2024-04-12T20:27:38.176 INFO:root:teuthology version: 0.0.1.dev267+g6c63784 2024-04-12T20:27:38.177 DEBUG:teuthology.run:Teuthology command: teuthology --archive /home/teuthworker/archive/vshankar-2024-04-12_03:11:38-fs-wip-vshankar-testing-20240411.061452-testing-default-smithi/7653578 --owner scheduled_vshankar@teuthology --description fs/upgrade/mds_upgrade_sequence/{bluestore-bitmap centos_9.stream conf/{client mds mgr mon osd} fail_fs/yes overrides/{ignorelist_health ignorelist_upgrade ignorelist_wrongly_marked_down pg-warn pg_health syntax} roles tasks/{0-from/quincy 1-volume/{0-create 1-ranks/2 2-allow_standby_replay/no 3-inline/no 4-verify} 2-client/fuse 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}} --verbose --name vshankar-2024-04-12_03:11:38-fs-wip-vshankar-testing-20240411.061452-testing-default-smithi -- /home/teuthworker/archive/vshankar-2024-04-12_03:11:38-fs-wip-vshankar-testing-20240411.061452-testing-default-smithi/7653578/orig.config.yaml 2024-04-12T20:27:38.234 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-12T20:27:38.325 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/vshankar-2024-04-12_03:11:38-fs-wip-vshankar-testing-20240411.061452-testing-default-smithi/7653578 branch: wip-vshankar-testing-20240411.061452 description: fs/upgrade/mds_upgrade_sequence/{bluestore-bitmap centos_9.stream conf/{client mds mgr mon osd} fail_fs/yes overrides/{ignorelist_health ignorelist_upgrade ignorelist_wrongly_marked_down pg-warn pg_health syntax} roles tasks/{0-from/quincy 1-volume/{0-create 1-ranks/2 2-allow_standby_replay/no 3-inline/no 4-verify} 2-client/fuse 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}} email: vshankar@redhat.com first_in_suite: false job_id: '7653578' kernel: flavor: default kdb: true sha1: 3f60ed14f7c110c7ce3c8d83dd2ed0d08e80d0c8 last_in_suite: false machine_type: smithi meta: - desc: 'setup ceph/quincy ' name: vshankar-2024-04-12_03:11:38-fs-wip-vshankar-testing-20240411.061452-testing-default-smithi no_nested_subset: false nuke-on-error: true os_type: centos os_version: 9.stream overrides: admin_socket: branch: wip-vshankar-testing-20240411.061452 ceph: conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 rados mon op timeout: 900 rados osd op timeout: 900 global: mon pg warn min per osd: 0 mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: debug client: 20 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon down mkfs grace: 300 mon op complaint time: 120 osd: bdev async discard: true bdev enable discard: true bluestore allocator: bitmap bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug ms: 1 debug osd: 20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - FS_DEGRADED - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - MDS_UP_LESS_THAN_MAX - filesystem is online with fewer MDS than max_mds - POOL_APP_NOT_ENABLED - overall HEALTH_ - Replacing daemon - deprecated feature inline_data - OSD_DOWN - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - PG_AVAILABILITY - PG_DEGRADED - Reduced data availability sha1: 324baf8ddb91dea46cac50cb535c306115791355 ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: osd default pool size: 2 osd: bdev async discard: true bdev enable discard: true bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs install: ceph: flavor: default sha1: 324baf8ddb91dea46cac50cb535c306115791355 kclient: syntax: v1 selinux: allowlist: - scontext=system_u:system_r:logrotate_t:s0 - scontext=system_u:system_r:getty_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-vshankar-testing-20240411.061452 sha1: 324baf8ddb91dea46cac50cb535c306115791355 owner: scheduled_vshankar@teuthology priority: 120 repo: https://git.ceph.com/ceph-ci.git roles: - - host.a - client.0 - osd.0 - osd.1 - osd.2 - - host.b - client.1 - osd.3 - osd.4 - osd.5 seed: 7105 sha1: 324baf8ddb91dea46cac50cb535c306115791355 sleep_before_teardown: 0 subset: 463/1024 suite: fs suite_branch: wip-vshankar-testing-20240411.061452 suite_path: /home/teuthworker/src/git.ceph.com_ceph-c_324baf8ddb91dea46cac50cb535c306115791355/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph-ci.git suite_sha1: 324baf8ddb91dea46cac50cb535c306115791355 targets: smithi018.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBNGyQkOmched09Efq4bSE3KgVpnlI3J6whhqkmy38BCo3vj5GRkDnqsXwwiryS9ADIjlegH6l0N7Y8JY4lxk/dQ= smithi186.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBKMqUaLnqL7jyJPKjPWFwXI6tZGa6LWRUQG61/E3XmkQnxUMVoO0pdZq24E8iyHT4Tq+F9Y+tckvwkjO0DzdbJY= tasks: - install: branch: quincy exclude_packages: - ceph-volume - print: '**** done install task...' - cephadm: cephadm_branch: quincy cephadm_git_url: https://github.com/ceph/ceph conf: osd: osd_class_default_list: '*' osd_class_load_list: '*' image: quay.ceph.io/ceph-ci/ceph:quincy roleless: true - print: '**** done end installing quincy cephadm ...' - cephadm.shell: host.a: - ceph config set mgr mgr/cephadm/use_repo_digest true --force - print: '**** done cephadm.shell ceph config set mgr...' - cephadm.shell: host.a: - ceph orch status - ceph orch ps - ceph orch ls - ceph orch host ls - ceph orch device ls - cephadm.shell: host.a: - ceph fs volume create cephfs --placement=4 - ceph fs dump - cephadm.shell: host.a: - ceph fs set cephfs max_mds 2 - cephadm.shell: host.a: - ceph fs set cephfs allow_standby_replay false - cephadm.shell: host.a: - ceph fs set cephfs inline_data false - cephadm.shell: host.a: - ceph fs dump - ceph --format=json fs dump | jq -e ".filesystems | length == 1" - while ! ceph --format=json mds versions | jq -e ". | add == 4"; do sleep 1; done - fs.pre_upgrade_save: null - ceph-fuse: null - print: '**** done client' - parallel: - upgrade-tasks - workload-tasks - cephadm.shell: host.a: - ceph fs dump - fs.post_upgrade_checks: null teuthology: fragments_dropped: [] meta: {} postmerge: [] variables: fail_fs: true teuthology_branch: main teuthology_sha1: 6c637841c215537a4502385240412f1966e0faab timestamp: 2024-04-12_03:11:38 tube: smithi upgrade-tasks: sequential: - cephadm.shell: env: - sha1 host.a: - ceph config set mon mon_warn_on_insecure_global_id_reclaim false --force - ceph config set mon mon_warn_on_insecure_global_id_reclaim_allowed false --force - ceph config set global log_to_journald false --force - ceph orch upgrade start --image quay.ceph.io/ceph-ci/ceph:$sha1 --daemon-types mgr - while ceph orch upgrade status | jq '.in_progress' | grep true && ! ceph orch upgrade status | jq '.message' | grep Error ; do ceph orch ps ; ceph versions ; ceph orch upgrade status ; sleep 30 ; done - ceph versions | jq -e '.mgr | length == 1' - ceph versions | jq -e '.mgr | keys' | grep $sha1 - ceph versions | jq -e '.overall | length == 2' - ceph orch upgrade check quay.ceph.io/ceph-ci/ceph:$sha1 | jq -e '.up_to_date | length == 2' - ceph orch ps - cephadm.shell: env: - sha1 host.a: - ceph config set mgr mgr/orchestrator/fail_fs true - cephadm.shell: env: - sha1 host.a: - ceph config set mon mon_warn_on_insecure_global_id_reclaim false --force - ceph config set mon mon_warn_on_insecure_global_id_reclaim_allowed false --force - ceph config set global log_to_journald false --force - ceph orch upgrade start --image quay.ceph.io/ceph-ci/ceph:$sha1 - cephadm.shell: env: - sha1 host.a: - while ceph orch upgrade status | jq '.in_progress' | grep true && ! ceph orch upgrade status | jq '.message' | grep Error ; do ceph orch ps ; ceph versions ; ceph fs dump; ceph orch upgrade status ; ceph health detail ; sleep 30 ; done - ceph orch ps - ceph orch upgrade status - ceph health detail - ceph versions - echo "wait for servicemap items w/ changing names to refresh" - sleep 60 - ceph orch ps - ceph versions - ceph versions | jq -e '.overall | length == 1' - ceph versions | jq -e '.overall | keys' | grep $sha1 user: vshankar verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885 workload-tasks: sequential: - workunit: clients: all: - suites/fsstress.sh 2024-04-12T20:27:38.325 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph-c_324baf8ddb91dea46cac50cb535c306115791355/qa; will attempt to use it 2024-04-12T20:27:38.326 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph-c_324baf8ddb91dea46cac50cb535c306115791355/qa/tasks 2024-04-12T20:27:38.326 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-04-12T20:27:38.327 INFO:teuthology.task.internal:Checking packages... 2024-04-12T20:27:38.351 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash '324baf8ddb91dea46cac50cb535c306115791355' 2024-04-12T20:27:38.352 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-04-12T20:27:38.352 INFO:teuthology.packaging:ref: None 2024-04-12T20:27:38.352 INFO:teuthology.packaging:tag: None 2024-04-12T20:27:38.352 INFO:teuthology.packaging:branch: wip-vshankar-testing-20240411.061452 2024-04-12T20:27:38.352 INFO:teuthology.packaging:sha1: 324baf8ddb91dea46cac50cb535c306115791355 2024-04-12T20:27:38.352 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=wip-vshankar-testing-20240411.061452 2024-04-12T20:27:38.616 INFO:teuthology.task.internal:Found packages for ceph version 19.0.0-2879.g324baf8d 2024-04-12T20:27:38.618 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-04-12T20:27:38.625 INFO:teuthology.task.internal:no buildpackages task found 2024-04-12T20:27:38.626 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-04-12T20:27:38.765 INFO:teuthology.task.internal:Saving configuration 2024-04-12T20:27:38.785 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-04-12T20:27:38.888 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-04-12T20:27:38.918 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi018.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/vshankar-2024-04-12_03:11:38-fs-wip-vshankar-testing-20240411.061452-testing-default-smithi/7653578', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-12 20:22:31.827896', 'locked_by': 'scheduled_vshankar@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBNGyQkOmched09Efq4bSE3KgVpnlI3J6whhqkmy38BCo3vj5GRkDnqsXwwiryS9ADIjlegH6l0N7Y8JY4lxk/dQ='} 2024-04-12T20:27:38.939 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi186.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/vshankar-2024-04-12_03:11:38-fs-wip-vshankar-testing-20240411.061452-testing-default-smithi/7653578', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-12 20:22:31.829527', 'locked_by': 'scheduled_vshankar@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBKMqUaLnqL7jyJPKjPWFwXI6tZGa6LWRUQG61/E3XmkQnxUMVoO0pdZq24E8iyHT4Tq+F9Y+tckvwkjO0DzdbJY='} 2024-04-12T20:27:38.939 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-04-12T20:27:38.996 INFO:teuthology.task.internal:roles: ubuntu@smithi018.front.sepia.ceph.com - ['host.a', 'client.0', 'osd.0', 'osd.1', 'osd.2'] 2024-04-12T20:27:38.996 INFO:teuthology.task.internal:roles: ubuntu@smithi186.front.sepia.ceph.com - ['host.b', 'client.1', 'osd.3', 'osd.4', 'osd.5'] 2024-04-12T20:27:38.997 INFO:teuthology.run_tasks:Running task console_log... 2024-04-12T20:27:39.197 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f949c42d160>, signals=[15]) 2024-04-12T20:27:39.198 INFO:teuthology.run_tasks:Running task internal.connect... 2024-04-12T20:27:39.222 INFO:teuthology.task.internal:Opening connections... 2024-04-12T20:27:39.222 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi018.front.sepia.ceph.com 2024-04-12T20:27:39.225 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi018.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-12T20:27:39.330 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi186.front.sepia.ceph.com 2024-04-12T20:27:39.331 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi186.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-12T20:27:39.418 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-04-12T20:27:39.433 DEBUG:teuthology.orchestra.run.smithi018:> uname -m 2024-04-12T20:27:39.451 INFO:teuthology.orchestra.run.smithi018.stdout:x86_64 2024-04-12T20:27:39.451 DEBUG:teuthology.orchestra.run.smithi018:> cat /etc/os-release 2024-04-12T20:27:39.508 INFO:teuthology.orchestra.run.smithi018.stdout:NAME="CentOS Stream" 2024-04-12T20:27:39.508 INFO:teuthology.orchestra.run.smithi018.stdout:VERSION="9" 2024-04-12T20:27:39.508 INFO:teuthology.orchestra.run.smithi018.stdout:ID="centos" 2024-04-12T20:27:39.508 INFO:teuthology.orchestra.run.smithi018.stdout:ID_LIKE="rhel fedora" 2024-04-12T20:27:39.508 INFO:teuthology.orchestra.run.smithi018.stdout:VERSION_ID="9" 2024-04-12T20:27:39.508 INFO:teuthology.orchestra.run.smithi018.stdout:PLATFORM_ID="platform:el9" 2024-04-12T20:27:39.508 INFO:teuthology.orchestra.run.smithi018.stdout:PRETTY_NAME="CentOS Stream 9" 2024-04-12T20:27:39.508 INFO:teuthology.orchestra.run.smithi018.stdout:ANSI_COLOR="0;31" 2024-04-12T20:27:39.508 INFO:teuthology.orchestra.run.smithi018.stdout:LOGO="fedora-logo-icon" 2024-04-12T20:27:39.508 INFO:teuthology.orchestra.run.smithi018.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-04-12T20:27:39.508 INFO:teuthology.orchestra.run.smithi018.stdout:HOME_URL="https://centos.org/" 2024-04-12T20:27:39.508 INFO:teuthology.orchestra.run.smithi018.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-04-12T20:27:39.509 INFO:teuthology.orchestra.run.smithi018.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-04-12T20:27:39.509 INFO:teuthology.orchestra.run.smithi018.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-04-12T20:27:39.509 INFO:teuthology.lock.ops:Updating smithi018.front.sepia.ceph.com on lock server 2024-04-12T20:27:39.535 DEBUG:teuthology.orchestra.run.smithi186:> uname -m 2024-04-12T20:27:39.552 INFO:teuthology.orchestra.run.smithi186.stdout:x86_64 2024-04-12T20:27:39.552 DEBUG:teuthology.orchestra.run.smithi186:> cat /etc/os-release 2024-04-12T20:27:39.609 INFO:teuthology.orchestra.run.smithi186.stdout:NAME="CentOS Stream" 2024-04-12T20:27:39.609 INFO:teuthology.orchestra.run.smithi186.stdout:VERSION="9" 2024-04-12T20:27:39.609 INFO:teuthology.orchestra.run.smithi186.stdout:ID="centos" 2024-04-12T20:27:39.609 INFO:teuthology.orchestra.run.smithi186.stdout:ID_LIKE="rhel fedora" 2024-04-12T20:27:39.610 INFO:teuthology.orchestra.run.smithi186.stdout:VERSION_ID="9" 2024-04-12T20:27:39.610 INFO:teuthology.orchestra.run.smithi186.stdout:PLATFORM_ID="platform:el9" 2024-04-12T20:27:39.610 INFO:teuthology.orchestra.run.smithi186.stdout:PRETTY_NAME="CentOS Stream 9" 2024-04-12T20:27:39.610 INFO:teuthology.orchestra.run.smithi186.stdout:ANSI_COLOR="0;31" 2024-04-12T20:27:39.610 INFO:teuthology.orchestra.run.smithi186.stdout:LOGO="fedora-logo-icon" 2024-04-12T20:27:39.610 INFO:teuthology.orchestra.run.smithi186.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-04-12T20:27:39.610 INFO:teuthology.orchestra.run.smithi186.stdout:HOME_URL="https://centos.org/" 2024-04-12T20:27:39.610 INFO:teuthology.orchestra.run.smithi186.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-04-12T20:27:39.610 INFO:teuthology.orchestra.run.smithi186.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-04-12T20:27:39.610 INFO:teuthology.orchestra.run.smithi186.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-04-12T20:27:39.611 INFO:teuthology.lock.ops:Updating smithi186.front.sepia.ceph.com on lock server 2024-04-12T20:27:39.635 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-04-12T20:27:39.782 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-04-12T20:27:39.829 INFO:teuthology.task.internal:Checking for old test directory... 2024-04-12T20:27:39.830 DEBUG:teuthology.orchestra.run.smithi018:> test '!' -e /home/ubuntu/cephtest 2024-04-12T20:27:39.832 DEBUG:teuthology.orchestra.run.smithi186:> test '!' -e /home/ubuntu/cephtest 2024-04-12T20:27:39.848 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-04-12T20:27:40.176 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-04-12T20:27:40.176 DEBUG:teuthology.orchestra.run.smithi018:> test -z $(ls -A /var/lib/ceph) 2024-04-12T20:27:40.179 DEBUG:teuthology.orchestra.run.smithi186:> test -z $(ls -A /var/lib/ceph) 2024-04-12T20:27:40.206 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-04-12T20:27:40.430 INFO:teuthology.run_tasks:Running task kernel... 2024-04-12T20:27:40.516 INFO:teuthology.task.kernel:normalize config orig: {'flavor': 'default', 'kdb': True, 'sha1': '3f60ed14f7c110c7ce3c8d83dd2ed0d08e80d0c8'} 2024-04-12T20:27:40.516 INFO:teuthology.task.kernel:config {'host.a': {'flavor': 'default', 'kdb': True, 'sha1': '3f60ed14f7c110c7ce3c8d83dd2ed0d08e80d0c8'}, 'host.b': {'flavor': 'default', 'kdb': True, 'sha1': '3f60ed14f7c110c7ce3c8d83dd2ed0d08e80d0c8'}}, timeout 300 2024-04-12T20:27:40.516 DEBUG:teuthology.orchestra.run.smithi018:> test -f /run/.containerenv -o -f /.dockerenv 2024-04-12T20:27:40.517 DEBUG:teuthology.orchestra.run.smithi186:> test -f /run/.containerenv -o -f /.dockerenv 2024-04-12T20:27:40.533 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-12T20:27:40.534 DEBUG:teuthology.task.kernel:sha1 for host.b is 3f60ed14f7c110c7ce3c8d83dd2ed0d08e80d0c8 2024-04-12T20:27:40.534 INFO:teuthology.task.kernel:Checking kernel version of host.b, want "3f60ed14f7c110c7ce3c8d83dd2ed0d08e80d0c8"... 2024-04-12T20:27:40.534 DEBUG:teuthology.orchestra.run.smithi186:> uname -r 2024-04-12T20:27:40.535 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-12T20:27:40.535 DEBUG:teuthology.task.kernel:sha1 for host.a is 3f60ed14f7c110c7ce3c8d83dd2ed0d08e80d0c8 2024-04-12T20:27:40.535 INFO:teuthology.task.kernel:Checking kernel version of host.a, want "3f60ed14f7c110c7ce3c8d83dd2ed0d08e80d0c8"... 2024-04-12T20:27:40.536 DEBUG:teuthology.orchestra.run.smithi018:> uname -r 2024-04-12T20:27:40.591 INFO:teuthology.orchestra.run.smithi186.stdout:5.14.0-361.el9.x86_64 2024-04-12T20:27:40.592 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-361.el9.x86_64 vs 3f60ed14f7c110c7ce3c8d83dd2ed0d08e80d0c8 2024-04-12T20:27:40.592 DEBUG:teuthology.task.kernel:failed to parse current kernel version 2024-04-12T20:27:40.592 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&sha1=3f60ed14f7c110c7ce3c8d83dd2ed0d08e80d0c8 2024-04-12T20:27:40.596 INFO:teuthology.orchestra.run.smithi018.stdout:5.14.0-361.el9.x86_64 2024-04-12T20:27:40.596 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-361.el9.x86_64 vs 3f60ed14f7c110c7ce3c8d83dd2ed0d08e80d0c8 2024-04-12T20:27:40.596 DEBUG:teuthology.task.kernel:failed to parse current kernel version 2024-04-12T20:27:40.597 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&sha1=3f60ed14f7c110c7ce3c8d83dd2ed0d08e80d0c8 2024-04-12T20:27:40.741 DEBUG:teuthology.orchestra.run.smithi018:> sudo yum upgrade -y linux-firmware 2024-04-12T20:27:40.822 DEBUG:teuthology.orchestra.run.smithi186:> sudo yum upgrade -y linux-firmware 2024-04-12T20:27:42.953 INFO:teuthology.orchestra.run.smithi018.stdout:CentOS-9 - AppStream 14 MB/s | 8.1 MB 00:00 2024-04-12T20:27:42.958 INFO:teuthology.orchestra.run.smithi186.stdout:CentOS-9 - AppStream 14 MB/s | 8.1 MB 00:00 2024-04-12T20:27:44.703 INFO:teuthology.orchestra.run.smithi018.stdout:CentOS-9 - BaseOS 6.9 MB/s | 2.5 MB 00:00 2024-04-12T20:27:44.901 INFO:teuthology.orchestra.run.smithi186.stdout:CentOS-9 - BaseOS 4.0 MB/s | 2.5 MB 00:00 2024-04-12T20:27:47.810 INFO:teuthology.orchestra.run.smithi018.stdout:CentOS Stream 9 - BaseOS 3.0 MB/s | 8.0 MB 00:02 2024-04-12T20:27:50.028 INFO:teuthology.orchestra.run.smithi018.stdout:CentOS Stream 9 - AppStream 18 MB/s | 19 MB 00:01 2024-04-12T20:27:54.612 INFO:teuthology.orchestra.run.smithi018.stdout:CentOS Stream 9 - CRB 7.3 MB/s | 6.2 MB 00:00 2024-04-12T20:27:54.782 INFO:teuthology.orchestra.run.smithi186.stdout:CentOS Stream 9 - BaseOS 874 kB/s | 8.0 MB 00:09 2024-04-12T20:27:56.029 INFO:teuthology.orchestra.run.smithi018.stdout:CentOS Stream 9 - Extras packages 73 kB/s | 16 kB 00:00 2024-04-12T20:27:57.258 INFO:teuthology.orchestra.run.smithi018.stdout:Extra Packages for Enterprise Linux 19 MB/s | 21 MB 00:01 2024-04-12T20:27:57.491 INFO:teuthology.orchestra.run.smithi186.stdout:CentOS Stream 9 - AppStream 12 MB/s | 19 MB 00:01 2024-04-12T20:28:01.625 INFO:teuthology.orchestra.run.smithi186.stdout:CentOS Stream 9 - CRB 12 MB/s | 6.2 MB 00:00 2024-04-12T20:28:02.464 INFO:teuthology.orchestra.run.smithi018.stdout:lab-extras 31 kB/s | 1.7 kB 00:00 2024-04-12T20:28:03.626 INFO:teuthology.orchestra.run.smithi186.stdout:CentOS Stream 9 - Extras packages 20 kB/s | 16 kB 00:00 2024-04-12T20:28:04.813 INFO:teuthology.orchestra.run.smithi018.stderr:Modular dependency problems: 2024-04-12T20:28:04.813 INFO:teuthology.orchestra.run.smithi018.stderr: 2024-04-12T20:28:04.814 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 1: conflicting requests 2024-04-12T20:28:04.814 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module container-tools:rhel8:820240321091303:20125149.x86_64 from CentOS-AppStream 2024-04-12T20:28:04.814 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 2: conflicting requests 2024-04-12T20:28:04.814 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module eclipse:rhel8:820201023100746:b230ed4e.x86_64 from CentOS-AppStream 2024-04-12T20:28:04.814 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 3: conflicting requests 2024-04-12T20:28:04.814 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module freeradius:3.0:8080020230117180605:89170a74.x86_64 from CentOS-AppStream 2024-04-12T20:28:04.814 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 4: conflicting requests 2024-04-12T20:28:04.814 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module gimp:2.8:8000020190628145146:4148dfdf.x86_64 from CentOS-AppStream 2024-04-12T20:28:04.814 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 5: conflicting requests 2024-04-12T20:28:04.814 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module go-toolset:rhel8:820240401160413:b754926a.x86_64 from CentOS-AppStream 2024-04-12T20:28:04.814 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 6: conflicting requests 2024-04-12T20:28:04.814 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module httpd:2.4:820240216084734:9edba152.x86_64 from CentOS-AppStream 2024-04-12T20:28:04.815 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 7: conflicting requests 2024-04-12T20:28:04.815 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module idm:client:820240307184541:49cc9d1b.x86_64 from CentOS-AppStream 2024-04-12T20:28:04.815 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 8: conflicting requests 2024-04-12T20:28:04.815 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module inkscape:0.92.3:8080020221205124429:3e031279.x86_64 from CentOS-AppStream 2024-04-12T20:28:04.815 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 9: conflicting requests 2024-04-12T20:28:04.815 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module jmc:rhel8:8050020211110222101:6392b1f8.x86_64 from CentOS-AppStream 2024-04-12T20:28:04.815 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 10: conflicting requests 2024-04-12T20:28:04.815 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module llvm-toolset:rhel8:820240205133135:9edba152.x86_64 from CentOS-AppStream 2024-04-12T20:28:04.815 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 11: conflicting requests 2024-04-12T20:28:04.815 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module log4j:2:8080020221205124743:9d367344.x86_64 from CentOS-AppStream 2024-04-12T20:28:04.815 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 12: conflicting requests 2024-04-12T20:28:04.815 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module mailman:2.1:820230727085757:77fc8825.x86_64 from CentOS-AppStream 2024-04-12T20:28:04.815 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 13: conflicting requests 2024-04-12T20:28:04.815 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module mariadb:10.3:8030020210419150013:30b713e6.x86_64 from CentOS-AppStream 2024-04-12T20:28:04.815 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 14: conflicting requests 2024-04-12T20:28:04.816 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module maven:3.5:8060020211203142015:c0229ad2.x86_64 from CentOS-AppStream 2024-04-12T20:28:04.816 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 15: conflicting requests 2024-04-12T20:28:04.816 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module mercurial:4.8:8000020190628020724:4148dfdf.x86_64 from CentOS-AppStream 2024-04-12T20:28:04.816 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 16: conflicting requests 2024-04-12T20:28:04.816 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module mod_auth_openidc:2.3:820230721123832:9edba152.x86_64 from CentOS-AppStream 2024-04-12T20:28:04.816 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 17: conflicting requests 2024-04-12T20:28:04.816 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module mysql:8.0:8040020210901180257:9f9e2e7e.x86_64 from CentOS-AppStream 2024-04-12T20:28:04.816 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 18: conflicting requests 2024-04-12T20:28:04.816 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module nginx:1.14:8000020211221191913:55190bc5.x86_64 from CentOS-AppStream 2024-04-12T20:28:04.816 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 19: conflicting requests 2024-04-12T20:28:04.816 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module nodejs:10:8040020210120182536:9f9e2e7e.x86_64 from CentOS-AppStream 2024-04-12T20:28:04.816 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 20: conflicting requests 2024-04-12T20:28:04.816 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl:5.26:8000020190628020724:55190bc5.x86_64 from CentOS-AppStream 2024-04-12T20:28:04.816 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 21: conflicting requests 2024-04-12T20:28:04.817 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module php:7.2:8020020200507003613:2c7ca891.x86_64 from CentOS-AppStream 2024-04-12T20:28:04.817 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 22: conflicting requests 2024-04-12T20:28:04.817 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module postgresql:10:820230223114800:9edba152.x86_64 from CentOS-AppStream 2024-04-12T20:28:04.817 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 23: conflicting requests 2024-04-12T20:28:04.817 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module python27:2.7:820240208011952:182f7c73.x86_64 from CentOS-AppStream 2024-04-12T20:28:04.817 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 24: conflicting requests 2024-04-12T20:28:04.817 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module python36:3.6:8050020210825152031:982725ab.x86_64 from CentOS-AppStream 2024-04-12T20:28:04.817 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 25: conflicting requests 2024-04-12T20:28:04.817 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module python38:3.8:820230810143931:64c1cd5a.x86_64 from CentOS-AppStream 2024-04-12T20:28:04.817 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 26: conflicting requests 2024-04-12T20:28:04.817 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module python39:3.9:820240214182535:17377f89.x86_64 from CentOS-AppStream 2024-04-12T20:28:04.817 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 27: conflicting requests 2024-04-12T20:28:04.817 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module redis:5:8040020211019153849:9f9e2e7e.x86_64 from CentOS-AppStream 2024-04-12T20:28:04.817 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 28: conflicting requests 2024-04-12T20:28:04.817 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module rhn-tools:1.0:8010020191114034948:f69d1239.x86_64 from CentOS-AppStream 2024-04-12T20:28:04.818 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 29: conflicting requests 2024-04-12T20:28:04.818 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module ruby:2.5:820230627084142:9edba152.x86_64 from CentOS-AppStream 2024-04-12T20:28:04.818 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 30: conflicting requests 2024-04-12T20:28:04.818 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module rust-toolset:rhel8:820240119204620:b09eea91.x86_64 from CentOS-AppStream 2024-04-12T20:28:04.818 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 31: conflicting requests 2024-04-12T20:28:04.818 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module satellite-5-client:1.0:8010020191114035551:cdc1202b.x86_64 from CentOS-AppStream 2024-04-12T20:28:04.818 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 32: conflicting requests 2024-04-12T20:28:04.818 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module squid:4:820240319173245:9edba152.x86_64 from CentOS-AppStream 2024-04-12T20:28:04.818 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 33: conflicting requests 2024-04-12T20:28:04.818 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module subversion:1.10:8070020220711155714:78111232.x86_64 from CentOS-AppStream 2024-04-12T20:28:04.818 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 34: conflicting requests 2024-04-12T20:28:04.818 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module swig:3.0:8030020200706142531:30b713e6.x86_64 from CentOS-AppStream 2024-04-12T20:28:04.818 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 35: conflicting requests 2024-04-12T20:28:04.818 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module varnish:6:8050020211109225449:b4937e53.x86_64 from CentOS-AppStream 2024-04-12T20:28:04.818 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 36: conflicting requests 2024-04-12T20:28:04.818 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module virt:rhel:820240314161907:9edba152.x86_64 from CentOS-AppStream 2024-04-12T20:28:04.819 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 37: conflicting requests 2024-04-12T20:28:04.819 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:8030020200716155257:7cc0a66d.x86_64 from CentOS-AppStream 2024-04-12T20:28:04.819 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:8030020200716155257:b967a9a2.x86_64 from CentOS-AppStream 2024-04-12T20:28:04.819 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:820230811133638:36f9fae6.x86_64 from CentOS-AppStream 2024-04-12T20:28:04.819 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 38: cannot install the best candidate for the job 2024-04-12T20:28:04.819 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:8010020191114031501:a5949e2e.x86_64 from CentOS-AppStream 2024-04-12T20:28:04.819 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:8010020191114031501:be2e4737.x86_64 from CentOS-AppStream 2024-04-12T20:28:04.819 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:820230223114759:2ab761e1.x86_64 from CentOS-AppStream 2024-04-12T20:28:04.819 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:820230223114759:86e45846.x86_64 from CentOS-AppStream 2024-04-12T20:28:04.819 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 39: conflicting requests 2024-04-12T20:28:04.819 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:8030020200715230104:1e4bbb35.x86_64 from CentOS-AppStream 2024-04-12T20:28:04.819 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:8030020200715230104:ea09926d.x86_64 from CentOS-AppStream 2024-04-12T20:28:04.819 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:082fdf2f.x86_64 from CentOS-AppStream 2024-04-12T20:28:04.820 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-04-12T20:28:04.820 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-04-12T20:28:04.820 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:fbe42456.x86_64 from CentOS-AppStream 2024-04-12T20:28:04.820 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 40: cannot install the best candidate for the job 2024-04-12T20:28:04.820 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:8010020191114031513:16b3ab4d.x86_64 from CentOS-AppStream 2024-04-12T20:28:04.820 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:8010020191114031513:e1f37755.x86_64 from CentOS-AppStream 2024-04-12T20:28:04.820 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-04-12T20:28:04.820 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-04-12T20:28:04.820 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 41: cannot install the best candidate for the job 2024-04-12T20:28:04.820 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:8030020200716150652:1e4bbb35.x86_64 from CentOS-AppStream 2024-04-12T20:28:04.820 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:8030020200716150652:ea09926d.x86_64 from CentOS-AppStream 2024-04-12T20:28:04.820 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-04-12T20:28:04.820 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-04-12T20:28:04.820 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 42: cannot install the best candidate for the job 2024-04-12T20:28:04.821 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:8030020200716174729:3a70019f.x86_64 from CentOS-AppStream 2024-04-12T20:28:04.821 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:8030020200716174729:cccafca5.x86_64 from CentOS-AppStream 2024-04-12T20:28:04.821 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:820230223114759:6fdf7e5d.x86_64 from CentOS-AppStream 2024-04-12T20:28:04.821 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:820230223114759:d4326aba.x86_64 from CentOS-AppStream 2024-04-12T20:28:04.821 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 43: cannot install the best candidate for the job 2024-04-12T20:28:04.821 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:8030020200716213634:56fce90f.x86_64 from CentOS-AppStream 2024-04-12T20:28:04.821 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:8030020200716213634:98a3c9d0.x86_64 from CentOS-AppStream 2024-04-12T20:28:04.821 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:820230223114759:06b7596e.x86_64 from CentOS-AppStream 2024-04-12T20:28:04.821 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:820230223114759:d2fc5d76.x86_64 from CentOS-AppStream 2024-04-12T20:28:04.821 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 44: cannot install the best candidate for the job 2024-04-12T20:28:04.821 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:8030020200716171139:3a70019f.x86_64 from CentOS-AppStream 2024-04-12T20:28:04.821 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:8030020200716171139:cccafca5.x86_64 from CentOS-AppStream 2024-04-12T20:28:04.821 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:820230223114759:6fdf7e5d.x86_64 from CentOS-AppStream 2024-04-12T20:28:04.822 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:820230223114759:d4326aba.x86_64 from CentOS-AppStream 2024-04-12T20:28:04.822 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 45: cannot install the best candidate for the job 2024-04-12T20:28:04.822 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:8010020191120175858:95fce66d.x86_64 from CentOS-AppStream 2024-04-12T20:28:04.822 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:8030020200723233049:09acf126.x86_64 from CentOS-AppStream 2024-04-12T20:28:04.822 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:820230223114759:436af1ae.x86_64 from CentOS-AppStream 2024-04-12T20:28:04.822 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:820230223114759:b12195ea.x86_64 from CentOS-AppStream 2024-04-12T20:28:05.397 INFO:teuthology.orchestra.run.smithi018.stdout:Dependencies resolved. 2024-04-12T20:28:05.408 INFO:teuthology.orchestra.run.smithi018.stdout:================================================================================ 2024-04-12T20:28:05.408 INFO:teuthology.orchestra.run.smithi018.stdout: Package Architecture Version Repository Size 2024-04-12T20:28:05.408 INFO:teuthology.orchestra.run.smithi018.stdout:================================================================================ 2024-04-12T20:28:05.408 INFO:teuthology.orchestra.run.smithi018.stdout:Upgrading: 2024-04-12T20:28:05.408 INFO:teuthology.orchestra.run.smithi018.stdout: linux-firmware noarch 20240219-143.el9 baseos 376 M 2024-04-12T20:28:05.408 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-12T20:28:05.409 INFO:teuthology.orchestra.run.smithi018.stdout:Transaction Summary 2024-04-12T20:28:05.409 INFO:teuthology.orchestra.run.smithi018.stdout:================================================================================ 2024-04-12T20:28:05.409 INFO:teuthology.orchestra.run.smithi018.stdout:Upgrade 1 Package 2024-04-12T20:28:05.409 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-12T20:28:05.409 INFO:teuthology.orchestra.run.smithi018.stdout:Total download size: 376 M 2024-04-12T20:28:05.409 INFO:teuthology.orchestra.run.smithi018.stdout:Downloading Packages: 2024-04-12T20:28:06.288 INFO:teuthology.orchestra.run.smithi186.stdout:Extra Packages for Enterprise Linux 8.3 MB/s | 21 MB 00:02 2024-04-12T20:28:11.385 INFO:teuthology.orchestra.run.smithi186.stdout:lab-extras 15 kB/s | 1.7 kB 00:00 2024-04-12T20:28:13.665 INFO:teuthology.orchestra.run.smithi186.stderr:Modular dependency problems: 2024-04-12T20:28:13.665 INFO:teuthology.orchestra.run.smithi186.stderr: 2024-04-12T20:28:13.665 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 1: conflicting requests 2024-04-12T20:28:13.665 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module container-tools:rhel8:820240321091303:20125149.x86_64 from CentOS-AppStream 2024-04-12T20:28:13.665 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 2: conflicting requests 2024-04-12T20:28:13.665 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module eclipse:rhel8:820201023100746:b230ed4e.x86_64 from CentOS-AppStream 2024-04-12T20:28:13.665 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 3: conflicting requests 2024-04-12T20:28:13.665 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module freeradius:3.0:8080020230117180605:89170a74.x86_64 from CentOS-AppStream 2024-04-12T20:28:13.665 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 4: conflicting requests 2024-04-12T20:28:13.666 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module gimp:2.8:8000020190628145146:4148dfdf.x86_64 from CentOS-AppStream 2024-04-12T20:28:13.666 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 5: conflicting requests 2024-04-12T20:28:13.666 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module go-toolset:rhel8:820240401160413:b754926a.x86_64 from CentOS-AppStream 2024-04-12T20:28:13.666 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 6: conflicting requests 2024-04-12T20:28:13.666 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module httpd:2.4:820240216084734:9edba152.x86_64 from CentOS-AppStream 2024-04-12T20:28:13.666 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 7: conflicting requests 2024-04-12T20:28:13.666 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module idm:client:820240307184541:49cc9d1b.x86_64 from CentOS-AppStream 2024-04-12T20:28:13.666 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 8: conflicting requests 2024-04-12T20:28:13.666 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module inkscape:0.92.3:8080020221205124429:3e031279.x86_64 from CentOS-AppStream 2024-04-12T20:28:13.666 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 9: conflicting requests 2024-04-12T20:28:13.666 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module jmc:rhel8:8050020211110222101:6392b1f8.x86_64 from CentOS-AppStream 2024-04-12T20:28:13.666 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 10: conflicting requests 2024-04-12T20:28:13.666 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module llvm-toolset:rhel8:820240205133135:9edba152.x86_64 from CentOS-AppStream 2024-04-12T20:28:13.667 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 11: conflicting requests 2024-04-12T20:28:13.667 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module log4j:2:8080020221205124743:9d367344.x86_64 from CentOS-AppStream 2024-04-12T20:28:13.667 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 12: conflicting requests 2024-04-12T20:28:13.667 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module mailman:2.1:820230727085757:77fc8825.x86_64 from CentOS-AppStream 2024-04-12T20:28:13.667 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 13: conflicting requests 2024-04-12T20:28:13.667 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module mariadb:10.3:8030020210419150013:30b713e6.x86_64 from CentOS-AppStream 2024-04-12T20:28:13.667 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 14: conflicting requests 2024-04-12T20:28:13.667 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module maven:3.5:8060020211203142015:c0229ad2.x86_64 from CentOS-AppStream 2024-04-12T20:28:13.667 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 15: conflicting requests 2024-04-12T20:28:13.667 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module mercurial:4.8:8000020190628020724:4148dfdf.x86_64 from CentOS-AppStream 2024-04-12T20:28:13.667 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 16: conflicting requests 2024-04-12T20:28:13.667 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module mod_auth_openidc:2.3:820230721123832:9edba152.x86_64 from CentOS-AppStream 2024-04-12T20:28:13.667 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 17: conflicting requests 2024-04-12T20:28:13.667 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module mysql:8.0:8040020210901180257:9f9e2e7e.x86_64 from CentOS-AppStream 2024-04-12T20:28:13.667 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 18: conflicting requests 2024-04-12T20:28:13.668 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module nginx:1.14:8000020211221191913:55190bc5.x86_64 from CentOS-AppStream 2024-04-12T20:28:13.668 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 19: conflicting requests 2024-04-12T20:28:13.668 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module nodejs:10:8040020210120182536:9f9e2e7e.x86_64 from CentOS-AppStream 2024-04-12T20:28:13.668 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 20: conflicting requests 2024-04-12T20:28:13.668 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl:5.26:8000020190628020724:55190bc5.x86_64 from CentOS-AppStream 2024-04-12T20:28:13.668 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 21: conflicting requests 2024-04-12T20:28:13.668 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module php:7.2:8020020200507003613:2c7ca891.x86_64 from CentOS-AppStream 2024-04-12T20:28:13.668 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 22: conflicting requests 2024-04-12T20:28:13.668 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module postgresql:10:820230223114800:9edba152.x86_64 from CentOS-AppStream 2024-04-12T20:28:13.668 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 23: conflicting requests 2024-04-12T20:28:13.668 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module python27:2.7:820240208011952:182f7c73.x86_64 from CentOS-AppStream 2024-04-12T20:28:13.668 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 24: conflicting requests 2024-04-12T20:28:13.668 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module python36:3.6:8050020210825152031:982725ab.x86_64 from CentOS-AppStream 2024-04-12T20:28:13.668 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 25: conflicting requests 2024-04-12T20:28:13.669 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module python38:3.8:820230810143931:64c1cd5a.x86_64 from CentOS-AppStream 2024-04-12T20:28:13.669 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 26: conflicting requests 2024-04-12T20:28:13.669 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module python39:3.9:820240214182535:17377f89.x86_64 from CentOS-AppStream 2024-04-12T20:28:13.669 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 27: conflicting requests 2024-04-12T20:28:13.669 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module redis:5:8040020211019153849:9f9e2e7e.x86_64 from CentOS-AppStream 2024-04-12T20:28:13.669 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 28: conflicting requests 2024-04-12T20:28:13.669 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module rhn-tools:1.0:8010020191114034948:f69d1239.x86_64 from CentOS-AppStream 2024-04-12T20:28:13.669 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 29: conflicting requests 2024-04-12T20:28:13.669 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module ruby:2.5:820230627084142:9edba152.x86_64 from CentOS-AppStream 2024-04-12T20:28:13.669 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 30: conflicting requests 2024-04-12T20:28:13.669 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module rust-toolset:rhel8:820240119204620:b09eea91.x86_64 from CentOS-AppStream 2024-04-12T20:28:13.669 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 31: conflicting requests 2024-04-12T20:28:13.669 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module satellite-5-client:1.0:8010020191114035551:cdc1202b.x86_64 from CentOS-AppStream 2024-04-12T20:28:13.669 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 32: conflicting requests 2024-04-12T20:28:13.670 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module squid:4:820240319173245:9edba152.x86_64 from CentOS-AppStream 2024-04-12T20:28:13.670 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 33: conflicting requests 2024-04-12T20:28:13.670 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module subversion:1.10:8070020220711155714:78111232.x86_64 from CentOS-AppStream 2024-04-12T20:28:13.670 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 34: conflicting requests 2024-04-12T20:28:13.670 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module swig:3.0:8030020200706142531:30b713e6.x86_64 from CentOS-AppStream 2024-04-12T20:28:13.670 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 35: conflicting requests 2024-04-12T20:28:13.670 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module varnish:6:8050020211109225449:b4937e53.x86_64 from CentOS-AppStream 2024-04-12T20:28:13.670 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 36: conflicting requests 2024-04-12T20:28:13.670 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module virt:rhel:820240314161907:9edba152.x86_64 from CentOS-AppStream 2024-04-12T20:28:13.670 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 37: conflicting requests 2024-04-12T20:28:13.670 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:8030020200716155257:7cc0a66d.x86_64 from CentOS-AppStream 2024-04-12T20:28:13.670 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:8030020200716155257:b967a9a2.x86_64 from CentOS-AppStream 2024-04-12T20:28:13.670 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:820230811133638:36f9fae6.x86_64 from CentOS-AppStream 2024-04-12T20:28:13.670 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 38: cannot install the best candidate for the job 2024-04-12T20:28:13.671 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:8010020191114031501:a5949e2e.x86_64 from CentOS-AppStream 2024-04-12T20:28:13.671 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:8010020191114031501:be2e4737.x86_64 from CentOS-AppStream 2024-04-12T20:28:13.671 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:820230223114759:2ab761e1.x86_64 from CentOS-AppStream 2024-04-12T20:28:13.671 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:820230223114759:86e45846.x86_64 from CentOS-AppStream 2024-04-12T20:28:13.671 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 39: conflicting requests 2024-04-12T20:28:13.671 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:8030020200715230104:1e4bbb35.x86_64 from CentOS-AppStream 2024-04-12T20:28:13.671 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:8030020200715230104:ea09926d.x86_64 from CentOS-AppStream 2024-04-12T20:28:13.671 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:082fdf2f.x86_64 from CentOS-AppStream 2024-04-12T20:28:13.671 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-04-12T20:28:13.671 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-04-12T20:28:13.671 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:fbe42456.x86_64 from CentOS-AppStream 2024-04-12T20:28:13.671 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 40: cannot install the best candidate for the job 2024-04-12T20:28:13.671 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:8010020191114031513:16b3ab4d.x86_64 from CentOS-AppStream 2024-04-12T20:28:13.671 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:8010020191114031513:e1f37755.x86_64 from CentOS-AppStream 2024-04-12T20:28:13.671 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-04-12T20:28:13.671 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-04-12T20:28:13.672 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 41: cannot install the best candidate for the job 2024-04-12T20:28:13.672 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:8030020200716150652:1e4bbb35.x86_64 from CentOS-AppStream 2024-04-12T20:28:13.672 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:8030020200716150652:ea09926d.x86_64 from CentOS-AppStream 2024-04-12T20:28:13.672 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-04-12T20:28:13.672 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-04-12T20:28:13.672 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 42: cannot install the best candidate for the job 2024-04-12T20:28:13.672 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:8030020200716174729:3a70019f.x86_64 from CentOS-AppStream 2024-04-12T20:28:13.672 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:8030020200716174729:cccafca5.x86_64 from CentOS-AppStream 2024-04-12T20:28:13.672 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:820230223114759:6fdf7e5d.x86_64 from CentOS-AppStream 2024-04-12T20:28:13.672 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:820230223114759:d4326aba.x86_64 from CentOS-AppStream 2024-04-12T20:28:13.672 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 43: cannot install the best candidate for the job 2024-04-12T20:28:13.672 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:8030020200716213634:56fce90f.x86_64 from CentOS-AppStream 2024-04-12T20:28:13.672 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:8030020200716213634:98a3c9d0.x86_64 from CentOS-AppStream 2024-04-12T20:28:13.672 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:820230223114759:06b7596e.x86_64 from CentOS-AppStream 2024-04-12T20:28:13.672 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:820230223114759:d2fc5d76.x86_64 from CentOS-AppStream 2024-04-12T20:28:13.672 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 44: cannot install the best candidate for the job 2024-04-12T20:28:13.673 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:8030020200716171139:3a70019f.x86_64 from CentOS-AppStream 2024-04-12T20:28:13.673 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:8030020200716171139:cccafca5.x86_64 from CentOS-AppStream 2024-04-12T20:28:13.673 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:820230223114759:6fdf7e5d.x86_64 from CentOS-AppStream 2024-04-12T20:28:13.673 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:820230223114759:d4326aba.x86_64 from CentOS-AppStream 2024-04-12T20:28:13.673 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 45: cannot install the best candidate for the job 2024-04-12T20:28:13.673 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:8010020191120175858:95fce66d.x86_64 from CentOS-AppStream 2024-04-12T20:28:13.673 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:8030020200723233049:09acf126.x86_64 from CentOS-AppStream 2024-04-12T20:28:13.673 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:820230223114759:436af1ae.x86_64 from CentOS-AppStream 2024-04-12T20:28:13.673 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:820230223114759:b12195ea.x86_64 from CentOS-AppStream 2024-04-12T20:28:14.137 INFO:teuthology.orchestra.run.smithi186.stdout:Dependencies resolved. 2024-04-12T20:28:14.144 INFO:teuthology.orchestra.run.smithi186.stdout:================================================================================ 2024-04-12T20:28:14.144 INFO:teuthology.orchestra.run.smithi186.stdout: Package Architecture Version Repository Size 2024-04-12T20:28:14.144 INFO:teuthology.orchestra.run.smithi186.stdout:================================================================================ 2024-04-12T20:28:14.144 INFO:teuthology.orchestra.run.smithi186.stdout:Upgrading: 2024-04-12T20:28:14.144 INFO:teuthology.orchestra.run.smithi186.stdout: linux-firmware noarch 20240219-143.el9 baseos 376 M 2024-04-12T20:28:14.145 INFO:teuthology.orchestra.run.smithi186.stdout: 2024-04-12T20:28:14.145 INFO:teuthology.orchestra.run.smithi186.stdout:Transaction Summary 2024-04-12T20:28:14.145 INFO:teuthology.orchestra.run.smithi186.stdout:================================================================================ 2024-04-12T20:28:14.145 INFO:teuthology.orchestra.run.smithi186.stdout:Upgrade 1 Package 2024-04-12T20:28:14.145 INFO:teuthology.orchestra.run.smithi186.stdout: 2024-04-12T20:28:14.145 INFO:teuthology.orchestra.run.smithi186.stdout:Total download size: 376 M 2024-04-12T20:28:14.145 INFO:teuthology.orchestra.run.smithi186.stdout:Downloading Packages: 2024-04-12T20:28:18.305 INFO:teuthology.orchestra.run.smithi018.stdout:linux-firmware-20240219-143.el9.noarch.rpm 29 MB/s | 376 MB 00:12 2024-04-12T20:28:18.306 INFO:teuthology.orchestra.run.smithi018.stdout:-------------------------------------------------------------------------------- 2024-04-12T20:28:18.307 INFO:teuthology.orchestra.run.smithi018.stdout:Total 29 MB/s | 376 MB 00:12 2024-04-12T20:28:21.492 INFO:teuthology.orchestra.run.smithi018.stdout:Running transaction check 2024-04-12T20:28:21.590 INFO:teuthology.orchestra.run.smithi018.stdout:Transaction check succeeded. 2024-04-12T20:28:21.590 INFO:teuthology.orchestra.run.smithi018.stdout:Running transaction test 2024-04-12T20:28:24.974 INFO:teuthology.orchestra.run.smithi018.stdout:Transaction test succeeded. 2024-04-12T20:28:24.974 INFO:teuthology.orchestra.run.smithi018.stdout:Running transaction 2024-04-12T20:28:32.460 INFO:teuthology.orchestra.run.smithi018.stdout: Preparing : 1/1 2024-04-12T20:28:32.789 INFO:teuthology.orchestra.run.smithi018.stdout: Upgrading : linux-firmware-20240219-143.el9.noarch 1/2 2024-04-12T20:28:32.877 INFO:teuthology.orchestra.run.smithi018.stdout: Cleanup : linux-firmware-20230814-139.el9.noarch 2/2 2024-04-12T20:28:33.938 INFO:teuthology.orchestra.run.smithi018.stdout: Running scriptlet: linux-firmware-20230814-139.el9.noarch 2/2 2024-04-12T20:28:33.938 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : linux-firmware-20240219-143.el9.noarch 1/2 2024-04-12T20:28:34.380 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : linux-firmware-20230814-139.el9.noarch 2/2 2024-04-12T20:28:34.381 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-12T20:28:34.381 INFO:teuthology.orchestra.run.smithi018.stdout:Upgraded: 2024-04-12T20:28:34.381 INFO:teuthology.orchestra.run.smithi018.stdout: linux-firmware-20240219-143.el9.noarch 2024-04-12T20:28:34.381 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-12T20:28:34.381 INFO:teuthology.orchestra.run.smithi018.stdout:Complete! 2024-04-12T20:28:34.684 INFO:teuthology.task.kernel:Downloading kernel 3f60ed14f7c110c7ce3c8d83dd2ed0d08e80d0c8 on host.a... 2024-04-12T20:28:34.684 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&sha1=3f60ed14f7c110c7ce3c8d83dd2ed0d08e80d0c8 2024-04-12T20:28:34.834 INFO:teuthology.task.kernel:fetching, builder baseurl is https://3.chacra.ceph.com/r/kernel/testing/3f60ed14f7c110c7ce3c8d83dd2ed0d08e80d0c8/centos/9/flavors/default/x86_64/ 2024-04-12T20:28:34.834 DEBUG:teuthology.orchestra.run.smithi018:> rm -f /tmp/kernel.x86_64.rpm && echo kernel-6.9.0_rc3_g3f60ed14f7c1-1.x86_64.rpm | wget -nv -O /tmp/kernel.x86_64.rpm --base=https://3.chacra.ceph.com/r/kernel/testing/3f60ed14f7c110c7ce3c8d83dd2ed0d08e80d0c8/centos/9/flavors/default/x86_64/ --input-file=- 2024-04-12T20:28:34.837 DEBUG:teuthology.task.kernel:Waiting for download/copy to ubuntu@smithi018.front.sepia.ceph.com to complete... 2024-04-12T20:28:35.918 INFO:teuthology.orchestra.run.smithi186.stdout:linux-firmware-20240219-143.el9.noarch.rpm 18 MB/s | 376 MB 00:21 2024-04-12T20:28:35.918 INFO:teuthology.orchestra.run.smithi186.stdout:-------------------------------------------------------------------------------- 2024-04-12T20:28:35.918 INFO:teuthology.orchestra.run.smithi186.stdout:Total 17 MB/s | 376 MB 00:21 2024-04-12T20:28:38.949 INFO:teuthology.orchestra.run.smithi186.stdout:Running transaction check 2024-04-12T20:28:39.327 INFO:teuthology.orchestra.run.smithi186.stdout:Transaction check succeeded. 2024-04-12T20:28:39.327 INFO:teuthology.orchestra.run.smithi186.stdout:Running transaction test 2024-04-12T20:28:42.578 INFO:teuthology.orchestra.run.smithi186.stdout:Transaction test succeeded. 2024-04-12T20:28:42.578 INFO:teuthology.orchestra.run.smithi186.stdout:Running transaction 2024-04-12T20:28:49.847 INFO:teuthology.orchestra.run.smithi186.stdout: Preparing : 1/1 2024-04-12T20:28:50.338 INFO:teuthology.orchestra.run.smithi186.stdout: Upgrading : linux-firmware-20240219-143.el9.noarch 1/2 2024-04-12T20:28:50.437 INFO:teuthology.orchestra.run.smithi186.stdout: Cleanup : linux-firmware-20230814-139.el9.noarch 2/2 2024-04-12T20:28:51.375 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: linux-firmware-20230814-139.el9.noarch 2/2 2024-04-12T20:28:51.375 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : linux-firmware-20240219-143.el9.noarch 1/2 2024-04-12T20:28:51.800 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : linux-firmware-20230814-139.el9.noarch 2/2 2024-04-12T20:28:51.801 INFO:teuthology.orchestra.run.smithi186.stdout: 2024-04-12T20:28:51.801 INFO:teuthology.orchestra.run.smithi186.stdout:Upgraded: 2024-04-12T20:28:51.801 INFO:teuthology.orchestra.run.smithi186.stdout: linux-firmware-20240219-143.el9.noarch 2024-04-12T20:28:51.801 INFO:teuthology.orchestra.run.smithi186.stdout: 2024-04-12T20:28:51.801 INFO:teuthology.orchestra.run.smithi186.stdout:Complete! 2024-04-12T20:28:52.123 INFO:teuthology.task.kernel:Downloading kernel 3f60ed14f7c110c7ce3c8d83dd2ed0d08e80d0c8 on host.b... 2024-04-12T20:28:52.124 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&sha1=3f60ed14f7c110c7ce3c8d83dd2ed0d08e80d0c8 2024-04-12T20:28:52.272 INFO:teuthology.task.kernel:fetching, builder baseurl is https://3.chacra.ceph.com/r/kernel/testing/3f60ed14f7c110c7ce3c8d83dd2ed0d08e80d0c8/centos/9/flavors/default/x86_64/ 2024-04-12T20:28:52.272 DEBUG:teuthology.orchestra.run.smithi186:> rm -f /tmp/kernel.x86_64.rpm && echo kernel-6.9.0_rc3_g3f60ed14f7c1-1.x86_64.rpm | wget -nv -O /tmp/kernel.x86_64.rpm --base=https://3.chacra.ceph.com/r/kernel/testing/3f60ed14f7c110c7ce3c8d83dd2ed0d08e80d0c8/centos/9/flavors/default/x86_64/ --input-file=- 2024-04-12T20:28:52.274 DEBUG:teuthology.task.kernel:Waiting for download/copy to ubuntu@smithi186.front.sepia.ceph.com to complete... 2024-04-12T20:29:37.617 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12 20:29:37 URL:https://3.chacra.ceph.com/r/kernel/testing/3f60ed14f7c110c7ce3c8d83dd2ed0d08e80d0c8/centos/9/flavors/default/x86_64/kernel-6.9.0_rc3_g3f60ed14f7c1-1.x86_64.rpm [357344123/357344123] -> "/tmp/kernel.x86_64.rpm" [1] 2024-04-12T20:29:37.618 INFO:teuthology.orchestra.run.smithi018.stderr:FINISHED --2024-04-12 20:29:37-- 2024-04-12T20:29:37.618 INFO:teuthology.orchestra.run.smithi018.stderr:Total wall clock time: 1m 3s 2024-04-12T20:29:37.618 INFO:teuthology.orchestra.run.smithi018.stderr:Downloaded: 1 files, 341M in 1m 2s (5.48 MB/s) 2024-04-12T20:29:37.621 INFO:teuthology.task.kernel:Installing kernel 3f60ed14f7c110c7ce3c8d83dd2ed0d08e80d0c8 on host.a... 2024-04-12T20:29:37.621 DEBUG:teuthology.orchestra.run.smithi018:> sudo rpm -ivh --oldpackage --replacefiles --replacepkgs /tmp/kernel.x86_64.rpm 2024-04-12T20:29:39.782 INFO:teuthology.orchestra.run.smithi018.stdout:Verifying... ######################################## 2024-04-12T20:29:39.845 INFO:teuthology.orchestra.run.smithi018.stdout:Preparing... ######################################## 2024-04-12T20:29:39.930 INFO:teuthology.orchestra.run.smithi018.stdout:Updating / installing... 2024-04-12T20:29:48.231 INFO:teuthology.orchestra.run.smithi018.stdout:kernel-6.9.0_rc3_g3f60ed14f7c1-1 ######################################## 2024-04-12T20:29:50.926 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12 20:29:50 URL:https://3.chacra.ceph.com/r/kernel/testing/3f60ed14f7c110c7ce3c8d83dd2ed0d08e80d0c8/centos/9/flavors/default/x86_64/kernel-6.9.0_rc3_g3f60ed14f7c1-1.x86_64.rpm [357344123/357344123] -> "/tmp/kernel.x86_64.rpm" [1] 2024-04-12T20:29:50.926 INFO:teuthology.orchestra.run.smithi186.stderr:FINISHED --2024-04-12 20:29:50-- 2024-04-12T20:29:50.926 INFO:teuthology.orchestra.run.smithi186.stderr:Total wall clock time: 59s 2024-04-12T20:29:50.926 INFO:teuthology.orchestra.run.smithi186.stderr:Downloaded: 1 files, 341M in 58s (5.88 MB/s) 2024-04-12T20:29:50.929 INFO:teuthology.task.kernel:Installing kernel 3f60ed14f7c110c7ce3c8d83dd2ed0d08e80d0c8 on host.b... 2024-04-12T20:29:50.929 DEBUG:teuthology.orchestra.run.smithi186:> sudo rpm -ivh --oldpackage --replacefiles --replacepkgs /tmp/kernel.x86_64.rpm 2024-04-12T20:29:52.981 INFO:teuthology.orchestra.run.smithi186.stdout:Verifying... ######################################## 2024-04-12T20:29:53.033 INFO:teuthology.orchestra.run.smithi186.stdout:Preparing... ######################################## 2024-04-12T20:29:53.112 INFO:teuthology.orchestra.run.smithi186.stdout:Updating / installing... 2024-04-12T20:30:01.068 INFO:teuthology.orchestra.run.smithi186.stdout:kernel-6.9.0_rc3_g3f60ed14f7c1-1 ######################################## 2024-04-12T20:30:02.853 INFO:teuthology.orchestra.run.smithi018.stderr:dracut: Disabling early microcode, because kernel does not support it. CONFIG_MICROCODE_[AMD|INTEL]!=y 2024-04-12T20:30:15.510 INFO:teuthology.orchestra.run.smithi186.stderr:dracut: Disabling early microcode, because kernel does not support it. CONFIG_MICROCODE_[AMD|INTEL]!=y 2024-04-12T20:30:26.708 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi018.front.sepia.ceph.com, path=/tmp/kernel.x86_64.rpm, version=None) 2024-04-12T20:30:26.708 DEBUG:teuthology.orchestra.run.smithi018:> rpm -qlp /tmp/kernel.x86_64.rpm 2024-04-12T20:30:26.780 INFO:teuthology.orchestra.run.smithi018.stdout:/boot/System.map-6.9.0-rc3-g3f60ed14f7c1 2024-04-12T20:30:26.780 INFO:teuthology.orchestra.run.smithi018.stdout:/boot/config-6.9.0-rc3-g3f60ed14f7c1 2024-04-12T20:30:26.780 INFO:teuthology.orchestra.run.smithi018.stdout:/boot/vmlinuz-6.9.0-rc3-g3f60ed14f7c1 2024-04-12T20:30:26.780 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/System.map 2024-04-12T20:30:26.780 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/config 2024-04-12T20:30:26.780 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel 2024-04-12T20:30:26.780 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch 2024-04-12T20:30:26.780 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch/x86 2024-04-12T20:30:26.780 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch/x86/crypto 2024-04-12T20:30:26.781 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch/x86/crypto/blowfish-x86_64.ko 2024-04-12T20:30:26.781 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch/x86/crypto/camellia-aesni-avx-x86_64.ko 2024-04-12T20:30:26.781 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch/x86/crypto/camellia-aesni-avx2.ko 2024-04-12T20:30:26.781 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch/x86/crypto/camellia-x86_64.ko 2024-04-12T20:30:26.781 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch/x86/crypto/cast5-avx-x86_64.ko 2024-04-12T20:30:26.781 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch/x86/crypto/cast6-avx-x86_64.ko 2024-04-12T20:30:26.781 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch/x86/crypto/crc32-pclmul.ko 2024-04-12T20:30:26.781 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch/x86/crypto/crc32c-intel.ko 2024-04-12T20:30:26.781 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch/x86/crypto/crct10dif-pclmul.ko 2024-04-12T20:30:26.781 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch/x86/crypto/des3_ede-x86_64.ko 2024-04-12T20:30:26.781 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch/x86/crypto/ghash-clmulni-intel.ko 2024-04-12T20:30:26.781 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch/x86/crypto/serpent-avx-x86_64.ko 2024-04-12T20:30:26.781 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch/x86/crypto/serpent-avx2.ko 2024-04-12T20:30:26.781 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch/x86/crypto/serpent-sse2-x86_64.ko 2024-04-12T20:30:26.782 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch/x86/crypto/sha1-ssse3.ko 2024-04-12T20:30:26.782 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch/x86/crypto/sha256-ssse3.ko 2024-04-12T20:30:26.782 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch/x86/crypto/sha512-ssse3.ko 2024-04-12T20:30:26.782 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch/x86/crypto/twofish-avx-x86_64.ko 2024-04-12T20:30:26.782 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch/x86/crypto/twofish-x86_64-3way.ko 2024-04-12T20:30:26.782 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch/x86/crypto/twofish-x86_64.ko 2024-04-12T20:30:26.782 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch/x86/kernel 2024-04-12T20:30:26.782 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch/x86/kernel/cpu 2024-04-12T20:30:26.782 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch/x86/kernel/cpu/mce 2024-04-12T20:30:26.782 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch/x86/kernel/cpu/mce/mce-inject.ko 2024-04-12T20:30:26.782 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch/x86/kvm 2024-04-12T20:30:26.782 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch/x86/kvm/kvm-amd.ko 2024-04-12T20:30:26.783 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch/x86/kvm/kvm-intel.ko 2024-04-12T20:30:26.783 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch/x86/kvm/kvm.ko 2024-04-12T20:30:26.783 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto 2024-04-12T20:30:26.783 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/ansi_cprng.ko 2024-04-12T20:30:26.783 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/anubis.ko 2024-04-12T20:30:26.783 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/arc4.ko 2024-04-12T20:30:26.783 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/async_tx 2024-04-12T20:30:26.783 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/async_tx/async_memcpy.ko 2024-04-12T20:30:26.783 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/async_tx/async_pq.ko 2024-04-12T20:30:26.783 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/async_tx/async_raid6_recov.ko 2024-04-12T20:30:26.783 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/async_tx/async_tx.ko 2024-04-12T20:30:26.783 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/async_tx/async_xor.ko 2024-04-12T20:30:26.783 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/async_tx/raid6test.ko 2024-04-12T20:30:26.783 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/authenc.ko 2024-04-12T20:30:26.784 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/authencesn.ko 2024-04-12T20:30:26.784 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/blake2b_generic.ko 2024-04-12T20:30:26.784 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/blowfish_common.ko 2024-04-12T20:30:26.784 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/blowfish_generic.ko 2024-04-12T20:30:26.784 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/camellia_generic.ko 2024-04-12T20:30:26.784 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/cast5_generic.ko 2024-04-12T20:30:26.784 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/cast6_generic.ko 2024-04-12T20:30:26.784 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/cast_common.ko 2024-04-12T20:30:26.784 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/ccm.ko 2024-04-12T20:30:26.784 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/cmac.ko 2024-04-12T20:30:26.784 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/crc32_generic.ko 2024-04-12T20:30:26.784 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/crypto_engine.ko 2024-04-12T20:30:26.784 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/crypto_user.ko 2024-04-12T20:30:26.784 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/cts.ko 2024-04-12T20:30:26.784 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/des_generic.ko 2024-04-12T20:30:26.785 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/dh_generic.ko 2024-04-12T20:30:26.785 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/echainiv.ko 2024-04-12T20:30:26.785 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/essiv.ko 2024-04-12T20:30:26.785 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/fcrypt.ko 2024-04-12T20:30:26.785 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/khazad.ko 2024-04-12T20:30:26.785 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/lz4.ko 2024-04-12T20:30:26.785 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/lz4hc.ko 2024-04-12T20:30:26.785 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/md4.ko 2024-04-12T20:30:26.785 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/michael_mic.ko 2024-04-12T20:30:26.785 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/pcbc.ko 2024-04-12T20:30:26.785 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/pcrypt.ko 2024-04-12T20:30:26.785 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/rmd160.ko 2024-04-12T20:30:26.785 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/seed.ko 2024-04-12T20:30:26.785 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/serpent_generic.ko 2024-04-12T20:30:26.785 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/tcrypt.ko 2024-04-12T20:30:26.786 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/tea.ko 2024-04-12T20:30:26.786 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/twofish_common.ko 2024-04-12T20:30:26.786 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/twofish_generic.ko 2024-04-12T20:30:26.786 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/vmac.ko 2024-04-12T20:30:26.786 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/wp512.ko 2024-04-12T20:30:26.786 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/xcbc.ko 2024-04-12T20:30:26.786 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/xor.ko 2024-04-12T20:30:26.786 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/xxhash_generic.ko 2024-04-12T20:30:26.786 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers 2024-04-12T20:30:26.786 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/acpi 2024-04-12T20:30:26.786 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/acpi/acpi_ipmi.ko 2024-04-12T20:30:26.786 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/acpi/acpi_pad.ko 2024-04-12T20:30:26.786 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/acpi/ec_sys.ko 2024-04-12T20:30:26.787 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/acpi/platform_profile.ko 2024-04-12T20:30:26.787 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/acpi/sbs.ko 2024-04-12T20:30:26.787 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/acpi/sbshc.ko 2024-04-12T20:30:26.787 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/acpi/video.ko 2024-04-12T20:30:26.787 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata 2024-04-12T20:30:26.787 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/acard-ahci.ko 2024-04-12T20:30:26.787 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/ahci_platform.ko 2024-04-12T20:30:26.787 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/ata_generic.ko 2024-04-12T20:30:26.787 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/libahci_platform.ko 2024-04-12T20:30:26.787 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_acpi.ko 2024-04-12T20:30:26.787 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_ali.ko 2024-04-12T20:30:26.787 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_amd.ko 2024-04-12T20:30:26.787 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_artop.ko 2024-04-12T20:30:26.787 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_atiixp.ko 2024-04-12T20:30:26.787 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_atp867x.ko 2024-04-12T20:30:26.788 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_cmd640.ko 2024-04-12T20:30:26.788 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_cmd64x.ko 2024-04-12T20:30:26.788 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_cypress.ko 2024-04-12T20:30:26.788 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_efar.ko 2024-04-12T20:30:26.788 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_hpt366.ko 2024-04-12T20:30:26.788 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_hpt37x.ko 2024-04-12T20:30:26.788 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_hpt3x2n.ko 2024-04-12T20:30:26.788 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_hpt3x3.ko 2024-04-12T20:30:26.788 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_it8213.ko 2024-04-12T20:30:26.788 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_it821x.ko 2024-04-12T20:30:26.788 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_jmicron.ko 2024-04-12T20:30:26.788 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_marvell.ko 2024-04-12T20:30:26.788 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_mpiix.ko 2024-04-12T20:30:26.788 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_netcell.ko 2024-04-12T20:30:26.789 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_ninja32.ko 2024-04-12T20:30:26.789 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_ns87410.ko 2024-04-12T20:30:26.789 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_ns87415.ko 2024-04-12T20:30:26.789 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_oldpiix.ko 2024-04-12T20:30:26.789 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_opti.ko 2024-04-12T20:30:26.789 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_optidma.ko 2024-04-12T20:30:26.789 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_pcmcia.ko 2024-04-12T20:30:26.789 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_pdc2027x.ko 2024-04-12T20:30:26.789 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_pdc202xx_old.ko 2024-04-12T20:30:26.789 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_piccolo.ko 2024-04-12T20:30:26.789 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_rdc.ko 2024-04-12T20:30:26.789 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_sch.ko 2024-04-12T20:30:26.790 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_serverworks.ko 2024-04-12T20:30:26.790 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_sil680.ko 2024-04-12T20:30:26.790 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_sis.ko 2024-04-12T20:30:26.790 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_sl82c105.ko 2024-04-12T20:30:26.790 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_triflex.ko 2024-04-12T20:30:26.790 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_via.ko 2024-04-12T20:30:26.790 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pdc_adma.ko 2024-04-12T20:30:26.790 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/sata_inic162x.ko 2024-04-12T20:30:26.790 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/sata_mv.ko 2024-04-12T20:30:26.790 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/sata_nv.ko 2024-04-12T20:30:26.790 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/sata_promise.ko 2024-04-12T20:30:26.790 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/sata_qstor.ko 2024-04-12T20:30:26.790 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/sata_sil.ko 2024-04-12T20:30:26.790 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/sata_sil24.ko 2024-04-12T20:30:26.791 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/sata_sis.ko 2024-04-12T20:30:26.791 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/sata_svw.ko 2024-04-12T20:30:26.791 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/sata_sx4.ko 2024-04-12T20:30:26.791 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/sata_uli.ko 2024-04-12T20:30:26.791 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/sata_via.ko 2024-04-12T20:30:26.791 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/sata_vsc.ko 2024-04-12T20:30:26.791 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/auxdisplay 2024-04-12T20:30:26.791 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/auxdisplay/ks0108.ko 2024-04-12T20:30:26.791 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/base 2024-04-12T20:30:26.791 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/base/regmap 2024-04-12T20:30:26.791 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/base/regmap/regmap-i2c.ko 2024-04-12T20:30:26.791 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/base/regmap/regmap-mmio.ko 2024-04-12T20:30:26.791 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/bcma 2024-04-12T20:30:26.792 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/bcma/bcma.ko 2024-04-12T20:30:26.792 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/block 2024-04-12T20:30:26.792 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/block/aoe 2024-04-12T20:30:26.792 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/block/aoe/aoe.ko 2024-04-12T20:30:26.792 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/block/brd.ko 2024-04-12T20:30:26.792 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/block/drbd 2024-04-12T20:30:26.792 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/block/drbd/drbd.ko 2024-04-12T20:30:26.792 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/block/floppy.ko 2024-04-12T20:30:26.792 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/block/loop.ko 2024-04-12T20:30:26.792 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/block/mtip32xx 2024-04-12T20:30:26.792 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/block/mtip32xx/mtip32xx.ko 2024-04-12T20:30:26.792 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/block/nbd.ko 2024-04-12T20:30:26.792 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/block/null_blk 2024-04-12T20:30:26.792 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/block/null_blk/null_blk.ko 2024-04-12T20:30:26.792 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/block/pktcdvd.ko 2024-04-12T20:30:26.793 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/block/rbd.ko 2024-04-12T20:30:26.793 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/block/virtio_blk.ko 2024-04-12T20:30:26.793 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/block/xen-blkback 2024-04-12T20:30:26.793 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/block/xen-blkback/xen-blkback.ko 2024-04-12T20:30:26.793 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/block/xen-blkfront.ko 2024-04-12T20:30:26.793 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/block/zram 2024-04-12T20:30:26.793 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/block/zram/zram.ko 2024-04-12T20:30:26.793 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/char 2024-04-12T20:30:26.793 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/char/hangcheck-timer.ko 2024-04-12T20:30:26.793 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/char/hw_random 2024-04-12T20:30:26.793 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/char/hw_random/amd-rng.ko 2024-04-12T20:30:26.793 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/char/hw_random/intel-rng.ko 2024-04-12T20:30:26.793 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/char/hw_random/timeriomem-rng.ko 2024-04-12T20:30:26.793 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/char/hw_random/via-rng.ko 2024-04-12T20:30:26.793 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/char/hw_random/virtio-rng.ko 2024-04-12T20:30:26.794 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/char/ipmi 2024-04-12T20:30:26.794 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/char/ipmi/ipmi_devintf.ko 2024-04-12T20:30:26.794 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/char/ipmi/ipmi_msghandler.ko 2024-04-12T20:30:26.794 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/char/ipmi/ipmi_poweroff.ko 2024-04-12T20:30:26.794 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/char/ipmi/ipmi_si.ko 2024-04-12T20:30:26.794 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/char/ipmi/ipmi_ssif.ko 2024-04-12T20:30:26.794 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/char/ipmi/ipmi_watchdog.ko 2024-04-12T20:30:26.794 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/char/lp.ko 2024-04-12T20:30:26.794 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/char/mwave 2024-04-12T20:30:26.794 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/char/mwave/mwave.ko 2024-04-12T20:30:26.794 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/char/ppdev.ko 2024-04-12T20:30:26.794 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/char/tlclk.ko 2024-04-12T20:30:26.794 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/char/tpm 2024-04-12T20:30:26.794 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/char/tpm/tpm.ko 2024-04-12T20:30:26.795 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/char/tpm/tpm_atmel.ko 2024-04-12T20:30:26.795 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/char/tpm/tpm_infineon.ko 2024-04-12T20:30:26.795 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/char/tpm/tpm_nsc.ko 2024-04-12T20:30:26.795 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/char/tpm/tpm_tis.ko 2024-04-12T20:30:26.795 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/char/tpm/tpm_tis_core.ko 2024-04-12T20:30:26.795 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/char/uv_mmtimer.ko 2024-04-12T20:30:26.795 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/char/virtio_console.ko 2024-04-12T20:30:26.795 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/cpufreq 2024-04-12T20:30:26.795 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/cpufreq/acpi-cpufreq.ko 2024-04-12T20:30:26.795 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/cpufreq/amd_freq_sensitivity.ko 2024-04-12T20:30:26.795 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/cpufreq/p4-clockmod.ko 2024-04-12T20:30:26.795 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/cpufreq/pcc-cpufreq.ko 2024-04-12T20:30:26.795 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/cpufreq/powernow-k8.ko 2024-04-12T20:30:26.795 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/cpufreq/speedstep-lib.ko 2024-04-12T20:30:26.795 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/crypto 2024-04-12T20:30:26.795 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/crypto/ccp 2024-04-12T20:30:26.796 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/crypto/ccp/ccp-crypto.ko 2024-04-12T20:30:26.796 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/crypto/ccp/ccp.ko 2024-04-12T20:30:26.796 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/crypto/intel 2024-04-12T20:30:26.796 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/crypto/intel/qat 2024-04-12T20:30:26.796 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/crypto/intel/qat/qat_common 2024-04-12T20:30:26.796 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/crypto/intel/qat/qat_common/intel_qat.ko 2024-04-12T20:30:26.796 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/crypto/intel/qat/qat_dh895xcc 2024-04-12T20:30:26.796 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/crypto/intel/qat/qat_dh895xcc/qat_dh895xcc.ko 2024-04-12T20:30:26.796 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/crypto/padlock-aes.ko 2024-04-12T20:30:26.796 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/crypto/padlock-sha.ko 2024-04-12T20:30:26.796 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/crypto/virtio 2024-04-12T20:30:26.796 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/crypto/virtio/virtio_crypto.ko 2024-04-12T20:30:26.796 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/dca 2024-04-12T20:30:26.796 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/dca/dca.ko 2024-04-12T20:30:26.797 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/devfreq 2024-04-12T20:30:26.797 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/devfreq/governor_simpleondemand.ko 2024-04-12T20:30:26.797 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/dma 2024-04-12T20:30:26.797 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/dma/dw 2024-04-12T20:30:26.797 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/dma/dw/dw_dmac.ko 2024-04-12T20:30:26.797 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/dma/ioat 2024-04-12T20:30:26.797 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/dma/ioat/ioatdma.ko 2024-04-12T20:30:26.797 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/edac 2024-04-12T20:30:26.797 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/edac/amd64_edac.ko 2024-04-12T20:30:26.797 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/edac/e752x_edac.ko 2024-04-12T20:30:26.797 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/edac/edac_mce_amd.ko 2024-04-12T20:30:26.797 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/edac/i3000_edac.ko 2024-04-12T20:30:26.797 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/edac/i3200_edac.ko 2024-04-12T20:30:26.798 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/edac/i5100_edac.ko 2024-04-12T20:30:26.798 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/edac/i5400_edac.ko 2024-04-12T20:30:26.798 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/edac/i7300_edac.ko 2024-04-12T20:30:26.798 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/edac/i7core_edac.ko 2024-04-12T20:30:26.798 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/edac/i82975x_edac.ko 2024-04-12T20:30:26.798 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/edac/ie31200_edac.ko 2024-04-12T20:30:26.798 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/edac/sb_edac.ko 2024-04-12T20:30:26.798 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/edac/x38_edac.ko 2024-04-12T20:30:26.798 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/firmware 2024-04-12T20:30:26.798 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/firmware/edd.ko 2024-04-12T20:30:26.798 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/firmware/iscsi_ibft.ko 2024-04-12T20:30:26.798 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/gpio 2024-04-12T20:30:26.798 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/gpio/gpio-ich.ko 2024-04-12T20:30:26.799 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/gpio/gpio-viperboard.ko 2024-04-12T20:30:26.799 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid 2024-04-12T20:30:26.799 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-appleir.ko 2024-04-12T20:30:26.799 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-aureal.ko 2024-04-12T20:30:26.799 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-axff.ko 2024-04-12T20:30:26.799 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-dr.ko 2024-04-12T20:30:26.799 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-elecom.ko 2024-04-12T20:30:26.799 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-elo.ko 2024-04-12T20:30:26.799 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-emsff.ko 2024-04-12T20:30:26.799 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-gaff.ko 2024-04-12T20:30:26.799 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-gt683r.ko 2024-04-12T20:30:26.799 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-gyration.ko 2024-04-12T20:30:26.799 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-holtek-kbd.ko 2024-04-12T20:30:26.799 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-holtek-mouse.ko 2024-04-12T20:30:26.799 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-holtekff.ko 2024-04-12T20:30:26.799 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-hyperv.ko 2024-04-12T20:30:26.800 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-icade.ko 2024-04-12T20:30:26.800 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-keytouch.ko 2024-04-12T20:30:26.800 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-kye.ko 2024-04-12T20:30:26.800 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-lcpower.ko 2024-04-12T20:30:26.800 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-led.ko 2024-04-12T20:30:26.800 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-lenovo.ko 2024-04-12T20:30:26.800 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-logitech-dj.ko 2024-04-12T20:30:26.800 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-logitech-hidpp.ko 2024-04-12T20:30:26.800 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-multitouch.ko 2024-04-12T20:30:26.800 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-ortek.ko 2024-04-12T20:30:26.800 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-penmount.ko 2024-04-12T20:30:26.800 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-petalynx.ko 2024-04-12T20:30:26.800 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-picolcd.ko 2024-04-12T20:30:26.800 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-pl.ko 2024-04-12T20:30:26.800 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-plantronics.ko 2024-04-12T20:30:26.801 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-primax.ko 2024-04-12T20:30:26.801 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-rmi.ko 2024-04-12T20:30:26.801 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-roccat-arvo.ko 2024-04-12T20:30:26.801 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-roccat-common.ko 2024-04-12T20:30:26.801 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-roccat-isku.ko 2024-04-12T20:30:26.801 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-roccat-kone.ko 2024-04-12T20:30:26.801 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-roccat-koneplus.ko 2024-04-12T20:30:26.801 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-roccat-konepure.ko 2024-04-12T20:30:26.801 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-roccat-kovaplus.ko 2024-04-12T20:30:26.801 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-roccat-lua.ko 2024-04-12T20:30:26.801 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-roccat-pyra.ko 2024-04-12T20:30:26.801 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-roccat-ryos.ko 2024-04-12T20:30:26.801 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-roccat-savu.ko 2024-04-12T20:30:26.801 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-roccat.ko 2024-04-12T20:30:26.801 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-saitek.ko 2024-04-12T20:30:26.801 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-samsung.ko 2024-04-12T20:30:26.802 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-sensor-hub.ko 2024-04-12T20:30:26.802 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-sjoy.ko 2024-04-12T20:30:26.802 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-sony.ko 2024-04-12T20:30:26.802 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-speedlink.ko 2024-04-12T20:30:26.802 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-steelseries.ko 2024-04-12T20:30:26.802 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-sunplus.ko 2024-04-12T20:30:26.802 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-thrustmaster.ko 2024-04-12T20:30:26.802 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-tivo.ko 2024-04-12T20:30:26.802 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-tmff.ko 2024-04-12T20:30:26.802 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-topseed.ko 2024-04-12T20:30:26.802 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-twinhan.ko 2024-04-12T20:30:26.802 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-uclogic.ko 2024-04-12T20:30:26.802 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-waltop.ko 2024-04-12T20:30:26.803 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-wiimote.ko 2024-04-12T20:30:26.803 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-xinmo.ko 2024-04-12T20:30:26.803 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-zpff.ko 2024-04-12T20:30:26.803 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-zydacron.ko 2024-04-12T20:30:26.803 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/uhid.ko 2024-04-12T20:30:26.803 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/wacom.ko 2024-04-12T20:30:26.803 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hv 2024-04-12T20:30:26.803 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hv/hv_balloon.ko 2024-04-12T20:30:26.803 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hv/hv_utils.ko 2024-04-12T20:30:26.803 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hv/hv_vmbus.ko 2024-04-12T20:30:26.803 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon 2024-04-12T20:30:26.803 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/abituguru.ko 2024-04-12T20:30:26.803 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/abituguru3.ko 2024-04-12T20:30:26.803 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/acpi_power_meter.ko 2024-04-12T20:30:26.803 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/ad7414.ko 2024-04-12T20:30:26.803 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/ad7418.ko 2024-04-12T20:30:26.804 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/adc128d818.ko 2024-04-12T20:30:26.804 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/adm1025.ko 2024-04-12T20:30:26.804 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/adm1026.ko 2024-04-12T20:30:26.804 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/adm1029.ko 2024-04-12T20:30:26.804 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/adm1031.ko 2024-04-12T20:30:26.804 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/adm9240.ko 2024-04-12T20:30:26.804 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/ads7828.ko 2024-04-12T20:30:26.804 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/adt7410.ko 2024-04-12T20:30:26.804 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/adt7411.ko 2024-04-12T20:30:26.804 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/adt7462.ko 2024-04-12T20:30:26.804 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/adt7470.ko 2024-04-12T20:30:26.804 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/adt7475.ko 2024-04-12T20:30:26.804 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/adt7x10.ko 2024-04-12T20:30:26.804 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/amc6821.ko 2024-04-12T20:30:26.804 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/applesmc.ko 2024-04-12T20:30:26.804 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/asb100.ko 2024-04-12T20:30:26.805 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/asc7621.ko 2024-04-12T20:30:26.805 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/asus_atk0110.ko 2024-04-12T20:30:26.805 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/atxp1.ko 2024-04-12T20:30:26.805 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/coretemp.ko 2024-04-12T20:30:26.805 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/dell-smm-hwmon.ko 2024-04-12T20:30:26.805 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/dme1737.ko 2024-04-12T20:30:26.805 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/ds1621.ko 2024-04-12T20:30:26.805 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/ds620.ko 2024-04-12T20:30:26.805 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/emc1403.ko 2024-04-12T20:30:26.805 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/emc6w201.ko 2024-04-12T20:30:26.805 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/f71805f.ko 2024-04-12T20:30:26.805 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/f71882fg.ko 2024-04-12T20:30:26.805 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/f75375s.ko 2024-04-12T20:30:26.805 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/fam15h_power.ko 2024-04-12T20:30:26.806 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/fschmd.ko 2024-04-12T20:30:26.806 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/g760a.ko 2024-04-12T20:30:26.806 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/g762.ko 2024-04-12T20:30:26.806 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/gl518sm.ko 2024-04-12T20:30:26.806 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/gl520sm.ko 2024-04-12T20:30:26.806 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/hwmon-vid.ko 2024-04-12T20:30:26.806 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/i5500_temp.ko 2024-04-12T20:30:26.806 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/i5k_amb.ko 2024-04-12T20:30:26.806 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/ibmaem.ko 2024-04-12T20:30:26.806 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/ibmpex.ko 2024-04-12T20:30:26.806 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/ina209.ko 2024-04-12T20:30:26.806 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/ina2xx.ko 2024-04-12T20:30:26.806 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/it87.ko 2024-04-12T20:30:26.807 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/k10temp.ko 2024-04-12T20:30:26.807 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/k8temp.ko 2024-04-12T20:30:26.807 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/lineage-pem.ko 2024-04-12T20:30:26.807 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/lm63.ko 2024-04-12T20:30:26.807 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/lm73.ko 2024-04-12T20:30:26.807 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/lm75.ko 2024-04-12T20:30:26.807 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/lm77.ko 2024-04-12T20:30:26.807 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/lm78.ko 2024-04-12T20:30:26.807 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/lm80.ko 2024-04-12T20:30:26.807 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/lm83.ko 2024-04-12T20:30:26.807 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/lm85.ko 2024-04-12T20:30:26.807 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/lm87.ko 2024-04-12T20:30:26.807 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/lm90.ko 2024-04-12T20:30:26.807 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/lm92.ko 2024-04-12T20:30:26.807 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/lm93.ko 2024-04-12T20:30:26.807 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/lm95234.ko 2024-04-12T20:30:26.808 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/lm95241.ko 2024-04-12T20:30:26.808 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/lm95245.ko 2024-04-12T20:30:26.808 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/ltc2945.ko 2024-04-12T20:30:26.808 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/ltc4151.ko 2024-04-12T20:30:26.808 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/ltc4215.ko 2024-04-12T20:30:26.808 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/ltc4222.ko 2024-04-12T20:30:26.808 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/ltc4245.ko 2024-04-12T20:30:26.808 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/ltc4260.ko 2024-04-12T20:30:26.808 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/ltc4261.ko 2024-04-12T20:30:26.808 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/max16065.ko 2024-04-12T20:30:26.808 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/max1619.ko 2024-04-12T20:30:26.808 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/max1668.ko 2024-04-12T20:30:26.808 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/max197.ko 2024-04-12T20:30:26.808 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/max6639.ko 2024-04-12T20:30:26.808 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/max6650.ko 2024-04-12T20:30:26.808 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/max6697.ko 2024-04-12T20:30:26.809 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/mcp3021.ko 2024-04-12T20:30:26.809 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/nct6683.ko 2024-04-12T20:30:26.809 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/nct6775-core.ko 2024-04-12T20:30:26.809 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/nct6775.ko 2024-04-12T20:30:26.809 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/nct7802.ko 2024-04-12T20:30:26.809 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/pc87360.ko 2024-04-12T20:30:26.809 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/pc87427.ko 2024-04-12T20:30:26.809 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/pcf8591.ko 2024-04-12T20:30:26.809 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/pmbus 2024-04-12T20:30:26.809 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/pmbus/adm1275.ko 2024-04-12T20:30:26.809 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/pmbus/lm25066.ko 2024-04-12T20:30:26.809 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/pmbus/ltc2978.ko 2024-04-12T20:30:26.809 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/pmbus/max16064.ko 2024-04-12T20:30:26.809 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/pmbus/max34440.ko 2024-04-12T20:30:26.809 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/pmbus/max8688.ko 2024-04-12T20:30:26.809 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/pmbus/pmbus.ko 2024-04-12T20:30:26.810 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/pmbus/pmbus_core.ko 2024-04-12T20:30:26.810 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/pmbus/tps40422.ko 2024-04-12T20:30:26.810 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/pmbus/ucd9000.ko 2024-04-12T20:30:26.810 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/pmbus/ucd9200.ko 2024-04-12T20:30:26.810 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/pmbus/zl6100.ko 2024-04-12T20:30:26.810 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/powr1220.ko 2024-04-12T20:30:26.810 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/sch5627.ko 2024-04-12T20:30:26.810 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/sch5636.ko 2024-04-12T20:30:26.810 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/sch56xx-common.ko 2024-04-12T20:30:26.810 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/sht15.ko 2024-04-12T20:30:26.810 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/sht21.ko 2024-04-12T20:30:26.810 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/shtc1.ko 2024-04-12T20:30:26.810 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/sis5595.ko 2024-04-12T20:30:26.810 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/smsc47b397.ko 2024-04-12T20:30:26.810 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/smsc47m1.ko 2024-04-12T20:30:26.810 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/smsc47m192.ko 2024-04-12T20:30:26.811 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/thmc50.ko 2024-04-12T20:30:26.811 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/tmp102.ko 2024-04-12T20:30:26.811 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/tmp103.ko 2024-04-12T20:30:26.811 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/tmp401.ko 2024-04-12T20:30:26.811 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/tmp421.ko 2024-04-12T20:30:26.811 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/via-cputemp.ko 2024-04-12T20:30:26.811 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/via686a.ko 2024-04-12T20:30:26.811 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/vt1211.ko 2024-04-12T20:30:26.811 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/vt8231.ko 2024-04-12T20:30:26.811 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/w83627ehf.ko 2024-04-12T20:30:26.811 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/w83627hf.ko 2024-04-12T20:30:26.811 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/w83781d.ko 2024-04-12T20:30:26.811 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/w83791d.ko 2024-04-12T20:30:26.811 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/w83792d.ko 2024-04-12T20:30:26.811 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/w83793.ko 2024-04-12T20:30:26.812 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/w83795.ko 2024-04-12T20:30:26.812 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/w83l785ts.ko 2024-04-12T20:30:26.812 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/w83l786ng.ko 2024-04-12T20:30:26.812 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c 2024-04-12T20:30:26.812 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/algos 2024-04-12T20:30:26.812 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/algos/i2c-algo-bit.ko 2024-04-12T20:30:26.812 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/algos/i2c-algo-pca.ko 2024-04-12T20:30:26.812 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/busses 2024-04-12T20:30:26.812 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/busses/i2c-amd756-s4882.ko 2024-04-12T20:30:26.812 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/busses/i2c-amd756.ko 2024-04-12T20:30:26.812 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/busses/i2c-amd8111.ko 2024-04-12T20:30:26.812 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/busses/i2c-ccgx-ucsi.ko 2024-04-12T20:30:26.812 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/busses/i2c-designware-core.ko 2024-04-12T20:30:26.812 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/busses/i2c-designware-pci.ko 2024-04-12T20:30:26.813 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/busses/i2c-designware-platform.ko 2024-04-12T20:30:26.813 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/busses/i2c-diolan-u2c.ko 2024-04-12T20:30:26.813 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/busses/i2c-i801.ko 2024-04-12T20:30:26.813 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/busses/i2c-isch.ko 2024-04-12T20:30:26.813 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/busses/i2c-ismt.ko 2024-04-12T20:30:26.813 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/busses/i2c-nforce2-s4985.ko 2024-04-12T20:30:26.813 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/busses/i2c-nforce2.ko 2024-04-12T20:30:26.813 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/busses/i2c-parport.ko 2024-04-12T20:30:26.813 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/busses/i2c-pca-platform.ko 2024-04-12T20:30:26.813 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/busses/i2c-piix4.ko 2024-04-12T20:30:26.813 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/busses/i2c-scmi.ko 2024-04-12T20:30:26.813 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/busses/i2c-simtec.ko 2024-04-12T20:30:26.813 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/busses/i2c-sis96x.ko 2024-04-12T20:30:26.813 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/busses/i2c-tiny-usb.ko 2024-04-12T20:30:26.813 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/busses/i2c-via.ko 2024-04-12T20:30:26.814 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/busses/i2c-viapro.ko 2024-04-12T20:30:26.814 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/busses/i2c-viperboard.ko 2024-04-12T20:30:26.814 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/i2c-dev.ko 2024-04-12T20:30:26.814 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/i2c-mux.ko 2024-04-12T20:30:26.814 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/i2c-slave-eeprom.ko 2024-04-12T20:30:26.814 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/i2c-smbus.ko 2024-04-12T20:30:26.814 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/i2c-stub.ko 2024-04-12T20:30:26.814 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/infiniband 2024-04-12T20:30:26.814 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/infiniband/core 2024-04-12T20:30:26.814 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/infiniband/core/ib_cm.ko 2024-04-12T20:30:26.814 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/infiniband/core/ib_core.ko 2024-04-12T20:30:26.814 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/infiniband/core/ib_umad.ko 2024-04-12T20:30:26.814 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/infiniband/core/ib_uverbs.ko 2024-04-12T20:30:26.814 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/infiniband/core/iw_cm.ko 2024-04-12T20:30:26.815 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/infiniband/core/rdma_cm.ko 2024-04-12T20:30:26.815 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/infiniband/core/rdma_ucm.ko 2024-04-12T20:30:26.815 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/infiniband/hw 2024-04-12T20:30:26.815 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/infiniband/hw/cxgb4 2024-04-12T20:30:26.815 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/infiniband/hw/cxgb4/iw_cxgb4.ko 2024-04-12T20:30:26.815 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/infiniband/hw/mlx4 2024-04-12T20:30:26.815 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/infiniband/hw/mlx4/mlx4_ib.ko 2024-04-12T20:30:26.815 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/infiniband/hw/mlx5 2024-04-12T20:30:26.815 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/infiniband/hw/mlx5/mlx5_ib.ko 2024-04-12T20:30:26.815 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/infiniband/hw/mthca 2024-04-12T20:30:26.815 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/infiniband/hw/mthca/ib_mthca.ko 2024-04-12T20:30:26.815 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/infiniband/ulp 2024-04-12T20:30:26.815 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/infiniband/ulp/ipoib 2024-04-12T20:30:26.815 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/infiniband/ulp/ipoib/ib_ipoib.ko 2024-04-12T20:30:26.815 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/infiniband/ulp/iser 2024-04-12T20:30:26.815 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/infiniband/ulp/iser/ib_iser.ko 2024-04-12T20:30:26.816 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/infiniband/ulp/isert 2024-04-12T20:30:26.816 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/infiniband/ulp/isert/ib_isert.ko 2024-04-12T20:30:26.816 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/infiniband/ulp/srp 2024-04-12T20:30:26.816 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/infiniband/ulp/srp/ib_srp.ko 2024-04-12T20:30:26.816 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/infiniband/ulp/srpt 2024-04-12T20:30:26.816 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/infiniband/ulp/srpt/ib_srpt.ko 2024-04-12T20:30:26.816 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/input 2024-04-12T20:30:26.816 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/input/gameport 2024-04-12T20:30:26.816 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/input/gameport/emu10k1-gp.ko 2024-04-12T20:30:26.816 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/input/gameport/fm801-gp.ko 2024-04-12T20:30:26.816 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/input/gameport/gameport.ko 2024-04-12T20:30:26.816 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/input/joydev.ko 2024-04-12T20:30:26.816 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/input/keyboard 2024-04-12T20:30:26.816 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/input/keyboard/gpio_keys.ko 2024-04-12T20:30:26.816 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/input/mouse 2024-04-12T20:30:26.816 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/input/mouse/appletouch.ko 2024-04-12T20:30:26.817 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/input/mouse/bcm5974.ko 2024-04-12T20:30:26.817 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/input/mouse/cyapatp.ko 2024-04-12T20:30:26.817 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/input/mouse/elan_i2c.ko 2024-04-12T20:30:26.817 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/input/mouse/sermouse.ko 2024-04-12T20:30:26.817 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/input/mouse/synaptics_i2c.ko 2024-04-12T20:30:26.817 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/input/mouse/synaptics_usb.ko 2024-04-12T20:30:26.817 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/input/mouse/vsxxxaa.ko 2024-04-12T20:30:26.817 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/input/rmi4 2024-04-12T20:30:26.817 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/input/rmi4/rmi_core.ko 2024-04-12T20:30:26.817 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/input/serio 2024-04-12T20:30:26.817 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/input/serio/altera_ps2.ko 2024-04-12T20:30:26.817 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/input/serio/arc_ps2.ko 2024-04-12T20:30:26.817 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/input/serio/hyperv-keyboard.ko 2024-04-12T20:30:26.817 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/input/serio/serio_raw.ko 2024-04-12T20:30:26.817 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/input/sparse-keymap.ko 2024-04-12T20:30:26.818 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/leds 2024-04-12T20:30:26.818 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/leds/leds-blinkm.ko 2024-04-12T20:30:26.818 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/leds/leds-lm3530.ko 2024-04-12T20:30:26.818 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/leds/leds-lp3944.ko 2024-04-12T20:30:26.818 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/leds/leds-ss4200.ko 2024-04-12T20:30:26.818 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/leds/trigger 2024-04-12T20:30:26.818 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/leds/trigger/ledtrig-audio.ko 2024-04-12T20:30:26.818 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/leds/trigger/ledtrig-backlight.ko 2024-04-12T20:30:26.818 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/leds/trigger/ledtrig-camera.ko 2024-04-12T20:30:26.818 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/leds/trigger/ledtrig-default-on.ko 2024-04-12T20:30:26.818 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/leds/trigger/ledtrig-gpio.ko 2024-04-12T20:30:26.818 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/leds/trigger/ledtrig-heartbeat.ko 2024-04-12T20:30:26.818 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/leds/trigger/ledtrig-oneshot.ko 2024-04-12T20:30:26.819 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/leds/trigger/ledtrig-timer.ko 2024-04-12T20:30:26.819 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/leds/trigger/ledtrig-transient.ko 2024-04-12T20:30:26.819 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/md 2024-04-12T20:30:26.819 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/md/bcache 2024-04-12T20:30:26.819 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/md/bcache/bcache.ko 2024-04-12T20:30:26.819 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/md/dm-bio-prison.ko 2024-04-12T20:30:26.819 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/md/dm-cache-smq.ko 2024-04-12T20:30:26.819 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/md/dm-cache.ko 2024-04-12T20:30:26.819 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/md/dm-crypt.ko 2024-04-12T20:30:26.819 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/md/dm-delay.ko 2024-04-12T20:30:26.819 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/md/dm-flakey.ko 2024-04-12T20:30:26.819 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/md/dm-log-userspace.ko 2024-04-12T20:30:26.819 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/md/dm-multipath.ko 2024-04-12T20:30:26.819 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/md/dm-queue-length.ko 2024-04-12T20:30:26.819 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/md/dm-raid.ko 2024-04-12T20:30:26.819 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/md/dm-round-robin.ko 2024-04-12T20:30:26.820 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/md/dm-service-time.ko 2024-04-12T20:30:26.820 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/md/dm-switch.ko 2024-04-12T20:30:26.820 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/md/dm-thin-pool.ko 2024-04-12T20:30:26.820 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/md/dm-verity.ko 2024-04-12T20:30:26.820 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/md/persistent-data 2024-04-12T20:30:26.820 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/md/persistent-data/dm-persistent-data.ko 2024-04-12T20:30:26.820 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/md/raid0.ko 2024-04-12T20:30:26.820 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/md/raid1.ko 2024-04-12T20:30:26.820 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/md/raid10.ko 2024-04-12T20:30:26.820 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/md/raid456.ko 2024-04-12T20:30:26.820 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/message 2024-04-12T20:30:26.820 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/message/fusion 2024-04-12T20:30:26.820 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/message/fusion/mptbase.ko 2024-04-12T20:30:26.820 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/message/fusion/mptctl.ko 2024-04-12T20:30:26.821 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/message/fusion/mptfc.ko 2024-04-12T20:30:26.821 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/message/fusion/mptlan.ko 2024-04-12T20:30:26.821 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/message/fusion/mptsas.ko 2024-04-12T20:30:26.821 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/message/fusion/mptscsih.ko 2024-04-12T20:30:26.821 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/message/fusion/mptspi.ko 2024-04-12T20:30:26.821 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/mfd 2024-04-12T20:30:26.821 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/mfd/lpc_ich.ko 2024-04-12T20:30:26.821 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/mfd/lpc_sch.ko 2024-04-12T20:30:26.821 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/mfd/mfd-core.ko 2024-04-12T20:30:26.821 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/mfd/sm501.ko 2024-04-12T20:30:26.821 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/mfd/viperboard.ko 2024-04-12T20:30:26.821 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/mfd/vx855.ko 2024-04-12T20:30:26.821 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/mfd/wl1273-core.ko 2024-04-12T20:30:26.821 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc 2024-04-12T20:30:26.821 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/altera-stapl 2024-04-12T20:30:26.821 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/altera-stapl/altera-stapl.ko 2024-04-12T20:30:26.822 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/apds9802als.ko 2024-04-12T20:30:26.822 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/apds990x.ko 2024-04-12T20:30:26.822 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/bh1770glc.ko 2024-04-12T20:30:26.822 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/cb710 2024-04-12T20:30:26.822 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/cb710/cb710.ko 2024-04-12T20:30:26.822 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/eeprom 2024-04-12T20:30:26.822 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/eeprom/at24.ko 2024-04-12T20:30:26.822 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/eeprom/eeprom_93cx6.ko 2024-04-12T20:30:26.822 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/eeprom/max6875.ko 2024-04-12T20:30:26.822 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/enclosure.ko 2024-04-12T20:30:26.822 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/hpilo.ko 2024-04-12T20:30:26.822 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/isl29003.ko 2024-04-12T20:30:26.822 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/isl29020.ko 2024-04-12T20:30:26.822 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/lis3lv02d 2024-04-12T20:30:26.822 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/lis3lv02d/lis3lv02d.ko 2024-04-12T20:30:26.822 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/lis3lv02d/lis3lv02d_i2c.ko 2024-04-12T20:30:26.823 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/mei 2024-04-12T20:30:26.823 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/mei/mei-me.ko 2024-04-12T20:30:26.823 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/mei/mei-txe.ko 2024-04-12T20:30:26.823 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/mei/mei.ko 2024-04-12T20:30:26.823 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/sgi-gru 2024-04-12T20:30:26.823 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/sgi-gru/gru.ko 2024-04-12T20:30:26.823 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/sgi-xp 2024-04-12T20:30:26.823 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/sgi-xp/xp.ko 2024-04-12T20:30:26.823 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/sgi-xp/xpc.ko 2024-04-12T20:30:26.823 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/sgi-xp/xpnet.ko 2024-04-12T20:30:26.823 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/tifm_7xx1.ko 2024-04-12T20:30:26.823 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/tifm_core.ko 2024-04-12T20:30:26.823 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/tsl2550.ko 2024-04-12T20:30:26.823 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/vmw_balloon.ko 2024-04-12T20:30:26.823 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/vmw_vmci 2024-04-12T20:30:26.824 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/vmw_vmci/vmw_vmci.ko 2024-04-12T20:30:26.824 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net 2024-04-12T20:30:26.824 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/bonding 2024-04-12T20:30:26.824 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/bonding/bonding.ko 2024-04-12T20:30:26.824 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/dsa 2024-04-12T20:30:26.824 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/dsa/b53 2024-04-12T20:30:26.824 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/dsa/b53/b53_common.ko 2024-04-12T20:30:26.824 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/dsa/bcm-sf2.ko 2024-04-12T20:30:26.824 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/dsa/mv88e6060.ko 2024-04-12T20:30:26.824 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/dsa/mv88e6xxx 2024-04-12T20:30:26.824 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/dsa/mv88e6xxx/mv88e6xxx.ko 2024-04-12T20:30:26.824 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/dummy.ko 2024-04-12T20:30:26.824 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/eql.ko 2024-04-12T20:30:26.824 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet 2024-04-12T20:30:26.825 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/3com 2024-04-12T20:30:26.825 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/3com/3c574_cs.ko 2024-04-12T20:30:26.825 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/3com/3c589_cs.ko 2024-04-12T20:30:26.825 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/3com/3c59x.ko 2024-04-12T20:30:26.825 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/3com/typhoon.ko 2024-04-12T20:30:26.825 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/8390 2024-04-12T20:30:26.825 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/8390/8390.ko 2024-04-12T20:30:26.825 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/8390/axnet_cs.ko 2024-04-12T20:30:26.825 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/8390/ne2k-pci.ko 2024-04-12T20:30:26.825 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/8390/pcnet_cs.ko 2024-04-12T20:30:26.825 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/adaptec 2024-04-12T20:30:26.825 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/adaptec/starfire.ko 2024-04-12T20:30:26.825 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/agere 2024-04-12T20:30:26.825 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/agere/et131x.ko 2024-04-12T20:30:26.826 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/alteon 2024-04-12T20:30:26.826 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/alteon/acenic.ko 2024-04-12T20:30:26.826 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/altera 2024-04-12T20:30:26.826 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/altera/altera_tse.ko 2024-04-12T20:30:26.826 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/amd 2024-04-12T20:30:26.826 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/amd/amd8111e.ko 2024-04-12T20:30:26.826 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/amd/nmclan_cs.ko 2024-04-12T20:30:26.826 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/amd/pcnet32.ko 2024-04-12T20:30:26.826 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/atheros 2024-04-12T20:30:26.826 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/atheros/alx 2024-04-12T20:30:26.826 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/atheros/alx/alx.ko 2024-04-12T20:30:26.826 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/atheros/atl1c 2024-04-12T20:30:26.826 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/atheros/atl1c/atl1c.ko 2024-04-12T20:30:26.826 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/atheros/atl1e 2024-04-12T20:30:26.827 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/atheros/atl1e/atl1e.ko 2024-04-12T20:30:26.827 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/atheros/atlx 2024-04-12T20:30:26.827 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/atheros/atlx/atl1.ko 2024-04-12T20:30:26.827 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/atheros/atlx/atl2.ko 2024-04-12T20:30:26.827 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/broadcom 2024-04-12T20:30:26.827 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/broadcom/b44.ko 2024-04-12T20:30:26.827 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/broadcom/bnx2.ko 2024-04-12T20:30:26.827 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/broadcom/bnx2x 2024-04-12T20:30:26.827 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/broadcom/bnx2x/bnx2x.ko 2024-04-12T20:30:26.827 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/broadcom/cnic.ko 2024-04-12T20:30:26.827 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/broadcom/genet 2024-04-12T20:30:26.827 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/broadcom/genet/genet.ko 2024-04-12T20:30:26.827 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/broadcom/tg3.ko 2024-04-12T20:30:26.827 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/brocade 2024-04-12T20:30:26.828 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/brocade/bna 2024-04-12T20:30:26.828 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/brocade/bna/bna.ko 2024-04-12T20:30:26.828 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/chelsio 2024-04-12T20:30:26.828 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/chelsio/cxgb 2024-04-12T20:30:26.828 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/chelsio/cxgb/cxgb.ko 2024-04-12T20:30:26.828 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/chelsio/cxgb3 2024-04-12T20:30:26.828 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/chelsio/cxgb3/cxgb3.ko 2024-04-12T20:30:26.828 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/chelsio/cxgb4 2024-04-12T20:30:26.828 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/chelsio/cxgb4/cxgb4.ko 2024-04-12T20:30:26.828 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/chelsio/cxgb4vf 2024-04-12T20:30:26.828 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/chelsio/cxgb4vf/cxgb4vf.ko 2024-04-12T20:30:26.828 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/chelsio/libcxgb 2024-04-12T20:30:26.828 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/chelsio/libcxgb/libcxgb.ko 2024-04-12T20:30:26.828 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/cisco 2024-04-12T20:30:26.829 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/cisco/enic 2024-04-12T20:30:26.829 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/cisco/enic/enic.ko 2024-04-12T20:30:26.829 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/dec 2024-04-12T20:30:26.829 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/dec/tulip 2024-04-12T20:30:26.829 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/dec/tulip/de2104x.ko 2024-04-12T20:30:26.829 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/dec/tulip/dmfe.ko 2024-04-12T20:30:26.829 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/dec/tulip/tulip.ko 2024-04-12T20:30:26.829 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/dec/tulip/uli526x.ko 2024-04-12T20:30:26.829 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/dec/tulip/winbond-840.ko 2024-04-12T20:30:26.829 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/dec/tulip/xircom_cb.ko 2024-04-12T20:30:26.829 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/dlink 2024-04-12T20:30:26.829 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/dlink/dl2k.ko 2024-04-12T20:30:26.829 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/dlink/sundance.ko 2024-04-12T20:30:26.829 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/dnet.ko 2024-04-12T20:30:26.829 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/emulex 2024-04-12T20:30:26.830 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/emulex/benet 2024-04-12T20:30:26.830 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/emulex/benet/be2net.ko 2024-04-12T20:30:26.830 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/ethoc.ko 2024-04-12T20:30:26.830 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/fealnx.ko 2024-04-12T20:30:26.830 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/intel 2024-04-12T20:30:26.830 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/intel/e100.ko 2024-04-12T20:30:26.830 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/intel/e1000 2024-04-12T20:30:26.830 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/intel/e1000/e1000.ko 2024-04-12T20:30:26.830 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/intel/e1000e 2024-04-12T20:30:26.830 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/intel/e1000e/e1000e.ko 2024-04-12T20:30:26.830 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/intel/fm10k 2024-04-12T20:30:26.830 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/intel/fm10k/fm10k.ko 2024-04-12T20:30:26.830 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/intel/i40e 2024-04-12T20:30:26.830 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/intel/i40e/i40e.ko 2024-04-12T20:30:26.831 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/intel/iavf 2024-04-12T20:30:26.831 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/intel/iavf/iavf.ko 2024-04-12T20:30:26.831 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/intel/igb 2024-04-12T20:30:26.831 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/intel/igb/igb.ko 2024-04-12T20:30:26.831 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/intel/igbvf 2024-04-12T20:30:26.831 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/intel/igbvf/igbvf.ko 2024-04-12T20:30:26.831 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/intel/ixgbe 2024-04-12T20:30:26.831 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/intel/ixgbe/ixgbe.ko 2024-04-12T20:30:26.831 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/intel/ixgbevf 2024-04-12T20:30:26.831 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/intel/ixgbevf/ixgbevf.ko 2024-04-12T20:30:26.831 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/jme.ko 2024-04-12T20:30:26.831 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/marvell 2024-04-12T20:30:26.831 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/marvell/mvmdio.ko 2024-04-12T20:30:26.832 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/marvell/skge.ko 2024-04-12T20:30:26.832 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/marvell/sky2.ko 2024-04-12T20:30:26.832 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/mellanox 2024-04-12T20:30:26.832 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/mellanox/mlx4 2024-04-12T20:30:26.832 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_core.ko 2024-04-12T20:30:26.832 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_en.ko 2024-04-12T20:30:26.832 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/mellanox/mlx5 2024-04-12T20:30:26.832 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/mellanox/mlx5/core 2024-04-12T20:30:26.832 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.ko 2024-04-12T20:30:26.832 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/micrel 2024-04-12T20:30:26.832 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/micrel/ksz884x.ko 2024-04-12T20:30:26.832 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/myricom 2024-04-12T20:30:26.832 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/myricom/myri10ge 2024-04-12T20:30:26.832 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/myricom/myri10ge/myri10ge.ko 2024-04-12T20:30:26.832 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/natsemi 2024-04-12T20:30:26.832 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/natsemi/natsemi.ko 2024-04-12T20:30:26.833 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/natsemi/ns83820.ko 2024-04-12T20:30:26.833 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/neterion 2024-04-12T20:30:26.833 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/neterion/s2io.ko 2024-04-12T20:30:26.833 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/nvidia 2024-04-12T20:30:26.833 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/nvidia/forcedeth.ko 2024-04-12T20:30:26.833 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/packetengines 2024-04-12T20:30:26.833 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/packetengines/hamachi.ko 2024-04-12T20:30:26.833 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/packetengines/yellowfin.ko 2024-04-12T20:30:26.833 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/qlogic 2024-04-12T20:30:26.833 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/qlogic/netxen 2024-04-12T20:30:26.833 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/qlogic/netxen/netxen_nic.ko 2024-04-12T20:30:26.833 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/qlogic/qla3xxx.ko 2024-04-12T20:30:26.833 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/qlogic/qlcnic 2024-04-12T20:30:26.833 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/qlogic/qlcnic/qlcnic.ko 2024-04-12T20:30:26.834 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/rdc 2024-04-12T20:30:26.834 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/rdc/r6040.ko 2024-04-12T20:30:26.834 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/realtek 2024-04-12T20:30:26.834 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/realtek/8139cp.ko 2024-04-12T20:30:26.834 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/realtek/8139too.ko 2024-04-12T20:30:26.834 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/realtek/atp.ko 2024-04-12T20:30:26.834 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/realtek/r8169.ko 2024-04-12T20:30:26.834 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/rocker 2024-04-12T20:30:26.834 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/rocker/rocker.ko 2024-04-12T20:30:26.834 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/sfc 2024-04-12T20:30:26.834 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/sfc/sfc.ko 2024-04-12T20:30:26.834 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/silan 2024-04-12T20:30:26.834 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/silan/sc92031.ko 2024-04-12T20:30:26.835 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/sis 2024-04-12T20:30:26.835 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/sis/sis190.ko 2024-04-12T20:30:26.835 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/sis/sis900.ko 2024-04-12T20:30:26.835 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/smsc 2024-04-12T20:30:26.835 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/smsc/epic100.ko 2024-04-12T20:30:26.835 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/smsc/smc91c92_cs.ko 2024-04-12T20:30:26.835 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/smsc/smsc911x.ko 2024-04-12T20:30:26.835 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/smsc/smsc9420.ko 2024-04-12T20:30:26.835 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/stmicro 2024-04-12T20:30:26.835 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/stmicro/stmmac 2024-04-12T20:30:26.835 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-intel.ko 2024-04-12T20:30:26.835 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac.ko 2024-04-12T20:30:26.835 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/sun 2024-04-12T20:30:26.835 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/sun/cassini.ko 2024-04-12T20:30:26.835 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/sun/niu.ko 2024-04-12T20:30:26.836 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/sun/sungem.ko 2024-04-12T20:30:26.836 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/sun/sunhme.ko 2024-04-12T20:30:26.836 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/tehuti 2024-04-12T20:30:26.836 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/tehuti/tehuti.ko 2024-04-12T20:30:26.836 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/ti 2024-04-12T20:30:26.836 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/ti/tlan.ko 2024-04-12T20:30:26.836 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/via 2024-04-12T20:30:26.836 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/via/via-rhine.ko 2024-04-12T20:30:26.836 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/via/via-velocity.ko 2024-04-12T20:30:26.836 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/wiznet 2024-04-12T20:30:26.836 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/wiznet/w5100.ko 2024-04-12T20:30:26.836 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/wiznet/w5300.ko 2024-04-12T20:30:26.836 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/xircom 2024-04-12T20:30:26.836 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/xircom/xirc2ps_cs.ko 2024-04-12T20:30:26.836 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/geneve.ko 2024-04-12T20:30:26.836 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/hyperv 2024-04-12T20:30:26.837 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/hyperv/hv_netvsc.ko 2024-04-12T20:30:26.837 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ieee802154 2024-04-12T20:30:26.837 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ieee802154/fakelb.ko 2024-04-12T20:30:26.837 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ifb.ko 2024-04-12T20:30:26.837 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ipvlan 2024-04-12T20:30:26.837 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ipvlan/ipvlan.ko 2024-04-12T20:30:26.837 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/macvlan.ko 2024-04-12T20:30:26.837 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/macvtap.ko 2024-04-12T20:30:26.837 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/mdio 2024-04-12T20:30:26.837 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/mdio.ko 2024-04-12T20:30:26.837 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/mdio/mdio-bcm-unimac.ko 2024-04-12T20:30:26.837 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/mdio/mdio-bitbang.ko 2024-04-12T20:30:26.837 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/mdio/mdio-regmap.ko 2024-04-12T20:30:26.837 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/mii.ko 2024-04-12T20:30:26.837 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/net_failover.ko 2024-04-12T20:30:26.838 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/netconsole.ko 2024-04-12T20:30:26.838 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/nlmon.ko 2024-04-12T20:30:26.838 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/pcs 2024-04-12T20:30:26.838 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/pcs/pcs-lynx.ko 2024-04-12T20:30:26.838 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/pcs/pcs_xpcs.ko 2024-04-12T20:30:26.838 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/phy 2024-04-12T20:30:26.838 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/phy/amd.ko 2024-04-12T20:30:26.838 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/phy/bcm-phy-lib.ko 2024-04-12T20:30:26.838 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/phy/bcm-phy-ptp.ko 2024-04-12T20:30:26.838 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/phy/bcm7xxx.ko 2024-04-12T20:30:26.838 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/phy/bcm87xx.ko 2024-04-12T20:30:26.838 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/phy/broadcom.ko 2024-04-12T20:30:26.838 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/phy/cicada.ko 2024-04-12T20:30:26.838 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/phy/davicom.ko 2024-04-12T20:30:26.839 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/phy/dp83640.ko 2024-04-12T20:30:26.839 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/phy/et1011c.ko 2024-04-12T20:30:26.839 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/phy/icplus.ko 2024-04-12T20:30:26.839 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/phy/lxt.ko 2024-04-12T20:30:26.839 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/phy/marvell.ko 2024-04-12T20:30:26.839 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/phy/micrel.ko 2024-04-12T20:30:26.839 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/phy/national.ko 2024-04-12T20:30:26.839 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/phy/phylink.ko 2024-04-12T20:30:26.839 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/phy/qsemi.ko 2024-04-12T20:30:26.839 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/phy/realtek.ko 2024-04-12T20:30:26.839 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/phy/smsc.ko 2024-04-12T20:30:26.839 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/phy/ste10Xp.ko 2024-04-12T20:30:26.839 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/phy/vitesse.ko 2024-04-12T20:30:26.839 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ppp 2024-04-12T20:30:26.839 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ppp/bsd_comp.ko 2024-04-12T20:30:26.839 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ppp/ppp_async.ko 2024-04-12T20:30:26.840 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ppp/ppp_deflate.ko 2024-04-12T20:30:26.840 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ppp/ppp_generic.ko 2024-04-12T20:30:26.840 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ppp/ppp_mppe.ko 2024-04-12T20:30:26.840 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ppp/ppp_synctty.ko 2024-04-12T20:30:26.840 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ppp/pppoe.ko 2024-04-12T20:30:26.840 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ppp/pppox.ko 2024-04-12T20:30:26.840 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ppp/pptp.ko 2024-04-12T20:30:26.840 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/slip 2024-04-12T20:30:26.840 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/slip/slhc.ko 2024-04-12T20:30:26.840 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/slip/slip.ko 2024-04-12T20:30:26.840 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/sungem_phy.ko 2024-04-12T20:30:26.840 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/tap.ko 2024-04-12T20:30:26.840 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/team 2024-04-12T20:30:26.840 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/team/team.ko 2024-04-12T20:30:26.840 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/team/team_mode_activebackup.ko 2024-04-12T20:30:26.840 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/team/team_mode_broadcast.ko 2024-04-12T20:30:26.841 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/team/team_mode_loadbalance.ko 2024-04-12T20:30:26.841 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/team/team_mode_random.ko 2024-04-12T20:30:26.841 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/team/team_mode_roundrobin.ko 2024-04-12T20:30:26.841 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/tun.ko 2024-04-12T20:30:26.841 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/veth.ko 2024-04-12T20:30:26.841 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/virtio_net.ko 2024-04-12T20:30:26.841 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/vmxnet3 2024-04-12T20:30:26.841 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/vmxnet3/vmxnet3.ko 2024-04-12T20:30:26.841 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/vxlan 2024-04-12T20:30:26.841 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/vxlan/vxlan.ko 2024-04-12T20:30:26.841 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/xen-netback 2024-04-12T20:30:26.841 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/xen-netback/xen-netback.ko 2024-04-12T20:30:26.841 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/xen-netfront.ko 2024-04-12T20:30:26.841 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ntb 2024-04-12T20:30:26.841 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ntb/ntb.ko 2024-04-12T20:30:26.841 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/nvme 2024-04-12T20:30:26.842 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/nvme/host 2024-04-12T20:30:26.842 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/nvme/host/nvme-core.ko 2024-04-12T20:30:26.842 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/nvme/host/nvme.ko 2024-04-12T20:30:26.842 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/parport 2024-04-12T20:30:26.842 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/parport/parport.ko 2024-04-12T20:30:26.842 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/parport/parport_cs.ko 2024-04-12T20:30:26.842 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/parport/parport_pc.ko 2024-04-12T20:30:26.842 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/parport/parport_serial.ko 2024-04-12T20:30:26.842 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/pci 2024-04-12T20:30:26.842 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/pci/hotplug 2024-04-12T20:30:26.842 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/pci/hotplug/acpiphp_ibm.ko 2024-04-12T20:30:26.842 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/pci/pcie 2024-04-12T20:30:26.842 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/pci/pcie/aer_inject.ko 2024-04-12T20:30:26.842 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/pci/xen-pcifront.ko 2024-04-12T20:30:26.842 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/pcmcia 2024-04-12T20:30:26.843 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/pcmcia/i82092.ko 2024-04-12T20:30:26.843 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/pcmcia/pd6729.ko 2024-04-12T20:30:26.843 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/pcmcia/yenta_socket.ko 2024-04-12T20:30:26.843 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/pinctrl 2024-04-12T20:30:26.843 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/pinctrl/intel 2024-04-12T20:30:26.843 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/pinctrl/intel/pinctrl-cherryview.ko 2024-04-12T20:30:26.843 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/platform 2024-04-12T20:30:26.843 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/platform/x86 2024-04-12T20:30:26.843 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/platform/x86/acer-wmi.ko 2024-04-12T20:30:26.843 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/platform/x86/acerhdf.ko 2024-04-12T20:30:26.843 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/platform/x86/apple-gmux.ko 2024-04-12T20:30:26.843 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/platform/x86/asus-laptop.ko 2024-04-12T20:30:26.843 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/platform/x86/asus-nb-wmi.ko 2024-04-12T20:30:26.843 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/platform/x86/asus-wmi.ko 2024-04-12T20:30:26.843 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/platform/x86/classmate-laptop.ko 2024-04-12T20:30:26.843 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/platform/x86/eeepc-laptop.ko 2024-04-12T20:30:26.844 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/platform/x86/eeepc-wmi.ko 2024-04-12T20:30:26.844 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/platform/x86/fujitsu-laptop.ko 2024-04-12T20:30:26.844 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/platform/x86/fujitsu-tablet.ko 2024-04-12T20:30:26.844 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/platform/x86/hdaps.ko 2024-04-12T20:30:26.844 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/platform/x86/intel 2024-04-12T20:30:26.844 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/platform/x86/intel/intel-rst.ko 2024-04-12T20:30:26.844 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/platform/x86/intel_ips.ko 2024-04-12T20:30:26.844 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/platform/x86/msi-wmi.ko 2024-04-12T20:30:26.844 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/platform/x86/mxm-wmi.ko 2024-04-12T20:30:26.844 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/platform/x86/panasonic-laptop.ko 2024-04-12T20:30:26.844 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/platform/x86/samsung-laptop.ko 2024-04-12T20:30:26.844 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/platform/x86/samsung-q10.ko 2024-04-12T20:30:26.844 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/platform/x86/topstar-laptop.ko 2024-04-12T20:30:26.844 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/platform/x86/toshiba_bluetooth.ko 2024-04-12T20:30:26.844 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/platform/x86/toshiba_haps.ko 2024-04-12T20:30:26.845 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/platform/x86/wmi-bmof.ko 2024-04-12T20:30:26.845 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/platform/x86/wmi.ko 2024-04-12T20:30:26.845 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/powercap 2024-04-12T20:30:26.845 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/powercap/intel_rapl_common.ko 2024-04-12T20:30:26.845 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/powercap/intel_rapl_msr.ko 2024-04-12T20:30:26.845 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/pps 2024-04-12T20:30:26.845 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/pps/clients 2024-04-12T20:30:26.845 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/pps/clients/pps-gpio.ko 2024-04-12T20:30:26.845 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/pps/clients/pps-ldisc.ko 2024-04-12T20:30:26.845 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/pps/clients/pps_parport.ko 2024-04-12T20:30:26.845 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/pps/pps_core.ko 2024-04-12T20:30:26.845 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ptp 2024-04-12T20:30:26.845 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ptp/ptp.ko 2024-04-12T20:30:26.845 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ptp/ptp_kvm.ko 2024-04-12T20:30:26.845 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ras 2024-04-12T20:30:26.846 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ras/amd 2024-04-12T20:30:26.846 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ras/amd/atl 2024-04-12T20:30:26.846 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ras/amd/atl/amd_atl.ko 2024-04-12T20:30:26.846 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ras/amd/fmpm.ko 2024-04-12T20:30:26.846 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc 2024-04-12T20:30:26.846 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-bq32k.ko 2024-04-12T20:30:26.846 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-ds1286.ko 2024-04-12T20:30:26.846 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-ds1307.ko 2024-04-12T20:30:26.846 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-ds1374.ko 2024-04-12T20:30:26.846 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-ds1511.ko 2024-04-12T20:30:26.846 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-ds1553.ko 2024-04-12T20:30:26.846 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-ds1672.ko 2024-04-12T20:30:26.846 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-ds1742.ko 2024-04-12T20:30:26.847 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-ds2404.ko 2024-04-12T20:30:26.847 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-ds3232.ko 2024-04-12T20:30:26.847 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-em3027.ko 2024-04-12T20:30:26.847 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-fm3130.ko 2024-04-12T20:30:26.847 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-isl12022.ko 2024-04-12T20:30:26.847 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-isl1208.ko 2024-04-12T20:30:26.847 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-m41t80.ko 2024-04-12T20:30:26.847 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-m48t35.ko 2024-04-12T20:30:26.847 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-m48t59.ko 2024-04-12T20:30:26.847 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-max6900.ko 2024-04-12T20:30:26.847 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-msm6242.ko 2024-04-12T20:30:26.847 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-pcf2127.ko 2024-04-12T20:30:26.847 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-pcf85063.ko 2024-04-12T20:30:26.847 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-pcf8523.ko 2024-04-12T20:30:26.847 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-pcf8563.ko 2024-04-12T20:30:26.847 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-pcf8583.ko 2024-04-12T20:30:26.848 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-rp5c01.ko 2024-04-12T20:30:26.848 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-rs5c372.ko 2024-04-12T20:30:26.848 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-rv3029c2.ko 2024-04-12T20:30:26.848 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-rx8025.ko 2024-04-12T20:30:26.848 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-rx8581.ko 2024-04-12T20:30:26.848 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-stk17ta8.ko 2024-04-12T20:30:26.848 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-x1205.ko 2024-04-12T20:30:26.848 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi 2024-04-12T20:30:26.848 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/3w-9xxx.ko 2024-04-12T20:30:26.848 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/3w-sas.ko 2024-04-12T20:30:26.848 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/3w-xxxx.ko 2024-04-12T20:30:26.848 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/BusLogic.ko 2024-04-12T20:30:26.848 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/a100u2w.ko 2024-04-12T20:30:26.849 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/aacraid 2024-04-12T20:30:26.849 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/aacraid/aacraid.ko 2024-04-12T20:30:26.849 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/advansys.ko 2024-04-12T20:30:26.849 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/aic7xxx 2024-04-12T20:30:26.849 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/aic7xxx/aic79xx.ko 2024-04-12T20:30:26.849 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/aic7xxx/aic7xxx.ko 2024-04-12T20:30:26.849 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/am53c974.ko 2024-04-12T20:30:26.849 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/arcmsr 2024-04-12T20:30:26.849 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/arcmsr/arcmsr.ko 2024-04-12T20:30:26.849 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/atp870u.ko 2024-04-12T20:30:26.849 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/be2iscsi 2024-04-12T20:30:26.849 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/be2iscsi/be2iscsi.ko 2024-04-12T20:30:26.849 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/bfa 2024-04-12T20:30:26.849 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/bfa/bfa.ko 2024-04-12T20:30:26.849 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/bnx2fc 2024-04-12T20:30:26.850 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/bnx2fc/bnx2fc.ko 2024-04-12T20:30:26.850 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/bnx2i 2024-04-12T20:30:26.850 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/bnx2i/bnx2i.ko 2024-04-12T20:30:26.850 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/ch.ko 2024-04-12T20:30:26.850 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/csiostor 2024-04-12T20:30:26.850 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/csiostor/csiostor.ko 2024-04-12T20:30:26.850 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/cxgbi 2024-04-12T20:30:26.850 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/cxgbi/cxgb3i 2024-04-12T20:30:26.850 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/cxgbi/cxgb3i/cxgb3i.ko 2024-04-12T20:30:26.850 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/cxgbi/cxgb4i 2024-04-12T20:30:26.850 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/cxgbi/cxgb4i/cxgb4i.ko 2024-04-12T20:30:26.850 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/cxgbi/libcxgbi.ko 2024-04-12T20:30:26.850 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/dc395x.ko 2024-04-12T20:30:26.851 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/device_handler 2024-04-12T20:30:26.851 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/device_handler/scsi_dh_alua.ko 2024-04-12T20:30:26.851 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/device_handler/scsi_dh_emc.ko 2024-04-12T20:30:26.851 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/device_handler/scsi_dh_hp_sw.ko 2024-04-12T20:30:26.851 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/device_handler/scsi_dh_rdac.ko 2024-04-12T20:30:26.851 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/dmx3191d.ko 2024-04-12T20:30:26.851 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/esas2r 2024-04-12T20:30:26.851 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/esas2r/esas2r.ko 2024-04-12T20:30:26.851 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/esp_scsi.ko 2024-04-12T20:30:26.851 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/fcoe 2024-04-12T20:30:26.851 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/fcoe/fcoe.ko 2024-04-12T20:30:26.851 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/fcoe/libfcoe.ko 2024-04-12T20:30:26.851 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/fnic 2024-04-12T20:30:26.851 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/fnic/fnic.ko 2024-04-12T20:30:26.852 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/hpsa.ko 2024-04-12T20:30:26.852 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/hptiop.ko 2024-04-12T20:30:26.852 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/hv_storvsc.ko 2024-04-12T20:30:26.852 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/initio.ko 2024-04-12T20:30:26.852 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/ipr.ko 2024-04-12T20:30:26.852 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/ips.ko 2024-04-12T20:30:26.852 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/isci 2024-04-12T20:30:26.852 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/isci/isci.ko 2024-04-12T20:30:26.852 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/iscsi_boot_sysfs.ko 2024-04-12T20:30:26.852 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/iscsi_tcp.ko 2024-04-12T20:30:26.852 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/libfc 2024-04-12T20:30:26.852 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/libfc/libfc.ko 2024-04-12T20:30:26.852 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/libiscsi.ko 2024-04-12T20:30:26.852 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/libiscsi_tcp.ko 2024-04-12T20:30:26.852 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/libsas 2024-04-12T20:30:26.853 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/libsas/libsas.ko 2024-04-12T20:30:26.853 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/lpfc 2024-04-12T20:30:26.853 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/lpfc/lpfc.ko 2024-04-12T20:30:26.853 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/megaraid 2024-04-12T20:30:26.853 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/megaraid.ko 2024-04-12T20:30:26.853 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/megaraid/megaraid_mbox.ko 2024-04-12T20:30:26.853 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/megaraid/megaraid_mm.ko 2024-04-12T20:30:26.853 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/megaraid/megaraid_sas.ko 2024-04-12T20:30:26.853 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/mpt3sas 2024-04-12T20:30:26.853 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/mpt3sas/mpt3sas.ko 2024-04-12T20:30:26.853 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/mvsas 2024-04-12T20:30:26.853 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/mvsas/mvsas.ko 2024-04-12T20:30:26.853 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/mvumi.ko 2024-04-12T20:30:26.853 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/pm8001 2024-04-12T20:30:26.853 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/pm8001/pm80xx.ko 2024-04-12T20:30:26.854 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/pmcraid.ko 2024-04-12T20:30:26.854 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/qla1280.ko 2024-04-12T20:30:26.854 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/qla2xxx 2024-04-12T20:30:26.854 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/qla2xxx/qla2xxx.ko 2024-04-12T20:30:26.854 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/qla2xxx/tcm_qla2xxx.ko 2024-04-12T20:30:26.854 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/qla4xxx 2024-04-12T20:30:26.854 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/qla4xxx/qla4xxx.ko 2024-04-12T20:30:26.854 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/raid_class.ko 2024-04-12T20:30:26.854 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/scsi_debug.ko 2024-04-12T20:30:26.854 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/scsi_transport_fc.ko 2024-04-12T20:30:26.854 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/scsi_transport_iscsi.ko 2024-04-12T20:30:26.855 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/scsi_transport_sas.ko 2024-04-12T20:30:26.855 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/scsi_transport_spi.ko 2024-04-12T20:30:26.855 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/scsi_transport_srp.ko 2024-04-12T20:30:26.855 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/ses.ko 2024-04-12T20:30:26.855 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/st.ko 2024-04-12T20:30:26.855 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/stex.ko 2024-04-12T20:30:26.855 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/sym53c8xx_2 2024-04-12T20:30:26.855 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/sym53c8xx_2/sym53c8xx.ko 2024-04-12T20:30:26.855 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/virtio_scsi.ko 2024-04-12T20:30:26.855 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/vmw_pvscsi.ko 2024-04-12T20:30:26.855 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/wd719x.ko 2024-04-12T20:30:26.855 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ssb 2024-04-12T20:30:26.856 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ssb/ssb.ko 2024-04-12T20:30:26.856 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/target 2024-04-12T20:30:26.856 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/target/iscsi 2024-04-12T20:30:26.856 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/target/iscsi/iscsi_target_mod.ko 2024-04-12T20:30:26.856 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/target/loopback 2024-04-12T20:30:26.856 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/target/loopback/tcm_loop.ko 2024-04-12T20:30:26.856 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/target/target_core_file.ko 2024-04-12T20:30:26.856 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/target/target_core_iblock.ko 2024-04-12T20:30:26.856 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/target/target_core_mod.ko 2024-04-12T20:30:26.856 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/target/target_core_pscsi.ko 2024-04-12T20:30:26.856 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/target/target_core_user.ko 2024-04-12T20:30:26.856 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/target/tcm_fc 2024-04-12T20:30:26.856 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/target/tcm_fc/tcm_fc.ko 2024-04-12T20:30:26.857 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/thermal 2024-04-12T20:30:26.857 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/thermal/intel 2024-04-12T20:30:26.857 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/thermal/intel/int340x_thermal 2024-04-12T20:30:26.857 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/thermal/intel/int340x_thermal/acpi_thermal_rel.ko 2024-04-12T20:30:26.857 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/thermal/intel/int340x_thermal/int3400_thermal.ko 2024-04-12T20:30:26.857 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/thermal/intel/int340x_thermal/int3401_thermal.ko 2024-04-12T20:30:26.857 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/thermal/intel/int340x_thermal/int3402_thermal.ko 2024-04-12T20:30:26.857 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/thermal/intel/int340x_thermal/int3403_thermal.ko 2024-04-12T20:30:26.857 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/thermal/intel/int340x_thermal/int340x_thermal_zone.ko 2024-04-12T20:30:26.857 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device.ko 2024-04-12T20:30:26.857 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci.ko 2024-04-12T20:30:26.857 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci_legacy.ko 2024-04-12T20:30:26.857 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_mbox.ko 2024-04-12T20:30:26.857 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_power_floor.ko 2024-04-12T20:30:26.858 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rapl.ko 2024-04-12T20:30:26.858 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rfim.ko 2024-04-12T20:30:26.858 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_hint.ko 2024-04-12T20:30:26.858 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_req.ko 2024-04-12T20:30:26.858 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/thermal/intel/intel_soc_dts_iosf.ko 2024-04-12T20:30:26.858 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/thermal/intel/intel_soc_dts_thermal.ko 2024-04-12T20:30:26.858 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/thermal/intel/x86_pkg_temp_thermal.ko 2024-04-12T20:30:26.858 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/tty 2024-04-12T20:30:26.858 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/tty/ipwireless 2024-04-12T20:30:26.858 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/tty/ipwireless/ipwireless.ko 2024-04-12T20:30:26.858 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/tty/n_gsm.ko 2024-04-12T20:30:26.858 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/tty/n_hdlc.ko 2024-04-12T20:30:26.859 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/tty/nozomi.ko 2024-04-12T20:30:26.859 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/tty/serial 2024-04-12T20:30:26.859 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/tty/serial/8250 2024-04-12T20:30:26.859 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/tty/serial/8250/serial_cs.ko 2024-04-12T20:30:26.859 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/tty/serial/arc_uart.ko 2024-04-12T20:30:26.859 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/tty/serial/jsm 2024-04-12T20:30:26.859 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/tty/serial/jsm/jsm.ko 2024-04-12T20:30:26.859 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ufs 2024-04-12T20:30:26.859 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ufs/core 2024-04-12T20:30:26.859 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ufs/core/ufshcd-core.ko 2024-04-12T20:30:26.859 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ufs/host 2024-04-12T20:30:26.859 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ufs/host/ufshcd-pci.ko 2024-04-12T20:30:26.859 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/uio 2024-04-12T20:30:26.859 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/uio/uio.ko 2024-04-12T20:30:26.859 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/uio/uio_aec.ko 2024-04-12T20:30:26.860 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/uio/uio_cif.ko 2024-04-12T20:30:26.860 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/uio/uio_pci_generic.ko 2024-04-12T20:30:26.860 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/uio/uio_sercos3.ko 2024-04-12T20:30:26.860 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb 2024-04-12T20:30:26.860 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/class 2024-04-12T20:30:26.860 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/class/cdc-acm.ko 2024-04-12T20:30:26.860 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/class/cdc-wdm.ko 2024-04-12T20:30:26.860 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/class/usblp.ko 2024-04-12T20:30:26.860 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/class/usbtmc.ko 2024-04-12T20:30:26.860 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/host 2024-04-12T20:30:26.860 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/host/sl811-hcd.ko 2024-04-12T20:30:26.860 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/image 2024-04-12T20:30:26.860 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/image/mdc800.ko 2024-04-12T20:30:26.860 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/image/microtek.ko 2024-04-12T20:30:26.860 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/misc 2024-04-12T20:30:26.861 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/misc/adutux.ko 2024-04-12T20:30:26.861 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/misc/appledisplay.ko 2024-04-12T20:30:26.861 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/misc/emi26.ko 2024-04-12T20:30:26.861 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/misc/emi62.ko 2024-04-12T20:30:26.861 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/misc/ezusb.ko 2024-04-12T20:30:26.861 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/misc/idmouse.ko 2024-04-12T20:30:26.861 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/misc/iowarrior.ko 2024-04-12T20:30:26.861 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/misc/isight_firmware.ko 2024-04-12T20:30:26.861 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/misc/ldusb.ko 2024-04-12T20:30:26.861 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/misc/legousbtower.ko 2024-04-12T20:30:26.861 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/misc/sisusbvga 2024-04-12T20:30:26.861 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/misc/sisusbvga/sisusbvga.ko 2024-04-12T20:30:26.861 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/misc/trancevibrator.ko 2024-04-12T20:30:26.861 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/misc/usb3503.ko 2024-04-12T20:30:26.861 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/misc/usblcd.ko 2024-04-12T20:30:26.862 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/misc/usbsevseg.ko 2024-04-12T20:30:26.862 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/misc/uss720.ko 2024-04-12T20:30:26.862 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/misc/yurex.ko 2024-04-12T20:30:26.862 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/phy 2024-04-12T20:30:26.862 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/phy/phy-generic.ko 2024-04-12T20:30:26.862 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial 2024-04-12T20:30:26.862 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/aircable.ko 2024-04-12T20:30:26.862 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/ark3116.ko 2024-04-12T20:30:26.862 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/belkin_sa.ko 2024-04-12T20:30:26.862 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/ch341.ko 2024-04-12T20:30:26.862 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/cp210x.ko 2024-04-12T20:30:26.862 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/cyberjack.ko 2024-04-12T20:30:26.862 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/cypress_m8.ko 2024-04-12T20:30:26.862 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/digi_acceleport.ko 2024-04-12T20:30:26.862 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/empeg.ko 2024-04-12T20:30:26.863 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/ftdi_sio.ko 2024-04-12T20:30:26.863 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/garmin_gps.ko 2024-04-12T20:30:26.863 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/io_edgeport.ko 2024-04-12T20:30:26.863 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/io_ti.ko 2024-04-12T20:30:26.863 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/ipaq.ko 2024-04-12T20:30:26.863 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/ipw.ko 2024-04-12T20:30:26.863 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/ir-usb.ko 2024-04-12T20:30:26.863 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/iuu_phoenix.ko 2024-04-12T20:30:26.863 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/keyspan.ko 2024-04-12T20:30:26.863 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/keyspan_pda.ko 2024-04-12T20:30:26.863 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/kl5kusb105.ko 2024-04-12T20:30:26.863 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/kobil_sct.ko 2024-04-12T20:30:26.863 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/mct_u232.ko 2024-04-12T20:30:26.863 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/mos7720.ko 2024-04-12T20:30:26.863 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/mos7840.ko 2024-04-12T20:30:26.864 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/navman.ko 2024-04-12T20:30:26.864 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/omninet.ko 2024-04-12T20:30:26.864 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/opticon.ko 2024-04-12T20:30:26.864 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/option.ko 2024-04-12T20:30:26.864 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/oti6858.ko 2024-04-12T20:30:26.864 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/pl2303.ko 2024-04-12T20:30:26.864 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/qcaux.ko 2024-04-12T20:30:26.864 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/qcserial.ko 2024-04-12T20:30:26.864 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/quatech2.ko 2024-04-12T20:30:26.864 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/safe_serial.ko 2024-04-12T20:30:26.864 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/sierra.ko 2024-04-12T20:30:26.864 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/spcp8x5.ko 2024-04-12T20:30:26.864 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/ssu100.ko 2024-04-12T20:30:26.864 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/symbolserial.ko 2024-04-12T20:30:26.864 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/ti_usb_3410_5052.ko 2024-04-12T20:30:26.864 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/usb-serial-simple.ko 2024-04-12T20:30:26.865 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/usb_debug.ko 2024-04-12T20:30:26.865 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/usb_wwan.ko 2024-04-12T20:30:26.865 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/visor.ko 2024-04-12T20:30:26.865 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/whiteheat.ko 2024-04-12T20:30:26.865 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/xsens_mt.ko 2024-04-12T20:30:26.865 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/storage 2024-04-12T20:30:26.865 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/storage/uas.ko 2024-04-12T20:30:26.865 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/storage/ums-alauda.ko 2024-04-12T20:30:26.865 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/storage/ums-cypress.ko 2024-04-12T20:30:26.865 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/storage/ums-datafab.ko 2024-04-12T20:30:26.865 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/storage/ums-eneub6250.ko 2024-04-12T20:30:26.865 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/storage/ums-freecom.ko 2024-04-12T20:30:26.865 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/storage/ums-isd200.ko 2024-04-12T20:30:26.865 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/storage/ums-jumpshot.ko 2024-04-12T20:30:26.865 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/storage/ums-karma.ko 2024-04-12T20:30:26.866 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/storage/ums-onetouch.ko 2024-04-12T20:30:26.866 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/storage/ums-realtek.ko 2024-04-12T20:30:26.866 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/storage/ums-sddr09.ko 2024-04-12T20:30:26.866 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/storage/ums-sddr55.ko 2024-04-12T20:30:26.866 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/storage/ums-usbat.ko 2024-04-12T20:30:26.866 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/storage/usb-storage.ko 2024-04-12T20:30:26.866 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/usbip 2024-04-12T20:30:26.866 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/usbip/usbip-core.ko 2024-04-12T20:30:26.866 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/usbip/usbip-host.ko 2024-04-12T20:30:26.866 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/usbip/vhci-hcd.ko 2024-04-12T20:30:26.866 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/vfio 2024-04-12T20:30:26.866 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/vfio/pci 2024-04-12T20:30:26.866 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/vfio/pci/vfio-pci-core.ko 2024-04-12T20:30:26.867 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/vfio/pci/vfio-pci.ko 2024-04-12T20:30:26.867 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/vfio/vfio.ko 2024-04-12T20:30:26.867 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/vfio/vfio_iommu_type1.ko 2024-04-12T20:30:26.867 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/vhost 2024-04-12T20:30:26.867 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/vhost/vhost.ko 2024-04-12T20:30:26.867 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/vhost/vhost_iotlb.ko 2024-04-12T20:30:26.867 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/vhost/vhost_net.ko 2024-04-12T20:30:26.867 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/vhost/vhost_scsi.ko 2024-04-12T20:30:26.867 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/video 2024-04-12T20:30:26.867 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/video/backlight 2024-04-12T20:30:26.867 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/video/backlight/apple_bl.ko 2024-04-12T20:30:26.867 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/video/backlight/lcd.ko 2024-04-12T20:30:26.867 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/video/backlight/platform_lcd.ko 2024-04-12T20:30:26.867 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/virtio 2024-04-12T20:30:26.868 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/virtio/virtio.ko 2024-04-12T20:30:26.868 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/virtio/virtio_balloon.ko 2024-04-12T20:30:26.868 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/virtio/virtio_mmio.ko 2024-04-12T20:30:26.868 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/virtio/virtio_pci.ko 2024-04-12T20:30:26.868 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/virtio/virtio_pci_legacy_dev.ko 2024-04-12T20:30:26.868 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/virtio/virtio_pci_modern_dev.ko 2024-04-12T20:30:26.868 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/virtio/virtio_ring.ko 2024-04-12T20:30:26.868 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/w1 2024-04-12T20:30:26.868 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/w1/masters 2024-04-12T20:30:26.868 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/w1/masters/ds2482.ko 2024-04-12T20:30:26.868 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/w1/masters/ds2490.ko 2024-04-12T20:30:26.868 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/w1/slaves 2024-04-12T20:30:26.868 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/w1/slaves/w1_ds2406.ko 2024-04-12T20:30:26.869 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/w1/slaves/w1_ds2408.ko 2024-04-12T20:30:26.869 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/w1/slaves/w1_ds2413.ko 2024-04-12T20:30:26.869 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/w1/slaves/w1_ds2423.ko 2024-04-12T20:30:26.869 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/w1/slaves/w1_ds2431.ko 2024-04-12T20:30:26.869 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/w1/slaves/w1_ds2433.ko 2024-04-12T20:30:26.869 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/w1/slaves/w1_ds2780.ko 2024-04-12T20:30:26.869 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/w1/slaves/w1_ds2781.ko 2024-04-12T20:30:26.869 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/w1/slaves/w1_ds28e04.ko 2024-04-12T20:30:26.869 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/w1/slaves/w1_smem.ko 2024-04-12T20:30:26.869 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/w1/slaves/w1_therm.ko 2024-04-12T20:30:26.869 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/w1/wire.ko 2024-04-12T20:30:26.869 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/watchdog 2024-04-12T20:30:26.869 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/watchdog/alim1535_wdt.ko 2024-04-12T20:30:26.870 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/watchdog/alim7101_wdt.ko 2024-04-12T20:30:26.870 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/watchdog/f71808e_wdt.ko 2024-04-12T20:30:26.870 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/watchdog/hpwdt.ko 2024-04-12T20:30:26.870 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/watchdog/i6300esb.ko 2024-04-12T20:30:26.870 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/watchdog/iTCO_vendor_support.ko 2024-04-12T20:30:26.870 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/watchdog/iTCO_wdt.ko 2024-04-12T20:30:26.870 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/watchdog/ib700wdt.ko 2024-04-12T20:30:26.870 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/watchdog/ibmasr.ko 2024-04-12T20:30:26.870 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/watchdog/ie6xx_wdt.ko 2024-04-12T20:30:26.870 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/watchdog/it8712f_wdt.ko 2024-04-12T20:30:26.870 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/watchdog/it87_wdt.ko 2024-04-12T20:30:26.870 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/watchdog/machzwd.ko 2024-04-12T20:30:26.871 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/watchdog/nv_tco.ko 2024-04-12T20:30:26.871 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/watchdog/pcwd_pci.ko 2024-04-12T20:30:26.871 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/watchdog/pcwd_usb.ko 2024-04-12T20:30:26.871 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/watchdog/sbc_fitpc2_wdt.ko 2024-04-12T20:30:26.871 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/watchdog/sch311x_wdt.ko 2024-04-12T20:30:26.871 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/watchdog/softdog.ko 2024-04-12T20:30:26.871 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/watchdog/sp5100_tco.ko 2024-04-12T20:30:26.871 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/watchdog/via_wdt.ko 2024-04-12T20:30:26.871 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/watchdog/w83627hf_wdt.ko 2024-04-12T20:30:26.871 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/watchdog/w83877f_wdt.ko 2024-04-12T20:30:26.871 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/watchdog/w83977f_wdt.ko 2024-04-12T20:30:26.871 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/watchdog/wdt_pci.ko 2024-04-12T20:30:26.871 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/watchdog/xen_wdt.ko 2024-04-12T20:30:26.871 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/xen 2024-04-12T20:30:26.872 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/xen/xen-acpi-processor.ko 2024-04-12T20:30:26.872 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/xen/xen-evtchn.ko 2024-04-12T20:30:26.872 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/xen/xen-gntalloc.ko 2024-04-12T20:30:26.872 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/xen/xen-gntdev.ko 2024-04-12T20:30:26.872 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/xen/xen-pciback 2024-04-12T20:30:26.872 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/xen/xen-pciback/xen-pciback.ko 2024-04-12T20:30:26.872 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/xen/xen-privcmd.ko 2024-04-12T20:30:26.872 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/xen/xenfs 2024-04-12T20:30:26.872 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/xen/xenfs/xenfs.ko 2024-04-12T20:30:26.872 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs 2024-04-12T20:30:26.872 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/9p 2024-04-12T20:30:26.872 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/9p/9p.ko 2024-04-12T20:30:26.872 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/affs 2024-04-12T20:30:26.872 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/affs/affs.ko 2024-04-12T20:30:26.873 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/befs 2024-04-12T20:30:26.873 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/befs/befs.ko 2024-04-12T20:30:26.873 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/binfmt_misc.ko 2024-04-12T20:30:26.873 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/btrfs 2024-04-12T20:30:26.873 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/btrfs/btrfs.ko 2024-04-12T20:30:26.873 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/cachefiles 2024-04-12T20:30:26.873 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/cachefiles/cachefiles.ko 2024-04-12T20:30:26.873 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/ceph 2024-04-12T20:30:26.873 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/ceph/ceph.ko 2024-04-12T20:30:26.873 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/coda 2024-04-12T20:30:26.873 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/coda/coda.ko 2024-04-12T20:30:26.873 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/cramfs 2024-04-12T20:30:26.873 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/cramfs/cramfs.ko 2024-04-12T20:30:26.873 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/dlm 2024-04-12T20:30:26.874 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/dlm/dlm.ko 2024-04-12T20:30:26.874 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/ecryptfs 2024-04-12T20:30:26.874 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/ecryptfs/ecryptfs.ko 2024-04-12T20:30:26.874 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/f2fs 2024-04-12T20:30:26.874 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/f2fs/f2fs.ko 2024-04-12T20:30:26.874 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/fat 2024-04-12T20:30:26.874 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/fat/fat.ko 2024-04-12T20:30:26.874 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/fat/msdos.ko 2024-04-12T20:30:26.874 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/fat/vfat.ko 2024-04-12T20:30:26.874 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/fuse 2024-04-12T20:30:26.874 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/fuse/cuse.ko 2024-04-12T20:30:26.874 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/fuse/fuse.ko 2024-04-12T20:30:26.874 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/gfs2 2024-04-12T20:30:26.875 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/gfs2/gfs2.ko 2024-04-12T20:30:26.875 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/hfs 2024-04-12T20:30:26.875 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/hfs/hfs.ko 2024-04-12T20:30:26.875 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/hfsplus 2024-04-12T20:30:26.875 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/hfsplus/hfsplus.ko 2024-04-12T20:30:26.875 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/isofs 2024-04-12T20:30:26.875 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/isofs/isofs.ko 2024-04-12T20:30:26.875 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/jfs 2024-04-12T20:30:26.875 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/jfs/jfs.ko 2024-04-12T20:30:26.875 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/lockd 2024-04-12T20:30:26.875 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/lockd/lockd.ko 2024-04-12T20:30:26.875 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/minix 2024-04-12T20:30:26.875 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/minix/minix.ko 2024-04-12T20:30:26.875 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/netfs 2024-04-12T20:30:26.876 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/netfs/netfs.ko 2024-04-12T20:30:26.876 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nfs 2024-04-12T20:30:26.876 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nfs/blocklayout 2024-04-12T20:30:26.876 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nfs/blocklayout/blocklayoutdriver.ko 2024-04-12T20:30:26.876 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nfs/filelayout 2024-04-12T20:30:26.876 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nfs/filelayout/nfs_layout_nfsv41_files.ko 2024-04-12T20:30:26.876 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nfs/flexfilelayout 2024-04-12T20:30:26.876 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nfs/flexfilelayout/nfs_layout_flexfiles.ko 2024-04-12T20:30:26.876 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nfs/nfs.ko 2024-04-12T20:30:26.876 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nfs/nfsv3.ko 2024-04-12T20:30:26.876 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nfs/nfsv4.ko 2024-04-12T20:30:26.876 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nfs_common 2024-04-12T20:30:26.876 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nfs_common/grace.ko 2024-04-12T20:30:26.877 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nfs_common/nfs_acl.ko 2024-04-12T20:30:26.877 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nfsd 2024-04-12T20:30:26.877 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nfsd/nfsd.ko 2024-04-12T20:30:26.877 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nilfs2 2024-04-12T20:30:26.877 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nilfs2/nilfs2.ko 2024-04-12T20:30:26.877 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls 2024-04-12T20:30:26.877 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/mac-celtic.ko 2024-04-12T20:30:26.877 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/mac-centeuro.ko 2024-04-12T20:30:26.877 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/mac-croatian.ko 2024-04-12T20:30:26.877 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/mac-cyrillic.ko 2024-04-12T20:30:26.877 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/mac-gaelic.ko 2024-04-12T20:30:26.877 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/mac-greek.ko 2024-04-12T20:30:26.877 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/mac-iceland.ko 2024-04-12T20:30:26.877 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/mac-inuit.ko 2024-04-12T20:30:26.877 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/mac-roman.ko 2024-04-12T20:30:26.878 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/mac-romanian.ko 2024-04-12T20:30:26.878 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/mac-turkish.ko 2024-04-12T20:30:26.878 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_cp1250.ko 2024-04-12T20:30:26.878 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_cp1251.ko 2024-04-12T20:30:26.878 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_cp1255.ko 2024-04-12T20:30:26.878 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_cp737.ko 2024-04-12T20:30:26.878 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_cp775.ko 2024-04-12T20:30:26.878 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_cp850.ko 2024-04-12T20:30:26.878 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_cp852.ko 2024-04-12T20:30:26.878 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_cp855.ko 2024-04-12T20:30:26.878 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_cp857.ko 2024-04-12T20:30:26.878 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_cp860.ko 2024-04-12T20:30:26.878 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_cp861.ko 2024-04-12T20:30:26.878 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_cp862.ko 2024-04-12T20:30:26.879 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_cp863.ko 2024-04-12T20:30:26.879 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_cp864.ko 2024-04-12T20:30:26.879 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_cp865.ko 2024-04-12T20:30:26.879 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_cp866.ko 2024-04-12T20:30:26.879 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_cp869.ko 2024-04-12T20:30:26.879 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_cp874.ko 2024-04-12T20:30:26.879 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_cp932.ko 2024-04-12T20:30:26.879 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_cp936.ko 2024-04-12T20:30:26.879 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_cp949.ko 2024-04-12T20:30:26.879 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_cp950.ko 2024-04-12T20:30:26.879 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_euc-jp.ko 2024-04-12T20:30:26.879 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_iso8859-1.ko 2024-04-12T20:30:26.879 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_iso8859-13.ko 2024-04-12T20:30:26.879 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_iso8859-14.ko 2024-04-12T20:30:26.879 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_iso8859-15.ko 2024-04-12T20:30:26.880 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_iso8859-2.ko 2024-04-12T20:30:26.880 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_iso8859-3.ko 2024-04-12T20:30:26.880 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_iso8859-4.ko 2024-04-12T20:30:26.880 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_iso8859-5.ko 2024-04-12T20:30:26.880 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_iso8859-6.ko 2024-04-12T20:30:26.880 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_iso8859-7.ko 2024-04-12T20:30:26.880 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_iso8859-9.ko 2024-04-12T20:30:26.880 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_koi8-r.ko 2024-04-12T20:30:26.880 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_koi8-ru.ko 2024-04-12T20:30:26.880 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_koi8-u.ko 2024-04-12T20:30:26.880 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_ucs2_utils.ko 2024-04-12T20:30:26.880 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_utf8.ko 2024-04-12T20:30:26.880 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/ocfs2 2024-04-12T20:30:26.880 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/ocfs2/cluster 2024-04-12T20:30:26.880 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/ocfs2/cluster/ocfs2_nodemanager.ko 2024-04-12T20:30:26.881 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/ocfs2/dlm 2024-04-12T20:30:26.881 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/ocfs2/dlm/ocfs2_dlm.ko 2024-04-12T20:30:26.881 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/ocfs2/dlmfs 2024-04-12T20:30:26.881 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/ocfs2/dlmfs/ocfs2_dlmfs.ko 2024-04-12T20:30:26.881 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/ocfs2/ocfs2.ko 2024-04-12T20:30:26.881 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/ocfs2/ocfs2_stack_o2cb.ko 2024-04-12T20:30:26.881 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/ocfs2/ocfs2_stack_user.ko 2024-04-12T20:30:26.881 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/ocfs2/ocfs2_stackglue.ko 2024-04-12T20:30:26.881 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/overlayfs 2024-04-12T20:30:26.881 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/overlayfs/overlay.ko 2024-04-12T20:30:26.881 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/pstore 2024-04-12T20:30:26.881 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/pstore/ramoops.ko 2024-04-12T20:30:26.881 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/reiserfs 2024-04-12T20:30:26.881 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/reiserfs/reiserfs.ko 2024-04-12T20:30:26.882 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/romfs 2024-04-12T20:30:26.882 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/romfs/romfs.ko 2024-04-12T20:30:26.882 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/smb 2024-04-12T20:30:26.882 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/smb/client 2024-04-12T20:30:26.882 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/smb/client/cifs.ko 2024-04-12T20:30:26.882 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/smb/common 2024-04-12T20:30:26.882 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/smb/common/cifs_arc4.ko 2024-04-12T20:30:26.882 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/smb/common/cifs_md4.ko 2024-04-12T20:30:26.882 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/squashfs 2024-04-12T20:30:26.882 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/squashfs/squashfs.ko 2024-04-12T20:30:26.882 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/sysv 2024-04-12T20:30:26.882 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/sysv/sysv.ko 2024-04-12T20:30:26.882 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/udf 2024-04-12T20:30:26.882 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/udf/udf.ko 2024-04-12T20:30:26.883 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/ufs 2024-04-12T20:30:26.883 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/ufs/ufs.ko 2024-04-12T20:30:26.883 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/xfs 2024-04-12T20:30:26.883 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/xfs/xfs.ko 2024-04-12T20:30:26.883 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/kernel 2024-04-12T20:30:26.883 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/kernel/rcu 2024-04-12T20:30:26.883 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/kernel/rcu/rcutorture.ko 2024-04-12T20:30:26.883 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/kernel/torture.ko 2024-04-12T20:30:26.883 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/kernel/trace 2024-04-12T20:30:26.883 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/kernel/trace/ring_buffer_benchmark.ko 2024-04-12T20:30:26.883 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/lib 2024-04-12T20:30:26.883 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/lib/asn1_encoder.ko 2024-04-12T20:30:26.883 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/lib/crc-itu-t.ko 2024-04-12T20:30:26.883 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/lib/crc8.ko 2024-04-12T20:30:26.883 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/lib/crypto 2024-04-12T20:30:26.884 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/lib/crypto/libarc4.ko 2024-04-12T20:30:26.884 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/lib/crypto/libdes.ko 2024-04-12T20:30:26.884 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/lib/libcrc32c.ko 2024-04-12T20:30:26.884 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/lib/lru_cache.ko 2024-04-12T20:30:26.884 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/lib/lz4 2024-04-12T20:30:26.884 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/lib/lz4/lz4_compress.ko 2024-04-12T20:30:26.884 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/lib/lz4/lz4hc_compress.ko 2024-04-12T20:30:26.884 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/lib/math 2024-04-12T20:30:26.884 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/lib/math/cordic.ko 2024-04-12T20:30:26.884 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/lib/raid6 2024-04-12T20:30:26.884 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/lib/raid6/raid6_pq.ko 2024-04-12T20:30:26.884 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/lib/reed_solomon 2024-04-12T20:30:26.884 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/lib/reed_solomon/reed_solomon.ko 2024-04-12T20:30:26.884 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/lib/ts_bm.ko 2024-04-12T20:30:26.884 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/lib/ts_fsm.ko 2024-04-12T20:30:26.885 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/lib/ts_kmp.ko 2024-04-12T20:30:26.885 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/lib/zstd 2024-04-12T20:30:26.885 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/lib/zstd/zstd_compress.ko 2024-04-12T20:30:26.885 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/mm 2024-04-12T20:30:26.885 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/mm/hwpoison-inject.ko 2024-04-12T20:30:26.885 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net 2024-04-12T20:30:26.885 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/6lowpan 2024-04-12T20:30:26.885 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/6lowpan/6lowpan.ko 2024-04-12T20:30:26.885 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/6lowpan/nhc_dest.ko 2024-04-12T20:30:26.885 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/6lowpan/nhc_fragment.ko 2024-04-12T20:30:26.885 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/6lowpan/nhc_hop.ko 2024-04-12T20:30:26.885 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/6lowpan/nhc_ipv6.ko 2024-04-12T20:30:26.885 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/6lowpan/nhc_mobility.ko 2024-04-12T20:30:26.885 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/6lowpan/nhc_routing.ko 2024-04-12T20:30:26.885 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/6lowpan/nhc_udp.ko 2024-04-12T20:30:26.886 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/802 2024-04-12T20:30:26.886 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/802/garp.ko 2024-04-12T20:30:26.886 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/802/mrp.ko 2024-04-12T20:30:26.886 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/802/p8022.ko 2024-04-12T20:30:26.886 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/802/psnap.ko 2024-04-12T20:30:26.886 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/802/stp.ko 2024-04-12T20:30:26.886 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/8021q 2024-04-12T20:30:26.886 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/8021q/8021q.ko 2024-04-12T20:30:26.886 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/9p 2024-04-12T20:30:26.886 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/9p/9pnet.ko 2024-04-12T20:30:26.886 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/9p/9pnet_fd.ko 2024-04-12T20:30:26.886 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/9p/9pnet_rdma.ko 2024-04-12T20:30:26.886 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/9p/9pnet_virtio.ko 2024-04-12T20:30:26.886 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/batman-adv 2024-04-12T20:30:26.887 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/batman-adv/batman-adv.ko 2024-04-12T20:30:26.887 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/bridge 2024-04-12T20:30:26.887 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/bridge/br_netfilter.ko 2024-04-12T20:30:26.887 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/bridge/bridge.ko 2024-04-12T20:30:26.887 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/bridge/netfilter 2024-04-12T20:30:26.887 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/bridge/netfilter/ebt_802_3.ko 2024-04-12T20:30:26.887 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/bridge/netfilter/ebt_among.ko 2024-04-12T20:30:26.887 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/bridge/netfilter/ebt_arp.ko 2024-04-12T20:30:26.887 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/bridge/netfilter/ebt_arpreply.ko 2024-04-12T20:30:26.887 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/bridge/netfilter/ebt_dnat.ko 2024-04-12T20:30:26.887 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/bridge/netfilter/ebt_ip.ko 2024-04-12T20:30:26.887 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/bridge/netfilter/ebt_ip6.ko 2024-04-12T20:30:26.887 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/bridge/netfilter/ebt_limit.ko 2024-04-12T20:30:26.887 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/bridge/netfilter/ebt_log.ko 2024-04-12T20:30:26.887 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/bridge/netfilter/ebt_mark.ko 2024-04-12T20:30:26.888 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/bridge/netfilter/ebt_mark_m.ko 2024-04-12T20:30:26.888 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/bridge/netfilter/ebt_nflog.ko 2024-04-12T20:30:26.888 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/bridge/netfilter/ebt_pkttype.ko 2024-04-12T20:30:26.888 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/bridge/netfilter/ebt_redirect.ko 2024-04-12T20:30:26.888 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/bridge/netfilter/ebt_snat.ko 2024-04-12T20:30:26.888 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/bridge/netfilter/ebt_stp.ko 2024-04-12T20:30:26.888 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/bridge/netfilter/ebt_vlan.ko 2024-04-12T20:30:26.888 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/bridge/netfilter/ebtable_broute.ko 2024-04-12T20:30:26.888 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/bridge/netfilter/ebtable_filter.ko 2024-04-12T20:30:26.888 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/bridge/netfilter/ebtable_nat.ko 2024-04-12T20:30:26.888 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/bridge/netfilter/ebtables.ko 2024-04-12T20:30:26.888 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/bridge/netfilter/nf_conntrack_bridge.ko 2024-04-12T20:30:26.888 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/bridge/netfilter/nft_meta_bridge.ko 2024-04-12T20:30:26.888 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/bridge/netfilter/nft_reject_bridge.ko 2024-04-12T20:30:26.889 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ceph 2024-04-12T20:30:26.889 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ceph/libceph.ko 2024-04-12T20:30:26.889 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/core 2024-04-12T20:30:26.889 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/core/failover.ko 2024-04-12T20:30:26.889 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/core/pktgen.ko 2024-04-12T20:30:26.889 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/dccp 2024-04-12T20:30:26.889 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/dccp/dccp.ko 2024-04-12T20:30:26.889 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/dccp/dccp_diag.ko 2024-04-12T20:30:26.889 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/dccp/dccp_ipv4.ko 2024-04-12T20:30:26.889 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/dccp/dccp_ipv6.ko 2024-04-12T20:30:26.889 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/dns_resolver 2024-04-12T20:30:26.889 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/dns_resolver/dns_resolver.ko 2024-04-12T20:30:26.889 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/dsa 2024-04-12T20:30:26.889 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/dsa/dsa_core.ko 2024-04-12T20:30:26.889 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/dsa/tag_brcm.ko 2024-04-12T20:30:26.890 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/dsa/tag_dsa.ko 2024-04-12T20:30:26.890 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/dsa/tag_none.ko 2024-04-12T20:30:26.890 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/dsa/tag_trailer.ko 2024-04-12T20:30:26.890 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ieee802154 2024-04-12T20:30:26.890 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ieee802154/6lowpan 2024-04-12T20:30:26.890 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ieee802154/6lowpan/ieee802154_6lowpan.ko 2024-04-12T20:30:26.890 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ieee802154/ieee802154.ko 2024-04-12T20:30:26.890 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ieee802154/ieee802154_socket.ko 2024-04-12T20:30:26.890 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4 2024-04-12T20:30:26.890 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/ah4.ko 2024-04-12T20:30:26.890 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/esp4.ko 2024-04-12T20:30:26.890 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/fou.ko 2024-04-12T20:30:26.890 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/gre.ko 2024-04-12T20:30:26.890 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/inet_diag.ko 2024-04-12T20:30:26.890 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/ip_gre.ko 2024-04-12T20:30:26.891 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/ip_tunnel.ko 2024-04-12T20:30:26.891 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/ip_vti.ko 2024-04-12T20:30:26.891 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/ipcomp.ko 2024-04-12T20:30:26.891 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/ipip.ko 2024-04-12T20:30:26.891 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/netfilter 2024-04-12T20:30:26.891 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/netfilter/arp_tables.ko 2024-04-12T20:30:26.891 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/netfilter/arpt_mangle.ko 2024-04-12T20:30:26.891 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/netfilter/arptable_filter.ko 2024-04-12T20:30:26.891 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/netfilter/ipt_ECN.ko 2024-04-12T20:30:26.891 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/netfilter/ipt_SYNPROXY.ko 2024-04-12T20:30:26.891 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/netfilter/ipt_ah.ko 2024-04-12T20:30:26.891 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/netfilter/ipt_rpfilter.ko 2024-04-12T20:30:26.891 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/netfilter/iptable_mangle.ko 2024-04-12T20:30:26.891 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/netfilter/iptable_nat.ko 2024-04-12T20:30:26.891 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/netfilter/iptable_raw.ko 2024-04-12T20:30:26.891 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/netfilter/iptable_security.ko 2024-04-12T20:30:26.891 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/netfilter/nf_defrag_ipv4.ko 2024-04-12T20:30:26.892 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/netfilter/nf_dup_ipv4.ko 2024-04-12T20:30:26.892 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/netfilter/nf_nat_h323.ko 2024-04-12T20:30:26.892 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/netfilter/nf_nat_pptp.ko 2024-04-12T20:30:26.892 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/netfilter/nf_nat_snmp_basic.ko 2024-04-12T20:30:26.892 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/netfilter/nf_socket_ipv4.ko 2024-04-12T20:30:26.892 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/netfilter/nf_tproxy_ipv4.ko 2024-04-12T20:30:26.892 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/netfilter/nft_dup_ipv4.ko 2024-04-12T20:30:26.892 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/netfilter/nft_fib_ipv4.ko 2024-04-12T20:30:26.892 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/netfilter/nft_reject_ipv4.ko 2024-04-12T20:30:26.892 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/tcp_bic.ko 2024-04-12T20:30:26.892 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/tcp_dctcp.ko 2024-04-12T20:30:26.892 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/tcp_diag.ko 2024-04-12T20:30:26.892 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/tcp_highspeed.ko 2024-04-12T20:30:26.892 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/tcp_htcp.ko 2024-04-12T20:30:26.893 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/tcp_hybla.ko 2024-04-12T20:30:26.893 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/tcp_illinois.ko 2024-04-12T20:30:26.893 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/tcp_lp.ko 2024-04-12T20:30:26.893 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/tcp_scalable.ko 2024-04-12T20:30:26.893 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/tcp_vegas.ko 2024-04-12T20:30:26.893 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/tcp_veno.ko 2024-04-12T20:30:26.893 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/tcp_westwood.ko 2024-04-12T20:30:26.893 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/tcp_yeah.ko 2024-04-12T20:30:26.893 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/tunnel4.ko 2024-04-12T20:30:26.893 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/udp_diag.ko 2024-04-12T20:30:26.893 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/udp_tunnel.ko 2024-04-12T20:30:26.893 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/xfrm4_tunnel.ko 2024-04-12T20:30:26.893 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6 2024-04-12T20:30:26.893 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/ah6.ko 2024-04-12T20:30:26.893 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/esp6.ko 2024-04-12T20:30:26.894 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/fou6.ko 2024-04-12T20:30:26.894 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/ip6_tunnel.ko 2024-04-12T20:30:26.894 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/ip6_udp_tunnel.ko 2024-04-12T20:30:26.894 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/ip6_vti.ko 2024-04-12T20:30:26.894 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/ipcomp6.ko 2024-04-12T20:30:26.894 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/netfilter 2024-04-12T20:30:26.894 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/netfilter/ip6_tables.ko 2024-04-12T20:30:26.894 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/netfilter/ip6t_REJECT.ko 2024-04-12T20:30:26.894 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/netfilter/ip6t_SYNPROXY.ko 2024-04-12T20:30:26.894 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/netfilter/ip6t_ah.ko 2024-04-12T20:30:26.894 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/netfilter/ip6t_eui64.ko 2024-04-12T20:30:26.894 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/netfilter/ip6t_frag.ko 2024-04-12T20:30:26.894 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/netfilter/ip6t_hbh.ko 2024-04-12T20:30:26.894 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/netfilter/ip6t_ipv6header.ko 2024-04-12T20:30:26.895 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/netfilter/ip6t_mh.ko 2024-04-12T20:30:26.895 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/netfilter/ip6t_rpfilter.ko 2024-04-12T20:30:26.895 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/netfilter/ip6t_rt.ko 2024-04-12T20:30:26.895 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/netfilter/ip6table_filter.ko 2024-04-12T20:30:26.895 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/netfilter/ip6table_mangle.ko 2024-04-12T20:30:26.895 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/netfilter/ip6table_nat.ko 2024-04-12T20:30:26.895 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/netfilter/ip6table_raw.ko 2024-04-12T20:30:26.895 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/netfilter/ip6table_security.ko 2024-04-12T20:30:26.895 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/netfilter/nf_defrag_ipv6.ko 2024-04-12T20:30:26.895 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/netfilter/nf_dup_ipv6.ko 2024-04-12T20:30:26.895 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/netfilter/nf_reject_ipv6.ko 2024-04-12T20:30:26.895 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/netfilter/nf_socket_ipv6.ko 2024-04-12T20:30:26.895 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/netfilter/nf_tproxy_ipv6.ko 2024-04-12T20:30:26.895 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/netfilter/nft_dup_ipv6.ko 2024-04-12T20:30:26.895 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/netfilter/nft_fib_ipv6.ko 2024-04-12T20:30:26.896 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/netfilter/nft_reject_ipv6.ko 2024-04-12T20:30:26.896 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/sit.ko 2024-04-12T20:30:26.896 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/tunnel6.ko 2024-04-12T20:30:26.896 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/xfrm6_tunnel.ko 2024-04-12T20:30:26.896 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/key 2024-04-12T20:30:26.896 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/key/af_key.ko 2024-04-12T20:30:26.896 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/l2tp 2024-04-12T20:30:26.896 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/l2tp/l2tp_core.ko 2024-04-12T20:30:26.896 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/l2tp/l2tp_debugfs.ko 2024-04-12T20:30:26.896 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/l2tp/l2tp_eth.ko 2024-04-12T20:30:26.896 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/l2tp/l2tp_ip.ko 2024-04-12T20:30:26.896 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/l2tp/l2tp_ip6.ko 2024-04-12T20:30:26.896 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/l2tp/l2tp_netlink.ko 2024-04-12T20:30:26.896 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/l2tp/l2tp_ppp.ko 2024-04-12T20:30:26.896 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/llc 2024-04-12T20:30:26.896 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/llc/llc.ko 2024-04-12T20:30:26.897 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/mac802154 2024-04-12T20:30:26.897 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/mac802154/mac802154.ko 2024-04-12T20:30:26.897 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/mpls 2024-04-12T20:30:26.897 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/mpls/mpls_gso.ko 2024-04-12T20:30:26.897 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter 2024-04-12T20:30:26.897 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipset 2024-04-12T20:30:26.897 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipset/ip_set.ko 2024-04-12T20:30:26.897 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipset/ip_set_bitmap_ip.ko 2024-04-12T20:30:26.897 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipset/ip_set_bitmap_ipmac.ko 2024-04-12T20:30:26.897 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipset/ip_set_bitmap_port.ko 2024-04-12T20:30:26.897 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipset/ip_set_hash_ip.ko 2024-04-12T20:30:26.897 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipset/ip_set_hash_ipmark.ko 2024-04-12T20:30:26.897 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipset/ip_set_hash_ipport.ko 2024-04-12T20:30:26.897 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipset/ip_set_hash_ipportip.ko 2024-04-12T20:30:26.898 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipset/ip_set_hash_ipportnet.ko 2024-04-12T20:30:26.898 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipset/ip_set_hash_mac.ko 2024-04-12T20:30:26.898 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipset/ip_set_hash_net.ko 2024-04-12T20:30:26.898 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipset/ip_set_hash_netiface.ko 2024-04-12T20:30:26.898 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipset/ip_set_hash_netnet.ko 2024-04-12T20:30:26.898 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipset/ip_set_hash_netport.ko 2024-04-12T20:30:26.898 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipset/ip_set_hash_netportnet.ko 2024-04-12T20:30:26.898 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipset/ip_set_list_set.ko 2024-04-12T20:30:26.898 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipvs 2024-04-12T20:30:26.898 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipvs/ip_vs.ko 2024-04-12T20:30:26.898 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipvs/ip_vs_dh.ko 2024-04-12T20:30:26.898 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipvs/ip_vs_fo.ko 2024-04-12T20:30:26.898 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipvs/ip_vs_ftp.ko 2024-04-12T20:30:26.899 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipvs/ip_vs_lblc.ko 2024-04-12T20:30:26.899 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipvs/ip_vs_lblcr.ko 2024-04-12T20:30:26.899 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipvs/ip_vs_lc.ko 2024-04-12T20:30:26.899 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipvs/ip_vs_nq.ko 2024-04-12T20:30:26.899 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipvs/ip_vs_pe_sip.ko 2024-04-12T20:30:26.899 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipvs/ip_vs_rr.ko 2024-04-12T20:30:26.899 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipvs/ip_vs_sed.ko 2024-04-12T20:30:26.899 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipvs/ip_vs_sh.ko 2024-04-12T20:30:26.899 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipvs/ip_vs_wlc.ko 2024-04-12T20:30:26.899 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipvs/ip_vs_wrr.ko 2024-04-12T20:30:26.899 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nf_conncount.ko 2024-04-12T20:30:26.899 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nf_conntrack.ko 2024-04-12T20:30:26.899 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nf_conntrack_amanda.ko 2024-04-12T20:30:26.900 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nf_conntrack_broadcast.ko 2024-04-12T20:30:26.900 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nf_conntrack_ftp.ko 2024-04-12T20:30:26.900 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nf_conntrack_h323.ko 2024-04-12T20:30:26.900 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nf_conntrack_irc.ko 2024-04-12T20:30:26.900 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nf_conntrack_netbios_ns.ko 2024-04-12T20:30:26.900 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nf_conntrack_netlink.ko 2024-04-12T20:30:26.900 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nf_conntrack_pptp.ko 2024-04-12T20:30:26.900 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nf_conntrack_sane.ko 2024-04-12T20:30:26.900 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nf_conntrack_sip.ko 2024-04-12T20:30:26.900 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nf_conntrack_snmp.ko 2024-04-12T20:30:26.900 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nf_conntrack_tftp.ko 2024-04-12T20:30:26.900 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nf_dup_netdev.ko 2024-04-12T20:30:26.900 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nf_flow_table.ko 2024-04-12T20:30:26.901 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nf_flow_table_inet.ko 2024-04-12T20:30:26.901 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nf_log_syslog.ko 2024-04-12T20:30:26.901 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nf_nat.ko 2024-04-12T20:30:26.901 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nf_nat_amanda.ko 2024-04-12T20:30:26.901 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nf_nat_ftp.ko 2024-04-12T20:30:26.901 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nf_nat_irc.ko 2024-04-12T20:30:26.901 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nf_nat_sip.ko 2024-04-12T20:30:26.901 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nf_nat_tftp.ko 2024-04-12T20:30:26.901 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nf_synproxy_core.ko 2024-04-12T20:30:26.901 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nf_tables.ko 2024-04-12T20:30:26.901 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nfnetlink.ko 2024-04-12T20:30:26.901 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nfnetlink_acct.ko 2024-04-12T20:30:26.901 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nfnetlink_log.ko 2024-04-12T20:30:26.901 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nfnetlink_osf.ko 2024-04-12T20:30:26.901 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nfnetlink_queue.ko 2024-04-12T20:30:26.902 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nft_chain_nat.ko 2024-04-12T20:30:26.902 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nft_compat.ko 2024-04-12T20:30:26.902 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nft_ct.ko 2024-04-12T20:30:26.902 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nft_dup_netdev.ko 2024-04-12T20:30:26.902 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nft_fib.ko 2024-04-12T20:30:26.902 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nft_fwd_netdev.ko 2024-04-12T20:30:26.902 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nft_hash.ko 2024-04-12T20:30:26.902 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nft_limit.ko 2024-04-12T20:30:26.902 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nft_log.ko 2024-04-12T20:30:26.902 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nft_masq.ko 2024-04-12T20:30:26.902 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nft_nat.ko 2024-04-12T20:30:26.902 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nft_numgen.ko 2024-04-12T20:30:26.902 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nft_queue.ko 2024-04-12T20:30:26.902 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nft_quota.ko 2024-04-12T20:30:26.903 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nft_redir.ko 2024-04-12T20:30:26.903 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nft_reject.ko 2024-04-12T20:30:26.903 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nft_reject_inet.ko 2024-04-12T20:30:26.903 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nft_socket.ko 2024-04-12T20:30:26.903 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nft_tunnel.ko 2024-04-12T20:30:26.903 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_AUDIT.ko 2024-04-12T20:30:26.903 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_CHECKSUM.ko 2024-04-12T20:30:26.903 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_CLASSIFY.ko 2024-04-12T20:30:26.903 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_CONNSECMARK.ko 2024-04-12T20:30:26.903 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_CT.ko 2024-04-12T20:30:26.903 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_DSCP.ko 2024-04-12T20:30:26.903 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_HL.ko 2024-04-12T20:30:26.903 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_HMARK.ko 2024-04-12T20:30:26.903 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_IDLETIMER.ko 2024-04-12T20:30:26.903 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_LED.ko 2024-04-12T20:30:26.903 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_LOG.ko 2024-04-12T20:30:26.904 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_MASQUERADE.ko 2024-04-12T20:30:26.904 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_NETMAP.ko 2024-04-12T20:30:26.904 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_NFLOG.ko 2024-04-12T20:30:26.904 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_NFQUEUE.ko 2024-04-12T20:30:26.904 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_RATEEST.ko 2024-04-12T20:30:26.904 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_REDIRECT.ko 2024-04-12T20:30:26.904 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_SECMARK.ko 2024-04-12T20:30:26.904 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_TCPMSS.ko 2024-04-12T20:30:26.904 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_TCPOPTSTRIP.ko 2024-04-12T20:30:26.904 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_TEE.ko 2024-04-12T20:30:26.904 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_TPROXY.ko 2024-04-12T20:30:26.904 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_TRACE.ko 2024-04-12T20:30:26.904 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_addrtype.ko 2024-04-12T20:30:26.904 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_bpf.ko 2024-04-12T20:30:26.904 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_cgroup.ko 2024-04-12T20:30:26.905 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_cluster.ko 2024-04-12T20:30:26.905 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_comment.ko 2024-04-12T20:30:26.905 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_connbytes.ko 2024-04-12T20:30:26.905 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_connlabel.ko 2024-04-12T20:30:26.905 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_connlimit.ko 2024-04-12T20:30:26.905 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_connmark.ko 2024-04-12T20:30:26.905 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_conntrack.ko 2024-04-12T20:30:26.905 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_cpu.ko 2024-04-12T20:30:26.905 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_dccp.ko 2024-04-12T20:30:26.905 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_devgroup.ko 2024-04-12T20:30:26.905 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_dscp.ko 2024-04-12T20:30:26.905 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_ecn.ko 2024-04-12T20:30:26.905 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_esp.ko 2024-04-12T20:30:26.905 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_hashlimit.ko 2024-04-12T20:30:26.905 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_helper.ko 2024-04-12T20:30:26.905 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_hl.ko 2024-04-12T20:30:26.905 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_ipcomp.ko 2024-04-12T20:30:26.906 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_iprange.ko 2024-04-12T20:30:26.906 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_ipvs.ko 2024-04-12T20:30:26.906 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_l2tp.ko 2024-04-12T20:30:26.906 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_length.ko 2024-04-12T20:30:26.906 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_limit.ko 2024-04-12T20:30:26.906 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_mac.ko 2024-04-12T20:30:26.906 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_mark.ko 2024-04-12T20:30:26.906 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_multiport.ko 2024-04-12T20:30:26.906 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_nat.ko 2024-04-12T20:30:26.906 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_nfacct.ko 2024-04-12T20:30:26.906 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_osf.ko 2024-04-12T20:30:26.906 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_owner.ko 2024-04-12T20:30:26.906 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_physdev.ko 2024-04-12T20:30:26.906 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_pkttype.ko 2024-04-12T20:30:26.906 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_policy.ko 2024-04-12T20:30:26.907 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_quota.ko 2024-04-12T20:30:26.907 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_rateest.ko 2024-04-12T20:30:26.907 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_realm.ko 2024-04-12T20:30:26.907 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_recent.ko 2024-04-12T20:30:26.907 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_sctp.ko 2024-04-12T20:30:26.907 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_set.ko 2024-04-12T20:30:26.907 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_socket.ko 2024-04-12T20:30:26.907 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_state.ko 2024-04-12T20:30:26.907 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_statistic.ko 2024-04-12T20:30:26.907 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_string.ko 2024-04-12T20:30:26.907 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_tcpmss.ko 2024-04-12T20:30:26.907 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_time.ko 2024-04-12T20:30:26.907 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_u32.ko 2024-04-12T20:30:26.907 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netlink 2024-04-12T20:30:26.908 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netlink/netlink_diag.ko 2024-04-12T20:30:26.908 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/nsh 2024-04-12T20:30:26.908 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/nsh/nsh.ko 2024-04-12T20:30:26.908 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/openvswitch 2024-04-12T20:30:26.908 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/openvswitch/openvswitch.ko 2024-04-12T20:30:26.908 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/openvswitch/vport-geneve.ko 2024-04-12T20:30:26.908 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/openvswitch/vport-gre.ko 2024-04-12T20:30:26.908 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/openvswitch/vport-vxlan.ko 2024-04-12T20:30:26.908 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/packet 2024-04-12T20:30:26.908 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/packet/af_packet_diag.ko 2024-04-12T20:30:26.908 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/rds 2024-04-12T20:30:26.908 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/rds/rds.ko 2024-04-12T20:30:26.908 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/rds/rds_rdma.ko 2024-04-12T20:30:26.908 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/rds/rds_tcp.ko 2024-04-12T20:30:26.908 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched 2024-04-12T20:30:26.908 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/act_csum.ko 2024-04-12T20:30:26.909 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/act_gact.ko 2024-04-12T20:30:26.909 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/act_mirred.ko 2024-04-12T20:30:26.909 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/act_nat.ko 2024-04-12T20:30:26.909 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/act_pedit.ko 2024-04-12T20:30:26.909 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/act_police.ko 2024-04-12T20:30:26.909 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/act_simple.ko 2024-04-12T20:30:26.909 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/act_skbedit.ko 2024-04-12T20:30:26.909 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/act_vlan.ko 2024-04-12T20:30:26.909 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/cls_basic.ko 2024-04-12T20:30:26.909 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/cls_bpf.ko 2024-04-12T20:30:26.909 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/cls_flow.ko 2024-04-12T20:30:26.909 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/cls_fw.ko 2024-04-12T20:30:26.910 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/cls_route.ko 2024-04-12T20:30:26.910 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/cls_u32.ko 2024-04-12T20:30:26.910 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/em_cmp.ko 2024-04-12T20:30:26.910 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/em_ipset.ko 2024-04-12T20:30:26.910 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/em_meta.ko 2024-04-12T20:30:26.910 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/em_nbyte.ko 2024-04-12T20:30:26.910 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/em_text.ko 2024-04-12T20:30:26.910 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/em_u32.ko 2024-04-12T20:30:26.910 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/sch_choke.ko 2024-04-12T20:30:26.910 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/sch_codel.ko 2024-04-12T20:30:26.910 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/sch_drr.ko 2024-04-12T20:30:26.910 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/sch_fq.ko 2024-04-12T20:30:26.910 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/sch_gred.ko 2024-04-12T20:30:26.910 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/sch_hfsc.ko 2024-04-12T20:30:26.911 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/sch_hhf.ko 2024-04-12T20:30:26.911 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/sch_htb.ko 2024-04-12T20:30:26.911 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/sch_ingress.ko 2024-04-12T20:30:26.911 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/sch_mqprio.ko 2024-04-12T20:30:26.911 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/sch_mqprio_lib.ko 2024-04-12T20:30:26.911 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/sch_multiq.ko 2024-04-12T20:30:26.911 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/sch_netem.ko 2024-04-12T20:30:26.911 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/sch_pie.ko 2024-04-12T20:30:26.911 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/sch_plug.ko 2024-04-12T20:30:26.911 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/sch_prio.ko 2024-04-12T20:30:26.911 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/sch_qfq.ko 2024-04-12T20:30:26.911 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/sch_red.ko 2024-04-12T20:30:26.911 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/sch_sfb.ko 2024-04-12T20:30:26.911 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/sch_sfq.ko 2024-04-12T20:30:26.912 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/sch_tbf.ko 2024-04-12T20:30:26.912 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/sch_teql.ko 2024-04-12T20:30:26.912 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sctp 2024-04-12T20:30:26.912 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sctp/sctp.ko 2024-04-12T20:30:26.912 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sctp/sctp_diag.ko 2024-04-12T20:30:26.912 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sunrpc 2024-04-12T20:30:26.912 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sunrpc/auth_gss 2024-04-12T20:30:26.912 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sunrpc/auth_gss/auth_rpcgss.ko 2024-04-12T20:30:26.912 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sunrpc/auth_gss/rpcsec_gss_krb5.ko 2024-04-12T20:30:26.912 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sunrpc/sunrpc.ko 2024-04-12T20:30:26.912 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sunrpc/xprtrdma 2024-04-12T20:30:26.912 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sunrpc/xprtrdma/rpcrdma.ko 2024-04-12T20:30:26.912 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/tipc 2024-04-12T20:30:26.912 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/tipc/tipc.ko 2024-04-12T20:30:26.912 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/tipc/tipc_diag.ko 2024-04-12T20:30:26.912 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/unix 2024-04-12T20:30:26.913 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/unix/unix_diag.ko 2024-04-12T20:30:26.913 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/vmw_vsock 2024-04-12T20:30:26.913 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/vmw_vsock/vmw_vsock_virtio_transport_common.ko 2024-04-12T20:30:26.913 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/vmw_vsock/vmw_vsock_vmci_transport.ko 2024-04-12T20:30:26.913 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/vmw_vsock/vsock.ko 2024-04-12T20:30:26.913 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/vmw_vsock/vsock_diag.ko 2024-04-12T20:30:26.913 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/vmw_vsock/vsock_loopback.ko 2024-04-12T20:30:26.913 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/xfrm 2024-04-12T20:30:26.913 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/xfrm/xfrm_ipcomp.ko 2024-04-12T20:30:26.913 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/security 2024-04-12T20:30:26.913 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/security/keys 2024-04-12T20:30:26.913 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/security/keys/encrypted-keys 2024-04-12T20:30:26.913 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/security/keys/encrypted-keys/encrypted-keys.ko 2024-04-12T20:30:26.913 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/security/keys/trusted-keys 2024-04-12T20:30:26.913 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/security/keys/trusted-keys/trusted.ko 2024-04-12T20:30:26.914 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/modules.alias 2024-04-12T20:30:26.914 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/modules.alias.bin 2024-04-12T20:30:26.914 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/modules.builtin 2024-04-12T20:30:26.914 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/modules.builtin.alias.bin 2024-04-12T20:30:26.914 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/modules.builtin.bin 2024-04-12T20:30:26.914 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/modules.builtin.modinfo 2024-04-12T20:30:26.914 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/modules.dep 2024-04-12T20:30:26.914 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/modules.dep.bin 2024-04-12T20:30:26.914 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/modules.devname 2024-04-12T20:30:26.914 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/modules.order 2024-04-12T20:30:26.914 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/modules.softdep 2024-04-12T20:30:26.914 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/modules.symbols 2024-04-12T20:30:26.914 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/modules.symbols.bin 2024-04-12T20:30:26.914 INFO:teuthology.orchestra.run.smithi018.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/vmlinuz 2024-04-12T20:30:26.918 DEBUG:teuthology.task.kernel:get_image_version: 6.9.0-rc3-g3f60ed14f7c1 2024-04-12T20:30:26.919 DEBUG:teuthology.orchestra.run.smithi018:> rpm --scripts -qp /tmp/kernel.x86_64.rpm 2024-04-12T20:30:26.942 INFO:teuthology.orchestra.run.smithi018.stdout:postinstall scriptlet (using /bin/sh): 2024-04-12T20:30:26.942 INFO:teuthology.orchestra.run.smithi018.stdout:if [ -x /usr/bin/kernel-install ]; then 2024-04-12T20:30:26.943 INFO:teuthology.orchestra.run.smithi018.stdout: /usr/bin/kernel-install add 6.9.0-rc3-g3f60ed14f7c1 /lib/modules/6.9.0-rc3-g3f60ed14f7c1/vmlinuz 2024-04-12T20:30:26.943 INFO:teuthology.orchestra.run.smithi018.stdout:fi 2024-04-12T20:30:26.943 INFO:teuthology.orchestra.run.smithi018.stdout:for file in vmlinuz System.map config; do 2024-04-12T20:30:26.943 INFO:teuthology.orchestra.run.smithi018.stdout: if ! cmp --silent "/lib/modules/6.9.0-rc3-g3f60ed14f7c1/${file}" "/boot/${file}-6.9.0-rc3-g3f60ed14f7c1"; then 2024-04-12T20:30:26.943 INFO:teuthology.orchestra.run.smithi018.stdout: cp "/lib/modules/6.9.0-rc3-g3f60ed14f7c1/${file}" "/boot/${file}-6.9.0-rc3-g3f60ed14f7c1" 2024-04-12T20:30:26.943 INFO:teuthology.orchestra.run.smithi018.stdout: fi 2024-04-12T20:30:26.943 INFO:teuthology.orchestra.run.smithi018.stdout:done 2024-04-12T20:30:26.943 INFO:teuthology.orchestra.run.smithi018.stdout:if [ -d "/lib/modules/6.9.0-rc3-g3f60ed14f7c1/dtb" ] && \ 2024-04-12T20:30:26.943 INFO:teuthology.orchestra.run.smithi018.stdout: ! diff -rq "/lib/modules/6.9.0-rc3-g3f60ed14f7c1/dtb" "/boot/dtb-6.9.0-rc3-g3f60ed14f7c1" >/dev/null 2>&1; then 2024-04-12T20:30:26.943 INFO:teuthology.orchestra.run.smithi018.stdout: rm -rf "/boot/dtb-6.9.0-rc3-g3f60ed14f7c1" 2024-04-12T20:30:26.943 INFO:teuthology.orchestra.run.smithi018.stdout: cp -r "/lib/modules/6.9.0-rc3-g3f60ed14f7c1/dtb" "/boot/dtb-6.9.0-rc3-g3f60ed14f7c1" 2024-04-12T20:30:26.943 INFO:teuthology.orchestra.run.smithi018.stdout:fi 2024-04-12T20:30:26.943 INFO:teuthology.orchestra.run.smithi018.stdout:if [ ! -e "/lib/modules/6.9.0-rc3-g3f60ed14f7c1/modules.dep" ]; then 2024-04-12T20:30:26.943 INFO:teuthology.orchestra.run.smithi018.stdout: /usr/sbin/depmod 6.9.0-rc3-g3f60ed14f7c1 2024-04-12T20:30:26.943 INFO:teuthology.orchestra.run.smithi018.stdout:fi 2024-04-12T20:30:26.944 INFO:teuthology.orchestra.run.smithi018.stdout:preuninstall scriptlet (using /bin/sh): 2024-04-12T20:30:26.944 INFO:teuthology.orchestra.run.smithi018.stdout:if [ -x /usr/bin/kernel-install ]; then 2024-04-12T20:30:26.944 INFO:teuthology.orchestra.run.smithi018.stdout:kernel-install remove 6.9.0-rc3-g3f60ed14f7c1 2024-04-12T20:30:26.944 INFO:teuthology.orchestra.run.smithi018.stdout:fi 2024-04-12T20:30:26.945 DEBUG:teuthology.orchestra.run.smithi018:> sudo rpm -qi grub2-tools 2024-04-12T20:30:27.025 INFO:teuthology.orchestra.run.smithi018.stdout:Name : grub2-tools 2024-04-12T20:30:27.025 INFO:teuthology.orchestra.run.smithi018.stdout:Epoch : 1 2024-04-12T20:30:27.025 INFO:teuthology.orchestra.run.smithi018.stdout:Version : 2.06 2024-04-12T20:30:27.025 INFO:teuthology.orchestra.run.smithi018.stdout:Release : 68.el9 2024-04-12T20:30:27.025 INFO:teuthology.orchestra.run.smithi018.stdout:Architecture: x86_64 2024-04-12T20:30:27.026 INFO:teuthology.orchestra.run.smithi018.stdout:Install Date: Fri 01 Sep 2023 09:14:30 PM UTC 2024-04-12T20:30:27.026 INFO:teuthology.orchestra.run.smithi018.stdout:Group : Unspecified 2024-04-12T20:30:27.026 INFO:teuthology.orchestra.run.smithi018.stdout:Size : 8264254 2024-04-12T20:30:27.026 INFO:teuthology.orchestra.run.smithi018.stdout:License : GPLv3+ 2024-04-12T20:30:27.026 INFO:teuthology.orchestra.run.smithi018.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2024-04-12T20:30:27.026 INFO:teuthology.orchestra.run.smithi018.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2024-04-12T20:30:27.026 INFO:teuthology.orchestra.run.smithi018.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2024-04-12T20:30:27.026 INFO:teuthology.orchestra.run.smithi018.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-04-12T20:30:27.026 INFO:teuthology.orchestra.run.smithi018.stdout:Packager : builder@centos.org 2024-04-12T20:30:27.026 INFO:teuthology.orchestra.run.smithi018.stdout:Vendor : CentOS 2024-04-12T20:30:27.026 INFO:teuthology.orchestra.run.smithi018.stdout:URL : http://www.gnu.org/software/grub/ 2024-04-12T20:30:27.026 INFO:teuthology.orchestra.run.smithi018.stdout:Summary : Support tools for GRUB. 2024-04-12T20:30:27.026 INFO:teuthology.orchestra.run.smithi018.stdout:Description : 2024-04-12T20:30:27.026 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-12T20:30:27.026 INFO:teuthology.orchestra.run.smithi018.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-04-12T20:30:27.027 INFO:teuthology.orchestra.run.smithi018.stdout:customizable bootloader with modular architecture. It supports a rich 2024-04-12T20:30:27.027 INFO:teuthology.orchestra.run.smithi018.stdout:variety of kernel formats, file systems, computer architectures and 2024-04-12T20:30:27.027 INFO:teuthology.orchestra.run.smithi018.stdout:hardware devices. 2024-04-12T20:30:27.027 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-12T20:30:27.027 INFO:teuthology.orchestra.run.smithi018.stdout:This subpackage provides tools for support of all platforms. 2024-04-12T20:30:27.028 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-04-12T20:30:27.028 INFO:teuthology.task.kernel:Updating grub on smithi018 to boot 6.9.0-rc3-g3f60ed14f7c1 2024-04-12T20:30:27.028 DEBUG:teuthology.orchestra.run.smithi018:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-04-12T20:30:27.732 INFO:teuthology.orchestra.run.smithi018.stderr:Generating grub configuration file ... 2024-04-12T20:30:29.087 INFO:teuthology.orchestra.run.smithi018.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-04-12T20:30:29.127 INFO:teuthology.orchestra.run.smithi018.stderr:done 2024-04-12T20:30:29.129 DEBUG:teuthology.orchestra.run.smithi018:> mktemp 2024-04-12T20:30:29.145 INFO:teuthology.orchestra.run.smithi018.stdout:/tmp/tmp.XXJ58i1WPe 2024-04-12T20:30:29.145 DEBUG:teuthology.orchestra.run.smithi018:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.XXJ58i1WPe 2024-04-12T20:30:29.217 DEBUG:teuthology.orchestra.run.smithi018:> sudo chmod 0666 /tmp/tmp.XXJ58i1WPe 2024-04-12T20:30:29.362 DEBUG:teuthology.orchestra.remote:smithi018:/tmp/tmp.XXJ58i1WPe is 6KB 2024-04-12T20:30:29.411 DEBUG:teuthology.orchestra.run.smithi018:> rm -fr /tmp/tmp.XXJ58i1WPe 2024-04-12T20:30:29.425 DEBUG:teuthology.orchestra.run.smithi018:> sudo /bin/ls /boot/loader/entries || true 2024-04-12T20:30:29.496 INFO:teuthology.orchestra.run.smithi018.stdout:15949e3c88704300882482eddfd7baae-0-rescue.conf 2024-04-12T20:30:29.496 INFO:teuthology.orchestra.run.smithi018.stdout:15949e3c88704300882482eddfd7baae-5.14.0-239.el9.x86_64.conf 2024-04-12T20:30:29.496 INFO:teuthology.orchestra.run.smithi018.stdout:15949e3c88704300882482eddfd7baae-5.14.0-361.el9.x86_64.conf 2024-04-12T20:30:29.496 INFO:teuthology.orchestra.run.smithi018.stdout:15949e3c88704300882482eddfd7baae-6.9.0-rc3-g3f60ed14f7c1.conf 2024-04-12T20:30:29.498 DEBUG:teuthology.orchestra.run.smithi018:> sudo grub2-set-default 15949e3c88704300882482eddfd7baae-6.9.0-rc3-g3f60ed14f7c1 2024-04-12T20:30:29.648 DEBUG:teuthology.orchestra.run.smithi018:> sudo shutdown -r now 2024-04-12T20:30:38.422 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi186.front.sepia.ceph.com, path=/tmp/kernel.x86_64.rpm, version=None) 2024-04-12T20:30:38.423 DEBUG:teuthology.orchestra.run.smithi186:> rpm -qlp /tmp/kernel.x86_64.rpm 2024-04-12T20:30:38.499 INFO:teuthology.orchestra.run.smithi186.stdout:/boot/System.map-6.9.0-rc3-g3f60ed14f7c1 2024-04-12T20:30:38.499 INFO:teuthology.orchestra.run.smithi186.stdout:/boot/config-6.9.0-rc3-g3f60ed14f7c1 2024-04-12T20:30:38.499 INFO:teuthology.orchestra.run.smithi186.stdout:/boot/vmlinuz-6.9.0-rc3-g3f60ed14f7c1 2024-04-12T20:30:38.499 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/System.map 2024-04-12T20:30:38.499 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/config 2024-04-12T20:30:38.499 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel 2024-04-12T20:30:38.499 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch 2024-04-12T20:30:38.499 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch/x86 2024-04-12T20:30:38.500 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch/x86/crypto 2024-04-12T20:30:38.500 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch/x86/crypto/blowfish-x86_64.ko 2024-04-12T20:30:38.500 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch/x86/crypto/camellia-aesni-avx-x86_64.ko 2024-04-12T20:30:38.500 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch/x86/crypto/camellia-aesni-avx2.ko 2024-04-12T20:30:38.500 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch/x86/crypto/camellia-x86_64.ko 2024-04-12T20:30:38.500 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch/x86/crypto/cast5-avx-x86_64.ko 2024-04-12T20:30:38.500 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch/x86/crypto/cast6-avx-x86_64.ko 2024-04-12T20:30:38.500 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch/x86/crypto/crc32-pclmul.ko 2024-04-12T20:30:38.500 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch/x86/crypto/crc32c-intel.ko 2024-04-12T20:30:38.500 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch/x86/crypto/crct10dif-pclmul.ko 2024-04-12T20:30:38.500 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch/x86/crypto/des3_ede-x86_64.ko 2024-04-12T20:30:38.500 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch/x86/crypto/ghash-clmulni-intel.ko 2024-04-12T20:30:38.500 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch/x86/crypto/serpent-avx-x86_64.ko 2024-04-12T20:30:38.500 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch/x86/crypto/serpent-avx2.ko 2024-04-12T20:30:38.501 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch/x86/crypto/serpent-sse2-x86_64.ko 2024-04-12T20:30:38.501 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch/x86/crypto/sha1-ssse3.ko 2024-04-12T20:30:38.501 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch/x86/crypto/sha256-ssse3.ko 2024-04-12T20:30:38.501 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch/x86/crypto/sha512-ssse3.ko 2024-04-12T20:30:38.501 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch/x86/crypto/twofish-avx-x86_64.ko 2024-04-12T20:30:38.501 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch/x86/crypto/twofish-x86_64-3way.ko 2024-04-12T20:30:38.501 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch/x86/crypto/twofish-x86_64.ko 2024-04-12T20:30:38.501 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch/x86/kernel 2024-04-12T20:30:38.501 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch/x86/kernel/cpu 2024-04-12T20:30:38.501 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch/x86/kernel/cpu/mce 2024-04-12T20:30:38.501 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch/x86/kernel/cpu/mce/mce-inject.ko 2024-04-12T20:30:38.501 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch/x86/kvm 2024-04-12T20:30:38.501 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch/x86/kvm/kvm-amd.ko 2024-04-12T20:30:38.501 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch/x86/kvm/kvm-intel.ko 2024-04-12T20:30:38.501 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch/x86/kvm/kvm.ko 2024-04-12T20:30:38.502 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto 2024-04-12T20:30:38.502 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/ansi_cprng.ko 2024-04-12T20:30:38.502 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/anubis.ko 2024-04-12T20:30:38.502 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/arc4.ko 2024-04-12T20:30:38.502 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/async_tx 2024-04-12T20:30:38.502 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/async_tx/async_memcpy.ko 2024-04-12T20:30:38.502 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/async_tx/async_pq.ko 2024-04-12T20:30:38.502 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/async_tx/async_raid6_recov.ko 2024-04-12T20:30:38.502 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/async_tx/async_tx.ko 2024-04-12T20:30:38.502 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/async_tx/async_xor.ko 2024-04-12T20:30:38.502 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/async_tx/raid6test.ko 2024-04-12T20:30:38.502 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/authenc.ko 2024-04-12T20:30:38.502 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/authencesn.ko 2024-04-12T20:30:38.503 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/blake2b_generic.ko 2024-04-12T20:30:38.503 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/blowfish_common.ko 2024-04-12T20:30:38.503 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/blowfish_generic.ko 2024-04-12T20:30:38.503 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/camellia_generic.ko 2024-04-12T20:30:38.503 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/cast5_generic.ko 2024-04-12T20:30:38.503 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/cast6_generic.ko 2024-04-12T20:30:38.503 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/cast_common.ko 2024-04-12T20:30:38.503 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/ccm.ko 2024-04-12T20:30:38.503 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/cmac.ko 2024-04-12T20:30:38.503 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/crc32_generic.ko 2024-04-12T20:30:38.503 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/crypto_engine.ko 2024-04-12T20:30:38.503 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/crypto_user.ko 2024-04-12T20:30:38.503 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/cts.ko 2024-04-12T20:30:38.504 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/des_generic.ko 2024-04-12T20:30:38.504 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/dh_generic.ko 2024-04-12T20:30:38.504 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/echainiv.ko 2024-04-12T20:30:38.504 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/essiv.ko 2024-04-12T20:30:38.504 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/fcrypt.ko 2024-04-12T20:30:38.504 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/khazad.ko 2024-04-12T20:30:38.504 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/lz4.ko 2024-04-12T20:30:38.504 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/lz4hc.ko 2024-04-12T20:30:38.504 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/md4.ko 2024-04-12T20:30:38.504 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/michael_mic.ko 2024-04-12T20:30:38.504 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/pcbc.ko 2024-04-12T20:30:38.504 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/pcrypt.ko 2024-04-12T20:30:38.504 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/rmd160.ko 2024-04-12T20:30:38.504 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/seed.ko 2024-04-12T20:30:38.505 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/serpent_generic.ko 2024-04-12T20:30:38.505 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/tcrypt.ko 2024-04-12T20:30:38.505 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/tea.ko 2024-04-12T20:30:38.505 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/twofish_common.ko 2024-04-12T20:30:38.505 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/twofish_generic.ko 2024-04-12T20:30:38.505 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/vmac.ko 2024-04-12T20:30:38.505 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/wp512.ko 2024-04-12T20:30:38.505 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/xcbc.ko 2024-04-12T20:30:38.505 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/xor.ko 2024-04-12T20:30:38.505 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/xxhash_generic.ko 2024-04-12T20:30:38.505 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers 2024-04-12T20:30:38.505 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/acpi 2024-04-12T20:30:38.505 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/acpi/acpi_ipmi.ko 2024-04-12T20:30:38.505 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/acpi/acpi_pad.ko 2024-04-12T20:30:38.505 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/acpi/ec_sys.ko 2024-04-12T20:30:38.506 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/acpi/platform_profile.ko 2024-04-12T20:30:38.506 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/acpi/sbs.ko 2024-04-12T20:30:38.506 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/acpi/sbshc.ko 2024-04-12T20:30:38.506 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/acpi/video.ko 2024-04-12T20:30:38.506 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata 2024-04-12T20:30:38.506 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/acard-ahci.ko 2024-04-12T20:30:38.506 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/ahci_platform.ko 2024-04-12T20:30:38.506 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/ata_generic.ko 2024-04-12T20:30:38.506 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/libahci_platform.ko 2024-04-12T20:30:38.506 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_acpi.ko 2024-04-12T20:30:38.506 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_ali.ko 2024-04-12T20:30:38.506 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_amd.ko 2024-04-12T20:30:38.506 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_artop.ko 2024-04-12T20:30:38.507 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_atiixp.ko 2024-04-12T20:30:38.507 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_atp867x.ko 2024-04-12T20:30:38.507 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_cmd640.ko 2024-04-12T20:30:38.507 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_cmd64x.ko 2024-04-12T20:30:38.507 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_cypress.ko 2024-04-12T20:30:38.507 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_efar.ko 2024-04-12T20:30:38.507 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_hpt366.ko 2024-04-12T20:30:38.507 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_hpt37x.ko 2024-04-12T20:30:38.507 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_hpt3x2n.ko 2024-04-12T20:30:38.507 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_hpt3x3.ko 2024-04-12T20:30:38.507 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_it8213.ko 2024-04-12T20:30:38.507 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_it821x.ko 2024-04-12T20:30:38.507 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_jmicron.ko 2024-04-12T20:30:38.507 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_marvell.ko 2024-04-12T20:30:38.507 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_mpiix.ko 2024-04-12T20:30:38.508 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_netcell.ko 2024-04-12T20:30:38.508 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_ninja32.ko 2024-04-12T20:30:38.508 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_ns87410.ko 2024-04-12T20:30:38.508 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_ns87415.ko 2024-04-12T20:30:38.508 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_oldpiix.ko 2024-04-12T20:30:38.508 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_opti.ko 2024-04-12T20:30:38.508 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_optidma.ko 2024-04-12T20:30:38.508 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_pcmcia.ko 2024-04-12T20:30:38.508 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_pdc2027x.ko 2024-04-12T20:30:38.508 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_pdc202xx_old.ko 2024-04-12T20:30:38.508 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_piccolo.ko 2024-04-12T20:30:38.508 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_rdc.ko 2024-04-12T20:30:38.508 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_sch.ko 2024-04-12T20:30:38.508 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_serverworks.ko 2024-04-12T20:30:38.509 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_sil680.ko 2024-04-12T20:30:38.509 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_sis.ko 2024-04-12T20:30:38.509 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_sl82c105.ko 2024-04-12T20:30:38.509 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_triflex.ko 2024-04-12T20:30:38.509 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_via.ko 2024-04-12T20:30:38.509 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pdc_adma.ko 2024-04-12T20:30:38.509 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/sata_inic162x.ko 2024-04-12T20:30:38.509 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/sata_mv.ko 2024-04-12T20:30:38.509 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/sata_nv.ko 2024-04-12T20:30:38.509 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/sata_promise.ko 2024-04-12T20:30:38.509 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/sata_qstor.ko 2024-04-12T20:30:38.509 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/sata_sil.ko 2024-04-12T20:30:38.509 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/sata_sil24.ko 2024-04-12T20:30:38.510 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/sata_sis.ko 2024-04-12T20:30:38.510 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/sata_svw.ko 2024-04-12T20:30:38.510 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/sata_sx4.ko 2024-04-12T20:30:38.510 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/sata_uli.ko 2024-04-12T20:30:38.510 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/sata_via.ko 2024-04-12T20:30:38.510 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/sata_vsc.ko 2024-04-12T20:30:38.510 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/auxdisplay 2024-04-12T20:30:38.510 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/auxdisplay/ks0108.ko 2024-04-12T20:30:38.510 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/base 2024-04-12T20:30:38.510 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/base/regmap 2024-04-12T20:30:38.510 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/base/regmap/regmap-i2c.ko 2024-04-12T20:30:38.510 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/base/regmap/regmap-mmio.ko 2024-04-12T20:30:38.510 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/bcma 2024-04-12T20:30:38.511 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/bcma/bcma.ko 2024-04-12T20:30:38.511 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/block 2024-04-12T20:30:38.511 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/block/aoe 2024-04-12T20:30:38.511 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/block/aoe/aoe.ko 2024-04-12T20:30:38.511 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/block/brd.ko 2024-04-12T20:30:38.511 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/block/drbd 2024-04-12T20:30:38.511 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/block/drbd/drbd.ko 2024-04-12T20:30:38.511 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/block/floppy.ko 2024-04-12T20:30:38.511 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/block/loop.ko 2024-04-12T20:30:38.511 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/block/mtip32xx 2024-04-12T20:30:38.511 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/block/mtip32xx/mtip32xx.ko 2024-04-12T20:30:38.511 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/block/nbd.ko 2024-04-12T20:30:38.511 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/block/null_blk 2024-04-12T20:30:38.512 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/block/null_blk/null_blk.ko 2024-04-12T20:30:38.512 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/block/pktcdvd.ko 2024-04-12T20:30:38.512 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/block/rbd.ko 2024-04-12T20:30:38.512 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/block/virtio_blk.ko 2024-04-12T20:30:38.512 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/block/xen-blkback 2024-04-12T20:30:38.512 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/block/xen-blkback/xen-blkback.ko 2024-04-12T20:30:38.512 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/block/xen-blkfront.ko 2024-04-12T20:30:38.512 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/block/zram 2024-04-12T20:30:38.512 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/block/zram/zram.ko 2024-04-12T20:30:38.512 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/char 2024-04-12T20:30:38.512 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/char/hangcheck-timer.ko 2024-04-12T20:30:38.512 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/char/hw_random 2024-04-12T20:30:38.512 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/char/hw_random/amd-rng.ko 2024-04-12T20:30:38.512 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/char/hw_random/intel-rng.ko 2024-04-12T20:30:38.513 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/char/hw_random/timeriomem-rng.ko 2024-04-12T20:30:38.513 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/char/hw_random/via-rng.ko 2024-04-12T20:30:38.513 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/char/hw_random/virtio-rng.ko 2024-04-12T20:30:38.513 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/char/ipmi 2024-04-12T20:30:38.513 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/char/ipmi/ipmi_devintf.ko 2024-04-12T20:30:38.513 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/char/ipmi/ipmi_msghandler.ko 2024-04-12T20:30:38.513 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/char/ipmi/ipmi_poweroff.ko 2024-04-12T20:30:38.513 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/char/ipmi/ipmi_si.ko 2024-04-12T20:30:38.513 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/char/ipmi/ipmi_ssif.ko 2024-04-12T20:30:38.513 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/char/ipmi/ipmi_watchdog.ko 2024-04-12T20:30:38.513 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/char/lp.ko 2024-04-12T20:30:38.513 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/char/mwave 2024-04-12T20:30:38.513 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/char/mwave/mwave.ko 2024-04-12T20:30:38.514 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/char/ppdev.ko 2024-04-12T20:30:38.514 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/char/tlclk.ko 2024-04-12T20:30:38.514 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/char/tpm 2024-04-12T20:30:38.514 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/char/tpm/tpm.ko 2024-04-12T20:30:38.514 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/char/tpm/tpm_atmel.ko 2024-04-12T20:30:38.514 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/char/tpm/tpm_infineon.ko 2024-04-12T20:30:38.514 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/char/tpm/tpm_nsc.ko 2024-04-12T20:30:38.514 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/char/tpm/tpm_tis.ko 2024-04-12T20:30:38.514 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/char/tpm/tpm_tis_core.ko 2024-04-12T20:30:38.514 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/char/uv_mmtimer.ko 2024-04-12T20:30:38.514 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/char/virtio_console.ko 2024-04-12T20:30:38.514 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/cpufreq 2024-04-12T20:30:38.514 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/cpufreq/acpi-cpufreq.ko 2024-04-12T20:30:38.515 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/cpufreq/amd_freq_sensitivity.ko 2024-04-12T20:30:38.515 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/cpufreq/p4-clockmod.ko 2024-04-12T20:30:38.515 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/cpufreq/pcc-cpufreq.ko 2024-04-12T20:30:38.515 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/cpufreq/powernow-k8.ko 2024-04-12T20:30:38.515 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/cpufreq/speedstep-lib.ko 2024-04-12T20:30:38.515 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/crypto 2024-04-12T20:30:38.515 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/crypto/ccp 2024-04-12T20:30:38.515 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/crypto/ccp/ccp-crypto.ko 2024-04-12T20:30:38.515 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/crypto/ccp/ccp.ko 2024-04-12T20:30:38.515 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/crypto/intel 2024-04-12T20:30:38.515 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/crypto/intel/qat 2024-04-12T20:30:38.515 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/crypto/intel/qat/qat_common 2024-04-12T20:30:38.515 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/crypto/intel/qat/qat_common/intel_qat.ko 2024-04-12T20:30:38.515 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/crypto/intel/qat/qat_dh895xcc 2024-04-12T20:30:38.516 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/crypto/intel/qat/qat_dh895xcc/qat_dh895xcc.ko 2024-04-12T20:30:38.516 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/crypto/padlock-aes.ko 2024-04-12T20:30:38.516 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/crypto/padlock-sha.ko 2024-04-12T20:30:38.516 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/crypto/virtio 2024-04-12T20:30:38.516 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/crypto/virtio/virtio_crypto.ko 2024-04-12T20:30:38.516 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/dca 2024-04-12T20:30:38.516 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/dca/dca.ko 2024-04-12T20:30:38.516 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/devfreq 2024-04-12T20:30:38.516 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/devfreq/governor_simpleondemand.ko 2024-04-12T20:30:38.516 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/dma 2024-04-12T20:30:38.516 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/dma/dw 2024-04-12T20:30:38.516 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/dma/dw/dw_dmac.ko 2024-04-12T20:30:38.516 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/dma/ioat 2024-04-12T20:30:38.516 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/dma/ioat/ioatdma.ko 2024-04-12T20:30:38.517 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/edac 2024-04-12T20:30:38.517 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/edac/amd64_edac.ko 2024-04-12T20:30:38.517 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/edac/e752x_edac.ko 2024-04-12T20:30:38.517 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/edac/edac_mce_amd.ko 2024-04-12T20:30:38.517 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/edac/i3000_edac.ko 2024-04-12T20:30:38.517 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/edac/i3200_edac.ko 2024-04-12T20:30:38.517 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/edac/i5100_edac.ko 2024-04-12T20:30:38.517 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/edac/i5400_edac.ko 2024-04-12T20:30:38.517 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/edac/i7300_edac.ko 2024-04-12T20:30:38.517 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/edac/i7core_edac.ko 2024-04-12T20:30:38.517 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/edac/i82975x_edac.ko 2024-04-12T20:30:38.517 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/edac/ie31200_edac.ko 2024-04-12T20:30:38.517 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/edac/sb_edac.ko 2024-04-12T20:30:38.518 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/edac/x38_edac.ko 2024-04-12T20:30:38.518 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/firmware 2024-04-12T20:30:38.518 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/firmware/edd.ko 2024-04-12T20:30:38.518 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/firmware/iscsi_ibft.ko 2024-04-12T20:30:38.518 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/gpio 2024-04-12T20:30:38.518 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/gpio/gpio-ich.ko 2024-04-12T20:30:38.518 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/gpio/gpio-viperboard.ko 2024-04-12T20:30:38.518 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid 2024-04-12T20:30:38.518 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-appleir.ko 2024-04-12T20:30:38.518 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-aureal.ko 2024-04-12T20:30:38.518 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-axff.ko 2024-04-12T20:30:38.518 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-dr.ko 2024-04-12T20:30:38.519 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-elecom.ko 2024-04-12T20:30:38.519 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-elo.ko 2024-04-12T20:30:38.519 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-emsff.ko 2024-04-12T20:30:38.519 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-gaff.ko 2024-04-12T20:30:38.519 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-gt683r.ko 2024-04-12T20:30:38.519 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-gyration.ko 2024-04-12T20:30:38.519 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-holtek-kbd.ko 2024-04-12T20:30:38.519 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-holtek-mouse.ko 2024-04-12T20:30:38.519 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-holtekff.ko 2024-04-12T20:30:38.519 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-hyperv.ko 2024-04-12T20:30:38.519 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-icade.ko 2024-04-12T20:30:38.519 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-keytouch.ko 2024-04-12T20:30:38.519 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-kye.ko 2024-04-12T20:30:38.519 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-lcpower.ko 2024-04-12T20:30:38.519 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-led.ko 2024-04-12T20:30:38.520 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-lenovo.ko 2024-04-12T20:30:38.520 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-logitech-dj.ko 2024-04-12T20:30:38.520 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-logitech-hidpp.ko 2024-04-12T20:30:38.520 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-multitouch.ko 2024-04-12T20:30:38.520 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-ortek.ko 2024-04-12T20:30:38.520 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-penmount.ko 2024-04-12T20:30:38.520 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-petalynx.ko 2024-04-12T20:30:38.520 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-picolcd.ko 2024-04-12T20:30:38.520 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-pl.ko 2024-04-12T20:30:38.520 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-plantronics.ko 2024-04-12T20:30:38.520 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-primax.ko 2024-04-12T20:30:38.520 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-rmi.ko 2024-04-12T20:30:38.520 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-roccat-arvo.ko 2024-04-12T20:30:38.521 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-roccat-common.ko 2024-04-12T20:30:38.521 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-roccat-isku.ko 2024-04-12T20:30:38.521 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-roccat-kone.ko 2024-04-12T20:30:38.521 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-roccat-koneplus.ko 2024-04-12T20:30:38.521 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-roccat-konepure.ko 2024-04-12T20:30:38.521 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-roccat-kovaplus.ko 2024-04-12T20:30:38.521 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-roccat-lua.ko 2024-04-12T20:30:38.521 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-roccat-pyra.ko 2024-04-12T20:30:38.521 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-roccat-ryos.ko 2024-04-12T20:30:38.521 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-roccat-savu.ko 2024-04-12T20:30:38.521 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-roccat.ko 2024-04-12T20:30:38.521 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-saitek.ko 2024-04-12T20:30:38.521 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-samsung.ko 2024-04-12T20:30:38.522 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-sensor-hub.ko 2024-04-12T20:30:38.522 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-sjoy.ko 2024-04-12T20:30:38.522 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-sony.ko 2024-04-12T20:30:38.522 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-speedlink.ko 2024-04-12T20:30:38.522 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-steelseries.ko 2024-04-12T20:30:38.522 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-sunplus.ko 2024-04-12T20:30:38.522 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-thrustmaster.ko 2024-04-12T20:30:38.522 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-tivo.ko 2024-04-12T20:30:38.522 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-tmff.ko 2024-04-12T20:30:38.522 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-topseed.ko 2024-04-12T20:30:38.522 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-twinhan.ko 2024-04-12T20:30:38.522 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-uclogic.ko 2024-04-12T20:30:38.522 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-waltop.ko 2024-04-12T20:30:38.523 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-wiimote.ko 2024-04-12T20:30:38.523 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-xinmo.ko 2024-04-12T20:30:38.523 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-zpff.ko 2024-04-12T20:30:38.523 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-zydacron.ko 2024-04-12T20:30:38.523 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/uhid.ko 2024-04-12T20:30:38.523 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/wacom.ko 2024-04-12T20:30:38.523 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hv 2024-04-12T20:30:38.523 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hv/hv_balloon.ko 2024-04-12T20:30:38.523 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hv/hv_utils.ko 2024-04-12T20:30:38.523 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hv/hv_vmbus.ko 2024-04-12T20:30:38.523 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon 2024-04-12T20:30:38.523 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/abituguru.ko 2024-04-12T20:30:38.523 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/abituguru3.ko 2024-04-12T20:30:38.523 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/acpi_power_meter.ko 2024-04-12T20:30:38.525 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/ad7414.ko 2024-04-12T20:30:38.525 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/ad7418.ko 2024-04-12T20:30:38.525 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/adc128d818.ko 2024-04-12T20:30:38.525 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/adm1025.ko 2024-04-12T20:30:38.525 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/adm1026.ko 2024-04-12T20:30:38.525 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/adm1029.ko 2024-04-12T20:30:38.525 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/adm1031.ko 2024-04-12T20:30:38.525 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/adm9240.ko 2024-04-12T20:30:38.525 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/ads7828.ko 2024-04-12T20:30:38.526 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/adt7410.ko 2024-04-12T20:30:38.526 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/adt7411.ko 2024-04-12T20:30:38.526 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/adt7462.ko 2024-04-12T20:30:38.526 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/adt7470.ko 2024-04-12T20:30:38.526 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/adt7475.ko 2024-04-12T20:30:38.526 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/adt7x10.ko 2024-04-12T20:30:38.526 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/amc6821.ko 2024-04-12T20:30:38.526 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/applesmc.ko 2024-04-12T20:30:38.526 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/asb100.ko 2024-04-12T20:30:38.526 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/asc7621.ko 2024-04-12T20:30:38.526 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/asus_atk0110.ko 2024-04-12T20:30:38.526 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/atxp1.ko 2024-04-12T20:30:38.527 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/coretemp.ko 2024-04-12T20:30:38.527 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/dell-smm-hwmon.ko 2024-04-12T20:30:38.527 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/dme1737.ko 2024-04-12T20:30:38.527 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/ds1621.ko 2024-04-12T20:30:38.527 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/ds620.ko 2024-04-12T20:30:38.527 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/emc1403.ko 2024-04-12T20:30:38.527 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/emc6w201.ko 2024-04-12T20:30:38.527 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/f71805f.ko 2024-04-12T20:30:38.527 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/f71882fg.ko 2024-04-12T20:30:38.527 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/f75375s.ko 2024-04-12T20:30:38.527 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/fam15h_power.ko 2024-04-12T20:30:38.527 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/fschmd.ko 2024-04-12T20:30:38.527 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/g760a.ko 2024-04-12T20:30:38.527 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/g762.ko 2024-04-12T20:30:38.528 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/gl518sm.ko 2024-04-12T20:30:38.528 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/gl520sm.ko 2024-04-12T20:30:38.528 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/hwmon-vid.ko 2024-04-12T20:30:38.528 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/i5500_temp.ko 2024-04-12T20:30:38.528 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/i5k_amb.ko 2024-04-12T20:30:38.528 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/ibmaem.ko 2024-04-12T20:30:38.528 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/ibmpex.ko 2024-04-12T20:30:38.528 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/ina209.ko 2024-04-12T20:30:38.528 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/ina2xx.ko 2024-04-12T20:30:38.528 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/it87.ko 2024-04-12T20:30:38.528 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/k10temp.ko 2024-04-12T20:30:38.528 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/k8temp.ko 2024-04-12T20:30:38.528 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/lineage-pem.ko 2024-04-12T20:30:38.528 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/lm63.ko 2024-04-12T20:30:38.529 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/lm73.ko 2024-04-12T20:30:38.529 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/lm75.ko 2024-04-12T20:30:38.529 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/lm77.ko 2024-04-12T20:30:38.529 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/lm78.ko 2024-04-12T20:30:38.529 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/lm80.ko 2024-04-12T20:30:38.529 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/lm83.ko 2024-04-12T20:30:38.529 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/lm85.ko 2024-04-12T20:30:38.529 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/lm87.ko 2024-04-12T20:30:38.529 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/lm90.ko 2024-04-12T20:30:38.529 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/lm92.ko 2024-04-12T20:30:38.529 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/lm93.ko 2024-04-12T20:30:38.529 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/lm95234.ko 2024-04-12T20:30:38.529 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/lm95241.ko 2024-04-12T20:30:38.529 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/lm95245.ko 2024-04-12T20:30:38.529 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/ltc2945.ko 2024-04-12T20:30:38.529 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/ltc4151.ko 2024-04-12T20:30:38.530 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/ltc4215.ko 2024-04-12T20:30:38.530 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/ltc4222.ko 2024-04-12T20:30:38.530 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/ltc4245.ko 2024-04-12T20:30:38.530 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/ltc4260.ko 2024-04-12T20:30:38.530 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/ltc4261.ko 2024-04-12T20:30:38.530 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/max16065.ko 2024-04-12T20:30:38.530 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/max1619.ko 2024-04-12T20:30:38.530 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/max1668.ko 2024-04-12T20:30:38.530 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/max197.ko 2024-04-12T20:30:38.530 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/max6639.ko 2024-04-12T20:30:38.530 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/max6650.ko 2024-04-12T20:30:38.530 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/max6697.ko 2024-04-12T20:30:38.530 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/mcp3021.ko 2024-04-12T20:30:38.531 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/nct6683.ko 2024-04-12T20:30:38.531 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/nct6775-core.ko 2024-04-12T20:30:38.531 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/nct6775.ko 2024-04-12T20:30:38.531 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/nct7802.ko 2024-04-12T20:30:38.531 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/pc87360.ko 2024-04-12T20:30:38.531 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/pc87427.ko 2024-04-12T20:30:38.531 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/pcf8591.ko 2024-04-12T20:30:38.531 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/pmbus 2024-04-12T20:30:38.531 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/pmbus/adm1275.ko 2024-04-12T20:30:38.531 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/pmbus/lm25066.ko 2024-04-12T20:30:38.531 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/pmbus/ltc2978.ko 2024-04-12T20:30:38.531 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/pmbus/max16064.ko 2024-04-12T20:30:38.531 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/pmbus/max34440.ko 2024-04-12T20:30:38.531 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/pmbus/max8688.ko 2024-04-12T20:30:38.532 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/pmbus/pmbus.ko 2024-04-12T20:30:38.532 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/pmbus/pmbus_core.ko 2024-04-12T20:30:38.532 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/pmbus/tps40422.ko 2024-04-12T20:30:38.532 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/pmbus/ucd9000.ko 2024-04-12T20:30:38.532 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/pmbus/ucd9200.ko 2024-04-12T20:30:38.532 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/pmbus/zl6100.ko 2024-04-12T20:30:38.532 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/powr1220.ko 2024-04-12T20:30:38.532 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/sch5627.ko 2024-04-12T20:30:38.532 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/sch5636.ko 2024-04-12T20:30:38.532 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/sch56xx-common.ko 2024-04-12T20:30:38.532 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/sht15.ko 2024-04-12T20:30:38.532 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/sht21.ko 2024-04-12T20:30:38.532 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/shtc1.ko 2024-04-12T20:30:38.532 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/sis5595.ko 2024-04-12T20:30:38.533 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/smsc47b397.ko 2024-04-12T20:30:38.533 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/smsc47m1.ko 2024-04-12T20:30:38.533 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/smsc47m192.ko 2024-04-12T20:30:38.533 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/thmc50.ko 2024-04-12T20:30:38.533 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/tmp102.ko 2024-04-12T20:30:38.533 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/tmp103.ko 2024-04-12T20:30:38.533 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/tmp401.ko 2024-04-12T20:30:38.533 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/tmp421.ko 2024-04-12T20:30:38.533 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/via-cputemp.ko 2024-04-12T20:30:38.533 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/via686a.ko 2024-04-12T20:30:38.533 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/vt1211.ko 2024-04-12T20:30:38.533 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/vt8231.ko 2024-04-12T20:30:38.533 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/w83627ehf.ko 2024-04-12T20:30:38.534 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/w83627hf.ko 2024-04-12T20:30:38.534 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/w83781d.ko 2024-04-12T20:30:38.534 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/w83791d.ko 2024-04-12T20:30:38.534 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/w83792d.ko 2024-04-12T20:30:38.534 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/w83793.ko 2024-04-12T20:30:38.534 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/w83795.ko 2024-04-12T20:30:38.534 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/w83l785ts.ko 2024-04-12T20:30:38.534 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/w83l786ng.ko 2024-04-12T20:30:38.534 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c 2024-04-12T20:30:38.534 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/algos 2024-04-12T20:30:38.534 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/algos/i2c-algo-bit.ko 2024-04-12T20:30:38.534 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/algos/i2c-algo-pca.ko 2024-04-12T20:30:38.534 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/busses 2024-04-12T20:30:38.535 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/busses/i2c-amd756-s4882.ko 2024-04-12T20:30:38.535 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/busses/i2c-amd756.ko 2024-04-12T20:30:38.535 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/busses/i2c-amd8111.ko 2024-04-12T20:30:38.535 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/busses/i2c-ccgx-ucsi.ko 2024-04-12T20:30:38.535 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/busses/i2c-designware-core.ko 2024-04-12T20:30:38.535 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/busses/i2c-designware-pci.ko 2024-04-12T20:30:38.535 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/busses/i2c-designware-platform.ko 2024-04-12T20:30:38.535 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/busses/i2c-diolan-u2c.ko 2024-04-12T20:30:38.535 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/busses/i2c-i801.ko 2024-04-12T20:30:38.535 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/busses/i2c-isch.ko 2024-04-12T20:30:38.535 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/busses/i2c-ismt.ko 2024-04-12T20:30:38.535 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/busses/i2c-nforce2-s4985.ko 2024-04-12T20:30:38.535 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/busses/i2c-nforce2.ko 2024-04-12T20:30:38.535 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/busses/i2c-parport.ko 2024-04-12T20:30:38.536 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/busses/i2c-pca-platform.ko 2024-04-12T20:30:38.536 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/busses/i2c-piix4.ko 2024-04-12T20:30:38.536 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/busses/i2c-scmi.ko 2024-04-12T20:30:38.536 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/busses/i2c-simtec.ko 2024-04-12T20:30:38.536 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/busses/i2c-sis96x.ko 2024-04-12T20:30:38.536 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/busses/i2c-tiny-usb.ko 2024-04-12T20:30:38.536 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/busses/i2c-via.ko 2024-04-12T20:30:38.536 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/busses/i2c-viapro.ko 2024-04-12T20:30:38.536 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/busses/i2c-viperboard.ko 2024-04-12T20:30:38.536 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/i2c-dev.ko 2024-04-12T20:30:38.536 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/i2c-mux.ko 2024-04-12T20:30:38.536 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/i2c-slave-eeprom.ko 2024-04-12T20:30:38.536 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/i2c-smbus.ko 2024-04-12T20:30:38.536 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/i2c-stub.ko 2024-04-12T20:30:38.536 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/infiniband 2024-04-12T20:30:38.537 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/infiniband/core 2024-04-12T20:30:38.537 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/infiniband/core/ib_cm.ko 2024-04-12T20:30:38.537 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/infiniband/core/ib_core.ko 2024-04-12T20:30:38.537 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/infiniband/core/ib_umad.ko 2024-04-12T20:30:38.537 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/infiniband/core/ib_uverbs.ko 2024-04-12T20:30:38.537 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/infiniband/core/iw_cm.ko 2024-04-12T20:30:38.537 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/infiniband/core/rdma_cm.ko 2024-04-12T20:30:38.537 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/infiniband/core/rdma_ucm.ko 2024-04-12T20:30:38.537 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/infiniband/hw 2024-04-12T20:30:38.537 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/infiniband/hw/cxgb4 2024-04-12T20:30:38.537 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/infiniband/hw/cxgb4/iw_cxgb4.ko 2024-04-12T20:30:38.537 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/infiniband/hw/mlx4 2024-04-12T20:30:38.537 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/infiniband/hw/mlx4/mlx4_ib.ko 2024-04-12T20:30:38.537 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/infiniband/hw/mlx5 2024-04-12T20:30:38.537 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/infiniband/hw/mlx5/mlx5_ib.ko 2024-04-12T20:30:38.537 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/infiniband/hw/mthca 2024-04-12T20:30:38.538 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/infiniband/hw/mthca/ib_mthca.ko 2024-04-12T20:30:38.538 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/infiniband/ulp 2024-04-12T20:30:38.538 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/infiniband/ulp/ipoib 2024-04-12T20:30:38.538 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/infiniband/ulp/ipoib/ib_ipoib.ko 2024-04-12T20:30:38.538 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/infiniband/ulp/iser 2024-04-12T20:30:38.538 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/infiniband/ulp/iser/ib_iser.ko 2024-04-12T20:30:38.538 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/infiniband/ulp/isert 2024-04-12T20:30:38.538 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/infiniband/ulp/isert/ib_isert.ko 2024-04-12T20:30:38.538 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/infiniband/ulp/srp 2024-04-12T20:30:38.538 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/infiniband/ulp/srp/ib_srp.ko 2024-04-12T20:30:38.538 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/infiniband/ulp/srpt 2024-04-12T20:30:38.538 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/infiniband/ulp/srpt/ib_srpt.ko 2024-04-12T20:30:38.538 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/input 2024-04-12T20:30:38.539 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/input/gameport 2024-04-12T20:30:38.539 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/input/gameport/emu10k1-gp.ko 2024-04-12T20:30:38.539 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/input/gameport/fm801-gp.ko 2024-04-12T20:30:38.539 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/input/gameport/gameport.ko 2024-04-12T20:30:38.539 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/input/joydev.ko 2024-04-12T20:30:38.539 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/input/keyboard 2024-04-12T20:30:38.539 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/input/keyboard/gpio_keys.ko 2024-04-12T20:30:38.539 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/input/mouse 2024-04-12T20:30:38.539 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/input/mouse/appletouch.ko 2024-04-12T20:30:38.539 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/input/mouse/bcm5974.ko 2024-04-12T20:30:38.539 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/input/mouse/cyapatp.ko 2024-04-12T20:30:38.539 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/input/mouse/elan_i2c.ko 2024-04-12T20:30:38.539 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/input/mouse/sermouse.ko 2024-04-12T20:30:38.539 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/input/mouse/synaptics_i2c.ko 2024-04-12T20:30:38.539 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/input/mouse/synaptics_usb.ko 2024-04-12T20:30:38.539 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/input/mouse/vsxxxaa.ko 2024-04-12T20:30:38.540 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/input/rmi4 2024-04-12T20:30:38.540 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/input/rmi4/rmi_core.ko 2024-04-12T20:30:38.540 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/input/serio 2024-04-12T20:30:38.540 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/input/serio/altera_ps2.ko 2024-04-12T20:30:38.540 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/input/serio/arc_ps2.ko 2024-04-12T20:30:38.540 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/input/serio/hyperv-keyboard.ko 2024-04-12T20:30:38.540 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/input/serio/serio_raw.ko 2024-04-12T20:30:38.540 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/input/sparse-keymap.ko 2024-04-12T20:30:38.540 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/leds 2024-04-12T20:30:38.540 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/leds/leds-blinkm.ko 2024-04-12T20:30:38.540 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/leds/leds-lm3530.ko 2024-04-12T20:30:38.540 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/leds/leds-lp3944.ko 2024-04-12T20:30:38.540 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/leds/leds-ss4200.ko 2024-04-12T20:30:38.540 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/leds/trigger 2024-04-12T20:30:38.540 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/leds/trigger/ledtrig-audio.ko 2024-04-12T20:30:38.540 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/leds/trigger/ledtrig-backlight.ko 2024-04-12T20:30:38.541 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/leds/trigger/ledtrig-camera.ko 2024-04-12T20:30:38.541 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/leds/trigger/ledtrig-default-on.ko 2024-04-12T20:30:38.541 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/leds/trigger/ledtrig-gpio.ko 2024-04-12T20:30:38.541 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/leds/trigger/ledtrig-heartbeat.ko 2024-04-12T20:30:38.541 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/leds/trigger/ledtrig-oneshot.ko 2024-04-12T20:30:38.541 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/leds/trigger/ledtrig-timer.ko 2024-04-12T20:30:38.541 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/leds/trigger/ledtrig-transient.ko 2024-04-12T20:30:38.541 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/md 2024-04-12T20:30:38.541 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/md/bcache 2024-04-12T20:30:38.541 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/md/bcache/bcache.ko 2024-04-12T20:30:38.542 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/md/dm-bio-prison.ko 2024-04-12T20:30:38.542 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/md/dm-cache-smq.ko 2024-04-12T20:30:38.542 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/md/dm-cache.ko 2024-04-12T20:30:38.542 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/md/dm-crypt.ko 2024-04-12T20:30:38.542 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/md/dm-delay.ko 2024-04-12T20:30:38.542 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/md/dm-flakey.ko 2024-04-12T20:30:38.542 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/md/dm-log-userspace.ko 2024-04-12T20:30:38.542 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/md/dm-multipath.ko 2024-04-12T20:30:38.542 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/md/dm-queue-length.ko 2024-04-12T20:30:38.542 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/md/dm-raid.ko 2024-04-12T20:30:38.542 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/md/dm-round-robin.ko 2024-04-12T20:30:38.542 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/md/dm-service-time.ko 2024-04-12T20:30:38.543 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/md/dm-switch.ko 2024-04-12T20:30:38.543 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/md/dm-thin-pool.ko 2024-04-12T20:30:38.543 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/md/dm-verity.ko 2024-04-12T20:30:38.543 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/md/persistent-data 2024-04-12T20:30:38.543 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/md/persistent-data/dm-persistent-data.ko 2024-04-12T20:30:38.543 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/md/raid0.ko 2024-04-12T20:30:38.543 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/md/raid1.ko 2024-04-12T20:30:38.543 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/md/raid10.ko 2024-04-12T20:30:38.543 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/md/raid456.ko 2024-04-12T20:30:38.543 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/message 2024-04-12T20:30:38.543 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/message/fusion 2024-04-12T20:30:38.543 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/message/fusion/mptbase.ko 2024-04-12T20:30:38.544 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/message/fusion/mptctl.ko 2024-04-12T20:30:38.544 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/message/fusion/mptfc.ko 2024-04-12T20:30:38.544 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/message/fusion/mptlan.ko 2024-04-12T20:30:38.544 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/message/fusion/mptsas.ko 2024-04-12T20:30:38.544 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/message/fusion/mptscsih.ko 2024-04-12T20:30:38.544 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/message/fusion/mptspi.ko 2024-04-12T20:30:38.544 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/mfd 2024-04-12T20:30:38.544 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/mfd/lpc_ich.ko 2024-04-12T20:30:38.544 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/mfd/lpc_sch.ko 2024-04-12T20:30:38.544 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/mfd/mfd-core.ko 2024-04-12T20:30:38.544 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/mfd/sm501.ko 2024-04-12T20:30:38.544 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/mfd/viperboard.ko 2024-04-12T20:30:38.544 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/mfd/vx855.ko 2024-04-12T20:30:38.545 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/mfd/wl1273-core.ko 2024-04-12T20:30:38.545 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc 2024-04-12T20:30:38.545 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/altera-stapl 2024-04-12T20:30:38.545 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/altera-stapl/altera-stapl.ko 2024-04-12T20:30:38.545 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/apds9802als.ko 2024-04-12T20:30:38.545 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/apds990x.ko 2024-04-12T20:30:38.545 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/bh1770glc.ko 2024-04-12T20:30:38.545 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/cb710 2024-04-12T20:30:38.545 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/cb710/cb710.ko 2024-04-12T20:30:38.545 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/eeprom 2024-04-12T20:30:38.545 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/eeprom/at24.ko 2024-04-12T20:30:38.546 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/eeprom/eeprom_93cx6.ko 2024-04-12T20:30:38.546 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/eeprom/max6875.ko 2024-04-12T20:30:38.546 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/enclosure.ko 2024-04-12T20:30:38.546 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/hpilo.ko 2024-04-12T20:30:38.546 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/isl29003.ko 2024-04-12T20:30:38.546 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/isl29020.ko 2024-04-12T20:30:38.546 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/lis3lv02d 2024-04-12T20:30:38.546 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/lis3lv02d/lis3lv02d.ko 2024-04-12T20:30:38.546 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/lis3lv02d/lis3lv02d_i2c.ko 2024-04-12T20:30:38.546 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/mei 2024-04-12T20:30:38.546 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/mei/mei-me.ko 2024-04-12T20:30:38.547 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/mei/mei-txe.ko 2024-04-12T20:30:38.547 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/mei/mei.ko 2024-04-12T20:30:38.547 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/sgi-gru 2024-04-12T20:30:38.547 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/sgi-gru/gru.ko 2024-04-12T20:30:38.547 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/sgi-xp 2024-04-12T20:30:38.547 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/sgi-xp/xp.ko 2024-04-12T20:30:38.547 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/sgi-xp/xpc.ko 2024-04-12T20:30:38.547 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/sgi-xp/xpnet.ko 2024-04-12T20:30:38.547 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/tifm_7xx1.ko 2024-04-12T20:30:38.547 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/tifm_core.ko 2024-04-12T20:30:38.547 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/tsl2550.ko 2024-04-12T20:30:38.547 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/vmw_balloon.ko 2024-04-12T20:30:38.548 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/vmw_vmci 2024-04-12T20:30:38.548 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/vmw_vmci/vmw_vmci.ko 2024-04-12T20:30:38.548 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net 2024-04-12T20:30:38.548 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/bonding 2024-04-12T20:30:38.548 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/bonding/bonding.ko 2024-04-12T20:30:38.548 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/dsa 2024-04-12T20:30:38.548 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/dsa/b53 2024-04-12T20:30:38.548 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/dsa/b53/b53_common.ko 2024-04-12T20:30:38.548 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/dsa/bcm-sf2.ko 2024-04-12T20:30:38.548 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/dsa/mv88e6060.ko 2024-04-12T20:30:38.548 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/dsa/mv88e6xxx 2024-04-12T20:30:38.548 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/dsa/mv88e6xxx/mv88e6xxx.ko 2024-04-12T20:30:38.549 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/dummy.ko 2024-04-12T20:30:38.549 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/eql.ko 2024-04-12T20:30:38.549 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet 2024-04-12T20:30:38.549 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/3com 2024-04-12T20:30:38.549 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/3com/3c574_cs.ko 2024-04-12T20:30:38.549 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/3com/3c589_cs.ko 2024-04-12T20:30:38.549 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/3com/3c59x.ko 2024-04-12T20:30:38.549 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/3com/typhoon.ko 2024-04-12T20:30:38.549 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/8390 2024-04-12T20:30:38.549 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/8390/8390.ko 2024-04-12T20:30:38.549 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/8390/axnet_cs.ko 2024-04-12T20:30:38.549 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/8390/ne2k-pci.ko 2024-04-12T20:30:38.549 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/8390/pcnet_cs.ko 2024-04-12T20:30:38.550 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/adaptec 2024-04-12T20:30:38.550 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/adaptec/starfire.ko 2024-04-12T20:30:38.550 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/agere 2024-04-12T20:30:38.550 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/agere/et131x.ko 2024-04-12T20:30:38.550 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/alteon 2024-04-12T20:30:38.550 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/alteon/acenic.ko 2024-04-12T20:30:38.550 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/altera 2024-04-12T20:30:38.550 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/altera/altera_tse.ko 2024-04-12T20:30:38.550 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/amd 2024-04-12T20:30:38.550 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/amd/amd8111e.ko 2024-04-12T20:30:38.550 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/amd/nmclan_cs.ko 2024-04-12T20:30:38.550 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/amd/pcnet32.ko 2024-04-12T20:30:38.551 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/atheros 2024-04-12T20:30:38.551 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/atheros/alx 2024-04-12T20:30:38.551 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/atheros/alx/alx.ko 2024-04-12T20:30:38.551 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/atheros/atl1c 2024-04-12T20:30:38.551 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/atheros/atl1c/atl1c.ko 2024-04-12T20:30:38.551 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/atheros/atl1e 2024-04-12T20:30:38.551 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/atheros/atl1e/atl1e.ko 2024-04-12T20:30:38.551 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/atheros/atlx 2024-04-12T20:30:38.551 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/atheros/atlx/atl1.ko 2024-04-12T20:30:38.551 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/atheros/atlx/atl2.ko 2024-04-12T20:30:38.551 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/broadcom 2024-04-12T20:30:38.551 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/broadcom/b44.ko 2024-04-12T20:30:38.551 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/broadcom/bnx2.ko 2024-04-12T20:30:38.552 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/broadcom/bnx2x 2024-04-12T20:30:38.552 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/broadcom/bnx2x/bnx2x.ko 2024-04-12T20:30:38.552 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/broadcom/cnic.ko 2024-04-12T20:30:38.552 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/broadcom/genet 2024-04-12T20:30:38.552 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/broadcom/genet/genet.ko 2024-04-12T20:30:38.552 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/broadcom/tg3.ko 2024-04-12T20:30:38.552 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/brocade 2024-04-12T20:30:38.552 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/brocade/bna 2024-04-12T20:30:38.552 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/brocade/bna/bna.ko 2024-04-12T20:30:38.552 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/chelsio 2024-04-12T20:30:38.552 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/chelsio/cxgb 2024-04-12T20:30:38.552 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/chelsio/cxgb/cxgb.ko 2024-04-12T20:30:38.553 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/chelsio/cxgb3 2024-04-12T20:30:38.553 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/chelsio/cxgb3/cxgb3.ko 2024-04-12T20:30:38.553 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/chelsio/cxgb4 2024-04-12T20:30:38.553 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/chelsio/cxgb4/cxgb4.ko 2024-04-12T20:30:38.553 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/chelsio/cxgb4vf 2024-04-12T20:30:38.553 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/chelsio/cxgb4vf/cxgb4vf.ko 2024-04-12T20:30:38.553 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/chelsio/libcxgb 2024-04-12T20:30:38.553 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/chelsio/libcxgb/libcxgb.ko 2024-04-12T20:30:38.553 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/cisco 2024-04-12T20:30:38.553 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/cisco/enic 2024-04-12T20:30:38.553 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/cisco/enic/enic.ko 2024-04-12T20:30:38.553 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/dec 2024-04-12T20:30:38.554 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/dec/tulip 2024-04-12T20:30:38.554 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/dec/tulip/de2104x.ko 2024-04-12T20:30:38.554 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/dec/tulip/dmfe.ko 2024-04-12T20:30:38.554 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/dec/tulip/tulip.ko 2024-04-12T20:30:38.554 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/dec/tulip/uli526x.ko 2024-04-12T20:30:38.554 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/dec/tulip/winbond-840.ko 2024-04-12T20:30:38.554 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/dec/tulip/xircom_cb.ko 2024-04-12T20:30:38.554 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/dlink 2024-04-12T20:30:38.554 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/dlink/dl2k.ko 2024-04-12T20:30:38.554 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/dlink/sundance.ko 2024-04-12T20:30:38.554 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/dnet.ko 2024-04-12T20:30:38.554 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/emulex 2024-04-12T20:30:38.555 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/emulex/benet 2024-04-12T20:30:38.555 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/emulex/benet/be2net.ko 2024-04-12T20:30:38.555 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/ethoc.ko 2024-04-12T20:30:38.555 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/fealnx.ko 2024-04-12T20:30:38.555 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/intel 2024-04-12T20:30:38.555 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/intel/e100.ko 2024-04-12T20:30:38.555 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/intel/e1000 2024-04-12T20:30:38.555 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/intel/e1000/e1000.ko 2024-04-12T20:30:38.555 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/intel/e1000e 2024-04-12T20:30:38.555 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/intel/e1000e/e1000e.ko 2024-04-12T20:30:38.555 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/intel/fm10k 2024-04-12T20:30:38.556 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/intel/fm10k/fm10k.ko 2024-04-12T20:30:38.556 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/intel/i40e 2024-04-12T20:30:38.556 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/intel/i40e/i40e.ko 2024-04-12T20:30:38.556 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/intel/iavf 2024-04-12T20:30:38.556 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/intel/iavf/iavf.ko 2024-04-12T20:30:38.556 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/intel/igb 2024-04-12T20:30:38.556 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/intel/igb/igb.ko 2024-04-12T20:30:38.556 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/intel/igbvf 2024-04-12T20:30:38.556 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/intel/igbvf/igbvf.ko 2024-04-12T20:30:38.556 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/intel/ixgbe 2024-04-12T20:30:38.556 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/intel/ixgbe/ixgbe.ko 2024-04-12T20:30:38.557 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/intel/ixgbevf 2024-04-12T20:30:38.557 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/intel/ixgbevf/ixgbevf.ko 2024-04-12T20:30:38.557 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/jme.ko 2024-04-12T20:30:38.557 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/marvell 2024-04-12T20:30:38.557 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/marvell/mvmdio.ko 2024-04-12T20:30:38.557 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/marvell/skge.ko 2024-04-12T20:30:38.557 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/marvell/sky2.ko 2024-04-12T20:30:38.557 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/mellanox 2024-04-12T20:30:38.557 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/mellanox/mlx4 2024-04-12T20:30:38.557 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_core.ko 2024-04-12T20:30:38.557 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_en.ko 2024-04-12T20:30:38.557 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/mellanox/mlx5 2024-04-12T20:30:38.557 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/mellanox/mlx5/core 2024-04-12T20:30:38.558 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.ko 2024-04-12T20:30:38.558 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/micrel 2024-04-12T20:30:38.558 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/micrel/ksz884x.ko 2024-04-12T20:30:38.558 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/myricom 2024-04-12T20:30:38.558 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/myricom/myri10ge 2024-04-12T20:30:38.558 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/myricom/myri10ge/myri10ge.ko 2024-04-12T20:30:38.558 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/natsemi 2024-04-12T20:30:38.558 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/natsemi/natsemi.ko 2024-04-12T20:30:38.558 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/natsemi/ns83820.ko 2024-04-12T20:30:38.558 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/neterion 2024-04-12T20:30:38.558 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/neterion/s2io.ko 2024-04-12T20:30:38.558 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/nvidia 2024-04-12T20:30:38.559 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/nvidia/forcedeth.ko 2024-04-12T20:30:38.559 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/packetengines 2024-04-12T20:30:38.559 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/packetengines/hamachi.ko 2024-04-12T20:30:38.559 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/packetengines/yellowfin.ko 2024-04-12T20:30:38.559 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/qlogic 2024-04-12T20:30:38.559 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/qlogic/netxen 2024-04-12T20:30:38.559 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/qlogic/netxen/netxen_nic.ko 2024-04-12T20:30:38.559 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/qlogic/qla3xxx.ko 2024-04-12T20:30:38.559 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/qlogic/qlcnic 2024-04-12T20:30:38.559 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/qlogic/qlcnic/qlcnic.ko 2024-04-12T20:30:38.559 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/rdc 2024-04-12T20:30:38.559 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/rdc/r6040.ko 2024-04-12T20:30:38.559 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/realtek 2024-04-12T20:30:38.560 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/realtek/8139cp.ko 2024-04-12T20:30:38.560 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/realtek/8139too.ko 2024-04-12T20:30:38.560 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/realtek/atp.ko 2024-04-12T20:30:38.560 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/realtek/r8169.ko 2024-04-12T20:30:38.560 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/rocker 2024-04-12T20:30:38.560 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/rocker/rocker.ko 2024-04-12T20:30:38.560 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/sfc 2024-04-12T20:30:38.560 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/sfc/sfc.ko 2024-04-12T20:30:38.560 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/silan 2024-04-12T20:30:38.560 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/silan/sc92031.ko 2024-04-12T20:30:38.560 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/sis 2024-04-12T20:30:38.560 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/sis/sis190.ko 2024-04-12T20:30:38.560 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/sis/sis900.ko 2024-04-12T20:30:38.561 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/smsc 2024-04-12T20:30:38.561 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/smsc/epic100.ko 2024-04-12T20:30:38.561 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/smsc/smc91c92_cs.ko 2024-04-12T20:30:38.561 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/smsc/smsc911x.ko 2024-04-12T20:30:38.561 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/smsc/smsc9420.ko 2024-04-12T20:30:38.561 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/stmicro 2024-04-12T20:30:38.561 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/stmicro/stmmac 2024-04-12T20:30:38.561 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-intel.ko 2024-04-12T20:30:38.561 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac.ko 2024-04-12T20:30:38.561 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/sun 2024-04-12T20:30:38.561 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/sun/cassini.ko 2024-04-12T20:30:38.561 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/sun/niu.ko 2024-04-12T20:30:38.561 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/sun/sungem.ko 2024-04-12T20:30:38.561 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/sun/sunhme.ko 2024-04-12T20:30:38.562 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/tehuti 2024-04-12T20:30:38.562 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/tehuti/tehuti.ko 2024-04-12T20:30:38.562 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/ti 2024-04-12T20:30:38.562 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/ti/tlan.ko 2024-04-12T20:30:38.562 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/via 2024-04-12T20:30:38.562 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/via/via-rhine.ko 2024-04-12T20:30:38.562 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/via/via-velocity.ko 2024-04-12T20:30:38.562 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/wiznet 2024-04-12T20:30:38.562 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/wiznet/w5100.ko 2024-04-12T20:30:38.562 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/wiznet/w5300.ko 2024-04-12T20:30:38.562 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/xircom 2024-04-12T20:30:38.562 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/xircom/xirc2ps_cs.ko 2024-04-12T20:30:38.562 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/geneve.ko 2024-04-12T20:30:38.563 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/hyperv 2024-04-12T20:30:38.563 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/hyperv/hv_netvsc.ko 2024-04-12T20:30:38.563 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ieee802154 2024-04-12T20:30:38.563 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ieee802154/fakelb.ko 2024-04-12T20:30:38.563 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ifb.ko 2024-04-12T20:30:38.563 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ipvlan 2024-04-12T20:30:38.563 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ipvlan/ipvlan.ko 2024-04-12T20:30:38.563 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/macvlan.ko 2024-04-12T20:30:38.563 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/macvtap.ko 2024-04-12T20:30:38.563 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/mdio 2024-04-12T20:30:38.563 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/mdio.ko 2024-04-12T20:30:38.563 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/mdio/mdio-bcm-unimac.ko 2024-04-12T20:30:38.563 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/mdio/mdio-bitbang.ko 2024-04-12T20:30:38.563 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/mdio/mdio-regmap.ko 2024-04-12T20:30:38.564 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/mii.ko 2024-04-12T20:30:38.564 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/net_failover.ko 2024-04-12T20:30:38.564 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/netconsole.ko 2024-04-12T20:30:38.564 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/nlmon.ko 2024-04-12T20:30:38.564 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/pcs 2024-04-12T20:30:38.564 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/pcs/pcs-lynx.ko 2024-04-12T20:30:38.564 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/pcs/pcs_xpcs.ko 2024-04-12T20:30:38.564 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/phy 2024-04-12T20:30:38.564 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/phy/amd.ko 2024-04-12T20:30:38.564 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/phy/bcm-phy-lib.ko 2024-04-12T20:30:38.564 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/phy/bcm-phy-ptp.ko 2024-04-12T20:30:38.564 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/phy/bcm7xxx.ko 2024-04-12T20:30:38.564 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/phy/bcm87xx.ko 2024-04-12T20:30:38.564 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/phy/broadcom.ko 2024-04-12T20:30:38.565 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/phy/cicada.ko 2024-04-12T20:30:38.565 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/phy/davicom.ko 2024-04-12T20:30:38.565 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/phy/dp83640.ko 2024-04-12T20:30:38.565 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/phy/et1011c.ko 2024-04-12T20:30:38.565 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/phy/icplus.ko 2024-04-12T20:30:38.565 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/phy/lxt.ko 2024-04-12T20:30:38.565 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/phy/marvell.ko 2024-04-12T20:30:38.565 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/phy/micrel.ko 2024-04-12T20:30:38.565 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/phy/national.ko 2024-04-12T20:30:38.565 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/phy/phylink.ko 2024-04-12T20:30:38.565 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/phy/qsemi.ko 2024-04-12T20:30:38.565 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/phy/realtek.ko 2024-04-12T20:30:38.565 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/phy/smsc.ko 2024-04-12T20:30:38.565 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/phy/ste10Xp.ko 2024-04-12T20:30:38.566 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/phy/vitesse.ko 2024-04-12T20:30:38.566 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ppp 2024-04-12T20:30:38.566 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ppp/bsd_comp.ko 2024-04-12T20:30:38.566 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ppp/ppp_async.ko 2024-04-12T20:30:38.566 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ppp/ppp_deflate.ko 2024-04-12T20:30:38.566 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ppp/ppp_generic.ko 2024-04-12T20:30:38.566 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ppp/ppp_mppe.ko 2024-04-12T20:30:38.566 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ppp/ppp_synctty.ko 2024-04-12T20:30:38.566 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ppp/pppoe.ko 2024-04-12T20:30:38.566 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ppp/pppox.ko 2024-04-12T20:30:38.566 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ppp/pptp.ko 2024-04-12T20:30:38.566 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/slip 2024-04-12T20:30:38.566 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/slip/slhc.ko 2024-04-12T20:30:38.567 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/slip/slip.ko 2024-04-12T20:30:38.567 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/sungem_phy.ko 2024-04-12T20:30:38.567 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/tap.ko 2024-04-12T20:30:38.567 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/team 2024-04-12T20:30:38.567 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/team/team.ko 2024-04-12T20:30:38.567 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/team/team_mode_activebackup.ko 2024-04-12T20:30:38.567 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/team/team_mode_broadcast.ko 2024-04-12T20:30:38.567 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/team/team_mode_loadbalance.ko 2024-04-12T20:30:38.567 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/team/team_mode_random.ko 2024-04-12T20:30:38.567 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/team/team_mode_roundrobin.ko 2024-04-12T20:30:38.567 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/tun.ko 2024-04-12T20:30:38.567 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/veth.ko 2024-04-12T20:30:38.568 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/virtio_net.ko 2024-04-12T20:30:38.568 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/vmxnet3 2024-04-12T20:30:38.568 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/vmxnet3/vmxnet3.ko 2024-04-12T20:30:38.568 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/vxlan 2024-04-12T20:30:38.568 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/vxlan/vxlan.ko 2024-04-12T20:30:38.568 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/xen-netback 2024-04-12T20:30:38.568 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/xen-netback/xen-netback.ko 2024-04-12T20:30:38.568 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/xen-netfront.ko 2024-04-12T20:30:38.568 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ntb 2024-04-12T20:30:38.568 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ntb/ntb.ko 2024-04-12T20:30:38.568 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/nvme 2024-04-12T20:30:38.568 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/nvme/host 2024-04-12T20:30:38.568 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/nvme/host/nvme-core.ko 2024-04-12T20:30:38.569 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/nvme/host/nvme.ko 2024-04-12T20:30:38.569 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/parport 2024-04-12T20:30:38.569 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/parport/parport.ko 2024-04-12T20:30:38.569 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/parport/parport_cs.ko 2024-04-12T20:30:38.569 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/parport/parport_pc.ko 2024-04-12T20:30:38.569 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/parport/parport_serial.ko 2024-04-12T20:30:38.569 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/pci 2024-04-12T20:30:38.569 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/pci/hotplug 2024-04-12T20:30:38.569 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/pci/hotplug/acpiphp_ibm.ko 2024-04-12T20:30:38.569 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/pci/pcie 2024-04-12T20:30:38.569 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/pci/pcie/aer_inject.ko 2024-04-12T20:30:38.569 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/pci/xen-pcifront.ko 2024-04-12T20:30:38.569 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/pcmcia 2024-04-12T20:30:38.569 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/pcmcia/i82092.ko 2024-04-12T20:30:38.570 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/pcmcia/pd6729.ko 2024-04-12T20:30:38.570 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/pcmcia/yenta_socket.ko 2024-04-12T20:30:38.570 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/pinctrl 2024-04-12T20:30:38.570 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/pinctrl/intel 2024-04-12T20:30:38.570 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/pinctrl/intel/pinctrl-cherryview.ko 2024-04-12T20:30:38.570 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/platform 2024-04-12T20:30:38.570 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/platform/x86 2024-04-12T20:30:38.570 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/platform/x86/acer-wmi.ko 2024-04-12T20:30:38.570 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/platform/x86/acerhdf.ko 2024-04-12T20:30:38.570 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/platform/x86/apple-gmux.ko 2024-04-12T20:30:38.570 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/platform/x86/asus-laptop.ko 2024-04-12T20:30:38.570 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/platform/x86/asus-nb-wmi.ko 2024-04-12T20:30:38.571 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/platform/x86/asus-wmi.ko 2024-04-12T20:30:38.571 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/platform/x86/classmate-laptop.ko 2024-04-12T20:30:38.571 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/platform/x86/eeepc-laptop.ko 2024-04-12T20:30:38.571 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/platform/x86/eeepc-wmi.ko 2024-04-12T20:30:38.571 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/platform/x86/fujitsu-laptop.ko 2024-04-12T20:30:38.571 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/platform/x86/fujitsu-tablet.ko 2024-04-12T20:30:38.571 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/platform/x86/hdaps.ko 2024-04-12T20:30:38.571 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/platform/x86/intel 2024-04-12T20:30:38.571 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/platform/x86/intel/intel-rst.ko 2024-04-12T20:30:38.571 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/platform/x86/intel_ips.ko 2024-04-12T20:30:38.571 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/platform/x86/msi-wmi.ko 2024-04-12T20:30:38.572 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/platform/x86/mxm-wmi.ko 2024-04-12T20:30:38.572 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/platform/x86/panasonic-laptop.ko 2024-04-12T20:30:38.572 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/platform/x86/samsung-laptop.ko 2024-04-12T20:30:38.572 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/platform/x86/samsung-q10.ko 2024-04-12T20:30:38.572 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/platform/x86/topstar-laptop.ko 2024-04-12T20:30:38.572 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/platform/x86/toshiba_bluetooth.ko 2024-04-12T20:30:38.572 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/platform/x86/toshiba_haps.ko 2024-04-12T20:30:38.572 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/platform/x86/wmi-bmof.ko 2024-04-12T20:30:38.572 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/platform/x86/wmi.ko 2024-04-12T20:30:38.572 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/powercap 2024-04-12T20:30:38.572 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/powercap/intel_rapl_common.ko 2024-04-12T20:30:38.572 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/powercap/intel_rapl_msr.ko 2024-04-12T20:30:38.573 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/pps 2024-04-12T20:30:38.573 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/pps/clients 2024-04-12T20:30:38.573 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/pps/clients/pps-gpio.ko 2024-04-12T20:30:38.573 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/pps/clients/pps-ldisc.ko 2024-04-12T20:30:38.573 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/pps/clients/pps_parport.ko 2024-04-12T20:30:38.573 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/pps/pps_core.ko 2024-04-12T20:30:38.573 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ptp 2024-04-12T20:30:38.573 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ptp/ptp.ko 2024-04-12T20:30:38.573 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ptp/ptp_kvm.ko 2024-04-12T20:30:38.573 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ras 2024-04-12T20:30:38.573 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ras/amd 2024-04-12T20:30:38.573 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ras/amd/atl 2024-04-12T20:30:38.574 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ras/amd/atl/amd_atl.ko 2024-04-12T20:30:38.574 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ras/amd/fmpm.ko 2024-04-12T20:30:38.574 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc 2024-04-12T20:30:38.574 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-bq32k.ko 2024-04-12T20:30:38.574 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-ds1286.ko 2024-04-12T20:30:38.574 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-ds1307.ko 2024-04-12T20:30:38.574 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-ds1374.ko 2024-04-12T20:30:38.574 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-ds1511.ko 2024-04-12T20:30:38.574 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-ds1553.ko 2024-04-12T20:30:38.574 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-ds1672.ko 2024-04-12T20:30:38.574 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-ds1742.ko 2024-04-12T20:30:38.574 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-ds2404.ko 2024-04-12T20:30:38.575 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-ds3232.ko 2024-04-12T20:30:38.575 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-em3027.ko 2024-04-12T20:30:38.575 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-fm3130.ko 2024-04-12T20:30:38.575 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-isl12022.ko 2024-04-12T20:30:38.575 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-isl1208.ko 2024-04-12T20:30:38.575 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-m41t80.ko 2024-04-12T20:30:38.575 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-m48t35.ko 2024-04-12T20:30:38.575 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-m48t59.ko 2024-04-12T20:30:38.575 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-max6900.ko 2024-04-12T20:30:38.575 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-msm6242.ko 2024-04-12T20:30:38.575 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-pcf2127.ko 2024-04-12T20:30:38.575 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-pcf85063.ko 2024-04-12T20:30:38.575 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-pcf8523.ko 2024-04-12T20:30:38.575 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-pcf8563.ko 2024-04-12T20:30:38.576 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-pcf8583.ko 2024-04-12T20:30:38.576 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-rp5c01.ko 2024-04-12T20:30:38.576 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-rs5c372.ko 2024-04-12T20:30:38.576 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-rv3029c2.ko 2024-04-12T20:30:38.576 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-rx8025.ko 2024-04-12T20:30:38.576 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-rx8581.ko 2024-04-12T20:30:38.576 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-stk17ta8.ko 2024-04-12T20:30:38.576 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-x1205.ko 2024-04-12T20:30:38.576 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi 2024-04-12T20:30:38.576 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/3w-9xxx.ko 2024-04-12T20:30:38.576 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/3w-sas.ko 2024-04-12T20:30:38.576 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/3w-xxxx.ko 2024-04-12T20:30:38.576 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/BusLogic.ko 2024-04-12T20:30:38.576 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/a100u2w.ko 2024-04-12T20:30:38.577 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/aacraid 2024-04-12T20:30:38.577 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/aacraid/aacraid.ko 2024-04-12T20:30:38.577 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/advansys.ko 2024-04-12T20:30:38.577 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/aic7xxx 2024-04-12T20:30:38.577 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/aic7xxx/aic79xx.ko 2024-04-12T20:30:38.577 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/aic7xxx/aic7xxx.ko 2024-04-12T20:30:38.577 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/am53c974.ko 2024-04-12T20:30:38.577 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/arcmsr 2024-04-12T20:30:38.577 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/arcmsr/arcmsr.ko 2024-04-12T20:30:38.577 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/atp870u.ko 2024-04-12T20:30:38.577 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/be2iscsi 2024-04-12T20:30:38.577 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/be2iscsi/be2iscsi.ko 2024-04-12T20:30:38.577 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/bfa 2024-04-12T20:30:38.577 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/bfa/bfa.ko 2024-04-12T20:30:38.578 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/bnx2fc 2024-04-12T20:30:38.578 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/bnx2fc/bnx2fc.ko 2024-04-12T20:30:38.578 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/bnx2i 2024-04-12T20:30:38.578 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/bnx2i/bnx2i.ko 2024-04-12T20:30:38.578 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/ch.ko 2024-04-12T20:30:38.578 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/csiostor 2024-04-12T20:30:38.578 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/csiostor/csiostor.ko 2024-04-12T20:30:38.578 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/cxgbi 2024-04-12T20:30:38.578 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/cxgbi/cxgb3i 2024-04-12T20:30:38.578 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/cxgbi/cxgb3i/cxgb3i.ko 2024-04-12T20:30:38.578 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/cxgbi/cxgb4i 2024-04-12T20:30:38.579 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/cxgbi/cxgb4i/cxgb4i.ko 2024-04-12T20:30:38.579 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/cxgbi/libcxgbi.ko 2024-04-12T20:30:38.579 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/dc395x.ko 2024-04-12T20:30:38.579 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/device_handler 2024-04-12T20:30:38.579 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/device_handler/scsi_dh_alua.ko 2024-04-12T20:30:38.579 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/device_handler/scsi_dh_emc.ko 2024-04-12T20:30:38.579 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/device_handler/scsi_dh_hp_sw.ko 2024-04-12T20:30:38.579 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/device_handler/scsi_dh_rdac.ko 2024-04-12T20:30:38.579 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/dmx3191d.ko 2024-04-12T20:30:38.579 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/esas2r 2024-04-12T20:30:38.579 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/esas2r/esas2r.ko 2024-04-12T20:30:38.579 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/esp_scsi.ko 2024-04-12T20:30:38.579 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/fcoe 2024-04-12T20:30:38.579 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/fcoe/fcoe.ko 2024-04-12T20:30:38.580 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/fcoe/libfcoe.ko 2024-04-12T20:30:38.580 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/fnic 2024-04-12T20:30:38.580 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/fnic/fnic.ko 2024-04-12T20:30:38.580 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/hpsa.ko 2024-04-12T20:30:38.580 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/hptiop.ko 2024-04-12T20:30:38.580 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/hv_storvsc.ko 2024-04-12T20:30:38.580 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/initio.ko 2024-04-12T20:30:38.580 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/ipr.ko 2024-04-12T20:30:38.580 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/ips.ko 2024-04-12T20:30:38.580 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/isci 2024-04-12T20:30:38.580 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/isci/isci.ko 2024-04-12T20:30:38.580 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/iscsi_boot_sysfs.ko 2024-04-12T20:30:38.580 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/iscsi_tcp.ko 2024-04-12T20:30:38.581 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/libfc 2024-04-12T20:30:38.581 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/libfc/libfc.ko 2024-04-12T20:30:38.581 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/libiscsi.ko 2024-04-12T20:30:38.581 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/libiscsi_tcp.ko 2024-04-12T20:30:38.581 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/libsas 2024-04-12T20:30:38.581 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/libsas/libsas.ko 2024-04-12T20:30:38.581 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/lpfc 2024-04-12T20:30:38.581 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/lpfc/lpfc.ko 2024-04-12T20:30:38.581 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/megaraid 2024-04-12T20:30:38.581 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/megaraid.ko 2024-04-12T20:30:38.581 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/megaraid/megaraid_mbox.ko 2024-04-12T20:30:38.581 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/megaraid/megaraid_mm.ko 2024-04-12T20:30:38.581 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/megaraid/megaraid_sas.ko 2024-04-12T20:30:38.581 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/mpt3sas 2024-04-12T20:30:38.582 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/mpt3sas/mpt3sas.ko 2024-04-12T20:30:38.582 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/mvsas 2024-04-12T20:30:38.582 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/mvsas/mvsas.ko 2024-04-12T20:30:38.582 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/mvumi.ko 2024-04-12T20:30:38.582 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/pm8001 2024-04-12T20:30:38.582 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/pm8001/pm80xx.ko 2024-04-12T20:30:38.582 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/pmcraid.ko 2024-04-12T20:30:38.582 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/qla1280.ko 2024-04-12T20:30:38.582 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/qla2xxx 2024-04-12T20:30:38.582 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/qla2xxx/qla2xxx.ko 2024-04-12T20:30:38.582 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/qla2xxx/tcm_qla2xxx.ko 2024-04-12T20:30:38.582 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/qla4xxx 2024-04-12T20:30:38.582 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/qla4xxx/qla4xxx.ko 2024-04-12T20:30:38.582 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/raid_class.ko 2024-04-12T20:30:38.583 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/scsi_debug.ko 2024-04-12T20:30:38.583 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/scsi_transport_fc.ko 2024-04-12T20:30:38.583 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/scsi_transport_iscsi.ko 2024-04-12T20:30:38.583 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/scsi_transport_sas.ko 2024-04-12T20:30:38.583 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/scsi_transport_spi.ko 2024-04-12T20:30:38.583 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/scsi_transport_srp.ko 2024-04-12T20:30:38.583 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/ses.ko 2024-04-12T20:30:38.583 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/st.ko 2024-04-12T20:30:38.583 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/stex.ko 2024-04-12T20:30:38.583 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/sym53c8xx_2 2024-04-12T20:30:38.583 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/sym53c8xx_2/sym53c8xx.ko 2024-04-12T20:30:38.583 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/virtio_scsi.ko 2024-04-12T20:30:38.583 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/vmw_pvscsi.ko 2024-04-12T20:30:38.583 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/wd719x.ko 2024-04-12T20:30:38.584 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ssb 2024-04-12T20:30:38.584 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ssb/ssb.ko 2024-04-12T20:30:38.584 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/target 2024-04-12T20:30:38.584 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/target/iscsi 2024-04-12T20:30:38.584 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/target/iscsi/iscsi_target_mod.ko 2024-04-12T20:30:38.584 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/target/loopback 2024-04-12T20:30:38.584 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/target/loopback/tcm_loop.ko 2024-04-12T20:30:38.584 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/target/target_core_file.ko 2024-04-12T20:30:38.584 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/target/target_core_iblock.ko 2024-04-12T20:30:38.584 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/target/target_core_mod.ko 2024-04-12T20:30:38.584 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/target/target_core_pscsi.ko 2024-04-12T20:30:38.584 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/target/target_core_user.ko 2024-04-12T20:30:38.585 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/target/tcm_fc 2024-04-12T20:30:38.585 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/target/tcm_fc/tcm_fc.ko 2024-04-12T20:30:38.585 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/thermal 2024-04-12T20:30:38.585 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/thermal/intel 2024-04-12T20:30:38.585 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/thermal/intel/int340x_thermal 2024-04-12T20:30:38.585 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/thermal/intel/int340x_thermal/acpi_thermal_rel.ko 2024-04-12T20:30:38.585 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/thermal/intel/int340x_thermal/int3400_thermal.ko 2024-04-12T20:30:38.585 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/thermal/intel/int340x_thermal/int3401_thermal.ko 2024-04-12T20:30:38.585 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/thermal/intel/int340x_thermal/int3402_thermal.ko 2024-04-12T20:30:38.585 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/thermal/intel/int340x_thermal/int3403_thermal.ko 2024-04-12T20:30:38.585 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/thermal/intel/int340x_thermal/int340x_thermal_zone.ko 2024-04-12T20:30:38.585 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device.ko 2024-04-12T20:30:38.585 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci.ko 2024-04-12T20:30:38.586 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci_legacy.ko 2024-04-12T20:30:38.586 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_mbox.ko 2024-04-12T20:30:38.586 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_power_floor.ko 2024-04-12T20:30:38.586 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rapl.ko 2024-04-12T20:30:38.586 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rfim.ko 2024-04-12T20:30:38.586 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_hint.ko 2024-04-12T20:30:38.586 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_req.ko 2024-04-12T20:30:38.586 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/thermal/intel/intel_soc_dts_iosf.ko 2024-04-12T20:30:38.586 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/thermal/intel/intel_soc_dts_thermal.ko 2024-04-12T20:30:38.586 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/thermal/intel/x86_pkg_temp_thermal.ko 2024-04-12T20:30:38.586 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/tty 2024-04-12T20:30:38.587 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/tty/ipwireless 2024-04-12T20:30:38.587 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/tty/ipwireless/ipwireless.ko 2024-04-12T20:30:38.587 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/tty/n_gsm.ko 2024-04-12T20:30:38.587 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/tty/n_hdlc.ko 2024-04-12T20:30:38.587 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/tty/nozomi.ko 2024-04-12T20:30:38.587 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/tty/serial 2024-04-12T20:30:38.587 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/tty/serial/8250 2024-04-12T20:30:38.587 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/tty/serial/8250/serial_cs.ko 2024-04-12T20:30:38.587 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/tty/serial/arc_uart.ko 2024-04-12T20:30:38.587 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/tty/serial/jsm 2024-04-12T20:30:38.587 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/tty/serial/jsm/jsm.ko 2024-04-12T20:30:38.587 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ufs 2024-04-12T20:30:38.587 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ufs/core 2024-04-12T20:30:38.588 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ufs/core/ufshcd-core.ko 2024-04-12T20:30:38.588 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ufs/host 2024-04-12T20:30:38.588 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ufs/host/ufshcd-pci.ko 2024-04-12T20:30:38.588 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/uio 2024-04-12T20:30:38.588 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/uio/uio.ko 2024-04-12T20:30:38.588 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/uio/uio_aec.ko 2024-04-12T20:30:38.588 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/uio/uio_cif.ko 2024-04-12T20:30:38.588 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/uio/uio_pci_generic.ko 2024-04-12T20:30:38.588 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/uio/uio_sercos3.ko 2024-04-12T20:30:38.588 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb 2024-04-12T20:30:38.588 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/class 2024-04-12T20:30:38.589 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/class/cdc-acm.ko 2024-04-12T20:30:38.589 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/class/cdc-wdm.ko 2024-04-12T20:30:38.589 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/class/usblp.ko 2024-04-12T20:30:38.589 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/class/usbtmc.ko 2024-04-12T20:30:38.589 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/host 2024-04-12T20:30:38.589 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/host/sl811-hcd.ko 2024-04-12T20:30:38.589 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/image 2024-04-12T20:30:38.589 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/image/mdc800.ko 2024-04-12T20:30:38.589 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/image/microtek.ko 2024-04-12T20:30:38.589 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/misc 2024-04-12T20:30:38.589 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/misc/adutux.ko 2024-04-12T20:30:38.589 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/misc/appledisplay.ko 2024-04-12T20:30:38.590 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/misc/emi26.ko 2024-04-12T20:30:38.590 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/misc/emi62.ko 2024-04-12T20:30:38.590 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/misc/ezusb.ko 2024-04-12T20:30:38.590 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/misc/idmouse.ko 2024-04-12T20:30:38.590 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/misc/iowarrior.ko 2024-04-12T20:30:38.590 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/misc/isight_firmware.ko 2024-04-12T20:30:38.590 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/misc/ldusb.ko 2024-04-12T20:30:38.590 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/misc/legousbtower.ko 2024-04-12T20:30:38.590 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/misc/sisusbvga 2024-04-12T20:30:38.590 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/misc/sisusbvga/sisusbvga.ko 2024-04-12T20:30:38.590 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/misc/trancevibrator.ko 2024-04-12T20:30:38.591 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/misc/usb3503.ko 2024-04-12T20:30:38.591 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/misc/usblcd.ko 2024-04-12T20:30:38.591 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/misc/usbsevseg.ko 2024-04-12T20:30:38.591 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/misc/uss720.ko 2024-04-12T20:30:38.591 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/misc/yurex.ko 2024-04-12T20:30:38.591 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/phy 2024-04-12T20:30:38.591 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/phy/phy-generic.ko 2024-04-12T20:30:38.591 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial 2024-04-12T20:30:38.591 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/aircable.ko 2024-04-12T20:30:38.591 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/ark3116.ko 2024-04-12T20:30:38.591 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/belkin_sa.ko 2024-04-12T20:30:38.592 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/ch341.ko 2024-04-12T20:30:38.592 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/cp210x.ko 2024-04-12T20:30:38.592 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/cyberjack.ko 2024-04-12T20:30:38.592 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/cypress_m8.ko 2024-04-12T20:30:38.592 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/digi_acceleport.ko 2024-04-12T20:30:38.592 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/empeg.ko 2024-04-12T20:30:38.592 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/ftdi_sio.ko 2024-04-12T20:30:38.592 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/garmin_gps.ko 2024-04-12T20:30:38.592 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/io_edgeport.ko 2024-04-12T20:30:38.592 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/io_ti.ko 2024-04-12T20:30:38.592 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/ipaq.ko 2024-04-12T20:30:38.592 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/ipw.ko 2024-04-12T20:30:38.593 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/ir-usb.ko 2024-04-12T20:30:38.593 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/iuu_phoenix.ko 2024-04-12T20:30:38.593 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/keyspan.ko 2024-04-12T20:30:38.593 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/keyspan_pda.ko 2024-04-12T20:30:38.593 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/kl5kusb105.ko 2024-04-12T20:30:38.593 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/kobil_sct.ko 2024-04-12T20:30:38.593 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/mct_u232.ko 2024-04-12T20:30:38.593 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/mos7720.ko 2024-04-12T20:30:38.593 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/mos7840.ko 2024-04-12T20:30:38.593 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/navman.ko 2024-04-12T20:30:38.593 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/omninet.ko 2024-04-12T20:30:38.593 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/opticon.ko 2024-04-12T20:30:38.593 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/option.ko 2024-04-12T20:30:38.594 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/oti6858.ko 2024-04-12T20:30:38.594 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/pl2303.ko 2024-04-12T20:30:38.594 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/qcaux.ko 2024-04-12T20:30:38.594 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/qcserial.ko 2024-04-12T20:30:38.594 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/quatech2.ko 2024-04-12T20:30:38.594 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/safe_serial.ko 2024-04-12T20:30:38.594 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/sierra.ko 2024-04-12T20:30:38.594 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/spcp8x5.ko 2024-04-12T20:30:38.594 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/ssu100.ko 2024-04-12T20:30:38.594 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/symbolserial.ko 2024-04-12T20:30:38.594 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/ti_usb_3410_5052.ko 2024-04-12T20:30:38.594 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/usb-serial-simple.ko 2024-04-12T20:30:38.594 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/usb_debug.ko 2024-04-12T20:30:38.595 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/usb_wwan.ko 2024-04-12T20:30:38.595 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/visor.ko 2024-04-12T20:30:38.595 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/whiteheat.ko 2024-04-12T20:30:38.595 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/xsens_mt.ko 2024-04-12T20:30:38.595 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/storage 2024-04-12T20:30:38.595 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/storage/uas.ko 2024-04-12T20:30:38.595 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/storage/ums-alauda.ko 2024-04-12T20:30:38.595 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/storage/ums-cypress.ko 2024-04-12T20:30:38.595 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/storage/ums-datafab.ko 2024-04-12T20:30:38.595 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/storage/ums-eneub6250.ko 2024-04-12T20:30:38.595 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/storage/ums-freecom.ko 2024-04-12T20:30:38.595 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/storage/ums-isd200.ko 2024-04-12T20:30:38.595 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/storage/ums-jumpshot.ko 2024-04-12T20:30:38.596 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/storage/ums-karma.ko 2024-04-12T20:30:38.596 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/storage/ums-onetouch.ko 2024-04-12T20:30:38.596 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/storage/ums-realtek.ko 2024-04-12T20:30:38.596 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/storage/ums-sddr09.ko 2024-04-12T20:30:38.596 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/storage/ums-sddr55.ko 2024-04-12T20:30:38.596 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/storage/ums-usbat.ko 2024-04-12T20:30:38.596 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/storage/usb-storage.ko 2024-04-12T20:30:38.596 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/usbip 2024-04-12T20:30:38.596 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/usbip/usbip-core.ko 2024-04-12T20:30:38.596 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/usbip/usbip-host.ko 2024-04-12T20:30:38.596 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/usbip/vhci-hcd.ko 2024-04-12T20:30:38.596 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/vfio 2024-04-12T20:30:38.597 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/vfio/pci 2024-04-12T20:30:38.597 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/vfio/pci/vfio-pci-core.ko 2024-04-12T20:30:38.597 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/vfio/pci/vfio-pci.ko 2024-04-12T20:30:38.597 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/vfio/vfio.ko 2024-04-12T20:30:38.597 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/vfio/vfio_iommu_type1.ko 2024-04-12T20:30:38.597 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/vhost 2024-04-12T20:30:38.597 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/vhost/vhost.ko 2024-04-12T20:30:38.597 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/vhost/vhost_iotlb.ko 2024-04-12T20:30:38.597 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/vhost/vhost_net.ko 2024-04-12T20:30:38.597 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/vhost/vhost_scsi.ko 2024-04-12T20:30:38.597 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/video 2024-04-12T20:30:38.597 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/video/backlight 2024-04-12T20:30:38.598 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/video/backlight/apple_bl.ko 2024-04-12T20:30:38.598 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/video/backlight/lcd.ko 2024-04-12T20:30:38.598 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/video/backlight/platform_lcd.ko 2024-04-12T20:30:38.598 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/virtio 2024-04-12T20:30:38.598 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/virtio/virtio.ko 2024-04-12T20:30:38.598 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/virtio/virtio_balloon.ko 2024-04-12T20:30:38.598 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/virtio/virtio_mmio.ko 2024-04-12T20:30:38.598 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/virtio/virtio_pci.ko 2024-04-12T20:30:38.598 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/virtio/virtio_pci_legacy_dev.ko 2024-04-12T20:30:38.598 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/virtio/virtio_pci_modern_dev.ko 2024-04-12T20:30:38.599 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/virtio/virtio_ring.ko 2024-04-12T20:30:38.599 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/w1 2024-04-12T20:30:38.599 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/w1/masters 2024-04-12T20:30:38.599 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/w1/masters/ds2482.ko 2024-04-12T20:30:38.599 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/w1/masters/ds2490.ko 2024-04-12T20:30:38.599 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/w1/slaves 2024-04-12T20:30:38.599 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/w1/slaves/w1_ds2406.ko 2024-04-12T20:30:38.599 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/w1/slaves/w1_ds2408.ko 2024-04-12T20:30:38.599 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/w1/slaves/w1_ds2413.ko 2024-04-12T20:30:38.599 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/w1/slaves/w1_ds2423.ko 2024-04-12T20:30:38.599 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/w1/slaves/w1_ds2431.ko 2024-04-12T20:30:38.599 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/w1/slaves/w1_ds2433.ko 2024-04-12T20:30:38.600 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/w1/slaves/w1_ds2780.ko 2024-04-12T20:30:38.600 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/w1/slaves/w1_ds2781.ko 2024-04-12T20:30:38.600 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/w1/slaves/w1_ds28e04.ko 2024-04-12T20:30:38.600 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/w1/slaves/w1_smem.ko 2024-04-12T20:30:38.600 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/w1/slaves/w1_therm.ko 2024-04-12T20:30:38.600 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/w1/wire.ko 2024-04-12T20:30:38.600 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/watchdog 2024-04-12T20:30:38.600 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/watchdog/alim1535_wdt.ko 2024-04-12T20:30:38.600 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/watchdog/alim7101_wdt.ko 2024-04-12T20:30:38.600 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/watchdog/f71808e_wdt.ko 2024-04-12T20:30:38.600 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/watchdog/hpwdt.ko 2024-04-12T20:30:38.600 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/watchdog/i6300esb.ko 2024-04-12T20:30:38.600 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/watchdog/iTCO_vendor_support.ko 2024-04-12T20:30:38.601 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/watchdog/iTCO_wdt.ko 2024-04-12T20:30:38.601 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/watchdog/ib700wdt.ko 2024-04-12T20:30:38.601 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/watchdog/ibmasr.ko 2024-04-12T20:30:38.601 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/watchdog/ie6xx_wdt.ko 2024-04-12T20:30:38.601 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/watchdog/it8712f_wdt.ko 2024-04-12T20:30:38.601 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/watchdog/it87_wdt.ko 2024-04-12T20:30:38.601 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/watchdog/machzwd.ko 2024-04-12T20:30:38.601 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/watchdog/nv_tco.ko 2024-04-12T20:30:38.601 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/watchdog/pcwd_pci.ko 2024-04-12T20:30:38.601 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/watchdog/pcwd_usb.ko 2024-04-12T20:30:38.601 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/watchdog/sbc_fitpc2_wdt.ko 2024-04-12T20:30:38.601 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/watchdog/sch311x_wdt.ko 2024-04-12T20:30:38.601 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/watchdog/softdog.ko 2024-04-12T20:30:38.602 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/watchdog/sp5100_tco.ko 2024-04-12T20:30:38.602 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/watchdog/via_wdt.ko 2024-04-12T20:30:38.602 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/watchdog/w83627hf_wdt.ko 2024-04-12T20:30:38.602 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/watchdog/w83877f_wdt.ko 2024-04-12T20:30:38.602 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/watchdog/w83977f_wdt.ko 2024-04-12T20:30:38.602 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/watchdog/wdt_pci.ko 2024-04-12T20:30:38.602 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/watchdog/xen_wdt.ko 2024-04-12T20:30:38.602 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/xen 2024-04-12T20:30:38.602 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/xen/xen-acpi-processor.ko 2024-04-12T20:30:38.602 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/xen/xen-evtchn.ko 2024-04-12T20:30:38.602 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/xen/xen-gntalloc.ko 2024-04-12T20:30:38.602 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/xen/xen-gntdev.ko 2024-04-12T20:30:38.602 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/xen/xen-pciback 2024-04-12T20:30:38.602 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/xen/xen-pciback/xen-pciback.ko 2024-04-12T20:30:38.603 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/xen/xen-privcmd.ko 2024-04-12T20:30:38.603 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/xen/xenfs 2024-04-12T20:30:38.603 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/xen/xenfs/xenfs.ko 2024-04-12T20:30:38.603 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs 2024-04-12T20:30:38.603 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/9p 2024-04-12T20:30:38.603 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/9p/9p.ko 2024-04-12T20:30:38.603 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/affs 2024-04-12T20:30:38.603 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/affs/affs.ko 2024-04-12T20:30:38.603 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/befs 2024-04-12T20:30:38.603 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/befs/befs.ko 2024-04-12T20:30:38.603 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/binfmt_misc.ko 2024-04-12T20:30:38.603 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/btrfs 2024-04-12T20:30:38.603 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/btrfs/btrfs.ko 2024-04-12T20:30:38.603 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/cachefiles 2024-04-12T20:30:38.604 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/cachefiles/cachefiles.ko 2024-04-12T20:30:38.604 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/ceph 2024-04-12T20:30:38.604 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/ceph/ceph.ko 2024-04-12T20:30:38.604 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/coda 2024-04-12T20:30:38.604 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/coda/coda.ko 2024-04-12T20:30:38.604 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/cramfs 2024-04-12T20:30:38.604 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/cramfs/cramfs.ko 2024-04-12T20:30:38.604 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/dlm 2024-04-12T20:30:38.604 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/dlm/dlm.ko 2024-04-12T20:30:38.604 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/ecryptfs 2024-04-12T20:30:38.604 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/ecryptfs/ecryptfs.ko 2024-04-12T20:30:38.604 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/f2fs 2024-04-12T20:30:38.604 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/f2fs/f2fs.ko 2024-04-12T20:30:38.605 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/fat 2024-04-12T20:30:38.605 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/fat/fat.ko 2024-04-12T20:30:38.605 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/fat/msdos.ko 2024-04-12T20:30:38.605 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/fat/vfat.ko 2024-04-12T20:30:38.605 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/fuse 2024-04-12T20:30:38.605 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/fuse/cuse.ko 2024-04-12T20:30:38.605 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/fuse/fuse.ko 2024-04-12T20:30:38.605 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/gfs2 2024-04-12T20:30:38.605 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/gfs2/gfs2.ko 2024-04-12T20:30:38.605 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/hfs 2024-04-12T20:30:38.605 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/hfs/hfs.ko 2024-04-12T20:30:38.605 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/hfsplus 2024-04-12T20:30:38.605 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/hfsplus/hfsplus.ko 2024-04-12T20:30:38.606 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/isofs 2024-04-12T20:30:38.606 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/isofs/isofs.ko 2024-04-12T20:30:38.606 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/jfs 2024-04-12T20:30:38.606 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/jfs/jfs.ko 2024-04-12T20:30:38.606 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/lockd 2024-04-12T20:30:38.606 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/lockd/lockd.ko 2024-04-12T20:30:38.606 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/minix 2024-04-12T20:30:38.606 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/minix/minix.ko 2024-04-12T20:30:38.606 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/netfs 2024-04-12T20:30:38.606 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/netfs/netfs.ko 2024-04-12T20:30:38.606 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nfs 2024-04-12T20:30:38.606 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nfs/blocklayout 2024-04-12T20:30:38.606 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nfs/blocklayout/blocklayoutdriver.ko 2024-04-12T20:30:38.606 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nfs/filelayout 2024-04-12T20:30:38.607 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nfs/filelayout/nfs_layout_nfsv41_files.ko 2024-04-12T20:30:38.607 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nfs/flexfilelayout 2024-04-12T20:30:38.607 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nfs/flexfilelayout/nfs_layout_flexfiles.ko 2024-04-12T20:30:38.607 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nfs/nfs.ko 2024-04-12T20:30:38.607 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nfs/nfsv3.ko 2024-04-12T20:30:38.607 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nfs/nfsv4.ko 2024-04-12T20:30:38.607 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nfs_common 2024-04-12T20:30:38.607 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nfs_common/grace.ko 2024-04-12T20:30:38.607 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nfs_common/nfs_acl.ko 2024-04-12T20:30:38.607 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nfsd 2024-04-12T20:30:38.607 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nfsd/nfsd.ko 2024-04-12T20:30:38.607 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nilfs2 2024-04-12T20:30:38.607 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nilfs2/nilfs2.ko 2024-04-12T20:30:38.608 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls 2024-04-12T20:30:38.608 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/mac-celtic.ko 2024-04-12T20:30:38.608 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/mac-centeuro.ko 2024-04-12T20:30:38.608 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/mac-croatian.ko 2024-04-12T20:30:38.608 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/mac-cyrillic.ko 2024-04-12T20:30:38.608 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/mac-gaelic.ko 2024-04-12T20:30:38.608 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/mac-greek.ko 2024-04-12T20:30:38.608 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/mac-iceland.ko 2024-04-12T20:30:38.608 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/mac-inuit.ko 2024-04-12T20:30:38.608 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/mac-roman.ko 2024-04-12T20:30:38.608 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/mac-romanian.ko 2024-04-12T20:30:38.608 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/mac-turkish.ko 2024-04-12T20:30:38.608 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_cp1250.ko 2024-04-12T20:30:38.608 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_cp1251.ko 2024-04-12T20:30:38.609 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_cp1255.ko 2024-04-12T20:30:38.609 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_cp737.ko 2024-04-12T20:30:38.609 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_cp775.ko 2024-04-12T20:30:38.609 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_cp850.ko 2024-04-12T20:30:38.609 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_cp852.ko 2024-04-12T20:30:38.609 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_cp855.ko 2024-04-12T20:30:38.609 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_cp857.ko 2024-04-12T20:30:38.609 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_cp860.ko 2024-04-12T20:30:38.609 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_cp861.ko 2024-04-12T20:30:38.609 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_cp862.ko 2024-04-12T20:30:38.609 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_cp863.ko 2024-04-12T20:30:38.610 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_cp864.ko 2024-04-12T20:30:38.610 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_cp865.ko 2024-04-12T20:30:38.610 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_cp866.ko 2024-04-12T20:30:38.610 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_cp869.ko 2024-04-12T20:30:38.610 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_cp874.ko 2024-04-12T20:30:38.610 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_cp932.ko 2024-04-12T20:30:38.610 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_cp936.ko 2024-04-12T20:30:38.610 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_cp949.ko 2024-04-12T20:30:38.610 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_cp950.ko 2024-04-12T20:30:38.610 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_euc-jp.ko 2024-04-12T20:30:38.610 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_iso8859-1.ko 2024-04-12T20:30:38.610 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_iso8859-13.ko 2024-04-12T20:30:38.611 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_iso8859-14.ko 2024-04-12T20:30:38.611 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_iso8859-15.ko 2024-04-12T20:30:38.611 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_iso8859-2.ko 2024-04-12T20:30:38.611 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_iso8859-3.ko 2024-04-12T20:30:38.611 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_iso8859-4.ko 2024-04-12T20:30:38.611 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_iso8859-5.ko 2024-04-12T20:30:38.611 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_iso8859-6.ko 2024-04-12T20:30:38.611 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_iso8859-7.ko 2024-04-12T20:30:38.611 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_iso8859-9.ko 2024-04-12T20:30:38.611 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_koi8-r.ko 2024-04-12T20:30:38.611 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_koi8-ru.ko 2024-04-12T20:30:38.611 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_koi8-u.ko 2024-04-12T20:30:38.612 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_ucs2_utils.ko 2024-04-12T20:30:38.612 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_utf8.ko 2024-04-12T20:30:38.612 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/ocfs2 2024-04-12T20:30:38.612 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/ocfs2/cluster 2024-04-12T20:30:38.612 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/ocfs2/cluster/ocfs2_nodemanager.ko 2024-04-12T20:30:38.612 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/ocfs2/dlm 2024-04-12T20:30:38.612 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/ocfs2/dlm/ocfs2_dlm.ko 2024-04-12T20:30:38.612 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/ocfs2/dlmfs 2024-04-12T20:30:38.612 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/ocfs2/dlmfs/ocfs2_dlmfs.ko 2024-04-12T20:30:38.612 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/ocfs2/ocfs2.ko 2024-04-12T20:30:38.612 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/ocfs2/ocfs2_stack_o2cb.ko 2024-04-12T20:30:38.612 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/ocfs2/ocfs2_stack_user.ko 2024-04-12T20:30:38.612 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/ocfs2/ocfs2_stackglue.ko 2024-04-12T20:30:38.613 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/overlayfs 2024-04-12T20:30:38.613 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/overlayfs/overlay.ko 2024-04-12T20:30:38.613 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/pstore 2024-04-12T20:30:38.613 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/pstore/ramoops.ko 2024-04-12T20:30:38.613 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/reiserfs 2024-04-12T20:30:38.613 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/reiserfs/reiserfs.ko 2024-04-12T20:30:38.613 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/romfs 2024-04-12T20:30:38.613 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/romfs/romfs.ko 2024-04-12T20:30:38.613 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/smb 2024-04-12T20:30:38.613 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/smb/client 2024-04-12T20:30:38.613 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/smb/client/cifs.ko 2024-04-12T20:30:38.613 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/smb/common 2024-04-12T20:30:38.613 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/smb/common/cifs_arc4.ko 2024-04-12T20:30:38.614 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/smb/common/cifs_md4.ko 2024-04-12T20:30:38.614 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/squashfs 2024-04-12T20:30:38.614 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/squashfs/squashfs.ko 2024-04-12T20:30:38.614 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/sysv 2024-04-12T20:30:38.614 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/sysv/sysv.ko 2024-04-12T20:30:38.614 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/udf 2024-04-12T20:30:38.614 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/udf/udf.ko 2024-04-12T20:30:38.614 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/ufs 2024-04-12T20:30:38.614 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/ufs/ufs.ko 2024-04-12T20:30:38.614 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/xfs 2024-04-12T20:30:38.614 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/xfs/xfs.ko 2024-04-12T20:30:38.615 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/kernel 2024-04-12T20:30:38.615 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/kernel/rcu 2024-04-12T20:30:38.615 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/kernel/rcu/rcutorture.ko 2024-04-12T20:30:38.615 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/kernel/torture.ko 2024-04-12T20:30:38.615 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/kernel/trace 2024-04-12T20:30:38.615 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/kernel/trace/ring_buffer_benchmark.ko 2024-04-12T20:30:38.615 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/lib 2024-04-12T20:30:38.615 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/lib/asn1_encoder.ko 2024-04-12T20:30:38.615 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/lib/crc-itu-t.ko 2024-04-12T20:30:38.615 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/lib/crc8.ko 2024-04-12T20:30:38.615 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/lib/crypto 2024-04-12T20:30:38.615 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/lib/crypto/libarc4.ko 2024-04-12T20:30:38.616 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/lib/crypto/libdes.ko 2024-04-12T20:30:38.616 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/lib/libcrc32c.ko 2024-04-12T20:30:38.616 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/lib/lru_cache.ko 2024-04-12T20:30:38.616 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/lib/lz4 2024-04-12T20:30:38.616 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/lib/lz4/lz4_compress.ko 2024-04-12T20:30:38.616 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/lib/lz4/lz4hc_compress.ko 2024-04-12T20:30:38.616 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/lib/math 2024-04-12T20:30:38.616 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/lib/math/cordic.ko 2024-04-12T20:30:38.616 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/lib/raid6 2024-04-12T20:30:38.616 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/lib/raid6/raid6_pq.ko 2024-04-12T20:30:38.616 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/lib/reed_solomon 2024-04-12T20:30:38.616 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/lib/reed_solomon/reed_solomon.ko 2024-04-12T20:30:38.617 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/lib/ts_bm.ko 2024-04-12T20:30:38.617 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/lib/ts_fsm.ko 2024-04-12T20:30:38.617 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/lib/ts_kmp.ko 2024-04-12T20:30:38.617 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/lib/zstd 2024-04-12T20:30:38.617 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/lib/zstd/zstd_compress.ko 2024-04-12T20:30:38.617 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/mm 2024-04-12T20:30:38.617 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/mm/hwpoison-inject.ko 2024-04-12T20:30:38.617 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net 2024-04-12T20:30:38.617 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/6lowpan 2024-04-12T20:30:38.617 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/6lowpan/6lowpan.ko 2024-04-12T20:30:38.617 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/6lowpan/nhc_dest.ko 2024-04-12T20:30:38.617 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/6lowpan/nhc_fragment.ko 2024-04-12T20:30:38.618 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/6lowpan/nhc_hop.ko 2024-04-12T20:30:38.618 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/6lowpan/nhc_ipv6.ko 2024-04-12T20:30:38.618 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/6lowpan/nhc_mobility.ko 2024-04-12T20:30:38.618 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/6lowpan/nhc_routing.ko 2024-04-12T20:30:38.618 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/6lowpan/nhc_udp.ko 2024-04-12T20:30:38.618 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/802 2024-04-12T20:30:38.618 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/802/garp.ko 2024-04-12T20:30:38.618 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/802/mrp.ko 2024-04-12T20:30:38.618 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/802/p8022.ko 2024-04-12T20:30:38.618 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/802/psnap.ko 2024-04-12T20:30:38.618 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/802/stp.ko 2024-04-12T20:30:38.619 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/8021q 2024-04-12T20:30:38.619 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/8021q/8021q.ko 2024-04-12T20:30:38.619 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/9p 2024-04-12T20:30:38.619 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/9p/9pnet.ko 2024-04-12T20:30:38.619 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/9p/9pnet_fd.ko 2024-04-12T20:30:38.619 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/9p/9pnet_rdma.ko 2024-04-12T20:30:38.619 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/9p/9pnet_virtio.ko 2024-04-12T20:30:38.619 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/batman-adv 2024-04-12T20:30:38.619 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/batman-adv/batman-adv.ko 2024-04-12T20:30:38.619 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/bridge 2024-04-12T20:30:38.619 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/bridge/br_netfilter.ko 2024-04-12T20:30:38.619 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/bridge/bridge.ko 2024-04-12T20:30:38.620 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/bridge/netfilter 2024-04-12T20:30:38.620 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/bridge/netfilter/ebt_802_3.ko 2024-04-12T20:30:38.620 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/bridge/netfilter/ebt_among.ko 2024-04-12T20:30:38.620 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/bridge/netfilter/ebt_arp.ko 2024-04-12T20:30:38.620 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/bridge/netfilter/ebt_arpreply.ko 2024-04-12T20:30:38.620 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/bridge/netfilter/ebt_dnat.ko 2024-04-12T20:30:38.620 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/bridge/netfilter/ebt_ip.ko 2024-04-12T20:30:38.620 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/bridge/netfilter/ebt_ip6.ko 2024-04-12T20:30:38.620 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/bridge/netfilter/ebt_limit.ko 2024-04-12T20:30:38.620 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/bridge/netfilter/ebt_log.ko 2024-04-12T20:30:38.620 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/bridge/netfilter/ebt_mark.ko 2024-04-12T20:30:38.620 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/bridge/netfilter/ebt_mark_m.ko 2024-04-12T20:30:38.620 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/bridge/netfilter/ebt_nflog.ko 2024-04-12T20:30:38.621 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/bridge/netfilter/ebt_pkttype.ko 2024-04-12T20:30:38.621 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/bridge/netfilter/ebt_redirect.ko 2024-04-12T20:30:38.621 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/bridge/netfilter/ebt_snat.ko 2024-04-12T20:30:38.621 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/bridge/netfilter/ebt_stp.ko 2024-04-12T20:30:38.621 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/bridge/netfilter/ebt_vlan.ko 2024-04-12T20:30:38.621 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/bridge/netfilter/ebtable_broute.ko 2024-04-12T20:30:38.621 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/bridge/netfilter/ebtable_filter.ko 2024-04-12T20:30:38.621 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/bridge/netfilter/ebtable_nat.ko 2024-04-12T20:30:38.621 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/bridge/netfilter/ebtables.ko 2024-04-12T20:30:38.621 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/bridge/netfilter/nf_conntrack_bridge.ko 2024-04-12T20:30:38.621 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/bridge/netfilter/nft_meta_bridge.ko 2024-04-12T20:30:38.621 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/bridge/netfilter/nft_reject_bridge.ko 2024-04-12T20:30:38.622 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ceph 2024-04-12T20:30:38.622 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ceph/libceph.ko 2024-04-12T20:30:38.622 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/core 2024-04-12T20:30:38.622 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/core/failover.ko 2024-04-12T20:30:38.622 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/core/pktgen.ko 2024-04-12T20:30:38.622 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/dccp 2024-04-12T20:30:38.622 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/dccp/dccp.ko 2024-04-12T20:30:38.622 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/dccp/dccp_diag.ko 2024-04-12T20:30:38.622 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/dccp/dccp_ipv4.ko 2024-04-12T20:30:38.622 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/dccp/dccp_ipv6.ko 2024-04-12T20:30:38.622 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/dns_resolver 2024-04-12T20:30:38.623 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/dns_resolver/dns_resolver.ko 2024-04-12T20:30:38.623 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/dsa 2024-04-12T20:30:38.623 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/dsa/dsa_core.ko 2024-04-12T20:30:38.623 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/dsa/tag_brcm.ko 2024-04-12T20:30:38.623 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/dsa/tag_dsa.ko 2024-04-12T20:30:38.623 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/dsa/tag_none.ko 2024-04-12T20:30:38.623 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/dsa/tag_trailer.ko 2024-04-12T20:30:38.623 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ieee802154 2024-04-12T20:30:38.623 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ieee802154/6lowpan 2024-04-12T20:30:38.623 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ieee802154/6lowpan/ieee802154_6lowpan.ko 2024-04-12T20:30:38.623 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ieee802154/ieee802154.ko 2024-04-12T20:30:38.623 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ieee802154/ieee802154_socket.ko 2024-04-12T20:30:38.624 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4 2024-04-12T20:30:38.624 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/ah4.ko 2024-04-12T20:30:38.624 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/esp4.ko 2024-04-12T20:30:38.624 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/fou.ko 2024-04-12T20:30:38.624 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/gre.ko 2024-04-12T20:30:38.624 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/inet_diag.ko 2024-04-12T20:30:38.624 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/ip_gre.ko 2024-04-12T20:30:38.624 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/ip_tunnel.ko 2024-04-12T20:30:38.624 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/ip_vti.ko 2024-04-12T20:30:38.624 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/ipcomp.ko 2024-04-12T20:30:38.624 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/ipip.ko 2024-04-12T20:30:38.624 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/netfilter 2024-04-12T20:30:38.625 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/netfilter/arp_tables.ko 2024-04-12T20:30:38.625 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/netfilter/arpt_mangle.ko 2024-04-12T20:30:38.625 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/netfilter/arptable_filter.ko 2024-04-12T20:30:38.625 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/netfilter/ipt_ECN.ko 2024-04-12T20:30:38.625 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/netfilter/ipt_SYNPROXY.ko 2024-04-12T20:30:38.625 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/netfilter/ipt_ah.ko 2024-04-12T20:30:38.625 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/netfilter/ipt_rpfilter.ko 2024-04-12T20:30:38.625 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/netfilter/iptable_mangle.ko 2024-04-12T20:30:38.625 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/netfilter/iptable_nat.ko 2024-04-12T20:30:38.625 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/netfilter/iptable_raw.ko 2024-04-12T20:30:38.625 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/netfilter/iptable_security.ko 2024-04-12T20:30:38.625 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/netfilter/nf_defrag_ipv4.ko 2024-04-12T20:30:38.626 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/netfilter/nf_dup_ipv4.ko 2024-04-12T20:30:38.626 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/netfilter/nf_nat_h323.ko 2024-04-12T20:30:38.626 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/netfilter/nf_nat_pptp.ko 2024-04-12T20:30:38.626 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/netfilter/nf_nat_snmp_basic.ko 2024-04-12T20:30:38.626 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/netfilter/nf_socket_ipv4.ko 2024-04-12T20:30:38.626 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/netfilter/nf_tproxy_ipv4.ko 2024-04-12T20:30:38.626 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/netfilter/nft_dup_ipv4.ko 2024-04-12T20:30:38.626 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/netfilter/nft_fib_ipv4.ko 2024-04-12T20:30:38.626 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/netfilter/nft_reject_ipv4.ko 2024-04-12T20:30:38.626 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/tcp_bic.ko 2024-04-12T20:30:38.626 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/tcp_dctcp.ko 2024-04-12T20:30:38.626 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/tcp_diag.ko 2024-04-12T20:30:38.627 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/tcp_highspeed.ko 2024-04-12T20:30:38.627 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/tcp_htcp.ko 2024-04-12T20:30:38.627 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/tcp_hybla.ko 2024-04-12T20:30:38.627 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/tcp_illinois.ko 2024-04-12T20:30:38.627 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/tcp_lp.ko 2024-04-12T20:30:38.627 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/tcp_scalable.ko 2024-04-12T20:30:38.627 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/tcp_vegas.ko 2024-04-12T20:30:38.627 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/tcp_veno.ko 2024-04-12T20:30:38.627 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/tcp_westwood.ko 2024-04-12T20:30:38.627 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/tcp_yeah.ko 2024-04-12T20:30:38.627 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/tunnel4.ko 2024-04-12T20:30:38.627 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/udp_diag.ko 2024-04-12T20:30:38.627 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/udp_tunnel.ko 2024-04-12T20:30:38.628 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/xfrm4_tunnel.ko 2024-04-12T20:30:38.628 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6 2024-04-12T20:30:38.628 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/ah6.ko 2024-04-12T20:30:38.628 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/esp6.ko 2024-04-12T20:30:38.628 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/fou6.ko 2024-04-12T20:30:38.628 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/ip6_tunnel.ko 2024-04-12T20:30:38.628 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/ip6_udp_tunnel.ko 2024-04-12T20:30:38.628 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/ip6_vti.ko 2024-04-12T20:30:38.628 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/ipcomp6.ko 2024-04-12T20:30:38.628 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/netfilter 2024-04-12T20:30:38.628 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/netfilter/ip6_tables.ko 2024-04-12T20:30:38.628 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/netfilter/ip6t_REJECT.ko 2024-04-12T20:30:38.629 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/netfilter/ip6t_SYNPROXY.ko 2024-04-12T20:30:38.629 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/netfilter/ip6t_ah.ko 2024-04-12T20:30:38.629 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/netfilter/ip6t_eui64.ko 2024-04-12T20:30:38.629 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/netfilter/ip6t_frag.ko 2024-04-12T20:30:38.629 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/netfilter/ip6t_hbh.ko 2024-04-12T20:30:38.629 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/netfilter/ip6t_ipv6header.ko 2024-04-12T20:30:38.629 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/netfilter/ip6t_mh.ko 2024-04-12T20:30:38.629 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/netfilter/ip6t_rpfilter.ko 2024-04-12T20:30:38.629 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/netfilter/ip6t_rt.ko 2024-04-12T20:30:38.629 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/netfilter/ip6table_filter.ko 2024-04-12T20:30:38.629 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/netfilter/ip6table_mangle.ko 2024-04-12T20:30:38.629 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/netfilter/ip6table_nat.ko 2024-04-12T20:30:38.629 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/netfilter/ip6table_raw.ko 2024-04-12T20:30:38.630 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/netfilter/ip6table_security.ko 2024-04-12T20:30:38.630 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/netfilter/nf_defrag_ipv6.ko 2024-04-12T20:30:38.630 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/netfilter/nf_dup_ipv6.ko 2024-04-12T20:30:38.630 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/netfilter/nf_reject_ipv6.ko 2024-04-12T20:30:38.630 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/netfilter/nf_socket_ipv6.ko 2024-04-12T20:30:38.630 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/netfilter/nf_tproxy_ipv6.ko 2024-04-12T20:30:38.630 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/netfilter/nft_dup_ipv6.ko 2024-04-12T20:30:38.630 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/netfilter/nft_fib_ipv6.ko 2024-04-12T20:30:38.630 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/netfilter/nft_reject_ipv6.ko 2024-04-12T20:30:38.630 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/sit.ko 2024-04-12T20:30:38.630 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/tunnel6.ko 2024-04-12T20:30:38.630 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/xfrm6_tunnel.ko 2024-04-12T20:30:38.630 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/key 2024-04-12T20:30:38.631 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/key/af_key.ko 2024-04-12T20:30:38.631 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/l2tp 2024-04-12T20:30:38.631 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/l2tp/l2tp_core.ko 2024-04-12T20:30:38.631 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/l2tp/l2tp_debugfs.ko 2024-04-12T20:30:38.631 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/l2tp/l2tp_eth.ko 2024-04-12T20:30:38.631 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/l2tp/l2tp_ip.ko 2024-04-12T20:30:38.631 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/l2tp/l2tp_ip6.ko 2024-04-12T20:30:38.631 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/l2tp/l2tp_netlink.ko 2024-04-12T20:30:38.631 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/l2tp/l2tp_ppp.ko 2024-04-12T20:30:38.631 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/llc 2024-04-12T20:30:38.631 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/llc/llc.ko 2024-04-12T20:30:38.631 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/mac802154 2024-04-12T20:30:38.631 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/mac802154/mac802154.ko 2024-04-12T20:30:38.631 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/mpls 2024-04-12T20:30:38.632 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/mpls/mpls_gso.ko 2024-04-12T20:30:38.632 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter 2024-04-12T20:30:38.632 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipset 2024-04-12T20:30:38.632 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipset/ip_set.ko 2024-04-12T20:30:38.632 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipset/ip_set_bitmap_ip.ko 2024-04-12T20:30:38.632 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipset/ip_set_bitmap_ipmac.ko 2024-04-12T20:30:38.632 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipset/ip_set_bitmap_port.ko 2024-04-12T20:30:38.632 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipset/ip_set_hash_ip.ko 2024-04-12T20:30:38.632 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipset/ip_set_hash_ipmark.ko 2024-04-12T20:30:38.632 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipset/ip_set_hash_ipport.ko 2024-04-12T20:30:38.632 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipset/ip_set_hash_ipportip.ko 2024-04-12T20:30:38.632 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipset/ip_set_hash_ipportnet.ko 2024-04-12T20:30:38.632 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipset/ip_set_hash_mac.ko 2024-04-12T20:30:38.633 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipset/ip_set_hash_net.ko 2024-04-12T20:30:38.633 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipset/ip_set_hash_netiface.ko 2024-04-12T20:30:38.633 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipset/ip_set_hash_netnet.ko 2024-04-12T20:30:38.633 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipset/ip_set_hash_netport.ko 2024-04-12T20:30:38.633 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipset/ip_set_hash_netportnet.ko 2024-04-12T20:30:38.633 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipset/ip_set_list_set.ko 2024-04-12T20:30:38.633 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipvs 2024-04-12T20:30:38.633 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipvs/ip_vs.ko 2024-04-12T20:30:38.633 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipvs/ip_vs_dh.ko 2024-04-12T20:30:38.633 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipvs/ip_vs_fo.ko 2024-04-12T20:30:38.633 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipvs/ip_vs_ftp.ko 2024-04-12T20:30:38.633 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipvs/ip_vs_lblc.ko 2024-04-12T20:30:38.633 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipvs/ip_vs_lblcr.ko 2024-04-12T20:30:38.633 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipvs/ip_vs_lc.ko 2024-04-12T20:30:38.634 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipvs/ip_vs_nq.ko 2024-04-12T20:30:38.634 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipvs/ip_vs_pe_sip.ko 2024-04-12T20:30:38.634 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipvs/ip_vs_rr.ko 2024-04-12T20:30:38.634 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipvs/ip_vs_sed.ko 2024-04-12T20:30:38.634 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipvs/ip_vs_sh.ko 2024-04-12T20:30:38.634 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipvs/ip_vs_wlc.ko 2024-04-12T20:30:38.634 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipvs/ip_vs_wrr.ko 2024-04-12T20:30:38.634 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nf_conncount.ko 2024-04-12T20:30:38.634 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nf_conntrack.ko 2024-04-12T20:30:38.634 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nf_conntrack_amanda.ko 2024-04-12T20:30:38.634 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nf_conntrack_broadcast.ko 2024-04-12T20:30:38.634 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nf_conntrack_ftp.ko 2024-04-12T20:30:38.635 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nf_conntrack_h323.ko 2024-04-12T20:30:38.635 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nf_conntrack_irc.ko 2024-04-12T20:30:38.635 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nf_conntrack_netbios_ns.ko 2024-04-12T20:30:38.635 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nf_conntrack_netlink.ko 2024-04-12T20:30:38.635 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nf_conntrack_pptp.ko 2024-04-12T20:30:38.635 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nf_conntrack_sane.ko 2024-04-12T20:30:38.635 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nf_conntrack_sip.ko 2024-04-12T20:30:38.635 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nf_conntrack_snmp.ko 2024-04-12T20:30:38.635 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nf_conntrack_tftp.ko 2024-04-12T20:30:38.635 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nf_dup_netdev.ko 2024-04-12T20:30:38.635 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nf_flow_table.ko 2024-04-12T20:30:38.635 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nf_flow_table_inet.ko 2024-04-12T20:30:38.635 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nf_log_syslog.ko 2024-04-12T20:30:38.635 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nf_nat.ko 2024-04-12T20:30:38.636 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nf_nat_amanda.ko 2024-04-12T20:30:38.636 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nf_nat_ftp.ko 2024-04-12T20:30:38.636 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nf_nat_irc.ko 2024-04-12T20:30:38.636 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nf_nat_sip.ko 2024-04-12T20:30:38.636 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nf_nat_tftp.ko 2024-04-12T20:30:38.636 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nf_synproxy_core.ko 2024-04-12T20:30:38.636 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nf_tables.ko 2024-04-12T20:30:38.636 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nfnetlink.ko 2024-04-12T20:30:38.636 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nfnetlink_acct.ko 2024-04-12T20:30:38.636 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nfnetlink_log.ko 2024-04-12T20:30:38.636 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nfnetlink_osf.ko 2024-04-12T20:30:38.636 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nfnetlink_queue.ko 2024-04-12T20:30:38.636 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nft_chain_nat.ko 2024-04-12T20:30:38.637 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nft_compat.ko 2024-04-12T20:30:38.637 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nft_ct.ko 2024-04-12T20:30:38.637 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nft_dup_netdev.ko 2024-04-12T20:30:38.637 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nft_fib.ko 2024-04-12T20:30:38.637 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nft_fwd_netdev.ko 2024-04-12T20:30:38.637 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nft_hash.ko 2024-04-12T20:30:38.637 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nft_limit.ko 2024-04-12T20:30:38.637 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nft_log.ko 2024-04-12T20:30:38.637 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nft_masq.ko 2024-04-12T20:30:38.637 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nft_nat.ko 2024-04-12T20:30:38.637 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nft_numgen.ko 2024-04-12T20:30:38.638 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nft_queue.ko 2024-04-12T20:30:38.638 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nft_quota.ko 2024-04-12T20:30:38.638 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nft_redir.ko 2024-04-12T20:30:38.638 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nft_reject.ko 2024-04-12T20:30:38.638 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nft_reject_inet.ko 2024-04-12T20:30:38.638 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nft_socket.ko 2024-04-12T20:30:38.638 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nft_tunnel.ko 2024-04-12T20:30:38.638 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_AUDIT.ko 2024-04-12T20:30:38.638 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_CHECKSUM.ko 2024-04-12T20:30:38.638 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_CLASSIFY.ko 2024-04-12T20:30:38.638 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_CONNSECMARK.ko 2024-04-12T20:30:38.639 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_CT.ko 2024-04-12T20:30:38.639 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_DSCP.ko 2024-04-12T20:30:38.639 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_HL.ko 2024-04-12T20:30:38.639 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_HMARK.ko 2024-04-12T20:30:38.639 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_IDLETIMER.ko 2024-04-12T20:30:38.639 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_LED.ko 2024-04-12T20:30:38.639 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_LOG.ko 2024-04-12T20:30:38.639 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_MASQUERADE.ko 2024-04-12T20:30:38.639 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_NETMAP.ko 2024-04-12T20:30:38.639 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_NFLOG.ko 2024-04-12T20:30:38.639 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_NFQUEUE.ko 2024-04-12T20:30:38.639 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_RATEEST.ko 2024-04-12T20:30:38.639 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_REDIRECT.ko 2024-04-12T20:30:38.640 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_SECMARK.ko 2024-04-12T20:30:38.640 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_TCPMSS.ko 2024-04-12T20:30:38.640 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_TCPOPTSTRIP.ko 2024-04-12T20:30:38.640 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_TEE.ko 2024-04-12T20:30:38.640 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_TPROXY.ko 2024-04-12T20:30:38.640 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_TRACE.ko 2024-04-12T20:30:38.640 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_addrtype.ko 2024-04-12T20:30:38.640 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_bpf.ko 2024-04-12T20:30:38.640 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_cgroup.ko 2024-04-12T20:30:38.640 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_cluster.ko 2024-04-12T20:30:38.640 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_comment.ko 2024-04-12T20:30:38.640 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_connbytes.ko 2024-04-12T20:30:38.640 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_connlabel.ko 2024-04-12T20:30:38.641 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_connlimit.ko 2024-04-12T20:30:38.641 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_connmark.ko 2024-04-12T20:30:38.641 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_conntrack.ko 2024-04-12T20:30:38.641 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_cpu.ko 2024-04-12T20:30:38.641 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_dccp.ko 2024-04-12T20:30:38.641 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_devgroup.ko 2024-04-12T20:30:38.641 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_dscp.ko 2024-04-12T20:30:38.641 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_ecn.ko 2024-04-12T20:30:38.641 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_esp.ko 2024-04-12T20:30:38.641 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_hashlimit.ko 2024-04-12T20:30:38.641 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_helper.ko 2024-04-12T20:30:38.641 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_hl.ko 2024-04-12T20:30:38.641 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_ipcomp.ko 2024-04-12T20:30:38.642 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_iprange.ko 2024-04-12T20:30:38.642 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_ipvs.ko 2024-04-12T20:30:38.642 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_l2tp.ko 2024-04-12T20:30:38.642 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_length.ko 2024-04-12T20:30:38.642 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_limit.ko 2024-04-12T20:30:38.642 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_mac.ko 2024-04-12T20:30:38.642 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_mark.ko 2024-04-12T20:30:38.642 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_multiport.ko 2024-04-12T20:30:38.642 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_nat.ko 2024-04-12T20:30:38.642 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_nfacct.ko 2024-04-12T20:30:38.642 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_osf.ko 2024-04-12T20:30:38.642 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_owner.ko 2024-04-12T20:30:38.642 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_physdev.ko 2024-04-12T20:30:38.643 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_pkttype.ko 2024-04-12T20:30:38.643 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_policy.ko 2024-04-12T20:30:38.643 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_quota.ko 2024-04-12T20:30:38.643 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_rateest.ko 2024-04-12T20:30:38.643 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_realm.ko 2024-04-12T20:30:38.643 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_recent.ko 2024-04-12T20:30:38.643 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_sctp.ko 2024-04-12T20:30:38.643 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_set.ko 2024-04-12T20:30:38.643 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_socket.ko 2024-04-12T20:30:38.643 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_state.ko 2024-04-12T20:30:38.643 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_statistic.ko 2024-04-12T20:30:38.643 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_string.ko 2024-04-12T20:30:38.643 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_tcpmss.ko 2024-04-12T20:30:38.644 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_time.ko 2024-04-12T20:30:38.644 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_u32.ko 2024-04-12T20:30:38.644 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netlink 2024-04-12T20:30:38.644 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netlink/netlink_diag.ko 2024-04-12T20:30:38.644 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/nsh 2024-04-12T20:30:38.644 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/nsh/nsh.ko 2024-04-12T20:30:38.644 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/openvswitch 2024-04-12T20:30:38.644 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/openvswitch/openvswitch.ko 2024-04-12T20:30:38.644 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/openvswitch/vport-geneve.ko 2024-04-12T20:30:38.644 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/openvswitch/vport-gre.ko 2024-04-12T20:30:38.644 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/openvswitch/vport-vxlan.ko 2024-04-12T20:30:38.644 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/packet 2024-04-12T20:30:38.644 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/packet/af_packet_diag.ko 2024-04-12T20:30:38.644 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/rds 2024-04-12T20:30:38.645 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/rds/rds.ko 2024-04-12T20:30:38.645 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/rds/rds_rdma.ko 2024-04-12T20:30:38.645 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/rds/rds_tcp.ko 2024-04-12T20:30:38.645 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched 2024-04-12T20:30:38.645 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/act_csum.ko 2024-04-12T20:30:38.645 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/act_gact.ko 2024-04-12T20:30:38.645 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/act_mirred.ko 2024-04-12T20:30:38.645 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/act_nat.ko 2024-04-12T20:30:38.645 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/act_pedit.ko 2024-04-12T20:30:38.645 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/act_police.ko 2024-04-12T20:30:38.645 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/act_simple.ko 2024-04-12T20:30:38.645 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/act_skbedit.ko 2024-04-12T20:30:38.645 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/act_vlan.ko 2024-04-12T20:30:38.645 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/cls_basic.ko 2024-04-12T20:30:38.646 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/cls_bpf.ko 2024-04-12T20:30:38.646 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/cls_flow.ko 2024-04-12T20:30:38.646 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/cls_fw.ko 2024-04-12T20:30:38.646 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/cls_route.ko 2024-04-12T20:30:38.646 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/cls_u32.ko 2024-04-12T20:30:38.646 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/em_cmp.ko 2024-04-12T20:30:38.646 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/em_ipset.ko 2024-04-12T20:30:38.646 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/em_meta.ko 2024-04-12T20:30:38.646 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/em_nbyte.ko 2024-04-12T20:30:38.646 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/em_text.ko 2024-04-12T20:30:38.646 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/em_u32.ko 2024-04-12T20:30:38.646 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/sch_choke.ko 2024-04-12T20:30:38.646 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/sch_codel.ko 2024-04-12T20:30:38.647 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/sch_drr.ko 2024-04-12T20:30:38.647 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/sch_fq.ko 2024-04-12T20:30:38.647 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/sch_gred.ko 2024-04-12T20:30:38.647 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/sch_hfsc.ko 2024-04-12T20:30:38.647 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/sch_hhf.ko 2024-04-12T20:30:38.647 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/sch_htb.ko 2024-04-12T20:30:38.647 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/sch_ingress.ko 2024-04-12T20:30:38.647 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/sch_mqprio.ko 2024-04-12T20:30:38.647 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/sch_mqprio_lib.ko 2024-04-12T20:30:38.647 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/sch_multiq.ko 2024-04-12T20:30:38.647 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/sch_netem.ko 2024-04-12T20:30:38.647 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/sch_pie.ko 2024-04-12T20:30:38.647 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/sch_plug.ko 2024-04-12T20:30:38.648 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/sch_prio.ko 2024-04-12T20:30:38.648 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/sch_qfq.ko 2024-04-12T20:30:38.648 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/sch_red.ko 2024-04-12T20:30:38.648 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/sch_sfb.ko 2024-04-12T20:30:38.648 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/sch_sfq.ko 2024-04-12T20:30:38.648 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/sch_tbf.ko 2024-04-12T20:30:38.648 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/sch_teql.ko 2024-04-12T20:30:38.648 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sctp 2024-04-12T20:30:38.648 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sctp/sctp.ko 2024-04-12T20:30:38.648 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sctp/sctp_diag.ko 2024-04-12T20:30:38.648 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sunrpc 2024-04-12T20:30:38.648 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sunrpc/auth_gss 2024-04-12T20:30:38.648 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sunrpc/auth_gss/auth_rpcgss.ko 2024-04-12T20:30:38.648 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sunrpc/auth_gss/rpcsec_gss_krb5.ko 2024-04-12T20:30:38.649 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sunrpc/sunrpc.ko 2024-04-12T20:30:38.649 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sunrpc/xprtrdma 2024-04-12T20:30:38.649 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sunrpc/xprtrdma/rpcrdma.ko 2024-04-12T20:30:38.649 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/tipc 2024-04-12T20:30:38.649 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/tipc/tipc.ko 2024-04-12T20:30:38.649 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/tipc/tipc_diag.ko 2024-04-12T20:30:38.649 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/unix 2024-04-12T20:30:38.649 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/unix/unix_diag.ko 2024-04-12T20:30:38.649 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/vmw_vsock 2024-04-12T20:30:38.649 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/vmw_vsock/vmw_vsock_virtio_transport_common.ko 2024-04-12T20:30:38.649 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/vmw_vsock/vmw_vsock_vmci_transport.ko 2024-04-12T20:30:38.649 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/vmw_vsock/vsock.ko 2024-04-12T20:30:38.649 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/vmw_vsock/vsock_diag.ko 2024-04-12T20:30:38.649 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/vmw_vsock/vsock_loopback.ko 2024-04-12T20:30:38.650 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/xfrm 2024-04-12T20:30:38.650 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/xfrm/xfrm_ipcomp.ko 2024-04-12T20:30:38.650 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/security 2024-04-12T20:30:38.650 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/security/keys 2024-04-12T20:30:38.650 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/security/keys/encrypted-keys 2024-04-12T20:30:38.650 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/security/keys/encrypted-keys/encrypted-keys.ko 2024-04-12T20:30:38.650 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/security/keys/trusted-keys 2024-04-12T20:30:38.650 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/security/keys/trusted-keys/trusted.ko 2024-04-12T20:30:38.650 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/modules.alias 2024-04-12T20:30:38.650 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/modules.alias.bin 2024-04-12T20:30:38.650 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/modules.builtin 2024-04-12T20:30:38.650 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/modules.builtin.alias.bin 2024-04-12T20:30:38.651 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/modules.builtin.bin 2024-04-12T20:30:38.651 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/modules.builtin.modinfo 2024-04-12T20:30:38.651 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/modules.dep 2024-04-12T20:30:38.651 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/modules.dep.bin 2024-04-12T20:30:38.651 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/modules.devname 2024-04-12T20:30:38.651 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/modules.order 2024-04-12T20:30:38.651 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/modules.softdep 2024-04-12T20:30:38.651 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/modules.symbols 2024-04-12T20:30:38.651 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/modules.symbols.bin 2024-04-12T20:30:38.651 INFO:teuthology.orchestra.run.smithi186.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/vmlinuz 2024-04-12T20:30:38.654 DEBUG:teuthology.task.kernel:get_image_version: 6.9.0-rc3-g3f60ed14f7c1 2024-04-12T20:30:38.654 DEBUG:teuthology.orchestra.run.smithi186:> rpm --scripts -qp /tmp/kernel.x86_64.rpm 2024-04-12T20:30:38.676 INFO:teuthology.orchestra.run.smithi186.stdout:postinstall scriptlet (using /bin/sh): 2024-04-12T20:30:38.676 INFO:teuthology.orchestra.run.smithi186.stdout:if [ -x /usr/bin/kernel-install ]; then 2024-04-12T20:30:38.677 INFO:teuthology.orchestra.run.smithi186.stdout: /usr/bin/kernel-install add 6.9.0-rc3-g3f60ed14f7c1 /lib/modules/6.9.0-rc3-g3f60ed14f7c1/vmlinuz 2024-04-12T20:30:38.677 INFO:teuthology.orchestra.run.smithi186.stdout:fi 2024-04-12T20:30:38.677 INFO:teuthology.orchestra.run.smithi186.stdout:for file in vmlinuz System.map config; do 2024-04-12T20:30:38.677 INFO:teuthology.orchestra.run.smithi186.stdout: if ! cmp --silent "/lib/modules/6.9.0-rc3-g3f60ed14f7c1/${file}" "/boot/${file}-6.9.0-rc3-g3f60ed14f7c1"; then 2024-04-12T20:30:38.677 INFO:teuthology.orchestra.run.smithi186.stdout: cp "/lib/modules/6.9.0-rc3-g3f60ed14f7c1/${file}" "/boot/${file}-6.9.0-rc3-g3f60ed14f7c1" 2024-04-12T20:30:38.677 INFO:teuthology.orchestra.run.smithi186.stdout: fi 2024-04-12T20:30:38.677 INFO:teuthology.orchestra.run.smithi186.stdout:done 2024-04-12T20:30:38.677 INFO:teuthology.orchestra.run.smithi186.stdout:if [ -d "/lib/modules/6.9.0-rc3-g3f60ed14f7c1/dtb" ] && \ 2024-04-12T20:30:38.677 INFO:teuthology.orchestra.run.smithi186.stdout: ! diff -rq "/lib/modules/6.9.0-rc3-g3f60ed14f7c1/dtb" "/boot/dtb-6.9.0-rc3-g3f60ed14f7c1" >/dev/null 2>&1; then 2024-04-12T20:30:38.677 INFO:teuthology.orchestra.run.smithi186.stdout: rm -rf "/boot/dtb-6.9.0-rc3-g3f60ed14f7c1" 2024-04-12T20:30:38.677 INFO:teuthology.orchestra.run.smithi186.stdout: cp -r "/lib/modules/6.9.0-rc3-g3f60ed14f7c1/dtb" "/boot/dtb-6.9.0-rc3-g3f60ed14f7c1" 2024-04-12T20:30:38.677 INFO:teuthology.orchestra.run.smithi186.stdout:fi 2024-04-12T20:30:38.677 INFO:teuthology.orchestra.run.smithi186.stdout:if [ ! -e "/lib/modules/6.9.0-rc3-g3f60ed14f7c1/modules.dep" ]; then 2024-04-12T20:30:38.678 INFO:teuthology.orchestra.run.smithi186.stdout: /usr/sbin/depmod 6.9.0-rc3-g3f60ed14f7c1 2024-04-12T20:30:38.678 INFO:teuthology.orchestra.run.smithi186.stdout:fi 2024-04-12T20:30:38.678 INFO:teuthology.orchestra.run.smithi186.stdout:preuninstall scriptlet (using /bin/sh): 2024-04-12T20:30:38.678 INFO:teuthology.orchestra.run.smithi186.stdout:if [ -x /usr/bin/kernel-install ]; then 2024-04-12T20:30:38.678 INFO:teuthology.orchestra.run.smithi186.stdout:kernel-install remove 6.9.0-rc3-g3f60ed14f7c1 2024-04-12T20:30:38.678 INFO:teuthology.orchestra.run.smithi186.stdout:fi 2024-04-12T20:30:38.679 DEBUG:teuthology.orchestra.run.smithi186:> sudo rpm -qi grub2-tools 2024-04-12T20:30:38.758 INFO:teuthology.orchestra.run.smithi186.stdout:Name : grub2-tools 2024-04-12T20:30:38.758 INFO:teuthology.orchestra.run.smithi186.stdout:Epoch : 1 2024-04-12T20:30:38.758 INFO:teuthology.orchestra.run.smithi186.stdout:Version : 2.06 2024-04-12T20:30:38.758 INFO:teuthology.orchestra.run.smithi186.stdout:Release : 68.el9 2024-04-12T20:30:38.758 INFO:teuthology.orchestra.run.smithi186.stdout:Architecture: x86_64 2024-04-12T20:30:38.758 INFO:teuthology.orchestra.run.smithi186.stdout:Install Date: Fri 01 Sep 2023 09:14:30 PM UTC 2024-04-12T20:30:38.758 INFO:teuthology.orchestra.run.smithi186.stdout:Group : Unspecified 2024-04-12T20:30:38.759 INFO:teuthology.orchestra.run.smithi186.stdout:Size : 8264254 2024-04-12T20:30:38.759 INFO:teuthology.orchestra.run.smithi186.stdout:License : GPLv3+ 2024-04-12T20:30:38.759 INFO:teuthology.orchestra.run.smithi186.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2024-04-12T20:30:38.759 INFO:teuthology.orchestra.run.smithi186.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2024-04-12T20:30:38.759 INFO:teuthology.orchestra.run.smithi186.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2024-04-12T20:30:38.759 INFO:teuthology.orchestra.run.smithi186.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-04-12T20:30:38.759 INFO:teuthology.orchestra.run.smithi186.stdout:Packager : builder@centos.org 2024-04-12T20:30:38.759 INFO:teuthology.orchestra.run.smithi186.stdout:Vendor : CentOS 2024-04-12T20:30:38.759 INFO:teuthology.orchestra.run.smithi186.stdout:URL : http://www.gnu.org/software/grub/ 2024-04-12T20:30:38.759 INFO:teuthology.orchestra.run.smithi186.stdout:Summary : Support tools for GRUB. 2024-04-12T20:30:38.759 INFO:teuthology.orchestra.run.smithi186.stdout:Description : 2024-04-12T20:30:38.759 INFO:teuthology.orchestra.run.smithi186.stdout: 2024-04-12T20:30:38.760 INFO:teuthology.orchestra.run.smithi186.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-04-12T20:30:38.760 INFO:teuthology.orchestra.run.smithi186.stdout:customizable bootloader with modular architecture. It supports a rich 2024-04-12T20:30:38.760 INFO:teuthology.orchestra.run.smithi186.stdout:variety of kernel formats, file systems, computer architectures and 2024-04-12T20:30:38.760 INFO:teuthology.orchestra.run.smithi186.stdout:hardware devices. 2024-04-12T20:30:38.760 INFO:teuthology.orchestra.run.smithi186.stdout: 2024-04-12T20:30:38.760 INFO:teuthology.orchestra.run.smithi186.stdout:This subpackage provides tools for support of all platforms. 2024-04-12T20:30:38.761 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-04-12T20:30:38.761 INFO:teuthology.task.kernel:Updating grub on smithi186 to boot 6.9.0-rc3-g3f60ed14f7c1 2024-04-12T20:30:38.761 DEBUG:teuthology.orchestra.run.smithi186:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-04-12T20:30:39.377 INFO:teuthology.orchestra.run.smithi186.stderr:Generating grub configuration file ... 2024-04-12T20:30:40.793 INFO:teuthology.orchestra.run.smithi186.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-04-12T20:30:40.831 INFO:teuthology.orchestra.run.smithi186.stderr:done 2024-04-12T20:30:40.833 DEBUG:teuthology.orchestra.run.smithi186:> mktemp 2024-04-12T20:30:40.848 INFO:teuthology.orchestra.run.smithi186.stdout:/tmp/tmp.ZAtXa7n0bp 2024-04-12T20:30:40.848 DEBUG:teuthology.orchestra.run.smithi186:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.ZAtXa7n0bp 2024-04-12T20:30:40.917 DEBUG:teuthology.orchestra.run.smithi186:> sudo chmod 0666 /tmp/tmp.ZAtXa7n0bp 2024-04-12T20:30:41.060 DEBUG:teuthology.orchestra.remote:smithi186:/tmp/tmp.ZAtXa7n0bp is 6KB 2024-04-12T20:30:41.108 DEBUG:teuthology.orchestra.run.smithi186:> rm -fr /tmp/tmp.ZAtXa7n0bp 2024-04-12T20:30:41.123 DEBUG:teuthology.orchestra.run.smithi186:> sudo /bin/ls /boot/loader/entries || true 2024-04-12T20:30:41.188 INFO:teuthology.orchestra.run.smithi186.stdout:15949e3c88704300882482eddfd7baae-0-rescue.conf 2024-04-12T20:30:41.188 INFO:teuthology.orchestra.run.smithi186.stdout:15949e3c88704300882482eddfd7baae-5.14.0-239.el9.x86_64.conf 2024-04-12T20:30:41.188 INFO:teuthology.orchestra.run.smithi186.stdout:15949e3c88704300882482eddfd7baae-5.14.0-361.el9.x86_64.conf 2024-04-12T20:30:41.188 INFO:teuthology.orchestra.run.smithi186.stdout:15949e3c88704300882482eddfd7baae-6.9.0-rc3-g3f60ed14f7c1.conf 2024-04-12T20:30:41.190 DEBUG:teuthology.orchestra.run.smithi186:> sudo grub2-set-default 15949e3c88704300882482eddfd7baae-6.9.0-rc3-g3f60ed14f7c1 2024-04-12T20:30:41.336 DEBUG:teuthology.orchestra.run.smithi186:> sudo shutdown -r now 2024-04-12T20:30:59.666 INFO:teuthology.task.kernel:Checking client host.a for new kernel version... 2024-04-12T20:30:59.667 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi018.front.sepia.ceph.com' 2024-04-12T20:30:59.667 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi018.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-12T20:31:11.350 INFO:teuthology.task.kernel:Checking client host.b for new kernel version... 2024-04-12T20:31:11.351 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi186.front.sepia.ceph.com' 2024-04-12T20:31:11.351 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi186.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-12T20:31:45.951 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.186 2024-04-12T20:31:54.961 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi186.front.sepia.ceph.com' 2024-04-12T20:31:54.962 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi186.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-12T20:31:59.671 DEBUG:teuthology.orchestra.remote:timed out 2024-04-12T20:32:08.678 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi018.front.sepia.ceph.com' 2024-04-12T20:32:08.679 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi018.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-12T20:32:10.081 DEBUG:teuthology.orchestra.run.smithi018:> true 2024-04-12T20:32:10.963 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi018.front.sepia.ceph.com' 2024-04-12T20:32:10.964 INFO:teuthology.task.kernel:Checking kernel version of host.a, want "6.9.0-rc3-g3f60ed14f7c1"... 2024-04-12T20:32:10.964 DEBUG:teuthology.orchestra.run.smithi018:> uname -r 2024-04-12T20:32:11.026 INFO:teuthology.orchestra.run.smithi018.stdout:6.9.0-rc3-g3f60ed14f7c1 2024-04-12T20:32:11.026 DEBUG:teuthology.task.kernel:current kernel version is 6.9.0-rc3-g3f60ed14f7c1 vs 6.9.0-rc3-g3f60ed14f7c1 2024-04-12T20:32:11.026 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-04-12T20:32:11.026 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-04-12T20:32:12.027 INFO:teuthology.task.kernel:Enabling kdb on host.a... 2024-04-12T20:32:12.027 DEBUG:teuthology.orchestra.run.smithi018:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-04-12T20:32:12.158 INFO:teuthology.orchestra.run.smithi018.stdout:ttyS1 2024-04-12T20:32:12.209 DEBUG:teuthology.parallel:result is None 2024-04-12T20:32:13.346 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.186 2024-04-12T20:32:25.347 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi186.front.sepia.ceph.com' 2024-04-12T20:32:25.348 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi186.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-12T20:32:28.414 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.186 2024-04-12T20:32:43.415 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi186.front.sepia.ceph.com' 2024-04-12T20:32:43.416 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi186.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-12T20:32:43.805 DEBUG:teuthology.orchestra.run.smithi186:> true 2024-04-12T20:32:44.398 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi186.front.sepia.ceph.com' 2024-04-12T20:32:44.398 INFO:teuthology.task.kernel:Checking kernel version of host.b, want "6.9.0-rc3-g3f60ed14f7c1"... 2024-04-12T20:32:44.399 DEBUG:teuthology.orchestra.run.smithi186:> uname -r 2024-04-12T20:32:44.438 INFO:teuthology.orchestra.run.smithi186.stdout:6.9.0-rc3-g3f60ed14f7c1 2024-04-12T20:32:44.438 DEBUG:teuthology.task.kernel:current kernel version is 6.9.0-rc3-g3f60ed14f7c1 vs 6.9.0-rc3-g3f60ed14f7c1 2024-04-12T20:32:44.438 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-04-12T20:32:44.438 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-04-12T20:32:45.439 INFO:teuthology.task.kernel:Enabling kdb on host.b... 2024-04-12T20:32:45.439 DEBUG:teuthology.orchestra.run.smithi186:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-04-12T20:32:45.550 INFO:teuthology.orchestra.run.smithi186.stdout:ttyS1 2024-04-12T20:32:45.584 DEBUG:teuthology.parallel:result is None 2024-04-12T20:32:45.584 INFO:teuthology.run_tasks:Running task internal.base... 2024-04-12T20:32:45.593 INFO:teuthology.task.internal:Creating test directory... 2024-04-12T20:32:45.593 DEBUG:teuthology.orchestra.run.smithi018:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-04-12T20:32:45.603 DEBUG:teuthology.orchestra.run.smithi186:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-04-12T20:32:45.659 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-04-12T20:32:45.681 INFO:teuthology.run_tasks:Running task internal.archive... 2024-04-12T20:32:45.746 INFO:teuthology.task.internal:Creating archive directory... 2024-04-12T20:32:45.747 DEBUG:teuthology.orchestra.run.smithi018:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-04-12T20:32:45.754 DEBUG:teuthology.orchestra.run.smithi186:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-04-12T20:32:45.822 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-04-12T20:32:45.830 INFO:teuthology.task.internal:Enabling coredump saving... 2024-04-12T20:32:45.830 DEBUG:teuthology.orchestra.run.smithi018:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-04-12T20:32:45.838 DEBUG:teuthology.orchestra.run.smithi186:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-04-12T20:32:45.914 INFO:teuthology.orchestra.run.smithi186.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-12T20:32:45.938 INFO:teuthology.orchestra.run.smithi018.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-12T20:32:45.948 INFO:teuthology.orchestra.run.smithi186.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-12T20:32:45.983 INFO:teuthology.orchestra.run.smithi018.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-12T20:32:45.988 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-04-12T20:32:45.996 INFO:teuthology.task.internal:Configuring sudo... 2024-04-12T20:32:45.997 DEBUG:teuthology.orchestra.run.smithi018:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-04-12T20:32:46.003 DEBUG:teuthology.orchestra.run.smithi186:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-04-12T20:32:46.099 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-04-12T20:32:46.110 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-04-12T20:32:46.110 DEBUG:teuthology.orchestra.run.smithi018:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-04-12T20:32:46.116 DEBUG:teuthology.orchestra.run.smithi186:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-04-12T20:32:46.165 DEBUG:teuthology.orchestra.run.smithi018:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-12T20:32:46.252 DEBUG:teuthology.orchestra.run.smithi018:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-12T20:32:46.402 DEBUG:teuthology.orchestra.run.smithi018:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-12T20:32:46.488 DEBUG:teuthology.orchestra.run.smithi018:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-12T20:32:46.590 DEBUG:teuthology.orchestra.run.smithi018:> set -ex 2024-04-12T20:32:46.590 DEBUG:teuthology.orchestra.run.smithi018:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-04-12T20:32:46.690 DEBUG:teuthology.orchestra.run.smithi186:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-12T20:32:46.769 DEBUG:teuthology.orchestra.run.smithi186:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-12T20:32:46.874 DEBUG:teuthology.orchestra.run.smithi186:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-12T20:32:46.949 DEBUG:teuthology.orchestra.run.smithi186:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-12T20:32:47.030 DEBUG:teuthology.orchestra.run.smithi186:> set -ex 2024-04-12T20:32:47.030 DEBUG:teuthology.orchestra.run.smithi186:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-04-12T20:32:47.107 DEBUG:teuthology.orchestra.run.smithi018:> sudo service rsyslog restart 2024-04-12T20:32:47.114 DEBUG:teuthology.orchestra.run.smithi186:> sudo service rsyslog restart 2024-04-12T20:32:47.248 INFO:teuthology.orchestra.run.smithi018.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-12T20:32:47.285 INFO:teuthology.orchestra.run.smithi186.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-12T20:32:47.687 INFO:teuthology.run_tasks:Running task internal.timer... 2024-04-12T20:32:47.701 INFO:teuthology.task.internal:Starting timer... 2024-04-12T20:32:47.702 INFO:teuthology.run_tasks:Running task pcp... 2024-04-12T20:32:47.746 INFO:teuthology.run_tasks:Running task selinux... 2024-04-12T20:32:47.760 DEBUG:teuthology.task:Applying overrides for task selinux: {'allowlist': ['scontext=system_u:system_r:logrotate_t:s0', 'scontext=system_u:system_r:getty_t:s0']} 2024-04-12T20:32:47.761 DEBUG:teuthology.orchestra.run.smithi018:> sudo service auditd rotate 2024-04-12T20:32:47.906 INFO:teuthology.orchestra.run.smithi018.stdout:Rotating logs: 2024-04-12T20:32:47.911 DEBUG:teuthology.orchestra.run.smithi186:> sudo service auditd rotate 2024-04-12T20:32:48.082 INFO:teuthology.orchestra.run.smithi186.stdout:Rotating logs: 2024-04-12T20:32:48.094 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-04-12T20:32:48.094 DEBUG:teuthology.orchestra.run.smithi018:> /usr/sbin/getenforce 2024-04-12T20:32:48.155 INFO:teuthology.orchestra.run.smithi018.stdout:Permissive 2024-04-12T20:32:48.155 DEBUG:teuthology.orchestra.run.smithi186:> /usr/sbin/getenforce 2024-04-12T20:32:48.231 INFO:teuthology.orchestra.run.smithi186.stdout:Permissive 2024-04-12T20:32:48.231 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi018.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi186.front.sepia.ceph.com': 'permissive'} 2024-04-12T20:32:48.231 DEBUG:teuthology.orchestra.run.smithi018:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-12T20:32:48.326 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-12T20:32:48.327 DEBUG:teuthology.orchestra.run.smithi186:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-12T20:32:48.464 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-12T20:32:48.465 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-04-12T20:32:48.465 DEBUG:teuthology.orchestra.run.smithi018:> sudo /usr/sbin/setenforce permissive 2024-04-12T20:32:48.551 DEBUG:teuthology.orchestra.run.smithi186:> sudo /usr/sbin/setenforce permissive 2024-04-12T20:32:48.704 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-04-12T20:32:48.714 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-04-12T20:32:48.730 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-04-12T20:32:48.732 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi018.front.sepia.ceph.com,smithi186.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-04-12T20:39:09.752 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi018.front.sepia.ceph.com'), Remote(name='ubuntu@smithi186.front.sepia.ceph.com')] 2024-04-12T20:39:09.753 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi018.front.sepia.ceph.com' 2024-04-12T20:39:09.754 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi018.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-12T20:39:09.904 DEBUG:teuthology.orchestra.run.smithi018:> true 2024-04-12T20:39:10.073 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi018.front.sepia.ceph.com' 2024-04-12T20:39:10.073 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi186.front.sepia.ceph.com' 2024-04-12T20:39:10.074 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi186.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-12T20:39:10.215 DEBUG:teuthology.orchestra.run.smithi186:> true 2024-04-12T20:39:10.378 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi186.front.sepia.ceph.com' 2024-04-12T20:39:10.379 INFO:teuthology.run_tasks:Running task clock... 2024-04-12T20:39:10.392 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-04-12T20:39:10.392 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-04-12T20:39:10.393 DEBUG:teuthology.orchestra.run.smithi018:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-12T20:39:10.401 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-04-12T20:39:10.401 DEBUG:teuthology.orchestra.run.smithi186:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-12T20:39:10.498 INFO:teuthology.orchestra.run.smithi186.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-04-12T20:39:10.503 INFO:teuthology.orchestra.run.smithi018.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-04-12T20:39:10.552 INFO:teuthology.orchestra.run.smithi186.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-04-12T20:39:10.554 INFO:teuthology.orchestra.run.smithi018.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-04-12T20:39:10.646 INFO:teuthology.orchestra.run.smithi186.stderr:sudo: ntpd: command not found 2024-04-12T20:39:10.650 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: ntpd: command not found 2024-04-12T20:39:10.686 INFO:teuthology.orchestra.run.smithi186.stdout:506 Cannot talk to daemon 2024-04-12T20:39:10.694 INFO:teuthology.orchestra.run.smithi018.stdout:506 Cannot talk to daemon 2024-04-12T20:39:10.737 INFO:teuthology.orchestra.run.smithi186.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-04-12T20:39:10.749 INFO:teuthology.orchestra.run.smithi018.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-04-12T20:39:10.791 INFO:teuthology.orchestra.run.smithi186.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-04-12T20:39:10.805 INFO:teuthology.orchestra.run.smithi018.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-04-12T20:39:10.936 INFO:teuthology.orchestra.run.smithi186.stderr:bash: line 1: ntpq: command not found 2024-04-12T20:39:10.946 INFO:teuthology.orchestra.run.smithi186.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-12T20:39:10.946 INFO:teuthology.orchestra.run.smithi186.stdout:=============================================================================== 2024-04-12T20:39:10.946 INFO:teuthology.orchestra.run.smithi186.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-12T20:39:10.946 INFO:teuthology.orchestra.run.smithi186.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-12T20:39:10.946 INFO:teuthology.orchestra.run.smithi186.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-12T20:39:10.946 INFO:teuthology.orchestra.run.smithi186.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-12T20:39:10.958 INFO:teuthology.orchestra.run.smithi018.stderr:bash: line 1: ntpq: command not found 2024-04-12T20:39:10.968 INFO:teuthology.orchestra.run.smithi018.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-12T20:39:10.968 INFO:teuthology.orchestra.run.smithi018.stdout:=============================================================================== 2024-04-12T20:39:10.968 INFO:teuthology.orchestra.run.smithi018.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-12T20:39:10.968 INFO:teuthology.orchestra.run.smithi018.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-12T20:39:10.968 INFO:teuthology.orchestra.run.smithi018.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-12T20:39:10.968 INFO:teuthology.orchestra.run.smithi018.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-12T20:39:10.970 INFO:teuthology.run_tasks:Running task install... 2024-04-12T20:39:10.979 DEBUG:teuthology.task.install:project ceph 2024-04-12T20:39:10.979 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '324baf8ddb91dea46cac50cb535c306115791355'}} 2024-04-12T20:39:10.979 DEBUG:teuthology.task.install:config {'branch': 'quincy', 'exclude_packages': ['ceph-volume'], 'flavor': 'default', 'sha1': '324baf8ddb91dea46cac50cb535c306115791355'} 2024-04-12T20:39:10.979 INFO:teuthology.task.install:Using flavor: default 2024-04-12T20:39:10.987 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-04-12T20:39:10.987 INFO:teuthology.task.install:extra packages: [] 2024-04-12T20:39:10.988 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': 'quincy', 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': ['ceph-volume'], 'extra_packages': [], 'extra_system_packages': [], 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '324baf8ddb91dea46cac50cb535c306115791355', 'tag': None, 'wait_for_package': False} 2024-04-12T20:39:10.988 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-04-12T20:39:10.988 INFO:teuthology.packaging:ref: None 2024-04-12T20:39:10.988 INFO:teuthology.packaging:tag: None 2024-04-12T20:39:10.988 INFO:teuthology.packaging:branch: quincy 2024-04-12T20:39:10.988 INFO:teuthology.packaging:sha1: 324baf8ddb91dea46cac50cb535c306115791355 2024-04-12T20:39:10.988 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=quincy 2024-04-12T20:39:10.990 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': 'quincy', 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': ['ceph-volume'], 'extra_packages': [], 'extra_system_packages': [], 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '324baf8ddb91dea46cac50cb535c306115791355', 'tag': None, 'wait_for_package': False} 2024-04-12T20:39:10.990 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-04-12T20:39:10.990 INFO:teuthology.packaging:ref: None 2024-04-12T20:39:10.990 INFO:teuthology.packaging:tag: None 2024-04-12T20:39:10.990 INFO:teuthology.packaging:branch: quincy 2024-04-12T20:39:10.990 INFO:teuthology.packaging:sha1: 324baf8ddb91dea46cac50cb535c306115791355 2024-04-12T20:39:10.991 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=quincy 2024-04-12T20:39:11.217 INFO:teuthology.task.install.rpm:Pulling from https://4.chacra.ceph.com/r/ceph/quincy/d3b5060a17449edfae9385cdf792afec109fca5d/centos/9/flavors/default/ 2024-04-12T20:39:11.217 INFO:teuthology.task.install.rpm:Package version is 17.2.7-827.gd3b5060a 2024-04-12T20:39:11.222 INFO:teuthology.task.install.rpm:Pulling from https://4.chacra.ceph.com/r/ceph/quincy/d3b5060a17449edfae9385cdf792afec109fca5d/centos/9/flavors/default/ 2024-04-12T20:39:11.222 INFO:teuthology.task.install.rpm:Package version is 17.2.7-827.gd3b5060a 2024-04-12T20:39:11.331 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://4.chacra.ceph.com/r/ceph/quincy/d3b5060a17449edfae9385cdf792afec109fca5d/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://4.chacra.ceph.com/r/ceph/quincy/d3b5060a17449edfae9385cdf792afec109fca5d/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://4.chacra.ceph.com/r/ceph/quincy/d3b5060a17449edfae9385cdf792afec109fca5d/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-04-12T20:39:11.332 DEBUG:teuthology.orchestra.run.smithi186:> set -ex 2024-04-12T20:39:11.332 DEBUG:teuthology.orchestra.run.smithi186:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-04-12T20:39:11.340 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://4.chacra.ceph.com/r/ceph/quincy/d3b5060a17449edfae9385cdf792afec109fca5d/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://4.chacra.ceph.com/r/ceph/quincy/d3b5060a17449edfae9385cdf792afec109fca5d/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://4.chacra.ceph.com/r/ceph/quincy/d3b5060a17449edfae9385cdf792afec109fca5d/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-04-12T20:39:11.340 DEBUG:teuthology.orchestra.run.smithi018:> set -ex 2024-04-12T20:39:11.340 DEBUG:teuthology.orchestra.run.smithi018:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-04-12T20:39:11.434 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd on remote rpm x86_64 2024-04-12T20:39:11.434 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-04-12T20:39:11.434 INFO:teuthology.packaging:ref: None 2024-04-12T20:39:11.434 INFO:teuthology.packaging:tag: None 2024-04-12T20:39:11.434 INFO:teuthology.packaging:branch: quincy 2024-04-12T20:39:11.434 INFO:teuthology.packaging:sha1: 324baf8ddb91dea46cac50cb535c306115791355 2024-04-12T20:39:11.434 DEBUG:teuthology.orchestra.run.smithi186:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;ref/quincy/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-04-12T20:39:11.448 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd on remote rpm x86_64 2024-04-12T20:39:11.448 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-04-12T20:39:11.448 INFO:teuthology.packaging:ref: None 2024-04-12T20:39:11.448 INFO:teuthology.packaging:tag: None 2024-04-12T20:39:11.448 INFO:teuthology.packaging:branch: quincy 2024-04-12T20:39:11.448 INFO:teuthology.packaging:sha1: 324baf8ddb91dea46cac50cb535c306115791355 2024-04-12T20:39:11.448 DEBUG:teuthology.orchestra.run.smithi018:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;ref/quincy/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-04-12T20:39:11.576 DEBUG:teuthology.orchestra.run.smithi186:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-04-12T20:39:11.593 DEBUG:teuthology.orchestra.run.smithi018:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-04-12T20:39:11.711 DEBUG:teuthology.orchestra.run.smithi186:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-04-12T20:39:11.734 DEBUG:teuthology.orchestra.run.smithi018:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-04-12T20:39:11.797 INFO:teuthology.orchestra.run.smithi186.stdout:check_obsoletes = 1 2024-04-12T20:39:11.801 DEBUG:teuthology.orchestra.run.smithi186:> sudo yum clean all 2024-04-12T20:39:11.831 INFO:teuthology.orchestra.run.smithi018.stdout:check_obsoletes = 1 2024-04-12T20:39:11.836 DEBUG:teuthology.orchestra.run.smithi018:> sudo yum clean all 2024-04-12T20:39:12.156 INFO:teuthology.orchestra.run.smithi186.stdout:74 files removed 2024-04-12T20:39:12.195 DEBUG:teuthology.orchestra.run.smithi186:> sudo yum -y install ceph-radosgw 2024-04-12T20:39:12.219 INFO:teuthology.orchestra.run.smithi018.stdout:74 files removed 2024-04-12T20:39:12.259 DEBUG:teuthology.orchestra.run.smithi018:> sudo yum -y install ceph-radosgw 2024-04-12T20:39:13.235 INFO:teuthology.orchestra.run.smithi186.stdout:ceph packages for x86_64 149 kB/s | 77 kB 00:00 2024-04-12T20:39:13.289 INFO:teuthology.orchestra.run.smithi018.stdout:ceph packages for x86_64 157 kB/s | 77 kB 00:00 2024-04-12T20:39:13.856 INFO:teuthology.orchestra.run.smithi018.stdout:ceph noarch packages 19 kB/s | 9.2 kB 00:00 2024-04-12T20:39:14.282 INFO:teuthology.orchestra.run.smithi018.stdout:ceph source packages 5.1 kB/s | 1.8 kB 00:00 2024-04-12T20:39:14.768 INFO:teuthology.orchestra.run.smithi186.stdout:ceph noarch packages 6.4 kB/s | 9.2 kB 00:01 2024-04-12T20:39:15.044 INFO:teuthology.orchestra.run.smithi018.stdout:CentOS-9 - AppStream 12 MB/s | 8.1 MB 00:00 2024-04-12T20:39:15.469 INFO:teuthology.orchestra.run.smithi186.stdout:ceph source packages 2.9 kB/s | 1.8 kB 00:00 2024-04-12T20:39:16.214 INFO:teuthology.orchestra.run.smithi186.stdout:CentOS-9 - AppStream 12 MB/s | 8.1 MB 00:00 2024-04-12T20:39:17.019 INFO:teuthology.orchestra.run.smithi018.stdout:CentOS-9 - BaseOS 5.8 MB/s | 2.5 MB 00:00 2024-04-12T20:39:18.173 INFO:teuthology.orchestra.run.smithi186.stdout:CentOS-9 - BaseOS 5.9 MB/s | 2.5 MB 00:00 2024-04-12T20:39:18.549 INFO:teuthology.orchestra.run.smithi018.stdout:CentOS Stream 9 - BaseOS 8.1 MB/s | 8.0 MB 00:00 2024-04-12T20:39:19.310 INFO:teuthology.orchestra.run.smithi186.stdout:CentOS Stream 9 - BaseOS 14 MB/s | 8.0 MB 00:00 2024-04-12T20:39:20.833 INFO:teuthology.orchestra.run.smithi018.stdout:CentOS Stream 9 - AppStream 19 MB/s | 19 MB 00:01 2024-04-12T20:39:21.412 INFO:teuthology.orchestra.run.smithi186.stdout:CentOS Stream 9 - AppStream 22 MB/s | 19 MB 00:00 2024-04-12T20:39:25.781 INFO:teuthology.orchestra.run.smithi018.stdout:CentOS Stream 9 - CRB 6.5 MB/s | 6.2 MB 00:00 2024-04-12T20:39:26.954 INFO:teuthology.orchestra.run.smithi186.stdout:CentOS Stream 9 - CRB 3.7 MB/s | 6.2 MB 00:01 2024-04-12T20:39:27.484 INFO:teuthology.orchestra.run.smithi018.stdout:CentOS Stream 9 - Extras packages 54 kB/s | 16 kB 00:00 2024-04-12T20:39:28.713 INFO:teuthology.orchestra.run.smithi186.stdout:CentOS Stream 9 - Extras packages 39 kB/s | 16 kB 00:00 2024-04-12T20:39:28.936 INFO:teuthology.orchestra.run.smithi018.stdout:Extra Packages for Enterprise Linux 17 MB/s | 21 MB 00:01 2024-04-12T20:39:31.048 INFO:teuthology.orchestra.run.smithi186.stdout:Extra Packages for Enterprise Linux 9.8 MB/s | 21 MB 00:02 2024-04-12T20:39:35.178 INFO:teuthology.orchestra.run.smithi018.stdout:lab-extras 14 kB/s | 1.7 kB 00:00 2024-04-12T20:39:37.190 INFO:teuthology.orchestra.run.smithi186.stdout:lab-extras 14 kB/s | 1.7 kB 00:00 2024-04-12T20:39:37.943 INFO:teuthology.orchestra.run.smithi018.stderr:Modular dependency problems: 2024-04-12T20:39:37.943 INFO:teuthology.orchestra.run.smithi018.stderr: 2024-04-12T20:39:37.943 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 1: conflicting requests 2024-04-12T20:39:37.943 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module container-tools:rhel8:820240321091303:20125149.x86_64 from CentOS-AppStream 2024-04-12T20:39:37.943 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 2: conflicting requests 2024-04-12T20:39:37.943 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module eclipse:rhel8:820201023100746:b230ed4e.x86_64 from CentOS-AppStream 2024-04-12T20:39:37.943 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 3: conflicting requests 2024-04-12T20:39:37.943 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module freeradius:3.0:8080020230117180605:89170a74.x86_64 from CentOS-AppStream 2024-04-12T20:39:37.943 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 4: conflicting requests 2024-04-12T20:39:37.943 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module gimp:2.8:8000020190628145146:4148dfdf.x86_64 from CentOS-AppStream 2024-04-12T20:39:37.944 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 5: conflicting requests 2024-04-12T20:39:37.944 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module go-toolset:rhel8:820240401160413:b754926a.x86_64 from CentOS-AppStream 2024-04-12T20:39:37.944 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 6: conflicting requests 2024-04-12T20:39:37.944 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module httpd:2.4:820240216084734:9edba152.x86_64 from CentOS-AppStream 2024-04-12T20:39:37.944 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 7: conflicting requests 2024-04-12T20:39:37.944 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module idm:client:820240307184541:49cc9d1b.x86_64 from CentOS-AppStream 2024-04-12T20:39:37.944 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 8: conflicting requests 2024-04-12T20:39:37.944 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module inkscape:0.92.3:8080020221205124429:3e031279.x86_64 from CentOS-AppStream 2024-04-12T20:39:37.944 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 9: conflicting requests 2024-04-12T20:39:37.944 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module jmc:rhel8:8050020211110222101:6392b1f8.x86_64 from CentOS-AppStream 2024-04-12T20:39:37.944 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 10: conflicting requests 2024-04-12T20:39:37.944 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module llvm-toolset:rhel8:820240205133135:9edba152.x86_64 from CentOS-AppStream 2024-04-12T20:39:37.944 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 11: conflicting requests 2024-04-12T20:39:37.944 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module log4j:2:8080020221205124743:9d367344.x86_64 from CentOS-AppStream 2024-04-12T20:39:37.945 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 12: conflicting requests 2024-04-12T20:39:37.945 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module mailman:2.1:820230727085757:77fc8825.x86_64 from CentOS-AppStream 2024-04-12T20:39:37.945 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 13: conflicting requests 2024-04-12T20:39:37.945 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module mariadb:10.3:8030020210419150013:30b713e6.x86_64 from CentOS-AppStream 2024-04-12T20:39:37.945 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 14: conflicting requests 2024-04-12T20:39:37.945 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module maven:3.5:8060020211203142015:c0229ad2.x86_64 from CentOS-AppStream 2024-04-12T20:39:37.945 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 15: conflicting requests 2024-04-12T20:39:37.945 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module mercurial:4.8:8000020190628020724:4148dfdf.x86_64 from CentOS-AppStream 2024-04-12T20:39:37.945 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 16: conflicting requests 2024-04-12T20:39:37.945 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module mod_auth_openidc:2.3:820230721123832:9edba152.x86_64 from CentOS-AppStream 2024-04-12T20:39:37.945 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 17: conflicting requests 2024-04-12T20:39:37.945 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module mysql:8.0:8040020210901180257:9f9e2e7e.x86_64 from CentOS-AppStream 2024-04-12T20:39:37.945 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 18: conflicting requests 2024-04-12T20:39:37.946 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module nginx:1.14:8000020211221191913:55190bc5.x86_64 from CentOS-AppStream 2024-04-12T20:39:37.946 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 19: conflicting requests 2024-04-12T20:39:37.946 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module nodejs:10:8040020210120182536:9f9e2e7e.x86_64 from CentOS-AppStream 2024-04-12T20:39:37.946 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 20: conflicting requests 2024-04-12T20:39:37.946 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl:5.26:8000020190628020724:55190bc5.x86_64 from CentOS-AppStream 2024-04-12T20:39:37.946 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 21: conflicting requests 2024-04-12T20:39:37.946 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module php:7.2:8020020200507003613:2c7ca891.x86_64 from CentOS-AppStream 2024-04-12T20:39:37.946 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 22: conflicting requests 2024-04-12T20:39:37.946 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module postgresql:10:820230223114800:9edba152.x86_64 from CentOS-AppStream 2024-04-12T20:39:37.946 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 23: conflicting requests 2024-04-12T20:39:37.946 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module python27:2.7:820240208011952:182f7c73.x86_64 from CentOS-AppStream 2024-04-12T20:39:37.946 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 24: conflicting requests 2024-04-12T20:39:37.947 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module python36:3.6:8050020210825152031:982725ab.x86_64 from CentOS-AppStream 2024-04-12T20:39:37.947 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 25: conflicting requests 2024-04-12T20:39:37.947 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module python38:3.8:820230810143931:64c1cd5a.x86_64 from CentOS-AppStream 2024-04-12T20:39:37.947 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 26: conflicting requests 2024-04-12T20:39:37.947 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module python39:3.9:820240214182535:17377f89.x86_64 from CentOS-AppStream 2024-04-12T20:39:37.947 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 27: conflicting requests 2024-04-12T20:39:37.947 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module redis:5:8040020211019153849:9f9e2e7e.x86_64 from CentOS-AppStream 2024-04-12T20:39:37.947 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 28: conflicting requests 2024-04-12T20:39:37.947 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module rhn-tools:1.0:8010020191114034948:f69d1239.x86_64 from CentOS-AppStream 2024-04-12T20:39:37.947 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 29: conflicting requests 2024-04-12T20:39:37.947 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module ruby:2.5:820230627084142:9edba152.x86_64 from CentOS-AppStream 2024-04-12T20:39:37.947 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 30: conflicting requests 2024-04-12T20:39:37.948 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module rust-toolset:rhel8:820240119204620:b09eea91.x86_64 from CentOS-AppStream 2024-04-12T20:39:37.948 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 31: conflicting requests 2024-04-12T20:39:37.948 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module satellite-5-client:1.0:8010020191114035551:cdc1202b.x86_64 from CentOS-AppStream 2024-04-12T20:39:37.948 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 32: conflicting requests 2024-04-12T20:39:37.948 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module squid:4:820240319173245:9edba152.x86_64 from CentOS-AppStream 2024-04-12T20:39:37.948 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 33: conflicting requests 2024-04-12T20:39:37.948 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module subversion:1.10:8070020220711155714:78111232.x86_64 from CentOS-AppStream 2024-04-12T20:39:37.948 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 34: conflicting requests 2024-04-12T20:39:37.948 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module swig:3.0:8030020200706142531:30b713e6.x86_64 from CentOS-AppStream 2024-04-12T20:39:37.948 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 35: conflicting requests 2024-04-12T20:39:37.948 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module varnish:6:8050020211109225449:b4937e53.x86_64 from CentOS-AppStream 2024-04-12T20:39:37.948 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 36: conflicting requests 2024-04-12T20:39:37.948 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module virt:rhel:820240314161907:9edba152.x86_64 from CentOS-AppStream 2024-04-12T20:39:37.949 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 37: conflicting requests 2024-04-12T20:39:37.949 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:8030020200716155257:7cc0a66d.x86_64 from CentOS-AppStream 2024-04-12T20:39:37.949 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:8030020200716155257:b967a9a2.x86_64 from CentOS-AppStream 2024-04-12T20:39:37.949 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:820230811133638:36f9fae6.x86_64 from CentOS-AppStream 2024-04-12T20:39:37.949 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 38: cannot install the best candidate for the job 2024-04-12T20:39:37.949 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:8010020191114031501:a5949e2e.x86_64 from CentOS-AppStream 2024-04-12T20:39:37.949 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:8010020191114031501:be2e4737.x86_64 from CentOS-AppStream 2024-04-12T20:39:37.949 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:820230223114759:2ab761e1.x86_64 from CentOS-AppStream 2024-04-12T20:39:37.949 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:820230223114759:86e45846.x86_64 from CentOS-AppStream 2024-04-12T20:39:37.949 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 39: conflicting requests 2024-04-12T20:39:37.949 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:8030020200715230104:1e4bbb35.x86_64 from CentOS-AppStream 2024-04-12T20:39:37.949 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:8030020200715230104:ea09926d.x86_64 from CentOS-AppStream 2024-04-12T20:39:37.949 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:082fdf2f.x86_64 from CentOS-AppStream 2024-04-12T20:39:37.950 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-04-12T20:39:37.950 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-04-12T20:39:37.950 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:fbe42456.x86_64 from CentOS-AppStream 2024-04-12T20:39:37.950 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 40: cannot install the best candidate for the job 2024-04-12T20:39:37.950 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:8010020191114031513:16b3ab4d.x86_64 from CentOS-AppStream 2024-04-12T20:39:37.950 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:8010020191114031513:e1f37755.x86_64 from CentOS-AppStream 2024-04-12T20:39:37.950 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-04-12T20:39:37.950 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-04-12T20:39:37.950 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 41: cannot install the best candidate for the job 2024-04-12T20:39:37.950 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:8030020200716150652:1e4bbb35.x86_64 from CentOS-AppStream 2024-04-12T20:39:37.950 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:8030020200716150652:ea09926d.x86_64 from CentOS-AppStream 2024-04-12T20:39:37.950 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-04-12T20:39:37.950 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-04-12T20:39:37.950 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 42: cannot install the best candidate for the job 2024-04-12T20:39:37.951 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:8030020200716174729:3a70019f.x86_64 from CentOS-AppStream 2024-04-12T20:39:37.951 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:8030020200716174729:cccafca5.x86_64 from CentOS-AppStream 2024-04-12T20:39:37.951 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:820230223114759:6fdf7e5d.x86_64 from CentOS-AppStream 2024-04-12T20:39:37.951 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:820230223114759:d4326aba.x86_64 from CentOS-AppStream 2024-04-12T20:39:37.951 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 43: cannot install the best candidate for the job 2024-04-12T20:39:37.951 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:8030020200716213634:56fce90f.x86_64 from CentOS-AppStream 2024-04-12T20:39:37.951 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:8030020200716213634:98a3c9d0.x86_64 from CentOS-AppStream 2024-04-12T20:39:37.951 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:820230223114759:06b7596e.x86_64 from CentOS-AppStream 2024-04-12T20:39:37.951 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:820230223114759:d2fc5d76.x86_64 from CentOS-AppStream 2024-04-12T20:39:37.951 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 44: cannot install the best candidate for the job 2024-04-12T20:39:37.951 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:8030020200716171139:3a70019f.x86_64 from CentOS-AppStream 2024-04-12T20:39:37.951 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:8030020200716171139:cccafca5.x86_64 from CentOS-AppStream 2024-04-12T20:39:37.951 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:820230223114759:6fdf7e5d.x86_64 from CentOS-AppStream 2024-04-12T20:39:37.952 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:820230223114759:d4326aba.x86_64 from CentOS-AppStream 2024-04-12T20:39:37.952 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 45: cannot install the best candidate for the job 2024-04-12T20:39:37.952 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:8010020191120175858:95fce66d.x86_64 from CentOS-AppStream 2024-04-12T20:39:37.952 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:8030020200723233049:09acf126.x86_64 from CentOS-AppStream 2024-04-12T20:39:37.952 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:820230223114759:436af1ae.x86_64 from CentOS-AppStream 2024-04-12T20:39:37.952 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:820230223114759:b12195ea.x86_64 from CentOS-AppStream 2024-04-12T20:39:38.650 INFO:teuthology.orchestra.run.smithi018.stdout:Dependencies resolved. 2024-04-12T20:39:38.665 INFO:teuthology.orchestra.run.smithi018.stdout:================================================================================ 2024-04-12T20:39:38.665 INFO:teuthology.orchestra.run.smithi018.stdout: Package Arch Version Repository Size 2024-04-12T20:39:38.665 INFO:teuthology.orchestra.run.smithi018.stdout:================================================================================ 2024-04-12T20:39:38.665 INFO:teuthology.orchestra.run.smithi018.stdout:Installing: 2024-04-12T20:39:38.666 INFO:teuthology.orchestra.run.smithi018.stdout: ceph-radosgw x86_64 2:17.2.7-827.gd3b5060a.el9 ceph 14 M 2024-04-12T20:39:38.666 INFO:teuthology.orchestra.run.smithi018.stdout:Upgrading: 2024-04-12T20:39:38.666 INFO:teuthology.orchestra.run.smithi018.stdout: librados2 x86_64 2:17.2.7-827.gd3b5060a.el9 ceph 3.1 M 2024-04-12T20:39:38.666 INFO:teuthology.orchestra.run.smithi018.stdout: librbd1 x86_64 2:17.2.7-827.gd3b5060a.el9 ceph 2.9 M 2024-04-12T20:39:38.666 INFO:teuthology.orchestra.run.smithi018.stdout:Installing dependencies: 2024-04-12T20:39:38.666 INFO:teuthology.orchestra.run.smithi018.stdout: ceph-base x86_64 2:17.2.7-827.gd3b5060a.el9 ceph 5.0 M 2024-04-12T20:39:38.666 INFO:teuthology.orchestra.run.smithi018.stdout: ceph-common x86_64 2:17.2.7-827.gd3b5060a.el9 ceph 22 M 2024-04-12T20:39:38.666 INFO:teuthology.orchestra.run.smithi018.stdout: ceph-selinux x86_64 2:17.2.7-827.gd3b5060a.el9 ceph 25 k 2024-04-12T20:39:38.666 INFO:teuthology.orchestra.run.smithi018.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-04-12T20:39:38.666 INFO:teuthology.orchestra.run.smithi018.stdout: libcephfs2 x86_64 2:17.2.7-827.gd3b5060a.el9 ceph 637 k 2024-04-12T20:39:38.666 INFO:teuthology.orchestra.run.smithi018.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-04-12T20:39:38.666 INFO:teuthology.orchestra.run.smithi018.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-04-12T20:39:38.666 INFO:teuthology.orchestra.run.smithi018.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-04-12T20:39:38.667 INFO:teuthology.orchestra.run.smithi018.stdout: libradosstriper1 x86_64 2:17.2.7-827.gd3b5060a.el9 ceph 458 k 2024-04-12T20:39:38.667 INFO:teuthology.orchestra.run.smithi018.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-04-12T20:39:38.667 INFO:teuthology.orchestra.run.smithi018.stdout: librgw2 x86_64 2:17.2.7-827.gd3b5060a.el9 ceph 4.9 M 2024-04-12T20:39:38.667 INFO:teuthology.orchestra.run.smithi018.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-04-12T20:39:38.667 INFO:teuthology.orchestra.run.smithi018.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-04-12T20:39:38.667 INFO:teuthology.orchestra.run.smithi018.stdout: python3-ceph-argparse x86_64 2:17.2.7-827.gd3b5060a.el9 ceph 45 k 2024-04-12T20:39:38.667 INFO:teuthology.orchestra.run.smithi018.stdout: python3-ceph-common x86_64 2:17.2.7-827.gd3b5060a.el9 ceph 117 k 2024-04-12T20:39:38.667 INFO:teuthology.orchestra.run.smithi018.stdout: python3-cephfs x86_64 2:17.2.7-827.gd3b5060a.el9 ceph 156 k 2024-04-12T20:39:38.667 INFO:teuthology.orchestra.run.smithi018.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-04-12T20:39:38.667 INFO:teuthology.orchestra.run.smithi018.stdout: python3-rados x86_64 2:17.2.7-827.gd3b5060a.el9 ceph 320 k 2024-04-12T20:39:38.667 INFO:teuthology.orchestra.run.smithi018.stdout: python3-rbd x86_64 2:17.2.7-827.gd3b5060a.el9 ceph 292 k 2024-04-12T20:39:38.667 INFO:teuthology.orchestra.run.smithi018.stdout: python3-rgw x86_64 2:17.2.7-827.gd3b5060a.el9 ceph 99 k 2024-04-12T20:39:38.667 INFO:teuthology.orchestra.run.smithi018.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-04-12T20:39:38.667 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-12T20:39:38.667 INFO:teuthology.orchestra.run.smithi018.stdout:Transaction Summary 2024-04-12T20:39:38.668 INFO:teuthology.orchestra.run.smithi018.stdout:================================================================================ 2024-04-12T20:39:38.668 INFO:teuthology.orchestra.run.smithi018.stdout:Install 22 Packages 2024-04-12T20:39:38.668 INFO:teuthology.orchestra.run.smithi018.stdout:Upgrade 2 Packages 2024-04-12T20:39:38.668 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-12T20:39:38.668 INFO:teuthology.orchestra.run.smithi018.stdout:Total download size: 58 M 2024-04-12T20:39:38.668 INFO:teuthology.orchestra.run.smithi018.stdout:Downloading Packages: 2024-04-12T20:39:39.837 INFO:teuthology.orchestra.run.smithi186.stderr:Modular dependency problems: 2024-04-12T20:39:39.837 INFO:teuthology.orchestra.run.smithi186.stderr: 2024-04-12T20:39:39.837 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 1: conflicting requests 2024-04-12T20:39:39.837 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module container-tools:rhel8:820240321091303:20125149.x86_64 from CentOS-AppStream 2024-04-12T20:39:39.837 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 2: conflicting requests 2024-04-12T20:39:39.838 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module eclipse:rhel8:820201023100746:b230ed4e.x86_64 from CentOS-AppStream 2024-04-12T20:39:39.838 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 3: conflicting requests 2024-04-12T20:39:39.838 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module freeradius:3.0:8080020230117180605:89170a74.x86_64 from CentOS-AppStream 2024-04-12T20:39:39.838 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 4: conflicting requests 2024-04-12T20:39:39.838 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module gimp:2.8:8000020190628145146:4148dfdf.x86_64 from CentOS-AppStream 2024-04-12T20:39:39.838 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 5: conflicting requests 2024-04-12T20:39:39.838 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module go-toolset:rhel8:820240401160413:b754926a.x86_64 from CentOS-AppStream 2024-04-12T20:39:39.838 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 6: conflicting requests 2024-04-12T20:39:39.838 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module httpd:2.4:820240216084734:9edba152.x86_64 from CentOS-AppStream 2024-04-12T20:39:39.838 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 7: conflicting requests 2024-04-12T20:39:39.838 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module idm:client:820240307184541:49cc9d1b.x86_64 from CentOS-AppStream 2024-04-12T20:39:39.838 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 8: conflicting requests 2024-04-12T20:39:39.839 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module inkscape:0.92.3:8080020221205124429:3e031279.x86_64 from CentOS-AppStream 2024-04-12T20:39:39.839 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 9: conflicting requests 2024-04-12T20:39:39.839 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module jmc:rhel8:8050020211110222101:6392b1f8.x86_64 from CentOS-AppStream 2024-04-12T20:39:39.839 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 10: conflicting requests 2024-04-12T20:39:39.839 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module llvm-toolset:rhel8:820240205133135:9edba152.x86_64 from CentOS-AppStream 2024-04-12T20:39:39.839 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 11: conflicting requests 2024-04-12T20:39:39.839 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module log4j:2:8080020221205124743:9d367344.x86_64 from CentOS-AppStream 2024-04-12T20:39:39.839 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 12: conflicting requests 2024-04-12T20:39:39.839 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module mailman:2.1:820230727085757:77fc8825.x86_64 from CentOS-AppStream 2024-04-12T20:39:39.839 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 13: conflicting requests 2024-04-12T20:39:39.839 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module mariadb:10.3:8030020210419150013:30b713e6.x86_64 from CentOS-AppStream 2024-04-12T20:39:39.839 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 14: conflicting requests 2024-04-12T20:39:39.839 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module maven:3.5:8060020211203142015:c0229ad2.x86_64 from CentOS-AppStream 2024-04-12T20:39:39.839 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 15: conflicting requests 2024-04-12T20:39:39.840 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module mercurial:4.8:8000020190628020724:4148dfdf.x86_64 from CentOS-AppStream 2024-04-12T20:39:39.840 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 16: conflicting requests 2024-04-12T20:39:39.840 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module mod_auth_openidc:2.3:820230721123832:9edba152.x86_64 from CentOS-AppStream 2024-04-12T20:39:39.840 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 17: conflicting requests 2024-04-12T20:39:39.840 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module mysql:8.0:8040020210901180257:9f9e2e7e.x86_64 from CentOS-AppStream 2024-04-12T20:39:39.840 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 18: conflicting requests 2024-04-12T20:39:39.840 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module nginx:1.14:8000020211221191913:55190bc5.x86_64 from CentOS-AppStream 2024-04-12T20:39:39.840 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 19: conflicting requests 2024-04-12T20:39:39.840 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module nodejs:10:8040020210120182536:9f9e2e7e.x86_64 from CentOS-AppStream 2024-04-12T20:39:39.840 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 20: conflicting requests 2024-04-12T20:39:39.840 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl:5.26:8000020190628020724:55190bc5.x86_64 from CentOS-AppStream 2024-04-12T20:39:39.840 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 21: conflicting requests 2024-04-12T20:39:39.840 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module php:7.2:8020020200507003613:2c7ca891.x86_64 from CentOS-AppStream 2024-04-12T20:39:39.840 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 22: conflicting requests 2024-04-12T20:39:39.840 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module postgresql:10:820230223114800:9edba152.x86_64 from CentOS-AppStream 2024-04-12T20:39:39.841 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 23: conflicting requests 2024-04-12T20:39:39.841 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module python27:2.7:820240208011952:182f7c73.x86_64 from CentOS-AppStream 2024-04-12T20:39:39.841 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 24: conflicting requests 2024-04-12T20:39:39.841 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module python36:3.6:8050020210825152031:982725ab.x86_64 from CentOS-AppStream 2024-04-12T20:39:39.841 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 25: conflicting requests 2024-04-12T20:39:39.841 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module python38:3.8:820230810143931:64c1cd5a.x86_64 from CentOS-AppStream 2024-04-12T20:39:39.841 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 26: conflicting requests 2024-04-12T20:39:39.841 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module python39:3.9:820240214182535:17377f89.x86_64 from CentOS-AppStream 2024-04-12T20:39:39.841 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 27: conflicting requests 2024-04-12T20:39:39.841 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module redis:5:8040020211019153849:9f9e2e7e.x86_64 from CentOS-AppStream 2024-04-12T20:39:39.841 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 28: conflicting requests 2024-04-12T20:39:39.841 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module rhn-tools:1.0:8010020191114034948:f69d1239.x86_64 from CentOS-AppStream 2024-04-12T20:39:39.841 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 29: conflicting requests 2024-04-12T20:39:39.841 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module ruby:2.5:820230627084142:9edba152.x86_64 from CentOS-AppStream 2024-04-12T20:39:39.842 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 30: conflicting requests 2024-04-12T20:39:39.842 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module rust-toolset:rhel8:820240119204620:b09eea91.x86_64 from CentOS-AppStream 2024-04-12T20:39:39.842 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 31: conflicting requests 2024-04-12T20:39:39.842 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module satellite-5-client:1.0:8010020191114035551:cdc1202b.x86_64 from CentOS-AppStream 2024-04-12T20:39:39.842 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 32: conflicting requests 2024-04-12T20:39:39.842 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module squid:4:820240319173245:9edba152.x86_64 from CentOS-AppStream 2024-04-12T20:39:39.842 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 33: conflicting requests 2024-04-12T20:39:39.842 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module subversion:1.10:8070020220711155714:78111232.x86_64 from CentOS-AppStream 2024-04-12T20:39:39.842 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 34: conflicting requests 2024-04-12T20:39:39.842 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module swig:3.0:8030020200706142531:30b713e6.x86_64 from CentOS-AppStream 2024-04-12T20:39:39.842 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 35: conflicting requests 2024-04-12T20:39:39.842 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module varnish:6:8050020211109225449:b4937e53.x86_64 from CentOS-AppStream 2024-04-12T20:39:39.842 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 36: conflicting requests 2024-04-12T20:39:39.842 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module virt:rhel:820240314161907:9edba152.x86_64 from CentOS-AppStream 2024-04-12T20:39:39.843 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 37: conflicting requests 2024-04-12T20:39:39.843 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:8030020200716155257:7cc0a66d.x86_64 from CentOS-AppStream 2024-04-12T20:39:39.843 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:8030020200716155257:b967a9a2.x86_64 from CentOS-AppStream 2024-04-12T20:39:39.843 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:820230811133638:36f9fae6.x86_64 from CentOS-AppStream 2024-04-12T20:39:39.843 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 38: cannot install the best candidate for the job 2024-04-12T20:39:39.843 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:8010020191114031501:a5949e2e.x86_64 from CentOS-AppStream 2024-04-12T20:39:39.843 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:8010020191114031501:be2e4737.x86_64 from CentOS-AppStream 2024-04-12T20:39:39.843 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:820230223114759:2ab761e1.x86_64 from CentOS-AppStream 2024-04-12T20:39:39.843 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:820230223114759:86e45846.x86_64 from CentOS-AppStream 2024-04-12T20:39:39.843 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 39: conflicting requests 2024-04-12T20:39:39.843 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:8030020200715230104:1e4bbb35.x86_64 from CentOS-AppStream 2024-04-12T20:39:39.843 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:8030020200715230104:ea09926d.x86_64 from CentOS-AppStream 2024-04-12T20:39:39.843 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:082fdf2f.x86_64 from CentOS-AppStream 2024-04-12T20:39:39.843 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-04-12T20:39:39.844 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-04-12T20:39:39.844 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:fbe42456.x86_64 from CentOS-AppStream 2024-04-12T20:39:39.844 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 40: cannot install the best candidate for the job 2024-04-12T20:39:39.844 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:8010020191114031513:16b3ab4d.x86_64 from CentOS-AppStream 2024-04-12T20:39:39.844 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:8010020191114031513:e1f37755.x86_64 from CentOS-AppStream 2024-04-12T20:39:39.844 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-04-12T20:39:39.844 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-04-12T20:39:39.844 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 41: cannot install the best candidate for the job 2024-04-12T20:39:39.844 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:8030020200716150652:1e4bbb35.x86_64 from CentOS-AppStream 2024-04-12T20:39:39.844 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:8030020200716150652:ea09926d.x86_64 from CentOS-AppStream 2024-04-12T20:39:39.844 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-04-12T20:39:39.844 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-04-12T20:39:39.844 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 42: cannot install the best candidate for the job 2024-04-12T20:39:39.844 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:8030020200716174729:3a70019f.x86_64 from CentOS-AppStream 2024-04-12T20:39:39.845 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:8030020200716174729:cccafca5.x86_64 from CentOS-AppStream 2024-04-12T20:39:39.845 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:820230223114759:6fdf7e5d.x86_64 from CentOS-AppStream 2024-04-12T20:39:39.845 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:820230223114759:d4326aba.x86_64 from CentOS-AppStream 2024-04-12T20:39:39.845 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 43: cannot install the best candidate for the job 2024-04-12T20:39:39.845 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:8030020200716213634:56fce90f.x86_64 from CentOS-AppStream 2024-04-12T20:39:39.845 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:8030020200716213634:98a3c9d0.x86_64 from CentOS-AppStream 2024-04-12T20:39:39.845 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:820230223114759:06b7596e.x86_64 from CentOS-AppStream 2024-04-12T20:39:39.845 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:820230223114759:d2fc5d76.x86_64 from CentOS-AppStream 2024-04-12T20:39:39.845 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 44: cannot install the best candidate for the job 2024-04-12T20:39:39.845 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:8030020200716171139:3a70019f.x86_64 from CentOS-AppStream 2024-04-12T20:39:39.845 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:8030020200716171139:cccafca5.x86_64 from CentOS-AppStream 2024-04-12T20:39:39.845 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:820230223114759:6fdf7e5d.x86_64 from CentOS-AppStream 2024-04-12T20:39:39.845 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:820230223114759:d4326aba.x86_64 from CentOS-AppStream 2024-04-12T20:39:39.845 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 45: cannot install the best candidate for the job 2024-04-12T20:39:39.846 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:8010020191120175858:95fce66d.x86_64 from CentOS-AppStream 2024-04-12T20:39:39.846 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:8030020200723233049:09acf126.x86_64 from CentOS-AppStream 2024-04-12T20:39:39.846 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:820230223114759:436af1ae.x86_64 from CentOS-AppStream 2024-04-12T20:39:39.846 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:820230223114759:b12195ea.x86_64 from CentOS-AppStream 2024-04-12T20:39:40.070 INFO:teuthology.orchestra.run.smithi018.stdout:(1/24): ceph-base-17.2.7-827.gd3b5060a.el9.x86_ 5.9 MB/s | 5.0 MB 00:00 2024-04-12T20:39:40.162 INFO:teuthology.orchestra.run.smithi018.stdout:(2/24): ceph-selinux-17.2.7-827.gd3b5060a.el9.x 278 kB/s | 25 kB 00:00 2024-04-12T20:39:40.529 INFO:teuthology.orchestra.run.smithi018.stdout:(3/24): libcephfs2-17.2.7-827.gd3b5060a.el9.x86 1.7 MB/s | 637 kB 00:00 2024-04-12T20:39:40.558 INFO:teuthology.orchestra.run.smithi186.stdout:Dependencies resolved. 2024-04-12T20:39:40.569 INFO:teuthology.orchestra.run.smithi186.stdout:================================================================================ 2024-04-12T20:39:40.569 INFO:teuthology.orchestra.run.smithi186.stdout: Package Arch Version Repository Size 2024-04-12T20:39:40.570 INFO:teuthology.orchestra.run.smithi186.stdout:================================================================================ 2024-04-12T20:39:40.570 INFO:teuthology.orchestra.run.smithi186.stdout:Installing: 2024-04-12T20:39:40.570 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-radosgw x86_64 2:17.2.7-827.gd3b5060a.el9 ceph 14 M 2024-04-12T20:39:40.570 INFO:teuthology.orchestra.run.smithi186.stdout:Upgrading: 2024-04-12T20:39:40.570 INFO:teuthology.orchestra.run.smithi186.stdout: librados2 x86_64 2:17.2.7-827.gd3b5060a.el9 ceph 3.1 M 2024-04-12T20:39:40.570 INFO:teuthology.orchestra.run.smithi186.stdout: librbd1 x86_64 2:17.2.7-827.gd3b5060a.el9 ceph 2.9 M 2024-04-12T20:39:40.570 INFO:teuthology.orchestra.run.smithi186.stdout:Installing dependencies: 2024-04-12T20:39:40.570 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-base x86_64 2:17.2.7-827.gd3b5060a.el9 ceph 5.0 M 2024-04-12T20:39:40.570 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-common x86_64 2:17.2.7-827.gd3b5060a.el9 ceph 22 M 2024-04-12T20:39:40.570 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-selinux x86_64 2:17.2.7-827.gd3b5060a.el9 ceph 25 k 2024-04-12T20:39:40.570 INFO:teuthology.orchestra.run.smithi186.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-04-12T20:39:40.570 INFO:teuthology.orchestra.run.smithi186.stdout: libcephfs2 x86_64 2:17.2.7-827.gd3b5060a.el9 ceph 637 k 2024-04-12T20:39:40.570 INFO:teuthology.orchestra.run.smithi186.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-04-12T20:39:40.571 INFO:teuthology.orchestra.run.smithi186.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-04-12T20:39:40.571 INFO:teuthology.orchestra.run.smithi186.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-04-12T20:39:40.571 INFO:teuthology.orchestra.run.smithi186.stdout: libradosstriper1 x86_64 2:17.2.7-827.gd3b5060a.el9 ceph 458 k 2024-04-12T20:39:40.571 INFO:teuthology.orchestra.run.smithi186.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-04-12T20:39:40.571 INFO:teuthology.orchestra.run.smithi186.stdout: librgw2 x86_64 2:17.2.7-827.gd3b5060a.el9 ceph 4.9 M 2024-04-12T20:39:40.571 INFO:teuthology.orchestra.run.smithi186.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-04-12T20:39:40.571 INFO:teuthology.orchestra.run.smithi186.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-04-12T20:39:40.571 INFO:teuthology.orchestra.run.smithi186.stdout: python3-ceph-argparse x86_64 2:17.2.7-827.gd3b5060a.el9 ceph 45 k 2024-04-12T20:39:40.571 INFO:teuthology.orchestra.run.smithi186.stdout: python3-ceph-common x86_64 2:17.2.7-827.gd3b5060a.el9 ceph 117 k 2024-04-12T20:39:40.571 INFO:teuthology.orchestra.run.smithi186.stdout: python3-cephfs x86_64 2:17.2.7-827.gd3b5060a.el9 ceph 156 k 2024-04-12T20:39:40.571 INFO:teuthology.orchestra.run.smithi186.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-04-12T20:39:40.571 INFO:teuthology.orchestra.run.smithi186.stdout: python3-rados x86_64 2:17.2.7-827.gd3b5060a.el9 ceph 320 k 2024-04-12T20:39:40.571 INFO:teuthology.orchestra.run.smithi186.stdout: python3-rbd x86_64 2:17.2.7-827.gd3b5060a.el9 ceph 292 k 2024-04-12T20:39:40.571 INFO:teuthology.orchestra.run.smithi186.stdout: python3-rgw x86_64 2:17.2.7-827.gd3b5060a.el9 ceph 99 k 2024-04-12T20:39:40.572 INFO:teuthology.orchestra.run.smithi186.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-04-12T20:39:40.572 INFO:teuthology.orchestra.run.smithi186.stdout: 2024-04-12T20:39:40.572 INFO:teuthology.orchestra.run.smithi186.stdout:Transaction Summary 2024-04-12T20:39:40.572 INFO:teuthology.orchestra.run.smithi186.stdout:================================================================================ 2024-04-12T20:39:40.572 INFO:teuthology.orchestra.run.smithi186.stdout:Install 22 Packages 2024-04-12T20:39:40.572 INFO:teuthology.orchestra.run.smithi186.stdout:Upgrade 2 Packages 2024-04-12T20:39:40.572 INFO:teuthology.orchestra.run.smithi186.stdout: 2024-04-12T20:39:40.572 INFO:teuthology.orchestra.run.smithi186.stdout:Total download size: 58 M 2024-04-12T20:39:40.572 INFO:teuthology.orchestra.run.smithi186.stdout:Downloading Packages: 2024-04-12T20:39:40.621 INFO:teuthology.orchestra.run.smithi018.stdout:(4/24): libradosstriper1-17.2.7-827.gd3b5060a.e 4.9 MB/s | 458 kB 00:00 2024-04-12T20:39:41.163 INFO:teuthology.orchestra.run.smithi018.stdout:(5/24): librgw2-17.2.7-827.gd3b5060a.el9.x86_64 9.1 MB/s | 4.9 MB 00:00 2024-04-12T20:39:41.221 INFO:teuthology.orchestra.run.smithi018.stdout:(6/24): python3-ceph-argparse-17.2.7-827.gd3b50 784 kB/s | 45 kB 00:00 2024-04-12T20:39:41.497 INFO:teuthology.orchestra.run.smithi018.stdout:(7/24): ceph-common-17.2.7-827.gd3b5060a.el9.x8 9.9 MB/s | 22 MB 00:02 2024-04-12T20:39:41.530 INFO:teuthology.orchestra.run.smithi018.stdout:(8/24): python3-ceph-common-17.2.7-827.gd3b5060 382 kB/s | 117 kB 00:00 2024-04-12T20:39:41.739 INFO:teuthology.orchestra.run.smithi018.stdout:(9/24): ceph-radosgw-17.2.7-827.gd3b5060a.el9.x 5.6 MB/s | 14 MB 00:02 2024-04-12T20:39:41.798 INFO:teuthology.orchestra.run.smithi018.stdout:(10/24): python3-cephfs-17.2.7-827.gd3b5060a.el 521 kB/s | 156 kB 00:00 2024-04-12T20:39:41.848 INFO:teuthology.orchestra.run.smithi018.stdout:(11/24): python3-rbd-17.2.7-827.gd3b5060a.el9.x 2.7 MB/s | 292 kB 00:00 2024-04-12T20:39:41.873 INFO:teuthology.orchestra.run.smithi018.stdout:(12/24): python3-rgw-17.2.7-827.gd3b5060a.el9.x 1.3 MB/s | 99 kB 00:00 2024-04-12T20:39:41.874 INFO:teuthology.orchestra.run.smithi186.stdout:(1/24): ceph-base-17.2.7-827.gd3b5060a.el9.x86_ 6.4 MB/s | 5.0 MB 00:00 2024-04-12T20:39:41.916 INFO:teuthology.orchestra.run.smithi186.stdout:(2/24): ceph-selinux-17.2.7-827.gd3b5060a.el9.x 621 kB/s | 25 kB 00:00 2024-04-12T20:39:42.000 INFO:teuthology.orchestra.run.smithi186.stdout:(3/24): libcephfs2-17.2.7-827.gd3b5060a.el9.x86 7.6 MB/s | 637 kB 00:00 2024-04-12T20:39:42.007 INFO:teuthology.orchestra.run.smithi018.stdout:(13/24): librabbitmq-0.11.0-7.el9.x86_64.rpm 342 kB/s | 45 kB 00:00 2024-04-12T20:39:42.033 INFO:teuthology.orchestra.run.smithi018.stdout:(14/24): libpmemobj-1.12.1-1.el9.x86_64.rpm 877 kB/s | 160 kB 00:00 2024-04-12T20:39:42.066 INFO:teuthology.orchestra.run.smithi018.stdout:(15/24): python3-rados-17.2.7-827.gd3b5060a.el9 599 kB/s | 320 kB 00:00 2024-04-12T20:39:42.092 INFO:teuthology.orchestra.run.smithi186.stdout:(4/24): libradosstriper1-17.2.7-827.gd3b5060a.e 4.9 MB/s | 458 kB 00:00 2024-04-12T20:39:42.125 INFO:teuthology.orchestra.run.smithi018.stdout:(16/24): librdkafka-1.6.1-102.el9.x86_64.rpm 5.5 MB/s | 662 kB 00:00 2024-04-12T20:39:42.158 INFO:teuthology.orchestra.run.smithi018.stdout:(17/24): lttng-ust-2.12.0-6.el9.x86_64.rpm 2.3 MB/s | 292 kB 00:00 2024-04-12T20:39:42.275 INFO:teuthology.orchestra.run.smithi018.stdout:(18/24): python3-prettytable-0.7.2-27.el9.noarc 203 kB/s | 42 kB 00:00 2024-04-12T20:39:42.434 INFO:teuthology.orchestra.run.smithi018.stdout:(19/24): gperftools-libs-2.9.1-2.el9.x86_64.rpm 1.0 MB/s | 309 kB 00:00 2024-04-12T20:39:42.459 INFO:teuthology.orchestra.run.smithi018.stdout:(20/24): libunwind-1.6.2-1.el9.x86_64.rpm 563 kB/s | 67 kB 00:00 2024-04-12T20:39:42.501 INFO:teuthology.orchestra.run.smithi018.stdout:(21/24): liboath-2.6.7-2.el9.x86_64.rpm 179 kB/s | 49 kB 00:00 2024-04-12T20:39:42.576 INFO:teuthology.orchestra.run.smithi186.stdout:(5/24): librgw2-17.2.7-827.gd3b5060a.el9.x86_64 10 MB/s | 4.9 MB 00:00 2024-04-12T20:39:43.018 INFO:teuthology.orchestra.run.smithi018.stdout:(22/24): librados2-17.2.7-827.gd3b5060a.el9.x86 5.5 MB/s | 3.1 MB 00:00 2024-04-12T20:39:43.026 INFO:teuthology.orchestra.run.smithi186.stdout:(6/24): ceph-common-17.2.7-827.gd3b5060a.el9.x8 12 MB/s | 22 MB 00:01 2024-04-12T20:39:43.082 INFO:teuthology.orchestra.run.smithi186.stdout:(7/24): python3-ceph-argparse-17.2.7-827.gd3b50 89 kB/s | 45 kB 00:00 2024-04-12T20:39:43.118 INFO:teuthology.orchestra.run.smithi186.stdout:(8/24): python3-ceph-common-17.2.7-827.gd3b5060 1.3 MB/s | 117 kB 00:00 2024-04-12T20:39:43.151 INFO:teuthology.orchestra.run.smithi186.stdout:(9/24): python3-cephfs-17.2.7-827.gd3b5060a.el9 2.2 MB/s | 156 kB 00:00 2024-04-12T20:39:43.316 INFO:teuthology.orchestra.run.smithi186.stdout:(10/24): python3-rados-17.2.7-827.gd3b5060a.el9 4.2 MB/s | 320 kB 00:00 2024-04-12T20:39:43.316 INFO:teuthology.orchestra.run.smithi186.stdout:(11/24): python3-rbd-17.2.7-827.gd3b5060a.el9.x 3.8 MB/s | 292 kB 00:00 2024-04-12T20:39:43.316 INFO:teuthology.orchestra.run.smithi186.stdout:(12/24): python3-rgw-17.2.7-827.gd3b5060a.el9.x 1.5 MB/s | 99 kB 00:00 2024-04-12T20:39:43.317 INFO:teuthology.orchestra.run.smithi018.stdout:(23/24): librbd1-17.2.7-827.gd3b5060a.el9.x86_6 4.1 MB/s | 2.9 MB 00:00 2024-04-12T20:39:43.317 INFO:teuthology.orchestra.run.smithi018.stdout:(24/24): thrift-0.15.0-2.el9.x86_64.rpm 1.8 MB/s | 1.6 MB 00:00 2024-04-12T20:39:43.317 INFO:teuthology.orchestra.run.smithi018.stdout:-------------------------------------------------------------------------------- 2024-04-12T20:39:43.317 INFO:teuthology.orchestra.run.smithi018.stdout:Total 12 MB/s | 58 MB 00:04 2024-04-12T20:39:43.502 INFO:teuthology.orchestra.run.smithi018.stdout:Running transaction check 2024-04-12T20:39:43.581 INFO:teuthology.orchestra.run.smithi018.stdout:Transaction check succeeded. 2024-04-12T20:39:43.581 INFO:teuthology.orchestra.run.smithi018.stdout:Running transaction test 2024-04-12T20:39:43.969 INFO:teuthology.orchestra.run.smithi186.stdout:(13/24): librabbitmq-0.11.0-7.el9.x86_64.rpm 64 kB/s | 45 kB 00:00 2024-04-12T20:39:44.119 INFO:teuthology.orchestra.run.smithi186.stdout:(14/24): libpmemobj-1.12.1-1.el9.x86_64.rpm 180 kB/s | 160 kB 00:00 2024-04-12T20:39:44.133 INFO:teuthology.orchestra.run.smithi018.stdout:Transaction test succeeded. 2024-04-12T20:39:44.133 INFO:teuthology.orchestra.run.smithi018.stdout:Running transaction 2024-04-12T20:39:45.145 INFO:teuthology.orchestra.run.smithi186.stdout:(15/24): lttng-ust-2.12.0-6.el9.x86_64.rpm 286 kB/s | 292 kB 00:01 2024-04-12T20:39:45.487 INFO:teuthology.orchestra.run.smithi186.stdout:(16/24): librdkafka-1.6.1-102.el9.x86_64.rpm 437 kB/s | 662 kB 00:01 2024-04-12T20:39:45.560 INFO:teuthology.orchestra.run.smithi018.stdout: Preparing : 1/1 2024-04-12T20:39:45.969 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 1/26 2024-04-12T20:39:45.970 INFO:teuthology.orchestra.run.smithi186.stdout:(17/24): ceph-radosgw-17.2.7-827.gd3b5060a.el9. 2.9 MB/s | 14 MB 00:04 2024-04-12T20:39:46.004 INFO:teuthology.orchestra.run.smithi186.stdout:(18/24): python3-prettytable-0.7.2-27.el9.noarc 49 kB/s | 42 kB 00:00 2024-04-12T20:39:46.012 INFO:teuthology.orchestra.run.smithi018.stdout: Upgrading : librados2-2:17.2.7-827.gd3b5060a.el9.x86_64 2/26 2024-04-12T20:39:46.190 INFO:teuthology.orchestra.run.smithi018.stdout: Running scriptlet: librados2-2:17.2.7-827.gd3b5060a.el9.x86_64 2/26 2024-04-12T20:39:46.262 INFO:teuthology.orchestra.run.smithi186.stdout:(19/24): gperftools-libs-2.9.1-2.el9.x86_64.rpm 399 kB/s | 309 kB 00:00 2024-04-12T20:39:46.294 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : python3-rados-2:17.2.7-827.gd3b5060a.el9.x86_64 3/26 2024-04-12T20:39:46.321 INFO:teuthology.orchestra.run.smithi186.stdout:(20/24): liboath-2.6.7-2.el9.x86_64.rpm 139 kB/s | 49 kB 00:00 2024-04-12T20:39:46.321 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : libcephfs2-2:17.2.7-827.gd3b5060a.el9.x86_64 4/26 2024-04-12T20:39:46.363 INFO:teuthology.orchestra.run.smithi186.stdout:(21/24): libunwind-1.6.2-1.el9.x86_64.rpm 188 kB/s | 67 kB 00:00 2024-04-12T20:39:46.486 INFO:teuthology.orchestra.run.smithi018.stdout: Running scriptlet: libcephfs2-2:17.2.7-827.gd3b5060a.el9.x86_64 4/26 2024-04-12T20:39:46.546 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : thrift-0.15.0-2.el9.x86_64 5/26 2024-04-12T20:39:46.587 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 6/26 2024-04-12T20:39:46.622 INFO:teuthology.orchestra.run.smithi186.stdout:(22/24): librados2-17.2.7-827.gd3b5060a.el9.x86 10 MB/s | 3.1 MB 00:00 2024-04-12T20:39:46.722 INFO:teuthology.orchestra.run.smithi186.stdout:(23/24): librbd1-17.2.7-827.gd3b5060a.el9.x86_6 8.2 MB/s | 2.9 MB 00:00 2024-04-12T20:39:46.830 INFO:teuthology.orchestra.run.smithi186.stdout:(24/24): thrift-0.15.0-2.el9.x86_64.rpm 2.8 MB/s | 1.6 MB 00:00 2024-04-12T20:39:46.836 INFO:teuthology.orchestra.run.smithi186.stdout:-------------------------------------------------------------------------------- 2024-04-12T20:39:46.836 INFO:teuthology.orchestra.run.smithi186.stdout:Total 9.2 MB/s | 58 MB 00:06 2024-04-12T20:39:47.037 INFO:teuthology.orchestra.run.smithi186.stdout:Running transaction check 2024-04-12T20:39:47.115 INFO:teuthology.orchestra.run.smithi186.stdout:Transaction check succeeded. 2024-04-12T20:39:47.115 INFO:teuthology.orchestra.run.smithi186.stdout:Running transaction test 2024-04-12T20:39:47.170 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 7/26 2024-04-12T20:39:47.200 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : librgw2-2:17.2.7-827.gd3b5060a.el9.x86_64 8/26 2024-04-12T20:39:47.283 INFO:teuthology.orchestra.run.smithi018.stdout: Running scriptlet: librgw2-2:17.2.7-827.gd3b5060a.el9.x86_64 8/26 2024-04-12T20:39:47.318 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : liboath-2.6.7-2.el9.x86_64 9/26 2024-04-12T20:39:47.378 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : python3-ceph-argparse-2:17.2.7-827.gd3b5060a.el9.x 10/26 2024-04-12T20:39:47.425 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : python3-cephfs-2:17.2.7-827.gd3b5060a.el9.x86_64 11/26 2024-04-12T20:39:47.529 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : python3-rgw-2:17.2.7-827.gd3b5060a.el9.x86_64 12/26 2024-04-12T20:39:47.561 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : libradosstriper1-2:17.2.7-827.gd3b5060a.el9.x86_64 13/26 2024-04-12T20:39:47.673 INFO:teuthology.orchestra.run.smithi186.stdout:Transaction test succeeded. 2024-04-12T20:39:47.673 INFO:teuthology.orchestra.run.smithi186.stdout:Running transaction 2024-04-12T20:39:47.697 INFO:teuthology.orchestra.run.smithi018.stdout: Running scriptlet: libradosstriper1-2:17.2.7-827.gd3b5060a.el9.x86_64 13/26 2024-04-12T20:39:47.775 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 14/26 2024-04-12T20:39:47.825 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 15/26 2024-04-12T20:39:47.873 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 16/26 2024-04-12T20:39:48.272 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 17/26 2024-04-12T20:39:48.306 INFO:teuthology.orchestra.run.smithi018.stdout: Upgrading : librbd1-2:17.2.7-827.gd3b5060a.el9.x86_64 18/26 2024-04-12T20:39:48.496 INFO:teuthology.orchestra.run.smithi018.stdout: Running scriptlet: librbd1-2:17.2.7-827.gd3b5060a.el9.x86_64 18/26 2024-04-12T20:39:48.606 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : python3-rbd-2:17.2.7-827.gd3b5060a.el9.x86_64 19/26 2024-04-12T20:39:48.634 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : python3-ceph-common-2:17.2.7-827.gd3b5060a.el9.x86 20/26 2024-04-12T20:39:49.064 INFO:teuthology.orchestra.run.smithi186.stdout: Preparing : 1/1 2024-04-12T20:39:49.518 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 1/26 2024-04-12T20:39:49.566 INFO:teuthology.orchestra.run.smithi186.stdout: Upgrading : librados2-2:17.2.7-827.gd3b5060a.el9.x86_64 2/26 2024-04-12T20:39:49.689 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: librados2-2:17.2.7-827.gd3b5060a.el9.x86_64 2/26 2024-04-12T20:39:49.785 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-rados-2:17.2.7-827.gd3b5060a.el9.x86_64 3/26 2024-04-12T20:39:49.817 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : libcephfs2-2:17.2.7-827.gd3b5060a.el9.x86_64 4/26 2024-04-12T20:39:49.945 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: libcephfs2-2:17.2.7-827.gd3b5060a.el9.x86_64 4/26 2024-04-12T20:39:50.005 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : thrift-0.15.0-2.el9.x86_64 5/26 2024-04-12T20:39:50.049 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 6/26 2024-04-12T20:39:50.593 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 7/26 2024-04-12T20:39:50.637 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : librgw2-2:17.2.7-827.gd3b5060a.el9.x86_64 8/26 2024-04-12T20:39:50.720 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: librgw2-2:17.2.7-827.gd3b5060a.el9.x86_64 8/26 2024-04-12T20:39:50.755 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : liboath-2.6.7-2.el9.x86_64 9/26 2024-04-12T20:39:50.817 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-ceph-argparse-2:17.2.7-827.gd3b5060a.el9.x 10/26 2024-04-12T20:39:50.861 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-cephfs-2:17.2.7-827.gd3b5060a.el9.x86_64 11/26 2024-04-12T20:39:50.948 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-rgw-2:17.2.7-827.gd3b5060a.el9.x86_64 12/26 2024-04-12T20:39:50.990 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : libradosstriper1-2:17.2.7-827.gd3b5060a.el9.x86_64 13/26 2024-04-12T20:39:51.090 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: libradosstriper1-2:17.2.7-827.gd3b5060a.el9.x86_64 13/26 2024-04-12T20:39:51.153 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 14/26 2024-04-12T20:39:51.199 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 15/26 2024-04-12T20:39:51.240 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 16/26 2024-04-12T20:39:51.583 INFO:teuthology.orchestra.run.smithi018.stdout: Running scriptlet: ceph-common-2:17.2.7-827.gd3b5060a.el9.x86_64 21/26 2024-04-12T20:39:51.632 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 17/26 2024-04-12T20:39:51.660 INFO:teuthology.orchestra.run.smithi186.stdout: Upgrading : librbd1-2:17.2.7-827.gd3b5060a.el9.x86_64 18/26 2024-04-12T20:39:51.764 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : ceph-common-2:17.2.7-827.gd3b5060a.el9.x86_64 21/26 2024-04-12T20:39:51.774 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: librbd1-2:17.2.7-827.gd3b5060a.el9.x86_64 18/26 2024-04-12T20:39:51.869 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-rbd-2:17.2.7-827.gd3b5060a.el9.x86_64 19/26 2024-04-12T20:39:51.896 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-ceph-common-2:17.2.7-827.gd3b5060a.el9.x86 20/26 2024-04-12T20:39:52.547 INFO:teuthology.orchestra.run.smithi018.stdout: Running scriptlet: ceph-common-2:17.2.7-827.gd3b5060a.el9.x86_64 21/26 2024-04-12T20:39:52.592 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : ceph-base-2:17.2.7-827.gd3b5060a.el9.x86_64 22/26 2024-04-12T20:39:52.779 INFO:teuthology.orchestra.run.smithi018.stdout: Running scriptlet: ceph-base-2:17.2.7-827.gd3b5060a.el9.x86_64 22/26 2024-04-12T20:39:52.779 INFO:teuthology.orchestra.run.smithi018.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-04-12T20:39:52.779 INFO:teuthology.orchestra.run.smithi018.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-04-12T20:39:52.779 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-12T20:39:52.843 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : ceph-selinux-2:17.2.7-827.gd3b5060a.el9.x86_64 23/26 2024-04-12T20:39:54.990 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: ceph-common-2:17.2.7-827.gd3b5060a.el9.x86_64 21/26 2024-04-12T20:39:55.268 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : ceph-common-2:17.2.7-827.gd3b5060a.el9.x86_64 21/26 2024-04-12T20:39:55.930 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: ceph-common-2:17.2.7-827.gd3b5060a.el9.x86_64 21/26 2024-04-12T20:39:55.972 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : ceph-base-2:17.2.7-827.gd3b5060a.el9.x86_64 22/26 2024-04-12T20:39:56.194 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: ceph-base-2:17.2.7-827.gd3b5060a.el9.x86_64 22/26 2024-04-12T20:39:56.194 INFO:teuthology.orchestra.run.smithi186.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-04-12T20:39:56.194 INFO:teuthology.orchestra.run.smithi186.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-04-12T20:39:56.194 INFO:teuthology.orchestra.run.smithi186.stdout: 2024-04-12T20:39:56.239 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : ceph-selinux-2:17.2.7-827.gd3b5060a.el9.x86_64 23/26 2024-04-12T20:40:04.485 INFO:teuthology.orchestra.run.smithi018.stdout: Running scriptlet: ceph-selinux-2:17.2.7-827.gd3b5060a.el9.x86_64 23/26 2024-04-12T20:40:04.485 INFO:teuthology.orchestra.run.smithi018.stdout:skipping the directory /sys 2024-04-12T20:40:04.486 INFO:teuthology.orchestra.run.smithi018.stdout:skipping the directory /proc 2024-04-12T20:40:04.486 INFO:teuthology.orchestra.run.smithi018.stdout:skipping the directory /mnt 2024-04-12T20:40:04.486 INFO:teuthology.orchestra.run.smithi018.stdout:skipping the directory /var/tmp 2024-04-12T20:40:04.486 INFO:teuthology.orchestra.run.smithi018.stdout:skipping the directory /home 2024-04-12T20:40:04.486 INFO:teuthology.orchestra.run.smithi018.stdout:skipping the directory /root 2024-04-12T20:40:04.486 INFO:teuthology.orchestra.run.smithi018.stdout:skipping the directory /tmp 2024-04-12T20:40:04.486 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-12T20:40:05.980 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : ceph-radosgw-2:17.2.7-827.gd3b5060a.el9.x86_64 24/26 2024-04-12T20:40:06.118 INFO:teuthology.orchestra.run.smithi018.stdout: Running scriptlet: ceph-radosgw-2:17.2.7-827.gd3b5060a.el9.x86_64 24/26 2024-04-12T20:40:06.118 INFO:teuthology.orchestra.run.smithi018.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-12T20:40:06.118 INFO:teuthology.orchestra.run.smithi018.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-04-12T20:40:06.119 INFO:teuthology.orchestra.run.smithi018.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-12T20:40:06.119 INFO:teuthology.orchestra.run.smithi018.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-12T20:40:06.119 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-12T20:40:06.123 INFO:teuthology.orchestra.run.smithi018.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 25/26 2024-04-12T20:40:06.252 INFO:teuthology.orchestra.run.smithi018.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 25/26 2024-04-12T20:40:06.252 INFO:teuthology.orchestra.run.smithi018.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 26/26 2024-04-12T20:40:06.924 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: ceph-selinux-2:17.2.7-827.gd3b5060a.el9.x86_64 23/26 2024-04-12T20:40:06.924 INFO:teuthology.orchestra.run.smithi186.stdout:skipping the directory /sys 2024-04-12T20:40:06.924 INFO:teuthology.orchestra.run.smithi186.stdout:skipping the directory /proc 2024-04-12T20:40:06.924 INFO:teuthology.orchestra.run.smithi186.stdout:skipping the directory /mnt 2024-04-12T20:40:06.924 INFO:teuthology.orchestra.run.smithi186.stdout:skipping the directory /var/tmp 2024-04-12T20:40:06.925 INFO:teuthology.orchestra.run.smithi186.stdout:skipping the directory /home 2024-04-12T20:40:06.925 INFO:teuthology.orchestra.run.smithi186.stdout:skipping the directory /root 2024-04-12T20:40:06.925 INFO:teuthology.orchestra.run.smithi186.stdout:skipping the directory /tmp 2024-04-12T20:40:06.925 INFO:teuthology.orchestra.run.smithi186.stdout: 2024-04-12T20:40:08.040 INFO:teuthology.orchestra.run.smithi018.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 26/26 2024-04-12T20:40:08.041 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : ceph-base-2:17.2.7-827.gd3b5060a.el9.x86_64 1/26 2024-04-12T20:40:08.041 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : ceph-common-2:17.2.7-827.gd3b5060a.el9.x86_64 2/26 2024-04-12T20:40:08.041 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : ceph-radosgw-2:17.2.7-827.gd3b5060a.el9.x86_64 3/26 2024-04-12T20:40:08.041 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : ceph-selinux-2:17.2.7-827.gd3b5060a.el9.x86_64 4/26 2024-04-12T20:40:08.041 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : libcephfs2-2:17.2.7-827.gd3b5060a.el9.x86_64 5/26 2024-04-12T20:40:08.041 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : libradosstriper1-2:17.2.7-827.gd3b5060a.el9.x86_64 6/26 2024-04-12T20:40:08.041 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : librgw2-2:17.2.7-827.gd3b5060a.el9.x86_64 7/26 2024-04-12T20:40:08.041 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : python3-ceph-argparse-2:17.2.7-827.gd3b5060a.el9.x 8/26 2024-04-12T20:40:08.041 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : python3-ceph-common-2:17.2.7-827.gd3b5060a.el9.x86 9/26 2024-04-12T20:40:08.041 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : python3-cephfs-2:17.2.7-827.gd3b5060a.el9.x86_64 10/26 2024-04-12T20:40:08.042 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : python3-rados-2:17.2.7-827.gd3b5060a.el9.x86_64 11/26 2024-04-12T20:40:08.042 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : python3-rbd-2:17.2.7-827.gd3b5060a.el9.x86_64 12/26 2024-04-12T20:40:08.042 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : python3-rgw-2:17.2.7-827.gd3b5060a.el9.x86_64 13/26 2024-04-12T20:40:08.042 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 14/26 2024-04-12T20:40:08.042 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 15/26 2024-04-12T20:40:08.042 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 16/26 2024-04-12T20:40:08.042 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 17/26 2024-04-12T20:40:08.042 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 18/26 2024-04-12T20:40:08.042 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 19/26 2024-04-12T20:40:08.043 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 20/26 2024-04-12T20:40:08.043 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 21/26 2024-04-12T20:40:08.043 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 22/26 2024-04-12T20:40:08.043 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : librados2-2:17.2.7-827.gd3b5060a.el9.x86_64 23/26 2024-04-12T20:40:08.043 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 24/26 2024-04-12T20:40:08.043 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : librbd1-2:17.2.7-827.gd3b5060a.el9.x86_64 25/26 2024-04-12T20:40:08.300 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : ceph-radosgw-2:17.2.7-827.gd3b5060a.el9.x86_64 24/26 2024-04-12T20:40:08.437 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: ceph-radosgw-2:17.2.7-827.gd3b5060a.el9.x86_64 24/26 2024-04-12T20:40:08.438 INFO:teuthology.orchestra.run.smithi186.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-12T20:40:08.438 INFO:teuthology.orchestra.run.smithi186.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-04-12T20:40:08.438 INFO:teuthology.orchestra.run.smithi186.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-12T20:40:08.438 INFO:teuthology.orchestra.run.smithi186.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-12T20:40:08.438 INFO:teuthology.orchestra.run.smithi186.stdout: 2024-04-12T20:40:08.442 INFO:teuthology.orchestra.run.smithi186.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 25/26 2024-04-12T20:40:08.552 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 25/26 2024-04-12T20:40:08.553 INFO:teuthology.orchestra.run.smithi186.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 26/26 2024-04-12T20:40:08.912 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 26/26 2024-04-12T20:40:08.912 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-12T20:40:08.912 INFO:teuthology.orchestra.run.smithi018.stdout:Upgraded: 2024-04-12T20:40:08.912 INFO:teuthology.orchestra.run.smithi018.stdout: librados2-2:17.2.7-827.gd3b5060a.el9.x86_64 2024-04-12T20:40:08.912 INFO:teuthology.orchestra.run.smithi018.stdout: librbd1-2:17.2.7-827.gd3b5060a.el9.x86_64 2024-04-12T20:40:08.912 INFO:teuthology.orchestra.run.smithi018.stdout:Installed: 2024-04-12T20:40:08.913 INFO:teuthology.orchestra.run.smithi018.stdout: ceph-base-2:17.2.7-827.gd3b5060a.el9.x86_64 2024-04-12T20:40:08.913 INFO:teuthology.orchestra.run.smithi018.stdout: ceph-common-2:17.2.7-827.gd3b5060a.el9.x86_64 2024-04-12T20:40:08.913 INFO:teuthology.orchestra.run.smithi018.stdout: ceph-radosgw-2:17.2.7-827.gd3b5060a.el9.x86_64 2024-04-12T20:40:08.913 INFO:teuthology.orchestra.run.smithi018.stdout: ceph-selinux-2:17.2.7-827.gd3b5060a.el9.x86_64 2024-04-12T20:40:08.913 INFO:teuthology.orchestra.run.smithi018.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-04-12T20:40:08.913 INFO:teuthology.orchestra.run.smithi018.stdout: libcephfs2-2:17.2.7-827.gd3b5060a.el9.x86_64 2024-04-12T20:40:08.913 INFO:teuthology.orchestra.run.smithi018.stdout: liboath-2.6.7-2.el9.x86_64 2024-04-12T20:40:08.913 INFO:teuthology.orchestra.run.smithi018.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-04-12T20:40:08.913 INFO:teuthology.orchestra.run.smithi018.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-04-12T20:40:08.913 INFO:teuthology.orchestra.run.smithi018.stdout: libradosstriper1-2:17.2.7-827.gd3b5060a.el9.x86_64 2024-04-12T20:40:08.913 INFO:teuthology.orchestra.run.smithi018.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-04-12T20:40:08.913 INFO:teuthology.orchestra.run.smithi018.stdout: librgw2-2:17.2.7-827.gd3b5060a.el9.x86_64 2024-04-12T20:40:08.913 INFO:teuthology.orchestra.run.smithi018.stdout: libunwind-1.6.2-1.el9.x86_64 2024-04-12T20:40:08.914 INFO:teuthology.orchestra.run.smithi018.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-04-12T20:40:08.914 INFO:teuthology.orchestra.run.smithi018.stdout: python3-ceph-argparse-2:17.2.7-827.gd3b5060a.el9.x86_64 2024-04-12T20:40:08.914 INFO:teuthology.orchestra.run.smithi018.stdout: python3-ceph-common-2:17.2.7-827.gd3b5060a.el9.x86_64 2024-04-12T20:40:08.914 INFO:teuthology.orchestra.run.smithi018.stdout: python3-cephfs-2:17.2.7-827.gd3b5060a.el9.x86_64 2024-04-12T20:40:08.914 INFO:teuthology.orchestra.run.smithi018.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-04-12T20:40:08.914 INFO:teuthology.orchestra.run.smithi018.stdout: python3-rados-2:17.2.7-827.gd3b5060a.el9.x86_64 2024-04-12T20:40:08.914 INFO:teuthology.orchestra.run.smithi018.stdout: python3-rbd-2:17.2.7-827.gd3b5060a.el9.x86_64 2024-04-12T20:40:08.914 INFO:teuthology.orchestra.run.smithi018.stdout: python3-rgw-2:17.2.7-827.gd3b5060a.el9.x86_64 2024-04-12T20:40:08.914 INFO:teuthology.orchestra.run.smithi018.stdout: thrift-0.15.0-2.el9.x86_64 2024-04-12T20:40:08.914 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-12T20:40:08.914 INFO:teuthology.orchestra.run.smithi018.stdout:Complete! 2024-04-12T20:40:09.413 DEBUG:teuthology.orchestra.run.smithi018:> sudo yum -y install ceph-test 2024-04-12T20:40:10.274 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 26/26 2024-04-12T20:40:10.274 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : ceph-base-2:17.2.7-827.gd3b5060a.el9.x86_64 1/26 2024-04-12T20:40:10.274 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : ceph-common-2:17.2.7-827.gd3b5060a.el9.x86_64 2/26 2024-04-12T20:40:10.274 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : ceph-radosgw-2:17.2.7-827.gd3b5060a.el9.x86_64 3/26 2024-04-12T20:40:10.275 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : ceph-selinux-2:17.2.7-827.gd3b5060a.el9.x86_64 4/26 2024-04-12T20:40:10.275 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : libcephfs2-2:17.2.7-827.gd3b5060a.el9.x86_64 5/26 2024-04-12T20:40:10.275 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : libradosstriper1-2:17.2.7-827.gd3b5060a.el9.x86_64 6/26 2024-04-12T20:40:10.275 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : librgw2-2:17.2.7-827.gd3b5060a.el9.x86_64 7/26 2024-04-12T20:40:10.275 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-ceph-argparse-2:17.2.7-827.gd3b5060a.el9.x 8/26 2024-04-12T20:40:10.275 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-ceph-common-2:17.2.7-827.gd3b5060a.el9.x86 9/26 2024-04-12T20:40:10.275 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-cephfs-2:17.2.7-827.gd3b5060a.el9.x86_64 10/26 2024-04-12T20:40:10.275 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-rados-2:17.2.7-827.gd3b5060a.el9.x86_64 11/26 2024-04-12T20:40:10.275 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-rbd-2:17.2.7-827.gd3b5060a.el9.x86_64 12/26 2024-04-12T20:40:10.275 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-rgw-2:17.2.7-827.gd3b5060a.el9.x86_64 13/26 2024-04-12T20:40:10.275 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 14/26 2024-04-12T20:40:10.275 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 15/26 2024-04-12T20:40:10.275 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 16/26 2024-04-12T20:40:10.275 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 17/26 2024-04-12T20:40:10.276 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 18/26 2024-04-12T20:40:10.276 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 19/26 2024-04-12T20:40:10.276 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 20/26 2024-04-12T20:40:10.276 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 21/26 2024-04-12T20:40:10.276 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 22/26 2024-04-12T20:40:10.276 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : librados2-2:17.2.7-827.gd3b5060a.el9.x86_64 23/26 2024-04-12T20:40:10.276 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 24/26 2024-04-12T20:40:10.276 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : librbd1-2:17.2.7-827.gd3b5060a.el9.x86_64 25/26 2024-04-12T20:40:10.322 INFO:teuthology.orchestra.run.smithi018.stdout:Last metadata expiration check: 0:00:35 ago on Fri 12 Apr 2024 08:39:35 PM UTC. 2024-04-12T20:40:10.632 INFO:teuthology.orchestra.run.smithi018.stderr:Modular dependency problems: 2024-04-12T20:40:10.632 INFO:teuthology.orchestra.run.smithi018.stderr: 2024-04-12T20:40:10.632 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 1: conflicting requests 2024-04-12T20:40:10.632 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module container-tools:rhel8:820240321091303:20125149.x86_64 from CentOS-AppStream 2024-04-12T20:40:10.632 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 2: conflicting requests 2024-04-12T20:40:10.632 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module eclipse:rhel8:820201023100746:b230ed4e.x86_64 from CentOS-AppStream 2024-04-12T20:40:10.632 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 3: conflicting requests 2024-04-12T20:40:10.632 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module freeradius:3.0:8080020230117180605:89170a74.x86_64 from CentOS-AppStream 2024-04-12T20:40:10.632 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 4: conflicting requests 2024-04-12T20:40:10.632 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module gimp:2.8:8000020190628145146:4148dfdf.x86_64 from CentOS-AppStream 2024-04-12T20:40:10.632 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 5: conflicting requests 2024-04-12T20:40:10.632 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module go-toolset:rhel8:820240401160413:b754926a.x86_64 from CentOS-AppStream 2024-04-12T20:40:10.632 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 6: conflicting requests 2024-04-12T20:40:10.633 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module httpd:2.4:820240216084734:9edba152.x86_64 from CentOS-AppStream 2024-04-12T20:40:10.633 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 7: conflicting requests 2024-04-12T20:40:10.633 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module idm:client:820240307184541:49cc9d1b.x86_64 from CentOS-AppStream 2024-04-12T20:40:10.633 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 8: conflicting requests 2024-04-12T20:40:10.633 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module inkscape:0.92.3:8080020221205124429:3e031279.x86_64 from CentOS-AppStream 2024-04-12T20:40:10.633 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 9: conflicting requests 2024-04-12T20:40:10.633 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module jmc:rhel8:8050020211110222101:6392b1f8.x86_64 from CentOS-AppStream 2024-04-12T20:40:10.633 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 10: conflicting requests 2024-04-12T20:40:10.633 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module llvm-toolset:rhel8:820240205133135:9edba152.x86_64 from CentOS-AppStream 2024-04-12T20:40:10.633 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 11: conflicting requests 2024-04-12T20:40:10.633 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module log4j:2:8080020221205124743:9d367344.x86_64 from CentOS-AppStream 2024-04-12T20:40:10.633 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 12: conflicting requests 2024-04-12T20:40:10.633 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module mailman:2.1:820230727085757:77fc8825.x86_64 from CentOS-AppStream 2024-04-12T20:40:10.633 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 13: conflicting requests 2024-04-12T20:40:10.633 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module mariadb:10.3:8030020210419150013:30b713e6.x86_64 from CentOS-AppStream 2024-04-12T20:40:10.634 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 14: conflicting requests 2024-04-12T20:40:10.634 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module maven:3.5:8060020211203142015:c0229ad2.x86_64 from CentOS-AppStream 2024-04-12T20:40:10.634 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 15: conflicting requests 2024-04-12T20:40:10.634 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module mercurial:4.8:8000020190628020724:4148dfdf.x86_64 from CentOS-AppStream 2024-04-12T20:40:10.634 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 16: conflicting requests 2024-04-12T20:40:10.634 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module mod_auth_openidc:2.3:820230721123832:9edba152.x86_64 from CentOS-AppStream 2024-04-12T20:40:10.634 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 17: conflicting requests 2024-04-12T20:40:10.634 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module mysql:8.0:8040020210901180257:9f9e2e7e.x86_64 from CentOS-AppStream 2024-04-12T20:40:10.634 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 18: conflicting requests 2024-04-12T20:40:10.634 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module nginx:1.14:8000020211221191913:55190bc5.x86_64 from CentOS-AppStream 2024-04-12T20:40:10.634 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 19: conflicting requests 2024-04-12T20:40:10.634 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module nodejs:10:8040020210120182536:9f9e2e7e.x86_64 from CentOS-AppStream 2024-04-12T20:40:10.634 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 20: conflicting requests 2024-04-12T20:40:10.635 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl:5.26:8000020190628020724:55190bc5.x86_64 from CentOS-AppStream 2024-04-12T20:40:10.635 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 21: conflicting requests 2024-04-12T20:40:10.635 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module php:7.2:8020020200507003613:2c7ca891.x86_64 from CentOS-AppStream 2024-04-12T20:40:10.635 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 22: conflicting requests 2024-04-12T20:40:10.635 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module postgresql:10:820230223114800:9edba152.x86_64 from CentOS-AppStream 2024-04-12T20:40:10.635 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 23: conflicting requests 2024-04-12T20:40:10.635 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module python27:2.7:820240208011952:182f7c73.x86_64 from CentOS-AppStream 2024-04-12T20:40:10.635 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 24: conflicting requests 2024-04-12T20:40:10.635 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module python36:3.6:8050020210825152031:982725ab.x86_64 from CentOS-AppStream 2024-04-12T20:40:10.635 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 25: conflicting requests 2024-04-12T20:40:10.635 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module python38:3.8:820230810143931:64c1cd5a.x86_64 from CentOS-AppStream 2024-04-12T20:40:10.635 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 26: conflicting requests 2024-04-12T20:40:10.635 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module python39:3.9:820240214182535:17377f89.x86_64 from CentOS-AppStream 2024-04-12T20:40:10.635 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 27: conflicting requests 2024-04-12T20:40:10.635 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module redis:5:8040020211019153849:9f9e2e7e.x86_64 from CentOS-AppStream 2024-04-12T20:40:10.636 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 28: conflicting requests 2024-04-12T20:40:10.636 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module rhn-tools:1.0:8010020191114034948:f69d1239.x86_64 from CentOS-AppStream 2024-04-12T20:40:10.636 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 29: conflicting requests 2024-04-12T20:40:10.636 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module ruby:2.5:820230627084142:9edba152.x86_64 from CentOS-AppStream 2024-04-12T20:40:10.636 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 30: conflicting requests 2024-04-12T20:40:10.636 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module rust-toolset:rhel8:820240119204620:b09eea91.x86_64 from CentOS-AppStream 2024-04-12T20:40:10.636 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 31: conflicting requests 2024-04-12T20:40:10.636 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module satellite-5-client:1.0:8010020191114035551:cdc1202b.x86_64 from CentOS-AppStream 2024-04-12T20:40:10.636 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 32: conflicting requests 2024-04-12T20:40:10.636 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module squid:4:820240319173245:9edba152.x86_64 from CentOS-AppStream 2024-04-12T20:40:10.636 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 33: conflicting requests 2024-04-12T20:40:10.636 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module subversion:1.10:8070020220711155714:78111232.x86_64 from CentOS-AppStream 2024-04-12T20:40:10.636 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 34: conflicting requests 2024-04-12T20:40:10.636 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module swig:3.0:8030020200706142531:30b713e6.x86_64 from CentOS-AppStream 2024-04-12T20:40:10.636 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 35: conflicting requests 2024-04-12T20:40:10.637 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module varnish:6:8050020211109225449:b4937e53.x86_64 from CentOS-AppStream 2024-04-12T20:40:10.637 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 36: conflicting requests 2024-04-12T20:40:10.637 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module virt:rhel:820240314161907:9edba152.x86_64 from CentOS-AppStream 2024-04-12T20:40:10.637 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 37: conflicting requests 2024-04-12T20:40:10.637 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:8030020200716155257:7cc0a66d.x86_64 from CentOS-AppStream 2024-04-12T20:40:10.637 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:8030020200716155257:b967a9a2.x86_64 from CentOS-AppStream 2024-04-12T20:40:10.637 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:820230811133638:36f9fae6.x86_64 from CentOS-AppStream 2024-04-12T20:40:10.637 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 38: cannot install the best candidate for the job 2024-04-12T20:40:10.637 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:8010020191114031501:a5949e2e.x86_64 from CentOS-AppStream 2024-04-12T20:40:10.637 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:8010020191114031501:be2e4737.x86_64 from CentOS-AppStream 2024-04-12T20:40:10.637 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:820230223114759:2ab761e1.x86_64 from CentOS-AppStream 2024-04-12T20:40:10.637 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:820230223114759:86e45846.x86_64 from CentOS-AppStream 2024-04-12T20:40:10.637 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 39: conflicting requests 2024-04-12T20:40:10.637 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:8030020200715230104:1e4bbb35.x86_64 from CentOS-AppStream 2024-04-12T20:40:10.637 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:8030020200715230104:ea09926d.x86_64 from CentOS-AppStream 2024-04-12T20:40:10.638 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:082fdf2f.x86_64 from CentOS-AppStream 2024-04-12T20:40:10.638 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-04-12T20:40:10.638 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-04-12T20:40:10.638 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:fbe42456.x86_64 from CentOS-AppStream 2024-04-12T20:40:10.638 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 40: cannot install the best candidate for the job 2024-04-12T20:40:10.638 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:8010020191114031513:16b3ab4d.x86_64 from CentOS-AppStream 2024-04-12T20:40:10.638 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:8010020191114031513:e1f37755.x86_64 from CentOS-AppStream 2024-04-12T20:40:10.638 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-04-12T20:40:10.638 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-04-12T20:40:10.638 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 41: cannot install the best candidate for the job 2024-04-12T20:40:10.638 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:8030020200716150652:1e4bbb35.x86_64 from CentOS-AppStream 2024-04-12T20:40:10.638 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:8030020200716150652:ea09926d.x86_64 from CentOS-AppStream 2024-04-12T20:40:10.638 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-04-12T20:40:10.638 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-04-12T20:40:10.639 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 42: cannot install the best candidate for the job 2024-04-12T20:40:10.639 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:8030020200716174729:3a70019f.x86_64 from CentOS-AppStream 2024-04-12T20:40:10.639 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:8030020200716174729:cccafca5.x86_64 from CentOS-AppStream 2024-04-12T20:40:10.639 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:820230223114759:6fdf7e5d.x86_64 from CentOS-AppStream 2024-04-12T20:40:10.639 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:820230223114759:d4326aba.x86_64 from CentOS-AppStream 2024-04-12T20:40:10.639 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 43: cannot install the best candidate for the job 2024-04-12T20:40:10.639 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:8030020200716213634:56fce90f.x86_64 from CentOS-AppStream 2024-04-12T20:40:10.639 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:8030020200716213634:98a3c9d0.x86_64 from CentOS-AppStream 2024-04-12T20:40:10.639 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:820230223114759:06b7596e.x86_64 from CentOS-AppStream 2024-04-12T20:40:10.639 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:820230223114759:d2fc5d76.x86_64 from CentOS-AppStream 2024-04-12T20:40:10.639 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 44: cannot install the best candidate for the job 2024-04-12T20:40:10.639 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:8030020200716171139:3a70019f.x86_64 from CentOS-AppStream 2024-04-12T20:40:10.639 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:8030020200716171139:cccafca5.x86_64 from CentOS-AppStream 2024-04-12T20:40:10.639 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:820230223114759:6fdf7e5d.x86_64 from CentOS-AppStream 2024-04-12T20:40:10.639 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:820230223114759:d4326aba.x86_64 from CentOS-AppStream 2024-04-12T20:40:10.640 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 45: cannot install the best candidate for the job 2024-04-12T20:40:10.640 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:8010020191120175858:95fce66d.x86_64 from CentOS-AppStream 2024-04-12T20:40:10.640 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:8030020200723233049:09acf126.x86_64 from CentOS-AppStream 2024-04-12T20:40:10.640 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:820230223114759:436af1ae.x86_64 from CentOS-AppStream 2024-04-12T20:40:10.640 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:820230223114759:b12195ea.x86_64 from CentOS-AppStream 2024-04-12T20:40:10.708 INFO:teuthology.orchestra.run.smithi018.stdout:Dependencies resolved. 2024-04-12T20:40:10.709 INFO:teuthology.orchestra.run.smithi018.stdout:================================================================================ 2024-04-12T20:40:10.709 INFO:teuthology.orchestra.run.smithi018.stdout: Package Arch Version Repository Size 2024-04-12T20:40:10.710 INFO:teuthology.orchestra.run.smithi018.stdout:================================================================================ 2024-04-12T20:40:10.710 INFO:teuthology.orchestra.run.smithi018.stdout:Installing: 2024-04-12T20:40:10.710 INFO:teuthology.orchestra.run.smithi018.stdout: ceph-test x86_64 2:17.2.7-827.gd3b5060a.el9 ceph 48 M 2024-04-12T20:40:10.710 INFO:teuthology.orchestra.run.smithi018.stdout:Installing dependencies: 2024-04-12T20:40:10.710 INFO:teuthology.orchestra.run.smithi018.stdout: jq x86_64 1.6-16.el9 baseos 187 k 2024-04-12T20:40:10.711 INFO:teuthology.orchestra.run.smithi018.stdout: libcephsqlite x86_64 2:17.2.7-827.gd3b5060a.el9 ceph 163 k 2024-04-12T20:40:10.711 INFO:teuthology.orchestra.run.smithi018.stdout: oniguruma x86_64 6.9.6-1.el9.5 baseos 218 k 2024-04-12T20:40:10.711 INFO:teuthology.orchestra.run.smithi018.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-04-12T20:40:10.711 INFO:teuthology.orchestra.run.smithi018.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-04-12T20:40:10.711 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-12T20:40:10.711 INFO:teuthology.orchestra.run.smithi018.stdout:Transaction Summary 2024-04-12T20:40:10.711 INFO:teuthology.orchestra.run.smithi018.stdout:================================================================================ 2024-04-12T20:40:10.711 INFO:teuthology.orchestra.run.smithi018.stdout:Install 6 Packages 2024-04-12T20:40:10.712 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-12T20:40:10.713 INFO:teuthology.orchestra.run.smithi018.stdout:Total download size: 49 M 2024-04-12T20:40:10.713 INFO:teuthology.orchestra.run.smithi018.stdout:Installed size: 193 M 2024-04-12T20:40:10.713 INFO:teuthology.orchestra.run.smithi018.stdout:Downloading Packages: 2024-04-12T20:40:11.212 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 26/26 2024-04-12T20:40:11.212 INFO:teuthology.orchestra.run.smithi186.stdout: 2024-04-12T20:40:11.212 INFO:teuthology.orchestra.run.smithi186.stdout:Upgraded: 2024-04-12T20:40:11.213 INFO:teuthology.orchestra.run.smithi186.stdout: librados2-2:17.2.7-827.gd3b5060a.el9.x86_64 2024-04-12T20:40:11.213 INFO:teuthology.orchestra.run.smithi186.stdout: librbd1-2:17.2.7-827.gd3b5060a.el9.x86_64 2024-04-12T20:40:11.213 INFO:teuthology.orchestra.run.smithi186.stdout:Installed: 2024-04-12T20:40:11.213 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-base-2:17.2.7-827.gd3b5060a.el9.x86_64 2024-04-12T20:40:11.213 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-common-2:17.2.7-827.gd3b5060a.el9.x86_64 2024-04-12T20:40:11.213 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-radosgw-2:17.2.7-827.gd3b5060a.el9.x86_64 2024-04-12T20:40:11.213 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-selinux-2:17.2.7-827.gd3b5060a.el9.x86_64 2024-04-12T20:40:11.213 INFO:teuthology.orchestra.run.smithi186.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-04-12T20:40:11.213 INFO:teuthology.orchestra.run.smithi186.stdout: libcephfs2-2:17.2.7-827.gd3b5060a.el9.x86_64 2024-04-12T20:40:11.213 INFO:teuthology.orchestra.run.smithi186.stdout: liboath-2.6.7-2.el9.x86_64 2024-04-12T20:40:11.213 INFO:teuthology.orchestra.run.smithi186.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-04-12T20:40:11.213 INFO:teuthology.orchestra.run.smithi186.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-04-12T20:40:11.213 INFO:teuthology.orchestra.run.smithi186.stdout: libradosstriper1-2:17.2.7-827.gd3b5060a.el9.x86_64 2024-04-12T20:40:11.213 INFO:teuthology.orchestra.run.smithi186.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-04-12T20:40:11.214 INFO:teuthology.orchestra.run.smithi186.stdout: librgw2-2:17.2.7-827.gd3b5060a.el9.x86_64 2024-04-12T20:40:11.214 INFO:teuthology.orchestra.run.smithi186.stdout: libunwind-1.6.2-1.el9.x86_64 2024-04-12T20:40:11.214 INFO:teuthology.orchestra.run.smithi186.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-04-12T20:40:11.214 INFO:teuthology.orchestra.run.smithi186.stdout: python3-ceph-argparse-2:17.2.7-827.gd3b5060a.el9.x86_64 2024-04-12T20:40:11.214 INFO:teuthology.orchestra.run.smithi186.stdout: python3-ceph-common-2:17.2.7-827.gd3b5060a.el9.x86_64 2024-04-12T20:40:11.214 INFO:teuthology.orchestra.run.smithi186.stdout: python3-cephfs-2:17.2.7-827.gd3b5060a.el9.x86_64 2024-04-12T20:40:11.214 INFO:teuthology.orchestra.run.smithi186.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-04-12T20:40:11.214 INFO:teuthology.orchestra.run.smithi186.stdout: python3-rados-2:17.2.7-827.gd3b5060a.el9.x86_64 2024-04-12T20:40:11.214 INFO:teuthology.orchestra.run.smithi186.stdout: python3-rbd-2:17.2.7-827.gd3b5060a.el9.x86_64 2024-04-12T20:40:11.214 INFO:teuthology.orchestra.run.smithi186.stdout: python3-rgw-2:17.2.7-827.gd3b5060a.el9.x86_64 2024-04-12T20:40:11.214 INFO:teuthology.orchestra.run.smithi186.stdout: thrift-0.15.0-2.el9.x86_64 2024-04-12T20:40:11.214 INFO:teuthology.orchestra.run.smithi186.stdout: 2024-04-12T20:40:11.214 INFO:teuthology.orchestra.run.smithi186.stdout:Complete! 2024-04-12T20:40:11.545 INFO:teuthology.orchestra.run.smithi018.stdout:(1/6): jq-1.6-16.el9.x86_64.rpm 773 kB/s | 187 kB 00:00 2024-04-12T20:40:11.612 INFO:teuthology.orchestra.run.smithi018.stdout:(2/6): oniguruma-6.9.6-1.el9.5.x86_64.rpm 3.3 MB/s | 218 kB 00:00 2024-04-12T20:40:11.621 DEBUG:teuthology.orchestra.run.smithi186:> sudo yum -y install ceph-test 2024-04-12T20:40:11.637 INFO:teuthology.orchestra.run.smithi018.stdout:(3/6): libcephsqlite-17.2.7-827.gd3b5060a.el9.x 486 kB/s | 163 kB 00:00 2024-04-12T20:40:12.362 INFO:teuthology.orchestra.run.smithi018.stdout:(4/6): xmlstarlet-1.6.1-20.el9.x86_64.rpm 88 kB/s | 64 kB 00:00 2024-04-12T20:40:12.464 INFO:teuthology.orchestra.run.smithi186.stdout:Last metadata expiration check: 0:00:35 ago on Fri 12 Apr 2024 08:39:37 PM UTC. 2024-04-12T20:40:12.721 INFO:teuthology.orchestra.run.smithi018.stdout:(5/6): socat-1.7.4.1-5.el9.x86_64.rpm 275 kB/s | 305 kB 00:01 2024-04-12T20:40:12.757 INFO:teuthology.orchestra.run.smithi186.stderr:Modular dependency problems: 2024-04-12T20:40:12.757 INFO:teuthology.orchestra.run.smithi186.stderr: 2024-04-12T20:40:12.757 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 1: conflicting requests 2024-04-12T20:40:12.757 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module container-tools:rhel8:820240321091303:20125149.x86_64 from CentOS-AppStream 2024-04-12T20:40:12.758 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 2: conflicting requests 2024-04-12T20:40:12.758 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module eclipse:rhel8:820201023100746:b230ed4e.x86_64 from CentOS-AppStream 2024-04-12T20:40:12.758 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 3: conflicting requests 2024-04-12T20:40:12.758 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module freeradius:3.0:8080020230117180605:89170a74.x86_64 from CentOS-AppStream 2024-04-12T20:40:12.758 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 4: conflicting requests 2024-04-12T20:40:12.758 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module gimp:2.8:8000020190628145146:4148dfdf.x86_64 from CentOS-AppStream 2024-04-12T20:40:12.758 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 5: conflicting requests 2024-04-12T20:40:12.758 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module go-toolset:rhel8:820240401160413:b754926a.x86_64 from CentOS-AppStream 2024-04-12T20:40:12.758 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 6: conflicting requests 2024-04-12T20:40:12.758 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module httpd:2.4:820240216084734:9edba152.x86_64 from CentOS-AppStream 2024-04-12T20:40:12.758 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 7: conflicting requests 2024-04-12T20:40:12.759 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module idm:client:820240307184541:49cc9d1b.x86_64 from CentOS-AppStream 2024-04-12T20:40:12.759 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 8: conflicting requests 2024-04-12T20:40:12.759 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module inkscape:0.92.3:8080020221205124429:3e031279.x86_64 from CentOS-AppStream 2024-04-12T20:40:12.759 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 9: conflicting requests 2024-04-12T20:40:12.759 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module jmc:rhel8:8050020211110222101:6392b1f8.x86_64 from CentOS-AppStream 2024-04-12T20:40:12.759 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 10: conflicting requests 2024-04-12T20:40:12.759 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module llvm-toolset:rhel8:820240205133135:9edba152.x86_64 from CentOS-AppStream 2024-04-12T20:40:12.759 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 11: conflicting requests 2024-04-12T20:40:12.759 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module log4j:2:8080020221205124743:9d367344.x86_64 from CentOS-AppStream 2024-04-12T20:40:12.759 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 12: conflicting requests 2024-04-12T20:40:12.759 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module mailman:2.1:820230727085757:77fc8825.x86_64 from CentOS-AppStream 2024-04-12T20:40:12.760 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 13: conflicting requests 2024-04-12T20:40:12.760 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module mariadb:10.3:8030020210419150013:30b713e6.x86_64 from CentOS-AppStream 2024-04-12T20:40:12.760 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 14: conflicting requests 2024-04-12T20:40:12.760 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module maven:3.5:8060020211203142015:c0229ad2.x86_64 from CentOS-AppStream 2024-04-12T20:40:12.760 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 15: conflicting requests 2024-04-12T20:40:12.760 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module mercurial:4.8:8000020190628020724:4148dfdf.x86_64 from CentOS-AppStream 2024-04-12T20:40:12.760 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 16: conflicting requests 2024-04-12T20:40:12.760 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module mod_auth_openidc:2.3:820230721123832:9edba152.x86_64 from CentOS-AppStream 2024-04-12T20:40:12.760 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 17: conflicting requests 2024-04-12T20:40:12.760 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module mysql:8.0:8040020210901180257:9f9e2e7e.x86_64 from CentOS-AppStream 2024-04-12T20:40:12.760 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 18: conflicting requests 2024-04-12T20:40:12.761 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module nginx:1.14:8000020211221191913:55190bc5.x86_64 from CentOS-AppStream 2024-04-12T20:40:12.761 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 19: conflicting requests 2024-04-12T20:40:12.761 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module nodejs:10:8040020210120182536:9f9e2e7e.x86_64 from CentOS-AppStream 2024-04-12T20:40:12.761 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 20: conflicting requests 2024-04-12T20:40:12.761 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl:5.26:8000020190628020724:55190bc5.x86_64 from CentOS-AppStream 2024-04-12T20:40:12.761 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 21: conflicting requests 2024-04-12T20:40:12.761 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module php:7.2:8020020200507003613:2c7ca891.x86_64 from CentOS-AppStream 2024-04-12T20:40:12.761 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 22: conflicting requests 2024-04-12T20:40:12.761 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module postgresql:10:820230223114800:9edba152.x86_64 from CentOS-AppStream 2024-04-12T20:40:12.761 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 23: conflicting requests 2024-04-12T20:40:12.762 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module python27:2.7:820240208011952:182f7c73.x86_64 from CentOS-AppStream 2024-04-12T20:40:12.762 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 24: conflicting requests 2024-04-12T20:40:12.762 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module python36:3.6:8050020210825152031:982725ab.x86_64 from CentOS-AppStream 2024-04-12T20:40:12.762 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 25: conflicting requests 2024-04-12T20:40:12.762 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module python38:3.8:820230810143931:64c1cd5a.x86_64 from CentOS-AppStream 2024-04-12T20:40:12.762 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 26: conflicting requests 2024-04-12T20:40:12.762 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module python39:3.9:820240214182535:17377f89.x86_64 from CentOS-AppStream 2024-04-12T20:40:12.762 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 27: conflicting requests 2024-04-12T20:40:12.762 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module redis:5:8040020211019153849:9f9e2e7e.x86_64 from CentOS-AppStream 2024-04-12T20:40:12.762 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 28: conflicting requests 2024-04-12T20:40:12.763 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module rhn-tools:1.0:8010020191114034948:f69d1239.x86_64 from CentOS-AppStream 2024-04-12T20:40:12.763 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 29: conflicting requests 2024-04-12T20:40:12.763 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module ruby:2.5:820230627084142:9edba152.x86_64 from CentOS-AppStream 2024-04-12T20:40:12.763 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 30: conflicting requests 2024-04-12T20:40:12.763 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module rust-toolset:rhel8:820240119204620:b09eea91.x86_64 from CentOS-AppStream 2024-04-12T20:40:12.763 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 31: conflicting requests 2024-04-12T20:40:12.763 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module satellite-5-client:1.0:8010020191114035551:cdc1202b.x86_64 from CentOS-AppStream 2024-04-12T20:40:12.763 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 32: conflicting requests 2024-04-12T20:40:12.763 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module squid:4:820240319173245:9edba152.x86_64 from CentOS-AppStream 2024-04-12T20:40:12.763 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 33: conflicting requests 2024-04-12T20:40:12.763 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module subversion:1.10:8070020220711155714:78111232.x86_64 from CentOS-AppStream 2024-04-12T20:40:12.764 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 34: conflicting requests 2024-04-12T20:40:12.764 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module swig:3.0:8030020200706142531:30b713e6.x86_64 from CentOS-AppStream 2024-04-12T20:40:12.764 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 35: conflicting requests 2024-04-12T20:40:12.764 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module varnish:6:8050020211109225449:b4937e53.x86_64 from CentOS-AppStream 2024-04-12T20:40:12.764 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 36: conflicting requests 2024-04-12T20:40:12.764 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module virt:rhel:820240314161907:9edba152.x86_64 from CentOS-AppStream 2024-04-12T20:40:12.764 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 37: conflicting requests 2024-04-12T20:40:12.764 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:8030020200716155257:7cc0a66d.x86_64 from CentOS-AppStream 2024-04-12T20:40:12.764 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:8030020200716155257:b967a9a2.x86_64 from CentOS-AppStream 2024-04-12T20:40:12.764 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:820230811133638:36f9fae6.x86_64 from CentOS-AppStream 2024-04-12T20:40:12.764 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 38: cannot install the best candidate for the job 2024-04-12T20:40:12.764 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:8010020191114031501:a5949e2e.x86_64 from CentOS-AppStream 2024-04-12T20:40:12.765 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:8010020191114031501:be2e4737.x86_64 from CentOS-AppStream 2024-04-12T20:40:12.765 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:820230223114759:2ab761e1.x86_64 from CentOS-AppStream 2024-04-12T20:40:12.765 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:820230223114759:86e45846.x86_64 from CentOS-AppStream 2024-04-12T20:40:12.765 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 39: conflicting requests 2024-04-12T20:40:12.765 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:8030020200715230104:1e4bbb35.x86_64 from CentOS-AppStream 2024-04-12T20:40:12.765 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:8030020200715230104:ea09926d.x86_64 from CentOS-AppStream 2024-04-12T20:40:12.765 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:082fdf2f.x86_64 from CentOS-AppStream 2024-04-12T20:40:12.765 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-04-12T20:40:12.765 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-04-12T20:40:12.765 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:fbe42456.x86_64 from CentOS-AppStream 2024-04-12T20:40:12.765 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 40: cannot install the best candidate for the job 2024-04-12T20:40:12.766 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:8010020191114031513:16b3ab4d.x86_64 from CentOS-AppStream 2024-04-12T20:40:12.766 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:8010020191114031513:e1f37755.x86_64 from CentOS-AppStream 2024-04-12T20:40:12.766 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-04-12T20:40:12.766 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-04-12T20:40:12.767 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 41: cannot install the best candidate for the job 2024-04-12T20:40:12.767 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:8030020200716150652:1e4bbb35.x86_64 from CentOS-AppStream 2024-04-12T20:40:12.770 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:8030020200716150652:ea09926d.x86_64 from CentOS-AppStream 2024-04-12T20:40:12.770 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-04-12T20:40:12.770 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-04-12T20:40:12.770 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 42: cannot install the best candidate for the job 2024-04-12T20:40:12.770 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:8030020200716174729:3a70019f.x86_64 from CentOS-AppStream 2024-04-12T20:40:12.770 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:8030020200716174729:cccafca5.x86_64 from CentOS-AppStream 2024-04-12T20:40:12.770 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:820230223114759:6fdf7e5d.x86_64 from CentOS-AppStream 2024-04-12T20:40:12.771 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:820230223114759:d4326aba.x86_64 from CentOS-AppStream 2024-04-12T20:40:12.771 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 43: cannot install the best candidate for the job 2024-04-12T20:40:12.771 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:8030020200716213634:56fce90f.x86_64 from CentOS-AppStream 2024-04-12T20:40:12.771 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:8030020200716213634:98a3c9d0.x86_64 from CentOS-AppStream 2024-04-12T20:40:12.771 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:820230223114759:06b7596e.x86_64 from CentOS-AppStream 2024-04-12T20:40:12.771 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:820230223114759:d2fc5d76.x86_64 from CentOS-AppStream 2024-04-12T20:40:12.771 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 44: cannot install the best candidate for the job 2024-04-12T20:40:12.771 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:8030020200716171139:3a70019f.x86_64 from CentOS-AppStream 2024-04-12T20:40:12.771 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:8030020200716171139:cccafca5.x86_64 from CentOS-AppStream 2024-04-12T20:40:12.771 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:820230223114759:6fdf7e5d.x86_64 from CentOS-AppStream 2024-04-12T20:40:12.771 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:820230223114759:d4326aba.x86_64 from CentOS-AppStream 2024-04-12T20:40:12.771 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 45: cannot install the best candidate for the job 2024-04-12T20:40:12.771 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:8010020191120175858:95fce66d.x86_64 from CentOS-AppStream 2024-04-12T20:40:12.771 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:8030020200723233049:09acf126.x86_64 from CentOS-AppStream 2024-04-12T20:40:12.772 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:820230223114759:436af1ae.x86_64 from CentOS-AppStream 2024-04-12T20:40:12.772 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:820230223114759:b12195ea.x86_64 from CentOS-AppStream 2024-04-12T20:40:12.834 INFO:teuthology.orchestra.run.smithi186.stdout:Dependencies resolved. 2024-04-12T20:40:12.834 INFO:teuthology.orchestra.run.smithi186.stdout:================================================================================ 2024-04-12T20:40:12.834 INFO:teuthology.orchestra.run.smithi186.stdout: Package Arch Version Repository Size 2024-04-12T20:40:12.834 INFO:teuthology.orchestra.run.smithi186.stdout:================================================================================ 2024-04-12T20:40:12.835 INFO:teuthology.orchestra.run.smithi186.stdout:Installing: 2024-04-12T20:40:12.835 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-test x86_64 2:17.2.7-827.gd3b5060a.el9 ceph 48 M 2024-04-12T20:40:12.835 INFO:teuthology.orchestra.run.smithi186.stdout:Installing dependencies: 2024-04-12T20:40:12.835 INFO:teuthology.orchestra.run.smithi186.stdout: jq x86_64 1.6-16.el9 baseos 187 k 2024-04-12T20:40:12.835 INFO:teuthology.orchestra.run.smithi186.stdout: libcephsqlite x86_64 2:17.2.7-827.gd3b5060a.el9 ceph 163 k 2024-04-12T20:40:12.835 INFO:teuthology.orchestra.run.smithi186.stdout: oniguruma x86_64 6.9.6-1.el9.5 baseos 218 k 2024-04-12T20:40:12.835 INFO:teuthology.orchestra.run.smithi186.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-04-12T20:40:12.835 INFO:teuthology.orchestra.run.smithi186.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-04-12T20:40:12.835 INFO:teuthology.orchestra.run.smithi186.stdout: 2024-04-12T20:40:12.835 INFO:teuthology.orchestra.run.smithi186.stdout:Transaction Summary 2024-04-12T20:40:12.835 INFO:teuthology.orchestra.run.smithi186.stdout:================================================================================ 2024-04-12T20:40:12.835 INFO:teuthology.orchestra.run.smithi186.stdout:Install 6 Packages 2024-04-12T20:40:12.835 INFO:teuthology.orchestra.run.smithi186.stdout: 2024-04-12T20:40:12.836 INFO:teuthology.orchestra.run.smithi186.stdout:Total download size: 49 M 2024-04-12T20:40:12.836 INFO:teuthology.orchestra.run.smithi186.stdout:Installed size: 193 M 2024-04-12T20:40:12.836 INFO:teuthology.orchestra.run.smithi186.stdout:Downloading Packages: 2024-04-12T20:40:13.556 INFO:teuthology.orchestra.run.smithi186.stdout:(1/6): jq-1.6-16.el9.x86_64.rpm 755 kB/s | 187 kB 00:00 2024-04-12T20:40:13.650 INFO:teuthology.orchestra.run.smithi186.stdout:(2/6): oniguruma-6.9.6-1.el9.5.x86_64.rpm 2.3 MB/s | 218 kB 00:00 2024-04-12T20:40:13.809 INFO:teuthology.orchestra.run.smithi186.stdout:(3/6): libcephsqlite-17.2.7-827.gd3b5060a.el9.x 324 kB/s | 163 kB 00:00 2024-04-12T20:40:13.942 INFO:teuthology.orchestra.run.smithi186.stdout:(4/6): socat-1.7.4.1-5.el9.x86_64.rpm 1.0 MB/s | 305 kB 00:00 2024-04-12T20:40:13.959 INFO:teuthology.orchestra.run.smithi186.stdout:(5/6): xmlstarlet-1.6.1-20.el9.x86_64.rpm 430 kB/s | 64 kB 00:00 2024-04-12T20:40:13.979 INFO:teuthology.orchestra.run.smithi018.stdout:(6/6): ceph-test-17.2.7-827.gd3b5060a.el9.x86_6 18 MB/s | 48 MB 00:02 2024-04-12T20:40:13.983 INFO:teuthology.orchestra.run.smithi018.stdout:-------------------------------------------------------------------------------- 2024-04-12T20:40:13.984 INFO:teuthology.orchestra.run.smithi018.stdout:Total 15 MB/s | 49 MB 00:03 2024-04-12T20:40:14.140 INFO:teuthology.orchestra.run.smithi018.stdout:Running transaction check 2024-04-12T20:40:14.199 INFO:teuthology.orchestra.run.smithi018.stdout:Transaction check succeeded. 2024-04-12T20:40:14.200 INFO:teuthology.orchestra.run.smithi018.stdout:Running transaction test 2024-04-12T20:40:14.673 INFO:teuthology.orchestra.run.smithi018.stdout:Transaction test succeeded. 2024-04-12T20:40:14.971 INFO:teuthology.orchestra.run.smithi018.stdout:Running transaction 2024-04-12T20:40:15.360 INFO:teuthology.orchestra.run.smithi018.stdout: Preparing : 1/1 2024-04-12T20:40:15.462 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 1/6 2024-04-12T20:40:15.521 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 2/6 2024-04-12T20:40:15.564 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : oniguruma-6.9.6-1.el9.5.x86_64 3/6 2024-04-12T20:40:15.618 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : jq-1.6-16.el9.x86_64 4/6 2024-04-12T20:40:15.649 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : libcephsqlite-2:17.2.7-827.gd3b5060a.el9.x86_64 5/6 2024-04-12T20:40:16.045 INFO:teuthology.orchestra.run.smithi186.stdout:(6/6): ceph-test-17.2.7-827.gd3b5060a.el9.x86_6 18 MB/s | 48 MB 00:02 2024-04-12T20:40:16.046 INFO:teuthology.orchestra.run.smithi186.stdout:-------------------------------------------------------------------------------- 2024-04-12T20:40:16.047 INFO:teuthology.orchestra.run.smithi186.stdout:Total 15 MB/s | 49 MB 00:03 2024-04-12T20:40:16.192 INFO:teuthology.orchestra.run.smithi186.stdout:Running transaction check 2024-04-12T20:40:16.245 INFO:teuthology.orchestra.run.smithi186.stdout:Transaction check succeeded. 2024-04-12T20:40:16.245 INFO:teuthology.orchestra.run.smithi186.stdout:Running transaction test 2024-04-12T20:40:16.709 INFO:teuthology.orchestra.run.smithi186.stdout:Transaction test succeeded. 2024-04-12T20:40:16.710 INFO:teuthology.orchestra.run.smithi186.stdout:Running transaction 2024-04-12T20:40:17.379 INFO:teuthology.orchestra.run.smithi186.stdout: Preparing : 1/1 2024-04-12T20:40:17.472 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 1/6 2024-04-12T20:40:17.516 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 2/6 2024-04-12T20:40:17.566 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : oniguruma-6.9.6-1.el9.5.x86_64 3/6 2024-04-12T20:40:17.634 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : jq-1.6-16.el9.x86_64 4/6 2024-04-12T20:40:17.662 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : libcephsqlite-2:17.2.7-827.gd3b5060a.el9.x86_64 5/6 2024-04-12T20:40:20.762 INFO:teuthology.orchestra.run.smithi018.stdout: Running scriptlet: libcephsqlite-2:17.2.7-827.gd3b5060a.el9.x86_64 5/6 2024-04-12T20:40:20.874 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : ceph-test-2:17.2.7-827.gd3b5060a.el9.x86_64 6/6 2024-04-12T20:40:22.475 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: libcephsqlite-2:17.2.7-827.gd3b5060a.el9.x86_64 5/6 2024-04-12T20:40:22.575 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : ceph-test-2:17.2.7-827.gd3b5060a.el9.x86_64 6/6 2024-04-12T20:40:23.430 INFO:teuthology.orchestra.run.smithi018.stdout: Running scriptlet: ceph-test-2:17.2.7-827.gd3b5060a.el9.x86_64 6/6 2024-04-12T20:40:23.430 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : ceph-test-2:17.2.7-827.gd3b5060a.el9.x86_64 1/6 2024-04-12T20:40:23.430 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : libcephsqlite-2:17.2.7-827.gd3b5060a.el9.x86_64 2/6 2024-04-12T20:40:23.430 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : jq-1.6-16.el9.x86_64 3/6 2024-04-12T20:40:23.430 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : oniguruma-6.9.6-1.el9.5.x86_64 4/6 2024-04-12T20:40:23.430 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 5/6 2024-04-12T20:40:23.878 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 6/6 2024-04-12T20:40:23.878 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-12T20:40:23.878 INFO:teuthology.orchestra.run.smithi018.stdout:Installed: 2024-04-12T20:40:23.879 INFO:teuthology.orchestra.run.smithi018.stdout: ceph-test-2:17.2.7-827.gd3b5060a.el9.x86_64 2024-04-12T20:40:23.879 INFO:teuthology.orchestra.run.smithi018.stdout: jq-1.6-16.el9.x86_64 2024-04-12T20:40:23.879 INFO:teuthology.orchestra.run.smithi018.stdout: libcephsqlite-2:17.2.7-827.gd3b5060a.el9.x86_64 2024-04-12T20:40:23.879 INFO:teuthology.orchestra.run.smithi018.stdout: oniguruma-6.9.6-1.el9.5.x86_64 2024-04-12T20:40:23.879 INFO:teuthology.orchestra.run.smithi018.stdout: socat-1.7.4.1-5.el9.x86_64 2024-04-12T20:40:23.879 INFO:teuthology.orchestra.run.smithi018.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-04-12T20:40:23.879 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-12T20:40:23.879 INFO:teuthology.orchestra.run.smithi018.stdout:Complete! 2024-04-12T20:40:24.294 DEBUG:teuthology.orchestra.run.smithi018:> sudo yum -y install ceph 2024-04-12T20:40:24.888 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: ceph-test-2:17.2.7-827.gd3b5060a.el9.x86_64 6/6 2024-04-12T20:40:24.888 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : ceph-test-2:17.2.7-827.gd3b5060a.el9.x86_64 1/6 2024-04-12T20:40:24.888 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : libcephsqlite-2:17.2.7-827.gd3b5060a.el9.x86_64 2/6 2024-04-12T20:40:24.888 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : jq-1.6-16.el9.x86_64 3/6 2024-04-12T20:40:24.888 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : oniguruma-6.9.6-1.el9.5.x86_64 4/6 2024-04-12T20:40:24.888 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 5/6 2024-04-12T20:40:25.222 INFO:teuthology.orchestra.run.smithi018.stdout:Last metadata expiration check: 0:00:50 ago on Fri 12 Apr 2024 08:39:35 PM UTC. 2024-04-12T20:40:25.255 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 6/6 2024-04-12T20:40:25.255 INFO:teuthology.orchestra.run.smithi186.stdout: 2024-04-12T20:40:25.255 INFO:teuthology.orchestra.run.smithi186.stdout:Installed: 2024-04-12T20:40:25.255 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-test-2:17.2.7-827.gd3b5060a.el9.x86_64 2024-04-12T20:40:25.255 INFO:teuthology.orchestra.run.smithi186.stdout: jq-1.6-16.el9.x86_64 2024-04-12T20:40:25.255 INFO:teuthology.orchestra.run.smithi186.stdout: libcephsqlite-2:17.2.7-827.gd3b5060a.el9.x86_64 2024-04-12T20:40:25.255 INFO:teuthology.orchestra.run.smithi186.stdout: oniguruma-6.9.6-1.el9.5.x86_64 2024-04-12T20:40:25.256 INFO:teuthology.orchestra.run.smithi186.stdout: socat-1.7.4.1-5.el9.x86_64 2024-04-12T20:40:25.256 INFO:teuthology.orchestra.run.smithi186.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-04-12T20:40:25.256 INFO:teuthology.orchestra.run.smithi186.stdout: 2024-04-12T20:40:25.256 INFO:teuthology.orchestra.run.smithi186.stdout:Complete! 2024-04-12T20:40:25.531 INFO:teuthology.orchestra.run.smithi018.stderr:Modular dependency problems: 2024-04-12T20:40:25.531 INFO:teuthology.orchestra.run.smithi018.stderr: 2024-04-12T20:40:25.531 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 1: conflicting requests 2024-04-12T20:40:25.531 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module container-tools:rhel8:820240321091303:20125149.x86_64 from CentOS-AppStream 2024-04-12T20:40:25.531 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 2: conflicting requests 2024-04-12T20:40:25.532 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module eclipse:rhel8:820201023100746:b230ed4e.x86_64 from CentOS-AppStream 2024-04-12T20:40:25.532 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 3: conflicting requests 2024-04-12T20:40:25.532 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module freeradius:3.0:8080020230117180605:89170a74.x86_64 from CentOS-AppStream 2024-04-12T20:40:25.532 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 4: conflicting requests 2024-04-12T20:40:25.532 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module gimp:2.8:8000020190628145146:4148dfdf.x86_64 from CentOS-AppStream 2024-04-12T20:40:25.532 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 5: conflicting requests 2024-04-12T20:40:25.532 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module go-toolset:rhel8:820240401160413:b754926a.x86_64 from CentOS-AppStream 2024-04-12T20:40:25.532 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 6: conflicting requests 2024-04-12T20:40:25.532 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module httpd:2.4:820240216084734:9edba152.x86_64 from CentOS-AppStream 2024-04-12T20:40:25.532 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 7: conflicting requests 2024-04-12T20:40:25.532 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module idm:client:820240307184541:49cc9d1b.x86_64 from CentOS-AppStream 2024-04-12T20:40:25.532 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 8: conflicting requests 2024-04-12T20:40:25.532 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module inkscape:0.92.3:8080020221205124429:3e031279.x86_64 from CentOS-AppStream 2024-04-12T20:40:25.532 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 9: conflicting requests 2024-04-12T20:40:25.533 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module jmc:rhel8:8050020211110222101:6392b1f8.x86_64 from CentOS-AppStream 2024-04-12T20:40:25.533 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 10: conflicting requests 2024-04-12T20:40:25.533 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module llvm-toolset:rhel8:820240205133135:9edba152.x86_64 from CentOS-AppStream 2024-04-12T20:40:25.533 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 11: conflicting requests 2024-04-12T20:40:25.533 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module log4j:2:8080020221205124743:9d367344.x86_64 from CentOS-AppStream 2024-04-12T20:40:25.533 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 12: conflicting requests 2024-04-12T20:40:25.533 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module mailman:2.1:820230727085757:77fc8825.x86_64 from CentOS-AppStream 2024-04-12T20:40:25.533 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 13: conflicting requests 2024-04-12T20:40:25.533 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module mariadb:10.3:8030020210419150013:30b713e6.x86_64 from CentOS-AppStream 2024-04-12T20:40:25.533 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 14: conflicting requests 2024-04-12T20:40:25.533 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module maven:3.5:8060020211203142015:c0229ad2.x86_64 from CentOS-AppStream 2024-04-12T20:40:25.533 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 15: conflicting requests 2024-04-12T20:40:25.533 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module mercurial:4.8:8000020190628020724:4148dfdf.x86_64 from CentOS-AppStream 2024-04-12T20:40:25.533 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 16: conflicting requests 2024-04-12T20:40:25.534 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module mod_auth_openidc:2.3:820230721123832:9edba152.x86_64 from CentOS-AppStream 2024-04-12T20:40:25.534 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 17: conflicting requests 2024-04-12T20:40:25.534 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module mysql:8.0:8040020210901180257:9f9e2e7e.x86_64 from CentOS-AppStream 2024-04-12T20:40:25.534 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 18: conflicting requests 2024-04-12T20:40:25.534 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module nginx:1.14:8000020211221191913:55190bc5.x86_64 from CentOS-AppStream 2024-04-12T20:40:25.534 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 19: conflicting requests 2024-04-12T20:40:25.534 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module nodejs:10:8040020210120182536:9f9e2e7e.x86_64 from CentOS-AppStream 2024-04-12T20:40:25.534 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 20: conflicting requests 2024-04-12T20:40:25.534 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl:5.26:8000020190628020724:55190bc5.x86_64 from CentOS-AppStream 2024-04-12T20:40:25.534 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 21: conflicting requests 2024-04-12T20:40:25.534 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module php:7.2:8020020200507003613:2c7ca891.x86_64 from CentOS-AppStream 2024-04-12T20:40:25.534 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 22: conflicting requests 2024-04-12T20:40:25.535 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module postgresql:10:820230223114800:9edba152.x86_64 from CentOS-AppStream 2024-04-12T20:40:25.535 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 23: conflicting requests 2024-04-12T20:40:25.535 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module python27:2.7:820240208011952:182f7c73.x86_64 from CentOS-AppStream 2024-04-12T20:40:25.535 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 24: conflicting requests 2024-04-12T20:40:25.535 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module python36:3.6:8050020210825152031:982725ab.x86_64 from CentOS-AppStream 2024-04-12T20:40:25.535 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 25: conflicting requests 2024-04-12T20:40:25.535 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module python38:3.8:820230810143931:64c1cd5a.x86_64 from CentOS-AppStream 2024-04-12T20:40:25.535 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 26: conflicting requests 2024-04-12T20:40:25.536 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module python39:3.9:820240214182535:17377f89.x86_64 from CentOS-AppStream 2024-04-12T20:40:25.536 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 27: conflicting requests 2024-04-12T20:40:25.536 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module redis:5:8040020211019153849:9f9e2e7e.x86_64 from CentOS-AppStream 2024-04-12T20:40:25.536 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 28: conflicting requests 2024-04-12T20:40:25.536 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module rhn-tools:1.0:8010020191114034948:f69d1239.x86_64 from CentOS-AppStream 2024-04-12T20:40:25.536 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 29: conflicting requests 2024-04-12T20:40:25.536 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module ruby:2.5:820230627084142:9edba152.x86_64 from CentOS-AppStream 2024-04-12T20:40:25.536 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 30: conflicting requests 2024-04-12T20:40:25.536 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module rust-toolset:rhel8:820240119204620:b09eea91.x86_64 from CentOS-AppStream 2024-04-12T20:40:25.536 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 31: conflicting requests 2024-04-12T20:40:25.537 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module satellite-5-client:1.0:8010020191114035551:cdc1202b.x86_64 from CentOS-AppStream 2024-04-12T20:40:25.537 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 32: conflicting requests 2024-04-12T20:40:25.537 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module squid:4:820240319173245:9edba152.x86_64 from CentOS-AppStream 2024-04-12T20:40:25.537 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 33: conflicting requests 2024-04-12T20:40:25.537 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module subversion:1.10:8070020220711155714:78111232.x86_64 from CentOS-AppStream 2024-04-12T20:40:25.537 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 34: conflicting requests 2024-04-12T20:40:25.537 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module swig:3.0:8030020200706142531:30b713e6.x86_64 from CentOS-AppStream 2024-04-12T20:40:25.537 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 35: conflicting requests 2024-04-12T20:40:25.537 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module varnish:6:8050020211109225449:b4937e53.x86_64 from CentOS-AppStream 2024-04-12T20:40:25.537 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 36: conflicting requests 2024-04-12T20:40:25.537 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module virt:rhel:820240314161907:9edba152.x86_64 from CentOS-AppStream 2024-04-12T20:40:25.537 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 37: conflicting requests 2024-04-12T20:40:25.537 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:8030020200716155257:7cc0a66d.x86_64 from CentOS-AppStream 2024-04-12T20:40:25.537 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:8030020200716155257:b967a9a2.x86_64 from CentOS-AppStream 2024-04-12T20:40:25.537 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:820230811133638:36f9fae6.x86_64 from CentOS-AppStream 2024-04-12T20:40:25.538 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 38: cannot install the best candidate for the job 2024-04-12T20:40:25.538 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:8010020191114031501:a5949e2e.x86_64 from CentOS-AppStream 2024-04-12T20:40:25.538 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:8010020191114031501:be2e4737.x86_64 from CentOS-AppStream 2024-04-12T20:40:25.538 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:820230223114759:2ab761e1.x86_64 from CentOS-AppStream 2024-04-12T20:40:25.538 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:820230223114759:86e45846.x86_64 from CentOS-AppStream 2024-04-12T20:40:25.538 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 39: conflicting requests 2024-04-12T20:40:25.538 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:8030020200715230104:1e4bbb35.x86_64 from CentOS-AppStream 2024-04-12T20:40:25.538 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:8030020200715230104:ea09926d.x86_64 from CentOS-AppStream 2024-04-12T20:40:25.538 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:082fdf2f.x86_64 from CentOS-AppStream 2024-04-12T20:40:25.538 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-04-12T20:40:25.538 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-04-12T20:40:25.538 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:fbe42456.x86_64 from CentOS-AppStream 2024-04-12T20:40:25.538 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 40: cannot install the best candidate for the job 2024-04-12T20:40:25.539 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:8010020191114031513:16b3ab4d.x86_64 from CentOS-AppStream 2024-04-12T20:40:25.539 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:8010020191114031513:e1f37755.x86_64 from CentOS-AppStream 2024-04-12T20:40:25.539 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-04-12T20:40:25.539 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-04-12T20:40:25.539 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 41: cannot install the best candidate for the job 2024-04-12T20:40:25.539 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:8030020200716150652:1e4bbb35.x86_64 from CentOS-AppStream 2024-04-12T20:40:25.539 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:8030020200716150652:ea09926d.x86_64 from CentOS-AppStream 2024-04-12T20:40:25.539 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-04-12T20:40:25.539 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-04-12T20:40:25.539 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 42: cannot install the best candidate for the job 2024-04-12T20:40:25.540 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:8030020200716174729:3a70019f.x86_64 from CentOS-AppStream 2024-04-12T20:40:25.540 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:8030020200716174729:cccafca5.x86_64 from CentOS-AppStream 2024-04-12T20:40:25.540 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:820230223114759:6fdf7e5d.x86_64 from CentOS-AppStream 2024-04-12T20:40:25.540 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:820230223114759:d4326aba.x86_64 from CentOS-AppStream 2024-04-12T20:40:25.540 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 43: cannot install the best candidate for the job 2024-04-12T20:40:25.540 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:8030020200716213634:56fce90f.x86_64 from CentOS-AppStream 2024-04-12T20:40:25.540 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:8030020200716213634:98a3c9d0.x86_64 from CentOS-AppStream 2024-04-12T20:40:25.540 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:820230223114759:06b7596e.x86_64 from CentOS-AppStream 2024-04-12T20:40:25.540 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:820230223114759:d2fc5d76.x86_64 from CentOS-AppStream 2024-04-12T20:40:25.540 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 44: cannot install the best candidate for the job 2024-04-12T20:40:25.540 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:8030020200716171139:3a70019f.x86_64 from CentOS-AppStream 2024-04-12T20:40:25.540 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:8030020200716171139:cccafca5.x86_64 from CentOS-AppStream 2024-04-12T20:40:25.540 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:820230223114759:6fdf7e5d.x86_64 from CentOS-AppStream 2024-04-12T20:40:25.541 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:820230223114759:d4326aba.x86_64 from CentOS-AppStream 2024-04-12T20:40:25.541 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 45: cannot install the best candidate for the job 2024-04-12T20:40:25.541 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:8010020191120175858:95fce66d.x86_64 from CentOS-AppStream 2024-04-12T20:40:25.541 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:8030020200723233049:09acf126.x86_64 from CentOS-AppStream 2024-04-12T20:40:25.541 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:820230223114759:436af1ae.x86_64 from CentOS-AppStream 2024-04-12T20:40:25.541 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:820230223114759:b12195ea.x86_64 from CentOS-AppStream 2024-04-12T20:40:25.631 INFO:teuthology.orchestra.run.smithi018.stdout:Dependencies resolved. 2024-04-12T20:40:25.633 INFO:teuthology.orchestra.run.smithi018.stdout:================================================================================ 2024-04-12T20:40:25.633 INFO:teuthology.orchestra.run.smithi018.stdout: Package Arch Version Repository Size 2024-04-12T20:40:25.633 INFO:teuthology.orchestra.run.smithi018.stdout:================================================================================ 2024-04-12T20:40:25.633 INFO:teuthology.orchestra.run.smithi018.stdout:Installing: 2024-04-12T20:40:25.634 INFO:teuthology.orchestra.run.smithi018.stdout: ceph x86_64 2:17.2.7-827.gd3b5060a.el9 ceph 6.5 k 2024-04-12T20:40:25.634 INFO:teuthology.orchestra.run.smithi018.stdout:Installing dependencies: 2024-04-12T20:40:25.634 INFO:teuthology.orchestra.run.smithi018.stdout: ceph-mds x86_64 2:17.2.7-827.gd3b5060a.el9 ceph 2.1 M 2024-04-12T20:40:25.634 INFO:teuthology.orchestra.run.smithi018.stdout: ceph-mgr x86_64 2:17.2.7-827.gd3b5060a.el9 ceph 1.4 M 2024-04-12T20:40:25.634 INFO:teuthology.orchestra.run.smithi018.stdout: ceph-mgr-modules-core noarch 2:17.2.7-827.gd3b5060a.el9 ceph-noarch 240 k 2024-04-12T20:40:25.634 INFO:teuthology.orchestra.run.smithi018.stdout: ceph-mon x86_64 2:17.2.7-827.gd3b5060a.el9 ceph 3.9 M 2024-04-12T20:40:25.634 INFO:teuthology.orchestra.run.smithi018.stdout: ceph-osd x86_64 2:17.2.7-827.gd3b5060a.el9 ceph 15 M 2024-04-12T20:40:25.634 INFO:teuthology.orchestra.run.smithi018.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-04-12T20:40:25.634 INFO:teuthology.orchestra.run.smithi018.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-04-12T20:40:25.634 INFO:teuthology.orchestra.run.smithi018.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-04-12T20:40:25.634 INFO:teuthology.orchestra.run.smithi018.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-04-12T20:40:25.635 INFO:teuthology.orchestra.run.smithi018.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-04-12T20:40:25.635 INFO:teuthology.orchestra.run.smithi018.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-04-12T20:40:25.635 INFO:teuthology.orchestra.run.smithi018.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-04-12T20:40:25.635 INFO:teuthology.orchestra.run.smithi018.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-04-12T20:40:25.635 INFO:teuthology.orchestra.run.smithi018.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-04-12T20:40:25.635 INFO:teuthology.orchestra.run.smithi018.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-04-12T20:40:25.635 INFO:teuthology.orchestra.run.smithi018.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-04-12T20:40:25.635 INFO:teuthology.orchestra.run.smithi018.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-04-12T20:40:25.635 INFO:teuthology.orchestra.run.smithi018.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-04-12T20:40:25.635 INFO:teuthology.orchestra.run.smithi018.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-04-12T20:40:25.635 INFO:teuthology.orchestra.run.smithi018.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-04-12T20:40:25.635 INFO:teuthology.orchestra.run.smithi018.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-04-12T20:40:25.635 INFO:teuthology.orchestra.run.smithi018.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-04-12T20:40:25.635 INFO:teuthology.orchestra.run.smithi018.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-04-12T20:40:25.636 INFO:teuthology.orchestra.run.smithi018.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-04-12T20:40:25.636 INFO:teuthology.orchestra.run.smithi018.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-04-12T20:40:25.636 INFO:teuthology.orchestra.run.smithi018.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-04-12T20:40:25.636 INFO:teuthology.orchestra.run.smithi018.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-04-12T20:40:25.636 INFO:teuthology.orchestra.run.smithi018.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-04-12T20:40:25.636 INFO:teuthology.orchestra.run.smithi018.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-04-12T20:40:25.636 INFO:teuthology.orchestra.run.smithi018.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-04-12T20:40:25.636 INFO:teuthology.orchestra.run.smithi018.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-04-12T20:40:25.636 INFO:teuthology.orchestra.run.smithi018.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-04-12T20:40:25.636 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-12T20:40:25.637 INFO:teuthology.orchestra.run.smithi018.stdout:Transaction Summary 2024-04-12T20:40:25.637 INFO:teuthology.orchestra.run.smithi018.stdout:================================================================================ 2024-04-12T20:40:25.637 INFO:teuthology.orchestra.run.smithi018.stdout:Install 33 Packages 2024-04-12T20:40:25.637 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-12T20:40:25.637 INFO:teuthology.orchestra.run.smithi018.stdout:Total download size: 27 M 2024-04-12T20:40:25.637 INFO:teuthology.orchestra.run.smithi018.stdout:Installed size: 96 M 2024-04-12T20:40:25.637 INFO:teuthology.orchestra.run.smithi018.stdout:Downloading Packages: 2024-04-12T20:40:25.732 DEBUG:teuthology.orchestra.run.smithi186:> sudo yum -y install ceph 2024-04-12T20:40:26.628 INFO:teuthology.orchestra.run.smithi186.stdout:Last metadata expiration check: 0:00:49 ago on Fri 12 Apr 2024 08:39:37 PM UTC. 2024-04-12T20:40:26.878 INFO:teuthology.orchestra.run.smithi018.stdout:(1/33): ceph-17.2.7-827.gd3b5060a.el9.x86_64.rp 21 kB/s | 6.5 kB 00:00 2024-04-12T20:40:26.928 INFO:teuthology.orchestra.run.smithi186.stderr:Modular dependency problems: 2024-04-12T20:40:26.928 INFO:teuthology.orchestra.run.smithi186.stderr: 2024-04-12T20:40:26.928 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 1: conflicting requests 2024-04-12T20:40:26.928 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module container-tools:rhel8:820240321091303:20125149.x86_64 from CentOS-AppStream 2024-04-12T20:40:26.928 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 2: conflicting requests 2024-04-12T20:40:26.929 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module eclipse:rhel8:820201023100746:b230ed4e.x86_64 from CentOS-AppStream 2024-04-12T20:40:26.929 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 3: conflicting requests 2024-04-12T20:40:26.929 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module freeradius:3.0:8080020230117180605:89170a74.x86_64 from CentOS-AppStream 2024-04-12T20:40:26.929 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 4: conflicting requests 2024-04-12T20:40:26.929 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module gimp:2.8:8000020190628145146:4148dfdf.x86_64 from CentOS-AppStream 2024-04-12T20:40:26.929 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 5: conflicting requests 2024-04-12T20:40:26.929 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module go-toolset:rhel8:820240401160413:b754926a.x86_64 from CentOS-AppStream 2024-04-12T20:40:26.929 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 6: conflicting requests 2024-04-12T20:40:26.929 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module httpd:2.4:820240216084734:9edba152.x86_64 from CentOS-AppStream 2024-04-12T20:40:26.929 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 7: conflicting requests 2024-04-12T20:40:26.929 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module idm:client:820240307184541:49cc9d1b.x86_64 from CentOS-AppStream 2024-04-12T20:40:26.929 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 8: conflicting requests 2024-04-12T20:40:26.929 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module inkscape:0.92.3:8080020221205124429:3e031279.x86_64 from CentOS-AppStream 2024-04-12T20:40:26.930 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 9: conflicting requests 2024-04-12T20:40:26.930 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module jmc:rhel8:8050020211110222101:6392b1f8.x86_64 from CentOS-AppStream 2024-04-12T20:40:26.930 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 10: conflicting requests 2024-04-12T20:40:26.930 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module llvm-toolset:rhel8:820240205133135:9edba152.x86_64 from CentOS-AppStream 2024-04-12T20:40:26.930 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 11: conflicting requests 2024-04-12T20:40:26.930 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module log4j:2:8080020221205124743:9d367344.x86_64 from CentOS-AppStream 2024-04-12T20:40:26.930 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 12: conflicting requests 2024-04-12T20:40:26.930 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module mailman:2.1:820230727085757:77fc8825.x86_64 from CentOS-AppStream 2024-04-12T20:40:26.930 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 13: conflicting requests 2024-04-12T20:40:26.930 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module mariadb:10.3:8030020210419150013:30b713e6.x86_64 from CentOS-AppStream 2024-04-12T20:40:26.930 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 14: conflicting requests 2024-04-12T20:40:26.930 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module maven:3.5:8060020211203142015:c0229ad2.x86_64 from CentOS-AppStream 2024-04-12T20:40:26.931 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 15: conflicting requests 2024-04-12T20:40:26.931 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module mercurial:4.8:8000020190628020724:4148dfdf.x86_64 from CentOS-AppStream 2024-04-12T20:40:26.931 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 16: conflicting requests 2024-04-12T20:40:26.931 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module mod_auth_openidc:2.3:820230721123832:9edba152.x86_64 from CentOS-AppStream 2024-04-12T20:40:26.931 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 17: conflicting requests 2024-04-12T20:40:26.931 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module mysql:8.0:8040020210901180257:9f9e2e7e.x86_64 from CentOS-AppStream 2024-04-12T20:40:26.931 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 18: conflicting requests 2024-04-12T20:40:26.931 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module nginx:1.14:8000020211221191913:55190bc5.x86_64 from CentOS-AppStream 2024-04-12T20:40:26.931 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 19: conflicting requests 2024-04-12T20:40:26.931 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module nodejs:10:8040020210120182536:9f9e2e7e.x86_64 from CentOS-AppStream 2024-04-12T20:40:26.931 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 20: conflicting requests 2024-04-12T20:40:26.931 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl:5.26:8000020190628020724:55190bc5.x86_64 from CentOS-AppStream 2024-04-12T20:40:26.931 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 21: conflicting requests 2024-04-12T20:40:26.932 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module php:7.2:8020020200507003613:2c7ca891.x86_64 from CentOS-AppStream 2024-04-12T20:40:26.932 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 22: conflicting requests 2024-04-12T20:40:26.932 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module postgresql:10:820230223114800:9edba152.x86_64 from CentOS-AppStream 2024-04-12T20:40:26.932 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 23: conflicting requests 2024-04-12T20:40:26.932 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module python27:2.7:820240208011952:182f7c73.x86_64 from CentOS-AppStream 2024-04-12T20:40:26.932 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 24: conflicting requests 2024-04-12T20:40:26.932 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module python36:3.6:8050020210825152031:982725ab.x86_64 from CentOS-AppStream 2024-04-12T20:40:26.932 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 25: conflicting requests 2024-04-12T20:40:26.932 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module python38:3.8:820230810143931:64c1cd5a.x86_64 from CentOS-AppStream 2024-04-12T20:40:26.932 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 26: conflicting requests 2024-04-12T20:40:26.932 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module python39:3.9:820240214182535:17377f89.x86_64 from CentOS-AppStream 2024-04-12T20:40:26.932 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 27: conflicting requests 2024-04-12T20:40:26.932 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module redis:5:8040020211019153849:9f9e2e7e.x86_64 from CentOS-AppStream 2024-04-12T20:40:26.933 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 28: conflicting requests 2024-04-12T20:40:26.933 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module rhn-tools:1.0:8010020191114034948:f69d1239.x86_64 from CentOS-AppStream 2024-04-12T20:40:26.933 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 29: conflicting requests 2024-04-12T20:40:26.933 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module ruby:2.5:820230627084142:9edba152.x86_64 from CentOS-AppStream 2024-04-12T20:40:26.933 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 30: conflicting requests 2024-04-12T20:40:26.933 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module rust-toolset:rhel8:820240119204620:b09eea91.x86_64 from CentOS-AppStream 2024-04-12T20:40:26.933 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 31: conflicting requests 2024-04-12T20:40:26.933 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module satellite-5-client:1.0:8010020191114035551:cdc1202b.x86_64 from CentOS-AppStream 2024-04-12T20:40:26.933 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 32: conflicting requests 2024-04-12T20:40:26.933 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module squid:4:820240319173245:9edba152.x86_64 from CentOS-AppStream 2024-04-12T20:40:26.933 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 33: conflicting requests 2024-04-12T20:40:26.934 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module subversion:1.10:8070020220711155714:78111232.x86_64 from CentOS-AppStream 2024-04-12T20:40:26.934 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 34: conflicting requests 2024-04-12T20:40:26.934 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module swig:3.0:8030020200706142531:30b713e6.x86_64 from CentOS-AppStream 2024-04-12T20:40:26.934 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 35: conflicting requests 2024-04-12T20:40:26.934 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module varnish:6:8050020211109225449:b4937e53.x86_64 from CentOS-AppStream 2024-04-12T20:40:26.934 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 36: conflicting requests 2024-04-12T20:40:26.935 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module virt:rhel:820240314161907:9edba152.x86_64 from CentOS-AppStream 2024-04-12T20:40:26.935 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 37: conflicting requests 2024-04-12T20:40:26.935 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:8030020200716155257:7cc0a66d.x86_64 from CentOS-AppStream 2024-04-12T20:40:26.935 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:8030020200716155257:b967a9a2.x86_64 from CentOS-AppStream 2024-04-12T20:40:26.935 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:820230811133638:36f9fae6.x86_64 from CentOS-AppStream 2024-04-12T20:40:26.935 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 38: cannot install the best candidate for the job 2024-04-12T20:40:26.935 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:8010020191114031501:a5949e2e.x86_64 from CentOS-AppStream 2024-04-12T20:40:26.935 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:8010020191114031501:be2e4737.x86_64 from CentOS-AppStream 2024-04-12T20:40:26.936 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:820230223114759:2ab761e1.x86_64 from CentOS-AppStream 2024-04-12T20:40:26.936 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:820230223114759:86e45846.x86_64 from CentOS-AppStream 2024-04-12T20:40:26.936 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 39: conflicting requests 2024-04-12T20:40:26.936 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:8030020200715230104:1e4bbb35.x86_64 from CentOS-AppStream 2024-04-12T20:40:26.936 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:8030020200715230104:ea09926d.x86_64 from CentOS-AppStream 2024-04-12T20:40:26.936 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:082fdf2f.x86_64 from CentOS-AppStream 2024-04-12T20:40:26.936 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-04-12T20:40:26.936 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-04-12T20:40:26.936 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:fbe42456.x86_64 from CentOS-AppStream 2024-04-12T20:40:26.936 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 40: cannot install the best candidate for the job 2024-04-12T20:40:26.936 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:8010020191114031513:16b3ab4d.x86_64 from CentOS-AppStream 2024-04-12T20:40:26.936 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:8010020191114031513:e1f37755.x86_64 from CentOS-AppStream 2024-04-12T20:40:26.936 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-04-12T20:40:26.937 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-04-12T20:40:26.937 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 41: cannot install the best candidate for the job 2024-04-12T20:40:26.937 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:8030020200716150652:1e4bbb35.x86_64 from CentOS-AppStream 2024-04-12T20:40:26.937 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:8030020200716150652:ea09926d.x86_64 from CentOS-AppStream 2024-04-12T20:40:26.937 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-04-12T20:40:26.937 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-04-12T20:40:26.937 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 42: cannot install the best candidate for the job 2024-04-12T20:40:26.937 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:8030020200716174729:3a70019f.x86_64 from CentOS-AppStream 2024-04-12T20:40:26.937 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:8030020200716174729:cccafca5.x86_64 from CentOS-AppStream 2024-04-12T20:40:26.937 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:820230223114759:6fdf7e5d.x86_64 from CentOS-AppStream 2024-04-12T20:40:26.937 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:820230223114759:d4326aba.x86_64 from CentOS-AppStream 2024-04-12T20:40:26.937 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 43: cannot install the best candidate for the job 2024-04-12T20:40:26.937 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:8030020200716213634:56fce90f.x86_64 from CentOS-AppStream 2024-04-12T20:40:26.937 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:8030020200716213634:98a3c9d0.x86_64 from CentOS-AppStream 2024-04-12T20:40:26.938 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:820230223114759:06b7596e.x86_64 from CentOS-AppStream 2024-04-12T20:40:26.938 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:820230223114759:d2fc5d76.x86_64 from CentOS-AppStream 2024-04-12T20:40:26.938 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 44: cannot install the best candidate for the job 2024-04-12T20:40:26.938 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:8030020200716171139:3a70019f.x86_64 from CentOS-AppStream 2024-04-12T20:40:26.938 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:8030020200716171139:cccafca5.x86_64 from CentOS-AppStream 2024-04-12T20:40:26.938 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:820230223114759:6fdf7e5d.x86_64 from CentOS-AppStream 2024-04-12T20:40:26.938 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:820230223114759:d4326aba.x86_64 from CentOS-AppStream 2024-04-12T20:40:26.938 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 45: cannot install the best candidate for the job 2024-04-12T20:40:26.938 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:8010020191120175858:95fce66d.x86_64 from CentOS-AppStream 2024-04-12T20:40:26.938 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:8030020200723233049:09acf126.x86_64 from CentOS-AppStream 2024-04-12T20:40:26.938 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:820230223114759:436af1ae.x86_64 from CentOS-AppStream 2024-04-12T20:40:26.938 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:820230223114759:b12195ea.x86_64 from CentOS-AppStream 2024-04-12T20:40:27.010 INFO:teuthology.orchestra.run.smithi186.stdout:Dependencies resolved. 2024-04-12T20:40:27.013 INFO:teuthology.orchestra.run.smithi186.stdout:================================================================================ 2024-04-12T20:40:27.013 INFO:teuthology.orchestra.run.smithi186.stdout: Package Arch Version Repository Size 2024-04-12T20:40:27.013 INFO:teuthology.orchestra.run.smithi186.stdout:================================================================================ 2024-04-12T20:40:27.013 INFO:teuthology.orchestra.run.smithi186.stdout:Installing: 2024-04-12T20:40:27.013 INFO:teuthology.orchestra.run.smithi186.stdout: ceph x86_64 2:17.2.7-827.gd3b5060a.el9 ceph 6.5 k 2024-04-12T20:40:27.013 INFO:teuthology.orchestra.run.smithi186.stdout:Installing dependencies: 2024-04-12T20:40:27.013 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-mds x86_64 2:17.2.7-827.gd3b5060a.el9 ceph 2.1 M 2024-04-12T20:40:27.013 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-mgr x86_64 2:17.2.7-827.gd3b5060a.el9 ceph 1.4 M 2024-04-12T20:40:27.013 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-mgr-modules-core noarch 2:17.2.7-827.gd3b5060a.el9 ceph-noarch 240 k 2024-04-12T20:40:27.013 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-mon x86_64 2:17.2.7-827.gd3b5060a.el9 ceph 3.9 M 2024-04-12T20:40:27.013 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-osd x86_64 2:17.2.7-827.gd3b5060a.el9 ceph 15 M 2024-04-12T20:40:27.014 INFO:teuthology.orchestra.run.smithi186.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-04-12T20:40:27.014 INFO:teuthology.orchestra.run.smithi186.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-04-12T20:40:27.014 INFO:teuthology.orchestra.run.smithi186.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-04-12T20:40:27.014 INFO:teuthology.orchestra.run.smithi186.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-04-12T20:40:27.014 INFO:teuthology.orchestra.run.smithi186.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-04-12T20:40:27.014 INFO:teuthology.orchestra.run.smithi186.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-04-12T20:40:27.014 INFO:teuthology.orchestra.run.smithi186.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-04-12T20:40:27.014 INFO:teuthology.orchestra.run.smithi186.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-04-12T20:40:27.014 INFO:teuthology.orchestra.run.smithi186.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-04-12T20:40:27.014 INFO:teuthology.orchestra.run.smithi186.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-04-12T20:40:27.014 INFO:teuthology.orchestra.run.smithi186.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-04-12T20:40:27.014 INFO:teuthology.orchestra.run.smithi186.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-04-12T20:40:27.014 INFO:teuthology.orchestra.run.smithi186.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-04-12T20:40:27.014 INFO:teuthology.orchestra.run.smithi186.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-04-12T20:40:27.015 INFO:teuthology.orchestra.run.smithi186.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-04-12T20:40:27.015 INFO:teuthology.orchestra.run.smithi186.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-04-12T20:40:27.015 INFO:teuthology.orchestra.run.smithi186.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-04-12T20:40:27.015 INFO:teuthology.orchestra.run.smithi186.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-04-12T20:40:27.015 INFO:teuthology.orchestra.run.smithi186.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-04-12T20:40:27.015 INFO:teuthology.orchestra.run.smithi186.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-04-12T20:40:27.015 INFO:teuthology.orchestra.run.smithi186.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-04-12T20:40:27.015 INFO:teuthology.orchestra.run.smithi186.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-04-12T20:40:27.015 INFO:teuthology.orchestra.run.smithi186.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-04-12T20:40:27.015 INFO:teuthology.orchestra.run.smithi186.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-04-12T20:40:27.015 INFO:teuthology.orchestra.run.smithi186.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-04-12T20:40:27.015 INFO:teuthology.orchestra.run.smithi186.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-04-12T20:40:27.015 INFO:teuthology.orchestra.run.smithi186.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-04-12T20:40:27.015 INFO:teuthology.orchestra.run.smithi186.stdout: 2024-04-12T20:40:27.015 INFO:teuthology.orchestra.run.smithi186.stdout:Transaction Summary 2024-04-12T20:40:27.015 INFO:teuthology.orchestra.run.smithi186.stdout:================================================================================ 2024-04-12T20:40:27.016 INFO:teuthology.orchestra.run.smithi186.stdout:Install 33 Packages 2024-04-12T20:40:27.016 INFO:teuthology.orchestra.run.smithi186.stdout: 2024-04-12T20:40:27.016 INFO:teuthology.orchestra.run.smithi186.stdout:Total download size: 27 M 2024-04-12T20:40:27.016 INFO:teuthology.orchestra.run.smithi186.stdout:Installed size: 96 M 2024-04-12T20:40:27.016 INFO:teuthology.orchestra.run.smithi186.stdout:Downloading Packages: 2024-04-12T20:40:27.225 INFO:teuthology.orchestra.run.smithi018.stdout:(2/33): ceph-mgr-17.2.7-827.gd3b5060a.el9.x86_6 2.1 MB/s | 1.4 MB 00:00 2024-04-12T20:40:27.533 INFO:teuthology.orchestra.run.smithi018.stdout:(3/33): ceph-mds-17.2.7-827.gd3b5060a.el9.x86_6 2.1 MB/s | 2.1 MB 00:00 2024-04-12T20:40:27.625 INFO:teuthology.orchestra.run.smithi018.stdout:(4/33): ceph-mgr-modules-core-17.2.7-827.gd3b50 2.6 MB/s | 240 kB 00:00 2024-04-12T20:40:27.767 INFO:teuthology.orchestra.run.smithi018.stdout:(5/33): python3-cffi-1.14.5-5.el9.x86_64.rpm 1.8 MB/s | 253 kB 00:00 2024-04-12T20:40:27.805 INFO:teuthology.orchestra.run.smithi186.stdout:(1/33): ceph-17.2.7-827.gd3b5060a.el9.x86_64.rp 24 kB/s | 6.5 kB 00:00 2024-04-12T20:40:27.903 INFO:teuthology.orchestra.run.smithi018.stdout:(6/33): ceph-mon-17.2.7-827.gd3b5060a.el9.x86_6 3.8 MB/s | 3.9 MB 00:01 2024-04-12T20:40:27.960 INFO:teuthology.orchestra.run.smithi018.stdout:(7/33): python3-cryptography-36.0.1-4.el9.x86_6 6.4 MB/s | 1.2 MB 00:00 2024-04-12T20:40:28.021 INFO:teuthology.orchestra.run.smithi018.stdout:(8/33): python3-pycparser-2.20-6.el9.noarch.rpm 2.2 MB/s | 135 kB 00:00 2024-04-12T20:40:28.064 INFO:teuthology.orchestra.run.smithi186.stdout:(2/33): ceph-mgr-17.2.7-827.gd3b5060a.el9.x86_6 2.6 MB/s | 1.4 MB 00:00 2024-04-12T20:40:28.068 INFO:teuthology.orchestra.run.smithi018.stdout:(9/33): python3-requests-2.25.1-8.el9.noarch.rp 2.6 MB/s | 125 kB 00:00 2024-04-12T20:40:28.102 INFO:teuthology.orchestra.run.smithi018.stdout:(10/33): python3-ply-3.11-14.el9.noarch.rpm 533 kB/s | 106 kB 00:00 2024-04-12T20:40:28.136 INFO:teuthology.orchestra.run.smithi018.stdout:(11/33): python3-urllib3-1.26.5-5.el9.noarch.rp 3.2 MB/s | 215 kB 00:00 2024-04-12T20:40:28.403 INFO:teuthology.orchestra.run.smithi018.stdout:(12/33): ceph-osd-17.2.7-827.gd3b5060a.el9.x86_ 13 MB/s | 15 MB 00:01 2024-04-12T20:40:28.428 INFO:teuthology.orchestra.run.smithi018.stdout:(13/33): python3-markupsafe-1.1.1-12.el9.x86_64 119 kB/s | 35 kB 00:00 2024-04-12T20:40:28.520 INFO:teuthology.orchestra.run.smithi018.stdout:(14/33): python3-mako-1.1.4-6.el9.noarch.rpm 413 kB/s | 172 kB 00:00 2024-04-12T20:40:28.554 INFO:teuthology.orchestra.run.smithi018.stdout:(15/33): python3-toml-0.10.2-6.el9.noarch.rpm 334 kB/s | 42 kB 00:00 2024-04-12T20:40:28.579 INFO:teuthology.orchestra.run.smithi018.stdout:(16/33): python3-pytz-2021.1-5.el9.noarch.rpm 291 kB/s | 51 kB 00:00 2024-04-12T20:40:28.871 INFO:teuthology.orchestra.run.smithi018.stdout:(17/33): python3-bcrypt-3.2.2-1.el9.x86_64.rpm 124 kB/s | 43 kB 00:00 2024-04-12T20:40:28.948 INFO:teuthology.orchestra.run.smithi186.stdout:(3/33): ceph-osd-17.2.7-827.gd3b5060a.el9.x86_6 17 MB/s | 15 MB 00:00 2024-04-12T20:40:28.979 INFO:teuthology.orchestra.run.smithi018.stdout:(18/33): python3-cheroot-8.6.0-4.el9.noarch.rpm 404 kB/s | 172 kB 00:00 2024-04-12T20:40:29.005 INFO:teuthology.orchestra.run.smithi018.stdout:(19/33): python3-jaraco-8.2.1-3.el9.noarch.rpm 80 kB/s | 11 kB 00:00 2024-04-12T20:40:29.039 INFO:teuthology.orchestra.run.smithi186.stdout:(4/33): ceph-mon-17.2.7-827.gd3b5060a.el9.x86_6 3.2 MB/s | 3.9 MB 00:01 2024-04-12T20:40:29.055 INFO:teuthology.orchestra.run.smithi018.stdout:(20/33): python3-cherrypy-18.6.1-2.el9.noarch.r 754 kB/s | 358 kB 00:00 2024-04-12T20:40:29.073 INFO:teuthology.orchestra.run.smithi186.stdout:(5/33): ceph-mgr-modules-core-17.2.7-827.gd3b50 1.9 MB/s | 240 kB 00:00 2024-04-12T20:40:29.106 INFO:teuthology.orchestra.run.smithi018.stdout:(21/33): python3-jaraco-collections-3.0.0-8.el9 234 kB/s | 23 kB 00:00 2024-04-12T20:40:29.131 INFO:teuthology.orchestra.run.smithi018.stdout:(22/33): python3-jaraco-classes-3.2.1-5.el9.noa 142 kB/s | 18 kB 00:00 2024-04-12T20:40:29.156 INFO:teuthology.orchestra.run.smithi018.stdout:(23/33): python3-jaraco-functools-3.5.0-2.el9.n 194 kB/s | 19 kB 00:00 2024-04-12T20:40:29.157 INFO:teuthology.orchestra.run.smithi186.stdout:(6/33): ceph-mds-17.2.7-827.gd3b5060a.el9.x86_6 1.3 MB/s | 2.1 MB 00:01 2024-04-12T20:40:29.206 INFO:teuthology.orchestra.run.smithi018.stdout:(24/33): python3-jaraco-text-3.2.0-6.el9.noarch 197 kB/s | 20 kB 00:00 2024-04-12T20:40:29.240 INFO:teuthology.orchestra.run.smithi018.stdout:(25/33): python3-logutils-0.3.5-21.el9.noarch.r 428 kB/s | 46 kB 00:00 2024-04-12T20:40:29.274 INFO:teuthology.orchestra.run.smithi018.stdout:(26/33): python3-more-itertools-8.12.0-2.el9.no 675 kB/s | 79 kB 00:00 2024-04-12T20:40:29.299 INFO:teuthology.orchestra.run.smithi186.stdout:(7/33): python3-cffi-1.14.5-5.el9.x86_64.rpm 981 kB/s | 253 kB 00:00 2024-04-12T20:40:29.349 INFO:teuthology.orchestra.run.smithi186.stdout:(8/33): python3-ply-3.11-14.el9.noarch.rpm 558 kB/s | 106 kB 00:00 2024-04-12T20:40:29.372 INFO:teuthology.orchestra.run.smithi018.stdout:(27/33): python3-portend-3.1.0-2.el9.noarch.rpm 125 kB/s | 16 kB 00:00 2024-04-12T20:40:29.383 INFO:teuthology.orchestra.run.smithi186.stdout:(9/33): python3-pycparser-2.20-6.el9.noarch.rpm 1.6 MB/s | 135 kB 00:00 2024-04-12T20:40:29.408 INFO:teuthology.orchestra.run.smithi018.stdout:(28/33): python3-pyOpenSSL-21.0.0-1.el9.noarch. 675 kB/s | 90 kB 00:00 2024-04-12T20:40:29.441 INFO:teuthology.orchestra.run.smithi186.stdout:(10/33): python3-requests-2.25.1-8.el9.noarch.r 1.3 MB/s | 125 kB 00:00 2024-04-12T20:40:29.475 INFO:teuthology.orchestra.run.smithi186.stdout:(11/33): python3-urllib3-1.26.5-5.el9.noarch.rp 2.3 MB/s | 215 kB 00:00 2024-04-12T20:40:29.491 INFO:teuthology.orchestra.run.smithi018.stdout:(29/33): python3-pecan-1.4.2-3.el9.noarch.rpm 959 kB/s | 272 kB 00:00 2024-04-12T20:40:29.516 INFO:teuthology.orchestra.run.smithi018.stdout:(30/33): python3-tempora-5.0.0-2.el9.noarch.rpm 250 kB/s | 36 kB 00:00 2024-04-12T20:40:29.525 INFO:teuthology.orchestra.run.smithi186.stdout:(12/33): python3-cryptography-36.0.1-4.el9.x86_ 2.7 MB/s | 1.2 MB 00:00 2024-04-12T20:40:29.550 INFO:teuthology.orchestra.run.smithi018.stdout:(31/33): python3-webob-1.8.7-6.el9.noarch.rpm 1.6 MB/s | 230 kB 00:00 2024-04-12T20:40:29.551 INFO:teuthology.orchestra.run.smithi186.stdout:(13/33): python3-markupsafe-1.1.1-12.el9.x86_64 465 kB/s | 35 kB 00:00 2024-04-12T20:40:29.576 INFO:teuthology.orchestra.run.smithi186.stdout:(14/33): python3-mako-1.1.4-6.el9.noarch.rpm 1.3 MB/s | 172 kB 00:00 2024-04-12T20:40:29.615 INFO:teuthology.orchestra.run.smithi186.stdout:(15/33): python3-pytz-2021.1-5.el9.noarch.rpm 574 kB/s | 51 kB 00:00 2024-04-12T20:40:29.633 INFO:teuthology.orchestra.run.smithi018.stdout:(32/33): python3-zc-lockfile-2.0-10.el9.noarch. 173 kB/s | 20 kB 00:00 2024-04-12T20:40:29.677 INFO:teuthology.orchestra.run.smithi186.stdout:(16/33): python3-toml-0.10.2-6.el9.noarch.rpm 333 kB/s | 42 kB 00:00 2024-04-12T20:40:29.717 INFO:teuthology.orchestra.run.smithi018.stdout:(33/33): python3-werkzeug-2.0.3-3.el9.1.noarch. 2.1 MB/s | 427 kB 00:00 2024-04-12T20:40:29.721 INFO:teuthology.orchestra.run.smithi018.stdout:-------------------------------------------------------------------------------- 2024-04-12T20:40:29.721 INFO:teuthology.orchestra.run.smithi018.stdout:Total 6.6 MB/s | 27 MB 00:04 2024-04-12T20:40:29.835 INFO:teuthology.orchestra.run.smithi186.stdout:(17/33): python3-bcrypt-3.2.2-1.el9.x86_64.rpm 168 kB/s | 43 kB 00:00 2024-04-12T20:40:29.910 INFO:teuthology.orchestra.run.smithi186.stdout:(18/33): python3-jaraco-8.2.1-3.el9.noarch.rpm 143 kB/s | 11 kB 00:00 2024-04-12T20:40:29.946 INFO:teuthology.orchestra.run.smithi186.stdout:(19/33): python3-cheroot-8.6.0-4.el9.noarch.rpm 524 kB/s | 172 kB 00:00 2024-04-12T20:40:29.994 INFO:teuthology.orchestra.run.smithi186.stdout:(20/33): python3-jaraco-classes-3.2.1-5.el9.noa 212 kB/s | 18 kB 00:00 2024-04-12T20:40:30.028 INFO:teuthology.orchestra.run.smithi186.stdout:(21/33): python3-jaraco-collections-3.0.0-8.el9 279 kB/s | 23 kB 00:00 2024-04-12T20:40:30.062 INFO:teuthology.orchestra.run.smithi186.stdout:(22/33): python3-cherrypy-18.6.1-2.el9.noarch.r 933 kB/s | 358 kB 00:00 2024-04-12T20:40:30.087 INFO:teuthology.orchestra.run.smithi186.stdout:(23/33): python3-jaraco-functools-3.5.0-2.el9.n 212 kB/s | 19 kB 00:00 2024-04-12T20:40:30.087 INFO:teuthology.orchestra.run.smithi018.stdout:Running transaction check 2024-04-12T20:40:30.112 INFO:teuthology.orchestra.run.smithi186.stdout:(24/33): python3-jaraco-text-3.2.0-6.el9.noarch 236 kB/s | 20 kB 00:00 2024-04-12T20:40:30.154 INFO:teuthology.orchestra.run.smithi186.stdout:(25/33): python3-logutils-0.3.5-21.el9.noarch.r 506 kB/s | 46 kB 00:00 2024-04-12T20:40:30.176 INFO:teuthology.orchestra.run.smithi018.stdout:Transaction check succeeded. 2024-04-12T20:40:30.176 INFO:teuthology.orchestra.run.smithi018.stdout:Running transaction test 2024-04-12T20:40:30.221 INFO:teuthology.orchestra.run.smithi186.stdout:(26/33): python3-more-itertools-8.12.0-2.el9.no 592 kB/s | 79 kB 00:00 2024-04-12T20:40:30.246 INFO:teuthology.orchestra.run.smithi186.stdout:(27/33): python3-portend-3.1.0-2.el9.noarch.rpm 180 kB/s | 16 kB 00:00 2024-04-12T20:40:30.280 INFO:teuthology.orchestra.run.smithi186.stdout:(28/33): python3-pecan-1.4.2-3.el9.noarch.rpm 1.6 MB/s | 272 kB 00:00 2024-04-12T20:40:30.305 INFO:teuthology.orchestra.run.smithi186.stdout:(29/33): python3-pyOpenSSL-21.0.0-1.el9.noarch. 1.1 MB/s | 90 kB 00:00 2024-04-12T20:40:30.331 INFO:teuthology.orchestra.run.smithi186.stdout:(30/33): python3-tempora-5.0.0-2.el9.noarch.rpm 429 kB/s | 36 kB 00:00 2024-04-12T20:40:30.373 INFO:teuthology.orchestra.run.smithi186.stdout:(31/33): python3-webob-1.8.7-6.el9.noarch.rpm 2.4 MB/s | 230 kB 00:00 2024-04-12T20:40:30.423 INFO:teuthology.orchestra.run.smithi186.stdout:(32/33): python3-zc-lockfile-2.0-10.el9.noarch. 219 kB/s | 20 kB 00:00 2024-04-12T20:40:30.465 INFO:teuthology.orchestra.run.smithi186.stdout:(33/33): python3-werkzeug-2.0.3-3.el9.1.noarch. 2.6 MB/s | 427 kB 00:00 2024-04-12T20:40:30.468 INFO:teuthology.orchestra.run.smithi186.stdout:-------------------------------------------------------------------------------- 2024-04-12T20:40:30.468 INFO:teuthology.orchestra.run.smithi186.stdout:Total 7.9 MB/s | 27 MB 00:03 2024-04-12T20:40:30.620 INFO:teuthology.orchestra.run.smithi018.stdout:Transaction test succeeded. 2024-04-12T20:40:30.620 INFO:teuthology.orchestra.run.smithi018.stdout:Running transaction 2024-04-12T20:40:30.838 INFO:teuthology.orchestra.run.smithi186.stdout:Running transaction check 2024-04-12T20:40:30.920 INFO:teuthology.orchestra.run.smithi186.stdout:Transaction check succeeded. 2024-04-12T20:40:30.921 INFO:teuthology.orchestra.run.smithi186.stdout:Running transaction test 2024-04-12T20:40:31.370 INFO:teuthology.orchestra.run.smithi186.stdout:Transaction test succeeded. 2024-04-12T20:40:31.371 INFO:teuthology.orchestra.run.smithi186.stdout:Running transaction 2024-04-12T20:40:32.077 INFO:teuthology.orchestra.run.smithi018.stdout: Preparing : 1/1 2024-04-12T20:40:32.137 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 1/33 2024-04-12T20:40:32.184 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 2/33 2024-04-12T20:40:32.236 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 3/33 2024-04-12T20:40:32.381 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 4/33 2024-04-12T20:40:32.485 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 5/33 2024-04-12T20:40:32.545 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 6/33 2024-04-12T20:40:32.587 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 7/33 2024-04-12T20:40:32.623 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 8/33 2024-04-12T20:40:32.661 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 9/33 2024-04-12T20:40:32.705 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 10/33 2024-04-12T20:40:32.768 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 11/33 2024-04-12T20:40:32.772 INFO:teuthology.orchestra.run.smithi186.stdout: Preparing : 1/1 2024-04-12T20:40:32.828 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 12/33 2024-04-12T20:40:32.873 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 13/33 2024-04-12T20:40:32.877 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 1/33 2024-04-12T20:40:32.937 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 2/33 2024-04-12T20:40:32.960 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 14/33 2024-04-12T20:40:32.989 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 3/33 2024-04-12T20:40:33.105 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 15/33 2024-04-12T20:40:33.142 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 4/33 2024-04-12T20:40:33.243 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 5/33 2024-04-12T20:40:33.300 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 6/33 2024-04-12T20:40:33.308 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 16/33 2024-04-12T20:40:33.341 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 7/33 2024-04-12T20:40:33.369 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 8/33 2024-04-12T20:40:33.404 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 9/33 2024-04-12T20:40:33.432 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 17/33 2024-04-12T20:40:33.441 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 10/33 2024-04-12T20:40:33.501 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 11/33 2024-04-12T20:40:33.507 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 18/33 2024-04-12T20:40:33.549 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 19/33 2024-04-12T20:40:33.564 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 12/33 2024-04-12T20:40:33.603 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 13/33 2024-04-12T20:40:33.616 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : python3-ply-3.11-14.el9.noarch 20/33 2024-04-12T20:40:33.667 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 14/33 2024-04-12T20:40:33.715 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 21/33 2024-04-12T20:40:33.797 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 15/33 2024-04-12T20:40:33.984 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 22/33 2024-04-12T20:40:33.989 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 16/33 2024-04-12T20:40:34.054 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 23/33 2024-04-12T20:40:34.125 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 17/33 2024-04-12T20:40:34.146 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 24/33 2024-04-12T20:40:34.187 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 18/33 2024-04-12T20:40:34.229 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 19/33 2024-04-12T20:40:34.267 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 25/33 2024-04-12T20:40:34.294 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-ply-3.11-14.el9.noarch 20/33 2024-04-12T20:40:34.325 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 26/33 2024-04-12T20:40:34.366 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 21/33 2024-04-12T20:40:34.488 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 27/33 2024-04-12T20:40:34.629 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 22/33 2024-04-12T20:40:34.665 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : ceph-mgr-modules-core-2:17.2.7-827.gd3b5060a.el9.n 28/33 2024-04-12T20:40:34.702 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 23/33 2024-04-12T20:40:34.702 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : ceph-mgr-2:17.2.7-827.gd3b5060a.el9.x86_64 29/33 2024-04-12T20:40:34.781 INFO:teuthology.orchestra.run.smithi018.stdout: Running scriptlet: ceph-mgr-2:17.2.7-827.gd3b5060a.el9.x86_64 29/33 2024-04-12T20:40:34.781 INFO:teuthology.orchestra.run.smithi018.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-12T20:40:34.781 INFO:teuthology.orchestra.run.smithi018.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-04-12T20:40:34.782 INFO:teuthology.orchestra.run.smithi018.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-12T20:40:34.782 INFO:teuthology.orchestra.run.smithi018.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-12T20:40:34.782 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-12T20:40:34.791 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 24/33 2024-04-12T20:40:34.904 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 25/33 2024-04-12T20:40:34.953 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 26/33 2024-04-12T20:40:35.096 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 27/33 2024-04-12T20:40:35.258 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : ceph-mgr-modules-core-2:17.2.7-827.gd3b5060a.el9.n 28/33 2024-04-12T20:40:35.289 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : ceph-mgr-2:17.2.7-827.gd3b5060a.el9.x86_64 29/33 2024-04-12T20:40:35.363 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: ceph-mgr-2:17.2.7-827.gd3b5060a.el9.x86_64 29/33 2024-04-12T20:40:35.363 INFO:teuthology.orchestra.run.smithi186.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-12T20:40:35.363 INFO:teuthology.orchestra.run.smithi186.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-04-12T20:40:35.363 INFO:teuthology.orchestra.run.smithi186.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-12T20:40:35.363 INFO:teuthology.orchestra.run.smithi186.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-12T20:40:35.363 INFO:teuthology.orchestra.run.smithi186.stdout: 2024-04-12T20:40:36.303 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : ceph-osd-2:17.2.7-827.gd3b5060a.el9.x86_64 30/33 2024-04-12T20:40:36.403 INFO:teuthology.orchestra.run.smithi018.stdout: Running scriptlet: ceph-osd-2:17.2.7-827.gd3b5060a.el9.x86_64 30/33 2024-04-12T20:40:36.403 INFO:teuthology.orchestra.run.smithi018.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-12T20:40:36.403 INFO:teuthology.orchestra.run.smithi018.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-04-12T20:40:36.403 INFO:teuthology.orchestra.run.smithi018.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-12T20:40:36.403 INFO:teuthology.orchestra.run.smithi018.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-12T20:40:36.404 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-12T20:40:36.773 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : ceph-osd-2:17.2.7-827.gd3b5060a.el9.x86_64 30/33 2024-04-12T20:40:36.828 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : ceph-mon-2:17.2.7-827.gd3b5060a.el9.x86_64 31/33 2024-04-12T20:40:36.854 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: ceph-osd-2:17.2.7-827.gd3b5060a.el9.x86_64 30/33 2024-04-12T20:40:36.855 INFO:teuthology.orchestra.run.smithi186.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-12T20:40:36.855 INFO:teuthology.orchestra.run.smithi186.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-04-12T20:40:36.855 INFO:teuthology.orchestra.run.smithi186.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-12T20:40:36.855 INFO:teuthology.orchestra.run.smithi186.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-12T20:40:36.855 INFO:teuthology.orchestra.run.smithi186.stdout: 2024-04-12T20:40:36.898 INFO:teuthology.orchestra.run.smithi018.stdout: Running scriptlet: ceph-mon-2:17.2.7-827.gd3b5060a.el9.x86_64 31/33 2024-04-12T20:40:36.899 INFO:teuthology.orchestra.run.smithi018.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-12T20:40:36.899 INFO:teuthology.orchestra.run.smithi018.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-04-12T20:40:36.899 INFO:teuthology.orchestra.run.smithi018.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-12T20:40:36.899 INFO:teuthology.orchestra.run.smithi018.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-12T20:40:36.899 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-12T20:40:37.237 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : ceph-mds-2:17.2.7-827.gd3b5060a.el9.x86_64 32/33 2024-04-12T20:40:37.257 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : ceph-mon-2:17.2.7-827.gd3b5060a.el9.x86_64 31/33 2024-04-12T20:40:37.320 INFO:teuthology.orchestra.run.smithi018.stdout: Running scriptlet: ceph-mds-2:17.2.7-827.gd3b5060a.el9.x86_64 32/33 2024-04-12T20:40:37.321 INFO:teuthology.orchestra.run.smithi018.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-12T20:40:37.321 INFO:teuthology.orchestra.run.smithi018.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-04-12T20:40:37.321 INFO:teuthology.orchestra.run.smithi018.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-12T20:40:37.321 INFO:teuthology.orchestra.run.smithi018.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-12T20:40:37.321 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-12T20:40:37.325 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: ceph-mon-2:17.2.7-827.gd3b5060a.el9.x86_64 31/33 2024-04-12T20:40:37.325 INFO:teuthology.orchestra.run.smithi186.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-12T20:40:37.325 INFO:teuthology.orchestra.run.smithi186.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-04-12T20:40:37.326 INFO:teuthology.orchestra.run.smithi186.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-12T20:40:37.326 INFO:teuthology.orchestra.run.smithi186.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-12T20:40:37.326 INFO:teuthology.orchestra.run.smithi186.stdout: 2024-04-12T20:40:37.397 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : ceph-2:17.2.7-827.gd3b5060a.el9.x86_64 33/33 2024-04-12T20:40:37.545 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : ceph-mds-2:17.2.7-827.gd3b5060a.el9.x86_64 32/33 2024-04-12T20:40:37.567 INFO:teuthology.orchestra.run.smithi018.stdout: Running scriptlet: ceph-2:17.2.7-827.gd3b5060a.el9.x86_64 33/33 2024-04-12T20:40:37.567 INFO:teuthology.orchestra.run.smithi018.stdout:Couldn't write '0' to 'kernel/yama/ptrace_scope', ignoring: No such file or directory 2024-04-12T20:40:37.567 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-12T20:40:37.616 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: ceph-mds-2:17.2.7-827.gd3b5060a.el9.x86_64 32/33 2024-04-12T20:40:37.617 INFO:teuthology.orchestra.run.smithi186.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-12T20:40:37.617 INFO:teuthology.orchestra.run.smithi186.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-04-12T20:40:37.617 INFO:teuthology.orchestra.run.smithi186.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-12T20:40:37.617 INFO:teuthology.orchestra.run.smithi186.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-12T20:40:37.617 INFO:teuthology.orchestra.run.smithi186.stdout: 2024-04-12T20:40:37.711 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : ceph-2:17.2.7-827.gd3b5060a.el9.x86_64 33/33 2024-04-12T20:40:37.842 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: ceph-2:17.2.7-827.gd3b5060a.el9.x86_64 33/33 2024-04-12T20:40:37.842 INFO:teuthology.orchestra.run.smithi186.stdout:Couldn't write '0' to 'kernel/yama/ptrace_scope', ignoring: No such file or directory 2024-04-12T20:40:37.842 INFO:teuthology.orchestra.run.smithi186.stdout: 2024-04-12T20:40:39.205 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : ceph-2:17.2.7-827.gd3b5060a.el9.x86_64 1/33 2024-04-12T20:40:39.206 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : ceph-mds-2:17.2.7-827.gd3b5060a.el9.x86_64 2/33 2024-04-12T20:40:39.206 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : ceph-mgr-2:17.2.7-827.gd3b5060a.el9.x86_64 3/33 2024-04-12T20:40:39.206 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : ceph-mon-2:17.2.7-827.gd3b5060a.el9.x86_64 4/33 2024-04-12T20:40:39.206 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : ceph-osd-2:17.2.7-827.gd3b5060a.el9.x86_64 5/33 2024-04-12T20:40:39.206 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : ceph-mgr-modules-core-2:17.2.7-827.gd3b5060a.el9.n 6/33 2024-04-12T20:40:39.206 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 7/33 2024-04-12T20:40:39.206 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 8/33 2024-04-12T20:40:39.206 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : python3-ply-3.11-14.el9.noarch 9/33 2024-04-12T20:40:39.207 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 10/33 2024-04-12T20:40:39.207 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 11/33 2024-04-12T20:40:39.207 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 12/33 2024-04-12T20:40:39.207 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 13/33 2024-04-12T20:40:39.207 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 14/33 2024-04-12T20:40:39.207 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 15/33 2024-04-12T20:40:39.207 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 16/33 2024-04-12T20:40:39.207 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 17/33 2024-04-12T20:40:39.207 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 18/33 2024-04-12T20:40:39.207 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 19/33 2024-04-12T20:40:39.208 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 20/33 2024-04-12T20:40:39.208 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 21/33 2024-04-12T20:40:39.208 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 22/33 2024-04-12T20:40:39.208 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 23/33 2024-04-12T20:40:39.208 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 24/33 2024-04-12T20:40:39.208 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 25/33 2024-04-12T20:40:39.208 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 26/33 2024-04-12T20:40:39.208 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 27/33 2024-04-12T20:40:39.208 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 28/33 2024-04-12T20:40:39.209 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 29/33 2024-04-12T20:40:39.209 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 30/33 2024-04-12T20:40:39.209 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 31/33 2024-04-12T20:40:39.209 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 32/33 2024-04-12T20:40:39.531 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : ceph-2:17.2.7-827.gd3b5060a.el9.x86_64 1/33 2024-04-12T20:40:39.531 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : ceph-mds-2:17.2.7-827.gd3b5060a.el9.x86_64 2/33 2024-04-12T20:40:39.531 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : ceph-mgr-2:17.2.7-827.gd3b5060a.el9.x86_64 3/33 2024-04-12T20:40:39.531 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : ceph-mon-2:17.2.7-827.gd3b5060a.el9.x86_64 4/33 2024-04-12T20:40:39.531 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : ceph-osd-2:17.2.7-827.gd3b5060a.el9.x86_64 5/33 2024-04-12T20:40:39.531 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : ceph-mgr-modules-core-2:17.2.7-827.gd3b5060a.el9.n 6/33 2024-04-12T20:40:39.531 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 7/33 2024-04-12T20:40:39.531 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 8/33 2024-04-12T20:40:39.531 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-ply-3.11-14.el9.noarch 9/33 2024-04-12T20:40:39.531 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 10/33 2024-04-12T20:40:39.531 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 11/33 2024-04-12T20:40:39.531 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 12/33 2024-04-12T20:40:39.531 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 13/33 2024-04-12T20:40:39.532 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 14/33 2024-04-12T20:40:39.532 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 15/33 2024-04-12T20:40:39.532 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 16/33 2024-04-12T20:40:39.532 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 17/33 2024-04-12T20:40:39.532 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 18/33 2024-04-12T20:40:39.532 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 19/33 2024-04-12T20:40:39.532 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 20/33 2024-04-12T20:40:39.532 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 21/33 2024-04-12T20:40:39.532 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 22/33 2024-04-12T20:40:39.532 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 23/33 2024-04-12T20:40:39.532 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 24/33 2024-04-12T20:40:39.532 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 25/33 2024-04-12T20:40:39.532 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 26/33 2024-04-12T20:40:39.533 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 27/33 2024-04-12T20:40:39.533 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 28/33 2024-04-12T20:40:39.533 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 29/33 2024-04-12T20:40:39.533 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 30/33 2024-04-12T20:40:39.534 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 31/33 2024-04-12T20:40:39.534 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 32/33 2024-04-12T20:40:40.339 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 33/33 2024-04-12T20:40:40.339 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-12T20:40:40.339 INFO:teuthology.orchestra.run.smithi018.stdout:Installed: 2024-04-12T20:40:40.340 INFO:teuthology.orchestra.run.smithi018.stdout: ceph-2:17.2.7-827.gd3b5060a.el9.x86_64 2024-04-12T20:40:40.340 INFO:teuthology.orchestra.run.smithi018.stdout: ceph-mds-2:17.2.7-827.gd3b5060a.el9.x86_64 2024-04-12T20:40:40.340 INFO:teuthology.orchestra.run.smithi018.stdout: ceph-mgr-2:17.2.7-827.gd3b5060a.el9.x86_64 2024-04-12T20:40:40.340 INFO:teuthology.orchestra.run.smithi018.stdout: ceph-mgr-modules-core-2:17.2.7-827.gd3b5060a.el9.noarch 2024-04-12T20:40:40.340 INFO:teuthology.orchestra.run.smithi018.stdout: ceph-mon-2:17.2.7-827.gd3b5060a.el9.x86_64 2024-04-12T20:40:40.340 INFO:teuthology.orchestra.run.smithi018.stdout: ceph-osd-2:17.2.7-827.gd3b5060a.el9.x86_64 2024-04-12T20:40:40.340 INFO:teuthology.orchestra.run.smithi018.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-04-12T20:40:40.340 INFO:teuthology.orchestra.run.smithi018.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-04-12T20:40:40.340 INFO:teuthology.orchestra.run.smithi018.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-04-12T20:40:40.340 INFO:teuthology.orchestra.run.smithi018.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-04-12T20:40:40.340 INFO:teuthology.orchestra.run.smithi018.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-04-12T20:40:40.340 INFO:teuthology.orchestra.run.smithi018.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-04-12T20:40:40.341 INFO:teuthology.orchestra.run.smithi018.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-04-12T20:40:40.341 INFO:teuthology.orchestra.run.smithi018.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-04-12T20:40:40.341 INFO:teuthology.orchestra.run.smithi018.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-04-12T20:40:40.341 INFO:teuthology.orchestra.run.smithi018.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-04-12T20:40:40.341 INFO:teuthology.orchestra.run.smithi018.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-04-12T20:40:40.341 INFO:teuthology.orchestra.run.smithi018.stdout: python3-mako-1.1.4-6.el9.noarch 2024-04-12T20:40:40.341 INFO:teuthology.orchestra.run.smithi018.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-04-12T20:40:40.341 INFO:teuthology.orchestra.run.smithi018.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-04-12T20:40:40.341 INFO:teuthology.orchestra.run.smithi018.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-04-12T20:40:40.341 INFO:teuthology.orchestra.run.smithi018.stdout: python3-ply-3.11-14.el9.noarch 2024-04-12T20:40:40.341 INFO:teuthology.orchestra.run.smithi018.stdout: python3-portend-3.1.0-2.el9.noarch 2024-04-12T20:40:40.341 INFO:teuthology.orchestra.run.smithi018.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-04-12T20:40:40.341 INFO:teuthology.orchestra.run.smithi018.stdout: python3-pycparser-2.20-6.el9.noarch 2024-04-12T20:40:40.341 INFO:teuthology.orchestra.run.smithi018.stdout: python3-pytz-2021.1-5.el9.noarch 2024-04-12T20:40:40.341 INFO:teuthology.orchestra.run.smithi018.stdout: python3-requests-2.25.1-8.el9.noarch 2024-04-12T20:40:40.342 INFO:teuthology.orchestra.run.smithi018.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-04-12T20:40:40.342 INFO:teuthology.orchestra.run.smithi018.stdout: python3-toml-0.10.2-6.el9.noarch 2024-04-12T20:40:40.342 INFO:teuthology.orchestra.run.smithi018.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-04-12T20:40:40.342 INFO:teuthology.orchestra.run.smithi018.stdout: python3-webob-1.8.7-6.el9.noarch 2024-04-12T20:40:40.342 INFO:teuthology.orchestra.run.smithi018.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-04-12T20:40:40.342 INFO:teuthology.orchestra.run.smithi018.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-04-12T20:40:40.342 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-12T20:40:40.342 INFO:teuthology.orchestra.run.smithi018.stdout:Complete! 2024-04-12T20:40:40.727 DEBUG:teuthology.orchestra.run.smithi018:> sudo yum -y install ceph-base 2024-04-12T20:40:40.739 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 33/33 2024-04-12T20:40:40.739 INFO:teuthology.orchestra.run.smithi186.stdout: 2024-04-12T20:40:40.740 INFO:teuthology.orchestra.run.smithi186.stdout:Installed: 2024-04-12T20:40:40.740 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-2:17.2.7-827.gd3b5060a.el9.x86_64 2024-04-12T20:40:40.740 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-mds-2:17.2.7-827.gd3b5060a.el9.x86_64 2024-04-12T20:40:40.740 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-mgr-2:17.2.7-827.gd3b5060a.el9.x86_64 2024-04-12T20:40:40.740 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-mgr-modules-core-2:17.2.7-827.gd3b5060a.el9.noarch 2024-04-12T20:40:40.740 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-mon-2:17.2.7-827.gd3b5060a.el9.x86_64 2024-04-12T20:40:40.740 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-osd-2:17.2.7-827.gd3b5060a.el9.x86_64 2024-04-12T20:40:40.740 INFO:teuthology.orchestra.run.smithi186.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-04-12T20:40:40.740 INFO:teuthology.orchestra.run.smithi186.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-04-12T20:40:40.740 INFO:teuthology.orchestra.run.smithi186.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-04-12T20:40:40.740 INFO:teuthology.orchestra.run.smithi186.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-04-12T20:40:40.740 INFO:teuthology.orchestra.run.smithi186.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-04-12T20:40:40.740 INFO:teuthology.orchestra.run.smithi186.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-04-12T20:40:40.740 INFO:teuthology.orchestra.run.smithi186.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-04-12T20:40:40.741 INFO:teuthology.orchestra.run.smithi186.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-04-12T20:40:40.741 INFO:teuthology.orchestra.run.smithi186.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-04-12T20:40:40.741 INFO:teuthology.orchestra.run.smithi186.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-04-12T20:40:40.741 INFO:teuthology.orchestra.run.smithi186.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-04-12T20:40:40.741 INFO:teuthology.orchestra.run.smithi186.stdout: python3-mako-1.1.4-6.el9.noarch 2024-04-12T20:40:40.741 INFO:teuthology.orchestra.run.smithi186.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-04-12T20:40:40.741 INFO:teuthology.orchestra.run.smithi186.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-04-12T20:40:40.741 INFO:teuthology.orchestra.run.smithi186.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-04-12T20:40:40.741 INFO:teuthology.orchestra.run.smithi186.stdout: python3-ply-3.11-14.el9.noarch 2024-04-12T20:40:40.741 INFO:teuthology.orchestra.run.smithi186.stdout: python3-portend-3.1.0-2.el9.noarch 2024-04-12T20:40:40.741 INFO:teuthology.orchestra.run.smithi186.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-04-12T20:40:40.741 INFO:teuthology.orchestra.run.smithi186.stdout: python3-pycparser-2.20-6.el9.noarch 2024-04-12T20:40:40.741 INFO:teuthology.orchestra.run.smithi186.stdout: python3-pytz-2021.1-5.el9.noarch 2024-04-12T20:40:40.741 INFO:teuthology.orchestra.run.smithi186.stdout: python3-requests-2.25.1-8.el9.noarch 2024-04-12T20:40:40.741 INFO:teuthology.orchestra.run.smithi186.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-04-12T20:40:40.742 INFO:teuthology.orchestra.run.smithi186.stdout: python3-toml-0.10.2-6.el9.noarch 2024-04-12T20:40:40.742 INFO:teuthology.orchestra.run.smithi186.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-04-12T20:40:40.742 INFO:teuthology.orchestra.run.smithi186.stdout: python3-webob-1.8.7-6.el9.noarch 2024-04-12T20:40:40.742 INFO:teuthology.orchestra.run.smithi186.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-04-12T20:40:40.742 INFO:teuthology.orchestra.run.smithi186.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-04-12T20:40:40.742 INFO:teuthology.orchestra.run.smithi186.stdout: 2024-04-12T20:40:40.742 INFO:teuthology.orchestra.run.smithi186.stdout:Complete! 2024-04-12T20:40:41.172 DEBUG:teuthology.orchestra.run.smithi186:> sudo yum -y install ceph-base 2024-04-12T20:40:41.633 INFO:teuthology.orchestra.run.smithi018.stdout:Last metadata expiration check: 0:01:06 ago on Fri 12 Apr 2024 08:39:35 PM UTC. 2024-04-12T20:40:41.951 INFO:teuthology.orchestra.run.smithi018.stderr:Modular dependency problems: 2024-04-12T20:40:41.951 INFO:teuthology.orchestra.run.smithi018.stderr: 2024-04-12T20:40:41.951 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 1: conflicting requests 2024-04-12T20:40:41.951 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module container-tools:rhel8:820240321091303:20125149.x86_64 from CentOS-AppStream 2024-04-12T20:40:41.951 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 2: conflicting requests 2024-04-12T20:40:41.951 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module eclipse:rhel8:820201023100746:b230ed4e.x86_64 from CentOS-AppStream 2024-04-12T20:40:41.951 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 3: conflicting requests 2024-04-12T20:40:41.951 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module freeradius:3.0:8080020230117180605:89170a74.x86_64 from CentOS-AppStream 2024-04-12T20:40:41.951 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 4: conflicting requests 2024-04-12T20:40:41.951 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module gimp:2.8:8000020190628145146:4148dfdf.x86_64 from CentOS-AppStream 2024-04-12T20:40:41.952 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 5: conflicting requests 2024-04-12T20:40:41.952 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module go-toolset:rhel8:820240401160413:b754926a.x86_64 from CentOS-AppStream 2024-04-12T20:40:41.952 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 6: conflicting requests 2024-04-12T20:40:41.952 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module httpd:2.4:820240216084734:9edba152.x86_64 from CentOS-AppStream 2024-04-12T20:40:41.952 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 7: conflicting requests 2024-04-12T20:40:41.952 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module idm:client:820240307184541:49cc9d1b.x86_64 from CentOS-AppStream 2024-04-12T20:40:41.952 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 8: conflicting requests 2024-04-12T20:40:41.952 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module inkscape:0.92.3:8080020221205124429:3e031279.x86_64 from CentOS-AppStream 2024-04-12T20:40:41.952 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 9: conflicting requests 2024-04-12T20:40:41.952 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module jmc:rhel8:8050020211110222101:6392b1f8.x86_64 from CentOS-AppStream 2024-04-12T20:40:41.952 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 10: conflicting requests 2024-04-12T20:40:41.952 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module llvm-toolset:rhel8:820240205133135:9edba152.x86_64 from CentOS-AppStream 2024-04-12T20:40:41.952 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 11: conflicting requests 2024-04-12T20:40:41.952 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module log4j:2:8080020221205124743:9d367344.x86_64 from CentOS-AppStream 2024-04-12T20:40:41.953 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 12: conflicting requests 2024-04-12T20:40:41.953 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module mailman:2.1:820230727085757:77fc8825.x86_64 from CentOS-AppStream 2024-04-12T20:40:41.953 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 13: conflicting requests 2024-04-12T20:40:41.953 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module mariadb:10.3:8030020210419150013:30b713e6.x86_64 from CentOS-AppStream 2024-04-12T20:40:41.953 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 14: conflicting requests 2024-04-12T20:40:41.953 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module maven:3.5:8060020211203142015:c0229ad2.x86_64 from CentOS-AppStream 2024-04-12T20:40:41.953 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 15: conflicting requests 2024-04-12T20:40:41.953 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module mercurial:4.8:8000020190628020724:4148dfdf.x86_64 from CentOS-AppStream 2024-04-12T20:40:41.953 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 16: conflicting requests 2024-04-12T20:40:41.953 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module mod_auth_openidc:2.3:820230721123832:9edba152.x86_64 from CentOS-AppStream 2024-04-12T20:40:41.953 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 17: conflicting requests 2024-04-12T20:40:41.953 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module mysql:8.0:8040020210901180257:9f9e2e7e.x86_64 from CentOS-AppStream 2024-04-12T20:40:41.953 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 18: conflicting requests 2024-04-12T20:40:41.953 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module nginx:1.14:8000020211221191913:55190bc5.x86_64 from CentOS-AppStream 2024-04-12T20:40:41.954 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 19: conflicting requests 2024-04-12T20:40:41.954 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module nodejs:10:8040020210120182536:9f9e2e7e.x86_64 from CentOS-AppStream 2024-04-12T20:40:41.954 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 20: conflicting requests 2024-04-12T20:40:41.954 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl:5.26:8000020190628020724:55190bc5.x86_64 from CentOS-AppStream 2024-04-12T20:40:41.954 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 21: conflicting requests 2024-04-12T20:40:41.954 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module php:7.2:8020020200507003613:2c7ca891.x86_64 from CentOS-AppStream 2024-04-12T20:40:41.954 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 22: conflicting requests 2024-04-12T20:40:41.954 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module postgresql:10:820230223114800:9edba152.x86_64 from CentOS-AppStream 2024-04-12T20:40:41.954 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 23: conflicting requests 2024-04-12T20:40:41.954 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module python27:2.7:820240208011952:182f7c73.x86_64 from CentOS-AppStream 2024-04-12T20:40:41.954 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 24: conflicting requests 2024-04-12T20:40:41.954 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module python36:3.6:8050020210825152031:982725ab.x86_64 from CentOS-AppStream 2024-04-12T20:40:41.955 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 25: conflicting requests 2024-04-12T20:40:41.955 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module python38:3.8:820230810143931:64c1cd5a.x86_64 from CentOS-AppStream 2024-04-12T20:40:41.955 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 26: conflicting requests 2024-04-12T20:40:41.955 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module python39:3.9:820240214182535:17377f89.x86_64 from CentOS-AppStream 2024-04-12T20:40:41.955 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 27: conflicting requests 2024-04-12T20:40:41.955 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module redis:5:8040020211019153849:9f9e2e7e.x86_64 from CentOS-AppStream 2024-04-12T20:40:41.955 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 28: conflicting requests 2024-04-12T20:40:41.955 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module rhn-tools:1.0:8010020191114034948:f69d1239.x86_64 from CentOS-AppStream 2024-04-12T20:40:41.955 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 29: conflicting requests 2024-04-12T20:40:41.955 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module ruby:2.5:820230627084142:9edba152.x86_64 from CentOS-AppStream 2024-04-12T20:40:41.955 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 30: conflicting requests 2024-04-12T20:40:41.955 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module rust-toolset:rhel8:820240119204620:b09eea91.x86_64 from CentOS-AppStream 2024-04-12T20:40:41.955 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 31: conflicting requests 2024-04-12T20:40:41.955 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module satellite-5-client:1.0:8010020191114035551:cdc1202b.x86_64 from CentOS-AppStream 2024-04-12T20:40:41.956 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 32: conflicting requests 2024-04-12T20:40:41.956 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module squid:4:820240319173245:9edba152.x86_64 from CentOS-AppStream 2024-04-12T20:40:41.956 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 33: conflicting requests 2024-04-12T20:40:41.956 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module subversion:1.10:8070020220711155714:78111232.x86_64 from CentOS-AppStream 2024-04-12T20:40:41.956 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 34: conflicting requests 2024-04-12T20:40:41.956 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module swig:3.0:8030020200706142531:30b713e6.x86_64 from CentOS-AppStream 2024-04-12T20:40:41.956 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 35: conflicting requests 2024-04-12T20:40:41.956 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module varnish:6:8050020211109225449:b4937e53.x86_64 from CentOS-AppStream 2024-04-12T20:40:41.956 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 36: conflicting requests 2024-04-12T20:40:41.956 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module virt:rhel:820240314161907:9edba152.x86_64 from CentOS-AppStream 2024-04-12T20:40:41.956 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 37: conflicting requests 2024-04-12T20:40:41.957 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:8030020200716155257:7cc0a66d.x86_64 from CentOS-AppStream 2024-04-12T20:40:41.957 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:8030020200716155257:b967a9a2.x86_64 from CentOS-AppStream 2024-04-12T20:40:41.957 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:820230811133638:36f9fae6.x86_64 from CentOS-AppStream 2024-04-12T20:40:41.957 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 38: cannot install the best candidate for the job 2024-04-12T20:40:41.957 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:8010020191114031501:a5949e2e.x86_64 from CentOS-AppStream 2024-04-12T20:40:41.957 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:8010020191114031501:be2e4737.x86_64 from CentOS-AppStream 2024-04-12T20:40:41.957 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:820230223114759:2ab761e1.x86_64 from CentOS-AppStream 2024-04-12T20:40:41.957 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:820230223114759:86e45846.x86_64 from CentOS-AppStream 2024-04-12T20:40:41.957 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 39: conflicting requests 2024-04-12T20:40:41.957 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:8030020200715230104:1e4bbb35.x86_64 from CentOS-AppStream 2024-04-12T20:40:41.957 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:8030020200715230104:ea09926d.x86_64 from CentOS-AppStream 2024-04-12T20:40:41.957 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:082fdf2f.x86_64 from CentOS-AppStream 2024-04-12T20:40:41.957 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-04-12T20:40:41.957 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-04-12T20:40:41.958 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:fbe42456.x86_64 from CentOS-AppStream 2024-04-12T20:40:41.958 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 40: cannot install the best candidate for the job 2024-04-12T20:40:41.958 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:8010020191114031513:16b3ab4d.x86_64 from CentOS-AppStream 2024-04-12T20:40:41.958 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:8010020191114031513:e1f37755.x86_64 from CentOS-AppStream 2024-04-12T20:40:41.958 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-04-12T20:40:41.958 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-04-12T20:40:41.958 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 41: cannot install the best candidate for the job 2024-04-12T20:40:41.958 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:8030020200716150652:1e4bbb35.x86_64 from CentOS-AppStream 2024-04-12T20:40:41.958 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:8030020200716150652:ea09926d.x86_64 from CentOS-AppStream 2024-04-12T20:40:41.958 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-04-12T20:40:41.958 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-04-12T20:40:41.958 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 42: cannot install the best candidate for the job 2024-04-12T20:40:41.958 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:8030020200716174729:3a70019f.x86_64 from CentOS-AppStream 2024-04-12T20:40:41.959 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:8030020200716174729:cccafca5.x86_64 from CentOS-AppStream 2024-04-12T20:40:41.959 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:820230223114759:6fdf7e5d.x86_64 from CentOS-AppStream 2024-04-12T20:40:41.959 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:820230223114759:d4326aba.x86_64 from CentOS-AppStream 2024-04-12T20:40:41.959 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 43: cannot install the best candidate for the job 2024-04-12T20:40:41.959 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:8030020200716213634:56fce90f.x86_64 from CentOS-AppStream 2024-04-12T20:40:41.959 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:8030020200716213634:98a3c9d0.x86_64 from CentOS-AppStream 2024-04-12T20:40:41.959 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:820230223114759:06b7596e.x86_64 from CentOS-AppStream 2024-04-12T20:40:41.959 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:820230223114759:d2fc5d76.x86_64 from CentOS-AppStream 2024-04-12T20:40:41.959 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 44: cannot install the best candidate for the job 2024-04-12T20:40:41.959 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:8030020200716171139:3a70019f.x86_64 from CentOS-AppStream 2024-04-12T20:40:41.959 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:8030020200716171139:cccafca5.x86_64 from CentOS-AppStream 2024-04-12T20:40:41.959 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:820230223114759:6fdf7e5d.x86_64 from CentOS-AppStream 2024-04-12T20:40:41.959 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:820230223114759:d4326aba.x86_64 from CentOS-AppStream 2024-04-12T20:40:41.959 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 45: cannot install the best candidate for the job 2024-04-12T20:40:41.960 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:8010020191120175858:95fce66d.x86_64 from CentOS-AppStream 2024-04-12T20:40:41.960 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:8030020200723233049:09acf126.x86_64 from CentOS-AppStream 2024-04-12T20:40:41.960 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:820230223114759:436af1ae.x86_64 from CentOS-AppStream 2024-04-12T20:40:41.960 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:820230223114759:b12195ea.x86_64 from CentOS-AppStream 2024-04-12T20:40:41.960 INFO:teuthology.orchestra.run.smithi018.stdout:Package ceph-base-2:17.2.7-827.gd3b5060a.el9.x86_64 is already installed. 2024-04-12T20:40:42.027 INFO:teuthology.orchestra.run.smithi018.stdout:Dependencies resolved. 2024-04-12T20:40:42.029 INFO:teuthology.orchestra.run.smithi018.stdout:Nothing to do. 2024-04-12T20:40:42.029 INFO:teuthology.orchestra.run.smithi018.stdout:Complete! 2024-04-12T20:40:42.049 INFO:teuthology.orchestra.run.smithi186.stdout:Last metadata expiration check: 0:01:05 ago on Fri 12 Apr 2024 08:39:37 PM UTC. 2024-04-12T20:40:42.102 DEBUG:teuthology.orchestra.run.smithi018:> sudo yum -y install cephadm 2024-04-12T20:40:42.351 INFO:teuthology.orchestra.run.smithi186.stderr:Modular dependency problems: 2024-04-12T20:40:42.351 INFO:teuthology.orchestra.run.smithi186.stderr: 2024-04-12T20:40:42.351 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 1: conflicting requests 2024-04-12T20:40:42.351 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module container-tools:rhel8:820240321091303:20125149.x86_64 from CentOS-AppStream 2024-04-12T20:40:42.351 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 2: conflicting requests 2024-04-12T20:40:42.351 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module eclipse:rhel8:820201023100746:b230ed4e.x86_64 from CentOS-AppStream 2024-04-12T20:40:42.351 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 3: conflicting requests 2024-04-12T20:40:42.351 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module freeradius:3.0:8080020230117180605:89170a74.x86_64 from CentOS-AppStream 2024-04-12T20:40:42.351 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 4: conflicting requests 2024-04-12T20:40:42.351 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module gimp:2.8:8000020190628145146:4148dfdf.x86_64 from CentOS-AppStream 2024-04-12T20:40:42.352 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 5: conflicting requests 2024-04-12T20:40:42.352 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module go-toolset:rhel8:820240401160413:b754926a.x86_64 from CentOS-AppStream 2024-04-12T20:40:42.352 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 6: conflicting requests 2024-04-12T20:40:42.352 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module httpd:2.4:820240216084734:9edba152.x86_64 from CentOS-AppStream 2024-04-12T20:40:42.352 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 7: conflicting requests 2024-04-12T20:40:42.352 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module idm:client:820240307184541:49cc9d1b.x86_64 from CentOS-AppStream 2024-04-12T20:40:42.352 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 8: conflicting requests 2024-04-12T20:40:42.352 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module inkscape:0.92.3:8080020221205124429:3e031279.x86_64 from CentOS-AppStream 2024-04-12T20:40:42.352 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 9: conflicting requests 2024-04-12T20:40:42.352 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module jmc:rhel8:8050020211110222101:6392b1f8.x86_64 from CentOS-AppStream 2024-04-12T20:40:42.352 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 10: conflicting requests 2024-04-12T20:40:42.352 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module llvm-toolset:rhel8:820240205133135:9edba152.x86_64 from CentOS-AppStream 2024-04-12T20:40:42.352 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 11: conflicting requests 2024-04-12T20:40:42.352 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module log4j:2:8080020221205124743:9d367344.x86_64 from CentOS-AppStream 2024-04-12T20:40:42.353 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 12: conflicting requests 2024-04-12T20:40:42.353 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module mailman:2.1:820230727085757:77fc8825.x86_64 from CentOS-AppStream 2024-04-12T20:40:42.353 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 13: conflicting requests 2024-04-12T20:40:42.353 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module mariadb:10.3:8030020210419150013:30b713e6.x86_64 from CentOS-AppStream 2024-04-12T20:40:42.353 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 14: conflicting requests 2024-04-12T20:40:42.353 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module maven:3.5:8060020211203142015:c0229ad2.x86_64 from CentOS-AppStream 2024-04-12T20:40:42.353 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 15: conflicting requests 2024-04-12T20:40:42.353 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module mercurial:4.8:8000020190628020724:4148dfdf.x86_64 from CentOS-AppStream 2024-04-12T20:40:42.353 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 16: conflicting requests 2024-04-12T20:40:42.353 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module mod_auth_openidc:2.3:820230721123832:9edba152.x86_64 from CentOS-AppStream 2024-04-12T20:40:42.353 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 17: conflicting requests 2024-04-12T20:40:42.353 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module mysql:8.0:8040020210901180257:9f9e2e7e.x86_64 from CentOS-AppStream 2024-04-12T20:40:42.353 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 18: conflicting requests 2024-04-12T20:40:42.354 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module nginx:1.14:8000020211221191913:55190bc5.x86_64 from CentOS-AppStream 2024-04-12T20:40:42.354 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 19: conflicting requests 2024-04-12T20:40:42.354 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module nodejs:10:8040020210120182536:9f9e2e7e.x86_64 from CentOS-AppStream 2024-04-12T20:40:42.354 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 20: conflicting requests 2024-04-12T20:40:42.354 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl:5.26:8000020190628020724:55190bc5.x86_64 from CentOS-AppStream 2024-04-12T20:40:42.354 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 21: conflicting requests 2024-04-12T20:40:42.354 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module php:7.2:8020020200507003613:2c7ca891.x86_64 from CentOS-AppStream 2024-04-12T20:40:42.354 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 22: conflicting requests 2024-04-12T20:40:42.354 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module postgresql:10:820230223114800:9edba152.x86_64 from CentOS-AppStream 2024-04-12T20:40:42.354 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 23: conflicting requests 2024-04-12T20:40:42.354 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module python27:2.7:820240208011952:182f7c73.x86_64 from CentOS-AppStream 2024-04-12T20:40:42.355 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 24: conflicting requests 2024-04-12T20:40:42.355 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module python36:3.6:8050020210825152031:982725ab.x86_64 from CentOS-AppStream 2024-04-12T20:40:42.355 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 25: conflicting requests 2024-04-12T20:40:42.355 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module python38:3.8:820230810143931:64c1cd5a.x86_64 from CentOS-AppStream 2024-04-12T20:40:42.355 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 26: conflicting requests 2024-04-12T20:40:42.355 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module python39:3.9:820240214182535:17377f89.x86_64 from CentOS-AppStream 2024-04-12T20:40:42.355 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 27: conflicting requests 2024-04-12T20:40:42.355 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module redis:5:8040020211019153849:9f9e2e7e.x86_64 from CentOS-AppStream 2024-04-12T20:40:42.355 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 28: conflicting requests 2024-04-12T20:40:42.355 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module rhn-tools:1.0:8010020191114034948:f69d1239.x86_64 from CentOS-AppStream 2024-04-12T20:40:42.355 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 29: conflicting requests 2024-04-12T20:40:42.356 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module ruby:2.5:820230627084142:9edba152.x86_64 from CentOS-AppStream 2024-04-12T20:40:42.356 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 30: conflicting requests 2024-04-12T20:40:42.356 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module rust-toolset:rhel8:820240119204620:b09eea91.x86_64 from CentOS-AppStream 2024-04-12T20:40:42.356 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 31: conflicting requests 2024-04-12T20:40:42.356 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module satellite-5-client:1.0:8010020191114035551:cdc1202b.x86_64 from CentOS-AppStream 2024-04-12T20:40:42.356 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 32: conflicting requests 2024-04-12T20:40:42.356 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module squid:4:820240319173245:9edba152.x86_64 from CentOS-AppStream 2024-04-12T20:40:42.356 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 33: conflicting requests 2024-04-12T20:40:42.356 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module subversion:1.10:8070020220711155714:78111232.x86_64 from CentOS-AppStream 2024-04-12T20:40:42.356 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 34: conflicting requests 2024-04-12T20:40:42.356 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module swig:3.0:8030020200706142531:30b713e6.x86_64 from CentOS-AppStream 2024-04-12T20:40:42.356 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 35: conflicting requests 2024-04-12T20:40:42.356 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module varnish:6:8050020211109225449:b4937e53.x86_64 from CentOS-AppStream 2024-04-12T20:40:42.357 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 36: conflicting requests 2024-04-12T20:40:42.357 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module virt:rhel:820240314161907:9edba152.x86_64 from CentOS-AppStream 2024-04-12T20:40:42.357 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 37: conflicting requests 2024-04-12T20:40:42.357 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:8030020200716155257:7cc0a66d.x86_64 from CentOS-AppStream 2024-04-12T20:40:42.357 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:8030020200716155257:b967a9a2.x86_64 from CentOS-AppStream 2024-04-12T20:40:42.357 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:820230811133638:36f9fae6.x86_64 from CentOS-AppStream 2024-04-12T20:40:42.357 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 38: cannot install the best candidate for the job 2024-04-12T20:40:42.357 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:8010020191114031501:a5949e2e.x86_64 from CentOS-AppStream 2024-04-12T20:40:42.357 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:8010020191114031501:be2e4737.x86_64 from CentOS-AppStream 2024-04-12T20:40:42.357 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:820230223114759:2ab761e1.x86_64 from CentOS-AppStream 2024-04-12T20:40:42.357 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:820230223114759:86e45846.x86_64 from CentOS-AppStream 2024-04-12T20:40:42.357 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 39: conflicting requests 2024-04-12T20:40:42.357 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:8030020200715230104:1e4bbb35.x86_64 from CentOS-AppStream 2024-04-12T20:40:42.357 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:8030020200715230104:ea09926d.x86_64 from CentOS-AppStream 2024-04-12T20:40:42.358 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:082fdf2f.x86_64 from CentOS-AppStream 2024-04-12T20:40:42.358 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-04-12T20:40:42.358 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-04-12T20:40:42.358 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:fbe42456.x86_64 from CentOS-AppStream 2024-04-12T20:40:42.358 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 40: cannot install the best candidate for the job 2024-04-12T20:40:42.358 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:8010020191114031513:16b3ab4d.x86_64 from CentOS-AppStream 2024-04-12T20:40:42.358 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:8010020191114031513:e1f37755.x86_64 from CentOS-AppStream 2024-04-12T20:40:42.358 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-04-12T20:40:42.358 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-04-12T20:40:42.358 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 41: cannot install the best candidate for the job 2024-04-12T20:40:42.358 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:8030020200716150652:1e4bbb35.x86_64 from CentOS-AppStream 2024-04-12T20:40:42.358 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:8030020200716150652:ea09926d.x86_64 from CentOS-AppStream 2024-04-12T20:40:42.358 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-04-12T20:40:42.358 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-04-12T20:40:42.359 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 42: cannot install the best candidate for the job 2024-04-12T20:40:42.359 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:8030020200716174729:3a70019f.x86_64 from CentOS-AppStream 2024-04-12T20:40:42.359 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:8030020200716174729:cccafca5.x86_64 from CentOS-AppStream 2024-04-12T20:40:42.359 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:820230223114759:6fdf7e5d.x86_64 from CentOS-AppStream 2024-04-12T20:40:42.359 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:820230223114759:d4326aba.x86_64 from CentOS-AppStream 2024-04-12T20:40:42.359 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 43: cannot install the best candidate for the job 2024-04-12T20:40:42.359 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:8030020200716213634:56fce90f.x86_64 from CentOS-AppStream 2024-04-12T20:40:42.359 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:8030020200716213634:98a3c9d0.x86_64 from CentOS-AppStream 2024-04-12T20:40:42.359 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:820230223114759:06b7596e.x86_64 from CentOS-AppStream 2024-04-12T20:40:42.359 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:820230223114759:d2fc5d76.x86_64 from CentOS-AppStream 2024-04-12T20:40:42.359 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 44: cannot install the best candidate for the job 2024-04-12T20:40:42.359 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:8030020200716171139:3a70019f.x86_64 from CentOS-AppStream 2024-04-12T20:40:42.359 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:8030020200716171139:cccafca5.x86_64 from CentOS-AppStream 2024-04-12T20:40:42.359 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:820230223114759:6fdf7e5d.x86_64 from CentOS-AppStream 2024-04-12T20:40:42.360 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:820230223114759:d4326aba.x86_64 from CentOS-AppStream 2024-04-12T20:40:42.360 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 45: cannot install the best candidate for the job 2024-04-12T20:40:42.360 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:8010020191120175858:95fce66d.x86_64 from CentOS-AppStream 2024-04-12T20:40:42.360 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:8030020200723233049:09acf126.x86_64 from CentOS-AppStream 2024-04-12T20:40:42.360 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:820230223114759:436af1ae.x86_64 from CentOS-AppStream 2024-04-12T20:40:42.360 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:820230223114759:b12195ea.x86_64 from CentOS-AppStream 2024-04-12T20:40:42.360 INFO:teuthology.orchestra.run.smithi186.stdout:Package ceph-base-2:17.2.7-827.gd3b5060a.el9.x86_64 is already installed. 2024-04-12T20:40:42.422 INFO:teuthology.orchestra.run.smithi186.stdout:Dependencies resolved. 2024-04-12T20:40:42.424 INFO:teuthology.orchestra.run.smithi186.stdout:Nothing to do. 2024-04-12T20:40:42.424 INFO:teuthology.orchestra.run.smithi186.stdout:Complete! 2024-04-12T20:40:42.493 DEBUG:teuthology.orchestra.run.smithi186:> sudo yum -y install cephadm 2024-04-12T20:40:43.017 INFO:teuthology.orchestra.run.smithi018.stdout:Last metadata expiration check: 0:01:08 ago on Fri 12 Apr 2024 08:39:35 PM UTC. 2024-04-12T20:40:43.333 INFO:teuthology.orchestra.run.smithi018.stderr:Modular dependency problems: 2024-04-12T20:40:43.333 INFO:teuthology.orchestra.run.smithi018.stderr: 2024-04-12T20:40:43.333 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 1: conflicting requests 2024-04-12T20:40:43.333 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module container-tools:rhel8:820240321091303:20125149.x86_64 from CentOS-AppStream 2024-04-12T20:40:43.333 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 2: conflicting requests 2024-04-12T20:40:43.333 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module eclipse:rhel8:820201023100746:b230ed4e.x86_64 from CentOS-AppStream 2024-04-12T20:40:43.334 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 3: conflicting requests 2024-04-12T20:40:43.334 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module freeradius:3.0:8080020230117180605:89170a74.x86_64 from CentOS-AppStream 2024-04-12T20:40:43.334 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 4: conflicting requests 2024-04-12T20:40:43.334 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module gimp:2.8:8000020190628145146:4148dfdf.x86_64 from CentOS-AppStream 2024-04-12T20:40:43.334 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 5: conflicting requests 2024-04-12T20:40:43.334 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module go-toolset:rhel8:820240401160413:b754926a.x86_64 from CentOS-AppStream 2024-04-12T20:40:43.334 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 6: conflicting requests 2024-04-12T20:40:43.334 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module httpd:2.4:820240216084734:9edba152.x86_64 from CentOS-AppStream 2024-04-12T20:40:43.334 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 7: conflicting requests 2024-04-12T20:40:43.334 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module idm:client:820240307184541:49cc9d1b.x86_64 from CentOS-AppStream 2024-04-12T20:40:43.334 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 8: conflicting requests 2024-04-12T20:40:43.334 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module inkscape:0.92.3:8080020221205124429:3e031279.x86_64 from CentOS-AppStream 2024-04-12T20:40:43.334 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 9: conflicting requests 2024-04-12T20:40:43.335 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module jmc:rhel8:8050020211110222101:6392b1f8.x86_64 from CentOS-AppStream 2024-04-12T20:40:43.335 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 10: conflicting requests 2024-04-12T20:40:43.335 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module llvm-toolset:rhel8:820240205133135:9edba152.x86_64 from CentOS-AppStream 2024-04-12T20:40:43.335 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 11: conflicting requests 2024-04-12T20:40:43.335 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module log4j:2:8080020221205124743:9d367344.x86_64 from CentOS-AppStream 2024-04-12T20:40:43.335 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 12: conflicting requests 2024-04-12T20:40:43.335 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module mailman:2.1:820230727085757:77fc8825.x86_64 from CentOS-AppStream 2024-04-12T20:40:43.335 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 13: conflicting requests 2024-04-12T20:40:43.335 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module mariadb:10.3:8030020210419150013:30b713e6.x86_64 from CentOS-AppStream 2024-04-12T20:40:43.335 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 14: conflicting requests 2024-04-12T20:40:43.335 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module maven:3.5:8060020211203142015:c0229ad2.x86_64 from CentOS-AppStream 2024-04-12T20:40:43.335 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 15: conflicting requests 2024-04-12T20:40:43.335 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module mercurial:4.8:8000020190628020724:4148dfdf.x86_64 from CentOS-AppStream 2024-04-12T20:40:43.336 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 16: conflicting requests 2024-04-12T20:40:43.336 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module mod_auth_openidc:2.3:820230721123832:9edba152.x86_64 from CentOS-AppStream 2024-04-12T20:40:43.336 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 17: conflicting requests 2024-04-12T20:40:43.336 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module mysql:8.0:8040020210901180257:9f9e2e7e.x86_64 from CentOS-AppStream 2024-04-12T20:40:43.336 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 18: conflicting requests 2024-04-12T20:40:43.336 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module nginx:1.14:8000020211221191913:55190bc5.x86_64 from CentOS-AppStream 2024-04-12T20:40:43.336 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 19: conflicting requests 2024-04-12T20:40:43.336 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module nodejs:10:8040020210120182536:9f9e2e7e.x86_64 from CentOS-AppStream 2024-04-12T20:40:43.336 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 20: conflicting requests 2024-04-12T20:40:43.336 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl:5.26:8000020190628020724:55190bc5.x86_64 from CentOS-AppStream 2024-04-12T20:40:43.336 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 21: conflicting requests 2024-04-12T20:40:43.336 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module php:7.2:8020020200507003613:2c7ca891.x86_64 from CentOS-AppStream 2024-04-12T20:40:43.336 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 22: conflicting requests 2024-04-12T20:40:43.337 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module postgresql:10:820230223114800:9edba152.x86_64 from CentOS-AppStream 2024-04-12T20:40:43.337 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 23: conflicting requests 2024-04-12T20:40:43.337 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module python27:2.7:820240208011952:182f7c73.x86_64 from CentOS-AppStream 2024-04-12T20:40:43.337 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 24: conflicting requests 2024-04-12T20:40:43.337 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module python36:3.6:8050020210825152031:982725ab.x86_64 from CentOS-AppStream 2024-04-12T20:40:43.337 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 25: conflicting requests 2024-04-12T20:40:43.337 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module python38:3.8:820230810143931:64c1cd5a.x86_64 from CentOS-AppStream 2024-04-12T20:40:43.337 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 26: conflicting requests 2024-04-12T20:40:43.337 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module python39:3.9:820240214182535:17377f89.x86_64 from CentOS-AppStream 2024-04-12T20:40:43.337 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 27: conflicting requests 2024-04-12T20:40:43.337 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module redis:5:8040020211019153849:9f9e2e7e.x86_64 from CentOS-AppStream 2024-04-12T20:40:43.338 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 28: conflicting requests 2024-04-12T20:40:43.338 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module rhn-tools:1.0:8010020191114034948:f69d1239.x86_64 from CentOS-AppStream 2024-04-12T20:40:43.338 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 29: conflicting requests 2024-04-12T20:40:43.338 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module ruby:2.5:820230627084142:9edba152.x86_64 from CentOS-AppStream 2024-04-12T20:40:43.338 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 30: conflicting requests 2024-04-12T20:40:43.338 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module rust-toolset:rhel8:820240119204620:b09eea91.x86_64 from CentOS-AppStream 2024-04-12T20:40:43.338 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 31: conflicting requests 2024-04-12T20:40:43.338 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module satellite-5-client:1.0:8010020191114035551:cdc1202b.x86_64 from CentOS-AppStream 2024-04-12T20:40:43.338 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 32: conflicting requests 2024-04-12T20:40:43.338 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module squid:4:820240319173245:9edba152.x86_64 from CentOS-AppStream 2024-04-12T20:40:43.338 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 33: conflicting requests 2024-04-12T20:40:43.338 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module subversion:1.10:8070020220711155714:78111232.x86_64 from CentOS-AppStream 2024-04-12T20:40:43.339 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 34: conflicting requests 2024-04-12T20:40:43.339 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module swig:3.0:8030020200706142531:30b713e6.x86_64 from CentOS-AppStream 2024-04-12T20:40:43.339 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 35: conflicting requests 2024-04-12T20:40:43.339 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module varnish:6:8050020211109225449:b4937e53.x86_64 from CentOS-AppStream 2024-04-12T20:40:43.339 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 36: conflicting requests 2024-04-12T20:40:43.339 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module virt:rhel:820240314161907:9edba152.x86_64 from CentOS-AppStream 2024-04-12T20:40:43.339 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 37: conflicting requests 2024-04-12T20:40:43.339 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:8030020200716155257:7cc0a66d.x86_64 from CentOS-AppStream 2024-04-12T20:40:43.339 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:8030020200716155257:b967a9a2.x86_64 from CentOS-AppStream 2024-04-12T20:40:43.339 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:820230811133638:36f9fae6.x86_64 from CentOS-AppStream 2024-04-12T20:40:43.339 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 38: cannot install the best candidate for the job 2024-04-12T20:40:43.339 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:8010020191114031501:a5949e2e.x86_64 from CentOS-AppStream 2024-04-12T20:40:43.339 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:8010020191114031501:be2e4737.x86_64 from CentOS-AppStream 2024-04-12T20:40:43.339 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:820230223114759:2ab761e1.x86_64 from CentOS-AppStream 2024-04-12T20:40:43.340 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:820230223114759:86e45846.x86_64 from CentOS-AppStream 2024-04-12T20:40:43.340 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 39: conflicting requests 2024-04-12T20:40:43.340 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:8030020200715230104:1e4bbb35.x86_64 from CentOS-AppStream 2024-04-12T20:40:43.340 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:8030020200715230104:ea09926d.x86_64 from CentOS-AppStream 2024-04-12T20:40:43.340 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:082fdf2f.x86_64 from CentOS-AppStream 2024-04-12T20:40:43.340 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-04-12T20:40:43.340 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-04-12T20:40:43.340 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:fbe42456.x86_64 from CentOS-AppStream 2024-04-12T20:40:43.340 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 40: cannot install the best candidate for the job 2024-04-12T20:40:43.340 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:8010020191114031513:16b3ab4d.x86_64 from CentOS-AppStream 2024-04-12T20:40:43.341 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:8010020191114031513:e1f37755.x86_64 from CentOS-AppStream 2024-04-12T20:40:43.341 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-04-12T20:40:43.341 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-04-12T20:40:43.341 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 41: cannot install the best candidate for the job 2024-04-12T20:40:43.341 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:8030020200716150652:1e4bbb35.x86_64 from CentOS-AppStream 2024-04-12T20:40:43.341 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:8030020200716150652:ea09926d.x86_64 from CentOS-AppStream 2024-04-12T20:40:43.341 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-04-12T20:40:43.341 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-04-12T20:40:43.341 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 42: cannot install the best candidate for the job 2024-04-12T20:40:43.341 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:8030020200716174729:3a70019f.x86_64 from CentOS-AppStream 2024-04-12T20:40:43.341 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:8030020200716174729:cccafca5.x86_64 from CentOS-AppStream 2024-04-12T20:40:43.341 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:820230223114759:6fdf7e5d.x86_64 from CentOS-AppStream 2024-04-12T20:40:43.342 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:820230223114759:d4326aba.x86_64 from CentOS-AppStream 2024-04-12T20:40:43.342 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 43: cannot install the best candidate for the job 2024-04-12T20:40:43.342 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:8030020200716213634:56fce90f.x86_64 from CentOS-AppStream 2024-04-12T20:40:43.342 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:8030020200716213634:98a3c9d0.x86_64 from CentOS-AppStream 2024-04-12T20:40:43.342 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:820230223114759:06b7596e.x86_64 from CentOS-AppStream 2024-04-12T20:40:43.342 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:820230223114759:d2fc5d76.x86_64 from CentOS-AppStream 2024-04-12T20:40:43.342 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 44: cannot install the best candidate for the job 2024-04-12T20:40:43.342 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:8030020200716171139:3a70019f.x86_64 from CentOS-AppStream 2024-04-12T20:40:43.342 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:8030020200716171139:cccafca5.x86_64 from CentOS-AppStream 2024-04-12T20:40:43.342 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:820230223114759:6fdf7e5d.x86_64 from CentOS-AppStream 2024-04-12T20:40:43.342 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:820230223114759:d4326aba.x86_64 from CentOS-AppStream 2024-04-12T20:40:43.343 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 45: cannot install the best candidate for the job 2024-04-12T20:40:43.343 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:8010020191120175858:95fce66d.x86_64 from CentOS-AppStream 2024-04-12T20:40:43.343 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:8030020200723233049:09acf126.x86_64 from CentOS-AppStream 2024-04-12T20:40:43.343 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:820230223114759:436af1ae.x86_64 from CentOS-AppStream 2024-04-12T20:40:43.343 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:820230223114759:b12195ea.x86_64 from CentOS-AppStream 2024-04-12T20:40:43.365 INFO:teuthology.orchestra.run.smithi186.stdout:Last metadata expiration check: 0:01:06 ago on Fri 12 Apr 2024 08:39:37 PM UTC. 2024-04-12T20:40:43.409 INFO:teuthology.orchestra.run.smithi018.stdout:Dependencies resolved. 2024-04-12T20:40:43.409 INFO:teuthology.orchestra.run.smithi018.stdout:================================================================================ 2024-04-12T20:40:43.409 INFO:teuthology.orchestra.run.smithi018.stdout: Package Arch Version Repository Size 2024-04-12T20:40:43.409 INFO:teuthology.orchestra.run.smithi018.stdout:================================================================================ 2024-04-12T20:40:43.410 INFO:teuthology.orchestra.run.smithi018.stdout:Installing: 2024-04-12T20:40:43.410 INFO:teuthology.orchestra.run.smithi018.stdout: cephadm noarch 2:17.2.7-827.gd3b5060a.el9 ceph-noarch 82 k 2024-04-12T20:40:43.410 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-12T20:40:43.410 INFO:teuthology.orchestra.run.smithi018.stdout:Transaction Summary 2024-04-12T20:40:43.410 INFO:teuthology.orchestra.run.smithi018.stdout:================================================================================ 2024-04-12T20:40:43.410 INFO:teuthology.orchestra.run.smithi018.stdout:Install 1 Package 2024-04-12T20:40:43.410 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-12T20:40:43.411 INFO:teuthology.orchestra.run.smithi018.stdout:Total download size: 82 k 2024-04-12T20:40:43.411 INFO:teuthology.orchestra.run.smithi018.stdout:Installed size: 372 k 2024-04-12T20:40:43.411 INFO:teuthology.orchestra.run.smithi018.stdout:Downloading Packages: 2024-04-12T20:40:43.669 INFO:teuthology.orchestra.run.smithi186.stderr:Modular dependency problems: 2024-04-12T20:40:43.669 INFO:teuthology.orchestra.run.smithi186.stderr: 2024-04-12T20:40:43.669 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 1: conflicting requests 2024-04-12T20:40:43.669 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module container-tools:rhel8:820240321091303:20125149.x86_64 from CentOS-AppStream 2024-04-12T20:40:43.669 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 2: conflicting requests 2024-04-12T20:40:43.669 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module eclipse:rhel8:820201023100746:b230ed4e.x86_64 from CentOS-AppStream 2024-04-12T20:40:43.669 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 3: conflicting requests 2024-04-12T20:40:43.669 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module freeradius:3.0:8080020230117180605:89170a74.x86_64 from CentOS-AppStream 2024-04-12T20:40:43.669 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 4: conflicting requests 2024-04-12T20:40:43.669 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module gimp:2.8:8000020190628145146:4148dfdf.x86_64 from CentOS-AppStream 2024-04-12T20:40:43.669 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 5: conflicting requests 2024-04-12T20:40:43.670 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module go-toolset:rhel8:820240401160413:b754926a.x86_64 from CentOS-AppStream 2024-04-12T20:40:43.670 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 6: conflicting requests 2024-04-12T20:40:43.670 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module httpd:2.4:820240216084734:9edba152.x86_64 from CentOS-AppStream 2024-04-12T20:40:43.670 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 7: conflicting requests 2024-04-12T20:40:43.670 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module idm:client:820240307184541:49cc9d1b.x86_64 from CentOS-AppStream 2024-04-12T20:40:43.670 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 8: conflicting requests 2024-04-12T20:40:43.670 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module inkscape:0.92.3:8080020221205124429:3e031279.x86_64 from CentOS-AppStream 2024-04-12T20:40:43.670 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 9: conflicting requests 2024-04-12T20:40:43.670 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module jmc:rhel8:8050020211110222101:6392b1f8.x86_64 from CentOS-AppStream 2024-04-12T20:40:43.670 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 10: conflicting requests 2024-04-12T20:40:43.670 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module llvm-toolset:rhel8:820240205133135:9edba152.x86_64 from CentOS-AppStream 2024-04-12T20:40:43.670 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 11: conflicting requests 2024-04-12T20:40:43.670 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module log4j:2:8080020221205124743:9d367344.x86_64 from CentOS-AppStream 2024-04-12T20:40:43.671 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 12: conflicting requests 2024-04-12T20:40:43.671 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module mailman:2.1:820230727085757:77fc8825.x86_64 from CentOS-AppStream 2024-04-12T20:40:43.671 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 13: conflicting requests 2024-04-12T20:40:43.671 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module mariadb:10.3:8030020210419150013:30b713e6.x86_64 from CentOS-AppStream 2024-04-12T20:40:43.671 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 14: conflicting requests 2024-04-12T20:40:43.671 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module maven:3.5:8060020211203142015:c0229ad2.x86_64 from CentOS-AppStream 2024-04-12T20:40:43.671 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 15: conflicting requests 2024-04-12T20:40:43.671 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module mercurial:4.8:8000020190628020724:4148dfdf.x86_64 from CentOS-AppStream 2024-04-12T20:40:43.671 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 16: conflicting requests 2024-04-12T20:40:43.671 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module mod_auth_openidc:2.3:820230721123832:9edba152.x86_64 from CentOS-AppStream 2024-04-12T20:40:43.671 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 17: conflicting requests 2024-04-12T20:40:43.671 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module mysql:8.0:8040020210901180257:9f9e2e7e.x86_64 from CentOS-AppStream 2024-04-12T20:40:43.671 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 18: conflicting requests 2024-04-12T20:40:43.671 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module nginx:1.14:8000020211221191913:55190bc5.x86_64 from CentOS-AppStream 2024-04-12T20:40:43.671 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 19: conflicting requests 2024-04-12T20:40:43.672 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module nodejs:10:8040020210120182536:9f9e2e7e.x86_64 from CentOS-AppStream 2024-04-12T20:40:43.672 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 20: conflicting requests 2024-04-12T20:40:43.672 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl:5.26:8000020190628020724:55190bc5.x86_64 from CentOS-AppStream 2024-04-12T20:40:43.672 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 21: conflicting requests 2024-04-12T20:40:43.672 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module php:7.2:8020020200507003613:2c7ca891.x86_64 from CentOS-AppStream 2024-04-12T20:40:43.672 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 22: conflicting requests 2024-04-12T20:40:43.672 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module postgresql:10:820230223114800:9edba152.x86_64 from CentOS-AppStream 2024-04-12T20:40:43.672 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 23: conflicting requests 2024-04-12T20:40:43.672 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module python27:2.7:820240208011952:182f7c73.x86_64 from CentOS-AppStream 2024-04-12T20:40:43.672 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 24: conflicting requests 2024-04-12T20:40:43.672 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module python36:3.6:8050020210825152031:982725ab.x86_64 from CentOS-AppStream 2024-04-12T20:40:43.672 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 25: conflicting requests 2024-04-12T20:40:43.672 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module python38:3.8:820230810143931:64c1cd5a.x86_64 from CentOS-AppStream 2024-04-12T20:40:43.672 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 26: conflicting requests 2024-04-12T20:40:43.673 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module python39:3.9:820240214182535:17377f89.x86_64 from CentOS-AppStream 2024-04-12T20:40:43.673 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 27: conflicting requests 2024-04-12T20:40:43.673 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module redis:5:8040020211019153849:9f9e2e7e.x86_64 from CentOS-AppStream 2024-04-12T20:40:43.673 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 28: conflicting requests 2024-04-12T20:40:43.673 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module rhn-tools:1.0:8010020191114034948:f69d1239.x86_64 from CentOS-AppStream 2024-04-12T20:40:43.673 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 29: conflicting requests 2024-04-12T20:40:43.673 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module ruby:2.5:820230627084142:9edba152.x86_64 from CentOS-AppStream 2024-04-12T20:40:43.673 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 30: conflicting requests 2024-04-12T20:40:43.673 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module rust-toolset:rhel8:820240119204620:b09eea91.x86_64 from CentOS-AppStream 2024-04-12T20:40:43.673 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 31: conflicting requests 2024-04-12T20:40:43.673 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module satellite-5-client:1.0:8010020191114035551:cdc1202b.x86_64 from CentOS-AppStream 2024-04-12T20:40:43.673 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 32: conflicting requests 2024-04-12T20:40:43.673 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module squid:4:820240319173245:9edba152.x86_64 from CentOS-AppStream 2024-04-12T20:40:43.673 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 33: conflicting requests 2024-04-12T20:40:43.673 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module subversion:1.10:8070020220711155714:78111232.x86_64 from CentOS-AppStream 2024-04-12T20:40:43.674 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 34: conflicting requests 2024-04-12T20:40:43.674 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module swig:3.0:8030020200706142531:30b713e6.x86_64 from CentOS-AppStream 2024-04-12T20:40:43.674 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 35: conflicting requests 2024-04-12T20:40:43.674 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module varnish:6:8050020211109225449:b4937e53.x86_64 from CentOS-AppStream 2024-04-12T20:40:43.674 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 36: conflicting requests 2024-04-12T20:40:43.674 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module virt:rhel:820240314161907:9edba152.x86_64 from CentOS-AppStream 2024-04-12T20:40:43.674 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 37: conflicting requests 2024-04-12T20:40:43.674 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:8030020200716155257:7cc0a66d.x86_64 from CentOS-AppStream 2024-04-12T20:40:43.674 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:8030020200716155257:b967a9a2.x86_64 from CentOS-AppStream 2024-04-12T20:40:43.674 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:820230811133638:36f9fae6.x86_64 from CentOS-AppStream 2024-04-12T20:40:43.674 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 38: cannot install the best candidate for the job 2024-04-12T20:40:43.674 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:8010020191114031501:a5949e2e.x86_64 from CentOS-AppStream 2024-04-12T20:40:43.674 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:8010020191114031501:be2e4737.x86_64 from CentOS-AppStream 2024-04-12T20:40:43.674 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:820230223114759:2ab761e1.x86_64 from CentOS-AppStream 2024-04-12T20:40:43.674 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:820230223114759:86e45846.x86_64 from CentOS-AppStream 2024-04-12T20:40:43.675 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 39: conflicting requests 2024-04-12T20:40:43.675 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:8030020200715230104:1e4bbb35.x86_64 from CentOS-AppStream 2024-04-12T20:40:43.675 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:8030020200715230104:ea09926d.x86_64 from CentOS-AppStream 2024-04-12T20:40:43.675 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:082fdf2f.x86_64 from CentOS-AppStream 2024-04-12T20:40:43.675 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-04-12T20:40:43.675 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-04-12T20:40:43.675 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:fbe42456.x86_64 from CentOS-AppStream 2024-04-12T20:40:43.675 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 40: cannot install the best candidate for the job 2024-04-12T20:40:43.675 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:8010020191114031513:16b3ab4d.x86_64 from CentOS-AppStream 2024-04-12T20:40:43.675 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:8010020191114031513:e1f37755.x86_64 from CentOS-AppStream 2024-04-12T20:40:43.675 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-04-12T20:40:43.675 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-04-12T20:40:43.675 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 41: cannot install the best candidate for the job 2024-04-12T20:40:43.676 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:8030020200716150652:1e4bbb35.x86_64 from CentOS-AppStream 2024-04-12T20:40:43.676 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:8030020200716150652:ea09926d.x86_64 from CentOS-AppStream 2024-04-12T20:40:43.676 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-04-12T20:40:43.676 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-04-12T20:40:43.676 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 42: cannot install the best candidate for the job 2024-04-12T20:40:43.676 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:8030020200716174729:3a70019f.x86_64 from CentOS-AppStream 2024-04-12T20:40:43.676 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:8030020200716174729:cccafca5.x86_64 from CentOS-AppStream 2024-04-12T20:40:43.676 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:820230223114759:6fdf7e5d.x86_64 from CentOS-AppStream 2024-04-12T20:40:43.676 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:820230223114759:d4326aba.x86_64 from CentOS-AppStream 2024-04-12T20:40:43.676 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 43: cannot install the best candidate for the job 2024-04-12T20:40:43.676 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:8030020200716213634:56fce90f.x86_64 from CentOS-AppStream 2024-04-12T20:40:43.677 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:8030020200716213634:98a3c9d0.x86_64 from CentOS-AppStream 2024-04-12T20:40:43.677 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:820230223114759:06b7596e.x86_64 from CentOS-AppStream 2024-04-12T20:40:43.677 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:820230223114759:d2fc5d76.x86_64 from CentOS-AppStream 2024-04-12T20:40:43.677 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 44: cannot install the best candidate for the job 2024-04-12T20:40:43.677 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:8030020200716171139:3a70019f.x86_64 from CentOS-AppStream 2024-04-12T20:40:43.677 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:8030020200716171139:cccafca5.x86_64 from CentOS-AppStream 2024-04-12T20:40:43.677 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:820230223114759:6fdf7e5d.x86_64 from CentOS-AppStream 2024-04-12T20:40:43.677 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:820230223114759:d4326aba.x86_64 from CentOS-AppStream 2024-04-12T20:40:43.677 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 45: cannot install the best candidate for the job 2024-04-12T20:40:43.677 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:8010020191120175858:95fce66d.x86_64 from CentOS-AppStream 2024-04-12T20:40:43.677 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:8030020200723233049:09acf126.x86_64 from CentOS-AppStream 2024-04-12T20:40:43.677 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:820230223114759:436af1ae.x86_64 from CentOS-AppStream 2024-04-12T20:40:43.677 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:820230223114759:b12195ea.x86_64 from CentOS-AppStream 2024-04-12T20:40:43.741 INFO:teuthology.orchestra.run.smithi186.stdout:Dependencies resolved. 2024-04-12T20:40:43.742 INFO:teuthology.orchestra.run.smithi186.stdout:================================================================================ 2024-04-12T20:40:43.742 INFO:teuthology.orchestra.run.smithi186.stdout: Package Arch Version Repository Size 2024-04-12T20:40:43.742 INFO:teuthology.orchestra.run.smithi186.stdout:================================================================================ 2024-04-12T20:40:43.742 INFO:teuthology.orchestra.run.smithi186.stdout:Installing: 2024-04-12T20:40:43.742 INFO:teuthology.orchestra.run.smithi186.stdout: cephadm noarch 2:17.2.7-827.gd3b5060a.el9 ceph-noarch 82 k 2024-04-12T20:40:43.742 INFO:teuthology.orchestra.run.smithi186.stdout: 2024-04-12T20:40:43.742 INFO:teuthology.orchestra.run.smithi186.stdout:Transaction Summary 2024-04-12T20:40:43.742 INFO:teuthology.orchestra.run.smithi186.stdout:================================================================================ 2024-04-12T20:40:43.743 INFO:teuthology.orchestra.run.smithi186.stdout:Install 1 Package 2024-04-12T20:40:43.743 INFO:teuthology.orchestra.run.smithi186.stdout: 2024-04-12T20:40:43.743 INFO:teuthology.orchestra.run.smithi186.stdout:Total download size: 82 k 2024-04-12T20:40:43.743 INFO:teuthology.orchestra.run.smithi186.stdout:Installed size: 372 k 2024-04-12T20:40:43.743 INFO:teuthology.orchestra.run.smithi186.stdout:Downloading Packages: 2024-04-12T20:40:43.754 INFO:teuthology.orchestra.run.smithi018.stdout:cephadm-17.2.7-827.gd3b5060a.el9.noarch.rpm 239 kB/s | 82 kB 00:00 2024-04-12T20:40:43.755 INFO:teuthology.orchestra.run.smithi018.stdout:-------------------------------------------------------------------------------- 2024-04-12T20:40:43.756 INFO:teuthology.orchestra.run.smithi018.stdout:Total 237 kB/s | 82 kB 00:00 2024-04-12T20:40:43.757 INFO:teuthology.orchestra.run.smithi018.stdout:Running transaction check 2024-04-12T20:40:43.775 INFO:teuthology.orchestra.run.smithi018.stdout:Transaction check succeeded. 2024-04-12T20:40:43.775 INFO:teuthology.orchestra.run.smithi018.stdout:Running transaction test 2024-04-12T20:40:43.782 INFO:teuthology.orchestra.run.smithi018.stdout:Transaction test succeeded. 2024-04-12T20:40:43.783 INFO:teuthology.orchestra.run.smithi018.stdout:Running transaction 2024-04-12T20:40:43.948 INFO:teuthology.orchestra.run.smithi018.stdout: Preparing : 1/1 2024-04-12T20:40:44.137 INFO:teuthology.orchestra.run.smithi186.stdout:cephadm-17.2.7-827.gd3b5060a.el9.noarch.rpm 208 kB/s | 82 kB 00:00 2024-04-12T20:40:44.139 INFO:teuthology.orchestra.run.smithi186.stdout:-------------------------------------------------------------------------------- 2024-04-12T20:40:44.139 INFO:teuthology.orchestra.run.smithi186.stdout:Total 207 kB/s | 82 kB 00:00 2024-04-12T20:40:44.140 INFO:teuthology.orchestra.run.smithi186.stdout:Running transaction check 2024-04-12T20:40:44.157 INFO:teuthology.orchestra.run.smithi186.stdout:Transaction check succeeded. 2024-04-12T20:40:44.157 INFO:teuthology.orchestra.run.smithi186.stdout:Running transaction test 2024-04-12T20:40:44.164 INFO:teuthology.orchestra.run.smithi186.stdout:Transaction test succeeded. 2024-04-12T20:40:44.166 INFO:teuthology.orchestra.run.smithi186.stdout:Running transaction 2024-04-12T20:40:44.327 INFO:teuthology.orchestra.run.smithi186.stdout: Preparing : 1/1 2024-04-12T20:40:44.569 INFO:teuthology.orchestra.run.smithi018.stdout: Running scriptlet: cephadm-2:17.2.7-827.gd3b5060a.el9.noarch 1/1 2024-04-12T20:40:44.751 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : cephadm-2:17.2.7-827.gd3b5060a.el9.noarch 1/1 2024-04-12T20:40:45.074 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: cephadm-2:17.2.7-827.gd3b5060a.el9.noarch 1/1 2024-04-12T20:40:45.232 INFO:teuthology.orchestra.run.smithi018.stdout: Running scriptlet: cephadm-2:17.2.7-827.gd3b5060a.el9.noarch 1/1 2024-04-12T20:40:45.265 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : cephadm-2:17.2.7-827.gd3b5060a.el9.noarch 1/1 2024-04-12T20:40:45.580 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : cephadm-2:17.2.7-827.gd3b5060a.el9.noarch 1/1 2024-04-12T20:40:45.581 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-12T20:40:45.581 INFO:teuthology.orchestra.run.smithi018.stdout:Installed: 2024-04-12T20:40:45.581 INFO:teuthology.orchestra.run.smithi018.stdout: cephadm-2:17.2.7-827.gd3b5060a.el9.noarch 2024-04-12T20:40:45.581 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-12T20:40:45.581 INFO:teuthology.orchestra.run.smithi018.stdout:Complete! 2024-04-12T20:40:45.767 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: cephadm-2:17.2.7-827.gd3b5060a.el9.noarch 1/1 2024-04-12T20:40:45.923 DEBUG:teuthology.orchestra.run.smithi018:> sudo yum -y install ceph-immutable-object-cache 2024-04-12T20:40:46.129 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : cephadm-2:17.2.7-827.gd3b5060a.el9.noarch 1/1 2024-04-12T20:40:46.129 INFO:teuthology.orchestra.run.smithi186.stdout: 2024-04-12T20:40:46.129 INFO:teuthology.orchestra.run.smithi186.stdout:Installed: 2024-04-12T20:40:46.129 INFO:teuthology.orchestra.run.smithi186.stdout: cephadm-2:17.2.7-827.gd3b5060a.el9.noarch 2024-04-12T20:40:46.129 INFO:teuthology.orchestra.run.smithi186.stdout: 2024-04-12T20:40:46.129 INFO:teuthology.orchestra.run.smithi186.stdout:Complete! 2024-04-12T20:40:46.434 DEBUG:teuthology.orchestra.run.smithi186:> sudo yum -y install ceph-immutable-object-cache 2024-04-12T20:40:46.842 INFO:teuthology.orchestra.run.smithi018.stdout:Last metadata expiration check: 0:01:11 ago on Fri 12 Apr 2024 08:39:35 PM UTC. 2024-04-12T20:40:47.155 INFO:teuthology.orchestra.run.smithi018.stderr:Modular dependency problems: 2024-04-12T20:40:47.155 INFO:teuthology.orchestra.run.smithi018.stderr: 2024-04-12T20:40:47.155 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 1: conflicting requests 2024-04-12T20:40:47.156 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module container-tools:rhel8:820240321091303:20125149.x86_64 from CentOS-AppStream 2024-04-12T20:40:47.156 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 2: conflicting requests 2024-04-12T20:40:47.156 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module eclipse:rhel8:820201023100746:b230ed4e.x86_64 from CentOS-AppStream 2024-04-12T20:40:47.156 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 3: conflicting requests 2024-04-12T20:40:47.156 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module freeradius:3.0:8080020230117180605:89170a74.x86_64 from CentOS-AppStream 2024-04-12T20:40:47.156 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 4: conflicting requests 2024-04-12T20:40:47.156 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module gimp:2.8:8000020190628145146:4148dfdf.x86_64 from CentOS-AppStream 2024-04-12T20:40:47.156 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 5: conflicting requests 2024-04-12T20:40:47.156 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module go-toolset:rhel8:820240401160413:b754926a.x86_64 from CentOS-AppStream 2024-04-12T20:40:47.156 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 6: conflicting requests 2024-04-12T20:40:47.156 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module httpd:2.4:820240216084734:9edba152.x86_64 from CentOS-AppStream 2024-04-12T20:40:47.156 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 7: conflicting requests 2024-04-12T20:40:47.156 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module idm:client:820240307184541:49cc9d1b.x86_64 from CentOS-AppStream 2024-04-12T20:40:47.157 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 8: conflicting requests 2024-04-12T20:40:47.157 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module inkscape:0.92.3:8080020221205124429:3e031279.x86_64 from CentOS-AppStream 2024-04-12T20:40:47.157 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 9: conflicting requests 2024-04-12T20:40:47.157 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module jmc:rhel8:8050020211110222101:6392b1f8.x86_64 from CentOS-AppStream 2024-04-12T20:40:47.157 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 10: conflicting requests 2024-04-12T20:40:47.157 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module llvm-toolset:rhel8:820240205133135:9edba152.x86_64 from CentOS-AppStream 2024-04-12T20:40:47.157 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 11: conflicting requests 2024-04-12T20:40:47.157 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module log4j:2:8080020221205124743:9d367344.x86_64 from CentOS-AppStream 2024-04-12T20:40:47.157 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 12: conflicting requests 2024-04-12T20:40:47.157 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module mailman:2.1:820230727085757:77fc8825.x86_64 from CentOS-AppStream 2024-04-12T20:40:47.157 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 13: conflicting requests 2024-04-12T20:40:47.157 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module mariadb:10.3:8030020210419150013:30b713e6.x86_64 from CentOS-AppStream 2024-04-12T20:40:47.157 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 14: conflicting requests 2024-04-12T20:40:47.157 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module maven:3.5:8060020211203142015:c0229ad2.x86_64 from CentOS-AppStream 2024-04-12T20:40:47.158 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 15: conflicting requests 2024-04-12T20:40:47.158 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module mercurial:4.8:8000020190628020724:4148dfdf.x86_64 from CentOS-AppStream 2024-04-12T20:40:47.158 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 16: conflicting requests 2024-04-12T20:40:47.158 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module mod_auth_openidc:2.3:820230721123832:9edba152.x86_64 from CentOS-AppStream 2024-04-12T20:40:47.158 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 17: conflicting requests 2024-04-12T20:40:47.158 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module mysql:8.0:8040020210901180257:9f9e2e7e.x86_64 from CentOS-AppStream 2024-04-12T20:40:47.158 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 18: conflicting requests 2024-04-12T20:40:47.158 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module nginx:1.14:8000020211221191913:55190bc5.x86_64 from CentOS-AppStream 2024-04-12T20:40:47.158 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 19: conflicting requests 2024-04-12T20:40:47.158 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module nodejs:10:8040020210120182536:9f9e2e7e.x86_64 from CentOS-AppStream 2024-04-12T20:40:47.158 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 20: conflicting requests 2024-04-12T20:40:47.158 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl:5.26:8000020190628020724:55190bc5.x86_64 from CentOS-AppStream 2024-04-12T20:40:47.159 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 21: conflicting requests 2024-04-12T20:40:47.159 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module php:7.2:8020020200507003613:2c7ca891.x86_64 from CentOS-AppStream 2024-04-12T20:40:47.159 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 22: conflicting requests 2024-04-12T20:40:47.159 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module postgresql:10:820230223114800:9edba152.x86_64 from CentOS-AppStream 2024-04-12T20:40:47.159 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 23: conflicting requests 2024-04-12T20:40:47.159 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module python27:2.7:820240208011952:182f7c73.x86_64 from CentOS-AppStream 2024-04-12T20:40:47.159 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 24: conflicting requests 2024-04-12T20:40:47.159 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module python36:3.6:8050020210825152031:982725ab.x86_64 from CentOS-AppStream 2024-04-12T20:40:47.159 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 25: conflicting requests 2024-04-12T20:40:47.160 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module python38:3.8:820230810143931:64c1cd5a.x86_64 from CentOS-AppStream 2024-04-12T20:40:47.161 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 26: conflicting requests 2024-04-12T20:40:47.161 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module python39:3.9:820240214182535:17377f89.x86_64 from CentOS-AppStream 2024-04-12T20:40:47.161 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 27: conflicting requests 2024-04-12T20:40:47.161 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module redis:5:8040020211019153849:9f9e2e7e.x86_64 from CentOS-AppStream 2024-04-12T20:40:47.161 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 28: conflicting requests 2024-04-12T20:40:47.161 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module rhn-tools:1.0:8010020191114034948:f69d1239.x86_64 from CentOS-AppStream 2024-04-12T20:40:47.162 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 29: conflicting requests 2024-04-12T20:40:47.162 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module ruby:2.5:820230627084142:9edba152.x86_64 from CentOS-AppStream 2024-04-12T20:40:47.162 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 30: conflicting requests 2024-04-12T20:40:47.162 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module rust-toolset:rhel8:820240119204620:b09eea91.x86_64 from CentOS-AppStream 2024-04-12T20:40:47.162 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 31: conflicting requests 2024-04-12T20:40:47.162 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module satellite-5-client:1.0:8010020191114035551:cdc1202b.x86_64 from CentOS-AppStream 2024-04-12T20:40:47.162 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 32: conflicting requests 2024-04-12T20:40:47.162 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module squid:4:820240319173245:9edba152.x86_64 from CentOS-AppStream 2024-04-12T20:40:47.162 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 33: conflicting requests 2024-04-12T20:40:47.162 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module subversion:1.10:8070020220711155714:78111232.x86_64 from CentOS-AppStream 2024-04-12T20:40:47.162 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 34: conflicting requests 2024-04-12T20:40:47.162 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module swig:3.0:8030020200706142531:30b713e6.x86_64 from CentOS-AppStream 2024-04-12T20:40:47.163 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 35: conflicting requests 2024-04-12T20:40:47.163 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module varnish:6:8050020211109225449:b4937e53.x86_64 from CentOS-AppStream 2024-04-12T20:40:47.163 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 36: conflicting requests 2024-04-12T20:40:47.163 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module virt:rhel:820240314161907:9edba152.x86_64 from CentOS-AppStream 2024-04-12T20:40:47.163 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 37: conflicting requests 2024-04-12T20:40:47.163 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:8030020200716155257:7cc0a66d.x86_64 from CentOS-AppStream 2024-04-12T20:40:47.163 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:8030020200716155257:b967a9a2.x86_64 from CentOS-AppStream 2024-04-12T20:40:47.163 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:820230811133638:36f9fae6.x86_64 from CentOS-AppStream 2024-04-12T20:40:47.163 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 38: cannot install the best candidate for the job 2024-04-12T20:40:47.163 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:8010020191114031501:a5949e2e.x86_64 from CentOS-AppStream 2024-04-12T20:40:47.163 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:8010020191114031501:be2e4737.x86_64 from CentOS-AppStream 2024-04-12T20:40:47.163 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:820230223114759:2ab761e1.x86_64 from CentOS-AppStream 2024-04-12T20:40:47.163 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:820230223114759:86e45846.x86_64 from CentOS-AppStream 2024-04-12T20:40:47.163 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 39: conflicting requests 2024-04-12T20:40:47.164 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:8030020200715230104:1e4bbb35.x86_64 from CentOS-AppStream 2024-04-12T20:40:47.164 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:8030020200715230104:ea09926d.x86_64 from CentOS-AppStream 2024-04-12T20:40:47.164 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:082fdf2f.x86_64 from CentOS-AppStream 2024-04-12T20:40:47.164 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-04-12T20:40:47.164 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-04-12T20:40:47.164 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:fbe42456.x86_64 from CentOS-AppStream 2024-04-12T20:40:47.164 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 40: cannot install the best candidate for the job 2024-04-12T20:40:47.164 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:8010020191114031513:16b3ab4d.x86_64 from CentOS-AppStream 2024-04-12T20:40:47.164 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:8010020191114031513:e1f37755.x86_64 from CentOS-AppStream 2024-04-12T20:40:47.164 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-04-12T20:40:47.164 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-04-12T20:40:47.164 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 41: cannot install the best candidate for the job 2024-04-12T20:40:47.164 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:8030020200716150652:1e4bbb35.x86_64 from CentOS-AppStream 2024-04-12T20:40:47.164 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:8030020200716150652:ea09926d.x86_64 from CentOS-AppStream 2024-04-12T20:40:47.164 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-04-12T20:40:47.165 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-04-12T20:40:47.165 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 42: cannot install the best candidate for the job 2024-04-12T20:40:47.165 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:8030020200716174729:3a70019f.x86_64 from CentOS-AppStream 2024-04-12T20:40:47.165 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:8030020200716174729:cccafca5.x86_64 from CentOS-AppStream 2024-04-12T20:40:47.165 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:820230223114759:6fdf7e5d.x86_64 from CentOS-AppStream 2024-04-12T20:40:47.165 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:820230223114759:d4326aba.x86_64 from CentOS-AppStream 2024-04-12T20:40:47.165 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 43: cannot install the best candidate for the job 2024-04-12T20:40:47.165 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:8030020200716213634:56fce90f.x86_64 from CentOS-AppStream 2024-04-12T20:40:47.165 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:8030020200716213634:98a3c9d0.x86_64 from CentOS-AppStream 2024-04-12T20:40:47.165 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:820230223114759:06b7596e.x86_64 from CentOS-AppStream 2024-04-12T20:40:47.165 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:820230223114759:d2fc5d76.x86_64 from CentOS-AppStream 2024-04-12T20:40:47.165 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 44: cannot install the best candidate for the job 2024-04-12T20:40:47.165 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:8030020200716171139:3a70019f.x86_64 from CentOS-AppStream 2024-04-12T20:40:47.165 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:8030020200716171139:cccafca5.x86_64 from CentOS-AppStream 2024-04-12T20:40:47.166 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:820230223114759:6fdf7e5d.x86_64 from CentOS-AppStream 2024-04-12T20:40:47.166 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:820230223114759:d4326aba.x86_64 from CentOS-AppStream 2024-04-12T20:40:47.166 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 45: cannot install the best candidate for the job 2024-04-12T20:40:47.166 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:8010020191120175858:95fce66d.x86_64 from CentOS-AppStream 2024-04-12T20:40:47.166 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:8030020200723233049:09acf126.x86_64 from CentOS-AppStream 2024-04-12T20:40:47.166 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:820230223114759:436af1ae.x86_64 from CentOS-AppStream 2024-04-12T20:40:47.166 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:820230223114759:b12195ea.x86_64 from CentOS-AppStream 2024-04-12T20:40:47.228 INFO:teuthology.orchestra.run.smithi018.stdout:Dependencies resolved. 2024-04-12T20:40:47.229 INFO:teuthology.orchestra.run.smithi018.stdout:================================================================================ 2024-04-12T20:40:47.229 INFO:teuthology.orchestra.run.smithi018.stdout: Package Arch Version Repo Size 2024-04-12T20:40:47.229 INFO:teuthology.orchestra.run.smithi018.stdout:================================================================================ 2024-04-12T20:40:47.229 INFO:teuthology.orchestra.run.smithi018.stdout:Installing: 2024-04-12T20:40:47.229 INFO:teuthology.orchestra.run.smithi018.stdout: ceph-immutable-object-cache x86_64 2:17.2.7-827.gd3b5060a.el9 ceph 137 k 2024-04-12T20:40:47.230 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-12T20:40:47.230 INFO:teuthology.orchestra.run.smithi018.stdout:Transaction Summary 2024-04-12T20:40:47.230 INFO:teuthology.orchestra.run.smithi018.stdout:================================================================================ 2024-04-12T20:40:47.230 INFO:teuthology.orchestra.run.smithi018.stdout:Install 1 Package 2024-04-12T20:40:47.230 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-12T20:40:47.230 INFO:teuthology.orchestra.run.smithi018.stdout:Total download size: 137 k 2024-04-12T20:40:47.230 INFO:teuthology.orchestra.run.smithi018.stdout:Installed size: 379 k 2024-04-12T20:40:47.230 INFO:teuthology.orchestra.run.smithi018.stdout:Downloading Packages: 2024-04-12T20:40:47.340 INFO:teuthology.orchestra.run.smithi186.stdout:Last metadata expiration check: 0:01:10 ago on Fri 12 Apr 2024 08:39:37 PM UTC. 2024-04-12T20:40:47.637 INFO:teuthology.orchestra.run.smithi186.stderr:Modular dependency problems: 2024-04-12T20:40:47.637 INFO:teuthology.orchestra.run.smithi186.stderr: 2024-04-12T20:40:47.637 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 1: conflicting requests 2024-04-12T20:40:47.637 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module container-tools:rhel8:820240321091303:20125149.x86_64 from CentOS-AppStream 2024-04-12T20:40:47.638 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 2: conflicting requests 2024-04-12T20:40:47.638 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module eclipse:rhel8:820201023100746:b230ed4e.x86_64 from CentOS-AppStream 2024-04-12T20:40:47.638 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 3: conflicting requests 2024-04-12T20:40:47.638 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module freeradius:3.0:8080020230117180605:89170a74.x86_64 from CentOS-AppStream 2024-04-12T20:40:47.638 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 4: conflicting requests 2024-04-12T20:40:47.639 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module gimp:2.8:8000020190628145146:4148dfdf.x86_64 from CentOS-AppStream 2024-04-12T20:40:47.639 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 5: conflicting requests 2024-04-12T20:40:47.639 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module go-toolset:rhel8:820240401160413:b754926a.x86_64 from CentOS-AppStream 2024-04-12T20:40:47.639 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 6: conflicting requests 2024-04-12T20:40:47.639 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module httpd:2.4:820240216084734:9edba152.x86_64 from CentOS-AppStream 2024-04-12T20:40:47.639 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 7: conflicting requests 2024-04-12T20:40:47.640 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module idm:client:820240307184541:49cc9d1b.x86_64 from CentOS-AppStream 2024-04-12T20:40:47.640 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 8: conflicting requests 2024-04-12T20:40:47.640 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module inkscape:0.92.3:8080020221205124429:3e031279.x86_64 from CentOS-AppStream 2024-04-12T20:40:47.640 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 9: conflicting requests 2024-04-12T20:40:47.640 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module jmc:rhel8:8050020211110222101:6392b1f8.x86_64 from CentOS-AppStream 2024-04-12T20:40:47.640 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 10: conflicting requests 2024-04-12T20:40:47.641 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module llvm-toolset:rhel8:820240205133135:9edba152.x86_64 from CentOS-AppStream 2024-04-12T20:40:47.641 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 11: conflicting requests 2024-04-12T20:40:47.641 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module log4j:2:8080020221205124743:9d367344.x86_64 from CentOS-AppStream 2024-04-12T20:40:47.641 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 12: conflicting requests 2024-04-12T20:40:47.641 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module mailman:2.1:820230727085757:77fc8825.x86_64 from CentOS-AppStream 2024-04-12T20:40:47.641 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 13: conflicting requests 2024-04-12T20:40:47.641 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module mariadb:10.3:8030020210419150013:30b713e6.x86_64 from CentOS-AppStream 2024-04-12T20:40:47.642 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 14: conflicting requests 2024-04-12T20:40:47.642 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module maven:3.5:8060020211203142015:c0229ad2.x86_64 from CentOS-AppStream 2024-04-12T20:40:47.642 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 15: conflicting requests 2024-04-12T20:40:47.642 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module mercurial:4.8:8000020190628020724:4148dfdf.x86_64 from CentOS-AppStream 2024-04-12T20:40:47.642 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 16: conflicting requests 2024-04-12T20:40:47.643 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module mod_auth_openidc:2.3:820230721123832:9edba152.x86_64 from CentOS-AppStream 2024-04-12T20:40:47.643 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 17: conflicting requests 2024-04-12T20:40:47.643 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module mysql:8.0:8040020210901180257:9f9e2e7e.x86_64 from CentOS-AppStream 2024-04-12T20:40:47.643 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 18: conflicting requests 2024-04-12T20:40:47.643 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module nginx:1.14:8000020211221191913:55190bc5.x86_64 from CentOS-AppStream 2024-04-12T20:40:47.643 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 19: conflicting requests 2024-04-12T20:40:47.643 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module nodejs:10:8040020210120182536:9f9e2e7e.x86_64 from CentOS-AppStream 2024-04-12T20:40:47.644 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 20: conflicting requests 2024-04-12T20:40:47.644 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl:5.26:8000020190628020724:55190bc5.x86_64 from CentOS-AppStream 2024-04-12T20:40:47.644 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 21: conflicting requests 2024-04-12T20:40:47.644 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module php:7.2:8020020200507003613:2c7ca891.x86_64 from CentOS-AppStream 2024-04-12T20:40:47.644 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 22: conflicting requests 2024-04-12T20:40:47.644 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module postgresql:10:820230223114800:9edba152.x86_64 from CentOS-AppStream 2024-04-12T20:40:47.644 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 23: conflicting requests 2024-04-12T20:40:47.644 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module python27:2.7:820240208011952:182f7c73.x86_64 from CentOS-AppStream 2024-04-12T20:40:47.644 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 24: conflicting requests 2024-04-12T20:40:47.644 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module python36:3.6:8050020210825152031:982725ab.x86_64 from CentOS-AppStream 2024-04-12T20:40:47.644 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 25: conflicting requests 2024-04-12T20:40:47.644 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module python38:3.8:820230810143931:64c1cd5a.x86_64 from CentOS-AppStream 2024-04-12T20:40:47.645 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 26: conflicting requests 2024-04-12T20:40:47.645 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module python39:3.9:820240214182535:17377f89.x86_64 from CentOS-AppStream 2024-04-12T20:40:47.645 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 27: conflicting requests 2024-04-12T20:40:47.645 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module redis:5:8040020211019153849:9f9e2e7e.x86_64 from CentOS-AppStream 2024-04-12T20:40:47.645 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 28: conflicting requests 2024-04-12T20:40:47.645 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module rhn-tools:1.0:8010020191114034948:f69d1239.x86_64 from CentOS-AppStream 2024-04-12T20:40:47.645 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 29: conflicting requests 2024-04-12T20:40:47.645 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module ruby:2.5:820230627084142:9edba152.x86_64 from CentOS-AppStream 2024-04-12T20:40:47.645 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 30: conflicting requests 2024-04-12T20:40:47.645 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module rust-toolset:rhel8:820240119204620:b09eea91.x86_64 from CentOS-AppStream 2024-04-12T20:40:47.645 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 31: conflicting requests 2024-04-12T20:40:47.646 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module satellite-5-client:1.0:8010020191114035551:cdc1202b.x86_64 from CentOS-AppStream 2024-04-12T20:40:47.646 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 32: conflicting requests 2024-04-12T20:40:47.646 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module squid:4:820240319173245:9edba152.x86_64 from CentOS-AppStream 2024-04-12T20:40:47.646 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 33: conflicting requests 2024-04-12T20:40:47.646 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module subversion:1.10:8070020220711155714:78111232.x86_64 from CentOS-AppStream 2024-04-12T20:40:47.646 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 34: conflicting requests 2024-04-12T20:40:47.646 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module swig:3.0:8030020200706142531:30b713e6.x86_64 from CentOS-AppStream 2024-04-12T20:40:47.646 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 35: conflicting requests 2024-04-12T20:40:47.646 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module varnish:6:8050020211109225449:b4937e53.x86_64 from CentOS-AppStream 2024-04-12T20:40:47.646 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 36: conflicting requests 2024-04-12T20:40:47.646 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module virt:rhel:820240314161907:9edba152.x86_64 from CentOS-AppStream 2024-04-12T20:40:47.646 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 37: conflicting requests 2024-04-12T20:40:47.646 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:8030020200716155257:7cc0a66d.x86_64 from CentOS-AppStream 2024-04-12T20:40:47.646 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:8030020200716155257:b967a9a2.x86_64 from CentOS-AppStream 2024-04-12T20:40:47.647 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:820230811133638:36f9fae6.x86_64 from CentOS-AppStream 2024-04-12T20:40:47.647 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 38: cannot install the best candidate for the job 2024-04-12T20:40:47.647 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:8010020191114031501:a5949e2e.x86_64 from CentOS-AppStream 2024-04-12T20:40:47.647 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:8010020191114031501:be2e4737.x86_64 from CentOS-AppStream 2024-04-12T20:40:47.647 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:820230223114759:2ab761e1.x86_64 from CentOS-AppStream 2024-04-12T20:40:47.647 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:820230223114759:86e45846.x86_64 from CentOS-AppStream 2024-04-12T20:40:47.647 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 39: conflicting requests 2024-04-12T20:40:47.647 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:8030020200715230104:1e4bbb35.x86_64 from CentOS-AppStream 2024-04-12T20:40:47.647 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:8030020200715230104:ea09926d.x86_64 from CentOS-AppStream 2024-04-12T20:40:47.647 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:082fdf2f.x86_64 from CentOS-AppStream 2024-04-12T20:40:47.647 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-04-12T20:40:47.647 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-04-12T20:40:47.648 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:fbe42456.x86_64 from CentOS-AppStream 2024-04-12T20:40:47.648 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 40: cannot install the best candidate for the job 2024-04-12T20:40:47.648 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:8010020191114031513:16b3ab4d.x86_64 from CentOS-AppStream 2024-04-12T20:40:47.648 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:8010020191114031513:e1f37755.x86_64 from CentOS-AppStream 2024-04-12T20:40:47.648 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-04-12T20:40:47.648 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-04-12T20:40:47.648 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 41: cannot install the best candidate for the job 2024-04-12T20:40:47.648 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:8030020200716150652:1e4bbb35.x86_64 from CentOS-AppStream 2024-04-12T20:40:47.648 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:8030020200716150652:ea09926d.x86_64 from CentOS-AppStream 2024-04-12T20:40:47.648 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-04-12T20:40:47.648 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-04-12T20:40:47.648 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 42: cannot install the best candidate for the job 2024-04-12T20:40:47.648 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:8030020200716174729:3a70019f.x86_64 from CentOS-AppStream 2024-04-12T20:40:47.649 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:8030020200716174729:cccafca5.x86_64 from CentOS-AppStream 2024-04-12T20:40:47.649 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:820230223114759:6fdf7e5d.x86_64 from CentOS-AppStream 2024-04-12T20:40:47.649 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:820230223114759:d4326aba.x86_64 from CentOS-AppStream 2024-04-12T20:40:47.649 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 43: cannot install the best candidate for the job 2024-04-12T20:40:47.649 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:8030020200716213634:56fce90f.x86_64 from CentOS-AppStream 2024-04-12T20:40:47.649 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:8030020200716213634:98a3c9d0.x86_64 from CentOS-AppStream 2024-04-12T20:40:47.649 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:820230223114759:06b7596e.x86_64 from CentOS-AppStream 2024-04-12T20:40:47.649 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:820230223114759:d2fc5d76.x86_64 from CentOS-AppStream 2024-04-12T20:40:47.649 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 44: cannot install the best candidate for the job 2024-04-12T20:40:47.649 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:8030020200716171139:3a70019f.x86_64 from CentOS-AppStream 2024-04-12T20:40:47.649 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:8030020200716171139:cccafca5.x86_64 from CentOS-AppStream 2024-04-12T20:40:47.649 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:820230223114759:6fdf7e5d.x86_64 from CentOS-AppStream 2024-04-12T20:40:47.649 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:820230223114759:d4326aba.x86_64 from CentOS-AppStream 2024-04-12T20:40:47.649 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 45: cannot install the best candidate for the job 2024-04-12T20:40:47.649 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:8010020191120175858:95fce66d.x86_64 from CentOS-AppStream 2024-04-12T20:40:47.650 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:8030020200723233049:09acf126.x86_64 from CentOS-AppStream 2024-04-12T20:40:47.650 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:820230223114759:436af1ae.x86_64 from CentOS-AppStream 2024-04-12T20:40:47.650 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:820230223114759:b12195ea.x86_64 from CentOS-AppStream 2024-04-12T20:40:47.707 INFO:teuthology.orchestra.run.smithi186.stdout:Dependencies resolved. 2024-04-12T20:40:47.708 INFO:teuthology.orchestra.run.smithi186.stdout:================================================================================ 2024-04-12T20:40:47.709 INFO:teuthology.orchestra.run.smithi186.stdout: Package Arch Version Repo Size 2024-04-12T20:40:47.709 INFO:teuthology.orchestra.run.smithi186.stdout:================================================================================ 2024-04-12T20:40:47.709 INFO:teuthology.orchestra.run.smithi186.stdout:Installing: 2024-04-12T20:40:47.709 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-immutable-object-cache x86_64 2:17.2.7-827.gd3b5060a.el9 ceph 137 k 2024-04-12T20:40:47.709 INFO:teuthology.orchestra.run.smithi186.stdout: 2024-04-12T20:40:47.709 INFO:teuthology.orchestra.run.smithi186.stdout:Transaction Summary 2024-04-12T20:40:47.710 INFO:teuthology.orchestra.run.smithi186.stdout:================================================================================ 2024-04-12T20:40:47.710 INFO:teuthology.orchestra.run.smithi186.stdout:Install 1 Package 2024-04-12T20:40:47.710 INFO:teuthology.orchestra.run.smithi186.stdout: 2024-04-12T20:40:47.710 INFO:teuthology.orchestra.run.smithi186.stdout:Total download size: 137 k 2024-04-12T20:40:47.710 INFO:teuthology.orchestra.run.smithi186.stdout:Installed size: 379 k 2024-04-12T20:40:47.710 INFO:teuthology.orchestra.run.smithi186.stdout:Downloading Packages: 2024-04-12T20:40:47.752 INFO:teuthology.orchestra.run.smithi018.stdout:ceph-immutable-object-cache-17.2.7-827.gd3b5060 263 kB/s | 137 kB 00:00 2024-04-12T20:40:47.754 INFO:teuthology.orchestra.run.smithi018.stdout:-------------------------------------------------------------------------------- 2024-04-12T20:40:47.755 INFO:teuthology.orchestra.run.smithi018.stdout:Total 262 kB/s | 137 kB 00:00 2024-04-12T20:40:47.755 INFO:teuthology.orchestra.run.smithi018.stdout:Running transaction check 2024-04-12T20:40:47.783 INFO:teuthology.orchestra.run.smithi018.stdout:Transaction check succeeded. 2024-04-12T20:40:47.784 INFO:teuthology.orchestra.run.smithi018.stdout:Running transaction test 2024-04-12T20:40:47.884 INFO:teuthology.orchestra.run.smithi018.stdout:Transaction test succeeded. 2024-04-12T20:40:47.885 INFO:teuthology.orchestra.run.smithi018.stdout:Running transaction 2024-04-12T20:40:47.943 INFO:teuthology.orchestra.run.smithi186.stdout:ceph-immutable-object-cache-17.2.7-827.gd3b5060 588 kB/s | 137 kB 00:00 2024-04-12T20:40:47.945 INFO:teuthology.orchestra.run.smithi186.stdout:-------------------------------------------------------------------------------- 2024-04-12T20:40:47.946 INFO:teuthology.orchestra.run.smithi186.stdout:Total 580 kB/s | 137 kB 00:00 2024-04-12T20:40:47.946 INFO:teuthology.orchestra.run.smithi186.stdout:Running transaction check 2024-04-12T20:40:47.971 INFO:teuthology.orchestra.run.smithi186.stdout:Transaction check succeeded. 2024-04-12T20:40:47.972 INFO:teuthology.orchestra.run.smithi186.stdout:Running transaction test 2024-04-12T20:40:48.067 INFO:teuthology.orchestra.run.smithi186.stdout:Transaction test succeeded. 2024-04-12T20:40:48.068 INFO:teuthology.orchestra.run.smithi186.stdout:Running transaction 2024-04-12T20:40:48.092 INFO:teuthology.orchestra.run.smithi018.stdout: Preparing : 1/1 2024-04-12T20:40:48.170 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : ceph-immutable-object-cache-2:17.2.7-827.gd3b5060a.e 1/1 2024-04-12T20:40:48.243 INFO:teuthology.orchestra.run.smithi186.stdout: Preparing : 1/1 2024-04-12T20:40:48.249 INFO:teuthology.orchestra.run.smithi018.stdout: Running scriptlet: ceph-immutable-object-cache-2:17.2.7-827.gd3b5060a.e 1/1 2024-04-12T20:40:48.249 INFO:teuthology.orchestra.run.smithi018.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-12T20:40:48.249 INFO:teuthology.orchestra.run.smithi018.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-04-12T20:40:48.249 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-12T20:40:48.328 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : ceph-immutable-object-cache-2:17.2.7-827.gd3b5060a.e 1/1 2024-04-12T20:40:48.400 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: ceph-immutable-object-cache-2:17.2.7-827.gd3b5060a.e 1/1 2024-04-12T20:40:48.400 INFO:teuthology.orchestra.run.smithi186.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-12T20:40:48.400 INFO:teuthology.orchestra.run.smithi186.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-04-12T20:40:48.400 INFO:teuthology.orchestra.run.smithi186.stdout: 2024-04-12T20:40:49.728 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : ceph-immutable-object-cache-2:17.2.7-827.gd3b5060a.e 1/1 2024-04-12T20:40:49.728 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-12T20:40:49.728 INFO:teuthology.orchestra.run.smithi018.stdout:Installed: 2024-04-12T20:40:49.728 INFO:teuthology.orchestra.run.smithi018.stdout: ceph-immutable-object-cache-2:17.2.7-827.gd3b5060a.el9.x86_64 2024-04-12T20:40:49.728 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-12T20:40:49.729 INFO:teuthology.orchestra.run.smithi018.stdout:Complete! 2024-04-12T20:40:49.835 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : ceph-immutable-object-cache-2:17.2.7-827.gd3b5060a.e 1/1 2024-04-12T20:40:49.836 INFO:teuthology.orchestra.run.smithi186.stdout: 2024-04-12T20:40:49.836 INFO:teuthology.orchestra.run.smithi186.stdout:Installed: 2024-04-12T20:40:49.836 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-immutable-object-cache-2:17.2.7-827.gd3b5060a.el9.x86_64 2024-04-12T20:40:49.836 INFO:teuthology.orchestra.run.smithi186.stdout: 2024-04-12T20:40:49.836 INFO:teuthology.orchestra.run.smithi186.stdout:Complete! 2024-04-12T20:40:50.108 DEBUG:teuthology.orchestra.run.smithi018:> sudo yum -y install ceph-mgr 2024-04-12T20:40:50.140 DEBUG:teuthology.orchestra.run.smithi186:> sudo yum -y install ceph-mgr 2024-04-12T20:40:51.027 INFO:teuthology.orchestra.run.smithi186.stdout:Last metadata expiration check: 0:01:14 ago on Fri 12 Apr 2024 08:39:37 PM UTC. 2024-04-12T20:40:51.037 INFO:teuthology.orchestra.run.smithi018.stdout:Last metadata expiration check: 0:01:16 ago on Fri 12 Apr 2024 08:39:35 PM UTC. 2024-04-12T20:40:51.325 INFO:teuthology.orchestra.run.smithi186.stderr:Modular dependency problems: 2024-04-12T20:40:51.325 INFO:teuthology.orchestra.run.smithi186.stderr: 2024-04-12T20:40:51.326 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 1: conflicting requests 2024-04-12T20:40:51.326 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module container-tools:rhel8:820240321091303:20125149.x86_64 from CentOS-AppStream 2024-04-12T20:40:51.326 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 2: conflicting requests 2024-04-12T20:40:51.326 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module eclipse:rhel8:820201023100746:b230ed4e.x86_64 from CentOS-AppStream 2024-04-12T20:40:51.326 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 3: conflicting requests 2024-04-12T20:40:51.326 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module freeradius:3.0:8080020230117180605:89170a74.x86_64 from CentOS-AppStream 2024-04-12T20:40:51.326 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 4: conflicting requests 2024-04-12T20:40:51.326 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module gimp:2.8:8000020190628145146:4148dfdf.x86_64 from CentOS-AppStream 2024-04-12T20:40:51.326 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 5: conflicting requests 2024-04-12T20:40:51.326 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module go-toolset:rhel8:820240401160413:b754926a.x86_64 from CentOS-AppStream 2024-04-12T20:40:51.326 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 6: conflicting requests 2024-04-12T20:40:51.327 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module httpd:2.4:820240216084734:9edba152.x86_64 from CentOS-AppStream 2024-04-12T20:40:51.327 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 7: conflicting requests 2024-04-12T20:40:51.327 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module idm:client:820240307184541:49cc9d1b.x86_64 from CentOS-AppStream 2024-04-12T20:40:51.327 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 8: conflicting requests 2024-04-12T20:40:51.327 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module inkscape:0.92.3:8080020221205124429:3e031279.x86_64 from CentOS-AppStream 2024-04-12T20:40:51.327 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 9: conflicting requests 2024-04-12T20:40:51.327 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module jmc:rhel8:8050020211110222101:6392b1f8.x86_64 from CentOS-AppStream 2024-04-12T20:40:51.327 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 10: conflicting requests 2024-04-12T20:40:51.327 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module llvm-toolset:rhel8:820240205133135:9edba152.x86_64 from CentOS-AppStream 2024-04-12T20:40:51.327 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 11: conflicting requests 2024-04-12T20:40:51.327 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module log4j:2:8080020221205124743:9d367344.x86_64 from CentOS-AppStream 2024-04-12T20:40:51.328 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 12: conflicting requests 2024-04-12T20:40:51.328 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module mailman:2.1:820230727085757:77fc8825.x86_64 from CentOS-AppStream 2024-04-12T20:40:51.328 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 13: conflicting requests 2024-04-12T20:40:51.328 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module mariadb:10.3:8030020210419150013:30b713e6.x86_64 from CentOS-AppStream 2024-04-12T20:40:51.328 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 14: conflicting requests 2024-04-12T20:40:51.328 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module maven:3.5:8060020211203142015:c0229ad2.x86_64 from CentOS-AppStream 2024-04-12T20:40:51.328 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 15: conflicting requests 2024-04-12T20:40:51.328 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module mercurial:4.8:8000020190628020724:4148dfdf.x86_64 from CentOS-AppStream 2024-04-12T20:40:51.328 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 16: conflicting requests 2024-04-12T20:40:51.329 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module mod_auth_openidc:2.3:820230721123832:9edba152.x86_64 from CentOS-AppStream 2024-04-12T20:40:51.329 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 17: conflicting requests 2024-04-12T20:40:51.329 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module mysql:8.0:8040020210901180257:9f9e2e7e.x86_64 from CentOS-AppStream 2024-04-12T20:40:51.329 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 18: conflicting requests 2024-04-12T20:40:51.329 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module nginx:1.14:8000020211221191913:55190bc5.x86_64 from CentOS-AppStream 2024-04-12T20:40:51.329 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 19: conflicting requests 2024-04-12T20:40:51.329 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module nodejs:10:8040020210120182536:9f9e2e7e.x86_64 from CentOS-AppStream 2024-04-12T20:40:51.329 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 20: conflicting requests 2024-04-12T20:40:51.329 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl:5.26:8000020190628020724:55190bc5.x86_64 from CentOS-AppStream 2024-04-12T20:40:51.329 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 21: conflicting requests 2024-04-12T20:40:51.330 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module php:7.2:8020020200507003613:2c7ca891.x86_64 from CentOS-AppStream 2024-04-12T20:40:51.330 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 22: conflicting requests 2024-04-12T20:40:51.330 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module postgresql:10:820230223114800:9edba152.x86_64 from CentOS-AppStream 2024-04-12T20:40:51.330 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 23: conflicting requests 2024-04-12T20:40:51.330 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module python27:2.7:820240208011952:182f7c73.x86_64 from CentOS-AppStream 2024-04-12T20:40:51.330 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 24: conflicting requests 2024-04-12T20:40:51.330 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module python36:3.6:8050020210825152031:982725ab.x86_64 from CentOS-AppStream 2024-04-12T20:40:51.330 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 25: conflicting requests 2024-04-12T20:40:51.330 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module python38:3.8:820230810143931:64c1cd5a.x86_64 from CentOS-AppStream 2024-04-12T20:40:51.330 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 26: conflicting requests 2024-04-12T20:40:51.331 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module python39:3.9:820240214182535:17377f89.x86_64 from CentOS-AppStream 2024-04-12T20:40:51.331 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 27: conflicting requests 2024-04-12T20:40:51.331 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module redis:5:8040020211019153849:9f9e2e7e.x86_64 from CentOS-AppStream 2024-04-12T20:40:51.331 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 28: conflicting requests 2024-04-12T20:40:51.331 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module rhn-tools:1.0:8010020191114034948:f69d1239.x86_64 from CentOS-AppStream 2024-04-12T20:40:51.331 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 29: conflicting requests 2024-04-12T20:40:51.331 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module ruby:2.5:820230627084142:9edba152.x86_64 from CentOS-AppStream 2024-04-12T20:40:51.331 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 30: conflicting requests 2024-04-12T20:40:51.331 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module rust-toolset:rhel8:820240119204620:b09eea91.x86_64 from CentOS-AppStream 2024-04-12T20:40:51.331 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 31: conflicting requests 2024-04-12T20:40:51.331 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module satellite-5-client:1.0:8010020191114035551:cdc1202b.x86_64 from CentOS-AppStream 2024-04-12T20:40:51.332 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 32: conflicting requests 2024-04-12T20:40:51.332 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module squid:4:820240319173245:9edba152.x86_64 from CentOS-AppStream 2024-04-12T20:40:51.332 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 33: conflicting requests 2024-04-12T20:40:51.332 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module subversion:1.10:8070020220711155714:78111232.x86_64 from CentOS-AppStream 2024-04-12T20:40:51.332 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 34: conflicting requests 2024-04-12T20:40:51.332 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module swig:3.0:8030020200706142531:30b713e6.x86_64 from CentOS-AppStream 2024-04-12T20:40:51.332 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 35: conflicting requests 2024-04-12T20:40:51.332 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module varnish:6:8050020211109225449:b4937e53.x86_64 from CentOS-AppStream 2024-04-12T20:40:51.332 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 36: conflicting requests 2024-04-12T20:40:51.332 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module virt:rhel:820240314161907:9edba152.x86_64 from CentOS-AppStream 2024-04-12T20:40:51.333 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 37: conflicting requests 2024-04-12T20:40:51.333 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:8030020200716155257:7cc0a66d.x86_64 from CentOS-AppStream 2024-04-12T20:40:51.333 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:8030020200716155257:b967a9a2.x86_64 from CentOS-AppStream 2024-04-12T20:40:51.333 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:820230811133638:36f9fae6.x86_64 from CentOS-AppStream 2024-04-12T20:40:51.333 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 38: cannot install the best candidate for the job 2024-04-12T20:40:51.333 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:8010020191114031501:a5949e2e.x86_64 from CentOS-AppStream 2024-04-12T20:40:51.333 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:8010020191114031501:be2e4737.x86_64 from CentOS-AppStream 2024-04-12T20:40:51.333 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:820230223114759:2ab761e1.x86_64 from CentOS-AppStream 2024-04-12T20:40:51.333 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:820230223114759:86e45846.x86_64 from CentOS-AppStream 2024-04-12T20:40:51.333 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 39: conflicting requests 2024-04-12T20:40:51.333 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:8030020200715230104:1e4bbb35.x86_64 from CentOS-AppStream 2024-04-12T20:40:51.334 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:8030020200715230104:ea09926d.x86_64 from CentOS-AppStream 2024-04-12T20:40:51.334 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:082fdf2f.x86_64 from CentOS-AppStream 2024-04-12T20:40:51.334 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-04-12T20:40:51.334 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-04-12T20:40:51.334 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:fbe42456.x86_64 from CentOS-AppStream 2024-04-12T20:40:51.334 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 40: cannot install the best candidate for the job 2024-04-12T20:40:51.334 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:8010020191114031513:16b3ab4d.x86_64 from CentOS-AppStream 2024-04-12T20:40:51.334 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:8010020191114031513:e1f37755.x86_64 from CentOS-AppStream 2024-04-12T20:40:51.334 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-04-12T20:40:51.334 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-04-12T20:40:51.335 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 41: cannot install the best candidate for the job 2024-04-12T20:40:51.335 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:8030020200716150652:1e4bbb35.x86_64 from CentOS-AppStream 2024-04-12T20:40:51.335 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:8030020200716150652:ea09926d.x86_64 from CentOS-AppStream 2024-04-12T20:40:51.335 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-04-12T20:40:51.335 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-04-12T20:40:51.335 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 42: cannot install the best candidate for the job 2024-04-12T20:40:51.335 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:8030020200716174729:3a70019f.x86_64 from CentOS-AppStream 2024-04-12T20:40:51.335 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:8030020200716174729:cccafca5.x86_64 from CentOS-AppStream 2024-04-12T20:40:51.335 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:820230223114759:6fdf7e5d.x86_64 from CentOS-AppStream 2024-04-12T20:40:51.335 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:820230223114759:d4326aba.x86_64 from CentOS-AppStream 2024-04-12T20:40:51.335 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 43: cannot install the best candidate for the job 2024-04-12T20:40:51.335 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:8030020200716213634:56fce90f.x86_64 from CentOS-AppStream 2024-04-12T20:40:51.335 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:8030020200716213634:98a3c9d0.x86_64 from CentOS-AppStream 2024-04-12T20:40:51.336 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:820230223114759:06b7596e.x86_64 from CentOS-AppStream 2024-04-12T20:40:51.336 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:820230223114759:d2fc5d76.x86_64 from CentOS-AppStream 2024-04-12T20:40:51.336 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 44: cannot install the best candidate for the job 2024-04-12T20:40:51.336 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:8030020200716171139:3a70019f.x86_64 from CentOS-AppStream 2024-04-12T20:40:51.336 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:8030020200716171139:cccafca5.x86_64 from CentOS-AppStream 2024-04-12T20:40:51.336 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:820230223114759:6fdf7e5d.x86_64 from CentOS-AppStream 2024-04-12T20:40:51.336 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:820230223114759:d4326aba.x86_64 from CentOS-AppStream 2024-04-12T20:40:51.336 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 45: cannot install the best candidate for the job 2024-04-12T20:40:51.336 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:8010020191120175858:95fce66d.x86_64 from CentOS-AppStream 2024-04-12T20:40:51.336 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:8030020200723233049:09acf126.x86_64 from CentOS-AppStream 2024-04-12T20:40:51.337 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:820230223114759:436af1ae.x86_64 from CentOS-AppStream 2024-04-12T20:40:51.337 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:820230223114759:b12195ea.x86_64 from CentOS-AppStream 2024-04-12T20:40:51.337 INFO:teuthology.orchestra.run.smithi186.stdout:Package ceph-mgr-2:17.2.7-827.gd3b5060a.el9.x86_64 is already installed. 2024-04-12T20:40:51.355 INFO:teuthology.orchestra.run.smithi018.stderr:Modular dependency problems: 2024-04-12T20:40:51.355 INFO:teuthology.orchestra.run.smithi018.stderr: 2024-04-12T20:40:51.355 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 1: conflicting requests 2024-04-12T20:40:51.355 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module container-tools:rhel8:820240321091303:20125149.x86_64 from CentOS-AppStream 2024-04-12T20:40:51.355 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 2: conflicting requests 2024-04-12T20:40:51.355 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module eclipse:rhel8:820201023100746:b230ed4e.x86_64 from CentOS-AppStream 2024-04-12T20:40:51.355 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 3: conflicting requests 2024-04-12T20:40:51.355 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module freeradius:3.0:8080020230117180605:89170a74.x86_64 from CentOS-AppStream 2024-04-12T20:40:51.355 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 4: conflicting requests 2024-04-12T20:40:51.356 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module gimp:2.8:8000020190628145146:4148dfdf.x86_64 from CentOS-AppStream 2024-04-12T20:40:51.356 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 5: conflicting requests 2024-04-12T20:40:51.356 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module go-toolset:rhel8:820240401160413:b754926a.x86_64 from CentOS-AppStream 2024-04-12T20:40:51.356 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 6: conflicting requests 2024-04-12T20:40:51.356 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module httpd:2.4:820240216084734:9edba152.x86_64 from CentOS-AppStream 2024-04-12T20:40:51.356 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 7: conflicting requests 2024-04-12T20:40:51.356 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module idm:client:820240307184541:49cc9d1b.x86_64 from CentOS-AppStream 2024-04-12T20:40:51.356 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 8: conflicting requests 2024-04-12T20:40:51.356 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module inkscape:0.92.3:8080020221205124429:3e031279.x86_64 from CentOS-AppStream 2024-04-12T20:40:51.356 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 9: conflicting requests 2024-04-12T20:40:51.356 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module jmc:rhel8:8050020211110222101:6392b1f8.x86_64 from CentOS-AppStream 2024-04-12T20:40:51.356 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 10: conflicting requests 2024-04-12T20:40:51.356 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module llvm-toolset:rhel8:820240205133135:9edba152.x86_64 from CentOS-AppStream 2024-04-12T20:40:51.356 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 11: conflicting requests 2024-04-12T20:40:51.357 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module log4j:2:8080020221205124743:9d367344.x86_64 from CentOS-AppStream 2024-04-12T20:40:51.357 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 12: conflicting requests 2024-04-12T20:40:51.357 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module mailman:2.1:820230727085757:77fc8825.x86_64 from CentOS-AppStream 2024-04-12T20:40:51.357 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 13: conflicting requests 2024-04-12T20:40:51.357 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module mariadb:10.3:8030020210419150013:30b713e6.x86_64 from CentOS-AppStream 2024-04-12T20:40:51.357 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 14: conflicting requests 2024-04-12T20:40:51.357 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module maven:3.5:8060020211203142015:c0229ad2.x86_64 from CentOS-AppStream 2024-04-12T20:40:51.357 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 15: conflicting requests 2024-04-12T20:40:51.357 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module mercurial:4.8:8000020190628020724:4148dfdf.x86_64 from CentOS-AppStream 2024-04-12T20:40:51.357 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 16: conflicting requests 2024-04-12T20:40:51.357 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module mod_auth_openidc:2.3:820230721123832:9edba152.x86_64 from CentOS-AppStream 2024-04-12T20:40:51.357 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 17: conflicting requests 2024-04-12T20:40:51.357 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module mysql:8.0:8040020210901180257:9f9e2e7e.x86_64 from CentOS-AppStream 2024-04-12T20:40:51.357 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 18: conflicting requests 2024-04-12T20:40:51.358 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module nginx:1.14:8000020211221191913:55190bc5.x86_64 from CentOS-AppStream 2024-04-12T20:40:51.358 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 19: conflicting requests 2024-04-12T20:40:51.358 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module nodejs:10:8040020210120182536:9f9e2e7e.x86_64 from CentOS-AppStream 2024-04-12T20:40:51.358 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 20: conflicting requests 2024-04-12T20:40:51.358 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl:5.26:8000020190628020724:55190bc5.x86_64 from CentOS-AppStream 2024-04-12T20:40:51.358 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 21: conflicting requests 2024-04-12T20:40:51.358 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module php:7.2:8020020200507003613:2c7ca891.x86_64 from CentOS-AppStream 2024-04-12T20:40:51.358 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 22: conflicting requests 2024-04-12T20:40:51.358 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module postgresql:10:820230223114800:9edba152.x86_64 from CentOS-AppStream 2024-04-12T20:40:51.358 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 23: conflicting requests 2024-04-12T20:40:51.358 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module python27:2.7:820240208011952:182f7c73.x86_64 from CentOS-AppStream 2024-04-12T20:40:51.358 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 24: conflicting requests 2024-04-12T20:40:51.358 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module python36:3.6:8050020210825152031:982725ab.x86_64 from CentOS-AppStream 2024-04-12T20:40:51.359 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 25: conflicting requests 2024-04-12T20:40:51.359 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module python38:3.8:820230810143931:64c1cd5a.x86_64 from CentOS-AppStream 2024-04-12T20:40:51.359 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 26: conflicting requests 2024-04-12T20:40:51.359 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module python39:3.9:820240214182535:17377f89.x86_64 from CentOS-AppStream 2024-04-12T20:40:51.359 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 27: conflicting requests 2024-04-12T20:40:51.359 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module redis:5:8040020211019153849:9f9e2e7e.x86_64 from CentOS-AppStream 2024-04-12T20:40:51.359 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 28: conflicting requests 2024-04-12T20:40:51.359 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module rhn-tools:1.0:8010020191114034948:f69d1239.x86_64 from CentOS-AppStream 2024-04-12T20:40:51.359 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 29: conflicting requests 2024-04-12T20:40:51.359 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module ruby:2.5:820230627084142:9edba152.x86_64 from CentOS-AppStream 2024-04-12T20:40:51.359 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 30: conflicting requests 2024-04-12T20:40:51.359 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module rust-toolset:rhel8:820240119204620:b09eea91.x86_64 from CentOS-AppStream 2024-04-12T20:40:51.359 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 31: conflicting requests 2024-04-12T20:40:51.359 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module satellite-5-client:1.0:8010020191114035551:cdc1202b.x86_64 from CentOS-AppStream 2024-04-12T20:40:51.360 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 32: conflicting requests 2024-04-12T20:40:51.360 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module squid:4:820240319173245:9edba152.x86_64 from CentOS-AppStream 2024-04-12T20:40:51.360 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 33: conflicting requests 2024-04-12T20:40:51.360 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module subversion:1.10:8070020220711155714:78111232.x86_64 from CentOS-AppStream 2024-04-12T20:40:51.360 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 34: conflicting requests 2024-04-12T20:40:51.360 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module swig:3.0:8030020200706142531:30b713e6.x86_64 from CentOS-AppStream 2024-04-12T20:40:51.360 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 35: conflicting requests 2024-04-12T20:40:51.360 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module varnish:6:8050020211109225449:b4937e53.x86_64 from CentOS-AppStream 2024-04-12T20:40:51.360 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 36: conflicting requests 2024-04-12T20:40:51.360 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module virt:rhel:820240314161907:9edba152.x86_64 from CentOS-AppStream 2024-04-12T20:40:51.360 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 37: conflicting requests 2024-04-12T20:40:51.360 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:8030020200716155257:7cc0a66d.x86_64 from CentOS-AppStream 2024-04-12T20:40:51.360 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:8030020200716155257:b967a9a2.x86_64 from CentOS-AppStream 2024-04-12T20:40:51.360 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:820230811133638:36f9fae6.x86_64 from CentOS-AppStream 2024-04-12T20:40:51.361 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 38: cannot install the best candidate for the job 2024-04-12T20:40:51.361 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:8010020191114031501:a5949e2e.x86_64 from CentOS-AppStream 2024-04-12T20:40:51.361 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:8010020191114031501:be2e4737.x86_64 from CentOS-AppStream 2024-04-12T20:40:51.361 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:820230223114759:2ab761e1.x86_64 from CentOS-AppStream 2024-04-12T20:40:51.361 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:820230223114759:86e45846.x86_64 from CentOS-AppStream 2024-04-12T20:40:51.361 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 39: conflicting requests 2024-04-12T20:40:51.361 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:8030020200715230104:1e4bbb35.x86_64 from CentOS-AppStream 2024-04-12T20:40:51.361 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:8030020200715230104:ea09926d.x86_64 from CentOS-AppStream 2024-04-12T20:40:51.361 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:082fdf2f.x86_64 from CentOS-AppStream 2024-04-12T20:40:51.361 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-04-12T20:40:51.361 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-04-12T20:40:51.361 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:fbe42456.x86_64 from CentOS-AppStream 2024-04-12T20:40:51.361 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 40: cannot install the best candidate for the job 2024-04-12T20:40:51.361 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:8010020191114031513:16b3ab4d.x86_64 from CentOS-AppStream 2024-04-12T20:40:51.361 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:8010020191114031513:e1f37755.x86_64 from CentOS-AppStream 2024-04-12T20:40:51.362 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-04-12T20:40:51.362 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-04-12T20:40:51.362 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 41: cannot install the best candidate for the job 2024-04-12T20:40:51.362 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:8030020200716150652:1e4bbb35.x86_64 from CentOS-AppStream 2024-04-12T20:40:51.362 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:8030020200716150652:ea09926d.x86_64 from CentOS-AppStream 2024-04-12T20:40:51.362 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-04-12T20:40:51.362 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-04-12T20:40:51.362 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 42: cannot install the best candidate for the job 2024-04-12T20:40:51.362 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:8030020200716174729:3a70019f.x86_64 from CentOS-AppStream 2024-04-12T20:40:51.362 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:8030020200716174729:cccafca5.x86_64 from CentOS-AppStream 2024-04-12T20:40:51.362 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:820230223114759:6fdf7e5d.x86_64 from CentOS-AppStream 2024-04-12T20:40:51.362 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:820230223114759:d4326aba.x86_64 from CentOS-AppStream 2024-04-12T20:40:51.362 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 43: cannot install the best candidate for the job 2024-04-12T20:40:51.363 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:8030020200716213634:56fce90f.x86_64 from CentOS-AppStream 2024-04-12T20:40:51.363 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:8030020200716213634:98a3c9d0.x86_64 from CentOS-AppStream 2024-04-12T20:40:51.363 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:820230223114759:06b7596e.x86_64 from CentOS-AppStream 2024-04-12T20:40:51.363 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:820230223114759:d2fc5d76.x86_64 from CentOS-AppStream 2024-04-12T20:40:51.363 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 44: cannot install the best candidate for the job 2024-04-12T20:40:51.363 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:8030020200716171139:3a70019f.x86_64 from CentOS-AppStream 2024-04-12T20:40:51.363 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:8030020200716171139:cccafca5.x86_64 from CentOS-AppStream 2024-04-12T20:40:51.363 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:820230223114759:6fdf7e5d.x86_64 from CentOS-AppStream 2024-04-12T20:40:51.363 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:820230223114759:d4326aba.x86_64 from CentOS-AppStream 2024-04-12T20:40:51.363 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 45: cannot install the best candidate for the job 2024-04-12T20:40:51.363 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:8010020191120175858:95fce66d.x86_64 from CentOS-AppStream 2024-04-12T20:40:51.363 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:8030020200723233049:09acf126.x86_64 from CentOS-AppStream 2024-04-12T20:40:51.363 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:820230223114759:436af1ae.x86_64 from CentOS-AppStream 2024-04-12T20:40:51.363 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:820230223114759:b12195ea.x86_64 from CentOS-AppStream 2024-04-12T20:40:51.364 INFO:teuthology.orchestra.run.smithi018.stdout:Package ceph-mgr-2:17.2.7-827.gd3b5060a.el9.x86_64 is already installed. 2024-04-12T20:40:51.397 INFO:teuthology.orchestra.run.smithi186.stdout:Dependencies resolved. 2024-04-12T20:40:51.398 INFO:teuthology.orchestra.run.smithi186.stdout:Nothing to do. 2024-04-12T20:40:51.398 INFO:teuthology.orchestra.run.smithi186.stdout:Complete! 2024-04-12T20:40:51.432 INFO:teuthology.orchestra.run.smithi018.stdout:Dependencies resolved. 2024-04-12T20:40:51.433 INFO:teuthology.orchestra.run.smithi018.stdout:Nothing to do. 2024-04-12T20:40:51.433 INFO:teuthology.orchestra.run.smithi018.stdout:Complete! 2024-04-12T20:40:51.475 DEBUG:teuthology.orchestra.run.smithi186:> sudo yum -y install ceph-mgr-dashboard 2024-04-12T20:40:51.511 DEBUG:teuthology.orchestra.run.smithi018:> sudo yum -y install ceph-mgr-dashboard 2024-04-12T20:40:52.350 INFO:teuthology.orchestra.run.smithi186.stdout:Last metadata expiration check: 0:01:15 ago on Fri 12 Apr 2024 08:39:37 PM UTC. 2024-04-12T20:40:52.397 INFO:teuthology.orchestra.run.smithi018.stdout:Last metadata expiration check: 0:01:17 ago on Fri 12 Apr 2024 08:39:35 PM UTC. 2024-04-12T20:40:52.646 INFO:teuthology.orchestra.run.smithi186.stderr:Modular dependency problems: 2024-04-12T20:40:52.647 INFO:teuthology.orchestra.run.smithi186.stderr: 2024-04-12T20:40:52.647 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 1: conflicting requests 2024-04-12T20:40:52.647 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module container-tools:rhel8:820240321091303:20125149.x86_64 from CentOS-AppStream 2024-04-12T20:40:52.647 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 2: conflicting requests 2024-04-12T20:40:52.647 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module eclipse:rhel8:820201023100746:b230ed4e.x86_64 from CentOS-AppStream 2024-04-12T20:40:52.647 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 3: conflicting requests 2024-04-12T20:40:52.647 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module freeradius:3.0:8080020230117180605:89170a74.x86_64 from CentOS-AppStream 2024-04-12T20:40:52.647 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 4: conflicting requests 2024-04-12T20:40:52.647 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module gimp:2.8:8000020190628145146:4148dfdf.x86_64 from CentOS-AppStream 2024-04-12T20:40:52.647 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 5: conflicting requests 2024-04-12T20:40:52.648 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module go-toolset:rhel8:820240401160413:b754926a.x86_64 from CentOS-AppStream 2024-04-12T20:40:52.648 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 6: conflicting requests 2024-04-12T20:40:52.648 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module httpd:2.4:820240216084734:9edba152.x86_64 from CentOS-AppStream 2024-04-12T20:40:52.648 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 7: conflicting requests 2024-04-12T20:40:52.648 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module idm:client:820240307184541:49cc9d1b.x86_64 from CentOS-AppStream 2024-04-12T20:40:52.648 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 8: conflicting requests 2024-04-12T20:40:52.648 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module inkscape:0.92.3:8080020221205124429:3e031279.x86_64 from CentOS-AppStream 2024-04-12T20:40:52.648 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 9: conflicting requests 2024-04-12T20:40:52.648 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module jmc:rhel8:8050020211110222101:6392b1f8.x86_64 from CentOS-AppStream 2024-04-12T20:40:52.648 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 10: conflicting requests 2024-04-12T20:40:52.648 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module llvm-toolset:rhel8:820240205133135:9edba152.x86_64 from CentOS-AppStream 2024-04-12T20:40:52.649 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 11: conflicting requests 2024-04-12T20:40:52.649 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module log4j:2:8080020221205124743:9d367344.x86_64 from CentOS-AppStream 2024-04-12T20:40:52.649 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 12: conflicting requests 2024-04-12T20:40:52.649 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module mailman:2.1:820230727085757:77fc8825.x86_64 from CentOS-AppStream 2024-04-12T20:40:52.649 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 13: conflicting requests 2024-04-12T20:40:52.649 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module mariadb:10.3:8030020210419150013:30b713e6.x86_64 from CentOS-AppStream 2024-04-12T20:40:52.649 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 14: conflicting requests 2024-04-12T20:40:52.649 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module maven:3.5:8060020211203142015:c0229ad2.x86_64 from CentOS-AppStream 2024-04-12T20:40:52.649 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 15: conflicting requests 2024-04-12T20:40:52.649 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module mercurial:4.8:8000020190628020724:4148dfdf.x86_64 from CentOS-AppStream 2024-04-12T20:40:52.649 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 16: conflicting requests 2024-04-12T20:40:52.649 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module mod_auth_openidc:2.3:820230721123832:9edba152.x86_64 from CentOS-AppStream 2024-04-12T20:40:52.650 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 17: conflicting requests 2024-04-12T20:40:52.650 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module mysql:8.0:8040020210901180257:9f9e2e7e.x86_64 from CentOS-AppStream 2024-04-12T20:40:52.650 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 18: conflicting requests 2024-04-12T20:40:52.650 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module nginx:1.14:8000020211221191913:55190bc5.x86_64 from CentOS-AppStream 2024-04-12T20:40:52.650 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 19: conflicting requests 2024-04-12T20:40:52.650 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module nodejs:10:8040020210120182536:9f9e2e7e.x86_64 from CentOS-AppStream 2024-04-12T20:40:52.650 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 20: conflicting requests 2024-04-12T20:40:52.650 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl:5.26:8000020190628020724:55190bc5.x86_64 from CentOS-AppStream 2024-04-12T20:40:52.650 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 21: conflicting requests 2024-04-12T20:40:52.650 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module php:7.2:8020020200507003613:2c7ca891.x86_64 from CentOS-AppStream 2024-04-12T20:40:52.650 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 22: conflicting requests 2024-04-12T20:40:52.651 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module postgresql:10:820230223114800:9edba152.x86_64 from CentOS-AppStream 2024-04-12T20:40:52.651 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 23: conflicting requests 2024-04-12T20:40:52.651 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module python27:2.7:820240208011952:182f7c73.x86_64 from CentOS-AppStream 2024-04-12T20:40:52.651 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 24: conflicting requests 2024-04-12T20:40:52.651 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module python36:3.6:8050020210825152031:982725ab.x86_64 from CentOS-AppStream 2024-04-12T20:40:52.651 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 25: conflicting requests 2024-04-12T20:40:52.651 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module python38:3.8:820230810143931:64c1cd5a.x86_64 from CentOS-AppStream 2024-04-12T20:40:52.651 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 26: conflicting requests 2024-04-12T20:40:52.651 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module python39:3.9:820240214182535:17377f89.x86_64 from CentOS-AppStream 2024-04-12T20:40:52.651 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 27: conflicting requests 2024-04-12T20:40:52.651 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module redis:5:8040020211019153849:9f9e2e7e.x86_64 from CentOS-AppStream 2024-04-12T20:40:52.651 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 28: conflicting requests 2024-04-12T20:40:52.651 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module rhn-tools:1.0:8010020191114034948:f69d1239.x86_64 from CentOS-AppStream 2024-04-12T20:40:52.651 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 29: conflicting requests 2024-04-12T20:40:52.652 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module ruby:2.5:820230627084142:9edba152.x86_64 from CentOS-AppStream 2024-04-12T20:40:52.652 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 30: conflicting requests 2024-04-12T20:40:52.652 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module rust-toolset:rhel8:820240119204620:b09eea91.x86_64 from CentOS-AppStream 2024-04-12T20:40:52.652 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 31: conflicting requests 2024-04-12T20:40:52.652 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module satellite-5-client:1.0:8010020191114035551:cdc1202b.x86_64 from CentOS-AppStream 2024-04-12T20:40:52.652 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 32: conflicting requests 2024-04-12T20:40:52.652 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module squid:4:820240319173245:9edba152.x86_64 from CentOS-AppStream 2024-04-12T20:40:52.652 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 33: conflicting requests 2024-04-12T20:40:52.652 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module subversion:1.10:8070020220711155714:78111232.x86_64 from CentOS-AppStream 2024-04-12T20:40:52.652 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 34: conflicting requests 2024-04-12T20:40:52.652 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module swig:3.0:8030020200706142531:30b713e6.x86_64 from CentOS-AppStream 2024-04-12T20:40:52.653 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 35: conflicting requests 2024-04-12T20:40:52.653 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module varnish:6:8050020211109225449:b4937e53.x86_64 from CentOS-AppStream 2024-04-12T20:40:52.653 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 36: conflicting requests 2024-04-12T20:40:52.653 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module virt:rhel:820240314161907:9edba152.x86_64 from CentOS-AppStream 2024-04-12T20:40:52.653 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 37: conflicting requests 2024-04-12T20:40:52.653 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:8030020200716155257:7cc0a66d.x86_64 from CentOS-AppStream 2024-04-12T20:40:52.653 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:8030020200716155257:b967a9a2.x86_64 from CentOS-AppStream 2024-04-12T20:40:52.653 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:820230811133638:36f9fae6.x86_64 from CentOS-AppStream 2024-04-12T20:40:52.653 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 38: cannot install the best candidate for the job 2024-04-12T20:40:52.653 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:8010020191114031501:a5949e2e.x86_64 from CentOS-AppStream 2024-04-12T20:40:52.653 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:8010020191114031501:be2e4737.x86_64 from CentOS-AppStream 2024-04-12T20:40:52.653 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:820230223114759:2ab761e1.x86_64 from CentOS-AppStream 2024-04-12T20:40:52.653 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:820230223114759:86e45846.x86_64 from CentOS-AppStream 2024-04-12T20:40:52.653 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 39: conflicting requests 2024-04-12T20:40:52.654 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:8030020200715230104:1e4bbb35.x86_64 from CentOS-AppStream 2024-04-12T20:40:52.654 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:8030020200715230104:ea09926d.x86_64 from CentOS-AppStream 2024-04-12T20:40:52.654 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:082fdf2f.x86_64 from CentOS-AppStream 2024-04-12T20:40:52.654 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-04-12T20:40:52.654 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-04-12T20:40:52.654 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:fbe42456.x86_64 from CentOS-AppStream 2024-04-12T20:40:52.654 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 40: cannot install the best candidate for the job 2024-04-12T20:40:52.654 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:8010020191114031513:16b3ab4d.x86_64 from CentOS-AppStream 2024-04-12T20:40:52.654 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:8010020191114031513:e1f37755.x86_64 from CentOS-AppStream 2024-04-12T20:40:52.655 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-04-12T20:40:52.655 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-04-12T20:40:52.655 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 41: cannot install the best candidate for the job 2024-04-12T20:40:52.655 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:8030020200716150652:1e4bbb35.x86_64 from CentOS-AppStream 2024-04-12T20:40:52.655 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:8030020200716150652:ea09926d.x86_64 from CentOS-AppStream 2024-04-12T20:40:52.655 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-04-12T20:40:52.655 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-04-12T20:40:52.655 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 42: cannot install the best candidate for the job 2024-04-12T20:40:52.655 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:8030020200716174729:3a70019f.x86_64 from CentOS-AppStream 2024-04-12T20:40:52.656 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:8030020200716174729:cccafca5.x86_64 from CentOS-AppStream 2024-04-12T20:40:52.656 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:820230223114759:6fdf7e5d.x86_64 from CentOS-AppStream 2024-04-12T20:40:52.656 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:820230223114759:d4326aba.x86_64 from CentOS-AppStream 2024-04-12T20:40:52.656 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 43: cannot install the best candidate for the job 2024-04-12T20:40:52.656 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:8030020200716213634:56fce90f.x86_64 from CentOS-AppStream 2024-04-12T20:40:52.656 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:8030020200716213634:98a3c9d0.x86_64 from CentOS-AppStream 2024-04-12T20:40:52.656 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:820230223114759:06b7596e.x86_64 from CentOS-AppStream 2024-04-12T20:40:52.656 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:820230223114759:d2fc5d76.x86_64 from CentOS-AppStream 2024-04-12T20:40:52.656 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 44: cannot install the best candidate for the job 2024-04-12T20:40:52.656 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:8030020200716171139:3a70019f.x86_64 from CentOS-AppStream 2024-04-12T20:40:52.656 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:8030020200716171139:cccafca5.x86_64 from CentOS-AppStream 2024-04-12T20:40:52.656 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:820230223114759:6fdf7e5d.x86_64 from CentOS-AppStream 2024-04-12T20:40:52.657 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:820230223114759:d4326aba.x86_64 from CentOS-AppStream 2024-04-12T20:40:52.657 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 45: cannot install the best candidate for the job 2024-04-12T20:40:52.657 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:8010020191120175858:95fce66d.x86_64 from CentOS-AppStream 2024-04-12T20:40:52.657 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:8030020200723233049:09acf126.x86_64 from CentOS-AppStream 2024-04-12T20:40:52.657 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:820230223114759:436af1ae.x86_64 from CentOS-AppStream 2024-04-12T20:40:52.657 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:820230223114759:b12195ea.x86_64 from CentOS-AppStream 2024-04-12T20:40:52.708 INFO:teuthology.orchestra.run.smithi018.stderr:Modular dependency problems: 2024-04-12T20:40:52.708 INFO:teuthology.orchestra.run.smithi018.stderr: 2024-04-12T20:40:52.708 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 1: conflicting requests 2024-04-12T20:40:52.708 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module container-tools:rhel8:820240321091303:20125149.x86_64 from CentOS-AppStream 2024-04-12T20:40:52.708 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 2: conflicting requests 2024-04-12T20:40:52.708 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module eclipse:rhel8:820201023100746:b230ed4e.x86_64 from CentOS-AppStream 2024-04-12T20:40:52.709 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 3: conflicting requests 2024-04-12T20:40:52.709 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module freeradius:3.0:8080020230117180605:89170a74.x86_64 from CentOS-AppStream 2024-04-12T20:40:52.709 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 4: conflicting requests 2024-04-12T20:40:52.709 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module gimp:2.8:8000020190628145146:4148dfdf.x86_64 from CentOS-AppStream 2024-04-12T20:40:52.709 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 5: conflicting requests 2024-04-12T20:40:52.709 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module go-toolset:rhel8:820240401160413:b754926a.x86_64 from CentOS-AppStream 2024-04-12T20:40:52.709 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 6: conflicting requests 2024-04-12T20:40:52.709 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module httpd:2.4:820240216084734:9edba152.x86_64 from CentOS-AppStream 2024-04-12T20:40:52.710 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 7: conflicting requests 2024-04-12T20:40:52.710 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module idm:client:820240307184541:49cc9d1b.x86_64 from CentOS-AppStream 2024-04-12T20:40:52.710 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 8: conflicting requests 2024-04-12T20:40:52.710 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module inkscape:0.92.3:8080020221205124429:3e031279.x86_64 from CentOS-AppStream 2024-04-12T20:40:52.710 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 9: conflicting requests 2024-04-12T20:40:52.710 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module jmc:rhel8:8050020211110222101:6392b1f8.x86_64 from CentOS-AppStream 2024-04-12T20:40:52.710 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 10: conflicting requests 2024-04-12T20:40:52.710 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module llvm-toolset:rhel8:820240205133135:9edba152.x86_64 from CentOS-AppStream 2024-04-12T20:40:52.711 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 11: conflicting requests 2024-04-12T20:40:52.711 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module log4j:2:8080020221205124743:9d367344.x86_64 from CentOS-AppStream 2024-04-12T20:40:52.711 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 12: conflicting requests 2024-04-12T20:40:52.711 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module mailman:2.1:820230727085757:77fc8825.x86_64 from CentOS-AppStream 2024-04-12T20:40:52.711 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 13: conflicting requests 2024-04-12T20:40:52.711 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module mariadb:10.3:8030020210419150013:30b713e6.x86_64 from CentOS-AppStream 2024-04-12T20:40:52.711 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 14: conflicting requests 2024-04-12T20:40:52.711 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module maven:3.5:8060020211203142015:c0229ad2.x86_64 from CentOS-AppStream 2024-04-12T20:40:52.711 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 15: conflicting requests 2024-04-12T20:40:52.712 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module mercurial:4.8:8000020190628020724:4148dfdf.x86_64 from CentOS-AppStream 2024-04-12T20:40:52.712 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 16: conflicting requests 2024-04-12T20:40:52.712 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module mod_auth_openidc:2.3:820230721123832:9edba152.x86_64 from CentOS-AppStream 2024-04-12T20:40:52.712 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 17: conflicting requests 2024-04-12T20:40:52.712 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module mysql:8.0:8040020210901180257:9f9e2e7e.x86_64 from CentOS-AppStream 2024-04-12T20:40:52.712 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 18: conflicting requests 2024-04-12T20:40:52.712 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module nginx:1.14:8000020211221191913:55190bc5.x86_64 from CentOS-AppStream 2024-04-12T20:40:52.713 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 19: conflicting requests 2024-04-12T20:40:52.713 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module nodejs:10:8040020210120182536:9f9e2e7e.x86_64 from CentOS-AppStream 2024-04-12T20:40:52.713 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 20: conflicting requests 2024-04-12T20:40:52.713 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl:5.26:8000020190628020724:55190bc5.x86_64 from CentOS-AppStream 2024-04-12T20:40:52.713 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 21: conflicting requests 2024-04-12T20:40:52.713 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module php:7.2:8020020200507003613:2c7ca891.x86_64 from CentOS-AppStream 2024-04-12T20:40:52.713 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 22: conflicting requests 2024-04-12T20:40:52.714 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module postgresql:10:820230223114800:9edba152.x86_64 from CentOS-AppStream 2024-04-12T20:40:52.714 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 23: conflicting requests 2024-04-12T20:40:52.714 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module python27:2.7:820240208011952:182f7c73.x86_64 from CentOS-AppStream 2024-04-12T20:40:52.714 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 24: conflicting requests 2024-04-12T20:40:52.714 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module python36:3.6:8050020210825152031:982725ab.x86_64 from CentOS-AppStream 2024-04-12T20:40:52.714 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 25: conflicting requests 2024-04-12T20:40:52.715 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module python38:3.8:820230810143931:64c1cd5a.x86_64 from CentOS-AppStream 2024-04-12T20:40:52.715 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 26: conflicting requests 2024-04-12T20:40:52.715 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module python39:3.9:820240214182535:17377f89.x86_64 from CentOS-AppStream 2024-04-12T20:40:52.715 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 27: conflicting requests 2024-04-12T20:40:52.715 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module redis:5:8040020211019153849:9f9e2e7e.x86_64 from CentOS-AppStream 2024-04-12T20:40:52.715 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 28: conflicting requests 2024-04-12T20:40:52.716 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module rhn-tools:1.0:8010020191114034948:f69d1239.x86_64 from CentOS-AppStream 2024-04-12T20:40:52.716 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 29: conflicting requests 2024-04-12T20:40:52.716 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module ruby:2.5:820230627084142:9edba152.x86_64 from CentOS-AppStream 2024-04-12T20:40:52.716 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 30: conflicting requests 2024-04-12T20:40:52.716 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module rust-toolset:rhel8:820240119204620:b09eea91.x86_64 from CentOS-AppStream 2024-04-12T20:40:52.716 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 31: conflicting requests 2024-04-12T20:40:52.716 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module satellite-5-client:1.0:8010020191114035551:cdc1202b.x86_64 from CentOS-AppStream 2024-04-12T20:40:52.717 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 32: conflicting requests 2024-04-12T20:40:52.717 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module squid:4:820240319173245:9edba152.x86_64 from CentOS-AppStream 2024-04-12T20:40:52.717 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 33: conflicting requests 2024-04-12T20:40:52.717 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module subversion:1.10:8070020220711155714:78111232.x86_64 from CentOS-AppStream 2024-04-12T20:40:52.717 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 34: conflicting requests 2024-04-12T20:40:52.717 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module swig:3.0:8030020200706142531:30b713e6.x86_64 from CentOS-AppStream 2024-04-12T20:40:52.717 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 35: conflicting requests 2024-04-12T20:40:52.717 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module varnish:6:8050020211109225449:b4937e53.x86_64 from CentOS-AppStream 2024-04-12T20:40:52.717 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 36: conflicting requests 2024-04-12T20:40:52.717 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module virt:rhel:820240314161907:9edba152.x86_64 from CentOS-AppStream 2024-04-12T20:40:52.717 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 37: conflicting requests 2024-04-12T20:40:52.718 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:8030020200716155257:7cc0a66d.x86_64 from CentOS-AppStream 2024-04-12T20:40:52.718 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:8030020200716155257:b967a9a2.x86_64 from CentOS-AppStream 2024-04-12T20:40:52.718 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:820230811133638:36f9fae6.x86_64 from CentOS-AppStream 2024-04-12T20:40:52.718 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 38: cannot install the best candidate for the job 2024-04-12T20:40:52.718 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:8010020191114031501:a5949e2e.x86_64 from CentOS-AppStream 2024-04-12T20:40:52.718 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:8010020191114031501:be2e4737.x86_64 from CentOS-AppStream 2024-04-12T20:40:52.718 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:820230223114759:2ab761e1.x86_64 from CentOS-AppStream 2024-04-12T20:40:52.718 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:820230223114759:86e45846.x86_64 from CentOS-AppStream 2024-04-12T20:40:52.718 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 39: conflicting requests 2024-04-12T20:40:52.718 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:8030020200715230104:1e4bbb35.x86_64 from CentOS-AppStream 2024-04-12T20:40:52.718 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:8030020200715230104:ea09926d.x86_64 from CentOS-AppStream 2024-04-12T20:40:52.718 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:082fdf2f.x86_64 from CentOS-AppStream 2024-04-12T20:40:52.718 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-04-12T20:40:52.718 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-04-12T20:40:52.719 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:fbe42456.x86_64 from CentOS-AppStream 2024-04-12T20:40:52.719 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 40: cannot install the best candidate for the job 2024-04-12T20:40:52.719 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:8010020191114031513:16b3ab4d.x86_64 from CentOS-AppStream 2024-04-12T20:40:52.719 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:8010020191114031513:e1f37755.x86_64 from CentOS-AppStream 2024-04-12T20:40:52.719 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-04-12T20:40:52.719 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-04-12T20:40:52.719 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 41: cannot install the best candidate for the job 2024-04-12T20:40:52.719 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:8030020200716150652:1e4bbb35.x86_64 from CentOS-AppStream 2024-04-12T20:40:52.719 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:8030020200716150652:ea09926d.x86_64 from CentOS-AppStream 2024-04-12T20:40:52.719 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-04-12T20:40:52.719 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-04-12T20:40:52.719 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 42: cannot install the best candidate for the job 2024-04-12T20:40:52.719 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:8030020200716174729:3a70019f.x86_64 from CentOS-AppStream 2024-04-12T20:40:52.719 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:8030020200716174729:cccafca5.x86_64 from CentOS-AppStream 2024-04-12T20:40:52.720 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:820230223114759:6fdf7e5d.x86_64 from CentOS-AppStream 2024-04-12T20:40:52.720 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:820230223114759:d4326aba.x86_64 from CentOS-AppStream 2024-04-12T20:40:52.720 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 43: cannot install the best candidate for the job 2024-04-12T20:40:52.720 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:8030020200716213634:56fce90f.x86_64 from CentOS-AppStream 2024-04-12T20:40:52.720 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:8030020200716213634:98a3c9d0.x86_64 from CentOS-AppStream 2024-04-12T20:40:52.720 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:820230223114759:06b7596e.x86_64 from CentOS-AppStream 2024-04-12T20:40:52.720 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:820230223114759:d2fc5d76.x86_64 from CentOS-AppStream 2024-04-12T20:40:52.720 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 44: cannot install the best candidate for the job 2024-04-12T20:40:52.720 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:8030020200716171139:3a70019f.x86_64 from CentOS-AppStream 2024-04-12T20:40:52.720 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:8030020200716171139:cccafca5.x86_64 from CentOS-AppStream 2024-04-12T20:40:52.720 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:820230223114759:6fdf7e5d.x86_64 from CentOS-AppStream 2024-04-12T20:40:52.720 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:820230223114759:d4326aba.x86_64 from CentOS-AppStream 2024-04-12T20:40:52.720 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 45: cannot install the best candidate for the job 2024-04-12T20:40:52.720 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:8010020191120175858:95fce66d.x86_64 from CentOS-AppStream 2024-04-12T20:40:52.721 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:8030020200723233049:09acf126.x86_64 from CentOS-AppStream 2024-04-12T20:40:52.721 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:820230223114759:436af1ae.x86_64 from CentOS-AppStream 2024-04-12T20:40:52.721 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:820230223114759:b12195ea.x86_64 from CentOS-AppStream 2024-04-12T20:40:52.722 INFO:teuthology.orchestra.run.smithi186.stdout:Dependencies resolved. 2024-04-12T20:40:52.722 INFO:teuthology.orchestra.run.smithi186.stdout:================================================================================ 2024-04-12T20:40:52.722 INFO:teuthology.orchestra.run.smithi186.stdout: Package Arch Version Repository Size 2024-04-12T20:40:52.722 INFO:teuthology.orchestra.run.smithi186.stdout:================================================================================ 2024-04-12T20:40:52.722 INFO:teuthology.orchestra.run.smithi186.stdout:Installing: 2024-04-12T20:40:52.722 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-mgr-dashboard noarch 2:17.2.7-827.gd3b5060a.el9 ceph-noarch 1.7 M 2024-04-12T20:40:52.722 INFO:teuthology.orchestra.run.smithi186.stdout:Installing dependencies: 2024-04-12T20:40:52.722 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-grafana-dashboards noarch 2:17.2.7-827.gd3b5060a.el9 ceph-noarch 24 k 2024-04-12T20:40:52.722 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-prometheus-alerts noarch 2:17.2.7-827.gd3b5060a.el9 ceph-noarch 15 k 2024-04-12T20:40:52.722 INFO:teuthology.orchestra.run.smithi186.stdout: python3-jwt noarch 2.4.0-1.el9 epel 41 k 2024-04-12T20:40:52.722 INFO:teuthology.orchestra.run.smithi186.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-04-12T20:40:52.723 INFO:teuthology.orchestra.run.smithi186.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-04-12T20:40:52.723 INFO:teuthology.orchestra.run.smithi186.stdout:Installing weak dependencies: 2024-04-12T20:40:52.723 INFO:teuthology.orchestra.run.smithi186.stdout: python3-jwt+crypto noarch 2.4.0-1.el9 epel 9.0 k 2024-04-12T20:40:52.723 INFO:teuthology.orchestra.run.smithi186.stdout: 2024-04-12T20:40:52.723 INFO:teuthology.orchestra.run.smithi186.stdout:Transaction Summary 2024-04-12T20:40:52.723 INFO:teuthology.orchestra.run.smithi186.stdout:================================================================================ 2024-04-12T20:40:52.723 INFO:teuthology.orchestra.run.smithi186.stdout:Install 7 Packages 2024-04-12T20:40:52.723 INFO:teuthology.orchestra.run.smithi186.stdout: 2024-04-12T20:40:52.723 INFO:teuthology.orchestra.run.smithi186.stdout:Total download size: 2.0 M 2024-04-12T20:40:52.723 INFO:teuthology.orchestra.run.smithi186.stdout:Installed size: 8.1 M 2024-04-12T20:40:52.723 INFO:teuthology.orchestra.run.smithi186.stdout:Downloading Packages: 2024-04-12T20:40:52.782 INFO:teuthology.orchestra.run.smithi018.stdout:Dependencies resolved. 2024-04-12T20:40:52.783 INFO:teuthology.orchestra.run.smithi018.stdout:================================================================================ 2024-04-12T20:40:52.783 INFO:teuthology.orchestra.run.smithi018.stdout: Package Arch Version Repository Size 2024-04-12T20:40:52.783 INFO:teuthology.orchestra.run.smithi018.stdout:================================================================================ 2024-04-12T20:40:52.783 INFO:teuthology.orchestra.run.smithi018.stdout:Installing: 2024-04-12T20:40:52.783 INFO:teuthology.orchestra.run.smithi018.stdout: ceph-mgr-dashboard noarch 2:17.2.7-827.gd3b5060a.el9 ceph-noarch 1.7 M 2024-04-12T20:40:52.783 INFO:teuthology.orchestra.run.smithi018.stdout:Installing dependencies: 2024-04-12T20:40:52.784 INFO:teuthology.orchestra.run.smithi018.stdout: ceph-grafana-dashboards noarch 2:17.2.7-827.gd3b5060a.el9 ceph-noarch 24 k 2024-04-12T20:40:52.784 INFO:teuthology.orchestra.run.smithi018.stdout: ceph-prometheus-alerts noarch 2:17.2.7-827.gd3b5060a.el9 ceph-noarch 15 k 2024-04-12T20:40:52.784 INFO:teuthology.orchestra.run.smithi018.stdout: python3-jwt noarch 2.4.0-1.el9 epel 41 k 2024-04-12T20:40:52.784 INFO:teuthology.orchestra.run.smithi018.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-04-12T20:40:52.784 INFO:teuthology.orchestra.run.smithi018.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-04-12T20:40:52.784 INFO:teuthology.orchestra.run.smithi018.stdout:Installing weak dependencies: 2024-04-12T20:40:52.784 INFO:teuthology.orchestra.run.smithi018.stdout: python3-jwt+crypto noarch 2.4.0-1.el9 epel 9.0 k 2024-04-12T20:40:52.784 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-12T20:40:52.784 INFO:teuthology.orchestra.run.smithi018.stdout:Transaction Summary 2024-04-12T20:40:52.784 INFO:teuthology.orchestra.run.smithi018.stdout:================================================================================ 2024-04-12T20:40:52.784 INFO:teuthology.orchestra.run.smithi018.stdout:Install 7 Packages 2024-04-12T20:40:52.784 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-12T20:40:52.785 INFO:teuthology.orchestra.run.smithi018.stdout:Total download size: 2.0 M 2024-04-12T20:40:52.785 INFO:teuthology.orchestra.run.smithi018.stdout:Installed size: 8.1 M 2024-04-12T20:40:52.785 INFO:teuthology.orchestra.run.smithi018.stdout:Downloading Packages: 2024-04-12T20:40:53.406 INFO:teuthology.orchestra.run.smithi186.stdout:(1/7): ceph-grafana-dashboards-17.2.7-827.gd3b5 75 kB/s | 24 kB 00:00 2024-04-12T20:40:53.424 INFO:teuthology.orchestra.run.smithi018.stdout:(1/7): ceph-grafana-dashboards-17.2.7-827.gd3b5 56 kB/s | 24 kB 00:00 2024-04-12T20:40:53.448 INFO:teuthology.orchestra.run.smithi186.stdout:(2/7): ceph-prometheus-alerts-17.2.7-827.gd3b50 40 kB/s | 15 kB 00:00 2024-04-12T20:40:53.474 INFO:teuthology.orchestra.run.smithi018.stdout:(2/7): ceph-prometheus-alerts-17.2.7-827.gd3b50 30 kB/s | 15 kB 00:00 2024-04-12T20:40:53.707 INFO:teuthology.orchestra.run.smithi186.stdout:(3/7): python3-jwt+crypto-2.4.0-1.el9.noarch.rp 30 kB/s | 9.0 kB 00:00 2024-04-12T20:40:53.748 INFO:teuthology.orchestra.run.smithi186.stdout:(4/7): python3-jwt-2.4.0-1.el9.noarch.rpm 137 kB/s | 41 kB 00:00 2024-04-12T20:40:53.766 INFO:teuthology.orchestra.run.smithi018.stdout:(3/7): python3-jwt+crypto-2.4.0-1.el9.noarch.rp 27 kB/s | 9.0 kB 00:00 2024-04-12T20:40:53.791 INFO:teuthology.orchestra.run.smithi186.stdout:(5/7): python3-repoze-lru-0.7-16.el9.noarch.rpm 371 kB/s | 31 kB 00:00 2024-04-12T20:40:53.883 INFO:teuthology.orchestra.run.smithi186.stdout:(6/7): python3-routes-2.5.1-5.el9.noarch.rpm 1.4 MB/s | 188 kB 00:00 2024-04-12T20:40:53.933 INFO:teuthology.orchestra.run.smithi018.stdout:(4/7): python3-repoze-lru-0.7-16.el9.noarch.rpm 185 kB/s | 31 kB 00:00 2024-04-12T20:40:53.959 INFO:teuthology.orchestra.run.smithi018.stdout:(5/7): python3-jwt-2.4.0-1.el9.noarch.rpm 85 kB/s | 41 kB 00:00 2024-04-12T20:40:54.149 INFO:teuthology.orchestra.run.smithi186.stdout:(7/7): ceph-mgr-dashboard-17.2.7-827.gd3b5060a. 1.6 MB/s | 1.7 MB 00:01 2024-04-12T20:40:54.159 INFO:teuthology.orchestra.run.smithi186.stdout:-------------------------------------------------------------------------------- 2024-04-12T20:40:54.159 INFO:teuthology.orchestra.run.smithi186.stdout:Total 1.4 MB/s | 2.0 MB 00:01 2024-04-12T20:40:54.159 INFO:teuthology.orchestra.run.smithi186.stdout:Running transaction check 2024-04-12T20:40:54.175 INFO:teuthology.orchestra.run.smithi018.stdout:(6/7): ceph-mgr-dashboard-17.2.7-827.gd3b5060a. 1.4 MB/s | 1.7 MB 00:01 2024-04-12T20:40:54.187 INFO:teuthology.orchestra.run.smithi186.stdout:Transaction check succeeded. 2024-04-12T20:40:54.187 INFO:teuthology.orchestra.run.smithi186.stdout:Running transaction test 2024-04-12T20:40:54.242 INFO:teuthology.orchestra.run.smithi018.stdout:(7/7): python3-routes-2.5.1-5.el9.noarch.rpm 612 kB/s | 188 kB 00:00 2024-04-12T20:40:54.251 INFO:teuthology.orchestra.run.smithi018.stdout:-------------------------------------------------------------------------------- 2024-04-12T20:40:54.252 INFO:teuthology.orchestra.run.smithi018.stdout:Total 1.4 MB/s | 2.0 MB 00:01 2024-04-12T20:40:54.252 INFO:teuthology.orchestra.run.smithi018.stdout:Running transaction check 2024-04-12T20:40:54.282 INFO:teuthology.orchestra.run.smithi018.stdout:Transaction check succeeded. 2024-04-12T20:40:54.282 INFO:teuthology.orchestra.run.smithi018.stdout:Running transaction test 2024-04-12T20:40:54.311 INFO:teuthology.orchestra.run.smithi186.stdout:Transaction test succeeded. 2024-04-12T20:40:54.311 INFO:teuthology.orchestra.run.smithi186.stdout:Running transaction 2024-04-12T20:40:54.420 INFO:teuthology.orchestra.run.smithi018.stdout:Transaction test succeeded. 2024-04-12T20:40:54.421 INFO:teuthology.orchestra.run.smithi018.stdout:Running transaction 2024-04-12T20:40:54.782 INFO:teuthology.orchestra.run.smithi186.stdout: Preparing : 1/1 2024-04-12T20:40:54.863 INFO:teuthology.orchestra.run.smithi018.stdout: Preparing : 1/1 2024-04-12T20:40:54.893 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-jwt-2.4.0-1.el9.noarch 1/7 2024-04-12T20:40:54.950 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-jwt+crypto-2.4.0-1.el9.noarch 2/7 2024-04-12T20:40:54.952 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : python3-jwt-2.4.0-1.el9.noarch 1/7 2024-04-12T20:40:55.004 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : python3-jwt+crypto-2.4.0-1.el9.noarch 2/7 2024-04-12T20:40:55.059 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 3/7 2024-04-12T20:40:55.101 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 4/7 2024-04-12T20:40:55.532 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : ceph-prometheus-alerts-2:17.2.7-827.gd3b5060a.el9.no 5/7 2024-04-12T20:40:55.534 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : ceph-grafana-dashboards-2:17.2.7-827.gd3b5060a.el9.n 6/7 2024-04-12T20:40:55.534 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : ceph-mgr-dashboard-2:17.2.7-827.gd3b5060a.el9.noarch 7/7 2024-04-12T20:40:55.534 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 3/7 2024-04-12T20:40:55.534 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 4/7 2024-04-12T20:40:55.535 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : ceph-prometheus-alerts-2:17.2.7-827.gd3b5060a.el9.no 5/7 2024-04-12T20:40:55.535 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : ceph-grafana-dashboards-2:17.2.7-827.gd3b5060a.el9.n 6/7 2024-04-12T20:40:55.556 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : ceph-mgr-dashboard-2:17.2.7-827.gd3b5060a.el9.noarch 7/7 2024-04-12T20:40:56.145 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: ceph-mgr-dashboard-2:17.2.7-827.gd3b5060a.el9.noarch 7/7 2024-04-12T20:40:56.145 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : ceph-grafana-dashboards-2:17.2.7-827.gd3b5060a.el9.n 1/7 2024-04-12T20:40:56.145 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : ceph-mgr-dashboard-2:17.2.7-827.gd3b5060a.el9.noarch 2/7 2024-04-12T20:40:56.145 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : ceph-prometheus-alerts-2:17.2.7-827.gd3b5060a.el9.no 3/7 2024-04-12T20:40:56.145 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-jwt+crypto-2.4.0-1.el9.noarch 4/7 2024-04-12T20:40:56.145 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-jwt-2.4.0-1.el9.noarch 5/7 2024-04-12T20:40:56.145 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 6/7 2024-04-12T20:40:56.299 INFO:teuthology.orchestra.run.smithi018.stdout: Running scriptlet: ceph-mgr-dashboard-2:17.2.7-827.gd3b5060a.el9.noarch 7/7 2024-04-12T20:40:56.299 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : ceph-grafana-dashboards-2:17.2.7-827.gd3b5060a.el9.n 1/7 2024-04-12T20:40:56.299 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : ceph-mgr-dashboard-2:17.2.7-827.gd3b5060a.el9.noarch 2/7 2024-04-12T20:40:56.299 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : ceph-prometheus-alerts-2:17.2.7-827.gd3b5060a.el9.no 3/7 2024-04-12T20:40:56.299 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : python3-jwt+crypto-2.4.0-1.el9.noarch 4/7 2024-04-12T20:40:56.299 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : python3-jwt-2.4.0-1.el9.noarch 5/7 2024-04-12T20:40:56.300 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 6/7 2024-04-12T20:40:56.572 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 7/7 2024-04-12T20:40:56.572 INFO:teuthology.orchestra.run.smithi186.stdout: 2024-04-12T20:40:56.572 INFO:teuthology.orchestra.run.smithi186.stdout:Installed: 2024-04-12T20:40:56.572 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-grafana-dashboards-2:17.2.7-827.gd3b5060a.el9.noarch 2024-04-12T20:40:56.572 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-mgr-dashboard-2:17.2.7-827.gd3b5060a.el9.noarch 2024-04-12T20:40:56.573 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-prometheus-alerts-2:17.2.7-827.gd3b5060a.el9.noarch 2024-04-12T20:40:56.573 INFO:teuthology.orchestra.run.smithi186.stdout: python3-jwt-2.4.0-1.el9.noarch 2024-04-12T20:40:56.573 INFO:teuthology.orchestra.run.smithi186.stdout: python3-jwt+crypto-2.4.0-1.el9.noarch 2024-04-12T20:40:56.573 INFO:teuthology.orchestra.run.smithi186.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-04-12T20:40:56.573 INFO:teuthology.orchestra.run.smithi186.stdout: python3-routes-2.5.1-5.el9.noarch 2024-04-12T20:40:56.573 INFO:teuthology.orchestra.run.smithi186.stdout: 2024-04-12T20:40:56.573 INFO:teuthology.orchestra.run.smithi186.stdout:Complete! 2024-04-12T20:40:56.729 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 7/7 2024-04-12T20:40:56.729 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-12T20:40:56.730 INFO:teuthology.orchestra.run.smithi018.stdout:Installed: 2024-04-12T20:40:56.730 INFO:teuthology.orchestra.run.smithi018.stdout: ceph-grafana-dashboards-2:17.2.7-827.gd3b5060a.el9.noarch 2024-04-12T20:40:56.730 INFO:teuthology.orchestra.run.smithi018.stdout: ceph-mgr-dashboard-2:17.2.7-827.gd3b5060a.el9.noarch 2024-04-12T20:40:56.730 INFO:teuthology.orchestra.run.smithi018.stdout: ceph-prometheus-alerts-2:17.2.7-827.gd3b5060a.el9.noarch 2024-04-12T20:40:56.730 INFO:teuthology.orchestra.run.smithi018.stdout: python3-jwt-2.4.0-1.el9.noarch 2024-04-12T20:40:56.730 INFO:teuthology.orchestra.run.smithi018.stdout: python3-jwt+crypto-2.4.0-1.el9.noarch 2024-04-12T20:40:56.730 INFO:teuthology.orchestra.run.smithi018.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-04-12T20:40:56.730 INFO:teuthology.orchestra.run.smithi018.stdout: python3-routes-2.5.1-5.el9.noarch 2024-04-12T20:40:56.730 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-12T20:40:56.730 INFO:teuthology.orchestra.run.smithi018.stdout:Complete! 2024-04-12T20:40:56.862 DEBUG:teuthology.orchestra.run.smithi186:> sudo yum -y install ceph-mgr-diskprediction-local 2024-04-12T20:40:57.055 DEBUG:teuthology.orchestra.run.smithi018:> sudo yum -y install ceph-mgr-diskprediction-local 2024-04-12T20:40:57.763 INFO:teuthology.orchestra.run.smithi186.stdout:Last metadata expiration check: 0:01:20 ago on Fri 12 Apr 2024 08:39:37 PM UTC. 2024-04-12T20:40:57.947 INFO:teuthology.orchestra.run.smithi018.stdout:Last metadata expiration check: 0:01:22 ago on Fri 12 Apr 2024 08:39:35 PM UTC. 2024-04-12T20:40:58.072 INFO:teuthology.orchestra.run.smithi186.stderr:Modular dependency problems: 2024-04-12T20:40:58.072 INFO:teuthology.orchestra.run.smithi186.stderr: 2024-04-12T20:40:58.073 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 1: conflicting requests 2024-04-12T20:40:58.073 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module container-tools:rhel8:820240321091303:20125149.x86_64 from CentOS-AppStream 2024-04-12T20:40:58.073 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 2: conflicting requests 2024-04-12T20:40:58.073 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module eclipse:rhel8:820201023100746:b230ed4e.x86_64 from CentOS-AppStream 2024-04-12T20:40:58.073 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 3: conflicting requests 2024-04-12T20:40:58.074 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module freeradius:3.0:8080020230117180605:89170a74.x86_64 from CentOS-AppStream 2024-04-12T20:40:58.074 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 4: conflicting requests 2024-04-12T20:40:58.074 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module gimp:2.8:8000020190628145146:4148dfdf.x86_64 from CentOS-AppStream 2024-04-12T20:40:58.074 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 5: conflicting requests 2024-04-12T20:40:58.075 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module go-toolset:rhel8:820240401160413:b754926a.x86_64 from CentOS-AppStream 2024-04-12T20:40:58.075 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 6: conflicting requests 2024-04-12T20:40:58.075 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module httpd:2.4:820240216084734:9edba152.x86_64 from CentOS-AppStream 2024-04-12T20:40:58.076 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 7: conflicting requests 2024-04-12T20:40:58.076 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module idm:client:820240307184541:49cc9d1b.x86_64 from CentOS-AppStream 2024-04-12T20:40:58.076 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 8: conflicting requests 2024-04-12T20:40:58.077 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module inkscape:0.92.3:8080020221205124429:3e031279.x86_64 from CentOS-AppStream 2024-04-12T20:40:58.077 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 9: conflicting requests 2024-04-12T20:40:58.077 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module jmc:rhel8:8050020211110222101:6392b1f8.x86_64 from CentOS-AppStream 2024-04-12T20:40:58.077 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 10: conflicting requests 2024-04-12T20:40:58.077 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module llvm-toolset:rhel8:820240205133135:9edba152.x86_64 from CentOS-AppStream 2024-04-12T20:40:58.078 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 11: conflicting requests 2024-04-12T20:40:58.078 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module log4j:2:8080020221205124743:9d367344.x86_64 from CentOS-AppStream 2024-04-12T20:40:58.078 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 12: conflicting requests 2024-04-12T20:40:58.078 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module mailman:2.1:820230727085757:77fc8825.x86_64 from CentOS-AppStream 2024-04-12T20:40:58.078 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 13: conflicting requests 2024-04-12T20:40:58.079 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module mariadb:10.3:8030020210419150013:30b713e6.x86_64 from CentOS-AppStream 2024-04-12T20:40:58.079 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 14: conflicting requests 2024-04-12T20:40:58.079 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module maven:3.5:8060020211203142015:c0229ad2.x86_64 from CentOS-AppStream 2024-04-12T20:40:58.079 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 15: conflicting requests 2024-04-12T20:40:58.079 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module mercurial:4.8:8000020190628020724:4148dfdf.x86_64 from CentOS-AppStream 2024-04-12T20:40:58.079 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 16: conflicting requests 2024-04-12T20:40:58.079 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module mod_auth_openidc:2.3:820230721123832:9edba152.x86_64 from CentOS-AppStream 2024-04-12T20:40:58.079 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 17: conflicting requests 2024-04-12T20:40:58.079 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module mysql:8.0:8040020210901180257:9f9e2e7e.x86_64 from CentOS-AppStream 2024-04-12T20:40:58.079 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 18: conflicting requests 2024-04-12T20:40:58.079 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module nginx:1.14:8000020211221191913:55190bc5.x86_64 from CentOS-AppStream 2024-04-12T20:40:58.079 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 19: conflicting requests 2024-04-12T20:40:58.080 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module nodejs:10:8040020210120182536:9f9e2e7e.x86_64 from CentOS-AppStream 2024-04-12T20:40:58.080 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 20: conflicting requests 2024-04-12T20:40:58.080 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl:5.26:8000020190628020724:55190bc5.x86_64 from CentOS-AppStream 2024-04-12T20:40:58.080 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 21: conflicting requests 2024-04-12T20:40:58.080 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module php:7.2:8020020200507003613:2c7ca891.x86_64 from CentOS-AppStream 2024-04-12T20:40:58.080 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 22: conflicting requests 2024-04-12T20:40:58.080 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module postgresql:10:820230223114800:9edba152.x86_64 from CentOS-AppStream 2024-04-12T20:40:58.080 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 23: conflicting requests 2024-04-12T20:40:58.080 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module python27:2.7:820240208011952:182f7c73.x86_64 from CentOS-AppStream 2024-04-12T20:40:58.080 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 24: conflicting requests 2024-04-12T20:40:58.080 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module python36:3.6:8050020210825152031:982725ab.x86_64 from CentOS-AppStream 2024-04-12T20:40:58.080 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 25: conflicting requests 2024-04-12T20:40:58.080 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module python38:3.8:820230810143931:64c1cd5a.x86_64 from CentOS-AppStream 2024-04-12T20:40:58.081 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 26: conflicting requests 2024-04-12T20:40:58.081 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module python39:3.9:820240214182535:17377f89.x86_64 from CentOS-AppStream 2024-04-12T20:40:58.081 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 27: conflicting requests 2024-04-12T20:40:58.081 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module redis:5:8040020211019153849:9f9e2e7e.x86_64 from CentOS-AppStream 2024-04-12T20:40:58.081 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 28: conflicting requests 2024-04-12T20:40:58.081 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module rhn-tools:1.0:8010020191114034948:f69d1239.x86_64 from CentOS-AppStream 2024-04-12T20:40:58.081 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 29: conflicting requests 2024-04-12T20:40:58.081 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module ruby:2.5:820230627084142:9edba152.x86_64 from CentOS-AppStream 2024-04-12T20:40:58.081 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 30: conflicting requests 2024-04-12T20:40:58.081 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module rust-toolset:rhel8:820240119204620:b09eea91.x86_64 from CentOS-AppStream 2024-04-12T20:40:58.081 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 31: conflicting requests 2024-04-12T20:40:58.081 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module satellite-5-client:1.0:8010020191114035551:cdc1202b.x86_64 from CentOS-AppStream 2024-04-12T20:40:58.081 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 32: conflicting requests 2024-04-12T20:40:58.082 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module squid:4:820240319173245:9edba152.x86_64 from CentOS-AppStream 2024-04-12T20:40:58.082 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 33: conflicting requests 2024-04-12T20:40:58.082 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module subversion:1.10:8070020220711155714:78111232.x86_64 from CentOS-AppStream 2024-04-12T20:40:58.082 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 34: conflicting requests 2024-04-12T20:40:58.082 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module swig:3.0:8030020200706142531:30b713e6.x86_64 from CentOS-AppStream 2024-04-12T20:40:58.082 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 35: conflicting requests 2024-04-12T20:40:58.082 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module varnish:6:8050020211109225449:b4937e53.x86_64 from CentOS-AppStream 2024-04-12T20:40:58.082 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 36: conflicting requests 2024-04-12T20:40:58.082 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module virt:rhel:820240314161907:9edba152.x86_64 from CentOS-AppStream 2024-04-12T20:40:58.082 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 37: conflicting requests 2024-04-12T20:40:58.082 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:8030020200716155257:7cc0a66d.x86_64 from CentOS-AppStream 2024-04-12T20:40:58.082 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:8030020200716155257:b967a9a2.x86_64 from CentOS-AppStream 2024-04-12T20:40:58.083 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:820230811133638:36f9fae6.x86_64 from CentOS-AppStream 2024-04-12T20:40:58.083 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 38: cannot install the best candidate for the job 2024-04-12T20:40:58.083 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:8010020191114031501:a5949e2e.x86_64 from CentOS-AppStream 2024-04-12T20:40:58.083 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:8010020191114031501:be2e4737.x86_64 from CentOS-AppStream 2024-04-12T20:40:58.083 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:820230223114759:2ab761e1.x86_64 from CentOS-AppStream 2024-04-12T20:40:58.083 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:820230223114759:86e45846.x86_64 from CentOS-AppStream 2024-04-12T20:40:58.083 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 39: conflicting requests 2024-04-12T20:40:58.083 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:8030020200715230104:1e4bbb35.x86_64 from CentOS-AppStream 2024-04-12T20:40:58.083 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:8030020200715230104:ea09926d.x86_64 from CentOS-AppStream 2024-04-12T20:40:58.083 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:082fdf2f.x86_64 from CentOS-AppStream 2024-04-12T20:40:58.083 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-04-12T20:40:58.083 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-04-12T20:40:58.083 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:fbe42456.x86_64 from CentOS-AppStream 2024-04-12T20:40:58.084 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 40: cannot install the best candidate for the job 2024-04-12T20:40:58.084 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:8010020191114031513:16b3ab4d.x86_64 from CentOS-AppStream 2024-04-12T20:40:58.084 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:8010020191114031513:e1f37755.x86_64 from CentOS-AppStream 2024-04-12T20:40:58.084 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-04-12T20:40:58.084 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-04-12T20:40:58.084 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 41: cannot install the best candidate for the job 2024-04-12T20:40:58.084 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:8030020200716150652:1e4bbb35.x86_64 from CentOS-AppStream 2024-04-12T20:40:58.084 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:8030020200716150652:ea09926d.x86_64 from CentOS-AppStream 2024-04-12T20:40:58.084 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-04-12T20:40:58.084 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-04-12T20:40:58.084 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 42: cannot install the best candidate for the job 2024-04-12T20:40:58.084 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:8030020200716174729:3a70019f.x86_64 from CentOS-AppStream 2024-04-12T20:40:58.085 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:8030020200716174729:cccafca5.x86_64 from CentOS-AppStream 2024-04-12T20:40:58.085 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:820230223114759:6fdf7e5d.x86_64 from CentOS-AppStream 2024-04-12T20:40:58.085 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:820230223114759:d4326aba.x86_64 from CentOS-AppStream 2024-04-12T20:40:58.085 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 43: cannot install the best candidate for the job 2024-04-12T20:40:58.085 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:8030020200716213634:56fce90f.x86_64 from CentOS-AppStream 2024-04-12T20:40:58.085 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:8030020200716213634:98a3c9d0.x86_64 from CentOS-AppStream 2024-04-12T20:40:58.085 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:820230223114759:06b7596e.x86_64 from CentOS-AppStream 2024-04-12T20:40:58.085 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:820230223114759:d2fc5d76.x86_64 from CentOS-AppStream 2024-04-12T20:40:58.085 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 44: cannot install the best candidate for the job 2024-04-12T20:40:58.085 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:8030020200716171139:3a70019f.x86_64 from CentOS-AppStream 2024-04-12T20:40:58.085 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:8030020200716171139:cccafca5.x86_64 from CentOS-AppStream 2024-04-12T20:40:58.085 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:820230223114759:6fdf7e5d.x86_64 from CentOS-AppStream 2024-04-12T20:40:58.085 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:820230223114759:d4326aba.x86_64 from CentOS-AppStream 2024-04-12T20:40:58.085 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 45: cannot install the best candidate for the job 2024-04-12T20:40:58.086 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:8010020191120175858:95fce66d.x86_64 from CentOS-AppStream 2024-04-12T20:40:58.086 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:8030020200723233049:09acf126.x86_64 from CentOS-AppStream 2024-04-12T20:40:58.086 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:820230223114759:436af1ae.x86_64 from CentOS-AppStream 2024-04-12T20:40:58.086 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:820230223114759:b12195ea.x86_64 from CentOS-AppStream 2024-04-12T20:40:58.146 INFO:teuthology.orchestra.run.smithi186.stdout:Dependencies resolved. 2024-04-12T20:40:58.147 INFO:teuthology.orchestra.run.smithi186.stdout:====================================================================================== 2024-04-12T20:40:58.147 INFO:teuthology.orchestra.run.smithi186.stdout: Package Arch Version Repository Size 2024-04-12T20:40:58.147 INFO:teuthology.orchestra.run.smithi186.stdout:====================================================================================== 2024-04-12T20:40:58.147 INFO:teuthology.orchestra.run.smithi186.stdout:Installing: 2024-04-12T20:40:58.147 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-mgr-diskprediction-local noarch 2:17.2.7-827.gd3b5060a.el9 ceph-noarch 7.4 M 2024-04-12T20:40:58.147 INFO:teuthology.orchestra.run.smithi186.stdout:Installing dependencies: 2024-04-12T20:40:58.147 INFO:teuthology.orchestra.run.smithi186.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-04-12T20:40:58.148 INFO:teuthology.orchestra.run.smithi186.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-04-12T20:40:58.148 INFO:teuthology.orchestra.run.smithi186.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-04-12T20:40:58.148 INFO:teuthology.orchestra.run.smithi186.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-04-12T20:40:58.148 INFO:teuthology.orchestra.run.smithi186.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-04-12T20:40:58.148 INFO:teuthology.orchestra.run.smithi186.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-04-12T20:40:58.148 INFO:teuthology.orchestra.run.smithi186.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-04-12T20:40:58.148 INFO:teuthology.orchestra.run.smithi186.stdout: python3-devel x86_64 3.9.17-2.el9 appstream 245 k 2024-04-12T20:40:58.148 INFO:teuthology.orchestra.run.smithi186.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-04-12T20:40:58.148 INFO:teuthology.orchestra.run.smithi186.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-04-12T20:40:58.148 INFO:teuthology.orchestra.run.smithi186.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-04-12T20:40:58.148 INFO:teuthology.orchestra.run.smithi186.stdout: 2024-04-12T20:40:58.148 INFO:teuthology.orchestra.run.smithi186.stdout:Transaction Summary 2024-04-12T20:40:58.148 INFO:teuthology.orchestra.run.smithi186.stdout:====================================================================================== 2024-04-12T20:40:58.148 INFO:teuthology.orchestra.run.smithi186.stdout:Install 12 Packages 2024-04-12T20:40:58.149 INFO:teuthology.orchestra.run.smithi186.stdout: 2024-04-12T20:40:58.149 INFO:teuthology.orchestra.run.smithi186.stdout:Total download size: 38 M 2024-04-12T20:40:58.149 INFO:teuthology.orchestra.run.smithi186.stdout:Installed size: 209 M 2024-04-12T20:40:58.149 INFO:teuthology.orchestra.run.smithi186.stdout:Downloading Packages: 2024-04-12T20:40:58.268 INFO:teuthology.orchestra.run.smithi018.stderr:Modular dependency problems: 2024-04-12T20:40:58.268 INFO:teuthology.orchestra.run.smithi018.stderr: 2024-04-12T20:40:58.268 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 1: conflicting requests 2024-04-12T20:40:58.268 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module container-tools:rhel8:820240321091303:20125149.x86_64 from CentOS-AppStream 2024-04-12T20:40:58.268 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 2: conflicting requests 2024-04-12T20:40:58.269 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module eclipse:rhel8:820201023100746:b230ed4e.x86_64 from CentOS-AppStream 2024-04-12T20:40:58.269 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 3: conflicting requests 2024-04-12T20:40:58.269 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module freeradius:3.0:8080020230117180605:89170a74.x86_64 from CentOS-AppStream 2024-04-12T20:40:58.269 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 4: conflicting requests 2024-04-12T20:40:58.269 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module gimp:2.8:8000020190628145146:4148dfdf.x86_64 from CentOS-AppStream 2024-04-12T20:40:58.269 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 5: conflicting requests 2024-04-12T20:40:58.269 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module go-toolset:rhel8:820240401160413:b754926a.x86_64 from CentOS-AppStream 2024-04-12T20:40:58.269 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 6: conflicting requests 2024-04-12T20:40:58.269 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module httpd:2.4:820240216084734:9edba152.x86_64 from CentOS-AppStream 2024-04-12T20:40:58.269 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 7: conflicting requests 2024-04-12T20:40:58.269 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module idm:client:820240307184541:49cc9d1b.x86_64 from CentOS-AppStream 2024-04-12T20:40:58.269 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 8: conflicting requests 2024-04-12T20:40:58.269 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module inkscape:0.92.3:8080020221205124429:3e031279.x86_64 from CentOS-AppStream 2024-04-12T20:40:58.269 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 9: conflicting requests 2024-04-12T20:40:58.269 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module jmc:rhel8:8050020211110222101:6392b1f8.x86_64 from CentOS-AppStream 2024-04-12T20:40:58.270 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 10: conflicting requests 2024-04-12T20:40:58.270 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module llvm-toolset:rhel8:820240205133135:9edba152.x86_64 from CentOS-AppStream 2024-04-12T20:40:58.270 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 11: conflicting requests 2024-04-12T20:40:58.270 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module log4j:2:8080020221205124743:9d367344.x86_64 from CentOS-AppStream 2024-04-12T20:40:58.270 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 12: conflicting requests 2024-04-12T20:40:58.270 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module mailman:2.1:820230727085757:77fc8825.x86_64 from CentOS-AppStream 2024-04-12T20:40:58.270 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 13: conflicting requests 2024-04-12T20:40:58.270 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module mariadb:10.3:8030020210419150013:30b713e6.x86_64 from CentOS-AppStream 2024-04-12T20:40:58.270 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 14: conflicting requests 2024-04-12T20:40:58.270 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module maven:3.5:8060020211203142015:c0229ad2.x86_64 from CentOS-AppStream 2024-04-12T20:40:58.270 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 15: conflicting requests 2024-04-12T20:40:58.270 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module mercurial:4.8:8000020190628020724:4148dfdf.x86_64 from CentOS-AppStream 2024-04-12T20:40:58.270 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 16: conflicting requests 2024-04-12T20:40:58.271 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module mod_auth_openidc:2.3:820230721123832:9edba152.x86_64 from CentOS-AppStream 2024-04-12T20:40:58.271 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 17: conflicting requests 2024-04-12T20:40:58.271 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module mysql:8.0:8040020210901180257:9f9e2e7e.x86_64 from CentOS-AppStream 2024-04-12T20:40:58.271 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 18: conflicting requests 2024-04-12T20:40:58.271 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module nginx:1.14:8000020211221191913:55190bc5.x86_64 from CentOS-AppStream 2024-04-12T20:40:58.271 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 19: conflicting requests 2024-04-12T20:40:58.271 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module nodejs:10:8040020210120182536:9f9e2e7e.x86_64 from CentOS-AppStream 2024-04-12T20:40:58.271 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 20: conflicting requests 2024-04-12T20:40:58.271 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl:5.26:8000020190628020724:55190bc5.x86_64 from CentOS-AppStream 2024-04-12T20:40:58.272 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 21: conflicting requests 2024-04-12T20:40:58.272 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module php:7.2:8020020200507003613:2c7ca891.x86_64 from CentOS-AppStream 2024-04-12T20:40:58.272 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 22: conflicting requests 2024-04-12T20:40:58.272 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module postgresql:10:820230223114800:9edba152.x86_64 from CentOS-AppStream 2024-04-12T20:40:58.272 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 23: conflicting requests 2024-04-12T20:40:58.272 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module python27:2.7:820240208011952:182f7c73.x86_64 from CentOS-AppStream 2024-04-12T20:40:58.272 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 24: conflicting requests 2024-04-12T20:40:58.273 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module python36:3.6:8050020210825152031:982725ab.x86_64 from CentOS-AppStream 2024-04-12T20:40:58.273 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 25: conflicting requests 2024-04-12T20:40:58.273 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module python38:3.8:820230810143931:64c1cd5a.x86_64 from CentOS-AppStream 2024-04-12T20:40:58.273 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 26: conflicting requests 2024-04-12T20:40:58.273 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module python39:3.9:820240214182535:17377f89.x86_64 from CentOS-AppStream 2024-04-12T20:40:58.273 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 27: conflicting requests 2024-04-12T20:40:58.274 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module redis:5:8040020211019153849:9f9e2e7e.x86_64 from CentOS-AppStream 2024-04-12T20:40:58.274 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 28: conflicting requests 2024-04-12T20:40:58.274 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module rhn-tools:1.0:8010020191114034948:f69d1239.x86_64 from CentOS-AppStream 2024-04-12T20:40:58.274 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 29: conflicting requests 2024-04-12T20:40:58.274 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module ruby:2.5:820230627084142:9edba152.x86_64 from CentOS-AppStream 2024-04-12T20:40:58.274 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 30: conflicting requests 2024-04-12T20:40:58.275 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module rust-toolset:rhel8:820240119204620:b09eea91.x86_64 from CentOS-AppStream 2024-04-12T20:40:58.275 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 31: conflicting requests 2024-04-12T20:40:58.275 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module satellite-5-client:1.0:8010020191114035551:cdc1202b.x86_64 from CentOS-AppStream 2024-04-12T20:40:58.275 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 32: conflicting requests 2024-04-12T20:40:58.275 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module squid:4:820240319173245:9edba152.x86_64 from CentOS-AppStream 2024-04-12T20:40:58.275 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 33: conflicting requests 2024-04-12T20:40:58.275 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module subversion:1.10:8070020220711155714:78111232.x86_64 from CentOS-AppStream 2024-04-12T20:40:58.276 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 34: conflicting requests 2024-04-12T20:40:58.276 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module swig:3.0:8030020200706142531:30b713e6.x86_64 from CentOS-AppStream 2024-04-12T20:40:58.276 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 35: conflicting requests 2024-04-12T20:40:58.276 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module varnish:6:8050020211109225449:b4937e53.x86_64 from CentOS-AppStream 2024-04-12T20:40:58.276 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 36: conflicting requests 2024-04-12T20:40:58.276 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module virt:rhel:820240314161907:9edba152.x86_64 from CentOS-AppStream 2024-04-12T20:40:58.277 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 37: conflicting requests 2024-04-12T20:40:58.277 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:8030020200716155257:7cc0a66d.x86_64 from CentOS-AppStream 2024-04-12T20:40:58.277 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:8030020200716155257:b967a9a2.x86_64 from CentOS-AppStream 2024-04-12T20:40:58.277 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:820230811133638:36f9fae6.x86_64 from CentOS-AppStream 2024-04-12T20:40:58.277 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 38: cannot install the best candidate for the job 2024-04-12T20:40:58.277 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:8010020191114031501:a5949e2e.x86_64 from CentOS-AppStream 2024-04-12T20:40:58.278 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:8010020191114031501:be2e4737.x86_64 from CentOS-AppStream 2024-04-12T20:40:58.278 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:820230223114759:2ab761e1.x86_64 from CentOS-AppStream 2024-04-12T20:40:58.278 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:820230223114759:86e45846.x86_64 from CentOS-AppStream 2024-04-12T20:40:58.278 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 39: conflicting requests 2024-04-12T20:40:58.278 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:8030020200715230104:1e4bbb35.x86_64 from CentOS-AppStream 2024-04-12T20:40:58.278 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:8030020200715230104:ea09926d.x86_64 from CentOS-AppStream 2024-04-12T20:40:58.279 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:082fdf2f.x86_64 from CentOS-AppStream 2024-04-12T20:40:58.279 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-04-12T20:40:58.279 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-04-12T20:40:58.279 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:fbe42456.x86_64 from CentOS-AppStream 2024-04-12T20:40:58.279 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 40: cannot install the best candidate for the job 2024-04-12T20:40:58.279 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:8010020191114031513:16b3ab4d.x86_64 from CentOS-AppStream 2024-04-12T20:40:58.279 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:8010020191114031513:e1f37755.x86_64 from CentOS-AppStream 2024-04-12T20:40:58.279 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-04-12T20:40:58.280 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-04-12T20:40:58.280 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 41: cannot install the best candidate for the job 2024-04-12T20:40:58.280 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:8030020200716150652:1e4bbb35.x86_64 from CentOS-AppStream 2024-04-12T20:40:58.280 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:8030020200716150652:ea09926d.x86_64 from CentOS-AppStream 2024-04-12T20:40:58.280 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-04-12T20:40:58.280 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-04-12T20:40:58.280 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 42: cannot install the best candidate for the job 2024-04-12T20:40:58.281 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:8030020200716174729:3a70019f.x86_64 from CentOS-AppStream 2024-04-12T20:40:58.281 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:8030020200716174729:cccafca5.x86_64 from CentOS-AppStream 2024-04-12T20:40:58.281 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:820230223114759:6fdf7e5d.x86_64 from CentOS-AppStream 2024-04-12T20:40:58.281 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:820230223114759:d4326aba.x86_64 from CentOS-AppStream 2024-04-12T20:40:58.281 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 43: cannot install the best candidate for the job 2024-04-12T20:40:58.281 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:8030020200716213634:56fce90f.x86_64 from CentOS-AppStream 2024-04-12T20:40:58.281 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:8030020200716213634:98a3c9d0.x86_64 from CentOS-AppStream 2024-04-12T20:40:58.281 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:820230223114759:06b7596e.x86_64 from CentOS-AppStream 2024-04-12T20:40:58.282 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:820230223114759:d2fc5d76.x86_64 from CentOS-AppStream 2024-04-12T20:40:58.282 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 44: cannot install the best candidate for the job 2024-04-12T20:40:58.282 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:8030020200716171139:3a70019f.x86_64 from CentOS-AppStream 2024-04-12T20:40:58.282 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:8030020200716171139:cccafca5.x86_64 from CentOS-AppStream 2024-04-12T20:40:58.282 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:820230223114759:6fdf7e5d.x86_64 from CentOS-AppStream 2024-04-12T20:40:58.282 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:820230223114759:d4326aba.x86_64 from CentOS-AppStream 2024-04-12T20:40:58.282 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 45: cannot install the best candidate for the job 2024-04-12T20:40:58.282 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:8010020191120175858:95fce66d.x86_64 from CentOS-AppStream 2024-04-12T20:40:58.282 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:8030020200723233049:09acf126.x86_64 from CentOS-AppStream 2024-04-12T20:40:58.282 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:820230223114759:436af1ae.x86_64 from CentOS-AppStream 2024-04-12T20:40:58.282 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:820230223114759:b12195ea.x86_64 from CentOS-AppStream 2024-04-12T20:40:58.345 INFO:teuthology.orchestra.run.smithi018.stdout:Dependencies resolved. 2024-04-12T20:40:58.347 INFO:teuthology.orchestra.run.smithi018.stdout:====================================================================================== 2024-04-12T20:40:58.347 INFO:teuthology.orchestra.run.smithi018.stdout: Package Arch Version Repository Size 2024-04-12T20:40:58.347 INFO:teuthology.orchestra.run.smithi018.stdout:====================================================================================== 2024-04-12T20:40:58.347 INFO:teuthology.orchestra.run.smithi018.stdout:Installing: 2024-04-12T20:40:58.347 INFO:teuthology.orchestra.run.smithi018.stdout: ceph-mgr-diskprediction-local noarch 2:17.2.7-827.gd3b5060a.el9 ceph-noarch 7.4 M 2024-04-12T20:40:58.347 INFO:teuthology.orchestra.run.smithi018.stdout:Installing dependencies: 2024-04-12T20:40:58.347 INFO:teuthology.orchestra.run.smithi018.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-04-12T20:40:58.347 INFO:teuthology.orchestra.run.smithi018.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-04-12T20:40:58.347 INFO:teuthology.orchestra.run.smithi018.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-04-12T20:40:58.347 INFO:teuthology.orchestra.run.smithi018.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-04-12T20:40:58.347 INFO:teuthology.orchestra.run.smithi018.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-04-12T20:40:58.347 INFO:teuthology.orchestra.run.smithi018.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-04-12T20:40:58.347 INFO:teuthology.orchestra.run.smithi018.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-04-12T20:40:58.348 INFO:teuthology.orchestra.run.smithi018.stdout: python3-devel x86_64 3.9.17-2.el9 appstream 245 k 2024-04-12T20:40:58.348 INFO:teuthology.orchestra.run.smithi018.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-04-12T20:40:58.348 INFO:teuthology.orchestra.run.smithi018.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-04-12T20:40:58.348 INFO:teuthology.orchestra.run.smithi018.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-04-12T20:40:58.348 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-12T20:40:58.348 INFO:teuthology.orchestra.run.smithi018.stdout:Transaction Summary 2024-04-12T20:40:58.348 INFO:teuthology.orchestra.run.smithi018.stdout:====================================================================================== 2024-04-12T20:40:58.348 INFO:teuthology.orchestra.run.smithi018.stdout:Install 12 Packages 2024-04-12T20:40:58.348 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-12T20:40:58.349 INFO:teuthology.orchestra.run.smithi018.stdout:Total download size: 38 M 2024-04-12T20:40:58.349 INFO:teuthology.orchestra.run.smithi018.stdout:Installed size: 209 M 2024-04-12T20:40:58.349 INFO:teuthology.orchestra.run.smithi018.stdout:Downloading Packages: 2024-04-12T20:40:58.516 INFO:teuthology.orchestra.run.smithi186.stdout:(1/12): libquadmath-11.4.1-3.el9.x86_64.rpm 980 kB/s | 195 kB 00:00 2024-04-12T20:40:58.566 INFO:teuthology.orchestra.run.smithi186.stdout:(2/12): libgfortran-11.4.1-3.el9.x86_64.rpm 3.1 MB/s | 803 kB 00:00 2024-04-12T20:40:58.850 INFO:teuthology.orchestra.run.smithi186.stdout:(3/12): flexiblas-3.0.4-8.el9.x86_64.rpm 99 kB/s | 33 kB 00:00 2024-04-12T20:40:58.933 INFO:teuthology.orchestra.run.smithi186.stdout:(4/12): flexiblas-openblas-openmp-3.0.4-8.el9.x 220 kB/s | 18 kB 00:00 2024-04-12T20:40:58.975 INFO:teuthology.orchestra.run.smithi018.stdout:(1/12): libquadmath-11.4.1-3.el9.x86_64.rpm 992 kB/s | 195 kB 00:00 2024-04-12T20:40:59.016 INFO:teuthology.orchestra.run.smithi018.stdout:(2/12): flexiblas-3.0.4-8.el9.x86_64.rpm 803 kB/s | 33 kB 00:00 2024-04-12T20:40:59.034 INFO:teuthology.orchestra.run.smithi186.stdout:(5/12): openblas-0.3.21-2.el9.x86_64.rpm 355 kB/s | 35 kB 00:00 2024-04-12T20:40:59.050 INFO:teuthology.orchestra.run.smithi018.stdout:(3/12): libgfortran-11.4.1-3.el9.x86_64.rpm 2.9 MB/s | 803 kB 00:00 2024-04-12T20:40:59.092 INFO:teuthology.orchestra.run.smithi018.stdout:(4/12): flexiblas-openblas-openmp-3.0.4-8.el9.x 446 kB/s | 18 kB 00:00 2024-04-12T20:40:59.134 INFO:teuthology.orchestra.run.smithi186.stdout:(6/12): flexiblas-netlib-3.0.4-8.el9.x86_64.rpm 5.3 MB/s | 3.0 MB 00:00 2024-04-12T20:40:59.151 INFO:teuthology.orchestra.run.smithi018.stdout:(5/12): openblas-0.3.21-2.el9.x86_64.rpm 621 kB/s | 35 kB 00:00 2024-04-12T20:40:59.209 INFO:teuthology.orchestra.run.smithi186.stdout:(7/12): python3-devel-3.9.17-2.el9.x86_64.rpm 3.2 MB/s | 245 kB 00:00 2024-04-12T20:40:59.260 INFO:teuthology.orchestra.run.smithi018.stdout:(6/12): flexiblas-netlib-3.0.4-8.el9.x86_64.rpm 12 MB/s | 3.0 MB 00:00 2024-04-12T20:40:59.318 INFO:teuthology.orchestra.run.smithi018.stdout:(7/12): python3-devel-3.9.17-2.el9.x86_64.rpm 4.2 MB/s | 245 kB 00:00 2024-04-12T20:40:59.393 INFO:teuthology.orchestra.run.smithi186.stdout:(8/12): openblas-openmp-0.3.21-2.el9.x86_64.rpm 13 MB/s | 4.7 MB 00:00 2024-04-12T20:40:59.460 INFO:teuthology.orchestra.run.smithi018.stdout:(8/12): openblas-openmp-0.3.21-2.el9.x86_64.rpm 15 MB/s | 4.7 MB 00:00 2024-04-12T20:40:59.511 INFO:teuthology.orchestra.run.smithi018.stdout:(9/12): python3-numpy-f2py-1.20.1-5.el9.x86_64. 5.0 MB/s | 250 kB 00:00 2024-04-12T20:40:59.518 INFO:teuthology.orchestra.run.smithi186.stdout:(9/12): ceph-mgr-diskprediction-local-17.2.7-82 6.2 MB/s | 7.4 MB 00:01 2024-04-12T20:40:59.577 INFO:teuthology.orchestra.run.smithi186.stdout:(10/12): python3-numpy-f2py-1.20.1-5.el9.x86_64 1.3 MB/s | 250 kB 00:00 2024-04-12T20:40:59.644 INFO:teuthology.orchestra.run.smithi018.stdout:(10/12): python3-numpy-1.20.1-5.el9.x86_64.rpm 16 MB/s | 5.1 MB 00:00 2024-04-12T20:40:59.736 INFO:teuthology.orchestra.run.smithi186.stdout:(11/12): python3-numpy-1.20.1-5.el9.x86_64.rpm 9.7 MB/s | 5.1 MB 00:00 2024-04-12T20:40:59.953 INFO:teuthology.orchestra.run.smithi018.stdout:(11/12): python3-scipy-1.6.2-8.el9.x86_64.rpm 37 MB/s | 16 MB 00:00 2024-04-12T20:41:00.319 INFO:teuthology.orchestra.run.smithi186.stdout:(12/12): python3-scipy-1.6.2-8.el9.x86_64.rpm 20 MB/s | 16 MB 00:00 2024-04-12T20:41:00.328 INFO:teuthology.orchestra.run.smithi186.stdout:-------------------------------------------------------------------------------- 2024-04-12T20:41:00.328 INFO:teuthology.orchestra.run.smithi186.stdout:Total 17 MB/s | 38 MB 00:02 2024-04-12T20:41:00.511 INFO:teuthology.orchestra.run.smithi018.stdout:(12/12): ceph-mgr-diskprediction-local-17.2.7-8 4.3 MB/s | 7.4 MB 00:01 2024-04-12T20:41:00.513 INFO:teuthology.orchestra.run.smithi018.stdout:-------------------------------------------------------------------------------- 2024-04-12T20:41:00.514 INFO:teuthology.orchestra.run.smithi018.stdout:Total 18 MB/s | 38 MB 00:02 2024-04-12T20:41:00.956 INFO:teuthology.orchestra.run.smithi186.stdout:Running transaction check 2024-04-12T20:41:01.034 INFO:teuthology.orchestra.run.smithi186.stdout:Transaction check succeeded. 2024-04-12T20:41:01.034 INFO:teuthology.orchestra.run.smithi186.stdout:Running transaction test 2024-04-12T20:41:01.191 INFO:teuthology.orchestra.run.smithi018.stdout:Running transaction check 2024-04-12T20:41:01.273 INFO:teuthology.orchestra.run.smithi018.stdout:Transaction check succeeded. 2024-04-12T20:41:01.273 INFO:teuthology.orchestra.run.smithi018.stdout:Running transaction test 2024-04-12T20:41:01.639 INFO:teuthology.orchestra.run.smithi186.stdout:Transaction test succeeded. 2024-04-12T20:41:01.640 INFO:teuthology.orchestra.run.smithi186.stdout:Running transaction 2024-04-12T20:41:01.874 INFO:teuthology.orchestra.run.smithi018.stdout:Transaction test succeeded. 2024-04-12T20:41:01.875 INFO:teuthology.orchestra.run.smithi018.stdout:Running transaction 2024-04-12T20:41:02.526 INFO:teuthology.orchestra.run.smithi186.stdout: Preparing : 1/1 2024-04-12T20:41:02.689 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 1/12 2024-04-12T20:41:02.766 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 2/12 2024-04-12T20:41:02.820 INFO:teuthology.orchestra.run.smithi018.stdout: Preparing : 1/1 2024-04-12T20:41:02.894 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 1/12 2024-04-12T20:41:02.938 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 3/12 2024-04-12T20:41:02.949 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 2/12 2024-04-12T20:41:02.988 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-devel-3.9.17-2.el9.x86_64 4/12 2024-04-12T20:41:03.114 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 3/12 2024-04-12T20:41:03.163 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : python3-devel-3.9.17-2.el9.x86_64 4/12 2024-04-12T20:41:03.261 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : openblas-0.3.21-2.el9.x86_64 5/12 2024-04-12T20:41:03.293 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 6/12 2024-04-12T20:41:03.406 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 7/12 2024-04-12T20:41:03.432 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : openblas-0.3.21-2.el9.x86_64 5/12 2024-04-12T20:41:03.469 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 6/12 2024-04-12T20:41:03.569 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 7/12 2024-04-12T20:41:04.269 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 8/12 2024-04-12T20:41:04.405 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 9/12 2024-04-12T20:41:04.462 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 8/12 2024-04-12T20:41:04.612 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 9/12 2024-04-12T20:41:05.938 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 10/12 2024-04-12T20:41:06.253 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 10/12 2024-04-12T20:41:07.101 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 11/12 2024-04-12T20:41:07.134 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : ceph-mgr-diskprediction-local-2:17.2.7-827.gd3b506 12/12 2024-04-12T20:41:07.474 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 11/12 2024-04-12T20:41:07.534 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : ceph-mgr-diskprediction-local-2:17.2.7-827.gd3b506 12/12 2024-04-12T20:41:09.817 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:17.2.7-827.gd3b506 12/12 2024-04-12T20:41:09.817 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : ceph-mgr-diskprediction-local-2:17.2.7-827.gd3b506 1/12 2024-04-12T20:41:09.817 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 2/12 2024-04-12T20:41:09.817 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 3/12 2024-04-12T20:41:09.817 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 4/12 2024-04-12T20:41:09.817 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 5/12 2024-04-12T20:41:09.817 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 6/12 2024-04-12T20:41:09.817 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 7/12 2024-04-12T20:41:09.817 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 8/12 2024-04-12T20:41:09.818 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-devel-3.9.17-2.el9.x86_64 9/12 2024-04-12T20:41:09.818 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 10/12 2024-04-12T20:41:09.818 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 11/12 2024-04-12T20:41:10.162 INFO:teuthology.orchestra.run.smithi018.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:17.2.7-827.gd3b506 12/12 2024-04-12T20:41:10.162 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : ceph-mgr-diskprediction-local-2:17.2.7-827.gd3b506 1/12 2024-04-12T20:41:10.162 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 2/12 2024-04-12T20:41:10.163 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 3/12 2024-04-12T20:41:10.163 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 4/12 2024-04-12T20:41:10.163 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 5/12 2024-04-12T20:41:10.163 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 6/12 2024-04-12T20:41:10.163 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 7/12 2024-04-12T20:41:10.163 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 8/12 2024-04-12T20:41:10.163 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : python3-devel-3.9.17-2.el9.x86_64 9/12 2024-04-12T20:41:10.163 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 10/12 2024-04-12T20:41:10.163 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 11/12 2024-04-12T20:41:10.456 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 12/12 2024-04-12T20:41:10.456 INFO:teuthology.orchestra.run.smithi186.stdout: 2024-04-12T20:41:10.456 INFO:teuthology.orchestra.run.smithi186.stdout:Installed: 2024-04-12T20:41:10.456 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-mgr-diskprediction-local-2:17.2.7-827.gd3b5060a.el9.noarch 2024-04-12T20:41:10.456 INFO:teuthology.orchestra.run.smithi186.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-04-12T20:41:10.456 INFO:teuthology.orchestra.run.smithi186.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-04-12T20:41:10.457 INFO:teuthology.orchestra.run.smithi186.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-04-12T20:41:10.457 INFO:teuthology.orchestra.run.smithi186.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-04-12T20:41:10.457 INFO:teuthology.orchestra.run.smithi186.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-04-12T20:41:10.457 INFO:teuthology.orchestra.run.smithi186.stdout: openblas-0.3.21-2.el9.x86_64 2024-04-12T20:41:10.457 INFO:teuthology.orchestra.run.smithi186.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-04-12T20:41:10.457 INFO:teuthology.orchestra.run.smithi186.stdout: python3-devel-3.9.17-2.el9.x86_64 2024-04-12T20:41:10.457 INFO:teuthology.orchestra.run.smithi186.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-04-12T20:41:10.457 INFO:teuthology.orchestra.run.smithi186.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-04-12T20:41:10.457 INFO:teuthology.orchestra.run.smithi186.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-04-12T20:41:10.457 INFO:teuthology.orchestra.run.smithi186.stdout: 2024-04-12T20:41:10.457 INFO:teuthology.orchestra.run.smithi186.stdout:Complete! 2024-04-12T20:41:10.656 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 12/12 2024-04-12T20:41:10.656 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-12T20:41:10.657 INFO:teuthology.orchestra.run.smithi018.stdout:Installed: 2024-04-12T20:41:10.657 INFO:teuthology.orchestra.run.smithi018.stdout: ceph-mgr-diskprediction-local-2:17.2.7-827.gd3b5060a.el9.noarch 2024-04-12T20:41:10.657 INFO:teuthology.orchestra.run.smithi018.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-04-12T20:41:10.657 INFO:teuthology.orchestra.run.smithi018.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-04-12T20:41:10.657 INFO:teuthology.orchestra.run.smithi018.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-04-12T20:41:10.657 INFO:teuthology.orchestra.run.smithi018.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-04-12T20:41:10.657 INFO:teuthology.orchestra.run.smithi018.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-04-12T20:41:10.657 INFO:teuthology.orchestra.run.smithi018.stdout: openblas-0.3.21-2.el9.x86_64 2024-04-12T20:41:10.657 INFO:teuthology.orchestra.run.smithi018.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-04-12T20:41:10.657 INFO:teuthology.orchestra.run.smithi018.stdout: python3-devel-3.9.17-2.el9.x86_64 2024-04-12T20:41:10.657 INFO:teuthology.orchestra.run.smithi018.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-04-12T20:41:10.657 INFO:teuthology.orchestra.run.smithi018.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-04-12T20:41:10.657 INFO:teuthology.orchestra.run.smithi018.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-04-12T20:41:10.657 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-12T20:41:10.658 INFO:teuthology.orchestra.run.smithi018.stdout:Complete! 2024-04-12T20:41:10.902 DEBUG:teuthology.orchestra.run.smithi186:> sudo yum -y install ceph-mgr-rook 2024-04-12T20:41:11.078 DEBUG:teuthology.orchestra.run.smithi018:> sudo yum -y install ceph-mgr-rook 2024-04-12T20:41:11.804 INFO:teuthology.orchestra.run.smithi186.stdout:Last metadata expiration check: 0:01:34 ago on Fri 12 Apr 2024 08:39:37 PM UTC. 2024-04-12T20:41:11.995 INFO:teuthology.orchestra.run.smithi018.stdout:Last metadata expiration check: 0:01:36 ago on Fri 12 Apr 2024 08:39:35 PM UTC. 2024-04-12T20:41:12.104 INFO:teuthology.orchestra.run.smithi186.stderr:Modular dependency problems: 2024-04-12T20:41:12.104 INFO:teuthology.orchestra.run.smithi186.stderr: 2024-04-12T20:41:12.104 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 1: conflicting requests 2024-04-12T20:41:12.104 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module container-tools:rhel8:820240321091303:20125149.x86_64 from CentOS-AppStream 2024-04-12T20:41:12.104 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 2: conflicting requests 2024-04-12T20:41:12.105 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module eclipse:rhel8:820201023100746:b230ed4e.x86_64 from CentOS-AppStream 2024-04-12T20:41:12.105 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 3: conflicting requests 2024-04-12T20:41:12.105 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module freeradius:3.0:8080020230117180605:89170a74.x86_64 from CentOS-AppStream 2024-04-12T20:41:12.105 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 4: conflicting requests 2024-04-12T20:41:12.105 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module gimp:2.8:8000020190628145146:4148dfdf.x86_64 from CentOS-AppStream 2024-04-12T20:41:12.105 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 5: conflicting requests 2024-04-12T20:41:12.105 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module go-toolset:rhel8:820240401160413:b754926a.x86_64 from CentOS-AppStream 2024-04-12T20:41:12.105 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 6: conflicting requests 2024-04-12T20:41:12.105 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module httpd:2.4:820240216084734:9edba152.x86_64 from CentOS-AppStream 2024-04-12T20:41:12.105 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 7: conflicting requests 2024-04-12T20:41:12.105 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module idm:client:820240307184541:49cc9d1b.x86_64 from CentOS-AppStream 2024-04-12T20:41:12.105 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 8: conflicting requests 2024-04-12T20:41:12.105 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module inkscape:0.92.3:8080020221205124429:3e031279.x86_64 from CentOS-AppStream 2024-04-12T20:41:12.106 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 9: conflicting requests 2024-04-12T20:41:12.106 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module jmc:rhel8:8050020211110222101:6392b1f8.x86_64 from CentOS-AppStream 2024-04-12T20:41:12.106 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 10: conflicting requests 2024-04-12T20:41:12.106 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module llvm-toolset:rhel8:820240205133135:9edba152.x86_64 from CentOS-AppStream 2024-04-12T20:41:12.106 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 11: conflicting requests 2024-04-12T20:41:12.106 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module log4j:2:8080020221205124743:9d367344.x86_64 from CentOS-AppStream 2024-04-12T20:41:12.106 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 12: conflicting requests 2024-04-12T20:41:12.106 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module mailman:2.1:820230727085757:77fc8825.x86_64 from CentOS-AppStream 2024-04-12T20:41:12.106 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 13: conflicting requests 2024-04-12T20:41:12.107 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module mariadb:10.3:8030020210419150013:30b713e6.x86_64 from CentOS-AppStream 2024-04-12T20:41:12.107 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 14: conflicting requests 2024-04-12T20:41:12.107 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module maven:3.5:8060020211203142015:c0229ad2.x86_64 from CentOS-AppStream 2024-04-12T20:41:12.107 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 15: conflicting requests 2024-04-12T20:41:12.107 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module mercurial:4.8:8000020190628020724:4148dfdf.x86_64 from CentOS-AppStream 2024-04-12T20:41:12.107 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 16: conflicting requests 2024-04-12T20:41:12.107 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module mod_auth_openidc:2.3:820230721123832:9edba152.x86_64 from CentOS-AppStream 2024-04-12T20:41:12.107 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 17: conflicting requests 2024-04-12T20:41:12.107 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module mysql:8.0:8040020210901180257:9f9e2e7e.x86_64 from CentOS-AppStream 2024-04-12T20:41:12.107 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 18: conflicting requests 2024-04-12T20:41:12.107 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module nginx:1.14:8000020211221191913:55190bc5.x86_64 from CentOS-AppStream 2024-04-12T20:41:12.107 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 19: conflicting requests 2024-04-12T20:41:12.107 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module nodejs:10:8040020210120182536:9f9e2e7e.x86_64 from CentOS-AppStream 2024-04-12T20:41:12.107 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 20: conflicting requests 2024-04-12T20:41:12.108 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl:5.26:8000020190628020724:55190bc5.x86_64 from CentOS-AppStream 2024-04-12T20:41:12.108 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 21: conflicting requests 2024-04-12T20:41:12.108 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module php:7.2:8020020200507003613:2c7ca891.x86_64 from CentOS-AppStream 2024-04-12T20:41:12.108 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 22: conflicting requests 2024-04-12T20:41:12.108 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module postgresql:10:820230223114800:9edba152.x86_64 from CentOS-AppStream 2024-04-12T20:41:12.108 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 23: conflicting requests 2024-04-12T20:41:12.108 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module python27:2.7:820240208011952:182f7c73.x86_64 from CentOS-AppStream 2024-04-12T20:41:12.108 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 24: conflicting requests 2024-04-12T20:41:12.108 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module python36:3.6:8050020210825152031:982725ab.x86_64 from CentOS-AppStream 2024-04-12T20:41:12.108 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 25: conflicting requests 2024-04-12T20:41:12.108 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module python38:3.8:820230810143931:64c1cd5a.x86_64 from CentOS-AppStream 2024-04-12T20:41:12.108 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 26: conflicting requests 2024-04-12T20:41:12.108 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module python39:3.9:820240214182535:17377f89.x86_64 from CentOS-AppStream 2024-04-12T20:41:12.109 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 27: conflicting requests 2024-04-12T20:41:12.109 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module redis:5:8040020211019153849:9f9e2e7e.x86_64 from CentOS-AppStream 2024-04-12T20:41:12.109 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 28: conflicting requests 2024-04-12T20:41:12.109 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module rhn-tools:1.0:8010020191114034948:f69d1239.x86_64 from CentOS-AppStream 2024-04-12T20:41:12.109 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 29: conflicting requests 2024-04-12T20:41:12.109 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module ruby:2.5:820230627084142:9edba152.x86_64 from CentOS-AppStream 2024-04-12T20:41:12.109 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 30: conflicting requests 2024-04-12T20:41:12.109 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module rust-toolset:rhel8:820240119204620:b09eea91.x86_64 from CentOS-AppStream 2024-04-12T20:41:12.109 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 31: conflicting requests 2024-04-12T20:41:12.109 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module satellite-5-client:1.0:8010020191114035551:cdc1202b.x86_64 from CentOS-AppStream 2024-04-12T20:41:12.109 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 32: conflicting requests 2024-04-12T20:41:12.109 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module squid:4:820240319173245:9edba152.x86_64 from CentOS-AppStream 2024-04-12T20:41:12.109 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 33: conflicting requests 2024-04-12T20:41:12.109 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module subversion:1.10:8070020220711155714:78111232.x86_64 from CentOS-AppStream 2024-04-12T20:41:12.110 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 34: conflicting requests 2024-04-12T20:41:12.110 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module swig:3.0:8030020200706142531:30b713e6.x86_64 from CentOS-AppStream 2024-04-12T20:41:12.110 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 35: conflicting requests 2024-04-12T20:41:12.111 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module varnish:6:8050020211109225449:b4937e53.x86_64 from CentOS-AppStream 2024-04-12T20:41:12.112 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 36: conflicting requests 2024-04-12T20:41:12.112 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module virt:rhel:820240314161907:9edba152.x86_64 from CentOS-AppStream 2024-04-12T20:41:12.112 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 37: conflicting requests 2024-04-12T20:41:12.112 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:8030020200716155257:7cc0a66d.x86_64 from CentOS-AppStream 2024-04-12T20:41:12.112 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:8030020200716155257:b967a9a2.x86_64 from CentOS-AppStream 2024-04-12T20:41:12.112 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:820230811133638:36f9fae6.x86_64 from CentOS-AppStream 2024-04-12T20:41:12.112 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 38: cannot install the best candidate for the job 2024-04-12T20:41:12.112 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:8010020191114031501:a5949e2e.x86_64 from CentOS-AppStream 2024-04-12T20:41:12.113 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:8010020191114031501:be2e4737.x86_64 from CentOS-AppStream 2024-04-12T20:41:12.114 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:820230223114759:2ab761e1.x86_64 from CentOS-AppStream 2024-04-12T20:41:12.114 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:820230223114759:86e45846.x86_64 from CentOS-AppStream 2024-04-12T20:41:12.114 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 39: conflicting requests 2024-04-12T20:41:12.114 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:8030020200715230104:1e4bbb35.x86_64 from CentOS-AppStream 2024-04-12T20:41:12.114 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:8030020200715230104:ea09926d.x86_64 from CentOS-AppStream 2024-04-12T20:41:12.114 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:082fdf2f.x86_64 from CentOS-AppStream 2024-04-12T20:41:12.114 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-04-12T20:41:12.114 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-04-12T20:41:12.114 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:fbe42456.x86_64 from CentOS-AppStream 2024-04-12T20:41:12.114 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 40: cannot install the best candidate for the job 2024-04-12T20:41:12.115 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:8010020191114031513:16b3ab4d.x86_64 from CentOS-AppStream 2024-04-12T20:41:12.115 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:8010020191114031513:e1f37755.x86_64 from CentOS-AppStream 2024-04-12T20:41:12.115 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-04-12T20:41:12.115 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-04-12T20:41:12.115 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 41: cannot install the best candidate for the job 2024-04-12T20:41:12.115 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:8030020200716150652:1e4bbb35.x86_64 from CentOS-AppStream 2024-04-12T20:41:12.115 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:8030020200716150652:ea09926d.x86_64 from CentOS-AppStream 2024-04-12T20:41:12.115 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-04-12T20:41:12.115 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-04-12T20:41:12.115 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 42: cannot install the best candidate for the job 2024-04-12T20:41:12.115 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:8030020200716174729:3a70019f.x86_64 from CentOS-AppStream 2024-04-12T20:41:12.115 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:8030020200716174729:cccafca5.x86_64 from CentOS-AppStream 2024-04-12T20:41:12.115 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:820230223114759:6fdf7e5d.x86_64 from CentOS-AppStream 2024-04-12T20:41:12.116 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:820230223114759:d4326aba.x86_64 from CentOS-AppStream 2024-04-12T20:41:12.116 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 43: cannot install the best candidate for the job 2024-04-12T20:41:12.116 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:8030020200716213634:56fce90f.x86_64 from CentOS-AppStream 2024-04-12T20:41:12.116 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:8030020200716213634:98a3c9d0.x86_64 from CentOS-AppStream 2024-04-12T20:41:12.116 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:820230223114759:06b7596e.x86_64 from CentOS-AppStream 2024-04-12T20:41:12.116 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:820230223114759:d2fc5d76.x86_64 from CentOS-AppStream 2024-04-12T20:41:12.116 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 44: cannot install the best candidate for the job 2024-04-12T20:41:12.116 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:8030020200716171139:3a70019f.x86_64 from CentOS-AppStream 2024-04-12T20:41:12.116 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:8030020200716171139:cccafca5.x86_64 from CentOS-AppStream 2024-04-12T20:41:12.116 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:820230223114759:6fdf7e5d.x86_64 from CentOS-AppStream 2024-04-12T20:41:12.116 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:820230223114759:d4326aba.x86_64 from CentOS-AppStream 2024-04-12T20:41:12.116 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 45: cannot install the best candidate for the job 2024-04-12T20:41:12.116 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:8010020191120175858:95fce66d.x86_64 from CentOS-AppStream 2024-04-12T20:41:12.116 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:8030020200723233049:09acf126.x86_64 from CentOS-AppStream 2024-04-12T20:41:12.117 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:820230223114759:436af1ae.x86_64 from CentOS-AppStream 2024-04-12T20:41:12.117 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:820230223114759:b12195ea.x86_64 from CentOS-AppStream 2024-04-12T20:41:12.169 INFO:teuthology.orchestra.run.smithi186.stdout:Dependencies resolved. 2024-04-12T20:41:12.171 INFO:teuthology.orchestra.run.smithi186.stdout:================================================================================ 2024-04-12T20:41:12.171 INFO:teuthology.orchestra.run.smithi186.stdout: Package Arch Version Repository Size 2024-04-12T20:41:12.171 INFO:teuthology.orchestra.run.smithi186.stdout:================================================================================ 2024-04-12T20:41:12.171 INFO:teuthology.orchestra.run.smithi186.stdout:Installing: 2024-04-12T20:41:12.171 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-mgr-rook noarch 2:17.2.7-827.gd3b5060a.el9 ceph-noarch 47 k 2024-04-12T20:41:12.171 INFO:teuthology.orchestra.run.smithi186.stdout:Installing dependencies: 2024-04-12T20:41:12.171 INFO:teuthology.orchestra.run.smithi186.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-04-12T20:41:12.171 INFO:teuthology.orchestra.run.smithi186.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-04-12T20:41:12.171 INFO:teuthology.orchestra.run.smithi186.stdout: python3-google-auth noarch 1:2.28.2-1.el9 epel 203 k 2024-04-12T20:41:12.171 INFO:teuthology.orchestra.run.smithi186.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-04-12T20:41:12.171 INFO:teuthology.orchestra.run.smithi186.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-04-12T20:41:12.171 INFO:teuthology.orchestra.run.smithi186.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-04-12T20:41:12.171 INFO:teuthology.orchestra.run.smithi186.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-04-12T20:41:12.172 INFO:teuthology.orchestra.run.smithi186.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-04-12T20:41:12.172 INFO:teuthology.orchestra.run.smithi186.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-04-12T20:41:12.172 INFO:teuthology.orchestra.run.smithi186.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-04-12T20:41:12.172 INFO:teuthology.orchestra.run.smithi186.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-04-12T20:41:12.172 INFO:teuthology.orchestra.run.smithi186.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-04-12T20:41:12.172 INFO:teuthology.orchestra.run.smithi186.stdout: 2024-04-12T20:41:12.172 INFO:teuthology.orchestra.run.smithi186.stdout:Transaction Summary 2024-04-12T20:41:12.172 INFO:teuthology.orchestra.run.smithi186.stdout:================================================================================ 2024-04-12T20:41:12.172 INFO:teuthology.orchestra.run.smithi186.stdout:Install 13 Packages 2024-04-12T20:41:12.172 INFO:teuthology.orchestra.run.smithi186.stdout: 2024-04-12T20:41:12.173 INFO:teuthology.orchestra.run.smithi186.stdout:Total download size: 2.2 M 2024-04-12T20:41:12.173 INFO:teuthology.orchestra.run.smithi186.stdout:Installed size: 26 M 2024-04-12T20:41:12.173 INFO:teuthology.orchestra.run.smithi186.stdout:Downloading Packages: 2024-04-12T20:41:12.307 INFO:teuthology.orchestra.run.smithi018.stderr:Modular dependency problems: 2024-04-12T20:41:12.307 INFO:teuthology.orchestra.run.smithi018.stderr: 2024-04-12T20:41:12.307 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 1: conflicting requests 2024-04-12T20:41:12.307 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module container-tools:rhel8:820240321091303:20125149.x86_64 from CentOS-AppStream 2024-04-12T20:41:12.307 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 2: conflicting requests 2024-04-12T20:41:12.307 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module eclipse:rhel8:820201023100746:b230ed4e.x86_64 from CentOS-AppStream 2024-04-12T20:41:12.307 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 3: conflicting requests 2024-04-12T20:41:12.307 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module freeradius:3.0:8080020230117180605:89170a74.x86_64 from CentOS-AppStream 2024-04-12T20:41:12.307 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 4: conflicting requests 2024-04-12T20:41:12.307 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module gimp:2.8:8000020190628145146:4148dfdf.x86_64 from CentOS-AppStream 2024-04-12T20:41:12.308 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 5: conflicting requests 2024-04-12T20:41:12.308 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module go-toolset:rhel8:820240401160413:b754926a.x86_64 from CentOS-AppStream 2024-04-12T20:41:12.308 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 6: conflicting requests 2024-04-12T20:41:12.308 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module httpd:2.4:820240216084734:9edba152.x86_64 from CentOS-AppStream 2024-04-12T20:41:12.308 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 7: conflicting requests 2024-04-12T20:41:12.308 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module idm:client:820240307184541:49cc9d1b.x86_64 from CentOS-AppStream 2024-04-12T20:41:12.308 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 8: conflicting requests 2024-04-12T20:41:12.308 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module inkscape:0.92.3:8080020221205124429:3e031279.x86_64 from CentOS-AppStream 2024-04-12T20:41:12.308 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 9: conflicting requests 2024-04-12T20:41:12.308 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module jmc:rhel8:8050020211110222101:6392b1f8.x86_64 from CentOS-AppStream 2024-04-12T20:41:12.308 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 10: conflicting requests 2024-04-12T20:41:12.309 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module llvm-toolset:rhel8:820240205133135:9edba152.x86_64 from CentOS-AppStream 2024-04-12T20:41:12.309 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 11: conflicting requests 2024-04-12T20:41:12.309 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module log4j:2:8080020221205124743:9d367344.x86_64 from CentOS-AppStream 2024-04-12T20:41:12.309 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 12: conflicting requests 2024-04-12T20:41:12.309 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module mailman:2.1:820230727085757:77fc8825.x86_64 from CentOS-AppStream 2024-04-12T20:41:12.309 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 13: conflicting requests 2024-04-12T20:41:12.309 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module mariadb:10.3:8030020210419150013:30b713e6.x86_64 from CentOS-AppStream 2024-04-12T20:41:12.309 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 14: conflicting requests 2024-04-12T20:41:12.309 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module maven:3.5:8060020211203142015:c0229ad2.x86_64 from CentOS-AppStream 2024-04-12T20:41:12.309 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 15: conflicting requests 2024-04-12T20:41:12.310 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module mercurial:4.8:8000020190628020724:4148dfdf.x86_64 from CentOS-AppStream 2024-04-12T20:41:12.310 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 16: conflicting requests 2024-04-12T20:41:12.310 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module mod_auth_openidc:2.3:820230721123832:9edba152.x86_64 from CentOS-AppStream 2024-04-12T20:41:12.310 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 17: conflicting requests 2024-04-12T20:41:12.310 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module mysql:8.0:8040020210901180257:9f9e2e7e.x86_64 from CentOS-AppStream 2024-04-12T20:41:12.310 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 18: conflicting requests 2024-04-12T20:41:12.310 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module nginx:1.14:8000020211221191913:55190bc5.x86_64 from CentOS-AppStream 2024-04-12T20:41:12.310 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 19: conflicting requests 2024-04-12T20:41:12.310 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module nodejs:10:8040020210120182536:9f9e2e7e.x86_64 from CentOS-AppStream 2024-04-12T20:41:12.310 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 20: conflicting requests 2024-04-12T20:41:12.310 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl:5.26:8000020190628020724:55190bc5.x86_64 from CentOS-AppStream 2024-04-12T20:41:12.311 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 21: conflicting requests 2024-04-12T20:41:12.311 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module php:7.2:8020020200507003613:2c7ca891.x86_64 from CentOS-AppStream 2024-04-12T20:41:12.311 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 22: conflicting requests 2024-04-12T20:41:12.311 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module postgresql:10:820230223114800:9edba152.x86_64 from CentOS-AppStream 2024-04-12T20:41:12.311 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 23: conflicting requests 2024-04-12T20:41:12.311 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module python27:2.7:820240208011952:182f7c73.x86_64 from CentOS-AppStream 2024-04-12T20:41:12.311 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 24: conflicting requests 2024-04-12T20:41:12.311 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module python36:3.6:8050020210825152031:982725ab.x86_64 from CentOS-AppStream 2024-04-12T20:41:12.311 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 25: conflicting requests 2024-04-12T20:41:12.311 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module python38:3.8:820230810143931:64c1cd5a.x86_64 from CentOS-AppStream 2024-04-12T20:41:12.311 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 26: conflicting requests 2024-04-12T20:41:12.311 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module python39:3.9:820240214182535:17377f89.x86_64 from CentOS-AppStream 2024-04-12T20:41:12.311 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 27: conflicting requests 2024-04-12T20:41:12.312 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module redis:5:8040020211019153849:9f9e2e7e.x86_64 from CentOS-AppStream 2024-04-12T20:41:12.312 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 28: conflicting requests 2024-04-12T20:41:12.312 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module rhn-tools:1.0:8010020191114034948:f69d1239.x86_64 from CentOS-AppStream 2024-04-12T20:41:12.312 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 29: conflicting requests 2024-04-12T20:41:12.312 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module ruby:2.5:820230627084142:9edba152.x86_64 from CentOS-AppStream 2024-04-12T20:41:12.312 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 30: conflicting requests 2024-04-12T20:41:12.312 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module rust-toolset:rhel8:820240119204620:b09eea91.x86_64 from CentOS-AppStream 2024-04-12T20:41:12.312 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 31: conflicting requests 2024-04-12T20:41:12.312 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module satellite-5-client:1.0:8010020191114035551:cdc1202b.x86_64 from CentOS-AppStream 2024-04-12T20:41:12.312 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 32: conflicting requests 2024-04-12T20:41:12.312 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module squid:4:820240319173245:9edba152.x86_64 from CentOS-AppStream 2024-04-12T20:41:12.312 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 33: conflicting requests 2024-04-12T20:41:12.313 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module subversion:1.10:8070020220711155714:78111232.x86_64 from CentOS-AppStream 2024-04-12T20:41:12.313 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 34: conflicting requests 2024-04-12T20:41:12.313 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module swig:3.0:8030020200706142531:30b713e6.x86_64 from CentOS-AppStream 2024-04-12T20:41:12.313 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 35: conflicting requests 2024-04-12T20:41:12.313 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module varnish:6:8050020211109225449:b4937e53.x86_64 from CentOS-AppStream 2024-04-12T20:41:12.313 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 36: conflicting requests 2024-04-12T20:41:12.313 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module virt:rhel:820240314161907:9edba152.x86_64 from CentOS-AppStream 2024-04-12T20:41:12.313 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 37: conflicting requests 2024-04-12T20:41:12.313 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:8030020200716155257:7cc0a66d.x86_64 from CentOS-AppStream 2024-04-12T20:41:12.313 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:8030020200716155257:b967a9a2.x86_64 from CentOS-AppStream 2024-04-12T20:41:12.313 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:820230811133638:36f9fae6.x86_64 from CentOS-AppStream 2024-04-12T20:41:12.313 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 38: cannot install the best candidate for the job 2024-04-12T20:41:12.314 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:8010020191114031501:a5949e2e.x86_64 from CentOS-AppStream 2024-04-12T20:41:12.314 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:8010020191114031501:be2e4737.x86_64 from CentOS-AppStream 2024-04-12T20:41:12.314 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:820230223114759:2ab761e1.x86_64 from CentOS-AppStream 2024-04-12T20:41:12.314 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:820230223114759:86e45846.x86_64 from CentOS-AppStream 2024-04-12T20:41:12.314 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 39: conflicting requests 2024-04-12T20:41:12.314 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:8030020200715230104:1e4bbb35.x86_64 from CentOS-AppStream 2024-04-12T20:41:12.314 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:8030020200715230104:ea09926d.x86_64 from CentOS-AppStream 2024-04-12T20:41:12.314 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:082fdf2f.x86_64 from CentOS-AppStream 2024-04-12T20:41:12.314 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-04-12T20:41:12.314 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-04-12T20:41:12.315 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:fbe42456.x86_64 from CentOS-AppStream 2024-04-12T20:41:12.315 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 40: cannot install the best candidate for the job 2024-04-12T20:41:12.315 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:8010020191114031513:16b3ab4d.x86_64 from CentOS-AppStream 2024-04-12T20:41:12.315 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:8010020191114031513:e1f37755.x86_64 from CentOS-AppStream 2024-04-12T20:41:12.315 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-04-12T20:41:12.315 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-04-12T20:41:12.315 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 41: cannot install the best candidate for the job 2024-04-12T20:41:12.315 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:8030020200716150652:1e4bbb35.x86_64 from CentOS-AppStream 2024-04-12T20:41:12.315 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:8030020200716150652:ea09926d.x86_64 from CentOS-AppStream 2024-04-12T20:41:12.315 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-04-12T20:41:12.315 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-04-12T20:41:12.315 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 42: cannot install the best candidate for the job 2024-04-12T20:41:12.316 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:8030020200716174729:3a70019f.x86_64 from CentOS-AppStream 2024-04-12T20:41:12.316 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:8030020200716174729:cccafca5.x86_64 from CentOS-AppStream 2024-04-12T20:41:12.316 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:820230223114759:6fdf7e5d.x86_64 from CentOS-AppStream 2024-04-12T20:41:12.316 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:820230223114759:d4326aba.x86_64 from CentOS-AppStream 2024-04-12T20:41:12.316 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 43: cannot install the best candidate for the job 2024-04-12T20:41:12.316 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:8030020200716213634:56fce90f.x86_64 from CentOS-AppStream 2024-04-12T20:41:12.316 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:8030020200716213634:98a3c9d0.x86_64 from CentOS-AppStream 2024-04-12T20:41:12.316 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:820230223114759:06b7596e.x86_64 from CentOS-AppStream 2024-04-12T20:41:12.316 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:820230223114759:d2fc5d76.x86_64 from CentOS-AppStream 2024-04-12T20:41:12.316 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 44: cannot install the best candidate for the job 2024-04-12T20:41:12.316 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:8030020200716171139:3a70019f.x86_64 from CentOS-AppStream 2024-04-12T20:41:12.316 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:8030020200716171139:cccafca5.x86_64 from CentOS-AppStream 2024-04-12T20:41:12.316 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:820230223114759:6fdf7e5d.x86_64 from CentOS-AppStream 2024-04-12T20:41:12.317 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:820230223114759:d4326aba.x86_64 from CentOS-AppStream 2024-04-12T20:41:12.317 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 45: cannot install the best candidate for the job 2024-04-12T20:41:12.317 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:8010020191120175858:95fce66d.x86_64 from CentOS-AppStream 2024-04-12T20:41:12.317 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:8030020200723233049:09acf126.x86_64 from CentOS-AppStream 2024-04-12T20:41:12.317 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:820230223114759:436af1ae.x86_64 from CentOS-AppStream 2024-04-12T20:41:12.317 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:820230223114759:b12195ea.x86_64 from CentOS-AppStream 2024-04-12T20:41:12.381 INFO:teuthology.orchestra.run.smithi018.stdout:Dependencies resolved. 2024-04-12T20:41:12.383 INFO:teuthology.orchestra.run.smithi018.stdout:================================================================================ 2024-04-12T20:41:12.383 INFO:teuthology.orchestra.run.smithi018.stdout: Package Arch Version Repository Size 2024-04-12T20:41:12.383 INFO:teuthology.orchestra.run.smithi018.stdout:================================================================================ 2024-04-12T20:41:12.383 INFO:teuthology.orchestra.run.smithi018.stdout:Installing: 2024-04-12T20:41:12.383 INFO:teuthology.orchestra.run.smithi018.stdout: ceph-mgr-rook noarch 2:17.2.7-827.gd3b5060a.el9 ceph-noarch 47 k 2024-04-12T20:41:12.383 INFO:teuthology.orchestra.run.smithi018.stdout:Installing dependencies: 2024-04-12T20:41:12.383 INFO:teuthology.orchestra.run.smithi018.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-04-12T20:41:12.383 INFO:teuthology.orchestra.run.smithi018.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-04-12T20:41:12.383 INFO:teuthology.orchestra.run.smithi018.stdout: python3-google-auth noarch 1:2.28.2-1.el9 epel 203 k 2024-04-12T20:41:12.383 INFO:teuthology.orchestra.run.smithi018.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-04-12T20:41:12.383 INFO:teuthology.orchestra.run.smithi018.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-04-12T20:41:12.383 INFO:teuthology.orchestra.run.smithi018.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-04-12T20:41:12.383 INFO:teuthology.orchestra.run.smithi018.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-04-12T20:41:12.384 INFO:teuthology.orchestra.run.smithi018.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-04-12T20:41:12.384 INFO:teuthology.orchestra.run.smithi018.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-04-12T20:41:12.384 INFO:teuthology.orchestra.run.smithi018.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-04-12T20:41:12.384 INFO:teuthology.orchestra.run.smithi018.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-04-12T20:41:12.384 INFO:teuthology.orchestra.run.smithi018.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-04-12T20:41:12.384 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-12T20:41:12.384 INFO:teuthology.orchestra.run.smithi018.stdout:Transaction Summary 2024-04-12T20:41:12.384 INFO:teuthology.orchestra.run.smithi018.stdout:================================================================================ 2024-04-12T20:41:12.384 INFO:teuthology.orchestra.run.smithi018.stdout:Install 13 Packages 2024-04-12T20:41:12.384 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-12T20:41:12.385 INFO:teuthology.orchestra.run.smithi018.stdout:Total download size: 2.2 M 2024-04-12T20:41:12.385 INFO:teuthology.orchestra.run.smithi018.stdout:Installed size: 26 M 2024-04-12T20:41:12.385 INFO:teuthology.orchestra.run.smithi018.stdout:Downloading Packages: 2024-04-12T20:41:12.904 INFO:teuthology.orchestra.run.smithi186.stdout:(1/13): ceph-mgr-rook-17.2.7-827.gd3b5060a.el9. 154 kB/s | 47 kB 00:00 2024-04-12T20:41:12.962 INFO:teuthology.orchestra.run.smithi186.stdout:(2/13): python3-jsonpointer-2.0-4.el9.noarch.rp 52 kB/s | 19 kB 00:00 2024-04-12T20:41:12.996 INFO:teuthology.orchestra.run.smithi186.stdout:(3/13): python3-jsonpatch-1.21-16.el9.noarch.rp 64 kB/s | 26 kB 00:00 2024-04-12T20:41:13.020 INFO:teuthology.orchestra.run.smithi018.stdout:(1/13): python3-jsonpatch-1.21-16.el9.noarch.rp 139 kB/s | 26 kB 00:00 2024-04-12T20:41:13.095 INFO:teuthology.orchestra.run.smithi018.stdout:(2/13): python3-jsonpointer-2.0-4.el9.noarch.rp 74 kB/s | 19 kB 00:00 2024-04-12T20:41:13.180 INFO:teuthology.orchestra.run.smithi186.stdout:(4/13): python3-oauthlib-3.1.1-5.el9.noarch.rpm 813 kB/s | 222 kB 00:00 2024-04-12T20:41:13.205 INFO:teuthology.orchestra.run.smithi186.stdout:(5/13): python3-pyasn1-0.4.8-6.el9.noarch.rpm 658 kB/s | 159 kB 00:00 2024-04-12T20:41:13.295 INFO:teuthology.orchestra.run.smithi018.stdout:(3/13): python3-pyasn1-0.4.8-6.el9.noarch.rpm 810 kB/s | 159 kB 00:00 2024-04-12T20:41:13.297 INFO:teuthology.orchestra.run.smithi186.stdout:(6/13): python3-requests-oauthlib-1.3.0-12.el9. 460 kB/s | 54 kB 00:00 2024-04-12T20:41:13.362 INFO:teuthology.orchestra.run.smithi018.stdout:(4/13): python3-oauthlib-3.1.1-5.el9.noarch.rpm 844 kB/s | 222 kB 00:00 2024-04-12T20:41:13.404 INFO:teuthology.orchestra.run.smithi018.stdout:(5/13): ceph-mgr-rook-17.2.7-827.gd3b5060a.el9. 83 kB/s | 47 kB 00:00 2024-04-12T20:41:13.438 INFO:teuthology.orchestra.run.smithi018.stdout:(6/13): python3-pyasn1-modules-0.4.8-6.el9.noar 1.9 MB/s | 279 kB 00:00 2024-04-12T20:41:13.504 INFO:teuthology.orchestra.run.smithi018.stdout:(7/13): python3-requests-oauthlib-1.3.0-12.el9. 380 kB/s | 54 kB 00:00 2024-04-12T20:41:13.572 INFO:teuthology.orchestra.run.smithi186.stdout:(7/13): python3-pyasn1-modules-0.4.8-6.el9.noar 485 kB/s | 279 kB 00:00 2024-04-12T20:41:13.772 INFO:teuthology.orchestra.run.smithi018.stdout:(8/13): python3-certifi-2021.10.8-2.el9.noarch. 56 kB/s | 15 kB 00:00 2024-04-12T20:41:13.772 INFO:teuthology.orchestra.run.smithi186.stdout:(8/13): python3-cachetools-4.2.4-1.el9.noarch.r 57 kB/s | 32 kB 00:00 2024-04-12T20:41:13.798 INFO:teuthology.orchestra.run.smithi186.stdout:(9/13): python3-certifi-2021.10.8-2.el9.noarch. 30 kB/s | 15 kB 00:00 2024-04-12T20:41:13.813 INFO:teuthology.orchestra.run.smithi018.stdout:(9/13): python3-cachetools-4.2.4-1.el9.noarch.r 79 kB/s | 32 kB 00:00 2024-04-12T20:41:14.014 INFO:teuthology.orchestra.run.smithi018.stdout:(10/13): python3-google-auth-2.28.2-1.el9.noarc 399 kB/s | 203 kB 00:00 2024-04-12T20:41:14.056 INFO:teuthology.orchestra.run.smithi018.stdout:(11/13): python3-rsa-4.9-2.el9.noarch.rpm 245 kB/s | 59 kB 00:00 2024-04-12T20:41:14.082 INFO:teuthology.orchestra.run.smithi186.stdout:(10/13): python3-rsa-4.9-2.el9.noarch.rpm 209 kB/s | 59 kB 00:00 2024-04-12T20:41:14.149 INFO:teuthology.orchestra.run.smithi018.stdout:(12/13): python3-websocket-client-1.2.3-2.el9.n 676 kB/s | 90 kB 00:00 2024-04-12T20:41:14.316 INFO:teuthology.orchestra.run.smithi186.stdout:(11/13): python3-websocket-client-1.2.3-2.el9.n 386 kB/s | 90 kB 00:00 2024-04-12T20:41:14.331 INFO:teuthology.orchestra.run.smithi018.stdout:(13/13): python3-kubernetes-26.1.0-2.el9.noarch 1.8 MB/s | 1.0 MB 00:00 2024-04-12T20:41:14.332 INFO:teuthology.orchestra.run.smithi018.stdout:-------------------------------------------------------------------------------- 2024-04-12T20:41:14.333 INFO:teuthology.orchestra.run.smithi018.stdout:Total 1.1 MB/s | 2.2 MB 00:01 2024-04-12T20:41:14.349 INFO:teuthology.orchestra.run.smithi186.stdout:(12/13): python3-google-auth-2.28.2-1.el9.noarc 262 kB/s | 203 kB 00:00 2024-04-12T20:41:14.449 INFO:teuthology.orchestra.run.smithi186.stdout:(13/13): python3-kubernetes-26.1.0-2.el9.noarch 1.5 MB/s | 1.0 MB 00:00 2024-04-12T20:41:14.462 INFO:teuthology.orchestra.run.smithi186.stdout:-------------------------------------------------------------------------------- 2024-04-12T20:41:14.463 INFO:teuthology.orchestra.run.smithi186.stdout:Total 982 kB/s | 2.2 MB 00:02 2024-04-12T20:41:14.564 INFO:teuthology.orchestra.run.smithi018.stdout:Running transaction check 2024-04-12T20:41:14.612 INFO:teuthology.orchestra.run.smithi018.stdout:Transaction check succeeded. 2024-04-12T20:41:14.612 INFO:teuthology.orchestra.run.smithi018.stdout:Running transaction test 2024-04-12T20:41:14.680 INFO:teuthology.orchestra.run.smithi186.stdout:Running transaction check 2024-04-12T20:41:14.727 INFO:teuthology.orchestra.run.smithi186.stdout:Transaction check succeeded. 2024-04-12T20:41:14.727 INFO:teuthology.orchestra.run.smithi186.stdout:Running transaction test 2024-04-12T20:41:14.781 INFO:teuthology.orchestra.run.smithi018.stdout:Transaction test succeeded. 2024-04-12T20:41:14.784 INFO:teuthology.orchestra.run.smithi018.stdout:Running transaction 2024-04-12T20:41:14.891 INFO:teuthology.orchestra.run.smithi186.stdout:Transaction test succeeded. 2024-04-12T20:41:14.892 INFO:teuthology.orchestra.run.smithi186.stdout:Running transaction 2024-04-12T20:41:15.470 INFO:teuthology.orchestra.run.smithi018.stdout: Preparing : 1/1 2024-04-12T20:41:15.574 INFO:teuthology.orchestra.run.smithi186.stdout: Preparing : 1/1 2024-04-12T20:41:15.707 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 1/13 2024-04-12T20:41:15.776 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 2/13 2024-04-12T20:41:15.825 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 1/13 2024-04-12T20:41:15.849 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : python3-rsa-4.9-2.el9.noarch 3/13 2024-04-12T20:41:15.887 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 4/13 2024-04-12T20:41:15.893 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 2/13 2024-04-12T20:41:15.942 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 5/13 2024-04-12T20:41:15.960 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-rsa-4.9-2.el9.noarch 3/13 2024-04-12T20:41:16.005 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 4/13 2024-04-12T20:41:16.049 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 5/13 2024-04-12T20:41:16.075 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 6/13 2024-04-12T20:41:16.186 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 6/13 2024-04-12T20:41:16.250 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : python3-google-auth-1:2.28.2-1.el9.noarch 7/13 2024-04-12T20:41:16.319 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 8/13 2024-04-12T20:41:16.325 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-google-auth-1:2.28.2-1.el9.noarch 7/13 2024-04-12T20:41:16.395 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 8/13 2024-04-12T20:41:17.303 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 9/13 2024-04-12T20:41:17.324 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 9/13 2024-04-12T20:41:17.360 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 10/13 2024-04-12T20:41:17.393 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 11/13 2024-04-12T20:41:17.402 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 10/13 2024-04-12T20:41:17.435 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 11/13 2024-04-12T20:41:17.446 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 12/13 2024-04-12T20:41:17.483 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 12/13 2024-04-12T20:41:17.485 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : ceph-mgr-rook-2:17.2.7-827.gd3b5060a.el9.noarch 13/13 2024-04-12T20:41:17.511 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : ceph-mgr-rook-2:17.2.7-827.gd3b5060a.el9.noarch 13/13 2024-04-12T20:41:18.480 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: ceph-mgr-rook-2:17.2.7-827.gd3b5060a.el9.noarch 13/13 2024-04-12T20:41:18.480 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : ceph-mgr-rook-2:17.2.7-827.gd3b5060a.el9.noarch 1/13 2024-04-12T20:41:18.481 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 2/13 2024-04-12T20:41:18.481 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 3/13 2024-04-12T20:41:18.481 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 4/13 2024-04-12T20:41:18.481 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 5/13 2024-04-12T20:41:18.481 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 6/13 2024-04-12T20:41:18.481 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 7/13 2024-04-12T20:41:18.481 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 8/13 2024-04-12T20:41:18.481 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 9/13 2024-04-12T20:41:18.481 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-google-auth-1:2.28.2-1.el9.noarch 10/13 2024-04-12T20:41:18.481 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 11/13 2024-04-12T20:41:18.481 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 12/13 2024-04-12T20:41:18.485 INFO:teuthology.orchestra.run.smithi018.stdout: Running scriptlet: ceph-mgr-rook-2:17.2.7-827.gd3b5060a.el9.noarch 13/13 2024-04-12T20:41:18.485 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : ceph-mgr-rook-2:17.2.7-827.gd3b5060a.el9.noarch 1/13 2024-04-12T20:41:18.485 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 2/13 2024-04-12T20:41:18.485 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 3/13 2024-04-12T20:41:18.485 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 4/13 2024-04-12T20:41:18.485 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 5/13 2024-04-12T20:41:18.485 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 6/13 2024-04-12T20:41:18.486 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 7/13 2024-04-12T20:41:18.486 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 8/13 2024-04-12T20:41:18.486 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 9/13 2024-04-12T20:41:18.486 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : python3-google-auth-1:2.28.2-1.el9.noarch 10/13 2024-04-12T20:41:18.486 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 11/13 2024-04-12T20:41:18.486 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 12/13 2024-04-12T20:41:18.956 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 13/13 2024-04-12T20:41:18.956 INFO:teuthology.orchestra.run.smithi186.stdout: 2024-04-12T20:41:18.956 INFO:teuthology.orchestra.run.smithi186.stdout:Installed: 2024-04-12T20:41:18.956 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-mgr-rook-2:17.2.7-827.gd3b5060a.el9.noarch 2024-04-12T20:41:18.956 INFO:teuthology.orchestra.run.smithi186.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-04-12T20:41:18.956 INFO:teuthology.orchestra.run.smithi186.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-04-12T20:41:18.956 INFO:teuthology.orchestra.run.smithi186.stdout: python3-google-auth-1:2.28.2-1.el9.noarch 2024-04-12T20:41:18.956 INFO:teuthology.orchestra.run.smithi186.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-04-12T20:41:18.957 INFO:teuthology.orchestra.run.smithi186.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-04-12T20:41:18.957 INFO:teuthology.orchestra.run.smithi186.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-04-12T20:41:18.957 INFO:teuthology.orchestra.run.smithi186.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-04-12T20:41:18.957 INFO:teuthology.orchestra.run.smithi186.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-04-12T20:41:18.957 INFO:teuthology.orchestra.run.smithi186.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-04-12T20:41:18.957 INFO:teuthology.orchestra.run.smithi186.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-04-12T20:41:18.957 INFO:teuthology.orchestra.run.smithi186.stdout: python3-rsa-4.9-2.el9.noarch 2024-04-12T20:41:18.957 INFO:teuthology.orchestra.run.smithi186.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-04-12T20:41:18.957 INFO:teuthology.orchestra.run.smithi186.stdout: 2024-04-12T20:41:18.957 INFO:teuthology.orchestra.run.smithi186.stdout:Complete! 2024-04-12T20:41:19.014 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 13/13 2024-04-12T20:41:19.014 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-12T20:41:19.014 INFO:teuthology.orchestra.run.smithi018.stdout:Installed: 2024-04-12T20:41:19.014 INFO:teuthology.orchestra.run.smithi018.stdout: ceph-mgr-rook-2:17.2.7-827.gd3b5060a.el9.noarch 2024-04-12T20:41:19.014 INFO:teuthology.orchestra.run.smithi018.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-04-12T20:41:19.015 INFO:teuthology.orchestra.run.smithi018.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-04-12T20:41:19.015 INFO:teuthology.orchestra.run.smithi018.stdout: python3-google-auth-1:2.28.2-1.el9.noarch 2024-04-12T20:41:19.015 INFO:teuthology.orchestra.run.smithi018.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-04-12T20:41:19.015 INFO:teuthology.orchestra.run.smithi018.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-04-12T20:41:19.015 INFO:teuthology.orchestra.run.smithi018.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-04-12T20:41:19.015 INFO:teuthology.orchestra.run.smithi018.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-04-12T20:41:19.015 INFO:teuthology.orchestra.run.smithi018.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-04-12T20:41:19.015 INFO:teuthology.orchestra.run.smithi018.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-04-12T20:41:19.015 INFO:teuthology.orchestra.run.smithi018.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-04-12T20:41:19.015 INFO:teuthology.orchestra.run.smithi018.stdout: python3-rsa-4.9-2.el9.noarch 2024-04-12T20:41:19.015 INFO:teuthology.orchestra.run.smithi018.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-04-12T20:41:19.015 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-12T20:41:19.015 INFO:teuthology.orchestra.run.smithi018.stdout:Complete! 2024-04-12T20:41:19.280 DEBUG:teuthology.orchestra.run.smithi186:> sudo yum -y install ceph-mgr-cephadm 2024-04-12T20:41:19.368 DEBUG:teuthology.orchestra.run.smithi018:> sudo yum -y install ceph-mgr-cephadm 2024-04-12T20:41:20.184 INFO:teuthology.orchestra.run.smithi186.stdout:Last metadata expiration check: 0:01:43 ago on Fri 12 Apr 2024 08:39:37 PM UTC. 2024-04-12T20:41:20.339 INFO:teuthology.orchestra.run.smithi018.stdout:Last metadata expiration check: 0:01:45 ago on Fri 12 Apr 2024 08:39:35 PM UTC. 2024-04-12T20:41:20.479 INFO:teuthology.orchestra.run.smithi186.stderr:Modular dependency problems: 2024-04-12T20:41:20.479 INFO:teuthology.orchestra.run.smithi186.stderr: 2024-04-12T20:41:20.479 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 1: conflicting requests 2024-04-12T20:41:20.480 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module container-tools:rhel8:820240321091303:20125149.x86_64 from CentOS-AppStream 2024-04-12T20:41:20.480 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 2: conflicting requests 2024-04-12T20:41:20.480 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module eclipse:rhel8:820201023100746:b230ed4e.x86_64 from CentOS-AppStream 2024-04-12T20:41:20.480 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 3: conflicting requests 2024-04-12T20:41:20.480 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module freeradius:3.0:8080020230117180605:89170a74.x86_64 from CentOS-AppStream 2024-04-12T20:41:20.480 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 4: conflicting requests 2024-04-12T20:41:20.480 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module gimp:2.8:8000020190628145146:4148dfdf.x86_64 from CentOS-AppStream 2024-04-12T20:41:20.480 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 5: conflicting requests 2024-04-12T20:41:20.480 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module go-toolset:rhel8:820240401160413:b754926a.x86_64 from CentOS-AppStream 2024-04-12T20:41:20.480 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 6: conflicting requests 2024-04-12T20:41:20.480 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module httpd:2.4:820240216084734:9edba152.x86_64 from CentOS-AppStream 2024-04-12T20:41:20.480 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 7: conflicting requests 2024-04-12T20:41:20.480 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module idm:client:820240307184541:49cc9d1b.x86_64 from CentOS-AppStream 2024-04-12T20:41:20.481 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 8: conflicting requests 2024-04-12T20:41:20.481 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module inkscape:0.92.3:8080020221205124429:3e031279.x86_64 from CentOS-AppStream 2024-04-12T20:41:20.481 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 9: conflicting requests 2024-04-12T20:41:20.481 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module jmc:rhel8:8050020211110222101:6392b1f8.x86_64 from CentOS-AppStream 2024-04-12T20:41:20.481 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 10: conflicting requests 2024-04-12T20:41:20.481 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module llvm-toolset:rhel8:820240205133135:9edba152.x86_64 from CentOS-AppStream 2024-04-12T20:41:20.481 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 11: conflicting requests 2024-04-12T20:41:20.481 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module log4j:2:8080020221205124743:9d367344.x86_64 from CentOS-AppStream 2024-04-12T20:41:20.481 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 12: conflicting requests 2024-04-12T20:41:20.481 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module mailman:2.1:820230727085757:77fc8825.x86_64 from CentOS-AppStream 2024-04-12T20:41:20.481 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 13: conflicting requests 2024-04-12T20:41:20.481 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module mariadb:10.3:8030020210419150013:30b713e6.x86_64 from CentOS-AppStream 2024-04-12T20:41:20.481 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 14: conflicting requests 2024-04-12T20:41:20.482 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module maven:3.5:8060020211203142015:c0229ad2.x86_64 from CentOS-AppStream 2024-04-12T20:41:20.482 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 15: conflicting requests 2024-04-12T20:41:20.482 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module mercurial:4.8:8000020190628020724:4148dfdf.x86_64 from CentOS-AppStream 2024-04-12T20:41:20.482 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 16: conflicting requests 2024-04-12T20:41:20.482 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module mod_auth_openidc:2.3:820230721123832:9edba152.x86_64 from CentOS-AppStream 2024-04-12T20:41:20.482 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 17: conflicting requests 2024-04-12T20:41:20.482 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module mysql:8.0:8040020210901180257:9f9e2e7e.x86_64 from CentOS-AppStream 2024-04-12T20:41:20.482 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 18: conflicting requests 2024-04-12T20:41:20.482 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module nginx:1.14:8000020211221191913:55190bc5.x86_64 from CentOS-AppStream 2024-04-12T20:41:20.482 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 19: conflicting requests 2024-04-12T20:41:20.482 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module nodejs:10:8040020210120182536:9f9e2e7e.x86_64 from CentOS-AppStream 2024-04-12T20:41:20.482 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 20: conflicting requests 2024-04-12T20:41:20.482 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl:5.26:8000020190628020724:55190bc5.x86_64 from CentOS-AppStream 2024-04-12T20:41:20.483 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 21: conflicting requests 2024-04-12T20:41:20.483 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module php:7.2:8020020200507003613:2c7ca891.x86_64 from CentOS-AppStream 2024-04-12T20:41:20.483 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 22: conflicting requests 2024-04-12T20:41:20.483 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module postgresql:10:820230223114800:9edba152.x86_64 from CentOS-AppStream 2024-04-12T20:41:20.483 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 23: conflicting requests 2024-04-12T20:41:20.483 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module python27:2.7:820240208011952:182f7c73.x86_64 from CentOS-AppStream 2024-04-12T20:41:20.483 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 24: conflicting requests 2024-04-12T20:41:20.483 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module python36:3.6:8050020210825152031:982725ab.x86_64 from CentOS-AppStream 2024-04-12T20:41:20.483 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 25: conflicting requests 2024-04-12T20:41:20.483 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module python38:3.8:820230810143931:64c1cd5a.x86_64 from CentOS-AppStream 2024-04-12T20:41:20.483 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 26: conflicting requests 2024-04-12T20:41:20.483 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module python39:3.9:820240214182535:17377f89.x86_64 from CentOS-AppStream 2024-04-12T20:41:20.483 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 27: conflicting requests 2024-04-12T20:41:20.484 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module redis:5:8040020211019153849:9f9e2e7e.x86_64 from CentOS-AppStream 2024-04-12T20:41:20.484 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 28: conflicting requests 2024-04-12T20:41:20.484 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module rhn-tools:1.0:8010020191114034948:f69d1239.x86_64 from CentOS-AppStream 2024-04-12T20:41:20.484 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 29: conflicting requests 2024-04-12T20:41:20.484 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module ruby:2.5:820230627084142:9edba152.x86_64 from CentOS-AppStream 2024-04-12T20:41:20.484 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 30: conflicting requests 2024-04-12T20:41:20.484 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module rust-toolset:rhel8:820240119204620:b09eea91.x86_64 from CentOS-AppStream 2024-04-12T20:41:20.484 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 31: conflicting requests 2024-04-12T20:41:20.484 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module satellite-5-client:1.0:8010020191114035551:cdc1202b.x86_64 from CentOS-AppStream 2024-04-12T20:41:20.484 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 32: conflicting requests 2024-04-12T20:41:20.484 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module squid:4:820240319173245:9edba152.x86_64 from CentOS-AppStream 2024-04-12T20:41:20.484 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 33: conflicting requests 2024-04-12T20:41:20.484 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module subversion:1.10:8070020220711155714:78111232.x86_64 from CentOS-AppStream 2024-04-12T20:41:20.484 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 34: conflicting requests 2024-04-12T20:41:20.485 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module swig:3.0:8030020200706142531:30b713e6.x86_64 from CentOS-AppStream 2024-04-12T20:41:20.485 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 35: conflicting requests 2024-04-12T20:41:20.485 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module varnish:6:8050020211109225449:b4937e53.x86_64 from CentOS-AppStream 2024-04-12T20:41:20.485 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 36: conflicting requests 2024-04-12T20:41:20.485 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module virt:rhel:820240314161907:9edba152.x86_64 from CentOS-AppStream 2024-04-12T20:41:20.485 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 37: conflicting requests 2024-04-12T20:41:20.485 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:8030020200716155257:7cc0a66d.x86_64 from CentOS-AppStream 2024-04-12T20:41:20.485 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:8030020200716155257:b967a9a2.x86_64 from CentOS-AppStream 2024-04-12T20:41:20.485 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:820230811133638:36f9fae6.x86_64 from CentOS-AppStream 2024-04-12T20:41:20.485 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 38: cannot install the best candidate for the job 2024-04-12T20:41:20.485 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:8010020191114031501:a5949e2e.x86_64 from CentOS-AppStream 2024-04-12T20:41:20.485 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:8010020191114031501:be2e4737.x86_64 from CentOS-AppStream 2024-04-12T20:41:20.486 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:820230223114759:2ab761e1.x86_64 from CentOS-AppStream 2024-04-12T20:41:20.486 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:820230223114759:86e45846.x86_64 from CentOS-AppStream 2024-04-12T20:41:20.486 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 39: conflicting requests 2024-04-12T20:41:20.486 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:8030020200715230104:1e4bbb35.x86_64 from CentOS-AppStream 2024-04-12T20:41:20.486 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:8030020200715230104:ea09926d.x86_64 from CentOS-AppStream 2024-04-12T20:41:20.486 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:082fdf2f.x86_64 from CentOS-AppStream 2024-04-12T20:41:20.486 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-04-12T20:41:20.486 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-04-12T20:41:20.486 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:fbe42456.x86_64 from CentOS-AppStream 2024-04-12T20:41:20.486 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 40: cannot install the best candidate for the job 2024-04-12T20:41:20.486 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:8010020191114031513:16b3ab4d.x86_64 from CentOS-AppStream 2024-04-12T20:41:20.486 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:8010020191114031513:e1f37755.x86_64 from CentOS-AppStream 2024-04-12T20:41:20.487 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-04-12T20:41:20.487 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-04-12T20:41:20.487 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 41: cannot install the best candidate for the job 2024-04-12T20:41:20.487 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:8030020200716150652:1e4bbb35.x86_64 from CentOS-AppStream 2024-04-12T20:41:20.487 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:8030020200716150652:ea09926d.x86_64 from CentOS-AppStream 2024-04-12T20:41:20.487 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-04-12T20:41:20.487 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-04-12T20:41:20.487 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 42: cannot install the best candidate for the job 2024-04-12T20:41:20.487 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:8030020200716174729:3a70019f.x86_64 from CentOS-AppStream 2024-04-12T20:41:20.487 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:8030020200716174729:cccafca5.x86_64 from CentOS-AppStream 2024-04-12T20:41:20.487 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:820230223114759:6fdf7e5d.x86_64 from CentOS-AppStream 2024-04-12T20:41:20.487 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:820230223114759:d4326aba.x86_64 from CentOS-AppStream 2024-04-12T20:41:20.487 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 43: cannot install the best candidate for the job 2024-04-12T20:41:20.488 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:8030020200716213634:56fce90f.x86_64 from CentOS-AppStream 2024-04-12T20:41:20.488 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:8030020200716213634:98a3c9d0.x86_64 from CentOS-AppStream 2024-04-12T20:41:20.488 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:820230223114759:06b7596e.x86_64 from CentOS-AppStream 2024-04-12T20:41:20.488 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:820230223114759:d2fc5d76.x86_64 from CentOS-AppStream 2024-04-12T20:41:20.488 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 44: cannot install the best candidate for the job 2024-04-12T20:41:20.488 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:8030020200716171139:3a70019f.x86_64 from CentOS-AppStream 2024-04-12T20:41:20.488 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:8030020200716171139:cccafca5.x86_64 from CentOS-AppStream 2024-04-12T20:41:20.488 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:820230223114759:6fdf7e5d.x86_64 from CentOS-AppStream 2024-04-12T20:41:20.488 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:820230223114759:d4326aba.x86_64 from CentOS-AppStream 2024-04-12T20:41:20.488 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 45: cannot install the best candidate for the job 2024-04-12T20:41:20.488 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:8010020191120175858:95fce66d.x86_64 from CentOS-AppStream 2024-04-12T20:41:20.488 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:8030020200723233049:09acf126.x86_64 from CentOS-AppStream 2024-04-12T20:41:20.488 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:820230223114759:436af1ae.x86_64 from CentOS-AppStream 2024-04-12T20:41:20.489 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:820230223114759:b12195ea.x86_64 from CentOS-AppStream 2024-04-12T20:41:20.553 INFO:teuthology.orchestra.run.smithi186.stdout:Dependencies resolved. 2024-04-12T20:41:20.554 INFO:teuthology.orchestra.run.smithi186.stdout:================================================================================ 2024-04-12T20:41:20.554 INFO:teuthology.orchestra.run.smithi186.stdout: Package Arch Version Repository Size 2024-04-12T20:41:20.554 INFO:teuthology.orchestra.run.smithi186.stdout:================================================================================ 2024-04-12T20:41:20.555 INFO:teuthology.orchestra.run.smithi186.stdout:Installing: 2024-04-12T20:41:20.555 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-mgr-cephadm noarch 2:17.2.7-827.gd3b5060a.el9 ceph-noarch 123 k 2024-04-12T20:41:20.555 INFO:teuthology.orchestra.run.smithi186.stdout:Installing dependencies: 2024-04-12T20:41:20.555 INFO:teuthology.orchestra.run.smithi186.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-04-12T20:41:20.555 INFO:teuthology.orchestra.run.smithi186.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-04-12T20:41:20.555 INFO:teuthology.orchestra.run.smithi186.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-04-12T20:41:20.555 INFO:teuthology.orchestra.run.smithi186.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-04-12T20:41:20.555 INFO:teuthology.orchestra.run.smithi186.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-04-12T20:41:20.555 INFO:teuthology.orchestra.run.smithi186.stdout: 2024-04-12T20:41:20.555 INFO:teuthology.orchestra.run.smithi186.stdout:Transaction Summary 2024-04-12T20:41:20.555 INFO:teuthology.orchestra.run.smithi186.stdout:================================================================================ 2024-04-12T20:41:20.555 INFO:teuthology.orchestra.run.smithi186.stdout:Install 6 Packages 2024-04-12T20:41:20.555 INFO:teuthology.orchestra.run.smithi186.stdout: 2024-04-12T20:41:20.556 INFO:teuthology.orchestra.run.smithi186.stdout:Total download size: 7.0 M 2024-04-12T20:41:20.556 INFO:teuthology.orchestra.run.smithi186.stdout:Installed size: 33 M 2024-04-12T20:41:20.556 INFO:teuthology.orchestra.run.smithi186.stdout:Downloading Packages: 2024-04-12T20:41:20.654 INFO:teuthology.orchestra.run.smithi018.stderr:Modular dependency problems: 2024-04-12T20:41:20.654 INFO:teuthology.orchestra.run.smithi018.stderr: 2024-04-12T20:41:20.654 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 1: conflicting requests 2024-04-12T20:41:20.654 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module container-tools:rhel8:820240321091303:20125149.x86_64 from CentOS-AppStream 2024-04-12T20:41:20.654 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 2: conflicting requests 2024-04-12T20:41:20.654 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module eclipse:rhel8:820201023100746:b230ed4e.x86_64 from CentOS-AppStream 2024-04-12T20:41:20.655 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 3: conflicting requests 2024-04-12T20:41:20.655 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module freeradius:3.0:8080020230117180605:89170a74.x86_64 from CentOS-AppStream 2024-04-12T20:41:20.655 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 4: conflicting requests 2024-04-12T20:41:20.655 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module gimp:2.8:8000020190628145146:4148dfdf.x86_64 from CentOS-AppStream 2024-04-12T20:41:20.655 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 5: conflicting requests 2024-04-12T20:41:20.655 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module go-toolset:rhel8:820240401160413:b754926a.x86_64 from CentOS-AppStream 2024-04-12T20:41:20.655 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 6: conflicting requests 2024-04-12T20:41:20.655 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module httpd:2.4:820240216084734:9edba152.x86_64 from CentOS-AppStream 2024-04-12T20:41:20.655 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 7: conflicting requests 2024-04-12T20:41:20.655 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module idm:client:820240307184541:49cc9d1b.x86_64 from CentOS-AppStream 2024-04-12T20:41:20.655 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 8: conflicting requests 2024-04-12T20:41:20.655 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module inkscape:0.92.3:8080020221205124429:3e031279.x86_64 from CentOS-AppStream 2024-04-12T20:41:20.655 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 9: conflicting requests 2024-04-12T20:41:20.656 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module jmc:rhel8:8050020211110222101:6392b1f8.x86_64 from CentOS-AppStream 2024-04-12T20:41:20.656 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 10: conflicting requests 2024-04-12T20:41:20.656 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module llvm-toolset:rhel8:820240205133135:9edba152.x86_64 from CentOS-AppStream 2024-04-12T20:41:20.656 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 11: conflicting requests 2024-04-12T20:41:20.656 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module log4j:2:8080020221205124743:9d367344.x86_64 from CentOS-AppStream 2024-04-12T20:41:20.656 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 12: conflicting requests 2024-04-12T20:41:20.656 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module mailman:2.1:820230727085757:77fc8825.x86_64 from CentOS-AppStream 2024-04-12T20:41:20.656 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 13: conflicting requests 2024-04-12T20:41:20.656 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module mariadb:10.3:8030020210419150013:30b713e6.x86_64 from CentOS-AppStream 2024-04-12T20:41:20.656 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 14: conflicting requests 2024-04-12T20:41:20.656 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module maven:3.5:8060020211203142015:c0229ad2.x86_64 from CentOS-AppStream 2024-04-12T20:41:20.656 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 15: conflicting requests 2024-04-12T20:41:20.657 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module mercurial:4.8:8000020190628020724:4148dfdf.x86_64 from CentOS-AppStream 2024-04-12T20:41:20.657 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 16: conflicting requests 2024-04-12T20:41:20.657 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module mod_auth_openidc:2.3:820230721123832:9edba152.x86_64 from CentOS-AppStream 2024-04-12T20:41:20.657 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 17: conflicting requests 2024-04-12T20:41:20.657 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module mysql:8.0:8040020210901180257:9f9e2e7e.x86_64 from CentOS-AppStream 2024-04-12T20:41:20.657 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 18: conflicting requests 2024-04-12T20:41:20.657 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module nginx:1.14:8000020211221191913:55190bc5.x86_64 from CentOS-AppStream 2024-04-12T20:41:20.657 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 19: conflicting requests 2024-04-12T20:41:20.657 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module nodejs:10:8040020210120182536:9f9e2e7e.x86_64 from CentOS-AppStream 2024-04-12T20:41:20.657 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 20: conflicting requests 2024-04-12T20:41:20.657 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl:5.26:8000020190628020724:55190bc5.x86_64 from CentOS-AppStream 2024-04-12T20:41:20.657 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 21: conflicting requests 2024-04-12T20:41:20.657 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module php:7.2:8020020200507003613:2c7ca891.x86_64 from CentOS-AppStream 2024-04-12T20:41:20.658 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 22: conflicting requests 2024-04-12T20:41:20.658 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module postgresql:10:820230223114800:9edba152.x86_64 from CentOS-AppStream 2024-04-12T20:41:20.658 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 23: conflicting requests 2024-04-12T20:41:20.658 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module python27:2.7:820240208011952:182f7c73.x86_64 from CentOS-AppStream 2024-04-12T20:41:20.658 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 24: conflicting requests 2024-04-12T20:41:20.658 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module python36:3.6:8050020210825152031:982725ab.x86_64 from CentOS-AppStream 2024-04-12T20:41:20.658 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 25: conflicting requests 2024-04-12T20:41:20.658 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module python38:3.8:820230810143931:64c1cd5a.x86_64 from CentOS-AppStream 2024-04-12T20:41:20.658 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 26: conflicting requests 2024-04-12T20:41:20.658 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module python39:3.9:820240214182535:17377f89.x86_64 from CentOS-AppStream 2024-04-12T20:41:20.658 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 27: conflicting requests 2024-04-12T20:41:20.658 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module redis:5:8040020211019153849:9f9e2e7e.x86_64 from CentOS-AppStream 2024-04-12T20:41:20.659 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 28: conflicting requests 2024-04-12T20:41:20.659 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module rhn-tools:1.0:8010020191114034948:f69d1239.x86_64 from CentOS-AppStream 2024-04-12T20:41:20.659 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 29: conflicting requests 2024-04-12T20:41:20.659 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module ruby:2.5:820230627084142:9edba152.x86_64 from CentOS-AppStream 2024-04-12T20:41:20.659 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 30: conflicting requests 2024-04-12T20:41:20.659 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module rust-toolset:rhel8:820240119204620:b09eea91.x86_64 from CentOS-AppStream 2024-04-12T20:41:20.659 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 31: conflicting requests 2024-04-12T20:41:20.659 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module satellite-5-client:1.0:8010020191114035551:cdc1202b.x86_64 from CentOS-AppStream 2024-04-12T20:41:20.659 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 32: conflicting requests 2024-04-12T20:41:20.659 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module squid:4:820240319173245:9edba152.x86_64 from CentOS-AppStream 2024-04-12T20:41:20.659 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 33: conflicting requests 2024-04-12T20:41:20.659 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module subversion:1.10:8070020220711155714:78111232.x86_64 from CentOS-AppStream 2024-04-12T20:41:20.659 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 34: conflicting requests 2024-04-12T20:41:20.660 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module swig:3.0:8030020200706142531:30b713e6.x86_64 from CentOS-AppStream 2024-04-12T20:41:20.660 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 35: conflicting requests 2024-04-12T20:41:20.660 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module varnish:6:8050020211109225449:b4937e53.x86_64 from CentOS-AppStream 2024-04-12T20:41:20.660 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 36: conflicting requests 2024-04-12T20:41:20.660 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module virt:rhel:820240314161907:9edba152.x86_64 from CentOS-AppStream 2024-04-12T20:41:20.660 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 37: conflicting requests 2024-04-12T20:41:20.660 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:8030020200716155257:7cc0a66d.x86_64 from CentOS-AppStream 2024-04-12T20:41:20.660 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:8030020200716155257:b967a9a2.x86_64 from CentOS-AppStream 2024-04-12T20:41:20.660 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:820230811133638:36f9fae6.x86_64 from CentOS-AppStream 2024-04-12T20:41:20.660 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 38: cannot install the best candidate for the job 2024-04-12T20:41:20.660 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:8010020191114031501:a5949e2e.x86_64 from CentOS-AppStream 2024-04-12T20:41:20.660 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:8010020191114031501:be2e4737.x86_64 from CentOS-AppStream 2024-04-12T20:41:20.660 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:820230223114759:2ab761e1.x86_64 from CentOS-AppStream 2024-04-12T20:41:20.660 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:820230223114759:86e45846.x86_64 from CentOS-AppStream 2024-04-12T20:41:20.661 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 39: conflicting requests 2024-04-12T20:41:20.661 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:8030020200715230104:1e4bbb35.x86_64 from CentOS-AppStream 2024-04-12T20:41:20.661 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:8030020200715230104:ea09926d.x86_64 from CentOS-AppStream 2024-04-12T20:41:20.661 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:082fdf2f.x86_64 from CentOS-AppStream 2024-04-12T20:41:20.661 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-04-12T20:41:20.661 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-04-12T20:41:20.661 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:fbe42456.x86_64 from CentOS-AppStream 2024-04-12T20:41:20.661 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 40: cannot install the best candidate for the job 2024-04-12T20:41:20.661 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:8010020191114031513:16b3ab4d.x86_64 from CentOS-AppStream 2024-04-12T20:41:20.661 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:8010020191114031513:e1f37755.x86_64 from CentOS-AppStream 2024-04-12T20:41:20.661 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-04-12T20:41:20.661 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-04-12T20:41:20.661 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 41: cannot install the best candidate for the job 2024-04-12T20:41:20.662 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:8030020200716150652:1e4bbb35.x86_64 from CentOS-AppStream 2024-04-12T20:41:20.662 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:8030020200716150652:ea09926d.x86_64 from CentOS-AppStream 2024-04-12T20:41:20.662 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-04-12T20:41:20.662 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-04-12T20:41:20.662 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 42: cannot install the best candidate for the job 2024-04-12T20:41:20.662 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:8030020200716174729:3a70019f.x86_64 from CentOS-AppStream 2024-04-12T20:41:20.662 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:8030020200716174729:cccafca5.x86_64 from CentOS-AppStream 2024-04-12T20:41:20.662 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:820230223114759:6fdf7e5d.x86_64 from CentOS-AppStream 2024-04-12T20:41:20.662 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:820230223114759:d4326aba.x86_64 from CentOS-AppStream 2024-04-12T20:41:20.662 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 43: cannot install the best candidate for the job 2024-04-12T20:41:20.662 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:8030020200716213634:56fce90f.x86_64 from CentOS-AppStream 2024-04-12T20:41:20.662 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:8030020200716213634:98a3c9d0.x86_64 from CentOS-AppStream 2024-04-12T20:41:20.662 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:820230223114759:06b7596e.x86_64 from CentOS-AppStream 2024-04-12T20:41:20.662 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:820230223114759:d2fc5d76.x86_64 from CentOS-AppStream 2024-04-12T20:41:20.663 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 44: cannot install the best candidate for the job 2024-04-12T20:41:20.663 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:8030020200716171139:3a70019f.x86_64 from CentOS-AppStream 2024-04-12T20:41:20.663 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:8030020200716171139:cccafca5.x86_64 from CentOS-AppStream 2024-04-12T20:41:20.663 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:820230223114759:6fdf7e5d.x86_64 from CentOS-AppStream 2024-04-12T20:41:20.663 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:820230223114759:d4326aba.x86_64 from CentOS-AppStream 2024-04-12T20:41:20.663 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 45: cannot install the best candidate for the job 2024-04-12T20:41:20.663 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:8010020191120175858:95fce66d.x86_64 from CentOS-AppStream 2024-04-12T20:41:20.663 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:8030020200723233049:09acf126.x86_64 from CentOS-AppStream 2024-04-12T20:41:20.663 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:820230223114759:436af1ae.x86_64 from CentOS-AppStream 2024-04-12T20:41:20.663 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:820230223114759:b12195ea.x86_64 from CentOS-AppStream 2024-04-12T20:41:20.732 INFO:teuthology.orchestra.run.smithi018.stdout:Dependencies resolved. 2024-04-12T20:41:20.733 INFO:teuthology.orchestra.run.smithi018.stdout:================================================================================ 2024-04-12T20:41:20.733 INFO:teuthology.orchestra.run.smithi018.stdout: Package Arch Version Repository Size 2024-04-12T20:41:20.733 INFO:teuthology.orchestra.run.smithi018.stdout:================================================================================ 2024-04-12T20:41:20.733 INFO:teuthology.orchestra.run.smithi018.stdout:Installing: 2024-04-12T20:41:20.733 INFO:teuthology.orchestra.run.smithi018.stdout: ceph-mgr-cephadm noarch 2:17.2.7-827.gd3b5060a.el9 ceph-noarch 123 k 2024-04-12T20:41:20.734 INFO:teuthology.orchestra.run.smithi018.stdout:Installing dependencies: 2024-04-12T20:41:20.734 INFO:teuthology.orchestra.run.smithi018.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-04-12T20:41:20.734 INFO:teuthology.orchestra.run.smithi018.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-04-12T20:41:20.734 INFO:teuthology.orchestra.run.smithi018.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-04-12T20:41:20.734 INFO:teuthology.orchestra.run.smithi018.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-04-12T20:41:20.734 INFO:teuthology.orchestra.run.smithi018.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-04-12T20:41:20.734 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-12T20:41:20.734 INFO:teuthology.orchestra.run.smithi018.stdout:Transaction Summary 2024-04-12T20:41:20.734 INFO:teuthology.orchestra.run.smithi018.stdout:================================================================================ 2024-04-12T20:41:20.734 INFO:teuthology.orchestra.run.smithi018.stdout:Install 6 Packages 2024-04-12T20:41:20.734 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-12T20:41:20.735 INFO:teuthology.orchestra.run.smithi018.stdout:Total download size: 7.0 M 2024-04-12T20:41:20.735 INFO:teuthology.orchestra.run.smithi018.stdout:Installed size: 33 M 2024-04-12T20:41:20.735 INFO:teuthology.orchestra.run.smithi018.stdout:Downloading Packages: 2024-04-12T20:41:21.427 INFO:teuthology.orchestra.run.smithi186.stdout:(1/6): ceph-mgr-cephadm-17.2.7-827.gd3b5060a.el 247 kB/s | 123 kB 00:00 2024-04-12T20:41:21.642 INFO:teuthology.orchestra.run.smithi018.stdout:(1/6): ceph-mgr-cephadm-17.2.7-827.gd3b5060a.el 342 kB/s | 123 kB 00:00 2024-04-12T20:41:21.644 INFO:teuthology.orchestra.run.smithi186.stdout:(2/6): python3-jinja2-2.11.3-5.el9.noarch.rpm 347 kB/s | 248 kB 00:00 2024-04-12T20:41:21.717 INFO:teuthology.orchestra.run.smithi018.stdout:(2/6): python3-jinja2-2.11.3-5.el9.noarch.rpm 570 kB/s | 248 kB 00:00 2024-04-12T20:41:21.817 INFO:teuthology.orchestra.run.smithi018.stdout:(3/6): python3-asyncssh-2.13.2-5.el9.noarch.rpm 3.1 MB/s | 548 kB 00:00 2024-04-12T20:41:21.834 INFO:teuthology.orchestra.run.smithi018.stdout:(4/6): python3-natsort-7.1.1-5.el9.noarch.rpm 500 kB/s | 58 kB 00:00 2024-04-12T20:41:21.859 INFO:teuthology.orchestra.run.smithi018.stdout:(5/6): python3-typing-extensions-4.4.0-2.el9.no 1.2 MB/s | 51 kB 00:00 2024-04-12T20:41:22.019 INFO:teuthology.orchestra.run.smithi186.stdout:(3/6): python3-natsort-7.1.1-5.el9.noarch.rpm 154 kB/s | 58 kB 00:00 2024-04-12T20:41:22.051 INFO:teuthology.orchestra.run.smithi018.stdout:(6/6): python3-babel-2.9.1-2.el9.noarch.rpm 7.8 MB/s | 6.0 MB 00:00 2024-04-12T20:41:22.052 INFO:teuthology.orchestra.run.smithi186.stdout:(4/6): python3-asyncssh-2.13.2-5.el9.noarch.rpm 878 kB/s | 548 kB 00:00 2024-04-12T20:41:22.057 INFO:teuthology.orchestra.run.smithi018.stdout:-------------------------------------------------------------------------------- 2024-04-12T20:41:22.058 INFO:teuthology.orchestra.run.smithi018.stdout:Total 5.3 MB/s | 7.0 MB 00:01 2024-04-12T20:41:22.119 INFO:teuthology.orchestra.run.smithi186.stdout:(5/6): python3-typing-extensions-4.4.0-2.el9.no 514 kB/s | 51 kB 00:00 2024-04-12T20:41:22.195 INFO:teuthology.orchestra.run.smithi018.stdout:Running transaction check 2024-04-12T20:41:22.236 INFO:teuthology.orchestra.run.smithi018.stdout:Transaction check succeeded. 2024-04-12T20:41:22.236 INFO:teuthology.orchestra.run.smithi018.stdout:Running transaction test 2024-04-12T20:41:22.400 INFO:teuthology.orchestra.run.smithi018.stdout:Transaction test succeeded. 2024-04-12T20:41:22.400 INFO:teuthology.orchestra.run.smithi018.stdout:Running transaction 2024-04-12T20:41:22.402 INFO:teuthology.orchestra.run.smithi186.stdout:(6/6): python3-babel-2.9.1-2.el9.noarch.rpm 4.0 MB/s | 6.0 MB 00:01 2024-04-12T20:41:22.407 INFO:teuthology.orchestra.run.smithi186.stdout:-------------------------------------------------------------------------------- 2024-04-12T20:41:22.408 INFO:teuthology.orchestra.run.smithi186.stdout:Total 3.8 MB/s | 7.0 MB 00:01 2024-04-12T20:41:22.536 INFO:teuthology.orchestra.run.smithi186.stdout:Running transaction check 2024-04-12T20:41:22.574 INFO:teuthology.orchestra.run.smithi186.stdout:Transaction check succeeded. 2024-04-12T20:41:22.574 INFO:teuthology.orchestra.run.smithi186.stdout:Running transaction test 2024-04-12T20:41:22.730 INFO:teuthology.orchestra.run.smithi186.stdout:Transaction test succeeded. 2024-04-12T20:41:22.730 INFO:teuthology.orchestra.run.smithi186.stdout:Running transaction 2024-04-12T20:41:22.850 INFO:teuthology.orchestra.run.smithi018.stdout: Preparing : 1/1 2024-04-12T20:41:23.102 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 1/6 2024-04-12T20:41:23.141 INFO:teuthology.orchestra.run.smithi186.stdout: Preparing : 1/1 2024-04-12T20:41:23.253 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 2/6 2024-04-12T20:41:23.399 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 1/6 2024-04-12T20:41:23.470 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 2/6 2024-04-12T20:41:23.994 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 3/6 2024-04-12T20:41:24.107 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 4/6 2024-04-12T20:41:24.187 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 5/6 2024-04-12T20:41:24.232 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 3/6 2024-04-12T20:41:24.236 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : ceph-mgr-cephadm-2:17.2.7-827.gd3b5060a.el9.noarch 6/6 2024-04-12T20:41:24.335 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 4/6 2024-04-12T20:41:24.402 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 5/6 2024-04-12T20:41:24.433 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : ceph-mgr-cephadm-2:17.2.7-827.gd3b5060a.el9.noarch 6/6 2024-04-12T20:41:25.162 INFO:teuthology.orchestra.run.smithi018.stdout: Running scriptlet: ceph-mgr-cephadm-2:17.2.7-827.gd3b5060a.el9.noarch 6/6 2024-04-12T20:41:25.162 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : ceph-mgr-cephadm-2:17.2.7-827.gd3b5060a.el9.noarch 1/6 2024-04-12T20:41:25.162 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 2/6 2024-04-12T20:41:25.162 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 3/6 2024-04-12T20:41:25.162 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 4/6 2024-04-12T20:41:25.162 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 5/6 2024-04-12T20:41:25.318 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: ceph-mgr-cephadm-2:17.2.7-827.gd3b5060a.el9.noarch 6/6 2024-04-12T20:41:25.318 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : ceph-mgr-cephadm-2:17.2.7-827.gd3b5060a.el9.noarch 1/6 2024-04-12T20:41:25.318 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 2/6 2024-04-12T20:41:25.318 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 3/6 2024-04-12T20:41:25.319 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 4/6 2024-04-12T20:41:25.319 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 5/6 2024-04-12T20:41:25.562 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 6/6 2024-04-12T20:41:25.562 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-12T20:41:25.562 INFO:teuthology.orchestra.run.smithi018.stdout:Installed: 2024-04-12T20:41:25.562 INFO:teuthology.orchestra.run.smithi018.stdout: ceph-mgr-cephadm-2:17.2.7-827.gd3b5060a.el9.noarch 2024-04-12T20:41:25.562 INFO:teuthology.orchestra.run.smithi018.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-04-12T20:41:25.562 INFO:teuthology.orchestra.run.smithi018.stdout: python3-babel-2.9.1-2.el9.noarch 2024-04-12T20:41:25.562 INFO:teuthology.orchestra.run.smithi018.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-04-12T20:41:25.562 INFO:teuthology.orchestra.run.smithi018.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-04-12T20:41:25.562 INFO:teuthology.orchestra.run.smithi018.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-04-12T20:41:25.562 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-12T20:41:25.562 INFO:teuthology.orchestra.run.smithi018.stdout:Complete! 2024-04-12T20:41:25.710 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 6/6 2024-04-12T20:41:25.710 INFO:teuthology.orchestra.run.smithi186.stdout: 2024-04-12T20:41:25.710 INFO:teuthology.orchestra.run.smithi186.stdout:Installed: 2024-04-12T20:41:25.710 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-mgr-cephadm-2:17.2.7-827.gd3b5060a.el9.noarch 2024-04-12T20:41:25.711 INFO:teuthology.orchestra.run.smithi186.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-04-12T20:41:25.711 INFO:teuthology.orchestra.run.smithi186.stdout: python3-babel-2.9.1-2.el9.noarch 2024-04-12T20:41:25.711 INFO:teuthology.orchestra.run.smithi186.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-04-12T20:41:25.711 INFO:teuthology.orchestra.run.smithi186.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-04-12T20:41:25.711 INFO:teuthology.orchestra.run.smithi186.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-04-12T20:41:25.711 INFO:teuthology.orchestra.run.smithi186.stdout: 2024-04-12T20:41:25.711 INFO:teuthology.orchestra.run.smithi186.stdout:Complete! 2024-04-12T20:41:25.882 DEBUG:teuthology.orchestra.run.smithi018:> sudo yum -y install ceph-fuse 2024-04-12T20:41:26.050 DEBUG:teuthology.orchestra.run.smithi186:> sudo yum -y install ceph-fuse 2024-04-12T20:41:26.801 INFO:teuthology.orchestra.run.smithi018.stdout:Last metadata expiration check: 0:01:51 ago on Fri 12 Apr 2024 08:39:35 PM UTC. 2024-04-12T20:41:26.972 INFO:teuthology.orchestra.run.smithi186.stdout:Last metadata expiration check: 0:01:49 ago on Fri 12 Apr 2024 08:39:37 PM UTC. 2024-04-12T20:41:27.112 INFO:teuthology.orchestra.run.smithi018.stderr:Modular dependency problems: 2024-04-12T20:41:27.112 INFO:teuthology.orchestra.run.smithi018.stderr: 2024-04-12T20:41:27.112 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 1: conflicting requests 2024-04-12T20:41:27.112 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module container-tools:rhel8:820240321091303:20125149.x86_64 from CentOS-AppStream 2024-04-12T20:41:27.112 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 2: conflicting requests 2024-04-12T20:41:27.112 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module eclipse:rhel8:820201023100746:b230ed4e.x86_64 from CentOS-AppStream 2024-04-12T20:41:27.112 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 3: conflicting requests 2024-04-12T20:41:27.112 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module freeradius:3.0:8080020230117180605:89170a74.x86_64 from CentOS-AppStream 2024-04-12T20:41:27.112 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 4: conflicting requests 2024-04-12T20:41:27.112 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module gimp:2.8:8000020190628145146:4148dfdf.x86_64 from CentOS-AppStream 2024-04-12T20:41:27.113 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 5: conflicting requests 2024-04-12T20:41:27.113 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module go-toolset:rhel8:820240401160413:b754926a.x86_64 from CentOS-AppStream 2024-04-12T20:41:27.113 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 6: conflicting requests 2024-04-12T20:41:27.113 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module httpd:2.4:820240216084734:9edba152.x86_64 from CentOS-AppStream 2024-04-12T20:41:27.113 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 7: conflicting requests 2024-04-12T20:41:27.113 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module idm:client:820240307184541:49cc9d1b.x86_64 from CentOS-AppStream 2024-04-12T20:41:27.113 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 8: conflicting requests 2024-04-12T20:41:27.113 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module inkscape:0.92.3:8080020221205124429:3e031279.x86_64 from CentOS-AppStream 2024-04-12T20:41:27.113 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 9: conflicting requests 2024-04-12T20:41:27.113 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module jmc:rhel8:8050020211110222101:6392b1f8.x86_64 from CentOS-AppStream 2024-04-12T20:41:27.113 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 10: conflicting requests 2024-04-12T20:41:27.113 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module llvm-toolset:rhel8:820240205133135:9edba152.x86_64 from CentOS-AppStream 2024-04-12T20:41:27.113 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 11: conflicting requests 2024-04-12T20:41:27.113 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module log4j:2:8080020221205124743:9d367344.x86_64 from CentOS-AppStream 2024-04-12T20:41:27.114 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 12: conflicting requests 2024-04-12T20:41:27.114 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module mailman:2.1:820230727085757:77fc8825.x86_64 from CentOS-AppStream 2024-04-12T20:41:27.114 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 13: conflicting requests 2024-04-12T20:41:27.114 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module mariadb:10.3:8030020210419150013:30b713e6.x86_64 from CentOS-AppStream 2024-04-12T20:41:27.114 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 14: conflicting requests 2024-04-12T20:41:27.114 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module maven:3.5:8060020211203142015:c0229ad2.x86_64 from CentOS-AppStream 2024-04-12T20:41:27.114 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 15: conflicting requests 2024-04-12T20:41:27.114 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module mercurial:4.8:8000020190628020724:4148dfdf.x86_64 from CentOS-AppStream 2024-04-12T20:41:27.114 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 16: conflicting requests 2024-04-12T20:41:27.114 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module mod_auth_openidc:2.3:820230721123832:9edba152.x86_64 from CentOS-AppStream 2024-04-12T20:41:27.114 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 17: conflicting requests 2024-04-12T20:41:27.114 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module mysql:8.0:8040020210901180257:9f9e2e7e.x86_64 from CentOS-AppStream 2024-04-12T20:41:27.115 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 18: conflicting requests 2024-04-12T20:41:27.115 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module nginx:1.14:8000020211221191913:55190bc5.x86_64 from CentOS-AppStream 2024-04-12T20:41:27.115 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 19: conflicting requests 2024-04-12T20:41:27.115 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module nodejs:10:8040020210120182536:9f9e2e7e.x86_64 from CentOS-AppStream 2024-04-12T20:41:27.115 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 20: conflicting requests 2024-04-12T20:41:27.115 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl:5.26:8000020190628020724:55190bc5.x86_64 from CentOS-AppStream 2024-04-12T20:41:27.115 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 21: conflicting requests 2024-04-12T20:41:27.115 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module php:7.2:8020020200507003613:2c7ca891.x86_64 from CentOS-AppStream 2024-04-12T20:41:27.115 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 22: conflicting requests 2024-04-12T20:41:27.115 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module postgresql:10:820230223114800:9edba152.x86_64 from CentOS-AppStream 2024-04-12T20:41:27.115 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 23: conflicting requests 2024-04-12T20:41:27.115 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module python27:2.7:820240208011952:182f7c73.x86_64 from CentOS-AppStream 2024-04-12T20:41:27.115 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 24: conflicting requests 2024-04-12T20:41:27.115 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module python36:3.6:8050020210825152031:982725ab.x86_64 from CentOS-AppStream 2024-04-12T20:41:27.116 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 25: conflicting requests 2024-04-12T20:41:27.116 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module python38:3.8:820230810143931:64c1cd5a.x86_64 from CentOS-AppStream 2024-04-12T20:41:27.116 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 26: conflicting requests 2024-04-12T20:41:27.116 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module python39:3.9:820240214182535:17377f89.x86_64 from CentOS-AppStream 2024-04-12T20:41:27.116 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 27: conflicting requests 2024-04-12T20:41:27.116 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module redis:5:8040020211019153849:9f9e2e7e.x86_64 from CentOS-AppStream 2024-04-12T20:41:27.116 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 28: conflicting requests 2024-04-12T20:41:27.116 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module rhn-tools:1.0:8010020191114034948:f69d1239.x86_64 from CentOS-AppStream 2024-04-12T20:41:27.116 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 29: conflicting requests 2024-04-12T20:41:27.116 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module ruby:2.5:820230627084142:9edba152.x86_64 from CentOS-AppStream 2024-04-12T20:41:27.116 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 30: conflicting requests 2024-04-12T20:41:27.116 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module rust-toolset:rhel8:820240119204620:b09eea91.x86_64 from CentOS-AppStream 2024-04-12T20:41:27.116 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 31: conflicting requests 2024-04-12T20:41:27.117 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module satellite-5-client:1.0:8010020191114035551:cdc1202b.x86_64 from CentOS-AppStream 2024-04-12T20:41:27.117 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 32: conflicting requests 2024-04-12T20:41:27.117 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module squid:4:820240319173245:9edba152.x86_64 from CentOS-AppStream 2024-04-12T20:41:27.117 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 33: conflicting requests 2024-04-12T20:41:27.117 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module subversion:1.10:8070020220711155714:78111232.x86_64 from CentOS-AppStream 2024-04-12T20:41:27.117 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 34: conflicting requests 2024-04-12T20:41:27.117 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module swig:3.0:8030020200706142531:30b713e6.x86_64 from CentOS-AppStream 2024-04-12T20:41:27.117 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 35: conflicting requests 2024-04-12T20:41:27.117 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module varnish:6:8050020211109225449:b4937e53.x86_64 from CentOS-AppStream 2024-04-12T20:41:27.117 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 36: conflicting requests 2024-04-12T20:41:27.117 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module virt:rhel:820240314161907:9edba152.x86_64 from CentOS-AppStream 2024-04-12T20:41:27.117 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 37: conflicting requests 2024-04-12T20:41:27.117 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:8030020200716155257:7cc0a66d.x86_64 from CentOS-AppStream 2024-04-12T20:41:27.117 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:8030020200716155257:b967a9a2.x86_64 from CentOS-AppStream 2024-04-12T20:41:27.117 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:820230811133638:36f9fae6.x86_64 from CentOS-AppStream 2024-04-12T20:41:27.118 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 38: cannot install the best candidate for the job 2024-04-12T20:41:27.118 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:8010020191114031501:a5949e2e.x86_64 from CentOS-AppStream 2024-04-12T20:41:27.118 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:8010020191114031501:be2e4737.x86_64 from CentOS-AppStream 2024-04-12T20:41:27.118 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:820230223114759:2ab761e1.x86_64 from CentOS-AppStream 2024-04-12T20:41:27.118 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:820230223114759:86e45846.x86_64 from CentOS-AppStream 2024-04-12T20:41:27.118 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 39: conflicting requests 2024-04-12T20:41:27.118 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:8030020200715230104:1e4bbb35.x86_64 from CentOS-AppStream 2024-04-12T20:41:27.118 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:8030020200715230104:ea09926d.x86_64 from CentOS-AppStream 2024-04-12T20:41:27.118 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:082fdf2f.x86_64 from CentOS-AppStream 2024-04-12T20:41:27.118 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-04-12T20:41:27.118 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-04-12T20:41:27.118 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:fbe42456.x86_64 from CentOS-AppStream 2024-04-12T20:41:27.118 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 40: cannot install the best candidate for the job 2024-04-12T20:41:27.119 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:8010020191114031513:16b3ab4d.x86_64 from CentOS-AppStream 2024-04-12T20:41:27.119 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:8010020191114031513:e1f37755.x86_64 from CentOS-AppStream 2024-04-12T20:41:27.119 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-04-12T20:41:27.119 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-04-12T20:41:27.119 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 41: cannot install the best candidate for the job 2024-04-12T20:41:27.119 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:8030020200716150652:1e4bbb35.x86_64 from CentOS-AppStream 2024-04-12T20:41:27.119 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:8030020200716150652:ea09926d.x86_64 from CentOS-AppStream 2024-04-12T20:41:27.119 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-04-12T20:41:27.119 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-04-12T20:41:27.119 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 42: cannot install the best candidate for the job 2024-04-12T20:41:27.119 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:8030020200716174729:3a70019f.x86_64 from CentOS-AppStream 2024-04-12T20:41:27.119 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:8030020200716174729:cccafca5.x86_64 from CentOS-AppStream 2024-04-12T20:41:27.119 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:820230223114759:6fdf7e5d.x86_64 from CentOS-AppStream 2024-04-12T20:41:27.119 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:820230223114759:d4326aba.x86_64 from CentOS-AppStream 2024-04-12T20:41:27.120 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 43: cannot install the best candidate for the job 2024-04-12T20:41:27.120 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:8030020200716213634:56fce90f.x86_64 from CentOS-AppStream 2024-04-12T20:41:27.120 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:8030020200716213634:98a3c9d0.x86_64 from CentOS-AppStream 2024-04-12T20:41:27.120 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:820230223114759:06b7596e.x86_64 from CentOS-AppStream 2024-04-12T20:41:27.120 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:820230223114759:d2fc5d76.x86_64 from CentOS-AppStream 2024-04-12T20:41:27.120 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 44: cannot install the best candidate for the job 2024-04-12T20:41:27.120 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:8030020200716171139:3a70019f.x86_64 from CentOS-AppStream 2024-04-12T20:41:27.120 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:8030020200716171139:cccafca5.x86_64 from CentOS-AppStream 2024-04-12T20:41:27.120 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:820230223114759:6fdf7e5d.x86_64 from CentOS-AppStream 2024-04-12T20:41:27.120 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:820230223114759:d4326aba.x86_64 from CentOS-AppStream 2024-04-12T20:41:27.120 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 45: cannot install the best candidate for the job 2024-04-12T20:41:27.120 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:8010020191120175858:95fce66d.x86_64 from CentOS-AppStream 2024-04-12T20:41:27.120 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:8030020200723233049:09acf126.x86_64 from CentOS-AppStream 2024-04-12T20:41:27.120 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:820230223114759:436af1ae.x86_64 from CentOS-AppStream 2024-04-12T20:41:27.121 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:820230223114759:b12195ea.x86_64 from CentOS-AppStream 2024-04-12T20:41:27.186 INFO:teuthology.orchestra.run.smithi018.stdout:Dependencies resolved. 2024-04-12T20:41:27.186 INFO:teuthology.orchestra.run.smithi018.stdout:================================================================================ 2024-04-12T20:41:27.187 INFO:teuthology.orchestra.run.smithi018.stdout: Package Arch Version Repository Size 2024-04-12T20:41:27.187 INFO:teuthology.orchestra.run.smithi018.stdout:================================================================================ 2024-04-12T20:41:27.187 INFO:teuthology.orchestra.run.smithi018.stdout:Installing: 2024-04-12T20:41:27.187 INFO:teuthology.orchestra.run.smithi018.stdout: ceph-fuse x86_64 2:17.2.7-827.gd3b5060a.el9 ceph 812 k 2024-04-12T20:41:27.187 INFO:teuthology.orchestra.run.smithi018.stdout:Installing dependencies: 2024-04-12T20:41:27.187 INFO:teuthology.orchestra.run.smithi018.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-04-12T20:41:27.187 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-12T20:41:27.187 INFO:teuthology.orchestra.run.smithi018.stdout:Transaction Summary 2024-04-12T20:41:27.187 INFO:teuthology.orchestra.run.smithi018.stdout:================================================================================ 2024-04-12T20:41:27.187 INFO:teuthology.orchestra.run.smithi018.stdout:Install 2 Packages 2024-04-12T20:41:27.187 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-12T20:41:27.188 INFO:teuthology.orchestra.run.smithi018.stdout:Total download size: 892 k 2024-04-12T20:41:27.188 INFO:teuthology.orchestra.run.smithi018.stdout:Installed size: 2.6 M 2024-04-12T20:41:27.188 INFO:teuthology.orchestra.run.smithi018.stdout:Downloading Packages: 2024-04-12T20:41:27.323 INFO:teuthology.orchestra.run.smithi186.stderr:Modular dependency problems: 2024-04-12T20:41:27.323 INFO:teuthology.orchestra.run.smithi186.stderr: 2024-04-12T20:41:27.323 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 1: conflicting requests 2024-04-12T20:41:27.323 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module container-tools:rhel8:820240321091303:20125149.x86_64 from CentOS-AppStream 2024-04-12T20:41:27.323 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 2: conflicting requests 2024-04-12T20:41:27.323 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module eclipse:rhel8:820201023100746:b230ed4e.x86_64 from CentOS-AppStream 2024-04-12T20:41:27.323 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 3: conflicting requests 2024-04-12T20:41:27.323 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module freeradius:3.0:8080020230117180605:89170a74.x86_64 from CentOS-AppStream 2024-04-12T20:41:27.323 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 4: conflicting requests 2024-04-12T20:41:27.323 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module gimp:2.8:8000020190628145146:4148dfdf.x86_64 from CentOS-AppStream 2024-04-12T20:41:27.324 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 5: conflicting requests 2024-04-12T20:41:27.324 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module go-toolset:rhel8:820240401160413:b754926a.x86_64 from CentOS-AppStream 2024-04-12T20:41:27.324 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 6: conflicting requests 2024-04-12T20:41:27.324 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module httpd:2.4:820240216084734:9edba152.x86_64 from CentOS-AppStream 2024-04-12T20:41:27.324 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 7: conflicting requests 2024-04-12T20:41:27.324 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module idm:client:820240307184541:49cc9d1b.x86_64 from CentOS-AppStream 2024-04-12T20:41:27.324 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 8: conflicting requests 2024-04-12T20:41:27.324 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module inkscape:0.92.3:8080020221205124429:3e031279.x86_64 from CentOS-AppStream 2024-04-12T20:41:27.324 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 9: conflicting requests 2024-04-12T20:41:27.324 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module jmc:rhel8:8050020211110222101:6392b1f8.x86_64 from CentOS-AppStream 2024-04-12T20:41:27.324 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 10: conflicting requests 2024-04-12T20:41:27.324 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module llvm-toolset:rhel8:820240205133135:9edba152.x86_64 from CentOS-AppStream 2024-04-12T20:41:27.324 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 11: conflicting requests 2024-04-12T20:41:27.324 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module log4j:2:8080020221205124743:9d367344.x86_64 from CentOS-AppStream 2024-04-12T20:41:27.325 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 12: conflicting requests 2024-04-12T20:41:27.325 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module mailman:2.1:820230727085757:77fc8825.x86_64 from CentOS-AppStream 2024-04-12T20:41:27.325 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 13: conflicting requests 2024-04-12T20:41:27.325 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module mariadb:10.3:8030020210419150013:30b713e6.x86_64 from CentOS-AppStream 2024-04-12T20:41:27.325 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 14: conflicting requests 2024-04-12T20:41:27.325 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module maven:3.5:8060020211203142015:c0229ad2.x86_64 from CentOS-AppStream 2024-04-12T20:41:27.325 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 15: conflicting requests 2024-04-12T20:41:27.325 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module mercurial:4.8:8000020190628020724:4148dfdf.x86_64 from CentOS-AppStream 2024-04-12T20:41:27.325 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 16: conflicting requests 2024-04-12T20:41:27.325 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module mod_auth_openidc:2.3:820230721123832:9edba152.x86_64 from CentOS-AppStream 2024-04-12T20:41:27.325 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 17: conflicting requests 2024-04-12T20:41:27.325 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module mysql:8.0:8040020210901180257:9f9e2e7e.x86_64 from CentOS-AppStream 2024-04-12T20:41:27.325 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 18: conflicting requests 2024-04-12T20:41:27.326 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module nginx:1.14:8000020211221191913:55190bc5.x86_64 from CentOS-AppStream 2024-04-12T20:41:27.326 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 19: conflicting requests 2024-04-12T20:41:27.326 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module nodejs:10:8040020210120182536:9f9e2e7e.x86_64 from CentOS-AppStream 2024-04-12T20:41:27.326 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 20: conflicting requests 2024-04-12T20:41:27.326 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl:5.26:8000020190628020724:55190bc5.x86_64 from CentOS-AppStream 2024-04-12T20:41:27.326 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 21: conflicting requests 2024-04-12T20:41:27.326 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module php:7.2:8020020200507003613:2c7ca891.x86_64 from CentOS-AppStream 2024-04-12T20:41:27.326 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 22: conflicting requests 2024-04-12T20:41:27.326 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module postgresql:10:820230223114800:9edba152.x86_64 from CentOS-AppStream 2024-04-12T20:41:27.326 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 23: conflicting requests 2024-04-12T20:41:27.326 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module python27:2.7:820240208011952:182f7c73.x86_64 from CentOS-AppStream 2024-04-12T20:41:27.326 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 24: conflicting requests 2024-04-12T20:41:27.326 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module python36:3.6:8050020210825152031:982725ab.x86_64 from CentOS-AppStream 2024-04-12T20:41:27.326 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 25: conflicting requests 2024-04-12T20:41:27.327 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module python38:3.8:820230810143931:64c1cd5a.x86_64 from CentOS-AppStream 2024-04-12T20:41:27.327 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 26: conflicting requests 2024-04-12T20:41:27.327 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module python39:3.9:820240214182535:17377f89.x86_64 from CentOS-AppStream 2024-04-12T20:41:27.327 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 27: conflicting requests 2024-04-12T20:41:27.327 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module redis:5:8040020211019153849:9f9e2e7e.x86_64 from CentOS-AppStream 2024-04-12T20:41:27.327 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 28: conflicting requests 2024-04-12T20:41:27.327 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module rhn-tools:1.0:8010020191114034948:f69d1239.x86_64 from CentOS-AppStream 2024-04-12T20:41:27.327 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 29: conflicting requests 2024-04-12T20:41:27.327 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module ruby:2.5:820230627084142:9edba152.x86_64 from CentOS-AppStream 2024-04-12T20:41:27.327 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 30: conflicting requests 2024-04-12T20:41:27.327 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module rust-toolset:rhel8:820240119204620:b09eea91.x86_64 from CentOS-AppStream 2024-04-12T20:41:27.327 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 31: conflicting requests 2024-04-12T20:41:27.327 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module satellite-5-client:1.0:8010020191114035551:cdc1202b.x86_64 from CentOS-AppStream 2024-04-12T20:41:27.327 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 32: conflicting requests 2024-04-12T20:41:27.327 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module squid:4:820240319173245:9edba152.x86_64 from CentOS-AppStream 2024-04-12T20:41:27.328 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 33: conflicting requests 2024-04-12T20:41:27.328 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module subversion:1.10:8070020220711155714:78111232.x86_64 from CentOS-AppStream 2024-04-12T20:41:27.328 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 34: conflicting requests 2024-04-12T20:41:27.328 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module swig:3.0:8030020200706142531:30b713e6.x86_64 from CentOS-AppStream 2024-04-12T20:41:27.328 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 35: conflicting requests 2024-04-12T20:41:27.328 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module varnish:6:8050020211109225449:b4937e53.x86_64 from CentOS-AppStream 2024-04-12T20:41:27.328 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 36: conflicting requests 2024-04-12T20:41:27.328 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module virt:rhel:820240314161907:9edba152.x86_64 from CentOS-AppStream 2024-04-12T20:41:27.328 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 37: conflicting requests 2024-04-12T20:41:27.328 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:8030020200716155257:7cc0a66d.x86_64 from CentOS-AppStream 2024-04-12T20:41:27.328 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:8030020200716155257:b967a9a2.x86_64 from CentOS-AppStream 2024-04-12T20:41:27.328 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:820230811133638:36f9fae6.x86_64 from CentOS-AppStream 2024-04-12T20:41:27.328 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 38: cannot install the best candidate for the job 2024-04-12T20:41:27.328 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:8010020191114031501:a5949e2e.x86_64 from CentOS-AppStream 2024-04-12T20:41:27.329 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:8010020191114031501:be2e4737.x86_64 from CentOS-AppStream 2024-04-12T20:41:27.329 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:820230223114759:2ab761e1.x86_64 from CentOS-AppStream 2024-04-12T20:41:27.329 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:820230223114759:86e45846.x86_64 from CentOS-AppStream 2024-04-12T20:41:27.329 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 39: conflicting requests 2024-04-12T20:41:27.329 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:8030020200715230104:1e4bbb35.x86_64 from CentOS-AppStream 2024-04-12T20:41:27.329 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:8030020200715230104:ea09926d.x86_64 from CentOS-AppStream 2024-04-12T20:41:27.329 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:082fdf2f.x86_64 from CentOS-AppStream 2024-04-12T20:41:27.329 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-04-12T20:41:27.329 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-04-12T20:41:27.329 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:fbe42456.x86_64 from CentOS-AppStream 2024-04-12T20:41:27.329 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 40: cannot install the best candidate for the job 2024-04-12T20:41:27.329 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:8010020191114031513:16b3ab4d.x86_64 from CentOS-AppStream 2024-04-12T20:41:27.329 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:8010020191114031513:e1f37755.x86_64 from CentOS-AppStream 2024-04-12T20:41:27.330 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-04-12T20:41:27.330 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-04-12T20:41:27.330 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 41: cannot install the best candidate for the job 2024-04-12T20:41:27.330 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:8030020200716150652:1e4bbb35.x86_64 from CentOS-AppStream 2024-04-12T20:41:27.330 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:8030020200716150652:ea09926d.x86_64 from CentOS-AppStream 2024-04-12T20:41:27.330 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-04-12T20:41:27.330 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-04-12T20:41:27.330 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 42: cannot install the best candidate for the job 2024-04-12T20:41:27.330 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:8030020200716174729:3a70019f.x86_64 from CentOS-AppStream 2024-04-12T20:41:27.330 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:8030020200716174729:cccafca5.x86_64 from CentOS-AppStream 2024-04-12T20:41:27.330 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:820230223114759:6fdf7e5d.x86_64 from CentOS-AppStream 2024-04-12T20:41:27.330 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:820230223114759:d4326aba.x86_64 from CentOS-AppStream 2024-04-12T20:41:27.330 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 43: cannot install the best candidate for the job 2024-04-12T20:41:27.330 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:8030020200716213634:56fce90f.x86_64 from CentOS-AppStream 2024-04-12T20:41:27.331 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:8030020200716213634:98a3c9d0.x86_64 from CentOS-AppStream 2024-04-12T20:41:27.331 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:820230223114759:06b7596e.x86_64 from CentOS-AppStream 2024-04-12T20:41:27.331 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:820230223114759:d2fc5d76.x86_64 from CentOS-AppStream 2024-04-12T20:41:27.331 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 44: cannot install the best candidate for the job 2024-04-12T20:41:27.331 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:8030020200716171139:3a70019f.x86_64 from CentOS-AppStream 2024-04-12T20:41:27.331 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:8030020200716171139:cccafca5.x86_64 from CentOS-AppStream 2024-04-12T20:41:27.331 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:820230223114759:6fdf7e5d.x86_64 from CentOS-AppStream 2024-04-12T20:41:27.331 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:820230223114759:d4326aba.x86_64 from CentOS-AppStream 2024-04-12T20:41:27.331 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 45: cannot install the best candidate for the job 2024-04-12T20:41:27.331 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:8010020191120175858:95fce66d.x86_64 from CentOS-AppStream 2024-04-12T20:41:27.331 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:8030020200723233049:09acf126.x86_64 from CentOS-AppStream 2024-04-12T20:41:27.331 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:820230223114759:436af1ae.x86_64 from CentOS-AppStream 2024-04-12T20:41:27.331 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:820230223114759:b12195ea.x86_64 from CentOS-AppStream 2024-04-12T20:41:27.397 INFO:teuthology.orchestra.run.smithi186.stdout:Dependencies resolved. 2024-04-12T20:41:27.397 INFO:teuthology.orchestra.run.smithi186.stdout:================================================================================ 2024-04-12T20:41:27.398 INFO:teuthology.orchestra.run.smithi186.stdout: Package Arch Version Repository Size 2024-04-12T20:41:27.398 INFO:teuthology.orchestra.run.smithi186.stdout:================================================================================ 2024-04-12T20:41:27.398 INFO:teuthology.orchestra.run.smithi186.stdout:Installing: 2024-04-12T20:41:27.398 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-fuse x86_64 2:17.2.7-827.gd3b5060a.el9 ceph 812 k 2024-04-12T20:41:27.398 INFO:teuthology.orchestra.run.smithi186.stdout:Installing dependencies: 2024-04-12T20:41:27.398 INFO:teuthology.orchestra.run.smithi186.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-04-12T20:41:27.398 INFO:teuthology.orchestra.run.smithi186.stdout: 2024-04-12T20:41:27.398 INFO:teuthology.orchestra.run.smithi186.stdout:Transaction Summary 2024-04-12T20:41:27.398 INFO:teuthology.orchestra.run.smithi186.stdout:================================================================================ 2024-04-12T20:41:27.398 INFO:teuthology.orchestra.run.smithi186.stdout:Install 2 Packages 2024-04-12T20:41:27.398 INFO:teuthology.orchestra.run.smithi186.stdout: 2024-04-12T20:41:27.399 INFO:teuthology.orchestra.run.smithi186.stdout:Total download size: 892 k 2024-04-12T20:41:27.399 INFO:teuthology.orchestra.run.smithi186.stdout:Installed size: 2.6 M 2024-04-12T20:41:27.399 INFO:teuthology.orchestra.run.smithi186.stdout:Downloading Packages: 2024-04-12T20:41:27.514 INFO:teuthology.orchestra.run.smithi018.stdout:(1/2): fuse-2.9.9-15.el9.x86_64.rpm 447 kB/s | 80 kB 00:00 2024-04-12T20:41:27.689 INFO:teuthology.orchestra.run.smithi018.stdout:(2/2): ceph-fuse-17.2.7-827.gd3b5060a.el9.x86_6 2.2 MB/s | 812 kB 00:00 2024-04-12T20:41:27.691 INFO:teuthology.orchestra.run.smithi018.stdout:-------------------------------------------------------------------------------- 2024-04-12T20:41:27.692 INFO:teuthology.orchestra.run.smithi018.stdout:Total 1.7 MB/s | 892 kB 00:00 2024-04-12T20:41:27.734 INFO:teuthology.orchestra.run.smithi018.stdout:Running transaction check 2024-04-12T20:41:27.809 INFO:teuthology.orchestra.run.smithi018.stdout:Transaction check succeeded. 2024-04-12T20:41:27.809 INFO:teuthology.orchestra.run.smithi018.stdout:Running transaction test 2024-04-12T20:41:27.916 INFO:teuthology.orchestra.run.smithi018.stdout:Transaction test succeeded. 2024-04-12T20:41:27.916 INFO:teuthology.orchestra.run.smithi018.stdout:Running transaction 2024-04-12T20:41:27.950 INFO:teuthology.orchestra.run.smithi186.stdout:(1/2): fuse-2.9.9-15.el9.x86_64.rpm 167 kB/s | 80 kB 00:00 2024-04-12T20:41:27.992 INFO:teuthology.orchestra.run.smithi186.stdout:(2/2): ceph-fuse-17.2.7-827.gd3b5060a.el9.x86_6 1.5 MB/s | 812 kB 00:00 2024-04-12T20:41:27.993 INFO:teuthology.orchestra.run.smithi186.stdout:-------------------------------------------------------------------------------- 2024-04-12T20:41:27.994 INFO:teuthology.orchestra.run.smithi186.stdout:Total 1.5 MB/s | 892 kB 00:00 2024-04-12T20:41:28.032 INFO:teuthology.orchestra.run.smithi186.stdout:Running transaction check 2024-04-12T20:41:28.105 INFO:teuthology.orchestra.run.smithi186.stdout:Transaction check succeeded. 2024-04-12T20:41:28.105 INFO:teuthology.orchestra.run.smithi186.stdout:Running transaction test 2024-04-12T20:41:28.196 INFO:teuthology.orchestra.run.smithi018.stdout: Preparing : 1/1 2024-04-12T20:41:28.210 INFO:teuthology.orchestra.run.smithi186.stdout:Transaction test succeeded. 2024-04-12T20:41:28.210 INFO:teuthology.orchestra.run.smithi186.stdout:Running transaction 2024-04-12T20:41:28.375 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : fuse-2.9.9-15.el9.x86_64 1/2 2024-04-12T20:41:28.460 INFO:teuthology.orchestra.run.smithi186.stdout: Preparing : 1/1 2024-04-12T20:41:28.465 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : ceph-fuse-2:17.2.7-827.gd3b5060a.el9.x86_64 2/2 2024-04-12T20:41:28.662 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : fuse-2.9.9-15.el9.x86_64 1/2 2024-04-12T20:41:28.738 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : ceph-fuse-2:17.2.7-827.gd3b5060a.el9.x86_64 2/2 2024-04-12T20:41:29.492 INFO:teuthology.orchestra.run.smithi018.stdout: Running scriptlet: ceph-fuse-2:17.2.7-827.gd3b5060a.el9.x86_64 2/2 2024-04-12T20:41:29.493 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : ceph-fuse-2:17.2.7-827.gd3b5060a.el9.x86_64 1/2 2024-04-12T20:41:29.756 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: ceph-fuse-2:17.2.7-827.gd3b5060a.el9.x86_64 2/2 2024-04-12T20:41:29.756 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : ceph-fuse-2:17.2.7-827.gd3b5060a.el9.x86_64 1/2 2024-04-12T20:41:29.870 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 2/2 2024-04-12T20:41:29.871 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-12T20:41:29.871 INFO:teuthology.orchestra.run.smithi018.stdout:Installed: 2024-04-12T20:41:29.871 INFO:teuthology.orchestra.run.smithi018.stdout: ceph-fuse-2:17.2.7-827.gd3b5060a.el9.x86_64 fuse-2.9.9-15.el9.x86_64 2024-04-12T20:41:29.871 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-12T20:41:29.871 INFO:teuthology.orchestra.run.smithi018.stdout:Complete! 2024-04-12T20:41:30.144 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 2/2 2024-04-12T20:41:30.145 INFO:teuthology.orchestra.run.smithi186.stdout: 2024-04-12T20:41:30.145 INFO:teuthology.orchestra.run.smithi186.stdout:Installed: 2024-04-12T20:41:30.145 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-fuse-2:17.2.7-827.gd3b5060a.el9.x86_64 fuse-2.9.9-15.el9.x86_64 2024-04-12T20:41:30.145 INFO:teuthology.orchestra.run.smithi186.stdout: 2024-04-12T20:41:30.145 INFO:teuthology.orchestra.run.smithi186.stdout:Complete! 2024-04-12T20:41:30.230 DEBUG:teuthology.orchestra.run.smithi018:> sudo yum -y install librados-devel 2024-04-12T20:41:30.524 DEBUG:teuthology.orchestra.run.smithi186:> sudo yum -y install librados-devel 2024-04-12T20:41:31.199 INFO:teuthology.orchestra.run.smithi018.stdout:Last metadata expiration check: 0:01:56 ago on Fri 12 Apr 2024 08:39:35 PM UTC. 2024-04-12T20:41:31.431 INFO:teuthology.orchestra.run.smithi186.stdout:Last metadata expiration check: 0:01:54 ago on Fri 12 Apr 2024 08:39:37 PM UTC. 2024-04-12T20:41:31.511 INFO:teuthology.orchestra.run.smithi018.stderr:Modular dependency problems: 2024-04-12T20:41:31.511 INFO:teuthology.orchestra.run.smithi018.stderr: 2024-04-12T20:41:31.511 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 1: conflicting requests 2024-04-12T20:41:31.511 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module container-tools:rhel8:820240321091303:20125149.x86_64 from CentOS-AppStream 2024-04-12T20:41:31.512 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 2: conflicting requests 2024-04-12T20:41:31.512 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module eclipse:rhel8:820201023100746:b230ed4e.x86_64 from CentOS-AppStream 2024-04-12T20:41:31.512 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 3: conflicting requests 2024-04-12T20:41:31.512 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module freeradius:3.0:8080020230117180605:89170a74.x86_64 from CentOS-AppStream 2024-04-12T20:41:31.512 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 4: conflicting requests 2024-04-12T20:41:31.512 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module gimp:2.8:8000020190628145146:4148dfdf.x86_64 from CentOS-AppStream 2024-04-12T20:41:31.512 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 5: conflicting requests 2024-04-12T20:41:31.512 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module go-toolset:rhel8:820240401160413:b754926a.x86_64 from CentOS-AppStream 2024-04-12T20:41:31.512 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 6: conflicting requests 2024-04-12T20:41:31.512 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module httpd:2.4:820240216084734:9edba152.x86_64 from CentOS-AppStream 2024-04-12T20:41:31.512 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 7: conflicting requests 2024-04-12T20:41:31.512 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module idm:client:820240307184541:49cc9d1b.x86_64 from CentOS-AppStream 2024-04-12T20:41:31.512 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 8: conflicting requests 2024-04-12T20:41:31.512 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module inkscape:0.92.3:8080020221205124429:3e031279.x86_64 from CentOS-AppStream 2024-04-12T20:41:31.513 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 9: conflicting requests 2024-04-12T20:41:31.513 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module jmc:rhel8:8050020211110222101:6392b1f8.x86_64 from CentOS-AppStream 2024-04-12T20:41:31.513 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 10: conflicting requests 2024-04-12T20:41:31.513 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module llvm-toolset:rhel8:820240205133135:9edba152.x86_64 from CentOS-AppStream 2024-04-12T20:41:31.513 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 11: conflicting requests 2024-04-12T20:41:31.513 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module log4j:2:8080020221205124743:9d367344.x86_64 from CentOS-AppStream 2024-04-12T20:41:31.513 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 12: conflicting requests 2024-04-12T20:41:31.513 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module mailman:2.1:820230727085757:77fc8825.x86_64 from CentOS-AppStream 2024-04-12T20:41:31.513 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 13: conflicting requests 2024-04-12T20:41:31.513 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module mariadb:10.3:8030020210419150013:30b713e6.x86_64 from CentOS-AppStream 2024-04-12T20:41:31.513 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 14: conflicting requests 2024-04-12T20:41:31.513 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module maven:3.5:8060020211203142015:c0229ad2.x86_64 from CentOS-AppStream 2024-04-12T20:41:31.513 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 15: conflicting requests 2024-04-12T20:41:31.514 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module mercurial:4.8:8000020190628020724:4148dfdf.x86_64 from CentOS-AppStream 2024-04-12T20:41:31.514 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 16: conflicting requests 2024-04-12T20:41:31.514 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module mod_auth_openidc:2.3:820230721123832:9edba152.x86_64 from CentOS-AppStream 2024-04-12T20:41:31.514 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 17: conflicting requests 2024-04-12T20:41:31.514 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module mysql:8.0:8040020210901180257:9f9e2e7e.x86_64 from CentOS-AppStream 2024-04-12T20:41:31.514 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 18: conflicting requests 2024-04-12T20:41:31.514 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module nginx:1.14:8000020211221191913:55190bc5.x86_64 from CentOS-AppStream 2024-04-12T20:41:31.514 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 19: conflicting requests 2024-04-12T20:41:31.514 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module nodejs:10:8040020210120182536:9f9e2e7e.x86_64 from CentOS-AppStream 2024-04-12T20:41:31.514 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 20: conflicting requests 2024-04-12T20:41:31.514 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl:5.26:8000020190628020724:55190bc5.x86_64 from CentOS-AppStream 2024-04-12T20:41:31.514 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 21: conflicting requests 2024-04-12T20:41:31.515 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module php:7.2:8020020200507003613:2c7ca891.x86_64 from CentOS-AppStream 2024-04-12T20:41:31.515 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 22: conflicting requests 2024-04-12T20:41:31.515 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module postgresql:10:820230223114800:9edba152.x86_64 from CentOS-AppStream 2024-04-12T20:41:31.515 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 23: conflicting requests 2024-04-12T20:41:31.515 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module python27:2.7:820240208011952:182f7c73.x86_64 from CentOS-AppStream 2024-04-12T20:41:31.515 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 24: conflicting requests 2024-04-12T20:41:31.515 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module python36:3.6:8050020210825152031:982725ab.x86_64 from CentOS-AppStream 2024-04-12T20:41:31.515 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 25: conflicting requests 2024-04-12T20:41:31.515 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module python38:3.8:820230810143931:64c1cd5a.x86_64 from CentOS-AppStream 2024-04-12T20:41:31.515 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 26: conflicting requests 2024-04-12T20:41:31.515 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module python39:3.9:820240214182535:17377f89.x86_64 from CentOS-AppStream 2024-04-12T20:41:31.515 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 27: conflicting requests 2024-04-12T20:41:31.515 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module redis:5:8040020211019153849:9f9e2e7e.x86_64 from CentOS-AppStream 2024-04-12T20:41:31.515 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 28: conflicting requests 2024-04-12T20:41:31.516 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module rhn-tools:1.0:8010020191114034948:f69d1239.x86_64 from CentOS-AppStream 2024-04-12T20:41:31.516 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 29: conflicting requests 2024-04-12T20:41:31.516 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module ruby:2.5:820230627084142:9edba152.x86_64 from CentOS-AppStream 2024-04-12T20:41:31.516 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 30: conflicting requests 2024-04-12T20:41:31.516 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module rust-toolset:rhel8:820240119204620:b09eea91.x86_64 from CentOS-AppStream 2024-04-12T20:41:31.516 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 31: conflicting requests 2024-04-12T20:41:31.516 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module satellite-5-client:1.0:8010020191114035551:cdc1202b.x86_64 from CentOS-AppStream 2024-04-12T20:41:31.516 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 32: conflicting requests 2024-04-12T20:41:31.516 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module squid:4:820240319173245:9edba152.x86_64 from CentOS-AppStream 2024-04-12T20:41:31.516 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 33: conflicting requests 2024-04-12T20:41:31.516 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module subversion:1.10:8070020220711155714:78111232.x86_64 from CentOS-AppStream 2024-04-12T20:41:31.516 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 34: conflicting requests 2024-04-12T20:41:31.516 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module swig:3.0:8030020200706142531:30b713e6.x86_64 from CentOS-AppStream 2024-04-12T20:41:31.516 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 35: conflicting requests 2024-04-12T20:41:31.517 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module varnish:6:8050020211109225449:b4937e53.x86_64 from CentOS-AppStream 2024-04-12T20:41:31.517 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 36: conflicting requests 2024-04-12T20:41:31.517 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module virt:rhel:820240314161907:9edba152.x86_64 from CentOS-AppStream 2024-04-12T20:41:31.517 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 37: conflicting requests 2024-04-12T20:41:31.517 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:8030020200716155257:7cc0a66d.x86_64 from CentOS-AppStream 2024-04-12T20:41:31.517 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:8030020200716155257:b967a9a2.x86_64 from CentOS-AppStream 2024-04-12T20:41:31.517 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:820230811133638:36f9fae6.x86_64 from CentOS-AppStream 2024-04-12T20:41:31.517 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 38: cannot install the best candidate for the job 2024-04-12T20:41:31.517 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:8010020191114031501:a5949e2e.x86_64 from CentOS-AppStream 2024-04-12T20:41:31.517 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:8010020191114031501:be2e4737.x86_64 from CentOS-AppStream 2024-04-12T20:41:31.518 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:820230223114759:2ab761e1.x86_64 from CentOS-AppStream 2024-04-12T20:41:31.518 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:820230223114759:86e45846.x86_64 from CentOS-AppStream 2024-04-12T20:41:31.518 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 39: conflicting requests 2024-04-12T20:41:31.518 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:8030020200715230104:1e4bbb35.x86_64 from CentOS-AppStream 2024-04-12T20:41:31.518 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:8030020200715230104:ea09926d.x86_64 from CentOS-AppStream 2024-04-12T20:41:31.518 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:082fdf2f.x86_64 from CentOS-AppStream 2024-04-12T20:41:31.518 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-04-12T20:41:31.518 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-04-12T20:41:31.518 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:fbe42456.x86_64 from CentOS-AppStream 2024-04-12T20:41:31.518 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 40: cannot install the best candidate for the job 2024-04-12T20:41:31.518 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:8010020191114031513:16b3ab4d.x86_64 from CentOS-AppStream 2024-04-12T20:41:31.518 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:8010020191114031513:e1f37755.x86_64 from CentOS-AppStream 2024-04-12T20:41:31.518 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-04-12T20:41:31.519 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-04-12T20:41:31.519 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 41: cannot install the best candidate for the job 2024-04-12T20:41:31.519 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:8030020200716150652:1e4bbb35.x86_64 from CentOS-AppStream 2024-04-12T20:41:31.519 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:8030020200716150652:ea09926d.x86_64 from CentOS-AppStream 2024-04-12T20:41:31.519 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-04-12T20:41:31.519 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-04-12T20:41:31.519 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 42: cannot install the best candidate for the job 2024-04-12T20:41:31.519 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:8030020200716174729:3a70019f.x86_64 from CentOS-AppStream 2024-04-12T20:41:31.519 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:8030020200716174729:cccafca5.x86_64 from CentOS-AppStream 2024-04-12T20:41:31.519 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:820230223114759:6fdf7e5d.x86_64 from CentOS-AppStream 2024-04-12T20:41:31.519 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:820230223114759:d4326aba.x86_64 from CentOS-AppStream 2024-04-12T20:41:31.519 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 43: cannot install the best candidate for the job 2024-04-12T20:41:31.519 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:8030020200716213634:56fce90f.x86_64 from CentOS-AppStream 2024-04-12T20:41:31.519 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:8030020200716213634:98a3c9d0.x86_64 from CentOS-AppStream 2024-04-12T20:41:31.520 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:820230223114759:06b7596e.x86_64 from CentOS-AppStream 2024-04-12T20:41:31.520 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:820230223114759:d2fc5d76.x86_64 from CentOS-AppStream 2024-04-12T20:41:31.520 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 44: cannot install the best candidate for the job 2024-04-12T20:41:31.520 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:8030020200716171139:3a70019f.x86_64 from CentOS-AppStream 2024-04-12T20:41:31.520 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:8030020200716171139:cccafca5.x86_64 from CentOS-AppStream 2024-04-12T20:41:31.520 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:820230223114759:6fdf7e5d.x86_64 from CentOS-AppStream 2024-04-12T20:41:31.520 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:820230223114759:d4326aba.x86_64 from CentOS-AppStream 2024-04-12T20:41:31.520 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 45: cannot install the best candidate for the job 2024-04-12T20:41:31.520 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:8010020191120175858:95fce66d.x86_64 from CentOS-AppStream 2024-04-12T20:41:31.520 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:8030020200723233049:09acf126.x86_64 from CentOS-AppStream 2024-04-12T20:41:31.520 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:820230223114759:436af1ae.x86_64 from CentOS-AppStream 2024-04-12T20:41:31.520 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:820230223114759:b12195ea.x86_64 from CentOS-AppStream 2024-04-12T20:41:31.588 INFO:teuthology.orchestra.run.smithi018.stdout:Dependencies resolved. 2024-04-12T20:41:31.589 INFO:teuthology.orchestra.run.smithi018.stdout:================================================================================ 2024-04-12T20:41:31.589 INFO:teuthology.orchestra.run.smithi018.stdout: Package Arch Version Repo Size 2024-04-12T20:41:31.589 INFO:teuthology.orchestra.run.smithi018.stdout:================================================================================ 2024-04-12T20:41:31.589 INFO:teuthology.orchestra.run.smithi018.stdout:Installing: 2024-04-12T20:41:31.589 INFO:teuthology.orchestra.run.smithi018.stdout: librados-devel x86_64 2:17.2.7-827.gd3b5060a.el9 ceph 126 k 2024-04-12T20:41:31.589 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-12T20:41:31.590 INFO:teuthology.orchestra.run.smithi018.stdout:Transaction Summary 2024-04-12T20:41:31.590 INFO:teuthology.orchestra.run.smithi018.stdout:================================================================================ 2024-04-12T20:41:31.590 INFO:teuthology.orchestra.run.smithi018.stdout:Install 1 Package 2024-04-12T20:41:31.590 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-12T20:41:31.590 INFO:teuthology.orchestra.run.smithi018.stdout:Total download size: 126 k 2024-04-12T20:41:31.590 INFO:teuthology.orchestra.run.smithi018.stdout:Installed size: 448 k 2024-04-12T20:41:31.590 INFO:teuthology.orchestra.run.smithi018.stdout:Downloading Packages: 2024-04-12T20:41:31.733 INFO:teuthology.orchestra.run.smithi186.stderr:Modular dependency problems: 2024-04-12T20:41:31.733 INFO:teuthology.orchestra.run.smithi186.stderr: 2024-04-12T20:41:31.734 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 1: conflicting requests 2024-04-12T20:41:31.734 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module container-tools:rhel8:820240321091303:20125149.x86_64 from CentOS-AppStream 2024-04-12T20:41:31.734 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 2: conflicting requests 2024-04-12T20:41:31.734 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module eclipse:rhel8:820201023100746:b230ed4e.x86_64 from CentOS-AppStream 2024-04-12T20:41:31.734 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 3: conflicting requests 2024-04-12T20:41:31.734 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module freeradius:3.0:8080020230117180605:89170a74.x86_64 from CentOS-AppStream 2024-04-12T20:41:31.735 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 4: conflicting requests 2024-04-12T20:41:31.735 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module gimp:2.8:8000020190628145146:4148dfdf.x86_64 from CentOS-AppStream 2024-04-12T20:41:31.735 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 5: conflicting requests 2024-04-12T20:41:31.735 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module go-toolset:rhel8:820240401160413:b754926a.x86_64 from CentOS-AppStream 2024-04-12T20:41:31.735 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 6: conflicting requests 2024-04-12T20:41:31.735 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module httpd:2.4:820240216084734:9edba152.x86_64 from CentOS-AppStream 2024-04-12T20:41:31.735 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 7: conflicting requests 2024-04-12T20:41:31.735 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module idm:client:820240307184541:49cc9d1b.x86_64 from CentOS-AppStream 2024-04-12T20:41:31.736 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 8: conflicting requests 2024-04-12T20:41:31.736 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module inkscape:0.92.3:8080020221205124429:3e031279.x86_64 from CentOS-AppStream 2024-04-12T20:41:31.736 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 9: conflicting requests 2024-04-12T20:41:31.736 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module jmc:rhel8:8050020211110222101:6392b1f8.x86_64 from CentOS-AppStream 2024-04-12T20:41:31.736 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 10: conflicting requests 2024-04-12T20:41:31.736 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module llvm-toolset:rhel8:820240205133135:9edba152.x86_64 from CentOS-AppStream 2024-04-12T20:41:31.736 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 11: conflicting requests 2024-04-12T20:41:31.736 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module log4j:2:8080020221205124743:9d367344.x86_64 from CentOS-AppStream 2024-04-12T20:41:31.736 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 12: conflicting requests 2024-04-12T20:41:31.736 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module mailman:2.1:820230727085757:77fc8825.x86_64 from CentOS-AppStream 2024-04-12T20:41:31.737 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 13: conflicting requests 2024-04-12T20:41:31.737 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module mariadb:10.3:8030020210419150013:30b713e6.x86_64 from CentOS-AppStream 2024-04-12T20:41:31.737 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 14: conflicting requests 2024-04-12T20:41:31.737 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module maven:3.5:8060020211203142015:c0229ad2.x86_64 from CentOS-AppStream 2024-04-12T20:41:31.737 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 15: conflicting requests 2024-04-12T20:41:31.737 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module mercurial:4.8:8000020190628020724:4148dfdf.x86_64 from CentOS-AppStream 2024-04-12T20:41:31.737 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 16: conflicting requests 2024-04-12T20:41:31.737 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module mod_auth_openidc:2.3:820230721123832:9edba152.x86_64 from CentOS-AppStream 2024-04-12T20:41:31.737 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 17: conflicting requests 2024-04-12T20:41:31.737 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module mysql:8.0:8040020210901180257:9f9e2e7e.x86_64 from CentOS-AppStream 2024-04-12T20:41:31.737 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 18: conflicting requests 2024-04-12T20:41:31.737 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module nginx:1.14:8000020211221191913:55190bc5.x86_64 from CentOS-AppStream 2024-04-12T20:41:31.737 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 19: conflicting requests 2024-04-12T20:41:31.737 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module nodejs:10:8040020210120182536:9f9e2e7e.x86_64 from CentOS-AppStream 2024-04-12T20:41:31.738 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 20: conflicting requests 2024-04-12T20:41:31.738 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl:5.26:8000020190628020724:55190bc5.x86_64 from CentOS-AppStream 2024-04-12T20:41:31.738 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 21: conflicting requests 2024-04-12T20:41:31.738 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module php:7.2:8020020200507003613:2c7ca891.x86_64 from CentOS-AppStream 2024-04-12T20:41:31.738 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 22: conflicting requests 2024-04-12T20:41:31.738 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module postgresql:10:820230223114800:9edba152.x86_64 from CentOS-AppStream 2024-04-12T20:41:31.738 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 23: conflicting requests 2024-04-12T20:41:31.738 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module python27:2.7:820240208011952:182f7c73.x86_64 from CentOS-AppStream 2024-04-12T20:41:31.738 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 24: conflicting requests 2024-04-12T20:41:31.738 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module python36:3.6:8050020210825152031:982725ab.x86_64 from CentOS-AppStream 2024-04-12T20:41:31.738 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 25: conflicting requests 2024-04-12T20:41:31.738 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module python38:3.8:820230810143931:64c1cd5a.x86_64 from CentOS-AppStream 2024-04-12T20:41:31.739 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 26: conflicting requests 2024-04-12T20:41:31.739 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module python39:3.9:820240214182535:17377f89.x86_64 from CentOS-AppStream 2024-04-12T20:41:31.739 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 27: conflicting requests 2024-04-12T20:41:31.739 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module redis:5:8040020211019153849:9f9e2e7e.x86_64 from CentOS-AppStream 2024-04-12T20:41:31.739 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 28: conflicting requests 2024-04-12T20:41:31.740 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module rhn-tools:1.0:8010020191114034948:f69d1239.x86_64 from CentOS-AppStream 2024-04-12T20:41:31.740 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 29: conflicting requests 2024-04-12T20:41:31.740 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module ruby:2.5:820230627084142:9edba152.x86_64 from CentOS-AppStream 2024-04-12T20:41:31.741 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 30: conflicting requests 2024-04-12T20:41:31.741 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module rust-toolset:rhel8:820240119204620:b09eea91.x86_64 from CentOS-AppStream 2024-04-12T20:41:31.741 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 31: conflicting requests 2024-04-12T20:41:31.741 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module satellite-5-client:1.0:8010020191114035551:cdc1202b.x86_64 from CentOS-AppStream 2024-04-12T20:41:31.741 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 32: conflicting requests 2024-04-12T20:41:31.741 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module squid:4:820240319173245:9edba152.x86_64 from CentOS-AppStream 2024-04-12T20:41:31.741 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 33: conflicting requests 2024-04-12T20:41:31.741 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module subversion:1.10:8070020220711155714:78111232.x86_64 from CentOS-AppStream 2024-04-12T20:41:31.741 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 34: conflicting requests 2024-04-12T20:41:31.741 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module swig:3.0:8030020200706142531:30b713e6.x86_64 from CentOS-AppStream 2024-04-12T20:41:31.742 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 35: conflicting requests 2024-04-12T20:41:31.742 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module varnish:6:8050020211109225449:b4937e53.x86_64 from CentOS-AppStream 2024-04-12T20:41:31.742 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 36: conflicting requests 2024-04-12T20:41:31.742 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module virt:rhel:820240314161907:9edba152.x86_64 from CentOS-AppStream 2024-04-12T20:41:31.742 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 37: conflicting requests 2024-04-12T20:41:31.742 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:8030020200716155257:7cc0a66d.x86_64 from CentOS-AppStream 2024-04-12T20:41:31.742 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:8030020200716155257:b967a9a2.x86_64 from CentOS-AppStream 2024-04-12T20:41:31.742 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:820230811133638:36f9fae6.x86_64 from CentOS-AppStream 2024-04-12T20:41:31.742 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 38: cannot install the best candidate for the job 2024-04-12T20:41:31.742 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:8010020191114031501:a5949e2e.x86_64 from CentOS-AppStream 2024-04-12T20:41:31.742 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:8010020191114031501:be2e4737.x86_64 from CentOS-AppStream 2024-04-12T20:41:31.743 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:820230223114759:2ab761e1.x86_64 from CentOS-AppStream 2024-04-12T20:41:31.743 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:820230223114759:86e45846.x86_64 from CentOS-AppStream 2024-04-12T20:41:31.743 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 39: conflicting requests 2024-04-12T20:41:31.743 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:8030020200715230104:1e4bbb35.x86_64 from CentOS-AppStream 2024-04-12T20:41:31.743 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:8030020200715230104:ea09926d.x86_64 from CentOS-AppStream 2024-04-12T20:41:31.743 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:082fdf2f.x86_64 from CentOS-AppStream 2024-04-12T20:41:31.743 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-04-12T20:41:31.743 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-04-12T20:41:31.743 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:fbe42456.x86_64 from CentOS-AppStream 2024-04-12T20:41:31.743 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 40: cannot install the best candidate for the job 2024-04-12T20:41:31.744 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:8010020191114031513:16b3ab4d.x86_64 from CentOS-AppStream 2024-04-12T20:41:31.744 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:8010020191114031513:e1f37755.x86_64 from CentOS-AppStream 2024-04-12T20:41:31.744 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-04-12T20:41:31.744 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-04-12T20:41:31.744 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 41: cannot install the best candidate for the job 2024-04-12T20:41:31.744 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:8030020200716150652:1e4bbb35.x86_64 from CentOS-AppStream 2024-04-12T20:41:31.744 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:8030020200716150652:ea09926d.x86_64 from CentOS-AppStream 2024-04-12T20:41:31.744 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-04-12T20:41:31.744 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-04-12T20:41:31.744 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 42: cannot install the best candidate for the job 2024-04-12T20:41:31.744 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:8030020200716174729:3a70019f.x86_64 from CentOS-AppStream 2024-04-12T20:41:31.745 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:8030020200716174729:cccafca5.x86_64 from CentOS-AppStream 2024-04-12T20:41:31.745 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:820230223114759:6fdf7e5d.x86_64 from CentOS-AppStream 2024-04-12T20:41:31.745 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:820230223114759:d4326aba.x86_64 from CentOS-AppStream 2024-04-12T20:41:31.745 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 43: cannot install the best candidate for the job 2024-04-12T20:41:31.745 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:8030020200716213634:56fce90f.x86_64 from CentOS-AppStream 2024-04-12T20:41:31.745 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:8030020200716213634:98a3c9d0.x86_64 from CentOS-AppStream 2024-04-12T20:41:31.745 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:820230223114759:06b7596e.x86_64 from CentOS-AppStream 2024-04-12T20:41:31.745 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:820230223114759:d2fc5d76.x86_64 from CentOS-AppStream 2024-04-12T20:41:31.745 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 44: cannot install the best candidate for the job 2024-04-12T20:41:31.745 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:8030020200716171139:3a70019f.x86_64 from CentOS-AppStream 2024-04-12T20:41:31.745 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:8030020200716171139:cccafca5.x86_64 from CentOS-AppStream 2024-04-12T20:41:31.746 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:820230223114759:6fdf7e5d.x86_64 from CentOS-AppStream 2024-04-12T20:41:31.746 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:820230223114759:d4326aba.x86_64 from CentOS-AppStream 2024-04-12T20:41:31.746 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 45: cannot install the best candidate for the job 2024-04-12T20:41:31.746 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:8010020191120175858:95fce66d.x86_64 from CentOS-AppStream 2024-04-12T20:41:31.746 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:8030020200723233049:09acf126.x86_64 from CentOS-AppStream 2024-04-12T20:41:31.746 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:820230223114759:436af1ae.x86_64 from CentOS-AppStream 2024-04-12T20:41:31.746 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:820230223114759:b12195ea.x86_64 from CentOS-AppStream 2024-04-12T20:41:31.804 INFO:teuthology.orchestra.run.smithi186.stdout:Dependencies resolved. 2024-04-12T20:41:31.805 INFO:teuthology.orchestra.run.smithi186.stdout:================================================================================ 2024-04-12T20:41:31.806 INFO:teuthology.orchestra.run.smithi186.stdout: Package Arch Version Repo Size 2024-04-12T20:41:31.806 INFO:teuthology.orchestra.run.smithi186.stdout:================================================================================ 2024-04-12T20:41:31.806 INFO:teuthology.orchestra.run.smithi186.stdout:Installing: 2024-04-12T20:41:31.806 INFO:teuthology.orchestra.run.smithi186.stdout: librados-devel x86_64 2:17.2.7-827.gd3b5060a.el9 ceph 126 k 2024-04-12T20:41:31.806 INFO:teuthology.orchestra.run.smithi186.stdout: 2024-04-12T20:41:31.806 INFO:teuthology.orchestra.run.smithi186.stdout:Transaction Summary 2024-04-12T20:41:31.806 INFO:teuthology.orchestra.run.smithi186.stdout:================================================================================ 2024-04-12T20:41:31.806 INFO:teuthology.orchestra.run.smithi186.stdout:Install 1 Package 2024-04-12T20:41:31.806 INFO:teuthology.orchestra.run.smithi186.stdout: 2024-04-12T20:41:31.806 INFO:teuthology.orchestra.run.smithi186.stdout:Total download size: 126 k 2024-04-12T20:41:31.806 INFO:teuthology.orchestra.run.smithi186.stdout:Installed size: 448 k 2024-04-12T20:41:31.807 INFO:teuthology.orchestra.run.smithi186.stdout:Downloading Packages: 2024-04-12T20:41:31.989 INFO:teuthology.orchestra.run.smithi018.stdout:librados-devel-17.2.7-827.gd3b5060a.el9.x86_64. 317 kB/s | 126 kB 00:00 2024-04-12T20:41:31.990 INFO:teuthology.orchestra.run.smithi018.stdout:-------------------------------------------------------------------------------- 2024-04-12T20:41:31.991 INFO:teuthology.orchestra.run.smithi018.stdout:Total 315 kB/s | 126 kB 00:00 2024-04-12T20:41:31.992 INFO:teuthology.orchestra.run.smithi018.stdout:Running transaction check 2024-04-12T20:41:32.012 INFO:teuthology.orchestra.run.smithi018.stdout:Transaction check succeeded. 2024-04-12T20:41:32.012 INFO:teuthology.orchestra.run.smithi018.stdout:Running transaction test 2024-04-12T20:41:32.031 INFO:teuthology.orchestra.run.smithi186.stdout:librados-devel-17.2.7-827.gd3b5060a.el9.x86_64. 562 kB/s | 126 kB 00:00 2024-04-12T20:41:32.033 INFO:teuthology.orchestra.run.smithi186.stdout:-------------------------------------------------------------------------------- 2024-04-12T20:41:32.034 INFO:teuthology.orchestra.run.smithi186.stdout:Total 554 kB/s | 126 kB 00:00 2024-04-12T20:41:32.034 INFO:teuthology.orchestra.run.smithi186.stdout:Running transaction check 2024-04-12T20:41:32.053 INFO:teuthology.orchestra.run.smithi186.stdout:Transaction check succeeded. 2024-04-12T20:41:32.053 INFO:teuthology.orchestra.run.smithi186.stdout:Running transaction test 2024-04-12T20:41:32.126 INFO:teuthology.orchestra.run.smithi018.stdout:Transaction test succeeded. 2024-04-12T20:41:32.126 INFO:teuthology.orchestra.run.smithi018.stdout:Running transaction 2024-04-12T20:41:32.159 INFO:teuthology.orchestra.run.smithi186.stdout:Transaction test succeeded. 2024-04-12T20:41:32.160 INFO:teuthology.orchestra.run.smithi186.stdout:Running transaction 2024-04-12T20:41:32.325 INFO:teuthology.orchestra.run.smithi018.stdout: Preparing : 1/1 2024-04-12T20:41:32.360 INFO:teuthology.orchestra.run.smithi186.stdout: Preparing : 1/1 2024-04-12T20:41:32.447 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : librados-devel-2:17.2.7-827.gd3b5060a.el9.x86_64 1/1 2024-04-12T20:41:32.513 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : librados-devel-2:17.2.7-827.gd3b5060a.el9.x86_64 1/1 2024-04-12T20:41:33.015 INFO:teuthology.orchestra.run.smithi018.stdout: Running scriptlet: librados-devel-2:17.2.7-827.gd3b5060a.el9.x86_64 1/1 2024-04-12T20:41:33.158 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: librados-devel-2:17.2.7-827.gd3b5060a.el9.x86_64 1/1 2024-04-12T20:41:33.367 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : librados-devel-2:17.2.7-827.gd3b5060a.el9.x86_64 1/1 2024-04-12T20:41:33.367 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-12T20:41:33.367 INFO:teuthology.orchestra.run.smithi018.stdout:Installed: 2024-04-12T20:41:33.367 INFO:teuthology.orchestra.run.smithi018.stdout: librados-devel-2:17.2.7-827.gd3b5060a.el9.x86_64 2024-04-12T20:41:33.367 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-12T20:41:33.367 INFO:teuthology.orchestra.run.smithi018.stdout:Complete! 2024-04-12T20:41:33.564 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : librados-devel-2:17.2.7-827.gd3b5060a.el9.x86_64 1/1 2024-04-12T20:41:33.564 INFO:teuthology.orchestra.run.smithi186.stdout: 2024-04-12T20:41:33.564 INFO:teuthology.orchestra.run.smithi186.stdout:Installed: 2024-04-12T20:41:33.564 INFO:teuthology.orchestra.run.smithi186.stdout: librados-devel-2:17.2.7-827.gd3b5060a.el9.x86_64 2024-04-12T20:41:33.564 INFO:teuthology.orchestra.run.smithi186.stdout: 2024-04-12T20:41:33.564 INFO:teuthology.orchestra.run.smithi186.stdout:Complete! 2024-04-12T20:41:33.711 DEBUG:teuthology.orchestra.run.smithi018:> sudo yum -y install libcephfs2 2024-04-12T20:41:33.881 DEBUG:teuthology.orchestra.run.smithi186:> sudo yum -y install libcephfs2 2024-04-12T20:41:34.641 INFO:teuthology.orchestra.run.smithi018.stdout:Last metadata expiration check: 0:01:59 ago on Fri 12 Apr 2024 08:39:35 PM UTC. 2024-04-12T20:41:34.776 INFO:teuthology.orchestra.run.smithi186.stdout:Last metadata expiration check: 0:01:57 ago on Fri 12 Apr 2024 08:39:37 PM UTC. 2024-04-12T20:41:34.964 INFO:teuthology.orchestra.run.smithi018.stderr:Modular dependency problems: 2024-04-12T20:41:34.964 INFO:teuthology.orchestra.run.smithi018.stderr: 2024-04-12T20:41:34.964 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 1: conflicting requests 2024-04-12T20:41:34.964 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module container-tools:rhel8:820240321091303:20125149.x86_64 from CentOS-AppStream 2024-04-12T20:41:34.964 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 2: conflicting requests 2024-04-12T20:41:34.964 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module eclipse:rhel8:820201023100746:b230ed4e.x86_64 from CentOS-AppStream 2024-04-12T20:41:34.964 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 3: conflicting requests 2024-04-12T20:41:34.964 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module freeradius:3.0:8080020230117180605:89170a74.x86_64 from CentOS-AppStream 2024-04-12T20:41:34.964 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 4: conflicting requests 2024-04-12T20:41:34.964 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module gimp:2.8:8000020190628145146:4148dfdf.x86_64 from CentOS-AppStream 2024-04-12T20:41:34.964 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 5: conflicting requests 2024-04-12T20:41:34.965 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module go-toolset:rhel8:820240401160413:b754926a.x86_64 from CentOS-AppStream 2024-04-12T20:41:34.965 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 6: conflicting requests 2024-04-12T20:41:34.965 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module httpd:2.4:820240216084734:9edba152.x86_64 from CentOS-AppStream 2024-04-12T20:41:34.965 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 7: conflicting requests 2024-04-12T20:41:34.965 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module idm:client:820240307184541:49cc9d1b.x86_64 from CentOS-AppStream 2024-04-12T20:41:34.965 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 8: conflicting requests 2024-04-12T20:41:34.965 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module inkscape:0.92.3:8080020221205124429:3e031279.x86_64 from CentOS-AppStream 2024-04-12T20:41:34.965 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 9: conflicting requests 2024-04-12T20:41:34.965 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module jmc:rhel8:8050020211110222101:6392b1f8.x86_64 from CentOS-AppStream 2024-04-12T20:41:34.965 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 10: conflicting requests 2024-04-12T20:41:34.965 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module llvm-toolset:rhel8:820240205133135:9edba152.x86_64 from CentOS-AppStream 2024-04-12T20:41:34.965 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 11: conflicting requests 2024-04-12T20:41:34.965 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module log4j:2:8080020221205124743:9d367344.x86_64 from CentOS-AppStream 2024-04-12T20:41:34.965 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 12: conflicting requests 2024-04-12T20:41:34.966 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module mailman:2.1:820230727085757:77fc8825.x86_64 from CentOS-AppStream 2024-04-12T20:41:34.966 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 13: conflicting requests 2024-04-12T20:41:34.966 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module mariadb:10.3:8030020210419150013:30b713e6.x86_64 from CentOS-AppStream 2024-04-12T20:41:34.966 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 14: conflicting requests 2024-04-12T20:41:34.966 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module maven:3.5:8060020211203142015:c0229ad2.x86_64 from CentOS-AppStream 2024-04-12T20:41:34.966 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 15: conflicting requests 2024-04-12T20:41:34.966 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module mercurial:4.8:8000020190628020724:4148dfdf.x86_64 from CentOS-AppStream 2024-04-12T20:41:34.966 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 16: conflicting requests 2024-04-12T20:41:34.966 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module mod_auth_openidc:2.3:820230721123832:9edba152.x86_64 from CentOS-AppStream 2024-04-12T20:41:34.966 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 17: conflicting requests 2024-04-12T20:41:34.966 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module mysql:8.0:8040020210901180257:9f9e2e7e.x86_64 from CentOS-AppStream 2024-04-12T20:41:34.966 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 18: conflicting requests 2024-04-12T20:41:34.966 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module nginx:1.14:8000020211221191913:55190bc5.x86_64 from CentOS-AppStream 2024-04-12T20:41:34.967 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 19: conflicting requests 2024-04-12T20:41:34.967 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module nodejs:10:8040020210120182536:9f9e2e7e.x86_64 from CentOS-AppStream 2024-04-12T20:41:34.967 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 20: conflicting requests 2024-04-12T20:41:34.967 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl:5.26:8000020190628020724:55190bc5.x86_64 from CentOS-AppStream 2024-04-12T20:41:34.967 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 21: conflicting requests 2024-04-12T20:41:34.967 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module php:7.2:8020020200507003613:2c7ca891.x86_64 from CentOS-AppStream 2024-04-12T20:41:34.967 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 22: conflicting requests 2024-04-12T20:41:34.967 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module postgresql:10:820230223114800:9edba152.x86_64 from CentOS-AppStream 2024-04-12T20:41:34.967 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 23: conflicting requests 2024-04-12T20:41:34.967 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module python27:2.7:820240208011952:182f7c73.x86_64 from CentOS-AppStream 2024-04-12T20:41:34.967 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 24: conflicting requests 2024-04-12T20:41:34.967 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module python36:3.6:8050020210825152031:982725ab.x86_64 from CentOS-AppStream 2024-04-12T20:41:34.967 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 25: conflicting requests 2024-04-12T20:41:34.968 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module python38:3.8:820230810143931:64c1cd5a.x86_64 from CentOS-AppStream 2024-04-12T20:41:34.968 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 26: conflicting requests 2024-04-12T20:41:34.968 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module python39:3.9:820240214182535:17377f89.x86_64 from CentOS-AppStream 2024-04-12T20:41:34.968 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 27: conflicting requests 2024-04-12T20:41:34.968 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module redis:5:8040020211019153849:9f9e2e7e.x86_64 from CentOS-AppStream 2024-04-12T20:41:34.968 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 28: conflicting requests 2024-04-12T20:41:34.968 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module rhn-tools:1.0:8010020191114034948:f69d1239.x86_64 from CentOS-AppStream 2024-04-12T20:41:34.968 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 29: conflicting requests 2024-04-12T20:41:34.968 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module ruby:2.5:820230627084142:9edba152.x86_64 from CentOS-AppStream 2024-04-12T20:41:34.968 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 30: conflicting requests 2024-04-12T20:41:34.968 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module rust-toolset:rhel8:820240119204620:b09eea91.x86_64 from CentOS-AppStream 2024-04-12T20:41:34.968 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 31: conflicting requests 2024-04-12T20:41:34.968 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module satellite-5-client:1.0:8010020191114035551:cdc1202b.x86_64 from CentOS-AppStream 2024-04-12T20:41:34.968 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 32: conflicting requests 2024-04-12T20:41:34.969 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module squid:4:820240319173245:9edba152.x86_64 from CentOS-AppStream 2024-04-12T20:41:34.969 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 33: conflicting requests 2024-04-12T20:41:34.969 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module subversion:1.10:8070020220711155714:78111232.x86_64 from CentOS-AppStream 2024-04-12T20:41:34.969 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 34: conflicting requests 2024-04-12T20:41:34.969 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module swig:3.0:8030020200706142531:30b713e6.x86_64 from CentOS-AppStream 2024-04-12T20:41:34.969 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 35: conflicting requests 2024-04-12T20:41:34.969 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module varnish:6:8050020211109225449:b4937e53.x86_64 from CentOS-AppStream 2024-04-12T20:41:34.969 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 36: conflicting requests 2024-04-12T20:41:34.969 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module virt:rhel:820240314161907:9edba152.x86_64 from CentOS-AppStream 2024-04-12T20:41:34.969 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 37: conflicting requests 2024-04-12T20:41:34.969 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:8030020200716155257:7cc0a66d.x86_64 from CentOS-AppStream 2024-04-12T20:41:34.969 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:8030020200716155257:b967a9a2.x86_64 from CentOS-AppStream 2024-04-12T20:41:34.969 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:820230811133638:36f9fae6.x86_64 from CentOS-AppStream 2024-04-12T20:41:34.969 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 38: cannot install the best candidate for the job 2024-04-12T20:41:34.970 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:8010020191114031501:a5949e2e.x86_64 from CentOS-AppStream 2024-04-12T20:41:34.970 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:8010020191114031501:be2e4737.x86_64 from CentOS-AppStream 2024-04-12T20:41:34.970 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:820230223114759:2ab761e1.x86_64 from CentOS-AppStream 2024-04-12T20:41:34.970 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:820230223114759:86e45846.x86_64 from CentOS-AppStream 2024-04-12T20:41:34.970 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 39: conflicting requests 2024-04-12T20:41:34.970 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:8030020200715230104:1e4bbb35.x86_64 from CentOS-AppStream 2024-04-12T20:41:34.970 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:8030020200715230104:ea09926d.x86_64 from CentOS-AppStream 2024-04-12T20:41:34.970 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:082fdf2f.x86_64 from CentOS-AppStream 2024-04-12T20:41:34.970 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-04-12T20:41:34.970 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-04-12T20:41:34.970 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:fbe42456.x86_64 from CentOS-AppStream 2024-04-12T20:41:34.970 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 40: cannot install the best candidate for the job 2024-04-12T20:41:34.970 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:8010020191114031513:16b3ab4d.x86_64 from CentOS-AppStream 2024-04-12T20:41:34.970 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:8010020191114031513:e1f37755.x86_64 from CentOS-AppStream 2024-04-12T20:41:34.971 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-04-12T20:41:34.971 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-04-12T20:41:34.971 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 41: cannot install the best candidate for the job 2024-04-12T20:41:34.971 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:8030020200716150652:1e4bbb35.x86_64 from CentOS-AppStream 2024-04-12T20:41:34.971 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:8030020200716150652:ea09926d.x86_64 from CentOS-AppStream 2024-04-12T20:41:34.971 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-04-12T20:41:34.971 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-04-12T20:41:34.971 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 42: cannot install the best candidate for the job 2024-04-12T20:41:34.971 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:8030020200716174729:3a70019f.x86_64 from CentOS-AppStream 2024-04-12T20:41:34.971 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:8030020200716174729:cccafca5.x86_64 from CentOS-AppStream 2024-04-12T20:41:34.971 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:820230223114759:6fdf7e5d.x86_64 from CentOS-AppStream 2024-04-12T20:41:34.971 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:820230223114759:d4326aba.x86_64 from CentOS-AppStream 2024-04-12T20:41:34.971 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 43: cannot install the best candidate for the job 2024-04-12T20:41:34.972 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:8030020200716213634:56fce90f.x86_64 from CentOS-AppStream 2024-04-12T20:41:34.972 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:8030020200716213634:98a3c9d0.x86_64 from CentOS-AppStream 2024-04-12T20:41:34.972 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:820230223114759:06b7596e.x86_64 from CentOS-AppStream 2024-04-12T20:41:34.972 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:820230223114759:d2fc5d76.x86_64 from CentOS-AppStream 2024-04-12T20:41:34.972 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 44: cannot install the best candidate for the job 2024-04-12T20:41:34.972 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:8030020200716171139:3a70019f.x86_64 from CentOS-AppStream 2024-04-12T20:41:34.972 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:8030020200716171139:cccafca5.x86_64 from CentOS-AppStream 2024-04-12T20:41:34.972 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:820230223114759:6fdf7e5d.x86_64 from CentOS-AppStream 2024-04-12T20:41:34.972 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:820230223114759:d4326aba.x86_64 from CentOS-AppStream 2024-04-12T20:41:34.972 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 45: cannot install the best candidate for the job 2024-04-12T20:41:34.972 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:8010020191120175858:95fce66d.x86_64 from CentOS-AppStream 2024-04-12T20:41:34.972 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:8030020200723233049:09acf126.x86_64 from CentOS-AppStream 2024-04-12T20:41:34.972 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:820230223114759:436af1ae.x86_64 from CentOS-AppStream 2024-04-12T20:41:34.972 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:820230223114759:b12195ea.x86_64 from CentOS-AppStream 2024-04-12T20:41:34.973 INFO:teuthology.orchestra.run.smithi018.stdout:Package libcephfs2-2:17.2.7-827.gd3b5060a.el9.x86_64 is already installed. 2024-04-12T20:41:35.042 INFO:teuthology.orchestra.run.smithi018.stdout:Dependencies resolved. 2024-04-12T20:41:35.043 INFO:teuthology.orchestra.run.smithi018.stdout:Nothing to do. 2024-04-12T20:41:35.043 INFO:teuthology.orchestra.run.smithi018.stdout:Complete! 2024-04-12T20:41:35.081 INFO:teuthology.orchestra.run.smithi186.stderr:Modular dependency problems: 2024-04-12T20:41:35.081 INFO:teuthology.orchestra.run.smithi186.stderr: 2024-04-12T20:41:35.081 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 1: conflicting requests 2024-04-12T20:41:35.081 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module container-tools:rhel8:820240321091303:20125149.x86_64 from CentOS-AppStream 2024-04-12T20:41:35.081 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 2: conflicting requests 2024-04-12T20:41:35.081 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module eclipse:rhel8:820201023100746:b230ed4e.x86_64 from CentOS-AppStream 2024-04-12T20:41:35.081 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 3: conflicting requests 2024-04-12T20:41:35.081 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module freeradius:3.0:8080020230117180605:89170a74.x86_64 from CentOS-AppStream 2024-04-12T20:41:35.081 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 4: conflicting requests 2024-04-12T20:41:35.081 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module gimp:2.8:8000020190628145146:4148dfdf.x86_64 from CentOS-AppStream 2024-04-12T20:41:35.081 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 5: conflicting requests 2024-04-12T20:41:35.082 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module go-toolset:rhel8:820240401160413:b754926a.x86_64 from CentOS-AppStream 2024-04-12T20:41:35.082 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 6: conflicting requests 2024-04-12T20:41:35.082 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module httpd:2.4:820240216084734:9edba152.x86_64 from CentOS-AppStream 2024-04-12T20:41:35.082 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 7: conflicting requests 2024-04-12T20:41:35.082 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module idm:client:820240307184541:49cc9d1b.x86_64 from CentOS-AppStream 2024-04-12T20:41:35.082 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 8: conflicting requests 2024-04-12T20:41:35.082 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module inkscape:0.92.3:8080020221205124429:3e031279.x86_64 from CentOS-AppStream 2024-04-12T20:41:35.082 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 9: conflicting requests 2024-04-12T20:41:35.082 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module jmc:rhel8:8050020211110222101:6392b1f8.x86_64 from CentOS-AppStream 2024-04-12T20:41:35.082 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 10: conflicting requests 2024-04-12T20:41:35.082 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module llvm-toolset:rhel8:820240205133135:9edba152.x86_64 from CentOS-AppStream 2024-04-12T20:41:35.082 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 11: conflicting requests 2024-04-12T20:41:35.082 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module log4j:2:8080020221205124743:9d367344.x86_64 from CentOS-AppStream 2024-04-12T20:41:35.082 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 12: conflicting requests 2024-04-12T20:41:35.083 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module mailman:2.1:820230727085757:77fc8825.x86_64 from CentOS-AppStream 2024-04-12T20:41:35.083 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 13: conflicting requests 2024-04-12T20:41:35.083 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module mariadb:10.3:8030020210419150013:30b713e6.x86_64 from CentOS-AppStream 2024-04-12T20:41:35.083 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 14: conflicting requests 2024-04-12T20:41:35.083 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module maven:3.5:8060020211203142015:c0229ad2.x86_64 from CentOS-AppStream 2024-04-12T20:41:35.083 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 15: conflicting requests 2024-04-12T20:41:35.083 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module mercurial:4.8:8000020190628020724:4148dfdf.x86_64 from CentOS-AppStream 2024-04-12T20:41:35.083 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 16: conflicting requests 2024-04-12T20:41:35.083 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module mod_auth_openidc:2.3:820230721123832:9edba152.x86_64 from CentOS-AppStream 2024-04-12T20:41:35.083 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 17: conflicting requests 2024-04-12T20:41:35.083 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module mysql:8.0:8040020210901180257:9f9e2e7e.x86_64 from CentOS-AppStream 2024-04-12T20:41:35.083 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 18: conflicting requests 2024-04-12T20:41:35.083 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module nginx:1.14:8000020211221191913:55190bc5.x86_64 from CentOS-AppStream 2024-04-12T20:41:35.083 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 19: conflicting requests 2024-04-12T20:41:35.084 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module nodejs:10:8040020210120182536:9f9e2e7e.x86_64 from CentOS-AppStream 2024-04-12T20:41:35.084 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 20: conflicting requests 2024-04-12T20:41:35.084 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl:5.26:8000020190628020724:55190bc5.x86_64 from CentOS-AppStream 2024-04-12T20:41:35.084 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 21: conflicting requests 2024-04-12T20:41:35.084 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module php:7.2:8020020200507003613:2c7ca891.x86_64 from CentOS-AppStream 2024-04-12T20:41:35.084 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 22: conflicting requests 2024-04-12T20:41:35.084 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module postgresql:10:820230223114800:9edba152.x86_64 from CentOS-AppStream 2024-04-12T20:41:35.084 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 23: conflicting requests 2024-04-12T20:41:35.084 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module python27:2.7:820240208011952:182f7c73.x86_64 from CentOS-AppStream 2024-04-12T20:41:35.084 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 24: conflicting requests 2024-04-12T20:41:35.084 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module python36:3.6:8050020210825152031:982725ab.x86_64 from CentOS-AppStream 2024-04-12T20:41:35.084 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 25: conflicting requests 2024-04-12T20:41:35.084 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module python38:3.8:820230810143931:64c1cd5a.x86_64 from CentOS-AppStream 2024-04-12T20:41:35.084 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 26: conflicting requests 2024-04-12T20:41:35.084 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module python39:3.9:820240214182535:17377f89.x86_64 from CentOS-AppStream 2024-04-12T20:41:35.085 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 27: conflicting requests 2024-04-12T20:41:35.085 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module redis:5:8040020211019153849:9f9e2e7e.x86_64 from CentOS-AppStream 2024-04-12T20:41:35.085 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 28: conflicting requests 2024-04-12T20:41:35.085 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module rhn-tools:1.0:8010020191114034948:f69d1239.x86_64 from CentOS-AppStream 2024-04-12T20:41:35.085 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 29: conflicting requests 2024-04-12T20:41:35.085 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module ruby:2.5:820230627084142:9edba152.x86_64 from CentOS-AppStream 2024-04-12T20:41:35.085 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 30: conflicting requests 2024-04-12T20:41:35.085 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module rust-toolset:rhel8:820240119204620:b09eea91.x86_64 from CentOS-AppStream 2024-04-12T20:41:35.085 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 31: conflicting requests 2024-04-12T20:41:35.085 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module satellite-5-client:1.0:8010020191114035551:cdc1202b.x86_64 from CentOS-AppStream 2024-04-12T20:41:35.085 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 32: conflicting requests 2024-04-12T20:41:35.085 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module squid:4:820240319173245:9edba152.x86_64 from CentOS-AppStream 2024-04-12T20:41:35.085 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 33: conflicting requests 2024-04-12T20:41:35.085 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module subversion:1.10:8070020220711155714:78111232.x86_64 from CentOS-AppStream 2024-04-12T20:41:35.086 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 34: conflicting requests 2024-04-12T20:41:35.086 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module swig:3.0:8030020200706142531:30b713e6.x86_64 from CentOS-AppStream 2024-04-12T20:41:35.086 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 35: conflicting requests 2024-04-12T20:41:35.086 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module varnish:6:8050020211109225449:b4937e53.x86_64 from CentOS-AppStream 2024-04-12T20:41:35.086 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 36: conflicting requests 2024-04-12T20:41:35.086 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module virt:rhel:820240314161907:9edba152.x86_64 from CentOS-AppStream 2024-04-12T20:41:35.086 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 37: conflicting requests 2024-04-12T20:41:35.086 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:8030020200716155257:7cc0a66d.x86_64 from CentOS-AppStream 2024-04-12T20:41:35.086 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:8030020200716155257:b967a9a2.x86_64 from CentOS-AppStream 2024-04-12T20:41:35.086 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:820230811133638:36f9fae6.x86_64 from CentOS-AppStream 2024-04-12T20:41:35.086 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 38: cannot install the best candidate for the job 2024-04-12T20:41:35.086 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:8010020191114031501:a5949e2e.x86_64 from CentOS-AppStream 2024-04-12T20:41:35.086 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:8010020191114031501:be2e4737.x86_64 from CentOS-AppStream 2024-04-12T20:41:35.086 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:820230223114759:2ab761e1.x86_64 from CentOS-AppStream 2024-04-12T20:41:35.087 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:820230223114759:86e45846.x86_64 from CentOS-AppStream 2024-04-12T20:41:35.087 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 39: conflicting requests 2024-04-12T20:41:35.087 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:8030020200715230104:1e4bbb35.x86_64 from CentOS-AppStream 2024-04-12T20:41:35.087 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:8030020200715230104:ea09926d.x86_64 from CentOS-AppStream 2024-04-12T20:41:35.087 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:082fdf2f.x86_64 from CentOS-AppStream 2024-04-12T20:41:35.087 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-04-12T20:41:35.087 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-04-12T20:41:35.087 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:fbe42456.x86_64 from CentOS-AppStream 2024-04-12T20:41:35.087 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 40: cannot install the best candidate for the job 2024-04-12T20:41:35.087 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:8010020191114031513:16b3ab4d.x86_64 from CentOS-AppStream 2024-04-12T20:41:35.087 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:8010020191114031513:e1f37755.x86_64 from CentOS-AppStream 2024-04-12T20:41:35.087 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-04-12T20:41:35.087 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-04-12T20:41:35.087 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 41: cannot install the best candidate for the job 2024-04-12T20:41:35.088 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:8030020200716150652:1e4bbb35.x86_64 from CentOS-AppStream 2024-04-12T20:41:35.088 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:8030020200716150652:ea09926d.x86_64 from CentOS-AppStream 2024-04-12T20:41:35.088 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-04-12T20:41:35.088 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-04-12T20:41:35.088 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 42: cannot install the best candidate for the job 2024-04-12T20:41:35.088 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:8030020200716174729:3a70019f.x86_64 from CentOS-AppStream 2024-04-12T20:41:35.088 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:8030020200716174729:cccafca5.x86_64 from CentOS-AppStream 2024-04-12T20:41:35.088 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:820230223114759:6fdf7e5d.x86_64 from CentOS-AppStream 2024-04-12T20:41:35.088 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:820230223114759:d4326aba.x86_64 from CentOS-AppStream 2024-04-12T20:41:35.088 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 43: cannot install the best candidate for the job 2024-04-12T20:41:35.088 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:8030020200716213634:56fce90f.x86_64 from CentOS-AppStream 2024-04-12T20:41:35.088 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:8030020200716213634:98a3c9d0.x86_64 from CentOS-AppStream 2024-04-12T20:41:35.088 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:820230223114759:06b7596e.x86_64 from CentOS-AppStream 2024-04-12T20:41:35.088 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:820230223114759:d2fc5d76.x86_64 from CentOS-AppStream 2024-04-12T20:41:35.089 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 44: cannot install the best candidate for the job 2024-04-12T20:41:35.089 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:8030020200716171139:3a70019f.x86_64 from CentOS-AppStream 2024-04-12T20:41:35.089 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:8030020200716171139:cccafca5.x86_64 from CentOS-AppStream 2024-04-12T20:41:35.089 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:820230223114759:6fdf7e5d.x86_64 from CentOS-AppStream 2024-04-12T20:41:35.089 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:820230223114759:d4326aba.x86_64 from CentOS-AppStream 2024-04-12T20:41:35.089 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 45: cannot install the best candidate for the job 2024-04-12T20:41:35.089 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:8010020191120175858:95fce66d.x86_64 from CentOS-AppStream 2024-04-12T20:41:35.089 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:8030020200723233049:09acf126.x86_64 from CentOS-AppStream 2024-04-12T20:41:35.089 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:820230223114759:436af1ae.x86_64 from CentOS-AppStream 2024-04-12T20:41:35.089 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:820230223114759:b12195ea.x86_64 from CentOS-AppStream 2024-04-12T20:41:35.089 INFO:teuthology.orchestra.run.smithi186.stdout:Package libcephfs2-2:17.2.7-827.gd3b5060a.el9.x86_64 is already installed. 2024-04-12T20:41:35.117 DEBUG:teuthology.orchestra.run.smithi018:> sudo yum -y install libcephfs-devel 2024-04-12T20:41:35.155 INFO:teuthology.orchestra.run.smithi186.stdout:Dependencies resolved. 2024-04-12T20:41:35.156 INFO:teuthology.orchestra.run.smithi186.stdout:Nothing to do. 2024-04-12T20:41:35.156 INFO:teuthology.orchestra.run.smithi186.stdout:Complete! 2024-04-12T20:41:35.231 DEBUG:teuthology.orchestra.run.smithi186:> sudo yum -y install libcephfs-devel 2024-04-12T20:41:36.029 INFO:teuthology.orchestra.run.smithi018.stdout:Last metadata expiration check: 0:02:01 ago on Fri 12 Apr 2024 08:39:35 PM UTC. 2024-04-12T20:41:36.095 INFO:teuthology.orchestra.run.smithi186.stdout:Last metadata expiration check: 0:01:59 ago on Fri 12 Apr 2024 08:39:37 PM UTC. 2024-04-12T20:41:36.343 INFO:teuthology.orchestra.run.smithi018.stderr:Modular dependency problems: 2024-04-12T20:41:36.343 INFO:teuthology.orchestra.run.smithi018.stderr: 2024-04-12T20:41:36.343 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 1: conflicting requests 2024-04-12T20:41:36.343 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module container-tools:rhel8:820240321091303:20125149.x86_64 from CentOS-AppStream 2024-04-12T20:41:36.343 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 2: conflicting requests 2024-04-12T20:41:36.343 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module eclipse:rhel8:820201023100746:b230ed4e.x86_64 from CentOS-AppStream 2024-04-12T20:41:36.343 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 3: conflicting requests 2024-04-12T20:41:36.343 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module freeradius:3.0:8080020230117180605:89170a74.x86_64 from CentOS-AppStream 2024-04-12T20:41:36.344 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 4: conflicting requests 2024-04-12T20:41:36.344 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module gimp:2.8:8000020190628145146:4148dfdf.x86_64 from CentOS-AppStream 2024-04-12T20:41:36.344 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 5: conflicting requests 2024-04-12T20:41:36.344 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module go-toolset:rhel8:820240401160413:b754926a.x86_64 from CentOS-AppStream 2024-04-12T20:41:36.344 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 6: conflicting requests 2024-04-12T20:41:36.344 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module httpd:2.4:820240216084734:9edba152.x86_64 from CentOS-AppStream 2024-04-12T20:41:36.344 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 7: conflicting requests 2024-04-12T20:41:36.344 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module idm:client:820240307184541:49cc9d1b.x86_64 from CentOS-AppStream 2024-04-12T20:41:36.344 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 8: conflicting requests 2024-04-12T20:41:36.344 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module inkscape:0.92.3:8080020221205124429:3e031279.x86_64 from CentOS-AppStream 2024-04-12T20:41:36.344 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 9: conflicting requests 2024-04-12T20:41:36.344 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module jmc:rhel8:8050020211110222101:6392b1f8.x86_64 from CentOS-AppStream 2024-04-12T20:41:36.344 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 10: conflicting requests 2024-04-12T20:41:36.345 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module llvm-toolset:rhel8:820240205133135:9edba152.x86_64 from CentOS-AppStream 2024-04-12T20:41:36.345 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 11: conflicting requests 2024-04-12T20:41:36.345 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module log4j:2:8080020221205124743:9d367344.x86_64 from CentOS-AppStream 2024-04-12T20:41:36.345 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 12: conflicting requests 2024-04-12T20:41:36.345 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module mailman:2.1:820230727085757:77fc8825.x86_64 from CentOS-AppStream 2024-04-12T20:41:36.345 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 13: conflicting requests 2024-04-12T20:41:36.345 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module mariadb:10.3:8030020210419150013:30b713e6.x86_64 from CentOS-AppStream 2024-04-12T20:41:36.345 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 14: conflicting requests 2024-04-12T20:41:36.345 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module maven:3.5:8060020211203142015:c0229ad2.x86_64 from CentOS-AppStream 2024-04-12T20:41:36.345 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 15: conflicting requests 2024-04-12T20:41:36.345 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module mercurial:4.8:8000020190628020724:4148dfdf.x86_64 from CentOS-AppStream 2024-04-12T20:41:36.345 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 16: conflicting requests 2024-04-12T20:41:36.345 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module mod_auth_openidc:2.3:820230721123832:9edba152.x86_64 from CentOS-AppStream 2024-04-12T20:41:36.345 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 17: conflicting requests 2024-04-12T20:41:36.346 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module mysql:8.0:8040020210901180257:9f9e2e7e.x86_64 from CentOS-AppStream 2024-04-12T20:41:36.346 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 18: conflicting requests 2024-04-12T20:41:36.346 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module nginx:1.14:8000020211221191913:55190bc5.x86_64 from CentOS-AppStream 2024-04-12T20:41:36.346 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 19: conflicting requests 2024-04-12T20:41:36.346 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module nodejs:10:8040020210120182536:9f9e2e7e.x86_64 from CentOS-AppStream 2024-04-12T20:41:36.346 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 20: conflicting requests 2024-04-12T20:41:36.346 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl:5.26:8000020190628020724:55190bc5.x86_64 from CentOS-AppStream 2024-04-12T20:41:36.346 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 21: conflicting requests 2024-04-12T20:41:36.346 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module php:7.2:8020020200507003613:2c7ca891.x86_64 from CentOS-AppStream 2024-04-12T20:41:36.346 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 22: conflicting requests 2024-04-12T20:41:36.346 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module postgresql:10:820230223114800:9edba152.x86_64 from CentOS-AppStream 2024-04-12T20:41:36.346 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 23: conflicting requests 2024-04-12T20:41:36.346 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module python27:2.7:820240208011952:182f7c73.x86_64 from CentOS-AppStream 2024-04-12T20:41:36.347 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 24: conflicting requests 2024-04-12T20:41:36.347 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module python36:3.6:8050020210825152031:982725ab.x86_64 from CentOS-AppStream 2024-04-12T20:41:36.347 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 25: conflicting requests 2024-04-12T20:41:36.347 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module python38:3.8:820230810143931:64c1cd5a.x86_64 from CentOS-AppStream 2024-04-12T20:41:36.347 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 26: conflicting requests 2024-04-12T20:41:36.347 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module python39:3.9:820240214182535:17377f89.x86_64 from CentOS-AppStream 2024-04-12T20:41:36.347 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 27: conflicting requests 2024-04-12T20:41:36.347 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module redis:5:8040020211019153849:9f9e2e7e.x86_64 from CentOS-AppStream 2024-04-12T20:41:36.347 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 28: conflicting requests 2024-04-12T20:41:36.347 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module rhn-tools:1.0:8010020191114034948:f69d1239.x86_64 from CentOS-AppStream 2024-04-12T20:41:36.347 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 29: conflicting requests 2024-04-12T20:41:36.347 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module ruby:2.5:820230627084142:9edba152.x86_64 from CentOS-AppStream 2024-04-12T20:41:36.347 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 30: conflicting requests 2024-04-12T20:41:36.347 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module rust-toolset:rhel8:820240119204620:b09eea91.x86_64 from CentOS-AppStream 2024-04-12T20:41:36.348 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 31: conflicting requests 2024-04-12T20:41:36.348 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module satellite-5-client:1.0:8010020191114035551:cdc1202b.x86_64 from CentOS-AppStream 2024-04-12T20:41:36.348 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 32: conflicting requests 2024-04-12T20:41:36.348 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module squid:4:820240319173245:9edba152.x86_64 from CentOS-AppStream 2024-04-12T20:41:36.348 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 33: conflicting requests 2024-04-12T20:41:36.348 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module subversion:1.10:8070020220711155714:78111232.x86_64 from CentOS-AppStream 2024-04-12T20:41:36.348 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 34: conflicting requests 2024-04-12T20:41:36.348 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module swig:3.0:8030020200706142531:30b713e6.x86_64 from CentOS-AppStream 2024-04-12T20:41:36.348 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 35: conflicting requests 2024-04-12T20:41:36.348 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module varnish:6:8050020211109225449:b4937e53.x86_64 from CentOS-AppStream 2024-04-12T20:41:36.348 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 36: conflicting requests 2024-04-12T20:41:36.348 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module virt:rhel:820240314161907:9edba152.x86_64 from CentOS-AppStream 2024-04-12T20:41:36.348 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 37: conflicting requests 2024-04-12T20:41:36.348 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:8030020200716155257:7cc0a66d.x86_64 from CentOS-AppStream 2024-04-12T20:41:36.349 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:8030020200716155257:b967a9a2.x86_64 from CentOS-AppStream 2024-04-12T20:41:36.349 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:820230811133638:36f9fae6.x86_64 from CentOS-AppStream 2024-04-12T20:41:36.349 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 38: cannot install the best candidate for the job 2024-04-12T20:41:36.349 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:8010020191114031501:a5949e2e.x86_64 from CentOS-AppStream 2024-04-12T20:41:36.349 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:8010020191114031501:be2e4737.x86_64 from CentOS-AppStream 2024-04-12T20:41:36.349 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:820230223114759:2ab761e1.x86_64 from CentOS-AppStream 2024-04-12T20:41:36.349 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:820230223114759:86e45846.x86_64 from CentOS-AppStream 2024-04-12T20:41:36.349 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 39: conflicting requests 2024-04-12T20:41:36.349 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:8030020200715230104:1e4bbb35.x86_64 from CentOS-AppStream 2024-04-12T20:41:36.349 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:8030020200715230104:ea09926d.x86_64 from CentOS-AppStream 2024-04-12T20:41:36.349 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:082fdf2f.x86_64 from CentOS-AppStream 2024-04-12T20:41:36.349 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-04-12T20:41:36.349 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-04-12T20:41:36.349 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:fbe42456.x86_64 from CentOS-AppStream 2024-04-12T20:41:36.349 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 40: cannot install the best candidate for the job 2024-04-12T20:41:36.350 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:8010020191114031513:16b3ab4d.x86_64 from CentOS-AppStream 2024-04-12T20:41:36.350 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:8010020191114031513:e1f37755.x86_64 from CentOS-AppStream 2024-04-12T20:41:36.350 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-04-12T20:41:36.350 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-04-12T20:41:36.350 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 41: cannot install the best candidate for the job 2024-04-12T20:41:36.350 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:8030020200716150652:1e4bbb35.x86_64 from CentOS-AppStream 2024-04-12T20:41:36.350 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:8030020200716150652:ea09926d.x86_64 from CentOS-AppStream 2024-04-12T20:41:36.350 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-04-12T20:41:36.350 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-04-12T20:41:36.350 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 42: cannot install the best candidate for the job 2024-04-12T20:41:36.350 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:8030020200716174729:3a70019f.x86_64 from CentOS-AppStream 2024-04-12T20:41:36.350 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:8030020200716174729:cccafca5.x86_64 from CentOS-AppStream 2024-04-12T20:41:36.350 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:820230223114759:6fdf7e5d.x86_64 from CentOS-AppStream 2024-04-12T20:41:36.351 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:820230223114759:d4326aba.x86_64 from CentOS-AppStream 2024-04-12T20:41:36.351 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 43: cannot install the best candidate for the job 2024-04-12T20:41:36.351 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:8030020200716213634:56fce90f.x86_64 from CentOS-AppStream 2024-04-12T20:41:36.351 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:8030020200716213634:98a3c9d0.x86_64 from CentOS-AppStream 2024-04-12T20:41:36.351 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:820230223114759:06b7596e.x86_64 from CentOS-AppStream 2024-04-12T20:41:36.351 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:820230223114759:d2fc5d76.x86_64 from CentOS-AppStream 2024-04-12T20:41:36.351 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 44: cannot install the best candidate for the job 2024-04-12T20:41:36.351 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:8030020200716171139:3a70019f.x86_64 from CentOS-AppStream 2024-04-12T20:41:36.351 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:8030020200716171139:cccafca5.x86_64 from CentOS-AppStream 2024-04-12T20:41:36.351 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:820230223114759:6fdf7e5d.x86_64 from CentOS-AppStream 2024-04-12T20:41:36.351 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:820230223114759:d4326aba.x86_64 from CentOS-AppStream 2024-04-12T20:41:36.351 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 45: cannot install the best candidate for the job 2024-04-12T20:41:36.351 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:8010020191120175858:95fce66d.x86_64 from CentOS-AppStream 2024-04-12T20:41:36.351 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:8030020200723233049:09acf126.x86_64 from CentOS-AppStream 2024-04-12T20:41:36.352 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:820230223114759:436af1ae.x86_64 from CentOS-AppStream 2024-04-12T20:41:36.352 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:820230223114759:b12195ea.x86_64 from CentOS-AppStream 2024-04-12T20:41:36.394 INFO:teuthology.orchestra.run.smithi186.stderr:Modular dependency problems: 2024-04-12T20:41:36.394 INFO:teuthology.orchestra.run.smithi186.stderr: 2024-04-12T20:41:36.394 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 1: conflicting requests 2024-04-12T20:41:36.394 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module container-tools:rhel8:820240321091303:20125149.x86_64 from CentOS-AppStream 2024-04-12T20:41:36.394 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 2: conflicting requests 2024-04-12T20:41:36.394 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module eclipse:rhel8:820201023100746:b230ed4e.x86_64 from CentOS-AppStream 2024-04-12T20:41:36.394 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 3: conflicting requests 2024-04-12T20:41:36.394 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module freeradius:3.0:8080020230117180605:89170a74.x86_64 from CentOS-AppStream 2024-04-12T20:41:36.394 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 4: conflicting requests 2024-04-12T20:41:36.395 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module gimp:2.8:8000020190628145146:4148dfdf.x86_64 from CentOS-AppStream 2024-04-12T20:41:36.395 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 5: conflicting requests 2024-04-12T20:41:36.395 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module go-toolset:rhel8:820240401160413:b754926a.x86_64 from CentOS-AppStream 2024-04-12T20:41:36.395 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 6: conflicting requests 2024-04-12T20:41:36.395 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module httpd:2.4:820240216084734:9edba152.x86_64 from CentOS-AppStream 2024-04-12T20:41:36.395 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 7: conflicting requests 2024-04-12T20:41:36.395 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module idm:client:820240307184541:49cc9d1b.x86_64 from CentOS-AppStream 2024-04-12T20:41:36.395 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 8: conflicting requests 2024-04-12T20:41:36.395 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module inkscape:0.92.3:8080020221205124429:3e031279.x86_64 from CentOS-AppStream 2024-04-12T20:41:36.395 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 9: conflicting requests 2024-04-12T20:41:36.395 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module jmc:rhel8:8050020211110222101:6392b1f8.x86_64 from CentOS-AppStream 2024-04-12T20:41:36.395 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 10: conflicting requests 2024-04-12T20:41:36.395 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module llvm-toolset:rhel8:820240205133135:9edba152.x86_64 from CentOS-AppStream 2024-04-12T20:41:36.395 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 11: conflicting requests 2024-04-12T20:41:36.395 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module log4j:2:8080020221205124743:9d367344.x86_64 from CentOS-AppStream 2024-04-12T20:41:36.396 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 12: conflicting requests 2024-04-12T20:41:36.396 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module mailman:2.1:820230727085757:77fc8825.x86_64 from CentOS-AppStream 2024-04-12T20:41:36.396 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 13: conflicting requests 2024-04-12T20:41:36.396 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module mariadb:10.3:8030020210419150013:30b713e6.x86_64 from CentOS-AppStream 2024-04-12T20:41:36.396 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 14: conflicting requests 2024-04-12T20:41:36.396 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module maven:3.5:8060020211203142015:c0229ad2.x86_64 from CentOS-AppStream 2024-04-12T20:41:36.396 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 15: conflicting requests 2024-04-12T20:41:36.396 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module mercurial:4.8:8000020190628020724:4148dfdf.x86_64 from CentOS-AppStream 2024-04-12T20:41:36.396 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 16: conflicting requests 2024-04-12T20:41:36.396 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module mod_auth_openidc:2.3:820230721123832:9edba152.x86_64 from CentOS-AppStream 2024-04-12T20:41:36.396 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 17: conflicting requests 2024-04-12T20:41:36.396 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module mysql:8.0:8040020210901180257:9f9e2e7e.x86_64 from CentOS-AppStream 2024-04-12T20:41:36.396 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 18: conflicting requests 2024-04-12T20:41:36.396 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module nginx:1.14:8000020211221191913:55190bc5.x86_64 from CentOS-AppStream 2024-04-12T20:41:36.397 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 19: conflicting requests 2024-04-12T20:41:36.397 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module nodejs:10:8040020210120182536:9f9e2e7e.x86_64 from CentOS-AppStream 2024-04-12T20:41:36.397 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 20: conflicting requests 2024-04-12T20:41:36.397 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl:5.26:8000020190628020724:55190bc5.x86_64 from CentOS-AppStream 2024-04-12T20:41:36.397 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 21: conflicting requests 2024-04-12T20:41:36.397 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module php:7.2:8020020200507003613:2c7ca891.x86_64 from CentOS-AppStream 2024-04-12T20:41:36.397 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 22: conflicting requests 2024-04-12T20:41:36.397 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module postgresql:10:820230223114800:9edba152.x86_64 from CentOS-AppStream 2024-04-12T20:41:36.397 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 23: conflicting requests 2024-04-12T20:41:36.397 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module python27:2.7:820240208011952:182f7c73.x86_64 from CentOS-AppStream 2024-04-12T20:41:36.397 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 24: conflicting requests 2024-04-12T20:41:36.397 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module python36:3.6:8050020210825152031:982725ab.x86_64 from CentOS-AppStream 2024-04-12T20:41:36.397 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 25: conflicting requests 2024-04-12T20:41:36.397 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module python38:3.8:820230810143931:64c1cd5a.x86_64 from CentOS-AppStream 2024-04-12T20:41:36.398 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 26: conflicting requests 2024-04-12T20:41:36.398 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module python39:3.9:820240214182535:17377f89.x86_64 from CentOS-AppStream 2024-04-12T20:41:36.398 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 27: conflicting requests 2024-04-12T20:41:36.398 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module redis:5:8040020211019153849:9f9e2e7e.x86_64 from CentOS-AppStream 2024-04-12T20:41:36.398 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 28: conflicting requests 2024-04-12T20:41:36.398 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module rhn-tools:1.0:8010020191114034948:f69d1239.x86_64 from CentOS-AppStream 2024-04-12T20:41:36.398 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 29: conflicting requests 2024-04-12T20:41:36.398 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module ruby:2.5:820230627084142:9edba152.x86_64 from CentOS-AppStream 2024-04-12T20:41:36.398 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 30: conflicting requests 2024-04-12T20:41:36.398 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module rust-toolset:rhel8:820240119204620:b09eea91.x86_64 from CentOS-AppStream 2024-04-12T20:41:36.398 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 31: conflicting requests 2024-04-12T20:41:36.398 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module satellite-5-client:1.0:8010020191114035551:cdc1202b.x86_64 from CentOS-AppStream 2024-04-12T20:41:36.398 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 32: conflicting requests 2024-04-12T20:41:36.399 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module squid:4:820240319173245:9edba152.x86_64 from CentOS-AppStream 2024-04-12T20:41:36.399 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 33: conflicting requests 2024-04-12T20:41:36.399 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module subversion:1.10:8070020220711155714:78111232.x86_64 from CentOS-AppStream 2024-04-12T20:41:36.399 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 34: conflicting requests 2024-04-12T20:41:36.399 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module swig:3.0:8030020200706142531:30b713e6.x86_64 from CentOS-AppStream 2024-04-12T20:41:36.399 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 35: conflicting requests 2024-04-12T20:41:36.399 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module varnish:6:8050020211109225449:b4937e53.x86_64 from CentOS-AppStream 2024-04-12T20:41:36.399 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 36: conflicting requests 2024-04-12T20:41:36.399 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module virt:rhel:820240314161907:9edba152.x86_64 from CentOS-AppStream 2024-04-12T20:41:36.399 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 37: conflicting requests 2024-04-12T20:41:36.399 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:8030020200716155257:7cc0a66d.x86_64 from CentOS-AppStream 2024-04-12T20:41:36.399 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:8030020200716155257:b967a9a2.x86_64 from CentOS-AppStream 2024-04-12T20:41:36.399 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:820230811133638:36f9fae6.x86_64 from CentOS-AppStream 2024-04-12T20:41:36.399 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 38: cannot install the best candidate for the job 2024-04-12T20:41:36.399 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:8010020191114031501:a5949e2e.x86_64 from CentOS-AppStream 2024-04-12T20:41:36.400 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:8010020191114031501:be2e4737.x86_64 from CentOS-AppStream 2024-04-12T20:41:36.400 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:820230223114759:2ab761e1.x86_64 from CentOS-AppStream 2024-04-12T20:41:36.400 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:820230223114759:86e45846.x86_64 from CentOS-AppStream 2024-04-12T20:41:36.400 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 39: conflicting requests 2024-04-12T20:41:36.400 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:8030020200715230104:1e4bbb35.x86_64 from CentOS-AppStream 2024-04-12T20:41:36.400 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:8030020200715230104:ea09926d.x86_64 from CentOS-AppStream 2024-04-12T20:41:36.400 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:082fdf2f.x86_64 from CentOS-AppStream 2024-04-12T20:41:36.400 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-04-12T20:41:36.400 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-04-12T20:41:36.400 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:fbe42456.x86_64 from CentOS-AppStream 2024-04-12T20:41:36.400 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 40: cannot install the best candidate for the job 2024-04-12T20:41:36.400 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:8010020191114031513:16b3ab4d.x86_64 from CentOS-AppStream 2024-04-12T20:41:36.401 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:8010020191114031513:e1f37755.x86_64 from CentOS-AppStream 2024-04-12T20:41:36.401 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-04-12T20:41:36.401 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-04-12T20:41:36.401 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 41: cannot install the best candidate for the job 2024-04-12T20:41:36.401 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:8030020200716150652:1e4bbb35.x86_64 from CentOS-AppStream 2024-04-12T20:41:36.401 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:8030020200716150652:ea09926d.x86_64 from CentOS-AppStream 2024-04-12T20:41:36.401 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-04-12T20:41:36.401 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-04-12T20:41:36.401 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 42: cannot install the best candidate for the job 2024-04-12T20:41:36.401 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:8030020200716174729:3a70019f.x86_64 from CentOS-AppStream 2024-04-12T20:41:36.401 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:8030020200716174729:cccafca5.x86_64 from CentOS-AppStream 2024-04-12T20:41:36.401 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:820230223114759:6fdf7e5d.x86_64 from CentOS-AppStream 2024-04-12T20:41:36.401 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:820230223114759:d4326aba.x86_64 from CentOS-AppStream 2024-04-12T20:41:36.401 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 43: cannot install the best candidate for the job 2024-04-12T20:41:36.402 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:8030020200716213634:56fce90f.x86_64 from CentOS-AppStream 2024-04-12T20:41:36.402 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:8030020200716213634:98a3c9d0.x86_64 from CentOS-AppStream 2024-04-12T20:41:36.402 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:820230223114759:06b7596e.x86_64 from CentOS-AppStream 2024-04-12T20:41:36.402 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:820230223114759:d2fc5d76.x86_64 from CentOS-AppStream 2024-04-12T20:41:36.402 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 44: cannot install the best candidate for the job 2024-04-12T20:41:36.402 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:8030020200716171139:3a70019f.x86_64 from CentOS-AppStream 2024-04-12T20:41:36.402 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:8030020200716171139:cccafca5.x86_64 from CentOS-AppStream 2024-04-12T20:41:36.402 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:820230223114759:6fdf7e5d.x86_64 from CentOS-AppStream 2024-04-12T20:41:36.402 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:820230223114759:d4326aba.x86_64 from CentOS-AppStream 2024-04-12T20:41:36.402 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 45: cannot install the best candidate for the job 2024-04-12T20:41:36.402 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:8010020191120175858:95fce66d.x86_64 from CentOS-AppStream 2024-04-12T20:41:36.402 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:8030020200723233049:09acf126.x86_64 from CentOS-AppStream 2024-04-12T20:41:36.402 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:820230223114759:436af1ae.x86_64 from CentOS-AppStream 2024-04-12T20:41:36.402 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:820230223114759:b12195ea.x86_64 from CentOS-AppStream 2024-04-12T20:41:36.421 INFO:teuthology.orchestra.run.smithi018.stdout:Dependencies resolved. 2024-04-12T20:41:36.421 INFO:teuthology.orchestra.run.smithi018.stdout:================================================================================ 2024-04-12T20:41:36.421 INFO:teuthology.orchestra.run.smithi018.stdout: Package Arch Version Repo Size 2024-04-12T20:41:36.421 INFO:teuthology.orchestra.run.smithi018.stdout:================================================================================ 2024-04-12T20:41:36.421 INFO:teuthology.orchestra.run.smithi018.stdout:Installing: 2024-04-12T20:41:36.421 INFO:teuthology.orchestra.run.smithi018.stdout: libcephfs-devel x86_64 2:17.2.7-827.gd3b5060a.el9 ceph 25 k 2024-04-12T20:41:36.422 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-12T20:41:36.422 INFO:teuthology.orchestra.run.smithi018.stdout:Transaction Summary 2024-04-12T20:41:36.422 INFO:teuthology.orchestra.run.smithi018.stdout:================================================================================ 2024-04-12T20:41:36.422 INFO:teuthology.orchestra.run.smithi018.stdout:Install 1 Package 2024-04-12T20:41:36.422 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-12T20:41:36.422 INFO:teuthology.orchestra.run.smithi018.stdout:Total download size: 25 k 2024-04-12T20:41:36.423 INFO:teuthology.orchestra.run.smithi018.stdout:Installed size: 110 k 2024-04-12T20:41:36.423 INFO:teuthology.orchestra.run.smithi018.stdout:Downloading Packages: 2024-04-12T20:41:36.468 INFO:teuthology.orchestra.run.smithi186.stdout:Dependencies resolved. 2024-04-12T20:41:36.468 INFO:teuthology.orchestra.run.smithi186.stdout:================================================================================ 2024-04-12T20:41:36.468 INFO:teuthology.orchestra.run.smithi186.stdout: Package Arch Version Repo Size 2024-04-12T20:41:36.468 INFO:teuthology.orchestra.run.smithi186.stdout:================================================================================ 2024-04-12T20:41:36.468 INFO:teuthology.orchestra.run.smithi186.stdout:Installing: 2024-04-12T20:41:36.468 INFO:teuthology.orchestra.run.smithi186.stdout: libcephfs-devel x86_64 2:17.2.7-827.gd3b5060a.el9 ceph 25 k 2024-04-12T20:41:36.469 INFO:teuthology.orchestra.run.smithi186.stdout: 2024-04-12T20:41:36.469 INFO:teuthology.orchestra.run.smithi186.stdout:Transaction Summary 2024-04-12T20:41:36.469 INFO:teuthology.orchestra.run.smithi186.stdout:================================================================================ 2024-04-12T20:41:36.469 INFO:teuthology.orchestra.run.smithi186.stdout:Install 1 Package 2024-04-12T20:41:36.469 INFO:teuthology.orchestra.run.smithi186.stdout: 2024-04-12T20:41:36.469 INFO:teuthology.orchestra.run.smithi186.stdout:Total download size: 25 k 2024-04-12T20:41:36.469 INFO:teuthology.orchestra.run.smithi186.stdout:Installed size: 110 k 2024-04-12T20:41:36.470 INFO:teuthology.orchestra.run.smithi186.stdout:Downloading Packages: 2024-04-12T20:41:36.740 INFO:teuthology.orchestra.run.smithi018.stdout:libcephfs-devel-17.2.7-827.gd3b5060a.el9.x86_64 79 kB/s | 25 kB 00:00 2024-04-12T20:41:36.741 INFO:teuthology.orchestra.run.smithi018.stdout:-------------------------------------------------------------------------------- 2024-04-12T20:41:36.742 INFO:teuthology.orchestra.run.smithi018.stdout:Total 78 kB/s | 25 kB 00:00 2024-04-12T20:41:36.742 INFO:teuthology.orchestra.run.smithi018.stdout:Running transaction check 2024-04-12T20:41:36.751 INFO:teuthology.orchestra.run.smithi018.stdout:Transaction check succeeded. 2024-04-12T20:41:36.752 INFO:teuthology.orchestra.run.smithi018.stdout:Running transaction test 2024-04-12T20:41:36.756 INFO:teuthology.orchestra.run.smithi018.stdout:Transaction test succeeded. 2024-04-12T20:41:36.757 INFO:teuthology.orchestra.run.smithi018.stdout:Running transaction 2024-04-12T20:41:36.769 INFO:teuthology.orchestra.run.smithi186.stdout:libcephfs-devel-17.2.7-827.gd3b5060a.el9.x86_64 83 kB/s | 25 kB 00:00 2024-04-12T20:41:36.770 INFO:teuthology.orchestra.run.smithi186.stdout:-------------------------------------------------------------------------------- 2024-04-12T20:41:36.771 INFO:teuthology.orchestra.run.smithi186.stdout:Total 83 kB/s | 25 kB 00:00 2024-04-12T20:41:36.771 INFO:teuthology.orchestra.run.smithi186.stdout:Running transaction check 2024-04-12T20:41:36.783 INFO:teuthology.orchestra.run.smithi186.stdout:Transaction check succeeded. 2024-04-12T20:41:36.783 INFO:teuthology.orchestra.run.smithi186.stdout:Running transaction test 2024-04-12T20:41:36.789 INFO:teuthology.orchestra.run.smithi186.stdout:Transaction test succeeded. 2024-04-12T20:41:36.791 INFO:teuthology.orchestra.run.smithi186.stdout:Running transaction 2024-04-12T20:41:36.892 INFO:teuthology.orchestra.run.smithi186.stdout: Preparing : 1/1 2024-04-12T20:41:36.918 INFO:teuthology.orchestra.run.smithi018.stdout: Preparing : 1/1 2024-04-12T20:41:37.086 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : libcephfs-devel-2:17.2.7-827.gd3b5060a.el9.x86_64 1/1 2024-04-12T20:41:37.117 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : libcephfs-devel-2:17.2.7-827.gd3b5060a.el9.x86_64 1/1 2024-04-12T20:41:37.649 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: libcephfs-devel-2:17.2.7-827.gd3b5060a.el9.x86_64 1/1 2024-04-12T20:41:37.722 INFO:teuthology.orchestra.run.smithi018.stdout: Running scriptlet: libcephfs-devel-2:17.2.7-827.gd3b5060a.el9.x86_64 1/1 2024-04-12T20:41:37.979 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : libcephfs-devel-2:17.2.7-827.gd3b5060a.el9.x86_64 1/1 2024-04-12T20:41:37.979 INFO:teuthology.orchestra.run.smithi186.stdout: 2024-04-12T20:41:37.979 INFO:teuthology.orchestra.run.smithi186.stdout:Installed: 2024-04-12T20:41:37.979 INFO:teuthology.orchestra.run.smithi186.stdout: libcephfs-devel-2:17.2.7-827.gd3b5060a.el9.x86_64 2024-04-12T20:41:37.980 INFO:teuthology.orchestra.run.smithi186.stdout: 2024-04-12T20:41:37.980 INFO:teuthology.orchestra.run.smithi186.stdout:Complete! 2024-04-12T20:41:38.045 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : libcephfs-devel-2:17.2.7-827.gd3b5060a.el9.x86_64 1/1 2024-04-12T20:41:38.045 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-12T20:41:38.045 INFO:teuthology.orchestra.run.smithi018.stdout:Installed: 2024-04-12T20:41:38.045 INFO:teuthology.orchestra.run.smithi018.stdout: libcephfs-devel-2:17.2.7-827.gd3b5060a.el9.x86_64 2024-04-12T20:41:38.045 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-12T20:41:38.045 INFO:teuthology.orchestra.run.smithi018.stdout:Complete! 2024-04-12T20:41:38.256 DEBUG:teuthology.orchestra.run.smithi186:> sudo yum -y install librados2 2024-04-12T20:41:38.270 DEBUG:teuthology.orchestra.run.smithi018:> sudo yum -y install librados2 2024-04-12T20:41:39.144 INFO:teuthology.orchestra.run.smithi186.stdout:Last metadata expiration check: 0:02:02 ago on Fri 12 Apr 2024 08:39:37 PM UTC. 2024-04-12T20:41:39.191 INFO:teuthology.orchestra.run.smithi018.stdout:Last metadata expiration check: 0:02:04 ago on Fri 12 Apr 2024 08:39:35 PM UTC. 2024-04-12T20:41:39.440 INFO:teuthology.orchestra.run.smithi186.stderr:Modular dependency problems: 2024-04-12T20:41:39.440 INFO:teuthology.orchestra.run.smithi186.stderr: 2024-04-12T20:41:39.440 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 1: conflicting requests 2024-04-12T20:41:39.440 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module container-tools:rhel8:820240321091303:20125149.x86_64 from CentOS-AppStream 2024-04-12T20:41:39.440 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 2: conflicting requests 2024-04-12T20:41:39.440 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module eclipse:rhel8:820201023100746:b230ed4e.x86_64 from CentOS-AppStream 2024-04-12T20:41:39.440 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 3: conflicting requests 2024-04-12T20:41:39.440 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module freeradius:3.0:8080020230117180605:89170a74.x86_64 from CentOS-AppStream 2024-04-12T20:41:39.440 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 4: conflicting requests 2024-04-12T20:41:39.440 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module gimp:2.8:8000020190628145146:4148dfdf.x86_64 from CentOS-AppStream 2024-04-12T20:41:39.440 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 5: conflicting requests 2024-04-12T20:41:39.440 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module go-toolset:rhel8:820240401160413:b754926a.x86_64 from CentOS-AppStream 2024-04-12T20:41:39.441 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 6: conflicting requests 2024-04-12T20:41:39.441 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module httpd:2.4:820240216084734:9edba152.x86_64 from CentOS-AppStream 2024-04-12T20:41:39.441 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 7: conflicting requests 2024-04-12T20:41:39.441 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module idm:client:820240307184541:49cc9d1b.x86_64 from CentOS-AppStream 2024-04-12T20:41:39.441 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 8: conflicting requests 2024-04-12T20:41:39.441 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module inkscape:0.92.3:8080020221205124429:3e031279.x86_64 from CentOS-AppStream 2024-04-12T20:41:39.441 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 9: conflicting requests 2024-04-12T20:41:39.441 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module jmc:rhel8:8050020211110222101:6392b1f8.x86_64 from CentOS-AppStream 2024-04-12T20:41:39.441 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 10: conflicting requests 2024-04-12T20:41:39.441 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module llvm-toolset:rhel8:820240205133135:9edba152.x86_64 from CentOS-AppStream 2024-04-12T20:41:39.441 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 11: conflicting requests 2024-04-12T20:41:39.441 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module log4j:2:8080020221205124743:9d367344.x86_64 from CentOS-AppStream 2024-04-12T20:41:39.441 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 12: conflicting requests 2024-04-12T20:41:39.441 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module mailman:2.1:820230727085757:77fc8825.x86_64 from CentOS-AppStream 2024-04-12T20:41:39.442 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 13: conflicting requests 2024-04-12T20:41:39.442 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module mariadb:10.3:8030020210419150013:30b713e6.x86_64 from CentOS-AppStream 2024-04-12T20:41:39.442 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 14: conflicting requests 2024-04-12T20:41:39.442 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module maven:3.5:8060020211203142015:c0229ad2.x86_64 from CentOS-AppStream 2024-04-12T20:41:39.442 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 15: conflicting requests 2024-04-12T20:41:39.442 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module mercurial:4.8:8000020190628020724:4148dfdf.x86_64 from CentOS-AppStream 2024-04-12T20:41:39.442 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 16: conflicting requests 2024-04-12T20:41:39.442 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module mod_auth_openidc:2.3:820230721123832:9edba152.x86_64 from CentOS-AppStream 2024-04-12T20:41:39.442 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 17: conflicting requests 2024-04-12T20:41:39.442 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module mysql:8.0:8040020210901180257:9f9e2e7e.x86_64 from CentOS-AppStream 2024-04-12T20:41:39.442 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 18: conflicting requests 2024-04-12T20:41:39.442 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module nginx:1.14:8000020211221191913:55190bc5.x86_64 from CentOS-AppStream 2024-04-12T20:41:39.442 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 19: conflicting requests 2024-04-12T20:41:39.442 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module nodejs:10:8040020210120182536:9f9e2e7e.x86_64 from CentOS-AppStream 2024-04-12T20:41:39.443 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 20: conflicting requests 2024-04-12T20:41:39.443 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl:5.26:8000020190628020724:55190bc5.x86_64 from CentOS-AppStream 2024-04-12T20:41:39.443 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 21: conflicting requests 2024-04-12T20:41:39.443 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module php:7.2:8020020200507003613:2c7ca891.x86_64 from CentOS-AppStream 2024-04-12T20:41:39.443 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 22: conflicting requests 2024-04-12T20:41:39.443 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module postgresql:10:820230223114800:9edba152.x86_64 from CentOS-AppStream 2024-04-12T20:41:39.443 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 23: conflicting requests 2024-04-12T20:41:39.443 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module python27:2.7:820240208011952:182f7c73.x86_64 from CentOS-AppStream 2024-04-12T20:41:39.443 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 24: conflicting requests 2024-04-12T20:41:39.443 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module python36:3.6:8050020210825152031:982725ab.x86_64 from CentOS-AppStream 2024-04-12T20:41:39.443 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 25: conflicting requests 2024-04-12T20:41:39.443 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module python38:3.8:820230810143931:64c1cd5a.x86_64 from CentOS-AppStream 2024-04-12T20:41:39.443 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 26: conflicting requests 2024-04-12T20:41:39.443 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module python39:3.9:820240214182535:17377f89.x86_64 from CentOS-AppStream 2024-04-12T20:41:39.444 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 27: conflicting requests 2024-04-12T20:41:39.444 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module redis:5:8040020211019153849:9f9e2e7e.x86_64 from CentOS-AppStream 2024-04-12T20:41:39.444 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 28: conflicting requests 2024-04-12T20:41:39.444 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module rhn-tools:1.0:8010020191114034948:f69d1239.x86_64 from CentOS-AppStream 2024-04-12T20:41:39.444 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 29: conflicting requests 2024-04-12T20:41:39.444 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module ruby:2.5:820230627084142:9edba152.x86_64 from CentOS-AppStream 2024-04-12T20:41:39.444 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 30: conflicting requests 2024-04-12T20:41:39.444 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module rust-toolset:rhel8:820240119204620:b09eea91.x86_64 from CentOS-AppStream 2024-04-12T20:41:39.444 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 31: conflicting requests 2024-04-12T20:41:39.444 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module satellite-5-client:1.0:8010020191114035551:cdc1202b.x86_64 from CentOS-AppStream 2024-04-12T20:41:39.444 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 32: conflicting requests 2024-04-12T20:41:39.444 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module squid:4:820240319173245:9edba152.x86_64 from CentOS-AppStream 2024-04-12T20:41:39.444 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 33: conflicting requests 2024-04-12T20:41:39.444 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module subversion:1.10:8070020220711155714:78111232.x86_64 from CentOS-AppStream 2024-04-12T20:41:39.445 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 34: conflicting requests 2024-04-12T20:41:39.445 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module swig:3.0:8030020200706142531:30b713e6.x86_64 from CentOS-AppStream 2024-04-12T20:41:39.445 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 35: conflicting requests 2024-04-12T20:41:39.445 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module varnish:6:8050020211109225449:b4937e53.x86_64 from CentOS-AppStream 2024-04-12T20:41:39.445 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 36: conflicting requests 2024-04-12T20:41:39.445 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module virt:rhel:820240314161907:9edba152.x86_64 from CentOS-AppStream 2024-04-12T20:41:39.445 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 37: conflicting requests 2024-04-12T20:41:39.445 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:8030020200716155257:7cc0a66d.x86_64 from CentOS-AppStream 2024-04-12T20:41:39.445 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:8030020200716155257:b967a9a2.x86_64 from CentOS-AppStream 2024-04-12T20:41:39.445 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:820230811133638:36f9fae6.x86_64 from CentOS-AppStream 2024-04-12T20:41:39.445 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 38: cannot install the best candidate for the job 2024-04-12T20:41:39.445 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:8010020191114031501:a5949e2e.x86_64 from CentOS-AppStream 2024-04-12T20:41:39.445 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:8010020191114031501:be2e4737.x86_64 from CentOS-AppStream 2024-04-12T20:41:39.445 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:820230223114759:2ab761e1.x86_64 from CentOS-AppStream 2024-04-12T20:41:39.446 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:820230223114759:86e45846.x86_64 from CentOS-AppStream 2024-04-12T20:41:39.446 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 39: conflicting requests 2024-04-12T20:41:39.446 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:8030020200715230104:1e4bbb35.x86_64 from CentOS-AppStream 2024-04-12T20:41:39.446 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:8030020200715230104:ea09926d.x86_64 from CentOS-AppStream 2024-04-12T20:41:39.446 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:082fdf2f.x86_64 from CentOS-AppStream 2024-04-12T20:41:39.446 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-04-12T20:41:39.446 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-04-12T20:41:39.446 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:fbe42456.x86_64 from CentOS-AppStream 2024-04-12T20:41:39.446 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 40: cannot install the best candidate for the job 2024-04-12T20:41:39.446 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:8010020191114031513:16b3ab4d.x86_64 from CentOS-AppStream 2024-04-12T20:41:39.446 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:8010020191114031513:e1f37755.x86_64 from CentOS-AppStream 2024-04-12T20:41:39.446 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-04-12T20:41:39.446 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-04-12T20:41:39.447 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 41: cannot install the best candidate for the job 2024-04-12T20:41:39.447 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:8030020200716150652:1e4bbb35.x86_64 from CentOS-AppStream 2024-04-12T20:41:39.447 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:8030020200716150652:ea09926d.x86_64 from CentOS-AppStream 2024-04-12T20:41:39.447 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-04-12T20:41:39.447 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-04-12T20:41:39.447 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 42: cannot install the best candidate for the job 2024-04-12T20:41:39.447 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:8030020200716174729:3a70019f.x86_64 from CentOS-AppStream 2024-04-12T20:41:39.447 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:8030020200716174729:cccafca5.x86_64 from CentOS-AppStream 2024-04-12T20:41:39.447 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:820230223114759:6fdf7e5d.x86_64 from CentOS-AppStream 2024-04-12T20:41:39.447 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:820230223114759:d4326aba.x86_64 from CentOS-AppStream 2024-04-12T20:41:39.447 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 43: cannot install the best candidate for the job 2024-04-12T20:41:39.447 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:8030020200716213634:56fce90f.x86_64 from CentOS-AppStream 2024-04-12T20:41:39.447 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:8030020200716213634:98a3c9d0.x86_64 from CentOS-AppStream 2024-04-12T20:41:39.448 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:820230223114759:06b7596e.x86_64 from CentOS-AppStream 2024-04-12T20:41:39.448 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:820230223114759:d2fc5d76.x86_64 from CentOS-AppStream 2024-04-12T20:41:39.448 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 44: cannot install the best candidate for the job 2024-04-12T20:41:39.448 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:8030020200716171139:3a70019f.x86_64 from CentOS-AppStream 2024-04-12T20:41:39.448 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:8030020200716171139:cccafca5.x86_64 from CentOS-AppStream 2024-04-12T20:41:39.448 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:820230223114759:6fdf7e5d.x86_64 from CentOS-AppStream 2024-04-12T20:41:39.448 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:820230223114759:d4326aba.x86_64 from CentOS-AppStream 2024-04-12T20:41:39.448 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 45: cannot install the best candidate for the job 2024-04-12T20:41:39.448 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:8010020191120175858:95fce66d.x86_64 from CentOS-AppStream 2024-04-12T20:41:39.448 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:8030020200723233049:09acf126.x86_64 from CentOS-AppStream 2024-04-12T20:41:39.448 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:820230223114759:436af1ae.x86_64 from CentOS-AppStream 2024-04-12T20:41:39.448 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:820230223114759:b12195ea.x86_64 from CentOS-AppStream 2024-04-12T20:41:39.449 INFO:teuthology.orchestra.run.smithi186.stdout:Package librados2-2:17.2.7-827.gd3b5060a.el9.x86_64 is already installed. 2024-04-12T20:41:39.503 INFO:teuthology.orchestra.run.smithi018.stderr:Modular dependency problems: 2024-04-12T20:41:39.503 INFO:teuthology.orchestra.run.smithi018.stderr: 2024-04-12T20:41:39.503 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 1: conflicting requests 2024-04-12T20:41:39.503 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module container-tools:rhel8:820240321091303:20125149.x86_64 from CentOS-AppStream 2024-04-12T20:41:39.503 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 2: conflicting requests 2024-04-12T20:41:39.503 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module eclipse:rhel8:820201023100746:b230ed4e.x86_64 from CentOS-AppStream 2024-04-12T20:41:39.503 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 3: conflicting requests 2024-04-12T20:41:39.503 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module freeradius:3.0:8080020230117180605:89170a74.x86_64 from CentOS-AppStream 2024-04-12T20:41:39.503 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 4: conflicting requests 2024-04-12T20:41:39.503 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module gimp:2.8:8000020190628145146:4148dfdf.x86_64 from CentOS-AppStream 2024-04-12T20:41:39.503 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 5: conflicting requests 2024-04-12T20:41:39.504 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module go-toolset:rhel8:820240401160413:b754926a.x86_64 from CentOS-AppStream 2024-04-12T20:41:39.504 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 6: conflicting requests 2024-04-12T20:41:39.504 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module httpd:2.4:820240216084734:9edba152.x86_64 from CentOS-AppStream 2024-04-12T20:41:39.504 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 7: conflicting requests 2024-04-12T20:41:39.504 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module idm:client:820240307184541:49cc9d1b.x86_64 from CentOS-AppStream 2024-04-12T20:41:39.504 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 8: conflicting requests 2024-04-12T20:41:39.504 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module inkscape:0.92.3:8080020221205124429:3e031279.x86_64 from CentOS-AppStream 2024-04-12T20:41:39.504 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 9: conflicting requests 2024-04-12T20:41:39.504 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module jmc:rhel8:8050020211110222101:6392b1f8.x86_64 from CentOS-AppStream 2024-04-12T20:41:39.504 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 10: conflicting requests 2024-04-12T20:41:39.504 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module llvm-toolset:rhel8:820240205133135:9edba152.x86_64 from CentOS-AppStream 2024-04-12T20:41:39.504 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 11: conflicting requests 2024-04-12T20:41:39.504 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module log4j:2:8080020221205124743:9d367344.x86_64 from CentOS-AppStream 2024-04-12T20:41:39.505 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 12: conflicting requests 2024-04-12T20:41:39.505 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module mailman:2.1:820230727085757:77fc8825.x86_64 from CentOS-AppStream 2024-04-12T20:41:39.505 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 13: conflicting requests 2024-04-12T20:41:39.505 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module mariadb:10.3:8030020210419150013:30b713e6.x86_64 from CentOS-AppStream 2024-04-12T20:41:39.505 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 14: conflicting requests 2024-04-12T20:41:39.505 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module maven:3.5:8060020211203142015:c0229ad2.x86_64 from CentOS-AppStream 2024-04-12T20:41:39.505 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 15: conflicting requests 2024-04-12T20:41:39.505 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module mercurial:4.8:8000020190628020724:4148dfdf.x86_64 from CentOS-AppStream 2024-04-12T20:41:39.505 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 16: conflicting requests 2024-04-12T20:41:39.505 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module mod_auth_openidc:2.3:820230721123832:9edba152.x86_64 from CentOS-AppStream 2024-04-12T20:41:39.505 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 17: conflicting requests 2024-04-12T20:41:39.505 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module mysql:8.0:8040020210901180257:9f9e2e7e.x86_64 from CentOS-AppStream 2024-04-12T20:41:39.505 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 18: conflicting requests 2024-04-12T20:41:39.506 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module nginx:1.14:8000020211221191913:55190bc5.x86_64 from CentOS-AppStream 2024-04-12T20:41:39.506 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 19: conflicting requests 2024-04-12T20:41:39.506 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module nodejs:10:8040020210120182536:9f9e2e7e.x86_64 from CentOS-AppStream 2024-04-12T20:41:39.506 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 20: conflicting requests 2024-04-12T20:41:39.506 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl:5.26:8000020190628020724:55190bc5.x86_64 from CentOS-AppStream 2024-04-12T20:41:39.506 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 21: conflicting requests 2024-04-12T20:41:39.506 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module php:7.2:8020020200507003613:2c7ca891.x86_64 from CentOS-AppStream 2024-04-12T20:41:39.506 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 22: conflicting requests 2024-04-12T20:41:39.506 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module postgresql:10:820230223114800:9edba152.x86_64 from CentOS-AppStream 2024-04-12T20:41:39.506 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 23: conflicting requests 2024-04-12T20:41:39.506 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module python27:2.7:820240208011952:182f7c73.x86_64 from CentOS-AppStream 2024-04-12T20:41:39.507 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 24: conflicting requests 2024-04-12T20:41:39.507 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module python36:3.6:8050020210825152031:982725ab.x86_64 from CentOS-AppStream 2024-04-12T20:41:39.507 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 25: conflicting requests 2024-04-12T20:41:39.507 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module python38:3.8:820230810143931:64c1cd5a.x86_64 from CentOS-AppStream 2024-04-12T20:41:39.507 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 26: conflicting requests 2024-04-12T20:41:39.507 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module python39:3.9:820240214182535:17377f89.x86_64 from CentOS-AppStream 2024-04-12T20:41:39.507 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 27: conflicting requests 2024-04-12T20:41:39.507 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module redis:5:8040020211019153849:9f9e2e7e.x86_64 from CentOS-AppStream 2024-04-12T20:41:39.507 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 28: conflicting requests 2024-04-12T20:41:39.507 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module rhn-tools:1.0:8010020191114034948:f69d1239.x86_64 from CentOS-AppStream 2024-04-12T20:41:39.507 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 29: conflicting requests 2024-04-12T20:41:39.507 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module ruby:2.5:820230627084142:9edba152.x86_64 from CentOS-AppStream 2024-04-12T20:41:39.507 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 30: conflicting requests 2024-04-12T20:41:39.508 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module rust-toolset:rhel8:820240119204620:b09eea91.x86_64 from CentOS-AppStream 2024-04-12T20:41:39.508 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 31: conflicting requests 2024-04-12T20:41:39.508 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module satellite-5-client:1.0:8010020191114035551:cdc1202b.x86_64 from CentOS-AppStream 2024-04-12T20:41:39.508 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 32: conflicting requests 2024-04-12T20:41:39.508 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module squid:4:820240319173245:9edba152.x86_64 from CentOS-AppStream 2024-04-12T20:41:39.508 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 33: conflicting requests 2024-04-12T20:41:39.508 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module subversion:1.10:8070020220711155714:78111232.x86_64 from CentOS-AppStream 2024-04-12T20:41:39.508 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 34: conflicting requests 2024-04-12T20:41:39.508 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module swig:3.0:8030020200706142531:30b713e6.x86_64 from CentOS-AppStream 2024-04-12T20:41:39.508 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 35: conflicting requests 2024-04-12T20:41:39.508 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module varnish:6:8050020211109225449:b4937e53.x86_64 from CentOS-AppStream 2024-04-12T20:41:39.508 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 36: conflicting requests 2024-04-12T20:41:39.508 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module virt:rhel:820240314161907:9edba152.x86_64 from CentOS-AppStream 2024-04-12T20:41:39.508 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 37: conflicting requests 2024-04-12T20:41:39.509 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:8030020200716155257:7cc0a66d.x86_64 from CentOS-AppStream 2024-04-12T20:41:39.509 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:8030020200716155257:b967a9a2.x86_64 from CentOS-AppStream 2024-04-12T20:41:39.509 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:820230811133638:36f9fae6.x86_64 from CentOS-AppStream 2024-04-12T20:41:39.509 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 38: cannot install the best candidate for the job 2024-04-12T20:41:39.509 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:8010020191114031501:a5949e2e.x86_64 from CentOS-AppStream 2024-04-12T20:41:39.509 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:8010020191114031501:be2e4737.x86_64 from CentOS-AppStream 2024-04-12T20:41:39.509 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:820230223114759:2ab761e1.x86_64 from CentOS-AppStream 2024-04-12T20:41:39.509 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:820230223114759:86e45846.x86_64 from CentOS-AppStream 2024-04-12T20:41:39.509 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 39: conflicting requests 2024-04-12T20:41:39.509 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:8030020200715230104:1e4bbb35.x86_64 from CentOS-AppStream 2024-04-12T20:41:39.509 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:8030020200715230104:ea09926d.x86_64 from CentOS-AppStream 2024-04-12T20:41:39.509 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:082fdf2f.x86_64 from CentOS-AppStream 2024-04-12T20:41:39.509 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-04-12T20:41:39.509 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-04-12T20:41:39.510 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:fbe42456.x86_64 from CentOS-AppStream 2024-04-12T20:41:39.510 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 40: cannot install the best candidate for the job 2024-04-12T20:41:39.510 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:8010020191114031513:16b3ab4d.x86_64 from CentOS-AppStream 2024-04-12T20:41:39.510 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:8010020191114031513:e1f37755.x86_64 from CentOS-AppStream 2024-04-12T20:41:39.510 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-04-12T20:41:39.510 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-04-12T20:41:39.510 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 41: cannot install the best candidate for the job 2024-04-12T20:41:39.510 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:8030020200716150652:1e4bbb35.x86_64 from CentOS-AppStream 2024-04-12T20:41:39.510 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:8030020200716150652:ea09926d.x86_64 from CentOS-AppStream 2024-04-12T20:41:39.510 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-04-12T20:41:39.510 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-04-12T20:41:39.510 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 42: cannot install the best candidate for the job 2024-04-12T20:41:39.510 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:8030020200716174729:3a70019f.x86_64 from CentOS-AppStream 2024-04-12T20:41:39.510 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:8030020200716174729:cccafca5.x86_64 from CentOS-AppStream 2024-04-12T20:41:39.511 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:820230223114759:6fdf7e5d.x86_64 from CentOS-AppStream 2024-04-12T20:41:39.511 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:820230223114759:d4326aba.x86_64 from CentOS-AppStream 2024-04-12T20:41:39.511 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 43: cannot install the best candidate for the job 2024-04-12T20:41:39.511 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:8030020200716213634:56fce90f.x86_64 from CentOS-AppStream 2024-04-12T20:41:39.511 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:8030020200716213634:98a3c9d0.x86_64 from CentOS-AppStream 2024-04-12T20:41:39.511 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:820230223114759:06b7596e.x86_64 from CentOS-AppStream 2024-04-12T20:41:39.511 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:820230223114759:d2fc5d76.x86_64 from CentOS-AppStream 2024-04-12T20:41:39.511 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 44: cannot install the best candidate for the job 2024-04-12T20:41:39.511 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:8030020200716171139:3a70019f.x86_64 from CentOS-AppStream 2024-04-12T20:41:39.511 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:8030020200716171139:cccafca5.x86_64 from CentOS-AppStream 2024-04-12T20:41:39.511 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:820230223114759:6fdf7e5d.x86_64 from CentOS-AppStream 2024-04-12T20:41:39.511 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:820230223114759:d4326aba.x86_64 from CentOS-AppStream 2024-04-12T20:41:39.511 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 45: cannot install the best candidate for the job 2024-04-12T20:41:39.512 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:8010020191120175858:95fce66d.x86_64 from CentOS-AppStream 2024-04-12T20:41:39.512 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:8030020200723233049:09acf126.x86_64 from CentOS-AppStream 2024-04-12T20:41:39.512 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:820230223114759:436af1ae.x86_64 from CentOS-AppStream 2024-04-12T20:41:39.512 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:820230223114759:b12195ea.x86_64 from CentOS-AppStream 2024-04-12T20:41:39.512 INFO:teuthology.orchestra.run.smithi186.stdout:Dependencies resolved. 2024-04-12T20:41:39.513 INFO:teuthology.orchestra.run.smithi018.stdout:Package librados2-2:17.2.7-827.gd3b5060a.el9.x86_64 is already installed. 2024-04-12T20:41:39.513 INFO:teuthology.orchestra.run.smithi186.stdout:Nothing to do. 2024-04-12T20:41:39.513 INFO:teuthology.orchestra.run.smithi186.stdout:Complete! 2024-04-12T20:41:39.580 INFO:teuthology.orchestra.run.smithi018.stdout:Dependencies resolved. 2024-04-12T20:41:39.581 INFO:teuthology.orchestra.run.smithi018.stdout:Nothing to do. 2024-04-12T20:41:39.581 INFO:teuthology.orchestra.run.smithi018.stdout:Complete! 2024-04-12T20:41:39.590 DEBUG:teuthology.orchestra.run.smithi186:> sudo yum -y install librbd1 2024-04-12T20:41:39.708 DEBUG:teuthology.orchestra.run.smithi018:> sudo yum -y install librbd1 2024-04-12T20:41:40.445 INFO:teuthology.orchestra.run.smithi186.stdout:Last metadata expiration check: 0:02:03 ago on Fri 12 Apr 2024 08:39:37 PM UTC. 2024-04-12T20:41:40.612 INFO:teuthology.orchestra.run.smithi018.stdout:Last metadata expiration check: 0:02:05 ago on Fri 12 Apr 2024 08:39:35 PM UTC. 2024-04-12T20:41:40.741 INFO:teuthology.orchestra.run.smithi186.stderr:Modular dependency problems: 2024-04-12T20:41:40.741 INFO:teuthology.orchestra.run.smithi186.stderr: 2024-04-12T20:41:40.742 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 1: conflicting requests 2024-04-12T20:41:40.742 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module container-tools:rhel8:820240321091303:20125149.x86_64 from CentOS-AppStream 2024-04-12T20:41:40.742 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 2: conflicting requests 2024-04-12T20:41:40.742 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module eclipse:rhel8:820201023100746:b230ed4e.x86_64 from CentOS-AppStream 2024-04-12T20:41:40.742 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 3: conflicting requests 2024-04-12T20:41:40.742 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module freeradius:3.0:8080020230117180605:89170a74.x86_64 from CentOS-AppStream 2024-04-12T20:41:40.742 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 4: conflicting requests 2024-04-12T20:41:40.742 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module gimp:2.8:8000020190628145146:4148dfdf.x86_64 from CentOS-AppStream 2024-04-12T20:41:40.742 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 5: conflicting requests 2024-04-12T20:41:40.742 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module go-toolset:rhel8:820240401160413:b754926a.x86_64 from CentOS-AppStream 2024-04-12T20:41:40.743 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 6: conflicting requests 2024-04-12T20:41:40.743 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module httpd:2.4:820240216084734:9edba152.x86_64 from CentOS-AppStream 2024-04-12T20:41:40.743 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 7: conflicting requests 2024-04-12T20:41:40.743 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module idm:client:820240307184541:49cc9d1b.x86_64 from CentOS-AppStream 2024-04-12T20:41:40.743 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 8: conflicting requests 2024-04-12T20:41:40.743 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module inkscape:0.92.3:8080020221205124429:3e031279.x86_64 from CentOS-AppStream 2024-04-12T20:41:40.743 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 9: conflicting requests 2024-04-12T20:41:40.743 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module jmc:rhel8:8050020211110222101:6392b1f8.x86_64 from CentOS-AppStream 2024-04-12T20:41:40.743 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 10: conflicting requests 2024-04-12T20:41:40.743 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module llvm-toolset:rhel8:820240205133135:9edba152.x86_64 from CentOS-AppStream 2024-04-12T20:41:40.743 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 11: conflicting requests 2024-04-12T20:41:40.744 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module log4j:2:8080020221205124743:9d367344.x86_64 from CentOS-AppStream 2024-04-12T20:41:40.744 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 12: conflicting requests 2024-04-12T20:41:40.744 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module mailman:2.1:820230727085757:77fc8825.x86_64 from CentOS-AppStream 2024-04-12T20:41:40.744 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 13: conflicting requests 2024-04-12T20:41:40.744 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module mariadb:10.3:8030020210419150013:30b713e6.x86_64 from CentOS-AppStream 2024-04-12T20:41:40.744 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 14: conflicting requests 2024-04-12T20:41:40.744 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module maven:3.5:8060020211203142015:c0229ad2.x86_64 from CentOS-AppStream 2024-04-12T20:41:40.744 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 15: conflicting requests 2024-04-12T20:41:40.744 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module mercurial:4.8:8000020190628020724:4148dfdf.x86_64 from CentOS-AppStream 2024-04-12T20:41:40.745 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 16: conflicting requests 2024-04-12T20:41:40.745 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module mod_auth_openidc:2.3:820230721123832:9edba152.x86_64 from CentOS-AppStream 2024-04-12T20:41:40.745 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 17: conflicting requests 2024-04-12T20:41:40.745 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module mysql:8.0:8040020210901180257:9f9e2e7e.x86_64 from CentOS-AppStream 2024-04-12T20:41:40.745 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 18: conflicting requests 2024-04-12T20:41:40.745 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module nginx:1.14:8000020211221191913:55190bc5.x86_64 from CentOS-AppStream 2024-04-12T20:41:40.745 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 19: conflicting requests 2024-04-12T20:41:40.745 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module nodejs:10:8040020210120182536:9f9e2e7e.x86_64 from CentOS-AppStream 2024-04-12T20:41:40.745 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 20: conflicting requests 2024-04-12T20:41:40.745 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl:5.26:8000020190628020724:55190bc5.x86_64 from CentOS-AppStream 2024-04-12T20:41:40.745 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 21: conflicting requests 2024-04-12T20:41:40.745 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module php:7.2:8020020200507003613:2c7ca891.x86_64 from CentOS-AppStream 2024-04-12T20:41:40.745 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 22: conflicting requests 2024-04-12T20:41:40.746 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module postgresql:10:820230223114800:9edba152.x86_64 from CentOS-AppStream 2024-04-12T20:41:40.746 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 23: conflicting requests 2024-04-12T20:41:40.746 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module python27:2.7:820240208011952:182f7c73.x86_64 from CentOS-AppStream 2024-04-12T20:41:40.746 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 24: conflicting requests 2024-04-12T20:41:40.746 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module python36:3.6:8050020210825152031:982725ab.x86_64 from CentOS-AppStream 2024-04-12T20:41:40.746 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 25: conflicting requests 2024-04-12T20:41:40.746 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module python38:3.8:820230810143931:64c1cd5a.x86_64 from CentOS-AppStream 2024-04-12T20:41:40.746 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 26: conflicting requests 2024-04-12T20:41:40.746 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module python39:3.9:820240214182535:17377f89.x86_64 from CentOS-AppStream 2024-04-12T20:41:40.746 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 27: conflicting requests 2024-04-12T20:41:40.746 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module redis:5:8040020211019153849:9f9e2e7e.x86_64 from CentOS-AppStream 2024-04-12T20:41:40.746 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 28: conflicting requests 2024-04-12T20:41:40.746 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module rhn-tools:1.0:8010020191114034948:f69d1239.x86_64 from CentOS-AppStream 2024-04-12T20:41:40.746 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 29: conflicting requests 2024-04-12T20:41:40.747 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module ruby:2.5:820230627084142:9edba152.x86_64 from CentOS-AppStream 2024-04-12T20:41:40.747 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 30: conflicting requests 2024-04-12T20:41:40.747 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module rust-toolset:rhel8:820240119204620:b09eea91.x86_64 from CentOS-AppStream 2024-04-12T20:41:40.747 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 31: conflicting requests 2024-04-12T20:41:40.747 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module satellite-5-client:1.0:8010020191114035551:cdc1202b.x86_64 from CentOS-AppStream 2024-04-12T20:41:40.747 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 32: conflicting requests 2024-04-12T20:41:40.747 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module squid:4:820240319173245:9edba152.x86_64 from CentOS-AppStream 2024-04-12T20:41:40.747 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 33: conflicting requests 2024-04-12T20:41:40.747 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module subversion:1.10:8070020220711155714:78111232.x86_64 from CentOS-AppStream 2024-04-12T20:41:40.747 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 34: conflicting requests 2024-04-12T20:41:40.747 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module swig:3.0:8030020200706142531:30b713e6.x86_64 from CentOS-AppStream 2024-04-12T20:41:40.747 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 35: conflicting requests 2024-04-12T20:41:40.747 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module varnish:6:8050020211109225449:b4937e53.x86_64 from CentOS-AppStream 2024-04-12T20:41:40.747 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 36: conflicting requests 2024-04-12T20:41:40.748 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module virt:rhel:820240314161907:9edba152.x86_64 from CentOS-AppStream 2024-04-12T20:41:40.748 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 37: conflicting requests 2024-04-12T20:41:40.748 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:8030020200716155257:7cc0a66d.x86_64 from CentOS-AppStream 2024-04-12T20:41:40.748 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:8030020200716155257:b967a9a2.x86_64 from CentOS-AppStream 2024-04-12T20:41:40.748 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:820230811133638:36f9fae6.x86_64 from CentOS-AppStream 2024-04-12T20:41:40.748 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 38: cannot install the best candidate for the job 2024-04-12T20:41:40.748 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:8010020191114031501:a5949e2e.x86_64 from CentOS-AppStream 2024-04-12T20:41:40.748 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:8010020191114031501:be2e4737.x86_64 from CentOS-AppStream 2024-04-12T20:41:40.748 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:820230223114759:2ab761e1.x86_64 from CentOS-AppStream 2024-04-12T20:41:40.748 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:820230223114759:86e45846.x86_64 from CentOS-AppStream 2024-04-12T20:41:40.748 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 39: conflicting requests 2024-04-12T20:41:40.748 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:8030020200715230104:1e4bbb35.x86_64 from CentOS-AppStream 2024-04-12T20:41:40.748 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:8030020200715230104:ea09926d.x86_64 from CentOS-AppStream 2024-04-12T20:41:40.748 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:082fdf2f.x86_64 from CentOS-AppStream 2024-04-12T20:41:40.748 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-04-12T20:41:40.749 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-04-12T20:41:40.749 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:fbe42456.x86_64 from CentOS-AppStream 2024-04-12T20:41:40.749 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 40: cannot install the best candidate for the job 2024-04-12T20:41:40.749 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:8010020191114031513:16b3ab4d.x86_64 from CentOS-AppStream 2024-04-12T20:41:40.749 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:8010020191114031513:e1f37755.x86_64 from CentOS-AppStream 2024-04-12T20:41:40.749 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-04-12T20:41:40.749 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-04-12T20:41:40.749 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 41: cannot install the best candidate for the job 2024-04-12T20:41:40.749 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:8030020200716150652:1e4bbb35.x86_64 from CentOS-AppStream 2024-04-12T20:41:40.749 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:8030020200716150652:ea09926d.x86_64 from CentOS-AppStream 2024-04-12T20:41:40.749 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-04-12T20:41:40.749 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-04-12T20:41:40.749 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 42: cannot install the best candidate for the job 2024-04-12T20:41:40.750 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:8030020200716174729:3a70019f.x86_64 from CentOS-AppStream 2024-04-12T20:41:40.750 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:8030020200716174729:cccafca5.x86_64 from CentOS-AppStream 2024-04-12T20:41:40.750 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:820230223114759:6fdf7e5d.x86_64 from CentOS-AppStream 2024-04-12T20:41:40.750 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:820230223114759:d4326aba.x86_64 from CentOS-AppStream 2024-04-12T20:41:40.750 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 43: cannot install the best candidate for the job 2024-04-12T20:41:40.750 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:8030020200716213634:56fce90f.x86_64 from CentOS-AppStream 2024-04-12T20:41:40.750 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:8030020200716213634:98a3c9d0.x86_64 from CentOS-AppStream 2024-04-12T20:41:40.750 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:820230223114759:06b7596e.x86_64 from CentOS-AppStream 2024-04-12T20:41:40.750 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:820230223114759:d2fc5d76.x86_64 from CentOS-AppStream 2024-04-12T20:41:40.750 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 44: cannot install the best candidate for the job 2024-04-12T20:41:40.751 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:8030020200716171139:3a70019f.x86_64 from CentOS-AppStream 2024-04-12T20:41:40.751 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:8030020200716171139:cccafca5.x86_64 from CentOS-AppStream 2024-04-12T20:41:40.751 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:820230223114759:6fdf7e5d.x86_64 from CentOS-AppStream 2024-04-12T20:41:40.751 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:820230223114759:d4326aba.x86_64 from CentOS-AppStream 2024-04-12T20:41:40.751 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 45: cannot install the best candidate for the job 2024-04-12T20:41:40.751 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:8010020191120175858:95fce66d.x86_64 from CentOS-AppStream 2024-04-12T20:41:40.751 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:8030020200723233049:09acf126.x86_64 from CentOS-AppStream 2024-04-12T20:41:40.751 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:820230223114759:436af1ae.x86_64 from CentOS-AppStream 2024-04-12T20:41:40.751 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:820230223114759:b12195ea.x86_64 from CentOS-AppStream 2024-04-12T20:41:40.752 INFO:teuthology.orchestra.run.smithi186.stdout:Package librbd1-2:17.2.7-827.gd3b5060a.el9.x86_64 is already installed. 2024-04-12T20:41:40.812 INFO:teuthology.orchestra.run.smithi186.stdout:Dependencies resolved. 2024-04-12T20:41:40.813 INFO:teuthology.orchestra.run.smithi186.stdout:Nothing to do. 2024-04-12T20:41:40.813 INFO:teuthology.orchestra.run.smithi186.stdout:Complete! 2024-04-12T20:41:40.886 DEBUG:teuthology.orchestra.run.smithi186:> sudo yum -y install python3-rados 2024-04-12T20:41:40.928 INFO:teuthology.orchestra.run.smithi018.stderr:Modular dependency problems: 2024-04-12T20:41:40.928 INFO:teuthology.orchestra.run.smithi018.stderr: 2024-04-12T20:41:40.928 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 1: conflicting requests 2024-04-12T20:41:40.928 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module container-tools:rhel8:820240321091303:20125149.x86_64 from CentOS-AppStream 2024-04-12T20:41:40.928 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 2: conflicting requests 2024-04-12T20:41:40.928 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module eclipse:rhel8:820201023100746:b230ed4e.x86_64 from CentOS-AppStream 2024-04-12T20:41:40.928 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 3: conflicting requests 2024-04-12T20:41:40.928 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module freeradius:3.0:8080020230117180605:89170a74.x86_64 from CentOS-AppStream 2024-04-12T20:41:40.929 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 4: conflicting requests 2024-04-12T20:41:40.929 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module gimp:2.8:8000020190628145146:4148dfdf.x86_64 from CentOS-AppStream 2024-04-12T20:41:40.929 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 5: conflicting requests 2024-04-12T20:41:40.929 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module go-toolset:rhel8:820240401160413:b754926a.x86_64 from CentOS-AppStream 2024-04-12T20:41:40.929 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 6: conflicting requests 2024-04-12T20:41:40.929 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module httpd:2.4:820240216084734:9edba152.x86_64 from CentOS-AppStream 2024-04-12T20:41:40.929 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 7: conflicting requests 2024-04-12T20:41:40.929 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module idm:client:820240307184541:49cc9d1b.x86_64 from CentOS-AppStream 2024-04-12T20:41:40.929 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 8: conflicting requests 2024-04-12T20:41:40.930 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module inkscape:0.92.3:8080020221205124429:3e031279.x86_64 from CentOS-AppStream 2024-04-12T20:41:40.930 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 9: conflicting requests 2024-04-12T20:41:40.930 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module jmc:rhel8:8050020211110222101:6392b1f8.x86_64 from CentOS-AppStream 2024-04-12T20:41:40.930 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 10: conflicting requests 2024-04-12T20:41:40.930 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module llvm-toolset:rhel8:820240205133135:9edba152.x86_64 from CentOS-AppStream 2024-04-12T20:41:40.930 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 11: conflicting requests 2024-04-12T20:41:40.930 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module log4j:2:8080020221205124743:9d367344.x86_64 from CentOS-AppStream 2024-04-12T20:41:40.930 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 12: conflicting requests 2024-04-12T20:41:40.930 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module mailman:2.1:820230727085757:77fc8825.x86_64 from CentOS-AppStream 2024-04-12T20:41:40.930 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 13: conflicting requests 2024-04-12T20:41:40.930 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module mariadb:10.3:8030020210419150013:30b713e6.x86_64 from CentOS-AppStream 2024-04-12T20:41:40.930 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 14: conflicting requests 2024-04-12T20:41:40.930 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module maven:3.5:8060020211203142015:c0229ad2.x86_64 from CentOS-AppStream 2024-04-12T20:41:40.931 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 15: conflicting requests 2024-04-12T20:41:40.931 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module mercurial:4.8:8000020190628020724:4148dfdf.x86_64 from CentOS-AppStream 2024-04-12T20:41:40.931 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 16: conflicting requests 2024-04-12T20:41:40.931 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module mod_auth_openidc:2.3:820230721123832:9edba152.x86_64 from CentOS-AppStream 2024-04-12T20:41:40.931 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 17: conflicting requests 2024-04-12T20:41:40.931 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module mysql:8.0:8040020210901180257:9f9e2e7e.x86_64 from CentOS-AppStream 2024-04-12T20:41:40.931 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 18: conflicting requests 2024-04-12T20:41:40.931 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module nginx:1.14:8000020211221191913:55190bc5.x86_64 from CentOS-AppStream 2024-04-12T20:41:40.931 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 19: conflicting requests 2024-04-12T20:41:40.931 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module nodejs:10:8040020210120182536:9f9e2e7e.x86_64 from CentOS-AppStream 2024-04-12T20:41:40.931 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 20: conflicting requests 2024-04-12T20:41:40.931 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl:5.26:8000020190628020724:55190bc5.x86_64 from CentOS-AppStream 2024-04-12T20:41:40.932 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 21: conflicting requests 2024-04-12T20:41:40.932 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module php:7.2:8020020200507003613:2c7ca891.x86_64 from CentOS-AppStream 2024-04-12T20:41:40.932 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 22: conflicting requests 2024-04-12T20:41:40.932 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module postgresql:10:820230223114800:9edba152.x86_64 from CentOS-AppStream 2024-04-12T20:41:40.932 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 23: conflicting requests 2024-04-12T20:41:40.932 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module python27:2.7:820240208011952:182f7c73.x86_64 from CentOS-AppStream 2024-04-12T20:41:40.932 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 24: conflicting requests 2024-04-12T20:41:40.932 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module python36:3.6:8050020210825152031:982725ab.x86_64 from CentOS-AppStream 2024-04-12T20:41:40.932 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 25: conflicting requests 2024-04-12T20:41:40.932 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module python38:3.8:820230810143931:64c1cd5a.x86_64 from CentOS-AppStream 2024-04-12T20:41:40.932 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 26: conflicting requests 2024-04-12T20:41:40.932 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module python39:3.9:820240214182535:17377f89.x86_64 from CentOS-AppStream 2024-04-12T20:41:40.932 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 27: conflicting requests 2024-04-12T20:41:40.932 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module redis:5:8040020211019153849:9f9e2e7e.x86_64 from CentOS-AppStream 2024-04-12T20:41:40.933 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 28: conflicting requests 2024-04-12T20:41:40.933 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module rhn-tools:1.0:8010020191114034948:f69d1239.x86_64 from CentOS-AppStream 2024-04-12T20:41:40.933 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 29: conflicting requests 2024-04-12T20:41:40.933 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module ruby:2.5:820230627084142:9edba152.x86_64 from CentOS-AppStream 2024-04-12T20:41:40.933 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 30: conflicting requests 2024-04-12T20:41:40.933 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module rust-toolset:rhel8:820240119204620:b09eea91.x86_64 from CentOS-AppStream 2024-04-12T20:41:40.933 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 31: conflicting requests 2024-04-12T20:41:40.933 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module satellite-5-client:1.0:8010020191114035551:cdc1202b.x86_64 from CentOS-AppStream 2024-04-12T20:41:40.933 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 32: conflicting requests 2024-04-12T20:41:40.933 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module squid:4:820240319173245:9edba152.x86_64 from CentOS-AppStream 2024-04-12T20:41:40.933 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 33: conflicting requests 2024-04-12T20:41:40.933 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module subversion:1.10:8070020220711155714:78111232.x86_64 from CentOS-AppStream 2024-04-12T20:41:40.933 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 34: conflicting requests 2024-04-12T20:41:40.933 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module swig:3.0:8030020200706142531:30b713e6.x86_64 from CentOS-AppStream 2024-04-12T20:41:40.934 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 35: conflicting requests 2024-04-12T20:41:40.934 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module varnish:6:8050020211109225449:b4937e53.x86_64 from CentOS-AppStream 2024-04-12T20:41:40.934 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 36: conflicting requests 2024-04-12T20:41:40.934 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module virt:rhel:820240314161907:9edba152.x86_64 from CentOS-AppStream 2024-04-12T20:41:40.934 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 37: conflicting requests 2024-04-12T20:41:40.934 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:8030020200716155257:7cc0a66d.x86_64 from CentOS-AppStream 2024-04-12T20:41:40.934 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:8030020200716155257:b967a9a2.x86_64 from CentOS-AppStream 2024-04-12T20:41:40.934 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:820230811133638:36f9fae6.x86_64 from CentOS-AppStream 2024-04-12T20:41:40.934 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 38: cannot install the best candidate for the job 2024-04-12T20:41:40.934 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:8010020191114031501:a5949e2e.x86_64 from CentOS-AppStream 2024-04-12T20:41:40.934 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:8010020191114031501:be2e4737.x86_64 from CentOS-AppStream 2024-04-12T20:41:40.934 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:820230223114759:2ab761e1.x86_64 from CentOS-AppStream 2024-04-12T20:41:40.934 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:820230223114759:86e45846.x86_64 from CentOS-AppStream 2024-04-12T20:41:40.934 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 39: conflicting requests 2024-04-12T20:41:40.935 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:8030020200715230104:1e4bbb35.x86_64 from CentOS-AppStream 2024-04-12T20:41:40.935 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:8030020200715230104:ea09926d.x86_64 from CentOS-AppStream 2024-04-12T20:41:40.935 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:082fdf2f.x86_64 from CentOS-AppStream 2024-04-12T20:41:40.935 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-04-12T20:41:40.935 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-04-12T20:41:40.935 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:fbe42456.x86_64 from CentOS-AppStream 2024-04-12T20:41:40.935 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 40: cannot install the best candidate for the job 2024-04-12T20:41:40.935 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:8010020191114031513:16b3ab4d.x86_64 from CentOS-AppStream 2024-04-12T20:41:40.935 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:8010020191114031513:e1f37755.x86_64 from CentOS-AppStream 2024-04-12T20:41:40.935 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-04-12T20:41:40.935 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-04-12T20:41:40.935 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 41: cannot install the best candidate for the job 2024-04-12T20:41:40.935 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:8030020200716150652:1e4bbb35.x86_64 from CentOS-AppStream 2024-04-12T20:41:40.935 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:8030020200716150652:ea09926d.x86_64 from CentOS-AppStream 2024-04-12T20:41:40.936 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-04-12T20:41:40.936 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-04-12T20:41:40.936 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 42: cannot install the best candidate for the job 2024-04-12T20:41:40.936 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:8030020200716174729:3a70019f.x86_64 from CentOS-AppStream 2024-04-12T20:41:40.936 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:8030020200716174729:cccafca5.x86_64 from CentOS-AppStream 2024-04-12T20:41:40.936 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:820230223114759:6fdf7e5d.x86_64 from CentOS-AppStream 2024-04-12T20:41:40.936 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:820230223114759:d4326aba.x86_64 from CentOS-AppStream 2024-04-12T20:41:40.936 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 43: cannot install the best candidate for the job 2024-04-12T20:41:40.936 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:8030020200716213634:56fce90f.x86_64 from CentOS-AppStream 2024-04-12T20:41:40.936 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:8030020200716213634:98a3c9d0.x86_64 from CentOS-AppStream 2024-04-12T20:41:40.936 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:820230223114759:06b7596e.x86_64 from CentOS-AppStream 2024-04-12T20:41:40.936 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:820230223114759:d2fc5d76.x86_64 from CentOS-AppStream 2024-04-12T20:41:40.936 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 44: cannot install the best candidate for the job 2024-04-12T20:41:40.937 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:8030020200716171139:3a70019f.x86_64 from CentOS-AppStream 2024-04-12T20:41:40.937 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:8030020200716171139:cccafca5.x86_64 from CentOS-AppStream 2024-04-12T20:41:40.937 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:820230223114759:6fdf7e5d.x86_64 from CentOS-AppStream 2024-04-12T20:41:40.937 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:820230223114759:d4326aba.x86_64 from CentOS-AppStream 2024-04-12T20:41:40.937 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 45: cannot install the best candidate for the job 2024-04-12T20:41:40.937 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:8010020191120175858:95fce66d.x86_64 from CentOS-AppStream 2024-04-12T20:41:40.937 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:8030020200723233049:09acf126.x86_64 from CentOS-AppStream 2024-04-12T20:41:40.937 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:820230223114759:436af1ae.x86_64 from CentOS-AppStream 2024-04-12T20:41:40.937 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:820230223114759:b12195ea.x86_64 from CentOS-AppStream 2024-04-12T20:41:40.937 INFO:teuthology.orchestra.run.smithi018.stdout:Package librbd1-2:17.2.7-827.gd3b5060a.el9.x86_64 is already installed. 2024-04-12T20:41:41.002 INFO:teuthology.orchestra.run.smithi018.stdout:Dependencies resolved. 2024-04-12T20:41:41.003 INFO:teuthology.orchestra.run.smithi018.stdout:Nothing to do. 2024-04-12T20:41:41.003 INFO:teuthology.orchestra.run.smithi018.stdout:Complete! 2024-04-12T20:41:41.086 DEBUG:teuthology.orchestra.run.smithi018:> sudo yum -y install python3-rados 2024-04-12T20:41:41.756 INFO:teuthology.orchestra.run.smithi186.stdout:Last metadata expiration check: 0:02:04 ago on Fri 12 Apr 2024 08:39:37 PM UTC. 2024-04-12T20:41:41.964 INFO:teuthology.orchestra.run.smithi018.stdout:Last metadata expiration check: 0:02:06 ago on Fri 12 Apr 2024 08:39:35 PM UTC. 2024-04-12T20:41:42.058 INFO:teuthology.orchestra.run.smithi186.stderr:Modular dependency problems: 2024-04-12T20:41:42.058 INFO:teuthology.orchestra.run.smithi186.stderr: 2024-04-12T20:41:42.058 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 1: conflicting requests 2024-04-12T20:41:42.058 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module container-tools:rhel8:820240321091303:20125149.x86_64 from CentOS-AppStream 2024-04-12T20:41:42.058 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 2: conflicting requests 2024-04-12T20:41:42.058 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module eclipse:rhel8:820201023100746:b230ed4e.x86_64 from CentOS-AppStream 2024-04-12T20:41:42.059 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 3: conflicting requests 2024-04-12T20:41:42.059 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module freeradius:3.0:8080020230117180605:89170a74.x86_64 from CentOS-AppStream 2024-04-12T20:41:42.059 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 4: conflicting requests 2024-04-12T20:41:42.059 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module gimp:2.8:8000020190628145146:4148dfdf.x86_64 from CentOS-AppStream 2024-04-12T20:41:42.059 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 5: conflicting requests 2024-04-12T20:41:42.059 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module go-toolset:rhel8:820240401160413:b754926a.x86_64 from CentOS-AppStream 2024-04-12T20:41:42.059 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 6: conflicting requests 2024-04-12T20:41:42.059 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module httpd:2.4:820240216084734:9edba152.x86_64 from CentOS-AppStream 2024-04-12T20:41:42.059 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 7: conflicting requests 2024-04-12T20:41:42.059 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module idm:client:820240307184541:49cc9d1b.x86_64 from CentOS-AppStream 2024-04-12T20:41:42.060 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 8: conflicting requests 2024-04-12T20:41:42.060 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module inkscape:0.92.3:8080020221205124429:3e031279.x86_64 from CentOS-AppStream 2024-04-12T20:41:42.060 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 9: conflicting requests 2024-04-12T20:41:42.060 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module jmc:rhel8:8050020211110222101:6392b1f8.x86_64 from CentOS-AppStream 2024-04-12T20:41:42.060 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 10: conflicting requests 2024-04-12T20:41:42.060 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module llvm-toolset:rhel8:820240205133135:9edba152.x86_64 from CentOS-AppStream 2024-04-12T20:41:42.060 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 11: conflicting requests 2024-04-12T20:41:42.060 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module log4j:2:8080020221205124743:9d367344.x86_64 from CentOS-AppStream 2024-04-12T20:41:42.060 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 12: conflicting requests 2024-04-12T20:41:42.061 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module mailman:2.1:820230727085757:77fc8825.x86_64 from CentOS-AppStream 2024-04-12T20:41:42.061 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 13: conflicting requests 2024-04-12T20:41:42.061 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module mariadb:10.3:8030020210419150013:30b713e6.x86_64 from CentOS-AppStream 2024-04-12T20:41:42.061 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 14: conflicting requests 2024-04-12T20:41:42.061 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module maven:3.5:8060020211203142015:c0229ad2.x86_64 from CentOS-AppStream 2024-04-12T20:41:42.061 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 15: conflicting requests 2024-04-12T20:41:42.061 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module mercurial:4.8:8000020190628020724:4148dfdf.x86_64 from CentOS-AppStream 2024-04-12T20:41:42.061 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 16: conflicting requests 2024-04-12T20:41:42.061 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module mod_auth_openidc:2.3:820230721123832:9edba152.x86_64 from CentOS-AppStream 2024-04-12T20:41:42.061 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 17: conflicting requests 2024-04-12T20:41:42.062 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module mysql:8.0:8040020210901180257:9f9e2e7e.x86_64 from CentOS-AppStream 2024-04-12T20:41:42.062 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 18: conflicting requests 2024-04-12T20:41:42.062 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module nginx:1.14:8000020211221191913:55190bc5.x86_64 from CentOS-AppStream 2024-04-12T20:41:42.062 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 19: conflicting requests 2024-04-12T20:41:42.062 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module nodejs:10:8040020210120182536:9f9e2e7e.x86_64 from CentOS-AppStream 2024-04-12T20:41:42.062 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 20: conflicting requests 2024-04-12T20:41:42.062 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl:5.26:8000020190628020724:55190bc5.x86_64 from CentOS-AppStream 2024-04-12T20:41:42.062 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 21: conflicting requests 2024-04-12T20:41:42.062 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module php:7.2:8020020200507003613:2c7ca891.x86_64 from CentOS-AppStream 2024-04-12T20:41:42.062 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 22: conflicting requests 2024-04-12T20:41:42.063 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module postgresql:10:820230223114800:9edba152.x86_64 from CentOS-AppStream 2024-04-12T20:41:42.063 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 23: conflicting requests 2024-04-12T20:41:42.063 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module python27:2.7:820240208011952:182f7c73.x86_64 from CentOS-AppStream 2024-04-12T20:41:42.063 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 24: conflicting requests 2024-04-12T20:41:42.063 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module python36:3.6:8050020210825152031:982725ab.x86_64 from CentOS-AppStream 2024-04-12T20:41:42.063 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 25: conflicting requests 2024-04-12T20:41:42.063 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module python38:3.8:820230810143931:64c1cd5a.x86_64 from CentOS-AppStream 2024-04-12T20:41:42.063 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 26: conflicting requests 2024-04-12T20:41:42.063 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module python39:3.9:820240214182535:17377f89.x86_64 from CentOS-AppStream 2024-04-12T20:41:42.064 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 27: conflicting requests 2024-04-12T20:41:42.064 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module redis:5:8040020211019153849:9f9e2e7e.x86_64 from CentOS-AppStream 2024-04-12T20:41:42.064 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 28: conflicting requests 2024-04-12T20:41:42.064 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module rhn-tools:1.0:8010020191114034948:f69d1239.x86_64 from CentOS-AppStream 2024-04-12T20:41:42.064 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 29: conflicting requests 2024-04-12T20:41:42.064 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module ruby:2.5:820230627084142:9edba152.x86_64 from CentOS-AppStream 2024-04-12T20:41:42.064 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 30: conflicting requests 2024-04-12T20:41:42.064 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module rust-toolset:rhel8:820240119204620:b09eea91.x86_64 from CentOS-AppStream 2024-04-12T20:41:42.064 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 31: conflicting requests 2024-04-12T20:41:42.065 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module satellite-5-client:1.0:8010020191114035551:cdc1202b.x86_64 from CentOS-AppStream 2024-04-12T20:41:42.065 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 32: conflicting requests 2024-04-12T20:41:42.065 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module squid:4:820240319173245:9edba152.x86_64 from CentOS-AppStream 2024-04-12T20:41:42.065 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 33: conflicting requests 2024-04-12T20:41:42.065 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module subversion:1.10:8070020220711155714:78111232.x86_64 from CentOS-AppStream 2024-04-12T20:41:42.065 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 34: conflicting requests 2024-04-12T20:41:42.065 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module swig:3.0:8030020200706142531:30b713e6.x86_64 from CentOS-AppStream 2024-04-12T20:41:42.065 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 35: conflicting requests 2024-04-12T20:41:42.066 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module varnish:6:8050020211109225449:b4937e53.x86_64 from CentOS-AppStream 2024-04-12T20:41:42.066 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 36: conflicting requests 2024-04-12T20:41:42.066 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module virt:rhel:820240314161907:9edba152.x86_64 from CentOS-AppStream 2024-04-12T20:41:42.066 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 37: conflicting requests 2024-04-12T20:41:42.066 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:8030020200716155257:7cc0a66d.x86_64 from CentOS-AppStream 2024-04-12T20:41:42.066 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:8030020200716155257:b967a9a2.x86_64 from CentOS-AppStream 2024-04-12T20:41:42.066 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:820230811133638:36f9fae6.x86_64 from CentOS-AppStream 2024-04-12T20:41:42.067 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 38: cannot install the best candidate for the job 2024-04-12T20:41:42.067 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:8010020191114031501:a5949e2e.x86_64 from CentOS-AppStream 2024-04-12T20:41:42.067 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:8010020191114031501:be2e4737.x86_64 from CentOS-AppStream 2024-04-12T20:41:42.067 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:820230223114759:2ab761e1.x86_64 from CentOS-AppStream 2024-04-12T20:41:42.067 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:820230223114759:86e45846.x86_64 from CentOS-AppStream 2024-04-12T20:41:42.067 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 39: conflicting requests 2024-04-12T20:41:42.067 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:8030020200715230104:1e4bbb35.x86_64 from CentOS-AppStream 2024-04-12T20:41:42.067 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:8030020200715230104:ea09926d.x86_64 from CentOS-AppStream 2024-04-12T20:41:42.067 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:082fdf2f.x86_64 from CentOS-AppStream 2024-04-12T20:41:42.067 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-04-12T20:41:42.068 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-04-12T20:41:42.068 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:fbe42456.x86_64 from CentOS-AppStream 2024-04-12T20:41:42.068 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 40: cannot install the best candidate for the job 2024-04-12T20:41:42.068 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:8010020191114031513:16b3ab4d.x86_64 from CentOS-AppStream 2024-04-12T20:41:42.068 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:8010020191114031513:e1f37755.x86_64 from CentOS-AppStream 2024-04-12T20:41:42.068 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-04-12T20:41:42.068 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-04-12T20:41:42.068 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 41: cannot install the best candidate for the job 2024-04-12T20:41:42.068 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:8030020200716150652:1e4bbb35.x86_64 from CentOS-AppStream 2024-04-12T20:41:42.068 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:8030020200716150652:ea09926d.x86_64 from CentOS-AppStream 2024-04-12T20:41:42.068 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-04-12T20:41:42.069 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-04-12T20:41:42.069 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 42: cannot install the best candidate for the job 2024-04-12T20:41:42.069 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:8030020200716174729:3a70019f.x86_64 from CentOS-AppStream 2024-04-12T20:41:42.069 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:8030020200716174729:cccafca5.x86_64 from CentOS-AppStream 2024-04-12T20:41:42.069 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:820230223114759:6fdf7e5d.x86_64 from CentOS-AppStream 2024-04-12T20:41:42.069 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:820230223114759:d4326aba.x86_64 from CentOS-AppStream 2024-04-12T20:41:42.069 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 43: cannot install the best candidate for the job 2024-04-12T20:41:42.069 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:8030020200716213634:56fce90f.x86_64 from CentOS-AppStream 2024-04-12T20:41:42.069 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:8030020200716213634:98a3c9d0.x86_64 from CentOS-AppStream 2024-04-12T20:41:42.069 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:820230223114759:06b7596e.x86_64 from CentOS-AppStream 2024-04-12T20:41:42.069 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:820230223114759:d2fc5d76.x86_64 from CentOS-AppStream 2024-04-12T20:41:42.069 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 44: cannot install the best candidate for the job 2024-04-12T20:41:42.069 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:8030020200716171139:3a70019f.x86_64 from CentOS-AppStream 2024-04-12T20:41:42.070 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:8030020200716171139:cccafca5.x86_64 from CentOS-AppStream 2024-04-12T20:41:42.070 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:820230223114759:6fdf7e5d.x86_64 from CentOS-AppStream 2024-04-12T20:41:42.070 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:820230223114759:d4326aba.x86_64 from CentOS-AppStream 2024-04-12T20:41:42.070 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 45: cannot install the best candidate for the job 2024-04-12T20:41:42.070 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:8010020191120175858:95fce66d.x86_64 from CentOS-AppStream 2024-04-12T20:41:42.070 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:8030020200723233049:09acf126.x86_64 from CentOS-AppStream 2024-04-12T20:41:42.070 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:820230223114759:436af1ae.x86_64 from CentOS-AppStream 2024-04-12T20:41:42.070 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:820230223114759:b12195ea.x86_64 from CentOS-AppStream 2024-04-12T20:41:42.071 INFO:teuthology.orchestra.run.smithi186.stdout:Package python3-rados-2:17.2.7-827.gd3b5060a.el9.x86_64 is already installed. 2024-04-12T20:41:42.130 INFO:teuthology.orchestra.run.smithi186.stdout:Dependencies resolved. 2024-04-12T20:41:42.132 INFO:teuthology.orchestra.run.smithi186.stdout:Nothing to do. 2024-04-12T20:41:42.132 INFO:teuthology.orchestra.run.smithi186.stdout:Complete! 2024-04-12T20:41:42.201 DEBUG:teuthology.orchestra.run.smithi186:> sudo yum -y install python3-rgw 2024-04-12T20:41:42.289 INFO:teuthology.orchestra.run.smithi018.stderr:Modular dependency problems: 2024-04-12T20:41:42.290 INFO:teuthology.orchestra.run.smithi018.stderr: 2024-04-12T20:41:42.290 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 1: conflicting requests 2024-04-12T20:41:42.290 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module container-tools:rhel8:820240321091303:20125149.x86_64 from CentOS-AppStream 2024-04-12T20:41:42.290 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 2: conflicting requests 2024-04-12T20:41:42.290 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module eclipse:rhel8:820201023100746:b230ed4e.x86_64 from CentOS-AppStream 2024-04-12T20:41:42.290 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 3: conflicting requests 2024-04-12T20:41:42.290 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module freeradius:3.0:8080020230117180605:89170a74.x86_64 from CentOS-AppStream 2024-04-12T20:41:42.290 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 4: conflicting requests 2024-04-12T20:41:42.290 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module gimp:2.8:8000020190628145146:4148dfdf.x86_64 from CentOS-AppStream 2024-04-12T20:41:42.290 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 5: conflicting requests 2024-04-12T20:41:42.290 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module go-toolset:rhel8:820240401160413:b754926a.x86_64 from CentOS-AppStream 2024-04-12T20:41:42.290 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 6: conflicting requests 2024-04-12T20:41:42.290 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module httpd:2.4:820240216084734:9edba152.x86_64 from CentOS-AppStream 2024-04-12T20:41:42.291 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 7: conflicting requests 2024-04-12T20:41:42.291 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module idm:client:820240307184541:49cc9d1b.x86_64 from CentOS-AppStream 2024-04-12T20:41:42.291 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 8: conflicting requests 2024-04-12T20:41:42.291 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module inkscape:0.92.3:8080020221205124429:3e031279.x86_64 from CentOS-AppStream 2024-04-12T20:41:42.291 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 9: conflicting requests 2024-04-12T20:41:42.291 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module jmc:rhel8:8050020211110222101:6392b1f8.x86_64 from CentOS-AppStream 2024-04-12T20:41:42.291 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 10: conflicting requests 2024-04-12T20:41:42.291 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module llvm-toolset:rhel8:820240205133135:9edba152.x86_64 from CentOS-AppStream 2024-04-12T20:41:42.291 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 11: conflicting requests 2024-04-12T20:41:42.291 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module log4j:2:8080020221205124743:9d367344.x86_64 from CentOS-AppStream 2024-04-12T20:41:42.291 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 12: conflicting requests 2024-04-12T20:41:42.291 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module mailman:2.1:820230727085757:77fc8825.x86_64 from CentOS-AppStream 2024-04-12T20:41:42.291 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 13: conflicting requests 2024-04-12T20:41:42.292 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module mariadb:10.3:8030020210419150013:30b713e6.x86_64 from CentOS-AppStream 2024-04-12T20:41:42.292 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 14: conflicting requests 2024-04-12T20:41:42.292 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module maven:3.5:8060020211203142015:c0229ad2.x86_64 from CentOS-AppStream 2024-04-12T20:41:42.292 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 15: conflicting requests 2024-04-12T20:41:42.292 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module mercurial:4.8:8000020190628020724:4148dfdf.x86_64 from CentOS-AppStream 2024-04-12T20:41:42.292 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 16: conflicting requests 2024-04-12T20:41:42.292 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module mod_auth_openidc:2.3:820230721123832:9edba152.x86_64 from CentOS-AppStream 2024-04-12T20:41:42.292 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 17: conflicting requests 2024-04-12T20:41:42.292 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module mysql:8.0:8040020210901180257:9f9e2e7e.x86_64 from CentOS-AppStream 2024-04-12T20:41:42.292 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 18: conflicting requests 2024-04-12T20:41:42.292 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module nginx:1.14:8000020211221191913:55190bc5.x86_64 from CentOS-AppStream 2024-04-12T20:41:42.292 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 19: conflicting requests 2024-04-12T20:41:42.292 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module nodejs:10:8040020210120182536:9f9e2e7e.x86_64 from CentOS-AppStream 2024-04-12T20:41:42.292 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 20: conflicting requests 2024-04-12T20:41:42.292 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl:5.26:8000020190628020724:55190bc5.x86_64 from CentOS-AppStream 2024-04-12T20:41:42.293 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 21: conflicting requests 2024-04-12T20:41:42.293 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module php:7.2:8020020200507003613:2c7ca891.x86_64 from CentOS-AppStream 2024-04-12T20:41:42.293 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 22: conflicting requests 2024-04-12T20:41:42.293 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module postgresql:10:820230223114800:9edba152.x86_64 from CentOS-AppStream 2024-04-12T20:41:42.293 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 23: conflicting requests 2024-04-12T20:41:42.293 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module python27:2.7:820240208011952:182f7c73.x86_64 from CentOS-AppStream 2024-04-12T20:41:42.293 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 24: conflicting requests 2024-04-12T20:41:42.293 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module python36:3.6:8050020210825152031:982725ab.x86_64 from CentOS-AppStream 2024-04-12T20:41:42.293 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 25: conflicting requests 2024-04-12T20:41:42.294 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module python38:3.8:820230810143931:64c1cd5a.x86_64 from CentOS-AppStream 2024-04-12T20:41:42.294 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 26: conflicting requests 2024-04-12T20:41:42.294 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module python39:3.9:820240214182535:17377f89.x86_64 from CentOS-AppStream 2024-04-12T20:41:42.294 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 27: conflicting requests 2024-04-12T20:41:42.294 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module redis:5:8040020211019153849:9f9e2e7e.x86_64 from CentOS-AppStream 2024-04-12T20:41:42.294 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 28: conflicting requests 2024-04-12T20:41:42.294 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module rhn-tools:1.0:8010020191114034948:f69d1239.x86_64 from CentOS-AppStream 2024-04-12T20:41:42.294 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 29: conflicting requests 2024-04-12T20:41:42.294 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module ruby:2.5:820230627084142:9edba152.x86_64 from CentOS-AppStream 2024-04-12T20:41:42.294 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 30: conflicting requests 2024-04-12T20:41:42.295 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module rust-toolset:rhel8:820240119204620:b09eea91.x86_64 from CentOS-AppStream 2024-04-12T20:41:42.295 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 31: conflicting requests 2024-04-12T20:41:42.295 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module satellite-5-client:1.0:8010020191114035551:cdc1202b.x86_64 from CentOS-AppStream 2024-04-12T20:41:42.295 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 32: conflicting requests 2024-04-12T20:41:42.295 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module squid:4:820240319173245:9edba152.x86_64 from CentOS-AppStream 2024-04-12T20:41:42.295 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 33: conflicting requests 2024-04-12T20:41:42.295 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module subversion:1.10:8070020220711155714:78111232.x86_64 from CentOS-AppStream 2024-04-12T20:41:42.295 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 34: conflicting requests 2024-04-12T20:41:42.295 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module swig:3.0:8030020200706142531:30b713e6.x86_64 from CentOS-AppStream 2024-04-12T20:41:42.295 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 35: conflicting requests 2024-04-12T20:41:42.295 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module varnish:6:8050020211109225449:b4937e53.x86_64 from CentOS-AppStream 2024-04-12T20:41:42.295 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 36: conflicting requests 2024-04-12T20:41:42.295 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module virt:rhel:820240314161907:9edba152.x86_64 from CentOS-AppStream 2024-04-12T20:41:42.295 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 37: conflicting requests 2024-04-12T20:41:42.296 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:8030020200716155257:7cc0a66d.x86_64 from CentOS-AppStream 2024-04-12T20:41:42.296 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:8030020200716155257:b967a9a2.x86_64 from CentOS-AppStream 2024-04-12T20:41:42.296 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:820230811133638:36f9fae6.x86_64 from CentOS-AppStream 2024-04-12T20:41:42.296 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 38: cannot install the best candidate for the job 2024-04-12T20:41:42.296 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:8010020191114031501:a5949e2e.x86_64 from CentOS-AppStream 2024-04-12T20:41:42.296 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:8010020191114031501:be2e4737.x86_64 from CentOS-AppStream 2024-04-12T20:41:42.296 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:820230223114759:2ab761e1.x86_64 from CentOS-AppStream 2024-04-12T20:41:42.296 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:820230223114759:86e45846.x86_64 from CentOS-AppStream 2024-04-12T20:41:42.296 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 39: conflicting requests 2024-04-12T20:41:42.296 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:8030020200715230104:1e4bbb35.x86_64 from CentOS-AppStream 2024-04-12T20:41:42.296 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:8030020200715230104:ea09926d.x86_64 from CentOS-AppStream 2024-04-12T20:41:42.296 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:082fdf2f.x86_64 from CentOS-AppStream 2024-04-12T20:41:42.296 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-04-12T20:41:42.296 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-04-12T20:41:42.297 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:fbe42456.x86_64 from CentOS-AppStream 2024-04-12T20:41:42.297 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 40: cannot install the best candidate for the job 2024-04-12T20:41:42.297 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:8010020191114031513:16b3ab4d.x86_64 from CentOS-AppStream 2024-04-12T20:41:42.297 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:8010020191114031513:e1f37755.x86_64 from CentOS-AppStream 2024-04-12T20:41:42.297 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-04-12T20:41:42.297 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-04-12T20:41:42.297 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 41: cannot install the best candidate for the job 2024-04-12T20:41:42.297 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:8030020200716150652:1e4bbb35.x86_64 from CentOS-AppStream 2024-04-12T20:41:42.297 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:8030020200716150652:ea09926d.x86_64 from CentOS-AppStream 2024-04-12T20:41:42.297 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-04-12T20:41:42.297 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-04-12T20:41:42.297 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 42: cannot install the best candidate for the job 2024-04-12T20:41:42.297 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:8030020200716174729:3a70019f.x86_64 from CentOS-AppStream 2024-04-12T20:41:42.297 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:8030020200716174729:cccafca5.x86_64 from CentOS-AppStream 2024-04-12T20:41:42.298 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:820230223114759:6fdf7e5d.x86_64 from CentOS-AppStream 2024-04-12T20:41:42.298 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:820230223114759:d4326aba.x86_64 from CentOS-AppStream 2024-04-12T20:41:42.298 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 43: cannot install the best candidate for the job 2024-04-12T20:41:42.298 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:8030020200716213634:56fce90f.x86_64 from CentOS-AppStream 2024-04-12T20:41:42.298 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:8030020200716213634:98a3c9d0.x86_64 from CentOS-AppStream 2024-04-12T20:41:42.298 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:820230223114759:06b7596e.x86_64 from CentOS-AppStream 2024-04-12T20:41:42.298 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:820230223114759:d2fc5d76.x86_64 from CentOS-AppStream 2024-04-12T20:41:42.298 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 44: cannot install the best candidate for the job 2024-04-12T20:41:42.298 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:8030020200716171139:3a70019f.x86_64 from CentOS-AppStream 2024-04-12T20:41:42.298 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:8030020200716171139:cccafca5.x86_64 from CentOS-AppStream 2024-04-12T20:41:42.298 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:820230223114759:6fdf7e5d.x86_64 from CentOS-AppStream 2024-04-12T20:41:42.298 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:820230223114759:d4326aba.x86_64 from CentOS-AppStream 2024-04-12T20:41:42.298 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 45: cannot install the best candidate for the job 2024-04-12T20:41:42.299 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:8010020191120175858:95fce66d.x86_64 from CentOS-AppStream 2024-04-12T20:41:42.299 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:8030020200723233049:09acf126.x86_64 from CentOS-AppStream 2024-04-12T20:41:42.299 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:820230223114759:436af1ae.x86_64 from CentOS-AppStream 2024-04-12T20:41:42.299 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:820230223114759:b12195ea.x86_64 from CentOS-AppStream 2024-04-12T20:41:42.299 INFO:teuthology.orchestra.run.smithi018.stdout:Package python3-rados-2:17.2.7-827.gd3b5060a.el9.x86_64 is already installed. 2024-04-12T20:41:42.370 INFO:teuthology.orchestra.run.smithi018.stdout:Dependencies resolved. 2024-04-12T20:41:42.371 INFO:teuthology.orchestra.run.smithi018.stdout:Nothing to do. 2024-04-12T20:41:42.371 INFO:teuthology.orchestra.run.smithi018.stdout:Complete! 2024-04-12T20:41:42.441 DEBUG:teuthology.orchestra.run.smithi018:> sudo yum -y install python3-rgw 2024-04-12T20:41:43.063 INFO:teuthology.orchestra.run.smithi186.stdout:Last metadata expiration check: 0:02:06 ago on Fri 12 Apr 2024 08:39:37 PM UTC. 2024-04-12T20:41:43.328 INFO:teuthology.orchestra.run.smithi018.stdout:Last metadata expiration check: 0:02:08 ago on Fri 12 Apr 2024 08:39:35 PM UTC. 2024-04-12T20:41:43.365 INFO:teuthology.orchestra.run.smithi186.stderr:Modular dependency problems: 2024-04-12T20:41:43.366 INFO:teuthology.orchestra.run.smithi186.stderr: 2024-04-12T20:41:43.366 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 1: conflicting requests 2024-04-12T20:41:43.366 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module container-tools:rhel8:820240321091303:20125149.x86_64 from CentOS-AppStream 2024-04-12T20:41:43.366 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 2: conflicting requests 2024-04-12T20:41:43.366 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module eclipse:rhel8:820201023100746:b230ed4e.x86_64 from CentOS-AppStream 2024-04-12T20:41:43.366 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 3: conflicting requests 2024-04-12T20:41:43.366 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module freeradius:3.0:8080020230117180605:89170a74.x86_64 from CentOS-AppStream 2024-04-12T20:41:43.366 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 4: conflicting requests 2024-04-12T20:41:43.366 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module gimp:2.8:8000020190628145146:4148dfdf.x86_64 from CentOS-AppStream 2024-04-12T20:41:43.366 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 5: conflicting requests 2024-04-12T20:41:43.367 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module go-toolset:rhel8:820240401160413:b754926a.x86_64 from CentOS-AppStream 2024-04-12T20:41:43.367 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 6: conflicting requests 2024-04-12T20:41:43.367 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module httpd:2.4:820240216084734:9edba152.x86_64 from CentOS-AppStream 2024-04-12T20:41:43.367 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 7: conflicting requests 2024-04-12T20:41:43.367 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module idm:client:820240307184541:49cc9d1b.x86_64 from CentOS-AppStream 2024-04-12T20:41:43.367 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 8: conflicting requests 2024-04-12T20:41:43.367 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module inkscape:0.92.3:8080020221205124429:3e031279.x86_64 from CentOS-AppStream 2024-04-12T20:41:43.367 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 9: conflicting requests 2024-04-12T20:41:43.367 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module jmc:rhel8:8050020211110222101:6392b1f8.x86_64 from CentOS-AppStream 2024-04-12T20:41:43.367 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 10: conflicting requests 2024-04-12T20:41:43.367 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module llvm-toolset:rhel8:820240205133135:9edba152.x86_64 from CentOS-AppStream 2024-04-12T20:41:43.367 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 11: conflicting requests 2024-04-12T20:41:43.367 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module log4j:2:8080020221205124743:9d367344.x86_64 from CentOS-AppStream 2024-04-12T20:41:43.367 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 12: conflicting requests 2024-04-12T20:41:43.368 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module mailman:2.1:820230727085757:77fc8825.x86_64 from CentOS-AppStream 2024-04-12T20:41:43.368 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 13: conflicting requests 2024-04-12T20:41:43.368 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module mariadb:10.3:8030020210419150013:30b713e6.x86_64 from CentOS-AppStream 2024-04-12T20:41:43.368 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 14: conflicting requests 2024-04-12T20:41:43.368 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module maven:3.5:8060020211203142015:c0229ad2.x86_64 from CentOS-AppStream 2024-04-12T20:41:43.368 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 15: conflicting requests 2024-04-12T20:41:43.368 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module mercurial:4.8:8000020190628020724:4148dfdf.x86_64 from CentOS-AppStream 2024-04-12T20:41:43.368 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 16: conflicting requests 2024-04-12T20:41:43.368 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module mod_auth_openidc:2.3:820230721123832:9edba152.x86_64 from CentOS-AppStream 2024-04-12T20:41:43.368 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 17: conflicting requests 2024-04-12T20:41:43.368 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module mysql:8.0:8040020210901180257:9f9e2e7e.x86_64 from CentOS-AppStream 2024-04-12T20:41:43.368 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 18: conflicting requests 2024-04-12T20:41:43.368 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module nginx:1.14:8000020211221191913:55190bc5.x86_64 from CentOS-AppStream 2024-04-12T20:41:43.368 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 19: conflicting requests 2024-04-12T20:41:43.369 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module nodejs:10:8040020210120182536:9f9e2e7e.x86_64 from CentOS-AppStream 2024-04-12T20:41:43.369 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 20: conflicting requests 2024-04-12T20:41:43.369 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl:5.26:8000020190628020724:55190bc5.x86_64 from CentOS-AppStream 2024-04-12T20:41:43.369 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 21: conflicting requests 2024-04-12T20:41:43.369 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module php:7.2:8020020200507003613:2c7ca891.x86_64 from CentOS-AppStream 2024-04-12T20:41:43.369 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 22: conflicting requests 2024-04-12T20:41:43.369 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module postgresql:10:820230223114800:9edba152.x86_64 from CentOS-AppStream 2024-04-12T20:41:43.369 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 23: conflicting requests 2024-04-12T20:41:43.369 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module python27:2.7:820240208011952:182f7c73.x86_64 from CentOS-AppStream 2024-04-12T20:41:43.369 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 24: conflicting requests 2024-04-12T20:41:43.369 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module python36:3.6:8050020210825152031:982725ab.x86_64 from CentOS-AppStream 2024-04-12T20:41:43.369 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 25: conflicting requests 2024-04-12T20:41:43.369 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module python38:3.8:820230810143931:64c1cd5a.x86_64 from CentOS-AppStream 2024-04-12T20:41:43.370 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 26: conflicting requests 2024-04-12T20:41:43.370 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module python39:3.9:820240214182535:17377f89.x86_64 from CentOS-AppStream 2024-04-12T20:41:43.370 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 27: conflicting requests 2024-04-12T20:41:43.370 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module redis:5:8040020211019153849:9f9e2e7e.x86_64 from CentOS-AppStream 2024-04-12T20:41:43.370 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 28: conflicting requests 2024-04-12T20:41:43.370 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module rhn-tools:1.0:8010020191114034948:f69d1239.x86_64 from CentOS-AppStream 2024-04-12T20:41:43.370 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 29: conflicting requests 2024-04-12T20:41:43.370 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module ruby:2.5:820230627084142:9edba152.x86_64 from CentOS-AppStream 2024-04-12T20:41:43.370 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 30: conflicting requests 2024-04-12T20:41:43.370 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module rust-toolset:rhel8:820240119204620:b09eea91.x86_64 from CentOS-AppStream 2024-04-12T20:41:43.370 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 31: conflicting requests 2024-04-12T20:41:43.370 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module satellite-5-client:1.0:8010020191114035551:cdc1202b.x86_64 from CentOS-AppStream 2024-04-12T20:41:43.370 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 32: conflicting requests 2024-04-12T20:41:43.370 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module squid:4:820240319173245:9edba152.x86_64 from CentOS-AppStream 2024-04-12T20:41:43.371 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 33: conflicting requests 2024-04-12T20:41:43.371 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module subversion:1.10:8070020220711155714:78111232.x86_64 from CentOS-AppStream 2024-04-12T20:41:43.371 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 34: conflicting requests 2024-04-12T20:41:43.371 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module swig:3.0:8030020200706142531:30b713e6.x86_64 from CentOS-AppStream 2024-04-12T20:41:43.371 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 35: conflicting requests 2024-04-12T20:41:43.371 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module varnish:6:8050020211109225449:b4937e53.x86_64 from CentOS-AppStream 2024-04-12T20:41:43.371 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 36: conflicting requests 2024-04-12T20:41:43.371 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module virt:rhel:820240314161907:9edba152.x86_64 from CentOS-AppStream 2024-04-12T20:41:43.371 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 37: conflicting requests 2024-04-12T20:41:43.371 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:8030020200716155257:7cc0a66d.x86_64 from CentOS-AppStream 2024-04-12T20:41:43.371 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:8030020200716155257:b967a9a2.x86_64 from CentOS-AppStream 2024-04-12T20:41:43.371 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:820230811133638:36f9fae6.x86_64 from CentOS-AppStream 2024-04-12T20:41:43.371 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 38: cannot install the best candidate for the job 2024-04-12T20:41:43.371 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:8010020191114031501:a5949e2e.x86_64 from CentOS-AppStream 2024-04-12T20:41:43.372 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:8010020191114031501:be2e4737.x86_64 from CentOS-AppStream 2024-04-12T20:41:43.372 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:820230223114759:2ab761e1.x86_64 from CentOS-AppStream 2024-04-12T20:41:43.372 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:820230223114759:86e45846.x86_64 from CentOS-AppStream 2024-04-12T20:41:43.372 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 39: conflicting requests 2024-04-12T20:41:43.372 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:8030020200715230104:1e4bbb35.x86_64 from CentOS-AppStream 2024-04-12T20:41:43.372 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:8030020200715230104:ea09926d.x86_64 from CentOS-AppStream 2024-04-12T20:41:43.372 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:082fdf2f.x86_64 from CentOS-AppStream 2024-04-12T20:41:43.372 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-04-12T20:41:43.372 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-04-12T20:41:43.372 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:fbe42456.x86_64 from CentOS-AppStream 2024-04-12T20:41:43.372 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 40: cannot install the best candidate for the job 2024-04-12T20:41:43.372 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:8010020191114031513:16b3ab4d.x86_64 from CentOS-AppStream 2024-04-12T20:41:43.372 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:8010020191114031513:e1f37755.x86_64 from CentOS-AppStream 2024-04-12T20:41:43.372 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-04-12T20:41:43.373 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-04-12T20:41:43.373 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 41: cannot install the best candidate for the job 2024-04-12T20:41:43.373 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:8030020200716150652:1e4bbb35.x86_64 from CentOS-AppStream 2024-04-12T20:41:43.373 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:8030020200716150652:ea09926d.x86_64 from CentOS-AppStream 2024-04-12T20:41:43.373 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-04-12T20:41:43.373 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-04-12T20:41:43.373 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 42: cannot install the best candidate for the job 2024-04-12T20:41:43.373 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:8030020200716174729:3a70019f.x86_64 from CentOS-AppStream 2024-04-12T20:41:43.373 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:8030020200716174729:cccafca5.x86_64 from CentOS-AppStream 2024-04-12T20:41:43.373 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:820230223114759:6fdf7e5d.x86_64 from CentOS-AppStream 2024-04-12T20:41:43.373 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:820230223114759:d4326aba.x86_64 from CentOS-AppStream 2024-04-12T20:41:43.373 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 43: cannot install the best candidate for the job 2024-04-12T20:41:43.373 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:8030020200716213634:56fce90f.x86_64 from CentOS-AppStream 2024-04-12T20:41:43.373 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:8030020200716213634:98a3c9d0.x86_64 from CentOS-AppStream 2024-04-12T20:41:43.374 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:820230223114759:06b7596e.x86_64 from CentOS-AppStream 2024-04-12T20:41:43.374 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:820230223114759:d2fc5d76.x86_64 from CentOS-AppStream 2024-04-12T20:41:43.374 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 44: cannot install the best candidate for the job 2024-04-12T20:41:43.374 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:8030020200716171139:3a70019f.x86_64 from CentOS-AppStream 2024-04-12T20:41:43.374 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:8030020200716171139:cccafca5.x86_64 from CentOS-AppStream 2024-04-12T20:41:43.374 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:820230223114759:6fdf7e5d.x86_64 from CentOS-AppStream 2024-04-12T20:41:43.374 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:820230223114759:d4326aba.x86_64 from CentOS-AppStream 2024-04-12T20:41:43.374 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 45: cannot install the best candidate for the job 2024-04-12T20:41:43.374 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:8010020191120175858:95fce66d.x86_64 from CentOS-AppStream 2024-04-12T20:41:43.374 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:8030020200723233049:09acf126.x86_64 from CentOS-AppStream 2024-04-12T20:41:43.374 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:820230223114759:436af1ae.x86_64 from CentOS-AppStream 2024-04-12T20:41:43.374 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:820230223114759:b12195ea.x86_64 from CentOS-AppStream 2024-04-12T20:41:43.375 INFO:teuthology.orchestra.run.smithi186.stdout:Package python3-rgw-2:17.2.7-827.gd3b5060a.el9.x86_64 is already installed. 2024-04-12T20:41:43.440 INFO:teuthology.orchestra.run.smithi186.stdout:Dependencies resolved. 2024-04-12T20:41:43.441 INFO:teuthology.orchestra.run.smithi186.stdout:Nothing to do. 2024-04-12T20:41:43.441 INFO:teuthology.orchestra.run.smithi186.stdout:Complete! 2024-04-12T20:41:43.531 DEBUG:teuthology.orchestra.run.smithi186:> sudo yum -y install python3-cephfs 2024-04-12T20:41:43.644 INFO:teuthology.orchestra.run.smithi018.stderr:Modular dependency problems: 2024-04-12T20:41:43.644 INFO:teuthology.orchestra.run.smithi018.stderr: 2024-04-12T20:41:43.644 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 1: conflicting requests 2024-04-12T20:41:43.645 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module container-tools:rhel8:820240321091303:20125149.x86_64 from CentOS-AppStream 2024-04-12T20:41:43.645 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 2: conflicting requests 2024-04-12T20:41:43.645 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module eclipse:rhel8:820201023100746:b230ed4e.x86_64 from CentOS-AppStream 2024-04-12T20:41:43.645 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 3: conflicting requests 2024-04-12T20:41:43.645 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module freeradius:3.0:8080020230117180605:89170a74.x86_64 from CentOS-AppStream 2024-04-12T20:41:43.645 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 4: conflicting requests 2024-04-12T20:41:43.645 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module gimp:2.8:8000020190628145146:4148dfdf.x86_64 from CentOS-AppStream 2024-04-12T20:41:43.645 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 5: conflicting requests 2024-04-12T20:41:43.645 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module go-toolset:rhel8:820240401160413:b754926a.x86_64 from CentOS-AppStream 2024-04-12T20:41:43.645 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 6: conflicting requests 2024-04-12T20:41:43.645 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module httpd:2.4:820240216084734:9edba152.x86_64 from CentOS-AppStream 2024-04-12T20:41:43.645 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 7: conflicting requests 2024-04-12T20:41:43.645 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module idm:client:820240307184541:49cc9d1b.x86_64 from CentOS-AppStream 2024-04-12T20:41:43.645 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 8: conflicting requests 2024-04-12T20:41:43.646 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module inkscape:0.92.3:8080020221205124429:3e031279.x86_64 from CentOS-AppStream 2024-04-12T20:41:43.646 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 9: conflicting requests 2024-04-12T20:41:43.646 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module jmc:rhel8:8050020211110222101:6392b1f8.x86_64 from CentOS-AppStream 2024-04-12T20:41:43.646 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 10: conflicting requests 2024-04-12T20:41:43.646 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module llvm-toolset:rhel8:820240205133135:9edba152.x86_64 from CentOS-AppStream 2024-04-12T20:41:43.646 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 11: conflicting requests 2024-04-12T20:41:43.646 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module log4j:2:8080020221205124743:9d367344.x86_64 from CentOS-AppStream 2024-04-12T20:41:43.646 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 12: conflicting requests 2024-04-12T20:41:43.646 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module mailman:2.1:820230727085757:77fc8825.x86_64 from CentOS-AppStream 2024-04-12T20:41:43.646 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 13: conflicting requests 2024-04-12T20:41:43.646 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module mariadb:10.3:8030020210419150013:30b713e6.x86_64 from CentOS-AppStream 2024-04-12T20:41:43.646 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 14: conflicting requests 2024-04-12T20:41:43.646 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module maven:3.5:8060020211203142015:c0229ad2.x86_64 from CentOS-AppStream 2024-04-12T20:41:43.647 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 15: conflicting requests 2024-04-12T20:41:43.647 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module mercurial:4.8:8000020190628020724:4148dfdf.x86_64 from CentOS-AppStream 2024-04-12T20:41:43.647 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 16: conflicting requests 2024-04-12T20:41:43.647 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module mod_auth_openidc:2.3:820230721123832:9edba152.x86_64 from CentOS-AppStream 2024-04-12T20:41:43.647 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 17: conflicting requests 2024-04-12T20:41:43.647 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module mysql:8.0:8040020210901180257:9f9e2e7e.x86_64 from CentOS-AppStream 2024-04-12T20:41:43.647 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 18: conflicting requests 2024-04-12T20:41:43.647 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module nginx:1.14:8000020211221191913:55190bc5.x86_64 from CentOS-AppStream 2024-04-12T20:41:43.647 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 19: conflicting requests 2024-04-12T20:41:43.647 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module nodejs:10:8040020210120182536:9f9e2e7e.x86_64 from CentOS-AppStream 2024-04-12T20:41:43.647 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 20: conflicting requests 2024-04-12T20:41:43.647 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl:5.26:8000020190628020724:55190bc5.x86_64 from CentOS-AppStream 2024-04-12T20:41:43.647 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 21: conflicting requests 2024-04-12T20:41:43.647 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module php:7.2:8020020200507003613:2c7ca891.x86_64 from CentOS-AppStream 2024-04-12T20:41:43.648 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 22: conflicting requests 2024-04-12T20:41:43.648 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module postgresql:10:820230223114800:9edba152.x86_64 from CentOS-AppStream 2024-04-12T20:41:43.648 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 23: conflicting requests 2024-04-12T20:41:43.648 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module python27:2.7:820240208011952:182f7c73.x86_64 from CentOS-AppStream 2024-04-12T20:41:43.648 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 24: conflicting requests 2024-04-12T20:41:43.648 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module python36:3.6:8050020210825152031:982725ab.x86_64 from CentOS-AppStream 2024-04-12T20:41:43.648 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 25: conflicting requests 2024-04-12T20:41:43.648 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module python38:3.8:820230810143931:64c1cd5a.x86_64 from CentOS-AppStream 2024-04-12T20:41:43.648 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 26: conflicting requests 2024-04-12T20:41:43.648 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module python39:3.9:820240214182535:17377f89.x86_64 from CentOS-AppStream 2024-04-12T20:41:43.648 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 27: conflicting requests 2024-04-12T20:41:43.648 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module redis:5:8040020211019153849:9f9e2e7e.x86_64 from CentOS-AppStream 2024-04-12T20:41:43.648 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 28: conflicting requests 2024-04-12T20:41:43.648 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module rhn-tools:1.0:8010020191114034948:f69d1239.x86_64 from CentOS-AppStream 2024-04-12T20:41:43.648 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 29: conflicting requests 2024-04-12T20:41:43.649 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module ruby:2.5:820230627084142:9edba152.x86_64 from CentOS-AppStream 2024-04-12T20:41:43.649 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 30: conflicting requests 2024-04-12T20:41:43.649 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module rust-toolset:rhel8:820240119204620:b09eea91.x86_64 from CentOS-AppStream 2024-04-12T20:41:43.649 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 31: conflicting requests 2024-04-12T20:41:43.649 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module satellite-5-client:1.0:8010020191114035551:cdc1202b.x86_64 from CentOS-AppStream 2024-04-12T20:41:43.649 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 32: conflicting requests 2024-04-12T20:41:43.649 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module squid:4:820240319173245:9edba152.x86_64 from CentOS-AppStream 2024-04-12T20:41:43.649 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 33: conflicting requests 2024-04-12T20:41:43.649 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module subversion:1.10:8070020220711155714:78111232.x86_64 from CentOS-AppStream 2024-04-12T20:41:43.649 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 34: conflicting requests 2024-04-12T20:41:43.649 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module swig:3.0:8030020200706142531:30b713e6.x86_64 from CentOS-AppStream 2024-04-12T20:41:43.649 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 35: conflicting requests 2024-04-12T20:41:43.649 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module varnish:6:8050020211109225449:b4937e53.x86_64 from CentOS-AppStream 2024-04-12T20:41:43.649 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 36: conflicting requests 2024-04-12T20:41:43.650 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module virt:rhel:820240314161907:9edba152.x86_64 from CentOS-AppStream 2024-04-12T20:41:43.650 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 37: conflicting requests 2024-04-12T20:41:43.650 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:8030020200716155257:7cc0a66d.x86_64 from CentOS-AppStream 2024-04-12T20:41:43.650 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:8030020200716155257:b967a9a2.x86_64 from CentOS-AppStream 2024-04-12T20:41:43.650 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:820230811133638:36f9fae6.x86_64 from CentOS-AppStream 2024-04-12T20:41:43.650 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 38: cannot install the best candidate for the job 2024-04-12T20:41:43.650 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:8010020191114031501:a5949e2e.x86_64 from CentOS-AppStream 2024-04-12T20:41:43.650 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:8010020191114031501:be2e4737.x86_64 from CentOS-AppStream 2024-04-12T20:41:43.650 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:820230223114759:2ab761e1.x86_64 from CentOS-AppStream 2024-04-12T20:41:43.650 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:820230223114759:86e45846.x86_64 from CentOS-AppStream 2024-04-12T20:41:43.650 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 39: conflicting requests 2024-04-12T20:41:43.650 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:8030020200715230104:1e4bbb35.x86_64 from CentOS-AppStream 2024-04-12T20:41:43.650 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:8030020200715230104:ea09926d.x86_64 from CentOS-AppStream 2024-04-12T20:41:43.650 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:082fdf2f.x86_64 from CentOS-AppStream 2024-04-12T20:41:43.651 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-04-12T20:41:43.651 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-04-12T20:41:43.651 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:fbe42456.x86_64 from CentOS-AppStream 2024-04-12T20:41:43.651 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 40: cannot install the best candidate for the job 2024-04-12T20:41:43.651 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:8010020191114031513:16b3ab4d.x86_64 from CentOS-AppStream 2024-04-12T20:41:43.651 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:8010020191114031513:e1f37755.x86_64 from CentOS-AppStream 2024-04-12T20:41:43.651 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-04-12T20:41:43.651 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-04-12T20:41:43.651 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 41: cannot install the best candidate for the job 2024-04-12T20:41:43.651 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:8030020200716150652:1e4bbb35.x86_64 from CentOS-AppStream 2024-04-12T20:41:43.651 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:8030020200716150652:ea09926d.x86_64 from CentOS-AppStream 2024-04-12T20:41:43.651 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-04-12T20:41:43.651 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-04-12T20:41:43.651 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 42: cannot install the best candidate for the job 2024-04-12T20:41:43.651 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:8030020200716174729:3a70019f.x86_64 from CentOS-AppStream 2024-04-12T20:41:43.652 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:8030020200716174729:cccafca5.x86_64 from CentOS-AppStream 2024-04-12T20:41:43.652 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:820230223114759:6fdf7e5d.x86_64 from CentOS-AppStream 2024-04-12T20:41:43.652 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:820230223114759:d4326aba.x86_64 from CentOS-AppStream 2024-04-12T20:41:43.652 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 43: cannot install the best candidate for the job 2024-04-12T20:41:43.652 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:8030020200716213634:56fce90f.x86_64 from CentOS-AppStream 2024-04-12T20:41:43.652 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:8030020200716213634:98a3c9d0.x86_64 from CentOS-AppStream 2024-04-12T20:41:43.652 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:820230223114759:06b7596e.x86_64 from CentOS-AppStream 2024-04-12T20:41:43.652 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:820230223114759:d2fc5d76.x86_64 from CentOS-AppStream 2024-04-12T20:41:43.652 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 44: cannot install the best candidate for the job 2024-04-12T20:41:43.652 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:8030020200716171139:3a70019f.x86_64 from CentOS-AppStream 2024-04-12T20:41:43.652 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:8030020200716171139:cccafca5.x86_64 from CentOS-AppStream 2024-04-12T20:41:43.652 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:820230223114759:6fdf7e5d.x86_64 from CentOS-AppStream 2024-04-12T20:41:43.652 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:820230223114759:d4326aba.x86_64 from CentOS-AppStream 2024-04-12T20:41:43.652 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 45: cannot install the best candidate for the job 2024-04-12T20:41:43.652 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:8010020191120175858:95fce66d.x86_64 from CentOS-AppStream 2024-04-12T20:41:43.653 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:8030020200723233049:09acf126.x86_64 from CentOS-AppStream 2024-04-12T20:41:43.653 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:820230223114759:436af1ae.x86_64 from CentOS-AppStream 2024-04-12T20:41:43.653 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:820230223114759:b12195ea.x86_64 from CentOS-AppStream 2024-04-12T20:41:43.653 INFO:teuthology.orchestra.run.smithi018.stdout:Package python3-rgw-2:17.2.7-827.gd3b5060a.el9.x86_64 is already installed. 2024-04-12T20:41:43.723 INFO:teuthology.orchestra.run.smithi018.stdout:Dependencies resolved. 2024-04-12T20:41:43.724 INFO:teuthology.orchestra.run.smithi018.stdout:Nothing to do. 2024-04-12T20:41:43.724 INFO:teuthology.orchestra.run.smithi018.stdout:Complete! 2024-04-12T20:41:43.804 DEBUG:teuthology.orchestra.run.smithi018:> sudo yum -y install python3-cephfs 2024-04-12T20:41:44.416 INFO:teuthology.orchestra.run.smithi186.stdout:Last metadata expiration check: 0:02:07 ago on Fri 12 Apr 2024 08:39:37 PM UTC. 2024-04-12T20:41:44.717 INFO:teuthology.orchestra.run.smithi186.stderr:Modular dependency problems: 2024-04-12T20:41:44.717 INFO:teuthology.orchestra.run.smithi186.stderr: 2024-04-12T20:41:44.717 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 1: conflicting requests 2024-04-12T20:41:44.717 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module container-tools:rhel8:820240321091303:20125149.x86_64 from CentOS-AppStream 2024-04-12T20:41:44.717 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 2: conflicting requests 2024-04-12T20:41:44.717 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module eclipse:rhel8:820201023100746:b230ed4e.x86_64 from CentOS-AppStream 2024-04-12T20:41:44.717 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 3: conflicting requests 2024-04-12T20:41:44.718 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module freeradius:3.0:8080020230117180605:89170a74.x86_64 from CentOS-AppStream 2024-04-12T20:41:44.718 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 4: conflicting requests 2024-04-12T20:41:44.718 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module gimp:2.8:8000020190628145146:4148dfdf.x86_64 from CentOS-AppStream 2024-04-12T20:41:44.718 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 5: conflicting requests 2024-04-12T20:41:44.718 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module go-toolset:rhel8:820240401160413:b754926a.x86_64 from CentOS-AppStream 2024-04-12T20:41:44.718 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 6: conflicting requests 2024-04-12T20:41:44.718 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module httpd:2.4:820240216084734:9edba152.x86_64 from CentOS-AppStream 2024-04-12T20:41:44.718 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 7: conflicting requests 2024-04-12T20:41:44.718 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module idm:client:820240307184541:49cc9d1b.x86_64 from CentOS-AppStream 2024-04-12T20:41:44.718 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 8: conflicting requests 2024-04-12T20:41:44.718 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module inkscape:0.92.3:8080020221205124429:3e031279.x86_64 from CentOS-AppStream 2024-04-12T20:41:44.718 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 9: conflicting requests 2024-04-12T20:41:44.718 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module jmc:rhel8:8050020211110222101:6392b1f8.x86_64 from CentOS-AppStream 2024-04-12T20:41:44.719 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 10: conflicting requests 2024-04-12T20:41:44.719 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module llvm-toolset:rhel8:820240205133135:9edba152.x86_64 from CentOS-AppStream 2024-04-12T20:41:44.719 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 11: conflicting requests 2024-04-12T20:41:44.719 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module log4j:2:8080020221205124743:9d367344.x86_64 from CentOS-AppStream 2024-04-12T20:41:44.719 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 12: conflicting requests 2024-04-12T20:41:44.719 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module mailman:2.1:820230727085757:77fc8825.x86_64 from CentOS-AppStream 2024-04-12T20:41:44.719 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 13: conflicting requests 2024-04-12T20:41:44.719 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module mariadb:10.3:8030020210419150013:30b713e6.x86_64 from CentOS-AppStream 2024-04-12T20:41:44.719 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 14: conflicting requests 2024-04-12T20:41:44.719 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module maven:3.5:8060020211203142015:c0229ad2.x86_64 from CentOS-AppStream 2024-04-12T20:41:44.719 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 15: conflicting requests 2024-04-12T20:41:44.719 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module mercurial:4.8:8000020190628020724:4148dfdf.x86_64 from CentOS-AppStream 2024-04-12T20:41:44.719 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 16: conflicting requests 2024-04-12T20:41:44.719 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module mod_auth_openidc:2.3:820230721123832:9edba152.x86_64 from CentOS-AppStream 2024-04-12T20:41:44.720 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 17: conflicting requests 2024-04-12T20:41:44.720 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module mysql:8.0:8040020210901180257:9f9e2e7e.x86_64 from CentOS-AppStream 2024-04-12T20:41:44.720 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 18: conflicting requests 2024-04-12T20:41:44.720 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module nginx:1.14:8000020211221191913:55190bc5.x86_64 from CentOS-AppStream 2024-04-12T20:41:44.720 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 19: conflicting requests 2024-04-12T20:41:44.720 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module nodejs:10:8040020210120182536:9f9e2e7e.x86_64 from CentOS-AppStream 2024-04-12T20:41:44.720 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 20: conflicting requests 2024-04-12T20:41:44.720 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl:5.26:8000020190628020724:55190bc5.x86_64 from CentOS-AppStream 2024-04-12T20:41:44.720 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 21: conflicting requests 2024-04-12T20:41:44.720 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module php:7.2:8020020200507003613:2c7ca891.x86_64 from CentOS-AppStream 2024-04-12T20:41:44.720 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 22: conflicting requests 2024-04-12T20:41:44.720 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module postgresql:10:820230223114800:9edba152.x86_64 from CentOS-AppStream 2024-04-12T20:41:44.720 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 23: conflicting requests 2024-04-12T20:41:44.720 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module python27:2.7:820240208011952:182f7c73.x86_64 from CentOS-AppStream 2024-04-12T20:41:44.721 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 24: conflicting requests 2024-04-12T20:41:44.721 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module python36:3.6:8050020210825152031:982725ab.x86_64 from CentOS-AppStream 2024-04-12T20:41:44.721 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 25: conflicting requests 2024-04-12T20:41:44.721 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module python38:3.8:820230810143931:64c1cd5a.x86_64 from CentOS-AppStream 2024-04-12T20:41:44.721 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 26: conflicting requests 2024-04-12T20:41:44.721 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module python39:3.9:820240214182535:17377f89.x86_64 from CentOS-AppStream 2024-04-12T20:41:44.721 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 27: conflicting requests 2024-04-12T20:41:44.721 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module redis:5:8040020211019153849:9f9e2e7e.x86_64 from CentOS-AppStream 2024-04-12T20:41:44.721 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 28: conflicting requests 2024-04-12T20:41:44.721 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module rhn-tools:1.0:8010020191114034948:f69d1239.x86_64 from CentOS-AppStream 2024-04-12T20:41:44.721 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 29: conflicting requests 2024-04-12T20:41:44.721 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module ruby:2.5:820230627084142:9edba152.x86_64 from CentOS-AppStream 2024-04-12T20:41:44.721 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 30: conflicting requests 2024-04-12T20:41:44.721 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module rust-toolset:rhel8:820240119204620:b09eea91.x86_64 from CentOS-AppStream 2024-04-12T20:41:44.721 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 31: conflicting requests 2024-04-12T20:41:44.722 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module satellite-5-client:1.0:8010020191114035551:cdc1202b.x86_64 from CentOS-AppStream 2024-04-12T20:41:44.722 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 32: conflicting requests 2024-04-12T20:41:44.722 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module squid:4:820240319173245:9edba152.x86_64 from CentOS-AppStream 2024-04-12T20:41:44.722 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 33: conflicting requests 2024-04-12T20:41:44.722 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module subversion:1.10:8070020220711155714:78111232.x86_64 from CentOS-AppStream 2024-04-12T20:41:44.722 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 34: conflicting requests 2024-04-12T20:41:44.722 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module swig:3.0:8030020200706142531:30b713e6.x86_64 from CentOS-AppStream 2024-04-12T20:41:44.722 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 35: conflicting requests 2024-04-12T20:41:44.722 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module varnish:6:8050020211109225449:b4937e53.x86_64 from CentOS-AppStream 2024-04-12T20:41:44.722 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 36: conflicting requests 2024-04-12T20:41:44.722 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module virt:rhel:820240314161907:9edba152.x86_64 from CentOS-AppStream 2024-04-12T20:41:44.722 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 37: conflicting requests 2024-04-12T20:41:44.722 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:8030020200716155257:7cc0a66d.x86_64 from CentOS-AppStream 2024-04-12T20:41:44.722 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:8030020200716155257:b967a9a2.x86_64 from CentOS-AppStream 2024-04-12T20:41:44.723 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:820230811133638:36f9fae6.x86_64 from CentOS-AppStream 2024-04-12T20:41:44.723 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 38: cannot install the best candidate for the job 2024-04-12T20:41:44.723 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:8010020191114031501:a5949e2e.x86_64 from CentOS-AppStream 2024-04-12T20:41:44.723 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:8010020191114031501:be2e4737.x86_64 from CentOS-AppStream 2024-04-12T20:41:44.723 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:820230223114759:2ab761e1.x86_64 from CentOS-AppStream 2024-04-12T20:41:44.723 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:820230223114759:86e45846.x86_64 from CentOS-AppStream 2024-04-12T20:41:44.723 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 39: conflicting requests 2024-04-12T20:41:44.723 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:8030020200715230104:1e4bbb35.x86_64 from CentOS-AppStream 2024-04-12T20:41:44.723 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:8030020200715230104:ea09926d.x86_64 from CentOS-AppStream 2024-04-12T20:41:44.723 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:082fdf2f.x86_64 from CentOS-AppStream 2024-04-12T20:41:44.723 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-04-12T20:41:44.723 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-04-12T20:41:44.723 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:fbe42456.x86_64 from CentOS-AppStream 2024-04-12T20:41:44.723 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 40: cannot install the best candidate for the job 2024-04-12T20:41:44.724 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:8010020191114031513:16b3ab4d.x86_64 from CentOS-AppStream 2024-04-12T20:41:44.724 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:8010020191114031513:e1f37755.x86_64 from CentOS-AppStream 2024-04-12T20:41:44.724 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-04-12T20:41:44.724 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-04-12T20:41:44.724 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 41: cannot install the best candidate for the job 2024-04-12T20:41:44.724 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:8030020200716150652:1e4bbb35.x86_64 from CentOS-AppStream 2024-04-12T20:41:44.724 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:8030020200716150652:ea09926d.x86_64 from CentOS-AppStream 2024-04-12T20:41:44.724 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-04-12T20:41:44.724 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-04-12T20:41:44.724 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 42: cannot install the best candidate for the job 2024-04-12T20:41:44.724 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:8030020200716174729:3a70019f.x86_64 from CentOS-AppStream 2024-04-12T20:41:44.724 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:8030020200716174729:cccafca5.x86_64 from CentOS-AppStream 2024-04-12T20:41:44.724 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:820230223114759:6fdf7e5d.x86_64 from CentOS-AppStream 2024-04-12T20:41:44.724 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:820230223114759:d4326aba.x86_64 from CentOS-AppStream 2024-04-12T20:41:44.725 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 43: cannot install the best candidate for the job 2024-04-12T20:41:44.725 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:8030020200716213634:56fce90f.x86_64 from CentOS-AppStream 2024-04-12T20:41:44.725 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:8030020200716213634:98a3c9d0.x86_64 from CentOS-AppStream 2024-04-12T20:41:44.725 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:820230223114759:06b7596e.x86_64 from CentOS-AppStream 2024-04-12T20:41:44.725 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:820230223114759:d2fc5d76.x86_64 from CentOS-AppStream 2024-04-12T20:41:44.725 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 44: cannot install the best candidate for the job 2024-04-12T20:41:44.725 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:8030020200716171139:3a70019f.x86_64 from CentOS-AppStream 2024-04-12T20:41:44.725 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:8030020200716171139:cccafca5.x86_64 from CentOS-AppStream 2024-04-12T20:41:44.725 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:820230223114759:6fdf7e5d.x86_64 from CentOS-AppStream 2024-04-12T20:41:44.725 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:820230223114759:d4326aba.x86_64 from CentOS-AppStream 2024-04-12T20:41:44.725 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 45: cannot install the best candidate for the job 2024-04-12T20:41:44.725 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:8010020191120175858:95fce66d.x86_64 from CentOS-AppStream 2024-04-12T20:41:44.725 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:8030020200723233049:09acf126.x86_64 from CentOS-AppStream 2024-04-12T20:41:44.725 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:820230223114759:436af1ae.x86_64 from CentOS-AppStream 2024-04-12T20:41:44.726 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:820230223114759:b12195ea.x86_64 from CentOS-AppStream 2024-04-12T20:41:44.726 INFO:teuthology.orchestra.run.smithi186.stdout:Package python3-cephfs-2:17.2.7-827.gd3b5060a.el9.x86_64 is already installed. 2024-04-12T20:41:44.741 INFO:teuthology.orchestra.run.smithi018.stdout:Last metadata expiration check: 0:02:09 ago on Fri 12 Apr 2024 08:39:35 PM UTC. 2024-04-12T20:41:44.791 INFO:teuthology.orchestra.run.smithi186.stdout:Dependencies resolved. 2024-04-12T20:41:44.792 INFO:teuthology.orchestra.run.smithi186.stdout:Nothing to do. 2024-04-12T20:41:44.792 INFO:teuthology.orchestra.run.smithi186.stdout:Complete! 2024-04-12T20:41:44.861 DEBUG:teuthology.orchestra.run.smithi186:> sudo yum -y install python3-rbd 2024-04-12T20:41:45.054 INFO:teuthology.orchestra.run.smithi018.stderr:Modular dependency problems: 2024-04-12T20:41:45.054 INFO:teuthology.orchestra.run.smithi018.stderr: 2024-04-12T20:41:45.054 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 1: conflicting requests 2024-04-12T20:41:45.054 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module container-tools:rhel8:820240321091303:20125149.x86_64 from CentOS-AppStream 2024-04-12T20:41:45.054 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 2: conflicting requests 2024-04-12T20:41:45.054 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module eclipse:rhel8:820201023100746:b230ed4e.x86_64 from CentOS-AppStream 2024-04-12T20:41:45.054 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 3: conflicting requests 2024-04-12T20:41:45.055 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module freeradius:3.0:8080020230117180605:89170a74.x86_64 from CentOS-AppStream 2024-04-12T20:41:45.055 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 4: conflicting requests 2024-04-12T20:41:45.055 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module gimp:2.8:8000020190628145146:4148dfdf.x86_64 from CentOS-AppStream 2024-04-12T20:41:45.055 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 5: conflicting requests 2024-04-12T20:41:45.055 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module go-toolset:rhel8:820240401160413:b754926a.x86_64 from CentOS-AppStream 2024-04-12T20:41:45.055 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 6: conflicting requests 2024-04-12T20:41:45.055 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module httpd:2.4:820240216084734:9edba152.x86_64 from CentOS-AppStream 2024-04-12T20:41:45.055 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 7: conflicting requests 2024-04-12T20:41:45.055 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module idm:client:820240307184541:49cc9d1b.x86_64 from CentOS-AppStream 2024-04-12T20:41:45.055 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 8: conflicting requests 2024-04-12T20:41:45.056 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module inkscape:0.92.3:8080020221205124429:3e031279.x86_64 from CentOS-AppStream 2024-04-12T20:41:45.056 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 9: conflicting requests 2024-04-12T20:41:45.056 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module jmc:rhel8:8050020211110222101:6392b1f8.x86_64 from CentOS-AppStream 2024-04-12T20:41:45.056 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 10: conflicting requests 2024-04-12T20:41:45.056 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module llvm-toolset:rhel8:820240205133135:9edba152.x86_64 from CentOS-AppStream 2024-04-12T20:41:45.056 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 11: conflicting requests 2024-04-12T20:41:45.056 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module log4j:2:8080020221205124743:9d367344.x86_64 from CentOS-AppStream 2024-04-12T20:41:45.056 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 12: conflicting requests 2024-04-12T20:41:45.056 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module mailman:2.1:820230727085757:77fc8825.x86_64 from CentOS-AppStream 2024-04-12T20:41:45.056 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 13: conflicting requests 2024-04-12T20:41:45.056 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module mariadb:10.3:8030020210419150013:30b713e6.x86_64 from CentOS-AppStream 2024-04-12T20:41:45.056 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 14: conflicting requests 2024-04-12T20:41:45.056 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module maven:3.5:8060020211203142015:c0229ad2.x86_64 from CentOS-AppStream 2024-04-12T20:41:45.057 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 15: conflicting requests 2024-04-12T20:41:45.057 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module mercurial:4.8:8000020190628020724:4148dfdf.x86_64 from CentOS-AppStream 2024-04-12T20:41:45.057 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 16: conflicting requests 2024-04-12T20:41:45.057 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module mod_auth_openidc:2.3:820230721123832:9edba152.x86_64 from CentOS-AppStream 2024-04-12T20:41:45.057 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 17: conflicting requests 2024-04-12T20:41:45.057 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module mysql:8.0:8040020210901180257:9f9e2e7e.x86_64 from CentOS-AppStream 2024-04-12T20:41:45.057 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 18: conflicting requests 2024-04-12T20:41:45.057 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module nginx:1.14:8000020211221191913:55190bc5.x86_64 from CentOS-AppStream 2024-04-12T20:41:45.057 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 19: conflicting requests 2024-04-12T20:41:45.057 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module nodejs:10:8040020210120182536:9f9e2e7e.x86_64 from CentOS-AppStream 2024-04-12T20:41:45.057 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 20: conflicting requests 2024-04-12T20:41:45.057 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl:5.26:8000020190628020724:55190bc5.x86_64 from CentOS-AppStream 2024-04-12T20:41:45.057 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 21: conflicting requests 2024-04-12T20:41:45.057 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module php:7.2:8020020200507003613:2c7ca891.x86_64 from CentOS-AppStream 2024-04-12T20:41:45.058 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 22: conflicting requests 2024-04-12T20:41:45.058 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module postgresql:10:820230223114800:9edba152.x86_64 from CentOS-AppStream 2024-04-12T20:41:45.058 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 23: conflicting requests 2024-04-12T20:41:45.058 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module python27:2.7:820240208011952:182f7c73.x86_64 from CentOS-AppStream 2024-04-12T20:41:45.058 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 24: conflicting requests 2024-04-12T20:41:45.058 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module python36:3.6:8050020210825152031:982725ab.x86_64 from CentOS-AppStream 2024-04-12T20:41:45.058 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 25: conflicting requests 2024-04-12T20:41:45.058 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module python38:3.8:820230810143931:64c1cd5a.x86_64 from CentOS-AppStream 2024-04-12T20:41:45.058 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 26: conflicting requests 2024-04-12T20:41:45.058 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module python39:3.9:820240214182535:17377f89.x86_64 from CentOS-AppStream 2024-04-12T20:41:45.058 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 27: conflicting requests 2024-04-12T20:41:45.058 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module redis:5:8040020211019153849:9f9e2e7e.x86_64 from CentOS-AppStream 2024-04-12T20:41:45.058 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 28: conflicting requests 2024-04-12T20:41:45.059 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module rhn-tools:1.0:8010020191114034948:f69d1239.x86_64 from CentOS-AppStream 2024-04-12T20:41:45.059 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 29: conflicting requests 2024-04-12T20:41:45.059 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module ruby:2.5:820230627084142:9edba152.x86_64 from CentOS-AppStream 2024-04-12T20:41:45.059 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 30: conflicting requests 2024-04-12T20:41:45.059 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module rust-toolset:rhel8:820240119204620:b09eea91.x86_64 from CentOS-AppStream 2024-04-12T20:41:45.059 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 31: conflicting requests 2024-04-12T20:41:45.059 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module satellite-5-client:1.0:8010020191114035551:cdc1202b.x86_64 from CentOS-AppStream 2024-04-12T20:41:45.059 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 32: conflicting requests 2024-04-12T20:41:45.059 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module squid:4:820240319173245:9edba152.x86_64 from CentOS-AppStream 2024-04-12T20:41:45.059 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 33: conflicting requests 2024-04-12T20:41:45.060 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module subversion:1.10:8070020220711155714:78111232.x86_64 from CentOS-AppStream 2024-04-12T20:41:45.060 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 34: conflicting requests 2024-04-12T20:41:45.060 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module swig:3.0:8030020200706142531:30b713e6.x86_64 from CentOS-AppStream 2024-04-12T20:41:45.060 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 35: conflicting requests 2024-04-12T20:41:45.060 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module varnish:6:8050020211109225449:b4937e53.x86_64 from CentOS-AppStream 2024-04-12T20:41:45.060 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 36: conflicting requests 2024-04-12T20:41:45.060 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module virt:rhel:820240314161907:9edba152.x86_64 from CentOS-AppStream 2024-04-12T20:41:45.060 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 37: conflicting requests 2024-04-12T20:41:45.060 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:8030020200716155257:7cc0a66d.x86_64 from CentOS-AppStream 2024-04-12T20:41:45.060 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:8030020200716155257:b967a9a2.x86_64 from CentOS-AppStream 2024-04-12T20:41:45.060 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:820230811133638:36f9fae6.x86_64 from CentOS-AppStream 2024-04-12T20:41:45.060 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 38: cannot install the best candidate for the job 2024-04-12T20:41:45.060 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:8010020191114031501:a5949e2e.x86_64 from CentOS-AppStream 2024-04-12T20:41:45.061 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:8010020191114031501:be2e4737.x86_64 from CentOS-AppStream 2024-04-12T20:41:45.061 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:820230223114759:2ab761e1.x86_64 from CentOS-AppStream 2024-04-12T20:41:45.061 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:820230223114759:86e45846.x86_64 from CentOS-AppStream 2024-04-12T20:41:45.061 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 39: conflicting requests 2024-04-12T20:41:45.061 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:8030020200715230104:1e4bbb35.x86_64 from CentOS-AppStream 2024-04-12T20:41:45.061 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:8030020200715230104:ea09926d.x86_64 from CentOS-AppStream 2024-04-12T20:41:45.061 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:082fdf2f.x86_64 from CentOS-AppStream 2024-04-12T20:41:45.061 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-04-12T20:41:45.061 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-04-12T20:41:45.061 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:fbe42456.x86_64 from CentOS-AppStream 2024-04-12T20:41:45.061 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 40: cannot install the best candidate for the job 2024-04-12T20:41:45.061 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:8010020191114031513:16b3ab4d.x86_64 from CentOS-AppStream 2024-04-12T20:41:45.061 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:8010020191114031513:e1f37755.x86_64 from CentOS-AppStream 2024-04-12T20:41:45.061 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-04-12T20:41:45.062 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-04-12T20:41:45.062 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 41: cannot install the best candidate for the job 2024-04-12T20:41:45.062 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:8030020200716150652:1e4bbb35.x86_64 from CentOS-AppStream 2024-04-12T20:41:45.062 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:8030020200716150652:ea09926d.x86_64 from CentOS-AppStream 2024-04-12T20:41:45.062 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-04-12T20:41:45.062 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-04-12T20:41:45.062 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 42: cannot install the best candidate for the job 2024-04-12T20:41:45.062 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:8030020200716174729:3a70019f.x86_64 from CentOS-AppStream 2024-04-12T20:41:45.062 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:8030020200716174729:cccafca5.x86_64 from CentOS-AppStream 2024-04-12T20:41:45.062 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:820230223114759:6fdf7e5d.x86_64 from CentOS-AppStream 2024-04-12T20:41:45.062 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:820230223114759:d4326aba.x86_64 from CentOS-AppStream 2024-04-12T20:41:45.062 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 43: cannot install the best candidate for the job 2024-04-12T20:41:45.062 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:8030020200716213634:56fce90f.x86_64 from CentOS-AppStream 2024-04-12T20:41:45.063 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:8030020200716213634:98a3c9d0.x86_64 from CentOS-AppStream 2024-04-12T20:41:45.063 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:820230223114759:06b7596e.x86_64 from CentOS-AppStream 2024-04-12T20:41:45.063 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:820230223114759:d2fc5d76.x86_64 from CentOS-AppStream 2024-04-12T20:41:45.063 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 44: cannot install the best candidate for the job 2024-04-12T20:41:45.063 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:8030020200716171139:3a70019f.x86_64 from CentOS-AppStream 2024-04-12T20:41:45.063 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:8030020200716171139:cccafca5.x86_64 from CentOS-AppStream 2024-04-12T20:41:45.063 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:820230223114759:6fdf7e5d.x86_64 from CentOS-AppStream 2024-04-12T20:41:45.063 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:820230223114759:d4326aba.x86_64 from CentOS-AppStream 2024-04-12T20:41:45.063 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 45: cannot install the best candidate for the job 2024-04-12T20:41:45.063 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:8010020191120175858:95fce66d.x86_64 from CentOS-AppStream 2024-04-12T20:41:45.063 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:8030020200723233049:09acf126.x86_64 from CentOS-AppStream 2024-04-12T20:41:45.063 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:820230223114759:436af1ae.x86_64 from CentOS-AppStream 2024-04-12T20:41:45.063 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:820230223114759:b12195ea.x86_64 from CentOS-AppStream 2024-04-12T20:41:45.064 INFO:teuthology.orchestra.run.smithi018.stdout:Package python3-cephfs-2:17.2.7-827.gd3b5060a.el9.x86_64 is already installed. 2024-04-12T20:41:45.132 INFO:teuthology.orchestra.run.smithi018.stdout:Dependencies resolved. 2024-04-12T20:41:45.133 INFO:teuthology.orchestra.run.smithi018.stdout:Nothing to do. 2024-04-12T20:41:45.133 INFO:teuthology.orchestra.run.smithi018.stdout:Complete! 2024-04-12T20:41:45.210 DEBUG:teuthology.orchestra.run.smithi018:> sudo yum -y install python3-rbd 2024-04-12T20:41:45.715 INFO:teuthology.orchestra.run.smithi186.stdout:Last metadata expiration check: 0:02:08 ago on Fri 12 Apr 2024 08:39:37 PM UTC. 2024-04-12T20:41:46.023 INFO:teuthology.orchestra.run.smithi186.stderr:Modular dependency problems: 2024-04-12T20:41:46.023 INFO:teuthology.orchestra.run.smithi186.stderr: 2024-04-12T20:41:46.023 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 1: conflicting requests 2024-04-12T20:41:46.023 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module container-tools:rhel8:820240321091303:20125149.x86_64 from CentOS-AppStream 2024-04-12T20:41:46.023 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 2: conflicting requests 2024-04-12T20:41:46.024 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module eclipse:rhel8:820201023100746:b230ed4e.x86_64 from CentOS-AppStream 2024-04-12T20:41:46.024 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 3: conflicting requests 2024-04-12T20:41:46.024 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module freeradius:3.0:8080020230117180605:89170a74.x86_64 from CentOS-AppStream 2024-04-12T20:41:46.024 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 4: conflicting requests 2024-04-12T20:41:46.024 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module gimp:2.8:8000020190628145146:4148dfdf.x86_64 from CentOS-AppStream 2024-04-12T20:41:46.024 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 5: conflicting requests 2024-04-12T20:41:46.025 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module go-toolset:rhel8:820240401160413:b754926a.x86_64 from CentOS-AppStream 2024-04-12T20:41:46.025 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 6: conflicting requests 2024-04-12T20:41:46.025 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module httpd:2.4:820240216084734:9edba152.x86_64 from CentOS-AppStream 2024-04-12T20:41:46.025 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 7: conflicting requests 2024-04-12T20:41:46.025 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module idm:client:820240307184541:49cc9d1b.x86_64 from CentOS-AppStream 2024-04-12T20:41:46.025 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 8: conflicting requests 2024-04-12T20:41:46.025 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module inkscape:0.92.3:8080020221205124429:3e031279.x86_64 from CentOS-AppStream 2024-04-12T20:41:46.026 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 9: conflicting requests 2024-04-12T20:41:46.026 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module jmc:rhel8:8050020211110222101:6392b1f8.x86_64 from CentOS-AppStream 2024-04-12T20:41:46.026 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 10: conflicting requests 2024-04-12T20:41:46.026 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module llvm-toolset:rhel8:820240205133135:9edba152.x86_64 from CentOS-AppStream 2024-04-12T20:41:46.026 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 11: conflicting requests 2024-04-12T20:41:46.026 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module log4j:2:8080020221205124743:9d367344.x86_64 from CentOS-AppStream 2024-04-12T20:41:46.026 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 12: conflicting requests 2024-04-12T20:41:46.027 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module mailman:2.1:820230727085757:77fc8825.x86_64 from CentOS-AppStream 2024-04-12T20:41:46.027 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 13: conflicting requests 2024-04-12T20:41:46.027 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module mariadb:10.3:8030020210419150013:30b713e6.x86_64 from CentOS-AppStream 2024-04-12T20:41:46.027 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 14: conflicting requests 2024-04-12T20:41:46.027 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module maven:3.5:8060020211203142015:c0229ad2.x86_64 from CentOS-AppStream 2024-04-12T20:41:46.027 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 15: conflicting requests 2024-04-12T20:41:46.028 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module mercurial:4.8:8000020190628020724:4148dfdf.x86_64 from CentOS-AppStream 2024-04-12T20:41:46.028 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 16: conflicting requests 2024-04-12T20:41:46.028 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module mod_auth_openidc:2.3:820230721123832:9edba152.x86_64 from CentOS-AppStream 2024-04-12T20:41:46.028 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 17: conflicting requests 2024-04-12T20:41:46.029 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module mysql:8.0:8040020210901180257:9f9e2e7e.x86_64 from CentOS-AppStream 2024-04-12T20:41:46.029 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 18: conflicting requests 2024-04-12T20:41:46.029 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module nginx:1.14:8000020211221191913:55190bc5.x86_64 from CentOS-AppStream 2024-04-12T20:41:46.029 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 19: conflicting requests 2024-04-12T20:41:46.029 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module nodejs:10:8040020210120182536:9f9e2e7e.x86_64 from CentOS-AppStream 2024-04-12T20:41:46.029 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 20: conflicting requests 2024-04-12T20:41:46.030 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl:5.26:8000020190628020724:55190bc5.x86_64 from CentOS-AppStream 2024-04-12T20:41:46.030 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 21: conflicting requests 2024-04-12T20:41:46.030 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module php:7.2:8020020200507003613:2c7ca891.x86_64 from CentOS-AppStream 2024-04-12T20:41:46.030 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 22: conflicting requests 2024-04-12T20:41:46.030 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module postgresql:10:820230223114800:9edba152.x86_64 from CentOS-AppStream 2024-04-12T20:41:46.030 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 23: conflicting requests 2024-04-12T20:41:46.030 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module python27:2.7:820240208011952:182f7c73.x86_64 from CentOS-AppStream 2024-04-12T20:41:46.030 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 24: conflicting requests 2024-04-12T20:41:46.031 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module python36:3.6:8050020210825152031:982725ab.x86_64 from CentOS-AppStream 2024-04-12T20:41:46.031 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 25: conflicting requests 2024-04-12T20:41:46.031 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module python38:3.8:820230810143931:64c1cd5a.x86_64 from CentOS-AppStream 2024-04-12T20:41:46.031 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 26: conflicting requests 2024-04-12T20:41:46.031 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module python39:3.9:820240214182535:17377f89.x86_64 from CentOS-AppStream 2024-04-12T20:41:46.031 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 27: conflicting requests 2024-04-12T20:41:46.031 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module redis:5:8040020211019153849:9f9e2e7e.x86_64 from CentOS-AppStream 2024-04-12T20:41:46.032 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 28: conflicting requests 2024-04-12T20:41:46.032 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module rhn-tools:1.0:8010020191114034948:f69d1239.x86_64 from CentOS-AppStream 2024-04-12T20:41:46.032 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 29: conflicting requests 2024-04-12T20:41:46.032 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module ruby:2.5:820230627084142:9edba152.x86_64 from CentOS-AppStream 2024-04-12T20:41:46.032 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 30: conflicting requests 2024-04-12T20:41:46.033 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module rust-toolset:rhel8:820240119204620:b09eea91.x86_64 from CentOS-AppStream 2024-04-12T20:41:46.033 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 31: conflicting requests 2024-04-12T20:41:46.033 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module satellite-5-client:1.0:8010020191114035551:cdc1202b.x86_64 from CentOS-AppStream 2024-04-12T20:41:46.033 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 32: conflicting requests 2024-04-12T20:41:46.033 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module squid:4:820240319173245:9edba152.x86_64 from CentOS-AppStream 2024-04-12T20:41:46.033 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 33: conflicting requests 2024-04-12T20:41:46.033 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module subversion:1.10:8070020220711155714:78111232.x86_64 from CentOS-AppStream 2024-04-12T20:41:46.033 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 34: conflicting requests 2024-04-12T20:41:46.033 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module swig:3.0:8030020200706142531:30b713e6.x86_64 from CentOS-AppStream 2024-04-12T20:41:46.033 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 35: conflicting requests 2024-04-12T20:41:46.033 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module varnish:6:8050020211109225449:b4937e53.x86_64 from CentOS-AppStream 2024-04-12T20:41:46.033 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 36: conflicting requests 2024-04-12T20:41:46.034 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module virt:rhel:820240314161907:9edba152.x86_64 from CentOS-AppStream 2024-04-12T20:41:46.034 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 37: conflicting requests 2024-04-12T20:41:46.034 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:8030020200716155257:7cc0a66d.x86_64 from CentOS-AppStream 2024-04-12T20:41:46.034 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:8030020200716155257:b967a9a2.x86_64 from CentOS-AppStream 2024-04-12T20:41:46.034 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:820230811133638:36f9fae6.x86_64 from CentOS-AppStream 2024-04-12T20:41:46.034 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 38: cannot install the best candidate for the job 2024-04-12T20:41:46.034 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:8010020191114031501:a5949e2e.x86_64 from CentOS-AppStream 2024-04-12T20:41:46.034 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:8010020191114031501:be2e4737.x86_64 from CentOS-AppStream 2024-04-12T20:41:46.035 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:820230223114759:2ab761e1.x86_64 from CentOS-AppStream 2024-04-12T20:41:46.035 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:820230223114759:86e45846.x86_64 from CentOS-AppStream 2024-04-12T20:41:46.035 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 39: conflicting requests 2024-04-12T20:41:46.035 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:8030020200715230104:1e4bbb35.x86_64 from CentOS-AppStream 2024-04-12T20:41:46.035 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:8030020200715230104:ea09926d.x86_64 from CentOS-AppStream 2024-04-12T20:41:46.035 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:082fdf2f.x86_64 from CentOS-AppStream 2024-04-12T20:41:46.035 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-04-12T20:41:46.036 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-04-12T20:41:46.036 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:fbe42456.x86_64 from CentOS-AppStream 2024-04-12T20:41:46.036 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 40: cannot install the best candidate for the job 2024-04-12T20:41:46.036 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:8010020191114031513:16b3ab4d.x86_64 from CentOS-AppStream 2024-04-12T20:41:46.036 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:8010020191114031513:e1f37755.x86_64 from CentOS-AppStream 2024-04-12T20:41:46.036 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-04-12T20:41:46.036 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-04-12T20:41:46.036 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 41: cannot install the best candidate for the job 2024-04-12T20:41:46.036 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:8030020200716150652:1e4bbb35.x86_64 from CentOS-AppStream 2024-04-12T20:41:46.036 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:8030020200716150652:ea09926d.x86_64 from CentOS-AppStream 2024-04-12T20:41:46.036 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-04-12T20:41:46.036 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-04-12T20:41:46.037 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 42: cannot install the best candidate for the job 2024-04-12T20:41:46.037 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:8030020200716174729:3a70019f.x86_64 from CentOS-AppStream 2024-04-12T20:41:46.037 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:8030020200716174729:cccafca5.x86_64 from CentOS-AppStream 2024-04-12T20:41:46.037 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:820230223114759:6fdf7e5d.x86_64 from CentOS-AppStream 2024-04-12T20:41:46.037 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:820230223114759:d4326aba.x86_64 from CentOS-AppStream 2024-04-12T20:41:46.037 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 43: cannot install the best candidate for the job 2024-04-12T20:41:46.037 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:8030020200716213634:56fce90f.x86_64 from CentOS-AppStream 2024-04-12T20:41:46.037 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:8030020200716213634:98a3c9d0.x86_64 from CentOS-AppStream 2024-04-12T20:41:46.037 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:820230223114759:06b7596e.x86_64 from CentOS-AppStream 2024-04-12T20:41:46.037 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:820230223114759:d2fc5d76.x86_64 from CentOS-AppStream 2024-04-12T20:41:46.037 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 44: cannot install the best candidate for the job 2024-04-12T20:41:46.037 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:8030020200716171139:3a70019f.x86_64 from CentOS-AppStream 2024-04-12T20:41:46.037 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:8030020200716171139:cccafca5.x86_64 from CentOS-AppStream 2024-04-12T20:41:46.038 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:820230223114759:6fdf7e5d.x86_64 from CentOS-AppStream 2024-04-12T20:41:46.038 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:820230223114759:d4326aba.x86_64 from CentOS-AppStream 2024-04-12T20:41:46.038 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 45: cannot install the best candidate for the job 2024-04-12T20:41:46.038 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:8010020191120175858:95fce66d.x86_64 from CentOS-AppStream 2024-04-12T20:41:46.038 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:8030020200723233049:09acf126.x86_64 from CentOS-AppStream 2024-04-12T20:41:46.038 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:820230223114759:436af1ae.x86_64 from CentOS-AppStream 2024-04-12T20:41:46.038 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:820230223114759:b12195ea.x86_64 from CentOS-AppStream 2024-04-12T20:41:46.038 INFO:teuthology.orchestra.run.smithi186.stdout:Package python3-rbd-2:17.2.7-827.gd3b5060a.el9.x86_64 is already installed. 2024-04-12T20:41:46.097 INFO:teuthology.orchestra.run.smithi186.stdout:Dependencies resolved. 2024-04-12T20:41:46.098 INFO:teuthology.orchestra.run.smithi018.stdout:Last metadata expiration check: 0:02:11 ago on Fri 12 Apr 2024 08:39:35 PM UTC. 2024-04-12T20:41:46.098 INFO:teuthology.orchestra.run.smithi186.stdout:Nothing to do. 2024-04-12T20:41:46.098 INFO:teuthology.orchestra.run.smithi186.stdout:Complete! 2024-04-12T20:41:46.180 DEBUG:teuthology.orchestra.run.smithi186:> sudo yum -y install rbd-fuse 2024-04-12T20:41:46.420 INFO:teuthology.orchestra.run.smithi018.stderr:Modular dependency problems: 2024-04-12T20:41:46.421 INFO:teuthology.orchestra.run.smithi018.stderr: 2024-04-12T20:41:46.421 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 1: conflicting requests 2024-04-12T20:41:46.421 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module container-tools:rhel8:820240321091303:20125149.x86_64 from CentOS-AppStream 2024-04-12T20:41:46.421 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 2: conflicting requests 2024-04-12T20:41:46.421 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module eclipse:rhel8:820201023100746:b230ed4e.x86_64 from CentOS-AppStream 2024-04-12T20:41:46.421 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 3: conflicting requests 2024-04-12T20:41:46.421 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module freeradius:3.0:8080020230117180605:89170a74.x86_64 from CentOS-AppStream 2024-04-12T20:41:46.422 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 4: conflicting requests 2024-04-12T20:41:46.422 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module gimp:2.8:8000020190628145146:4148dfdf.x86_64 from CentOS-AppStream 2024-04-12T20:41:46.422 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 5: conflicting requests 2024-04-12T20:41:46.422 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module go-toolset:rhel8:820240401160413:b754926a.x86_64 from CentOS-AppStream 2024-04-12T20:41:46.422 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 6: conflicting requests 2024-04-12T20:41:46.422 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module httpd:2.4:820240216084734:9edba152.x86_64 from CentOS-AppStream 2024-04-12T20:41:46.422 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 7: conflicting requests 2024-04-12T20:41:46.423 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module idm:client:820240307184541:49cc9d1b.x86_64 from CentOS-AppStream 2024-04-12T20:41:46.423 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 8: conflicting requests 2024-04-12T20:41:46.423 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module inkscape:0.92.3:8080020221205124429:3e031279.x86_64 from CentOS-AppStream 2024-04-12T20:41:46.423 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 9: conflicting requests 2024-04-12T20:41:46.423 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module jmc:rhel8:8050020211110222101:6392b1f8.x86_64 from CentOS-AppStream 2024-04-12T20:41:46.423 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 10: conflicting requests 2024-04-12T20:41:46.423 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module llvm-toolset:rhel8:820240205133135:9edba152.x86_64 from CentOS-AppStream 2024-04-12T20:41:46.423 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 11: conflicting requests 2024-04-12T20:41:46.424 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module log4j:2:8080020221205124743:9d367344.x86_64 from CentOS-AppStream 2024-04-12T20:41:46.424 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 12: conflicting requests 2024-04-12T20:41:46.424 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module mailman:2.1:820230727085757:77fc8825.x86_64 from CentOS-AppStream 2024-04-12T20:41:46.424 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 13: conflicting requests 2024-04-12T20:41:46.424 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module mariadb:10.3:8030020210419150013:30b713e6.x86_64 from CentOS-AppStream 2024-04-12T20:41:46.424 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 14: conflicting requests 2024-04-12T20:41:46.424 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module maven:3.5:8060020211203142015:c0229ad2.x86_64 from CentOS-AppStream 2024-04-12T20:41:46.424 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 15: conflicting requests 2024-04-12T20:41:46.425 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module mercurial:4.8:8000020190628020724:4148dfdf.x86_64 from CentOS-AppStream 2024-04-12T20:41:46.425 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 16: conflicting requests 2024-04-12T20:41:46.425 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module mod_auth_openidc:2.3:820230721123832:9edba152.x86_64 from CentOS-AppStream 2024-04-12T20:41:46.425 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 17: conflicting requests 2024-04-12T20:41:46.425 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module mysql:8.0:8040020210901180257:9f9e2e7e.x86_64 from CentOS-AppStream 2024-04-12T20:41:46.425 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 18: conflicting requests 2024-04-12T20:41:46.425 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module nginx:1.14:8000020211221191913:55190bc5.x86_64 from CentOS-AppStream 2024-04-12T20:41:46.426 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 19: conflicting requests 2024-04-12T20:41:46.426 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module nodejs:10:8040020210120182536:9f9e2e7e.x86_64 from CentOS-AppStream 2024-04-12T20:41:46.426 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 20: conflicting requests 2024-04-12T20:41:46.426 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl:5.26:8000020190628020724:55190bc5.x86_64 from CentOS-AppStream 2024-04-12T20:41:46.426 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 21: conflicting requests 2024-04-12T20:41:46.426 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module php:7.2:8020020200507003613:2c7ca891.x86_64 from CentOS-AppStream 2024-04-12T20:41:46.426 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 22: conflicting requests 2024-04-12T20:41:46.427 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module postgresql:10:820230223114800:9edba152.x86_64 from CentOS-AppStream 2024-04-12T20:41:46.427 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 23: conflicting requests 2024-04-12T20:41:46.427 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module python27:2.7:820240208011952:182f7c73.x86_64 from CentOS-AppStream 2024-04-12T20:41:46.427 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 24: conflicting requests 2024-04-12T20:41:46.427 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module python36:3.6:8050020210825152031:982725ab.x86_64 from CentOS-AppStream 2024-04-12T20:41:46.427 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 25: conflicting requests 2024-04-12T20:41:46.427 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module python38:3.8:820230810143931:64c1cd5a.x86_64 from CentOS-AppStream 2024-04-12T20:41:46.428 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 26: conflicting requests 2024-04-12T20:41:46.428 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module python39:3.9:820240214182535:17377f89.x86_64 from CentOS-AppStream 2024-04-12T20:41:46.428 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 27: conflicting requests 2024-04-12T20:41:46.428 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module redis:5:8040020211019153849:9f9e2e7e.x86_64 from CentOS-AppStream 2024-04-12T20:41:46.428 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 28: conflicting requests 2024-04-12T20:41:46.428 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module rhn-tools:1.0:8010020191114034948:f69d1239.x86_64 from CentOS-AppStream 2024-04-12T20:41:46.428 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 29: conflicting requests 2024-04-12T20:41:46.428 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module ruby:2.5:820230627084142:9edba152.x86_64 from CentOS-AppStream 2024-04-12T20:41:46.429 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 30: conflicting requests 2024-04-12T20:41:46.429 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module rust-toolset:rhel8:820240119204620:b09eea91.x86_64 from CentOS-AppStream 2024-04-12T20:41:46.429 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 31: conflicting requests 2024-04-12T20:41:46.429 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module satellite-5-client:1.0:8010020191114035551:cdc1202b.x86_64 from CentOS-AppStream 2024-04-12T20:41:46.429 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 32: conflicting requests 2024-04-12T20:41:46.429 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module squid:4:820240319173245:9edba152.x86_64 from CentOS-AppStream 2024-04-12T20:41:46.429 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 33: conflicting requests 2024-04-12T20:41:46.430 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module subversion:1.10:8070020220711155714:78111232.x86_64 from CentOS-AppStream 2024-04-12T20:41:46.430 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 34: conflicting requests 2024-04-12T20:41:46.430 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module swig:3.0:8030020200706142531:30b713e6.x86_64 from CentOS-AppStream 2024-04-12T20:41:46.430 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 35: conflicting requests 2024-04-12T20:41:46.430 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module varnish:6:8050020211109225449:b4937e53.x86_64 from CentOS-AppStream 2024-04-12T20:41:46.430 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 36: conflicting requests 2024-04-12T20:41:46.430 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module virt:rhel:820240314161907:9edba152.x86_64 from CentOS-AppStream 2024-04-12T20:41:46.431 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 37: conflicting requests 2024-04-12T20:41:46.431 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:8030020200716155257:7cc0a66d.x86_64 from CentOS-AppStream 2024-04-12T20:41:46.431 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:8030020200716155257:b967a9a2.x86_64 from CentOS-AppStream 2024-04-12T20:41:46.431 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:820230811133638:36f9fae6.x86_64 from CentOS-AppStream 2024-04-12T20:41:46.431 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 38: cannot install the best candidate for the job 2024-04-12T20:41:46.432 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:8010020191114031501:a5949e2e.x86_64 from CentOS-AppStream 2024-04-12T20:41:46.432 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:8010020191114031501:be2e4737.x86_64 from CentOS-AppStream 2024-04-12T20:41:46.432 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:820230223114759:2ab761e1.x86_64 from CentOS-AppStream 2024-04-12T20:41:46.432 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:820230223114759:86e45846.x86_64 from CentOS-AppStream 2024-04-12T20:41:46.432 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 39: conflicting requests 2024-04-12T20:41:46.432 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:8030020200715230104:1e4bbb35.x86_64 from CentOS-AppStream 2024-04-12T20:41:46.432 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:8030020200715230104:ea09926d.x86_64 from CentOS-AppStream 2024-04-12T20:41:46.432 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:082fdf2f.x86_64 from CentOS-AppStream 2024-04-12T20:41:46.433 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-04-12T20:41:46.433 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-04-12T20:41:46.433 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:fbe42456.x86_64 from CentOS-AppStream 2024-04-12T20:41:46.433 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 40: cannot install the best candidate for the job 2024-04-12T20:41:46.433 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:8010020191114031513:16b3ab4d.x86_64 from CentOS-AppStream 2024-04-12T20:41:46.433 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:8010020191114031513:e1f37755.x86_64 from CentOS-AppStream 2024-04-12T20:41:46.433 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-04-12T20:41:46.434 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-04-12T20:41:46.434 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 41: cannot install the best candidate for the job 2024-04-12T20:41:46.434 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:8030020200716150652:1e4bbb35.x86_64 from CentOS-AppStream 2024-04-12T20:41:46.434 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:8030020200716150652:ea09926d.x86_64 from CentOS-AppStream 2024-04-12T20:41:46.434 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-04-12T20:41:46.434 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-04-12T20:41:46.434 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 42: cannot install the best candidate for the job 2024-04-12T20:41:46.435 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:8030020200716174729:3a70019f.x86_64 from CentOS-AppStream 2024-04-12T20:41:46.435 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:8030020200716174729:cccafca5.x86_64 from CentOS-AppStream 2024-04-12T20:41:46.435 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:820230223114759:6fdf7e5d.x86_64 from CentOS-AppStream 2024-04-12T20:41:46.435 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:820230223114759:d4326aba.x86_64 from CentOS-AppStream 2024-04-12T20:41:46.435 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 43: cannot install the best candidate for the job 2024-04-12T20:41:46.435 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:8030020200716213634:56fce90f.x86_64 from CentOS-AppStream 2024-04-12T20:41:46.435 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:8030020200716213634:98a3c9d0.x86_64 from CentOS-AppStream 2024-04-12T20:41:46.436 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:820230223114759:06b7596e.x86_64 from CentOS-AppStream 2024-04-12T20:41:46.436 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:820230223114759:d2fc5d76.x86_64 from CentOS-AppStream 2024-04-12T20:41:46.436 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 44: cannot install the best candidate for the job 2024-04-12T20:41:46.436 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:8030020200716171139:3a70019f.x86_64 from CentOS-AppStream 2024-04-12T20:41:46.436 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:8030020200716171139:cccafca5.x86_64 from CentOS-AppStream 2024-04-12T20:41:46.436 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:820230223114759:6fdf7e5d.x86_64 from CentOS-AppStream 2024-04-12T20:41:46.436 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:820230223114759:d4326aba.x86_64 from CentOS-AppStream 2024-04-12T20:41:46.436 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 45: cannot install the best candidate for the job 2024-04-12T20:41:46.437 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:8010020191120175858:95fce66d.x86_64 from CentOS-AppStream 2024-04-12T20:41:46.437 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:8030020200723233049:09acf126.x86_64 from CentOS-AppStream 2024-04-12T20:41:46.437 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:820230223114759:436af1ae.x86_64 from CentOS-AppStream 2024-04-12T20:41:46.437 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:820230223114759:b12195ea.x86_64 from CentOS-AppStream 2024-04-12T20:41:46.438 INFO:teuthology.orchestra.run.smithi018.stdout:Package python3-rbd-2:17.2.7-827.gd3b5060a.el9.x86_64 is already installed. 2024-04-12T20:41:46.501 INFO:teuthology.orchestra.run.smithi018.stdout:Dependencies resolved. 2024-04-12T20:41:46.502 INFO:teuthology.orchestra.run.smithi018.stdout:Nothing to do. 2024-04-12T20:41:46.502 INFO:teuthology.orchestra.run.smithi018.stdout:Complete! 2024-04-12T20:41:46.573 DEBUG:teuthology.orchestra.run.smithi018:> sudo yum -y install rbd-fuse 2024-04-12T20:41:47.056 INFO:teuthology.orchestra.run.smithi186.stdout:Last metadata expiration check: 0:02:10 ago on Fri 12 Apr 2024 08:39:37 PM UTC. 2024-04-12T20:41:47.359 INFO:teuthology.orchestra.run.smithi186.stderr:Modular dependency problems: 2024-04-12T20:41:47.359 INFO:teuthology.orchestra.run.smithi186.stderr: 2024-04-12T20:41:47.359 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 1: conflicting requests 2024-04-12T20:41:47.359 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module container-tools:rhel8:820240321091303:20125149.x86_64 from CentOS-AppStream 2024-04-12T20:41:47.359 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 2: conflicting requests 2024-04-12T20:41:47.359 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module eclipse:rhel8:820201023100746:b230ed4e.x86_64 from CentOS-AppStream 2024-04-12T20:41:47.360 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 3: conflicting requests 2024-04-12T20:41:47.360 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module freeradius:3.0:8080020230117180605:89170a74.x86_64 from CentOS-AppStream 2024-04-12T20:41:47.360 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 4: conflicting requests 2024-04-12T20:41:47.360 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module gimp:2.8:8000020190628145146:4148dfdf.x86_64 from CentOS-AppStream 2024-04-12T20:41:47.360 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 5: conflicting requests 2024-04-12T20:41:47.360 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module go-toolset:rhel8:820240401160413:b754926a.x86_64 from CentOS-AppStream 2024-04-12T20:41:47.360 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 6: conflicting requests 2024-04-12T20:41:47.360 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module httpd:2.4:820240216084734:9edba152.x86_64 from CentOS-AppStream 2024-04-12T20:41:47.360 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 7: conflicting requests 2024-04-12T20:41:47.360 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module idm:client:820240307184541:49cc9d1b.x86_64 from CentOS-AppStream 2024-04-12T20:41:47.360 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 8: conflicting requests 2024-04-12T20:41:47.360 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module inkscape:0.92.3:8080020221205124429:3e031279.x86_64 from CentOS-AppStream 2024-04-12T20:41:47.360 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 9: conflicting requests 2024-04-12T20:41:47.360 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module jmc:rhel8:8050020211110222101:6392b1f8.x86_64 from CentOS-AppStream 2024-04-12T20:41:47.360 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 10: conflicting requests 2024-04-12T20:41:47.361 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module llvm-toolset:rhel8:820240205133135:9edba152.x86_64 from CentOS-AppStream 2024-04-12T20:41:47.361 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 11: conflicting requests 2024-04-12T20:41:47.361 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module log4j:2:8080020221205124743:9d367344.x86_64 from CentOS-AppStream 2024-04-12T20:41:47.361 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 12: conflicting requests 2024-04-12T20:41:47.361 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module mailman:2.1:820230727085757:77fc8825.x86_64 from CentOS-AppStream 2024-04-12T20:41:47.361 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 13: conflicting requests 2024-04-12T20:41:47.361 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module mariadb:10.3:8030020210419150013:30b713e6.x86_64 from CentOS-AppStream 2024-04-12T20:41:47.361 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 14: conflicting requests 2024-04-12T20:41:47.361 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module maven:3.5:8060020211203142015:c0229ad2.x86_64 from CentOS-AppStream 2024-04-12T20:41:47.361 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 15: conflicting requests 2024-04-12T20:41:47.361 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module mercurial:4.8:8000020190628020724:4148dfdf.x86_64 from CentOS-AppStream 2024-04-12T20:41:47.361 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 16: conflicting requests 2024-04-12T20:41:47.361 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module mod_auth_openidc:2.3:820230721123832:9edba152.x86_64 from CentOS-AppStream 2024-04-12T20:41:47.361 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 17: conflicting requests 2024-04-12T20:41:47.362 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module mysql:8.0:8040020210901180257:9f9e2e7e.x86_64 from CentOS-AppStream 2024-04-12T20:41:47.362 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 18: conflicting requests 2024-04-12T20:41:47.362 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module nginx:1.14:8000020211221191913:55190bc5.x86_64 from CentOS-AppStream 2024-04-12T20:41:47.362 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 19: conflicting requests 2024-04-12T20:41:47.362 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module nodejs:10:8040020210120182536:9f9e2e7e.x86_64 from CentOS-AppStream 2024-04-12T20:41:47.362 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 20: conflicting requests 2024-04-12T20:41:47.362 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl:5.26:8000020190628020724:55190bc5.x86_64 from CentOS-AppStream 2024-04-12T20:41:47.362 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 21: conflicting requests 2024-04-12T20:41:47.362 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module php:7.2:8020020200507003613:2c7ca891.x86_64 from CentOS-AppStream 2024-04-12T20:41:47.362 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 22: conflicting requests 2024-04-12T20:41:47.362 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module postgresql:10:820230223114800:9edba152.x86_64 from CentOS-AppStream 2024-04-12T20:41:47.362 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 23: conflicting requests 2024-04-12T20:41:47.362 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module python27:2.7:820240208011952:182f7c73.x86_64 from CentOS-AppStream 2024-04-12T20:41:47.362 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 24: conflicting requests 2024-04-12T20:41:47.362 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module python36:3.6:8050020210825152031:982725ab.x86_64 from CentOS-AppStream 2024-04-12T20:41:47.363 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 25: conflicting requests 2024-04-12T20:41:47.363 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module python38:3.8:820230810143931:64c1cd5a.x86_64 from CentOS-AppStream 2024-04-12T20:41:47.363 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 26: conflicting requests 2024-04-12T20:41:47.363 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module python39:3.9:820240214182535:17377f89.x86_64 from CentOS-AppStream 2024-04-12T20:41:47.363 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 27: conflicting requests 2024-04-12T20:41:47.363 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module redis:5:8040020211019153849:9f9e2e7e.x86_64 from CentOS-AppStream 2024-04-12T20:41:47.363 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 28: conflicting requests 2024-04-12T20:41:47.363 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module rhn-tools:1.0:8010020191114034948:f69d1239.x86_64 from CentOS-AppStream 2024-04-12T20:41:47.363 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 29: conflicting requests 2024-04-12T20:41:47.363 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module ruby:2.5:820230627084142:9edba152.x86_64 from CentOS-AppStream 2024-04-12T20:41:47.364 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 30: conflicting requests 2024-04-12T20:41:47.364 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module rust-toolset:rhel8:820240119204620:b09eea91.x86_64 from CentOS-AppStream 2024-04-12T20:41:47.364 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 31: conflicting requests 2024-04-12T20:41:47.364 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module satellite-5-client:1.0:8010020191114035551:cdc1202b.x86_64 from CentOS-AppStream 2024-04-12T20:41:47.364 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 32: conflicting requests 2024-04-12T20:41:47.364 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module squid:4:820240319173245:9edba152.x86_64 from CentOS-AppStream 2024-04-12T20:41:47.364 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 33: conflicting requests 2024-04-12T20:41:47.364 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module subversion:1.10:8070020220711155714:78111232.x86_64 from CentOS-AppStream 2024-04-12T20:41:47.364 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 34: conflicting requests 2024-04-12T20:41:47.364 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module swig:3.0:8030020200706142531:30b713e6.x86_64 from CentOS-AppStream 2024-04-12T20:41:47.364 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 35: conflicting requests 2024-04-12T20:41:47.364 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module varnish:6:8050020211109225449:b4937e53.x86_64 from CentOS-AppStream 2024-04-12T20:41:47.364 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 36: conflicting requests 2024-04-12T20:41:47.364 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module virt:rhel:820240314161907:9edba152.x86_64 from CentOS-AppStream 2024-04-12T20:41:47.365 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 37: conflicting requests 2024-04-12T20:41:47.365 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:8030020200716155257:7cc0a66d.x86_64 from CentOS-AppStream 2024-04-12T20:41:47.365 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:8030020200716155257:b967a9a2.x86_64 from CentOS-AppStream 2024-04-12T20:41:47.365 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:820230811133638:36f9fae6.x86_64 from CentOS-AppStream 2024-04-12T20:41:47.365 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 38: cannot install the best candidate for the job 2024-04-12T20:41:47.365 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:8010020191114031501:a5949e2e.x86_64 from CentOS-AppStream 2024-04-12T20:41:47.365 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:8010020191114031501:be2e4737.x86_64 from CentOS-AppStream 2024-04-12T20:41:47.365 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:820230223114759:2ab761e1.x86_64 from CentOS-AppStream 2024-04-12T20:41:47.365 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:820230223114759:86e45846.x86_64 from CentOS-AppStream 2024-04-12T20:41:47.365 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 39: conflicting requests 2024-04-12T20:41:47.365 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:8030020200715230104:1e4bbb35.x86_64 from CentOS-AppStream 2024-04-12T20:41:47.365 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:8030020200715230104:ea09926d.x86_64 from CentOS-AppStream 2024-04-12T20:41:47.365 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:082fdf2f.x86_64 from CentOS-AppStream 2024-04-12T20:41:47.365 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-04-12T20:41:47.365 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-04-12T20:41:47.366 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:fbe42456.x86_64 from CentOS-AppStream 2024-04-12T20:41:47.366 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 40: cannot install the best candidate for the job 2024-04-12T20:41:47.366 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:8010020191114031513:16b3ab4d.x86_64 from CentOS-AppStream 2024-04-12T20:41:47.366 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:8010020191114031513:e1f37755.x86_64 from CentOS-AppStream 2024-04-12T20:41:47.366 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-04-12T20:41:47.366 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-04-12T20:41:47.366 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 41: cannot install the best candidate for the job 2024-04-12T20:41:47.366 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:8030020200716150652:1e4bbb35.x86_64 from CentOS-AppStream 2024-04-12T20:41:47.366 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:8030020200716150652:ea09926d.x86_64 from CentOS-AppStream 2024-04-12T20:41:47.366 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-04-12T20:41:47.366 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-04-12T20:41:47.366 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 42: cannot install the best candidate for the job 2024-04-12T20:41:47.366 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:8030020200716174729:3a70019f.x86_64 from CentOS-AppStream 2024-04-12T20:41:47.366 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:8030020200716174729:cccafca5.x86_64 from CentOS-AppStream 2024-04-12T20:41:47.367 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:820230223114759:6fdf7e5d.x86_64 from CentOS-AppStream 2024-04-12T20:41:47.367 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:820230223114759:d4326aba.x86_64 from CentOS-AppStream 2024-04-12T20:41:47.367 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 43: cannot install the best candidate for the job 2024-04-12T20:41:47.367 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:8030020200716213634:56fce90f.x86_64 from CentOS-AppStream 2024-04-12T20:41:47.367 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:8030020200716213634:98a3c9d0.x86_64 from CentOS-AppStream 2024-04-12T20:41:47.367 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:820230223114759:06b7596e.x86_64 from CentOS-AppStream 2024-04-12T20:41:47.367 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:820230223114759:d2fc5d76.x86_64 from CentOS-AppStream 2024-04-12T20:41:47.367 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 44: cannot install the best candidate for the job 2024-04-12T20:41:47.367 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:8030020200716171139:3a70019f.x86_64 from CentOS-AppStream 2024-04-12T20:41:47.367 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:8030020200716171139:cccafca5.x86_64 from CentOS-AppStream 2024-04-12T20:41:47.367 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:820230223114759:6fdf7e5d.x86_64 from CentOS-AppStream 2024-04-12T20:41:47.367 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:820230223114759:d4326aba.x86_64 from CentOS-AppStream 2024-04-12T20:41:47.367 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 45: cannot install the best candidate for the job 2024-04-12T20:41:47.367 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:8010020191120175858:95fce66d.x86_64 from CentOS-AppStream 2024-04-12T20:41:47.367 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:8030020200723233049:09acf126.x86_64 from CentOS-AppStream 2024-04-12T20:41:47.368 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:820230223114759:436af1ae.x86_64 from CentOS-AppStream 2024-04-12T20:41:47.368 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:820230223114759:b12195ea.x86_64 from CentOS-AppStream 2024-04-12T20:41:47.428 INFO:teuthology.orchestra.run.smithi186.stdout:Dependencies resolved. 2024-04-12T20:41:47.428 INFO:teuthology.orchestra.run.smithi186.stdout:================================================================================ 2024-04-12T20:41:47.428 INFO:teuthology.orchestra.run.smithi186.stdout: Package Architecture Version Repository Size 2024-04-12T20:41:47.429 INFO:teuthology.orchestra.run.smithi186.stdout:================================================================================ 2024-04-12T20:41:47.429 INFO:teuthology.orchestra.run.smithi186.stdout:Installing: 2024-04-12T20:41:47.429 INFO:teuthology.orchestra.run.smithi186.stdout: rbd-fuse x86_64 2:17.2.7-827.gd3b5060a.el9 ceph 82 k 2024-04-12T20:41:47.429 INFO:teuthology.orchestra.run.smithi186.stdout: 2024-04-12T20:41:47.429 INFO:teuthology.orchestra.run.smithi186.stdout:Transaction Summary 2024-04-12T20:41:47.429 INFO:teuthology.orchestra.run.smithi186.stdout:================================================================================ 2024-04-12T20:41:47.429 INFO:teuthology.orchestra.run.smithi186.stdout:Install 1 Package 2024-04-12T20:41:47.429 INFO:teuthology.orchestra.run.smithi186.stdout: 2024-04-12T20:41:47.430 INFO:teuthology.orchestra.run.smithi186.stdout:Total download size: 82 k 2024-04-12T20:41:47.430 INFO:teuthology.orchestra.run.smithi186.stdout:Installed size: 222 k 2024-04-12T20:41:47.430 INFO:teuthology.orchestra.run.smithi186.stdout:Downloading Packages: 2024-04-12T20:41:47.455 INFO:teuthology.orchestra.run.smithi018.stdout:Last metadata expiration check: 0:02:12 ago on Fri 12 Apr 2024 08:39:35 PM UTC. 2024-04-12T20:41:47.673 INFO:teuthology.orchestra.run.smithi186.stdout:rbd-fuse-17.2.7-827.gd3b5060a.el9.x86_64.rpm 338 kB/s | 82 kB 00:00 2024-04-12T20:41:47.674 INFO:teuthology.orchestra.run.smithi186.stdout:-------------------------------------------------------------------------------- 2024-04-12T20:41:47.676 INFO:teuthology.orchestra.run.smithi186.stdout:Total 334 kB/s | 82 kB 00:00 2024-04-12T20:41:47.676 INFO:teuthology.orchestra.run.smithi186.stdout:Running transaction check 2024-04-12T20:41:47.699 INFO:teuthology.orchestra.run.smithi186.stdout:Transaction check succeeded. 2024-04-12T20:41:47.700 INFO:teuthology.orchestra.run.smithi186.stdout:Running transaction test 2024-04-12T20:41:47.767 INFO:teuthology.orchestra.run.smithi018.stderr:Modular dependency problems: 2024-04-12T20:41:47.767 INFO:teuthology.orchestra.run.smithi018.stderr: 2024-04-12T20:41:47.767 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 1: conflicting requests 2024-04-12T20:41:47.767 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module container-tools:rhel8:820240321091303:20125149.x86_64 from CentOS-AppStream 2024-04-12T20:41:47.767 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 2: conflicting requests 2024-04-12T20:41:47.767 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module eclipse:rhel8:820201023100746:b230ed4e.x86_64 from CentOS-AppStream 2024-04-12T20:41:47.768 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 3: conflicting requests 2024-04-12T20:41:47.768 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module freeradius:3.0:8080020230117180605:89170a74.x86_64 from CentOS-AppStream 2024-04-12T20:41:47.768 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 4: conflicting requests 2024-04-12T20:41:47.768 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module gimp:2.8:8000020190628145146:4148dfdf.x86_64 from CentOS-AppStream 2024-04-12T20:41:47.768 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 5: conflicting requests 2024-04-12T20:41:47.768 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module go-toolset:rhel8:820240401160413:b754926a.x86_64 from CentOS-AppStream 2024-04-12T20:41:47.768 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 6: conflicting requests 2024-04-12T20:41:47.768 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module httpd:2.4:820240216084734:9edba152.x86_64 from CentOS-AppStream 2024-04-12T20:41:47.768 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 7: conflicting requests 2024-04-12T20:41:47.768 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module idm:client:820240307184541:49cc9d1b.x86_64 from CentOS-AppStream 2024-04-12T20:41:47.768 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 8: conflicting requests 2024-04-12T20:41:47.768 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module inkscape:0.92.3:8080020221205124429:3e031279.x86_64 from CentOS-AppStream 2024-04-12T20:41:47.768 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 9: conflicting requests 2024-04-12T20:41:47.768 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module jmc:rhel8:8050020211110222101:6392b1f8.x86_64 from CentOS-AppStream 2024-04-12T20:41:47.769 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 10: conflicting requests 2024-04-12T20:41:47.769 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module llvm-toolset:rhel8:820240205133135:9edba152.x86_64 from CentOS-AppStream 2024-04-12T20:41:47.769 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 11: conflicting requests 2024-04-12T20:41:47.769 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module log4j:2:8080020221205124743:9d367344.x86_64 from CentOS-AppStream 2024-04-12T20:41:47.769 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 12: conflicting requests 2024-04-12T20:41:47.769 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module mailman:2.1:820230727085757:77fc8825.x86_64 from CentOS-AppStream 2024-04-12T20:41:47.769 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 13: conflicting requests 2024-04-12T20:41:47.769 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module mariadb:10.3:8030020210419150013:30b713e6.x86_64 from CentOS-AppStream 2024-04-12T20:41:47.769 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 14: conflicting requests 2024-04-12T20:41:47.769 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module maven:3.5:8060020211203142015:c0229ad2.x86_64 from CentOS-AppStream 2024-04-12T20:41:47.769 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 15: conflicting requests 2024-04-12T20:41:47.769 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module mercurial:4.8:8000020190628020724:4148dfdf.x86_64 from CentOS-AppStream 2024-04-12T20:41:47.769 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 16: conflicting requests 2024-04-12T20:41:47.769 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module mod_auth_openidc:2.3:820230721123832:9edba152.x86_64 from CentOS-AppStream 2024-04-12T20:41:47.770 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 17: conflicting requests 2024-04-12T20:41:47.770 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module mysql:8.0:8040020210901180257:9f9e2e7e.x86_64 from CentOS-AppStream 2024-04-12T20:41:47.770 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 18: conflicting requests 2024-04-12T20:41:47.770 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module nginx:1.14:8000020211221191913:55190bc5.x86_64 from CentOS-AppStream 2024-04-12T20:41:47.770 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 19: conflicting requests 2024-04-12T20:41:47.770 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module nodejs:10:8040020210120182536:9f9e2e7e.x86_64 from CentOS-AppStream 2024-04-12T20:41:47.770 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 20: conflicting requests 2024-04-12T20:41:47.770 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl:5.26:8000020190628020724:55190bc5.x86_64 from CentOS-AppStream 2024-04-12T20:41:47.770 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 21: conflicting requests 2024-04-12T20:41:47.770 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module php:7.2:8020020200507003613:2c7ca891.x86_64 from CentOS-AppStream 2024-04-12T20:41:47.770 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 22: conflicting requests 2024-04-12T20:41:47.770 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module postgresql:10:820230223114800:9edba152.x86_64 from CentOS-AppStream 2024-04-12T20:41:47.770 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 23: conflicting requests 2024-04-12T20:41:47.770 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module python27:2.7:820240208011952:182f7c73.x86_64 from CentOS-AppStream 2024-04-12T20:41:47.770 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 24: conflicting requests 2024-04-12T20:41:47.771 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module python36:3.6:8050020210825152031:982725ab.x86_64 from CentOS-AppStream 2024-04-12T20:41:47.771 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 25: conflicting requests 2024-04-12T20:41:47.771 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module python38:3.8:820230810143931:64c1cd5a.x86_64 from CentOS-AppStream 2024-04-12T20:41:47.771 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 26: conflicting requests 2024-04-12T20:41:47.771 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module python39:3.9:820240214182535:17377f89.x86_64 from CentOS-AppStream 2024-04-12T20:41:47.771 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 27: conflicting requests 2024-04-12T20:41:47.771 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module redis:5:8040020211019153849:9f9e2e7e.x86_64 from CentOS-AppStream 2024-04-12T20:41:47.771 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 28: conflicting requests 2024-04-12T20:41:47.771 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module rhn-tools:1.0:8010020191114034948:f69d1239.x86_64 from CentOS-AppStream 2024-04-12T20:41:47.771 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 29: conflicting requests 2024-04-12T20:41:47.771 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module ruby:2.5:820230627084142:9edba152.x86_64 from CentOS-AppStream 2024-04-12T20:41:47.771 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 30: conflicting requests 2024-04-12T20:41:47.771 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module rust-toolset:rhel8:820240119204620:b09eea91.x86_64 from CentOS-AppStream 2024-04-12T20:41:47.771 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 31: conflicting requests 2024-04-12T20:41:47.772 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module satellite-5-client:1.0:8010020191114035551:cdc1202b.x86_64 from CentOS-AppStream 2024-04-12T20:41:47.772 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 32: conflicting requests 2024-04-12T20:41:47.772 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module squid:4:820240319173245:9edba152.x86_64 from CentOS-AppStream 2024-04-12T20:41:47.772 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 33: conflicting requests 2024-04-12T20:41:47.772 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module subversion:1.10:8070020220711155714:78111232.x86_64 from CentOS-AppStream 2024-04-12T20:41:47.772 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 34: conflicting requests 2024-04-12T20:41:47.772 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module swig:3.0:8030020200706142531:30b713e6.x86_64 from CentOS-AppStream 2024-04-12T20:41:47.772 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 35: conflicting requests 2024-04-12T20:41:47.772 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module varnish:6:8050020211109225449:b4937e53.x86_64 from CentOS-AppStream 2024-04-12T20:41:47.772 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 36: conflicting requests 2024-04-12T20:41:47.772 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module virt:rhel:820240314161907:9edba152.x86_64 from CentOS-AppStream 2024-04-12T20:41:47.773 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 37: conflicting requests 2024-04-12T20:41:47.773 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:8030020200716155257:7cc0a66d.x86_64 from CentOS-AppStream 2024-04-12T20:41:47.773 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:8030020200716155257:b967a9a2.x86_64 from CentOS-AppStream 2024-04-12T20:41:47.773 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:820230811133638:36f9fae6.x86_64 from CentOS-AppStream 2024-04-12T20:41:47.773 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 38: cannot install the best candidate for the job 2024-04-12T20:41:47.773 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:8010020191114031501:a5949e2e.x86_64 from CentOS-AppStream 2024-04-12T20:41:47.773 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:8010020191114031501:be2e4737.x86_64 from CentOS-AppStream 2024-04-12T20:41:47.774 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:820230223114759:2ab761e1.x86_64 from CentOS-AppStream 2024-04-12T20:41:47.774 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:820230223114759:86e45846.x86_64 from CentOS-AppStream 2024-04-12T20:41:47.774 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 39: conflicting requests 2024-04-12T20:41:47.774 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:8030020200715230104:1e4bbb35.x86_64 from CentOS-AppStream 2024-04-12T20:41:47.774 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:8030020200715230104:ea09926d.x86_64 from CentOS-AppStream 2024-04-12T20:41:47.774 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:082fdf2f.x86_64 from CentOS-AppStream 2024-04-12T20:41:47.774 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-04-12T20:41:47.774 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-04-12T20:41:47.774 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:fbe42456.x86_64 from CentOS-AppStream 2024-04-12T20:41:47.774 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 40: cannot install the best candidate for the job 2024-04-12T20:41:47.774 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:8010020191114031513:16b3ab4d.x86_64 from CentOS-AppStream 2024-04-12T20:41:47.774 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:8010020191114031513:e1f37755.x86_64 from CentOS-AppStream 2024-04-12T20:41:47.774 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-04-12T20:41:47.775 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-04-12T20:41:47.775 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 41: cannot install the best candidate for the job 2024-04-12T20:41:47.775 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:8030020200716150652:1e4bbb35.x86_64 from CentOS-AppStream 2024-04-12T20:41:47.775 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:8030020200716150652:ea09926d.x86_64 from CentOS-AppStream 2024-04-12T20:41:47.775 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-04-12T20:41:47.775 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-04-12T20:41:47.775 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 42: cannot install the best candidate for the job 2024-04-12T20:41:47.775 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:8030020200716174729:3a70019f.x86_64 from CentOS-AppStream 2024-04-12T20:41:47.775 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:8030020200716174729:cccafca5.x86_64 from CentOS-AppStream 2024-04-12T20:41:47.775 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:820230223114759:6fdf7e5d.x86_64 from CentOS-AppStream 2024-04-12T20:41:47.775 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:820230223114759:d4326aba.x86_64 from CentOS-AppStream 2024-04-12T20:41:47.775 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 43: cannot install the best candidate for the job 2024-04-12T20:41:47.775 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:8030020200716213634:56fce90f.x86_64 from CentOS-AppStream 2024-04-12T20:41:47.775 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:8030020200716213634:98a3c9d0.x86_64 from CentOS-AppStream 2024-04-12T20:41:47.775 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:820230223114759:06b7596e.x86_64 from CentOS-AppStream 2024-04-12T20:41:47.776 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:820230223114759:d2fc5d76.x86_64 from CentOS-AppStream 2024-04-12T20:41:47.776 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 44: cannot install the best candidate for the job 2024-04-12T20:41:47.776 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:8030020200716171139:3a70019f.x86_64 from CentOS-AppStream 2024-04-12T20:41:47.776 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:8030020200716171139:cccafca5.x86_64 from CentOS-AppStream 2024-04-12T20:41:47.776 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:820230223114759:6fdf7e5d.x86_64 from CentOS-AppStream 2024-04-12T20:41:47.776 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:820230223114759:d4326aba.x86_64 from CentOS-AppStream 2024-04-12T20:41:47.776 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 45: cannot install the best candidate for the job 2024-04-12T20:41:47.776 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:8010020191120175858:95fce66d.x86_64 from CentOS-AppStream 2024-04-12T20:41:47.776 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:8030020200723233049:09acf126.x86_64 from CentOS-AppStream 2024-04-12T20:41:47.776 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:820230223114759:436af1ae.x86_64 from CentOS-AppStream 2024-04-12T20:41:47.776 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:820230223114759:b12195ea.x86_64 from CentOS-AppStream 2024-04-12T20:41:47.805 INFO:teuthology.orchestra.run.smithi186.stdout:Transaction test succeeded. 2024-04-12T20:41:47.805 INFO:teuthology.orchestra.run.smithi186.stdout:Running transaction 2024-04-12T20:41:47.841 INFO:teuthology.orchestra.run.smithi018.stdout:Dependencies resolved. 2024-04-12T20:41:47.841 INFO:teuthology.orchestra.run.smithi018.stdout:================================================================================ 2024-04-12T20:41:47.841 INFO:teuthology.orchestra.run.smithi018.stdout: Package Architecture Version Repository Size 2024-04-12T20:41:47.841 INFO:teuthology.orchestra.run.smithi018.stdout:================================================================================ 2024-04-12T20:41:47.842 INFO:teuthology.orchestra.run.smithi018.stdout:Installing: 2024-04-12T20:41:47.842 INFO:teuthology.orchestra.run.smithi018.stdout: rbd-fuse x86_64 2:17.2.7-827.gd3b5060a.el9 ceph 82 k 2024-04-12T20:41:47.842 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-12T20:41:47.842 INFO:teuthology.orchestra.run.smithi018.stdout:Transaction Summary 2024-04-12T20:41:47.842 INFO:teuthology.orchestra.run.smithi018.stdout:================================================================================ 2024-04-12T20:41:47.842 INFO:teuthology.orchestra.run.smithi018.stdout:Install 1 Package 2024-04-12T20:41:47.842 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-12T20:41:47.843 INFO:teuthology.orchestra.run.smithi018.stdout:Total download size: 82 k 2024-04-12T20:41:47.843 INFO:teuthology.orchestra.run.smithi018.stdout:Installed size: 222 k 2024-04-12T20:41:47.843 INFO:teuthology.orchestra.run.smithi018.stdout:Downloading Packages: 2024-04-12T20:41:48.000 INFO:teuthology.orchestra.run.smithi186.stdout: Preparing : 1/1 2024-04-12T20:41:48.137 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : rbd-fuse-2:17.2.7-827.gd3b5060a.el9.x86_64 1/1 2024-04-12T20:41:48.293 INFO:teuthology.orchestra.run.smithi018.stdout:rbd-fuse-17.2.7-827.gd3b5060a.el9.x86_64.rpm 182 kB/s | 82 kB 00:00 2024-04-12T20:41:48.295 INFO:teuthology.orchestra.run.smithi018.stdout:-------------------------------------------------------------------------------- 2024-04-12T20:41:48.296 INFO:teuthology.orchestra.run.smithi018.stdout:Total 180 kB/s | 82 kB 00:00 2024-04-12T20:41:48.296 INFO:teuthology.orchestra.run.smithi018.stdout:Running transaction check 2024-04-12T20:41:48.321 INFO:teuthology.orchestra.run.smithi018.stdout:Transaction check succeeded. 2024-04-12T20:41:48.321 INFO:teuthology.orchestra.run.smithi018.stdout:Running transaction test 2024-04-12T20:41:48.432 INFO:teuthology.orchestra.run.smithi018.stdout:Transaction test succeeded. 2024-04-12T20:41:48.432 INFO:teuthology.orchestra.run.smithi018.stdout:Running transaction 2024-04-12T20:41:48.669 INFO:teuthology.orchestra.run.smithi018.stdout: Preparing : 1/1 2024-04-12T20:41:48.764 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: rbd-fuse-2:17.2.7-827.gd3b5060a.el9.x86_64 1/1 2024-04-12T20:41:48.842 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : rbd-fuse-2:17.2.7-827.gd3b5060a.el9.x86_64 1/1 2024-04-12T20:41:49.095 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : rbd-fuse-2:17.2.7-827.gd3b5060a.el9.x86_64 1/1 2024-04-12T20:41:49.095 INFO:teuthology.orchestra.run.smithi186.stdout: 2024-04-12T20:41:49.095 INFO:teuthology.orchestra.run.smithi186.stdout:Installed: 2024-04-12T20:41:49.095 INFO:teuthology.orchestra.run.smithi186.stdout: rbd-fuse-2:17.2.7-827.gd3b5060a.el9.x86_64 2024-04-12T20:41:49.095 INFO:teuthology.orchestra.run.smithi186.stdout: 2024-04-12T20:41:49.095 INFO:teuthology.orchestra.run.smithi186.stdout:Complete! 2024-04-12T20:41:49.385 INFO:teuthology.orchestra.run.smithi018.stdout: Running scriptlet: rbd-fuse-2:17.2.7-827.gd3b5060a.el9.x86_64 1/1 2024-04-12T20:41:49.399 DEBUG:teuthology.orchestra.run.smithi186:> sudo yum -y install rbd-mirror 2024-04-12T20:41:49.770 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : rbd-fuse-2:17.2.7-827.gd3b5060a.el9.x86_64 1/1 2024-04-12T20:41:49.770 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-12T20:41:49.771 INFO:teuthology.orchestra.run.smithi018.stdout:Installed: 2024-04-12T20:41:49.771 INFO:teuthology.orchestra.run.smithi018.stdout: rbd-fuse-2:17.2.7-827.gd3b5060a.el9.x86_64 2024-04-12T20:41:49.771 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-12T20:41:49.771 INFO:teuthology.orchestra.run.smithi018.stdout:Complete! 2024-04-12T20:41:50.131 DEBUG:teuthology.orchestra.run.smithi018:> sudo yum -y install rbd-mirror 2024-04-12T20:41:50.289 INFO:teuthology.orchestra.run.smithi186.stdout:Last metadata expiration check: 0:02:13 ago on Fri 12 Apr 2024 08:39:37 PM UTC. 2024-04-12T20:41:50.595 INFO:teuthology.orchestra.run.smithi186.stderr:Modular dependency problems: 2024-04-12T20:41:50.595 INFO:teuthology.orchestra.run.smithi186.stderr: 2024-04-12T20:41:50.595 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 1: conflicting requests 2024-04-12T20:41:50.595 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module container-tools:rhel8:820240321091303:20125149.x86_64 from CentOS-AppStream 2024-04-12T20:41:50.595 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 2: conflicting requests 2024-04-12T20:41:50.595 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module eclipse:rhel8:820201023100746:b230ed4e.x86_64 from CentOS-AppStream 2024-04-12T20:41:50.595 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 3: conflicting requests 2024-04-12T20:41:50.595 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module freeradius:3.0:8080020230117180605:89170a74.x86_64 from CentOS-AppStream 2024-04-12T20:41:50.595 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 4: conflicting requests 2024-04-12T20:41:50.595 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module gimp:2.8:8000020190628145146:4148dfdf.x86_64 from CentOS-AppStream 2024-04-12T20:41:50.596 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 5: conflicting requests 2024-04-12T20:41:50.596 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module go-toolset:rhel8:820240401160413:b754926a.x86_64 from CentOS-AppStream 2024-04-12T20:41:50.596 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 6: conflicting requests 2024-04-12T20:41:50.596 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module httpd:2.4:820240216084734:9edba152.x86_64 from CentOS-AppStream 2024-04-12T20:41:50.596 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 7: conflicting requests 2024-04-12T20:41:50.596 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module idm:client:820240307184541:49cc9d1b.x86_64 from CentOS-AppStream 2024-04-12T20:41:50.596 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 8: conflicting requests 2024-04-12T20:41:50.596 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module inkscape:0.92.3:8080020221205124429:3e031279.x86_64 from CentOS-AppStream 2024-04-12T20:41:50.596 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 9: conflicting requests 2024-04-12T20:41:50.596 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module jmc:rhel8:8050020211110222101:6392b1f8.x86_64 from CentOS-AppStream 2024-04-12T20:41:50.596 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 10: conflicting requests 2024-04-12T20:41:50.596 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module llvm-toolset:rhel8:820240205133135:9edba152.x86_64 from CentOS-AppStream 2024-04-12T20:41:50.596 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 11: conflicting requests 2024-04-12T20:41:50.597 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module log4j:2:8080020221205124743:9d367344.x86_64 from CentOS-AppStream 2024-04-12T20:41:50.597 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 12: conflicting requests 2024-04-12T20:41:50.597 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module mailman:2.1:820230727085757:77fc8825.x86_64 from CentOS-AppStream 2024-04-12T20:41:50.597 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 13: conflicting requests 2024-04-12T20:41:50.597 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module mariadb:10.3:8030020210419150013:30b713e6.x86_64 from CentOS-AppStream 2024-04-12T20:41:50.597 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 14: conflicting requests 2024-04-12T20:41:50.597 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module maven:3.5:8060020211203142015:c0229ad2.x86_64 from CentOS-AppStream 2024-04-12T20:41:50.597 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 15: conflicting requests 2024-04-12T20:41:50.597 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module mercurial:4.8:8000020190628020724:4148dfdf.x86_64 from CentOS-AppStream 2024-04-12T20:41:50.597 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 16: conflicting requests 2024-04-12T20:41:50.597 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module mod_auth_openidc:2.3:820230721123832:9edba152.x86_64 from CentOS-AppStream 2024-04-12T20:41:50.597 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 17: conflicting requests 2024-04-12T20:41:50.597 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module mysql:8.0:8040020210901180257:9f9e2e7e.x86_64 from CentOS-AppStream 2024-04-12T20:41:50.597 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 18: conflicting requests 2024-04-12T20:41:50.598 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module nginx:1.14:8000020211221191913:55190bc5.x86_64 from CentOS-AppStream 2024-04-12T20:41:50.598 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 19: conflicting requests 2024-04-12T20:41:50.598 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module nodejs:10:8040020210120182536:9f9e2e7e.x86_64 from CentOS-AppStream 2024-04-12T20:41:50.598 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 20: conflicting requests 2024-04-12T20:41:50.598 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl:5.26:8000020190628020724:55190bc5.x86_64 from CentOS-AppStream 2024-04-12T20:41:50.598 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 21: conflicting requests 2024-04-12T20:41:50.598 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module php:7.2:8020020200507003613:2c7ca891.x86_64 from CentOS-AppStream 2024-04-12T20:41:50.598 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 22: conflicting requests 2024-04-12T20:41:50.598 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module postgresql:10:820230223114800:9edba152.x86_64 from CentOS-AppStream 2024-04-12T20:41:50.598 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 23: conflicting requests 2024-04-12T20:41:50.598 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module python27:2.7:820240208011952:182f7c73.x86_64 from CentOS-AppStream 2024-04-12T20:41:50.598 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 24: conflicting requests 2024-04-12T20:41:50.598 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module python36:3.6:8050020210825152031:982725ab.x86_64 from CentOS-AppStream 2024-04-12T20:41:50.599 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 25: conflicting requests 2024-04-12T20:41:50.599 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module python38:3.8:820230810143931:64c1cd5a.x86_64 from CentOS-AppStream 2024-04-12T20:41:50.599 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 26: conflicting requests 2024-04-12T20:41:50.599 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module python39:3.9:820240214182535:17377f89.x86_64 from CentOS-AppStream 2024-04-12T20:41:50.599 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 27: conflicting requests 2024-04-12T20:41:50.599 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module redis:5:8040020211019153849:9f9e2e7e.x86_64 from CentOS-AppStream 2024-04-12T20:41:50.599 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 28: conflicting requests 2024-04-12T20:41:50.599 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module rhn-tools:1.0:8010020191114034948:f69d1239.x86_64 from CentOS-AppStream 2024-04-12T20:41:50.599 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 29: conflicting requests 2024-04-12T20:41:50.599 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module ruby:2.5:820230627084142:9edba152.x86_64 from CentOS-AppStream 2024-04-12T20:41:50.599 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 30: conflicting requests 2024-04-12T20:41:50.599 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module rust-toolset:rhel8:820240119204620:b09eea91.x86_64 from CentOS-AppStream 2024-04-12T20:41:50.599 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 31: conflicting requests 2024-04-12T20:41:50.599 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module satellite-5-client:1.0:8010020191114035551:cdc1202b.x86_64 from CentOS-AppStream 2024-04-12T20:41:50.599 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 32: conflicting requests 2024-04-12T20:41:50.600 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module squid:4:820240319173245:9edba152.x86_64 from CentOS-AppStream 2024-04-12T20:41:50.600 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 33: conflicting requests 2024-04-12T20:41:50.600 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module subversion:1.10:8070020220711155714:78111232.x86_64 from CentOS-AppStream 2024-04-12T20:41:50.600 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 34: conflicting requests 2024-04-12T20:41:50.600 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module swig:3.0:8030020200706142531:30b713e6.x86_64 from CentOS-AppStream 2024-04-12T20:41:50.600 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 35: conflicting requests 2024-04-12T20:41:50.600 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module varnish:6:8050020211109225449:b4937e53.x86_64 from CentOS-AppStream 2024-04-12T20:41:50.600 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 36: conflicting requests 2024-04-12T20:41:50.600 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module virt:rhel:820240314161907:9edba152.x86_64 from CentOS-AppStream 2024-04-12T20:41:50.600 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 37: conflicting requests 2024-04-12T20:41:50.600 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:8030020200716155257:7cc0a66d.x86_64 from CentOS-AppStream 2024-04-12T20:41:50.600 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:8030020200716155257:b967a9a2.x86_64 from CentOS-AppStream 2024-04-12T20:41:50.600 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:820230811133638:36f9fae6.x86_64 from CentOS-AppStream 2024-04-12T20:41:50.600 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 38: cannot install the best candidate for the job 2024-04-12T20:41:50.600 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:8010020191114031501:a5949e2e.x86_64 from CentOS-AppStream 2024-04-12T20:41:50.601 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:8010020191114031501:be2e4737.x86_64 from CentOS-AppStream 2024-04-12T20:41:50.601 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:820230223114759:2ab761e1.x86_64 from CentOS-AppStream 2024-04-12T20:41:50.601 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:820230223114759:86e45846.x86_64 from CentOS-AppStream 2024-04-12T20:41:50.601 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 39: conflicting requests 2024-04-12T20:41:50.601 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:8030020200715230104:1e4bbb35.x86_64 from CentOS-AppStream 2024-04-12T20:41:50.601 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:8030020200715230104:ea09926d.x86_64 from CentOS-AppStream 2024-04-12T20:41:50.601 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:082fdf2f.x86_64 from CentOS-AppStream 2024-04-12T20:41:50.601 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-04-12T20:41:50.601 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-04-12T20:41:50.601 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:fbe42456.x86_64 from CentOS-AppStream 2024-04-12T20:41:50.601 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 40: cannot install the best candidate for the job 2024-04-12T20:41:50.601 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:8010020191114031513:16b3ab4d.x86_64 from CentOS-AppStream 2024-04-12T20:41:50.601 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:8010020191114031513:e1f37755.x86_64 from CentOS-AppStream 2024-04-12T20:41:50.602 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-04-12T20:41:50.602 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-04-12T20:41:50.602 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 41: cannot install the best candidate for the job 2024-04-12T20:41:50.602 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:8030020200716150652:1e4bbb35.x86_64 from CentOS-AppStream 2024-04-12T20:41:50.602 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:8030020200716150652:ea09926d.x86_64 from CentOS-AppStream 2024-04-12T20:41:50.602 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-04-12T20:41:50.602 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-04-12T20:41:50.602 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 42: cannot install the best candidate for the job 2024-04-12T20:41:50.602 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:8030020200716174729:3a70019f.x86_64 from CentOS-AppStream 2024-04-12T20:41:50.602 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:8030020200716174729:cccafca5.x86_64 from CentOS-AppStream 2024-04-12T20:41:50.602 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:820230223114759:6fdf7e5d.x86_64 from CentOS-AppStream 2024-04-12T20:41:50.603 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:820230223114759:d4326aba.x86_64 from CentOS-AppStream 2024-04-12T20:41:50.603 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 43: cannot install the best candidate for the job 2024-04-12T20:41:50.603 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:8030020200716213634:56fce90f.x86_64 from CentOS-AppStream 2024-04-12T20:41:50.603 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:8030020200716213634:98a3c9d0.x86_64 from CentOS-AppStream 2024-04-12T20:41:50.603 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:820230223114759:06b7596e.x86_64 from CentOS-AppStream 2024-04-12T20:41:50.603 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:820230223114759:d2fc5d76.x86_64 from CentOS-AppStream 2024-04-12T20:41:50.603 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 44: cannot install the best candidate for the job 2024-04-12T20:41:50.603 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:8030020200716171139:3a70019f.x86_64 from CentOS-AppStream 2024-04-12T20:41:50.603 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:8030020200716171139:cccafca5.x86_64 from CentOS-AppStream 2024-04-12T20:41:50.604 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:820230223114759:6fdf7e5d.x86_64 from CentOS-AppStream 2024-04-12T20:41:50.604 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:820230223114759:d4326aba.x86_64 from CentOS-AppStream 2024-04-12T20:41:50.604 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 45: cannot install the best candidate for the job 2024-04-12T20:41:50.604 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:8010020191120175858:95fce66d.x86_64 from CentOS-AppStream 2024-04-12T20:41:50.604 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:8030020200723233049:09acf126.x86_64 from CentOS-AppStream 2024-04-12T20:41:50.604 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:820230223114759:436af1ae.x86_64 from CentOS-AppStream 2024-04-12T20:41:50.604 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:820230223114759:b12195ea.x86_64 from CentOS-AppStream 2024-04-12T20:41:50.669 INFO:teuthology.orchestra.run.smithi186.stdout:Dependencies resolved. 2024-04-12T20:41:50.671 INFO:teuthology.orchestra.run.smithi186.stdout:================================================================================ 2024-04-12T20:41:50.671 INFO:teuthology.orchestra.run.smithi186.stdout: Package Arch Version Repo Size 2024-04-12T20:41:50.671 INFO:teuthology.orchestra.run.smithi186.stdout:================================================================================ 2024-04-12T20:41:50.671 INFO:teuthology.orchestra.run.smithi186.stdout:Installing: 2024-04-12T20:41:50.671 INFO:teuthology.orchestra.run.smithi186.stdout: rbd-mirror x86_64 2:17.2.7-827.gd3b5060a.el9 ceph 2.9 M 2024-04-12T20:41:50.672 INFO:teuthology.orchestra.run.smithi186.stdout: 2024-04-12T20:41:50.672 INFO:teuthology.orchestra.run.smithi186.stdout:Transaction Summary 2024-04-12T20:41:50.672 INFO:teuthology.orchestra.run.smithi186.stdout:================================================================================ 2024-04-12T20:41:50.672 INFO:teuthology.orchestra.run.smithi186.stdout:Install 1 Package 2024-04-12T20:41:50.672 INFO:teuthology.orchestra.run.smithi186.stdout: 2024-04-12T20:41:50.672 INFO:teuthology.orchestra.run.smithi186.stdout:Total download size: 2.9 M 2024-04-12T20:41:50.673 INFO:teuthology.orchestra.run.smithi186.stdout:Installed size: 12 M 2024-04-12T20:41:50.673 INFO:teuthology.orchestra.run.smithi186.stdout:Downloading Packages: 2024-04-12T20:41:51.060 INFO:teuthology.orchestra.run.smithi018.stdout:Last metadata expiration check: 0:02:16 ago on Fri 12 Apr 2024 08:39:35 PM UTC. 2024-04-12T20:41:51.076 INFO:teuthology.orchestra.run.smithi186.stdout:rbd-mirror-17.2.7-827.gd3b5060a.el9.x86_64.rpm 7.3 MB/s | 2.9 MB 00:00 2024-04-12T20:41:51.077 INFO:teuthology.orchestra.run.smithi186.stdout:-------------------------------------------------------------------------------- 2024-04-12T20:41:51.078 INFO:teuthology.orchestra.run.smithi186.stdout:Total 7.2 MB/s | 2.9 MB 00:00 2024-04-12T20:41:51.079 INFO:teuthology.orchestra.run.smithi186.stdout:Running transaction check 2024-04-12T20:41:51.111 INFO:teuthology.orchestra.run.smithi186.stdout:Transaction check succeeded. 2024-04-12T20:41:51.111 INFO:teuthology.orchestra.run.smithi186.stdout:Running transaction test 2024-04-12T20:41:51.223 INFO:teuthology.orchestra.run.smithi186.stdout:Transaction test succeeded. 2024-04-12T20:41:51.224 INFO:teuthology.orchestra.run.smithi186.stdout:Running transaction 2024-04-12T20:41:51.377 INFO:teuthology.orchestra.run.smithi018.stderr:Modular dependency problems: 2024-04-12T20:41:51.377 INFO:teuthology.orchestra.run.smithi018.stderr: 2024-04-12T20:41:51.377 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 1: conflicting requests 2024-04-12T20:41:51.377 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module container-tools:rhel8:820240321091303:20125149.x86_64 from CentOS-AppStream 2024-04-12T20:41:51.378 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 2: conflicting requests 2024-04-12T20:41:51.378 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module eclipse:rhel8:820201023100746:b230ed4e.x86_64 from CentOS-AppStream 2024-04-12T20:41:51.378 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 3: conflicting requests 2024-04-12T20:41:51.378 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module freeradius:3.0:8080020230117180605:89170a74.x86_64 from CentOS-AppStream 2024-04-12T20:41:51.378 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 4: conflicting requests 2024-04-12T20:41:51.378 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module gimp:2.8:8000020190628145146:4148dfdf.x86_64 from CentOS-AppStream 2024-04-12T20:41:51.378 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 5: conflicting requests 2024-04-12T20:41:51.378 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module go-toolset:rhel8:820240401160413:b754926a.x86_64 from CentOS-AppStream 2024-04-12T20:41:51.378 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 6: conflicting requests 2024-04-12T20:41:51.378 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module httpd:2.4:820240216084734:9edba152.x86_64 from CentOS-AppStream 2024-04-12T20:41:51.378 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 7: conflicting requests 2024-04-12T20:41:51.378 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module idm:client:820240307184541:49cc9d1b.x86_64 from CentOS-AppStream 2024-04-12T20:41:51.378 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 8: conflicting requests 2024-04-12T20:41:51.379 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module inkscape:0.92.3:8080020221205124429:3e031279.x86_64 from CentOS-AppStream 2024-04-12T20:41:51.379 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 9: conflicting requests 2024-04-12T20:41:51.379 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module jmc:rhel8:8050020211110222101:6392b1f8.x86_64 from CentOS-AppStream 2024-04-12T20:41:51.379 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 10: conflicting requests 2024-04-12T20:41:51.379 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module llvm-toolset:rhel8:820240205133135:9edba152.x86_64 from CentOS-AppStream 2024-04-12T20:41:51.379 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 11: conflicting requests 2024-04-12T20:41:51.379 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module log4j:2:8080020221205124743:9d367344.x86_64 from CentOS-AppStream 2024-04-12T20:41:51.379 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 12: conflicting requests 2024-04-12T20:41:51.379 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module mailman:2.1:820230727085757:77fc8825.x86_64 from CentOS-AppStream 2024-04-12T20:41:51.379 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 13: conflicting requests 2024-04-12T20:41:51.379 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module mariadb:10.3:8030020210419150013:30b713e6.x86_64 from CentOS-AppStream 2024-04-12T20:41:51.379 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 14: conflicting requests 2024-04-12T20:41:51.379 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module maven:3.5:8060020211203142015:c0229ad2.x86_64 from CentOS-AppStream 2024-04-12T20:41:51.379 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 15: conflicting requests 2024-04-12T20:41:51.380 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module mercurial:4.8:8000020190628020724:4148dfdf.x86_64 from CentOS-AppStream 2024-04-12T20:41:51.380 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 16: conflicting requests 2024-04-12T20:41:51.380 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module mod_auth_openidc:2.3:820230721123832:9edba152.x86_64 from CentOS-AppStream 2024-04-12T20:41:51.380 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 17: conflicting requests 2024-04-12T20:41:51.380 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module mysql:8.0:8040020210901180257:9f9e2e7e.x86_64 from CentOS-AppStream 2024-04-12T20:41:51.380 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 18: conflicting requests 2024-04-12T20:41:51.380 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module nginx:1.14:8000020211221191913:55190bc5.x86_64 from CentOS-AppStream 2024-04-12T20:41:51.380 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 19: conflicting requests 2024-04-12T20:41:51.380 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module nodejs:10:8040020210120182536:9f9e2e7e.x86_64 from CentOS-AppStream 2024-04-12T20:41:51.380 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 20: conflicting requests 2024-04-12T20:41:51.380 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl:5.26:8000020190628020724:55190bc5.x86_64 from CentOS-AppStream 2024-04-12T20:41:51.380 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 21: conflicting requests 2024-04-12T20:41:51.380 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module php:7.2:8020020200507003613:2c7ca891.x86_64 from CentOS-AppStream 2024-04-12T20:41:51.380 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 22: conflicting requests 2024-04-12T20:41:51.381 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module postgresql:10:820230223114800:9edba152.x86_64 from CentOS-AppStream 2024-04-12T20:41:51.381 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 23: conflicting requests 2024-04-12T20:41:51.381 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module python27:2.7:820240208011952:182f7c73.x86_64 from CentOS-AppStream 2024-04-12T20:41:51.381 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 24: conflicting requests 2024-04-12T20:41:51.381 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module python36:3.6:8050020210825152031:982725ab.x86_64 from CentOS-AppStream 2024-04-12T20:41:51.381 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 25: conflicting requests 2024-04-12T20:41:51.381 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module python38:3.8:820230810143931:64c1cd5a.x86_64 from CentOS-AppStream 2024-04-12T20:41:51.381 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 26: conflicting requests 2024-04-12T20:41:51.381 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module python39:3.9:820240214182535:17377f89.x86_64 from CentOS-AppStream 2024-04-12T20:41:51.381 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 27: conflicting requests 2024-04-12T20:41:51.381 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module redis:5:8040020211019153849:9f9e2e7e.x86_64 from CentOS-AppStream 2024-04-12T20:41:51.381 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 28: conflicting requests 2024-04-12T20:41:51.382 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module rhn-tools:1.0:8010020191114034948:f69d1239.x86_64 from CentOS-AppStream 2024-04-12T20:41:51.382 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 29: conflicting requests 2024-04-12T20:41:51.382 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module ruby:2.5:820230627084142:9edba152.x86_64 from CentOS-AppStream 2024-04-12T20:41:51.382 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 30: conflicting requests 2024-04-12T20:41:51.382 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module rust-toolset:rhel8:820240119204620:b09eea91.x86_64 from CentOS-AppStream 2024-04-12T20:41:51.382 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 31: conflicting requests 2024-04-12T20:41:51.382 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module satellite-5-client:1.0:8010020191114035551:cdc1202b.x86_64 from CentOS-AppStream 2024-04-12T20:41:51.382 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 32: conflicting requests 2024-04-12T20:41:51.382 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module squid:4:820240319173245:9edba152.x86_64 from CentOS-AppStream 2024-04-12T20:41:51.382 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 33: conflicting requests 2024-04-12T20:41:51.382 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module subversion:1.10:8070020220711155714:78111232.x86_64 from CentOS-AppStream 2024-04-12T20:41:51.382 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 34: conflicting requests 2024-04-12T20:41:51.383 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module swig:3.0:8030020200706142531:30b713e6.x86_64 from CentOS-AppStream 2024-04-12T20:41:51.383 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 35: conflicting requests 2024-04-12T20:41:51.383 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module varnish:6:8050020211109225449:b4937e53.x86_64 from CentOS-AppStream 2024-04-12T20:41:51.383 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 36: conflicting requests 2024-04-12T20:41:51.383 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module virt:rhel:820240314161907:9edba152.x86_64 from CentOS-AppStream 2024-04-12T20:41:51.383 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 37: conflicting requests 2024-04-12T20:41:51.383 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:8030020200716155257:7cc0a66d.x86_64 from CentOS-AppStream 2024-04-12T20:41:51.383 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:8030020200716155257:b967a9a2.x86_64 from CentOS-AppStream 2024-04-12T20:41:51.383 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:820230811133638:36f9fae6.x86_64 from CentOS-AppStream 2024-04-12T20:41:51.383 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 38: cannot install the best candidate for the job 2024-04-12T20:41:51.383 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:8010020191114031501:a5949e2e.x86_64 from CentOS-AppStream 2024-04-12T20:41:51.383 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:8010020191114031501:be2e4737.x86_64 from CentOS-AppStream 2024-04-12T20:41:51.383 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:820230223114759:2ab761e1.x86_64 from CentOS-AppStream 2024-04-12T20:41:51.383 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:820230223114759:86e45846.x86_64 from CentOS-AppStream 2024-04-12T20:41:51.383 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 39: conflicting requests 2024-04-12T20:41:51.384 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:8030020200715230104:1e4bbb35.x86_64 from CentOS-AppStream 2024-04-12T20:41:51.384 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:8030020200715230104:ea09926d.x86_64 from CentOS-AppStream 2024-04-12T20:41:51.384 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:082fdf2f.x86_64 from CentOS-AppStream 2024-04-12T20:41:51.384 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-04-12T20:41:51.384 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-04-12T20:41:51.384 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:fbe42456.x86_64 from CentOS-AppStream 2024-04-12T20:41:51.384 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 40: cannot install the best candidate for the job 2024-04-12T20:41:51.384 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:8010020191114031513:16b3ab4d.x86_64 from CentOS-AppStream 2024-04-12T20:41:51.384 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:8010020191114031513:e1f37755.x86_64 from CentOS-AppStream 2024-04-12T20:41:51.384 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-04-12T20:41:51.384 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-04-12T20:41:51.384 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 41: cannot install the best candidate for the job 2024-04-12T20:41:51.385 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:8030020200716150652:1e4bbb35.x86_64 from CentOS-AppStream 2024-04-12T20:41:51.385 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:8030020200716150652:ea09926d.x86_64 from CentOS-AppStream 2024-04-12T20:41:51.385 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-04-12T20:41:51.385 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-04-12T20:41:51.385 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 42: cannot install the best candidate for the job 2024-04-12T20:41:51.385 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:8030020200716174729:3a70019f.x86_64 from CentOS-AppStream 2024-04-12T20:41:51.385 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:8030020200716174729:cccafca5.x86_64 from CentOS-AppStream 2024-04-12T20:41:51.385 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:820230223114759:6fdf7e5d.x86_64 from CentOS-AppStream 2024-04-12T20:41:51.385 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:820230223114759:d4326aba.x86_64 from CentOS-AppStream 2024-04-12T20:41:51.385 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 43: cannot install the best candidate for the job 2024-04-12T20:41:51.385 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:8030020200716213634:56fce90f.x86_64 from CentOS-AppStream 2024-04-12T20:41:51.385 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:8030020200716213634:98a3c9d0.x86_64 from CentOS-AppStream 2024-04-12T20:41:51.385 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:820230223114759:06b7596e.x86_64 from CentOS-AppStream 2024-04-12T20:41:51.385 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:820230223114759:d2fc5d76.x86_64 from CentOS-AppStream 2024-04-12T20:41:51.386 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 44: cannot install the best candidate for the job 2024-04-12T20:41:51.386 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:8030020200716171139:3a70019f.x86_64 from CentOS-AppStream 2024-04-12T20:41:51.386 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:8030020200716171139:cccafca5.x86_64 from CentOS-AppStream 2024-04-12T20:41:51.386 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:820230223114759:6fdf7e5d.x86_64 from CentOS-AppStream 2024-04-12T20:41:51.386 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:820230223114759:d4326aba.x86_64 from CentOS-AppStream 2024-04-12T20:41:51.386 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 45: cannot install the best candidate for the job 2024-04-12T20:41:51.386 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:8010020191120175858:95fce66d.x86_64 from CentOS-AppStream 2024-04-12T20:41:51.386 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:8030020200723233049:09acf126.x86_64 from CentOS-AppStream 2024-04-12T20:41:51.386 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:820230223114759:436af1ae.x86_64 from CentOS-AppStream 2024-04-12T20:41:51.386 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:820230223114759:b12195ea.x86_64 from CentOS-AppStream 2024-04-12T20:41:51.456 INFO:teuthology.orchestra.run.smithi018.stdout:Dependencies resolved. 2024-04-12T20:41:51.456 INFO:teuthology.orchestra.run.smithi018.stdout:================================================================================ 2024-04-12T20:41:51.457 INFO:teuthology.orchestra.run.smithi018.stdout: Package Arch Version Repo Size 2024-04-12T20:41:51.457 INFO:teuthology.orchestra.run.smithi018.stdout:================================================================================ 2024-04-12T20:41:51.457 INFO:teuthology.orchestra.run.smithi018.stdout:Installing: 2024-04-12T20:41:51.457 INFO:teuthology.orchestra.run.smithi018.stdout: rbd-mirror x86_64 2:17.2.7-827.gd3b5060a.el9 ceph 2.9 M 2024-04-12T20:41:51.457 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-12T20:41:51.457 INFO:teuthology.orchestra.run.smithi018.stdout:Transaction Summary 2024-04-12T20:41:51.457 INFO:teuthology.orchestra.run.smithi018.stdout:================================================================================ 2024-04-12T20:41:51.457 INFO:teuthology.orchestra.run.smithi018.stdout:Install 1 Package 2024-04-12T20:41:51.457 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-12T20:41:51.457 INFO:teuthology.orchestra.run.smithi018.stdout:Total download size: 2.9 M 2024-04-12T20:41:51.457 INFO:teuthology.orchestra.run.smithi018.stdout:Installed size: 12 M 2024-04-12T20:41:51.457 INFO:teuthology.orchestra.run.smithi018.stdout:Downloading Packages: 2024-04-12T20:41:51.753 INFO:teuthology.orchestra.run.smithi186.stdout: Preparing : 1/1 2024-04-12T20:41:51.843 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : rbd-mirror-2:17.2.7-827.gd3b5060a.el9.x86_64 1/1 2024-04-12T20:41:51.872 INFO:teuthology.orchestra.run.smithi018.stdout:rbd-mirror-17.2.7-827.gd3b5060a.el9.x86_64.rpm 7.1 MB/s | 2.9 MB 00:00 2024-04-12T20:41:51.874 INFO:teuthology.orchestra.run.smithi018.stdout:-------------------------------------------------------------------------------- 2024-04-12T20:41:51.875 INFO:teuthology.orchestra.run.smithi018.stdout:Total 7.0 MB/s | 2.9 MB 00:00 2024-04-12T20:41:51.875 INFO:teuthology.orchestra.run.smithi018.stdout:Running transaction check 2024-04-12T20:41:51.908 INFO:teuthology.orchestra.run.smithi018.stdout:Transaction check succeeded. 2024-04-12T20:41:51.908 INFO:teuthology.orchestra.run.smithi018.stdout:Running transaction test 2024-04-12T20:41:51.933 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: rbd-mirror-2:17.2.7-827.gd3b5060a.el9.x86_64 1/1 2024-04-12T20:41:51.933 INFO:teuthology.orchestra.run.smithi186.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-12T20:41:51.933 INFO:teuthology.orchestra.run.smithi186.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-04-12T20:41:51.934 INFO:teuthology.orchestra.run.smithi186.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-12T20:41:51.934 INFO:teuthology.orchestra.run.smithi186.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-12T20:41:51.934 INFO:teuthology.orchestra.run.smithi186.stdout: 2024-04-12T20:41:52.026 INFO:teuthology.orchestra.run.smithi018.stdout:Transaction test succeeded. 2024-04-12T20:41:52.027 INFO:teuthology.orchestra.run.smithi018.stdout:Running transaction 2024-04-12T20:41:52.579 INFO:teuthology.orchestra.run.smithi018.stdout: Preparing : 1/1 2024-04-12T20:41:52.656 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : rbd-mirror-2:17.2.7-827.gd3b5060a.el9.x86_64 1/1 2024-04-12T20:41:52.741 INFO:teuthology.orchestra.run.smithi018.stdout: Running scriptlet: rbd-mirror-2:17.2.7-827.gd3b5060a.el9.x86_64 1/1 2024-04-12T20:41:52.741 INFO:teuthology.orchestra.run.smithi018.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-12T20:41:52.741 INFO:teuthology.orchestra.run.smithi018.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-04-12T20:41:52.741 INFO:teuthology.orchestra.run.smithi018.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-12T20:41:52.741 INFO:teuthology.orchestra.run.smithi018.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-12T20:41:52.741 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-12T20:41:53.483 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : rbd-mirror-2:17.2.7-827.gd3b5060a.el9.x86_64 1/1 2024-04-12T20:41:53.483 INFO:teuthology.orchestra.run.smithi186.stdout: 2024-04-12T20:41:53.483 INFO:teuthology.orchestra.run.smithi186.stdout:Installed: 2024-04-12T20:41:53.483 INFO:teuthology.orchestra.run.smithi186.stdout: rbd-mirror-2:17.2.7-827.gd3b5060a.el9.x86_64 2024-04-12T20:41:53.483 INFO:teuthology.orchestra.run.smithi186.stdout: 2024-04-12T20:41:53.483 INFO:teuthology.orchestra.run.smithi186.stdout:Complete! 2024-04-12T20:41:53.779 DEBUG:teuthology.orchestra.run.smithi186:> sudo yum -y install rbd-nbd 2024-04-12T20:41:54.438 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : rbd-mirror-2:17.2.7-827.gd3b5060a.el9.x86_64 1/1 2024-04-12T20:41:54.438 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-12T20:41:54.438 INFO:teuthology.orchestra.run.smithi018.stdout:Installed: 2024-04-12T20:41:54.438 INFO:teuthology.orchestra.run.smithi018.stdout: rbd-mirror-2:17.2.7-827.gd3b5060a.el9.x86_64 2024-04-12T20:41:54.438 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-12T20:41:54.438 INFO:teuthology.orchestra.run.smithi018.stdout:Complete! 2024-04-12T20:41:54.674 INFO:teuthology.orchestra.run.smithi186.stdout:Last metadata expiration check: 0:02:17 ago on Fri 12 Apr 2024 08:39:37 PM UTC. 2024-04-12T20:41:54.762 DEBUG:teuthology.orchestra.run.smithi018:> sudo yum -y install rbd-nbd 2024-04-12T20:41:54.969 INFO:teuthology.orchestra.run.smithi186.stderr:Modular dependency problems: 2024-04-12T20:41:54.969 INFO:teuthology.orchestra.run.smithi186.stderr: 2024-04-12T20:41:54.970 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 1: conflicting requests 2024-04-12T20:41:54.970 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module container-tools:rhel8:820240321091303:20125149.x86_64 from CentOS-AppStream 2024-04-12T20:41:54.970 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 2: conflicting requests 2024-04-12T20:41:54.970 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module eclipse:rhel8:820201023100746:b230ed4e.x86_64 from CentOS-AppStream 2024-04-12T20:41:54.970 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 3: conflicting requests 2024-04-12T20:41:54.970 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module freeradius:3.0:8080020230117180605:89170a74.x86_64 from CentOS-AppStream 2024-04-12T20:41:54.970 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 4: conflicting requests 2024-04-12T20:41:54.970 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module gimp:2.8:8000020190628145146:4148dfdf.x86_64 from CentOS-AppStream 2024-04-12T20:41:54.970 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 5: conflicting requests 2024-04-12T20:41:54.970 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module go-toolset:rhel8:820240401160413:b754926a.x86_64 from CentOS-AppStream 2024-04-12T20:41:54.970 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 6: conflicting requests 2024-04-12T20:41:54.970 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module httpd:2.4:820240216084734:9edba152.x86_64 from CentOS-AppStream 2024-04-12T20:41:54.970 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 7: conflicting requests 2024-04-12T20:41:54.971 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module idm:client:820240307184541:49cc9d1b.x86_64 from CentOS-AppStream 2024-04-12T20:41:54.971 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 8: conflicting requests 2024-04-12T20:41:54.971 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module inkscape:0.92.3:8080020221205124429:3e031279.x86_64 from CentOS-AppStream 2024-04-12T20:41:54.971 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 9: conflicting requests 2024-04-12T20:41:54.971 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module jmc:rhel8:8050020211110222101:6392b1f8.x86_64 from CentOS-AppStream 2024-04-12T20:41:54.971 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 10: conflicting requests 2024-04-12T20:41:54.971 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module llvm-toolset:rhel8:820240205133135:9edba152.x86_64 from CentOS-AppStream 2024-04-12T20:41:54.971 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 11: conflicting requests 2024-04-12T20:41:54.971 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module log4j:2:8080020221205124743:9d367344.x86_64 from CentOS-AppStream 2024-04-12T20:41:54.971 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 12: conflicting requests 2024-04-12T20:41:54.971 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module mailman:2.1:820230727085757:77fc8825.x86_64 from CentOS-AppStream 2024-04-12T20:41:54.971 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 13: conflicting requests 2024-04-12T20:41:54.971 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module mariadb:10.3:8030020210419150013:30b713e6.x86_64 from CentOS-AppStream 2024-04-12T20:41:54.971 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 14: conflicting requests 2024-04-12T20:41:54.972 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module maven:3.5:8060020211203142015:c0229ad2.x86_64 from CentOS-AppStream 2024-04-12T20:41:54.972 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 15: conflicting requests 2024-04-12T20:41:54.972 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module mercurial:4.8:8000020190628020724:4148dfdf.x86_64 from CentOS-AppStream 2024-04-12T20:41:54.972 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 16: conflicting requests 2024-04-12T20:41:54.972 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module mod_auth_openidc:2.3:820230721123832:9edba152.x86_64 from CentOS-AppStream 2024-04-12T20:41:54.972 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 17: conflicting requests 2024-04-12T20:41:54.972 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module mysql:8.0:8040020210901180257:9f9e2e7e.x86_64 from CentOS-AppStream 2024-04-12T20:41:54.972 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 18: conflicting requests 2024-04-12T20:41:54.972 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module nginx:1.14:8000020211221191913:55190bc5.x86_64 from CentOS-AppStream 2024-04-12T20:41:54.972 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 19: conflicting requests 2024-04-12T20:41:54.972 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module nodejs:10:8040020210120182536:9f9e2e7e.x86_64 from CentOS-AppStream 2024-04-12T20:41:54.972 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 20: conflicting requests 2024-04-12T20:41:54.972 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl:5.26:8000020190628020724:55190bc5.x86_64 from CentOS-AppStream 2024-04-12T20:41:54.972 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 21: conflicting requests 2024-04-12T20:41:54.973 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module php:7.2:8020020200507003613:2c7ca891.x86_64 from CentOS-AppStream 2024-04-12T20:41:54.973 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 22: conflicting requests 2024-04-12T20:41:54.973 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module postgresql:10:820230223114800:9edba152.x86_64 from CentOS-AppStream 2024-04-12T20:41:54.973 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 23: conflicting requests 2024-04-12T20:41:54.973 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module python27:2.7:820240208011952:182f7c73.x86_64 from CentOS-AppStream 2024-04-12T20:41:54.973 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 24: conflicting requests 2024-04-12T20:41:54.973 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module python36:3.6:8050020210825152031:982725ab.x86_64 from CentOS-AppStream 2024-04-12T20:41:54.973 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 25: conflicting requests 2024-04-12T20:41:54.973 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module python38:3.8:820230810143931:64c1cd5a.x86_64 from CentOS-AppStream 2024-04-12T20:41:54.973 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 26: conflicting requests 2024-04-12T20:41:54.973 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module python39:3.9:820240214182535:17377f89.x86_64 from CentOS-AppStream 2024-04-12T20:41:54.973 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 27: conflicting requests 2024-04-12T20:41:54.973 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module redis:5:8040020211019153849:9f9e2e7e.x86_64 from CentOS-AppStream 2024-04-12T20:41:54.974 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 28: conflicting requests 2024-04-12T20:41:54.974 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module rhn-tools:1.0:8010020191114034948:f69d1239.x86_64 from CentOS-AppStream 2024-04-12T20:41:54.974 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 29: conflicting requests 2024-04-12T20:41:54.974 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module ruby:2.5:820230627084142:9edba152.x86_64 from CentOS-AppStream 2024-04-12T20:41:54.974 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 30: conflicting requests 2024-04-12T20:41:54.974 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module rust-toolset:rhel8:820240119204620:b09eea91.x86_64 from CentOS-AppStream 2024-04-12T20:41:54.974 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 31: conflicting requests 2024-04-12T20:41:54.974 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module satellite-5-client:1.0:8010020191114035551:cdc1202b.x86_64 from CentOS-AppStream 2024-04-12T20:41:54.974 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 32: conflicting requests 2024-04-12T20:41:54.974 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module squid:4:820240319173245:9edba152.x86_64 from CentOS-AppStream 2024-04-12T20:41:54.974 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 33: conflicting requests 2024-04-12T20:41:54.974 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module subversion:1.10:8070020220711155714:78111232.x86_64 from CentOS-AppStream 2024-04-12T20:41:54.974 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 34: conflicting requests 2024-04-12T20:41:54.974 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module swig:3.0:8030020200706142531:30b713e6.x86_64 from CentOS-AppStream 2024-04-12T20:41:54.975 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 35: conflicting requests 2024-04-12T20:41:54.975 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module varnish:6:8050020211109225449:b4937e53.x86_64 from CentOS-AppStream 2024-04-12T20:41:54.975 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 36: conflicting requests 2024-04-12T20:41:54.975 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module virt:rhel:820240314161907:9edba152.x86_64 from CentOS-AppStream 2024-04-12T20:41:54.975 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 37: conflicting requests 2024-04-12T20:41:54.975 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:8030020200716155257:7cc0a66d.x86_64 from CentOS-AppStream 2024-04-12T20:41:54.975 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:8030020200716155257:b967a9a2.x86_64 from CentOS-AppStream 2024-04-12T20:41:54.975 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:820230811133638:36f9fae6.x86_64 from CentOS-AppStream 2024-04-12T20:41:54.975 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 38: cannot install the best candidate for the job 2024-04-12T20:41:54.975 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:8010020191114031501:a5949e2e.x86_64 from CentOS-AppStream 2024-04-12T20:41:54.975 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:8010020191114031501:be2e4737.x86_64 from CentOS-AppStream 2024-04-12T20:41:54.975 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:820230223114759:2ab761e1.x86_64 from CentOS-AppStream 2024-04-12T20:41:54.975 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:820230223114759:86e45846.x86_64 from CentOS-AppStream 2024-04-12T20:41:54.975 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 39: conflicting requests 2024-04-12T20:41:54.976 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:8030020200715230104:1e4bbb35.x86_64 from CentOS-AppStream 2024-04-12T20:41:54.976 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:8030020200715230104:ea09926d.x86_64 from CentOS-AppStream 2024-04-12T20:41:54.976 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:082fdf2f.x86_64 from CentOS-AppStream 2024-04-12T20:41:54.976 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-04-12T20:41:54.976 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-04-12T20:41:54.976 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:fbe42456.x86_64 from CentOS-AppStream 2024-04-12T20:41:54.976 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 40: cannot install the best candidate for the job 2024-04-12T20:41:54.976 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:8010020191114031513:16b3ab4d.x86_64 from CentOS-AppStream 2024-04-12T20:41:54.976 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:8010020191114031513:e1f37755.x86_64 from CentOS-AppStream 2024-04-12T20:41:54.976 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-04-12T20:41:54.976 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-04-12T20:41:54.976 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 41: cannot install the best candidate for the job 2024-04-12T20:41:54.976 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:8030020200716150652:1e4bbb35.x86_64 from CentOS-AppStream 2024-04-12T20:41:54.976 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:8030020200716150652:ea09926d.x86_64 from CentOS-AppStream 2024-04-12T20:41:54.977 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-04-12T20:41:54.977 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-04-12T20:41:54.977 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 42: cannot install the best candidate for the job 2024-04-12T20:41:54.977 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:8030020200716174729:3a70019f.x86_64 from CentOS-AppStream 2024-04-12T20:41:54.977 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:8030020200716174729:cccafca5.x86_64 from CentOS-AppStream 2024-04-12T20:41:54.977 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:820230223114759:6fdf7e5d.x86_64 from CentOS-AppStream 2024-04-12T20:41:54.977 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:820230223114759:d4326aba.x86_64 from CentOS-AppStream 2024-04-12T20:41:54.977 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 43: cannot install the best candidate for the job 2024-04-12T20:41:54.977 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:8030020200716213634:56fce90f.x86_64 from CentOS-AppStream 2024-04-12T20:41:54.977 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:8030020200716213634:98a3c9d0.x86_64 from CentOS-AppStream 2024-04-12T20:41:54.977 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:820230223114759:06b7596e.x86_64 from CentOS-AppStream 2024-04-12T20:41:54.977 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:820230223114759:d2fc5d76.x86_64 from CentOS-AppStream 2024-04-12T20:41:54.977 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 44: cannot install the best candidate for the job 2024-04-12T20:41:54.977 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:8030020200716171139:3a70019f.x86_64 from CentOS-AppStream 2024-04-12T20:41:54.977 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:8030020200716171139:cccafca5.x86_64 from CentOS-AppStream 2024-04-12T20:41:54.978 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:820230223114759:6fdf7e5d.x86_64 from CentOS-AppStream 2024-04-12T20:41:54.978 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:820230223114759:d4326aba.x86_64 from CentOS-AppStream 2024-04-12T20:41:54.978 INFO:teuthology.orchestra.run.smithi186.stderr: Problem 45: cannot install the best candidate for the job 2024-04-12T20:41:54.978 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:8010020191120175858:95fce66d.x86_64 from CentOS-AppStream 2024-04-12T20:41:54.978 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:8030020200723233049:09acf126.x86_64 from CentOS-AppStream 2024-04-12T20:41:54.978 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:820230223114759:436af1ae.x86_64 from CentOS-AppStream 2024-04-12T20:41:54.978 INFO:teuthology.orchestra.run.smithi186.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:820230223114759:b12195ea.x86_64 from CentOS-AppStream 2024-04-12T20:41:55.039 INFO:teuthology.orchestra.run.smithi186.stdout:Dependencies resolved. 2024-04-12T20:41:55.040 INFO:teuthology.orchestra.run.smithi186.stdout:================================================================================ 2024-04-12T20:41:55.040 INFO:teuthology.orchestra.run.smithi186.stdout: Package Architecture Version Repository Size 2024-04-12T20:41:55.041 INFO:teuthology.orchestra.run.smithi186.stdout:================================================================================ 2024-04-12T20:41:55.041 INFO:teuthology.orchestra.run.smithi186.stdout:Installing: 2024-04-12T20:41:55.041 INFO:teuthology.orchestra.run.smithi186.stdout: rbd-nbd x86_64 2:17.2.7-827.gd3b5060a.el9 ceph 165 k 2024-04-12T20:41:55.041 INFO:teuthology.orchestra.run.smithi186.stdout: 2024-04-12T20:41:55.041 INFO:teuthology.orchestra.run.smithi186.stdout:Transaction Summary 2024-04-12T20:41:55.041 INFO:teuthology.orchestra.run.smithi186.stdout:================================================================================ 2024-04-12T20:41:55.041 INFO:teuthology.orchestra.run.smithi186.stdout:Install 1 Package 2024-04-12T20:41:55.041 INFO:teuthology.orchestra.run.smithi186.stdout: 2024-04-12T20:41:55.041 INFO:teuthology.orchestra.run.smithi186.stdout:Total download size: 165 k 2024-04-12T20:41:55.041 INFO:teuthology.orchestra.run.smithi186.stdout:Installed size: 477 k 2024-04-12T20:41:55.041 INFO:teuthology.orchestra.run.smithi186.stdout:Downloading Packages: 2024-04-12T20:41:55.313 INFO:teuthology.orchestra.run.smithi186.stdout:rbd-nbd-17.2.7-827.gd3b5060a.el9.x86_64.rpm 608 kB/s | 165 kB 00:00 2024-04-12T20:41:55.314 INFO:teuthology.orchestra.run.smithi186.stdout:-------------------------------------------------------------------------------- 2024-04-12T20:41:55.315 INFO:teuthology.orchestra.run.smithi186.stdout:Total 602 kB/s | 165 kB 00:00 2024-04-12T20:41:55.316 INFO:teuthology.orchestra.run.smithi186.stdout:Running transaction check 2024-04-12T20:41:55.342 INFO:teuthology.orchestra.run.smithi186.stdout:Transaction check succeeded. 2024-04-12T20:41:55.342 INFO:teuthology.orchestra.run.smithi186.stdout:Running transaction test 2024-04-12T20:41:55.448 INFO:teuthology.orchestra.run.smithi186.stdout:Transaction test succeeded. 2024-04-12T20:41:55.448 INFO:teuthology.orchestra.run.smithi186.stdout:Running transaction 2024-04-12T20:41:55.654 INFO:teuthology.orchestra.run.smithi186.stdout: Preparing : 1/1 2024-04-12T20:41:55.702 INFO:teuthology.orchestra.run.smithi018.stdout:Last metadata expiration check: 0:02:20 ago on Fri 12 Apr 2024 08:39:35 PM UTC. 2024-04-12T20:41:55.811 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : rbd-nbd-2:17.2.7-827.gd3b5060a.el9.x86_64 1/1 2024-04-12T20:41:56.018 INFO:teuthology.orchestra.run.smithi018.stderr:Modular dependency problems: 2024-04-12T20:41:56.018 INFO:teuthology.orchestra.run.smithi018.stderr: 2024-04-12T20:41:56.018 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 1: conflicting requests 2024-04-12T20:41:56.018 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module container-tools:rhel8:820240321091303:20125149.x86_64 from CentOS-AppStream 2024-04-12T20:41:56.018 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 2: conflicting requests 2024-04-12T20:41:56.018 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module eclipse:rhel8:820201023100746:b230ed4e.x86_64 from CentOS-AppStream 2024-04-12T20:41:56.018 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 3: conflicting requests 2024-04-12T20:41:56.018 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module freeradius:3.0:8080020230117180605:89170a74.x86_64 from CentOS-AppStream 2024-04-12T20:41:56.019 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 4: conflicting requests 2024-04-12T20:41:56.019 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module gimp:2.8:8000020190628145146:4148dfdf.x86_64 from CentOS-AppStream 2024-04-12T20:41:56.019 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 5: conflicting requests 2024-04-12T20:41:56.019 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module go-toolset:rhel8:820240401160413:b754926a.x86_64 from CentOS-AppStream 2024-04-12T20:41:56.019 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 6: conflicting requests 2024-04-12T20:41:56.019 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module httpd:2.4:820240216084734:9edba152.x86_64 from CentOS-AppStream 2024-04-12T20:41:56.019 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 7: conflicting requests 2024-04-12T20:41:56.019 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module idm:client:820240307184541:49cc9d1b.x86_64 from CentOS-AppStream 2024-04-12T20:41:56.019 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 8: conflicting requests 2024-04-12T20:41:56.019 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module inkscape:0.92.3:8080020221205124429:3e031279.x86_64 from CentOS-AppStream 2024-04-12T20:41:56.019 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 9: conflicting requests 2024-04-12T20:41:56.020 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module jmc:rhel8:8050020211110222101:6392b1f8.x86_64 from CentOS-AppStream 2024-04-12T20:41:56.020 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 10: conflicting requests 2024-04-12T20:41:56.020 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module llvm-toolset:rhel8:820240205133135:9edba152.x86_64 from CentOS-AppStream 2024-04-12T20:41:56.020 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 11: conflicting requests 2024-04-12T20:41:56.020 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module log4j:2:8080020221205124743:9d367344.x86_64 from CentOS-AppStream 2024-04-12T20:41:56.020 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 12: conflicting requests 2024-04-12T20:41:56.020 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module mailman:2.1:820230727085757:77fc8825.x86_64 from CentOS-AppStream 2024-04-12T20:41:56.020 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 13: conflicting requests 2024-04-12T20:41:56.020 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module mariadb:10.3:8030020210419150013:30b713e6.x86_64 from CentOS-AppStream 2024-04-12T20:41:56.020 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 14: conflicting requests 2024-04-12T20:41:56.020 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module maven:3.5:8060020211203142015:c0229ad2.x86_64 from CentOS-AppStream 2024-04-12T20:41:56.021 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 15: conflicting requests 2024-04-12T20:41:56.021 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module mercurial:4.8:8000020190628020724:4148dfdf.x86_64 from CentOS-AppStream 2024-04-12T20:41:56.021 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 16: conflicting requests 2024-04-12T20:41:56.021 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module mod_auth_openidc:2.3:820230721123832:9edba152.x86_64 from CentOS-AppStream 2024-04-12T20:41:56.021 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 17: conflicting requests 2024-04-12T20:41:56.021 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module mysql:8.0:8040020210901180257:9f9e2e7e.x86_64 from CentOS-AppStream 2024-04-12T20:41:56.021 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 18: conflicting requests 2024-04-12T20:41:56.021 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module nginx:1.14:8000020211221191913:55190bc5.x86_64 from CentOS-AppStream 2024-04-12T20:41:56.021 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 19: conflicting requests 2024-04-12T20:41:56.021 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module nodejs:10:8040020210120182536:9f9e2e7e.x86_64 from CentOS-AppStream 2024-04-12T20:41:56.021 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 20: conflicting requests 2024-04-12T20:41:56.022 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl:5.26:8000020190628020724:55190bc5.x86_64 from CentOS-AppStream 2024-04-12T20:41:56.022 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 21: conflicting requests 2024-04-12T20:41:56.022 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module php:7.2:8020020200507003613:2c7ca891.x86_64 from CentOS-AppStream 2024-04-12T20:41:56.022 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 22: conflicting requests 2024-04-12T20:41:56.022 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module postgresql:10:820230223114800:9edba152.x86_64 from CentOS-AppStream 2024-04-12T20:41:56.022 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 23: conflicting requests 2024-04-12T20:41:56.022 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module python27:2.7:820240208011952:182f7c73.x86_64 from CentOS-AppStream 2024-04-12T20:41:56.022 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 24: conflicting requests 2024-04-12T20:41:56.022 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module python36:3.6:8050020210825152031:982725ab.x86_64 from CentOS-AppStream 2024-04-12T20:41:56.022 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 25: conflicting requests 2024-04-12T20:41:56.022 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module python38:3.8:820230810143931:64c1cd5a.x86_64 from CentOS-AppStream 2024-04-12T20:41:56.023 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 26: conflicting requests 2024-04-12T20:41:56.023 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module python39:3.9:820240214182535:17377f89.x86_64 from CentOS-AppStream 2024-04-12T20:41:56.023 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 27: conflicting requests 2024-04-12T20:41:56.023 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module redis:5:8040020211019153849:9f9e2e7e.x86_64 from CentOS-AppStream 2024-04-12T20:41:56.023 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 28: conflicting requests 2024-04-12T20:41:56.023 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module rhn-tools:1.0:8010020191114034948:f69d1239.x86_64 from CentOS-AppStream 2024-04-12T20:41:56.023 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 29: conflicting requests 2024-04-12T20:41:56.023 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module ruby:2.5:820230627084142:9edba152.x86_64 from CentOS-AppStream 2024-04-12T20:41:56.023 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 30: conflicting requests 2024-04-12T20:41:56.023 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module rust-toolset:rhel8:820240119204620:b09eea91.x86_64 from CentOS-AppStream 2024-04-12T20:41:56.023 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 31: conflicting requests 2024-04-12T20:41:56.023 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module satellite-5-client:1.0:8010020191114035551:cdc1202b.x86_64 from CentOS-AppStream 2024-04-12T20:41:56.024 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 32: conflicting requests 2024-04-12T20:41:56.024 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module squid:4:820240319173245:9edba152.x86_64 from CentOS-AppStream 2024-04-12T20:41:56.024 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 33: conflicting requests 2024-04-12T20:41:56.024 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module subversion:1.10:8070020220711155714:78111232.x86_64 from CentOS-AppStream 2024-04-12T20:41:56.024 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 34: conflicting requests 2024-04-12T20:41:56.024 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module swig:3.0:8030020200706142531:30b713e6.x86_64 from CentOS-AppStream 2024-04-12T20:41:56.024 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 35: conflicting requests 2024-04-12T20:41:56.024 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module varnish:6:8050020211109225449:b4937e53.x86_64 from CentOS-AppStream 2024-04-12T20:41:56.024 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 36: conflicting requests 2024-04-12T20:41:56.024 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module virt:rhel:820240314161907:9edba152.x86_64 from CentOS-AppStream 2024-04-12T20:41:56.024 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 37: conflicting requests 2024-04-12T20:41:56.024 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:8030020200716155257:7cc0a66d.x86_64 from CentOS-AppStream 2024-04-12T20:41:56.024 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:8030020200716155257:b967a9a2.x86_64 from CentOS-AppStream 2024-04-12T20:41:56.024 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:820230811133638:36f9fae6.x86_64 from CentOS-AppStream 2024-04-12T20:41:56.025 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 38: cannot install the best candidate for the job 2024-04-12T20:41:56.025 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:8010020191114031501:a5949e2e.x86_64 from CentOS-AppStream 2024-04-12T20:41:56.025 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:8010020191114031501:be2e4737.x86_64 from CentOS-AppStream 2024-04-12T20:41:56.025 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:820230223114759:2ab761e1.x86_64 from CentOS-AppStream 2024-04-12T20:41:56.025 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:820230223114759:86e45846.x86_64 from CentOS-AppStream 2024-04-12T20:41:56.025 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 39: conflicting requests 2024-04-12T20:41:56.025 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:8030020200715230104:1e4bbb35.x86_64 from CentOS-AppStream 2024-04-12T20:41:56.025 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:8030020200715230104:ea09926d.x86_64 from CentOS-AppStream 2024-04-12T20:41:56.025 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:082fdf2f.x86_64 from CentOS-AppStream 2024-04-12T20:41:56.025 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-04-12T20:41:56.025 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-04-12T20:41:56.025 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:fbe42456.x86_64 from CentOS-AppStream 2024-04-12T20:41:56.025 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 40: cannot install the best candidate for the job 2024-04-12T20:41:56.025 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:8010020191114031513:16b3ab4d.x86_64 from CentOS-AppStream 2024-04-12T20:41:56.026 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:8010020191114031513:e1f37755.x86_64 from CentOS-AppStream 2024-04-12T20:41:56.026 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-04-12T20:41:56.026 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-04-12T20:41:56.026 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 41: cannot install the best candidate for the job 2024-04-12T20:41:56.026 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:8030020200716150652:1e4bbb35.x86_64 from CentOS-AppStream 2024-04-12T20:41:56.026 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:8030020200716150652:ea09926d.x86_64 from CentOS-AppStream 2024-04-12T20:41:56.026 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-04-12T20:41:56.026 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-04-12T20:41:56.026 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 42: cannot install the best candidate for the job 2024-04-12T20:41:56.026 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:8030020200716174729:3a70019f.x86_64 from CentOS-AppStream 2024-04-12T20:41:56.026 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:8030020200716174729:cccafca5.x86_64 from CentOS-AppStream 2024-04-12T20:41:56.026 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:820230223114759:6fdf7e5d.x86_64 from CentOS-AppStream 2024-04-12T20:41:56.026 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:820230223114759:d4326aba.x86_64 from CentOS-AppStream 2024-04-12T20:41:56.027 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 43: cannot install the best candidate for the job 2024-04-12T20:41:56.027 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:8030020200716213634:56fce90f.x86_64 from CentOS-AppStream 2024-04-12T20:41:56.027 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:8030020200716213634:98a3c9d0.x86_64 from CentOS-AppStream 2024-04-12T20:41:56.027 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:820230223114759:06b7596e.x86_64 from CentOS-AppStream 2024-04-12T20:41:56.027 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:820230223114759:d2fc5d76.x86_64 from CentOS-AppStream 2024-04-12T20:41:56.027 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 44: cannot install the best candidate for the job 2024-04-12T20:41:56.027 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:8030020200716171139:3a70019f.x86_64 from CentOS-AppStream 2024-04-12T20:41:56.027 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:8030020200716171139:cccafca5.x86_64 from CentOS-AppStream 2024-04-12T20:41:56.027 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:820230223114759:6fdf7e5d.x86_64 from CentOS-AppStream 2024-04-12T20:41:56.027 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:820230223114759:d4326aba.x86_64 from CentOS-AppStream 2024-04-12T20:41:56.027 INFO:teuthology.orchestra.run.smithi018.stderr: Problem 45: cannot install the best candidate for the job 2024-04-12T20:41:56.027 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:8010020191120175858:95fce66d.x86_64 from CentOS-AppStream 2024-04-12T20:41:56.027 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:8030020200723233049:09acf126.x86_64 from CentOS-AppStream 2024-04-12T20:41:56.027 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:820230223114759:436af1ae.x86_64 from CentOS-AppStream 2024-04-12T20:41:56.028 INFO:teuthology.orchestra.run.smithi018.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:820230223114759:b12195ea.x86_64 from CentOS-AppStream 2024-04-12T20:41:56.092 INFO:teuthology.orchestra.run.smithi018.stdout:Dependencies resolved. 2024-04-12T20:41:56.093 INFO:teuthology.orchestra.run.smithi018.stdout:================================================================================ 2024-04-12T20:41:56.093 INFO:teuthology.orchestra.run.smithi018.stdout: Package Architecture Version Repository Size 2024-04-12T20:41:56.093 INFO:teuthology.orchestra.run.smithi018.stdout:================================================================================ 2024-04-12T20:41:56.094 INFO:teuthology.orchestra.run.smithi018.stdout:Installing: 2024-04-12T20:41:56.094 INFO:teuthology.orchestra.run.smithi018.stdout: rbd-nbd x86_64 2:17.2.7-827.gd3b5060a.el9 ceph 165 k 2024-04-12T20:41:56.094 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-12T20:41:56.094 INFO:teuthology.orchestra.run.smithi018.stdout:Transaction Summary 2024-04-12T20:41:56.094 INFO:teuthology.orchestra.run.smithi018.stdout:================================================================================ 2024-04-12T20:41:56.094 INFO:teuthology.orchestra.run.smithi018.stdout:Install 1 Package 2024-04-12T20:41:56.094 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-12T20:41:56.094 INFO:teuthology.orchestra.run.smithi018.stdout:Total download size: 165 k 2024-04-12T20:41:56.094 INFO:teuthology.orchestra.run.smithi018.stdout:Installed size: 477 k 2024-04-12T20:41:56.094 INFO:teuthology.orchestra.run.smithi018.stdout:Downloading Packages: 2024-04-12T20:41:56.386 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: rbd-nbd-2:17.2.7-827.gd3b5060a.el9.x86_64 1/1 2024-04-12T20:41:56.476 INFO:teuthology.orchestra.run.smithi018.stdout:rbd-nbd-17.2.7-827.gd3b5060a.el9.x86_64.rpm 433 kB/s | 165 kB 00:00 2024-04-12T20:41:56.478 INFO:teuthology.orchestra.run.smithi018.stdout:-------------------------------------------------------------------------------- 2024-04-12T20:41:56.478 INFO:teuthology.orchestra.run.smithi018.stdout:Total 429 kB/s | 165 kB 00:00 2024-04-12T20:41:56.479 INFO:teuthology.orchestra.run.smithi018.stdout:Running transaction check 2024-04-12T20:41:56.506 INFO:teuthology.orchestra.run.smithi018.stdout:Transaction check succeeded. 2024-04-12T20:41:56.507 INFO:teuthology.orchestra.run.smithi018.stdout:Running transaction test 2024-04-12T20:41:56.612 INFO:teuthology.orchestra.run.smithi018.stdout:Transaction test succeeded. 2024-04-12T20:41:56.613 INFO:teuthology.orchestra.run.smithi018.stdout:Running transaction 2024-04-12T20:41:56.725 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : rbd-nbd-2:17.2.7-827.gd3b5060a.el9.x86_64 1/1 2024-04-12T20:41:56.726 INFO:teuthology.orchestra.run.smithi186.stdout: 2024-04-12T20:41:56.726 INFO:teuthology.orchestra.run.smithi186.stdout:Installed: 2024-04-12T20:41:56.726 INFO:teuthology.orchestra.run.smithi186.stdout: rbd-nbd-2:17.2.7-827.gd3b5060a.el9.x86_64 2024-04-12T20:41:56.726 INFO:teuthology.orchestra.run.smithi186.stdout: 2024-04-12T20:41:56.726 INFO:teuthology.orchestra.run.smithi186.stdout:Complete! 2024-04-12T20:41:56.840 INFO:teuthology.orchestra.run.smithi018.stdout: Preparing : 1/1 2024-04-12T20:41:56.952 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : rbd-nbd-2:17.2.7-827.gd3b5060a.el9.x86_64 1/1 2024-04-12T20:41:57.105 DEBUG:teuthology.parallel:result is None 2024-04-12T20:41:57.540 INFO:teuthology.orchestra.run.smithi018.stdout: Running scriptlet: rbd-nbd-2:17.2.7-827.gd3b5060a.el9.x86_64 1/1 2024-04-12T20:41:57.965 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : rbd-nbd-2:17.2.7-827.gd3b5060a.el9.x86_64 1/1 2024-04-12T20:41:57.965 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-12T20:41:57.966 INFO:teuthology.orchestra.run.smithi018.stdout:Installed: 2024-04-12T20:41:57.966 INFO:teuthology.orchestra.run.smithi018.stdout: rbd-nbd-2:17.2.7-827.gd3b5060a.el9.x86_64 2024-04-12T20:41:57.966 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-12T20:41:57.966 INFO:teuthology.orchestra.run.smithi018.stdout:Complete! 2024-04-12T20:41:58.280 DEBUG:teuthology.parallel:result is None 2024-04-12T20:41:58.281 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-04-12T20:41:58.281 INFO:teuthology.packaging:ref: None 2024-04-12T20:41:58.281 INFO:teuthology.packaging:tag: None 2024-04-12T20:41:58.281 INFO:teuthology.packaging:branch: quincy 2024-04-12T20:41:58.281 INFO:teuthology.packaging:sha1: 324baf8ddb91dea46cac50cb535c306115791355 2024-04-12T20:41:58.281 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=quincy 2024-04-12T20:41:58.619 DEBUG:teuthology.orchestra.run.smithi018:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-04-12T20:41:58.688 INFO:teuthology.orchestra.run.smithi018.stdout:17.2.7-827.gd3b5060a.el9 2024-04-12T20:41:58.689 INFO:teuthology.packaging:The installed version of ceph is 17.2.7-827.gd3b5060a.el9 2024-04-12T20:41:58.689 INFO:teuthology.task.install:The correct ceph version 17.2.7-827.gd3b5060a is installed. 2024-04-12T20:41:58.690 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-04-12T20:41:58.690 INFO:teuthology.packaging:ref: None 2024-04-12T20:41:58.691 INFO:teuthology.packaging:tag: None 2024-04-12T20:41:58.691 INFO:teuthology.packaging:branch: quincy 2024-04-12T20:41:58.691 INFO:teuthology.packaging:sha1: 324baf8ddb91dea46cac50cb535c306115791355 2024-04-12T20:41:58.691 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=quincy 2024-04-12T20:41:58.907 DEBUG:teuthology.orchestra.run.smithi186:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-04-12T20:41:58.972 INFO:teuthology.orchestra.run.smithi186.stdout:17.2.7-827.gd3b5060a.el9 2024-04-12T20:41:58.972 INFO:teuthology.packaging:The installed version of ceph is 17.2.7-827.gd3b5060a.el9 2024-04-12T20:41:58.972 INFO:teuthology.task.install:The correct ceph version 17.2.7-827.gd3b5060a is installed. 2024-04-12T20:41:58.974 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-04-12T20:41:58.974 DEBUG:teuthology.orchestra.run.smithi018:> set -ex 2024-04-12T20:41:58.975 DEBUG:teuthology.orchestra.run.smithi018:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-04-12T20:41:59.075 DEBUG:teuthology.orchestra.run.smithi186:> set -ex 2024-04-12T20:41:59.075 DEBUG:teuthology.orchestra.run.smithi186:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-04-12T20:41:59.179 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-04-12T20:41:59.179 DEBUG:teuthology.orchestra.run.smithi018:> set -ex 2024-04-12T20:41:59.179 DEBUG:teuthology.orchestra.run.smithi018:> sudo dd of=/usr/bin/daemon-helper 2024-04-12T20:41:59.284 DEBUG:teuthology.orchestra.run.smithi018:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-04-12T20:41:59.427 DEBUG:teuthology.orchestra.run.smithi186:> set -ex 2024-04-12T20:41:59.427 DEBUG:teuthology.orchestra.run.smithi186:> sudo dd of=/usr/bin/daemon-helper 2024-04-12T20:41:59.530 DEBUG:teuthology.orchestra.run.smithi186:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-04-12T20:41:59.664 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-04-12T20:41:59.664 DEBUG:teuthology.orchestra.run.smithi018:> set -ex 2024-04-12T20:41:59.664 DEBUG:teuthology.orchestra.run.smithi018:> sudo dd of=/usr/bin/adjust-ulimits 2024-04-12T20:41:59.768 DEBUG:teuthology.orchestra.run.smithi018:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-04-12T20:41:59.914 DEBUG:teuthology.orchestra.run.smithi186:> set -ex 2024-04-12T20:41:59.914 DEBUG:teuthology.orchestra.run.smithi186:> sudo dd of=/usr/bin/adjust-ulimits 2024-04-12T20:42:00.014 DEBUG:teuthology.orchestra.run.smithi186:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-04-12T20:42:00.156 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-04-12T20:42:00.157 DEBUG:teuthology.orchestra.run.smithi018:> set -ex 2024-04-12T20:42:00.157 DEBUG:teuthology.orchestra.run.smithi018:> sudo dd of=/usr/bin/stdin-killer 2024-04-12T20:42:00.266 DEBUG:teuthology.orchestra.run.smithi018:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-04-12T20:42:00.371 DEBUG:teuthology.orchestra.run.smithi186:> set -ex 2024-04-12T20:42:00.371 DEBUG:teuthology.orchestra.run.smithi186:> sudo dd of=/usr/bin/stdin-killer 2024-04-12T20:42:00.468 DEBUG:teuthology.orchestra.run.smithi186:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-04-12T20:42:00.562 INFO:teuthology.run_tasks:Running task print... 2024-04-12T20:42:00.572 INFO:teuthology.task.print:**** done install task... 2024-04-12T20:42:00.572 INFO:teuthology.run_tasks:Running task cephadm... 2024-04-12T20:42:00.717 INFO:tasks.cephadm:Config: {'cephadm_branch': 'quincy', 'cephadm_git_url': 'https://github.com/ceph/ceph', 'conf': {'osd': {'osd_class_default_list': '*', 'osd_class_load_list': '*', 'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}, 'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'mon pg warn min per osd': 0}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120}}, 'image': 'quay.ceph.io/ceph-ci/ceph:quincy', 'roleless': True, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'OSD_DOWN', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability'], 'sha1': '324baf8ddb91dea46cac50cb535c306115791355'} 2024-04-12T20:42:00.718 INFO:tasks.cephadm:Cluster image is quay.ceph.io/ceph-ci/ceph:quincy 2024-04-12T20:42:00.718 INFO:tasks.cephadm:Cluster fsid is 1cbe407a-f90d-11ee-bc8d-c7b262605968 2024-04-12T20:42:00.718 INFO:tasks.cephadm:Choosing monitor IPs and ports... 2024-04-12T20:42:00.718 INFO:tasks.cephadm:No mon roles; fabricating mons 2024-04-12T20:42:00.719 INFO:tasks.cephadm:Monitor IPs: {'mon.smithi018': '172.21.15.18', 'mon.smithi186': '172.21.15.186'} 2024-04-12T20:42:00.719 INFO:tasks.cephadm:Normalizing hostnames... 2024-04-12T20:42:00.719 DEBUG:teuthology.orchestra.run.smithi018:> sudo hostname $(hostname -s) 2024-04-12T20:42:00.827 DEBUG:teuthology.orchestra.run.smithi186:> sudo hostname $(hostname -s) 2024-04-12T20:42:00.929 INFO:tasks.cephadm:Downloading cephadm (repo https://github.com/ceph/ceph ref quincy)... 2024-04-12T20:42:00.929 DEBUG:teuthology.orchestra.run.smithi018:> curl --silent https://raw.githubusercontent.com/ceph/ceph/quincy/src/cephadm/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-04-12T20:42:01.263 INFO:teuthology.orchestra.run.smithi018.stdout:-rw-r--r--. 1 ubuntu ubuntu 375153 Apr 12 20:42 /home/ubuntu/cephtest/cephadm 2024-04-12T20:42:01.263 DEBUG:teuthology.orchestra.run.smithi186:> curl --silent https://raw.githubusercontent.com/ceph/ceph/quincy/src/cephadm/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-04-12T20:42:01.517 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r--. 1 ubuntu ubuntu 375153 Apr 12 20:42 /home/ubuntu/cephtest/cephadm 2024-04-12T20:42:01.517 DEBUG:teuthology.orchestra.run.smithi018:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-04-12T20:42:01.581 DEBUG:teuthology.orchestra.run.smithi186:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-04-12T20:42:01.651 INFO:tasks.cephadm:Pulling image quay.ceph.io/ceph-ci/ceph:quincy on all hosts... 2024-04-12T20:42:01.652 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy pull 2024-04-12T20:42:01.659 DEBUG:teuthology.orchestra.run.smithi186:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy pull 2024-04-12T20:42:02.801 INFO:teuthology.orchestra.run.smithi186.stderr:Pulling container image quay.ceph.io/ceph-ci/ceph:quincy... 2024-04-12T20:42:02.904 INFO:teuthology.orchestra.run.smithi018.stderr:Pulling container image quay.ceph.io/ceph-ci/ceph:quincy... 2024-04-12T20:42:36.214 INFO:teuthology.orchestra.run.smithi186.stdout:{ 2024-04-12T20:42:36.214 INFO:teuthology.orchestra.run.smithi186.stdout: "ceph_version": "ceph version 17.2.7-827-gd3b5060a (d3b5060a17449edfae9385cdf792afec109fca5d) quincy (stable)", 2024-04-12T20:42:36.214 INFO:teuthology.orchestra.run.smithi186.stdout: "image_id": "21de9b7369ad8457bc44c6aca16b9dfec25bf172fb7b6ebd80f97d4df2005241", 2024-04-12T20:42:36.214 INFO:teuthology.orchestra.run.smithi186.stdout: "repo_digests": [ 2024-04-12T20:42:36.214 INFO:teuthology.orchestra.run.smithi186.stdout: "quay.ceph.io/ceph-ci/ceph@sha256:5e334df698f432180b78bc9c46936bfdfbc92bc9a8983463e523f780d23765fe" 2024-04-12T20:42:36.214 INFO:teuthology.orchestra.run.smithi186.stdout: ] 2024-04-12T20:42:36.214 INFO:teuthology.orchestra.run.smithi186.stdout:} 2024-04-12T20:42:38.011 INFO:teuthology.orchestra.run.smithi018.stdout:{ 2024-04-12T20:42:38.011 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph_version": "ceph version 17.2.7-827-gd3b5060a (d3b5060a17449edfae9385cdf792afec109fca5d) quincy (stable)", 2024-04-12T20:42:38.011 INFO:teuthology.orchestra.run.smithi018.stdout: "image_id": "21de9b7369ad8457bc44c6aca16b9dfec25bf172fb7b6ebd80f97d4df2005241", 2024-04-12T20:42:38.011 INFO:teuthology.orchestra.run.smithi018.stdout: "repo_digests": [ 2024-04-12T20:42:38.012 INFO:teuthology.orchestra.run.smithi018.stdout: "quay.ceph.io/ceph-ci/ceph@sha256:5e334df698f432180b78bc9c46936bfdfbc92bc9a8983463e523f780d23765fe" 2024-04-12T20:42:38.012 INFO:teuthology.orchestra.run.smithi018.stdout: ] 2024-04-12T20:42:38.012 INFO:teuthology.orchestra.run.smithi018.stdout:} 2024-04-12T20:42:38.067 DEBUG:teuthology.orchestra.run.smithi018:> sudo mkdir -p /etc/ceph 2024-04-12T20:42:38.170 DEBUG:teuthology.orchestra.run.smithi186:> sudo mkdir -p /etc/ceph 2024-04-12T20:42:38.274 DEBUG:teuthology.orchestra.run.smithi018:> sudo chmod 777 /etc/ceph 2024-04-12T20:42:38.377 DEBUG:teuthology.orchestra.run.smithi186:> sudo chmod 777 /etc/ceph 2024-04-12T20:42:38.476 INFO:tasks.cephadm:Writing seed config... 2024-04-12T20:42:38.477 INFO:tasks.cephadm: override: [osd] osd_class_default_list = * 2024-04-12T20:42:38.477 INFO:tasks.cephadm: override: [osd] osd_class_load_list = * 2024-04-12T20:42:38.477 INFO:tasks.cephadm: override: [osd] bdev async discard = True 2024-04-12T20:42:38.477 INFO:tasks.cephadm: override: [osd] bdev enable discard = True 2024-04-12T20:42:38.477 INFO:tasks.cephadm: override: [osd] bluestore allocator = bitmap 2024-04-12T20:42:38.478 INFO:tasks.cephadm: override: [osd] bluestore block size = 96636764160 2024-04-12T20:42:38.478 INFO:tasks.cephadm: override: [osd] bluestore fsck on mount = True 2024-04-12T20:42:38.478 INFO:tasks.cephadm: override: [osd] debug bluefs = 1/20 2024-04-12T20:42:38.478 INFO:tasks.cephadm: override: [osd] debug bluestore = 1/20 2024-04-12T20:42:38.478 INFO:tasks.cephadm: override: [osd] debug ms = 1 2024-04-12T20:42:38.478 INFO:tasks.cephadm: override: [osd] debug osd = 20 2024-04-12T20:42:38.478 INFO:tasks.cephadm: override: [osd] debug rocksdb = 4/10 2024-04-12T20:42:38.478 INFO:tasks.cephadm: override: [osd] mon osd backfillfull_ratio = 0.85 2024-04-12T20:42:38.478 INFO:tasks.cephadm: override: [osd] mon osd full ratio = 0.9 2024-04-12T20:42:38.478 INFO:tasks.cephadm: override: [osd] mon osd nearfull ratio = 0.8 2024-04-12T20:42:38.478 INFO:tasks.cephadm: override: [osd] osd failsafe full ratio = 0.95 2024-04-12T20:42:38.478 INFO:tasks.cephadm: override: [osd] osd objectstore = bluestore 2024-04-12T20:42:38.478 INFO:tasks.cephadm: override: [osd] osd op complaint time = 180 2024-04-12T20:42:38.479 INFO:tasks.cephadm: override: [client] client mount timeout = 600 2024-04-12T20:42:38.479 INFO:tasks.cephadm: override: [client] debug client = 20 2024-04-12T20:42:38.479 INFO:tasks.cephadm: override: [client] debug ms = 1 2024-04-12T20:42:38.479 INFO:tasks.cephadm: override: [client] rados mon op timeout = 900 2024-04-12T20:42:38.479 INFO:tasks.cephadm: override: [client] rados osd op timeout = 900 2024-04-12T20:42:38.479 INFO:tasks.cephadm: override: [global] mon pg warn min per osd = 0 2024-04-12T20:42:38.479 INFO:tasks.cephadm: override: [mds] debug mds = 20 2024-04-12T20:42:38.479 INFO:tasks.cephadm: override: [mds] debug mds balancer = 20 2024-04-12T20:42:38.479 INFO:tasks.cephadm: override: [mds] debug ms = 1 2024-04-12T20:42:38.479 INFO:tasks.cephadm: override: [mds] mds debug frag = True 2024-04-12T20:42:38.479 INFO:tasks.cephadm: override: [mds] mds debug scatterstat = True 2024-04-12T20:42:38.479 INFO:tasks.cephadm: override: [mds] mds op complaint time = 180 2024-04-12T20:42:38.479 INFO:tasks.cephadm: override: [mds] mds verify scatter = True 2024-04-12T20:42:38.480 INFO:tasks.cephadm: override: [mds] osd op complaint time = 180 2024-04-12T20:42:38.480 INFO:tasks.cephadm: override: [mds] rados mon op timeout = 900 2024-04-12T20:42:38.480 INFO:tasks.cephadm: override: [mds] rados osd op timeout = 900 2024-04-12T20:42:38.480 INFO:tasks.cephadm: override: [mgr] debug client = 20 2024-04-12T20:42:38.480 INFO:tasks.cephadm: override: [mgr] debug mgr = 20 2024-04-12T20:42:38.480 INFO:tasks.cephadm: override: [mgr] debug ms = 1 2024-04-12T20:42:38.480 INFO:tasks.cephadm: override: [mon] debug mon = 20 2024-04-12T20:42:38.480 INFO:tasks.cephadm: override: [mon] debug ms = 1 2024-04-12T20:42:38.480 INFO:tasks.cephadm: override: [mon] debug paxos = 20 2024-04-12T20:42:38.480 INFO:tasks.cephadm: override: [mon] mon down mkfs grace = 300 2024-04-12T20:42:38.480 INFO:tasks.cephadm: override: [mon] mon op complaint time = 120 2024-04-12T20:42:38.481 DEBUG:teuthology.orchestra.run.smithi018:> set -ex 2024-04-12T20:42:38.481 DEBUG:teuthology.orchestra.run.smithi018:> dd of=/home/ubuntu/cephtest/seed.ceph.conf 2024-04-12T20:42:38.542 DEBUG:tasks.cephadm:Final config: [global] # make logging friendly to teuthology log_to_file = true log_to_stderr = false log to journald = false mon cluster log to file = true mon cluster log file level = debug mon clock drift allowed = 1.000 # replicate across OSDs, not hosts osd crush chooseleaf type = 0 #osd pool default size = 2 osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd # enable some debugging auth debug = true ms die on old message = true ms die on bug = true debug asserts on shutdown = true # adjust warnings mon max pg per osd = 10000# >= luminous mon pg warn max object skew = 0 mon osd allow primary affinity = true mon osd allow pg remap = true mon warn on legacy crush tunables = false mon warn on crush straw calc version zero = false mon warn on no sortbitwise = false mon warn on osd down out interval zero = false mon warn on too few osds = false mon_warn_on_pool_pg_num_not_power_of_two = false # disable pg_autoscaler by default for new pools osd_pool_default_pg_autoscale_mode = off # tests delete pools mon allow pool delete = true fsid = 1cbe407a-f90d-11ee-bc8d-c7b262605968 mon pg warn min per osd = 0 [osd] osd scrub load threshold = 5.0 osd scrub max interval = 600 osd mclock profile = high_recovery_ops osd recover clone overlap = true osd recovery max chunk = 1048576 osd deep scrub update digest min age = 30 osd map max advance = 10 osd memory target autotune = true # debugging osd debug shutdown = true osd debug op order = true osd debug verify stray on activate = true osd debug pg log writeout = true osd debug verify cached snaps = true osd debug verify missing on start = true osd debug misdirected ops = true osd op queue = debug_random osd op queue cut off = debug_random osd shutdown pgref assert = true bdev debug aio = true osd sloppy crc = true osd_class_default_list = * osd_class_load_list = * bdev async discard = True bdev enable discard = True bluestore allocator = bitmap bluestore block size = 96636764160 bluestore fsck on mount = True debug bluefs = 1/20 debug bluestore = 1/20 debug ms = 1 debug osd = 20 debug rocksdb = 4/10 mon osd backfillfull_ratio = 0.85 mon osd full ratio = 0.9 mon osd nearfull ratio = 0.8 osd failsafe full ratio = 0.95 osd objectstore = bluestore osd op complaint time = 180 [mgr] mon reweight min pgs per osd = 4 mon reweight min bytes per osd = 10 mgr/telemetry/nag = false debug client = 20 debug mgr = 20 debug ms = 1 [mon] mon data avail warn = 5 mon mgr mkfs grace = 240 mon reweight min pgs per osd = 4 mon osd reporter subtree level = osd mon osd prime pg temp = true mon reweight min bytes per osd = 10 # rotate auth tickets quickly to exercise renewal paths auth mon ticket ttl = 660# 11m auth service ticket ttl = 240# 4m # don't complain about global id reclaim mon_warn_on_insecure_global_id_reclaim = false mon_warn_on_insecure_global_id_reclaim_allowed = false debug mon = 20 debug ms = 1 debug paxos = 20 mon down mkfs grace = 300 mon op complaint time = 120 [client.rgw] rgw cache enabled = true rgw enable ops log = true rgw enable usage log = true [client] client mount timeout = 600 debug client = 20 debug ms = 1 rados mon op timeout = 900 rados osd op timeout = 900 [mds] debug mds = 20 debug mds balancer = 20 debug ms = 1 mds debug frag = True mds debug scatterstat = True mds op complaint time = 180 mds verify scatter = True osd op complaint time = 180 rados mon op timeout = 900 rados osd op timeout = 900 2024-04-12T20:42:38.543 DEBUG:teuthology.orchestra.run.smithi018:mon.smithi018> sudo journalctl -f -n 0 -u ceph-1cbe407a-f90d-11ee-bc8d-c7b262605968@mon.smithi018.service 2024-04-12T20:42:38.590 INFO:tasks.cephadm:Bootstrapping... 2024-04-12T20:42:38.590 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy -v bootstrap --fsid 1cbe407a-f90d-11ee-bc8d-c7b262605968 --config /home/ubuntu/cephtest/seed.ceph.conf --output-config /etc/ceph/ceph.conf --output-keyring /etc/ceph/ceph.client.admin.keyring --output-pub-ssh-key /home/ubuntu/cephtest/ceph.pub --mon-ip 172.21.15.18 --skip-admin-label && sudo chmod +r /etc/ceph/ceph.client.admin.keyring 2024-04-12T20:42:38.907 INFO:teuthology.orchestra.run.smithi018.stdout:-------------------------------------------------------------------------------- 2024-04-12T20:42:38.907 INFO:teuthology.orchestra.run.smithi018.stdout:cephadm ['--image', 'quay.ceph.io/ceph-ci/ceph:quincy', '-v', 'bootstrap', '--fsid', '1cbe407a-f90d-11ee-bc8d-c7b262605968', '--config', '/home/ubuntu/cephtest/seed.ceph.conf', '--output-config', '/etc/ceph/ceph.conf', '--output-keyring', '/etc/ceph/ceph.client.admin.keyring', '--output-pub-ssh-key', '/home/ubuntu/cephtest/ceph.pub', '--mon-ip', '172.21.15.18', '--skip-admin-label'] 2024-04-12T20:42:38.964 INFO:teuthology.orchestra.run.smithi018.stdout:/bin/podman: stdout 4.9.4-dev 2024-04-12T20:42:38.965 INFO:teuthology.orchestra.run.smithi018.stderr:Specifying an fsid for your cluster offers no advantages and may increase the likelihood of fsid conflicts. 2024-04-12T20:42:38.965 INFO:teuthology.orchestra.run.smithi018.stdout:Verifying podman|docker is present... 2024-04-12T20:42:39.017 INFO:teuthology.orchestra.run.smithi018.stdout:/bin/podman: stdout 4.9.4-dev 2024-04-12T20:42:39.018 INFO:teuthology.orchestra.run.smithi018.stdout:Verifying lvm2 is present... 2024-04-12T20:42:39.018 INFO:teuthology.orchestra.run.smithi018.stdout:Verifying time synchronization is in place... 2024-04-12T20:42:39.035 INFO:teuthology.orchestra.run.smithi018.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-04-12T20:42:39.035 INFO:teuthology.orchestra.run.smithi018.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-04-12T20:42:39.053 INFO:teuthology.orchestra.run.smithi018.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-04-12T20:42:39.054 INFO:teuthology.orchestra.run.smithi018.stdout:systemctl: stdout inactive 2024-04-12T20:42:39.078 INFO:teuthology.orchestra.run.smithi018.stdout:systemctl: stdout enabled 2024-04-12T20:42:39.099 INFO:teuthology.orchestra.run.smithi018.stdout:systemctl: stdout active 2024-04-12T20:42:39.099 INFO:teuthology.orchestra.run.smithi018.stdout:Unit chronyd.service is enabled and running 2024-04-12T20:42:39.099 INFO:teuthology.orchestra.run.smithi018.stdout:Repeating the final host check... 2024-04-12T20:42:39.153 INFO:teuthology.orchestra.run.smithi018.stdout:/bin/podman: stdout 4.9.4-dev 2024-04-12T20:42:39.153 INFO:teuthology.orchestra.run.smithi018.stdout:podman (/bin/podman) version 4.9.4 is present 2024-04-12T20:42:39.153 INFO:teuthology.orchestra.run.smithi018.stdout:systemctl is present 2024-04-12T20:42:39.153 INFO:teuthology.orchestra.run.smithi018.stdout:lvcreate is present 2024-04-12T20:42:39.168 INFO:teuthology.orchestra.run.smithi018.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-04-12T20:42:39.168 INFO:teuthology.orchestra.run.smithi018.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-04-12T20:42:39.183 INFO:teuthology.orchestra.run.smithi018.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-04-12T20:42:39.183 INFO:teuthology.orchestra.run.smithi018.stdout:systemctl: stdout inactive 2024-04-12T20:42:39.204 INFO:teuthology.orchestra.run.smithi018.stdout:systemctl: stdout enabled 2024-04-12T20:42:39.224 INFO:teuthology.orchestra.run.smithi018.stdout:systemctl: stdout active 2024-04-12T20:42:39.224 INFO:teuthology.orchestra.run.smithi018.stdout:Unit chronyd.service is enabled and running 2024-04-12T20:42:39.225 INFO:teuthology.orchestra.run.smithi018.stdout:Host looks OK 2024-04-12T20:42:39.225 INFO:teuthology.orchestra.run.smithi018.stdout:Cluster fsid: 1cbe407a-f90d-11ee-bc8d-c7b262605968 2024-04-12T20:42:39.225 INFO:teuthology.orchestra.run.smithi018.stdout:Acquiring lock 139953634690144 on /run/cephadm/1cbe407a-f90d-11ee-bc8d-c7b262605968.lock 2024-04-12T20:42:39.226 INFO:teuthology.orchestra.run.smithi018.stdout:Lock 139953634690144 acquired on /run/cephadm/1cbe407a-f90d-11ee-bc8d-c7b262605968.lock 2024-04-12T20:42:39.227 INFO:teuthology.orchestra.run.smithi018.stdout:Verifying IP 172.21.15.18 port 3300 ... 2024-04-12T20:42:39.227 INFO:teuthology.orchestra.run.smithi018.stdout:Verifying IP 172.21.15.18 port 6789 ... 2024-04-12T20:42:39.228 INFO:teuthology.orchestra.run.smithi018.stdout:Base mon IP(s) is [172.21.15.18:3300, 172.21.15.18:6789], mon addrv is [v2:172.21.15.18:3300,v1:172.21.15.18:6789] 2024-04-12T20:42:39.242 INFO:teuthology.orchestra.run.smithi018.stdout:/sbin/ip: stdout default via 172.21.15.254 dev ens1f0 proto dhcp src 172.21.15.18 metric 100 2024-04-12T20:42:39.242 INFO:teuthology.orchestra.run.smithi018.stdout:/sbin/ip: stdout 172.21.0.0/20 dev ens1f0 proto kernel scope link src 172.21.15.18 metric 100 2024-04-12T20:42:39.254 INFO:teuthology.orchestra.run.smithi018.stdout:/sbin/ip: stdout ::1 dev lo proto kernel metric 256 pref medium 2024-04-12T20:42:39.254 INFO:teuthology.orchestra.run.smithi018.stdout:/sbin/ip: stdout fe80::/64 dev ens1f0 proto kernel metric 1024 pref medium 2024-04-12T20:42:39.254 INFO:teuthology.orchestra.run.smithi018.stdout:/sbin/ip: stdout default via fe80::327c:5e00:6487:71e0 dev ens1f0 proto ra metric 100 pref medium 2024-04-12T20:42:39.265 INFO:teuthology.orchestra.run.smithi018.stdout:/sbin/ip: stdout 1: lo: mtu 65536 state UNKNOWN qlen 1000 2024-04-12T20:42:39.266 INFO:teuthology.orchestra.run.smithi018.stdout:/sbin/ip: stdout inet6 ::1/128 scope host 2024-04-12T20:42:39.266 INFO:teuthology.orchestra.run.smithi018.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-04-12T20:42:39.266 INFO:teuthology.orchestra.run.smithi018.stdout:/sbin/ip: stdout 4: ens1f0: mtu 1500 state UP qlen 1000 2024-04-12T20:42:39.266 INFO:teuthology.orchestra.run.smithi018.stdout:/sbin/ip: stdout inet6 fe80::ec4:7aff:febd:1362/64 scope link noprefixroute 2024-04-12T20:42:39.266 INFO:teuthology.orchestra.run.smithi018.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-04-12T20:42:39.268 INFO:teuthology.orchestra.run.smithi018.stdout:Mon IP `172.21.15.18` is in CIDR network `172.21.0.0/20` 2024-04-12T20:42:39.269 INFO:teuthology.orchestra.run.smithi018.stdout:Mon IP `172.21.15.18` is in CIDR network `172.21.0.0/20` 2024-04-12T20:42:39.269 INFO:teuthology.orchestra.run.smithi018.stdout:Inferred mon public CIDR from local network configuration ['172.21.0.0/20', '172.21.0.0/20'] 2024-04-12T20:42:39.271 INFO:teuthology.orchestra.run.smithi018.stdout:Internal network (--cluster-network) has not been provided, OSD replication will default to the public_network 2024-04-12T20:42:39.273 INFO:teuthology.orchestra.run.smithi018.stdout:Pulling container image quay.ceph.io/ceph-ci/ceph:quincy... 2024-04-12T20:42:40.052 INFO:teuthology.orchestra.run.smithi018.stdout:/bin/podman: stdout 21de9b7369ad8457bc44c6aca16b9dfec25bf172fb7b6ebd80f97d4df2005241 2024-04-12T20:42:40.052 INFO:teuthology.orchestra.run.smithi018.stdout:/bin/podman: stderr Trying to pull quay.ceph.io/ceph-ci/ceph:quincy... 2024-04-12T20:42:40.052 INFO:teuthology.orchestra.run.smithi018.stdout:/bin/podman: stderr Getting image source signatures 2024-04-12T20:42:40.052 INFO:teuthology.orchestra.run.smithi018.stdout:/bin/podman: stderr Copying blob sha256:6a38c9a31ea4c6c1e3d079c8edc2f3b418b00f3ce1d3a48ffe92dcb2ef7b4e51 2024-04-12T20:42:40.052 INFO:teuthology.orchestra.run.smithi018.stdout:/bin/podman: stderr Copying blob sha256:9793b5b5582798bde394f4383d142e9f248a25e4ebfd72ee05ea9ec6251a877d 2024-04-12T20:42:40.052 INFO:teuthology.orchestra.run.smithi018.stdout:/bin/podman: stderr Copying config sha256:21de9b7369ad8457bc44c6aca16b9dfec25bf172fb7b6ebd80f97d4df2005241 2024-04-12T20:42:40.052 INFO:teuthology.orchestra.run.smithi018.stdout:/bin/podman: stderr Writing manifest to image destination 2024-04-12T20:42:41.239 INFO:teuthology.orchestra.run.smithi018.stdout:ceph: stdout ceph version 17.2.7-827-gd3b5060a (d3b5060a17449edfae9385cdf792afec109fca5d) quincy (stable) 2024-04-12T20:42:41.240 INFO:teuthology.orchestra.run.smithi018.stdout:Ceph version: ceph version 17.2.7-827-gd3b5060a (d3b5060a17449edfae9385cdf792afec109fca5d) quincy (stable) 2024-04-12T20:42:41.241 INFO:teuthology.orchestra.run.smithi018.stdout:Extracting ceph user uid/gid from container image... 2024-04-12T20:42:52.111 INFO:teuthology.orchestra.run.smithi018.stdout:stat: stdout 167 167 2024-04-12T20:42:52.111 INFO:teuthology.orchestra.run.smithi018.stdout:Creating initial keys... 2024-04-12T20:42:53.261 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph-authtool: stdout AQDMnBlmydm7IxAAtjb7mSoxC0jRtByXn3/GcQ== 2024-04-12T20:42:54.499 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph-authtool: stdout AQDNnBlmkTrLMRAAT/qSVFZqRIwgnvS47pcgGA== 2024-04-12T20:42:55.641 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph-authtool: stdout AQDPnBlmSUdnABAAHPPvZICT7AxWgqhG7730iQ== 2024-04-12T20:42:55.644 INFO:teuthology.orchestra.run.smithi018.stdout:Creating initial monmap... 2024-04-12T20:42:56.773 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: monmap file /tmp/monmap 2024-04-12T20:42:56.773 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/monmaptool: stdout setting min_mon_release = octopus 2024-04-12T20:42:56.773 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: set fsid to 1cbe407a-f90d-11ee-bc8d-c7b262605968 2024-04-12T20:42:56.774 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-04-12T20:42:56.774 INFO:teuthology.orchestra.run.smithi018.stdout:monmaptool for smithi018 [v2:172.21.15.18:3300,v1:172.21.15.18:6789] on /usr/bin/monmaptool: monmap file /tmp/monmap 2024-04-12T20:42:56.774 INFO:teuthology.orchestra.run.smithi018.stdout:setting min_mon_release = octopus 2024-04-12T20:42:56.774 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/monmaptool: set fsid to 1cbe407a-f90d-11ee-bc8d-c7b262605968 2024-04-12T20:42:56.774 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-04-12T20:42:56.774 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-12T20:42:56.775 INFO:teuthology.orchestra.run.smithi018.stdout:Creating mon... 2024-04-12T20:42:58.409 INFO:teuthology.orchestra.run.smithi018.stdout:create mon.smithi018 on 2024-04-12T20:42:58.778 INFO:teuthology.orchestra.run.smithi018.stdout:systemctl: stderr Removed "/etc/systemd/system/multi-user.target.wants/ceph.target". 2024-04-12T20:42:59.011 INFO:teuthology.orchestra.run.smithi018.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /etc/systemd/system/ceph.target. 2024-04-12T20:42:59.275 INFO:teuthology.orchestra.run.smithi018.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph-1cbe407a-f90d-11ee-bc8d-c7b262605968.target → /etc/systemd/system/ceph-1cbe407a-f90d-11ee-bc8d-c7b262605968.target. 2024-04-12T20:42:59.275 INFO:teuthology.orchestra.run.smithi018.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph.target.wants/ceph-1cbe407a-f90d-11ee-bc8d-c7b262605968.target → /etc/systemd/system/ceph-1cbe407a-f90d-11ee-bc8d-c7b262605968.target. 2024-04-12T20:42:59.592 INFO:teuthology.orchestra.run.smithi018.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-1cbe407a-f90d-11ee-bc8d-c7b262605968@mon.smithi018 2024-04-12T20:42:59.593 INFO:teuthology.orchestra.run.smithi018.stdout:systemctl: stderr Failed to reset failed state of unit ceph-1cbe407a-f90d-11ee-bc8d-c7b262605968@mon.smithi018.service: Unit ceph-1cbe407a-f90d-11ee-bc8d-c7b262605968@mon.smithi018.service not loaded. 2024-04-12T20:42:59.908 INFO:teuthology.orchestra.run.smithi018.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-1cbe407a-f90d-11ee-bc8d-c7b262605968.target.wants/ceph-1cbe407a-f90d-11ee-bc8d-c7b262605968@mon.smithi018.service → /etc/systemd/system/ceph-1cbe407a-f90d-11ee-bc8d-c7b262605968@.service. 2024-04-12T20:43:00.606 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:43:00 smithi018 podman[28413]: 2024-04-12 20:43:00.369741344 +0000 UTC m=+0.178042848 container create 851ef11d1ec1cc3b0356f09000f3b3b2f7a7c78924d1fede7be81426bea4fca6 (image=quay.ceph.io/ceph-ci/ceph:quincy, name=ceph-1cbe407a-f90d-11ee-bc8d-c7b262605968-mon-smithi018, org.label-schema.build-date=20240409, maintainer=Guillaume Abrioux , ceph=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, io.buildah.version=1.33.5, RELEASE=quincy-d3b5060, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD) 2024-04-12T20:43:00.606 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:43:00 smithi018 podman[28413]: 2024-04-12 20:43:00.604838618 +0000 UTC m=+0.413140192 container init 851ef11d1ec1cc3b0356f09000f3b3b2f7a7c78924d1fede7be81426bea4fca6 (image=quay.ceph.io/ceph-ci/ceph:quincy, name=ceph-1cbe407a-f90d-11ee-bc8d-c7b262605968-mon-smithi018, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, GIT_CLEAN=True, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240409, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, RELEASE=quincy-d3b5060, ceph=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-12T20:43:00.722 INFO:teuthology.orchestra.run.smithi018.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-04-12T20:43:00.723 INFO:teuthology.orchestra.run.smithi018.stdout:systemctl: stdout disabled 2024-04-12T20:43:00.738 INFO:teuthology.orchestra.run.smithi018.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-04-12T20:43:00.738 INFO:teuthology.orchestra.run.smithi018.stdout:systemctl: stdout inactive 2024-04-12T20:43:00.738 INFO:teuthology.orchestra.run.smithi018.stdout:firewalld.service is not enabled 2024-04-12T20:43:00.738 INFO:teuthology.orchestra.run.smithi018.stdout:Not possible to enable service . firewalld.service is not available 2024-04-12T20:43:00.739 INFO:teuthology.orchestra.run.smithi018.stdout:Waiting for mon to start... 2024-04-12T20:43:00.739 INFO:teuthology.orchestra.run.smithi018.stdout:Waiting for mon... 2024-04-12T20:43:00.890 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:43:00 smithi018 podman[28413]: 2024-04-12 20:43:00.617172236 +0000 UTC m=+0.425473740 container start 851ef11d1ec1cc3b0356f09000f3b3b2f7a7c78924d1fede7be81426bea4fca6 (image=quay.ceph.io/ceph-ci/ceph:quincy, name=ceph-1cbe407a-f90d-11ee-bc8d-c7b262605968-mon-smithi018, maintainer=Guillaume Abrioux , RELEASE=quincy-d3b5060, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.build-date=20240409) 2024-04-12T20:43:00.890 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:43:00 smithi018 bash[28413]: 851ef11d1ec1cc3b0356f09000f3b3b2f7a7c78924d1fede7be81426bea4fca6 2024-04-12T20:43:00.890 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:43:00 smithi018 ceph-mon[28431]: mkfs 1cbe407a-f90d-11ee-bc8d-c7b262605968 2024-04-12T20:43:00.890 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:43:00 smithi018 ceph-mon[28431]: mon.smithi018 is new leader, mons smithi018 in quorum (ranks 0) 2024-04-12T20:43:00.890 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:43:00 smithi018 systemd[1]: Started Ceph mon.smithi018 for 1cbe407a-f90d-11ee-bc8d-c7b262605968. 2024-04-12T20:43:02.028 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:43:01 smithi018 ceph-mon[28431]: mon.smithi018 is new leader, mons smithi018 in quorum (ranks 0) 2024-04-12T20:43:02.028 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:43:01 smithi018 ceph-mon[28431]: monmap e1: 1 mons at {smithi018=[v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-04-12T20:43:02.028 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:43:01 smithi018 ceph-mon[28431]: fsmap 2024-04-12T20:43:02.028 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:43:01 smithi018 ceph-mon[28431]: osdmap e1: 0 total, 0 up, 0 in 2024-04-12T20:43:02.028 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:43:01 smithi018 ceph-mon[28431]: mgrmap e1: no daemons active 2024-04-12T20:43:02.028 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:43:01 smithi018 ceph-mon[28431]: from='client.? 172.21.15.18:0/1975446664' entity='client.admin' cmd=[{"prefix": "status"}]: dispatch 2024-04-12T20:43:02.447 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout cluster: 2024-04-12T20:43:02.447 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout id: 1cbe407a-f90d-11ee-bc8d-c7b262605968 2024-04-12T20:43:02.447 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout health: HEALTH_OK 2024-04-12T20:43:02.447 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout 2024-04-12T20:43:02.447 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout services: 2024-04-12T20:43:02.448 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout mon: 1 daemons, quorum smithi018 (age 0.748487s) 2024-04-12T20:43:02.448 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout mgr: no daemons active 2024-04-12T20:43:02.448 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout osd: 0 osds: 0 up, 0 in 2024-04-12T20:43:02.448 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout 2024-04-12T20:43:02.448 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout data: 2024-04-12T20:43:02.448 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout pools: 0 pools, 0 pgs 2024-04-12T20:43:02.448 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout objects: 0 objects, 0 B 2024-04-12T20:43:02.448 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout usage: 0 B used, 0 B / 0 B avail 2024-04-12T20:43:02.448 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout pgs: 2024-04-12T20:43:02.448 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout 2024-04-12T20:43:02.448 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:01.345+0000 7f5b4a800700 1 Processor -- start 2024-04-12T20:43:02.448 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:01.345+0000 7f5b4a800700 1 -- start start 2024-04-12T20:43:02.448 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:01.345+0000 7f5b4a800700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5b4407cf90 0x7f5b4410bcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T20:43:02.448 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:01.345+0000 7f5b4a800700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b4407d430 con 0x7f5b4407cf90 2024-04-12T20:43:02.449 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:01.346+0000 7f5b43400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5b4407cf90 0x7f5b4410bcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T20:43:02.449 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:01.347+0000 7f5b43400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5b4407cf90 0x7f5b4410bcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:45566/0 (socket says 172.21.15.18:45566) 2024-04-12T20:43:02.449 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:01.347+0000 7f5b43400700 1 -- 172.21.15.18:0/22080127 learned_addr learned my addr 172.21.15.18:0/22080127 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T20:43:02.449 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:01.347+0000 7f5b43400700 1 -- 172.21.15.18:0/22080127 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b4410c210 con 0x7f5b4407cf90 2024-04-12T20:43:02.449 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:01.348+0000 7f5b43400700 1 --2- 172.21.15.18:0/22080127 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5b4407cf90 0x7f5b4410bcd0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f5b340098e0 tx=0x7f5b34009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bfc7a43c98c3828b server_cookie=0 in_seq=0 out_seq=0 2024-04-12T20:43:02.449 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:01.348+0000 7f5b42000700 1 -- 172.21.15.18:0/22080127 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f5b3400b860 con 0x7f5b4407cf90 2024-04-12T20:43:02.449 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:01.348+0000 7f5b42000700 1 -- 172.21.15.18:0/22080127 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b3400b9c0 con 0x7f5b4407cf90 2024-04-12T20:43:02.449 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:01.348+0000 7f5b42000700 1 -- 172.21.15.18:0/22080127 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f5b3400bc90 con 0x7f5b4407cf90 2024-04-12T20:43:02.449 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:01.349+0000 7f5b4a800700 1 -- 172.21.15.18:0/22080127 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5b4407cf90 msgr2=0x7f5b4410bcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:43:02.449 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:01.349+0000 7f5b4a800700 1 --2- 172.21.15.18:0/22080127 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5b4407cf90 0x7f5b4410bcd0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f5b340098e0 tx=0x7f5b34009250 comp rx=0 tx=0).stop 2024-04-12T20:43:02.449 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:01.349+0000 7f5b4a800700 1 -- 172.21.15.18:0/22080127 shutdown_connections 2024-04-12T20:43:02.449 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:01.349+0000 7f5b4a800700 1 --2- 172.21.15.18:0/22080127 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5b4407cf90 0x7f5b4410bcd0 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:43:02.449 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:01.349+0000 7f5b4a800700 1 -- 172.21.15.18:0/22080127 >> 172.21.15.18:0/22080127 conn(0x7f5b440782e0 msgr2=0x7f5b4407a6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T20:43:02.449 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:01.349+0000 7f5b4a800700 1 -- 172.21.15.18:0/22080127 shutdown_connections 2024-04-12T20:43:02.450 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:01.349+0000 7f5b4a800700 1 -- 172.21.15.18:0/22080127 wait complete. 2024-04-12T20:43:02.450 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:01.350+0000 7f5b4a800700 1 Processor -- start 2024-04-12T20:43:02.450 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:01.350+0000 7f5b4a800700 1 -- start start 2024-04-12T20:43:02.450 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:01.351+0000 7f5b4a800700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5b4407cf90 0x7f5b441173f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T20:43:02.450 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:01.351+0000 7f5b4a800700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b44117930 con 0x7f5b4407cf90 2024-04-12T20:43:02.450 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:01.351+0000 7f5b43400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5b4407cf90 0x7f5b441173f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T20:43:02.450 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:01.351+0000 7f5b43400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5b4407cf90 0x7f5b441173f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:45570/0 (socket says 172.21.15.18:45570) 2024-04-12T20:43:02.450 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:01.351+0000 7f5b43400700 1 -- 172.21.15.18:0/1975446664 learned_addr learned my addr 172.21.15.18:0/1975446664 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T20:43:02.450 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:01.352+0000 7f5b43400700 1 -- 172.21.15.18:0/1975446664 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b340094f0 con 0x7f5b4407cf90 2024-04-12T20:43:02.450 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:01.352+0000 7f5b43400700 1 --2- 172.21.15.18:0/1975446664 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5b4407cf90 0x7f5b441173f0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f5b34009290 tx=0x7f5b34016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T20:43:02.450 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:01.352+0000 7f5b3b400700 1 -- 172.21.15.18:0/1975446664 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f5b3401c5c0 con 0x7f5b4407cf90 2024-04-12T20:43:02.450 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:01.352+0000 7f5b3b400700 1 -- 172.21.15.18:0/1975446664 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b3401c740 con 0x7f5b4407cf90 2024-04-12T20:43:02.450 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:01.352+0000 7f5b3b400700 1 -- 172.21.15.18:0/1975446664 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f5b3401ca50 con 0x7f5b4407cf90 2024-04-12T20:43:02.451 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:01.352+0000 7f5b4a800700 1 -- 172.21.15.18:0/1975446664 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5b44117b30 con 0x7f5b4407cf90 2024-04-12T20:43:02.451 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:01.352+0000 7f5b4a800700 1 -- 172.21.15.18:0/1975446664 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5b441186a0 con 0x7f5b4407cf90 2024-04-12T20:43:02.451 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:01.353+0000 7f5b3b400700 1 -- 172.21.15.18:0/1975446664 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 1) v1 ==== 521+0+0 (secure 0 0 0) 0x7f5b3401a450 con 0x7f5b4407cf90 2024-04-12T20:43:02.451 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:01.353+0000 7f5b3b400700 1 -- 172.21.15.18:0/1975446664 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 721+0+0 (secure 0 0 0) 0x7f5b3402e790 con 0x7f5b4407cf90 2024-04-12T20:43:02.451 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:01.354+0000 7f5b4a800700 1 -- 172.21.15.18:0/1975446664 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5b4405f5f0 con 0x7f5b4407cf90 2024-04-12T20:43:02.451 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:01.357+0000 7f5b3b400700 1 -- 172.21.15.18:0/1975446664 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71349 (secure 0 0 0) 0x7f5b3401ce30 con 0x7f5b4407cf90 2024-04-12T20:43:02.451 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:01.437+0000 7f5b4a800700 1 -- 172.21.15.18:0/1975446664 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "status"} v 0) v1 -- 0x7f5b44119170 con 0x7f5b4407cf90 2024-04-12T20:43:02.451 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:01.438+0000 7f5b3b400700 1 -- 172.21.15.18:0/1975446664 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "status"}]=0 v0) v1 ==== 54+0+325 (secure 0 0 0) 0x7f5b3401a890 con 0x7f5b4407cf90 2024-04-12T20:43:02.451 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:01.439+0000 7f5b4a800700 1 -- 172.21.15.18:0/1975446664 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5b4407cf90 msgr2=0x7f5b441173f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:43:02.451 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:01.439+0000 7f5b4a800700 1 --2- 172.21.15.18:0/1975446664 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5b4407cf90 0x7f5b441173f0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f5b34009290 tx=0x7f5b34016000 comp rx=0 tx=0).stop 2024-04-12T20:43:02.451 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:01.439+0000 7f5b4a800700 1 -- 172.21.15.18:0/1975446664 shutdown_connections 2024-04-12T20:43:02.451 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:01.439+0000 7f5b4a800700 1 --2- 172.21.15.18:0/1975446664 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5b4407cf90 0x7f5b441173f0 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:43:02.451 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:01.439+0000 7f5b4a800700 1 -- 172.21.15.18:0/1975446664 >> 172.21.15.18:0/1975446664 conn(0x7f5b440782e0 msgr2=0x7f5b4407a660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T20:43:02.451 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:01.440+0000 7f5b4a800700 1 -- 172.21.15.18:0/1975446664 shutdown_connections 2024-04-12T20:43:02.452 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:01.440+0000 7f5b4a800700 1 -- 172.21.15.18:0/1975446664 wait complete. 2024-04-12T20:43:02.452 INFO:teuthology.orchestra.run.smithi018.stdout:mon is available 2024-04-12T20:43:02.452 INFO:teuthology.orchestra.run.smithi018.stdout:Assimilating anything we can from ceph.conf... 2024-04-12T20:43:03.353 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:43:03 smithi018 ceph-mon[28431]: from='client.? 172.21.15.18:0/2553082359' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-04-12T20:43:03.353 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:43:03 smithi018 ceph-mon[28431]: from='client.? 172.21.15.18:0/2553082359' entity='client.admin' cmd='[{"prefix": "config assimilate-conf"}]': finished 2024-04-12T20:43:03.800 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout 2024-04-12T20:43:03.800 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout [global] 2024-04-12T20:43:03.800 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout fsid = 1cbe407a-f90d-11ee-bc8d-c7b262605968 2024-04-12T20:43:03.800 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.18:3300,v1:172.21.15.18:6789] 2024-04-12T20:43:03.801 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-04-12T20:43:03.801 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-04-12T20:43:03.801 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-04-12T20:43:03.801 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-04-12T20:43:03.801 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout 2024-04-12T20:43:03.801 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout [mgr] 2024-04-12T20:43:03.805 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-04-12T20:43:03.805 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout 2024-04-12T20:43:03.805 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout [osd] 2024-04-12T20:43:03.805 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-04-12T20:43:03.805 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-04-12T20:43:03.805 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-04-12T20:43:03.805 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-04-12T20:43:03.805 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-04-12T20:43:03.805 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:03.086+0000 7f393e600700 1 Processor -- start 2024-04-12T20:43:03.805 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:03.086+0000 7f393e600700 1 -- start start 2024-04-12T20:43:03.805 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:03.086+0000 7f393e600700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f393807cf90 0x7f393810bcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T20:43:03.805 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:03.086+0000 7f393e600700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f393807d430 con 0x7f393807cf90 2024-04-12T20:43:03.805 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:03.087+0000 7f3937400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f393807cf90 0x7f393810bcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T20:43:03.806 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:03.087+0000 7f3937400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f393807cf90 0x7f393810bcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:45576/0 (socket says 172.21.15.18:45576) 2024-04-12T20:43:03.806 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:03.087+0000 7f3937400700 1 -- 172.21.15.18:0/1403358892 learned_addr learned my addr 172.21.15.18:0/1403358892 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T20:43:03.806 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:03.087+0000 7f3937400700 1 -- 172.21.15.18:0/1403358892 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f393810c210 con 0x7f393807cf90 2024-04-12T20:43:03.806 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:03.088+0000 7f3937400700 1 --2- 172.21.15.18:0/1403358892 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f393807cf90 0x7f393810bcd0 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f39280098e0 tx=0x7f3928009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a86981ccabf8eb7 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T20:43:03.806 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:03.088+0000 7f3936000700 1 -- 172.21.15.18:0/1403358892 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f392800b860 con 0x7f393807cf90 2024-04-12T20:43:03.806 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:03.088+0000 7f3936000700 1 -- 172.21.15.18:0/1403358892 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f392800b9c0 con 0x7f393807cf90 2024-04-12T20:43:03.806 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:03.088+0000 7f3936000700 1 -- 172.21.15.18:0/1403358892 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f392800bc90 con 0x7f393807cf90 2024-04-12T20:43:03.806 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:03.088+0000 7f393e600700 1 -- 172.21.15.18:0/1403358892 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f393807cf90 msgr2=0x7f393810bcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:43:03.806 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:03.088+0000 7f393e600700 1 --2- 172.21.15.18:0/1403358892 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f393807cf90 0x7f393810bcd0 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f39280098e0 tx=0x7f3928009250 comp rx=0 tx=0).stop 2024-04-12T20:43:03.806 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:03.089+0000 7f393e600700 1 -- 172.21.15.18:0/1403358892 shutdown_connections 2024-04-12T20:43:03.806 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:03.089+0000 7f393e600700 1 --2- 172.21.15.18:0/1403358892 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f393807cf90 0x7f393810bcd0 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:43:03.806 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:03.089+0000 7f393e600700 1 -- 172.21.15.18:0/1403358892 >> 172.21.15.18:0/1403358892 conn(0x7f39380782e0 msgr2=0x7f393807a6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T20:43:03.807 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:03.089+0000 7f393e600700 1 -- 172.21.15.18:0/1403358892 shutdown_connections 2024-04-12T20:43:03.807 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:03.089+0000 7f393e600700 1 -- 172.21.15.18:0/1403358892 wait complete. 2024-04-12T20:43:03.807 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:03.090+0000 7f393e600700 1 Processor -- start 2024-04-12T20:43:03.807 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:03.090+0000 7f393e600700 1 -- start start 2024-04-12T20:43:03.807 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:03.090+0000 7f393e600700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f393807cf90 0x7f39381173f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T20:43:03.807 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:03.090+0000 7f393e600700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3938117930 con 0x7f393807cf90 2024-04-12T20:43:03.807 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:03.091+0000 7f3937400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f393807cf90 0x7f39381173f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T20:43:03.807 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:03.091+0000 7f3937400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f393807cf90 0x7f39381173f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:45592/0 (socket says 172.21.15.18:45592) 2024-04-12T20:43:03.810 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:03.091+0000 7f3937400700 1 -- 172.21.15.18:0/2553082359 learned_addr learned my addr 172.21.15.18:0/2553082359 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T20:43:03.810 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:03.091+0000 7f3937400700 1 -- 172.21.15.18:0/2553082359 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39280094f0 con 0x7f393807cf90 2024-04-12T20:43:03.810 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:03.092+0000 7f3937400700 1 --2- 172.21.15.18:0/2553082359 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f393807cf90 0x7f39381173f0 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f3928009290 tx=0x7f3928016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T20:43:03.810 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:03.092+0000 7f392f400700 1 -- 172.21.15.18:0/2553082359 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f392801c5c0 con 0x7f393807cf90 2024-04-12T20:43:03.810 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:03.092+0000 7f392f400700 1 -- 172.21.15.18:0/2553082359 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f392801c740 con 0x7f393807cf90 2024-04-12T20:43:03.810 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:03.092+0000 7f392f400700 1 -- 172.21.15.18:0/2553082359 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f392801ca50 con 0x7f393807cf90 2024-04-12T20:43:03.810 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:03.092+0000 7f393e600700 1 -- 172.21.15.18:0/2553082359 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3938117b30 con 0x7f393807cf90 2024-04-12T20:43:03.810 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:03.092+0000 7f393e600700 1 -- 172.21.15.18:0/2553082359 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f39381186a0 con 0x7f393807cf90 2024-04-12T20:43:03.810 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:03.093+0000 7f392f400700 1 -- 172.21.15.18:0/2553082359 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 1) v1 ==== 521+0+0 (secure 0 0 0) 0x7f392801a450 con 0x7f393807cf90 2024-04-12T20:43:03.810 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:03.093+0000 7f392f400700 1 -- 172.21.15.18:0/2553082359 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 721+0+0 (secure 0 0 0) 0x7f392802e790 con 0x7f393807cf90 2024-04-12T20:43:03.810 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:03.093+0000 7f393e600700 1 -- 172.21.15.18:0/2553082359 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f393805f5f0 con 0x7f393807cf90 2024-04-12T20:43:03.811 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:03.096+0000 7f392f400700 1 -- 172.21.15.18:0/2553082359 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71349 (secure 0 0 0) 0x7f392801ce30 con 0x7f393807cf90 2024-04-12T20:43:03.811 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:03.172+0000 7f393e600700 1 -- 172.21.15.18:0/2553082359 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) v1 -- 0x7f393810c590 con 0x7f393807cf90 2024-04-12T20:43:03.811 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:03.183+0000 7f392f400700 1 -- 172.21.15.18:0/2553082359 <== mon.0 v2:172.21.15.18:3300/0 7 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7f392802e9f0 con 0x7f393807cf90 2024-04-12T20:43:03.811 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:03.183+0000 7f392f400700 1 -- 172.21.15.18:0/2553082359 <== mon.0 v2:172.21.15.18:3300/0 8 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v2) v1 ==== 70+0+429 (secure 0 0 0) 0x7f393810c590 con 0x7f393807cf90 2024-04-12T20:43:03.811 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:03.185+0000 7f393e600700 1 -- 172.21.15.18:0/2553082359 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f393807cf90 msgr2=0x7f39381173f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:43:03.811 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:03.185+0000 7f393e600700 1 --2- 172.21.15.18:0/2553082359 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f393807cf90 0x7f39381173f0 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f3928009290 tx=0x7f3928016000 comp rx=0 tx=0).stop 2024-04-12T20:43:03.811 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:03.185+0000 7f393e600700 1 -- 172.21.15.18:0/2553082359 shutdown_connections 2024-04-12T20:43:03.811 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:03.185+0000 7f393e600700 1 --2- 172.21.15.18:0/2553082359 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f393807cf90 0x7f39381173f0 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:43:03.811 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:03.185+0000 7f393e600700 1 -- 172.21.15.18:0/2553082359 >> 172.21.15.18:0/2553082359 conn(0x7f39380782e0 msgr2=0x7f393807a660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T20:43:03.811 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:03.185+0000 7f393e600700 1 -- 172.21.15.18:0/2553082359 shutdown_connections 2024-04-12T20:43:03.811 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:03.185+0000 7f393e600700 1 -- 172.21.15.18:0/2553082359 wait complete. 2024-04-12T20:43:03.811 INFO:teuthology.orchestra.run.smithi018.stdout:Generating new minimal ceph.conf... 2024-04-12T20:43:04.774 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:43:04 smithi018 ceph-mon[28431]: from='client.? 172.21.15.18:0/1331258775' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-12T20:43:05.046 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:04.405+0000 7f8f08e00700 1 Processor -- start 2024-04-12T20:43:05.046 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:04.405+0000 7f8f08e00700 1 -- start start 2024-04-12T20:43:05.046 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:04.406+0000 7f8f08e00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8f04071270 0x7f8f04105520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T20:43:05.046 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:04.406+0000 7f8f08e00700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f04071680 con 0x7f8f04071270 2024-04-12T20:43:05.047 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:04.406+0000 7f8f01e00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8f04071270 0x7f8f04105520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T20:43:05.047 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:04.406+0000 7f8f01e00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8f04071270 0x7f8f04105520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:45608/0 (socket says 172.21.15.18:45608) 2024-04-12T20:43:05.047 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:04.406+0000 7f8f01e00700 1 -- 172.21.15.18:0/3824396789 learned_addr learned my addr 172.21.15.18:0/3824396789 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T20:43:05.047 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:04.407+0000 7f8f01e00700 1 -- 172.21.15.18:0/3824396789 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f04105a60 con 0x7f8f04071270 2024-04-12T20:43:05.047 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:04.407+0000 7f8f01e00700 1 --2- 172.21.15.18:0/3824396789 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8f04071270 0x7f8f04105520 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f8ef40098e0 tx=0x7f8ef4009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=390a2fd4b36c9061 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T20:43:05.047 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:04.407+0000 7f8f00a00700 1 -- 172.21.15.18:0/3824396789 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f8ef400b860 con 0x7f8f04071270 2024-04-12T20:43:05.047 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:04.407+0000 7f8f00a00700 1 -- 172.21.15.18:0/3824396789 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7f8ef400be50 con 0x7f8f04071270 2024-04-12T20:43:05.047 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:04.407+0000 7f8f00a00700 1 -- 172.21.15.18:0/3824396789 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f8ef4010380 con 0x7f8f04071270 2024-04-12T20:43:05.047 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:04.408+0000 7f8f08e00700 1 -- 172.21.15.18:0/3824396789 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8f04071270 msgr2=0x7f8f04105520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:43:05.047 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:04.408+0000 7f8f08e00700 1 --2- 172.21.15.18:0/3824396789 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8f04071270 0x7f8f04105520 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f8ef40098e0 tx=0x7f8ef4009250 comp rx=0 tx=0).stop 2024-04-12T20:43:05.047 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:04.408+0000 7f8f08e00700 1 -- 172.21.15.18:0/3824396789 shutdown_connections 2024-04-12T20:43:05.047 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:04.408+0000 7f8f08e00700 1 --2- 172.21.15.18:0/3824396789 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8f04071270 0x7f8f04105520 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:43:05.047 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:04.408+0000 7f8f08e00700 1 -- 172.21.15.18:0/3824396789 >> 172.21.15.18:0/3824396789 conn(0x7f8f040fcfb0 msgr2=0x7f8f040ff3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T20:43:05.047 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:04.408+0000 7f8f08e00700 1 -- 172.21.15.18:0/3824396789 shutdown_connections 2024-04-12T20:43:05.047 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:04.408+0000 7f8f08e00700 1 -- 172.21.15.18:0/3824396789 wait complete. 2024-04-12T20:43:05.048 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:04.409+0000 7f8f08e00700 1 Processor -- start 2024-04-12T20:43:05.048 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:04.409+0000 7f8f08e00700 1 -- start start 2024-04-12T20:43:05.048 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:04.410+0000 7f8f08e00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8f04071270 0x7f8f04113940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T20:43:05.048 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:04.410+0000 7f8f08e00700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f04113e80 con 0x7f8f04071270 2024-04-12T20:43:05.048 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:04.410+0000 7f8f01e00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8f04071270 0x7f8f04113940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T20:43:05.048 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:04.410+0000 7f8f01e00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8f04071270 0x7f8f04113940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:45612/0 (socket says 172.21.15.18:45612) 2024-04-12T20:43:05.048 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:04.410+0000 7f8f01e00700 1 -- 172.21.15.18:0/1331258775 learned_addr learned my addr 172.21.15.18:0/1331258775 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T20:43:05.048 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:04.410+0000 7f8f01e00700 1 -- 172.21.15.18:0/1331258775 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ef40094f0 con 0x7f8f04071270 2024-04-12T20:43:05.048 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:04.411+0000 7f8f01e00700 1 --2- 172.21.15.18:0/1331258775 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8f04071270 0x7f8f04113940 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f8ef4011bd0 tx=0x7f8ef4009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T20:43:05.048 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:04.411+0000 7f8efa000700 1 -- 172.21.15.18:0/1331258775 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f8ef40107c0 con 0x7f8f04071270 2024-04-12T20:43:05.048 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:04.411+0000 7f8f08e00700 1 -- 172.21.15.18:0/1331258775 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8f04114080 con 0x7f8f04071270 2024-04-12T20:43:05.048 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:04.411+0000 7f8efa000700 1 -- 172.21.15.18:0/1331258775 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7f8ef4010db0 con 0x7f8f04071270 2024-04-12T20:43:05.048 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:04.411+0000 7f8efa000700 1 -- 172.21.15.18:0/1331258775 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f8ef401ad60 con 0x7f8f04071270 2024-04-12T20:43:05.048 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:04.412+0000 7f8f08e00700 1 -- 172.21.15.18:0/1331258775 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8f04114bf0 con 0x7f8f04071270 2024-04-12T20:43:05.049 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:04.412+0000 7f8efa000700 1 -- 172.21.15.18:0/1331258775 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 1) v1 ==== 521+0+0 (secure 0 0 0) 0x7f8ef402e6c0 con 0x7f8f04071270 2024-04-12T20:43:05.049 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:04.412+0000 7f8efa000700 1 -- 172.21.15.18:0/1331258775 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 721+0+0 (secure 0 0 0) 0x7f8ef40248d0 con 0x7f8f04071270 2024-04-12T20:43:05.049 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:04.413+0000 7f8f08e00700 1 -- 172.21.15.18:0/1331258775 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8f0405f5f0 con 0x7f8f04071270 2024-04-12T20:43:05.049 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:04.417+0000 7f8efa000700 1 -- 172.21.15.18:0/1331258775 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71349 (secure 0 0 0) 0x7f8ef401a5c0 con 0x7f8f04071270 2024-04-12T20:43:05.049 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:04.493+0000 7f8f08e00700 1 -- 172.21.15.18:0/1331258775 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) v1 -- 0x7f8f041068a0 con 0x7f8f04071270 2024-04-12T20:43:05.049 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:04.494+0000 7f8efa000700 1 -- 172.21.15.18:0/1331258775 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v2) v1 ==== 76+0+175 (secure 0 0 0) 0x7f8ef4010920 con 0x7f8f04071270 2024-04-12T20:43:05.049 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:04.496+0000 7f8f08e00700 1 -- 172.21.15.18:0/1331258775 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8f04071270 msgr2=0x7f8f04113940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:43:05.049 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:04.496+0000 7f8f08e00700 1 --2- 172.21.15.18:0/1331258775 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8f04071270 0x7f8f04113940 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f8ef4011bd0 tx=0x7f8ef4009250 comp rx=0 tx=0).stop 2024-04-12T20:43:05.049 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:04.496+0000 7f8f08e00700 1 -- 172.21.15.18:0/1331258775 shutdown_connections 2024-04-12T20:43:05.049 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:04.496+0000 7f8f08e00700 1 --2- 172.21.15.18:0/1331258775 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8f04071270 0x7f8f04113940 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:43:05.049 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:04.496+0000 7f8f08e00700 1 -- 172.21.15.18:0/1331258775 >> 172.21.15.18:0/1331258775 conn(0x7f8f040fcfb0 msgr2=0x7f8f040fdba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T20:43:05.049 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:04.496+0000 7f8f08e00700 1 -- 172.21.15.18:0/1331258775 shutdown_connections 2024-04-12T20:43:05.049 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:04.496+0000 7f8f08e00700 1 -- 172.21.15.18:0/1331258775 wait complete. 2024-04-12T20:43:05.050 INFO:teuthology.orchestra.run.smithi018.stdout:Restarting the monitor... 2024-04-12T20:43:05.353 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:43:05 smithi018 systemd[1]: Stopping Ceph mon.smithi018 for 1cbe407a-f90d-11ee-bc8d-c7b262605968... 2024-04-12T20:43:05.854 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:43:05 smithi018 ceph-1cbe407a-f90d-11ee-bc8d-c7b262605968-mon-smithi018[28427]: 2024-04-12T20:43:05.513+0000 7fdbf3200700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.smithi018 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-04-12T20:43:05.854 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:43:05 smithi018 ceph-1cbe407a-f90d-11ee-bc8d-c7b262605968-mon-smithi018[28427]: 2024-04-12T20:43:05.513+0000 7fdbf3200700 -1 mon.smithi018@0(leader) e1 *** Got Signal Terminated *** 2024-04-12T20:43:05.854 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:43:05 smithi018 podman[28677]: 2024-04-12 20:43:05.762060101 +0000 UTC m=+0.382059343 container died 851ef11d1ec1cc3b0356f09000f3b3b2f7a7c78924d1fede7be81426bea4fca6 (image=quay.ceph.io/ceph-ci/ceph:quincy, name=ceph-1cbe407a-f90d-11ee-bc8d-c7b262605968-mon-smithi018, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.build-date=20240409, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , RELEASE=quincy-d3b5060, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, GIT_CLEAN=True, ceph=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD) 2024-04-12T20:43:06.340 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:43:06 smithi018 podman[28677]: 2024-04-12 20:43:06.1650376 +0000 UTC m=+0.785036855 container cleanup 851ef11d1ec1cc3b0356f09000f3b3b2f7a7c78924d1fede7be81426bea4fca6 (image=quay.ceph.io/ceph-ci/ceph:quincy, name=ceph-1cbe407a-f90d-11ee-bc8d-c7b262605968-mon-smithi018, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240409, org.label-schema.license=GPLv2, ceph=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=quincy-d3b5060) 2024-04-12T20:43:06.340 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:43:06 smithi018 bash[28677]: ceph-1cbe407a-f90d-11ee-bc8d-c7b262605968-mon-smithi018 2024-04-12T20:43:06.592 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:43:06 smithi018 podman[28691]: 2024-04-12 20:43:06.340776463 +0000 UTC m=+0.564131551 container remove 851ef11d1ec1cc3b0356f09000f3b3b2f7a7c78924d1fede7be81426bea4fca6 (image=quay.ceph.io/ceph-ci/ceph:quincy, name=ceph-1cbe407a-f90d-11ee-bc8d-c7b262605968-mon-smithi018, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240409, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, RELEASE=quincy-d3b5060, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS) 2024-04-12T20:43:07.241 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:43:06 smithi018 systemd[1]: ceph-1cbe407a-f90d-11ee-bc8d-c7b262605968@mon.smithi018.service: Deactivated successfully. 2024-04-12T20:43:07.241 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:43:06 smithi018 systemd[1]: Stopped Ceph mon.smithi018 for 1cbe407a-f90d-11ee-bc8d-c7b262605968. 2024-04-12T20:43:07.241 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:43:06 smithi018 systemd[1]: ceph-1cbe407a-f90d-11ee-bc8d-c7b262605968@mon.smithi018.service: Consumed 1.958s CPU time. 2024-04-12T20:43:07.242 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:43:06 smithi018 systemd[1]: Starting Ceph mon.smithi018 for 1cbe407a-f90d-11ee-bc8d-c7b262605968... 2024-04-12T20:43:07.599 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:43:07 smithi018 podman[28780]: 2024-04-12 20:43:07.147731822 +0000 UTC m=+0.031783514 image pull 21de9b7369ad8457bc44c6aca16b9dfec25bf172fb7b6ebd80f97d4df2005241 quay.ceph.io/ceph-ci/ceph:quincy 2024-04-12T20:43:07.853 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:43:07 smithi018 podman[28780]: 2024-04-12 20:43:07.600179423 +0000 UTC m=+0.484231155 container create cd84caafa056950f73264f0fa479839383ed2eb657507a99926d9f5eea267f9b (image=quay.ceph.io/ceph-ci/ceph:quincy, name=ceph-1cbe407a-f90d-11ee-bc8d-c7b262605968-mon-smithi018, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, RELEASE=quincy-d3b5060, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_CLEAN=True, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240409, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, GIT_BRANCH=HEAD) 2024-04-12T20:43:07.853 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:43:07 smithi018 podman[28780]: 2024-04-12 20:43:07.83686012 +0000 UTC m=+0.720911856 container init cd84caafa056950f73264f0fa479839383ed2eb657507a99926d9f5eea267f9b (image=quay.ceph.io/ceph-ci/ceph:quincy, name=ceph-1cbe407a-f90d-11ee-bc8d-c7b262605968-mon-smithi018, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, ceph=True, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, org.label-schema.build-date=20240409, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=quincy-d3b5060, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=) 2024-04-12T20:43:07.853 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:43:07 smithi018 podman[28780]: 2024-04-12 20:43:07.846980597 +0000 UTC m=+0.731032312 container start cd84caafa056950f73264f0fa479839383ed2eb657507a99926d9f5eea267f9b (image=quay.ceph.io/ceph-ci/ceph:quincy, name=ceph-1cbe407a-f90d-11ee-bc8d-c7b262605968-mon-smithi018, org.label-schema.schema-version=1.0, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.build-date=20240409, RELEASE=quincy-d3b5060, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, io.buildah.version=1.33.5) 2024-04-12T20:43:07.935 INFO:teuthology.orchestra.run.smithi018.stdout:Setting public_network to 172.21.0.0/20 in mon config section 2024-04-12T20:43:08.253 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:43:07 smithi018 ceph-mon[28799]: set uid:gid to 167:167 (ceph:ceph) 2024-04-12T20:43:08.253 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:43:07 smithi018 ceph-mon[28799]: ceph version 17.2.7-827-gd3b5060a (d3b5060a17449edfae9385cdf792afec109fca5d) quincy (stable), process ceph-mon, pid 2 2024-04-12T20:43:08.253 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:43:07 smithi018 ceph-mon[28799]: pidfile_write: ignore empty --pid-file 2024-04-12T20:43:08.253 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:43:07 smithi018 ceph-mon[28799]: load: jerasure load: lrc 2024-04-12T20:43:08.253 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:43:07 smithi018 ceph-mon[28799]: rocksdb: RocksDB version: 6.15.5 2024-04-12T20:43:08.253 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:43:07 smithi018 ceph-mon[28799]: rocksdb: Git sha rocksdb_build_git_sha:@0@ 2024-04-12T20:43:08.254 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:43:07 smithi018 ceph-mon[28799]: rocksdb: Compile date Apr 9 2024 2024-04-12T20:43:08.254 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:43:07 smithi018 ceph-mon[28799]: rocksdb: DB SUMMARY 2024-04-12T20:43:08.254 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:43:07 smithi018 ceph-mon[28799]: rocksdb: DB Session ID: PY8C6ZGLCERCAWB0DVAO 2024-04-12T20:43:08.254 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:43:07 smithi018 ceph-mon[28799]: rocksdb: CURRENT file: CURRENT 2024-04-12T20:43:08.254 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:43:07 smithi018 ceph-mon[28799]: rocksdb: IDENTITY file: IDENTITY 2024-04-12T20:43:08.254 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:43:07 smithi018 ceph-mon[28799]: rocksdb: MANIFEST file: MANIFEST-000009 size: 131 Bytes 2024-04-12T20:43:08.254 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:43:07 smithi018 ceph-mon[28799]: rocksdb: SST files in /var/lib/ceph/mon/ceph-smithi018/store.db dir, Total Num: 1, files: 000008.sst 2024-04-12T20:43:08.254 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:43:07 smithi018 ceph-mon[28799]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-smithi018/store.db: 000010.log size: 98154 ; 2024-04-12T20:43:08.254 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:43:07 smithi018 ceph-mon[28799]: rocksdb: Options.error_if_exists: 0 2024-04-12T20:43:08.254 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:43:07 smithi018 ceph-mon[28799]: rocksdb: Options.create_if_missing: 0 2024-04-12T20:43:08.254 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:43:07 smithi018 ceph-mon[28799]: rocksdb: Options.paranoid_checks: 1 2024-04-12T20:43:08.255 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:43:07 smithi018 ceph-mon[28799]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-04-12T20:43:08.255 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:43:07 smithi018 ceph-mon[28799]: rocksdb: Options.env: 0x558d28294a80 2024-04-12T20:43:08.255 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:43:07 smithi018 ceph-mon[28799]: rocksdb: Options.fs: Posix File System 2024-04-12T20:43:08.255 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:43:07 smithi018 ceph-mon[28799]: rocksdb: Options.info_log: 0x558d5050fc00 2024-04-12T20:43:08.255 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:43:07 smithi018 ceph-mon[28799]: rocksdb: Options.max_file_opening_threads: 16 2024-04-12T20:43:08.255 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:43:07 smithi018 ceph-mon[28799]: rocksdb: Options.statistics: (nil) 2024-04-12T20:43:08.255 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:43:07 smithi018 ceph-mon[28799]: rocksdb: Options.use_fsync: 0 2024-04-12T20:43:08.255 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:43:07 smithi018 ceph-mon[28799]: rocksdb: Options.max_log_file_size: 0 2024-04-12T20:43:08.255 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:43:07 smithi018 ceph-mon[28799]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-04-12T20:43:08.255 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:43:07 smithi018 ceph-mon[28799]: rocksdb: Options.log_file_time_to_roll: 0 2024-04-12T20:43:08.255 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:43:07 smithi018 ceph-mon[28799]: rocksdb: Options.keep_log_file_num: 1000 2024-04-12T20:43:08.255 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:43:07 smithi018 ceph-mon[28799]: rocksdb: Options.recycle_log_file_num: 0 2024-04-12T20:43:08.256 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:43:07 smithi018 ceph-mon[28799]: rocksdb: Options.allow_fallocate: 1 2024-04-12T20:43:08.256 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:43:07 smithi018 ceph-mon[28799]: rocksdb: Options.allow_mmap_reads: 0 2024-04-12T20:43:08.256 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:43:07 smithi018 ceph-mon[28799]: rocksdb: Options.allow_mmap_writes: 0 2024-04-12T20:43:08.256 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:43:07 smithi018 ceph-mon[28799]: rocksdb: Options.use_direct_reads: 0 2024-04-12T20:43:08.256 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:43:07 smithi018 ceph-mon[28799]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-04-12T20:43:08.256 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:43:07 smithi018 ceph-mon[28799]: rocksdb: Options.create_missing_column_families: 0 2024-04-12T20:43:08.256 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:43:07 smithi018 ceph-mon[28799]: rocksdb: Options.db_log_dir: 2024-04-12T20:43:08.256 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:43:07 smithi018 ceph-mon[28799]: rocksdb: Options.wal_dir: /var/lib/ceph/mon/ceph-smithi018/store.db 2024-04-12T20:43:08.256 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:43:07 smithi018 ceph-mon[28799]: rocksdb: Options.table_cache_numshardbits: 6 2024-04-12T20:43:08.256 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:43:07 smithi018 ceph-mon[28799]: rocksdb: Options.WAL_ttl_seconds: 0 2024-04-12T20:43:08.258 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:43:07 smithi018 ceph-mon[28799]: rocksdb: Options.WAL_size_limit_MB: 0 2024-04-12T20:43:08.258 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:43:07 smithi018 ceph-mon[28799]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-04-12T20:43:08.258 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:43:07 smithi018 ceph-mon[28799]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-04-12T20:43:08.258 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:43:07 smithi018 ceph-mon[28799]: rocksdb: Options.is_fd_close_on_exec: 1 2024-04-12T20:43:08.258 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:43:07 smithi018 ceph-mon[28799]: rocksdb: Options.advise_random_on_open: 1 2024-04-12T20:43:08.258 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:43:07 smithi018 ceph-mon[28799]: rocksdb: Options.db_write_buffer_size: 0 2024-04-12T20:43:08.259 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:43:07 smithi018 ceph-mon[28799]: rocksdb: Options.write_buffer_manager: 0x558d512849f0 2024-04-12T20:43:08.259 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:43:07 smithi018 ceph-mon[28799]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-04-12T20:43:08.259 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:43:07 smithi018 ceph-mon[28799]: rocksdb: Options.new_table_reader_for_compaction_inputs: 0 2024-04-12T20:43:08.259 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:43:07 smithi018 ceph-mon[28799]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-04-12T20:43:08.259 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:43:07 smithi018 ceph-mon[28799]: rocksdb: Options.use_adaptive_mutex: 0 2024-04-12T20:43:08.259 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:43:07 smithi018 ceph-mon[28799]: rocksdb: Options.rate_limiter: (nil) 2024-04-12T20:43:08.259 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:43:07 smithi018 ceph-mon[28799]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-04-12T20:43:08.259 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:43:07 smithi018 ceph-mon[28799]: rocksdb: Options.wal_recovery_mode: 2 2024-04-12T20:43:08.259 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:43:07 smithi018 ceph-mon[28799]: rocksdb: Options.enable_thread_tracking: 0 2024-04-12T20:43:08.259 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:43:07 smithi018 ceph-mon[28799]: rocksdb: Options.enable_pipelined_write: 0 2024-04-12T20:43:08.259 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:43:07 smithi018 ceph-mon[28799]: rocksdb: Options.unordered_write: 0 2024-04-12T20:43:08.259 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:43:07 smithi018 ceph-mon[28799]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-04-12T20:43:08.259 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:43:07 smithi018 ceph-mon[28799]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-04-12T20:43:08.259 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:43:07 smithi018 ceph-mon[28799]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-04-12T20:43:08.260 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:43:07 smithi018 ceph-mon[28799]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-04-12T20:43:08.260 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:43:07 smithi018 ceph-mon[28799]: rocksdb: Options.row_cache: None 2024-04-12T20:43:08.260 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:43:07 smithi018 ceph-mon[28799]: rocksdb: Options.wal_filter: None 2024-04-12T20:43:08.260 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:43:07 smithi018 ceph-mon[28799]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-04-12T20:43:08.260 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:43:07 smithi018 ceph-mon[28799]: rocksdb: Options.allow_ingest_behind: 0 2024-04-12T20:43:08.260 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:43:07 smithi018 ceph-mon[28799]: rocksdb: Options.preserve_deletes: 0 2024-04-12T20:43:08.260 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:43:07 smithi018 ceph-mon[28799]: rocksdb: Options.two_write_queues: 0 2024-04-12T20:43:08.260 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:43:07 smithi018 ceph-mon[28799]: rocksdb: Options.manual_wal_flush: 0 2024-04-12T20:43:08.260 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:43:07 smithi018 ceph-mon[28799]: rocksdb: Options.atomic_flush: 0 2024-04-12T20:43:08.260 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:43:07 smithi018 ceph-mon[28799]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-04-12T20:43:08.260 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:43:07 smithi018 ceph-mon[28799]: rocksdb: Options.persist_stats_to_disk: 0 2024-04-12T20:43:08.260 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:43:07 smithi018 ceph-mon[28799]: rocksdb: Options.write_dbid_to_manifest: 0 2024-04-12T20:43:08.260 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:43:07 smithi018 ceph-mon[28799]: rocksdb: Options.log_readahead_size: 0 2024-04-12T20:43:08.261 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:43:07 smithi018 ceph-mon[28799]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-04-12T20:43:08.261 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:43:07 smithi018 ceph-mon[28799]: rocksdb: Options.best_efforts_recovery: 0 2024-04-12T20:43:08.261 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:43:07 smithi018 ceph-mon[28799]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-04-12T20:43:08.261 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:43:07 smithi018 ceph-mon[28799]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-04-12T20:43:08.261 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:43:07 smithi018 ceph-mon[28799]: rocksdb: Options.allow_data_in_errors: 0 2024-04-12T20:43:08.261 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:43:07 smithi018 ceph-mon[28799]: rocksdb: Options.db_host_id: __hostname__ 2024-04-12T20:43:08.261 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:43:07 smithi018 ceph-mon[28799]: rocksdb: Options.max_background_jobs: 2 2024-04-12T20:43:08.261 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:43:07 smithi018 ceph-mon[28799]: rocksdb: Options.max_background_compactions: -1 2024-04-12T20:43:08.261 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:43:07 smithi018 ceph-mon[28799]: rocksdb: Options.max_subcompactions: 1 2024-04-12T20:43:08.261 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:43:07 smithi018 ceph-mon[28799]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-04-12T20:43:08.261 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:43:07 smithi018 ceph-mon[28799]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-04-12T20:43:08.261 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:43:07 smithi018 ceph-mon[28799]: rocksdb: Options.delayed_write_rate : 16777216 2024-04-12T20:43:08.261 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:43:07 smithi018 ceph-mon[28799]: rocksdb: Options.max_total_wal_size: 0 2024-04-12T20:43:08.261 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:43:07 smithi018 ceph-mon[28799]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-04-12T20:43:08.262 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:43:07 smithi018 ceph-mon[28799]: rocksdb: Options.stats_dump_period_sec: 600 2024-04-12T20:43:08.262 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:43:07 smithi018 ceph-mon[28799]: rocksdb: Options.stats_persist_period_sec: 600 2024-04-12T20:43:08.262 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:43:07 smithi018 ceph-mon[28799]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-04-12T20:43:08.262 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:43:07 smithi018 ceph-mon[28799]: rocksdb: Options.max_open_files: -1 2024-04-12T20:43:08.262 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:43:07 smithi018 ceph-mon[28799]: rocksdb: Options.bytes_per_sync: 0 2024-04-12T20:43:08.262 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:43:07 smithi018 ceph-mon[28799]: rocksdb: Options.wal_bytes_per_sync: 0 2024-04-12T20:43:08.262 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:43:07 smithi018 ceph-mon[28799]: rocksdb: Options.strict_bytes_per_sync: 0 2024-04-12T20:43:08.262 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:43:07 smithi018 ceph-mon[28799]: rocksdb: Options.compaction_readahead_size: 0 2024-04-12T20:43:08.262 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:43:07 smithi018 ceph-mon[28799]: rocksdb: Options.max_background_flushes: -1 2024-04-12T20:43:08.262 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:43:07 smithi018 ceph-mon[28799]: rocksdb: Compression algorithms supported: 2024-04-12T20:43:08.262 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:43:07 smithi018 ceph-mon[28799]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-04-12T20:43:08.262 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:43:07 smithi018 ceph-mon[28799]: rocksdb: kZSTD supported: 0 2024-04-12T20:43:08.262 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:43:07 smithi018 ceph-mon[28799]: rocksdb: kXpressCompression supported: 0 2024-04-12T20:43:08.262 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:43:07 smithi018 ceph-mon[28799]: rocksdb: kLZ4HCCompression supported: 1 2024-04-12T20:43:08.262 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:43:07 smithi018 ceph-mon[28799]: rocksdb: kZlibCompression supported: 1 2024-04-12T20:43:08.263 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:43:07 smithi018 ceph-mon[28799]: rocksdb: kSnappyCompression supported: 1 2024-04-12T20:43:08.263 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:43:07 smithi018 ceph-mon[28799]: rocksdb: kLZ4Compression supported: 1 2024-04-12T20:43:08.263 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:43:07 smithi018 ceph-mon[28799]: rocksdb: kBZip2Compression supported: 0 2024-04-12T20:43:08.263 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:43:07 smithi018 ceph-mon[28799]: rocksdb: Fast CRC32 supported: Supported on x86 2024-04-12T20:43:08.263 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:43:07 smithi018 ceph-mon[28799]: rocksdb: [db/version_set.cc:4724] Recovering from manifest file: /var/lib/ceph/mon/ceph-smithi018/store.db/MANIFEST-000009 2024-04-12T20:43:08.263 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:43:07 smithi018 ceph-mon[28799]: rocksdb: [db/column_family.cc:595] --------------- Options for column family [default]: 2024-04-12T20:43:08.263 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:43:07 smithi018 ceph-mon[28799]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-04-12T20:43:08.263 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:43:07 smithi018 ceph-mon[28799]: rocksdb: Options.merge_operator: 2024-04-12T20:43:08.263 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:43:07 smithi018 ceph-mon[28799]: rocksdb: Options.compaction_filter: None 2024-04-12T20:43:08.263 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:43:07 smithi018 ceph-mon[28799]: rocksdb: Options.compaction_filter_factory: None 2024-04-12T20:43:08.263 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:43:07 smithi018 ceph-mon[28799]: rocksdb: Options.sst_partitioner_factory: None 2024-04-12T20:43:08.263 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:43:07 smithi018 ceph-mon[28799]: rocksdb: Options.memtable_factory: SkipListFactory 2024-04-12T20:43:08.263 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:43:07 smithi018 ceph-mon[28799]: rocksdb: Options.table_factory: BlockBasedTable 2024-04-12T20:43:08.263 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:43:07 smithi018 ceph-mon[28799]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x558d50564f58) 2024-04-12T20:43:08.263 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout: cache_index_and_filter_blocks: 1 2024-04-12T20:43:08.264 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-04-12T20:43:08.264 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-04-12T20:43:08.264 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout: pin_top_level_index_and_filter: 1 2024-04-12T20:43:08.264 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout: index_type: 0 2024-04-12T20:43:08.264 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout: data_block_index_type: 0 2024-04-12T20:43:08.264 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout: index_shortening: 1 2024-04-12T20:43:08.264 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout: data_block_hash_table_util_ratio: 0.750000 2024-04-12T20:43:08.264 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout: hash_index_allow_collision: 1 2024-04-12T20:43:08.264 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout: checksum: 1 2024-04-12T20:43:08.264 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout: no_block_cache: 0 2024-04-12T20:43:08.264 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout: block_cache: 0x558d50597090 2024-04-12T20:43:08.264 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout: block_cache_name: BinnedLRUCache 2024-04-12T20:43:08.264 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout: block_cache_options: 2024-04-12T20:43:08.264 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout: capacity : 536870912 2024-04-12T20:43:08.265 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout: num_shard_bits : 4 2024-04-12T20:43:08.265 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout: strict_capacity_limit : 0 2024-04-12T20:43:08.265 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout: high_pri_pool_ratio: 0.000 2024-04-12T20:43:08.265 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout: block_cache_compressed: (nil) 2024-04-12T20:43:08.265 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout: persistent_cache: (nil) 2024-04-12T20:43:08.265 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout: block_size: 4096 2024-04-12T20:43:08.265 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout: block_size_deviation: 10 2024-04-12T20:43:08.265 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout: block_restart_interval: 16 2024-04-12T20:43:08.265 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout: index_block_restart_interval: 1 2024-04-12T20:43:08.265 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout: metadata_block_size: 4096 2024-04-12T20:43:08.265 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout: partition_filters: 0 2024-04-12T20:43:08.265 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout: use_delta_encoding: 1 2024-04-12T20:43:08.265 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout: filter_policy: rocksdb.BuiltinBloomFilter 2024-04-12T20:43:08.265 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout: whole_key_filtering: 1 2024-04-12T20:43:08.266 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout: verify_compression: 0 2024-04-12T20:43:08.266 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout: read_amp_bytes_per_bit: 0 2024-04-12T20:43:08.266 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout: format_version: 4 2024-04-12T20:43:08.266 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout: enable_index_compression: 1 2024-04-12T20:43:08.266 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout: block_align: 0 2024-04-12T20:43:08.266 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:43:07 smithi018 ceph-mon[28799]: rocksdb: Options.write_buffer_size: 33554432 2024-04-12T20:43:08.266 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:43:07 smithi018 ceph-mon[28799]: rocksdb: Options.max_write_buffer_number: 2 2024-04-12T20:43:08.266 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:43:07 smithi018 ceph-mon[28799]: rocksdb: Options.compression: NoCompression 2024-04-12T20:43:08.266 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:43:07 smithi018 ceph-mon[28799]: rocksdb: Options.bottommost_compression: Disabled 2024-04-12T20:43:08.266 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:43:07 smithi018 ceph-mon[28799]: rocksdb: Options.prefix_extractor: nullptr 2024-04-12T20:43:08.266 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:43:07 smithi018 ceph-mon[28799]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-04-12T20:43:08.266 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:43:07 smithi018 ceph-mon[28799]: rocksdb: Options.num_levels: 7 2024-04-12T20:43:08.266 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:43:07 smithi018 ceph-mon[28799]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-04-12T20:43:08.266 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:43:07 smithi018 ceph-mon[28799]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-04-12T20:43:08.267 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:43:07 smithi018 ceph-mon[28799]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-04-12T20:43:08.267 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:43:07 smithi018 ceph-mon[28799]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-04-12T20:43:08.267 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:43:07 smithi018 ceph-mon[28799]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-04-12T20:43:08.267 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:43:07 smithi018 ceph-mon[28799]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-04-12T20:43:08.267 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:43:07 smithi018 ceph-mon[28799]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-04-12T20:43:08.267 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:43:07 smithi018 ceph-mon[28799]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-04-12T20:43:08.267 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:43:07 smithi018 ceph-mon[28799]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-04-12T20:43:08.267 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:43:07 smithi018 ceph-mon[28799]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-04-12T20:43:08.267 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:43:07 smithi018 ceph-mon[28799]: rocksdb: Options.compression_opts.window_bits: -14 2024-04-12T20:43:08.267 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:43:07 smithi018 ceph-mon[28799]: rocksdb: Options.compression_opts.level: 32767 2024-04-12T20:43:08.267 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:43:07 smithi018 ceph-mon[28799]: rocksdb: Options.compression_opts.strategy: 0 2024-04-12T20:43:08.267 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:43:07 smithi018 ceph-mon[28799]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-04-12T20:43:08.267 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:43:07 smithi018 ceph-mon[28799]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-04-12T20:43:08.267 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:43:07 smithi018 ceph-mon[28799]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-04-12T20:43:08.267 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:43:07 smithi018 ceph-mon[28799]: rocksdb: Options.compression_opts.enabled: false 2024-04-12T20:43:08.268 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:43:07 smithi018 ceph-mon[28799]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-04-12T20:43:08.268 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:43:07 smithi018 ceph-mon[28799]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-04-12T20:43:08.268 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:43:07 smithi018 ceph-mon[28799]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-04-12T20:43:08.268 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:43:07 smithi018 ceph-mon[28799]: rocksdb: Options.target_file_size_base: 67108864 2024-04-12T20:43:08.268 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:43:07 smithi018 ceph-mon[28799]: rocksdb: Options.target_file_size_multiplier: 1 2024-04-12T20:43:08.268 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:43:07 smithi018 ceph-mon[28799]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-04-12T20:43:08.268 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:43:07 smithi018 ceph-mon[28799]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-04-12T20:43:08.268 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:43:07 smithi018 ceph-mon[28799]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-04-12T20:43:08.268 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:43:07 smithi018 ceph-mon[28799]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-04-12T20:43:08.268 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:43:07 smithi018 ceph-mon[28799]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-04-12T20:43:08.268 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:43:07 smithi018 ceph-mon[28799]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-04-12T20:43:08.268 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:43:07 smithi018 ceph-mon[28799]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-04-12T20:43:08.268 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:43:07 smithi018 ceph-mon[28799]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-04-12T20:43:08.268 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:43:07 smithi018 ceph-mon[28799]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-04-12T20:43:08.269 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:43:07 smithi018 ceph-mon[28799]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-04-12T20:43:08.269 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:43:07 smithi018 ceph-mon[28799]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-04-12T20:43:08.269 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:43:07 smithi018 ceph-mon[28799]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-04-12T20:43:08.269 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:43:07 smithi018 ceph-mon[28799]: rocksdb: Options.arena_block_size: 4194304 2024-04-12T20:43:08.269 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:43:07 smithi018 ceph-mon[28799]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-04-12T20:43:08.269 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:43:07 smithi018 ceph-mon[28799]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-04-12T20:43:08.269 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:43:07 smithi018 ceph-mon[28799]: rocksdb: Options.rate_limit_delay_max_milliseconds: 100 2024-04-12T20:43:08.269 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:43:07 smithi018 ceph-mon[28799]: rocksdb: Options.disable_auto_compactions: 0 2024-04-12T20:43:08.269 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:43:07 smithi018 ceph-mon[28799]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-04-12T20:43:08.269 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:43:07 smithi018 ceph-mon[28799]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-04-12T20:43:08.269 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:43:07 smithi018 ceph-mon[28799]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-04-12T20:43:08.269 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:43:07 smithi018 ceph-mon[28799]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-04-12T20:43:08.269 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:43:07 smithi018 ceph-mon[28799]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-04-12T20:43:08.269 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:43:07 smithi018 ceph-mon[28799]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-04-12T20:43:08.270 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:43:07 smithi018 ceph-mon[28799]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-04-12T20:43:08.270 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:43:07 smithi018 ceph-mon[28799]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-04-12T20:43:08.270 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:43:07 smithi018 ceph-mon[28799]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-04-12T20:43:08.270 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:43:07 smithi018 ceph-mon[28799]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-04-12T20:43:08.270 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:43:07 smithi018 ceph-mon[28799]: rocksdb: Options.table_properties_collectors: 2024-04-12T20:43:08.270 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:43:07 smithi018 ceph-mon[28799]: rocksdb: Options.inplace_update_support: 0 2024-04-12T20:43:08.270 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:43:07 smithi018 ceph-mon[28799]: rocksdb: Options.inplace_update_num_locks: 10000 2024-04-12T20:43:08.270 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:43:07 smithi018 ceph-mon[28799]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-04-12T20:43:08.270 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:43:07 smithi018 ceph-mon[28799]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-04-12T20:43:08.270 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:43:07 smithi018 ceph-mon[28799]: rocksdb: Options.memtable_huge_page_size: 0 2024-04-12T20:43:08.270 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:43:07 smithi018 ceph-mon[28799]: rocksdb: Options.bloom_locality: 0 2024-04-12T20:43:08.270 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:43:07 smithi018 ceph-mon[28799]: rocksdb: Options.max_successive_merges: 0 2024-04-12T20:43:08.270 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:43:07 smithi018 ceph-mon[28799]: rocksdb: Options.optimize_filters_for_hits: 0 2024-04-12T20:43:08.271 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:43:07 smithi018 ceph-mon[28799]: rocksdb: Options.paranoid_file_checks: 0 2024-04-12T20:43:08.271 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:43:07 smithi018 ceph-mon[28799]: rocksdb: Options.force_consistency_checks: 1 2024-04-12T20:43:08.271 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:43:07 smithi018 ceph-mon[28799]: rocksdb: Options.report_bg_io_stats: 0 2024-04-12T20:43:08.271 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:43:07 smithi018 ceph-mon[28799]: rocksdb: Options.ttl: 2592000 2024-04-12T20:43:08.271 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:43:07 smithi018 ceph-mon[28799]: rocksdb: Options.periodic_compaction_seconds: 0 2024-04-12T20:43:08.271 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:43:07 smithi018 ceph-mon[28799]: rocksdb: Options.enable_blob_files: false 2024-04-12T20:43:08.271 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:43:07 smithi018 ceph-mon[28799]: rocksdb: Options.min_blob_size: 0 2024-04-12T20:43:08.271 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:43:07 smithi018 ceph-mon[28799]: rocksdb: Options.blob_file_size: 268435456 2024-04-12T20:43:08.271 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:43:07 smithi018 ceph-mon[28799]: rocksdb: Options.blob_compression_type: NoCompression 2024-04-12T20:43:08.271 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:43:07 smithi018 ceph-mon[28799]: rocksdb: Options.enable_blob_garbage_collection: false 2024-04-12T20:43:08.271 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:43:07 smithi018 ceph-mon[28799]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-04-12T20:43:08.271 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:43:07 smithi018 ceph-mon[28799]: rocksdb: [db/version_set.cc:4764] Recovered from manifest file:/var/lib/ceph/mon/ceph-smithi018/store.db/MANIFEST-000009 succeeded,manifest_file_number is 9, next_file_number is 11, last_sequence is 5, log_number is 5,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-04-12T20:43:08.272 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:43:07 smithi018 ceph-mon[28799]: rocksdb: [db/version_set.cc:4779] Column family [default] (ID 0), log number is 5 2024-04-12T20:43:08.272 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:43:07 smithi018 ceph-mon[28799]: rocksdb: [db/version_set.cc:4082] Creating manifest 13 2024-04-12T20:43:08.272 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:43:07 smithi018 ceph-mon[28799]: rocksdb: EVENT_LOG_v1 {"time_micros": 1712954587900892, "job": 1, "event": "recovery_started", "wal_files": [10]} 2024-04-12T20:43:08.272 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:43:07 smithi018 ceph-mon[28799]: rocksdb: [db/db_impl/db_impl_open.cc:845] Recovering log #10 mode 2 2024-04-12T20:43:08.272 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:43:07 smithi018 ceph-mon[28799]: rocksdb: [table/block_based/filter_policy.cc:991] Using legacy Bloom filter with high (20) bits/key. Dramatic filter space and/or accuracy improvement is available with format_version>=5. 2024-04-12T20:43:08.272 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:43:07 smithi018 ceph-mon[28799]: rocksdb: EVENT_LOG_v1 {"time_micros": 1712954587903843, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 14, "file_size": 93360, "file_checksum": "", "file_checksum_func_name": "Unknown", "table_properties": {"data_size": 91472, "index_size": 248, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 709, "raw_key_size": 13576, "raw_average_key_size": 49, "raw_value_size": 84036, "raw_average_value_size": 307, "num_data_blocks": 11, "num_entries": 273, "num_deletions": 3, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "rocksdb.BuiltinBloomFilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; ", "creation_time": 1712954587, "oldest_key_time": 0, "file_creation_time": 0, "db_id": "eaf74de4-f45b-484d-b35e-7954c73b0bf5", "db_session_id": "PY8C6ZGLCERCAWB0DVAO"}} 2024-04-12T20:43:08.272 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:43:07 smithi018 ceph-mon[28799]: rocksdb: [db/version_set.cc:4082] Creating manifest 15 2024-04-12T20:43:08.272 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:43:07 smithi018 ceph-mon[28799]: rocksdb: EVENT_LOG_v1 {"time_micros": 1712954587904898, "job": 1, "event": "recovery_finished"} 2024-04-12T20:43:08.272 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:43:07 smithi018 ceph-mon[28799]: rocksdb: [file/delete_scheduler.cc:69] Deleted file /var/lib/ceph/mon/ceph-smithi018/store.db/000010.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-04-12T20:43:08.272 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:43:07 smithi018 ceph-mon[28799]: rocksdb: [db/db_impl/db_impl_open.cc:1700] SstFileManager instance 0x558d5052b180 2024-04-12T20:43:08.272 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:43:07 smithi018 ceph-mon[28799]: rocksdb: DB pointer 0x558d50600000 2024-04-12T20:43:08.272 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:43:07 smithi018 ceph-mon[28799]: rocksdb: [db/db_impl/db_impl.cc:901] ------- DUMPING STATS ------- 2024-04-12T20:43:08.273 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:43:07 smithi018 ceph-mon[28799]: rocksdb: [db/db_impl/db_impl.cc:903] 2024-04-12T20:43:08.273 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout: ** DB Stats ** 2024-04-12T20:43:08.273 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-04-12T20:43:08.273 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-04-12T20:43:08.273 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-04-12T20:43:08.273 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-04-12T20:43:08.273 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-04-12T20:43:08.273 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 MB, 0.00 MB/s 2024-04-12T20:43:08.273 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-04-12T20:43:08.273 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout: 2024-04-12T20:43:08.273 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout: ** Compaction Stats [default] ** 2024-04-12T20:43:08.273 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2024-04-12T20:43:08.273 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout: ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-04-12T20:43:08.273 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout: L0 2/0 92.97 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 44.5 0.00 0.00 1 0.002 0 0 2024-04-12T20:43:08.274 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout: Sum 2/0 92.97 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 44.5 0.00 0.00 1 0.002 0 0 2024-04-12T20:43:08.274 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 44.5 0.00 0.00 1 0.002 0 0 2024-04-12T20:43:08.274 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout: 2024-04-12T20:43:08.274 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout: ** Compaction Stats [default] ** 2024-04-12T20:43:08.274 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2024-04-12T20:43:08.274 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-04-12T20:43:08.274 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 44.5 0.00 0.00 1 0.002 0 0 2024-04-12T20:43:08.274 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-04-12T20:43:08.274 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-04-12T20:43:08.274 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-04-12T20:43:08.274 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-04-12T20:43:08.274 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-04-12T20:43:08.274 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout: AddFile(Keys): cumulative 0, interval 0 2024-04-12T20:43:08.275 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout: Cumulative compaction: 0.00 GB write, 8.25 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-04-12T20:43:08.275 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout: Interval compaction: 0.00 GB write, 8.25 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-04-12T20:43:08.275 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-04-12T20:43:08.275 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout: 2024-04-12T20:43:08.275 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout: ** File Read Latency Histogram By Level [default] ** 2024-04-12T20:43:08.275 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout: 2024-04-12T20:43:08.275 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout: ** Compaction Stats [default] ** 2024-04-12T20:43:08.275 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2024-04-12T20:43:08.275 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout: ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-04-12T20:43:08.275 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout: L0 2/0 92.97 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 44.5 0.00 0.00 1 0.002 0 0 2024-04-12T20:43:08.275 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout: Sum 2/0 92.97 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 44.5 0.00 0.00 1 0.002 0 0 2024-04-12T20:43:08.275 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 2024-04-12T20:43:08.275 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout: 2024-04-12T20:43:08.276 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout: ** Compaction Stats [default] ** 2024-04-12T20:43:08.276 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2024-04-12T20:43:08.276 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-04-12T20:43:08.276 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 44.5 0.00 0.00 1 0.002 0 0 2024-04-12T20:43:08.276 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-04-12T20:43:08.276 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-04-12T20:43:08.276 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-04-12T20:43:08.276 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-04-12T20:43:08.276 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-04-12T20:43:08.276 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout: AddFile(Keys): cumulative 0, interval 0 2024-04-12T20:43:08.276 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout: Cumulative compaction: 0.00 GB write, 8.22 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-04-12T20:43:08.276 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout: Interval compaction: 0.00 GB write, 0.00 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-04-12T20:43:08.277 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-04-12T20:43:08.277 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout: 2024-04-12T20:43:08.277 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout: ** File Read Latency Histogram By Level [default] ** 2024-04-12T20:43:08.277 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:43:07 smithi018 ceph-mon[28799]: starting mon.smithi018 rank 0 at public addrs [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] at bind addrs [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] mon_data /var/lib/ceph/mon/ceph-smithi018 fsid 1cbe407a-f90d-11ee-bc8d-c7b262605968 2024-04-12T20:43:08.277 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:43:07 smithi018 ceph-mon[28799]: mon.smithi018@-1(???) e1 preinit fsid 1cbe407a-f90d-11ee-bc8d-c7b262605968 2024-04-12T20:43:08.277 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:43:07 smithi018 ceph-mon[28799]: mon.smithi018@-1(???).mds e1 new map 2024-04-12T20:43:08.277 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:43:07 smithi018 ceph-mon[28799]: mon.smithi018@-1(???).mds e1 print_map 2024-04-12T20:43:08.277 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout: e1 2024-04-12T20:43:08.277 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-04-12T20:43:08.277 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-12T20:43:08.277 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout: legacy client fscid: -1 2024-04-12T20:43:08.277 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout: 2024-04-12T20:43:08.278 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout: No filesystems configured 2024-04-12T20:43:08.278 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:43:07 smithi018 ceph-mon[28799]: mon.smithi018@-1(???).osd e1 crush map has features 3314932999778484224, adjusting msgr requires 2024-04-12T20:43:08.278 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:43:07 smithi018 ceph-mon[28799]: mon.smithi018@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-04-12T20:43:08.278 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:43:07 smithi018 ceph-mon[28799]: mon.smithi018@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-04-12T20:43:08.278 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:43:07 smithi018 ceph-mon[28799]: mon.smithi018@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-04-12T20:43:08.278 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:43:07 smithi018 ceph-mon[28799]: mon.smithi018@-1(???).paxosservice(auth 1..2) refresh upgraded, format 0 -> 3 2024-04-12T20:43:08.278 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:43:07 smithi018 bash[28780]: cd84caafa056950f73264f0fa479839383ed2eb657507a99926d9f5eea267f9b 2024-04-12T20:43:08.278 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:43:07 smithi018 systemd[1]: Started Ceph mon.smithi018 for 1cbe407a-f90d-11ee-bc8d-c7b262605968. 2024-04-12T20:43:08.278 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:43:07 smithi018 ceph-mon[28799]: mon.smithi018 is new leader, mons smithi018 in quorum (ranks 0) 2024-04-12T20:43:08.278 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:43:07 smithi018 ceph-mon[28799]: monmap e1: 1 mons at {smithi018=[v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-04-12T20:43:08.278 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:43:07 smithi018 ceph-mon[28799]: fsmap 2024-04-12T20:43:08.278 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:43:07 smithi018 ceph-mon[28799]: osdmap e1: 0 total, 0 up, 0 in 2024-04-12T20:43:08.278 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:43:07 smithi018 ceph-mon[28799]: mgrmap e1: no daemons active 2024-04-12T20:43:09.294 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:08.538+0000 7f46ae800700 1 Processor -- start 2024-04-12T20:43:09.299 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:08.539+0000 7f46ae800700 1 -- start start 2024-04-12T20:43:09.299 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:08.539+0000 7f46ae800700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f46a8077b00 0x7f46a8076100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T20:43:09.299 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:08.539+0000 7f46ae800700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46a8077f10 con 0x7f46a8077b00 2024-04-12T20:43:09.299 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:08.540+0000 7f46a7400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f46a8077b00 0x7f46a8076100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T20:43:09.299 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:08.540+0000 7f46a7400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f46a8077b00 0x7f46a8076100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:45616/0 (socket says 172.21.15.18:45616) 2024-04-12T20:43:09.299 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:08.540+0000 7f46a7400700 1 -- 172.21.15.18:0/486532987 learned_addr learned my addr 172.21.15.18:0/486532987 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T20:43:09.299 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:08.540+0000 7f46a7400700 1 -- 172.21.15.18:0/486532987 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46a8076640 con 0x7f46a8077b00 2024-04-12T20:43:09.299 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:08.541+0000 7f46a7400700 1 --2- 172.21.15.18:0/486532987 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f46a8077b00 0x7f46a8076100 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f46980098e0 tx=0x7f4698009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ab4370b10443fe0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T20:43:09.299 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:08.541+0000 7f46a6000700 1 -- 172.21.15.18:0/486532987 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f469800b860 con 0x7f46a8077b00 2024-04-12T20:43:09.300 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:08.541+0000 7f46a6000700 1 -- 172.21.15.18:0/486532987 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7f469800be50 con 0x7f46a8077b00 2024-04-12T20:43:09.300 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:08.541+0000 7f46a6000700 1 -- 172.21.15.18:0/486532987 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f4698010380 con 0x7f46a8077b00 2024-04-12T20:43:09.300 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:08.542+0000 7f46ae800700 1 -- 172.21.15.18:0/486532987 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f46a8077b00 msgr2=0x7f46a8076100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:43:09.300 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:08.542+0000 7f46ae800700 1 --2- 172.21.15.18:0/486532987 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f46a8077b00 0x7f46a8076100 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f46980098e0 tx=0x7f4698009250 comp rx=0 tx=0).stop 2024-04-12T20:43:09.300 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:08.542+0000 7f46ae800700 1 -- 172.21.15.18:0/486532987 shutdown_connections 2024-04-12T20:43:09.300 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:08.542+0000 7f46ae800700 1 --2- 172.21.15.18:0/486532987 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f46a8077b00 0x7f46a8076100 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:43:09.300 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:08.542+0000 7f46ae800700 1 -- 172.21.15.18:0/486532987 >> 172.21.15.18:0/486532987 conn(0x7f46a80fe090 msgr2=0x7f46a81004c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T20:43:09.300 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:08.542+0000 7f46ae800700 1 -- 172.21.15.18:0/486532987 shutdown_connections 2024-04-12T20:43:09.300 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:08.542+0000 7f46ae800700 1 -- 172.21.15.18:0/486532987 wait complete. 2024-04-12T20:43:09.300 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:08.543+0000 7f46ae800700 1 Processor -- start 2024-04-12T20:43:09.300 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:08.544+0000 7f46ae800700 1 -- start start 2024-04-12T20:43:09.300 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:08.544+0000 7f46ae800700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f46a8077b00 0x7f46a81139b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T20:43:09.300 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:08.544+0000 7f46ae800700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46a8113ef0 con 0x7f46a8077b00 2024-04-12T20:43:09.300 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:08.544+0000 7f46a7400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f46a8077b00 0x7f46a81139b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T20:43:09.301 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:08.544+0000 7f46a7400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f46a8077b00 0x7f46a81139b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:45630/0 (socket says 172.21.15.18:45630) 2024-04-12T20:43:09.301 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:08.544+0000 7f46a7400700 1 -- 172.21.15.18:0/1385216281 learned_addr learned my addr 172.21.15.18:0/1385216281 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T20:43:09.301 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:08.545+0000 7f46a7400700 1 -- 172.21.15.18:0/1385216281 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46980094f0 con 0x7f46a8077b00 2024-04-12T20:43:09.301 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:08.545+0000 7f46a7400700 1 --2- 172.21.15.18:0/1385216281 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f46a8077b00 0x7f46a81139b0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f4698011bd0 tx=0x7f4698009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T20:43:09.301 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:08.545+0000 7f469f400700 1 -- 172.21.15.18:0/1385216281 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f46980107c0 con 0x7f46a8077b00 2024-04-12T20:43:09.301 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:08.545+0000 7f469f400700 1 -- 172.21.15.18:0/1385216281 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7f4698010db0 con 0x7f46a8077b00 2024-04-12T20:43:09.301 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:08.545+0000 7f46ae800700 1 -- 172.21.15.18:0/1385216281 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f46a81140f0 con 0x7f46a8077b00 2024-04-12T20:43:09.301 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:08.546+0000 7f469f400700 1 -- 172.21.15.18:0/1385216281 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f469801ad00 con 0x7f46a8077b00 2024-04-12T20:43:09.301 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:08.546+0000 7f46ae800700 1 -- 172.21.15.18:0/1385216281 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f46a8114c60 con 0x7f46a8077b00 2024-04-12T20:43:09.301 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:08.546+0000 7f469f400700 1 -- 172.21.15.18:0/1385216281 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 1) v1 ==== 521+0+0 (secure 0 0 0) 0x7f46980256d0 con 0x7f46a8077b00 2024-04-12T20:43:09.301 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:08.546+0000 7f469f400700 1 -- 172.21.15.18:0/1385216281 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 721+0+0 (secure 0 0 0) 0x7f4698019890 con 0x7f46a8077b00 2024-04-12T20:43:09.301 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:08.547+0000 7f46ae800700 1 -- 172.21.15.18:0/1385216281 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f46a805f5f0 con 0x7f46a8077b00 2024-04-12T20:43:09.301 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:08.551+0000 7f469f400700 1 -- 172.21.15.18:0/1385216281 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71349 (secure 0 0 0) 0x7f469802a060 con 0x7f46a8077b00 2024-04-12T20:43:09.301 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:08.627+0000 7f46ae800700 1 -- 172.21.15.18:0/1385216281 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command([{prefix=config set, name=public_network}] v 0) v1 -- 0x7f46a8108b10 con 0x7f46a8077b00 2024-04-12T20:43:09.302 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:08.630+0000 7f469f400700 1 -- 172.21.15.18:0/1385216281 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{prefix=config set, name=public_network}]=0 v3)=0 v3) v1 ==== 124+0+0 (secure 0 0 0) 0x7f4698020070 con 0x7f46a8077b00 2024-04-12T20:43:09.302 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:08.632+0000 7f46ae800700 1 -- 172.21.15.18:0/1385216281 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f46a8077b00 msgr2=0x7f46a81139b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:43:09.302 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:08.632+0000 7f46ae800700 1 --2- 172.21.15.18:0/1385216281 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f46a8077b00 0x7f46a81139b0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f4698011bd0 tx=0x7f4698009250 comp rx=0 tx=0).stop 2024-04-12T20:43:09.302 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:08.632+0000 7f46ae800700 1 -- 172.21.15.18:0/1385216281 shutdown_connections 2024-04-12T20:43:09.302 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:08.632+0000 7f46ae800700 1 --2- 172.21.15.18:0/1385216281 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f46a8077b00 0x7f46a81139b0 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:43:09.302 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:08.632+0000 7f46ae800700 1 -- 172.21.15.18:0/1385216281 >> 172.21.15.18:0/1385216281 conn(0x7f46a80fe090 msgr2=0x7f46a80fed00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T20:43:09.302 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:08.632+0000 7f46ae800700 1 -- 172.21.15.18:0/1385216281 shutdown_connections 2024-04-12T20:43:09.302 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:08.632+0000 7f46ae800700 1 -- 172.21.15.18:0/1385216281 wait complete. 2024-04-12T20:43:09.302 INFO:teuthology.orchestra.run.smithi018.stdout:Wrote config to /etc/ceph/ceph.conf 2024-04-12T20:43:09.302 INFO:teuthology.orchestra.run.smithi018.stdout:Wrote keyring to /etc/ceph/ceph.client.admin.keyring 2024-04-12T20:43:09.302 INFO:teuthology.orchestra.run.smithi018.stdout:Creating mgr... 2024-04-12T20:43:09.303 INFO:teuthology.orchestra.run.smithi018.stdout:Verifying port 9283 ... 2024-04-12T20:43:09.638 INFO:teuthology.orchestra.run.smithi018.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-1cbe407a-f90d-11ee-bc8d-c7b262605968@mgr.smithi018.xjzwga 2024-04-12T20:43:09.638 INFO:teuthology.orchestra.run.smithi018.stdout:systemctl: stderr Failed to reset failed state of unit ceph-1cbe407a-f90d-11ee-bc8d-c7b262605968@mgr.smithi018.xjzwga.service: Unit ceph-1cbe407a-f90d-11ee-bc8d-c7b262605968@mgr.smithi018.xjzwga.service not loaded. 2024-04-12T20:43:09.904 INFO:teuthology.orchestra.run.smithi018.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-1cbe407a-f90d-11ee-bc8d-c7b262605968.target.wants/ceph-1cbe407a-f90d-11ee-bc8d-c7b262605968@mgr.smithi018.xjzwga.service → /etc/systemd/system/ceph-1cbe407a-f90d-11ee-bc8d-c7b262605968@.service. 2024-04-12T20:43:09.948 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:43:09 smithi018 ceph-mon[28799]: from='client.? 172.21.15.18:0/1385216281' entity='client.admin' 2024-04-12T20:43:10.713 INFO:teuthology.orchestra.run.smithi018.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-04-12T20:43:10.713 INFO:teuthology.orchestra.run.smithi018.stdout:systemctl: stdout disabled 2024-04-12T20:43:10.733 INFO:teuthology.orchestra.run.smithi018.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-04-12T20:43:10.733 INFO:teuthology.orchestra.run.smithi018.stdout:systemctl: stdout inactive 2024-04-12T20:43:10.733 INFO:teuthology.orchestra.run.smithi018.stdout:firewalld.service is not enabled 2024-04-12T20:43:10.733 INFO:teuthology.orchestra.run.smithi018.stdout:Not possible to enable service . firewalld.service is not available 2024-04-12T20:43:10.759 INFO:teuthology.orchestra.run.smithi018.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-04-12T20:43:10.759 INFO:teuthology.orchestra.run.smithi018.stdout:systemctl: stdout disabled 2024-04-12T20:43:10.782 INFO:teuthology.orchestra.run.smithi018.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-04-12T20:43:10.782 INFO:teuthology.orchestra.run.smithi018.stdout:systemctl: stdout inactive 2024-04-12T20:43:10.782 INFO:teuthology.orchestra.run.smithi018.stdout:firewalld.service is not enabled 2024-04-12T20:43:10.782 INFO:teuthology.orchestra.run.smithi018.stdout:Not possible to open ports <[9283]>. firewalld.service is not available 2024-04-12T20:43:10.782 INFO:teuthology.orchestra.run.smithi018.stdout:Waiting for mgr to start... 2024-04-12T20:43:10.782 INFO:teuthology.orchestra.run.smithi018.stdout:Waiting for mgr... 2024-04-12T20:43:11.853 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:43:11 smithi018 ceph-mon[28799]: from='client.? 172.21.15.18:0/2226362487' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-04-12T20:43:12.473 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout 2024-04-12T20:43:12.475 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout { 2024-04-12T20:43:12.475 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "fsid": "1cbe407a-f90d-11ee-bc8d-c7b262605968", 2024-04-12T20:43:12.475 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "health": { 2024-04-12T20:43:12.475 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-04-12T20:43:12.476 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-04-12T20:43:12.476 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-04-12T20:43:12.476 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout }, 2024-04-12T20:43:12.476 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-04-12T20:43:12.476 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-04-12T20:43:12.476 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout 0 2024-04-12T20:43:12.476 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout ], 2024-04-12T20:43:12.476 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-04-12T20:43:12.476 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "smithi018" 2024-04-12T20:43:12.476 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout ], 2024-04-12T20:43:12.476 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "quorum_age": 3, 2024-04-12T20:43:12.476 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "monmap": { 2024-04-12T20:43:12.476 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-12T20:43:12.476 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "quincy", 2024-04-12T20:43:12.476 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-04-12T20:43:12.477 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout }, 2024-04-12T20:43:12.477 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-04-12T20:43:12.477 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-12T20:43:12.477 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-04-12T20:43:12.477 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-04-12T20:43:12.477 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-04-12T20:43:12.477 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-04-12T20:43:12.477 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-04-12T20:43:12.477 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-04-12T20:43:12.477 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout }, 2024-04-12T20:43:12.484 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-04-12T20:43:12.484 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-04-12T20:43:12.484 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-04-12T20:43:12.484 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-04-12T20:43:12.484 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-04-12T20:43:12.484 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-04-12T20:43:12.485 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-04-12T20:43:12.485 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-04-12T20:43:12.485 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-04-12T20:43:12.485 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout }, 2024-04-12T20:43:12.485 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-04-12T20:43:12.485 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-12T20:43:12.485 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-04-12T20:43:12.485 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-04-12T20:43:12.485 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout }, 2024-04-12T20:43:12.485 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-04-12T20:43:12.485 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "available": false, 2024-04-12T20:43:12.485 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-04-12T20:43:12.485 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "modules": [ 2024-04-12T20:43:12.485 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "iostat", 2024-04-12T20:43:12.486 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "nfs", 2024-04-12T20:43:12.486 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "restful" 2024-04-12T20:43:12.486 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout ], 2024-04-12T20:43:12.486 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-12T20:43:12.486 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout }, 2024-04-12T20:43:12.486 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-04-12T20:43:12.486 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-12T20:43:12.486 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "modified": "2024-04-12T20:43:00.692941+0000", 2024-04-12T20:43:12.486 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-12T20:43:12.486 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout }, 2024-04-12T20:43:12.486 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-04-12T20:43:12.487 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout } 2024-04-12T20:43:12.487 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:11.374+0000 7f79ba600700 1 Processor -- start 2024-04-12T20:43:12.487 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:11.374+0000 7f79ba600700 1 -- start start 2024-04-12T20:43:12.487 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:11.375+0000 7f79ba600700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f79b4102970 0x7f79b4102d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T20:43:12.487 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:11.375+0000 7f79ba600700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79b41032c0 con 0x7f79b4102970 2024-04-12T20:43:12.487 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:11.375+0000 7f79b3400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f79b4102970 0x7f79b4102d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T20:43:12.487 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:11.375+0000 7f79b3400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f79b4102970 0x7f79b4102d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:60174/0 (socket says 172.21.15.18:60174) 2024-04-12T20:43:12.487 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:11.375+0000 7f79b3400700 1 -- 172.21.15.18:0/2014045312 learned_addr learned my addr 172.21.15.18:0/2014045312 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T20:43:12.487 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:11.376+0000 7f79b3400700 1 -- 172.21.15.18:0/2014045312 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79b4103400 con 0x7f79b4102970 2024-04-12T20:43:12.487 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:11.376+0000 7f79b3400700 1 --2- 172.21.15.18:0/2014045312 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f79b4102970 0x7f79b4102d80 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f79a40098e0 tx=0x7f79a4009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=81e87a4e83cbd6fd server_cookie=0 in_seq=0 out_seq=0 2024-04-12T20:43:12.487 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:11.377+0000 7f79b2000700 1 -- 172.21.15.18:0/2014045312 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f79a400b860 con 0x7f79b4102970 2024-04-12T20:43:12.487 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:11.377+0000 7f79b2000700 1 -- 172.21.15.18:0/2014045312 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7f79a400be50 con 0x7f79b4102970 2024-04-12T20:43:12.487 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:11.377+0000 7f79b2000700 1 -- 172.21.15.18:0/2014045312 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f79a4010380 con 0x7f79b4102970 2024-04-12T20:43:12.488 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:11.377+0000 7f79ba600700 1 -- 172.21.15.18:0/2014045312 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f79b4102970 msgr2=0x7f79b4102d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:43:12.488 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:11.377+0000 7f79ba600700 1 --2- 172.21.15.18:0/2014045312 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f79b4102970 0x7f79b4102d80 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f79a40098e0 tx=0x7f79a4009250 comp rx=0 tx=0).stop 2024-04-12T20:43:12.488 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:11.378+0000 7f79ba600700 1 -- 172.21.15.18:0/2014045312 shutdown_connections 2024-04-12T20:43:12.488 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:11.378+0000 7f79ba600700 1 --2- 172.21.15.18:0/2014045312 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f79b4102970 0x7f79b4102d80 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:43:12.488 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:11.378+0000 7f79ba600700 1 -- 172.21.15.18:0/2014045312 >> 172.21.15.18:0/2014045312 conn(0x7f79b40fe090 msgr2=0x7f79b4100500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T20:43:12.488 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:11.378+0000 7f79ba600700 1 -- 172.21.15.18:0/2014045312 shutdown_connections 2024-04-12T20:43:12.488 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:11.378+0000 7f79ba600700 1 -- 172.21.15.18:0/2014045312 wait complete. 2024-04-12T20:43:12.488 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:11.379+0000 7f79ba600700 1 Processor -- start 2024-04-12T20:43:12.488 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:11.379+0000 7f79ba600700 1 -- start start 2024-04-12T20:43:12.488 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:11.379+0000 7f79ba600700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f79b4102970 0x7f79b4111720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T20:43:12.488 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:11.379+0000 7f79ba600700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79b4111c60 con 0x7f79b4102970 2024-04-12T20:43:12.488 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:11.380+0000 7f79b3400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f79b4102970 0x7f79b4111720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T20:43:12.488 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:11.380+0000 7f79b3400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f79b4102970 0x7f79b4111720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:60180/0 (socket says 172.21.15.18:60180) 2024-04-12T20:43:12.488 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:11.380+0000 7f79b3400700 1 -- 172.21.15.18:0/2226362487 learned_addr learned my addr 172.21.15.18:0/2226362487 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T20:43:12.489 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:11.380+0000 7f79b3400700 1 -- 172.21.15.18:0/2226362487 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79a40094f0 con 0x7f79b4102970 2024-04-12T20:43:12.489 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:11.380+0000 7f79b3400700 1 --2- 172.21.15.18:0/2226362487 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f79b4102970 0x7f79b4111720 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f79a400ba90 tx=0x7f79a4009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T20:43:12.489 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:11.381+0000 7f79ab400700 1 -- 172.21.15.18:0/2226362487 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f79a40107c0 con 0x7f79b4102970 2024-04-12T20:43:12.489 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:11.381+0000 7f79ab400700 1 -- 172.21.15.18:0/2226362487 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7f79a4010db0 con 0x7f79b4102970 2024-04-12T20:43:12.489 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:11.381+0000 7f79ba600700 1 -- 172.21.15.18:0/2226362487 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f79b4111e60 con 0x7f79b4102970 2024-04-12T20:43:12.489 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:11.381+0000 7f79ab400700 1 -- 172.21.15.18:0/2226362487 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f79a401add0 con 0x7f79b4102970 2024-04-12T20:43:12.489 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:11.381+0000 7f79ba600700 1 -- 172.21.15.18:0/2226362487 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f79b41129d0 con 0x7f79b4102970 2024-04-12T20:43:12.490 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:11.382+0000 7f79ab400700 1 -- 172.21.15.18:0/2226362487 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 1) v1 ==== 521+0+0 (secure 0 0 0) 0x7f79a402e6c0 con 0x7f79b4102970 2024-04-12T20:43:12.490 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:11.382+0000 7f79ab400700 1 -- 172.21.15.18:0/2226362487 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 721+0+0 (secure 0 0 0) 0x7f79a4024490 con 0x7f79b4102970 2024-04-12T20:43:12.490 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:11.382+0000 7f79ba600700 1 -- 172.21.15.18:0/2226362487 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f79b405f5f0 con 0x7f79b4102970 2024-04-12T20:43:12.490 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:11.385+0000 7f79ab400700 1 -- 172.21.15.18:0/2226362487 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71349 (secure 0 0 0) 0x7f79a4029070 con 0x7f79b4102970 2024-04-12T20:43:12.490 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:11.463+0000 7f79ba600700 1 -- 172.21.15.18:0/2226362487 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7f79b41046a0 con 0x7f79b4102970 2024-04-12T20:43:12.490 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:11.464+0000 7f79ab400700 1 -- 172.21.15.18:0/2226362487 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1248 (secure 0 0 0) 0x7f79a401a650 con 0x7f79b4102970 2024-04-12T20:43:12.490 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:11.466+0000 7f79ba600700 1 -- 172.21.15.18:0/2226362487 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f79b4102970 msgr2=0x7f79b4111720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:43:12.490 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:11.466+0000 7f79ba600700 1 --2- 172.21.15.18:0/2226362487 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f79b4102970 0x7f79b4111720 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f79a400ba90 tx=0x7f79a4009250 comp rx=0 tx=0).stop 2024-04-12T20:43:12.490 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:11.466+0000 7f79ba600700 1 -- 172.21.15.18:0/2226362487 shutdown_connections 2024-04-12T20:43:12.490 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:11.466+0000 7f79ba600700 1 --2- 172.21.15.18:0/2226362487 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f79b4102970 0x7f79b4111720 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:43:12.490 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:11.466+0000 7f79ba600700 1 -- 172.21.15.18:0/2226362487 >> 172.21.15.18:0/2226362487 conn(0x7f79b40fe090 msgr2=0x7f79b40fec70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T20:43:12.490 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:11.466+0000 7f79ba600700 1 -- 172.21.15.18:0/2226362487 shutdown_connections 2024-04-12T20:43:12.490 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:11.466+0000 7f79ba600700 1 -- 172.21.15.18:0/2226362487 wait complete. 2024-04-12T20:43:12.490 INFO:teuthology.orchestra.run.smithi018.stdout:mgr not available, waiting (1/15)... 2024-04-12T20:43:15.573 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:43:15 smithi018 ceph-mon[28799]: from='client.? 172.21.15.18:0/2352444335' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-04-12T20:43:15.968 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout 2024-04-12T20:43:15.968 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout { 2024-04-12T20:43:15.968 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "fsid": "1cbe407a-f90d-11ee-bc8d-c7b262605968", 2024-04-12T20:43:15.968 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "health": { 2024-04-12T20:43:15.968 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-04-12T20:43:15.968 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-04-12T20:43:15.969 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-04-12T20:43:15.969 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout }, 2024-04-12T20:43:15.969 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-04-12T20:43:15.969 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-04-12T20:43:15.969 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout 0 2024-04-12T20:43:15.969 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout ], 2024-04-12T20:43:15.969 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-04-12T20:43:15.969 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "smithi018" 2024-04-12T20:43:15.969 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout ], 2024-04-12T20:43:15.969 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "quorum_age": 7, 2024-04-12T20:43:15.969 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "monmap": { 2024-04-12T20:43:15.969 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-12T20:43:15.969 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "quincy", 2024-04-12T20:43:15.969 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-04-12T20:43:15.970 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout }, 2024-04-12T20:43:15.970 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-04-12T20:43:15.970 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-12T20:43:15.970 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-04-12T20:43:15.970 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-04-12T20:43:15.970 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-04-12T20:43:15.970 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-04-12T20:43:15.978 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-04-12T20:43:15.978 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-04-12T20:43:15.978 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout }, 2024-04-12T20:43:15.978 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-04-12T20:43:15.978 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-04-12T20:43:15.978 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-04-12T20:43:15.978 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-04-12T20:43:15.978 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-04-12T20:43:15.978 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-04-12T20:43:15.979 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-04-12T20:43:15.979 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-04-12T20:43:15.979 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-04-12T20:43:15.979 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout }, 2024-04-12T20:43:15.979 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-04-12T20:43:15.979 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-12T20:43:15.979 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-04-12T20:43:15.979 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-04-12T20:43:15.979 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout }, 2024-04-12T20:43:15.979 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-04-12T20:43:15.979 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "available": false, 2024-04-12T20:43:15.979 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-04-12T20:43:15.979 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "modules": [ 2024-04-12T20:43:15.979 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "iostat", 2024-04-12T20:43:15.979 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "nfs", 2024-04-12T20:43:15.980 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "restful" 2024-04-12T20:43:15.980 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout ], 2024-04-12T20:43:15.980 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-12T20:43:15.980 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout }, 2024-04-12T20:43:15.980 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-04-12T20:43:15.980 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-12T20:43:15.980 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "modified": "2024-04-12T20:43:00.692941+0000", 2024-04-12T20:43:15.980 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-12T20:43:15.980 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout }, 2024-04-12T20:43:15.980 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-04-12T20:43:15.980 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout } 2024-04-12T20:43:15.980 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:15.080+0000 7f7487e00700 1 Processor -- start 2024-04-12T20:43:15.980 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:15.080+0000 7f7487e00700 1 -- start start 2024-04-12T20:43:15.980 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:15.081+0000 7f7487e00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7480077b80 0x7f7480076180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T20:43:15.981 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:15.081+0000 7f7487e00700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7480077f90 con 0x7f7480077b80 2024-04-12T20:43:15.981 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:15.081+0000 7f7485400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7480077b80 0x7f7480076180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T20:43:15.981 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:15.081+0000 7f7485400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7480077b80 0x7f7480076180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:60196/0 (socket says 172.21.15.18:60196) 2024-04-12T20:43:15.981 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:15.081+0000 7f7485400700 1 -- 172.21.15.18:0/3754776470 learned_addr learned my addr 172.21.15.18:0/3754776470 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T20:43:15.981 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:15.081+0000 7f7485400700 1 -- 172.21.15.18:0/3754776470 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74800766c0 con 0x7f7480077b80 2024-04-12T20:43:15.981 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:15.082+0000 7f7485400700 1 --2- 172.21.15.18:0/3754776470 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7480077b80 0x7f7480076180 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f74740098e0 tx=0x7f7474009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf8ba5c974376e20 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T20:43:15.981 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:15.082+0000 7f747fe00700 1 -- 172.21.15.18:0/3754776470 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f747400b860 con 0x7f7480077b80 2024-04-12T20:43:15.981 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:15.082+0000 7f747fe00700 1 -- 172.21.15.18:0/3754776470 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7f747400be50 con 0x7f7480077b80 2024-04-12T20:43:15.981 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:15.082+0000 7f747fe00700 1 -- 172.21.15.18:0/3754776470 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f7474010380 con 0x7f7480077b80 2024-04-12T20:43:15.981 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:15.083+0000 7f7487e00700 1 -- 172.21.15.18:0/3754776470 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7480077b80 msgr2=0x7f7480076180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:43:15.981 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:15.083+0000 7f7487e00700 1 --2- 172.21.15.18:0/3754776470 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7480077b80 0x7f7480076180 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f74740098e0 tx=0x7f7474009250 comp rx=0 tx=0).stop 2024-04-12T20:43:15.981 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:15.083+0000 7f7487e00700 1 -- 172.21.15.18:0/3754776470 shutdown_connections 2024-04-12T20:43:15.981 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:15.083+0000 7f7487e00700 1 --2- 172.21.15.18:0/3754776470 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7480077b80 0x7f7480076180 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:43:15.982 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:15.083+0000 7f7487e00700 1 -- 172.21.15.18:0/3754776470 >> 172.21.15.18:0/3754776470 conn(0x7f74800fe090 msgr2=0x7f74801004c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T20:43:15.982 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:15.083+0000 7f7487e00700 1 -- 172.21.15.18:0/3754776470 shutdown_connections 2024-04-12T20:43:15.982 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:15.083+0000 7f7487e00700 1 -- 172.21.15.18:0/3754776470 wait complete. 2024-04-12T20:43:15.982 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:15.084+0000 7f7487e00700 1 Processor -- start 2024-04-12T20:43:15.982 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:15.084+0000 7f7487e00700 1 -- start start 2024-04-12T20:43:15.982 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:15.084+0000 7f7487e00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7480077b80 0x7f74801139f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T20:43:15.982 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:15.084+0000 7f7487e00700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7480113f30 con 0x7f7480077b80 2024-04-12T20:43:15.982 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:15.085+0000 7f7485400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7480077b80 0x7f74801139f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T20:43:15.982 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:15.085+0000 7f7485400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7480077b80 0x7f74801139f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:60204/0 (socket says 172.21.15.18:60204) 2024-04-12T20:43:15.982 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:15.085+0000 7f7485400700 1 -- 172.21.15.18:0/2352444335 learned_addr learned my addr 172.21.15.18:0/2352444335 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T20:43:15.982 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:15.085+0000 7f7485400700 1 -- 172.21.15.18:0/2352444335 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74740094f0 con 0x7f7480077b80 2024-04-12T20:43:15.982 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:15.085+0000 7f7485400700 1 --2- 172.21.15.18:0/2352444335 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7480077b80 0x7f74801139f0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f7474006d40 tx=0x7f7474009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T20:43:15.982 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:15.086+0000 7f747d600700 1 -- 172.21.15.18:0/2352444335 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f74740107c0 con 0x7f7480077b80 2024-04-12T20:43:15.982 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:15.086+0000 7f747d600700 1 -- 172.21.15.18:0/2352444335 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7f7474010db0 con 0x7f7480077b80 2024-04-12T20:43:15.983 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:15.086+0000 7f747d600700 1 -- 172.21.15.18:0/2352444335 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f747401ad40 con 0x7f7480077b80 2024-04-12T20:43:15.983 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:15.086+0000 7f7487e00700 1 -- 172.21.15.18:0/2352444335 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7480114130 con 0x7f7480077b80 2024-04-12T20:43:15.983 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:15.086+0000 7f7487e00700 1 -- 172.21.15.18:0/2352444335 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7480114ca0 con 0x7f7480077b80 2024-04-12T20:43:15.983 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:15.086+0000 7f747d600700 1 -- 172.21.15.18:0/2352444335 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 1) v1 ==== 521+0+0 (secure 0 0 0) 0x7f74740256d0 con 0x7f7480077b80 2024-04-12T20:43:15.983 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:15.086+0000 7f747d600700 1 -- 172.21.15.18:0/2352444335 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 721+0+0 (secure 0 0 0) 0x7f7474019710 con 0x7f7480077b80 2024-04-12T20:43:15.983 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:15.087+0000 7f7487e00700 1 -- 172.21.15.18:0/2352444335 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f748005f5f0 con 0x7f7480077b80 2024-04-12T20:43:15.983 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:15.091+0000 7f747d600700 1 -- 172.21.15.18:0/2352444335 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71349 (secure 0 0 0) 0x7f747402a060 con 0x7f7480077b80 2024-04-12T20:43:15.983 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:15.168+0000 7f7487e00700 1 -- 172.21.15.18:0/2352444335 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7f7480114f50 con 0x7f7480077b80 2024-04-12T20:43:15.983 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:15.169+0000 7f747d600700 1 -- 172.21.15.18:0/2352444335 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1248 (secure 0 0 0) 0x7f7474020070 con 0x7f7480077b80 2024-04-12T20:43:15.983 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:15.171+0000 7f7487e00700 1 -- 172.21.15.18:0/2352444335 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7480077b80 msgr2=0x7f74801139f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:43:15.983 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:15.171+0000 7f7487e00700 1 --2- 172.21.15.18:0/2352444335 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7480077b80 0x7f74801139f0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f7474006d40 tx=0x7f7474009250 comp rx=0 tx=0).stop 2024-04-12T20:43:15.983 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:15.171+0000 7f7487e00700 1 -- 172.21.15.18:0/2352444335 shutdown_connections 2024-04-12T20:43:15.983 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:15.171+0000 7f7487e00700 1 --2- 172.21.15.18:0/2352444335 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7480077b80 0x7f74801139f0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:43:15.983 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:15.171+0000 7f7487e00700 1 -- 172.21.15.18:0/2352444335 >> 172.21.15.18:0/2352444335 conn(0x7f74800fe090 msgr2=0x7f74800fed10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T20:43:15.983 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:15.171+0000 7f7487e00700 1 -- 172.21.15.18:0/2352444335 shutdown_connections 2024-04-12T20:43:15.984 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:15.171+0000 7f7487e00700 1 -- 172.21.15.18:0/2352444335 wait complete. 2024-04-12T20:43:15.984 INFO:teuthology.orchestra.run.smithi018.stdout:mgr not available, waiting (2/15)... 2024-04-12T20:43:19.097 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:43:18 smithi018 ceph-mon[28799]: from='client.? 172.21.15.18:0/4155106942' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-04-12T20:43:19.365 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout 2024-04-12T20:43:19.365 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout { 2024-04-12T20:43:19.365 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "fsid": "1cbe407a-f90d-11ee-bc8d-c7b262605968", 2024-04-12T20:43:19.365 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "health": { 2024-04-12T20:43:19.365 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-04-12T20:43:19.365 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-04-12T20:43:19.365 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-04-12T20:43:19.365 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout }, 2024-04-12T20:43:19.366 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-04-12T20:43:19.366 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-04-12T20:43:19.366 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout 0 2024-04-12T20:43:19.366 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout ], 2024-04-12T20:43:19.366 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-04-12T20:43:19.366 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "smithi018" 2024-04-12T20:43:19.366 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout ], 2024-04-12T20:43:19.366 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "quorum_age": 10, 2024-04-12T20:43:19.366 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "monmap": { 2024-04-12T20:43:19.366 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-12T20:43:19.366 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "quincy", 2024-04-12T20:43:19.366 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-04-12T20:43:19.366 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout }, 2024-04-12T20:43:19.366 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-04-12T20:43:19.367 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-12T20:43:19.367 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-04-12T20:43:19.367 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-04-12T20:43:19.367 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-04-12T20:43:19.375 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-04-12T20:43:19.375 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-04-12T20:43:19.375 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-04-12T20:43:19.375 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout }, 2024-04-12T20:43:19.375 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-04-12T20:43:19.375 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-04-12T20:43:19.376 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-04-12T20:43:19.376 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-04-12T20:43:19.376 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-04-12T20:43:19.376 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-04-12T20:43:19.376 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-04-12T20:43:19.376 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-04-12T20:43:19.376 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-04-12T20:43:19.376 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout }, 2024-04-12T20:43:19.376 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-04-12T20:43:19.376 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-12T20:43:19.376 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-04-12T20:43:19.376 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-04-12T20:43:19.376 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout }, 2024-04-12T20:43:19.376 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-04-12T20:43:19.377 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "available": false, 2024-04-12T20:43:19.377 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-04-12T20:43:19.377 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "modules": [ 2024-04-12T20:43:19.377 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "iostat", 2024-04-12T20:43:19.377 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "nfs", 2024-04-12T20:43:19.377 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "restful" 2024-04-12T20:43:19.377 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout ], 2024-04-12T20:43:19.377 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-12T20:43:19.377 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout }, 2024-04-12T20:43:19.377 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-04-12T20:43:19.377 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-12T20:43:19.377 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "modified": "2024-04-12T20:43:00.692941+0000", 2024-04-12T20:43:19.377 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-12T20:43:19.377 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout }, 2024-04-12T20:43:19.377 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-04-12T20:43:19.378 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout } 2024-04-12T20:43:19.378 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:18.609+0000 7f6b75e00700 1 Processor -- start 2024-04-12T20:43:19.378 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:18.609+0000 7f6b75e00700 1 -- start start 2024-04-12T20:43:19.378 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:18.610+0000 7f6b75e00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6b70071230 0x7f6b70105560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T20:43:19.378 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:18.610+0000 7f6b75e00700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b70071640 con 0x7f6b70071230 2024-04-12T20:43:19.378 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:18.610+0000 7f6b6ea00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6b70071230 0x7f6b70105560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T20:43:19.378 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:18.610+0000 7f6b6ea00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6b70071230 0x7f6b70105560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:60208/0 (socket says 172.21.15.18:60208) 2024-04-12T20:43:19.378 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:18.610+0000 7f6b6ea00700 1 -- 172.21.15.18:0/2380860455 learned_addr learned my addr 172.21.15.18:0/2380860455 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T20:43:19.378 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:18.611+0000 7f6b6ea00700 1 -- 172.21.15.18:0/2380860455 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b70105aa0 con 0x7f6b70071230 2024-04-12T20:43:19.378 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:18.611+0000 7f6b6ea00700 1 --2- 172.21.15.18:0/2380860455 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6b70071230 0x7f6b70105560 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f6b600098e0 tx=0x7f6b60009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=527282b3edfd3f33 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T20:43:19.378 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:18.611+0000 7f6b6d600700 1 -- 172.21.15.18:0/2380860455 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f6b6000b860 con 0x7f6b70071230 2024-04-12T20:43:19.378 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:18.611+0000 7f6b6d600700 1 -- 172.21.15.18:0/2380860455 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7f6b6000be50 con 0x7f6b70071230 2024-04-12T20:43:19.378 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:18.611+0000 7f6b6d600700 1 -- 172.21.15.18:0/2380860455 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f6b60010380 con 0x7f6b70071230 2024-04-12T20:43:19.379 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:18.612+0000 7f6b75e00700 1 -- 172.21.15.18:0/2380860455 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6b70071230 msgr2=0x7f6b70105560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:43:19.379 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:18.612+0000 7f6b75e00700 1 --2- 172.21.15.18:0/2380860455 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6b70071230 0x7f6b70105560 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f6b600098e0 tx=0x7f6b60009250 comp rx=0 tx=0).stop 2024-04-12T20:43:19.379 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:18.612+0000 7f6b75e00700 1 -- 172.21.15.18:0/2380860455 shutdown_connections 2024-04-12T20:43:19.379 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:18.612+0000 7f6b75e00700 1 --2- 172.21.15.18:0/2380860455 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6b70071230 0x7f6b70105560 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:43:19.379 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:18.612+0000 7f6b75e00700 1 -- 172.21.15.18:0/2380860455 >> 172.21.15.18:0/2380860455 conn(0x7f6b700fcfb0 msgr2=0x7f6b700ff3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T20:43:19.379 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:18.612+0000 7f6b75e00700 1 -- 172.21.15.18:0/2380860455 shutdown_connections 2024-04-12T20:43:19.379 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:18.612+0000 7f6b75e00700 1 -- 172.21.15.18:0/2380860455 wait complete. 2024-04-12T20:43:19.379 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:18.613+0000 7f6b75e00700 1 Processor -- start 2024-04-12T20:43:19.379 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:18.613+0000 7f6b75e00700 1 -- start start 2024-04-12T20:43:19.379 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:18.614+0000 7f6b75e00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6b70071230 0x7f6b70113960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T20:43:19.379 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:18.614+0000 7f6b75e00700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b70113ea0 con 0x7f6b70071230 2024-04-12T20:43:19.379 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:18.614+0000 7f6b6ea00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6b70071230 0x7f6b70113960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T20:43:19.379 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:18.614+0000 7f6b6ea00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6b70071230 0x7f6b70113960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:60218/0 (socket says 172.21.15.18:60218) 2024-04-12T20:43:19.379 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:18.614+0000 7f6b6ea00700 1 -- 172.21.15.18:0/4155106942 learned_addr learned my addr 172.21.15.18:0/4155106942 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T20:43:19.379 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:18.614+0000 7f6b6ea00700 1 -- 172.21.15.18:0/4155106942 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b600094f0 con 0x7f6b70071230 2024-04-12T20:43:19.380 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:18.615+0000 7f6b6ea00700 1 --2- 172.21.15.18:0/4155106942 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6b70071230 0x7f6b70113960 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f6b60011bd0 tx=0x7f6b60009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T20:43:19.380 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:18.615+0000 7f6b66a00700 1 -- 172.21.15.18:0/4155106942 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f6b600107c0 con 0x7f6b70071230 2024-04-12T20:43:19.380 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:18.615+0000 7f6b66a00700 1 -- 172.21.15.18:0/4155106942 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7f6b60010db0 con 0x7f6b70071230 2024-04-12T20:43:19.380 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:18.615+0000 7f6b66a00700 1 -- 172.21.15.18:0/4155106942 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f6b6001ad60 con 0x7f6b70071230 2024-04-12T20:43:19.380 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:18.615+0000 7f6b75e00700 1 -- 172.21.15.18:0/4155106942 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6b701140a0 con 0x7f6b70071230 2024-04-12T20:43:19.380 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:18.615+0000 7f6b75e00700 1 -- 172.21.15.18:0/4155106942 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6b70114c10 con 0x7f6b70071230 2024-04-12T20:43:19.380 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:18.616+0000 7f6b66a00700 1 -- 172.21.15.18:0/4155106942 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 1) v1 ==== 521+0+0 (secure 0 0 0) 0x7f6b6002e6c0 con 0x7f6b70071230 2024-04-12T20:43:19.380 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:18.616+0000 7f6b66a00700 1 -- 172.21.15.18:0/4155106942 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 721+0+0 (secure 0 0 0) 0x7f6b60024870 con 0x7f6b70071230 2024-04-12T20:43:19.380 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:18.616+0000 7f6b75e00700 1 -- 172.21.15.18:0/4155106942 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6b7005f5f0 con 0x7f6b70071230 2024-04-12T20:43:19.380 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:18.619+0000 7f6b66a00700 1 -- 172.21.15.18:0/4155106942 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71349 (secure 0 0 0) 0x7f6b6001a5c0 con 0x7f6b70071230 2024-04-12T20:43:19.380 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:18.698+0000 7f6b75e00700 1 -- 172.21.15.18:0/4155106942 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7f6b70106880 con 0x7f6b70071230 2024-04-12T20:43:19.380 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:18.698+0000 7f6b66a00700 1 -- 172.21.15.18:0/4155106942 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1249 (secure 0 0 0) 0x7f6b60010920 con 0x7f6b70071230 2024-04-12T20:43:19.380 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:18.700+0000 7f6b75e00700 1 -- 172.21.15.18:0/4155106942 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6b70071230 msgr2=0x7f6b70113960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:43:19.380 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:18.700+0000 7f6b75e00700 1 --2- 172.21.15.18:0/4155106942 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6b70071230 0x7f6b70113960 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f6b60011bd0 tx=0x7f6b60009250 comp rx=0 tx=0).stop 2024-04-12T20:43:19.380 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:18.700+0000 7f6b75e00700 1 -- 172.21.15.18:0/4155106942 shutdown_connections 2024-04-12T20:43:19.381 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:18.700+0000 7f6b75e00700 1 --2- 172.21.15.18:0/4155106942 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6b70071230 0x7f6b70113960 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:43:19.381 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:18.700+0000 7f6b75e00700 1 -- 172.21.15.18:0/4155106942 >> 172.21.15.18:0/4155106942 conn(0x7f6b700fcfb0 msgr2=0x7f6b700fdb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T20:43:19.381 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:18.700+0000 7f6b75e00700 1 -- 172.21.15.18:0/4155106942 shutdown_connections 2024-04-12T20:43:19.381 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:18.700+0000 7f6b75e00700 1 -- 172.21.15.18:0/4155106942 wait complete. 2024-04-12T20:43:19.381 INFO:teuthology.orchestra.run.smithi018.stdout:mgr not available, waiting (3/15)... 2024-04-12T20:43:20.103 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:43:19 smithi018 ceph-mon[28799]: Activating manager daemon smithi018.xjzwga 2024-04-12T20:43:20.103 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:43:19 smithi018 ceph-mon[28799]: mgrmap e2: smithi018.xjzwga(active, starting, since 0.0105876s) 2024-04-12T20:43:20.104 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:43:19 smithi018 ceph-mon[28799]: from='mgr.14100 172.21.15.18:0/1684830192' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-12T20:43:20.104 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:43:19 smithi018 ceph-mon[28799]: from='mgr.14100 172.21.15.18:0/1684830192' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-12T20:43:20.104 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:43:19 smithi018 ceph-mon[28799]: from='mgr.14100 172.21.15.18:0/1684830192' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-12T20:43:20.104 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:43:19 smithi018 ceph-mon[28799]: from='mgr.14100 172.21.15.18:0/1684830192' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "mon metadata", "id": "smithi018"}]: dispatch 2024-04-12T20:43:20.104 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:43:19 smithi018 ceph-mon[28799]: from='mgr.14100 172.21.15.18:0/1684830192' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "mgr metadata", "who": "smithi018.xjzwga", "id": "smithi018.xjzwga"}]: dispatch 2024-04-12T20:43:20.104 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:43:19 smithi018 ceph-mon[28799]: Manager daemon smithi018.xjzwga is now available 2024-04-12T20:43:20.104 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:43:19 smithi018 ceph-mon[28799]: from='mgr.14100 172.21.15.18:0/1684830192' entity='mgr.smithi018.xjzwga' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi018.xjzwga/mirror_snapshot_schedule"}]: dispatch 2024-04-12T20:43:20.104 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:43:19 smithi018 ceph-mon[28799]: from='mgr.14100 172.21.15.18:0/1684830192' entity='mgr.smithi018.xjzwga' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi018.xjzwga/trash_purge_schedule"}]: dispatch 2024-04-12T20:43:20.104 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:43:19 smithi018 ceph-mon[28799]: from='mgr.14100 172.21.15.18:0/1684830192' entity='mgr.smithi018.xjzwga' 2024-04-12T20:43:20.104 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:43:19 smithi018 ceph-mon[28799]: from='mgr.14100 172.21.15.18:0/1684830192' entity='mgr.smithi018.xjzwga' 2024-04-12T20:43:20.104 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:43:19 smithi018 ceph-mon[28799]: from='mgr.14100 172.21.15.18:0/1684830192' entity='mgr.smithi018.xjzwga' 2024-04-12T20:43:21.676 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:43:21 smithi018 ceph-mon[28799]: mgrmap e3: smithi018.xjzwga(active, since 1.01927s) 2024-04-12T20:43:22.603 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:43:22 smithi018 ceph-mon[28799]: mgrmap e4: smithi018.xjzwga(active, since 2s) 2024-04-12T20:43:22.603 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:43:22 smithi018 ceph-mon[28799]: from='client.? 172.21.15.18:0/3870580130' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-04-12T20:43:22.936 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout 2024-04-12T20:43:22.936 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout { 2024-04-12T20:43:22.936 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "fsid": "1cbe407a-f90d-11ee-bc8d-c7b262605968", 2024-04-12T20:43:22.936 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "health": { 2024-04-12T20:43:22.936 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-04-12T20:43:22.936 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-04-12T20:43:22.936 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-04-12T20:43:22.936 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout }, 2024-04-12T20:43:22.936 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-04-12T20:43:22.936 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-04-12T20:43:22.937 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout 0 2024-04-12T20:43:22.937 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout ], 2024-04-12T20:43:22.937 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-04-12T20:43:22.937 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "smithi018" 2024-04-12T20:43:22.937 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout ], 2024-04-12T20:43:22.937 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "quorum_age": 14, 2024-04-12T20:43:22.937 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "monmap": { 2024-04-12T20:43:22.937 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-12T20:43:22.937 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "quincy", 2024-04-12T20:43:22.937 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-04-12T20:43:22.937 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout }, 2024-04-12T20:43:22.937 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-04-12T20:43:22.937 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-12T20:43:22.937 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-04-12T20:43:22.937 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-04-12T20:43:22.938 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-04-12T20:43:22.938 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-04-12T20:43:22.938 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-04-12T20:43:22.938 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-04-12T20:43:22.938 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout }, 2024-04-12T20:43:22.938 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-04-12T20:43:22.938 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-04-12T20:43:22.938 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-04-12T20:43:22.938 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-04-12T20:43:22.938 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-04-12T20:43:22.938 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-04-12T20:43:22.938 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-04-12T20:43:22.938 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-04-12T20:43:22.939 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-04-12T20:43:22.939 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout }, 2024-04-12T20:43:22.939 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-04-12T20:43:22.939 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-12T20:43:22.939 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-04-12T20:43:22.939 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-04-12T20:43:22.939 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout }, 2024-04-12T20:43:22.939 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-04-12T20:43:22.939 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "available": true, 2024-04-12T20:43:22.939 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-04-12T20:43:22.939 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "modules": [ 2024-04-12T20:43:22.939 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "iostat", 2024-04-12T20:43:22.939 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "nfs", 2024-04-12T20:43:22.939 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "restful" 2024-04-12T20:43:22.939 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout ], 2024-04-12T20:43:22.940 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-12T20:43:22.940 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout }, 2024-04-12T20:43:22.940 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-04-12T20:43:22.940 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-12T20:43:22.940 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "modified": "2024-04-12T20:43:00.692941+0000", 2024-04-12T20:43:22.940 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-12T20:43:22.940 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout }, 2024-04-12T20:43:22.940 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-04-12T20:43:22.940 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout } 2024-04-12T20:43:22.940 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:21.933+0000 7fcc39000700 1 Processor -- start 2024-04-12T20:43:22.940 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:21.933+0000 7fcc39000700 1 -- start start 2024-04-12T20:43:22.940 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:21.934+0000 7fcc39000700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fcc3407cf90 0x7fcc3410bcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T20:43:22.940 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:21.934+0000 7fcc39000700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc3407d430 con 0x7fcc3407cf90 2024-04-12T20:43:22.940 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:21.934+0000 7fcc31e00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fcc3407cf90 0x7fcc3410bcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T20:43:22.941 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:21.934+0000 7fcc31e00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fcc3407cf90 0x7fcc3410bcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:58054/0 (socket says 172.21.15.18:58054) 2024-04-12T20:43:22.941 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:21.934+0000 7fcc31e00700 1 -- 172.21.15.18:0/4070380492 learned_addr learned my addr 172.21.15.18:0/4070380492 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T20:43:22.941 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:21.935+0000 7fcc31e00700 1 -- 172.21.15.18:0/4070380492 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc3410c210 con 0x7fcc3407cf90 2024-04-12T20:43:22.941 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:21.936+0000 7fcc31e00700 1 --2- 172.21.15.18:0/4070380492 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fcc3407cf90 0x7fcc3410bcd0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fcc240098e0 tx=0x7fcc24009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=28c7e6e694f15f42 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T20:43:22.941 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:21.936+0000 7fcc30a00700 1 -- 172.21.15.18:0/4070380492 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fcc2400b860 con 0x7fcc3407cf90 2024-04-12T20:43:22.941 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:21.936+0000 7fcc30a00700 1 -- 172.21.15.18:0/4070380492 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7fcc2400be50 con 0x7fcc3407cf90 2024-04-12T20:43:22.941 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:21.936+0000 7fcc30a00700 1 -- 172.21.15.18:0/4070380492 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fcc24010380 con 0x7fcc3407cf90 2024-04-12T20:43:22.941 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:21.937+0000 7fcc39000700 1 -- 172.21.15.18:0/4070380492 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fcc3407cf90 msgr2=0x7fcc3410bcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:43:22.941 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:21.937+0000 7fcc39000700 1 --2- 172.21.15.18:0/4070380492 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fcc3407cf90 0x7fcc3410bcd0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fcc240098e0 tx=0x7fcc24009250 comp rx=0 tx=0).stop 2024-04-12T20:43:22.941 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:21.937+0000 7fcc39000700 1 -- 172.21.15.18:0/4070380492 shutdown_connections 2024-04-12T20:43:22.941 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:21.937+0000 7fcc39000700 1 --2- 172.21.15.18:0/4070380492 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fcc3407cf90 0x7fcc3410bcd0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:43:22.941 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:21.937+0000 7fcc39000700 1 -- 172.21.15.18:0/4070380492 >> 172.21.15.18:0/4070380492 conn(0x7fcc340782e0 msgr2=0x7fcc3407a6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T20:43:22.941 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:21.937+0000 7fcc39000700 1 -- 172.21.15.18:0/4070380492 shutdown_connections 2024-04-12T20:43:22.941 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:21.937+0000 7fcc39000700 1 -- 172.21.15.18:0/4070380492 wait complete. 2024-04-12T20:43:22.941 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:21.938+0000 7fcc39000700 1 Processor -- start 2024-04-12T20:43:22.942 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:21.939+0000 7fcc39000700 1 -- start start 2024-04-12T20:43:22.942 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:21.939+0000 7fcc39000700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fcc3407cf90 0x7fcc34117c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T20:43:22.942 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:21.939+0000 7fcc39000700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc34118190 con 0x7fcc3407cf90 2024-04-12T20:43:22.942 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:21.939+0000 7fcc31e00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fcc3407cf90 0x7fcc34117c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T20:43:22.942 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:21.939+0000 7fcc31e00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fcc3407cf90 0x7fcc34117c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:58070/0 (socket says 172.21.15.18:58070) 2024-04-12T20:43:22.942 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:21.939+0000 7fcc31e00700 1 -- 172.21.15.18:0/3870580130 learned_addr learned my addr 172.21.15.18:0/3870580130 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T20:43:22.942 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:21.940+0000 7fcc31e00700 1 -- 172.21.15.18:0/3870580130 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc240094f0 con 0x7fcc3407cf90 2024-04-12T20:43:22.942 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:21.940+0000 7fcc31e00700 1 --2- 172.21.15.18:0/3870580130 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fcc3407cf90 0x7fcc34117c50 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fcc24006b00 tx=0x7fcc24009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T20:43:22.942 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:21.940+0000 7fcc2a000700 1 -- 172.21.15.18:0/3870580130 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fcc240107c0 con 0x7fcc3407cf90 2024-04-12T20:43:22.942 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:21.940+0000 7fcc2a000700 1 -- 172.21.15.18:0/3870580130 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7fcc24010db0 con 0x7fcc3407cf90 2024-04-12T20:43:22.942 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:21.940+0000 7fcc2a000700 1 -- 172.21.15.18:0/3870580130 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fcc24024cd0 con 0x7fcc3407cf90 2024-04-12T20:43:22.942 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:21.940+0000 7fcc39000700 1 -- 172.21.15.18:0/3870580130 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcc34118390 con 0x7fcc3407cf90 2024-04-12T20:43:22.942 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:21.941+0000 7fcc39000700 1 -- 172.21.15.18:0/3870580130 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcc34118f00 con 0x7fcc3407cf90 2024-04-12T20:43:22.943 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:21.942+0000 7fcc2a000700 1 -- 172.21.15.18:0/3870580130 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 4) v1 ==== 43357+0+0 (secure 0 0 0) 0x7fcc24024540 con 0x7fcc3407cf90 2024-04-12T20:43:22.943 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:21.942+0000 7fcc39000700 1 -- 172.21.15.18:0/3870580130 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcc3405f5f0 con 0x7fcc3407cf90 2024-04-12T20:43:22.943 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:21.942+0000 7fcc2a000700 1 --2- 172.21.15.18:0/3870580130 >> [v2:172.21.15.18:6800/1312802974,v1:172.21.15.18:6801/1312802974] conn(0x7fcc18036790 0x7fcc18038c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T20:43:22.943 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:21.942+0000 7fcc2a000700 1 -- 172.21.15.18:0/3870580130 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 721+0+0 (secure 0 0 0) 0x7fcc2404a4c0 con 0x7fcc3407cf90 2024-04-12T20:43:22.943 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:21.942+0000 7fcc31400700 1 --2- 172.21.15.18:0/3870580130 >> [v2:172.21.15.18:6800/1312802974,v1:172.21.15.18:6801/1312802974] conn(0x7fcc18036790 0x7fcc18038c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T20:43:22.943 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:21.943+0000 7fcc31400700 1 --2- 172.21.15.18:0/3870580130 >> [v2:172.21.15.18:6800/1312802974,v1:172.21.15.18:6801/1312802974] conn(0x7fcc18036790 0x7fcc18038c40 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fcc20009990 tx=0x7fcc20006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T20:43:22.943 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:21.949+0000 7fcc2a000700 1 -- 172.21.15.18:0/3870580130 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7fcc24010920 con 0x7fcc3407cf90 2024-04-12T20:43:22.943 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:22.244+0000 7fcc39000700 1 -- 172.21.15.18:0/3870580130 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7fcc3404bed0 con 0x7fcc3407cf90 2024-04-12T20:43:22.943 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:22.246+0000 7fcc2a000700 1 -- 172.21.15.18:0/3870580130 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1248 (secure 0 0 0) 0x7fcc2402ce40 con 0x7fcc3407cf90 2024-04-12T20:43:22.943 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:22.248+0000 7fcc39000700 1 -- 172.21.15.18:0/3870580130 >> [v2:172.21.15.18:6800/1312802974,v1:172.21.15.18:6801/1312802974] conn(0x7fcc18036790 msgr2=0x7fcc18038c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:43:22.943 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:22.248+0000 7fcc39000700 1 --2- 172.21.15.18:0/3870580130 >> [v2:172.21.15.18:6800/1312802974,v1:172.21.15.18:6801/1312802974] conn(0x7fcc18036790 0x7fcc18038c40 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fcc20009990 tx=0x7fcc20006e40 comp rx=0 tx=0).stop 2024-04-12T20:43:22.943 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:22.248+0000 7fcc39000700 1 -- 172.21.15.18:0/3870580130 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fcc3407cf90 msgr2=0x7fcc34117c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:43:22.943 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:22.248+0000 7fcc39000700 1 --2- 172.21.15.18:0/3870580130 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fcc3407cf90 0x7fcc34117c50 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fcc24006b00 tx=0x7fcc24009250 comp rx=0 tx=0).stop 2024-04-12T20:43:22.943 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:22.248+0000 7fcc39000700 1 -- 172.21.15.18:0/3870580130 shutdown_connections 2024-04-12T20:43:22.944 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:22.248+0000 7fcc39000700 1 --2- 172.21.15.18:0/3870580130 >> [v2:172.21.15.18:6800/1312802974,v1:172.21.15.18:6801/1312802974] conn(0x7fcc18036790 0x7fcc18038c40 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:43:22.944 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:22.248+0000 7fcc39000700 1 --2- 172.21.15.18:0/3870580130 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fcc3407cf90 0x7fcc34117c50 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:43:22.944 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:22.248+0000 7fcc39000700 1 -- 172.21.15.18:0/3870580130 >> 172.21.15.18:0/3870580130 conn(0x7fcc340782e0 msgr2=0x7fcc34078e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T20:43:22.944 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:22.249+0000 7fcc39000700 1 -- 172.21.15.18:0/3870580130 shutdown_connections 2024-04-12T20:43:22.944 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:22.249+0000 7fcc39000700 1 -- 172.21.15.18:0/3870580130 wait complete. 2024-04-12T20:43:22.944 INFO:teuthology.orchestra.run.smithi018.stdout:mgr is available 2024-04-12T20:43:24.244 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:43:23 smithi018 ceph-mon[28799]: from='client.? 172.21.15.18:0/2711930960' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-04-12T20:43:24.591 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout 2024-04-12T20:43:24.591 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout [global] 2024-04-12T20:43:24.591 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout fsid = 1cbe407a-f90d-11ee-bc8d-c7b262605968 2024-04-12T20:43:24.591 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-04-12T20:43:24.591 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-04-12T20:43:24.591 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-04-12T20:43:24.591 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-04-12T20:43:24.591 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout 2024-04-12T20:43:24.592 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout [mgr] 2024-04-12T20:43:24.592 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-04-12T20:43:24.592 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout 2024-04-12T20:43:24.592 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout [osd] 2024-04-12T20:43:24.592 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-04-12T20:43:24.592 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-04-12T20:43:24.592 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-04-12T20:43:24.592 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-04-12T20:43:24.592 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-04-12T20:43:24.592 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:23.711+0000 7fb9eb600700 1 Processor -- start 2024-04-12T20:43:24.592 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:23.711+0000 7fb9eb600700 1 -- start start 2024-04-12T20:43:24.592 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:23.711+0000 7fb9eb600700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb9e407cf90 0x7fb9e410bcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T20:43:24.592 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:23.711+0000 7fb9eb600700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9e407d430 con 0x7fb9e407cf90 2024-04-12T20:43:24.592 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:23.712+0000 7fb9e8c00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb9e407cf90 0x7fb9e410bcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T20:43:24.593 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:23.712+0000 7fb9e8c00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb9e407cf90 0x7fb9e410bcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:58072/0 (socket says 172.21.15.18:58072) 2024-04-12T20:43:24.593 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:23.712+0000 7fb9e8c00700 1 -- 172.21.15.18:0/3622285005 learned_addr learned my addr 172.21.15.18:0/3622285005 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T20:43:24.593 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:23.713+0000 7fb9e8c00700 1 -- 172.21.15.18:0/3622285005 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9e410c210 con 0x7fb9e407cf90 2024-04-12T20:43:24.593 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:23.713+0000 7fb9e8c00700 1 --2- 172.21.15.18:0/3622285005 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb9e407cf90 0x7fb9e410bcd0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fb9d80098e0 tx=0x7fb9d8009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e3ca4405d94a35e1 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T20:43:24.593 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:23.713+0000 7fb9e3400700 1 -- 172.21.15.18:0/3622285005 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fb9d800b860 con 0x7fb9e407cf90 2024-04-12T20:43:24.593 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:23.713+0000 7fb9e3400700 1 -- 172.21.15.18:0/3622285005 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7fb9d800be50 con 0x7fb9e407cf90 2024-04-12T20:43:24.593 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:23.713+0000 7fb9e3400700 1 -- 172.21.15.18:0/3622285005 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fb9d8010380 con 0x7fb9e407cf90 2024-04-12T20:43:24.593 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:23.714+0000 7fb9eb600700 1 -- 172.21.15.18:0/3622285005 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb9e407cf90 msgr2=0x7fb9e410bcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:43:24.593 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:23.714+0000 7fb9eb600700 1 --2- 172.21.15.18:0/3622285005 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb9e407cf90 0x7fb9e410bcd0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fb9d80098e0 tx=0x7fb9d8009250 comp rx=0 tx=0).stop 2024-04-12T20:43:24.593 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:23.714+0000 7fb9eb600700 1 -- 172.21.15.18:0/3622285005 shutdown_connections 2024-04-12T20:43:24.593 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:23.714+0000 7fb9eb600700 1 --2- 172.21.15.18:0/3622285005 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb9e407cf90 0x7fb9e410bcd0 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:43:24.593 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:23.714+0000 7fb9eb600700 1 -- 172.21.15.18:0/3622285005 >> 172.21.15.18:0/3622285005 conn(0x7fb9e40782e0 msgr2=0x7fb9e407a6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T20:43:24.593 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:23.714+0000 7fb9eb600700 1 -- 172.21.15.18:0/3622285005 shutdown_connections 2024-04-12T20:43:24.594 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:23.714+0000 7fb9eb600700 1 -- 172.21.15.18:0/3622285005 wait complete. 2024-04-12T20:43:24.594 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:23.715+0000 7fb9eb600700 1 Processor -- start 2024-04-12T20:43:24.594 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:23.715+0000 7fb9eb600700 1 -- start start 2024-04-12T20:43:24.594 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:23.716+0000 7fb9eb600700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb9e407cf90 0x7fb9e4117c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T20:43:24.594 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:23.716+0000 7fb9eb600700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9e4118190 con 0x7fb9e407cf90 2024-04-12T20:43:24.594 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:23.716+0000 7fb9e8c00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb9e407cf90 0x7fb9e4117c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T20:43:24.594 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:23.716+0000 7fb9e8c00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb9e407cf90 0x7fb9e4117c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:58080/0 (socket says 172.21.15.18:58080) 2024-04-12T20:43:24.594 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:23.716+0000 7fb9e8c00700 1 -- 172.21.15.18:0/2711930960 learned_addr learned my addr 172.21.15.18:0/2711930960 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T20:43:24.594 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:23.716+0000 7fb9e8c00700 1 -- 172.21.15.18:0/2711930960 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9d80094f0 con 0x7fb9e407cf90 2024-04-12T20:43:24.594 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:23.717+0000 7fb9e8c00700 1 --2- 172.21.15.18:0/2711930960 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb9e407cf90 0x7fb9e4117c50 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fb9d8006b00 tx=0x7fb9d8009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T20:43:24.594 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:23.717+0000 7fb9e0c00700 1 -- 172.21.15.18:0/2711930960 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fb9d80107c0 con 0x7fb9e407cf90 2024-04-12T20:43:24.594 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:23.717+0000 7fb9eb600700 1 -- 172.21.15.18:0/2711930960 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb9e4118390 con 0x7fb9e407cf90 2024-04-12T20:43:24.594 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:23.717+0000 7fb9e0c00700 1 -- 172.21.15.18:0/2711930960 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7fb9d8010db0 con 0x7fb9e407cf90 2024-04-12T20:43:24.594 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:23.717+0000 7fb9e0c00700 1 -- 172.21.15.18:0/2711930960 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fb9d801acf0 con 0x7fb9e407cf90 2024-04-12T20:43:24.594 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:23.717+0000 7fb9eb600700 1 -- 172.21.15.18:0/2711930960 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb9e4118f00 con 0x7fb9e407cf90 2024-04-12T20:43:24.595 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:23.718+0000 7fb9e0c00700 1 -- 172.21.15.18:0/2711930960 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 4) v1 ==== 43357+0+0 (secure 0 0 0) 0x7fb9d8010920 con 0x7fb9e407cf90 2024-04-12T20:43:24.595 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:23.718+0000 7fb9eb600700 1 -- 172.21.15.18:0/2711930960 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb9e405f5f0 con 0x7fb9e407cf90 2024-04-12T20:43:24.595 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:23.719+0000 7fb9e0c00700 1 --2- 172.21.15.18:0/2711930960 >> [v2:172.21.15.18:6800/1312802974,v1:172.21.15.18:6801/1312802974] conn(0x7fb9cc036790 0x7fb9cc038c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T20:43:24.595 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:23.719+0000 7fb9e0c00700 1 -- 172.21.15.18:0/2711930960 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 721+0+0 (secure 0 0 0) 0x7fb9d804a5b0 con 0x7fb9e407cf90 2024-04-12T20:43:24.595 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:23.719+0000 7fb9e3e00700 1 --2- 172.21.15.18:0/2711930960 >> [v2:172.21.15.18:6800/1312802974,v1:172.21.15.18:6801/1312802974] conn(0x7fb9cc036790 0x7fb9cc038c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T20:43:24.595 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:23.720+0000 7fb9e3e00700 1 --2- 172.21.15.18:0/2711930960 >> [v2:172.21.15.18:6800/1312802974,v1:172.21.15.18:6801/1312802974] conn(0x7fb9cc036790 0x7fb9cc038c40 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fb9d0009990 tx=0x7fb9d0006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T20:43:24.595 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:23.724+0000 7fb9e0c00700 1 -- 172.21.15.18:0/2711930960 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7fb9d8010bd0 con 0x7fb9e407cf90 2024-04-12T20:43:24.595 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:23.930+0000 7fb9eb600700 1 -- 172.21.15.18:0/2711930960 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) v1 -- 0x7fb9e404bed0 con 0x7fb9e407cf90 2024-04-12T20:43:24.595 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:23.931+0000 7fb9e0c00700 1 -- 172.21.15.18:0/2711930960 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v3) v1 ==== 70+0+373 (secure 0 0 0) 0x7fb9d802be40 con 0x7fb9e407cf90 2024-04-12T20:43:24.595 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:23.934+0000 7fb9eb600700 1 -- 172.21.15.18:0/2711930960 >> [v2:172.21.15.18:6800/1312802974,v1:172.21.15.18:6801/1312802974] conn(0x7fb9cc036790 msgr2=0x7fb9cc038c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:43:24.595 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:23.934+0000 7fb9eb600700 1 --2- 172.21.15.18:0/2711930960 >> [v2:172.21.15.18:6800/1312802974,v1:172.21.15.18:6801/1312802974] conn(0x7fb9cc036790 0x7fb9cc038c40 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fb9d0009990 tx=0x7fb9d0006e40 comp rx=0 tx=0).stop 2024-04-12T20:43:24.595 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:23.934+0000 7fb9eb600700 1 -- 172.21.15.18:0/2711930960 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb9e407cf90 msgr2=0x7fb9e4117c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:43:24.595 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:23.934+0000 7fb9eb600700 1 --2- 172.21.15.18:0/2711930960 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb9e407cf90 0x7fb9e4117c50 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fb9d8006b00 tx=0x7fb9d8009250 comp rx=0 tx=0).stop 2024-04-12T20:43:24.596 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:23.934+0000 7fb9eb600700 1 -- 172.21.15.18:0/2711930960 shutdown_connections 2024-04-12T20:43:24.596 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:23.934+0000 7fb9eb600700 1 --2- 172.21.15.18:0/2711930960 >> [v2:172.21.15.18:6800/1312802974,v1:172.21.15.18:6801/1312802974] conn(0x7fb9cc036790 0x7fb9cc038c40 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:43:24.596 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:23.934+0000 7fb9eb600700 1 --2- 172.21.15.18:0/2711930960 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb9e407cf90 0x7fb9e4117c50 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:43:24.596 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:23.934+0000 7fb9eb600700 1 -- 172.21.15.18:0/2711930960 >> 172.21.15.18:0/2711930960 conn(0x7fb9e40782e0 msgr2=0x7fb9e4078e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T20:43:24.596 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:23.935+0000 7fb9eb600700 1 -- 172.21.15.18:0/2711930960 shutdown_connections 2024-04-12T20:43:24.596 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:23.935+0000 7fb9eb600700 1 -- 172.21.15.18:0/2711930960 wait complete. 2024-04-12T20:43:24.596 INFO:teuthology.orchestra.run.smithi018.stdout:Enabling cephadm module... 2024-04-12T20:43:25.853 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:43:25 smithi018 ceph-mon[28799]: from='client.? 172.21.15.18:0/3861917165' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "cephadm"}]: dispatch 2024-04-12T20:43:26.267 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:25.184+0000 7f3d66a00700 1 Processor -- start 2024-04-12T20:43:26.267 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:25.184+0000 7f3d66a00700 1 -- start start 2024-04-12T20:43:26.267 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:25.185+0000 7f3d66a00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3d60102920 0x7f3d60102d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T20:43:26.267 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:25.185+0000 7f3d66a00700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d60103300 con 0x7f3d60102920 2024-04-12T20:43:26.267 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:25.185+0000 7f3d5fe00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3d60102920 0x7f3d60102d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T20:43:26.268 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:25.185+0000 7f3d5fe00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3d60102920 0x7f3d60102d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:58094/0 (socket says 172.21.15.18:58094) 2024-04-12T20:43:26.268 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:25.185+0000 7f3d5fe00700 1 -- 172.21.15.18:0/151360626 learned_addr learned my addr 172.21.15.18:0/151360626 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T20:43:26.268 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:25.186+0000 7f3d5fe00700 1 -- 172.21.15.18:0/151360626 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d60103440 con 0x7f3d60102920 2024-04-12T20:43:26.268 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:25.186+0000 7f3d5fe00700 1 --2- 172.21.15.18:0/151360626 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3d60102920 0x7f3d60102d30 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f3d500098e0 tx=0x7f3d50009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6601de9771738f2c server_cookie=0 in_seq=0 out_seq=0 2024-04-12T20:43:26.268 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:25.186+0000 7f3d5ea00700 1 -- 172.21.15.18:0/151360626 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f3d5000b860 con 0x7f3d60102920 2024-04-12T20:43:26.268 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:25.187+0000 7f3d5ea00700 1 -- 172.21.15.18:0/151360626 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7f3d5000be50 con 0x7f3d60102920 2024-04-12T20:43:26.268 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:25.187+0000 7f3d5ea00700 1 -- 172.21.15.18:0/151360626 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f3d50010380 con 0x7f3d60102920 2024-04-12T20:43:26.268 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:25.187+0000 7f3d66a00700 1 -- 172.21.15.18:0/151360626 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3d60102920 msgr2=0x7f3d60102d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:43:26.268 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:25.187+0000 7f3d66a00700 1 --2- 172.21.15.18:0/151360626 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3d60102920 0x7f3d60102d30 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f3d500098e0 tx=0x7f3d50009250 comp rx=0 tx=0).stop 2024-04-12T20:43:26.268 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:25.187+0000 7f3d66a00700 1 -- 172.21.15.18:0/151360626 shutdown_connections 2024-04-12T20:43:26.268 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:25.187+0000 7f3d66a00700 1 --2- 172.21.15.18:0/151360626 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3d60102920 0x7f3d60102d30 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:43:26.268 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:25.187+0000 7f3d66a00700 1 -- 172.21.15.18:0/151360626 >> 172.21.15.18:0/151360626 conn(0x7f3d600fe0a0 msgr2=0x7f3d601004b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T20:43:26.268 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:25.188+0000 7f3d66a00700 1 -- 172.21.15.18:0/151360626 shutdown_connections 2024-04-12T20:43:26.269 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:25.188+0000 7f3d66a00700 1 -- 172.21.15.18:0/151360626 wait complete. 2024-04-12T20:43:26.269 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:25.189+0000 7f3d66a00700 1 Processor -- start 2024-04-12T20:43:26.269 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:25.189+0000 7f3d66a00700 1 -- start start 2024-04-12T20:43:26.269 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:25.189+0000 7f3d66a00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3d60102920 0x7f3d60113910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T20:43:26.269 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:25.189+0000 7f3d66a00700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d60113e50 con 0x7f3d60102920 2024-04-12T20:43:26.269 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:25.189+0000 7f3d5fe00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3d60102920 0x7f3d60113910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T20:43:26.269 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:25.189+0000 7f3d5fe00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3d60102920 0x7f3d60113910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:58096/0 (socket says 172.21.15.18:58096) 2024-04-12T20:43:26.269 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:25.189+0000 7f3d5fe00700 1 -- 172.21.15.18:0/3861917165 learned_addr learned my addr 172.21.15.18:0/3861917165 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T20:43:26.269 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:25.190+0000 7f3d5fe00700 1 -- 172.21.15.18:0/3861917165 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d500094f0 con 0x7f3d60102920 2024-04-12T20:43:26.269 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:25.190+0000 7f3d5fe00700 1 --2- 172.21.15.18:0/3861917165 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3d60102920 0x7f3d60113910 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f3d50000c00 tx=0x7f3d50009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T20:43:26.269 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:25.190+0000 7f3d57e00700 1 -- 172.21.15.18:0/3861917165 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f3d50010380 con 0x7f3d60102920 2024-04-12T20:43:26.269 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:25.190+0000 7f3d57e00700 1 -- 172.21.15.18:0/3861917165 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7f3d50010c50 con 0x7f3d60102920 2024-04-12T20:43:26.269 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:25.190+0000 7f3d57e00700 1 -- 172.21.15.18:0/3861917165 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f3d5001ac50 con 0x7f3d60102920 2024-04-12T20:43:26.270 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:25.190+0000 7f3d66a00700 1 -- 172.21.15.18:0/3861917165 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3d60114050 con 0x7f3d60102920 2024-04-12T20:43:26.270 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:25.191+0000 7f3d66a00700 1 -- 172.21.15.18:0/3861917165 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3d60114b70 con 0x7f3d60102920 2024-04-12T20:43:26.270 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:25.192+0000 7f3d57e00700 1 -- 172.21.15.18:0/3861917165 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 4) v1 ==== 43357+0+0 (secure 0 0 0) 0x7f3d500107c0 con 0x7f3d60102920 2024-04-12T20:43:26.270 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:25.192+0000 7f3d66a00700 1 -- 172.21.15.18:0/3861917165 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3d6005f5f0 con 0x7f3d60102920 2024-04-12T20:43:26.270 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:25.192+0000 7f3d57e00700 1 --2- 172.21.15.18:0/3861917165 >> [v2:172.21.15.18:6800/1312802974,v1:172.21.15.18:6801/1312802974] conn(0x7f3d44036740 0x7f3d44038bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T20:43:26.270 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:25.192+0000 7f3d57e00700 1 -- 172.21.15.18:0/3861917165 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 721+0+0 (secure 0 0 0) 0x7f3d5004a500 con 0x7f3d60102920 2024-04-12T20:43:26.270 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:25.192+0000 7f3d5f400700 1 --2- 172.21.15.18:0/3861917165 >> [v2:172.21.15.18:6800/1312802974,v1:172.21.15.18:6801/1312802974] conn(0x7f3d44036740 0x7f3d44038bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T20:43:26.270 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:25.193+0000 7f3d5f400700 1 --2- 172.21.15.18:0/3861917165 >> [v2:172.21.15.18:6800/1312802974,v1:172.21.15.18:6801/1312802974] conn(0x7f3d44036740 0x7f3d44038bf0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f3d4c009990 tx=0x7f3d4c006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T20:43:26.270 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:25.197+0000 7f3d57e00700 1 -- 172.21.15.18:0/3861917165 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7f3d5001a430 con 0x7f3d60102920 2024-04-12T20:43:26.271 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:25.469+0000 7f3d66a00700 1 -- 172.21.15.18:0/3861917165 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "cephadm"} v 0) v1 -- 0x7f3d6004bed0 con 0x7f3d60102920 2024-04-12T20:43:26.271 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:25.526+0000 7f3d57e00700 1 -- 172.21.15.18:0/3861917165 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mgrmap(e 5) v1 ==== 43368+0+0 (secure 0 0 0) 0x7f3d50019740 con 0x7f3d60102920 2024-04-12T20:43:26.271 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:25.528+0000 7f3d57e00700 1 -- 172.21.15.18:0/3861917165 <== mon.0 v2:172.21.15.18:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "cephadm"}]=0 v5) v1 ==== 86+0+0 (secure 0 0 0) 0x7f3d5004c4a0 con 0x7f3d60102920 2024-04-12T20:43:26.271 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:25.531+0000 7f3d66a00700 1 -- 172.21.15.18:0/3861917165 >> [v2:172.21.15.18:6800/1312802974,v1:172.21.15.18:6801/1312802974] conn(0x7f3d44036740 msgr2=0x7f3d44038bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:43:26.271 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:25.531+0000 7f3d66a00700 1 --2- 172.21.15.18:0/3861917165 >> [v2:172.21.15.18:6800/1312802974,v1:172.21.15.18:6801/1312802974] conn(0x7f3d44036740 0x7f3d44038bf0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f3d4c009990 tx=0x7f3d4c006e40 comp rx=0 tx=0).stop 2024-04-12T20:43:26.271 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:25.531+0000 7f3d66a00700 1 -- 172.21.15.18:0/3861917165 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3d60102920 msgr2=0x7f3d60113910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:43:26.271 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:25.531+0000 7f3d66a00700 1 --2- 172.21.15.18:0/3861917165 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3d60102920 0x7f3d60113910 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f3d50000c00 tx=0x7f3d50009250 comp rx=0 tx=0).stop 2024-04-12T20:43:26.271 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:25.531+0000 7f3d66a00700 1 -- 172.21.15.18:0/3861917165 shutdown_connections 2024-04-12T20:43:26.271 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:25.531+0000 7f3d66a00700 1 --2- 172.21.15.18:0/3861917165 >> [v2:172.21.15.18:6800/1312802974,v1:172.21.15.18:6801/1312802974] conn(0x7f3d44036740 0x7f3d44038bf0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:43:26.271 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:25.531+0000 7f3d66a00700 1 --2- 172.21.15.18:0/3861917165 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3d60102920 0x7f3d60113910 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:43:26.271 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:25.531+0000 7f3d66a00700 1 -- 172.21.15.18:0/3861917165 >> 172.21.15.18:0/3861917165 conn(0x7f3d600fe0a0 msgr2=0x7f3d600febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T20:43:26.272 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:25.531+0000 7f3d66a00700 1 -- 172.21.15.18:0/3861917165 shutdown_connections 2024-04-12T20:43:26.272 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:25.531+0000 7f3d66a00700 1 -- 172.21.15.18:0/3861917165 wait complete. 2024-04-12T20:43:26.836 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:43:26 smithi018 ceph-mon[28799]: from='client.? 172.21.15.18:0/3861917165' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "cephadm"}]': finished 2024-04-12T20:43:26.837 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:43:26 smithi018 ceph-mon[28799]: mgrmap e5: smithi018.xjzwga(active, since 6s) 2024-04-12T20:43:27.752 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:43:27 smithi018 ceph-mon[28799]: from='client.? 172.21.15.18:0/3891782035' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-04-12T20:43:27.768 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout { 2024-04-12T20:43:27.768 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "epoch": 5, 2024-04-12T20:43:27.768 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "available": true, 2024-04-12T20:43:27.768 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "active_name": "smithi018.xjzwga", 2024-04-12T20:43:27.768 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-04-12T20:43:27.768 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout } 2024-04-12T20:43:27.768 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:26.836+0000 7fd57b400700 1 Processor -- start 2024-04-12T20:43:27.769 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:26.836+0000 7fd57b400700 1 -- start start 2024-04-12T20:43:27.769 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:26.837+0000 7fd57b400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd574071290 0x7fd574105500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T20:43:27.769 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:26.837+0000 7fd57b400700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5740716a0 con 0x7fd574071290 2024-04-12T20:43:27.769 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:26.837+0000 7fd578a00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd574071290 0x7fd574105500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T20:43:27.769 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:26.838+0000 7fd578a00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd574071290 0x7fd574105500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:58130/0 (socket says 172.21.15.18:58130) 2024-04-12T20:43:27.769 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:26.838+0000 7fd578a00700 1 -- 172.21.15.18:0/1484706676 learned_addr learned my addr 172.21.15.18:0/1484706676 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T20:43:27.769 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:26.838+0000 7fd578a00700 1 -- 172.21.15.18:0/1484706676 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd574105a40 con 0x7fd574071290 2024-04-12T20:43:27.769 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:26.838+0000 7fd578a00700 1 --2- 172.21.15.18:0/1484706676 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd574071290 0x7fd574105500 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fd5680098e0 tx=0x7fd568009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1cf9bf03acf0a288 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T20:43:27.769 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:26.839+0000 7fd573400700 1 -- 172.21.15.18:0/1484706676 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fd56800b860 con 0x7fd574071290 2024-04-12T20:43:27.769 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:26.839+0000 7fd573400700 1 -- 172.21.15.18:0/1484706676 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7fd56800be50 con 0x7fd574071290 2024-04-12T20:43:27.769 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:26.839+0000 7fd573400700 1 -- 172.21.15.18:0/1484706676 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fd568010380 con 0x7fd574071290 2024-04-12T20:43:27.770 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:26.839+0000 7fd57b400700 1 -- 172.21.15.18:0/1484706676 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd574071290 msgr2=0x7fd574105500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:43:27.770 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:26.839+0000 7fd57b400700 1 --2- 172.21.15.18:0/1484706676 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd574071290 0x7fd574105500 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fd5680098e0 tx=0x7fd568009250 comp rx=0 tx=0).stop 2024-04-12T20:43:27.770 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:26.839+0000 7fd57b400700 1 -- 172.21.15.18:0/1484706676 shutdown_connections 2024-04-12T20:43:27.770 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:26.839+0000 7fd57b400700 1 --2- 172.21.15.18:0/1484706676 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd574071290 0x7fd574105500 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:43:27.770 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:26.839+0000 7fd57b400700 1 -- 172.21.15.18:0/1484706676 >> 172.21.15.18:0/1484706676 conn(0x7fd5740fcfb0 msgr2=0x7fd5740ff3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T20:43:27.770 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:26.840+0000 7fd57b400700 1 -- 172.21.15.18:0/1484706676 shutdown_connections 2024-04-12T20:43:27.770 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:26.840+0000 7fd57b400700 1 -- 172.21.15.18:0/1484706676 wait complete. 2024-04-12T20:43:27.770 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:26.840+0000 7fd57b400700 1 Processor -- start 2024-04-12T20:43:27.770 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:26.841+0000 7fd57b400700 1 -- start start 2024-04-12T20:43:27.770 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:26.841+0000 7fd57b400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd574071290 0x7fd574113930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T20:43:27.770 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:26.841+0000 7fd57b400700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd574113e70 con 0x7fd574071290 2024-04-12T20:43:27.770 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:26.841+0000 7fd578a00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd574071290 0x7fd574113930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T20:43:27.770 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:26.841+0000 7fd578a00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd574071290 0x7fd574113930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:58136/0 (socket says 172.21.15.18:58136) 2024-04-12T20:43:27.770 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:26.841+0000 7fd578a00700 1 -- 172.21.15.18:0/3891782035 learned_addr learned my addr 172.21.15.18:0/3891782035 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T20:43:27.771 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:26.842+0000 7fd578a00700 1 -- 172.21.15.18:0/3891782035 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5680094f0 con 0x7fd574071290 2024-04-12T20:43:27.771 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:26.842+0000 7fd578a00700 1 --2- 172.21.15.18:0/3891782035 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd574071290 0x7fd574113930 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fd568011bd0 tx=0x7fd568009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T20:43:27.771 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:26.842+0000 7fd570c00700 1 -- 172.21.15.18:0/3891782035 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fd5680107c0 con 0x7fd574071290 2024-04-12T20:43:27.771 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:26.842+0000 7fd570c00700 1 -- 172.21.15.18:0/3891782035 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7fd568010db0 con 0x7fd574071290 2024-04-12T20:43:27.771 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:26.842+0000 7fd57b400700 1 -- 172.21.15.18:0/3891782035 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd574114070 con 0x7fd574071290 2024-04-12T20:43:27.771 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:26.842+0000 7fd570c00700 1 -- 172.21.15.18:0/3891782035 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fd56801ad60 con 0x7fd574071290 2024-04-12T20:43:27.771 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:26.843+0000 7fd57b400700 1 -- 172.21.15.18:0/3891782035 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd574114be0 con 0x7fd574071290 2024-04-12T20:43:27.771 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:26.844+0000 7fd570c00700 1 -- 172.21.15.18:0/3891782035 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 5) v1 ==== 43368+0+0 (secure 0 0 0) 0x7fd56801a5c0 con 0x7fd574071290 2024-04-12T20:43:27.771 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:26.844+0000 7fd57b400700 1 -- 172.21.15.18:0/3891782035 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd57405f5f0 con 0x7fd574071290 2024-04-12T20:43:27.771 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:26.844+0000 7fd570c00700 1 --2- 172.21.15.18:0/3891782035 >> [v2:172.21.15.18:6800/1312802974,v1:172.21.15.18:6801/1312802974] conn(0x7fd55c036830 0x7fd55c038ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T20:43:27.771 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:26.844+0000 7fd570c00700 1 -- 172.21.15.18:0/3891782035 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 721+0+0 (secure 0 0 0) 0x7fd56804ab10 con 0x7fd574071290 2024-04-12T20:43:27.771 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:26.845+0000 7fd573e00700 1 -- 172.21.15.18:0/3891782035 >> [v2:172.21.15.18:6800/1312802974,v1:172.21.15.18:6801/1312802974] conn(0x7fd55c036830 msgr2=0x7fd55c038ce0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.18:6800/1312802974 2024-04-12T20:43:27.771 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:26.845+0000 7fd573e00700 1 --2- 172.21.15.18:0/3891782035 >> [v2:172.21.15.18:6800/1312802974,v1:172.21.15.18:6801/1312802974] conn(0x7fd55c036830 0x7fd55c038ce0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-12T20:43:27.772 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:26.850+0000 7fd570c00700 1 -- 172.21.15.18:0/3891782035 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7fd568023bc0 con 0x7fd574071290 2024-04-12T20:43:27.772 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:27.046+0000 7fd573e00700 1 -- 172.21.15.18:0/3891782035 >> [v2:172.21.15.18:6800/1312802974,v1:172.21.15.18:6801/1312802974] conn(0x7fd55c036830 msgr2=0x7fd55c038ce0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.18:6800/1312802974 2024-04-12T20:43:27.772 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:27.046+0000 7fd573e00700 1 --2- 172.21.15.18:0/3891782035 >> [v2:172.21.15.18:6800/1312802974,v1:172.21.15.18:6801/1312802974] conn(0x7fd55c036830 0x7fd55c038ce0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-12T20:43:27.772 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:27.119+0000 7fd57b400700 1 -- 172.21.15.18:0/3891782035 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) v1 -- 0x7fd574115750 con 0x7fd574071290 2024-04-12T20:43:27.772 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:27.120+0000 7fd570c00700 1 -- 172.21.15.18:0/3891782035 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v5) v1 ==== 56+0+103 (secure 0 0 0) 0x7fd568023390 con 0x7fd574071290 2024-04-12T20:43:27.772 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:27.123+0000 7fd57b400700 1 -- 172.21.15.18:0/3891782035 >> [v2:172.21.15.18:6800/1312802974,v1:172.21.15.18:6801/1312802974] conn(0x7fd55c036830 msgr2=0x7fd55c038ce0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-12T20:43:27.772 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:27.123+0000 7fd57b400700 1 --2- 172.21.15.18:0/3891782035 >> [v2:172.21.15.18:6800/1312802974,v1:172.21.15.18:6801/1312802974] conn(0x7fd55c036830 0x7fd55c038ce0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:43:27.772 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:27.123+0000 7fd57b400700 1 -- 172.21.15.18:0/3891782035 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd574071290 msgr2=0x7fd574113930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:43:27.772 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:27.123+0000 7fd57b400700 1 --2- 172.21.15.18:0/3891782035 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd574071290 0x7fd574113930 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fd568011bd0 tx=0x7fd568009250 comp rx=0 tx=0).stop 2024-04-12T20:43:27.772 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:27.123+0000 7fd57b400700 1 -- 172.21.15.18:0/3891782035 shutdown_connections 2024-04-12T20:43:27.772 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:27.123+0000 7fd57b400700 1 --2- 172.21.15.18:0/3891782035 >> [v2:172.21.15.18:6800/1312802974,v1:172.21.15.18:6801/1312802974] conn(0x7fd55c036830 0x7fd55c038ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:43:27.772 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:27.123+0000 7fd57b400700 1 --2- 172.21.15.18:0/3891782035 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd574071290 0x7fd574113930 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:43:27.772 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:27.123+0000 7fd57b400700 1 -- 172.21.15.18:0/3891782035 >> 172.21.15.18:0/3891782035 conn(0x7fd5740fcfb0 msgr2=0x7fd574102f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T20:43:27.772 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:27.124+0000 7fd57b400700 1 -- 172.21.15.18:0/3891782035 shutdown_connections 2024-04-12T20:43:27.773 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:27.124+0000 7fd57b400700 1 -- 172.21.15.18:0/3891782035 wait complete. 2024-04-12T20:43:27.773 INFO:teuthology.orchestra.run.smithi018.stdout:Waiting for the mgr to restart... 2024-04-12T20:43:27.773 INFO:teuthology.orchestra.run.smithi018.stdout:Waiting for mgr epoch 5... 2024-04-12T20:43:34.607 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:43:34 smithi018 ceph-mon[28799]: Active manager daemon smithi018.xjzwga restarted 2024-04-12T20:43:34.607 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:43:34 smithi018 ceph-mon[28799]: Activating manager daemon smithi018.xjzwga 2024-04-12T20:43:34.607 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:43:34 smithi018 ceph-mon[28799]: osdmap e2: 0 total, 0 up, 0 in 2024-04-12T20:43:34.607 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:43:34 smithi018 ceph-mon[28799]: mgrmap e6: smithi018.xjzwga(active, starting, since 0.00912757s) 2024-04-12T20:43:34.607 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:43:34 smithi018 ceph-mon[28799]: from='mgr.14120 172.21.15.18:0/4270481350' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "mon metadata", "id": "smithi018"}]: dispatch 2024-04-12T20:43:34.607 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:43:34 smithi018 ceph-mon[28799]: from='mgr.14120 172.21.15.18:0/4270481350' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "mgr metadata", "who": "smithi018.xjzwga", "id": "smithi018.xjzwga"}]: dispatch 2024-04-12T20:43:34.607 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:43:34 smithi018 ceph-mon[28799]: from='mgr.14120 172.21.15.18:0/4270481350' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-12T20:43:34.607 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:43:34 smithi018 ceph-mon[28799]: from='mgr.14120 172.21.15.18:0/4270481350' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-12T20:43:34.607 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:43:34 smithi018 ceph-mon[28799]: from='mgr.14120 172.21.15.18:0/4270481350' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-12T20:43:34.608 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:43:34 smithi018 ceph-mon[28799]: Manager daemon smithi018.xjzwga is now available 2024-04-12T20:43:34.608 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:43:34 smithi018 ceph-mon[28799]: from='mgr.14120 172.21.15.18:0/4270481350' entity='mgr.smithi018.xjzwga' 2024-04-12T20:43:35.603 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:43:35 smithi018 ceph-mon[28799]: Found migration_current of "None". Setting to last migration. 2024-04-12T20:43:35.603 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:43:35 smithi018 ceph-mon[28799]: from='mgr.14120 172.21.15.18:0/4270481350' entity='mgr.smithi018.xjzwga' 2024-04-12T20:43:35.604 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:43:35 smithi018 ceph-mon[28799]: from='mgr.14120 172.21.15.18:0/4270481350' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-12T20:43:35.604 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:43:35 smithi018 ceph-mon[28799]: from='mgr.14120 172.21.15.18:0/4270481350' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-12T20:43:35.604 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:43:35 smithi018 ceph-mon[28799]: from='mgr.14120 172.21.15.18:0/4270481350' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-12T20:43:35.604 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:43:35 smithi018 ceph-mon[28799]: from='mgr.14120 172.21.15.18:0/4270481350' entity='mgr.smithi018.xjzwga' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi018.xjzwga/mirror_snapshot_schedule"}]: dispatch 2024-04-12T20:43:35.604 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:43:35 smithi018 ceph-mon[28799]: from='mgr.14120 172.21.15.18:0/4270481350' entity='mgr.smithi018.xjzwga' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi018.xjzwga/trash_purge_schedule"}]: dispatch 2024-04-12T20:43:35.604 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:43:35 smithi018 ceph-mon[28799]: mgrmap e7: smithi018.xjzwga(active, since 1.01679s) 2024-04-12T20:43:35.956 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout { 2024-04-12T20:43:35.956 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 7, 2024-04-12T20:43:35.957 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "initialized": true 2024-04-12T20:43:35.957 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout } 2024-04-12T20:43:35.957 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:28.362+0000 7f982f600700 1 Processor -- start 2024-04-12T20:43:35.957 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:28.362+0000 7f982f600700 1 -- start start 2024-04-12T20:43:35.957 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:28.362+0000 7f982f600700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f98280711f0 0x7f98281079a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T20:43:35.957 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:28.362+0000 7f982f600700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9828071600 con 0x7f98280711f0 2024-04-12T20:43:35.957 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:28.363+0000 7f982cc00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f98280711f0 0x7f98281079a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T20:43:35.957 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:28.363+0000 7f982cc00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f98280711f0 0x7f98281079a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:58146/0 (socket says 172.21.15.18:58146) 2024-04-12T20:43:35.957 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:28.363+0000 7f982cc00700 1 -- 172.21.15.18:0/3680916765 learned_addr learned my addr 172.21.15.18:0/3680916765 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T20:43:35.957 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:28.363+0000 7f982cc00700 1 -- 172.21.15.18:0/3680916765 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9828103390 con 0x7f98280711f0 2024-04-12T20:43:35.957 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:28.364+0000 7f982cc00700 1 --2- 172.21.15.18:0/3680916765 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f98280711f0 0x7f98281079a0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f98180098e0 tx=0x7f9818009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4961b18db41a0d45 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T20:43:35.957 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:28.364+0000 7f9827e00700 1 -- 172.21.15.18:0/3680916765 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f981800b860 con 0x7f98280711f0 2024-04-12T20:43:35.957 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:28.364+0000 7f9827e00700 1 -- 172.21.15.18:0/3680916765 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7f981800be50 con 0x7f98280711f0 2024-04-12T20:43:35.958 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:28.364+0000 7f9827e00700 1 -- 172.21.15.18:0/3680916765 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f9818010380 con 0x7f98280711f0 2024-04-12T20:43:35.958 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:28.365+0000 7f982f600700 1 -- 172.21.15.18:0/3680916765 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f98280711f0 msgr2=0x7f98281079a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:43:35.958 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:28.365+0000 7f982f600700 1 --2- 172.21.15.18:0/3680916765 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f98280711f0 0x7f98281079a0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f98180098e0 tx=0x7f9818009250 comp rx=0 tx=0).stop 2024-04-12T20:43:35.958 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:28.365+0000 7f982f600700 1 -- 172.21.15.18:0/3680916765 shutdown_connections 2024-04-12T20:43:35.958 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:28.365+0000 7f982f600700 1 --2- 172.21.15.18:0/3680916765 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f98280711f0 0x7f98281079a0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:43:35.958 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:28.365+0000 7f982f600700 1 -- 172.21.15.18:0/3680916765 >> 172.21.15.18:0/3680916765 conn(0x7f98280fcfb0 msgr2=0x7f98280ff3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T20:43:35.958 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:28.365+0000 7f982f600700 1 -- 172.21.15.18:0/3680916765 shutdown_connections 2024-04-12T20:43:35.958 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:28.365+0000 7f982f600700 1 -- 172.21.15.18:0/3680916765 wait complete. 2024-04-12T20:43:35.958 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:28.366+0000 7f982f600700 1 Processor -- start 2024-04-12T20:43:35.958 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:28.366+0000 7f982f600700 1 -- start start 2024-04-12T20:43:35.958 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:28.366+0000 7f982f600700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f98280711f0 0x7f9828113920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T20:43:35.958 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:28.366+0000 7f982f600700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9828113e60 con 0x7f98280711f0 2024-04-12T20:43:35.958 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:28.366+0000 7f982cc00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f98280711f0 0x7f9828113920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T20:43:35.959 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:28.366+0000 7f982cc00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f98280711f0 0x7f9828113920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:58160/0 (socket says 172.21.15.18:58160) 2024-04-12T20:43:35.959 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:28.366+0000 7f982cc00700 1 -- 172.21.15.18:0/3240366219 learned_addr learned my addr 172.21.15.18:0/3240366219 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T20:43:35.959 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:28.367+0000 7f982cc00700 1 -- 172.21.15.18:0/3240366219 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98180094f0 con 0x7f98280711f0 2024-04-12T20:43:35.959 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:28.367+0000 7f982cc00700 1 --2- 172.21.15.18:0/3240366219 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f98280711f0 0x7f9828113920 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f9818011bd0 tx=0x7f9818009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T20:43:35.959 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:28.367+0000 7f9825600700 1 -- 172.21.15.18:0/3240366219 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f98180107c0 con 0x7f98280711f0 2024-04-12T20:43:35.959 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:28.367+0000 7f9825600700 1 -- 172.21.15.18:0/3240366219 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7f9818010db0 con 0x7f98280711f0 2024-04-12T20:43:35.959 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:28.367+0000 7f982f600700 1 -- 172.21.15.18:0/3240366219 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9828114060 con 0x7f98280711f0 2024-04-12T20:43:35.959 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:28.367+0000 7f9825600700 1 -- 172.21.15.18:0/3240366219 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f981801ad00 con 0x7f98280711f0 2024-04-12T20:43:35.959 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:28.368+0000 7f982f600700 1 -- 172.21.15.18:0/3240366219 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9828114bd0 con 0x7f98280711f0 2024-04-12T20:43:35.959 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:28.369+0000 7f9825600700 1 -- 172.21.15.18:0/3240366219 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 5) v1 ==== 43368+0+0 (secure 0 0 0) 0x7f98180193c0 con 0x7f98280711f0 2024-04-12T20:43:35.959 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:28.369+0000 7f9825600700 1 --2- 172.21.15.18:0/3240366219 >> [v2:172.21.15.18:6800/1312802974,v1:172.21.15.18:6801/1312802974] conn(0x7f980c036790 0x7f980c038c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T20:43:35.959 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:28.369+0000 7f982f600700 1 -- 172.21.15.18:0/3240366219 --> [v2:172.21.15.18:6800/1312802974,v1:172.21.15.18:6801/1312802974] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7f982805f5f0 con 0x7f980c036790 2024-04-12T20:43:35.959 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:28.369+0000 7f9825600700 1 -- 172.21.15.18:0/3240366219 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 721+0+0 (secure 0 0 0) 0x7f981804a090 con 0x7f98280711f0 2024-04-12T20:43:35.959 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:28.369+0000 7f981fe00700 1 -- 172.21.15.18:0/3240366219 >> [v2:172.21.15.18:6800/1312802974,v1:172.21.15.18:6801/1312802974] conn(0x7f980c036790 msgr2=0x7f980c038c40 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.18:6800/1312802974 2024-04-12T20:43:35.960 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:28.369+0000 7f981fe00700 1 --2- 172.21.15.18:0/3240366219 >> [v2:172.21.15.18:6800/1312802974,v1:172.21.15.18:6801/1312802974] conn(0x7f980c036790 0x7f980c038c40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-12T20:43:35.960 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:28.570+0000 7f981fe00700 1 -- 172.21.15.18:0/3240366219 >> [v2:172.21.15.18:6800/1312802974,v1:172.21.15.18:6801/1312802974] conn(0x7f980c036790 msgr2=0x7f980c038c40 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.18:6800/1312802974 2024-04-12T20:43:35.960 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:28.570+0000 7f981fe00700 1 --2- 172.21.15.18:0/3240366219 >> [v2:172.21.15.18:6800/1312802974,v1:172.21.15.18:6801/1312802974] conn(0x7f980c036790 0x7f980c038c40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-12T20:43:35.960 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:28.971+0000 7f981fe00700 1 -- 172.21.15.18:0/3240366219 >> [v2:172.21.15.18:6800/1312802974,v1:172.21.15.18:6801/1312802974] conn(0x7f980c036790 msgr2=0x7f980c038c40 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.18:6800/1312802974 2024-04-12T20:43:35.960 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:28.971+0000 7f981fe00700 1 --2- 172.21.15.18:0/3240366219 >> [v2:172.21.15.18:6800/1312802974,v1:172.21.15.18:6801/1312802974] conn(0x7f980c036790 0x7f980c038c40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-12T20:43:35.960 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:29.772+0000 7f981fe00700 1 -- 172.21.15.18:0/3240366219 >> [v2:172.21.15.18:6800/1312802974,v1:172.21.15.18:6801/1312802974] conn(0x7f980c036790 msgr2=0x7f980c038c40 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.18:6800/1312802974 2024-04-12T20:43:35.960 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:29.772+0000 7f981fe00700 1 --2- 172.21.15.18:0/3240366219 >> [v2:172.21.15.18:6800/1312802974,v1:172.21.15.18:6801/1312802974] conn(0x7f980c036790 0x7f980c038c40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-12T20:43:35.960 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:31.374+0000 7f981fe00700 1 -- 172.21.15.18:0/3240366219 >> [v2:172.21.15.18:6800/1312802974,v1:172.21.15.18:6801/1312802974] conn(0x7f980c036790 msgr2=0x7f980c038c40 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.18:6800/1312802974 2024-04-12T20:43:35.960 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:31.374+0000 7f981fe00700 1 --2- 172.21.15.18:0/3240366219 >> [v2:172.21.15.18:6800/1312802974,v1:172.21.15.18:6801/1312802974] conn(0x7f980c036790 0x7f980c038c40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-04-12T20:43:35.960 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:34.274+0000 7f9825600700 1 -- 172.21.15.18:0/3240366219 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mgrmap(e 6) v1 ==== 43135+0+0 (secure 0 0 0) 0x7f981801a560 con 0x7f98280711f0 2024-04-12T20:43:35.960 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:34.274+0000 7f9825600700 1 -- 172.21.15.18:0/3240366219 >> [v2:172.21.15.18:6800/1312802974,v1:172.21.15.18:6801/1312802974] conn(0x7f980c036790 msgr2=0x7f980c038c40 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-12T20:43:35.960 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:34.274+0000 7f9825600700 1 --2- 172.21.15.18:0/3240366219 >> [v2:172.21.15.18:6800/1312802974,v1:172.21.15.18:6801/1312802974] conn(0x7f980c036790 0x7f980c038c40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:43:35.960 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:35.279+0000 7f9825600700 1 -- 172.21.15.18:0/3240366219 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mgrmap(e 7) v1 ==== 43262+0+0 (secure 0 0 0) 0x7f9818019670 con 0x7f98280711f0 2024-04-12T20:43:35.961 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:35.279+0000 7f9825600700 1 --2- 172.21.15.18:0/3240366219 >> [v2:172.21.15.18:6800/3403971131,v1:172.21.15.18:6801/3403971131] conn(0x7f980c039f50 0x7f980c03c330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T20:43:35.961 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:35.279+0000 7f9825600700 1 -- 172.21.15.18:0/3240366219 --> [v2:172.21.15.18:6800/3403971131,v1:172.21.15.18:6801/3403971131] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7f982805f5f0 con 0x7f980c039f50 2024-04-12T20:43:35.961 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:35.282+0000 7f981fe00700 1 --2- 172.21.15.18:0/3240366219 >> [v2:172.21.15.18:6800/3403971131,v1:172.21.15.18:6801/3403971131] conn(0x7f980c039f50 0x7f980c03c330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T20:43:35.961 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:35.285+0000 7f981fe00700 1 --2- 172.21.15.18:0/3240366219 >> [v2:172.21.15.18:6800/3403971131,v1:172.21.15.18:6801/3403971131] conn(0x7f980c039f50 0x7f980c03c330 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f98100039d0 tx=0x7f9810009350 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T20:43:35.961 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:35.286+0000 7f9825600700 1 -- 172.21.15.18:0/3240366219 <== mgr.14120 v2:172.21.15.18:6800/3403971131 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+6549 (secure 0 0 0) 0x7f982805f5f0 con 0x7f980c039f50 2024-04-12T20:43:35.961 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:35.293+0000 7f982f600700 1 -- 172.21.15.18:0/3240366219 --> [v2:172.21.15.18:6800/3403971131,v1:172.21.15.18:6801/3403971131] -- command(tid 1: {"prefix": "mgr_status"}) v1 -- 0x7f982804bed0 con 0x7f980c039f50 2024-04-12T20:43:35.961 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:35.294+0000 7f9825600700 1 -- 172.21.15.18:0/3240366219 <== mgr.14120 v2:172.21.15.18:6800/3403971131 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+51 (secure 0 0 0) 0x7f982804bed0 con 0x7f980c039f50 2024-04-12T20:43:35.961 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:35.294+0000 7f982f600700 1 -- 172.21.15.18:0/3240366219 >> [v2:172.21.15.18:6800/3403971131,v1:172.21.15.18:6801/3403971131] conn(0x7f980c039f50 msgr2=0x7f980c03c330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:43:35.961 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:35.294+0000 7f982f600700 1 --2- 172.21.15.18:0/3240366219 >> [v2:172.21.15.18:6800/3403971131,v1:172.21.15.18:6801/3403971131] conn(0x7f980c039f50 0x7f980c03c330 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f98100039d0 tx=0x7f9810009350 comp rx=0 tx=0).stop 2024-04-12T20:43:35.961 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:35.295+0000 7f982f600700 1 -- 172.21.15.18:0/3240366219 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f98280711f0 msgr2=0x7f9828113920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:43:35.961 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:35.295+0000 7f982f600700 1 --2- 172.21.15.18:0/3240366219 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f98280711f0 0x7f9828113920 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f9818011bd0 tx=0x7f9818009250 comp rx=0 tx=0).stop 2024-04-12T20:43:35.961 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:35.295+0000 7f982f600700 1 -- 172.21.15.18:0/3240366219 shutdown_connections 2024-04-12T20:43:35.961 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:35.295+0000 7f982f600700 1 --2- 172.21.15.18:0/3240366219 >> [v2:172.21.15.18:6800/3403971131,v1:172.21.15.18:6801/3403971131] conn(0x7f980c039f50 0x7f980c03c330 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:43:35.961 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:35.295+0000 7f982f600700 1 --2- 172.21.15.18:0/3240366219 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f98280711f0 0x7f9828113920 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:43:35.962 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:35.295+0000 7f982f600700 1 --2- 172.21.15.18:0/3240366219 >> [v2:172.21.15.18:6800/1312802974,v1:172.21.15.18:6801/1312802974] conn(0x7f980c036790 0x7f980c038c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:43:35.962 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:35.295+0000 7f982f600700 1 -- 172.21.15.18:0/3240366219 >> 172.21.15.18:0/3240366219 conn(0x7f98280fcfb0 msgr2=0x7f98280fdbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T20:43:35.962 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:35.295+0000 7f982f600700 1 -- 172.21.15.18:0/3240366219 shutdown_connections 2024-04-12T20:43:35.962 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:35.295+0000 7f982f600700 1 -- 172.21.15.18:0/3240366219 wait complete. 2024-04-12T20:43:35.962 INFO:teuthology.orchestra.run.smithi018.stdout:mgr epoch 5 is available 2024-04-12T20:43:35.962 INFO:teuthology.orchestra.run.smithi018.stdout:Setting orchestrator backend to cephadm... 2024-04-12T20:43:36.526 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:43:36 smithi018 ceph-mon[28799]: from='client.14124 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-04-12T20:43:36.527 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:43:36 smithi018 ceph-mon[28799]: from='client.14124 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-04-12T20:43:37.353 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:43:37 smithi018 ceph-mon[28799]: mgrmap e8: smithi018.xjzwga(active, since 2s) 2024-04-12T20:43:37.353 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:43:37 smithi018 ceph-mon[28799]: [12/Apr/2024:20:43:36] ENGINE Bus STARTING 2024-04-12T20:43:37.353 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:43:37 smithi018 ceph-mon[28799]: from='mgr.14120 172.21.15.18:0/4270481350' entity='mgr.smithi018.xjzwga' 2024-04-12T20:43:37.353 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:43:37 smithi018 ceph-mon[28799]: from='mgr.14120 172.21.15.18:0/4270481350' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-12T20:43:37.354 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:43:37 smithi018 ceph-mon[28799]: from='mgr.14120 172.21.15.18:0/4270481350' entity='mgr.smithi018.xjzwga' 2024-04-12T20:43:37.354 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:43:37 smithi018 ceph-mon[28799]: from='mgr.14120 172.21.15.18:0/4270481350' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-12T20:43:37.469 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:36.526+0000 7fa0e1c00700 1 Processor -- start 2024-04-12T20:43:37.469 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:36.527+0000 7fa0e1c00700 1 -- start start 2024-04-12T20:43:37.469 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:36.527+0000 7fa0e1c00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa0dc077b00 0x7fa0dc076100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T20:43:37.469 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:36.527+0000 7fa0e1c00700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0dc077f10 con 0x7fa0dc077b00 2024-04-12T20:43:37.469 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:36.528+0000 7fa0daa00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa0dc077b00 0x7fa0dc076100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T20:43:37.469 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:36.528+0000 7fa0daa00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa0dc077b00 0x7fa0dc076100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:56270/0 (socket says 172.21.15.18:56270) 2024-04-12T20:43:37.470 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:36.528+0000 7fa0daa00700 1 -- 172.21.15.18:0/2633920481 learned_addr learned my addr 172.21.15.18:0/2633920481 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T20:43:37.470 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:36.528+0000 7fa0daa00700 1 -- 172.21.15.18:0/2633920481 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0dc076640 con 0x7fa0dc077b00 2024-04-12T20:43:37.470 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:36.528+0000 7fa0daa00700 1 --2- 172.21.15.18:0/2633920481 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa0dc077b00 0x7fa0dc076100 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fa0cc0098e0 tx=0x7fa0cc009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce0d7b2275eba352 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T20:43:37.470 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:36.529+0000 7fa0da000700 1 -- 172.21.15.18:0/2633920481 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fa0cc00b860 con 0x7fa0dc077b00 2024-04-12T20:43:37.470 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:36.529+0000 7fa0da000700 1 -- 172.21.15.18:0/2633920481 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7fa0cc00be50 con 0x7fa0dc077b00 2024-04-12T20:43:37.470 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:36.529+0000 7fa0da000700 1 -- 172.21.15.18:0/2633920481 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fa0cc010380 con 0x7fa0dc077b00 2024-04-12T20:43:37.470 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:36.529+0000 7fa0e1c00700 1 -- 172.21.15.18:0/2633920481 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa0dc077b00 msgr2=0x7fa0dc076100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:43:37.470 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:36.529+0000 7fa0e1c00700 1 --2- 172.21.15.18:0/2633920481 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa0dc077b00 0x7fa0dc076100 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fa0cc0098e0 tx=0x7fa0cc009250 comp rx=0 tx=0).stop 2024-04-12T20:43:37.470 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:36.530+0000 7fa0e1c00700 1 -- 172.21.15.18:0/2633920481 shutdown_connections 2024-04-12T20:43:37.470 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:36.530+0000 7fa0e1c00700 1 --2- 172.21.15.18:0/2633920481 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa0dc077b00 0x7fa0dc076100 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:43:37.470 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:36.530+0000 7fa0e1c00700 1 -- 172.21.15.18:0/2633920481 >> 172.21.15.18:0/2633920481 conn(0x7fa0dc0fe090 msgr2=0x7fa0dc1004c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T20:43:37.470 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:36.530+0000 7fa0e1c00700 1 -- 172.21.15.18:0/2633920481 shutdown_connections 2024-04-12T20:43:37.470 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:36.530+0000 7fa0e1c00700 1 -- 172.21.15.18:0/2633920481 wait complete. 2024-04-12T20:43:37.470 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:36.530+0000 7fa0e1c00700 1 Processor -- start 2024-04-12T20:43:37.471 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:36.531+0000 7fa0e1c00700 1 -- start start 2024-04-12T20:43:37.471 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:36.531+0000 7fa0e1c00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa0dc077b00 0x7fa0dc1139b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T20:43:37.471 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:36.531+0000 7fa0e1c00700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0dc113ef0 con 0x7fa0dc077b00 2024-04-12T20:43:37.471 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:36.531+0000 7fa0daa00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa0dc077b00 0x7fa0dc1139b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T20:43:37.471 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:36.531+0000 7fa0daa00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa0dc077b00 0x7fa0dc1139b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:56282/0 (socket says 172.21.15.18:56282) 2024-04-12T20:43:37.471 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:36.531+0000 7fa0daa00700 1 -- 172.21.15.18:0/2943004686 learned_addr learned my addr 172.21.15.18:0/2943004686 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T20:43:37.471 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:36.532+0000 7fa0daa00700 1 -- 172.21.15.18:0/2943004686 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0cc0094f0 con 0x7fa0dc077b00 2024-04-12T20:43:37.471 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:36.532+0000 7fa0daa00700 1 --2- 172.21.15.18:0/2943004686 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa0dc077b00 0x7fa0dc1139b0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fa0cc006fd0 tx=0x7fa0cc009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T20:43:37.471 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:36.532+0000 7fa0d3400700 1 -- 172.21.15.18:0/2943004686 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fa0cc010760 con 0x7fa0dc077b00 2024-04-12T20:43:37.471 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:36.532+0000 7fa0d3400700 1 -- 172.21.15.18:0/2943004686 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7fa0cc010d50 con 0x7fa0dc077b00 2024-04-12T20:43:37.471 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:36.532+0000 7fa0d3400700 1 -- 172.21.15.18:0/2943004686 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fa0cc01adc0 con 0x7fa0dc077b00 2024-04-12T20:43:37.471 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:36.532+0000 7fa0e1c00700 1 -- 172.21.15.18:0/2943004686 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa0dc1140f0 con 0x7fa0dc077b00 2024-04-12T20:43:37.471 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:36.532+0000 7fa0e1c00700 1 -- 172.21.15.18:0/2943004686 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa0dc114c60 con 0x7fa0dc077b00 2024-04-12T20:43:37.471 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:36.533+0000 7fa0e1c00700 1 -- 172.21.15.18:0/2943004686 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa0dc05f5f0 con 0x7fa0dc077b00 2024-04-12T20:43:37.471 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:36.534+0000 7fa0d3400700 1 -- 172.21.15.18:0/2943004686 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 8) v1 ==== 43368+0+0 (secure 0 0 0) 0x7fa0cc0108c0 con 0x7fa0dc077b00 2024-04-12T20:43:37.471 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:36.534+0000 7fa0d3400700 1 --2- 172.21.15.18:0/2943004686 >> [v2:172.21.15.18:6800/3403971131,v1:172.21.15.18:6801/3403971131] conn(0x7fa0c00367e0 0x7fa0c0038c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T20:43:37.472 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:36.534+0000 7fa0d3400700 1 -- 172.21.15.18:0/2943004686 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 936+0+0 (secure 0 0 0) 0x7fa0cc049f70 con 0x7fa0dc077b00 2024-04-12T20:43:37.472 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:36.535+0000 7fa0d2000700 1 --2- 172.21.15.18:0/2943004686 >> [v2:172.21.15.18:6800/3403971131,v1:172.21.15.18:6801/3403971131] conn(0x7fa0c00367e0 0x7fa0c0038c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T20:43:37.472 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:36.535+0000 7fa0d2000700 1 --2- 172.21.15.18:0/2943004686 >> [v2:172.21.15.18:6800/3403971131,v1:172.21.15.18:6801/3403971131] conn(0x7fa0c00367e0 0x7fa0c0038c90 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fa0c4009990 tx=0x7fa0c4006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T20:43:37.472 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:36.540+0000 7fa0d3400700 1 -- 172.21.15.18:0/2943004686 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7fa0cc022770 con 0x7fa0dc077b00 2024-04-12T20:43:37.472 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:36.770+0000 7fa0e1c00700 1 -- 172.21.15.18:0/2943004686 --> [v2:172.21.15.18:6800/3403971131,v1:172.21.15.18:6801/3403971131] -- mgr_command(tid 0: {"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}) v1 -- 0x7fa0dc0ff4d0 con 0x7fa0c00367e0 2024-04-12T20:43:37.472 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:36.783+0000 7fa0d3400700 1 -- 172.21.15.18:0/2943004686 <== mgr.14120 v2:172.21.15.18:6800/3403971131 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7fa0dc0ff4d0 con 0x7fa0c00367e0 2024-04-12T20:43:37.472 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:36.787+0000 7fa0e1c00700 1 -- 172.21.15.18:0/2943004686 >> [v2:172.21.15.18:6800/3403971131,v1:172.21.15.18:6801/3403971131] conn(0x7fa0c00367e0 msgr2=0x7fa0c0038c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:43:37.472 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:36.787+0000 7fa0e1c00700 1 --2- 172.21.15.18:0/2943004686 >> [v2:172.21.15.18:6800/3403971131,v1:172.21.15.18:6801/3403971131] conn(0x7fa0c00367e0 0x7fa0c0038c90 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fa0c4009990 tx=0x7fa0c4006e40 comp rx=0 tx=0).stop 2024-04-12T20:43:37.472 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:36.787+0000 7fa0e1c00700 1 -- 172.21.15.18:0/2943004686 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa0dc077b00 msgr2=0x7fa0dc1139b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:43:37.472 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:36.787+0000 7fa0e1c00700 1 --2- 172.21.15.18:0/2943004686 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa0dc077b00 0x7fa0dc1139b0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fa0cc006fd0 tx=0x7fa0cc009250 comp rx=0 tx=0).stop 2024-04-12T20:43:37.472 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:36.787+0000 7fa0e1c00700 1 -- 172.21.15.18:0/2943004686 shutdown_connections 2024-04-12T20:43:37.472 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:36.787+0000 7fa0e1c00700 1 --2- 172.21.15.18:0/2943004686 >> [v2:172.21.15.18:6800/3403971131,v1:172.21.15.18:6801/3403971131] conn(0x7fa0c00367e0 0x7fa0c0038c90 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:43:37.472 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:36.787+0000 7fa0e1c00700 1 --2- 172.21.15.18:0/2943004686 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa0dc077b00 0x7fa0dc1139b0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:43:37.472 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:36.787+0000 7fa0e1c00700 1 -- 172.21.15.18:0/2943004686 >> 172.21.15.18:0/2943004686 conn(0x7fa0dc0fe090 msgr2=0x7fa0dc0fed00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T20:43:37.472 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:36.788+0000 7fa0e1c00700 1 -- 172.21.15.18:0/2943004686 shutdown_connections 2024-04-12T20:43:37.472 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:36.788+0000 7fa0e1c00700 1 -- 172.21.15.18:0/2943004686 wait complete. 2024-04-12T20:43:38.533 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:43:38 smithi018 ceph-mon[28799]: [12/Apr/2024:20:43:36] ENGINE Serving on https://172.21.15.18:7150 2024-04-12T20:43:38.533 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:43:38 smithi018 ceph-mon[28799]: [12/Apr/2024:20:43:36] ENGINE Bus STARTED 2024-04-12T20:43:38.533 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:43:38 smithi018 ceph-mon[28799]: from='client.14132 -' entity='client.admin' cmd=[{"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T20:43:39.073 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout value unchanged 2024-04-12T20:43:39.075 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:38.103+0000 7f46d3200700 1 Processor -- start 2024-04-12T20:43:39.075 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:38.103+0000 7f46d3200700 1 -- start start 2024-04-12T20:43:39.075 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:38.103+0000 7f46d3200700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f46cc102890 0x7f46cc102ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T20:43:39.075 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:38.103+0000 7f46d3200700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46cc1031e0 con 0x7f46cc102890 2024-04-12T20:43:39.075 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:38.104+0000 7f46cbe00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f46cc102890 0x7f46cc102ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T20:43:39.075 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:38.104+0000 7f46cbe00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f46cc102890 0x7f46cc102ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:56298/0 (socket says 172.21.15.18:56298) 2024-04-12T20:43:39.075 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:38.104+0000 7f46cbe00700 1 -- 172.21.15.18:0/2145728115 learned_addr learned my addr 172.21.15.18:0/2145728115 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T20:43:39.080 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:38.104+0000 7f46cbe00700 1 -- 172.21.15.18:0/2145728115 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46cc103320 con 0x7f46cc102890 2024-04-12T20:43:39.080 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:38.105+0000 7f46cbe00700 1 --2- 172.21.15.18:0/2145728115 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f46cc102890 0x7f46cc102ca0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f46bc0098e0 tx=0x7f46bc009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=617a00201b56157b server_cookie=0 in_seq=0 out_seq=0 2024-04-12T20:43:39.080 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:38.105+0000 7f46caa00700 1 -- 172.21.15.18:0/2145728115 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f46bc00b860 con 0x7f46cc102890 2024-04-12T20:43:39.080 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:38.105+0000 7f46caa00700 1 -- 172.21.15.18:0/2145728115 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7f46bc00be50 con 0x7f46cc102890 2024-04-12T20:43:39.080 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:38.105+0000 7f46caa00700 1 -- 172.21.15.18:0/2145728115 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f46bc010380 con 0x7f46cc102890 2024-04-12T20:43:39.080 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:38.106+0000 7f46d3200700 1 -- 172.21.15.18:0/2145728115 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f46cc102890 msgr2=0x7f46cc102ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:43:39.080 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:38.106+0000 7f46d3200700 1 --2- 172.21.15.18:0/2145728115 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f46cc102890 0x7f46cc102ca0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f46bc0098e0 tx=0x7f46bc009250 comp rx=0 tx=0).stop 2024-04-12T20:43:39.080 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:38.106+0000 7f46d3200700 1 -- 172.21.15.18:0/2145728115 shutdown_connections 2024-04-12T20:43:39.080 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:38.106+0000 7f46d3200700 1 --2- 172.21.15.18:0/2145728115 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f46cc102890 0x7f46cc102ca0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:43:39.080 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:38.106+0000 7f46d3200700 1 -- 172.21.15.18:0/2145728115 >> 172.21.15.18:0/2145728115 conn(0x7f46cc0fe0a0 msgr2=0x7f46cc1004b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T20:43:39.081 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:38.106+0000 7f46d3200700 1 -- 172.21.15.18:0/2145728115 shutdown_connections 2024-04-12T20:43:39.081 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:38.106+0000 7f46d3200700 1 -- 172.21.15.18:0/2145728115 wait complete. 2024-04-12T20:43:39.081 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:38.107+0000 7f46d3200700 1 Processor -- start 2024-04-12T20:43:39.081 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:38.107+0000 7f46d3200700 1 -- start start 2024-04-12T20:43:39.081 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:38.108+0000 7f46d3200700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f46cc102890 0x7f46cc113920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T20:43:39.081 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:38.108+0000 7f46d3200700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46cc113e60 con 0x7f46cc102890 2024-04-12T20:43:39.081 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:38.108+0000 7f46cbe00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f46cc102890 0x7f46cc113920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T20:43:39.081 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:38.108+0000 7f46cbe00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f46cc102890 0x7f46cc113920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:56310/0 (socket says 172.21.15.18:56310) 2024-04-12T20:43:39.081 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:38.108+0000 7f46cbe00700 1 -- 172.21.15.18:0/3733398103 learned_addr learned my addr 172.21.15.18:0/3733398103 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T20:43:39.081 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:38.108+0000 7f46cbe00700 1 -- 172.21.15.18:0/3733398103 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46bc0094f0 con 0x7f46cc102890 2024-04-12T20:43:39.081 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:38.109+0000 7f46cbe00700 1 --2- 172.21.15.18:0/3733398103 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f46cc102890 0x7f46cc113920 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f46bc000c00 tx=0x7f46bc009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T20:43:39.081 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:38.109+0000 7f46c3e00700 1 -- 172.21.15.18:0/3733398103 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f46bc0107c0 con 0x7f46cc102890 2024-04-12T20:43:39.081 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:38.109+0000 7f46c3e00700 1 -- 172.21.15.18:0/3733398103 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7f46bc010db0 con 0x7f46cc102890 2024-04-12T20:43:39.082 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:38.109+0000 7f46c3e00700 1 -- 172.21.15.18:0/3733398103 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f46bc01ad00 con 0x7f46cc102890 2024-04-12T20:43:39.082 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:38.109+0000 7f46d3200700 1 -- 172.21.15.18:0/3733398103 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f46cc114060 con 0x7f46cc102890 2024-04-12T20:43:39.082 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:38.109+0000 7f46d3200700 1 -- 172.21.15.18:0/3733398103 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f46cc114bd0 con 0x7f46cc102890 2024-04-12T20:43:39.082 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:38.110+0000 7f46d3200700 1 -- 172.21.15.18:0/3733398103 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f46cc05f5f0 con 0x7f46cc102890 2024-04-12T20:43:39.082 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:38.110+0000 7f46c3e00700 1 -- 172.21.15.18:0/3733398103 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 8) v1 ==== 43368+0+0 (secure 0 0 0) 0x7f46bc01a530 con 0x7f46cc102890 2024-04-12T20:43:39.082 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:38.111+0000 7f46c3e00700 1 --2- 172.21.15.18:0/3733398103 >> [v2:172.21.15.18:6800/3403971131,v1:172.21.15.18:6801/3403971131] conn(0x7f46b00367e0 0x7f46b0038c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T20:43:39.082 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:38.111+0000 7f46c3e00700 1 -- 172.21.15.18:0/3733398103 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 936+0+0 (secure 0 0 0) 0x7f46bc04a650 con 0x7f46cc102890 2024-04-12T20:43:39.082 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:38.111+0000 7f46cb400700 1 --2- 172.21.15.18:0/3733398103 >> [v2:172.21.15.18:6800/3403971131,v1:172.21.15.18:6801/3403971131] conn(0x7f46b00367e0 0x7f46b0038c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T20:43:39.082 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:38.112+0000 7f46cb400700 1 --2- 172.21.15.18:0/3733398103 >> [v2:172.21.15.18:6800/3403971131,v1:172.21.15.18:6801/3403971131] conn(0x7f46b00367e0 0x7f46b0038c90 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f46b8009990 tx=0x7f46b8006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T20:43:39.082 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:38.121+0000 7f46c3e00700 1 -- 172.21.15.18:0/3733398103 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7f46bc010920 con 0x7f46cc102890 2024-04-12T20:43:39.082 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:38.341+0000 7f46d3200700 1 -- 172.21.15.18:0/3733398103 --> [v2:172.21.15.18:6800/3403971131,v1:172.21.15.18:6801/3403971131] -- mgr_command(tid 0: {"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}) v1 -- 0x7f46cc0ff280 con 0x7f46b00367e0 2024-04-12T20:43:39.082 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:38.342+0000 7f46c3e00700 1 -- 172.21.15.18:0/3733398103 <== mgr.14120 v2:172.21.15.18:6800/3403971131 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+16 (secure 0 0 0) 0x7f46cc0ff280 con 0x7f46b00367e0 2024-04-12T20:43:39.082 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:38.344+0000 7f46d3200700 1 -- 172.21.15.18:0/3733398103 >> [v2:172.21.15.18:6800/3403971131,v1:172.21.15.18:6801/3403971131] conn(0x7f46b00367e0 msgr2=0x7f46b0038c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:43:39.084 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:38.344+0000 7f46d3200700 1 --2- 172.21.15.18:0/3733398103 >> [v2:172.21.15.18:6800/3403971131,v1:172.21.15.18:6801/3403971131] conn(0x7f46b00367e0 0x7f46b0038c90 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f46b8009990 tx=0x7f46b8006e40 comp rx=0 tx=0).stop 2024-04-12T20:43:39.084 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:38.344+0000 7f46d3200700 1 -- 172.21.15.18:0/3733398103 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f46cc102890 msgr2=0x7f46cc113920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:43:39.084 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:38.344+0000 7f46d3200700 1 --2- 172.21.15.18:0/3733398103 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f46cc102890 0x7f46cc113920 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f46bc000c00 tx=0x7f46bc009250 comp rx=0 tx=0).stop 2024-04-12T20:43:39.084 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:38.344+0000 7f46d3200700 1 -- 172.21.15.18:0/3733398103 shutdown_connections 2024-04-12T20:43:39.084 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:38.344+0000 7f46d3200700 1 --2- 172.21.15.18:0/3733398103 >> [v2:172.21.15.18:6800/3403971131,v1:172.21.15.18:6801/3403971131] conn(0x7f46b00367e0 0x7f46b0038c90 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:43:39.084 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:38.344+0000 7f46d3200700 1 --2- 172.21.15.18:0/3733398103 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f46cc102890 0x7f46cc113920 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:43:39.084 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:38.344+0000 7f46d3200700 1 -- 172.21.15.18:0/3733398103 >> 172.21.15.18:0/3733398103 conn(0x7f46cc0fe0a0 msgr2=0x7f46cc0feb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T20:43:39.084 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:38.345+0000 7f46d3200700 1 -- 172.21.15.18:0/3733398103 shutdown_connections 2024-04-12T20:43:39.085 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:38.345+0000 7f46d3200700 1 -- 172.21.15.18:0/3733398103 wait complete. 2024-04-12T20:43:39.085 INFO:teuthology.orchestra.run.smithi018.stdout:Generating ssh key... 2024-04-12T20:43:39.598 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:43:39 smithi018 ceph-mon[28799]: from='client.14134 -' entity='client.admin' cmd=[{"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T20:43:40.969 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:39.674+0000 7f4e1b000700 1 Processor -- start 2024-04-12T20:43:40.970 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:39.674+0000 7f4e1b000700 1 -- start start 2024-04-12T20:43:40.970 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:39.675+0000 7f4e1b000700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4e14101830 0x7f4e14101c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T20:43:40.970 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:39.675+0000 7f4e1b000700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e14102180 con 0x7f4e14101830 2024-04-12T20:43:40.971 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:39.675+0000 7f4e13e00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4e14101830 0x7f4e14101c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T20:43:40.971 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:39.676+0000 7f4e13e00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4e14101830 0x7f4e14101c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:56320/0 (socket says 172.21.15.18:56320) 2024-04-12T20:43:40.971 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:39.676+0000 7f4e13e00700 1 -- 172.21.15.18:0/872255397 learned_addr learned my addr 172.21.15.18:0/872255397 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T20:43:40.971 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:39.676+0000 7f4e13e00700 1 -- 172.21.15.18:0/872255397 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e141022c0 con 0x7f4e14101830 2024-04-12T20:43:40.972 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:39.676+0000 7f4e13e00700 1 --2- 172.21.15.18:0/872255397 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4e14101830 0x7f4e14101c40 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f4e040098e0 tx=0x7f4e04009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=71bb89a357a630ab server_cookie=0 in_seq=0 out_seq=0 2024-04-12T20:43:40.972 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:39.677+0000 7f4e13400700 1 -- 172.21.15.18:0/872255397 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f4e0400b860 con 0x7f4e14101830 2024-04-12T20:43:40.972 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:39.677+0000 7f4e13400700 1 -- 172.21.15.18:0/872255397 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7f4e0400be50 con 0x7f4e14101830 2024-04-12T20:43:40.972 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:39.677+0000 7f4e13400700 1 -- 172.21.15.18:0/872255397 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f4e04010380 con 0x7f4e14101830 2024-04-12T20:43:40.972 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:39.677+0000 7f4e1b000700 1 -- 172.21.15.18:0/872255397 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4e14101830 msgr2=0x7f4e14101c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:43:40.973 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:39.677+0000 7f4e1b000700 1 --2- 172.21.15.18:0/872255397 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4e14101830 0x7f4e14101c40 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f4e040098e0 tx=0x7f4e04009250 comp rx=0 tx=0).stop 2024-04-12T20:43:40.973 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:39.677+0000 7f4e1b000700 1 -- 172.21.15.18:0/872255397 shutdown_connections 2024-04-12T20:43:40.973 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:39.677+0000 7f4e1b000700 1 --2- 172.21.15.18:0/872255397 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4e14101830 0x7f4e14101c40 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:43:40.974 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:39.677+0000 7f4e1b000700 1 -- 172.21.15.18:0/872255397 >> 172.21.15.18:0/872255397 conn(0x7f4e140fcfb0 msgr2=0x7f4e140ff3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T20:43:40.974 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:39.678+0000 7f4e1b000700 1 -- 172.21.15.18:0/872255397 shutdown_connections 2024-04-12T20:43:40.974 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:39.678+0000 7f4e1b000700 1 -- 172.21.15.18:0/872255397 wait complete. 2024-04-12T20:43:40.975 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:39.678+0000 7f4e1b000700 1 Processor -- start 2024-04-12T20:43:40.975 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:39.679+0000 7f4e1b000700 1 -- start start 2024-04-12T20:43:40.975 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:39.679+0000 7f4e1b000700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4e14101830 0x7f4e14113940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T20:43:40.975 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:39.679+0000 7f4e1b000700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e14113e80 con 0x7f4e14101830 2024-04-12T20:43:40.975 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:39.679+0000 7f4e13e00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4e14101830 0x7f4e14113940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T20:43:40.976 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:39.679+0000 7f4e13e00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4e14101830 0x7f4e14113940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:56330/0 (socket says 172.21.15.18:56330) 2024-04-12T20:43:40.976 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:39.679+0000 7f4e13e00700 1 -- 172.21.15.18:0/2789130303 learned_addr learned my addr 172.21.15.18:0/2789130303 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T20:43:40.976 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:39.680+0000 7f4e13e00700 1 -- 172.21.15.18:0/2789130303 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e040094f0 con 0x7f4e14101830 2024-04-12T20:43:40.976 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:39.680+0000 7f4e13e00700 1 --2- 172.21.15.18:0/2789130303 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4e14101830 0x7f4e14113940 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f4e0400ba90 tx=0x7f4e04009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T20:43:40.976 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:39.680+0000 7f4e10c00700 1 -- 172.21.15.18:0/2789130303 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f4e040107c0 con 0x7f4e14101830 2024-04-12T20:43:40.977 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:39.680+0000 7f4e10c00700 1 -- 172.21.15.18:0/2789130303 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7f4e04010db0 con 0x7f4e14101830 2024-04-12T20:43:40.977 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:39.680+0000 7f4e1b000700 1 -- 172.21.15.18:0/2789130303 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4e14114080 con 0x7f4e14101830 2024-04-12T20:43:40.977 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:39.680+0000 7f4e10c00700 1 -- 172.21.15.18:0/2789130303 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f4e0401add0 con 0x7f4e14101830 2024-04-12T20:43:40.979 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:39.680+0000 7f4e1b000700 1 -- 172.21.15.18:0/2789130303 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4e14114bf0 con 0x7f4e14101830 2024-04-12T20:43:40.979 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:39.681+0000 7f4e10c00700 1 -- 172.21.15.18:0/2789130303 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 8) v1 ==== 43368+0+0 (secure 0 0 0) 0x7f4e04010920 con 0x7f4e14101830 2024-04-12T20:43:40.980 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:39.681+0000 7f4e1b000700 1 -- 172.21.15.18:0/2789130303 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4e1405f5f0 con 0x7f4e14101830 2024-04-12T20:43:40.980 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:39.682+0000 7f4e10c00700 1 --2- 172.21.15.18:0/2789130303 >> [v2:172.21.15.18:6800/3403971131,v1:172.21.15.18:6801/3403971131] conn(0x7f4df80367e0 0x7f4df8038c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T20:43:40.980 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:39.682+0000 7f4e10c00700 1 -- 172.21.15.18:0/2789130303 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 936+0+0 (secure 0 0 0) 0x7f4e0404a2b0 con 0x7f4e14101830 2024-04-12T20:43:40.980 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:39.682+0000 7f4e0be00700 1 --2- 172.21.15.18:0/2789130303 >> [v2:172.21.15.18:6800/3403971131,v1:172.21.15.18:6801/3403971131] conn(0x7f4df80367e0 0x7f4df8038c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T20:43:40.980 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:39.683+0000 7f4e0be00700 1 --2- 172.21.15.18:0/2789130303 >> [v2:172.21.15.18:6800/3403971131,v1:172.21.15.18:6801/3403971131] conn(0x7f4df80367e0 0x7f4df8038c90 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f4dfc009990 tx=0x7f4dfc006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T20:43:40.980 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:39.687+0000 7f4e10c00700 1 -- 172.21.15.18:0/2789130303 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7f4e0401a430 con 0x7f4e14101830 2024-04-12T20:43:40.980 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:39.901+0000 7f4e1b000700 1 -- 172.21.15.18:0/2789130303 --> [v2:172.21.15.18:6800/3403971131,v1:172.21.15.18:6801/3403971131] -- mgr_command(tid 0: {"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}) v1 -- 0x7f4e141157d0 con 0x7f4df80367e0 2024-04-12T20:43:40.980 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:40.281+0000 7f4e10c00700 1 -- 172.21.15.18:0/2789130303 <== mgr.14120 v2:172.21.15.18:6800/3403971131 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f4e141157d0 con 0x7f4df80367e0 2024-04-12T20:43:40.980 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:40.284+0000 7f4e1b000700 1 -- 172.21.15.18:0/2789130303 >> [v2:172.21.15.18:6800/3403971131,v1:172.21.15.18:6801/3403971131] conn(0x7f4df80367e0 msgr2=0x7f4df8038c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:43:40.980 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:40.284+0000 7f4e1b000700 1 --2- 172.21.15.18:0/2789130303 >> [v2:172.21.15.18:6800/3403971131,v1:172.21.15.18:6801/3403971131] conn(0x7f4df80367e0 0x7f4df8038c90 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f4dfc009990 tx=0x7f4dfc006e40 comp rx=0 tx=0).stop 2024-04-12T20:43:40.980 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:40.284+0000 7f4e1b000700 1 -- 172.21.15.18:0/2789130303 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4e14101830 msgr2=0x7f4e14113940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:43:40.980 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:40.284+0000 7f4e1b000700 1 --2- 172.21.15.18:0/2789130303 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4e14101830 0x7f4e14113940 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f4e0400ba90 tx=0x7f4e04009250 comp rx=0 tx=0).stop 2024-04-12T20:43:40.980 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:40.284+0000 7f4e1b000700 1 -- 172.21.15.18:0/2789130303 shutdown_connections 2024-04-12T20:43:40.981 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:40.284+0000 7f4e1b000700 1 --2- 172.21.15.18:0/2789130303 >> [v2:172.21.15.18:6800/3403971131,v1:172.21.15.18:6801/3403971131] conn(0x7f4df80367e0 0x7f4df8038c90 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:43:40.981 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:40.284+0000 7f4e1b000700 1 --2- 172.21.15.18:0/2789130303 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4e14101830 0x7f4e14113940 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:43:40.981 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:40.284+0000 7f4e1b000700 1 -- 172.21.15.18:0/2789130303 >> 172.21.15.18:0/2789130303 conn(0x7f4e140fcfb0 msgr2=0x7f4e140fded0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T20:43:40.981 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:40.285+0000 7f4e1b000700 1 -- 172.21.15.18:0/2789130303 shutdown_connections 2024-04-12T20:43:40.981 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:40.285+0000 7f4e1b000700 1 -- 172.21.15.18:0/2789130303 wait complete. 2024-04-12T20:43:41.521 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:43:41 smithi018 ceph-mon[28799]: from='client.14136 -' entity='client.admin' cmd=[{"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T20:43:41.521 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:43:41 smithi018 ceph-mon[28799]: Generating ssh key... 2024-04-12T20:43:41.521 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:43:41 smithi018 ceph-mon[28799]: from='mgr.14120 172.21.15.18:0/4270481350' entity='mgr.smithi018.xjzwga' 2024-04-12T20:43:41.521 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:43:41 smithi018 ceph-mon[28799]: from='mgr.14120 172.21.15.18:0/4270481350' entity='mgr.smithi018.xjzwga' 2024-04-12T20:43:42.505 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDa2TaF+4LzQRv/R3PRY5IlWt3/AQKzjtsznp8j5IwgrPKCMpavq2mOHgOpPo2CU0RVf7c9TXM1TQWklow1cM/yJpCG0lc6NcjAdw87bb2BAZQVlYVBn8NP5prb0iMjlv1FjMgN96KMcNrDXtBD8RYvx/ozDqcc8hTP7wncSt6/z5zJttr+u/ExQvMSdid/8SbpH42wtSzSQhrwmiQByagI4Rr1R0DfLOQA4g0vsJqN1NxsWdNxu7G13y8bpliBKG8s9rNE8J4d4GHJcrXkSb67judK6nQKAGbiFylGsrLTX6IiLuUcPU3ypDWtIUddVlLQeeOciiHd47tUlgx62wi8NNtn6ZPqtQ3MiG0OB8hYWrH0kqrWcUVE+8JXbftVCX39J1XJE9lyHpJNc9q88S3NIv9DZE6rv2GR/+efFLPaAaaDlq8KNNIwWik/nToPORdW1zDtr6v+YHvSlR3TrKXlPsWKImAlQhnBJuL3nuINwalFX8RXyVaG8eATf2DUqyU= ceph-1cbe407a-f90d-11ee-bc8d-c7b262605968 2024-04-12T20:43:42.519 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:41.595+0000 7f85cba00700 1 Processor -- start 2024-04-12T20:43:42.519 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:41.595+0000 7f85cba00700 1 -- start start 2024-04-12T20:43:42.519 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:41.596+0000 7f85cba00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f85c4077ab0 0x7f85c4076100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T20:43:42.519 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:41.596+0000 7f85cba00700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85c4077ec0 con 0x7f85c4077ab0 2024-04-12T20:43:42.519 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:41.596+0000 7f85c9000700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f85c4077ab0 0x7f85c4076100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T20:43:42.519 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:41.596+0000 7f85c9000700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f85c4077ab0 0x7f85c4076100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:44192/0 (socket says 172.21.15.18:44192) 2024-04-12T20:43:42.520 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:41.596+0000 7f85c9000700 1 -- 172.21.15.18:0/715477145 learned_addr learned my addr 172.21.15.18:0/715477145 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T20:43:42.520 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:41.597+0000 7f85c9000700 1 -- 172.21.15.18:0/715477145 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85c4078000 con 0x7f85c4077ab0 2024-04-12T20:43:42.520 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:41.597+0000 7f85c9000700 1 --2- 172.21.15.18:0/715477145 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f85c4077ab0 0x7f85c4076100 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f85b80098e0 tx=0x7f85b8009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d6f81ee31b90ab3 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T20:43:42.520 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:41.597+0000 7f85c3400700 1 -- 172.21.15.18:0/715477145 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f85b800b860 con 0x7f85c4077ab0 2024-04-12T20:43:42.520 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:41.597+0000 7f85c3400700 1 -- 172.21.15.18:0/715477145 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7f85b800be50 con 0x7f85c4077ab0 2024-04-12T20:43:42.520 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:41.597+0000 7f85c3400700 1 -- 172.21.15.18:0/715477145 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f85b8010380 con 0x7f85c4077ab0 2024-04-12T20:43:42.520 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:41.598+0000 7f85cba00700 1 -- 172.21.15.18:0/715477145 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f85c4077ab0 msgr2=0x7f85c4076100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:43:42.520 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:41.598+0000 7f85cba00700 1 --2- 172.21.15.18:0/715477145 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f85c4077ab0 0x7f85c4076100 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f85b80098e0 tx=0x7f85b8009250 comp rx=0 tx=0).stop 2024-04-12T20:43:42.520 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:41.598+0000 7f85cba00700 1 -- 172.21.15.18:0/715477145 shutdown_connections 2024-04-12T20:43:42.520 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:41.598+0000 7f85cba00700 1 --2- 172.21.15.18:0/715477145 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f85c4077ab0 0x7f85c4076100 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:43:42.520 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:41.598+0000 7f85cba00700 1 -- 172.21.15.18:0/715477145 >> 172.21.15.18:0/715477145 conn(0x7f85c40fe090 msgr2=0x7f85c4100500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T20:43:42.520 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:41.599+0000 7f85cba00700 1 -- 172.21.15.18:0/715477145 shutdown_connections 2024-04-12T20:43:42.521 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:41.599+0000 7f85cba00700 1 -- 172.21.15.18:0/715477145 wait complete. 2024-04-12T20:43:42.521 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:41.599+0000 7f85cba00700 1 Processor -- start 2024-04-12T20:43:42.521 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:41.600+0000 7f85cba00700 1 -- start start 2024-04-12T20:43:42.521 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:41.600+0000 7f85cba00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f85c4077ab0 0x7f85c4113990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T20:43:42.521 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:41.600+0000 7f85cba00700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85c4113ed0 con 0x7f85c4077ab0 2024-04-12T20:43:42.521 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:41.600+0000 7f85c9000700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f85c4077ab0 0x7f85c4113990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T20:43:42.521 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:41.600+0000 7f85c9000700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f85c4077ab0 0x7f85c4113990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:44206/0 (socket says 172.21.15.18:44206) 2024-04-12T20:43:42.521 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:41.600+0000 7f85c9000700 1 -- 172.21.15.18:0/4016781426 learned_addr learned my addr 172.21.15.18:0/4016781426 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T20:43:42.521 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:41.601+0000 7f85c9000700 1 -- 172.21.15.18:0/4016781426 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85b80094f0 con 0x7f85c4077ab0 2024-04-12T20:43:42.521 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:41.601+0000 7f85c9000700 1 --2- 172.21.15.18:0/4016781426 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f85c4077ab0 0x7f85c4113990 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f85b8011bd0 tx=0x7f85b8009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T20:43:42.521 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:41.601+0000 7f85c0c00700 1 -- 172.21.15.18:0/4016781426 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f85b80107c0 con 0x7f85c4077ab0 2024-04-12T20:43:42.521 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:41.601+0000 7f85c0c00700 1 -- 172.21.15.18:0/4016781426 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7f85b8010db0 con 0x7f85c4077ab0 2024-04-12T20:43:42.521 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:41.601+0000 7f85c0c00700 1 -- 172.21.15.18:0/4016781426 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f85b801ad00 con 0x7f85c4077ab0 2024-04-12T20:43:42.521 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:41.601+0000 7f85cba00700 1 -- 172.21.15.18:0/4016781426 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f85c41140d0 con 0x7f85c4077ab0 2024-04-12T20:43:42.522 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:41.601+0000 7f85cba00700 1 -- 172.21.15.18:0/4016781426 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f85c4114c40 con 0x7f85c4077ab0 2024-04-12T20:43:42.522 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:41.602+0000 7f85c0c00700 1 -- 172.21.15.18:0/4016781426 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 8) v1 ==== 43368+0+0 (secure 0 0 0) 0x7f85b801a560 con 0x7f85c4077ab0 2024-04-12T20:43:42.522 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:41.603+0000 7f85cba00700 1 -- 172.21.15.18:0/4016781426 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f85c405f5f0 con 0x7f85c4077ab0 2024-04-12T20:43:42.522 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:41.603+0000 7f85c0c00700 1 --2- 172.21.15.18:0/4016781426 >> [v2:172.21.15.18:6800/3403971131,v1:172.21.15.18:6801/3403971131] conn(0x7f85ac036790 0x7f85ac038c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T20:43:42.522 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:41.603+0000 7f85c0c00700 1 -- 172.21.15.18:0/4016781426 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 936+0+0 (secure 0 0 0) 0x7f85b804a190 con 0x7f85c4077ab0 2024-04-12T20:43:42.522 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:41.603+0000 7f85c3e00700 1 --2- 172.21.15.18:0/4016781426 >> [v2:172.21.15.18:6800/3403971131,v1:172.21.15.18:6801/3403971131] conn(0x7f85ac036790 0x7f85ac038c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T20:43:42.522 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:41.604+0000 7f85c3e00700 1 --2- 172.21.15.18:0/4016781426 >> [v2:172.21.15.18:6800/3403971131,v1:172.21.15.18:6801/3403971131] conn(0x7f85ac036790 0x7f85ac038c40 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f85b0009990 tx=0x7f85b0006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T20:43:42.522 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:41.608+0000 7f85c0c00700 1 -- 172.21.15.18:0/4016781426 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7f85b8024590 con 0x7f85c4077ab0 2024-04-12T20:43:42.522 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:41.824+0000 7f85cba00700 1 -- 172.21.15.18:0/4016781426 --> [v2:172.21.15.18:6800/3403971131,v1:172.21.15.18:6801/3403971131] -- mgr_command(tid 0: {"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}) v1 -- 0x7f85c40ff4c0 con 0x7f85ac036790 2024-04-12T20:43:42.522 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:41.826+0000 7f85c0c00700 1 -- 172.21.15.18:0/4016781426 <== mgr.14120 v2:172.21.15.18:6800/3403971131 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+595 (secure 0 0 0) 0x7f85c40ff4c0 con 0x7f85ac036790 2024-04-12T20:43:42.522 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:41.828+0000 7f85cba00700 1 -- 172.21.15.18:0/4016781426 >> [v2:172.21.15.18:6800/3403971131,v1:172.21.15.18:6801/3403971131] conn(0x7f85ac036790 msgr2=0x7f85ac038c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:43:42.522 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:41.829+0000 7f85cba00700 1 --2- 172.21.15.18:0/4016781426 >> [v2:172.21.15.18:6800/3403971131,v1:172.21.15.18:6801/3403971131] conn(0x7f85ac036790 0x7f85ac038c40 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f85b0009990 tx=0x7f85b0006e40 comp rx=0 tx=0).stop 2024-04-12T20:43:42.522 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:41.829+0000 7f85cba00700 1 -- 172.21.15.18:0/4016781426 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f85c4077ab0 msgr2=0x7f85c4113990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:43:42.523 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:41.829+0000 7f85cba00700 1 --2- 172.21.15.18:0/4016781426 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f85c4077ab0 0x7f85c4113990 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f85b8011bd0 tx=0x7f85b8009250 comp rx=0 tx=0).stop 2024-04-12T20:43:42.523 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:41.829+0000 7f85cba00700 1 -- 172.21.15.18:0/4016781426 shutdown_connections 2024-04-12T20:43:42.523 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:41.829+0000 7f85cba00700 1 --2- 172.21.15.18:0/4016781426 >> [v2:172.21.15.18:6800/3403971131,v1:172.21.15.18:6801/3403971131] conn(0x7f85ac036790 0x7f85ac038c40 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:43:42.523 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:41.829+0000 7f85cba00700 1 --2- 172.21.15.18:0/4016781426 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f85c4077ab0 0x7f85c4113990 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:43:42.523 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:41.829+0000 7f85cba00700 1 -- 172.21.15.18:0/4016781426 >> 172.21.15.18:0/4016781426 conn(0x7f85c40fe090 msgr2=0x7f85c40fecf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T20:43:42.523 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:41.829+0000 7f85cba00700 1 -- 172.21.15.18:0/4016781426 shutdown_connections 2024-04-12T20:43:42.523 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:41.829+0000 7f85cba00700 1 -- 172.21.15.18:0/4016781426 wait complete. 2024-04-12T20:43:42.524 INFO:teuthology.orchestra.run.smithi018.stdout:Wrote public SSH key to /home/ubuntu/cephtest/ceph.pub 2024-04-12T20:43:42.524 INFO:teuthology.orchestra.run.smithi018.stdout:Adding key to root@localhost authorized_keys... 2024-04-12T20:43:42.524 INFO:teuthology.orchestra.run.smithi018.stdout:Adding host smithi018... 2024-04-12T20:43:42.798 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:43:42 smithi018 ceph-mon[28799]: from='client.14138 -' entity='client.admin' cmd=[{"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T20:43:43.706 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:43:43 smithi018 ceph-mon[28799]: from='client.14140 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi018", "addr": "172.21.15.18", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T20:43:46.103 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:43:45 smithi018 ceph-mon[28799]: Deploying cephadm binary to smithi018 2024-04-12T20:43:48.557 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout Added host 'smithi018' with addr '172.21.15.18' 2024-04-12T20:43:48.558 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:43.221+0000 7f8126c00700 1 Processor -- start 2024-04-12T20:43:48.558 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:43.221+0000 7f8126c00700 1 -- start start 2024-04-12T20:43:48.558 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:43.222+0000 7f8126c00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8120077b20 0x7f8120076120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T20:43:48.558 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:43.222+0000 7f8126c00700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8120077f30 con 0x7f8120077b20 2024-04-12T20:43:48.558 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:43.222+0000 7f811fe00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8120077b20 0x7f8120076120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T20:43:48.559 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:43.222+0000 7f811fe00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8120077b20 0x7f8120076120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:44216/0 (socket says 172.21.15.18:44216) 2024-04-12T20:43:48.559 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:43.222+0000 7f811fe00700 1 -- 172.21.15.18:0/3884092262 learned_addr learned my addr 172.21.15.18:0/3884092262 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T20:43:48.559 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:43.223+0000 7f811fe00700 1 -- 172.21.15.18:0/3884092262 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8120076660 con 0x7f8120077b20 2024-04-12T20:43:48.559 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:43.223+0000 7f811fe00700 1 --2- 172.21.15.18:0/3884092262 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8120077b20 0x7f8120076120 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f81100098e0 tx=0x7f8110009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ecf5f2215a02e762 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T20:43:48.560 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:43.223+0000 7f811f400700 1 -- 172.21.15.18:0/3884092262 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f811000b860 con 0x7f8120077b20 2024-04-12T20:43:48.560 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:43.223+0000 7f811f400700 1 -- 172.21.15.18:0/3884092262 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7f811000be50 con 0x7f8120077b20 2024-04-12T20:43:48.560 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:43.223+0000 7f811f400700 1 -- 172.21.15.18:0/3884092262 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f8110010380 con 0x7f8120077b20 2024-04-12T20:43:48.560 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:43.224+0000 7f8126c00700 1 -- 172.21.15.18:0/3884092262 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8120077b20 msgr2=0x7f8120076120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:43:48.560 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:43.224+0000 7f8126c00700 1 --2- 172.21.15.18:0/3884092262 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8120077b20 0x7f8120076120 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f81100098e0 tx=0x7f8110009250 comp rx=0 tx=0).stop 2024-04-12T20:43:48.562 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:43.224+0000 7f8126c00700 1 -- 172.21.15.18:0/3884092262 shutdown_connections 2024-04-12T20:43:48.562 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:43.224+0000 7f8126c00700 1 --2- 172.21.15.18:0/3884092262 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8120077b20 0x7f8120076120 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:43:48.562 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:43.224+0000 7f8126c00700 1 -- 172.21.15.18:0/3884092262 >> 172.21.15.18:0/3884092262 conn(0x7f81200fe090 msgr2=0x7f81201004c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T20:43:48.562 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:43.224+0000 7f8126c00700 1 -- 172.21.15.18:0/3884092262 shutdown_connections 2024-04-12T20:43:48.562 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:43.224+0000 7f8126c00700 1 -- 172.21.15.18:0/3884092262 wait complete. 2024-04-12T20:43:48.562 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:43.225+0000 7f8126c00700 1 Processor -- start 2024-04-12T20:43:48.562 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:43.225+0000 7f8126c00700 1 -- start start 2024-04-12T20:43:48.562 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:43.226+0000 7f8126c00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8120077b20 0x7f8120113a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T20:43:48.562 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:43.226+0000 7f8126c00700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8120113f40 con 0x7f8120077b20 2024-04-12T20:43:48.563 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:43.226+0000 7f811fe00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8120077b20 0x7f8120113a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T20:43:48.563 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:43.226+0000 7f811fe00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8120077b20 0x7f8120113a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:44222/0 (socket says 172.21.15.18:44222) 2024-04-12T20:43:48.563 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:43.226+0000 7f811fe00700 1 -- 172.21.15.18:0/88742882 learned_addr learned my addr 172.21.15.18:0/88742882 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T20:43:48.563 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:43.226+0000 7f811fe00700 1 -- 172.21.15.18:0/88742882 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81100094f0 con 0x7f8120077b20 2024-04-12T20:43:48.563 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:43.226+0000 7f811fe00700 1 --2- 172.21.15.18:0/88742882 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8120077b20 0x7f8120113a00 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f8110006d40 tx=0x7f8110009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T20:43:48.565 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:43.227+0000 7f811cc00700 1 -- 172.21.15.18:0/88742882 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f81100107c0 con 0x7f8120077b20 2024-04-12T20:43:48.565 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:43.227+0000 7f811cc00700 1 -- 172.21.15.18:0/88742882 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7f8110010db0 con 0x7f8120077b20 2024-04-12T20:43:48.566 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:43.227+0000 7f811cc00700 1 -- 172.21.15.18:0/88742882 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f811001ad40 con 0x7f8120077b20 2024-04-12T20:43:48.566 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:43.227+0000 7f8126c00700 1 -- 172.21.15.18:0/88742882 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8120114140 con 0x7f8120077b20 2024-04-12T20:43:48.566 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:43.227+0000 7f8126c00700 1 -- 172.21.15.18:0/88742882 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8120114cb0 con 0x7f8120077b20 2024-04-12T20:43:48.566 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:43.228+0000 7f811cc00700 1 -- 172.21.15.18:0/88742882 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 8) v1 ==== 43368+0+0 (secure 0 0 0) 0x7f811001a5f0 con 0x7f8120077b20 2024-04-12T20:43:48.566 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:43.228+0000 7f8126c00700 1 -- 172.21.15.18:0/88742882 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f812005f5f0 con 0x7f8120077b20 2024-04-12T20:43:48.566 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:43.228+0000 7f811cc00700 1 --2- 172.21.15.18:0/88742882 >> [v2:172.21.15.18:6800/3403971131,v1:172.21.15.18:6801/3403971131] conn(0x7f81040367e0 0x7f8104038c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T20:43:48.566 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:43.229+0000 7f811cc00700 1 -- 172.21.15.18:0/88742882 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 936+0+0 (secure 0 0 0) 0x7f8110049fc0 con 0x7f8120077b20 2024-04-12T20:43:48.566 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:43.229+0000 7f8117400700 1 --2- 172.21.15.18:0/88742882 >> [v2:172.21.15.18:6800/3403971131,v1:172.21.15.18:6801/3403971131] conn(0x7f81040367e0 0x7f8104038c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T20:43:48.566 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:43.229+0000 7f8117400700 1 --2- 172.21.15.18:0/88742882 >> [v2:172.21.15.18:6800/3403971131,v1:172.21.15.18:6801/3403971131] conn(0x7f81040367e0 0x7f8104038c90 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f8108009990 tx=0x7f8108006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T20:43:48.566 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:43.236+0000 7f811cc00700 1 -- 172.21.15.18:0/88742882 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7f8110006d70 con 0x7f8120077b20 2024-04-12T20:43:48.566 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:43.452+0000 7f8126c00700 1 -- 172.21.15.18:0/88742882 --> [v2:172.21.15.18:6800/3403971131,v1:172.21.15.18:6801/3403971131] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi018", "addr": "172.21.15.18", "target": ["mon-mgr", ""]}) v1 -- 0x7f81200ff520 con 0x7f81040367e0 2024-04-12T20:43:48.566 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:47.869+0000 7f811cc00700 1 -- 172.21.15.18:0/88742882 <== mgr.14120 v2:172.21.15.18:6800/3403971131 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+48 (secure 0 0 0) 0x7f81200ff520 con 0x7f81040367e0 2024-04-12T20:43:48.566 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:47.874+0000 7f8126c00700 1 -- 172.21.15.18:0/88742882 >> [v2:172.21.15.18:6800/3403971131,v1:172.21.15.18:6801/3403971131] conn(0x7f81040367e0 msgr2=0x7f8104038c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:43:48.567 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:47.874+0000 7f8126c00700 1 --2- 172.21.15.18:0/88742882 >> [v2:172.21.15.18:6800/3403971131,v1:172.21.15.18:6801/3403971131] conn(0x7f81040367e0 0x7f8104038c90 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f8108009990 tx=0x7f8108006e40 comp rx=0 tx=0).stop 2024-04-12T20:43:48.567 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:47.874+0000 7f8126c00700 1 -- 172.21.15.18:0/88742882 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8120077b20 msgr2=0x7f8120113a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:43:48.567 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:47.874+0000 7f8126c00700 1 --2- 172.21.15.18:0/88742882 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8120077b20 0x7f8120113a00 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f8110006d40 tx=0x7f8110009250 comp rx=0 tx=0).stop 2024-04-12T20:43:48.567 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:47.874+0000 7f8126c00700 1 -- 172.21.15.18:0/88742882 shutdown_connections 2024-04-12T20:43:48.567 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:47.874+0000 7f8126c00700 1 --2- 172.21.15.18:0/88742882 >> [v2:172.21.15.18:6800/3403971131,v1:172.21.15.18:6801/3403971131] conn(0x7f81040367e0 0x7f8104038c90 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:43:48.567 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:47.874+0000 7f8126c00700 1 --2- 172.21.15.18:0/88742882 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8120077b20 0x7f8120113a00 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:43:48.567 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:47.874+0000 7f8126c00700 1 -- 172.21.15.18:0/88742882 >> 172.21.15.18:0/88742882 conn(0x7f81200fe090 msgr2=0x7f81200fed30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T20:43:48.567 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:47.874+0000 7f8126c00700 1 -- 172.21.15.18:0/88742882 shutdown_connections 2024-04-12T20:43:48.567 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:47.875+0000 7f8126c00700 1 -- 172.21.15.18:0/88742882 wait complete. 2024-04-12T20:43:48.567 INFO:teuthology.orchestra.run.smithi018.stdout:Deploying mon service with default placement... 2024-04-12T20:43:49.204 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:43:48 smithi018 ceph-mon[28799]: from='mgr.14120 172.21.15.18:0/4270481350' entity='mgr.smithi018.xjzwga' 2024-04-12T20:43:49.204 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:43:48 smithi018 ceph-mon[28799]: Added host smithi018 2024-04-12T20:43:49.204 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:43:48 smithi018 ceph-mon[28799]: from='mgr.14120 172.21.15.18:0/4270481350' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-12T20:43:50.853 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:43:50 smithi018 ceph-mon[28799]: from='client.14142 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T20:43:50.853 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:43:50 smithi018 ceph-mon[28799]: Saving service mon spec with placement count:5 2024-04-12T20:43:50.853 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:43:50 smithi018 ceph-mon[28799]: from='mgr.14120 172.21.15.18:0/4270481350' entity='mgr.smithi018.xjzwga' 2024-04-12T20:43:50.868 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout Scheduled mon update... 2024-04-12T20:43:50.868 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:49.375+0000 7f5eeb000700 1 Processor -- start 2024-04-12T20:43:50.868 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:49.375+0000 7f5eeb000700 1 -- start start 2024-04-12T20:43:50.868 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:49.375+0000 7f5eeb000700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5ee4102970 0x7f5ee4102d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T20:43:50.873 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:49.375+0000 7f5eeb000700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ee41032c0 con 0x7f5ee4102970 2024-04-12T20:43:50.873 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:49.376+0000 7f5ee3e00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5ee4102970 0x7f5ee4102d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T20:43:50.873 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:49.376+0000 7f5ee3e00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5ee4102970 0x7f5ee4102d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:44224/0 (socket says 172.21.15.18:44224) 2024-04-12T20:43:50.874 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:49.376+0000 7f5ee3e00700 1 -- 172.21.15.18:0/1388915441 learned_addr learned my addr 172.21.15.18:0/1388915441 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T20:43:50.874 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:49.376+0000 7f5ee3e00700 1 -- 172.21.15.18:0/1388915441 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ee4103400 con 0x7f5ee4102970 2024-04-12T20:43:50.874 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:49.377+0000 7f5ee3e00700 1 --2- 172.21.15.18:0/1388915441 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5ee4102970 0x7f5ee4102d80 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f5ed40098e0 tx=0x7f5ed4009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ff4bf8e3ac60f98 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T20:43:50.874 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:49.377+0000 7f5ee2a00700 1 -- 172.21.15.18:0/1388915441 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f5ed400b860 con 0x7f5ee4102970 2024-04-12T20:43:50.874 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:49.377+0000 7f5ee2a00700 1 -- 172.21.15.18:0/1388915441 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7f5ed400be50 con 0x7f5ee4102970 2024-04-12T20:43:50.874 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:49.377+0000 7f5ee2a00700 1 -- 172.21.15.18:0/1388915441 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f5ed4010380 con 0x7f5ee4102970 2024-04-12T20:43:50.874 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:49.378+0000 7f5eeb000700 1 -- 172.21.15.18:0/1388915441 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5ee4102970 msgr2=0x7f5ee4102d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:43:50.874 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:49.378+0000 7f5eeb000700 1 --2- 172.21.15.18:0/1388915441 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5ee4102970 0x7f5ee4102d80 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f5ed40098e0 tx=0x7f5ed4009250 comp rx=0 tx=0).stop 2024-04-12T20:43:50.874 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:49.378+0000 7f5eeb000700 1 -- 172.21.15.18:0/1388915441 shutdown_connections 2024-04-12T20:43:50.874 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:49.378+0000 7f5eeb000700 1 --2- 172.21.15.18:0/1388915441 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5ee4102970 0x7f5ee4102d80 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:43:50.874 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:49.378+0000 7f5eeb000700 1 -- 172.21.15.18:0/1388915441 >> 172.21.15.18:0/1388915441 conn(0x7f5ee40fe090 msgr2=0x7f5ee4100500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T20:43:50.874 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:49.378+0000 7f5eeb000700 1 -- 172.21.15.18:0/1388915441 shutdown_connections 2024-04-12T20:43:50.874 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:49.378+0000 7f5eeb000700 1 -- 172.21.15.18:0/1388915441 wait complete. 2024-04-12T20:43:50.875 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:49.379+0000 7f5eeb000700 1 Processor -- start 2024-04-12T20:43:50.875 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:49.379+0000 7f5eeb000700 1 -- start start 2024-04-12T20:43:50.875 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:49.380+0000 7f5eeb000700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5ee4102970 0x7f5ee4113950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T20:43:50.875 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:49.380+0000 7f5eeb000700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ee4113e90 con 0x7f5ee4102970 2024-04-12T20:43:50.875 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:49.380+0000 7f5ee3e00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5ee4102970 0x7f5ee4113950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T20:43:50.875 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:49.380+0000 7f5ee3e00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5ee4102970 0x7f5ee4113950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:44228/0 (socket says 172.21.15.18:44228) 2024-04-12T20:43:50.875 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:49.380+0000 7f5ee3e00700 1 -- 172.21.15.18:0/3321956282 learned_addr learned my addr 172.21.15.18:0/3321956282 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T20:43:50.875 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:49.380+0000 7f5ee3e00700 1 -- 172.21.15.18:0/3321956282 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ed40094f0 con 0x7f5ee4102970 2024-04-12T20:43:50.875 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:49.381+0000 7f5ee3e00700 1 --2- 172.21.15.18:0/3321956282 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5ee4102970 0x7f5ee4113950 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f5ed400ba90 tx=0x7f5ed4009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T20:43:50.875 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:49.381+0000 7f5edbe00700 1 -- 172.21.15.18:0/3321956282 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f5ed40107c0 con 0x7f5ee4102970 2024-04-12T20:43:50.875 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:49.381+0000 7f5edbe00700 1 -- 172.21.15.18:0/3321956282 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7f5ed4010db0 con 0x7f5ee4102970 2024-04-12T20:43:50.875 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:49.381+0000 7f5eeb000700 1 -- 172.21.15.18:0/3321956282 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5ee4114090 con 0x7f5ee4102970 2024-04-12T20:43:50.875 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:49.381+0000 7f5edbe00700 1 -- 172.21.15.18:0/3321956282 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f5ed401add0 con 0x7f5ee4102970 2024-04-12T20:43:50.875 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:49.381+0000 7f5eeb000700 1 -- 172.21.15.18:0/3321956282 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5ee4114c00 con 0x7f5ee4102970 2024-04-12T20:43:50.876 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:49.382+0000 7f5edbe00700 1 -- 172.21.15.18:0/3321956282 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 8) v1 ==== 43368+0+0 (secure 0 0 0) 0x7f5ed4010920 con 0x7f5ee4102970 2024-04-12T20:43:50.876 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:49.383+0000 7f5eeb000700 1 -- 172.21.15.18:0/3321956282 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5ee405f5f0 con 0x7f5ee4102970 2024-04-12T20:43:50.876 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:49.383+0000 7f5edbe00700 1 --2- 172.21.15.18:0/3321956282 >> [v2:172.21.15.18:6800/3403971131,v1:172.21.15.18:6801/3403971131] conn(0x7f5ec8036790 0x7f5ec8038c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T20:43:50.876 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:49.383+0000 7f5edbe00700 1 -- 172.21.15.18:0/3321956282 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 936+0+0 (secure 0 0 0) 0x7f5ed404a680 con 0x7f5ee4102970 2024-04-12T20:43:50.876 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:49.384+0000 7f5ee3400700 1 --2- 172.21.15.18:0/3321956282 >> [v2:172.21.15.18:6800/3403971131,v1:172.21.15.18:6801/3403971131] conn(0x7f5ec8036790 0x7f5ec8038c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T20:43:50.876 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:49.384+0000 7f5ee3400700 1 --2- 172.21.15.18:0/3321956282 >> [v2:172.21.15.18:6800/3403971131,v1:172.21.15.18:6801/3403971131] conn(0x7f5ec8036790 0x7f5ec8038c40 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f5ed0009990 tx=0x7f5ed0006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T20:43:50.876 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:49.389+0000 7f5edbe00700 1 -- 172.21.15.18:0/3321956282 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7f5ed4018b40 con 0x7f5ee4102970 2024-04-12T20:43:50.876 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:49.659+0000 7f5eeb000700 1 -- 172.21.15.18:0/3321956282 --> [v2:172.21.15.18:6800/3403971131,v1:172.21.15.18:6801/3403971131] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "target": ["mon-mgr", ""]}) v1 -- 0x7f5ee40ff320 con 0x7f5ec8036790 2024-04-12T20:43:50.876 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:49.670+0000 7f5edbe00700 1 -- 172.21.15.18:0/3321956282 <== mgr.14120 v2:172.21.15.18:6800/3403971131 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7f5ee40ff320 con 0x7f5ec8036790 2024-04-12T20:43:50.876 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:49.673+0000 7f5eeb000700 1 -- 172.21.15.18:0/3321956282 >> [v2:172.21.15.18:6800/3403971131,v1:172.21.15.18:6801/3403971131] conn(0x7f5ec8036790 msgr2=0x7f5ec8038c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:43:50.876 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:49.673+0000 7f5eeb000700 1 --2- 172.21.15.18:0/3321956282 >> [v2:172.21.15.18:6800/3403971131,v1:172.21.15.18:6801/3403971131] conn(0x7f5ec8036790 0x7f5ec8038c40 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f5ed0009990 tx=0x7f5ed0006e40 comp rx=0 tx=0).stop 2024-04-12T20:43:50.876 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:49.673+0000 7f5eeb000700 1 -- 172.21.15.18:0/3321956282 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5ee4102970 msgr2=0x7f5ee4113950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:43:50.876 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:49.673+0000 7f5eeb000700 1 --2- 172.21.15.18:0/3321956282 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5ee4102970 0x7f5ee4113950 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f5ed400ba90 tx=0x7f5ed4009250 comp rx=0 tx=0).stop 2024-04-12T20:43:50.876 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:49.673+0000 7f5eeb000700 1 -- 172.21.15.18:0/3321956282 shutdown_connections 2024-04-12T20:43:50.877 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:49.673+0000 7f5eeb000700 1 --2- 172.21.15.18:0/3321956282 >> [v2:172.21.15.18:6800/3403971131,v1:172.21.15.18:6801/3403971131] conn(0x7f5ec8036790 0x7f5ec8038c40 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:43:50.877 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:49.673+0000 7f5eeb000700 1 --2- 172.21.15.18:0/3321956282 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5ee4102970 0x7f5ee4113950 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:43:50.877 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:49.673+0000 7f5eeb000700 1 -- 172.21.15.18:0/3321956282 >> 172.21.15.18:0/3321956282 conn(0x7f5ee40fe090 msgr2=0x7f5ee40fec10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T20:43:50.877 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:49.674+0000 7f5eeb000700 1 -- 172.21.15.18:0/3321956282 shutdown_connections 2024-04-12T20:43:50.877 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:49.674+0000 7f5eeb000700 1 -- 172.21.15.18:0/3321956282 wait complete. 2024-04-12T20:43:50.877 INFO:teuthology.orchestra.run.smithi018.stdout:Deploying mgr service with default placement... 2024-04-12T20:43:52.853 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:43:52 smithi018 ceph-mon[28799]: from='mgr.14120 172.21.15.18:0/4270481350' entity='mgr.smithi018.xjzwga' 2024-04-12T20:43:52.853 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:43:52 smithi018 ceph-mon[28799]: from='mgr.14120 172.21.15.18:0/4270481350' entity='mgr.smithi018.xjzwga' 2024-04-12T20:43:52.853 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:43:52 smithi018 ceph-mon[28799]: from='mgr.14120 172.21.15.18:0/4270481350' entity='mgr.smithi018.xjzwga' 2024-04-12T20:43:52.922 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout Scheduled mgr update... 2024-04-12T20:43:52.922 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:51.892+0000 7f7615800700 1 Processor -- start 2024-04-12T20:43:52.922 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:51.892+0000 7f7615800700 1 -- start start 2024-04-12T20:43:52.923 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:51.892+0000 7f7615800700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7610104b80 0x7f7610104f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T20:43:52.923 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:51.893+0000 7f7615800700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7610105560 con 0x7f7610104b80 2024-04-12T20:43:52.923 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:51.893+0000 7f760ea00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7610104b80 0x7f7610104f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T20:43:52.923 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:51.893+0000 7f760ea00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7610104b80 0x7f7610104f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:33400/0 (socket says 172.21.15.18:33400) 2024-04-12T20:43:52.923 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:51.893+0000 7f760ea00700 1 -- 172.21.15.18:0/2066229602 learned_addr learned my addr 172.21.15.18:0/2066229602 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T20:43:52.923 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:51.894+0000 7f760ea00700 1 -- 172.21.15.18:0/2066229602 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76101056a0 con 0x7f7610104b80 2024-04-12T20:43:52.923 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:51.894+0000 7f760ea00700 1 --2- 172.21.15.18:0/2066229602 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7610104b80 0x7f7610104f90 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f76000098e0 tx=0x7f7600009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=283a3fe3ef88d078 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T20:43:52.923 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:51.894+0000 7f760d600700 1 -- 172.21.15.18:0/2066229602 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f760000b860 con 0x7f7610104b80 2024-04-12T20:43:52.923 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:51.894+0000 7f760d600700 1 -- 172.21.15.18:0/2066229602 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f760000be90 con 0x7f7610104b80 2024-04-12T20:43:52.924 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:51.894+0000 7f760d600700 1 -- 172.21.15.18:0/2066229602 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f76000103c0 con 0x7f7610104b80 2024-04-12T20:43:52.924 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:51.895+0000 7f7615800700 1 -- 172.21.15.18:0/2066229602 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7610104b80 msgr2=0x7f7610104f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:43:52.924 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:51.895+0000 7f7615800700 1 --2- 172.21.15.18:0/2066229602 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7610104b80 0x7f7610104f90 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f76000098e0 tx=0x7f7600009250 comp rx=0 tx=0).stop 2024-04-12T20:43:52.924 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:51.895+0000 7f7615800700 1 -- 172.21.15.18:0/2066229602 shutdown_connections 2024-04-12T20:43:52.924 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:51.895+0000 7f7615800700 1 --2- 172.21.15.18:0/2066229602 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7610104b80 0x7f7610104f90 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:43:52.924 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:51.895+0000 7f7615800700 1 -- 172.21.15.18:0/2066229602 >> 172.21.15.18:0/2066229602 conn(0x7f7610078320 msgr2=0x7f7610078720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T20:43:52.924 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:51.895+0000 7f7615800700 1 -- 172.21.15.18:0/2066229602 shutdown_connections 2024-04-12T20:43:52.924 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:51.895+0000 7f7615800700 1 -- 172.21.15.18:0/2066229602 wait complete. 2024-04-12T20:43:52.924 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:51.896+0000 7f7615800700 1 Processor -- start 2024-04-12T20:43:52.925 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:51.896+0000 7f7615800700 1 -- start start 2024-04-12T20:43:52.925 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:51.897+0000 7f7615800700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7610104b80 0x7f76101138e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T20:43:52.925 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:51.897+0000 7f7615800700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7610113e20 con 0x7f7610104b80 2024-04-12T20:43:52.925 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:51.897+0000 7f760ea00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7610104b80 0x7f76101138e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T20:43:52.926 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:51.897+0000 7f760ea00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7610104b80 0x7f76101138e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:33402/0 (socket says 172.21.15.18:33402) 2024-04-12T20:43:52.926 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:51.897+0000 7f760ea00700 1 -- 172.21.15.18:0/1796298625 learned_addr learned my addr 172.21.15.18:0/1796298625 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T20:43:52.926 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:51.898+0000 7f760ea00700 1 -- 172.21.15.18:0/1796298625 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76000094f0 con 0x7f7610104b80 2024-04-12T20:43:52.926 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:51.898+0000 7f760ea00700 1 --2- 172.21.15.18:0/1796298625 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7610104b80 0x7f76101138e0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f7600006b00 tx=0x7f7600009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T20:43:52.926 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:51.898+0000 7f7606a00700 1 -- 172.21.15.18:0/1796298625 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f7600010840 con 0x7f7610104b80 2024-04-12T20:43:52.926 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:51.898+0000 7f7606a00700 1 -- 172.21.15.18:0/1796298625 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f7600010e70 con 0x7f7610104b80 2024-04-12T20:43:52.926 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:51.898+0000 7f7615800700 1 -- 172.21.15.18:0/1796298625 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7610114020 con 0x7f7610104b80 2024-04-12T20:43:52.926 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:51.898+0000 7f7606a00700 1 -- 172.21.15.18:0/1796298625 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f76000243f0 con 0x7f7610104b80 2024-04-12T20:43:52.926 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:51.899+0000 7f7615800700 1 -- 172.21.15.18:0/1796298625 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7610114b90 con 0x7f7610104b80 2024-04-12T20:43:52.927 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:51.900+0000 7f7606a00700 1 -- 172.21.15.18:0/1796298625 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 8) v1 ==== 43368+0+0 (secure 0 0 0) 0x7f76000109a0 con 0x7f7610104b80 2024-04-12T20:43:52.927 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:51.900+0000 7f7615800700 1 -- 172.21.15.18:0/1796298625 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f761005f5f0 con 0x7f7610104b80 2024-04-12T20:43:52.927 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:51.900+0000 7f7606a00700 1 --2- 172.21.15.18:0/1796298625 >> [v2:172.21.15.18:6800/3403971131,v1:172.21.15.18:6801/3403971131] conn(0x7f75f40367e0 0x7f75f4038c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T20:43:52.927 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:51.900+0000 7f7606a00700 1 -- 172.21.15.18:0/1796298625 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 936+0+0 (secure 0 0 0) 0x7f760004a770 con 0x7f7610104b80 2024-04-12T20:43:52.927 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:51.901+0000 7f760e000700 1 --2- 172.21.15.18:0/1796298625 >> [v2:172.21.15.18:6800/3403971131,v1:172.21.15.18:6801/3403971131] conn(0x7f75f40367e0 0x7f75f4038c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T20:43:52.927 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:51.901+0000 7f760e000700 1 --2- 172.21.15.18:0/1796298625 >> [v2:172.21.15.18:6800/3403971131,v1:172.21.15.18:6801/3403971131] conn(0x7f75f40367e0 0x7f75f4038c90 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f75fc009990 tx=0x7f75fc006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T20:43:52.927 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:51.905+0000 7f7606a00700 1 -- 172.21.15.18:0/1796298625 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7f760001a7c0 con 0x7f7610104b80 2024-04-12T20:43:52.927 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:52.151+0000 7f7615800700 1 -- 172.21.15.18:0/1796298625 --> [v2:172.21.15.18:6800/3403971131,v1:172.21.15.18:6801/3403971131] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "target": ["mon-mgr", ""]}) v1 -- 0x7f7610102c10 con 0x7f75f40367e0 2024-04-12T20:43:52.927 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:52.161+0000 7f7606a00700 1 -- 172.21.15.18:0/1796298625 <== mgr.14120 v2:172.21.15.18:6800/3403971131 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7f7610102c10 con 0x7f75f40367e0 2024-04-12T20:43:52.928 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:52.165+0000 7f7615800700 1 -- 172.21.15.18:0/1796298625 >> [v2:172.21.15.18:6800/3403971131,v1:172.21.15.18:6801/3403971131] conn(0x7f75f40367e0 msgr2=0x7f75f4038c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:43:52.928 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:52.165+0000 7f7615800700 1 --2- 172.21.15.18:0/1796298625 >> [v2:172.21.15.18:6800/3403971131,v1:172.21.15.18:6801/3403971131] conn(0x7f75f40367e0 0x7f75f4038c90 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f75fc009990 tx=0x7f75fc006e40 comp rx=0 tx=0).stop 2024-04-12T20:43:52.928 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:52.165+0000 7f7615800700 1 -- 172.21.15.18:0/1796298625 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7610104b80 msgr2=0x7f76101138e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:43:52.928 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:52.165+0000 7f7615800700 1 --2- 172.21.15.18:0/1796298625 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7610104b80 0x7f76101138e0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f7600006b00 tx=0x7f7600009250 comp rx=0 tx=0).stop 2024-04-12T20:43:52.928 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:52.165+0000 7f7615800700 1 -- 172.21.15.18:0/1796298625 shutdown_connections 2024-04-12T20:43:52.928 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:52.165+0000 7f7615800700 1 --2- 172.21.15.18:0/1796298625 >> [v2:172.21.15.18:6800/3403971131,v1:172.21.15.18:6801/3403971131] conn(0x7f75f40367e0 0x7f75f4038c90 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:43:52.928 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:52.165+0000 7f7615800700 1 --2- 172.21.15.18:0/1796298625 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7610104b80 0x7f76101138e0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:43:52.928 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:52.165+0000 7f7615800700 1 -- 172.21.15.18:0/1796298625 >> 172.21.15.18:0/1796298625 conn(0x7f7610078320 msgr2=0x7f76101024e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T20:43:52.928 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:52.166+0000 7f7615800700 1 -- 172.21.15.18:0/1796298625 shutdown_connections 2024-04-12T20:43:52.928 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:52.166+0000 7f7615800700 1 -- 172.21.15.18:0/1796298625 wait complete. 2024-04-12T20:43:52.928 INFO:teuthology.orchestra.run.smithi018.stdout:Deploying crash service with default placement... 2024-04-12T20:43:53.762 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:43:53 smithi018 ceph-mon[28799]: from='client.14144 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T20:43:53.763 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:43:53 smithi018 ceph-mon[28799]: Saving service mgr spec with placement count:2 2024-04-12T20:43:54.853 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:43:54 smithi018 ceph-mon[28799]: from='client.14146 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "crash", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T20:43:54.853 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:43:54 smithi018 ceph-mon[28799]: Saving service crash spec with placement * 2024-04-12T20:43:54.853 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:43:54 smithi018 ceph-mon[28799]: from='mgr.14120 172.21.15.18:0/4270481350' entity='mgr.smithi018.xjzwga' 2024-04-12T20:43:54.853 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:43:54 smithi018 ceph-mon[28799]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-12T20:43:54.853 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:43:54 smithi018 ceph-mon[28799]: from='mgr.14120 172.21.15.18:0/4270481350' entity='mgr.smithi018.xjzwga' 2024-04-12T20:43:54.928 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout Scheduled crash update... 2024-04-12T20:43:54.928 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:53.515+0000 7fba9f200700 1 Processor -- start 2024-04-12T20:43:54.928 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:53.516+0000 7fba9f200700 1 -- start start 2024-04-12T20:43:54.928 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:53.516+0000 7fba9f200700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fba98077ab0 0x7fba98076100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T20:43:54.928 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:53.516+0000 7fba9f200700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba98077ec0 con 0x7fba98077ab0 2024-04-12T20:43:54.928 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:53.516+0000 7fba97e00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fba98077ab0 0x7fba98076100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T20:43:54.928 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:53.516+0000 7fba97e00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fba98077ab0 0x7fba98076100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:33406/0 (socket says 172.21.15.18:33406) 2024-04-12T20:43:54.928 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:53.516+0000 7fba97e00700 1 -- 172.21.15.18:0/3572774027 learned_addr learned my addr 172.21.15.18:0/3572774027 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T20:43:54.929 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:53.517+0000 7fba97e00700 1 -- 172.21.15.18:0/3572774027 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba98078000 con 0x7fba98077ab0 2024-04-12T20:43:54.929 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:53.517+0000 7fba97e00700 1 --2- 172.21.15.18:0/3572774027 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fba98077ab0 0x7fba98076100 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fba880098e0 tx=0x7fba88009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=41a957c4a2defb4 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T20:43:54.929 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:53.517+0000 7fba96a00700 1 -- 172.21.15.18:0/3572774027 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fba8800b860 con 0x7fba98077ab0 2024-04-12T20:43:54.929 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:53.517+0000 7fba96a00700 1 -- 172.21.15.18:0/3572774027 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fba8800be90 con 0x7fba98077ab0 2024-04-12T20:43:54.929 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:53.517+0000 7fba96a00700 1 -- 172.21.15.18:0/3572774027 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fba880103c0 con 0x7fba98077ab0 2024-04-12T20:43:54.929 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:53.518+0000 7fba9f200700 1 -- 172.21.15.18:0/3572774027 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fba98077ab0 msgr2=0x7fba98076100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:43:54.929 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:53.518+0000 7fba9f200700 1 --2- 172.21.15.18:0/3572774027 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fba98077ab0 0x7fba98076100 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fba880098e0 tx=0x7fba88009250 comp rx=0 tx=0).stop 2024-04-12T20:43:54.929 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:53.518+0000 7fba9f200700 1 -- 172.21.15.18:0/3572774027 shutdown_connections 2024-04-12T20:43:54.929 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:53.518+0000 7fba9f200700 1 --2- 172.21.15.18:0/3572774027 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fba98077ab0 0x7fba98076100 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:43:54.929 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:53.518+0000 7fba9f200700 1 -- 172.21.15.18:0/3572774027 >> 172.21.15.18:0/3572774027 conn(0x7fba980fe090 msgr2=0x7fba98100500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T20:43:54.929 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:53.519+0000 7fba9f200700 1 -- 172.21.15.18:0/3572774027 shutdown_connections 2024-04-12T20:43:54.929 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:53.519+0000 7fba9f200700 1 -- 172.21.15.18:0/3572774027 wait complete. 2024-04-12T20:43:54.929 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:53.520+0000 7fba9f200700 1 Processor -- start 2024-04-12T20:43:54.929 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:53.520+0000 7fba9f200700 1 -- start start 2024-04-12T20:43:54.929 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:53.520+0000 7fba9f200700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fba98077ab0 0x7fba98113990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T20:43:54.930 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:53.520+0000 7fba9f200700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba98113ed0 con 0x7fba98077ab0 2024-04-12T20:43:54.930 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:53.521+0000 7fba97e00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fba98077ab0 0x7fba98113990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T20:43:54.930 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:53.521+0000 7fba97e00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fba98077ab0 0x7fba98113990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:33414/0 (socket says 172.21.15.18:33414) 2024-04-12T20:43:54.930 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:53.521+0000 7fba97e00700 1 -- 172.21.15.18:0/1672803983 learned_addr learned my addr 172.21.15.18:0/1672803983 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T20:43:54.930 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:53.521+0000 7fba97e00700 1 -- 172.21.15.18:0/1672803983 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba880094f0 con 0x7fba98077ab0 2024-04-12T20:43:54.930 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:53.521+0000 7fba97e00700 1 --2- 172.21.15.18:0/1672803983 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fba98077ab0 0x7fba98113990 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fba880105b0 tx=0x7fba88009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T20:43:54.930 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:53.522+0000 7fba8fe00700 1 -- 172.21.15.18:0/1672803983 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fba88010840 con 0x7fba98077ab0 2024-04-12T20:43:54.930 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:53.522+0000 7fba8fe00700 1 -- 172.21.15.18:0/1672803983 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fba88010e70 con 0x7fba98077ab0 2024-04-12T20:43:54.930 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:53.522+0000 7fba9f200700 1 -- 172.21.15.18:0/1672803983 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fba981140d0 con 0x7fba98077ab0 2024-04-12T20:43:54.930 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:53.522+0000 7fba9f200700 1 -- 172.21.15.18:0/1672803983 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fba98114c40 con 0x7fba98077ab0 2024-04-12T20:43:54.930 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:53.522+0000 7fba8fe00700 1 -- 172.21.15.18:0/1672803983 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fba8801ae30 con 0x7fba98077ab0 2024-04-12T20:43:54.930 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:53.524+0000 7fba8fe00700 1 -- 172.21.15.18:0/1672803983 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 8) v1 ==== 43368+0+0 (secure 0 0 0) 0x7fba8802e440 con 0x7fba98077ab0 2024-04-12T20:43:54.930 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:53.524+0000 7fba9f200700 1 -- 172.21.15.18:0/1672803983 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fba780052e0 con 0x7fba98077ab0 2024-04-12T20:43:54.931 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:53.524+0000 7fba8fe00700 1 --2- 172.21.15.18:0/1672803983 >> [v2:172.21.15.18:6800/3403971131,v1:172.21.15.18:6801/3403971131] conn(0x7fba7c0367e0 0x7fba7c038c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T20:43:54.931 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:53.524+0000 7fba8fe00700 1 -- 172.21.15.18:0/1672803983 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 936+0+0 (secure 0 0 0) 0x7fba8804a5c0 con 0x7fba98077ab0 2024-04-12T20:43:54.931 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:53.525+0000 7fba97400700 1 --2- 172.21.15.18:0/1672803983 >> [v2:172.21.15.18:6800/3403971131,v1:172.21.15.18:6801/3403971131] conn(0x7fba7c0367e0 0x7fba7c038c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T20:43:54.931 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:53.526+0000 7fba97400700 1 --2- 172.21.15.18:0/1672803983 >> [v2:172.21.15.18:6800/3403971131,v1:172.21.15.18:6801/3403971131] conn(0x7fba7c0367e0 0x7fba7c038c90 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fba84009990 tx=0x7fba84006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T20:43:54.931 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:53.530+0000 7fba8fe00700 1 -- 172.21.15.18:0/1672803983 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7fba88018940 con 0x7fba98077ab0 2024-04-12T20:43:54.931 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:53.757+0000 7fba9f200700 1 -- 172.21.15.18:0/1672803983 --> [v2:172.21.15.18:6800/3403971131,v1:172.21.15.18:6801/3403971131] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "crash", "target": ["mon-mgr", ""]}) v1 -- 0x7fba78000bf0 con 0x7fba7c0367e0 2024-04-12T20:43:54.931 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:53.763+0000 7fba8fe00700 1 -- 172.21.15.18:0/1672803983 <== mgr.14120 v2:172.21.15.18:6800/3403971131 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+26 (secure 0 0 0) 0x7fba78000bf0 con 0x7fba7c0367e0 2024-04-12T20:43:54.931 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:53.766+0000 7fba9f200700 1 -- 172.21.15.18:0/1672803983 >> [v2:172.21.15.18:6800/3403971131,v1:172.21.15.18:6801/3403971131] conn(0x7fba7c0367e0 msgr2=0x7fba7c038c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:43:54.931 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:53.766+0000 7fba9f200700 1 --2- 172.21.15.18:0/1672803983 >> [v2:172.21.15.18:6800/3403971131,v1:172.21.15.18:6801/3403971131] conn(0x7fba7c0367e0 0x7fba7c038c90 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fba84009990 tx=0x7fba84006e40 comp rx=0 tx=0).stop 2024-04-12T20:43:54.931 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:53.766+0000 7fba9f200700 1 -- 172.21.15.18:0/1672803983 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fba98077ab0 msgr2=0x7fba98113990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:43:54.931 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:53.766+0000 7fba9f200700 1 --2- 172.21.15.18:0/1672803983 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fba98077ab0 0x7fba98113990 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fba880105b0 tx=0x7fba88009250 comp rx=0 tx=0).stop 2024-04-12T20:43:54.931 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:53.766+0000 7fba9f200700 1 -- 172.21.15.18:0/1672803983 shutdown_connections 2024-04-12T20:43:54.932 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:53.766+0000 7fba9f200700 1 --2- 172.21.15.18:0/1672803983 >> [v2:172.21.15.18:6800/3403971131,v1:172.21.15.18:6801/3403971131] conn(0x7fba7c0367e0 0x7fba7c038c90 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:43:54.932 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:53.766+0000 7fba9f200700 1 --2- 172.21.15.18:0/1672803983 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fba98077ab0 0x7fba98113990 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:43:54.932 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:53.766+0000 7fba9f200700 1 -- 172.21.15.18:0/1672803983 >> 172.21.15.18:0/1672803983 conn(0x7fba980fe090 msgr2=0x7fba980fecf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T20:43:54.932 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:53.767+0000 7fba9f200700 1 -- 172.21.15.18:0/1672803983 shutdown_connections 2024-04-12T20:43:54.932 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:53.767+0000 7fba9f200700 1 -- 172.21.15.18:0/1672803983 wait complete. 2024-04-12T20:43:54.932 INFO:teuthology.orchestra.run.smithi018.stdout:Deploying prometheus service with default placement... 2024-04-12T20:43:56.430 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout Scheduled prometheus update... 2024-04-12T20:43:56.430 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:55.473+0000 7fede6000700 1 Processor -- start 2024-04-12T20:43:56.430 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:55.473+0000 7fede6000700 1 -- start start 2024-04-12T20:43:56.430 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:55.473+0000 7fede6000700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fede007cfb0 0x7fede010bcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T20:43:56.430 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:55.473+0000 7fede6000700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fede007d450 con 0x7fede007cfb0 2024-04-12T20:43:56.430 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:55.474+0000 7feddf400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fede007cfb0 0x7fede010bcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T20:43:56.431 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:55.474+0000 7feddf400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fede007cfb0 0x7fede010bcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:33426/0 (socket says 172.21.15.18:33426) 2024-04-12T20:43:56.431 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:55.474+0000 7feddf400700 1 -- 172.21.15.18:0/3082129725 learned_addr learned my addr 172.21.15.18:0/3082129725 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T20:43:56.431 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:55.474+0000 7feddf400700 1 -- 172.21.15.18:0/3082129725 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fede010c210 con 0x7fede007cfb0 2024-04-12T20:43:56.431 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:55.475+0000 7feddf400700 1 --2- 172.21.15.18:0/3082129725 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fede007cfb0 0x7fede010bcd0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fedd00098e0 tx=0x7fedd0009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a80b60e9e5e47b34 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T20:43:56.431 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:55.475+0000 7fedde000700 1 -- 172.21.15.18:0/3082129725 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fedd000b860 con 0x7fede007cfb0 2024-04-12T20:43:56.431 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:55.475+0000 7fedde000700 1 -- 172.21.15.18:0/3082129725 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fedd000be90 con 0x7fede007cfb0 2024-04-12T20:43:56.431 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:55.475+0000 7fedde000700 1 -- 172.21.15.18:0/3082129725 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fedd00103c0 con 0x7fede007cfb0 2024-04-12T20:43:56.431 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:55.476+0000 7fede6000700 1 -- 172.21.15.18:0/3082129725 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fede007cfb0 msgr2=0x7fede010bcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:43:56.431 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:55.476+0000 7fede6000700 1 --2- 172.21.15.18:0/3082129725 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fede007cfb0 0x7fede010bcd0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fedd00098e0 tx=0x7fedd0009250 comp rx=0 tx=0).stop 2024-04-12T20:43:56.431 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:55.476+0000 7fede6000700 1 -- 172.21.15.18:0/3082129725 shutdown_connections 2024-04-12T20:43:56.431 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:55.476+0000 7fede6000700 1 --2- 172.21.15.18:0/3082129725 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fede007cfb0 0x7fede010bcd0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:43:56.431 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:55.476+0000 7fede6000700 1 -- 172.21.15.18:0/3082129725 >> 172.21.15.18:0/3082129725 conn(0x7fede0078530 msgr2=0x7fede007a940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T20:43:56.431 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:55.476+0000 7fede6000700 1 -- 172.21.15.18:0/3082129725 shutdown_connections 2024-04-12T20:43:56.432 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:55.476+0000 7fede6000700 1 -- 172.21.15.18:0/3082129725 wait complete. 2024-04-12T20:43:56.432 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:55.477+0000 7fede6000700 1 Processor -- start 2024-04-12T20:43:56.432 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:55.477+0000 7fede6000700 1 -- start start 2024-04-12T20:43:56.432 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:55.478+0000 7fede6000700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fede007cfb0 0x7fede0117c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T20:43:56.432 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:55.478+0000 7fede6000700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fede01181b0 con 0x7fede007cfb0 2024-04-12T20:43:56.432 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:55.478+0000 7feddf400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fede007cfb0 0x7fede0117c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T20:43:56.432 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:55.478+0000 7feddf400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fede007cfb0 0x7fede0117c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:33438/0 (socket says 172.21.15.18:33438) 2024-04-12T20:43:56.432 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:55.478+0000 7feddf400700 1 -- 172.21.15.18:0/3519077577 learned_addr learned my addr 172.21.15.18:0/3519077577 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T20:43:56.432 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:55.478+0000 7feddf400700 1 -- 172.21.15.18:0/3519077577 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fedd00094f0 con 0x7fede007cfb0 2024-04-12T20:43:56.432 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:55.479+0000 7feddf400700 1 --2- 172.21.15.18:0/3519077577 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fede007cfb0 0x7fede0117c70 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fedd00090b0 tx=0x7fedd0009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T20:43:56.432 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:55.479+0000 7fedd7400700 1 -- 172.21.15.18:0/3519077577 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fedd0010840 con 0x7fede007cfb0 2024-04-12T20:43:56.432 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:55.479+0000 7fedd7400700 1 -- 172.21.15.18:0/3519077577 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fedd0010e70 con 0x7fede007cfb0 2024-04-12T20:43:56.432 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:55.479+0000 7fede6000700 1 -- 172.21.15.18:0/3519077577 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fede01183b0 con 0x7fede007cfb0 2024-04-12T20:43:56.432 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:55.479+0000 7fedd7400700 1 -- 172.21.15.18:0/3519077577 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fedd0024e40 con 0x7fede007cfb0 2024-04-12T20:43:56.432 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:55.479+0000 7fede6000700 1 -- 172.21.15.18:0/3519077577 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fede0118f20 con 0x7fede007cfb0 2024-04-12T20:43:56.433 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:55.481+0000 7fedd7400700 1 -- 172.21.15.18:0/3519077577 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 8) v1 ==== 43368+0+0 (secure 0 0 0) 0x7fedd00109a0 con 0x7fede007cfb0 2024-04-12T20:43:56.433 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:55.481+0000 7fede6000700 1 -- 172.21.15.18:0/3519077577 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fede005f5f0 con 0x7fede007cfb0 2024-04-12T20:43:56.433 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:55.481+0000 7fedd7400700 1 --2- 172.21.15.18:0/3519077577 >> [v2:172.21.15.18:6800/3403971131,v1:172.21.15.18:6801/3403971131] conn(0x7fedc4036790 0x7fedc4038c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T20:43:56.433 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:55.481+0000 7fedd7400700 1 -- 172.21.15.18:0/3519077577 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 936+0+0 (secure 0 0 0) 0x7fedd004a800 con 0x7fede007cfb0 2024-04-12T20:43:56.433 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:55.482+0000 7feddea00700 1 --2- 172.21.15.18:0/3519077577 >> [v2:172.21.15.18:6800/3403971131,v1:172.21.15.18:6801/3403971131] conn(0x7fedc4036790 0x7fedc4038c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T20:43:56.433 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:55.483+0000 7feddea00700 1 --2- 172.21.15.18:0/3519077577 >> [v2:172.21.15.18:6800/3403971131,v1:172.21.15.18:6801/3403971131] conn(0x7fedc4036790 0x7fedc4038c40 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fedcc009990 tx=0x7fedcc006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T20:43:56.433 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:55.486+0000 7fedd7400700 1 -- 172.21.15.18:0/3519077577 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7fedd002c520 con 0x7fede007cfb0 2024-04-12T20:43:56.433 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:55.715+0000 7fede6000700 1 -- 172.21.15.18:0/3519077577 --> [v2:172.21.15.18:6800/3403971131,v1:172.21.15.18:6801/3403971131] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "prometheus", "target": ["mon-mgr", ""]}) v1 -- 0x7fede007a790 con 0x7fedc4036790 2024-04-12T20:43:56.433 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:55.729+0000 7fedd7400700 1 -- 172.21.15.18:0/3519077577 <== mgr.14120 v2:172.21.15.18:6800/3403971131 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+31 (secure 0 0 0) 0x7fede007a790 con 0x7fedc4036790 2024-04-12T20:43:56.433 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:55.733+0000 7fede6000700 1 -- 172.21.15.18:0/3519077577 >> [v2:172.21.15.18:6800/3403971131,v1:172.21.15.18:6801/3403971131] conn(0x7fedc4036790 msgr2=0x7fedc4038c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:43:56.433 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:55.733+0000 7fede6000700 1 --2- 172.21.15.18:0/3519077577 >> [v2:172.21.15.18:6800/3403971131,v1:172.21.15.18:6801/3403971131] conn(0x7fedc4036790 0x7fedc4038c40 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fedcc009990 tx=0x7fedcc006e40 comp rx=0 tx=0).stop 2024-04-12T20:43:56.433 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:55.733+0000 7fede6000700 1 -- 172.21.15.18:0/3519077577 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fede007cfb0 msgr2=0x7fede0117c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:43:56.433 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:55.733+0000 7fede6000700 1 --2- 172.21.15.18:0/3519077577 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fede007cfb0 0x7fede0117c70 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fedd00090b0 tx=0x7fedd0009250 comp rx=0 tx=0).stop 2024-04-12T20:43:56.433 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:55.734+0000 7fede6000700 1 -- 172.21.15.18:0/3519077577 shutdown_connections 2024-04-12T20:43:56.433 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:55.734+0000 7fede6000700 1 --2- 172.21.15.18:0/3519077577 >> [v2:172.21.15.18:6800/3403971131,v1:172.21.15.18:6801/3403971131] conn(0x7fedc4036790 0x7fedc4038c40 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:43:56.434 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:55.734+0000 7fede6000700 1 --2- 172.21.15.18:0/3519077577 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fede007cfb0 0x7fede0117c70 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:43:56.434 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:55.734+0000 7fede6000700 1 -- 172.21.15.18:0/3519077577 >> 172.21.15.18:0/3519077577 conn(0x7fede0078530 msgr2=0x7fede007a080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T20:43:56.434 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:55.734+0000 7fede6000700 1 -- 172.21.15.18:0/3519077577 shutdown_connections 2024-04-12T20:43:56.434 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:55.734+0000 7fede6000700 1 -- 172.21.15.18:0/3519077577 wait complete. 2024-04-12T20:43:56.434 INFO:teuthology.orchestra.run.smithi018.stdout:Deploying grafana service with default placement... 2024-04-12T20:43:56.969 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:43:56 smithi018 ceph-mon[28799]: from='client.14148 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "prometheus", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T20:43:56.969 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:43:56 smithi018 ceph-mon[28799]: Saving service prometheus spec with placement count:1 2024-04-12T20:43:56.969 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:43:56 smithi018 ceph-mon[28799]: from='mgr.14120 172.21.15.18:0/4270481350' entity='mgr.smithi018.xjzwga' 2024-04-12T20:43:56.969 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:43:56 smithi018 ceph-mon[28799]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-12T20:43:56.969 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:43:56 smithi018 ceph-mon[28799]: from='mgr.14120 172.21.15.18:0/4270481350' entity='mgr.smithi018.xjzwga' 2024-04-12T20:43:58.345 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:43:58 smithi018 ceph-mon[28799]: from='client.14150 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "grafana", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T20:43:58.345 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:43:58 smithi018 ceph-mon[28799]: Saving service grafana spec with placement count:1 2024-04-12T20:43:58.345 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:43:58 smithi018 ceph-mon[28799]: from='mgr.14120 172.21.15.18:0/4270481350' entity='mgr.smithi018.xjzwga' 2024-04-12T20:43:58.361 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout Scheduled grafana update... 2024-04-12T20:43:58.362 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:56.970+0000 7fec03c00700 1 Processor -- start 2024-04-12T20:43:58.362 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:56.970+0000 7fec03c00700 1 -- start start 2024-04-12T20:43:58.362 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:56.970+0000 7fec03c00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7febfc071210 0x7febfc105580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T20:43:58.362 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:56.970+0000 7fec03c00700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7febfc071620 con 0x7febfc071210 2024-04-12T20:43:58.362 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:56.971+0000 7fec01200700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7febfc071210 0x7febfc105580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T20:43:58.362 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:56.971+0000 7fec01200700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7febfc071210 0x7febfc105580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:33454/0 (socket says 172.21.15.18:33454) 2024-04-12T20:43:58.362 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:56.971+0000 7fec01200700 1 -- 172.21.15.18:0/1679936239 learned_addr learned my addr 172.21.15.18:0/1679936239 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T20:43:58.362 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:56.972+0000 7fec01200700 1 -- 172.21.15.18:0/1679936239 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7febfc101360 con 0x7febfc071210 2024-04-12T20:43:58.362 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:56.972+0000 7fec01200700 1 --2- 172.21.15.18:0/1679936239 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7febfc071210 0x7febfc105580 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7febf00098e0 tx=0x7febf0009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d354a6a367c13f06 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T20:43:58.362 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:56.972+0000 7febfb400700 1 -- 172.21.15.18:0/1679936239 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7febf000b860 con 0x7febfc071210 2024-04-12T20:43:58.362 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:56.972+0000 7febfb400700 1 -- 172.21.15.18:0/1679936239 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7febf000be90 con 0x7febfc071210 2024-04-12T20:43:58.362 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:56.972+0000 7febfb400700 1 -- 172.21.15.18:0/1679936239 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7febf00103c0 con 0x7febfc071210 2024-04-12T20:43:58.363 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:56.973+0000 7fec03c00700 1 -- 172.21.15.18:0/1679936239 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7febfc071210 msgr2=0x7febfc105580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:43:58.363 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:56.973+0000 7fec03c00700 1 --2- 172.21.15.18:0/1679936239 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7febfc071210 0x7febfc105580 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7febf00098e0 tx=0x7febf0009250 comp rx=0 tx=0).stop 2024-04-12T20:43:58.363 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:56.973+0000 7fec03c00700 1 -- 172.21.15.18:0/1679936239 shutdown_connections 2024-04-12T20:43:58.363 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:56.973+0000 7fec03c00700 1 --2- 172.21.15.18:0/1679936239 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7febfc071210 0x7febfc105580 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:43:58.363 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:56.973+0000 7fec03c00700 1 -- 172.21.15.18:0/1679936239 >> 172.21.15.18:0/1679936239 conn(0x7febfc0fcfb0 msgr2=0x7febfc0ff3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T20:43:58.363 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:56.973+0000 7fec03c00700 1 -- 172.21.15.18:0/1679936239 shutdown_connections 2024-04-12T20:43:58.363 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:56.974+0000 7fec03c00700 1 -- 172.21.15.18:0/1679936239 wait complete. 2024-04-12T20:43:58.363 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:56.974+0000 7fec03c00700 1 Processor -- start 2024-04-12T20:43:58.363 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:56.975+0000 7fec03c00700 1 -- start start 2024-04-12T20:43:58.363 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:56.975+0000 7fec03c00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7febfc071210 0x7febfc113970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T20:43:58.363 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:56.975+0000 7fec03c00700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7febfc113eb0 con 0x7febfc071210 2024-04-12T20:43:58.363 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:56.975+0000 7fec01200700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7febfc071210 0x7febfc113970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T20:43:58.363 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:56.975+0000 7fec01200700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7febfc071210 0x7febfc113970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:33468/0 (socket says 172.21.15.18:33468) 2024-04-12T20:43:58.363 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:56.975+0000 7fec01200700 1 -- 172.21.15.18:0/673962321 learned_addr learned my addr 172.21.15.18:0/673962321 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T20:43:58.364 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:56.976+0000 7fec01200700 1 -- 172.21.15.18:0/673962321 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7febf00094f0 con 0x7febfc071210 2024-04-12T20:43:58.364 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:56.976+0000 7fec01200700 1 --2- 172.21.15.18:0/673962321 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7febfc071210 0x7febfc113970 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7febf00105b0 tx=0x7febf0009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T20:43:58.364 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:56.976+0000 7febf8c00700 1 -- 172.21.15.18:0/673962321 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7febf0010840 con 0x7febfc071210 2024-04-12T20:43:58.364 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:56.976+0000 7fec03c00700 1 -- 172.21.15.18:0/673962321 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7febfc1140b0 con 0x7febfc071210 2024-04-12T20:43:58.364 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:56.976+0000 7febf8c00700 1 -- 172.21.15.18:0/673962321 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7febf0010e70 con 0x7febfc071210 2024-04-12T20:43:58.364 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:56.976+0000 7febf8c00700 1 -- 172.21.15.18:0/673962321 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7febf001ae30 con 0x7febfc071210 2024-04-12T20:43:58.364 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:56.976+0000 7fec03c00700 1 -- 172.21.15.18:0/673962321 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7febfc114c20 con 0x7febfc071210 2024-04-12T20:43:58.364 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:56.978+0000 7febf8c00700 1 -- 172.21.15.18:0/673962321 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 8) v1 ==== 43368+0+0 (secure 0 0 0) 0x7febf001a430 con 0x7febfc071210 2024-04-12T20:43:58.364 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:56.978+0000 7febf8c00700 1 --2- 172.21.15.18:0/673962321 >> [v2:172.21.15.18:6800/3403971131,v1:172.21.15.18:6801/3403971131] conn(0x7febe4036790 0x7febe4038c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T20:43:58.364 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:56.978+0000 7febf8c00700 1 -- 172.21.15.18:0/673962321 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 936+0+0 (secure 0 0 0) 0x7febf004a1a0 con 0x7febfc071210 2024-04-12T20:43:58.364 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:56.979+0000 7febfbe00700 1 --2- 172.21.15.18:0/673962321 >> [v2:172.21.15.18:6800/3403971131,v1:172.21.15.18:6801/3403971131] conn(0x7febe4036790 0x7febe4038c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T20:43:58.364 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:56.979+0000 7fec03c00700 1 -- 172.21.15.18:0/673962321 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7febfc05f5f0 con 0x7febfc071210 2024-04-12T20:43:58.364 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:56.979+0000 7febfbe00700 1 --2- 172.21.15.18:0/673962321 >> [v2:172.21.15.18:6800/3403971131,v1:172.21.15.18:6801/3403971131] conn(0x7febe4036790 0x7febe4038c40 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7febe8009990 tx=0x7febe8006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T20:43:58.364 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:56.984+0000 7febf8c00700 1 -- 172.21.15.18:0/673962321 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7febf002b4d0 con 0x7febfc071210 2024-04-12T20:43:58.365 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:57.229+0000 7fec03c00700 1 -- 172.21.15.18:0/673962321 --> [v2:172.21.15.18:6800/3403971131,v1:172.21.15.18:6801/3403971131] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "grafana", "target": ["mon-mgr", ""]}) v1 -- 0x7febfc114f20 con 0x7febe4036790 2024-04-12T20:43:58.365 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:57.237+0000 7febf8c00700 1 -- 172.21.15.18:0/673962321 <== mgr.14120 v2:172.21.15.18:6800/3403971131 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+28 (secure 0 0 0) 0x7febfc114f20 con 0x7febe4036790 2024-04-12T20:43:58.365 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:57.240+0000 7fec03c00700 1 -- 172.21.15.18:0/673962321 >> [v2:172.21.15.18:6800/3403971131,v1:172.21.15.18:6801/3403971131] conn(0x7febe4036790 msgr2=0x7febe4038c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:43:58.365 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:57.240+0000 7fec03c00700 1 --2- 172.21.15.18:0/673962321 >> [v2:172.21.15.18:6800/3403971131,v1:172.21.15.18:6801/3403971131] conn(0x7febe4036790 0x7febe4038c40 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7febe8009990 tx=0x7febe8006e40 comp rx=0 tx=0).stop 2024-04-12T20:43:58.365 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:57.240+0000 7fec03c00700 1 -- 172.21.15.18:0/673962321 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7febfc071210 msgr2=0x7febfc113970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:43:58.365 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:57.240+0000 7fec03c00700 1 --2- 172.21.15.18:0/673962321 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7febfc071210 0x7febfc113970 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7febf00105b0 tx=0x7febf0009250 comp rx=0 tx=0).stop 2024-04-12T20:43:58.365 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:57.240+0000 7fec03c00700 1 -- 172.21.15.18:0/673962321 shutdown_connections 2024-04-12T20:43:58.365 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:57.240+0000 7fec03c00700 1 --2- 172.21.15.18:0/673962321 >> [v2:172.21.15.18:6800/3403971131,v1:172.21.15.18:6801/3403971131] conn(0x7febe4036790 0x7febe4038c40 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:43:58.365 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:57.240+0000 7fec03c00700 1 --2- 172.21.15.18:0/673962321 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7febfc071210 0x7febfc113970 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:43:58.365 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:57.240+0000 7fec03c00700 1 -- 172.21.15.18:0/673962321 >> 172.21.15.18:0/673962321 conn(0x7febfc0fcfb0 msgr2=0x7febfc1031a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T20:43:58.365 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:57.240+0000 7fec03c00700 1 -- 172.21.15.18:0/673962321 shutdown_connections 2024-04-12T20:43:58.365 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:57.240+0000 7fec03c00700 1 -- 172.21.15.18:0/673962321 wait complete. 2024-04-12T20:43:58.365 INFO:teuthology.orchestra.run.smithi018.stdout:Deploying node-exporter service with default placement... 2024-04-12T20:43:59.342 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:43:59 smithi018 ceph-mon[28799]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-12T20:44:00.346 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout Scheduled node-exporter update... 2024-04-12T20:44:00.346 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:59.376+0000 7fa0c2a00700 1 Processor -- start 2024-04-12T20:44:00.346 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:59.376+0000 7fa0c2a00700 1 -- start start 2024-04-12T20:44:00.346 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:59.376+0000 7fa0c2a00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa0bc07ce60 0x7fa0bc10bcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T20:44:00.346 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:59.376+0000 7fa0c2a00700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0bc07d300 con 0x7fa0bc07ce60 2024-04-12T20:44:00.346 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:59.377+0000 7fa0c1600700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa0bc07ce60 0x7fa0bc10bcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T20:44:00.346 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:59.377+0000 7fa0c1600700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa0bc07ce60 0x7fa0bc10bcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:33478/0 (socket says 172.21.15.18:33478) 2024-04-12T20:44:00.346 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:59.377+0000 7fa0c1600700 1 -- 172.21.15.18:0/70706285 learned_addr learned my addr 172.21.15.18:0/70706285 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T20:44:00.346 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:59.377+0000 7fa0c1600700 1 -- 172.21.15.18:0/70706285 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0bc07d440 con 0x7fa0bc07ce60 2024-04-12T20:44:00.347 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:59.378+0000 7fa0c1600700 1 --2- 172.21.15.18:0/70706285 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa0bc07ce60 0x7fa0bc10bcd0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fa0b00098e0 tx=0x7fa0b0009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fec59b1a43813ca1 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T20:44:00.347 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:59.378+0000 7fa0bbe00700 1 -- 172.21.15.18:0/70706285 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fa0b000b860 con 0x7fa0bc07ce60 2024-04-12T20:44:00.347 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:59.378+0000 7fa0bbe00700 1 -- 172.21.15.18:0/70706285 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa0b000be90 con 0x7fa0bc07ce60 2024-04-12T20:44:00.347 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:59.378+0000 7fa0bbe00700 1 -- 172.21.15.18:0/70706285 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fa0b00103c0 con 0x7fa0bc07ce60 2024-04-12T20:44:00.347 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:59.378+0000 7fa0c2a00700 1 -- 172.21.15.18:0/70706285 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa0bc07ce60 msgr2=0x7fa0bc10bcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:44:00.347 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:59.378+0000 7fa0c2a00700 1 --2- 172.21.15.18:0/70706285 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa0bc07ce60 0x7fa0bc10bcd0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fa0b00098e0 tx=0x7fa0b0009250 comp rx=0 tx=0).stop 2024-04-12T20:44:00.347 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:59.379+0000 7fa0c2a00700 1 -- 172.21.15.18:0/70706285 shutdown_connections 2024-04-12T20:44:00.347 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:59.379+0000 7fa0c2a00700 1 --2- 172.21.15.18:0/70706285 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa0bc07ce60 0x7fa0bc10bcd0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:44:00.347 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:59.379+0000 7fa0c2a00700 1 -- 172.21.15.18:0/70706285 >> 172.21.15.18:0/70706285 conn(0x7fa0bc0782f0 msgr2=0x7fa0bc07a700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T20:44:00.347 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:59.379+0000 7fa0c2a00700 1 -- 172.21.15.18:0/70706285 shutdown_connections 2024-04-12T20:44:00.347 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:59.379+0000 7fa0c2a00700 1 -- 172.21.15.18:0/70706285 wait complete. 2024-04-12T20:44:00.347 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:59.380+0000 7fa0c2a00700 1 Processor -- start 2024-04-12T20:44:00.347 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:59.380+0000 7fa0c2a00700 1 -- start start 2024-04-12T20:44:00.348 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:59.380+0000 7fa0c2a00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa0bc07ce60 0x7fa0bc117ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T20:44:00.348 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:59.380+0000 7fa0c2a00700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0bc118000 con 0x7fa0bc07ce60 2024-04-12T20:44:00.348 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:59.380+0000 7fa0c1600700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa0bc07ce60 0x7fa0bc117ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T20:44:00.348 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:59.380+0000 7fa0c1600700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa0bc07ce60 0x7fa0bc117ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:33490/0 (socket says 172.21.15.18:33490) 2024-04-12T20:44:00.348 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:59.380+0000 7fa0c1600700 1 -- 172.21.15.18:0/1508791145 learned_addr learned my addr 172.21.15.18:0/1508791145 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T20:44:00.348 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:59.381+0000 7fa0c1600700 1 -- 172.21.15.18:0/1508791145 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0b00094f0 con 0x7fa0bc07ce60 2024-04-12T20:44:00.348 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:59.381+0000 7fa0c1600700 1 --2- 172.21.15.18:0/1508791145 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa0bc07ce60 0x7fa0bc117ac0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fa0b0003080 tx=0x7fa0b0009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T20:44:00.348 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:59.381+0000 7fa0b9600700 1 -- 172.21.15.18:0/1508791145 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fa0b0011920 con 0x7fa0bc07ce60 2024-04-12T20:44:00.348 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:59.381+0000 7fa0c2a00700 1 -- 172.21.15.18:0/1508791145 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa0bc118200 con 0x7fa0bc07ce60 2024-04-12T20:44:00.348 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:59.381+0000 7fa0b9600700 1 -- 172.21.15.18:0/1508791145 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa0b0010d10 con 0x7fa0bc07ce60 2024-04-12T20:44:00.348 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:59.381+0000 7fa0b9600700 1 -- 172.21.15.18:0/1508791145 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fa0b001ae50 con 0x7fa0bc07ce60 2024-04-12T20:44:00.348 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:59.382+0000 7fa0c2a00700 1 -- 172.21.15.18:0/1508791145 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa0bc118d70 con 0x7fa0bc07ce60 2024-04-12T20:44:00.348 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:59.383+0000 7fa0b9600700 1 -- 172.21.15.18:0/1508791145 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 8) v1 ==== 43368+0+0 (secure 0 0 0) 0x7fa0b001a430 con 0x7fa0bc07ce60 2024-04-12T20:44:00.348 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:59.383+0000 7fa0c2a00700 1 -- 172.21.15.18:0/1508791145 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa0bc05f5f0 con 0x7fa0bc07ce60 2024-04-12T20:44:00.348 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:59.383+0000 7fa0b9600700 1 --2- 172.21.15.18:0/1508791145 >> [v2:172.21.15.18:6800/3403971131,v1:172.21.15.18:6801/3403971131] conn(0x7fa0a4036740 0x7fa0a4038bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T20:44:00.349 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:59.383+0000 7fa0b9600700 1 -- 172.21.15.18:0/1508791145 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 936+0+0 (secure 0 0 0) 0x7fa0b004a640 con 0x7fa0bc07ce60 2024-04-12T20:44:00.349 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:59.383+0000 7fa0c0c00700 1 --2- 172.21.15.18:0/1508791145 >> [v2:172.21.15.18:6800/3403971131,v1:172.21.15.18:6801/3403971131] conn(0x7fa0a4036740 0x7fa0a4038bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T20:44:00.349 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:59.384+0000 7fa0c0c00700 1 --2- 172.21.15.18:0/1508791145 >> [v2:172.21.15.18:6800/3403971131,v1:172.21.15.18:6801/3403971131] conn(0x7fa0a4036740 0x7fa0a4038bf0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fa0a8009990 tx=0x7fa0a8006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T20:44:00.349 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:59.389+0000 7fa0b9600700 1 -- 172.21.15.18:0/1508791145 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7fa0b0010e80 con 0x7fa0bc07ce60 2024-04-12T20:44:00.349 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:59.617+0000 7fa0c2a00700 1 -- 172.21.15.18:0/1508791145 --> [v2:172.21.15.18:6800/3403971131,v1:172.21.15.18:6801/3403971131] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "node-exporter", "target": ["mon-mgr", ""]}) v1 -- 0x7fa0bc07a540 con 0x7fa0a4036740 2024-04-12T20:44:00.349 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:59.624+0000 7fa0b9600700 1 -- 172.21.15.18:0/1508791145 <== mgr.14120 v2:172.21.15.18:6800/3403971131 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+34 (secure 0 0 0) 0x7fa0bc07a540 con 0x7fa0a4036740 2024-04-12T20:44:00.349 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:59.627+0000 7fa0c2a00700 1 -- 172.21.15.18:0/1508791145 >> [v2:172.21.15.18:6800/3403971131,v1:172.21.15.18:6801/3403971131] conn(0x7fa0a4036740 msgr2=0x7fa0a4038bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:44:00.349 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:59.627+0000 7fa0c2a00700 1 --2- 172.21.15.18:0/1508791145 >> [v2:172.21.15.18:6800/3403971131,v1:172.21.15.18:6801/3403971131] conn(0x7fa0a4036740 0x7fa0a4038bf0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fa0a8009990 tx=0x7fa0a8006e40 comp rx=0 tx=0).stop 2024-04-12T20:44:00.349 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:59.627+0000 7fa0c2a00700 1 -- 172.21.15.18:0/1508791145 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa0bc07ce60 msgr2=0x7fa0bc117ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:44:00.349 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:59.627+0000 7fa0c2a00700 1 --2- 172.21.15.18:0/1508791145 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa0bc07ce60 0x7fa0bc117ac0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fa0b0003080 tx=0x7fa0b0009250 comp rx=0 tx=0).stop 2024-04-12T20:44:00.349 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:59.627+0000 7fa0c2a00700 1 -- 172.21.15.18:0/1508791145 shutdown_connections 2024-04-12T20:44:00.349 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:59.627+0000 7fa0c2a00700 1 --2- 172.21.15.18:0/1508791145 >> [v2:172.21.15.18:6800/3403971131,v1:172.21.15.18:6801/3403971131] conn(0x7fa0a4036740 0x7fa0a4038bf0 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:44:00.349 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:59.627+0000 7fa0c2a00700 1 --2- 172.21.15.18:0/1508791145 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa0bc07ce60 0x7fa0bc117ac0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:44:00.350 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:59.627+0000 7fa0c2a00700 1 -- 172.21.15.18:0/1508791145 >> 172.21.15.18:0/1508791145 conn(0x7fa0bc0782f0 msgr2=0x7fa0bc079e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T20:44:00.350 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:59.627+0000 7fa0c2a00700 1 -- 172.21.15.18:0/1508791145 shutdown_connections 2024-04-12T20:44:00.350 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:43:59.627+0000 7fa0c2a00700 1 -- 172.21.15.18:0/1508791145 wait complete. 2024-04-12T20:44:00.350 INFO:teuthology.orchestra.run.smithi018.stdout:Deploying alertmanager service with default placement... 2024-04-12T20:44:00.853 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:44:00 smithi018 ceph-mon[28799]: from='client.14152 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "node-exporter", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T20:44:00.853 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:44:00 smithi018 ceph-mon[28799]: Saving service node-exporter spec with placement * 2024-04-12T20:44:00.853 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:44:00 smithi018 ceph-mon[28799]: from='mgr.14120 172.21.15.18:0/4270481350' entity='mgr.smithi018.xjzwga' 2024-04-12T20:44:01.834 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:44:01 smithi018 ceph-mon[28799]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-12T20:44:01.834 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:44:01 smithi018 ceph-mon[28799]: from='mgr.14120 172.21.15.18:0/4270481350' entity='mgr.smithi018.xjzwga' 2024-04-12T20:44:01.848 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout Scheduled alertmanager update... 2024-04-12T20:44:01.852 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:00.915+0000 7fbca1600700 1 Processor -- start 2024-04-12T20:44:01.852 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:00.915+0000 7fbca1600700 1 -- start start 2024-04-12T20:44:01.852 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:00.916+0000 7fbca1600700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fbc9c078c90 0x7fbc9c077290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T20:44:01.852 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:00.916+0000 7fbca1600700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc9c0790a0 con 0x7fbc9c078c90 2024-04-12T20:44:01.852 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:00.916+0000 7fbc9aa00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fbc9c078c90 0x7fbc9c077290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T20:44:01.852 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:00.916+0000 7fbc9aa00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fbc9c078c90 0x7fbc9c077290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:38116/0 (socket says 172.21.15.18:38116) 2024-04-12T20:44:01.852 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:00.916+0000 7fbc9aa00700 1 -- 172.21.15.18:0/2793299076 learned_addr learned my addr 172.21.15.18:0/2793299076 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T20:44:01.852 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:00.917+0000 7fbc9aa00700 1 -- 172.21.15.18:0/2793299076 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc9c0777d0 con 0x7fbc9c078c90 2024-04-12T20:44:01.852 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:00.917+0000 7fbc9aa00700 1 --2- 172.21.15.18:0/2793299076 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fbc9c078c90 0x7fbc9c077290 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fbc8c0098e0 tx=0x7fbc8c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce5c756cf6d41cac server_cookie=0 in_seq=0 out_seq=0 2024-04-12T20:44:01.852 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:00.917+0000 7fbc99600700 1 -- 172.21.15.18:0/2793299076 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fbc8c00b860 con 0x7fbc9c078c90 2024-04-12T20:44:01.853 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:00.917+0000 7fbc99600700 1 -- 172.21.15.18:0/2793299076 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fbc8c00be90 con 0x7fbc9c078c90 2024-04-12T20:44:01.853 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:00.917+0000 7fbc99600700 1 -- 172.21.15.18:0/2793299076 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fbc8c0103c0 con 0x7fbc9c078c90 2024-04-12T20:44:01.853 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:00.918+0000 7fbca1600700 1 -- 172.21.15.18:0/2793299076 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fbc9c078c90 msgr2=0x7fbc9c077290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:44:01.853 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:00.918+0000 7fbca1600700 1 --2- 172.21.15.18:0/2793299076 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fbc9c078c90 0x7fbc9c077290 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fbc8c0098e0 tx=0x7fbc8c009250 comp rx=0 tx=0).stop 2024-04-12T20:44:01.853 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:00.918+0000 7fbca1600700 1 -- 172.21.15.18:0/2793299076 shutdown_connections 2024-04-12T20:44:01.853 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:00.918+0000 7fbca1600700 1 --2- 172.21.15.18:0/2793299076 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fbc9c078c90 0x7fbc9c077290 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:44:01.853 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:00.918+0000 7fbca1600700 1 -- 172.21.15.18:0/2793299076 >> 172.21.15.18:0/2793299076 conn(0x7fbc9c0fe070 msgr2=0x7fbc9c100480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T20:44:01.853 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:00.919+0000 7fbca1600700 1 -- 172.21.15.18:0/2793299076 shutdown_connections 2024-04-12T20:44:01.853 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:00.919+0000 7fbca1600700 1 -- 172.21.15.18:0/2793299076 wait complete. 2024-04-12T20:44:01.853 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:00.919+0000 7fbca1600700 1 Processor -- start 2024-04-12T20:44:01.853 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:00.920+0000 7fbca1600700 1 -- start start 2024-04-12T20:44:01.853 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:00.920+0000 7fbca1600700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fbc9c078c90 0x7fbc9c117e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T20:44:01.853 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:00.920+0000 7fbca1600700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc9c118340 con 0x7fbc9c078c90 2024-04-12T20:44:01.853 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:00.920+0000 7fbc9aa00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fbc9c078c90 0x7fbc9c117e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T20:44:01.853 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:00.920+0000 7fbc9aa00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fbc9c078c90 0x7fbc9c117e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:38118/0 (socket says 172.21.15.18:38118) 2024-04-12T20:44:01.854 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:00.920+0000 7fbc9aa00700 1 -- 172.21.15.18:0/3810573752 learned_addr learned my addr 172.21.15.18:0/3810573752 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T20:44:01.856 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:00.921+0000 7fbc9aa00700 1 -- 172.21.15.18:0/3810573752 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc8c0094f0 con 0x7fbc9c078c90 2024-04-12T20:44:01.856 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:00.921+0000 7fbc9aa00700 1 --2- 172.21.15.18:0/3810573752 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fbc9c078c90 0x7fbc9c117e00 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fbc8c006fd0 tx=0x7fbc8c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T20:44:01.857 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:00.921+0000 7fbc92a00700 1 -- 172.21.15.18:0/3810573752 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fbc8c010630 con 0x7fbc9c078c90 2024-04-12T20:44:01.857 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:00.921+0000 7fbc92a00700 1 -- 172.21.15.18:0/3810573752 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fbc8c010c60 con 0x7fbc9c078c90 2024-04-12T20:44:01.857 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:00.921+0000 7fbca1600700 1 -- 172.21.15.18:0/3810573752 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbc9c118540 con 0x7fbc9c078c90 2024-04-12T20:44:01.857 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:00.921+0000 7fbc92a00700 1 -- 172.21.15.18:0/3810573752 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fbc8c01ace0 con 0x7fbc9c078c90 2024-04-12T20:44:01.857 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:00.921+0000 7fbca1600700 1 -- 172.21.15.18:0/3810573752 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbc9c118f70 con 0x7fbc9c078c90 2024-04-12T20:44:01.857 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:00.922+0000 7fbc92a00700 1 -- 172.21.15.18:0/3810573752 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 8) v1 ==== 43368+0+0 (secure 0 0 0) 0x7fbc8c010790 con 0x7fbc9c078c90 2024-04-12T20:44:01.857 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:00.922+0000 7fbca1600700 1 -- 172.21.15.18:0/3810573752 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbc9c05f5f0 con 0x7fbc9c078c90 2024-04-12T20:44:01.857 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:00.923+0000 7fbc92a00700 1 --2- 172.21.15.18:0/3810573752 >> [v2:172.21.15.18:6800/3403971131,v1:172.21.15.18:6801/3403971131] conn(0x7fbc80036790 0x7fbc80038c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T20:44:01.857 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:00.923+0000 7fbc92a00700 1 -- 172.21.15.18:0/3810573752 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 936+0+0 (secure 0 0 0) 0x7fbc8c01e070 con 0x7fbc9c078c90 2024-04-12T20:44:01.857 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:00.923+0000 7fbc9a000700 1 --2- 172.21.15.18:0/3810573752 >> [v2:172.21.15.18:6800/3403971131,v1:172.21.15.18:6801/3403971131] conn(0x7fbc80036790 0x7fbc80038c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T20:44:01.857 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:00.924+0000 7fbc9a000700 1 --2- 172.21.15.18:0/3810573752 >> [v2:172.21.15.18:6800/3403971131,v1:172.21.15.18:6801/3403971131] conn(0x7fbc80036790 0x7fbc80038c40 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fbc88009990 tx=0x7fbc88006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T20:44:01.857 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:00.928+0000 7fbc92a00700 1 -- 172.21.15.18:0/3810573752 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7fbc8c010dd0 con 0x7fbc9c078c90 2024-04-12T20:44:01.857 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:01.157+0000 7fbca1600700 1 -- 172.21.15.18:0/3810573752 --> [v2:172.21.15.18:6800/3403971131,v1:172.21.15.18:6801/3403971131] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "alertmanager", "target": ["mon-mgr", ""]}) v1 -- 0x7fbc9c119c80 con 0x7fbc80036790 2024-04-12T20:44:01.857 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:01.166+0000 7fbc92a00700 1 -- 172.21.15.18:0/3810573752 <== mgr.14120 v2:172.21.15.18:6800/3403971131 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+33 (secure 0 0 0) 0x7fbc9c119c80 con 0x7fbc80036790 2024-04-12T20:44:01.858 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:01.168+0000 7fbca1600700 1 -- 172.21.15.18:0/3810573752 >> [v2:172.21.15.18:6800/3403971131,v1:172.21.15.18:6801/3403971131] conn(0x7fbc80036790 msgr2=0x7fbc80038c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:44:01.858 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:01.168+0000 7fbca1600700 1 --2- 172.21.15.18:0/3810573752 >> [v2:172.21.15.18:6800/3403971131,v1:172.21.15.18:6801/3403971131] conn(0x7fbc80036790 0x7fbc80038c40 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fbc88009990 tx=0x7fbc88006e40 comp rx=0 tx=0).stop 2024-04-12T20:44:01.858 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:01.168+0000 7fbca1600700 1 -- 172.21.15.18:0/3810573752 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fbc9c078c90 msgr2=0x7fbc9c117e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:44:01.858 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:01.168+0000 7fbca1600700 1 --2- 172.21.15.18:0/3810573752 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fbc9c078c90 0x7fbc9c117e00 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fbc8c006fd0 tx=0x7fbc8c009250 comp rx=0 tx=0).stop 2024-04-12T20:44:01.858 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:01.169+0000 7fbca1600700 1 -- 172.21.15.18:0/3810573752 shutdown_connections 2024-04-12T20:44:01.858 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:01.169+0000 7fbca1600700 1 --2- 172.21.15.18:0/3810573752 >> [v2:172.21.15.18:6800/3403971131,v1:172.21.15.18:6801/3403971131] conn(0x7fbc80036790 0x7fbc80038c40 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:44:01.858 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:01.169+0000 7fbca1600700 1 --2- 172.21.15.18:0/3810573752 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fbc9c078c90 0x7fbc9c117e00 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:44:01.858 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:01.169+0000 7fbca1600700 1 -- 172.21.15.18:0/3810573752 >> 172.21.15.18:0/3810573752 conn(0x7fbc9c0fe070 msgr2=0x7fbc9c10f020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T20:44:01.858 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:01.169+0000 7fbca1600700 1 -- 172.21.15.18:0/3810573752 shutdown_connections 2024-04-12T20:44:01.858 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:01.169+0000 7fbca1600700 1 -- 172.21.15.18:0/3810573752 wait complete. 2024-04-12T20:44:02.719 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:44:02 smithi018 ceph-mon[28799]: from='client.14154 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "alertmanager", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T20:44:02.720 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:44:02 smithi018 ceph-mon[28799]: Saving service alertmanager spec with placement count:1 2024-04-12T20:44:03.409 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:02.493+0000 7fc3d1400700 1 Processor -- start 2024-04-12T20:44:03.409 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:02.493+0000 7fc3d1400700 1 -- start start 2024-04-12T20:44:03.409 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:02.493+0000 7fc3d1400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc3cc07cde0 0x7fc3cc10bcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T20:44:03.409 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:02.494+0000 7fc3d1400700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3cc07d280 con 0x7fc3cc07cde0 2024-04-12T20:44:03.409 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:02.494+0000 7fc3cbe00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc3cc07cde0 0x7fc3cc10bcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T20:44:03.409 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:02.494+0000 7fc3cbe00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc3cc07cde0 0x7fc3cc10bcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:38134/0 (socket says 172.21.15.18:38134) 2024-04-12T20:44:03.409 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:02.494+0000 7fc3cbe00700 1 -- 172.21.15.18:0/4095127584 learned_addr learned my addr 172.21.15.18:0/4095127584 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T20:44:03.409 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:02.495+0000 7fc3cbe00700 1 -- 172.21.15.18:0/4095127584 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3cc07d3c0 con 0x7fc3cc07cde0 2024-04-12T20:44:03.409 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:02.495+0000 7fc3cbe00700 1 --2- 172.21.15.18:0/4095127584 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc3cc07cde0 0x7fc3cc10bcd0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fc3bc0098e0 tx=0x7fc3bc009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=48342ef7131f490f server_cookie=0 in_seq=0 out_seq=0 2024-04-12T20:44:03.409 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:02.495+0000 7fc3caa00700 1 -- 172.21.15.18:0/4095127584 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fc3bc00b860 con 0x7fc3cc07cde0 2024-04-12T20:44:03.409 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:02.495+0000 7fc3caa00700 1 -- 172.21.15.18:0/4095127584 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc3bc00be90 con 0x7fc3cc07cde0 2024-04-12T20:44:03.409 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:02.495+0000 7fc3caa00700 1 -- 172.21.15.18:0/4095127584 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fc3bc0103c0 con 0x7fc3cc07cde0 2024-04-12T20:44:03.410 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:02.496+0000 7fc3d1400700 1 -- 172.21.15.18:0/4095127584 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc3cc07cde0 msgr2=0x7fc3cc10bcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:44:03.410 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:02.496+0000 7fc3d1400700 1 --2- 172.21.15.18:0/4095127584 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc3cc07cde0 0x7fc3cc10bcd0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fc3bc0098e0 tx=0x7fc3bc009250 comp rx=0 tx=0).stop 2024-04-12T20:44:03.410 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:02.496+0000 7fc3d1400700 1 -- 172.21.15.18:0/4095127584 shutdown_connections 2024-04-12T20:44:03.410 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:02.496+0000 7fc3d1400700 1 --2- 172.21.15.18:0/4095127584 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc3cc07cde0 0x7fc3cc10bcd0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:44:03.410 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:02.496+0000 7fc3d1400700 1 -- 172.21.15.18:0/4095127584 >> 172.21.15.18:0/4095127584 conn(0x7fc3cc078320 msgr2=0x7fc3cc07a730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T20:44:03.410 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:02.496+0000 7fc3d1400700 1 -- 172.21.15.18:0/4095127584 shutdown_connections 2024-04-12T20:44:03.410 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:02.496+0000 7fc3d1400700 1 -- 172.21.15.18:0/4095127584 wait complete. 2024-04-12T20:44:03.410 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:02.497+0000 7fc3d1400700 1 Processor -- start 2024-04-12T20:44:03.410 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:02.497+0000 7fc3d1400700 1 -- start start 2024-04-12T20:44:03.410 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:02.498+0000 7fc3d1400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc3cc07cde0 0x7fc3cc117ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T20:44:03.410 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:02.498+0000 7fc3d1400700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3cc118000 con 0x7fc3cc07cde0 2024-04-12T20:44:03.410 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:02.498+0000 7fc3cbe00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc3cc07cde0 0x7fc3cc117ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T20:44:03.410 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:02.498+0000 7fc3cbe00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc3cc07cde0 0x7fc3cc117ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:38142/0 (socket says 172.21.15.18:38142) 2024-04-12T20:44:03.410 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:02.498+0000 7fc3cbe00700 1 -- 172.21.15.18:0/3254869481 learned_addr learned my addr 172.21.15.18:0/3254869481 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T20:44:03.411 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:02.498+0000 7fc3cbe00700 1 -- 172.21.15.18:0/3254869481 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3bc0094f0 con 0x7fc3cc07cde0 2024-04-12T20:44:03.411 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:02.499+0000 7fc3cbe00700 1 --2- 172.21.15.18:0/3254869481 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc3cc07cde0 0x7fc3cc117ac0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fc3bc006d40 tx=0x7fc3bc009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T20:44:03.411 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:02.499+0000 7fc3c3e00700 1 -- 172.21.15.18:0/3254869481 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fc3bc010840 con 0x7fc3cc07cde0 2024-04-12T20:44:03.411 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:02.499+0000 7fc3c3e00700 1 -- 172.21.15.18:0/3254869481 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc3bc010e70 con 0x7fc3cc07cde0 2024-04-12T20:44:03.411 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:02.499+0000 7fc3d1400700 1 -- 172.21.15.18:0/3254869481 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc3cc118200 con 0x7fc3cc07cde0 2024-04-12T20:44:03.411 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:02.499+0000 7fc3c3e00700 1 -- 172.21.15.18:0/3254869481 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fc3bc024ea0 con 0x7fc3cc07cde0 2024-04-12T20:44:03.411 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:02.499+0000 7fc3d1400700 1 -- 172.21.15.18:0/3254869481 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc3cc118d70 con 0x7fc3cc07cde0 2024-04-12T20:44:03.411 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:02.500+0000 7fc3c3e00700 1 -- 172.21.15.18:0/3254869481 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 8) v1 ==== 43368+0+0 (secure 0 0 0) 0x7fc3bc024780 con 0x7fc3cc07cde0 2024-04-12T20:44:03.411 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:02.500+0000 7fc3d1400700 1 -- 172.21.15.18:0/3254869481 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc3cc05f5f0 con 0x7fc3cc07cde0 2024-04-12T20:44:03.411 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:02.501+0000 7fc3c3e00700 1 --2- 172.21.15.18:0/3254869481 >> [v2:172.21.15.18:6800/3403971131,v1:172.21.15.18:6801/3403971131] conn(0x7fc3b00367e0 0x7fc3b0038c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T20:44:03.411 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:02.501+0000 7fc3c3e00700 1 -- 172.21.15.18:0/3254869481 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 936+0+0 (secure 0 0 0) 0x7fc3bc04a740 con 0x7fc3cc07cde0 2024-04-12T20:44:03.411 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:02.501+0000 7fc3cb400700 1 --2- 172.21.15.18:0/3254869481 >> [v2:172.21.15.18:6800/3403971131,v1:172.21.15.18:6801/3403971131] conn(0x7fc3b00367e0 0x7fc3b0038c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T20:44:03.411 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:02.502+0000 7fc3cb400700 1 --2- 172.21.15.18:0/3254869481 >> [v2:172.21.15.18:6800/3403971131,v1:172.21.15.18:6801/3403971131] conn(0x7fc3b00367e0 0x7fc3b0038c90 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fc3b8009990 tx=0x7fc3b8006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T20:44:03.411 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:02.506+0000 7fc3c3e00700 1 -- 172.21.15.18:0/3254869481 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7fc3bc02c930 con 0x7fc3cc07cde0 2024-04-12T20:44:03.411 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:02.720+0000 7fc3d1400700 1 -- 172.21.15.18:0/3254869481 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/container_init}] v 0) v1 -- 0x7fc3cc04bed0 con 0x7fc3cc07cde0 2024-04-12T20:44:03.412 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:02.729+0000 7fc3c3e00700 1 -- 172.21.15.18:0/3254869481 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/container_init}]=0 v7)=0 v7) v1 ==== 142+0+0 (secure 0 0 0) 0x7fc3bc02c340 con 0x7fc3cc07cde0 2024-04-12T20:44:03.412 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:02.732+0000 7fc3d1400700 1 -- 172.21.15.18:0/3254869481 >> [v2:172.21.15.18:6800/3403971131,v1:172.21.15.18:6801/3403971131] conn(0x7fc3b00367e0 msgr2=0x7fc3b0038c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:44:03.412 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:02.732+0000 7fc3d1400700 1 --2- 172.21.15.18:0/3254869481 >> [v2:172.21.15.18:6800/3403971131,v1:172.21.15.18:6801/3403971131] conn(0x7fc3b00367e0 0x7fc3b0038c90 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fc3b8009990 tx=0x7fc3b8006e40 comp rx=0 tx=0).stop 2024-04-12T20:44:03.412 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:02.732+0000 7fc3d1400700 1 -- 172.21.15.18:0/3254869481 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc3cc07cde0 msgr2=0x7fc3cc117ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:44:03.412 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:02.732+0000 7fc3d1400700 1 --2- 172.21.15.18:0/3254869481 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc3cc07cde0 0x7fc3cc117ac0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fc3bc006d40 tx=0x7fc3bc009250 comp rx=0 tx=0).stop 2024-04-12T20:44:03.412 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:02.732+0000 7fc3d1400700 1 -- 172.21.15.18:0/3254869481 shutdown_connections 2024-04-12T20:44:03.412 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:02.732+0000 7fc3d1400700 1 --2- 172.21.15.18:0/3254869481 >> [v2:172.21.15.18:6800/3403971131,v1:172.21.15.18:6801/3403971131] conn(0x7fc3b00367e0 0x7fc3b0038c90 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:44:03.412 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:02.732+0000 7fc3d1400700 1 --2- 172.21.15.18:0/3254869481 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc3cc07cde0 0x7fc3cc117ac0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:44:03.412 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:02.732+0000 7fc3d1400700 1 -- 172.21.15.18:0/3254869481 >> 172.21.15.18:0/3254869481 conn(0x7fc3cc078320 msgr2=0x7fc3cc079e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T20:44:03.412 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:02.733+0000 7fc3d1400700 1 -- 172.21.15.18:0/3254869481 shutdown_connections 2024-04-12T20:44:03.412 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:02.733+0000 7fc3d1400700 1 -- 172.21.15.18:0/3254869481 wait complete. 2024-04-12T20:44:03.698 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:44:03 smithi018 ceph-mon[28799]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-12T20:44:03.698 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:44:03 smithi018 ceph-mon[28799]: from='client.? 172.21.15.18:0/3254869481' entity='client.admin' 2024-04-12T20:44:04.899 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:04.002+0000 7f08d4800700 1 Processor -- start 2024-04-12T20:44:04.899 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:04.003+0000 7f08d4800700 1 -- start start 2024-04-12T20:44:04.899 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:04.003+0000 7f08d4800700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f08cc104bd0 0x7f08cc104fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T20:44:04.899 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:04.003+0000 7f08d4800700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08cc1055b0 con 0x7f08cc104bd0 2024-04-12T20:44:04.900 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:04.004+0000 7f08d1e00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f08cc104bd0 0x7f08cc104fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T20:44:04.900 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:04.004+0000 7f08d1e00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f08cc104bd0 0x7f08cc104fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:38148/0 (socket says 172.21.15.18:38148) 2024-04-12T20:44:04.900 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:04.004+0000 7f08d1e00700 1 -- 172.21.15.18:0/2855901207 learned_addr learned my addr 172.21.15.18:0/2855901207 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T20:44:04.900 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:04.004+0000 7f08d1e00700 1 -- 172.21.15.18:0/2855901207 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08cc1056f0 con 0x7f08cc104bd0 2024-04-12T20:44:04.900 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:04.004+0000 7f08d1e00700 1 --2- 172.21.15.18:0/2855901207 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f08cc104bd0 0x7f08cc104fe0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f08c00098e0 tx=0x7f08c0009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=52a6f75b54d8ae54 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T20:44:04.900 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:04.005+0000 7f08d0a00700 1 -- 172.21.15.18:0/2855901207 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f08c000b860 con 0x7f08cc104bd0 2024-04-12T20:44:04.900 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:04.005+0000 7f08d0a00700 1 -- 172.21.15.18:0/2855901207 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f08c000be90 con 0x7f08cc104bd0 2024-04-12T20:44:04.900 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:04.005+0000 7f08d0a00700 1 -- 172.21.15.18:0/2855901207 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f08c00103c0 con 0x7f08cc104bd0 2024-04-12T20:44:04.900 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:04.005+0000 7f08d4800700 1 -- 172.21.15.18:0/2855901207 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f08cc104bd0 msgr2=0x7f08cc104fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:44:04.900 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:04.005+0000 7f08d4800700 1 --2- 172.21.15.18:0/2855901207 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f08cc104bd0 0x7f08cc104fe0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f08c00098e0 tx=0x7f08c0009250 comp rx=0 tx=0).stop 2024-04-12T20:44:04.900 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:04.006+0000 7f08d4800700 1 -- 172.21.15.18:0/2855901207 shutdown_connections 2024-04-12T20:44:04.900 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:04.006+0000 7f08d4800700 1 --2- 172.21.15.18:0/2855901207 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f08cc104bd0 0x7f08cc104fe0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:44:04.900 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:04.006+0000 7f08d4800700 1 -- 172.21.15.18:0/2855901207 >> 172.21.15.18:0/2855901207 conn(0x7f08cc078320 msgr2=0x7f08cc078740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T20:44:04.901 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:04.006+0000 7f08d4800700 1 -- 172.21.15.18:0/2855901207 shutdown_connections 2024-04-12T20:44:04.901 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:04.006+0000 7f08d4800700 1 -- 172.21.15.18:0/2855901207 wait complete. 2024-04-12T20:44:04.901 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:04.007+0000 7f08d4800700 1 Processor -- start 2024-04-12T20:44:04.901 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:04.007+0000 7f08d4800700 1 -- start start 2024-04-12T20:44:04.901 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:04.007+0000 7f08d4800700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f08cc104bd0 0x7f08cc113890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T20:44:04.901 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:04.007+0000 7f08d4800700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08cc113dd0 con 0x7f08cc104bd0 2024-04-12T20:44:04.901 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:04.007+0000 7f08d1e00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f08cc104bd0 0x7f08cc113890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T20:44:04.901 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:04.007+0000 7f08d1e00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f08cc104bd0 0x7f08cc113890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:38162/0 (socket says 172.21.15.18:38162) 2024-04-12T20:44:04.901 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:04.007+0000 7f08d1e00700 1 -- 172.21.15.18:0/346192930 learned_addr learned my addr 172.21.15.18:0/346192930 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T20:44:04.901 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:04.008+0000 7f08d1e00700 1 -- 172.21.15.18:0/346192930 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08c00094f0 con 0x7f08cc104bd0 2024-04-12T20:44:04.901 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:04.008+0000 7f08d1e00700 1 --2- 172.21.15.18:0/346192930 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f08cc104bd0 0x7f08cc113890 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f08c0006d40 tx=0x7f08c0009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T20:44:04.901 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:04.008+0000 7f08c6000700 1 -- 172.21.15.18:0/346192930 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f08c0010840 con 0x7f08cc104bd0 2024-04-12T20:44:04.901 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:04.008+0000 7f08c6000700 1 -- 172.21.15.18:0/346192930 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f08c0010e70 con 0x7f08cc104bd0 2024-04-12T20:44:04.901 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:04.008+0000 7f08d4800700 1 -- 172.21.15.18:0/346192930 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f08cc113fd0 con 0x7f08cc104bd0 2024-04-12T20:44:04.901 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:04.008+0000 7f08c6000700 1 -- 172.21.15.18:0/346192930 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f08c0024ea0 con 0x7f08cc104bd0 2024-04-12T20:44:04.902 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:04.009+0000 7f08d4800700 1 -- 172.21.15.18:0/346192930 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f08cc114b40 con 0x7f08cc104bd0 2024-04-12T20:44:04.902 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:04.010+0000 7f08c6000700 1 -- 172.21.15.18:0/346192930 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 8) v1 ==== 43368+0+0 (secure 0 0 0) 0x7f08c00109a0 con 0x7f08cc104bd0 2024-04-12T20:44:04.902 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:04.010+0000 7f08d4800700 1 -- 172.21.15.18:0/346192930 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f08cc05f5f0 con 0x7f08cc104bd0 2024-04-12T20:44:04.902 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:04.010+0000 7f08c6000700 1 --2- 172.21.15.18:0/346192930 >> [v2:172.21.15.18:6800/3403971131,v1:172.21.15.18:6801/3403971131] conn(0x7f08b4036740 0x7f08b4038bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T20:44:04.902 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:04.010+0000 7f08c6000700 1 -- 172.21.15.18:0/346192930 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 936+0+0 (secure 0 0 0) 0x7f08c004a790 con 0x7f08cc104bd0 2024-04-12T20:44:04.902 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:04.010+0000 7f08d1400700 1 --2- 172.21.15.18:0/346192930 >> [v2:172.21.15.18:6800/3403971131,v1:172.21.15.18:6801/3403971131] conn(0x7f08b4036740 0x7f08b4038bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T20:44:04.902 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:04.011+0000 7f08d1400700 1 --2- 172.21.15.18:0/346192930 >> [v2:172.21.15.18:6800/3403971131,v1:172.21.15.18:6801/3403971131] conn(0x7f08b4036740 0x7f08b4038bf0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f08bc009990 tx=0x7f08bc006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T20:44:04.902 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:04.016+0000 7f08c6000700 1 -- 172.21.15.18:0/346192930 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7f08c0024430 con 0x7f08cc104bd0 2024-04-12T20:44:04.902 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:04.230+0000 7f08d4800700 1 -- 172.21.15.18:0/346192930 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command([{prefix=config set, name=mgr/dashboard/ssl_server_port}] v 0) v1 -- 0x7f08cc04bed0 con 0x7f08cc104bd0 2024-04-12T20:44:04.902 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:04.238+0000 7f08c6000700 1 -- 172.21.15.18:0/346192930 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/dashboard/ssl_server_port}]=0 v8)=0 v8) v1 ==== 130+0+0 (secure 0 0 0) 0x7f08c002c340 con 0x7f08cc104bd0 2024-04-12T20:44:04.902 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:04.241+0000 7f08d4800700 1 -- 172.21.15.18:0/346192930 >> [v2:172.21.15.18:6800/3403971131,v1:172.21.15.18:6801/3403971131] conn(0x7f08b4036740 msgr2=0x7f08b4038bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:44:04.902 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:04.241+0000 7f08d4800700 1 --2- 172.21.15.18:0/346192930 >> [v2:172.21.15.18:6800/3403971131,v1:172.21.15.18:6801/3403971131] conn(0x7f08b4036740 0x7f08b4038bf0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f08bc009990 tx=0x7f08bc006e40 comp rx=0 tx=0).stop 2024-04-12T20:44:04.902 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:04.241+0000 7f08d4800700 1 -- 172.21.15.18:0/346192930 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f08cc104bd0 msgr2=0x7f08cc113890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:44:04.902 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:04.241+0000 7f08d4800700 1 --2- 172.21.15.18:0/346192930 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f08cc104bd0 0x7f08cc113890 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f08c0006d40 tx=0x7f08c0009250 comp rx=0 tx=0).stop 2024-04-12T20:44:04.903 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:04.241+0000 7f08d4800700 1 -- 172.21.15.18:0/346192930 shutdown_connections 2024-04-12T20:44:04.903 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:04.241+0000 7f08d4800700 1 --2- 172.21.15.18:0/346192930 >> [v2:172.21.15.18:6800/3403971131,v1:172.21.15.18:6801/3403971131] conn(0x7f08b4036740 0x7f08b4038bf0 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:44:04.903 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:04.241+0000 7f08d4800700 1 --2- 172.21.15.18:0/346192930 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f08cc104bd0 0x7f08cc113890 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:44:04.903 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:04.241+0000 7f08d4800700 1 -- 172.21.15.18:0/346192930 >> 172.21.15.18:0/346192930 conn(0x7f08cc078320 msgr2=0x7f08cc1024d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T20:44:04.903 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:04.242+0000 7f08d4800700 1 -- 172.21.15.18:0/346192930 shutdown_connections 2024-04-12T20:44:04.903 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:04.242+0000 7f08d4800700 1 -- 172.21.15.18:0/346192930 wait complete. 2024-04-12T20:44:04.903 INFO:teuthology.orchestra.run.smithi018.stdout:Enabling the dashboard module... 2024-04-12T20:44:05.534 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:44:05 smithi018 ceph-mon[28799]: from='client.? 172.21.15.18:0/346192930' entity='client.admin' 2024-04-12T20:44:05.535 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:44:05 smithi018 ceph-mon[28799]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-12T20:44:06.603 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:44:06 smithi018 ceph-mon[28799]: from='client.? 172.21.15.18:0/2765612530' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "dashboard"}]: dispatch 2024-04-12T20:44:07.053 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:05.535+0000 7f6fa0a00700 1 Processor -- start 2024-04-12T20:44:07.054 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:05.536+0000 7f6fa0a00700 1 -- start start 2024-04-12T20:44:07.054 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:05.536+0000 7f6fa0a00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6f9c104bd0 0x7f6f9c104fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T20:44:07.054 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:05.536+0000 7f6fa0a00700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f9c1055b0 con 0x7f6f9c104bd0 2024-04-12T20:44:07.054 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:05.537+0000 7f6f99e00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6f9c104bd0 0x7f6f9c104fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T20:44:07.055 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:05.537+0000 7f6f99e00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6f9c104bd0 0x7f6f9c104fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:38170/0 (socket says 172.21.15.18:38170) 2024-04-12T20:44:07.056 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:05.537+0000 7f6f99e00700 1 -- 172.21.15.18:0/3655966257 learned_addr learned my addr 172.21.15.18:0/3655966257 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T20:44:07.056 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:05.537+0000 7f6f99e00700 1 -- 172.21.15.18:0/3655966257 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f9c1056f0 con 0x7f6f9c104bd0 2024-04-12T20:44:07.056 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:05.537+0000 7f6f99e00700 1 --2- 172.21.15.18:0/3655966257 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6f9c104bd0 0x7f6f9c104fe0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f6f8c0098e0 tx=0x7f6f8c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ab15542d5e6482d server_cookie=0 in_seq=0 out_seq=0 2024-04-12T20:44:07.056 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:05.538+0000 7f6f98a00700 1 -- 172.21.15.18:0/3655966257 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f6f8c00b860 con 0x7f6f9c104bd0 2024-04-12T20:44:07.056 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:05.538+0000 7f6f98a00700 1 -- 172.21.15.18:0/3655966257 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f6f8c00be90 con 0x7f6f9c104bd0 2024-04-12T20:44:07.056 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:05.538+0000 7f6f98a00700 1 -- 172.21.15.18:0/3655966257 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f6f8c0103c0 con 0x7f6f9c104bd0 2024-04-12T20:44:07.056 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:05.538+0000 7f6fa0a00700 1 -- 172.21.15.18:0/3655966257 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6f9c104bd0 msgr2=0x7f6f9c104fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:44:07.056 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:05.538+0000 7f6fa0a00700 1 --2- 172.21.15.18:0/3655966257 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6f9c104bd0 0x7f6f9c104fe0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f6f8c0098e0 tx=0x7f6f8c009250 comp rx=0 tx=0).stop 2024-04-12T20:44:07.056 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:05.539+0000 7f6fa0a00700 1 -- 172.21.15.18:0/3655966257 shutdown_connections 2024-04-12T20:44:07.058 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:05.539+0000 7f6fa0a00700 1 --2- 172.21.15.18:0/3655966257 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6f9c104bd0 0x7f6f9c104fe0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:44:07.058 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:05.539+0000 7f6fa0a00700 1 -- 172.21.15.18:0/3655966257 >> 172.21.15.18:0/3655966257 conn(0x7f6f9c0782f0 msgr2=0x7f6f9c0786f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T20:44:07.058 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:05.539+0000 7f6fa0a00700 1 -- 172.21.15.18:0/3655966257 shutdown_connections 2024-04-12T20:44:07.058 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:05.539+0000 7f6fa0a00700 1 -- 172.21.15.18:0/3655966257 wait complete. 2024-04-12T20:44:07.058 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:05.540+0000 7f6fa0a00700 1 Processor -- start 2024-04-12T20:44:07.058 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:05.540+0000 7f6fa0a00700 1 -- start start 2024-04-12T20:44:07.058 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:05.540+0000 7f6fa0a00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6f9c104bd0 0x7f6f9c113890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T20:44:07.059 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:05.540+0000 7f6fa0a00700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f9c113dd0 con 0x7f6f9c104bd0 2024-04-12T20:44:07.059 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:05.541+0000 7f6f99e00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6f9c104bd0 0x7f6f9c113890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T20:44:07.059 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:05.541+0000 7f6f99e00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6f9c104bd0 0x7f6f9c113890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:38186/0 (socket says 172.21.15.18:38186) 2024-04-12T20:44:07.059 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:05.541+0000 7f6f99e00700 1 -- 172.21.15.18:0/2765612530 learned_addr learned my addr 172.21.15.18:0/2765612530 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T20:44:07.059 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:05.541+0000 7f6f99e00700 1 -- 172.21.15.18:0/2765612530 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f8c0094f0 con 0x7f6f9c104bd0 2024-04-12T20:44:07.062 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:05.541+0000 7f6f99e00700 1 --2- 172.21.15.18:0/2765612530 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6f9c104bd0 0x7f6f9c113890 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f6f8c006d40 tx=0x7f6f8c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T20:44:07.062 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:05.542+0000 7f6f92000700 1 -- 172.21.15.18:0/2765612530 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f6f8c010840 con 0x7f6f9c104bd0 2024-04-12T20:44:07.062 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:05.542+0000 7f6f92000700 1 -- 172.21.15.18:0/2765612530 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f6f8c010e70 con 0x7f6f9c104bd0 2024-04-12T20:44:07.062 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:05.542+0000 7f6fa0a00700 1 -- 172.21.15.18:0/2765612530 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6f9c113fd0 con 0x7f6f9c104bd0 2024-04-12T20:44:07.062 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:05.542+0000 7f6f92000700 1 -- 172.21.15.18:0/2765612530 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f6f8c024ea0 con 0x7f6f9c104bd0 2024-04-12T20:44:07.062 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:05.542+0000 7f6fa0a00700 1 -- 172.21.15.18:0/2765612530 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6f9c114b40 con 0x7f6f9c104bd0 2024-04-12T20:44:07.062 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:05.543+0000 7f6f92000700 1 -- 172.21.15.18:0/2765612530 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 8) v1 ==== 43368+0+0 (secure 0 0 0) 0x7f6f8c024780 con 0x7f6f9c104bd0 2024-04-12T20:44:07.062 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:05.543+0000 7f6fa0a00700 1 -- 172.21.15.18:0/2765612530 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6f9c05f5f0 con 0x7f6f9c104bd0 2024-04-12T20:44:07.062 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:05.543+0000 7f6f92000700 1 --2- 172.21.15.18:0/2765612530 >> [v2:172.21.15.18:6800/3403971131,v1:172.21.15.18:6801/3403971131] conn(0x7f6f800367e0 0x7f6f80038c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T20:44:07.062 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:05.543+0000 7f6f92000700 1 -- 172.21.15.18:0/2765612530 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 936+0+0 (secure 0 0 0) 0x7f6f8c04a740 con 0x7f6f9c104bd0 2024-04-12T20:44:07.062 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:05.544+0000 7f6f99400700 1 --2- 172.21.15.18:0/2765612530 >> [v2:172.21.15.18:6800/3403971131,v1:172.21.15.18:6801/3403971131] conn(0x7f6f800367e0 0x7f6f80038c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T20:44:07.062 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:05.544+0000 7f6f99400700 1 --2- 172.21.15.18:0/2765612530 >> [v2:172.21.15.18:6800/3403971131,v1:172.21.15.18:6801/3403971131] conn(0x7f6f800367e0 0x7f6f80038c90 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f6f88009990 tx=0x7f6f88006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T20:44:07.063 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:05.548+0000 7f6f92000700 1 -- 172.21.15.18:0/2765612530 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7f6f8c02c930 con 0x7f6f9c104bd0 2024-04-12T20:44:07.063 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:05.826+0000 7f6fa0a00700 1 -- 172.21.15.18:0/2765612530 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "dashboard"} v 0) v1 -- 0x7f6f9c04bed0 con 0x7f6f9c104bd0 2024-04-12T20:44:07.063 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:06.251+0000 7f6f92000700 1 -- 172.21.15.18:0/2765612530 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mgrmap(e 9) v1 ==== 43381+0+0 (secure 0 0 0) 0x7f6f8c02e440 con 0x7f6f9c104bd0 2024-04-12T20:44:07.063 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:06.254+0000 7f6f92000700 1 -- 172.21.15.18:0/2765612530 <== mon.0 v2:172.21.15.18:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "dashboard"}]=0 v9) v1 ==== 88+0+0 (secure 0 0 0) 0x7f6f8c018350 con 0x7f6f9c104bd0 2024-04-12T20:44:07.063 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:06.258+0000 7f6fa0a00700 1 -- 172.21.15.18:0/2765612530 >> [v2:172.21.15.18:6800/3403971131,v1:172.21.15.18:6801/3403971131] conn(0x7f6f800367e0 msgr2=0x7f6f80038c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:44:07.063 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:06.258+0000 7f6fa0a00700 1 --2- 172.21.15.18:0/2765612530 >> [v2:172.21.15.18:6800/3403971131,v1:172.21.15.18:6801/3403971131] conn(0x7f6f800367e0 0x7f6f80038c90 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f6f88009990 tx=0x7f6f88006e40 comp rx=0 tx=0).stop 2024-04-12T20:44:07.063 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:06.258+0000 7f6fa0a00700 1 -- 172.21.15.18:0/2765612530 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6f9c104bd0 msgr2=0x7f6f9c113890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:44:07.063 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:06.258+0000 7f6fa0a00700 1 --2- 172.21.15.18:0/2765612530 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6f9c104bd0 0x7f6f9c113890 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f6f8c006d40 tx=0x7f6f8c009250 comp rx=0 tx=0).stop 2024-04-12T20:44:07.063 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:06.258+0000 7f6fa0a00700 1 -- 172.21.15.18:0/2765612530 shutdown_connections 2024-04-12T20:44:07.063 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:06.258+0000 7f6fa0a00700 1 --2- 172.21.15.18:0/2765612530 >> [v2:172.21.15.18:6800/3403971131,v1:172.21.15.18:6801/3403971131] conn(0x7f6f800367e0 0x7f6f80038c90 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:44:07.063 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:06.258+0000 7f6fa0a00700 1 --2- 172.21.15.18:0/2765612530 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6f9c104bd0 0x7f6f9c113890 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:44:07.063 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:06.258+0000 7f6fa0a00700 1 -- 172.21.15.18:0/2765612530 >> 172.21.15.18:0/2765612530 conn(0x7f6f9c0782f0 msgr2=0x7f6f9c102520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T20:44:07.063 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:06.258+0000 7f6fa0a00700 1 -- 172.21.15.18:0/2765612530 shutdown_connections 2024-04-12T20:44:07.064 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:06.258+0000 7f6fa0a00700 1 -- 172.21.15.18:0/2765612530 wait complete. 2024-04-12T20:44:07.332 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:44:07 smithi018 ceph-mon[28799]: from='client.? 172.21.15.18:0/2765612530' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "dashboard"}]': finished 2024-04-12T20:44:07.332 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:44:07 smithi018 ceph-mon[28799]: mgrmap e9: smithi018.xjzwga(active, since 31s) 2024-04-12T20:44:08.544 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:44:08 smithi018 ceph-mon[28799]: from='client.? 172.21.15.18:0/3241014435' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-04-12T20:44:08.558 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout { 2024-04-12T20:44:08.558 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "epoch": 9, 2024-04-12T20:44:08.558 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "available": true, 2024-04-12T20:44:08.559 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "active_name": "smithi018.xjzwga", 2024-04-12T20:44:08.559 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-04-12T20:44:08.559 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout } 2024-04-12T20:44:08.559 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:07.622+0000 7f42fb200700 1 Processor -- start 2024-04-12T20:44:08.559 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:07.622+0000 7f42fb200700 1 -- start start 2024-04-12T20:44:08.559 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:07.622+0000 7f42fb200700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f42f4077ab0 0x7f42f4076100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T20:44:08.559 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:07.622+0000 7f42fb200700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42f4077ec0 con 0x7f42f4077ab0 2024-04-12T20:44:08.559 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:07.623+0000 7f42f3e00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f42f4077ab0 0x7f42f4076100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T20:44:08.559 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:07.623+0000 7f42f3e00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f42f4077ab0 0x7f42f4076100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:38218/0 (socket says 172.21.15.18:38218) 2024-04-12T20:44:08.559 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:07.623+0000 7f42f3e00700 1 -- 172.21.15.18:0/3134133408 learned_addr learned my addr 172.21.15.18:0/3134133408 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T20:44:08.559 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:07.623+0000 7f42f3e00700 1 -- 172.21.15.18:0/3134133408 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42f4078000 con 0x7f42f4077ab0 2024-04-12T20:44:08.559 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:07.624+0000 7f42f3e00700 1 --2- 172.21.15.18:0/3134133408 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f42f4077ab0 0x7f42f4076100 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f42e40098e0 tx=0x7f42e4009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=30dcd8faa2df9d54 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T20:44:08.559 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:07.624+0000 7f42f3400700 1 -- 172.21.15.18:0/3134133408 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f42e400b860 con 0x7f42f4077ab0 2024-04-12T20:44:08.560 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:07.624+0000 7f42f3400700 1 -- 172.21.15.18:0/3134133408 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f42e400be90 con 0x7f42f4077ab0 2024-04-12T20:44:08.560 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:07.624+0000 7f42f3400700 1 -- 172.21.15.18:0/3134133408 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f42e40103c0 con 0x7f42f4077ab0 2024-04-12T20:44:08.560 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:07.625+0000 7f42fb200700 1 -- 172.21.15.18:0/3134133408 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f42f4077ab0 msgr2=0x7f42f4076100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:44:08.560 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:07.625+0000 7f42fb200700 1 --2- 172.21.15.18:0/3134133408 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f42f4077ab0 0x7f42f4076100 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f42e40098e0 tx=0x7f42e4009250 comp rx=0 tx=0).stop 2024-04-12T20:44:08.560 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:07.625+0000 7f42fb200700 1 -- 172.21.15.18:0/3134133408 shutdown_connections 2024-04-12T20:44:08.560 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:07.625+0000 7f42fb200700 1 --2- 172.21.15.18:0/3134133408 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f42f4077ab0 0x7f42f4076100 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:44:08.560 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:07.625+0000 7f42fb200700 1 -- 172.21.15.18:0/3134133408 >> 172.21.15.18:0/3134133408 conn(0x7f42f40fe090 msgr2=0x7f42f4100500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T20:44:08.560 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:07.625+0000 7f42fb200700 1 -- 172.21.15.18:0/3134133408 shutdown_connections 2024-04-12T20:44:08.560 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:07.625+0000 7f42fb200700 1 -- 172.21.15.18:0/3134133408 wait complete. 2024-04-12T20:44:08.560 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:07.626+0000 7f42fb200700 1 Processor -- start 2024-04-12T20:44:08.560 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:07.626+0000 7f42fb200700 1 -- start start 2024-04-12T20:44:08.560 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:07.626+0000 7f42fb200700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f42f4077ab0 0x7f42f4113990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T20:44:08.560 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:07.626+0000 7f42fb200700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42f4113ed0 con 0x7f42f4077ab0 2024-04-12T20:44:08.560 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:07.627+0000 7f42f3e00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f42f4077ab0 0x7f42f4113990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T20:44:08.560 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:07.627+0000 7f42f3e00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f42f4077ab0 0x7f42f4113990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:38232/0 (socket says 172.21.15.18:38232) 2024-04-12T20:44:08.560 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:07.627+0000 7f42f3e00700 1 -- 172.21.15.18:0/3241014435 learned_addr learned my addr 172.21.15.18:0/3241014435 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T20:44:08.561 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:07.627+0000 7f42f3e00700 1 -- 172.21.15.18:0/3241014435 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42e40094f0 con 0x7f42f4077ab0 2024-04-12T20:44:08.561 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:07.627+0000 7f42f3e00700 1 --2- 172.21.15.18:0/3241014435 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f42f4077ab0 0x7f42f4113990 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f42e40105b0 tx=0x7f42e4009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T20:44:08.561 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:07.628+0000 7f42f0c00700 1 -- 172.21.15.18:0/3241014435 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f42e4010840 con 0x7f42f4077ab0 2024-04-12T20:44:08.561 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:07.628+0000 7f42f0c00700 1 -- 172.21.15.18:0/3241014435 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f42e4010e70 con 0x7f42f4077ab0 2024-04-12T20:44:08.561 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:07.628+0000 7f42fb200700 1 -- 172.21.15.18:0/3241014435 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f42f41140d0 con 0x7f42f4077ab0 2024-04-12T20:44:08.561 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:07.628+0000 7f42f0c00700 1 -- 172.21.15.18:0/3241014435 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f42e401ae30 con 0x7f42f4077ab0 2024-04-12T20:44:08.561 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:07.628+0000 7f42fb200700 1 -- 172.21.15.18:0/3241014435 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f42f4114c40 con 0x7f42f4077ab0 2024-04-12T20:44:08.561 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:07.629+0000 7f42fb200700 1 -- 172.21.15.18:0/3241014435 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f42f405f5f0 con 0x7f42f4077ab0 2024-04-12T20:44:08.561 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:07.629+0000 7f42f0c00700 1 -- 172.21.15.18:0/3241014435 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 9) v1 ==== 43381+0+0 (secure 0 0 0) 0x7f42e401a430 con 0x7f42f4077ab0 2024-04-12T20:44:08.561 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:07.629+0000 7f42f0c00700 1 --2- 172.21.15.18:0/3241014435 >> [v2:172.21.15.18:6800/3403971131,v1:172.21.15.18:6801/3403971131] conn(0x7f42d8036830 0x7f42d8038ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T20:44:08.561 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:07.629+0000 7f42f0c00700 1 -- 172.21.15.18:0/3241014435 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 936+0+0 (secure 0 0 0) 0x7f42e404a6b0 con 0x7f42f4077ab0 2024-04-12T20:44:08.561 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:07.630+0000 7f42ebe00700 1 -- 172.21.15.18:0/3241014435 >> [v2:172.21.15.18:6800/3403971131,v1:172.21.15.18:6801/3403971131] conn(0x7f42d8036830 msgr2=0x7f42d8038ce0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.18:6800/3403971131 2024-04-12T20:44:08.561 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:07.630+0000 7f42ebe00700 1 --2- 172.21.15.18:0/3241014435 >> [v2:172.21.15.18:6800/3403971131,v1:172.21.15.18:6801/3403971131] conn(0x7f42d8036830 0x7f42d8038ce0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-12T20:44:08.561 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:07.634+0000 7f42f0c00700 1 -- 172.21.15.18:0/3241014435 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7f42e402c930 con 0x7f42f4077ab0 2024-04-12T20:44:08.561 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:07.831+0000 7f42ebe00700 1 -- 172.21.15.18:0/3241014435 >> [v2:172.21.15.18:6800/3403971131,v1:172.21.15.18:6801/3403971131] conn(0x7f42d8036830 msgr2=0x7f42d8038ce0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.18:6800/3403971131 2024-04-12T20:44:08.561 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:07.831+0000 7f42ebe00700 1 --2- 172.21.15.18:0/3241014435 >> [v2:172.21.15.18:6800/3403971131,v1:172.21.15.18:6801/3403971131] conn(0x7f42d8036830 0x7f42d8038ce0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-12T20:44:08.562 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:07.901+0000 7f42fb200700 1 -- 172.21.15.18:0/3241014435 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) v1 -- 0x7f42f4114f80 con 0x7f42f4077ab0 2024-04-12T20:44:08.562 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:07.902+0000 7f42f0c00700 1 -- 172.21.15.18:0/3241014435 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v9) v1 ==== 56+0+103 (secure 0 0 0) 0x7f42e402c340 con 0x7f42f4077ab0 2024-04-12T20:44:08.562 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:07.905+0000 7f42fb200700 1 -- 172.21.15.18:0/3241014435 >> [v2:172.21.15.18:6800/3403971131,v1:172.21.15.18:6801/3403971131] conn(0x7f42d8036830 msgr2=0x7f42d8038ce0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-12T20:44:08.562 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:07.905+0000 7f42fb200700 1 --2- 172.21.15.18:0/3241014435 >> [v2:172.21.15.18:6800/3403971131,v1:172.21.15.18:6801/3403971131] conn(0x7f42d8036830 0x7f42d8038ce0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:44:08.562 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:07.905+0000 7f42fb200700 1 -- 172.21.15.18:0/3241014435 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f42f4077ab0 msgr2=0x7f42f4113990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:44:08.562 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:07.905+0000 7f42fb200700 1 --2- 172.21.15.18:0/3241014435 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f42f4077ab0 0x7f42f4113990 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f42e40105b0 tx=0x7f42e4009250 comp rx=0 tx=0).stop 2024-04-12T20:44:08.562 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:07.905+0000 7f42fb200700 1 -- 172.21.15.18:0/3241014435 shutdown_connections 2024-04-12T20:44:08.562 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:07.905+0000 7f42fb200700 1 --2- 172.21.15.18:0/3241014435 >> [v2:172.21.15.18:6800/3403971131,v1:172.21.15.18:6801/3403971131] conn(0x7f42d8036830 0x7f42d8038ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:44:08.562 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:07.905+0000 7f42fb200700 1 --2- 172.21.15.18:0/3241014435 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f42f4077ab0 0x7f42f4113990 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:44:08.562 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:07.905+0000 7f42fb200700 1 -- 172.21.15.18:0/3241014435 >> 172.21.15.18:0/3241014435 conn(0x7f42f40fe090 msgr2=0x7f42f40fecf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T20:44:08.562 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:07.906+0000 7f42fb200700 1 -- 172.21.15.18:0/3241014435 shutdown_connections 2024-04-12T20:44:08.562 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:07.906+0000 7f42fb200700 1 -- 172.21.15.18:0/3241014435 wait complete. 2024-04-12T20:44:08.562 INFO:teuthology.orchestra.run.smithi018.stdout:Waiting for the mgr to restart... 2024-04-12T20:44:08.562 INFO:teuthology.orchestra.run.smithi018.stdout:Waiting for mgr epoch 9... 2024-04-12T20:44:15.603 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:44:15 smithi018 ceph-mon[28799]: Active manager daemon smithi018.xjzwga restarted 2024-04-12T20:44:15.603 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:44:15 smithi018 ceph-mon[28799]: Activating manager daemon smithi018.xjzwga 2024-04-12T20:44:15.603 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:44:15 smithi018 ceph-mon[28799]: osdmap e3: 0 total, 0 up, 0 in 2024-04-12T20:44:15.603 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:44:15 smithi018 ceph-mon[28799]: mgrmap e10: smithi018.xjzwga(active, starting, since 0.00898533s) 2024-04-12T20:44:15.603 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:44:15 smithi018 ceph-mon[28799]: from='mgr.14162 172.21.15.18:0/2711576919' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "mon metadata", "id": "smithi018"}]: dispatch 2024-04-12T20:44:15.603 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:44:15 smithi018 ceph-mon[28799]: from='mgr.14162 172.21.15.18:0/2711576919' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "mgr metadata", "who": "smithi018.xjzwga", "id": "smithi018.xjzwga"}]: dispatch 2024-04-12T20:44:15.603 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:44:15 smithi018 ceph-mon[28799]: from='mgr.14162 172.21.15.18:0/2711576919' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-12T20:44:15.603 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:44:15 smithi018 ceph-mon[28799]: from='mgr.14162 172.21.15.18:0/2711576919' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-12T20:44:15.603 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:44:15 smithi018 ceph-mon[28799]: from='mgr.14162 172.21.15.18:0/2711576919' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-12T20:44:15.603 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:44:15 smithi018 ceph-mon[28799]: Manager daemon smithi018.xjzwga is now available 2024-04-12T20:44:16.488 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:44:16 smithi018 ceph-mon[28799]: from='mgr.14162 172.21.15.18:0/2711576919' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-12T20:44:16.488 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:44:16 smithi018 ceph-mon[28799]: from='mgr.14162 172.21.15.18:0/2711576919' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-12T20:44:16.488 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:44:16 smithi018 ceph-mon[28799]: from='mgr.14162 172.21.15.18:0/2711576919' entity='mgr.smithi018.xjzwga' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi018.xjzwga/mirror_snapshot_schedule"}]: dispatch 2024-04-12T20:44:16.488 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:44:16 smithi018 ceph-mon[28799]: from='mgr.14162 172.21.15.18:0/2711576919' entity='mgr.smithi018.xjzwga' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi018.xjzwga/trash_purge_schedule"}]: dispatch 2024-04-12T20:44:16.488 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:44:16 smithi018 ceph-mon[28799]: from='mgr.14162 172.21.15.18:0/2711576919' entity='mgr.smithi018.xjzwga' 2024-04-12T20:44:16.488 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:44:16 smithi018 ceph-mon[28799]: mgrmap e11: smithi018.xjzwga(active, since 1.00825s) 2024-04-12T20:44:17.412 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout { 2024-04-12T20:44:17.412 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 11, 2024-04-12T20:44:17.413 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "initialized": true 2024-04-12T20:44:17.413 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout } 2024-04-12T20:44:17.413 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:09.145+0000 7f450b400700 1 Processor -- start 2024-04-12T20:44:17.414 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:09.146+0000 7f450b400700 1 -- start start 2024-04-12T20:44:17.414 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:09.146+0000 7f450b400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4504077ad0 0x7f4504076120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T20:44:17.414 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:09.146+0000 7f450b400700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4504077ee0 con 0x7f4504077ad0 2024-04-12T20:44:17.414 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:09.147+0000 7f4508a00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4504077ad0 0x7f4504076120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T20:44:17.414 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:09.147+0000 7f4508a00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4504077ad0 0x7f4504076120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:38234/0 (socket says 172.21.15.18:38234) 2024-04-12T20:44:17.415 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:09.147+0000 7f4508a00700 1 -- 172.21.15.18:0/1921021256 learned_addr learned my addr 172.21.15.18:0/1921021256 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T20:44:17.415 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:09.147+0000 7f4508a00700 1 -- 172.21.15.18:0/1921021256 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4504078020 con 0x7f4504077ad0 2024-04-12T20:44:17.415 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:09.148+0000 7f4508a00700 1 --2- 172.21.15.18:0/1921021256 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4504077ad0 0x7f4504076120 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f44f80098e0 tx=0x7f44f8009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f2040f0108dda3ac server_cookie=0 in_seq=0 out_seq=0 2024-04-12T20:44:17.415 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:09.148+0000 7f4503400700 1 -- 172.21.15.18:0/1921021256 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f44f800b860 con 0x7f4504077ad0 2024-04-12T20:44:17.415 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:09.148+0000 7f4503400700 1 -- 172.21.15.18:0/1921021256 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f44f800be90 con 0x7f4504077ad0 2024-04-12T20:44:17.415 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:09.148+0000 7f4503400700 1 -- 172.21.15.18:0/1921021256 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f44f80103c0 con 0x7f4504077ad0 2024-04-12T20:44:17.416 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:09.149+0000 7f450b400700 1 -- 172.21.15.18:0/1921021256 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4504077ad0 msgr2=0x7f4504076120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:44:17.417 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:09.149+0000 7f450b400700 1 --2- 172.21.15.18:0/1921021256 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4504077ad0 0x7f4504076120 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f44f80098e0 tx=0x7f44f8009250 comp rx=0 tx=0).stop 2024-04-12T20:44:17.417 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:09.149+0000 7f450b400700 1 -- 172.21.15.18:0/1921021256 shutdown_connections 2024-04-12T20:44:17.417 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:09.149+0000 7f450b400700 1 --2- 172.21.15.18:0/1921021256 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4504077ad0 0x7f4504076120 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:44:17.417 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:09.149+0000 7f450b400700 1 -- 172.21.15.18:0/1921021256 >> 172.21.15.18:0/1921021256 conn(0x7f45040fe090 msgr2=0x7f4504100500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T20:44:17.417 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:09.149+0000 7f450b400700 1 -- 172.21.15.18:0/1921021256 shutdown_connections 2024-04-12T20:44:17.417 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:09.149+0000 7f450b400700 1 -- 172.21.15.18:0/1921021256 wait complete. 2024-04-12T20:44:17.417 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:09.150+0000 7f450b400700 1 Processor -- start 2024-04-12T20:44:17.418 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:09.150+0000 7f450b400700 1 -- start start 2024-04-12T20:44:17.418 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:09.151+0000 7f450b400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4504077ad0 0x7f4504113990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T20:44:17.418 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:09.151+0000 7f450b400700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4504113ed0 con 0x7f4504077ad0 2024-04-12T20:44:17.418 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:09.151+0000 7f4508a00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4504077ad0 0x7f4504113990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T20:44:17.418 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:09.151+0000 7f4508a00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4504077ad0 0x7f4504113990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:38242/0 (socket says 172.21.15.18:38242) 2024-04-12T20:44:17.419 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:09.151+0000 7f4508a00700 1 -- 172.21.15.18:0/1228740242 learned_addr learned my addr 172.21.15.18:0/1228740242 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T20:44:17.419 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:09.151+0000 7f4508a00700 1 -- 172.21.15.18:0/1228740242 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44f80094f0 con 0x7f4504077ad0 2024-04-12T20:44:17.419 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:09.152+0000 7f4508a00700 1 --2- 172.21.15.18:0/1228740242 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4504077ad0 0x7f4504113990 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f44f80105b0 tx=0x7f44f8009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T20:44:17.419 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:09.152+0000 7f4500c00700 1 -- 172.21.15.18:0/1228740242 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f44f8010840 con 0x7f4504077ad0 2024-04-12T20:44:17.419 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:09.152+0000 7f4500c00700 1 -- 172.21.15.18:0/1228740242 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f44f8010e70 con 0x7f4504077ad0 2024-04-12T20:44:17.420 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:09.152+0000 7f4500c00700 1 -- 172.21.15.18:0/1228740242 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f44f801ae30 con 0x7f4504077ad0 2024-04-12T20:44:17.421 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:09.152+0000 7f450b400700 1 -- 172.21.15.18:0/1228740242 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f45041140d0 con 0x7f4504077ad0 2024-04-12T20:44:17.421 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:09.152+0000 7f450b400700 1 -- 172.21.15.18:0/1228740242 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4504114c40 con 0x7f4504077ad0 2024-04-12T20:44:17.421 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:09.153+0000 7f4500c00700 1 -- 172.21.15.18:0/1228740242 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 9) v1 ==== 43381+0+0 (secure 0 0 0) 0x7f44f80109a0 con 0x7f4504077ad0 2024-04-12T20:44:17.421 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:09.154+0000 7f4500c00700 1 --2- 172.21.15.18:0/1228740242 >> [v2:172.21.15.18:6800/3403971131,v1:172.21.15.18:6801/3403971131] conn(0x7f44ec0367e0 0x7f44ec038c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T20:44:17.421 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:09.154+0000 7f4500c00700 1 -- 172.21.15.18:0/1228740242 --> [v2:172.21.15.18:6800/3403971131,v1:172.21.15.18:6801/3403971131] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7f44ec0393a0 con 0x7f44ec0367e0 2024-04-12T20:44:17.421 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:09.154+0000 7f4500c00700 1 -- 172.21.15.18:0/1228740242 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 936+0+0 (secure 0 0 0) 0x7f44f804a320 con 0x7f4504077ad0 2024-04-12T20:44:17.421 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:09.154+0000 7f4503e00700 1 -- 172.21.15.18:0/1228740242 >> [v2:172.21.15.18:6800/3403971131,v1:172.21.15.18:6801/3403971131] conn(0x7f44ec0367e0 msgr2=0x7f44ec038c90 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.18:6800/3403971131 2024-04-12T20:44:17.421 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:09.154+0000 7f4503e00700 1 --2- 172.21.15.18:0/1228740242 >> [v2:172.21.15.18:6800/3403971131,v1:172.21.15.18:6801/3403971131] conn(0x7f44ec0367e0 0x7f44ec038c90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-12T20:44:17.421 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:09.355+0000 7f4503e00700 1 -- 172.21.15.18:0/1228740242 >> [v2:172.21.15.18:6800/3403971131,v1:172.21.15.18:6801/3403971131] conn(0x7f44ec0367e0 msgr2=0x7f44ec038c90 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.18:6800/3403971131 2024-04-12T20:44:17.421 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:09.355+0000 7f4503e00700 1 --2- 172.21.15.18:0/1228740242 >> [v2:172.21.15.18:6800/3403971131,v1:172.21.15.18:6801/3403971131] conn(0x7f44ec0367e0 0x7f44ec038c90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-12T20:44:17.421 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:09.756+0000 7f4503e00700 1 -- 172.21.15.18:0/1228740242 >> [v2:172.21.15.18:6800/3403971131,v1:172.21.15.18:6801/3403971131] conn(0x7f44ec0367e0 msgr2=0x7f44ec038c90 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.18:6800/3403971131 2024-04-12T20:44:17.421 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:09.756+0000 7f4503e00700 1 --2- 172.21.15.18:0/1228740242 >> [v2:172.21.15.18:6800/3403971131,v1:172.21.15.18:6801/3403971131] conn(0x7f44ec0367e0 0x7f44ec038c90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-12T20:44:17.421 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:10.557+0000 7f4503e00700 1 -- 172.21.15.18:0/1228740242 >> [v2:172.21.15.18:6800/3403971131,v1:172.21.15.18:6801/3403971131] conn(0x7f44ec0367e0 msgr2=0x7f44ec038c90 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.18:6800/3403971131 2024-04-12T20:44:17.423 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:10.557+0000 7f4503e00700 1 --2- 172.21.15.18:0/1228740242 >> [v2:172.21.15.18:6800/3403971131,v1:172.21.15.18:6801/3403971131] conn(0x7f44ec0367e0 0x7f44ec038c90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-12T20:44:17.423 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:12.159+0000 7f4503e00700 1 -- 172.21.15.18:0/1228740242 >> [v2:172.21.15.18:6800/3403971131,v1:172.21.15.18:6801/3403971131] conn(0x7f44ec0367e0 msgr2=0x7f44ec038c90 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.18:6800/3403971131 2024-04-12T20:44:17.423 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:12.159+0000 7f4503e00700 1 --2- 172.21.15.18:0/1228740242 >> [v2:172.21.15.18:6800/3403971131,v1:172.21.15.18:6801/3403971131] conn(0x7f44ec0367e0 0x7f44ec038c90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-04-12T20:44:17.423 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:15.149+0000 7f4500c00700 1 -- 172.21.15.18:0/1228740242 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mgrmap(e 10) v1 ==== 43148+0+0 (secure 0 0 0) 0x7f44f801a710 con 0x7f4504077ad0 2024-04-12T20:44:17.423 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:15.149+0000 7f4500c00700 1 -- 172.21.15.18:0/1228740242 >> [v2:172.21.15.18:6800/3403971131,v1:172.21.15.18:6801/3403971131] conn(0x7f44ec0367e0 msgr2=0x7f44ec038c90 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-12T20:44:17.423 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:15.149+0000 7f4500c00700 1 --2- 172.21.15.18:0/1228740242 >> [v2:172.21.15.18:6800/3403971131,v1:172.21.15.18:6801/3403971131] conn(0x7f44ec0367e0 0x7f44ec038c90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:44:17.423 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:16.149+0000 7f4500c00700 1 -- 172.21.15.18:0/1228740242 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mgrmap(e 11) v1 ==== 43275+0+0 (secure 0 0 0) 0x7f44f804baa0 con 0x7f4504077ad0 2024-04-12T20:44:17.423 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:16.149+0000 7f4500c00700 1 --2- 172.21.15.18:0/1228740242 >> [v2:172.21.15.18:6800/1391635591,v1:172.21.15.18:6801/1391635591] conn(0x7f44ec03a070 0x7f44ec03c450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T20:44:17.424 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:16.149+0000 7f4500c00700 1 -- 172.21.15.18:0/1228740242 --> [v2:172.21.15.18:6800/1391635591,v1:172.21.15.18:6801/1391635591] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7f44ec0393a0 con 0x7f44ec03a070 2024-04-12T20:44:17.424 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:16.150+0000 7f4503e00700 1 --2- 172.21.15.18:0/1228740242 >> [v2:172.21.15.18:6800/1391635591,v1:172.21.15.18:6801/1391635591] conn(0x7f44ec03a070 0x7f44ec03c450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T20:44:17.424 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:16.152+0000 7f4503e00700 1 --2- 172.21.15.18:0/1228740242 >> [v2:172.21.15.18:6800/1391635591,v1:172.21.15.18:6801/1391635591] conn(0x7f44ec03a070 0x7f44ec03c450 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f44f00039d0 tx=0x7f44f00092f0 comp rx=0 tx=0).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T20:44:17.424 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:16.152+0000 7f4500c00700 1 -- 172.21.15.18:0/1228740242 <== mgr.14162 v2:172.21.15.18:6800/1391635591 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+6549 (secure 0 0 0) 0x7f44ec0393a0 con 0x7f44ec03a070 2024-04-12T20:44:17.424 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:16.159+0000 7f450b400700 1 -- 172.21.15.18:0/1228740242 --> [v2:172.21.15.18:6800/1391635591,v1:172.21.15.18:6801/1391635591] -- command(tid 1: {"prefix": "mgr_status"}) v1 -- 0x7f4504114560 con 0x7f44ec03a070 2024-04-12T20:44:17.424 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:16.160+0000 7f4500c00700 1 -- 172.21.15.18:0/1228740242 <== mgr.14162 v2:172.21.15.18:6800/1391635591 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+52 (secure 0 0 0) 0x7f4504114560 con 0x7f44ec03a070 2024-04-12T20:44:17.424 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:16.160+0000 7f450b400700 1 -- 172.21.15.18:0/1228740242 >> [v2:172.21.15.18:6800/1391635591,v1:172.21.15.18:6801/1391635591] conn(0x7f44ec03a070 msgr2=0x7f44ec03c450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:44:17.424 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:16.160+0000 7f450b400700 1 --2- 172.21.15.18:0/1228740242 >> [v2:172.21.15.18:6800/1391635591,v1:172.21.15.18:6801/1391635591] conn(0x7f44ec03a070 0x7f44ec03c450 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f44f00039d0 tx=0x7f44f00092f0 comp rx=0 tx=0).stop 2024-04-12T20:44:17.425 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:16.160+0000 7f450b400700 1 -- 172.21.15.18:0/1228740242 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4504077ad0 msgr2=0x7f4504113990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:44:17.425 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:16.160+0000 7f450b400700 1 --2- 172.21.15.18:0/1228740242 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4504077ad0 0x7f4504113990 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f44f80105b0 tx=0x7f44f8009250 comp rx=0 tx=0).stop 2024-04-12T20:44:17.425 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:16.160+0000 7f450b400700 1 -- 172.21.15.18:0/1228740242 shutdown_connections 2024-04-12T20:44:17.426 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:16.160+0000 7f450b400700 1 --2- 172.21.15.18:0/1228740242 >> [v2:172.21.15.18:6800/1391635591,v1:172.21.15.18:6801/1391635591] conn(0x7f44ec03a070 0x7f44ec03c450 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:44:17.426 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:16.160+0000 7f450b400700 1 --2- 172.21.15.18:0/1228740242 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4504077ad0 0x7f4504113990 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:44:17.426 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:16.160+0000 7f450b400700 1 --2- 172.21.15.18:0/1228740242 >> [v2:172.21.15.18:6800/3403971131,v1:172.21.15.18:6801/3403971131] conn(0x7f44ec0367e0 0x7f44ec038c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:44:17.426 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:16.160+0000 7f450b400700 1 -- 172.21.15.18:0/1228740242 >> 172.21.15.18:0/1228740242 conn(0x7f45040fe090 msgr2=0x7f45040fed60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T20:44:17.426 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:16.161+0000 7f450b400700 1 -- 172.21.15.18:0/1228740242 shutdown_connections 2024-04-12T20:44:17.426 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:16.161+0000 7f450b400700 1 -- 172.21.15.18:0/1228740242 wait complete. 2024-04-12T20:44:17.426 INFO:teuthology.orchestra.run.smithi018.stdout:mgr epoch 9 is available 2024-04-12T20:44:17.426 INFO:teuthology.orchestra.run.smithi018.stdout:Generating a dashboard self-signed certificate... 2024-04-12T20:44:17.586 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:44:17 smithi018 ceph-mon[28799]: [12/Apr/2024:20:44:15] ENGINE Bus STARTING 2024-04-12T20:44:17.586 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:44:17 smithi018 ceph-mon[28799]: [12/Apr/2024:20:44:15] ENGINE Serving on https://172.21.15.18:7150 2024-04-12T20:44:17.586 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:44:17 smithi018 ceph-mon[28799]: [12/Apr/2024:20:44:15] ENGINE Bus STARTED 2024-04-12T20:44:17.586 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:44:17 smithi018 ceph-mon[28799]: from='client.14166 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-04-12T20:44:17.586 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:44:17 smithi018 ceph-mon[28799]: from='client.14166 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-04-12T20:44:18.455 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:44:18 smithi018 ceph-mon[28799]: mgrmap e12: smithi018.xjzwga(active, since 2s) 2024-04-12T20:44:18.455 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:44:18 smithi018 ceph-mon[28799]: from='mgr.14162 172.21.15.18:0/2711576919' entity='mgr.smithi018.xjzwga' 2024-04-12T20:44:19.071 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout Self-signed certificate created 2024-04-12T20:44:19.072 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:18.031+0000 7fd6b9a00700 1 Processor -- start 2024-04-12T20:44:19.072 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:18.031+0000 7fd6b9a00700 1 -- start start 2024-04-12T20:44:19.072 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:18.032+0000 7fd6b9a00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd6b4071270 0x7fd6b4105520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T20:44:19.072 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:18.032+0000 7fd6b9a00700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6b4071680 con 0x7fd6b4071270 2024-04-12T20:44:19.072 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:18.033+0000 7fd6b2a00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd6b4071270 0x7fd6b4105520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T20:44:19.073 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:18.033+0000 7fd6b2a00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd6b4071270 0x7fd6b4105520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:42538/0 (socket says 172.21.15.18:42538) 2024-04-12T20:44:19.073 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:18.033+0000 7fd6b2a00700 1 -- 172.21.15.18:0/1743389240 learned_addr learned my addr 172.21.15.18:0/1743389240 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T20:44:19.073 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:18.034+0000 7fd6b2a00700 1 -- 172.21.15.18:0/1743389240 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6b4105a60 con 0x7fd6b4071270 2024-04-12T20:44:19.074 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:18.034+0000 7fd6b2a00700 1 --2- 172.21.15.18:0/1743389240 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd6b4071270 0x7fd6b4105520 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fd6a40098e0 tx=0x7fd6a4009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7bf908dcc5876e4b server_cookie=0 in_seq=0 out_seq=0 2024-04-12T20:44:19.075 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:18.034+0000 7fd6b1600700 1 -- 172.21.15.18:0/1743389240 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fd6a400b860 con 0x7fd6b4071270 2024-04-12T20:44:19.075 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:18.034+0000 7fd6b1600700 1 -- 172.21.15.18:0/1743389240 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd6a400be90 con 0x7fd6b4071270 2024-04-12T20:44:19.075 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:18.034+0000 7fd6b1600700 1 -- 172.21.15.18:0/1743389240 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fd6a40103c0 con 0x7fd6b4071270 2024-04-12T20:44:19.075 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:18.035+0000 7fd6b9a00700 1 -- 172.21.15.18:0/1743389240 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd6b4071270 msgr2=0x7fd6b4105520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:44:19.075 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:18.035+0000 7fd6b9a00700 1 --2- 172.21.15.18:0/1743389240 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd6b4071270 0x7fd6b4105520 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fd6a40098e0 tx=0x7fd6a4009250 comp rx=0 tx=0).stop 2024-04-12T20:44:19.075 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:18.035+0000 7fd6b9a00700 1 -- 172.21.15.18:0/1743389240 shutdown_connections 2024-04-12T20:44:19.079 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:18.035+0000 7fd6b9a00700 1 --2- 172.21.15.18:0/1743389240 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd6b4071270 0x7fd6b4105520 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:44:19.079 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:18.035+0000 7fd6b9a00700 1 -- 172.21.15.18:0/1743389240 >> 172.21.15.18:0/1743389240 conn(0x7fd6b40fcfb0 msgr2=0x7fd6b40ff3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T20:44:19.079 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:18.035+0000 7fd6b9a00700 1 -- 172.21.15.18:0/1743389240 shutdown_connections 2024-04-12T20:44:19.080 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:18.035+0000 7fd6b9a00700 1 -- 172.21.15.18:0/1743389240 wait complete. 2024-04-12T20:44:19.080 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:18.036+0000 7fd6b9a00700 1 Processor -- start 2024-04-12T20:44:19.080 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:18.036+0000 7fd6b9a00700 1 -- start start 2024-04-12T20:44:19.080 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:18.037+0000 7fd6b9a00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd6b4071270 0x7fd6b4113980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T20:44:19.080 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:18.037+0000 7fd6b9a00700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6b4113ec0 con 0x7fd6b4071270 2024-04-12T20:44:19.080 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:18.037+0000 7fd6b2a00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd6b4071270 0x7fd6b4113980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T20:44:19.080 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:18.037+0000 7fd6b2a00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd6b4071270 0x7fd6b4113980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:42540/0 (socket says 172.21.15.18:42540) 2024-04-12T20:44:19.080 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:18.037+0000 7fd6b2a00700 1 -- 172.21.15.18:0/2388682266 learned_addr learned my addr 172.21.15.18:0/2388682266 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T20:44:19.080 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:18.037+0000 7fd6b2a00700 1 -- 172.21.15.18:0/2388682266 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6a40094f0 con 0x7fd6b4071270 2024-04-12T20:44:19.080 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:18.038+0000 7fd6b2a00700 1 --2- 172.21.15.18:0/2388682266 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd6b4071270 0x7fd6b4113980 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fd6a40094c0 tx=0x7fd6a4009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T20:44:19.080 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:18.038+0000 7fd6aaa00700 1 -- 172.21.15.18:0/2388682266 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fd6a40118f0 con 0x7fd6b4071270 2024-04-12T20:44:19.080 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:18.038+0000 7fd6aaa00700 1 -- 172.21.15.18:0/2388682266 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd6a4010d60 con 0x7fd6b4071270 2024-04-12T20:44:19.080 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:18.038+0000 7fd6aaa00700 1 -- 172.21.15.18:0/2388682266 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fd6a401a430 con 0x7fd6b4071270 2024-04-12T20:44:19.080 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:18.038+0000 7fd6b9a00700 1 -- 172.21.15.18:0/2388682266 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd6b41140c0 con 0x7fd6b4071270 2024-04-12T20:44:19.081 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:18.038+0000 7fd6b9a00700 1 -- 172.21.15.18:0/2388682266 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd6b4114c30 con 0x7fd6b4071270 2024-04-12T20:44:19.081 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:18.039+0000 7fd6b9a00700 1 -- 172.21.15.18:0/2388682266 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd6b405f5f0 con 0x7fd6b4071270 2024-04-12T20:44:19.081 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:18.040+0000 7fd6aaa00700 1 -- 172.21.15.18:0/2388682266 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 12) v1 ==== 43381+0+0 (secure 0 0 0) 0x7fd6a4024440 con 0x7fd6b4071270 2024-04-12T20:44:19.081 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:18.040+0000 7fd6aaa00700 1 --2- 172.21.15.18:0/2388682266 >> [v2:172.21.15.18:6800/1391635591,v1:172.21.15.18:6801/1391635591] conn(0x7fd6980367e0 0x7fd698038c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T20:44:19.081 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:18.040+0000 7fd6aaa00700 1 -- 172.21.15.18:0/2388682266 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1151+0+0 (secure 0 0 0) 0x7fd6a404a6e0 con 0x7fd6b4071270 2024-04-12T20:44:19.081 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:18.041+0000 7fd6b2000700 1 --2- 172.21.15.18:0/2388682266 >> [v2:172.21.15.18:6800/1391635591,v1:172.21.15.18:6801/1391635591] conn(0x7fd6980367e0 0x7fd698038c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T20:44:19.081 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:18.041+0000 7fd6b2000700 1 --2- 172.21.15.18:0/2388682266 >> [v2:172.21.15.18:6800/1391635591,v1:172.21.15.18:6801/1391635591] conn(0x7fd6980367e0 0x7fd698038c90 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fd6a0009990 tx=0x7fd6a0006e40 comp rx=0 tx=0).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T20:44:19.081 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:18.046+0000 7fd6aaa00700 1 -- 172.21.15.18:0/2388682266 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7fd6a4024760 con 0x7fd6b4071270 2024-04-12T20:44:19.081 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:18.269+0000 7fd6b9a00700 1 -- 172.21.15.18:0/2388682266 --> [v2:172.21.15.18:6800/1391635591,v1:172.21.15.18:6801/1391635591] -- mgr_command(tid 0: {"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}) v1 -- 0x7fd6b40fe340 con 0x7fd6980367e0 2024-04-12T20:44:19.081 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:18.450+0000 7fd6aaa00700 1 -- 172.21.15.18:0/2388682266 <== mgr.14162 v2:172.21.15.18:6800/1391635591 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+32 (secure 0 0 0) 0x7fd6b40fe340 con 0x7fd6980367e0 2024-04-12T20:44:19.081 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:18.455+0000 7fd6b9a00700 1 -- 172.21.15.18:0/2388682266 >> [v2:172.21.15.18:6800/1391635591,v1:172.21.15.18:6801/1391635591] conn(0x7fd6980367e0 msgr2=0x7fd698038c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:44:19.081 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:18.455+0000 7fd6b9a00700 1 --2- 172.21.15.18:0/2388682266 >> [v2:172.21.15.18:6800/1391635591,v1:172.21.15.18:6801/1391635591] conn(0x7fd6980367e0 0x7fd698038c90 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fd6a0009990 tx=0x7fd6a0006e40 comp rx=0 tx=0).stop 2024-04-12T20:44:19.081 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:18.455+0000 7fd6b9a00700 1 -- 172.21.15.18:0/2388682266 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd6b4071270 msgr2=0x7fd6b4113980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:44:19.081 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:18.455+0000 7fd6b9a00700 1 --2- 172.21.15.18:0/2388682266 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd6b4071270 0x7fd6b4113980 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fd6a40094c0 tx=0x7fd6a4009250 comp rx=0 tx=0).stop 2024-04-12T20:44:19.082 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:18.455+0000 7fd6b9a00700 1 -- 172.21.15.18:0/2388682266 shutdown_connections 2024-04-12T20:44:19.082 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:18.455+0000 7fd6b9a00700 1 --2- 172.21.15.18:0/2388682266 >> [v2:172.21.15.18:6800/1391635591,v1:172.21.15.18:6801/1391635591] conn(0x7fd6980367e0 0x7fd698038c90 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:44:19.082 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:18.455+0000 7fd6b9a00700 1 --2- 172.21.15.18:0/2388682266 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd6b4071270 0x7fd6b4113980 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:44:19.082 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:18.455+0000 7fd6b9a00700 1 -- 172.21.15.18:0/2388682266 >> 172.21.15.18:0/2388682266 conn(0x7fd6b40fcfb0 msgr2=0x7fd6b40fdc30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T20:44:19.082 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:18.455+0000 7fd6b9a00700 1 -- 172.21.15.18:0/2388682266 shutdown_connections 2024-04-12T20:44:19.082 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:18.455+0000 7fd6b9a00700 1 -- 172.21.15.18:0/2388682266 wait complete. 2024-04-12T20:44:19.082 INFO:teuthology.orchestra.run.smithi018.stdout:Creating initial admin user... 2024-04-12T20:44:19.459 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:44:19 smithi018 ceph-mon[28799]: from='client.14174 -' entity='client.admin' cmd=[{"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T20:44:19.459 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:44:19 smithi018 ceph-mon[28799]: from='mgr.14162 172.21.15.18:0/2711576919' entity='mgr.smithi018.xjzwga' 2024-04-12T20:44:19.459 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:44:19 smithi018 ceph-mon[28799]: from='mgr.14162 172.21.15.18:0/2711576919' entity='mgr.smithi018.xjzwga' 2024-04-12T20:44:19.459 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:44:19 smithi018 ceph-mon[28799]: from='mgr.14162 172.21.15.18:0/2711576919' entity='mgr.smithi018.xjzwga' 2024-04-12T20:44:21.360 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout {"username": "admin", "password": "$2b$12$hIExPvppUxVQ7wcB2tXlke.meDCyMN2tvrVID2o3MpBbfVZh7afQe", "roles": ["administrator"], "name": null, "email": null, "lastUpdate": 1712954660, "enabled": true, "pwdExpirationDate": null, "pwdUpdateRequired": true} 2024-04-12T20:44:21.360 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:19.762+0000 7f5119400700 1 Processor -- start 2024-04-12T20:44:21.360 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:19.762+0000 7f5119400700 1 -- start start 2024-04-12T20:44:21.361 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:19.762+0000 7f5119400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5114077ab0 0x7f5114076100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T20:44:21.364 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:19.762+0000 7f5119400700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5114077ec0 con 0x7f5114077ab0 2024-04-12T20:44:21.364 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:19.763+0000 7f5112800700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5114077ab0 0x7f5114076100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T20:44:21.365 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:19.763+0000 7f5112800700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5114077ab0 0x7f5114076100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:42552/0 (socket says 172.21.15.18:42552) 2024-04-12T20:44:21.365 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:19.763+0000 7f5112800700 1 -- 172.21.15.18:0/1476255755 learned_addr learned my addr 172.21.15.18:0/1476255755 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T20:44:21.365 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:19.763+0000 7f5112800700 1 -- 172.21.15.18:0/1476255755 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5114078000 con 0x7f5114077ab0 2024-04-12T20:44:21.365 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:19.764+0000 7f5112800700 1 --2- 172.21.15.18:0/1476255755 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5114077ab0 0x7f5114076100 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f51040098e0 tx=0x7f5104009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e85ceca3b7f7d23e server_cookie=0 in_seq=0 out_seq=0 2024-04-12T20:44:21.365 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:19.764+0000 7f5111e00700 1 -- 172.21.15.18:0/1476255755 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f510400b860 con 0x7f5114077ab0 2024-04-12T20:44:21.365 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:19.764+0000 7f5111e00700 1 -- 172.21.15.18:0/1476255755 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f510400be90 con 0x7f5114077ab0 2024-04-12T20:44:21.365 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:19.764+0000 7f5111e00700 1 -- 172.21.15.18:0/1476255755 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f51040103c0 con 0x7f5114077ab0 2024-04-12T20:44:21.365 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:19.765+0000 7f5119400700 1 -- 172.21.15.18:0/1476255755 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5114077ab0 msgr2=0x7f5114076100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:44:21.365 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:19.765+0000 7f5119400700 1 --2- 172.21.15.18:0/1476255755 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5114077ab0 0x7f5114076100 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f51040098e0 tx=0x7f5104009250 comp rx=0 tx=0).stop 2024-04-12T20:44:21.365 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:19.765+0000 7f5119400700 1 -- 172.21.15.18:0/1476255755 shutdown_connections 2024-04-12T20:44:21.365 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:19.765+0000 7f5119400700 1 --2- 172.21.15.18:0/1476255755 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5114077ab0 0x7f5114076100 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:44:21.365 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:19.765+0000 7f5119400700 1 -- 172.21.15.18:0/1476255755 >> 172.21.15.18:0/1476255755 conn(0x7f51140fe090 msgr2=0x7f5114100500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T20:44:21.365 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:19.765+0000 7f5119400700 1 -- 172.21.15.18:0/1476255755 shutdown_connections 2024-04-12T20:44:21.365 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:19.765+0000 7f5119400700 1 -- 172.21.15.18:0/1476255755 wait complete. 2024-04-12T20:44:21.366 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:19.766+0000 7f5119400700 1 Processor -- start 2024-04-12T20:44:21.366 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:19.766+0000 7f5119400700 1 -- start start 2024-04-12T20:44:21.366 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:19.767+0000 7f5119400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5114077ab0 0x7f5114113990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T20:44:21.366 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:19.767+0000 7f5119400700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5114113ed0 con 0x7f5114077ab0 2024-04-12T20:44:21.366 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:19.767+0000 7f5112800700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5114077ab0 0x7f5114113990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T20:44:21.366 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:19.767+0000 7f5112800700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5114077ab0 0x7f5114113990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:42558/0 (socket says 172.21.15.18:42558) 2024-04-12T20:44:21.366 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:19.767+0000 7f5112800700 1 -- 172.21.15.18:0/514605050 learned_addr learned my addr 172.21.15.18:0/514605050 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T20:44:21.366 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:19.767+0000 7f5112800700 1 -- 172.21.15.18:0/514605050 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51040094f0 con 0x7f5114077ab0 2024-04-12T20:44:21.366 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:19.768+0000 7f5112800700 1 --2- 172.21.15.18:0/514605050 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5114077ab0 0x7f5114113990 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f51040105b0 tx=0x7f5104009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T20:44:21.366 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:19.768+0000 7f510aa00700 1 -- 172.21.15.18:0/514605050 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f5104010840 con 0x7f5114077ab0 2024-04-12T20:44:21.366 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:19.768+0000 7f510aa00700 1 -- 172.21.15.18:0/514605050 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f5104010e70 con 0x7f5114077ab0 2024-04-12T20:44:21.366 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:19.768+0000 7f5119400700 1 -- 172.21.15.18:0/514605050 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f51141140d0 con 0x7f5114077ab0 2024-04-12T20:44:21.366 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:19.768+0000 7f510aa00700 1 -- 172.21.15.18:0/514605050 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f510401ae30 con 0x7f5114077ab0 2024-04-12T20:44:21.367 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:19.768+0000 7f5119400700 1 -- 172.21.15.18:0/514605050 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5114114c40 con 0x7f5114077ab0 2024-04-12T20:44:21.367 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:19.770+0000 7f510aa00700 1 -- 172.21.15.18:0/514605050 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 12) v1 ==== 43381+0+0 (secure 0 0 0) 0x7f51040109a0 con 0x7f5114077ab0 2024-04-12T20:44:21.367 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:19.770+0000 7f5119400700 1 -- 172.21.15.18:0/514605050 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f511405f5f0 con 0x7f5114077ab0 2024-04-12T20:44:21.367 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:19.770+0000 7f510aa00700 1 --2- 172.21.15.18:0/514605050 >> [v2:172.21.15.18:6800/1391635591,v1:172.21.15.18:6801/1391635591] conn(0x7f50f8036830 0x7f50f8038ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T20:44:21.369 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:19.770+0000 7f510aa00700 1 -- 172.21.15.18:0/514605050 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1151+0+0 (secure 0 0 0) 0x7f510404a2a0 con 0x7f5114077ab0 2024-04-12T20:44:21.369 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:19.771+0000 7f510be00700 1 --2- 172.21.15.18:0/514605050 >> [v2:172.21.15.18:6800/1391635591,v1:172.21.15.18:6801/1391635591] conn(0x7f50f8036830 0x7f50f8038ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T20:44:21.369 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:19.772+0000 7f510be00700 1 --2- 172.21.15.18:0/514605050 >> [v2:172.21.15.18:6800/1391635591,v1:172.21.15.18:6801/1391635591] conn(0x7f50f8036830 0x7f50f8038ce0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f50fc009990 tx=0x7f50fc006e40 comp rx=0 tx=0).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T20:44:21.369 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:19.775+0000 7f510aa00700 1 -- 172.21.15.18:0/514605050 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7f510402b4d0 con 0x7f5114077ab0 2024-04-12T20:44:21.369 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:20.021+0000 7f5119400700 1 -- 172.21.15.18:0/514605050 --> [v2:172.21.15.18:6800/1391635591,v1:172.21.15.18:6801/1391635591] -- mgr_command(tid 0: {"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}) v1 -- 0x7f511405f410 con 0x7f50f8036830 2024-04-12T20:44:21.369 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:20.288+0000 7f510aa00700 1 -- 172.21.15.18:0/514605050 <== mgr.14162 v2:172.21.15.18:6800/1391635591 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+252 (secure 0 0 0) 0x7f511405f410 con 0x7f50f8036830 2024-04-12T20:44:21.369 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:20.291+0000 7f5119400700 1 -- 172.21.15.18:0/514605050 >> [v2:172.21.15.18:6800/1391635591,v1:172.21.15.18:6801/1391635591] conn(0x7f50f8036830 msgr2=0x7f50f8038ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:44:21.369 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:20.291+0000 7f5119400700 1 --2- 172.21.15.18:0/514605050 >> [v2:172.21.15.18:6800/1391635591,v1:172.21.15.18:6801/1391635591] conn(0x7f50f8036830 0x7f50f8038ce0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f50fc009990 tx=0x7f50fc006e40 comp rx=0 tx=0).stop 2024-04-12T20:44:21.370 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:20.291+0000 7f5119400700 1 -- 172.21.15.18:0/514605050 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5114077ab0 msgr2=0x7f5114113990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:44:21.370 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:20.291+0000 7f5119400700 1 --2- 172.21.15.18:0/514605050 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5114077ab0 0x7f5114113990 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f51040105b0 tx=0x7f5104009250 comp rx=0 tx=0).stop 2024-04-12T20:44:21.370 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:20.291+0000 7f5119400700 1 -- 172.21.15.18:0/514605050 shutdown_connections 2024-04-12T20:44:21.370 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:20.291+0000 7f5119400700 1 --2- 172.21.15.18:0/514605050 >> [v2:172.21.15.18:6800/1391635591,v1:172.21.15.18:6801/1391635591] conn(0x7f50f8036830 0x7f50f8038ce0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:44:21.370 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:20.291+0000 7f5119400700 1 --2- 172.21.15.18:0/514605050 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5114077ab0 0x7f5114113990 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:44:21.370 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:20.291+0000 7f5119400700 1 -- 172.21.15.18:0/514605050 >> 172.21.15.18:0/514605050 conn(0x7f51140fe090 msgr2=0x7f51140fecf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T20:44:21.370 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:20.291+0000 7f5119400700 1 -- 172.21.15.18:0/514605050 shutdown_connections 2024-04-12T20:44:21.370 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:20.292+0000 7f5119400700 1 -- 172.21.15.18:0/514605050 wait complete. 2024-04-12T20:44:21.370 INFO:teuthology.orchestra.run.smithi018.stdout:Fetching dashboard port number... 2024-04-12T20:44:21.603 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:44:21 smithi018 ceph-mon[28799]: from='client.14176 -' entity='client.admin' cmd=[{"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T20:44:21.603 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:44:21 smithi018 ceph-mon[28799]: from='mgr.14162 172.21.15.18:0/2711576919' entity='mgr.smithi018.xjzwga' 2024-04-12T20:44:22.603 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:44:22 smithi018 ceph-mon[28799]: mgrmap e13: smithi018.xjzwga(active, since 6s) 2024-04-12T20:44:22.980 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout 8443 2024-04-12T20:44:22.980 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:22.095+0000 7f5420e00700 1 Processor -- start 2024-04-12T20:44:22.980 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:22.096+0000 7f5420e00700 1 -- start start 2024-04-12T20:44:22.980 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:22.096+0000 7f5420e00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f541c104c20 0x7f541c105030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T20:44:22.981 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:22.096+0000 7f5420e00700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f541c105600 con 0x7f541c104c20 2024-04-12T20:44:22.983 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:22.097+0000 7f5419e00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f541c104c20 0x7f541c105030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T20:44:22.983 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:22.097+0000 7f5419e00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f541c104c20 0x7f541c105030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:51352/0 (socket says 172.21.15.18:51352) 2024-04-12T20:44:22.983 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:22.097+0000 7f5419e00700 1 -- 172.21.15.18:0/2913926175 learned_addr learned my addr 172.21.15.18:0/2913926175 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T20:44:22.983 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:22.097+0000 7f5419e00700 1 -- 172.21.15.18:0/2913926175 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f541c105740 con 0x7f541c104c20 2024-04-12T20:44:22.983 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:22.097+0000 7f5419e00700 1 --2- 172.21.15.18:0/2913926175 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f541c104c20 0x7f541c105030 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f540c0098e0 tx=0x7f540c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee981e24cf6c3ec server_cookie=0 in_seq=0 out_seq=0 2024-04-12T20:44:22.983 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:22.098+0000 7f5418a00700 1 -- 172.21.15.18:0/2913926175 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f540c00b860 con 0x7f541c104c20 2024-04-12T20:44:22.983 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:22.098+0000 7f5418a00700 1 -- 172.21.15.18:0/2913926175 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f540c00be90 con 0x7f541c104c20 2024-04-12T20:44:22.983 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:22.098+0000 7f5418a00700 1 -- 172.21.15.18:0/2913926175 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f540c0103c0 con 0x7f541c104c20 2024-04-12T20:44:22.983 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:22.098+0000 7f5420e00700 1 -- 172.21.15.18:0/2913926175 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f541c104c20 msgr2=0x7f541c105030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:44:22.983 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:22.098+0000 7f5420e00700 1 --2- 172.21.15.18:0/2913926175 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f541c104c20 0x7f541c105030 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f540c0098e0 tx=0x7f540c009250 comp rx=0 tx=0).stop 2024-04-12T20:44:22.984 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:22.099+0000 7f5420e00700 1 -- 172.21.15.18:0/2913926175 shutdown_connections 2024-04-12T20:44:22.984 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:22.099+0000 7f5420e00700 1 --2- 172.21.15.18:0/2913926175 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f541c104c20 0x7f541c105030 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:44:22.984 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:22.099+0000 7f5420e00700 1 -- 172.21.15.18:0/2913926175 >> 172.21.15.18:0/2913926175 conn(0x7f541c078530 msgr2=0x7f541c078930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T20:44:22.984 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:22.099+0000 7f5420e00700 1 -- 172.21.15.18:0/2913926175 shutdown_connections 2024-04-12T20:44:22.984 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:22.099+0000 7f5420e00700 1 -- 172.21.15.18:0/2913926175 wait complete. 2024-04-12T20:44:22.984 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:22.100+0000 7f5420e00700 1 Processor -- start 2024-04-12T20:44:22.984 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:22.100+0000 7f5420e00700 1 -- start start 2024-04-12T20:44:22.984 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:22.100+0000 7f5420e00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f541c104c20 0x7f541c1138a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T20:44:22.984 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:22.100+0000 7f5420e00700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f541c113de0 con 0x7f541c104c20 2024-04-12T20:44:22.988 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:22.100+0000 7f5419e00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f541c104c20 0x7f541c1138a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T20:44:22.988 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:22.100+0000 7f5419e00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f541c104c20 0x7f541c1138a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:51362/0 (socket says 172.21.15.18:51362) 2024-04-12T20:44:22.988 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:22.100+0000 7f5419e00700 1 -- 172.21.15.18:0/1177655336 learned_addr learned my addr 172.21.15.18:0/1177655336 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T20:44:22.988 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:22.101+0000 7f5419e00700 1 -- 172.21.15.18:0/1177655336 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f540c0094f0 con 0x7f541c104c20 2024-04-12T20:44:22.988 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:22.101+0000 7f5419e00700 1 --2- 172.21.15.18:0/1177655336 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f541c104c20 0x7f541c1138a0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f540c0094c0 tx=0x7f540c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T20:44:22.988 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:22.101+0000 7f5412000700 1 -- 172.21.15.18:0/1177655336 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f540c010890 con 0x7f541c104c20 2024-04-12T20:44:22.988 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:22.101+0000 7f5412000700 1 -- 172.21.15.18:0/1177655336 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f540c010ec0 con 0x7f541c104c20 2024-04-12T20:44:22.988 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:22.101+0000 7f5412000700 1 -- 172.21.15.18:0/1177655336 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f540c0243f0 con 0x7f541c104c20 2024-04-12T20:44:22.988 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:22.101+0000 7f5420e00700 1 -- 172.21.15.18:0/1177655336 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f541c113fe0 con 0x7f541c104c20 2024-04-12T20:44:22.988 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:22.102+0000 7f5420e00700 1 -- 172.21.15.18:0/1177655336 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f541c114b50 con 0x7f541c104c20 2024-04-12T20:44:22.988 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:22.103+0000 7f5412000700 1 -- 172.21.15.18:0/1177655336 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 13) v1 ==== 43424+0+0 (secure 0 0 0) 0x7f540c01a7e0 con 0x7f541c104c20 2024-04-12T20:44:22.988 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:22.103+0000 7f5420e00700 1 -- 172.21.15.18:0/1177655336 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f541c05f5f0 con 0x7f541c104c20 2024-04-12T20:44:22.989 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:22.103+0000 7f5412000700 1 --2- 172.21.15.18:0/1177655336 >> [v2:172.21.15.18:6800/1391635591,v1:172.21.15.18:6801/1391635591] conn(0x7f54000368d0 0x7f5400038d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T20:44:22.989 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:22.103+0000 7f5412000700 1 -- 172.21.15.18:0/1177655336 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1151+0+0 (secure 0 0 0) 0x7f540c04a4f0 con 0x7f541c104c20 2024-04-12T20:44:22.989 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:22.103+0000 7f5419400700 1 --2- 172.21.15.18:0/1177655336 >> [v2:172.21.15.18:6800/1391635591,v1:172.21.15.18:6801/1391635591] conn(0x7f54000368d0 0x7f5400038d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T20:44:22.989 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:22.104+0000 7f5419400700 1 --2- 172.21.15.18:0/1177655336 >> [v2:172.21.15.18:6800/1391635591,v1:172.21.15.18:6801/1391635591] conn(0x7f54000368d0 0x7f5400038d80 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f5408009990 tx=0x7f5408006e40 comp rx=0 tx=0).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T20:44:22.989 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:22.109+0000 7f5412000700 1 -- 172.21.15.18:0/1177655336 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7f540c02b8e0 con 0x7f541c104c20 2024-04-12T20:44:22.989 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:22.326+0000 7f5420e00700 1 -- 172.21.15.18:0/1177655336 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"} v 0) v1 -- 0x7f541c04bed0 con 0x7f541c104c20 2024-04-12T20:44:22.989 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:22.327+0000 7f5412000700 1 -- 172.21.15.18:0/1177655336 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]=0 v8) v1 ==== 112+0+5 (secure 0 0 0) 0x7f540c02b2f0 con 0x7f541c104c20 2024-04-12T20:44:22.989 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:22.329+0000 7f5420e00700 1 -- 172.21.15.18:0/1177655336 >> [v2:172.21.15.18:6800/1391635591,v1:172.21.15.18:6801/1391635591] conn(0x7f54000368d0 msgr2=0x7f5400038d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:44:22.989 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:22.329+0000 7f5420e00700 1 --2- 172.21.15.18:0/1177655336 >> [v2:172.21.15.18:6800/1391635591,v1:172.21.15.18:6801/1391635591] conn(0x7f54000368d0 0x7f5400038d80 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f5408009990 tx=0x7f5408006e40 comp rx=0 tx=0).stop 2024-04-12T20:44:22.989 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:22.329+0000 7f5420e00700 1 -- 172.21.15.18:0/1177655336 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f541c104c20 msgr2=0x7f541c1138a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:44:22.989 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:22.329+0000 7f5420e00700 1 --2- 172.21.15.18:0/1177655336 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f541c104c20 0x7f541c1138a0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f540c0094c0 tx=0x7f540c009250 comp rx=0 tx=0).stop 2024-04-12T20:44:22.989 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:22.330+0000 7f5420e00700 1 -- 172.21.15.18:0/1177655336 shutdown_connections 2024-04-12T20:44:22.989 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:22.330+0000 7f5420e00700 1 --2- 172.21.15.18:0/1177655336 >> [v2:172.21.15.18:6800/1391635591,v1:172.21.15.18:6801/1391635591] conn(0x7f54000368d0 0x7f5400038d80 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:44:22.990 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:22.330+0000 7f5420e00700 1 --2- 172.21.15.18:0/1177655336 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f541c104c20 0x7f541c1138a0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:44:22.990 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:22.330+0000 7f5420e00700 1 -- 172.21.15.18:0/1177655336 >> 172.21.15.18:0/1177655336 conn(0x7f541c078530 msgr2=0x7f541c1024f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T20:44:22.990 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:22.330+0000 7f5420e00700 1 -- 172.21.15.18:0/1177655336 shutdown_connections 2024-04-12T20:44:22.990 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:22.330+0000 7f5420e00700 1 -- 172.21.15.18:0/1177655336 wait complete. 2024-04-12T20:44:23.019 INFO:teuthology.orchestra.run.smithi018.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-04-12T20:44:23.019 INFO:teuthology.orchestra.run.smithi018.stdout:systemctl: stdout disabled 2024-04-12T20:44:23.043 INFO:teuthology.orchestra.run.smithi018.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-04-12T20:44:23.043 INFO:teuthology.orchestra.run.smithi018.stdout:systemctl: stdout inactive 2024-04-12T20:44:23.043 INFO:teuthology.orchestra.run.smithi018.stdout:firewalld.service is not enabled 2024-04-12T20:44:23.043 INFO:teuthology.orchestra.run.smithi018.stdout:Not possible to open ports <[8443]>. firewalld.service is not available 2024-04-12T20:44:23.051 INFO:teuthology.orchestra.run.smithi018.stdout:Ceph Dashboard is now available at: 2024-04-12T20:44:23.051 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-12T20:44:23.051 INFO:teuthology.orchestra.run.smithi018.stdout: URL: https://smithi018.front.sepia.ceph.com:8443/ 2024-04-12T20:44:23.051 INFO:teuthology.orchestra.run.smithi018.stdout: User: admin 2024-04-12T20:44:23.051 INFO:teuthology.orchestra.run.smithi018.stdout: Password: nwgmwhzo3s 2024-04-12T20:44:23.051 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-12T20:44:23.052 INFO:teuthology.orchestra.run.smithi018.stdout:Saving cluster configuration to /var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/config directory 2024-04-12T20:44:23.054 INFO:teuthology.orchestra.run.smithi018.stdout:Enabling autotune for osd_memory_target 2024-04-12T20:44:23.559 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:44:23 smithi018 ceph-mon[28799]: from='client.? 172.21.15.18:0/1177655336' entity='client.admin' cmd=[{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]: dispatch 2024-04-12T20:44:24.772 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:23.730+0000 7f3ff9200700 1 Processor -- start 2024-04-12T20:44:24.773 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:23.730+0000 7f3ff9200700 1 -- start start 2024-04-12T20:44:24.773 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:23.731+0000 7f3ff9200700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3ff41017f0 0x7f3ff4101c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T20:44:24.773 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:23.731+0000 7f3ff9200700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ff4102140 con 0x7f3ff41017f0 2024-04-12T20:44:24.773 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:23.731+0000 7f3ff1e00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3ff41017f0 0x7f3ff4101c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T20:44:24.773 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:23.731+0000 7f3ff1e00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3ff41017f0 0x7f3ff4101c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:51364/0 (socket says 172.21.15.18:51364) 2024-04-12T20:44:24.778 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:23.731+0000 7f3ff1e00700 1 -- 172.21.15.18:0/4112962140 learned_addr learned my addr 172.21.15.18:0/4112962140 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T20:44:24.778 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:23.732+0000 7f3ff1e00700 1 -- 172.21.15.18:0/4112962140 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ff4102280 con 0x7f3ff41017f0 2024-04-12T20:44:24.778 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:23.732+0000 7f3ff1e00700 1 --2- 172.21.15.18:0/4112962140 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3ff41017f0 0x7f3ff4101c00 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f3fe400bb80 tx=0x7f3fe400b4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e0f03b55f3f60a3 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T20:44:24.778 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:23.732+0000 7f3ff0a00700 1 -- 172.21.15.18:0/4112962140 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f3fe400dd00 con 0x7f3ff41017f0 2024-04-12T20:44:24.778 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:23.733+0000 7f3ff0a00700 1 -- 172.21.15.18:0/4112962140 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f3fe400de60 con 0x7f3ff41017f0 2024-04-12T20:44:24.778 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:23.733+0000 7f3ff0a00700 1 -- 172.21.15.18:0/4112962140 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f3fe4018920 con 0x7f3ff41017f0 2024-04-12T20:44:24.778 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:23.733+0000 7f3ff9200700 1 -- 172.21.15.18:0/4112962140 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3ff41017f0 msgr2=0x7f3ff4101c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:44:24.778 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:23.733+0000 7f3ff9200700 1 --2- 172.21.15.18:0/4112962140 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3ff41017f0 0x7f3ff4101c00 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f3fe400bb80 tx=0x7f3fe400b4f0 comp rx=0 tx=0).stop 2024-04-12T20:44:24.778 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:23.733+0000 7f3ff9200700 1 -- 172.21.15.18:0/4112962140 shutdown_connections 2024-04-12T20:44:24.778 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:23.733+0000 7f3ff9200700 1 --2- 172.21.15.18:0/4112962140 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3ff41017f0 0x7f3ff4101c00 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:44:24.779 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:23.733+0000 7f3ff9200700 1 -- 172.21.15.18:0/4112962140 >> 172.21.15.18:0/4112962140 conn(0x7f3ff40fcfb0 msgr2=0x7f3ff40ff3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T20:44:24.779 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:23.734+0000 7f3ff9200700 1 -- 172.21.15.18:0/4112962140 shutdown_connections 2024-04-12T20:44:24.779 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:23.734+0000 7f3ff9200700 1 -- 172.21.15.18:0/4112962140 wait complete. 2024-04-12T20:44:24.779 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:23.734+0000 7f3ff9200700 1 Processor -- start 2024-04-12T20:44:24.779 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:23.735+0000 7f3ff9200700 1 -- start start 2024-04-12T20:44:24.779 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:23.735+0000 7f3ff9200700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3ff41017f0 0x7f3ff41138c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T20:44:24.779 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:23.735+0000 7f3ff9200700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ff4113e00 con 0x7f3ff41017f0 2024-04-12T20:44:24.779 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:23.735+0000 7f3ff1e00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3ff41017f0 0x7f3ff41138c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T20:44:24.779 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:23.735+0000 7f3ff1e00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3ff41017f0 0x7f3ff41138c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:51376/0 (socket says 172.21.15.18:51376) 2024-04-12T20:44:24.779 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:23.735+0000 7f3ff1e00700 1 -- 172.21.15.18:0/4183839474 learned_addr learned my addr 172.21.15.18:0/4183839474 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T20:44:24.779 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:23.736+0000 7f3ff1e00700 1 -- 172.21.15.18:0/4183839474 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3fe400b790 con 0x7f3ff41017f0 2024-04-12T20:44:24.779 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:23.736+0000 7f3ff1e00700 1 --2- 172.21.15.18:0/4183839474 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3ff41017f0 0x7f3ff41138c0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f3fe40106a0 tx=0x7f3fe400b4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T20:44:24.779 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:23.736+0000 7f3fea000700 1 -- 172.21.15.18:0/4183839474 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f3fe400dc70 con 0x7f3ff41017f0 2024-04-12T20:44:24.779 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:23.736+0000 7f3fea000700 1 -- 172.21.15.18:0/4183839474 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f3fe4010c30 con 0x7f3ff41017f0 2024-04-12T20:44:24.779 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:23.736+0000 7f3fea000700 1 -- 172.21.15.18:0/4183839474 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f3fe4021550 con 0x7f3ff41017f0 2024-04-12T20:44:24.780 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:23.736+0000 7f3ff9200700 1 -- 172.21.15.18:0/4183839474 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3ff4114000 con 0x7f3ff41017f0 2024-04-12T20:44:24.780 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:23.737+0000 7f3ff9200700 1 -- 172.21.15.18:0/4183839474 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3ff4114b70 con 0x7f3ff41017f0 2024-04-12T20:44:24.780 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:23.738+0000 7f3fea000700 1 -- 172.21.15.18:0/4183839474 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 13) v1 ==== 43424+0+0 (secure 0 0 0) 0x7f3fe40223f0 con 0x7f3ff41017f0 2024-04-12T20:44:24.780 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:23.738+0000 7f3ff9200700 1 -- 172.21.15.18:0/4183839474 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3ff405f5f0 con 0x7f3ff41017f0 2024-04-12T20:44:24.780 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:23.738+0000 7f3fea000700 1 --2- 172.21.15.18:0/4183839474 >> [v2:172.21.15.18:6800/1391635591,v1:172.21.15.18:6801/1391635591] conn(0x7f3fd8036920 0x7f3fd8038dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T20:44:24.780 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:23.738+0000 7f3fea000700 1 -- 172.21.15.18:0/4183839474 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1151+0+0 (secure 0 0 0) 0x7f3fe4014070 con 0x7f3ff41017f0 2024-04-12T20:44:24.780 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:23.739+0000 7f3ff1400700 1 --2- 172.21.15.18:0/4183839474 >> [v2:172.21.15.18:6800/1391635591,v1:172.21.15.18:6801/1391635591] conn(0x7f3fd8036920 0x7f3fd8038dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T20:44:24.780 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:23.739+0000 7f3ff1400700 1 --2- 172.21.15.18:0/4183839474 >> [v2:172.21.15.18:6800/1391635591,v1:172.21.15.18:6801/1391635591] conn(0x7f3fd8036920 0x7f3fd8038dd0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f3fe0009990 tx=0x7f3fe0006e40 comp rx=0 tx=0).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T20:44:24.780 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:23.744+0000 7f3fea000700 1 -- 172.21.15.18:0/4183839474 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7f3fe40217c0 con 0x7f3ff41017f0 2024-04-12T20:44:24.780 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:23.962+0000 7f3ff9200700 1 -- 172.21.15.18:0/4183839474 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command([{prefix=config set, name=osd_memory_target_autotune}] v 0) v1 -- 0x7f3ff404bed0 con 0x7f3ff41017f0 2024-04-12T20:44:24.780 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:23.963+0000 7f3fea000700 1 -- 172.21.15.18:0/4183839474 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{prefix=config set, name=osd_memory_target_autotune}]=0 v8)=0 v8) v1 ==== 127+0+0 (secure 0 0 0) 0x7f3fe400e7d0 con 0x7f3ff41017f0 2024-04-12T20:44:24.780 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:23.965+0000 7f3ff9200700 1 -- 172.21.15.18:0/4183839474 >> [v2:172.21.15.18:6800/1391635591,v1:172.21.15.18:6801/1391635591] conn(0x7f3fd8036920 msgr2=0x7f3fd8038dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:44:24.780 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:23.965+0000 7f3ff9200700 1 --2- 172.21.15.18:0/4183839474 >> [v2:172.21.15.18:6800/1391635591,v1:172.21.15.18:6801/1391635591] conn(0x7f3fd8036920 0x7f3fd8038dd0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f3fe0009990 tx=0x7f3fe0006e40 comp rx=0 tx=0).stop 2024-04-12T20:44:24.780 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:23.965+0000 7f3ff9200700 1 -- 172.21.15.18:0/4183839474 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3ff41017f0 msgr2=0x7f3ff41138c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:44:24.780 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:23.965+0000 7f3ff9200700 1 --2- 172.21.15.18:0/4183839474 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3ff41017f0 0x7f3ff41138c0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f3fe40106a0 tx=0x7f3fe400b4f0 comp rx=0 tx=0).stop 2024-04-12T20:44:24.780 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:23.965+0000 7f3ff9200700 1 -- 172.21.15.18:0/4183839474 shutdown_connections 2024-04-12T20:44:24.781 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:23.965+0000 7f3ff9200700 1 --2- 172.21.15.18:0/4183839474 >> [v2:172.21.15.18:6800/1391635591,v1:172.21.15.18:6801/1391635591] conn(0x7f3fd8036920 0x7f3fd8038dd0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:44:24.781 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:23.965+0000 7f3ff9200700 1 --2- 172.21.15.18:0/4183839474 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3ff41017f0 0x7f3ff41138c0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:44:24.781 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:23.965+0000 7f3ff9200700 1 -- 172.21.15.18:0/4183839474 >> 172.21.15.18:0/4183839474 conn(0x7f3ff40fcfb0 msgr2=0x7f3ff410a090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T20:44:24.781 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:23.966+0000 7f3ff9200700 1 -- 172.21.15.18:0/4183839474 shutdown_connections 2024-04-12T20:44:24.781 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:23.966+0000 7f3ff9200700 1 -- 172.21.15.18:0/4183839474 wait complete. 2024-04-12T20:44:26.580 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:25.507+0000 7f26f3e00700 1 Processor -- start 2024-04-12T20:44:26.580 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:25.508+0000 7f26f3e00700 1 -- start start 2024-04-12T20:44:26.580 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:25.508+0000 7f26f3e00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f26f41027b0 0x7f26f4102bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T20:44:26.580 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:25.508+0000 7f26f3e00700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26f4103100 con 0x7f26f41027b0 2024-04-12T20:44:26.580 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:25.509+0000 7f26f2a00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f26f41027b0 0x7f26f4102bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T20:44:26.580 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:25.509+0000 7f26f2a00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f26f41027b0 0x7f26f4102bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:51388/0 (socket says 172.21.15.18:51388) 2024-04-12T20:44:26.580 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:25.509+0000 7f26f2a00700 1 -- 172.21.15.18:0/2044061384 learned_addr learned my addr 172.21.15.18:0/2044061384 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T20:44:26.580 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:25.509+0000 7f26f2a00700 1 -- 172.21.15.18:0/2044061384 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26f4103240 con 0x7f26f41027b0 2024-04-12T20:44:26.580 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:25.509+0000 7f26f2a00700 1 --2- 172.21.15.18:0/2044061384 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f26f41027b0 0x7f26f4102bc0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f26e40098e0 tx=0x7f26e4009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=934b8513b8f0166a server_cookie=0 in_seq=0 out_seq=0 2024-04-12T20:44:26.584 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:25.510+0000 7f26f2000700 1 -- 172.21.15.18:0/2044061384 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f26e400b8f0 con 0x7f26f41027b0 2024-04-12T20:44:26.584 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:25.510+0000 7f26f2000700 1 -- 172.21.15.18:0/2044061384 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f26e4011420 con 0x7f26f41027b0 2024-04-12T20:44:26.584 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:25.510+0000 7f26f2000700 1 -- 172.21.15.18:0/2044061384 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f26e4010540 con 0x7f26f41027b0 2024-04-12T20:44:26.584 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:25.510+0000 7f26f3e00700 1 -- 172.21.15.18:0/2044061384 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f26f41027b0 msgr2=0x7f26f4102bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:44:26.584 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:25.510+0000 7f26f3e00700 1 --2- 172.21.15.18:0/2044061384 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f26f41027b0 0x7f26f4102bc0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f26e40098e0 tx=0x7f26e4009250 comp rx=0 tx=0).stop 2024-04-12T20:44:26.584 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:25.511+0000 7f26f3e00700 1 -- 172.21.15.18:0/2044061384 shutdown_connections 2024-04-12T20:44:26.584 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:25.511+0000 7f26f3e00700 1 --2- 172.21.15.18:0/2044061384 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f26f41027b0 0x7f26f4102bc0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:44:26.584 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:25.511+0000 7f26f3e00700 1 -- 172.21.15.18:0/2044061384 >> 172.21.15.18:0/2044061384 conn(0x7f26f40fdef0 msgr2=0x7f26f4100340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T20:44:26.585 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:25.511+0000 7f26f3e00700 1 -- 172.21.15.18:0/2044061384 shutdown_connections 2024-04-12T20:44:26.585 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:25.511+0000 7f26f3e00700 1 -- 172.21.15.18:0/2044061384 wait complete. 2024-04-12T20:44:26.585 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:25.512+0000 7f26f3e00700 1 Processor -- start 2024-04-12T20:44:26.585 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:25.512+0000 7f26f3e00700 1 -- start start 2024-04-12T20:44:26.585 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:25.512+0000 7f26f3e00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f26f41027b0 0x7f26f4113770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T20:44:26.585 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:25.512+0000 7f26f3e00700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26f4113cb0 con 0x7f26f41027b0 2024-04-12T20:44:26.585 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:25.513+0000 7f26f2a00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f26f41027b0 0x7f26f4113770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T20:44:26.585 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:25.513+0000 7f26f2a00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f26f41027b0 0x7f26f4113770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:51398/0 (socket says 172.21.15.18:51398) 2024-04-12T20:44:26.585 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:25.513+0000 7f26f2a00700 1 -- 172.21.15.18:0/236467164 learned_addr learned my addr 172.21.15.18:0/236467164 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T20:44:26.585 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:25.513+0000 7f26f2a00700 1 -- 172.21.15.18:0/236467164 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26e40094f0 con 0x7f26f41027b0 2024-04-12T20:44:26.585 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:25.513+0000 7f26f2a00700 1 --2- 172.21.15.18:0/236467164 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f26f41027b0 0x7f26f4113770 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f26e40090b0 tx=0x7f26e4009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T20:44:26.585 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:25.513+0000 7f26eb400700 1 -- 172.21.15.18:0/236467164 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f26e40107b0 con 0x7f26f41027b0 2024-04-12T20:44:26.585 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:25.514+0000 7f26eb400700 1 -- 172.21.15.18:0/236467164 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f26e4010de0 con 0x7f26f41027b0 2024-04-12T20:44:26.585 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:25.514+0000 7f26eb400700 1 -- 172.21.15.18:0/236467164 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f26e401ae50 con 0x7f26f41027b0 2024-04-12T20:44:26.586 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:25.514+0000 7f26f3e00700 1 -- 172.21.15.18:0/236467164 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f26f4113eb0 con 0x7f26f41027b0 2024-04-12T20:44:26.586 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:25.514+0000 7f26f3e00700 1 -- 172.21.15.18:0/236467164 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f26f4114a20 con 0x7f26f41027b0 2024-04-12T20:44:26.586 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:25.515+0000 7f26eb400700 1 -- 172.21.15.18:0/236467164 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 13) v1 ==== 43424+0+0 (secure 0 0 0) 0x7f26e401a430 con 0x7f26f41027b0 2024-04-12T20:44:26.586 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:25.515+0000 7f26f3e00700 1 -- 172.21.15.18:0/236467164 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f26f405f5f0 con 0x7f26f41027b0 2024-04-12T20:44:26.586 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:25.515+0000 7f26eb400700 1 --2- 172.21.15.18:0/236467164 >> [v2:172.21.15.18:6800/1391635591,v1:172.21.15.18:6801/1391635591] conn(0x7f26d8036910 0x7f26d8038dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T20:44:26.586 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:25.515+0000 7f26eb400700 1 -- 172.21.15.18:0/236467164 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1151+0+0 (secure 0 0 0) 0x7f26e404a2a0 con 0x7f26f41027b0 2024-04-12T20:44:26.586 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:25.516+0000 7f26ea000700 1 --2- 172.21.15.18:0/236467164 >> [v2:172.21.15.18:6800/1391635591,v1:172.21.15.18:6801/1391635591] conn(0x7f26d8036910 0x7f26d8038dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T20:44:26.586 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:25.516+0000 7f26ea000700 1 --2- 172.21.15.18:0/236467164 >> [v2:172.21.15.18:6800/1391635591,v1:172.21.15.18:6801/1391635591] conn(0x7f26d8036910 0x7f26d8038dc0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f26dc009990 tx=0x7f26dc006e40 comp rx=0 tx=0).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T20:44:26.586 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:25.521+0000 7f26eb400700 1 -- 172.21.15.18:0/236467164 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7f26e402b8c0 con 0x7f26f41027b0 2024-04-12T20:44:26.586 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:25.856+0000 7f26f3e00700 1 -- 172.21.15.18:0/236467164 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command([{prefix=config-key set, key=mgr/dashboard/cluster/status}] v 0) v1 -- 0x7f26f404bed0 con 0x7f26f41027b0 2024-04-12T20:44:26.588 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:25.862+0000 7f26eb400700 1 -- 172.21.15.18:0/236467164 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{prefix=config-key set, key=mgr/dashboard/cluster/status}]=0 set mgr/dashboard/cluster/status v33)=0 set mgr/dashboard/cluster/status v33) v1 ==== 153+0+0 (secure 0 0 0) 0x7f26e402b2f0 con 0x7f26f41027b0 2024-04-12T20:44:26.588 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:25.865+0000 7f26f3e00700 1 -- 172.21.15.18:0/236467164 >> [v2:172.21.15.18:6800/1391635591,v1:172.21.15.18:6801/1391635591] conn(0x7f26d8036910 msgr2=0x7f26d8038dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:44:26.588 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:25.865+0000 7f26f3e00700 1 --2- 172.21.15.18:0/236467164 >> [v2:172.21.15.18:6800/1391635591,v1:172.21.15.18:6801/1391635591] conn(0x7f26d8036910 0x7f26d8038dc0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f26dc009990 tx=0x7f26dc006e40 comp rx=0 tx=0).stop 2024-04-12T20:44:26.588 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:25.865+0000 7f26f3e00700 1 -- 172.21.15.18:0/236467164 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f26f41027b0 msgr2=0x7f26f4113770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:44:26.588 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:25.865+0000 7f26f3e00700 1 --2- 172.21.15.18:0/236467164 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f26f41027b0 0x7f26f4113770 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f26e40090b0 tx=0x7f26e4009250 comp rx=0 tx=0).stop 2024-04-12T20:44:26.588 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:25.865+0000 7f26f3e00700 1 -- 172.21.15.18:0/236467164 shutdown_connections 2024-04-12T20:44:26.588 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:25.865+0000 7f26f3e00700 1 --2- 172.21.15.18:0/236467164 >> [v2:172.21.15.18:6800/1391635591,v1:172.21.15.18:6801/1391635591] conn(0x7f26d8036910 0x7f26d8038dc0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:44:26.589 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:25.865+0000 7f26f3e00700 1 --2- 172.21.15.18:0/236467164 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f26f41027b0 0x7f26f4113770 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:44:26.589 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:25.865+0000 7f26f3e00700 1 -- 172.21.15.18:0/236467164 >> 172.21.15.18:0/236467164 conn(0x7f26f40fdef0 msgr2=0x7f26f40fea10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T20:44:26.589 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:25.865+0000 7f26f3e00700 1 -- 172.21.15.18:0/236467164 shutdown_connections 2024-04-12T20:44:26.589 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr 2024-04-12T20:44:25.866+0000 7f26f3e00700 1 -- 172.21.15.18:0/236467164 wait complete. 2024-04-12T20:44:26.589 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr set mgr/dashboard/cluster/status 2024-04-12T20:44:26.589 INFO:teuthology.orchestra.run.smithi018.stdout:You can access the Ceph CLI as following in case of multi-cluster or non-default config: 2024-04-12T20:44:26.589 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-12T20:44:26.589 INFO:teuthology.orchestra.run.smithi018.stdout: sudo /home/ubuntu/cephtest/cephadm shell --fsid 1cbe407a-f90d-11ee-bc8d-c7b262605968 -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring 2024-04-12T20:44:26.589 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-12T20:44:26.589 INFO:teuthology.orchestra.run.smithi018.stdout:Or, if you are only running a single cluster on this host: 2024-04-12T20:44:26.589 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-12T20:44:26.589 INFO:teuthology.orchestra.run.smithi018.stdout: sudo /home/ubuntu/cephtest/cephadm shell 2024-04-12T20:44:26.589 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-12T20:44:26.589 INFO:teuthology.orchestra.run.smithi018.stdout:Please consider enabling telemetry to help improve Ceph: 2024-04-12T20:44:26.590 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-12T20:44:26.590 INFO:teuthology.orchestra.run.smithi018.stdout: ceph telemetry on 2024-04-12T20:44:26.590 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-12T20:44:26.590 INFO:teuthology.orchestra.run.smithi018.stdout:For more information see: 2024-04-12T20:44:26.590 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-12T20:44:26.590 INFO:teuthology.orchestra.run.smithi018.stdout: https://docs.ceph.com/docs/master/mgr/telemetry/ 2024-04-12T20:44:26.590 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-12T20:44:26.590 INFO:teuthology.orchestra.run.smithi018.stdout:Bootstrap complete. 2024-04-12T20:44:26.719 INFO:tasks.cephadm:Fetching config... 2024-04-12T20:44:26.719 DEBUG:teuthology.orchestra.run.smithi018:> set -ex 2024-04-12T20:44:26.719 DEBUG:teuthology.orchestra.run.smithi018:> dd if=/etc/ceph/ceph.conf of=/dev/stdout 2024-04-12T20:44:26.804 INFO:tasks.cephadm:Fetching client.admin keyring... 2024-04-12T20:44:26.804 DEBUG:teuthology.orchestra.run.smithi018:> set -ex 2024-04-12T20:44:26.804 DEBUG:teuthology.orchestra.run.smithi018:> dd if=/etc/ceph/ceph.client.admin.keyring of=/dev/stdout 2024-04-12T20:44:26.893 INFO:tasks.cephadm:Fetching mon keyring... 2024-04-12T20:44:26.893 DEBUG:teuthology.orchestra.run.smithi018:> set -ex 2024-04-12T20:44:26.893 DEBUG:teuthology.orchestra.run.smithi018:> sudo dd if=/var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/mon.smithi018/keyring of=/dev/stdout 2024-04-12T20:44:27.031 INFO:tasks.cephadm:Fetching pub ssh key... 2024-04-12T20:44:27.032 DEBUG:teuthology.orchestra.run.smithi018:> set -ex 2024-04-12T20:44:27.032 DEBUG:teuthology.orchestra.run.smithi018:> dd if=/home/ubuntu/cephtest/ceph.pub of=/dev/stdout 2024-04-12T20:44:27.083 INFO:tasks.cephadm:Installing pub ssh key for root users... 2024-04-12T20:44:27.083 DEBUG:teuthology.orchestra.run.smithi018:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDa2TaF+4LzQRv/R3PRY5IlWt3/AQKzjtsznp8j5IwgrPKCMpavq2mOHgOpPo2CU0RVf7c9TXM1TQWklow1cM/yJpCG0lc6NcjAdw87bb2BAZQVlYVBn8NP5prb0iMjlv1FjMgN96KMcNrDXtBD8RYvx/ozDqcc8hTP7wncSt6/z5zJttr+u/ExQvMSdid/8SbpH42wtSzSQhrwmiQByagI4Rr1R0DfLOQA4g0vsJqN1NxsWdNxu7G13y8bpliBKG8s9rNE8J4d4GHJcrXkSb67judK6nQKAGbiFylGsrLTX6IiLuUcPU3ypDWtIUddVlLQeeOciiHd47tUlgx62wi8NNtn6ZPqtQ3MiG0OB8hYWrH0kqrWcUVE+8JXbftVCX39J1XJE9lyHpJNc9q88S3NIv9DZE6rv2GR/+efFLPaAaaDlq8KNNIwWik/nToPORdW1zDtr6v+YHvSlR3TrKXlPsWKImAlQhnBJuL3nuINwalFX8RXyVaG8eATf2DUqyU= ceph-1cbe407a-f90d-11ee-bc8d-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-04-12T20:44:27.146 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:44:26 smithi018 ceph-mon[28799]: from='client.? 172.21.15.18:0/236467164' entity='client.admin' 2024-04-12T20:44:27.199 INFO:teuthology.orchestra.run.smithi018.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDa2TaF+4LzQRv/R3PRY5IlWt3/AQKzjtsznp8j5IwgrPKCMpavq2mOHgOpPo2CU0RVf7c9TXM1TQWklow1cM/yJpCG0lc6NcjAdw87bb2BAZQVlYVBn8NP5prb0iMjlv1FjMgN96KMcNrDXtBD8RYvx/ozDqcc8hTP7wncSt6/z5zJttr+u/ExQvMSdid/8SbpH42wtSzSQhrwmiQByagI4Rr1R0DfLOQA4g0vsJqN1NxsWdNxu7G13y8bpliBKG8s9rNE8J4d4GHJcrXkSb67judK6nQKAGbiFylGsrLTX6IiLuUcPU3ypDWtIUddVlLQeeOciiHd47tUlgx62wi8NNtn6ZPqtQ3MiG0OB8hYWrH0kqrWcUVE+8JXbftVCX39J1XJE9lyHpJNc9q88S3NIv9DZE6rv2GR/+efFLPaAaaDlq8KNNIwWik/nToPORdW1zDtr6v+YHvSlR3TrKXlPsWKImAlQhnBJuL3nuINwalFX8RXyVaG8eATf2DUqyU= ceph-1cbe407a-f90d-11ee-bc8d-c7b262605968 2024-04-12T20:44:27.258 DEBUG:teuthology.orchestra.run.smithi186:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDa2TaF+4LzQRv/R3PRY5IlWt3/AQKzjtsznp8j5IwgrPKCMpavq2mOHgOpPo2CU0RVf7c9TXM1TQWklow1cM/yJpCG0lc6NcjAdw87bb2BAZQVlYVBn8NP5prb0iMjlv1FjMgN96KMcNrDXtBD8RYvx/ozDqcc8hTP7wncSt6/z5zJttr+u/ExQvMSdid/8SbpH42wtSzSQhrwmiQByagI4Rr1R0DfLOQA4g0vsJqN1NxsWdNxu7G13y8bpliBKG8s9rNE8J4d4GHJcrXkSb67judK6nQKAGbiFylGsrLTX6IiLuUcPU3ypDWtIUddVlLQeeOciiHd47tUlgx62wi8NNtn6ZPqtQ3MiG0OB8hYWrH0kqrWcUVE+8JXbftVCX39J1XJE9lyHpJNc9q88S3NIv9DZE6rv2GR/+efFLPaAaaDlq8KNNIwWik/nToPORdW1zDtr6v+YHvSlR3TrKXlPsWKImAlQhnBJuL3nuINwalFX8RXyVaG8eATf2DUqyU= ceph-1cbe407a-f90d-11ee-bc8d-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-04-12T20:44:27.389 INFO:teuthology.orchestra.run.smithi186.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDa2TaF+4LzQRv/R3PRY5IlWt3/AQKzjtsznp8j5IwgrPKCMpavq2mOHgOpPo2CU0RVf7c9TXM1TQWklow1cM/yJpCG0lc6NcjAdw87bb2BAZQVlYVBn8NP5prb0iMjlv1FjMgN96KMcNrDXtBD8RYvx/ozDqcc8hTP7wncSt6/z5zJttr+u/ExQvMSdid/8SbpH42wtSzSQhrwmiQByagI4Rr1R0DfLOQA4g0vsJqN1NxsWdNxu7G13y8bpliBKG8s9rNE8J4d4GHJcrXkSb67judK6nQKAGbiFylGsrLTX6IiLuUcPU3ypDWtIUddVlLQeeOciiHd47tUlgx62wi8NNtn6ZPqtQ3MiG0OB8hYWrH0kqrWcUVE+8JXbftVCX39J1XJE9lyHpJNc9q88S3NIv9DZE6rv2GR/+efFLPaAaaDlq8KNNIwWik/nToPORdW1zDtr6v+YHvSlR3TrKXlPsWKImAlQhnBJuL3nuINwalFX8RXyVaG8eATf2DUqyU= ceph-1cbe407a-f90d-11ee-bc8d-c7b262605968 2024-04-12T20:44:27.448 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1cbe407a-f90d-11ee-bc8d-c7b262605968 -- ceph config set mgr mgr/cephadm/allow_ptrace true 2024-04-12T20:44:27.911 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/mon.smithi018/config 2024-04-12T20:44:30.789 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:44:30.789+0000 7f1c89c00700 1 -- 172.21.15.18:0/235706634 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1c840fd050 msgr2=0x7f1c840fd460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:44:30.789 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:44:30.789+0000 7f1c89c00700 1 --2- 172.21.15.18:0/235706634 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1c840fd050 0x7f1c840fd460 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f1c740048c0 tx=0x7f1c74009e30 comp rx=0 tx=0).stop 2024-04-12T20:44:30.789 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:44:30.789+0000 7f1c89c00700 1 -- 172.21.15.18:0/235706634 shutdown_connections 2024-04-12T20:44:30.789 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:44:30.789+0000 7f1c89c00700 1 --2- 172.21.15.18:0/235706634 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1c840fd050 0x7f1c840fd460 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:44:30.790 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:44:30.789+0000 7f1c89c00700 1 -- 172.21.15.18:0/235706634 >> 172.21.15.18:0/235706634 conn(0x7f1c840f70d0 msgr2=0x7f1c840f9500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T20:44:30.790 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:44:30.789+0000 7f1c89c00700 1 -- 172.21.15.18:0/235706634 shutdown_connections 2024-04-12T20:44:30.790 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:44:30.789+0000 7f1c89c00700 1 -- 172.21.15.18:0/235706634 wait complete. 2024-04-12T20:44:30.790 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:44:30.790+0000 7f1c89c00700 1 Processor -- start 2024-04-12T20:44:30.790 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:44:30.790+0000 7f1c89c00700 1 -- start start 2024-04-12T20:44:30.791 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:44:30.791+0000 7f1c89c00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1c840fd050 0x7f1c8410d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T20:44:30.791 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:44:30.791+0000 7f1c89c00700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c8410d600 con 0x7f1c840fd050 2024-04-12T20:44:30.791 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:44:30.791+0000 7f1c82a00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1c840fd050 0x7f1c8410d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T20:44:30.791 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:44:30.791+0000 7f1c82a00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1c840fd050 0x7f1c8410d0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:51412/0 (socket says 172.21.15.18:51412) 2024-04-12T20:44:30.791 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:44:30.791+0000 7f1c82a00700 1 -- 172.21.15.18:0/1274413095 learned_addr learned my addr 172.21.15.18:0/1274413095 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T20:44:30.791 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:44:30.792+0000 7f1c82a00700 1 -- 172.21.15.18:0/1274413095 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c74005040 con 0x7f1c840fd050 2024-04-12T20:44:30.792 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:44:30.792+0000 7f1c82a00700 1 --2- 172.21.15.18:0/1274413095 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1c840fd050 0x7f1c8410d0c0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f1c74009680 tx=0x7f1c74009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T20:44:30.792 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:44:30.792+0000 7f1c7aa00700 1 -- 172.21.15.18:0/1274413095 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f1c7400b5c0 con 0x7f1c840fd050 2024-04-12T20:44:30.792 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:44:30.792+0000 7f1c7aa00700 1 -- 172.21.15.18:0/1274413095 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f1c7400be70 con 0x7f1c840fd050 2024-04-12T20:44:30.792 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:44:30.792+0000 7f1c89c00700 1 -- 172.21.15.18:0/1274413095 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1c8410d800 con 0x7f1c840fd050 2024-04-12T20:44:30.792 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:44:30.793+0000 7f1c7aa00700 1 -- 172.21.15.18:0/1274413095 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f1c740135a0 con 0x7f1c840fd050 2024-04-12T20:44:30.792 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:44:30.793+0000 7f1c89c00700 1 -- 172.21.15.18:0/1274413095 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1c8410e370 con 0x7f1c840fd050 2024-04-12T20:44:30.794 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:44:30.794+0000 7f1c7aa00700 1 -- 172.21.15.18:0/1274413095 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 13) v1 ==== 43424+0+0 (secure 0 0 0) 0x7f1c74004d30 con 0x7f1c840fd050 2024-04-12T20:44:30.794 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:44:30.794+0000 7f1c89c00700 1 -- 172.21.15.18:0/1274413095 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1c84101e70 con 0x7f1c840fd050 2024-04-12T20:44:30.794 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:44:30.794+0000 7f1c7aa00700 1 --2- 172.21.15.18:0/1274413095 >> [v2:172.21.15.18:6800/1391635591,v1:172.21.15.18:6801/1391635591] conn(0x7f1c68036880 0x7f1c68038d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T20:44:30.794 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:44:30.794+0000 7f1c7aa00700 1 -- 172.21.15.18:0/1274413095 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1151+0+0 (secure 0 0 0) 0x7f1c74049340 con 0x7f1c840fd050 2024-04-12T20:44:30.794 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:44:30.795+0000 7f1c82000700 1 --2- 172.21.15.18:0/1274413095 >> [v2:172.21.15.18:6800/1391635591,v1:172.21.15.18:6801/1391635591] conn(0x7f1c68036880 0x7f1c68038d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T20:44:30.795 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:44:30.795+0000 7f1c82000700 1 --2- 172.21.15.18:0/1274413095 >> [v2:172.21.15.18:6800/1391635591,v1:172.21.15.18:6801/1391635591] conn(0x7f1c68036880 0x7f1c68038d30 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f1c70009990 tx=0x7f1c70006e40 comp rx=0 tx=0).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T20:44:30.799 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:44:30.800+0000 7f1c7aa00700 1 -- 172.21.15.18:0/1274413095 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7f1c74011940 con 0x7f1c840fd050 2024-04-12T20:44:31.016 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:44:31.016+0000 7f1c89c00700 1 -- 172.21.15.18:0/1274413095 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/allow_ptrace}] v 0) v1 -- 0x7f1c841017e0 con 0x7f1c840fd050 2024-04-12T20:44:31.022 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:44:31.022+0000 7f1c7aa00700 1 -- 172.21.15.18:0/1274413095 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/allow_ptrace}]=0 v9)=0 v9) v1 ==== 125+0+0 (secure 0 0 0) 0x7f1c74021020 con 0x7f1c840fd050 2024-04-12T20:44:31.025 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:44:31.025+0000 7f1c89c00700 1 -- 172.21.15.18:0/1274413095 >> [v2:172.21.15.18:6800/1391635591,v1:172.21.15.18:6801/1391635591] conn(0x7f1c68036880 msgr2=0x7f1c68038d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:44:31.025 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:44:31.025+0000 7f1c89c00700 1 --2- 172.21.15.18:0/1274413095 >> [v2:172.21.15.18:6800/1391635591,v1:172.21.15.18:6801/1391635591] conn(0x7f1c68036880 0x7f1c68038d30 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f1c70009990 tx=0x7f1c70006e40 comp rx=0 tx=0).stop 2024-04-12T20:44:31.026 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:44:31.025+0000 7f1c89c00700 1 -- 172.21.15.18:0/1274413095 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1c840fd050 msgr2=0x7f1c8410d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:44:31.026 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:44:31.025+0000 7f1c89c00700 1 --2- 172.21.15.18:0/1274413095 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1c840fd050 0x7f1c8410d0c0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f1c74009680 tx=0x7f1c74009e30 comp rx=0 tx=0).stop 2024-04-12T20:44:31.026 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:44:31.026+0000 7f1c89c00700 1 -- 172.21.15.18:0/1274413095 shutdown_connections 2024-04-12T20:44:31.026 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:44:31.026+0000 7f1c89c00700 1 --2- 172.21.15.18:0/1274413095 >> [v2:172.21.15.18:6800/1391635591,v1:172.21.15.18:6801/1391635591] conn(0x7f1c68036880 0x7f1c68038d30 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:44:31.026 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:44:31.026+0000 7f1c89c00700 1 --2- 172.21.15.18:0/1274413095 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1c840fd050 0x7f1c8410d0c0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:44:31.026 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:44:31.026+0000 7f1c89c00700 1 -- 172.21.15.18:0/1274413095 >> 172.21.15.18:0/1274413095 conn(0x7f1c840f70d0 msgr2=0x7f1c84065070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T20:44:31.026 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:44:31.026+0000 7f1c89c00700 1 -- 172.21.15.18:0/1274413095 shutdown_connections 2024-04-12T20:44:31.026 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:44:31.026+0000 7f1c89c00700 1 -- 172.21.15.18:0/1274413095 wait complete. 2024-04-12T20:44:31.775 INFO:tasks.cephadm:Distributing conf and client.admin keyring to all hosts + 0755 2024-04-12T20:44:31.775 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1cbe407a-f90d-11ee-bc8d-c7b262605968 -- ceph orch client-keyring set client.admin '*' --mode 0755 2024-04-12T20:44:32.317 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:44:32 smithi018 ceph-mon[28799]: from='client.? 172.21.15.18:0/1274413095' entity='client.admin' 2024-04-12T20:44:32.683 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/mon.smithi018/config 2024-04-12T20:44:33.801 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:44:33 smithi018 ceph-mon[28799]: from='mgr.14162 172.21.15.18:0/2711576919' entity='mgr.smithi018.xjzwga' 2024-04-12T20:44:33.802 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:44:33 smithi018 ceph-mon[28799]: from='mgr.14162 172.21.15.18:0/2711576919' entity='mgr.smithi018.xjzwga' 2024-04-12T20:44:33.802 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:44:33 smithi018 ceph-mon[28799]: from='mgr.14162 172.21.15.18:0/2711576919' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "config rm", "who": "osd/host:smithi018", "name": "osd_memory_target"}]: dispatch 2024-04-12T20:44:33.802 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:44:33 smithi018 ceph-mon[28799]: from='mgr.14162 172.21.15.18:0/2711576919' entity='mgr.smithi018.xjzwga' 2024-04-12T20:44:33.802 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:44:33 smithi018 ceph-mon[28799]: from='mgr.14162 172.21.15.18:0/2711576919' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi018", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-04-12T20:44:33.802 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:44:33 smithi018 ceph-mon[28799]: from='mgr.14162 172.21.15.18:0/2711576919' entity='mgr.smithi018.xjzwga' cmd='[{"prefix": "auth get-or-create", "entity": "client.crash.smithi018", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]': finished 2024-04-12T20:44:33.802 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:44:33 smithi018 ceph-mon[28799]: from='mgr.14162 172.21.15.18:0/2711576919' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-12T20:44:33.802 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:44:33 smithi018 ceph-mon[28799]: Deploying daemon crash.smithi018 on smithi018 2024-04-12T20:44:34.762 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:44:34.762+0000 7f1511e00700 1 -- 172.21.15.18:0/1766546406 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f150c0feae0 msgr2=0x7f150c0feef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:44:34.762 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:44:34.762+0000 7f1511e00700 1 --2- 172.21.15.18:0/1766546406 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f150c0feae0 0x7f150c0feef0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f15000048c0 tx=0x7f1500009e30 comp rx=0 tx=0).stop 2024-04-12T20:44:34.763 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:44:34.762+0000 7f1511e00700 1 -- 172.21.15.18:0/1766546406 shutdown_connections 2024-04-12T20:44:34.763 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:44:34.762+0000 7f1511e00700 1 --2- 172.21.15.18:0/1766546406 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f150c0feae0 0x7f150c0feef0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:44:34.763 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:44:34.762+0000 7f1511e00700 1 -- 172.21.15.18:0/1766546406 >> 172.21.15.18:0/1766546406 conn(0x7f150c0fa5d0 msgr2=0x7f150c0fc9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T20:44:34.763 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:44:34.762+0000 7f1511e00700 1 -- 172.21.15.18:0/1766546406 shutdown_connections 2024-04-12T20:44:34.763 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:44:34.762+0000 7f1511e00700 1 -- 172.21.15.18:0/1766546406 wait complete. 2024-04-12T20:44:34.763 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:44:34.763+0000 7f1511e00700 1 Processor -- start 2024-04-12T20:44:34.763 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:44:34.764+0000 7f1511e00700 1 -- start start 2024-04-12T20:44:34.764 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:44:34.764+0000 7f1511e00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f150c0feae0 0x7f150c10f100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T20:44:34.764 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:44:34.764+0000 7f1511e00700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f150c10f640 con 0x7f150c0feae0 2024-04-12T20:44:34.764 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:44:34.764+0000 7f1510a00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f150c0feae0 0x7f150c10f100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T20:44:34.764 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:44:34.764+0000 7f1510a00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f150c0feae0 0x7f150c10f100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:47330/0 (socket says 172.21.15.18:47330) 2024-04-12T20:44:34.764 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:44:34.764+0000 7f1510a00700 1 -- 172.21.15.18:0/337893462 learned_addr learned my addr 172.21.15.18:0/337893462 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T20:44:34.764 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:44:34.765+0000 7f1510a00700 1 -- 172.21.15.18:0/337893462 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1500005040 con 0x7f150c0feae0 2024-04-12T20:44:34.765 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:44:34.765+0000 7f1510a00700 1 --2- 172.21.15.18:0/337893462 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f150c0feae0 0x7f150c10f100 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f1500009680 tx=0x7f1500009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T20:44:34.765 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:44:34.766+0000 7f1508c00700 1 -- 172.21.15.18:0/337893462 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f150000b5c0 con 0x7f150c0feae0 2024-04-12T20:44:34.765 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:44:34.766+0000 7f1508c00700 1 -- 172.21.15.18:0/337893462 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f150000be70 con 0x7f150c0feae0 2024-04-12T20:44:34.765 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:44:34.766+0000 7f1511e00700 1 -- 172.21.15.18:0/337893462 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f150c10f840 con 0x7f150c0feae0 2024-04-12T20:44:34.765 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:44:34.766+0000 7f1508c00700 1 -- 172.21.15.18:0/337893462 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f15000135a0 con 0x7f150c0feae0 2024-04-12T20:44:34.766 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:44:34.766+0000 7f1511e00700 1 -- 172.21.15.18:0/337893462 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f150c1103b0 con 0x7f150c0feae0 2024-04-12T20:44:34.767 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:44:34.767+0000 7f1511e00700 1 -- 172.21.15.18:0/337893462 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f150c104ff0 con 0x7f150c0feae0 2024-04-12T20:44:34.767 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:44:34.767+0000 7f1508c00700 1 -- 172.21.15.18:0/337893462 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 13) v1 ==== 43424+0+0 (secure 0 0 0) 0x7f1500013810 con 0x7f150c0feae0 2024-04-12T20:44:34.767 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:44:34.768+0000 7f1508c00700 1 --2- 172.21.15.18:0/337893462 >> [v2:172.21.15.18:6800/1391635591,v1:172.21.15.18:6801/1391635591] conn(0x7f14f40368d0 0x7f14f4038d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T20:44:34.767 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:44:34.768+0000 7f1508c00700 1 -- 172.21.15.18:0/337893462 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1151+0+0 (secure 0 0 0) 0x7f15000495e0 con 0x7f150c0feae0 2024-04-12T20:44:34.768 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:44:34.768+0000 7f150be00700 1 --2- 172.21.15.18:0/337893462 >> [v2:172.21.15.18:6800/1391635591,v1:172.21.15.18:6801/1391635591] conn(0x7f14f40368d0 0x7f14f4038d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T20:44:34.768 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:44:34.769+0000 7f150be00700 1 --2- 172.21.15.18:0/337893462 >> [v2:172.21.15.18:6800/1391635591,v1:172.21.15.18:6801/1391635591] conn(0x7f14f40368d0 0x7f14f4038d80 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f14f8009990 tx=0x7f14f8006e40 comp rx=0 tx=0).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T20:44:34.773 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:44:34.773+0000 7f1508c00700 1 -- 172.21.15.18:0/337893462 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7f1500011940 con 0x7f150c0feae0 2024-04-12T20:44:34.998 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:44:34.998+0000 7f1511e00700 1 -- 172.21.15.18:0/337893462 --> [v2:172.21.15.18:6800/1391635591,v1:172.21.15.18:6801/1391635591] -- mgr_command(tid 0: {"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}) v1 -- 0x7f150c05f690 con 0x7f14f40368d0 2024-04-12T20:44:35.004 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:44:35.005+0000 7f1508c00700 1 -- 172.21.15.18:0/337893462 <== mgr.14162 v2:172.21.15.18:6800/1391635591 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f150c05f690 con 0x7f14f40368d0 2024-04-12T20:44:35.008 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:44:35.008+0000 7f1511e00700 1 -- 172.21.15.18:0/337893462 >> [v2:172.21.15.18:6800/1391635591,v1:172.21.15.18:6801/1391635591] conn(0x7f14f40368d0 msgr2=0x7f14f4038d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:44:35.008 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:44:35.008+0000 7f1511e00700 1 --2- 172.21.15.18:0/337893462 >> [v2:172.21.15.18:6800/1391635591,v1:172.21.15.18:6801/1391635591] conn(0x7f14f40368d0 0x7f14f4038d80 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f14f8009990 tx=0x7f14f8006e40 comp rx=0 tx=0).stop 2024-04-12T20:44:35.008 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:44:35.008+0000 7f1511e00700 1 -- 172.21.15.18:0/337893462 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f150c0feae0 msgr2=0x7f150c10f100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:44:35.008 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:44:35.008+0000 7f1511e00700 1 --2- 172.21.15.18:0/337893462 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f150c0feae0 0x7f150c10f100 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f1500009680 tx=0x7f1500009e30 comp rx=0 tx=0).stop 2024-04-12T20:44:35.008 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:44:35.008+0000 7f1511e00700 1 -- 172.21.15.18:0/337893462 shutdown_connections 2024-04-12T20:44:35.008 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:44:35.008+0000 7f1511e00700 1 --2- 172.21.15.18:0/337893462 >> [v2:172.21.15.18:6800/1391635591,v1:172.21.15.18:6801/1391635591] conn(0x7f14f40368d0 0x7f14f4038d80 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:44:35.008 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:44:35.008+0000 7f1511e00700 1 --2- 172.21.15.18:0/337893462 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f150c0feae0 0x7f150c10f100 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:44:35.008 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:44:35.008+0000 7f1511e00700 1 -- 172.21.15.18:0/337893462 >> 172.21.15.18:0/337893462 conn(0x7f150c0fa5d0 msgr2=0x7f150c0fb170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T20:44:35.009 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:44:35.008+0000 7f1511e00700 1 -- 172.21.15.18:0/337893462 shutdown_connections 2024-04-12T20:44:35.009 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:44:35.008+0000 7f1511e00700 1 -- 172.21.15.18:0/337893462 wait complete. 2024-04-12T20:44:36.062 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi186 2024-04-12T20:44:36.063 DEBUG:teuthology.orchestra.run.smithi186:> set -ex 2024-04-12T20:44:36.063 DEBUG:teuthology.orchestra.run.smithi186:> dd of=/etc/ceph/ceph.conf 2024-04-12T20:44:36.124 DEBUG:teuthology.orchestra.run.smithi186:> set -ex 2024-04-12T20:44:36.124 DEBUG:teuthology.orchestra.run.smithi186:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-04-12T20:44:36.219 INFO:tasks.cephadm:Adding host smithi186 to orchestrator... 2024-04-12T20:44:36.219 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1cbe407a-f90d-11ee-bc8d-c7b262605968 -- ceph orch host add smithi186 2024-04-12T20:44:36.303 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:44:36 smithi018 ceph-mon[28799]: from='client.14186 -' entity='client.admin' cmd=[{"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T20:44:36.303 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:44:36 smithi018 ceph-mon[28799]: from='mgr.14162 172.21.15.18:0/2711576919' entity='mgr.smithi018.xjzwga' 2024-04-12T20:44:36.303 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:44:36 smithi018 ceph-mon[28799]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-12T20:44:37.095 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/mon.smithi018/config 2024-04-12T20:44:38.295 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:44:38 smithi018 ceph-mon[28799]: from='mgr.14162 172.21.15.18:0/2711576919' entity='mgr.smithi018.xjzwga' 2024-04-12T20:44:38.295 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:44:38 smithi018 ceph-mon[28799]: from='mgr.14162 172.21.15.18:0/2711576919' entity='mgr.smithi018.xjzwga' 2024-04-12T20:44:38.295 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:44:38 smithi018 ceph-mon[28799]: from='mgr.14162 172.21.15.18:0/2711576919' entity='mgr.smithi018.xjzwga' 2024-04-12T20:44:38.295 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:44:38 smithi018 ceph-mon[28799]: from='mgr.14162 172.21.15.18:0/2711576919' entity='mgr.smithi018.xjzwga' 2024-04-12T20:44:38.295 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:44:38 smithi018 ceph-mon[28799]: Deploying daemon node-exporter.smithi018 on smithi018 2024-04-12T20:44:38.295 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:44:38 smithi018 ceph-mon[28799]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-12T20:44:39.117 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:44:39.115+0000 7fbd71000700 1 -- 172.21.15.18:0/1714963915 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fbd6c0fca70 msgr2=0x7fbd6c0fce80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:44:39.117 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:44:39.115+0000 7fbd71000700 1 --2- 172.21.15.18:0/1714963915 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fbd6c0fca70 0x7fbd6c0fce80 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fbd5c0048c0 tx=0x7fbd5c009e30 comp rx=0 tx=0).stop 2024-04-12T20:44:39.117 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:44:39.115+0000 7fbd71000700 1 -- 172.21.15.18:0/1714963915 shutdown_connections 2024-04-12T20:44:39.117 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:44:39.115+0000 7fbd71000700 1 --2- 172.21.15.18:0/1714963915 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fbd6c0fca70 0x7fbd6c0fce80 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:44:39.118 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:44:39.115+0000 7fbd71000700 1 -- 172.21.15.18:0/1714963915 >> 172.21.15.18:0/1714963915 conn(0x7fbd6c0f81d0 msgr2=0x7fbd6c0fa600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T20:44:39.118 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:44:39.116+0000 7fbd71000700 1 -- 172.21.15.18:0/1714963915 shutdown_connections 2024-04-12T20:44:39.118 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:44:39.116+0000 7fbd71000700 1 -- 172.21.15.18:0/1714963915 wait complete. 2024-04-12T20:44:39.118 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:44:39.117+0000 7fbd71000700 1 Processor -- start 2024-04-12T20:44:39.118 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:44:39.117+0000 7fbd71000700 1 -- start start 2024-04-12T20:44:39.118 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:44:39.117+0000 7fbd71000700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fbd6c0fca70 0x7fbd6c103f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T20:44:39.118 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:44:39.117+0000 7fbd71000700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd6c1044a0 con 0x7fbd6c0fca70 2024-04-12T20:44:39.118 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:44:39.117+0000 7fbd69e00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fbd6c0fca70 0x7fbd6c103f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T20:44:39.118 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:44:39.118+0000 7fbd69e00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fbd6c0fca70 0x7fbd6c103f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:47360/0 (socket says 172.21.15.18:47360) 2024-04-12T20:44:39.118 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:44:39.118+0000 7fbd69e00700 1 -- 172.21.15.18:0/2699802102 learned_addr learned my addr 172.21.15.18:0/2699802102 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T20:44:39.119 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:44:39.118+0000 7fbd69e00700 1 -- 172.21.15.18:0/2699802102 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd5c005040 con 0x7fbd6c0fca70 2024-04-12T20:44:39.119 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:44:39.118+0000 7fbd69e00700 1 --2- 172.21.15.18:0/2699802102 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fbd6c0fca70 0x7fbd6c103f60 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fbd5c009680 tx=0x7fbd5c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T20:44:39.119 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:44:39.119+0000 7fbd62000700 1 -- 172.21.15.18:0/2699802102 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fbd5c00b5c0 con 0x7fbd6c0fca70 2024-04-12T20:44:39.119 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:44:39.119+0000 7fbd62000700 1 -- 172.21.15.18:0/2699802102 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fbd5c00be70 con 0x7fbd6c0fca70 2024-04-12T20:44:39.119 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:44:39.119+0000 7fbd71000700 1 -- 172.21.15.18:0/2699802102 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbd6c1046a0 con 0x7fbd6c0fca70 2024-04-12T20:44:39.119 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:44:39.119+0000 7fbd62000700 1 -- 172.21.15.18:0/2699802102 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fbd5c0135a0 con 0x7fbd6c0fca70 2024-04-12T20:44:39.120 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:44:39.119+0000 7fbd71000700 1 -- 172.21.15.18:0/2699802102 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbd6c105260 con 0x7fbd6c0fca70 2024-04-12T20:44:39.120 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:44:39.120+0000 7fbd62000700 1 -- 172.21.15.18:0/2699802102 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 13) v1 ==== 43424+0+0 (secure 0 0 0) 0x7fbd5c013860 con 0x7fbd6c0fca70 2024-04-12T20:44:39.120 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:44:39.121+0000 7fbd71000700 1 -- 172.21.15.18:0/2699802102 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbd6c05f5f0 con 0x7fbd6c0fca70 2024-04-12T20:44:39.120 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:44:39.121+0000 7fbd62000700 1 --2- 172.21.15.18:0/2699802102 >> [v2:172.21.15.18:6800/1391635591,v1:172.21.15.18:6801/1391635591] conn(0x7fbd500368d0 0x7fbd50038d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T20:44:39.121 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:44:39.121+0000 7fbd62000700 1 -- 172.21.15.18:0/2699802102 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1151+0+0 (secure 0 0 0) 0x7fbd5c049640 con 0x7fbd6c0fca70 2024-04-12T20:44:39.122 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:44:39.123+0000 7fbd69400700 1 --2- 172.21.15.18:0/2699802102 >> [v2:172.21.15.18:6800/1391635591,v1:172.21.15.18:6801/1391635591] conn(0x7fbd500368d0 0x7fbd50038d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T20:44:39.123 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:44:39.123+0000 7fbd69400700 1 --2- 172.21.15.18:0/2699802102 >> [v2:172.21.15.18:6800/1391635591,v1:172.21.15.18:6801/1391635591] conn(0x7fbd500368d0 0x7fbd50038d80 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fbd58009990 tx=0x7fbd58006e40 comp rx=0 tx=0).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T20:44:39.127 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:44:39.127+0000 7fbd62000700 1 -- 172.21.15.18:0/2699802102 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7fbd5c011960 con 0x7fbd6c0fca70 2024-04-12T20:44:39.368 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:44:39.369+0000 7fbd71000700 1 -- 172.21.15.18:0/2699802102 --> [v2:172.21.15.18:6800/1391635591,v1:172.21.15.18:6801/1391635591] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi186", "target": ["mon-mgr", ""]}) v1 -- 0x7fbd6c071700 con 0x7fbd500368d0 2024-04-12T20:44:40.353 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:44:40 smithi018 ceph-mon[28799]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-12T20:44:40.353 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:44:40 smithi018 ceph-mon[28799]: from='client.14188 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi186", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T20:44:41.603 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:44:41 smithi018 ceph-mon[28799]: from='mgr.14162 172.21.15.18:0/2711576919' entity='mgr.smithi018.xjzwga' 2024-04-12T20:44:42.353 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:44:42 smithi018 ceph-mon[28799]: Deploying cephadm binary to smithi186 2024-04-12T20:44:42.353 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:44:42 smithi018 ceph-mon[28799]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-12T20:44:43.783 INFO:teuthology.orchestra.run.smithi018.stdout:Added host 'smithi186' with addr '172.21.15.186' 2024-04-12T20:44:43.784 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:44:43.782+0000 7fbd62000700 1 -- 172.21.15.18:0/2699802102 <== mgr.14162 v2:172.21.15.18:6800/1391635591 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+49 (secure 0 0 0) 0x7fbd6c071700 con 0x7fbd500368d0 2024-04-12T20:44:43.786 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:44:43.786+0000 7fbd71000700 1 -- 172.21.15.18:0/2699802102 >> [v2:172.21.15.18:6800/1391635591,v1:172.21.15.18:6801/1391635591] conn(0x7fbd500368d0 msgr2=0x7fbd50038d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:44:43.786 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:44:43.786+0000 7fbd71000700 1 --2- 172.21.15.18:0/2699802102 >> [v2:172.21.15.18:6800/1391635591,v1:172.21.15.18:6801/1391635591] conn(0x7fbd500368d0 0x7fbd50038d80 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fbd58009990 tx=0x7fbd58006e40 comp rx=0 tx=0).stop 2024-04-12T20:44:43.786 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:44:43.786+0000 7fbd71000700 1 -- 172.21.15.18:0/2699802102 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fbd6c0fca70 msgr2=0x7fbd6c103f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:44:43.786 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:44:43.786+0000 7fbd71000700 1 --2- 172.21.15.18:0/2699802102 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fbd6c0fca70 0x7fbd6c103f60 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fbd5c009680 tx=0x7fbd5c009e30 comp rx=0 tx=0).stop 2024-04-12T20:44:43.786 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:44:43.786+0000 7fbd71000700 1 -- 172.21.15.18:0/2699802102 shutdown_connections 2024-04-12T20:44:43.786 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:44:43.786+0000 7fbd71000700 1 --2- 172.21.15.18:0/2699802102 >> [v2:172.21.15.18:6800/1391635591,v1:172.21.15.18:6801/1391635591] conn(0x7fbd500368d0 0x7fbd50038d80 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:44:43.787 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:44:43.786+0000 7fbd71000700 1 --2- 172.21.15.18:0/2699802102 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fbd6c0fca70 0x7fbd6c103f60 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:44:43.787 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:44:43.786+0000 7fbd71000700 1 -- 172.21.15.18:0/2699802102 >> 172.21.15.18:0/2699802102 conn(0x7fbd6c0f81d0 msgr2=0x7fbd6c0fa600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T20:44:43.787 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:44:43.786+0000 7fbd71000700 1 -- 172.21.15.18:0/2699802102 shutdown_connections 2024-04-12T20:44:43.787 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:44:43.786+0000 7fbd71000700 1 -- 172.21.15.18:0/2699802102 wait complete. 2024-04-12T20:44:44.418 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:44:44 smithi018 ceph-mon[28799]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-12T20:44:44.418 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:44:44 smithi018 ceph-mon[28799]: from='mgr.14162 172.21.15.18:0/2711576919' entity='mgr.smithi018.xjzwga' 2024-04-12T20:44:44.418 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:44:44 smithi018 ceph-mon[28799]: from='mgr.14162 172.21.15.18:0/2711576919' entity='mgr.smithi018.xjzwga' 2024-04-12T20:44:44.418 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:44:44 smithi018 ceph-mon[28799]: from='mgr.14162 172.21.15.18:0/2711576919' entity='mgr.smithi018.xjzwga' 2024-04-12T20:44:44.418 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:44:44 smithi018 ceph-mon[28799]: from='mgr.14162 172.21.15.18:0/2711576919' entity='mgr.smithi018.xjzwga' 2024-04-12T20:44:44.418 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:44:44 smithi018 ceph-mon[28799]: Deploying daemon alertmanager.smithi018 on smithi018 2024-04-12T20:44:44.418 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:44:44 smithi018 ceph-mon[28799]: from='mgr.14162 172.21.15.18:0/2711576919' entity='mgr.smithi018.xjzwga' 2024-04-12T20:44:44.483 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1cbe407a-f90d-11ee-bc8d-c7b262605968 -- ceph orch host ls --format=json 2024-04-12T20:44:44.976 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/mon.smithi018/config 2024-04-12T20:44:45.442 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:44:45 smithi018 ceph-mon[28799]: Added host smithi186 2024-04-12T20:44:46.603 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:44:46 smithi018 ceph-mon[28799]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-12T20:44:46.603 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:44:46 smithi018 ceph-mon[28799]: from='mgr.14162 172.21.15.18:0/2711576919' entity='mgr.smithi018.xjzwga' 2024-04-12T20:44:47.876 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:44:47.876+0000 7fb4fd800700 1 -- 172.21.15.18:0/1326591719 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb4f80fc8c0 msgr2=0x7fb4f80fccd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:44:47.876 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:44:47.876+0000 7fb4fd800700 1 --2- 172.21.15.18:0/1326591719 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb4f80fc8c0 0x7fb4f80fccd0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fb4e80048c0 tx=0x7fb4e8009e30 comp rx=0 tx=0).stop 2024-04-12T20:44:47.877 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:44:47.876+0000 7fb4fd800700 1 -- 172.21.15.18:0/1326591719 shutdown_connections 2024-04-12T20:44:47.877 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:44:47.876+0000 7fb4fd800700 1 --2- 172.21.15.18:0/1326591719 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb4f80fc8c0 0x7fb4f80fccd0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:44:47.877 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:44:47.876+0000 7fb4fd800700 1 -- 172.21.15.18:0/1326591719 >> 172.21.15.18:0/1326591719 conn(0x7fb4f80f8020 msgr2=0x7fb4f80fa450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T20:44:47.877 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:44:47.876+0000 7fb4fd800700 1 -- 172.21.15.18:0/1326591719 shutdown_connections 2024-04-12T20:44:47.877 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:44:47.876+0000 7fb4fd800700 1 -- 172.21.15.18:0/1326591719 wait complete. 2024-04-12T20:44:47.878 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:44:47.877+0000 7fb4fd800700 1 Processor -- start 2024-04-12T20:44:47.878 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:44:47.877+0000 7fb4fd800700 1 -- start start 2024-04-12T20:44:47.878 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:44:47.878+0000 7fb4fd800700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb4f80fc8c0 0x7fb4f8071d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T20:44:47.878 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:44:47.878+0000 7fb4fd800700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4f80722a0 con 0x7fb4f80fc8c0 2024-04-12T20:44:47.878 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:44:47.878+0000 7fb4f7e00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb4f80fc8c0 0x7fb4f8071d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T20:44:47.878 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:44:47.878+0000 7fb4f7e00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb4f80fc8c0 0x7fb4f8071d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:52534/0 (socket says 172.21.15.18:52534) 2024-04-12T20:44:47.878 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:44:47.878+0000 7fb4f7e00700 1 -- 172.21.15.18:0/1518676749 learned_addr learned my addr 172.21.15.18:0/1518676749 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T20:44:47.879 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:44:47.879+0000 7fb4f7e00700 1 -- 172.21.15.18:0/1518676749 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4e8005040 con 0x7fb4f80fc8c0 2024-04-12T20:44:47.879 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:44:47.879+0000 7fb4f7e00700 1 --2- 172.21.15.18:0/1518676749 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb4f80fc8c0 0x7fb4f8071d60 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fb4e8009680 tx=0x7fb4e8009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T20:44:47.879 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:44:47.879+0000 7fb4efe00700 1 -- 172.21.15.18:0/1518676749 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fb4e800b5c0 con 0x7fb4f80fc8c0 2024-04-12T20:44:47.879 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:44:47.879+0000 7fb4efe00700 1 -- 172.21.15.18:0/1518676749 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb4e800be70 con 0x7fb4f80fc8c0 2024-04-12T20:44:47.879 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:44:47.879+0000 7fb4fd800700 1 -- 172.21.15.18:0/1518676749 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb4f8070370 con 0x7fb4f80fc8c0 2024-04-12T20:44:47.879 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:44:47.880+0000 7fb4efe00700 1 -- 172.21.15.18:0/1518676749 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fb4e80135a0 con 0x7fb4f80fc8c0 2024-04-12T20:44:47.880 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:44:47.880+0000 7fb4fd800700 1 -- 172.21.15.18:0/1518676749 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb4f8070860 con 0x7fb4f80fc8c0 2024-04-12T20:44:47.881 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:44:47.881+0000 7fb4efe00700 1 -- 172.21.15.18:0/1518676749 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 13) v1 ==== 43424+0+0 (secure 0 0 0) 0x7fb4e801ca90 con 0x7fb4f80fc8c0 2024-04-12T20:44:47.881 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:44:47.882+0000 7fb4fd800700 1 -- 172.21.15.18:0/1518676749 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb4f805f5f0 con 0x7fb4f80fc8c0 2024-04-12T20:44:47.881 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:44:47.882+0000 7fb4efe00700 1 --2- 172.21.15.18:0/1518676749 >> [v2:172.21.15.18:6800/1391635591,v1:172.21.15.18:6801/1391635591] conn(0x7fb4dc0368d0 0x7fb4dc038d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T20:44:47.881 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:44:47.882+0000 7fb4efe00700 1 -- 172.21.15.18:0/1518676749 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1151+0+0 (secure 0 0 0) 0x7fb4e80493f0 con 0x7fb4f80fc8c0 2024-04-12T20:44:47.882 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:44:47.882+0000 7fb4f7400700 1 --2- 172.21.15.18:0/1518676749 >> [v2:172.21.15.18:6800/1391635591,v1:172.21.15.18:6801/1391635591] conn(0x7fb4dc0368d0 0x7fb4dc038d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T20:44:47.882 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:44:47.883+0000 7fb4f7400700 1 --2- 172.21.15.18:0/1518676749 >> [v2:172.21.15.18:6800/1391635591,v1:172.21.15.18:6801/1391635591] conn(0x7fb4dc0368d0 0x7fb4dc038d80 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fb4e4009990 tx=0x7fb4e4006e40 comp rx=0 tx=0).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T20:44:47.888 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:44:47.888+0000 7fb4efe00700 1 -- 172.21.15.18:0/1518676749 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7fb4e8011960 con 0x7fb4f80fc8c0 2024-04-12T20:44:48.189 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:44:48.189+0000 7fb4fd800700 1 -- 172.21.15.18:0/1518676749 --> [v2:172.21.15.18:6800/1391635591,v1:172.21.15.18:6801/1391635591] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fb4f8106310 con 0x7fb4dc0368d0 2024-04-12T20:44:48.192 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:44:48.192+0000 7fb4efe00700 1 -- 172.21.15.18:0/1518676749 <== mgr.14162 v2:172.21.15.18:6800/1391635591 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+160 (secure 0 0 0) 0x7fb4f8106310 con 0x7fb4dc0368d0 2024-04-12T20:44:48.192 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-12T20:44:48.192 INFO:teuthology.orchestra.run.smithi018.stdout:[{"addr": "172.21.15.18", "hostname": "smithi018", "labels": [], "status": ""}, {"addr": "172.21.15.186", "hostname": "smithi186", "labels": [], "status": ""}] 2024-04-12T20:44:48.196 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:44:48.196+0000 7fb4fd800700 1 -- 172.21.15.18:0/1518676749 >> [v2:172.21.15.18:6800/1391635591,v1:172.21.15.18:6801/1391635591] conn(0x7fb4dc0368d0 msgr2=0x7fb4dc038d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:44:48.197 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:44:48.196+0000 7fb4fd800700 1 --2- 172.21.15.18:0/1518676749 >> [v2:172.21.15.18:6800/1391635591,v1:172.21.15.18:6801/1391635591] conn(0x7fb4dc0368d0 0x7fb4dc038d80 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fb4e4009990 tx=0x7fb4e4006e40 comp rx=0 tx=0).stop 2024-04-12T20:44:48.197 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:44:48.196+0000 7fb4fd800700 1 -- 172.21.15.18:0/1518676749 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb4f80fc8c0 msgr2=0x7fb4f8071d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:44:48.197 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:44:48.196+0000 7fb4fd800700 1 --2- 172.21.15.18:0/1518676749 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb4f80fc8c0 0x7fb4f8071d60 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fb4e8009680 tx=0x7fb4e8009e30 comp rx=0 tx=0).stop 2024-04-12T20:44:48.197 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:44:48.196+0000 7fb4fd800700 1 -- 172.21.15.18:0/1518676749 shutdown_connections 2024-04-12T20:44:48.197 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:44:48.196+0000 7fb4fd800700 1 --2- 172.21.15.18:0/1518676749 >> [v2:172.21.15.18:6800/1391635591,v1:172.21.15.18:6801/1391635591] conn(0x7fb4dc0368d0 0x7fb4dc038d80 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:44:48.197 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:44:48.196+0000 7fb4fd800700 1 --2- 172.21.15.18:0/1518676749 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb4f80fc8c0 0x7fb4f8071d60 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:44:48.197 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:44:48.196+0000 7fb4fd800700 1 -- 172.21.15.18:0/1518676749 >> 172.21.15.18:0/1518676749 conn(0x7fb4f80f8020 msgr2=0x7fb4f8103a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T20:44:48.197 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:44:48.197+0000 7fb4fd800700 1 -- 172.21.15.18:0/1518676749 shutdown_connections 2024-04-12T20:44:48.197 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:44:48.197+0000 7fb4fd800700 1 -- 172.21.15.18:0/1518676749 wait complete. 2024-04-12T20:44:48.603 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:44:48 smithi018 ceph-mon[28799]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-12T20:44:49.603 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:44:49 smithi018 ceph-mon[28799]: from='client.14190 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-12T20:44:49.896 INFO:tasks.cephadm:Setting crush tunables to default 2024-04-12T20:44:49.897 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1cbe407a-f90d-11ee-bc8d-c7b262605968 -- ceph osd crush tunables default 2024-04-12T20:44:50.603 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:44:50 smithi018 ceph-mon[28799]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-12T20:44:50.635 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/mon.smithi018/config 2024-04-12T20:44:52.603 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:44:52 smithi018 ceph-mon[28799]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-12T20:44:54.500 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:44:54 smithi018 ceph-mon[28799]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-12T20:44:55.126 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:44:55.124+0000 7f8d03200700 1 -- 172.21.15.18:0/688062330 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8cfc0fb780 msgr2=0x7f8cfc0fbb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:44:55.126 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:44:55.124+0000 7f8d03200700 1 --2- 172.21.15.18:0/688062330 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8cfc0fb780 0x7f8cfc0fbb90 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f8cf00048c0 tx=0x7f8cf0009e30 comp rx=0 tx=0).stop 2024-04-12T20:44:55.126 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:44:55.124+0000 7f8d03200700 1 -- 172.21.15.18:0/688062330 shutdown_connections 2024-04-12T20:44:55.126 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:44:55.124+0000 7f8d03200700 1 --2- 172.21.15.18:0/688062330 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8cfc0fb780 0x7f8cfc0fbb90 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:44:55.126 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:44:55.124+0000 7f8d03200700 1 -- 172.21.15.18:0/688062330 >> 172.21.15.18:0/688062330 conn(0x7f8cfc0f6ee0 msgr2=0x7f8cfc0f9310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T20:44:55.126 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:44:55.124+0000 7f8d03200700 1 -- 172.21.15.18:0/688062330 shutdown_connections 2024-04-12T20:44:55.126 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:44:55.124+0000 7f8d03200700 1 -- 172.21.15.18:0/688062330 wait complete. 2024-04-12T20:44:55.126 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:44:55.125+0000 7f8d03200700 1 Processor -- start 2024-04-12T20:44:55.127 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:44:55.126+0000 7f8d03200700 1 -- start start 2024-04-12T20:44:55.127 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:44:55.126+0000 7f8d03200700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8cfc0fb780 0x7f8cfc10f0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T20:44:55.127 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:44:55.126+0000 7f8d03200700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8cfc10f600 con 0x7f8cfc0fb780 2024-04-12T20:44:55.127 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:44:55.126+0000 7f8d01e00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8cfc0fb780 0x7f8cfc10f0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T20:44:55.127 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:44:55.126+0000 7f8d01e00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8cfc0fb780 0x7f8cfc10f0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:36716/0 (socket says 172.21.15.18:36716) 2024-04-12T20:44:55.127 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:44:55.126+0000 7f8d01e00700 1 -- 172.21.15.18:0/511652442 learned_addr learned my addr 172.21.15.18:0/511652442 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T20:44:55.128 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:44:55.127+0000 7f8d01e00700 1 -- 172.21.15.18:0/511652442 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8cf0005040 con 0x7f8cfc0fb780 2024-04-12T20:44:55.128 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:44:55.127+0000 7f8d01e00700 1 --2- 172.21.15.18:0/511652442 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8cfc0fb780 0x7f8cfc10f0c0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f8cf0009680 tx=0x7f8cf0009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T20:44:55.128 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:44:55.127+0000 7f8cf6000700 1 -- 172.21.15.18:0/511652442 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f8cf000b5c0 con 0x7f8cfc0fb780 2024-04-12T20:44:55.128 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:44:55.127+0000 7f8cf6000700 1 -- 172.21.15.18:0/511652442 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f8cf000be70 con 0x7f8cfc0fb780 2024-04-12T20:44:55.128 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:44:55.127+0000 7f8d03200700 1 -- 172.21.15.18:0/511652442 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8cfc10f800 con 0x7f8cfc0fb780 2024-04-12T20:44:55.128 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:44:55.127+0000 7f8cf6000700 1 -- 172.21.15.18:0/511652442 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f8cf00135a0 con 0x7f8cfc0fb780 2024-04-12T20:44:55.128 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:44:55.128+0000 7f8d03200700 1 -- 172.21.15.18:0/511652442 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8cfc110370 con 0x7f8cfc0fb780 2024-04-12T20:44:55.129 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:44:55.129+0000 7f8cf6000700 1 -- 172.21.15.18:0/511652442 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 13) v1 ==== 43424+0+0 (secure 0 0 0) 0x7f8cf001c430 con 0x7f8cfc0fb780 2024-04-12T20:44:55.129 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:44:55.129+0000 7f8d03200700 1 -- 172.21.15.18:0/511652442 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8cfc101c90 con 0x7f8cfc0fb780 2024-04-12T20:44:55.129 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:44:55.129+0000 7f8cf6000700 1 --2- 172.21.15.18:0/511652442 >> [v2:172.21.15.18:6800/1391635591,v1:172.21.15.18:6801/1391635591] conn(0x7f8ce40368d0 0x7f8ce4038d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T20:44:55.130 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:44:55.129+0000 7f8cf6000700 1 -- 172.21.15.18:0/511652442 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1151+0+0 (secure 0 0 0) 0x7f8cf0049370 con 0x7f8cfc0fb780 2024-04-12T20:44:55.130 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:44:55.130+0000 7f8d01400700 1 --2- 172.21.15.18:0/511652442 >> [v2:172.21.15.18:6800/1391635591,v1:172.21.15.18:6801/1391635591] conn(0x7f8ce40368d0 0x7f8ce4038d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T20:44:55.130 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:44:55.130+0000 7f8d01400700 1 --2- 172.21.15.18:0/511652442 >> [v2:172.21.15.18:6800/1391635591,v1:172.21.15.18:6801/1391635591] conn(0x7f8ce40368d0 0x7f8ce4038d80 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f8cec009990 tx=0x7f8cec006e40 comp rx=0 tx=0).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T20:44:55.135 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:44:55.135+0000 7f8cf6000700 1 -- 172.21.15.18:0/511652442 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7f8cf0004d70 con 0x7f8cfc0fb780 2024-04-12T20:44:55.366 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:44:55.365+0000 7f8d03200700 1 -- 172.21.15.18:0/511652442 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) v1 -- 0x7f8cfc1105b0 con 0x7f8cfc0fb780 2024-04-12T20:44:56.246 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:44:56.246+0000 7f8cf6000700 1 -- 172.21.15.18:0/511652442 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v4) v1 ==== 124+0+0 (secure 0 0 0) 0x7f8cf0021020 con 0x7f8cfc0fb780 2024-04-12T20:44:56.250 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:44:56.250+0000 7f8d03200700 1 -- 172.21.15.18:0/511652442 >> [v2:172.21.15.18:6800/1391635591,v1:172.21.15.18:6801/1391635591] conn(0x7f8ce40368d0 msgr2=0x7f8ce4038d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:44:56.251 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:44:56.250+0000 7f8d03200700 1 --2- 172.21.15.18:0/511652442 >> [v2:172.21.15.18:6800/1391635591,v1:172.21.15.18:6801/1391635591] conn(0x7f8ce40368d0 0x7f8ce4038d80 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f8cec009990 tx=0x7f8cec006e40 comp rx=0 tx=0).stop 2024-04-12T20:44:56.251 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:44:56.250+0000 7f8d03200700 1 -- 172.21.15.18:0/511652442 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8cfc0fb780 msgr2=0x7f8cfc10f0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:44:56.251 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:44:56.250+0000 7f8d03200700 1 --2- 172.21.15.18:0/511652442 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8cfc0fb780 0x7f8cfc10f0c0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f8cf0009680 tx=0x7f8cf0009e30 comp rx=0 tx=0).stop 2024-04-12T20:44:56.251 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:44:56.251+0000 7f8d03200700 1 -- 172.21.15.18:0/511652442 shutdown_connections 2024-04-12T20:44:56.251 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:44:56.251+0000 7f8d03200700 1 --2- 172.21.15.18:0/511652442 >> [v2:172.21.15.18:6800/1391635591,v1:172.21.15.18:6801/1391635591] conn(0x7f8ce40368d0 0x7f8ce4038d80 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:44:56.251 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:44:56.251+0000 7f8d03200700 1 --2- 172.21.15.18:0/511652442 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8cfc0fb780 0x7f8cfc10f0c0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:44:56.251 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:44:56.251+0000 7f8d03200700 1 -- 172.21.15.18:0/511652442 >> 172.21.15.18:0/511652442 conn(0x7f8cfc0f6ee0 msgr2=0x7f8cfc0f9310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T20:44:56.251 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:44:56.251+0000 7f8d03200700 1 -- 172.21.15.18:0/511652442 shutdown_connections 2024-04-12T20:44:56.251 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:44:56.251+0000 7f8d03200700 1 -- 172.21.15.18:0/511652442 wait complete. 2024-04-12T20:44:56.252 INFO:teuthology.orchestra.run.smithi018.stderr:adjusted tunables profile to default 2024-04-12T20:44:56.603 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:44:56 smithi018 ceph-mon[28799]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-12T20:44:56.603 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:44:56 smithi018 ceph-mon[28799]: from='client.? 172.21.15.18:0/511652442' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-04-12T20:44:56.603 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:44:56 smithi018 ceph-mon[28799]: from='mgr.14162 172.21.15.18:0/2711576919' entity='mgr.smithi018.xjzwga' 2024-04-12T20:44:56.604 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:44:56 smithi018 ceph-mon[28799]: from='mgr.14162 172.21.15.18:0/2711576919' entity='mgr.smithi018.xjzwga' 2024-04-12T20:44:56.604 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:44:56 smithi018 ceph-mon[28799]: from='mgr.14162 172.21.15.18:0/2711576919' entity='mgr.smithi018.xjzwga' 2024-04-12T20:44:56.604 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:44:56 smithi018 ceph-mon[28799]: from='mgr.14162 172.21.15.18:0/2711576919' entity='mgr.smithi018.xjzwga' 2024-04-12T20:44:57.016 INFO:tasks.cephadm:Adding mon.smithi018 on smithi018 2024-04-12T20:44:57.016 INFO:tasks.cephadm:Adding mon.smithi186 on smithi186 2024-04-12T20:44:57.016 DEBUG:teuthology.orchestra.run.smithi186:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1cbe407a-f90d-11ee-bc8d-c7b262605968 -- ceph orch apply mon '2;smithi018:172.21.15.18=smithi018;smithi186:172.21.15.186=smithi186' 2024-04-12T20:44:57.468 INFO:teuthology.orchestra.run.smithi186.stderr:Inferring config /etc/ceph/ceph.conf 2024-04-12T20:44:57.567 INFO:teuthology.orchestra.run.smithi186.stderr:Inferring config /etc/ceph/ceph.conf 2024-04-12T20:44:57.603 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:44:57 smithi018 ceph-mon[28799]: from='client.? 172.21.15.18:0/511652442' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-04-12T20:44:57.604 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:44:57 smithi018 ceph-mon[28799]: osdmap e4: 0 total, 0 up, 0 in 2024-04-12T20:44:57.604 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:44:57 smithi018 ceph-mon[28799]: from='mgr.14162 172.21.15.18:0/2711576919' entity='mgr.smithi018.xjzwga' 2024-04-12T20:44:57.604 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:44:57 smithi018 ceph-mon[28799]: from='mgr.14162 172.21.15.18:0/2711576919' entity='mgr.smithi018.xjzwga' 2024-04-12T20:44:57.604 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:44:57 smithi018 ceph-mon[28799]: from='mgr.14162 172.21.15.18:0/2711576919' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2024-04-12T20:44:57.604 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:44:57 smithi018 ceph-mon[28799]: from='mgr.14162 172.21.15.18:0/2711576919' entity='mgr.smithi018.xjzwga' 2024-04-12T20:44:58.603 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:44:58 smithi018 ceph-mon[28799]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2024-04-12T20:44:58.603 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:44:58 smithi018 ceph-mon[28799]: Deploying daemon grafana.smithi018 on smithi018 2024-04-12T20:44:58.603 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:44:58 smithi018 ceph-mon[28799]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-12T20:44:59.604 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:44:59.604+0000 7f0518600700 1 -- 172.21.15.186:0/3776932526 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f05100669c0 msgr2=0x7f05100ff640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:44:59.604 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:44:59.604+0000 7f0518600700 1 --2- 172.21.15.186:0/3776932526 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f05100669c0 0x7f05100ff640 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f05040048c0 tx=0x7f0504009e30 comp rx=0 tx=0).stop 2024-04-12T20:44:59.604 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:44:59.604+0000 7f0518600700 1 -- 172.21.15.186:0/3776932526 shutdown_connections 2024-04-12T20:44:59.604 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:44:59.604+0000 7f0518600700 1 --2- 172.21.15.186:0/3776932526 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f05100669c0 0x7f05100ff640 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:44:59.605 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:44:59.604+0000 7f0518600700 1 -- 172.21.15.186:0/3776932526 >> 172.21.15.186:0/3776932526 conn(0x7f05100f7090 msgr2=0x7f05100f94e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T20:44:59.605 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:44:59.604+0000 7f0518600700 1 -- 172.21.15.186:0/3776932526 shutdown_connections 2024-04-12T20:44:59.605 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:44:59.604+0000 7f0518600700 1 -- 172.21.15.186:0/3776932526 wait complete. 2024-04-12T20:44:59.605 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:44:59.605+0000 7f0518600700 1 Processor -- start 2024-04-12T20:44:59.605 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:44:59.605+0000 7f0518600700 1 -- start start 2024-04-12T20:44:59.605 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:44:59.606+0000 7f0518600700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f05100669c0 0x7f05100fcf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T20:44:59.605 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:44:59.606+0000 7f0518600700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05100fd4d0 con 0x7f05100669c0 2024-04-12T20:44:59.606 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:44:59.606+0000 7f0515c00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f05100669c0 0x7f05100fcf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T20:44:59.606 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:44:59.606+0000 7f0515c00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f05100669c0 0x7f05100fcf90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.186:57400/0 (socket says 172.21.15.186:57400) 2024-04-12T20:44:59.606 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:44:59.606+0000 7f0515c00700 1 -- 172.21.15.186:0/418135307 learned_addr learned my addr 172.21.15.186:0/418135307 (peer_addr_for_me v2:172.21.15.186:0/0) 2024-04-12T20:44:59.606 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:44:59.607+0000 7f0515c00700 1 -- 172.21.15.186:0/418135307 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0504005040 con 0x7f05100669c0 2024-04-12T20:44:59.606 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:44:59.608+0000 7f0515c00700 1 --2- 172.21.15.186:0/418135307 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f05100669c0 0x7f05100fcf90 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f050400b900 tx=0x7f0504009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T20:44:59.607 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:44:59.608+0000 7f050d600700 1 -- 172.21.15.186:0/418135307 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f050400b5c0 con 0x7f05100669c0 2024-04-12T20:44:59.607 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:44:59.608+0000 7f0518600700 1 -- 172.21.15.186:0/418135307 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f05100fb5a0 con 0x7f05100669c0 2024-04-12T20:44:59.607 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:44:59.608+0000 7f050d600700 1 -- 172.21.15.186:0/418135307 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f050400be70 con 0x7f05100669c0 2024-04-12T20:44:59.607 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:44:59.608+0000 7f050d600700 1 -- 172.21.15.186:0/418135307 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f050401c590 con 0x7f05100669c0 2024-04-12T20:44:59.607 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:44:59.608+0000 7f0518600700 1 -- 172.21.15.186:0/418135307 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f05100fba90 con 0x7f05100669c0 2024-04-12T20:44:59.608 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:44:59.610+0000 7f0518600700 1 -- 172.21.15.186:0/418135307 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f051005f5f0 con 0x7f05100669c0 2024-04-12T20:44:59.609 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:44:59.610+0000 7f050d600700 1 -- 172.21.15.186:0/418135307 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 13) v1 ==== 43424+0+0 (secure 0 0 0) 0x7f0504004d30 con 0x7f05100669c0 2024-04-12T20:44:59.609 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:44:59.610+0000 7f050d600700 1 --2- 172.21.15.186:0/418135307 >> [v2:172.21.15.18:6800/1391635591,v1:172.21.15.18:6801/1391635591] conn(0x7f04f80368d0 0x7f04f8038d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T20:44:59.609 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:44:59.611+0000 7f050d600700 1 -- 172.21.15.186:0/418135307 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1151+0+0 (secure 0 0 0) 0x7f05040493f0 con 0x7f05100669c0 2024-04-12T20:44:59.610 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:44:59.611+0000 7f0515200700 1 --2- 172.21.15.186:0/418135307 >> [v2:172.21.15.18:6800/1391635591,v1:172.21.15.18:6801/1391635591] conn(0x7f04f80368d0 0x7f04f8038d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T20:44:59.611 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:44:59.613+0000 7f0515200700 1 --2- 172.21.15.186:0/418135307 >> [v2:172.21.15.18:6800/1391635591,v1:172.21.15.18:6801/1391635591] conn(0x7f04f80368d0 0x7f04f8038d80 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f04fc009990 tx=0x7f04fc006e40 comp rx=0 tx=0).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T20:44:59.628 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:44:59.629+0000 7f050d600700 1 -- 172.21.15.186:0/418135307 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7f0504011d90 con 0x7f05100669c0 2024-04-12T20:44:59.850 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:44:59.851+0000 7f0518600700 1 -- 172.21.15.186:0/418135307 --> [v2:172.21.15.18:6800/1391635591,v1:172.21.15.18:6801/1391635591] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "placement": "2;smithi018:172.21.15.18=smithi018;smithi186:172.21.15.186=smithi186", "target": ["mon-mgr", ""]}) v1 -- 0x7f0510106540 con 0x7f04f80368d0 2024-04-12T20:44:59.865 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:44:59.866+0000 7f050d600700 1 -- 172.21.15.186:0/418135307 <== mgr.14162 v2:172.21.15.18:6800/1391635591 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7f0510106540 con 0x7f04f80368d0 2024-04-12T20:44:59.865 INFO:teuthology.orchestra.run.smithi186.stdout:Scheduled mon update... 2024-04-12T20:44:59.868 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:44:59.868+0000 7f0518600700 1 -- 172.21.15.186:0/418135307 >> [v2:172.21.15.18:6800/1391635591,v1:172.21.15.18:6801/1391635591] conn(0x7f04f80368d0 msgr2=0x7f04f8038d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:44:59.868 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:44:59.868+0000 7f0518600700 1 --2- 172.21.15.186:0/418135307 >> [v2:172.21.15.18:6800/1391635591,v1:172.21.15.18:6801/1391635591] conn(0x7f04f80368d0 0x7f04f8038d80 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f04fc009990 tx=0x7f04fc006e40 comp rx=0 tx=0).stop 2024-04-12T20:44:59.868 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:44:59.869+0000 7f0518600700 1 -- 172.21.15.186:0/418135307 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f05100669c0 msgr2=0x7f05100fcf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:44:59.868 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:44:59.869+0000 7f0518600700 1 --2- 172.21.15.186:0/418135307 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f05100669c0 0x7f05100fcf90 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f050400b900 tx=0x7f0504009e30 comp rx=0 tx=0).stop 2024-04-12T20:44:59.868 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:44:59.869+0000 7f0518600700 1 -- 172.21.15.186:0/418135307 shutdown_connections 2024-04-12T20:44:59.868 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:44:59.869+0000 7f0518600700 1 --2- 172.21.15.186:0/418135307 >> [v2:172.21.15.18:6800/1391635591,v1:172.21.15.18:6801/1391635591] conn(0x7f04f80368d0 0x7f04f8038d80 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:44:59.868 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:44:59.869+0000 7f0518600700 1 --2- 172.21.15.186:0/418135307 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f05100669c0 0x7f05100fcf90 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:44:59.869 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:44:59.869+0000 7f0518600700 1 -- 172.21.15.186:0/418135307 >> 172.21.15.186:0/418135307 conn(0x7f05100f7090 msgr2=0x7f0510103b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T20:44:59.869 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:44:59.869+0000 7f0518600700 1 -- 172.21.15.186:0/418135307 shutdown_connections 2024-04-12T20:44:59.869 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:44:59.869+0000 7f0518600700 1 -- 172.21.15.186:0/418135307 wait complete. 2024-04-12T20:45:00.573 DEBUG:teuthology.orchestra.run.smithi186:mon.smithi186> sudo journalctl -f -n 0 -u ceph-1cbe407a-f90d-11ee-bc8d-c7b262605968@mon.smithi186.service 2024-04-12T20:45:00.581 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-04-12T20:45:00.581 DEBUG:teuthology.orchestra.run.smithi186:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1cbe407a-f90d-11ee-bc8d-c7b262605968 -- ceph mon dump -f json 2024-04-12T20:45:00.603 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:45:00 smithi018 ceph-mon[28799]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-12T20:45:00.603 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:45:00 smithi018 ceph-mon[28799]: from='mgr.14162 172.21.15.18:0/2711576919' entity='mgr.smithi018.xjzwga' 2024-04-12T20:45:00.603 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:45:00 smithi018 ceph-mon[28799]: from='mgr.14162 172.21.15.18:0/2711576919' entity='mgr.smithi018.xjzwga' 2024-04-12T20:45:01.053 INFO:teuthology.orchestra.run.smithi186.stderr:Inferring config /etc/ceph/ceph.conf 2024-04-12T20:45:01.157 INFO:teuthology.orchestra.run.smithi186.stderr:Inferring config /etc/ceph/ceph.conf 2024-04-12T20:45:01.603 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:45:01 smithi018 ceph-mon[28799]: from='client.14194 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "2;smithi018:172.21.15.18=smithi018;smithi186:172.21.15.186=smithi186", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T20:45:01.603 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:45:01 smithi018 ceph-mon[28799]: Saving service mon spec with placement smithi018:172.21.15.18=smithi018;smithi186:172.21.15.186=smithi186;count:2 2024-04-12T20:45:02.603 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:45:02 smithi018 ceph-mon[28799]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-12T20:45:02.972 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:45:02.972+0000 7f662dc00700 1 -- 172.21.15.186:0/2797472692 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f66281030c0 msgr2=0x7f66281034d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:45:02.972 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:45:02.972+0000 7f662dc00700 1 --2- 172.21.15.186:0/2797472692 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f66281030c0 0x7f66281034d0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f66180048c0 tx=0x7f6618009e30 comp rx=0 tx=0).stop 2024-04-12T20:45:02.972 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:45:02.972+0000 7f662dc00700 1 -- 172.21.15.186:0/2797472692 shutdown_connections 2024-04-12T20:45:02.972 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:45:02.972+0000 7f662dc00700 1 --2- 172.21.15.186:0/2797472692 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f66281030c0 0x7f66281034d0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:45:02.972 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:45:02.972+0000 7f662dc00700 1 -- 172.21.15.186:0/2797472692 >> 172.21.15.186:0/2797472692 conn(0x7f6628072300 msgr2=0x7f6628074710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T20:45:02.972 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:45:02.973+0000 7f662dc00700 1 -- 172.21.15.186:0/2797472692 shutdown_connections 2024-04-12T20:45:02.972 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:45:02.973+0000 7f662dc00700 1 -- 172.21.15.186:0/2797472692 wait complete. 2024-04-12T20:45:02.973 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:45:02.974+0000 7f662dc00700 1 Processor -- start 2024-04-12T20:45:02.973 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:45:02.974+0000 7f662dc00700 1 -- start start 2024-04-12T20:45:02.973 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:45:02.974+0000 7f662dc00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f66281030c0 0x7f66281136f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T20:45:02.973 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:45:02.974+0000 7f662dc00700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6628113c30 con 0x7f66281030c0 2024-04-12T20:45:02.974 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:45:02.975+0000 7f6626a00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f66281030c0 0x7f66281136f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T20:45:02.974 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:45:02.975+0000 7f6626a00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f66281030c0 0x7f66281136f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.186:57428/0 (socket says 172.21.15.186:57428) 2024-04-12T20:45:02.974 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:45:02.975+0000 7f6626a00700 1 -- 172.21.15.186:0/985283561 learned_addr learned my addr 172.21.15.186:0/985283561 (peer_addr_for_me v2:172.21.15.186:0/0) 2024-04-12T20:45:02.975 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:45:02.976+0000 7f6626a00700 1 -- 172.21.15.186:0/985283561 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6618005040 con 0x7f66281030c0 2024-04-12T20:45:02.975 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:45:02.976+0000 7f6626a00700 1 --2- 172.21.15.186:0/985283561 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f66281030c0 0x7f66281136f0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f661800b900 tx=0x7f6618009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T20:45:02.976 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:45:02.977+0000 7f661ea00700 1 -- 172.21.15.186:0/985283561 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f661800b5c0 con 0x7f66281030c0 2024-04-12T20:45:02.976 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:45:02.977+0000 7f662dc00700 1 -- 172.21.15.186:0/985283561 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6628113e30 con 0x7f66281030c0 2024-04-12T20:45:02.976 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:45:02.977+0000 7f661ea00700 1 -- 172.21.15.186:0/985283561 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f661800be70 con 0x7f66281030c0 2024-04-12T20:45:02.976 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:45:02.977+0000 7f661ea00700 1 -- 172.21.15.186:0/985283561 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f661801c590 con 0x7f66281030c0 2024-04-12T20:45:02.976 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:45:02.977+0000 7f662dc00700 1 -- 172.21.15.186:0/985283561 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f66281149a0 con 0x7f66281030c0 2024-04-12T20:45:02.976 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:45:02.978+0000 7f662dc00700 1 -- 172.21.15.186:0/985283561 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6628109560 con 0x7f66281030c0 2024-04-12T20:45:02.977 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:45:02.978+0000 7f661ea00700 1 -- 172.21.15.186:0/985283561 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 13) v1 ==== 43424+0+0 (secure 0 0 0) 0x7f6618004d30 con 0x7f66281030c0 2024-04-12T20:45:02.978 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:45:02.979+0000 7f661ea00700 1 --2- 172.21.15.186:0/985283561 >> [v2:172.21.15.18:6800/1391635591,v1:172.21.15.18:6801/1391635591] conn(0x7f660c0368d0 0x7f660c038d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T20:45:02.978 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:45:02.979+0000 7f661ea00700 1 -- 172.21.15.186:0/985283561 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1151+0+0 (secure 0 0 0) 0x7f6618049320 con 0x7f66281030c0 2024-04-12T20:45:02.978 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:45:02.980+0000 7f661fe00700 1 --2- 172.21.15.186:0/985283561 >> [v2:172.21.15.18:6800/1391635591,v1:172.21.15.18:6801/1391635591] conn(0x7f660c0368d0 0x7f660c038d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T20:45:02.980 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:45:02.981+0000 7f661fe00700 1 --2- 172.21.15.186:0/985283561 >> [v2:172.21.15.18:6800/1391635591,v1:172.21.15.18:6801/1391635591] conn(0x7f660c0368d0 0x7f660c038d80 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f6610009990 tx=0x7f6610006e40 comp rx=0 tx=0).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T20:45:02.991 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:45:02.992+0000 7f661ea00700 1 -- 172.21.15.186:0/985283561 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7f661801b350 con 0x7f66281030c0 2024-04-12T20:45:03.283 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:45:03.284+0000 7f662dc00700 1 -- 172.21.15.186:0/985283561 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f6628115470 con 0x7f66281030c0 2024-04-12T20:45:03.284 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:45:03.285+0000 7f661ea00700 1 -- 172.21.15.186:0/985283561 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+739 (secure 0 0 0) 0x7f6618021030 con 0x7f66281030c0 2024-04-12T20:45:03.284 INFO:teuthology.orchestra.run.smithi186.stdout: 2024-04-12T20:45:03.284 INFO:teuthology.orchestra.run.smithi186.stdout:{"epoch":1,"fsid":"1cbe407a-f90d-11ee-bc8d-c7b262605968","modified":"2024-04-12T20:42:56.111400Z","created":"2024-04-12T20:42:56.111400Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"smithi018","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:3300","nonce":0},{"type":"v1","addr":"172.21.15.18:6789","nonce":0}]},"addr":"172.21.15.18:6789/0","public_addr":"172.21.15.18:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-04-12T20:45:03.286 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:45:03.288+0000 7f662dc00700 1 -- 172.21.15.186:0/985283561 >> [v2:172.21.15.18:6800/1391635591,v1:172.21.15.18:6801/1391635591] conn(0x7f660c0368d0 msgr2=0x7f660c038d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:45:03.287 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:45:03.288+0000 7f662dc00700 1 --2- 172.21.15.186:0/985283561 >> [v2:172.21.15.18:6800/1391635591,v1:172.21.15.18:6801/1391635591] conn(0x7f660c0368d0 0x7f660c038d80 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f6610009990 tx=0x7f6610006e40 comp rx=0 tx=0).stop 2024-04-12T20:45:03.287 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:45:03.288+0000 7f662dc00700 1 -- 172.21.15.186:0/985283561 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f66281030c0 msgr2=0x7f66281136f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:45:03.287 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:45:03.288+0000 7f662dc00700 1 --2- 172.21.15.186:0/985283561 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f66281030c0 0x7f66281136f0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f661800b900 tx=0x7f6618009e30 comp rx=0 tx=0).stop 2024-04-12T20:45:03.287 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:45:03.288+0000 7f662dc00700 1 -- 172.21.15.186:0/985283561 shutdown_connections 2024-04-12T20:45:03.287 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:45:03.288+0000 7f662dc00700 1 --2- 172.21.15.186:0/985283561 >> [v2:172.21.15.18:6800/1391635591,v1:172.21.15.18:6801/1391635591] conn(0x7f660c0368d0 0x7f660c038d80 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:45:03.288 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:45:03.288+0000 7f662dc00700 1 --2- 172.21.15.186:0/985283561 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f66281030c0 0x7f66281136f0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:45:03.288 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:45:03.288+0000 7f662dc00700 1 -- 172.21.15.186:0/985283561 >> 172.21.15.186:0/985283561 conn(0x7f6628072300 msgr2=0x7f66280741c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T20:45:03.288 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:45:03.288+0000 7f662dc00700 1 -- 172.21.15.186:0/985283561 shutdown_connections 2024-04-12T20:45:03.288 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:45:03.288+0000 7f662dc00700 1 -- 172.21.15.186:0/985283561 wait complete. 2024-04-12T20:45:03.289 INFO:teuthology.orchestra.run.smithi186.stderr:dumped monmap epoch 1 2024-04-12T20:45:04.603 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:45:04 smithi018 ceph-mon[28799]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-12T20:45:04.603 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:45:04 smithi018 ceph-mon[28799]: from='client.? 172.21.15.186:0/985283561' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-12T20:45:04.922 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-04-12T20:45:04.922 DEBUG:teuthology.orchestra.run.smithi186:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1cbe407a-f90d-11ee-bc8d-c7b262605968 -- ceph mon dump -f json 2024-04-12T20:45:05.381 INFO:teuthology.orchestra.run.smithi186.stderr:Inferring config /etc/ceph/ceph.conf 2024-04-12T20:45:05.484 INFO:teuthology.orchestra.run.smithi186.stderr:Inferring config /etc/ceph/ceph.conf 2024-04-12T20:45:06.603 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:45:06 smithi018 ceph-mon[28799]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-12T20:45:07.440 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:45:07.441+0000 7fa586000700 1 -- 172.21.15.186:0/1961592025 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa5801030c0 msgr2=0x7fa5801034d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:45:07.441 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:45:07.441+0000 7fa586000700 1 --2- 172.21.15.186:0/1961592025 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa5801030c0 0x7fa5801034d0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fa5700048c0 tx=0x7fa570009e30 comp rx=0 tx=0).stop 2024-04-12T20:45:07.441 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:45:07.441+0000 7fa586000700 1 -- 172.21.15.186:0/1961592025 shutdown_connections 2024-04-12T20:45:07.441 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:45:07.441+0000 7fa586000700 1 --2- 172.21.15.186:0/1961592025 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa5801030c0 0x7fa5801034d0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:45:07.441 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:45:07.441+0000 7fa586000700 1 -- 172.21.15.186:0/1961592025 >> 172.21.15.186:0/1961592025 conn(0x7fa580072300 msgr2=0x7fa580074710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T20:45:07.441 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:45:07.441+0000 7fa586000700 1 -- 172.21.15.186:0/1961592025 shutdown_connections 2024-04-12T20:45:07.441 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:45:07.442+0000 7fa586000700 1 -- 172.21.15.186:0/1961592025 wait complete. 2024-04-12T20:45:07.442 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:45:07.443+0000 7fa586000700 1 Processor -- start 2024-04-12T20:45:07.442 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:45:07.443+0000 7fa586000700 1 -- start start 2024-04-12T20:45:07.442 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:45:07.443+0000 7fa586000700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa5801030c0 0x7fa5801136f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T20:45:07.442 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:45:07.443+0000 7fa586000700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa580113c30 con 0x7fa5801030c0 2024-04-12T20:45:07.442 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:45:07.444+0000 7fa57f400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa5801030c0 0x7fa5801136f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T20:45:07.443 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:45:07.444+0000 7fa57f400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa5801030c0 0x7fa5801136f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.186:42056/0 (socket says 172.21.15.186:42056) 2024-04-12T20:45:07.443 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:45:07.444+0000 7fa57f400700 1 -- 172.21.15.186:0/209345777 learned_addr learned my addr 172.21.15.186:0/209345777 (peer_addr_for_me v2:172.21.15.186:0/0) 2024-04-12T20:45:07.443 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:45:07.445+0000 7fa57f400700 1 -- 172.21.15.186:0/209345777 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa570005040 con 0x7fa5801030c0 2024-04-12T20:45:07.444 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:45:07.445+0000 7fa57f400700 1 --2- 172.21.15.186:0/209345777 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa5801030c0 0x7fa5801136f0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fa57000b900 tx=0x7fa570009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T20:45:07.444 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:45:07.445+0000 7fa577400700 1 -- 172.21.15.186:0/209345777 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fa57000b5c0 con 0x7fa5801030c0 2024-04-12T20:45:07.445 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:45:07.446+0000 7fa586000700 1 -- 172.21.15.186:0/209345777 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa580113e30 con 0x7fa5801030c0 2024-04-12T20:45:07.445 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:45:07.446+0000 7fa577400700 1 -- 172.21.15.186:0/209345777 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa57000be70 con 0x7fa5801030c0 2024-04-12T20:45:07.445 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:45:07.446+0000 7fa577400700 1 -- 172.21.15.186:0/209345777 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fa57001c590 con 0x7fa5801030c0 2024-04-12T20:45:07.445 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:45:07.446+0000 7fa586000700 1 -- 172.21.15.186:0/209345777 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa5801149a0 con 0x7fa5801030c0 2024-04-12T20:45:07.446 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:45:07.447+0000 7fa586000700 1 -- 172.21.15.186:0/209345777 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa580109560 con 0x7fa5801030c0 2024-04-12T20:45:07.446 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:45:07.447+0000 7fa577400700 1 -- 172.21.15.186:0/209345777 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 13) v1 ==== 43424+0+0 (secure 0 0 0) 0x7fa570004d30 con 0x7fa5801030c0 2024-04-12T20:45:07.446 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:45:07.448+0000 7fa577400700 1 --2- 172.21.15.186:0/209345777 >> [v2:172.21.15.18:6800/1391635591,v1:172.21.15.18:6801/1391635591] conn(0x7fa5640368d0 0x7fa564038d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T20:45:07.447 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:45:07.448+0000 7fa577400700 1 -- 172.21.15.186:0/209345777 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1151+0+0 (secure 0 0 0) 0x7fa570049400 con 0x7fa5801030c0 2024-04-12T20:45:07.447 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:45:07.448+0000 7fa57ea00700 1 --2- 172.21.15.186:0/209345777 >> [v2:172.21.15.18:6800/1391635591,v1:172.21.15.18:6801/1391635591] conn(0x7fa5640368d0 0x7fa564038d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T20:45:07.448 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:45:07.449+0000 7fa57ea00700 1 --2- 172.21.15.186:0/209345777 >> [v2:172.21.15.18:6800/1391635591,v1:172.21.15.18:6801/1391635591] conn(0x7fa5640368d0 0x7fa564038d80 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fa56c009990 tx=0x7fa56c006e40 comp rx=0 tx=0).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T20:45:07.462 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:45:07.463+0000 7fa577400700 1 -- 172.21.15.186:0/209345777 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7fa57001b350 con 0x7fa5801030c0 2024-04-12T20:45:07.760 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:45:07.760+0000 7fa586000700 1 -- 172.21.15.186:0/209345777 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7fa5801153b0 con 0x7fa5801030c0 2024-04-12T20:45:07.760 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:45:07.761+0000 7fa577400700 1 -- 172.21.15.186:0/209345777 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+739 (secure 0 0 0) 0x7fa570021020 con 0x7fa5801030c0 2024-04-12T20:45:07.761 INFO:teuthology.orchestra.run.smithi186.stdout: 2024-04-12T20:45:07.761 INFO:teuthology.orchestra.run.smithi186.stdout:{"epoch":1,"fsid":"1cbe407a-f90d-11ee-bc8d-c7b262605968","modified":"2024-04-12T20:42:56.111400Z","created":"2024-04-12T20:42:56.111400Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"smithi018","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:3300","nonce":0},{"type":"v1","addr":"172.21.15.18:6789","nonce":0}]},"addr":"172.21.15.18:6789/0","public_addr":"172.21.15.18:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-04-12T20:45:07.763 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:45:07.764+0000 7fa586000700 1 -- 172.21.15.186:0/209345777 >> [v2:172.21.15.18:6800/1391635591,v1:172.21.15.18:6801/1391635591] conn(0x7fa5640368d0 msgr2=0x7fa564038d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:45:07.763 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:45:07.764+0000 7fa586000700 1 --2- 172.21.15.186:0/209345777 >> [v2:172.21.15.18:6800/1391635591,v1:172.21.15.18:6801/1391635591] conn(0x7fa5640368d0 0x7fa564038d80 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fa56c009990 tx=0x7fa56c006e40 comp rx=0 tx=0).stop 2024-04-12T20:45:07.763 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:45:07.764+0000 7fa586000700 1 -- 172.21.15.186:0/209345777 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa5801030c0 msgr2=0x7fa5801136f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:45:07.763 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:45:07.764+0000 7fa586000700 1 --2- 172.21.15.186:0/209345777 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa5801030c0 0x7fa5801136f0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fa57000b900 tx=0x7fa570009e30 comp rx=0 tx=0).stop 2024-04-12T20:45:07.763 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:45:07.764+0000 7fa586000700 1 -- 172.21.15.186:0/209345777 shutdown_connections 2024-04-12T20:45:07.764 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:45:07.764+0000 7fa586000700 1 --2- 172.21.15.186:0/209345777 >> [v2:172.21.15.18:6800/1391635591,v1:172.21.15.18:6801/1391635591] conn(0x7fa5640368d0 0x7fa564038d80 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:45:07.764 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:45:07.764+0000 7fa586000700 1 --2- 172.21.15.186:0/209345777 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa5801030c0 0x7fa5801136f0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:45:07.764 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:45:07.764+0000 7fa586000700 1 -- 172.21.15.186:0/209345777 >> 172.21.15.186:0/209345777 conn(0x7fa580072300 msgr2=0x7fa5800741c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T20:45:07.764 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:45:07.764+0000 7fa586000700 1 -- 172.21.15.186:0/209345777 shutdown_connections 2024-04-12T20:45:07.764 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:45:07.764+0000 7fa586000700 1 -- 172.21.15.186:0/209345777 wait complete. 2024-04-12T20:45:07.764 INFO:teuthology.orchestra.run.smithi186.stderr:dumped monmap epoch 1 2024-04-12T20:45:08.353 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:45:08 smithi018 ceph-mon[28799]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-12T20:45:08.354 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:45:08 smithi018 ceph-mon[28799]: from='client.? 172.21.15.186:0/209345777' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-12T20:45:09.408 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-04-12T20:45:09.408 DEBUG:teuthology.orchestra.run.smithi186:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1cbe407a-f90d-11ee-bc8d-c7b262605968 -- ceph mon dump -f json 2024-04-12T20:45:09.862 INFO:teuthology.orchestra.run.smithi186.stderr:Inferring config /etc/ceph/ceph.conf 2024-04-12T20:45:09.968 INFO:teuthology.orchestra.run.smithi186.stderr:Inferring config /etc/ceph/ceph.conf 2024-04-12T20:45:10.606 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:45:10 smithi018 ceph-mon[28799]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-12T20:45:11.896 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:45:11.895+0000 7f38ec400700 1 -- 172.21.15.186:0/1899998281 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f38e40fca60 msgr2=0x7f38e40fce70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:45:11.896 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:45:11.895+0000 7f38ec400700 1 --2- 172.21.15.186:0/1899998281 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f38e40fca60 0x7f38e40fce70 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f38d80048c0 tx=0x7f38d8009e30 comp rx=0 tx=0).stop 2024-04-12T20:45:11.896 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:45:11.896+0000 7f38ec400700 1 -- 172.21.15.186:0/1899998281 shutdown_connections 2024-04-12T20:45:11.896 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:45:11.896+0000 7f38ec400700 1 --2- 172.21.15.186:0/1899998281 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f38e40fca60 0x7f38e40fce70 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:45:11.896 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:45:11.896+0000 7f38ec400700 1 -- 172.21.15.186:0/1899998281 >> 172.21.15.186:0/1899998281 conn(0x7f38e40f81c0 msgr2=0x7f38e40fa5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T20:45:11.896 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:45:11.896+0000 7f38ec400700 1 -- 172.21.15.186:0/1899998281 shutdown_connections 2024-04-12T20:45:11.896 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:45:11.896+0000 7f38ec400700 1 -- 172.21.15.186:0/1899998281 wait complete. 2024-04-12T20:45:11.897 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:45:11.897+0000 7f38ec400700 1 Processor -- start 2024-04-12T20:45:11.897 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:45:11.897+0000 7f38ec400700 1 -- start start 2024-04-12T20:45:11.897 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:45:11.897+0000 7f38ec400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f38e40fca60 0x7f38e410d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T20:45:11.897 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:45:11.897+0000 7f38ec400700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38e410d5f0 con 0x7f38e40fca60 2024-04-12T20:45:11.898 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:45:11.898+0000 7f38e9a00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f38e40fca60 0x7f38e410d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T20:45:11.898 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:45:11.898+0000 7f38e9a00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f38e40fca60 0x7f38e410d0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.186:42068/0 (socket says 172.21.15.186:42068) 2024-04-12T20:45:11.898 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:45:11.898+0000 7f38e9a00700 1 -- 172.21.15.186:0/1613581302 learned_addr learned my addr 172.21.15.186:0/1613581302 (peer_addr_for_me v2:172.21.15.186:0/0) 2024-04-12T20:45:11.898 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:45:11.899+0000 7f38e9a00700 1 -- 172.21.15.186:0/1613581302 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38d8005040 con 0x7f38e40fca60 2024-04-12T20:45:11.899 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:45:11.899+0000 7f38e9a00700 1 --2- 172.21.15.186:0/1613581302 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f38e40fca60 0x7f38e410d0b0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f38d8009680 tx=0x7f38d8009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T20:45:11.899 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:45:11.899+0000 7f38e1600700 1 -- 172.21.15.186:0/1613581302 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f38d800b5c0 con 0x7f38e40fca60 2024-04-12T20:45:11.899 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:45:11.899+0000 7f38ec400700 1 -- 172.21.15.186:0/1613581302 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f38e410d7f0 con 0x7f38e40fca60 2024-04-12T20:45:11.900 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:45:11.900+0000 7f38ec400700 1 -- 172.21.15.186:0/1613581302 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f38e410e360 con 0x7f38e40fca60 2024-04-12T20:45:11.900 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:45:11.900+0000 7f38e1600700 1 -- 172.21.15.186:0/1613581302 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f38d800be70 con 0x7f38e40fca60 2024-04-12T20:45:11.900 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:45:11.900+0000 7f38e1600700 1 -- 172.21.15.186:0/1613581302 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f38d80135c0 con 0x7f38e40fca60 2024-04-12T20:45:11.900 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:45:11.901+0000 7f38ec400700 1 -- 172.21.15.186:0/1613581302 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f38e4102f70 con 0x7f38e40fca60 2024-04-12T20:45:11.901 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:45:11.901+0000 7f38e1600700 1 -- 172.21.15.186:0/1613581302 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 13) v1 ==== 43424+0+0 (secure 0 0 0) 0x7f38d8004d30 con 0x7f38e40fca60 2024-04-12T20:45:11.901 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:45:11.902+0000 7f38e1600700 1 --2- 172.21.15.186:0/1613581302 >> [v2:172.21.15.18:6800/1391635591,v1:172.21.15.18:6801/1391635591] conn(0x7f38cc036880 0x7f38cc038d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T20:45:11.901 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:45:11.902+0000 7f38e1600700 1 -- 172.21.15.186:0/1613581302 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1151+0+0 (secure 0 0 0) 0x7f38d80493d0 con 0x7f38e40fca60 2024-04-12T20:45:11.902 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:45:11.902+0000 7f38e9000700 1 --2- 172.21.15.186:0/1613581302 >> [v2:172.21.15.18:6800/1391635591,v1:172.21.15.18:6801/1391635591] conn(0x7f38cc036880 0x7f38cc038d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T20:45:11.903 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:45:11.904+0000 7f38e9000700 1 --2- 172.21.15.186:0/1613581302 >> [v2:172.21.15.18:6800/1391635591,v1:172.21.15.18:6801/1391635591] conn(0x7f38cc036880 0x7f38cc038d30 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f38d0009990 tx=0x7f38d0006e40 comp rx=0 tx=0).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T20:45:11.916 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:45:11.916+0000 7f38e1600700 1 -- 172.21.15.186:0/1613581302 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7f38d8011550 con 0x7f38e40fca60 2024-04-12T20:45:12.207 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:45:12.207+0000 7f38ec400700 1 -- 172.21.15.186:0/1613581302 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f38e405f5f0 con 0x7f38e40fca60 2024-04-12T20:45:12.208 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:45:12.208+0000 7f38e1600700 1 -- 172.21.15.186:0/1613581302 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+739 (secure 0 0 0) 0x7f38d8021020 con 0x7f38e40fca60 2024-04-12T20:45:12.208 INFO:teuthology.orchestra.run.smithi186.stdout: 2024-04-12T20:45:12.208 INFO:teuthology.orchestra.run.smithi186.stdout:{"epoch":1,"fsid":"1cbe407a-f90d-11ee-bc8d-c7b262605968","modified":"2024-04-12T20:42:56.111400Z","created":"2024-04-12T20:42:56.111400Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"smithi018","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:3300","nonce":0},{"type":"v1","addr":"172.21.15.18:6789","nonce":0}]},"addr":"172.21.15.18:6789/0","public_addr":"172.21.15.18:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-04-12T20:45:12.211 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:45:12.211+0000 7f38ec400700 1 -- 172.21.15.186:0/1613581302 >> [v2:172.21.15.18:6800/1391635591,v1:172.21.15.18:6801/1391635591] conn(0x7f38cc036880 msgr2=0x7f38cc038d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:45:12.211 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:45:12.211+0000 7f38ec400700 1 --2- 172.21.15.186:0/1613581302 >> [v2:172.21.15.18:6800/1391635591,v1:172.21.15.18:6801/1391635591] conn(0x7f38cc036880 0x7f38cc038d30 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f38d0009990 tx=0x7f38d0006e40 comp rx=0 tx=0).stop 2024-04-12T20:45:12.211 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:45:12.211+0000 7f38ec400700 1 -- 172.21.15.186:0/1613581302 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f38e40fca60 msgr2=0x7f38e410d0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:45:12.211 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:45:12.211+0000 7f38ec400700 1 --2- 172.21.15.186:0/1613581302 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f38e40fca60 0x7f38e410d0b0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f38d8009680 tx=0x7f38d8009e30 comp rx=0 tx=0).stop 2024-04-12T20:45:12.211 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:45:12.211+0000 7f38ec400700 1 -- 172.21.15.186:0/1613581302 shutdown_connections 2024-04-12T20:45:12.211 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:45:12.211+0000 7f38ec400700 1 --2- 172.21.15.186:0/1613581302 >> [v2:172.21.15.18:6800/1391635591,v1:172.21.15.18:6801/1391635591] conn(0x7f38cc036880 0x7f38cc038d30 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:45:12.211 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:45:12.211+0000 7f38ec400700 1 --2- 172.21.15.186:0/1613581302 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f38e40fca60 0x7f38e410d0b0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:45:12.211 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:45:12.211+0000 7f38ec400700 1 -- 172.21.15.186:0/1613581302 >> 172.21.15.186:0/1613581302 conn(0x7f38e40f81c0 msgr2=0x7f38e4103ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T20:45:12.211 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:45:12.211+0000 7f38ec400700 1 -- 172.21.15.186:0/1613581302 shutdown_connections 2024-04-12T20:45:12.211 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:45:12.211+0000 7f38ec400700 1 -- 172.21.15.186:0/1613581302 wait complete. 2024-04-12T20:45:12.212 INFO:teuthology.orchestra.run.smithi186.stderr:dumped monmap epoch 1 2024-04-12T20:45:12.534 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:45:12 smithi018 ceph-mon[28799]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-12T20:45:12.534 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:45:12 smithi018 ceph-mon[28799]: from='client.? 172.21.15.186:0/1613581302' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-12T20:45:13.862 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-04-12T20:45:13.863 DEBUG:teuthology.orchestra.run.smithi186:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1cbe407a-f90d-11ee-bc8d-c7b262605968 -- ceph mon dump -f json 2024-04-12T20:45:14.303 INFO:teuthology.orchestra.run.smithi186.stderr:Inferring config /etc/ceph/ceph.conf 2024-04-12T20:45:14.398 INFO:teuthology.orchestra.run.smithi186.stderr:Inferring config /etc/ceph/ceph.conf 2024-04-12T20:45:14.417 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:45:14 smithi018 ceph-mon[28799]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-12T20:45:16.286 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:45:16 smithi018 ceph-mon[28799]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-12T20:45:16.286 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:45:16 smithi018 ceph-mon[28799]: from='mgr.14162 172.21.15.18:0/2711576919' entity='mgr.smithi018.xjzwga' 2024-04-12T20:45:16.286 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:45:16 smithi018 ceph-mon[28799]: from='mgr.14162 172.21.15.18:0/2711576919' entity='mgr.smithi018.xjzwga' 2024-04-12T20:45:16.286 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:45:16 smithi018 ceph-mon[28799]: from='mgr.14162 172.21.15.18:0/2711576919' entity='mgr.smithi018.xjzwga' 2024-04-12T20:45:16.286 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:45:16 smithi018 ceph-mon[28799]: from='mgr.14162 172.21.15.18:0/2711576919' entity='mgr.smithi018.xjzwga' 2024-04-12T20:45:16.286 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:45:16 smithi018 ceph-mon[28799]: from='mgr.14162 172.21.15.18:0/2711576919' entity='mgr.smithi018.xjzwga' 2024-04-12T20:45:16.286 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:45:16 smithi018 ceph-mon[28799]: from='mgr.14162 172.21.15.18:0/2711576919' entity='mgr.smithi018.xjzwga' 2024-04-12T20:45:16.335 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:45:16.334+0000 7fe60fc00700 1 -- 172.21.15.186:0/227797346 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe6081030c0 msgr2=0x7fe6081034d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:45:16.335 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:45:16.334+0000 7fe60fc00700 1 --2- 172.21.15.186:0/227797346 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe6081030c0 0x7fe6081034d0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fe5fc0048c0 tx=0x7fe5fc009e30 comp rx=0 tx=0).stop 2024-04-12T20:45:16.335 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:45:16.334+0000 7fe60fc00700 1 -- 172.21.15.186:0/227797346 shutdown_connections 2024-04-12T20:45:16.336 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:45:16.334+0000 7fe60fc00700 1 --2- 172.21.15.186:0/227797346 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe6081030c0 0x7fe6081034d0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:45:16.336 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:45:16.334+0000 7fe60fc00700 1 -- 172.21.15.186:0/227797346 >> 172.21.15.186:0/227797346 conn(0x7fe608072300 msgr2=0x7fe608074710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T20:45:16.336 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:45:16.335+0000 7fe60fc00700 1 -- 172.21.15.186:0/227797346 shutdown_connections 2024-04-12T20:45:16.336 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:45:16.335+0000 7fe60fc00700 1 -- 172.21.15.186:0/227797346 wait complete. 2024-04-12T20:45:16.336 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:45:16.336+0000 7fe60fc00700 1 Processor -- start 2024-04-12T20:45:16.337 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:45:16.336+0000 7fe60fc00700 1 -- start start 2024-04-12T20:45:16.337 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:45:16.336+0000 7fe60fc00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe6081030c0 0x7fe6081136f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T20:45:16.337 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:45:16.336+0000 7fe60fc00700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe608113c30 con 0x7fe6081030c0 2024-04-12T20:45:16.337 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:45:16.337+0000 7fe60d200700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe6081030c0 0x7fe6081136f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T20:45:16.337 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:45:16.337+0000 7fe60d200700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe6081030c0 0x7fe6081136f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.186:56858/0 (socket says 172.21.15.186:56858) 2024-04-12T20:45:16.337 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:45:16.337+0000 7fe60d200700 1 -- 172.21.15.186:0/995251818 learned_addr learned my addr 172.21.15.186:0/995251818 (peer_addr_for_me v2:172.21.15.186:0/0) 2024-04-12T20:45:16.337 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:45:16.337+0000 7fe60d200700 1 -- 172.21.15.186:0/995251818 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5fc005040 con 0x7fe6081030c0 2024-04-12T20:45:16.338 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:45:16.337+0000 7fe60d200700 1 --2- 172.21.15.186:0/995251818 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe6081030c0 0x7fe6081136f0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fe5fc00b900 tx=0x7fe5fc009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T20:45:16.338 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:45:16.338+0000 7fe604c00700 1 -- 172.21.15.186:0/995251818 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fe5fc00b5c0 con 0x7fe6081030c0 2024-04-12T20:45:16.338 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:45:16.338+0000 7fe604c00700 1 -- 172.21.15.186:0/995251818 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe5fc00be70 con 0x7fe6081030c0 2024-04-12T20:45:16.338 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:45:16.338+0000 7fe60fc00700 1 -- 172.21.15.186:0/995251818 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe608113e30 con 0x7fe6081030c0 2024-04-12T20:45:16.338 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:45:16.338+0000 7fe604c00700 1 -- 172.21.15.186:0/995251818 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fe5fc01c590 con 0x7fe6081030c0 2024-04-12T20:45:16.339 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:45:16.338+0000 7fe60fc00700 1 -- 172.21.15.186:0/995251818 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe6081149a0 con 0x7fe6081030c0 2024-04-12T20:45:16.339 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:45:16.339+0000 7fe60fc00700 1 -- 172.21.15.186:0/995251818 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe608109560 con 0x7fe6081030c0 2024-04-12T20:45:16.339 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:45:16.339+0000 7fe604c00700 1 -- 172.21.15.186:0/995251818 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 13) v1 ==== 43424+0+0 (secure 0 0 0) 0x7fe5fc004d30 con 0x7fe6081030c0 2024-04-12T20:45:16.340 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:45:16.340+0000 7fe604c00700 1 --2- 172.21.15.186:0/995251818 >> [v2:172.21.15.18:6800/1391635591,v1:172.21.15.18:6801/1391635591] conn(0x7fe5f0036880 0x7fe5f0038d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T20:45:16.340 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:45:16.340+0000 7fe604c00700 1 -- 172.21.15.186:0/995251818 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1151+0+0 (secure 0 0 0) 0x7fe5fc049450 con 0x7fe6081030c0 2024-04-12T20:45:16.340 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:45:16.340+0000 7fe607e00700 1 --2- 172.21.15.186:0/995251818 >> [v2:172.21.15.18:6800/1391635591,v1:172.21.15.18:6801/1391635591] conn(0x7fe5f0036880 0x7fe5f0038d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T20:45:16.341 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:45:16.341+0000 7fe607e00700 1 --2- 172.21.15.186:0/995251818 >> [v2:172.21.15.18:6800/1391635591,v1:172.21.15.18:6801/1391635591] conn(0x7fe5f0036880 0x7fe5f0038d30 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fe5f4009990 tx=0x7fe5f4006e40 comp rx=0 tx=0).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T20:45:16.346 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:45:16.346+0000 7fe604c00700 1 -- 172.21.15.186:0/995251818 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7fe5fc01b350 con 0x7fe6081030c0 2024-04-12T20:45:16.640 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:45:16.640+0000 7fe60fc00700 1 -- 172.21.15.186:0/995251818 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7fe6081153b0 con 0x7fe6081030c0 2024-04-12T20:45:16.641 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:45:16.641+0000 7fe604c00700 1 -- 172.21.15.186:0/995251818 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+739 (secure 0 0 0) 0x7fe5fc021020 con 0x7fe6081030c0 2024-04-12T20:45:16.642 INFO:teuthology.orchestra.run.smithi186.stdout: 2024-04-12T20:45:16.642 INFO:teuthology.orchestra.run.smithi186.stdout:{"epoch":1,"fsid":"1cbe407a-f90d-11ee-bc8d-c7b262605968","modified":"2024-04-12T20:42:56.111400Z","created":"2024-04-12T20:42:56.111400Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"smithi018","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:3300","nonce":0},{"type":"v1","addr":"172.21.15.18:6789","nonce":0}]},"addr":"172.21.15.18:6789/0","public_addr":"172.21.15.18:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-04-12T20:45:16.644 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:45:16.644+0000 7fe60fc00700 1 -- 172.21.15.186:0/995251818 >> [v2:172.21.15.18:6800/1391635591,v1:172.21.15.18:6801/1391635591] conn(0x7fe5f0036880 msgr2=0x7fe5f0038d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:45:16.644 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:45:16.644+0000 7fe60fc00700 1 --2- 172.21.15.186:0/995251818 >> [v2:172.21.15.18:6800/1391635591,v1:172.21.15.18:6801/1391635591] conn(0x7fe5f0036880 0x7fe5f0038d30 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fe5f4009990 tx=0x7fe5f4006e40 comp rx=0 tx=0).stop 2024-04-12T20:45:16.644 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:45:16.644+0000 7fe60fc00700 1 -- 172.21.15.186:0/995251818 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe6081030c0 msgr2=0x7fe6081136f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:45:16.645 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:45:16.644+0000 7fe60fc00700 1 --2- 172.21.15.186:0/995251818 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe6081030c0 0x7fe6081136f0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fe5fc00b900 tx=0x7fe5fc009e30 comp rx=0 tx=0).stop 2024-04-12T20:45:16.645 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:45:16.644+0000 7fe60fc00700 1 -- 172.21.15.186:0/995251818 shutdown_connections 2024-04-12T20:45:16.645 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:45:16.644+0000 7fe60fc00700 1 --2- 172.21.15.186:0/995251818 >> [v2:172.21.15.18:6800/1391635591,v1:172.21.15.18:6801/1391635591] conn(0x7fe5f0036880 0x7fe5f0038d30 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:45:16.645 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:45:16.644+0000 7fe60fc00700 1 --2- 172.21.15.186:0/995251818 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe6081030c0 0x7fe6081136f0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:45:16.645 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:45:16.644+0000 7fe60fc00700 1 -- 172.21.15.186:0/995251818 >> 172.21.15.186:0/995251818 conn(0x7fe608072300 msgr2=0x7fe6080741c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T20:45:16.645 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:45:16.645+0000 7fe60fc00700 1 -- 172.21.15.186:0/995251818 shutdown_connections 2024-04-12T20:45:16.645 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:45:16.645+0000 7fe60fc00700 1 -- 172.21.15.186:0/995251818 wait complete. 2024-04-12T20:45:16.646 INFO:teuthology.orchestra.run.smithi186.stderr:dumped monmap epoch 1 2024-04-12T20:45:17.603 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:45:17 smithi018 ceph-mon[28799]: Deploying daemon prometheus.smithi018 on smithi018 2024-04-12T20:45:17.603 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:45:17 smithi018 ceph-mon[28799]: from='client.? 172.21.15.186:0/995251818' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-12T20:45:18.371 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-04-12T20:45:18.372 DEBUG:teuthology.orchestra.run.smithi186:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1cbe407a-f90d-11ee-bc8d-c7b262605968 -- ceph mon dump -f json 2024-04-12T20:45:18.603 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:45:18 smithi018 ceph-mon[28799]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-12T20:45:18.823 INFO:teuthology.orchestra.run.smithi186.stderr:Inferring config /etc/ceph/ceph.conf 2024-04-12T20:45:18.934 INFO:teuthology.orchestra.run.smithi186.stderr:Inferring config /etc/ceph/ceph.conf 2024-04-12T20:45:20.603 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:45:20 smithi018 ceph-mon[28799]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-12T20:45:20.603 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:45:20 smithi018 ceph-mon[28799]: from='mgr.14162 172.21.15.18:0/2711576919' entity='mgr.smithi018.xjzwga' 2024-04-12T20:45:20.751 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:45:20.750+0000 7fae82c00700 1 -- 172.21.15.186:0/1362415484 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fae7c0fec70 msgr2=0x7fae7c0ff080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:45:20.751 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:45:20.750+0000 7fae82c00700 1 --2- 172.21.15.186:0/1362415484 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fae7c0fec70 0x7fae7c0ff080 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fae6c0048c0 tx=0x7fae6c009e30 comp rx=0 tx=0).stop 2024-04-12T20:45:20.751 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:45:20.750+0000 7fae82c00700 1 -- 172.21.15.186:0/1362415484 shutdown_connections 2024-04-12T20:45:20.751 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:45:20.750+0000 7fae82c00700 1 --2- 172.21.15.186:0/1362415484 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fae7c0fec70 0x7fae7c0ff080 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:45:20.751 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:45:20.750+0000 7fae82c00700 1 -- 172.21.15.186:0/1362415484 >> 172.21.15.186:0/1362415484 conn(0x7fae7c0fa5d0 msgr2=0x7fae7c0fc9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T20:45:20.751 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:45:20.751+0000 7fae82c00700 1 -- 172.21.15.186:0/1362415484 shutdown_connections 2024-04-12T20:45:20.751 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:45:20.751+0000 7fae82c00700 1 -- 172.21.15.186:0/1362415484 wait complete. 2024-04-12T20:45:20.752 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:45:20.752+0000 7fae82c00700 1 Processor -- start 2024-04-12T20:45:20.752 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:45:20.752+0000 7fae82c00700 1 -- start start 2024-04-12T20:45:20.752 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:45:20.752+0000 7fae82c00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fae7c0fec70 0x7fae7c10f2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T20:45:20.752 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:45:20.752+0000 7fae82c00700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae7c10f830 con 0x7fae7c0fec70 2024-04-12T20:45:20.753 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:45:20.753+0000 7fae7be00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fae7c0fec70 0x7fae7c10f2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T20:45:20.753 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:45:20.753+0000 7fae7be00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fae7c0fec70 0x7fae7c10f2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.186:56882/0 (socket says 172.21.15.186:56882) 2024-04-12T20:45:20.753 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:45:20.753+0000 7fae7be00700 1 -- 172.21.15.186:0/4237619884 learned_addr learned my addr 172.21.15.186:0/4237619884 (peer_addr_for_me v2:172.21.15.186:0/0) 2024-04-12T20:45:20.753 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:45:20.754+0000 7fae7be00700 1 -- 172.21.15.186:0/4237619884 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae6c005040 con 0x7fae7c0fec70 2024-04-12T20:45:20.754 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:45:20.754+0000 7fae7be00700 1 --2- 172.21.15.186:0/4237619884 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fae7c0fec70 0x7fae7c10f2f0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fae6c009680 tx=0x7fae6c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T20:45:20.754 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:45:20.754+0000 7fae73e00700 1 -- 172.21.15.186:0/4237619884 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fae6c00b5c0 con 0x7fae7c0fec70 2024-04-12T20:45:20.755 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:45:20.755+0000 7fae82c00700 1 -- 172.21.15.186:0/4237619884 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fae7c10fa30 con 0x7fae7c0fec70 2024-04-12T20:45:20.755 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:45:20.755+0000 7fae82c00700 1 -- 172.21.15.186:0/4237619884 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fae7c110550 con 0x7fae7c0fec70 2024-04-12T20:45:20.755 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:45:20.755+0000 7fae73e00700 1 -- 172.21.15.186:0/4237619884 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fae6c00be70 con 0x7fae7c0fec70 2024-04-12T20:45:20.755 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:45:20.755+0000 7fae73e00700 1 -- 172.21.15.186:0/4237619884 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fae6c0135c0 con 0x7fae7c0fec70 2024-04-12T20:45:20.755 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:45:20.756+0000 7fae82c00700 1 -- 172.21.15.186:0/4237619884 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fae7c105180 con 0x7fae7c0fec70 2024-04-12T20:45:20.756 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:45:20.757+0000 7fae73e00700 1 -- 172.21.15.186:0/4237619884 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 13) v1 ==== 43424+0+0 (secure 0 0 0) 0x7fae6c004d30 con 0x7fae7c0fec70 2024-04-12T20:45:20.757 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:45:20.757+0000 7fae73e00700 1 --2- 172.21.15.186:0/4237619884 >> [v2:172.21.15.18:6800/1391635591,v1:172.21.15.18:6801/1391635591] conn(0x7fae60036830 0x7fae60038ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T20:45:20.757 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:45:20.757+0000 7fae73e00700 1 -- 172.21.15.186:0/4237619884 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1151+0+0 (secure 0 0 0) 0x7fae6c049650 con 0x7fae7c0fec70 2024-04-12T20:45:20.757 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:45:20.758+0000 7fae7b400700 1 --2- 172.21.15.186:0/4237619884 >> [v2:172.21.15.18:6800/1391635591,v1:172.21.15.18:6801/1391635591] conn(0x7fae60036830 0x7fae60038ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T20:45:20.758 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:45:20.759+0000 7fae7b400700 1 --2- 172.21.15.186:0/4237619884 >> [v2:172.21.15.18:6800/1391635591,v1:172.21.15.18:6801/1391635591] conn(0x7fae60036830 0x7fae60038ce0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fae68009990 tx=0x7fae68006e40 comp rx=0 tx=0).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T20:45:20.773 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:45:20.773+0000 7fae73e00700 1 -- 172.21.15.186:0/4237619884 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7fae6c011bd0 con 0x7fae7c0fec70 2024-04-12T20:45:21.066 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:45:21.066+0000 7fae82c00700 1 -- 172.21.15.186:0/4237619884 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7fae7c05f5f0 con 0x7fae7c0fec70 2024-04-12T20:45:21.067 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:45:21.067+0000 7fae73e00700 1 -- 172.21.15.186:0/4237619884 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+739 (secure 0 0 0) 0x7fae6c021030 con 0x7fae7c0fec70 2024-04-12T20:45:21.067 INFO:teuthology.orchestra.run.smithi186.stdout: 2024-04-12T20:45:21.067 INFO:teuthology.orchestra.run.smithi186.stdout:{"epoch":1,"fsid":"1cbe407a-f90d-11ee-bc8d-c7b262605968","modified":"2024-04-12T20:42:56.111400Z","created":"2024-04-12T20:42:56.111400Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"smithi018","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:3300","nonce":0},{"type":"v1","addr":"172.21.15.18:6789","nonce":0}]},"addr":"172.21.15.18:6789/0","public_addr":"172.21.15.18:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-04-12T20:45:21.070 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:45:21.070+0000 7fae82c00700 1 -- 172.21.15.186:0/4237619884 >> [v2:172.21.15.18:6800/1391635591,v1:172.21.15.18:6801/1391635591] conn(0x7fae60036830 msgr2=0x7fae60038ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:45:21.070 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:45:21.070+0000 7fae82c00700 1 --2- 172.21.15.186:0/4237619884 >> [v2:172.21.15.18:6800/1391635591,v1:172.21.15.18:6801/1391635591] conn(0x7fae60036830 0x7fae60038ce0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fae68009990 tx=0x7fae68006e40 comp rx=0 tx=0).stop 2024-04-12T20:45:21.071 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:45:21.070+0000 7fae82c00700 1 -- 172.21.15.186:0/4237619884 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fae7c0fec70 msgr2=0x7fae7c10f2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:45:21.071 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:45:21.070+0000 7fae82c00700 1 --2- 172.21.15.186:0/4237619884 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fae7c0fec70 0x7fae7c10f2f0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fae6c009680 tx=0x7fae6c009e30 comp rx=0 tx=0).stop 2024-04-12T20:45:21.071 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:45:21.070+0000 7fae82c00700 1 -- 172.21.15.186:0/4237619884 shutdown_connections 2024-04-12T20:45:21.071 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:45:21.070+0000 7fae82c00700 1 --2- 172.21.15.186:0/4237619884 >> [v2:172.21.15.18:6800/1391635591,v1:172.21.15.18:6801/1391635591] conn(0x7fae60036830 0x7fae60038ce0 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:45:21.071 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:45:21.070+0000 7fae82c00700 1 --2- 172.21.15.186:0/4237619884 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fae7c0fec70 0x7fae7c10f2f0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:45:21.071 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:45:21.070+0000 7fae82c00700 1 -- 172.21.15.186:0/4237619884 >> 172.21.15.186:0/4237619884 conn(0x7fae7c0fa5d0 msgr2=0x7fae7c0fc440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T20:45:21.071 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:45:21.071+0000 7fae82c00700 1 -- 172.21.15.186:0/4237619884 shutdown_connections 2024-04-12T20:45:21.071 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:45:21.071+0000 7fae82c00700 1 -- 172.21.15.186:0/4237619884 wait complete. 2024-04-12T20:45:21.072 INFO:teuthology.orchestra.run.smithi186.stderr:dumped monmap epoch 1 2024-04-12T20:45:21.603 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:45:21 smithi018 ceph-mon[28799]: from='client.? 172.21.15.186:0/4237619884' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-12T20:45:22.603 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:45:22 smithi018 ceph-mon[28799]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-12T20:45:22.728 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-04-12T20:45:22.728 DEBUG:teuthology.orchestra.run.smithi186:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1cbe407a-f90d-11ee-bc8d-c7b262605968 -- ceph mon dump -f json 2024-04-12T20:45:23.163 INFO:teuthology.orchestra.run.smithi186.stderr:Inferring config /etc/ceph/ceph.conf 2024-04-12T20:45:23.262 INFO:teuthology.orchestra.run.smithi186.stderr:Inferring config /etc/ceph/ceph.conf 2024-04-12T20:45:24.603 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:45:24 smithi018 ceph-mon[28799]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-12T20:45:25.196 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:45:25.195+0000 7fa8c9e00700 1 -- 172.21.15.186:0/1939052667 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa8c4072820 msgr2=0x7fa8c4072c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:45:25.196 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:45:25.195+0000 7fa8c9e00700 1 --2- 172.21.15.186:0/1939052667 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa8c4072820 0x7fa8c4072c30 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fa8b80048c0 tx=0x7fa8b8009e30 comp rx=0 tx=0).stop 2024-04-12T20:45:25.196 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:45:25.195+0000 7fa8c9e00700 1 -- 172.21.15.186:0/1939052667 shutdown_connections 2024-04-12T20:45:25.196 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:45:25.195+0000 7fa8c9e00700 1 --2- 172.21.15.186:0/1939052667 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa8c4072820 0x7fa8c4072c30 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:45:25.196 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:45:25.195+0000 7fa8c9e00700 1 -- 172.21.15.186:0/1939052667 >> 172.21.15.186:0/1939052667 conn(0x7fa8c4067610 msgr2=0x7fa8c40fc630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T20:45:25.196 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:45:25.195+0000 7fa8c9e00700 1 -- 172.21.15.186:0/1939052667 shutdown_connections 2024-04-12T20:45:25.196 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:45:25.195+0000 7fa8c9e00700 1 -- 172.21.15.186:0/1939052667 wait complete. 2024-04-12T20:45:25.196 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:45:25.196+0000 7fa8c9e00700 1 Processor -- start 2024-04-12T20:45:25.197 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:45:25.196+0000 7fa8c9e00700 1 -- start start 2024-04-12T20:45:25.197 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:45:25.197+0000 7fa8c9e00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa8c4072820 0x7fa8c4113550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T20:45:25.197 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:45:25.197+0000 7fa8c9e00700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8c4113a90 con 0x7fa8c4072820 2024-04-12T20:45:25.197 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:45:25.197+0000 7fa8c8a00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa8c4072820 0x7fa8c4113550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T20:45:25.197 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:45:25.197+0000 7fa8c8a00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa8c4072820 0x7fa8c4113550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.186:46836/0 (socket says 172.21.15.186:46836) 2024-04-12T20:45:25.197 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:45:25.197+0000 7fa8c8a00700 1 -- 172.21.15.186:0/3441944583 learned_addr learned my addr 172.21.15.186:0/3441944583 (peer_addr_for_me v2:172.21.15.186:0/0) 2024-04-12T20:45:25.198 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:45:25.198+0000 7fa8c8a00700 1 -- 172.21.15.186:0/3441944583 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8b8005040 con 0x7fa8c4072820 2024-04-12T20:45:25.198 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:45:25.199+0000 7fa8c8a00700 1 --2- 172.21.15.186:0/3441944583 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa8c4072820 0x7fa8c4113550 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fa8b8009680 tx=0x7fa8b8009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T20:45:25.199 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:45:25.199+0000 7fa8c0c00700 1 -- 172.21.15.186:0/3441944583 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fa8b800bed0 con 0x7fa8c4072820 2024-04-12T20:45:25.199 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:45:25.199+0000 7fa8c0c00700 1 -- 172.21.15.186:0/3441944583 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa8b800b5c0 con 0x7fa8c4072820 2024-04-12T20:45:25.199 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:45:25.199+0000 7fa8c9e00700 1 -- 172.21.15.186:0/3441944583 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa8c4113c90 con 0x7fa8c4072820 2024-04-12T20:45:25.199 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:45:25.199+0000 7fa8c0c00700 1 -- 172.21.15.186:0/3441944583 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fa8b801c5b0 con 0x7fa8c4072820 2024-04-12T20:45:25.199 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:45:25.199+0000 7fa8c9e00700 1 -- 172.21.15.186:0/3441944583 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa8c4114800 con 0x7fa8c4072820 2024-04-12T20:45:25.200 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:45:25.200+0000 7fa8c9e00700 1 -- 172.21.15.186:0/3441944583 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa8c41093a0 con 0x7fa8c4072820 2024-04-12T20:45:25.200 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:45:25.201+0000 7fa8c0c00700 1 -- 172.21.15.186:0/3441944583 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 13) v1 ==== 43424+0+0 (secure 0 0 0) 0x7fa8b8004d30 con 0x7fa8c4072820 2024-04-12T20:45:25.201 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:45:25.201+0000 7fa8c0c00700 1 --2- 172.21.15.186:0/3441944583 >> [v2:172.21.15.18:6800/1391635591,v1:172.21.15.18:6801/1391635591] conn(0x7fa8ac0368d0 0x7fa8ac038d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T20:45:25.201 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:45:25.201+0000 7fa8c0c00700 1 -- 172.21.15.186:0/3441944583 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1151+0+0 (secure 0 0 0) 0x7fa8b8049900 con 0x7fa8c4072820 2024-04-12T20:45:25.202 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:45:25.202+0000 7fa8c3e00700 1 --2- 172.21.15.186:0/3441944583 >> [v2:172.21.15.18:6800/1391635591,v1:172.21.15.18:6801/1391635591] conn(0x7fa8ac0368d0 0x7fa8ac038d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T20:45:25.204 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:45:25.204+0000 7fa8c3e00700 1 --2- 172.21.15.186:0/3441944583 >> [v2:172.21.15.18:6800/1391635591,v1:172.21.15.18:6801/1391635591] conn(0x7fa8ac0368d0 0x7fa8ac038d80 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fa8b0009990 tx=0x7fa8b0006e40 comp rx=0 tx=0).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T20:45:25.218 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:45:25.219+0000 7fa8c0c00700 1 -- 172.21.15.186:0/3441944583 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7fa8b8011550 con 0x7fa8c4072820 2024-04-12T20:45:25.506 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:45:25.506+0000 7fa8c9e00700 1 -- 172.21.15.186:0/3441944583 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7fa8c4115210 con 0x7fa8c4072820 2024-04-12T20:45:25.507 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:45:25.507+0000 7fa8c0c00700 1 -- 172.21.15.186:0/3441944583 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+739 (secure 0 0 0) 0x7fa8b8011370 con 0x7fa8c4072820 2024-04-12T20:45:25.508 INFO:teuthology.orchestra.run.smithi186.stdout: 2024-04-12T20:45:25.508 INFO:teuthology.orchestra.run.smithi186.stdout:{"epoch":1,"fsid":"1cbe407a-f90d-11ee-bc8d-c7b262605968","modified":"2024-04-12T20:42:56.111400Z","created":"2024-04-12T20:42:56.111400Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"smithi018","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:3300","nonce":0},{"type":"v1","addr":"172.21.15.18:6789","nonce":0}]},"addr":"172.21.15.18:6789/0","public_addr":"172.21.15.18:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-04-12T20:45:25.510 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:45:25.511+0000 7fa8c9e00700 1 -- 172.21.15.186:0/3441944583 >> [v2:172.21.15.18:6800/1391635591,v1:172.21.15.18:6801/1391635591] conn(0x7fa8ac0368d0 msgr2=0x7fa8ac038d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:45:25.510 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:45:25.511+0000 7fa8c9e00700 1 --2- 172.21.15.186:0/3441944583 >> [v2:172.21.15.18:6800/1391635591,v1:172.21.15.18:6801/1391635591] conn(0x7fa8ac0368d0 0x7fa8ac038d80 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fa8b0009990 tx=0x7fa8b0006e40 comp rx=0 tx=0).stop 2024-04-12T20:45:25.510 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:45:25.511+0000 7fa8c9e00700 1 -- 172.21.15.186:0/3441944583 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa8c4072820 msgr2=0x7fa8c4113550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:45:25.510 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:45:25.511+0000 7fa8c9e00700 1 --2- 172.21.15.186:0/3441944583 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa8c4072820 0x7fa8c4113550 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fa8b8009680 tx=0x7fa8b8009e30 comp rx=0 tx=0).stop 2024-04-12T20:45:25.510 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:45:25.511+0000 7fa8c9e00700 1 -- 172.21.15.186:0/3441944583 shutdown_connections 2024-04-12T20:45:25.510 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:45:25.511+0000 7fa8c9e00700 1 --2- 172.21.15.186:0/3441944583 >> [v2:172.21.15.18:6800/1391635591,v1:172.21.15.18:6801/1391635591] conn(0x7fa8ac0368d0 0x7fa8ac038d80 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:45:25.510 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:45:25.511+0000 7fa8c9e00700 1 --2- 172.21.15.186:0/3441944583 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa8c4072820 0x7fa8c4113550 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:45:25.511 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:45:25.511+0000 7fa8c9e00700 1 -- 172.21.15.186:0/3441944583 >> 172.21.15.186:0/3441944583 conn(0x7fa8c4067610 msgr2=0x7fa8c4109fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T20:45:25.511 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:45:25.511+0000 7fa8c9e00700 1 -- 172.21.15.186:0/3441944583 shutdown_connections 2024-04-12T20:45:25.511 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:45:25.511+0000 7fa8c9e00700 1 -- 172.21.15.186:0/3441944583 wait complete. 2024-04-12T20:45:25.511 INFO:teuthology.orchestra.run.smithi186.stderr:dumped monmap epoch 1 2024-04-12T20:45:26.604 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:45:26 smithi018 ceph-mon[28799]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-12T20:45:26.604 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:45:26 smithi018 ceph-mon[28799]: from='client.? 172.21.15.186:0/3441944583' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-12T20:45:27.227 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-04-12T20:45:27.227 DEBUG:teuthology.orchestra.run.smithi186:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1cbe407a-f90d-11ee-bc8d-c7b262605968 -- ceph mon dump -f json 2024-04-12T20:45:27.687 INFO:teuthology.orchestra.run.smithi186.stderr:Inferring config /etc/ceph/ceph.conf 2024-04-12T20:45:27.790 INFO:teuthology.orchestra.run.smithi186.stderr:Inferring config /etc/ceph/ceph.conf 2024-04-12T20:45:28.603 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:45:28 smithi018 ceph-mon[28799]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-12T20:45:29.611 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:45:29.612+0000 7fb91fe00700 1 -- 172.21.15.186:0/3442764345 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb9200feae0 msgr2=0x7fb9200feef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:45:29.612 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:45:29.612+0000 7fb91fe00700 1 --2- 172.21.15.186:0/3442764345 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb9200feae0 0x7fb9200feef0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fb9100048c0 tx=0x7fb910009e30 comp rx=0 tx=0).stop 2024-04-12T20:45:29.612 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:45:29.612+0000 7fb91fe00700 1 -- 172.21.15.186:0/3442764345 shutdown_connections 2024-04-12T20:45:29.612 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:45:29.612+0000 7fb91fe00700 1 --2- 172.21.15.186:0/3442764345 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb9200feae0 0x7fb9200feef0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:45:29.612 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:45:29.612+0000 7fb91fe00700 1 -- 172.21.15.186:0/3442764345 >> 172.21.15.186:0/3442764345 conn(0x7fb9200fa5d0 msgr2=0x7fb9200fc9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T20:45:29.612 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:45:29.612+0000 7fb91fe00700 1 -- 172.21.15.186:0/3442764345 shutdown_connections 2024-04-12T20:45:29.612 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:45:29.612+0000 7fb91fe00700 1 -- 172.21.15.186:0/3442764345 wait complete. 2024-04-12T20:45:29.613 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:45:29.613+0000 7fb91fe00700 1 Processor -- start 2024-04-12T20:45:29.613 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:45:29.613+0000 7fb91fe00700 1 -- start start 2024-04-12T20:45:29.613 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:45:29.614+0000 7fb91fe00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb9200feae0 0x7fb92010f0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T20:45:29.613 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:45:29.614+0000 7fb91fe00700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb92010f630 con 0x7fb9200feae0 2024-04-12T20:45:29.613 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:45:29.615+0000 7fb91ea00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb9200feae0 0x7fb92010f0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T20:45:29.613 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:45:29.615+0000 7fb91ea00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb9200feae0 0x7fb92010f0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.186:46856/0 (socket says 172.21.15.186:46856) 2024-04-12T20:45:29.614 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:45:29.615+0000 7fb91ea00700 1 -- 172.21.15.186:0/3625005961 learned_addr learned my addr 172.21.15.186:0/3625005961 (peer_addr_for_me v2:172.21.15.186:0/0) 2024-04-12T20:45:29.614 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:45:29.616+0000 7fb91ea00700 1 -- 172.21.15.186:0/3625005961 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb910005040 con 0x7fb9200feae0 2024-04-12T20:45:29.615 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:45:29.616+0000 7fb91ea00700 1 --2- 172.21.15.186:0/3625005961 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb9200feae0 0x7fb92010f0f0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fb9100096b0 tx=0x7fb910009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T20:45:29.615 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:45:29.617+0000 7fb916a00700 1 -- 172.21.15.186:0/3625005961 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fb91000bed0 con 0x7fb9200feae0 2024-04-12T20:45:29.616 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:45:29.617+0000 7fb91fe00700 1 -- 172.21.15.186:0/3625005961 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb92010f830 con 0x7fb9200feae0 2024-04-12T20:45:29.616 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:45:29.617+0000 7fb916a00700 1 -- 172.21.15.186:0/3625005961 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb91000b5c0 con 0x7fb9200feae0 2024-04-12T20:45:29.616 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:45:29.617+0000 7fb916a00700 1 -- 172.21.15.186:0/3625005961 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fb91001c5b0 con 0x7fb9200feae0 2024-04-12T20:45:29.616 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:45:29.617+0000 7fb91fe00700 1 -- 172.21.15.186:0/3625005961 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb9201103a0 con 0x7fb9200feae0 2024-04-12T20:45:29.617 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:45:29.618+0000 7fb91fe00700 1 -- 172.21.15.186:0/3625005961 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb920104ff0 con 0x7fb9200feae0 2024-04-12T20:45:29.618 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:45:29.619+0000 7fb916a00700 1 -- 172.21.15.186:0/3625005961 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 13) v1 ==== 43424+0+0 (secure 0 0 0) 0x7fb910004d30 con 0x7fb9200feae0 2024-04-12T20:45:29.618 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:45:29.620+0000 7fb916a00700 1 --2- 172.21.15.186:0/3625005961 >> [v2:172.21.15.18:6800/1391635591,v1:172.21.15.18:6801/1391635591] conn(0x7fb9040368d0 0x7fb904038d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T20:45:29.618 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:45:29.620+0000 7fb916a00700 1 -- 172.21.15.186:0/3625005961 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1151+0+0 (secure 0 0 0) 0x7fb91004a5b0 con 0x7fb9200feae0 2024-04-12T20:45:29.619 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:45:29.620+0000 7fb91e000700 1 --2- 172.21.15.186:0/3625005961 >> [v2:172.21.15.18:6800/1391635591,v1:172.21.15.18:6801/1391635591] conn(0x7fb9040368d0 0x7fb904038d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T20:45:29.620 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:45:29.621+0000 7fb91e000700 1 --2- 172.21.15.186:0/3625005961 >> [v2:172.21.15.18:6800/1391635591,v1:172.21.15.18:6801/1391635591] conn(0x7fb9040368d0 0x7fb904038d80 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fb90c009990 tx=0x7fb90c006e40 comp rx=0 tx=0).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T20:45:29.634 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:45:29.635+0000 7fb916a00700 1 -- 172.21.15.186:0/3625005961 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7fb91001b980 con 0x7fb9200feae0 2024-04-12T20:45:29.927 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:45:29.928+0000 7fb91fe00700 1 -- 172.21.15.186:0/3625005961 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7fb920110e70 con 0x7fb9200feae0 2024-04-12T20:45:29.928 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:45:29.929+0000 7fb916a00700 1 -- 172.21.15.186:0/3625005961 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+739 (secure 0 0 0) 0x7fb91001b390 con 0x7fb9200feae0 2024-04-12T20:45:29.928 INFO:teuthology.orchestra.run.smithi186.stdout: 2024-04-12T20:45:29.928 INFO:teuthology.orchestra.run.smithi186.stdout:{"epoch":1,"fsid":"1cbe407a-f90d-11ee-bc8d-c7b262605968","modified":"2024-04-12T20:42:56.111400Z","created":"2024-04-12T20:42:56.111400Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"smithi018","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:3300","nonce":0},{"type":"v1","addr":"172.21.15.18:6789","nonce":0}]},"addr":"172.21.15.18:6789/0","public_addr":"172.21.15.18:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-04-12T20:45:29.931 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:45:29.932+0000 7fb91fe00700 1 -- 172.21.15.186:0/3625005961 >> [v2:172.21.15.18:6800/1391635591,v1:172.21.15.18:6801/1391635591] conn(0x7fb9040368d0 msgr2=0x7fb904038d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:45:29.931 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:45:29.932+0000 7fb91fe00700 1 --2- 172.21.15.186:0/3625005961 >> [v2:172.21.15.18:6800/1391635591,v1:172.21.15.18:6801/1391635591] conn(0x7fb9040368d0 0x7fb904038d80 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fb90c009990 tx=0x7fb90c006e40 comp rx=0 tx=0).stop 2024-04-12T20:45:29.931 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:45:29.932+0000 7fb91fe00700 1 -- 172.21.15.186:0/3625005961 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb9200feae0 msgr2=0x7fb92010f0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:45:29.931 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:45:29.932+0000 7fb91fe00700 1 --2- 172.21.15.186:0/3625005961 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb9200feae0 0x7fb92010f0f0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fb9100096b0 tx=0x7fb910009e30 comp rx=0 tx=0).stop 2024-04-12T20:45:29.931 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:45:29.932+0000 7fb91fe00700 1 -- 172.21.15.186:0/3625005961 shutdown_connections 2024-04-12T20:45:29.931 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:45:29.932+0000 7fb91fe00700 1 --2- 172.21.15.186:0/3625005961 >> [v2:172.21.15.18:6800/1391635591,v1:172.21.15.18:6801/1391635591] conn(0x7fb9040368d0 0x7fb904038d80 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:45:29.931 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:45:29.932+0000 7fb91fe00700 1 --2- 172.21.15.186:0/3625005961 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb9200feae0 0x7fb92010f0f0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:45:29.931 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:45:29.932+0000 7fb91fe00700 1 -- 172.21.15.186:0/3625005961 >> 172.21.15.186:0/3625005961 conn(0x7fb9200fa5d0 msgr2=0x7fb9200fc400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T20:45:29.931 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:45:29.932+0000 7fb91fe00700 1 -- 172.21.15.186:0/3625005961 shutdown_connections 2024-04-12T20:45:29.931 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:45:29.932+0000 7fb91fe00700 1 -- 172.21.15.186:0/3625005961 wait complete. 2024-04-12T20:45:29.932 INFO:teuthology.orchestra.run.smithi186.stderr:dumped monmap epoch 1 2024-04-12T20:45:30.589 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:45:30 smithi018 ceph-mon[28799]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-12T20:45:30.589 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:45:30 smithi018 ceph-mon[28799]: from='client.? 172.21.15.186:0/3625005961' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-12T20:45:31.654 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-04-12T20:45:31.654 DEBUG:teuthology.orchestra.run.smithi186:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1cbe407a-f90d-11ee-bc8d-c7b262605968 -- ceph mon dump -f json 2024-04-12T20:45:32.108 INFO:teuthology.orchestra.run.smithi186.stderr:Inferring config /etc/ceph/ceph.conf 2024-04-12T20:45:32.208 INFO:teuthology.orchestra.run.smithi186.stderr:Inferring config /etc/ceph/ceph.conf 2024-04-12T20:45:32.603 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:45:32 smithi018 ceph-mon[28799]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-12T20:45:34.469 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:45:34.469+0000 7f5c49a00700 1 -- 172.21.15.186:0/1043369031 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5c441030c0 msgr2=0x7f5c441034d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:45:34.469 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:45:34.469+0000 7f5c49a00700 1 --2- 172.21.15.186:0/1043369031 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5c441030c0 0x7f5c441034d0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f5c340048c0 tx=0x7f5c34009e30 comp rx=0 tx=0).stop 2024-04-12T20:45:34.469 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:45:34.470+0000 7f5c49a00700 1 -- 172.21.15.186:0/1043369031 shutdown_connections 2024-04-12T20:45:34.469 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:45:34.470+0000 7f5c49a00700 1 --2- 172.21.15.186:0/1043369031 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5c441030c0 0x7f5c441034d0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:45:34.469 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:45:34.470+0000 7f5c49a00700 1 -- 172.21.15.186:0/1043369031 >> 172.21.15.186:0/1043369031 conn(0x7f5c44072300 msgr2=0x7f5c44074710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T20:45:34.469 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:45:34.470+0000 7f5c49a00700 1 -- 172.21.15.186:0/1043369031 shutdown_connections 2024-04-12T20:45:34.469 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:45:34.470+0000 7f5c49a00700 1 -- 172.21.15.186:0/1043369031 wait complete. 2024-04-12T20:45:34.470 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:45:34.471+0000 7f5c49a00700 1 Processor -- start 2024-04-12T20:45:34.470 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:45:34.471+0000 7f5c49a00700 1 -- start start 2024-04-12T20:45:34.470 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:45:34.471+0000 7f5c49a00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5c441030c0 0x7f5c441136f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T20:45:34.470 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:45:34.471+0000 7f5c49a00700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c44113c30 con 0x7f5c441030c0 2024-04-12T20:45:34.471 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:45:34.472+0000 7f5c42a00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5c441030c0 0x7f5c441136f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T20:45:34.471 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:45:34.472+0000 7f5c42a00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5c441030c0 0x7f5c441136f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.186:46872/0 (socket says 172.21.15.186:46872) 2024-04-12T20:45:34.471 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:45:34.472+0000 7f5c42a00700 1 -- 172.21.15.186:0/699787554 learned_addr learned my addr 172.21.15.186:0/699787554 (peer_addr_for_me v2:172.21.15.186:0/0) 2024-04-12T20:45:34.471 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:45:34.473+0000 7f5c42a00700 1 -- 172.21.15.186:0/699787554 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c34005040 con 0x7f5c441030c0 2024-04-12T20:45:34.472 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:45:34.473+0000 7f5c42a00700 1 --2- 172.21.15.186:0/699787554 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5c441030c0 0x7f5c441136f0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f5c34009680 tx=0x7f5c34009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T20:45:34.472 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:45:34.474+0000 7f5c3aa00700 1 -- 172.21.15.186:0/699787554 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f5c3400bed0 con 0x7f5c441030c0 2024-04-12T20:45:34.472 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:45:34.474+0000 7f5c49a00700 1 -- 172.21.15.186:0/699787554 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5c44113e30 con 0x7f5c441030c0 2024-04-12T20:45:34.472 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:45:34.474+0000 7f5c3aa00700 1 -- 172.21.15.186:0/699787554 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f5c3400b5c0 con 0x7f5c441030c0 2024-04-12T20:45:34.473 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:45:34.474+0000 7f5c3aa00700 1 -- 172.21.15.186:0/699787554 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f5c3401c5b0 con 0x7f5c441030c0 2024-04-12T20:45:34.473 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:45:34.474+0000 7f5c49a00700 1 -- 172.21.15.186:0/699787554 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5c441149a0 con 0x7f5c441030c0 2024-04-12T20:45:34.473 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:45:34.475+0000 7f5c49a00700 1 -- 172.21.15.186:0/699787554 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5c44109560 con 0x7f5c441030c0 2024-04-12T20:45:34.474 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:45:34.475+0000 7f5c3aa00700 1 -- 172.21.15.186:0/699787554 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 13) v1 ==== 43424+0+0 (secure 0 0 0) 0x7f5c34004d30 con 0x7f5c441030c0 2024-04-12T20:45:34.474 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:45:34.476+0000 7f5c3aa00700 1 --2- 172.21.15.186:0/699787554 >> [v2:172.21.15.18:6800/1391635591,v1:172.21.15.18:6801/1391635591] conn(0x7f5c28036880 0x7f5c28038d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T20:45:34.474 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:45:34.476+0000 7f5c3aa00700 1 -- 172.21.15.186:0/699787554 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1151+0+0 (secure 0 0 0) 0x7f5c34049930 con 0x7f5c441030c0 2024-04-12T20:45:34.475 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:45:34.476+0000 7f5c42000700 1 --2- 172.21.15.186:0/699787554 >> [v2:172.21.15.18:6800/1391635591,v1:172.21.15.18:6801/1391635591] conn(0x7f5c28036880 0x7f5c28038d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T20:45:34.477 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:45:34.478+0000 7f5c42000700 1 --2- 172.21.15.186:0/699787554 >> [v2:172.21.15.18:6800/1391635591,v1:172.21.15.18:6801/1391635591] conn(0x7f5c28036880 0x7f5c28038d30 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f5c30009990 tx=0x7f5c30006e40 comp rx=0 tx=0).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T20:45:34.491 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:45:34.492+0000 7f5c3aa00700 1 -- 172.21.15.186:0/699787554 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7f5c34011550 con 0x7f5c441030c0 2024-04-12T20:45:34.556 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:45:34 smithi018 ceph-mon[28799]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-12T20:45:34.781 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:45:34.782+0000 7f5c49a00700 1 -- 172.21.15.186:0/699787554 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f5c441153b0 con 0x7f5c441030c0 2024-04-12T20:45:34.782 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:45:34.783+0000 7f5c3aa00700 1 -- 172.21.15.186:0/699787554 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+739 (secure 0 0 0) 0x7f5c34011370 con 0x7f5c441030c0 2024-04-12T20:45:34.783 INFO:teuthology.orchestra.run.smithi186.stdout: 2024-04-12T20:45:34.783 INFO:teuthology.orchestra.run.smithi186.stdout:{"epoch":1,"fsid":"1cbe407a-f90d-11ee-bc8d-c7b262605968","modified":"2024-04-12T20:42:56.111400Z","created":"2024-04-12T20:42:56.111400Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"smithi018","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:3300","nonce":0},{"type":"v1","addr":"172.21.15.18:6789","nonce":0}]},"addr":"172.21.15.18:6789/0","public_addr":"172.21.15.18:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-04-12T20:45:34.785 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:45:34.786+0000 7f5c49a00700 1 -- 172.21.15.186:0/699787554 >> [v2:172.21.15.18:6800/1391635591,v1:172.21.15.18:6801/1391635591] conn(0x7f5c28036880 msgr2=0x7f5c28038d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:45:34.785 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:45:34.786+0000 7f5c49a00700 1 --2- 172.21.15.186:0/699787554 >> [v2:172.21.15.18:6800/1391635591,v1:172.21.15.18:6801/1391635591] conn(0x7f5c28036880 0x7f5c28038d30 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f5c30009990 tx=0x7f5c30006e40 comp rx=0 tx=0).stop 2024-04-12T20:45:34.785 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:45:34.786+0000 7f5c49a00700 1 -- 172.21.15.186:0/699787554 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5c441030c0 msgr2=0x7f5c441136f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:45:34.785 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:45:34.786+0000 7f5c49a00700 1 --2- 172.21.15.186:0/699787554 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5c441030c0 0x7f5c441136f0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f5c34009680 tx=0x7f5c34009e30 comp rx=0 tx=0).stop 2024-04-12T20:45:34.785 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:45:34.786+0000 7f5c49a00700 1 -- 172.21.15.186:0/699787554 shutdown_connections 2024-04-12T20:45:34.785 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:45:34.786+0000 7f5c49a00700 1 --2- 172.21.15.186:0/699787554 >> [v2:172.21.15.18:6800/1391635591,v1:172.21.15.18:6801/1391635591] conn(0x7f5c28036880 0x7f5c28038d30 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:45:34.785 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:45:34.786+0000 7f5c49a00700 1 --2- 172.21.15.186:0/699787554 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5c441030c0 0x7f5c441136f0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:45:34.786 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:45:34.786+0000 7f5c49a00700 1 -- 172.21.15.186:0/699787554 >> 172.21.15.186:0/699787554 conn(0x7f5c44072300 msgr2=0x7f5c440741c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T20:45:34.786 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:45:34.786+0000 7f5c49a00700 1 -- 172.21.15.186:0/699787554 shutdown_connections 2024-04-12T20:45:34.786 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:45:34.786+0000 7f5c49a00700 1 -- 172.21.15.186:0/699787554 wait complete. 2024-04-12T20:45:34.786 INFO:teuthology.orchestra.run.smithi186.stderr:dumped monmap epoch 1 2024-04-12T20:45:35.603 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:45:35 smithi018 ceph-mon[28799]: from='client.? 172.21.15.186:0/699787554' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-12T20:45:35.603 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:45:35 smithi018 ceph-mon[28799]: from='mgr.14162 172.21.15.18:0/2711576919' entity='mgr.smithi018.xjzwga' 2024-04-12T20:45:35.603 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:45:35 smithi018 ceph-mon[28799]: from='mgr.14162 172.21.15.18:0/2711576919' entity='mgr.smithi018.xjzwga' 2024-04-12T20:45:35.603 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:45:35 smithi018 ceph-mon[28799]: from='mgr.14162 172.21.15.18:0/2711576919' entity='mgr.smithi018.xjzwga' 2024-04-12T20:45:35.604 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:45:35 smithi018 ceph-mon[28799]: from='mgr.14162 172.21.15.18:0/2711576919' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "mgr module enable", "module": "prometheus"}]: dispatch 2024-04-12T20:45:35.604 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:45:35 smithi018 ceph-mon[28799]: from='mgr.14162 172.21.15.18:0/2711576919' entity='mgr.smithi018.xjzwga' 2024-04-12T20:45:36.603 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:45:36 smithi018 ceph-mon[28799]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-12T20:45:36.603 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:45:36 smithi018 ceph-mon[28799]: from='mgr.14162 172.21.15.18:0/2711576919' entity='mgr.smithi018.xjzwga' cmd='[{"prefix": "mgr module enable", "module": "prometheus"}]': finished 2024-04-12T20:45:36.603 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:45:36 smithi018 ceph-mon[28799]: mgrmap e14: smithi018.xjzwga(active, since 80s) 2024-04-12T20:45:36.774 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-04-12T20:45:36.774 DEBUG:teuthology.orchestra.run.smithi186:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1cbe407a-f90d-11ee-bc8d-c7b262605968 -- ceph mon dump -f json 2024-04-12T20:45:37.231 INFO:teuthology.orchestra.run.smithi186.stderr:Inferring config /etc/ceph/ceph.conf 2024-04-12T20:45:37.335 INFO:teuthology.orchestra.run.smithi186.stderr:Inferring config /etc/ceph/ceph.conf 2024-04-12T20:45:39.160 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:45:39.159+0000 7f5c35000700 1 -- 172.21.15.186:0/3869869888 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5c300fca60 msgr2=0x7f5c300fce70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:45:39.160 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:45:39.159+0000 7f5c35000700 1 --2- 172.21.15.186:0/3869869888 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5c300fca60 0x7f5c300fce70 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f5c200048c0 tx=0x7f5c20009e30 comp rx=0 tx=0).stop 2024-04-12T20:45:39.160 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:45:39.159+0000 7f5c35000700 1 -- 172.21.15.186:0/3869869888 shutdown_connections 2024-04-12T20:45:39.160 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:45:39.159+0000 7f5c35000700 1 --2- 172.21.15.186:0/3869869888 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5c300fca60 0x7f5c300fce70 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:45:39.160 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:45:39.159+0000 7f5c35000700 1 -- 172.21.15.186:0/3869869888 >> 172.21.15.186:0/3869869888 conn(0x7f5c300f81c0 msgr2=0x7f5c300fa5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T20:45:39.160 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:45:39.159+0000 7f5c35000700 1 -- 172.21.15.186:0/3869869888 shutdown_connections 2024-04-12T20:45:39.161 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:45:39.160+0000 7f5c35000700 1 -- 172.21.15.186:0/3869869888 wait complete. 2024-04-12T20:45:39.161 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:45:39.161+0000 7f5c35000700 1 Processor -- start 2024-04-12T20:45:39.161 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:45:39.161+0000 7f5c35000700 1 -- start start 2024-04-12T20:45:39.161 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:45:39.161+0000 7f5c35000700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5c300fca60 0x7f5c30071d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T20:45:39.161 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:45:39.161+0000 7f5c35000700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c300722a0 con 0x7f5c300fca60 2024-04-12T20:45:39.162 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:45:39.162+0000 7f5c2de00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5c300fca60 0x7f5c30071d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T20:45:39.162 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:45:39.162+0000 7f5c2de00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5c300fca60 0x7f5c30071d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.186:36332/0 (socket says 172.21.15.186:36332) 2024-04-12T20:45:39.162 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:45:39.162+0000 7f5c2de00700 1 -- 172.21.15.186:0/3813928811 learned_addr learned my addr 172.21.15.186:0/3813928811 (peer_addr_for_me v2:172.21.15.186:0/0) 2024-04-12T20:45:39.163 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:45:39.162+0000 7f5c2de00700 1 -- 172.21.15.186:0/3813928811 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c20005040 con 0x7f5c300fca60 2024-04-12T20:45:39.163 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:45:39.163+0000 7f5c2de00700 1 --2- 172.21.15.186:0/3813928811 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5c300fca60 0x7f5c30071d60 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f5c20009680 tx=0x7f5c20009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T20:45:39.163 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:45:39.163+0000 7f5c26000700 1 -- 172.21.15.186:0/3813928811 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f5c2000b5c0 con 0x7f5c300fca60 2024-04-12T20:45:39.163 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:45:39.163+0000 7f5c35000700 1 -- 172.21.15.186:0/3813928811 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5c30070370 con 0x7f5c300fca60 2024-04-12T20:45:39.163 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:45:39.163+0000 7f5c26000700 1 -- 172.21.15.186:0/3813928811 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f5c2000be70 con 0x7f5c300fca60 2024-04-12T20:45:39.163 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:45:39.163+0000 7f5c26000700 1 -- 172.21.15.186:0/3813928811 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f5c200135a0 con 0x7f5c300fca60 2024-04-12T20:45:39.163 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:45:39.163+0000 7f5c35000700 1 -- 172.21.15.186:0/3813928811 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5c30070860 con 0x7f5c300fca60 2024-04-12T20:45:39.164 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:45:39.164+0000 7f5c35000700 1 -- 172.21.15.186:0/3813928811 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5c3005f5f0 con 0x7f5c300fca60 2024-04-12T20:45:39.164 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:45:39.165+0000 7f5c26000700 1 -- 172.21.15.186:0/3813928811 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 14) v1 ==== 43438+0+0 (secure 0 0 0) 0x7f5c20004d30 con 0x7f5c300fca60 2024-04-12T20:45:39.164 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:45:39.165+0000 7f5c26000700 1 --2- 172.21.15.186:0/3813928811 >> [v2:172.21.15.18:6800/1391635591,v1:172.21.15.18:6801/1391635591] conn(0x7f5c140368d0 0x7f5c14038d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T20:45:39.164 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:45:39.165+0000 7f5c26000700 1 -- 172.21.15.186:0/3813928811 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1151+0+0 (secure 0 0 0) 0x7f5c20049490 con 0x7f5c300fca60 2024-04-12T20:45:39.164 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:45:39.165+0000 7f5c2d400700 1 -- 172.21.15.186:0/3813928811 >> [v2:172.21.15.18:6800/1391635591,v1:172.21.15.18:6801/1391635591] conn(0x7f5c140368d0 msgr2=0x7f5c14038d80 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.18:6800/1391635591 2024-04-12T20:45:39.164 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:45:39.166+0000 7f5c2d400700 1 --2- 172.21.15.186:0/3813928811 >> [v2:172.21.15.18:6800/1391635591,v1:172.21.15.18:6801/1391635591] conn(0x7f5c140368d0 0x7f5c14038d80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-12T20:45:39.178 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:45:39.180+0000 7f5c26000700 1 -- 172.21.15.186:0/3813928811 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7f5c20011960 con 0x7f5c300fca60 2024-04-12T20:45:39.366 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:45:39.367+0000 7f5c2d400700 1 -- 172.21.15.186:0/3813928811 >> [v2:172.21.15.18:6800/1391635591,v1:172.21.15.18:6801/1391635591] conn(0x7f5c140368d0 msgr2=0x7f5c14038d80 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.18:6800/1391635591 2024-04-12T20:45:39.366 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:45:39.367+0000 7f5c2d400700 1 --2- 172.21.15.186:0/3813928811 >> [v2:172.21.15.18:6800/1391635591,v1:172.21.15.18:6801/1391635591] conn(0x7f5c140368d0 0x7f5c14038d80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-12T20:45:39.471 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:45:39.473+0000 7f5c35000700 1 -- 172.21.15.186:0/3813928811 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f5c30106440 con 0x7f5c300fca60 2024-04-12T20:45:39.473 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:45:39.474+0000 7f5c26000700 1 -- 172.21.15.186:0/3813928811 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+739 (secure 0 0 0) 0x7f5c20021070 con 0x7f5c300fca60 2024-04-12T20:45:39.473 INFO:teuthology.orchestra.run.smithi186.stdout: 2024-04-12T20:45:39.473 INFO:teuthology.orchestra.run.smithi186.stdout:{"epoch":1,"fsid":"1cbe407a-f90d-11ee-bc8d-c7b262605968","modified":"2024-04-12T20:42:56.111400Z","created":"2024-04-12T20:42:56.111400Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"smithi018","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:3300","nonce":0},{"type":"v1","addr":"172.21.15.18:6789","nonce":0}]},"addr":"172.21.15.18:6789/0","public_addr":"172.21.15.18:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-04-12T20:45:39.476 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:45:39.477+0000 7f5c35000700 1 -- 172.21.15.186:0/3813928811 >> [v2:172.21.15.18:6800/1391635591,v1:172.21.15.18:6801/1391635591] conn(0x7f5c140368d0 msgr2=0x7f5c14038d80 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-12T20:45:39.476 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:45:39.477+0000 7f5c35000700 1 --2- 172.21.15.186:0/3813928811 >> [v2:172.21.15.18:6800/1391635591,v1:172.21.15.18:6801/1391635591] conn(0x7f5c140368d0 0x7f5c14038d80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:45:39.476 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:45:39.477+0000 7f5c35000700 1 -- 172.21.15.186:0/3813928811 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5c300fca60 msgr2=0x7f5c30071d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:45:39.476 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:45:39.477+0000 7f5c35000700 1 --2- 172.21.15.186:0/3813928811 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5c300fca60 0x7f5c30071d60 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f5c20009680 tx=0x7f5c20009e30 comp rx=0 tx=0).stop 2024-04-12T20:45:39.476 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:45:39.477+0000 7f5c35000700 1 -- 172.21.15.186:0/3813928811 shutdown_connections 2024-04-12T20:45:39.476 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:45:39.477+0000 7f5c35000700 1 --2- 172.21.15.186:0/3813928811 >> [v2:172.21.15.18:6800/1391635591,v1:172.21.15.18:6801/1391635591] conn(0x7f5c140368d0 0x7f5c14038d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:45:39.476 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:45:39.477+0000 7f5c35000700 1 --2- 172.21.15.186:0/3813928811 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5c300fca60 0x7f5c30071d60 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:45:39.476 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:45:39.477+0000 7f5c35000700 1 -- 172.21.15.186:0/3813928811 >> 172.21.15.186:0/3813928811 conn(0x7f5c300f81c0 msgr2=0x7f5c30103ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T20:45:39.476 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:45:39.477+0000 7f5c35000700 1 -- 172.21.15.186:0/3813928811 shutdown_connections 2024-04-12T20:45:39.476 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:45:39.477+0000 7f5c35000700 1 -- 172.21.15.186:0/3813928811 wait complete. 2024-04-12T20:45:39.477 INFO:teuthology.orchestra.run.smithi186.stderr:dumped monmap epoch 1 2024-04-12T20:45:39.853 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:45:39 smithi018 ceph-mon[28799]: from='client.? 172.21.15.186:0/3813928811' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-12T20:45:41.142 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-04-12T20:45:41.142 DEBUG:teuthology.orchestra.run.smithi186:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1cbe407a-f90d-11ee-bc8d-c7b262605968 -- ceph mon dump -f json 2024-04-12T20:45:41.595 INFO:teuthology.orchestra.run.smithi186.stderr:Inferring config /etc/ceph/ceph.conf 2024-04-12T20:45:41.692 INFO:teuthology.orchestra.run.smithi186.stderr:Inferring config /etc/ceph/ceph.conf 2024-04-12T20:45:43.531 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:45:43.532+0000 7fbad3000700 1 -- 172.21.15.186:0/3304769179 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fbacc0665e0 msgr2=0x7fbacc0ff650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:45:43.532 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:45:43.532+0000 7fbad3000700 1 --2- 172.21.15.186:0/3304769179 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fbacc0665e0 0x7fbacc0ff650 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fbabc0048c0 tx=0x7fbabc009e30 comp rx=0 tx=0).stop 2024-04-12T20:45:43.532 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:45:43.532+0000 7fbad3000700 1 -- 172.21.15.186:0/3304769179 shutdown_connections 2024-04-12T20:45:43.532 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:45:43.532+0000 7fbad3000700 1 --2- 172.21.15.186:0/3304769179 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fbacc0665e0 0x7fbacc0ff650 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:45:43.532 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:45:43.532+0000 7fbad3000700 1 -- 172.21.15.186:0/3304769179 >> 172.21.15.186:0/3304769179 conn(0x7fbacc0f70c0 msgr2=0x7fbacc0f94f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T20:45:43.532 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:45:43.532+0000 7fbad3000700 1 -- 172.21.15.186:0/3304769179 shutdown_connections 2024-04-12T20:45:43.532 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:45:43.532+0000 7fbad3000700 1 -- 172.21.15.186:0/3304769179 wait complete. 2024-04-12T20:45:43.533 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:45:43.533+0000 7fbad3000700 1 Processor -- start 2024-04-12T20:45:43.533 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:45:43.533+0000 7fbad3000700 1 -- start start 2024-04-12T20:45:43.533 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:45:43.534+0000 7fbad3000700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fbacc0665e0 0x7fbacc10f2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T20:45:43.533 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:45:43.534+0000 7fbad3000700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbacc10f820 con 0x7fbacc0665e0 2024-04-12T20:45:43.533 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:45:43.534+0000 7fbacbe00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fbacc0665e0 0x7fbacc10f2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T20:45:43.534 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:45:43.534+0000 7fbacbe00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fbacc0665e0 0x7fbacc10f2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.186:36356/0 (socket says 172.21.15.186:36356) 2024-04-12T20:45:43.534 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:45:43.534+0000 7fbacbe00700 1 -- 172.21.15.186:0/405499217 learned_addr learned my addr 172.21.15.186:0/405499217 (peer_addr_for_me v2:172.21.15.186:0/0) 2024-04-12T20:45:43.534 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:45:43.535+0000 7fbacbe00700 1 -- 172.21.15.186:0/405499217 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbabc005040 con 0x7fbacc0665e0 2024-04-12T20:45:43.534 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:45:43.536+0000 7fbacbe00700 1 --2- 172.21.15.186:0/405499217 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fbacc0665e0 0x7fbacc10f2e0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fbabc009680 tx=0x7fbabc009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T20:45:43.535 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:45:43.536+0000 7fbac8c00700 1 -- 172.21.15.186:0/405499217 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fbabc00bed0 con 0x7fbacc0665e0 2024-04-12T20:45:43.535 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:45:43.536+0000 7fbad3000700 1 -- 172.21.15.186:0/405499217 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbacc10fa20 con 0x7fbacc0665e0 2024-04-12T20:45:43.535 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:45:43.536+0000 7fbac8c00700 1 -- 172.21.15.186:0/405499217 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fbabc00b5c0 con 0x7fbacc0665e0 2024-04-12T20:45:43.535 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:45:43.536+0000 7fbac8c00700 1 -- 172.21.15.186:0/405499217 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fbabc01c5b0 con 0x7fbacc0665e0 2024-04-12T20:45:43.535 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:45:43.536+0000 7fbad3000700 1 -- 172.21.15.186:0/405499217 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbacc110590 con 0x7fbacc0665e0 2024-04-12T20:45:43.536 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:45:43.537+0000 7fbad3000700 1 -- 172.21.15.186:0/405499217 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbacc102f60 con 0x7fbacc0665e0 2024-04-12T20:45:43.536 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:45:43.538+0000 7fbac8c00700 1 -- 172.21.15.186:0/405499217 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 14) v1 ==== 43438+0+0 (secure 0 0 0) 0x7fbabc004d30 con 0x7fbacc0665e0 2024-04-12T20:45:43.537 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:45:43.538+0000 7fbac8c00700 1 --2- 172.21.15.186:0/405499217 >> [v2:172.21.15.18:6800/1391635591,v1:172.21.15.18:6801/1391635591] conn(0x7fbab0036960 0x7fbab0038e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T20:45:43.537 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:45:43.538+0000 7fbac8c00700 1 -- 172.21.15.186:0/405499217 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1151+0+0 (secure 0 0 0) 0x7fbabc04a510 con 0x7fbacc0665e0 2024-04-12T20:45:43.537 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:45:43.538+0000 7fbac3e00700 1 -- 172.21.15.186:0/405499217 >> [v2:172.21.15.18:6800/1391635591,v1:172.21.15.18:6801/1391635591] conn(0x7fbab0036960 msgr2=0x7fbab0038e10 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.18:6800/1391635591 2024-04-12T20:45:43.537 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:45:43.539+0000 7fbac3e00700 1 --2- 172.21.15.186:0/405499217 >> [v2:172.21.15.18:6800/1391635591,v1:172.21.15.18:6801/1391635591] conn(0x7fbab0036960 0x7fbab0038e10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-12T20:45:43.551 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:45:43.552+0000 7fbac8c00700 1 -- 172.21.15.186:0/405499217 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7fbabc02d970 con 0x7fbacc0665e0 2024-04-12T20:45:43.739 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:45:43.740+0000 7fbac3e00700 1 -- 172.21.15.186:0/405499217 >> [v2:172.21.15.18:6800/1391635591,v1:172.21.15.18:6801/1391635591] conn(0x7fbab0036960 msgr2=0x7fbab0038e10 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.18:6800/1391635591 2024-04-12T20:45:43.739 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:45:43.740+0000 7fbac3e00700 1 --2- 172.21.15.186:0/405499217 >> [v2:172.21.15.18:6800/1391635591,v1:172.21.15.18:6801/1391635591] conn(0x7fbab0036960 0x7fbab0038e10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-12T20:45:43.837 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:45:43.838+0000 7fbad3000700 1 -- 172.21.15.186:0/405499217 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7fbacc110890 con 0x7fbacc0665e0 2024-04-12T20:45:43.838 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:45:43.839+0000 7fbac8c00700 1 -- 172.21.15.186:0/405499217 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+739 (secure 0 0 0) 0x7fbabc02ecb0 con 0x7fbacc0665e0 2024-04-12T20:45:43.838 INFO:teuthology.orchestra.run.smithi186.stdout: 2024-04-12T20:45:43.839 INFO:teuthology.orchestra.run.smithi186.stdout:{"epoch":1,"fsid":"1cbe407a-f90d-11ee-bc8d-c7b262605968","modified":"2024-04-12T20:42:56.111400Z","created":"2024-04-12T20:42:56.111400Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"smithi018","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:3300","nonce":0},{"type":"v1","addr":"172.21.15.18:6789","nonce":0}]},"addr":"172.21.15.18:6789/0","public_addr":"172.21.15.18:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-04-12T20:45:43.841 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:45:43.842+0000 7fbad3000700 1 -- 172.21.15.186:0/405499217 >> [v2:172.21.15.18:6800/1391635591,v1:172.21.15.18:6801/1391635591] conn(0x7fbab0036960 msgr2=0x7fbab0038e10 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-12T20:45:43.841 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:45:43.842+0000 7fbad3000700 1 --2- 172.21.15.186:0/405499217 >> [v2:172.21.15.18:6800/1391635591,v1:172.21.15.18:6801/1391635591] conn(0x7fbab0036960 0x7fbab0038e10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:45:43.841 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:45:43.842+0000 7fbad3000700 1 -- 172.21.15.186:0/405499217 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fbacc0665e0 msgr2=0x7fbacc10f2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:45:43.841 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:45:43.842+0000 7fbad3000700 1 --2- 172.21.15.186:0/405499217 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fbacc0665e0 0x7fbacc10f2e0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fbabc009680 tx=0x7fbabc009e30 comp rx=0 tx=0).stop 2024-04-12T20:45:43.842 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:45:43.842+0000 7fbad3000700 1 -- 172.21.15.186:0/405499217 shutdown_connections 2024-04-12T20:45:43.842 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:45:43.842+0000 7fbad3000700 1 --2- 172.21.15.186:0/405499217 >> [v2:172.21.15.18:6800/1391635591,v1:172.21.15.18:6801/1391635591] conn(0x7fbab0036960 0x7fbab0038e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:45:43.842 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:45:43.842+0000 7fbad3000700 1 --2- 172.21.15.186:0/405499217 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fbacc0665e0 0x7fbacc10f2e0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:45:43.842 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:45:43.842+0000 7fbad3000700 1 -- 172.21.15.186:0/405499217 >> 172.21.15.186:0/405499217 conn(0x7fbacc0f70c0 msgr2=0x7fbacc0f94f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T20:45:43.842 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:45:43.842+0000 7fbad3000700 1 -- 172.21.15.186:0/405499217 shutdown_connections 2024-04-12T20:45:43.842 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:45:43.842+0000 7fbad3000700 1 -- 172.21.15.186:0/405499217 wait complete. 2024-04-12T20:45:43.842 INFO:teuthology.orchestra.run.smithi186.stderr:dumped monmap epoch 1 2024-04-12T20:45:44.103 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:45:43 smithi018 ceph-mon[28799]: from='client.? 172.21.15.186:0/405499217' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-12T20:45:45.075 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:45:44 smithi018 ceph-mon[28799]: Active manager daemon smithi018.xjzwga restarted 2024-04-12T20:45:45.075 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:45:44 smithi018 ceph-mon[28799]: Activating manager daemon smithi018.xjzwga 2024-04-12T20:45:45.075 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:45:44 smithi018 ceph-mon[28799]: osdmap e5: 0 total, 0 up, 0 in 2024-04-12T20:45:45.075 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:45:44 smithi018 ceph-mon[28799]: mgrmap e15: smithi018.xjzwga(active, starting, since 0.00880627s) 2024-04-12T20:45:45.075 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:45:44 smithi018 ceph-mon[28799]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "mon metadata", "id": "smithi018"}]: dispatch 2024-04-12T20:45:45.075 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:45:44 smithi018 ceph-mon[28799]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "mgr metadata", "who": "smithi018.xjzwga", "id": "smithi018.xjzwga"}]: dispatch 2024-04-12T20:45:45.075 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:45:44 smithi018 ceph-mon[28799]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-12T20:45:45.076 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:45:44 smithi018 ceph-mon[28799]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-12T20:45:45.076 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:45:44 smithi018 ceph-mon[28799]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-12T20:45:45.076 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:45:44 smithi018 ceph-mon[28799]: Manager daemon smithi018.xjzwga is now available 2024-04-12T20:45:45.076 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:45:44 smithi018 ceph-mon[28799]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-12T20:45:45.076 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:45:44 smithi018 ceph-mon[28799]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-12T20:45:45.076 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:45:44 smithi018 ceph-mon[28799]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' 2024-04-12T20:45:45.076 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:45:44 smithi018 ceph-mon[28799]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T20:45:45.076 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:45:44 smithi018 ceph-mon[28799]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi018.xjzwga/mirror_snapshot_schedule"}]: dispatch 2024-04-12T20:45:45.076 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:45:44 smithi018 ceph-mon[28799]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi018.xjzwga/trash_purge_schedule"}]: dispatch 2024-04-12T20:45:45.468 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-04-12T20:45:45.468 DEBUG:teuthology.orchestra.run.smithi186:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1cbe407a-f90d-11ee-bc8d-c7b262605968 -- ceph mon dump -f json 2024-04-12T20:45:45.892 INFO:teuthology.orchestra.run.smithi186.stderr:Inferring config /etc/ceph/ceph.conf 2024-04-12T20:45:45.986 INFO:teuthology.orchestra.run.smithi186.stderr:Inferring config /etc/ceph/ceph.conf 2024-04-12T20:45:46.604 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:45:46 smithi018 ceph-mon[28799]: [12/Apr/2024:20:45:45] ENGINE Bus STARTING 2024-04-12T20:45:46.604 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:45:46 smithi018 ceph-mon[28799]: [12/Apr/2024:20:45:45] ENGINE Serving on https://172.21.15.18:7150 2024-04-12T20:45:46.604 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:45:46 smithi018 ceph-mon[28799]: [12/Apr/2024:20:45:45] ENGINE Bus STARTED 2024-04-12T20:45:46.604 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:45:46 smithi018 ceph-mon[28799]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' 2024-04-12T20:45:46.604 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:45:46 smithi018 ceph-mon[28799]: mgrmap e16: smithi018.xjzwga(active, since 1.00949s) 2024-04-12T20:45:46.604 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:45:46 smithi018 ceph-mon[28799]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' 2024-04-12T20:45:47.803 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:45:47.803+0000 7f9e12e00700 1 -- 172.21.15.186:0/115401994 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9e0c0fca70 msgr2=0x7f9e0c0fce80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:45:47.803 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:45:47.803+0000 7f9e12e00700 1 --2- 172.21.15.186:0/115401994 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9e0c0fca70 0x7f9e0c0fce80 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f9dfc0048c0 tx=0x7f9dfc009e30 comp rx=0 tx=0).stop 2024-04-12T20:45:47.803 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:45:47.803+0000 7f9e12e00700 1 -- 172.21.15.186:0/115401994 shutdown_connections 2024-04-12T20:45:47.803 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:45:47.803+0000 7f9e12e00700 1 --2- 172.21.15.186:0/115401994 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9e0c0fca70 0x7f9e0c0fce80 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:45:47.803 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:45:47.803+0000 7f9e12e00700 1 -- 172.21.15.186:0/115401994 >> 172.21.15.186:0/115401994 conn(0x7f9e0c0f81d0 msgr2=0x7f9e0c0fa600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T20:45:47.803 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:45:47.804+0000 7f9e12e00700 1 -- 172.21.15.186:0/115401994 shutdown_connections 2024-04-12T20:45:47.803 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:45:47.804+0000 7f9e12e00700 1 -- 172.21.15.186:0/115401994 wait complete. 2024-04-12T20:45:47.804 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:45:47.805+0000 7f9e12e00700 1 Processor -- start 2024-04-12T20:45:47.804 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:45:47.805+0000 7f9e12e00700 1 -- start start 2024-04-12T20:45:47.804 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:45:47.805+0000 7f9e12e00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9e0c0fca70 0x7f9e0c10d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T20:45:47.804 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:45:47.805+0000 7f9e12e00700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e0c10d600 con 0x7f9e0c0fca70 2024-04-12T20:45:47.804 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:45:47.806+0000 7f9e0be00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9e0c0fca70 0x7f9e0c10d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T20:45:47.804 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:45:47.806+0000 7f9e0be00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9e0c0fca70 0x7f9e0c10d0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.186:43324/0 (socket says 172.21.15.186:43324) 2024-04-12T20:45:47.805 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:45:47.806+0000 7f9e0be00700 1 -- 172.21.15.186:0/1008675771 learned_addr learned my addr 172.21.15.186:0/1008675771 (peer_addr_for_me v2:172.21.15.186:0/0) 2024-04-12T20:45:47.805 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:45:47.806+0000 7f9e0be00700 1 -- 172.21.15.186:0/1008675771 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9dfc005040 con 0x7f9e0c0fca70 2024-04-12T20:45:47.805 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:45:47.807+0000 7f9e0be00700 1 --2- 172.21.15.186:0/1008675771 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9e0c0fca70 0x7f9e0c10d0c0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f9dfc0096b0 tx=0x7f9dfc009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T20:45:47.806 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:45:47.807+0000 7f9e03e00700 1 -- 172.21.15.186:0/1008675771 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f9dfc00bed0 con 0x7f9e0c0fca70 2024-04-12T20:45:47.806 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:45:47.807+0000 7f9e12e00700 1 -- 172.21.15.186:0/1008675771 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9e0c10d800 con 0x7f9e0c0fca70 2024-04-12T20:45:47.806 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:45:47.807+0000 7f9e12e00700 1 -- 172.21.15.186:0/1008675771 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9e0c10e370 con 0x7f9e0c0fca70 2024-04-12T20:45:47.806 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:45:47.807+0000 7f9e03e00700 1 -- 172.21.15.186:0/1008675771 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f9dfc00b5c0 con 0x7f9e0c0fca70 2024-04-12T20:45:47.806 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:45:47.807+0000 7f9e03e00700 1 -- 172.21.15.186:0/1008675771 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f9dfc0255c0 con 0x7f9e0c0fca70 2024-04-12T20:45:47.807 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:45:47.808+0000 7f9e03e00700 1 -- 172.21.15.186:0/1008675771 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 17) v1 ==== 43481+0+0 (secure 0 0 0) 0x7f9dfc013420 con 0x7f9e0c0fca70 2024-04-12T20:45:47.807 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:45:47.809+0000 7f9e12e00700 1 -- 172.21.15.186:0/1008675771 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9e0c102f80 con 0x7f9e0c0fca70 2024-04-12T20:45:47.808 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:45:47.809+0000 7f9e03e00700 1 --2- 172.21.15.186:0/1008675771 >> [v2:172.21.15.18:6800/3048894687,v1:172.21.15.18:6801/3048894687] conn(0x7f9df0036970 0x7f9df0038e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T20:45:47.808 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:45:47.809+0000 7f9e03e00700 1 -- 172.21.15.186:0/1008675771 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1366+0+0 (secure 0 0 0) 0x7f9dfc049d80 con 0x7f9e0c0fca70 2024-04-12T20:45:47.808 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:45:47.809+0000 7f9e0b400700 1 --2- 172.21.15.186:0/1008675771 >> [v2:172.21.15.18:6800/3048894687,v1:172.21.15.18:6801/3048894687] conn(0x7f9df0036970 0x7f9df0038e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T20:45:47.809 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:45:47.810+0000 7f9e0b400700 1 --2- 172.21.15.186:0/1008675771 >> [v2:172.21.15.18:6800/3048894687,v1:172.21.15.18:6801/3048894687] conn(0x7f9df0036970 0x7f9df0038e20 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f9df8009990 tx=0x7f9df8006e40 comp rx=0 tx=0).ready entity=mgr.14212 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T20:45:47.816 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:45:47.817+0000 7f9e03e00700 1 -- 172.21.15.186:0/1008675771 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7f9dfc024540 con 0x7f9e0c0fca70 2024-04-12T20:45:48.113 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:45:48.114+0000 7f9e12e00700 1 -- 172.21.15.186:0/1008675771 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f9e0c102f80 con 0x7f9e0c0fca70 2024-04-12T20:45:48.114 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:45:48.115+0000 7f9e03e00700 1 -- 172.21.15.186:0/1008675771 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+739 (secure 0 0 0) 0x7f9dfc024360 con 0x7f9e0c0fca70 2024-04-12T20:45:48.114 INFO:teuthology.orchestra.run.smithi186.stdout: 2024-04-12T20:45:48.114 INFO:teuthology.orchestra.run.smithi186.stdout:{"epoch":1,"fsid":"1cbe407a-f90d-11ee-bc8d-c7b262605968","modified":"2024-04-12T20:42:56.111400Z","created":"2024-04-12T20:42:56.111400Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"smithi018","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:3300","nonce":0},{"type":"v1","addr":"172.21.15.18:6789","nonce":0}]},"addr":"172.21.15.18:6789/0","public_addr":"172.21.15.18:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-04-12T20:45:48.117 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:45:48.118+0000 7f9e12e00700 1 -- 172.21.15.186:0/1008675771 >> [v2:172.21.15.18:6800/3048894687,v1:172.21.15.18:6801/3048894687] conn(0x7f9df0036970 msgr2=0x7f9df0038e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:45:48.117 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:45:48.118+0000 7f9e12e00700 1 --2- 172.21.15.186:0/1008675771 >> [v2:172.21.15.18:6800/3048894687,v1:172.21.15.18:6801/3048894687] conn(0x7f9df0036970 0x7f9df0038e20 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f9df8009990 tx=0x7f9df8006e40 comp rx=0 tx=0).stop 2024-04-12T20:45:48.117 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:45:48.118+0000 7f9e12e00700 1 -- 172.21.15.186:0/1008675771 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9e0c0fca70 msgr2=0x7f9e0c10d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:45:48.117 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:45:48.118+0000 7f9e12e00700 1 --2- 172.21.15.186:0/1008675771 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9e0c0fca70 0x7f9e0c10d0c0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f9dfc0096b0 tx=0x7f9dfc009e30 comp rx=0 tx=0).stop 2024-04-12T20:45:48.117 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:45:48.118+0000 7f9e12e00700 1 -- 172.21.15.186:0/1008675771 shutdown_connections 2024-04-12T20:45:48.117 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:45:48.118+0000 7f9e12e00700 1 --2- 172.21.15.186:0/1008675771 >> [v2:172.21.15.18:6800/3048894687,v1:172.21.15.18:6801/3048894687] conn(0x7f9df0036970 0x7f9df0038e20 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:45:48.117 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:45:48.118+0000 7f9e12e00700 1 --2- 172.21.15.186:0/1008675771 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9e0c0fca70 0x7f9e0c10d0c0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:45:48.118 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:45:48.118+0000 7f9e12e00700 1 -- 172.21.15.186:0/1008675771 >> 172.21.15.186:0/1008675771 conn(0x7f9e0c0f81d0 msgr2=0x7f9e0c103bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T20:45:48.118 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:45:48.118+0000 7f9e12e00700 1 -- 172.21.15.186:0/1008675771 shutdown_connections 2024-04-12T20:45:48.118 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:45:48.118+0000 7f9e12e00700 1 -- 172.21.15.186:0/1008675771 wait complete. 2024-04-12T20:45:48.118 INFO:teuthology.orchestra.run.smithi186.stderr:dumped monmap epoch 1 2024-04-12T20:45:48.603 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:45:48 smithi018 ceph-mon[28799]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' 2024-04-12T20:45:48.603 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:45:48 smithi018 ceph-mon[28799]: mgrmap e17: smithi018.xjzwga(active, since 2s) 2024-04-12T20:45:48.603 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:45:48 smithi018 ceph-mon[28799]: from='client.? 172.21.15.186:0/1008675771' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-12T20:45:49.811 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-04-12T20:45:49.811 DEBUG:teuthology.orchestra.run.smithi186:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1cbe407a-f90d-11ee-bc8d-c7b262605968 -- ceph mon dump -f json 2024-04-12T20:45:50.171 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:45:49 smithi018 ceph-mon[28799]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' 2024-04-12T20:45:50.674 INFO:teuthology.orchestra.run.smithi186.stderr:Inferring config /etc/ceph/ceph.conf 2024-04-12T20:45:50.772 INFO:teuthology.orchestra.run.smithi186.stderr:Inferring config /etc/ceph/ceph.conf 2024-04-12T20:45:51.853 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:45:51 smithi018 ceph-mon[28799]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' 2024-04-12T20:45:51.853 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:45:51 smithi018 ceph-mon[28799]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' 2024-04-12T20:45:53.355 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:45:53.355+0000 7fbeb1600700 1 -- 172.21.15.186:0/173209077 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fbeac1030c0 msgr2=0x7fbeac1034d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:45:53.355 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:45:53.355+0000 7fbeb1600700 1 --2- 172.21.15.186:0/173209077 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fbeac1030c0 0x7fbeac1034d0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fbe9c0048c0 tx=0x7fbe9c009e30 comp rx=0 tx=0).stop 2024-04-12T20:45:53.355 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:45:53.355+0000 7fbeb1600700 1 -- 172.21.15.186:0/173209077 shutdown_connections 2024-04-12T20:45:53.355 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:45:53.355+0000 7fbeb1600700 1 --2- 172.21.15.186:0/173209077 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fbeac1030c0 0x7fbeac1034d0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:45:53.355 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:45:53.355+0000 7fbeb1600700 1 -- 172.21.15.186:0/173209077 >> 172.21.15.186:0/173209077 conn(0x7fbeac072300 msgr2=0x7fbeac074710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T20:45:53.355 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:45:53.355+0000 7fbeb1600700 1 -- 172.21.15.186:0/173209077 shutdown_connections 2024-04-12T20:45:53.355 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:45:53.355+0000 7fbeb1600700 1 -- 172.21.15.186:0/173209077 wait complete. 2024-04-12T20:45:53.356 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:45:53.356+0000 7fbeb1600700 1 Processor -- start 2024-04-12T20:45:53.356 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:45:53.356+0000 7fbeb1600700 1 -- start start 2024-04-12T20:45:53.356 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:45:53.357+0000 7fbeb1600700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fbeac1030c0 0x7fbeac1136f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T20:45:53.356 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:45:53.357+0000 7fbeb1600700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbeac113c30 con 0x7fbeac1030c0 2024-04-12T20:45:53.356 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:45:53.357+0000 7fbeaaa00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fbeac1030c0 0x7fbeac1136f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T20:45:53.356 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:45:53.357+0000 7fbeaaa00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fbeac1030c0 0x7fbeac1136f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.186:43344/0 (socket says 172.21.15.186:43344) 2024-04-12T20:45:53.357 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:45:53.357+0000 7fbeaaa00700 1 -- 172.21.15.186:0/1359875603 learned_addr learned my addr 172.21.15.186:0/1359875603 (peer_addr_for_me v2:172.21.15.186:0/0) 2024-04-12T20:45:53.357 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:45:53.358+0000 7fbeaaa00700 1 -- 172.21.15.186:0/1359875603 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe9c005040 con 0x7fbeac1030c0 2024-04-12T20:45:53.357 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:45:53.359+0000 7fbeaaa00700 1 --2- 172.21.15.186:0/1359875603 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fbeac1030c0 0x7fbeac1136f0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fbe9c00b900 tx=0x7fbe9c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T20:45:53.358 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:45:53.359+0000 7fbea2a00700 1 -- 172.21.15.186:0/1359875603 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fbe9c00b5c0 con 0x7fbeac1030c0 2024-04-12T20:45:53.358 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:45:53.359+0000 7fbeb1600700 1 -- 172.21.15.186:0/1359875603 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbeac113e30 con 0x7fbeac1030c0 2024-04-12T20:45:53.358 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:45:53.359+0000 7fbea2a00700 1 -- 172.21.15.186:0/1359875603 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fbe9c00be70 con 0x7fbeac1030c0 2024-04-12T20:45:53.358 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:45:53.359+0000 7fbea2a00700 1 -- 172.21.15.186:0/1359875603 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fbe9c01c590 con 0x7fbeac1030c0 2024-04-12T20:45:53.358 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:45:53.359+0000 7fbeb1600700 1 -- 172.21.15.186:0/1359875603 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbeac1149a0 con 0x7fbeac1030c0 2024-04-12T20:45:53.359 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:45:53.360+0000 7fbeb1600700 1 -- 172.21.15.186:0/1359875603 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbeac109560 con 0x7fbeac1030c0 2024-04-12T20:45:53.359 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:45:53.361+0000 7fbea2a00700 1 -- 172.21.15.186:0/1359875603 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 17) v1 ==== 43481+0+0 (secure 0 0 0) 0x7fbe9c004d30 con 0x7fbeac1030c0 2024-04-12T20:45:53.360 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:45:53.361+0000 7fbea2a00700 1 --2- 172.21.15.186:0/1359875603 >> [v2:172.21.15.18:6800/3048894687,v1:172.21.15.18:6801/3048894687] conn(0x7fbe90036970 0x7fbe90038e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T20:45:53.360 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:45:53.361+0000 7fbea2a00700 1 -- 172.21.15.186:0/1359875603 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1366+0+0 (secure 0 0 0) 0x7fbe9c049650 con 0x7fbeac1030c0 2024-04-12T20:45:53.361 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:45:53.362+0000 7fbeaa000700 1 --2- 172.21.15.186:0/1359875603 >> [v2:172.21.15.18:6800/3048894687,v1:172.21.15.18:6801/3048894687] conn(0x7fbe90036970 0x7fbe90038e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T20:45:53.362 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:45:53.363+0000 7fbeaa000700 1 --2- 172.21.15.186:0/1359875603 >> [v2:172.21.15.18:6800/3048894687,v1:172.21.15.18:6801/3048894687] conn(0x7fbe90036970 0x7fbe90038e20 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fbe98009990 tx=0x7fbe98006e40 comp rx=0 tx=0).ready entity=mgr.14212 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T20:45:53.376 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:45:53.377+0000 7fbea2a00700 1 -- 172.21.15.186:0/1359875603 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7fbe9c01b350 con 0x7fbeac1030c0 2024-04-12T20:45:53.603 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:45:53 smithi018 ceph-mon[28799]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' 2024-04-12T20:45:53.603 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:45:53 smithi018 ceph-mon[28799]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' 2024-04-12T20:45:53.603 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:45:53 smithi018 ceph-mon[28799]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "config rm", "who": "osd/host:smithi018", "name": "osd_memory_target"}]: dispatch 2024-04-12T20:45:53.676 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:45:53.677+0000 7fbeb1600700 1 -- 172.21.15.186:0/1359875603 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7fbeac115470 con 0x7fbeac1030c0 2024-04-12T20:45:53.677 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:45:53.678+0000 7fbea2a00700 1 -- 172.21.15.186:0/1359875603 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+739 (secure 0 0 0) 0x7fbe9c021030 con 0x7fbeac1030c0 2024-04-12T20:45:53.677 INFO:teuthology.orchestra.run.smithi186.stdout: 2024-04-12T20:45:53.678 INFO:teuthology.orchestra.run.smithi186.stdout:{"epoch":1,"fsid":"1cbe407a-f90d-11ee-bc8d-c7b262605968","modified":"2024-04-12T20:42:56.111400Z","created":"2024-04-12T20:42:56.111400Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"smithi018","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:3300","nonce":0},{"type":"v1","addr":"172.21.15.18:6789","nonce":0}]},"addr":"172.21.15.18:6789/0","public_addr":"172.21.15.18:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-04-12T20:45:53.680 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:45:53.681+0000 7fbeb1600700 1 -- 172.21.15.186:0/1359875603 >> [v2:172.21.15.18:6800/3048894687,v1:172.21.15.18:6801/3048894687] conn(0x7fbe90036970 msgr2=0x7fbe90038e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:45:53.680 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:45:53.681+0000 7fbeb1600700 1 --2- 172.21.15.186:0/1359875603 >> [v2:172.21.15.18:6800/3048894687,v1:172.21.15.18:6801/3048894687] conn(0x7fbe90036970 0x7fbe90038e20 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fbe98009990 tx=0x7fbe98006e40 comp rx=0 tx=0).stop 2024-04-12T20:45:53.680 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:45:53.681+0000 7fbeb1600700 1 -- 172.21.15.186:0/1359875603 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fbeac1030c0 msgr2=0x7fbeac1136f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:45:53.680 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:45:53.681+0000 7fbeb1600700 1 --2- 172.21.15.186:0/1359875603 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fbeac1030c0 0x7fbeac1136f0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fbe9c00b900 tx=0x7fbe9c009e30 comp rx=0 tx=0).stop 2024-04-12T20:45:53.680 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:45:53.681+0000 7fbeb1600700 1 -- 172.21.15.186:0/1359875603 shutdown_connections 2024-04-12T20:45:53.681 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:45:53.681+0000 7fbeb1600700 1 --2- 172.21.15.186:0/1359875603 >> [v2:172.21.15.18:6800/3048894687,v1:172.21.15.18:6801/3048894687] conn(0x7fbe90036970 0x7fbe90038e20 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:45:53.681 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:45:53.681+0000 7fbeb1600700 1 --2- 172.21.15.186:0/1359875603 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fbeac1030c0 0x7fbeac1136f0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:45:53.681 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:45:53.681+0000 7fbeb1600700 1 -- 172.21.15.186:0/1359875603 >> 172.21.15.186:0/1359875603 conn(0x7fbeac072300 msgr2=0x7fbeac0741c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T20:45:53.681 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:45:53.681+0000 7fbeb1600700 1 -- 172.21.15.186:0/1359875603 shutdown_connections 2024-04-12T20:45:53.681 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:45:53.681+0000 7fbeb1600700 1 -- 172.21.15.186:0/1359875603 wait complete. 2024-04-12T20:45:53.681 INFO:teuthology.orchestra.run.smithi186.stderr:dumped monmap epoch 1 2024-04-12T20:45:54.603 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:45:54 smithi018 ceph-mon[28799]: from='client.? 172.21.15.186:0/1359875603' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-12T20:45:54.603 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:45:54 smithi018 ceph-mon[28799]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' 2024-04-12T20:45:54.603 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:45:54 smithi018 ceph-mon[28799]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' 2024-04-12T20:45:54.603 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:45:54 smithi018 ceph-mon[28799]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' 2024-04-12T20:45:54.603 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:45:54 smithi018 ceph-mon[28799]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' 2024-04-12T20:45:54.603 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:45:54 smithi018 ceph-mon[28799]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "config rm", "who": "osd/host:smithi186", "name": "osd_memory_target"}]: dispatch 2024-04-12T20:45:54.603 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:45:54 smithi018 ceph-mon[28799]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-12T20:45:54.603 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:45:54 smithi018 ceph-mon[28799]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-12T20:45:54.604 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:45:54 smithi018 ceph-mon[28799]: Updating smithi018:/etc/ceph/ceph.conf 2024-04-12T20:45:54.604 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:45:54 smithi018 ceph-mon[28799]: Updating smithi186:/etc/ceph/ceph.conf 2024-04-12T20:45:55.440 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-04-12T20:45:55.441 DEBUG:teuthology.orchestra.run.smithi186:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1cbe407a-f90d-11ee-bc8d-c7b262605968 -- ceph mon dump -f json 2024-04-12T20:45:55.603 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:45:55 smithi018 ceph-mon[28799]: Updating smithi186:/var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/config/ceph.conf 2024-04-12T20:45:55.603 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:45:55 smithi018 ceph-mon[28799]: Updating smithi018:/var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/config/ceph.conf 2024-04-12T20:45:55.865 INFO:teuthology.orchestra.run.smithi186.stderr:Inferring config /var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/config/ceph.conf 2024-04-12T20:45:56.603 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:45:56 smithi018 ceph-mon[28799]: Updating smithi186:/etc/ceph/ceph.client.admin.keyring 2024-04-12T20:45:56.603 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:45:56 smithi018 ceph-mon[28799]: Updating smithi018:/etc/ceph/ceph.client.admin.keyring 2024-04-12T20:45:56.603 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:45:56 smithi018 ceph-mon[28799]: Updating smithi186:/var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/config/ceph.client.admin.keyring 2024-04-12T20:45:57.767 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:45:57.767+0000 7f7891200700 1 -- 172.21.15.186:0/2625415551 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f788c104c70 msgr2=0x7f788c105040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:45:57.767 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:45:57.767+0000 7f7891200700 1 --2- 172.21.15.186:0/2625415551 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f788c104c70 0x7f788c105040 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f787c0048c0 tx=0x7f787c009e30 comp rx=0 tx=0).stop 2024-04-12T20:45:57.768 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:45:57.767+0000 7f7891200700 1 -- 172.21.15.186:0/2625415551 shutdown_connections 2024-04-12T20:45:57.768 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:45:57.767+0000 7f7891200700 1 --2- 172.21.15.186:0/2625415551 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f788c104c70 0x7f788c105040 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:45:57.768 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:45:57.767+0000 7f7891200700 1 -- 172.21.15.186:0/2625415551 >> 172.21.15.186:0/2625415551 conn(0x7f788c0fab10 msgr2=0x7f788c0fcef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T20:45:57.768 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:45:57.767+0000 7f7891200700 1 -- 172.21.15.186:0/2625415551 shutdown_connections 2024-04-12T20:45:57.769 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:45:57.767+0000 7f7891200700 1 -- 172.21.15.186:0/2625415551 wait complete. 2024-04-12T20:45:57.769 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:45:57.768+0000 7f7891200700 1 Processor -- start 2024-04-12T20:45:57.769 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:45:57.768+0000 7f7891200700 1 -- start start 2024-04-12T20:45:57.771 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:45:57.769+0000 7f7891200700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f788c104c70 0x7f788c1152f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T20:45:57.771 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:45:57.769+0000 7f7891200700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f788c115830 con 0x7f788c104c70 2024-04-12T20:45:57.771 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:45:57.769+0000 7f7889e00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f788c104c70 0x7f788c1152f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T20:45:57.771 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:45:57.769+0000 7f7889e00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f788c104c70 0x7f788c1152f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.186:55072/0 (socket says 172.21.15.186:55072) 2024-04-12T20:45:57.772 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:45:57.769+0000 7f7889e00700 1 -- 172.21.15.186:0/3716716920 learned_addr learned my addr 172.21.15.186:0/3716716920 (peer_addr_for_me v2:172.21.15.186:0/0) 2024-04-12T20:45:57.772 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:45:57.770+0000 7f7889e00700 1 -- 172.21.15.186:0/3716716920 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f787c005040 con 0x7f788c104c70 2024-04-12T20:45:57.772 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:45:57.771+0000 7f7889e00700 1 --2- 172.21.15.186:0/3716716920 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f788c104c70 0x7f788c1152f0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f787c00b7f0 tx=0x7f787c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T20:45:57.772 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:45:57.771+0000 7f7882000700 1 -- 172.21.15.186:0/3716716920 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f787c009370 con 0x7f788c104c70 2024-04-12T20:45:57.772 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:45:57.771+0000 7f7891200700 1 -- 172.21.15.186:0/3716716920 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f788c115a30 con 0x7f788c104c70 2024-04-12T20:45:57.772 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:45:57.771+0000 7f7882000700 1 -- 172.21.15.186:0/3716716920 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f787c013420 con 0x7f788c104c70 2024-04-12T20:45:57.772 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:45:57.771+0000 7f7882000700 1 -- 172.21.15.186:0/3716716920 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f787c01c590 con 0x7f788c104c70 2024-04-12T20:45:57.772 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:45:57.771+0000 7f7891200700 1 -- 172.21.15.186:0/3716716920 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f788c115f20 con 0x7f788c104c70 2024-04-12T20:45:57.773 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:45:57.772+0000 7f7891200700 1 -- 172.21.15.186:0/3716716920 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f788c1003c0 con 0x7f788c104c70 2024-04-12T20:45:57.774 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:45:57.773+0000 7f7882000700 1 -- 172.21.15.186:0/3716716920 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 17) v1 ==== 43481+0+0 (secure 0 0 0) 0x7f787c013a70 con 0x7f788c104c70 2024-04-12T20:45:57.774 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:45:57.773+0000 7f7882000700 1 --2- 172.21.15.186:0/3716716920 >> [v2:172.21.15.18:6800/3048894687,v1:172.21.15.18:6801/3048894687] conn(0x7f7870036970 0x7f7870038e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T20:45:57.774 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:45:57.773+0000 7f7882000700 1 -- 172.21.15.186:0/3716716920 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1366+0+0 (secure 0 0 0) 0x7f787c049400 con 0x7f788c104c70 2024-04-12T20:45:57.774 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:45:57.774+0000 7f7889400700 1 --2- 172.21.15.186:0/3716716920 >> [v2:172.21.15.18:6800/3048894687,v1:172.21.15.18:6801/3048894687] conn(0x7f7870036970 0x7f7870038e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T20:45:57.774 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:45:57.775+0000 7f7889400700 1 --2- 172.21.15.186:0/3716716920 >> [v2:172.21.15.18:6800/3048894687,v1:172.21.15.18:6801/3048894687] conn(0x7f7870036970 0x7f7870038e20 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f7878009990 tx=0x7f7878006e40 comp rx=0 tx=0).ready entity=mgr.14212 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T20:45:57.787 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:45:57.788+0000 7f7882000700 1 -- 172.21.15.186:0/3716716920 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7f787c024b50 con 0x7f788c104c70 2024-04-12T20:45:58.087 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:45:58.087+0000 7f7891200700 1 -- 172.21.15.186:0/3716716920 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f788c05f5f0 con 0x7f788c104c70 2024-04-12T20:45:58.087 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:45:58.089+0000 7f7882000700 1 -- 172.21.15.186:0/3716716920 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+739 (secure 0 0 0) 0x7f787c021070 con 0x7f788c104c70 2024-04-12T20:45:58.088 INFO:teuthology.orchestra.run.smithi186.stdout: 2024-04-12T20:45:58.088 INFO:teuthology.orchestra.run.smithi186.stdout:{"epoch":1,"fsid":"1cbe407a-f90d-11ee-bc8d-c7b262605968","modified":"2024-04-12T20:42:56.111400Z","created":"2024-04-12T20:42:56.111400Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"smithi018","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:3300","nonce":0},{"type":"v1","addr":"172.21.15.18:6789","nonce":0}]},"addr":"172.21.15.18:6789/0","public_addr":"172.21.15.18:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-04-12T20:45:58.091 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:45:58.091+0000 7f7891200700 1 -- 172.21.15.186:0/3716716920 >> [v2:172.21.15.18:6800/3048894687,v1:172.21.15.18:6801/3048894687] conn(0x7f7870036970 msgr2=0x7f7870038e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:45:58.091 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:45:58.091+0000 7f7891200700 1 --2- 172.21.15.186:0/3716716920 >> [v2:172.21.15.18:6800/3048894687,v1:172.21.15.18:6801/3048894687] conn(0x7f7870036970 0x7f7870038e20 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f7878009990 tx=0x7f7878006e40 comp rx=0 tx=0).stop 2024-04-12T20:45:58.091 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:45:58.091+0000 7f7891200700 1 -- 172.21.15.186:0/3716716920 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f788c104c70 msgr2=0x7f788c1152f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:45:58.091 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:45:58.091+0000 7f7891200700 1 --2- 172.21.15.186:0/3716716920 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f788c104c70 0x7f788c1152f0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f787c00b7f0 tx=0x7f787c009e30 comp rx=0 tx=0).stop 2024-04-12T20:45:58.091 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:45:58.092+0000 7f7891200700 1 -- 172.21.15.186:0/3716716920 shutdown_connections 2024-04-12T20:45:58.091 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:45:58.092+0000 7f7891200700 1 --2- 172.21.15.186:0/3716716920 >> [v2:172.21.15.18:6800/3048894687,v1:172.21.15.18:6801/3048894687] conn(0x7f7870036970 0x7f7870038e20 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:45:58.091 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:45:58.092+0000 7f7891200700 1 --2- 172.21.15.186:0/3716716920 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f788c104c70 0x7f788c1152f0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:45:58.091 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:45:58.092+0000 7f7891200700 1 -- 172.21.15.186:0/3716716920 >> 172.21.15.186:0/3716716920 conn(0x7f788c0fab10 msgr2=0x7f788c0fd4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T20:45:58.092 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:45:58.092+0000 7f7891200700 1 -- 172.21.15.186:0/3716716920 shutdown_connections 2024-04-12T20:45:58.092 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:45:58.092+0000 7f7891200700 1 -- 172.21.15.186:0/3716716920 wait complete. 2024-04-12T20:45:58.092 INFO:teuthology.orchestra.run.smithi186.stderr:dumped monmap epoch 1 2024-04-12T20:45:58.103 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:45:57 smithi018 ceph-mon[28799]: Updating smithi018:/var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/config/ceph.client.admin.keyring 2024-04-12T20:45:58.103 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:45:57 smithi018 ceph-mon[28799]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' 2024-04-12T20:45:58.103 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:45:57 smithi018 ceph-mon[28799]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' 2024-04-12T20:45:58.104 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:45:57 smithi018 ceph-mon[28799]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' 2024-04-12T20:45:58.104 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:45:57 smithi018 ceph-mon[28799]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' 2024-04-12T20:45:58.104 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:45:57 smithi018 ceph-mon[28799]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' 2024-04-12T20:45:58.104 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:45:57 smithi018 ceph-mon[28799]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi186", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-04-12T20:45:58.104 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:45:57 smithi018 ceph-mon[28799]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' cmd='[{"prefix": "auth get-or-create", "entity": "client.crash.smithi186", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]': finished 2024-04-12T20:45:58.104 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:45:57 smithi018 ceph-mon[28799]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-12T20:45:58.104 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:45:57 smithi018 ceph-mon[28799]: Deploying daemon crash.smithi186 on smithi186 2024-04-12T20:45:59.103 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:45:58 smithi018 ceph-mon[28799]: from='client.? 172.21.15.186:0/3716716920' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-12T20:46:00.235 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-04-12T20:46:00.235 DEBUG:teuthology.orchestra.run.smithi186:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1cbe407a-f90d-11ee-bc8d-c7b262605968 -- ceph mon dump -f json 2024-04-12T20:46:00.354 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:45:59 smithi018 ceph-mon[28799]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T20:46:00.690 INFO:teuthology.orchestra.run.smithi186.stderr:Inferring config /var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/config/ceph.conf 2024-04-12T20:46:02.892 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:46:02.867+0000 7f3580a00700 1 -- 172.21.15.186:0/641388747 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f3574009700 con 0x7f3584071f90 2024-04-12T20:46:02.892 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:46:02.867+0000 7f3589200700 1 -- 172.21.15.186:0/641388747 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3584071f90 msgr2=0x7f3584070360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:46:02.892 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:46:02.867+0000 7f3589200700 1 --2- 172.21.15.186:0/641388747 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3584071f90 0x7f3584070360 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f35740048c0 tx=0x7f3574009e30 comp rx=0 tx=0).stop 2024-04-12T20:46:02.892 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:46:02.868+0000 7f3589200700 1 -- 172.21.15.186:0/641388747 shutdown_connections 2024-04-12T20:46:02.892 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:46:02.868+0000 7f3589200700 1 --2- 172.21.15.186:0/641388747 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3584071f90 0x7f3584070360 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:46:02.892 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:46:02.868+0000 7f3589200700 1 -- 172.21.15.186:0/641388747 >> 172.21.15.186:0/641388747 conn(0x7f35840f85e0 msgr2=0x7f35840fa9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T20:46:02.892 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:46:02.868+0000 7f3589200700 1 -- 172.21.15.186:0/641388747 shutdown_connections 2024-04-12T20:46:02.892 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:46:02.868+0000 7f3589200700 1 -- 172.21.15.186:0/641388747 wait complete. 2024-04-12T20:46:02.892 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:46:02.868+0000 7f3589200700 1 Processor -- start 2024-04-12T20:46:02.892 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:46:02.869+0000 7f3589200700 1 -- start start 2024-04-12T20:46:02.892 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:46:02.869+0000 7f3589200700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3584071f90 0x7f35841197f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T20:46:02.893 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:46:02.869+0000 7f3589200700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3584119d30 con 0x7f3584071f90 2024-04-12T20:46:02.893 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:46:02.870+0000 7f3581e00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3584071f90 0x7f35841197f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T20:46:02.893 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:46:02.870+0000 7f3581e00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3584071f90 0x7f35841197f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.186:55100/0 (socket says 172.21.15.186:55100) 2024-04-12T20:46:02.893 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:46:02.870+0000 7f3581e00700 1 -- 172.21.15.186:0/3983440824 learned_addr learned my addr 172.21.15.186:0/3983440824 (peer_addr_for_me v2:172.21.15.186:0/0) 2024-04-12T20:46:02.893 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:46:02.870+0000 7f3581e00700 1 -- 172.21.15.186:0/3983440824 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3574005040 con 0x7f3584071f90 2024-04-12T20:46:02.893 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:46:02.871+0000 7f3581e00700 1 --2- 172.21.15.186:0/3983440824 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3584071f90 0x7f35841197f0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f357400b890 tx=0x7f3574009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T20:46:02.893 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:46:02.871+0000 7f357a000700 1 -- 172.21.15.186:0/3983440824 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f35740098d0 con 0x7f3584071f90 2024-04-12T20:46:02.893 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:46:02.871+0000 7f3589200700 1 -- 172.21.15.186:0/3983440824 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3584119f30 con 0x7f3584071f90 2024-04-12T20:46:02.893 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:46:02.871+0000 7f3589200700 1 -- 172.21.15.186:0/3983440824 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f358411a3a0 con 0x7f3584071f90 2024-04-12T20:46:02.893 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:46:02.871+0000 7f357a000700 1 -- 172.21.15.186:0/3983440824 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f357400bc10 con 0x7f3584071f90 2024-04-12T20:46:02.893 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:46:02.871+0000 7f357a000700 1 -- 172.21.15.186:0/3983440824 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f35740255c0 con 0x7f3584071f90 2024-04-12T20:46:02.893 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:46:02.873+0000 7f357a000700 1 -- 172.21.15.186:0/3983440824 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 17) v1 ==== 43481+0+0 (secure 0 0 0) 0x7f3574013420 con 0x7f3584071f90 2024-04-12T20:46:02.893 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:46:02.873+0000 7f3589200700 1 -- 172.21.15.186:0/3983440824 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3584071aa0 con 0x7f3584071f90 2024-04-12T20:46:02.894 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:46:02.874+0000 7f357a000700 1 --2- 172.21.15.186:0/3983440824 >> [v2:172.21.15.18:6800/3048894687,v1:172.21.15.18:6801/3048894687] conn(0x7f3568036a00 0x7f3568038eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T20:46:02.894 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:46:02.874+0000 7f357a000700 1 -- 172.21.15.186:0/3983440824 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1366+0+0 (secure 0 0 0) 0x7f3574049c00 con 0x7f3584071f90 2024-04-12T20:46:02.894 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:46:02.874+0000 7f3581400700 1 --2- 172.21.15.186:0/3983440824 >> [v2:172.21.15.18:6800/3048894687,v1:172.21.15.18:6801/3048894687] conn(0x7f3568036a00 0x7f3568038eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T20:46:02.894 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:46:02.875+0000 7f3581400700 1 --2- 172.21.15.186:0/3983440824 >> [v2:172.21.15.18:6800/3048894687,v1:172.21.15.18:6801/3048894687] conn(0x7f3568036a00 0x7f3568038eb0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f3570009990 tx=0x7f3570006e40 comp rx=0 tx=0).ready entity=mgr.14212 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T20:46:02.894 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:46:02.891+0000 7f357a000700 1 -- 172.21.15.186:0/3983440824 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7f35740249b0 con 0x7f3584071f90 2024-04-12T20:46:03.185 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:46:03.185+0000 7f3589200700 1 -- 172.21.15.186:0/3983440824 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f358410d440 con 0x7f3584071f90 2024-04-12T20:46:03.185 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:46:03.187+0000 7f357a000700 1 -- 172.21.15.186:0/3983440824 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+739 (secure 0 0 0) 0x7f3574024360 con 0x7f3584071f90 2024-04-12T20:46:03.186 INFO:teuthology.orchestra.run.smithi186.stdout: 2024-04-12T20:46:03.186 INFO:teuthology.orchestra.run.smithi186.stdout:{"epoch":1,"fsid":"1cbe407a-f90d-11ee-bc8d-c7b262605968","modified":"2024-04-12T20:42:56.111400Z","created":"2024-04-12T20:42:56.111400Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"smithi018","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:3300","nonce":0},{"type":"v1","addr":"172.21.15.18:6789","nonce":0}]},"addr":"172.21.15.18:6789/0","public_addr":"172.21.15.18:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-04-12T20:46:03.190 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:46:03.191+0000 7f3589200700 1 -- 172.21.15.186:0/3983440824 >> [v2:172.21.15.18:6800/3048894687,v1:172.21.15.18:6801/3048894687] conn(0x7f3568036a00 msgr2=0x7f3568038eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:46:03.190 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:46:03.191+0000 7f3589200700 1 --2- 172.21.15.186:0/3983440824 >> [v2:172.21.15.18:6800/3048894687,v1:172.21.15.18:6801/3048894687] conn(0x7f3568036a00 0x7f3568038eb0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f3570009990 tx=0x7f3570006e40 comp rx=0 tx=0).stop 2024-04-12T20:46:03.190 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:46:03.191+0000 7f3589200700 1 -- 172.21.15.186:0/3983440824 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3584071f90 msgr2=0x7f35841197f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:46:03.190 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:46:03.191+0000 7f3589200700 1 --2- 172.21.15.186:0/3983440824 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3584071f90 0x7f35841197f0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f357400b890 tx=0x7f3574009e30 comp rx=0 tx=0).stop 2024-04-12T20:46:03.190 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:46:03.191+0000 7f3589200700 1 -- 172.21.15.186:0/3983440824 shutdown_connections 2024-04-12T20:46:03.190 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:46:03.191+0000 7f3589200700 1 --2- 172.21.15.186:0/3983440824 >> [v2:172.21.15.18:6800/3048894687,v1:172.21.15.18:6801/3048894687] conn(0x7f3568036a00 0x7f3568038eb0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:46:03.190 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:46:03.191+0000 7f3589200700 1 --2- 172.21.15.186:0/3983440824 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3584071f90 0x7f35841197f0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:46:03.190 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:46:03.191+0000 7f3589200700 1 -- 172.21.15.186:0/3983440824 >> 172.21.15.186:0/3983440824 conn(0x7f35840f85e0 msgr2=0x7f35840fa0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T20:46:03.191 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:46:03.191+0000 7f3589200700 1 -- 172.21.15.186:0/3983440824 shutdown_connections 2024-04-12T20:46:03.191 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:46:03.191+0000 7f3589200700 1 -- 172.21.15.186:0/3983440824 wait complete. 2024-04-12T20:46:03.191 INFO:teuthology.orchestra.run.smithi186.stderr:dumped monmap epoch 1 2024-04-12T20:46:04.353 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:46:03 smithi018 ceph-mon[28799]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' 2024-04-12T20:46:04.353 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:46:03 smithi018 ceph-mon[28799]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' 2024-04-12T20:46:04.353 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:46:03 smithi018 ceph-mon[28799]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' 2024-04-12T20:46:04.353 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:46:03 smithi018 ceph-mon[28799]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' 2024-04-12T20:46:04.353 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:46:03 smithi018 ceph-mon[28799]: Deploying daemon node-exporter.smithi186 on smithi186 2024-04-12T20:46:04.354 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:46:03 smithi018 ceph-mon[28799]: from='client.? 172.21.15.186:0/3983440824' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-12T20:46:04.867 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-04-12T20:46:04.867 DEBUG:teuthology.orchestra.run.smithi186:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1cbe407a-f90d-11ee-bc8d-c7b262605968 -- ceph mon dump -f json 2024-04-12T20:46:05.327 INFO:teuthology.orchestra.run.smithi186.stderr:Inferring config /var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/config/ceph.conf 2024-04-12T20:46:06.155 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:46:05 smithi018 ceph-mon[28799]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-12T20:46:06.155 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:46:05 smithi018 ceph-mon[28799]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' 2024-04-12T20:46:08.353 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:46:07 smithi018 ceph-mon[28799]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-12T20:46:08.827 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:46:08.826+0000 7f9abe600700 1 -- 172.21.15.186:0/896133957 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9ab8102ac0 msgr2=0x7f9ab8102e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:46:08.828 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:46:08.826+0000 7f9abe600700 1 --2- 172.21.15.186:0/896133957 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9ab8102ac0 0x7f9ab8102e90 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f9aa80048c0 tx=0x7f9aa8009e30 comp rx=0 tx=0).stop 2024-04-12T20:46:08.828 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:46:08.826+0000 7f9abe600700 1 -- 172.21.15.186:0/896133957 shutdown_connections 2024-04-12T20:46:08.828 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:46:08.826+0000 7f9abe600700 1 --2- 172.21.15.186:0/896133957 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9ab8102ac0 0x7f9ab8102e90 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:46:08.828 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:46:08.826+0000 7f9abe600700 1 -- 172.21.15.186:0/896133957 >> 172.21.15.186:0/896133957 conn(0x7f9ab80f85e0 msgr2=0x7f9ab80fa9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T20:46:08.828 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:46:08.827+0000 7f9abe600700 1 -- 172.21.15.186:0/896133957 shutdown_connections 2024-04-12T20:46:08.828 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:46:08.827+0000 7f9abe600700 1 -- 172.21.15.186:0/896133957 wait complete. 2024-04-12T20:46:08.828 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:46:08.827+0000 7f9abe600700 1 Processor -- start 2024-04-12T20:46:08.828 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:46:08.828+0000 7f9abe600700 1 -- start start 2024-04-12T20:46:08.829 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:46:08.828+0000 7f9abe600700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9ab8102ac0 0x7f9ab8113120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T20:46:08.829 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:46:08.828+0000 7f9abe600700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ab8113660 con 0x7f9ab8102ac0 2024-04-12T20:46:08.829 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:46:08.829+0000 7f9ab7400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9ab8102ac0 0x7f9ab8113120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T20:46:08.829 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:46:08.829+0000 7f9ab7400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9ab8102ac0 0x7f9ab8113120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.186:39434/0 (socket says 172.21.15.186:39434) 2024-04-12T20:46:08.829 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:46:08.829+0000 7f9ab7400700 1 -- 172.21.15.186:0/1664864883 learned_addr learned my addr 172.21.15.186:0/1664864883 (peer_addr_for_me v2:172.21.15.186:0/0) 2024-04-12T20:46:08.829 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:46:08.829+0000 7f9ab7400700 1 -- 172.21.15.186:0/1664864883 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9aa8005040 con 0x7f9ab8102ac0 2024-04-12T20:46:08.830 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:46:08.830+0000 7f9ab7400700 1 --2- 172.21.15.186:0/1664864883 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9ab8102ac0 0x7f9ab8113120 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f9aa800b820 tx=0x7f9aa8009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T20:46:08.830 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:46:08.830+0000 7f9aaf400700 1 -- 172.21.15.186:0/1664864883 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f9aa80093c0 con 0x7f9ab8102ac0 2024-04-12T20:46:08.830 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:46:08.830+0000 7f9abe600700 1 -- 172.21.15.186:0/1664864883 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9ab8113860 con 0x7f9ab8102ac0 2024-04-12T20:46:08.830 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:46:08.830+0000 7f9abe600700 1 -- 172.21.15.186:0/1664864883 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9ab8113d50 con 0x7f9ab8102ac0 2024-04-12T20:46:08.830 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:46:08.830+0000 7f9aaf400700 1 -- 172.21.15.186:0/1664864883 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f9aa8013420 con 0x7f9ab8102ac0 2024-04-12T20:46:08.830 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:46:08.830+0000 7f9aaf400700 1 -- 172.21.15.186:0/1664864883 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f9aa80255c0 con 0x7f9ab8102ac0 2024-04-12T20:46:08.831 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:46:08.831+0000 7f9abe600700 1 -- 172.21.15.186:0/1664864883 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9ab80fe1c0 con 0x7f9ab8102ac0 2024-04-12T20:46:08.831 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:46:08.832+0000 7f9aaf400700 1 -- 172.21.15.186:0/1664864883 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 17) v1 ==== 43481+0+0 (secure 0 0 0) 0x7f9aa8013590 con 0x7f9ab8102ac0 2024-04-12T20:46:08.832 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:46:08.832+0000 7f9aaf400700 1 --2- 172.21.15.186:0/1664864883 >> [v2:172.21.15.18:6800/3048894687,v1:172.21.15.18:6801/3048894687] conn(0x7f9a9c0369c0 0x7f9a9c038e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T20:46:08.832 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:46:08.833+0000 7f9aaf400700 1 -- 172.21.15.186:0/1664864883 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1366+0+0 (secure 0 0 0) 0x7f9aa8049b50 con 0x7f9ab8102ac0 2024-04-12T20:46:08.833 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:46:08.833+0000 7f9ab6a00700 1 --2- 172.21.15.186:0/1664864883 >> [v2:172.21.15.18:6800/3048894687,v1:172.21.15.18:6801/3048894687] conn(0x7f9a9c0369c0 0x7f9a9c038e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T20:46:08.834 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:46:08.834+0000 7f9ab6a00700 1 --2- 172.21.15.186:0/1664864883 >> [v2:172.21.15.18:6800/3048894687,v1:172.21.15.18:6801/3048894687] conn(0x7f9a9c0369c0 0x7f9a9c038e70 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f9aa4009990 tx=0x7f9aa4006e40 comp rx=0 tx=0).ready entity=mgr.14212 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T20:46:08.848 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:46:08.849+0000 7f9aaf400700 1 -- 172.21.15.186:0/1664864883 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7f9aa80238f0 con 0x7f9ab8102ac0 2024-04-12T20:46:09.165 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:46:09.163+0000 7f9abe600700 1 -- 172.21.15.186:0/1664864883 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f9ab805f5f0 con 0x7f9ab8102ac0 2024-04-12T20:46:09.166 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:46:09.164+0000 7f9aaf400700 1 -- 172.21.15.186:0/1664864883 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+739 (secure 0 0 0) 0x7f9aa8011370 con 0x7f9ab8102ac0 2024-04-12T20:46:09.166 INFO:teuthology.orchestra.run.smithi186.stdout: 2024-04-12T20:46:09.166 INFO:teuthology.orchestra.run.smithi186.stdout:{"epoch":1,"fsid":"1cbe407a-f90d-11ee-bc8d-c7b262605968","modified":"2024-04-12T20:42:56.111400Z","created":"2024-04-12T20:42:56.111400Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"smithi018","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:3300","nonce":0},{"type":"v1","addr":"172.21.15.18:6789","nonce":0}]},"addr":"172.21.15.18:6789/0","public_addr":"172.21.15.18:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-04-12T20:46:09.167 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:46:09.165+0000 7f9abe600700 1 -- 172.21.15.186:0/1664864883 >> [v2:172.21.15.18:6800/3048894687,v1:172.21.15.18:6801/3048894687] conn(0x7f9a9c0369c0 msgr2=0x7f9a9c038e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:46:09.167 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:46:09.165+0000 7f9abe600700 1 --2- 172.21.15.186:0/1664864883 >> [v2:172.21.15.18:6800/3048894687,v1:172.21.15.18:6801/3048894687] conn(0x7f9a9c0369c0 0x7f9a9c038e70 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f9aa4009990 tx=0x7f9aa4006e40 comp rx=0 tx=0).stop 2024-04-12T20:46:09.168 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:46:09.165+0000 7f9abe600700 1 -- 172.21.15.186:0/1664864883 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9ab8102ac0 msgr2=0x7f9ab8113120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:46:09.168 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:46:09.165+0000 7f9abe600700 1 --2- 172.21.15.186:0/1664864883 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9ab8102ac0 0x7f9ab8113120 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f9aa800b820 tx=0x7f9aa8009e30 comp rx=0 tx=0).stop 2024-04-12T20:46:09.168 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:46:09.165+0000 7f9abe600700 1 -- 172.21.15.186:0/1664864883 shutdown_connections 2024-04-12T20:46:09.168 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:46:09.165+0000 7f9abe600700 1 --2- 172.21.15.186:0/1664864883 >> [v2:172.21.15.18:6800/3048894687,v1:172.21.15.18:6801/3048894687] conn(0x7f9a9c0369c0 0x7f9a9c038e70 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:46:09.168 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:46:09.165+0000 7f9abe600700 1 --2- 172.21.15.186:0/1664864883 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9ab8102ac0 0x7f9ab8113120 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:46:09.168 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:46:09.165+0000 7f9abe600700 1 -- 172.21.15.186:0/1664864883 >> 172.21.15.186:0/1664864883 conn(0x7f9ab80f85e0 msgr2=0x7f9ab80f9bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T20:46:09.168 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:46:09.168+0000 7f9abe600700 1 -- 172.21.15.186:0/1664864883 shutdown_connections 2024-04-12T20:46:09.169 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:46:09.168+0000 7f9abe600700 1 -- 172.21.15.186:0/1664864883 wait complete. 2024-04-12T20:46:09.170 INFO:teuthology.orchestra.run.smithi186.stderr:dumped monmap epoch 1 2024-04-12T20:46:09.853 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:46:09 smithi018 ceph-mon[28799]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' 2024-04-12T20:46:09.854 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:46:09 smithi018 ceph-mon[28799]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' 2024-04-12T20:46:09.854 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:46:09 smithi018 ceph-mon[28799]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' 2024-04-12T20:46:09.854 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:46:09 smithi018 ceph-mon[28799]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' 2024-04-12T20:46:09.854 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:46:09 smithi018 ceph-mon[28799]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi186.kqcfxk", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-12T20:46:09.854 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:46:09 smithi018 ceph-mon[28799]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.smithi186.kqcfxk", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-04-12T20:46:09.854 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:46:09 smithi018 ceph-mon[28799]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-12T20:46:09.854 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:46:09 smithi018 ceph-mon[28799]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-12T20:46:09.854 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:46:09 smithi018 ceph-mon[28799]: Deploying daemon mgr.smithi186.kqcfxk on smithi186 2024-04-12T20:46:09.854 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:46:09 smithi018 ceph-mon[28799]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-12T20:46:09.854 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:46:09 smithi018 ceph-mon[28799]: from='client.? 172.21.15.186:0/1664864883' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-12T20:46:11.353 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:46:10 smithi018 ceph-mon[28799]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' 2024-04-12T20:46:11.520 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-04-12T20:46:11.520 DEBUG:teuthology.orchestra.run.smithi186:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1cbe407a-f90d-11ee-bc8d-c7b262605968 -- ceph mon dump -f json 2024-04-12T20:46:12.353 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:46:11 smithi018 ceph-mon[28799]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-12T20:46:12.363 INFO:teuthology.orchestra.run.smithi186.stderr:Inferring config /var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/config/ceph.conf 2024-04-12T20:46:13.854 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:46:13 smithi018 ceph-mon[28799]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' 2024-04-12T20:46:13.854 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:46:13 smithi018 ceph-mon[28799]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' 2024-04-12T20:46:13.854 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:46:13 smithi018 ceph-mon[28799]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' 2024-04-12T20:46:13.854 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:46:13 smithi018 ceph-mon[28799]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' 2024-04-12T20:46:13.854 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:46:13 smithi018 ceph-mon[28799]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-12T20:46:13.854 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:46:13 smithi018 ceph-mon[28799]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-12T20:46:13.854 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:46:13 smithi018 ceph-mon[28799]: Deploying daemon mon.smithi186 on smithi186 2024-04-12T20:46:13.854 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:46:13 smithi018 ceph-mon[28799]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-12T20:46:14.595 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:46:14.594+0000 7febc6400700 1 -- 172.21.15.186:0/863630153 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7febc0104af0 msgr2=0x7febc0104ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:46:14.595 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:46:14.594+0000 7febc6400700 1 --2- 172.21.15.186:0/863630153 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7febc0104af0 0x7febc0104ec0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7febb40048c0 tx=0x7febb4009e30 comp rx=0 tx=0).stop 2024-04-12T20:46:14.595 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:46:14.594+0000 7febc6400700 1 -- 172.21.15.186:0/863630153 shutdown_connections 2024-04-12T20:46:14.595 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:46:14.594+0000 7febc6400700 1 --2- 172.21.15.186:0/863630153 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7febc0104af0 0x7febc0104ec0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:46:14.595 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:46:14.594+0000 7febc6400700 1 -- 172.21.15.186:0/863630153 >> 172.21.15.186:0/863630153 conn(0x7febc00fa950 msgr2=0x7febc00fcd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T20:46:14.595 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:46:14.594+0000 7febc6400700 1 -- 172.21.15.186:0/863630153 shutdown_connections 2024-04-12T20:46:14.595 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:46:14.594+0000 7febc6400700 1 -- 172.21.15.186:0/863630153 wait complete. 2024-04-12T20:46:14.596 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:46:14.595+0000 7febc6400700 1 Processor -- start 2024-04-12T20:46:14.596 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:46:14.595+0000 7febc6400700 1 -- start start 2024-04-12T20:46:14.597 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:46:14.595+0000 7febc6400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7febc0104af0 0x7febc0115180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T20:46:14.597 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:46:14.595+0000 7febc6400700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7febc01156c0 con 0x7febc0104af0 2024-04-12T20:46:14.597 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:46:14.596+0000 7febc5000700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7febc0104af0 0x7febc0115180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T20:46:14.598 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:46:14.596+0000 7febc5000700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7febc0104af0 0x7febc0115180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.186:39486/0 (socket says 172.21.15.186:39486) 2024-04-12T20:46:14.598 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:46:14.596+0000 7febc5000700 1 -- 172.21.15.186:0/3657706470 learned_addr learned my addr 172.21.15.186:0/3657706470 (peer_addr_for_me v2:172.21.15.186:0/0) 2024-04-12T20:46:14.598 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:46:14.597+0000 7febc5000700 1 -- 172.21.15.186:0/3657706470 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7febb4005040 con 0x7febc0104af0 2024-04-12T20:46:14.598 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:46:14.597+0000 7febc5000700 1 --2- 172.21.15.186:0/3657706470 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7febc0104af0 0x7febc0115180 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7febb400b820 tx=0x7febb4009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T20:46:14.598 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:46:14.597+0000 7febbcc00700 1 -- 172.21.15.186:0/3657706470 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7febb40093c0 con 0x7febc0104af0 2024-04-12T20:46:14.598 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:46:14.597+0000 7febc6400700 1 -- 172.21.15.186:0/3657706470 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7febc01158c0 con 0x7febc0104af0 2024-04-12T20:46:14.598 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:46:14.598+0000 7febc6400700 1 -- 172.21.15.186:0/3657706470 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7febc0115db0 con 0x7febc0104af0 2024-04-12T20:46:14.598 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:46:14.598+0000 7febbcc00700 1 -- 172.21.15.186:0/3657706470 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7febb4013420 con 0x7febc0104af0 2024-04-12T20:46:14.598 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:46:14.598+0000 7febbcc00700 1 -- 172.21.15.186:0/3657706470 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7febb40255c0 con 0x7febc0104af0 2024-04-12T20:46:14.599 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:46:14.599+0000 7febc6400700 1 -- 172.21.15.186:0/3657706470 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feba00052b0 con 0x7febc0104af0 2024-04-12T20:46:14.599 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:46:14.599+0000 7febbcc00700 1 -- 172.21.15.186:0/3657706470 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 17) v1 ==== 43481+0+0 (secure 0 0 0) 0x7febb4013590 con 0x7febc0104af0 2024-04-12T20:46:14.600 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:46:14.600+0000 7febbcc00700 1 --2- 172.21.15.186:0/3657706470 >> [v2:172.21.15.18:6800/3048894687,v1:172.21.15.18:6801/3048894687] conn(0x7feba80369c0 0x7feba8038e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T20:46:14.600 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:46:14.600+0000 7febbcc00700 1 -- 172.21.15.186:0/3657706470 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1366+0+0 (secure 0 0 0) 0x7febb4049b20 con 0x7febc0104af0 2024-04-12T20:46:14.601 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:46:14.601+0000 7febbfe00700 1 --2- 172.21.15.186:0/3657706470 >> [v2:172.21.15.18:6800/3048894687,v1:172.21.15.18:6801/3048894687] conn(0x7feba80369c0 0x7feba8038e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T20:46:14.601 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:46:14.602+0000 7febbfe00700 1 --2- 172.21.15.186:0/3657706470 >> [v2:172.21.15.18:6800/3048894687,v1:172.21.15.18:6801/3048894687] conn(0x7feba80369c0 0x7feba8038e70 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7febac009990 tx=0x7febac006e40 comp rx=0 tx=0).ready entity=mgr.14212 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T20:46:14.615 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:46:14.615+0000 7febbcc00700 1 -- 172.21.15.186:0/3657706470 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7febb4024dc0 con 0x7febc0104af0 2024-04-12T20:46:14.907 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:46:14.907+0000 7febc6400700 1 -- 172.21.15.186:0/3657706470 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7feba0005560 con 0x7febc0104af0 2024-04-12T20:46:14.909 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:46:14.909+0000 7febbcc00700 1 -- 172.21.15.186:0/3657706470 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+739 (secure 0 0 0) 0x7febb4024360 con 0x7febc0104af0 2024-04-12T20:46:14.909 INFO:teuthology.orchestra.run.smithi186.stdout: 2024-04-12T20:46:14.909 INFO:teuthology.orchestra.run.smithi186.stdout:{"epoch":1,"fsid":"1cbe407a-f90d-11ee-bc8d-c7b262605968","modified":"2024-04-12T20:42:56.111400Z","created":"2024-04-12T20:42:56.111400Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"smithi018","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:3300","nonce":0},{"type":"v1","addr":"172.21.15.18:6789","nonce":0}]},"addr":"172.21.15.18:6789/0","public_addr":"172.21.15.18:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-04-12T20:46:14.912 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:46:14.912+0000 7febc6400700 1 -- 172.21.15.186:0/3657706470 >> [v2:172.21.15.18:6800/3048894687,v1:172.21.15.18:6801/3048894687] conn(0x7feba80369c0 msgr2=0x7feba8038e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:46:14.912 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:46:14.912+0000 7febc6400700 1 --2- 172.21.15.186:0/3657706470 >> [v2:172.21.15.18:6800/3048894687,v1:172.21.15.18:6801/3048894687] conn(0x7feba80369c0 0x7feba8038e70 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7febac009990 tx=0x7febac006e40 comp rx=0 tx=0).stop 2024-04-12T20:46:14.912 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:46:14.912+0000 7febc6400700 1 -- 172.21.15.186:0/3657706470 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7febc0104af0 msgr2=0x7febc0115180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:46:14.912 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:46:14.912+0000 7febc6400700 1 --2- 172.21.15.186:0/3657706470 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7febc0104af0 0x7febc0115180 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7febb400b820 tx=0x7febb4009e30 comp rx=0 tx=0).stop 2024-04-12T20:46:14.912 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:46:14.912+0000 7febc6400700 1 -- 172.21.15.186:0/3657706470 shutdown_connections 2024-04-12T20:46:14.912 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:46:14.912+0000 7febc6400700 1 --2- 172.21.15.186:0/3657706470 >> [v2:172.21.15.18:6800/3048894687,v1:172.21.15.18:6801/3048894687] conn(0x7feba80369c0 0x7feba8038e70 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:46:14.912 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:46:14.912+0000 7febc6400700 1 --2- 172.21.15.186:0/3657706470 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7febc0104af0 0x7febc0115180 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:46:14.912 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:46:14.912+0000 7febc6400700 1 -- 172.21.15.186:0/3657706470 >> 172.21.15.186:0/3657706470 conn(0x7febc00fa950 msgr2=0x7febc00fb130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T20:46:14.913 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:46:14.912+0000 7febc6400700 1 -- 172.21.15.186:0/3657706470 shutdown_connections 2024-04-12T20:46:14.913 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:46:14.912+0000 7febc6400700 1 -- 172.21.15.186:0/3657706470 wait complete. 2024-04-12T20:46:14.913 INFO:teuthology.orchestra.run.smithi186.stderr:dumped monmap epoch 1 2024-04-12T20:46:16.103 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:46:15 smithi018 ceph-mon[28799]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-12T20:46:16.103 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:46:15 smithi018 ceph-mon[28799]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T20:46:16.103 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:46:15 smithi018 ceph-mon[28799]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' 2024-04-12T20:46:16.104 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:46:15 smithi018 ceph-mon[28799]: from='client.? 172.21.15.186:0/3657706470' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-12T20:46:16.913 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-04-12T20:46:16.913 DEBUG:teuthology.orchestra.run.smithi186:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1cbe407a-f90d-11ee-bc8d-c7b262605968 -- ceph mon dump -f json 2024-04-12T20:46:17.414 INFO:teuthology.orchestra.run.smithi186.stderr:Inferring config /var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/mon.smithi186/config 2024-04-12T20:46:18.103 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:46:17 smithi018 ceph-mon[28799]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-12T20:46:18.935 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:46:18 smithi186 podman[36548]: 2024-04-12 20:46:18.830779547 +0000 UTC m=+0.467323504 container create d7263c772b737048a211b0d8729e54a447612bd7a9019ad582ef84c8def87867 (image=quay.ceph.io/ceph-ci/ceph@sha256:5e334df698f432180b78bc9c46936bfdfbc92bc9a8983463e523f780d23765fe, name=ceph-1cbe407a-f90d-11ee-bc8d-c7b262605968-mon-smithi186, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=quincy-d3b5060, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240409, ceph=True) 2024-04-12T20:46:19.461 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:46:19 smithi186 podman[36548]: 2024-04-12 20:46:19.269267251 +0000 UTC m=+0.905811214 container init d7263c772b737048a211b0d8729e54a447612bd7a9019ad582ef84c8def87867 (image=quay.ceph.io/ceph-ci/ceph@sha256:5e334df698f432180b78bc9c46936bfdfbc92bc9a8983463e523f780d23765fe, name=ceph-1cbe407a-f90d-11ee-bc8d-c7b262605968-mon-smithi186, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, RELEASE=quincy-d3b5060, GIT_BRANCH=HEAD, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_CLEAN=True, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240409) 2024-04-12T20:46:19.462 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:46:19 smithi186 podman[36548]: 2024-04-12 20:46:19.278570991 +0000 UTC m=+0.915114957 container start d7263c772b737048a211b0d8729e54a447612bd7a9019ad582ef84c8def87867 (image=quay.ceph.io/ceph-ci/ceph@sha256:5e334df698f432180b78bc9c46936bfdfbc92bc9a8983463e523f780d23765fe, name=ceph-1cbe407a-f90d-11ee-bc8d-c7b262605968-mon-smithi186, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240409, io.buildah.version=1.33.5, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, ceph=True, GIT_CLEAN=True, org.label-schema.license=GPLv2, RELEASE=quincy-d3b5060, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD) 2024-04-12T20:46:19.462 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:46:19 smithi186 ceph-mon[36597]: set uid:gid to 167:167 (ceph:ceph) 2024-04-12T20:46:19.462 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:46:19 smithi186 ceph-mon[36597]: ceph version 17.2.7-827-gd3b5060a (d3b5060a17449edfae9385cdf792afec109fca5d) quincy (stable), process ceph-mon, pid 2 2024-04-12T20:46:19.462 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:46:19 smithi186 ceph-mon[36597]: pidfile_write: ignore empty --pid-file 2024-04-12T20:46:19.462 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:46:19 smithi186 ceph-mon[36597]: load: jerasure load: lrc 2024-04-12T20:46:19.462 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:46:19 smithi186 ceph-mon[36597]: rocksdb: RocksDB version: 6.15.5 2024-04-12T20:46:19.462 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:46:19 smithi186 ceph-mon[36597]: rocksdb: Git sha rocksdb_build_git_sha:@0@ 2024-04-12T20:46:19.462 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:46:19 smithi186 ceph-mon[36597]: rocksdb: Compile date Apr 9 2024 2024-04-12T20:46:19.462 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:46:19 smithi186 ceph-mon[36597]: rocksdb: DB SUMMARY 2024-04-12T20:46:19.462 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:46:19 smithi186 ceph-mon[36597]: rocksdb: DB Session ID: M21MUUTFTQIVNSJS34XV 2024-04-12T20:46:19.462 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:46:19 smithi186 ceph-mon[36597]: rocksdb: CURRENT file: CURRENT 2024-04-12T20:46:19.462 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:46:19 smithi186 ceph-mon[36597]: rocksdb: IDENTITY file: IDENTITY 2024-04-12T20:46:19.462 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:46:19 smithi186 ceph-mon[36597]: rocksdb: MANIFEST file: MANIFEST-000003 size: 57 Bytes 2024-04-12T20:46:19.463 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:46:19 smithi186 ceph-mon[36597]: rocksdb: SST files in /var/lib/ceph/mon/ceph-smithi186/store.db dir, Total Num: 0, files: 2024-04-12T20:46:19.463 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:46:19 smithi186 ceph-mon[36597]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-smithi186/store.db: 000004.log size: 511 ; 2024-04-12T20:46:19.463 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:46:19 smithi186 ceph-mon[36597]: rocksdb: Options.error_if_exists: 0 2024-04-12T20:46:19.463 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:46:19 smithi186 ceph-mon[36597]: rocksdb: Options.create_if_missing: 0 2024-04-12T20:46:19.463 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:46:19 smithi186 ceph-mon[36597]: rocksdb: Options.paranoid_checks: 1 2024-04-12T20:46:19.463 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:46:19 smithi186 ceph-mon[36597]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-04-12T20:46:19.463 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:46:19 smithi186 ceph-mon[36597]: rocksdb: Options.env: 0x55b2b9cc4a80 2024-04-12T20:46:19.463 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:46:19 smithi186 ceph-mon[36597]: rocksdb: Options.fs: Posix File System 2024-04-12T20:46:19.463 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:46:19 smithi186 ceph-mon[36597]: rocksdb: Options.info_log: 0x55b2c538fc00 2024-04-12T20:46:19.463 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:46:19 smithi186 ceph-mon[36597]: rocksdb: Options.max_file_opening_threads: 16 2024-04-12T20:46:19.463 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:46:19 smithi186 ceph-mon[36597]: rocksdb: Options.statistics: (nil) 2024-04-12T20:46:19.463 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:46:19 smithi186 ceph-mon[36597]: rocksdb: Options.use_fsync: 0 2024-04-12T20:46:19.464 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:46:19 smithi186 ceph-mon[36597]: rocksdb: Options.max_log_file_size: 0 2024-04-12T20:46:19.464 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:46:19 smithi186 ceph-mon[36597]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-04-12T20:46:19.464 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:46:19 smithi186 ceph-mon[36597]: rocksdb: Options.log_file_time_to_roll: 0 2024-04-12T20:46:19.464 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:46:19 smithi186 ceph-mon[36597]: rocksdb: Options.keep_log_file_num: 1000 2024-04-12T20:46:19.464 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:46:19 smithi186 ceph-mon[36597]: rocksdb: Options.recycle_log_file_num: 0 2024-04-12T20:46:19.464 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:46:19 smithi186 ceph-mon[36597]: rocksdb: Options.allow_fallocate: 1 2024-04-12T20:46:19.464 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:46:19 smithi186 ceph-mon[36597]: rocksdb: Options.allow_mmap_reads: 0 2024-04-12T20:46:19.464 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:46:19 smithi186 ceph-mon[36597]: rocksdb: Options.allow_mmap_writes: 0 2024-04-12T20:46:19.465 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:46:19 smithi186 ceph-mon[36597]: rocksdb: Options.use_direct_reads: 0 2024-04-12T20:46:19.465 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:46:19 smithi186 ceph-mon[36597]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-04-12T20:46:19.465 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:46:19 smithi186 ceph-mon[36597]: rocksdb: Options.create_missing_column_families: 0 2024-04-12T20:46:19.465 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:46:19 smithi186 ceph-mon[36597]: rocksdb: Options.db_log_dir: 2024-04-12T20:46:19.465 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:46:19 smithi186 ceph-mon[36597]: rocksdb: Options.wal_dir: /var/lib/ceph/mon/ceph-smithi186/store.db 2024-04-12T20:46:19.465 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:46:19 smithi186 ceph-mon[36597]: rocksdb: Options.table_cache_numshardbits: 6 2024-04-12T20:46:19.465 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:46:19 smithi186 ceph-mon[36597]: rocksdb: Options.WAL_ttl_seconds: 0 2024-04-12T20:46:19.465 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:46:19 smithi186 ceph-mon[36597]: rocksdb: Options.WAL_size_limit_MB: 0 2024-04-12T20:46:19.465 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:46:19 smithi186 ceph-mon[36597]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-04-12T20:46:19.465 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:46:19 smithi186 ceph-mon[36597]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-04-12T20:46:19.465 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:46:19 smithi186 ceph-mon[36597]: rocksdb: Options.is_fd_close_on_exec: 1 2024-04-12T20:46:19.465 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:46:19 smithi186 ceph-mon[36597]: rocksdb: Options.advise_random_on_open: 1 2024-04-12T20:46:19.465 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:46:19 smithi186 ceph-mon[36597]: rocksdb: Options.db_write_buffer_size: 0 2024-04-12T20:46:19.466 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:46:19 smithi186 ceph-mon[36597]: rocksdb: Options.write_buffer_manager: 0x55b2c61049f0 2024-04-12T20:46:19.466 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:46:19 smithi186 ceph-mon[36597]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-04-12T20:46:19.466 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:46:19 smithi186 ceph-mon[36597]: rocksdb: Options.new_table_reader_for_compaction_inputs: 0 2024-04-12T20:46:19.466 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:46:19 smithi186 ceph-mon[36597]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-04-12T20:46:19.466 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:46:19 smithi186 ceph-mon[36597]: rocksdb: Options.use_adaptive_mutex: 0 2024-04-12T20:46:19.466 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:46:19 smithi186 ceph-mon[36597]: rocksdb: Options.rate_limiter: (nil) 2024-04-12T20:46:19.466 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:46:19 smithi186 ceph-mon[36597]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-04-12T20:46:19.466 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:46:19 smithi186 ceph-mon[36597]: rocksdb: Options.wal_recovery_mode: 2 2024-04-12T20:46:19.466 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:46:19 smithi186 ceph-mon[36597]: rocksdb: Options.enable_thread_tracking: 0 2024-04-12T20:46:19.466 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:46:19 smithi186 ceph-mon[36597]: rocksdb: Options.enable_pipelined_write: 0 2024-04-12T20:46:19.466 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:46:19 smithi186 ceph-mon[36597]: rocksdb: Options.unordered_write: 0 2024-04-12T20:46:19.466 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:46:19 smithi186 ceph-mon[36597]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-04-12T20:46:19.466 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:46:19 smithi186 ceph-mon[36597]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-04-12T20:46:19.466 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:46:19 smithi186 ceph-mon[36597]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-04-12T20:46:19.467 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:46:19 smithi186 ceph-mon[36597]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-04-12T20:46:19.467 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:46:19 smithi186 ceph-mon[36597]: rocksdb: Options.row_cache: None 2024-04-12T20:46:19.467 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:46:19 smithi186 ceph-mon[36597]: rocksdb: Options.wal_filter: None 2024-04-12T20:46:19.467 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:46:19 smithi186 ceph-mon[36597]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-04-12T20:46:19.467 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:46:19 smithi186 ceph-mon[36597]: rocksdb: Options.allow_ingest_behind: 0 2024-04-12T20:46:19.467 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:46:19 smithi186 ceph-mon[36597]: rocksdb: Options.preserve_deletes: 0 2024-04-12T20:46:19.467 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:46:19 smithi186 ceph-mon[36597]: rocksdb: Options.two_write_queues: 0 2024-04-12T20:46:19.467 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:46:19 smithi186 ceph-mon[36597]: rocksdb: Options.manual_wal_flush: 0 2024-04-12T20:46:19.467 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:46:19 smithi186 ceph-mon[36597]: rocksdb: Options.atomic_flush: 0 2024-04-12T20:46:19.467 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:46:19 smithi186 ceph-mon[36597]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-04-12T20:46:19.467 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:46:19 smithi186 ceph-mon[36597]: rocksdb: Options.persist_stats_to_disk: 0 2024-04-12T20:46:19.467 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:46:19 smithi186 ceph-mon[36597]: rocksdb: Options.write_dbid_to_manifest: 0 2024-04-12T20:46:19.467 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:46:19 smithi186 ceph-mon[36597]: rocksdb: Options.log_readahead_size: 0 2024-04-12T20:46:19.467 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:46:19 smithi186 ceph-mon[36597]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-04-12T20:46:19.467 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:46:19 smithi186 ceph-mon[36597]: rocksdb: Options.best_efforts_recovery: 0 2024-04-12T20:46:19.468 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:46:19 smithi186 ceph-mon[36597]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-04-12T20:46:19.468 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:46:19 smithi186 ceph-mon[36597]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-04-12T20:46:19.468 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:46:19 smithi186 ceph-mon[36597]: rocksdb: Options.allow_data_in_errors: 0 2024-04-12T20:46:19.468 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:46:19 smithi186 ceph-mon[36597]: rocksdb: Options.db_host_id: __hostname__ 2024-04-12T20:46:19.468 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:46:19 smithi186 ceph-mon[36597]: rocksdb: Options.max_background_jobs: 2 2024-04-12T20:46:19.468 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:46:19 smithi186 ceph-mon[36597]: rocksdb: Options.max_background_compactions: -1 2024-04-12T20:46:19.468 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:46:19 smithi186 ceph-mon[36597]: rocksdb: Options.max_subcompactions: 1 2024-04-12T20:46:19.468 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:46:19 smithi186 ceph-mon[36597]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-04-12T20:46:19.468 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:46:19 smithi186 ceph-mon[36597]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-04-12T20:46:19.468 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:46:19 smithi186 ceph-mon[36597]: rocksdb: Options.delayed_write_rate : 16777216 2024-04-12T20:46:19.468 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:46:19 smithi186 ceph-mon[36597]: rocksdb: Options.max_total_wal_size: 0 2024-04-12T20:46:19.468 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:46:19 smithi186 ceph-mon[36597]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-04-12T20:46:19.468 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:46:19 smithi186 ceph-mon[36597]: rocksdb: Options.stats_dump_period_sec: 600 2024-04-12T20:46:19.468 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:46:19 smithi186 ceph-mon[36597]: rocksdb: Options.stats_persist_period_sec: 600 2024-04-12T20:46:19.468 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:46:19 smithi186 ceph-mon[36597]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-04-12T20:46:19.469 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:46:19 smithi186 ceph-mon[36597]: rocksdb: Options.max_open_files: -1 2024-04-12T20:46:19.469 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:46:19 smithi186 ceph-mon[36597]: rocksdb: Options.bytes_per_sync: 0 2024-04-12T20:46:19.469 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:46:19 smithi186 ceph-mon[36597]: rocksdb: Options.wal_bytes_per_sync: 0 2024-04-12T20:46:19.469 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:46:19 smithi186 ceph-mon[36597]: rocksdb: Options.strict_bytes_per_sync: 0 2024-04-12T20:46:19.469 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:46:19 smithi186 ceph-mon[36597]: rocksdb: Options.compaction_readahead_size: 0 2024-04-12T20:46:19.469 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:46:19 smithi186 ceph-mon[36597]: rocksdb: Options.max_background_flushes: -1 2024-04-12T20:46:19.469 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:46:19 smithi186 ceph-mon[36597]: rocksdb: Compression algorithms supported: 2024-04-12T20:46:19.469 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:46:19 smithi186 ceph-mon[36597]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-04-12T20:46:19.469 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:46:19 smithi186 ceph-mon[36597]: rocksdb: kZSTD supported: 0 2024-04-12T20:46:19.469 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:46:19 smithi186 ceph-mon[36597]: rocksdb: kXpressCompression supported: 0 2024-04-12T20:46:19.469 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:46:19 smithi186 ceph-mon[36597]: rocksdb: kLZ4HCCompression supported: 1 2024-04-12T20:46:19.469 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:46:19 smithi186 ceph-mon[36597]: rocksdb: kZlibCompression supported: 1 2024-04-12T20:46:19.469 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:46:19 smithi186 ceph-mon[36597]: rocksdb: kSnappyCompression supported: 1 2024-04-12T20:46:19.469 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:46:19 smithi186 ceph-mon[36597]: rocksdb: kLZ4Compression supported: 1 2024-04-12T20:46:19.470 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:46:19 smithi186 ceph-mon[36597]: rocksdb: kBZip2Compression supported: 0 2024-04-12T20:46:19.471 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:46:19 smithi186 ceph-mon[36597]: rocksdb: Fast CRC32 supported: Supported on x86 2024-04-12T20:46:19.471 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:46:19 smithi186 ceph-mon[36597]: rocksdb: [db/version_set.cc:4724] Recovering from manifest file: /var/lib/ceph/mon/ceph-smithi186/store.db/MANIFEST-000003 2024-04-12T20:46:19.471 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:46:19 smithi186 ceph-mon[36597]: rocksdb: [db/column_family.cc:595] --------------- Options for column family [default]: 2024-04-12T20:46:19.471 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:46:19 smithi186 ceph-mon[36597]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-04-12T20:46:19.471 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:46:19 smithi186 ceph-mon[36597]: rocksdb: Options.merge_operator: 2024-04-12T20:46:19.471 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:46:19 smithi186 ceph-mon[36597]: rocksdb: Options.compaction_filter: None 2024-04-12T20:46:19.471 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:46:19 smithi186 ceph-mon[36597]: rocksdb: Options.compaction_filter_factory: None 2024-04-12T20:46:19.471 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:46:19 smithi186 ceph-mon[36597]: rocksdb: Options.sst_partitioner_factory: None 2024-04-12T20:46:19.472 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:46:19 smithi186 ceph-mon[36597]: rocksdb: Options.memtable_factory: SkipListFactory 2024-04-12T20:46:19.472 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:46:19 smithi186 ceph-mon[36597]: rocksdb: Options.table_factory: BlockBasedTable 2024-04-12T20:46:19.472 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:46:19 smithi186 ceph-mon[36597]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x55b2c53e4f58) 2024-04-12T20:46:19.472 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout: cache_index_and_filter_blocks: 1 2024-04-12T20:46:19.472 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-04-12T20:46:19.472 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-04-12T20:46:19.472 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout: pin_top_level_index_and_filter: 1 2024-04-12T20:46:19.472 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout: index_type: 0 2024-04-12T20:46:19.472 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout: data_block_index_type: 0 2024-04-12T20:46:19.472 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout: index_shortening: 1 2024-04-12T20:46:19.472 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout: data_block_hash_table_util_ratio: 0.750000 2024-04-12T20:46:19.472 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout: hash_index_allow_collision: 1 2024-04-12T20:46:19.472 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout: checksum: 1 2024-04-12T20:46:19.472 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout: no_block_cache: 0 2024-04-12T20:46:19.473 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout: block_cache: 0x55b2c5417090 2024-04-12T20:46:19.473 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout: block_cache_name: BinnedLRUCache 2024-04-12T20:46:19.473 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout: block_cache_options: 2024-04-12T20:46:19.473 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout: capacity : 536870912 2024-04-12T20:46:19.473 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout: num_shard_bits : 4 2024-04-12T20:46:19.473 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout: strict_capacity_limit : 0 2024-04-12T20:46:19.473 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout: high_pri_pool_ratio: 0.000 2024-04-12T20:46:19.473 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout: block_cache_compressed: (nil) 2024-04-12T20:46:19.473 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout: persistent_cache: (nil) 2024-04-12T20:46:19.473 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout: block_size: 4096 2024-04-12T20:46:19.473 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout: block_size_deviation: 10 2024-04-12T20:46:19.473 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout: block_restart_interval: 16 2024-04-12T20:46:19.473 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout: index_block_restart_interval: 1 2024-04-12T20:46:19.473 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout: metadata_block_size: 4096 2024-04-12T20:46:19.473 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout: partition_filters: 0 2024-04-12T20:46:19.474 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout: use_delta_encoding: 1 2024-04-12T20:46:19.474 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout: filter_policy: rocksdb.BuiltinBloomFilter 2024-04-12T20:46:19.474 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout: whole_key_filtering: 1 2024-04-12T20:46:19.474 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout: verify_compression: 0 2024-04-12T20:46:19.474 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout: read_amp_bytes_per_bit: 0 2024-04-12T20:46:19.474 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout: format_version: 4 2024-04-12T20:46:19.474 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout: enable_index_compression: 1 2024-04-12T20:46:19.474 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout: block_align: 0 2024-04-12T20:46:19.474 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:46:19 smithi186 ceph-mon[36597]: rocksdb: Options.write_buffer_size: 33554432 2024-04-12T20:46:19.474 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:46:19 smithi186 ceph-mon[36597]: rocksdb: Options.max_write_buffer_number: 2 2024-04-12T20:46:19.474 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:46:19 smithi186 ceph-mon[36597]: rocksdb: Options.compression: NoCompression 2024-04-12T20:46:19.474 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:46:19 smithi186 ceph-mon[36597]: rocksdb: Options.bottommost_compression: Disabled 2024-04-12T20:46:19.474 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:46:19 smithi186 ceph-mon[36597]: rocksdb: Options.prefix_extractor: nullptr 2024-04-12T20:46:19.474 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:46:19 smithi186 ceph-mon[36597]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-04-12T20:46:19.474 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:46:19 smithi186 ceph-mon[36597]: rocksdb: Options.num_levels: 7 2024-04-12T20:46:19.475 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:46:19 smithi186 ceph-mon[36597]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-04-12T20:46:19.475 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:46:19 smithi186 ceph-mon[36597]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-04-12T20:46:19.475 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:46:19 smithi186 ceph-mon[36597]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-04-12T20:46:19.475 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:46:19 smithi186 ceph-mon[36597]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-04-12T20:46:19.475 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:46:19 smithi186 ceph-mon[36597]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-04-12T20:46:19.475 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:46:19 smithi186 ceph-mon[36597]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-04-12T20:46:19.475 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:46:19 smithi186 ceph-mon[36597]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-04-12T20:46:19.475 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:46:19 smithi186 ceph-mon[36597]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-04-12T20:46:19.475 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:46:19 smithi186 ceph-mon[36597]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-04-12T20:46:19.475 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:46:19 smithi186 ceph-mon[36597]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-04-12T20:46:19.475 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:46:19 smithi186 ceph-mon[36597]: rocksdb: Options.compression_opts.window_bits: -14 2024-04-12T20:46:19.475 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:46:19 smithi186 ceph-mon[36597]: rocksdb: Options.compression_opts.level: 32767 2024-04-12T20:46:19.475 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:46:19 smithi186 ceph-mon[36597]: rocksdb: Options.compression_opts.strategy: 0 2024-04-12T20:46:19.475 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:46:19 smithi186 ceph-mon[36597]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-04-12T20:46:19.475 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:46:19 smithi186 ceph-mon[36597]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-04-12T20:46:19.476 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:46:19 smithi186 ceph-mon[36597]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-04-12T20:46:19.476 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:46:19 smithi186 ceph-mon[36597]: rocksdb: Options.compression_opts.enabled: false 2024-04-12T20:46:19.476 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:46:19 smithi186 ceph-mon[36597]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-04-12T20:46:19.476 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:46:19 smithi186 ceph-mon[36597]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-04-12T20:46:19.476 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:46:19 smithi186 ceph-mon[36597]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-04-12T20:46:19.476 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:46:19 smithi186 ceph-mon[36597]: rocksdb: Options.target_file_size_base: 67108864 2024-04-12T20:46:19.476 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:46:19 smithi186 ceph-mon[36597]: rocksdb: Options.target_file_size_multiplier: 1 2024-04-12T20:46:19.476 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:46:19 smithi186 ceph-mon[36597]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-04-12T20:46:19.476 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:46:19 smithi186 ceph-mon[36597]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-04-12T20:46:19.476 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:46:19 smithi186 ceph-mon[36597]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-04-12T20:46:19.476 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:46:19 smithi186 ceph-mon[36597]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-04-12T20:46:19.476 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:46:19 smithi186 ceph-mon[36597]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-04-12T20:46:19.476 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:46:19 smithi186 ceph-mon[36597]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-04-12T20:46:19.476 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:46:19 smithi186 ceph-mon[36597]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-04-12T20:46:19.476 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:46:19 smithi186 ceph-mon[36597]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-04-12T20:46:19.477 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:46:19 smithi186 ceph-mon[36597]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-04-12T20:46:19.477 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:46:19 smithi186 ceph-mon[36597]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-04-12T20:46:19.477 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:46:19 smithi186 ceph-mon[36597]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-04-12T20:46:19.477 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:46:19 smithi186 ceph-mon[36597]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-04-12T20:46:19.477 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:46:19 smithi186 ceph-mon[36597]: rocksdb: Options.arena_block_size: 4194304 2024-04-12T20:46:19.477 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:46:19 smithi186 ceph-mon[36597]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-04-12T20:46:19.477 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:46:19 smithi186 ceph-mon[36597]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-04-12T20:46:19.477 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:46:19 smithi186 ceph-mon[36597]: rocksdb: Options.rate_limit_delay_max_milliseconds: 100 2024-04-12T20:46:19.477 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:46:19 smithi186 ceph-mon[36597]: rocksdb: Options.disable_auto_compactions: 0 2024-04-12T20:46:19.477 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:46:19 smithi186 ceph-mon[36597]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-04-12T20:46:19.477 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:46:19 smithi186 ceph-mon[36597]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-04-12T20:46:19.477 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:46:19 smithi186 ceph-mon[36597]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-04-12T20:46:19.477 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:46:19 smithi186 ceph-mon[36597]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-04-12T20:46:19.477 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:46:19 smithi186 ceph-mon[36597]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-04-12T20:46:19.478 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:46:19 smithi186 ceph-mon[36597]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-04-12T20:46:19.478 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:46:19 smithi186 ceph-mon[36597]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-04-12T20:46:19.478 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:46:19 smithi186 ceph-mon[36597]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-04-12T20:46:19.478 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:46:19 smithi186 ceph-mon[36597]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-04-12T20:46:19.478 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:46:19 smithi186 ceph-mon[36597]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-04-12T20:46:19.478 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:46:19 smithi186 ceph-mon[36597]: rocksdb: Options.table_properties_collectors: 2024-04-12T20:46:19.478 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:46:19 smithi186 ceph-mon[36597]: rocksdb: Options.inplace_update_support: 0 2024-04-12T20:46:19.478 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:46:19 smithi186 ceph-mon[36597]: rocksdb: Options.inplace_update_num_locks: 10000 2024-04-12T20:46:19.478 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:46:19 smithi186 ceph-mon[36597]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-04-12T20:46:19.478 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:46:19 smithi186 ceph-mon[36597]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-04-12T20:46:19.478 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:46:19 smithi186 ceph-mon[36597]: rocksdb: Options.memtable_huge_page_size: 0 2024-04-12T20:46:19.478 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:46:19 smithi186 ceph-mon[36597]: rocksdb: Options.bloom_locality: 0 2024-04-12T20:46:19.478 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:46:19 smithi186 ceph-mon[36597]: rocksdb: Options.max_successive_merges: 0 2024-04-12T20:46:19.478 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:46:19 smithi186 ceph-mon[36597]: rocksdb: Options.optimize_filters_for_hits: 0 2024-04-12T20:46:19.479 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:46:19 smithi186 ceph-mon[36597]: rocksdb: Options.paranoid_file_checks: 0 2024-04-12T20:46:19.479 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:46:19 smithi186 ceph-mon[36597]: rocksdb: Options.force_consistency_checks: 1 2024-04-12T20:46:19.479 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:46:19 smithi186 ceph-mon[36597]: rocksdb: Options.report_bg_io_stats: 0 2024-04-12T20:46:19.479 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:46:19 smithi186 ceph-mon[36597]: rocksdb: Options.ttl: 2592000 2024-04-12T20:46:19.479 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:46:19 smithi186 ceph-mon[36597]: rocksdb: Options.periodic_compaction_seconds: 0 2024-04-12T20:46:19.479 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:46:19 smithi186 ceph-mon[36597]: rocksdb: Options.enable_blob_files: false 2024-04-12T20:46:19.479 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:46:19 smithi186 ceph-mon[36597]: rocksdb: Options.min_blob_size: 0 2024-04-12T20:46:19.479 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:46:19 smithi186 ceph-mon[36597]: rocksdb: Options.blob_file_size: 268435456 2024-04-12T20:46:19.479 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:46:19 smithi186 ceph-mon[36597]: rocksdb: Options.blob_compression_type: NoCompression 2024-04-12T20:46:19.479 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:46:19 smithi186 ceph-mon[36597]: rocksdb: Options.enable_blob_garbage_collection: false 2024-04-12T20:46:19.479 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:46:19 smithi186 ceph-mon[36597]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-04-12T20:46:19.479 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:46:19 smithi186 ceph-mon[36597]: rocksdb: [db/version_set.cc:4764] Recovered from manifest file:/var/lib/ceph/mon/ceph-smithi186/store.db/MANIFEST-000003 succeeded,manifest_file_number is 3, next_file_number is 5, last_sequence is 0, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-04-12T20:46:19.479 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:46:19 smithi186 ceph-mon[36597]: rocksdb: [db/version_set.cc:4779] Column family [default] (ID 0), log number is 0 2024-04-12T20:46:19.479 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:46:19 smithi186 ceph-mon[36597]: rocksdb: [db/version_set.cc:4082] Creating manifest 7 2024-04-12T20:46:19.479 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:46:19 smithi186 ceph-mon[36597]: rocksdb: EVENT_LOG_v1 {"time_micros": 1712954779327487, "job": 1, "event": "recovery_started", "wal_files": [4]} 2024-04-12T20:46:19.480 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:46:19 smithi186 ceph-mon[36597]: rocksdb: [db/db_impl/db_impl_open.cc:845] Recovering log #4 mode 2 2024-04-12T20:46:19.480 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:46:19 smithi186 ceph-mon[36597]: rocksdb: [table/block_based/filter_policy.cc:991] Using legacy Bloom filter with high (20) bits/key. Dramatic filter space and/or accuracy improvement is available with format_version>=5. 2024-04-12T20:46:19.480 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:46:19 smithi186 ceph-mon[36597]: rocksdb: EVENT_LOG_v1 {"time_micros": 1712954779328206, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 8, "file_size": 1545, "file_checksum": "", "file_checksum_func_name": "Unknown", "table_properties": {"data_size": 523, "index_size": 31, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 69, "raw_key_size": 115, "raw_average_key_size": 23, "raw_value_size": 401, "raw_average_value_size": 80, "num_data_blocks": 1, "num_entries": 5, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "rocksdb.BuiltinBloomFilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; ", "creation_time": 1712954779, "oldest_key_time": 0, "file_creation_time": 0, "db_id": "34cdea1e-5e97-499c-bcd6-6ed0ef5f8b73", "db_session_id": "M21MUUTFTQIVNSJS34XV"}} 2024-04-12T20:46:19.480 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:46:19 smithi186 ceph-mon[36597]: rocksdb: [db/version_set.cc:4082] Creating manifest 9 2024-04-12T20:46:19.480 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:46:19 smithi186 ceph-mon[36597]: rocksdb: EVENT_LOG_v1 {"time_micros": 1712954779329059, "job": 1, "event": "recovery_finished"} 2024-04-12T20:46:19.480 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:46:19 smithi186 ceph-mon[36597]: rocksdb: [file/delete_scheduler.cc:69] Deleted file /var/lib/ceph/mon/ceph-smithi186/store.db/000004.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-04-12T20:46:19.480 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:46:19 smithi186 ceph-mon[36597]: rocksdb: [db/db_impl/db_impl_open.cc:1700] SstFileManager instance 0x55b2c53ab180 2024-04-12T20:46:19.480 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:46:19 smithi186 ceph-mon[36597]: rocksdb: DB pointer 0x55b2c5480000 2024-04-12T20:46:19.480 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:46:19 smithi186 ceph-mon[36597]: rocksdb: [db/db_impl/db_impl.cc:901] ------- DUMPING STATS ------- 2024-04-12T20:46:19.480 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:46:19 smithi186 ceph-mon[36597]: rocksdb: [db/db_impl/db_impl.cc:903] 2024-04-12T20:46:19.480 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout: ** DB Stats ** 2024-04-12T20:46:19.480 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-04-12T20:46:19.480 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-04-12T20:46:19.480 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-04-12T20:46:19.480 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-04-12T20:46:19.481 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-04-12T20:46:19.481 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 MB, 0.00 MB/s 2024-04-12T20:46:19.481 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-04-12T20:46:19.481 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout: 2024-04-12T20:46:19.481 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout: ** Compaction Stats [default] ** 2024-04-12T20:46:19.481 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2024-04-12T20:46:19.481 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout: ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-04-12T20:46:19.481 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout: L0 1/0 1.51 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 2.1 0.00 0.00 1 0.001 0 0 2024-04-12T20:46:19.481 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout: Sum 1/0 1.51 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 2.1 0.00 0.00 1 0.001 0 0 2024-04-12T20:46:19.481 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 2.1 0.00 0.00 1 0.001 0 0 2024-04-12T20:46:19.481 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout: 2024-04-12T20:46:19.481 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout: ** Compaction Stats [default] ** 2024-04-12T20:46:19.481 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2024-04-12T20:46:19.481 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-04-12T20:46:19.482 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 2.1 0.00 0.00 1 0.001 0 0 2024-04-12T20:46:19.482 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-04-12T20:46:19.482 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-04-12T20:46:19.482 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-04-12T20:46:19.482 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-04-12T20:46:19.482 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-04-12T20:46:19.482 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout: AddFile(Keys): cumulative 0, interval 0 2024-04-12T20:46:19.482 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout: Cumulative compaction: 0.00 GB write, 0.22 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-04-12T20:46:19.482 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout: Interval compaction: 0.00 GB write, 0.22 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-04-12T20:46:19.482 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-04-12T20:46:19.482 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout: 2024-04-12T20:46:19.482 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout: ** File Read Latency Histogram By Level [default] ** 2024-04-12T20:46:19.482 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout: 2024-04-12T20:46:19.482 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout: ** Compaction Stats [default] ** 2024-04-12T20:46:19.483 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2024-04-12T20:46:19.483 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout: ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-04-12T20:46:19.483 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout: L0 1/0 1.51 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 2.1 0.00 0.00 1 0.001 0 0 2024-04-12T20:46:19.483 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout: Sum 1/0 1.51 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 2.1 0.00 0.00 1 0.001 0 0 2024-04-12T20:46:19.483 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 2024-04-12T20:46:19.483 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout: 2024-04-12T20:46:19.483 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout: ** Compaction Stats [default] ** 2024-04-12T20:46:19.483 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2024-04-12T20:46:19.483 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-04-12T20:46:19.483 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 2.1 0.00 0.00 1 0.001 0 0 2024-04-12T20:46:19.483 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-04-12T20:46:19.483 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-04-12T20:46:19.483 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-04-12T20:46:19.483 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-04-12T20:46:19.484 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-04-12T20:46:19.484 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout: AddFile(Keys): cumulative 0, interval 0 2024-04-12T20:46:19.484 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout: Cumulative compaction: 0.00 GB write, 0.22 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-04-12T20:46:19.484 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout: Interval compaction: 0.00 GB write, 0.00 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-04-12T20:46:19.484 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-04-12T20:46:19.484 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout: 2024-04-12T20:46:19.484 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout: ** File Read Latency Histogram By Level [default] ** 2024-04-12T20:46:19.484 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:46:19 smithi186 ceph-mon[36597]: mon.smithi186 does not exist in monmap, will attempt to join an existing cluster 2024-04-12T20:46:19.484 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:46:19 smithi186 ceph-mon[36597]: using public_addr v2:172.21.15.186:0/0 -> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] 2024-04-12T20:46:19.484 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:46:19 smithi186 ceph-mon[36597]: starting mon.smithi186 rank -1 at public addrs [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] at bind addrs [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] mon_data /var/lib/ceph/mon/ceph-smithi186 fsid 1cbe407a-f90d-11ee-bc8d-c7b262605968 2024-04-12T20:46:19.484 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:46:19 smithi186 ceph-mon[36597]: mon.smithi186@-1(???) e0 preinit fsid 1cbe407a-f90d-11ee-bc8d-c7b262605968 2024-04-12T20:46:19.484 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:46:19 smithi186 ceph-mon[36597]: mon.smithi186@-1(synchronizing).mds e1 new map 2024-04-12T20:46:19.484 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:46:19 smithi186 ceph-mon[36597]: mon.smithi186@-1(synchronizing).mds e1 print_map 2024-04-12T20:46:19.484 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout: e1 2024-04-12T20:46:19.484 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-04-12T20:46:19.485 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-12T20:46:19.485 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout: legacy client fscid: -1 2024-04-12T20:46:19.485 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout: 2024-04-12T20:46:19.485 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout: No filesystems configured 2024-04-12T20:46:19.485 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:46:19 smithi186 ceph-mon[36597]: mon.smithi186@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2024-04-12T20:46:19.485 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:46:19 smithi186 ceph-mon[36597]: mon.smithi186@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2024-04-12T20:46:19.485 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:46:19 smithi186 ceph-mon[36597]: mon.smithi186@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2024-04-12T20:46:19.485 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:46:19 smithi186 ceph-mon[36597]: mon.smithi186@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2024-04-12T20:46:19.485 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:46:19 smithi186 ceph-mon[36597]: mon.smithi186@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2024-04-12T20:46:19.485 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:46:19 smithi186 ceph-mon[36597]: mon.smithi186@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2024-04-12T20:46:19.485 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:46:19 smithi186 ceph-mon[36597]: mon.smithi186@-1(synchronizing).osd e5 e5: 0 total, 0 up, 0 in 2024-04-12T20:46:19.485 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:46:19 smithi186 ceph-mon[36597]: mon.smithi186@-1(synchronizing).osd e5 crush map has features 3314932999778484224, adjusting msgr requires 2024-04-12T20:46:19.485 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:46:19 smithi186 ceph-mon[36597]: mon.smithi186@-1(synchronizing).osd e5 crush map has features 288514050185494528, adjusting msgr requires 2024-04-12T20:46:19.485 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:46:19 smithi186 ceph-mon[36597]: mon.smithi186@-1(synchronizing).osd e5 crush map has features 288514050185494528, adjusting msgr requires 2024-04-12T20:46:19.486 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:46:19 smithi186 ceph-mon[36597]: mon.smithi186@-1(synchronizing).osd e5 crush map has features 288514050185494528, adjusting msgr requires 2024-04-12T20:46:19.486 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:46:19 smithi186 ceph-mon[36597]: Updating smithi186:/var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/config/ceph.conf 2024-04-12T20:46:19.486 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:46:19 smithi186 ceph-mon[36597]: Updating smithi018:/var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/config/ceph.conf 2024-04-12T20:46:19.486 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:46:19 smithi186 ceph-mon[36597]: Updating smithi186:/etc/ceph/ceph.client.admin.keyring 2024-04-12T20:46:19.486 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:46:19 smithi186 ceph-mon[36597]: Updating smithi018:/etc/ceph/ceph.client.admin.keyring 2024-04-12T20:46:19.486 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:46:19 smithi186 ceph-mon[36597]: Updating smithi186:/var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/config/ceph.client.admin.keyring 2024-04-12T20:46:19.486 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:46:19 smithi186 ceph-mon[36597]: Updating smithi018:/var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/config/ceph.client.admin.keyring 2024-04-12T20:46:19.486 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:46:19 smithi186 ceph-mon[36597]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' 2024-04-12T20:46:19.486 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:46:19 smithi186 ceph-mon[36597]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' 2024-04-12T20:46:19.486 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:46:19 smithi186 ceph-mon[36597]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' 2024-04-12T20:46:19.486 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:46:19 smithi186 ceph-mon[36597]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' 2024-04-12T20:46:19.486 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:46:19 smithi186 ceph-mon[36597]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' 2024-04-12T20:46:19.486 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:46:19 smithi186 ceph-mon[36597]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi186", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-04-12T20:46:19.486 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:46:19 smithi186 ceph-mon[36597]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' cmd='[{"prefix": "auth get-or-create", "entity": "client.crash.smithi186", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]': finished 2024-04-12T20:46:19.486 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:46:19 smithi186 ceph-mon[36597]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-12T20:46:19.487 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:46:19 smithi186 ceph-mon[36597]: Deploying daemon crash.smithi186 on smithi186 2024-04-12T20:46:19.487 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:46:19 smithi186 ceph-mon[36597]: from='client.? 172.21.15.186:0/3716716920' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-12T20:46:19.487 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:46:19 smithi186 ceph-mon[36597]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T20:46:19.487 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:46:19 smithi186 ceph-mon[36597]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' 2024-04-12T20:46:19.487 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:46:19 smithi186 ceph-mon[36597]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' 2024-04-12T20:46:19.487 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:46:19 smithi186 ceph-mon[36597]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' 2024-04-12T20:46:19.487 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:46:19 smithi186 ceph-mon[36597]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' 2024-04-12T20:46:19.487 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:46:19 smithi186 ceph-mon[36597]: Deploying daemon node-exporter.smithi186 on smithi186 2024-04-12T20:46:19.487 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:46:19 smithi186 ceph-mon[36597]: from='client.? 172.21.15.186:0/3983440824' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-12T20:46:19.487 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:46:19 smithi186 ceph-mon[36597]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-12T20:46:19.487 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:46:19 smithi186 ceph-mon[36597]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' 2024-04-12T20:46:19.487 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:46:19 smithi186 ceph-mon[36597]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-12T20:46:19.488 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:46:19 smithi186 ceph-mon[36597]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' 2024-04-12T20:46:19.488 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:46:19 smithi186 ceph-mon[36597]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' 2024-04-12T20:46:19.488 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:46:19 smithi186 ceph-mon[36597]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' 2024-04-12T20:46:19.488 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:46:19 smithi186 ceph-mon[36597]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' 2024-04-12T20:46:19.488 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:46:19 smithi186 ceph-mon[36597]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi186.kqcfxk", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-12T20:46:19.489 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:46:19 smithi186 ceph-mon[36597]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.smithi186.kqcfxk", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-04-12T20:46:19.489 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:46:19 smithi186 ceph-mon[36597]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-12T20:46:19.489 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:46:19 smithi186 ceph-mon[36597]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-12T20:46:19.489 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:46:19 smithi186 ceph-mon[36597]: Deploying daemon mgr.smithi186.kqcfxk on smithi186 2024-04-12T20:46:19.489 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:46:19 smithi186 ceph-mon[36597]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-12T20:46:19.489 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:46:19 smithi186 ceph-mon[36597]: from='client.? 172.21.15.186:0/1664864883' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-12T20:46:19.489 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:46:19 smithi186 ceph-mon[36597]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' 2024-04-12T20:46:19.489 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:46:19 smithi186 ceph-mon[36597]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-12T20:46:19.489 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:46:19 smithi186 ceph-mon[36597]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' 2024-04-12T20:46:19.489 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:46:19 smithi186 ceph-mon[36597]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' 2024-04-12T20:46:19.489 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:46:19 smithi186 ceph-mon[36597]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' 2024-04-12T20:46:19.489 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:46:19 smithi186 ceph-mon[36597]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' 2024-04-12T20:46:19.489 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:46:19 smithi186 ceph-mon[36597]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-12T20:46:19.489 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:46:19 smithi186 ceph-mon[36597]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-12T20:46:19.490 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:46:19 smithi186 ceph-mon[36597]: Deploying daemon mon.smithi186 on smithi186 2024-04-12T20:46:19.490 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:46:19 smithi186 ceph-mon[36597]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-12T20:46:19.490 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:46:19 smithi186 ceph-mon[36597]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-12T20:46:19.490 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:46:19 smithi186 ceph-mon[36597]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T20:46:19.490 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:46:19 smithi186 ceph-mon[36597]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' 2024-04-12T20:46:19.490 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:46:19 smithi186 ceph-mon[36597]: from='client.? 172.21.15.186:0/3657706470' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-12T20:46:19.490 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:46:19 smithi186 ceph-mon[36597]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-12T20:46:19.490 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:46:19 smithi186 ceph-mon[36597]: mon.smithi186@-1(synchronizing).paxosservice(auth 1..6) refresh upgraded, format 0 -> 3 2024-04-12T20:46:19.490 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:46:19 smithi186 bash[36548]: d7263c772b737048a211b0d8729e54a447612bd7a9019ad582ef84c8def87867 2024-04-12T20:46:19.490 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:46:19 smithi186 systemd[1]: Started Ceph mon.smithi186 for 1cbe407a-f90d-11ee-bc8d-c7b262605968. 2024-04-12T20:46:24.782 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:46:24 smithi186 ceph-mon[36597]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "mon metadata", "id": "smithi186"}]: dispatch 2024-04-12T20:46:24.783 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:46:24 smithi186 ceph-mon[36597]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "mon metadata", "id": "smithi018"}]: dispatch 2024-04-12T20:46:24.783 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:46:24 smithi186 ceph-mon[36597]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "mon metadata", "id": "smithi186"}]: dispatch 2024-04-12T20:46:24.783 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:46:24 smithi186 ceph-mon[36597]: mon.smithi018 calling monitor election 2024-04-12T20:46:24.783 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:46:24 smithi186 ceph-mon[36597]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "mon metadata", "id": "smithi186"}]: dispatch 2024-04-12T20:46:24.783 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:46:24 smithi186 ceph-mon[36597]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-12T20:46:24.783 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:46:24 smithi186 ceph-mon[36597]: from='mgr.? 172.21.15.186:0/302456701' entity='mgr.smithi186.kqcfxk' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi186.kqcfxk/crt"}]: dispatch 2024-04-12T20:46:24.783 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:46:24 smithi186 ceph-mon[36597]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "mon metadata", "id": "smithi186"}]: dispatch 2024-04-12T20:46:24.783 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:46:24 smithi186 ceph-mon[36597]: mon.smithi186 calling monitor election 2024-04-12T20:46:24.783 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:46:24 smithi186 ceph-mon[36597]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "mon metadata", "id": "smithi186"}]: dispatch 2024-04-12T20:46:24.783 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:46:24 smithi186 ceph-mon[36597]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-12T20:46:24.783 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:46:24 smithi186 ceph-mon[36597]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "mon metadata", "id": "smithi186"}]: dispatch 2024-04-12T20:46:24.783 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:46:24 smithi186 ceph-mon[36597]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "mon metadata", "id": "smithi186"}]: dispatch 2024-04-12T20:46:24.783 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:46:24 smithi186 ceph-mon[36597]: mon.smithi018 is new leader, mons smithi018,smithi186 in quorum (ranks 0,1) 2024-04-12T20:46:24.783 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:46:24 smithi186 ceph-mon[36597]: monmap e2: 2 mons at {smithi018=[v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0],smithi186=[v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-04-12T20:46:24.784 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:46:24 smithi186 ceph-mon[36597]: fsmap 2024-04-12T20:46:24.784 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:46:24 smithi186 ceph-mon[36597]: osdmap e5: 0 total, 0 up, 0 in 2024-04-12T20:46:24.784 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:46:24 smithi186 ceph-mon[36597]: mgrmap e17: smithi018.xjzwga(active, since 39s) 2024-04-12T20:46:24.784 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:46:24 smithi186 ceph-mon[36597]: overall HEALTH_OK 2024-04-12T20:46:24.784 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:46:24 smithi186 ceph-mon[36597]: Standby manager daemon smithi186.kqcfxk started 2024-04-12T20:46:24.784 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:46:24 smithi186 ceph-mon[36597]: from='mgr.? 172.21.15.186:0/302456701' entity='mgr.smithi186.kqcfxk' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-12T20:46:24.784 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:46:24 smithi186 ceph-mon[36597]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' 2024-04-12T20:46:24.784 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:46:24 smithi186 ceph-mon[36597]: from='mgr.? 172.21.15.186:0/302456701' entity='mgr.smithi186.kqcfxk' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi186.kqcfxk/key"}]: dispatch 2024-04-12T20:46:24.784 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:46:24 smithi186 ceph-mon[36597]: from='mgr.? 172.21.15.186:0/302456701' entity='mgr.smithi186.kqcfxk' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-12T20:46:24.784 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:46:24 smithi186 ceph-mon[36597]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' 2024-04-12T20:46:24.784 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:46:24 smithi186 ceph-mon[36597]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' 2024-04-12T20:46:24.854 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:46:24 smithi018 ceph-mon[28799]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "mon metadata", "id": "smithi186"}]: dispatch 2024-04-12T20:46:24.854 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:46:24 smithi018 ceph-mon[28799]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "mon metadata", "id": "smithi018"}]: dispatch 2024-04-12T20:46:24.854 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:46:24 smithi018 ceph-mon[28799]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "mon metadata", "id": "smithi186"}]: dispatch 2024-04-12T20:46:24.854 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:46:24 smithi018 ceph-mon[28799]: mon.smithi018 calling monitor election 2024-04-12T20:46:24.854 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:46:24 smithi018 ceph-mon[28799]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "mon metadata", "id": "smithi186"}]: dispatch 2024-04-12T20:46:24.854 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:46:24 smithi018 ceph-mon[28799]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-12T20:46:24.854 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:46:24 smithi018 ceph-mon[28799]: from='mgr.? 172.21.15.186:0/302456701' entity='mgr.smithi186.kqcfxk' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi186.kqcfxk/crt"}]: dispatch 2024-04-12T20:46:24.854 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:46:24 smithi018 ceph-mon[28799]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "mon metadata", "id": "smithi186"}]: dispatch 2024-04-12T20:46:24.855 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:46:24 smithi018 ceph-mon[28799]: mon.smithi186 calling monitor election 2024-04-12T20:46:24.855 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:46:24 smithi018 ceph-mon[28799]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "mon metadata", "id": "smithi186"}]: dispatch 2024-04-12T20:46:24.855 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:46:24 smithi018 ceph-mon[28799]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-12T20:46:24.855 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:46:24 smithi018 ceph-mon[28799]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "mon metadata", "id": "smithi186"}]: dispatch 2024-04-12T20:46:24.855 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:46:24 smithi018 ceph-mon[28799]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "mon metadata", "id": "smithi186"}]: dispatch 2024-04-12T20:46:24.855 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:46:24 smithi018 ceph-mon[28799]: mon.smithi018 is new leader, mons smithi018,smithi186 in quorum (ranks 0,1) 2024-04-12T20:46:24.855 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:46:24 smithi018 ceph-mon[28799]: monmap e2: 2 mons at {smithi018=[v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0],smithi186=[v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-04-12T20:46:24.855 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:46:24 smithi018 ceph-mon[28799]: fsmap 2024-04-12T20:46:24.855 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:46:24 smithi018 ceph-mon[28799]: osdmap e5: 0 total, 0 up, 0 in 2024-04-12T20:46:24.855 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:46:24 smithi018 ceph-mon[28799]: mgrmap e17: smithi018.xjzwga(active, since 39s) 2024-04-12T20:46:24.855 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:46:24 smithi018 ceph-mon[28799]: overall HEALTH_OK 2024-04-12T20:46:24.855 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:46:24 smithi018 ceph-mon[28799]: Standby manager daemon smithi186.kqcfxk started 2024-04-12T20:46:24.855 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:46:24 smithi018 ceph-mon[28799]: from='mgr.? 172.21.15.186:0/302456701' entity='mgr.smithi186.kqcfxk' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-12T20:46:24.855 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:46:24 smithi018 ceph-mon[28799]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' 2024-04-12T20:46:24.855 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:46:24 smithi018 ceph-mon[28799]: from='mgr.? 172.21.15.186:0/302456701' entity='mgr.smithi186.kqcfxk' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi186.kqcfxk/key"}]: dispatch 2024-04-12T20:46:24.855 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:46:24 smithi018 ceph-mon[28799]: from='mgr.? 172.21.15.186:0/302456701' entity='mgr.smithi186.kqcfxk' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-12T20:46:24.856 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:46:24 smithi018 ceph-mon[28799]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' 2024-04-12T20:46:24.856 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:46:24 smithi018 ceph-mon[28799]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' 2024-04-12T20:46:25.510 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:46:25.509+0000 7f6295400700 1 -- 172.21.15.186:0/901027124 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f62740061f0 msgr2=0x7f6274008650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:46:25.511 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:46:25.509+0000 7f6295400700 1 --2- 172.21.15.186:0/901027124 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f62740061f0 0x7f6274008650 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f62900fda70 tx=0x7f6280002040 comp rx=0 tx=0).stop 2024-04-12T20:46:25.511 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:46:25.509+0000 7f6295400700 1 -- 172.21.15.186:0/901027124 shutdown_connections 2024-04-12T20:46:25.511 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:46:25.509+0000 7f6295400700 1 --2- 172.21.15.186:0/901027124 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f62740061f0 0x7f6274008650 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:46:25.511 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:46:25.509+0000 7f6295400700 1 -- 172.21.15.186:0/901027124 >> 172.21.15.186:0/901027124 conn(0x7f62900f8510 msgr2=0x7f62900fa920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T20:46:25.511 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:46:25.509+0000 7f6295400700 1 -- 172.21.15.186:0/901027124 shutdown_connections 2024-04-12T20:46:25.511 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:46:25.509+0000 7f6295400700 1 -- 172.21.15.186:0/901027124 wait complete. 2024-04-12T20:46:25.511 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:46:25.511+0000 7f6295400700 1 Processor -- start 2024-04-12T20:46:25.511 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:46:25.512+0000 7f6295400700 1 -- start start 2024-04-12T20:46:25.512 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:46:25.512+0000 7f6295400700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f62901154f0 0x7f62901158c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T20:46:25.512 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:46:25.512+0000 7f6295400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6290115e00 0x7f629010f470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T20:46:25.512 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:46:25.512+0000 7f6295400700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f629010fb00 con 0x7f6290115e00 2024-04-12T20:46:25.512 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:46:25.512+0000 7f6295400700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f629010fc70 con 0x7f62901154f0 2024-04-12T20:46:25.512 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:46:25.513+0000 7f628fe00700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f62901154f0 0x7f62901158c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T20:46:25.512 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:46:25.513+0000 7f628fe00700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f62901154f0 0x7f62901158c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.186:49748/0 (socket says 172.21.15.186:49748) 2024-04-12T20:46:25.513 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:46:25.513+0000 7f628fe00700 1 -- 172.21.15.186:0/1102979752 learned_addr learned my addr 172.21.15.186:0/1102979752 (peer_addr_for_me v2:172.21.15.186:0/0) 2024-04-12T20:46:25.513 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:46:25.513+0000 7f628f400700 1 --2- 172.21.15.186:0/1102979752 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6290115e00 0x7f629010f470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T20:46:25.513 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:46:25.513+0000 7f628fe00700 1 -- 172.21.15.186:0/1102979752 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f62901154f0 msgr2=0x7f62901158c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 12 2024-04-12T20:46:25.513 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:46:25.513+0000 7f628fe00700 1 -- 172.21.15.186:0/1102979752 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f62901154f0 msgr2=0x7f62901158c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-04-12T20:46:25.513 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:46:25.513+0000 7f628fe00700 1 --2- 172.21.15.186:0/1102979752 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f62901154f0 0x7f62901158c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2024-04-12T20:46:25.513 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:46:25.513+0000 7f628fe00700 1 --2- 172.21.15.186:0/1102979752 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f62901154f0 0x7f62901158c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-12T20:46:25.513 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:46:25.514+0000 7f628f400700 1 -- 172.21.15.186:0/1102979752 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f62901154f0 msgr2=0x7f62901158c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-12T20:46:25.514 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:46:25.514+0000 7f628f400700 1 --2- 172.21.15.186:0/1102979752 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f62901154f0 0x7f62901158c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:46:25.514 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:46:25.514+0000 7f628f400700 1 -- 172.21.15.186:0/1102979752 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6280005040 con 0x7f6290115e00 2024-04-12T20:46:25.514 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:46:25.515+0000 7f628f400700 1 --2- 172.21.15.186:0/1102979752 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6290115e00 0x7f629010f470 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f627c00b790 tx=0x7f627c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T20:46:25.514 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:46:25.515+0000 7f6287e00700 1 -- 172.21.15.186:0/1102979752 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f627c011810 con 0x7f6290115e00 2024-04-12T20:46:25.514 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:46:25.515+0000 7f6295400700 1 -- 172.21.15.186:0/1102979752 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f629010ff50 con 0x7f6290115e00 2024-04-12T20:46:25.515 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:46:25.515+0000 7f6287e00700 1 -- 172.21.15.186:0/1102979752 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f627c011e40 con 0x7f6290115e00 2024-04-12T20:46:25.515 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:46:25.515+0000 7f6287e00700 1 -- 172.21.15.186:0/1102979752 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f627c00f2e0 con 0x7f6290115e00 2024-04-12T20:46:25.515 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:46:25.515+0000 7f6295400700 1 -- 172.21.15.186:0/1102979752 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f62901102d0 con 0x7f6290115e00 2024-04-12T20:46:25.515 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:46:25.517+0000 7f6295400700 1 -- 172.21.15.186:0/1102979752 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6290100580 con 0x7f6290115e00 2024-04-12T20:46:25.517 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:46:25.519+0000 7f6287e00700 1 -- 172.21.15.186:0/1102979752 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 18) v1 ==== 86565+0+0 (secure 0 0 0) 0x7f627c010380 con 0x7f6290115e00 2024-04-12T20:46:25.518 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:46:25.520+0000 7f6287e00700 1 --2- 172.21.15.186:0/1102979752 >> [v2:172.21.15.18:6800/3048894687,v1:172.21.15.18:6801/3048894687] conn(0x7f6270069360 0x7f627006b810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T20:46:25.518 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:46:25.520+0000 7f6287e00700 1 -- 172.21.15.186:0/1102979752 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1366+0+0 (secure 0 0 0) 0x7f627c0868f0 con 0x7f6290115e00 2024-04-12T20:46:25.519 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:46:25.520+0000 7f628fe00700 1 --2- 172.21.15.186:0/1102979752 >> [v2:172.21.15.18:6800/3048894687,v1:172.21.15.18:6801/3048894687] conn(0x7f6270069360 0x7f627006b810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T20:46:25.520 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:46:25.522+0000 7f628fe00700 1 --2- 172.21.15.186:0/1102979752 >> [v2:172.21.15.18:6800/3048894687,v1:172.21.15.18:6801/3048894687] conn(0x7f6270069360 0x7f627006b810 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f62800045a0 tx=0x7f6280002040 comp rx=0 tx=0).ready entity=mgr.14212 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T20:46:25.534 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:46:25.533+0000 7f6287e00700 1 -- 172.21.15.186:0/1102979752 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7f627c0576f0 con 0x7f6290115e00 2024-04-12T20:46:25.782 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:46:25 smithi186 ceph-mon[36597]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' 2024-04-12T20:46:25.782 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:46:25 smithi186 ceph-mon[36597]: mgrmap e18: smithi018.xjzwga(active, since 39s), standbys: smithi186.kqcfxk 2024-04-12T20:46:25.782 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:46:25 smithi186 ceph-mon[36597]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "mgr metadata", "who": "smithi186.kqcfxk", "id": "smithi186.kqcfxk"}]: dispatch 2024-04-12T20:46:25.782 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:46:25 smithi186 ceph-mon[36597]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' 2024-04-12T20:46:25.782 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:46:25 smithi186 ceph-mon[36597]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-12T20:46:25.782 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:46:25 smithi186 ceph-mon[36597]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-12T20:46:25.782 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:46:25 smithi186 ceph-mon[36597]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' 2024-04-12T20:46:25.783 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:46:25 smithi186 ceph-mon[36597]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "mon metadata", "id": "smithi186"}]: dispatch 2024-04-12T20:46:25.840 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:46:25.841+0000 7f6295400700 1 -- 172.21.15.186:0/1102979752 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f62901046a0 con 0x7f6290115e00 2024-04-12T20:46:25.841 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:46:25.842+0000 7f6287e00700 1 -- 172.21.15.186:0/1102979752 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 2 v2) v1 ==== 95+0+1017 (secure 0 0 0) 0x7f627c015090 con 0x7f6290115e00 2024-04-12T20:46:25.841 INFO:teuthology.orchestra.run.smithi186.stdout: 2024-04-12T20:46:25.841 INFO:teuthology.orchestra.run.smithi186.stdout:{"epoch":2,"fsid":"1cbe407a-f90d-11ee-bc8d-c7b262605968","modified":"2024-04-12T20:46:19.383203Z","created":"2024-04-12T20:42:56.111400Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"smithi018","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:3300","nonce":0},{"type":"v1","addr":"172.21.15.18:6789","nonce":0}]},"addr":"172.21.15.18:6789/0","public_addr":"172.21.15.18:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"smithi186","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:3300","nonce":0},{"type":"v1","addr":"172.21.15.186:6789","nonce":0}]},"addr":"172.21.15.186:6789/0","public_addr":"172.21.15.186:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-04-12T20:46:25.843 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:46:25.845+0000 7f6295400700 1 -- 172.21.15.186:0/1102979752 >> [v2:172.21.15.18:6800/3048894687,v1:172.21.15.18:6801/3048894687] conn(0x7f6270069360 msgr2=0x7f627006b810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:46:25.844 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:46:25.845+0000 7f6295400700 1 --2- 172.21.15.186:0/1102979752 >> [v2:172.21.15.18:6800/3048894687,v1:172.21.15.18:6801/3048894687] conn(0x7f6270069360 0x7f627006b810 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f62800045a0 tx=0x7f6280002040 comp rx=0 tx=0).stop 2024-04-12T20:46:25.844 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:46:25.845+0000 7f6295400700 1 -- 172.21.15.186:0/1102979752 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6290115e00 msgr2=0x7f629010f470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:46:25.844 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:46:25.845+0000 7f6295400700 1 --2- 172.21.15.186:0/1102979752 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6290115e00 0x7f629010f470 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f627c00b790 tx=0x7f627c0091a0 comp rx=0 tx=0).stop 2024-04-12T20:46:25.844 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:46:25.845+0000 7f6295400700 1 -- 172.21.15.186:0/1102979752 shutdown_connections 2024-04-12T20:46:25.844 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:46:25.845+0000 7f6295400700 1 --2- 172.21.15.186:0/1102979752 >> [v2:172.21.15.18:6800/3048894687,v1:172.21.15.18:6801/3048894687] conn(0x7f6270069360 0x7f627006b810 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:46:25.844 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:46:25.845+0000 7f6295400700 1 --2- 172.21.15.186:0/1102979752 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f62901154f0 0x7f62901158c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:46:25.844 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:46:25.845+0000 7f6295400700 1 --2- 172.21.15.186:0/1102979752 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6290115e00 0x7f629010f470 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:46:25.844 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:46:25.845+0000 7f6295400700 1 -- 172.21.15.186:0/1102979752 >> 172.21.15.186:0/1102979752 conn(0x7f62900f8510 msgr2=0x7f6290101430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T20:46:25.844 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:46:25.845+0000 7f6295400700 1 -- 172.21.15.186:0/1102979752 shutdown_connections 2024-04-12T20:46:25.845 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:46:25.845+0000 7f6295400700 1 -- 172.21.15.186:0/1102979752 wait complete. 2024-04-12T20:46:25.845 INFO:teuthology.orchestra.run.smithi186.stderr:dumped monmap epoch 2 2024-04-12T20:46:25.853 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:46:25 smithi018 ceph-mon[28799]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' 2024-04-12T20:46:25.853 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:46:25 smithi018 ceph-mon[28799]: mgrmap e18: smithi018.xjzwga(active, since 39s), standbys: smithi186.kqcfxk 2024-04-12T20:46:25.853 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:46:25 smithi018 ceph-mon[28799]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "mgr metadata", "who": "smithi186.kqcfxk", "id": "smithi186.kqcfxk"}]: dispatch 2024-04-12T20:46:25.853 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:46:25 smithi018 ceph-mon[28799]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' 2024-04-12T20:46:25.853 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:46:25 smithi018 ceph-mon[28799]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-12T20:46:25.853 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:46:25 smithi018 ceph-mon[28799]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-12T20:46:25.853 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:46:25 smithi018 ceph-mon[28799]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' 2024-04-12T20:46:25.853 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:46:25 smithi018 ceph-mon[28799]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "mon metadata", "id": "smithi186"}]: dispatch 2024-04-12T20:46:26.532 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:46:26 smithi186 ceph-mon[36597]: from='client.? 172.21.15.186:0/1102979752' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-12T20:46:26.854 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:46:26 smithi018 ceph-mon[28799]: from='client.? 172.21.15.186:0/1102979752' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-12T20:46:27.066 INFO:tasks.cephadm:Generating final ceph.conf file... 2024-04-12T20:46:27.066 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1cbe407a-f90d-11ee-bc8d-c7b262605968 -- ceph config generate-minimal-conf 2024-04-12T20:46:27.556 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:46:27 smithi186 ceph-mon[36597]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-12T20:46:27.590 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/mon.smithi018/config 2024-04-12T20:46:27.787 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:46:27 smithi018 ceph-mon[28799]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-12T20:46:29.282 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:46:29 smithi186 ceph-mon[36597]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' 2024-04-12T20:46:29.282 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:46:29 smithi186 ceph-mon[36597]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' 2024-04-12T20:46:29.282 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:46:29 smithi186 ceph-mon[36597]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-12T20:46:29.282 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:46:29 smithi186 ceph-mon[36597]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-12T20:46:29.282 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:46:29 smithi186 ceph-mon[36597]: Updating smithi018:/etc/ceph/ceph.conf 2024-04-12T20:46:29.282 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:46:29 smithi186 ceph-mon[36597]: Updating smithi186:/etc/ceph/ceph.conf 2024-04-12T20:46:29.282 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:46:29 smithi186 ceph-mon[36597]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-12T20:46:29.282 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:46:29 smithi186 ceph-mon[36597]: Updating smithi018:/var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/config/ceph.conf 2024-04-12T20:46:29.282 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:46:29 smithi186 ceph-mon[36597]: Updating smithi186:/var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/config/ceph.conf 2024-04-12T20:46:29.426 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:46:29 smithi018 ceph-mon[28799]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' 2024-04-12T20:46:29.426 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:46:29 smithi018 ceph-mon[28799]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' 2024-04-12T20:46:29.426 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:46:29 smithi018 ceph-mon[28799]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-12T20:46:29.426 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:46:29 smithi018 ceph-mon[28799]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-12T20:46:29.426 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:46:29 smithi018 ceph-mon[28799]: Updating smithi018:/etc/ceph/ceph.conf 2024-04-12T20:46:29.426 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:46:29 smithi018 ceph-mon[28799]: Updating smithi186:/etc/ceph/ceph.conf 2024-04-12T20:46:29.426 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:46:29 smithi018 ceph-mon[28799]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-12T20:46:29.426 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:46:29 smithi018 ceph-mon[28799]: Updating smithi018:/var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/config/ceph.conf 2024-04-12T20:46:29.426 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:46:29 smithi018 ceph-mon[28799]: Updating smithi186:/var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/config/ceph.conf 2024-04-12T20:46:29.702 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:46:29.702+0000 7f5e19200700 1 -- 172.21.15.18:0/1548934608 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5e14102ac0 msgr2=0x7f5e14102e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:46:29.703 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:46:29.702+0000 7f5e19200700 1 --2- 172.21.15.18:0/1548934608 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5e14102ac0 0x7f5e14102e90 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f5e040048c0 tx=0x7f5e04009e30 comp rx=0 tx=0).stop 2024-04-12T20:46:29.703 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:46:29.702+0000 7f5e19200700 1 -- 172.21.15.18:0/1548934608 shutdown_connections 2024-04-12T20:46:29.703 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:46:29.702+0000 7f5e19200700 1 --2- 172.21.15.18:0/1548934608 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5e14102ac0 0x7f5e14102e90 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:46:29.703 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:46:29.702+0000 7f5e19200700 1 -- 172.21.15.18:0/1548934608 >> 172.21.15.18:0/1548934608 conn(0x7f5e140f85e0 msgr2=0x7f5e140fa9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T20:46:29.703 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:46:29.702+0000 7f5e19200700 1 -- 172.21.15.18:0/1548934608 shutdown_connections 2024-04-12T20:46:29.703 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:46:29.702+0000 7f5e19200700 1 -- 172.21.15.18:0/1548934608 wait complete. 2024-04-12T20:46:29.704 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:46:29.703+0000 7f5e19200700 1 Processor -- start 2024-04-12T20:46:29.704 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:46:29.704+0000 7f5e19200700 1 -- start start 2024-04-12T20:46:29.704 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:46:29.704+0000 7f5e19200700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5e14102ac0 0x7f5e14113180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T20:46:29.704 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:46:29.704+0000 7f5e19200700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5e141136c0 0x7f5e1410d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T20:46:29.704 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:46:29.704+0000 7f5e19200700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e14113cd0 con 0x7f5e141136c0 2024-04-12T20:46:29.704 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:46:29.704+0000 7f5e19200700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e14113e40 con 0x7f5e14102ac0 2024-04-12T20:46:29.704 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:46:29.704+0000 7f5e11400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5e141136c0 0x7f5e1410d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T20:46:29.704 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:46:29.704+0000 7f5e11400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5e141136c0 0x7f5e1410d190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:36186/0 (socket says 172.21.15.18:36186) 2024-04-12T20:46:29.704 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:46:29.704+0000 7f5e11400700 1 -- 172.21.15.18:0/3825252857 learned_addr learned my addr 172.21.15.18:0/3825252857 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T20:46:29.705 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:46:29.705+0000 7f5e11400700 1 -- 172.21.15.18:0/3825252857 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5e14102ac0 msgr2=0x7f5e14113180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:46:29.705 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:46:29.705+0000 7f5e11400700 1 --2- 172.21.15.18:0/3825252857 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5e14102ac0 0x7f5e14113180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:46:29.705 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:46:29.705+0000 7f5e11400700 1 -- 172.21.15.18:0/3825252857 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e04005040 con 0x7f5e141136c0 2024-04-12T20:46:29.706 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:46:29.705+0000 7f5e11400700 1 --2- 172.21.15.18:0/3825252857 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5e141136c0 0x7f5e1410d190 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f5e0000b790 tx=0x7f5e00009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T20:46:29.706 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:46:29.705+0000 7f5e0a000700 1 -- 172.21.15.18:0/3825252857 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5e00010870 con 0x7f5e141136c0 2024-04-12T20:46:29.706 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:46:29.705+0000 7f5e19200700 1 -- 172.21.15.18:0/3825252857 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5e1410d7f0 con 0x7f5e141136c0 2024-04-12T20:46:29.706 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:46:29.705+0000 7f5e0a000700 1 -- 172.21.15.18:0/3825252857 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f5e0000f3e0 con 0x7f5e141136c0 2024-04-12T20:46:29.706 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:46:29.706+0000 7f5e0a000700 1 -- 172.21.15.18:0/3825252857 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5e00010ea0 con 0x7f5e141136c0 2024-04-12T20:46:29.706 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:46:29.706+0000 7f5e19200700 1 -- 172.21.15.18:0/3825252857 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5e1410ddb0 con 0x7f5e141136c0 2024-04-12T20:46:29.706 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:46:29.707+0000 7f5e19200700 1 -- 172.21.15.18:0/3825252857 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5e14101d30 con 0x7f5e141136c0 2024-04-12T20:46:29.708 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:46:29.708+0000 7f5e0a000700 1 -- 172.21.15.18:0/3825252857 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 18) v1 ==== 86565+0+0 (secure 0 0 0) 0x7f5e0000f550 con 0x7f5e141136c0 2024-04-12T20:46:29.709 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:46:29.709+0000 7f5e0a000700 1 --2- 172.21.15.18:0/3825252857 >> [v2:172.21.15.18:6800/3048894687,v1:172.21.15.18:6801/3048894687] conn(0x7f5df8069360 0x7f5df806b810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T20:46:29.710 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:46:29.710+0000 7f5e0a000700 1 -- 172.21.15.18:0/3825252857 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1366+0+0 (secure 0 0 0) 0x7f5e00086690 con 0x7f5e141136c0 2024-04-12T20:46:29.710 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:46:29.710+0000 7f5e11e00700 1 --2- 172.21.15.18:0/3825252857 >> [v2:172.21.15.18:6800/3048894687,v1:172.21.15.18:6801/3048894687] conn(0x7f5df8069360 0x7f5df806b810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T20:46:29.710 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:46:29.711+0000 7f5e11e00700 1 --2- 172.21.15.18:0/3825252857 >> [v2:172.21.15.18:6800/3048894687,v1:172.21.15.18:6801/3048894687] conn(0x7f5df8069360 0x7f5df806b810 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f5e04005010 tx=0x7f5e04009e30 comp rx=0 tx=0).ready entity=mgr.14212 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T20:46:29.714 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:46:29.715+0000 7f5e0a000700 1 -- 172.21.15.18:0/3825252857 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7f5e00053a00 con 0x7f5e141136c0 2024-04-12T20:46:29.937 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:46:29.938+0000 7f5e19200700 1 -- 172.21.15.18:0/3825252857 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) v1 -- 0x7f5e140fe1c0 con 0x7f5e141136c0 2024-04-12T20:46:29.938 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:46:29.938+0000 7f5e0a000700 1 -- 172.21.15.18:0/3825252857 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v10) v1 ==== 76+0+225 (secure 0 0 0) 0x7f5e00014100 con 0x7f5e141136c0 2024-04-12T20:46:29.938 INFO:teuthology.orchestra.run.smithi018.stdout:# minimal ceph.conf for 1cbe407a-f90d-11ee-bc8d-c7b262605968 2024-04-12T20:46:29.939 INFO:teuthology.orchestra.run.smithi018.stdout:[global] 2024-04-12T20:46:29.939 INFO:teuthology.orchestra.run.smithi018.stdout: fsid = 1cbe407a-f90d-11ee-bc8d-c7b262605968 2024-04-12T20:46:29.939 INFO:teuthology.orchestra.run.smithi018.stdout: mon_host = [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] 2024-04-12T20:46:29.940 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:46:29.941+0000 7f5e19200700 1 -- 172.21.15.18:0/3825252857 >> [v2:172.21.15.18:6800/3048894687,v1:172.21.15.18:6801/3048894687] conn(0x7f5df8069360 msgr2=0x7f5df806b810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:46:29.941 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:46:29.941+0000 7f5e19200700 1 --2- 172.21.15.18:0/3825252857 >> [v2:172.21.15.18:6800/3048894687,v1:172.21.15.18:6801/3048894687] conn(0x7f5df8069360 0x7f5df806b810 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f5e04005010 tx=0x7f5e04009e30 comp rx=0 tx=0).stop 2024-04-12T20:46:29.941 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:46:29.941+0000 7f5e19200700 1 -- 172.21.15.18:0/3825252857 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5e141136c0 msgr2=0x7f5e1410d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:46:29.941 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:46:29.941+0000 7f5e19200700 1 --2- 172.21.15.18:0/3825252857 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5e141136c0 0x7f5e1410d190 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f5e0000b790 tx=0x7f5e00009250 comp rx=0 tx=0).stop 2024-04-12T20:46:29.941 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:46:29.941+0000 7f5e19200700 1 -- 172.21.15.18:0/3825252857 shutdown_connections 2024-04-12T20:46:29.941 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:46:29.941+0000 7f5e19200700 1 --2- 172.21.15.18:0/3825252857 >> [v2:172.21.15.18:6800/3048894687,v1:172.21.15.18:6801/3048894687] conn(0x7f5df8069360 0x7f5df806b810 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:46:29.942 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:46:29.941+0000 7f5e19200700 1 --2- 172.21.15.18:0/3825252857 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5e14102ac0 0x7f5e14113180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:46:29.942 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:46:29.941+0000 7f5e19200700 1 --2- 172.21.15.18:0/3825252857 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5e141136c0 0x7f5e1410d190 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:46:29.942 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:46:29.941+0000 7f5e19200700 1 -- 172.21.15.18:0/3825252857 >> 172.21.15.18:0/3825252857 conn(0x7f5e140f85e0 msgr2=0x7f5e140f9fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T20:46:29.942 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:46:29.941+0000 7f5e19200700 1 -- 172.21.15.18:0/3825252857 shutdown_connections 2024-04-12T20:46:29.942 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:46:29.941+0000 7f5e19200700 1 -- 172.21.15.18:0/3825252857 wait complete. 2024-04-12T20:46:30.603 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:46:30 smithi018 ceph-mon[28799]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' 2024-04-12T20:46:30.603 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:46:30 smithi018 ceph-mon[28799]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' 2024-04-12T20:46:30.603 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:46:30 smithi018 ceph-mon[28799]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' 2024-04-12T20:46:30.603 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:46:30 smithi018 ceph-mon[28799]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' 2024-04-12T20:46:30.604 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:46:30 smithi018 ceph-mon[28799]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' 2024-04-12T20:46:30.604 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:46:30 smithi018 ceph-mon[28799]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T20:46:30.604 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:46:30 smithi018 ceph-mon[28799]: Reconfiguring mon.smithi018 (unknown last config time)... 2024-04-12T20:46:30.604 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:46:30 smithi018 ceph-mon[28799]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-12T20:46:30.604 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:46:30 smithi018 ceph-mon[28799]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-12T20:46:30.604 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:46:30 smithi018 ceph-mon[28799]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-12T20:46:30.604 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:46:30 smithi018 ceph-mon[28799]: Reconfiguring daemon mon.smithi018 on smithi018 2024-04-12T20:46:30.604 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:46:30 smithi018 ceph-mon[28799]: from='client.? 172.21.15.18:0/3825252857' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-12T20:46:30.745 INFO:tasks.cephadm:Distributing (final) config and client.admin keyring... 2024-04-12T20:46:30.746 DEBUG:teuthology.orchestra.run.smithi018:> set -ex 2024-04-12T20:46:30.746 DEBUG:teuthology.orchestra.run.smithi018:> sudo dd of=/etc/ceph/ceph.conf 2024-04-12T20:46:30.831 DEBUG:teuthology.orchestra.run.smithi018:> set -ex 2024-04-12T20:46:30.831 DEBUG:teuthology.orchestra.run.smithi018:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-04-12T20:46:30.956 DEBUG:teuthology.orchestra.run.smithi186:> set -ex 2024-04-12T20:46:30.956 DEBUG:teuthology.orchestra.run.smithi186:> sudo dd of=/etc/ceph/ceph.conf 2024-04-12T20:46:31.032 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:46:30 smithi186 ceph-mon[36597]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' 2024-04-12T20:46:31.032 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:46:30 smithi186 ceph-mon[36597]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' 2024-04-12T20:46:31.032 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:46:30 smithi186 ceph-mon[36597]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' 2024-04-12T20:46:31.032 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:46:30 smithi186 ceph-mon[36597]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' 2024-04-12T20:46:31.032 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:46:30 smithi186 ceph-mon[36597]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' 2024-04-12T20:46:31.032 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:46:30 smithi186 ceph-mon[36597]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T20:46:31.032 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:46:30 smithi186 ceph-mon[36597]: Reconfiguring mon.smithi018 (unknown last config time)... 2024-04-12T20:46:31.032 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:46:30 smithi186 ceph-mon[36597]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-12T20:46:31.032 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:46:30 smithi186 ceph-mon[36597]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-12T20:46:31.033 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:46:30 smithi186 ceph-mon[36597]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-12T20:46:31.033 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:46:30 smithi186 ceph-mon[36597]: Reconfiguring daemon mon.smithi018 on smithi018 2024-04-12T20:46:31.033 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:46:30 smithi186 ceph-mon[36597]: from='client.? 172.21.15.18:0/3825252857' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-12T20:46:31.061 DEBUG:teuthology.orchestra.run.smithi186:> set -ex 2024-04-12T20:46:31.061 DEBUG:teuthology.orchestra.run.smithi186:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-04-12T20:46:31.196 INFO:tasks.cephadm:Deploying OSDs... 2024-04-12T20:46:31.197 DEBUG:teuthology.orchestra.run.smithi018:> set -ex 2024-04-12T20:46:31.197 DEBUG:teuthology.orchestra.run.smithi018:> dd if=/scratch_devs of=/dev/stdout 2024-04-12T20:46:31.245 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-04-12T20:46:31.246 DEBUG:teuthology.orchestra.run.smithi018:> stat /dev/vg_nvme/lv_1 2024-04-12T20:46:31.328 INFO:teuthology.orchestra.run.smithi018.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-04-12T20:46:31.329 INFO:teuthology.orchestra.run.smithi018.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-12T20:46:31.329 INFO:teuthology.orchestra.run.smithi018.stdout:Device: 6h/6d Inode: 631 Links: 1 2024-04-12T20:46:31.329 INFO:teuthology.orchestra.run.smithi018.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-12T20:46:31.329 INFO:teuthology.orchestra.run.smithi018.stdout:Context: system_u:object_r:device_t:s0 2024-04-12T20:46:31.329 INFO:teuthology.orchestra.run.smithi018.stdout:Access: 2024-04-12 20:45:33.843206027 +0000 2024-04-12T20:46:31.329 INFO:teuthology.orchestra.run.smithi018.stdout:Modify: 2024-04-12 20:38:35.691345783 +0000 2024-04-12T20:46:31.329 INFO:teuthology.orchestra.run.smithi018.stdout:Change: 2024-04-12 20:38:35.691345783 +0000 2024-04-12T20:46:31.329 INFO:teuthology.orchestra.run.smithi018.stdout: Birth: 2024-04-12 20:38:35.691345783 +0000 2024-04-12T20:46:31.329 DEBUG:teuthology.orchestra.run.smithi018:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-04-12T20:46:31.463 INFO:teuthology.orchestra.run.smithi018.stderr:1+0 records in 2024-04-12T20:46:31.463 INFO:teuthology.orchestra.run.smithi018.stderr:1+0 records out 2024-04-12T20:46:31.463 INFO:teuthology.orchestra.run.smithi018.stderr:512 bytes copied, 0.000287633 s, 1.8 MB/s 2024-04-12T20:46:31.467 DEBUG:teuthology.orchestra.run.smithi018:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-04-12T20:46:31.519 DEBUG:teuthology.orchestra.run.smithi018:> stat /dev/vg_nvme/lv_2 2024-04-12T20:46:31.565 INFO:teuthology.orchestra.run.smithi018.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-04-12T20:46:31.565 INFO:teuthology.orchestra.run.smithi018.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-12T20:46:31.565 INFO:teuthology.orchestra.run.smithi018.stdout:Device: 6h/6d Inode: 653 Links: 1 2024-04-12T20:46:31.565 INFO:teuthology.orchestra.run.smithi018.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-12T20:46:31.565 INFO:teuthology.orchestra.run.smithi018.stdout:Context: system_u:object_r:device_t:s0 2024-04-12T20:46:31.565 INFO:teuthology.orchestra.run.smithi018.stdout:Access: 2024-04-12 20:45:33.844206016 +0000 2024-04-12T20:46:31.565 INFO:teuthology.orchestra.run.smithi018.stdout:Modify: 2024-04-12 20:38:36.336338310 +0000 2024-04-12T20:46:31.565 INFO:teuthology.orchestra.run.smithi018.stdout:Change: 2024-04-12 20:38:36.336338310 +0000 2024-04-12T20:46:31.565 INFO:teuthology.orchestra.run.smithi018.stdout: Birth: 2024-04-12 20:38:36.336338310 +0000 2024-04-12T20:46:31.566 DEBUG:teuthology.orchestra.run.smithi018:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-04-12T20:46:31.696 INFO:teuthology.orchestra.run.smithi018.stderr:1+0 records in 2024-04-12T20:46:31.696 INFO:teuthology.orchestra.run.smithi018.stderr:1+0 records out 2024-04-12T20:46:31.696 INFO:teuthology.orchestra.run.smithi018.stderr:512 bytes copied, 0.000191361 s, 2.7 MB/s 2024-04-12T20:46:31.699 DEBUG:teuthology.orchestra.run.smithi018:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-04-12T20:46:31.751 DEBUG:teuthology.orchestra.run.smithi018:> stat /dev/vg_nvme/lv_3 2024-04-12T20:46:31.797 INFO:teuthology.orchestra.run.smithi018.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-04-12T20:46:31.797 INFO:teuthology.orchestra.run.smithi018.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-12T20:46:31.797 INFO:teuthology.orchestra.run.smithi018.stdout:Device: 6h/6d Inode: 671 Links: 1 2024-04-12T20:46:31.797 INFO:teuthology.orchestra.run.smithi018.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-12T20:46:31.797 INFO:teuthology.orchestra.run.smithi018.stdout:Context: system_u:object_r:device_t:s0 2024-04-12T20:46:31.797 INFO:teuthology.orchestra.run.smithi018.stdout:Access: 2024-04-12 20:45:33.844206016 +0000 2024-04-12T20:46:31.797 INFO:teuthology.orchestra.run.smithi018.stdout:Modify: 2024-04-12 20:38:36.973330930 +0000 2024-04-12T20:46:31.797 INFO:teuthology.orchestra.run.smithi018.stdout:Change: 2024-04-12 20:38:36.973330930 +0000 2024-04-12T20:46:31.797 INFO:teuthology.orchestra.run.smithi018.stdout: Birth: 2024-04-12 20:38:36.973330930 +0000 2024-04-12T20:46:31.798 DEBUG:teuthology.orchestra.run.smithi018:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-04-12T20:46:31.853 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:46:31 smithi018 ceph-mon[28799]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-12T20:46:31.932 INFO:teuthology.orchestra.run.smithi018.stderr:1+0 records in 2024-04-12T20:46:31.932 INFO:teuthology.orchestra.run.smithi018.stderr:1+0 records out 2024-04-12T20:46:31.932 INFO:teuthology.orchestra.run.smithi018.stderr:512 bytes copied, 0.000228001 s, 2.2 MB/s 2024-04-12T20:46:31.935 DEBUG:teuthology.orchestra.run.smithi018:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-04-12T20:46:32.027 DEBUG:teuthology.orchestra.run.smithi018:> stat /dev/vg_nvme/lv_4 2024-04-12T20:46:32.032 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:46:31 smithi186 ceph-mon[36597]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-12T20:46:32.072 INFO:teuthology.orchestra.run.smithi018.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-04-12T20:46:32.072 INFO:teuthology.orchestra.run.smithi018.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-12T20:46:32.073 INFO:teuthology.orchestra.run.smithi018.stdout:Device: 6h/6d Inode: 691 Links: 1 2024-04-12T20:46:32.073 INFO:teuthology.orchestra.run.smithi018.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-12T20:46:32.073 INFO:teuthology.orchestra.run.smithi018.stdout:Context: system_u:object_r:device_t:s0 2024-04-12T20:46:32.073 INFO:teuthology.orchestra.run.smithi018.stdout:Access: 2024-04-12 20:45:33.845206004 +0000 2024-04-12T20:46:32.073 INFO:teuthology.orchestra.run.smithi018.stdout:Modify: 2024-04-12 20:38:37.619323445 +0000 2024-04-12T20:46:32.073 INFO:teuthology.orchestra.run.smithi018.stdout:Change: 2024-04-12 20:38:37.619323445 +0000 2024-04-12T20:46:32.073 INFO:teuthology.orchestra.run.smithi018.stdout: Birth: 2024-04-12 20:38:37.619323445 +0000 2024-04-12T20:46:32.073 DEBUG:teuthology.orchestra.run.smithi018:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-04-12T20:46:32.193 INFO:teuthology.orchestra.run.smithi018.stderr:1+0 records in 2024-04-12T20:46:32.193 INFO:teuthology.orchestra.run.smithi018.stderr:1+0 records out 2024-04-12T20:46:32.193 INFO:teuthology.orchestra.run.smithi018.stderr:512 bytes copied, 0.00020783 s, 2.5 MB/s 2024-04-12T20:46:32.195 DEBUG:teuthology.orchestra.run.smithi018:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-04-12T20:46:32.240 DEBUG:teuthology.orchestra.run.smithi186:> set -ex 2024-04-12T20:46:32.241 DEBUG:teuthology.orchestra.run.smithi186:> dd if=/scratch_devs of=/dev/stdout 2024-04-12T20:46:32.299 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-04-12T20:46:32.299 DEBUG:teuthology.orchestra.run.smithi186:> stat /dev/vg_nvme/lv_1 2024-04-12T20:46:32.352 INFO:teuthology.orchestra.run.smithi186.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-04-12T20:46:32.352 INFO:teuthology.orchestra.run.smithi186.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-12T20:46:32.352 INFO:teuthology.orchestra.run.smithi186.stdout:Device: 6h/6d Inode: 633 Links: 1 2024-04-12T20:46:32.352 INFO:teuthology.orchestra.run.smithi186.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-12T20:46:32.352 INFO:teuthology.orchestra.run.smithi186.stdout:Context: system_u:object_r:device_t:s0 2024-04-12T20:46:32.352 INFO:teuthology.orchestra.run.smithi186.stdout:Access: 2024-04-12 20:46:17.574843214 +0000 2024-04-12T20:46:32.352 INFO:teuthology.orchestra.run.smithi186.stdout:Modify: 2024-04-12 20:38:24.794010639 +0000 2024-04-12T20:46:32.352 INFO:teuthology.orchestra.run.smithi186.stdout:Change: 2024-04-12 20:38:24.794010639 +0000 2024-04-12T20:46:32.352 INFO:teuthology.orchestra.run.smithi186.stdout: Birth: 2024-04-12 20:38:24.794010639 +0000 2024-04-12T20:46:32.352 DEBUG:teuthology.orchestra.run.smithi186:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-04-12T20:46:32.441 INFO:teuthology.orchestra.run.smithi186.stderr:1+0 records in 2024-04-12T20:46:32.441 INFO:teuthology.orchestra.run.smithi186.stderr:1+0 records out 2024-04-12T20:46:32.441 INFO:teuthology.orchestra.run.smithi186.stderr:512 bytes copied, 0.000456896 s, 1.1 MB/s 2024-04-12T20:46:32.444 DEBUG:teuthology.orchestra.run.smithi186:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-04-12T20:46:32.502 DEBUG:teuthology.orchestra.run.smithi186:> stat /dev/vg_nvme/lv_2 2024-04-12T20:46:32.552 INFO:teuthology.orchestra.run.smithi186.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-04-12T20:46:32.553 INFO:teuthology.orchestra.run.smithi186.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-12T20:46:32.553 INFO:teuthology.orchestra.run.smithi186.stdout:Device: 6h/6d Inode: 651 Links: 1 2024-04-12T20:46:32.553 INFO:teuthology.orchestra.run.smithi186.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-12T20:46:32.553 INFO:teuthology.orchestra.run.smithi186.stdout:Context: system_u:object_r:device_t:s0 2024-04-12T20:46:32.553 INFO:teuthology.orchestra.run.smithi186.stdout:Access: 2024-04-12 20:46:17.575843196 +0000 2024-04-12T20:46:32.553 INFO:teuthology.orchestra.run.smithi186.stdout:Modify: 2024-04-12 20:38:25.379000361 +0000 2024-04-12T20:46:32.553 INFO:teuthology.orchestra.run.smithi186.stdout:Change: 2024-04-12 20:38:25.379000361 +0000 2024-04-12T20:46:32.553 INFO:teuthology.orchestra.run.smithi186.stdout: Birth: 2024-04-12 20:38:25.379000361 +0000 2024-04-12T20:46:32.553 DEBUG:teuthology.orchestra.run.smithi186:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-04-12T20:46:32.634 INFO:teuthology.orchestra.run.smithi186.stderr:1+0 records in 2024-04-12T20:46:32.634 INFO:teuthology.orchestra.run.smithi186.stderr:1+0 records out 2024-04-12T20:46:32.634 INFO:teuthology.orchestra.run.smithi186.stderr:512 bytes copied, 0.000336303 s, 1.5 MB/s 2024-04-12T20:46:32.638 DEBUG:teuthology.orchestra.run.smithi186:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-04-12T20:46:32.691 DEBUG:teuthology.orchestra.run.smithi186:> stat /dev/vg_nvme/lv_3 2024-04-12T20:46:32.742 INFO:teuthology.orchestra.run.smithi186.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-04-12T20:46:32.742 INFO:teuthology.orchestra.run.smithi186.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-12T20:46:32.742 INFO:teuthology.orchestra.run.smithi186.stdout:Device: 6h/6d Inode: 670 Links: 1 2024-04-12T20:46:32.742 INFO:teuthology.orchestra.run.smithi186.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-12T20:46:32.743 INFO:teuthology.orchestra.run.smithi186.stdout:Context: system_u:object_r:device_t:s0 2024-04-12T20:46:32.743 INFO:teuthology.orchestra.run.smithi186.stdout:Access: 2024-04-12 20:46:17.576843179 +0000 2024-04-12T20:46:32.743 INFO:teuthology.orchestra.run.smithi186.stdout:Modify: 2024-04-12 20:38:25.991989590 +0000 2024-04-12T20:46:32.743 INFO:teuthology.orchestra.run.smithi186.stdout:Change: 2024-04-12 20:38:25.991989590 +0000 2024-04-12T20:46:32.743 INFO:teuthology.orchestra.run.smithi186.stdout: Birth: 2024-04-12 20:38:25.991989590 +0000 2024-04-12T20:46:32.743 DEBUG:teuthology.orchestra.run.smithi186:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-04-12T20:46:32.830 INFO:teuthology.orchestra.run.smithi186.stderr:1+0 records in 2024-04-12T20:46:32.831 INFO:teuthology.orchestra.run.smithi186.stderr:1+0 records out 2024-04-12T20:46:32.831 INFO:teuthology.orchestra.run.smithi186.stderr:512 bytes copied, 0.000345849 s, 1.5 MB/s 2024-04-12T20:46:32.835 DEBUG:teuthology.orchestra.run.smithi186:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-04-12T20:46:32.891 DEBUG:teuthology.orchestra.run.smithi186:> stat /dev/vg_nvme/lv_4 2024-04-12T20:46:32.940 INFO:teuthology.orchestra.run.smithi186.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-04-12T20:46:32.940 INFO:teuthology.orchestra.run.smithi186.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-12T20:46:32.940 INFO:teuthology.orchestra.run.smithi186.stdout:Device: 6h/6d Inode: 690 Links: 1 2024-04-12T20:46:32.940 INFO:teuthology.orchestra.run.smithi186.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-12T20:46:32.940 INFO:teuthology.orchestra.run.smithi186.stdout:Context: system_u:object_r:device_t:s0 2024-04-12T20:46:32.940 INFO:teuthology.orchestra.run.smithi186.stdout:Access: 2024-04-12 20:46:17.577843161 +0000 2024-04-12T20:46:32.940 INFO:teuthology.orchestra.run.smithi186.stdout:Modify: 2024-04-12 20:38:26.617978591 +0000 2024-04-12T20:46:32.940 INFO:teuthology.orchestra.run.smithi186.stdout:Change: 2024-04-12 20:38:26.617978591 +0000 2024-04-12T20:46:32.940 INFO:teuthology.orchestra.run.smithi186.stdout: Birth: 2024-04-12 20:38:26.617978591 +0000 2024-04-12T20:46:32.941 DEBUG:teuthology.orchestra.run.smithi186:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-04-12T20:46:33.030 INFO:teuthology.orchestra.run.smithi186.stderr:1+0 records in 2024-04-12T20:46:33.030 INFO:teuthology.orchestra.run.smithi186.stderr:1+0 records out 2024-04-12T20:46:33.030 INFO:teuthology.orchestra.run.smithi186.stderr:512 bytes copied, 0.000370093 s, 1.4 MB/s 2024-04-12T20:46:33.034 DEBUG:teuthology.orchestra.run.smithi186:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-04-12T20:46:33.092 INFO:tasks.cephadm:Deploying osd.0 on smithi018 with /dev/vg_nvme/lv_4... 2024-04-12T20:46:33.092 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1cbe407a-f90d-11ee-bc8d-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-04-12T20:46:33.353 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:46:33 smithi018 ceph-mon[28799]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' 2024-04-12T20:46:33.353 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:46:33 smithi018 ceph-mon[28799]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' 2024-04-12T20:46:33.353 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:46:33 smithi018 ceph-mon[28799]: Reconfiguring mgr.smithi018.xjzwga (unknown last config time)... 2024-04-12T20:46:33.353 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:46:33 smithi018 ceph-mon[28799]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi018.xjzwga", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-12T20:46:33.353 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:46:33 smithi018 ceph-mon[28799]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-12T20:46:33.353 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:46:33 smithi018 ceph-mon[28799]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-12T20:46:33.353 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:46:33 smithi018 ceph-mon[28799]: Reconfiguring daemon mgr.smithi018.xjzwga on smithi018 2024-04-12T20:46:33.354 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:46:33 smithi018 ceph-mon[28799]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-12T20:46:33.532 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:46:33 smithi186 ceph-mon[36597]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' 2024-04-12T20:46:33.532 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:46:33 smithi186 ceph-mon[36597]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' 2024-04-12T20:46:33.532 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:46:33 smithi186 ceph-mon[36597]: Reconfiguring mgr.smithi018.xjzwga (unknown last config time)... 2024-04-12T20:46:33.533 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:46:33 smithi186 ceph-mon[36597]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi018.xjzwga", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-12T20:46:33.533 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:46:33 smithi186 ceph-mon[36597]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-12T20:46:33.533 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:46:33 smithi186 ceph-mon[36597]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-12T20:46:33.533 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:46:33 smithi186 ceph-mon[36597]: Reconfiguring daemon mgr.smithi018.xjzwga on smithi018 2024-04-12T20:46:33.533 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:46:33 smithi186 ceph-mon[36597]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-12T20:46:34.188 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/mon.smithi018/config 2024-04-12T20:46:35.505 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:46:35 smithi018 ceph-mon[28799]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' 2024-04-12T20:46:35.505 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:46:35 smithi018 ceph-mon[28799]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' 2024-04-12T20:46:35.505 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:46:35 smithi018 ceph-mon[28799]: Reconfiguring crash.smithi018 (monmap changed)... 2024-04-12T20:46:35.505 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:46:35 smithi018 ceph-mon[28799]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi018", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-04-12T20:46:35.505 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:46:35 smithi018 ceph-mon[28799]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-12T20:46:35.505 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:46:35 smithi018 ceph-mon[28799]: Reconfiguring daemon crash.smithi018 on smithi018 2024-04-12T20:46:35.505 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:46:35 smithi018 ceph-mon[28799]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-12T20:46:35.532 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:46:35 smithi186 ceph-mon[36597]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' 2024-04-12T20:46:35.532 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:46:35 smithi186 ceph-mon[36597]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' 2024-04-12T20:46:35.532 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:46:35 smithi186 ceph-mon[36597]: Reconfiguring crash.smithi018 (monmap changed)... 2024-04-12T20:46:35.532 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:46:35 smithi186 ceph-mon[36597]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi018", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-04-12T20:46:35.532 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:46:35 smithi186 ceph-mon[36597]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-12T20:46:35.532 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:46:35 smithi186 ceph-mon[36597]: Reconfiguring daemon crash.smithi018 on smithi018 2024-04-12T20:46:35.533 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:46:35 smithi186 ceph-mon[36597]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-12T20:46:37.690 INFO:teuthology.orchestra.run.smithi018.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-04-12T20:46:37.690 INFO:teuthology.orchestra.run.smithi018.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-04-12T20:46:37.690 INFO:teuthology.orchestra.run.smithi018.stderr: stderr: 10+0 records in 2024-04-12T20:46:37.690 INFO:teuthology.orchestra.run.smithi018.stderr:10+0 records out 2024-04-12T20:46:37.691 INFO:teuthology.orchestra.run.smithi018.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.03526 s, 297 MB/s 2024-04-12T20:46:37.691 INFO:teuthology.orchestra.run.smithi018.stderr:--> Zapping successful for: 2024-04-12T20:46:38.065 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:46:37 smithi018 ceph-mon[28799]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-12T20:46:38.065 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:46:37 smithi018 ceph-mon[28799]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' 2024-04-12T20:46:38.065 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:46:37 smithi018 ceph-mon[28799]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' 2024-04-12T20:46:38.282 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:46:37 smithi186 ceph-mon[36597]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-12T20:46:38.282 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:46:37 smithi186 ceph-mon[36597]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' 2024-04-12T20:46:38.282 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:46:37 smithi186 ceph-mon[36597]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' 2024-04-12T20:46:38.496 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1cbe407a-f90d-11ee-bc8d-c7b262605968 -- ceph orch daemon add osd smithi018:vg_nvme/lv_4 2024-04-12T20:46:39.011 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:46:38 smithi018 ceph-mon[28799]: Reconfiguring alertmanager.smithi018 (dependencies changed)... 2024-04-12T20:46:39.011 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:46:38 smithi018 ceph-mon[28799]: Reconfiguring daemon alertmanager.smithi018 on smithi018 2024-04-12T20:46:39.282 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:46:38 smithi186 ceph-mon[36597]: Reconfiguring alertmanager.smithi018 (dependencies changed)... 2024-04-12T20:46:39.282 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:46:38 smithi186 ceph-mon[36597]: Reconfiguring daemon alertmanager.smithi018 on smithi018 2024-04-12T20:46:40.054 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/mon.smithi018/config 2024-04-12T20:46:40.198 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:46:39 smithi018 ceph-mon[28799]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-12T20:46:40.282 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:46:39 smithi186 ceph-mon[36597]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-12T20:46:42.103 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:46:41 smithi018 ceph-mon[28799]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-12T20:46:42.282 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:46:41 smithi186 ceph-mon[36597]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-12T20:46:43.983 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:46:43.738+0000 7f4ea3600700 1 -- 172.21.15.18:0/2440636085 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4e9c066b70 msgr2=0x7f4e9c0ff9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:46:43.983 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:46:43.738+0000 7f4ea3600700 1 --2- 172.21.15.18:0/2440636085 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4e9c066b70 0x7f4e9c0ff9d0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f4e90004940 tx=0x7f4e90009e30 comp rx=0 tx=0).stop 2024-04-12T20:46:43.983 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:46:43.738+0000 7f4ea3600700 1 -- 172.21.15.18:0/2440636085 shutdown_connections 2024-04-12T20:46:43.983 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:46:43.738+0000 7f4ea3600700 1 --2- 172.21.15.18:0/2440636085 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4e9c0fff10 0x7f4e9c102330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:46:43.983 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:46:43.738+0000 7f4ea3600700 1 --2- 172.21.15.18:0/2440636085 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4e9c066b70 0x7f4e9c0ff9d0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:46:43.983 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:46:43.738+0000 7f4ea3600700 1 -- 172.21.15.18:0/2440636085 >> 172.21.15.18:0/2440636085 conn(0x7f4e9c0f7440 msgr2=0x7f4e9c0f9870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T20:46:43.983 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:46:43.738+0000 7f4ea3600700 1 -- 172.21.15.18:0/2440636085 shutdown_connections 2024-04-12T20:46:43.983 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:46:43.738+0000 7f4ea3600700 1 -- 172.21.15.18:0/2440636085 wait complete. 2024-04-12T20:46:43.983 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:46:43.739+0000 7f4ea3600700 1 Processor -- start 2024-04-12T20:46:43.984 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:46:43.739+0000 7f4ea3600700 1 -- start start 2024-04-12T20:46:43.984 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:46:43.740+0000 7f4ea3600700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4e9c066b70 0x7f4e9c10fd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T20:46:43.984 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:46:43.740+0000 7f4ea3600700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4e9c0fff10 0x7f4e9c110280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T20:46:43.984 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:46:43.740+0000 7f4ea3600700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e9c110880 con 0x7f4e9c066b70 2024-04-12T20:46:43.984 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:46:43.740+0000 7f4ea3600700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e9c110fc0 con 0x7f4e9c0fff10 2024-04-12T20:46:43.984 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:46:43.740+0000 7f4ea0c00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4e9c066b70 0x7f4e9c10fd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T20:46:43.984 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:46:43.740+0000 7f4ea0c00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4e9c066b70 0x7f4e9c10fd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:33966/0 (socket says 172.21.15.18:33966) 2024-04-12T20:46:43.984 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:46:43.740+0000 7f4ea0c00700 1 -- 172.21.15.18:0/3889514408 learned_addr learned my addr 172.21.15.18:0/3889514408 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T20:46:43.984 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:46:43.741+0000 7f4e9be00700 1 --2- 172.21.15.18:0/3889514408 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4e9c0fff10 0x7f4e9c110280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T20:46:43.984 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:46:43.741+0000 7f4ea0c00700 1 -- 172.21.15.18:0/3889514408 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4e9c0fff10 msgr2=0x7f4e9c110280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:46:43.984 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:46:43.741+0000 7f4ea0c00700 1 --2- 172.21.15.18:0/3889514408 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4e9c0fff10 0x7f4e9c110280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:46:43.984 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:46:43.741+0000 7f4ea0c00700 1 -- 172.21.15.18:0/3889514408 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e90005040 con 0x7f4e9c066b70 2024-04-12T20:46:43.984 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:46:43.741+0000 7f4ea0c00700 1 --2- 172.21.15.18:0/3889514408 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4e9c066b70 0x7f4e9c10fd40 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f4e9000b970 tx=0x7f4e90009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T20:46:43.985 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:46:43.741+0000 7f4e98c00700 1 -- 172.21.15.18:0/3889514408 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4e9000e040 con 0x7f4e9c066b70 2024-04-12T20:46:43.985 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:46:43.742+0000 7f4e98c00700 1 -- 172.21.15.18:0/3889514408 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f4e9000bbf0 con 0x7f4e9c066b70 2024-04-12T20:46:43.985 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:46:43.742+0000 7f4e98c00700 1 -- 172.21.15.18:0/3889514408 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4e9001c630 con 0x7f4e9c066b70 2024-04-12T20:46:43.985 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:46:43.742+0000 7f4ea3600700 1 -- 172.21.15.18:0/3889514408 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4e9c111240 con 0x7f4e9c066b70 2024-04-12T20:46:43.985 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:46:43.742+0000 7f4ea3600700 1 -- 172.21.15.18:0/3889514408 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4e9c111800 con 0x7f4e9c066b70 2024-04-12T20:46:43.985 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:46:43.743+0000 7f4ea3600700 1 -- 172.21.15.18:0/3889514408 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4e9c05f5f0 con 0x7f4e9c066b70 2024-04-12T20:46:43.985 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:46:43.745+0000 7f4e98c00700 1 -- 172.21.15.18:0/3889514408 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 18) v1 ==== 86565+0+0 (secure 0 0 0) 0x7f4e9000bd60 con 0x7f4e9c066b70 2024-04-12T20:46:43.985 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:46:43.746+0000 7f4e98c00700 1 --2- 172.21.15.18:0/3889514408 >> [v2:172.21.15.18:6800/3048894687,v1:172.21.15.18:6801/3048894687] conn(0x7f4e84069210 0x7f4e8406b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T20:46:43.985 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:46:43.746+0000 7f4e98c00700 1 -- 172.21.15.18:0/3889514408 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1366+0+0 (secure 0 0 0) 0x7f4e90087430 con 0x7f4e9c066b70 2024-04-12T20:46:43.985 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:46:43.746+0000 7f4e9be00700 1 --2- 172.21.15.18:0/3889514408 >> [v2:172.21.15.18:6800/3048894687,v1:172.21.15.18:6801/3048894687] conn(0x7f4e84069210 0x7f4e8406b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T20:46:43.985 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:46:43.747+0000 7f4e9be00700 1 --2- 172.21.15.18:0/3889514408 >> [v2:172.21.15.18:6800/3048894687,v1:172.21.15.18:6801/3048894687] conn(0x7f4e84069210 0x7f4e8406b6c0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f4e88007fd0 tx=0x7f4e88009040 comp rx=0 tx=0).ready entity=mgr.14212 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T20:46:43.985 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:46:43.751+0000 7f4e98c00700 1 -- 172.21.15.18:0/3889514408 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7f4e90058360 con 0x7f4e9c066b70 2024-04-12T20:46:43.985 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:46:43.984+0000 7f4ea3600700 1 -- 172.21.15.18:0/3889514408 --> [v2:172.21.15.18:6800/3048894687,v1:172.21.15.18:6801/3048894687] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi018:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) v1 -- 0x7f4e9c111b80 con 0x7f4e84069210 2024-04-12T20:46:44.102 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:46:43 smithi018 ceph-mon[28799]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-12T20:46:44.282 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:46:43 smithi186 ceph-mon[36597]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-12T20:46:45.063 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:46:44 smithi018 ceph-mon[28799]: from='client.14248 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi018:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T20:46:45.064 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:46:44 smithi018 ceph-mon[28799]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-12T20:46:45.064 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:46:44 smithi018 ceph-mon[28799]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-12T20:46:45.064 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:46:44 smithi018 ceph-mon[28799]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-12T20:46:45.064 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:46:44 smithi018 ceph-mon[28799]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T20:46:45.282 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:46:44 smithi186 ceph-mon[36597]: from='client.14248 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi018:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T20:46:45.282 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:46:44 smithi186 ceph-mon[36597]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-12T20:46:45.282 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:46:44 smithi186 ceph-mon[36597]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-12T20:46:45.282 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:46:44 smithi186 ceph-mon[36597]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-12T20:46:45.282 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:46:44 smithi186 ceph-mon[36597]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T20:46:46.103 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:46:45 smithi018 ceph-mon[28799]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-12T20:46:46.282 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:46:45 smithi186 ceph-mon[36597]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-12T20:46:48.032 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:46:47 smithi186 ceph-mon[36597]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' 2024-04-12T20:46:48.032 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:46:47 smithi186 ceph-mon[36597]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' 2024-04-12T20:46:48.032 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:46:47 smithi186 ceph-mon[36597]: Reconfiguring grafana.smithi018 (dependencies changed)... 2024-04-12T20:46:48.032 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:46:47 smithi186 ceph-mon[36597]: Reconfiguring daemon grafana.smithi018 on smithi018 2024-04-12T20:46:48.033 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:46:47 smithi186 ceph-mon[36597]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-12T20:46:48.103 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:46:47 smithi018 ceph-mon[28799]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' 2024-04-12T20:46:48.103 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:46:47 smithi018 ceph-mon[28799]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' 2024-04-12T20:46:48.103 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:46:47 smithi018 ceph-mon[28799]: Reconfiguring grafana.smithi018 (dependencies changed)... 2024-04-12T20:46:48.104 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:46:47 smithi018 ceph-mon[28799]: Reconfiguring daemon grafana.smithi018 on smithi018 2024-04-12T20:46:48.104 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:46:47 smithi018 ceph-mon[28799]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-12T20:46:48.853 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:46:48 smithi018 ceph-mon[28799]: from='client.? 172.21.15.18:0/3468867780' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2af887cc-0447-42aa-a8f3-1aa7fcd4dc8a"}]: dispatch 2024-04-12T20:46:48.853 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:46:48 smithi018 ceph-mon[28799]: from='client.? 172.21.15.18:0/3468867780' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "2af887cc-0447-42aa-a8f3-1aa7fcd4dc8a"}]': finished 2024-04-12T20:46:48.853 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:46:48 smithi018 ceph-mon[28799]: osdmap e6: 1 total, 0 up, 1 in 2024-04-12T20:46:48.854 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:46:48 smithi018 ceph-mon[28799]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-12T20:46:49.032 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:46:48 smithi186 ceph-mon[36597]: from='client.? 172.21.15.18:0/3468867780' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2af887cc-0447-42aa-a8f3-1aa7fcd4dc8a"}]: dispatch 2024-04-12T20:46:49.032 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:46:48 smithi186 ceph-mon[36597]: from='client.? 172.21.15.18:0/3468867780' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "2af887cc-0447-42aa-a8f3-1aa7fcd4dc8a"}]': finished 2024-04-12T20:46:49.032 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:46:48 smithi186 ceph-mon[36597]: osdmap e6: 1 total, 0 up, 1 in 2024-04-12T20:46:49.032 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:46:48 smithi186 ceph-mon[36597]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-12T20:46:50.032 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:46:49 smithi186 ceph-mon[36597]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-12T20:46:50.032 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:46:49 smithi186 ceph-mon[36597]: from='client.? 172.21.15.18:0/2398489993' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-12T20:46:50.103 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:46:49 smithi018 ceph-mon[28799]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-12T20:46:50.103 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:46:49 smithi018 ceph-mon[28799]: from='client.? 172.21.15.18:0/2398489993' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-12T20:46:52.282 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:46:51 smithi186 ceph-mon[36597]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-12T20:46:52.353 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:46:51 smithi018 ceph-mon[28799]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-12T20:46:54.060 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:46:53 smithi018 ceph-mon[28799]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-12T20:46:54.282 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:46:53 smithi186 ceph-mon[36597]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-12T20:46:56.103 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:46:55 smithi018 ceph-mon[28799]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-12T20:46:56.282 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:46:55 smithi186 ceph-mon[36597]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-12T20:46:58.103 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:46:57 smithi018 ceph-mon[28799]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-12T20:46:58.283 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:46:57 smithi186 ceph-mon[36597]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-12T20:47:00.032 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:46:59 smithi186 ceph-mon[36597]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-12T20:47:00.032 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:46:59 smithi186 ceph-mon[36597]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T20:47:00.187 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:46:59 smithi018 ceph-mon[28799]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-12T20:47:00.188 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:46:59 smithi018 ceph-mon[28799]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T20:47:01.699 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:47:01 smithi018 ceph-mon[28799]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' 2024-04-12T20:47:01.699 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:47:01 smithi018 ceph-mon[28799]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' 2024-04-12T20:47:01.700 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:47:01 smithi018 ceph-mon[28799]: Reconfiguring prometheus.smithi018 (dependencies changed)... 2024-04-12T20:47:01.700 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:47:01 smithi018 ceph-mon[28799]: Reconfiguring daemon prometheus.smithi018 on smithi018 2024-04-12T20:47:01.700 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:47:01 smithi018 ceph-mon[28799]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-12T20:47:01.782 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:47:01 smithi186 ceph-mon[36597]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' 2024-04-12T20:47:01.782 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:47:01 smithi186 ceph-mon[36597]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' 2024-04-12T20:47:01.782 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:47:01 smithi186 ceph-mon[36597]: Reconfiguring prometheus.smithi018 (dependencies changed)... 2024-04-12T20:47:01.782 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:47:01 smithi186 ceph-mon[36597]: Reconfiguring daemon prometheus.smithi018 on smithi018 2024-04-12T20:47:01.782 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:47:01 smithi186 ceph-mon[36597]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-12T20:47:02.500 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:47:02 smithi018 ceph-mon[28799]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-04-12T20:47:02.501 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:47:02 smithi018 ceph-mon[28799]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-12T20:47:02.782 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:47:02 smithi186 ceph-mon[36597]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-04-12T20:47:02.782 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:47:02 smithi186 ceph-mon[36597]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-12T20:47:03.782 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:47:03 smithi186 ceph-mon[36597]: Deploying daemon osd.0 on smithi018 2024-04-12T20:47:03.782 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:47:03 smithi186 ceph-mon[36597]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-12T20:47:03.853 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:47:03 smithi018 ceph-mon[28799]: Deploying daemon osd.0 on smithi018 2024-04-12T20:47:03.853 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:47:03 smithi018 ceph-mon[28799]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-12T20:47:06.103 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:47:05 smithi018 ceph-mon[28799]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-12T20:47:06.282 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:47:05 smithi186 ceph-mon[36597]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-12T20:47:08.148 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:47:07 smithi018 ceph-mon[28799]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-12T20:47:08.282 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:47:07 smithi186 ceph-mon[36597]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-12T20:47:09.782 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:47:09 smithi186 ceph-mon[36597]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' 2024-04-12T20:47:09.782 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:47:09 smithi186 ceph-mon[36597]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' 2024-04-12T20:47:09.782 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:47:09 smithi186 ceph-mon[36597]: Reconfiguring crash.smithi186 (monmap changed)... 2024-04-12T20:47:09.783 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:47:09 smithi186 ceph-mon[36597]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi186", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-04-12T20:47:09.783 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:47:09 smithi186 ceph-mon[36597]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-12T20:47:09.783 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:47:09 smithi186 ceph-mon[36597]: Reconfiguring daemon crash.smithi186 on smithi186 2024-04-12T20:47:09.783 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:47:09 smithi186 ceph-mon[36597]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-12T20:47:09.854 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:47:09 smithi018 ceph-mon[28799]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' 2024-04-12T20:47:09.854 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:47:09 smithi018 ceph-mon[28799]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' 2024-04-12T20:47:09.854 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:47:09 smithi018 ceph-mon[28799]: Reconfiguring crash.smithi186 (monmap changed)... 2024-04-12T20:47:09.854 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:47:09 smithi018 ceph-mon[28799]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi186", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-04-12T20:47:09.854 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:47:09 smithi018 ceph-mon[28799]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-12T20:47:09.854 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:47:09 smithi018 ceph-mon[28799]: Reconfiguring daemon crash.smithi186 on smithi186 2024-04-12T20:47:09.854 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:47:09 smithi018 ceph-mon[28799]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-12T20:47:11.353 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:47:11 smithi018 ceph-mon[28799]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' 2024-04-12T20:47:11.353 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:47:11 smithi018 ceph-mon[28799]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' 2024-04-12T20:47:11.353 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:47:11 smithi018 ceph-mon[28799]: Reconfiguring mgr.smithi186.kqcfxk (monmap changed)... 2024-04-12T20:47:11.354 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:47:11 smithi018 ceph-mon[28799]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi186.kqcfxk", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-12T20:47:11.354 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:47:11 smithi018 ceph-mon[28799]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-12T20:47:11.354 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:47:11 smithi018 ceph-mon[28799]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-12T20:47:11.354 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:47:11 smithi018 ceph-mon[28799]: Reconfiguring daemon mgr.smithi186.kqcfxk on smithi186 2024-04-12T20:47:11.354 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:47:11 smithi018 ceph-mon[28799]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-12T20:47:11.488 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:47:11 smithi186 ceph-mon[36597]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' 2024-04-12T20:47:11.488 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:47:11 smithi186 ceph-mon[36597]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' 2024-04-12T20:47:11.488 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:47:11 smithi186 ceph-mon[36597]: Reconfiguring mgr.smithi186.kqcfxk (monmap changed)... 2024-04-12T20:47:11.488 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:47:11 smithi186 ceph-mon[36597]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi186.kqcfxk", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-12T20:47:11.488 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:47:11 smithi186 ceph-mon[36597]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-12T20:47:11.489 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:47:11 smithi186 ceph-mon[36597]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-12T20:47:11.489 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:47:11 smithi186 ceph-mon[36597]: Reconfiguring daemon mgr.smithi186.kqcfxk on smithi186 2024-04-12T20:47:11.489 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:47:11 smithi186 ceph-mon[36597]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-12T20:47:13.282 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:47:13 smithi186 ceph-mon[36597]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' 2024-04-12T20:47:13.282 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:47:13 smithi186 ceph-mon[36597]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' 2024-04-12T20:47:13.283 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:47:13 smithi186 ceph-mon[36597]: Reconfiguring mon.smithi186 (monmap changed)... 2024-04-12T20:47:13.283 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:47:13 smithi186 ceph-mon[36597]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-12T20:47:13.283 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:47:13 smithi186 ceph-mon[36597]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-12T20:47:13.283 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:47:13 smithi186 ceph-mon[36597]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-12T20:47:13.283 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:47:13 smithi186 ceph-mon[36597]: Reconfiguring daemon mon.smithi186 on smithi186 2024-04-12T20:47:13.283 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:47:13 smithi186 ceph-mon[36597]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-12T20:47:13.431 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:47:13 smithi018 ceph-mon[28799]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' 2024-04-12T20:47:13.432 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:47:13 smithi018 ceph-mon[28799]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' 2024-04-12T20:47:13.432 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:47:13 smithi018 ceph-mon[28799]: Reconfiguring mon.smithi186 (monmap changed)... 2024-04-12T20:47:13.432 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:47:13 smithi018 ceph-mon[28799]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-12T20:47:13.432 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:47:13 smithi018 ceph-mon[28799]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-12T20:47:13.432 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:47:13 smithi018 ceph-mon[28799]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-12T20:47:13.432 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:47:13 smithi018 ceph-mon[28799]: Reconfiguring daemon mon.smithi186 on smithi186 2024-04-12T20:47:13.432 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:47:13 smithi018 ceph-mon[28799]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-12T20:47:15.333 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:47:15 smithi018 ceph-mon[28799]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' 2024-04-12T20:47:15.333 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:47:15 smithi018 ceph-mon[28799]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' 2024-04-12T20:47:15.333 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:47:15 smithi018 ceph-mon[28799]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-04-12T20:47:15.333 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:47:15 smithi018 ceph-mon[28799]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-04-12T20:47:15.334 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:47:15 smithi018 ceph-mon[28799]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi018.front.sepia.ceph.com:9093"}]: dispatch 2024-04-12T20:47:15.334 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:47:15 smithi018 ceph-mon[28799]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi018.front.sepia.ceph.com:9093"}]: dispatch 2024-04-12T20:47:15.334 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:47:15 smithi018 ceph-mon[28799]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' 2024-04-12T20:47:15.334 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:47:15 smithi018 ceph-mon[28799]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-04-12T20:47:15.334 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:47:15 smithi018 ceph-mon[28799]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-04-12T20:47:15.334 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:47:15 smithi018 ceph-mon[28799]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi018.front.sepia.ceph.com:3000"}]: dispatch 2024-04-12T20:47:15.334 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:47:15 smithi018 ceph-mon[28799]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi018.front.sepia.ceph.com:3000"}]: dispatch 2024-04-12T20:47:15.334 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:47:15 smithi018 ceph-mon[28799]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' 2024-04-12T20:47:15.334 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:47:15 smithi018 ceph-mon[28799]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-04-12T20:47:15.334 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:47:15 smithi018 ceph-mon[28799]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-04-12T20:47:15.334 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:47:15 smithi018 ceph-mon[28799]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi018.front.sepia.ceph.com:9095"}]: dispatch 2024-04-12T20:47:15.334 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:47:15 smithi018 ceph-mon[28799]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi018.front.sepia.ceph.com:9095"}]: dispatch 2024-04-12T20:47:15.334 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:47:15 smithi018 ceph-mon[28799]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' 2024-04-12T20:47:15.335 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:47:15 smithi018 ceph-mon[28799]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-12T20:47:15.335 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:47:15 smithi018 ceph-mon[28799]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-12T20:47:15.335 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:47:15 smithi018 ceph-mon[28799]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T20:47:15.460 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:47:15 smithi186 ceph-mon[36597]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' 2024-04-12T20:47:15.461 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:47:15 smithi186 ceph-mon[36597]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' 2024-04-12T20:47:15.461 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:47:15 smithi186 ceph-mon[36597]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-04-12T20:47:15.461 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:47:15 smithi186 ceph-mon[36597]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-04-12T20:47:15.461 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:47:15 smithi186 ceph-mon[36597]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi018.front.sepia.ceph.com:9093"}]: dispatch 2024-04-12T20:47:15.461 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:47:15 smithi186 ceph-mon[36597]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi018.front.sepia.ceph.com:9093"}]: dispatch 2024-04-12T20:47:15.461 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:47:15 smithi186 ceph-mon[36597]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' 2024-04-12T20:47:15.461 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:47:15 smithi186 ceph-mon[36597]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-04-12T20:47:15.461 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:47:15 smithi186 ceph-mon[36597]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-04-12T20:47:15.461 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:47:15 smithi186 ceph-mon[36597]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi018.front.sepia.ceph.com:3000"}]: dispatch 2024-04-12T20:47:15.461 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:47:15 smithi186 ceph-mon[36597]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi018.front.sepia.ceph.com:3000"}]: dispatch 2024-04-12T20:47:15.461 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:47:15 smithi186 ceph-mon[36597]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' 2024-04-12T20:47:15.462 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:47:15 smithi186 ceph-mon[36597]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-04-12T20:47:15.462 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:47:15 smithi186 ceph-mon[36597]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-04-12T20:47:15.462 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:47:15 smithi186 ceph-mon[36597]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi018.front.sepia.ceph.com:9095"}]: dispatch 2024-04-12T20:47:15.462 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:47:15 smithi186 ceph-mon[36597]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi018.front.sepia.ceph.com:9095"}]: dispatch 2024-04-12T20:47:15.462 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:47:15 smithi186 ceph-mon[36597]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' 2024-04-12T20:47:15.462 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:47:15 smithi186 ceph-mon[36597]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-12T20:47:15.462 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:47:15 smithi186 ceph-mon[36597]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-12T20:47:15.462 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:47:15 smithi186 ceph-mon[36597]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T20:47:17.519 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:47:17 smithi018 ceph-mon[28799]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' 2024-04-12T20:47:17.519 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:47:17 smithi018 ceph-mon[28799]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' 2024-04-12T20:47:17.519 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:47:17 smithi018 ceph-mon[28799]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' 2024-04-12T20:47:17.520 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:47:17 smithi018 ceph-mon[28799]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' 2024-04-12T20:47:17.520 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:47:17 smithi018 ceph-mon[28799]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-12T20:47:17.532 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:47:17 smithi186 ceph-mon[36597]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' 2024-04-12T20:47:17.532 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:47:17 smithi186 ceph-mon[36597]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' 2024-04-12T20:47:17.532 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:47:17 smithi186 ceph-mon[36597]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' 2024-04-12T20:47:17.532 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:47:17 smithi186 ceph-mon[36597]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' 2024-04-12T20:47:17.532 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:47:17 smithi186 ceph-mon[36597]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-12T20:47:20.178 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:47:19 smithi018 ceph-mon[28799]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-12T20:47:20.282 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:47:19 smithi186 ceph-mon[36597]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-12T20:47:21.048 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:47:20 smithi018 ceph-mon[28799]: from='osd.0 [v2:172.21.15.18:6802/585613146,v1:172.21.15.18:6803/585613146]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-04-12T20:47:21.282 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:47:20 smithi186 ceph-mon[36597]: from='osd.0 [v2:172.21.15.18:6802/585613146,v1:172.21.15.18:6803/585613146]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-04-12T20:47:21.982 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:47:21 smithi018 ceph-mon[28799]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-12T20:47:21.982 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:47:21 smithi018 ceph-mon[28799]: from='osd.0 [v2:172.21.15.18:6802/585613146,v1:172.21.15.18:6803/585613146]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-04-12T20:47:21.983 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:47:21 smithi018 ceph-mon[28799]: osdmap e7: 1 total, 0 up, 1 in 2024-04-12T20:47:21.983 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:47:21 smithi018 ceph-mon[28799]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-12T20:47:21.983 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:47:21 smithi018 ceph-mon[28799]: from='osd.0 [v2:172.21.15.18:6802/585613146,v1:172.21.15.18:6803/585613146]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi018", "root=default"]}]: dispatch 2024-04-12T20:47:22.059 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:47:22.059+0000 7f4e98c00700 1 -- 172.21.15.18:0/3889514408 <== mgr.14212 v2:172.21.15.18:6800/3048894687 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f4e9c111b80 con 0x7f4e84069210 2024-04-12T20:47:22.060 INFO:teuthology.orchestra.run.smithi018.stdout:Created osd(s) 0 on host 'smithi018' 2024-04-12T20:47:22.062 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:47:22.063+0000 7f4ea3600700 1 -- 172.21.15.18:0/3889514408 >> [v2:172.21.15.18:6800/3048894687,v1:172.21.15.18:6801/3048894687] conn(0x7f4e84069210 msgr2=0x7f4e8406b6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:47:22.063 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:47:22.063+0000 7f4ea3600700 1 --2- 172.21.15.18:0/3889514408 >> [v2:172.21.15.18:6800/3048894687,v1:172.21.15.18:6801/3048894687] conn(0x7f4e84069210 0x7f4e8406b6c0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f4e88007fd0 tx=0x7f4e88009040 comp rx=0 tx=0).stop 2024-04-12T20:47:22.063 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:47:22.063+0000 7f4ea3600700 1 -- 172.21.15.18:0/3889514408 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4e9c066b70 msgr2=0x7f4e9c10fd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:47:22.063 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:47:22.063+0000 7f4ea3600700 1 --2- 172.21.15.18:0/3889514408 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4e9c066b70 0x7f4e9c10fd40 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f4e9000b970 tx=0x7f4e90009e30 comp rx=0 tx=0).stop 2024-04-12T20:47:22.063 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:47:22.063+0000 7f4ea3600700 1 -- 172.21.15.18:0/3889514408 shutdown_connections 2024-04-12T20:47:22.063 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:47:22.063+0000 7f4ea3600700 1 --2- 172.21.15.18:0/3889514408 >> [v2:172.21.15.18:6800/3048894687,v1:172.21.15.18:6801/3048894687] conn(0x7f4e84069210 0x7f4e8406b6c0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:47:22.063 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:47:22.063+0000 7f4ea3600700 1 --2- 172.21.15.18:0/3889514408 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4e9c066b70 0x7f4e9c10fd40 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:47:22.063 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:47:22.063+0000 7f4ea3600700 1 --2- 172.21.15.18:0/3889514408 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4e9c0fff10 0x7f4e9c110280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:47:22.063 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:47:22.063+0000 7f4ea3600700 1 -- 172.21.15.18:0/3889514408 >> 172.21.15.18:0/3889514408 conn(0x7f4e9c0f7440 msgr2=0x7f4e9c0fbab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T20:47:22.063 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:47:22.063+0000 7f4ea3600700 1 -- 172.21.15.18:0/3889514408 shutdown_connections 2024-04-12T20:47:22.064 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:47:22.063+0000 7f4ea3600700 1 -- 172.21.15.18:0/3889514408 wait complete. 2024-04-12T20:47:22.169 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:47:21 smithi186 ceph-mon[36597]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-12T20:47:22.169 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:47:21 smithi186 ceph-mon[36597]: from='osd.0 [v2:172.21.15.18:6802/585613146,v1:172.21.15.18:6803/585613146]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-04-12T20:47:22.169 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:47:21 smithi186 ceph-mon[36597]: osdmap e7: 1 total, 0 up, 1 in 2024-04-12T20:47:22.170 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:47:21 smithi186 ceph-mon[36597]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-12T20:47:22.170 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:47:21 smithi186 ceph-mon[36597]: from='osd.0 [v2:172.21.15.18:6802/585613146,v1:172.21.15.18:6803/585613146]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi018", "root=default"]}]: dispatch 2024-04-12T20:47:23.145 DEBUG:teuthology.orchestra.run.smithi018:osd.0> sudo journalctl -f -n 0 -u ceph-1cbe407a-f90d-11ee-bc8d-c7b262605968@osd.0.service 2024-04-12T20:47:23.154 INFO:tasks.cephadm:Deploying osd.1 on smithi018 with /dev/vg_nvme/lv_3... 2024-04-12T20:47:23.154 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1cbe407a-f90d-11ee-bc8d-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-04-12T20:47:23.240 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:47:22 smithi018 ceph-mon[28799]: from='osd.0 [v2:172.21.15.18:6802/585613146,v1:172.21.15.18:6803/585613146]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi018", "root=default"]}]': finished 2024-04-12T20:47:23.241 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:47:22 smithi018 ceph-mon[28799]: osdmap e8: 1 total, 0 up, 1 in 2024-04-12T20:47:23.241 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:47:22 smithi018 ceph-mon[28799]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-12T20:47:23.241 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:47:22 smithi018 ceph-mon[28799]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-12T20:47:23.241 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:47:22 smithi018 ceph-mon[28799]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' 2024-04-12T20:47:23.241 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:47:22 smithi018 ceph-mon[28799]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' 2024-04-12T20:47:23.282 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:47:22 smithi186 ceph-mon[36597]: from='osd.0 [v2:172.21.15.18:6802/585613146,v1:172.21.15.18:6803/585613146]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi018", "root=default"]}]': finished 2024-04-12T20:47:23.282 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:47:22 smithi186 ceph-mon[36597]: osdmap e8: 1 total, 0 up, 1 in 2024-04-12T20:47:23.282 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:47:22 smithi186 ceph-mon[36597]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-12T20:47:23.282 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:47:22 smithi186 ceph-mon[36597]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-12T20:47:23.282 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:47:22 smithi186 ceph-mon[36597]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' 2024-04-12T20:47:23.282 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:47:22 smithi186 ceph-mon[36597]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' 2024-04-12T20:47:23.691 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/mon.smithi018/config 2024-04-12T20:47:24.101 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:47:23 smithi018 ceph-mon[28799]: purged_snaps scrub starts 2024-04-12T20:47:24.101 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:47:23 smithi018 ceph-mon[28799]: purged_snaps scrub ok 2024-04-12T20:47:24.101 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:47:23 smithi018 ceph-mon[28799]: pgmap v45: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-12T20:47:24.101 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:47:23 smithi018 ceph-mon[28799]: osd.0 [v2:172.21.15.18:6802/585613146,v1:172.21.15.18:6803/585613146] boot 2024-04-12T20:47:24.101 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:47:23 smithi018 ceph-mon[28799]: osdmap e9: 1 total, 1 up, 1 in 2024-04-12T20:47:24.101 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:47:23 smithi018 ceph-mon[28799]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-12T20:47:24.101 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:47:23 smithi018 ceph-mon[28799]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' 2024-04-12T20:47:24.101 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:47:23 smithi018 ceph-mon[28799]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' 2024-04-12T20:47:24.282 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:47:23 smithi186 ceph-mon[36597]: purged_snaps scrub starts 2024-04-12T20:47:24.282 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:47:23 smithi186 ceph-mon[36597]: purged_snaps scrub ok 2024-04-12T20:47:24.283 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:47:23 smithi186 ceph-mon[36597]: pgmap v45: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-12T20:47:24.283 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:47:23 smithi186 ceph-mon[36597]: osd.0 [v2:172.21.15.18:6802/585613146,v1:172.21.15.18:6803/585613146] boot 2024-04-12T20:47:24.283 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:47:23 smithi186 ceph-mon[36597]: osdmap e9: 1 total, 1 up, 1 in 2024-04-12T20:47:24.283 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:47:23 smithi186 ceph-mon[36597]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-12T20:47:24.283 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:47:23 smithi186 ceph-mon[36597]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' 2024-04-12T20:47:24.283 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:47:23 smithi186 ceph-mon[36597]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' 2024-04-12T20:47:25.342 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:47:25 smithi018 ceph-mon[28799]: osdmap e10: 1 total, 1 up, 1 in 2024-04-12T20:47:25.456 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:47:25 smithi186 ceph-mon[36597]: osdmap e10: 1 total, 1 up, 1 in 2024-04-12T20:47:26.255 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:47:26 smithi018 ceph-mon[28799]: pgmap v48: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-04-12T20:47:26.532 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:47:26 smithi186 ceph-mon[36597]: pgmap v48: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-04-12T20:47:26.579 INFO:teuthology.orchestra.run.smithi018.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-04-12T20:47:26.579 INFO:teuthology.orchestra.run.smithi018.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-04-12T20:47:26.579 INFO:teuthology.orchestra.run.smithi018.stderr: stderr: 10+0 records in 2024-04-12T20:47:26.579 INFO:teuthology.orchestra.run.smithi018.stderr:10+0 records out 2024-04-12T20:47:26.579 INFO:teuthology.orchestra.run.smithi018.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0348183 s, 301 MB/s 2024-04-12T20:47:26.579 INFO:teuthology.orchestra.run.smithi018.stderr:--> Zapping successful for: 2024-04-12T20:47:27.795 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1cbe407a-f90d-11ee-bc8d-c7b262605968 -- ceph orch daemon add osd smithi018:vg_nvme/lv_3 2024-04-12T20:47:28.331 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:47:28 smithi018 ceph-mon[28799]: pgmap v49: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-04-12T20:47:28.335 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:47:28 smithi186 ceph-mon[36597]: pgmap v49: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-04-12T20:47:28.486 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/mon.smithi018/config 2024-04-12T20:47:30.282 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:47:30 smithi018 ceph-mon[28799]: pgmap v50: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-04-12T20:47:30.282 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:47:30 smithi018 ceph-mon[28799]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T20:47:30.282 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:47:30 smithi186 ceph-mon[36597]: pgmap v50: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-04-12T20:47:30.282 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:47:30 smithi186 ceph-mon[36597]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T20:47:31.012 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:47:31.011+0000 7fd95a000700 1 -- 172.21.15.18:0/577972178 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd954071ea0 msgr2=0x7fd9540703b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:47:31.012 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:47:31.011+0000 7fd95a000700 1 --2- 172.21.15.18:0/577972178 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd954071ea0 0x7fd9540703b0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fd944004940 tx=0x7fd944009e30 comp rx=0 tx=0).stop 2024-04-12T20:47:31.012 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:47:31.012+0000 7fd95a000700 1 -- 172.21.15.18:0/577972178 shutdown_connections 2024-04-12T20:47:31.012 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:47:31.012+0000 7fd95a000700 1 --2- 172.21.15.18:0/577972178 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd9540708f0 0x7fd954070d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:47:31.012 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:47:31.012+0000 7fd95a000700 1 --2- 172.21.15.18:0/577972178 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd954071ea0 0x7fd9540703b0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:47:31.013 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:47:31.012+0000 7fd95a000700 1 -- 172.21.15.18:0/577972178 >> 172.21.15.18:0/577972178 conn(0x7fd9540f8550 msgr2=0x7fd9540fa960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T20:47:31.013 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:47:31.012+0000 7fd95a000700 1 -- 172.21.15.18:0/577972178 shutdown_connections 2024-04-12T20:47:31.013 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:47:31.012+0000 7fd95a000700 1 -- 172.21.15.18:0/577972178 wait complete. 2024-04-12T20:47:31.013 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:47:31.013+0000 7fd95a000700 1 Processor -- start 2024-04-12T20:47:31.014 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:47:31.013+0000 7fd95a000700 1 -- start start 2024-04-12T20:47:31.014 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:47:31.013+0000 7fd95a000700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd9540708f0 0x7fd95410fb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T20:47:31.014 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:47:31.013+0000 7fd95a000700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd954071ea0 0x7fd9541100d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T20:47:31.015 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:47:31.013+0000 7fd95a000700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9541106d0 con 0x7fd9540708f0 2024-04-12T20:47:31.015 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:47:31.013+0000 7fd95a000700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd954110840 con 0x7fd954071ea0 2024-04-12T20:47:31.015 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:47:31.014+0000 7fd953400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd9540708f0 0x7fd95410fb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T20:47:31.015 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:47:31.014+0000 7fd953400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd9540708f0 0x7fd95410fb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:53608/0 (socket says 172.21.15.18:53608) 2024-04-12T20:47:31.015 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:47:31.014+0000 7fd953400700 1 -- 172.21.15.18:0/475456907 learned_addr learned my addr 172.21.15.18:0/475456907 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T20:47:31.015 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:47:31.014+0000 7fd952a00700 1 --2- 172.21.15.18:0/475456907 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd954071ea0 0x7fd9541100d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T20:47:31.015 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:47:31.014+0000 7fd953400700 1 -- 172.21.15.18:0/475456907 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd954071ea0 msgr2=0x7fd9541100d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:47:31.015 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:47:31.014+0000 7fd953400700 1 --2- 172.21.15.18:0/475456907 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd954071ea0 0x7fd9541100d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:47:31.015 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:47:31.014+0000 7fd953400700 1 -- 172.21.15.18:0/475456907 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd944005040 con 0x7fd9540708f0 2024-04-12T20:47:31.015 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:47:31.014+0000 7fd952a00700 1 --2- 172.21.15.18:0/475456907 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd954071ea0 0x7fd9541100d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-12T20:47:31.015 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:47:31.014+0000 7fd953400700 1 --2- 172.21.15.18:0/475456907 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd9540708f0 0x7fd95410fb90 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fd94400b970 tx=0x7fd944009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T20:47:31.015 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:47:31.015+0000 7fd94b400700 1 -- 172.21.15.18:0/475456907 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd94400e040 con 0x7fd9540708f0 2024-04-12T20:47:31.015 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:47:31.015+0000 7fd94b400700 1 -- 172.21.15.18:0/475456907 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd944004290 con 0x7fd9540708f0 2024-04-12T20:47:31.015 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:47:31.015+0000 7fd94b400700 1 -- 172.21.15.18:0/475456907 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd94401c630 con 0x7fd9540708f0 2024-04-12T20:47:31.016 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:47:31.015+0000 7fd95a000700 1 -- 172.21.15.18:0/475456907 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd954115240 con 0x7fd9540708f0 2024-04-12T20:47:31.016 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:47:31.015+0000 7fd95a000700 1 -- 172.21.15.18:0/475456907 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd954115800 con 0x7fd9540708f0 2024-04-12T20:47:31.016 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:47:31.016+0000 7fd95a000700 1 -- 172.21.15.18:0/475456907 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd95404c710 con 0x7fd9540708f0 2024-04-12T20:47:31.016 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:47:31.017+0000 7fd94b400700 1 -- 172.21.15.18:0/475456907 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 18) v1 ==== 86565+0+0 (secure 0 0 0) 0x7fd94400bbf0 con 0x7fd9540708f0 2024-04-12T20:47:31.017 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:47:31.018+0000 7fd94b400700 1 --2- 172.21.15.18:0/475456907 >> [v2:172.21.15.18:6800/3048894687,v1:172.21.15.18:6801/3048894687] conn(0x7fd938069210 0x7fd93806b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T20:47:31.018 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:47:31.018+0000 7fd94b400700 1 -- 172.21.15.18:0/475456907 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(10..10 src has 1..10) v4 ==== 1978+0+0 (secure 0 0 0) 0x7fd944087660 con 0x7fd9540708f0 2024-04-12T20:47:31.018 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:47:31.018+0000 7fd952a00700 1 --2- 172.21.15.18:0/475456907 >> [v2:172.21.15.18:6800/3048894687,v1:172.21.15.18:6801/3048894687] conn(0x7fd938069210 0x7fd93806b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T20:47:31.018 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:47:31.019+0000 7fd952a00700 1 --2- 172.21.15.18:0/475456907 >> [v2:172.21.15.18:6800/3048894687,v1:172.21.15.18:6801/3048894687] conn(0x7fd938069210 0x7fd93806b6c0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fd940009fd0 tx=0x7fd94000b040 comp rx=0 tx=0).ready entity=mgr.14212 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T20:47:31.022 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:47:31.023+0000 7fd94b400700 1 -- 172.21.15.18:0/475456907 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7fd944058280 con 0x7fd9540708f0 2024-04-12T20:47:31.253 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:47:31.253+0000 7fd95a000700 1 -- 172.21.15.18:0/475456907 --> [v2:172.21.15.18:6800/3048894687,v1:172.21.15.18:6801/3048894687] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi018:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) v1 -- 0x7fd954103d10 con 0x7fd938069210 2024-04-12T20:47:32.294 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:47:32 smithi018 ceph-mon[28799]: pgmap v51: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-04-12T20:47:32.294 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:47:32 smithi018 ceph-mon[28799]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-12T20:47:32.294 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:47:32 smithi018 ceph-mon[28799]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-12T20:47:32.294 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:47:32 smithi018 ceph-mon[28799]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-12T20:47:32.294 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:47:32 smithi018 ceph-mon[28799]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' 2024-04-12T20:47:32.294 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:47:32 smithi018 ceph-mon[28799]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' 2024-04-12T20:47:32.294 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:47:32 smithi018 ceph-mon[28799]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-12T20:47:32.295 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:47:32 smithi018 ceph-mon[28799]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' 2024-04-12T20:47:32.295 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:47:32 smithi018 ceph-mon[28799]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-12T20:47:32.295 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:47:32 smithi018 ceph-mon[28799]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-12T20:47:32.295 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:47:32 smithi018 ceph-mon[28799]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' 2024-04-12T20:47:32.295 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:47:32 smithi018 ceph-mon[28799]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-12T20:47:32.532 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:47:32 smithi186 ceph-mon[36597]: pgmap v51: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-04-12T20:47:32.532 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:47:32 smithi186 ceph-mon[36597]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-12T20:47:32.532 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:47:32 smithi186 ceph-mon[36597]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-12T20:47:32.532 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:47:32 smithi186 ceph-mon[36597]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-12T20:47:32.533 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:47:32 smithi186 ceph-mon[36597]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' 2024-04-12T20:47:32.533 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:47:32 smithi186 ceph-mon[36597]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' 2024-04-12T20:47:32.533 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:47:32 smithi186 ceph-mon[36597]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-12T20:47:32.533 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:47:32 smithi186 ceph-mon[36597]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' 2024-04-12T20:47:32.533 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:47:32 smithi186 ceph-mon[36597]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-12T20:47:32.533 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:47:32 smithi186 ceph-mon[36597]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-12T20:47:32.533 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:47:32 smithi186 ceph-mon[36597]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' 2024-04-12T20:47:32.533 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:47:32 smithi186 ceph-mon[36597]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-12T20:47:33.325 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:47:33 smithi018 ceph-mon[28799]: from='client.14266 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi018:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T20:47:33.325 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:47:33 smithi018 ceph-mon[28799]: Detected new or changed devices on smithi018 2024-04-12T20:47:33.325 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:47:33 smithi018 ceph-mon[28799]: Adjusting osd_memory_target on smithi018 to 12870M 2024-04-12T20:47:33.532 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:47:33 smithi186 ceph-mon[36597]: from='client.14266 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi018:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T20:47:33.532 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:47:33 smithi186 ceph-mon[36597]: Detected new or changed devices on smithi018 2024-04-12T20:47:33.532 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:47:33 smithi186 ceph-mon[36597]: Adjusting osd_memory_target on smithi018 to 12870M 2024-04-12T20:47:34.353 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:47:34 smithi018 ceph-mon[28799]: pgmap v52: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-04-12T20:47:34.532 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:47:34 smithi186 ceph-mon[36597]: pgmap v52: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-04-12T20:47:35.782 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:47:35 smithi186 ceph-mon[36597]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' 2024-04-12T20:47:35.782 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:47:35 smithi186 ceph-mon[36597]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' 2024-04-12T20:47:35.782 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:47:35 smithi186 ceph-mon[36597]: pgmap v53: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-04-12T20:47:35.853 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:47:35 smithi018 ceph-mon[28799]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' 2024-04-12T20:47:35.853 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:47:35 smithi018 ceph-mon[28799]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' 2024-04-12T20:47:35.853 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:47:35 smithi018 ceph-mon[28799]: pgmap v53: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-04-12T20:47:36.752 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:47:36 smithi018 ceph-mon[28799]: from='client.? 172.21.15.18:0/2596669692' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6ba7b0d3-a03d-45cc-b4e9-89a92c8c6ca8"}]: dispatch 2024-04-12T20:47:36.752 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:47:36 smithi018 ceph-mon[28799]: from='client.? 172.21.15.18:0/2596669692' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "6ba7b0d3-a03d-45cc-b4e9-89a92c8c6ca8"}]': finished 2024-04-12T20:47:36.752 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:47:36 smithi018 ceph-mon[28799]: osdmap e11: 2 total, 1 up, 2 in 2024-04-12T20:47:36.752 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:47:36 smithi018 ceph-mon[28799]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-12T20:47:36.782 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:47:36 smithi186 ceph-mon[36597]: from='client.? 172.21.15.18:0/2596669692' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6ba7b0d3-a03d-45cc-b4e9-89a92c8c6ca8"}]: dispatch 2024-04-12T20:47:36.782 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:47:36 smithi186 ceph-mon[36597]: from='client.? 172.21.15.18:0/2596669692' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "6ba7b0d3-a03d-45cc-b4e9-89a92c8c6ca8"}]': finished 2024-04-12T20:47:36.782 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:47:36 smithi186 ceph-mon[36597]: osdmap e11: 2 total, 1 up, 2 in 2024-04-12T20:47:36.782 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:47:36 smithi186 ceph-mon[36597]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-12T20:47:37.763 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:47:37 smithi018 ceph-mon[28799]: from='client.? 172.21.15.18:0/59686754' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-12T20:47:37.763 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:47:37 smithi018 ceph-mon[28799]: pgmap v55: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-04-12T20:47:37.782 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:47:37 smithi186 ceph-mon[36597]: from='client.? 172.21.15.18:0/59686754' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-12T20:47:37.782 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:47:37 smithi186 ceph-mon[36597]: pgmap v55: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-04-12T20:47:39.682 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:47:39 smithi018 ceph-mon[28799]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' 2024-04-12T20:47:39.682 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:47:39 smithi018 ceph-mon[28799]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' 2024-04-12T20:47:39.682 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:47:39 smithi018 ceph-mon[28799]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-12T20:47:39.682 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:47:39 smithi018 ceph-mon[28799]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-12T20:47:39.683 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:47:39 smithi018 ceph-mon[28799]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' 2024-04-12T20:47:39.683 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:47:39 smithi018 ceph-mon[28799]: pgmap v56: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-04-12T20:47:39.782 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:47:39 smithi186 ceph-mon[36597]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' 2024-04-12T20:47:39.782 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:47:39 smithi186 ceph-mon[36597]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' 2024-04-12T20:47:39.782 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:47:39 smithi186 ceph-mon[36597]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-12T20:47:39.782 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:47:39 smithi186 ceph-mon[36597]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-12T20:47:39.782 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:47:39 smithi186 ceph-mon[36597]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' 2024-04-12T20:47:39.782 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:47:39 smithi186 ceph-mon[36597]: pgmap v56: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-04-12T20:47:42.103 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:47:41 smithi018 ceph-mon[28799]: pgmap v57: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-04-12T20:47:42.282 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:47:41 smithi186 ceph-mon[36597]: pgmap v57: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-04-12T20:47:44.103 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:47:43 smithi018 ceph-mon[28799]: pgmap v58: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-04-12T20:47:44.282 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:47:43 smithi186 ceph-mon[36597]: pgmap v58: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-04-12T20:47:45.282 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:47:44 smithi186 ceph-mon[36597]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-04-12T20:47:45.282 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:47:44 smithi186 ceph-mon[36597]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-12T20:47:45.282 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:47:44 smithi186 ceph-mon[36597]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T20:47:45.313 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:47:44 smithi018 ceph-mon[28799]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-04-12T20:47:45.313 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:47:44 smithi018 ceph-mon[28799]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-12T20:47:45.314 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:47:44 smithi018 ceph-mon[28799]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T20:47:46.103 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:47:45 smithi018 ceph-mon[28799]: Deploying daemon osd.1 on smithi018 2024-04-12T20:47:46.103 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:47:45 smithi018 ceph-mon[28799]: pgmap v59: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-04-12T20:47:46.282 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:47:45 smithi186 ceph-mon[36597]: Deploying daemon osd.1 on smithi018 2024-04-12T20:47:46.282 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:47:45 smithi186 ceph-mon[36597]: pgmap v59: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-04-12T20:47:48.282 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:47:47 smithi186 ceph-mon[36597]: pgmap v60: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-04-12T20:47:48.353 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:47:47 smithi018 ceph-mon[28799]: pgmap v60: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-04-12T20:47:50.045 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:47:49 smithi018 ceph-mon[28799]: pgmap v61: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-04-12T20:47:50.282 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:47:49 smithi186 ceph-mon[36597]: pgmap v61: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-04-12T20:47:52.036 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:47:51 smithi018 ceph-mon[28799]: pgmap v62: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-04-12T20:47:52.282 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:47:51 smithi186 ceph-mon[36597]: pgmap v62: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-04-12T20:47:53.652 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:47:53 smithi018 ceph-mon[28799]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' 2024-04-12T20:47:53.652 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:47:53 smithi018 ceph-mon[28799]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-12T20:47:53.653 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:47:53 smithi018 ceph-mon[28799]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' 2024-04-12T20:47:53.653 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:47:53 smithi018 ceph-mon[28799]: pgmap v63: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-04-12T20:47:54.032 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:47:53 smithi186 ceph-mon[36597]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' 2024-04-12T20:47:54.032 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:47:53 smithi186 ceph-mon[36597]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-12T20:47:54.032 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:47:53 smithi186 ceph-mon[36597]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' 2024-04-12T20:47:54.032 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:47:53 smithi186 ceph-mon[36597]: pgmap v63: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-04-12T20:47:56.103 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:47:55 smithi018 ceph-mon[28799]: pgmap v64: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-04-12T20:47:56.103 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:47:55 smithi018 ceph-mon[28799]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' 2024-04-12T20:47:56.103 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:47:55 smithi018 ceph-mon[28799]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' 2024-04-12T20:47:56.282 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:47:55 smithi186 ceph-mon[36597]: pgmap v64: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-04-12T20:47:56.282 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:47:55 smithi186 ceph-mon[36597]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' 2024-04-12T20:47:56.282 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:47:55 smithi186 ceph-mon[36597]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' 2024-04-12T20:47:57.054 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:47:56 smithi018 ceph-mon[28799]: from='osd.1 [v2:172.21.15.18:6810/181951035,v1:172.21.15.18:6811/181951035]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-12T20:47:57.282 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:47:56 smithi186 ceph-mon[36597]: from='osd.1 [v2:172.21.15.18:6810/181951035,v1:172.21.15.18:6811/181951035]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-12T20:47:57.604 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:47:57.604+0000 7fd94b400700 1 -- 172.21.15.18:0/475456907 <== mgr.14212 v2:172.21.15.18:6800/3048894687 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7fd954103d10 con 0x7fd938069210 2024-04-12T20:47:57.605 INFO:teuthology.orchestra.run.smithi018.stdout:Created osd(s) 1 on host 'smithi018' 2024-04-12T20:47:57.608 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:47:57.608+0000 7fd95a000700 1 -- 172.21.15.18:0/475456907 >> [v2:172.21.15.18:6800/3048894687,v1:172.21.15.18:6801/3048894687] conn(0x7fd938069210 msgr2=0x7fd93806b6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:47:57.608 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:47:57.608+0000 7fd95a000700 1 --2- 172.21.15.18:0/475456907 >> [v2:172.21.15.18:6800/3048894687,v1:172.21.15.18:6801/3048894687] conn(0x7fd938069210 0x7fd93806b6c0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fd940009fd0 tx=0x7fd94000b040 comp rx=0 tx=0).stop 2024-04-12T20:47:57.608 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:47:57.608+0000 7fd95a000700 1 -- 172.21.15.18:0/475456907 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd9540708f0 msgr2=0x7fd95410fb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:47:57.608 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:47:57.608+0000 7fd95a000700 1 --2- 172.21.15.18:0/475456907 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd9540708f0 0x7fd95410fb90 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fd94400b970 tx=0x7fd944009e30 comp rx=0 tx=0).stop 2024-04-12T20:47:57.608 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:47:57.608+0000 7fd95a000700 1 -- 172.21.15.18:0/475456907 shutdown_connections 2024-04-12T20:47:57.608 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:47:57.608+0000 7fd95a000700 1 --2- 172.21.15.18:0/475456907 >> [v2:172.21.15.18:6800/3048894687,v1:172.21.15.18:6801/3048894687] conn(0x7fd938069210 0x7fd93806b6c0 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:47:57.608 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:47:57.608+0000 7fd95a000700 1 --2- 172.21.15.18:0/475456907 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd9540708f0 0x7fd95410fb90 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:47:57.608 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:47:57.608+0000 7fd95a000700 1 --2- 172.21.15.18:0/475456907 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd954071ea0 0x7fd9541100d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:47:57.608 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:47:57.608+0000 7fd95a000700 1 -- 172.21.15.18:0/475456907 >> 172.21.15.18:0/475456907 conn(0x7fd9540f8550 msgr2=0x7fd9540fa030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T20:47:57.609 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:47:57.609+0000 7fd95a000700 1 -- 172.21.15.18:0/475456907 shutdown_connections 2024-04-12T20:47:57.609 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:47:57.609+0000 7fd95a000700 1 -- 172.21.15.18:0/475456907 wait complete. 2024-04-12T20:47:58.184 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:47:57 smithi186 ceph-mon[36597]: pgmap v65: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-04-12T20:47:58.184 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:47:57 smithi186 ceph-mon[36597]: from='osd.1 [v2:172.21.15.18:6810/181951035,v1:172.21.15.18:6811/181951035]' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-04-12T20:47:58.184 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:47:57 smithi186 ceph-mon[36597]: osdmap e12: 2 total, 1 up, 2 in 2024-04-12T20:47:58.184 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:47:57 smithi186 ceph-mon[36597]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-12T20:47:58.184 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:47:57 smithi186 ceph-mon[36597]: from='osd.1 [v2:172.21.15.18:6810/181951035,v1:172.21.15.18:6811/181951035]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi018", "root=default"]}]: dispatch 2024-04-12T20:47:58.184 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:47:57 smithi186 ceph-mon[36597]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' 2024-04-12T20:47:58.185 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:47:57 smithi186 ceph-mon[36597]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' 2024-04-12T20:47:58.272 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:47:57 smithi018 ceph-mon[28799]: pgmap v65: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-04-12T20:47:58.272 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:47:57 smithi018 ceph-mon[28799]: from='osd.1 [v2:172.21.15.18:6810/181951035,v1:172.21.15.18:6811/181951035]' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-04-12T20:47:58.272 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:47:57 smithi018 ceph-mon[28799]: osdmap e12: 2 total, 1 up, 2 in 2024-04-12T20:47:58.272 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:47:57 smithi018 ceph-mon[28799]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-12T20:47:58.272 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:47:57 smithi018 ceph-mon[28799]: from='osd.1 [v2:172.21.15.18:6810/181951035,v1:172.21.15.18:6811/181951035]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi018", "root=default"]}]: dispatch 2024-04-12T20:47:58.272 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:47:57 smithi018 ceph-mon[28799]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' 2024-04-12T20:47:58.272 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:47:57 smithi018 ceph-mon[28799]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' 2024-04-12T20:47:59.072 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:47:58 smithi018 ceph-mon[28799]: from='osd.1 [v2:172.21.15.18:6810/181951035,v1:172.21.15.18:6811/181951035]' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi018", "root=default"]}]': finished 2024-04-12T20:47:59.072 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:47:58 smithi018 ceph-mon[28799]: osdmap e13: 2 total, 1 up, 2 in 2024-04-12T20:47:59.072 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:47:58 smithi018 ceph-mon[28799]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-12T20:47:59.072 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:47:58 smithi018 ceph-mon[28799]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-12T20:47:59.148 DEBUG:teuthology.orchestra.run.smithi018:osd.1> sudo journalctl -f -n 0 -u ceph-1cbe407a-f90d-11ee-bc8d-c7b262605968@osd.1.service 2024-04-12T20:47:59.154 INFO:tasks.cephadm:Deploying osd.2 on smithi018 with /dev/vg_nvme/lv_2... 2024-04-12T20:47:59.154 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1cbe407a-f90d-11ee-bc8d-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-04-12T20:47:59.211 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:47:58 smithi186 ceph-mon[36597]: from='osd.1 [v2:172.21.15.18:6810/181951035,v1:172.21.15.18:6811/181951035]' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi018", "root=default"]}]': finished 2024-04-12T20:47:59.211 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:47:58 smithi186 ceph-mon[36597]: osdmap e13: 2 total, 1 up, 2 in 2024-04-12T20:47:59.211 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:47:58 smithi186 ceph-mon[36597]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-12T20:47:59.212 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:47:58 smithi186 ceph-mon[36597]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-12T20:48:00.076 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/mon.smithi018/config 2024-04-12T20:48:00.228 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:47:59 smithi018 ceph-mon[28799]: purged_snaps scrub starts 2024-04-12T20:48:00.228 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:47:59 smithi018 ceph-mon[28799]: purged_snaps scrub ok 2024-04-12T20:48:00.228 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:47:59 smithi018 ceph-mon[28799]: pgmap v68: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-04-12T20:48:00.228 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:47:59 smithi018 ceph-mon[28799]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-12T20:48:00.228 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:47:59 smithi018 ceph-mon[28799]: osd.1 [v2:172.21.15.18:6810/181951035,v1:172.21.15.18:6811/181951035] boot 2024-04-12T20:48:00.228 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:47:59 smithi018 ceph-mon[28799]: osdmap e14: 2 total, 2 up, 2 in 2024-04-12T20:48:00.228 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:47:59 smithi018 ceph-mon[28799]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-12T20:48:00.228 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:47:59 smithi018 ceph-mon[28799]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T20:48:00.282 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:47:59 smithi186 ceph-mon[36597]: purged_snaps scrub starts 2024-04-12T20:48:00.282 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:47:59 smithi186 ceph-mon[36597]: purged_snaps scrub ok 2024-04-12T20:48:00.282 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:47:59 smithi186 ceph-mon[36597]: pgmap v68: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-04-12T20:48:00.282 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:47:59 smithi186 ceph-mon[36597]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-12T20:48:00.282 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:47:59 smithi186 ceph-mon[36597]: osd.1 [v2:172.21.15.18:6810/181951035,v1:172.21.15.18:6811/181951035] boot 2024-04-12T20:48:00.282 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:47:59 smithi186 ceph-mon[36597]: osdmap e14: 2 total, 2 up, 2 in 2024-04-12T20:48:00.283 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:47:59 smithi186 ceph-mon[36597]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-12T20:48:00.283 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:47:59 smithi186 ceph-mon[36597]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T20:48:01.022 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:48:00 smithi018 ceph-mon[28799]: osdmap e15: 2 total, 2 up, 2 in 2024-04-12T20:48:01.032 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:48:00 smithi186 ceph-mon[36597]: osdmap e15: 2 total, 2 up, 2 in 2024-04-12T20:48:02.282 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:48:01 smithi186 ceph-mon[36597]: pgmap v71: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-04-12T20:48:02.294 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:48:01 smithi018 ceph-mon[28799]: pgmap v71: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-04-12T20:48:03.493 INFO:teuthology.orchestra.run.smithi018.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-04-12T20:48:03.493 INFO:teuthology.orchestra.run.smithi018.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-04-12T20:48:03.493 INFO:teuthology.orchestra.run.smithi018.stderr: stderr: 10+0 records in 2024-04-12T20:48:03.493 INFO:teuthology.orchestra.run.smithi018.stderr:10+0 records out 2024-04-12T20:48:03.493 INFO:teuthology.orchestra.run.smithi018.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0341479 s, 307 MB/s 2024-04-12T20:48:03.493 INFO:teuthology.orchestra.run.smithi018.stderr:--> Zapping successful for: 2024-04-12T20:48:04.103 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:48:03 smithi018 ceph-mon[28799]: pgmap v72: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-04-12T20:48:04.104 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:48:03 smithi018 ceph-mon[28799]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' 2024-04-12T20:48:04.104 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:48:03 smithi018 ceph-mon[28799]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' 2024-04-12T20:48:04.282 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:48:03 smithi186 ceph-mon[36597]: pgmap v72: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-04-12T20:48:04.282 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:48:03 smithi186 ceph-mon[36597]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' 2024-04-12T20:48:04.282 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:48:03 smithi186 ceph-mon[36597]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' 2024-04-12T20:48:05.089 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1cbe407a-f90d-11ee-bc8d-c7b262605968 -- ceph orch daemon add osd smithi018:vg_nvme/lv_2 2024-04-12T20:48:05.792 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/mon.smithi018/config 2024-04-12T20:48:05.942 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:48:05 smithi018 ceph-mon[28799]: pgmap v73: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-04-12T20:48:06.282 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:48:05 smithi186 ceph-mon[36597]: pgmap v73: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-04-12T20:48:08.017 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:48:07 smithi018 ceph-mon[28799]: pgmap v74: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-04-12T20:48:08.282 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:48:07 smithi186 ceph-mon[36597]: pgmap v74: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-04-12T20:48:08.486 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:48:08.487+0000 7f01b0200700 1 -- 172.21.15.18:0/2955090260 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f01a80fff70 msgr2=0x7f01a8102350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:48:08.486 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:48:08.487+0000 7f01b0200700 1 --2- 172.21.15.18:0/2955090260 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f01a80fff70 0x7f01a8102350 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f0194004940 tx=0x7f0194009e30 comp rx=0 tx=0).stop 2024-04-12T20:48:08.487 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:48:08.487+0000 7f01b0200700 1 -- 172.21.15.18:0/2955090260 shutdown_connections 2024-04-12T20:48:08.487 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:48:08.487+0000 7f01b0200700 1 --2- 172.21.15.18:0/2955090260 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f01a80fff70 0x7f01a8102350 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:48:08.487 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:48:08.487+0000 7f01b0200700 1 --2- 172.21.15.18:0/2955090260 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f01a8066b10 0x7f01a80ffa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:48:08.487 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:48:08.487+0000 7f01b0200700 1 -- 172.21.15.18:0/2955090260 >> 172.21.15.18:0/2955090260 conn(0x7f01a80f7480 msgr2=0x7f01a80f9890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T20:48:08.487 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:48:08.487+0000 7f01b0200700 1 -- 172.21.15.18:0/2955090260 shutdown_connections 2024-04-12T20:48:08.487 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:48:08.488+0000 7f01b0200700 1 -- 172.21.15.18:0/2955090260 wait complete. 2024-04-12T20:48:08.488 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:48:08.489+0000 7f01b0200700 1 Processor -- start 2024-04-12T20:48:08.488 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:48:08.489+0000 7f01b0200700 1 -- start start 2024-04-12T20:48:08.488 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:48:08.489+0000 7f01b0200700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f01a8066b10 0x7f01a810fc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T20:48:08.488 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:48:08.489+0000 7f01b0200700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f01a80fff70 0x7f01a8110170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T20:48:08.488 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:48:08.489+0000 7f01b0200700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01a8110770 con 0x7f01a8066b10 2024-04-12T20:48:08.488 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:48:08.489+0000 7f01b0200700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01a8110fc0 con 0x7f01a80fff70 2024-04-12T20:48:08.488 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:48:08.489+0000 7f01ad800700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f01a8066b10 0x7f01a810fc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T20:48:08.488 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:48:08.490+0000 7f01ad800700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f01a8066b10 0x7f01a810fc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:34564/0 (socket says 172.21.15.18:34564) 2024-04-12T20:48:08.488 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:48:08.490+0000 7f01ad800700 1 -- 172.21.15.18:0/2148091875 learned_addr learned my addr 172.21.15.18:0/2148091875 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T20:48:08.490 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:48:08.490+0000 7f01ace00700 1 --2- 172.21.15.18:0/2148091875 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f01a80fff70 0x7f01a8110170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T20:48:08.490 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:48:08.490+0000 7f01ad800700 1 -- 172.21.15.18:0/2148091875 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f01a80fff70 msgr2=0x7f01a8110170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:48:08.490 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:48:08.490+0000 7f01ad800700 1 --2- 172.21.15.18:0/2148091875 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f01a80fff70 0x7f01a8110170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:48:08.490 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:48:08.490+0000 7f01ad800700 1 -- 172.21.15.18:0/2148091875 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0194005040 con 0x7f01a8066b10 2024-04-12T20:48:08.490 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:48:08.490+0000 7f01ad800700 1 --2- 172.21.15.18:0/2148091875 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f01a8066b10 0x7f01a810fc30 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f019c00c8e0 tx=0x7f019c0044c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T20:48:08.490 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:48:08.491+0000 7f01a5600700 1 -- 172.21.15.18:0/2148091875 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f019c00a940 con 0x7f01a8066b10 2024-04-12T20:48:08.490 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:48:08.491+0000 7f01a5600700 1 -- 172.21.15.18:0/2148091875 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f019c00ce80 con 0x7f01a8066b10 2024-04-12T20:48:08.490 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:48:08.491+0000 7f01b0200700 1 -- 172.21.15.18:0/2148091875 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f01a81112a0 con 0x7f01a8066b10 2024-04-12T20:48:08.490 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:48:08.491+0000 7f01a5600700 1 -- 172.21.15.18:0/2148091875 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f019c0113a0 con 0x7f01a8066b10 2024-04-12T20:48:08.491 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:48:08.491+0000 7f01b0200700 1 -- 172.21.15.18:0/2148091875 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f01a81117e0 con 0x7f01a8066b10 2024-04-12T20:48:08.491 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:48:08.492+0000 7f01b0200700 1 -- 172.21.15.18:0/2148091875 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f01a805f5f0 con 0x7f01a8066b10 2024-04-12T20:48:08.491 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:48:08.493+0000 7f01a5600700 1 -- 172.21.15.18:0/2148091875 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 18) v1 ==== 86565+0+0 (secure 0 0 0) 0x7f019c012400 con 0x7f01a8066b10 2024-04-12T20:48:08.492 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:48:08.494+0000 7f01a5600700 1 --2- 172.21.15.18:0/2148091875 >> [v2:172.21.15.18:6800/3048894687,v1:172.21.15.18:6801/3048894687] conn(0x7f01900692e0 0x7f019006b790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T20:48:08.493 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:48:08.494+0000 7f01a5600700 1 -- 172.21.15.18:0/2148091875 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(15..15 src has 1..15) v4 ==== 2410+0+0 (secure 0 0 0) 0x7f019c086570 con 0x7f01a8066b10 2024-04-12T20:48:08.493 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:48:08.494+0000 7f01ace00700 1 --2- 172.21.15.18:0/2148091875 >> [v2:172.21.15.18:6800/3048894687,v1:172.21.15.18:6801/3048894687] conn(0x7f01900692e0 0x7f019006b790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T20:48:08.493 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:48:08.495+0000 7f01ace00700 1 --2- 172.21.15.18:0/2148091875 >> [v2:172.21.15.18:6800/3048894687,v1:172.21.15.18:6801/3048894687] conn(0x7f01900692e0 0x7f019006b790 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f019400b3b0 tx=0x7f0194009e30 comp rx=0 tx=0).ready entity=mgr.14212 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T20:48:08.497 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:48:08.499+0000 7f01a5600700 1 -- 172.21.15.18:0/2148091875 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7f019c056fe0 con 0x7f01a8066b10 2024-04-12T20:48:08.724 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:48:08.725+0000 7f01b0200700 1 -- 172.21.15.18:0/2148091875 --> [v2:172.21.15.18:6800/3048894687,v1:172.21.15.18:6801/3048894687] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi018:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) v1 -- 0x7f01a8111b50 con 0x7f01900692e0 2024-04-12T20:48:09.062 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:48:08 smithi018 ceph-mon[28799]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-12T20:48:09.062 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:48:08 smithi018 ceph-mon[28799]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-12T20:48:09.063 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:48:08 smithi018 ceph-mon[28799]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-12T20:48:09.282 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:48:08 smithi186 ceph-mon[36597]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-12T20:48:09.282 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:48:08 smithi186 ceph-mon[36597]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-12T20:48:09.282 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:48:08 smithi186 ceph-mon[36597]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-12T20:48:10.103 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:48:09 smithi018 ceph-mon[28799]: from='client.14284 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi018:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T20:48:10.103 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:48:09 smithi018 ceph-mon[28799]: pgmap v75: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-04-12T20:48:10.103 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:48:09 smithi018 ceph-mon[28799]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' 2024-04-12T20:48:10.103 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:48:09 smithi018 ceph-mon[28799]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' 2024-04-12T20:48:10.103 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:48:09 smithi018 ceph-mon[28799]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-12T20:48:10.104 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:48:09 smithi018 ceph-mon[28799]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-12T20:48:10.104 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:48:09 smithi018 ceph-mon[28799]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' 2024-04-12T20:48:10.104 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:48:09 smithi018 ceph-mon[28799]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-12T20:48:10.104 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:48:09 smithi018 ceph-mon[28799]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-12T20:48:10.104 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:48:09 smithi018 ceph-mon[28799]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' 2024-04-12T20:48:10.104 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:48:09 smithi018 ceph-mon[28799]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-12T20:48:10.213 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:48:09 smithi186 ceph-mon[36597]: from='client.14284 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi018:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T20:48:10.213 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:48:09 smithi186 ceph-mon[36597]: pgmap v75: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-04-12T20:48:10.213 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:48:09 smithi186 ceph-mon[36597]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' 2024-04-12T20:48:10.214 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:48:09 smithi186 ceph-mon[36597]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' 2024-04-12T20:48:10.214 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:48:09 smithi186 ceph-mon[36597]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-12T20:48:10.214 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:48:09 smithi186 ceph-mon[36597]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-12T20:48:10.214 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:48:09 smithi186 ceph-mon[36597]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' 2024-04-12T20:48:10.214 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:48:09 smithi186 ceph-mon[36597]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-12T20:48:10.214 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:48:09 smithi186 ceph-mon[36597]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-12T20:48:10.214 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:48:09 smithi186 ceph-mon[36597]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' 2024-04-12T20:48:10.214 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:48:09 smithi186 ceph-mon[36597]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-12T20:48:10.953 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:48:10 smithi018 ceph-mon[28799]: Detected new or changed devices on smithi018 2024-04-12T20:48:10.953 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:48:10 smithi018 ceph-mon[28799]: Adjusting osd_memory_target on smithi018 to 6435M 2024-04-12T20:48:11.282 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:48:10 smithi186 ceph-mon[36597]: Detected new or changed devices on smithi018 2024-04-12T20:48:11.282 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:48:10 smithi186 ceph-mon[36597]: Adjusting osd_memory_target on smithi018 to 6435M 2024-04-12T20:48:12.103 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:48:11 smithi018 ceph-mon[28799]: pgmap v76: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-04-12T20:48:12.282 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:48:11 smithi186 ceph-mon[36597]: pgmap v76: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-04-12T20:48:13.184 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:48:12 smithi018 ceph-mon[28799]: from='client.? 172.21.15.18:0/2946587017' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5bb1137d-5a3c-40bc-ae08-fc792b8ee547"}]: dispatch 2024-04-12T20:48:13.185 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:48:12 smithi018 ceph-mon[28799]: from='client.? 172.21.15.18:0/2946587017' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "5bb1137d-5a3c-40bc-ae08-fc792b8ee547"}]': finished 2024-04-12T20:48:13.185 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:48:12 smithi018 ceph-mon[28799]: osdmap e16: 3 total, 2 up, 3 in 2024-04-12T20:48:13.185 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:48:12 smithi018 ceph-mon[28799]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-12T20:48:13.282 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:48:12 smithi186 ceph-mon[36597]: from='client.? 172.21.15.18:0/2946587017' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5bb1137d-5a3c-40bc-ae08-fc792b8ee547"}]: dispatch 2024-04-12T20:48:13.283 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:48:12 smithi186 ceph-mon[36597]: from='client.? 172.21.15.18:0/2946587017' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "5bb1137d-5a3c-40bc-ae08-fc792b8ee547"}]': finished 2024-04-12T20:48:13.283 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:48:12 smithi186 ceph-mon[36597]: osdmap e16: 3 total, 2 up, 3 in 2024-04-12T20:48:13.283 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:48:12 smithi186 ceph-mon[36597]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-12T20:48:14.282 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:48:13 smithi186 ceph-mon[36597]: pgmap v78: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-04-12T20:48:14.282 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:48:13 smithi186 ceph-mon[36597]: from='client.? 172.21.15.18:0/1411019748' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-12T20:48:14.353 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:48:13 smithi018 ceph-mon[28799]: pgmap v78: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-04-12T20:48:14.353 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:48:13 smithi018 ceph-mon[28799]: from='client.? 172.21.15.18:0/1411019748' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-12T20:48:15.103 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:48:14 smithi018 ceph-mon[28799]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T20:48:15.282 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:48:14 smithi186 ceph-mon[36597]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T20:48:16.280 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:48:15 smithi018 ceph-mon[28799]: pgmap v79: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-04-12T20:48:16.281 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:48:15 smithi186 ceph-mon[36597]: pgmap v79: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-04-12T20:48:17.532 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:48:17 smithi186 ceph-mon[36597]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' 2024-04-12T20:48:17.532 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:48:17 smithi186 ceph-mon[36597]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' 2024-04-12T20:48:17.532 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:48:17 smithi186 ceph-mon[36597]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-12T20:48:17.533 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:48:17 smithi186 ceph-mon[36597]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-12T20:48:17.533 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:48:17 smithi186 ceph-mon[36597]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' 2024-04-12T20:48:17.533 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:48:17 smithi186 ceph-mon[36597]: pgmap v80: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-04-12T20:48:17.603 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:48:17 smithi018 ceph-mon[28799]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' 2024-04-12T20:48:17.603 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:48:17 smithi018 ceph-mon[28799]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' 2024-04-12T20:48:17.603 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:48:17 smithi018 ceph-mon[28799]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-12T20:48:17.603 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:48:17 smithi018 ceph-mon[28799]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-12T20:48:17.603 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:48:17 smithi018 ceph-mon[28799]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' 2024-04-12T20:48:17.603 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:48:17 smithi018 ceph-mon[28799]: pgmap v80: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-04-12T20:48:20.103 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:48:19 smithi018 ceph-mon[28799]: pgmap v81: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-04-12T20:48:20.282 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:48:19 smithi186 ceph-mon[36597]: pgmap v81: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-04-12T20:48:22.103 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:48:21 smithi018 ceph-mon[28799]: pgmap v82: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-04-12T20:48:22.103 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:48:21 smithi018 ceph-mon[28799]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-04-12T20:48:22.103 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:48:21 smithi018 ceph-mon[28799]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-12T20:48:22.282 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:48:21 smithi186 ceph-mon[36597]: pgmap v82: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-04-12T20:48:22.282 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:48:21 smithi186 ceph-mon[36597]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-04-12T20:48:22.282 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:48:21 smithi186 ceph-mon[36597]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-12T20:48:23.282 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:48:22 smithi186 ceph-mon[36597]: Deploying daemon osd.2 on smithi018 2024-04-12T20:48:23.353 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:48:22 smithi018 ceph-mon[28799]: Deploying daemon osd.2 on smithi018 2024-04-12T20:48:23.963 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:48:23 smithi018 ceph-mon[28799]: pgmap v83: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-04-12T20:48:24.282 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:48:23 smithi186 ceph-mon[36597]: pgmap v83: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-04-12T20:48:26.103 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:48:25 smithi018 ceph-mon[28799]: pgmap v84: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-04-12T20:48:26.282 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:48:25 smithi186 ceph-mon[36597]: pgmap v84: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-04-12T20:48:28.282 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:48:27 smithi186 ceph-mon[36597]: pgmap v85: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-04-12T20:48:28.353 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:48:27 smithi018 ceph-mon[28799]: pgmap v85: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-04-12T20:48:30.282 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:48:29 smithi186 ceph-mon[36597]: pgmap v86: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-04-12T20:48:30.282 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:48:29 smithi186 ceph-mon[36597]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T20:48:30.292 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:48:29 smithi018 ceph-mon[28799]: pgmap v86: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-04-12T20:48:30.292 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:48:29 smithi018 ceph-mon[28799]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T20:48:31.353 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:48:31 smithi018 ceph-mon[28799]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' 2024-04-12T20:48:31.353 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:48:31 smithi018 ceph-mon[28799]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' 2024-04-12T20:48:31.353 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:48:31 smithi018 ceph-mon[28799]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-12T20:48:31.353 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:48:31 smithi018 ceph-mon[28799]: pgmap v87: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-04-12T20:48:31.532 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:48:31 smithi186 ceph-mon[36597]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' 2024-04-12T20:48:31.532 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:48:31 smithi186 ceph-mon[36597]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' 2024-04-12T20:48:31.532 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:48:31 smithi186 ceph-mon[36597]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-12T20:48:31.532 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:48:31 smithi186 ceph-mon[36597]: pgmap v87: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-04-12T20:48:34.151 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:48:33 smithi018 ceph-mon[28799]: pgmap v88: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-04-12T20:48:34.282 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:48:33 smithi186 ceph-mon[36597]: pgmap v88: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-04-12T20:48:35.100 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:48:34 smithi018 ceph-mon[28799]: from='osd.2 [v2:172.21.15.18:6818/3573247809,v1:172.21.15.18:6819/3573247809]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-12T20:48:35.214 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:48:35.215+0000 7f01a5600700 1 -- 172.21.15.18:0/2148091875 <== mgr.14212 v2:172.21.15.18:6800/3048894687 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f01a8111b50 con 0x7f01900692e0 2024-04-12T20:48:35.214 INFO:teuthology.orchestra.run.smithi018.stdout:Created osd(s) 2 on host 'smithi018' 2024-04-12T20:48:35.218 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:48:35.219+0000 7f01b0200700 1 -- 172.21.15.18:0/2148091875 >> [v2:172.21.15.18:6800/3048894687,v1:172.21.15.18:6801/3048894687] conn(0x7f01900692e0 msgr2=0x7f019006b790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:48:35.219 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:48:35.219+0000 7f01b0200700 1 --2- 172.21.15.18:0/2148091875 >> [v2:172.21.15.18:6800/3048894687,v1:172.21.15.18:6801/3048894687] conn(0x7f01900692e0 0x7f019006b790 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f019400b3b0 tx=0x7f0194009e30 comp rx=0 tx=0).stop 2024-04-12T20:48:35.219 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:48:35.219+0000 7f01b0200700 1 -- 172.21.15.18:0/2148091875 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f01a8066b10 msgr2=0x7f01a810fc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:48:35.219 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:48:35.219+0000 7f01b0200700 1 --2- 172.21.15.18:0/2148091875 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f01a8066b10 0x7f01a810fc30 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f019c00c8e0 tx=0x7f019c0044c0 comp rx=0 tx=0).stop 2024-04-12T20:48:35.219 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:48:35.219+0000 7f01b0200700 1 -- 172.21.15.18:0/2148091875 shutdown_connections 2024-04-12T20:48:35.219 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:48:35.219+0000 7f01b0200700 1 --2- 172.21.15.18:0/2148091875 >> [v2:172.21.15.18:6800/3048894687,v1:172.21.15.18:6801/3048894687] conn(0x7f01900692e0 0x7f019006b790 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:48:35.219 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:48:35.219+0000 7f01b0200700 1 --2- 172.21.15.18:0/2148091875 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f01a8066b10 0x7f01a810fc30 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:48:35.220 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:48:35.219+0000 7f01b0200700 1 --2- 172.21.15.18:0/2148091875 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f01a80fff70 0x7f01a8110170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:48:35.220 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:48:35.219+0000 7f01b0200700 1 -- 172.21.15.18:0/2148091875 >> 172.21.15.18:0/2148091875 conn(0x7f01a80f7480 msgr2=0x7f01a80fba30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T20:48:35.220 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:48:35.220+0000 7f01b0200700 1 -- 172.21.15.18:0/2148091875 shutdown_connections 2024-04-12T20:48:35.220 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:48:35.220+0000 7f01b0200700 1 -- 172.21.15.18:0/2148091875 wait complete. 2024-04-12T20:48:35.282 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:48:34 smithi186 ceph-mon[36597]: from='osd.2 [v2:172.21.15.18:6818/3573247809,v1:172.21.15.18:6819/3573247809]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-12T20:48:36.224 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:48:35 smithi018 ceph-mon[28799]: pgmap v89: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-04-12T20:48:36.224 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:48:35 smithi018 ceph-mon[28799]: from='osd.2 [v2:172.21.15.18:6818/3573247809,v1:172.21.15.18:6819/3573247809]' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-04-12T20:48:36.224 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:48:35 smithi018 ceph-mon[28799]: osdmap e17: 3 total, 2 up, 3 in 2024-04-12T20:48:36.224 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:48:35 smithi018 ceph-mon[28799]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-12T20:48:36.224 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:48:35 smithi018 ceph-mon[28799]: from='osd.2 [v2:172.21.15.18:6818/3573247809,v1:172.21.15.18:6819/3573247809]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi018", "root=default"]}]: dispatch 2024-04-12T20:48:36.224 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:48:35 smithi018 ceph-mon[28799]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' 2024-04-12T20:48:36.225 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:48:35 smithi018 ceph-mon[28799]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' 2024-04-12T20:48:36.282 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:48:35 smithi186 ceph-mon[36597]: pgmap v89: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-04-12T20:48:36.282 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:48:35 smithi186 ceph-mon[36597]: from='osd.2 [v2:172.21.15.18:6818/3573247809,v1:172.21.15.18:6819/3573247809]' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-04-12T20:48:36.282 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:48:35 smithi186 ceph-mon[36597]: osdmap e17: 3 total, 2 up, 3 in 2024-04-12T20:48:36.283 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:48:35 smithi186 ceph-mon[36597]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-12T20:48:36.283 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:48:35 smithi186 ceph-mon[36597]: from='osd.2 [v2:172.21.15.18:6818/3573247809,v1:172.21.15.18:6819/3573247809]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi018", "root=default"]}]: dispatch 2024-04-12T20:48:36.283 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:48:35 smithi186 ceph-mon[36597]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' 2024-04-12T20:48:36.283 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:48:35 smithi186 ceph-mon[36597]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' 2024-04-12T20:48:36.288 DEBUG:teuthology.orchestra.run.smithi018:osd.2> sudo journalctl -f -n 0 -u ceph-1cbe407a-f90d-11ee-bc8d-c7b262605968@osd.2.service 2024-04-12T20:48:36.295 INFO:tasks.cephadm:Deploying osd.3 on smithi186 with /dev/vg_nvme/lv_4... 2024-04-12T20:48:36.296 DEBUG:teuthology.orchestra.run.smithi186:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1cbe407a-f90d-11ee-bc8d-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-04-12T20:48:36.767 INFO:teuthology.orchestra.run.smithi186.stderr:Inferring config /var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/mon.smithi186/config 2024-04-12T20:48:37.187 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:48:36 smithi018 ceph-mon[28799]: from='osd.2 [v2:172.21.15.18:6818/3573247809,v1:172.21.15.18:6819/3573247809]' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi018", "root=default"]}]': finished 2024-04-12T20:48:37.187 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:48:36 smithi018 ceph-mon[28799]: osdmap e18: 3 total, 2 up, 3 in 2024-04-12T20:48:37.187 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:48:36 smithi018 ceph-mon[28799]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-12T20:48:37.187 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:48:36 smithi018 ceph-mon[28799]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-12T20:48:37.210 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:48:36 smithi186 ceph-mon[36597]: from='osd.2 [v2:172.21.15.18:6818/3573247809,v1:172.21.15.18:6819/3573247809]' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi018", "root=default"]}]': finished 2024-04-12T20:48:37.210 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:48:36 smithi186 ceph-mon[36597]: osdmap e18: 3 total, 2 up, 3 in 2024-04-12T20:48:37.210 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:48:36 smithi186 ceph-mon[36597]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-12T20:48:37.210 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:48:36 smithi186 ceph-mon[36597]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-12T20:48:38.177 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:48:37 smithi018 ceph-mon[28799]: purged_snaps scrub starts 2024-04-12T20:48:38.177 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:48:37 smithi018 ceph-mon[28799]: purged_snaps scrub ok 2024-04-12T20:48:38.177 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:48:37 smithi018 ceph-mon[28799]: pgmap v92: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-04-12T20:48:38.177 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:48:37 smithi018 ceph-mon[28799]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-12T20:48:38.178 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:48:37 smithi018 ceph-mon[28799]: osd.2 [v2:172.21.15.18:6818/3573247809,v1:172.21.15.18:6819/3573247809] boot 2024-04-12T20:48:38.178 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:48:37 smithi018 ceph-mon[28799]: osdmap e19: 3 total, 3 up, 3 in 2024-04-12T20:48:38.178 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:48:37 smithi018 ceph-mon[28799]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-12T20:48:38.178 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:48:37 smithi018 ceph-mon[28799]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32}]: dispatch 2024-04-12T20:48:38.199 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:48:37 smithi186 ceph-mon[36597]: purged_snaps scrub starts 2024-04-12T20:48:38.199 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:48:37 smithi186 ceph-mon[36597]: purged_snaps scrub ok 2024-04-12T20:48:38.199 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:48:37 smithi186 ceph-mon[36597]: pgmap v92: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-04-12T20:48:38.199 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:48:37 smithi186 ceph-mon[36597]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-12T20:48:38.199 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:48:37 smithi186 ceph-mon[36597]: osd.2 [v2:172.21.15.18:6818/3573247809,v1:172.21.15.18:6819/3573247809] boot 2024-04-12T20:48:38.199 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:48:37 smithi186 ceph-mon[36597]: osdmap e19: 3 total, 3 up, 3 in 2024-04-12T20:48:38.200 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:48:37 smithi186 ceph-mon[36597]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-12T20:48:38.200 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:48:37 smithi186 ceph-mon[36597]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32}]: dispatch 2024-04-12T20:48:39.282 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:48:38 smithi186 ceph-mon[36597]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32}]': finished 2024-04-12T20:48:39.282 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:48:38 smithi186 ceph-mon[36597]: osdmap e20: 3 total, 3 up, 3 in 2024-04-12T20:48:39.282 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:48:38 smithi186 ceph-mon[36597]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-04-12T20:48:39.353 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:48:38 smithi018 ceph-mon[28799]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32}]': finished 2024-04-12T20:48:39.353 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:48:38 smithi018 ceph-mon[28799]: osdmap e20: 3 total, 3 up, 3 in 2024-04-12T20:48:39.353 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:48:38 smithi018 ceph-mon[28799]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-04-12T20:48:39.444 INFO:teuthology.orchestra.run.smithi186.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-04-12T20:48:39.444 INFO:teuthology.orchestra.run.smithi186.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-04-12T20:48:39.444 INFO:teuthology.orchestra.run.smithi186.stderr: stderr: 10+0 records in 2024-04-12T20:48:39.444 INFO:teuthology.orchestra.run.smithi186.stderr:10+0 records out 2024-04-12T20:48:39.444 INFO:teuthology.orchestra.run.smithi186.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0323688 s, 324 MB/s 2024-04-12T20:48:39.444 INFO:teuthology.orchestra.run.smithi186.stderr:--> Zapping successful for: 2024-04-12T20:48:40.092 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:48:39 smithi186 ceph-mon[36597]: pgmap v95: 1 pgs: 1 unknown; 0 B data, 872 MiB used, 267 GiB / 268 GiB avail 2024-04-12T20:48:40.092 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:48:39 smithi186 ceph-mon[36597]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-12T20:48:40.092 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:48:39 smithi186 ceph-mon[36597]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-04-12T20:48:40.092 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:48:39 smithi186 ceph-mon[36597]: osdmap e21: 3 total, 3 up, 3 in 2024-04-12T20:48:40.092 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:48:39 smithi186 ceph-mon[36597]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' 2024-04-12T20:48:40.092 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:48:39 smithi186 ceph-mon[36597]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' 2024-04-12T20:48:40.225 DEBUG:teuthology.orchestra.run.smithi186:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1cbe407a-f90d-11ee-bc8d-c7b262605968 -- ceph orch daemon add osd smithi186:vg_nvme/lv_4 2024-04-12T20:48:40.292 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:48:39 smithi018 ceph-mon[28799]: pgmap v95: 1 pgs: 1 unknown; 0 B data, 872 MiB used, 267 GiB / 268 GiB avail 2024-04-12T20:48:40.293 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:48:39 smithi018 ceph-mon[28799]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-12T20:48:40.293 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:48:39 smithi018 ceph-mon[28799]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-04-12T20:48:40.293 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:48:39 smithi018 ceph-mon[28799]: osdmap e21: 3 total, 3 up, 3 in 2024-04-12T20:48:40.293 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:48:39 smithi018 ceph-mon[28799]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' 2024-04-12T20:48:40.293 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:48:39 smithi018 ceph-mon[28799]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' 2024-04-12T20:48:40.293 INFO:journalctl@ceph.osd.0.smithi018.stdout:Apr 12 20:48:39 smithi018 sudo[63952]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-12T20:48:40.293 INFO:journalctl@ceph.osd.0.smithi018.stdout:Apr 12 20:48:39 smithi018 sudo[63952]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-12T20:48:40.293 INFO:journalctl@ceph.osd.0.smithi018.stdout:Apr 12 20:48:39 smithi018 sudo[63952]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-12T20:48:40.293 INFO:journalctl@ceph.osd.0.smithi018.stdout:Apr 12 20:48:39 smithi018 sudo[63952]: pam_unix(sudo:session): session closed for user root 2024-04-12T20:48:40.548 INFO:journalctl@ceph.osd.0.smithi018.stdout:Apr 12 20:48:40 smithi018 sudo[64031]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-12T20:48:40.548 INFO:journalctl@ceph.osd.0.smithi018.stdout:Apr 12 20:48:40 smithi018 sudo[64031]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-12T20:48:40.548 INFO:journalctl@ceph.osd.0.smithi018.stdout:Apr 12 20:48:40 smithi018 sudo[64031]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-12T20:48:40.548 INFO:journalctl@ceph.osd.0.smithi018.stdout:Apr 12 20:48:40 smithi018 sudo[64031]: pam_unix(sudo:session): session closed for user root 2024-04-12T20:48:40.689 INFO:teuthology.orchestra.run.smithi186.stderr:Inferring config /var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/mon.smithi186/config 2024-04-12T20:48:41.197 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:48:41 smithi018 ceph-mon[28799]: osdmap e22: 3 total, 3 up, 3 in 2024-04-12T20:48:41.198 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:48:41 smithi018 ceph-mon[28799]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' 2024-04-12T20:48:41.198 INFO:journalctl@ceph.osd.1.smithi018.stdout:Apr 12 20:48:41 smithi018 sudo[64088]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-12T20:48:41.198 INFO:journalctl@ceph.osd.1.smithi018.stdout:Apr 12 20:48:41 smithi018 sudo[64088]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-12T20:48:41.198 INFO:journalctl@ceph.osd.1.smithi018.stdout:Apr 12 20:48:41 smithi018 sudo[64088]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-12T20:48:41.532 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:48:41 smithi186 ceph-mon[36597]: osdmap e22: 3 total, 3 up, 3 in 2024-04-12T20:48:41.532 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:48:41 smithi186 ceph-mon[36597]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' 2024-04-12T20:48:41.603 INFO:journalctl@ceph.osd.1.smithi018.stdout:Apr 12 20:48:41 smithi018 sudo[64088]: pam_unix(sudo:session): session closed for user root 2024-04-12T20:48:42.103 INFO:journalctl@ceph.osd.1.smithi018.stdout:Apr 12 20:48:41 smithi018 sudo[64110]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-12T20:48:42.103 INFO:journalctl@ceph.osd.1.smithi018.stdout:Apr 12 20:48:41 smithi018 sudo[64110]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-12T20:48:42.103 INFO:journalctl@ceph.osd.1.smithi018.stdout:Apr 12 20:48:41 smithi018 sudo[64110]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-12T20:48:42.103 INFO:journalctl@ceph.osd.1.smithi018.stdout:Apr 12 20:48:41 smithi018 sudo[64110]: pam_unix(sudo:session): session closed for user root 2024-04-12T20:48:42.236 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:48:42 smithi186 ceph-mon[36597]: pgmap v98: 1 pgs: 1 active+clean; 449 KiB data, 872 MiB used, 267 GiB / 268 GiB avail 2024-04-12T20:48:42.236 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:48:42 smithi186 ceph-mon[36597]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-04-12T20:48:42.236 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:48:42 smithi186 ceph-mon[36597]: Cluster is now healthy 2024-04-12T20:48:42.236 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:48:42 smithi186 ceph-mon[36597]: mgrmap e19: smithi018.xjzwga(active, since 2m), standbys: smithi186.kqcfxk 2024-04-12T20:48:42.417 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:48:42 smithi018 ceph-mon[28799]: pgmap v98: 1 pgs: 1 active+clean; 449 KiB data, 872 MiB used, 267 GiB / 268 GiB avail 2024-04-12T20:48:42.418 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:48:42 smithi018 ceph-mon[28799]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-04-12T20:48:42.418 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:48:42 smithi018 ceph-mon[28799]: Cluster is now healthy 2024-04-12T20:48:42.418 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:48:42 smithi018 ceph-mon[28799]: mgrmap e19: smithi018.xjzwga(active, since 2m), standbys: smithi186.kqcfxk 2024-04-12T20:48:42.669 INFO:journalctl@ceph.osd.2.smithi018.stdout:Apr 12 20:48:42 smithi018 sudo[64133]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-12T20:48:42.669 INFO:journalctl@ceph.osd.2.smithi018.stdout:Apr 12 20:48:42 smithi018 sudo[64133]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-12T20:48:42.670 INFO:journalctl@ceph.osd.2.smithi018.stdout:Apr 12 20:48:42 smithi018 sudo[64133]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-12T20:48:42.670 INFO:journalctl@ceph.osd.2.smithi018.stdout:Apr 12 20:48:42 smithi018 sudo[64133]: pam_unix(sudo:session): session closed for user root 2024-04-12T20:48:42.711 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:48:42.711+0000 7f2340a00700 1 -- 172.21.15.186:0/1549722471 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f233c103420 msgr2=0x7f233c103830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:48:42.711 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:48:42.711+0000 7f2340a00700 1 --2- 172.21.15.186:0/1549722471 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f233c103420 0x7f233c103830 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f232c004940 tx=0x7f232c009e30 comp rx=0 tx=0).stop 2024-04-12T20:48:42.711 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:48:42.711+0000 7f2340a00700 1 -- 172.21.15.186:0/1549722471 shutdown_connections 2024-04-12T20:48:42.711 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:48:42.711+0000 7f2340a00700 1 --2- 172.21.15.186:0/1549722471 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f233c1046d0 0x7f233c104b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:48:42.711 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:48:42.711+0000 7f2340a00700 1 --2- 172.21.15.186:0/1549722471 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f233c103420 0x7f233c103830 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:48:42.712 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:48:42.711+0000 7f2340a00700 1 -- 172.21.15.186:0/1549722471 >> 172.21.15.186:0/1549722471 conn(0x7f233c0725a0 msgr2=0x7f233c0749d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T20:48:42.712 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:48:42.711+0000 7f2340a00700 1 -- 172.21.15.186:0/1549722471 shutdown_connections 2024-04-12T20:48:42.712 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:48:42.711+0000 7f2340a00700 1 -- 172.21.15.186:0/1549722471 wait complete. 2024-04-12T20:48:42.712 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:48:42.713+0000 7f2340a00700 1 Processor -- start 2024-04-12T20:48:42.713 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:48:42.713+0000 7f2340a00700 1 -- start start 2024-04-12T20:48:42.713 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:48:42.713+0000 7f2340a00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f233c103420 0x7f233c113ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T20:48:42.714 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:48:42.713+0000 7f2340a00700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f233c1046d0 0x7f233c114420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T20:48:42.714 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:48:42.713+0000 7f2340a00700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f233c114a20 con 0x7f233c103420 2024-04-12T20:48:42.714 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:48:42.713+0000 7f2340a00700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f233c114b90 con 0x7f233c1046d0 2024-04-12T20:48:42.714 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:48:42.713+0000 7f2339400700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f233c1046d0 0x7f233c114420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T20:48:42.714 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:48:42.714+0000 7f2339400700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f233c1046d0 0x7f233c114420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.186:39226/0 (socket says 172.21.15.186:39226) 2024-04-12T20:48:42.714 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:48:42.714+0000 7f2339400700 1 -- 172.21.15.186:0/407955004 learned_addr learned my addr 172.21.15.186:0/407955004 (peer_addr_for_me v2:172.21.15.186:0/0) 2024-04-12T20:48:42.714 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:48:42.714+0000 7f2339e00700 1 --2- 172.21.15.186:0/407955004 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f233c103420 0x7f233c113ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T20:48:42.714 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:48:42.714+0000 7f2339400700 1 -- 172.21.15.186:0/407955004 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f233c103420 msgr2=0x7f233c113ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:48:42.714 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:48:42.714+0000 7f2339400700 1 --2- 172.21.15.186:0/407955004 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f233c103420 0x7f233c113ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:48:42.714 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:48:42.714+0000 7f2339400700 1 -- 172.21.15.186:0/407955004 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f232c005040 con 0x7f233c1046d0 2024-04-12T20:48:42.714 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:48:42.714+0000 7f2339e00700 1 --2- 172.21.15.186:0/407955004 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f233c103420 0x7f233c113ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-12T20:48:42.714 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:48:42.714+0000 7f2339400700 1 --2- 172.21.15.186:0/407955004 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f233c1046d0 0x7f233c114420 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f2328002a80 tx=0x7f23280046b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T20:48:42.714 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:48:42.715+0000 7f2332000700 1 -- 172.21.15.186:0/407955004 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2328007cc0 con 0x7f233c1046d0 2024-04-12T20:48:42.715 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:48:42.715+0000 7f2332000700 1 -- 172.21.15.186:0/407955004 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f232800f040 con 0x7f233c1046d0 2024-04-12T20:48:42.715 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:48:42.715+0000 7f2332000700 1 -- 172.21.15.186:0/407955004 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f23280096f0 con 0x7f233c1046d0 2024-04-12T20:48:42.715 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:48:42.715+0000 7f2340a00700 1 -- 172.21.15.186:0/407955004 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f233c1195f0 con 0x7f233c1046d0 2024-04-12T20:48:42.715 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:48:42.715+0000 7f2340a00700 1 -- 172.21.15.186:0/407955004 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f233c119bb0 con 0x7f233c1046d0 2024-04-12T20:48:42.715 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:48:42.716+0000 7f2340a00700 1 -- 172.21.15.186:0/407955004 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f233c04c710 con 0x7f233c1046d0 2024-04-12T20:48:42.715 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:48:42.717+0000 7f2332000700 1 -- 172.21.15.186:0/407955004 <== mon.1 v2:172.21.15.186:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7f232800d690 con 0x7f233c1046d0 2024-04-12T20:48:42.716 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:48:42.718+0000 7f2332000700 1 --2- 172.21.15.186:0/407955004 >> [v2:172.21.15.18:6800/3048894687,v1:172.21.15.18:6801/3048894687] conn(0x7f23200692a0 0x7f232006b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T20:48:42.717 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:48:42.718+0000 7f2332000700 1 -- 172.21.15.186:0/407955004 <== mon.1 v2:172.21.15.186:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 3228+0+0 (secure 0 0 0) 0x7f2328085fa0 con 0x7f233c1046d0 2024-04-12T20:48:42.717 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:48:42.718+0000 7f2339e00700 1 --2- 172.21.15.186:0/407955004 >> [v2:172.21.15.18:6800/3048894687,v1:172.21.15.18:6801/3048894687] conn(0x7f23200692a0 0x7f232006b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T20:48:42.718 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:48:42.719+0000 7f2339e00700 1 --2- 172.21.15.186:0/407955004 >> [v2:172.21.15.18:6800/3048894687,v1:172.21.15.18:6801/3048894687] conn(0x7f23200692a0 0x7f232006b750 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f232c00b020 tx=0x7f232c009e30 comp rx=0 tx=0).ready entity=mgr.14212 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T20:48:42.722 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:48:42.723+0000 7f2332000700 1 -- 172.21.15.186:0/407955004 <== mon.1 v2:172.21.15.186:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7f232804d190 con 0x7f233c1046d0 2024-04-12T20:48:42.942 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:48:42.943+0000 7f2340a00700 1 -- 172.21.15.186:0/407955004 --> [v2:172.21.15.18:6800/3048894687,v1:172.21.15.18:6801/3048894687] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi186:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) v1 -- 0x7f233c108970 con 0x7f23200692a0 2024-04-12T20:48:43.353 INFO:journalctl@ceph.osd.2.smithi018.stdout:Apr 12 20:48:43 smithi018 sudo[64144]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-12T20:48:43.353 INFO:journalctl@ceph.osd.2.smithi018.stdout:Apr 12 20:48:43 smithi018 sudo[64144]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-12T20:48:43.353 INFO:journalctl@ceph.osd.2.smithi018.stdout:Apr 12 20:48:43 smithi018 sudo[64144]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-12T20:48:43.353 INFO:journalctl@ceph.osd.2.smithi018.stdout:Apr 12 20:48:43 smithi018 sudo[64144]: pam_unix(sudo:session): session closed for user root 2024-04-12T20:48:44.103 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:48:43 smithi018 sudo[64166]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-12T20:48:44.103 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:48:43 smithi018 sudo[64166]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-12T20:48:44.103 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:48:43 smithi018 sudo[64166]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-12T20:48:44.103 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:48:43 smithi018 sudo[64166]: pam_unix(sudo:session): session closed for user root 2024-04-12T20:48:44.590 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:48:44 smithi018 sudo[66060]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-12T20:48:44.590 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:48:44 smithi018 sudo[66060]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-12T20:48:44.590 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:48:44 smithi018 sudo[66060]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-12T20:48:44.590 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:48:44 smithi018 sudo[66060]: pam_unix(sudo:session): session closed for user root 2024-04-12T20:48:44.590 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:48:44 smithi018 ceph-mon[28799]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-12T20:48:44.590 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:48:44 smithi018 ceph-mon[28799]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-12T20:48:44.590 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:48:44 smithi018 ceph-mon[28799]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-12T20:48:44.590 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:48:44 smithi018 ceph-mon[28799]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-12T20:48:45.029 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:48:44 smithi186 sudo[39862]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-12T20:48:45.029 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:48:44 smithi186 sudo[39862]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-12T20:48:45.029 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:48:44 smithi186 sudo[39862]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-12T20:48:45.029 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:48:44 smithi186 sudo[39862]: pam_unix(sudo:session): session closed for user root 2024-04-12T20:48:45.782 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:48:45 smithi186 sudo[39890]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-12T20:48:45.782 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:48:45 smithi186 sudo[39890]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-12T20:48:45.782 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:48:45 smithi186 sudo[39890]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-12T20:48:45.782 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:48:45 smithi186 sudo[39890]: pam_unix(sudo:session): session closed for user root 2024-04-12T20:48:45.782 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:48:45 smithi186 ceph-mon[36597]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-12T20:48:45.783 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:48:45 smithi186 ceph-mon[36597]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-12T20:48:45.783 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:48:45 smithi186 ceph-mon[36597]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-12T20:48:45.783 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:48:45 smithi186 ceph-mon[36597]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-12T20:48:45.783 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:48:45 smithi186 ceph-mon[36597]: pgmap v99: 1 pgs: 1 active+clean; 449 KiB data, 872 MiB used, 267 GiB / 268 GiB avail 2024-04-12T20:48:45.783 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:48:45 smithi186 ceph-mon[36597]: from='client.24127 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi186:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T20:48:45.783 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:48:45 smithi186 ceph-mon[36597]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-12T20:48:45.783 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:48:45 smithi186 ceph-mon[36597]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "mon metadata", "id": "smithi018"}]: dispatch 2024-04-12T20:48:45.783 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:48:45 smithi186 ceph-mon[36597]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "mon metadata", "id": "smithi186"}]: dispatch 2024-04-12T20:48:45.783 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:48:45 smithi186 ceph-mon[36597]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "mon metadata", "id": "smithi018"}]: dispatch 2024-04-12T20:48:45.783 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:48:45 smithi186 ceph-mon[36597]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "mon metadata", "id": "smithi186"}]: dispatch 2024-04-12T20:48:45.783 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:48:45 smithi186 ceph-mon[36597]: pgmap v100: 1 pgs: 1 active+clean; 449 KiB data, 873 MiB used, 267 GiB / 268 GiB avail 2024-04-12T20:48:45.783 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:48:45 smithi186 ceph-mon[36597]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T20:48:45.853 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:48:45 smithi018 ceph-mon[28799]: pgmap v99: 1 pgs: 1 active+clean; 449 KiB data, 872 MiB used, 267 GiB / 268 GiB avail 2024-04-12T20:48:45.853 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:48:45 smithi018 ceph-mon[28799]: from='client.24127 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi186:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T20:48:45.853 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:48:45 smithi018 ceph-mon[28799]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-12T20:48:45.853 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:48:45 smithi018 ceph-mon[28799]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "mon metadata", "id": "smithi018"}]: dispatch 2024-04-12T20:48:45.853 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:48:45 smithi018 ceph-mon[28799]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "mon metadata", "id": "smithi186"}]: dispatch 2024-04-12T20:48:45.853 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:48:45 smithi018 ceph-mon[28799]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "mon metadata", "id": "smithi018"}]: dispatch 2024-04-12T20:48:45.853 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:48:45 smithi018 ceph-mon[28799]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "mon metadata", "id": "smithi186"}]: dispatch 2024-04-12T20:48:45.854 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:48:45 smithi018 ceph-mon[28799]: pgmap v100: 1 pgs: 1 active+clean; 449 KiB data, 873 MiB used, 267 GiB / 268 GiB avail 2024-04-12T20:48:45.854 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:48:45 smithi018 ceph-mon[28799]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T20:48:46.783 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:48:46 smithi186 ceph-mon[36597]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-12T20:48:46.783 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:48:46 smithi186 ceph-mon[36597]: Detected new or changed devices on smithi018 2024-04-12T20:48:46.783 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:48:46 smithi186 ceph-mon[36597]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-12T20:48:46.783 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:48:46 smithi186 ceph-mon[36597]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' 2024-04-12T20:48:46.783 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:48:46 smithi186 ceph-mon[36597]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' 2024-04-12T20:48:46.783 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:48:46 smithi186 ceph-mon[36597]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-12T20:48:46.783 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:48:46 smithi186 ceph-mon[36597]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-12T20:48:46.783 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:48:46 smithi186 ceph-mon[36597]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-12T20:48:46.783 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:48:46 smithi186 ceph-mon[36597]: Adjusting osd_memory_target on smithi018 to 4290M 2024-04-12T20:48:46.783 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:48:46 smithi186 ceph-mon[36597]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' 2024-04-12T20:48:46.783 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:48:46 smithi186 ceph-mon[36597]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-12T20:48:46.783 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:48:46 smithi186 ceph-mon[36597]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-12T20:48:46.783 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:48:46 smithi186 ceph-mon[36597]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' 2024-04-12T20:48:46.784 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:48:46 smithi186 ceph-mon[36597]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' 2024-04-12T20:48:46.784 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:48:46 smithi186 ceph-mon[36597]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-12T20:48:46.784 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:48:46 smithi186 ceph-mon[36597]: from='client.? 172.21.15.186:0/1358123669' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a5140a33-1e90-4980-aa9a-b0afe6b155f0"}]: dispatch 2024-04-12T20:48:46.784 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:48:46 smithi186 ceph-mon[36597]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a5140a33-1e90-4980-aa9a-b0afe6b155f0"}]: dispatch 2024-04-12T20:48:46.784 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:48:46 smithi186 ceph-mon[36597]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a5140a33-1e90-4980-aa9a-b0afe6b155f0"}]': finished 2024-04-12T20:48:46.784 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:48:46 smithi186 ceph-mon[36597]: osdmap e23: 4 total, 3 up, 4 in 2024-04-12T20:48:46.784 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:48:46 smithi186 ceph-mon[36597]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-12T20:48:46.854 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:48:46 smithi018 ceph-mon[28799]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-12T20:48:46.854 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:48:46 smithi018 ceph-mon[28799]: Detected new or changed devices on smithi018 2024-04-12T20:48:46.854 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:48:46 smithi018 ceph-mon[28799]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-12T20:48:46.854 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:48:46 smithi018 ceph-mon[28799]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' 2024-04-12T20:48:46.854 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:48:46 smithi018 ceph-mon[28799]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' 2024-04-12T20:48:46.854 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:48:46 smithi018 ceph-mon[28799]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-12T20:48:46.854 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:48:46 smithi018 ceph-mon[28799]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-12T20:48:46.854 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:48:46 smithi018 ceph-mon[28799]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-12T20:48:46.854 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:48:46 smithi018 ceph-mon[28799]: Adjusting osd_memory_target on smithi018 to 4290M 2024-04-12T20:48:46.855 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:48:46 smithi018 ceph-mon[28799]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' 2024-04-12T20:48:46.855 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:48:46 smithi018 ceph-mon[28799]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-12T20:48:46.855 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:48:46 smithi018 ceph-mon[28799]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-12T20:48:46.855 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:48:46 smithi018 ceph-mon[28799]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' 2024-04-12T20:48:46.855 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:48:46 smithi018 ceph-mon[28799]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' 2024-04-12T20:48:46.855 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:48:46 smithi018 ceph-mon[28799]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-12T20:48:46.855 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:48:46 smithi018 ceph-mon[28799]: from='client.? 172.21.15.186:0/1358123669' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a5140a33-1e90-4980-aa9a-b0afe6b155f0"}]: dispatch 2024-04-12T20:48:46.855 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:48:46 smithi018 ceph-mon[28799]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a5140a33-1e90-4980-aa9a-b0afe6b155f0"}]: dispatch 2024-04-12T20:48:46.855 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:48:46 smithi018 ceph-mon[28799]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a5140a33-1e90-4980-aa9a-b0afe6b155f0"}]': finished 2024-04-12T20:48:46.855 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:48:46 smithi018 ceph-mon[28799]: osdmap e23: 4 total, 3 up, 4 in 2024-04-12T20:48:46.855 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:48:46 smithi018 ceph-mon[28799]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-12T20:48:47.782 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:48:47 smithi186 ceph-mon[36597]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail; 1.3 KiB/s rd, 78 KiB/s wr, 5 op/s 2024-04-12T20:48:47.783 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:48:47 smithi186 ceph-mon[36597]: from='client.? 172.21.15.186:0/3670430337' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-12T20:48:47.853 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:48:47 smithi018 ceph-mon[28799]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail; 1.3 KiB/s rd, 78 KiB/s wr, 5 op/s 2024-04-12T20:48:47.853 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:48:47 smithi018 ceph-mon[28799]: from='client.? 172.21.15.186:0/3670430337' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-12T20:48:50.103 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:48:49 smithi018 ceph-mon[28799]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail; 1.0 KiB/s rd, 60 KiB/s wr, 3 op/s 2024-04-12T20:48:50.250 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:48:49 smithi186 ceph-mon[36597]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail; 1.0 KiB/s rd, 60 KiB/s wr, 3 op/s 2024-04-12T20:48:52.282 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:48:51 smithi186 ceph-mon[36597]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail; 2.7 KiB/s rd, 131 KiB/s wr, 8 op/s 2024-04-12T20:48:52.282 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:48:51 smithi186 ceph-mon[36597]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' 2024-04-12T20:48:52.282 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:48:51 smithi186 ceph-mon[36597]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' 2024-04-12T20:48:52.282 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:48:51 smithi186 ceph-mon[36597]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-12T20:48:52.282 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:48:51 smithi186 ceph-mon[36597]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-12T20:48:52.282 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:48:51 smithi186 ceph-mon[36597]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' 2024-04-12T20:48:52.353 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:48:51 smithi018 ceph-mon[28799]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail; 2.7 KiB/s rd, 131 KiB/s wr, 8 op/s 2024-04-12T20:48:52.353 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:48:51 smithi018 ceph-mon[28799]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' 2024-04-12T20:48:52.353 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:48:51 smithi018 ceph-mon[28799]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' 2024-04-12T20:48:52.353 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:48:51 smithi018 ceph-mon[28799]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-12T20:48:52.353 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:48:51 smithi018 ceph-mon[28799]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-12T20:48:52.353 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:48:51 smithi018 ceph-mon[28799]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' 2024-04-12T20:48:54.032 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:48:53 smithi186 ceph-mon[36597]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail; 2.7 KiB/s rd, 131 KiB/s wr, 8 op/s 2024-04-12T20:48:54.353 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:48:53 smithi018 ceph-mon[28799]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail; 2.7 KiB/s rd, 131 KiB/s wr, 8 op/s 2024-04-12T20:48:56.282 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:48:55 smithi186 ceph-mon[36597]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail; 2.7 KiB/s rd, 131 KiB/s wr, 8 op/s 2024-04-12T20:48:56.282 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:48:55 smithi186 ceph-mon[36597]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-04-12T20:48:56.282 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:48:55 smithi186 ceph-mon[36597]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-12T20:48:56.353 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:48:55 smithi018 ceph-mon[28799]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail; 2.7 KiB/s rd, 131 KiB/s wr, 8 op/s 2024-04-12T20:48:56.353 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:48:55 smithi018 ceph-mon[28799]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-04-12T20:48:56.353 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:48:55 smithi018 ceph-mon[28799]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-12T20:48:57.032 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:48:56 smithi186 ceph-mon[36597]: Deploying daemon osd.3 on smithi186 2024-04-12T20:48:57.353 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:48:56 smithi018 ceph-mon[28799]: Deploying daemon osd.3 on smithi186 2024-04-12T20:48:58.183 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:48:57 smithi186 ceph-mon[36597]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail; 1.7 KiB/s rd, 75 KiB/s wr, 5 op/s 2024-04-12T20:48:58.353 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:48:57 smithi018 ceph-mon[28799]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail; 1.7 KiB/s rd, 75 KiB/s wr, 5 op/s 2024-04-12T20:49:00.282 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:48:59 smithi186 ceph-mon[36597]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail; 1.5 KiB/s rd, 65 KiB/s wr, 4 op/s 2024-04-12T20:49:00.283 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:48:59 smithi186 ceph-mon[36597]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T20:49:00.292 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:48:59 smithi018 ceph-mon[28799]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail; 1.5 KiB/s rd, 65 KiB/s wr, 4 op/s 2024-04-12T20:49:00.292 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:48:59 smithi018 ceph-mon[28799]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T20:49:02.033 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:49:01 smithi186 ceph-mon[36597]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail; 1.5 KiB/s rd, 65 KiB/s wr, 4 op/s 2024-04-12T20:49:02.353 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:49:01 smithi018 ceph-mon[28799]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail; 1.5 KiB/s rd, 65 KiB/s wr, 4 op/s 2024-04-12T20:49:04.199 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:49:03 smithi186 ceph-mon[36597]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail 2024-04-12T20:49:04.199 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:49:03 smithi186 ceph-mon[36597]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' 2024-04-12T20:49:04.199 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:49:03 smithi186 ceph-mon[36597]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-12T20:49:04.199 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:49:03 smithi186 ceph-mon[36597]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' 2024-04-12T20:49:04.353 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:49:03 smithi018 ceph-mon[28799]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail 2024-04-12T20:49:04.353 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:49:03 smithi018 ceph-mon[28799]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' 2024-04-12T20:49:04.353 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:49:03 smithi018 ceph-mon[28799]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-12T20:49:04.353 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:49:03 smithi018 ceph-mon[28799]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' 2024-04-12T20:49:06.100 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:49:05 smithi186 ceph-mon[36597]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail 2024-04-12T20:49:06.188 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:49:05 smithi018 ceph-mon[28799]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail 2024-04-12T20:49:07.639 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:49:07.639+0000 7f2332000700 1 -- 172.21.15.186:0/407955004 <== mgr.14212 v2:172.21.15.18:6800/3048894687 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f233c108970 con 0x7f23200692a0 2024-04-12T20:49:07.639 INFO:teuthology.orchestra.run.smithi186.stdout:Created osd(s) 3 on host 'smithi186' 2024-04-12T20:49:07.642 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:49:07.642+0000 7f2340a00700 1 -- 172.21.15.186:0/407955004 >> [v2:172.21.15.18:6800/3048894687,v1:172.21.15.18:6801/3048894687] conn(0x7f23200692a0 msgr2=0x7f232006b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:49:07.643 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:49:07.642+0000 7f2340a00700 1 --2- 172.21.15.186:0/407955004 >> [v2:172.21.15.18:6800/3048894687,v1:172.21.15.18:6801/3048894687] conn(0x7f23200692a0 0x7f232006b750 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f232c00b020 tx=0x7f232c009e30 comp rx=0 tx=0).stop 2024-04-12T20:49:07.643 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:49:07.642+0000 7f2340a00700 1 -- 172.21.15.186:0/407955004 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f233c1046d0 msgr2=0x7f233c114420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:49:07.643 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:49:07.642+0000 7f2340a00700 1 --2- 172.21.15.186:0/407955004 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f233c1046d0 0x7f233c114420 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f2328002a80 tx=0x7f23280046b0 comp rx=0 tx=0).stop 2024-04-12T20:49:07.643 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:49:07.642+0000 7f2340a00700 1 -- 172.21.15.186:0/407955004 shutdown_connections 2024-04-12T20:49:07.643 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:49:07.642+0000 7f2340a00700 1 --2- 172.21.15.186:0/407955004 >> [v2:172.21.15.18:6800/3048894687,v1:172.21.15.18:6801/3048894687] conn(0x7f23200692a0 0x7f232006b750 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:49:07.643 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:49:07.642+0000 7f2340a00700 1 --2- 172.21.15.186:0/407955004 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f233c103420 0x7f233c113ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:49:07.643 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:49:07.642+0000 7f2340a00700 1 --2- 172.21.15.186:0/407955004 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f233c1046d0 0x7f233c114420 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:49:07.643 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:49:07.642+0000 7f2340a00700 1 -- 172.21.15.186:0/407955004 >> 172.21.15.186:0/407955004 conn(0x7f233c0725a0 msgr2=0x7f233c073c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T20:49:07.643 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:49:07.643+0000 7f2340a00700 1 -- 172.21.15.186:0/407955004 shutdown_connections 2024-04-12T20:49:07.643 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:49:07.643+0000 7f2340a00700 1 -- 172.21.15.186:0/407955004 wait complete. 2024-04-12T20:49:08.103 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:49:07 smithi018 ceph-mon[28799]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail 2024-04-12T20:49:08.104 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:49:07 smithi018 ceph-mon[28799]: from='osd.3 [v2:172.21.15.186:6800/2184799172,v1:172.21.15.186:6801/2184799172]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-04-12T20:49:08.104 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:49:07 smithi018 ceph-mon[28799]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-04-12T20:49:08.104 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:49:07 smithi018 ceph-mon[28799]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' 2024-04-12T20:49:08.104 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:49:07 smithi018 ceph-mon[28799]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' 2024-04-12T20:49:08.104 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:49:07 smithi018 ceph-mon[28799]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' 2024-04-12T20:49:08.104 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:49:07 smithi018 ceph-mon[28799]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' 2024-04-12T20:49:08.282 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:49:07 smithi186 ceph-mon[36597]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail 2024-04-12T20:49:08.282 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:49:07 smithi186 ceph-mon[36597]: from='osd.3 [v2:172.21.15.186:6800/2184799172,v1:172.21.15.186:6801/2184799172]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-04-12T20:49:08.282 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:49:07 smithi186 ceph-mon[36597]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-04-12T20:49:08.282 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:49:07 smithi186 ceph-mon[36597]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' 2024-04-12T20:49:08.282 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:49:07 smithi186 ceph-mon[36597]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' 2024-04-12T20:49:08.282 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:49:07 smithi186 ceph-mon[36597]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' 2024-04-12T20:49:08.283 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:49:07 smithi186 ceph-mon[36597]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' 2024-04-12T20:49:08.404 DEBUG:teuthology.orchestra.run.smithi186:osd.3> sudo journalctl -f -n 0 -u ceph-1cbe407a-f90d-11ee-bc8d-c7b262605968@osd.3.service 2024-04-12T20:49:08.409 INFO:tasks.cephadm:Deploying osd.4 on smithi186 with /dev/vg_nvme/lv_3... 2024-04-12T20:49:08.409 DEBUG:teuthology.orchestra.run.smithi186:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1cbe407a-f90d-11ee-bc8d-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-04-12T20:49:09.282 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:49:08 smithi186 ceph-mon[36597]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-04-12T20:49:09.282 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:49:08 smithi186 ceph-mon[36597]: osdmap e24: 4 total, 3 up, 4 in 2024-04-12T20:49:09.282 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:49:08 smithi186 ceph-mon[36597]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-12T20:49:09.282 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:49:08 smithi186 ceph-mon[36597]: from='osd.3 [v2:172.21.15.186:6800/2184799172,v1:172.21.15.186:6801/2184799172]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi186", "root=default"]}]: dispatch 2024-04-12T20:49:09.282 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:49:08 smithi186 ceph-mon[36597]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi186", "root=default"]}]: dispatch 2024-04-12T20:49:09.283 INFO:journalctl@ceph.osd.3.smithi186.stdout:Apr 12 20:49:08 smithi186 ceph-1cbe407a-f90d-11ee-bc8d-c7b262605968-osd-3[42045]: 2024-04-12T20:49:08.975+0000 7f541a600700 -1 osd.3 0 waiting for initial osdmap 2024-04-12T20:49:09.283 INFO:journalctl@ceph.osd.3.smithi186.stdout:Apr 12 20:49:08 smithi186 ceph-1cbe407a-f90d-11ee-bc8d-c7b262605968-osd-3[42045]: 2024-04-12T20:49:08.988+0000 7f5414200700 -1 osd.3 25 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-12T20:49:09.353 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:49:08 smithi018 ceph-mon[28799]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-04-12T20:49:09.353 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:49:08 smithi018 ceph-mon[28799]: osdmap e24: 4 total, 3 up, 4 in 2024-04-12T20:49:09.353 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:49:08 smithi018 ceph-mon[28799]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-12T20:49:09.353 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:49:08 smithi018 ceph-mon[28799]: from='osd.3 [v2:172.21.15.186:6800/2184799172,v1:172.21.15.186:6801/2184799172]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi186", "root=default"]}]: dispatch 2024-04-12T20:49:09.353 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:49:08 smithi018 ceph-mon[28799]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi186", "root=default"]}]: dispatch 2024-04-12T20:49:09.690 INFO:teuthology.orchestra.run.smithi186.stderr:Inferring config /var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/mon.smithi186/config 2024-04-12T20:49:10.206 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:49:09 smithi186 ceph-mon[36597]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail 2024-04-12T20:49:10.206 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:49:09 smithi186 ceph-mon[36597]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi186", "root=default"]}]': finished 2024-04-12T20:49:10.206 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:49:09 smithi186 ceph-mon[36597]: osdmap e25: 4 total, 3 up, 4 in 2024-04-12T20:49:10.206 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:49:09 smithi186 ceph-mon[36597]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-12T20:49:10.206 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:49:09 smithi186 ceph-mon[36597]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-12T20:49:10.295 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:49:09 smithi018 ceph-mon[28799]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail 2024-04-12T20:49:10.295 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:49:09 smithi018 ceph-mon[28799]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi186", "root=default"]}]': finished 2024-04-12T20:49:10.295 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:49:09 smithi018 ceph-mon[28799]: osdmap e25: 4 total, 3 up, 4 in 2024-04-12T20:49:10.295 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:49:09 smithi018 ceph-mon[28799]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-12T20:49:10.295 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:49:09 smithi018 ceph-mon[28799]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-12T20:49:11.282 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:49:10 smithi186 ceph-mon[36597]: purged_snaps scrub starts 2024-04-12T20:49:11.282 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:49:10 smithi186 ceph-mon[36597]: purged_snaps scrub ok 2024-04-12T20:49:11.282 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:49:10 smithi186 ceph-mon[36597]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-12T20:49:11.282 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:49:10 smithi186 ceph-mon[36597]: osd.3 [v2:172.21.15.186:6800/2184799172,v1:172.21.15.186:6801/2184799172] boot 2024-04-12T20:49:11.282 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:49:10 smithi186 ceph-mon[36597]: osdmap e26: 4 total, 4 up, 4 in 2024-04-12T20:49:11.282 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:49:10 smithi186 ceph-mon[36597]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-12T20:49:11.353 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:49:10 smithi018 ceph-mon[28799]: purged_snaps scrub starts 2024-04-12T20:49:11.353 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:49:10 smithi018 ceph-mon[28799]: purged_snaps scrub ok 2024-04-12T20:49:11.353 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:49:10 smithi018 ceph-mon[28799]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-12T20:49:11.354 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:49:10 smithi018 ceph-mon[28799]: osd.3 [v2:172.21.15.186:6800/2184799172,v1:172.21.15.186:6801/2184799172] boot 2024-04-12T20:49:11.354 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:49:10 smithi018 ceph-mon[28799]: osdmap e26: 4 total, 4 up, 4 in 2024-04-12T20:49:11.354 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:49:10 smithi018 ceph-mon[28799]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-12T20:49:12.245 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:49:11 smithi186 ceph-mon[36597]: pgmap v117: 1 pgs: 1 peering; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail 2024-04-12T20:49:12.245 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:49:11 smithi186 ceph-mon[36597]: osdmap e27: 4 total, 4 up, 4 in 2024-04-12T20:49:12.353 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:49:11 smithi018 ceph-mon[28799]: pgmap v117: 1 pgs: 1 peering; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail 2024-04-12T20:49:12.353 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:49:11 smithi018 ceph-mon[28799]: osdmap e27: 4 total, 4 up, 4 in 2024-04-12T20:49:12.608 INFO:teuthology.orchestra.run.smithi186.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-04-12T20:49:12.608 INFO:teuthology.orchestra.run.smithi186.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-04-12T20:49:12.609 INFO:teuthology.orchestra.run.smithi186.stderr: stderr: 10+0 records in 2024-04-12T20:49:12.609 INFO:teuthology.orchestra.run.smithi186.stderr:10+0 records out 2024-04-12T20:49:12.609 INFO:teuthology.orchestra.run.smithi186.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0366021 s, 286 MB/s 2024-04-12T20:49:12.609 INFO:teuthology.orchestra.run.smithi186.stderr:--> Zapping successful for: 2024-04-12T20:49:13.279 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:49:13 smithi186 ceph-mon[36597]: osdmap e28: 4 total, 4 up, 4 in 2024-04-12T20:49:13.353 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:49:13 smithi018 ceph-mon[28799]: osdmap e28: 4 total, 4 up, 4 in 2024-04-12T20:49:13.686 DEBUG:teuthology.orchestra.run.smithi186:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1cbe407a-f90d-11ee-bc8d-c7b262605968 -- ceph orch daemon add osd smithi186:vg_nvme/lv_3 2024-04-12T20:49:14.151 INFO:teuthology.orchestra.run.smithi186.stderr:Inferring config /var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/mon.smithi186/config 2024-04-12T20:49:14.282 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:49:14 smithi186 ceph-mon[36597]: pgmap v120: 1 pgs: 1 peering; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-04-12T20:49:14.283 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:49:14 smithi186 ceph-mon[36597]: osdmap e29: 4 total, 4 up, 4 in 2024-04-12T20:49:14.283 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:49:14 smithi186 ceph-mon[36597]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' 2024-04-12T20:49:14.283 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:49:14 smithi186 ceph-mon[36597]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' 2024-04-12T20:49:14.283 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:49:14 smithi186 ceph-mon[36597]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-12T20:49:14.283 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:49:14 smithi186 ceph-mon[36597]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' 2024-04-12T20:49:14.283 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:49:14 smithi186 ceph-mon[36597]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-12T20:49:14.283 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:49:14 smithi186 ceph-mon[36597]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-12T20:49:14.283 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:49:14 smithi186 ceph-mon[36597]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' 2024-04-12T20:49:14.283 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:49:14 smithi186 ceph-mon[36597]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-12T20:49:14.283 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:49:14 smithi186 ceph-mon[36597]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-12T20:49:14.283 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:49:14 smithi186 ceph-mon[36597]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-12T20:49:14.283 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:49:14 smithi186 ceph-mon[36597]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' 2024-04-12T20:49:14.353 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:49:14 smithi018 ceph-mon[28799]: pgmap v120: 1 pgs: 1 peering; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-04-12T20:49:14.353 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:49:14 smithi018 ceph-mon[28799]: osdmap e29: 4 total, 4 up, 4 in 2024-04-12T20:49:14.353 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:49:14 smithi018 ceph-mon[28799]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' 2024-04-12T20:49:14.353 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:49:14 smithi018 ceph-mon[28799]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' 2024-04-12T20:49:14.354 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:49:14 smithi018 ceph-mon[28799]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-12T20:49:14.354 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:49:14 smithi018 ceph-mon[28799]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' 2024-04-12T20:49:14.354 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:49:14 smithi018 ceph-mon[28799]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-12T20:49:14.354 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:49:14 smithi018 ceph-mon[28799]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-12T20:49:14.354 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:49:14 smithi018 ceph-mon[28799]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' 2024-04-12T20:49:14.354 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:49:14 smithi018 ceph-mon[28799]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-12T20:49:14.354 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:49:14 smithi018 ceph-mon[28799]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-12T20:49:14.354 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:49:14 smithi018 ceph-mon[28799]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-12T20:49:14.354 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:49:14 smithi018 ceph-mon[28799]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' 2024-04-12T20:49:15.282 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:49:15 smithi186 ceph-mon[36597]: Detected new or changed devices on smithi186 2024-04-12T20:49:15.282 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:49:15 smithi186 ceph-mon[36597]: Adjusting osd_memory_target on smithi186 to 15937M 2024-04-12T20:49:15.282 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:49:15 smithi186 ceph-mon[36597]: osdmap e30: 4 total, 4 up, 4 in 2024-04-12T20:49:15.282 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:49:15 smithi186 ceph-mon[36597]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T20:49:15.353 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:49:15 smithi018 ceph-mon[28799]: Detected new or changed devices on smithi186 2024-04-12T20:49:15.353 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:49:15 smithi018 ceph-mon[28799]: Adjusting osd_memory_target on smithi186 to 15937M 2024-04-12T20:49:15.353 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:49:15 smithi018 ceph-mon[28799]: osdmap e30: 4 total, 4 up, 4 in 2024-04-12T20:49:15.353 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:49:15 smithi018 ceph-mon[28799]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T20:49:16.060 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:49:16.060+0000 7f9b51e00700 1 -- 172.21.15.186:0/3046529894 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9b4c0709a0 msgr2=0x7f9b4c070e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:49:16.061 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:49:16.060+0000 7f9b51e00700 1 --2- 172.21.15.186:0/3046529894 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9b4c0709a0 0x7f9b4c070e10 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f9b34004940 tx=0x7f9b34009e30 comp rx=0 tx=0).stop 2024-04-12T20:49:16.061 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:49:16.060+0000 7f9b51e00700 1 -- 172.21.15.186:0/3046529894 shutdown_connections 2024-04-12T20:49:16.061 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:49:16.060+0000 7f9b51e00700 1 --2- 172.21.15.186:0/3046529894 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9b4c0709a0 0x7f9b4c070e10 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:49:16.061 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:49:16.060+0000 7f9b51e00700 1 --2- 172.21.15.186:0/3046529894 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9b4c071f50 0x7f9b4c070460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:49:16.061 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:49:16.060+0000 7f9b51e00700 1 -- 172.21.15.186:0/3046529894 >> 172.21.15.186:0/3046529894 conn(0x7f9b4c0f8560 msgr2=0x7f9b4c0fa970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T20:49:16.061 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:49:16.061+0000 7f9b51e00700 1 -- 172.21.15.186:0/3046529894 shutdown_connections 2024-04-12T20:49:16.061 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:49:16.061+0000 7f9b51e00700 1 -- 172.21.15.186:0/3046529894 wait complete. 2024-04-12T20:49:16.061 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:49:16.062+0000 7f9b51e00700 1 Processor -- start 2024-04-12T20:49:16.061 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:49:16.062+0000 7f9b51e00700 1 -- start start 2024-04-12T20:49:16.063 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:49:16.062+0000 7f9b51e00700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9b4c0709a0 0x7f9b4c114050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T20:49:16.063 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:49:16.062+0000 7f9b51e00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9b4c071f50 0x7f9b4c114590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T20:49:16.063 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:49:16.062+0000 7f9b51e00700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b4c114b90 con 0x7f9b4c071f50 2024-04-12T20:49:16.063 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:49:16.062+0000 7f9b51e00700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b4c1153e0 con 0x7f9b4c0709a0 2024-04-12T20:49:16.063 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:49:16.062+0000 7f9b4aa00700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9b4c0709a0 0x7f9b4c114050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T20:49:16.063 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:49:16.063+0000 7f9b4aa00700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9b4c0709a0 0x7f9b4c114050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.186:52004/0 (socket says 172.21.15.186:52004) 2024-04-12T20:49:16.063 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:49:16.063+0000 7f9b4aa00700 1 -- 172.21.15.186:0/3910593251 learned_addr learned my addr 172.21.15.186:0/3910593251 (peer_addr_for_me v2:172.21.15.186:0/0) 2024-04-12T20:49:16.063 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:49:16.063+0000 7f9b42000700 1 --2- 172.21.15.186:0/3910593251 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9b4c071f50 0x7f9b4c114590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T20:49:16.063 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:49:16.063+0000 7f9b4aa00700 1 -- 172.21.15.186:0/3910593251 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9b4c071f50 msgr2=0x7f9b4c114590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:49:16.064 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:49:16.063+0000 7f9b4aa00700 1 --2- 172.21.15.186:0/3910593251 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9b4c071f50 0x7f9b4c114590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:49:16.064 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:49:16.063+0000 7f9b4aa00700 1 -- 172.21.15.186:0/3910593251 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b34005040 con 0x7f9b4c0709a0 2024-04-12T20:49:16.064 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:49:16.063+0000 7f9b42000700 1 --2- 172.21.15.186:0/3910593251 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9b4c071f50 0x7f9b4c114590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-12T20:49:16.064 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:49:16.063+0000 7f9b4aa00700 1 --2- 172.21.15.186:0/3910593251 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9b4c0709a0 0x7f9b4c114050 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f9b3c002a80 tx=0x7f9b3c004600 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T20:49:16.064 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:49:16.064+0000 7f9b43400700 1 -- 172.21.15.186:0/3910593251 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9b3c007cc0 con 0x7f9b4c0709a0 2024-04-12T20:49:16.064 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:49:16.064+0000 7f9b51e00700 1 -- 172.21.15.186:0/3910593251 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9b4c1156c0 con 0x7f9b4c0709a0 2024-04-12T20:49:16.064 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:49:16.064+0000 7f9b43400700 1 -- 172.21.15.186:0/3910593251 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f9b3c00f040 con 0x7f9b4c0709a0 2024-04-12T20:49:16.064 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:49:16.064+0000 7f9b43400700 1 -- 172.21.15.186:0/3910593251 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9b3c00dad0 con 0x7f9b4c0709a0 2024-04-12T20:49:16.064 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:49:16.064+0000 7f9b51e00700 1 -- 172.21.15.186:0/3910593251 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9b4c115c00 con 0x7f9b4c0709a0 2024-04-12T20:49:16.065 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:49:16.065+0000 7f9b51e00700 1 -- 172.21.15.186:0/3910593251 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9b4c05f5f0 con 0x7f9b4c0709a0 2024-04-12T20:49:16.065 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:49:16.066+0000 7f9b43400700 1 -- 172.21.15.186:0/3910593251 <== mon.1 v2:172.21.15.186:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7f9b3c00dc30 con 0x7f9b4c0709a0 2024-04-12T20:49:16.065 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:49:16.067+0000 7f9b43400700 1 --2- 172.21.15.186:0/3910593251 >> [v2:172.21.15.18:6800/3048894687,v1:172.21.15.18:6801/3048894687] conn(0x7f9b300692b0 0x7f9b3006b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T20:49:16.065 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:49:16.067+0000 7f9b43400700 1 -- 172.21.15.186:0/3910593251 <== mon.1 v2:172.21.15.186:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 3770+0+0 (secure 0 0 0) 0x7f9b3c0867d0 con 0x7f9b4c0709a0 2024-04-12T20:49:16.066 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:49:16.067+0000 7f9b42000700 1 --2- 172.21.15.186:0/3910593251 >> [v2:172.21.15.18:6800/3048894687,v1:172.21.15.18:6801/3048894687] conn(0x7f9b300692b0 0x7f9b3006b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T20:49:16.067 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:49:16.069+0000 7f9b42000700 1 --2- 172.21.15.186:0/3910593251 >> [v2:172.21.15.18:6800/3048894687,v1:172.21.15.18:6801/3048894687] conn(0x7f9b300692b0 0x7f9b3006b760 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f9b3400b7a0 tx=0x7f9b34009e30 comp rx=0 tx=0).ready entity=mgr.14212 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T20:49:16.070 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:49:16.072+0000 7f9b43400700 1 -- 172.21.15.186:0/3910593251 <== mon.1 v2:172.21.15.186:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7f9b3c056c30 con 0x7f9b4c0709a0 2024-04-12T20:49:16.282 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:49:16 smithi186 ceph-mon[36597]: pgmap v123: 1 pgs: 1 peering; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-04-12T20:49:16.293 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:49:16.294+0000 7f9b51e00700 1 -- 172.21.15.186:0/3910593251 --> [v2:172.21.15.18:6800/3048894687,v1:172.21.15.18:6801/3048894687] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi186:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) v1 -- 0x7f9b4c115fe0 con 0x7f9b300692b0 2024-04-12T20:49:16.353 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:49:16 smithi018 ceph-mon[28799]: pgmap v123: 1 pgs: 1 peering; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-04-12T20:49:17.353 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:49:17 smithi018 ceph-mon[28799]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-12T20:49:17.353 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:49:17 smithi018 ceph-mon[28799]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-12T20:49:17.353 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:49:17 smithi018 ceph-mon[28799]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-12T20:49:17.363 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:49:17 smithi186 ceph-mon[36597]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-12T20:49:17.363 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:49:17 smithi186 ceph-mon[36597]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-12T20:49:17.363 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:49:17 smithi186 ceph-mon[36597]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-12T20:49:18.353 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:49:18 smithi018 ceph-mon[28799]: from='client.24147 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi186:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T20:49:18.353 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:49:18 smithi018 ceph-mon[28799]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail; 99 KiB/s, 0 objects/s recovering 2024-04-12T20:49:18.356 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:49:18 smithi186 ceph-mon[36597]: from='client.24147 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi186:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T20:49:18.356 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:49:18 smithi186 ceph-mon[36597]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail; 99 KiB/s, 0 objects/s recovering 2024-04-12T20:49:20.353 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:49:20 smithi018 ceph-mon[28799]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail; 84 KiB/s, 0 objects/s recovering 2024-04-12T20:49:20.353 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:49:20 smithi018 ceph-mon[28799]: from='client.? 172.21.15.186:0/1463637605' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d6b2c6b3-6c78-4261-bcbe-c33358407198"}]: dispatch 2024-04-12T20:49:20.353 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:49:20 smithi018 ceph-mon[28799]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d6b2c6b3-6c78-4261-bcbe-c33358407198"}]: dispatch 2024-04-12T20:49:20.353 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:49:20 smithi018 ceph-mon[28799]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d6b2c6b3-6c78-4261-bcbe-c33358407198"}]': finished 2024-04-12T20:49:20.353 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:49:20 smithi018 ceph-mon[28799]: osdmap e31: 5 total, 4 up, 5 in 2024-04-12T20:49:20.354 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:49:20 smithi018 ceph-mon[28799]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-12T20:49:20.442 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:49:20 smithi186 ceph-mon[36597]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail; 84 KiB/s, 0 objects/s recovering 2024-04-12T20:49:20.442 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:49:20 smithi186 ceph-mon[36597]: from='client.? 172.21.15.186:0/1463637605' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d6b2c6b3-6c78-4261-bcbe-c33358407198"}]: dispatch 2024-04-12T20:49:20.442 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:49:20 smithi186 ceph-mon[36597]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d6b2c6b3-6c78-4261-bcbe-c33358407198"}]: dispatch 2024-04-12T20:49:20.442 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:49:20 smithi186 ceph-mon[36597]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d6b2c6b3-6c78-4261-bcbe-c33358407198"}]': finished 2024-04-12T20:49:20.442 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:49:20 smithi186 ceph-mon[36597]: osdmap e31: 5 total, 4 up, 5 in 2024-04-12T20:49:20.442 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:49:20 smithi186 ceph-mon[36597]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-12T20:49:21.532 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:49:21 smithi186 ceph-mon[36597]: from='client.? 172.21.15.186:0/4212785231' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-12T20:49:21.603 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:49:21 smithi018 ceph-mon[28799]: from='client.? 172.21.15.186:0/4212785231' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-12T20:49:22.532 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:49:22 smithi186 ceph-mon[36597]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail; 74 KiB/s, 0 objects/s recovering 2024-04-12T20:49:22.603 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:49:22 smithi018 ceph-mon[28799]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail; 74 KiB/s, 0 objects/s recovering 2024-04-12T20:49:23.531 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:49:23 smithi186 ceph-mon[36597]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail; 65 KiB/s, 0 objects/s recovering 2024-04-12T20:49:23.603 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:49:23 smithi018 ceph-mon[28799]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail; 65 KiB/s, 0 objects/s recovering 2024-04-12T20:49:26.093 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:49:25 smithi186 ceph-mon[36597]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-12T20:49:26.353 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:49:25 smithi018 ceph-mon[28799]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-12T20:49:28.282 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:49:27 smithi186 ceph-mon[36597]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-04-12T20:49:28.353 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:49:27 smithi018 ceph-mon[28799]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-04-12T20:49:29.032 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:49:28 smithi186 ceph-mon[36597]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-04-12T20:49:29.032 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:49:28 smithi186 ceph-mon[36597]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-12T20:49:29.353 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:49:28 smithi018 ceph-mon[28799]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-04-12T20:49:29.353 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:49:28 smithi018 ceph-mon[28799]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-12T20:49:30.095 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:49:29 smithi186 ceph-mon[36597]: Deploying daemon osd.4 on smithi186 2024-04-12T20:49:30.095 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:49:29 smithi186 ceph-mon[36597]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-04-12T20:49:30.095 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:49:29 smithi186 ceph-mon[36597]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T20:49:30.293 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:49:29 smithi018 ceph-mon[28799]: Deploying daemon osd.4 on smithi186 2024-04-12T20:49:30.293 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:49:29 smithi018 ceph-mon[28799]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-04-12T20:49:30.293 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:49:29 smithi018 ceph-mon[28799]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T20:49:32.161 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:49:31 smithi186 ceph-mon[36597]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-04-12T20:49:32.353 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:49:31 smithi018 ceph-mon[28799]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-04-12T20:49:34.033 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:49:33 smithi186 ceph-mon[36597]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-04-12T20:49:34.353 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:49:33 smithi018 ceph-mon[28799]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-04-12T20:49:35.970 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:49:35 smithi186 ceph-mon[36597]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-04-12T20:49:36.353 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:49:35 smithi018 ceph-mon[28799]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-04-12T20:49:37.233 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:49:37 smithi186 ceph-mon[36597]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' 2024-04-12T20:49:37.233 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:49:37 smithi186 ceph-mon[36597]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-12T20:49:37.233 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:49:37 smithi186 ceph-mon[36597]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' 2024-04-12T20:49:37.233 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:49:37 smithi186 ceph-mon[36597]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-04-12T20:49:37.606 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:49:37 smithi018 ceph-mon[28799]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' 2024-04-12T20:49:37.606 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:49:37 smithi018 ceph-mon[28799]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-12T20:49:37.606 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:49:37 smithi018 ceph-mon[28799]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' 2024-04-12T20:49:37.606 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:49:37 smithi018 ceph-mon[28799]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-04-12T20:49:40.103 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:49:39 smithi186 ceph-mon[36597]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-04-12T20:49:40.292 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:49:39 smithi018 ceph-mon[28799]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-04-12T20:49:41.257 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:49:40 smithi186 ceph-mon[36597]: from='osd.4 [v2:172.21.15.186:6808/3584843287,v1:172.21.15.186:6809/3584843287]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-12T20:49:41.257 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:49:40 smithi186 ceph-mon[36597]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-12T20:49:41.257 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:49:40 smithi186 ceph-mon[36597]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' 2024-04-12T20:49:41.257 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:49:40 smithi186 ceph-mon[36597]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' 2024-04-12T20:49:41.349 INFO:teuthology.orchestra.run.smithi186.stdout:Created osd(s) 4 on host 'smithi186' 2024-04-12T20:49:41.349 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:49:41.348+0000 7f9b43400700 1 -- 172.21.15.186:0/3910593251 <== mgr.14212 v2:172.21.15.18:6800/3048894687 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f9b4c115fe0 con 0x7f9b300692b0 2024-04-12T20:49:41.351 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:49:41.352+0000 7f9b51e00700 1 -- 172.21.15.186:0/3910593251 >> [v2:172.21.15.18:6800/3048894687,v1:172.21.15.18:6801/3048894687] conn(0x7f9b300692b0 msgr2=0x7f9b3006b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:49:41.351 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:49:41.352+0000 7f9b51e00700 1 --2- 172.21.15.186:0/3910593251 >> [v2:172.21.15.18:6800/3048894687,v1:172.21.15.18:6801/3048894687] conn(0x7f9b300692b0 0x7f9b3006b760 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f9b3400b7a0 tx=0x7f9b34009e30 comp rx=0 tx=0).stop 2024-04-12T20:49:41.351 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:49:41.352+0000 7f9b51e00700 1 -- 172.21.15.186:0/3910593251 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9b4c0709a0 msgr2=0x7f9b4c114050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:49:41.352 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:49:41.352+0000 7f9b51e00700 1 --2- 172.21.15.186:0/3910593251 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9b4c0709a0 0x7f9b4c114050 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f9b3c002a80 tx=0x7f9b3c004600 comp rx=0 tx=0).stop 2024-04-12T20:49:41.352 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:49:41.353+0000 7f9b51e00700 1 -- 172.21.15.186:0/3910593251 shutdown_connections 2024-04-12T20:49:41.352 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:49:41.353+0000 7f9b51e00700 1 --2- 172.21.15.186:0/3910593251 >> [v2:172.21.15.18:6800/3048894687,v1:172.21.15.18:6801/3048894687] conn(0x7f9b300692b0 0x7f9b3006b760 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:49:41.352 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:49:41.353+0000 7f9b51e00700 1 --2- 172.21.15.186:0/3910593251 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9b4c0709a0 0x7f9b4c114050 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:49:41.352 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:49:41.353+0000 7f9b51e00700 1 --2- 172.21.15.186:0/3910593251 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9b4c071f50 0x7f9b4c114590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:49:41.352 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:49:41.353+0000 7f9b51e00700 1 -- 172.21.15.186:0/3910593251 >> 172.21.15.186:0/3910593251 conn(0x7f9b4c0f8560 msgr2=0x7f9b4c0f9dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T20:49:41.352 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:49:41.353+0000 7f9b51e00700 1 -- 172.21.15.186:0/3910593251 shutdown_connections 2024-04-12T20:49:41.352 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:49:41.353+0000 7f9b51e00700 1 -- 172.21.15.186:0/3910593251 wait complete. 2024-04-12T20:49:41.353 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:49:40 smithi018 ceph-mon[28799]: from='osd.4 [v2:172.21.15.186:6808/3584843287,v1:172.21.15.186:6809/3584843287]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-12T20:49:41.353 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:49:40 smithi018 ceph-mon[28799]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-12T20:49:41.353 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:49:40 smithi018 ceph-mon[28799]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' 2024-04-12T20:49:41.353 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:49:40 smithi018 ceph-mon[28799]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' 2024-04-12T20:49:42.002 DEBUG:teuthology.orchestra.run.smithi186:osd.4> sudo journalctl -f -n 0 -u ceph-1cbe407a-f90d-11ee-bc8d-c7b262605968@osd.4.service 2024-04-12T20:49:42.007 INFO:tasks.cephadm:Deploying osd.5 on smithi186 with /dev/vg_nvme/lv_2... 2024-04-12T20:49:42.008 DEBUG:teuthology.orchestra.run.smithi186:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1cbe407a-f90d-11ee-bc8d-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-04-12T20:49:42.260 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:49:41 smithi186 ceph-mon[36597]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-04-12T20:49:42.260 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:49:41 smithi186 ceph-mon[36597]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-04-12T20:49:42.260 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:49:41 smithi186 ceph-mon[36597]: osdmap e32: 5 total, 4 up, 5 in 2024-04-12T20:49:42.261 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:49:41 smithi186 ceph-mon[36597]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-12T20:49:42.261 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:49:41 smithi186 ceph-mon[36597]: from='osd.4 [v2:172.21.15.186:6808/3584843287,v1:172.21.15.186:6809/3584843287]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi186", "root=default"]}]: dispatch 2024-04-12T20:49:42.261 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:49:41 smithi186 ceph-mon[36597]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi186", "root=default"]}]: dispatch 2024-04-12T20:49:42.261 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:49:41 smithi186 ceph-mon[36597]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' 2024-04-12T20:49:42.261 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:49:41 smithi186 ceph-mon[36597]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' 2024-04-12T20:49:42.353 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:49:41 smithi018 ceph-mon[28799]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-04-12T20:49:42.353 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:49:41 smithi018 ceph-mon[28799]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-04-12T20:49:42.353 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:49:41 smithi018 ceph-mon[28799]: osdmap e32: 5 total, 4 up, 5 in 2024-04-12T20:49:42.353 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:49:41 smithi018 ceph-mon[28799]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-12T20:49:42.354 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:49:41 smithi018 ceph-mon[28799]: from='osd.4 [v2:172.21.15.186:6808/3584843287,v1:172.21.15.186:6809/3584843287]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi186", "root=default"]}]: dispatch 2024-04-12T20:49:42.354 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:49:41 smithi018 ceph-mon[28799]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi186", "root=default"]}]: dispatch 2024-04-12T20:49:42.354 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:49:41 smithi018 ceph-mon[28799]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' 2024-04-12T20:49:42.354 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:49:41 smithi018 ceph-mon[28799]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' 2024-04-12T20:49:43.282 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:49:42 smithi186 ceph-mon[36597]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi186", "root=default"]}]': finished 2024-04-12T20:49:43.283 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:49:42 smithi186 ceph-mon[36597]: osdmap e33: 5 total, 4 up, 5 in 2024-04-12T20:49:43.283 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:49:42 smithi186 ceph-mon[36597]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-12T20:49:43.283 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:49:42 smithi186 ceph-mon[36597]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-12T20:49:43.353 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:49:42 smithi018 ceph-mon[28799]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi186", "root=default"]}]': finished 2024-04-12T20:49:43.353 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:49:42 smithi018 ceph-mon[28799]: osdmap e33: 5 total, 4 up, 5 in 2024-04-12T20:49:43.353 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:49:42 smithi018 ceph-mon[28799]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-12T20:49:43.353 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:49:42 smithi018 ceph-mon[28799]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-12T20:49:43.372 INFO:teuthology.orchestra.run.smithi186.stderr:Inferring config /var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/mon.smithi186/config 2024-04-12T20:49:44.282 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:49:43 smithi186 ceph-mon[36597]: purged_snaps scrub starts 2024-04-12T20:49:44.282 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:49:43 smithi186 ceph-mon[36597]: purged_snaps scrub ok 2024-04-12T20:49:44.282 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:49:43 smithi186 ceph-mon[36597]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-04-12T20:49:44.282 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:49:43 smithi186 ceph-mon[36597]: osd.4 [v2:172.21.15.186:6808/3584843287,v1:172.21.15.186:6809/3584843287] boot 2024-04-12T20:49:44.282 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:49:43 smithi186 ceph-mon[36597]: osdmap e34: 5 total, 5 up, 5 in 2024-04-12T20:49:44.282 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:49:43 smithi186 ceph-mon[36597]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-12T20:49:44.354 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:49:43 smithi018 ceph-mon[28799]: purged_snaps scrub starts 2024-04-12T20:49:44.354 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:49:43 smithi018 ceph-mon[28799]: purged_snaps scrub ok 2024-04-12T20:49:44.354 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:49:43 smithi018 ceph-mon[28799]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-04-12T20:49:44.354 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:49:43 smithi018 ceph-mon[28799]: osd.4 [v2:172.21.15.186:6808/3584843287,v1:172.21.15.186:6809/3584843287] boot 2024-04-12T20:49:44.354 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:49:43 smithi018 ceph-mon[28799]: osdmap e34: 5 total, 5 up, 5 in 2024-04-12T20:49:44.354 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:49:43 smithi018 ceph-mon[28799]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-12T20:49:45.241 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:49:44 smithi186 ceph-mon[36597]: osdmap e35: 5 total, 5 up, 5 in 2024-04-12T20:49:45.242 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:49:44 smithi186 ceph-mon[36597]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T20:49:45.353 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:49:44 smithi018 ceph-mon[28799]: osdmap e35: 5 total, 5 up, 5 in 2024-04-12T20:49:45.353 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:49:44 smithi018 ceph-mon[28799]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T20:49:46.281 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:49:45 smithi186 ceph-mon[36597]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-04-12T20:49:46.353 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:49:45 smithi018 ceph-mon[28799]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-04-12T20:49:46.353 INFO:teuthology.orchestra.run.smithi186.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-04-12T20:49:46.353 INFO:teuthology.orchestra.run.smithi186.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-04-12T20:49:46.354 INFO:teuthology.orchestra.run.smithi186.stderr: stderr: 10+0 records in 2024-04-12T20:49:46.354 INFO:teuthology.orchestra.run.smithi186.stderr:10+0 records out 2024-04-12T20:49:46.354 INFO:teuthology.orchestra.run.smithi186.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0328483 s, 319 MB/s 2024-04-12T20:49:46.354 INFO:teuthology.orchestra.run.smithi186.stderr:--> Zapping successful for: 2024-04-12T20:49:47.809 DEBUG:teuthology.orchestra.run.smithi186:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1cbe407a-f90d-11ee-bc8d-c7b262605968 -- ceph orch daemon add osd smithi186:vg_nvme/lv_2 2024-04-12T20:49:48.032 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:49:47 smithi186 ceph-mon[36597]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-04-12T20:49:48.032 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:49:47 smithi186 ceph-mon[36597]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' 2024-04-12T20:49:48.032 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:49:47 smithi186 ceph-mon[36597]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' 2024-04-12T20:49:48.032 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:49:47 smithi186 ceph-mon[36597]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-12T20:49:48.033 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:49:47 smithi186 ceph-mon[36597]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-12T20:49:48.033 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:49:47 smithi186 ceph-mon[36597]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' 2024-04-12T20:49:48.033 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:49:47 smithi186 ceph-mon[36597]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-12T20:49:48.033 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:49:47 smithi186 ceph-mon[36597]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-12T20:49:48.033 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:49:47 smithi186 ceph-mon[36597]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' 2024-04-12T20:49:48.033 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:49:47 smithi186 ceph-mon[36597]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-12T20:49:48.290 INFO:teuthology.orchestra.run.smithi186.stderr:Inferring config /var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/mon.smithi186/config 2024-04-12T20:49:48.353 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:49:47 smithi018 ceph-mon[28799]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-04-12T20:49:48.353 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:49:47 smithi018 ceph-mon[28799]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' 2024-04-12T20:49:48.353 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:49:47 smithi018 ceph-mon[28799]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' 2024-04-12T20:49:48.353 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:49:47 smithi018 ceph-mon[28799]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-12T20:49:48.353 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:49:47 smithi018 ceph-mon[28799]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-12T20:49:48.353 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:49:47 smithi018 ceph-mon[28799]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' 2024-04-12T20:49:48.353 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:49:47 smithi018 ceph-mon[28799]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-12T20:49:48.353 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:49:47 smithi018 ceph-mon[28799]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-12T20:49:48.353 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:49:47 smithi018 ceph-mon[28799]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' 2024-04-12T20:49:48.353 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:49:47 smithi018 ceph-mon[28799]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-12T20:49:49.032 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:49:48 smithi186 ceph-mon[36597]: Detected new or changed devices on smithi186 2024-04-12T20:49:49.032 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:49:48 smithi186 ceph-mon[36597]: Adjusting osd_memory_target on smithi186 to 7968M 2024-04-12T20:49:49.032 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:49:48 smithi186 ceph-mon[36597]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-12T20:49:49.032 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:49:48 smithi186 ceph-mon[36597]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-12T20:49:49.033 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:49:48 smithi186 ceph-mon[36597]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' 2024-04-12T20:49:49.353 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:49:48 smithi018 ceph-mon[28799]: Detected new or changed devices on smithi186 2024-04-12T20:49:49.353 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:49:48 smithi018 ceph-mon[28799]: Adjusting osd_memory_target on smithi186 to 7968M 2024-04-12T20:49:49.353 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:49:48 smithi018 ceph-mon[28799]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-12T20:49:49.353 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:49:48 smithi018 ceph-mon[28799]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-12T20:49:49.353 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:49:48 smithi018 ceph-mon[28799]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' 2024-04-12T20:49:50.144 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:49:49 smithi186 ceph-mon[36597]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-04-12T20:49:50.294 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:49:49 smithi018 ceph-mon[28799]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-04-12T20:49:50.308 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:49:50.308+0000 7ff8cec00700 1 -- 172.21.15.186:0/2222970091 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff8c80fce70 msgr2=0x7ff8c80fd280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:49:50.308 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:49:50.308+0000 7ff8cec00700 1 --2- 172.21.15.186:0/2222970091 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff8c80fce70 0x7ff8c80fd280 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7ff8b8004940 tx=0x7ff8b8009e30 comp rx=0 tx=0).stop 2024-04-12T20:49:50.308 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:49:50.308+0000 7ff8cec00700 1 -- 172.21.15.186:0/2222970091 shutdown_connections 2024-04-12T20:49:50.308 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:49:50.308+0000 7ff8cec00700 1 --2- 172.21.15.186:0/2222970091 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff8c80fdfe0 0x7ff8c80fe450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:49:50.308 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:49:50.308+0000 7ff8cec00700 1 --2- 172.21.15.186:0/2222970091 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff8c80fce70 0x7ff8c80fd280 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:49:50.308 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:49:50.308+0000 7ff8cec00700 1 -- 172.21.15.186:0/2222970091 >> 172.21.15.186:0/2222970091 conn(0x7ff8c80f8550 msgr2=0x7ff8c80fa960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T20:49:50.308 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:49:50.308+0000 7ff8cec00700 1 -- 172.21.15.186:0/2222970091 shutdown_connections 2024-04-12T20:49:50.308 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:49:50.308+0000 7ff8cec00700 1 -- 172.21.15.186:0/2222970091 wait complete. 2024-04-12T20:49:50.309 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:49:50.309+0000 7ff8cec00700 1 Processor -- start 2024-04-12T20:49:50.309 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:49:50.310+0000 7ff8cec00700 1 -- start start 2024-04-12T20:49:50.310 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:49:50.310+0000 7ff8cec00700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff8c80fce70 0x7ff8c810fb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T20:49:50.310 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:49:50.310+0000 7ff8cec00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff8c80fdfe0 0x7ff8c8110090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T20:49:50.310 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:49:50.310+0000 7ff8cec00700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8c8110720 con 0x7ff8c80fdfe0 2024-04-12T20:49:50.310 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:49:50.310+0000 7ff8cec00700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8c81150c0 con 0x7ff8c80fce70 2024-04-12T20:49:50.310 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:49:50.310+0000 7ff8c7e00700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff8c80fce70 0x7ff8c810fb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T20:49:50.311 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:49:50.310+0000 7ff8c7e00700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff8c80fce70 0x7ff8c810fb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.186:44428/0 (socket says 172.21.15.186:44428) 2024-04-12T20:49:50.311 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:49:50.310+0000 7ff8c7e00700 1 -- 172.21.15.186:0/1230854663 learned_addr learned my addr 172.21.15.186:0/1230854663 (peer_addr_for_me v2:172.21.15.186:0/0) 2024-04-12T20:49:50.311 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:49:50.311+0000 7ff8c7400700 1 --2- 172.21.15.186:0/1230854663 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff8c80fdfe0 0x7ff8c8110090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T20:49:50.311 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:49:50.311+0000 7ff8c7e00700 1 -- 172.21.15.186:0/1230854663 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff8c80fdfe0 msgr2=0x7ff8c8110090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:49:50.311 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:49:50.311+0000 7ff8c7e00700 1 --2- 172.21.15.186:0/1230854663 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff8c80fdfe0 0x7ff8c8110090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:49:50.311 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:49:50.311+0000 7ff8c7e00700 1 -- 172.21.15.186:0/1230854663 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8b8005040 con 0x7ff8c80fce70 2024-04-12T20:49:50.311 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:49:50.311+0000 7ff8c7400700 1 --2- 172.21.15.186:0/1230854663 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff8c80fdfe0 0x7ff8c8110090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-12T20:49:50.311 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:49:50.311+0000 7ff8c7e00700 1 --2- 172.21.15.186:0/1230854663 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff8c80fce70 0x7ff8c810fb50 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7ff8b800b3b0 tx=0x7ff8b8009e30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T20:49:50.311 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:49:50.311+0000 7ff8bfe00700 1 -- 172.21.15.186:0/1230854663 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff8b800e040 con 0x7ff8c80fce70 2024-04-12T20:49:50.311 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:49:50.311+0000 7ff8bfe00700 1 -- 172.21.15.186:0/1230854663 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff8b8004290 con 0x7ff8c80fce70 2024-04-12T20:49:50.311 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:49:50.311+0000 7ff8cec00700 1 -- 172.21.15.186:0/1230854663 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff8c81152c0 con 0x7ff8c80fce70 2024-04-12T20:49:50.311 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:49:50.311+0000 7ff8bfe00700 1 -- 172.21.15.186:0/1230854663 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff8b801c630 con 0x7ff8c80fce70 2024-04-12T20:49:50.311 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:49:50.312+0000 7ff8cec00700 1 -- 172.21.15.186:0/1230854663 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff8c8115830 con 0x7ff8c80fce70 2024-04-12T20:49:50.312 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:49:50.313+0000 7ff8cec00700 1 -- 172.21.15.186:0/1230854663 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff8c8101970 con 0x7ff8c80fce70 2024-04-12T20:49:50.312 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:49:50.314+0000 7ff8bfe00700 1 -- 172.21.15.186:0/1230854663 <== mon.1 v2:172.21.15.186:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7ff8b800bbf0 con 0x7ff8c80fce70 2024-04-12T20:49:50.314 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:49:50.315+0000 7ff8bfe00700 1 --2- 172.21.15.186:0/1230854663 >> [v2:172.21.15.18:6800/3048894687,v1:172.21.15.18:6801/3048894687] conn(0x7ff8ac0692a0 0x7ff8ac06b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T20:49:50.314 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:49:50.315+0000 7ff8bfe00700 1 -- 172.21.15.186:0/1230854663 <== mon.1 v2:172.21.15.186:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 4202+0+0 (secure 0 0 0) 0x7ff8b80881b0 con 0x7ff8c80fce70 2024-04-12T20:49:50.314 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:49:50.316+0000 7ff8c7400700 1 --2- 172.21.15.186:0/1230854663 >> [v2:172.21.15.18:6800/3048894687,v1:172.21.15.18:6801/3048894687] conn(0x7ff8ac0692a0 0x7ff8ac06b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T20:49:50.315 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:49:50.317+0000 7ff8c7400700 1 --2- 172.21.15.186:0/1230854663 >> [v2:172.21.15.18:6800/3048894687,v1:172.21.15.18:6801/3048894687] conn(0x7ff8ac0692a0 0x7ff8ac06b750 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7ff8b4006fd0 tx=0x7ff8b4008040 comp rx=0 tx=0).ready entity=mgr.14212 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T20:49:50.319 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:49:50.320+0000 7ff8bfe00700 1 -- 172.21.15.186:0/1230854663 <== mon.1 v2:172.21.15.186:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7ff8b80583c0 con 0x7ff8c80fce70 2024-04-12T20:49:50.537 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:49:50.538+0000 7ff8cec00700 1 -- 172.21.15.186:0/1230854663 --> [v2:172.21.15.18:6800/3048894687,v1:172.21.15.18:6801/3048894687] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi186:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) v1 -- 0x7ff8c802b450 con 0x7ff8ac0692a0 2024-04-12T20:49:51.282 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:49:50 smithi186 ceph-mon[36597]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-12T20:49:51.282 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:49:50 smithi186 ceph-mon[36597]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-12T20:49:51.282 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:49:50 smithi186 ceph-mon[36597]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-12T20:49:51.353 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:49:50 smithi018 ceph-mon[28799]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-12T20:49:51.353 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:49:50 smithi018 ceph-mon[28799]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-12T20:49:51.353 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:49:50 smithi018 ceph-mon[28799]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-12T20:49:52.282 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:49:51 smithi186 ceph-mon[36597]: from='client.24165 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi186:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T20:49:52.282 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:49:51 smithi186 ceph-mon[36597]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-04-12T20:49:52.353 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:49:51 smithi018 ceph-mon[28799]: from='client.24165 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi186:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T20:49:52.353 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:49:51 smithi018 ceph-mon[28799]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-04-12T20:49:54.281 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:49:54 smithi186 ceph-mon[36597]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-04-12T20:49:54.353 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:49:54 smithi018 ceph-mon[28799]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-04-12T20:49:55.282 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:49:55 smithi186 ceph-mon[36597]: from='client.? 172.21.15.186:0/1606603589' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7b5c9f17-6306-4fb5-aa77-fc9a73bcc663"}]: dispatch 2024-04-12T20:49:55.282 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:49:55 smithi186 ceph-mon[36597]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7b5c9f17-6306-4fb5-aa77-fc9a73bcc663"}]: dispatch 2024-04-12T20:49:55.282 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:49:55 smithi186 ceph-mon[36597]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "7b5c9f17-6306-4fb5-aa77-fc9a73bcc663"}]': finished 2024-04-12T20:49:55.282 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:49:55 smithi186 ceph-mon[36597]: osdmap e36: 6 total, 5 up, 6 in 2024-04-12T20:49:55.282 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:49:55 smithi186 ceph-mon[36597]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-12T20:49:55.353 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:49:55 smithi018 ceph-mon[28799]: from='client.? 172.21.15.186:0/1606603589' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7b5c9f17-6306-4fb5-aa77-fc9a73bcc663"}]: dispatch 2024-04-12T20:49:55.353 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:49:55 smithi018 ceph-mon[28799]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7b5c9f17-6306-4fb5-aa77-fc9a73bcc663"}]: dispatch 2024-04-12T20:49:55.353 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:49:55 smithi018 ceph-mon[28799]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "7b5c9f17-6306-4fb5-aa77-fc9a73bcc663"}]': finished 2024-04-12T20:49:55.353 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:49:55 smithi018 ceph-mon[28799]: osdmap e36: 6 total, 5 up, 6 in 2024-04-12T20:49:55.353 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:49:55 smithi018 ceph-mon[28799]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-12T20:49:56.282 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:49:56 smithi186 ceph-mon[36597]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-04-12T20:49:56.282 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:49:56 smithi186 ceph-mon[36597]: from='client.? 172.21.15.186:0/2538721206' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-12T20:49:56.353 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:49:56 smithi018 ceph-mon[28799]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-04-12T20:49:56.353 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:49:56 smithi018 ceph-mon[28799]: from='client.? 172.21.15.186:0/2538721206' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-12T20:49:58.282 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:49:58 smithi186 ceph-mon[36597]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-04-12T20:49:58.353 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:49:58 smithi018 ceph-mon[28799]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-04-12T20:50:00.275 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:50:00 smithi186 ceph-mon[36597]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-04-12T20:50:00.276 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:50:00 smithi186 ceph-mon[36597]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T20:50:00.276 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:50:00 smithi186 ceph-mon[36597]: overall HEALTH_OK 2024-04-12T20:50:00.293 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:50:00 smithi018 ceph-mon[28799]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-04-12T20:50:00.293 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:50:00 smithi018 ceph-mon[28799]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T20:50:00.293 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:50:00 smithi018 ceph-mon[28799]: overall HEALTH_OK 2024-04-12T20:50:02.282 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:50:02 smithi186 ceph-mon[36597]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-04-12T20:50:02.353 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:50:02 smithi018 ceph-mon[28799]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-04-12T20:50:04.282 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:50:04 smithi186 ceph-mon[36597]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-04-12T20:50:04.282 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:50:04 smithi186 ceph-mon[36597]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-04-12T20:50:04.282 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:50:04 smithi186 ceph-mon[36597]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-12T20:50:04.353 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:50:04 smithi018 ceph-mon[28799]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-04-12T20:50:04.353 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:50:04 smithi018 ceph-mon[28799]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-04-12T20:50:04.353 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:50:04 smithi018 ceph-mon[28799]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-12T20:50:05.281 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:50:05 smithi186 ceph-mon[36597]: Deploying daemon osd.5 on smithi186 2024-04-12T20:50:05.353 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:50:05 smithi018 ceph-mon[28799]: Deploying daemon osd.5 on smithi186 2024-04-12T20:50:06.353 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:50:06 smithi018 ceph-mon[28799]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-04-12T20:50:06.532 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:50:06 smithi186 ceph-mon[36597]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-04-12T20:50:08.265 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:50:08 smithi186 ceph-mon[36597]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-04-12T20:50:08.353 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:50:08 smithi018 ceph-mon[28799]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-04-12T20:50:10.351 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:50:10 smithi186 ceph-mon[36597]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-04-12T20:50:10.352 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:50:10 smithi018 ceph-mon[28799]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-04-12T20:50:12.282 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:50:11 smithi186 ceph-mon[36597]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-04-12T20:50:12.282 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:50:11 smithi186 ceph-mon[36597]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' 2024-04-12T20:50:12.282 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:50:11 smithi186 ceph-mon[36597]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' 2024-04-12T20:50:12.282 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:50:11 smithi186 ceph-mon[36597]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-12T20:50:12.353 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:50:11 smithi018 ceph-mon[28799]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-04-12T20:50:12.353 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:50:11 smithi018 ceph-mon[28799]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' 2024-04-12T20:50:12.353 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:50:11 smithi018 ceph-mon[28799]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' 2024-04-12T20:50:12.353 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:50:11 smithi018 ceph-mon[28799]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-12T20:50:14.282 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:50:13 smithi186 ceph-mon[36597]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-04-12T20:50:14.352 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:50:13 smithi018 ceph-mon[28799]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-04-12T20:50:15.282 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:50:15 smithi186 ceph-mon[36597]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T20:50:15.282 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:50:15 smithi186 ceph-mon[36597]: from='osd.5 [v2:172.21.15.186:6816/3543055150,v1:172.21.15.186:6817/3543055150]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-12T20:50:15.282 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:50:15 smithi186 ceph-mon[36597]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-12T20:50:15.353 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:50:15 smithi018 ceph-mon[28799]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T20:50:15.353 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:50:15 smithi018 ceph-mon[28799]: from='osd.5 [v2:172.21.15.186:6816/3543055150,v1:172.21.15.186:6817/3543055150]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-12T20:50:15.353 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:50:15 smithi018 ceph-mon[28799]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-12T20:50:15.652 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:50:15.652+0000 7ff8bfe00700 1 -- 172.21.15.186:0/1230854663 <== mgr.14212 v2:172.21.15.18:6800/3048894687 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7ff8c802b450 con 0x7ff8ac0692a0 2024-04-12T20:50:15.652 INFO:teuthology.orchestra.run.smithi186.stdout:Created osd(s) 5 on host 'smithi186' 2024-04-12T20:50:15.654 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:50:15.655+0000 7ff8cec00700 1 -- 172.21.15.186:0/1230854663 >> [v2:172.21.15.18:6800/3048894687,v1:172.21.15.18:6801/3048894687] conn(0x7ff8ac0692a0 msgr2=0x7ff8ac06b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:50:15.654 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:50:15.655+0000 7ff8cec00700 1 --2- 172.21.15.186:0/1230854663 >> [v2:172.21.15.18:6800/3048894687,v1:172.21.15.18:6801/3048894687] conn(0x7ff8ac0692a0 0x7ff8ac06b750 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7ff8b4006fd0 tx=0x7ff8b4008040 comp rx=0 tx=0).stop 2024-04-12T20:50:15.654 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:50:15.655+0000 7ff8cec00700 1 -- 172.21.15.186:0/1230854663 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff8c80fce70 msgr2=0x7ff8c810fb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:50:15.655 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:50:15.655+0000 7ff8cec00700 1 --2- 172.21.15.186:0/1230854663 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff8c80fce70 0x7ff8c810fb50 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7ff8b800b3b0 tx=0x7ff8b8009e30 comp rx=0 tx=0).stop 2024-04-12T20:50:15.655 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:50:15.655+0000 7ff8cec00700 1 -- 172.21.15.186:0/1230854663 shutdown_connections 2024-04-12T20:50:15.655 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:50:15.655+0000 7ff8cec00700 1 --2- 172.21.15.186:0/1230854663 >> [v2:172.21.15.18:6800/3048894687,v1:172.21.15.18:6801/3048894687] conn(0x7ff8ac0692a0 0x7ff8ac06b750 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:50:15.655 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:50:15.655+0000 7ff8cec00700 1 --2- 172.21.15.186:0/1230854663 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff8c80fce70 0x7ff8c810fb50 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:50:15.655 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:50:15.655+0000 7ff8cec00700 1 --2- 172.21.15.186:0/1230854663 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff8c80fdfe0 0x7ff8c8110090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:50:15.655 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:50:15.655+0000 7ff8cec00700 1 -- 172.21.15.186:0/1230854663 >> 172.21.15.186:0/1230854663 conn(0x7ff8c80f8550 msgr2=0x7ff8c80f9c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T20:50:15.656 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:50:15.656+0000 7ff8cec00700 1 -- 172.21.15.186:0/1230854663 shutdown_connections 2024-04-12T20:50:15.656 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:50:15.656+0000 7ff8cec00700 1 -- 172.21.15.186:0/1230854663 wait complete. 2024-04-12T20:50:16.281 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:50:16 smithi186 ceph-mon[36597]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-04-12T20:50:16.282 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:50:16 smithi186 ceph-mon[36597]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-04-12T20:50:16.282 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:50:16 smithi186 ceph-mon[36597]: osdmap e37: 6 total, 5 up, 6 in 2024-04-12T20:50:16.282 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:50:16 smithi186 ceph-mon[36597]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-12T20:50:16.282 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:50:16 smithi186 ceph-mon[36597]: from='osd.5 [v2:172.21.15.186:6816/3543055150,v1:172.21.15.186:6817/3543055150]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi186", "root=default"]}]: dispatch 2024-04-12T20:50:16.282 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:50:16 smithi186 ceph-mon[36597]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi186", "root=default"]}]: dispatch 2024-04-12T20:50:16.282 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:50:16 smithi186 ceph-mon[36597]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' 2024-04-12T20:50:16.282 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:50:16 smithi186 ceph-mon[36597]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' 2024-04-12T20:50:16.353 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:50:16 smithi018 ceph-mon[28799]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-04-12T20:50:16.353 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:50:16 smithi018 ceph-mon[28799]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-04-12T20:50:16.353 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:50:16 smithi018 ceph-mon[28799]: osdmap e37: 6 total, 5 up, 6 in 2024-04-12T20:50:16.353 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:50:16 smithi018 ceph-mon[28799]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-12T20:50:16.353 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:50:16 smithi018 ceph-mon[28799]: from='osd.5 [v2:172.21.15.186:6816/3543055150,v1:172.21.15.186:6817/3543055150]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi186", "root=default"]}]: dispatch 2024-04-12T20:50:16.353 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:50:16 smithi018 ceph-mon[28799]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi186", "root=default"]}]: dispatch 2024-04-12T20:50:16.353 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:50:16 smithi018 ceph-mon[28799]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' 2024-04-12T20:50:16.354 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:50:16 smithi018 ceph-mon[28799]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' 2024-04-12T20:50:16.517 DEBUG:teuthology.orchestra.run.smithi186:osd.5> sudo journalctl -f -n 0 -u ceph-1cbe407a-f90d-11ee-bc8d-c7b262605968@osd.5.service 2024-04-12T20:50:16.523 INFO:tasks.cephadm:Waiting for 6 OSDs to come up... 2024-04-12T20:50:16.523 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1cbe407a-f90d-11ee-bc8d-c7b262605968 -- ceph osd stat -f json 2024-04-12T20:50:17.051 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/mon.smithi018/config 2024-04-12T20:50:17.282 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:50:17 smithi186 ceph-mon[36597]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi186", "root=default"]}]': finished 2024-04-12T20:50:17.282 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:50:17 smithi186 ceph-mon[36597]: osdmap e38: 6 total, 5 up, 6 in 2024-04-12T20:50:17.282 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:50:17 smithi186 ceph-mon[36597]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-12T20:50:17.282 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:50:17 smithi186 ceph-mon[36597]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-12T20:50:17.282 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:50:17 smithi186 ceph-mon[36597]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' 2024-04-12T20:50:17.282 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:50:17 smithi186 ceph-mon[36597]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' 2024-04-12T20:50:17.353 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:50:17 smithi018 ceph-mon[28799]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi186", "root=default"]}]': finished 2024-04-12T20:50:17.353 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:50:17 smithi018 ceph-mon[28799]: osdmap e38: 6 total, 5 up, 6 in 2024-04-12T20:50:17.354 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:50:17 smithi018 ceph-mon[28799]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-12T20:50:17.354 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:50:17 smithi018 ceph-mon[28799]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-12T20:50:17.354 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:50:17 smithi018 ceph-mon[28799]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' 2024-04-12T20:50:17.354 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:50:17 smithi018 ceph-mon[28799]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' 2024-04-12T20:50:18.045 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:50:18 smithi186 ceph-mon[36597]: purged_snaps scrub starts 2024-04-12T20:50:18.045 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:50:18 smithi186 ceph-mon[36597]: purged_snaps scrub ok 2024-04-12T20:50:18.045 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:50:18 smithi186 ceph-mon[36597]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-04-12T20:50:18.045 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:50:18 smithi186 ceph-mon[36597]: osd.5 [v2:172.21.15.186:6816/3543055150,v1:172.21.15.186:6817/3543055150] boot 2024-04-12T20:50:18.046 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:50:18 smithi186 ceph-mon[36597]: osdmap e39: 6 total, 6 up, 6 in 2024-04-12T20:50:18.046 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:50:18 smithi186 ceph-mon[36597]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-12T20:50:18.353 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:50:18 smithi018 ceph-mon[28799]: purged_snaps scrub starts 2024-04-12T20:50:18.353 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:50:18 smithi018 ceph-mon[28799]: purged_snaps scrub ok 2024-04-12T20:50:18.353 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:50:18 smithi018 ceph-mon[28799]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-04-12T20:50:18.353 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:50:18 smithi018 ceph-mon[28799]: osd.5 [v2:172.21.15.186:6816/3543055150,v1:172.21.15.186:6817/3543055150] boot 2024-04-12T20:50:18.353 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:50:18 smithi018 ceph-mon[28799]: osdmap e39: 6 total, 6 up, 6 in 2024-04-12T20:50:18.353 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:50:18 smithi018 ceph-mon[28799]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-12T20:50:19.306 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:19.306+0000 7f3ccc600700 1 -- 172.21.15.18:0/1690301170 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3cc4071ea0 msgr2=0x7f3cc40703b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:50:19.306 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:19.306+0000 7f3ccc600700 1 --2- 172.21.15.18:0/1690301170 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3cc4071ea0 0x7f3cc40703b0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f3cb8009ce0 tx=0x7f3cb8004d10 comp rx=0 tx=0).stop 2024-04-12T20:50:19.306 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:19.306+0000 7f3ccc600700 1 -- 172.21.15.18:0/1690301170 shutdown_connections 2024-04-12T20:50:19.306 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:19.306+0000 7f3ccc600700 1 --2- 172.21.15.18:0/1690301170 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3cc40708f0 0x7f3cc4070d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:50:19.306 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:19.306+0000 7f3ccc600700 1 --2- 172.21.15.18:0/1690301170 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3cc4071ea0 0x7f3cc40703b0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:50:19.306 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:19.306+0000 7f3ccc600700 1 -- 172.21.15.18:0/1690301170 >> 172.21.15.18:0/1690301170 conn(0x7f3cc40f8560 msgr2=0x7f3cc40fa970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T20:50:19.306 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:19.306+0000 7f3ccc600700 1 -- 172.21.15.18:0/1690301170 shutdown_connections 2024-04-12T20:50:19.306 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:19.306+0000 7f3ccc600700 1 -- 172.21.15.18:0/1690301170 wait complete. 2024-04-12T20:50:19.307 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:50:19 smithi018 ceph-mon[28799]: osdmap e40: 6 total, 6 up, 6 in 2024-04-12T20:50:19.307 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:50:19 smithi018 ceph-mon[28799]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-12T20:50:19.308 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:19.307+0000 7f3ccc600700 1 Processor -- start 2024-04-12T20:50:19.308 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:19.307+0000 7f3ccc600700 1 -- start start 2024-04-12T20:50:19.308 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:19.307+0000 7f3ccc600700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3cc40708f0 0x7f3cc410fea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T20:50:19.308 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:19.307+0000 7f3ccc600700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3cc4071ea0 0x7f3cc41103e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T20:50:19.308 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:19.307+0000 7f3ccc600700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3cc4111150 con 0x7f3cc4071ea0 2024-04-12T20:50:19.308 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:19.307+0000 7f3ccc600700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3cc41112c0 con 0x7f3cc40708f0 2024-04-12T20:50:19.308 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:19.308+0000 7f3cc9200700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3cc4071ea0 0x7f3cc41103e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T20:50:19.308 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:19.308+0000 7f3cc9200700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3cc4071ea0 0x7f3cc41103e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:57312/0 (socket says 172.21.15.18:57312) 2024-04-12T20:50:19.308 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:19.308+0000 7f3cc9200700 1 -- 172.21.15.18:0/1354760889 learned_addr learned my addr 172.21.15.18:0/1354760889 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T20:50:19.308 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:19.308+0000 7f3cc9c00700 1 --2- 172.21.15.18:0/1354760889 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3cc40708f0 0x7f3cc410fea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T20:50:19.308 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:19.308+0000 7f3cc9200700 1 -- 172.21.15.18:0/1354760889 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3cc40708f0 msgr2=0x7f3cc410fea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:50:19.309 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:19.308+0000 7f3cc9200700 1 --2- 172.21.15.18:0/1354760889 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3cc40708f0 0x7f3cc410fea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:50:19.309 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:19.308+0000 7f3cc9200700 1 -- 172.21.15.18:0/1354760889 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3cb8009940 con 0x7f3cc4071ea0 2024-04-12T20:50:19.309 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:19.309+0000 7f3cc9200700 1 --2- 172.21.15.18:0/1354760889 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3cc4071ea0 0x7f3cc41103e0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f3cb0002a50 tx=0x7f3cb00046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T20:50:19.309 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:19.309+0000 7f3cc1600700 1 -- 172.21.15.18:0/1354760889 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3cb0007cc0 con 0x7f3cc4071ea0 2024-04-12T20:50:19.309 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:19.309+0000 7f3ccc600700 1 -- 172.21.15.18:0/1354760889 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3cc4111540 con 0x7f3cc4071ea0 2024-04-12T20:50:19.309 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:19.309+0000 7f3cc1600700 1 -- 172.21.15.18:0/1354760889 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f3cb000f040 con 0x7f3cc4071ea0 2024-04-12T20:50:19.309 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:19.309+0000 7f3cc1600700 1 -- 172.21.15.18:0/1354760889 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3cb0009640 con 0x7f3cc4071ea0 2024-04-12T20:50:19.309 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:19.309+0000 7f3ccc600700 1 -- 172.21.15.18:0/1354760889 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3cc4111b00 con 0x7f3cc4071ea0 2024-04-12T20:50:19.310 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:19.310+0000 7f3ccc600700 1 -- 172.21.15.18:0/1354760889 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3cc405f5f0 con 0x7f3cc4071ea0 2024-04-12T20:50:19.310 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:19.312+0000 7f3cc1600700 1 -- 172.21.15.18:0/1354760889 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7f3cb00097a0 con 0x7f3cc4071ea0 2024-04-12T20:50:19.313 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:19.314+0000 7f3cc1600700 1 --2- 172.21.15.18:0/1354760889 >> [v2:172.21.15.18:6800/3048894687,v1:172.21.15.18:6801/3048894687] conn(0x7f3cac0692b0 0x7f3cac06b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T20:50:19.313 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:19.314+0000 7f3cc1600700 1 -- 172.21.15.18:0/1354760889 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4634+0+0 (secure 0 0 0) 0x7f3cb0086d20 con 0x7f3cc4071ea0 2024-04-12T20:50:19.313 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:19.314+0000 7f3cc9c00700 1 --2- 172.21.15.18:0/1354760889 >> [v2:172.21.15.18:6800/3048894687,v1:172.21.15.18:6801/3048894687] conn(0x7f3cac0692b0 0x7f3cac06b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T20:50:19.314 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:19.315+0000 7f3cc9c00700 1 --2- 172.21.15.18:0/1354760889 >> [v2:172.21.15.18:6800/3048894687,v1:172.21.15.18:6801/3048894687] conn(0x7f3cac0692b0 0x7f3cac06b760 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f3cb80094a0 tx=0x7f3cb8004d10 comp rx=0 tx=0).ready entity=mgr.14212 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T20:50:19.318 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:19.319+0000 7f3cc1600700 1 -- 172.21.15.18:0/1354760889 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7f3cb0053270 con 0x7f3cc4071ea0 2024-04-12T20:50:19.532 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:50:19 smithi186 ceph-mon[36597]: osdmap e40: 6 total, 6 up, 6 in 2024-04-12T20:50:19.532 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:50:19 smithi186 ceph-mon[36597]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-12T20:50:19.535 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:19.535+0000 7f3ccc600700 1 -- 172.21.15.18:0/1354760889 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) v1 -- 0x7f3cc4107ae0 con 0x7f3cc4071ea0 2024-04-12T20:50:19.536 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:19.536+0000 7f3cc1600700 1 -- 172.21.15.18:0/1354760889 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v40) v1 ==== 74+0+130 (secure 0 0 0) 0x7f3cb0056890 con 0x7f3cc4071ea0 2024-04-12T20:50:19.536 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-12T20:50:19.538 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:19.539+0000 7f3ccc600700 1 -- 172.21.15.18:0/1354760889 >> [v2:172.21.15.18:6800/3048894687,v1:172.21.15.18:6801/3048894687] conn(0x7f3cac0692b0 msgr2=0x7f3cac06b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:50:19.539 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:19.539+0000 7f3ccc600700 1 --2- 172.21.15.18:0/1354760889 >> [v2:172.21.15.18:6800/3048894687,v1:172.21.15.18:6801/3048894687] conn(0x7f3cac0692b0 0x7f3cac06b760 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f3cb80094a0 tx=0x7f3cb8004d10 comp rx=0 tx=0).stop 2024-04-12T20:50:19.539 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:19.539+0000 7f3ccc600700 1 -- 172.21.15.18:0/1354760889 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3cc4071ea0 msgr2=0x7f3cc41103e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:50:19.539 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:19.539+0000 7f3ccc600700 1 --2- 172.21.15.18:0/1354760889 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3cc4071ea0 0x7f3cc41103e0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f3cb0002a50 tx=0x7f3cb00046b0 comp rx=0 tx=0).stop 2024-04-12T20:50:19.539 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:19.540+0000 7f3ccc600700 1 -- 172.21.15.18:0/1354760889 shutdown_connections 2024-04-12T20:50:19.539 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:19.540+0000 7f3ccc600700 1 --2- 172.21.15.18:0/1354760889 >> [v2:172.21.15.18:6800/3048894687,v1:172.21.15.18:6801/3048894687] conn(0x7f3cac0692b0 0x7f3cac06b760 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:50:19.539 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:19.540+0000 7f3ccc600700 1 --2- 172.21.15.18:0/1354760889 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3cc40708f0 0x7f3cc410fea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:50:19.539 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:19.540+0000 7f3ccc600700 1 --2- 172.21.15.18:0/1354760889 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3cc4071ea0 0x7f3cc41103e0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:50:19.540 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:19.540+0000 7f3ccc600700 1 -- 172.21.15.18:0/1354760889 >> 172.21.15.18:0/1354760889 conn(0x7f3cc40f8560 msgr2=0x7f3cc40fa080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T20:50:19.540 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:19.540+0000 7f3ccc600700 1 -- 172.21.15.18:0/1354760889 shutdown_connections 2024-04-12T20:50:19.540 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:19.540+0000 7f3ccc600700 1 -- 172.21.15.18:0/1354760889 wait complete. 2024-04-12T20:50:20.411 INFO:teuthology.orchestra.run.smithi018.stdout:{"epoch":40,"num_osds":6,"num_up_osds":6,"osd_up_since":1712955017,"num_in_osds":6,"osd_in_since":1712954994,"num_remapped_pgs":0} 2024-04-12T20:50:20.411 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid 1cbe407a-f90d-11ee-bc8d-c7b262605968 -- ceph osd dump --format=json 2024-04-12T20:50:20.491 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:50:20 smithi186 ceph-mon[36597]: from='client.? 172.21.15.18:0/1354760889' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-04-12T20:50:20.504 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:50:20 smithi018 ceph-mon[28799]: from='client.? 172.21.15.18:0/1354760889' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-04-12T20:50:20.922 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/mon.smithi018/config 2024-04-12T20:50:21.323 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:50:21 smithi018 ceph-mon[28799]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-12T20:50:21.532 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:50:21 smithi186 ceph-mon[36597]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-12T20:50:22.732 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:50:22 smithi018 ceph-mon[28799]: Detected new or changed devices on smithi186 2024-04-12T20:50:22.733 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:50:22 smithi018 ceph-mon[28799]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' 2024-04-12T20:50:22.733 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:50:22 smithi018 ceph-mon[28799]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' 2024-04-12T20:50:22.733 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:50:22 smithi018 ceph-mon[28799]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-12T20:50:22.733 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:50:22 smithi018 ceph-mon[28799]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-12T20:50:22.733 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:50:22 smithi018 ceph-mon[28799]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-12T20:50:22.733 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:50:22 smithi018 ceph-mon[28799]: Adjusting osd_memory_target on smithi186 to 5312M 2024-04-12T20:50:22.733 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:50:22 smithi018 ceph-mon[28799]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' 2024-04-12T20:50:22.733 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:50:22 smithi018 ceph-mon[28799]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-12T20:50:22.733 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:50:22 smithi018 ceph-mon[28799]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-12T20:50:22.733 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:50:22 smithi018 ceph-mon[28799]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' 2024-04-12T20:50:22.733 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:50:22 smithi018 ceph-mon[28799]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-12T20:50:22.733 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:50:22 smithi018 ceph-mon[28799]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-12T20:50:22.733 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:50:22 smithi018 ceph-mon[28799]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-12T20:50:22.733 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:50:22 smithi018 ceph-mon[28799]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' 2024-04-12T20:50:22.758 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:50:22 smithi186 ceph-mon[36597]: Detected new or changed devices on smithi186 2024-04-12T20:50:22.758 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:50:22 smithi186 ceph-mon[36597]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' 2024-04-12T20:50:22.758 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:50:22 smithi186 ceph-mon[36597]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' 2024-04-12T20:50:22.758 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:50:22 smithi186 ceph-mon[36597]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-12T20:50:22.758 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:50:22 smithi186 ceph-mon[36597]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-12T20:50:22.758 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:50:22 smithi186 ceph-mon[36597]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-12T20:50:22.758 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:50:22 smithi186 ceph-mon[36597]: Adjusting osd_memory_target on smithi186 to 5312M 2024-04-12T20:50:22.758 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:50:22 smithi186 ceph-mon[36597]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' 2024-04-12T20:50:22.758 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:50:22 smithi186 ceph-mon[36597]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-12T20:50:22.759 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:50:22 smithi186 ceph-mon[36597]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-12T20:50:22.759 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:50:22 smithi186 ceph-mon[36597]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' 2024-04-12T20:50:22.759 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:50:22 smithi186 ceph-mon[36597]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-12T20:50:22.759 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:50:22 smithi186 ceph-mon[36597]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-12T20:50:22.759 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:50:22 smithi186 ceph-mon[36597]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-12T20:50:22.759 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:50:22 smithi186 ceph-mon[36597]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' 2024-04-12T20:50:23.216 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:23.216+0000 7fe8cd000700 1 -- 172.21.15.18:0/594899281 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe8c81059c0 msgr2=0x7fe8c80ff080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:50:23.217 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:23.216+0000 7fe8cd000700 1 --2- 172.21.15.18:0/594899281 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe8c81059c0 0x7fe8c80ff080 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fe8b4004940 tx=0x7fe8b4009e30 comp rx=0 tx=0).stop 2024-04-12T20:50:23.217 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:23.217+0000 7fe8cd000700 1 -- 172.21.15.18:0/594899281 shutdown_connections 2024-04-12T20:50:23.217 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:23.217+0000 7fe8cd000700 1 --2- 172.21.15.18:0/594899281 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe8c81059c0 0x7fe8c80ff080 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:50:23.217 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:23.217+0000 7fe8cd000700 1 --2- 172.21.15.18:0/594899281 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe8c8105020 0x7fe8c81053f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:50:23.217 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:23.217+0000 7fe8cd000700 1 -- 172.21.15.18:0/594899281 >> 172.21.15.18:0/594899281 conn(0x7fe8c80faba0 msgr2=0x7fe8c80fcfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T20:50:23.217 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:23.217+0000 7fe8cd000700 1 -- 172.21.15.18:0/594899281 shutdown_connections 2024-04-12T20:50:23.217 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:23.217+0000 7fe8cd000700 1 -- 172.21.15.18:0/594899281 wait complete. 2024-04-12T20:50:23.218 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:23.218+0000 7fe8cd000700 1 Processor -- start 2024-04-12T20:50:23.219 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:23.218+0000 7fe8cd000700 1 -- start start 2024-04-12T20:50:23.219 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:23.218+0000 7fe8cd000700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe8c8105020 0x7fe8c8115ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T20:50:23.219 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:23.218+0000 7fe8cd000700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe8c81059c0 0x7fe8c81160e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T20:50:23.219 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:23.218+0000 7fe8cd000700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8c8116730 con 0x7fe8c8105020 2024-04-12T20:50:23.219 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:23.218+0000 7fe8cd000700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8c810fd10 con 0x7fe8c81059c0 2024-04-12T20:50:23.219 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:23.219+0000 7fe8c5e00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe8c8105020 0x7fe8c8115ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T20:50:23.219 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:23.219+0000 7fe8c5e00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe8c8105020 0x7fe8c8115ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:48624/0 (socket says 172.21.15.18:48624) 2024-04-12T20:50:23.219 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:23.219+0000 7fe8c5e00700 1 -- 172.21.15.18:0/2169425458 learned_addr learned my addr 172.21.15.18:0/2169425458 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T20:50:23.219 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:23.219+0000 7fe8c5400700 1 --2- 172.21.15.18:0/2169425458 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe8c81059c0 0x7fe8c81160e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T20:50:23.219 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:23.219+0000 7fe8c5e00700 1 -- 172.21.15.18:0/2169425458 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe8c81059c0 msgr2=0x7fe8c81160e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:50:23.219 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:23.219+0000 7fe8c5e00700 1 --2- 172.21.15.18:0/2169425458 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe8c81059c0 0x7fe8c81160e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:50:23.219 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:23.219+0000 7fe8c5e00700 1 -- 172.21.15.18:0/2169425458 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8b4005040 con 0x7fe8c8105020 2024-04-12T20:50:23.219 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:23.220+0000 7fe8c5e00700 1 --2- 172.21.15.18:0/2169425458 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe8c8105020 0x7fe8c8115ba0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fe8b800c830 tx=0x7fe8b8004470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T20:50:23.220 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:23.220+0000 7fe8be000700 1 -- 172.21.15.18:0/2169425458 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe8b800a760 con 0x7fe8c8105020 2024-04-12T20:50:23.220 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:23.220+0000 7fe8be000700 1 -- 172.21.15.18:0/2169425458 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe8b800ad90 con 0x7fe8c8105020 2024-04-12T20:50:23.220 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:23.220+0000 7fe8be000700 1 -- 172.21.15.18:0/2169425458 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe8b80113a0 con 0x7fe8c8105020 2024-04-12T20:50:23.220 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:23.220+0000 7fe8cd000700 1 -- 172.21.15.18:0/2169425458 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe8c810fff0 con 0x7fe8c8105020 2024-04-12T20:50:23.220 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:23.220+0000 7fe8cd000700 1 -- 172.21.15.18:0/2169425458 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe8c81105b0 con 0x7fe8c8105020 2024-04-12T20:50:23.221 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:23.221+0000 7fe8cd000700 1 -- 172.21.15.18:0/2169425458 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe8c8100770 con 0x7fe8c8105020 2024-04-12T20:50:23.221 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:23.222+0000 7fe8be000700 1 -- 172.21.15.18:0/2169425458 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7fe8b8012400 con 0x7fe8c8105020 2024-04-12T20:50:23.223 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:23.223+0000 7fe8be000700 1 --2- 172.21.15.18:0/2169425458 >> [v2:172.21.15.18:6800/3048894687,v1:172.21.15.18:6801/3048894687] conn(0x7fe8ac0692a0 0x7fe8ac06b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T20:50:23.223 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:23.224+0000 7fe8be000700 1 -- 172.21.15.18:0/2169425458 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4634+0+0 (secure 0 0 0) 0x7fe8b8086ec0 con 0x7fe8c8105020 2024-04-12T20:50:23.223 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:23.224+0000 7fe8c5400700 1 --2- 172.21.15.18:0/2169425458 >> [v2:172.21.15.18:6800/3048894687,v1:172.21.15.18:6801/3048894687] conn(0x7fe8ac0692a0 0x7fe8ac06b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T20:50:23.223 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:23.224+0000 7fe8c5400700 1 --2- 172.21.15.18:0/2169425458 >> [v2:172.21.15.18:6800/3048894687,v1:172.21.15.18:6801/3048894687] conn(0x7fe8ac0692a0 0x7fe8ac06b750 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fe8b400b020 tx=0x7fe8b4009e30 comp rx=0 tx=0).ready entity=mgr.14212 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T20:50:23.228 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:23.229+0000 7fe8be000700 1 -- 172.21.15.18:0/2169425458 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7fe8b8056fa0 con 0x7fe8c8105020 2024-04-12T20:50:23.448 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:23.449+0000 7fe8cd000700 1 -- 172.21.15.18:0/2169425458 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe8c804c710 con 0x7fe8c8105020 2024-04-12T20:50:23.449 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:23.450+0000 7fe8be000700 1 -- 172.21.15.18:0/2169425458 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v40) v1 ==== 74+0+11072 (secure 0 0 0) 0x7fe8b8056b30 con 0x7fe8c8105020 2024-04-12T20:50:23.450 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-12T20:50:23.450 INFO:teuthology.orchestra.run.smithi018.stdout:{"epoch":40,"fsid":"1cbe407a-f90d-11ee-bc8d-c7b262605968","created":"2024-04-12T20:43:00.692361+0000","modified":"2024-04-12T20:50:18.227750+0000","last_up_change":"2024-04-12T20:50:17.011011+0000","last_in_change":"2024-04-12T20:49:54.190644+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":14,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-12T20:48:37.529081+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}}],"osds":[{"osd":0,"uuid":"2af887cc-0447-42aa-a8f3-1aa7fcd4dc8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6802","nonce":585613146},{"type":"v1","addr":"172.21.15.18:6803","nonce":585613146}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6804","nonce":585613146},{"type":"v1","addr":"172.21.15.18:6805","nonce":585613146}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6808","nonce":585613146},{"type":"v1","addr":"172.21.15.18:6809","nonce":585613146}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6806","nonce":585613146},{"type":"v1","addr":"172.21.15.18:6807","nonce":585613146}]},"public_addr":"172.21.15.18:6803/585613146","cluster_addr":"172.21.15.18:6805/585613146","heartbeat_back_addr":"172.21.15.18:6809/585613146","heartbeat_front_addr":"172.21.15.18:6807/585613146","state":["exists","up"]},{"osd":1,"uuid":"6ba7b0d3-a03d-45cc-b4e9-89a92c8c6ca8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6810","nonce":181951035},{"type":"v1","addr":"172.21.15.18:6811","nonce":181951035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6812","nonce":181951035},{"type":"v1","addr":"172.21.15.18:6813","nonce":181951035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6816","nonce":181951035},{"type":"v1","addr":"172.21.15.18:6817","nonce":181951035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6814","nonce":181951035},{"type":"v1","addr":"172.21.15.18:6815","nonce":181951035}]},"public_addr":"172.21.15.18:6811/181951035","cluster_addr":"172.21.15.18:6813/181951035","heartbeat_back_addr":"172.21.15.18:6817/181951035","heartbeat_front_addr":"172.21.15.18:6815/181951035","state":["exists","up"]},{"osd":2,"uuid":"5bb1137d-5a3c-40bc-ae08-fc792b8ee547","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6818","nonce":3573247809},{"type":"v1","addr":"172.21.15.18:6819","nonce":3573247809}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6820","nonce":3573247809},{"type":"v1","addr":"172.21.15.18:6821","nonce":3573247809}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6824","nonce":3573247809},{"type":"v1","addr":"172.21.15.18:6825","nonce":3573247809}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6822","nonce":3573247809},{"type":"v1","addr":"172.21.15.18:6823","nonce":3573247809}]},"public_addr":"172.21.15.18:6819/3573247809","cluster_addr":"172.21.15.18:6821/3573247809","heartbeat_back_addr":"172.21.15.18:6825/3573247809","heartbeat_front_addr":"172.21.15.18:6823/3573247809","state":["exists","up"]},{"osd":3,"uuid":"a5140a33-1e90-4980-aa9a-b0afe6b155f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6800","nonce":2184799172},{"type":"v1","addr":"172.21.15.186:6801","nonce":2184799172}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6802","nonce":2184799172},{"type":"v1","addr":"172.21.15.186:6803","nonce":2184799172}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6806","nonce":2184799172},{"type":"v1","addr":"172.21.15.186:6807","nonce":2184799172}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6804","nonce":2184799172},{"type":"v1","addr":"172.21.15.186:6805","nonce":2184799172}]},"public_addr":"172.21.15.186:6801/2184799172","cluster_addr":"172.21.15.186:6803/2184799172","heartbeat_back_addr":"172.21.15.186:6807/2184799172","heartbeat_front_addr":"172.21.15.186:6805/2184799172","state":["exists","up"]},{"osd":4,"uuid":"d6b2c6b3-6c78-4261-bcbe-c33358407198","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6808","nonce":3584843287},{"type":"v1","addr":"172.21.15.186:6809","nonce":3584843287}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6810","nonce":3584843287},{"type":"v1","addr":"172.21.15.186:6811","nonce":3584843287}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6814","nonce":3584843287},{"type":"v1","addr":"172.21.15.186:6815","nonce":3584843287}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6812","nonce":3584843287},{"type":"v1","addr":"172.21.15.186:6813","nonce":3584843287}]},"public_addr":"172.21.15.186:6809/3584843287","cluster_addr":"172.21.15.186:6811/3584843287","heartbeat_back_addr":"172.21.15.186:6815/3584843287","heartbeat_front_addr":"172.21.15.186:6813/3584843287","state":["exists","up"]},{"osd":5,"uuid":"7b5c9f17-6306-4fb5-aa77-fc9a73bcc663","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6816","nonce":3543055150},{"type":"v1","addr":"172.21.15.186:6817","nonce":3543055150}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6818","nonce":3543055150},{"type":"v1","addr":"172.21.15.186:6819","nonce":3543055150}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6822","nonce":3543055150},{"type":"v1","addr":"172.21.15.186:6823","nonce":3543055150}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6820","nonce":3543055150},{"type":"v1","addr":"172.21.15.186:6821","nonce":3543055150}]},"public_addr":"172.21.15.186:6817/3543055150","cluster_addr":"172.21.15.186:6819/3543055150","heartbeat_back_addr":"172.21.15.186:6823/3543055150","heartbeat_front_addr":"172.21.15.186:6821/3543055150","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-04-12T20:47:21.231060+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-04-12T20:47:57.699205+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-04-12T20:48:35.096648+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-04-12T20:49:08.252810+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-04-12T20:49:41.102187+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-04-12T20:50:16.009813+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.18:0/1720919549":"2024-04-13T20:45:44.776072+0000","172.21.15.18:0/3896210349":"2024-04-13T20:44:15.143139+0000","172.21.15.18:6801/1391635591":"2024-04-13T20:45:44.776072+0000","172.21.15.18:6800/1391635591":"2024-04-13T20:45:44.776072+0000","172.21.15.18:0/3387911038":"2024-04-13T20:44:15.143139+0000","172.21.15.18:0/4018777476":"2024-04-13T20:45:44.776072+0000","172.21.15.18:6801/3403971131":"2024-04-13T20:44:15.143139+0000","172.21.15.18:0/2945432483":"2024-04-13T20:44:15.143139+0000","172.21.15.18:0/1732042827":"2024-04-13T20:43:34.269559+0000","172.21.15.18:0/766256564":"2024-04-13T20:45:44.776072+0000","172.21.15.18:0/2517252399":"2024-04-13T20:43:34.269559+0000","172.21.15.18:0/3295870133":"2024-04-13T20:43:34.269559+0000","172.21.15.18:6800/3403971131":"2024-04-13T20:44:15.143139+0000","172.21.15.18:6800/1312802974":"2024-04-13T20:43:34.269559+0000","172.21.15.18:6801/1312802974":"2024-04-13T20:43:34.269559+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-12T20:50:23.453 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:23.453+0000 7fe8cd000700 1 -- 172.21.15.18:0/2169425458 >> [v2:172.21.15.18:6800/3048894687,v1:172.21.15.18:6801/3048894687] conn(0x7fe8ac0692a0 msgr2=0x7fe8ac06b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:50:23.453 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:23.453+0000 7fe8cd000700 1 --2- 172.21.15.18:0/2169425458 >> [v2:172.21.15.18:6800/3048894687,v1:172.21.15.18:6801/3048894687] conn(0x7fe8ac0692a0 0x7fe8ac06b750 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fe8b400b020 tx=0x7fe8b4009e30 comp rx=0 tx=0).stop 2024-04-12T20:50:23.453 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:23.453+0000 7fe8cd000700 1 -- 172.21.15.18:0/2169425458 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe8c8105020 msgr2=0x7fe8c8115ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:50:23.453 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:23.453+0000 7fe8cd000700 1 --2- 172.21.15.18:0/2169425458 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe8c8105020 0x7fe8c8115ba0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fe8b800c830 tx=0x7fe8b8004470 comp rx=0 tx=0).stop 2024-04-12T20:50:23.453 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:23.453+0000 7fe8cd000700 1 -- 172.21.15.18:0/2169425458 shutdown_connections 2024-04-12T20:50:23.453 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:23.453+0000 7fe8cd000700 1 --2- 172.21.15.18:0/2169425458 >> [v2:172.21.15.18:6800/3048894687,v1:172.21.15.18:6801/3048894687] conn(0x7fe8ac0692a0 0x7fe8ac06b750 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:50:23.453 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:23.453+0000 7fe8cd000700 1 --2- 172.21.15.18:0/2169425458 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe8c8105020 0x7fe8c8115ba0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:50:23.453 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:23.453+0000 7fe8cd000700 1 --2- 172.21.15.18:0/2169425458 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe8c81059c0 0x7fe8c81160e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:50:23.453 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:23.453+0000 7fe8cd000700 1 -- 172.21.15.18:0/2169425458 >> 172.21.15.18:0/2169425458 conn(0x7fe8c80faba0 msgr2=0x7fe8c80fbb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T20:50:23.453 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:23.453+0000 7fe8cd000700 1 -- 172.21.15.18:0/2169425458 shutdown_connections 2024-04-12T20:50:23.453 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:23.453+0000 7fe8cd000700 1 -- 172.21.15.18:0/2169425458 wait complete. 2024-04-12T20:50:23.781 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:50:23 smithi186 ceph-mon[36597]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-12T20:50:23.813 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:50:23 smithi018 ceph-mon[28799]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-12T20:50:24.203 INFO:tasks.cephadm.ceph_manager.ceph:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-12T20:48:37.529081+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}] 2024-04-12T20:50:24.204 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid 1cbe407a-f90d-11ee-bc8d-c7b262605968 -- ceph osd pool get .mgr pg_num 2024-04-12T20:50:24.702 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/mon.smithi018/config 2024-04-12T20:50:24.781 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:50:24 smithi186 ceph-mon[36597]: from='client.? 172.21.15.18:0/2169425458' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-12T20:50:24.852 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:50:24 smithi018 ceph-mon[28799]: from='client.? 172.21.15.18:0/2169425458' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-12T20:50:25.603 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:50:25 smithi018 ceph-mon[28799]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-12T20:50:25.781 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:50:25 smithi186 ceph-mon[36597]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-12T20:50:26.650 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:26.650+0000 7f4150800700 1 -- 172.21.15.18:0/3481555257 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4148101d70 msgr2=0x7f4148102140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:50:26.651 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:26.650+0000 7f4150800700 1 --2- 172.21.15.18:0/3481555257 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4148101d70 0x7f4148102140 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f413c004940 tx=0x7f413c009e30 comp rx=0 tx=0).stop 2024-04-12T20:50:26.651 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:26.650+0000 7f4150800700 1 -- 172.21.15.18:0/3481555257 shutdown_connections 2024-04-12T20:50:26.651 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:26.650+0000 7f4150800700 1 --2- 172.21.15.18:0/3481555257 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4148102710 0x7f41480fbd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:50:26.651 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:26.650+0000 7f4150800700 1 --2- 172.21.15.18:0/3481555257 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4148101d70 0x7f4148102140 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:50:26.651 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:26.650+0000 7f4150800700 1 -- 172.21.15.18:0/3481555257 >> 172.21.15.18:0/3481555257 conn(0x7f4148072bc0 msgr2=0x7f4148072fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T20:50:26.651 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:26.650+0000 7f4150800700 1 -- 172.21.15.18:0/3481555257 shutdown_connections 2024-04-12T20:50:26.651 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:26.650+0000 7f4150800700 1 -- 172.21.15.18:0/3481555257 wait complete. 2024-04-12T20:50:26.651 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:26.651+0000 7f4150800700 1 Processor -- start 2024-04-12T20:50:26.652 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:26.652+0000 7f4150800700 1 -- start start 2024-04-12T20:50:26.653 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:26.652+0000 7f4150800700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4148101d70 0x7f414810eaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T20:50:26.653 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:26.652+0000 7f4150800700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4148102710 0x7f414810f030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T20:50:26.653 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:26.652+0000 7f4150800700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f414810f710 con 0x7f4148102710 2024-04-12T20:50:26.653 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:26.652+0000 7f4150800700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f414810f850 con 0x7f4148101d70 2024-04-12T20:50:26.653 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:26.653+0000 7f414d400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4148102710 0x7f414810f030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T20:50:26.653 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:26.653+0000 7f414d400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4148102710 0x7f414810f030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:48636/0 (socket says 172.21.15.18:48636) 2024-04-12T20:50:26.653 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:26.653+0000 7f414d400700 1 -- 172.21.15.18:0/1466463808 learned_addr learned my addr 172.21.15.18:0/1466463808 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T20:50:26.653 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:26.653+0000 7f414de00700 1 --2- 172.21.15.18:0/1466463808 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4148101d70 0x7f414810eaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T20:50:26.653 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:26.653+0000 7f414d400700 1 -- 172.21.15.18:0/1466463808 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4148101d70 msgr2=0x7f414810eaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:50:26.653 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:26.653+0000 7f414d400700 1 --2- 172.21.15.18:0/1466463808 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4148101d70 0x7f414810eaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:50:26.654 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:26.653+0000 7f414d400700 1 -- 172.21.15.18:0/1466463808 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f413c005040 con 0x7f4148102710 2024-04-12T20:50:26.654 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:26.653+0000 7f414d400700 1 --2- 172.21.15.18:0/1466463808 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4148102710 0x7f414810f030 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f413800b790 tx=0x7f4138009220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T20:50:26.654 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:26.654+0000 7f4142000700 1 -- 172.21.15.18:0/1466463808 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f413800f6f0 con 0x7f4148102710 2024-04-12T20:50:26.654 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:26.654+0000 7f4150800700 1 -- 172.21.15.18:0/1466463808 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4148117bc0 con 0x7f4148102710 2024-04-12T20:50:26.654 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:26.654+0000 7f4142000700 1 -- 172.21.15.18:0/1466463808 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f413800fd20 con 0x7f4148102710 2024-04-12T20:50:26.655 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:26.654+0000 7f4142000700 1 -- 172.21.15.18:0/1466463808 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f41380103a0 con 0x7f4148102710 2024-04-12T20:50:26.655 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:26.654+0000 7f4150800700 1 -- 172.21.15.18:0/1466463808 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4148118150 con 0x7f4148102710 2024-04-12T20:50:26.655 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:26.655+0000 7f4150800700 1 -- 172.21.15.18:0/1466463808 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f41480ff080 con 0x7f4148102710 2024-04-12T20:50:26.655 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:26.656+0000 7f4142000700 1 -- 172.21.15.18:0/1466463808 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7f4138011400 con 0x7f4148102710 2024-04-12T20:50:26.656 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:26.657+0000 7f4142000700 1 --2- 172.21.15.18:0/1466463808 >> [v2:172.21.15.18:6800/3048894687,v1:172.21.15.18:6801/3048894687] conn(0x7f41300692b0 0x7f413006b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T20:50:26.656 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:26.657+0000 7f4142000700 1 -- 172.21.15.18:0/1466463808 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4634+0+0 (secure 0 0 0) 0x7f4138086d50 con 0x7f4148102710 2024-04-12T20:50:26.657 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:26.657+0000 7f414de00700 1 --2- 172.21.15.18:0/1466463808 >> [v2:172.21.15.18:6800/3048894687,v1:172.21.15.18:6801/3048894687] conn(0x7f41300692b0 0x7f413006b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T20:50:26.657 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:26.658+0000 7f414de00700 1 --2- 172.21.15.18:0/1466463808 >> [v2:172.21.15.18:6800/3048894687,v1:172.21.15.18:6801/3048894687] conn(0x7f41300692b0 0x7f413006b760 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f413c00b3b0 tx=0x7f413c009e30 comp rx=0 tx=0).ready entity=mgr.14212 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T20:50:26.660 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:26.662+0000 7f4142000700 1 -- 172.21.15.18:0/1466463808 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7f41380532a0 con 0x7f4148102710 2024-04-12T20:50:26.876 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:26.876+0000 7f4150800700 1 -- 172.21.15.18:0/1466463808 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f4148108ee0 con 0x7f4148102710 2024-04-12T20:50:26.877 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:26.877+0000 7f4142000700 1 -- 172.21.15.18:0/1466463808 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v40) v1 ==== 93+0+10 (secure 0 0 0) 0x7f41380568c0 con 0x7f4148102710 2024-04-12T20:50:26.877 INFO:teuthology.orchestra.run.smithi018.stdout:pg_num: 1 2024-04-12T20:50:26.879 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:26.880+0000 7f4150800700 1 -- 172.21.15.18:0/1466463808 >> [v2:172.21.15.18:6800/3048894687,v1:172.21.15.18:6801/3048894687] conn(0x7f41300692b0 msgr2=0x7f413006b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:50:26.879 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:26.880+0000 7f4150800700 1 --2- 172.21.15.18:0/1466463808 >> [v2:172.21.15.18:6800/3048894687,v1:172.21.15.18:6801/3048894687] conn(0x7f41300692b0 0x7f413006b760 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f413c00b3b0 tx=0x7f413c009e30 comp rx=0 tx=0).stop 2024-04-12T20:50:26.879 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:26.880+0000 7f4150800700 1 -- 172.21.15.18:0/1466463808 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4148102710 msgr2=0x7f414810f030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:50:26.880 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:26.880+0000 7f4150800700 1 --2- 172.21.15.18:0/1466463808 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4148102710 0x7f414810f030 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f413800b790 tx=0x7f4138009220 comp rx=0 tx=0).stop 2024-04-12T20:50:26.880 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:26.880+0000 7f4150800700 1 -- 172.21.15.18:0/1466463808 shutdown_connections 2024-04-12T20:50:26.880 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:26.880+0000 7f4150800700 1 --2- 172.21.15.18:0/1466463808 >> [v2:172.21.15.18:6800/3048894687,v1:172.21.15.18:6801/3048894687] conn(0x7f41300692b0 0x7f413006b760 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:50:26.880 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:26.880+0000 7f4150800700 1 --2- 172.21.15.18:0/1466463808 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4148101d70 0x7f414810eaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:50:26.880 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:26.880+0000 7f4150800700 1 --2- 172.21.15.18:0/1466463808 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4148102710 0x7f414810f030 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:50:26.880 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:26.880+0000 7f4150800700 1 -- 172.21.15.18:0/1466463808 >> 172.21.15.18:0/1466463808 conn(0x7f4148072bc0 msgr2=0x7f41480faed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T20:50:26.880 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:26.881+0000 7f4150800700 1 -- 172.21.15.18:0/1466463808 shutdown_connections 2024-04-12T20:50:26.881 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:26.881+0000 7f4150800700 1 -- 172.21.15.18:0/1466463808 wait complete. 2024-04-12T20:50:27.243 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:50:26 smithi018 ceph-mon[28799]: from='client.? 172.21.15.18:0/1466463808' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-12T20:50:27.281 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:50:26 smithi186 ceph-mon[36597]: from='client.? 172.21.15.18:0/1466463808' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-12T20:50:27.691 INFO:tasks.cephadm:Setting up client nodes... 2024-04-12T20:50:27.691 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1cbe407a-f90d-11ee-bc8d-c7b262605968 -- ceph auth get-or-create client.0 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-04-12T20:50:28.185 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:50:27 smithi186 ceph-mon[36597]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-12T20:50:28.202 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/mon.smithi018/config 2024-04-12T20:50:28.352 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:50:27 smithi018 ceph-mon[28799]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-12T20:50:30.023 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:50:29 smithi018 ceph-mon[28799]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-12T20:50:30.024 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:50:29 smithi018 ceph-mon[28799]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T20:50:30.031 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:50:29 smithi186 ceph-mon[36597]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-12T20:50:30.031 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:50:29 smithi186 ceph-mon[36597]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T20:50:30.202 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:30.202+0000 7f3726600700 1 -- 172.21.15.18:0/2032261711 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3720104520 msgr2=0x7f3720104990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:50:30.203 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:30.202+0000 7f3726600700 1 --2- 172.21.15.18:0/2032261711 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3720104520 0x7f3720104990 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f370c004940 tx=0x7f370c009e30 comp rx=0 tx=0).stop 2024-04-12T20:50:30.203 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:30.203+0000 7f3726600700 1 -- 172.21.15.18:0/2032261711 shutdown_connections 2024-04-12T20:50:30.203 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:30.203+0000 7f3726600700 1 --2- 172.21.15.18:0/2032261711 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3720104520 0x7f3720104990 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:50:30.203 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:30.203+0000 7f3726600700 1 --2- 172.21.15.18:0/2032261711 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3720103270 0x7f3720103680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:50:30.203 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:30.203+0000 7f3726600700 1 -- 172.21.15.18:0/2032261711 >> 172.21.15.18:0/2032261711 conn(0x7f3720072530 msgr2=0x7f3720074980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T20:50:30.203 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:30.203+0000 7f3726600700 1 -- 172.21.15.18:0/2032261711 shutdown_connections 2024-04-12T20:50:30.203 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:30.203+0000 7f3726600700 1 -- 172.21.15.18:0/2032261711 wait complete. 2024-04-12T20:50:30.204 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:30.204+0000 7f3726600700 1 Processor -- start 2024-04-12T20:50:30.204 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:30.204+0000 7f3726600700 1 -- start start 2024-04-12T20:50:30.204 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:30.205+0000 7f3726600700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3720103270 0x7f3720113f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T20:50:30.204 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:30.205+0000 7f3726600700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3720104520 0x7f3720114440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T20:50:30.204 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:30.205+0000 7f3726600700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3720114a40 con 0x7f3720103270 2024-04-12T20:50:30.205 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:30.205+0000 7f3726600700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3720115180 con 0x7f3720104520 2024-04-12T20:50:30.205 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:30.205+0000 7f3725200700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3720103270 0x7f3720113f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T20:50:30.205 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:30.205+0000 7f3725200700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3720103270 0x7f3720113f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:48658/0 (socket says 172.21.15.18:48658) 2024-04-12T20:50:30.205 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:30.205+0000 7f3725200700 1 -- 172.21.15.18:0/19879074 learned_addr learned my addr 172.21.15.18:0/19879074 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T20:50:30.206 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:30.205+0000 7f371fe00700 1 --2- 172.21.15.18:0/19879074 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3720104520 0x7f3720114440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T20:50:30.206 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:30.205+0000 7f3725200700 1 -- 172.21.15.18:0/19879074 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3720104520 msgr2=0x7f3720114440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:50:30.206 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:30.205+0000 7f3725200700 1 --2- 172.21.15.18:0/19879074 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3720104520 0x7f3720114440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:50:30.206 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:30.206+0000 7f3725200700 1 -- 172.21.15.18:0/19879074 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f370c005040 con 0x7f3720103270 2024-04-12T20:50:30.206 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:30.206+0000 7f3725200700 1 --2- 172.21.15.18:0/19879074 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3720103270 0x7f3720113f00 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f371400c830 tx=0x7f3714004470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T20:50:30.206 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:30.206+0000 7f371cc00700 1 -- 172.21.15.18:0/19879074 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f371400a760 con 0x7f3720103270 2024-04-12T20:50:30.206 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:30.206+0000 7f371cc00700 1 -- 172.21.15.18:0/19879074 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f371400ad90 con 0x7f3720103270 2024-04-12T20:50:30.206 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:30.206+0000 7f371cc00700 1 -- 172.21.15.18:0/19879074 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f37140113a0 con 0x7f3720103270 2024-04-12T20:50:30.206 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:30.206+0000 7f3726600700 1 -- 172.21.15.18:0/19879074 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3720115460 con 0x7f3720103270 2024-04-12T20:50:30.206 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:30.206+0000 7f3726600700 1 -- 172.21.15.18:0/19879074 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3720115a20 con 0x7f3720103270 2024-04-12T20:50:30.207 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:30.207+0000 7f3726600700 1 -- 172.21.15.18:0/19879074 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f37200760a0 con 0x7f3720103270 2024-04-12T20:50:30.207 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:30.208+0000 7f371cc00700 1 -- 172.21.15.18:0/19879074 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7f3714012400 con 0x7f3720103270 2024-04-12T20:50:30.209 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:30.209+0000 7f371cc00700 1 --2- 172.21.15.18:0/19879074 >> [v2:172.21.15.18:6800/3048894687,v1:172.21.15.18:6801/3048894687] conn(0x7f37080692a0 0x7f370806b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T20:50:30.209 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:30.210+0000 7f371cc00700 1 -- 172.21.15.18:0/19879074 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4634+0+0 (secure 0 0 0) 0x7f3714086ec0 con 0x7f3720103270 2024-04-12T20:50:30.209 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:30.210+0000 7f371fe00700 1 --2- 172.21.15.18:0/19879074 >> [v2:172.21.15.18:6800/3048894687,v1:172.21.15.18:6801/3048894687] conn(0x7f37080692a0 0x7f370806b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T20:50:30.209 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:30.210+0000 7f371fe00700 1 --2- 172.21.15.18:0/19879074 >> [v2:172.21.15.18:6800/3048894687,v1:172.21.15.18:6801/3048894687] conn(0x7f37080692a0 0x7f370806b750 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f370c004f90 tx=0x7f370c009e30 comp rx=0 tx=0).ready entity=mgr.14212 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T20:50:30.213 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:30.214+0000 7f371cc00700 1 -- 172.21.15.18:0/19879074 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7f3714056fa0 con 0x7f3720103270 2024-04-12T20:50:30.519 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:30.520+0000 7f3726600700 1 -- 172.21.15.18:0/19879074 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} v 0) v1 -- 0x7f372004bed0 con 0x7f3720103270 2024-04-12T20:50:30.525 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:30.525+0000 7f371cc00700 1 -- 172.21.15.18:0/19879074 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]=0 v14) v1 ==== 170+0+59 (secure 0 0 0) 0x7f3714056b30 con 0x7f3720103270 2024-04-12T20:50:30.525 INFO:teuthology.orchestra.run.smithi018.stdout:[client.0] 2024-04-12T20:50:30.525 INFO:teuthology.orchestra.run.smithi018.stdout: key = AQCWnhlmmNUiHxAAIv0zJzNYMlgePxjcKJeZaQ== 2024-04-12T20:50:30.528 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:30.528+0000 7f3726600700 1 -- 172.21.15.18:0/19879074 >> [v2:172.21.15.18:6800/3048894687,v1:172.21.15.18:6801/3048894687] conn(0x7f37080692a0 msgr2=0x7f370806b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:50:30.528 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:30.528+0000 7f3726600700 1 --2- 172.21.15.18:0/19879074 >> [v2:172.21.15.18:6800/3048894687,v1:172.21.15.18:6801/3048894687] conn(0x7f37080692a0 0x7f370806b750 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f370c004f90 tx=0x7f370c009e30 comp rx=0 tx=0).stop 2024-04-12T20:50:30.528 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:30.528+0000 7f3726600700 1 -- 172.21.15.18:0/19879074 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3720103270 msgr2=0x7f3720113f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:50:30.528 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:30.528+0000 7f3726600700 1 --2- 172.21.15.18:0/19879074 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3720103270 0x7f3720113f00 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f371400c830 tx=0x7f3714004470 comp rx=0 tx=0).stop 2024-04-12T20:50:30.528 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:30.528+0000 7f3726600700 1 -- 172.21.15.18:0/19879074 shutdown_connections 2024-04-12T20:50:30.528 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:30.528+0000 7f3726600700 1 --2- 172.21.15.18:0/19879074 >> [v2:172.21.15.18:6800/3048894687,v1:172.21.15.18:6801/3048894687] conn(0x7f37080692a0 0x7f370806b750 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:50:30.528 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:30.528+0000 7f3726600700 1 --2- 172.21.15.18:0/19879074 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3720103270 0x7f3720113f00 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:50:30.528 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:30.528+0000 7f3726600700 1 --2- 172.21.15.18:0/19879074 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3720104520 0x7f3720114440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:50:30.528 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:30.528+0000 7f3726600700 1 -- 172.21.15.18:0/19879074 >> 172.21.15.18:0/19879074 conn(0x7f3720072530 msgr2=0x7f3720074be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T20:50:30.528 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:30.529+0000 7f3726600700 1 -- 172.21.15.18:0/19879074 shutdown_connections 2024-04-12T20:50:30.529 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:30.529+0000 7f3726600700 1 -- 172.21.15.18:0/19879074 wait complete. 2024-04-12T20:50:31.282 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:50:30 smithi186 ceph-mon[36597]: from='client.? 172.21.15.18:0/19879074' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-04-12T20:50:31.282 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:50:30 smithi186 ceph-mon[36597]: from='client.? 172.21.15.18:0/19879074' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-04-12T20:50:31.335 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:50:30 smithi018 ceph-mon[28799]: from='client.? 172.21.15.18:0/19879074' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-04-12T20:50:31.335 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:50:30 smithi018 ceph-mon[28799]: from='client.? 172.21.15.18:0/19879074' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-04-12T20:50:31.428 DEBUG:teuthology.orchestra.run.smithi018:> set -ex 2024-04-12T20:50:31.428 DEBUG:teuthology.orchestra.run.smithi018:> sudo dd of=/etc/ceph/ceph.client.0.keyring 2024-04-12T20:50:31.428 DEBUG:teuthology.orchestra.run.smithi018:> sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-04-12T20:50:31.602 DEBUG:teuthology.orchestra.run.smithi186:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1cbe407a-f90d-11ee-bc8d-c7b262605968 -- ceph auth get-or-create client.1 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-04-12T20:50:32.087 INFO:teuthology.orchestra.run.smithi186.stderr:Inferring config /var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/mon.smithi186/config 2024-04-12T20:50:32.231 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:50:31 smithi186 ceph-mon[36597]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-12T20:50:32.353 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:50:31 smithi018 ceph-mon[28799]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-12T20:50:34.179 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:50:34.180+0000 7f3f1b600700 1 -- 172.21.15.186:0/631240936 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3f14071ea0 msgr2=0x7f3f140703b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:50:34.180 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:50:34.180+0000 7f3f1b600700 1 --2- 172.21.15.186:0/631240936 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3f14071ea0 0x7f3f140703b0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f3f08004940 tx=0x7f3f08009e30 comp rx=0 tx=0).stop 2024-04-12T20:50:34.180 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:50:34.181+0000 7f3f1b600700 1 -- 172.21.15.186:0/631240936 shutdown_connections 2024-04-12T20:50:34.180 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:50:34.181+0000 7f3f1b600700 1 --2- 172.21.15.186:0/631240936 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3f140708f0 0x7f3f14070d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:50:34.180 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:50:34.181+0000 7f3f1b600700 1 --2- 172.21.15.186:0/631240936 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3f14071ea0 0x7f3f140703b0 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:50:34.180 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:50:34.181+0000 7f3f1b600700 1 -- 172.21.15.186:0/631240936 >> 172.21.15.186:0/631240936 conn(0x7f3f140f8540 msgr2=0x7f3f140fa9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T20:50:34.180 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:50:34.181+0000 7f3f1b600700 1 -- 172.21.15.186:0/631240936 shutdown_connections 2024-04-12T20:50:34.180 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:50:34.181+0000 7f3f1b600700 1 -- 172.21.15.186:0/631240936 wait complete. 2024-04-12T20:50:34.181 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:50:34.182+0000 7f3f1b600700 1 Processor -- start 2024-04-12T20:50:34.181 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:50:34.182+0000 7f3f1b600700 1 -- start start 2024-04-12T20:50:34.182 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:50:34.182+0000 7f3f1b600700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3f140708f0 0x7f3f14113f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T20:50:34.182 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:50:34.182+0000 7f3f1b600700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3f14071ea0 0x7f3f14114480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T20:50:34.182 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:50:34.182+0000 7f3f1b600700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f14114a80 con 0x7f3f140708f0 2024-04-12T20:50:34.182 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:50:34.182+0000 7f3f1b600700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f14114bf0 con 0x7f3f14071ea0 2024-04-12T20:50:34.182 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:50:34.182+0000 7f3f13e00700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3f14071ea0 0x7f3f14114480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T20:50:34.182 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:50:34.183+0000 7f3f13e00700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3f14071ea0 0x7f3f14114480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.186:57606/0 (socket says 172.21.15.186:57606) 2024-04-12T20:50:34.182 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:50:34.183+0000 7f3f13e00700 1 -- 172.21.15.186:0/682392361 learned_addr learned my addr 172.21.15.186:0/682392361 (peer_addr_for_me v2:172.21.15.186:0/0) 2024-04-12T20:50:34.182 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:50:34.183+0000 7f3f18c00700 1 --2- 172.21.15.186:0/682392361 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3f140708f0 0x7f3f14113f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T20:50:34.182 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:50:34.183+0000 7f3f13e00700 1 -- 172.21.15.186:0/682392361 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3f140708f0 msgr2=0x7f3f14113f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:50:34.182 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:50:34.183+0000 7f3f13e00700 1 --2- 172.21.15.186:0/682392361 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3f140708f0 0x7f3f14113f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:50:34.182 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:50:34.183+0000 7f3f13e00700 1 -- 172.21.15.186:0/682392361 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f08005040 con 0x7f3f14071ea0 2024-04-12T20:50:34.182 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:50:34.183+0000 7f3f13e00700 1 --2- 172.21.15.186:0/682392361 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3f14071ea0 0x7f3f14114480 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f3f0000c8c0 tx=0x7f3f000044c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T20:50:34.183 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:50:34.184+0000 7f3f10c00700 1 -- 172.21.15.186:0/682392361 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3f0000a980 con 0x7f3f14071ea0 2024-04-12T20:50:34.183 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:50:34.184+0000 7f3f1b600700 1 -- 172.21.15.186:0/682392361 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3f14119650 con 0x7f3f14071ea0 2024-04-12T20:50:34.183 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:50:34.184+0000 7f3f10c00700 1 -- 172.21.15.186:0/682392361 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f3f0000ce60 con 0x7f3f14071ea0 2024-04-12T20:50:34.183 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:50:34.184+0000 7f3f10c00700 1 -- 172.21.15.186:0/682392361 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3f000115e0 con 0x7f3f14071ea0 2024-04-12T20:50:34.184 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:50:34.184+0000 7f3f1b600700 1 -- 172.21.15.186:0/682392361 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3f14119c10 con 0x7f3f14071ea0 2024-04-12T20:50:34.184 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:50:34.185+0000 7f3f1b600700 1 -- 172.21.15.186:0/682392361 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3f1404c710 con 0x7f3f14071ea0 2024-04-12T20:50:34.184 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:50:34.186+0000 7f3f10c00700 1 -- 172.21.15.186:0/682392361 <== mon.1 v2:172.21.15.186:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7f3f00012400 con 0x7f3f14071ea0 2024-04-12T20:50:34.185 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:50:34.187+0000 7f3f10c00700 1 --2- 172.21.15.186:0/682392361 >> [v2:172.21.15.18:6800/3048894687,v1:172.21.15.18:6801/3048894687] conn(0x7f3efc0692a0 0x7f3efc06b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T20:50:34.186 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:50:34.187+0000 7f3f10c00700 1 -- 172.21.15.186:0/682392361 <== mon.1 v2:172.21.15.186:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4634+0+0 (secure 0 0 0) 0x7f3f00086f10 con 0x7f3f14071ea0 2024-04-12T20:50:34.186 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:50:34.188+0000 7f3f18c00700 1 --2- 172.21.15.186:0/682392361 >> [v2:172.21.15.18:6800/3048894687,v1:172.21.15.18:6801/3048894687] conn(0x7f3efc0692a0 0x7f3efc06b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T20:50:34.187 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:50:34.189+0000 7f3f18c00700 1 --2- 172.21.15.186:0/682392361 >> [v2:172.21.15.18:6800/3048894687,v1:172.21.15.18:6801/3048894687] conn(0x7f3efc0692a0 0x7f3efc06b750 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f3f0800b970 tx=0x7f3f08009e30 comp rx=0 tx=0).ready entity=mgr.14212 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T20:50:34.190 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:50:34.192+0000 7f3f10c00700 1 -- 172.21.15.186:0/682392361 <== mon.1 v2:172.21.15.186:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7f3f00056ff0 con 0x7f3f14071ea0 2024-04-12T20:50:34.281 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:50:33 smithi186 ceph-mon[36597]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-12T20:50:34.353 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:50:33 smithi018 ceph-mon[28799]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-12T20:50:34.486 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:50:34.487+0000 7f3f1b600700 1 -- 172.21.15.186:0/682392361 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} v 0) v1 -- 0x7f3f1410bf20 con 0x7f3f14071ea0 2024-04-12T20:50:34.496 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:50:34.498+0000 7f3f10c00700 1 -- 172.21.15.186:0/682392361 <== mon.1 v2:172.21.15.186:3300/0 7 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]=0 v15) v1 ==== 170+0+59 (secure 0 0 0) 0x7f3f000126b0 con 0x7f3f14071ea0 2024-04-12T20:50:34.497 INFO:teuthology.orchestra.run.smithi186.stdout:[client.1] 2024-04-12T20:50:34.497 INFO:teuthology.orchestra.run.smithi186.stdout: key = AQCanhlmxZc1HRAAZv7+7H1wZbthiY1QSEVFWw== 2024-04-12T20:50:34.500 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:50:34.501+0000 7f3f1b600700 1 -- 172.21.15.186:0/682392361 >> [v2:172.21.15.18:6800/3048894687,v1:172.21.15.18:6801/3048894687] conn(0x7f3efc0692a0 msgr2=0x7f3efc06b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:50:34.500 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:50:34.501+0000 7f3f1b600700 1 --2- 172.21.15.186:0/682392361 >> [v2:172.21.15.18:6800/3048894687,v1:172.21.15.18:6801/3048894687] conn(0x7f3efc0692a0 0x7f3efc06b750 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f3f0800b970 tx=0x7f3f08009e30 comp rx=0 tx=0).stop 2024-04-12T20:50:34.500 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:50:34.501+0000 7f3f1b600700 1 -- 172.21.15.186:0/682392361 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3f14071ea0 msgr2=0x7f3f14114480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:50:34.500 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:50:34.501+0000 7f3f1b600700 1 --2- 172.21.15.186:0/682392361 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3f14071ea0 0x7f3f14114480 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f3f0000c8c0 tx=0x7f3f000044c0 comp rx=0 tx=0).stop 2024-04-12T20:50:34.500 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:50:34.501+0000 7f3f1b600700 1 -- 172.21.15.186:0/682392361 shutdown_connections 2024-04-12T20:50:34.500 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:50:34.501+0000 7f3f1b600700 1 --2- 172.21.15.186:0/682392361 >> [v2:172.21.15.18:6800/3048894687,v1:172.21.15.18:6801/3048894687] conn(0x7f3efc0692a0 0x7f3efc06b750 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:50:34.500 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:50:34.501+0000 7f3f1b600700 1 --2- 172.21.15.186:0/682392361 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3f140708f0 0x7f3f14113f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:50:34.500 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:50:34.501+0000 7f3f1b600700 1 --2- 172.21.15.186:0/682392361 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3f14071ea0 0x7f3f14114480 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:50:34.500 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:50:34.501+0000 7f3f1b600700 1 -- 172.21.15.186:0/682392361 >> 172.21.15.186:0/682392361 conn(0x7f3f140f8540 msgr2=0x7f3f140f9c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T20:50:34.500 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:50:34.501+0000 7f3f1b600700 1 -- 172.21.15.186:0/682392361 shutdown_connections 2024-04-12T20:50:34.500 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:50:34.501+0000 7f3f1b600700 1 -- 172.21.15.186:0/682392361 wait complete. 2024-04-12T20:50:35.270 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:50:34 smithi186 ceph-mon[36597]: from='client.? 172.21.15.186:0/682392361' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-04-12T20:50:35.271 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:50:34 smithi186 ceph-mon[36597]: from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-04-12T20:50:35.271 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:50:34 smithi186 ceph-mon[36597]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-04-12T20:50:35.318 DEBUG:teuthology.orchestra.run.smithi186:> set -ex 2024-04-12T20:50:35.318 DEBUG:teuthology.orchestra.run.smithi186:> sudo dd of=/etc/ceph/ceph.client.1.keyring 2024-04-12T20:50:35.318 DEBUG:teuthology.orchestra.run.smithi186:> sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2024-04-12T20:50:35.353 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:50:34 smithi018 ceph-mon[28799]: from='client.? 172.21.15.186:0/682392361' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-04-12T20:50:35.353 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:50:34 smithi018 ceph-mon[28799]: from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-04-12T20:50:35.353 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:50:34 smithi018 ceph-mon[28799]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-04-12T20:50:35.467 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-04-12T20:50:35.467 INFO:tasks.cephadm.ceph_manager.ceph:waiting for mgr available 2024-04-12T20:50:35.468 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid 1cbe407a-f90d-11ee-bc8d-c7b262605968 -- ceph mgr dump --format=json 2024-04-12T20:50:35.981 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/mon.smithi018/config 2024-04-12T20:50:36.281 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:50:35 smithi186 ceph-mon[36597]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-12T20:50:36.319 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:50:35 smithi018 ceph-mon[28799]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-12T20:50:38.103 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:50:37 smithi018 ceph-mon[28799]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-12T20:50:38.281 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:50:37 smithi186 ceph-mon[36597]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-12T20:50:38.369 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:38.369+0000 7f4e0a600700 1 -- 172.21.15.18:0/2554124540 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4e04105800 msgr2=0x7f4e040fee70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:50:38.370 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:38.369+0000 7f4e0a600700 1 --2- 172.21.15.18:0/2554124540 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4e04105800 0x7f4e040fee70 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f4df0004940 tx=0x7f4df0009e30 comp rx=0 tx=0).stop 2024-04-12T20:50:38.370 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:38.369+0000 7f4e0a600700 1 -- 172.21.15.18:0/2554124540 shutdown_connections 2024-04-12T20:50:38.370 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:38.369+0000 7f4e0a600700 1 --2- 172.21.15.18:0/2554124540 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4e04105800 0x7f4e040fee70 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:50:38.370 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:38.369+0000 7f4e0a600700 1 --2- 172.21.15.18:0/2554124540 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4e04104e60 0x7f4e04105230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:50:38.370 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:38.369+0000 7f4e0a600700 1 -- 172.21.15.18:0/2554124540 >> 172.21.15.18:0/2554124540 conn(0x7f4e040fab10 msgr2=0x7f4e040fcef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T20:50:38.370 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:38.369+0000 7f4e0a600700 1 -- 172.21.15.18:0/2554124540 shutdown_connections 2024-04-12T20:50:38.370 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:38.369+0000 7f4e0a600700 1 -- 172.21.15.18:0/2554124540 wait complete. 2024-04-12T20:50:38.370 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:38.370+0000 7f4e0a600700 1 Processor -- start 2024-04-12T20:50:38.370 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:38.370+0000 7f4e0a600700 1 -- start start 2024-04-12T20:50:38.372 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:38.371+0000 7f4e0a600700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4e04104e60 0x7f4e04115c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T20:50:38.372 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:38.371+0000 7f4e0a600700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4e04105800 0x7f4e04116170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T20:50:38.372 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:38.371+0000 7f4e0a600700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e04116850 con 0x7f4e04104e60 2024-04-12T20:50:38.372 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:38.371+0000 7f4e0a600700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e0410fb90 con 0x7f4e04105800 2024-04-12T20:50:38.372 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:38.371+0000 7f4e09200700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4e04104e60 0x7f4e04115c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T20:50:38.372 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:38.371+0000 7f4e09200700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4e04104e60 0x7f4e04115c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:43220/0 (socket says 172.21.15.18:43220) 2024-04-12T20:50:38.372 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:38.371+0000 7f4e09200700 1 -- 172.21.15.18:0/991558458 learned_addr learned my addr 172.21.15.18:0/991558458 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T20:50:38.372 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:38.371+0000 7f4e03e00700 1 --2- 172.21.15.18:0/991558458 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4e04105800 0x7f4e04116170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T20:50:38.372 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:38.371+0000 7f4e09200700 1 -- 172.21.15.18:0/991558458 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4e04105800 msgr2=0x7f4e04116170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:50:38.372 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:38.371+0000 7f4e09200700 1 --2- 172.21.15.18:0/991558458 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4e04105800 0x7f4e04116170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:50:38.373 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:38.371+0000 7f4e09200700 1 -- 172.21.15.18:0/991558458 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4df0005040 con 0x7f4e04104e60 2024-04-12T20:50:38.373 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:38.372+0000 7f4e09200700 1 --2- 172.21.15.18:0/991558458 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4e04104e60 0x7f4e04115c30 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f4df800b790 tx=0x7f4df8009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T20:50:38.373 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:38.372+0000 7f4e00c00700 1 -- 172.21.15.18:0/991558458 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4df800f770 con 0x7f4e04104e60 2024-04-12T20:50:38.373 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:38.372+0000 7f4e00c00700 1 -- 172.21.15.18:0/991558458 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f4df800fda0 con 0x7f4e04104e60 2024-04-12T20:50:38.373 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:38.372+0000 7f4e00c00700 1 -- 172.21.15.18:0/991558458 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4df800d310 con 0x7f4e04104e60 2024-04-12T20:50:38.373 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:38.372+0000 7f4e0a600700 1 -- 172.21.15.18:0/991558458 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4e0410fe10 con 0x7f4e04104e60 2024-04-12T20:50:38.373 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:38.372+0000 7f4e0a600700 1 -- 172.21.15.18:0/991558458 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4e041103d0 con 0x7f4e04104e60 2024-04-12T20:50:38.373 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:38.373+0000 7f4e0a600700 1 -- 172.21.15.18:0/991558458 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4e041005b0 con 0x7f4e04104e60 2024-04-12T20:50:38.374 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:38.374+0000 7f4e00c00700 1 -- 172.21.15.18:0/991558458 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7f4df8010420 con 0x7f4e04104e60 2024-04-12T20:50:38.374 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:38.375+0000 7f4e00c00700 1 --2- 172.21.15.18:0/991558458 >> [v2:172.21.15.18:6800/3048894687,v1:172.21.15.18:6801/3048894687] conn(0x7f4dec0692a0 0x7f4dec06b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T20:50:38.375 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:38.375+0000 7f4e00c00700 1 -- 172.21.15.18:0/991558458 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4634+0+0 (secure 0 0 0) 0x7f4df8086e20 con 0x7f4e04104e60 2024-04-12T20:50:38.375 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:38.376+0000 7f4e03e00700 1 --2- 172.21.15.18:0/991558458 >> [v2:172.21.15.18:6800/3048894687,v1:172.21.15.18:6801/3048894687] conn(0x7f4dec0692a0 0x7f4dec06b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T20:50:38.375 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:38.376+0000 7f4e03e00700 1 --2- 172.21.15.18:0/991558458 >> [v2:172.21.15.18:6800/3048894687,v1:172.21.15.18:6801/3048894687] conn(0x7f4dec0692a0 0x7f4dec06b750 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f4df0000c00 tx=0x7f4df0009e30 comp rx=0 tx=0).ready entity=mgr.14212 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T20:50:38.379 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:38.380+0000 7f4e00c00700 1 -- 172.21.15.18:0/991558458 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7f4df8056f00 con 0x7f4e04104e60 2024-04-12T20:50:38.651 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:38.651+0000 7f4e0a600700 1 -- 172.21.15.18:0/991558458 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) v1 -- 0x7f4e0405f5f0 con 0x7f4e04104e60 2024-04-12T20:50:38.657 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:38.658+0000 7f4e00c00700 1 -- 172.21.15.18:0/991558458 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v19) v1 ==== 74+0+167168 (secure 0 0 0) 0x7f4df8056a90 con 0x7f4e04104e60 2024-04-12T20:50:38.659 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-12T20:50:38.663 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:38.662+0000 7f4e0a600700 1 -- 172.21.15.18:0/991558458 >> [v2:172.21.15.18:6800/3048894687,v1:172.21.15.18:6801/3048894687] conn(0x7f4dec0692a0 msgr2=0x7f4dec06b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:50:38.663 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:38.662+0000 7f4e0a600700 1 --2- 172.21.15.18:0/991558458 >> [v2:172.21.15.18:6800/3048894687,v1:172.21.15.18:6801/3048894687] conn(0x7f4dec0692a0 0x7f4dec06b750 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f4df0000c00 tx=0x7f4df0009e30 comp rx=0 tx=0).stop 2024-04-12T20:50:38.663 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:38.662+0000 7f4e0a600700 1 -- 172.21.15.18:0/991558458 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4e04104e60 msgr2=0x7f4e04115c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:50:38.663 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:38.662+0000 7f4e0a600700 1 --2- 172.21.15.18:0/991558458 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4e04104e60 0x7f4e04115c30 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f4df800b790 tx=0x7f4df8009250 comp rx=0 tx=0).stop 2024-04-12T20:50:38.664 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:38.662+0000 7f4e0a600700 1 -- 172.21.15.18:0/991558458 shutdown_connections 2024-04-12T20:50:38.664 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:38.662+0000 7f4e0a600700 1 --2- 172.21.15.18:0/991558458 >> [v2:172.21.15.18:6800/3048894687,v1:172.21.15.18:6801/3048894687] conn(0x7f4dec0692a0 0x7f4dec06b750 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:50:38.664 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:38.662+0000 7f4e0a600700 1 --2- 172.21.15.18:0/991558458 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4e04104e60 0x7f4e04115c30 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:50:38.664 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:38.662+0000 7f4e0a600700 1 --2- 172.21.15.18:0/991558458 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4e04105800 0x7f4e04116170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:50:38.664 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:38.662+0000 7f4e0a600700 1 -- 172.21.15.18:0/991558458 >> 172.21.15.18:0/991558458 conn(0x7f4e040fab10 msgr2=0x7f4e040fcae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T20:50:38.664 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:38.663+0000 7f4e0a600700 1 -- 172.21.15.18:0/991558458 shutdown_connections 2024-04-12T20:50:38.664 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:38.663+0000 7f4e0a600700 1 -- 172.21.15.18:0/991558458 wait complete. 2024-04-12T20:50:39.251 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:50:38 smithi186 ceph-mon[36597]: from='client.? 172.21.15.18:0/991558458' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-04-12T20:50:39.353 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:50:38 smithi018 ceph-mon[28799]: from='client.? 172.21.15.18:0/991558458' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-04-12T20:50:39.533 INFO:teuthology.orchestra.run.smithi018.stdout:{"epoch":19,"active_gid":14212,"active_name":"smithi018.xjzwga","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6800","nonce":3048894687},{"type":"v1","addr":"172.21.15.18:6801","nonce":3048894687}]},"active_addr":"172.21.15.18:6801/3048894687","active_change":"2024-04-12T20:45:44.776342+0000","active_mgr_features":4540138320759226367,"available":true,"standbys":[{"gid":14234,"name":"smithi186.kqcfxk","mgr_features":4540138320759226367,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2400","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"7","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.1.5","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["cephadm","dashboard","iostat","nfs","prometheus","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2400","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"7","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.1.5","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{"dashboard":"https://172.21.15.18:8443/","prometheus":"http://172.21.15.18:9283/"},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":5,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.18:0","nonce":2956836690}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.18:0","nonce":714775767}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.18:0","nonce":3227425848}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.18:0","nonce":2330205732}]}]} 2024-04-12T20:50:39.537 INFO:tasks.cephadm.ceph_manager.ceph:mgr available! 2024-04-12T20:50:39.537 INFO:tasks.cephadm.ceph_manager.ceph:waiting for all up 2024-04-12T20:50:39.537 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid 1cbe407a-f90d-11ee-bc8d-c7b262605968 -- ceph osd dump --format=json 2024-04-12T20:50:40.039 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/mon.smithi018/config 2024-04-12T20:50:40.281 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:50:39 smithi186 ceph-mon[36597]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-12T20:50:40.294 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:50:39 smithi018 ceph-mon[28799]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-12T20:50:42.242 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:42.242+0000 7fd896400700 1 -- 172.21.15.18:0/1335918828 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd8900fe830 msgr2=0x7fd890106f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:50:42.242 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:42.242+0000 7fd896400700 1 --2- 172.21.15.18:0/1335918828 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd8900fe830 0x7fd890106f00 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fd87c004940 tx=0x7fd87c009e30 comp rx=0 tx=0).stop 2024-04-12T20:50:42.242 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:50:41 smithi018 ceph-mon[28799]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-12T20:50:42.243 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:42.243+0000 7fd896400700 1 -- 172.21.15.18:0/1335918828 shutdown_connections 2024-04-12T20:50:42.243 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:42.243+0000 7fd896400700 1 --2- 172.21.15.18:0/1335918828 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd8900fe830 0x7fd890106f00 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:50:42.243 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:42.243+0000 7fd896400700 1 --2- 172.21.15.18:0/1335918828 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd8900fde90 0x7fd8900fe260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:50:42.243 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:42.243+0000 7fd896400700 1 -- 172.21.15.18:0/1335918828 >> 172.21.15.18:0/1335918828 conn(0x7fd890072bc0 msgr2=0x7fd890072fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T20:50:42.243 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:42.243+0000 7fd896400700 1 -- 172.21.15.18:0/1335918828 shutdown_connections 2024-04-12T20:50:42.243 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:42.243+0000 7fd896400700 1 -- 172.21.15.18:0/1335918828 wait complete. 2024-04-12T20:50:42.244 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:42.244+0000 7fd896400700 1 Processor -- start 2024-04-12T20:50:42.244 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:42.244+0000 7fd896400700 1 -- start start 2024-04-12T20:50:42.245 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:42.245+0000 7fd896400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd8900fde90 0x7fd890113ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T20:50:42.245 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:42.245+0000 7fd896400700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd8900fe830 0x7fd8901140e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T20:50:42.245 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:42.245+0000 7fd896400700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd890114730 con 0x7fd8900fde90 2024-04-12T20:50:42.245 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:42.245+0000 7fd896400700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd89010db20 con 0x7fd8900fe830 2024-04-12T20:50:42.245 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:42.245+0000 7fd88f400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd8900fde90 0x7fd890113ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T20:50:42.245 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:42.245+0000 7fd88f400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd8900fde90 0x7fd890113ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:44140/0 (socket says 172.21.15.18:44140) 2024-04-12T20:50:42.245 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:42.245+0000 7fd88f400700 1 -- 172.21.15.18:0/1983671979 learned_addr learned my addr 172.21.15.18:0/1983671979 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T20:50:42.245 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:42.245+0000 7fd88ea00700 1 --2- 172.21.15.18:0/1983671979 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd8900fe830 0x7fd8901140e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T20:50:42.245 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:42.245+0000 7fd88f400700 1 -- 172.21.15.18:0/1983671979 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd8900fe830 msgr2=0x7fd8901140e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:50:42.245 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:42.245+0000 7fd88f400700 1 --2- 172.21.15.18:0/1983671979 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd8900fe830 0x7fd8901140e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:50:42.246 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:42.245+0000 7fd88f400700 1 -- 172.21.15.18:0/1983671979 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd87c005040 con 0x7fd8900fde90 2024-04-12T20:50:42.246 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:42.245+0000 7fd88ea00700 1 --2- 172.21.15.18:0/1983671979 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd8900fe830 0x7fd8901140e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-12T20:50:42.246 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:42.246+0000 7fd88f400700 1 --2- 172.21.15.18:0/1983671979 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd8900fde90 0x7fd890113ba0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fd880002a50 tx=0x7fd8800046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T20:50:42.246 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:42.246+0000 7fd887400700 1 -- 172.21.15.18:0/1983671979 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd880007cc0 con 0x7fd8900fde90 2024-04-12T20:50:42.246 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:42.246+0000 7fd887400700 1 -- 172.21.15.18:0/1983671979 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd88000f040 con 0x7fd8900fde90 2024-04-12T20:50:42.246 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:42.246+0000 7fd887400700 1 -- 172.21.15.18:0/1983671979 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd880009640 con 0x7fd8900fde90 2024-04-12T20:50:42.246 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:42.246+0000 7fd896400700 1 -- 172.21.15.18:0/1983671979 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd89010de00 con 0x7fd8900fde90 2024-04-12T20:50:42.246 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:42.246+0000 7fd896400700 1 -- 172.21.15.18:0/1983671979 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd89010e340 con 0x7fd8900fde90 2024-04-12T20:50:42.247 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:42.247+0000 7fd896400700 1 -- 172.21.15.18:0/1983671979 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd890065e30 con 0x7fd8900fde90 2024-04-12T20:50:42.247 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:42.248+0000 7fd887400700 1 -- 172.21.15.18:0/1983671979 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7fd8800097a0 con 0x7fd8900fde90 2024-04-12T20:50:42.248 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:42.249+0000 7fd887400700 1 --2- 172.21.15.18:0/1983671979 >> [v2:172.21.15.18:6800/3048894687,v1:172.21.15.18:6801/3048894687] conn(0x7fd8740692b0 0x7fd87406b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T20:50:42.248 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:42.249+0000 7fd887400700 1 -- 172.21.15.18:0/1983671979 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4634+0+0 (secure 0 0 0) 0x7fd880086c10 con 0x7fd8900fde90 2024-04-12T20:50:42.249 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:42.249+0000 7fd88ea00700 1 --2- 172.21.15.18:0/1983671979 >> [v2:172.21.15.18:6800/3048894687,v1:172.21.15.18:6801/3048894687] conn(0x7fd8740692b0 0x7fd87406b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T20:50:42.249 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:42.250+0000 7fd88ea00700 1 --2- 172.21.15.18:0/1983671979 >> [v2:172.21.15.18:6800/3048894687,v1:172.21.15.18:6801/3048894687] conn(0x7fd8740692b0 0x7fd87406b760 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fd87c00b3b0 tx=0x7fd87c009e30 comp rx=0 tx=0).ready entity=mgr.14212 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T20:50:42.253 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:42.254+0000 7fd887400700 1 -- 172.21.15.18:0/1983671979 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7fd880056c70 con 0x7fd8900fde90 2024-04-12T20:50:42.281 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:50:41 smithi186 ceph-mon[36597]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-12T20:50:42.469 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:42.470+0000 7fd896400700 1 -- 172.21.15.18:0/1983671979 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd890109f70 con 0x7fd8900fde90 2024-04-12T20:50:42.471 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:42.472+0000 7fd887400700 1 -- 172.21.15.18:0/1983671979 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v40) v1 ==== 74+0+11072 (secure 0 0 0) 0x7fd880056800 con 0x7fd8900fde90 2024-04-12T20:50:42.472 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-12T20:50:42.472 INFO:teuthology.orchestra.run.smithi018.stdout:{"epoch":40,"fsid":"1cbe407a-f90d-11ee-bc8d-c7b262605968","created":"2024-04-12T20:43:00.692361+0000","modified":"2024-04-12T20:50:18.227750+0000","last_up_change":"2024-04-12T20:50:17.011011+0000","last_in_change":"2024-04-12T20:49:54.190644+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":14,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-12T20:48:37.529081+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}}],"osds":[{"osd":0,"uuid":"2af887cc-0447-42aa-a8f3-1aa7fcd4dc8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6802","nonce":585613146},{"type":"v1","addr":"172.21.15.18:6803","nonce":585613146}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6804","nonce":585613146},{"type":"v1","addr":"172.21.15.18:6805","nonce":585613146}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6808","nonce":585613146},{"type":"v1","addr":"172.21.15.18:6809","nonce":585613146}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6806","nonce":585613146},{"type":"v1","addr":"172.21.15.18:6807","nonce":585613146}]},"public_addr":"172.21.15.18:6803/585613146","cluster_addr":"172.21.15.18:6805/585613146","heartbeat_back_addr":"172.21.15.18:6809/585613146","heartbeat_front_addr":"172.21.15.18:6807/585613146","state":["exists","up"]},{"osd":1,"uuid":"6ba7b0d3-a03d-45cc-b4e9-89a92c8c6ca8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6810","nonce":181951035},{"type":"v1","addr":"172.21.15.18:6811","nonce":181951035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6812","nonce":181951035},{"type":"v1","addr":"172.21.15.18:6813","nonce":181951035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6816","nonce":181951035},{"type":"v1","addr":"172.21.15.18:6817","nonce":181951035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6814","nonce":181951035},{"type":"v1","addr":"172.21.15.18:6815","nonce":181951035}]},"public_addr":"172.21.15.18:6811/181951035","cluster_addr":"172.21.15.18:6813/181951035","heartbeat_back_addr":"172.21.15.18:6817/181951035","heartbeat_front_addr":"172.21.15.18:6815/181951035","state":["exists","up"]},{"osd":2,"uuid":"5bb1137d-5a3c-40bc-ae08-fc792b8ee547","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6818","nonce":3573247809},{"type":"v1","addr":"172.21.15.18:6819","nonce":3573247809}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6820","nonce":3573247809},{"type":"v1","addr":"172.21.15.18:6821","nonce":3573247809}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6824","nonce":3573247809},{"type":"v1","addr":"172.21.15.18:6825","nonce":3573247809}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6822","nonce":3573247809},{"type":"v1","addr":"172.21.15.18:6823","nonce":3573247809}]},"public_addr":"172.21.15.18:6819/3573247809","cluster_addr":"172.21.15.18:6821/3573247809","heartbeat_back_addr":"172.21.15.18:6825/3573247809","heartbeat_front_addr":"172.21.15.18:6823/3573247809","state":["exists","up"]},{"osd":3,"uuid":"a5140a33-1e90-4980-aa9a-b0afe6b155f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6800","nonce":2184799172},{"type":"v1","addr":"172.21.15.186:6801","nonce":2184799172}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6802","nonce":2184799172},{"type":"v1","addr":"172.21.15.186:6803","nonce":2184799172}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6806","nonce":2184799172},{"type":"v1","addr":"172.21.15.186:6807","nonce":2184799172}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6804","nonce":2184799172},{"type":"v1","addr":"172.21.15.186:6805","nonce":2184799172}]},"public_addr":"172.21.15.186:6801/2184799172","cluster_addr":"172.21.15.186:6803/2184799172","heartbeat_back_addr":"172.21.15.186:6807/2184799172","heartbeat_front_addr":"172.21.15.186:6805/2184799172","state":["exists","up"]},{"osd":4,"uuid":"d6b2c6b3-6c78-4261-bcbe-c33358407198","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6808","nonce":3584843287},{"type":"v1","addr":"172.21.15.186:6809","nonce":3584843287}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6810","nonce":3584843287},{"type":"v1","addr":"172.21.15.186:6811","nonce":3584843287}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6814","nonce":3584843287},{"type":"v1","addr":"172.21.15.186:6815","nonce":3584843287}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6812","nonce":3584843287},{"type":"v1","addr":"172.21.15.186:6813","nonce":3584843287}]},"public_addr":"172.21.15.186:6809/3584843287","cluster_addr":"172.21.15.186:6811/3584843287","heartbeat_back_addr":"172.21.15.186:6815/3584843287","heartbeat_front_addr":"172.21.15.186:6813/3584843287","state":["exists","up"]},{"osd":5,"uuid":"7b5c9f17-6306-4fb5-aa77-fc9a73bcc663","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6816","nonce":3543055150},{"type":"v1","addr":"172.21.15.186:6817","nonce":3543055150}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6818","nonce":3543055150},{"type":"v1","addr":"172.21.15.186:6819","nonce":3543055150}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6822","nonce":3543055150},{"type":"v1","addr":"172.21.15.186:6823","nonce":3543055150}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6820","nonce":3543055150},{"type":"v1","addr":"172.21.15.186:6821","nonce":3543055150}]},"public_addr":"172.21.15.186:6817/3543055150","cluster_addr":"172.21.15.186:6819/3543055150","heartbeat_back_addr":"172.21.15.186:6823/3543055150","heartbeat_front_addr":"172.21.15.186:6821/3543055150","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-04-12T20:47:21.231060+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-04-12T20:47:57.699205+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-04-12T20:48:35.096648+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-04-12T20:49:08.252810+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-04-12T20:49:41.102187+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-04-12T20:50:16.009813+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.18:0/1720919549":"2024-04-13T20:45:44.776072+0000","172.21.15.18:0/3896210349":"2024-04-13T20:44:15.143139+0000","172.21.15.18:6801/1391635591":"2024-04-13T20:45:44.776072+0000","172.21.15.18:6800/1391635591":"2024-04-13T20:45:44.776072+0000","172.21.15.18:0/3387911038":"2024-04-13T20:44:15.143139+0000","172.21.15.18:0/4018777476":"2024-04-13T20:45:44.776072+0000","172.21.15.18:6801/3403971131":"2024-04-13T20:44:15.143139+0000","172.21.15.18:0/2945432483":"2024-04-13T20:44:15.143139+0000","172.21.15.18:0/1732042827":"2024-04-13T20:43:34.269559+0000","172.21.15.18:0/766256564":"2024-04-13T20:45:44.776072+0000","172.21.15.18:0/2517252399":"2024-04-13T20:43:34.269559+0000","172.21.15.18:0/3295870133":"2024-04-13T20:43:34.269559+0000","172.21.15.18:6800/3403971131":"2024-04-13T20:44:15.143139+0000","172.21.15.18:6800/1312802974":"2024-04-13T20:43:34.269559+0000","172.21.15.18:6801/1312802974":"2024-04-13T20:43:34.269559+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-12T20:50:42.474 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:42.475+0000 7fd896400700 1 -- 172.21.15.18:0/1983671979 >> [v2:172.21.15.18:6800/3048894687,v1:172.21.15.18:6801/3048894687] conn(0x7fd8740692b0 msgr2=0x7fd87406b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:50:42.474 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:42.475+0000 7fd896400700 1 --2- 172.21.15.18:0/1983671979 >> [v2:172.21.15.18:6800/3048894687,v1:172.21.15.18:6801/3048894687] conn(0x7fd8740692b0 0x7fd87406b760 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fd87c00b3b0 tx=0x7fd87c009e30 comp rx=0 tx=0).stop 2024-04-12T20:50:42.474 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:42.475+0000 7fd896400700 1 -- 172.21.15.18:0/1983671979 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd8900fde90 msgr2=0x7fd890113ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:50:42.474 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:42.475+0000 7fd896400700 1 --2- 172.21.15.18:0/1983671979 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd8900fde90 0x7fd890113ba0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fd880002a50 tx=0x7fd8800046b0 comp rx=0 tx=0).stop 2024-04-12T20:50:42.474 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:42.475+0000 7fd896400700 1 -- 172.21.15.18:0/1983671979 shutdown_connections 2024-04-12T20:50:42.475 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:42.475+0000 7fd896400700 1 --2- 172.21.15.18:0/1983671979 >> [v2:172.21.15.18:6800/3048894687,v1:172.21.15.18:6801/3048894687] conn(0x7fd8740692b0 0x7fd87406b760 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:50:42.475 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:42.475+0000 7fd896400700 1 --2- 172.21.15.18:0/1983671979 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd8900fde90 0x7fd890113ba0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:50:42.475 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:42.475+0000 7fd896400700 1 --2- 172.21.15.18:0/1983671979 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd8900fe830 0x7fd8901140e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:50:42.475 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:42.475+0000 7fd896400700 1 -- 172.21.15.18:0/1983671979 >> 172.21.15.18:0/1983671979 conn(0x7fd890072bc0 msgr2=0x7fd8901037b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T20:50:42.475 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:42.475+0000 7fd896400700 1 -- 172.21.15.18:0/1983671979 shutdown_connections 2024-04-12T20:50:42.475 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:42.475+0000 7fd896400700 1 -- 172.21.15.18:0/1983671979 wait complete. 2024-04-12T20:50:43.279 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:50:42 smithi018 ceph-mon[28799]: from='client.? 172.21.15.18:0/1983671979' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-12T20:50:43.281 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:50:42 smithi186 ceph-mon[36597]: from='client.? 172.21.15.18:0/1983671979' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-12T20:50:43.363 INFO:tasks.cephadm.ceph_manager.ceph:all up! 2024-04-12T20:50:43.363 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid 1cbe407a-f90d-11ee-bc8d-c7b262605968 -- ceph osd dump --format=json 2024-04-12T20:50:43.868 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/mon.smithi018/config 2024-04-12T20:50:44.277 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:50:43 smithi018 ceph-mon[28799]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-12T20:50:44.281 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:50:43 smithi186 ceph-mon[36597]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-12T20:50:45.235 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:50:45 smithi018 ceph-mon[28799]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T20:50:45.281 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:50:45 smithi186 ceph-mon[36597]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T20:50:46.095 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:46.094+0000 7f6410c00700 1 -- 172.21.15.18:0/4034174653 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f640c102d80 msgr2=0x7f640c103150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:50:46.095 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:46.094+0000 7f6410c00700 1 --2- 172.21.15.18:0/4034174653 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f640c102d80 0x7f640c103150 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f63fc009ce0 tx=0x7f63fc004d10 comp rx=0 tx=0).stop 2024-04-12T20:50:46.095 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:46.095+0000 7f6410c00700 1 -- 172.21.15.18:0/4034174653 shutdown_connections 2024-04-12T20:50:46.095 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:46.095+0000 7f6410c00700 1 --2- 172.21.15.18:0/4034174653 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f640c103720 0x7f640c0fcd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:50:46.095 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:46.095+0000 7f6410c00700 1 --2- 172.21.15.18:0/4034174653 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f640c102d80 0x7f640c103150 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:50:46.095 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:46.095+0000 7f6410c00700 1 -- 172.21.15.18:0/4034174653 >> 172.21.15.18:0/4034174653 conn(0x7f640c0f8930 msgr2=0x7f640c0fad40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T20:50:46.095 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:46.095+0000 7f6410c00700 1 -- 172.21.15.18:0/4034174653 shutdown_connections 2024-04-12T20:50:46.095 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:46.095+0000 7f6410c00700 1 -- 172.21.15.18:0/4034174653 wait complete. 2024-04-12T20:50:46.096 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:46.096+0000 7f6410c00700 1 Processor -- start 2024-04-12T20:50:46.096 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:46.096+0000 7f6410c00700 1 -- start start 2024-04-12T20:50:46.097 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:46.097+0000 7f6410c00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f640c102d80 0x7f640c0702c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T20:50:46.097 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:46.097+0000 7f6410c00700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f640c103720 0x7f640c070800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T20:50:46.097 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:46.097+0000 7f6410c00700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f640c070dd0 con 0x7f640c102d80 2024-04-12T20:50:46.097 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:46.097+0000 7f6410c00700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f640c070f10 con 0x7f640c103720 2024-04-12T20:50:46.097 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:46.098+0000 7f6409e00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f640c102d80 0x7f640c0702c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T20:50:46.097 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:46.098+0000 7f6409e00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f640c102d80 0x7f640c0702c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:44158/0 (socket says 172.21.15.18:44158) 2024-04-12T20:50:46.097 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:46.098+0000 7f6409e00700 1 -- 172.21.15.18:0/1664518800 learned_addr learned my addr 172.21.15.18:0/1664518800 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T20:50:46.098 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:46.098+0000 7f6409400700 1 --2- 172.21.15.18:0/1664518800 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f640c103720 0x7f640c070800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T20:50:46.098 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:46.098+0000 7f6409e00700 1 -- 172.21.15.18:0/1664518800 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f640c103720 msgr2=0x7f640c070800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:50:46.098 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:46.098+0000 7f6409e00700 1 --2- 172.21.15.18:0/1664518800 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f640c103720 0x7f640c070800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:50:46.099 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:46.098+0000 7f6409e00700 1 -- 172.21.15.18:0/1664518800 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63fc009940 con 0x7f640c102d80 2024-04-12T20:50:46.099 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:46.098+0000 7f6409e00700 1 --2- 172.21.15.18:0/1664518800 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f640c102d80 0x7f640c0702c0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f63fc005010 tx=0x7f63fc004d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T20:50:46.099 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:46.099+0000 7f6402000700 1 -- 172.21.15.18:0/1664518800 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f63fc00e040 con 0x7f640c102d80 2024-04-12T20:50:46.099 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:46.099+0000 7f6402000700 1 -- 172.21.15.18:0/1664518800 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f63fc008e20 con 0x7f640c102d80 2024-04-12T20:50:46.099 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:46.099+0000 7f6402000700 1 -- 172.21.15.18:0/1664518800 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f63fc01c660 con 0x7f640c102d80 2024-04-12T20:50:46.099 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:46.099+0000 7f6410c00700 1 -- 172.21.15.18:0/1664518800 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f640c11a020 con 0x7f640c102d80 2024-04-12T20:50:46.099 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:46.099+0000 7f6410c00700 1 -- 172.21.15.18:0/1664518800 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f640c11a4a0 con 0x7f640c102d80 2024-04-12T20:50:46.099 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:46.100+0000 7f6410c00700 1 -- 172.21.15.18:0/1664518800 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f640c100090 con 0x7f640c102d80 2024-04-12T20:50:46.100 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:46.101+0000 7f6402000700 1 -- 172.21.15.18:0/1664518800 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7f63fc012a70 con 0x7f640c102d80 2024-04-12T20:50:46.102 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:46.103+0000 7f6402000700 1 --2- 172.21.15.18:0/1664518800 >> [v2:172.21.15.18:6800/3048894687,v1:172.21.15.18:6801/3048894687] conn(0x7f63f0069300 0x7f63f006b7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T20:50:46.102 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:46.103+0000 7f6402000700 1 -- 172.21.15.18:0/1664518800 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4634+0+0 (secure 0 0 0) 0x7f63fc088b40 con 0x7f640c102d80 2024-04-12T20:50:46.102 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:46.103+0000 7f6409400700 1 --2- 172.21.15.18:0/1664518800 >> [v2:172.21.15.18:6800/3048894687,v1:172.21.15.18:6801/3048894687] conn(0x7f63f0069300 0x7f63f006b7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T20:50:46.103 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:46.104+0000 7f6409400700 1 --2- 172.21.15.18:0/1664518800 >> [v2:172.21.15.18:6800/3048894687,v1:172.21.15.18:6801/3048894687] conn(0x7f63f0069300 0x7f63f006b7b0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f63f8006fd0 tx=0x7f63f8008040 comp rx=0 tx=0).ready entity=mgr.14212 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T20:50:46.106 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:46.107+0000 7f6402000700 1 -- 172.21.15.18:0/1664518800 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7f63fc055090 con 0x7f640c102d80 2024-04-12T20:50:46.281 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:50:46 smithi186 ceph-mon[36597]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-12T20:50:46.322 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:46.323+0000 7f6410c00700 1 -- 172.21.15.18:0/1664518800 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f640c0fe4d0 con 0x7f640c102d80 2024-04-12T20:50:46.322 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:50:46 smithi018 ceph-mon[28799]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-12T20:50:46.323 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:46.324+0000 7f6402000700 1 -- 172.21.15.18:0/1664518800 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v40) v1 ==== 74+0+11072 (secure 0 0 0) 0x7f63fc0586b0 con 0x7f640c102d80 2024-04-12T20:50:46.324 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-12T20:50:46.324 INFO:teuthology.orchestra.run.smithi018.stdout:{"epoch":40,"fsid":"1cbe407a-f90d-11ee-bc8d-c7b262605968","created":"2024-04-12T20:43:00.692361+0000","modified":"2024-04-12T20:50:18.227750+0000","last_up_change":"2024-04-12T20:50:17.011011+0000","last_in_change":"2024-04-12T20:49:54.190644+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":14,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-12T20:48:37.529081+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}}],"osds":[{"osd":0,"uuid":"2af887cc-0447-42aa-a8f3-1aa7fcd4dc8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6802","nonce":585613146},{"type":"v1","addr":"172.21.15.18:6803","nonce":585613146}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6804","nonce":585613146},{"type":"v1","addr":"172.21.15.18:6805","nonce":585613146}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6808","nonce":585613146},{"type":"v1","addr":"172.21.15.18:6809","nonce":585613146}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6806","nonce":585613146},{"type":"v1","addr":"172.21.15.18:6807","nonce":585613146}]},"public_addr":"172.21.15.18:6803/585613146","cluster_addr":"172.21.15.18:6805/585613146","heartbeat_back_addr":"172.21.15.18:6809/585613146","heartbeat_front_addr":"172.21.15.18:6807/585613146","state":["exists","up"]},{"osd":1,"uuid":"6ba7b0d3-a03d-45cc-b4e9-89a92c8c6ca8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6810","nonce":181951035},{"type":"v1","addr":"172.21.15.18:6811","nonce":181951035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6812","nonce":181951035},{"type":"v1","addr":"172.21.15.18:6813","nonce":181951035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6816","nonce":181951035},{"type":"v1","addr":"172.21.15.18:6817","nonce":181951035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6814","nonce":181951035},{"type":"v1","addr":"172.21.15.18:6815","nonce":181951035}]},"public_addr":"172.21.15.18:6811/181951035","cluster_addr":"172.21.15.18:6813/181951035","heartbeat_back_addr":"172.21.15.18:6817/181951035","heartbeat_front_addr":"172.21.15.18:6815/181951035","state":["exists","up"]},{"osd":2,"uuid":"5bb1137d-5a3c-40bc-ae08-fc792b8ee547","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6818","nonce":3573247809},{"type":"v1","addr":"172.21.15.18:6819","nonce":3573247809}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6820","nonce":3573247809},{"type":"v1","addr":"172.21.15.18:6821","nonce":3573247809}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6824","nonce":3573247809},{"type":"v1","addr":"172.21.15.18:6825","nonce":3573247809}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6822","nonce":3573247809},{"type":"v1","addr":"172.21.15.18:6823","nonce":3573247809}]},"public_addr":"172.21.15.18:6819/3573247809","cluster_addr":"172.21.15.18:6821/3573247809","heartbeat_back_addr":"172.21.15.18:6825/3573247809","heartbeat_front_addr":"172.21.15.18:6823/3573247809","state":["exists","up"]},{"osd":3,"uuid":"a5140a33-1e90-4980-aa9a-b0afe6b155f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6800","nonce":2184799172},{"type":"v1","addr":"172.21.15.186:6801","nonce":2184799172}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6802","nonce":2184799172},{"type":"v1","addr":"172.21.15.186:6803","nonce":2184799172}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6806","nonce":2184799172},{"type":"v1","addr":"172.21.15.186:6807","nonce":2184799172}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6804","nonce":2184799172},{"type":"v1","addr":"172.21.15.186:6805","nonce":2184799172}]},"public_addr":"172.21.15.186:6801/2184799172","cluster_addr":"172.21.15.186:6803/2184799172","heartbeat_back_addr":"172.21.15.186:6807/2184799172","heartbeat_front_addr":"172.21.15.186:6805/2184799172","state":["exists","up"]},{"osd":4,"uuid":"d6b2c6b3-6c78-4261-bcbe-c33358407198","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6808","nonce":3584843287},{"type":"v1","addr":"172.21.15.186:6809","nonce":3584843287}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6810","nonce":3584843287},{"type":"v1","addr":"172.21.15.186:6811","nonce":3584843287}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6814","nonce":3584843287},{"type":"v1","addr":"172.21.15.186:6815","nonce":3584843287}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6812","nonce":3584843287},{"type":"v1","addr":"172.21.15.186:6813","nonce":3584843287}]},"public_addr":"172.21.15.186:6809/3584843287","cluster_addr":"172.21.15.186:6811/3584843287","heartbeat_back_addr":"172.21.15.186:6815/3584843287","heartbeat_front_addr":"172.21.15.186:6813/3584843287","state":["exists","up"]},{"osd":5,"uuid":"7b5c9f17-6306-4fb5-aa77-fc9a73bcc663","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6816","nonce":3543055150},{"type":"v1","addr":"172.21.15.186:6817","nonce":3543055150}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6818","nonce":3543055150},{"type":"v1","addr":"172.21.15.186:6819","nonce":3543055150}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6822","nonce":3543055150},{"type":"v1","addr":"172.21.15.186:6823","nonce":3543055150}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6820","nonce":3543055150},{"type":"v1","addr":"172.21.15.186:6821","nonce":3543055150}]},"public_addr":"172.21.15.186:6817/3543055150","cluster_addr":"172.21.15.186:6819/3543055150","heartbeat_back_addr":"172.21.15.186:6823/3543055150","heartbeat_front_addr":"172.21.15.186:6821/3543055150","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-04-12T20:47:21.231060+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-04-12T20:47:57.699205+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-04-12T20:48:35.096648+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-04-12T20:49:08.252810+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-04-12T20:49:41.102187+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-04-12T20:50:16.009813+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.18:0/1720919549":"2024-04-13T20:45:44.776072+0000","172.21.15.18:0/3896210349":"2024-04-13T20:44:15.143139+0000","172.21.15.18:6801/1391635591":"2024-04-13T20:45:44.776072+0000","172.21.15.18:6800/1391635591":"2024-04-13T20:45:44.776072+0000","172.21.15.18:0/3387911038":"2024-04-13T20:44:15.143139+0000","172.21.15.18:0/4018777476":"2024-04-13T20:45:44.776072+0000","172.21.15.18:6801/3403971131":"2024-04-13T20:44:15.143139+0000","172.21.15.18:0/2945432483":"2024-04-13T20:44:15.143139+0000","172.21.15.18:0/1732042827":"2024-04-13T20:43:34.269559+0000","172.21.15.18:0/766256564":"2024-04-13T20:45:44.776072+0000","172.21.15.18:0/2517252399":"2024-04-13T20:43:34.269559+0000","172.21.15.18:0/3295870133":"2024-04-13T20:43:34.269559+0000","172.21.15.18:6800/3403971131":"2024-04-13T20:44:15.143139+0000","172.21.15.18:6800/1312802974":"2024-04-13T20:43:34.269559+0000","172.21.15.18:6801/1312802974":"2024-04-13T20:43:34.269559+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-12T20:50:46.327 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:46.327+0000 7f6410c00700 1 -- 172.21.15.18:0/1664518800 >> [v2:172.21.15.18:6800/3048894687,v1:172.21.15.18:6801/3048894687] conn(0x7f63f0069300 msgr2=0x7f63f006b7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:50:46.327 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:46.327+0000 7f6410c00700 1 --2- 172.21.15.18:0/1664518800 >> [v2:172.21.15.18:6800/3048894687,v1:172.21.15.18:6801/3048894687] conn(0x7f63f0069300 0x7f63f006b7b0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f63f8006fd0 tx=0x7f63f8008040 comp rx=0 tx=0).stop 2024-04-12T20:50:46.327 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:46.327+0000 7f6410c00700 1 -- 172.21.15.18:0/1664518800 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f640c102d80 msgr2=0x7f640c0702c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:50:46.327 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:46.327+0000 7f6410c00700 1 --2- 172.21.15.18:0/1664518800 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f640c102d80 0x7f640c0702c0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f63fc005010 tx=0x7f63fc004d10 comp rx=0 tx=0).stop 2024-04-12T20:50:46.327 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:46.327+0000 7f6410c00700 1 -- 172.21.15.18:0/1664518800 shutdown_connections 2024-04-12T20:50:46.327 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:46.328+0000 7f6410c00700 1 --2- 172.21.15.18:0/1664518800 >> [v2:172.21.15.18:6800/3048894687,v1:172.21.15.18:6801/3048894687] conn(0x7f63f0069300 0x7f63f006b7b0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:50:46.328 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:46.328+0000 7f6410c00700 1 --2- 172.21.15.18:0/1664518800 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f640c102d80 0x7f640c0702c0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:50:46.328 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:46.328+0000 7f6410c00700 1 --2- 172.21.15.18:0/1664518800 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f640c103720 0x7f640c070800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:50:46.328 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:46.328+0000 7f6410c00700 1 -- 172.21.15.18:0/1664518800 >> 172.21.15.18:0/1664518800 conn(0x7f640c0f8930 msgr2=0x7f640c107340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T20:50:46.328 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:46.328+0000 7f6410c00700 1 -- 172.21.15.18:0/1664518800 shutdown_connections 2024-04-12T20:50:46.328 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:46.328+0000 7f6410c00700 1 -- 172.21.15.18:0/1664518800 wait complete. 2024-04-12T20:50:47.114 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid 1cbe407a-f90d-11ee-bc8d-c7b262605968 -- ceph tell osd.0 flush_pg_stats 2024-04-12T20:50:47.115 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid 1cbe407a-f90d-11ee-bc8d-c7b262605968 -- ceph tell osd.1 flush_pg_stats 2024-04-12T20:50:47.115 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid 1cbe407a-f90d-11ee-bc8d-c7b262605968 -- ceph tell osd.2 flush_pg_stats 2024-04-12T20:50:47.115 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid 1cbe407a-f90d-11ee-bc8d-c7b262605968 -- ceph tell osd.3 flush_pg_stats 2024-04-12T20:50:47.116 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid 1cbe407a-f90d-11ee-bc8d-c7b262605968 -- ceph tell osd.4 flush_pg_stats 2024-04-12T20:50:47.116 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid 1cbe407a-f90d-11ee-bc8d-c7b262605968 -- ceph tell osd.5 flush_pg_stats 2024-04-12T20:50:47.282 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:50:47 smithi186 ceph-mon[36597]: from='client.? 172.21.15.18:0/1664518800' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-12T20:50:47.352 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:50:47 smithi018 ceph-mon[28799]: from='client.? 172.21.15.18:0/1664518800' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-12T20:50:47.656 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/mon.smithi018/config 2024-04-12T20:50:47.681 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/mon.smithi018/config 2024-04-12T20:50:48.097 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:50:48 smithi018 ceph-mon[28799]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-12T20:50:48.282 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:50:48 smithi186 ceph-mon[36597]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-12T20:50:49.696 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/mon.smithi018/config 2024-04-12T20:50:50.293 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:50:50 smithi018 ceph-mon[28799]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-12T20:50:50.531 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:50:50 smithi186 ceph-mon[36597]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-12T20:50:50.801 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/mon.smithi018/config 2024-04-12T20:50:50.810 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/mon.smithi018/config 2024-04-12T20:50:51.747 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:51.491+0000 7f43b7200700 1 -- 172.21.15.18:0/3591792293 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f43b8102c40 msgr2=0x7f43b8103010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:50:51.747 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:51.491+0000 7f43b7200700 1 --2- 172.21.15.18:0/3591792293 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f43b8102c40 0x7f43b8103010 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f43a8004940 tx=0x7f43a8009e30 comp rx=0 tx=0).stop 2024-04-12T20:50:51.747 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:51.491+0000 7f43b7200700 1 -- 172.21.15.18:0/3591792293 shutdown_connections 2024-04-12T20:50:51.747 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:51.491+0000 7f43b7200700 1 --2- 172.21.15.18:0/3591792293 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f43b81035e0 0x7f43b80fcc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:50:51.747 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:51.491+0000 7f43b7200700 1 --2- 172.21.15.18:0/3591792293 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f43b8102c40 0x7f43b8103010 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:50:51.748 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:51.491+0000 7f43b7200700 1 -- 172.21.15.18:0/3591792293 >> 172.21.15.18:0/3591792293 conn(0x7f43b80f87f0 msgr2=0x7f43b80fac00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T20:50:51.748 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:51.491+0000 7f43b7200700 1 -- 172.21.15.18:0/3591792293 shutdown_connections 2024-04-12T20:50:51.748 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:51.491+0000 7f43b7200700 1 -- 172.21.15.18:0/3591792293 wait complete. 2024-04-12T20:50:51.748 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:51.492+0000 7f43b7200700 1 Processor -- start 2024-04-12T20:50:51.748 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:51.492+0000 7f43b7200700 1 -- start start 2024-04-12T20:50:51.748 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:51.493+0000 7f43b7200700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f43b8102c40 0x7f43b8115c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T20:50:51.749 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:51.493+0000 7f43b7200700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f43b81035e0 0x7f43b8116170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T20:50:51.749 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:51.493+0000 7f43b7200700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43b81167c0 con 0x7f43b81035e0 2024-04-12T20:50:51.749 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:51.493+0000 7f43b7200700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43b810fb90 con 0x7f43b8102c40 2024-04-12T20:50:51.749 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:51.493+0000 7f43b5400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f43b81035e0 0x7f43b8116170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T20:50:51.749 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:51.493+0000 7f43b5400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f43b81035e0 0x7f43b8116170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:50426/0 (socket says 172.21.15.18:50426) 2024-04-12T20:50:51.749 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:51.493+0000 7f43b5400700 1 -- 172.21.15.18:0/3217453740 learned_addr learned my addr 172.21.15.18:0/3217453740 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T20:50:51.749 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:51.494+0000 7f43b5400700 1 -- 172.21.15.18:0/3217453740 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f43b8102c40 msgr2=0x7f43b8115c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:50:51.749 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:51.494+0000 7f43b5400700 1 --2- 172.21.15.18:0/3217453740 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f43b8102c40 0x7f43b8115c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:50:51.749 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:51.494+0000 7f43b5400700 1 -- 172.21.15.18:0/3217453740 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43a8005040 con 0x7f43b81035e0 2024-04-12T20:50:51.749 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:51.494+0000 7f43b5400700 1 --2- 172.21.15.18:0/3217453740 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f43b81035e0 0x7f43b8116170 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f43a400c890 tx=0x7f43a40044c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T20:50:51.749 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:51.494+0000 7f43ae000700 1 -- 172.21.15.18:0/3217453740 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f43a400a7d0 con 0x7f43b81035e0 2024-04-12T20:50:51.749 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:51.494+0000 7f43ae000700 1 -- 172.21.15.18:0/3217453740 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f43a400ae00 con 0x7f43b81035e0 2024-04-12T20:50:51.749 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:51.494+0000 7f43b7200700 1 -- 172.21.15.18:0/3217453740 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f43b810fe70 con 0x7f43b81035e0 2024-04-12T20:50:51.750 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:51.494+0000 7f43ae000700 1 -- 172.21.15.18:0/3217453740 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f43a40113a0 con 0x7f43b81035e0 2024-04-12T20:50:51.750 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:51.495+0000 7f43b7200700 1 -- 172.21.15.18:0/3217453740 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f43b8110430 con 0x7f43b81035e0 2024-04-12T20:50:51.750 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:51.495+0000 7f43b7200700 1 -- 172.21.15.18:0/3217453740 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f43b8110970 con 0x7f43b81035e0 2024-04-12T20:50:51.750 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:51.497+0000 7f43ae000700 1 -- 172.21.15.18:0/3217453740 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7f43a4012400 con 0x7f43b81035e0 2024-04-12T20:50:51.750 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:51.498+0000 7f43ae000700 1 --2- 172.21.15.18:0/3217453740 >> [v2:172.21.15.18:6800/3048894687,v1:172.21.15.18:6801/3048894687] conn(0x7f439c0692a0 0x7f439c06b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T20:50:51.750 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:51.498+0000 7f43ae000700 1 -- 172.21.15.18:0/3217453740 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4634+0+0 (secure 0 0 0) 0x7f43a4086fd0 con 0x7f43b81035e0 2024-04-12T20:50:51.750 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:51.498+0000 7f43b5e00700 1 --2- 172.21.15.18:0/3217453740 >> [v2:172.21.15.18:6800/3048894687,v1:172.21.15.18:6801/3048894687] conn(0x7f439c0692a0 0x7f439c06b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T20:50:51.750 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:51.499+0000 7f43ae000700 1 --2- 172.21.15.18:0/3217453740 >> [v2:172.21.15.18:6810/181951035,v1:172.21.15.18:6811/181951035] conn(0x7f439c06ef70 0x7f439c071380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T20:50:51.750 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:51.499+0000 7f43ae000700 1 -- 172.21.15.18:0/3217453740 --> [v2:172.21.15.18:6810/181951035,v1:172.21.15.18:6811/181951035] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f439c071a30 con 0x7f439c06ef70 2024-04-12T20:50:51.750 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:51.499+0000 7f43ae000700 1 -- 172.21.15.18:0/3217453740 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7f43a4089530 con 0x7f43b81035e0 2024-04-12T20:50:51.751 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:51.499+0000 7f43b6800700 1 --2- 172.21.15.18:0/3217453740 >> [v2:172.21.15.18:6810/181951035,v1:172.21.15.18:6811/181951035] conn(0x7f439c06ef70 0x7f439c071380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T20:50:51.751 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:51.499+0000 7f43b5e00700 1 --2- 172.21.15.18:0/3217453740 >> [v2:172.21.15.18:6800/3048894687,v1:172.21.15.18:6801/3048894687] conn(0x7f439c0692a0 0x7f439c06b750 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f43a8004f90 tx=0x7f43a8009e30 comp rx=0 tx=0).ready entity=mgr.14212 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T20:50:51.751 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:51.500+0000 7f43b6800700 1 --2- 172.21.15.18:0/3217453740 >> [v2:172.21.15.18:6810/181951035,v1:172.21.15.18:6811/181951035] conn(0x7f439c06ef70 0x7f439c071380 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T20:50:51.751 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:51.501+0000 7f43ae000700 1 -- 172.21.15.18:0/3217453740 <== osd.1 v2:172.21.15.18:6810/181951035 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+24147 (crc 0 0 0) 0x7f439c071a30 con 0x7f439c06ef70 2024-04-12T20:50:51.751 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:51.517+0000 7f43b7200700 1 -- 172.21.15.18:0/3217453740 --> [v2:172.21.15.18:6810/181951035,v1:172.21.15.18:6811/181951035] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f43b80705f0 con 0x7f439c06ef70 2024-04-12T20:50:51.751 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:51.518+0000 7f43ae000700 1 -- 172.21.15.18:0/3217453740 <== osd.1 v2:172.21.15.18:6810/181951035 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f43b80705f0 con 0x7f439c06ef70 2024-04-12T20:50:51.751 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:51.518+0000 7f43b7200700 1 -- 172.21.15.18:0/3217453740 >> [v2:172.21.15.18:6810/181951035,v1:172.21.15.18:6811/181951035] conn(0x7f439c06ef70 msgr2=0x7f439c071380 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:50:51.751 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:51.518+0000 7f43b7200700 1 --2- 172.21.15.18:0/3217453740 >> [v2:172.21.15.18:6810/181951035,v1:172.21.15.18:6811/181951035] conn(0x7f439c06ef70 0x7f439c071380 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:50:51.751 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:51.518+0000 7f43b7200700 1 -- 172.21.15.18:0/3217453740 >> [v2:172.21.15.18:6800/3048894687,v1:172.21.15.18:6801/3048894687] conn(0x7f439c0692a0 msgr2=0x7f439c06b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:50:51.751 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:51.518+0000 7f43b7200700 1 --2- 172.21.15.18:0/3217453740 >> [v2:172.21.15.18:6800/3048894687,v1:172.21.15.18:6801/3048894687] conn(0x7f439c0692a0 0x7f439c06b750 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f43a8004f90 tx=0x7f43a8009e30 comp rx=0 tx=0).stop 2024-04-12T20:50:51.751 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:51.518+0000 7f43b7200700 1 -- 172.21.15.18:0/3217453740 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f43b81035e0 msgr2=0x7f43b8116170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:50:51.751 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:51.518+0000 7f43b7200700 1 --2- 172.21.15.18:0/3217453740 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f43b81035e0 0x7f43b8116170 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f43a400c890 tx=0x7f43a40044c0 comp rx=0 tx=0).stop 2024-04-12T20:50:51.751 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:51.518+0000 7f43b7200700 1 -- 172.21.15.18:0/3217453740 shutdown_connections 2024-04-12T20:50:51.751 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:51.518+0000 7f43b7200700 1 --2- 172.21.15.18:0/3217453740 >> [v2:172.21.15.18:6810/181951035,v1:172.21.15.18:6811/181951035] conn(0x7f439c06ef70 0x7f439c071380 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:50:51.752 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:51.518+0000 7f43b7200700 1 --2- 172.21.15.18:0/3217453740 >> [v2:172.21.15.18:6800/3048894687,v1:172.21.15.18:6801/3048894687] conn(0x7f439c0692a0 0x7f439c06b750 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:50:51.752 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:51.518+0000 7f43b7200700 1 --2- 172.21.15.18:0/3217453740 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f43b8102c40 0x7f43b8115c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:50:51.752 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:51.518+0000 7f43b7200700 1 --2- 172.21.15.18:0/3217453740 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f43b81035e0 0x7f43b8116170 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:50:51.752 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:51.518+0000 7f43b7200700 1 -- 172.21.15.18:0/3217453740 >> 172.21.15.18:0/3217453740 conn(0x7f43b80f87f0 msgr2=0x7f43b8071af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T20:50:51.752 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:51.518+0000 7f43b7200700 1 -- 172.21.15.18:0/3217453740 shutdown_connections 2024-04-12T20:50:51.752 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:51.519+0000 7f43b7200700 1 -- 172.21.15.18:0/3217453740 wait complete. 2024-04-12T20:50:52.313 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/mon.smithi018/config 2024-04-12T20:50:52.352 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:50:52 smithi018 ceph-mon[28799]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-12T20:50:52.531 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:50:52 smithi186 ceph-mon[36597]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-12T20:50:53.641 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:53.641+0000 7f9a92a00700 1 -- 172.21.15.18:0/2033281969 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9a8c1035d0 msgr2=0x7f9a8c0fcc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:50:53.641 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:53.641+0000 7f9a92a00700 1 --2- 172.21.15.18:0/2033281969 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9a8c1035d0 0x7f9a8c0fcc40 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f9a78004940 tx=0x7f9a78009e30 comp rx=0 tx=0).stop 2024-04-12T20:50:53.642 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:53.641+0000 7f9a92a00700 1 -- 172.21.15.18:0/2033281969 shutdown_connections 2024-04-12T20:50:53.642 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:53.641+0000 7f9a92a00700 1 --2- 172.21.15.18:0/2033281969 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9a8c1035d0 0x7f9a8c0fcc40 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:50:53.642 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:53.641+0000 7f9a92a00700 1 --2- 172.21.15.18:0/2033281969 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9a8c102c30 0x7f9a8c103000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:50:53.642 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:53.642+0000 7f9a92a00700 1 -- 172.21.15.18:0/2033281969 >> 172.21.15.18:0/2033281969 conn(0x7f9a8c0f8760 msgr2=0x7f9a8c0fab70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T20:50:53.642 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:53.642+0000 7f9a92a00700 1 -- 172.21.15.18:0/2033281969 shutdown_connections 2024-04-12T20:50:53.642 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:53.642+0000 7f9a92a00700 1 -- 172.21.15.18:0/2033281969 wait complete. 2024-04-12T20:50:53.642 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:53.642+0000 7f9a92a00700 1 Processor -- start 2024-04-12T20:50:53.642 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:53.643+0000 7f9a92a00700 1 -- start start 2024-04-12T20:50:53.643 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:53.643+0000 7f9a92a00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9a8c102c30 0x7f9a8c115c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T20:50:53.643 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:53.643+0000 7f9a92a00700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9a8c1035d0 0x7f9a8c116140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T20:50:53.643 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:53.643+0000 7f9a92a00700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a8c116790 con 0x7f9a8c102c30 2024-04-12T20:50:53.644 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:53.643+0000 7f9a92a00700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a8c10fb80 con 0x7f9a8c1035d0 2024-04-12T20:50:53.644 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:53.643+0000 7f9a91600700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9a8c102c30 0x7f9a8c115c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T20:50:53.644 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:53.643+0000 7f9a91600700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9a8c102c30 0x7f9a8c115c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:50442/0 (socket says 172.21.15.18:50442) 2024-04-12T20:50:53.644 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:53.643+0000 7f9a91600700 1 -- 172.21.15.18:0/378888107 learned_addr learned my addr 172.21.15.18:0/378888107 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T20:50:53.644 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:53.644+0000 7f9a90c00700 1 --2- 172.21.15.18:0/378888107 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9a8c1035d0 0x7f9a8c116140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T20:50:53.644 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:53.644+0000 7f9a91600700 1 -- 172.21.15.18:0/378888107 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9a8c1035d0 msgr2=0x7f9a8c116140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:50:53.644 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:53.644+0000 7f9a91600700 1 --2- 172.21.15.18:0/378888107 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9a8c1035d0 0x7f9a8c116140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:50:53.644 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:53.644+0000 7f9a91600700 1 -- 172.21.15.18:0/378888107 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a78005040 con 0x7f9a8c102c30 2024-04-12T20:50:53.644 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:53.644+0000 7f9a91600700 1 --2- 172.21.15.18:0/378888107 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9a8c102c30 0x7f9a8c115c00 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f9a8000eb40 tx=0x7f9a8000b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T20:50:53.645 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:53.645+0000 7f9a89600700 1 -- 172.21.15.18:0/378888107 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9a80007cc0 con 0x7f9a8c102c30 2024-04-12T20:50:53.645 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:53.645+0000 7f9a89600700 1 -- 172.21.15.18:0/378888107 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f9a8000f040 con 0x7f9a8c102c30 2024-04-12T20:50:53.645 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:53.645+0000 7f9a89600700 1 -- 172.21.15.18:0/378888107 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9a80009620 con 0x7f9a8c102c30 2024-04-12T20:50:53.646 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:53.645+0000 7f9a92a00700 1 -- 172.21.15.18:0/378888107 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9a8c10fe00 con 0x7f9a8c102c30 2024-04-12T20:50:53.646 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:53.645+0000 7f9a92a00700 1 -- 172.21.15.18:0/378888107 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9a8c1102c0 con 0x7f9a8c102c30 2024-04-12T20:50:53.646 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:53.646+0000 7f9a92a00700 1 -- 172.21.15.18:0/378888107 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f9a8c110870 con 0x7f9a8c102c30 2024-04-12T20:50:53.646 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:53.647+0000 7f9a89600700 1 -- 172.21.15.18:0/378888107 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7f9a80009780 con 0x7f9a8c102c30 2024-04-12T20:50:53.647 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:53.648+0000 7f9a89600700 1 --2- 172.21.15.18:0/378888107 >> [v2:172.21.15.18:6800/3048894687,v1:172.21.15.18:6801/3048894687] conn(0x7f9a740692a0 0x7f9a7406b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T20:50:53.647 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:53.648+0000 7f9a89600700 1 -- 172.21.15.18:0/378888107 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4634+0+0 (secure 0 0 0) 0x7f9a80086af0 con 0x7f9a8c102c30 2024-04-12T20:50:53.648 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:53.648+0000 7f9a89600700 1 --2- 172.21.15.18:0/378888107 >> [v2:172.21.15.18:6802/585613146,v1:172.21.15.18:6803/585613146] conn(0x7f9a7406ef70 0x7f9a74071380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T20:50:53.648 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:53.648+0000 7f9a90c00700 1 --2- 172.21.15.18:0/378888107 >> [v2:172.21.15.18:6800/3048894687,v1:172.21.15.18:6801/3048894687] conn(0x7f9a740692a0 0x7f9a7406b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T20:50:53.648 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:53.648+0000 7f9a89600700 1 -- 172.21.15.18:0/378888107 --> [v2:172.21.15.18:6802/585613146,v1:172.21.15.18:6803/585613146] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f9a74071a30 con 0x7f9a7406ef70 2024-04-12T20:50:53.648 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:53.649+0000 7f9a89600700 1 -- 172.21.15.18:0/378888107 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9a8008b220 con 0x7f9a8c102c30 2024-04-12T20:50:53.648 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:53.649+0000 7f9a92000700 1 --2- 172.21.15.18:0/378888107 >> [v2:172.21.15.18:6802/585613146,v1:172.21.15.18:6803/585613146] conn(0x7f9a7406ef70 0x7f9a74071380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T20:50:53.648 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:53.649+0000 7f9a90c00700 1 --2- 172.21.15.18:0/378888107 >> [v2:172.21.15.18:6800/3048894687,v1:172.21.15.18:6801/3048894687] conn(0x7f9a740692a0 0x7f9a7406b750 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f9a78009b30 tx=0x7f9a78009e30 comp rx=0 tx=0).ready entity=mgr.14212 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T20:50:53.649 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:53.650+0000 7f9a92000700 1 --2- 172.21.15.18:0/378888107 >> [v2:172.21.15.18:6802/585613146,v1:172.21.15.18:6803/585613146] conn(0x7f9a7406ef70 0x7f9a74071380 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T20:50:53.650 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:53.651+0000 7f9a89600700 1 -- 172.21.15.18:0/378888107 <== osd.0 v2:172.21.15.18:6802/585613146 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+24147 (crc 0 0 0) 0x7f9a74071a30 con 0x7f9a7406ef70 2024-04-12T20:50:53.681 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:53.681+0000 7f9a92a00700 1 -- 172.21.15.18:0/378888107 --> [v2:172.21.15.18:6802/585613146,v1:172.21.15.18:6803/585613146] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f9a8c0fe380 con 0x7f9a7406ef70 2024-04-12T20:50:53.681 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:53.682+0000 7f9a89600700 1 -- 172.21.15.18:0/378888107 <== osd.0 v2:172.21.15.18:6802/585613146 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f9a8c0fe380 con 0x7f9a7406ef70 2024-04-12T20:50:53.682 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:53.683+0000 7f9a92a00700 1 -- 172.21.15.18:0/378888107 >> [v2:172.21.15.18:6802/585613146,v1:172.21.15.18:6803/585613146] conn(0x7f9a7406ef70 msgr2=0x7f9a74071380 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:50:53.682 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:53.683+0000 7f9a92a00700 1 --2- 172.21.15.18:0/378888107 >> [v2:172.21.15.18:6802/585613146,v1:172.21.15.18:6803/585613146] conn(0x7f9a7406ef70 0x7f9a74071380 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:50:53.683 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:53.683+0000 7f9a92a00700 1 -- 172.21.15.18:0/378888107 >> [v2:172.21.15.18:6800/3048894687,v1:172.21.15.18:6801/3048894687] conn(0x7f9a740692a0 msgr2=0x7f9a7406b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:50:53.683 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:53.683+0000 7f9a92a00700 1 --2- 172.21.15.18:0/378888107 >> [v2:172.21.15.18:6800/3048894687,v1:172.21.15.18:6801/3048894687] conn(0x7f9a740692a0 0x7f9a7406b750 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f9a78009b30 tx=0x7f9a78009e30 comp rx=0 tx=0).stop 2024-04-12T20:50:53.683 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:53.683+0000 7f9a92a00700 1 -- 172.21.15.18:0/378888107 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9a8c102c30 msgr2=0x7f9a8c115c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:50:53.683 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:53.683+0000 7f9a92a00700 1 --2- 172.21.15.18:0/378888107 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9a8c102c30 0x7f9a8c115c00 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f9a8000eb40 tx=0x7f9a8000b040 comp rx=0 tx=0).stop 2024-04-12T20:50:53.683 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:53.683+0000 7f9a92a00700 1 -- 172.21.15.18:0/378888107 shutdown_connections 2024-04-12T20:50:53.683 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:53.683+0000 7f9a92a00700 1 --2- 172.21.15.18:0/378888107 >> [v2:172.21.15.18:6800/3048894687,v1:172.21.15.18:6801/3048894687] conn(0x7f9a740692a0 0x7f9a7406b750 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:50:53.683 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:53.683+0000 7f9a92a00700 1 --2- 172.21.15.18:0/378888107 >> [v2:172.21.15.18:6802/585613146,v1:172.21.15.18:6803/585613146] conn(0x7f9a7406ef70 0x7f9a74071380 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:50:53.683 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:53.683+0000 7f9a92a00700 1 --2- 172.21.15.18:0/378888107 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9a8c102c30 0x7f9a8c115c00 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:50:53.683 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:53.683+0000 7f9a92a00700 1 --2- 172.21.15.18:0/378888107 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9a8c1035d0 0x7f9a8c116140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:50:53.684 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:53.683+0000 7f9a92a00700 1 -- 172.21.15.18:0/378888107 >> 172.21.15.18:0/378888107 conn(0x7f9a8c0f8760 msgr2=0x7f9a8c071af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T20:50:53.684 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:53.683+0000 7f9a92a00700 1 -- 172.21.15.18:0/378888107 shutdown_connections 2024-04-12T20:50:53.684 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:53.683+0000 7f9a92a00700 1 -- 172.21.15.18:0/378888107 wait complete. 2024-04-12T20:50:54.304 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:50:54 smithi018 ceph-mon[28799]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-12T20:50:54.531 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:50:54 smithi186 ceph-mon[36597]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-12T20:50:56.194 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:50:56 smithi018 ceph-mon[28799]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-12T20:50:56.531 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:50:56 smithi186 ceph-mon[36597]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-12T20:50:57.434 INFO:teuthology.orchestra.run.smithi018.stdout:38654705709 2024-04-12T20:50:57.435 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid 1cbe407a-f90d-11ee-bc8d-c7b262605968 -- ceph osd last-stat-seq osd.0 2024-04-12T20:50:57.605 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:57.217+0000 7f20b8c00700 1 -- 172.21.15.18:0/3539304986 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f20b4070bf0 msgr2=0x7f20b41091b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:50:57.606 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:57.217+0000 7f20b8c00700 1 --2- 172.21.15.18:0/3539304986 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f20b4070bf0 0x7f20b41091b0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f20a0004940 tx=0x7f20a0009e30 comp rx=0 tx=0).stop 2024-04-12T20:50:57.606 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:57.217+0000 7f20b8c00700 1 -- 172.21.15.18:0/3539304986 shutdown_connections 2024-04-12T20:50:57.606 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:57.217+0000 7f20b8c00700 1 --2- 172.21.15.18:0/3539304986 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f20b4070bf0 0x7f20b41091b0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:50:57.606 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:57.217+0000 7f20b8c00700 1 --2- 172.21.15.18:0/3539304986 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f20b40702e0 0x7f20b40706b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:50:57.606 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:57.217+0000 7f20b8c00700 1 -- 172.21.15.18:0/3539304986 >> 172.21.15.18:0/3539304986 conn(0x7f20b40f8970 msgr2=0x7f20b40fad80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T20:50:57.606 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:57.217+0000 7f20b8c00700 1 -- 172.21.15.18:0/3539304986 shutdown_connections 2024-04-12T20:50:57.606 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:57.217+0000 7f20b8c00700 1 -- 172.21.15.18:0/3539304986 wait complete. 2024-04-12T20:50:57.606 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:57.218+0000 7f20b8c00700 1 Processor -- start 2024-04-12T20:50:57.606 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:57.218+0000 7f20b8c00700 1 -- start start 2024-04-12T20:50:57.606 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:57.218+0000 7f20b8c00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f20b40702e0 0x7f20b4115e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T20:50:57.607 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:57.219+0000 7f20b8c00700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f20b4070bf0 0x7f20b4116370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T20:50:57.607 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:57.219+0000 7f20b8c00700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20b41169c0 con 0x7f20b40702e0 2024-04-12T20:50:57.607 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:57.219+0000 7f20b8c00700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20b410fd90 con 0x7f20b4070bf0 2024-04-12T20:50:57.607 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:57.219+0000 7f20b1e00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f20b40702e0 0x7f20b4115e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T20:50:57.607 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:57.219+0000 7f20b1e00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f20b40702e0 0x7f20b4115e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:50460/0 (socket says 172.21.15.18:50460) 2024-04-12T20:50:57.607 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:57.219+0000 7f20b1e00700 1 -- 172.21.15.18:0/2584136676 learned_addr learned my addr 172.21.15.18:0/2584136676 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T20:50:57.607 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:57.219+0000 7f20b1400700 1 --2- 172.21.15.18:0/2584136676 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f20b4070bf0 0x7f20b4116370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T20:50:57.607 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:57.219+0000 7f20b1e00700 1 -- 172.21.15.18:0/2584136676 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f20b4070bf0 msgr2=0x7f20b4116370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:50:57.607 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:57.219+0000 7f20b1e00700 1 --2- 172.21.15.18:0/2584136676 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f20b4070bf0 0x7f20b4116370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:50:57.607 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:57.219+0000 7f20b1e00700 1 -- 172.21.15.18:0/2584136676 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20a0005040 con 0x7f20b40702e0 2024-04-12T20:50:57.607 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:57.220+0000 7f20b1e00700 1 --2- 172.21.15.18:0/2584136676 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f20b40702e0 0x7f20b4115e30 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f20a400c830 tx=0x7f20a4004470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T20:50:57.607 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:57.220+0000 7f20aa000700 1 -- 172.21.15.18:0/2584136676 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f20a400a760 con 0x7f20b40702e0 2024-04-12T20:50:57.607 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:57.220+0000 7f20aa000700 1 -- 172.21.15.18:0/2584136676 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f20a400ad90 con 0x7f20b40702e0 2024-04-12T20:50:57.607 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:57.220+0000 7f20b8c00700 1 -- 172.21.15.18:0/2584136676 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f20b4110070 con 0x7f20b40702e0 2024-04-12T20:50:57.608 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:57.220+0000 7f20aa000700 1 -- 172.21.15.18:0/2584136676 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f20a40113a0 con 0x7f20b40702e0 2024-04-12T20:50:57.608 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:57.220+0000 7f20b8c00700 1 -- 172.21.15.18:0/2584136676 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f20b41105b0 con 0x7f20b40702e0 2024-04-12T20:50:57.608 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:57.221+0000 7f20b8c00700 1 -- 172.21.15.18:0/2584136676 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f20b4110b40 con 0x7f20b40702e0 2024-04-12T20:50:57.608 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:57.222+0000 7f20aa000700 1 -- 172.21.15.18:0/2584136676 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7f20a4012400 con 0x7f20b40702e0 2024-04-12T20:50:57.608 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:57.223+0000 7f20aa000700 1 --2- 172.21.15.18:0/2584136676 >> [v2:172.21.15.18:6800/3048894687,v1:172.21.15.18:6801/3048894687] conn(0x7f20980692b0 0x7f209806b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T20:50:57.608 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:57.223+0000 7f20aa000700 1 -- 172.21.15.18:0/2584136676 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4634+0+0 (secure 0 0 0) 0x7f20a4086e00 con 0x7f20b40702e0 2024-04-12T20:50:57.608 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:57.223+0000 7f20b1400700 1 --2- 172.21.15.18:0/2584136676 >> [v2:172.21.15.18:6800/3048894687,v1:172.21.15.18:6801/3048894687] conn(0x7f20980692b0 0x7f209806b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T20:50:57.608 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:57.223+0000 7f20aa000700 1 --2- 172.21.15.18:0/2584136676 >> [v2:172.21.15.18:6818/3573247809,v1:172.21.15.18:6819/3573247809] conn(0x7f209806ef80 0x7f2098071390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T20:50:57.608 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:57.224+0000 7f20aa000700 1 -- 172.21.15.18:0/2584136676 --> [v2:172.21.15.18:6818/3573247809,v1:172.21.15.18:6819/3573247809] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f2098071a40 con 0x7f209806ef80 2024-04-12T20:50:57.608 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:57.224+0000 7f20aa000700 1 -- 172.21.15.18:0/2584136676 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7f20a4089410 con 0x7f20b40702e0 2024-04-12T20:50:57.608 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:57.224+0000 7f20b2800700 1 --2- 172.21.15.18:0/2584136676 >> [v2:172.21.15.18:6818/3573247809,v1:172.21.15.18:6819/3573247809] conn(0x7f209806ef80 0x7f2098071390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T20:50:57.609 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:57.224+0000 7f20b1400700 1 --2- 172.21.15.18:0/2584136676 >> [v2:172.21.15.18:6800/3048894687,v1:172.21.15.18:6801/3048894687] conn(0x7f20980692b0 0x7f209806b760 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f20a000b020 tx=0x7f20a0009e30 comp rx=0 tx=0).ready entity=mgr.14212 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T20:50:57.609 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:57.225+0000 7f20b2800700 1 --2- 172.21.15.18:0/2584136676 >> [v2:172.21.15.18:6818/3573247809,v1:172.21.15.18:6819/3573247809] conn(0x7f209806ef80 0x7f2098071390 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T20:50:57.609 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:57.225+0000 7f20aa000700 1 -- 172.21.15.18:0/2584136676 <== osd.2 v2:172.21.15.18:6818/3573247809 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+24147 (crc 0 0 0) 0x7f2098071a40 con 0x7f209806ef80 2024-04-12T20:50:57.609 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:57.243+0000 7f20b8c00700 1 -- 172.21.15.18:0/2584136676 --> [v2:172.21.15.18:6818/3573247809,v1:172.21.15.18:6819/3573247809] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f20b4071a20 con 0x7f209806ef80 2024-04-12T20:50:57.609 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:57.243+0000 7f20aa000700 1 -- 172.21.15.18:0/2584136676 <== osd.2 v2:172.21.15.18:6818/3573247809 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f20b4071a20 con 0x7f209806ef80 2024-04-12T20:50:57.609 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:57.243+0000 7f20b8c00700 1 -- 172.21.15.18:0/2584136676 >> [v2:172.21.15.18:6818/3573247809,v1:172.21.15.18:6819/3573247809] conn(0x7f209806ef80 msgr2=0x7f2098071390 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:50:57.610 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:57.244+0000 7f20b8c00700 1 --2- 172.21.15.18:0/2584136676 >> [v2:172.21.15.18:6818/3573247809,v1:172.21.15.18:6819/3573247809] conn(0x7f209806ef80 0x7f2098071390 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:50:57.610 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:57.244+0000 7f20b8c00700 1 -- 172.21.15.18:0/2584136676 >> [v2:172.21.15.18:6800/3048894687,v1:172.21.15.18:6801/3048894687] conn(0x7f20980692b0 msgr2=0x7f209806b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:50:57.610 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:57.244+0000 7f20b8c00700 1 --2- 172.21.15.18:0/2584136676 >> [v2:172.21.15.18:6800/3048894687,v1:172.21.15.18:6801/3048894687] conn(0x7f20980692b0 0x7f209806b760 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f20a000b020 tx=0x7f20a0009e30 comp rx=0 tx=0).stop 2024-04-12T20:50:57.610 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:57.244+0000 7f20b8c00700 1 -- 172.21.15.18:0/2584136676 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f20b40702e0 msgr2=0x7f20b4115e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:50:57.610 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:57.244+0000 7f20b8c00700 1 --2- 172.21.15.18:0/2584136676 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f20b40702e0 0x7f20b4115e30 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f20a400c830 tx=0x7f20a4004470 comp rx=0 tx=0).stop 2024-04-12T20:50:57.610 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:57.244+0000 7f20b8c00700 1 -- 172.21.15.18:0/2584136676 shutdown_connections 2024-04-12T20:50:57.610 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:57.244+0000 7f20b8c00700 1 --2- 172.21.15.18:0/2584136676 >> [v2:172.21.15.18:6818/3573247809,v1:172.21.15.18:6819/3573247809] conn(0x7f209806ef80 0x7f2098071390 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:50:57.610 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:57.244+0000 7f20b8c00700 1 --2- 172.21.15.18:0/2584136676 >> [v2:172.21.15.18:6800/3048894687,v1:172.21.15.18:6801/3048894687] conn(0x7f20980692b0 0x7f209806b760 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:50:57.610 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:57.244+0000 7f20b8c00700 1 --2- 172.21.15.18:0/2584136676 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f20b40702e0 0x7f20b4115e30 secure :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f20a400c830 tx=0x7f20a4004470 comp rx=0 tx=0).stop 2024-04-12T20:50:57.611 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:57.244+0000 7f20b8c00700 1 --2- 172.21.15.18:0/2584136676 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f20b4070bf0 0x7f20b4116370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:50:57.611 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:57.244+0000 7f20b8c00700 1 -- 172.21.15.18:0/2584136676 >> 172.21.15.18:0/2584136676 conn(0x7f20b40f8970 msgr2=0x7f20b40f9ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T20:50:57.611 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:57.244+0000 7f20b8c00700 1 -- 172.21.15.18:0/2584136676 shutdown_connections 2024-04-12T20:50:57.611 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:57.244+0000 7f20b8c00700 1 -- 172.21.15.18:0/2584136676 wait complete. 2024-04-12T20:50:58.157 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:50:58 smithi018 ceph-mon[28799]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-12T20:50:58.335 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:50:58 smithi186 ceph-mon[36597]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-12T20:50:58.342 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:58.190+0000 7f7ca4600700 1 -- 172.21.15.18:0/3614169825 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7c9c0fe7b0 msgr2=0x7f7c9c106df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:50:58.342 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:58.190+0000 7f7ca4600700 1 --2- 172.21.15.18:0/3614169825 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7c9c0fe7b0 0x7f7c9c106df0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f7c88004940 tx=0x7f7c88009e30 comp rx=0 tx=0).stop 2024-04-12T20:50:58.342 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:58.190+0000 7f7ca4600700 1 -- 172.21.15.18:0/3614169825 shutdown_connections 2024-04-12T20:50:58.342 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:58.190+0000 7f7ca4600700 1 --2- 172.21.15.18:0/3614169825 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7c9c0fe7b0 0x7f7c9c106df0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:50:58.342 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:58.190+0000 7f7ca4600700 1 --2- 172.21.15.18:0/3614169825 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7c9c0fde10 0x7f7c9c0fe1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:50:58.342 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:58.190+0000 7f7ca4600700 1 -- 172.21.15.18:0/3614169825 >> 172.21.15.18:0/3614169825 conn(0x7f7c9c0f7880 msgr2=0x7f7c9c0f9c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T20:50:58.342 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:58.190+0000 7f7ca4600700 1 -- 172.21.15.18:0/3614169825 shutdown_connections 2024-04-12T20:50:58.342 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:58.190+0000 7f7ca4600700 1 -- 172.21.15.18:0/3614169825 wait complete. 2024-04-12T20:50:58.343 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:58.191+0000 7f7ca4600700 1 Processor -- start 2024-04-12T20:50:58.343 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:58.191+0000 7f7ca4600700 1 -- start start 2024-04-12T20:50:58.343 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:58.192+0000 7f7ca4600700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7c9c0fde10 0x7f7c9c0fbc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T20:50:58.343 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:58.192+0000 7f7ca4600700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7c9c0fe7b0 0x7f7c9c0fc180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T20:50:58.343 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:58.192+0000 7f7ca4600700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c9c0fc6c0 con 0x7f7c9c0fde10 2024-04-12T20:50:58.343 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:58.192+0000 7f7ca4600700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c9c0fc800 con 0x7f7c9c0fe7b0 2024-04-12T20:50:58.343 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:58.192+0000 7f7ca1c00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7c9c0fde10 0x7f7c9c0fbc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T20:50:58.343 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:58.192+0000 7f7ca1c00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7c9c0fde10 0x7f7c9c0fbc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:50482/0 (socket says 172.21.15.18:50482) 2024-04-12T20:50:58.343 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:58.192+0000 7f7ca1c00700 1 -- 172.21.15.18:0/3335170791 learned_addr learned my addr 172.21.15.18:0/3335170791 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T20:50:58.343 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:58.192+0000 7f7ca1200700 1 --2- 172.21.15.18:0/3335170791 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7c9c0fe7b0 0x7f7c9c0fc180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T20:50:58.343 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:58.193+0000 7f7ca1c00700 1 -- 172.21.15.18:0/3335170791 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7c9c0fe7b0 msgr2=0x7f7c9c0fc180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:50:58.343 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:58.193+0000 7f7ca1c00700 1 --2- 172.21.15.18:0/3335170791 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7c9c0fe7b0 0x7f7c9c0fc180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:50:58.343 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:58.193+0000 7f7ca1c00700 1 -- 172.21.15.18:0/3335170791 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c88005040 con 0x7f7c9c0fde10 2024-04-12T20:50:58.343 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:58.193+0000 7f7ca1c00700 1 --2- 172.21.15.18:0/3335170791 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7c9c0fde10 0x7f7c9c0fbc40 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f7c90002a50 tx=0x7f7c900046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T20:50:58.344 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:58.193+0000 7f7c99600700 1 -- 172.21.15.18:0/3335170791 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7c90007cc0 con 0x7f7c9c0fde10 2024-04-12T20:50:58.344 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:58.193+0000 7f7c99600700 1 -- 172.21.15.18:0/3335170791 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f7c9000f040 con 0x7f7c9c0fde10 2024-04-12T20:50:58.344 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:58.193+0000 7f7c99600700 1 -- 172.21.15.18:0/3335170791 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7c9000db70 con 0x7f7c9c0fde10 2024-04-12T20:50:58.344 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:58.194+0000 7f7ca4600700 1 -- 172.21.15.18:0/3335170791 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7c9c0fcab0 con 0x7f7c9c0fde10 2024-04-12T20:50:58.344 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:58.194+0000 7f7ca4600700 1 -- 172.21.15.18:0/3335170791 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7c9c11a590 con 0x7f7c9c0fde10 2024-04-12T20:50:58.344 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:58.195+0000 7f7ca4600700 1 -- 172.21.15.18:0/3335170791 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f7c9c066770 con 0x7f7c9c0fde10 2024-04-12T20:50:58.344 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:58.196+0000 7f7c99600700 1 -- 172.21.15.18:0/3335170791 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7f7c9000dd10 con 0x7f7c9c0fde10 2024-04-12T20:50:58.344 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:58.197+0000 7f7c99600700 1 --2- 172.21.15.18:0/3335170791 >> [v2:172.21.15.18:6800/3048894687,v1:172.21.15.18:6801/3048894687] conn(0x7f7c840692b0 0x7f7c8406b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T20:50:58.344 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:58.197+0000 7f7c99600700 1 -- 172.21.15.18:0/3335170791 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4634+0+0 (secure 0 0 0) 0x7f7c900863c0 con 0x7f7c9c0fde10 2024-04-12T20:50:58.344 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:58.197+0000 7f7ca1200700 1 --2- 172.21.15.18:0/3335170791 >> [v2:172.21.15.18:6800/3048894687,v1:172.21.15.18:6801/3048894687] conn(0x7f7c840692b0 0x7f7c8406b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T20:50:58.344 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:58.197+0000 7f7c99600700 1 --2- 172.21.15.18:0/3335170791 >> [v2:172.21.15.186:6816/3543055150,v1:172.21.15.186:6817/3543055150] conn(0x7f7c8406ef80 0x7f7c84071390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T20:50:58.344 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:58.197+0000 7f7c99600700 1 -- 172.21.15.18:0/3335170791 --> [v2:172.21.15.186:6816/3543055150,v1:172.21.15.186:6817/3543055150] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f7c84071a40 con 0x7f7c8406ef80 2024-04-12T20:50:58.344 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:58.197+0000 7f7c99600700 1 -- 172.21.15.18:0/3335170791 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7c90088970 con 0x7f7c9c0fde10 2024-04-12T20:50:58.344 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:58.198+0000 7f7ca1200700 1 --2- 172.21.15.18:0/3335170791 >> [v2:172.21.15.18:6800/3048894687,v1:172.21.15.18:6801/3048894687] conn(0x7f7c840692b0 0x7f7c8406b760 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f7c9c0fd530 tx=0x7f7c88009e30 comp rx=0 tx=0).ready entity=mgr.14212 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T20:50:58.345 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:58.198+0000 7f7ca2600700 1 --2- 172.21.15.18:0/3335170791 >> [v2:172.21.15.186:6816/3543055150,v1:172.21.15.186:6817/3543055150] conn(0x7f7c8406ef80 0x7f7c84071390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T20:50:58.345 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:58.199+0000 7f7ca2600700 1 --2- 172.21.15.18:0/3335170791 >> [v2:172.21.15.186:6816/3543055150,v1:172.21.15.186:6817/3543055150] conn(0x7f7c8406ef80 0x7f7c84071390 crc :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T20:50:58.345 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:58.201+0000 7f7c99600700 1 -- 172.21.15.18:0/3335170791 <== osd.5 v2:172.21.15.186:6816/3543055150 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+24147 (crc 0 0 0) 0x7f7c84071a40 con 0x7f7c8406ef80 2024-04-12T20:50:58.345 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:58.218+0000 7f7ca4600700 1 -- 172.21.15.18:0/3335170791 --> [v2:172.21.15.186:6816/3543055150,v1:172.21.15.186:6817/3543055150] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f7c9c0fca50 con 0x7f7c8406ef80 2024-04-12T20:50:58.345 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:58.219+0000 7f7c99600700 1 -- 172.21.15.18:0/3335170791 <== osd.5 v2:172.21.15.186:6816/3543055150 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f7c9c0fca50 con 0x7f7c8406ef80 2024-04-12T20:50:58.345 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:58.219+0000 7f7ca4600700 1 -- 172.21.15.18:0/3335170791 >> [v2:172.21.15.186:6816/3543055150,v1:172.21.15.186:6817/3543055150] conn(0x7f7c8406ef80 msgr2=0x7f7c84071390 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:50:58.346 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:58.219+0000 7f7ca4600700 1 --2- 172.21.15.18:0/3335170791 >> [v2:172.21.15.186:6816/3543055150,v1:172.21.15.186:6817/3543055150] conn(0x7f7c8406ef80 0x7f7c84071390 crc :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:50:58.346 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:58.219+0000 7f7ca4600700 1 -- 172.21.15.18:0/3335170791 >> [v2:172.21.15.18:6800/3048894687,v1:172.21.15.18:6801/3048894687] conn(0x7f7c840692b0 msgr2=0x7f7c8406b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:50:58.346 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:58.219+0000 7f7ca4600700 1 --2- 172.21.15.18:0/3335170791 >> [v2:172.21.15.18:6800/3048894687,v1:172.21.15.18:6801/3048894687] conn(0x7f7c840692b0 0x7f7c8406b760 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f7c9c0fd530 tx=0x7f7c88009e30 comp rx=0 tx=0).stop 2024-04-12T20:50:58.346 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:58.219+0000 7f7ca4600700 1 -- 172.21.15.18:0/3335170791 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7c9c0fde10 msgr2=0x7f7c9c0fbc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:50:58.346 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:58.219+0000 7f7ca4600700 1 --2- 172.21.15.18:0/3335170791 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7c9c0fde10 0x7f7c9c0fbc40 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f7c90002a50 tx=0x7f7c900046b0 comp rx=0 tx=0).stop 2024-04-12T20:50:58.346 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:58.219+0000 7f7ca4600700 1 -- 172.21.15.18:0/3335170791 shutdown_connections 2024-04-12T20:50:58.346 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:58.219+0000 7f7ca4600700 1 --2- 172.21.15.18:0/3335170791 >> [v2:172.21.15.186:6816/3543055150,v1:172.21.15.186:6817/3543055150] conn(0x7f7c8406ef80 0x7f7c84071390 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:50:58.346 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:58.219+0000 7f7ca4600700 1 --2- 172.21.15.18:0/3335170791 >> [v2:172.21.15.18:6800/3048894687,v1:172.21.15.18:6801/3048894687] conn(0x7f7c840692b0 0x7f7c8406b760 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:50:58.347 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:58.219+0000 7f7ca4600700 1 --2- 172.21.15.18:0/3335170791 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7c9c0fde10 0x7f7c9c0fbc40 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:50:58.347 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:58.219+0000 7f7ca4600700 1 --2- 172.21.15.18:0/3335170791 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7c9c0fe7b0 0x7f7c9c0fc180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:50:58.347 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:58.219+0000 7f7ca4600700 1 -- 172.21.15.18:0/3335170791 >> 172.21.15.18:0/3335170791 conn(0x7f7c9c0f7880 msgr2=0x7f7c9c0f8ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T20:50:58.347 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:58.220+0000 7f7ca4600700 1 -- 172.21.15.18:0/3335170791 shutdown_connections 2024-04-12T20:50:58.347 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:58.220+0000 7f7ca4600700 1 -- 172.21.15.18:0/3335170791 wait complete. 2024-04-12T20:50:59.284 INFO:teuthology.orchestra.run.smithi018.stdout:60129542180 2024-04-12T20:50:59.285 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid 1cbe407a-f90d-11ee-bc8d-c7b262605968 -- ceph osd last-stat-seq osd.1 2024-04-12T20:50:59.860 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:59.861+0000 7faa7d200700 1 -- 172.21.15.18:0/3506010949 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7faa78102e10 msgr2=0x7faa781031e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:50:59.860 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:59.861+0000 7faa7d200700 1 --2- 172.21.15.18:0/3506010949 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7faa78102e10 0x7faa781031e0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7faa68004940 tx=0x7faa68009e30 comp rx=0 tx=0).stop 2024-04-12T20:50:59.860 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:59.861+0000 7faa7d200700 1 -- 172.21.15.18:0/3506010949 shutdown_connections 2024-04-12T20:50:59.860 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:59.861+0000 7faa7d200700 1 --2- 172.21.15.18:0/3506010949 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7faa781037b0 0x7faa780fce20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:50:59.860 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:59.861+0000 7faa7d200700 1 --2- 172.21.15.18:0/3506010949 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7faa78102e10 0x7faa781031e0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:50:59.860 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:59.861+0000 7faa7d200700 1 -- 172.21.15.18:0/3506010949 >> 172.21.15.18:0/3506010949 conn(0x7faa780f8980 msgr2=0x7faa780fad90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T20:50:59.860 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:59.861+0000 7faa7d200700 1 -- 172.21.15.18:0/3506010949 shutdown_connections 2024-04-12T20:50:59.861 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:59.862+0000 7faa7d200700 1 -- 172.21.15.18:0/3506010949 wait complete. 2024-04-12T20:50:59.862 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:59.862+0000 7faa7d200700 1 Processor -- start 2024-04-12T20:50:59.862 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:59.863+0000 7faa7d200700 1 -- start start 2024-04-12T20:50:59.862 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:59.863+0000 7faa7d200700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7faa78102e10 0x7faa78115dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T20:50:59.862 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:59.863+0000 7faa7d200700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7faa781037b0 0x7faa78116300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T20:50:59.862 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:59.863+0000 7faa7d200700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa78116950 con 0x7faa78102e10 2024-04-12T20:50:59.862 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:59.863+0000 7faa7d200700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa7810fd70 con 0x7faa781037b0 2024-04-12T20:50:59.862 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:59.863+0000 7faa75e00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7faa78102e10 0x7faa78115dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T20:50:59.863 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:59.863+0000 7faa75e00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7faa78102e10 0x7faa78115dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:50502/0 (socket says 172.21.15.18:50502) 2024-04-12T20:50:59.863 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:59.863+0000 7faa75e00700 1 -- 172.21.15.18:0/916547452 learned_addr learned my addr 172.21.15.18:0/916547452 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T20:50:59.863 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:59.864+0000 7faa6fe00700 1 --2- 172.21.15.18:0/916547452 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7faa781037b0 0x7faa78116300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T20:50:59.863 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:59.864+0000 7faa75e00700 1 -- 172.21.15.18:0/916547452 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7faa781037b0 msgr2=0x7faa78116300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:50:59.863 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:59.864+0000 7faa75e00700 1 --2- 172.21.15.18:0/916547452 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7faa781037b0 0x7faa78116300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:50:59.863 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:59.864+0000 7faa75e00700 1 -- 172.21.15.18:0/916547452 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa68005040 con 0x7faa78102e10 2024-04-12T20:50:59.863 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:59.864+0000 7faa6fe00700 1 --2- 172.21.15.18:0/916547452 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7faa781037b0 0x7faa78116300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-12T20:50:59.863 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:59.864+0000 7faa75e00700 1 --2- 172.21.15.18:0/916547452 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7faa78102e10 0x7faa78115dc0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7faa68009180 tx=0x7faa68009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T20:50:59.864 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:59.864+0000 7faa6e000700 1 -- 172.21.15.18:0/916547452 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7faa6800e040 con 0x7faa78102e10 2024-04-12T20:50:59.864 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:59.865+0000 7faa6e000700 1 -- 172.21.15.18:0/916547452 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7faa6800bbc0 con 0x7faa78102e10 2024-04-12T20:50:59.864 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:59.865+0000 7faa6e000700 1 -- 172.21.15.18:0/916547452 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7faa6801c630 con 0x7faa78102e10 2024-04-12T20:50:59.864 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:59.865+0000 7faa7d200700 1 -- 172.21.15.18:0/916547452 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faa7810fff0 con 0x7faa78102e10 2024-04-12T20:50:59.864 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:59.865+0000 7faa7d200700 1 -- 172.21.15.18:0/916547452 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faa781105b0 con 0x7faa78102e10 2024-04-12T20:50:59.864 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:59.866+0000 7faa7d200700 1 -- 172.21.15.18:0/916547452 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7faa78110ba0 con 0x7faa78102e10 2024-04-12T20:50:59.865 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:59.867+0000 7faa6e000700 1 -- 172.21.15.18:0/916547452 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7faa6800bd30 con 0x7faa78102e10 2024-04-12T20:50:59.866 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:59.868+0000 7faa6e000700 1 --2- 172.21.15.18:0/916547452 >> [v2:172.21.15.18:6800/3048894687,v1:172.21.15.18:6801/3048894687] conn(0x7faa5c0692b0 0x7faa5c06b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T20:50:59.866 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:59.868+0000 7faa6e000700 1 -- 172.21.15.18:0/916547452 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4634+0+0 (secure 0 0 0) 0x7faa68088210 con 0x7faa78102e10 2024-04-12T20:50:59.866 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:59.868+0000 7faa6fe00700 1 --2- 172.21.15.18:0/916547452 >> [v2:172.21.15.18:6800/3048894687,v1:172.21.15.18:6801/3048894687] conn(0x7faa5c0692b0 0x7faa5c06b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T20:50:59.867 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:59.868+0000 7faa6e000700 1 --2- 172.21.15.18:0/916547452 >> [v2:172.21.15.186:6808/3584843287,v1:172.21.15.186:6809/3584843287] conn(0x7faa5c06ef80 0x7faa5c071390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T20:50:59.867 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:59.868+0000 7faa6e000700 1 -- 172.21.15.18:0/916547452 --> [v2:172.21.15.186:6808/3584843287,v1:172.21.15.186:6809/3584843287] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7faa5c071a40 con 0x7faa5c06ef80 2024-04-12T20:50:59.867 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:59.868+0000 7faa6e000700 1 -- 172.21.15.18:0/916547452 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7faa6808a760 con 0x7faa78102e10 2024-04-12T20:50:59.867 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:59.869+0000 7faa6fe00700 1 --2- 172.21.15.18:0/916547452 >> [v2:172.21.15.18:6800/3048894687,v1:172.21.15.18:6801/3048894687] conn(0x7faa5c0692b0 0x7faa5c06b760 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7faa60009fd0 tx=0x7faa60009ee0 comp rx=0 tx=0).ready entity=mgr.14212 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T20:50:59.867 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:59.869+0000 7faa76800700 1 --2- 172.21.15.18:0/916547452 >> [v2:172.21.15.186:6808/3584843287,v1:172.21.15.186:6809/3584843287] conn(0x7faa5c06ef80 0x7faa5c071390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T20:50:59.867 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:59.870+0000 7faa76800700 1 --2- 172.21.15.18:0/916547452 >> [v2:172.21.15.186:6808/3584843287,v1:172.21.15.186:6809/3584843287] conn(0x7faa5c06ef80 0x7faa5c071390 crc :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T20:50:59.870 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:59.872+0000 7faa6e000700 1 -- 172.21.15.18:0/916547452 <== osd.4 v2:172.21.15.186:6808/3584843287 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+24147 (crc 0 0 0) 0x7faa5c071a40 con 0x7faa5c06ef80 2024-04-12T20:50:59.887 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:59.889+0000 7faa7d200700 1 -- 172.21.15.18:0/916547452 --> [v2:172.21.15.186:6808/3584843287,v1:172.21.15.186:6809/3584843287] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7faa78110180 con 0x7faa5c06ef80 2024-04-12T20:50:59.887 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:59.889+0000 7faa6e000700 1 -- 172.21.15.18:0/916547452 <== osd.4 v2:172.21.15.186:6808/3584843287 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7faa78110180 con 0x7faa5c06ef80 2024-04-12T20:50:59.888 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:59.890+0000 7faa7d200700 1 -- 172.21.15.18:0/916547452 >> [v2:172.21.15.186:6808/3584843287,v1:172.21.15.186:6809/3584843287] conn(0x7faa5c06ef80 msgr2=0x7faa5c071390 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:50:59.888 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:59.890+0000 7faa7d200700 1 --2- 172.21.15.18:0/916547452 >> [v2:172.21.15.186:6808/3584843287,v1:172.21.15.186:6809/3584843287] conn(0x7faa5c06ef80 0x7faa5c071390 crc :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:50:59.888 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:59.890+0000 7faa7d200700 1 -- 172.21.15.18:0/916547452 >> [v2:172.21.15.18:6800/3048894687,v1:172.21.15.18:6801/3048894687] conn(0x7faa5c0692b0 msgr2=0x7faa5c06b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:50:59.889 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:59.890+0000 7faa7d200700 1 --2- 172.21.15.18:0/916547452 >> [v2:172.21.15.18:6800/3048894687,v1:172.21.15.18:6801/3048894687] conn(0x7faa5c0692b0 0x7faa5c06b760 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7faa60009fd0 tx=0x7faa60009ee0 comp rx=0 tx=0).stop 2024-04-12T20:50:59.889 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:59.890+0000 7faa7d200700 1 -- 172.21.15.18:0/916547452 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7faa78102e10 msgr2=0x7faa78115dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:50:59.889 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:59.890+0000 7faa7d200700 1 --2- 172.21.15.18:0/916547452 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7faa78102e10 0x7faa78115dc0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7faa68009180 tx=0x7faa68009e30 comp rx=0 tx=0).stop 2024-04-12T20:50:59.889 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:59.890+0000 7faa7d200700 1 -- 172.21.15.18:0/916547452 shutdown_connections 2024-04-12T20:50:59.889 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:59.890+0000 7faa7d200700 1 --2- 172.21.15.18:0/916547452 >> [v2:172.21.15.186:6808/3584843287,v1:172.21.15.186:6809/3584843287] conn(0x7faa5c06ef80 0x7faa5c071390 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:50:59.889 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:59.890+0000 7faa7d200700 1 --2- 172.21.15.18:0/916547452 >> [v2:172.21.15.18:6800/3048894687,v1:172.21.15.18:6801/3048894687] conn(0x7faa5c0692b0 0x7faa5c06b760 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:50:59.889 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:59.890+0000 7faa7d200700 1 --2- 172.21.15.18:0/916547452 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7faa78102e10 0x7faa78115dc0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:50:59.890 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:59.890+0000 7faa7d200700 1 --2- 172.21.15.18:0/916547452 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7faa781037b0 0x7faa78116300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:50:59.890 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:59.890+0000 7faa7d200700 1 -- 172.21.15.18:0/916547452 >> 172.21.15.18:0/916547452 conn(0x7faa780f8980 msgr2=0x7faa78107400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T20:50:59.890 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:59.891+0000 7faa7d200700 1 -- 172.21.15.18:0/916547452 shutdown_connections 2024-04-12T20:50:59.890 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:50:59.891+0000 7faa7d200700 1 -- 172.21.15.18:0/916547452 wait complete. 2024-04-12T20:51:00.282 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:51:00 smithi186 ceph-mon[36597]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-12T20:51:00.282 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:51:00 smithi186 ceph-mon[36597]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T20:51:00.301 INFO:teuthology.orchestra.run.smithi018.stdout:81604378654 2024-04-12T20:51:00.301 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid 1cbe407a-f90d-11ee-bc8d-c7b262605968 -- ceph osd last-stat-seq osd.2 2024-04-12T20:51:00.352 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:51:00 smithi018 ceph-mon[28799]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-12T20:51:00.352 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:51:00 smithi018 ceph-mon[28799]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T20:51:01.241 INFO:teuthology.orchestra.run.smithi018.stdout:167503724555 2024-04-12T20:51:01.241 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid 1cbe407a-f90d-11ee-bc8d-c7b262605968 -- ceph osd last-stat-seq osd.5 2024-04-12T20:51:01.797 INFO:teuthology.orchestra.run.smithi018.stdout:146028888082 2024-04-12T20:51:01.798 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid 1cbe407a-f90d-11ee-bc8d-c7b262605968 -- ceph osd last-stat-seq osd.4 2024-04-12T20:51:01.828 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:01.808+0000 7fc50e000700 1 -- 172.21.15.18:0/52092116 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc5080fbc70 msgr2=0x7fc5080fc040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:51:01.828 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:01.808+0000 7fc50e000700 1 --2- 172.21.15.18:0/52092116 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc5080fbc70 0x7fc5080fc040 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fc4f8004940 tx=0x7fc4f8009e30 comp rx=0 tx=0).stop 2024-04-12T20:51:01.828 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:01.808+0000 7fc50e000700 1 -- 172.21.15.18:0/52092116 shutdown_connections 2024-04-12T20:51:01.828 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:01.808+0000 7fc50e000700 1 --2- 172.21.15.18:0/52092116 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc5080fc610 0x7fc508100c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:51:01.828 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:01.808+0000 7fc50e000700 1 --2- 172.21.15.18:0/52092116 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc5080fbc70 0x7fc5080fc040 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:51:01.829 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:01.808+0000 7fc50e000700 1 -- 172.21.15.18:0/52092116 >> 172.21.15.18:0/52092116 conn(0x7fc5080f78f0 msgr2=0x7fc5080f9d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T20:51:01.829 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:01.809+0000 7fc50e000700 1 -- 172.21.15.18:0/52092116 shutdown_connections 2024-04-12T20:51:01.829 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:01.809+0000 7fc50e000700 1 -- 172.21.15.18:0/52092116 wait complete. 2024-04-12T20:51:01.829 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:01.809+0000 7fc50e000700 1 Processor -- start 2024-04-12T20:51:01.829 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:01.810+0000 7fc50e000700 1 -- start start 2024-04-12T20:51:01.829 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:01.810+0000 7fc50e000700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc5080fbc70 0x7fc508119f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T20:51:01.829 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:01.810+0000 7fc50e000700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc5080fc610 0x7fc50811a4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T20:51:01.829 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:01.810+0000 7fc50e000700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc50811ab10 con 0x7fc5080fc610 2024-04-12T20:51:01.829 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:01.810+0000 7fc50e000700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5081140f0 con 0x7fc5080fbc70 2024-04-12T20:51:01.829 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:01.810+0000 7fc506a00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc5080fc610 0x7fc50811a4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T20:51:01.829 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:01.810+0000 7fc506a00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc5080fc610 0x7fc50811a4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:49362/0 (socket says 172.21.15.18:49362) 2024-04-12T20:51:01.829 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:01.810+0000 7fc506a00700 1 -- 172.21.15.18:0/3399016506 learned_addr learned my addr 172.21.15.18:0/3399016506 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T20:51:01.829 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:01.811+0000 7fc507400700 1 --2- 172.21.15.18:0/3399016506 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc5080fbc70 0x7fc508119f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T20:51:01.830 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:01.811+0000 7fc506a00700 1 -- 172.21.15.18:0/3399016506 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc5080fbc70 msgr2=0x7fc508119f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:51:01.830 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:01.811+0000 7fc506a00700 1 --2- 172.21.15.18:0/3399016506 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc5080fbc70 0x7fc508119f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:51:01.830 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:01.811+0000 7fc506a00700 1 -- 172.21.15.18:0/3399016506 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4f8005040 con 0x7fc5080fc610 2024-04-12T20:51:01.830 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:01.811+0000 7fc507400700 1 --2- 172.21.15.18:0/3399016506 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc5080fbc70 0x7fc508119f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-12T20:51:01.830 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:01.811+0000 7fc506a00700 1 --2- 172.21.15.18:0/3399016506 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc5080fc610 0x7fc50811a4c0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fc4f400c830 tx=0x7fc4f4004470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T20:51:01.830 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:01.812+0000 7fc4ff400700 1 -- 172.21.15.18:0/3399016506 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc4f400a760 con 0x7fc5080fc610 2024-04-12T20:51:01.830 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:01.812+0000 7fc4ff400700 1 -- 172.21.15.18:0/3399016506 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc4f400ad90 con 0x7fc5080fc610 2024-04-12T20:51:01.830 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:01.812+0000 7fc50e000700 1 -- 172.21.15.18:0/3399016506 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc5081143d0 con 0x7fc5080fc610 2024-04-12T20:51:01.830 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:01.812+0000 7fc4ff400700 1 -- 172.21.15.18:0/3399016506 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc4f40113a0 con 0x7fc5080fc610 2024-04-12T20:51:01.830 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:01.812+0000 7fc50e000700 1 -- 172.21.15.18:0/3399016506 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc508114910 con 0x7fc5080fc610 2024-04-12T20:51:01.830 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:01.813+0000 7fc50e000700 1 -- 172.21.15.18:0/3399016506 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fc508114e50 con 0x7fc5080fc610 2024-04-12T20:51:01.830 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:01.814+0000 7fc4ff400700 1 -- 172.21.15.18:0/3399016506 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7fc4f4012400 con 0x7fc5080fc610 2024-04-12T20:51:01.830 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:01.815+0000 7fc4ff400700 1 --2- 172.21.15.18:0/3399016506 >> [v2:172.21.15.18:6800/3048894687,v1:172.21.15.18:6801/3048894687] conn(0x7fc4ec0692a0 0x7fc4ec06b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T20:51:01.831 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:01.815+0000 7fc4ff400700 1 -- 172.21.15.18:0/3399016506 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4634+0+0 (secure 0 0 0) 0x7fc4f4086f30 con 0x7fc5080fc610 2024-04-12T20:51:01.831 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:01.815+0000 7fc4ff400700 1 --2- 172.21.15.18:0/3399016506 >> [v2:172.21.15.186:6800/2184799172,v1:172.21.15.186:6801/2184799172] conn(0x7fc4ec06ef70 0x7fc4ec071380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T20:51:01.831 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:01.815+0000 7fc507400700 1 --2- 172.21.15.18:0/3399016506 >> [v2:172.21.15.18:6800/3048894687,v1:172.21.15.18:6801/3048894687] conn(0x7fc4ec0692a0 0x7fc4ec06b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T20:51:01.831 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:01.815+0000 7fc4ff400700 1 -- 172.21.15.18:0/3399016506 --> [v2:172.21.15.186:6800/2184799172,v1:172.21.15.186:6801/2184799172] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fc4ec071a30 con 0x7fc4ec06ef70 2024-04-12T20:51:01.831 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:01.815+0000 7fc4ff400700 1 -- 172.21.15.18:0/3399016506 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc4f408c4a0 con 0x7fc5080fc610 2024-04-12T20:51:01.831 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:01.816+0000 7fc507e00700 1 --2- 172.21.15.18:0/3399016506 >> [v2:172.21.15.186:6800/2184799172,v1:172.21.15.186:6801/2184799172] conn(0x7fc4ec06ef70 0x7fc4ec071380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T20:51:01.831 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:01.816+0000 7fc507400700 1 --2- 172.21.15.18:0/3399016506 >> [v2:172.21.15.18:6800/3048894687,v1:172.21.15.18:6801/3048894687] conn(0x7fc4ec0692a0 0x7fc4ec06b750 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fc4f8000c00 tx=0x7fc4f8009e30 comp rx=0 tx=0).ready entity=mgr.14212 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T20:51:01.831 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:01.817+0000 7fc507e00700 1 --2- 172.21.15.18:0/3399016506 >> [v2:172.21.15.186:6800/2184799172,v1:172.21.15.186:6801/2184799172] conn(0x7fc4ec06ef70 0x7fc4ec071380 crc :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T20:51:01.831 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:01.819+0000 7fc4ff400700 1 -- 172.21.15.18:0/3399016506 <== osd.3 v2:172.21.15.186:6800/2184799172 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+24147 (crc 0 0 0) 0x7fc4ec071a30 con 0x7fc4ec06ef70 2024-04-12T20:51:01.834 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:01.836+0000 7fc50e000700 1 -- 172.21.15.18:0/3399016506 --> [v2:172.21.15.186:6800/2184799172,v1:172.21.15.186:6801/2184799172] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fc50810b420 con 0x7fc4ec06ef70 2024-04-12T20:51:01.835 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:01.837+0000 7fc4ff400700 1 -- 172.21.15.18:0/3399016506 <== osd.3 v2:172.21.15.186:6800/2184799172 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7fc50810b420 con 0x7fc4ec06ef70 2024-04-12T20:51:01.836 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:01.838+0000 7fc50e000700 1 -- 172.21.15.18:0/3399016506 >> [v2:172.21.15.186:6800/2184799172,v1:172.21.15.186:6801/2184799172] conn(0x7fc4ec06ef70 msgr2=0x7fc4ec071380 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:51:01.836 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:01.838+0000 7fc50e000700 1 --2- 172.21.15.18:0/3399016506 >> [v2:172.21.15.186:6800/2184799172,v1:172.21.15.186:6801/2184799172] conn(0x7fc4ec06ef70 0x7fc4ec071380 crc :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:51:01.837 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:01.838+0000 7fc50e000700 1 -- 172.21.15.18:0/3399016506 >> [v2:172.21.15.18:6800/3048894687,v1:172.21.15.18:6801/3048894687] conn(0x7fc4ec0692a0 msgr2=0x7fc4ec06b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:51:01.837 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:01.838+0000 7fc50e000700 1 --2- 172.21.15.18:0/3399016506 >> [v2:172.21.15.18:6800/3048894687,v1:172.21.15.18:6801/3048894687] conn(0x7fc4ec0692a0 0x7fc4ec06b750 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fc4f8000c00 tx=0x7fc4f8009e30 comp rx=0 tx=0).stop 2024-04-12T20:51:01.837 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:01.838+0000 7fc50e000700 1 -- 172.21.15.18:0/3399016506 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc5080fc610 msgr2=0x7fc50811a4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:51:01.837 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:01.838+0000 7fc50e000700 1 --2- 172.21.15.18:0/3399016506 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc5080fc610 0x7fc50811a4c0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fc4f400c830 tx=0x7fc4f4004470 comp rx=0 tx=0).stop 2024-04-12T20:51:01.837 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:01.838+0000 7fc50e000700 1 -- 172.21.15.18:0/3399016506 shutdown_connections 2024-04-12T20:51:01.837 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:01.838+0000 7fc50e000700 1 --2- 172.21.15.18:0/3399016506 >> [v2:172.21.15.186:6800/2184799172,v1:172.21.15.186:6801/2184799172] conn(0x7fc4ec06ef70 0x7fc4ec071380 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:51:01.838 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:01.838+0000 7fc50e000700 1 --2- 172.21.15.18:0/3399016506 >> [v2:172.21.15.18:6800/3048894687,v1:172.21.15.18:6801/3048894687] conn(0x7fc4ec0692a0 0x7fc4ec06b750 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:51:01.838 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:01.838+0000 7fc50e000700 1 --2- 172.21.15.18:0/3399016506 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc5080fbc70 0x7fc508119f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:51:01.838 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:01.838+0000 7fc50e000700 1 --2- 172.21.15.18:0/3399016506 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc5080fc610 0x7fc50811a4c0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:51:01.838 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:01.838+0000 7fc50e000700 1 -- 172.21.15.18:0/3399016506 >> 172.21.15.18:0/3399016506 conn(0x7fc5080f78f0 msgr2=0x7fc50810aa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T20:51:01.838 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:01.839+0000 7fc50e000700 1 -- 172.21.15.18:0/3399016506 shutdown_connections 2024-04-12T20:51:01.838 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:01.839+0000 7fc50e000700 1 -- 172.21.15.18:0/3399016506 wait complete. 2024-04-12T20:51:02.342 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:51:02 smithi018 ceph-mon[28799]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-12T20:51:02.531 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:51:02 smithi186 ceph-mon[36597]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-12T20:51:02.647 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/mon.smithi018/config 2024-04-12T20:51:02.653 INFO:teuthology.orchestra.run.smithi018.stdout:111669149721 2024-04-12T20:51:02.653 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid 1cbe407a-f90d-11ee-bc8d-c7b262605968 -- ceph osd last-stat-seq osd.3 2024-04-12T20:51:02.656 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/mon.smithi018/config 2024-04-12T20:51:02.760 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/mon.smithi018/config 2024-04-12T20:51:03.396 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/mon.smithi018/config 2024-04-12T20:51:04.102 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:51:04 smithi018 ceph-mon[28799]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-12T20:51:04.531 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:51:04 smithi186 ceph-mon[36597]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-12T20:51:05.068 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/mon.smithi018/config 2024-04-12T20:51:06.353 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:51:06 smithi018 ceph-mon[28799]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-12T20:51:06.531 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:51:06 smithi186 ceph-mon[36597]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-12T20:51:07.957 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:07.957+0000 7f304f800700 1 -- 172.21.15.18:0/1692455951 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f30480fe830 msgr2=0x7f3048106f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:51:07.957 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:07.957+0000 7f304f800700 1 --2- 172.21.15.18:0/1692455951 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f30480fe830 0x7f3048106f00 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f3030004940 tx=0x7f3030009e30 comp rx=0 tx=0).stop 2024-04-12T20:51:07.957 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:07.957+0000 7f304f800700 1 -- 172.21.15.18:0/1692455951 shutdown_connections 2024-04-12T20:51:07.957 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:07.957+0000 7f304f800700 1 --2- 172.21.15.18:0/1692455951 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f30480fe830 0x7f3048106f00 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:51:07.957 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:07.957+0000 7f304f800700 1 --2- 172.21.15.18:0/1692455951 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f30480fde90 0x7f30480fe260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:51:07.957 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:07.957+0000 7f304f800700 1 -- 172.21.15.18:0/1692455951 >> 172.21.15.18:0/1692455951 conn(0x7f3048072bc0 msgr2=0x7f3048072fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T20:51:07.957 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:07.957+0000 7f304f800700 1 -- 172.21.15.18:0/1692455951 shutdown_connections 2024-04-12T20:51:07.957 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:07.957+0000 7f304f800700 1 -- 172.21.15.18:0/1692455951 wait complete. 2024-04-12T20:51:07.957 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:07.958+0000 7f304f800700 1 Processor -- start 2024-04-12T20:51:07.958 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:07.958+0000 7f304f800700 1 -- start start 2024-04-12T20:51:07.958 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:07.958+0000 7f304f800700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f30480fde90 0x7f3048115da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T20:51:07.958 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:07.959+0000 7f304f800700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f30480fe830 0x7f30481162e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T20:51:07.958 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:07.959+0000 7f304f800700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3048116930 con 0x7f30480fe830 2024-04-12T20:51:07.958 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:07.959+0000 7f304f800700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f304810fd00 con 0x7f30480fde90 2024-04-12T20:51:07.958 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:07.959+0000 7f303fe00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f30480fe830 0x7f30481162e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T20:51:07.958 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:07.959+0000 7f303fe00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f30480fe830 0x7f30481162e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:49376/0 (socket says 172.21.15.18:49376) 2024-04-12T20:51:07.958 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:07.959+0000 7f303fe00700 1 -- 172.21.15.18:0/1765610666 learned_addr learned my addr 172.21.15.18:0/1765610666 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T20:51:07.959 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:07.959+0000 7f304ce00700 1 --2- 172.21.15.18:0/1765610666 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f30480fde90 0x7f3048115da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T20:51:07.959 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:07.959+0000 7f303fe00700 1 -- 172.21.15.18:0/1765610666 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f30480fde90 msgr2=0x7f3048115da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:51:07.959 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:07.959+0000 7f303fe00700 1 --2- 172.21.15.18:0/1765610666 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f30480fde90 0x7f3048115da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:51:07.959 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:07.959+0000 7f303fe00700 1 -- 172.21.15.18:0/1765610666 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3030005040 con 0x7f30480fe830 2024-04-12T20:51:07.959 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:07.960+0000 7f303fe00700 1 --2- 172.21.15.18:0/1765610666 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f30480fe830 0x7f30481162e0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f3030000c00 tx=0x7f3030009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T20:51:07.960 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:07.960+0000 7f3045600700 1 -- 172.21.15.18:0/1765610666 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f303000e040 con 0x7f30480fe830 2024-04-12T20:51:07.960 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:07.960+0000 7f3045600700 1 -- 172.21.15.18:0/1765610666 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f3030013420 con 0x7f30480fe830 2024-04-12T20:51:07.960 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:07.960+0000 7f304f800700 1 -- 172.21.15.18:0/1765610666 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f304810ff80 con 0x7f30480fe830 2024-04-12T20:51:07.960 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:07.960+0000 7f3045600700 1 -- 172.21.15.18:0/1765610666 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f303001c610 con 0x7f30480fe830 2024-04-12T20:51:07.960 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:07.960+0000 7f304f800700 1 -- 172.21.15.18:0/1765610666 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f30481104c0 con 0x7f30480fe830 2024-04-12T20:51:07.960 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:07.961+0000 7f304f800700 1 -- 172.21.15.18:0/1765610666 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3048106a20 con 0x7f30480fe830 2024-04-12T20:51:07.960 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:07.962+0000 7f3045600700 1 -- 172.21.15.18:0/1765610666 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7f3030013590 con 0x7f30480fe830 2024-04-12T20:51:07.961 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:07.963+0000 7f3045600700 1 --2- 172.21.15.18:0/1765610666 >> [v2:172.21.15.18:6800/3048894687,v1:172.21.15.18:6801/3048894687] conn(0x7f302c0692b0 0x7f302c06b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T20:51:07.961 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:07.963+0000 7f3045600700 1 -- 172.21.15.18:0/1765610666 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4634+0+0 (secure 0 0 0) 0x7f3030088160 con 0x7f30480fe830 2024-04-12T20:51:07.961 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:07.963+0000 7f304ce00700 1 --2- 172.21.15.18:0/1765610666 >> [v2:172.21.15.18:6800/3048894687,v1:172.21.15.18:6801/3048894687] conn(0x7f302c0692b0 0x7f302c06b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T20:51:07.962 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:07.964+0000 7f304ce00700 1 --2- 172.21.15.18:0/1765610666 >> [v2:172.21.15.18:6800/3048894687,v1:172.21.15.18:6801/3048894687] conn(0x7f302c0692b0 0x7f302c06b760 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f3038007fd0 tx=0x7f3038009040 comp rx=0 tx=0).ready entity=mgr.14212 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T20:51:07.966 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:07.968+0000 7f3045600700 1 -- 172.21.15.18:0/1765610666 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7f3030058140 con 0x7f30480fe830 2024-04-12T20:51:08.188 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:08.190+0000 7f304f800700 1 -- 172.21.15.18:0/1765610666 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f304805f620 con 0x7f30480fe830 2024-04-12T20:51:08.189 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:08.190+0000 7f3045600700 1 -- 172.21.15.18:0/1765610666 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f3030057cd0 con 0x7f30480fe830 2024-04-12T20:51:08.189 INFO:teuthology.orchestra.run.smithi018.stdout:60129542183 2024-04-12T20:51:08.192 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:08.193+0000 7f304f800700 1 -- 172.21.15.18:0/1765610666 >> [v2:172.21.15.18:6800/3048894687,v1:172.21.15.18:6801/3048894687] conn(0x7f302c0692b0 msgr2=0x7f302c06b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:51:08.192 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:08.193+0000 7f304f800700 1 --2- 172.21.15.18:0/1765610666 >> [v2:172.21.15.18:6800/3048894687,v1:172.21.15.18:6801/3048894687] conn(0x7f302c0692b0 0x7f302c06b760 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f3038007fd0 tx=0x7f3038009040 comp rx=0 tx=0).stop 2024-04-12T20:51:08.192 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:08.193+0000 7f304f800700 1 -- 172.21.15.18:0/1765610666 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f30480fe830 msgr2=0x7f30481162e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:51:08.192 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:08.193+0000 7f304f800700 1 --2- 172.21.15.18:0/1765610666 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f30480fe830 0x7f30481162e0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f3030000c00 tx=0x7f3030009e30 comp rx=0 tx=0).stop 2024-04-12T20:51:08.192 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:08.193+0000 7f304f800700 1 -- 172.21.15.18:0/1765610666 shutdown_connections 2024-04-12T20:51:08.192 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:08.193+0000 7f304f800700 1 --2- 172.21.15.18:0/1765610666 >> [v2:172.21.15.18:6800/3048894687,v1:172.21.15.18:6801/3048894687] conn(0x7f302c0692b0 0x7f302c06b760 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:51:08.192 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:08.193+0000 7f304f800700 1 --2- 172.21.15.18:0/1765610666 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f30480fde90 0x7f3048115da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:51:08.192 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:08.193+0000 7f304f800700 1 --2- 172.21.15.18:0/1765610666 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f30480fe830 0x7f30481162e0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:51:08.192 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:08.193+0000 7f304f800700 1 -- 172.21.15.18:0/1765610666 >> 172.21.15.18:0/1765610666 conn(0x7f3048072bc0 msgr2=0x7f30480fb6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T20:51:08.192 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:08.194+0000 7f304f800700 1 -- 172.21.15.18:0/1765610666 shutdown_connections 2024-04-12T20:51:08.192 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:08.194+0000 7f304f800700 1 -- 172.21.15.18:0/1765610666 wait complete. 2024-04-12T20:51:08.352 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:51:08 smithi018 ceph-mon[28799]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-12T20:51:08.531 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:51:08 smithi186 ceph-mon[36597]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-12T20:51:09.247 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:09.248+0000 7f9522a00700 1 -- 172.21.15.18:0/1916618329 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f951c070300 msgr2=0x7f951c0706d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:51:09.247 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:09.248+0000 7f9522a00700 1 --2- 172.21.15.18:0/1916618329 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f951c070300 0x7f951c0706d0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f950c004970 tx=0x7f950c009e30 comp rx=0 tx=0).stop 2024-04-12T20:51:09.247 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:09.248+0000 7f9522a00700 1 -- 172.21.15.18:0/1916618329 shutdown_connections 2024-04-12T20:51:09.247 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:09.248+0000 7f9522a00700 1 --2- 172.21.15.18:0/1916618329 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f951c070c10 0x7f951c109170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:51:09.247 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:09.248+0000 7f9522a00700 1 --2- 172.21.15.18:0/1916618329 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f951c070300 0x7f951c0706d0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:51:09.247 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:09.248+0000 7f9522a00700 1 -- 172.21.15.18:0/1916618329 >> 172.21.15.18:0/1916618329 conn(0x7f951c0f89b0 msgr2=0x7f951c0fadc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T20:51:09.247 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:51:09 smithi018 ceph-mon[28799]: from='client.? 172.21.15.18:0/1765610666' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-04-12T20:51:09.249 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:09.248+0000 7f9522a00700 1 -- 172.21.15.18:0/1916618329 shutdown_connections 2024-04-12T20:51:09.249 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:09.249+0000 7f9522a00700 1 -- 172.21.15.18:0/1916618329 wait complete. 2024-04-12T20:51:09.249 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:09.250+0000 7f9522a00700 1 Processor -- start 2024-04-12T20:51:09.249 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:09.250+0000 7f9522a00700 1 -- start start 2024-04-12T20:51:09.249 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:09.250+0000 7f9522a00700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f951c070300 0x7f951c115af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T20:51:09.249 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:09.250+0000 7f9522a00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f951c070c10 0x7f951c116030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T20:51:09.249 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:09.250+0000 7f9522a00700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f951c116680 con 0x7f951c070c10 2024-04-12T20:51:09.249 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:09.250+0000 7f9522a00700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f951c10fca0 con 0x7f951c070300 2024-04-12T20:51:09.249 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:09.250+0000 7f951b400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f951c070c10 0x7f951c116030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T20:51:09.249 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:09.251+0000 7f951b400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f951c070c10 0x7f951c116030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:49400/0 (socket says 172.21.15.18:49400) 2024-04-12T20:51:09.249 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:09.251+0000 7f951b400700 1 -- 172.21.15.18:0/1497424572 learned_addr learned my addr 172.21.15.18:0/1497424572 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T20:51:09.250 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:09.251+0000 7f951be00700 1 --2- 172.21.15.18:0/1497424572 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f951c070300 0x7f951c115af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T20:51:09.250 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:09.251+0000 7f951b400700 1 -- 172.21.15.18:0/1497424572 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f951c070300 msgr2=0x7f951c115af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:51:09.250 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:09.251+0000 7f951b400700 1 --2- 172.21.15.18:0/1497424572 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f951c070300 0x7f951c115af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:51:09.250 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:09.251+0000 7f951b400700 1 -- 172.21.15.18:0/1497424572 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f950c005040 con 0x7f951c070c10 2024-04-12T20:51:09.250 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:09.251+0000 7f951b400700 1 --2- 172.21.15.18:0/1497424572 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f951c070c10 0x7f951c116030 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f9508002a50 tx=0x7f95080046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T20:51:09.251 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:09.252+0000 7f9513e00700 1 -- 172.21.15.18:0/1497424572 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9508007cc0 con 0x7f951c070c10 2024-04-12T20:51:09.251 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:09.252+0000 7f9513e00700 1 -- 172.21.15.18:0/1497424572 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f950800f040 con 0x7f951c070c10 2024-04-12T20:51:09.251 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:09.252+0000 7f9522a00700 1 -- 172.21.15.18:0/1497424572 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f951c10ff80 con 0x7f951c070c10 2024-04-12T20:51:09.251 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:09.252+0000 7f9513e00700 1 -- 172.21.15.18:0/1497424572 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9508009640 con 0x7f951c070c10 2024-04-12T20:51:09.251 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:09.252+0000 7f9522a00700 1 -- 172.21.15.18:0/1497424572 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f951c1104c0 con 0x7f951c070c10 2024-04-12T20:51:09.251 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:09.253+0000 7f9522a00700 1 -- 172.21.15.18:0/1497424572 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f951c071a40 con 0x7f951c070c10 2024-04-12T20:51:09.252 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:09.254+0000 7f9513e00700 1 -- 172.21.15.18:0/1497424572 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7f95080097a0 con 0x7f951c070c10 2024-04-12T20:51:09.253 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:09.255+0000 7f9513e00700 1 --2- 172.21.15.18:0/1497424572 >> [v2:172.21.15.18:6800/3048894687,v1:172.21.15.18:6801/3048894687] conn(0x7f95000692a0 0x7f950006b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T20:51:09.253 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:09.255+0000 7f9513e00700 1 -- 172.21.15.18:0/1497424572 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4634+0+0 (secure 0 0 0) 0x7f9508086d20 con 0x7f951c070c10 2024-04-12T20:51:09.253 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:09.255+0000 7f951be00700 1 --2- 172.21.15.18:0/1497424572 >> [v2:172.21.15.18:6800/3048894687,v1:172.21.15.18:6801/3048894687] conn(0x7f95000692a0 0x7f950006b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T20:51:09.254 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:09.256+0000 7f951be00700 1 --2- 172.21.15.18:0/1497424572 >> [v2:172.21.15.18:6800/3048894687,v1:172.21.15.18:6801/3048894687] conn(0x7f95000692a0 0x7f950006b750 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f950c009fd0 tx=0x7f950c009e30 comp rx=0 tx=0).ready entity=mgr.14212 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T20:51:09.258 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:09.260+0000 7f9513e00700 1 -- 172.21.15.18:0/1497424572 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7f9508053270 con 0x7f951c070c10 2024-04-12T20:51:09.391 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:51:09 smithi186 ceph-mon[36597]: from='client.? 172.21.15.18:0/1765610666' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-04-12T20:51:09.476 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:09.477+0000 7f9522a00700 1 -- 172.21.15.18:0/1497424572 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f951c10c0f0 con 0x7f951c070c10 2024-04-12T20:51:09.476 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:09.478+0000 7f9513e00700 1 -- 172.21.15.18:0/1497424572 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f9508056890 con 0x7f951c070c10 2024-04-12T20:51:09.476 INFO:teuthology.orchestra.run.smithi018.stdout:38654705712 2024-04-12T20:51:09.479 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:09.481+0000 7f9522a00700 1 -- 172.21.15.18:0/1497424572 >> [v2:172.21.15.18:6800/3048894687,v1:172.21.15.18:6801/3048894687] conn(0x7f95000692a0 msgr2=0x7f950006b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:51:09.480 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:09.481+0000 7f9522a00700 1 --2- 172.21.15.18:0/1497424572 >> [v2:172.21.15.18:6800/3048894687,v1:172.21.15.18:6801/3048894687] conn(0x7f95000692a0 0x7f950006b750 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f950c009fd0 tx=0x7f950c009e30 comp rx=0 tx=0).stop 2024-04-12T20:51:09.480 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:09.481+0000 7f9522a00700 1 -- 172.21.15.18:0/1497424572 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f951c070c10 msgr2=0x7f951c116030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:51:09.480 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:09.481+0000 7f9522a00700 1 --2- 172.21.15.18:0/1497424572 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f951c070c10 0x7f951c116030 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f9508002a50 tx=0x7f95080046b0 comp rx=0 tx=0).stop 2024-04-12T20:51:09.480 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:09.481+0000 7f9522a00700 1 -- 172.21.15.18:0/1497424572 shutdown_connections 2024-04-12T20:51:09.480 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:09.481+0000 7f9522a00700 1 --2- 172.21.15.18:0/1497424572 >> [v2:172.21.15.18:6800/3048894687,v1:172.21.15.18:6801/3048894687] conn(0x7f95000692a0 0x7f950006b750 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:51:09.480 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:09.481+0000 7f9522a00700 1 --2- 172.21.15.18:0/1497424572 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f951c070300 0x7f951c115af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:51:09.480 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:09.481+0000 7f9522a00700 1 --2- 172.21.15.18:0/1497424572 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f951c070c10 0x7f951c116030 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:51:09.480 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:09.481+0000 7f9522a00700 1 -- 172.21.15.18:0/1497424572 >> 172.21.15.18:0/1497424572 conn(0x7f951c0f89b0 msgr2=0x7f951c0f9de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T20:51:09.480 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:09.482+0000 7f9522a00700 1 -- 172.21.15.18:0/1497424572 shutdown_connections 2024-04-12T20:51:09.480 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:09.482+0000 7f9522a00700 1 -- 172.21.15.18:0/1497424572 wait complete. 2024-04-12T20:51:10.281 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:51:10 smithi186 ceph-mon[36597]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-12T20:51:10.281 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:51:10 smithi186 ceph-mon[36597]: from='client.? 172.21.15.18:0/1497424572' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-04-12T20:51:10.283 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:51:10 smithi018 ceph-mon[28799]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-12T20:51:10.283 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:51:10 smithi018 ceph-mon[28799]: from='client.? 172.21.15.18:0/1497424572' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-04-12T20:51:10.964 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/mon.smithi018/config 2024-04-12T20:51:11.353 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:51:11 smithi018 ceph-mon[28799]: from='client.? 172.21.15.18:0/2918598517' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-04-12T20:51:11.531 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:51:11 smithi186 ceph-mon[36597]: from='client.? 172.21.15.18:0/2918598517' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-04-12T20:51:11.636 INFO:tasks.cephadm.ceph_manager.ceph:need seq 38654705709 got 38654705712 for osd.0 2024-04-12T20:51:11.636 DEBUG:teuthology.parallel:result is None 2024-04-12T20:51:12.327 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:10.850+0000 7f818b000700 1 -- 172.21.15.18:0/897701772 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f81841059a0 msgr2=0x7f81840ff010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:51:12.327 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:10.851+0000 7f818b000700 1 --2- 172.21.15.18:0/897701772 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f81841059a0 0x7f81840ff010 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f816c004940 tx=0x7f816c009e30 comp rx=0 tx=0).stop 2024-04-12T20:51:12.328 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:10.851+0000 7f818b000700 1 -- 172.21.15.18:0/897701772 shutdown_connections 2024-04-12T20:51:12.328 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:10.851+0000 7f818b000700 1 --2- 172.21.15.18:0/897701772 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f81841059a0 0x7f81840ff010 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:51:12.328 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:10.851+0000 7f818b000700 1 --2- 172.21.15.18:0/897701772 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8184105000 0x7f81841053d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:51:12.328 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:10.851+0000 7f818b000700 1 -- 172.21.15.18:0/897701772 >> 172.21.15.18:0/897701772 conn(0x7f81840fabd0 msgr2=0x7f81840fcfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T20:51:12.328 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:10.851+0000 7f818b000700 1 -- 172.21.15.18:0/897701772 shutdown_connections 2024-04-12T20:51:12.328 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:10.851+0000 7f818b000700 1 -- 172.21.15.18:0/897701772 wait complete. 2024-04-12T20:51:12.328 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:10.852+0000 7f818b000700 1 Processor -- start 2024-04-12T20:51:12.328 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:10.852+0000 7f818b000700 1 -- start start 2024-04-12T20:51:12.328 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:10.852+0000 7f818b000700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8184105000 0x7f81841190c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T20:51:12.328 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:10.852+0000 7f818b000700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f81841059a0 0x7f8184119600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T20:51:12.328 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:10.852+0000 7f818b000700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8184119c30 con 0x7f8184105000 2024-04-12T20:51:12.328 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:10.852+0000 7f818b000700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8184074f50 con 0x7f81841059a0 2024-04-12T20:51:12.328 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:10.852+0000 7f8183e00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8184105000 0x7f81841190c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T20:51:12.329 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:10.853+0000 7f8183e00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8184105000 0x7f81841190c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:48172/0 (socket says 172.21.15.18:48172) 2024-04-12T20:51:12.329 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:10.853+0000 7f8183e00700 1 -- 172.21.15.18:0/2918598517 learned_addr learned my addr 172.21.15.18:0/2918598517 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T20:51:12.329 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:10.853+0000 7f8183e00700 1 -- 172.21.15.18:0/2918598517 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f81841059a0 msgr2=0x7f8184119600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:51:12.329 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:10.853+0000 7f8183e00700 1 --2- 172.21.15.18:0/2918598517 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f81841059a0 0x7f8184119600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:51:12.329 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:10.853+0000 7f8183e00700 1 -- 172.21.15.18:0/2918598517 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f816c005040 con 0x7f8184105000 2024-04-12T20:51:12.329 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:10.853+0000 7f8183e00700 1 --2- 172.21.15.18:0/2918598517 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8184105000 0x7f81841190c0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f817400c8c0 tx=0x7f81740044c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T20:51:12.329 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:10.853+0000 7f8180c00700 1 -- 172.21.15.18:0/2918598517 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f817400a980 con 0x7f8184105000 2024-04-12T20:51:12.329 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:10.854+0000 7f8180c00700 1 -- 172.21.15.18:0/2918598517 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f817400ce60 con 0x7f8184105000 2024-04-12T20:51:12.329 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:10.854+0000 7f8180c00700 1 -- 172.21.15.18:0/2918598517 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f81740115e0 con 0x7f8184105000 2024-04-12T20:51:12.329 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:10.854+0000 7f818b000700 1 -- 172.21.15.18:0/2918598517 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f81840751e0 con 0x7f8184105000 2024-04-12T20:51:12.329 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:10.854+0000 7f818b000700 1 -- 172.21.15.18:0/2918598517 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f81840757a0 con 0x7f8184105000 2024-04-12T20:51:12.329 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:10.855+0000 7f818b000700 1 -- 172.21.15.18:0/2918598517 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8184100750 con 0x7f8184105000 2024-04-12T20:51:12.329 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:10.856+0000 7f8180c00700 1 -- 172.21.15.18:0/2918598517 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7f8174012400 con 0x7f8184105000 2024-04-12T20:51:12.329 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:10.857+0000 7f8180c00700 1 --2- 172.21.15.18:0/2918598517 >> [v2:172.21.15.18:6800/3048894687,v1:172.21.15.18:6801/3048894687] conn(0x7f81680692b0 0x7f816806b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T20:51:12.329 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:10.857+0000 7f8180c00700 1 -- 172.21.15.18:0/2918598517 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4634+0+0 (secure 0 0 0) 0x7f8174086e50 con 0x7f8184105000 2024-04-12T20:51:12.330 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:10.857+0000 7f817be00700 1 --2- 172.21.15.18:0/2918598517 >> [v2:172.21.15.18:6800/3048894687,v1:172.21.15.18:6801/3048894687] conn(0x7f81680692b0 0x7f816806b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T20:51:12.330 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:10.858+0000 7f817be00700 1 --2- 172.21.15.18:0/2918598517 >> [v2:172.21.15.18:6800/3048894687,v1:172.21.15.18:6801/3048894687] conn(0x7f81680692b0 0x7f816806b760 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f816c004f90 tx=0x7f816c009e30 comp rx=0 tx=0).ready entity=mgr.14212 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T20:51:12.330 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:10.861+0000 7f8180c00700 1 -- 172.21.15.18:0/2918598517 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7f8174056ff0 con 0x7f8184105000 2024-04-12T20:51:12.330 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:11.100+0000 7f818b000700 1 -- 172.21.15.18:0/2918598517 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7f818410c0e0 con 0x7f8184105000 2024-04-12T20:51:12.330 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:11.101+0000 7f8180c00700 1 -- 172.21.15.18:0/2918598517 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f81740126b0 con 0x7f8184105000 2024-04-12T20:51:12.330 INFO:teuthology.orchestra.run.smithi018.stdout:167503724557 2024-04-12T20:51:12.330 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:11.103+0000 7f818b000700 1 -- 172.21.15.18:0/2918598517 >> [v2:172.21.15.18:6800/3048894687,v1:172.21.15.18:6801/3048894687] conn(0x7f81680692b0 msgr2=0x7f816806b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:51:12.331 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:11.103+0000 7f818b000700 1 --2- 172.21.15.18:0/2918598517 >> [v2:172.21.15.18:6800/3048894687,v1:172.21.15.18:6801/3048894687] conn(0x7f81680692b0 0x7f816806b760 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f816c004f90 tx=0x7f816c009e30 comp rx=0 tx=0).stop 2024-04-12T20:51:12.331 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:11.103+0000 7f818b000700 1 -- 172.21.15.18:0/2918598517 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8184105000 msgr2=0x7f81841190c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:51:12.331 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:11.103+0000 7f818b000700 1 --2- 172.21.15.18:0/2918598517 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8184105000 0x7f81841190c0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f817400c8c0 tx=0x7f81740044c0 comp rx=0 tx=0).stop 2024-04-12T20:51:12.331 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:11.104+0000 7f818b000700 1 -- 172.21.15.18:0/2918598517 shutdown_connections 2024-04-12T20:51:12.331 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:11.104+0000 7f818b000700 1 --2- 172.21.15.18:0/2918598517 >> [v2:172.21.15.18:6800/3048894687,v1:172.21.15.18:6801/3048894687] conn(0x7f81680692b0 0x7f816806b760 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:51:12.331 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:11.104+0000 7f818b000700 1 --2- 172.21.15.18:0/2918598517 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8184105000 0x7f81841190c0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:51:12.331 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:11.104+0000 7f818b000700 1 --2- 172.21.15.18:0/2918598517 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f81841059a0 0x7f8184119600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:51:12.331 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:11.104+0000 7f818b000700 1 -- 172.21.15.18:0/2918598517 >> 172.21.15.18:0/2918598517 conn(0x7f81840fabd0 msgr2=0x7f81840fbcc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T20:51:12.331 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:11.104+0000 7f818b000700 1 -- 172.21.15.18:0/2918598517 shutdown_connections 2024-04-12T20:51:12.331 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:11.104+0000 7f818b000700 1 -- 172.21.15.18:0/2918598517 wait complete. 2024-04-12T20:51:12.531 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:51:12 smithi186 ceph-mon[36597]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-12T20:51:12.602 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:51:12 smithi018 ceph-mon[28799]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-12T20:51:13.176 INFO:tasks.cephadm.ceph_manager.ceph:need seq 60129542180 got 60129542183 for osd.1 2024-04-12T20:51:13.177 DEBUG:teuthology.parallel:result is None 2024-04-12T20:51:13.982 INFO:tasks.cephadm.ceph_manager.ceph:need seq 167503724555 got 167503724557 for osd.5 2024-04-12T20:51:13.982 DEBUG:teuthology.parallel:result is None 2024-04-12T20:51:14.353 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:51:14 smithi018 ceph-mon[28799]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-12T20:51:14.531 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:51:14 smithi186 ceph-mon[36597]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-12T20:51:15.175 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:15.168+0000 7fe0b9a00700 1 -- 172.21.15.18:0/2447013503 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe0b40fdeb0 msgr2=0x7fe0b40fe280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:51:15.176 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:15.168+0000 7fe0b9a00700 1 --2- 172.21.15.18:0/2447013503 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe0b40fdeb0 0x7fe0b40fe280 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fe0a4004970 tx=0x7fe0a4009e30 comp rx=0 tx=0).stop 2024-04-12T20:51:15.176 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:15.169+0000 7fe0b9a00700 1 -- 172.21.15.18:0/2447013503 shutdown_connections 2024-04-12T20:51:15.176 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:15.169+0000 7fe0b9a00700 1 --2- 172.21.15.18:0/2447013503 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe0b40fe850 0x7fe0b4106f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:51:15.176 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:15.169+0000 7fe0b9a00700 1 --2- 172.21.15.18:0/2447013503 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe0b40fdeb0 0x7fe0b40fe280 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:51:15.176 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:15.169+0000 7fe0b9a00700 1 -- 172.21.15.18:0/2447013503 >> 172.21.15.18:0/2447013503 conn(0x7fe0b4072bc0 msgr2=0x7fe0b4072fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T20:51:15.176 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:15.169+0000 7fe0b9a00700 1 -- 172.21.15.18:0/2447013503 shutdown_connections 2024-04-12T20:51:15.176 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:15.169+0000 7fe0b9a00700 1 -- 172.21.15.18:0/2447013503 wait complete. 2024-04-12T20:51:15.176 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:15.170+0000 7fe0b9a00700 1 Processor -- start 2024-04-12T20:51:15.176 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:15.170+0000 7fe0b9a00700 1 -- start start 2024-04-12T20:51:15.176 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:15.170+0000 7fe0b9a00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe0b40fdeb0 0x7fe0b4115d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T20:51:15.176 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:15.170+0000 7fe0b9a00700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe0b40fe850 0x7fe0b41162a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T20:51:15.176 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:15.170+0000 7fe0b9a00700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0b41168f0 con 0x7fe0b40fdeb0 2024-04-12T20:51:15.176 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:15.170+0000 7fe0b9a00700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0b410fd00 con 0x7fe0b40fe850 2024-04-12T20:51:15.176 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:15.170+0000 7fe0b2a00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe0b40fdeb0 0x7fe0b4115d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T20:51:15.177 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:15.171+0000 7fe0b2a00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe0b40fdeb0 0x7fe0b4115d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:48188/0 (socket says 172.21.15.18:48188) 2024-04-12T20:51:15.177 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:15.171+0000 7fe0b2a00700 1 -- 172.21.15.18:0/1704798550 learned_addr learned my addr 172.21.15.18:0/1704798550 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T20:51:15.177 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:15.171+0000 7fe0b2000700 1 --2- 172.21.15.18:0/1704798550 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe0b40fe850 0x7fe0b41162a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T20:51:15.177 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:15.171+0000 7fe0b2a00700 1 -- 172.21.15.18:0/1704798550 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe0b40fe850 msgr2=0x7fe0b41162a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:51:15.177 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:15.171+0000 7fe0b2a00700 1 --2- 172.21.15.18:0/1704798550 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe0b40fe850 0x7fe0b41162a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:51:15.177 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:15.171+0000 7fe0b2a00700 1 -- 172.21.15.18:0/1704798550 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0a4005040 con 0x7fe0b40fdeb0 2024-04-12T20:51:15.177 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:15.171+0000 7fe0b2000700 1 --2- 172.21.15.18:0/1704798550 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe0b40fe850 0x7fe0b41162a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-12T20:51:15.177 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:15.171+0000 7fe0b2a00700 1 --2- 172.21.15.18:0/1704798550 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe0b40fdeb0 0x7fe0b4115d60 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fe0a4004f50 tx=0x7fe0a4009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T20:51:15.177 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:15.172+0000 7fe0aaa00700 1 -- 172.21.15.18:0/1704798550 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe0a400e040 con 0x7fe0b40fdeb0 2024-04-12T20:51:15.177 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:15.172+0000 7fe0aaa00700 1 -- 172.21.15.18:0/1704798550 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe0a400bc10 con 0x7fe0b40fdeb0 2024-04-12T20:51:15.177 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:15.172+0000 7fe0b9a00700 1 -- 172.21.15.18:0/1704798550 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe0b410ff80 con 0x7fe0b40fdeb0 2024-04-12T20:51:15.177 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:15.172+0000 7fe0aaa00700 1 -- 172.21.15.18:0/1704798550 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe0a401c630 con 0x7fe0b40fdeb0 2024-04-12T20:51:15.177 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:15.172+0000 7fe0b9a00700 1 -- 172.21.15.18:0/1704798550 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe0b41104c0 con 0x7fe0b40fdeb0 2024-04-12T20:51:15.177 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:15.173+0000 7fe0b9a00700 1 -- 172.21.15.18:0/1704798550 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe0b41069f0 con 0x7fe0b40fdeb0 2024-04-12T20:51:15.178 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:15.174+0000 7fe0aaa00700 1 -- 172.21.15.18:0/1704798550 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7fe0a4013420 con 0x7fe0b40fdeb0 2024-04-12T20:51:15.178 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:15.175+0000 7fe0aaa00700 1 --2- 172.21.15.18:0/1704798550 >> [v2:172.21.15.18:6800/3048894687,v1:172.21.15.18:6801/3048894687] conn(0x7fe0980692b0 0x7fe09806b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T20:51:15.178 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:15.175+0000 7fe0aaa00700 1 -- 172.21.15.18:0/1704798550 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4634+0+0 (secure 0 0 0) 0x7fe0a40881a0 con 0x7fe0b40fdeb0 2024-04-12T20:51:15.178 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:15.175+0000 7fe0b2000700 1 --2- 172.21.15.18:0/1704798550 >> [v2:172.21.15.18:6800/3048894687,v1:172.21.15.18:6801/3048894687] conn(0x7fe0980692b0 0x7fe09806b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T20:51:15.178 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:15.176+0000 7fe0b2000700 1 --2- 172.21.15.18:0/1704798550 >> [v2:172.21.15.18:6800/3048894687,v1:172.21.15.18:6801/3048894687] conn(0x7fe0980692b0 0x7fe09806b760 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fe0a0007fd0 tx=0x7fe0a0009040 comp rx=0 tx=0).ready entity=mgr.14212 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T20:51:15.182 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:15.184+0000 7fe0aaa00700 1 -- 172.21.15.18:0/1704798550 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7fe0a4054810 con 0x7fe0b40fdeb0 2024-04-12T20:51:15.255 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:51:15 smithi018 ceph-mon[28799]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T20:51:15.352 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:15.256+0000 7f55c4800700 1 -- 172.21.15.18:0/2683982593 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f55bc0fbc80 msgr2=0x7f55bc0fc050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:51:15.352 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:15.256+0000 7f55c4800700 1 --2- 172.21.15.18:0/2683982593 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f55bc0fbc80 0x7f55bc0fc050 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f55b0004940 tx=0x7f55b0009e30 comp rx=0 tx=0).stop 2024-04-12T20:51:15.352 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:15.256+0000 7f55c4800700 1 -- 172.21.15.18:0/2683982593 shutdown_connections 2024-04-12T20:51:15.352 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:15.256+0000 7f55c4800700 1 --2- 172.21.15.18:0/2683982593 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f55bc0fc590 0x7f55bc1092f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:51:15.352 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:15.256+0000 7f55c4800700 1 --2- 172.21.15.18:0/2683982593 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f55bc0fbc80 0x7f55bc0fc050 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:51:15.352 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:15.256+0000 7f55c4800700 1 -- 172.21.15.18:0/2683982593 >> 172.21.15.18:0/2683982593 conn(0x7f55bc072bc0 msgr2=0x7f55bc072fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T20:51:15.352 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:15.256+0000 7f55c4800700 1 -- 172.21.15.18:0/2683982593 shutdown_connections 2024-04-12T20:51:15.352 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:15.256+0000 7f55c4800700 1 -- 172.21.15.18:0/2683982593 wait complete. 2024-04-12T20:51:15.353 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:15.257+0000 7f55c4800700 1 Processor -- start 2024-04-12T20:51:15.353 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:15.257+0000 7f55c4800700 1 -- start start 2024-04-12T20:51:15.353 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:15.257+0000 7f55c4800700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f55bc0fbc80 0x7f55bc114c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T20:51:15.353 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:15.257+0000 7f55c4800700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f55bc0fc590 0x7f55bc1151a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T20:51:15.353 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:15.257+0000 7f55c4800700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55bc1157a0 con 0x7f55bc0fc590 2024-04-12T20:51:15.353 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:15.257+0000 7f55c4800700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55bc10fd70 con 0x7f55bc0fbc80 2024-04-12T20:51:15.353 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:15.258+0000 7f55c1400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f55bc0fc590 0x7f55bc1151a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T20:51:15.353 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:15.258+0000 7f55c1400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f55bc0fc590 0x7f55bc1151a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:48198/0 (socket says 172.21.15.18:48198) 2024-04-12T20:51:15.353 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:15.258+0000 7f55c1400700 1 -- 172.21.15.18:0/2944524834 learned_addr learned my addr 172.21.15.18:0/2944524834 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T20:51:15.353 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:15.258+0000 7f55c1e00700 1 --2- 172.21.15.18:0/2944524834 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f55bc0fbc80 0x7f55bc114c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T20:51:15.353 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:15.258+0000 7f55c1400700 1 -- 172.21.15.18:0/2944524834 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f55bc0fbc80 msgr2=0x7f55bc114c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:51:15.353 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:15.258+0000 7f55c1400700 1 --2- 172.21.15.18:0/2944524834 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f55bc0fbc80 0x7f55bc114c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:51:15.353 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:15.258+0000 7f55c1400700 1 -- 172.21.15.18:0/2944524834 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55b0005040 con 0x7f55bc0fc590 2024-04-12T20:51:15.353 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:15.258+0000 7f55c1e00700 1 --2- 172.21.15.18:0/2944524834 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f55bc0fbc80 0x7f55bc114c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-12T20:51:15.354 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:15.259+0000 7f55c1400700 1 --2- 172.21.15.18:0/2944524834 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f55bc0fc590 0x7f55bc1151a0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f55ac00c8c0 tx=0x7f55ac0044c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T20:51:15.354 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:15.259+0000 7f55b6000700 1 -- 172.21.15.18:0/2944524834 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f55ac00a980 con 0x7f55bc0fc590 2024-04-12T20:51:15.354 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:15.259+0000 7f55c4800700 1 -- 172.21.15.18:0/2944524834 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f55bc110050 con 0x7f55bc0fc590 2024-04-12T20:51:15.354 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:15.259+0000 7f55b6000700 1 -- 172.21.15.18:0/2944524834 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f55ac00ce60 con 0x7f55bc0fc590 2024-04-12T20:51:15.354 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:15.259+0000 7f55b6000700 1 -- 172.21.15.18:0/2944524834 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f55ac0115e0 con 0x7f55bc0fc590 2024-04-12T20:51:15.354 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:15.259+0000 7f55c4800700 1 -- 172.21.15.18:0/2944524834 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f55bc110610 con 0x7f55bc0fc590 2024-04-12T20:51:15.354 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:15.260+0000 7f55c4800700 1 -- 172.21.15.18:0/2944524834 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f55bc065e30 con 0x7f55bc0fc590 2024-04-12T20:51:15.354 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:15.261+0000 7f55b6000700 1 -- 172.21.15.18:0/2944524834 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7f55ac012400 con 0x7f55bc0fc590 2024-04-12T20:51:15.354 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:15.262+0000 7f55b6000700 1 --2- 172.21.15.18:0/2944524834 >> [v2:172.21.15.18:6800/3048894687,v1:172.21.15.18:6801/3048894687] conn(0x7f55a40692b0 0x7f55a406b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T20:51:15.354 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:15.262+0000 7f55b6000700 1 -- 172.21.15.18:0/2944524834 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4634+0+0 (secure 0 0 0) 0x7f55ac086f50 con 0x7f55bc0fc590 2024-04-12T20:51:15.354 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:15.263+0000 7f55c1e00700 1 --2- 172.21.15.18:0/2944524834 >> [v2:172.21.15.18:6800/3048894687,v1:172.21.15.18:6801/3048894687] conn(0x7f55a40692b0 0x7f55a406b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T20:51:15.354 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:15.263+0000 7f55c1e00700 1 --2- 172.21.15.18:0/2944524834 >> [v2:172.21.15.18:6800/3048894687,v1:172.21.15.18:6801/3048894687] conn(0x7f55a40692b0 0x7f55a406b760 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f55b0000c00 tx=0x7f55b0009e30 comp rx=0 tx=0).ready entity=mgr.14212 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T20:51:15.354 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:15.271+0000 7f55b6000700 1 -- 172.21.15.18:0/2944524834 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7f55ac0535c0 con 0x7f55bc0fc590 2024-04-12T20:51:15.413 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:15.415+0000 7fe0b9a00700 1 -- 172.21.15.18:0/1704798550 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7fe0b404bed0 con 0x7fe0b40fdeb0 2024-04-12T20:51:15.413 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:15.415+0000 7fe0aaa00700 1 -- 172.21.15.18:0/1704798550 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fe0a4057e30 con 0x7fe0b40fdeb0 2024-04-12T20:51:15.414 INFO:teuthology.orchestra.run.smithi018.stdout:81604378658 2024-04-12T20:51:15.416 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:15.418+0000 7fe0b9a00700 1 -- 172.21.15.18:0/1704798550 >> [v2:172.21.15.18:6800/3048894687,v1:172.21.15.18:6801/3048894687] conn(0x7fe0980692b0 msgr2=0x7fe09806b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:51:15.416 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:15.418+0000 7fe0b9a00700 1 --2- 172.21.15.18:0/1704798550 >> [v2:172.21.15.18:6800/3048894687,v1:172.21.15.18:6801/3048894687] conn(0x7fe0980692b0 0x7fe09806b760 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fe0a0007fd0 tx=0x7fe0a0009040 comp rx=0 tx=0).stop 2024-04-12T20:51:15.416 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:15.418+0000 7fe0b9a00700 1 -- 172.21.15.18:0/1704798550 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe0b40fdeb0 msgr2=0x7fe0b4115d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:51:15.416 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:15.418+0000 7fe0b9a00700 1 --2- 172.21.15.18:0/1704798550 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe0b40fdeb0 0x7fe0b4115d60 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fe0a4004f50 tx=0x7fe0a4009e30 comp rx=0 tx=0).stop 2024-04-12T20:51:15.417 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:15.418+0000 7fe0b9a00700 1 -- 172.21.15.18:0/1704798550 shutdown_connections 2024-04-12T20:51:15.417 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:15.418+0000 7fe0b9a00700 1 --2- 172.21.15.18:0/1704798550 >> [v2:172.21.15.18:6800/3048894687,v1:172.21.15.18:6801/3048894687] conn(0x7fe0980692b0 0x7fe09806b760 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:51:15.417 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:15.418+0000 7fe0b9a00700 1 --2- 172.21.15.18:0/1704798550 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe0b40fdeb0 0x7fe0b4115d60 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:51:15.417 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:15.418+0000 7fe0b9a00700 1 --2- 172.21.15.18:0/1704798550 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe0b40fe850 0x7fe0b41162a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:51:15.417 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:15.418+0000 7fe0b9a00700 1 -- 172.21.15.18:0/1704798550 >> 172.21.15.18:0/1704798550 conn(0x7fe0b4072bc0 msgr2=0x7fe0b40fd470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T20:51:15.417 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:15.418+0000 7fe0b9a00700 1 -- 172.21.15.18:0/1704798550 shutdown_connections 2024-04-12T20:51:15.417 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:15.419+0000 7fe0b9a00700 1 -- 172.21.15.18:0/1704798550 wait complete. 2024-04-12T20:51:15.455 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:51:15 smithi186 ceph-mon[36597]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T20:51:15.495 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:15.497+0000 7f55c4800700 1 -- 172.21.15.18:0/2944524834 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7f55bc10c160 con 0x7f55bc0fc590 2024-04-12T20:51:15.496 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:15.498+0000 7f55b6000700 1 -- 172.21.15.18:0/2944524834 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f55ac0126b0 con 0x7f55bc0fc590 2024-04-12T20:51:15.496 INFO:teuthology.orchestra.run.smithi018.stdout:146028888085 2024-04-12T20:51:15.499 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:15.501+0000 7f55c4800700 1 -- 172.21.15.18:0/2944524834 >> [v2:172.21.15.18:6800/3048894687,v1:172.21.15.18:6801/3048894687] conn(0x7f55a40692b0 msgr2=0x7f55a406b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:51:15.499 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:15.501+0000 7f55c4800700 1 --2- 172.21.15.18:0/2944524834 >> [v2:172.21.15.18:6800/3048894687,v1:172.21.15.18:6801/3048894687] conn(0x7f55a40692b0 0x7f55a406b760 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f55b0000c00 tx=0x7f55b0009e30 comp rx=0 tx=0).stop 2024-04-12T20:51:15.499 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:15.501+0000 7f55c4800700 1 -- 172.21.15.18:0/2944524834 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f55bc0fc590 msgr2=0x7f55bc1151a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:51:15.499 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:15.501+0000 7f55c4800700 1 --2- 172.21.15.18:0/2944524834 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f55bc0fc590 0x7f55bc1151a0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f55ac00c8c0 tx=0x7f55ac0044c0 comp rx=0 tx=0).stop 2024-04-12T20:51:15.499 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:15.501+0000 7f55c4800700 1 -- 172.21.15.18:0/2944524834 shutdown_connections 2024-04-12T20:51:15.499 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:15.501+0000 7f55c4800700 1 --2- 172.21.15.18:0/2944524834 >> [v2:172.21.15.18:6800/3048894687,v1:172.21.15.18:6801/3048894687] conn(0x7f55a40692b0 0x7f55a406b760 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:51:15.500 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:15.501+0000 7f55c4800700 1 --2- 172.21.15.18:0/2944524834 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f55bc0fbc80 0x7f55bc114c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:51:15.500 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:15.501+0000 7f55c4800700 1 --2- 172.21.15.18:0/2944524834 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f55bc0fc590 0x7f55bc1151a0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:51:15.500 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:15.501+0000 7f55c4800700 1 -- 172.21.15.18:0/2944524834 >> 172.21.15.18:0/2944524834 conn(0x7f55bc072bc0 msgr2=0x7f55bc105ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T20:51:15.500 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:15.501+0000 7f55c4800700 1 -- 172.21.15.18:0/2944524834 shutdown_connections 2024-04-12T20:51:15.500 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:15.501+0000 7f55c4800700 1 -- 172.21.15.18:0/2944524834 wait complete. 2024-04-12T20:51:16.261 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:51:16 smithi018 ceph-mon[28799]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-12T20:51:16.261 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:51:16 smithi018 ceph-mon[28799]: from='client.? 172.21.15.18:0/1704798550' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-04-12T20:51:16.261 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:51:16 smithi018 ceph-mon[28799]: from='client.? 172.21.15.18:0/2944524834' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-04-12T20:51:16.433 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:16.432+0000 7f84c6800700 1 -- 172.21.15.18:0/2097516504 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f84c0070360 msgr2=0x7f84c0070730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:51:16.433 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:16.432+0000 7f84c6800700 1 --2- 172.21.15.18:0/2097516504 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f84c0070360 0x7f84c0070730 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f84b0004970 tx=0x7f84b0009e30 comp rx=0 tx=0).stop 2024-04-12T20:51:16.433 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:16.433+0000 7f84c6800700 1 -- 172.21.15.18:0/2097516504 shutdown_connections 2024-04-12T20:51:16.433 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:16.433+0000 7f84c6800700 1 --2- 172.21.15.18:0/2097516504 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f84c0070c70 0x7f84c0109110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:51:16.433 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:16.433+0000 7f84c6800700 1 --2- 172.21.15.18:0/2097516504 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f84c0070360 0x7f84c0070730 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:51:16.433 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:16.433+0000 7f84c6800700 1 -- 172.21.15.18:0/2097516504 >> 172.21.15.18:0/2097516504 conn(0x7f84c00f89b0 msgr2=0x7f84c00fadc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T20:51:16.433 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:16.433+0000 7f84c6800700 1 -- 172.21.15.18:0/2097516504 shutdown_connections 2024-04-12T20:51:16.433 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:16.433+0000 7f84c6800700 1 -- 172.21.15.18:0/2097516504 wait complete. 2024-04-12T20:51:16.433 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:16.433+0000 7f84c6800700 1 Processor -- start 2024-04-12T20:51:16.433 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:16.434+0000 7f84c6800700 1 -- start start 2024-04-12T20:51:16.433 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:16.434+0000 7f84c6800700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f84c0070360 0x7f84c0115d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T20:51:16.434 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:16.434+0000 7f84c6800700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f84c0070c70 0x7f84c0116290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T20:51:16.434 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:16.434+0000 7f84c6800700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84c01168e0 con 0x7f84c0070360 2024-04-12T20:51:16.434 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:16.434+0000 7f84c6800700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84c010fcb0 con 0x7f84c0070c70 2024-04-12T20:51:16.434 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:16.434+0000 7f84bf400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f84c0070360 0x7f84c0115d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T20:51:16.434 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:16.434+0000 7f84bf400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f84c0070360 0x7f84c0115d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:48220/0 (socket says 172.21.15.18:48220) 2024-04-12T20:51:16.434 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:16.434+0000 7f84bf400700 1 -- 172.21.15.18:0/2702106689 learned_addr learned my addr 172.21.15.18:0/2702106689 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T20:51:16.434 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:16.435+0000 7f84b6a00700 1 --2- 172.21.15.18:0/2702106689 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f84c0070c70 0x7f84c0116290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T20:51:16.434 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:16.435+0000 7f84bf400700 1 -- 172.21.15.18:0/2702106689 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f84c0070c70 msgr2=0x7f84c0116290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:51:16.434 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:16.435+0000 7f84bf400700 1 --2- 172.21.15.18:0/2702106689 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f84c0070c70 0x7f84c0116290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:51:16.434 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:16.435+0000 7f84bf400700 1 -- 172.21.15.18:0/2702106689 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84b0005040 con 0x7f84c0070360 2024-04-12T20:51:16.435 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:16.435+0000 7f84bf400700 1 --2- 172.21.15.18:0/2702106689 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f84c0070360 0x7f84c0115d50 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f84b0000c00 tx=0x7f84b0009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T20:51:16.435 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:16.435+0000 7f84b7e00700 1 -- 172.21.15.18:0/2702106689 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f84b000e040 con 0x7f84c0070360 2024-04-12T20:51:16.435 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:16.435+0000 7f84b7e00700 1 -- 172.21.15.18:0/2702106689 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f84b000bc10 con 0x7f84c0070360 2024-04-12T20:51:16.435 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:16.435+0000 7f84c6800700 1 -- 172.21.15.18:0/2702106689 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f84c010ff30 con 0x7f84c0070360 2024-04-12T20:51:16.435 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:16.435+0000 7f84b7e00700 1 -- 172.21.15.18:0/2702106689 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f84b001c3f0 con 0x7f84c0070360 2024-04-12T20:51:16.435 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:16.436+0000 7f84c6800700 1 -- 172.21.15.18:0/2702106689 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f84c0110470 con 0x7f84c0070360 2024-04-12T20:51:16.435 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:16.437+0000 7f84c6800700 1 -- 172.21.15.18:0/2702106689 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f84c0071aa0 con 0x7f84c0070360 2024-04-12T20:51:16.436 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:16.438+0000 7f84b7e00700 1 -- 172.21.15.18:0/2702106689 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7f84b000bd80 con 0x7f84c0070360 2024-04-12T20:51:16.437 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:16.439+0000 7f84b7e00700 1 --2- 172.21.15.18:0/2702106689 >> [v2:172.21.15.18:6800/3048894687,v1:172.21.15.18:6801/3048894687] conn(0x7f84a40692a0 0x7f84a406b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T20:51:16.437 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:16.439+0000 7f84b7e00700 1 -- 172.21.15.18:0/2702106689 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4634+0+0 (secure 0 0 0) 0x7f84b0088230 con 0x7f84c0070360 2024-04-12T20:51:16.438 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:16.439+0000 7f84b6a00700 1 --2- 172.21.15.18:0/2702106689 >> [v2:172.21.15.18:6800/3048894687,v1:172.21.15.18:6801/3048894687] conn(0x7f84a40692a0 0x7f84a406b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T20:51:16.438 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:16.440+0000 7f84b6a00700 1 --2- 172.21.15.18:0/2702106689 >> [v2:172.21.15.18:6800/3048894687,v1:172.21.15.18:6801/3048894687] conn(0x7f84a40692a0 0x7f84a406b750 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f84c01115f0 tx=0x7f84a800b040 comp rx=0 tx=0).ready entity=mgr.14212 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T20:51:16.442 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:16.444+0000 7f84b7e00700 1 -- 172.21.15.18:0/2702106689 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7f84b0058340 con 0x7f84c0070360 2024-04-12T20:51:16.531 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:51:16 smithi186 ceph-mon[36597]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-12T20:51:16.532 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:51:16 smithi186 ceph-mon[36597]: from='client.? 172.21.15.18:0/1704798550' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-04-12T20:51:16.532 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:51:16 smithi186 ceph-mon[36597]: from='client.? 172.21.15.18:0/2944524834' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-04-12T20:51:16.665 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:16.667+0000 7f84c6800700 1 -- 172.21.15.18:0/2702106689 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f84c004bed0 con 0x7f84c0070360 2024-04-12T20:51:16.666 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:16.668+0000 7f84b7e00700 1 -- 172.21.15.18:0/2702106689 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f84b0057ed0 con 0x7f84c0070360 2024-04-12T20:51:16.666 INFO:teuthology.orchestra.run.smithi018.stdout:111669149723 2024-04-12T20:51:16.669 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:16.671+0000 7f84c6800700 1 -- 172.21.15.18:0/2702106689 >> [v2:172.21.15.18:6800/3048894687,v1:172.21.15.18:6801/3048894687] conn(0x7f84a40692a0 msgr2=0x7f84a406b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:51:16.669 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:16.671+0000 7f84c6800700 1 --2- 172.21.15.18:0/2702106689 >> [v2:172.21.15.18:6800/3048894687,v1:172.21.15.18:6801/3048894687] conn(0x7f84a40692a0 0x7f84a406b750 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f84c01115f0 tx=0x7f84a800b040 comp rx=0 tx=0).stop 2024-04-12T20:51:16.669 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:16.671+0000 7f84c6800700 1 -- 172.21.15.18:0/2702106689 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f84c0070360 msgr2=0x7f84c0115d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:51:16.669 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:16.671+0000 7f84c6800700 1 --2- 172.21.15.18:0/2702106689 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f84c0070360 0x7f84c0115d50 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f84b0000c00 tx=0x7f84b0009e30 comp rx=0 tx=0).stop 2024-04-12T20:51:16.669 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:16.671+0000 7f84c6800700 1 -- 172.21.15.18:0/2702106689 shutdown_connections 2024-04-12T20:51:16.669 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:16.671+0000 7f84c6800700 1 --2- 172.21.15.18:0/2702106689 >> [v2:172.21.15.18:6800/3048894687,v1:172.21.15.18:6801/3048894687] conn(0x7f84a40692a0 0x7f84a406b750 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:51:16.670 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:16.671+0000 7f84c6800700 1 --2- 172.21.15.18:0/2702106689 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f84c0070360 0x7f84c0115d50 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:51:16.670 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:16.671+0000 7f84c6800700 1 --2- 172.21.15.18:0/2702106689 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f84c0070c70 0x7f84c0116290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:51:16.670 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:16.671+0000 7f84c6800700 1 -- 172.21.15.18:0/2702106689 >> 172.21.15.18:0/2702106689 conn(0x7f84c00f89b0 msgr2=0x7f84c00f9e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T20:51:16.670 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:16.671+0000 7f84c6800700 1 -- 172.21.15.18:0/2702106689 shutdown_connections 2024-04-12T20:51:16.670 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:16.671+0000 7f84c6800700 1 -- 172.21.15.18:0/2702106689 wait complete. 2024-04-12T20:51:16.866 INFO:tasks.cephadm.ceph_manager.ceph:need seq 81604378654 got 81604378658 for osd.2 2024-04-12T20:51:16.867 DEBUG:teuthology.parallel:result is None 2024-04-12T20:51:17.352 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:51:17 smithi018 ceph-mon[28799]: from='client.? 172.21.15.18:0/2702106689' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-04-12T20:51:17.459 INFO:tasks.cephadm.ceph_manager.ceph:need seq 111669149721 got 111669149723 for osd.3 2024-04-12T20:51:17.460 DEBUG:teuthology.parallel:result is None 2024-04-12T20:51:17.531 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:51:17 smithi186 ceph-mon[36597]: from='client.? 172.21.15.18:0/2702106689' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-04-12T20:51:17.792 INFO:tasks.cephadm.ceph_manager.ceph:need seq 146028888082 got 146028888085 for osd.4 2024-04-12T20:51:17.792 DEBUG:teuthology.parallel:result is None 2024-04-12T20:51:17.793 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-04-12T20:51:17.793 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid 1cbe407a-f90d-11ee-bc8d-c7b262605968 -- ceph pg dump --format=json 2024-04-12T20:51:18.281 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/mon.smithi018/config 2024-04-12T20:51:18.433 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:51:18 smithi018 ceph-mon[28799]: pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-12T20:51:18.531 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:51:18 smithi186 ceph-mon[36597]: pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-12T20:51:19.476 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:51:19 smithi018 ceph-mon[28799]: pgmap v195: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-12T20:51:19.531 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:51:19 smithi186 ceph-mon[36597]: pgmap v195: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-12T20:51:20.396 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:20.396+0000 7f1d33a00700 1 -- 172.21.15.18:0/3206081370 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1d2c102de0 msgr2=0x7f1d2c1031b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:51:20.396 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:20.396+0000 7f1d33a00700 1 --2- 172.21.15.18:0/3206081370 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1d2c102de0 0x7f1d2c1031b0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f1d20004940 tx=0x7f1d20009e30 comp rx=0 tx=0).stop 2024-04-12T20:51:20.396 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:20.396+0000 7f1d33a00700 1 -- 172.21.15.18:0/3206081370 shutdown_connections 2024-04-12T20:51:20.396 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:20.396+0000 7f1d33a00700 1 --2- 172.21.15.18:0/3206081370 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1d2c103780 0x7f1d2c0fcdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:51:20.396 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:20.396+0000 7f1d33a00700 1 --2- 172.21.15.18:0/3206081370 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1d2c102de0 0x7f1d2c1031b0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:51:20.396 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:20.396+0000 7f1d33a00700 1 -- 172.21.15.18:0/3206081370 >> 172.21.15.18:0/3206081370 conn(0x7f1d2c0f8970 msgr2=0x7f1d2c0fad80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T20:51:20.397 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:20.397+0000 7f1d33a00700 1 -- 172.21.15.18:0/3206081370 shutdown_connections 2024-04-12T20:51:20.397 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:20.397+0000 7f1d33a00700 1 -- 172.21.15.18:0/3206081370 wait complete. 2024-04-12T20:51:20.397 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:20.397+0000 7f1d33a00700 1 Processor -- start 2024-04-12T20:51:20.397 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:20.398+0000 7f1d33a00700 1 -- start start 2024-04-12T20:51:20.399 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:20.398+0000 7f1d33a00700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1d2c102de0 0x7f1d2c113b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T20:51:20.399 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:20.398+0000 7f1d33a00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1d2c103780 0x7f1d2c1140d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T20:51:20.399 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:20.398+0000 7f1d33a00700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d2c114720 con 0x7f1d2c103780 2024-04-12T20:51:20.399 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:20.398+0000 7f1d33a00700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d2c10db30 con 0x7f1d2c102de0 2024-04-12T20:51:20.399 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:20.398+0000 7f1d2be00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1d2c103780 0x7f1d2c1140d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T20:51:20.399 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:20.399+0000 7f1d2be00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1d2c103780 0x7f1d2c1140d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:48230/0 (socket says 172.21.15.18:48230) 2024-04-12T20:51:20.399 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:20.399+0000 7f1d2be00700 1 -- 172.21.15.18:0/1015026150 learned_addr learned my addr 172.21.15.18:0/1015026150 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T20:51:20.399 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:20.399+0000 7f1d31000700 1 --2- 172.21.15.18:0/1015026150 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1d2c102de0 0x7f1d2c113b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T20:51:20.400 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:20.399+0000 7f1d2be00700 1 -- 172.21.15.18:0/1015026150 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1d2c102de0 msgr2=0x7f1d2c113b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:51:20.400 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:20.399+0000 7f1d2be00700 1 --2- 172.21.15.18:0/1015026150 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1d2c102de0 0x7f1d2c113b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:51:20.400 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:20.399+0000 7f1d2be00700 1 -- 172.21.15.18:0/1015026150 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d20005040 con 0x7f1d2c103780 2024-04-12T20:51:20.400 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:20.399+0000 7f1d2be00700 1 --2- 172.21.15.18:0/1015026150 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1d2c103780 0x7f1d2c1140d0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f1d18002a50 tx=0x7f1d18004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T20:51:20.400 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:20.400+0000 7f1d28c00700 1 -- 172.21.15.18:0/1015026150 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1d18007cc0 con 0x7f1d2c103780 2024-04-12T20:51:20.400 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:20.400+0000 7f1d33a00700 1 -- 172.21.15.18:0/1015026150 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1d2c10de10 con 0x7f1d2c103780 2024-04-12T20:51:20.400 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:20.400+0000 7f1d28c00700 1 -- 172.21.15.18:0/1015026150 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f1d1800f040 con 0x7f1d2c103780 2024-04-12T20:51:20.400 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:20.400+0000 7f1d28c00700 1 -- 172.21.15.18:0/1015026150 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1d1800da40 con 0x7f1d2c103780 2024-04-12T20:51:20.400 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:20.400+0000 7f1d33a00700 1 -- 172.21.15.18:0/1015026150 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1d2c10e3d0 con 0x7f1d2c103780 2024-04-12T20:51:20.401 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:20.401+0000 7f1d33a00700 1 -- 172.21.15.18:0/1015026150 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1d2c1000f0 con 0x7f1d2c103780 2024-04-12T20:51:20.401 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:20.402+0000 7f1d28c00700 1 -- 172.21.15.18:0/1015026150 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7f1d180097c0 con 0x7f1d2c103780 2024-04-12T20:51:20.402 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:20.403+0000 7f1d28c00700 1 --2- 172.21.15.18:0/1015026150 >> [v2:172.21.15.18:6800/3048894687,v1:172.21.15.18:6801/3048894687] conn(0x7f1d140692a0 0x7f1d1406b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T20:51:20.402 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:20.403+0000 7f1d28c00700 1 -- 172.21.15.18:0/1015026150 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4634+0+0 (secure 0 0 0) 0x7f1d18086c40 con 0x7f1d2c103780 2024-04-12T20:51:20.402 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:20.404+0000 7f1d31000700 1 --2- 172.21.15.18:0/1015026150 >> [v2:172.21.15.18:6800/3048894687,v1:172.21.15.18:6801/3048894687] conn(0x7f1d140692a0 0x7f1d1406b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T20:51:20.402 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:20.404+0000 7f1d31000700 1 --2- 172.21.15.18:0/1015026150 >> [v2:172.21.15.18:6800/3048894687,v1:172.21.15.18:6801/3048894687] conn(0x7f1d140692a0 0x7f1d1406b750 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f1d20004f90 tx=0x7f1d20009e30 comp rx=0 tx=0).ready entity=mgr.14212 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T20:51:20.406 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:20.408+0000 7f1d28c00700 1 -- 172.21.15.18:0/1015026150 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7f1d18056ca0 con 0x7f1d2c103780 2024-04-12T20:51:20.624 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:20.625+0000 7f1d33a00700 1 -- 172.21.15.18:0/1015026150 --> [v2:172.21.15.18:6800/3048894687,v1:172.21.15.18:6801/3048894687] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f1d2c10f1e0 con 0x7f1d140692a0 2024-04-12T20:51:20.625 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:20.627+0000 7f1d28c00700 1 -- 172.21.15.18:0/1015026150 <== mgr.14212 v2:172.21.15.18:6800/3048894687 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+27019 (secure 0 0 0) 0x7f1d2c10f1e0 con 0x7f1d140692a0 2024-04-12T20:51:20.626 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-12T20:51:20.628 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:20.630+0000 7f1d33a00700 1 -- 172.21.15.18:0/1015026150 >> [v2:172.21.15.18:6800/3048894687,v1:172.21.15.18:6801/3048894687] conn(0x7f1d140692a0 msgr2=0x7f1d1406b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:51:20.628 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:20.630+0000 7f1d33a00700 1 --2- 172.21.15.18:0/1015026150 >> [v2:172.21.15.18:6800/3048894687,v1:172.21.15.18:6801/3048894687] conn(0x7f1d140692a0 0x7f1d1406b750 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f1d20004f90 tx=0x7f1d20009e30 comp rx=0 tx=0).stop 2024-04-12T20:51:20.628 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:20.630+0000 7f1d33a00700 1 -- 172.21.15.18:0/1015026150 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1d2c103780 msgr2=0x7f1d2c1140d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:51:20.629 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:20.630+0000 7f1d33a00700 1 --2- 172.21.15.18:0/1015026150 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1d2c103780 0x7f1d2c1140d0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f1d18002a50 tx=0x7f1d18004600 comp rx=0 tx=0).stop 2024-04-12T20:51:20.629 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:20.630+0000 7f1d33a00700 1 -- 172.21.15.18:0/1015026150 shutdown_connections 2024-04-12T20:51:20.629 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:20.630+0000 7f1d33a00700 1 --2- 172.21.15.18:0/1015026150 >> [v2:172.21.15.18:6800/3048894687,v1:172.21.15.18:6801/3048894687] conn(0x7f1d140692a0 0x7f1d1406b750 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:51:20.629 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:20.630+0000 7f1d33a00700 1 --2- 172.21.15.18:0/1015026150 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1d2c102de0 0x7f1d2c113b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:51:20.629 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:20.630+0000 7f1d33a00700 1 --2- 172.21.15.18:0/1015026150 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1d2c103780 0x7f1d2c1140d0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:51:20.629 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:20.630+0000 7f1d33a00700 1 -- 172.21.15.18:0/1015026150 >> 172.21.15.18:0/1015026150 conn(0x7f1d2c0f8970 msgr2=0x7f1d2c1073d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T20:51:20.629 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:20.630+0000 7f1d33a00700 1 -- 172.21.15.18:0/1015026150 shutdown_connections 2024-04-12T20:51:20.629 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:20.630+0000 7f1d33a00700 1 -- 172.21.15.18:0/1015026150 wait complete. 2024-04-12T20:51:20.629 INFO:teuthology.orchestra.run.smithi018.stderr:dumped all 2024-04-12T20:51:21.411 INFO:teuthology.orchestra.run.smithi018.stdout:{"pg_ready":true,"pg_map":{"version":195,"stamp":"2024-04-12T20:51:18.868108+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":3,"acting":3,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":3,"num_osds":6,"num_per_pool_osds":6,"num_per_pool_omap_osds":3,"kb":562470912,"kb_used":1788024,"kb_used_data":3876,"kb_used_omap":9,"kb_used_meta":1784054,"kb_avail":560682888,"statfs":{"total":575970213888,"available":574139277312,"internally_reserved":0,"allocated":3969024,"data_stored":2606262,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":10224,"internal_metadata":1826871312},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003187"},"pg_stats":[{"pgid":"1.0","version":"22'117","reported_seq":190,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-12T20:50:17.019483+0000","last_change":"2024-04-12T20:49:42.928259+0000","last_active":"2024-04-12T20:50:17.019483+0000","last_peered":"2024-04-12T20:50:17.019483+0000","last_clean":"2024-04-12T20:50:17.019483+0000","last_became_active":"2024-04-12T20:49:42.927797+0000","last_became_peered":"2024-04-12T20:49:42.927797+0000","last_unstale":"2024-04-12T20:50:17.019483+0000","last_undegraded":"2024-04-12T20:50:17.019483+0000","last_fullsized":"2024-04-12T20:50:17.019483+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T20:48:37.909936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T20:48:37.909936+0000","last_clean_scrub_stamp":"2024-04-12T20:48:37.909936+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T03:39:03.960945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2375680,"data_stored":2361472,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":3,"acting":3,"num_store_stats":4}],"osd_stats":[{"osd":5,"up_from":39,"seq":167503724559,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":297712,"kb_used_data":356,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93447440,"statfs":{"total":95995035648,"available":95690178560,"internally_reserved":0,"allocated":364544,"data_stored":139193,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82799999999999996}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1970000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0409999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1359999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.222}]}]},{"osd":4,"up_from":34,"seq":146028888086,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":297712,"kb_used_data":356,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93447440,"statfs":{"total":95995035648,"available":95690178560,"internally_reserved":0,"allocated":364544,"data_stored":139193,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Apr 12 20:50:44 2024","interfaces":[{"interface":"back","average":{"1min":0.98599999999999999,"5min":0.98599999999999999,"15min":0.98599999999999999},"min":{"1min":0.61099999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"max":{"1min":1.4330000000000001,"5min":1.4330000000000001,"15min":1.4330000000000001},"last":1.367},{"interface":"front","average":{"1min":0.95599999999999996,"5min":0.95599999999999996,"15min":0.95599999999999996},"min":{"1min":0.69299999999999995,"5min":0.69299999999999995,"15min":0.69299999999999995},"max":{"1min":1.345,"5min":1.345,"15min":1.345},"last":1.0840000000000001}]},{"osd":1,"last update":"Fri Apr 12 20:50:44 2024","interfaces":[{"interface":"back","average":{"1min":1.0109999999999999,"5min":1.0109999999999999,"15min":1.0109999999999999},"min":{"1min":0.69299999999999995,"5min":0.69299999999999995,"15min":0.69299999999999995},"max":{"1min":1.456,"5min":1.456,"15min":1.456},"last":1.23},{"interface":"front","average":{"1min":0.96699999999999997,"5min":0.96699999999999997,"15min":0.96699999999999997},"min":{"1min":0.67500000000000004,"5min":0.67500000000000004,"15min":0.67500000000000004},"max":{"1min":1.3740000000000001,"5min":1.3740000000000001,"15min":1.3740000000000001},"last":1.3049999999999999}]},{"osd":2,"last update":"Fri Apr 12 20:50:44 2024","interfaces":[{"interface":"back","average":{"1min":1.0820000000000001,"5min":1.0820000000000001,"15min":1.0820000000000001},"min":{"1min":0.76700000000000002,"5min":0.76700000000000002,"15min":0.76700000000000002},"max":{"1min":1.579,"5min":1.579,"15min":1.579},"last":1.1839999999999999},{"interface":"front","average":{"1min":1.129,"5min":1.129,"15min":1.129},"min":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.76400000000000001},"max":{"1min":1.732,"5min":1.732,"15min":1.732},"last":1.1479999999999999}]},{"osd":3,"last update":"Fri Apr 12 20:50:44 2024","interfaces":[{"interface":"back","average":{"1min":0.94099999999999995,"5min":0.94099999999999995,"15min":0.94099999999999995},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":1.329,"5min":1.329,"15min":1.329},"last":1.0029999999999999},{"interface":"front","average":{"1min":0.873,"5min":0.873,"15min":0.873},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":1.2070000000000001,"5min":1.2070000000000001,"15min":1.2070000000000001},"last":1.0980000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0309999999999999}]}]},{"osd":3,"up_from":26,"seq":111669149724,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":298300,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93446852,"statfs":{"total":95995035648,"available":95689576448,"internally_reserved":0,"allocated":958464,"data_stored":729561,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Apr 12 20:51:13 2024","interfaces":[{"interface":"back","average":{"1min":1.0780000000000001,"5min":1.024,"15min":1.0149999999999999},"min":{"1min":0.70099999999999996,"5min":0.65000000000000002,"15min":0.65000000000000002},"max":{"1min":1.506,"5min":1.506,"15min":1.506},"last":0.98599999999999999},{"interface":"front","average":{"1min":1.0629999999999999,"5min":1.0409999999999999,"15min":1.0369999999999999},"min":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"max":{"1min":1.5600000000000001,"5min":1.5600000000000001,"15min":1.5600000000000001},"last":0.76700000000000002}]},{"osd":1,"last update":"Fri Apr 12 20:51:13 2024","interfaces":[{"interface":"back","average":{"1min":1.1000000000000001,"5min":1.0089999999999999,"15min":0.99399999999999999},"min":{"1min":0.61599999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"max":{"1min":1.3600000000000001,"5min":1.3600000000000001,"15min":1.3600000000000001},"last":1.2370000000000001},{"interface":"front","average":{"1min":1.1120000000000001,"5min":1.0669999999999999,"15min":1.0589999999999999},"min":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"max":{"1min":2.1110000000000002,"5min":2.1110000000000002,"15min":2.1110000000000002},"last":1.071}]},{"osd":2,"last update":"Fri Apr 12 20:51:13 2024","interfaces":[{"interface":"back","average":{"1min":1.0940000000000001,"5min":1.052,"15min":1.0449999999999999},"min":{"1min":0.71599999999999997,"5min":0.66900000000000004,"15min":0.66900000000000004},"max":{"1min":1.351,"5min":1.361,"15min":1.361},"last":1.1859999999999999},{"interface":"front","average":{"1min":1.252,"5min":1.141,"15min":1.123},"min":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.77400000000000002},"max":{"1min":1.577,"5min":1.577,"15min":1.577},"last":1.361}]},{"osd":4,"last update":"Fri Apr 12 20:50:43 2024","interfaces":[{"interface":"back","average":{"1min":1.0680000000000001,"5min":1.0680000000000001,"15min":1.0680000000000001},"min":{"1min":0.5,"5min":0.5,"15min":0.5},"max":{"1min":1.389,"5min":1.389,"15min":1.389},"last":1.165},{"interface":"front","average":{"1min":1.1799999999999999,"5min":1.1799999999999999,"15min":1.1799999999999999},"min":{"1min":0.502,"5min":0.502,"15min":0.502},"max":{"1min":1.9930000000000001,"5min":1.9930000000000001,"15min":1.9930000000000001},"last":0.94599999999999995}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2929999999999999}]}]},{"osd":2,"up_from":19,"seq":81604378659,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":297712,"kb_used_data":356,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93447440,"statfs":{"total":95995035648,"available":95690178560,"internally_reserved":0,"allocated":364544,"data_stored":139193,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Apr 12 20:50:46 2024","interfaces":[{"interface":"back","average":{"1min":0.91000000000000003,"5min":0.90500000000000003,"15min":0.90400000000000003},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":1.8560000000000001,"5min":1.8560000000000001,"15min":1.8560000000000001},"last":0.96599999999999997},{"interface":"front","average":{"1min":0.91700000000000004,"5min":0.90700000000000003,"15min":0.90500000000000003},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":1.429,"5min":1.429,"15min":1.429},"last":1.091}]},{"osd":1,"last update":"Fri Apr 12 20:50:46 2024","interfaces":[{"interface":"back","average":{"1min":1.0840000000000001,"5min":0.94599999999999995,"15min":0.92300000000000004},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":2.5179999999999998,"5min":2.5179999999999998,"15min":2.5179999999999998},"last":1.3640000000000001},{"interface":"front","average":{"1min":1.016,"5min":0.96599999999999997,"15min":0.95799999999999996},"min":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":1.302,"5min":1.302,"15min":1.302},"last":1.141}]},{"osd":3,"last update":"Fri Apr 12 20:51:12 2024","interfaces":[{"interface":"back","average":{"1min":1.2050000000000001,"5min":1.2170000000000001,"15min":1.2190000000000001},"min":{"1min":0.90200000000000002,"5min":0.90200000000000002,"15min":0.90200000000000002},"max":{"1min":1.5029999999999999,"5min":1.653,"15min":1.653},"last":1.274},{"interface":"front","average":{"1min":1.3220000000000001,"5min":1.1739999999999999,"15min":1.1499999999999999},"min":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"max":{"1min":3.2799999999999998,"5min":3.2799999999999998,"15min":3.2799999999999998},"last":1.2310000000000001}]},{"osd":4,"last update":"Fri Apr 12 20:50:46 2024","interfaces":[{"interface":"back","average":{"1min":1.2589999999999999,"5min":1.2589999999999999,"15min":1.2589999999999999},"min":{"1min":0.78800000000000003,"5min":0.78800000000000003,"15min":0.78800000000000003},"max":{"1min":2.2559999999999998,"5min":2.2559999999999998,"15min":2.2559999999999998},"last":1.51},{"interface":"front","average":{"1min":1.244,"5min":1.244,"15min":1.244},"min":{"1min":0.95599999999999996,"5min":0.95599999999999996,"15min":0.95599999999999996},"max":{"1min":1.671,"5min":1.671,"15min":1.671},"last":1.4239999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.323}]}]},{"osd":0,"up_from":9,"seq":38654705714,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":298296,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93446856,"statfs":{"total":95995035648,"available":95689580544,"internally_reserved":0,"allocated":958464,"data_stored":729561,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Apr 12 20:51:05 2024","interfaces":[{"interface":"back","average":{"1min":0.88700000000000001,"5min":0.78800000000000003,"15min":0.754},"min":{"1min":0.48999999999999999,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":1.841,"5min":1.841,"15min":1.841},"last":0.90100000000000002},{"interface":"front","average":{"1min":0.99099999999999999,"5min":0.88900000000000001,"15min":0.85799999999999998},"min":{"1min":0.52200000000000002,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":1.3680000000000001,"5min":2.1099999999999999,"15min":2.1099999999999999},"last":0.74399999999999999}]},{"osd":2,"last update":"Fri Apr 12 20:50:42 2024","interfaces":[{"interface":"back","average":{"1min":0.93300000000000005,"5min":0.95299999999999996,"15min":0.95599999999999996},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":1.3740000000000001,"5min":1.3740000000000001,"15min":1.3740000000000001},"last":0.95299999999999996},{"interface":"front","average":{"1min":1.038,"5min":1.0169999999999999,"15min":1.0129999999999999},"min":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":1.431,"5min":1.431,"15min":1.431},"last":0.83199999999999996}]},{"osd":3,"last update":"Fri Apr 12 20:50:12 2024","interfaces":[{"interface":"back","average":{"1min":1.1970000000000001,"5min":1.1970000000000001,"15min":1.1970000000000001},"min":{"1min":0.93100000000000005,"5min":0.93100000000000005,"15min":0.93100000000000005},"max":{"1min":1.4990000000000001,"5min":1.4990000000000001,"15min":1.4990000000000001},"last":1.1739999999999999},{"interface":"front","average":{"1min":1.222,"5min":1.222,"15min":1.222},"min":{"1min":0.85499999999999998,"5min":0.85499999999999998,"15min":0.85499999999999998},"max":{"1min":1.613,"5min":1.613,"15min":1.613},"last":1.105}]},{"osd":4,"last update":"Fri Apr 12 20:50:44 2024","interfaces":[{"interface":"back","average":{"1min":1.2589999999999999,"5min":1.2589999999999999,"15min":1.2589999999999999},"min":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"max":{"1min":2.0070000000000001,"5min":2.0070000000000001,"15min":2.0070000000000001},"last":1.121},{"interface":"front","average":{"1min":1.341,"5min":1.341,"15min":1.341},"min":{"1min":0.97999999999999998,"5min":0.97999999999999998,"15min":0.97999999999999998},"max":{"1min":1.8899999999999999,"5min":1.8899999999999999,"15min":1.8899999999999999},"last":1.0700000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2150000000000001}]}]},{"osd":1,"up_from":14,"seq":60129542185,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":298292,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93446860,"statfs":{"total":95995035648,"available":95689584640,"internally_reserved":0,"allocated":958464,"data_stored":729561,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Apr 12 20:51:04 2024","interfaces":[{"interface":"back","average":{"1min":0.87,"5min":0.79600000000000004,"15min":0.76600000000000001},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":1.234,"5min":1.234,"15min":1.234},"last":0.91000000000000003},{"interface":"front","average":{"1min":0.95599999999999996,"5min":0.85799999999999998,"15min":0.82199999999999995},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":2.077,"5min":2.077,"15min":2.077},"last":0.875}]},{"osd":2,"last update":"Fri Apr 12 20:50:41 2024","interfaces":[{"interface":"back","average":{"1min":0.96299999999999997,"5min":0.91000000000000003,"15min":0.90100000000000002},"min":{"1min":0.501,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":1.591,"5min":1.591,"15min":1.591},"last":0.96799999999999997},{"interface":"front","average":{"1min":1.093,"5min":1.0329999999999999,"15min":1.0229999999999999},"min":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":1.7230000000000001,"5min":1.7230000000000001,"15min":1.7230000000000001},"last":1.091}]},{"osd":3,"last update":"Fri Apr 12 20:50:13 2024","interfaces":[{"interface":"back","average":{"1min":1.208,"5min":1.208,"15min":1.208},"min":{"1min":0.98199999999999998,"5min":0.98199999999999998,"15min":0.98199999999999998},"max":{"1min":1.738,"5min":1.738,"15min":1.738},"last":1.306},{"interface":"front","average":{"1min":1.2350000000000001,"5min":1.2350000000000001,"15min":1.2350000000000001},"min":{"1min":0.86299999999999999,"5min":0.86299999999999999,"15min":0.86299999999999999},"max":{"1min":1.925,"5min":1.925,"15min":1.925},"last":1.2689999999999999}]},{"osd":4,"last update":"Fri Apr 12 20:50:45 2024","interfaces":[{"interface":"back","average":{"1min":1.181,"5min":1.181,"15min":1.181},"min":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"max":{"1min":1.786,"5min":1.786,"15min":1.786},"last":1.1739999999999999},{"interface":"front","average":{"1min":1.5409999999999999,"5min":1.5409999999999999,"15min":1.5409999999999999},"min":{"1min":0.753,"5min":0.753,"15min":0.753},"max":{"1min":8.1099999999999994,"5min":8.1099999999999994,"15min":8.1099999999999994},"last":1.661}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.6220000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-12T20:51:21.413 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid 1cbe407a-f90d-11ee-bc8d-c7b262605968 -- ceph pg dump --format=json 2024-04-12T20:51:21.878 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/mon.smithi018/config 2024-04-12T20:51:22.029 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:51:21 smithi018 ceph-mon[28799]: from='client.14412 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-12T20:51:22.029 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:51:21 smithi018 ceph-mon[28799]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-12T20:51:22.281 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:51:21 smithi186 ceph-mon[36597]: from='client.14412 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-12T20:51:22.281 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:51:21 smithi186 ceph-mon[36597]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-12T20:51:23.102 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:51:22 smithi018 ceph-mon[28799]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-12T20:51:23.281 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:51:22 smithi186 ceph-mon[36597]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-12T20:51:24.047 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:24.048+0000 7f678de00700 1 -- 172.21.15.18:0/1798543129 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6788105800 msgr2=0x7f67880fee70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:51:24.047 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:24.048+0000 7f678de00700 1 --2- 172.21.15.18:0/1798543129 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6788105800 0x7f67880fee70 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f6774004940 tx=0x7f6774009e30 comp rx=0 tx=0).stop 2024-04-12T20:51:24.047 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:24.048+0000 7f678de00700 1 -- 172.21.15.18:0/1798543129 shutdown_connections 2024-04-12T20:51:24.047 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:24.048+0000 7f678de00700 1 --2- 172.21.15.18:0/1798543129 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6788105800 0x7f67880fee70 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:51:24.047 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:24.048+0000 7f678de00700 1 --2- 172.21.15.18:0/1798543129 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6788104e60 0x7f6788105230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:51:24.047 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:24.048+0000 7f678de00700 1 -- 172.21.15.18:0/1798543129 >> 172.21.15.18:0/1798543129 conn(0x7f67880fab10 msgr2=0x7f67880fcef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T20:51:24.048 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:24.049+0000 7f678de00700 1 -- 172.21.15.18:0/1798543129 shutdown_connections 2024-04-12T20:51:24.048 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:24.049+0000 7f678de00700 1 -- 172.21.15.18:0/1798543129 wait complete. 2024-04-12T20:51:24.048 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:51:23 smithi018 ceph-mon[28799]: pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-12T20:51:24.048 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:51:23 smithi018 ceph-mon[28799]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-12T20:51:24.048 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:51:23 smithi018 ceph-mon[28799]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-12T20:51:24.048 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:51:23 smithi018 ceph-mon[28799]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' 2024-04-12T20:51:24.049 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:24.050+0000 7f678de00700 1 Processor -- start 2024-04-12T20:51:24.049 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:24.050+0000 7f678de00700 1 -- start start 2024-04-12T20:51:24.050 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:24.050+0000 7f678de00700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6788104e60 0x7f6788115b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T20:51:24.050 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:24.050+0000 7f678de00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6788105800 0x7f6788116070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T20:51:24.050 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:24.050+0000 7f678de00700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67881166c0 con 0x7f6788105800 2024-04-12T20:51:24.050 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:24.050+0000 7f678de00700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f678810fab0 con 0x7f6788104e60 2024-04-12T20:51:24.050 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:24.051+0000 7f6787e00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6788105800 0x7f6788116070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T20:51:24.050 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:24.051+0000 7f6787e00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6788105800 0x7f6788116070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:43688/0 (socket says 172.21.15.18:43688) 2024-04-12T20:51:24.050 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:24.051+0000 7f6787e00700 1 -- 172.21.15.18:0/297285224 learned_addr learned my addr 172.21.15.18:0/297285224 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T20:51:24.050 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:24.051+0000 7f678ca00700 1 --2- 172.21.15.18:0/297285224 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6788104e60 0x7f6788115b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T20:51:24.050 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:24.051+0000 7f6787e00700 1 -- 172.21.15.18:0/297285224 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6788104e60 msgr2=0x7f6788115b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:51:24.050 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:24.051+0000 7f6787e00700 1 --2- 172.21.15.18:0/297285224 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6788104e60 0x7f6788115b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:51:24.050 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:24.051+0000 7f6787e00700 1 -- 172.21.15.18:0/297285224 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6774005040 con 0x7f6788105800 2024-04-12T20:51:24.050 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:24.051+0000 7f6787e00700 1 --2- 172.21.15.18:0/297285224 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6788105800 0x7f6788116070 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f67740040d0 tx=0x7f6774009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T20:51:24.051 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:24.052+0000 7f6784c00700 1 -- 172.21.15.18:0/297285224 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f677400e030 con 0x7f6788105800 2024-04-12T20:51:24.051 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:24.052+0000 7f678de00700 1 -- 172.21.15.18:0/297285224 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f678810fd30 con 0x7f6788105800 2024-04-12T20:51:24.051 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:24.052+0000 7f6784c00700 1 -- 172.21.15.18:0/297285224 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f6774013c00 con 0x7f6788105800 2024-04-12T20:51:24.051 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:24.052+0000 7f6784c00700 1 -- 172.21.15.18:0/297285224 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f677400b650 con 0x7f6788105800 2024-04-12T20:51:24.051 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:24.052+0000 7f678de00700 1 -- 172.21.15.18:0/297285224 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f67881102f0 con 0x7f6788105800 2024-04-12T20:51:24.051 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:24.053+0000 7f678de00700 1 -- 172.21.15.18:0/297285224 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6788102170 con 0x7f6788105800 2024-04-12T20:51:24.052 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:24.054+0000 7f6784c00700 1 -- 172.21.15.18:0/297285224 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7f677400b8c0 con 0x7f6788105800 2024-04-12T20:51:24.053 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:24.055+0000 7f6784c00700 1 --2- 172.21.15.18:0/297285224 >> [v2:172.21.15.18:6800/3048894687,v1:172.21.15.18:6801/3048894687] conn(0x7f67700692b0 0x7f677006b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T20:51:24.053 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:24.055+0000 7f6784c00700 1 -- 172.21.15.18:0/297285224 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4634+0+0 (secure 0 0 0) 0x7f6774088bd0 con 0x7f6788105800 2024-04-12T20:51:24.053 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:24.055+0000 7f678ca00700 1 --2- 172.21.15.18:0/297285224 >> [v2:172.21.15.18:6800/3048894687,v1:172.21.15.18:6801/3048894687] conn(0x7f67700692b0 0x7f677006b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T20:51:24.054 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:24.056+0000 7f678ca00700 1 --2- 172.21.15.18:0/297285224 >> [v2:172.21.15.18:6800/3048894687,v1:172.21.15.18:6801/3048894687] conn(0x7f67700692b0 0x7f677006b760 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f677c009fd0 tx=0x7f677c00b040 comp rx=0 tx=0).ready entity=mgr.14212 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T20:51:24.058 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:24.060+0000 7f6784c00700 1 -- 172.21.15.18:0/297285224 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7f6774058cb0 con 0x7f6788105800 2024-04-12T20:51:24.267 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:24.269+0000 7f678de00700 1 -- 172.21.15.18:0/297285224 --> [v2:172.21.15.18:6800/3048894687,v1:172.21.15.18:6801/3048894687] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f678805f660 con 0x7f67700692b0 2024-04-12T20:51:24.271 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:24.274+0000 7f6784c00700 1 -- 172.21.15.18:0/297285224 <== mgr.14212 v2:172.21.15.18:6800/3048894687 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+27761 (secure 0 0 0) 0x7f678805f660 con 0x7f67700692b0 2024-04-12T20:51:24.272 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-12T20:51:24.275 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:24.277+0000 7f678de00700 1 -- 172.21.15.18:0/297285224 >> [v2:172.21.15.18:6800/3048894687,v1:172.21.15.18:6801/3048894687] conn(0x7f67700692b0 msgr2=0x7f677006b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:51:24.275 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:24.277+0000 7f678de00700 1 --2- 172.21.15.18:0/297285224 >> [v2:172.21.15.18:6800/3048894687,v1:172.21.15.18:6801/3048894687] conn(0x7f67700692b0 0x7f677006b760 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f677c009fd0 tx=0x7f677c00b040 comp rx=0 tx=0).stop 2024-04-12T20:51:24.275 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:24.277+0000 7f678de00700 1 -- 172.21.15.18:0/297285224 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6788105800 msgr2=0x7f6788116070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:51:24.275 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:24.277+0000 7f678de00700 1 --2- 172.21.15.18:0/297285224 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6788105800 0x7f6788116070 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f67740040d0 tx=0x7f6774009e30 comp rx=0 tx=0).stop 2024-04-12T20:51:24.275 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:24.277+0000 7f678de00700 1 -- 172.21.15.18:0/297285224 shutdown_connections 2024-04-12T20:51:24.276 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:24.277+0000 7f678de00700 1 --2- 172.21.15.18:0/297285224 >> [v2:172.21.15.18:6800/3048894687,v1:172.21.15.18:6801/3048894687] conn(0x7f67700692b0 0x7f677006b760 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:51:24.276 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:24.277+0000 7f678de00700 1 --2- 172.21.15.18:0/297285224 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6788104e60 0x7f6788115b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:51:24.276 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:24.277+0000 7f678de00700 1 --2- 172.21.15.18:0/297285224 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6788105800 0x7f6788116070 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:51:24.276 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:24.277+0000 7f678de00700 1 -- 172.21.15.18:0/297285224 >> 172.21.15.18:0/297285224 conn(0x7f67880fab10 msgr2=0x7f67880fd8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T20:51:24.276 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:24.277+0000 7f678de00700 1 -- 172.21.15.18:0/297285224 shutdown_connections 2024-04-12T20:51:24.276 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:24.277+0000 7f678de00700 1 -- 172.21.15.18:0/297285224 wait complete. 2024-04-12T20:51:24.277 INFO:teuthology.orchestra.run.smithi018.stderr:dumped all 2024-04-12T20:51:24.281 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:51:23 smithi186 ceph-mon[36597]: pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-12T20:51:24.281 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:51:23 smithi186 ceph-mon[36597]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-12T20:51:24.281 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:51:23 smithi186 ceph-mon[36597]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-12T20:51:24.281 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:51:23 smithi186 ceph-mon[36597]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' 2024-04-12T20:51:24.987 INFO:teuthology.orchestra.run.smithi018.stdout:{"pg_ready":true,"pg_map":{"version":197,"stamp":"2024-04-12T20:51:22.869131+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":3,"acting":3,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":3,"num_osds":6,"num_per_pool_osds":6,"num_per_pool_omap_osds":3,"kb":562470912,"kb_used":1788024,"kb_used_data":3876,"kb_used_omap":9,"kb_used_meta":1784054,"kb_avail":560682888,"statfs":{"total":575970213888,"available":574139277312,"internally_reserved":0,"allocated":3969024,"data_stored":2606262,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":10224,"internal_metadata":1826871312},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003175"},"pg_stats":[{"pgid":"1.0","version":"22'117","reported_seq":190,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-12T20:50:17.019483+0000","last_change":"2024-04-12T20:49:42.928259+0000","last_active":"2024-04-12T20:50:17.019483+0000","last_peered":"2024-04-12T20:50:17.019483+0000","last_clean":"2024-04-12T20:50:17.019483+0000","last_became_active":"2024-04-12T20:49:42.927797+0000","last_became_peered":"2024-04-12T20:49:42.927797+0000","last_unstale":"2024-04-12T20:50:17.019483+0000","last_undegraded":"2024-04-12T20:50:17.019483+0000","last_fullsized":"2024-04-12T20:50:17.019483+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T20:48:37.909936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T20:48:37.909936+0000","last_clean_scrub_stamp":"2024-04-12T20:48:37.909936+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-14T03:39:03.960945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2375680,"data_stored":2361472,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":3,"acting":3,"num_store_stats":4}],"osd_stats":[{"osd":5,"up_from":39,"seq":167503724559,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":297712,"kb_used_data":356,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93447440,"statfs":{"total":95995035648,"available":95690178560,"internally_reserved":0,"allocated":364544,"data_stored":139193,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82799999999999996}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1970000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0409999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1359999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.222}]}]},{"osd":4,"up_from":34,"seq":146028888086,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":297712,"kb_used_data":356,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93447440,"statfs":{"total":95995035648,"available":95690178560,"internally_reserved":0,"allocated":364544,"data_stored":139193,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Apr 12 20:50:44 2024","interfaces":[{"interface":"back","average":{"1min":0.98599999999999999,"5min":0.98599999999999999,"15min":0.98599999999999999},"min":{"1min":0.61099999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"max":{"1min":1.4330000000000001,"5min":1.4330000000000001,"15min":1.4330000000000001},"last":1.367},{"interface":"front","average":{"1min":0.95599999999999996,"5min":0.95599999999999996,"15min":0.95599999999999996},"min":{"1min":0.69299999999999995,"5min":0.69299999999999995,"15min":0.69299999999999995},"max":{"1min":1.345,"5min":1.345,"15min":1.345},"last":1.0840000000000001}]},{"osd":1,"last update":"Fri Apr 12 20:50:44 2024","interfaces":[{"interface":"back","average":{"1min":1.0109999999999999,"5min":1.0109999999999999,"15min":1.0109999999999999},"min":{"1min":0.69299999999999995,"5min":0.69299999999999995,"15min":0.69299999999999995},"max":{"1min":1.456,"5min":1.456,"15min":1.456},"last":1.23},{"interface":"front","average":{"1min":0.96699999999999997,"5min":0.96699999999999997,"15min":0.96699999999999997},"min":{"1min":0.67500000000000004,"5min":0.67500000000000004,"15min":0.67500000000000004},"max":{"1min":1.3740000000000001,"5min":1.3740000000000001,"15min":1.3740000000000001},"last":1.3049999999999999}]},{"osd":2,"last update":"Fri Apr 12 20:50:44 2024","interfaces":[{"interface":"back","average":{"1min":1.0820000000000001,"5min":1.0820000000000001,"15min":1.0820000000000001},"min":{"1min":0.76700000000000002,"5min":0.76700000000000002,"15min":0.76700000000000002},"max":{"1min":1.579,"5min":1.579,"15min":1.579},"last":1.1839999999999999},{"interface":"front","average":{"1min":1.129,"5min":1.129,"15min":1.129},"min":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.76400000000000001},"max":{"1min":1.732,"5min":1.732,"15min":1.732},"last":1.1479999999999999}]},{"osd":3,"last update":"Fri Apr 12 20:50:44 2024","interfaces":[{"interface":"back","average":{"1min":0.94099999999999995,"5min":0.94099999999999995,"15min":0.94099999999999995},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":1.329,"5min":1.329,"15min":1.329},"last":1.0029999999999999},{"interface":"front","average":{"1min":0.873,"5min":0.873,"15min":0.873},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":1.2070000000000001,"5min":1.2070000000000001,"15min":1.2070000000000001},"last":1.0980000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0309999999999999}]}]},{"osd":3,"up_from":26,"seq":111669149725,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":298300,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93446852,"statfs":{"total":95995035648,"available":95689576448,"internally_reserved":0,"allocated":958464,"data_stored":729561,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Apr 12 20:51:13 2024","interfaces":[{"interface":"back","average":{"1min":1.0780000000000001,"5min":1.024,"15min":1.0149999999999999},"min":{"1min":0.70099999999999996,"5min":0.65000000000000002,"15min":0.65000000000000002},"max":{"1min":1.506,"5min":1.506,"15min":1.506},"last":1.577},{"interface":"front","average":{"1min":1.0629999999999999,"5min":1.0409999999999999,"15min":1.0369999999999999},"min":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"max":{"1min":1.5600000000000001,"5min":1.5600000000000001,"15min":1.5600000000000001},"last":1.1839999999999999}]},{"osd":1,"last update":"Fri Apr 12 20:51:13 2024","interfaces":[{"interface":"back","average":{"1min":1.1000000000000001,"5min":1.0089999999999999,"15min":0.99399999999999999},"min":{"1min":0.61599999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"max":{"1min":1.3600000000000001,"5min":1.3600000000000001,"15min":1.3600000000000001},"last":1.456},{"interface":"front","average":{"1min":1.1120000000000001,"5min":1.0669999999999999,"15min":1.0589999999999999},"min":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"max":{"1min":2.1110000000000002,"5min":2.1110000000000002,"15min":2.1110000000000002},"last":1.036}]},{"osd":2,"last update":"Fri Apr 12 20:51:13 2024","interfaces":[{"interface":"back","average":{"1min":1.0940000000000001,"5min":1.052,"15min":1.0449999999999999},"min":{"1min":0.71599999999999997,"5min":0.66900000000000004,"15min":0.66900000000000004},"max":{"1min":1.351,"5min":1.361,"15min":1.361},"last":1.337},{"interface":"front","average":{"1min":1.252,"5min":1.141,"15min":1.123},"min":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.77400000000000002},"max":{"1min":1.577,"5min":1.577,"15min":1.577},"last":1.518}]},{"osd":4,"last update":"Fri Apr 12 20:50:43 2024","interfaces":[{"interface":"back","average":{"1min":1.0680000000000001,"5min":1.0680000000000001,"15min":1.0680000000000001},"min":{"1min":0.5,"5min":0.5,"15min":0.5},"max":{"1min":1.389,"5min":1.389,"15min":1.389},"last":1.1160000000000001},{"interface":"front","average":{"1min":1.1799999999999999,"5min":1.1799999999999999,"15min":1.1799999999999999},"min":{"1min":0.502,"5min":0.502,"15min":0.502},"max":{"1min":1.9930000000000001,"5min":1.9930000000000001,"15min":1.9930000000000001},"last":0.93600000000000005}]},{"osd":5,"last update":"Fri Apr 12 20:51:18 2024","interfaces":[{"interface":"back","average":{"1min":1.254,"5min":1.254,"15min":1.254},"min":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":1.6359999999999999,"5min":1.6359999999999999,"15min":1.6359999999999999},"last":1.3560000000000001},{"interface":"front","average":{"1min":1.056,"5min":1.056,"15min":1.056},"min":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":1.492,"5min":1.492,"15min":1.492},"last":1.0609999999999999}]}]},{"osd":2,"up_from":19,"seq":81604378659,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":297712,"kb_used_data":356,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93447440,"statfs":{"total":95995035648,"available":95690178560,"internally_reserved":0,"allocated":364544,"data_stored":139193,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Apr 12 20:50:46 2024","interfaces":[{"interface":"back","average":{"1min":0.91000000000000003,"5min":0.90500000000000003,"15min":0.90400000000000003},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":1.8560000000000001,"5min":1.8560000000000001,"15min":1.8560000000000001},"last":0.96599999999999997},{"interface":"front","average":{"1min":0.91700000000000004,"5min":0.90700000000000003,"15min":0.90500000000000003},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":1.429,"5min":1.429,"15min":1.429},"last":1.091}]},{"osd":1,"last update":"Fri Apr 12 20:50:46 2024","interfaces":[{"interface":"back","average":{"1min":1.0840000000000001,"5min":0.94599999999999995,"15min":0.92300000000000004},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":2.5179999999999998,"5min":2.5179999999999998,"15min":2.5179999999999998},"last":1.3640000000000001},{"interface":"front","average":{"1min":1.016,"5min":0.96599999999999997,"15min":0.95799999999999996},"min":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":1.302,"5min":1.302,"15min":1.302},"last":1.141}]},{"osd":3,"last update":"Fri Apr 12 20:51:12 2024","interfaces":[{"interface":"back","average":{"1min":1.2050000000000001,"5min":1.2170000000000001,"15min":1.2190000000000001},"min":{"1min":0.90200000000000002,"5min":0.90200000000000002,"15min":0.90200000000000002},"max":{"1min":1.5029999999999999,"5min":1.653,"15min":1.653},"last":1.274},{"interface":"front","average":{"1min":1.3220000000000001,"5min":1.1739999999999999,"15min":1.1499999999999999},"min":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"max":{"1min":3.2799999999999998,"5min":3.2799999999999998,"15min":3.2799999999999998},"last":1.2310000000000001}]},{"osd":4,"last update":"Fri Apr 12 20:50:46 2024","interfaces":[{"interface":"back","average":{"1min":1.2589999999999999,"5min":1.2589999999999999,"15min":1.2589999999999999},"min":{"1min":0.78800000000000003,"5min":0.78800000000000003,"15min":0.78800000000000003},"max":{"1min":2.2559999999999998,"5min":2.2559999999999998,"15min":2.2559999999999998},"last":1.51},{"interface":"front","average":{"1min":1.244,"5min":1.244,"15min":1.244},"min":{"1min":0.95599999999999996,"5min":0.95599999999999996,"15min":0.95599999999999996},"max":{"1min":1.671,"5min":1.671,"15min":1.671},"last":1.4239999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.323}]}]},{"osd":0,"up_from":9,"seq":38654705715,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":298296,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93446856,"statfs":{"total":95995035648,"available":95689580544,"internally_reserved":0,"allocated":958464,"data_stored":729561,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Apr 12 20:51:05 2024","interfaces":[{"interface":"back","average":{"1min":0.88700000000000001,"5min":0.78800000000000003,"15min":0.754},"min":{"1min":0.48999999999999999,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":1.841,"5min":1.841,"15min":1.841},"last":1.24},{"interface":"front","average":{"1min":0.99099999999999999,"5min":0.88900000000000001,"15min":0.85799999999999998},"min":{"1min":0.52200000000000002,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":1.3680000000000001,"5min":2.1099999999999999,"15min":2.1099999999999999},"last":1.0820000000000001}]},{"osd":2,"last update":"Fri Apr 12 20:50:42 2024","interfaces":[{"interface":"back","average":{"1min":0.93300000000000005,"5min":0.95299999999999996,"15min":0.95599999999999996},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":1.3740000000000001,"5min":1.3740000000000001,"15min":1.3740000000000001},"last":1.1339999999999999},{"interface":"front","average":{"1min":1.038,"5min":1.0169999999999999,"15min":1.0129999999999999},"min":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":1.431,"5min":1.431,"15min":1.431},"last":1.6579999999999999}]},{"osd":3,"last update":"Fri Apr 12 20:51:14 2024","interfaces":[{"interface":"back","average":{"1min":1.294,"5min":1.216,"15min":1.2030000000000001},"min":{"1min":0.95999999999999996,"5min":0.93100000000000005,"15min":0.93100000000000005},"max":{"1min":1.6599999999999999,"5min":1.6599999999999999,"15min":1.6599999999999999},"last":1.6040000000000001},{"interface":"front","average":{"1min":1.276,"5min":1.232,"15min":1.2250000000000001},"min":{"1min":0.93000000000000005,"5min":0.85499999999999998,"15min":0.85499999999999998},"max":{"1min":1.6930000000000001,"5min":1.6930000000000001,"15min":1.6930000000000001},"last":1.405}]},{"osd":4,"last update":"Fri Apr 12 20:50:44 2024","interfaces":[{"interface":"back","average":{"1min":1.2589999999999999,"5min":1.2589999999999999,"15min":1.2589999999999999},"min":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"max":{"1min":2.0070000000000001,"5min":2.0070000000000001,"15min":2.0070000000000001},"last":1.5569999999999999},{"interface":"front","average":{"1min":1.341,"5min":1.341,"15min":1.341},"min":{"1min":0.97999999999999998,"5min":0.97999999999999998,"15min":0.97999999999999998},"max":{"1min":1.8899999999999999,"5min":1.8899999999999999,"15min":1.8899999999999999},"last":1.3069999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.722}]}]},{"osd":1,"up_from":14,"seq":60129542186,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":298292,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93446860,"statfs":{"total":95995035648,"available":95689584640,"internally_reserved":0,"allocated":958464,"data_stored":729561,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Apr 12 20:51:04 2024","interfaces":[{"interface":"back","average":{"1min":0.87,"5min":0.79600000000000004,"15min":0.76600000000000001},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":1.234,"5min":1.234,"15min":1.234},"last":0.875},{"interface":"front","average":{"1min":0.95599999999999996,"5min":0.85799999999999998,"15min":0.82199999999999995},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":2.077,"5min":2.077,"15min":2.077},"last":0.61099999999999999}]},{"osd":2,"last update":"Fri Apr 12 20:50:41 2024","interfaces":[{"interface":"back","average":{"1min":0.96299999999999997,"5min":0.91000000000000003,"15min":0.90100000000000002},"min":{"1min":0.501,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":1.591,"5min":1.591,"15min":1.591},"last":0.754},{"interface":"front","average":{"1min":1.093,"5min":1.0329999999999999,"15min":1.0229999999999999},"min":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":1.7230000000000001,"5min":1.7230000000000001,"15min":1.7230000000000001},"last":1.0349999999999999}]},{"osd":3,"last update":"Fri Apr 12 20:51:15 2024","interfaces":[{"interface":"back","average":{"1min":1.2889999999999999,"5min":1.224,"15min":1.2130000000000001},"min":{"1min":0.96599999999999997,"5min":0.96599999999999997,"15min":0.96599999999999997},"max":{"1min":1.909,"5min":1.909,"15min":1.909},"last":1.3480000000000001},{"interface":"front","average":{"1min":1.1930000000000001,"5min":1.226,"15min":1.232},"min":{"1min":0.66800000000000004,"5min":0.66800000000000004,"15min":0.66800000000000004},"max":{"1min":1.573,"5min":1.925,"15min":1.925},"last":1.3149999999999999}]},{"osd":4,"last update":"Fri Apr 12 20:50:45 2024","interfaces":[{"interface":"back","average":{"1min":1.181,"5min":1.181,"15min":1.181},"min":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"max":{"1min":1.786,"5min":1.786,"15min":1.786},"last":0.96099999999999997},{"interface":"front","average":{"1min":1.5409999999999999,"5min":1.5409999999999999,"15min":1.5409999999999999},"min":{"1min":0.753,"5min":0.753,"15min":0.753},"max":{"1min":8.1099999999999994,"5min":8.1099999999999994,"15min":8.1099999999999994},"last":1.119}]},{"osd":5,"last update":"Fri Apr 12 20:51:18 2024","interfaces":[{"interface":"back","average":{"1min":1.3109999999999999,"5min":1.3109999999999999,"15min":1.3109999999999999},"min":{"1min":0.79000000000000004,"5min":0.79000000000000004,"15min":0.79000000000000004},"max":{"1min":1.8029999999999999,"5min":1.8029999999999999,"15min":1.8029999999999999},"last":1.1539999999999999},{"interface":"front","average":{"1min":1.244,"5min":1.244,"15min":1.244},"min":{"1min":0.73699999999999999,"5min":0.73699999999999999,"15min":0.73699999999999999},"max":{"1min":1.585,"5min":1.585,"15min":1.585},"last":1.2350000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-12T20:51:24.989 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-04-12T20:51:24.989 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-04-12T20:51:24.989 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy 2024-04-12T20:51:24.989 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid 1cbe407a-f90d-11ee-bc8d-c7b262605968 -- ceph health --format=json 2024-04-12T20:51:25.500 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/mon.smithi018/config 2024-04-12T20:51:26.268 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:51:25 smithi018 ceph-mon[28799]: from='client.14416 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-12T20:51:26.268 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:51:25 smithi018 ceph-mon[28799]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-12T20:51:26.281 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:51:25 smithi186 ceph-mon[36597]: from='client.14416 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-12T20:51:26.281 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:51:25 smithi186 ceph-mon[36597]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-12T20:51:27.869 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:27.870+0000 7f0029000700 1 -- 172.21.15.18:0/1570663066 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0024109430 msgr2=0x7f0024109800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:51:27.869 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:27.870+0000 7f0029000700 1 --2- 172.21.15.18:0/1570663066 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0024109430 0x7f0024109800 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f0014004940 tx=0x7f0014009e30 comp rx=0 tx=0).stop 2024-04-12T20:51:27.869 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:27.870+0000 7f0029000700 1 -- 172.21.15.18:0/1570663066 shutdown_connections 2024-04-12T20:51:27.869 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:27.870+0000 7f0029000700 1 --2- 172.21.15.18:0/1570663066 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0024109dd0 0x7f0024103440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:51:27.869 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:27.870+0000 7f0029000700 1 --2- 172.21.15.18:0/1570663066 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0024109430 0x7f0024109800 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:51:27.869 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:27.870+0000 7f0029000700 1 -- 172.21.15.18:0/1570663066 >> 172.21.15.18:0/1570663066 conn(0x7f0024072c00 msgr2=0x7f0024075010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T20:51:27.869 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:27.870+0000 7f0029000700 1 -- 172.21.15.18:0/1570663066 shutdown_connections 2024-04-12T20:51:27.869 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:27.870+0000 7f0029000700 1 -- 172.21.15.18:0/1570663066 wait complete. 2024-04-12T20:51:27.870 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:27.871+0000 7f0029000700 1 Processor -- start 2024-04-12T20:51:27.870 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:27.872+0000 7f0029000700 1 -- start start 2024-04-12T20:51:27.870 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:27.872+0000 7f0029000700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0024109430 0x7f0024119f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T20:51:27.870 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:27.872+0000 7f0029000700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0024109dd0 0x7f002411a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T20:51:27.870 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:27.872+0000 7f0029000700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f002411aad0 con 0x7f0024109430 2024-04-12T20:51:27.870 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:27.872+0000 7f0029000700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0024114100 con 0x7f0024109dd0 2024-04-12T20:51:27.870 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:27.872+0000 7f0021e00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0024109430 0x7f0024119f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T20:51:27.871 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:27.872+0000 7f0021e00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0024109430 0x7f0024119f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:43704/0 (socket says 172.21.15.18:43704) 2024-04-12T20:51:27.871 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:27.872+0000 7f0021e00700 1 -- 172.21.15.18:0/1526192814 learned_addr learned my addr 172.21.15.18:0/1526192814 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T20:51:27.872 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:27.873+0000 7f0021400700 1 --2- 172.21.15.18:0/1526192814 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0024109dd0 0x7f002411a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T20:51:27.872 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:27.873+0000 7f0021e00700 1 -- 172.21.15.18:0/1526192814 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0024109dd0 msgr2=0x7f002411a480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:51:27.872 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:27.873+0000 7f0021e00700 1 --2- 172.21.15.18:0/1526192814 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0024109dd0 0x7f002411a480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:51:27.872 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:27.873+0000 7f0021e00700 1 -- 172.21.15.18:0/1526192814 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0014005040 con 0x7f0024109430 2024-04-12T20:51:27.872 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:27.873+0000 7f0021e00700 1 --2- 172.21.15.18:0/1526192814 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0024109430 0x7f0024119f40 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f0014009150 tx=0x7f0014009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T20:51:27.872 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:27.873+0000 7f001a000700 1 -- 172.21.15.18:0/1526192814 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f001400e040 con 0x7f0024109430 2024-04-12T20:51:27.872 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:27.873+0000 7f001a000700 1 -- 172.21.15.18:0/1526192814 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f0014013420 con 0x7f0024109430 2024-04-12T20:51:27.872 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:27.874+0000 7f0029000700 1 -- 172.21.15.18:0/1526192814 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0024114380 con 0x7f0024109430 2024-04-12T20:51:27.872 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:27.874+0000 7f001a000700 1 -- 172.21.15.18:0/1526192814 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f001401c630 con 0x7f0024109430 2024-04-12T20:51:27.872 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:27.874+0000 7f0029000700 1 -- 172.21.15.18:0/1526192814 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0024114940 con 0x7f0024109430 2024-04-12T20:51:27.873 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:27.875+0000 7f0029000700 1 -- 172.21.15.18:0/1526192814 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0024104b80 con 0x7f0024109430 2024-04-12T20:51:27.873 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:27.876+0000 7f001a000700 1 -- 172.21.15.18:0/1526192814 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7f0014013590 con 0x7f0024109430 2024-04-12T20:51:27.874 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:27.876+0000 7f001a000700 1 --2- 172.21.15.18:0/1526192814 >> [v2:172.21.15.18:6800/3048894687,v1:172.21.15.18:6801/3048894687] conn(0x7f00080692a0 0x7f000806b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T20:51:27.875 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:27.877+0000 7f001a000700 1 -- 172.21.15.18:0/1526192814 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4634+0+0 (secure 0 0 0) 0x7f00140880e0 con 0x7f0024109430 2024-04-12T20:51:27.875 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:27.877+0000 7f0021400700 1 --2- 172.21.15.18:0/1526192814 >> [v2:172.21.15.18:6800/3048894687,v1:172.21.15.18:6801/3048894687] conn(0x7f00080692a0 0x7f000806b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T20:51:27.875 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:27.878+0000 7f0021400700 1 --2- 172.21.15.18:0/1526192814 >> [v2:172.21.15.18:6800/3048894687,v1:172.21.15.18:6801/3048894687] conn(0x7f00080692a0 0x7f000806b750 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f00241157e0 tx=0x7f0010009ee0 comp rx=0 tx=0).ready entity=mgr.14212 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T20:51:27.879 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:27.881+0000 7f001a000700 1 -- 172.21.15.18:0/1526192814 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7f00140581c0 con 0x7f0024109430 2024-04-12T20:51:28.102 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:51:27 smithi018 ceph-mon[28799]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-12T20:51:28.146 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:28.147+0000 7f0029000700 1 -- 172.21.15.18:0/1526192814 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f002404c710 con 0x7f0024109430 2024-04-12T20:51:28.146 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:28.148+0000 7f001a000700 1 -- 172.21.15.18:0/1526192814 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+46 (secure 0 0 0) 0x7f0014057d50 con 0x7f0024109430 2024-04-12T20:51:28.147 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-12T20:51:28.147 INFO:teuthology.orchestra.run.smithi018.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-04-12T20:51:28.150 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:28.151+0000 7f0029000700 1 -- 172.21.15.18:0/1526192814 >> [v2:172.21.15.18:6800/3048894687,v1:172.21.15.18:6801/3048894687] conn(0x7f00080692a0 msgr2=0x7f000806b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:51:28.150 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:28.151+0000 7f0029000700 1 --2- 172.21.15.18:0/1526192814 >> [v2:172.21.15.18:6800/3048894687,v1:172.21.15.18:6801/3048894687] conn(0x7f00080692a0 0x7f000806b750 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f00241157e0 tx=0x7f0010009ee0 comp rx=0 tx=0).stop 2024-04-12T20:51:28.150 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:28.151+0000 7f0029000700 1 -- 172.21.15.18:0/1526192814 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0024109430 msgr2=0x7f0024119f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:51:28.150 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:28.151+0000 7f0029000700 1 --2- 172.21.15.18:0/1526192814 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0024109430 0x7f0024119f40 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f0014009150 tx=0x7f0014009e30 comp rx=0 tx=0).stop 2024-04-12T20:51:28.150 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:28.151+0000 7f0029000700 1 -- 172.21.15.18:0/1526192814 shutdown_connections 2024-04-12T20:51:28.150 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:28.151+0000 7f0029000700 1 --2- 172.21.15.18:0/1526192814 >> [v2:172.21.15.18:6800/3048894687,v1:172.21.15.18:6801/3048894687] conn(0x7f00080692a0 0x7f000806b750 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:51:28.150 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:28.151+0000 7f0029000700 1 --2- 172.21.15.18:0/1526192814 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0024109430 0x7f0024119f40 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:51:28.150 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:28.151+0000 7f0029000700 1 --2- 172.21.15.18:0/1526192814 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0024109dd0 0x7f002411a480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:51:28.150 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:28.151+0000 7f0029000700 1 -- 172.21.15.18:0/1526192814 >> 172.21.15.18:0/1526192814 conn(0x7f0024072c00 msgr2=0x7f0024074c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T20:51:28.151 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:28.152+0000 7f0029000700 1 -- 172.21.15.18:0/1526192814 shutdown_connections 2024-04-12T20:51:28.151 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:28.152+0000 7f0029000700 1 -- 172.21.15.18:0/1526192814 wait complete. 2024-04-12T20:51:28.281 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:51:27 smithi186 ceph-mon[36597]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-12T20:51:28.969 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy done 2024-04-12T20:51:28.969 INFO:tasks.cephadm:Setup complete, yielding 2024-04-12T20:51:28.969 INFO:teuthology.run_tasks:Running task print... 2024-04-12T20:51:28.979 INFO:teuthology.task.print:**** done end installing quincy cephadm ... 2024-04-12T20:51:28.979 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-04-12T20:51:29.000 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi018.front.sepia.ceph.com 2024-04-12T20:51:29.000 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1cbe407a-f90d-11ee-bc8d-c7b262605968 -- bash -c 'ceph config set mgr mgr/cephadm/use_repo_digest true --force' 2024-04-12T20:51:29.281 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:51:28 smithi186 ceph-mon[36597]: from='client.? 172.21.15.18:0/1526192814' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-04-12T20:51:29.352 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:51:28 smithi018 ceph-mon[28799]: from='client.? 172.21.15.18:0/1526192814' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-04-12T20:51:29.502 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/mon.smithi018/config 2024-04-12T20:51:30.281 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:51:29 smithi186 ceph-mon[36597]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-12T20:51:30.281 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:51:29 smithi186 ceph-mon[36597]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T20:51:30.293 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:51:29 smithi018 ceph-mon[28799]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-12T20:51:30.294 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:51:29 smithi018 ceph-mon[28799]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T20:51:31.808 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:31.809+0000 7fa2d6c00700 1 -- 172.21.15.18:0/4129919408 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa2d0066b70 msgr2=0x7fa2d00ff9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:51:31.809 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:31.809+0000 7fa2d6c00700 1 --2- 172.21.15.18:0/4129919408 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa2d0066b70 0x7fa2d00ff9d0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fa2c0004940 tx=0x7fa2c0009e30 comp rx=0 tx=0).stop 2024-04-12T20:51:31.809 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:31.809+0000 7fa2d6c00700 1 -- 172.21.15.18:0/4129919408 shutdown_connections 2024-04-12T20:51:31.809 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:31.809+0000 7fa2d6c00700 1 --2- 172.21.15.18:0/4129919408 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa2d00fff10 0x7fa2d0102330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:51:31.809 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:31.809+0000 7fa2d6c00700 1 --2- 172.21.15.18:0/4129919408 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa2d0066b70 0x7fa2d00ff9d0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:51:31.810 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:31.809+0000 7fa2d6c00700 1 -- 172.21.15.18:0/4129919408 >> 172.21.15.18:0/4129919408 conn(0x7fa2d00f7440 msgr2=0x7fa2d00f9870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T20:51:31.810 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:31.809+0000 7fa2d6c00700 1 -- 172.21.15.18:0/4129919408 shutdown_connections 2024-04-12T20:51:31.810 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:31.809+0000 7fa2d6c00700 1 -- 172.21.15.18:0/4129919408 wait complete. 2024-04-12T20:51:31.810 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:31.810+0000 7fa2d6c00700 1 Processor -- start 2024-04-12T20:51:31.810 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:31.811+0000 7fa2d6c00700 1 -- start start 2024-04-12T20:51:31.813 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:31.811+0000 7fa2d6c00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa2d0066b70 0x7fa2d010da90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T20:51:31.813 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:31.811+0000 7fa2d6c00700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa2d00fff10 0x7fa2d010dfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T20:51:31.813 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:31.811+0000 7fa2d6c00700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2d010e5d0 con 0x7fa2d0066b70 2024-04-12T20:51:31.813 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:31.811+0000 7fa2d6c00700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2d010ee20 con 0x7fa2d00fff10 2024-04-12T20:51:31.814 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:31.811+0000 7fa2cfe00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa2d0066b70 0x7fa2d010da90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T20:51:31.814 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:31.811+0000 7fa2cfe00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa2d0066b70 0x7fa2d010da90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:33004/0 (socket says 172.21.15.18:33004) 2024-04-12T20:51:31.814 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:31.811+0000 7fa2cfe00700 1 -- 172.21.15.18:0/1596767490 learned_addr learned my addr 172.21.15.18:0/1596767490 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T20:51:31.814 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:31.812+0000 7fa2cf400700 1 --2- 172.21.15.18:0/1596767490 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa2d00fff10 0x7fa2d010dfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T20:51:31.814 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:31.812+0000 7fa2cfe00700 1 -- 172.21.15.18:0/1596767490 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa2d00fff10 msgr2=0x7fa2d010dfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:51:31.814 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:31.812+0000 7fa2cfe00700 1 --2- 172.21.15.18:0/1596767490 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa2d00fff10 0x7fa2d010dfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:51:31.814 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:31.812+0000 7fa2cfe00700 1 -- 172.21.15.18:0/1596767490 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2c0005040 con 0x7fa2d0066b70 2024-04-12T20:51:31.814 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:31.812+0000 7fa2cfe00700 1 --2- 172.21.15.18:0/1596767490 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa2d0066b70 0x7fa2d010da90 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fa2c000b970 tx=0x7fa2c0009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T20:51:31.814 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:31.813+0000 7fa2c7e00700 1 -- 172.21.15.18:0/1596767490 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa2c000e040 con 0x7fa2d0066b70 2024-04-12T20:51:31.814 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:31.813+0000 7fa2c7e00700 1 -- 172.21.15.18:0/1596767490 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa2c0004290 con 0x7fa2d0066b70 2024-04-12T20:51:31.814 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:31.813+0000 7fa2d6c00700 1 -- 172.21.15.18:0/1596767490 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa2d010f0a0 con 0x7fa2d0066b70 2024-04-12T20:51:31.815 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:31.813+0000 7fa2c7e00700 1 -- 172.21.15.18:0/1596767490 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa2c001c630 con 0x7fa2d0066b70 2024-04-12T20:51:31.815 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:31.813+0000 7fa2d6c00700 1 -- 172.21.15.18:0/1596767490 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa2d010f5e0 con 0x7fa2d0066b70 2024-04-12T20:51:31.815 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:31.814+0000 7fa2d6c00700 1 -- 172.21.15.18:0/1596767490 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa2d005f5f0 con 0x7fa2d0066b70 2024-04-12T20:51:31.816 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:31.815+0000 7fa2c7e00700 1 -- 172.21.15.18:0/1596767490 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7fa2c000bbf0 con 0x7fa2d0066b70 2024-04-12T20:51:31.816 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:31.816+0000 7fa2c7e00700 1 --2- 172.21.15.18:0/1596767490 >> [v2:172.21.15.18:6800/3048894687,v1:172.21.15.18:6801/3048894687] conn(0x7fa2b40692b0 0x7fa2b406b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T20:51:31.816 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:31.817+0000 7fa2c7e00700 1 -- 172.21.15.18:0/1596767490 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4634+0+0 (secure 0 0 0) 0x7fa2c00882c0 con 0x7fa2d0066b70 2024-04-12T20:51:31.816 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:31.817+0000 7fa2cf400700 1 --2- 172.21.15.18:0/1596767490 >> [v2:172.21.15.18:6800/3048894687,v1:172.21.15.18:6801/3048894687] conn(0x7fa2b40692b0 0x7fa2b406b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T20:51:31.816 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:31.818+0000 7fa2cf400700 1 --2- 172.21.15.18:0/1596767490 >> [v2:172.21.15.18:6800/3048894687,v1:172.21.15.18:6801/3048894687] conn(0x7fa2b40692b0 0x7fa2b406b760 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fa2bc004990 tx=0x7fa2bc009ee0 comp rx=0 tx=0).ready entity=mgr.14212 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T20:51:31.820 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:31.822+0000 7fa2c7e00700 1 -- 172.21.15.18:0/1596767490 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7fa2c0058320 con 0x7fa2d0066b70 2024-04-12T20:51:32.042 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:32.043+0000 7fa2d6c00700 1 -- 172.21.15.18:0/1596767490 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/use_repo_digest}] v 0) v1 -- 0x7fa2d004bed0 con 0x7fa2d0066b70 2024-04-12T20:51:32.052 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:32.054+0000 7fa2c7e00700 1 -- 172.21.15.18:0/1596767490 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/use_repo_digest}]=0 v20)=0 v20) v1 ==== 143+0+0 (secure 0 0 0) 0x7fa2c0013e60 con 0x7fa2d0066b70 2024-04-12T20:51:32.055 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:32.057+0000 7fa2d6c00700 1 -- 172.21.15.18:0/1596767490 >> [v2:172.21.15.18:6800/3048894687,v1:172.21.15.18:6801/3048894687] conn(0x7fa2b40692b0 msgr2=0x7fa2b406b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:51:32.055 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:32.057+0000 7fa2d6c00700 1 --2- 172.21.15.18:0/1596767490 >> [v2:172.21.15.18:6800/3048894687,v1:172.21.15.18:6801/3048894687] conn(0x7fa2b40692b0 0x7fa2b406b760 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fa2bc004990 tx=0x7fa2bc009ee0 comp rx=0 tx=0).stop 2024-04-12T20:51:32.055 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:32.057+0000 7fa2d6c00700 1 -- 172.21.15.18:0/1596767490 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa2d0066b70 msgr2=0x7fa2d010da90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:51:32.055 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:32.057+0000 7fa2d6c00700 1 --2- 172.21.15.18:0/1596767490 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa2d0066b70 0x7fa2d010da90 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fa2c000b970 tx=0x7fa2c0009e30 comp rx=0 tx=0).stop 2024-04-12T20:51:32.056 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:32.057+0000 7fa2d6c00700 1 -- 172.21.15.18:0/1596767490 shutdown_connections 2024-04-12T20:51:32.056 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:32.057+0000 7fa2d6c00700 1 --2- 172.21.15.18:0/1596767490 >> [v2:172.21.15.18:6800/3048894687,v1:172.21.15.18:6801/3048894687] conn(0x7fa2b40692b0 0x7fa2b406b760 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:51:32.056 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:32.057+0000 7fa2d6c00700 1 --2- 172.21.15.18:0/1596767490 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa2d0066b70 0x7fa2d010da90 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:51:32.056 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:32.057+0000 7fa2d6c00700 1 --2- 172.21.15.18:0/1596767490 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa2d00fff10 0x7fa2d010dfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:51:32.056 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:32.057+0000 7fa2d6c00700 1 -- 172.21.15.18:0/1596767490 >> 172.21.15.18:0/1596767490 conn(0x7fa2d00f7440 msgr2=0x7fa2d00f7860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T20:51:32.056 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:32.057+0000 7fa2d6c00700 1 -- 172.21.15.18:0/1596767490 shutdown_connections 2024-04-12T20:51:32.056 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:32.058+0000 7fa2d6c00700 1 -- 172.21.15.18:0/1596767490 wait complete. 2024-04-12T20:51:32.281 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:51:31 smithi186 ceph-mon[36597]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-12T20:51:32.352 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:51:31 smithi018 ceph-mon[28799]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-12T20:51:33.023 INFO:teuthology.run_tasks:Running task print... 2024-04-12T20:51:33.032 INFO:teuthology.task.print:**** done cephadm.shell ceph config set mgr... 2024-04-12T20:51:33.032 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-04-12T20:51:33.055 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi018.front.sepia.ceph.com 2024-04-12T20:51:33.056 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1cbe407a-f90d-11ee-bc8d-c7b262605968 -- bash -c 'ceph orch status' 2024-04-12T20:51:33.352 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:51:33 smithi018 ceph-mon[28799]: from='client.? 172.21.15.18:0/1596767490' entity='client.admin' 2024-04-12T20:51:33.352 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:51:33 smithi018 ceph-mon[28799]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-12T20:51:33.352 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:51:33 smithi018 ceph-mon[28799]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-12T20:51:33.353 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:51:33 smithi018 ceph-mon[28799]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-12T20:51:33.353 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:51:33 smithi018 ceph-mon[28799]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' 2024-04-12T20:51:33.531 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:51:33 smithi186 ceph-mon[36597]: from='client.? 172.21.15.18:0/1596767490' entity='client.admin' 2024-04-12T20:51:33.532 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:51:33 smithi186 ceph-mon[36597]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-12T20:51:33.532 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:51:33 smithi186 ceph-mon[36597]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-12T20:51:33.532 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:51:33 smithi186 ceph-mon[36597]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-12T20:51:33.532 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:51:33 smithi186 ceph-mon[36597]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' 2024-04-12T20:51:33.547 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/mon.smithi018/config 2024-04-12T20:51:34.352 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:51:34 smithi018 ceph-mon[28799]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-12T20:51:34.531 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:51:34 smithi186 ceph-mon[36597]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-12T20:51:35.841 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:35.842+0000 7fc03e000700 1 -- 172.21.15.18:0/667507959 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc0380fce70 msgr2=0x7fc0380fd280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:51:35.841 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:35.842+0000 7fc03e000700 1 --2- 172.21.15.18:0/667507959 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc0380fce70 0x7fc0380fd280 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fc028004940 tx=0x7fc028009e30 comp rx=0 tx=0).stop 2024-04-12T20:51:35.842 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:35.843+0000 7fc03e000700 1 -- 172.21.15.18:0/667507959 shutdown_connections 2024-04-12T20:51:35.842 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:35.843+0000 7fc03e000700 1 --2- 172.21.15.18:0/667507959 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc0380fdfe0 0x7fc0380fe450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:51:35.842 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:35.843+0000 7fc03e000700 1 --2- 172.21.15.18:0/667507959 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc0380fce70 0x7fc0380fd280 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:51:35.842 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:35.843+0000 7fc03e000700 1 -- 172.21.15.18:0/667507959 >> 172.21.15.18:0/667507959 conn(0x7fc0380f8550 msgr2=0x7fc0380fa960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T20:51:35.842 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:35.843+0000 7fc03e000700 1 -- 172.21.15.18:0/667507959 shutdown_connections 2024-04-12T20:51:35.842 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:35.843+0000 7fc03e000700 1 -- 172.21.15.18:0/667507959 wait complete. 2024-04-12T20:51:35.842 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:35.844+0000 7fc03e000700 1 Processor -- start 2024-04-12T20:51:35.847 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:35.844+0000 7fc03e000700 1 -- start start 2024-04-12T20:51:35.847 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:35.844+0000 7fc03e000700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc0380fce70 0x7fc03810fb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T20:51:35.847 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:35.844+0000 7fc03e000700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc0380fdfe0 0x7fc038110070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T20:51:35.847 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:35.844+0000 7fc03e000700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc038110670 con 0x7fc0380fdfe0 2024-04-12T20:51:35.847 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:35.844+0000 7fc03e000700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0381107e0 con 0x7fc0380fce70 2024-04-12T20:51:35.847 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:35.844+0000 7fc036a00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc0380fdfe0 0x7fc038110070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T20:51:35.847 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:35.845+0000 7fc036a00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc0380fdfe0 0x7fc038110070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:33016/0 (socket says 172.21.15.18:33016) 2024-04-12T20:51:35.847 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:35.845+0000 7fc036a00700 1 -- 172.21.15.18:0/1395687034 learned_addr learned my addr 172.21.15.18:0/1395687034 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T20:51:35.847 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:35.845+0000 7fc037400700 1 --2- 172.21.15.18:0/1395687034 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc0380fce70 0x7fc03810fb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T20:51:35.847 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:35.845+0000 7fc036a00700 1 -- 172.21.15.18:0/1395687034 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc0380fce70 msgr2=0x7fc03810fb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:51:35.847 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:35.845+0000 7fc036a00700 1 --2- 172.21.15.18:0/1395687034 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc0380fce70 0x7fc03810fb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:51:35.847 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:35.845+0000 7fc036a00700 1 -- 172.21.15.18:0/1395687034 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc028005040 con 0x7fc0380fdfe0 2024-04-12T20:51:35.847 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:35.845+0000 7fc036a00700 1 --2- 172.21.15.18:0/1395687034 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc0380fdfe0 0x7fc038110070 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fc024002a50 tx=0x7fc0240046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T20:51:35.848 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:35.846+0000 7fc02f400700 1 -- 172.21.15.18:0/1395687034 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc024007cc0 con 0x7fc0380fdfe0 2024-04-12T20:51:35.848 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:35.846+0000 7fc02f400700 1 -- 172.21.15.18:0/1395687034 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc02400f040 con 0x7fc0380fdfe0 2024-04-12T20:51:35.848 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:35.846+0000 7fc02f400700 1 -- 172.21.15.18:0/1395687034 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc024009640 con 0x7fc0380fdfe0 2024-04-12T20:51:35.848 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:35.846+0000 7fc03e000700 1 -- 172.21.15.18:0/1395687034 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc038115240 con 0x7fc0380fdfe0 2024-04-12T20:51:35.848 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:35.846+0000 7fc03e000700 1 -- 172.21.15.18:0/1395687034 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc038115800 con 0x7fc0380fdfe0 2024-04-12T20:51:35.848 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:35.847+0000 7fc03e000700 1 -- 172.21.15.18:0/1395687034 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc038101970 con 0x7fc0380fdfe0 2024-04-12T20:51:35.848 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:35.848+0000 7fc02f400700 1 -- 172.21.15.18:0/1395687034 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7fc0240097a0 con 0x7fc0380fdfe0 2024-04-12T20:51:35.848 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:35.849+0000 7fc02f400700 1 --2- 172.21.15.18:0/1395687034 >> [v2:172.21.15.18:6800/3048894687,v1:172.21.15.18:6801/3048894687] conn(0x7fc01c0692a0 0x7fc01c06b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T20:51:35.849 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:35.849+0000 7fc02f400700 1 -- 172.21.15.18:0/1395687034 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4634+0+0 (secure 0 0 0) 0x7fc024086b70 con 0x7fc0380fdfe0 2024-04-12T20:51:35.849 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:35.849+0000 7fc037400700 1 --2- 172.21.15.18:0/1395687034 >> [v2:172.21.15.18:6800/3048894687,v1:172.21.15.18:6801/3048894687] conn(0x7fc01c0692a0 0x7fc01c06b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T20:51:35.849 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:35.850+0000 7fc037400700 1 --2- 172.21.15.18:0/1395687034 >> [v2:172.21.15.18:6800/3048894687,v1:172.21.15.18:6801/3048894687] conn(0x7fc01c0692a0 0x7fc01c06b750 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fc02800b3b0 tx=0x7fc028009e30 comp rx=0 tx=0).ready entity=mgr.14212 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T20:51:35.852 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:35.854+0000 7fc02f400700 1 -- 172.21.15.18:0/1395687034 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7fc024056c70 con 0x7fc0380fdfe0 2024-04-12T20:51:36.087 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:36.089+0000 7fc03e000700 1 -- 172.21.15.18:0/1395687034 --> [v2:172.21.15.18:6800/3048894687,v1:172.21.15.18:6801/3048894687] -- mgr_command(tid 0: {"prefix": "orch status", "target": ["mon-mgr", ""]}) v1 -- 0x7fc03802b340 con 0x7fc01c0692a0 2024-04-12T20:51:36.091 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:36.093+0000 7fc02f400700 1 -- 172.21.15.18:0/1395687034 <== mgr.14212 v2:172.21.15.18:6800/3048894687 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+43 (secure 0 0 0) 0x7fc03802b340 con 0x7fc01c0692a0 2024-04-12T20:51:36.091 INFO:teuthology.orchestra.run.smithi018.stdout:Backend: cephadm 2024-04-12T20:51:36.091 INFO:teuthology.orchestra.run.smithi018.stdout:Available: Yes 2024-04-12T20:51:36.092 INFO:teuthology.orchestra.run.smithi018.stdout:Paused: No 2024-04-12T20:51:36.094 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:36.096+0000 7fc03e000700 1 -- 172.21.15.18:0/1395687034 >> [v2:172.21.15.18:6800/3048894687,v1:172.21.15.18:6801/3048894687] conn(0x7fc01c0692a0 msgr2=0x7fc01c06b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:51:36.094 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:36.096+0000 7fc03e000700 1 --2- 172.21.15.18:0/1395687034 >> [v2:172.21.15.18:6800/3048894687,v1:172.21.15.18:6801/3048894687] conn(0x7fc01c0692a0 0x7fc01c06b750 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fc02800b3b0 tx=0x7fc028009e30 comp rx=0 tx=0).stop 2024-04-12T20:51:36.094 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:36.096+0000 7fc03e000700 1 -- 172.21.15.18:0/1395687034 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc0380fdfe0 msgr2=0x7fc038110070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:51:36.095 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:36.096+0000 7fc03e000700 1 --2- 172.21.15.18:0/1395687034 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc0380fdfe0 0x7fc038110070 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fc024002a50 tx=0x7fc0240046b0 comp rx=0 tx=0).stop 2024-04-12T20:51:36.095 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:36.096+0000 7fc03e000700 1 -- 172.21.15.18:0/1395687034 shutdown_connections 2024-04-12T20:51:36.095 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:36.096+0000 7fc03e000700 1 --2- 172.21.15.18:0/1395687034 >> [v2:172.21.15.18:6800/3048894687,v1:172.21.15.18:6801/3048894687] conn(0x7fc01c0692a0 0x7fc01c06b750 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:51:36.095 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:36.096+0000 7fc03e000700 1 --2- 172.21.15.18:0/1395687034 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc0380fce70 0x7fc03810fb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:51:36.095 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:36.096+0000 7fc03e000700 1 --2- 172.21.15.18:0/1395687034 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc0380fdfe0 0x7fc038110070 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:51:36.095 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:36.096+0000 7fc03e000700 1 -- 172.21.15.18:0/1395687034 >> 172.21.15.18:0/1395687034 conn(0x7fc0380f8550 msgr2=0x7fc0380f9bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T20:51:36.095 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:36.097+0000 7fc03e000700 1 -- 172.21.15.18:0/1395687034 shutdown_connections 2024-04-12T20:51:36.095 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:36.097+0000 7fc03e000700 1 -- 172.21.15.18:0/1395687034 wait complete. 2024-04-12T20:51:36.352 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:51:36 smithi018 ceph-mon[28799]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-12T20:51:36.531 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:51:36 smithi186 ceph-mon[36597]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-12T20:51:37.084 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1cbe407a-f90d-11ee-bc8d-c7b262605968 -- bash -c 'ceph orch ps' 2024-04-12T20:51:37.352 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:51:37 smithi018 ceph-mon[28799]: from='client.14428 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T20:51:37.531 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:51:37 smithi186 ceph-mon[36597]: from='client.14428 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T20:51:37.555 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/mon.smithi018/config 2024-04-12T20:51:38.353 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:51:38 smithi018 ceph-mon[28799]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-12T20:51:38.531 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:51:38 smithi186 ceph-mon[36597]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-12T20:51:39.868 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:39.868+0000 7f6d66600700 1 -- 172.21.15.18:0/1121556016 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6d60104500 msgr2=0x7f6d60104970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:51:39.868 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:39.868+0000 7f6d66600700 1 --2- 172.21.15.18:0/1121556016 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6d60104500 0x7f6d60104970 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f6d4c004940 tx=0x7f6d4c009e30 comp rx=0 tx=0).stop 2024-04-12T20:51:39.868 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:39.868+0000 7f6d66600700 1 -- 172.21.15.18:0/1121556016 shutdown_connections 2024-04-12T20:51:39.868 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:39.868+0000 7f6d66600700 1 --2- 172.21.15.18:0/1121556016 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6d60104500 0x7f6d60104970 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:51:39.868 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:39.868+0000 7f6d66600700 1 --2- 172.21.15.18:0/1121556016 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6d60103250 0x7f6d60103660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:51:39.868 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:39.868+0000 7f6d66600700 1 -- 172.21.15.18:0/1121556016 >> 172.21.15.18:0/1121556016 conn(0x7f6d60072530 msgr2=0x7f6d60074960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T20:51:39.868 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:39.868+0000 7f6d66600700 1 -- 172.21.15.18:0/1121556016 shutdown_connections 2024-04-12T20:51:39.869 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:39.869+0000 7f6d66600700 1 -- 172.21.15.18:0/1121556016 wait complete. 2024-04-12T20:51:39.870 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:39.870+0000 7f6d66600700 1 Processor -- start 2024-04-12T20:51:39.870 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:39.870+0000 7f6d66600700 1 -- start start 2024-04-12T20:51:39.870 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:39.870+0000 7f6d66600700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6d60103250 0x7f6d60113e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T20:51:39.870 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:39.870+0000 7f6d66600700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6d60104500 0x7f6d601143d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T20:51:39.870 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:39.870+0000 7f6d66600700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d60115140 con 0x7f6d60103250 2024-04-12T20:51:39.870 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:39.870+0000 7f6d66600700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d601152b0 con 0x7f6d60104500 2024-04-12T20:51:39.870 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:39.871+0000 7f6d65200700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6d60103250 0x7f6d60113e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T20:51:39.870 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:39.871+0000 7f6d65200700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6d60103250 0x7f6d60113e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:33032/0 (socket says 172.21.15.18:33032) 2024-04-12T20:51:39.870 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:39.871+0000 7f6d65200700 1 -- 172.21.15.18:0/3355747883 learned_addr learned my addr 172.21.15.18:0/3355747883 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T20:51:39.871 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:39.871+0000 7f6d5fe00700 1 --2- 172.21.15.18:0/3355747883 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6d60104500 0x7f6d601143d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T20:51:39.871 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:39.871+0000 7f6d65200700 1 -- 172.21.15.18:0/3355747883 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6d60104500 msgr2=0x7f6d601143d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:51:39.871 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:39.871+0000 7f6d65200700 1 --2- 172.21.15.18:0/3355747883 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6d60104500 0x7f6d601143d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:51:39.871 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:39.871+0000 7f6d65200700 1 -- 172.21.15.18:0/3355747883 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d4c005040 con 0x7f6d60103250 2024-04-12T20:51:39.871 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:39.871+0000 7f6d65200700 1 --2- 172.21.15.18:0/3355747883 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6d60103250 0x7f6d60113e90 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f6d54002a50 tx=0x7f6d540046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T20:51:39.872 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:39.872+0000 7f6d5cc00700 1 -- 172.21.15.18:0/3355747883 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6d54007cc0 con 0x7f6d60103250 2024-04-12T20:51:39.872 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:39.872+0000 7f6d5cc00700 1 -- 172.21.15.18:0/3355747883 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f6d5400f040 con 0x7f6d60103250 2024-04-12T20:51:39.872 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:39.872+0000 7f6d5cc00700 1 -- 172.21.15.18:0/3355747883 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6d54009640 con 0x7f6d60103250 2024-04-12T20:51:39.872 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:39.872+0000 7f6d66600700 1 -- 172.21.15.18:0/3355747883 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6d60115530 con 0x7f6d60103250 2024-04-12T20:51:39.872 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:39.872+0000 7f6d66600700 1 -- 172.21.15.18:0/3355747883 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6d60115af0 con 0x7f6d60103250 2024-04-12T20:51:39.872 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:39.873+0000 7f6d66600700 1 -- 172.21.15.18:0/3355747883 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6d60076080 con 0x7f6d60103250 2024-04-12T20:51:39.873 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:39.874+0000 7f6d5cc00700 1 -- 172.21.15.18:0/3355747883 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7f6d540097a0 con 0x7f6d60103250 2024-04-12T20:51:39.874 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:39.875+0000 7f6d5cc00700 1 --2- 172.21.15.18:0/3355747883 >> [v2:172.21.15.18:6800/3048894687,v1:172.21.15.18:6801/3048894687] conn(0x7f6d480692a0 0x7f6d4806b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T20:51:39.874 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:39.875+0000 7f6d5cc00700 1 -- 172.21.15.18:0/3355747883 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4634+0+0 (secure 0 0 0) 0x7f6d54086b70 con 0x7f6d60103250 2024-04-12T20:51:39.874 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:39.875+0000 7f6d5fe00700 1 --2- 172.21.15.18:0/3355747883 >> [v2:172.21.15.18:6800/3048894687,v1:172.21.15.18:6801/3048894687] conn(0x7f6d480692a0 0x7f6d4806b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T20:51:39.875 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:39.876+0000 7f6d5fe00700 1 --2- 172.21.15.18:0/3355747883 >> [v2:172.21.15.18:6800/3048894687,v1:172.21.15.18:6801/3048894687] conn(0x7f6d480692a0 0x7f6d4806b750 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f6d4c000c00 tx=0x7f6d4c009e30 comp rx=0 tx=0).ready entity=mgr.14212 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T20:51:39.878 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:39.880+0000 7f6d5cc00700 1 -- 172.21.15.18:0/3355747883 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7f6d540531e0 con 0x7f6d60103250 2024-04-12T20:51:40.105 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:40.106+0000 7f6d66600700 1 -- 172.21.15.18:0/3355747883 --> [v2:172.21.15.18:6800/3048894687,v1:172.21.15.18:6801/3048894687] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f6d60115d60 con 0x7f6d480692a0 2024-04-12T20:51:40.121 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:40.123+0000 7f6d5cc00700 1 -- 172.21.15.18:0/3355747883 <== mgr.14212 v2:172.21.15.18:6800/3048894687 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+2682 (secure 0 0 0) 0x7f6d60115d60 con 0x7f6d480692a0 2024-04-12T20:51:40.122 INFO:teuthology.orchestra.run.smithi018.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-12T20:51:40.122 INFO:teuthology.orchestra.run.smithi018.stdout:alertmanager.smithi018 smithi018 *:9093,9094 running (4m) 2m ago 6m 13.7M - 0.25.0 c8568f914cd2 f43ec346f9e5 2024-04-12T20:51:40.122 INFO:teuthology.orchestra.run.smithi018.stdout:crash.smithi018 smithi018 running (7m) 2m ago 7m 6660k - 17.2.7-827-gd3b5060a 21de9b7369ad 1ac7603bddfd 2024-04-12T20:51:40.122 INFO:teuthology.orchestra.run.smithi018.stdout:crash.smithi186 smithi186 running (5m) 83s ago 5m 6669k - 17.2.7-827-gd3b5060a 21de9b7369ad 14bf8a1672eb 2024-04-12T20:51:40.122 INFO:teuthology.orchestra.run.smithi018.stdout:grafana.smithi018 smithi018 *:3000 running (4m) 2m ago 6m 73.9M - 9.4.7 954c08fa6188 a7c3fe73c4d7 2024-04-12T20:51:40.123 INFO:teuthology.orchestra.run.smithi018.stdout:mgr.smithi018.xjzwga smithi018 *:9283 running (8m) 2m ago 8m 473M - 17.2.7-827-gd3b5060a 21de9b7369ad c72f43857311 2024-04-12T20:51:40.123 INFO:teuthology.orchestra.run.smithi018.stdout:mgr.smithi186.kqcfxk smithi186 *:8443,9283 running (5m) 83s ago 5m 431M - 17.2.7-827-gd3b5060a 21de9b7369ad 2c790daf3cc7 2024-04-12T20:51:40.123 INFO:teuthology.orchestra.run.smithi018.stdout:mon.smithi018 smithi018 running (8m) 2m ago 8m 40.8M 2048M 17.2.7-827-gd3b5060a 21de9b7369ad cd84caafa056 2024-04-12T20:51:40.123 INFO:teuthology.orchestra.run.smithi018.stdout:mon.smithi186 smithi186 running (5m) 83s ago 5m 37.6M 2048M 17.2.7-827-gd3b5060a 21de9b7369ad d7263c772b73 2024-04-12T20:51:40.123 INFO:teuthology.orchestra.run.smithi018.stdout:node-exporter.smithi018 smithi018 *:9100 running (6m) 2m ago 6m 7884k - 1.5.0 0da6a335fe13 a23e90d473fe 2024-04-12T20:51:40.123 INFO:teuthology.orchestra.run.smithi018.stdout:node-exporter.smithi186 smithi186 *:9100 running (5m) 83s ago 5m 7894k - 1.5.0 0da6a335fe13 57160d1060f2 2024-04-12T20:51:40.123 INFO:teuthology.orchestra.run.smithi018.stdout:osd.0 smithi018 running (4m) 2m ago 4m 38.0M 4290M 17.2.7-827-gd3b5060a 21de9b7369ad 0d83e027bf0a 2024-04-12T20:51:40.123 INFO:teuthology.orchestra.run.smithi018.stdout:osd.1 smithi018 running (3m) 2m ago 3m 38.3M 4290M 17.2.7-827-gd3b5060a 21de9b7369ad 6517c9cc7f87 2024-04-12T20:51:40.123 INFO:teuthology.orchestra.run.smithi018.stdout:osd.2 smithi018 running (3m) 2m ago 3m 37.9M 4290M 17.2.7-827-gd3b5060a 21de9b7369ad 8a83692acaa5 2024-04-12T20:51:40.123 INFO:teuthology.orchestra.run.smithi018.stdout:osd.3 smithi186 running (2m) 83s ago 2m 38.7M 5312M 17.2.7-827-gd3b5060a 21de9b7369ad c2def7a90d76 2024-04-12T20:51:40.123 INFO:teuthology.orchestra.run.smithi018.stdout:osd.4 smithi186 running (2m) 83s ago 2m 35.7M 5312M 17.2.7-827-gd3b5060a 21de9b7369ad 757e7c71b7b9 2024-04-12T20:51:40.123 INFO:teuthology.orchestra.run.smithi018.stdout:osd.5 smithi186 running (89s) 83s ago 89s 10.5M 5312M 17.2.7-827-gd3b5060a 21de9b7369ad 2ee072a436d0 2024-04-12T20:51:40.123 INFO:teuthology.orchestra.run.smithi018.stdout:prometheus.smithi018 smithi018 *:9095 running (4m) 2m ago 6m 36.5M - 2.43.0 a07b618ecd1d aecd046db448 2024-04-12T20:51:40.125 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:40.125+0000 7f6d66600700 1 -- 172.21.15.18:0/3355747883 >> [v2:172.21.15.18:6800/3048894687,v1:172.21.15.18:6801/3048894687] conn(0x7f6d480692a0 msgr2=0x7f6d4806b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:51:40.125 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:40.125+0000 7f6d66600700 1 --2- 172.21.15.18:0/3355747883 >> [v2:172.21.15.18:6800/3048894687,v1:172.21.15.18:6801/3048894687] conn(0x7f6d480692a0 0x7f6d4806b750 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f6d4c000c00 tx=0x7f6d4c009e30 comp rx=0 tx=0).stop 2024-04-12T20:51:40.125 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:40.126+0000 7f6d66600700 1 -- 172.21.15.18:0/3355747883 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6d60103250 msgr2=0x7f6d60113e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:51:40.125 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:40.126+0000 7f6d66600700 1 --2- 172.21.15.18:0/3355747883 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6d60103250 0x7f6d60113e90 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f6d54002a50 tx=0x7f6d540046b0 comp rx=0 tx=0).stop 2024-04-12T20:51:40.125 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:40.126+0000 7f6d66600700 1 -- 172.21.15.18:0/3355747883 shutdown_connections 2024-04-12T20:51:40.125 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:40.126+0000 7f6d66600700 1 --2- 172.21.15.18:0/3355747883 >> [v2:172.21.15.18:6800/3048894687,v1:172.21.15.18:6801/3048894687] conn(0x7f6d480692a0 0x7f6d4806b750 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:51:40.125 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:40.126+0000 7f6d66600700 1 --2- 172.21.15.18:0/3355747883 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6d60103250 0x7f6d60113e90 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:51:40.125 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:40.126+0000 7f6d66600700 1 --2- 172.21.15.18:0/3355747883 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6d60104500 0x7f6d601143d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:51:40.126 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:40.126+0000 7f6d66600700 1 -- 172.21.15.18:0/3355747883 >> 172.21.15.18:0/3355747883 conn(0x7f6d60072530 msgr2=0x7f6d60074b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T20:51:40.126 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:40.126+0000 7f6d66600700 1 -- 172.21.15.18:0/3355747883 shutdown_connections 2024-04-12T20:51:40.126 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:40.126+0000 7f6d66600700 1 -- 172.21.15.18:0/3355747883 wait complete. 2024-04-12T20:51:40.412 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:51:40 smithi018 ceph-mon[28799]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-12T20:51:40.531 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:51:40 smithi186 ceph-mon[36597]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-12T20:51:40.955 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1cbe407a-f90d-11ee-bc8d-c7b262605968 -- bash -c 'ceph orch ls' 2024-04-12T20:51:41.352 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:51:41 smithi018 ceph-mon[28799]: from='client.14432 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T20:51:41.467 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/mon.smithi018/config 2024-04-12T20:51:41.531 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:51:41 smithi186 ceph-mon[36597]: from='client.14432 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T20:51:42.352 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:51:42 smithi018 ceph-mon[28799]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-12T20:51:42.532 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:51:42 smithi186 ceph-mon[36597]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-12T20:51:43.733 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:43.733+0000 7f588ca00700 1 -- 172.21.15.18:0/3255411661 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f58880ffd80 msgr2=0x7f58881021a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:51:43.733 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:43.733+0000 7f588ca00700 1 --2- 172.21.15.18:0/3255411661 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f58880ffd80 0x7f58881021a0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f5874004940 tx=0x7f5874009e30 comp rx=0 tx=0).stop 2024-04-12T20:51:43.733 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:43.733+0000 7f588ca00700 1 -- 172.21.15.18:0/3255411661 shutdown_connections 2024-04-12T20:51:43.733 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:43.733+0000 7f588ca00700 1 --2- 172.21.15.18:0/3255411661 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f58880ffd80 0x7f58881021a0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:51:43.734 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:43.733+0000 7f588ca00700 1 --2- 172.21.15.18:0/3255411661 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5888066d10 0x7f58880ff840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:51:43.734 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:43.733+0000 7f588ca00700 1 -- 172.21.15.18:0/3255411661 >> 172.21.15.18:0/3255411661 conn(0x7f58880f72d0 msgr2=0x7f58880f96e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T20:51:43.734 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:43.733+0000 7f588ca00700 1 -- 172.21.15.18:0/3255411661 shutdown_connections 2024-04-12T20:51:43.734 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:43.733+0000 7f588ca00700 1 -- 172.21.15.18:0/3255411661 wait complete. 2024-04-12T20:51:43.734 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:43.734+0000 7f588ca00700 1 Processor -- start 2024-04-12T20:51:43.734 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:43.734+0000 7f588ca00700 1 -- start start 2024-04-12T20:51:43.735 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:43.735+0000 7f588ca00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5888066d10 0x7f58880fd220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T20:51:43.735 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:43.735+0000 7f588ca00700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f58880ffd80 0x7f58880fb740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T20:51:43.735 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:43.735+0000 7f588ca00700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58880fbd60 con 0x7f5888066d10 2024-04-12T20:51:43.735 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:43.735+0000 7f588ca00700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58880fbea0 con 0x7f58880ffd80 2024-04-12T20:51:43.735 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:43.735+0000 7f5886800700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5888066d10 0x7f58880fd220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T20:51:43.735 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:43.735+0000 7f5886800700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5888066d10 0x7f58880fd220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:39770/0 (socket says 172.21.15.18:39770) 2024-04-12T20:51:43.736 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:43.735+0000 7f5886800700 1 -- 172.21.15.18:0/4219702292 learned_addr learned my addr 172.21.15.18:0/4219702292 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T20:51:43.736 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:43.736+0000 7f5885e00700 1 --2- 172.21.15.18:0/4219702292 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f58880ffd80 0x7f58880fb740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T20:51:43.736 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:43.736+0000 7f5886800700 1 -- 172.21.15.18:0/4219702292 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f58880ffd80 msgr2=0x7f58880fb740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:51:43.736 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:43.736+0000 7f5886800700 1 --2- 172.21.15.18:0/4219702292 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f58880ffd80 0x7f58880fb740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:51:43.736 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:43.736+0000 7f5886800700 1 -- 172.21.15.18:0/4219702292 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5874005040 con 0x7f5888066d10 2024-04-12T20:51:43.736 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:43.736+0000 7f5886800700 1 --2- 172.21.15.18:0/4219702292 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5888066d10 0x7f58880fd220 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f587800b790 tx=0x7f5878009220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T20:51:43.736 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:43.736+0000 7f587ea00700 1 -- 172.21.15.18:0/4219702292 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f587800f6f0 con 0x7f5888066d10 2024-04-12T20:51:43.736 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:43.736+0000 7f588ca00700 1 -- 172.21.15.18:0/4219702292 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f58880fc180 con 0x7f5888066d10 2024-04-12T20:51:43.736 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:43.736+0000 7f587ea00700 1 -- 172.21.15.18:0/4219702292 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f587800fd20 con 0x7f5888066d10 2024-04-12T20:51:43.736 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:43.736+0000 7f588ca00700 1 -- 172.21.15.18:0/4219702292 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5888115910 con 0x7f5888066d10 2024-04-12T20:51:43.736 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:43.736+0000 7f587ea00700 1 -- 172.21.15.18:0/4219702292 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f587801a400 con 0x7f5888066d10 2024-04-12T20:51:43.737 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:43.738+0000 7f588ca00700 1 -- 172.21.15.18:0/4219702292 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f58880faf50 con 0x7f5888066d10 2024-04-12T20:51:43.737 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:43.739+0000 7f587ea00700 1 -- 172.21.15.18:0/4219702292 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7f5878011400 con 0x7f5888066d10 2024-04-12T20:51:43.739 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:43.740+0000 7f587ea00700 1 --2- 172.21.15.18:0/4219702292 >> [v2:172.21.15.18:6800/3048894687,v1:172.21.15.18:6801/3048894687] conn(0x7f586c0692a0 0x7f586c06b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T20:51:43.739 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:43.740+0000 7f587ea00700 1 -- 172.21.15.18:0/4219702292 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4634+0+0 (secure 0 0 0) 0x7f5878086cd0 con 0x7f5888066d10 2024-04-12T20:51:43.739 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:43.740+0000 7f5885e00700 1 --2- 172.21.15.18:0/4219702292 >> [v2:172.21.15.18:6800/3048894687,v1:172.21.15.18:6801/3048894687] conn(0x7f586c0692a0 0x7f586c06b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T20:51:43.740 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:43.741+0000 7f5885e00700 1 --2- 172.21.15.18:0/4219702292 >> [v2:172.21.15.18:6800/3048894687,v1:172.21.15.18:6801/3048894687] conn(0x7f586c0692a0 0x7f586c06b750 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f587400b3b0 tx=0x7f5874009e30 comp rx=0 tx=0).ready entity=mgr.14212 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T20:51:43.744 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:43.745+0000 7f587ea00700 1 -- 172.21.15.18:0/4219702292 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7f58780532c0 con 0x7f5888066d10 2024-04-12T20:51:43.977 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:43.977+0000 7f588ca00700 1 -- 172.21.15.18:0/4219702292 --> [v2:172.21.15.18:6800/3048894687,v1:172.21.15.18:6801/3048894687] -- mgr_command(tid 0: {"prefix": "orch ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f58880f8dc0 con 0x7f586c0692a0 2024-04-12T20:51:43.988 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:43.989+0000 7f587ea00700 1 -- 172.21.15.18:0/4219702292 <== mgr.14212 v2:172.21.15.18:6800/3048894687 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+1170 (secure 0 0 0) 0x7f58880f8dc0 con 0x7f586c0692a0 2024-04-12T20:51:43.989 INFO:teuthology.orchestra.run.smithi018.stdout:NAME PORTS RUNNING REFRESHED AGE PLACEMENT 2024-04-12T20:51:43.989 INFO:teuthology.orchestra.run.smithi018.stdout:alertmanager ?:9093,9094 1/1 2m ago 7m count:1 2024-04-12T20:51:43.989 INFO:teuthology.orchestra.run.smithi018.stdout:crash 2/2 2m ago 7m * 2024-04-12T20:51:43.989 INFO:teuthology.orchestra.run.smithi018.stdout:grafana ?:3000 1/1 2m ago 7m count:1 2024-04-12T20:51:43.989 INFO:teuthology.orchestra.run.smithi018.stdout:mgr 2/2 2m ago 7m count:2 2024-04-12T20:51:43.989 INFO:teuthology.orchestra.run.smithi018.stdout:mon 2/2 2m ago 6m smithi018:172.21.15.18=smithi018;smithi186:172.21.15.186=smithi186;count:2 2024-04-12T20:51:43.989 INFO:teuthology.orchestra.run.smithi018.stdout:node-exporter ?:9100 2/2 2m ago 7m * 2024-04-12T20:51:43.989 INFO:teuthology.orchestra.run.smithi018.stdout:osd 6 2m ago - 2024-04-12T20:51:43.989 INFO:teuthology.orchestra.run.smithi018.stdout:prometheus ?:9095 1/1 2m ago 7m count:1 2024-04-12T20:51:43.991 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:43.992+0000 7f588ca00700 1 -- 172.21.15.18:0/4219702292 >> [v2:172.21.15.18:6800/3048894687,v1:172.21.15.18:6801/3048894687] conn(0x7f586c0692a0 msgr2=0x7f586c06b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:51:43.991 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:43.992+0000 7f588ca00700 1 --2- 172.21.15.18:0/4219702292 >> [v2:172.21.15.18:6800/3048894687,v1:172.21.15.18:6801/3048894687] conn(0x7f586c0692a0 0x7f586c06b750 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f587400b3b0 tx=0x7f5874009e30 comp rx=0 tx=0).stop 2024-04-12T20:51:43.991 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:43.992+0000 7f588ca00700 1 -- 172.21.15.18:0/4219702292 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5888066d10 msgr2=0x7f58880fd220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:51:43.991 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:43.992+0000 7f588ca00700 1 --2- 172.21.15.18:0/4219702292 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5888066d10 0x7f58880fd220 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f587800b790 tx=0x7f5878009220 comp rx=0 tx=0).stop 2024-04-12T20:51:43.992 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:43.993+0000 7f588ca00700 1 -- 172.21.15.18:0/4219702292 shutdown_connections 2024-04-12T20:51:43.992 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:43.993+0000 7f588ca00700 1 --2- 172.21.15.18:0/4219702292 >> [v2:172.21.15.18:6800/3048894687,v1:172.21.15.18:6801/3048894687] conn(0x7f586c0692a0 0x7f586c06b750 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:51:43.992 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:43.993+0000 7f588ca00700 1 --2- 172.21.15.18:0/4219702292 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5888066d10 0x7f58880fd220 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:51:43.992 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:43.993+0000 7f588ca00700 1 --2- 172.21.15.18:0/4219702292 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f58880ffd80 0x7f58880fb740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:51:43.992 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:43.993+0000 7f588ca00700 1 -- 172.21.15.18:0/4219702292 >> 172.21.15.18:0/4219702292 conn(0x7f58880f72d0 msgr2=0x7f58880f76a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T20:51:43.992 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:43.993+0000 7f588ca00700 1 -- 172.21.15.18:0/4219702292 shutdown_connections 2024-04-12T20:51:43.992 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:43.993+0000 7f588ca00700 1 -- 172.21.15.18:0/4219702292 wait complete. 2024-04-12T20:51:44.352 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:51:44 smithi018 ceph-mon[28799]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-12T20:51:44.531 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:51:44 smithi186 ceph-mon[36597]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-12T20:51:44.867 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1cbe407a-f90d-11ee-bc8d-c7b262605968 -- bash -c 'ceph orch host ls' 2024-04-12T20:51:45.380 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/mon.smithi018/config 2024-04-12T20:51:45.454 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:51:45 smithi186 ceph-mon[36597]: from='client.14436 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T20:51:45.454 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:51:45 smithi186 ceph-mon[36597]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T20:51:45.524 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:51:45 smithi018 ceph-mon[28799]: from='client.14436 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T20:51:45.525 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:51:45 smithi018 ceph-mon[28799]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T20:51:46.352 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:51:46 smithi018 ceph-mon[28799]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-12T20:51:46.531 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:51:46 smithi186 ceph-mon[36597]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-12T20:51:47.360 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:51:47 smithi018 ceph-mon[28799]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-12T20:51:47.531 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:51:47 smithi186 ceph-mon[36597]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-12T20:51:47.653 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:47.653+0000 7f2fb1c00700 1 -- 172.21.15.18:0/323668443 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2fac0708f0 msgr2=0x7f2fac070d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:51:47.653 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:47.653+0000 7f2fb1c00700 1 --2- 172.21.15.18:0/323668443 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2fac0708f0 0x7f2fac070d60 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f2f98004940 tx=0x7f2f98009e30 comp rx=0 tx=0).stop 2024-04-12T20:51:47.653 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:47.653+0000 7f2fb1c00700 1 -- 172.21.15.18:0/323668443 shutdown_connections 2024-04-12T20:51:47.653 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:47.653+0000 7f2fb1c00700 1 --2- 172.21.15.18:0/323668443 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2fac0708f0 0x7f2fac070d60 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:51:47.654 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:47.653+0000 7f2fb1c00700 1 --2- 172.21.15.18:0/323668443 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2fac071ea0 0x7f2fac0703b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:51:47.654 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:47.653+0000 7f2fb1c00700 1 -- 172.21.15.18:0/323668443 >> 172.21.15.18:0/323668443 conn(0x7f2fac0f8510 msgr2=0x7f2fac0fa960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T20:51:47.654 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:47.653+0000 7f2fb1c00700 1 -- 172.21.15.18:0/323668443 shutdown_connections 2024-04-12T20:51:47.654 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:47.654+0000 7f2fb1c00700 1 -- 172.21.15.18:0/323668443 wait complete. 2024-04-12T20:51:47.655 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:47.654+0000 7f2fb1c00700 1 Processor -- start 2024-04-12T20:51:47.656 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:47.655+0000 7f2fb1c00700 1 -- start start 2024-04-12T20:51:47.656 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:47.655+0000 7f2fb1c00700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2fac0708f0 0x7f2fac114000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T20:51:47.656 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:47.655+0000 7f2fb1c00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2fac071ea0 0x7f2fac114540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T20:51:47.656 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:47.655+0000 7f2fb1c00700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fac114b40 con 0x7f2fac071ea0 2024-04-12T20:51:47.656 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:47.655+0000 7f2fb1c00700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fac115390 con 0x7f2fac0708f0 2024-04-12T20:51:47.656 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:47.655+0000 7f2faa000700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2fac071ea0 0x7f2fac114540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T20:51:47.656 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:47.655+0000 7f2faa000700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2fac071ea0 0x7f2fac114540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:39790/0 (socket says 172.21.15.18:39790) 2024-04-12T20:51:47.656 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:47.655+0000 7f2faa000700 1 -- 172.21.15.18:0/766505314 learned_addr learned my addr 172.21.15.18:0/766505314 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T20:51:47.656 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:47.656+0000 7f2faaa00700 1 --2- 172.21.15.18:0/766505314 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2fac0708f0 0x7f2fac114000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T20:51:47.656 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:47.656+0000 7f2faa000700 1 -- 172.21.15.18:0/766505314 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2fac0708f0 msgr2=0x7f2fac114000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:51:47.656 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:47.656+0000 7f2faa000700 1 --2- 172.21.15.18:0/766505314 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2fac0708f0 0x7f2fac114000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:51:47.656 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:47.656+0000 7f2faa000700 1 -- 172.21.15.18:0/766505314 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f98005040 con 0x7f2fac071ea0 2024-04-12T20:51:47.656 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:47.656+0000 7f2faa000700 1 --2- 172.21.15.18:0/766505314 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2fac071ea0 0x7f2fac114540 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f2f9800b3b0 tx=0x7f2f98009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T20:51:47.656 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:47.657+0000 7f2fa2a00700 1 -- 172.21.15.18:0/766505314 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2f9800e040 con 0x7f2fac071ea0 2024-04-12T20:51:47.656 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:47.657+0000 7f2fa2a00700 1 -- 172.21.15.18:0/766505314 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f2f98004290 con 0x7f2fac071ea0 2024-04-12T20:51:47.657 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:47.657+0000 7f2fa2a00700 1 -- 172.21.15.18:0/766505314 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2f9801c630 con 0x7f2fac071ea0 2024-04-12T20:51:47.657 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:47.657+0000 7f2fb1c00700 1 -- 172.21.15.18:0/766505314 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2fac115610 con 0x7f2fac071ea0 2024-04-12T20:51:47.657 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:47.657+0000 7f2fb1c00700 1 -- 172.21.15.18:0/766505314 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2fac115b50 con 0x7f2fac071ea0 2024-04-12T20:51:47.657 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:47.658+0000 7f2fb1c00700 1 -- 172.21.15.18:0/766505314 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2fac05f5f0 con 0x7f2fac071ea0 2024-04-12T20:51:47.658 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:47.659+0000 7f2fa2a00700 1 -- 172.21.15.18:0/766505314 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7f2f9800bbf0 con 0x7f2fac071ea0 2024-04-12T20:51:47.659 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:47.660+0000 7f2fa2a00700 1 --2- 172.21.15.18:0/766505314 >> [v2:172.21.15.18:6800/3048894687,v1:172.21.15.18:6801/3048894687] conn(0x7f2f900692b0 0x7f2f9006b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T20:51:47.659 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:47.660+0000 7f2fa2a00700 1 -- 172.21.15.18:0/766505314 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4634+0+0 (secure 0 0 0) 0x7f2f98088360 con 0x7f2fac071ea0 2024-04-12T20:51:47.659 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:47.660+0000 7f2faaa00700 1 --2- 172.21.15.18:0/766505314 >> [v2:172.21.15.18:6800/3048894687,v1:172.21.15.18:6801/3048894687] conn(0x7f2f900692b0 0x7f2f9006b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T20:51:47.660 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:47.661+0000 7f2faaa00700 1 --2- 172.21.15.18:0/766505314 >> [v2:172.21.15.18:6800/3048894687,v1:172.21.15.18:6801/3048894687] conn(0x7f2f900692b0 0x7f2f9006b760 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f2f9c007fd0 tx=0x7f2f9c009040 comp rx=0 tx=0).ready entity=mgr.14212 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T20:51:47.663 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:47.664+0000 7f2fa2a00700 1 -- 172.21.15.18:0/766505314 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7f2f980583c0 con 0x7f2fac071ea0 2024-04-12T20:51:47.887 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:47.888+0000 7f2fb1c00700 1 -- 172.21.15.18:0/766505314 --> [v2:172.21.15.18:6800/3048894687,v1:172.21.15.18:6801/3048894687] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f2fac115e30 con 0x7f2f900692b0 2024-04-12T20:51:47.890 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:47.891+0000 7f2fa2a00700 1 -- 172.21.15.18:0/766505314 <== mgr.14212 v2:172.21.15.18:6800/3048894687 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+148 (secure 0 0 0) 0x7f2fac115e30 con 0x7f2f900692b0 2024-04-12T20:51:47.890 INFO:teuthology.orchestra.run.smithi018.stdout:HOST ADDR LABELS STATUS 2024-04-12T20:51:47.890 INFO:teuthology.orchestra.run.smithi018.stdout:smithi018 172.21.15.18 2024-04-12T20:51:47.890 INFO:teuthology.orchestra.run.smithi018.stdout:smithi186 172.21.15.186 2024-04-12T20:51:47.891 INFO:teuthology.orchestra.run.smithi018.stdout:2 hosts in cluster 2024-04-12T20:51:47.893 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:47.894+0000 7f2fb1c00700 1 -- 172.21.15.18:0/766505314 >> [v2:172.21.15.18:6800/3048894687,v1:172.21.15.18:6801/3048894687] conn(0x7f2f900692b0 msgr2=0x7f2f9006b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:51:47.893 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:47.894+0000 7f2fb1c00700 1 --2- 172.21.15.18:0/766505314 >> [v2:172.21.15.18:6800/3048894687,v1:172.21.15.18:6801/3048894687] conn(0x7f2f900692b0 0x7f2f9006b760 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f2f9c007fd0 tx=0x7f2f9c009040 comp rx=0 tx=0).stop 2024-04-12T20:51:47.893 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:47.894+0000 7f2fb1c00700 1 -- 172.21.15.18:0/766505314 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2fac071ea0 msgr2=0x7f2fac114540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:51:47.894 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:47.894+0000 7f2fb1c00700 1 --2- 172.21.15.18:0/766505314 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2fac071ea0 0x7f2fac114540 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f2f9800b3b0 tx=0x7f2f98009e30 comp rx=0 tx=0).stop 2024-04-12T20:51:47.894 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:47.894+0000 7f2fb1c00700 1 -- 172.21.15.18:0/766505314 shutdown_connections 2024-04-12T20:51:47.894 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:47.894+0000 7f2fb1c00700 1 --2- 172.21.15.18:0/766505314 >> [v2:172.21.15.18:6800/3048894687,v1:172.21.15.18:6801/3048894687] conn(0x7f2f900692b0 0x7f2f9006b760 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:51:47.894 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:47.894+0000 7f2fb1c00700 1 --2- 172.21.15.18:0/766505314 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2fac0708f0 0x7f2fac114000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:51:47.894 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:47.894+0000 7f2fb1c00700 1 --2- 172.21.15.18:0/766505314 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2fac071ea0 0x7f2fac114540 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:51:47.894 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:47.894+0000 7f2fb1c00700 1 -- 172.21.15.18:0/766505314 >> 172.21.15.18:0/766505314 conn(0x7f2fac0f8510 msgr2=0x7f2fac0f9d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T20:51:47.894 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:47.894+0000 7f2fb1c00700 1 -- 172.21.15.18:0/766505314 shutdown_connections 2024-04-12T20:51:47.894 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:47.894+0000 7f2fb1c00700 1 -- 172.21.15.18:0/766505314 wait complete. 2024-04-12T20:51:48.531 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:51:48 smithi186 ceph-mon[36597]: from='client.14440 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T20:51:48.603 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:51:48 smithi018 ceph-mon[28799]: from='client.14440 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T20:51:48.742 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1cbe407a-f90d-11ee-bc8d-c7b262605968 -- bash -c 'ceph orch device ls' 2024-04-12T20:51:49.271 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/mon.smithi018/config 2024-04-12T20:51:49.459 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:51:49 smithi018 ceph-mon[28799]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-12T20:51:49.531 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:51:49 smithi186 ceph-mon[36597]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-12T20:51:51.436 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:51.436+0000 7f8a57600700 1 -- 172.21.15.18:0/613384272 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8a500662e0 msgr2=0x7f8a500ff9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:51:51.436 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:51.436+0000 7f8a57600700 1 --2- 172.21.15.18:0/613384272 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8a500662e0 0x7f8a500ff9c0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f8a44004940 tx=0x7f8a44009e30 comp rx=0 tx=0).stop 2024-04-12T20:51:51.436 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:51.436+0000 7f8a57600700 1 -- 172.21.15.18:0/613384272 shutdown_connections 2024-04-12T20:51:51.437 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:51.436+0000 7f8a57600700 1 --2- 172.21.15.18:0/613384272 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8a500fff00 0x7f8a50102320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:51:51.437 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:51.436+0000 7f8a57600700 1 --2- 172.21.15.18:0/613384272 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8a500662e0 0x7f8a500ff9c0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:51:51.437 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:51.436+0000 7f8a57600700 1 -- 172.21.15.18:0/613384272 >> 172.21.15.18:0/613384272 conn(0x7f8a500f7430 msgr2=0x7f8a500f9860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T20:51:51.437 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:51.436+0000 7f8a57600700 1 -- 172.21.15.18:0/613384272 shutdown_connections 2024-04-12T20:51:51.437 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:51.437+0000 7f8a57600700 1 -- 172.21.15.18:0/613384272 wait complete. 2024-04-12T20:51:51.438 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:51.438+0000 7f8a57600700 1 Processor -- start 2024-04-12T20:51:51.438 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:51.438+0000 7f8a57600700 1 -- start start 2024-04-12T20:51:51.438 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:51.438+0000 7f8a57600700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8a500662e0 0x7f8a5010da30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T20:51:51.438 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:51.438+0000 7f8a57600700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8a500fff00 0x7f8a5010df70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T20:51:51.438 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:51.438+0000 7f8a57600700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a5010e570 con 0x7f8a500662e0 2024-04-12T20:51:51.438 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:51.438+0000 7f8a57600700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a5010edc0 con 0x7f8a500fff00 2024-04-12T20:51:51.438 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:51.439+0000 7f8a54c00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8a500662e0 0x7f8a5010da30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T20:51:51.438 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:51.439+0000 7f8a54c00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8a500662e0 0x7f8a5010da30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:42326/0 (socket says 172.21.15.18:42326) 2024-04-12T20:51:51.438 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:51.439+0000 7f8a54c00700 1 -- 172.21.15.18:0/3641915990 learned_addr learned my addr 172.21.15.18:0/3641915990 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T20:51:51.438 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:51.439+0000 7f8a4fe00700 1 --2- 172.21.15.18:0/3641915990 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8a500fff00 0x7f8a5010df70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T20:51:51.439 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:51.439+0000 7f8a54c00700 1 -- 172.21.15.18:0/3641915990 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8a500fff00 msgr2=0x7f8a5010df70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:51:51.439 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:51.439+0000 7f8a54c00700 1 --2- 172.21.15.18:0/3641915990 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8a500fff00 0x7f8a5010df70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:51:51.439 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:51.439+0000 7f8a54c00700 1 -- 172.21.15.18:0/3641915990 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a44005040 con 0x7f8a500662e0 2024-04-12T20:51:51.439 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:51.440+0000 7f8a54c00700 1 --2- 172.21.15.18:0/3641915990 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8a500662e0 0x7f8a5010da30 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f8a4400b970 tx=0x7f8a44009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T20:51:51.439 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:51.440+0000 7f8a4cc00700 1 -- 172.21.15.18:0/3641915990 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8a4400e040 con 0x7f8a500662e0 2024-04-12T20:51:51.439 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:51.440+0000 7f8a4cc00700 1 -- 172.21.15.18:0/3641915990 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f8a44004290 con 0x7f8a500662e0 2024-04-12T20:51:51.439 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:51.440+0000 7f8a4cc00700 1 -- 172.21.15.18:0/3641915990 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8a4401c630 con 0x7f8a500662e0 2024-04-12T20:51:51.439 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:51.440+0000 7f8a57600700 1 -- 172.21.15.18:0/3641915990 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8a5010f040 con 0x7f8a500662e0 2024-04-12T20:51:51.440 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:51.440+0000 7f8a57600700 1 -- 172.21.15.18:0/3641915990 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8a5010f580 con 0x7f8a500662e0 2024-04-12T20:51:51.440 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:51.442+0000 7f8a57600700 1 -- 172.21.15.18:0/3641915990 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8a5005f5f0 con 0x7f8a500662e0 2024-04-12T20:51:51.441 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:51.443+0000 7f8a4cc00700 1 -- 172.21.15.18:0/3641915990 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7f8a4400bbf0 con 0x7f8a500662e0 2024-04-12T20:51:51.443 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:51.444+0000 7f8a4cc00700 1 --2- 172.21.15.18:0/3641915990 >> [v2:172.21.15.18:6800/3048894687,v1:172.21.15.18:6801/3048894687] conn(0x7f8a380692b0 0x7f8a3806b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T20:51:51.443 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:51.444+0000 7f8a4cc00700 1 -- 172.21.15.18:0/3641915990 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4634+0+0 (secure 0 0 0) 0x7f8a440881d0 con 0x7f8a500662e0 2024-04-12T20:51:51.443 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:51.444+0000 7f8a4fe00700 1 --2- 172.21.15.18:0/3641915990 >> [v2:172.21.15.18:6800/3048894687,v1:172.21.15.18:6801/3048894687] conn(0x7f8a380692b0 0x7f8a3806b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T20:51:51.444 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:51.445+0000 7f8a4fe00700 1 --2- 172.21.15.18:0/3641915990 >> [v2:172.21.15.18:6800/3048894687,v1:172.21.15.18:6801/3048894687] conn(0x7f8a380692b0 0x7f8a3806b760 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f8a3c004990 tx=0x7f8a3c009ee0 comp rx=0 tx=0).ready entity=mgr.14212 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T20:51:51.447 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:51.448+0000 7f8a4cc00700 1 -- 172.21.15.18:0/3641915990 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7f8a440582b0 con 0x7f8a500662e0 2024-04-12T20:51:51.674 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:51.675+0000 7f8a57600700 1 -- 172.21.15.18:0/3641915990 --> [v2:172.21.15.18:6800/3048894687,v1:172.21.15.18:6801/3048894687] -- mgr_command(tid 0: {"prefix": "orch device ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f8a5010f9a0 con 0x7f8a380692b0 2024-04-12T20:51:51.676 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:51.677+0000 7f8a4cc00700 1 -- 172.21.15.18:0/3641915990 <== mgr.14212 v2:172.21.15.18:6800/3048894687 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+399 (secure 0 0 0) 0x7f8a5010f9a0 con 0x7f8a380692b0 2024-04-12T20:51:51.677 INFO:teuthology.orchestra.run.smithi018.stdout:HOST PATH TYPE DEVICE ID SIZE AVAILABLE REFRESHED REJECT REASONS 2024-04-12T20:51:51.677 INFO:teuthology.orchestra.run.smithi018.stdout:smithi018 /dev/nvme0n1 ssd INTEL_SSDPED1D480GA_PHMB75120091480DGN 447G No 3m ago Has a FileSystem, LVM detected 2024-04-12T20:51:51.677 INFO:teuthology.orchestra.run.smithi018.stdout:smithi186 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_CVFT623300CB400BGN 372G No 90s ago Has a FileSystem, LVM detected 2024-04-12T20:51:51.679 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:51.680+0000 7f8a57600700 1 -- 172.21.15.18:0/3641915990 >> [v2:172.21.15.18:6800/3048894687,v1:172.21.15.18:6801/3048894687] conn(0x7f8a380692b0 msgr2=0x7f8a3806b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:51:51.679 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:51.680+0000 7f8a57600700 1 --2- 172.21.15.18:0/3641915990 >> [v2:172.21.15.18:6800/3048894687,v1:172.21.15.18:6801/3048894687] conn(0x7f8a380692b0 0x7f8a3806b760 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f8a3c004990 tx=0x7f8a3c009ee0 comp rx=0 tx=0).stop 2024-04-12T20:51:51.679 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:51.680+0000 7f8a57600700 1 -- 172.21.15.18:0/3641915990 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8a500662e0 msgr2=0x7f8a5010da30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:51:51.679 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:51.680+0000 7f8a57600700 1 --2- 172.21.15.18:0/3641915990 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8a500662e0 0x7f8a5010da30 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f8a4400b970 tx=0x7f8a44009e30 comp rx=0 tx=0).stop 2024-04-12T20:51:51.679 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:51.680+0000 7f8a57600700 1 -- 172.21.15.18:0/3641915990 shutdown_connections 2024-04-12T20:51:51.679 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:51.680+0000 7f8a57600700 1 --2- 172.21.15.18:0/3641915990 >> [v2:172.21.15.18:6800/3048894687,v1:172.21.15.18:6801/3048894687] conn(0x7f8a380692b0 0x7f8a3806b760 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:51:51.680 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:51.680+0000 7f8a57600700 1 --2- 172.21.15.18:0/3641915990 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8a500662e0 0x7f8a5010da30 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:51:51.680 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:51.680+0000 7f8a57600700 1 --2- 172.21.15.18:0/3641915990 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8a500fff00 0x7f8a5010df70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:51:51.680 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:51.680+0000 7f8a57600700 1 -- 172.21.15.18:0/3641915990 >> 172.21.15.18:0/3641915990 conn(0x7f8a500f7430 msgr2=0x7f8a500f7850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T20:51:51.680 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:51.680+0000 7f8a57600700 1 -- 172.21.15.18:0/3641915990 shutdown_connections 2024-04-12T20:51:51.680 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:51.680+0000 7f8a57600700 1 -- 172.21.15.18:0/3641915990 wait complete. 2024-04-12T20:51:52.103 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:51:51 smithi018 ceph-mon[28799]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-12T20:51:52.281 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:51:51 smithi186 ceph-mon[36597]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-12T20:51:52.635 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-04-12T20:51:52.645 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi018.front.sepia.ceph.com 2024-04-12T20:51:52.646 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1cbe407a-f90d-11ee-bc8d-c7b262605968 -- bash -c 'ceph fs volume create cephfs --placement=4' 2024-04-12T20:51:53.102 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:51:52 smithi018 ceph-mon[28799]: from='client.14444 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T20:51:53.130 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/mon.smithi018/config 2024-04-12T20:51:53.281 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:51:52 smithi186 ceph-mon[36597]: from='client.14444 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T20:51:54.281 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:51:53 smithi186 ceph-mon[36597]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-12T20:51:54.352 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:51:53 smithi018 ceph-mon[28799]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-12T20:51:55.443 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:55.443+0000 7fcee3000700 1 -- 172.21.15.18:0/3310554463 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fcedc1002c0 msgr2=0x7fcedc100730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:51:55.443 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:55.443+0000 7fcee3000700 1 --2- 172.21.15.18:0/3310554463 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fcedc1002c0 0x7fcedc100730 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fcec8004940 tx=0x7fcec8009e30 comp rx=0 tx=0).stop 2024-04-12T20:51:55.443 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:55.444+0000 7fcee3000700 1 -- 172.21.15.18:0/3310554463 shutdown_connections 2024-04-12T20:51:55.443 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:55.444+0000 7fcee3000700 1 --2- 172.21.15.18:0/3310554463 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fcedc1002c0 0x7fcedc100730 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:51:55.443 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:55.444+0000 7fcee3000700 1 --2- 172.21.15.18:0/3310554463 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcedc0ff010 0x7fcedc0ff420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:51:55.443 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:55.444+0000 7fcee3000700 1 -- 172.21.15.18:0/3310554463 >> 172.21.15.18:0/3310554463 conn(0x7fcedc0fa790 msgr2=0x7fcedc0fcba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T20:51:55.443 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:55.444+0000 7fcee3000700 1 -- 172.21.15.18:0/3310554463 shutdown_connections 2024-04-12T20:51:55.444 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:55.444+0000 7fcee3000700 1 -- 172.21.15.18:0/3310554463 wait complete. 2024-04-12T20:51:55.445 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:55.445+0000 7fcee3000700 1 Processor -- start 2024-04-12T20:51:55.445 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:55.445+0000 7fcee3000700 1 -- start start 2024-04-12T20:51:55.445 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:55.445+0000 7fcee3000700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcedc0ff010 0x7fcedc10fb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T20:51:55.445 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:55.445+0000 7fcee3000700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fcedc1002c0 0x7fcedc110090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T20:51:55.445 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:55.445+0000 7fcee3000700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcedc110690 con 0x7fcedc1002c0 2024-04-12T20:51:55.445 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:55.445+0000 7fcee3000700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcedc110800 con 0x7fcedc0ff010 2024-04-12T20:51:55.445 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:55.446+0000 7fcedb400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fcedc1002c0 0x7fcedc110090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T20:51:55.445 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:55.446+0000 7fcedb400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fcedc1002c0 0x7fcedc110090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:42334/0 (socket says 172.21.15.18:42334) 2024-04-12T20:51:55.445 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:55.446+0000 7fcedb400700 1 -- 172.21.15.18:0/2894081390 learned_addr learned my addr 172.21.15.18:0/2894081390 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T20:51:55.445 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:55.446+0000 7fcedbe00700 1 --2- 172.21.15.18:0/2894081390 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcedc0ff010 0x7fcedc10fb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T20:51:55.445 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:55.446+0000 7fcedb400700 1 -- 172.21.15.18:0/2894081390 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcedc0ff010 msgr2=0x7fcedc10fb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:51:55.445 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:55.446+0000 7fcedb400700 1 --2- 172.21.15.18:0/2894081390 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcedc0ff010 0x7fcedc10fb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:51:55.445 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:55.446+0000 7fcedb400700 1 -- 172.21.15.18:0/2894081390 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcec8005040 con 0x7fcedc1002c0 2024-04-12T20:51:55.446 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:55.446+0000 7fcedb400700 1 --2- 172.21.15.18:0/2894081390 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fcedc1002c0 0x7fcedc110090 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fcec800b3b0 tx=0x7fcec8009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T20:51:55.446 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:55.447+0000 7fced3e00700 1 -- 172.21.15.18:0/2894081390 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcec800e040 con 0x7fcedc1002c0 2024-04-12T20:51:55.447 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:55.447+0000 7fced3e00700 1 -- 172.21.15.18:0/2894081390 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fcec8004290 con 0x7fcedc1002c0 2024-04-12T20:51:55.447 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:55.447+0000 7fcee3000700 1 -- 172.21.15.18:0/2894081390 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcedc115200 con 0x7fcedc1002c0 2024-04-12T20:51:55.447 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:55.447+0000 7fced3e00700 1 -- 172.21.15.18:0/2894081390 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcec801c630 con 0x7fcedc1002c0 2024-04-12T20:51:55.447 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:55.447+0000 7fcee3000700 1 -- 172.21.15.18:0/2894081390 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcedc1157c0 con 0x7fcedc1002c0 2024-04-12T20:51:55.447 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:55.448+0000 7fcee3000700 1 -- 172.21.15.18:0/2894081390 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcedc04c710 con 0x7fcedc1002c0 2024-04-12T20:51:55.448 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:55.449+0000 7fced3e00700 1 -- 172.21.15.18:0/2894081390 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7fcec800bbf0 con 0x7fcedc1002c0 2024-04-12T20:51:55.449 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:55.450+0000 7fced3e00700 1 --2- 172.21.15.18:0/2894081390 >> [v2:172.21.15.18:6800/3048894687,v1:172.21.15.18:6801/3048894687] conn(0x7fcec00692a0 0x7fcec006b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T20:51:55.449 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:55.450+0000 7fced3e00700 1 -- 172.21.15.18:0/2894081390 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4634+0+0 (secure 0 0 0) 0x7fcec80882c0 con 0x7fcedc1002c0 2024-04-12T20:51:55.449 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:55.450+0000 7fcedbe00700 1 --2- 172.21.15.18:0/2894081390 >> [v2:172.21.15.18:6800/3048894687,v1:172.21.15.18:6801/3048894687] conn(0x7fcec00692a0 0x7fcec006b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T20:51:55.450 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:55.451+0000 7fcedbe00700 1 --2- 172.21.15.18:0/2894081390 >> [v2:172.21.15.18:6800/3048894687,v1:172.21.15.18:6801/3048894687] conn(0x7fcec00692a0 0x7fcec006b750 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fcecc007fd0 tx=0x7fcecc009040 comp rx=0 tx=0).ready entity=mgr.14212 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T20:51:55.454 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:55.455+0000 7fced3e00700 1 -- 172.21.15.18:0/2894081390 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7fcec80583a0 con 0x7fcedc1002c0 2024-04-12T20:51:55.705 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:55.706+0000 7fcee3000700 1 -- 172.21.15.18:0/2894081390 --> [v2:172.21.15.18:6800/3048894687,v1:172.21.15.18:6801/3048894687] -- mgr_command(tid 0: {"prefix": "fs volume create", "name": "cephfs", "placement": "4", "target": ["mon-mgr", ""]}) v1 -- 0x7fcedc104620 con 0x7fcec00692a0 2024-04-12T20:51:56.602 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:51:56 smithi018 ceph-mon[28799]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-12T20:51:56.781 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:51:56 smithi186 ceph-mon[36597]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-12T20:51:57.400 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:57.400+0000 7fced3e00700 1 -- 172.21.15.18:0/2894081390 <== mgr.14212 v2:172.21.15.18:6800/3048894687 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7fcedc104620 con 0x7fcec00692a0 2024-04-12T20:51:57.403 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:57.404+0000 7fcee3000700 1 -- 172.21.15.18:0/2894081390 >> [v2:172.21.15.18:6800/3048894687,v1:172.21.15.18:6801/3048894687] conn(0x7fcec00692a0 msgr2=0x7fcec006b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:51:57.403 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:57.404+0000 7fcee3000700 1 --2- 172.21.15.18:0/2894081390 >> [v2:172.21.15.18:6800/3048894687,v1:172.21.15.18:6801/3048894687] conn(0x7fcec00692a0 0x7fcec006b750 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fcecc007fd0 tx=0x7fcecc009040 comp rx=0 tx=0).stop 2024-04-12T20:51:57.404 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:57.404+0000 7fcee3000700 1 -- 172.21.15.18:0/2894081390 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fcedc1002c0 msgr2=0x7fcedc110090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:51:57.404 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:57.404+0000 7fcee3000700 1 --2- 172.21.15.18:0/2894081390 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fcedc1002c0 0x7fcedc110090 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fcec800b3b0 tx=0x7fcec8009e30 comp rx=0 tx=0).stop 2024-04-12T20:51:57.404 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:57.404+0000 7fcee3000700 1 -- 172.21.15.18:0/2894081390 shutdown_connections 2024-04-12T20:51:57.404 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:57.404+0000 7fcee3000700 1 --2- 172.21.15.18:0/2894081390 >> [v2:172.21.15.18:6800/3048894687,v1:172.21.15.18:6801/3048894687] conn(0x7fcec00692a0 0x7fcec006b750 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:51:57.404 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:57.404+0000 7fcee3000700 1 --2- 172.21.15.18:0/2894081390 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcedc0ff010 0x7fcedc10fb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:51:57.404 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:57.404+0000 7fcee3000700 1 --2- 172.21.15.18:0/2894081390 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fcedc1002c0 0x7fcedc110090 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:51:57.405 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:57.404+0000 7fcee3000700 1 -- 172.21.15.18:0/2894081390 >> 172.21.15.18:0/2894081390 conn(0x7fcedc0fa790 msgr2=0x7fcedc0fbe70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T20:51:57.405 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:57.404+0000 7fcee3000700 1 -- 172.21.15.18:0/2894081390 shutdown_connections 2024-04-12T20:51:57.405 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:51:57.404+0000 7fcee3000700 1 -- 172.21.15.18:0/2894081390 wait complete. 2024-04-12T20:51:57.602 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:51:57 smithi018 ceph-mon[28799]: from='client.14448 -' entity='client.admin' cmd=[{"prefix": "fs volume create", "name": "cephfs", "placement": "4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T20:51:57.602 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:51:57 smithi018 ceph-mon[28799]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd pool create", "pool": "cephfs.cephfs.meta"}]: dispatch 2024-04-12T20:51:57.602 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:51:57 smithi018 ceph-mon[28799]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' cmd='[{"prefix": "osd pool create", "pool": "cephfs.cephfs.meta"}]': finished 2024-04-12T20:51:57.602 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:51:57 smithi018 ceph-mon[28799]: osdmap e41: 6 total, 6 up, 6 in 2024-04-12T20:51:57.602 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:51:57 smithi018 ceph-mon[28799]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' cmd=[{"bulk": true, "prefix": "osd pool create", "pool": "cephfs.cephfs.data"}]: dispatch 2024-04-12T20:51:57.602 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:51:57 smithi018 ceph-mon[28799]: pgmap v215: 33 pgs: 32 unknown, 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-12T20:51:57.603 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:51:57 smithi018 ceph-1cbe407a-f90d-11ee-bc8d-c7b262605968-mon-smithi018[28794]: 2024-04-12T20:51:57.361+0000 7fa25da00700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-12T20:51:57.781 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:51:57 smithi186 ceph-mon[36597]: from='client.14448 -' entity='client.admin' cmd=[{"prefix": "fs volume create", "name": "cephfs", "placement": "4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T20:51:57.781 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:51:57 smithi186 ceph-mon[36597]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd pool create", "pool": "cephfs.cephfs.meta"}]: dispatch 2024-04-12T20:51:57.781 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:51:57 smithi186 ceph-mon[36597]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' cmd='[{"prefix": "osd pool create", "pool": "cephfs.cephfs.meta"}]': finished 2024-04-12T20:51:57.782 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:51:57 smithi186 ceph-mon[36597]: osdmap e41: 6 total, 6 up, 6 in 2024-04-12T20:51:57.782 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:51:57 smithi186 ceph-mon[36597]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' cmd=[{"bulk": true, "prefix": "osd pool create", "pool": "cephfs.cephfs.data"}]: dispatch 2024-04-12T20:51:57.782 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:51:57 smithi186 ceph-mon[36597]: pgmap v215: 33 pgs: 32 unknown, 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-12T20:51:58.463 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1cbe407a-f90d-11ee-bc8d-c7b262605968 -- bash -c 'ceph fs dump' 2024-04-12T20:51:58.602 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:51:58 smithi018 ceph-mon[28799]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-12T20:51:58.603 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:51:58 smithi018 ceph-mon[28799]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' cmd='[{"bulk": true, "prefix": "osd pool create", "pool": "cephfs.cephfs.data"}]': finished 2024-04-12T20:51:58.603 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:51:58 smithi018 ceph-mon[28799]: osdmap e42: 6 total, 6 up, 6 in 2024-04-12T20:51:58.603 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:51:58 smithi018 ceph-mon[28799]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs.cephfs.meta", "data": "cephfs.cephfs.data"}]: dispatch 2024-04-12T20:51:58.603 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:51:58 smithi018 ceph-mon[28799]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-12T20:51:58.603 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:51:58 smithi018 ceph-mon[28799]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-04-12T20:51:58.603 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:51:58 smithi018 ceph-mon[28799]: osdmap e43: 6 total, 6 up, 6 in 2024-04-12T20:51:58.603 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:51:58 smithi018 ceph-mon[28799]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs.cephfs.meta", "data": "cephfs.cephfs.data"}]': finished 2024-04-12T20:51:58.603 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:51:58 smithi018 ceph-mon[28799]: fsmap cephfs:0 2024-04-12T20:51:58.603 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:51:58 smithi018 ceph-mon[28799]: Saving service mds.cephfs spec with placement count:4 2024-04-12T20:51:58.603 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:51:58 smithi018 ceph-mon[28799]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' 2024-04-12T20:51:58.603 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:51:58 smithi018 ceph-mon[28799]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-12T20:51:58.603 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:51:58 smithi018 ceph-mon[28799]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-12T20:51:58.603 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:51:58 smithi018 ceph-mon[28799]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-12T20:51:58.604 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:51:58 smithi018 ceph-mon[28799]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' 2024-04-12T20:51:58.604 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:51:58 smithi018 ceph-mon[28799]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi018.acqbsc", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-12T20:51:58.604 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:51:58 smithi018 ceph-mon[28799]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' cmd='[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi018.acqbsc", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-12T20:51:58.604 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:51:58 smithi018 ceph-mon[28799]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-12T20:51:58.604 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:51:58 smithi018 ceph-mon[28799]: Deploying daemon mds.cephfs.smithi018.acqbsc on smithi018 2024-04-12T20:51:58.684 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:51:58 smithi186 ceph-mon[36597]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-12T20:51:58.685 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:51:58 smithi186 ceph-mon[36597]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' cmd='[{"bulk": true, "prefix": "osd pool create", "pool": "cephfs.cephfs.data"}]': finished 2024-04-12T20:51:58.685 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:51:58 smithi186 ceph-mon[36597]: osdmap e42: 6 total, 6 up, 6 in 2024-04-12T20:51:58.685 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:51:58 smithi186 ceph-mon[36597]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs.cephfs.meta", "data": "cephfs.cephfs.data"}]: dispatch 2024-04-12T20:51:58.685 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:51:58 smithi186 ceph-mon[36597]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-12T20:51:58.685 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:51:58 smithi186 ceph-mon[36597]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-04-12T20:51:58.685 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:51:58 smithi186 ceph-mon[36597]: osdmap e43: 6 total, 6 up, 6 in 2024-04-12T20:51:58.685 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:51:58 smithi186 ceph-mon[36597]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs.cephfs.meta", "data": "cephfs.cephfs.data"}]': finished 2024-04-12T20:51:58.685 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:51:58 smithi186 ceph-mon[36597]: fsmap cephfs:0 2024-04-12T20:51:58.685 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:51:58 smithi186 ceph-mon[36597]: Saving service mds.cephfs spec with placement count:4 2024-04-12T20:51:58.685 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:51:58 smithi186 ceph-mon[36597]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' 2024-04-12T20:51:58.685 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:51:58 smithi186 ceph-mon[36597]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-12T20:51:58.685 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:51:58 smithi186 ceph-mon[36597]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-12T20:51:58.685 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:51:58 smithi186 ceph-mon[36597]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-12T20:51:58.685 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:51:58 smithi186 ceph-mon[36597]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' 2024-04-12T20:51:58.686 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:51:58 smithi186 ceph-mon[36597]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi018.acqbsc", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-12T20:51:58.686 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:51:58 smithi186 ceph-mon[36597]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' cmd='[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi018.acqbsc", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-12T20:51:58.686 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:51:58 smithi186 ceph-mon[36597]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-12T20:51:58.686 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:51:58 smithi186 ceph-mon[36597]: Deploying daemon mds.cephfs.smithi018.acqbsc on smithi018 2024-04-12T20:51:59.410 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/mon.smithi018/config 2024-04-12T20:51:59.781 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:51:59 smithi186 ceph-mon[36597]: osdmap e44: 6 total, 6 up, 6 in 2024-04-12T20:51:59.781 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:51:59 smithi186 ceph-mon[36597]: pgmap v219: 65 pgs: 15 creating+peering, 14 active+clean, 36 unknown; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-12T20:51:59.852 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:51:59 smithi018 ceph-mon[28799]: osdmap e44: 6 total, 6 up, 6 in 2024-04-12T20:51:59.852 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:51:59 smithi018 ceph-mon[28799]: pgmap v219: 65 pgs: 15 creating+peering, 14 active+clean, 36 unknown; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-12T20:52:00.603 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:52:00 smithi018 ceph-mon[28799]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-04-12T20:52:00.603 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:52:00 smithi018 ceph-mon[28799]: osdmap e45: 6 total, 6 up, 6 in 2024-04-12T20:52:00.603 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:52:00 smithi018 ceph-mon[28799]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' 2024-04-12T20:52:00.603 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:52:00 smithi018 ceph-mon[28799]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T20:52:00.781 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:52:00 smithi186 ceph-mon[36597]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-04-12T20:52:00.781 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:52:00 smithi186 ceph-mon[36597]: osdmap e45: 6 total, 6 up, 6 in 2024-04-12T20:52:00.781 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:52:00 smithi186 ceph-mon[36597]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' 2024-04-12T20:52:00.781 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:52:00 smithi186 ceph-mon[36597]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T20:52:01.669 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:52:01 smithi018 ceph-mon[28799]: pgmap v221: 65 pgs: 15 creating+peering, 35 active+clean, 15 unknown; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-12T20:52:01.781 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:52:01 smithi186 ceph-mon[36597]: pgmap v221: 65 pgs: 15 creating+peering, 35 active+clean, 15 unknown; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-12T20:52:02.658 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:02.658+0000 7fdeaaa00700 1 -- 172.21.15.18:0/1787925981 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdea4071e20 msgr2=0x7fdea4070330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:52:02.658 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:02.658+0000 7fdeaaa00700 1 --2- 172.21.15.18:0/1787925981 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdea4071e20 0x7fdea4070330 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fde940048b0 tx=0x7fde94009e30 comp rx=0 tx=0).stop 2024-04-12T20:52:02.658 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:02.658+0000 7fdeaaa00700 1 -- 172.21.15.18:0/1787925981 shutdown_connections 2024-04-12T20:52:02.658 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:02.658+0000 7fdeaaa00700 1 --2- 172.21.15.18:0/1787925981 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fdea4070870 0x7fdea4070ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:52:02.659 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:02.658+0000 7fdeaaa00700 1 --2- 172.21.15.18:0/1787925981 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdea4071e20 0x7fdea4070330 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:52:02.659 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:02.658+0000 7fdeaaa00700 1 -- 172.21.15.18:0/1787925981 >> 172.21.15.18:0/1787925981 conn(0x7fdea40f8560 msgr2=0x7fdea40fa970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T20:52:02.659 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:02.658+0000 7fdeaaa00700 1 -- 172.21.15.18:0/1787925981 shutdown_connections 2024-04-12T20:52:02.659 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:02.658+0000 7fdeaaa00700 1 -- 172.21.15.18:0/1787925981 wait complete. 2024-04-12T20:52:02.659 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:02.659+0000 7fdeaaa00700 1 Processor -- start 2024-04-12T20:52:02.660 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:02.660+0000 7fdeaaa00700 1 -- start start 2024-04-12T20:52:02.660 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:02.660+0000 7fdeaaa00700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdea4070870 0x7fdea410fc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T20:52:02.660 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:02.660+0000 7fdeaaa00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fdea4071e20 0x7fdea4110150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T20:52:02.661 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:02.660+0000 7fdeaaa00700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdea4110750 con 0x7fdea4071e20 2024-04-12T20:52:02.661 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:02.660+0000 7fdeaaa00700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdea4110f50 con 0x7fdea4070870 2024-04-12T20:52:02.661 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:02.660+0000 7fdea3400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fdea4071e20 0x7fdea4110150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T20:52:02.661 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:02.660+0000 7fdea3400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fdea4071e20 0x7fdea4110150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:33984/0 (socket says 172.21.15.18:33984) 2024-04-12T20:52:02.661 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:02.660+0000 7fdea3400700 1 -- 172.21.15.18:0/4146740180 learned_addr learned my addr 172.21.15.18:0/4146740180 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T20:52:02.661 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:02.661+0000 7fdea3e00700 1 --2- 172.21.15.18:0/4146740180 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdea4070870 0x7fdea410fc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T20:52:02.661 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:02.661+0000 7fdea3400700 1 -- 172.21.15.18:0/4146740180 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdea4070870 msgr2=0x7fdea410fc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:52:02.661 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:02.661+0000 7fdea3400700 1 --2- 172.21.15.18:0/4146740180 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdea4070870 0x7fdea410fc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:52:02.661 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:02.661+0000 7fdea3400700 1 -- 172.21.15.18:0/4146740180 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde94005040 con 0x7fdea4071e20 2024-04-12T20:52:02.661 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:02.662+0000 7fdea3400700 1 --2- 172.21.15.18:0/4146740180 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fdea4071e20 0x7fdea4110150 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fde900041d0 tx=0x7fde9000b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T20:52:02.662 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:02.662+0000 7fde9be00700 1 -- 172.21.15.18:0/4146740180 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fde90007cc0 con 0x7fdea4071e20 2024-04-12T20:52:02.662 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:02.662+0000 7fde9be00700 1 -- 172.21.15.18:0/4146740180 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fde9000f040 con 0x7fdea4071e20 2024-04-12T20:52:02.662 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:02.662+0000 7fdeaaa00700 1 -- 172.21.15.18:0/4146740180 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdea41111d0 con 0x7fdea4071e20 2024-04-12T20:52:02.662 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:02.662+0000 7fde9be00700 1 -- 172.21.15.18:0/4146740180 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fde9000ca50 con 0x7fdea4071e20 2024-04-12T20:52:02.662 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:02.662+0000 7fdeaaa00700 1 -- 172.21.15.18:0/4146740180 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdea4111710 con 0x7fdea4071e20 2024-04-12T20:52:02.662 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:02.663+0000 7fdeaaa00700 1 -- 172.21.15.18:0/4146740180 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdea405f5f0 con 0x7fdea4071e20 2024-04-12T20:52:02.664 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:02.665+0000 7fde9be00700 1 -- 172.21.15.18:0/4146740180 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7fde9000ccc0 con 0x7fdea4071e20 2024-04-12T20:52:02.665 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:02.666+0000 7fde9be00700 1 --2- 172.21.15.18:0/4146740180 >> [v2:172.21.15.18:6800/3048894687,v1:172.21.15.18:6801/3048894687] conn(0x7fde880692b0 0x7fde8806b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T20:52:02.665 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:02.667+0000 7fde9be00700 1 -- 172.21.15.18:0/4146740180 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5464+0+0 (secure 0 0 0) 0x7fde90086c90 con 0x7fdea4071e20 2024-04-12T20:52:02.665 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:02.667+0000 7fdea3e00700 1 --2- 172.21.15.18:0/4146740180 >> [v2:172.21.15.18:6800/3048894687,v1:172.21.15.18:6801/3048894687] conn(0x7fde880692b0 0x7fde8806b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T20:52:02.666 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:02.667+0000 7fdea3e00700 1 --2- 172.21.15.18:0/4146740180 >> [v2:172.21.15.18:6800/3048894687,v1:172.21.15.18:6801/3048894687] conn(0x7fde880692b0 0x7fde8806b760 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fde94009fd0 tx=0x7fde94009e30 comp rx=0 tx=0).ready entity=mgr.14212 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T20:52:02.670 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:02.672+0000 7fde9be00700 1 -- 172.21.15.18:0/4146740180 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7fde9004e190 con 0x7fdea4071e20 2024-04-12T20:52:02.935 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:02.936+0000 7fdeaaa00700 1 -- 172.21.15.18:0/4146740180 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fdea4107ae0 con 0x7fdea4071e20 2024-04-12T20:52:02.936 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:02.937+0000 7fde9be00700 1 -- 172.21.15.18:0/4146740180 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 4 v4) v1 ==== 75+0+1247 (secure 0 0 0) 0x7fde900194f0 con 0x7fdea4071e20 2024-04-12T20:52:02.937 INFO:teuthology.orchestra.run.smithi018.stdout:e4 2024-04-12T20:52:02.937 INFO:teuthology.orchestra.run.smithi018.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-12T20:52:02.937 INFO:teuthology.orchestra.run.smithi018.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-12T20:52:02.937 INFO:teuthology.orchestra.run.smithi018.stdout:legacy client fscid: 1 2024-04-12T20:52:02.937 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-12T20:52:02.937 INFO:teuthology.orchestra.run.smithi018.stdout:Filesystem 'cephfs' (1) 2024-04-12T20:52:02.938 INFO:teuthology.orchestra.run.smithi018.stdout:fs_name cephfs 2024-04-12T20:52:02.938 INFO:teuthology.orchestra.run.smithi018.stdout:epoch 4 2024-04-12T20:52:02.938 INFO:teuthology.orchestra.run.smithi018.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-04-12T20:52:02.938 INFO:teuthology.orchestra.run.smithi018.stdout:created 2024-04-12T20:51:57.362850+0000 2024-04-12T20:52:02.938 INFO:teuthology.orchestra.run.smithi018.stdout:modified 2024-04-12T20:52:02.410436+0000 2024-04-12T20:52:02.938 INFO:teuthology.orchestra.run.smithi018.stdout:tableserver 0 2024-04-12T20:52:02.938 INFO:teuthology.orchestra.run.smithi018.stdout:root 0 2024-04-12T20:52:02.938 INFO:teuthology.orchestra.run.smithi018.stdout:session_timeout 60 2024-04-12T20:52:02.938 INFO:teuthology.orchestra.run.smithi018.stdout:session_autoclose 300 2024-04-12T20:52:02.938 INFO:teuthology.orchestra.run.smithi018.stdout:max_file_size 1099511627776 2024-04-12T20:52:02.938 INFO:teuthology.orchestra.run.smithi018.stdout:required_client_features {} 2024-04-12T20:52:02.938 INFO:teuthology.orchestra.run.smithi018.stdout:last_failure 0 2024-04-12T20:52:02.938 INFO:teuthology.orchestra.run.smithi018.stdout:last_failure_osd_epoch 0 2024-04-12T20:52:02.938 INFO:teuthology.orchestra.run.smithi018.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-12T20:52:02.939 INFO:teuthology.orchestra.run.smithi018.stdout:max_mds 1 2024-04-12T20:52:02.939 INFO:teuthology.orchestra.run.smithi018.stdout:in 0 2024-04-12T20:52:02.939 INFO:teuthology.orchestra.run.smithi018.stdout:up {0=14452} 2024-04-12T20:52:02.939 INFO:teuthology.orchestra.run.smithi018.stdout:failed 2024-04-12T20:52:02.939 INFO:teuthology.orchestra.run.smithi018.stdout:damaged 2024-04-12T20:52:02.939 INFO:teuthology.orchestra.run.smithi018.stdout:stopped 2024-04-12T20:52:02.939 INFO:teuthology.orchestra.run.smithi018.stdout:data_pools [3] 2024-04-12T20:52:02.939 INFO:teuthology.orchestra.run.smithi018.stdout:metadata_pool 2 2024-04-12T20:52:02.939 INFO:teuthology.orchestra.run.smithi018.stdout:inline_data disabled 2024-04-12T20:52:02.939 INFO:teuthology.orchestra.run.smithi018.stdout:balancer 2024-04-12T20:52:02.939 INFO:teuthology.orchestra.run.smithi018.stdout:standby_count_wanted 0 2024-04-12T20:52:02.939 INFO:teuthology.orchestra.run.smithi018.stdout:[mds.cephfs.smithi018.acqbsc{0:14452} state up:creating seq 1 addr [v2:172.21.15.18:6826/1564065434,v1:172.21.15.18:6827/1564065434] compat {c=[1],r=[1],i=[7ff]}] 2024-04-12T20:52:02.939 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-12T20:52:02.939 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-12T20:52:02.940 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:02.941+0000 7fdeaaa00700 1 -- 172.21.15.18:0/4146740180 >> [v2:172.21.15.18:6800/3048894687,v1:172.21.15.18:6801/3048894687] conn(0x7fde880692b0 msgr2=0x7fde8806b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:52:02.940 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:02.941+0000 7fdeaaa00700 1 --2- 172.21.15.18:0/4146740180 >> [v2:172.21.15.18:6800/3048894687,v1:172.21.15.18:6801/3048894687] conn(0x7fde880692b0 0x7fde8806b760 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fde94009fd0 tx=0x7fde94009e30 comp rx=0 tx=0).stop 2024-04-12T20:52:02.940 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:02.941+0000 7fdeaaa00700 1 -- 172.21.15.18:0/4146740180 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fdea4071e20 msgr2=0x7fdea4110150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:52:02.940 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:02.941+0000 7fdeaaa00700 1 --2- 172.21.15.18:0/4146740180 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fdea4071e20 0x7fdea4110150 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fde900041d0 tx=0x7fde9000b040 comp rx=0 tx=0).stop 2024-04-12T20:52:02.941 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:02.942+0000 7fdeaaa00700 1 -- 172.21.15.18:0/4146740180 shutdown_connections 2024-04-12T20:52:02.941 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:02.942+0000 7fdeaaa00700 1 --2- 172.21.15.18:0/4146740180 >> [v2:172.21.15.18:6800/3048894687,v1:172.21.15.18:6801/3048894687] conn(0x7fde880692b0 0x7fde8806b760 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:52:02.941 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:02.942+0000 7fdeaaa00700 1 --2- 172.21.15.18:0/4146740180 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdea4070870 0x7fdea410fc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:52:02.941 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:02.942+0000 7fdeaaa00700 1 --2- 172.21.15.18:0/4146740180 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fdea4071e20 0x7fdea4110150 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:52:02.941 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:02.942+0000 7fdeaaa00700 1 -- 172.21.15.18:0/4146740180 >> 172.21.15.18:0/4146740180 conn(0x7fdea40f8560 msgr2=0x7fdea40f9c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T20:52:02.941 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:02.942+0000 7fdeaaa00700 1 -- 172.21.15.18:0/4146740180 shutdown_connections 2024-04-12T20:52:02.941 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:02.942+0000 7fdeaaa00700 1 -- 172.21.15.18:0/4146740180 wait complete. 2024-04-12T20:52:02.942 INFO:teuthology.orchestra.run.smithi018.stderr:dumped fsmap epoch 4 2024-04-12T20:52:03.636 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:52:03 smithi186 ceph-mon[36597]: mds.? [v2:172.21.15.18:6826/1564065434,v1:172.21.15.18:6827/1564065434] up:boot 2024-04-12T20:52:03.636 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:52:03 smithi186 ceph-mon[36597]: daemon mds.cephfs.smithi018.acqbsc assigned to filesystem cephfs as rank 0 (now has 1 ranks) 2024-04-12T20:52:03.636 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:52:03 smithi186 ceph-mon[36597]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-04-12T20:52:03.636 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:52:03 smithi186 ceph-mon[36597]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-04-12T20:52:03.636 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:52:03 smithi186 ceph-mon[36597]: Cluster is now healthy 2024-04-12T20:52:03.636 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:52:03 smithi186 ceph-mon[36597]: fsmap cephfs:0 1 up:standby 2024-04-12T20:52:03.637 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:52:03 smithi186 ceph-mon[36597]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi018.acqbsc"}]: dispatch 2024-04-12T20:52:03.637 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:52:03 smithi186 ceph-mon[36597]: fsmap cephfs:1 {0=cephfs.smithi018.acqbsc=up:creating} 2024-04-12T20:52:03.637 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:52:03 smithi186 ceph-mon[36597]: daemon mds.cephfs.smithi018.acqbsc is now active in filesystem cephfs as rank 0 2024-04-12T20:52:03.637 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:52:03 smithi186 ceph-mon[36597]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' 2024-04-12T20:52:03.637 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:52:03 smithi186 ceph-mon[36597]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' 2024-04-12T20:52:03.637 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:52:03 smithi186 ceph-mon[36597]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' 2024-04-12T20:52:03.637 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:52:03 smithi186 ceph-mon[36597]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi186.jdllyn", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-12T20:52:03.637 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:52:03 smithi186 ceph-mon[36597]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' cmd='[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi186.jdllyn", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-12T20:52:03.637 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:52:03 smithi186 ceph-mon[36597]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-12T20:52:03.637 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:52:03 smithi186 ceph-mon[36597]: Deploying daemon mds.cephfs.smithi186.jdllyn on smithi186 2024-04-12T20:52:03.637 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:52:03 smithi186 ceph-mon[36597]: pgmap v222: 65 pgs: 15 creating+peering, 50 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-12T20:52:03.638 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:52:03 smithi186 ceph-mon[36597]: from='client.? 172.21.15.18:0/4146740180' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-12T20:52:03.843 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:52:03 smithi018 ceph-mon[28799]: mds.? [v2:172.21.15.18:6826/1564065434,v1:172.21.15.18:6827/1564065434] up:boot 2024-04-12T20:52:03.843 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:52:03 smithi018 ceph-mon[28799]: daemon mds.cephfs.smithi018.acqbsc assigned to filesystem cephfs as rank 0 (now has 1 ranks) 2024-04-12T20:52:03.843 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:52:03 smithi018 ceph-mon[28799]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-04-12T20:52:03.843 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:52:03 smithi018 ceph-mon[28799]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-04-12T20:52:03.843 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:52:03 smithi018 ceph-mon[28799]: Cluster is now healthy 2024-04-12T20:52:03.843 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:52:03 smithi018 ceph-mon[28799]: fsmap cephfs:0 1 up:standby 2024-04-12T20:52:03.843 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:52:03 smithi018 ceph-mon[28799]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi018.acqbsc"}]: dispatch 2024-04-12T20:52:03.843 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:52:03 smithi018 ceph-mon[28799]: fsmap cephfs:1 {0=cephfs.smithi018.acqbsc=up:creating} 2024-04-12T20:52:03.844 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:52:03 smithi018 ceph-mon[28799]: daemon mds.cephfs.smithi018.acqbsc is now active in filesystem cephfs as rank 0 2024-04-12T20:52:03.844 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:52:03 smithi018 ceph-mon[28799]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' 2024-04-12T20:52:03.844 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:52:03 smithi018 ceph-mon[28799]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' 2024-04-12T20:52:03.844 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:52:03 smithi018 ceph-mon[28799]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' 2024-04-12T20:52:03.844 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:52:03 smithi018 ceph-mon[28799]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi186.jdllyn", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-12T20:52:03.844 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:52:03 smithi018 ceph-mon[28799]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' cmd='[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi186.jdllyn", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-12T20:52:03.844 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:52:03 smithi018 ceph-mon[28799]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-12T20:52:03.844 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:52:03 smithi018 ceph-mon[28799]: Deploying daemon mds.cephfs.smithi186.jdllyn on smithi186 2024-04-12T20:52:03.844 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:52:03 smithi018 ceph-mon[28799]: pgmap v222: 65 pgs: 15 creating+peering, 50 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-12T20:52:03.844 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:52:03 smithi018 ceph-mon[28799]: from='client.? 172.21.15.18:0/4146740180' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-12T20:52:03.895 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-04-12T20:52:03.905 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi018.front.sepia.ceph.com 2024-04-12T20:52:03.906 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1cbe407a-f90d-11ee-bc8d-c7b262605968 -- bash -c 'ceph fs set cephfs max_mds 2' 2024-04-12T20:52:04.405 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/mon.smithi018/config 2024-04-12T20:52:04.718 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:52:04 smithi018 ceph-mon[28799]: mds.? [v2:172.21.15.18:6826/1564065434,v1:172.21.15.18:6827/1564065434] up:active 2024-04-12T20:52:04.718 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:52:04 smithi018 ceph-mon[28799]: fsmap cephfs:1 {0=cephfs.smithi018.acqbsc=up:active} 2024-04-12T20:52:04.781 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:52:04 smithi186 ceph-mon[36597]: mds.? [v2:172.21.15.18:6826/1564065434,v1:172.21.15.18:6827/1564065434] up:active 2024-04-12T20:52:04.781 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:52:04 smithi186 ceph-mon[36597]: fsmap cephfs:1 {0=cephfs.smithi018.acqbsc=up:active} 2024-04-12T20:52:05.723 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:52:05 smithi186 ceph-mon[36597]: pgmap v223: 65 pgs: 65 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.1 KiB/s wr, 3 op/s 2024-04-12T20:52:05.848 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:52:05 smithi018 ceph-mon[28799]: pgmap v223: 65 pgs: 65 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.1 KiB/s wr, 3 op/s 2024-04-12T20:52:06.772 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:06.772+0000 7f41fdc00700 1 -- 172.21.15.18:0/662631757 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f41f80fce80 msgr2=0x7f41f80fd290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:52:06.772 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:06.772+0000 7f41fdc00700 1 --2- 172.21.15.18:0/662631757 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f41f80fce80 0x7f41f80fd290 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f41e8004940 tx=0x7f41e8009e30 comp rx=0 tx=0).stop 2024-04-12T20:52:06.772 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:06.772+0000 7f41fdc00700 1 -- 172.21.15.18:0/662631757 shutdown_connections 2024-04-12T20:52:06.772 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:06.772+0000 7f41fdc00700 1 --2- 172.21.15.18:0/662631757 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f41f80fdff0 0x7f41f80fe460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:52:06.772 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:06.772+0000 7f41fdc00700 1 --2- 172.21.15.18:0/662631757 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f41f80fce80 0x7f41f80fd290 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:52:06.772 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:06.772+0000 7f41fdc00700 1 -- 172.21.15.18:0/662631757 >> 172.21.15.18:0/662631757 conn(0x7f41f80f8540 msgr2=0x7f41f80fa970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T20:52:06.772 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:06.772+0000 7f41fdc00700 1 -- 172.21.15.18:0/662631757 shutdown_connections 2024-04-12T20:52:06.772 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:06.772+0000 7f41fdc00700 1 -- 172.21.15.18:0/662631757 wait complete. 2024-04-12T20:52:06.773 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:06.774+0000 7f41fdc00700 1 Processor -- start 2024-04-12T20:52:06.773 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:06.774+0000 7f41fdc00700 1 -- start start 2024-04-12T20:52:06.773 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:06.774+0000 7f41fdc00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f41f80fce80 0x7f41f810da40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T20:52:06.773 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:06.774+0000 7f41fdc00700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f41f80fdff0 0x7f41f810df80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T20:52:06.773 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:06.774+0000 7f41fdc00700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41f810e580 con 0x7f41f80fce80 2024-04-12T20:52:06.773 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:06.774+0000 7f41fdc00700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41f810edd0 con 0x7f41f80fdff0 2024-04-12T20:52:06.774 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:06.775+0000 7f41f6a00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f41f80fce80 0x7f41f810da40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T20:52:06.774 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:06.775+0000 7f41f6a00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f41f80fce80 0x7f41f810da40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:33992/0 (socket says 172.21.15.18:33992) 2024-04-12T20:52:06.774 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:06.775+0000 7f41f6a00700 1 -- 172.21.15.18:0/953908081 learned_addr learned my addr 172.21.15.18:0/953908081 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T20:52:06.774 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:06.775+0000 7f41f6000700 1 --2- 172.21.15.18:0/953908081 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f41f80fdff0 0x7f41f810df80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T20:52:06.775 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:06.775+0000 7f41f6a00700 1 -- 172.21.15.18:0/953908081 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f41f80fdff0 msgr2=0x7f41f810df80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:52:06.775 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:06.775+0000 7f41f6a00700 1 --2- 172.21.15.18:0/953908081 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f41f80fdff0 0x7f41f810df80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:52:06.775 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:06.775+0000 7f41f6a00700 1 -- 172.21.15.18:0/953908081 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41e8005040 con 0x7f41f80fce80 2024-04-12T20:52:06.775 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:06.776+0000 7f41f6a00700 1 --2- 172.21.15.18:0/953908081 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f41f80fce80 0x7f41f810da40 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f41e800b3b0 tx=0x7f41e8009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T20:52:06.775 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:06.776+0000 7f41eea00700 1 -- 172.21.15.18:0/953908081 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f41e800e040 con 0x7f41f80fce80 2024-04-12T20:52:06.775 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:06.776+0000 7f41eea00700 1 -- 172.21.15.18:0/953908081 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f41e8004290 con 0x7f41f80fce80 2024-04-12T20:52:06.775 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:06.776+0000 7f41fdc00700 1 -- 172.21.15.18:0/953908081 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f41f810f050 con 0x7f41f80fce80 2024-04-12T20:52:06.776 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:06.776+0000 7f41eea00700 1 -- 172.21.15.18:0/953908081 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f41e801c3f0 con 0x7f41f80fce80 2024-04-12T20:52:06.776 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:06.776+0000 7f41fdc00700 1 -- 172.21.15.18:0/953908081 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f41f810f590 con 0x7f41f80fce80 2024-04-12T20:52:06.776 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:06.778+0000 7f41fdc00700 1 -- 172.21.15.18:0/953908081 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f41f805f5f0 con 0x7f41f80fce80 2024-04-12T20:52:06.778 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:06.780+0000 7f41eea00700 1 -- 172.21.15.18:0/953908081 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7f41e800bbf0 con 0x7f41f80fce80 2024-04-12T20:52:06.780 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:06.781+0000 7f41eea00700 1 --2- 172.21.15.18:0/953908081 >> [v2:172.21.15.18:6800/3048894687,v1:172.21.15.18:6801/3048894687] conn(0x7f41dc0692b0 0x7f41dc06b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T20:52:06.780 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:06.781+0000 7f41eea00700 1 -- 172.21.15.18:0/953908081 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5464+0+0 (secure 0 0 0) 0x7f41e8088480 con 0x7f41f80fce80 2024-04-12T20:52:06.780 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:06.781+0000 7f41f6000700 1 --2- 172.21.15.18:0/953908081 >> [v2:172.21.15.18:6800/3048894687,v1:172.21.15.18:6801/3048894687] conn(0x7f41dc0692b0 0x7f41dc06b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T20:52:06.781 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:06.782+0000 7f41f6000700 1 --2- 172.21.15.18:0/953908081 >> [v2:172.21.15.18:6800/3048894687,v1:172.21.15.18:6801/3048894687] conn(0x7f41dc0692b0 0x7f41dc06b760 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f41e4004f40 tx=0x7f41e400b040 comp rx=0 tx=0).ready entity=mgr.14212 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T20:52:06.789 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:06.790+0000 7f41eea00700 1 -- 172.21.15.18:0/953908081 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7f41e8058220 con 0x7f41f80fce80 2024-04-12T20:52:07.093 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:07.094+0000 7f41fdc00700 1 -- 172.21.15.18:0/953908081 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "2"} v 0) v1 -- 0x7f41f810f910 con 0x7f41f80fce80 2024-04-12T20:52:07.304 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:52:07 smithi018 ceph-mon[28799]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' 2024-04-12T20:52:07.304 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:52:07 smithi018 ceph-mon[28799]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' 2024-04-12T20:52:07.304 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:52:07 smithi018 ceph-mon[28799]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' 2024-04-12T20:52:07.304 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:52:07 smithi018 ceph-mon[28799]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi018.szjluo", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-12T20:52:07.304 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:52:07 smithi018 ceph-mon[28799]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' cmd='[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi018.szjluo", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-12T20:52:07.304 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:52:07 smithi018 ceph-mon[28799]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-12T20:52:07.304 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:52:07 smithi018 ceph-mon[28799]: Deploying daemon mds.cephfs.smithi018.szjluo on smithi018 2024-04-12T20:52:07.304 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:52:07 smithi018 ceph-mon[28799]: mds.? [v2:172.21.15.186:6824/1645616336,v1:172.21.15.186:6825/1645616336] up:boot 2024-04-12T20:52:07.304 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:52:07 smithi018 ceph-mon[28799]: fsmap cephfs:1 {0=cephfs.smithi018.acqbsc=up:active} 1 up:standby 2024-04-12T20:52:07.304 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:52:07 smithi018 ceph-mon[28799]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi186.jdllyn"}]: dispatch 2024-04-12T20:52:07.305 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:52:07 smithi018 ceph-mon[28799]: fsmap cephfs:1 {0=cephfs.smithi018.acqbsc=up:active} 1 up:standby 2024-04-12T20:52:07.532 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:52:07 smithi186 ceph-mon[36597]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' 2024-04-12T20:52:07.532 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:52:07 smithi186 ceph-mon[36597]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' 2024-04-12T20:52:07.532 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:52:07 smithi186 ceph-mon[36597]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' 2024-04-12T20:52:07.532 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:52:07 smithi186 ceph-mon[36597]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi018.szjluo", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-12T20:52:07.532 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:52:07 smithi186 ceph-mon[36597]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' cmd='[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi018.szjluo", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-12T20:52:07.533 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:52:07 smithi186 ceph-mon[36597]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-12T20:52:07.533 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:52:07 smithi186 ceph-mon[36597]: Deploying daemon mds.cephfs.smithi018.szjluo on smithi018 2024-04-12T20:52:07.533 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:52:07 smithi186 ceph-mon[36597]: mds.? [v2:172.21.15.186:6824/1645616336,v1:172.21.15.186:6825/1645616336] up:boot 2024-04-12T20:52:07.533 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:52:07 smithi186 ceph-mon[36597]: fsmap cephfs:1 {0=cephfs.smithi018.acqbsc=up:active} 1 up:standby 2024-04-12T20:52:07.533 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:52:07 smithi186 ceph-mon[36597]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi186.jdllyn"}]: dispatch 2024-04-12T20:52:07.533 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:52:07 smithi186 ceph-mon[36597]: fsmap cephfs:1 {0=cephfs.smithi018.acqbsc=up:active} 1 up:standby 2024-04-12T20:52:08.050 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:08.049+0000 7f41eea00700 1 -- 172.21.15.18:0/953908081 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "2"}]=0 v8) v1 ==== 105+0+0 (secure 0 0 0) 0x7f41e8057db0 con 0x7f41f80fce80 2024-04-12T20:52:08.053 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:08.054+0000 7f41fdc00700 1 -- 172.21.15.18:0/953908081 >> [v2:172.21.15.18:6800/3048894687,v1:172.21.15.18:6801/3048894687] conn(0x7f41dc0692b0 msgr2=0x7f41dc06b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:52:08.053 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:08.054+0000 7f41fdc00700 1 --2- 172.21.15.18:0/953908081 >> [v2:172.21.15.18:6800/3048894687,v1:172.21.15.18:6801/3048894687] conn(0x7f41dc0692b0 0x7f41dc06b760 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f41e4004f40 tx=0x7f41e400b040 comp rx=0 tx=0).stop 2024-04-12T20:52:08.053 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:08.054+0000 7f41fdc00700 1 -- 172.21.15.18:0/953908081 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f41f80fce80 msgr2=0x7f41f810da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:52:08.053 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:08.054+0000 7f41fdc00700 1 --2- 172.21.15.18:0/953908081 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f41f80fce80 0x7f41f810da40 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f41e800b3b0 tx=0x7f41e8009e30 comp rx=0 tx=0).stop 2024-04-12T20:52:08.053 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:08.054+0000 7f41fdc00700 1 -- 172.21.15.18:0/953908081 shutdown_connections 2024-04-12T20:52:08.053 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:08.054+0000 7f41fdc00700 1 --2- 172.21.15.18:0/953908081 >> [v2:172.21.15.18:6800/3048894687,v1:172.21.15.18:6801/3048894687] conn(0x7f41dc0692b0 0x7f41dc06b760 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:52:08.053 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:08.054+0000 7f41fdc00700 1 --2- 172.21.15.18:0/953908081 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f41f80fce80 0x7f41f810da40 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:52:08.053 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:08.054+0000 7f41fdc00700 1 --2- 172.21.15.18:0/953908081 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f41f80fdff0 0x7f41f810df80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:52:08.053 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:08.054+0000 7f41fdc00700 1 -- 172.21.15.18:0/953908081 >> 172.21.15.18:0/953908081 conn(0x7f41f80f8540 msgr2=0x7f41f80f9c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T20:52:08.054 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:08.055+0000 7f41fdc00700 1 -- 172.21.15.18:0/953908081 shutdown_connections 2024-04-12T20:52:08.054 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:08.055+0000 7f41fdc00700 1 -- 172.21.15.18:0/953908081 wait complete. 2024-04-12T20:52:08.352 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:52:08 smithi018 ceph-mon[28799]: pgmap v224: 65 pgs: 65 active+clean; 579 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.5 KiB/s wr, 5 op/s 2024-04-12T20:52:08.352 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:52:08 smithi018 ceph-mon[28799]: from='client.? 172.21.15.18:0/953908081' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "2"}]: dispatch 2024-04-12T20:52:08.531 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:52:08 smithi186 ceph-mon[36597]: pgmap v224: 65 pgs: 65 active+clean; 579 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.5 KiB/s wr, 5 op/s 2024-04-12T20:52:08.531 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:52:08 smithi186 ceph-mon[36597]: from='client.? 172.21.15.18:0/953908081' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "2"}]: dispatch 2024-04-12T20:52:09.147 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:52:09 smithi018 ceph-mon[28799]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-04-12T20:52:09.147 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:52:09 smithi018 ceph-mon[28799]: from='client.? 172.21.15.18:0/953908081' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "2"}]': finished 2024-04-12T20:52:09.148 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:52:09 smithi018 ceph-mon[28799]: daemon mds.cephfs.smithi186.jdllyn assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-04-12T20:52:09.148 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:52:09 smithi018 ceph-mon[28799]: Health check failed: insufficient standby MDS daemons available (MDS_INSUFFICIENT_STANDBY) 2024-04-12T20:52:09.148 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:52:09 smithi018 ceph-mon[28799]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-04-12T20:52:09.148 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:52:09 smithi018 ceph-mon[28799]: fsmap cephfs:1 {0=cephfs.smithi018.acqbsc=up:active} 1 up:standby 2024-04-12T20:52:09.148 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:52:09 smithi018 ceph-mon[28799]: fsmap cephfs:2 {0=cephfs.smithi018.acqbsc=up:active,1=cephfs.smithi186.jdllyn=up:creating} 2024-04-12T20:52:09.148 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:52:09 smithi018 ceph-mon[28799]: daemon mds.cephfs.smithi186.jdllyn is now active in filesystem cephfs as rank 1 2024-04-12T20:52:09.151 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-04-12T20:52:09.162 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi018.front.sepia.ceph.com 2024-04-12T20:52:09.162 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1cbe407a-f90d-11ee-bc8d-c7b262605968 -- bash -c 'ceph fs set cephfs allow_standby_replay false' 2024-04-12T20:52:09.531 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:52:09 smithi186 ceph-mon[36597]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-04-12T20:52:09.531 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:52:09 smithi186 ceph-mon[36597]: from='client.? 172.21.15.18:0/953908081' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "2"}]': finished 2024-04-12T20:52:09.532 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:52:09 smithi186 ceph-mon[36597]: daemon mds.cephfs.smithi186.jdllyn assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-04-12T20:52:09.532 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:52:09 smithi186 ceph-mon[36597]: Health check failed: insufficient standby MDS daemons available (MDS_INSUFFICIENT_STANDBY) 2024-04-12T20:52:09.532 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:52:09 smithi186 ceph-mon[36597]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-04-12T20:52:09.532 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:52:09 smithi186 ceph-mon[36597]: fsmap cephfs:1 {0=cephfs.smithi018.acqbsc=up:active} 1 up:standby 2024-04-12T20:52:09.532 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:52:09 smithi186 ceph-mon[36597]: fsmap cephfs:2 {0=cephfs.smithi018.acqbsc=up:active,1=cephfs.smithi186.jdllyn=up:creating} 2024-04-12T20:52:09.532 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:52:09 smithi186 ceph-mon[36597]: daemon mds.cephfs.smithi186.jdllyn is now active in filesystem cephfs as rank 1 2024-04-12T20:52:10.180 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/mon.smithi018/config 2024-04-12T20:52:10.342 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:52:10 smithi018 ceph-mon[28799]: pgmap v225: 65 pgs: 65 active+clean; 579 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.3 KiB/s wr, 4 op/s 2024-04-12T20:52:10.342 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:52:10 smithi018 ceph-mon[28799]: mds.? [v2:172.21.15.186:6824/1645616336,v1:172.21.15.186:6825/1645616336] up:active 2024-04-12T20:52:10.342 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:52:10 smithi018 ceph-mon[28799]: fsmap cephfs:2 {0=cephfs.smithi018.acqbsc=up:active,1=cephfs.smithi186.jdllyn=up:active} 2024-04-12T20:52:10.343 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:52:10 smithi018 ceph-mon[28799]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' 2024-04-12T20:52:10.531 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:52:10 smithi186 ceph-mon[36597]: pgmap v225: 65 pgs: 65 active+clean; 579 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.3 KiB/s wr, 4 op/s 2024-04-12T20:52:10.531 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:52:10 smithi186 ceph-mon[36597]: mds.? [v2:172.21.15.186:6824/1645616336,v1:172.21.15.186:6825/1645616336] up:active 2024-04-12T20:52:10.531 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:52:10 smithi186 ceph-mon[36597]: fsmap cephfs:2 {0=cephfs.smithi018.acqbsc=up:active,1=cephfs.smithi186.jdllyn=up:active} 2024-04-12T20:52:10.531 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:52:10 smithi186 ceph-mon[36597]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' 2024-04-12T20:52:11.470 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:52:11 smithi186 ceph-mon[36597]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' 2024-04-12T20:52:11.471 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:52:11 smithi186 ceph-mon[36597]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' 2024-04-12T20:52:11.471 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:52:11 smithi186 ceph-mon[36597]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' 2024-04-12T20:52:11.471 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:52:11 smithi186 ceph-mon[36597]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi186.wjnafs", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-12T20:52:11.471 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:52:11 smithi186 ceph-mon[36597]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' cmd='[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi186.wjnafs", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-12T20:52:11.471 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:52:11 smithi186 ceph-mon[36597]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-12T20:52:11.471 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:52:11 smithi186 ceph-mon[36597]: Deploying daemon mds.cephfs.smithi186.wjnafs on smithi186 2024-04-12T20:52:11.471 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:52:11 smithi186 ceph-mon[36597]: pgmap v226: 65 pgs: 65 active+clean; 580 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.5 KiB/s wr, 4 op/s 2024-04-12T20:52:11.471 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:52:11 smithi186 ceph-mon[36597]: Health check cleared: MDS_INSUFFICIENT_STANDBY (was: insufficient standby MDS daemons available) 2024-04-12T20:52:11.471 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:52:11 smithi186 ceph-mon[36597]: Cluster is now healthy 2024-04-12T20:52:11.471 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:52:11 smithi186 ceph-mon[36597]: mds.? [v2:172.21.15.18:6828/722443353,v1:172.21.15.18:6829/722443353] up:boot 2024-04-12T20:52:11.471 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:52:11 smithi186 ceph-mon[36597]: fsmap cephfs:2 {0=cephfs.smithi018.acqbsc=up:active,1=cephfs.smithi186.jdllyn=up:active} 1 up:standby 2024-04-12T20:52:11.471 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:52:11 smithi186 ceph-mon[36597]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi018.szjluo"}]: dispatch 2024-04-12T20:52:11.603 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:52:11 smithi018 ceph-mon[28799]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' 2024-04-12T20:52:11.603 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:52:11 smithi018 ceph-mon[28799]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' 2024-04-12T20:52:11.603 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:52:11 smithi018 ceph-mon[28799]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' 2024-04-12T20:52:11.603 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:52:11 smithi018 ceph-mon[28799]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi186.wjnafs", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-12T20:52:11.603 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:52:11 smithi018 ceph-mon[28799]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' cmd='[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi186.wjnafs", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-12T20:52:11.603 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:52:11 smithi018 ceph-mon[28799]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-12T20:52:11.604 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:52:11 smithi018 ceph-mon[28799]: Deploying daemon mds.cephfs.smithi186.wjnafs on smithi186 2024-04-12T20:52:11.604 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:52:11 smithi018 ceph-mon[28799]: pgmap v226: 65 pgs: 65 active+clean; 580 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.5 KiB/s wr, 4 op/s 2024-04-12T20:52:11.604 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:52:11 smithi018 ceph-mon[28799]: Health check cleared: MDS_INSUFFICIENT_STANDBY (was: insufficient standby MDS daemons available) 2024-04-12T20:52:11.604 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:52:11 smithi018 ceph-mon[28799]: Cluster is now healthy 2024-04-12T20:52:11.604 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:52:11 smithi018 ceph-mon[28799]: mds.? [v2:172.21.15.18:6828/722443353,v1:172.21.15.18:6829/722443353] up:boot 2024-04-12T20:52:11.604 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:52:11 smithi018 ceph-mon[28799]: fsmap cephfs:2 {0=cephfs.smithi018.acqbsc=up:active,1=cephfs.smithi186.jdllyn=up:active} 1 up:standby 2024-04-12T20:52:11.604 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:52:11 smithi018 ceph-mon[28799]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi018.szjluo"}]: dispatch 2024-04-12T20:52:12.595 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:12.595+0000 7f72fda00700 1 -- 172.21.15.18:0/1702480462 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f72f8073170 msgr2=0x7f72f8073b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:52:12.595 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:12.595+0000 7f72fda00700 1 --2- 172.21.15.18:0/1702480462 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f72f8073170 0x7f72f8073b80 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f72e4004940 tx=0x7f72e4009e30 comp rx=0 tx=0).stop 2024-04-12T20:52:12.595 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:12.595+0000 7f72fda00700 1 -- 172.21.15.18:0/1702480462 shutdown_connections 2024-04-12T20:52:12.595 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:12.595+0000 7f72fda00700 1 --2- 172.21.15.18:0/1702480462 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f72f8073170 0x7f72f8073b80 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:52:12.595 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:12.595+0000 7f72fda00700 1 --2- 172.21.15.18:0/1702480462 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f72f8072820 0x7f72f8072c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:52:12.595 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:12.595+0000 7f72fda00700 1 -- 172.21.15.18:0/1702480462 >> 172.21.15.18:0/1702480462 conn(0x7f72f80fa5d0 msgr2=0x7f72f80fc9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T20:52:12.595 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:12.596+0000 7f72fda00700 1 -- 172.21.15.18:0/1702480462 shutdown_connections 2024-04-12T20:52:12.595 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:12.596+0000 7f72fda00700 1 -- 172.21.15.18:0/1702480462 wait complete. 2024-04-12T20:52:12.596 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:12.596+0000 7f72fda00700 1 Processor -- start 2024-04-12T20:52:12.597 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:12.597+0000 7f72fda00700 1 -- start start 2024-04-12T20:52:12.597 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:12.597+0000 7f72fda00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f72f8072820 0x7f72f8114020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T20:52:12.597 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:12.597+0000 7f72fda00700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f72f8114560 0x7f72f81153b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T20:52:12.597 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:12.597+0000 7f72fda00700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72f8114a20 con 0x7f72f8072820 2024-04-12T20:52:12.597 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:12.597+0000 7f72fda00700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72f8114b60 con 0x7f72f8114560 2024-04-12T20:52:12.597 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:12.597+0000 7f72f7e00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f72f8072820 0x7f72f8114020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T20:52:12.597 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:12.597+0000 7f72f7e00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f72f8072820 0x7f72f8114020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:35818/0 (socket says 172.21.15.18:35818) 2024-04-12T20:52:12.597 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:12.597+0000 7f72f7e00700 1 -- 172.21.15.18:0/520190145 learned_addr learned my addr 172.21.15.18:0/520190145 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T20:52:12.597 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:12.597+0000 7f72f7400700 1 --2- 172.21.15.18:0/520190145 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f72f8114560 0x7f72f81153b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T20:52:12.597 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:12.598+0000 7f72f7e00700 1 -- 172.21.15.18:0/520190145 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f72f8114560 msgr2=0x7f72f81153b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:52:12.597 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:12.598+0000 7f72f7e00700 1 --2- 172.21.15.18:0/520190145 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f72f8114560 0x7f72f81153b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:52:12.597 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:12.598+0000 7f72f7e00700 1 -- 172.21.15.18:0/520190145 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72e4005040 con 0x7f72f8072820 2024-04-12T20:52:12.597 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:12.598+0000 7f72f7e00700 1 --2- 172.21.15.18:0/520190145 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f72f8072820 0x7f72f8114020 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f72e8002a80 tx=0x7f72e8004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T20:52:12.598 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:12.598+0000 7f72efe00700 1 -- 172.21.15.18:0/520190145 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f72e8007cc0 con 0x7f72f8072820 2024-04-12T20:52:12.598 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:12.598+0000 7f72efe00700 1 -- 172.21.15.18:0/520190145 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f72e800f040 con 0x7f72f8072820 2024-04-12T20:52:12.598 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:12.598+0000 7f72fda00700 1 -- 172.21.15.18:0/520190145 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f72f8115a10 con 0x7f72f8072820 2024-04-12T20:52:12.599 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:12.598+0000 7f72efe00700 1 -- 172.21.15.18:0/520190145 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f72e800dad0 con 0x7f72f8072820 2024-04-12T20:52:12.599 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:12.598+0000 7f72fda00700 1 -- 172.21.15.18:0/520190145 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f72f8115fa0 con 0x7f72f8072820 2024-04-12T20:52:12.599 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:12.600+0000 7f72fda00700 1 -- 172.21.15.18:0/520190145 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f72f810c540 con 0x7f72f8072820 2024-04-12T20:52:12.599 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:12.601+0000 7f72efe00700 1 -- 172.21.15.18:0/520190145 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7f72e800dc30 con 0x7f72f8072820 2024-04-12T20:52:12.601 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:12.602+0000 7f72efe00700 1 --2- 172.21.15.18:0/520190145 >> [v2:172.21.15.18:6800/3048894687,v1:172.21.15.18:6801/3048894687] conn(0x7f72dc0692b0 0x7f72dc06b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T20:52:12.601 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:12.602+0000 7f72efe00700 1 -- 172.21.15.18:0/520190145 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5464+0+0 (secure 0 0 0) 0x7f72e8086fe0 con 0x7f72f8072820 2024-04-12T20:52:12.601 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:12.602+0000 7f72f7400700 1 --2- 172.21.15.18:0/520190145 >> [v2:172.21.15.18:6800/3048894687,v1:172.21.15.18:6801/3048894687] conn(0x7f72dc0692b0 0x7f72dc06b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T20:52:12.601 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:12.603+0000 7f72f7400700 1 --2- 172.21.15.18:0/520190145 >> [v2:172.21.15.18:6800/3048894687,v1:172.21.15.18:6801/3048894687] conn(0x7f72dc0692b0 0x7f72dc06b760 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f72e400b970 tx=0x7f72e4009e30 comp rx=0 tx=0).ready entity=mgr.14212 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T20:52:12.605 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:12.607+0000 7f72efe00700 1 -- 172.21.15.18:0/520190145 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7f72e8056c80 con 0x7f72f8072820 2024-04-12T20:52:12.878 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:12.879+0000 7f72fda00700 1 -- 172.21.15.18:0/520190145 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f72f810bd30 con 0x7f72f8072820 2024-04-12T20:52:12.938 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:12.938+0000 7f72efe00700 1 -- 172.21.15.18:0/520190145 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v12) v1 ==== 122+0+0 (secure 0 0 0) 0x7f72e8056810 con 0x7f72f8072820 2024-04-12T20:52:12.941 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:12.942+0000 7f72fda00700 1 -- 172.21.15.18:0/520190145 >> [v2:172.21.15.18:6800/3048894687,v1:172.21.15.18:6801/3048894687] conn(0x7f72dc0692b0 msgr2=0x7f72dc06b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:52:12.941 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:12.942+0000 7f72fda00700 1 --2- 172.21.15.18:0/520190145 >> [v2:172.21.15.18:6800/3048894687,v1:172.21.15.18:6801/3048894687] conn(0x7f72dc0692b0 0x7f72dc06b760 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f72e400b970 tx=0x7f72e4009e30 comp rx=0 tx=0).stop 2024-04-12T20:52:12.941 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:12.942+0000 7f72fda00700 1 -- 172.21.15.18:0/520190145 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f72f8072820 msgr2=0x7f72f8114020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:52:12.941 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:12.942+0000 7f72fda00700 1 --2- 172.21.15.18:0/520190145 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f72f8072820 0x7f72f8114020 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f72e8002a80 tx=0x7f72e8004600 comp rx=0 tx=0).stop 2024-04-12T20:52:12.941 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:12.942+0000 7f72fda00700 1 -- 172.21.15.18:0/520190145 shutdown_connections 2024-04-12T20:52:12.942 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:12.942+0000 7f72fda00700 1 --2- 172.21.15.18:0/520190145 >> [v2:172.21.15.18:6800/3048894687,v1:172.21.15.18:6801/3048894687] conn(0x7f72dc0692b0 0x7f72dc06b760 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:52:12.942 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:12.942+0000 7f72fda00700 1 --2- 172.21.15.18:0/520190145 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f72f8072820 0x7f72f8114020 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:52:12.942 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:12.942+0000 7f72fda00700 1 --2- 172.21.15.18:0/520190145 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f72f8114560 0x7f72f81153b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:52:12.942 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:12.942+0000 7f72fda00700 1 -- 172.21.15.18:0/520190145 >> 172.21.15.18:0/520190145 conn(0x7f72f80fa5d0 msgr2=0x7f72f80fbdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T20:52:12.942 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:12.943+0000 7f72fda00700 1 -- 172.21.15.18:0/520190145 shutdown_connections 2024-04-12T20:52:12.942 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:12.943+0000 7f72fda00700 1 -- 172.21.15.18:0/520190145 wait complete. 2024-04-12T20:52:13.281 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:52:12 smithi186 ceph-mon[36597]: from='client.? 172.21.15.18:0/520190145' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]: dispatch 2024-04-12T20:52:13.352 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:52:12 smithi018 ceph-mon[28799]: from='client.? 172.21.15.18:0/520190145' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]: dispatch 2024-04-12T20:52:13.792 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-04-12T20:52:13.802 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi018.front.sepia.ceph.com 2024-04-12T20:52:13.802 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1cbe407a-f90d-11ee-bc8d-c7b262605968 -- bash -c 'ceph fs set cephfs inline_data false' 2024-04-12T20:52:14.103 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:52:13 smithi018 ceph-mon[28799]: pgmap v227: 65 pgs: 65 active+clean; 580 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.5 KiB/s wr, 5 op/s 2024-04-12T20:52:14.103 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:52:13 smithi018 ceph-mon[28799]: from='client.? 172.21.15.18:0/520190145' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]': finished 2024-04-12T20:52:14.103 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:52:13 smithi018 ceph-mon[28799]: fsmap cephfs:2 {0=cephfs.smithi018.acqbsc=up:active,1=cephfs.smithi186.jdllyn=up:active} 1 up:standby 2024-04-12T20:52:14.103 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:52:13 smithi018 ceph-mon[28799]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' 2024-04-12T20:52:14.103 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:52:13 smithi018 ceph-mon[28799]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' 2024-04-12T20:52:14.103 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:52:13 smithi018 ceph-mon[28799]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' 2024-04-12T20:52:14.103 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:52:13 smithi018 ceph-mon[28799]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' 2024-04-12T20:52:14.103 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:52:13 smithi018 ceph-mon[28799]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' 2024-04-12T20:52:14.281 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:52:13 smithi186 ceph-mon[36597]: pgmap v227: 65 pgs: 65 active+clean; 580 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.5 KiB/s wr, 5 op/s 2024-04-12T20:52:14.281 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:52:13 smithi186 ceph-mon[36597]: from='client.? 172.21.15.18:0/520190145' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]': finished 2024-04-12T20:52:14.281 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:52:13 smithi186 ceph-mon[36597]: fsmap cephfs:2 {0=cephfs.smithi018.acqbsc=up:active,1=cephfs.smithi186.jdllyn=up:active} 1 up:standby 2024-04-12T20:52:14.281 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:52:13 smithi186 ceph-mon[36597]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' 2024-04-12T20:52:14.282 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:52:13 smithi186 ceph-mon[36597]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' 2024-04-12T20:52:14.282 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:52:13 smithi186 ceph-mon[36597]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' 2024-04-12T20:52:14.282 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:52:13 smithi186 ceph-mon[36597]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' 2024-04-12T20:52:14.282 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:52:13 smithi186 ceph-mon[36597]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' 2024-04-12T20:52:14.314 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/mon.smithi018/config 2024-04-12T20:52:15.103 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:52:14 smithi018 ceph-mon[28799]: mds.? [v2:172.21.15.186:6826/438347207,v1:172.21.15.186:6827/438347207] up:boot 2024-04-12T20:52:15.103 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:52:14 smithi018 ceph-mon[28799]: fsmap cephfs:2 {0=cephfs.smithi018.acqbsc=up:active,1=cephfs.smithi186.jdllyn=up:active} 2 up:standby 2024-04-12T20:52:15.103 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:52:14 smithi018 ceph-mon[28799]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi186.wjnafs"}]: dispatch 2024-04-12T20:52:15.103 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:52:14 smithi018 ceph-mon[28799]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-12T20:52:15.103 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:52:14 smithi018 ceph-mon[28799]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' 2024-04-12T20:52:15.103 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:52:14 smithi018 ceph-mon[28799]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T20:52:15.281 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:52:14 smithi186 ceph-mon[36597]: mds.? [v2:172.21.15.186:6826/438347207,v1:172.21.15.186:6827/438347207] up:boot 2024-04-12T20:52:15.282 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:52:14 smithi186 ceph-mon[36597]: fsmap cephfs:2 {0=cephfs.smithi018.acqbsc=up:active,1=cephfs.smithi186.jdllyn=up:active} 2 up:standby 2024-04-12T20:52:15.282 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:52:14 smithi186 ceph-mon[36597]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi186.wjnafs"}]: dispatch 2024-04-12T20:52:15.282 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:52:14 smithi186 ceph-mon[36597]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-12T20:52:15.282 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:52:14 smithi186 ceph-mon[36597]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' 2024-04-12T20:52:15.282 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:52:14 smithi186 ceph-mon[36597]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T20:52:16.102 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:52:15 smithi018 ceph-mon[28799]: pgmap v228: 65 pgs: 65 active+clean; 580 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.0 KiB/s wr, 6 op/s 2024-04-12T20:52:16.103 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:52:15 smithi018 ceph-mon[28799]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' 2024-04-12T20:52:16.103 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:52:15 smithi018 ceph-mon[28799]: mds.? [v2:172.21.15.18:6828/722443353,v1:172.21.15.18:6829/722443353] up:standby 2024-04-12T20:52:16.103 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:52:15 smithi018 ceph-mon[28799]: mds.? [v2:172.21.15.18:6826/1564065434,v1:172.21.15.18:6827/1564065434] up:active 2024-04-12T20:52:16.103 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:52:15 smithi018 ceph-mon[28799]: Dropping low affinity active daemon mds.cephfs.smithi186.jdllyn in favor of higher affinity standby. 2024-04-12T20:52:16.103 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:52:15 smithi018 ceph-mon[28799]: Replacing daemon mds.cephfs.smithi186.jdllyn as rank 1 with standby daemon mds.cephfs.smithi018.szjluo 2024-04-12T20:52:16.103 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:52:15 smithi018 ceph-mon[28799]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-04-12T20:52:16.103 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:52:15 smithi018 ceph-mon[28799]: fsmap cephfs:2 {0=cephfs.smithi018.acqbsc=up:active,1=cephfs.smithi186.jdllyn=up:active} 2 up:standby 2024-04-12T20:52:16.103 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:52:15 smithi018 ceph-mon[28799]: osdmap e46: 6 total, 6 up, 6 in 2024-04-12T20:52:16.103 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:52:15 smithi018 ceph-mon[28799]: fsmap cephfs:2/2 {0=cephfs.smithi018.acqbsc=up:active,1=cephfs.smithi018.szjluo=up:replay} 1 up:standby 2024-04-12T20:52:16.273 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:52:15 smithi186 ceph-mon[36597]: pgmap v228: 65 pgs: 65 active+clean; 580 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.0 KiB/s wr, 6 op/s 2024-04-12T20:52:16.273 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:52:15 smithi186 ceph-mon[36597]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' 2024-04-12T20:52:16.273 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:52:15 smithi186 ceph-mon[36597]: mds.? [v2:172.21.15.18:6828/722443353,v1:172.21.15.18:6829/722443353] up:standby 2024-04-12T20:52:16.273 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:52:15 smithi186 ceph-mon[36597]: mds.? [v2:172.21.15.18:6826/1564065434,v1:172.21.15.18:6827/1564065434] up:active 2024-04-12T20:52:16.273 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:52:15 smithi186 ceph-mon[36597]: Dropping low affinity active daemon mds.cephfs.smithi186.jdllyn in favor of higher affinity standby. 2024-04-12T20:52:16.273 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:52:15 smithi186 ceph-mon[36597]: Replacing daemon mds.cephfs.smithi186.jdllyn as rank 1 with standby daemon mds.cephfs.smithi018.szjluo 2024-04-12T20:52:16.273 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:52:15 smithi186 ceph-mon[36597]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-04-12T20:52:16.273 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:52:15 smithi186 ceph-mon[36597]: fsmap cephfs:2 {0=cephfs.smithi018.acqbsc=up:active,1=cephfs.smithi186.jdllyn=up:active} 2 up:standby 2024-04-12T20:52:16.273 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:52:15 smithi186 ceph-mon[36597]: osdmap e46: 6 total, 6 up, 6 in 2024-04-12T20:52:16.273 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:52:15 smithi186 ceph-mon[36597]: fsmap cephfs:2/2 {0=cephfs.smithi018.acqbsc=up:active,1=cephfs.smithi018.szjluo=up:replay} 1 up:standby 2024-04-12T20:52:17.271 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:52:16 smithi018 ceph-mon[28799]: mds.? [v2:172.21.15.18:6828/722443353,v1:172.21.15.18:6829/722443353] up:resolve 2024-04-12T20:52:17.271 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:52:16 smithi018 ceph-mon[28799]: mds.? [v2:172.21.15.186:6824/1464359952,v1:172.21.15.186:6825/1464359952] up:boot 2024-04-12T20:52:17.271 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:52:16 smithi018 ceph-mon[28799]: fsmap cephfs:2/2 {0=cephfs.smithi018.acqbsc=up:active,1=cephfs.smithi018.szjluo=up:resolve} 2 up:standby 2024-04-12T20:52:17.271 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:52:16 smithi018 ceph-mon[28799]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi186.jdllyn"}]: dispatch 2024-04-12T20:52:17.281 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:52:16 smithi186 ceph-mon[36597]: mds.? [v2:172.21.15.18:6828/722443353,v1:172.21.15.18:6829/722443353] up:resolve 2024-04-12T20:52:17.281 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:52:16 smithi186 ceph-mon[36597]: mds.? [v2:172.21.15.186:6824/1464359952,v1:172.21.15.186:6825/1464359952] up:boot 2024-04-12T20:52:17.281 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:52:16 smithi186 ceph-mon[36597]: fsmap cephfs:2/2 {0=cephfs.smithi018.acqbsc=up:active,1=cephfs.smithi018.szjluo=up:resolve} 2 up:standby 2024-04-12T20:52:17.281 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:52:16 smithi186 ceph-mon[36597]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi186.jdllyn"}]: dispatch 2024-04-12T20:52:17.332 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:17.332+0000 7f836ce00700 1 -- 172.21.15.18:0/325574284 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f83680662e0 msgr2=0x7f83680ff9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:52:17.332 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:17.332+0000 7f836ce00700 1 --2- 172.21.15.18:0/325574284 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f83680662e0 0x7f83680ff9c0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f8358004940 tx=0x7f8358009e30 comp rx=0 tx=0).stop 2024-04-12T20:52:17.332 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:17.332+0000 7f836ce00700 1 -- 172.21.15.18:0/325574284 shutdown_connections 2024-04-12T20:52:17.332 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:17.332+0000 7f836ce00700 1 --2- 172.21.15.18:0/325574284 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f83680fff00 0x7f8368102320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:52:17.332 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:17.332+0000 7f836ce00700 1 --2- 172.21.15.18:0/325574284 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f83680662e0 0x7f83680ff9c0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:52:17.333 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:17.332+0000 7f836ce00700 1 -- 172.21.15.18:0/325574284 >> 172.21.15.18:0/325574284 conn(0x7f83680f7430 msgr2=0x7f83680f9860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T20:52:17.333 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:17.333+0000 7f836ce00700 1 -- 172.21.15.18:0/325574284 shutdown_connections 2024-04-12T20:52:17.333 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:17.333+0000 7f836ce00700 1 -- 172.21.15.18:0/325574284 wait complete. 2024-04-12T20:52:17.333 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:17.333+0000 7f836ce00700 1 Processor -- start 2024-04-12T20:52:17.335 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:17.334+0000 7f836ce00700 1 -- start start 2024-04-12T20:52:17.335 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:17.334+0000 7f836ce00700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f83680662e0 0x7f836810fc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T20:52:17.335 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:17.334+0000 7f836ce00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f83680fff00 0x7f83681101a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T20:52:17.335 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:17.334+0000 7f836ce00700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83681107a0 con 0x7f83680fff00 2024-04-12T20:52:17.335 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:17.334+0000 7f836ce00700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8368110fa0 con 0x7f83680662e0 2024-04-12T20:52:17.335 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:17.334+0000 7f8365400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f83680fff00 0x7f83681101a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T20:52:17.335 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:17.334+0000 7f8365400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f83680fff00 0x7f83681101a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:35834/0 (socket says 172.21.15.18:35834) 2024-04-12T20:52:17.335 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:17.334+0000 7f8365400700 1 -- 172.21.15.18:0/4208615591 learned_addr learned my addr 172.21.15.18:0/4208615591 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T20:52:17.335 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:17.334+0000 7f8365e00700 1 --2- 172.21.15.18:0/4208615591 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f83680662e0 0x7f836810fc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T20:52:17.335 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:17.335+0000 7f8365400700 1 -- 172.21.15.18:0/4208615591 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f83680662e0 msgr2=0x7f836810fc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:52:17.335 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:17.335+0000 7f8365400700 1 --2- 172.21.15.18:0/4208615591 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f83680662e0 0x7f836810fc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:52:17.336 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:17.335+0000 7f8365400700 1 -- 172.21.15.18:0/4208615591 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8358005040 con 0x7f83680fff00 2024-04-12T20:52:17.336 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:17.335+0000 7f8365400700 1 --2- 172.21.15.18:0/4208615591 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f83680fff00 0x7f83681101a0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f8354002a80 tx=0x7f83540046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T20:52:17.336 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:17.336+0000 7f835e000700 1 -- 172.21.15.18:0/4208615591 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8354007cc0 con 0x7f83680fff00 2024-04-12T20:52:17.336 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:17.336+0000 7f835e000700 1 -- 172.21.15.18:0/4208615591 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f835400f040 con 0x7f83680fff00 2024-04-12T20:52:17.336 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:17.336+0000 7f836ce00700 1 -- 172.21.15.18:0/4208615591 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8368111280 con 0x7f83680fff00 2024-04-12T20:52:17.336 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:17.336+0000 7f835e000700 1 -- 172.21.15.18:0/4208615591 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f835400dac0 con 0x7f83680fff00 2024-04-12T20:52:17.336 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:17.336+0000 7f836ce00700 1 -- 172.21.15.18:0/4208615591 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f83681117c0 con 0x7f83680fff00 2024-04-12T20:52:17.336 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:17.337+0000 7f836ce00700 1 -- 172.21.15.18:0/4208615591 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f836805f5f0 con 0x7f83680fff00 2024-04-12T20:52:17.336 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:17.338+0000 7f835e000700 1 -- 172.21.15.18:0/4208615591 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7f835400dc20 con 0x7f83680fff00 2024-04-12T20:52:17.338 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:17.339+0000 7f835e000700 1 --2- 172.21.15.18:0/4208615591 >> [v2:172.21.15.18:6800/3048894687,v1:172.21.15.18:6801/3048894687] conn(0x7f834c0692b0 0x7f834c06b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T20:52:17.338 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:17.340+0000 7f835e000700 1 -- 172.21.15.18:0/4208615591 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5550+0+0 (secure 0 0 0) 0x7f8354086e00 con 0x7f83680fff00 2024-04-12T20:52:17.339 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:17.340+0000 7f8365e00700 1 --2- 172.21.15.18:0/4208615591 >> [v2:172.21.15.18:6800/3048894687,v1:172.21.15.18:6801/3048894687] conn(0x7f834c0692b0 0x7f834c06b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T20:52:17.339 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:17.340+0000 7f8365e00700 1 --2- 172.21.15.18:0/4208615591 >> [v2:172.21.15.18:6800/3048894687,v1:172.21.15.18:6801/3048894687] conn(0x7f834c0692b0 0x7f834c06b760 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f835800b3b0 tx=0x7f8358009e30 comp rx=0 tx=0).ready entity=mgr.14212 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T20:52:17.343 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:17.345+0000 7f835e000700 1 -- 172.21.15.18:0/4208615591 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7f8354056ad0 con 0x7f83680fff00 2024-04-12T20:52:17.633 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:17.634+0000 7f836ce00700 1 -- 172.21.15.18:0/4208615591 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "inline_data", "val": "false"} v 0) v1 -- 0x7f836804bed0 con 0x7f83680fff00 2024-04-12T20:52:18.000 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:18.001+0000 7f835e000700 1 -- 172.21.15.18:0/4208615591 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "inline_data", "val": "false"}]=0 inline data disabled v18) v1 ==== 133+0+0 (secure 0 0 0) 0x7f8354056660 con 0x7f83680fff00 2024-04-12T20:52:18.003 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:18.004+0000 7f836ce00700 1 -- 172.21.15.18:0/4208615591 >> [v2:172.21.15.18:6800/3048894687,v1:172.21.15.18:6801/3048894687] conn(0x7f834c0692b0 msgr2=0x7f834c06b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:52:18.003 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:18.004+0000 7f836ce00700 1 --2- 172.21.15.18:0/4208615591 >> [v2:172.21.15.18:6800/3048894687,v1:172.21.15.18:6801/3048894687] conn(0x7f834c0692b0 0x7f834c06b760 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f835800b3b0 tx=0x7f8358009e30 comp rx=0 tx=0).stop 2024-04-12T20:52:18.004 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:18.004+0000 7f836ce00700 1 -- 172.21.15.18:0/4208615591 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f83680fff00 msgr2=0x7f83681101a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:52:18.004 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:18.004+0000 7f836ce00700 1 --2- 172.21.15.18:0/4208615591 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f83680fff00 0x7f83681101a0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f8354002a80 tx=0x7f83540046b0 comp rx=0 tx=0).stop 2024-04-12T20:52:18.004 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:18.004+0000 7f836ce00700 1 -- 172.21.15.18:0/4208615591 shutdown_connections 2024-04-12T20:52:18.004 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:18.004+0000 7f836ce00700 1 --2- 172.21.15.18:0/4208615591 >> [v2:172.21.15.18:6800/3048894687,v1:172.21.15.18:6801/3048894687] conn(0x7f834c0692b0 0x7f834c06b760 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:52:18.004 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:18.004+0000 7f836ce00700 1 --2- 172.21.15.18:0/4208615591 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f83680662e0 0x7f836810fc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:52:18.004 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:18.004+0000 7f836ce00700 1 --2- 172.21.15.18:0/4208615591 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f83680fff00 0x7f83681101a0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:52:18.004 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:18.004+0000 7f836ce00700 1 -- 172.21.15.18:0/4208615591 >> 172.21.15.18:0/4208615591 conn(0x7f83680f7430 msgr2=0x7f83680fba40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T20:52:18.004 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:18.004+0000 7f836ce00700 1 -- 172.21.15.18:0/4208615591 shutdown_connections 2024-04-12T20:52:18.004 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:18.005+0000 7f836ce00700 1 -- 172.21.15.18:0/4208615591 wait complete. 2024-04-12T20:52:18.005 INFO:teuthology.orchestra.run.smithi018.stderr:inline data disabled 2024-04-12T20:52:18.281 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:52:17 smithi186 ceph-mon[36597]: pgmap v230: 65 pgs: 65 active+clean; 580 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 102 B/s rd, 1.1 KiB/s wr, 4 op/s 2024-04-12T20:52:18.281 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:52:17 smithi186 ceph-mon[36597]: mds.? [v2:172.21.15.18:6828/722443353,v1:172.21.15.18:6829/722443353] up:reconnect 2024-04-12T20:52:18.281 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:52:17 smithi186 ceph-mon[36597]: fsmap cephfs:2/2 {0=cephfs.smithi018.acqbsc=up:active,1=cephfs.smithi018.szjluo=up:reconnect} 2 up:standby 2024-04-12T20:52:18.282 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:52:17 smithi186 ceph-mon[36597]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' 2024-04-12T20:52:18.282 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:52:17 smithi186 ceph-mon[36597]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' 2024-04-12T20:52:18.282 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:52:17 smithi186 ceph-mon[36597]: from='client.? 172.21.15.18:0/4208615591' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "inline_data", "val": "false"}]: dispatch 2024-04-12T20:52:18.333 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:52:17 smithi018 ceph-mon[28799]: pgmap v230: 65 pgs: 65 active+clean; 580 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 102 B/s rd, 1.1 KiB/s wr, 4 op/s 2024-04-12T20:52:18.333 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:52:17 smithi018 ceph-mon[28799]: mds.? [v2:172.21.15.18:6828/722443353,v1:172.21.15.18:6829/722443353] up:reconnect 2024-04-12T20:52:18.333 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:52:17 smithi018 ceph-mon[28799]: fsmap cephfs:2/2 {0=cephfs.smithi018.acqbsc=up:active,1=cephfs.smithi018.szjluo=up:reconnect} 2 up:standby 2024-04-12T20:52:18.333 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:52:17 smithi018 ceph-mon[28799]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' 2024-04-12T20:52:18.333 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:52:17 smithi018 ceph-mon[28799]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' 2024-04-12T20:52:18.333 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:52:17 smithi018 ceph-mon[28799]: from='client.? 172.21.15.18:0/4208615591' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "inline_data", "val": "false"}]: dispatch 2024-04-12T20:52:19.103 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:52:19 smithi018 ceph-mon[28799]: mds.? [v2:172.21.15.18:6828/722443353,v1:172.21.15.18:6829/722443353] up:rejoin 2024-04-12T20:52:19.103 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:52:19 smithi018 ceph-mon[28799]: from='client.? 172.21.15.18:0/4208615591' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "inline_data", "val": "false"}]': finished 2024-04-12T20:52:19.103 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:52:19 smithi018 ceph-mon[28799]: mds.? [v2:172.21.15.186:6826/438347207,v1:172.21.15.186:6827/438347207] up:standby 2024-04-12T20:52:19.103 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:52:19 smithi018 ceph-mon[28799]: fsmap cephfs:2/2 {0=cephfs.smithi018.acqbsc=up:active,1=cephfs.smithi018.szjluo=up:rejoin} 2 up:standby 2024-04-12T20:52:19.103 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:52:19 smithi018 ceph-mon[28799]: daemon mds.cephfs.smithi018.szjluo is now active in filesystem cephfs as rank 1 2024-04-12T20:52:19.281 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:52:19 smithi186 ceph-mon[36597]: mds.? [v2:172.21.15.18:6828/722443353,v1:172.21.15.18:6829/722443353] up:rejoin 2024-04-12T20:52:19.281 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:52:19 smithi186 ceph-mon[36597]: from='client.? 172.21.15.18:0/4208615591' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "inline_data", "val": "false"}]': finished 2024-04-12T20:52:19.281 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:52:19 smithi186 ceph-mon[36597]: mds.? [v2:172.21.15.186:6826/438347207,v1:172.21.15.186:6827/438347207] up:standby 2024-04-12T20:52:19.281 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:52:19 smithi186 ceph-mon[36597]: fsmap cephfs:2/2 {0=cephfs.smithi018.acqbsc=up:active,1=cephfs.smithi018.szjluo=up:rejoin} 2 up:standby 2024-04-12T20:52:19.282 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:52:19 smithi186 ceph-mon[36597]: daemon mds.cephfs.smithi018.szjluo is now active in filesystem cephfs as rank 1 2024-04-12T20:52:19.340 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-04-12T20:52:19.351 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi018.front.sepia.ceph.com 2024-04-12T20:52:19.351 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1cbe407a-f90d-11ee-bc8d-c7b262605968 -- bash -c 'ceph fs dump' 2024-04-12T20:52:20.236 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/mon.smithi018/config 2024-04-12T20:52:20.281 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:52:20 smithi186 ceph-mon[36597]: pgmap v231: 65 pgs: 65 active+clean; 580 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 409 B/s rd, 1.1 KiB/s wr, 4 op/s 2024-04-12T20:52:20.281 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:52:20 smithi186 ceph-mon[36597]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-04-12T20:52:20.281 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:52:20 smithi186 ceph-mon[36597]: Cluster is now healthy 2024-04-12T20:52:20.281 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:52:20 smithi186 ceph-mon[36597]: mds.? [v2:172.21.15.18:6828/722443353,v1:172.21.15.18:6829/722443353] up:active 2024-04-12T20:52:20.281 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:52:20 smithi186 ceph-mon[36597]: fsmap cephfs:2 {0=cephfs.smithi018.acqbsc=up:active,1=cephfs.smithi018.szjluo=up:active} 2 up:standby 2024-04-12T20:52:20.292 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:52:20 smithi018 ceph-mon[28799]: pgmap v231: 65 pgs: 65 active+clean; 580 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 409 B/s rd, 1.1 KiB/s wr, 4 op/s 2024-04-12T20:52:20.292 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:52:20 smithi018 ceph-mon[28799]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-04-12T20:52:20.292 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:52:20 smithi018 ceph-mon[28799]: Cluster is now healthy 2024-04-12T20:52:20.292 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:52:20 smithi018 ceph-mon[28799]: mds.? [v2:172.21.15.18:6828/722443353,v1:172.21.15.18:6829/722443353] up:active 2024-04-12T20:52:20.292 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:52:20 smithi018 ceph-mon[28799]: fsmap cephfs:2 {0=cephfs.smithi018.acqbsc=up:active,1=cephfs.smithi018.szjluo=up:active} 2 up:standby 2024-04-12T20:52:22.239 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:52:22 smithi018 ceph-mon[28799]: pgmap v232: 65 pgs: 65 active+clean; 580 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.3 KiB/s rd, 716 B/s wr, 4 op/s 2024-04-12T20:52:22.281 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:52:22 smithi186 ceph-mon[36597]: pgmap v232: 65 pgs: 65 active+clean; 580 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.3 KiB/s rd, 716 B/s wr, 4 op/s 2024-04-12T20:52:22.909 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:22.909+0000 7f4ad3c00700 1 -- 172.21.15.18:0/1524512810 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4acc0709a0 msgr2=0x7f4acc070e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:52:22.909 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:22.909+0000 7f4ad3c00700 1 --2- 172.21.15.18:0/1524512810 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4acc0709a0 0x7f4acc070e10 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f4ab8004940 tx=0x7f4ab8009e30 comp rx=0 tx=0).stop 2024-04-12T20:52:22.909 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:22.909+0000 7f4ad3c00700 1 -- 172.21.15.18:0/1524512810 shutdown_connections 2024-04-12T20:52:22.909 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:22.909+0000 7f4ad3c00700 1 --2- 172.21.15.18:0/1524512810 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4acc0709a0 0x7f4acc070e10 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:52:22.909 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:22.909+0000 7f4ad3c00700 1 --2- 172.21.15.18:0/1524512810 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4acc071f50 0x7f4acc070460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:52:22.909 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:22.909+0000 7f4ad3c00700 1 -- 172.21.15.18:0/1524512810 >> 172.21.15.18:0/1524512810 conn(0x7f4acc0f8560 msgr2=0x7f4acc0fa970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T20:52:22.909 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:22.909+0000 7f4ad3c00700 1 -- 172.21.15.18:0/1524512810 shutdown_connections 2024-04-12T20:52:22.909 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:22.909+0000 7f4ad3c00700 1 -- 172.21.15.18:0/1524512810 wait complete. 2024-04-12T20:52:22.910 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:22.910+0000 7f4ad3c00700 1 Processor -- start 2024-04-12T20:52:22.910 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:22.911+0000 7f4ad3c00700 1 -- start start 2024-04-12T20:52:22.910 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:22.911+0000 7f4ad3c00700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4acc0709a0 0x7f4acc06e360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T20:52:22.911 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:22.911+0000 7f4ad3c00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4acc071f50 0x7f4acc06e8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T20:52:22.911 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:22.911+0000 7f4ad3c00700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4acc06ffa0 con 0x7f4acc071f50 2024-04-12T20:52:22.911 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:22.911+0000 7f4ad3c00700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4acc06ede0 con 0x7f4acc0709a0 2024-04-12T20:52:22.911 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:22.911+0000 7f4acbe00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4acc071f50 0x7f4acc06e8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T20:52:22.911 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:22.912+0000 7f4acbe00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4acc071f50 0x7f4acc06e8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:46042/0 (socket says 172.21.15.18:46042) 2024-04-12T20:52:22.911 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:22.912+0000 7f4acbe00700 1 -- 172.21.15.18:0/1457812694 learned_addr learned my addr 172.21.15.18:0/1457812694 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T20:52:22.911 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:22.912+0000 7f4ad1200700 1 --2- 172.21.15.18:0/1457812694 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4acc0709a0 0x7f4acc06e360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T20:52:22.912 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:22.912+0000 7f4acbe00700 1 -- 172.21.15.18:0/1457812694 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4acc0709a0 msgr2=0x7f4acc06e360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:52:22.912 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:22.912+0000 7f4acbe00700 1 --2- 172.21.15.18:0/1457812694 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4acc0709a0 0x7f4acc06e360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:52:22.912 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:22.912+0000 7f4acbe00700 1 -- 172.21.15.18:0/1457812694 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ab8005040 con 0x7f4acc071f50 2024-04-12T20:52:22.912 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:22.912+0000 7f4ad1200700 1 --2- 172.21.15.18:0/1457812694 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4acc0709a0 0x7f4acc06e360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-12T20:52:22.912 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:22.912+0000 7f4acbe00700 1 --2- 172.21.15.18:0/1457812694 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4acc071f50 0x7f4acc06e8a0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f4ab800b3b0 tx=0x7f4ab8009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T20:52:22.912 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:22.913+0000 7f4ac8c00700 1 -- 172.21.15.18:0/1457812694 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4ab800e040 con 0x7f4acc071f50 2024-04-12T20:52:22.912 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:22.913+0000 7f4ad3c00700 1 -- 172.21.15.18:0/1457812694 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4acc06f060 con 0x7f4acc071f50 2024-04-12T20:52:22.912 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:22.913+0000 7f4ac8c00700 1 -- 172.21.15.18:0/1457812694 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f4ab8004290 con 0x7f4acc071f50 2024-04-12T20:52:22.912 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:22.913+0000 7f4ac8c00700 1 -- 172.21.15.18:0/1457812694 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4ab801c630 con 0x7f4acc071f50 2024-04-12T20:52:22.912 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:22.913+0000 7f4ad3c00700 1 -- 172.21.15.18:0/1457812694 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4acc122630 con 0x7f4acc071f50 2024-04-12T20:52:22.913 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:22.914+0000 7f4ad3c00700 1 -- 172.21.15.18:0/1457812694 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4acc05f5f0 con 0x7f4acc071f50 2024-04-12T20:52:22.914 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:22.916+0000 7f4ac8c00700 1 -- 172.21.15.18:0/1457812694 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7f4ab800bbf0 con 0x7f4acc071f50 2024-04-12T20:52:22.915 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:22.917+0000 7f4ac8c00700 1 --2- 172.21.15.18:0/1457812694 >> [v2:172.21.15.18:6800/3048894687,v1:172.21.15.18:6801/3048894687] conn(0x7f4ab40692b0 0x7f4ab406b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T20:52:22.916 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:22.917+0000 7f4ac8c00700 1 -- 172.21.15.18:0/1457812694 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5550+0+0 (secure 0 0 0) 0x7f4ab8088710 con 0x7f4acc071f50 2024-04-12T20:52:22.916 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:22.917+0000 7f4ad1200700 1 --2- 172.21.15.18:0/1457812694 >> [v2:172.21.15.18:6800/3048894687,v1:172.21.15.18:6801/3048894687] conn(0x7f4ab40692b0 0x7f4ab406b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T20:52:22.916 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:22.918+0000 7f4ad1200700 1 --2- 172.21.15.18:0/1457812694 >> [v2:172.21.15.18:6800/3048894687,v1:172.21.15.18:6801/3048894687] conn(0x7f4ab40692b0 0x7f4ab406b760 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f4acc06fc20 tx=0x7f4ac0009040 comp rx=0 tx=0).ready entity=mgr.14212 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T20:52:22.925 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:22.926+0000 7f4ac8c00700 1 -- 172.21.15.18:0/1457812694 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7f4ab8054950 con 0x7f4acc071f50 2024-04-12T20:52:23.200 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:23.199+0000 7f4ad3c00700 1 -- 172.21.15.18:0/1457812694 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f4acc122940 con 0x7f4acc071f50 2024-04-12T20:52:23.200 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:23.200+0000 7f4ac8c00700 1 -- 172.21.15.18:0/1457812694 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 19 v19) v1 ==== 76+0+1816 (secure 0 0 0) 0x7f4ab8057f70 con 0x7f4acc071f50 2024-04-12T20:52:23.200 INFO:teuthology.orchestra.run.smithi018.stdout:e19 2024-04-12T20:52:23.200 INFO:teuthology.orchestra.run.smithi018.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-12T20:52:23.200 INFO:teuthology.orchestra.run.smithi018.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-12T20:52:23.200 INFO:teuthology.orchestra.run.smithi018.stdout:legacy client fscid: 1 2024-04-12T20:52:23.200 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-12T20:52:23.200 INFO:teuthology.orchestra.run.smithi018.stdout:Filesystem 'cephfs' (1) 2024-04-12T20:52:23.200 INFO:teuthology.orchestra.run.smithi018.stdout:fs_name cephfs 2024-04-12T20:52:23.200 INFO:teuthology.orchestra.run.smithi018.stdout:epoch 19 2024-04-12T20:52:23.201 INFO:teuthology.orchestra.run.smithi018.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-04-12T20:52:23.201 INFO:teuthology.orchestra.run.smithi018.stdout:created 2024-04-12T20:51:57.362850+0000 2024-04-12T20:52:23.201 INFO:teuthology.orchestra.run.smithi018.stdout:modified 2024-04-12T20:52:19.000070+0000 2024-04-12T20:52:23.201 INFO:teuthology.orchestra.run.smithi018.stdout:tableserver 0 2024-04-12T20:52:23.201 INFO:teuthology.orchestra.run.smithi018.stdout:root 0 2024-04-12T20:52:23.201 INFO:teuthology.orchestra.run.smithi018.stdout:session_timeout 60 2024-04-12T20:52:23.201 INFO:teuthology.orchestra.run.smithi018.stdout:session_autoclose 300 2024-04-12T20:52:23.201 INFO:teuthology.orchestra.run.smithi018.stdout:max_file_size 1099511627776 2024-04-12T20:52:23.201 INFO:teuthology.orchestra.run.smithi018.stdout:required_client_features {} 2024-04-12T20:52:23.201 INFO:teuthology.orchestra.run.smithi018.stdout:last_failure 0 2024-04-12T20:52:23.201 INFO:teuthology.orchestra.run.smithi018.stdout:last_failure_osd_epoch 46 2024-04-12T20:52:23.201 INFO:teuthology.orchestra.run.smithi018.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-12T20:52:23.201 INFO:teuthology.orchestra.run.smithi018.stdout:max_mds 2 2024-04-12T20:52:23.201 INFO:teuthology.orchestra.run.smithi018.stdout:in 0,1 2024-04-12T20:52:23.201 INFO:teuthology.orchestra.run.smithi018.stdout:up {0=14452,1=14464} 2024-04-12T20:52:23.201 INFO:teuthology.orchestra.run.smithi018.stdout:failed 2024-04-12T20:52:23.202 INFO:teuthology.orchestra.run.smithi018.stdout:damaged 2024-04-12T20:52:23.202 INFO:teuthology.orchestra.run.smithi018.stdout:stopped 2024-04-12T20:52:23.202 INFO:teuthology.orchestra.run.smithi018.stdout:data_pools [3] 2024-04-12T20:52:23.202 INFO:teuthology.orchestra.run.smithi018.stdout:metadata_pool 2 2024-04-12T20:52:23.202 INFO:teuthology.orchestra.run.smithi018.stdout:inline_data disabled 2024-04-12T20:52:23.202 INFO:teuthology.orchestra.run.smithi018.stdout:balancer 2024-04-12T20:52:23.202 INFO:teuthology.orchestra.run.smithi018.stdout:standby_count_wanted 1 2024-04-12T20:52:23.202 INFO:teuthology.orchestra.run.smithi018.stdout:[mds.cephfs.smithi018.acqbsc{0:14452} state up:active seq 5 join_fscid=1 addr [v2:172.21.15.18:6826/1564065434,v1:172.21.15.18:6827/1564065434] compat {c=[1],r=[1],i=[7ff]}] 2024-04-12T20:52:23.202 INFO:teuthology.orchestra.run.smithi018.stdout:[mds.cephfs.smithi018.szjluo{1:14464} state up:active seq 6 join_fscid=1 addr [v2:172.21.15.18:6828/722443353,v1:172.21.15.18:6829/722443353] compat {c=[1],r=[1],i=[7ff]}] 2024-04-12T20:52:23.202 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-12T20:52:23.202 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-12T20:52:23.202 INFO:teuthology.orchestra.run.smithi018.stdout:Standby daemons: 2024-04-12T20:52:23.202 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-12T20:52:23.202 INFO:teuthology.orchestra.run.smithi018.stdout:[mds.cephfs.smithi186.wjnafs{-1:24277} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.186:6826/438347207,v1:172.21.15.186:6827/438347207] compat {c=[1],r=[1],i=[7ff]}] 2024-04-12T20:52:23.203 INFO:teuthology.orchestra.run.smithi018.stdout:[mds.cephfs.smithi186.jdllyn{-1:24281} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.186:6824/1464359952,v1:172.21.15.186:6825/1464359952] compat {c=[1],r=[1],i=[7ff]}] 2024-04-12T20:52:23.203 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:23.203+0000 7f4ad3c00700 1 -- 172.21.15.18:0/1457812694 >> [v2:172.21.15.18:6800/3048894687,v1:172.21.15.18:6801/3048894687] conn(0x7f4ab40692b0 msgr2=0x7f4ab406b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:52:23.203 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:23.203+0000 7f4ad3c00700 1 --2- 172.21.15.18:0/1457812694 >> [v2:172.21.15.18:6800/3048894687,v1:172.21.15.18:6801/3048894687] conn(0x7f4ab40692b0 0x7f4ab406b760 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f4acc06fc20 tx=0x7f4ac0009040 comp rx=0 tx=0).stop 2024-04-12T20:52:23.203 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:23.203+0000 7f4ad3c00700 1 -- 172.21.15.18:0/1457812694 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4acc071f50 msgr2=0x7f4acc06e8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:52:23.203 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:23.203+0000 7f4ad3c00700 1 --2- 172.21.15.18:0/1457812694 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4acc071f50 0x7f4acc06e8a0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f4ab800b3b0 tx=0x7f4ab8009e30 comp rx=0 tx=0).stop 2024-04-12T20:52:23.203 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:23.203+0000 7f4ad3c00700 1 -- 172.21.15.18:0/1457812694 shutdown_connections 2024-04-12T20:52:23.204 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:23.203+0000 7f4ad3c00700 1 --2- 172.21.15.18:0/1457812694 >> [v2:172.21.15.18:6800/3048894687,v1:172.21.15.18:6801/3048894687] conn(0x7f4ab40692b0 0x7f4ab406b760 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:52:23.204 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:23.203+0000 7f4ad3c00700 1 --2- 172.21.15.18:0/1457812694 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4acc0709a0 0x7f4acc06e360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:52:23.204 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:23.203+0000 7f4ad3c00700 1 --2- 172.21.15.18:0/1457812694 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4acc071f50 0x7f4acc06e8a0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:52:23.204 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:23.203+0000 7f4ad3c00700 1 -- 172.21.15.18:0/1457812694 >> 172.21.15.18:0/1457812694 conn(0x7f4acc0f8560 msgr2=0x7f4acc0f9d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T20:52:23.204 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:23.203+0000 7f4ad3c00700 1 -- 172.21.15.18:0/1457812694 shutdown_connections 2024-04-12T20:52:23.204 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:23.203+0000 7f4ad3c00700 1 -- 172.21.15.18:0/1457812694 wait complete. 2024-04-12T20:52:23.204 INFO:teuthology.orchestra.run.smithi018.stderr:dumped fsmap epoch 19 2024-04-12T20:52:24.281 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:52:24 smithi186 ceph-mon[36597]: pgmap v233: 65 pgs: 65 active+clean; 580 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.8 KiB/s rd, 614 B/s wr, 4 op/s 2024-04-12T20:52:24.281 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:52:24 smithi186 ceph-mon[36597]: from='client.? 172.21.15.18:0/1457812694' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-12T20:52:24.352 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:52:24 smithi018 ceph-mon[28799]: pgmap v233: 65 pgs: 65 active+clean; 580 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.8 KiB/s rd, 614 B/s wr, 4 op/s 2024-04-12T20:52:24.352 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:52:24 smithi018 ceph-mon[28799]: from='client.? 172.21.15.18:0/1457812694' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-12T20:52:24.659 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1cbe407a-f90d-11ee-bc8d-c7b262605968 -- bash -c 'ceph --format=json fs dump | jq -e ".filesystems | length == 1"' 2024-04-12T20:52:25.190 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/mon.smithi018/config 2024-04-12T20:52:26.409 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:52:26 smithi018 ceph-mon[28799]: pgmap v234: 65 pgs: 65 active+clean; 580 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 4.6 KiB/s rd, 0 B/s wr, 4 op/s 2024-04-12T20:52:26.409 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:52:26 smithi018 ceph-mon[28799]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' 2024-04-12T20:52:26.409 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:52:26 smithi018 ceph-mon[28799]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' 2024-04-12T20:52:26.410 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:52:26 smithi018 ceph-mon[28799]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-12T20:52:26.410 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:52:26 smithi018 ceph-mon[28799]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-12T20:52:26.410 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:52:26 smithi018 ceph-mon[28799]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' 2024-04-12T20:52:26.531 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:52:26 smithi186 ceph-mon[36597]: pgmap v234: 65 pgs: 65 active+clean; 580 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 4.6 KiB/s rd, 0 B/s wr, 4 op/s 2024-04-12T20:52:26.531 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:52:26 smithi186 ceph-mon[36597]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' 2024-04-12T20:52:26.531 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:52:26 smithi186 ceph-mon[36597]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' 2024-04-12T20:52:26.531 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:52:26 smithi186 ceph-mon[36597]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-12T20:52:26.532 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:52:26 smithi186 ceph-mon[36597]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-12T20:52:26.532 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:52:26 smithi186 ceph-mon[36597]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' 2024-04-12T20:52:27.222 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:52:27 smithi018 ceph-mon[28799]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-12T20:52:27.491 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:52:27 smithi186 ceph-mon[36597]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-12T20:52:27.544 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:27.545+0000 7f8acfa00700 1 -- 172.21.15.18:0/641095737 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8ac80fff00 msgr2=0x7f8ac8102320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:52:27.544 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:27.545+0000 7f8acfa00700 1 --2- 172.21.15.18:0/641095737 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8ac80fff00 0x7f8ac8102320 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f8ab4004940 tx=0x7f8ab4009e30 comp rx=0 tx=0).stop 2024-04-12T20:52:27.544 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:27.546+0000 7f8acfa00700 1 -- 172.21.15.18:0/641095737 shutdown_connections 2024-04-12T20:52:27.544 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:27.546+0000 7f8acfa00700 1 --2- 172.21.15.18:0/641095737 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8ac80fff00 0x7f8ac8102320 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:52:27.544 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:27.546+0000 7f8acfa00700 1 --2- 172.21.15.18:0/641095737 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8ac80662e0 0x7f8ac80ff9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:52:27.544 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:27.546+0000 7f8acfa00700 1 -- 172.21.15.18:0/641095737 >> 172.21.15.18:0/641095737 conn(0x7f8ac80f7430 msgr2=0x7f8ac80f9860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T20:52:27.544 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:27.546+0000 7f8acfa00700 1 -- 172.21.15.18:0/641095737 shutdown_connections 2024-04-12T20:52:27.544 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:27.546+0000 7f8acfa00700 1 -- 172.21.15.18:0/641095737 wait complete. 2024-04-12T20:52:27.545 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:27.547+0000 7f8acfa00700 1 Processor -- start 2024-04-12T20:52:27.545 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:27.547+0000 7f8acfa00700 1 -- start start 2024-04-12T20:52:27.545 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:27.547+0000 7f8acfa00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8ac80662e0 0x7f8ac80fd3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T20:52:27.546 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:27.547+0000 7f8acfa00700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8ac80fff00 0x7f8ac80fb8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T20:52:27.546 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:27.547+0000 7f8acfa00700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ac80fbe70 con 0x7f8ac80662e0 2024-04-12T20:52:27.546 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:27.547+0000 7f8acfa00700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ac80fbfe0 con 0x7f8ac80fff00 2024-04-12T20:52:27.546 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:27.547+0000 7f8acd000700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8ac80662e0 0x7f8ac80fd3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T20:52:27.546 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:27.548+0000 7f8acd000700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8ac80662e0 0x7f8ac80fd3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:46064/0 (socket says 172.21.15.18:46064) 2024-04-12T20:52:27.546 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:27.548+0000 7f8acd000700 1 -- 172.21.15.18:0/2915625811 learned_addr learned my addr 172.21.15.18:0/2915625811 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T20:52:27.546 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:27.548+0000 7f8ac7e00700 1 --2- 172.21.15.18:0/2915625811 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8ac80fff00 0x7f8ac80fb8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T20:52:27.547 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:27.548+0000 7f8acd000700 1 -- 172.21.15.18:0/2915625811 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8ac80fff00 msgr2=0x7f8ac80fb8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:52:27.547 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:27.548+0000 7f8acd000700 1 --2- 172.21.15.18:0/2915625811 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8ac80fff00 0x7f8ac80fb8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:52:27.547 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:27.548+0000 7f8acd000700 1 -- 172.21.15.18:0/2915625811 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ab4005040 con 0x7f8ac80662e0 2024-04-12T20:52:27.547 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:27.548+0000 7f8acd000700 1 --2- 172.21.15.18:0/2915625811 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8ac80662e0 0x7f8ac80fd3f0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f8abc00c800 tx=0x7f8abc004470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T20:52:27.547 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:27.549+0000 7f8ac4c00700 1 -- 172.21.15.18:0/2915625811 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8abc00a760 con 0x7f8ac80662e0 2024-04-12T20:52:27.547 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:27.549+0000 7f8ac4c00700 1 -- 172.21.15.18:0/2915625811 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f8abc00ad90 con 0x7f8ac80662e0 2024-04-12T20:52:27.547 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:27.549+0000 7f8ac4c00700 1 -- 172.21.15.18:0/2915625811 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8abc0113a0 con 0x7f8ac80662e0 2024-04-12T20:52:27.547 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:27.549+0000 7f8acfa00700 1 -- 172.21.15.18:0/2915625811 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8ac80fc2c0 con 0x7f8ac80662e0 2024-04-12T20:52:27.547 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:27.549+0000 7f8acfa00700 1 -- 172.21.15.18:0/2915625811 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8ac8115a90 con 0x7f8ac80662e0 2024-04-12T20:52:27.548 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:27.550+0000 7f8acfa00700 1 -- 172.21.15.18:0/2915625811 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8ac805f5f0 con 0x7f8ac80662e0 2024-04-12T20:52:27.550 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:27.552+0000 7f8ac4c00700 1 -- 172.21.15.18:0/2915625811 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7f8abc012400 con 0x7f8ac80662e0 2024-04-12T20:52:27.551 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:27.553+0000 7f8ac4c00700 1 --2- 172.21.15.18:0/2915625811 >> [v2:172.21.15.18:6800/3048894687,v1:172.21.15.18:6801/3048894687] conn(0x7f8ab00692b0 0x7f8ab006b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T20:52:27.552 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:27.553+0000 7f8ac4c00700 1 -- 172.21.15.18:0/2915625811 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5550+0+0 (secure 0 0 0) 0x7f8abc0871f0 con 0x7f8ac80662e0 2024-04-12T20:52:27.552 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:27.554+0000 7f8ac7e00700 1 --2- 172.21.15.18:0/2915625811 >> [v2:172.21.15.18:6800/3048894687,v1:172.21.15.18:6801/3048894687] conn(0x7f8ab00692b0 0x7f8ab006b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T20:52:27.552 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:27.555+0000 7f8ac7e00700 1 --2- 172.21.15.18:0/2915625811 >> [v2:172.21.15.18:6800/3048894687,v1:172.21.15.18:6801/3048894687] conn(0x7f8ab00692b0 0x7f8ab006b760 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f8ab4004f90 tx=0x7f8ab4009e30 comp rx=0 tx=0).ready entity=mgr.14212 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T20:52:27.556 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:27.558+0000 7f8ac4c00700 1 -- 172.21.15.18:0/2915625811 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7f8abc0534b0 con 0x7f8ac80662e0 2024-04-12T20:52:27.858 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:27.860+0000 7f8acfa00700 1 -- 172.21.15.18:0/2915625811 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8ac804bed0 con 0x7f8ac80662e0 2024-04-12T20:52:27.859 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:27.861+0000 7f8ac4c00700 1 -- 172.21.15.18:0/2915625811 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 19 v19) v1 ==== 94+0+4703 (secure 0 0 0) 0x7f8abc056ad0 con 0x7f8ac80662e0 2024-04-12T20:52:27.862 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:27.864+0000 7f8acfa00700 1 -- 172.21.15.18:0/2915625811 >> [v2:172.21.15.18:6800/3048894687,v1:172.21.15.18:6801/3048894687] conn(0x7f8ab00692b0 msgr2=0x7f8ab006b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:52:27.862 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:27.864+0000 7f8acfa00700 1 --2- 172.21.15.18:0/2915625811 >> [v2:172.21.15.18:6800/3048894687,v1:172.21.15.18:6801/3048894687] conn(0x7f8ab00692b0 0x7f8ab006b760 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f8ab4004f90 tx=0x7f8ab4009e30 comp rx=0 tx=0).stop 2024-04-12T20:52:27.862 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:27.864+0000 7f8acfa00700 1 -- 172.21.15.18:0/2915625811 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8ac80662e0 msgr2=0x7f8ac80fd3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:52:27.863 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:27.864+0000 7f8acfa00700 1 --2- 172.21.15.18:0/2915625811 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8ac80662e0 0x7f8ac80fd3f0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f8abc00c800 tx=0x7f8abc004470 comp rx=0 tx=0).stop 2024-04-12T20:52:27.863 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:27.864+0000 7f8acfa00700 1 -- 172.21.15.18:0/2915625811 shutdown_connections 2024-04-12T20:52:27.863 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:27.864+0000 7f8acfa00700 1 --2- 172.21.15.18:0/2915625811 >> [v2:172.21.15.18:6800/3048894687,v1:172.21.15.18:6801/3048894687] conn(0x7f8ab00692b0 0x7f8ab006b760 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:52:27.863 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:27.864+0000 7f8acfa00700 1 --2- 172.21.15.18:0/2915625811 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8ac80662e0 0x7f8ac80fd3f0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:52:27.863 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:27.864+0000 7f8acfa00700 1 --2- 172.21.15.18:0/2915625811 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8ac80fff00 0x7f8ac80fb8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:52:27.863 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:27.864+0000 7f8acfa00700 1 -- 172.21.15.18:0/2915625811 >> 172.21.15.18:0/2915625811 conn(0x7f8ac80f7430 msgr2=0x7f8ac80f7850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T20:52:27.863 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:27.864+0000 7f8acfa00700 1 -- 172.21.15.18:0/2915625811 shutdown_connections 2024-04-12T20:52:27.863 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:27.864+0000 7f8acfa00700 1 -- 172.21.15.18:0/2915625811 wait complete. 2024-04-12T20:52:27.863 INFO:teuthology.orchestra.run.smithi018.stderr:dumped fsmap epoch 19 2024-04-12T20:52:27.878 INFO:teuthology.orchestra.run.smithi018.stdout:true 2024-04-12T20:52:28.460 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:52:28 smithi186 ceph-mon[36597]: pgmap v235: 65 pgs: 65 active+clean; 580 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 3.9 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-12T20:52:28.460 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:52:28 smithi186 ceph-mon[36597]: from='client.? 172.21.15.18:0/2915625811' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-12T20:52:28.511 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:52:28 smithi018 ceph-mon[28799]: pgmap v235: 65 pgs: 65 active+clean; 580 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 3.9 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-12T20:52:28.511 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:52:28 smithi018 ceph-mon[28799]: from='client.? 172.21.15.18:0/2915625811' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-12T20:52:29.266 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1cbe407a-f90d-11ee-bc8d-c7b262605968 -- bash -c 'while ! ceph --format=json mds versions | jq -e ". | add == 4"; do sleep 1; done' 2024-04-12T20:52:30.253 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/mon.smithi018/config 2024-04-12T20:52:30.292 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:52:30 smithi018 ceph-mon[28799]: pgmap v236: 65 pgs: 65 active+clean; 580 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 3.7 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-12T20:52:30.292 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:52:30 smithi018 ceph-mon[28799]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' 2024-04-12T20:52:30.292 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:52:30 smithi018 ceph-mon[28799]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' 2024-04-12T20:52:30.292 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:52:30 smithi018 ceph-mon[28799]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T20:52:30.531 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:52:30 smithi186 ceph-mon[36597]: pgmap v236: 65 pgs: 65 active+clean; 580 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 3.7 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-12T20:52:30.531 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:52:30 smithi186 ceph-mon[36597]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' 2024-04-12T20:52:30.531 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:52:30 smithi186 ceph-mon[36597]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' 2024-04-12T20:52:30.531 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:52:30 smithi186 ceph-mon[36597]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T20:52:32.352 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:52:32 smithi018 ceph-mon[28799]: pgmap v237: 65 pgs: 65 active+clean; 580 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 3.5 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-12T20:52:32.531 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:52:32 smithi186 ceph-mon[36597]: pgmap v237: 65 pgs: 65 active+clean; 580 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 3.5 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-12T20:52:33.859 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:33.860+0000 7fba9d200700 1 -- 172.21.15.18:0/3943633442 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fba98071e40 msgr2=0x7fba98070350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:52:33.859 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:33.860+0000 7fba9d200700 1 --2- 172.21.15.18:0/3943633442 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fba98071e40 0x7fba98070350 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fba88004940 tx=0x7fba88009e30 comp rx=0 tx=0).stop 2024-04-12T20:52:33.859 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:33.860+0000 7fba9d200700 1 -- 172.21.15.18:0/3943633442 shutdown_connections 2024-04-12T20:52:33.859 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:33.860+0000 7fba9d200700 1 --2- 172.21.15.18:0/3943633442 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fba98070890 0x7fba98070d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:52:33.859 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:33.860+0000 7fba9d200700 1 --2- 172.21.15.18:0/3943633442 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fba98071e40 0x7fba98070350 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:52:33.859 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:33.860+0000 7fba9d200700 1 -- 172.21.15.18:0/3943633442 >> 172.21.15.18:0/3943633442 conn(0x7fba980f8580 msgr2=0x7fba980fa990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T20:52:33.859 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:33.860+0000 7fba9d200700 1 -- 172.21.15.18:0/3943633442 shutdown_connections 2024-04-12T20:52:33.859 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:33.860+0000 7fba9d200700 1 -- 172.21.15.18:0/3943633442 wait complete. 2024-04-12T20:52:33.859 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:33.861+0000 7fba9d200700 1 Processor -- start 2024-04-12T20:52:33.859 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:33.861+0000 7fba9d200700 1 -- start start 2024-04-12T20:52:33.861 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:33.861+0000 7fba9d200700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fba98070890 0x7fba9810fb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T20:52:33.861 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:33.861+0000 7fba9d200700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fba98071e40 0x7fba981100b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T20:52:33.861 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:33.861+0000 7fba9d200700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba981106b0 con 0x7fba98070890 2024-04-12T20:52:33.861 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:33.861+0000 7fba9d200700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba98110820 con 0x7fba98071e40 2024-04-12T20:52:33.861 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:33.862+0000 7fba95e00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fba98070890 0x7fba9810fb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T20:52:33.861 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:33.862+0000 7fba95e00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fba98070890 0x7fba9810fb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:35546/0 (socket says 172.21.15.18:35546) 2024-04-12T20:52:33.861 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:33.862+0000 7fba95e00700 1 -- 172.21.15.18:0/1862678422 learned_addr learned my addr 172.21.15.18:0/1862678422 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T20:52:33.861 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:33.862+0000 7fba95400700 1 --2- 172.21.15.18:0/1862678422 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fba98071e40 0x7fba981100b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T20:52:33.861 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:33.862+0000 7fba95e00700 1 -- 172.21.15.18:0/1862678422 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fba98071e40 msgr2=0x7fba981100b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:52:33.861 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:33.862+0000 7fba95e00700 1 --2- 172.21.15.18:0/1862678422 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fba98071e40 0x7fba981100b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:52:33.861 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:33.862+0000 7fba95e00700 1 -- 172.21.15.18:0/1862678422 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba88005040 con 0x7fba98070890 2024-04-12T20:52:33.861 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:33.862+0000 7fba95400700 1 --2- 172.21.15.18:0/1862678422 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fba98071e40 0x7fba981100b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-12T20:52:33.862 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:33.862+0000 7fba95e00700 1 --2- 172.21.15.18:0/1862678422 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fba98070890 0x7fba9810fb70 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fba8800b970 tx=0x7fba88009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T20:52:33.862 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:33.863+0000 7fba8e000700 1 -- 172.21.15.18:0/1862678422 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fba8800e040 con 0x7fba98070890 2024-04-12T20:52:33.862 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:33.863+0000 7fba8e000700 1 -- 172.21.15.18:0/1862678422 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fba88004290 con 0x7fba98070890 2024-04-12T20:52:33.862 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:33.863+0000 7fba9d200700 1 -- 172.21.15.18:0/1862678422 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fba98115220 con 0x7fba98070890 2024-04-12T20:52:33.862 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:33.863+0000 7fba8e000700 1 -- 172.21.15.18:0/1862678422 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fba8801c630 con 0x7fba98070890 2024-04-12T20:52:33.862 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:33.863+0000 7fba9d200700 1 -- 172.21.15.18:0/1862678422 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fba981157e0 con 0x7fba98070890 2024-04-12T20:52:33.862 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:33.864+0000 7fba9d200700 1 -- 172.21.15.18:0/1862678422 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fba9804c710 con 0x7fba98070890 2024-04-12T20:52:33.863 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:33.865+0000 7fba8e000700 1 -- 172.21.15.18:0/1862678422 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7fba8800bbf0 con 0x7fba98070890 2024-04-12T20:52:33.865 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:33.867+0000 7fba8e000700 1 --2- 172.21.15.18:0/1862678422 >> [v2:172.21.15.18:6800/3048894687,v1:172.21.15.18:6801/3048894687] conn(0x7fba7c0692a0 0x7fba7c06b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T20:52:33.865 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:33.867+0000 7fba8e000700 1 -- 172.21.15.18:0/1862678422 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5550+0+0 (secure 0 0 0) 0x7fba88088680 con 0x7fba98070890 2024-04-12T20:52:33.865 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:33.868+0000 7fba95400700 1 --2- 172.21.15.18:0/1862678422 >> [v2:172.21.15.18:6800/3048894687,v1:172.21.15.18:6801/3048894687] conn(0x7fba7c0692a0 0x7fba7c06b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T20:52:33.866 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:33.868+0000 7fba95400700 1 --2- 172.21.15.18:0/1862678422 >> [v2:172.21.15.18:6800/3048894687,v1:172.21.15.18:6801/3048894687] conn(0x7fba7c0692a0 0x7fba7c06b750 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fba84009fd0 tx=0x7fba8400b040 comp rx=0 tx=0).ready entity=mgr.14212 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T20:52:33.869 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:33.871+0000 7fba8e000700 1 -- 172.21.15.18:0/1862678422 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7fba88058350 con 0x7fba98070890 2024-04-12T20:52:34.241 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:34.244+0000 7fba9d200700 1 -- 172.21.15.18:0/1862678422 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "mds versions", "format": "json"} v 0) v1 -- 0x7fba98107b20 con 0x7fba98070890 2024-04-12T20:52:34.242 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:34.244+0000 7fba8e000700 1 -- 172.21.15.18:0/1862678422 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "mds versions", "format": "json"}]=0 v19) v1 ==== 78+0+99 (secure 0 0 0) 0x7fba88013e60 con 0x7fba98070890 2024-04-12T20:52:34.246 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:34.248+0000 7fba9d200700 1 -- 172.21.15.18:0/1862678422 >> [v2:172.21.15.18:6800/3048894687,v1:172.21.15.18:6801/3048894687] conn(0x7fba7c0692a0 msgr2=0x7fba7c06b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:52:34.246 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:34.248+0000 7fba9d200700 1 --2- 172.21.15.18:0/1862678422 >> [v2:172.21.15.18:6800/3048894687,v1:172.21.15.18:6801/3048894687] conn(0x7fba7c0692a0 0x7fba7c06b750 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fba84009fd0 tx=0x7fba8400b040 comp rx=0 tx=0).stop 2024-04-12T20:52:34.246 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:34.248+0000 7fba9d200700 1 -- 172.21.15.18:0/1862678422 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fba98070890 msgr2=0x7fba9810fb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:52:34.246 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:34.248+0000 7fba9d200700 1 --2- 172.21.15.18:0/1862678422 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fba98070890 0x7fba9810fb70 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fba8800b970 tx=0x7fba88009e30 comp rx=0 tx=0).stop 2024-04-12T20:52:34.246 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:34.248+0000 7fba9d200700 1 -- 172.21.15.18:0/1862678422 shutdown_connections 2024-04-12T20:52:34.246 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:34.248+0000 7fba9d200700 1 --2- 172.21.15.18:0/1862678422 >> [v2:172.21.15.18:6800/3048894687,v1:172.21.15.18:6801/3048894687] conn(0x7fba7c0692a0 0x7fba7c06b750 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:52:34.247 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:34.248+0000 7fba9d200700 1 --2- 172.21.15.18:0/1862678422 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fba98070890 0x7fba9810fb70 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:52:34.247 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:34.248+0000 7fba9d200700 1 --2- 172.21.15.18:0/1862678422 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fba98071e40 0x7fba981100b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:52:34.247 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:34.248+0000 7fba9d200700 1 -- 172.21.15.18:0/1862678422 >> 172.21.15.18:0/1862678422 conn(0x7fba980f8580 msgr2=0x7fba980f9c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T20:52:34.247 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:34.248+0000 7fba9d200700 1 -- 172.21.15.18:0/1862678422 shutdown_connections 2024-04-12T20:52:34.247 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:34.249+0000 7fba9d200700 1 -- 172.21.15.18:0/1862678422 wait complete. 2024-04-12T20:52:34.264 INFO:teuthology.orchestra.run.smithi018.stdout:true 2024-04-12T20:52:34.531 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:52:34 smithi186 ceph-mon[36597]: pgmap v238: 65 pgs: 65 active+clean; 580 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.7 KiB/s rd, 0 B/s wr, 2 op/s 2024-04-12T20:52:34.550 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:52:34 smithi018 ceph-mon[28799]: pgmap v238: 65 pgs: 65 active+clean; 580 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.7 KiB/s rd, 0 B/s wr, 2 op/s 2024-04-12T20:52:35.352 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:52:35 smithi018 ceph-mon[28799]: from='client.? 172.21.15.18:0/1862678422' entity='client.admin' cmd=[{"prefix": "mds versions", "format": "json"}]: dispatch 2024-04-12T20:52:35.455 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:52:35 smithi186 ceph-mon[36597]: from='client.? 172.21.15.18:0/1862678422' entity='client.admin' cmd=[{"prefix": "mds versions", "format": "json"}]: dispatch 2024-04-12T20:52:35.466 INFO:teuthology.run_tasks:Running task fs.pre_upgrade_save... 2024-04-12T20:52:35.480 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-12T20:52:35.699 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:35.701+0000 7fc84d400640 1 -- 172.21.15.18:0/239110667 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc848072660 msgr2=0x7fc848073050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:52:35.699 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:35.701+0000 7fc84d400640 1 --2- 172.21.15.18:0/239110667 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc848072660 0x7fc848073050 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fc8300048d0 tx=0x7fc830009da0 comp rx=0 tx=0).stop 2024-04-12T20:52:35.699 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:35.701+0000 7fc84d400640 1 -- 172.21.15.18:0/239110667 shutdown_connections 2024-04-12T20:52:35.699 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:35.701+0000 7fc84d400640 1 --2- 172.21.15.18:0/239110667 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc848072660 0x7fc848073050 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:52:35.699 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:35.701+0000 7fc84d400640 1 --2- 172.21.15.18:0/239110667 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc848071d10 0x7fc848072120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:52:35.699 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:35.701+0000 7fc84d400640 1 -- 172.21.15.18:0/239110667 >> 172.21.15.18:0/239110667 conn(0x7fc8480f9c80 msgr2=0x7fc8480fc060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T20:52:35.699 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:35.701+0000 7fc84d400640 1 -- 172.21.15.18:0/239110667 shutdown_connections 2024-04-12T20:52:35.699 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:35.701+0000 7fc84d400640 1 -- 172.21.15.18:0/239110667 wait complete. 2024-04-12T20:52:35.701 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:35.702+0000 7fc84d400640 1 Processor -- start 2024-04-12T20:52:35.701 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:35.702+0000 7fc84d400640 1 -- start start 2024-04-12T20:52:35.701 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:35.702+0000 7fc84d400640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc848071d10 0x7fc8481132e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T20:52:35.701 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:35.702+0000 7fc84d400640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc848072660 0x7fc848113820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T20:52:35.701 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:35.703+0000 7fc84d400640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc848113e70 con 0x7fc848071d10 2024-04-12T20:52:35.701 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:35.703+0000 7fc84d400640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc848118870 con 0x7fc848072660 2024-04-12T20:52:35.701 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:35.703+0000 7fc846800640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc848071d10 0x7fc8481132e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T20:52:35.701 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:35.703+0000 7fc846800640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc848071d10 0x7fc8481132e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:35560/0 (socket says 172.21.15.18:35560) 2024-04-12T20:52:35.701 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:35.703+0000 7fc846800640 1 -- 172.21.15.18:0/637303810 learned_addr learned my addr 172.21.15.18:0/637303810 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T20:52:35.701 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:35.703+0000 7fc83fe00640 1 --2- 172.21.15.18:0/637303810 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc848072660 0x7fc848113820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T20:52:35.701 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:35.703+0000 7fc846800640 1 -- 172.21.15.18:0/637303810 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc848072660 msgr2=0x7fc848113820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:52:35.701 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:35.703+0000 7fc846800640 1 --2- 172.21.15.18:0/637303810 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc848072660 0x7fc848113820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:52:35.701 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:35.703+0000 7fc846800640 1 -- 172.21.15.18:0/637303810 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc830005040 con 0x7fc848071d10 2024-04-12T20:52:35.702 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:35.704+0000 7fc846800640 1 --2- 172.21.15.18:0/637303810 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc848071d10 0x7fc8481132e0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fc838002a80 tx=0x7fc838004570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T20:52:35.702 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:35.704+0000 7fc83ea00640 1 -- 172.21.15.18:0/637303810 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc838007c30 con 0x7fc848071d10 2024-04-12T20:52:35.702 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:35.704+0000 7fc83ea00640 1 -- 172.21.15.18:0/637303810 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc83800f040 con 0x7fc848071d10 2024-04-12T20:52:35.702 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:35.704+0000 7fc84d400640 1 -- 172.21.15.18:0/637303810 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc848118a10 con 0x7fc848071d10 2024-04-12T20:52:35.703 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:35.704+0000 7fc83ea00640 1 -- 172.21.15.18:0/637303810 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc83800dba0 con 0x7fc848071d10 2024-04-12T20:52:35.703 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:35.704+0000 7fc84d400640 1 -- 172.21.15.18:0/637303810 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc848118f00 con 0x7fc848071d10 2024-04-12T20:52:35.703 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:35.705+0000 7fc84d400640 1 -- 172.21.15.18:0/637303810 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc84804bcc0 con 0x7fc848071d10 2024-04-12T20:52:35.705 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:35.707+0000 7fc83ea00640 1 -- 172.21.15.18:0/637303810 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7fc83800dd00 con 0x7fc848071d10 2024-04-12T20:52:35.706 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:35.708+0000 7fc83ea00640 1 --2- 172.21.15.18:0/637303810 >> [v2:172.21.15.18:6800/3048894687,v1:172.21.15.18:6801/3048894687] conn(0x7fc82c069210 0x7fc82c06b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T20:52:35.706 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:35.708+0000 7fc83ea00640 1 -- 172.21.15.18:0/637303810 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5550+0+0 (secure 0 0 0) 0x7fc8380871e0 con 0x7fc848071d10 2024-04-12T20:52:35.706 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:35.708+0000 7fc83fe00640 1 --2- 172.21.15.18:0/637303810 >> [v2:172.21.15.18:6800/3048894687,v1:172.21.15.18:6801/3048894687] conn(0x7fc82c069210 0x7fc82c06b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T20:52:35.707 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:35.709+0000 7fc83fe00640 1 --2- 172.21.15.18:0/637303810 >> [v2:172.21.15.18:6800/3048894687,v1:172.21.15.18:6801/3048894687] conn(0x7fc82c069210 0x7fc82c06b6c0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fc83002bda0 tx=0x7fc830009da0 comp rx=0 tx=0).ready entity=mgr.14212 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T20:52:35.710 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:35.713+0000 7fc83ea00640 1 -- 172.21.15.18:0/637303810 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7fc8380533a0 con 0x7fc848071d10 2024-04-12T20:52:35.920 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:35.923+0000 7fc84d400640 1 -- 172.21.15.18:0/637303810 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc8481193c0 con 0x7fc848071d10 2024-04-12T20:52:35.922 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:35.924+0000 7fc83ea00640 1 -- 172.21.15.18:0/637303810 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v46) v1 ==== 74+0+14389 (secure 0 0 0) 0x7fc8380569c0 con 0x7fc848071d10 2024-04-12T20:52:35.922 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-12T20:52:35.922 INFO:teuthology.orchestra.run.smithi018.stdout:{"epoch":46,"fsid":"1cbe407a-f90d-11ee-bc8d-c7b262605968","created":"2024-04-12T20:43:00.692361+0000","modified":"2024-04-12T20:52:14.968750+0000","last_up_change":"2024-04-12T20:50:17.011011+0000","last_in_change":"2024-04-12T20:49:54.190644+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":14,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-12T20:48:37.529081+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"cephfs.cephfs.meta","create_time":"2024-04-12T20:51:56.284414+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"44","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":3,"pool_name":"cephfs.cephfs.data","create_time":"2024-04-12T20:51:56.927802+0000","flags":131073,"flags_names":"hashpspool,bulk","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"45","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"2af887cc-0447-42aa-a8f3-1aa7fcd4dc8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6802","nonce":585613146},{"type":"v1","addr":"172.21.15.18:6803","nonce":585613146}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6804","nonce":585613146},{"type":"v1","addr":"172.21.15.18:6805","nonce":585613146}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6808","nonce":585613146},{"type":"v1","addr":"172.21.15.18:6809","nonce":585613146}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6806","nonce":585613146},{"type":"v1","addr":"172.21.15.18:6807","nonce":585613146}]},"public_addr":"172.21.15.18:6803/585613146","cluster_addr":"172.21.15.18:6805/585613146","heartbeat_back_addr":"172.21.15.18:6809/585613146","heartbeat_front_addr":"172.21.15.18:6807/585613146","state":["exists","up"]},{"osd":1,"uuid":"6ba7b0d3-a03d-45cc-b4e9-89a92c8c6ca8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6810","nonce":181951035},{"type":"v1","addr":"172.21.15.18:6811","nonce":181951035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6812","nonce":181951035},{"type":"v1","addr":"172.21.15.18:6813","nonce":181951035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6816","nonce":181951035},{"type":"v1","addr":"172.21.15.18:6817","nonce":181951035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6814","nonce":181951035},{"type":"v1","addr":"172.21.15.18:6815","nonce":181951035}]},"public_addr":"172.21.15.18:6811/181951035","cluster_addr":"172.21.15.18:6813/181951035","heartbeat_back_addr":"172.21.15.18:6817/181951035","heartbeat_front_addr":"172.21.15.18:6815/181951035","state":["exists","up"]},{"osd":2,"uuid":"5bb1137d-5a3c-40bc-ae08-fc792b8ee547","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6818","nonce":3573247809},{"type":"v1","addr":"172.21.15.18:6819","nonce":3573247809}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6820","nonce":3573247809},{"type":"v1","addr":"172.21.15.18:6821","nonce":3573247809}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6824","nonce":3573247809},{"type":"v1","addr":"172.21.15.18:6825","nonce":3573247809}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6822","nonce":3573247809},{"type":"v1","addr":"172.21.15.18:6823","nonce":3573247809}]},"public_addr":"172.21.15.18:6819/3573247809","cluster_addr":"172.21.15.18:6821/3573247809","heartbeat_back_addr":"172.21.15.18:6825/3573247809","heartbeat_front_addr":"172.21.15.18:6823/3573247809","state":["exists","up"]},{"osd":3,"uuid":"a5140a33-1e90-4980-aa9a-b0afe6b155f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6800","nonce":2184799172},{"type":"v1","addr":"172.21.15.186:6801","nonce":2184799172}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6802","nonce":2184799172},{"type":"v1","addr":"172.21.15.186:6803","nonce":2184799172}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6806","nonce":2184799172},{"type":"v1","addr":"172.21.15.186:6807","nonce":2184799172}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6804","nonce":2184799172},{"type":"v1","addr":"172.21.15.186:6805","nonce":2184799172}]},"public_addr":"172.21.15.186:6801/2184799172","cluster_addr":"172.21.15.186:6803/2184799172","heartbeat_back_addr":"172.21.15.186:6807/2184799172","heartbeat_front_addr":"172.21.15.186:6805/2184799172","state":["exists","up"]},{"osd":4,"uuid":"d6b2c6b3-6c78-4261-bcbe-c33358407198","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6808","nonce":3584843287},{"type":"v1","addr":"172.21.15.186:6809","nonce":3584843287}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6810","nonce":3584843287},{"type":"v1","addr":"172.21.15.186:6811","nonce":3584843287}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6814","nonce":3584843287},{"type":"v1","addr":"172.21.15.186:6815","nonce":3584843287}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6812","nonce":3584843287},{"type":"v1","addr":"172.21.15.186:6813","nonce":3584843287}]},"public_addr":"172.21.15.186:6809/3584843287","cluster_addr":"172.21.15.186:6811/3584843287","heartbeat_back_addr":"172.21.15.186:6815/3584843287","heartbeat_front_addr":"172.21.15.186:6813/3584843287","state":["exists","up"]},{"osd":5,"uuid":"7b5c9f17-6306-4fb5-aa77-fc9a73bcc663","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6816","nonce":3543055150},{"type":"v1","addr":"172.21.15.186:6817","nonce":3543055150}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6818","nonce":3543055150},{"type":"v1","addr":"172.21.15.186:6819","nonce":3543055150}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6822","nonce":3543055150},{"type":"v1","addr":"172.21.15.186:6823","nonce":3543055150}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6820","nonce":3543055150},{"type":"v1","addr":"172.21.15.186:6821","nonce":3543055150}]},"public_addr":"172.21.15.186:6817/3543055150","cluster_addr":"172.21.15.186:6819/3543055150","heartbeat_back_addr":"172.21.15.186:6823/3543055150","heartbeat_front_addr":"172.21.15.186:6821/3543055150","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-04-12T20:47:21.231060+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-04-12T20:47:57.699205+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-04-12T20:48:35.096648+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-04-12T20:49:08.252810+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-04-12T20:49:41.102187+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-04-12T20:50:16.009813+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.186:6825/1645616336":"2024-04-13T20:52:14.968672+0000","172.21.15.18:6801/1312802974":"2024-04-13T20:43:34.269559+0000","172.21.15.18:6800/1312802974":"2024-04-13T20:43:34.269559+0000","172.21.15.18:0/3295870133":"2024-04-13T20:43:34.269559+0000","172.21.15.18:0/2517252399":"2024-04-13T20:43:34.269559+0000","172.21.15.18:0/766256564":"2024-04-13T20:45:44.776072+0000","172.21.15.18:6800/3403971131":"2024-04-13T20:44:15.143139+0000","172.21.15.18:0/1732042827":"2024-04-13T20:43:34.269559+0000","172.21.15.18:0/2945432483":"2024-04-13T20:44:15.143139+0000","172.21.15.18:6801/3403971131":"2024-04-13T20:44:15.143139+0000","172.21.15.18:0/3387911038":"2024-04-13T20:44:15.143139+0000","172.21.15.18:0/4018777476":"2024-04-13T20:45:44.776072+0000","172.21.15.186:6824/1645616336":"2024-04-13T20:52:14.968672+0000","172.21.15.18:6800/1391635591":"2024-04-13T20:45:44.776072+0000","172.21.15.18:6801/1391635591":"2024-04-13T20:45:44.776072+0000","172.21.15.18:0/3896210349":"2024-04-13T20:44:15.143139+0000","172.21.15.18:0/1720919549":"2024-04-13T20:45:44.776072+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-12T20:52:35.924 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:35.926+0000 7fc84d400640 1 -- 172.21.15.18:0/637303810 >> [v2:172.21.15.18:6800/3048894687,v1:172.21.15.18:6801/3048894687] conn(0x7fc82c069210 msgr2=0x7fc82c06b6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:52:35.925 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:35.926+0000 7fc84d400640 1 --2- 172.21.15.18:0/637303810 >> [v2:172.21.15.18:6800/3048894687,v1:172.21.15.18:6801/3048894687] conn(0x7fc82c069210 0x7fc82c06b6c0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fc83002bda0 tx=0x7fc830009da0 comp rx=0 tx=0).stop 2024-04-12T20:52:35.925 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:35.926+0000 7fc84d400640 1 -- 172.21.15.18:0/637303810 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc848071d10 msgr2=0x7fc8481132e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:52:35.925 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:35.926+0000 7fc84d400640 1 --2- 172.21.15.18:0/637303810 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc848071d10 0x7fc8481132e0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fc838002a80 tx=0x7fc838004570 comp rx=0 tx=0).stop 2024-04-12T20:52:35.925 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:35.927+0000 7fc84d400640 1 -- 172.21.15.18:0/637303810 shutdown_connections 2024-04-12T20:52:35.925 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:35.927+0000 7fc84d400640 1 --2- 172.21.15.18:0/637303810 >> [v2:172.21.15.18:6800/3048894687,v1:172.21.15.18:6801/3048894687] conn(0x7fc82c069210 0x7fc82c06b6c0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:52:35.925 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:35.927+0000 7fc84d400640 1 --2- 172.21.15.18:0/637303810 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc848072660 0x7fc848113820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:52:35.925 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:35.927+0000 7fc84d400640 1 --2- 172.21.15.18:0/637303810 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc848071d10 0x7fc8481132e0 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:52:35.925 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:35.927+0000 7fc84d400640 1 -- 172.21.15.18:0/637303810 >> 172.21.15.18:0/637303810 conn(0x7fc8480f9c80 msgr2=0x7fc8480fb2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T20:52:35.925 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:35.927+0000 7fc84d400640 1 -- 172.21.15.18:0/637303810 shutdown_connections 2024-04-12T20:52:35.925 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:35.927+0000 7fc84d400640 1 -- 172.21.15.18:0/637303810 wait complete. 2024-04-12T20:52:35.943 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-12T20:48:37.529081+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 2, 'pool_name': 'cephfs.cephfs.meta', 'create_time': '2024-04-12T20:51:56.284414+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '44', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 3, 'pool_name': 'cephfs.cephfs.data', 'create_time': '2024-04-12T20:51:56.927802+0000', 'flags': 131073, 'flags_names': 'hashpspool,bulk', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '45', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-12T20:52:35.943 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-12T20:52:36.158 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:36.160+0000 7faa6f200640 1 -- 172.21.15.18:0/958517336 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7faa70065ce0 msgr2=0x7faa700fec10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:52:36.159 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:36.160+0000 7faa6f200640 1 --2- 172.21.15.18:0/958517336 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7faa70065ce0 0x7faa700fec10 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7faa600048d0 tx=0x7faa60009da0 comp rx=0 tx=0).stop 2024-04-12T20:52:36.159 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:36.161+0000 7faa6f200640 1 -- 172.21.15.18:0/958517336 shutdown_connections 2024-04-12T20:52:36.159 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:36.161+0000 7faa6f200640 1 --2- 172.21.15.18:0/958517336 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7faa700ff150 0x7faa70101530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:52:36.159 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:36.161+0000 7faa6f200640 1 --2- 172.21.15.18:0/958517336 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7faa70065ce0 0x7faa700fec10 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:52:36.159 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:36.161+0000 7faa6f200640 1 -- 172.21.15.18:0/958517336 >> 172.21.15.18:0/958517336 conn(0x7faa700f66e0 msgr2=0x7faa700f8af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T20:52:36.159 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:36.161+0000 7faa6f200640 1 -- 172.21.15.18:0/958517336 shutdown_connections 2024-04-12T20:52:36.159 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:36.161+0000 7faa6f200640 1 -- 172.21.15.18:0/958517336 wait complete. 2024-04-12T20:52:36.160 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:36.162+0000 7faa6f200640 1 Processor -- start 2024-04-12T20:52:36.160 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:36.162+0000 7faa6f200640 1 -- start start 2024-04-12T20:52:36.160 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:36.162+0000 7faa6f200640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7faa70065ce0 0x7faa7010cb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T20:52:36.160 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:36.162+0000 7faa6f200640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7faa700ff150 0x7faa7010d080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T20:52:36.160 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:36.162+0000 7faa6f200640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa7010d6d0 con 0x7faa70065ce0 2024-04-12T20:52:36.161 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:36.162+0000 7faa6f200640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa70112080 con 0x7faa700ff150 2024-04-12T20:52:36.161 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:36.163+0000 7faa6de00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7faa70065ce0 0x7faa7010cb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T20:52:36.161 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:36.163+0000 7faa6de00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7faa70065ce0 0x7faa7010cb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:35578/0 (socket says 172.21.15.18:35578) 2024-04-12T20:52:36.161 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:36.163+0000 7faa6de00640 1 -- 172.21.15.18:0/1593861509 learned_addr learned my addr 172.21.15.18:0/1593861509 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T20:52:36.162 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:36.163+0000 7faa6d400640 1 --2- 172.21.15.18:0/1593861509 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7faa700ff150 0x7faa7010d080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T20:52:36.162 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:36.163+0000 7faa6de00640 1 -- 172.21.15.18:0/1593861509 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7faa700ff150 msgr2=0x7faa7010d080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:52:36.162 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:36.163+0000 7faa6de00640 1 --2- 172.21.15.18:0/1593861509 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7faa700ff150 0x7faa7010d080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:52:36.162 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:36.163+0000 7faa6de00640 1 -- 172.21.15.18:0/1593861509 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa60005040 con 0x7faa70065ce0 2024-04-12T20:52:36.162 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:36.163+0000 7faa6de00640 1 --2- 172.21.15.18:0/1593861509 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7faa70065ce0 0x7faa7010cb40 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7faa6002bda0 tx=0x7faa60009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T20:52:36.162 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:36.164+0000 7faa66000640 1 -- 172.21.15.18:0/1593861509 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7faa60009590 con 0x7faa70065ce0 2024-04-12T20:52:36.162 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:36.164+0000 7faa66000640 1 -- 172.21.15.18:0/1593861509 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7faa600096f0 con 0x7faa70065ce0 2024-04-12T20:52:36.162 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:36.164+0000 7faa66000640 1 -- 172.21.15.18:0/1593861509 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7faa6003b640 con 0x7faa70065ce0 2024-04-12T20:52:36.162 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:36.164+0000 7faa6f200640 1 -- 172.21.15.18:0/1593861509 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faa701121c0 con 0x7faa70065ce0 2024-04-12T20:52:36.163 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:36.164+0000 7faa6f200640 1 -- 172.21.15.18:0/1593861509 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faa701126b0 con 0x7faa70065ce0 2024-04-12T20:52:36.163 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:36.165+0000 7faa6f200640 1 -- 172.21.15.18:0/1593861509 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faa7004bcc0 con 0x7faa70065ce0 2024-04-12T20:52:36.163 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:36.166+0000 7faa66000640 1 -- 172.21.15.18:0/1593861509 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7faa6002f070 con 0x7faa70065ce0 2024-04-12T20:52:36.165 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:36.167+0000 7faa66000640 1 --2- 172.21.15.18:0/1593861509 >> [v2:172.21.15.18:6800/3048894687,v1:172.21.15.18:6801/3048894687] conn(0x7faa54069210 0x7faa5406b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T20:52:36.165 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:36.167+0000 7faa66000640 1 -- 172.21.15.18:0/1593861509 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5550+0+0 (secure 0 0 0) 0x7faa600a9a50 con 0x7faa70065ce0 2024-04-12T20:52:36.165 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:36.168+0000 7faa6d400640 1 --2- 172.21.15.18:0/1593861509 >> [v2:172.21.15.18:6800/3048894687,v1:172.21.15.18:6801/3048894687] conn(0x7faa54069210 0x7faa5406b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T20:52:36.166 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:36.168+0000 7faa6d400640 1 --2- 172.21.15.18:0/1593861509 >> [v2:172.21.15.18:6800/3048894687,v1:172.21.15.18:6801/3048894687] conn(0x7faa54069210 0x7faa5406b6c0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7faa7010dff0 tx=0x7faa5c009040 comp rx=0 tx=0).ready entity=mgr.14212 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T20:52:36.170 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:36.172+0000 7faa66000640 1 -- 172.21.15.18:0/1593861509 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7faa60075c10 con 0x7faa70065ce0 2024-04-12T20:52:36.352 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:52:36 smithi018 ceph-mon[28799]: pgmap v239: 65 pgs: 65 active+clean; 580 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.5 KiB/s rd, 0 B/s wr, 1 op/s 2024-04-12T20:52:36.352 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:52:36 smithi018 ceph-mon[28799]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' 2024-04-12T20:52:36.353 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:52:36 smithi018 ceph-mon[28799]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' 2024-04-12T20:52:36.353 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:52:36 smithi018 ceph-mon[28799]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-12T20:52:36.353 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:52:36 smithi018 ceph-mon[28799]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-12T20:52:36.353 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:52:36 smithi018 ceph-mon[28799]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' 2024-04-12T20:52:36.353 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:52:36 smithi018 ceph-mon[28799]: from='client.? 172.21.15.18:0/637303810' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-12T20:52:36.362 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:36.365+0000 7faa6f200640 1 -- 172.21.15.18:0/1593861509 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7faa70112840 con 0x7faa70065ce0 2024-04-12T20:52:36.363 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:36.366+0000 7faa66000640 1 -- 172.21.15.18:0/1593861509 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v46) v1 ==== 93+0+10 (secure 0 0 0) 0x7faa60079230 con 0x7faa70065ce0 2024-04-12T20:52:36.363 INFO:teuthology.orchestra.run.smithi018.stdout:pg_num: 1 2024-04-12T20:52:36.366 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:36.368+0000 7faa6f200640 1 -- 172.21.15.18:0/1593861509 >> [v2:172.21.15.18:6800/3048894687,v1:172.21.15.18:6801/3048894687] conn(0x7faa54069210 msgr2=0x7faa5406b6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:52:36.366 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:36.368+0000 7faa6f200640 1 --2- 172.21.15.18:0/1593861509 >> [v2:172.21.15.18:6800/3048894687,v1:172.21.15.18:6801/3048894687] conn(0x7faa54069210 0x7faa5406b6c0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7faa7010dff0 tx=0x7faa5c009040 comp rx=0 tx=0).stop 2024-04-12T20:52:36.366 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:36.368+0000 7faa6f200640 1 -- 172.21.15.18:0/1593861509 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7faa70065ce0 msgr2=0x7faa7010cb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:52:36.367 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:36.368+0000 7faa6f200640 1 --2- 172.21.15.18:0/1593861509 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7faa70065ce0 0x7faa7010cb40 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7faa6002bda0 tx=0x7faa60009da0 comp rx=0 tx=0).stop 2024-04-12T20:52:36.367 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:36.368+0000 7faa6f200640 1 -- 172.21.15.18:0/1593861509 shutdown_connections 2024-04-12T20:52:36.367 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:36.368+0000 7faa6f200640 1 --2- 172.21.15.18:0/1593861509 >> [v2:172.21.15.18:6800/3048894687,v1:172.21.15.18:6801/3048894687] conn(0x7faa54069210 0x7faa5406b6c0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:52:36.367 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:36.368+0000 7faa6f200640 1 --2- 172.21.15.18:0/1593861509 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7faa700ff150 0x7faa7010d080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:52:36.367 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:36.368+0000 7faa6f200640 1 --2- 172.21.15.18:0/1593861509 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7faa70065ce0 0x7faa7010cb40 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:52:36.367 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:36.368+0000 7faa6f200640 1 -- 172.21.15.18:0/1593861509 >> 172.21.15.18:0/1593861509 conn(0x7faa700f66e0 msgr2=0x7faa700f6ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T20:52:36.367 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:36.369+0000 7faa6f200640 1 -- 172.21.15.18:0/1593861509 shutdown_connections 2024-04-12T20:52:36.367 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:36.369+0000 7faa6f200640 1 -- 172.21.15.18:0/1593861509 wait complete. 2024-04-12T20:52:36.385 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs.cephfs.meta pg_num 2024-04-12T20:52:36.531 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:52:36 smithi186 ceph-mon[36597]: pgmap v239: 65 pgs: 65 active+clean; 580 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.5 KiB/s rd, 0 B/s wr, 1 op/s 2024-04-12T20:52:36.531 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:52:36 smithi186 ceph-mon[36597]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' 2024-04-12T20:52:36.531 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:52:36 smithi186 ceph-mon[36597]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' 2024-04-12T20:52:36.531 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:52:36 smithi186 ceph-mon[36597]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-12T20:52:36.532 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:52:36 smithi186 ceph-mon[36597]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-12T20:52:36.532 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:52:36 smithi186 ceph-mon[36597]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' 2024-04-12T20:52:36.532 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:52:36 smithi186 ceph-mon[36597]: from='client.? 172.21.15.18:0/637303810' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-12T20:52:36.605 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:36.607+0000 7fdf6e400640 1 -- 172.21.15.18:0/1159415580 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fdf680ff450 msgr2=0x7fdf680ff8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:52:36.605 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:36.607+0000 7fdf6e400640 1 --2- 172.21.15.18:0/1159415580 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fdf680ff450 0x7fdf680ff8c0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fdf540048d0 tx=0x7fdf54009da0 comp rx=0 tx=0).stop 2024-04-12T20:52:36.605 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:36.607+0000 7fdf6e400640 1 -- 172.21.15.18:0/1159415580 shutdown_connections 2024-04-12T20:52:36.605 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:36.607+0000 7fdf6e400640 1 --2- 172.21.15.18:0/1159415580 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fdf680ff450 0x7fdf680ff8c0 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:52:36.605 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:36.607+0000 7fdf6e400640 1 --2- 172.21.15.18:0/1159415580 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdf680fe220 0x7fdf680fe630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:52:36.605 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:36.607+0000 7fdf6e400640 1 -- 172.21.15.18:0/1159415580 >> 172.21.15.18:0/1159415580 conn(0x7fdf680f9ae0 msgr2=0x7fdf680fbec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T20:52:36.605 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:36.607+0000 7fdf6e400640 1 -- 172.21.15.18:0/1159415580 shutdown_connections 2024-04-12T20:52:36.605 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:36.607+0000 7fdf6e400640 1 -- 172.21.15.18:0/1159415580 wait complete. 2024-04-12T20:52:36.607 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:36.608+0000 7fdf6e400640 1 Processor -- start 2024-04-12T20:52:36.607 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:36.608+0000 7fdf6e400640 1 -- start start 2024-04-12T20:52:36.607 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:36.608+0000 7fdf6e400640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fdf680fe220 0x7fdf6810ed30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T20:52:36.607 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:36.608+0000 7fdf6e400640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdf680ff450 0x7fdf6810f270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T20:52:36.607 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:36.608+0000 7fdf6e400640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf6810f8c0 con 0x7fdf680fe220 2024-04-12T20:52:36.607 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:36.608+0000 7fdf6e400640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf68114280 con 0x7fdf680ff450 2024-04-12T20:52:36.608 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:36.608+0000 7fdf6d000640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fdf680fe220 0x7fdf6810ed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T20:52:36.608 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:36.608+0000 7fdf6d000640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fdf680fe220 0x7fdf6810ed30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:35596/0 (socket says 172.21.15.18:35596) 2024-04-12T20:52:36.608 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:36.608+0000 7fdf6d000640 1 -- 172.21.15.18:0/2899552598 learned_addr learned my addr 172.21.15.18:0/2899552598 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T20:52:36.608 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:36.609+0000 7fdf6d000640 1 -- 172.21.15.18:0/2899552598 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdf680ff450 msgr2=0x7fdf6810f270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:52:36.608 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:36.609+0000 7fdf6d000640 1 --2- 172.21.15.18:0/2899552598 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdf680ff450 0x7fdf6810f270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:52:36.608 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:36.609+0000 7fdf6d000640 1 -- 172.21.15.18:0/2899552598 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf54005040 con 0x7fdf680fe220 2024-04-12T20:52:36.608 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:36.609+0000 7fdf6d000640 1 --2- 172.21.15.18:0/2899552598 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fdf680fe220 0x7fdf6810ed30 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fdf5c002840 tx=0x7fdf5c004570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T20:52:36.608 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:36.609+0000 7fdf64c00640 1 -- 172.21.15.18:0/2899552598 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdf5c007c30 con 0x7fdf680fe220 2024-04-12T20:52:36.608 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:36.609+0000 7fdf64c00640 1 -- 172.21.15.18:0/2899552598 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fdf5c00f040 con 0x7fdf680fe220 2024-04-12T20:52:36.608 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:36.609+0000 7fdf6e400640 1 -- 172.21.15.18:0/2899552598 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdf68114420 con 0x7fdf680fe220 2024-04-12T20:52:36.608 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:36.610+0000 7fdf64c00640 1 -- 172.21.15.18:0/2899552598 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdf5c009590 con 0x7fdf680fe220 2024-04-12T20:52:36.608 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:36.610+0000 7fdf6e400640 1 -- 172.21.15.18:0/2899552598 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdf681148f0 con 0x7fdf680fe220 2024-04-12T20:52:36.609 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:36.611+0000 7fdf6e400640 1 -- 172.21.15.18:0/2899552598 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdf6804bcc0 con 0x7fdf680fe220 2024-04-12T20:52:36.609 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:36.612+0000 7fdf64c00640 1 -- 172.21.15.18:0/2899552598 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7fdf5c00d6b0 con 0x7fdf680fe220 2024-04-12T20:52:36.611 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:36.613+0000 7fdf64c00640 1 --2- 172.21.15.18:0/2899552598 >> [v2:172.21.15.18:6800/3048894687,v1:172.21.15.18:6801/3048894687] conn(0x7fdf50069220 0x7fdf5006b6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T20:52:36.611 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:36.613+0000 7fdf64c00640 1 -- 172.21.15.18:0/2899552598 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5550+0+0 (secure 0 0 0) 0x7fdf5c087130 con 0x7fdf680fe220 2024-04-12T20:52:36.611 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:36.614+0000 7fdf67e00640 1 --2- 172.21.15.18:0/2899552598 >> [v2:172.21.15.18:6800/3048894687,v1:172.21.15.18:6801/3048894687] conn(0x7fdf50069220 0x7fdf5006b6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T20:52:36.612 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:36.614+0000 7fdf67e00640 1 --2- 172.21.15.18:0/2899552598 >> [v2:172.21.15.18:6800/3048894687,v1:172.21.15.18:6801/3048894687] conn(0x7fdf50069220 0x7fdf5006b6d0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fdf54034040 tx=0x7fdf54009da0 comp rx=0 tx=0).ready entity=mgr.14212 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T20:52:36.614 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:36.617+0000 7fdf64c00640 1 -- 172.21.15.18:0/2899552598 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7fdf5c053370 con 0x7fdf680fe220 2024-04-12T20:52:36.807 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:36.810+0000 7fdf6e400640 1 -- 172.21.15.18:0/2899552598 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"} v 0) v1 -- 0x7fdf68114b60 con 0x7fdf680fe220 2024-04-12T20:52:36.808 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:36.810+0000 7fdf64c00640 1 -- 172.21.15.18:0/2899552598 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"}]=0 v46) v1 ==== 107+0+11 (secure 0 0 0) 0x7fdf5c056990 con 0x7fdf680fe220 2024-04-12T20:52:36.808 INFO:teuthology.orchestra.run.smithi018.stdout:pg_num: 32 2024-04-12T20:52:36.811 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:36.813+0000 7fdf6e400640 1 -- 172.21.15.18:0/2899552598 >> [v2:172.21.15.18:6800/3048894687,v1:172.21.15.18:6801/3048894687] conn(0x7fdf50069220 msgr2=0x7fdf5006b6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:52:36.811 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:36.813+0000 7fdf6e400640 1 --2- 172.21.15.18:0/2899552598 >> [v2:172.21.15.18:6800/3048894687,v1:172.21.15.18:6801/3048894687] conn(0x7fdf50069220 0x7fdf5006b6d0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fdf54034040 tx=0x7fdf54009da0 comp rx=0 tx=0).stop 2024-04-12T20:52:36.811 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:36.813+0000 7fdf6e400640 1 -- 172.21.15.18:0/2899552598 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fdf680fe220 msgr2=0x7fdf6810ed30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:52:36.811 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:36.813+0000 7fdf6e400640 1 --2- 172.21.15.18:0/2899552598 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fdf680fe220 0x7fdf6810ed30 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fdf5c002840 tx=0x7fdf5c004570 comp rx=0 tx=0).stop 2024-04-12T20:52:36.811 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:36.813+0000 7fdf6e400640 1 -- 172.21.15.18:0/2899552598 shutdown_connections 2024-04-12T20:52:36.811 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:36.813+0000 7fdf6e400640 1 --2- 172.21.15.18:0/2899552598 >> [v2:172.21.15.18:6800/3048894687,v1:172.21.15.18:6801/3048894687] conn(0x7fdf50069220 0x7fdf5006b6d0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:52:36.811 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:36.813+0000 7fdf6e400640 1 --2- 172.21.15.18:0/2899552598 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdf680ff450 0x7fdf6810f270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:52:36.812 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:36.813+0000 7fdf6e400640 1 --2- 172.21.15.18:0/2899552598 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fdf680fe220 0x7fdf6810ed30 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:52:36.812 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:36.813+0000 7fdf6e400640 1 -- 172.21.15.18:0/2899552598 >> 172.21.15.18:0/2899552598 conn(0x7fdf680f9ae0 msgr2=0x7fdf680fc070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T20:52:36.812 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:36.814+0000 7fdf6e400640 1 -- 172.21.15.18:0/2899552598 shutdown_connections 2024-04-12T20:52:36.812 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:36.814+0000 7fdf6e400640 1 -- 172.21.15.18:0/2899552598 wait complete. 2024-04-12T20:52:36.830 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs.cephfs.data pg_num 2024-04-12T20:52:37.047 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:37.049+0000 7f53efa00640 1 -- 172.21.15.18:0/2302197005 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f53e8071330 msgr2=0x7f53e806f860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:52:37.048 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:37.049+0000 7f53efa00640 1 --2- 172.21.15.18:0/2302197005 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f53e8071330 0x7f53e806f860 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f53dc0048d0 tx=0x7f53dc009da0 comp rx=0 tx=0).stop 2024-04-12T20:52:37.048 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:37.050+0000 7f53efa00640 1 -- 172.21.15.18:0/2302197005 shutdown_connections 2024-04-12T20:52:37.048 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:37.050+0000 7f53efa00640 1 --2- 172.21.15.18:0/2302197005 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f53e806fda0 0x7f53e8070210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:52:37.048 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:37.050+0000 7f53efa00640 1 --2- 172.21.15.18:0/2302197005 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f53e8071330 0x7f53e806f860 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:52:37.048 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:37.050+0000 7f53efa00640 1 -- 172.21.15.18:0/2302197005 >> 172.21.15.18:0/2302197005 conn(0x7f53e80f7960 msgr2=0x7f53e80f9d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T20:52:37.048 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:37.050+0000 7f53efa00640 1 -- 172.21.15.18:0/2302197005 shutdown_connections 2024-04-12T20:52:37.048 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:37.050+0000 7f53efa00640 1 -- 172.21.15.18:0/2302197005 wait complete. 2024-04-12T20:52:37.049 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:37.050+0000 7f53efa00640 1 Processor -- start 2024-04-12T20:52:37.049 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:37.051+0000 7f53efa00640 1 -- start start 2024-04-12T20:52:37.049 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:37.051+0000 7f53efa00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f53e806fda0 0x7f53e810ef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T20:52:37.050 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:37.051+0000 7f53efa00640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f53e8071330 0x7f53e810f4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T20:52:37.050 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:37.051+0000 7f53efa00640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53e810faf0 con 0x7f53e806fda0 2024-04-12T20:52:37.050 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:37.051+0000 7f53efa00640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53e81144a0 con 0x7f53e8071330 2024-04-12T20:52:37.050 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:37.051+0000 7f53ed000640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f53e806fda0 0x7f53e810ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T20:52:37.050 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:37.051+0000 7f53ed000640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f53e806fda0 0x7f53e810ef60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:35608/0 (socket says 172.21.15.18:35608) 2024-04-12T20:52:37.050 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:37.051+0000 7f53ed000640 1 -- 172.21.15.18:0/3023242466 learned_addr learned my addr 172.21.15.18:0/3023242466 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T20:52:37.050 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:37.051+0000 7f53e7e00640 1 --2- 172.21.15.18:0/3023242466 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f53e8071330 0x7f53e810f4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T20:52:37.050 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:37.052+0000 7f53ed000640 1 -- 172.21.15.18:0/3023242466 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f53e8071330 msgr2=0x7f53e810f4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:52:37.050 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:37.052+0000 7f53ed000640 1 --2- 172.21.15.18:0/3023242466 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f53e8071330 0x7f53e810f4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:52:37.050 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:37.052+0000 7f53ed000640 1 -- 172.21.15.18:0/3023242466 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53dc005040 con 0x7f53e806fda0 2024-04-12T20:52:37.050 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:37.052+0000 7f53e7e00640 1 --2- 172.21.15.18:0/3023242466 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f53e8071330 0x7f53e810f4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-12T20:52:37.050 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:37.052+0000 7f53ed000640 1 --2- 172.21.15.18:0/3023242466 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f53e806fda0 0x7f53e810ef60 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f53dc000c00 tx=0x7f53dc009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T20:52:37.051 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:37.052+0000 7f53e4c00640 1 -- 172.21.15.18:0/3023242466 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f53dc0094c0 con 0x7f53e806fda0 2024-04-12T20:52:37.051 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:37.052+0000 7f53e4c00640 1 -- 172.21.15.18:0/3023242466 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f53dc032e90 con 0x7f53e806fda0 2024-04-12T20:52:37.051 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:37.052+0000 7f53efa00640 1 -- 172.21.15.18:0/3023242466 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f53e81145e0 con 0x7f53e806fda0 2024-04-12T20:52:37.051 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:37.052+0000 7f53e4c00640 1 -- 172.21.15.18:0/3023242466 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f53dc0435f0 con 0x7f53e806fda0 2024-04-12T20:52:37.051 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:37.052+0000 7f53efa00640 1 -- 172.21.15.18:0/3023242466 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f53e8114a70 con 0x7f53e806fda0 2024-04-12T20:52:37.051 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:37.053+0000 7f53efa00640 1 -- 172.21.15.18:0/3023242466 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f53e804bcc0 con 0x7f53e806fda0 2024-04-12T20:52:37.052 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:37.054+0000 7f53e4c00640 1 -- 172.21.15.18:0/3023242466 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7f53dc02f070 con 0x7f53e806fda0 2024-04-12T20:52:37.054 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:37.056+0000 7f53e4c00640 1 --2- 172.21.15.18:0/3023242466 >> [v2:172.21.15.18:6800/3048894687,v1:172.21.15.18:6801/3048894687] conn(0x7f53d0069220 0x7f53d006b6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T20:52:37.054 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:37.056+0000 7f53e4c00640 1 -- 172.21.15.18:0/3023242466 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5550+0+0 (secure 0 0 0) 0x7f53dc0a9090 con 0x7f53e806fda0 2024-04-12T20:52:37.054 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:37.056+0000 7f53e7e00640 1 --2- 172.21.15.18:0/3023242466 >> [v2:172.21.15.18:6800/3048894687,v1:172.21.15.18:6801/3048894687] conn(0x7f53d0069220 0x7f53d006b6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T20:52:37.054 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:37.057+0000 7f53e7e00640 1 --2- 172.21.15.18:0/3023242466 >> [v2:172.21.15.18:6800/3048894687,v1:172.21.15.18:6801/3048894687] conn(0x7f53d0069220 0x7f53d006b6d0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f53e8110410 tx=0x7f53d400b040 comp rx=0 tx=0).ready entity=mgr.14212 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T20:52:37.057 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:37.060+0000 7f53e4c00640 1 -- 172.21.15.18:0/3023242466 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7f53dc0752f0 con 0x7f53e806fda0 2024-04-12T20:52:37.283 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:37.285+0000 7f53efa00640 1 -- 172.21.15.18:0/3023242466 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"} v 0) v1 -- 0x7f53e8114ce0 con 0x7f53e806fda0 2024-04-12T20:52:37.284 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:37.286+0000 7f53e4c00640 1 -- 172.21.15.18:0/3023242466 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"}]=0 v46) v1 ==== 107+0+11 (secure 0 0 0) 0x7f53dc078910 con 0x7f53e806fda0 2024-04-12T20:52:37.284 INFO:teuthology.orchestra.run.smithi018.stdout:pg_num: 32 2024-04-12T20:52:37.287 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:37.288+0000 7f53efa00640 1 -- 172.21.15.18:0/3023242466 >> [v2:172.21.15.18:6800/3048894687,v1:172.21.15.18:6801/3048894687] conn(0x7f53d0069220 msgr2=0x7f53d006b6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:52:37.287 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:37.288+0000 7f53efa00640 1 --2- 172.21.15.18:0/3023242466 >> [v2:172.21.15.18:6800/3048894687,v1:172.21.15.18:6801/3048894687] conn(0x7f53d0069220 0x7f53d006b6d0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f53e8110410 tx=0x7f53d400b040 comp rx=0 tx=0).stop 2024-04-12T20:52:37.287 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:37.289+0000 7f53efa00640 1 -- 172.21.15.18:0/3023242466 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f53e806fda0 msgr2=0x7f53e810ef60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:52:37.287 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:37.289+0000 7f53efa00640 1 --2- 172.21.15.18:0/3023242466 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f53e806fda0 0x7f53e810ef60 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f53dc000c00 tx=0x7f53dc009da0 comp rx=0 tx=0).stop 2024-04-12T20:52:37.287 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:37.289+0000 7f53efa00640 1 -- 172.21.15.18:0/3023242466 shutdown_connections 2024-04-12T20:52:37.287 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:37.289+0000 7f53efa00640 1 --2- 172.21.15.18:0/3023242466 >> [v2:172.21.15.18:6800/3048894687,v1:172.21.15.18:6801/3048894687] conn(0x7f53d0069220 0x7f53d006b6d0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:52:37.287 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:37.289+0000 7f53efa00640 1 --2- 172.21.15.18:0/3023242466 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f53e8071330 0x7f53e810f4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:52:37.287 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:37.289+0000 7f53efa00640 1 --2- 172.21.15.18:0/3023242466 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f53e806fda0 0x7f53e810ef60 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:52:37.287 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:37.289+0000 7f53efa00640 1 -- 172.21.15.18:0/3023242466 >> 172.21.15.18:0/3023242466 conn(0x7f53e80f7960 msgr2=0x7f53e80f9020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T20:52:37.287 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:37.289+0000 7f53efa00640 1 -- 172.21.15.18:0/3023242466 shutdown_connections 2024-04-12T20:52:37.288 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:37.289+0000 7f53efa00640 1 -- 172.21.15.18:0/3023242466 wait complete. 2024-04-12T20:52:37.304 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-12T20:52:37.352 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:52:37 smithi018 ceph-mon[28799]: from='client.? 172.21.15.18:0/1593861509' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-12T20:52:37.352 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:52:37 smithi018 ceph-mon[28799]: from='client.? 172.21.15.18:0/2899552598' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"}]: dispatch 2024-04-12T20:52:37.508 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:37.509+0000 7fc490e00640 1 -- 172.21.15.18:0/4060696780 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc48c066210 msgr2=0x7fc48c0fedc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:52:37.508 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:37.510+0000 7fc490e00640 1 --2- 172.21.15.18:0/4060696780 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc48c066210 0x7fc48c0fedc0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fc47c009c50 tx=0x7fc47c004d10 comp rx=0 tx=0).stop 2024-04-12T20:52:37.508 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:37.510+0000 7fc490e00640 1 -- 172.21.15.18:0/4060696780 shutdown_connections 2024-04-12T20:52:37.508 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:37.510+0000 7fc490e00640 1 --2- 172.21.15.18:0/4060696780 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc48c0ff300 0x7fc48c1016e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:52:37.508 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:37.510+0000 7fc490e00640 1 --2- 172.21.15.18:0/4060696780 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc48c066210 0x7fc48c0fedc0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:52:37.508 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:37.510+0000 7fc490e00640 1 -- 172.21.15.18:0/4060696780 >> 172.21.15.18:0/4060696780 conn(0x7fc48c0f6850 msgr2=0x7fc48c0f8ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T20:52:37.508 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:37.510+0000 7fc490e00640 1 -- 172.21.15.18:0/4060696780 shutdown_connections 2024-04-12T20:52:37.508 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:37.510+0000 7fc490e00640 1 -- 172.21.15.18:0/4060696780 wait complete. 2024-04-12T20:52:37.510 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:37.511+0000 7fc490e00640 1 Processor -- start 2024-04-12T20:52:37.510 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:37.511+0000 7fc490e00640 1 -- start start 2024-04-12T20:52:37.510 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:37.511+0000 7fc490e00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc48c066210 0x7fc48c10ef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T20:52:37.510 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:37.511+0000 7fc490e00640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc48c0ff300 0x7fc48c10f4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T20:52:37.510 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:37.511+0000 7fc490e00640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc48c10faf0 con 0x7fc48c066210 2024-04-12T20:52:37.510 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:37.511+0000 7fc490e00640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc48c1144a0 con 0x7fc48c0ff300 2024-04-12T20:52:37.510 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:37.511+0000 7fc489e00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc48c066210 0x7fc48c10ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T20:52:37.510 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:37.511+0000 7fc489e00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc48c066210 0x7fc48c10ef60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:35620/0 (socket says 172.21.15.18:35620) 2024-04-12T20:52:37.510 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:37.511+0000 7fc489e00640 1 -- 172.21.15.18:0/498604343 learned_addr learned my addr 172.21.15.18:0/498604343 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T20:52:37.510 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:37.512+0000 7fc489400640 1 --2- 172.21.15.18:0/498604343 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc48c0ff300 0x7fc48c10f4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T20:52:37.510 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:37.512+0000 7fc489e00640 1 -- 172.21.15.18:0/498604343 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc48c0ff300 msgr2=0x7fc48c10f4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:52:37.511 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:37.512+0000 7fc489e00640 1 --2- 172.21.15.18:0/498604343 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc48c0ff300 0x7fc48c10f4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:52:37.511 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:37.512+0000 7fc489e00640 1 -- 172.21.15.18:0/498604343 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc47c0098b0 con 0x7fc48c066210 2024-04-12T20:52:37.511 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:37.512+0000 7fc489e00640 1 --2- 172.21.15.18:0/498604343 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc48c066210 0x7fc48c10ef60 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fc47c004a30 tx=0x7fc47c004d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T20:52:37.511 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:37.513+0000 7fc482000640 1 -- 172.21.15.18:0/498604343 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc47c03a070 con 0x7fc48c066210 2024-04-12T20:52:37.511 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:37.513+0000 7fc482000640 1 -- 172.21.15.18:0/498604343 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc47c009710 con 0x7fc48c066210 2024-04-12T20:52:37.511 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:37.513+0000 7fc490e00640 1 -- 172.21.15.18:0/498604343 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc48c1145e0 con 0x7fc48c066210 2024-04-12T20:52:37.512 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:37.513+0000 7fc482000640 1 -- 172.21.15.18:0/498604343 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc47c03eaf0 con 0x7fc48c066210 2024-04-12T20:52:37.512 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:37.513+0000 7fc490e00640 1 -- 172.21.15.18:0/498604343 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc48c114b50 con 0x7fc48c066210 2024-04-12T20:52:37.512 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:37.514+0000 7fc490e00640 1 -- 172.21.15.18:0/498604343 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc48c04bcc0 con 0x7fc48c066210 2024-04-12T20:52:37.512 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:37.515+0000 7fc482000640 1 -- 172.21.15.18:0/498604343 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7fc47c032070 con 0x7fc48c066210 2024-04-12T20:52:37.513 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:37.516+0000 7fc482000640 1 --2- 172.21.15.18:0/498604343 >> [v2:172.21.15.18:6800/3048894687,v1:172.21.15.18:6801/3048894687] conn(0x7fc470069210 0x7fc47006b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T20:52:37.514 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:37.516+0000 7fc482000640 1 -- 172.21.15.18:0/498604343 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5550+0+0 (secure 0 0 0) 0x7fc47c0a93a0 con 0x7fc48c066210 2024-04-12T20:52:37.514 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:37.516+0000 7fc489400640 1 --2- 172.21.15.18:0/498604343 >> [v2:172.21.15.18:6800/3048894687,v1:172.21.15.18:6801/3048894687] conn(0x7fc470069210 0x7fc47006b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T20:52:37.514 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:37.517+0000 7fc489400640 1 --2- 172.21.15.18:0/498604343 >> [v2:172.21.15.18:6800/3048894687,v1:172.21.15.18:6801/3048894687] conn(0x7fc470069210 0x7fc47006b6c0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fc48c110410 tx=0x7fc47800b040 comp rx=0 tx=0).ready entity=mgr.14212 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T20:52:37.522 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:37.524+0000 7fc482000640 1 -- 172.21.15.18:0/498604343 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7fc47c075600 con 0x7fc48c066210 2024-04-12T20:52:37.531 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:52:37 smithi186 ceph-mon[36597]: from='client.? 172.21.15.18:0/1593861509' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-12T20:52:37.531 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:52:37 smithi186 ceph-mon[36597]: from='client.? 172.21.15.18:0/2899552598' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"}]: dispatch 2024-04-12T20:52:37.752 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:37.754+0000 7fc490e00640 1 -- 172.21.15.18:0/498604343 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc48c104d20 con 0x7fc48c066210 2024-04-12T20:52:37.753 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:37.755+0000 7fc482000640 1 -- 172.21.15.18:0/498604343 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 19 v19) v1 ==== 94+0+4703 (secure 0 0 0) 0x7fc47c078c20 con 0x7fc48c066210 2024-04-12T20:52:37.753 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-12T20:52:37.753 INFO:teuthology.orchestra.run.smithi018.stdout:{"epoch":19,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24277,"name":"cephfs.smithi186.wjnafs","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.186:6827/438347207","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6826","nonce":438347207},{"type":"v1","addr":"172.21.15.186:6827","nonce":438347207}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":18},{"gid":24281,"name":"cephfs.smithi186.jdllyn","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6825/1464359952","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6824","nonce":1464359952},{"type":"v1","addr":"172.21.15.186:6825","nonce":1464359952}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":16}],"filesystems":[{"mdsmap":{"epoch":19,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-12T20:51:57.362850+0000","modified":"2024-04-12T20:52:19.000070+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":46,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":14452,"mds_1":14464},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14452":{"gid":14452,"name":"cephfs.smithi018.acqbsc","rank":0,"incarnation":4,"state":"up:active","state_seq":5,"addr":"172.21.15.18:6827/1564065434","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6826","nonce":1564065434},{"type":"v1","addr":"172.21.15.18:6827","nonce":1564065434}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_14464":{"gid":14464,"name":"cephfs.smithi018.szjluo","rank":1,"incarnation":15,"state":"up:active","state_seq":6,"addr":"172.21.15.18:6829/722443353","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6828","nonce":722443353},{"type":"v1","addr":"172.21.15.18:6829","nonce":722443353}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":1},"id":1}]} 2024-04-12T20:52:37.754 INFO:teuthology.orchestra.run.smithi018.stderr:dumped fsmap epoch 19 2024-04-12T20:52:37.756 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:37.758+0000 7fc490e00640 1 -- 172.21.15.18:0/498604343 >> [v2:172.21.15.18:6800/3048894687,v1:172.21.15.18:6801/3048894687] conn(0x7fc470069210 msgr2=0x7fc47006b6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:52:37.756 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:37.758+0000 7fc490e00640 1 --2- 172.21.15.18:0/498604343 >> [v2:172.21.15.18:6800/3048894687,v1:172.21.15.18:6801/3048894687] conn(0x7fc470069210 0x7fc47006b6c0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fc48c110410 tx=0x7fc47800b040 comp rx=0 tx=0).stop 2024-04-12T20:52:37.756 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:37.758+0000 7fc490e00640 1 -- 172.21.15.18:0/498604343 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc48c066210 msgr2=0x7fc48c10ef60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:52:37.756 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:37.758+0000 7fc490e00640 1 --2- 172.21.15.18:0/498604343 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc48c066210 0x7fc48c10ef60 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fc47c004a30 tx=0x7fc47c004d10 comp rx=0 tx=0).stop 2024-04-12T20:52:37.756 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:37.758+0000 7fc490e00640 1 -- 172.21.15.18:0/498604343 shutdown_connections 2024-04-12T20:52:37.756 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:37.758+0000 7fc490e00640 1 --2- 172.21.15.18:0/498604343 >> [v2:172.21.15.18:6800/3048894687,v1:172.21.15.18:6801/3048894687] conn(0x7fc470069210 0x7fc47006b6c0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:52:37.756 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:37.758+0000 7fc490e00640 1 --2- 172.21.15.18:0/498604343 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc48c0ff300 0x7fc48c10f4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:52:37.756 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:37.758+0000 7fc490e00640 1 --2- 172.21.15.18:0/498604343 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc48c066210 0x7fc48c10ef60 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:52:37.756 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:37.758+0000 7fc490e00640 1 -- 172.21.15.18:0/498604343 >> 172.21.15.18:0/498604343 conn(0x7fc48c0f6850 msgr2=0x7fc48c0fae40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T20:52:37.757 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:37.758+0000 7fc490e00640 1 -- 172.21.15.18:0/498604343 shutdown_connections 2024-04-12T20:52:37.757 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:37.758+0000 7fc490e00640 1 -- 172.21.15.18:0/498604343 wait complete. 2024-04-12T20:52:37.772 DEBUG:tasks.fs:fs fscid=1,name=cephfs state = {'epoch': 19, 'max_mds': 2, 'flags': 18} 2024-04-12T20:52:37.772 INFO:teuthology.run_tasks:Running task ceph-fuse... 2024-04-12T20:52:37.797 INFO:tasks.ceph_fuse:Running ceph_fuse task... 2024-04-12T20:52:37.797 INFO:tasks.ceph_fuse:config is {'client.0': {}, 'client.1': {}} 2024-04-12T20:52:37.797 INFO:tasks.ceph_fuse:client.0 config is {} 2024-04-12T20:52:37.798 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-04-12T20:52:37.798 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-12T20:52:37.798 INFO:tasks.ceph_fuse:client.1 config is {} 2024-04-12T20:52:37.798 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-04-12T20:52:37.798 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-04-12T20:52:37.798 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-12T20:52:37.798 DEBUG:teuthology.orchestra.run.smithi186:> ip netns list 2024-04-12T20:52:37.858 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-12T20:52:37.858 DEBUG:teuthology.orchestra.run.smithi186:> sudo ip link delete ceph-brx 2024-04-12T20:52:37.964 INFO:teuthology.orchestra.run.smithi186.stderr:Cannot find device "ceph-brx" 2024-04-12T20:52:37.967 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-12T20:52:37.968 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-12T20:52:37.968 DEBUG:teuthology.orchestra.run.smithi018:> ip netns list 2024-04-12T20:52:38.016 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-12T20:52:38.016 DEBUG:teuthology.orchestra.run.smithi018:> sudo ip link delete ceph-brx 2024-04-12T20:52:38.105 INFO:teuthology.orchestra.run.smithi018.stderr:Cannot find device "ceph-brx" 2024-04-12T20:52:38.108 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-12T20:52:38.108 INFO:tasks.ceph_fuse:Mounting ceph-fuse clients... 2024-04-12T20:52:38.109 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid 1cbe407a-f90d-11ee-bc8d-c7b262605968 -- ceph fs ls 2024-04-12T20:52:38.281 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:52:38 smithi186 ceph-mon[36597]: pgmap v240: 65 pgs: 65 active+clean; 580 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 0 B/s wr, 0 op/s 2024-04-12T20:52:38.281 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:52:38 smithi186 ceph-mon[36597]: from='client.? 172.21.15.18:0/3023242466' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"}]: dispatch 2024-04-12T20:52:38.281 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:52:38 smithi186 ceph-mon[36597]: from='client.? 172.21.15.18:0/498604343' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-12T20:52:38.352 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:52:38 smithi018 ceph-mon[28799]: pgmap v240: 65 pgs: 65 active+clean; 580 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 0 B/s wr, 0 op/s 2024-04-12T20:52:38.352 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:52:38 smithi018 ceph-mon[28799]: from='client.? 172.21.15.18:0/3023242466' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"}]: dispatch 2024-04-12T20:52:38.352 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:52:38 smithi018 ceph-mon[28799]: from='client.? 172.21.15.18:0/498604343' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-12T20:52:38.593 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/mon.smithi018/config 2024-04-12T20:52:39.531 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:52:39 smithi186 ceph-mon[36597]: pgmap v241: 65 pgs: 65 active+clean; 580 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-12T20:52:39.603 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:52:39 smithi018 ceph-mon[28799]: pgmap v241: 65 pgs: 65 active+clean; 580 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-12T20:52:40.983 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:40.985+0000 7fd133e00700 1 -- 172.21.15.18:0/1590082953 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd1341092a0 msgr2=0x7fd134109670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:52:40.984 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:40.985+0000 7fd133e00700 1 --2- 172.21.15.18:0/1590082953 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd1341092a0 0x7fd134109670 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fd124004940 tx=0x7fd124009e30 comp rx=0 tx=0).stop 2024-04-12T20:52:40.984 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:40.985+0000 7fd133e00700 1 -- 172.21.15.18:0/1590082953 shutdown_connections 2024-04-12T20:52:40.984 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:40.985+0000 7fd133e00700 1 --2- 172.21.15.18:0/1590082953 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd134109c40 0x7fd1341032b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:52:40.984 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:40.985+0000 7fd133e00700 1 --2- 172.21.15.18:0/1590082953 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd1341092a0 0x7fd134109670 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:52:40.984 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:40.985+0000 7fd133e00700 1 -- 172.21.15.18:0/1590082953 >> 172.21.15.18:0/1590082953 conn(0x7fd134072a30 msgr2=0x7fd134074e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T20:52:40.984 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:40.986+0000 7fd133e00700 1 -- 172.21.15.18:0/1590082953 shutdown_connections 2024-04-12T20:52:40.984 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:40.986+0000 7fd133e00700 1 -- 172.21.15.18:0/1590082953 wait complete. 2024-04-12T20:52:40.985 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:40.986+0000 7fd133e00700 1 Processor -- start 2024-04-12T20:52:40.986 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:40.986+0000 7fd133e00700 1 -- start start 2024-04-12T20:52:40.986 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:40.987+0000 7fd133e00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd1341092a0 0x7fd134119ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T20:52:40.986 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:40.987+0000 7fd133e00700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd134109c40 0x7fd13411a530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T20:52:40.986 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:40.987+0000 7fd133e00700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd13411ab80 con 0x7fd1341092a0 2024-04-12T20:52:40.986 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:40.987+0000 7fd133e00700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd134113f50 con 0x7fd134109c40 2024-04-12T20:52:40.986 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:40.987+0000 7fd132a00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd1341092a0 0x7fd134119ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T20:52:40.986 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:40.987+0000 7fd132a00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd1341092a0 0x7fd134119ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:35194/0 (socket says 172.21.15.18:35194) 2024-04-12T20:52:40.986 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:40.987+0000 7fd132a00700 1 -- 172.21.15.18:0/1565854072 learned_addr learned my addr 172.21.15.18:0/1565854072 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T20:52:40.986 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:40.987+0000 7fd132000700 1 --2- 172.21.15.18:0/1565854072 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd134109c40 0x7fd13411a530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T20:52:40.986 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:40.988+0000 7fd132a00700 1 -- 172.21.15.18:0/1565854072 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd134109c40 msgr2=0x7fd13411a530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:52:40.986 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:40.988+0000 7fd132a00700 1 --2- 172.21.15.18:0/1565854072 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd134109c40 0x7fd13411a530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:52:40.986 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:40.988+0000 7fd132a00700 1 -- 172.21.15.18:0/1565854072 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd124005040 con 0x7fd1341092a0 2024-04-12T20:52:40.986 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:40.988+0000 7fd132a00700 1 --2- 172.21.15.18:0/1565854072 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd1341092a0 0x7fd134119ff0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fd124009180 tx=0x7fd124009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T20:52:40.987 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:40.988+0000 7fd12aa00700 1 -- 172.21.15.18:0/1565854072 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd12400e040 con 0x7fd1341092a0 2024-04-12T20:52:40.987 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:40.988+0000 7fd133e00700 1 -- 172.21.15.18:0/1565854072 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd1341141d0 con 0x7fd1341092a0 2024-04-12T20:52:40.987 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:40.988+0000 7fd12aa00700 1 -- 172.21.15.18:0/1565854072 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd12400bbc0 con 0x7fd1341092a0 2024-04-12T20:52:40.987 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:40.988+0000 7fd12aa00700 1 -- 172.21.15.18:0/1565854072 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd12401c630 con 0x7fd1341092a0 2024-04-12T20:52:40.987 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:40.988+0000 7fd133e00700 1 -- 172.21.15.18:0/1565854072 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd134114790 con 0x7fd1341092a0 2024-04-12T20:52:40.988 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:40.990+0000 7fd133e00700 1 -- 172.21.15.18:0/1565854072 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd1341049f0 con 0x7fd1341092a0 2024-04-12T20:52:40.988 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:40.991+0000 7fd12aa00700 1 -- 172.21.15.18:0/1565854072 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7fd12400bd30 con 0x7fd1341092a0 2024-04-12T20:52:40.990 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:40.992+0000 7fd12aa00700 1 --2- 172.21.15.18:0/1565854072 >> [v2:172.21.15.18:6800/3048894687,v1:172.21.15.18:6801/3048894687] conn(0x7fd1180692a0 0x7fd11806b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T20:52:40.990 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:40.992+0000 7fd12aa00700 1 -- 172.21.15.18:0/1565854072 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5550+0+0 (secure 0 0 0) 0x7fd1240885a0 con 0x7fd1341092a0 2024-04-12T20:52:40.990 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:40.993+0000 7fd132000700 1 --2- 172.21.15.18:0/1565854072 >> [v2:172.21.15.18:6800/3048894687,v1:172.21.15.18:6801/3048894687] conn(0x7fd1180692a0 0x7fd11806b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T20:52:40.991 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:40.993+0000 7fd132000700 1 --2- 172.21.15.18:0/1565854072 >> [v2:172.21.15.18:6800/3048894687,v1:172.21.15.18:6801/3048894687] conn(0x7fd1180692a0 0x7fd11806b750 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fd120007fd0 tx=0x7fd120009040 comp rx=0 tx=0).ready entity=mgr.14212 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T20:52:40.995 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:40.997+0000 7fd12aa00700 1 -- 172.21.15.18:0/1565854072 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7fd1240582f0 con 0x7fd1341092a0 2024-04-12T20:52:41.259 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:41.261+0000 7fd133e00700 1 -- 172.21.15.18:0/1565854072 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fd13404bed0 con 0x7fd1341092a0 2024-04-12T20:52:41.260 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:41.262+0000 7fd12aa00700 1 -- 172.21.15.18:0/1565854072 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v19) v1 ==== 53+0+83 (secure 0 0 0) 0x7fd124057e80 con 0x7fd1341092a0 2024-04-12T20:52:41.260 INFO:teuthology.orchestra.run.smithi018.stdout:name: cephfs, metadata pool: cephfs.cephfs.meta, data pools: [cephfs.cephfs.data ] 2024-04-12T20:52:41.264 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:41.265+0000 7fd133e00700 1 -- 172.21.15.18:0/1565854072 >> [v2:172.21.15.18:6800/3048894687,v1:172.21.15.18:6801/3048894687] conn(0x7fd1180692a0 msgr2=0x7fd11806b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:52:41.264 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:41.265+0000 7fd133e00700 1 --2- 172.21.15.18:0/1565854072 >> [v2:172.21.15.18:6800/3048894687,v1:172.21.15.18:6801/3048894687] conn(0x7fd1180692a0 0x7fd11806b750 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fd120007fd0 tx=0x7fd120009040 comp rx=0 tx=0).stop 2024-04-12T20:52:41.264 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:41.265+0000 7fd133e00700 1 -- 172.21.15.18:0/1565854072 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd1341092a0 msgr2=0x7fd134119ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:52:41.264 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:41.265+0000 7fd133e00700 1 --2- 172.21.15.18:0/1565854072 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd1341092a0 0x7fd134119ff0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fd124009180 tx=0x7fd124009e30 comp rx=0 tx=0).stop 2024-04-12T20:52:41.264 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:41.266+0000 7fd133e00700 1 -- 172.21.15.18:0/1565854072 shutdown_connections 2024-04-12T20:52:41.264 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:41.266+0000 7fd133e00700 1 --2- 172.21.15.18:0/1565854072 >> [v2:172.21.15.18:6800/3048894687,v1:172.21.15.18:6801/3048894687] conn(0x7fd1180692a0 0x7fd11806b750 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:52:41.264 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:41.266+0000 7fd133e00700 1 --2- 172.21.15.18:0/1565854072 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd1341092a0 0x7fd134119ff0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:52:41.264 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:41.266+0000 7fd133e00700 1 --2- 172.21.15.18:0/1565854072 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd134109c40 0x7fd13411a530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:52:41.265 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:41.266+0000 7fd133e00700 1 -- 172.21.15.18:0/1565854072 >> 172.21.15.18:0/1565854072 conn(0x7fd134072a30 msgr2=0x7fd1340748d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T20:52:41.265 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:41.266+0000 7fd133e00700 1 -- 172.21.15.18:0/1565854072 shutdown_connections 2024-04-12T20:52:41.265 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:41.267+0000 7fd133e00700 1 -- 172.21.15.18:0/1565854072 wait complete. 2024-04-12T20:52:42.102 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:52:41 smithi018 ceph-mon[28799]: pgmap v242: 65 pgs: 65 active+clean; 580 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-12T20:52:42.102 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:52:41 smithi018 ceph-mon[28799]: from='client.? 172.21.15.18:0/1565854072' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-04-12T20:52:42.281 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:52:41 smithi186 ceph-mon[36597]: pgmap v242: 65 pgs: 65 active+clean; 580 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-12T20:52:42.281 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:52:41 smithi186 ceph-mon[36597]: from='client.? 172.21.15.18:0/1565854072' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-04-12T20:52:42.301 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-12T20:52:42.301 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-12T20:52:42.301 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi018.front.sepia.ceph.com 2024-04-12T20:52:42.301 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-12T20:52:42.301 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-12T20:52:42.301 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-12T20:52:42.302 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-12T20:52:42.302 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-12T20:52:42.302 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-12T20:52:42.302 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-12T20:52:42.302 DEBUG:teuthology.orchestra.run.smithi018:> ip addr 2024-04-12T20:52:42.365 INFO:teuthology.orchestra.run.smithi018.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-12T20:52:42.365 INFO:teuthology.orchestra.run.smithi018.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-12T20:52:42.365 INFO:teuthology.orchestra.run.smithi018.stdout: inet 127.0.0.1/8 scope host lo 2024-04-12T20:52:42.365 INFO:teuthology.orchestra.run.smithi018.stdout: valid_lft forever preferred_lft forever 2024-04-12T20:52:42.366 INFO:teuthology.orchestra.run.smithi018.stdout: inet6 ::1/128 scope host 2024-04-12T20:52:42.366 INFO:teuthology.orchestra.run.smithi018.stdout: valid_lft forever preferred_lft forever 2024-04-12T20:52:42.366 INFO:teuthology.orchestra.run.smithi018.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-12T20:52:42.366 INFO:teuthology.orchestra.run.smithi018.stdout: link/ether 0c:c4:7a:6b:fd:98 brd ff:ff:ff:ff:ff:ff 2024-04-12T20:52:42.366 INFO:teuthology.orchestra.run.smithi018.stdout: altname enp7s0f0 2024-04-12T20:52:42.366 INFO:teuthology.orchestra.run.smithi018.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-12T20:52:42.366 INFO:teuthology.orchestra.run.smithi018.stdout: link/ether 0c:c4:7a:6b:fd:99 brd ff:ff:ff:ff:ff:ff 2024-04-12T20:52:42.366 INFO:teuthology.orchestra.run.smithi018.stdout: altname enp7s0f1 2024-04-12T20:52:42.366 INFO:teuthology.orchestra.run.smithi018.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-12T20:52:42.366 INFO:teuthology.orchestra.run.smithi018.stdout: link/ether 0c:c4:7a:bd:13:62 brd ff:ff:ff:ff:ff:ff 2024-04-12T20:52:42.366 INFO:teuthology.orchestra.run.smithi018.stdout: altname enp1s0f0 2024-04-12T20:52:42.367 INFO:teuthology.orchestra.run.smithi018.stdout: inet 172.21.15.18/20 brd 172.21.15.255 scope global dynamic ens1f0 2024-04-12T20:52:42.367 INFO:teuthology.orchestra.run.smithi018.stdout: valid_lft 41967sec preferred_lft 41967sec 2024-04-12T20:52:42.367 INFO:teuthology.orchestra.run.smithi018.stdout: inet6 fe80::ec4:7aff:febd:1362/64 scope link noprefixroute 2024-04-12T20:52:42.367 INFO:teuthology.orchestra.run.smithi018.stdout: valid_lft forever preferred_lft forever 2024-04-12T20:52:42.367 INFO:teuthology.orchestra.run.smithi018.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-12T20:52:42.367 INFO:teuthology.orchestra.run.smithi018.stdout: link/ether 0c:c4:7a:bd:13:63 brd ff:ff:ff:ff:ff:ff 2024-04-12T20:52:42.367 INFO:teuthology.orchestra.run.smithi018.stdout: altname enp1s0f1 2024-04-12T20:52:42.368 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-04-12T20:52:42.369 DEBUG:teuthology.orchestra.run.smithi018:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-12T20:52:42.369 DEBUG:teuthology.orchestra.run.smithi018:> set -e 2024-04-12T20:52:42.369 DEBUG:teuthology.orchestra.run.smithi018:> sudo ip link add name ceph-brx type bridge 2024-04-12T20:52:42.369 DEBUG:teuthology.orchestra.run.smithi018:> sudo ip addr flush dev ceph-brx 2024-04-12T20:52:42.369 DEBUG:teuthology.orchestra.run.smithi018:> sudo ip link set ceph-brx up 2024-04-12T20:52:42.369 DEBUG:teuthology.orchestra.run.smithi018:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-04-12T20:52:42.369 DEBUG:teuthology.orchestra.run.smithi018:> ') 2024-04-12T20:52:42.481 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:42 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-12T20:52:42.928 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:42 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-12T20:52:42.938 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-12T20:52:42.938 DEBUG:teuthology.orchestra.run.smithi018:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-12T20:52:43.026 INFO:teuthology.orchestra.run.smithi018.stdout:1 2024-04-12T20:52:43.030 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-12T20:52:43.030 DEBUG:teuthology.orchestra.run.smithi018:> route 2024-04-12T20:52:43.084 INFO:teuthology.orchestra.run.smithi018.stdout:Kernel IP routing table 2024-04-12T20:52:43.084 INFO:teuthology.orchestra.run.smithi018.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-12T20:52:43.084 INFO:teuthology.orchestra.run.smithi018.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-04-12T20:52:43.084 INFO:teuthology.orchestra.run.smithi018.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-04-12T20:52:43.084 INFO:teuthology.orchestra.run.smithi018.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-12T20:52:43.085 DEBUG:teuthology.orchestra.run.smithi018:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-12T20:52:43.085 DEBUG:teuthology.orchestra.run.smithi018:> set -e 2024-04-12T20:52:43.085 DEBUG:teuthology.orchestra.run.smithi018:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-04-12T20:52:43.086 DEBUG:teuthology.orchestra.run.smithi018:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-04-12T20:52:43.086 DEBUG:teuthology.orchestra.run.smithi018:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-04-12T20:52:43.086 DEBUG:teuthology.orchestra.run.smithi018:> ') 2024-04-12T20:52:43.189 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:43 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-12T20:52:43.907 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:43 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-12T20:52:43.917 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-12T20:52:43.917 DEBUG:teuthology.orchestra.run.smithi018:> ip netns list 2024-04-12T20:52:43.973 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-12T20:52:43.974 DEBUG:teuthology.orchestra.run.smithi018:> ip netns list-id 2024-04-12T20:52:44.024 DEBUG:teuthology.orchestra.run.smithi018:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-12T20:52:44.024 DEBUG:teuthology.orchestra.run.smithi018:> set -e 2024-04-12T20:52:44.024 DEBUG:teuthology.orchestra.run.smithi018:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-12T20:52:44.024 DEBUG:teuthology.orchestra.run.smithi018:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-04-12T20:52:44.024 DEBUG:teuthology.orchestra.run.smithi018:> ') 2024-04-12T20:52:44.129 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:44 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-12T20:52:44.165 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:52:43 smithi018 ceph-mon[28799]: pgmap v243: 65 pgs: 65 active+clean; 580 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-12T20:52:44.225 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:44 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-12T20:52:44.235 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-04-12T20:52:44.235 DEBUG:teuthology.orchestra.run.smithi018:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-12T20:52:44.235 DEBUG:teuthology.orchestra.run.smithi018:> set -e 2024-04-12T20:52:44.235 DEBUG:teuthology.orchestra.run.smithi018:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-04-12T20:52:44.235 DEBUG:teuthology.orchestra.run.smithi018:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-04-12T20:52:44.235 DEBUG:teuthology.orchestra.run.smithi018:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-04-12T20:52:44.236 DEBUG:teuthology.orchestra.run.smithi018:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-04-12T20:52:44.236 DEBUG:teuthology.orchestra.run.smithi018:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-04-12T20:52:44.236 DEBUG:teuthology.orchestra.run.smithi018:> ') 2024-04-12T20:52:44.281 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:52:43 smithi186 ceph-mon[36597]: pgmap v243: 65 pgs: 65 active+clean; 580 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-12T20:52:44.350 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:44 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-12T20:52:44.620 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:44 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-12T20:52:44.630 DEBUG:teuthology.orchestra.run.smithi018:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-12T20:52:44.630 DEBUG:teuthology.orchestra.run.smithi018:> set -e 2024-04-12T20:52:44.630 DEBUG:teuthology.orchestra.run.smithi018:> sudo ip link set brx.0 up 2024-04-12T20:52:44.631 DEBUG:teuthology.orchestra.run.smithi018:> sudo ip link set dev brx.0 master ceph-brx 2024-04-12T20:52:44.631 DEBUG:teuthology.orchestra.run.smithi018:> ') 2024-04-12T20:52:44.741 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:44 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-12T20:52:44.871 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:44 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-12T20:52:44.881 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-04-12T20:52:44.881 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-12T20:52:44.882 DEBUG:teuthology.orchestra.run.smithi018:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-12T20:52:44.926 INFO:teuthology.orchestra.run.smithi018.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-12T20:52:44.926 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-12T20:52:44.926 DEBUG:teuthology.orchestra.run.smithi018:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-12T20:52:45.016 DEBUG:teuthology.orchestra.run.smithi018:> sudo modprobe fuse 2024-04-12T20:52:45.093 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:52:44 smithi018 ceph-mon[28799]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T20:52:45.111 DEBUG:teuthology.orchestra.run.smithi018:> cat /proc/self/mounts | awk '{print $2}' 2024-04-12T20:52:45.170 INFO:teuthology.orchestra.run.smithi018.stdout:/proc 2024-04-12T20:52:45.170 INFO:teuthology.orchestra.run.smithi018.stdout:/sys 2024-04-12T20:52:45.170 INFO:teuthology.orchestra.run.smithi018.stdout:/dev 2024-04-12T20:52:45.170 INFO:teuthology.orchestra.run.smithi018.stdout:/sys/kernel/security 2024-04-12T20:52:45.171 INFO:teuthology.orchestra.run.smithi018.stdout:/dev/shm 2024-04-12T20:52:45.171 INFO:teuthology.orchestra.run.smithi018.stdout:/dev/pts 2024-04-12T20:52:45.171 INFO:teuthology.orchestra.run.smithi018.stdout:/run 2024-04-12T20:52:45.171 INFO:teuthology.orchestra.run.smithi018.stdout:/sys/fs/cgroup 2024-04-12T20:52:45.171 INFO:teuthology.orchestra.run.smithi018.stdout:/sys/fs/pstore 2024-04-12T20:52:45.171 INFO:teuthology.orchestra.run.smithi018.stdout:/sys/fs/bpf 2024-04-12T20:52:45.171 INFO:teuthology.orchestra.run.smithi018.stdout:/ 2024-04-12T20:52:45.171 INFO:teuthology.orchestra.run.smithi018.stdout:/sys/fs/selinux 2024-04-12T20:52:45.171 INFO:teuthology.orchestra.run.smithi018.stdout:/proc/sys/fs/binfmt_misc 2024-04-12T20:52:45.171 INFO:teuthology.orchestra.run.smithi018.stdout:/dev/hugepages 2024-04-12T20:52:45.171 INFO:teuthology.orchestra.run.smithi018.stdout:/dev/mqueue 2024-04-12T20:52:45.171 INFO:teuthology.orchestra.run.smithi018.stdout:/sys/kernel/debug 2024-04-12T20:52:45.171 INFO:teuthology.orchestra.run.smithi018.stdout:/sys/kernel/tracing 2024-04-12T20:52:45.172 INFO:teuthology.orchestra.run.smithi018.stdout:/sys/fs/fuse/connections 2024-04-12T20:52:45.172 INFO:teuthology.orchestra.run.smithi018.stdout:/sys/kernel/config 2024-04-12T20:52:45.172 INFO:teuthology.orchestra.run.smithi018.stdout:/run/credentials/systemd-sysctl.service 2024-04-12T20:52:45.172 INFO:teuthology.orchestra.run.smithi018.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-12T20:52:45.172 INFO:teuthology.orchestra.run.smithi018.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-12T20:52:45.172 INFO:teuthology.orchestra.run.smithi018.stdout:/var/lib/nfs/rpc_pipefs 2024-04-12T20:52:45.172 INFO:teuthology.orchestra.run.smithi018.stdout:/run/user/1000 2024-04-12T20:52:45.172 INFO:teuthology.orchestra.run.smithi018.stdout:/var/lib/ceph 2024-04-12T20:52:45.172 INFO:teuthology.orchestra.run.smithi018.stdout:/var/lib/containers/storage/overlay 2024-04-12T20:52:45.172 INFO:teuthology.orchestra.run.smithi018.stdout:/var/lib/containers/storage/overlay/39ac019fe1de9002b6b8d30d2738609e21d77263b7a490e9cb343cc265014169/merged 2024-04-12T20:52:45.172 INFO:teuthology.orchestra.run.smithi018.stdout:/var/lib/containers/storage/overlay/414367f3d669fbf4aace5082e8f58b5cf315128b3de12fd35f8fb57a4263e7ba/merged 2024-04-12T20:52:45.172 INFO:teuthology.orchestra.run.smithi018.stdout:/run/user/0 2024-04-12T20:52:45.172 INFO:teuthology.orchestra.run.smithi018.stdout:/proc/sys/fs/binfmt_misc 2024-04-12T20:52:45.172 INFO:teuthology.orchestra.run.smithi018.stdout:/var/lib/containers/storage/overlay/ce1b8f89a829d065388ca48407234cd722eb89961bc2cb40705c80804bf46dcc/merged 2024-04-12T20:52:45.172 INFO:teuthology.orchestra.run.smithi018.stdout:/var/lib/containers/storage/overlay/46ac7285f6219c0613d6839beeb8335378e6eb3ef66eed70e0a0838da6356675/merged 2024-04-12T20:52:45.173 INFO:teuthology.orchestra.run.smithi018.stdout:/var/lib/containers/storage/overlay/9f432fc771864b05d65181a509d5a48fc9aaca3e3cab4b7db7167b7b2cce2c72/merged 2024-04-12T20:52:45.173 INFO:teuthology.orchestra.run.smithi018.stdout:/var/lib/containers/storage/overlay/b34b37371e0c9f3faed142107ce162ac9afbca8b5c9f2f44917256b2bb00885b/merged 2024-04-12T20:52:45.173 INFO:teuthology.orchestra.run.smithi018.stdout:/var/lib/containers/storage/overlay/7f1b2f149242288ff9668cd922f3fc1af28a2a7fbb05eeeb4406591209a8fbc6/merged 2024-04-12T20:52:45.173 INFO:teuthology.orchestra.run.smithi018.stdout:/var/lib/containers/storage/overlay/460f7508e26cefd335148eb310bddd7f75945ae81d35635352e03fe0ef629d71/merged 2024-04-12T20:52:45.173 INFO:teuthology.orchestra.run.smithi018.stdout:/var/lib/containers/storage/overlay/ff966e5fe9afe390c678df9af150302a82bd5de52c44fb5017016d706c338339/merged 2024-04-12T20:52:45.173 INFO:teuthology.orchestra.run.smithi018.stdout:/var/lib/containers/storage/overlay/8c89911b92384140e7e50017e44d3336858fe1622172a7e6fd96c45700ea435d/merged 2024-04-12T20:52:45.173 INFO:teuthology.orchestra.run.smithi018.stdout:/var/lib/containers/storage/overlay/92a3424e27c738e68c99c4911a67f4951508300577a320018b18aafe7fff830e/merged 2024-04-12T20:52:45.173 INFO:teuthology.orchestra.run.smithi018.stdout:/var/lib/containers/storage/overlay/6b261a3f93021e8a997e303090f7a1e4ca85ba5274f643b6e2f7baaa7b980f37/merged 2024-04-12T20:52:45.173 INFO:teuthology.orchestra.run.smithi018.stdout:/run/netns 2024-04-12T20:52:45.173 INFO:teuthology.orchestra.run.smithi018.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-12T20:52:45.173 INFO:teuthology.orchestra.run.smithi018.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-12T20:52:45.174 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-12T20:52:45.175 DEBUG:teuthology.orchestra.run.smithi018:> ls /sys/fs/fuse/connections 2024-04-12T20:52:45.230 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-04-12T20:52:45.230 DEBUG:teuthology.orchestra.run.smithi018:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-04-12T20:52:45.237 DEBUG:teuthology.orchestra.run.smithi018:> sudo modprobe fuse 2024-04-12T20:52:45.281 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:52:44 smithi186 ceph-mon[36597]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T20:52:45.321 DEBUG:teuthology.orchestra.run.smithi018:> cat /proc/self/mounts | awk '{print $2}' 2024-04-12T20:52:45.370 INFO:teuthology.orchestra.run.smithi018.stdout:/proc 2024-04-12T20:52:45.371 INFO:teuthology.orchestra.run.smithi018.stdout:/sys 2024-04-12T20:52:45.371 INFO:teuthology.orchestra.run.smithi018.stdout:/dev 2024-04-12T20:52:45.371 INFO:teuthology.orchestra.run.smithi018.stdout:/sys/kernel/security 2024-04-12T20:52:45.371 INFO:teuthology.orchestra.run.smithi018.stdout:/dev/shm 2024-04-12T20:52:45.371 INFO:teuthology.orchestra.run.smithi018.stdout:/dev/pts 2024-04-12T20:52:45.371 INFO:teuthology.orchestra.run.smithi018.stdout:/run 2024-04-12T20:52:45.371 INFO:teuthology.orchestra.run.smithi018.stdout:/sys/fs/cgroup 2024-04-12T20:52:45.371 INFO:teuthology.orchestra.run.smithi018.stdout:/sys/fs/pstore 2024-04-12T20:52:45.371 INFO:teuthology.orchestra.run.smithi018.stdout:/sys/fs/bpf 2024-04-12T20:52:45.371 INFO:teuthology.orchestra.run.smithi018.stdout:/ 2024-04-12T20:52:45.371 INFO:teuthology.orchestra.run.smithi018.stdout:/sys/fs/selinux 2024-04-12T20:52:45.371 INFO:teuthology.orchestra.run.smithi018.stdout:/proc/sys/fs/binfmt_misc 2024-04-12T20:52:45.371 INFO:teuthology.orchestra.run.smithi018.stdout:/dev/hugepages 2024-04-12T20:52:45.371 INFO:teuthology.orchestra.run.smithi018.stdout:/dev/mqueue 2024-04-12T20:52:45.372 INFO:teuthology.orchestra.run.smithi018.stdout:/sys/kernel/debug 2024-04-12T20:52:45.372 INFO:teuthology.orchestra.run.smithi018.stdout:/sys/kernel/tracing 2024-04-12T20:52:45.372 INFO:teuthology.orchestra.run.smithi018.stdout:/sys/fs/fuse/connections 2024-04-12T20:52:45.372 INFO:teuthology.orchestra.run.smithi018.stdout:/sys/kernel/config 2024-04-12T20:52:45.372 INFO:teuthology.orchestra.run.smithi018.stdout:/run/credentials/systemd-sysctl.service 2024-04-12T20:52:45.372 INFO:teuthology.orchestra.run.smithi018.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-12T20:52:45.372 INFO:teuthology.orchestra.run.smithi018.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-12T20:52:45.372 INFO:teuthology.orchestra.run.smithi018.stdout:/var/lib/nfs/rpc_pipefs 2024-04-12T20:52:45.372 INFO:teuthology.orchestra.run.smithi018.stdout:/run/user/1000 2024-04-12T20:52:45.372 INFO:teuthology.orchestra.run.smithi018.stdout:/var/lib/ceph 2024-04-12T20:52:45.372 INFO:teuthology.orchestra.run.smithi018.stdout:/var/lib/containers/storage/overlay 2024-04-12T20:52:45.373 INFO:teuthology.orchestra.run.smithi018.stdout:/var/lib/containers/storage/overlay/39ac019fe1de9002b6b8d30d2738609e21d77263b7a490e9cb343cc265014169/merged 2024-04-12T20:52:45.373 INFO:teuthology.orchestra.run.smithi018.stdout:/var/lib/containers/storage/overlay/414367f3d669fbf4aace5082e8f58b5cf315128b3de12fd35f8fb57a4263e7ba/merged 2024-04-12T20:52:45.373 INFO:teuthology.orchestra.run.smithi018.stdout:/run/user/0 2024-04-12T20:52:45.373 INFO:teuthology.orchestra.run.smithi018.stdout:/proc/sys/fs/binfmt_misc 2024-04-12T20:52:45.373 INFO:teuthology.orchestra.run.smithi018.stdout:/var/lib/containers/storage/overlay/ce1b8f89a829d065388ca48407234cd722eb89961bc2cb40705c80804bf46dcc/merged 2024-04-12T20:52:45.373 INFO:teuthology.orchestra.run.smithi018.stdout:/var/lib/containers/storage/overlay/46ac7285f6219c0613d6839beeb8335378e6eb3ef66eed70e0a0838da6356675/merged 2024-04-12T20:52:45.373 INFO:teuthology.orchestra.run.smithi018.stdout:/var/lib/containers/storage/overlay/9f432fc771864b05d65181a509d5a48fc9aaca3e3cab4b7db7167b7b2cce2c72/merged 2024-04-12T20:52:45.373 INFO:teuthology.orchestra.run.smithi018.stdout:/var/lib/containers/storage/overlay/b34b37371e0c9f3faed142107ce162ac9afbca8b5c9f2f44917256b2bb00885b/merged 2024-04-12T20:52:45.373 INFO:teuthology.orchestra.run.smithi018.stdout:/var/lib/containers/storage/overlay/7f1b2f149242288ff9668cd922f3fc1af28a2a7fbb05eeeb4406591209a8fbc6/merged 2024-04-12T20:52:45.373 INFO:teuthology.orchestra.run.smithi018.stdout:/var/lib/containers/storage/overlay/460f7508e26cefd335148eb310bddd7f75945ae81d35635352e03fe0ef629d71/merged 2024-04-12T20:52:45.373 INFO:teuthology.orchestra.run.smithi018.stdout:/var/lib/containers/storage/overlay/ff966e5fe9afe390c678df9af150302a82bd5de52c44fb5017016d706c338339/merged 2024-04-12T20:52:45.373 INFO:teuthology.orchestra.run.smithi018.stdout:/var/lib/containers/storage/overlay/8c89911b92384140e7e50017e44d3336858fe1622172a7e6fd96c45700ea435d/merged 2024-04-12T20:52:45.373 INFO:teuthology.orchestra.run.smithi018.stdout:/var/lib/containers/storage/overlay/92a3424e27c738e68c99c4911a67f4951508300577a320018b18aafe7fff830e/merged 2024-04-12T20:52:45.373 INFO:teuthology.orchestra.run.smithi018.stdout:/var/lib/containers/storage/overlay/6b261a3f93021e8a997e303090f7a1e4ca85ba5274f643b6e2f7baaa7b980f37/merged 2024-04-12T20:52:45.373 INFO:teuthology.orchestra.run.smithi018.stdout:/run/netns 2024-04-12T20:52:45.374 INFO:teuthology.orchestra.run.smithi018.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-12T20:52:45.374 INFO:teuthology.orchestra.run.smithi018.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-12T20:52:45.375 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-12T20:52:45.375 DEBUG:teuthology.orchestra.run.smithi018:> ls /sys/fs/fuse/connections 2024-04-12T20:52:45.463 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi018.stderr:2024-04-12T20:52:45.465+0000 7ff8ed4c6300 -1 init, newargv = 0x55f2485d3dc0 newargc=15 2024-04-12T20:52:45.463 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi018.stderr:ceph-fuse[76022]: starting ceph client 2024-04-12T20:52:45.483 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi018.stderr:ceph-fuse[76022]: starting fuse 2024-04-12T20:52:46.281 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:52:45 smithi186 ceph-mon[36597]: pgmap v244: 65 pgs: 65 active+clean; 580 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-12T20:52:46.352 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:52:45 smithi018 ceph-mon[28799]: pgmap v244: 65 pgs: 65 active+clean; 580 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-12T20:52:46.521 DEBUG:teuthology.orchestra.run.smithi018:> sudo modprobe fuse 2024-04-12T20:52:46.630 DEBUG:teuthology.orchestra.run.smithi018:> cat /proc/self/mounts | awk '{print $2}' 2024-04-12T20:52:46.691 INFO:teuthology.orchestra.run.smithi018.stdout:/proc 2024-04-12T20:52:46.691 INFO:teuthology.orchestra.run.smithi018.stdout:/sys 2024-04-12T20:52:46.691 INFO:teuthology.orchestra.run.smithi018.stdout:/dev 2024-04-12T20:52:46.691 INFO:teuthology.orchestra.run.smithi018.stdout:/sys/kernel/security 2024-04-12T20:52:46.691 INFO:teuthology.orchestra.run.smithi018.stdout:/dev/shm 2024-04-12T20:52:46.691 INFO:teuthology.orchestra.run.smithi018.stdout:/dev/pts 2024-04-12T20:52:46.691 INFO:teuthology.orchestra.run.smithi018.stdout:/run 2024-04-12T20:52:46.691 INFO:teuthology.orchestra.run.smithi018.stdout:/sys/fs/cgroup 2024-04-12T20:52:46.692 INFO:teuthology.orchestra.run.smithi018.stdout:/sys/fs/pstore 2024-04-12T20:52:46.692 INFO:teuthology.orchestra.run.smithi018.stdout:/sys/fs/bpf 2024-04-12T20:52:46.692 INFO:teuthology.orchestra.run.smithi018.stdout:/ 2024-04-12T20:52:46.692 INFO:teuthology.orchestra.run.smithi018.stdout:/sys/fs/selinux 2024-04-12T20:52:46.692 INFO:teuthology.orchestra.run.smithi018.stdout:/proc/sys/fs/binfmt_misc 2024-04-12T20:52:46.692 INFO:teuthology.orchestra.run.smithi018.stdout:/dev/hugepages 2024-04-12T20:52:46.692 INFO:teuthology.orchestra.run.smithi018.stdout:/dev/mqueue 2024-04-12T20:52:46.692 INFO:teuthology.orchestra.run.smithi018.stdout:/sys/kernel/debug 2024-04-12T20:52:46.692 INFO:teuthology.orchestra.run.smithi018.stdout:/sys/kernel/tracing 2024-04-12T20:52:46.692 INFO:teuthology.orchestra.run.smithi018.stdout:/sys/fs/fuse/connections 2024-04-12T20:52:46.692 INFO:teuthology.orchestra.run.smithi018.stdout:/sys/kernel/config 2024-04-12T20:52:46.692 INFO:teuthology.orchestra.run.smithi018.stdout:/run/credentials/systemd-sysctl.service 2024-04-12T20:52:46.692 INFO:teuthology.orchestra.run.smithi018.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-12T20:52:46.692 INFO:teuthology.orchestra.run.smithi018.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-12T20:52:46.693 INFO:teuthology.orchestra.run.smithi018.stdout:/var/lib/nfs/rpc_pipefs 2024-04-12T20:52:46.693 INFO:teuthology.orchestra.run.smithi018.stdout:/run/user/1000 2024-04-12T20:52:46.693 INFO:teuthology.orchestra.run.smithi018.stdout:/var/lib/ceph 2024-04-12T20:52:46.693 INFO:teuthology.orchestra.run.smithi018.stdout:/var/lib/containers/storage/overlay 2024-04-12T20:52:46.693 INFO:teuthology.orchestra.run.smithi018.stdout:/var/lib/containers/storage/overlay/39ac019fe1de9002b6b8d30d2738609e21d77263b7a490e9cb343cc265014169/merged 2024-04-12T20:52:46.693 INFO:teuthology.orchestra.run.smithi018.stdout:/var/lib/containers/storage/overlay/414367f3d669fbf4aace5082e8f58b5cf315128b3de12fd35f8fb57a4263e7ba/merged 2024-04-12T20:52:46.693 INFO:teuthology.orchestra.run.smithi018.stdout:/run/user/0 2024-04-12T20:52:46.693 INFO:teuthology.orchestra.run.smithi018.stdout:/proc/sys/fs/binfmt_misc 2024-04-12T20:52:46.693 INFO:teuthology.orchestra.run.smithi018.stdout:/var/lib/containers/storage/overlay/ce1b8f89a829d065388ca48407234cd722eb89961bc2cb40705c80804bf46dcc/merged 2024-04-12T20:52:46.693 INFO:teuthology.orchestra.run.smithi018.stdout:/var/lib/containers/storage/overlay/46ac7285f6219c0613d6839beeb8335378e6eb3ef66eed70e0a0838da6356675/merged 2024-04-12T20:52:46.693 INFO:teuthology.orchestra.run.smithi018.stdout:/var/lib/containers/storage/overlay/9f432fc771864b05d65181a509d5a48fc9aaca3e3cab4b7db7167b7b2cce2c72/merged 2024-04-12T20:52:46.693 INFO:teuthology.orchestra.run.smithi018.stdout:/var/lib/containers/storage/overlay/b34b37371e0c9f3faed142107ce162ac9afbca8b5c9f2f44917256b2bb00885b/merged 2024-04-12T20:52:46.693 INFO:teuthology.orchestra.run.smithi018.stdout:/var/lib/containers/storage/overlay/7f1b2f149242288ff9668cd922f3fc1af28a2a7fbb05eeeb4406591209a8fbc6/merged 2024-04-12T20:52:46.693 INFO:teuthology.orchestra.run.smithi018.stdout:/var/lib/containers/storage/overlay/460f7508e26cefd335148eb310bddd7f75945ae81d35635352e03fe0ef629d71/merged 2024-04-12T20:52:46.693 INFO:teuthology.orchestra.run.smithi018.stdout:/var/lib/containers/storage/overlay/ff966e5fe9afe390c678df9af150302a82bd5de52c44fb5017016d706c338339/merged 2024-04-12T20:52:46.694 INFO:teuthology.orchestra.run.smithi018.stdout:/var/lib/containers/storage/overlay/8c89911b92384140e7e50017e44d3336858fe1622172a7e6fd96c45700ea435d/merged 2024-04-12T20:52:46.694 INFO:teuthology.orchestra.run.smithi018.stdout:/var/lib/containers/storage/overlay/92a3424e27c738e68c99c4911a67f4951508300577a320018b18aafe7fff830e/merged 2024-04-12T20:52:46.694 INFO:teuthology.orchestra.run.smithi018.stdout:/var/lib/containers/storage/overlay/6b261a3f93021e8a997e303090f7a1e4ca85ba5274f643b6e2f7baaa7b980f37/merged 2024-04-12T20:52:46.694 INFO:teuthology.orchestra.run.smithi018.stdout:/run/netns 2024-04-12T20:52:46.694 INFO:teuthology.orchestra.run.smithi018.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-12T20:52:46.694 INFO:teuthology.orchestra.run.smithi018.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-12T20:52:46.694 INFO:teuthology.orchestra.run.smithi018.stdout:/home/ubuntu/cephtest/mnt.0 2024-04-12T20:52:46.695 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-12T20:52:46.695 DEBUG:teuthology.orchestra.run.smithi018:> ls /sys/fs/fuse/connections 2024-04-12T20:52:46.749 INFO:teuthology.orchestra.run.smithi018.stdout:148 2024-04-12T20:52:46.749 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [148] 2024-04-12T20:52:46.749 DEBUG:teuthology.orchestra.run.smithi018:> sudo stdin-killer -- python3 -c ' 2024-04-12T20:52:46.749 DEBUG:teuthology.orchestra.run.smithi018:> import glob 2024-04-12T20:52:46.750 DEBUG:teuthology.orchestra.run.smithi018:> import re 2024-04-12T20:52:46.750 DEBUG:teuthology.orchestra.run.smithi018:> import os 2024-04-12T20:52:46.750 DEBUG:teuthology.orchestra.run.smithi018:> import subprocess 2024-04-12T20:52:46.750 DEBUG:teuthology.orchestra.run.smithi018:> 2024-04-12T20:52:46.750 DEBUG:teuthology.orchestra.run.smithi018:> def _find_admin_socket(client_name): 2024-04-12T20:52:46.750 DEBUG:teuthology.orchestra.run.smithi018:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-04-12T20:52:46.750 DEBUG:teuthology.orchestra.run.smithi018:> files = glob.glob(asok_path) 2024-04-12T20:52:46.750 DEBUG:teuthology.orchestra.run.smithi018:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-04-12T20:52:46.750 DEBUG:teuthology.orchestra.run.smithi018:> 2024-04-12T20:52:46.750 DEBUG:teuthology.orchestra.run.smithi018:> # Given a non-glob path, it better be there 2024-04-12T20:52:46.750 DEBUG:teuthology.orchestra.run.smithi018:> if "*" not in asok_path: 2024-04-12T20:52:46.750 DEBUG:teuthology.orchestra.run.smithi018:> assert(len(files) == 1) 2024-04-12T20:52:46.750 DEBUG:teuthology.orchestra.run.smithi018:> return files[0] 2024-04-12T20:52:46.750 DEBUG:teuthology.orchestra.run.smithi018:> 2024-04-12T20:52:46.750 DEBUG:teuthology.orchestra.run.smithi018:> for f in files: 2024-04-12T20:52:46.751 DEBUG:teuthology.orchestra.run.smithi018:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-04-12T20:52:46.751 DEBUG:teuthology.orchestra.run.smithi018:> if os.path.exists("/proc/{0}".format(pid)): 2024-04-12T20:52:46.751 DEBUG:teuthology.orchestra.run.smithi018:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-04-12T20:52:46.751 DEBUG:teuthology.orchestra.run.smithi018:> contents = proc_f.read() 2024-04-12T20:52:46.751 DEBUG:teuthology.orchestra.run.smithi018:> if mountpoint in contents: 2024-04-12T20:52:46.751 DEBUG:teuthology.orchestra.run.smithi018:> return f 2024-04-12T20:52:46.751 DEBUG:teuthology.orchestra.run.smithi018:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-04-12T20:52:46.751 DEBUG:teuthology.orchestra.run.smithi018:> 2024-04-12T20:52:46.751 DEBUG:teuthology.orchestra.run.smithi018:> print(_find_admin_socket("client.0")) 2024-04-12T20:52:46.751 DEBUG:teuthology.orchestra.run.smithi018:> ' 2024-04-12T20:52:46.931 INFO:teuthology.orchestra.run.smithi018.stdout:/var/run/ceph/ceph-client.0.76022.asok 2024-04-12T20:52:46.936 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:46 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-12T20:52:46.950 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.76022.asok 2024-04-12T20:52:46.950 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-12T20:52:46.950 DEBUG:teuthology.orchestra.run.smithi018:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.76022.asok status 2024-04-12T20:52:47.189 INFO:teuthology.orchestra.run.smithi018.stdout:{ 2024-04-12T20:52:47.190 INFO:teuthology.orchestra.run.smithi018.stdout: "metadata": { 2024-04-12T20:52:47.190 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph_sha1": "d3b5060a17449edfae9385cdf792afec109fca5d", 2024-04-12T20:52:47.190 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph_version": "ceph version 17.2.7-827-gd3b5060a (d3b5060a17449edfae9385cdf792afec109fca5d) quincy (stable)", 2024-04-12T20:52:47.190 INFO:teuthology.orchestra.run.smithi018.stdout: "entity_id": "0", 2024-04-12T20:52:47.190 INFO:teuthology.orchestra.run.smithi018.stdout: "hostname": "smithi018", 2024-04-12T20:52:47.190 INFO:teuthology.orchestra.run.smithi018.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-04-12T20:52:47.190 INFO:teuthology.orchestra.run.smithi018.stdout: "pid": "76022", 2024-04-12T20:52:47.190 INFO:teuthology.orchestra.run.smithi018.stdout: "root": "/" 2024-04-12T20:52:47.190 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-12T20:52:47.190 INFO:teuthology.orchestra.run.smithi018.stdout: "dentry_count": 0, 2024-04-12T20:52:47.190 INFO:teuthology.orchestra.run.smithi018.stdout: "dentry_pinned_count": 0, 2024-04-12T20:52:47.190 INFO:teuthology.orchestra.run.smithi018.stdout: "id": 14516, 2024-04-12T20:52:47.190 INFO:teuthology.orchestra.run.smithi018.stdout: "inst": { 2024-04-12T20:52:47.191 INFO:teuthology.orchestra.run.smithi018.stdout: "name": { 2024-04-12T20:52:47.191 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "client", 2024-04-12T20:52:47.191 INFO:teuthology.orchestra.run.smithi018.stdout: "num": 14516 2024-04-12T20:52:47.191 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-12T20:52:47.191 INFO:teuthology.orchestra.run.smithi018.stdout: "addr": { 2024-04-12T20:52:47.191 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "v1", 2024-04-12T20:52:47.191 INFO:teuthology.orchestra.run.smithi018.stdout: "addr": "192.168.0.1:0", 2024-04-12T20:52:47.191 INFO:teuthology.orchestra.run.smithi018.stdout: "nonce": 3927222862 2024-04-12T20:52:47.191 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-04-12T20:52:47.191 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-12T20:52:47.191 INFO:teuthology.orchestra.run.smithi018.stdout: "addr": { 2024-04-12T20:52:47.191 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "v1", 2024-04-12T20:52:47.191 INFO:teuthology.orchestra.run.smithi018.stdout: "addr": "192.168.0.1:0", 2024-04-12T20:52:47.191 INFO:teuthology.orchestra.run.smithi018.stdout: "nonce": 3927222862 2024-04-12T20:52:47.191 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-12T20:52:47.192 INFO:teuthology.orchestra.run.smithi018.stdout: "inst_str": "client.14516 192.168.0.1:0/3927222862", 2024-04-12T20:52:47.192 INFO:teuthology.orchestra.run.smithi018.stdout: "addr_str": "192.168.0.1:0/3927222862", 2024-04-12T20:52:47.192 INFO:teuthology.orchestra.run.smithi018.stdout: "inode_count": 1, 2024-04-12T20:52:47.192 INFO:teuthology.orchestra.run.smithi018.stdout: "mds_epoch": 19, 2024-04-12T20:52:47.192 INFO:teuthology.orchestra.run.smithi018.stdout: "osd_epoch": 46, 2024-04-12T20:52:47.192 INFO:teuthology.orchestra.run.smithi018.stdout: "osd_epoch_barrier": 0, 2024-04-12T20:52:47.192 INFO:teuthology.orchestra.run.smithi018.stdout: "blocklisted": false, 2024-04-12T20:52:47.192 INFO:teuthology.orchestra.run.smithi018.stdout: "fs_name": "cephfs" 2024-04-12T20:52:47.192 INFO:teuthology.orchestra.run.smithi018.stdout:} 2024-04-12T20:52:47.205 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid 1cbe407a-f90d-11ee-bc8d-c7b262605968 -- ceph fs ls 2024-04-12T20:52:47.712 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/mon.smithi018/config 2024-04-12T20:52:48.129 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:52:47 smithi018 ceph-mon[28799]: pgmap v245: 65 pgs: 65 active+clean; 581 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 170 B/s wr, 0 op/s 2024-04-12T20:52:48.281 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:52:47 smithi186 ceph-mon[36597]: pgmap v245: 65 pgs: 65 active+clean; 581 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 170 B/s wr, 0 op/s 2024-04-12T20:52:50.055 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:50.056+0000 7f1521400700 1 -- 172.21.15.18:0/3383307662 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f151c109dd0 msgr2=0x7f151c103440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:52:50.055 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:50.056+0000 7f1521400700 1 --2- 172.21.15.18:0/3383307662 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f151c109dd0 0x7f151c103440 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f1504004940 tx=0x7f1504009e30 comp rx=0 tx=0).stop 2024-04-12T20:52:50.055 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:50.056+0000 7f1521400700 1 -- 172.21.15.18:0/3383307662 shutdown_connections 2024-04-12T20:52:50.055 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:50.056+0000 7f1521400700 1 --2- 172.21.15.18:0/3383307662 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f151c109dd0 0x7f151c103440 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:52:50.055 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:50.056+0000 7f1521400700 1 --2- 172.21.15.18:0/3383307662 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f151c109430 0x7f151c109800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:52:50.055 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:50.056+0000 7f1521400700 1 -- 172.21.15.18:0/3383307662 >> 172.21.15.18:0/3383307662 conn(0x7f151c072c00 msgr2=0x7f151c075010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T20:52:50.055 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:50.057+0000 7f1521400700 1 -- 172.21.15.18:0/3383307662 shutdown_connections 2024-04-12T20:52:50.055 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:50.057+0000 7f1521400700 1 -- 172.21.15.18:0/3383307662 wait complete. 2024-04-12T20:52:50.057 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:50.058+0000 7f1521400700 1 Processor -- start 2024-04-12T20:52:50.057 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:50.058+0000 7f1521400700 1 -- start start 2024-04-12T20:52:50.057 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:50.058+0000 7f1521400700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f151c109430 0x7f151c119fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T20:52:50.057 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:50.058+0000 7f1521400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f151c109dd0 0x7f151c11a520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T20:52:50.058 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:50.058+0000 7f1521400700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f151c11ab70 con 0x7f151c109dd0 2024-04-12T20:52:50.058 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:50.058+0000 7f1521400700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f151c114150 con 0x7f151c109430 2024-04-12T20:52:50.058 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:50.059+0000 7f1511e00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f151c109dd0 0x7f151c11a520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T20:52:50.058 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:50.059+0000 7f1511e00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f151c109dd0 0x7f151c11a520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:35210/0 (socket says 172.21.15.18:35210) 2024-04-12T20:52:50.058 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:50.059+0000 7f1511e00700 1 -- 172.21.15.18:0/1639858486 learned_addr learned my addr 172.21.15.18:0/1639858486 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T20:52:50.058 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:50.059+0000 7f151a800700 1 --2- 172.21.15.18:0/1639858486 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f151c109430 0x7f151c119fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T20:52:50.058 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:50.059+0000 7f1511e00700 1 -- 172.21.15.18:0/1639858486 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f151c109430 msgr2=0x7f151c119fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:52:50.058 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:50.059+0000 7f1511e00700 1 --2- 172.21.15.18:0/1639858486 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f151c109430 0x7f151c119fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:52:50.058 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:50.059+0000 7f1511e00700 1 -- 172.21.15.18:0/1639858486 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1504005040 con 0x7f151c109dd0 2024-04-12T20:52:50.058 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:50.060+0000 7f1511e00700 1 --2- 172.21.15.18:0/1639858486 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f151c109dd0 0x7f151c11a520 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f1504009100 tx=0x7f1504009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T20:52:50.059 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:50.060+0000 7f1513400700 1 -- 172.21.15.18:0/1639858486 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f150400e040 con 0x7f151c109dd0 2024-04-12T20:52:50.059 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:50.060+0000 7f1513400700 1 -- 172.21.15.18:0/1639858486 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f1504013420 con 0x7f151c109dd0 2024-04-12T20:52:50.059 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:50.060+0000 7f1521400700 1 -- 172.21.15.18:0/1639858486 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f151c1143d0 con 0x7f151c109dd0 2024-04-12T20:52:50.059 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:50.060+0000 7f1513400700 1 -- 172.21.15.18:0/1639858486 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f150401c630 con 0x7f151c109dd0 2024-04-12T20:52:50.060 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:50.060+0000 7f1521400700 1 -- 172.21.15.18:0/1639858486 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f151c114990 con 0x7f151c109dd0 2024-04-12T20:52:50.060 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:50.061+0000 7f1521400700 1 -- 172.21.15.18:0/1639858486 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f151c104b80 con 0x7f151c109dd0 2024-04-12T20:52:50.060 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:50.063+0000 7f1513400700 1 -- 172.21.15.18:0/1639858486 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7f1504013590 con 0x7f151c109dd0 2024-04-12T20:52:50.062 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:50.064+0000 7f1513400700 1 --2- 172.21.15.18:0/1639858486 >> [v2:172.21.15.18:6800/3048894687,v1:172.21.15.18:6801/3048894687] conn(0x7f15000692a0 0x7f150006b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T20:52:50.062 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:50.064+0000 7f1513400700 1 -- 172.21.15.18:0/1639858486 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5550+0+0 (secure 0 0 0) 0x7f1504088f70 con 0x7f151c109dd0 2024-04-12T20:52:50.062 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:50.065+0000 7f151a800700 1 --2- 172.21.15.18:0/1639858486 >> [v2:172.21.15.18:6800/3048894687,v1:172.21.15.18:6801/3048894687] conn(0x7f15000692a0 0x7f150006b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T20:52:50.063 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:50.065+0000 7f151a800700 1 --2- 172.21.15.18:0/1639858486 >> [v2:172.21.15.18:6800/3048894687,v1:172.21.15.18:6801/3048894687] conn(0x7f15000692a0 0x7f150006b750 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f150c007fd0 tx=0x7f150c009d70 comp rx=0 tx=0).ready entity=mgr.14212 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T20:52:50.066 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:50.068+0000 7f1513400700 1 -- 172.21.15.18:0/1639858486 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7f1504058cc0 con 0x7f151c109dd0 2024-04-12T20:52:50.281 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:52:49 smithi186 ceph-mon[36597]: pgmap v246: 65 pgs: 65 active+clean; 581 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 255 B/s wr, 0 op/s 2024-04-12T20:52:50.293 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:52:49 smithi018 ceph-mon[28799]: pgmap v246: 65 pgs: 65 active+clean; 581 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 255 B/s wr, 0 op/s 2024-04-12T20:52:50.332 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:50.334+0000 7f1521400700 1 -- 172.21.15.18:0/1639858486 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f151c04c710 con 0x7f151c109dd0 2024-04-12T20:52:50.333 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:50.335+0000 7f1513400700 1 -- 172.21.15.18:0/1639858486 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v19) v1 ==== 53+0+83 (secure 0 0 0) 0x7f1504058850 con 0x7f151c109dd0 2024-04-12T20:52:50.333 INFO:teuthology.orchestra.run.smithi018.stdout:name: cephfs, metadata pool: cephfs.cephfs.meta, data pools: [cephfs.cephfs.data ] 2024-04-12T20:52:50.336 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:50.338+0000 7f1521400700 1 -- 172.21.15.18:0/1639858486 >> [v2:172.21.15.18:6800/3048894687,v1:172.21.15.18:6801/3048894687] conn(0x7f15000692a0 msgr2=0x7f150006b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:52:50.336 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:50.338+0000 7f1521400700 1 --2- 172.21.15.18:0/1639858486 >> [v2:172.21.15.18:6800/3048894687,v1:172.21.15.18:6801/3048894687] conn(0x7f15000692a0 0x7f150006b750 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f150c007fd0 tx=0x7f150c009d70 comp rx=0 tx=0).stop 2024-04-12T20:52:50.336 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:50.338+0000 7f1521400700 1 -- 172.21.15.18:0/1639858486 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f151c109dd0 msgr2=0x7f151c11a520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:52:50.336 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:50.338+0000 7f1521400700 1 --2- 172.21.15.18:0/1639858486 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f151c109dd0 0x7f151c11a520 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f1504009100 tx=0x7f1504009e30 comp rx=0 tx=0).stop 2024-04-12T20:52:50.336 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:50.338+0000 7f1521400700 1 -- 172.21.15.18:0/1639858486 shutdown_connections 2024-04-12T20:52:50.336 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:50.338+0000 7f1521400700 1 --2- 172.21.15.18:0/1639858486 >> [v2:172.21.15.18:6800/3048894687,v1:172.21.15.18:6801/3048894687] conn(0x7f15000692a0 0x7f150006b750 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:52:50.336 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:50.338+0000 7f1521400700 1 --2- 172.21.15.18:0/1639858486 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f151c109430 0x7f151c119fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:52:50.336 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:50.338+0000 7f1521400700 1 --2- 172.21.15.18:0/1639858486 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f151c109dd0 0x7f151c11a520 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:52:50.336 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:50.338+0000 7f1521400700 1 -- 172.21.15.18:0/1639858486 >> 172.21.15.18:0/1639858486 conn(0x7f151c072c00 msgr2=0x7f151c074be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T20:52:50.337 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:50.338+0000 7f1521400700 1 -- 172.21.15.18:0/1639858486 shutdown_connections 2024-04-12T20:52:50.337 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:52:50.339+0000 7f1521400700 1 -- 172.21.15.18:0/1639858486 wait complete. 2024-04-12T20:52:51.102 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:52:50 smithi018 ceph-mon[28799]: from='client.? 172.21.15.18:0/1639858486' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-04-12T20:52:51.268 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-12T20:52:51.269 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-12T20:52:51.269 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi186.front.sepia.ceph.com 2024-04-12T20:52:51.269 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-04-12T20:52:51.269 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-04-12T20:52:51.269 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-12T20:52:51.269 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-12T20:52:51.269 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-12T20:52:51.269 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.1' 2024-04-12T20:52:51.269 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-12T20:52:51.269 DEBUG:teuthology.orchestra.run.smithi186:> ip addr 2024-04-12T20:52:51.281 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:52:50 smithi186 ceph-mon[36597]: from='client.? 172.21.15.18:0/1639858486' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-04-12T20:52:51.328 INFO:teuthology.orchestra.run.smithi186.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-12T20:52:51.328 INFO:teuthology.orchestra.run.smithi186.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-12T20:52:51.328 INFO:teuthology.orchestra.run.smithi186.stdout: inet 127.0.0.1/8 scope host lo 2024-04-12T20:52:51.328 INFO:teuthology.orchestra.run.smithi186.stdout: valid_lft forever preferred_lft forever 2024-04-12T20:52:51.328 INFO:teuthology.orchestra.run.smithi186.stdout: inet6 ::1/128 scope host 2024-04-12T20:52:51.328 INFO:teuthology.orchestra.run.smithi186.stdout: valid_lft forever preferred_lft forever 2024-04-12T20:52:51.329 INFO:teuthology.orchestra.run.smithi186.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-12T20:52:51.329 INFO:teuthology.orchestra.run.smithi186.stdout: link/ether 0c:c4:7a:d9:38:2c brd ff:ff:ff:ff:ff:ff 2024-04-12T20:52:51.329 INFO:teuthology.orchestra.run.smithi186.stdout: altname enp7s0f0 2024-04-12T20:52:51.329 INFO:teuthology.orchestra.run.smithi186.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-12T20:52:51.329 INFO:teuthology.orchestra.run.smithi186.stdout: link/ether 0c:c4:7a:d9:38:2d brd ff:ff:ff:ff:ff:ff 2024-04-12T20:52:51.329 INFO:teuthology.orchestra.run.smithi186.stdout: altname enp7s0f1 2024-04-12T20:52:51.329 INFO:teuthology.orchestra.run.smithi186.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-12T20:52:51.329 INFO:teuthology.orchestra.run.smithi186.stdout: link/ether 0c:c4:7a:88:6d:f4 brd ff:ff:ff:ff:ff:ff 2024-04-12T20:52:51.329 INFO:teuthology.orchestra.run.smithi186.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-12T20:52:51.329 INFO:teuthology.orchestra.run.smithi186.stdout: link/ether 0c:c4:7a:88:6d:f5 brd ff:ff:ff:ff:ff:ff 2024-04-12T20:52:51.329 INFO:teuthology.orchestra.run.smithi186.stdout: inet 172.21.15.186/20 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-04-12T20:52:51.329 INFO:teuthology.orchestra.run.smithi186.stdout: valid_lft 41982sec preferred_lft 41982sec 2024-04-12T20:52:51.329 INFO:teuthology.orchestra.run.smithi186.stdout: inet6 fe80::ec4:7aff:fe88:6df5/64 scope link 2024-04-12T20:52:51.330 INFO:teuthology.orchestra.run.smithi186.stdout: valid_lft forever preferred_lft forever 2024-04-12T20:52:51.331 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-04-12T20:52:51.331 DEBUG:teuthology.orchestra.run.smithi186:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-12T20:52:51.331 DEBUG:teuthology.orchestra.run.smithi186:> set -e 2024-04-12T20:52:51.331 DEBUG:teuthology.orchestra.run.smithi186:> sudo ip link add name ceph-brx type bridge 2024-04-12T20:52:51.331 DEBUG:teuthology.orchestra.run.smithi186:> sudo ip addr flush dev ceph-brx 2024-04-12T20:52:51.332 DEBUG:teuthology.orchestra.run.smithi186:> sudo ip link set ceph-brx up 2024-04-12T20:52:51.332 DEBUG:teuthology.orchestra.run.smithi186:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-04-12T20:52:51.332 DEBUG:teuthology.orchestra.run.smithi186:> ') 2024-04-12T20:52:51.487 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:52:51 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-12T20:52:51.905 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:52:51 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-12T20:52:51.915 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-12T20:52:51.915 DEBUG:teuthology.orchestra.run.smithi186:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-12T20:52:51.988 INFO:teuthology.orchestra.run.smithi186.stdout:1 2024-04-12T20:52:51.992 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-12T20:52:51.992 DEBUG:teuthology.orchestra.run.smithi186:> route 2024-04-12T20:52:52.031 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:52:51 smithi186 ceph-mon[36597]: pgmap v247: 65 pgs: 65 active+clean; 581 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 341 B/s wr, 0 op/s 2024-04-12T20:52:52.042 INFO:teuthology.orchestra.run.smithi186.stdout:Kernel IP routing table 2024-04-12T20:52:52.043 INFO:teuthology.orchestra.run.smithi186.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-12T20:52:52.043 INFO:teuthology.orchestra.run.smithi186.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-12T20:52:52.043 INFO:teuthology.orchestra.run.smithi186.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-12T20:52:52.043 INFO:teuthology.orchestra.run.smithi186.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-12T20:52:52.043 DEBUG:teuthology.orchestra.run.smithi186:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-12T20:52:52.043 DEBUG:teuthology.orchestra.run.smithi186:> set -e 2024-04-12T20:52:52.044 DEBUG:teuthology.orchestra.run.smithi186:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-12T20:52:52.044 DEBUG:teuthology.orchestra.run.smithi186:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-12T20:52:52.044 DEBUG:teuthology.orchestra.run.smithi186:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-12T20:52:52.044 DEBUG:teuthology.orchestra.run.smithi186:> ') 2024-04-12T20:52:52.176 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:52:52 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-12T20:52:52.295 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:52:51 smithi018 ceph-mon[28799]: pgmap v247: 65 pgs: 65 active+clean; 581 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 341 B/s wr, 0 op/s 2024-04-12T20:52:52.819 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:52:52 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-12T20:52:52.827 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-12T20:52:52.827 DEBUG:teuthology.orchestra.run.smithi186:> ip netns list 2024-04-12T20:52:52.872 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-12T20:52:52.872 DEBUG:teuthology.orchestra.run.smithi186:> ip netns list-id 2024-04-12T20:52:52.963 DEBUG:teuthology.orchestra.run.smithi186:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-12T20:52:52.963 DEBUG:teuthology.orchestra.run.smithi186:> set -e 2024-04-12T20:52:52.963 DEBUG:teuthology.orchestra.run.smithi186:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-12T20:52:52.963 DEBUG:teuthology.orchestra.run.smithi186:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.1 0 2024-04-12T20:52:52.963 DEBUG:teuthology.orchestra.run.smithi186:> ') 2024-04-12T20:52:53.060 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:52:53 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-12T20:52:53.143 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:52:53 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-12T20:52:53.152 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.1' with 192.168.0.1/16 2024-04-12T20:52:53.152 DEBUG:teuthology.orchestra.run.smithi186:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-12T20:52:53.152 DEBUG:teuthology.orchestra.run.smithi186:> set -e 2024-04-12T20:52:53.153 DEBUG:teuthology.orchestra.run.smithi186:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.1 type veth peer name brx.0 2024-04-12T20:52:53.153 DEBUG:teuthology.orchestra.run.smithi186:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-04-12T20:52:53.153 DEBUG:teuthology.orchestra.run.smithi186:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set veth0 up 2024-04-12T20:52:53.153 DEBUG:teuthology.orchestra.run.smithi186:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set lo up 2024-04-12T20:52:53.153 DEBUG:teuthology.orchestra.run.smithi186:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip route add default via 192.168.255.254 2024-04-12T20:52:53.153 DEBUG:teuthology.orchestra.run.smithi186:> ') 2024-04-12T20:52:53.247 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:52:53 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-12T20:52:53.466 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:52:53 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-12T20:52:53.476 DEBUG:teuthology.orchestra.run.smithi186:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-12T20:52:53.476 DEBUG:teuthology.orchestra.run.smithi186:> set -e 2024-04-12T20:52:53.476 DEBUG:teuthology.orchestra.run.smithi186:> sudo ip link set brx.0 up 2024-04-12T20:52:53.476 DEBUG:teuthology.orchestra.run.smithi186:> sudo ip link set dev brx.0 master ceph-brx 2024-04-12T20:52:53.476 DEBUG:teuthology.orchestra.run.smithi186:> ') 2024-04-12T20:52:53.577 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:52:53 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-12T20:52:53.697 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:52:53 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-12T20:52:53.706 INFO:tasks.cephfs.fuse_mount:Client client.1 config is {} 2024-04-12T20:52:53.706 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-12T20:52:53.706 DEBUG:teuthology.orchestra.run.smithi186:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-04-12T20:52:53.751 INFO:teuthology.orchestra.run.smithi186.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-04-12T20:52:53.751 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-12T20:52:53.751 DEBUG:teuthology.orchestra.run.smithi186:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-04-12T20:52:53.840 DEBUG:teuthology.orchestra.run.smithi186:> sudo modprobe fuse 2024-04-12T20:52:53.979 DEBUG:teuthology.orchestra.run.smithi186:> cat /proc/self/mounts | awk '{print $2}' 2024-04-12T20:52:54.035 INFO:teuthology.orchestra.run.smithi186.stdout:/proc 2024-04-12T20:52:54.036 INFO:teuthology.orchestra.run.smithi186.stdout:/sys 2024-04-12T20:52:54.036 INFO:teuthology.orchestra.run.smithi186.stdout:/dev 2024-04-12T20:52:54.036 INFO:teuthology.orchestra.run.smithi186.stdout:/sys/kernel/security 2024-04-12T20:52:54.036 INFO:teuthology.orchestra.run.smithi186.stdout:/dev/shm 2024-04-12T20:52:54.036 INFO:teuthology.orchestra.run.smithi186.stdout:/dev/pts 2024-04-12T20:52:54.036 INFO:teuthology.orchestra.run.smithi186.stdout:/run 2024-04-12T20:52:54.036 INFO:teuthology.orchestra.run.smithi186.stdout:/sys/fs/cgroup 2024-04-12T20:52:54.036 INFO:teuthology.orchestra.run.smithi186.stdout:/sys/fs/pstore 2024-04-12T20:52:54.036 INFO:teuthology.orchestra.run.smithi186.stdout:/sys/fs/bpf 2024-04-12T20:52:54.036 INFO:teuthology.orchestra.run.smithi186.stdout:/ 2024-04-12T20:52:54.036 INFO:teuthology.orchestra.run.smithi186.stdout:/sys/fs/selinux 2024-04-12T20:52:54.037 INFO:teuthology.orchestra.run.smithi186.stdout:/proc/sys/fs/binfmt_misc 2024-04-12T20:52:54.037 INFO:teuthology.orchestra.run.smithi186.stdout:/dev/mqueue 2024-04-12T20:52:54.037 INFO:teuthology.orchestra.run.smithi186.stdout:/dev/hugepages 2024-04-12T20:52:54.037 INFO:teuthology.orchestra.run.smithi186.stdout:/sys/kernel/debug 2024-04-12T20:52:54.037 INFO:teuthology.orchestra.run.smithi186.stdout:/sys/kernel/tracing 2024-04-12T20:52:54.037 INFO:teuthology.orchestra.run.smithi186.stdout:/sys/fs/fuse/connections 2024-04-12T20:52:54.037 INFO:teuthology.orchestra.run.smithi186.stdout:/sys/kernel/config 2024-04-12T20:52:54.037 INFO:teuthology.orchestra.run.smithi186.stdout:/run/credentials/systemd-sysctl.service 2024-04-12T20:52:54.037 INFO:teuthology.orchestra.run.smithi186.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-12T20:52:54.037 INFO:teuthology.orchestra.run.smithi186.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-12T20:52:54.037 INFO:teuthology.orchestra.run.smithi186.stdout:/var/lib/nfs/rpc_pipefs 2024-04-12T20:52:54.037 INFO:teuthology.orchestra.run.smithi186.stdout:/run/user/1000 2024-04-12T20:52:54.037 INFO:teuthology.orchestra.run.smithi186.stdout:/var/lib/ceph 2024-04-12T20:52:54.038 INFO:teuthology.orchestra.run.smithi186.stdout:/run/user/0 2024-04-12T20:52:54.038 INFO:teuthology.orchestra.run.smithi186.stdout:/proc/sys/fs/binfmt_misc 2024-04-12T20:52:54.038 INFO:teuthology.orchestra.run.smithi186.stdout:/var/lib/containers/storage/overlay 2024-04-12T20:52:54.038 INFO:teuthology.orchestra.run.smithi186.stdout:/var/lib/containers/storage/overlay/87dc767b69aa91b4f8cbdf8139ec13e1db8bb517ba28a6649bea8a0373ad383c/merged 2024-04-12T20:52:54.038 INFO:teuthology.orchestra.run.smithi186.stdout:/var/lib/containers/storage/overlay/aa240d65e1d8e5509a2cdaa303816fc3c8c6f5e1257902fd619c10aa49486e96/merged 2024-04-12T20:52:54.038 INFO:teuthology.orchestra.run.smithi186.stdout:/var/lib/containers/storage/overlay/27fe6211ed2e375c10007ee138a5d85ebf2360f4d5bb472e0822d4de2d7e0667/merged 2024-04-12T20:52:54.038 INFO:teuthology.orchestra.run.smithi186.stdout:/var/lib/containers/storage/overlay/29e0504e1742d861bab3b8c6ade6c2ff618765efe514a64a92fc44d346d6f02e/merged 2024-04-12T20:52:54.038 INFO:teuthology.orchestra.run.smithi186.stdout:/var/lib/containers/storage/overlay/50dcf6f17c443229f3c680a040042b4932ceba22d244974122d772600e9b0287/merged 2024-04-12T20:52:54.038 INFO:teuthology.orchestra.run.smithi186.stdout:/var/lib/containers/storage/overlay/b09826eb97bd3f527d6a3eefd54e85ab04ac95e897ad33b0acc5c46747beaf7b/merged 2024-04-12T20:52:54.038 INFO:teuthology.orchestra.run.smithi186.stdout:/var/lib/containers/storage/overlay/237d19265be4f6aacf9dcfa82804b3f8ebb8057de8759f1395c1ac23cc534572/merged 2024-04-12T20:52:54.038 INFO:teuthology.orchestra.run.smithi186.stdout:/var/lib/containers/storage/overlay/8ee8aee72db28a01e0e40f791e8c4571845f0d12d739beec0ff488435441f467/merged 2024-04-12T20:52:54.038 INFO:teuthology.orchestra.run.smithi186.stdout:/var/lib/containers/storage/overlay/080b1947d86b67befe994ad9bcba0168ff28d5b23879da2be9dca51f68476e28/merged 2024-04-12T20:52:54.038 INFO:teuthology.orchestra.run.smithi186.stdout:/run/netns 2024-04-12T20:52:54.038 INFO:teuthology.orchestra.run.smithi186.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-12T20:52:54.039 INFO:teuthology.orchestra.run.smithi186.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-12T20:52:54.039 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-12T20:52:54.040 DEBUG:teuthology.orchestra.run.smithi186:> ls /sys/fs/fuse/connections 2024-04-12T20:52:54.089 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-04-12T20:52:54.090 DEBUG:teuthology.orchestra.run.smithi186:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.1 --id 1) 2024-04-12T20:52:54.094 DEBUG:teuthology.orchestra.run.smithi186:> sudo modprobe fuse 2024-04-12T20:52:54.173 DEBUG:teuthology.orchestra.run.smithi186:> cat /proc/self/mounts | awk '{print $2}' 2024-04-12T20:52:54.218 INFO:teuthology.orchestra.run.smithi186.stdout:/proc 2024-04-12T20:52:54.218 INFO:teuthology.orchestra.run.smithi186.stdout:/sys 2024-04-12T20:52:54.218 INFO:teuthology.orchestra.run.smithi186.stdout:/dev 2024-04-12T20:52:54.219 INFO:teuthology.orchestra.run.smithi186.stdout:/sys/kernel/security 2024-04-12T20:52:54.219 INFO:teuthology.orchestra.run.smithi186.stdout:/dev/shm 2024-04-12T20:52:54.219 INFO:teuthology.orchestra.run.smithi186.stdout:/dev/pts 2024-04-12T20:52:54.219 INFO:teuthology.orchestra.run.smithi186.stdout:/run 2024-04-12T20:52:54.219 INFO:teuthology.orchestra.run.smithi186.stdout:/sys/fs/cgroup 2024-04-12T20:52:54.219 INFO:teuthology.orchestra.run.smithi186.stdout:/sys/fs/pstore 2024-04-12T20:52:54.219 INFO:teuthology.orchestra.run.smithi186.stdout:/sys/fs/bpf 2024-04-12T20:52:54.219 INFO:teuthology.orchestra.run.smithi186.stdout:/ 2024-04-12T20:52:54.219 INFO:teuthology.orchestra.run.smithi186.stdout:/sys/fs/selinux 2024-04-12T20:52:54.219 INFO:teuthology.orchestra.run.smithi186.stdout:/proc/sys/fs/binfmt_misc 2024-04-12T20:52:54.219 INFO:teuthology.orchestra.run.smithi186.stdout:/dev/mqueue 2024-04-12T20:52:54.219 INFO:teuthology.orchestra.run.smithi186.stdout:/dev/hugepages 2024-04-12T20:52:54.219 INFO:teuthology.orchestra.run.smithi186.stdout:/sys/kernel/debug 2024-04-12T20:52:54.219 INFO:teuthology.orchestra.run.smithi186.stdout:/sys/kernel/tracing 2024-04-12T20:52:54.220 INFO:teuthology.orchestra.run.smithi186.stdout:/sys/fs/fuse/connections 2024-04-12T20:52:54.220 INFO:teuthology.orchestra.run.smithi186.stdout:/sys/kernel/config 2024-04-12T20:52:54.220 INFO:teuthology.orchestra.run.smithi186.stdout:/run/credentials/systemd-sysctl.service 2024-04-12T20:52:54.220 INFO:teuthology.orchestra.run.smithi186.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-12T20:52:54.220 INFO:teuthology.orchestra.run.smithi186.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-12T20:52:54.220 INFO:teuthology.orchestra.run.smithi186.stdout:/var/lib/nfs/rpc_pipefs 2024-04-12T20:52:54.220 INFO:teuthology.orchestra.run.smithi186.stdout:/run/user/1000 2024-04-12T20:52:54.220 INFO:teuthology.orchestra.run.smithi186.stdout:/var/lib/ceph 2024-04-12T20:52:54.220 INFO:teuthology.orchestra.run.smithi186.stdout:/run/user/0 2024-04-12T20:52:54.220 INFO:teuthology.orchestra.run.smithi186.stdout:/proc/sys/fs/binfmt_misc 2024-04-12T20:52:54.220 INFO:teuthology.orchestra.run.smithi186.stdout:/var/lib/containers/storage/overlay 2024-04-12T20:52:54.220 INFO:teuthology.orchestra.run.smithi186.stdout:/var/lib/containers/storage/overlay/87dc767b69aa91b4f8cbdf8139ec13e1db8bb517ba28a6649bea8a0373ad383c/merged 2024-04-12T20:52:54.220 INFO:teuthology.orchestra.run.smithi186.stdout:/var/lib/containers/storage/overlay/aa240d65e1d8e5509a2cdaa303816fc3c8c6f5e1257902fd619c10aa49486e96/merged 2024-04-12T20:52:54.221 INFO:teuthology.orchestra.run.smithi186.stdout:/var/lib/containers/storage/overlay/27fe6211ed2e375c10007ee138a5d85ebf2360f4d5bb472e0822d4de2d7e0667/merged 2024-04-12T20:52:54.221 INFO:teuthology.orchestra.run.smithi186.stdout:/var/lib/containers/storage/overlay/29e0504e1742d861bab3b8c6ade6c2ff618765efe514a64a92fc44d346d6f02e/merged 2024-04-12T20:52:54.221 INFO:teuthology.orchestra.run.smithi186.stdout:/var/lib/containers/storage/overlay/50dcf6f17c443229f3c680a040042b4932ceba22d244974122d772600e9b0287/merged 2024-04-12T20:52:54.221 INFO:teuthology.orchestra.run.smithi186.stdout:/var/lib/containers/storage/overlay/b09826eb97bd3f527d6a3eefd54e85ab04ac95e897ad33b0acc5c46747beaf7b/merged 2024-04-12T20:52:54.221 INFO:teuthology.orchestra.run.smithi186.stdout:/var/lib/containers/storage/overlay/237d19265be4f6aacf9dcfa82804b3f8ebb8057de8759f1395c1ac23cc534572/merged 2024-04-12T20:52:54.221 INFO:teuthology.orchestra.run.smithi186.stdout:/var/lib/containers/storage/overlay/8ee8aee72db28a01e0e40f791e8c4571845f0d12d739beec0ff488435441f467/merged 2024-04-12T20:52:54.221 INFO:teuthology.orchestra.run.smithi186.stdout:/var/lib/containers/storage/overlay/080b1947d86b67befe994ad9bcba0168ff28d5b23879da2be9dca51f68476e28/merged 2024-04-12T20:52:54.221 INFO:teuthology.orchestra.run.smithi186.stdout:/run/netns 2024-04-12T20:52:54.221 INFO:teuthology.orchestra.run.smithi186.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-12T20:52:54.221 INFO:teuthology.orchestra.run.smithi186.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-12T20:52:54.222 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-12T20:52:54.222 DEBUG:teuthology.orchestra.run.smithi186:> ls /sys/fs/fuse/connections 2024-04-12T20:52:54.223 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:52:54 smithi186 ceph-mon[36597]: pgmap v248: 65 pgs: 65 active+clean; 581 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 341 B/s wr, 0 op/s 2024-04-12T20:52:54.325 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi186.stderr:2024-04-12T20:52:54.326+0000 7f73c44cb300 -1 init, newargv = 0x555c9b362e50 newargc=15 2024-04-12T20:52:54.325 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi186.stderr:ceph-fuse[61757]: starting ceph client 2024-04-12T20:52:54.344 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi186.stderr:ceph-fuse[61757]: starting fuse 2024-04-12T20:52:54.352 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:52:54 smithi018 ceph-mon[28799]: pgmap v248: 65 pgs: 65 active+clean; 581 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 341 B/s wr, 0 op/s 2024-04-12T20:52:55.413 DEBUG:teuthology.orchestra.run.smithi186:> sudo modprobe fuse 2024-04-12T20:52:55.519 DEBUG:teuthology.orchestra.run.smithi186:> cat /proc/self/mounts | awk '{print $2}' 2024-04-12T20:52:55.577 INFO:teuthology.orchestra.run.smithi186.stdout:/proc 2024-04-12T20:52:55.577 INFO:teuthology.orchestra.run.smithi186.stdout:/sys 2024-04-12T20:52:55.577 INFO:teuthology.orchestra.run.smithi186.stdout:/dev 2024-04-12T20:52:55.577 INFO:teuthology.orchestra.run.smithi186.stdout:/sys/kernel/security 2024-04-12T20:52:55.577 INFO:teuthology.orchestra.run.smithi186.stdout:/dev/shm 2024-04-12T20:52:55.577 INFO:teuthology.orchestra.run.smithi186.stdout:/dev/pts 2024-04-12T20:52:55.577 INFO:teuthology.orchestra.run.smithi186.stdout:/run 2024-04-12T20:52:55.577 INFO:teuthology.orchestra.run.smithi186.stdout:/sys/fs/cgroup 2024-04-12T20:52:55.577 INFO:teuthology.orchestra.run.smithi186.stdout:/sys/fs/pstore 2024-04-12T20:52:55.577 INFO:teuthology.orchestra.run.smithi186.stdout:/sys/fs/bpf 2024-04-12T20:52:55.577 INFO:teuthology.orchestra.run.smithi186.stdout:/ 2024-04-12T20:52:55.577 INFO:teuthology.orchestra.run.smithi186.stdout:/sys/fs/selinux 2024-04-12T20:52:55.578 INFO:teuthology.orchestra.run.smithi186.stdout:/proc/sys/fs/binfmt_misc 2024-04-12T20:52:55.578 INFO:teuthology.orchestra.run.smithi186.stdout:/dev/mqueue 2024-04-12T20:52:55.578 INFO:teuthology.orchestra.run.smithi186.stdout:/dev/hugepages 2024-04-12T20:52:55.578 INFO:teuthology.orchestra.run.smithi186.stdout:/sys/kernel/debug 2024-04-12T20:52:55.578 INFO:teuthology.orchestra.run.smithi186.stdout:/sys/kernel/tracing 2024-04-12T20:52:55.578 INFO:teuthology.orchestra.run.smithi186.stdout:/sys/fs/fuse/connections 2024-04-12T20:52:55.578 INFO:teuthology.orchestra.run.smithi186.stdout:/sys/kernel/config 2024-04-12T20:52:55.578 INFO:teuthology.orchestra.run.smithi186.stdout:/run/credentials/systemd-sysctl.service 2024-04-12T20:52:55.578 INFO:teuthology.orchestra.run.smithi186.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-12T20:52:55.578 INFO:teuthology.orchestra.run.smithi186.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-12T20:52:55.578 INFO:teuthology.orchestra.run.smithi186.stdout:/var/lib/nfs/rpc_pipefs 2024-04-12T20:52:55.578 INFO:teuthology.orchestra.run.smithi186.stdout:/run/user/1000 2024-04-12T20:52:55.578 INFO:teuthology.orchestra.run.smithi186.stdout:/var/lib/ceph 2024-04-12T20:52:55.578 INFO:teuthology.orchestra.run.smithi186.stdout:/run/user/0 2024-04-12T20:52:55.578 INFO:teuthology.orchestra.run.smithi186.stdout:/proc/sys/fs/binfmt_misc 2024-04-12T20:52:55.579 INFO:teuthology.orchestra.run.smithi186.stdout:/var/lib/containers/storage/overlay 2024-04-12T20:52:55.579 INFO:teuthology.orchestra.run.smithi186.stdout:/var/lib/containers/storage/overlay/87dc767b69aa91b4f8cbdf8139ec13e1db8bb517ba28a6649bea8a0373ad383c/merged 2024-04-12T20:52:55.579 INFO:teuthology.orchestra.run.smithi186.stdout:/var/lib/containers/storage/overlay/aa240d65e1d8e5509a2cdaa303816fc3c8c6f5e1257902fd619c10aa49486e96/merged 2024-04-12T20:52:55.579 INFO:teuthology.orchestra.run.smithi186.stdout:/var/lib/containers/storage/overlay/27fe6211ed2e375c10007ee138a5d85ebf2360f4d5bb472e0822d4de2d7e0667/merged 2024-04-12T20:52:55.579 INFO:teuthology.orchestra.run.smithi186.stdout:/var/lib/containers/storage/overlay/29e0504e1742d861bab3b8c6ade6c2ff618765efe514a64a92fc44d346d6f02e/merged 2024-04-12T20:52:55.579 INFO:teuthology.orchestra.run.smithi186.stdout:/var/lib/containers/storage/overlay/50dcf6f17c443229f3c680a040042b4932ceba22d244974122d772600e9b0287/merged 2024-04-12T20:52:55.579 INFO:teuthology.orchestra.run.smithi186.stdout:/var/lib/containers/storage/overlay/b09826eb97bd3f527d6a3eefd54e85ab04ac95e897ad33b0acc5c46747beaf7b/merged 2024-04-12T20:52:55.579 INFO:teuthology.orchestra.run.smithi186.stdout:/var/lib/containers/storage/overlay/237d19265be4f6aacf9dcfa82804b3f8ebb8057de8759f1395c1ac23cc534572/merged 2024-04-12T20:52:55.579 INFO:teuthology.orchestra.run.smithi186.stdout:/var/lib/containers/storage/overlay/8ee8aee72db28a01e0e40f791e8c4571845f0d12d739beec0ff488435441f467/merged 2024-04-12T20:52:55.579 INFO:teuthology.orchestra.run.smithi186.stdout:/var/lib/containers/storage/overlay/080b1947d86b67befe994ad9bcba0168ff28d5b23879da2be9dca51f68476e28/merged 2024-04-12T20:52:55.579 INFO:teuthology.orchestra.run.smithi186.stdout:/run/netns 2024-04-12T20:52:55.579 INFO:teuthology.orchestra.run.smithi186.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-12T20:52:55.579 INFO:teuthology.orchestra.run.smithi186.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-12T20:52:55.579 INFO:teuthology.orchestra.run.smithi186.stdout:/home/ubuntu/cephtest/mnt.1 2024-04-12T20:52:55.580 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-12T20:52:55.580 DEBUG:teuthology.orchestra.run.smithi186:> ls /sys/fs/fuse/connections 2024-04-12T20:52:55.633 INFO:teuthology.orchestra.run.smithi186.stdout:115 2024-04-12T20:52:55.633 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [115] 2024-04-12T20:52:55.634 DEBUG:teuthology.orchestra.run.smithi186:> sudo stdin-killer -- python3 -c ' 2024-04-12T20:52:55.634 DEBUG:teuthology.orchestra.run.smithi186:> import glob 2024-04-12T20:52:55.634 DEBUG:teuthology.orchestra.run.smithi186:> import re 2024-04-12T20:52:55.634 DEBUG:teuthology.orchestra.run.smithi186:> import os 2024-04-12T20:52:55.634 DEBUG:teuthology.orchestra.run.smithi186:> import subprocess 2024-04-12T20:52:55.634 DEBUG:teuthology.orchestra.run.smithi186:> 2024-04-12T20:52:55.634 DEBUG:teuthology.orchestra.run.smithi186:> def _find_admin_socket(client_name): 2024-04-12T20:52:55.634 DEBUG:teuthology.orchestra.run.smithi186:> asok_path = "/var/run/ceph/ceph-client.1.*.asok" 2024-04-12T20:52:55.634 DEBUG:teuthology.orchestra.run.smithi186:> files = glob.glob(asok_path) 2024-04-12T20:52:55.634 DEBUG:teuthology.orchestra.run.smithi186:> mountpoint = "/home/ubuntu/cephtest/mnt.1" 2024-04-12T20:52:55.634 DEBUG:teuthology.orchestra.run.smithi186:> 2024-04-12T20:52:55.634 DEBUG:teuthology.orchestra.run.smithi186:> # Given a non-glob path, it better be there 2024-04-12T20:52:55.634 DEBUG:teuthology.orchestra.run.smithi186:> if "*" not in asok_path: 2024-04-12T20:52:55.634 DEBUG:teuthology.orchestra.run.smithi186:> assert(len(files) == 1) 2024-04-12T20:52:55.635 DEBUG:teuthology.orchestra.run.smithi186:> return files[0] 2024-04-12T20:52:55.635 DEBUG:teuthology.orchestra.run.smithi186:> 2024-04-12T20:52:55.635 DEBUG:teuthology.orchestra.run.smithi186:> for f in files: 2024-04-12T20:52:55.635 DEBUG:teuthology.orchestra.run.smithi186:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-04-12T20:52:55.635 DEBUG:teuthology.orchestra.run.smithi186:> if os.path.exists("/proc/{0}".format(pid)): 2024-04-12T20:52:55.635 DEBUG:teuthology.orchestra.run.smithi186:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-04-12T20:52:55.635 DEBUG:teuthology.orchestra.run.smithi186:> contents = proc_f.read() 2024-04-12T20:52:55.635 DEBUG:teuthology.orchestra.run.smithi186:> if mountpoint in contents: 2024-04-12T20:52:55.635 DEBUG:teuthology.orchestra.run.smithi186:> return f 2024-04-12T20:52:55.635 DEBUG:teuthology.orchestra.run.smithi186:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-04-12T20:52:55.635 DEBUG:teuthology.orchestra.run.smithi186:> 2024-04-12T20:52:55.635 DEBUG:teuthology.orchestra.run.smithi186:> print(_find_admin_socket("client.1")) 2024-04-12T20:52:55.635 DEBUG:teuthology.orchestra.run.smithi186:> ' 2024-04-12T20:52:55.809 INFO:teuthology.orchestra.run.smithi186.stdout:/var/run/ceph/ceph-client.1.61757.asok 2024-04-12T20:52:55.813 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-12T20:52:55 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-12T20:52:55.827 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.1.61757.asok 2024-04-12T20:52:55.827 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-12T20:52:55.827 DEBUG:teuthology.orchestra.run.smithi186:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.1.61757.asok status 2024-04-12T20:52:56.059 INFO:teuthology.orchestra.run.smithi186.stdout:{ 2024-04-12T20:52:56.059 INFO:teuthology.orchestra.run.smithi186.stdout: "metadata": { 2024-04-12T20:52:56.059 INFO:teuthology.orchestra.run.smithi186.stdout: "ceph_sha1": "d3b5060a17449edfae9385cdf792afec109fca5d", 2024-04-12T20:52:56.059 INFO:teuthology.orchestra.run.smithi186.stdout: "ceph_version": "ceph version 17.2.7-827-gd3b5060a (d3b5060a17449edfae9385cdf792afec109fca5d) quincy (stable)", 2024-04-12T20:52:56.059 INFO:teuthology.orchestra.run.smithi186.stdout: "entity_id": "1", 2024-04-12T20:52:56.059 INFO:teuthology.orchestra.run.smithi186.stdout: "hostname": "smithi186", 2024-04-12T20:52:56.060 INFO:teuthology.orchestra.run.smithi186.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.1", 2024-04-12T20:52:56.060 INFO:teuthology.orchestra.run.smithi186.stdout: "pid": "61757", 2024-04-12T20:52:56.060 INFO:teuthology.orchestra.run.smithi186.stdout: "root": "/" 2024-04-12T20:52:56.060 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-12T20:52:56.060 INFO:teuthology.orchestra.run.smithi186.stdout: "dentry_count": 0, 2024-04-12T20:52:56.060 INFO:teuthology.orchestra.run.smithi186.stdout: "dentry_pinned_count": 0, 2024-04-12T20:52:56.060 INFO:teuthology.orchestra.run.smithi186.stdout: "id": 24313, 2024-04-12T20:52:56.060 INFO:teuthology.orchestra.run.smithi186.stdout: "inst": { 2024-04-12T20:52:56.060 INFO:teuthology.orchestra.run.smithi186.stdout: "name": { 2024-04-12T20:52:56.060 INFO:teuthology.orchestra.run.smithi186.stdout: "type": "client", 2024-04-12T20:52:56.060 INFO:teuthology.orchestra.run.smithi186.stdout: "num": 24313 2024-04-12T20:52:56.060 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-12T20:52:56.060 INFO:teuthology.orchestra.run.smithi186.stdout: "addr": { 2024-04-12T20:52:56.060 INFO:teuthology.orchestra.run.smithi186.stdout: "type": "v1", 2024-04-12T20:52:56.061 INFO:teuthology.orchestra.run.smithi186.stdout: "addr": "192.168.0.1:0", 2024-04-12T20:52:56.061 INFO:teuthology.orchestra.run.smithi186.stdout: "nonce": 2102418092 2024-04-12T20:52:56.061 INFO:teuthology.orchestra.run.smithi186.stdout: } 2024-04-12T20:52:56.061 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-12T20:52:56.061 INFO:teuthology.orchestra.run.smithi186.stdout: "addr": { 2024-04-12T20:52:56.061 INFO:teuthology.orchestra.run.smithi186.stdout: "type": "v1", 2024-04-12T20:52:56.061 INFO:teuthology.orchestra.run.smithi186.stdout: "addr": "192.168.0.1:0", 2024-04-12T20:52:56.061 INFO:teuthology.orchestra.run.smithi186.stdout: "nonce": 2102418092 2024-04-12T20:52:56.061 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-12T20:52:56.061 INFO:teuthology.orchestra.run.smithi186.stdout: "inst_str": "client.24313 192.168.0.1:0/2102418092", 2024-04-12T20:52:56.061 INFO:teuthology.orchestra.run.smithi186.stdout: "addr_str": "192.168.0.1:0/2102418092", 2024-04-12T20:52:56.061 INFO:teuthology.orchestra.run.smithi186.stdout: "inode_count": 1, 2024-04-12T20:52:56.061 INFO:teuthology.orchestra.run.smithi186.stdout: "mds_epoch": 19, 2024-04-12T20:52:56.061 INFO:teuthology.orchestra.run.smithi186.stdout: "osd_epoch": 46, 2024-04-12T20:52:56.062 INFO:teuthology.orchestra.run.smithi186.stdout: "osd_epoch_barrier": 0, 2024-04-12T20:52:56.062 INFO:teuthology.orchestra.run.smithi186.stdout: "blocklisted": false, 2024-04-12T20:52:56.062 INFO:teuthology.orchestra.run.smithi186.stdout: "fs_name": "cephfs" 2024-04-12T20:52:56.062 INFO:teuthology.orchestra.run.smithi186.stdout:} 2024-04-12T20:52:56.074 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-12T20:52:56.074 DEBUG:teuthology.orchestra.run.smithi018:> stat --file-system '--printf=%T 2024-04-12T20:52:56.074 DEBUG:teuthology.orchestra.run.smithi018:> ' -- /home/ubuntu/cephtest/mnt.0 2024-04-12T20:52:56.142 INFO:teuthology.orchestra.run.smithi018.stdout:fuseblk 2024-04-12T20:52:56.142 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-04-12T20:52:56.142 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-12T20:52:56.142 DEBUG:teuthology.orchestra.run.smithi018:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-12T20:52:56.257 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-12T20:52:56.257 DEBUG:teuthology.orchestra.run.smithi186:> stat --file-system '--printf=%T 2024-04-12T20:52:56.257 DEBUG:teuthology.orchestra.run.smithi186:> ' -- /home/ubuntu/cephtest/mnt.1 2024-04-12T20:52:56.280 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:52:56 smithi186 ceph-mon[36597]: pgmap v249: 65 pgs: 65 active+clean; 581 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 341 B/s wr, 0 op/s 2024-04-12T20:52:56.317 INFO:teuthology.orchestra.run.smithi186.stdout:fuseblk 2024-04-12T20:52:56.317 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2024-04-12T20:52:56.317 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-12T20:52:56.318 DEBUG:teuthology.orchestra.run.smithi186:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-04-12T20:52:56.352 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:52:56 smithi018 ceph-mon[28799]: pgmap v249: 65 pgs: 65 active+clean; 581 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 341 B/s wr, 0 op/s 2024-04-12T20:52:57.358 INFO:teuthology.run_tasks:Running task print... 2024-04-12T20:52:57.370 INFO:teuthology.task.print:**** done client 2024-04-12T20:52:57.370 INFO:teuthology.run_tasks:Running task parallel... 2024-04-12T20:52:57.384 INFO:teuthology.task.parallel:starting parallel... 2024-04-12T20:52:57.384 INFO:teuthology.task.parallel:In parallel, running task sequential... 2024-04-12T20:52:57.385 INFO:teuthology.task.sequential:In sequential, running task cephadm.shell... 2024-04-12T20:52:57.385 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi018.front.sepia.ceph.com 2024-04-12T20:52:57.386 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1cbe407a-f90d-11ee-bc8d-c7b262605968 -e sha1=324baf8ddb91dea46cac50cb535c306115791355 -- bash -c 'ceph config set mon mon_warn_on_insecure_global_id_reclaim false --force' 2024-04-12T20:52:57.386 INFO:teuthology.task.parallel:In parallel, running task sequential... 2024-04-12T20:52:57.386 INFO:teuthology.task.sequential:In sequential, running task workunit... 2024-04-12T20:52:57.390 INFO:tasks.workunit:Pulling workunits from ref 324baf8ddb91dea46cac50cb535c306115791355 2024-04-12T20:52:57.390 INFO:tasks.workunit:Making a separate scratch dir for every client... 2024-04-12T20:52:57.391 INFO:tasks.workunit:timeout=3h 2024-04-12T20:52:57.391 INFO:tasks.workunit:cleanup=True 2024-04-12T20:52:57.391 DEBUG:teuthology.orchestra.run.smithi018:> stat -- /home/ubuntu/cephtest/mnt.0 2024-04-12T20:52:57.452 INFO:teuthology.orchestra.run.smithi018.stdout: File: /home/ubuntu/cephtest/mnt.0 2024-04-12T20:52:57.452 INFO:teuthology.orchestra.run.smithi018.stdout: Size: 0 Blocks: 1 IO Block: 4194304 directory 2024-04-12T20:52:57.452 INFO:teuthology.orchestra.run.smithi018.stdout:Device: 94h/148d Inode: 1 Links: 2 2024-04-12T20:52:57.452 INFO:teuthology.orchestra.run.smithi018.stdout:Access: (1777/drwxrwxrwt) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-12T20:52:57.452 INFO:teuthology.orchestra.run.smithi018.stdout:Context: system_u:object_r:fusefs_t:s0 2024-04-12T20:52:57.452 INFO:teuthology.orchestra.run.smithi018.stdout:Access: 1970-01-01 00:00:00.000000000 +0000 2024-04-12T20:52:57.453 INFO:teuthology.orchestra.run.smithi018.stdout:Modify: 2024-04-12 20:52:02.422373755 +0000 2024-04-12T20:52:57.453 INFO:teuthology.orchestra.run.smithi018.stdout:Change: 2024-04-12 20:52:56.251279847 +0000 2024-04-12T20:52:57.453 INFO:teuthology.orchestra.run.smithi018.stdout: Birth: - 2024-04-12T20:52:57.453 INFO:tasks.workunit:Did not need to create dir /home/ubuntu/cephtest/mnt.0 2024-04-12T20:52:57.453 DEBUG:teuthology.orchestra.run.smithi018:> cd -- /home/ubuntu/cephtest/mnt.0 && sudo install -d -m 0755 --owner=ubuntu -- client.0 2024-04-12T20:52:57.593 DEBUG:teuthology.orchestra.run.smithi186:> stat -- /home/ubuntu/cephtest/mnt.1 2024-04-12T20:52:57.661 INFO:teuthology.orchestra.run.smithi186.stdout: File: /home/ubuntu/cephtest/mnt.1 2024-04-12T20:52:57.661 INFO:teuthology.orchestra.run.smithi186.stdout: Size: 0 Blocks: 1 IO Block: 4194304 directory 2024-04-12T20:52:57.662 INFO:teuthology.orchestra.run.smithi186.stdout:Device: 73h/115d Inode: 1 Links: 3 2024-04-12T20:52:57.662 INFO:teuthology.orchestra.run.smithi186.stdout:Access: (1777/drwxrwxrwt) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-12T20:52:57.662 INFO:teuthology.orchestra.run.smithi186.stdout:Context: system_u:object_r:fusefs_t:s0 2024-04-12T20:52:57.662 INFO:teuthology.orchestra.run.smithi186.stdout:Access: 1970-01-01 00:00:00.000000000 +0000 2024-04-12T20:52:57.662 INFO:teuthology.orchestra.run.smithi186.stdout:Modify: 2024-04-12 20:52:57.580245296 +0000 2024-04-12T20:52:57.662 INFO:teuthology.orchestra.run.smithi186.stdout:Change: 2024-04-12 20:52:57.580245296 +0000 2024-04-12T20:52:57.662 INFO:teuthology.orchestra.run.smithi186.stdout: Birth: - 2024-04-12T20:52:57.662 INFO:tasks.workunit:Did not need to create dir /home/ubuntu/cephtest/mnt.1 2024-04-12T20:52:57.662 DEBUG:teuthology.orchestra.run.smithi186:> cd -- /home/ubuntu/cephtest/mnt.1 && sudo install -d -m 0755 --owner=ubuntu -- client.1 2024-04-12T20:52:57.771 DEBUG:teuthology.orchestra.run.smithi018:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone https://git.ceph.com/ceph-ci.git /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout 324baf8ddb91dea46cac50cb535c306115791355 2024-04-12T20:52:57.771 DEBUG:teuthology.orchestra.run.smithi186:> rm -rf /home/ubuntu/cephtest/clone.client.1 && git clone https://git.ceph.com/ceph-ci.git /home/ubuntu/cephtest/clone.client.1 && cd /home/ubuntu/cephtest/clone.client.1 && git checkout 324baf8ddb91dea46cac50cb535c306115791355 2024-04-12T20:52:57.817 INFO:tasks.workunit.client.0.smithi018.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2024-04-12T20:52:57.818 INFO:tasks.workunit.client.1.smithi186.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.1'... 2024-04-12T20:52:57.870 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/mon.smithi018/config 2024-04-12T20:52:58.281 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:52:58 smithi186 ceph-mon[36597]: pgmap v250: 65 pgs: 65 active+clean; 582 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 511 B/s wr, 0 op/s 2024-04-12T20:52:58.297 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:52:58 smithi018 ceph-mon[28799]: pgmap v250: 65 pgs: 65 active+clean; 582 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 511 B/s wr, 0 op/s 2024-04-12T20:53:00.251 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:53:00.251+0000 7f92fec00700 1 -- 172.21.15.18:0/3696025139 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f92f80fe4c0 msgr2=0x7f92f80fe8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:53:00.251 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:53:00.251+0000 7f92fec00700 1 --2- 172.21.15.18:0/3696025139 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f92f80fe4c0 0x7f92f80fe8d0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f92e8004940 tx=0x7f92e8009e30 comp rx=0 tx=0).stop 2024-04-12T20:53:00.251 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:53:00.252+0000 7f92fec00700 1 -- 172.21.15.18:0/3696025139 shutdown_connections 2024-04-12T20:53:00.251 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:53:00.252+0000 7f92fec00700 1 --2- 172.21.15.18:0/3696025139 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f92f80fee10 0x7f92f80faf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:53:00.251 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:53:00.252+0000 7f92fec00700 1 --2- 172.21.15.18:0/3696025139 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f92f80fe4c0 0x7f92f80fe8d0 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:53:00.251 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:53:00.252+0000 7f92fec00700 1 -- 172.21.15.18:0/3696025139 >> 172.21.15.18:0/3696025139 conn(0x7f92f80f6ba0 msgr2=0x7f92f80f8fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T20:53:00.251 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:53:00.252+0000 7f92fec00700 1 -- 172.21.15.18:0/3696025139 shutdown_connections 2024-04-12T20:53:00.252 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:53:00.252+0000 7f92fec00700 1 -- 172.21.15.18:0/3696025139 wait complete. 2024-04-12T20:53:00.252 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:53:00.253+0000 7f92fec00700 1 Processor -- start 2024-04-12T20:53:00.252 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:53:00.253+0000 7f92fec00700 1 -- start start 2024-04-12T20:53:00.252 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:53:00.253+0000 7f92fec00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f92f80fe4c0 0x7f92f810fc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T20:53:00.252 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:53:00.253+0000 7f92fec00700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f92f80fee10 0x7f92f8110160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T20:53:00.252 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:53:00.253+0000 7f92fec00700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92f8110760 con 0x7f92f80fe4c0 2024-04-12T20:53:00.252 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:53:00.253+0000 7f92fec00700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92f8110fb0 con 0x7f92f80fee10 2024-04-12T20:53:00.252 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:53:00.253+0000 7f92f7e00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f92f80fe4c0 0x7f92f810fc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T20:53:00.252 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:53:00.253+0000 7f92f7e00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f92f80fe4c0 0x7f92f810fc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:47342/0 (socket says 172.21.15.18:47342) 2024-04-12T20:53:00.252 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:53:00.253+0000 7f92f7e00700 1 -- 172.21.15.18:0/3090195141 learned_addr learned my addr 172.21.15.18:0/3090195141 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T20:53:00.253 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:53:00.254+0000 7f92f7e00700 1 -- 172.21.15.18:0/3090195141 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f92f80fee10 msgr2=0x7f92f8110160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:53:00.253 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:53:00.254+0000 7f92f7400700 1 --2- 172.21.15.18:0/3090195141 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f92f80fee10 0x7f92f8110160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T20:53:00.253 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:53:00.254+0000 7f92f7e00700 1 --2- 172.21.15.18:0/3090195141 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f92f80fee10 0x7f92f8110160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:53:00.253 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:53:00.254+0000 7f92f7e00700 1 -- 172.21.15.18:0/3090195141 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92e8005040 con 0x7f92f80fe4c0 2024-04-12T20:53:00.253 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:53:00.254+0000 7f92f7e00700 1 --2- 172.21.15.18:0/3090195141 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f92f80fe4c0 0x7f92f810fc20 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f92e800b3b0 tx=0x7f92e8009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T20:53:00.253 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:53:00.255+0000 7f92efe00700 1 -- 172.21.15.18:0/3090195141 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f92e800e040 con 0x7f92f80fe4c0 2024-04-12T20:53:00.253 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:53:00.255+0000 7f92efe00700 1 -- 172.21.15.18:0/3090195141 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f92e8004290 con 0x7f92f80fe4c0 2024-04-12T20:53:00.254 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:53:00.255+0000 7f92fec00700 1 -- 172.21.15.18:0/3090195141 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f92f8111230 con 0x7f92f80fe4c0 2024-04-12T20:53:00.254 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:53:00.255+0000 7f92efe00700 1 -- 172.21.15.18:0/3090195141 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f92e801c630 con 0x7f92f80fe4c0 2024-04-12T20:53:00.254 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:53:00.255+0000 7f92fec00700 1 -- 172.21.15.18:0/3090195141 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f92f8111770 con 0x7f92f80fe4c0 2024-04-12T20:53:00.254 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:53:00.256+0000 7f92fec00700 1 -- 172.21.15.18:0/3090195141 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f92f8101c00 con 0x7f92f80fe4c0 2024-04-12T20:53:00.255 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:53:00.257+0000 7f92efe00700 1 -- 172.21.15.18:0/3090195141 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7f92e800bbf0 con 0x7f92f80fe4c0 2024-04-12T20:53:00.256 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:53:00.259+0000 7f92efe00700 1 --2- 172.21.15.18:0/3090195141 >> [v2:172.21.15.18:6800/3048894687,v1:172.21.15.18:6801/3048894687] conn(0x7f92dc0692b0 0x7f92dc06b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T20:53:00.257 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:53:00.259+0000 7f92efe00700 1 -- 172.21.15.18:0/3090195141 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5550+0+0 (secure 0 0 0) 0x7f92e80886f0 con 0x7f92f80fe4c0 2024-04-12T20:53:00.257 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:53:00.259+0000 7f92f7400700 1 --2- 172.21.15.18:0/3090195141 >> [v2:172.21.15.18:6800/3048894687,v1:172.21.15.18:6801/3048894687] conn(0x7f92dc0692b0 0x7f92dc06b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T20:53:00.257 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:53:00.260+0000 7f92f7400700 1 --2- 172.21.15.18:0/3090195141 >> [v2:172.21.15.18:6800/3048894687,v1:172.21.15.18:6801/3048894687] conn(0x7f92dc0692b0 0x7f92dc06b760 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f92e4004d60 tx=0x7f92e400b040 comp rx=0 tx=0).ready entity=mgr.14212 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T20:53:00.261 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:53:00.264+0000 7f92efe00700 1 -- 172.21.15.18:0/3090195141 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7f92e80583c0 con 0x7f92f80fe4c0 2024-04-12T20:53:00.281 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:53:00 smithi186 ceph-mon[36597]: pgmap v251: 65 pgs: 65 active+clean; 582 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 341 B/s wr, 0 op/s 2024-04-12T20:53:00.281 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:53:00 smithi186 ceph-mon[36597]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T20:53:00.293 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:53:00 smithi018 ceph-mon[28799]: pgmap v251: 65 pgs: 65 active+clean; 582 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 341 B/s wr, 0 op/s 2024-04-12T20:53:00.293 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:53:00 smithi018 ceph-mon[28799]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T20:53:00.481 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:53:00.483+0000 7f92fec00700 1 -- 172.21.15.18:0/3090195141 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command([{prefix=config set, name=mon_warn_on_insecure_global_id_reclaim}] v 0) v1 -- 0x7f92f81113c0 con 0x7f92f80fe4c0 2024-04-12T20:53:00.482 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:53:00.484+0000 7f92efe00700 1 -- 172.21.15.18:0/3090195141 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_warn_on_insecure_global_id_reclaim}]=0 v21)=0 v21) v1 ==== 155+0+0 (secure 0 0 0) 0x7f92f81113c0 con 0x7f92f80fe4c0 2024-04-12T20:53:00.484 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:53:00.487+0000 7f92fec00700 1 -- 172.21.15.18:0/3090195141 >> [v2:172.21.15.18:6800/3048894687,v1:172.21.15.18:6801/3048894687] conn(0x7f92dc0692b0 msgr2=0x7f92dc06b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:53:00.484 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:53:00.487+0000 7f92fec00700 1 --2- 172.21.15.18:0/3090195141 >> [v2:172.21.15.18:6800/3048894687,v1:172.21.15.18:6801/3048894687] conn(0x7f92dc0692b0 0x7f92dc06b760 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f92e4004d60 tx=0x7f92e400b040 comp rx=0 tx=0).stop 2024-04-12T20:53:00.484 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:53:00.487+0000 7f92fec00700 1 -- 172.21.15.18:0/3090195141 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f92f80fe4c0 msgr2=0x7f92f810fc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:53:00.485 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:53:00.487+0000 7f92fec00700 1 --2- 172.21.15.18:0/3090195141 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f92f80fe4c0 0x7f92f810fc20 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f92e800b3b0 tx=0x7f92e8009e30 comp rx=0 tx=0).stop 2024-04-12T20:53:00.485 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:53:00.487+0000 7f92fec00700 1 -- 172.21.15.18:0/3090195141 shutdown_connections 2024-04-12T20:53:00.485 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:53:00.487+0000 7f92fec00700 1 --2- 172.21.15.18:0/3090195141 >> [v2:172.21.15.18:6800/3048894687,v1:172.21.15.18:6801/3048894687] conn(0x7f92dc0692b0 0x7f92dc06b760 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:53:00.485 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:53:00.487+0000 7f92fec00700 1 --2- 172.21.15.18:0/3090195141 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f92f80fe4c0 0x7f92f810fc20 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:53:00.485 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:53:00.487+0000 7f92fec00700 1 --2- 172.21.15.18:0/3090195141 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f92f80fee10 0x7f92f8110160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:53:00.486 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:53:00.487+0000 7f92fec00700 1 -- 172.21.15.18:0/3090195141 >> 172.21.15.18:0/3090195141 conn(0x7f92f80f6ba0 msgr2=0x7f92f80f82e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T20:53:00.486 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:53:00.487+0000 7f92fec00700 1 -- 172.21.15.18:0/3090195141 shutdown_connections 2024-04-12T20:53:00.486 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:53:00.488+0000 7f92fec00700 1 -- 172.21.15.18:0/3090195141 wait complete. 2024-04-12T20:53:01.465 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1cbe407a-f90d-11ee-bc8d-c7b262605968 -e sha1=324baf8ddb91dea46cac50cb535c306115791355 -- bash -c 'ceph config set mon mon_warn_on_insecure_global_id_reclaim_allowed false --force' 2024-04-12T20:53:01.992 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/mon.smithi018/config 2024-04-12T20:53:02.281 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:53:02 smithi186 ceph-mon[36597]: pgmap v252: 65 pgs: 65 active+clean; 582 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 341 B/s wr, 0 op/s 2024-04-12T20:53:02.352 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:53:02 smithi018 ceph-mon[28799]: pgmap v252: 65 pgs: 65 active+clean; 582 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 341 B/s wr, 0 op/s 2024-04-12T20:53:04.285 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:53:04 smithi018 ceph-mon[28799]: pgmap v253: 65 pgs: 65 active+clean; 587 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 682 B/s wr, 0 op/s 2024-04-12T20:53:04.531 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:53:04 smithi186 ceph-mon[36597]: pgmap v253: 65 pgs: 65 active+clean; 587 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 682 B/s wr, 0 op/s 2024-04-12T20:53:04.649 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:53:04.651+0000 7fdd5b200700 1 -- 172.21.15.18:0/976760500 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fdd5c104500 msgr2=0x7fdd5c104970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:53:04.650 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:53:04.651+0000 7fdd5b200700 1 --2- 172.21.15.18:0/976760500 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fdd5c104500 0x7fdd5c104970 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fdd48004940 tx=0x7fdd48009e30 comp rx=0 tx=0).stop 2024-04-12T20:53:04.650 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:53:04.652+0000 7fdd5b200700 1 -- 172.21.15.18:0/976760500 shutdown_connections 2024-04-12T20:53:04.650 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:53:04.652+0000 7fdd5b200700 1 --2- 172.21.15.18:0/976760500 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fdd5c104500 0x7fdd5c104970 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:53:04.650 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:53:04.652+0000 7fdd5b200700 1 --2- 172.21.15.18:0/976760500 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdd5c103250 0x7fdd5c103660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:53:04.650 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:53:04.652+0000 7fdd5b200700 1 -- 172.21.15.18:0/976760500 >> 172.21.15.18:0/976760500 conn(0x7fdd5c072530 msgr2=0x7fdd5c074960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T20:53:04.650 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:53:04.652+0000 7fdd5b200700 1 -- 172.21.15.18:0/976760500 shutdown_connections 2024-04-12T20:53:04.650 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:53:04.652+0000 7fdd5b200700 1 -- 172.21.15.18:0/976760500 wait complete. 2024-04-12T20:53:04.652 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:53:04.653+0000 7fdd5b200700 1 Processor -- start 2024-04-12T20:53:04.652 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:53:04.653+0000 7fdd5b200700 1 -- start start 2024-04-12T20:53:04.652 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:53:04.653+0000 7fdd5b200700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fdd5c103250 0x7fdd5c113e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T20:53:04.652 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:53:04.653+0000 7fdd5b200700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdd5c104500 0x7fdd5c1143d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T20:53:04.652 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:53:04.653+0000 7fdd5b200700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd5c115140 con 0x7fdd5c103250 2024-04-12T20:53:04.652 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:53:04.653+0000 7fdd5b200700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd5c1152b0 con 0x7fdd5c104500 2024-04-12T20:53:04.652 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:53:04.653+0000 7fdd59e00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fdd5c103250 0x7fdd5c113e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T20:53:04.652 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:53:04.654+0000 7fdd59e00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fdd5c103250 0x7fdd5c113e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:39364/0 (socket says 172.21.15.18:39364) 2024-04-12T20:53:04.652 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:53:04.654+0000 7fdd59e00700 1 -- 172.21.15.18:0/2143668026 learned_addr learned my addr 172.21.15.18:0/2143668026 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T20:53:04.652 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:53:04.654+0000 7fdd59400700 1 --2- 172.21.15.18:0/2143668026 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdd5c104500 0x7fdd5c1143d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T20:53:04.653 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:53:04.654+0000 7fdd59e00700 1 -- 172.21.15.18:0/2143668026 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdd5c104500 msgr2=0x7fdd5c1143d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:53:04.653 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:53:04.654+0000 7fdd59e00700 1 --2- 172.21.15.18:0/2143668026 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdd5c104500 0x7fdd5c1143d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:53:04.653 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:53:04.654+0000 7fdd59e00700 1 -- 172.21.15.18:0/2143668026 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd48005040 con 0x7fdd5c103250 2024-04-12T20:53:04.653 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:53:04.654+0000 7fdd59400700 1 --2- 172.21.15.18:0/2143668026 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdd5c104500 0x7fdd5c1143d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-12T20:53:04.653 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:53:04.654+0000 7fdd59e00700 1 --2- 172.21.15.18:0/2143668026 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fdd5c103250 0x7fdd5c113e90 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fdd4c002a50 tx=0x7fdd4c0046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T20:53:04.654 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:53:04.655+0000 7fdd52000700 1 -- 172.21.15.18:0/2143668026 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdd4c007cc0 con 0x7fdd5c103250 2024-04-12T20:53:04.654 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:53:04.655+0000 7fdd52000700 1 -- 172.21.15.18:0/2143668026 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fdd4c00f040 con 0x7fdd5c103250 2024-04-12T20:53:04.654 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:53:04.655+0000 7fdd5b200700 1 -- 172.21.15.18:0/2143668026 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdd5c115530 con 0x7fdd5c103250 2024-04-12T20:53:04.654 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:53:04.655+0000 7fdd52000700 1 -- 172.21.15.18:0/2143668026 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdd4c009640 con 0x7fdd5c103250 2024-04-12T20:53:04.654 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:53:04.655+0000 7fdd5b200700 1 -- 172.21.15.18:0/2143668026 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdd5c115af0 con 0x7fdd5c103250 2024-04-12T20:53:04.654 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:53:04.656+0000 7fdd5b200700 1 -- 172.21.15.18:0/2143668026 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdd5c076080 con 0x7fdd5c103250 2024-04-12T20:53:04.654 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:53:04.657+0000 7fdd52000700 1 -- 172.21.15.18:0/2143668026 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7fdd4c0097a0 con 0x7fdd5c103250 2024-04-12T20:53:04.656 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:53:04.658+0000 7fdd52000700 1 --2- 172.21.15.18:0/2143668026 >> [v2:172.21.15.18:6800/3048894687,v1:172.21.15.18:6801/3048894687] conn(0x7fdd400692a0 0x7fdd4006b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T20:53:04.656 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:53:04.658+0000 7fdd52000700 1 -- 172.21.15.18:0/2143668026 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5550+0+0 (secure 0 0 0) 0x7fdd4c086fa0 con 0x7fdd5c103250 2024-04-12T20:53:04.656 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:53:04.658+0000 7fdd59400700 1 --2- 172.21.15.18:0/2143668026 >> [v2:172.21.15.18:6800/3048894687,v1:172.21.15.18:6801/3048894687] conn(0x7fdd400692a0 0x7fdd4006b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T20:53:04.657 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:53:04.659+0000 7fdd59400700 1 --2- 172.21.15.18:0/2143668026 >> [v2:172.21.15.18:6800/3048894687,v1:172.21.15.18:6801/3048894687] conn(0x7fdd400692a0 0x7fdd4006b750 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fdd4800b7a0 tx=0x7fdd48009e30 comp rx=0 tx=0).ready entity=mgr.14212 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T20:53:04.660 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:53:04.663+0000 7fdd52000700 1 -- 172.21.15.18:0/2143668026 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7fdd4c0531e0 con 0x7fdd5c103250 2024-04-12T20:53:04.877 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:53:04.880+0000 7fdd5b200700 1 -- 172.21.15.18:0/2143668026 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command([{prefix=config set, name=mon_warn_on_insecure_global_id_reclaim_allowed}] v 0) v1 -- 0x7fdd5c04bed0 con 0x7fdd5c103250 2024-04-12T20:53:04.878 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:53:04.881+0000 7fdd52000700 1 -- 172.21.15.18:0/2143668026 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_warn_on_insecure_global_id_reclaim_allowed}]=0 v21)=0 v21) v1 ==== 163+0+0 (secure 0 0 0) 0x7fdd5c04bed0 con 0x7fdd5c103250 2024-04-12T20:53:04.881 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:53:04.883+0000 7fdd5b200700 1 -- 172.21.15.18:0/2143668026 >> [v2:172.21.15.18:6800/3048894687,v1:172.21.15.18:6801/3048894687] conn(0x7fdd400692a0 msgr2=0x7fdd4006b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:53:04.881 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:53:04.883+0000 7fdd5b200700 1 --2- 172.21.15.18:0/2143668026 >> [v2:172.21.15.18:6800/3048894687,v1:172.21.15.18:6801/3048894687] conn(0x7fdd400692a0 0x7fdd4006b750 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fdd4800b7a0 tx=0x7fdd48009e30 comp rx=0 tx=0).stop 2024-04-12T20:53:04.881 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:53:04.883+0000 7fdd5b200700 1 -- 172.21.15.18:0/2143668026 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fdd5c103250 msgr2=0x7fdd5c113e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:53:04.881 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:53:04.883+0000 7fdd5b200700 1 --2- 172.21.15.18:0/2143668026 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fdd5c103250 0x7fdd5c113e90 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fdd4c002a50 tx=0x7fdd4c0046b0 comp rx=0 tx=0).stop 2024-04-12T20:53:04.882 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:53:04.884+0000 7fdd5b200700 1 -- 172.21.15.18:0/2143668026 shutdown_connections 2024-04-12T20:53:04.882 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:53:04.884+0000 7fdd5b200700 1 --2- 172.21.15.18:0/2143668026 >> [v2:172.21.15.18:6800/3048894687,v1:172.21.15.18:6801/3048894687] conn(0x7fdd400692a0 0x7fdd4006b750 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:53:04.882 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:53:04.884+0000 7fdd5b200700 1 --2- 172.21.15.18:0/2143668026 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fdd5c103250 0x7fdd5c113e90 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:53:04.882 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:53:04.884+0000 7fdd5b200700 1 --2- 172.21.15.18:0/2143668026 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdd5c104500 0x7fdd5c1143d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:53:04.882 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:53:04.884+0000 7fdd5b200700 1 -- 172.21.15.18:0/2143668026 >> 172.21.15.18:0/2143668026 conn(0x7fdd5c072530 msgr2=0x7fdd5c074b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T20:53:04.883 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:53:04.884+0000 7fdd5b200700 1 -- 172.21.15.18:0/2143668026 shutdown_connections 2024-04-12T20:53:04.883 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:53:04.884+0000 7fdd5b200700 1 -- 172.21.15.18:0/2143668026 wait complete. 2024-04-12T20:53:05.791 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1cbe407a-f90d-11ee-bc8d-c7b262605968 -e sha1=324baf8ddb91dea46cac50cb535c306115791355 -- bash -c 'ceph config set global log_to_journald false --force' 2024-04-12T20:53:06.313 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/mon.smithi018/config 2024-04-12T20:53:06.352 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:53:06 smithi018 ceph-mon[28799]: pgmap v254: 65 pgs: 65 active+clean; 587 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 682 B/s wr, 0 op/s 2024-04-12T20:53:06.531 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:53:06 smithi186 ceph-mon[36597]: pgmap v254: 65 pgs: 65 active+clean; 587 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 682 B/s wr, 0 op/s 2024-04-12T20:53:08.317 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:53:08 smithi018 ceph-mon[28799]: pgmap v255: 65 pgs: 65 active+clean; 587 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 767 B/s wr, 0 op/s 2024-04-12T20:53:08.531 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:53:08 smithi186 ceph-mon[36597]: pgmap v255: 65 pgs: 65 active+clean; 587 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 767 B/s wr, 0 op/s 2024-04-12T20:53:08.651 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:53:08.651+0000 7f9792400700 1 -- 172.21.15.18:0/555358331 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f978c0709a0 msgr2=0x7f978c070e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:53:08.651 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:53:08.651+0000 7f9792400700 1 --2- 172.21.15.18:0/555358331 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f978c0709a0 0x7f978c070e10 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f9774004940 tx=0x7f9774009e30 comp rx=0 tx=0).stop 2024-04-12T20:53:08.651 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:53:08.651+0000 7f9792400700 1 -- 172.21.15.18:0/555358331 shutdown_connections 2024-04-12T20:53:08.651 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:53:08.651+0000 7f9792400700 1 --2- 172.21.15.18:0/555358331 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f978c0709a0 0x7f978c070e10 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:53:08.651 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:53:08.651+0000 7f9792400700 1 --2- 172.21.15.18:0/555358331 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f978c071f50 0x7f978c070460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:53:08.651 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:53:08.651+0000 7f9792400700 1 -- 172.21.15.18:0/555358331 >> 172.21.15.18:0/555358331 conn(0x7f978c0f8560 msgr2=0x7f978c0fa970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T20:53:08.651 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:53:08.652+0000 7f9792400700 1 -- 172.21.15.18:0/555358331 shutdown_connections 2024-04-12T20:53:08.651 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:53:08.652+0000 7f9792400700 1 -- 172.21.15.18:0/555358331 wait complete. 2024-04-12T20:53:08.651 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:53:08.652+0000 7f9792400700 1 Processor -- start 2024-04-12T20:53:08.653 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:53:08.652+0000 7f9792400700 1 -- start start 2024-04-12T20:53:08.653 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:53:08.653+0000 7f9792400700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f978c0709a0 0x7f978c10fc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T20:53:08.653 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:53:08.653+0000 7f9792400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f978c071f50 0x7f978c1101a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T20:53:08.653 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:53:08.653+0000 7f9792400700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f978c110830 con 0x7f978c071f50 2024-04-12T20:53:08.653 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:53:08.653+0000 7f9792400700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f978c111020 con 0x7f978c0709a0 2024-04-12T20:53:08.653 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:53:08.653+0000 7f9783e00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f978c071f50 0x7f978c1101a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T20:53:08.653 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:53:08.653+0000 7f9783e00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f978c071f50 0x7f978c1101a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:39388/0 (socket says 172.21.15.18:39388) 2024-04-12T20:53:08.653 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:53:08.653+0000 7f9783e00700 1 -- 172.21.15.18:0/204695945 learned_addr learned my addr 172.21.15.18:0/204695945 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T20:53:08.653 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:53:08.653+0000 7f978b400700 1 --2- 172.21.15.18:0/204695945 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f978c0709a0 0x7f978c10fc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T20:53:08.653 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:53:08.654+0000 7f9783e00700 1 -- 172.21.15.18:0/204695945 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f978c0709a0 msgr2=0x7f978c10fc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:53:08.654 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:53:08.654+0000 7f9783e00700 1 --2- 172.21.15.18:0/204695945 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f978c0709a0 0x7f978c10fc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:53:08.654 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:53:08.654+0000 7f9783e00700 1 -- 172.21.15.18:0/204695945 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9774005040 con 0x7f978c071f50 2024-04-12T20:53:08.654 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:53:08.654+0000 7f978b400700 1 --2- 172.21.15.18:0/204695945 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f978c0709a0 0x7f978c10fc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-12T20:53:08.654 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:53:08.654+0000 7f9783e00700 1 --2- 172.21.15.18:0/204695945 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f978c071f50 0x7f978c1101a0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f977400b3b0 tx=0x7f9774009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T20:53:08.654 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:53:08.654+0000 7f9783400700 1 -- 172.21.15.18:0/204695945 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f977400e040 con 0x7f978c071f50 2024-04-12T20:53:08.654 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:53:08.654+0000 7f9783400700 1 -- 172.21.15.18:0/204695945 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f9774004290 con 0x7f978c071f50 2024-04-12T20:53:08.654 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:53:08.654+0000 7f9792400700 1 -- 172.21.15.18:0/204695945 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f978c1112a0 con 0x7f978c071f50 2024-04-12T20:53:08.654 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:53:08.654+0000 7f9783400700 1 -- 172.21.15.18:0/204695945 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f977401c630 con 0x7f978c071f50 2024-04-12T20:53:08.654 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:53:08.654+0000 7f9792400700 1 -- 172.21.15.18:0/204695945 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f978c111860 con 0x7f978c071f50 2024-04-12T20:53:08.654 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:53:08.656+0000 7f9792400700 1 -- 172.21.15.18:0/204695945 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f978c1095b0 con 0x7f978c071f50 2024-04-12T20:53:08.655 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:53:08.657+0000 7f9783400700 1 -- 172.21.15.18:0/204695945 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7f977400bbf0 con 0x7f978c071f50 2024-04-12T20:53:08.656 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:53:08.658+0000 7f9783400700 1 --2- 172.21.15.18:0/204695945 >> [v2:172.21.15.18:6800/3048894687,v1:172.21.15.18:6801/3048894687] conn(0x7f97700692b0 0x7f977006b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T20:53:08.657 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:53:08.658+0000 7f9783400700 1 -- 172.21.15.18:0/204695945 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5550+0+0 (secure 0 0 0) 0x7f9774088630 con 0x7f978c071f50 2024-04-12T20:53:08.657 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:53:08.658+0000 7f978b400700 1 --2- 172.21.15.18:0/204695945 >> [v2:172.21.15.18:6800/3048894687,v1:172.21.15.18:6801/3048894687] conn(0x7f97700692b0 0x7f977006b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T20:53:08.657 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:53:08.659+0000 7f978b400700 1 --2- 172.21.15.18:0/204695945 >> [v2:172.21.15.18:6800/3048894687,v1:172.21.15.18:6801/3048894687] conn(0x7f97700692b0 0x7f977006b760 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f977c006fd0 tx=0x7f977c008040 comp rx=0 tx=0).ready entity=mgr.14212 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T20:53:08.662 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:53:08.663+0000 7f9783400700 1 -- 172.21.15.18:0/204695945 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7f97740583a0 con 0x7f978c071f50 2024-04-12T20:53:08.870 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:53:08.871+0000 7f9792400700 1 -- 172.21.15.18:0/204695945 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command([{prefix=config set, name=log_to_journald}] v 0) v1 -- 0x7f978c04bed0 con 0x7f978c071f50 2024-04-12T20:53:08.870 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:53:08.872+0000 7f9783400700 1 -- 172.21.15.18:0/204695945 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{prefix=config set, name=log_to_journald}]=0 v21)=0 v21) v1 ==== 135+0+0 (secure 0 0 0) 0x7f9774057f30 con 0x7f978c071f50 2024-04-12T20:53:08.873 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:53:08.874+0000 7f9792400700 1 -- 172.21.15.18:0/204695945 >> [v2:172.21.15.18:6800/3048894687,v1:172.21.15.18:6801/3048894687] conn(0x7f97700692b0 msgr2=0x7f977006b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:53:08.873 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:53:08.874+0000 7f9792400700 1 --2- 172.21.15.18:0/204695945 >> [v2:172.21.15.18:6800/3048894687,v1:172.21.15.18:6801/3048894687] conn(0x7f97700692b0 0x7f977006b760 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f977c006fd0 tx=0x7f977c008040 comp rx=0 tx=0).stop 2024-04-12T20:53:08.873 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:53:08.874+0000 7f9792400700 1 -- 172.21.15.18:0/204695945 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f978c071f50 msgr2=0x7f978c1101a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:53:08.873 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:53:08.874+0000 7f9792400700 1 --2- 172.21.15.18:0/204695945 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f978c071f50 0x7f978c1101a0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f977400b3b0 tx=0x7f9774009e30 comp rx=0 tx=0).stop 2024-04-12T20:53:08.873 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:53:08.874+0000 7f9792400700 1 -- 172.21.15.18:0/204695945 shutdown_connections 2024-04-12T20:53:08.874 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:53:08.874+0000 7f9792400700 1 --2- 172.21.15.18:0/204695945 >> [v2:172.21.15.18:6800/3048894687,v1:172.21.15.18:6801/3048894687] conn(0x7f97700692b0 0x7f977006b760 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:53:08.874 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:53:08.874+0000 7f9792400700 1 --2- 172.21.15.18:0/204695945 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f978c0709a0 0x7f978c10fc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:53:08.874 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:53:08.874+0000 7f9792400700 1 --2- 172.21.15.18:0/204695945 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f978c071f50 0x7f978c1101a0 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:53:08.874 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:53:08.874+0000 7f9792400700 1 -- 172.21.15.18:0/204695945 >> 172.21.15.18:0/204695945 conn(0x7f978c0f8560 msgr2=0x7f978c102d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T20:53:08.874 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:53:08.875+0000 7f9792400700 1 -- 172.21.15.18:0/204695945 shutdown_connections 2024-04-12T20:53:08.874 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:53:08.875+0000 7f9792400700 1 -- 172.21.15.18:0/204695945 wait complete. 2024-04-12T20:53:09.828 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1cbe407a-f90d-11ee-bc8d-c7b262605968 -e sha1=324baf8ddb91dea46cac50cb535c306115791355 -- bash -c 'ceph orch upgrade start --image quay.ceph.io/ceph-ci/ceph:$sha1 --daemon-types mgr' 2024-04-12T20:53:10.351 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:53:10 smithi018 ceph-mon[28799]: pgmap v256: 65 pgs: 65 active+clean; 587 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 597 B/s wr, 0 op/s 2024-04-12T20:53:10.364 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/mon.smithi018/config 2024-04-12T20:53:10.531 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:53:10 smithi186 ceph-mon[36597]: pgmap v256: 65 pgs: 65 active+clean; 587 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 597 B/s wr, 0 op/s 2024-04-12T20:53:12.341 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:53:12 smithi018 ceph-mon[28799]: pgmap v257: 65 pgs: 65 active+clean; 587 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 682 B/s wr, 0 op/s 2024-04-12T20:53:12.531 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:53:12 smithi186 ceph-mon[36597]: pgmap v257: 65 pgs: 65 active+clean; 587 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 682 B/s wr, 0 op/s 2024-04-12T20:53:12.672 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:53:12.673+0000 7fddd2800700 1 -- 172.21.15.18:0/2963898181 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fddcc1023f0 msgr2=0x7fddcc1047d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:53:12.672 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:53:12.673+0000 7fddd2800700 1 --2- 172.21.15.18:0/2963898181 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fddcc1023f0 0x7fddcc1047d0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fddb4004940 tx=0x7fddb4009e30 comp rx=0 tx=0).stop 2024-04-12T20:53:12.672 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:53:12.673+0000 7fddd2800700 1 -- 172.21.15.18:0/2963898181 shutdown_connections 2024-04-12T20:53:12.672 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:53:12.673+0000 7fddd2800700 1 --2- 172.21.15.18:0/2963898181 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fddcc1023f0 0x7fddcc1047d0 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:53:12.672 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:53:12.673+0000 7fddd2800700 1 --2- 172.21.15.18:0/2963898181 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fddcc066b10 0x7fddcc101eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:53:12.672 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:53:12.673+0000 7fddd2800700 1 -- 172.21.15.18:0/2963898181 >> 172.21.15.18:0/2963898181 conn(0x7fddcc0f74e0 msgr2=0x7fddcc0f98f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T20:53:12.673 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:53:12.673+0000 7fddd2800700 1 -- 172.21.15.18:0/2963898181 shutdown_connections 2024-04-12T20:53:12.673 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:53:12.673+0000 7fddd2800700 1 -- 172.21.15.18:0/2963898181 wait complete. 2024-04-12T20:53:12.674 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:53:12.674+0000 7fddd2800700 1 Processor -- start 2024-04-12T20:53:12.674 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:53:12.674+0000 7fddd2800700 1 -- start start 2024-04-12T20:53:12.674 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:53:12.675+0000 7fddd2800700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fddcc066b10 0x7fddcc10fc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T20:53:12.674 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:53:12.675+0000 7fddd2800700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fddcc1023f0 0x7fddcc1101a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T20:53:12.674 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:53:12.675+0000 7fddd2800700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fddcc110830 con 0x7fddcc066b10 2024-04-12T20:53:12.674 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:53:12.675+0000 7fddd2800700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fddcc111020 con 0x7fddcc1023f0 2024-04-12T20:53:12.674 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:53:12.675+0000 7fddcb400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fddcc066b10 0x7fddcc10fc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T20:53:12.674 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:53:12.675+0000 7fddcb400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fddcc066b10 0x7fddcc10fc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:36362/0 (socket says 172.21.15.18:36362) 2024-04-12T20:53:12.674 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:53:12.675+0000 7fddcb400700 1 -- 172.21.15.18:0/4167222178 learned_addr learned my addr 172.21.15.18:0/4167222178 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T20:53:12.674 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:53:12.675+0000 7fddc2a00700 1 --2- 172.21.15.18:0/4167222178 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fddcc1023f0 0x7fddcc1101a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T20:53:12.675 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:53:12.676+0000 7fddcb400700 1 -- 172.21.15.18:0/4167222178 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fddcc1023f0 msgr2=0x7fddcc1101a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:53:12.675 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:53:12.676+0000 7fddcb400700 1 --2- 172.21.15.18:0/4167222178 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fddcc1023f0 0x7fddcc1101a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:53:12.675 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:53:12.676+0000 7fddcb400700 1 -- 172.21.15.18:0/4167222178 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fddb4005040 con 0x7fddcc066b10 2024-04-12T20:53:12.675 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:53:12.676+0000 7fddcb400700 1 --2- 172.21.15.18:0/4167222178 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fddcc066b10 0x7fddcc10fc60 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fddbc00b790 tx=0x7fddbc009220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T20:53:12.676 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:53:12.676+0000 7fddc3e00700 1 -- 172.21.15.18:0/4167222178 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fddbc00f6f0 con 0x7fddcc066b10 2024-04-12T20:53:12.676 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:53:12.676+0000 7fddd2800700 1 -- 172.21.15.18:0/4167222178 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fddcc111300 con 0x7fddcc066b10 2024-04-12T20:53:12.676 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:53:12.676+0000 7fddc3e00700 1 -- 172.21.15.18:0/4167222178 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fddbc00fd20 con 0x7fddcc066b10 2024-04-12T20:53:12.676 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:53:12.676+0000 7fddc3e00700 1 -- 172.21.15.18:0/4167222178 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fddbc0103a0 con 0x7fddcc066b10 2024-04-12T20:53:12.676 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:53:12.677+0000 7fddd2800700 1 -- 172.21.15.18:0/4167222178 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fddcc1118c0 con 0x7fddcc066b10 2024-04-12T20:53:12.676 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:53:12.678+0000 7fddd2800700 1 -- 172.21.15.18:0/4167222178 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fddcc05f5f0 con 0x7fddcc066b10 2024-04-12T20:53:12.677 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:53:12.679+0000 7fddc3e00700 1 -- 172.21.15.18:0/4167222178 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7fddbc011400 con 0x7fddcc066b10 2024-04-12T20:53:12.678 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:53:12.680+0000 7fddc3e00700 1 --2- 172.21.15.18:0/4167222178 >> [v2:172.21.15.18:6800/3048894687,v1:172.21.15.18:6801/3048894687] conn(0x7fddb00692b0 0x7fddb006b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T20:53:12.678 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:53:12.680+0000 7fddc3e00700 1 -- 172.21.15.18:0/4167222178 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5550+0+0 (secure 0 0 0) 0x7fddbc087120 con 0x7fddcc066b10 2024-04-12T20:53:12.679 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:53:12.680+0000 7fddc2a00700 1 --2- 172.21.15.18:0/4167222178 >> [v2:172.21.15.18:6800/3048894687,v1:172.21.15.18:6801/3048894687] conn(0x7fddb00692b0 0x7fddb006b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T20:53:12.679 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:53:12.681+0000 7fddc2a00700 1 --2- 172.21.15.18:0/4167222178 >> [v2:172.21.15.18:6800/3048894687,v1:172.21.15.18:6801/3048894687] conn(0x7fddb00692b0 0x7fddb006b760 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fddb4004f90 tx=0x7fddb4009e30 comp rx=0 tx=0).ready entity=mgr.14212 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T20:53:12.683 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:53:12.684+0000 7fddc3e00700 1 -- 172.21.15.18:0/4167222178 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7fddbc056d70 con 0x7fddcc066b10 2024-04-12T20:53:12.919 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:53:12.920+0000 7fddd2800700 1 -- 172.21.15.18:0/4167222178 --> [v2:172.21.15.18:6800/3048894687,v1:172.21.15.18:6801/3048894687] -- mgr_command(tid 0: {"prefix": "orch upgrade start", "image": "quay.ceph.io/ceph-ci/ceph:324baf8ddb91dea46cac50cb535c306115791355", "daemon_types": "mgr", "target": ["mon-mgr", ""]}) v1 -- 0x7fddcc111bb0 con 0x7fddb00692b0 2024-04-12T20:53:14.352 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:53:14 smithi018 ceph-mon[28799]: pgmap v258: 65 pgs: 65 active+clean; 590 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1023 B/s wr, 0 op/s 2024-04-12T20:53:14.352 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:53:14 smithi018 ceph-mon[28799]: from='client.14540 -' entity='client.admin' cmd=[{"prefix": "orch upgrade start", "image": "quay.ceph.io/ceph-ci/ceph:324baf8ddb91dea46cac50cb535c306115791355", "daemon_types": "mgr", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T20:53:14.531 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:53:14 smithi186 ceph-mon[36597]: pgmap v258: 65 pgs: 65 active+clean; 590 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1023 B/s wr, 0 op/s 2024-04-12T20:53:14.531 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:53:14 smithi186 ceph-mon[36597]: from='client.14540 -' entity='client.admin' cmd=[{"prefix": "orch upgrade start", "image": "quay.ceph.io/ceph-ci/ceph:324baf8ddb91dea46cac50cb535c306115791355", "daemon_types": "mgr", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T20:53:15.352 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:53:15 smithi018 ceph-mon[28799]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T20:53:15.454 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:53:15 smithi186 ceph-mon[36597]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T20:53:16.352 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:53:16 smithi018 ceph-mon[28799]: pgmap v259: 65 pgs: 65 active+clean; 590 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 597 B/s wr, 0 op/s 2024-04-12T20:53:16.531 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:53:16 smithi186 ceph-mon[36597]: pgmap v259: 65 pgs: 65 active+clean; 590 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 597 B/s wr, 0 op/s 2024-04-12T20:53:18.352 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:53:18 smithi018 ceph-mon[28799]: pgmap v260: 65 pgs: 65 active+clean; 590 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 597 B/s wr, 0 op/s 2024-04-12T20:53:18.531 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:53:18 smithi186 ceph-mon[36597]: pgmap v260: 65 pgs: 65 active+clean; 590 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 597 B/s wr, 0 op/s 2024-04-12T20:53:20.352 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:53:20 smithi018 ceph-mon[28799]: pgmap v261: 65 pgs: 65 active+clean; 590 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 511 B/s wr, 0 op/s 2024-04-12T20:53:20.531 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:53:20 smithi186 ceph-mon[36597]: pgmap v261: 65 pgs: 65 active+clean; 590 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 511 B/s wr, 0 op/s 2024-04-12T20:53:22.352 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:53:22 smithi018 ceph-mon[28799]: pgmap v262: 65 pgs: 65 active+clean; 590 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 511 B/s wr, 0 op/s 2024-04-12T20:53:22.531 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:53:22 smithi186 ceph-mon[36597]: pgmap v262: 65 pgs: 65 active+clean; 590 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 511 B/s wr, 0 op/s 2024-04-12T20:53:24.531 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:53:24 smithi186 ceph-mon[36597]: pgmap v263: 65 pgs: 65 active+clean; 590 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 426 B/s wr, 0 op/s 2024-04-12T20:53:24.602 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:53:24 smithi018 ceph-mon[28799]: pgmap v263: 65 pgs: 65 active+clean; 590 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 426 B/s wr, 0 op/s 2024-04-12T20:53:26.531 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:53:26 smithi186 ceph-mon[36597]: pgmap v264: 65 pgs: 65 active+clean; 590 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-12T20:53:26.602 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:53:26 smithi018 ceph-mon[28799]: pgmap v264: 65 pgs: 65 active+clean; 590 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-12T20:53:28.531 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:53:28 smithi186 ceph-mon[36597]: pgmap v265: 65 pgs: 65 active+clean; 590 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-12T20:53:28.602 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:53:28 smithi018 ceph-mon[28799]: pgmap v265: 65 pgs: 65 active+clean; 590 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-12T20:53:30.531 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:53:30 smithi186 ceph-mon[36597]: pgmap v266: 65 pgs: 65 active+clean; 590 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-12T20:53:30.531 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:53:30 smithi186 ceph-mon[36597]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T20:53:30.602 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:53:30 smithi018 ceph-mon[28799]: pgmap v266: 65 pgs: 65 active+clean; 590 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-12T20:53:30.602 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:53:30 smithi018 ceph-mon[28799]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T20:53:32.531 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:53:32 smithi186 ceph-mon[36597]: pgmap v267: 65 pgs: 65 active+clean; 590 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-12T20:53:32.602 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:53:32 smithi018 ceph-mon[28799]: pgmap v267: 65 pgs: 65 active+clean; 590 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-12T20:53:34.531 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:53:34 smithi186 ceph-mon[36597]: pgmap v268: 65 pgs: 65 active+clean; 590 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-12T20:53:34.602 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:53:34 smithi018 ceph-mon[28799]: pgmap v268: 65 pgs: 65 active+clean; 590 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-12T20:53:36.407 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:53:36 smithi018 ceph-mon[28799]: pgmap v269: 65 pgs: 65 active+clean; 590 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-12T20:53:36.408 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:53:36 smithi018 ceph-mon[28799]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-12T20:53:36.531 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:53:36 smithi186 ceph-mon[36597]: pgmap v269: 65 pgs: 65 active+clean; 590 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-12T20:53:36.531 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:53:36 smithi186 ceph-mon[36597]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-12T20:53:38.352 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:53:38 smithi018 ceph-mon[28799]: pgmap v270: 65 pgs: 65 active+clean; 590 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-12T20:53:38.352 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:53:38 smithi018 ceph-mon[28799]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-12T20:53:38.352 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:53:38 smithi018 ceph-mon[28799]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-12T20:53:38.352 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:53:38 smithi018 ceph-mon[28799]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' 2024-04-12T20:53:38.531 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:53:38 smithi186 ceph-mon[36597]: pgmap v270: 65 pgs: 65 active+clean; 590 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-12T20:53:38.531 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:53:38 smithi186 ceph-mon[36597]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-12T20:53:38.531 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:53:38 smithi186 ceph-mon[36597]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-12T20:53:38.532 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:53:38 smithi186 ceph-mon[36597]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' 2024-04-12T20:53:40.531 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:53:40 smithi186 ceph-mon[36597]: pgmap v271: 65 pgs: 65 active+clean; 590 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-12T20:53:40.602 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:53:40 smithi018 ceph-mon[28799]: pgmap v271: 65 pgs: 65 active+clean; 590 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-12T20:53:41.531 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:53:41 smithi186 ceph-mon[36597]: pgmap v272: 65 pgs: 65 active+clean; 590 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-12T20:53:41.602 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:53:41 smithi018 ceph-mon[28799]: pgmap v272: 65 pgs: 65 active+clean; 590 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-12T20:53:44.281 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:53:43 smithi186 ceph-mon[36597]: pgmap v273: 65 pgs: 65 active+clean; 590 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-12T20:53:44.351 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:53:43 smithi018 ceph-mon[28799]: pgmap v273: 65 pgs: 65 active+clean; 590 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-12T20:53:45.281 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:53:44 smithi186 ceph-mon[36597]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T20:53:45.351 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:53:44 smithi018 ceph-mon[28799]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T20:53:46.281 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:53:45 smithi186 ceph-mon[36597]: pgmap v274: 65 pgs: 65 active+clean; 590 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-12T20:53:46.352 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:53:45 smithi018 ceph-mon[28799]: pgmap v274: 65 pgs: 65 active+clean; 590 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-12T20:53:48.281 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:53:47 smithi186 ceph-mon[36597]: pgmap v275: 65 pgs: 65 active+clean; 590 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-12T20:53:48.351 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:53:47 smithi018 ceph-mon[28799]: pgmap v275: 65 pgs: 65 active+clean; 590 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-12T20:53:50.281 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:53:49 smithi186 ceph-mon[36597]: pgmap v276: 65 pgs: 65 active+clean; 590 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-12T20:53:50.291 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:53:49 smithi018 ceph-mon[28799]: pgmap v276: 65 pgs: 65 active+clean; 590 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-12T20:53:51.789 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:53:51.791+0000 7fddc3e00700 1 -- 172.21.15.18:0/4167222178 <== mgr.14212 v2:172.21.15.18:6800/3048894687 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+89 (secure 0 0 0) 0x7fddcc111bb0 con 0x7fddb00692b0 2024-04-12T20:53:51.789 INFO:teuthology.orchestra.run.smithi018.stdout:Initiating upgrade to quay.ceph.io/ceph-ci/ceph:324baf8ddb91dea46cac50cb535c306115791355 2024-04-12T20:53:51.792 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:53:51.794+0000 7fddd2800700 1 -- 172.21.15.18:0/4167222178 >> [v2:172.21.15.18:6800/3048894687,v1:172.21.15.18:6801/3048894687] conn(0x7fddb00692b0 msgr2=0x7fddb006b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:53:51.792 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:53:51.794+0000 7fddd2800700 1 --2- 172.21.15.18:0/4167222178 >> [v2:172.21.15.18:6800/3048894687,v1:172.21.15.18:6801/3048894687] conn(0x7fddb00692b0 0x7fddb006b760 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fddb4004f90 tx=0x7fddb4009e30 comp rx=0 tx=0).stop 2024-04-12T20:53:51.792 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:53:51.794+0000 7fddd2800700 1 -- 172.21.15.18:0/4167222178 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fddcc066b10 msgr2=0x7fddcc10fc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:53:51.792 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:53:51.794+0000 7fddd2800700 1 --2- 172.21.15.18:0/4167222178 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fddcc066b10 0x7fddcc10fc60 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fddbc00b790 tx=0x7fddbc009220 comp rx=0 tx=0).stop 2024-04-12T20:53:51.792 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:53:51.794+0000 7fddd2800700 1 -- 172.21.15.18:0/4167222178 shutdown_connections 2024-04-12T20:53:51.792 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:53:51.794+0000 7fddd2800700 1 --2- 172.21.15.18:0/4167222178 >> [v2:172.21.15.18:6800/3048894687,v1:172.21.15.18:6801/3048894687] conn(0x7fddb00692b0 0x7fddb006b760 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:53:51.792 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:53:51.794+0000 7fddd2800700 1 --2- 172.21.15.18:0/4167222178 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fddcc066b10 0x7fddcc10fc60 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:53:51.792 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:53:51.794+0000 7fddd2800700 1 --2- 172.21.15.18:0/4167222178 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fddcc1023f0 0x7fddcc1101a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:53:51.792 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:53:51.794+0000 7fddd2800700 1 -- 172.21.15.18:0/4167222178 >> 172.21.15.18:0/4167222178 conn(0x7fddcc0f74e0 msgr2=0x7fddcc0f78b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T20:53:51.792 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:53:51.794+0000 7fddd2800700 1 -- 172.21.15.18:0/4167222178 shutdown_connections 2024-04-12T20:53:51.793 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:53:51.795+0000 7fddd2800700 1 -- 172.21.15.18:0/4167222178 wait complete. 2024-04-12T20:53:52.281 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:53:52 smithi186 ceph-mon[36597]: pgmap v277: 65 pgs: 65 active+clean; 590 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-12T20:53:52.281 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:53:52 smithi186 ceph-mon[36597]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' 2024-04-12T20:53:52.281 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:53:52 smithi186 ceph-mon[36597]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-12T20:53:52.281 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:53:52 smithi186 ceph-mon[36597]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-12T20:53:52.281 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:53:52 smithi186 ceph-mon[36597]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-12T20:53:52.281 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:53:52 smithi186 ceph-mon[36597]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' 2024-04-12T20:53:52.352 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:53:52 smithi018 ceph-mon[28799]: pgmap v277: 65 pgs: 65 active+clean; 590 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-12T20:53:52.352 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:53:52 smithi018 ceph-mon[28799]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' 2024-04-12T20:53:52.353 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:53:52 smithi018 ceph-mon[28799]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-12T20:53:52.353 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:53:52 smithi018 ceph-mon[28799]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-12T20:53:52.353 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:53:52 smithi018 ceph-mon[28799]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-12T20:53:52.353 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:53:52 smithi018 ceph-mon[28799]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' 2024-04-12T20:53:52.641 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1cbe407a-f90d-11ee-bc8d-c7b262605968 -e sha1=324baf8ddb91dea46cac50cb535c306115791355 -- bash -c 'while ceph orch upgrade status | jq '"'"'.in_progress'"'"' | grep true && ! ceph orch upgrade status | jq '"'"'.message'"'"' | grep Error ; do ceph orch ps ; ceph versions ; ceph orch upgrade status ; sleep 30 ; done' 2024-04-12T20:53:53.261 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:53:53 smithi018 ceph-mon[28799]: Upgrade: Started with target quay.ceph.io/ceph-ci/ceph:324baf8ddb91dea46cac50cb535c306115791355 2024-04-12T20:53:53.261 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:53:53 smithi018 ceph-mon[28799]: Upgrade: First pull of quay.ceph.io/ceph-ci/ceph:324baf8ddb91dea46cac50cb535c306115791355 2024-04-12T20:53:53.281 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:53:53 smithi186 ceph-mon[36597]: Upgrade: Started with target quay.ceph.io/ceph-ci/ceph:324baf8ddb91dea46cac50cb535c306115791355 2024-04-12T20:53:53.281 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:53:53 smithi186 ceph-mon[36597]: Upgrade: First pull of quay.ceph.io/ceph-ci/ceph:324baf8ddb91dea46cac50cb535c306115791355 2024-04-12T20:53:53.286 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/mon.smithi018/config 2024-04-12T20:53:54.280 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:53:54 smithi186 ceph-mon[36597]: pgmap v278: 65 pgs: 65 active+clean; 590 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-12T20:53:54.312 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:53:54 smithi018 ceph-mon[28799]: pgmap v278: 65 pgs: 65 active+clean; 590 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-12T20:53:56.280 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:53:56 smithi186 ceph-mon[36597]: pgmap v279: 65 pgs: 65 active+clean; 590 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-12T20:53:56.281 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:53:56 smithi186 ceph-mon[36597]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' 2024-04-12T20:53:56.281 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:53:56 smithi186 ceph-mon[36597]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-12T20:53:56.281 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:53:56 smithi186 ceph-mon[36597]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' 2024-04-12T20:53:56.323 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:53:56 smithi018 ceph-mon[28799]: pgmap v279: 65 pgs: 65 active+clean; 590 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-12T20:53:56.323 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:53:56 smithi018 ceph-mon[28799]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' 2024-04-12T20:53:56.323 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:53:56 smithi018 ceph-mon[28799]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-12T20:53:56.323 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:53:56 smithi018 ceph-mon[28799]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' 2024-04-12T20:53:56.847 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:53:56.847+0000 7f0cb3e00700 1 -- 172.21.15.18:0/1748539597 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0cac0fe0d0 msgr2=0x7f0cac0fe540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:53:56.847 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:53:56.847+0000 7f0cb3e00700 1 --2- 172.21.15.18:0/1748539597 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0cac0fe0d0 0x7f0cac0fe540 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f0c98004940 tx=0x7f0c98009e30 comp rx=0 tx=0).stop 2024-04-12T20:53:56.847 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:53:56.849+0000 7f0cb3e00700 1 -- 172.21.15.18:0/1748539597 shutdown_connections 2024-04-12T20:53:56.847 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:53:56.849+0000 7f0cb3e00700 1 --2- 172.21.15.18:0/1748539597 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0cac0fe0d0 0x7f0cac0fe540 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:53:56.847 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:53:56.849+0000 7f0cb3e00700 1 --2- 172.21.15.18:0/1748539597 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0cac0fcee0 0x7f0cac0fd310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:53:56.847 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:53:56.849+0000 7f0cb3e00700 1 -- 172.21.15.18:0/1748539597 >> 172.21.15.18:0/1748539597 conn(0x7f0cac0f8560 msgr2=0x7f0cac0fa970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T20:53:56.847 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:53:56.849+0000 7f0cb3e00700 1 -- 172.21.15.18:0/1748539597 shutdown_connections 2024-04-12T20:53:56.847 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:53:56.850+0000 7f0cb3e00700 1 -- 172.21.15.18:0/1748539597 wait complete. 2024-04-12T20:53:56.848 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:53:56.851+0000 7f0cb3e00700 1 Processor -- start 2024-04-12T20:53:56.848 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:53:56.851+0000 7f0cb3e00700 1 -- start start 2024-04-12T20:53:56.848 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:53:56.851+0000 7f0cb3e00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0cac0fcee0 0x7f0cac10fd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T20:53:56.849 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:53:56.851+0000 7f0cb3e00700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0cac0fe0d0 0x7f0cac110250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T20:53:56.849 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:53:56.851+0000 7f0cb3e00700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0cac110850 con 0x7f0cac0fcee0 2024-04-12T20:53:56.850 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:53:56.851+0000 7f0cb3e00700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0cac110f90 con 0x7f0cac0fe0d0 2024-04-12T20:53:56.850 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:53:56.852+0000 7f0cb0a00700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0cac0fe0d0 0x7f0cac110250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T20:53:56.850 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:53:56.852+0000 7f0cb0a00700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0cac0fe0d0 0x7f0cac110250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.18:33160/0 (socket says 172.21.15.18:33160) 2024-04-12T20:53:56.850 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:53:56.852+0000 7f0cb0a00700 1 -- 172.21.15.18:0/4259499559 learned_addr learned my addr 172.21.15.18:0/4259499559 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T20:53:56.850 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:53:56.852+0000 7f0cb0a00700 1 -- 172.21.15.18:0/4259499559 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0cac0fcee0 msgr2=0x7f0cac10fd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:53:56.851 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:53:56.852+0000 7f0cb0a00700 1 --2- 172.21.15.18:0/4259499559 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0cac0fcee0 0x7f0cac10fd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:53:56.851 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:53:56.852+0000 7f0cb0a00700 1 -- 172.21.15.18:0/4259499559 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c98005040 con 0x7f0cac0fe0d0 2024-04-12T20:53:56.851 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:53:56.852+0000 7f0cb0a00700 1 --2- 172.21.15.18:0/4259499559 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0cac0fe0d0 0x7f0cac110250 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f0c9800b3b0 tx=0x7f0c98009e30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T20:53:56.851 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:53:56.853+0000 7f0ca9600700 1 -- 172.21.15.18:0/4259499559 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0c9800e040 con 0x7f0cac0fe0d0 2024-04-12T20:53:56.851 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:53:56.853+0000 7f0ca9600700 1 -- 172.21.15.18:0/4259499559 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f0c98004290 con 0x7f0cac0fe0d0 2024-04-12T20:53:56.851 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:53:56.853+0000 7f0cb3e00700 1 -- 172.21.15.18:0/4259499559 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0cac111210 con 0x7f0cac0fe0d0 2024-04-12T20:53:56.851 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:53:56.853+0000 7f0cb3e00700 1 -- 172.21.15.18:0/4259499559 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0cac1117d0 con 0x7f0cac0fe0d0 2024-04-12T20:53:56.852 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:53:56.853+0000 7f0ca9600700 1 -- 172.21.15.18:0/4259499559 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0c9801c630 con 0x7f0cac0fe0d0 2024-04-12T20:53:56.853 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:53:56.856+0000 7f0cb3e00700 1 -- 172.21.15.18:0/4259499559 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0c8c0052e0 con 0x7f0cac0fe0d0 2024-04-12T20:53:56.854 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:53:56.856+0000 7f0ca9600700 1 -- 172.21.15.18:0/4259499559 <== mon.1 v2:172.21.15.186:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7f0c9800bbf0 con 0x7f0cac0fe0d0 2024-04-12T20:53:56.855 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:53:56.857+0000 7f0ca9600700 1 --2- 172.21.15.18:0/4259499559 >> [v2:172.21.15.18:6800/3048894687,v1:172.21.15.18:6801/3048894687] conn(0x7f0c940692b0 0x7f0c9406b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T20:53:56.855 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:53:56.858+0000 7f0ca9600700 1 -- 172.21.15.18:0/4259499559 <== mon.1 v2:172.21.15.186:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5550+0+0 (secure 0 0 0) 0x7f0c98088570 con 0x7f0cac0fe0d0 2024-04-12T20:53:56.855 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:53:56.858+0000 7f0cb1400700 1 --2- 172.21.15.18:0/4259499559 >> [v2:172.21.15.18:6800/3048894687,v1:172.21.15.18:6801/3048894687] conn(0x7f0c940692b0 0x7f0c9406b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T20:53:56.856 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:53:56.859+0000 7f0cb1400700 1 --2- 172.21.15.18:0/4259499559 >> [v2:172.21.15.18:6800/3048894687,v1:172.21.15.18:6801/3048894687] conn(0x7f0c940692b0 0x7f0c9406b760 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f0cac0fcc20 tx=0x7f0ca0009ce0 comp rx=0 tx=0).ready entity=mgr.14212 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T20:53:56.863 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:53:56.866+0000 7f0ca9600700 1 -- 172.21.15.18:0/4259499559 <== mon.1 v2:172.21.15.186:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7f0c980582c0 con 0x7f0cac0fe0d0 2024-04-12T20:53:57.022 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:53:57 smithi018 ceph-mon[28799]: Upgrade: Target is version 19.0.0-2879-g324baf8d (unknown) 2024-04-12T20:53:57.022 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:53:57 smithi018 ceph-mon[28799]: Upgrade: Target container is quay.ceph.io/ceph-ci/ceph@sha256:f4452def3be4d350f4a8e5feaf7ebb1511be9298fe68478ba7a2d6fbd7d69a63, digests ['quay.ceph.io/ceph-ci/ceph@sha256:f4452def3be4d350f4a8e5feaf7ebb1511be9298fe68478ba7a2d6fbd7d69a63'] 2024-04-12T20:53:57.022 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:53:57 smithi018 ceph-mon[28799]: Upgrade: Need to upgrade myself (mgr.smithi018.xjzwga) 2024-04-12T20:53:57.022 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:53:57 smithi018 ceph-mon[28799]: Upgrade: Pulling quay.ceph.io/ceph-ci/ceph@sha256:f4452def3be4d350f4a8e5feaf7ebb1511be9298fe68478ba7a2d6fbd7d69a63 on smithi186 2024-04-12T20:53:57.193 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:53:57.196+0000 7f0cb3e00700 1 -- 172.21.15.18:0/4259499559 --> [v2:172.21.15.18:6800/3048894687,v1:172.21.15.18:6801/3048894687] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f0c8c000bf0 con 0x7f0c940692b0 2024-04-12T20:53:57.195 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:53:57.197+0000 7f0ca9600700 1 -- 172.21.15.18:0/4259499559 <== mgr.14212 v2:172.21.15.18:6800/3048894687 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+439 (secure 0 0 0) 0x7f0c8c000bf0 con 0x7f0c940692b0 2024-04-12T20:53:57.198 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:53:57.200+0000 7f0cb3e00700 1 -- 172.21.15.18:0/4259499559 >> [v2:172.21.15.18:6800/3048894687,v1:172.21.15.18:6801/3048894687] conn(0x7f0c940692b0 msgr2=0x7f0c9406b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:53:57.198 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:53:57.200+0000 7f0cb3e00700 1 --2- 172.21.15.18:0/4259499559 >> [v2:172.21.15.18:6800/3048894687,v1:172.21.15.18:6801/3048894687] conn(0x7f0c940692b0 0x7f0c9406b760 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f0cac0fcc20 tx=0x7f0ca0009ce0 comp rx=0 tx=0).stop 2024-04-12T20:53:57.198 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:53:57.201+0000 7f0cb3e00700 1 -- 172.21.15.18:0/4259499559 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0cac0fe0d0 msgr2=0x7f0cac110250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:53:57.198 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:53:57.201+0000 7f0cb3e00700 1 --2- 172.21.15.18:0/4259499559 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0cac0fe0d0 0x7f0cac110250 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f0c9800b3b0 tx=0x7f0c98009e30 comp rx=0 tx=0).stop 2024-04-12T20:53:57.199 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:53:57.201+0000 7f0cb3e00700 1 -- 172.21.15.18:0/4259499559 shutdown_connections 2024-04-12T20:53:57.199 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:53:57.201+0000 7f0cb3e00700 1 --2- 172.21.15.18:0/4259499559 >> [v2:172.21.15.18:6800/3048894687,v1:172.21.15.18:6801/3048894687] conn(0x7f0c940692b0 0x7f0c9406b760 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:53:57.199 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:53:57.201+0000 7f0cb3e00700 1 --2- 172.21.15.18:0/4259499559 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0cac0fcee0 0x7f0cac10fd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:53:57.199 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:53:57.201+0000 7f0cb3e00700 1 --2- 172.21.15.18:0/4259499559 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0cac0fe0d0 0x7f0cac110250 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:53:57.199 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:53:57.201+0000 7f0cb3e00700 1 -- 172.21.15.18:0/4259499559 >> 172.21.15.18:0/4259499559 conn(0x7f0cac0f8560 msgr2=0x7f0cac0f9d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T20:53:57.199 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:53:57.201+0000 7f0cb3e00700 1 -- 172.21.15.18:0/4259499559 shutdown_connections 2024-04-12T20:53:57.199 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:53:57.201+0000 7f0cb3e00700 1 -- 172.21.15.18:0/4259499559 wait complete. 2024-04-12T20:53:57.214 INFO:teuthology.orchestra.run.smithi018.stdout:true 2024-04-12T20:53:57.280 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:53:57 smithi186 ceph-mon[36597]: Upgrade: Target is version 19.0.0-2879-g324baf8d (unknown) 2024-04-12T20:53:57.281 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:53:57 smithi186 ceph-mon[36597]: Upgrade: Target container is quay.ceph.io/ceph-ci/ceph@sha256:f4452def3be4d350f4a8e5feaf7ebb1511be9298fe68478ba7a2d6fbd7d69a63, digests ['quay.ceph.io/ceph-ci/ceph@sha256:f4452def3be4d350f4a8e5feaf7ebb1511be9298fe68478ba7a2d6fbd7d69a63'] 2024-04-12T20:53:57.281 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:53:57 smithi186 ceph-mon[36597]: Upgrade: Need to upgrade myself (mgr.smithi018.xjzwga) 2024-04-12T20:53:57.281 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:53:57 smithi186 ceph-mon[36597]: Upgrade: Pulling quay.ceph.io/ceph-ci/ceph@sha256:f4452def3be4d350f4a8e5feaf7ebb1511be9298fe68478ba7a2d6fbd7d69a63 on smithi186 2024-04-12T20:53:57.350 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:53:57.352+0000 7fa4bce00700 1 -- 172.21.15.18:0/475005524 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa4b80fcee0 msgr2=0x7fa4b80fd310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:53:57.351 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:53:57.352+0000 7fa4bce00700 1 --2- 172.21.15.18:0/475005524 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa4b80fcee0 0x7fa4b80fd310 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fa4a8004940 tx=0x7fa4a8009e30 comp rx=0 tx=0).stop 2024-04-12T20:53:57.351 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:53:57.353+0000 7fa4bce00700 1 -- 172.21.15.18:0/475005524 shutdown_connections 2024-04-12T20:53:57.351 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:53:57.353+0000 7fa4bce00700 1 --2- 172.21.15.18:0/475005524 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa4b80fe0d0 0x7fa4b80fe540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:53:57.351 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:53:57.353+0000 7fa4bce00700 1 --2- 172.21.15.18:0/475005524 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa4b80fcee0 0x7fa4b80fd310 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:53:57.351 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:53:57.353+0000 7fa4bce00700 1 -- 172.21.15.18:0/475005524 >> 172.21.15.18:0/475005524 conn(0x7fa4b80f8560 msgr2=0x7fa4b80fa970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T20:53:57.351 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:53:57.353+0000 7fa4bce00700 1 -- 172.21.15.18:0/475005524 shutdown_connections 2024-04-12T20:53:57.351 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:53:57.353+0000 7fa4bce00700 1 -- 172.21.15.18:0/475005524 wait complete. 2024-04-12T20:53:57.352 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:53:57.354+0000 7fa4bce00700 1 Processor -- start 2024-04-12T20:53:57.352 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:53:57.354+0000 7fa4bce00700 1 -- start start 2024-04-12T20:53:57.352 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:53:57.355+0000 7fa4bce00700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa4b80fcee0 0x7fa4b810fd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T20:53:57.353 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:53:57.355+0000 7fa4bce00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa4b80fe0d0 0x7fa4b8110260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T20:53:57.353 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:53:57.355+0000 7fa4bce00700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4b8110860 con 0x7fa4b80fe0d0 2024-04-12T20:53:57.353 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:53:57.355+0000 7fa4bce00700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4b8110fa0 con 0x7fa4b80fcee0 2024-04-12T20:53:57.353 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:53:57.355+0000 7fa4b5e00700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa4b80fcee0 0x7fa4b810fd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T20:53:57.353 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:53:57.355+0000 7fa4b5e00700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa4b80fcee0 0x7fa4b810fd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.18:33178/0 (socket says 172.21.15.18:33178) 2024-04-12T20:53:57.353 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:53:57.355+0000 7fa4b5e00700 1 -- 172.21.15.18:0/1015348862 learned_addr learned my addr 172.21.15.18:0/1015348862 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T20:53:57.353 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:53:57.355+0000 7fa4b5400700 1 --2- 172.21.15.18:0/1015348862 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa4b80fe0d0 0x7fa4b8110260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T20:53:57.354 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:53:57.356+0000 7fa4b5400700 1 -- 172.21.15.18:0/1015348862 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa4b80fcee0 msgr2=0x7fa4b810fd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:53:57.354 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:53:57.356+0000 7fa4b5400700 1 --2- 172.21.15.18:0/1015348862 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa4b80fcee0 0x7fa4b810fd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:53:57.354 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:53:57.356+0000 7fa4b5400700 1 -- 172.21.15.18:0/1015348862 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4a8005040 con 0x7fa4b80fe0d0 2024-04-12T20:53:57.354 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:53:57.357+0000 7fa4b5400700 1 --2- 172.21.15.18:0/1015348862 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa4b80fe0d0 0x7fa4b8110260 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fa4a4002820 tx=0x7fa4a4004570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T20:53:57.355 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:53:57.357+0000 7fa4ae000700 1 -- 172.21.15.18:0/1015348862 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa4a4007cc0 con 0x7fa4b80fe0d0 2024-04-12T20:53:57.355 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:53:57.357+0000 7fa4ae000700 1 -- 172.21.15.18:0/1015348862 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa4a400f040 con 0x7fa4b80fe0d0 2024-04-12T20:53:57.355 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:53:57.357+0000 7fa4ae000700 1 -- 172.21.15.18:0/1015348862 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa4a4013920 con 0x7fa4b80fe0d0 2024-04-12T20:53:57.355 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:53:57.357+0000 7fa4bce00700 1 -- 172.21.15.18:0/1015348862 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa4b8111280 con 0x7fa4b80fe0d0 2024-04-12T20:53:57.356 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:53:57.357+0000 7fa4bce00700 1 -- 172.21.15.18:0/1015348862 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa4b8111840 con 0x7fa4b80fe0d0 2024-04-12T20:53:57.356 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:53:57.359+0000 7fa4bce00700 1 -- 172.21.15.18:0/1015348862 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa4b8101a60 con 0x7fa4b80fe0d0 2024-04-12T20:53:57.358 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:53:57.361+0000 7fa4ae000700 1 -- 172.21.15.18:0/1015348862 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7fa4a4013aa0 con 0x7fa4b80fe0d0 2024-04-12T20:53:57.359 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:53:57.362+0000 7fa4ae000700 1 --2- 172.21.15.18:0/1015348862 >> [v2:172.21.15.18:6800/3048894687,v1:172.21.15.18:6801/3048894687] conn(0x7fa49c0692b0 0x7fa49c06b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T20:53:57.360 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:53:57.362+0000 7fa4ae000700 1 -- 172.21.15.18:0/1015348862 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5550+0+0 (secure 0 0 0) 0x7fa4a40874e0 con 0x7fa4b80fe0d0 2024-04-12T20:53:57.360 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:53:57.363+0000 7fa4b5e00700 1 --2- 172.21.15.18:0/1015348862 >> [v2:172.21.15.18:6800/3048894687,v1:172.21.15.18:6801/3048894687] conn(0x7fa49c0692b0 0x7fa49c06b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T20:53:57.361 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:53:57.363+0000 7fa4b5e00700 1 --2- 172.21.15.18:0/1015348862 >> [v2:172.21.15.18:6800/3048894687,v1:172.21.15.18:6801/3048894687] conn(0x7fa49c0692b0 0x7fa49c06b760 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fa4a8009fd0 tx=0x7fa4a8009e30 comp rx=0 tx=0).ready entity=mgr.14212 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T20:53:57.366 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:53:57.368+0000 7fa4ae000700 1 -- 172.21.15.18:0/1015348862 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7fa4a4057230 con 0x7fa4b80fe0d0 2024-04-12T20:53:57.629 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:53:57.632+0000 7fa4bce00700 1 -- 172.21.15.18:0/1015348862 --> [v2:172.21.15.18:6800/3048894687,v1:172.21.15.18:6801/3048894687] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fa4b8111b20 con 0x7fa49c0692b0 2024-04-12T20:53:57.631 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:53:57.634+0000 7fa4ae000700 1 -- 172.21.15.18:0/1015348862 <== mgr.14212 v2:172.21.15.18:6800/3048894687 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+439 (secure 0 0 0) 0x7fa4b8111b20 con 0x7fa49c0692b0 2024-04-12T20:53:57.634 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:53:57.637+0000 7fa4bce00700 1 -- 172.21.15.18:0/1015348862 >> [v2:172.21.15.18:6800/3048894687,v1:172.21.15.18:6801/3048894687] conn(0x7fa49c0692b0 msgr2=0x7fa49c06b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:53:57.634 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:53:57.637+0000 7fa4bce00700 1 --2- 172.21.15.18:0/1015348862 >> [v2:172.21.15.18:6800/3048894687,v1:172.21.15.18:6801/3048894687] conn(0x7fa49c0692b0 0x7fa49c06b760 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fa4a8009fd0 tx=0x7fa4a8009e30 comp rx=0 tx=0).stop 2024-04-12T20:53:57.634 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:53:57.637+0000 7fa4bce00700 1 -- 172.21.15.18:0/1015348862 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa4b80fe0d0 msgr2=0x7fa4b8110260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:53:57.635 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:53:57.637+0000 7fa4bce00700 1 --2- 172.21.15.18:0/1015348862 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa4b80fe0d0 0x7fa4b8110260 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fa4a4002820 tx=0x7fa4a4004570 comp rx=0 tx=0).stop 2024-04-12T20:53:57.635 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:53:57.637+0000 7fa4bce00700 1 -- 172.21.15.18:0/1015348862 shutdown_connections 2024-04-12T20:53:57.635 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:53:57.637+0000 7fa4bce00700 1 --2- 172.21.15.18:0/1015348862 >> [v2:172.21.15.18:6800/3048894687,v1:172.21.15.18:6801/3048894687] conn(0x7fa49c0692b0 0x7fa49c06b760 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:53:57.635 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:53:57.637+0000 7fa4bce00700 1 --2- 172.21.15.18:0/1015348862 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa4b80fcee0 0x7fa4b810fd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:53:57.635 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:53:57.637+0000 7fa4bce00700 1 --2- 172.21.15.18:0/1015348862 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa4b80fe0d0 0x7fa4b8110260 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:53:57.635 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:53:57.637+0000 7fa4bce00700 1 -- 172.21.15.18:0/1015348862 >> 172.21.15.18:0/1015348862 conn(0x7fa4b80f8560 msgr2=0x7fa4b80f9c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T20:53:57.636 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:53:57.638+0000 7fa4bce00700 1 -- 172.21.15.18:0/1015348862 shutdown_connections 2024-04-12T20:53:57.636 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:53:57.638+0000 7fa4bce00700 1 -- 172.21.15.18:0/1015348862 wait complete. 2024-04-12T20:53:57.818 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:53:57.820+0000 7f15c4c00700 1 -- 172.21.15.18:0/190566273 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f15c0071e40 msgr2=0x7f15c0070350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:53:57.818 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:53:57.820+0000 7f15c4c00700 1 --2- 172.21.15.18:0/190566273 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f15c0071e40 0x7f15c0070350 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f15b0004940 tx=0x7f15b0009e30 comp rx=0 tx=0).stop 2024-04-12T20:53:57.818 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:53:57.820+0000 7f15c4c00700 1 -- 172.21.15.18:0/190566273 shutdown_connections 2024-04-12T20:53:57.818 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:53:57.820+0000 7f15c4c00700 1 --2- 172.21.15.18:0/190566273 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f15c0070890 0x7f15c0070d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:53:57.818 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:53:57.820+0000 7f15c4c00700 1 --2- 172.21.15.18:0/190566273 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f15c0071e40 0x7f15c0070350 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:53:57.819 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:53:57.820+0000 7f15c4c00700 1 -- 172.21.15.18:0/190566273 >> 172.21.15.18:0/190566273 conn(0x7f15c00f8540 msgr2=0x7f15c00fa970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T20:53:57.819 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:53:57.820+0000 7f15c4c00700 1 -- 172.21.15.18:0/190566273 shutdown_connections 2024-04-12T20:53:57.819 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:53:57.821+0000 7f15c4c00700 1 -- 172.21.15.18:0/190566273 wait complete. 2024-04-12T20:53:57.820 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:53:57.821+0000 7f15c4c00700 1 Processor -- start 2024-04-12T20:53:57.820 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:53:57.821+0000 7f15c4c00700 1 -- start start 2024-04-12T20:53:57.820 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:53:57.822+0000 7f15c4c00700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f15c0070890 0x7f15c010fc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T20:53:57.820 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:53:57.822+0000 7f15c4c00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f15c0071e40 0x7f15c0110150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T20:53:57.820 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:53:57.822+0000 7f15c4c00700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15c0110750 con 0x7f15c0071e40 2024-04-12T20:53:57.820 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:53:57.822+0000 7f15c4c00700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15c0110fa0 con 0x7f15c0070890 2024-04-12T20:53:57.820 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:53:57.822+0000 7f15bd400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f15c0071e40 0x7f15c0110150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T20:53:57.820 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:53:57.822+0000 7f15bd400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f15c0071e40 0x7f15c0110150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:59636/0 (socket says 172.21.15.18:59636) 2024-04-12T20:53:57.820 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:53:57.822+0000 7f15bd400700 1 -- 172.21.15.18:0/1188886551 learned_addr learned my addr 172.21.15.18:0/1188886551 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T20:53:57.821 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:53:57.823+0000 7f15bde00700 1 --2- 172.21.15.18:0/1188886551 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f15c0070890 0x7f15c010fc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T20:53:57.822 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:53:57.823+0000 7f15bd400700 1 -- 172.21.15.18:0/1188886551 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f15c0070890 msgr2=0x7f15c010fc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:53:57.822 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:53:57.823+0000 7f15bd400700 1 --2- 172.21.15.18:0/1188886551 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f15c0070890 0x7f15c010fc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:53:57.822 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:53:57.823+0000 7f15bd400700 1 -- 172.21.15.18:0/1188886551 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15b0005040 con 0x7f15c0071e40 2024-04-12T20:53:57.822 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:53:57.823+0000 7f15bde00700 1 --2- 172.21.15.18:0/1188886551 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f15c0070890 0x7f15c010fc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-12T20:53:57.822 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:53:57.823+0000 7f15bd400700 1 --2- 172.21.15.18:0/1188886551 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f15c0071e40 0x7f15c0110150 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f15ac00eb40 tx=0x7f15ac00b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T20:53:57.822 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:53:57.824+0000 7f15b6000700 1 -- 172.21.15.18:0/1188886551 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f15ac007cc0 con 0x7f15c0071e40 2024-04-12T20:53:57.822 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:53:57.824+0000 7f15c4c00700 1 -- 172.21.15.18:0/1188886551 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f15c0111220 con 0x7f15c0071e40 2024-04-12T20:53:57.822 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:53:57.824+0000 7f15c4c00700 1 -- 172.21.15.18:0/1188886551 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f15c0111760 con 0x7f15c0071e40 2024-04-12T20:53:57.822 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:53:57.824+0000 7f15b6000700 1 -- 172.21.15.18:0/1188886551 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f15ac00f040 con 0x7f15c0071e40 2024-04-12T20:53:57.822 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:53:57.824+0000 7f15b6000700 1 -- 172.21.15.18:0/1188886551 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f15ac009620 con 0x7f15c0071e40 2024-04-12T20:53:57.823 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:53:57.826+0000 7f15c4c00700 1 -- 172.21.15.18:0/1188886551 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f15a00052e0 con 0x7f15c0071e40 2024-04-12T20:53:57.824 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:53:57.827+0000 7f15b6000700 1 -- 172.21.15.18:0/1188886551 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7f15ac009780 con 0x7f15c0071e40 2024-04-12T20:53:57.825 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:53:57.828+0000 7f15b6000700 1 --2- 172.21.15.18:0/1188886551 >> [v2:172.21.15.18:6800/3048894687,v1:172.21.15.18:6801/3048894687] conn(0x7f15a4069360 0x7f15a406b810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T20:53:57.825 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:53:57.828+0000 7f15b6000700 1 -- 172.21.15.18:0/1188886551 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5550+0+0 (secure 0 0 0) 0x7f15ac086e50 con 0x7f15c0071e40 2024-04-12T20:53:57.826 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:53:57.829+0000 7f15bde00700 1 --2- 172.21.15.18:0/1188886551 >> [v2:172.21.15.18:6800/3048894687,v1:172.21.15.18:6801/3048894687] conn(0x7f15a4069360 0x7f15a406b810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T20:53:57.829 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:53:57.831+0000 7f15bde00700 1 --2- 172.21.15.18:0/1188886551 >> [v2:172.21.15.18:6800/3048894687,v1:172.21.15.18:6801/3048894687] conn(0x7f15a4069360 0x7f15a406b810 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f15b000b0d0 tx=0x7f15b0009e30 comp rx=0 tx=0).ready entity=mgr.14212 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T20:53:57.834 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:53:57.836+0000 7f15b6000700 1 -- 172.21.15.18:0/1188886551 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7f15ac04e190 con 0x7f15c0071e40 2024-04-12T20:53:58.185 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:53:58.187+0000 7f15c4c00700 1 -- 172.21.15.18:0/1188886551 --> [v2:172.21.15.18:6800/3048894687,v1:172.21.15.18:6801/3048894687] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f15a0000bf0 con 0x7f15a4069360 2024-04-12T20:53:58.202 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:53:58.204+0000 7f15b6000700 1 -- 172.21.15.18:0/1188886551 <== mgr.14212 v2:172.21.15.18:6800/3048894687 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3410 (secure 0 0 0) 0x7f15a0000bf0 con 0x7f15a4069360 2024-04-12T20:53:58.202 INFO:teuthology.orchestra.run.smithi018.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-12T20:53:58.202 INFO:teuthology.orchestra.run.smithi018.stdout:alertmanager.smithi018 smithi018 *:9093,9094 running (7m) 82s ago 9m 13.1M - 0.25.0 c8568f914cd2 f43ec346f9e5 2024-04-12T20:53:58.202 INFO:teuthology.orchestra.run.smithi018.stdout:crash.smithi018 smithi018 running (9m) 82s ago 9m 6660k - 17.2.7-827-gd3b5060a 21de9b7369ad 1ac7603bddfd 2024-04-12T20:53:58.202 INFO:teuthology.orchestra.run.smithi018.stdout:crash.smithi186 smithi186 running (7m) 88s ago 7m 6669k - 17.2.7-827-gd3b5060a 21de9b7369ad 14bf8a1672eb 2024-04-12T20:53:58.202 INFO:teuthology.orchestra.run.smithi018.stdout:grafana.smithi018 smithi018 *:3000 running (6m) 82s ago 8m 73.7M - 9.4.7 954c08fa6188 a7c3fe73c4d7 2024-04-12T20:53:58.203 INFO:teuthology.orchestra.run.smithi018.stdout:mds.cephfs.smithi018.acqbsc smithi018 running (116s) 82s ago 115s 13.5M - 17.2.7-827-gd3b5060a 21de9b7369ad 5b5fce65f18a 2024-04-12T20:53:58.203 INFO:teuthology.orchestra.run.smithi018.stdout:mds.cephfs.smithi018.szjluo smithi018 running (108s) 82s ago 108s 12.8M - 17.2.7-827-gd3b5060a 21de9b7369ad cf9bf0c5031f 2024-04-12T20:53:58.203 INFO:teuthology.orchestra.run.smithi018.stdout:mds.cephfs.smithi186.jdllyn smithi186 running (112s) 88s ago 112s 10.4M - 17.2.7-827-gd3b5060a 21de9b7369ad 2b2bc46bd29a 2024-04-12T20:53:58.203 INFO:teuthology.orchestra.run.smithi018.stdout:mds.cephfs.smithi186.wjnafs smithi186 running (104s) 88s ago 104s 10.5M - 17.2.7-827-gd3b5060a 21de9b7369ad 74b115d6d6fd 2024-04-12T20:53:58.203 INFO:teuthology.orchestra.run.smithi018.stdout:mgr.smithi018.xjzwga smithi018 *:9283 running (10m) 82s ago 10m 481M - 17.2.7-827-gd3b5060a 21de9b7369ad c72f43857311 2024-04-12T20:53:58.203 INFO:teuthology.orchestra.run.smithi018.stdout:mgr.smithi186.kqcfxk smithi186 *:8443,9283 running (7m) 88s ago 7m 432M - 17.2.7-827-gd3b5060a 21de9b7369ad 2c790daf3cc7 2024-04-12T20:53:58.203 INFO:teuthology.orchestra.run.smithi018.stdout:mon.smithi018 smithi018 running (10m) 82s ago 10m 41.1M 2048M 17.2.7-827-gd3b5060a 21de9b7369ad cd84caafa056 2024-04-12T20:53:58.203 INFO:teuthology.orchestra.run.smithi018.stdout:mon.smithi186 smithi186 running (7m) 88s ago 7m 35.0M 2048M 17.2.7-827-gd3b5060a 21de9b7369ad d7263c772b73 2024-04-12T20:53:58.203 INFO:teuthology.orchestra.run.smithi018.stdout:node-exporter.smithi018 smithi018 *:9100 running (9m) 82s ago 9m 7625k - 1.5.0 0da6a335fe13 a23e90d473fe 2024-04-12T20:53:58.203 INFO:teuthology.orchestra.run.smithi018.stdout:node-exporter.smithi186 smithi186 *:9100 running (7m) 88s ago 7m 7382k - 1.5.0 0da6a335fe13 57160d1060f2 2024-04-12T20:53:58.203 INFO:teuthology.orchestra.run.smithi018.stdout:osd.0 smithi018 running (6m) 82s ago 6m 44.4M 4290M 17.2.7-827-gd3b5060a 21de9b7369ad 0d83e027bf0a 2024-04-12T20:53:58.203 INFO:teuthology.orchestra.run.smithi018.stdout:osd.1 smithi018 running (6m) 82s ago 6m 47.9M 4290M 17.2.7-827-gd3b5060a 21de9b7369ad 6517c9cc7f87 2024-04-12T20:53:58.203 INFO:teuthology.orchestra.run.smithi018.stdout:osd.2 smithi018 running (5m) 82s ago 5m 44.3M 4290M 17.2.7-827-gd3b5060a 21de9b7369ad 8a83692acaa5 2024-04-12T20:53:58.203 INFO:teuthology.orchestra.run.smithi018.stdout:osd.3 smithi186 running (4m) 88s ago 4m 43.4M 5312M 17.2.7-827-gd3b5060a 21de9b7369ad c2def7a90d76 2024-04-12T20:53:58.204 INFO:teuthology.orchestra.run.smithi018.stdout:osd.4 smithi186 running (4m) 88s ago 4m 41.0M 5312M 17.2.7-827-gd3b5060a 21de9b7369ad 757e7c71b7b9 2024-04-12T20:53:58.204 INFO:teuthology.orchestra.run.smithi018.stdout:osd.5 smithi186 running (3m) 88s ago 3m 40.8M 5312M 17.2.7-827-gd3b5060a 21de9b7369ad 2ee072a436d0 2024-04-12T20:53:58.204 INFO:teuthology.orchestra.run.smithi018.stdout:prometheus.smithi018 smithi018 *:9095 running (6m) 82s ago 8m 46.7M - 2.43.0 a07b618ecd1d aecd046db448 2024-04-12T20:53:58.205 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:53:58.208+0000 7f15c4c00700 1 -- 172.21.15.18:0/1188886551 >> [v2:172.21.15.18:6800/3048894687,v1:172.21.15.18:6801/3048894687] conn(0x7f15a4069360 msgr2=0x7f15a406b810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:53:58.206 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:53:58.208+0000 7f15c4c00700 1 --2- 172.21.15.18:0/1188886551 >> [v2:172.21.15.18:6800/3048894687,v1:172.21.15.18:6801/3048894687] conn(0x7f15a4069360 0x7f15a406b810 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f15b000b0d0 tx=0x7f15b0009e30 comp rx=0 tx=0).stop 2024-04-12T20:53:58.206 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:53:58.208+0000 7f15c4c00700 1 -- 172.21.15.18:0/1188886551 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f15c0071e40 msgr2=0x7f15c0110150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:53:58.206 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:53:58.208+0000 7f15c4c00700 1 --2- 172.21.15.18:0/1188886551 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f15c0071e40 0x7f15c0110150 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f15ac00eb40 tx=0x7f15ac00b040 comp rx=0 tx=0).stop 2024-04-12T20:53:58.206 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:53:58.208+0000 7f15c4c00700 1 -- 172.21.15.18:0/1188886551 shutdown_connections 2024-04-12T20:53:58.206 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:53:58.208+0000 7f15c4c00700 1 --2- 172.21.15.18:0/1188886551 >> [v2:172.21.15.18:6800/3048894687,v1:172.21.15.18:6801/3048894687] conn(0x7f15a4069360 0x7f15a406b810 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:53:58.207 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:53:58.208+0000 7f15c4c00700 1 --2- 172.21.15.18:0/1188886551 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f15c0070890 0x7f15c010fc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:53:58.207 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:53:58.208+0000 7f15c4c00700 1 --2- 172.21.15.18:0/1188886551 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f15c0071e40 0x7f15c0110150 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:53:58.207 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:53:58.208+0000 7f15c4c00700 1 -- 172.21.15.18:0/1188886551 >> 172.21.15.18:0/1188886551 conn(0x7f15c00f8540 msgr2=0x7f15c00f9c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T20:53:58.207 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:53:58.209+0000 7f15c4c00700 1 -- 172.21.15.18:0/1188886551 shutdown_connections 2024-04-12T20:53:58.207 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:53:58.209+0000 7f15c4c00700 1 -- 172.21.15.18:0/1188886551 wait complete. 2024-04-12T20:53:58.280 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:53:58 smithi186 ceph-mon[36597]: pgmap v280: 65 pgs: 65 active+clean; 590 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-12T20:53:58.351 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:53:58.353+0000 7f1ab3e00700 1 -- 172.21.15.18:0/2066696427 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1ab4071e40 msgr2=0x7f1ab4070350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:53:58.351 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:53:58.353+0000 7f1ab3e00700 1 --2- 172.21.15.18:0/2066696427 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1ab4071e40 0x7f1ab4070350 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f1aa4004940 tx=0x7f1aa4009e30 comp rx=0 tx=0).stop 2024-04-12T20:53:58.351 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:53:58.353+0000 7f1ab3e00700 1 -- 172.21.15.18:0/2066696427 shutdown_connections 2024-04-12T20:53:58.352 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:53:58.353+0000 7f1ab3e00700 1 --2- 172.21.15.18:0/2066696427 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1ab4070890 0x7f1ab4070d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:53:58.352 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:53:58.353+0000 7f1ab3e00700 1 --2- 172.21.15.18:0/2066696427 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1ab4071e40 0x7f1ab4070350 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:53:58.352 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:53:58.354+0000 7f1ab3e00700 1 -- 172.21.15.18:0/2066696427 >> 172.21.15.18:0/2066696427 conn(0x7f1ab40f83d0 msgr2=0x7f1ab40fa7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T20:53:58.352 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:53:58 smithi018 ceph-mon[28799]: pgmap v280: 65 pgs: 65 active+clean; 590 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-12T20:53:58.352 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:53:58.354+0000 7f1ab3e00700 1 -- 172.21.15.18:0/2066696427 shutdown_connections 2024-04-12T20:53:58.352 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:53:58.354+0000 7f1ab3e00700 1 -- 172.21.15.18:0/2066696427 wait complete. 2024-04-12T20:53:58.353 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:53:58.355+0000 7f1ab3e00700 1 Processor -- start 2024-04-12T20:53:58.353 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:53:58.355+0000 7f1ab3e00700 1 -- start start 2024-04-12T20:53:58.353 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:53:58.356+0000 7f1ab3e00700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1ab4070890 0x7f1ab40fcb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T20:53:58.354 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:53:58.356+0000 7f1ab3e00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1ab4071e40 0x7f1ab40fd090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T20:53:58.354 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:53:58.356+0000 7f1ab3e00700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ab40fd6e0 con 0x7f1ab4071e40 2024-04-12T20:53:58.354 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:53:58.356+0000 7f1ab3e00700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ab40fdf30 con 0x7f1ab4070890 2024-04-12T20:53:58.354 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:53:58.356+0000 7f1ab2a00700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1ab4070890 0x7f1ab40fcb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T20:53:58.355 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:53:58.356+0000 7f1ab2a00700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1ab4070890 0x7f1ab40fcb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.18:33210/0 (socket says 172.21.15.18:33210) 2024-04-12T20:53:58.355 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:53:58.356+0000 7f1ab2a00700 1 -- 172.21.15.18:0/2772402301 learned_addr learned my addr 172.21.15.18:0/2772402301 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T20:53:58.355 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:53:58.357+0000 7f1ab2000700 1 --2- 172.21.15.18:0/2772402301 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1ab4071e40 0x7f1ab40fd090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T20:53:58.355 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:53:58.357+0000 7f1ab2a00700 1 -- 172.21.15.18:0/2772402301 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1ab4071e40 msgr2=0x7f1ab40fd090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:53:58.355 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:53:58.357+0000 7f1ab2a00700 1 --2- 172.21.15.18:0/2772402301 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1ab4071e40 0x7f1ab40fd090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:53:58.356 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:53:58.357+0000 7f1ab2a00700 1 -- 172.21.15.18:0/2772402301 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1aa4005040 con 0x7f1ab4070890 2024-04-12T20:53:58.356 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:53:58.358+0000 7f1ab2a00700 1 --2- 172.21.15.18:0/2772402301 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1ab4070890 0x7f1ab40fcb50 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f1aa400b970 tx=0x7f1aa4009e30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T20:53:58.356 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:53:58.358+0000 7f1aaaa00700 1 -- 172.21.15.18:0/2772402301 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1aa400e040 con 0x7f1ab4070890 2024-04-12T20:53:58.356 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:53:58.358+0000 7f1ab3e00700 1 -- 172.21.15.18:0/2772402301 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1ab40fe1b0 con 0x7f1ab4070890 2024-04-12T20:53:58.356 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:53:58.359+0000 7f1ab3e00700 1 -- 172.21.15.18:0/2772402301 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1ab40fe770 con 0x7f1ab4070890 2024-04-12T20:53:58.357 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:53:58.359+0000 7f1aaaa00700 1 -- 172.21.15.18:0/2772402301 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f1aa400bbf0 con 0x7f1ab4070890 2024-04-12T20:53:58.357 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:53:58.359+0000 7f1aaaa00700 1 -- 172.21.15.18:0/2772402301 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1aa401c3f0 con 0x7f1ab4070890 2024-04-12T20:53:58.357 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:53:58.360+0000 7f1ab3e00700 1 -- 172.21.15.18:0/2772402301 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1a940052e0 con 0x7f1ab4070890 2024-04-12T20:53:58.359 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:53:58.361+0000 7f1aaaa00700 1 -- 172.21.15.18:0/2772402301 <== mon.1 v2:172.21.15.186:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7f1aa4013420 con 0x7f1ab4070890 2024-04-12T20:53:58.360 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:53:58.362+0000 7f1aaaa00700 1 --2- 172.21.15.18:0/2772402301 >> [v2:172.21.15.18:6800/3048894687,v1:172.21.15.18:6801/3048894687] conn(0x7f1a980692b0 0x7f1a9806b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T20:53:58.360 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:53:58.363+0000 7f1aaaa00700 1 -- 172.21.15.18:0/2772402301 <== mon.1 v2:172.21.15.186:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5550+0+0 (secure 0 0 0) 0x7f1aa40884b0 con 0x7f1ab4070890 2024-04-12T20:53:58.360 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:53:58.363+0000 7f1ab2000700 1 --2- 172.21.15.18:0/2772402301 >> [v2:172.21.15.18:6800/3048894687,v1:172.21.15.18:6801/3048894687] conn(0x7f1a980692b0 0x7f1a9806b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T20:53:58.361 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:53:58.364+0000 7f1ab2000700 1 --2- 172.21.15.18:0/2772402301 >> [v2:172.21.15.18:6800/3048894687,v1:172.21.15.18:6801/3048894687] conn(0x7f1a980692b0 0x7f1a9806b760 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f1aa0009fd0 tx=0x7f1aa0009f20 comp rx=0 tx=0).ready entity=mgr.14212 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T20:53:58.369 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:53:58.371+0000 7f1aaaa00700 1 -- 172.21.15.18:0/2772402301 <== mon.1 v2:172.21.15.186:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7f1aa4054770 con 0x7f1ab4070890 2024-04-12T20:53:58.703 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:53:58.705+0000 7f1ab3e00700 1 -- 172.21.15.18:0/2772402301 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f1a94005da0 con 0x7f1ab4070890 2024-04-12T20:53:58.704 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:53:58.706+0000 7f1aaaa00700 1 -- 172.21.15.18:0/2772402301 <== mon.1 v2:172.21.15.186:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+638 (secure 0 0 0) 0x7f1aa4057d90 con 0x7f1ab4070890 2024-04-12T20:53:58.704 INFO:teuthology.orchestra.run.smithi018.stdout:{ 2024-04-12T20:53:58.704 INFO:teuthology.orchestra.run.smithi018.stdout: "mon": { 2024-04-12T20:53:58.705 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 17.2.7-827-gd3b5060a (d3b5060a17449edfae9385cdf792afec109fca5d) quincy (stable)": 2 2024-04-12T20:53:58.705 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-12T20:53:58.705 INFO:teuthology.orchestra.run.smithi018.stdout: "mgr": { 2024-04-12T20:53:58.705 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 17.2.7-827-gd3b5060a (d3b5060a17449edfae9385cdf792afec109fca5d) quincy (stable)": 2 2024-04-12T20:53:58.705 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-12T20:53:58.705 INFO:teuthology.orchestra.run.smithi018.stdout: "osd": { 2024-04-12T20:53:58.706 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 17.2.7-827-gd3b5060a (d3b5060a17449edfae9385cdf792afec109fca5d) quincy (stable)": 6 2024-04-12T20:53:58.706 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-12T20:53:58.706 INFO:teuthology.orchestra.run.smithi018.stdout: "mds": { 2024-04-12T20:53:58.706 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 17.2.7-827-gd3b5060a (d3b5060a17449edfae9385cdf792afec109fca5d) quincy (stable)": 4 2024-04-12T20:53:58.706 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-12T20:53:58.706 INFO:teuthology.orchestra.run.smithi018.stdout: "overall": { 2024-04-12T20:53:58.707 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 17.2.7-827-gd3b5060a (d3b5060a17449edfae9385cdf792afec109fca5d) quincy (stable)": 14 2024-04-12T20:53:58.707 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-04-12T20:53:58.707 INFO:teuthology.orchestra.run.smithi018.stdout:} 2024-04-12T20:53:58.708 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:53:58.709+0000 7f1ab3e00700 1 -- 172.21.15.18:0/2772402301 >> [v2:172.21.15.18:6800/3048894687,v1:172.21.15.18:6801/3048894687] conn(0x7f1a980692b0 msgr2=0x7f1a9806b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:53:58.708 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:53:58.709+0000 7f1ab3e00700 1 --2- 172.21.15.18:0/2772402301 >> [v2:172.21.15.18:6800/3048894687,v1:172.21.15.18:6801/3048894687] conn(0x7f1a980692b0 0x7f1a9806b760 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f1aa0009fd0 tx=0x7f1aa0009f20 comp rx=0 tx=0).stop 2024-04-12T20:53:58.709 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:53:58.709+0000 7f1ab3e00700 1 -- 172.21.15.18:0/2772402301 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1ab4070890 msgr2=0x7f1ab40fcb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:53:58.709 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:53:58.709+0000 7f1ab3e00700 1 --2- 172.21.15.18:0/2772402301 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1ab4070890 0x7f1ab40fcb50 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f1aa400b970 tx=0x7f1aa4009e30 comp rx=0 tx=0).stop 2024-04-12T20:53:58.709 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:53:58.709+0000 7f1ab3e00700 1 -- 172.21.15.18:0/2772402301 shutdown_connections 2024-04-12T20:53:58.709 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:53:58.709+0000 7f1ab3e00700 1 --2- 172.21.15.18:0/2772402301 >> [v2:172.21.15.18:6800/3048894687,v1:172.21.15.18:6801/3048894687] conn(0x7f1a980692b0 0x7f1a9806b760 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:53:58.709 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:53:58.709+0000 7f1ab3e00700 1 --2- 172.21.15.18:0/2772402301 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1ab4070890 0x7f1ab40fcb50 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:53:58.710 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:53:58.709+0000 7f1ab3e00700 1 --2- 172.21.15.18:0/2772402301 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1ab4071e40 0x7f1ab40fd090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:53:58.710 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:53:58.709+0000 7f1ab3e00700 1 -- 172.21.15.18:0/2772402301 >> 172.21.15.18:0/2772402301 conn(0x7f1ab40f83d0 msgr2=0x7f1ab40f9ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T20:53:58.710 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:53:58.710+0000 7f1ab3e00700 1 -- 172.21.15.18:0/2772402301 shutdown_connections 2024-04-12T20:53:58.710 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:53:58.710+0000 7f1ab3e00700 1 -- 172.21.15.18:0/2772402301 wait complete. 2024-04-12T20:53:58.905 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:53:58.907+0000 7fd219000700 1 -- 172.21.15.18:0/2705382350 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd2140662e0 msgr2=0x7fd2140ff9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:53:58.905 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:53:58.907+0000 7fd219000700 1 --2- 172.21.15.18:0/2705382350 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd2140662e0 0x7fd2140ff9c0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fd204004940 tx=0x7fd204009e30 comp rx=0 tx=0).stop 2024-04-12T20:53:58.905 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:53:58.907+0000 7fd219000700 1 -- 172.21.15.18:0/2705382350 shutdown_connections 2024-04-12T20:53:58.906 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:53:58.907+0000 7fd219000700 1 --2- 172.21.15.18:0/2705382350 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd2140fff00 0x7fd214102320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:53:58.906 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:53:58.907+0000 7fd219000700 1 --2- 172.21.15.18:0/2705382350 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd2140662e0 0x7fd2140ff9c0 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:53:58.906 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:53:58.907+0000 7fd219000700 1 -- 172.21.15.18:0/2705382350 >> 172.21.15.18:0/2705382350 conn(0x7fd2140f7430 msgr2=0x7fd2140f9860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T20:53:58.906 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:53:58.907+0000 7fd219000700 1 -- 172.21.15.18:0/2705382350 shutdown_connections 2024-04-12T20:53:58.906 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:53:58.908+0000 7fd219000700 1 -- 172.21.15.18:0/2705382350 wait complete. 2024-04-12T20:53:58.907 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:53:58.908+0000 7fd219000700 1 Processor -- start 2024-04-12T20:53:58.907 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:53:58.908+0000 7fd219000700 1 -- start start 2024-04-12T20:53:58.907 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:53:58.909+0000 7fd219000700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd2140fff00 0x7fd21410dbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T20:53:58.907 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:53:58.909+0000 7fd219000700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd21410e0f0 0x7fd21410eea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T20:53:58.907 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:53:58.909+0000 7fd219000700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd21410e5b0 con 0x7fd2140fff00 2024-04-12T20:53:58.907 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:53:58.909+0000 7fd219000700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd21410e6f0 con 0x7fd21410e0f0 2024-04-12T20:53:58.907 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:53:58.909+0000 7fd211e00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd2140fff00 0x7fd21410dbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T20:53:58.907 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:53:58.909+0000 7fd211400700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd21410e0f0 0x7fd21410eea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T20:53:58.907 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:53:58.909+0000 7fd211e00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd2140fff00 0x7fd21410dbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:59666/0 (socket says 172.21.15.18:59666) 2024-04-12T20:53:58.907 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:53:58.909+0000 7fd211400700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd21410e0f0 0x7fd21410eea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.18:33236/0 (socket says 172.21.15.18:33236) 2024-04-12T20:53:58.907 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:53:58.909+0000 7fd211e00700 1 -- 172.21.15.18:0/3624110181 learned_addr learned my addr 172.21.15.18:0/3624110181 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T20:53:58.908 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:53:58.910+0000 7fd211400700 1 -- 172.21.15.18:0/3624110181 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd2140fff00 msgr2=0x7fd21410dbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:53:58.908 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:53:58.910+0000 7fd211400700 1 --2- 172.21.15.18:0/3624110181 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd2140fff00 0x7fd21410dbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:53:58.908 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:53:58.910+0000 7fd211400700 1 -- 172.21.15.18:0/3624110181 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd204005040 con 0x7fd21410e0f0 2024-04-12T20:53:58.908 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:53:58.910+0000 7fd211400700 1 --2- 172.21.15.18:0/3624110181 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd21410e0f0 0x7fd21410eea0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fd200009fd0 tx=0x7fd20000b040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T20:53:58.908 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:53:58.910+0000 7fd20a000700 1 -- 172.21.15.18:0/3624110181 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd200004e90 con 0x7fd21410e0f0 2024-04-12T20:53:58.909 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:53:58.911+0000 7fd20a000700 1 -- 172.21.15.18:0/3624110181 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd20000f040 con 0x7fd21410e0f0 2024-04-12T20:53:58.909 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:53:58.911+0000 7fd20a000700 1 -- 172.21.15.18:0/3624110181 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd20000c9b0 con 0x7fd21410e0f0 2024-04-12T20:53:58.909 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:53:58.911+0000 7fd219000700 1 -- 172.21.15.18:0/3624110181 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd21410f4a0 con 0x7fd21410e0f0 2024-04-12T20:53:58.909 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:53:58.911+0000 7fd219000700 1 -- 172.21.15.18:0/3624110181 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd21410fa30 con 0x7fd21410e0f0 2024-04-12T20:53:58.910 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:53:58.912+0000 7fd219000700 1 -- 172.21.15.18:0/3624110181 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd21405f5f0 con 0x7fd21410e0f0 2024-04-12T20:53:58.911 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:53:58.913+0000 7fd20a000700 1 -- 172.21.15.18:0/3624110181 <== mon.1 v2:172.21.15.186:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7fd200007cc0 con 0x7fd21410e0f0 2024-04-12T20:53:58.912 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:53:58.915+0000 7fd20a000700 1 --2- 172.21.15.18:0/3624110181 >> [v2:172.21.15.18:6800/3048894687,v1:172.21.15.18:6801/3048894687] conn(0x7fd1f8069300 0x7fd1f806b7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T20:53:58.913 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:53:58.915+0000 7fd20a000700 1 -- 172.21.15.18:0/3624110181 <== mon.1 v2:172.21.15.186:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5550+0+0 (secure 0 0 0) 0x7fd200086bf0 con 0x7fd21410e0f0 2024-04-12T20:53:58.913 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:53:58.915+0000 7fd211e00700 1 --2- 172.21.15.18:0/3624110181 >> [v2:172.21.15.18:6800/3048894687,v1:172.21.15.18:6801/3048894687] conn(0x7fd1f8069300 0x7fd1f806b7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T20:53:58.914 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:53:58.916+0000 7fd211e00700 1 --2- 172.21.15.18:0/3624110181 >> [v2:172.21.15.18:6800/3048894687,v1:172.21.15.18:6801/3048894687] conn(0x7fd1f8069300 0x7fd1f806b7b0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fd204013870 tx=0x7fd204009e30 comp rx=0 tx=0).ready entity=mgr.14212 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T20:53:58.918 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:53:58.921+0000 7fd20a000700 1 -- 172.21.15.18:0/3624110181 <== mon.1 v2:172.21.15.186:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7fd2000569f0 con 0x7fd21410e0f0 2024-04-12T20:53:59.027 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:53:59 smithi018 ceph-mon[28799]: from='client.24323 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T20:53:59.027 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:53:59 smithi018 ceph-mon[28799]: from='client.14546 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T20:53:59.027 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:53:59 smithi018 ceph-mon[28799]: from='client.? 172.21.15.18:0/2772402301' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T20:53:59.277 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:53:59.279+0000 7fd219000700 1 -- 172.21.15.18:0/3624110181 --> [v2:172.21.15.18:6800/3048894687,v1:172.21.15.18:6801/3048894687] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fd2140f8f70 con 0x7fd1f8069300 2024-04-12T20:53:59.278 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:53:59.281+0000 7fd20a000700 1 -- 172.21.15.18:0/3624110181 <== mgr.14212 v2:172.21.15.18:6800/3048894687 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+439 (secure 0 0 0) 0x7fd2140f8f70 con 0x7fd1f8069300 2024-04-12T20:53:59.278 INFO:teuthology.orchestra.run.smithi018.stdout:{ 2024-04-12T20:53:59.278 INFO:teuthology.orchestra.run.smithi018.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:f4452def3be4d350f4a8e5feaf7ebb1511be9298fe68478ba7a2d6fbd7d69a63", 2024-04-12T20:53:59.279 INFO:teuthology.orchestra.run.smithi018.stdout: "in_progress": true, 2024-04-12T20:53:59.279 INFO:teuthology.orchestra.run.smithi018.stdout: "which": "Upgrading daemons of type(s) mgr", 2024-04-12T20:53:59.279 INFO:teuthology.orchestra.run.smithi018.stdout: "services_complete": [], 2024-04-12T20:53:59.279 INFO:teuthology.orchestra.run.smithi018.stdout: "progress": "0/2 daemons upgraded", 2024-04-12T20:53:59.279 INFO:teuthology.orchestra.run.smithi018.stdout: "message": "Pulling quay.ceph.io/ceph-ci/ceph@sha256:f4452def3be4d350f4a8e5feaf7ebb1511be9298fe68478ba7a2d6fbd7d69a63 image on host smithi186", 2024-04-12T20:53:59.279 INFO:teuthology.orchestra.run.smithi018.stdout: "is_paused": false 2024-04-12T20:53:59.279 INFO:teuthology.orchestra.run.smithi018.stdout:} 2024-04-12T20:53:59.280 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:53:59 smithi186 ceph-mon[36597]: from='client.24323 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T20:53:59.280 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:53:59 smithi186 ceph-mon[36597]: from='client.14546 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T20:53:59.280 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:53:59 smithi186 ceph-mon[36597]: from='client.? 172.21.15.18:0/2772402301' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T20:53:59.282 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:53:59.284+0000 7fd219000700 1 -- 172.21.15.18:0/3624110181 >> [v2:172.21.15.18:6800/3048894687,v1:172.21.15.18:6801/3048894687] conn(0x7fd1f8069300 msgr2=0x7fd1f806b7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:53:59.282 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:53:59.284+0000 7fd219000700 1 --2- 172.21.15.18:0/3624110181 >> [v2:172.21.15.18:6800/3048894687,v1:172.21.15.18:6801/3048894687] conn(0x7fd1f8069300 0x7fd1f806b7b0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fd204013870 tx=0x7fd204009e30 comp rx=0 tx=0).stop 2024-04-12T20:53:59.282 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:53:59.284+0000 7fd219000700 1 -- 172.21.15.18:0/3624110181 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd21410e0f0 msgr2=0x7fd21410eea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:53:59.282 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:53:59.284+0000 7fd219000700 1 --2- 172.21.15.18:0/3624110181 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd21410e0f0 0x7fd21410eea0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fd200009fd0 tx=0x7fd20000b040 comp rx=0 tx=0).stop 2024-04-12T20:53:59.282 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:53:59.285+0000 7fd219000700 1 -- 172.21.15.18:0/3624110181 shutdown_connections 2024-04-12T20:53:59.282 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:53:59.285+0000 7fd219000700 1 --2- 172.21.15.18:0/3624110181 >> [v2:172.21.15.18:6800/3048894687,v1:172.21.15.18:6801/3048894687] conn(0x7fd1f8069300 0x7fd1f806b7b0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:53:59.282 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:53:59.285+0000 7fd219000700 1 --2- 172.21.15.18:0/3624110181 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd2140fff00 0x7fd21410dbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:53:59.282 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:53:59.285+0000 7fd219000700 1 --2- 172.21.15.18:0/3624110181 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd21410e0f0 0x7fd21410eea0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:53:59.282 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:53:59.285+0000 7fd219000700 1 -- 172.21.15.18:0/3624110181 >> 172.21.15.18:0/3624110181 conn(0x7fd2140f7430 msgr2=0x7fd2140f7850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T20:53:59.283 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:53:59.285+0000 7fd219000700 1 -- 172.21.15.18:0/3624110181 shutdown_connections 2024-04-12T20:53:59.283 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:53:59.285+0000 7fd219000700 1 -- 172.21.15.18:0/3624110181 wait complete. 2024-04-12T20:54:00.282 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:54:00 smithi186 ceph-mon[36597]: from='client.14550 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T20:54:00.282 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:54:00 smithi186 ceph-mon[36597]: pgmap v281: 65 pgs: 65 active+clean; 590 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-12T20:54:00.282 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:54:00 smithi186 ceph-mon[36597]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T20:54:00.291 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:54:00 smithi018 ceph-mon[28799]: from='client.14550 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T20:54:00.291 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:54:00 smithi018 ceph-mon[28799]: pgmap v281: 65 pgs: 65 active+clean; 590 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-12T20:54:00.292 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:54:00 smithi018 ceph-mon[28799]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T20:54:01.351 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:54:01 smithi018 ceph-mon[28799]: from='client.24339 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T20:54:01.531 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:54:01 smithi186 ceph-mon[36597]: from='client.24339 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T20:54:02.352 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:54:02 smithi018 ceph-mon[28799]: pgmap v282: 65 pgs: 65 active+clean; 590 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-12T20:54:02.530 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:54:02 smithi186 ceph-mon[36597]: pgmap v282: 65 pgs: 65 active+clean; 590 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-12T20:54:04.351 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:54:04 smithi018 ceph-mon[28799]: pgmap v283: 65 pgs: 65 active+clean; 590 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-12T20:54:04.530 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:54:04 smithi186 ceph-mon[36597]: pgmap v283: 65 pgs: 65 active+clean; 590 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-12T20:54:06.351 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:54:06 smithi018 ceph-mon[28799]: pgmap v284: 65 pgs: 65 active+clean; 590 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-12T20:54:06.530 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:54:06 smithi186 ceph-mon[36597]: pgmap v284: 65 pgs: 65 active+clean; 590 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-12T20:54:08.354 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:54:08 smithi018 ceph-mon[28799]: pgmap v285: 65 pgs: 65 active+clean; 590 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-12T20:54:08.530 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:54:08 smithi186 ceph-mon[36597]: pgmap v285: 65 pgs: 65 active+clean; 590 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-12T20:54:10.351 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:54:10 smithi018 ceph-mon[28799]: pgmap v286: 65 pgs: 65 active+clean; 590 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-12T20:54:10.381 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:54:10 smithi186 ceph-mon[36597]: pgmap v286: 65 pgs: 65 active+clean; 590 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-12T20:54:12.351 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:54:12 smithi018 ceph-mon[28799]: pgmap v287: 65 pgs: 65 active+clean; 590 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-12T20:54:12.531 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:54:12 smithi186 ceph-mon[36597]: pgmap v287: 65 pgs: 65 active+clean; 590 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-12T20:54:14.352 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:54:14 smithi018 ceph-mon[28799]: pgmap v288: 65 pgs: 65 active+clean; 590 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-12T20:54:14.530 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:54:14 smithi186 ceph-mon[36597]: pgmap v288: 65 pgs: 65 active+clean; 590 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-12T20:54:15.351 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:54:15 smithi018 ceph-mon[28799]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T20:54:15.453 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:54:15 smithi186 ceph-mon[36597]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T20:54:16.351 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:54:16 smithi018 ceph-mon[28799]: pgmap v289: 65 pgs: 65 active+clean; 590 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-12T20:54:16.531 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:54:16 smithi186 ceph-mon[36597]: pgmap v289: 65 pgs: 65 active+clean; 590 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-12T20:54:18.352 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:54:18 smithi018 ceph-mon[28799]: pgmap v290: 65 pgs: 65 active+clean; 590 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-12T20:54:18.530 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:54:18 smithi186 ceph-mon[36597]: pgmap v290: 65 pgs: 65 active+clean; 590 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-12T20:54:20.351 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:54:20 smithi018 ceph-mon[28799]: pgmap v291: 65 pgs: 65 active+clean; 590 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-12T20:54:20.531 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:54:20 smithi186 ceph-mon[36597]: pgmap v291: 65 pgs: 65 active+clean; 590 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-12T20:54:22.351 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:54:22 smithi018 ceph-mon[28799]: pgmap v292: 65 pgs: 65 active+clean; 590 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-12T20:54:22.530 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:54:22 smithi186 ceph-mon[36597]: pgmap v292: 65 pgs: 65 active+clean; 590 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-12T20:54:24.352 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:54:24 smithi018 ceph-mon[28799]: pgmap v293: 65 pgs: 65 active+clean; 590 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-12T20:54:24.530 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:54:24 smithi186 ceph-mon[36597]: pgmap v293: 65 pgs: 65 active+clean; 590 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-12T20:54:26.351 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:54:26 smithi018 ceph-mon[28799]: pgmap v294: 65 pgs: 65 active+clean; 590 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-12T20:54:26.531 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:54:26 smithi186 ceph-mon[36597]: pgmap v294: 65 pgs: 65 active+clean; 590 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-12T20:54:26.801 INFO:tasks.workunit.client.0.smithi018.stderr:Updating files: 58% (7225/12271) Updating files: 59% (7240/12271) Updating files: 60% (7363/12271) Updating files: 61% (7486/12271) Updating files: 62% (7609/12271) Updating files: 63% (7731/12271) Updating files: 64% (7854/12271) Updating files: 65% (7977/12271) Updating files: 66% (8099/12271) Updating files: 67% (8222/12271) Updating files: 68% (8345/12271) Updating files: 69% (8467/12271) Updating files: 70% (8590/12271) Updating files: 71% (8713/12271) Updating files: 72% (8836/12271) Updating files: 73% (8958/12271) Updating files: 74% (9081/12271) Updating files: 75% (9204/12271) Updating files: 76% (9326/12271) Updating files: 77% (9449/12271) Updating files: 78% (9572/12271) Updating files: 79% (9695/12271) Updating files: 80% (9817/12271) Updating files: 81% (9940/12271) Updating files: 82% (10063/12271) Updating files: 83% (10185/12271) Updating files: 84% (10308/12271) Updating files: 85% (10431/12271) Updating files: 86% (10554/12271) Updating files: 87% (10676/12271) Updating files: 88% (10799/12271) Updating files: 89% (10922/12271) Updating files: 90% (11044/12271) Updating files: 91% (11167/12271) Updating files: 92% (11290/12271) Updating files: 93% (11413/12271) Updating files: 94% (11535/12271) Updating files: 95% (11658/12271) Updating files: 96% (11781/12271) Updating files: 96% (11864/12271) Updating files: 97% (11903/12271) Updating files: 98% (12026/12271) Updating files: 99% (12149/12271) Updating files: 100% (12271/12271) Updating files: 100% (12271/12271), done. 2024-04-12T20:54:27.735 INFO:tasks.workunit.client.0.smithi018.stderr:Note: switching to '324baf8ddb91dea46cac50cb535c306115791355'. 2024-04-12T20:54:27.736 INFO:tasks.workunit.client.0.smithi018.stderr: 2024-04-12T20:54:27.736 INFO:tasks.workunit.client.0.smithi018.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2024-04-12T20:54:27.736 INFO:tasks.workunit.client.0.smithi018.stderr:changes and commit them, and you can discard any commits you make in this 2024-04-12T20:54:27.736 INFO:tasks.workunit.client.0.smithi018.stderr:state without impacting any branches by switching back to a branch. 2024-04-12T20:54:27.736 INFO:tasks.workunit.client.0.smithi018.stderr: 2024-04-12T20:54:27.736 INFO:tasks.workunit.client.0.smithi018.stderr:If you want to create a new branch to retain commits you create, you may 2024-04-12T20:54:27.736 INFO:tasks.workunit.client.0.smithi018.stderr:do so (now or later) by using -c with the switch command. Example: 2024-04-12T20:54:27.736 INFO:tasks.workunit.client.0.smithi018.stderr: 2024-04-12T20:54:27.736 INFO:tasks.workunit.client.0.smithi018.stderr: git switch -c 2024-04-12T20:54:27.736 INFO:tasks.workunit.client.0.smithi018.stderr: 2024-04-12T20:54:27.736 INFO:tasks.workunit.client.0.smithi018.stderr:Or undo this operation with: 2024-04-12T20:54:27.736 INFO:tasks.workunit.client.0.smithi018.stderr: 2024-04-12T20:54:27.736 INFO:tasks.workunit.client.0.smithi018.stderr: git switch - 2024-04-12T20:54:27.737 INFO:tasks.workunit.client.0.smithi018.stderr: 2024-04-12T20:54:27.737 INFO:tasks.workunit.client.0.smithi018.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2024-04-12T20:54:27.737 INFO:tasks.workunit.client.0.smithi018.stderr: 2024-04-12T20:54:27.737 INFO:tasks.workunit.client.0.smithi018.stderr:HEAD is now at 324baf8ddb9 Merge PR #55421 into wip-vshankar-testing-20240411.061452 2024-04-12T20:54:27.749 DEBUG:teuthology.orchestra.run.smithi018:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2024-04-12T20:54:27.841 INFO:tasks.workunit.client.0.smithi018.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2024-04-12T20:54:27.849 INFO:tasks.workunit.client.0.smithi018.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-04-12T20:54:27.849 INFO:tasks.workunit.client.0.smithi018.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2024-04-12T20:54:28.531 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:54:28 smithi186 ceph-mon[36597]: pgmap v295: 65 pgs: 65 active+clean; 590 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-12T20:54:28.602 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:54:28 smithi018 ceph-mon[28799]: pgmap v295: 65 pgs: 65 active+clean; 590 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-12T20:54:28.731 INFO:tasks.workunit.client.0.smithi018.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2024-04-12T20:54:28.872 INFO:tasks.workunit.client.0.smithi018.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2024-04-12T20:54:28.961 INFO:tasks.workunit.client.0.smithi018.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-04-12T20:54:28.967 INFO:tasks.workunit.client.0.smithi018.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-04-12T20:54:28.967 INFO:tasks.workunit.client.0.smithi018.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2024-04-12T20:54:29.055 INFO:tasks.workunit.client.0.smithi018.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-04-12T20:54:29.066 DEBUG:teuthology.orchestra.run.smithi018:> set -ex 2024-04-12T20:54:29.066 DEBUG:teuthology.orchestra.run.smithi018:> dd if=/home/ubuntu/cephtest/workunits.list.client.0 of=/dev/stdout 2024-04-12T20:54:29.120 INFO:tasks.workunit:Running workunits matching suites/fsstress.sh on client.0... 2024-04-12T20:54:29.123 INFO:tasks.workunit:Running workunit suites/fsstress.sh... 2024-04-12T20:54:29.123 DEBUG:teuthology.orchestra.run.smithi018:workunit test suites/fsstress.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=324baf8ddb91dea46cac50cb535c306115791355 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/fsstress.sh 2024-04-12T20:54:29.221 INFO:tasks.workunit.client.0.smithi018.stderr:+ mkdir -p fsstress 2024-04-12T20:54:29.229 INFO:tasks.workunit.client.0.smithi018.stderr:+ pushd fsstress 2024-04-12T20:54:29.237 INFO:tasks.workunit.client.0.smithi018.stdout:~/cephtest/mnt.0/client.0/tmp/fsstress ~/cephtest/mnt.0/client.0/tmp 2024-04-12T20:54:29.237 INFO:tasks.workunit.client.0.smithi018.stderr:+ wget -q -O ltp-full.tgz http://download.ceph.com/qa/ltp-full-20091231.tgz 2024-04-12T20:54:29.474 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:54:29.476+0000 7fd79f400700 1 -- 172.21.15.18:0/1304501710 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd798066260 msgr2=0x7fd798101e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:54:29.474 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:54:29.476+0000 7fd79f400700 1 --2- 172.21.15.18:0/1304501710 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd798066260 0x7fd798101e40 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fd788009ce0 tx=0x7fd788004d10 comp rx=0 tx=0).stop 2024-04-12T20:54:29.474 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:54:29.476+0000 7fd79f400700 1 -- 172.21.15.18:0/1304501710 shutdown_connections 2024-04-12T20:54:29.474 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:54:29.476+0000 7fd79f400700 1 --2- 172.21.15.18:0/1304501710 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd798102380 0x7fd798104760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:54:29.475 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:54:29.476+0000 7fd79f400700 1 --2- 172.21.15.18:0/1304501710 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd798066260 0x7fd798101e40 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:54:29.475 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:54:29.476+0000 7fd79f400700 1 -- 172.21.15.18:0/1304501710 >> 172.21.15.18:0/1304501710 conn(0x7fd7980f7430 msgr2=0x7fd7980f9860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T20:54:29.475 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:54:29.476+0000 7fd79f400700 1 -- 172.21.15.18:0/1304501710 shutdown_connections 2024-04-12T20:54:29.475 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:54:29.476+0000 7fd79f400700 1 -- 172.21.15.18:0/1304501710 wait complete. 2024-04-12T20:54:29.476 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:54:29.477+0000 7fd79f400700 1 Processor -- start 2024-04-12T20:54:29.476 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:54:29.478+0000 7fd79f400700 1 -- start start 2024-04-12T20:54:29.476 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:54:29.478+0000 7fd79f400700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd798066260 0x7fd79810ff00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T20:54:29.476 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:54:29.478+0000 7fd79f400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd798102380 0x7fd798110440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T20:54:29.476 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:54:29.478+0000 7fd79f400700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd798111150 con 0x7fd798102380 2024-04-12T20:54:29.476 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:54:29.478+0000 7fd79f400700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7981112c0 con 0x7fd798066260 2024-04-12T20:54:29.476 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:54:29.478+0000 7fd78fe00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd798102380 0x7fd798110440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T20:54:29.476 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:54:29.479+0000 7fd78fe00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd798102380 0x7fd798110440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:52010/0 (socket says 172.21.15.18:52010) 2024-04-12T20:54:29.476 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:54:29.479+0000 7fd78fe00700 1 -- 172.21.15.18:0/2432919614 learned_addr learned my addr 172.21.15.18:0/2432919614 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T20:54:29.476 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:54:29.479+0000 7fd79ca00700 1 --2- 172.21.15.18:0/2432919614 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd798066260 0x7fd79810ff00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T20:54:29.478 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:54:29.479+0000 7fd78fe00700 1 -- 172.21.15.18:0/2432919614 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd798066260 msgr2=0x7fd79810ff00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:54:29.478 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:54:29.479+0000 7fd78fe00700 1 --2- 172.21.15.18:0/2432919614 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd798066260 0x7fd79810ff00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:54:29.478 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:54:29.479+0000 7fd78fe00700 1 -- 172.21.15.18:0/2432919614 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd788009940 con 0x7fd798102380 2024-04-12T20:54:29.478 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:54:29.479+0000 7fd78fe00700 1 --2- 172.21.15.18:0/2432919614 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd798102380 0x7fd798110440 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fd78000b790 tx=0x7fd780009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T20:54:29.478 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:54:29.480+0000 7fd795600700 1 -- 172.21.15.18:0/2432919614 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd78000f6f0 con 0x7fd798102380 2024-04-12T20:54:29.478 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:54:29.480+0000 7fd795600700 1 -- 172.21.15.18:0/2432919614 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd78000fd20 con 0x7fd798102380 2024-04-12T20:54:29.478 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:54:29.480+0000 7fd795600700 1 -- 172.21.15.18:0/2432919614 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd7800103a0 con 0x7fd798102380 2024-04-12T20:54:29.478 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:54:29.480+0000 7fd79f400700 1 -- 172.21.15.18:0/2432919614 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd7981115a0 con 0x7fd798102380 2024-04-12T20:54:29.478 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:54:29.480+0000 7fd79f400700 1 -- 172.21.15.18:0/2432919614 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd798111b60 con 0x7fd798102380 2024-04-12T20:54:29.479 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:54:29.481+0000 7fd79f400700 1 -- 172.21.15.18:0/2432919614 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd798108360 con 0x7fd798102380 2024-04-12T20:54:29.479 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:54:29.482+0000 7fd795600700 1 -- 172.21.15.18:0/2432919614 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7fd780011400 con 0x7fd798102380 2024-04-12T20:54:29.480 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:54:29.483+0000 7fd795600700 1 --2- 172.21.15.18:0/2432919614 >> [v2:172.21.15.18:6800/3048894687,v1:172.21.15.18:6801/3048894687] conn(0x7fd77c0692b0 0x7fd77c06b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T20:54:29.481 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:54:29.483+0000 7fd795600700 1 -- 172.21.15.18:0/2432919614 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5550+0+0 (secure 0 0 0) 0x7fd780086fa0 con 0x7fd798102380 2024-04-12T20:54:29.481 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:54:29.484+0000 7fd79ca00700 1 --2- 172.21.15.18:0/2432919614 >> [v2:172.21.15.18:6800/3048894687,v1:172.21.15.18:6801/3048894687] conn(0x7fd77c0692b0 0x7fd77c06b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T20:54:29.482 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:54:29.485+0000 7fd79ca00700 1 --2- 172.21.15.18:0/2432919614 >> [v2:172.21.15.18:6800/3048894687,v1:172.21.15.18:6801/3048894687] conn(0x7fd77c0692b0 0x7fd77c06b760 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fd7880094d0 tx=0x7fd788004d10 comp rx=0 tx=0).ready entity=mgr.14212 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T20:54:29.486 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:54:29.489+0000 7fd795600700 1 -- 172.21.15.18:0/2432919614 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7fd780056cf0 con 0x7fd798102380 2024-04-12T20:54:29.744 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:54:29.746+0000 7fd79f400700 1 -- 172.21.15.18:0/2432919614 --> [v2:172.21.15.18:6800/3048894687,v1:172.21.15.18:6801/3048894687] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fd798107b50 con 0x7fd77c0692b0 2024-04-12T20:54:29.745 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:54:29.748+0000 7fd795600700 1 -- 172.21.15.18:0/2432919614 <== mgr.14212 v2:172.21.15.18:6800/3048894687 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+439 (secure 0 0 0) 0x7fd798107b50 con 0x7fd77c0692b0 2024-04-12T20:54:29.748 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:54:29.751+0000 7fd79f400700 1 -- 172.21.15.18:0/2432919614 >> [v2:172.21.15.18:6800/3048894687,v1:172.21.15.18:6801/3048894687] conn(0x7fd77c0692b0 msgr2=0x7fd77c06b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:54:29.748 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:54:29.751+0000 7fd79f400700 1 --2- 172.21.15.18:0/2432919614 >> [v2:172.21.15.18:6800/3048894687,v1:172.21.15.18:6801/3048894687] conn(0x7fd77c0692b0 0x7fd77c06b760 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fd7880094d0 tx=0x7fd788004d10 comp rx=0 tx=0).stop 2024-04-12T20:54:29.749 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:54:29.751+0000 7fd79f400700 1 -- 172.21.15.18:0/2432919614 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd798102380 msgr2=0x7fd798110440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:54:29.749 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:54:29.751+0000 7fd79f400700 1 --2- 172.21.15.18:0/2432919614 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd798102380 0x7fd798110440 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fd78000b790 tx=0x7fd780009250 comp rx=0 tx=0).stop 2024-04-12T20:54:29.749 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:54:29.751+0000 7fd79f400700 1 -- 172.21.15.18:0/2432919614 shutdown_connections 2024-04-12T20:54:29.749 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:54:29.751+0000 7fd79f400700 1 --2- 172.21.15.18:0/2432919614 >> [v2:172.21.15.18:6800/3048894687,v1:172.21.15.18:6801/3048894687] conn(0x7fd77c0692b0 0x7fd77c06b760 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:54:29.749 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:54:29.751+0000 7fd79f400700 1 --2- 172.21.15.18:0/2432919614 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd798066260 0x7fd79810ff00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:54:29.749 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:54:29.751+0000 7fd79f400700 1 --2- 172.21.15.18:0/2432919614 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd798102380 0x7fd798110440 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:54:29.749 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:54:29.751+0000 7fd79f400700 1 -- 172.21.15.18:0/2432919614 >> 172.21.15.18:0/2432919614 conn(0x7fd7980f7430 msgr2=0x7fd7980f7850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T20:54:29.749 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:54:29.751+0000 7fd79f400700 1 -- 172.21.15.18:0/2432919614 shutdown_connections 2024-04-12T20:54:29.749 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:54:29.752+0000 7fd79f400700 1 -- 172.21.15.18:0/2432919614 wait complete. 2024-04-12T20:54:29.766 INFO:teuthology.orchestra.run.smithi018.stdout:true 2024-04-12T20:54:29.931 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:54:29.933+0000 7ff5d8e00700 1 -- 172.21.15.18:0/3300516713 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff5d40708f0 msgr2=0x7ff5d4070d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:54:29.932 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:54:29.933+0000 7ff5d8e00700 1 --2- 172.21.15.18:0/3300516713 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff5d40708f0 0x7ff5d4070d60 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7ff5c0004940 tx=0x7ff5c0009e30 comp rx=0 tx=0).stop 2024-04-12T20:54:29.932 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:54:29.934+0000 7ff5d8e00700 1 -- 172.21.15.18:0/3300516713 shutdown_connections 2024-04-12T20:54:29.932 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:54:29.934+0000 7ff5d8e00700 1 --2- 172.21.15.18:0/3300516713 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff5d40708f0 0x7ff5d4070d60 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:54:29.932 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:54:29.934+0000 7ff5d8e00700 1 --2- 172.21.15.18:0/3300516713 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff5d4071ea0 0x7ff5d40703b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:54:29.932 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:54:29.934+0000 7ff5d8e00700 1 -- 172.21.15.18:0/3300516713 >> 172.21.15.18:0/3300516713 conn(0x7ff5d40f83d0 msgr2=0x7ff5d40fa7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T20:54:29.932 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:54:29.934+0000 7ff5d8e00700 1 -- 172.21.15.18:0/3300516713 shutdown_connections 2024-04-12T20:54:29.932 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:54:29.934+0000 7ff5d8e00700 1 -- 172.21.15.18:0/3300516713 wait complete. 2024-04-12T20:54:29.933 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:54:29.935+0000 7ff5d8e00700 1 Processor -- start 2024-04-12T20:54:29.933 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:54:29.935+0000 7ff5d8e00700 1 -- start start 2024-04-12T20:54:29.933 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:54:29.936+0000 7ff5d8e00700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff5d40708f0 0x7ff5d410fa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T20:54:29.933 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:54:29.936+0000 7ff5d8e00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff5d4071ea0 0x7ff5d410ff70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T20:54:29.933 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:54:29.936+0000 7ff5d8e00700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5d4110570 con 0x7ff5d4071ea0 2024-04-12T20:54:29.933 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:54:29.936+0000 7ff5d8e00700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5d4110dc0 con 0x7ff5d40708f0 2024-04-12T20:54:29.934 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:54:29.936+0000 7ff5d2a00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff5d4071ea0 0x7ff5d410ff70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T20:54:29.934 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:54:29.936+0000 7ff5d2a00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff5d4071ea0 0x7ff5d410ff70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:52030/0 (socket says 172.21.15.18:52030) 2024-04-12T20:54:29.934 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:54:29.936+0000 7ff5d2a00700 1 -- 172.21.15.18:0/2725571528 learned_addr learned my addr 172.21.15.18:0/2725571528 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T20:54:29.934 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:54:29.937+0000 7ff5d3400700 1 --2- 172.21.15.18:0/2725571528 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff5d40708f0 0x7ff5d410fa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T20:54:29.935 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:54:29.937+0000 7ff5d2a00700 1 -- 172.21.15.18:0/2725571528 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff5d40708f0 msgr2=0x7ff5d410fa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:54:29.935 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:54:29.937+0000 7ff5d2a00700 1 --2- 172.21.15.18:0/2725571528 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff5d40708f0 0x7ff5d410fa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:54:29.935 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:54:29.937+0000 7ff5d2a00700 1 -- 172.21.15.18:0/2725571528 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5c0005040 con 0x7ff5d4071ea0 2024-04-12T20:54:29.935 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:54:29.937+0000 7ff5d3400700 1 --2- 172.21.15.18:0/2725571528 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff5d40708f0 0x7ff5d410fa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-12T20:54:29.936 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:54:29.937+0000 7ff5d2a00700 1 --2- 172.21.15.18:0/2725571528 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff5d4071ea0 0x7ff5d410ff70 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7ff5c000b3b0 tx=0x7ff5c0009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T20:54:29.936 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:54:29.938+0000 7ff5cb400700 1 -- 172.21.15.18:0/2725571528 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff5c000e040 con 0x7ff5d4071ea0 2024-04-12T20:54:29.936 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:54:29.938+0000 7ff5cb400700 1 -- 172.21.15.18:0/2725571528 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff5c0004290 con 0x7ff5d4071ea0 2024-04-12T20:54:29.936 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:54:29.938+0000 7ff5d8e00700 1 -- 172.21.15.18:0/2725571528 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff5d4111040 con 0x7ff5d4071ea0 2024-04-12T20:54:29.936 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:54:29.938+0000 7ff5cb400700 1 -- 172.21.15.18:0/2725571528 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff5c001c630 con 0x7ff5d4071ea0 2024-04-12T20:54:29.936 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:54:29.938+0000 7ff5d8e00700 1 -- 172.21.15.18:0/2725571528 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff5d4111580 con 0x7ff5d4071ea0 2024-04-12T20:54:29.937 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:54:29.940+0000 7ff5d8e00700 1 -- 172.21.15.18:0/2725571528 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff5d405f5f0 con 0x7ff5d4071ea0 2024-04-12T20:54:29.938 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:54:29.941+0000 7ff5cb400700 1 -- 172.21.15.18:0/2725571528 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7ff5c000bbf0 con 0x7ff5d4071ea0 2024-04-12T20:54:29.939 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:54:29.942+0000 7ff5cb400700 1 --2- 172.21.15.18:0/2725571528 >> [v2:172.21.15.18:6800/3048894687,v1:172.21.15.18:6801/3048894687] conn(0x7ff5b80692b0 0x7ff5b806b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T20:54:29.940 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:54:29.942+0000 7ff5cb400700 1 -- 172.21.15.18:0/2725571528 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5550+0+0 (secure 0 0 0) 0x7ff5c0088650 con 0x7ff5d4071ea0 2024-04-12T20:54:29.940 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:54:29.943+0000 7ff5d3400700 1 --2- 172.21.15.18:0/2725571528 >> [v2:172.21.15.18:6800/3048894687,v1:172.21.15.18:6801/3048894687] conn(0x7ff5b80692b0 0x7ff5b806b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T20:54:29.941 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:54:29.944+0000 7ff5d3400700 1 --2- 172.21.15.18:0/2725571528 >> [v2:172.21.15.18:6800/3048894687,v1:172.21.15.18:6801/3048894687] conn(0x7ff5b80692b0 0x7ff5b806b760 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7ff5c4009fd0 tx=0x7ff5c4009f00 comp rx=0 tx=0).ready entity=mgr.14212 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T20:54:29.947 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:54:29.950+0000 7ff5cb400700 1 -- 172.21.15.18:0/2725571528 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7ff5c00583a0 con 0x7ff5d4071ea0 2024-04-12T20:54:30.210 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:54:30.212+0000 7ff5d8e00700 1 -- 172.21.15.18:0/2725571528 --> [v2:172.21.15.18:6800/3048894687,v1:172.21.15.18:6801/3048894687] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7ff5d4111920 con 0x7ff5b80692b0 2024-04-12T20:54:30.212 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:54:30.215+0000 7ff5cb400700 1 -- 172.21.15.18:0/2725571528 <== mgr.14212 v2:172.21.15.18:6800/3048894687 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+439 (secure 0 0 0) 0x7ff5d4111920 con 0x7ff5b80692b0 2024-04-12T20:54:30.216 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:54:30.218+0000 7ff5d8e00700 1 -- 172.21.15.18:0/2725571528 >> [v2:172.21.15.18:6800/3048894687,v1:172.21.15.18:6801/3048894687] conn(0x7ff5b80692b0 msgr2=0x7ff5b806b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:54:30.216 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:54:30.218+0000 7ff5d8e00700 1 --2- 172.21.15.18:0/2725571528 >> [v2:172.21.15.18:6800/3048894687,v1:172.21.15.18:6801/3048894687] conn(0x7ff5b80692b0 0x7ff5b806b760 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7ff5c4009fd0 tx=0x7ff5c4009f00 comp rx=0 tx=0).stop 2024-04-12T20:54:30.216 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:54:30.218+0000 7ff5d8e00700 1 -- 172.21.15.18:0/2725571528 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff5d4071ea0 msgr2=0x7ff5d410ff70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:54:30.216 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:54:30.218+0000 7ff5d8e00700 1 --2- 172.21.15.18:0/2725571528 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff5d4071ea0 0x7ff5d410ff70 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7ff5c000b3b0 tx=0x7ff5c0009e30 comp rx=0 tx=0).stop 2024-04-12T20:54:30.216 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:54:30.218+0000 7ff5d8e00700 1 -- 172.21.15.18:0/2725571528 shutdown_connections 2024-04-12T20:54:30.216 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:54:30.218+0000 7ff5d8e00700 1 --2- 172.21.15.18:0/2725571528 >> [v2:172.21.15.18:6800/3048894687,v1:172.21.15.18:6801/3048894687] conn(0x7ff5b80692b0 0x7ff5b806b760 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:54:30.216 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:54:30.218+0000 7ff5d8e00700 1 --2- 172.21.15.18:0/2725571528 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff5d40708f0 0x7ff5d410fa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:54:30.216 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:54:30.218+0000 7ff5d8e00700 1 --2- 172.21.15.18:0/2725571528 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff5d4071ea0 0x7ff5d410ff70 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:54:30.217 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:54:30.219+0000 7ff5d8e00700 1 -- 172.21.15.18:0/2725571528 >> 172.21.15.18:0/2725571528 conn(0x7ff5d40f83d0 msgr2=0x7ff5d40f9a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T20:54:30.217 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:54:30.219+0000 7ff5d8e00700 1 -- 172.21.15.18:0/2725571528 shutdown_connections 2024-04-12T20:54:30.217 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:54:30.219+0000 7ff5d8e00700 1 -- 172.21.15.18:0/2725571528 wait complete. 2024-04-12T20:54:30.393 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:54:30.395+0000 7f5f75200700 1 -- 172.21.15.18:0/2966917604 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5f70102310 msgr2=0x7f5f701046f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:54:30.393 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:54:30.395+0000 7f5f75200700 1 --2- 172.21.15.18:0/2966917604 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5f70102310 0x7f5f701046f0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f5f58004940 tx=0x7f5f58009e30 comp rx=0 tx=0).stop 2024-04-12T20:54:30.394 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:54:30.396+0000 7f5f75200700 1 -- 172.21.15.18:0/2966917604 shutdown_connections 2024-04-12T20:54:30.394 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:54:30.396+0000 7f5f75200700 1 --2- 172.21.15.18:0/2966917604 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5f70102310 0x7f5f701046f0 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:54:30.394 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:54:30 smithi018 ceph-mon[28799]: pgmap v296: 65 pgs: 65 active+clean; 590 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-12T20:54:30.394 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:54:30 smithi018 ceph-mon[28799]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T20:54:30.395 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:54:30.396+0000 7f5f75200700 1 --2- 172.21.15.18:0/2966917604 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5f70066b70 0x7f5f70101dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:54:30.395 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:54:30.396+0000 7f5f75200700 1 -- 172.21.15.18:0/2966917604 >> 172.21.15.18:0/2966917604 conn(0x7f5f700f7440 msgr2=0x7f5f700f9870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T20:54:30.395 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:54:30.396+0000 7f5f75200700 1 -- 172.21.15.18:0/2966917604 shutdown_connections 2024-04-12T20:54:30.395 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:54:30.396+0000 7f5f75200700 1 -- 172.21.15.18:0/2966917604 wait complete. 2024-04-12T20:54:30.395 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:54:30.397+0000 7f5f75200700 1 Processor -- start 2024-04-12T20:54:30.395 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:54:30.397+0000 7f5f75200700 1 -- start start 2024-04-12T20:54:30.396 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:54:30.398+0000 7f5f75200700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5f70066b70 0x7f5f7010fc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T20:54:30.396 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:54:30.398+0000 7f5f75200700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5f70102310 0x7f5f701101a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T20:54:30.396 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:54:30.398+0000 7f5f75200700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f701107a0 con 0x7f5f70066b70 2024-04-12T20:54:30.396 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:54:30.398+0000 7f5f75200700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f70110ff0 con 0x7f5f70102310 2024-04-12T20:54:30.396 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:54:30.398+0000 7f5f67e00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5f70066b70 0x7f5f7010fc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T20:54:30.396 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:54:30.398+0000 7f5f67e00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5f70066b70 0x7f5f7010fc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:52040/0 (socket says 172.21.15.18:52040) 2024-04-12T20:54:30.396 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:54:30.398+0000 7f5f67e00700 1 -- 172.21.15.18:0/1781810216 learned_addr learned my addr 172.21.15.18:0/1781810216 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T20:54:30.396 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:54:30.399+0000 7f5f6de00700 1 --2- 172.21.15.18:0/1781810216 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5f70102310 0x7f5f701101a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T20:54:30.397 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:54:30.399+0000 7f5f67e00700 1 -- 172.21.15.18:0/1781810216 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5f70102310 msgr2=0x7f5f701101a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:54:30.397 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:54:30.399+0000 7f5f67e00700 1 --2- 172.21.15.18:0/1781810216 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5f70102310 0x7f5f701101a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:54:30.397 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:54:30.399+0000 7f5f67e00700 1 -- 172.21.15.18:0/1781810216 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f58005040 con 0x7f5f70066b70 2024-04-12T20:54:30.397 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:54:30.399+0000 7f5f67e00700 1 --2- 172.21.15.18:0/1781810216 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5f70066b70 0x7f5f7010fc60 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f5f60002a50 tx=0x7f5f60004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T20:54:30.397 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:54:30.400+0000 7f5f66000700 1 -- 172.21.15.18:0/1781810216 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5f60007cc0 con 0x7f5f70066b70 2024-04-12T20:54:30.397 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:54:30.400+0000 7f5f66000700 1 -- 172.21.15.18:0/1781810216 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f5f6000f040 con 0x7f5f70066b70 2024-04-12T20:54:30.397 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:54:30.400+0000 7f5f75200700 1 -- 172.21.15.18:0/1781810216 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5f701112d0 con 0x7f5f70066b70 2024-04-12T20:54:30.398 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:54:30.400+0000 7f5f66000700 1 -- 172.21.15.18:0/1781810216 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5f6000d9c0 con 0x7f5f70066b70 2024-04-12T20:54:30.398 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:54:30.400+0000 7f5f75200700 1 -- 172.21.15.18:0/1781810216 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5f70111810 con 0x7f5f70066b70 2024-04-12T20:54:30.399 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:54:30.402+0000 7f5f75200700 1 -- 172.21.15.18:0/1781810216 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5f7005f5f0 con 0x7f5f70066b70 2024-04-12T20:54:30.400 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:54:30.403+0000 7f5f66000700 1 -- 172.21.15.18:0/1781810216 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7f5f60009730 con 0x7f5f70066b70 2024-04-12T20:54:30.401 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:54:30.404+0000 7f5f66000700 1 --2- 172.21.15.18:0/1781810216 >> [v2:172.21.15.18:6800/3048894687,v1:172.21.15.18:6801/3048894687] conn(0x7f5f540692b0 0x7f5f5406b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T20:54:30.401 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:54:30.404+0000 7f5f66000700 1 -- 172.21.15.18:0/1781810216 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5550+0+0 (secure 0 0 0) 0x7f5f60086c40 con 0x7f5f70066b70 2024-04-12T20:54:30.401 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:54:30.404+0000 7f5f6de00700 1 --2- 172.21.15.18:0/1781810216 >> [v2:172.21.15.18:6800/3048894687,v1:172.21.15.18:6801/3048894687] conn(0x7f5f540692b0 0x7f5f5406b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T20:54:30.402 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:54:30.405+0000 7f5f6de00700 1 --2- 172.21.15.18:0/1781810216 >> [v2:172.21.15.18:6800/3048894687,v1:172.21.15.18:6801/3048894687] conn(0x7f5f540692b0 0x7f5f5406b760 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f5f5800b3b0 tx=0x7f5f58009e30 comp rx=0 tx=0).ready entity=mgr.14212 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T20:54:30.406 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:54:30.409+0000 7f5f66000700 1 -- 172.21.15.18:0/1781810216 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7f5f60056930 con 0x7f5f70066b70 2024-04-12T20:54:30.530 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:54:30 smithi186 ceph-mon[36597]: pgmap v296: 65 pgs: 65 active+clean; 590 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-12T20:54:30.531 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:54:30 smithi186 ceph-mon[36597]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T20:54:30.690 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:54:30.692+0000 7f5f75200700 1 -- 172.21.15.18:0/1781810216 --> [v2:172.21.15.18:6800/3048894687,v1:172.21.15.18:6801/3048894687] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f5f70111af0 con 0x7f5f540692b0 2024-04-12T20:54:30.702 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:54:30.705+0000 7f5f66000700 1 -- 172.21.15.18:0/1781810216 <== mgr.14212 v2:172.21.15.18:6800/3048894687 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3366 (secure 0 0 0) 0x7f5f70111af0 con 0x7f5f540692b0 2024-04-12T20:54:30.703 INFO:teuthology.orchestra.run.smithi018.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-12T20:54:30.703 INFO:teuthology.orchestra.run.smithi018.stdout:alertmanager.smithi018 smithi018 *:9093,9094 running (7m) 114s ago 9m 13.1M - 0.25.0 c8568f914cd2 f43ec346f9e5 2024-04-12T20:54:30.703 INFO:teuthology.orchestra.run.smithi018.stdout:crash.smithi018 smithi018 running (9m) 114s ago 9m 6660k - 17.2.7-827-gd3b5060a 21de9b7369ad 1ac7603bddfd 2024-04-12T20:54:30.703 INFO:teuthology.orchestra.run.smithi018.stdout:crash.smithi186 smithi186 running (8m) 2m ago 8m 6669k - 17.2.7-827-gd3b5060a 21de9b7369ad 14bf8a1672eb 2024-04-12T20:54:30.703 INFO:teuthology.orchestra.run.smithi018.stdout:grafana.smithi018 smithi018 *:3000 running (7m) 114s ago 9m 73.7M - 9.4.7 954c08fa6188 a7c3fe73c4d7 2024-04-12T20:54:30.704 INFO:teuthology.orchestra.run.smithi018.stdout:mds.cephfs.smithi018.acqbsc smithi018 running (2m) 114s ago 2m 13.5M - 17.2.7-827-gd3b5060a 21de9b7369ad 5b5fce65f18a 2024-04-12T20:54:30.704 INFO:teuthology.orchestra.run.smithi018.stdout:mds.cephfs.smithi018.szjluo smithi018 running (2m) 114s ago 2m 12.8M - 17.2.7-827-gd3b5060a 21de9b7369ad cf9bf0c5031f 2024-04-12T20:54:30.704 INFO:teuthology.orchestra.run.smithi018.stdout:mds.cephfs.smithi186.jdllyn smithi186 running (2m) 2m ago 2m 10.4M - 17.2.7-827-gd3b5060a 21de9b7369ad 2b2bc46bd29a 2024-04-12T20:54:30.704 INFO:teuthology.orchestra.run.smithi018.stdout:mds.cephfs.smithi186.wjnafs smithi186 running (2m) 2m ago 2m 10.5M - 17.2.7-827-gd3b5060a 21de9b7369ad 74b115d6d6fd 2024-04-12T20:54:30.704 INFO:teuthology.orchestra.run.smithi018.stdout:mgr.smithi018.xjzwga smithi018 *:9283 running (11m) 114s ago 11m 481M - 17.2.7-827-gd3b5060a 21de9b7369ad c72f43857311 2024-04-12T20:54:30.704 INFO:teuthology.orchestra.run.smithi018.stdout:mgr.smithi186.kqcfxk smithi186 *:8443,9283 running (8m) 2m ago 8m 432M - 17.2.7-827-gd3b5060a 21de9b7369ad 2c790daf3cc7 2024-04-12T20:54:30.704 INFO:teuthology.orchestra.run.smithi018.stdout:mon.smithi018 smithi018 running (11m) 114s ago 11m 41.1M 2048M 17.2.7-827-gd3b5060a 21de9b7369ad cd84caafa056 2024-04-12T20:54:30.704 INFO:teuthology.orchestra.run.smithi018.stdout:mon.smithi186 smithi186 running (8m) 2m ago 8m 35.0M 2048M 17.2.7-827-gd3b5060a 21de9b7369ad d7263c772b73 2024-04-12T20:54:30.704 INFO:teuthology.orchestra.run.smithi018.stdout:node-exporter.smithi018 smithi018 *:9100 running (9m) 114s ago 9m 7625k - 1.5.0 0da6a335fe13 a23e90d473fe 2024-04-12T20:54:30.704 INFO:teuthology.orchestra.run.smithi018.stdout:node-exporter.smithi186 smithi186 *:9100 running (8m) 2m ago 8m 7382k - 1.5.0 0da6a335fe13 57160d1060f2 2024-04-12T20:54:30.704 INFO:teuthology.orchestra.run.smithi018.stdout:osd.0 smithi018 running (7m) 114s ago 7m 44.4M 4290M 17.2.7-827-gd3b5060a 21de9b7369ad 0d83e027bf0a 2024-04-12T20:54:30.704 INFO:teuthology.orchestra.run.smithi018.stdout:osd.1 smithi018 running (6m) 114s ago 6m 47.9M 4290M 17.2.7-827-gd3b5060a 21de9b7369ad 6517c9cc7f87 2024-04-12T20:54:30.704 INFO:teuthology.orchestra.run.smithi018.stdout:osd.2 smithi018 running (6m) 114s ago 6m 44.3M 4290M 17.2.7-827-gd3b5060a 21de9b7369ad 8a83692acaa5 2024-04-12T20:54:30.704 INFO:teuthology.orchestra.run.smithi018.stdout:osd.3 smithi186 running (5m) 2m ago 5m 43.4M 5312M 17.2.7-827-gd3b5060a 21de9b7369ad c2def7a90d76 2024-04-12T20:54:30.704 INFO:teuthology.orchestra.run.smithi018.stdout:osd.4 smithi186 running (4m) 2m ago 4m 41.0M 5312M 17.2.7-827-gd3b5060a 21de9b7369ad 757e7c71b7b9 2024-04-12T20:54:30.705 INFO:teuthology.orchestra.run.smithi018.stdout:osd.5 smithi186 running (4m) 2m ago 4m 40.8M 5312M 17.2.7-827-gd3b5060a 21de9b7369ad 2ee072a436d0 2024-04-12T20:54:30.705 INFO:teuthology.orchestra.run.smithi018.stdout:prometheus.smithi018 smithi018 *:9095 running (7m) 114s ago 8m 46.7M - 2.43.0 a07b618ecd1d aecd046db448 2024-04-12T20:54:30.706 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:54:30.709+0000 7f5f75200700 1 -- 172.21.15.18:0/1781810216 >> [v2:172.21.15.18:6800/3048894687,v1:172.21.15.18:6801/3048894687] conn(0x7f5f540692b0 msgr2=0x7f5f5406b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:54:30.707 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:54:30.709+0000 7f5f75200700 1 --2- 172.21.15.18:0/1781810216 >> [v2:172.21.15.18:6800/3048894687,v1:172.21.15.18:6801/3048894687] conn(0x7f5f540692b0 0x7f5f5406b760 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f5f5800b3b0 tx=0x7f5f58009e30 comp rx=0 tx=0).stop 2024-04-12T20:54:30.707 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:54:30.709+0000 7f5f75200700 1 -- 172.21.15.18:0/1781810216 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5f70066b70 msgr2=0x7f5f7010fc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:54:30.707 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:54:30.709+0000 7f5f75200700 1 --2- 172.21.15.18:0/1781810216 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5f70066b70 0x7f5f7010fc60 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f5f60002a50 tx=0x7f5f60004600 comp rx=0 tx=0).stop 2024-04-12T20:54:30.707 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:54:30.709+0000 7f5f75200700 1 -- 172.21.15.18:0/1781810216 shutdown_connections 2024-04-12T20:54:30.707 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:54:30.709+0000 7f5f75200700 1 --2- 172.21.15.18:0/1781810216 >> [v2:172.21.15.18:6800/3048894687,v1:172.21.15.18:6801/3048894687] conn(0x7f5f540692b0 0x7f5f5406b760 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:54:30.707 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:54:30.709+0000 7f5f75200700 1 --2- 172.21.15.18:0/1781810216 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5f70066b70 0x7f5f7010fc60 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:54:30.707 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:54:30.709+0000 7f5f75200700 1 --2- 172.21.15.18:0/1781810216 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5f70102310 0x7f5f701101a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:54:30.707 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:54:30.709+0000 7f5f75200700 1 -- 172.21.15.18:0/1781810216 >> 172.21.15.18:0/1781810216 conn(0x7f5f700f7440 msgr2=0x7f5f700f7860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T20:54:30.708 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:54:30.710+0000 7f5f75200700 1 -- 172.21.15.18:0/1781810216 shutdown_connections 2024-04-12T20:54:30.708 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:54:30.710+0000 7f5f75200700 1 -- 172.21.15.18:0/1781810216 wait complete. 2024-04-12T20:54:30.708 INFO:tasks.workunit.client.0.smithi018.stderr:+ tar xzf ltp-full.tgz 2024-04-12T20:54:30.898 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:54:30.900+0000 7f5f6d400700 1 -- 172.21.15.18:0/431649911 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5f68071e40 msgr2=0x7f5f68070350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:54:30.898 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:54:30.900+0000 7f5f6d400700 1 --2- 172.21.15.18:0/431649911 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5f68071e40 0x7f5f68070350 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f5f58004940 tx=0x7f5f58009e30 comp rx=0 tx=0).stop 2024-04-12T20:54:30.898 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:54:30.900+0000 7f5f6d400700 1 -- 172.21.15.18:0/431649911 shutdown_connections 2024-04-12T20:54:30.898 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:54:30.900+0000 7f5f6d400700 1 --2- 172.21.15.18:0/431649911 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5f68070890 0x7f5f68070d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:54:30.898 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:54:30.900+0000 7f5f6d400700 1 --2- 172.21.15.18:0/431649911 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5f68071e40 0x7f5f68070350 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:54:30.898 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:54:30.900+0000 7f5f6d400700 1 -- 172.21.15.18:0/431649911 >> 172.21.15.18:0/431649911 conn(0x7f5f680f8550 msgr2=0x7f5f680fa960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T20:54:30.898 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:54:30.900+0000 7f5f6d400700 1 -- 172.21.15.18:0/431649911 shutdown_connections 2024-04-12T20:54:30.898 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:54:30.900+0000 7f5f6d400700 1 -- 172.21.15.18:0/431649911 wait complete. 2024-04-12T20:54:30.899 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:54:30.901+0000 7f5f6d400700 1 Processor -- start 2024-04-12T20:54:30.899 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:54:30.901+0000 7f5f6d400700 1 -- start start 2024-04-12T20:54:30.899 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:54:30.902+0000 7f5f6d400700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5f68070890 0x7f5f6810fb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T20:54:30.899 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:54:30.902+0000 7f5f6d400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5f68071e40 0x7f5f681100b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T20:54:30.899 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:54:30.902+0000 7f5f6d400700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f681106b0 con 0x7f5f68071e40 2024-04-12T20:54:30.900 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:54:30.902+0000 7f5f6d400700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f68110820 con 0x7f5f68070890 2024-04-12T20:54:30.900 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:54:30.902+0000 7f5f65e00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5f68071e40 0x7f5f681100b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T20:54:30.900 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:54:30.902+0000 7f5f65e00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5f68071e40 0x7f5f681100b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:48954/0 (socket says 172.21.15.18:48954) 2024-04-12T20:54:30.900 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:54:30.902+0000 7f5f65e00700 1 -- 172.21.15.18:0/2740542435 learned_addr learned my addr 172.21.15.18:0/2740542435 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T20:54:30.900 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:54:30.903+0000 7f5f66800700 1 --2- 172.21.15.18:0/2740542435 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5f68070890 0x7f5f6810fb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T20:54:30.901 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:54:30.903+0000 7f5f65e00700 1 -- 172.21.15.18:0/2740542435 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5f68070890 msgr2=0x7f5f6810fb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:54:30.901 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:54:30.903+0000 7f5f65e00700 1 --2- 172.21.15.18:0/2740542435 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5f68070890 0x7f5f6810fb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:54:30.901 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:54:30.903+0000 7f5f65e00700 1 -- 172.21.15.18:0/2740542435 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f58005040 con 0x7f5f68071e40 2024-04-12T20:54:30.901 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:54:30.904+0000 7f5f65e00700 1 --2- 172.21.15.18:0/2740542435 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5f68071e40 0x7f5f681100b0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f5f5400c830 tx=0x7f5f54004470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T20:54:30.902 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:54:30.904+0000 7f5f5ea00700 1 -- 172.21.15.18:0/2740542435 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5f540106f0 con 0x7f5f68071e40 2024-04-12T20:54:30.902 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:54:30.904+0000 7f5f5ea00700 1 -- 172.21.15.18:0/2740542435 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f5f54010d20 con 0x7f5f68071e40 2024-04-12T20:54:30.902 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:54:30.904+0000 7f5f5ea00700 1 -- 172.21.15.18:0/2740542435 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5f540113a0 con 0x7f5f68071e40 2024-04-12T20:54:30.902 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:54:30.904+0000 7f5f6d400700 1 -- 172.21.15.18:0/2740542435 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5f68115280 con 0x7f5f68071e40 2024-04-12T20:54:30.902 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:54:30.905+0000 7f5f6d400700 1 -- 172.21.15.18:0/2740542435 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5f68115840 con 0x7f5f68071e40 2024-04-12T20:54:30.903 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:54:30.906+0000 7f5f6d400700 1 -- 172.21.15.18:0/2740542435 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5f6804c710 con 0x7f5f68071e40 2024-04-12T20:54:30.904 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:54:30.907+0000 7f5f5ea00700 1 -- 172.21.15.18:0/2740542435 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7f5f54012400 con 0x7f5f68071e40 2024-04-12T20:54:30.906 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:54:30.908+0000 7f5f5ea00700 1 --2- 172.21.15.18:0/2740542435 >> [v2:172.21.15.18:6800/3048894687,v1:172.21.15.18:6801/3048894687] conn(0x7f5f4c0692a0 0x7f5f4c06b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T20:54:30.906 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:54:30.909+0000 7f5f5ea00700 1 -- 172.21.15.18:0/2740542435 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5550+0+0 (secure 0 0 0) 0x7f5f54086f30 con 0x7f5f68071e40 2024-04-12T20:54:30.906 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:54:30.909+0000 7f5f66800700 1 --2- 172.21.15.18:0/2740542435 >> [v2:172.21.15.18:6800/3048894687,v1:172.21.15.18:6801/3048894687] conn(0x7f5f4c0692a0 0x7f5f4c06b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T20:54:30.907 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:54:30.910+0000 7f5f66800700 1 --2- 172.21.15.18:0/2740542435 >> [v2:172.21.15.18:6800/3048894687,v1:172.21.15.18:6801/3048894687] conn(0x7f5f4c0692a0 0x7f5f4c06b750 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f5f5800b3b0 tx=0x7f5f58009e30 comp rx=0 tx=0).ready entity=mgr.14212 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T20:54:30.911 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:54:30.913+0000 7f5f5ea00700 1 -- 172.21.15.18:0/2740542435 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7f5f54056c80 con 0x7f5f68071e40 2024-04-12T20:54:31.305 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:54:31.307+0000 7f5f6d400700 1 -- 172.21.15.18:0/2740542435 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f5f68115b20 con 0x7f5f68071e40 2024-04-12T20:54:31.306 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:54:31.308+0000 7f5f5ea00700 1 -- 172.21.15.18:0/2740542435 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+638 (secure 0 0 0) 0x7f5f68115b20 con 0x7f5f68071e40 2024-04-12T20:54:31.306 INFO:teuthology.orchestra.run.smithi018.stdout:{ 2024-04-12T20:54:31.306 INFO:teuthology.orchestra.run.smithi018.stdout: "mon": { 2024-04-12T20:54:31.306 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 17.2.7-827-gd3b5060a (d3b5060a17449edfae9385cdf792afec109fca5d) quincy (stable)": 2 2024-04-12T20:54:31.306 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-12T20:54:31.306 INFO:teuthology.orchestra.run.smithi018.stdout: "mgr": { 2024-04-12T20:54:31.306 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 17.2.7-827-gd3b5060a (d3b5060a17449edfae9385cdf792afec109fca5d) quincy (stable)": 2 2024-04-12T20:54:31.307 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-12T20:54:31.307 INFO:teuthology.orchestra.run.smithi018.stdout: "osd": { 2024-04-12T20:54:31.307 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 17.2.7-827-gd3b5060a (d3b5060a17449edfae9385cdf792afec109fca5d) quincy (stable)": 6 2024-04-12T20:54:31.307 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-12T20:54:31.307 INFO:teuthology.orchestra.run.smithi018.stdout: "mds": { 2024-04-12T20:54:31.307 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 17.2.7-827-gd3b5060a (d3b5060a17449edfae9385cdf792afec109fca5d) quincy (stable)": 4 2024-04-12T20:54:31.307 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-12T20:54:31.307 INFO:teuthology.orchestra.run.smithi018.stdout: "overall": { 2024-04-12T20:54:31.307 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 17.2.7-827-gd3b5060a (d3b5060a17449edfae9385cdf792afec109fca5d) quincy (stable)": 14 2024-04-12T20:54:31.307 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-04-12T20:54:31.307 INFO:teuthology.orchestra.run.smithi018.stdout:} 2024-04-12T20:54:31.309 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:54:31.311+0000 7f5f6d400700 1 -- 172.21.15.18:0/2740542435 >> [v2:172.21.15.18:6800/3048894687,v1:172.21.15.18:6801/3048894687] conn(0x7f5f4c0692a0 msgr2=0x7f5f4c06b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:54:31.309 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:54:31.311+0000 7f5f6d400700 1 --2- 172.21.15.18:0/2740542435 >> [v2:172.21.15.18:6800/3048894687,v1:172.21.15.18:6801/3048894687] conn(0x7f5f4c0692a0 0x7f5f4c06b750 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f5f5800b3b0 tx=0x7f5f58009e30 comp rx=0 tx=0).stop 2024-04-12T20:54:31.309 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:54:31.311+0000 7f5f6d400700 1 -- 172.21.15.18:0/2740542435 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5f68071e40 msgr2=0x7f5f681100b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:54:31.309 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:54:31.311+0000 7f5f6d400700 1 --2- 172.21.15.18:0/2740542435 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5f68071e40 0x7f5f681100b0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f5f5400c830 tx=0x7f5f54004470 comp rx=0 tx=0).stop 2024-04-12T20:54:31.310 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:54:31.312+0000 7f5f6d400700 1 -- 172.21.15.18:0/2740542435 shutdown_connections 2024-04-12T20:54:31.310 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:54:31.312+0000 7f5f6d400700 1 --2- 172.21.15.18:0/2740542435 >> [v2:172.21.15.18:6800/3048894687,v1:172.21.15.18:6801/3048894687] conn(0x7f5f4c0692a0 0x7f5f4c06b750 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:54:31.310 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:54:31.312+0000 7f5f6d400700 1 --2- 172.21.15.18:0/2740542435 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5f68070890 0x7f5f6810fb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:54:31.310 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:54:31.312+0000 7f5f6d400700 1 --2- 172.21.15.18:0/2740542435 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5f68071e40 0x7f5f681100b0 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:54:31.310 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:54:31.312+0000 7f5f6d400700 1 -- 172.21.15.18:0/2740542435 >> 172.21.15.18:0/2740542435 conn(0x7f5f680f8550 msgr2=0x7f5f680fa010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T20:54:31.310 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:54:31.312+0000 7f5f6d400700 1 -- 172.21.15.18:0/2740542435 shutdown_connections 2024-04-12T20:54:31.310 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:54:31.312+0000 7f5f6d400700 1 -- 172.21.15.18:0/2740542435 wait complete. 2024-04-12T20:54:31.490 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:54:31.493+0000 7f5235e00700 1 -- 172.21.15.18:0/849609649 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f52300709a0 msgr2=0x7f5230070e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:54:31.491 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:54:31.493+0000 7f5235e00700 1 --2- 172.21.15.18:0/849609649 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f52300709a0 0x7f5230070e10 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f521c004880 tx=0x7f521c009e30 comp rx=0 tx=0).stop 2024-04-12T20:54:31.491 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:54:31 smithi018 ceph-mon[28799]: from='client.14562 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T20:54:31.492 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:54:31.493+0000 7f5235e00700 1 -- 172.21.15.18:0/849609649 shutdown_connections 2024-04-12T20:54:31.492 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:54:31.493+0000 7f5235e00700 1 --2- 172.21.15.18:0/849609649 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f52300709a0 0x7f5230070e10 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:54:31.492 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:54:31.493+0000 7f5235e00700 1 --2- 172.21.15.18:0/849609649 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5230071f50 0x7f5230070460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:54:31.492 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:54:31.493+0000 7f5235e00700 1 -- 172.21.15.18:0/849609649 >> 172.21.15.18:0/849609649 conn(0x7f52300f8560 msgr2=0x7f52300fa970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T20:54:31.492 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:54:31.493+0000 7f5235e00700 1 -- 172.21.15.18:0/849609649 shutdown_connections 2024-04-12T20:54:31.492 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:54:31.493+0000 7f5235e00700 1 -- 172.21.15.18:0/849609649 wait complete. 2024-04-12T20:54:31.492 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:54:31.495+0000 7f5235e00700 1 Processor -- start 2024-04-12T20:54:31.493 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:54:31.495+0000 7f5235e00700 1 -- start start 2024-04-12T20:54:31.493 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:54:31.495+0000 7f5235e00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f52300709a0 0x7f5230114000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T20:54:31.493 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:54:31.495+0000 7f5235e00700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5230071f50 0x7f5230114540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T20:54:31.493 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:54:31.495+0000 7f5235e00700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5230114b40 con 0x7f52300709a0 2024-04-12T20:54:31.493 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:54:31.495+0000 7f5235e00700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5230115390 con 0x7f5230071f50 2024-04-12T20:54:31.493 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:54:31.496+0000 7f522ea00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f52300709a0 0x7f5230114000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T20:54:31.493 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:54:31.496+0000 7f522ea00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f52300709a0 0x7f5230114000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:48974/0 (socket says 172.21.15.18:48974) 2024-04-12T20:54:31.493 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:54:31.496+0000 7f522ea00700 1 -- 172.21.15.18:0/2183133745 learned_addr learned my addr 172.21.15.18:0/2183133745 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T20:54:31.494 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:54:31.496+0000 7f522e000700 1 --2- 172.21.15.18:0/2183133745 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5230071f50 0x7f5230114540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T20:54:31.495 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:54:31.496+0000 7f522ea00700 1 -- 172.21.15.18:0/2183133745 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5230071f50 msgr2=0x7f5230114540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:54:31.495 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:54:31.496+0000 7f522ea00700 1 --2- 172.21.15.18:0/2183133745 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5230071f50 0x7f5230114540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:54:31.495 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:54:31.496+0000 7f522ea00700 1 -- 172.21.15.18:0/2183133745 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f521c005040 con 0x7f52300709a0 2024-04-12T20:54:31.495 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:54:31.497+0000 7f522ea00700 1 --2- 172.21.15.18:0/2183133745 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f52300709a0 0x7f5230114000 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f522000eb40 tx=0x7f522000b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T20:54:31.495 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:54:31.497+0000 7f5226a00700 1 -- 172.21.15.18:0/2183133745 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5220007cc0 con 0x7f52300709a0 2024-04-12T20:54:31.495 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:54:31.497+0000 7f5226a00700 1 -- 172.21.15.18:0/2183133745 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f522000f040 con 0x7f52300709a0 2024-04-12T20:54:31.495 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:54:31.497+0000 7f5235e00700 1 -- 172.21.15.18:0/2183133745 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5230115670 con 0x7f52300709a0 2024-04-12T20:54:31.495 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:54:31.497+0000 7f5226a00700 1 -- 172.21.15.18:0/2183133745 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5220009680 con 0x7f52300709a0 2024-04-12T20:54:31.496 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:54:31.497+0000 7f5235e00700 1 -- 172.21.15.18:0/2183133745 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5230115bb0 con 0x7f52300709a0 2024-04-12T20:54:31.497 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:54:31.499+0000 7f5235e00700 1 -- 172.21.15.18:0/2183133745 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f523005f5f0 con 0x7f52300709a0 2024-04-12T20:54:31.498 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:54:31.500+0000 7f5226a00700 1 -- 172.21.15.18:0/2183133745 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7f52200097e0 con 0x7f52300709a0 2024-04-12T20:54:31.499 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:54:31.502+0000 7f5226a00700 1 --2- 172.21.15.18:0/2183133745 >> [v2:172.21.15.18:6800/3048894687,v1:172.21.15.18:6801/3048894687] conn(0x7f52140692b0 0x7f521406b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T20:54:31.499 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:54:31.502+0000 7f5226a00700 1 -- 172.21.15.18:0/2183133745 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5550+0+0 (secure 0 0 0) 0x7f5220086cc0 con 0x7f52300709a0 2024-04-12T20:54:31.500 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:54:31.502+0000 7f522e000700 1 --2- 172.21.15.18:0/2183133745 >> [v2:172.21.15.18:6800/3048894687,v1:172.21.15.18:6801/3048894687] conn(0x7f52140692b0 0x7f521406b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T20:54:31.500 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:54:31.503+0000 7f522e000700 1 --2- 172.21.15.18:0/2183133745 >> [v2:172.21.15.18:6800/3048894687,v1:172.21.15.18:6801/3048894687] conn(0x7f52140692b0 0x7f521406b760 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f521c009fd0 tx=0x7f521c009e30 comp rx=0 tx=0).ready entity=mgr.14212 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T20:54:31.504 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:54:31.507+0000 7f5226a00700 1 -- 172.21.15.18:0/2183133745 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7f522004e190 con 0x7f52300709a0 2024-04-12T20:54:31.531 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:54:31 smithi186 ceph-mon[36597]: from='client.14562 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T20:54:31.811 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:54:31.812+0000 7f5235e00700 1 -- 172.21.15.18:0/2183133745 --> [v2:172.21.15.18:6800/3048894687,v1:172.21.15.18:6801/3048894687] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f5230115ff0 con 0x7f52140692b0 2024-04-12T20:54:31.812 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:54:31.814+0000 7f5226a00700 1 -- 172.21.15.18:0/2183133745 <== mgr.14212 v2:172.21.15.18:6800/3048894687 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+439 (secure 0 0 0) 0x7f5230115ff0 con 0x7f52140692b0 2024-04-12T20:54:31.813 INFO:teuthology.orchestra.run.smithi018.stdout:{ 2024-04-12T20:54:31.813 INFO:teuthology.orchestra.run.smithi018.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:f4452def3be4d350f4a8e5feaf7ebb1511be9298fe68478ba7a2d6fbd7d69a63", 2024-04-12T20:54:31.813 INFO:teuthology.orchestra.run.smithi018.stdout: "in_progress": true, 2024-04-12T20:54:31.813 INFO:teuthology.orchestra.run.smithi018.stdout: "which": "Upgrading daemons of type(s) mgr", 2024-04-12T20:54:31.813 INFO:teuthology.orchestra.run.smithi018.stdout: "services_complete": [], 2024-04-12T20:54:31.813 INFO:teuthology.orchestra.run.smithi018.stdout: "progress": "0/2 daemons upgraded", 2024-04-12T20:54:31.813 INFO:teuthology.orchestra.run.smithi018.stdout: "message": "Pulling quay.ceph.io/ceph-ci/ceph@sha256:f4452def3be4d350f4a8e5feaf7ebb1511be9298fe68478ba7a2d6fbd7d69a63 image on host smithi186", 2024-04-12T20:54:31.813 INFO:teuthology.orchestra.run.smithi018.stdout: "is_paused": false 2024-04-12T20:54:31.813 INFO:teuthology.orchestra.run.smithi018.stdout:} 2024-04-12T20:54:31.815 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:54:31.817+0000 7f5235e00700 1 -- 172.21.15.18:0/2183133745 >> [v2:172.21.15.18:6800/3048894687,v1:172.21.15.18:6801/3048894687] conn(0x7f52140692b0 msgr2=0x7f521406b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:54:31.815 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:54:31.817+0000 7f5235e00700 1 --2- 172.21.15.18:0/2183133745 >> [v2:172.21.15.18:6800/3048894687,v1:172.21.15.18:6801/3048894687] conn(0x7f52140692b0 0x7f521406b760 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f521c009fd0 tx=0x7f521c009e30 comp rx=0 tx=0).stop 2024-04-12T20:54:31.815 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:54:31.817+0000 7f5235e00700 1 -- 172.21.15.18:0/2183133745 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f52300709a0 msgr2=0x7f5230114000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:54:31.816 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:54:31.817+0000 7f5235e00700 1 --2- 172.21.15.18:0/2183133745 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f52300709a0 0x7f5230114000 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f522000eb40 tx=0x7f522000b040 comp rx=0 tx=0).stop 2024-04-12T20:54:31.816 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:54:31.817+0000 7f5235e00700 1 -- 172.21.15.18:0/2183133745 shutdown_connections 2024-04-12T20:54:31.816 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:54:31.817+0000 7f5235e00700 1 --2- 172.21.15.18:0/2183133745 >> [v2:172.21.15.18:6800/3048894687,v1:172.21.15.18:6801/3048894687] conn(0x7f52140692b0 0x7f521406b760 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:54:31.817 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:54:31.817+0000 7f5235e00700 1 --2- 172.21.15.18:0/2183133745 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f52300709a0 0x7f5230114000 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:54:31.817 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:54:31.817+0000 7f5235e00700 1 --2- 172.21.15.18:0/2183133745 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5230071f50 0x7f5230114540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:54:31.817 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:54:31.817+0000 7f5235e00700 1 -- 172.21.15.18:0/2183133745 >> 172.21.15.18:0/2183133745 conn(0x7f52300f8560 msgr2=0x7f52300f9d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T20:54:31.817 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:54:31.818+0000 7f5235e00700 1 -- 172.21.15.18:0/2183133745 shutdown_connections 2024-04-12T20:54:31.817 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:54:31.818+0000 7f5235e00700 1 -- 172.21.15.18:0/2183133745 wait complete. 2024-04-12T20:54:32.435 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:54:32 smithi186 ceph-mon[36597]: from='client.14566 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T20:54:32.435 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:54:32 smithi186 ceph-mon[36597]: from='client.14570 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T20:54:32.435 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:54:32 smithi186 ceph-mon[36597]: pgmap v297: 65 pgs: 65 active+clean; 590 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-12T20:54:32.436 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:54:32 smithi186 ceph-mon[36597]: from='client.? 172.21.15.18:0/2740542435' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T20:54:32.601 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:54:32 smithi018 ceph-mon[28799]: from='client.14566 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T20:54:32.602 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:54:32 smithi018 ceph-mon[28799]: from='client.14570 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T20:54:32.602 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:54:32 smithi018 ceph-mon[28799]: pgmap v297: 65 pgs: 65 active+clean; 590 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-12T20:54:32.602 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:54:32 smithi018 ceph-mon[28799]: from='client.? 172.21.15.18:0/2740542435' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T20:54:33.533 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:54:33 smithi186 ceph-mon[36597]: from='client.14578 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T20:54:33.602 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:54:33 smithi018 ceph-mon[28799]: from='client.14578 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T20:54:34.216 INFO:tasks.workunit.client.1.smithi186.stderr:Updating files: 55% (6813/12271) Updating files: 56% (6872/12271) Updating files: 57% (6995/12271) Updating files: 58% (7118/12271) Updating files: 59% (7240/12271) Updating files: 60% (7363/12271) Updating files: 61% (7486/12271) Updating files: 62% (7609/12271) Updating files: 63% (7731/12271) Updating files: 64% (7854/12271) Updating files: 65% (7977/12271) Updating files: 66% (8099/12271) Updating files: 67% (8222/12271) Updating files: 68% (8345/12271) Updating files: 69% (8467/12271) Updating files: 70% (8590/12271) Updating files: 71% (8713/12271) Updating files: 72% (8836/12271) Updating files: 73% (8958/12271) Updating files: 74% (9081/12271) Updating files: 75% (9204/12271) Updating files: 76% (9326/12271) Updating files: 77% (9449/12271) Updating files: 78% (9572/12271) Updating files: 79% (9695/12271) Updating files: 80% (9817/12271) Updating files: 81% (9940/12271) Updating files: 82% (10063/12271) Updating files: 83% (10185/12271) Updating files: 84% (10308/12271) Updating files: 85% (10431/12271) Updating files: 86% (10554/12271) Updating files: 87% (10676/12271) Updating files: 88% (10799/12271) Updating files: 89% (10922/12271) Updating files: 90% (11044/12271) Updating files: 91% (11167/12271) Updating files: 91% (11179/12271) Updating files: 92% (11290/12271) Updating files: 93% (11413/12271) Updating files: 94% (11535/12271) Updating files: 95% (11658/12271) Updating files: 96% (11781/12271) Updating files: 97% (11903/12271) Updating files: 98% (12026/12271) Updating files: 99% (12149/12271) Updating files: 100% (12271/12271) Updating files: 100% (12271/12271), done. 2024-04-12T20:54:34.530 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:54:34 smithi186 ceph-mon[36597]: pgmap v298: 65 pgs: 65 active+clean; 2.9 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 197 KiB/s wr, 1 op/s 2024-04-12T20:54:34.602 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:54:34 smithi018 ceph-mon[28799]: pgmap v298: 65 pgs: 65 active+clean; 2.9 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 197 KiB/s wr, 1 op/s 2024-04-12T20:54:34.800 INFO:tasks.workunit.client.1.smithi186.stderr:Note: switching to '324baf8ddb91dea46cac50cb535c306115791355'. 2024-04-12T20:54:34.800 INFO:tasks.workunit.client.1.smithi186.stderr: 2024-04-12T20:54:34.800 INFO:tasks.workunit.client.1.smithi186.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2024-04-12T20:54:34.800 INFO:tasks.workunit.client.1.smithi186.stderr:changes and commit them, and you can discard any commits you make in this 2024-04-12T20:54:34.800 INFO:tasks.workunit.client.1.smithi186.stderr:state without impacting any branches by switching back to a branch. 2024-04-12T20:54:34.800 INFO:tasks.workunit.client.1.smithi186.stderr: 2024-04-12T20:54:34.800 INFO:tasks.workunit.client.1.smithi186.stderr:If you want to create a new branch to retain commits you create, you may 2024-04-12T20:54:34.800 INFO:tasks.workunit.client.1.smithi186.stderr:do so (now or later) by using -c with the switch command. Example: 2024-04-12T20:54:34.800 INFO:tasks.workunit.client.1.smithi186.stderr: 2024-04-12T20:54:34.800 INFO:tasks.workunit.client.1.smithi186.stderr: git switch -c 2024-04-12T20:54:34.800 INFO:tasks.workunit.client.1.smithi186.stderr: 2024-04-12T20:54:34.800 INFO:tasks.workunit.client.1.smithi186.stderr:Or undo this operation with: 2024-04-12T20:54:34.801 INFO:tasks.workunit.client.1.smithi186.stderr: 2024-04-12T20:54:34.801 INFO:tasks.workunit.client.1.smithi186.stderr: git switch - 2024-04-12T20:54:34.801 INFO:tasks.workunit.client.1.smithi186.stderr: 2024-04-12T20:54:34.801 INFO:tasks.workunit.client.1.smithi186.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2024-04-12T20:54:34.801 INFO:tasks.workunit.client.1.smithi186.stderr: 2024-04-12T20:54:34.801 INFO:tasks.workunit.client.1.smithi186.stderr:HEAD is now at 324baf8ddb9 Merge PR #55421 into wip-vshankar-testing-20240411.061452 2024-04-12T20:54:34.814 DEBUG:teuthology.orchestra.run.smithi186:> cd -- /home/ubuntu/cephtest/clone.client.1/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.1 2024-04-12T20:54:34.952 INFO:tasks.workunit.client.1.smithi186.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2024-04-12T20:54:34.962 INFO:tasks.workunit.client.1.smithi186.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.1/qa/workunits/direct_io' 2024-04-12T20:54:34.962 INFO:tasks.workunit.client.1.smithi186.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2024-04-12T20:54:35.455 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:54:35 smithi186 ceph-mon[36597]: Upgrade: Updating mgr.smithi186.kqcfxk 2024-04-12T20:54:35.455 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:54:35 smithi186 ceph-mon[36597]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' 2024-04-12T20:54:35.455 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:54:35 smithi186 ceph-mon[36597]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi186.kqcfxk", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-12T20:54:35.455 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:54:35 smithi186 ceph-mon[36597]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-12T20:54:35.455 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:54:35 smithi186 ceph-mon[36597]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-12T20:54:35.456 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:54:35 smithi186 ceph-mon[36597]: Deploying daemon mgr.smithi186.kqcfxk on smithi186 2024-04-12T20:54:35.456 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:54:35 smithi186 ceph-mon[36597]: pgmap v299: 65 pgs: 65 active+clean; 12 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 959 KiB/s wr, 18 op/s 2024-04-12T20:54:35.602 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:54:35 smithi018 ceph-mon[28799]: Upgrade: Updating mgr.smithi186.kqcfxk 2024-04-12T20:54:35.602 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:54:35 smithi018 ceph-mon[28799]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' 2024-04-12T20:54:35.603 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:54:35 smithi018 ceph-mon[28799]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi186.kqcfxk", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-12T20:54:35.603 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:54:35 smithi018 ceph-mon[28799]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-12T20:54:35.603 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:54:35 smithi018 ceph-mon[28799]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-12T20:54:35.603 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:54:35 smithi018 ceph-mon[28799]: Deploying daemon mgr.smithi186.kqcfxk on smithi186 2024-04-12T20:54:35.603 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:54:35 smithi018 ceph-mon[28799]: pgmap v299: 65 pgs: 65 active+clean; 12 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 959 KiB/s wr, 18 op/s 2024-04-12T20:54:37.504 INFO:tasks.workunit.client.1.smithi186.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2024-04-12T20:54:37.619 INFO:tasks.workunit.client.1.smithi186.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2024-04-12T20:54:37.694 INFO:tasks.workunit.client.1.smithi186.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.1/qa/workunits/direct_io' 2024-04-12T20:54:37.699 INFO:tasks.workunit.client.1.smithi186.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.1/qa/workunits/fs' 2024-04-12T20:54:37.699 INFO:tasks.workunit.client.1.smithi186.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2024-04-12T20:54:37.778 INFO:tasks.workunit.client.1.smithi186.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.1/qa/workunits/fs' 2024-04-12T20:54:37.786 DEBUG:teuthology.orchestra.run.smithi186:> set -ex 2024-04-12T20:54:37.786 DEBUG:teuthology.orchestra.run.smithi186:> dd if=/home/ubuntu/cephtest/workunits.list.client.1 of=/dev/stdout 2024-04-12T20:54:37.828 INFO:tasks.workunit:Running workunits matching suites/fsstress.sh on client.1... 2024-04-12T20:54:37.829 INFO:tasks.workunit:Running workunit suites/fsstress.sh... 2024-04-12T20:54:37.829 DEBUG:teuthology.orchestra.run.smithi186:workunit test suites/fsstress.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.1/client.1/tmp && cd -- /home/ubuntu/cephtest/mnt.1/client.1/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=324baf8ddb91dea46cac50cb535c306115791355 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="1" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.1 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.1 CEPH_MNT=/home/ubuntu/cephtest/mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.1/qa/workunits/suites/fsstress.sh 2024-04-12T20:54:37.943 INFO:tasks.workunit.client.1.smithi186.stderr:+ mkdir -p fsstress 2024-04-12T20:54:37.951 INFO:tasks.workunit.client.1.smithi186.stderr:+ pushd fsstress 2024-04-12T20:54:37.954 INFO:tasks.workunit.client.1.smithi186.stdout:~/cephtest/mnt.1/client.1/tmp/fsstress ~/cephtest/mnt.1/client.1/tmp 2024-04-12T20:54:37.955 INFO:tasks.workunit.client.1.smithi186.stderr:+ wget -q -O ltp-full.tgz http://download.ceph.com/qa/ltp-full-20091231.tgz 2024-04-12T20:54:38.281 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:54:37 smithi186 ceph-mon[36597]: pgmap v300: 65 pgs: 65 active+clean; 13 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.1 MiB/s wr, 32 op/s 2024-04-12T20:54:38.305 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:54:37 smithi018 ceph-mon[28799]: pgmap v300: 65 pgs: 65 active+clean; 13 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.1 MiB/s wr, 32 op/s 2024-04-12T20:54:39.168 INFO:tasks.workunit.client.1.smithi186.stderr:+ tar xzf ltp-full.tgz 2024-04-12T20:54:40.101 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:54:39 smithi018 ceph-mon[28799]: pgmap v301: 65 pgs: 65 active+clean; 13 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.1 MiB/s wr, 45 op/s 2024-04-12T20:54:40.234 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:54:39 smithi186 ceph-mon[36597]: pgmap v301: 65 pgs: 65 active+clean; 13 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.1 MiB/s wr, 45 op/s 2024-04-12T20:54:41.781 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:54:41 smithi186 ceph-mon[36597]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' 2024-04-12T20:54:41.781 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:54:41 smithi186 ceph-mon[36597]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' 2024-04-12T20:54:41.781 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:54:41 smithi186 ceph-mon[36597]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-12T20:54:41.781 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:54:41 smithi186 ceph-mon[36597]: pgmap v302: 65 pgs: 65 active+clean; 20 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 1.7 MiB/s wr, 58 op/s 2024-04-12T20:54:41.781 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:54:41 smithi186 ceph-mon[36597]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' 2024-04-12T20:54:41.781 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:54:41 smithi186 ceph-mon[36597]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' 2024-04-12T20:54:41.851 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:54:41 smithi018 ceph-mon[28799]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' 2024-04-12T20:54:41.852 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:54:41 smithi018 ceph-mon[28799]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' 2024-04-12T20:54:41.852 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:54:41 smithi018 ceph-mon[28799]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-12T20:54:41.852 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:54:41 smithi018 ceph-mon[28799]: pgmap v302: 65 pgs: 65 active+clean; 20 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 1.7 MiB/s wr, 58 op/s 2024-04-12T20:54:41.852 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:54:41 smithi018 ceph-mon[28799]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' 2024-04-12T20:54:41.852 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:54:41 smithi018 ceph-mon[28799]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' 2024-04-12T20:54:44.083 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:54:43 smithi186 ceph-mon[36597]: pgmap v303: 65 pgs: 65 active+clean; 24 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 2.0 MiB/s wr, 71 op/s 2024-04-12T20:54:44.351 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:54:43 smithi018 ceph-mon[28799]: pgmap v303: 65 pgs: 65 active+clean; 24 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 2.0 MiB/s wr, 71 op/s 2024-04-12T20:54:45.253 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:54:44 smithi186 ceph-mon[36597]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T20:54:45.352 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:54:44 smithi018 ceph-mon[28799]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T20:54:46.280 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:54:46 smithi186 ceph-mon[36597]: pgmap v304: 65 pgs: 65 active+clean; 31 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 2.4 MiB/s wr, 139 op/s 2024-04-12T20:54:46.351 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:54:46 smithi018 ceph-mon[28799]: pgmap v304: 65 pgs: 65 active+clean; 31 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 2.4 MiB/s wr, 139 op/s 2024-04-12T20:54:47.601 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:54:47 smithi018 ceph-mon[28799]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' 2024-04-12T20:54:47.601 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:54:47 smithi018 ceph-mon[28799]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' 2024-04-12T20:54:47.601 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:54:47 smithi018 ceph-mon[28799]: pgmap v305: 65 pgs: 65 active+clean; 35 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 2.0 MiB/s wr, 163 op/s 2024-04-12T20:54:47.689 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:54:47 smithi186 ceph-mon[36597]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' 2024-04-12T20:54:47.689 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:54:47 smithi186 ceph-mon[36597]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' 2024-04-12T20:54:47.689 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:54:47 smithi186 ceph-mon[36597]: pgmap v305: 65 pgs: 65 active+clean; 35 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 2.0 MiB/s wr, 163 op/s 2024-04-12T20:54:49.141 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:54:49 smithi186 ceph-mon[36597]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' 2024-04-12T20:54:49.141 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:54:49 smithi186 ceph-mon[36597]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' 2024-04-12T20:54:49.141 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:54:49 smithi186 ceph-mon[36597]: fsmap cephfs:2 {0=cephfs.smithi018.acqbsc=up:active,1=cephfs.smithi018.szjluo=up:active} 2 up:standby 2024-04-12T20:54:49.352 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:54:49 smithi018 ceph-mon[28799]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' 2024-04-12T20:54:49.352 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:54:49 smithi018 ceph-mon[28799]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' 2024-04-12T20:54:49.352 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:54:49 smithi018 ceph-mon[28799]: fsmap cephfs:2 {0=cephfs.smithi018.acqbsc=up:active,1=cephfs.smithi018.szjluo=up:active} 2 up:standby 2024-04-12T20:54:50.007 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:54:50 smithi186 ceph-mon[36597]: pgmap v306: 65 pgs: 65 active+clean; 36 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail; 2.1 MiB/s wr, 193 op/s 2024-04-12T20:54:50.292 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:54:50 smithi018 ceph-mon[28799]: pgmap v306: 65 pgs: 65 active+clean; 36 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail; 2.1 MiB/s wr, 193 op/s 2024-04-12T20:54:52.281 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:54:52 smithi186 ceph-mon[36597]: pgmap v307: 65 pgs: 65 active+clean; 41 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail; 2.4 MiB/s wr, 208 op/s 2024-04-12T20:54:52.281 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:54:52 smithi186 ceph-mon[36597]: Standby manager daemon smithi186.kqcfxk restarted 2024-04-12T20:54:52.281 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:54:52 smithi186 ceph-mon[36597]: Standby manager daemon smithi186.kqcfxk started 2024-04-12T20:54:52.281 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:54:52 smithi186 ceph-mon[36597]: from='mgr.? 172.21.15.186:0/1315906828' entity='mgr.smithi186.kqcfxk' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi186.kqcfxk/crt"}]: dispatch 2024-04-12T20:54:52.281 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:54:52 smithi186 ceph-mon[36597]: from='mgr.? 172.21.15.186:0/1315906828' entity='mgr.smithi186.kqcfxk' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-12T20:54:52.281 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:54:52 smithi186 ceph-mon[36597]: from='mgr.? 172.21.15.186:0/1315906828' entity='mgr.smithi186.kqcfxk' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi186.kqcfxk/key"}]: dispatch 2024-04-12T20:54:52.281 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:54:52 smithi186 ceph-mon[36597]: from='mgr.? 172.21.15.186:0/1315906828' entity='mgr.smithi186.kqcfxk' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-12T20:54:52.354 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:54:52 smithi018 ceph-mon[28799]: pgmap v307: 65 pgs: 65 active+clean; 41 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail; 2.4 MiB/s wr, 208 op/s 2024-04-12T20:54:52.354 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:54:52 smithi018 ceph-mon[28799]: Standby manager daemon smithi186.kqcfxk restarted 2024-04-12T20:54:52.354 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:54:52 smithi018 ceph-mon[28799]: Standby manager daemon smithi186.kqcfxk started 2024-04-12T20:54:52.354 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:54:52 smithi018 ceph-mon[28799]: from='mgr.? 172.21.15.186:0/1315906828' entity='mgr.smithi186.kqcfxk' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi186.kqcfxk/crt"}]: dispatch 2024-04-12T20:54:52.354 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:54:52 smithi018 ceph-mon[28799]: from='mgr.? 172.21.15.186:0/1315906828' entity='mgr.smithi186.kqcfxk' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-12T20:54:52.354 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:54:52 smithi018 ceph-mon[28799]: from='mgr.? 172.21.15.186:0/1315906828' entity='mgr.smithi186.kqcfxk' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi186.kqcfxk/key"}]: dispatch 2024-04-12T20:54:52.354 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:54:52 smithi018 ceph-mon[28799]: from='mgr.? 172.21.15.186:0/1315906828' entity='mgr.smithi186.kqcfxk' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-12T20:54:53.351 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:54:53 smithi018 ceph-mon[28799]: mgrmap e20: smithi018.xjzwga(active, since 9m), standbys: smithi186.kqcfxk 2024-04-12T20:54:53.530 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:54:53 smithi186 ceph-mon[36597]: mgrmap e20: smithi018.xjzwga(active, since 9m), standbys: smithi186.kqcfxk 2024-04-12T20:54:54.353 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:54:54 smithi018 ceph-mon[28799]: pgmap v308: 65 pgs: 65 active+clean; 45 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail; 2.2 MiB/s wr, 216 op/s 2024-04-12T20:54:54.531 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:54:54 smithi186 ceph-mon[36597]: pgmap v308: 65 pgs: 65 active+clean; 45 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail; 2.2 MiB/s wr, 216 op/s 2024-04-12T20:54:56.351 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:54:56 smithi018 ceph-mon[28799]: pgmap v309: 65 pgs: 65 active+clean; 50 MiB data, 1.9 GiB used, 534 GiB / 536 GiB avail; 2.3 MiB/s wr, 263 op/s 2024-04-12T20:54:56.530 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:54:56 smithi186 ceph-mon[36597]: pgmap v309: 65 pgs: 65 active+clean; 50 MiB data, 1.9 GiB used, 534 GiB / 536 GiB avail; 2.3 MiB/s wr, 263 op/s 2024-04-12T20:54:58.333 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:54:58 smithi186 ceph-mon[36597]: pgmap v310: 65 pgs: 65 active+clean; 53 MiB data, 1.9 GiB used, 534 GiB / 536 GiB avail; 2.0 MiB/s wr, 236 op/s 2024-04-12T20:54:58.351 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:54:58 smithi018 ceph-mon[28799]: pgmap v310: 65 pgs: 65 active+clean; 53 MiB data, 1.9 GiB used, 534 GiB / 536 GiB avail; 2.0 MiB/s wr, 236 op/s 2024-04-12T20:55:00.281 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:55:00 smithi186 ceph-mon[36597]: pgmap v311: 65 pgs: 65 active+clean; 56 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 1.9 MiB/s wr, 242 op/s 2024-04-12T20:55:00.281 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:55:00 smithi186 ceph-mon[36597]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' 2024-04-12T20:55:00.281 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:55:00 smithi186 ceph-mon[36597]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' 2024-04-12T20:55:00.281 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:55:00 smithi186 ceph-mon[36597]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-12T20:55:00.281 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:55:00 smithi186 ceph-mon[36597]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-12T20:55:00.281 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:55:00 smithi186 ceph-mon[36597]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' 2024-04-12T20:55:00.282 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:55:00 smithi186 ceph-mon[36597]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-12T20:55:00.282 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:55:00 smithi186 ceph-mon[36597]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "mgr fail", "who": "smithi018.xjzwga"}]: dispatch 2024-04-12T20:55:00.282 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:55:00 smithi186 ceph-mon[36597]: osdmap e47: 6 total, 6 up, 6 in 2024-04-12T20:55:00.282 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:55:00 smithi186 ceph-mon[36597]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' cmd='[{"prefix": "mgr fail", "who": "smithi018.xjzwga"}]': finished 2024-04-12T20:55:00.282 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:55:00 smithi186 ceph-mon[36597]: mgrmap e21: smithi186.kqcfxk(active, starting, since 0.0143539s) 2024-04-12T20:55:00.282 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:55:00 smithi186 ceph-mon[36597]: from='mgr.24359 172.21.15.186:0/1315906828' entity='mgr.smithi186.kqcfxk' cmd=[{"prefix": "mon metadata", "id": "smithi018"}]: dispatch 2024-04-12T20:55:00.282 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:55:00 smithi186 ceph-mon[36597]: from='mgr.24359 172.21.15.186:0/1315906828' entity='mgr.smithi186.kqcfxk' cmd=[{"prefix": "mon metadata", "id": "smithi186"}]: dispatch 2024-04-12T20:55:00.282 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:55:00 smithi186 ceph-mon[36597]: from='mgr.24359 172.21.15.186:0/1315906828' entity='mgr.smithi186.kqcfxk' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi018.acqbsc"}]: dispatch 2024-04-12T20:55:00.282 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:55:00 smithi186 ceph-mon[36597]: from='mgr.24359 172.21.15.186:0/1315906828' entity='mgr.smithi186.kqcfxk' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi018.szjluo"}]: dispatch 2024-04-12T20:55:00.282 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:55:00 smithi186 ceph-mon[36597]: from='mgr.24359 172.21.15.186:0/1315906828' entity='mgr.smithi186.kqcfxk' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi186.wjnafs"}]: dispatch 2024-04-12T20:55:00.282 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:55:00 smithi186 ceph-mon[36597]: from='mgr.24359 172.21.15.186:0/1315906828' entity='mgr.smithi186.kqcfxk' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi186.jdllyn"}]: dispatch 2024-04-12T20:55:00.282 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:55:00 smithi186 ceph-mon[36597]: from='mgr.24359 172.21.15.186:0/1315906828' entity='mgr.smithi186.kqcfxk' cmd=[{"prefix": "mgr metadata", "who": "smithi186.kqcfxk", "id": "smithi186.kqcfxk"}]: dispatch 2024-04-12T20:55:00.283 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:55:00 smithi186 ceph-mon[36597]: from='mgr.24359 172.21.15.186:0/1315906828' entity='mgr.smithi186.kqcfxk' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-12T20:55:00.283 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:55:00 smithi186 ceph-mon[36597]: from='mgr.24359 172.21.15.186:0/1315906828' entity='mgr.smithi186.kqcfxk' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-12T20:55:00.283 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:55:00 smithi186 ceph-mon[36597]: from='mgr.24359 172.21.15.186:0/1315906828' entity='mgr.smithi186.kqcfxk' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-12T20:55:00.283 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:55:00 smithi186 ceph-mon[36597]: from='mgr.24359 172.21.15.186:0/1315906828' entity='mgr.smithi186.kqcfxk' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-12T20:55:00.283 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:55:00 smithi186 ceph-mon[36597]: from='mgr.24359 172.21.15.186:0/1315906828' entity='mgr.smithi186.kqcfxk' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-12T20:55:00.283 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:55:00 smithi186 ceph-mon[36597]: from='mgr.24359 172.21.15.186:0/1315906828' entity='mgr.smithi186.kqcfxk' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-12T20:55:00.283 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:55:00 smithi186 ceph-mon[36597]: from='mgr.24359 172.21.15.186:0/1315906828' entity='mgr.smithi186.kqcfxk' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-12T20:55:00.283 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:55:00 smithi186 ceph-mon[36597]: from='mgr.24359 172.21.15.186:0/1315906828' entity='mgr.smithi186.kqcfxk' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-12T20:55:00.283 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:55:00 smithi186 ceph-mon[36597]: from='mgr.24359 172.21.15.186:0/1315906828' entity='mgr.smithi186.kqcfxk' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-12T20:55:00.283 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:55:00 smithi186 ceph-mon[36597]: Manager daemon smithi186.kqcfxk is now available 2024-04-12T20:55:00.284 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:55:00 smithi186 ceph-mon[36597]: from='mgr.24359 172.21.15.186:0/1315906828' entity='mgr.smithi186.kqcfxk' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-12T20:55:00.284 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:55:00 smithi186 ceph-mon[36597]: from='mgr.24359 172.21.15.186:0/1315906828' entity='mgr.smithi186.kqcfxk' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T20:55:00.284 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:55:00 smithi186 ceph-mon[36597]: from='mgr.24359 172.21.15.186:0/1315906828' entity='mgr.smithi186.kqcfxk' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi186.kqcfxk/mirror_snapshot_schedule"}]: dispatch 2024-04-12T20:55:00.284 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:55:00 smithi186 ceph-mon[36597]: from='mgr.24359 172.21.15.186:0/1315906828' entity='mgr.smithi186.kqcfxk' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi186.kqcfxk/trash_purge_schedule"}]: dispatch 2024-04-12T20:55:00.352 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:55:00 smithi018 ceph-mon[28799]: pgmap v311: 65 pgs: 65 active+clean; 56 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 1.9 MiB/s wr, 242 op/s 2024-04-12T20:55:00.352 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:55:00 smithi018 ceph-mon[28799]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' 2024-04-12T20:55:00.352 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:55:00 smithi018 ceph-mon[28799]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' 2024-04-12T20:55:00.352 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:55:00 smithi018 ceph-mon[28799]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-12T20:55:00.353 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:55:00 smithi018 ceph-mon[28799]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-12T20:55:00.353 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:55:00 smithi018 ceph-mon[28799]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' 2024-04-12T20:55:00.353 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:55:00 smithi018 ceph-mon[28799]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-12T20:55:00.353 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:55:00 smithi018 ceph-mon[28799]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "mgr fail", "who": "smithi018.xjzwga"}]: dispatch 2024-04-12T20:55:00.353 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:55:00 smithi018 ceph-mon[28799]: osdmap e47: 6 total, 6 up, 6 in 2024-04-12T20:55:00.353 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:55:00 smithi018 ceph-mon[28799]: from='mgr.14212 172.21.15.18:0/2425955143' entity='mgr.smithi018.xjzwga' cmd='[{"prefix": "mgr fail", "who": "smithi018.xjzwga"}]': finished 2024-04-12T20:55:00.353 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:55:00 smithi018 ceph-mon[28799]: mgrmap e21: smithi186.kqcfxk(active, starting, since 0.0143539s) 2024-04-12T20:55:00.353 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:55:00 smithi018 ceph-mon[28799]: from='mgr.24359 172.21.15.186:0/1315906828' entity='mgr.smithi186.kqcfxk' cmd=[{"prefix": "mon metadata", "id": "smithi018"}]: dispatch 2024-04-12T20:55:00.353 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:55:00 smithi018 ceph-mon[28799]: from='mgr.24359 172.21.15.186:0/1315906828' entity='mgr.smithi186.kqcfxk' cmd=[{"prefix": "mon metadata", "id": "smithi186"}]: dispatch 2024-04-12T20:55:00.353 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:55:00 smithi018 ceph-mon[28799]: from='mgr.24359 172.21.15.186:0/1315906828' entity='mgr.smithi186.kqcfxk' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi018.acqbsc"}]: dispatch 2024-04-12T20:55:00.353 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:55:00 smithi018 ceph-mon[28799]: from='mgr.24359 172.21.15.186:0/1315906828' entity='mgr.smithi186.kqcfxk' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi018.szjluo"}]: dispatch 2024-04-12T20:55:00.353 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:55:00 smithi018 ceph-mon[28799]: from='mgr.24359 172.21.15.186:0/1315906828' entity='mgr.smithi186.kqcfxk' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi186.wjnafs"}]: dispatch 2024-04-12T20:55:00.353 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:55:00 smithi018 ceph-mon[28799]: from='mgr.24359 172.21.15.186:0/1315906828' entity='mgr.smithi186.kqcfxk' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi186.jdllyn"}]: dispatch 2024-04-12T20:55:00.353 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:55:00 smithi018 ceph-mon[28799]: from='mgr.24359 172.21.15.186:0/1315906828' entity='mgr.smithi186.kqcfxk' cmd=[{"prefix": "mgr metadata", "who": "smithi186.kqcfxk", "id": "smithi186.kqcfxk"}]: dispatch 2024-04-12T20:55:00.353 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:55:00 smithi018 ceph-mon[28799]: from='mgr.24359 172.21.15.186:0/1315906828' entity='mgr.smithi186.kqcfxk' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-12T20:55:00.353 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:55:00 smithi018 ceph-mon[28799]: from='mgr.24359 172.21.15.186:0/1315906828' entity='mgr.smithi186.kqcfxk' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-12T20:55:00.354 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:55:00 smithi018 ceph-mon[28799]: from='mgr.24359 172.21.15.186:0/1315906828' entity='mgr.smithi186.kqcfxk' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-12T20:55:00.354 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:55:00 smithi018 ceph-mon[28799]: from='mgr.24359 172.21.15.186:0/1315906828' entity='mgr.smithi186.kqcfxk' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-12T20:55:00.354 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:55:00 smithi018 ceph-mon[28799]: from='mgr.24359 172.21.15.186:0/1315906828' entity='mgr.smithi186.kqcfxk' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-12T20:55:00.354 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:55:00 smithi018 ceph-mon[28799]: from='mgr.24359 172.21.15.186:0/1315906828' entity='mgr.smithi186.kqcfxk' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-12T20:55:00.354 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:55:00 smithi018 ceph-mon[28799]: from='mgr.24359 172.21.15.186:0/1315906828' entity='mgr.smithi186.kqcfxk' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-12T20:55:00.354 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:55:00 smithi018 ceph-mon[28799]: from='mgr.24359 172.21.15.186:0/1315906828' entity='mgr.smithi186.kqcfxk' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-12T20:55:00.354 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:55:00 smithi018 ceph-mon[28799]: from='mgr.24359 172.21.15.186:0/1315906828' entity='mgr.smithi186.kqcfxk' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-12T20:55:00.354 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:55:00 smithi018 ceph-mon[28799]: Manager daemon smithi186.kqcfxk is now available 2024-04-12T20:55:00.354 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:55:00 smithi018 ceph-mon[28799]: from='mgr.24359 172.21.15.186:0/1315906828' entity='mgr.smithi186.kqcfxk' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-12T20:55:00.354 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:55:00 smithi018 ceph-mon[28799]: from='mgr.24359 172.21.15.186:0/1315906828' entity='mgr.smithi186.kqcfxk' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T20:55:00.354 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:55:00 smithi018 ceph-mon[28799]: from='mgr.24359 172.21.15.186:0/1315906828' entity='mgr.smithi186.kqcfxk' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi186.kqcfxk/mirror_snapshot_schedule"}]: dispatch 2024-04-12T20:55:00.354 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:55:00 smithi018 ceph-mon[28799]: from='mgr.24359 172.21.15.186:0/1315906828' entity='mgr.smithi186.kqcfxk' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi186.kqcfxk/trash_purge_schedule"}]: dispatch 2024-04-12T20:55:01.530 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:55:01 smithi186 ceph-mon[36597]: from='mgr.24359 172.21.15.186:0/1315906828' entity='mgr.smithi186.kqcfxk' 2024-04-12T20:55:01.530 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:55:01 smithi186 ceph-mon[36597]: from='mgr.24359 172.21.15.186:0/1315906828' entity='mgr.smithi186.kqcfxk' 2024-04-12T20:55:01.531 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:55:01 smithi186 ceph-mon[36597]: mgrmap e22: smithi186.kqcfxk(active, since 1.01706s) 2024-04-12T20:55:01.601 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:55:01 smithi018 ceph-mon[28799]: from='mgr.24359 172.21.15.186:0/1315906828' entity='mgr.smithi186.kqcfxk' 2024-04-12T20:55:01.602 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:55:01 smithi018 ceph-mon[28799]: from='mgr.24359 172.21.15.186:0/1315906828' entity='mgr.smithi186.kqcfxk' 2024-04-12T20:55:01.602 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:55:01 smithi018 ceph-mon[28799]: mgrmap e22: smithi186.kqcfxk(active, since 1.01706s) 2024-04-12T20:55:02.020 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:55:02.020+0000 7f5cf6000700 1 -- 172.21.15.18:0/516445481 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5ce8013b70 con 0x7f5cf8071e40 2024-04-12T20:55:02.021 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:55:02.021+0000 7f5cfe200700 1 -- 172.21.15.18:0/516445481 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5cf8071e40 msgr2=0x7f5cf8070350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:55:02.021 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:55:02.021+0000 7f5cfe200700 1 --2- 172.21.15.18:0/516445481 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5cf8071e40 0x7f5cf8070350 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f5ce8004940 tx=0x7f5ce8009e30 comp rx=0 tx=0).stop 2024-04-12T20:55:02.021 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:55:02.022+0000 7f5cfe200700 1 -- 172.21.15.18:0/516445481 shutdown_connections 2024-04-12T20:55:02.021 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:55:02.022+0000 7f5cfe200700 1 --2- 172.21.15.18:0/516445481 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5cf8070890 0x7f5cf8070d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:55:02.021 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:55:02.022+0000 7f5cfe200700 1 --2- 172.21.15.18:0/516445481 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5cf8071e40 0x7f5cf8070350 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:55:02.021 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:55:02.022+0000 7f5cfe200700 1 -- 172.21.15.18:0/516445481 >> 172.21.15.18:0/516445481 conn(0x7f5cf80f8580 msgr2=0x7f5cf80fa990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T20:55:02.021 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:55:02.023+0000 7f5cfe200700 1 -- 172.21.15.18:0/516445481 shutdown_connections 2024-04-12T20:55:02.021 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:55:02.023+0000 7f5cfe200700 1 -- 172.21.15.18:0/516445481 wait complete. 2024-04-12T20:55:02.022 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:55:02.024+0000 7f5cfe200700 1 Processor -- start 2024-04-12T20:55:02.022 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:55:02.024+0000 7f5cfe200700 1 -- start start 2024-04-12T20:55:02.023 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:55:02.024+0000 7f5cfe200700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5cf8070890 0x7f5cf810fbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T20:55:02.023 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:55:02.024+0000 7f5cfe200700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5cf8071e40 0x7f5cf8110120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T20:55:02.023 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:55:02.024+0000 7f5cfe200700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5cf8110720 con 0x7f5cf8070890 2024-04-12T20:55:02.023 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:55:02.024+0000 7f5cfe200700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5cf8110f70 con 0x7f5cf8071e40 2024-04-12T20:55:02.024 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:55:02.025+0000 7f5cf7400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5cf8070890 0x7f5cf810fbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T20:55:02.024 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:55:02.025+0000 7f5cf7400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5cf8070890 0x7f5cf810fbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:34028/0 (socket says 172.21.15.18:34028) 2024-04-12T20:55:02.024 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:55:02.025+0000 7f5cf7400700 1 -- 172.21.15.18:0/1145772687 learned_addr learned my addr 172.21.15.18:0/1145772687 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T20:55:02.024 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:55:02.025+0000 7f5cf6a00700 1 --2- 172.21.15.18:0/1145772687 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5cf8071e40 0x7f5cf8110120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T20:55:02.024 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:55:02.025+0000 7f5cf7400700 1 -- 172.21.15.18:0/1145772687 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5cf8071e40 msgr2=0x7f5cf8110120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:55:02.024 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:55:02.026+0000 7f5cf7400700 1 --2- 172.21.15.18:0/1145772687 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5cf8071e40 0x7f5cf8110120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:55:02.024 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:55:02.026+0000 7f5cf7400700 1 -- 172.21.15.18:0/1145772687 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ce8005040 con 0x7f5cf8070890 2024-04-12T20:55:02.025 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:55:02.026+0000 7f5cf7400700 1 --2- 172.21.15.18:0/1145772687 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5cf8070890 0x7f5cf810fbe0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f5ce8000c00 tx=0x7f5ce8009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T20:55:02.025 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:55:02.027+0000 7f5cef400700 1 -- 172.21.15.18:0/1145772687 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5ce800e070 con 0x7f5cf8070890 2024-04-12T20:55:02.025 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:55:02.027+0000 7f5cef400700 1 -- 172.21.15.18:0/1145772687 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f5ce801c400 con 0x7f5cf8070890 2024-04-12T20:55:02.025 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:55:02.027+0000 7f5cef400700 1 -- 172.21.15.18:0/1145772687 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5ce800b650 con 0x7f5cf8070890 2024-04-12T20:55:02.025 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:55:02.027+0000 7f5cfe200700 1 -- 172.21.15.18:0/1145772687 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5cf81111f0 con 0x7f5cf8070890 2024-04-12T20:55:02.026 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:55:02.027+0000 7f5cfe200700 1 -- 172.21.15.18:0/1145772687 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5cf81116d0 con 0x7f5cf8070890 2024-04-12T20:55:02.028 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:55:02.028+0000 7f5cef400700 1 -- 172.21.15.18:0/1145772687 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 22) v1 ==== 46402+0+0 (secure 0 0 0) 0x7f5ce800b7b0 con 0x7f5cf8070890 2024-04-12T20:55:02.028 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:55:02.029+0000 7f5cfe200700 1 -- 172.21.15.18:0/1145772687 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5cd80052e0 con 0x7f5cf8070890 2024-04-12T20:55:02.028 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:55:02.029+0000 7f5cef400700 1 --2- 172.21.15.18:0/1145772687 >> [v2:172.21.15.186:6828/2316349902,v1:172.21.15.186:6829/2316349902] conn(0x7f5cdc039310 0x7f5cdc03b7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T20:55:02.028 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:55:02.029+0000 7f5cef400700 1 -- 172.21.15.18:0/1145772687 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5808+0+0 (secure 0 0 0) 0x7f5ce804f5d0 con 0x7f5cf8070890 2024-04-12T20:55:02.028 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:55:02.030+0000 7f5cf6a00700 1 --2- 172.21.15.18:0/1145772687 >> [v2:172.21.15.186:6828/2316349902,v1:172.21.15.186:6829/2316349902] conn(0x7f5cdc039310 0x7f5cdc03b7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T20:55:02.029 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:55:02.031+0000 7f5cf6a00700 1 --2- 172.21.15.18:0/1145772687 >> [v2:172.21.15.186:6828/2316349902,v1:172.21.15.186:6829/2316349902] conn(0x7f5cdc039310 0x7f5cdc03b7c0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f5ce4004600 tx=0x7f5ce400b040 comp rx=0 tx=0).ready entity=mgr.24359 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T20:55:02.035 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:55:02.037+0000 7f5cef400700 1 -- 172.21.15.18:0/1145772687 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186454 (secure 0 0 0) 0x7f5ce801bd40 con 0x7f5cf8070890 2024-04-12T20:55:02.316 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:55:02 smithi018 ceph-mon[28799]: Deploying cephadm binary to smithi186 2024-04-12T20:55:02.316 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:55:02 smithi018 ceph-mon[28799]: Deploying cephadm binary to smithi018 2024-04-12T20:55:02.316 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:55:02 smithi018 ceph-mon[28799]: pgmap v3: 65 pgs: 65 active+clean; 72 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail 2024-04-12T20:55:02.405 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:55:02.407+0000 7f5cfe200700 1 -- 172.21.15.18:0/1145772687 --> [v2:172.21.15.186:6828/2316349902,v1:172.21.15.186:6829/2316349902] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f5cd8000bf0 con 0x7f5cdc039310 2024-04-12T20:55:02.417 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:55:02.419+0000 7f5cef400700 1 -- 172.21.15.18:0/1145772687 <== mgr.24359 v2:172.21.15.186:6828/2316349902 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+310 (secure 0 0 0) 0x7f5cd8000bf0 con 0x7f5cdc039310 2024-04-12T20:55:02.422 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:55:02.422+0000 7f5cfe200700 1 -- 172.21.15.18:0/1145772687 >> [v2:172.21.15.186:6828/2316349902,v1:172.21.15.186:6829/2316349902] conn(0x7f5cdc039310 msgr2=0x7f5cdc03b7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:55:02.422 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:55:02.422+0000 7f5cfe200700 1 --2- 172.21.15.18:0/1145772687 >> [v2:172.21.15.186:6828/2316349902,v1:172.21.15.186:6829/2316349902] conn(0x7f5cdc039310 0x7f5cdc03b7c0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f5ce4004600 tx=0x7f5ce400b040 comp rx=0 tx=0).stop 2024-04-12T20:55:02.422 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:55:02.422+0000 7f5cfe200700 1 -- 172.21.15.18:0/1145772687 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5cf8070890 msgr2=0x7f5cf810fbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:55:02.422 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:55:02.422+0000 7f5cfe200700 1 --2- 172.21.15.18:0/1145772687 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5cf8070890 0x7f5cf810fbe0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f5ce8000c00 tx=0x7f5ce8009e30 comp rx=0 tx=0).stop 2024-04-12T20:55:02.422 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:55:02.423+0000 7f5cfe200700 1 -- 172.21.15.18:0/1145772687 shutdown_connections 2024-04-12T20:55:02.422 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:55:02.423+0000 7f5cfe200700 1 --2- 172.21.15.18:0/1145772687 >> [v2:172.21.15.186:6828/2316349902,v1:172.21.15.186:6829/2316349902] conn(0x7f5cdc039310 0x7f5cdc03b7c0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:55:02.422 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:55:02.423+0000 7f5cfe200700 1 --2- 172.21.15.18:0/1145772687 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5cf8070890 0x7f5cf810fbe0 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:55:02.422 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:55:02.423+0000 7f5cfe200700 1 --2- 172.21.15.18:0/1145772687 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5cf8071e40 0x7f5cf8110120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:55:02.422 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:55:02.423+0000 7f5cfe200700 1 -- 172.21.15.18:0/1145772687 >> 172.21.15.18:0/1145772687 conn(0x7f5cf80f8580 msgr2=0x7f5cf80f9c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T20:55:02.423 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:55:02.424+0000 7f5cfe200700 1 -- 172.21.15.18:0/1145772687 shutdown_connections 2024-04-12T20:55:02.423 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:55:02.424+0000 7f5cfe200700 1 -- 172.21.15.18:0/1145772687 wait complete. 2024-04-12T20:55:02.450 INFO:teuthology.orchestra.run.smithi018.stdout:true 2024-04-12T20:55:02.531 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:55:02 smithi186 ceph-mon[36597]: Deploying cephadm binary to smithi186 2024-04-12T20:55:02.531 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:55:02 smithi186 ceph-mon[36597]: Deploying cephadm binary to smithi018 2024-04-12T20:55:02.531 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:55:02 smithi186 ceph-mon[36597]: pgmap v3: 65 pgs: 65 active+clean; 72 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail 2024-04-12T20:55:02.646 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:55:02.647+0000 7f522f800700 1 -- 172.21.15.18:0/100057708 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f522806e700 msgr2=0x7f522806eb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:55:02.646 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:55:02.647+0000 7f522f800700 1 --2- 172.21.15.18:0/100057708 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f522806e700 0x7f522806eb10 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f521c004940 tx=0x7f521c009e30 comp rx=0 tx=0).stop 2024-04-12T20:55:02.647 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:55:02.648+0000 7f522f800700 1 -- 172.21.15.18:0/100057708 shutdown_connections 2024-04-12T20:55:02.648 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:55:02.648+0000 7f522f800700 1 --2- 172.21.15.18:0/100057708 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f522806f050 0x7f5228108fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:55:02.648 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:55:02.648+0000 7f522f800700 1 --2- 172.21.15.18:0/100057708 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f522806e700 0x7f522806eb10 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:55:02.648 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:55:02.648+0000 7f522f800700 1 -- 172.21.15.18:0/100057708 >> 172.21.15.18:0/100057708 conn(0x7f5228069de0 msgr2=0x7f522806c210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T20:55:02.648 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:55:02.649+0000 7f522f800700 1 -- 172.21.15.18:0/100057708 shutdown_connections 2024-04-12T20:55:02.648 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:55:02.649+0000 7f522f800700 1 -- 172.21.15.18:0/100057708 wait complete. 2024-04-12T20:55:02.649 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:55:02.651+0000 7f522f800700 1 Processor -- start 2024-04-12T20:55:02.649 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:55:02.651+0000 7f522f800700 1 -- start start 2024-04-12T20:55:02.650 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:55:02.652+0000 7f522f800700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f522806e700 0x7f5228113fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T20:55:02.650 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:55:02.652+0000 7f522f800700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f522806f050 0x7f5228114500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T20:55:02.655 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:55:02.652+0000 7f522ce00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f522806e700 0x7f5228113fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T20:55:02.655 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:55:02.652+0000 7f522f800700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5228114b00 con 0x7f522806e700 2024-04-12T20:55:02.655 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:55:02.652+0000 7f522ce00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f522806e700 0x7f5228113fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:34048/0 (socket says 172.21.15.18:34048) 2024-04-12T20:55:02.655 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:55:02.653+0000 7f522f800700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5228114c70 con 0x7f522806f050 2024-04-12T20:55:02.655 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:55:02.653+0000 7f522ce00700 1 -- 172.21.15.18:0/387032207 learned_addr learned my addr 172.21.15.18:0/387032207 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T20:55:02.655 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:55:02.653+0000 7f5227e00700 1 --2- 172.21.15.18:0/387032207 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f522806f050 0x7f5228114500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T20:55:02.655 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:55:02.653+0000 7f5227e00700 1 -- 172.21.15.18:0/387032207 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f522806e700 msgr2=0x7f5228113fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:55:02.656 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:55:02.653+0000 7f5227e00700 1 --2- 172.21.15.18:0/387032207 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f522806e700 0x7f5228113fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:55:02.656 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:55:02.653+0000 7f5227e00700 1 -- 172.21.15.18:0/387032207 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f521c005040 con 0x7f522806f050 2024-04-12T20:55:02.656 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:55:02.654+0000 7f5227e00700 1 --2- 172.21.15.18:0/387032207 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f522806f050 0x7f5228114500 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f5214004c20 tx=0x7f5214009e30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T20:55:02.656 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:55:02.654+0000 7f5224c00700 1 -- 172.21.15.18:0/387032207 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5214007e00 con 0x7f522806f050 2024-04-12T20:55:02.656 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:55:02.654+0000 7f522f800700 1 -- 172.21.15.18:0/387032207 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f52281196d0 con 0x7f522806f050 2024-04-12T20:55:02.656 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:55:02.655+0000 7f522f800700 1 -- 172.21.15.18:0/387032207 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5228119c10 con 0x7f522806f050 2024-04-12T20:55:02.656 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:55:02.655+0000 7f5224c00700 1 -- 172.21.15.18:0/387032207 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f521400f040 con 0x7f522806f050 2024-04-12T20:55:02.656 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:55:02.655+0000 7f5224c00700 1 -- 172.21.15.18:0/387032207 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5214015c40 con 0x7f522806f050 2024-04-12T20:55:02.657 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:55:02.656+0000 7f5224c00700 1 -- 172.21.15.18:0/387032207 <== mon.1 v2:172.21.15.186:3300/0 4 ==== mgrmap(e 23) v1 ==== 46560+0+0 (secure 0 0 0) 0x7f521401e400 con 0x7f522806f050 2024-04-12T20:55:02.657 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:55:02.657+0000 7f5224c00700 1 --2- 172.21.15.18:0/387032207 >> [v2:172.21.15.186:6828/2316349902,v1:172.21.15.186:6829/2316349902] conn(0x7f5210039470 0x7f521003b920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T20:55:02.657 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:55:02.657+0000 7f5224c00700 1 -- 172.21.15.18:0/387032207 <== mon.1 v2:172.21.15.186:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5808+0+0 (secure 0 0 0) 0x7f5214015400 con 0x7f522806f050 2024-04-12T20:55:02.657 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:55:02.658+0000 7f522ce00700 1 --2- 172.21.15.18:0/387032207 >> [v2:172.21.15.186:6828/2316349902,v1:172.21.15.186:6829/2316349902] conn(0x7f5210039470 0x7f521003b920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T20:55:02.657 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:55:02.659+0000 7f522f800700 1 -- 172.21.15.18:0/387032207 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f52080052e0 con 0x7f522806f050 2024-04-12T20:55:02.658 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:55:02.659+0000 7f522ce00700 1 --2- 172.21.15.18:0/387032207 >> [v2:172.21.15.186:6828/2316349902,v1:172.21.15.186:6829/2316349902] conn(0x7f5210039470 0x7f521003b920 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f521c004580 tx=0x7f521c009e30 comp rx=0 tx=0).ready entity=mgr.24359 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T20:55:02.677 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:55:02.679+0000 7f5224c00700 1 -- 172.21.15.18:0/387032207 <== mon.1 v2:172.21.15.186:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186454 (secure 0 0 0) 0x7f52140136f0 con 0x7f522806f050 2024-04-12T20:55:03.026 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:55:03.026+0000 7f522f800700 1 -- 172.21.15.18:0/387032207 --> [v2:172.21.15.186:6828/2316349902,v1:172.21.15.186:6829/2316349902] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f5208000bf0 con 0x7f5210039470 2024-04-12T20:55:03.031 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:55:03.033+0000 7f5224c00700 1 -- 172.21.15.18:0/387032207 <== mgr.24359 v2:172.21.15.186:6828/2316349902 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+310 (secure 0 0 0) 0x7f5208000bf0 con 0x7f5210039470 2024-04-12T20:55:03.039 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:55:03.036+0000 7f522f800700 1 -- 172.21.15.18:0/387032207 >> [v2:172.21.15.186:6828/2316349902,v1:172.21.15.186:6829/2316349902] conn(0x7f5210039470 msgr2=0x7f521003b920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:55:03.039 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:55:03.036+0000 7f522f800700 1 --2- 172.21.15.18:0/387032207 >> [v2:172.21.15.186:6828/2316349902,v1:172.21.15.186:6829/2316349902] conn(0x7f5210039470 0x7f521003b920 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f521c004580 tx=0x7f521c009e30 comp rx=0 tx=0).stop 2024-04-12T20:55:03.039 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:55:03.037+0000 7f522f800700 1 -- 172.21.15.18:0/387032207 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f522806f050 msgr2=0x7f5228114500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:55:03.039 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:55:03.037+0000 7f522f800700 1 --2- 172.21.15.18:0/387032207 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f522806f050 0x7f5228114500 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f5214004c20 tx=0x7f5214009e30 comp rx=0 tx=0).stop 2024-04-12T20:55:03.039 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:55:03.037+0000 7f522f800700 1 -- 172.21.15.18:0/387032207 shutdown_connections 2024-04-12T20:55:03.039 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:55:03.037+0000 7f522f800700 1 --2- 172.21.15.18:0/387032207 >> [v2:172.21.15.186:6828/2316349902,v1:172.21.15.186:6829/2316349902] conn(0x7f5210039470 0x7f521003b920 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:55:03.039 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:55:03.037+0000 7f522f800700 1 --2- 172.21.15.18:0/387032207 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f522806e700 0x7f5228113fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:55:03.039 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:55:03.037+0000 7f522f800700 1 --2- 172.21.15.18:0/387032207 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f522806f050 0x7f5228114500 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:55:03.039 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:55:03.037+0000 7f522f800700 1 -- 172.21.15.18:0/387032207 >> 172.21.15.18:0/387032207 conn(0x7f5228069de0 msgr2=0x7f5228107860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T20:55:03.039 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:55:03.037+0000 7f522f800700 1 -- 172.21.15.18:0/387032207 shutdown_connections 2024-04-12T20:55:03.039 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:55:03.038+0000 7f522f800700 1 -- 172.21.15.18:0/387032207 wait complete. 2024-04-12T20:55:03.232 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:55:03.233+0000 7f9069c00700 1 -- 172.21.15.18:0/41288269 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9064066e50 msgr2=0x7f90640ffa90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:55:03.233 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:55:03.233+0000 7f9069c00700 1 --2- 172.21.15.18:0/41288269 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9064066e50 0x7f90640ffa90 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f9054004940 tx=0x7f9054009e30 comp rx=0 tx=0).stop 2024-04-12T20:55:03.233 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:55:03.234+0000 7f9069c00700 1 -- 172.21.15.18:0/41288269 shutdown_connections 2024-04-12T20:55:03.233 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:55:03.234+0000 7f9069c00700 1 --2- 172.21.15.18:0/41288269 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f90640fffd0 0x7f90641023b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:55:03.233 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:55:03.234+0000 7f9069c00700 1 --2- 172.21.15.18:0/41288269 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9064066e50 0x7f90640ffa90 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:55:03.233 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:55:03.234+0000 7f9069c00700 1 -- 172.21.15.18:0/41288269 >> 172.21.15.18:0/41288269 conn(0x7f90640f7440 msgr2=0x7f90640f9890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T20:55:03.233 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:55:03.234+0000 7f9069c00700 1 -- 172.21.15.18:0/41288269 shutdown_connections 2024-04-12T20:55:03.233 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:55:03.234+0000 7f9069c00700 1 -- 172.21.15.18:0/41288269 wait complete. 2024-04-12T20:55:03.235 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:55:03.235+0000 7f9069c00700 1 Processor -- start 2024-04-12T20:55:03.235 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:55:03.235+0000 7f9069c00700 1 -- start start 2024-04-12T20:55:03.235 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:55:03.236+0000 7f9069c00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9064066e50 0x7f906410b790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T20:55:03.235 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:55:03.236+0000 7f9069c00700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f90640fffd0 0x7f906410bcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T20:55:03.235 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:55:03.236+0000 7f9069c00700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f906410c2d0 con 0x7f9064066e50 2024-04-12T20:55:03.235 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:55:03.236+0000 7f9069c00700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f906410c440 con 0x7f90640fffd0 2024-04-12T20:55:03.236 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:55:03.236+0000 7f9062000700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f90640fffd0 0x7f906410bcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T20:55:03.236 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:55:03.237+0000 7f9062000700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f90640fffd0 0x7f906410bcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.18:50058/0 (socket says 172.21.15.18:50058) 2024-04-12T20:55:03.236 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:55:03.237+0000 7f9062000700 1 -- 172.21.15.18:0/2624537901 learned_addr learned my addr 172.21.15.18:0/2624537901 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T20:55:03.237 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:55:03.238+0000 7f9062a00700 1 --2- 172.21.15.18:0/2624537901 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9064066e50 0x7f906410b790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T20:55:03.237 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:55:03.238+0000 7f9062000700 1 -- 172.21.15.18:0/2624537901 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9064066e50 msgr2=0x7f906410b790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:55:03.237 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:55:03.238+0000 7f9062000700 1 --2- 172.21.15.18:0/2624537901 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9064066e50 0x7f906410b790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:55:03.237 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:55:03.238+0000 7f9062000700 1 -- 172.21.15.18:0/2624537901 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9054005040 con 0x7f90640fffd0 2024-04-12T20:55:03.237 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:55:03.239+0000 7f9062000700 1 --2- 172.21.15.18:0/2624537901 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f90640fffd0 0x7f906410bcd0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f9050002a50 tx=0x7f90500046b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T20:55:03.238 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:55:03.239+0000 7f905aa00700 1 -- 172.21.15.18:0/2624537901 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9050007cc0 con 0x7f90640fffd0 2024-04-12T20:55:03.238 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:55:03.239+0000 7f905aa00700 1 -- 172.21.15.18:0/2624537901 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f905000f040 con 0x7f90640fffd0 2024-04-12T20:55:03.238 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:55:03.239+0000 7f905aa00700 1 -- 172.21.15.18:0/2624537901 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9050009640 con 0x7f90640fffd0 2024-04-12T20:55:03.238 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:55:03.240+0000 7f9069c00700 1 -- 172.21.15.18:0/2624537901 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9064110e50 con 0x7f90640fffd0 2024-04-12T20:55:03.238 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:55:03.240+0000 7f9069c00700 1 -- 172.21.15.18:0/2624537901 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f90641113e0 con 0x7f90640fffd0 2024-04-12T20:55:03.242 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:55:03.241+0000 7f9069c00700 1 -- 172.21.15.18:0/2624537901 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f906404c710 con 0x7f90640fffd0 2024-04-12T20:55:03.242 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:55:03.242+0000 7f905aa00700 1 -- 172.21.15.18:0/2624537901 <== mon.1 v2:172.21.15.186:3300/0 4 ==== mgrmap(e 23) v1 ==== 46560+0+0 (secure 0 0 0) 0x7f90500097c0 con 0x7f90640fffd0 2024-04-12T20:55:03.242 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:55:03.243+0000 7f905aa00700 1 --2- 172.21.15.18:0/2624537901 >> [v2:172.21.15.186:6828/2316349902,v1:172.21.15.186:6829/2316349902] conn(0x7f904803d8d0 0x7f904803fd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T20:55:03.242 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:55:03.243+0000 7f905aa00700 1 -- 172.21.15.18:0/2624537901 <== mon.1 v2:172.21.15.186:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5808+0+0 (secure 0 0 0) 0x7f905004cfd0 con 0x7f90640fffd0 2024-04-12T20:55:03.243 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:55:03.244+0000 7f9062a00700 1 --2- 172.21.15.18:0/2624537901 >> [v2:172.21.15.186:6828/2316349902,v1:172.21.15.186:6829/2316349902] conn(0x7f904803d8d0 0x7f904803fd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T20:55:03.243 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:55:03.245+0000 7f9062a00700 1 --2- 172.21.15.18:0/2624537901 >> [v2:172.21.15.186:6828/2316349902,v1:172.21.15.186:6829/2316349902] conn(0x7f904803d8d0 0x7f904803fd80 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f905400b970 tx=0x7f9054009e30 comp rx=0 tx=0).ready entity=mgr.24359 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T20:55:03.250 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:55:03.252+0000 7f905aa00700 1 -- 172.21.15.18:0/2624537901 <== mon.1 v2:172.21.15.186:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186454 (secure 0 0 0) 0x7f9050019490 con 0x7f90640fffd0 2024-04-12T20:55:03.351 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:55:03 smithi018 ceph-mon[28799]: [12/Apr/2024:20:55:01] ENGINE Bus STARTING 2024-04-12T20:55:03.351 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:55:03 smithi018 ceph-mon[28799]: pgmap v4: 65 pgs: 65 active+clean; 72 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail 2024-04-12T20:55:03.351 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:55:03 smithi018 ceph-mon[28799]: [12/Apr/2024:20:55:01] ENGINE Serving on http://172.21.15.186:8765 2024-04-12T20:55:03.352 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:55:03 smithi018 ceph-mon[28799]: [12/Apr/2024:20:55:01] ENGINE Serving on https://172.21.15.186:7150 2024-04-12T20:55:03.352 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:55:03 smithi018 ceph-mon[28799]: [12/Apr/2024:20:55:01] ENGINE Bus STARTED 2024-04-12T20:55:03.352 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:55:03 smithi018 ceph-mon[28799]: mgrmap e23: smithi186.kqcfxk(active, since 3s) 2024-04-12T20:55:03.531 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:55:03 smithi186 ceph-mon[36597]: [12/Apr/2024:20:55:01] ENGINE Bus STARTING 2024-04-12T20:55:03.531 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:55:03 smithi186 ceph-mon[36597]: pgmap v4: 65 pgs: 65 active+clean; 72 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail 2024-04-12T20:55:03.531 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:55:03 smithi186 ceph-mon[36597]: [12/Apr/2024:20:55:01] ENGINE Serving on http://172.21.15.186:8765 2024-04-12T20:55:03.531 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:55:03 smithi186 ceph-mon[36597]: [12/Apr/2024:20:55:01] ENGINE Serving on https://172.21.15.186:7150 2024-04-12T20:55:03.532 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:55:03 smithi186 ceph-mon[36597]: [12/Apr/2024:20:55:01] ENGINE Bus STARTED 2024-04-12T20:55:03.532 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:55:03 smithi186 ceph-mon[36597]: mgrmap e23: smithi186.kqcfxk(active, since 3s) 2024-04-12T20:55:03.731 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:55:03.732+0000 7f9069c00700 1 -- 172.21.15.18:0/2624537901 --> [v2:172.21.15.186:6828/2316349902,v1:172.21.15.186:6829/2316349902] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f90640f8fa0 con 0x7f904803d8d0 2024-04-12T20:55:03.784 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:55:03.786+0000 7f905aa00700 1 -- 172.21.15.18:0/2624537901 <== mgr.24359 v2:172.21.15.186:6828/2316349902 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3388 (secure 0 0 0) 0x7f90640f8fa0 con 0x7f904803d8d0 2024-04-12T20:55:03.785 INFO:teuthology.orchestra.run.smithi018.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-12T20:55:03.785 INFO:teuthology.orchestra.run.smithi018.stdout:alertmanager.smithi018 smithi018 *:9093,9094 running (8m) 2m ago 10m 13.1M - 0.25.0 c8568f914cd2 f43ec346f9e5 2024-04-12T20:55:03.785 INFO:teuthology.orchestra.run.smithi018.stdout:crash.smithi018 smithi018 running (10m) 2m ago 10m 6660k - 17.2.7-827-gd3b5060a 21de9b7369ad 1ac7603bddfd 2024-04-12T20:55:03.785 INFO:teuthology.orchestra.run.smithi018.stdout:crash.smithi186 smithi186 running (9m) 17s ago 9m 6669k - 17.2.7-827-gd3b5060a 21de9b7369ad 14bf8a1672eb 2024-04-12T20:55:03.785 INFO:teuthology.orchestra.run.smithi018.stdout:grafana.smithi018 smithi018 *:3000 running (8m) 2m ago 9m 73.7M - 9.4.7 954c08fa6188 a7c3fe73c4d7 2024-04-12T20:55:03.785 INFO:teuthology.orchestra.run.smithi018.stdout:mds.cephfs.smithi018.acqbsc smithi018 running (3m) 2m ago 3m 13.5M - 17.2.7-827-gd3b5060a 21de9b7369ad 5b5fce65f18a 2024-04-12T20:55:03.785 INFO:teuthology.orchestra.run.smithi018.stdout:mds.cephfs.smithi018.szjluo smithi018 running (2m) 2m ago 2m 12.8M - 17.2.7-827-gd3b5060a 21de9b7369ad cf9bf0c5031f 2024-04-12T20:55:03.785 INFO:teuthology.orchestra.run.smithi018.stdout:mds.cephfs.smithi186.jdllyn smithi186 running (2m) 17s ago 2m 11.1M - 17.2.7-827-gd3b5060a 21de9b7369ad 2b2bc46bd29a 2024-04-12T20:55:03.785 INFO:teuthology.orchestra.run.smithi018.stdout:mds.cephfs.smithi186.wjnafs smithi186 running (2m) 17s ago 2m 11.2M - 17.2.7-827-gd3b5060a 21de9b7369ad 74b115d6d6fd 2024-04-12T20:55:03.785 INFO:teuthology.orchestra.run.smithi018.stdout:mgr.smithi018.xjzwga smithi018 *:9283 running (11m) 2m ago 11m 481M - 17.2.7-827-gd3b5060a 21de9b7369ad c72f43857311 2024-04-12T20:55:03.785 INFO:teuthology.orchestra.run.smithi018.stdout:mgr.smithi186.kqcfxk smithi186 *:8443,9283 running (23s) 17s ago 8m 53.7M - 19.0.0-2879-g324baf8d 8717720dc5e5 bd4971d5260e 2024-04-12T20:55:03.786 INFO:teuthology.orchestra.run.smithi018.stdout:mon.smithi018 smithi018 running (11m) 2m ago 12m 41.1M 2048M 17.2.7-827-gd3b5060a 21de9b7369ad cd84caafa056 2024-04-12T20:55:03.786 INFO:teuthology.orchestra.run.smithi018.stdout:mon.smithi186 smithi186 running (8m) 17s ago 8m 32.8M 2048M 17.2.7-827-gd3b5060a 21de9b7369ad d7263c772b73 2024-04-12T20:55:03.786 INFO:teuthology.orchestra.run.smithi018.stdout:node-exporter.smithi018 smithi018 *:9100 running (10m) 2m ago 10m 7625k - 1.5.0 0da6a335fe13 a23e90d473fe 2024-04-12T20:55:03.786 INFO:teuthology.orchestra.run.smithi018.stdout:node-exporter.smithi186 smithi186 *:9100 running (8m) 17s ago 8m 7994k - 1.5.0 0da6a335fe13 57160d1060f2 2024-04-12T20:55:03.786 INFO:teuthology.orchestra.run.smithi018.stdout:osd.0 smithi018 running (7m) 2m ago 7m 44.4M 4290M 17.2.7-827-gd3b5060a 21de9b7369ad 0d83e027bf0a 2024-04-12T20:55:03.786 INFO:teuthology.orchestra.run.smithi018.stdout:osd.1 smithi018 running (7m) 2m ago 7m 47.9M 4290M 17.2.7-827-gd3b5060a 21de9b7369ad 6517c9cc7f87 2024-04-12T20:55:03.786 INFO:teuthology.orchestra.run.smithi018.stdout:osd.2 smithi018 running (6m) 2m ago 6m 44.3M 4290M 17.2.7-827-gd3b5060a 21de9b7369ad 8a83692acaa5 2024-04-12T20:55:03.786 INFO:teuthology.orchestra.run.smithi018.stdout:osd.3 smithi186 running (6m) 17s ago 6m 59.1M 5312M 17.2.7-827-gd3b5060a 21de9b7369ad c2def7a90d76 2024-04-12T20:55:03.786 INFO:teuthology.orchestra.run.smithi018.stdout:osd.4 smithi186 running (5m) 17s ago 5m 55.8M 5312M 17.2.7-827-gd3b5060a 21de9b7369ad 757e7c71b7b9 2024-04-12T20:55:03.786 INFO:teuthology.orchestra.run.smithi018.stdout:osd.5 smithi186 running (4m) 17s ago 4m 54.7M 5312M 17.2.7-827-gd3b5060a 21de9b7369ad 2ee072a436d0 2024-04-12T20:55:03.786 INFO:teuthology.orchestra.run.smithi018.stdout:prometheus.smithi018 smithi018 *:9095 running (7m) 2m ago 9m 46.7M - 2.43.0 a07b618ecd1d aecd046db448 2024-04-12T20:55:03.789 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:55:03.791+0000 7f9069c00700 1 -- 172.21.15.18:0/2624537901 >> [v2:172.21.15.186:6828/2316349902,v1:172.21.15.186:6829/2316349902] conn(0x7f904803d8d0 msgr2=0x7f904803fd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:55:03.789 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:55:03.791+0000 7f9069c00700 1 --2- 172.21.15.18:0/2624537901 >> [v2:172.21.15.186:6828/2316349902,v1:172.21.15.186:6829/2316349902] conn(0x7f904803d8d0 0x7f904803fd80 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f905400b970 tx=0x7f9054009e30 comp rx=0 tx=0).stop 2024-04-12T20:55:03.789 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:55:03.791+0000 7f9069c00700 1 -- 172.21.15.18:0/2624537901 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f90640fffd0 msgr2=0x7f906410bcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:55:03.789 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:55:03.791+0000 7f9069c00700 1 --2- 172.21.15.18:0/2624537901 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f90640fffd0 0x7f906410bcd0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f9050002a50 tx=0x7f90500046b0 comp rx=0 tx=0).stop 2024-04-12T20:55:03.790 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:55:03.791+0000 7f9069c00700 1 -- 172.21.15.18:0/2624537901 shutdown_connections 2024-04-12T20:55:03.790 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:55:03.791+0000 7f9069c00700 1 --2- 172.21.15.18:0/2624537901 >> [v2:172.21.15.186:6828/2316349902,v1:172.21.15.186:6829/2316349902] conn(0x7f904803d8d0 0x7f904803fd80 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:55:03.791 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:55:03.791+0000 7f9069c00700 1 --2- 172.21.15.18:0/2624537901 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9064066e50 0x7f906410b790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:55:03.791 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:55:03.791+0000 7f9069c00700 1 --2- 172.21.15.18:0/2624537901 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f90640fffd0 0x7f906410bcd0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:55:03.791 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:55:03.791+0000 7f9069c00700 1 -- 172.21.15.18:0/2624537901 >> 172.21.15.18:0/2624537901 conn(0x7f90640f7440 msgr2=0x7f90640f7880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T20:55:03.791 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:55:03.792+0000 7f9069c00700 1 -- 172.21.15.18:0/2624537901 shutdown_connections 2024-04-12T20:55:03.791 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:55:03.792+0000 7f9069c00700 1 -- 172.21.15.18:0/2624537901 wait complete. 2024-04-12T20:55:04.011 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:55:04.013+0000 7f529a400700 1 -- 172.21.15.18:0/2572068787 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5294072820 msgr2=0x7f5294072c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:55:04.011 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:55:04.013+0000 7f529a400700 1 --2- 172.21.15.18:0/2572068787 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5294072820 0x7f5294072c30 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f527c004940 tx=0x7f527c009e30 comp rx=0 tx=0).stop 2024-04-12T20:55:04.012 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:55:04.014+0000 7f529a400700 1 -- 172.21.15.18:0/2572068787 shutdown_connections 2024-04-12T20:55:04.012 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:55:04.014+0000 7f529a400700 1 --2- 172.21.15.18:0/2572068787 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5294073170 0x7f5294073b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:55:04.012 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:55:04.014+0000 7f529a400700 1 --2- 172.21.15.18:0/2572068787 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5294072820 0x7f5294072c30 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:55:04.012 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:55:04.014+0000 7f529a400700 1 -- 172.21.15.18:0/2572068787 >> 172.21.15.18:0/2572068787 conn(0x7f52940fa950 msgr2=0x7f52940fcd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T20:55:04.013 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:55:04.015+0000 7f529a400700 1 -- 172.21.15.18:0/2572068787 shutdown_connections 2024-04-12T20:55:04.014 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:55:04.015+0000 7f529a400700 1 -- 172.21.15.18:0/2572068787 wait complete. 2024-04-12T20:55:04.014 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:55:04.016+0000 7f529a400700 1 Processor -- start 2024-04-12T20:55:04.016 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:55:04.017+0000 7f529a400700 1 -- start start 2024-04-12T20:55:04.017 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:55:04.017+0000 7f529a400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5294072820 0x7f529406e400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T20:55:04.017 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:55:04.017+0000 7f529a400700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f529406e940 0x7f529406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T20:55:04.017 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:55:04.017+0000 7f529a400700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f529406ff30 con 0x7f5294072820 2024-04-12T20:55:04.017 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:55:04.017+0000 7f529a400700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52940700a0 con 0x7f529406e940 2024-04-12T20:55:04.017 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:55:04.018+0000 7f5292a00700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f529406e940 0x7f529406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T20:55:04.019 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:55:04.019+0000 7f5293400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5294072820 0x7f529406e400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T20:55:04.019 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:55:04.020+0000 7f5293400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5294072820 0x7f529406e400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:34078/0 (socket says 172.21.15.18:34078) 2024-04-12T20:55:04.019 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:55:04.020+0000 7f5293400700 1 -- 172.21.15.18:0/2046272903 learned_addr learned my addr 172.21.15.18:0/2046272903 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T20:55:04.021 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:55:04.021+0000 7f5292a00700 1 -- 172.21.15.18:0/2046272903 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5294072820 msgr2=0x7f529406e400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:55:04.021 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:55:04.021+0000 7f5292a00700 1 --2- 172.21.15.18:0/2046272903 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5294072820 0x7f529406e400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:55:04.021 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:55:04.021+0000 7f5292a00700 1 -- 172.21.15.18:0/2046272903 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f527c005040 con 0x7f529406e940 2024-04-12T20:55:04.022 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:55:04.024+0000 7f5292a00700 1 --2- 172.21.15.18:0/2046272903 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f529406e940 0x7f529406edb0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f5284009fd0 tx=0x7f528400b040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T20:55:04.023 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:55:04.025+0000 7f528b400700 1 -- 172.21.15.18:0/2046272903 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5284004e90 con 0x7f529406e940 2024-04-12T20:55:04.024 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:55:04.025+0000 7f528b400700 1 -- 172.21.15.18:0/2046272903 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f528400f040 con 0x7f529406e940 2024-04-12T20:55:04.024 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:55:04.025+0000 7f528b400700 1 -- 172.21.15.18:0/2046272903 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f528400c9b0 con 0x7f529406e940 2024-04-12T20:55:04.024 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:55:04.025+0000 7f529a400700 1 -- 172.21.15.18:0/2046272903 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f52940fec40 con 0x7f529406e940 2024-04-12T20:55:04.024 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:55:04.025+0000 7f529a400700 1 -- 172.21.15.18:0/2046272903 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f52940ff1b0 con 0x7f529406e940 2024-04-12T20:55:04.025 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:55:04.027+0000 7f528b400700 1 -- 172.21.15.18:0/2046272903 <== mon.1 v2:172.21.15.186:3300/0 4 ==== mgrmap(e 23) v1 ==== 46560+0+0 (secure 0 0 0) 0x7f5284007cc0 con 0x7f529406e940 2024-04-12T20:55:04.025 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:55:04.027+0000 7f529a400700 1 -- 172.21.15.18:0/2046272903 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f529405f5f0 con 0x7f529406e940 2024-04-12T20:55:04.026 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:55:04.027+0000 7f528b400700 1 --2- 172.21.15.18:0/2046272903 >> [v2:172.21.15.186:6828/2316349902,v1:172.21.15.186:6829/2316349902] conn(0x7f5278039510 0x7f527803b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T20:55:04.026 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:55:04.028+0000 7f528b400700 1 -- 172.21.15.18:0/2046272903 <== mon.1 v2:172.21.15.186:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5808+0+0 (secure 0 0 0) 0x7f5284012c10 con 0x7f529406e940 2024-04-12T20:55:04.026 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:55:04.028+0000 7f5293400700 1 --2- 172.21.15.18:0/2046272903 >> [v2:172.21.15.186:6828/2316349902,v1:172.21.15.186:6829/2316349902] conn(0x7f5278039510 0x7f527803b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T20:55:04.028 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:55:04.029+0000 7f5293400700 1 --2- 172.21.15.18:0/2046272903 >> [v2:172.21.15.186:6828/2316349902,v1:172.21.15.186:6829/2316349902] conn(0x7f5278039510 0x7f527803b9c0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f527c00b0d0 tx=0x7f527c009e30 comp rx=0 tx=0).ready entity=mgr.24359 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T20:55:04.033 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:55:04.035+0000 7f528b400700 1 -- 172.21.15.18:0/2046272903 <== mon.1 v2:172.21.15.186:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186454 (secure 0 0 0) 0x7f5284009410 con 0x7f529406e940 2024-04-12T20:55:04.352 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:55:04 smithi018 ceph-mon[28799]: from='client.14606 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T20:55:04.352 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:55:04 smithi018 ceph-mon[28799]: from='client.24391 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T20:55:04.530 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:55:04 smithi186 ceph-mon[36597]: from='client.14606 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T20:55:04.530 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:55:04 smithi186 ceph-mon[36597]: from='client.24391 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T20:55:04.562 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:55:04.563+0000 7f529a400700 1 -- 172.21.15.18:0/2046272903 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f529410bee0 con 0x7f529406e940 2024-04-12T20:55:04.563 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:55:04.564+0000 7f528b400700 1 -- 172.21.15.18:0/2046272903 <== mon.1 v2:172.21.15.186:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+739 (secure 0 0 0) 0x7f5284019020 con 0x7f529406e940 2024-04-12T20:55:04.563 INFO:teuthology.orchestra.run.smithi018.stdout:{ 2024-04-12T20:55:04.563 INFO:teuthology.orchestra.run.smithi018.stdout: "mon": { 2024-04-12T20:55:04.564 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 17.2.7-827-gd3b5060a (d3b5060a17449edfae9385cdf792afec109fca5d) quincy (stable)": 2 2024-04-12T20:55:04.564 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-12T20:55:04.564 INFO:teuthology.orchestra.run.smithi018.stdout: "mgr": { 2024-04-12T20:55:04.564 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 19.0.0-2879-g324baf8d (324baf8ddb91dea46cac50cb535c306115791355) squid (dev)": 1 2024-04-12T20:55:04.564 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-12T20:55:04.564 INFO:teuthology.orchestra.run.smithi018.stdout: "osd": { 2024-04-12T20:55:04.564 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 17.2.7-827-gd3b5060a (d3b5060a17449edfae9385cdf792afec109fca5d) quincy (stable)": 6 2024-04-12T20:55:04.564 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-12T20:55:04.564 INFO:teuthology.orchestra.run.smithi018.stdout: "mds": { 2024-04-12T20:55:04.564 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 17.2.7-827-gd3b5060a (d3b5060a17449edfae9385cdf792afec109fca5d) quincy (stable)": 4 2024-04-12T20:55:04.564 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-12T20:55:04.564 INFO:teuthology.orchestra.run.smithi018.stdout: "overall": { 2024-04-12T20:55:04.565 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 17.2.7-827-gd3b5060a (d3b5060a17449edfae9385cdf792afec109fca5d) quincy (stable)": 12, 2024-04-12T20:55:04.565 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 19.0.0-2879-g324baf8d (324baf8ddb91dea46cac50cb535c306115791355) squid (dev)": 1 2024-04-12T20:55:04.565 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-04-12T20:55:04.565 INFO:teuthology.orchestra.run.smithi018.stdout:} 2024-04-12T20:55:04.568 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:55:04.569+0000 7f529a400700 1 -- 172.21.15.18:0/2046272903 >> [v2:172.21.15.186:6828/2316349902,v1:172.21.15.186:6829/2316349902] conn(0x7f5278039510 msgr2=0x7f527803b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:55:04.568 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:55:04.569+0000 7f529a400700 1 --2- 172.21.15.18:0/2046272903 >> [v2:172.21.15.186:6828/2316349902,v1:172.21.15.186:6829/2316349902] conn(0x7f5278039510 0x7f527803b9c0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f527c00b0d0 tx=0x7f527c009e30 comp rx=0 tx=0).stop 2024-04-12T20:55:04.568 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:55:04.569+0000 7f529a400700 1 -- 172.21.15.18:0/2046272903 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f529406e940 msgr2=0x7f529406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:55:04.568 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:55:04.569+0000 7f529a400700 1 --2- 172.21.15.18:0/2046272903 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f529406e940 0x7f529406edb0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f5284009fd0 tx=0x7f528400b040 comp rx=0 tx=0).stop 2024-04-12T20:55:04.568 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:55:04.569+0000 7f529a400700 1 -- 172.21.15.18:0/2046272903 shutdown_connections 2024-04-12T20:55:04.569 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:55:04.569+0000 7f529a400700 1 --2- 172.21.15.18:0/2046272903 >> [v2:172.21.15.186:6828/2316349902,v1:172.21.15.186:6829/2316349902] conn(0x7f5278039510 0x7f527803b9c0 secure :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f527c00b0d0 tx=0x7f527c009e30 comp rx=0 tx=0).stop 2024-04-12T20:55:04.569 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:55:04.569+0000 7f529a400700 1 --2- 172.21.15.18:0/2046272903 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5294072820 0x7f529406e400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:55:04.569 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:55:04.569+0000 7f529a400700 1 --2- 172.21.15.18:0/2046272903 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f529406e940 0x7f529406edb0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:55:04.569 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:55:04.569+0000 7f529a400700 1 -- 172.21.15.18:0/2046272903 >> 172.21.15.18:0/2046272903 conn(0x7f52940fa950 msgr2=0x7f52940fcfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T20:55:04.569 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:55:04.570+0000 7f529a400700 1 -- 172.21.15.18:0/2046272903 shutdown_connections 2024-04-12T20:55:04.569 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:55:04.570+0000 7f529a400700 1 -- 172.21.15.18:0/2046272903 wait complete. 2024-04-12T20:55:04.770 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:55:04.772+0000 7fd628200700 1 -- 172.21.15.18:0/3567603293 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd6200fe0d0 msgr2=0x7fd6200fe540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:55:04.771 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:55:04.772+0000 7fd628200700 1 --2- 172.21.15.18:0/3567603293 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd6200fe0d0 0x7fd6200fe540 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fd60c004940 tx=0x7fd60c009e30 comp rx=0 tx=0).stop 2024-04-12T20:55:04.771 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:55:04.772+0000 7fd628200700 1 -- 172.21.15.18:0/3567603293 shutdown_connections 2024-04-12T20:55:04.771 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:55:04.772+0000 7fd628200700 1 --2- 172.21.15.18:0/3567603293 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd6200fe0d0 0x7fd6200fe540 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:55:04.771 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:55:04.772+0000 7fd628200700 1 --2- 172.21.15.18:0/3567603293 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd6200fcee0 0x7fd6200fd310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:55:04.771 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:55:04.772+0000 7fd628200700 1 -- 172.21.15.18:0/3567603293 >> 172.21.15.18:0/3567603293 conn(0x7fd6200f8560 msgr2=0x7fd6200fa970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T20:55:04.771 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:55:04.773+0000 7fd628200700 1 -- 172.21.15.18:0/3567603293 shutdown_connections 2024-04-12T20:55:04.771 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:55:04.773+0000 7fd628200700 1 -- 172.21.15.18:0/3567603293 wait complete. 2024-04-12T20:55:04.772 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:55:04.774+0000 7fd628200700 1 Processor -- start 2024-04-12T20:55:04.773 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:55:04.774+0000 7fd628200700 1 -- start start 2024-04-12T20:55:04.774 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:55:04.775+0000 7fd628200700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd6200fcee0 0x7fd62010da10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T20:55:04.774 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:55:04.775+0000 7fd628200700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd6200fe0d0 0x7fd62010df50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T20:55:04.774 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:55:04.775+0000 7fd628200700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd62010e550 con 0x7fd6200fe0d0 2024-04-12T20:55:04.774 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:55:04.775+0000 7fd628200700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd62010ed50 con 0x7fd6200fcee0 2024-04-12T20:55:04.774 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:55:04.775+0000 7fd624e00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd6200fe0d0 0x7fd62010df50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T20:55:04.774 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:55:04.775+0000 7fd625800700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd6200fcee0 0x7fd62010da10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T20:55:04.775 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:55:04.775+0000 7fd625800700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd6200fcee0 0x7fd62010da10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.18:50082/0 (socket says 172.21.15.18:50082) 2024-04-12T20:55:04.775 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:55:04.775+0000 7fd625800700 1 -- 172.21.15.18:0/1297214701 learned_addr learned my addr 172.21.15.18:0/1297214701 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T20:55:04.775 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:55:04.776+0000 7fd625800700 1 -- 172.21.15.18:0/1297214701 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd6200fe0d0 msgr2=0x7fd62010df50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:55:04.775 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:55:04.777+0000 7fd625800700 1 --2- 172.21.15.18:0/1297214701 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd6200fe0d0 0x7fd62010df50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:55:04.775 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:55:04.777+0000 7fd625800700 1 -- 172.21.15.18:0/1297214701 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd60c005040 con 0x7fd6200fcee0 2024-04-12T20:55:04.775 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:55:04.777+0000 7fd624e00700 1 --2- 172.21.15.18:0/1297214701 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd6200fe0d0 0x7fd62010df50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-12T20:55:04.777 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:55:04.779+0000 7fd625800700 1 --2- 172.21.15.18:0/1297214701 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd6200fcee0 0x7fd62010da10 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fd61400eb70 tx=0x7fd61400b040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T20:55:04.778 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:55:04.779+0000 7fd61d600700 1 -- 172.21.15.18:0/1297214701 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd614007cc0 con 0x7fd6200fcee0 2024-04-12T20:55:04.778 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:55:04.779+0000 7fd61d600700 1 -- 172.21.15.18:0/1297214701 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd61400f040 con 0x7fd6200fcee0 2024-04-12T20:55:04.778 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:55:04.779+0000 7fd61d600700 1 -- 172.21.15.18:0/1297214701 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd61400cb20 con 0x7fd6200fcee0 2024-04-12T20:55:04.778 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:55:04.779+0000 7fd628200700 1 -- 172.21.15.18:0/1297214701 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd62010efd0 con 0x7fd6200fcee0 2024-04-12T20:55:04.778 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:55:04.780+0000 7fd628200700 1 -- 172.21.15.18:0/1297214701 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd62010f450 con 0x7fd6200fcee0 2024-04-12T20:55:04.780 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:55:04.781+0000 7fd61d600700 1 -- 172.21.15.18:0/1297214701 <== mon.1 v2:172.21.15.186:3300/0 4 ==== mgrmap(e 23) v1 ==== 46560+0+0 (secure 0 0 0) 0x7fd61400cc80 con 0x7fd6200fcee0 2024-04-12T20:55:04.780 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:55:04.782+0000 7fd61d600700 1 --2- 172.21.15.18:0/1297214701 >> [v2:172.21.15.186:6828/2316349902,v1:172.21.15.186:6829/2316349902] conn(0x7fd60803d890 0x7fd60803fd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T20:55:04.780 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:55:04.782+0000 7fd628200700 1 -- 172.21.15.18:0/1297214701 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd62005f5f0 con 0x7fd6200fcee0 2024-04-12T20:55:04.781 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:55:04.782+0000 7fd61d600700 1 -- 172.21.15.18:0/1297214701 <== mon.1 v2:172.21.15.186:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5808+0+0 (secure 0 0 0) 0x7fd61404ccf0 con 0x7fd6200fcee0 2024-04-12T20:55:04.782 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:55:04.783+0000 7fd624e00700 1 --2- 172.21.15.18:0/1297214701 >> [v2:172.21.15.186:6828/2316349902,v1:172.21.15.186:6829/2316349902] conn(0x7fd60803d890 0x7fd60803fd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T20:55:04.782 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:55:04.784+0000 7fd624e00700 1 --2- 172.21.15.18:0/1297214701 >> [v2:172.21.15.186:6828/2316349902,v1:172.21.15.186:6829/2316349902] conn(0x7fd60803d890 0x7fd60803fd40 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fd60c009fd0 tx=0x7fd60c009e30 comp rx=0 tx=0).ready entity=mgr.24359 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T20:55:04.793 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:55:04.793+0000 7fd61d600700 1 -- 172.21.15.18:0/1297214701 <== mon.1 v2:172.21.15.186:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186454 (secure 0 0 0) 0x7fd614012db0 con 0x7fd6200fcee0 2024-04-12T20:55:05.190 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:55:05.191+0000 7fd628200700 1 -- 172.21.15.18:0/1297214701 --> [v2:172.21.15.186:6828/2316349902,v1:172.21.15.186:6829/2316349902] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fd62010f8e0 con 0x7fd60803d890 2024-04-12T20:55:05.212 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:55:05.212+0000 7fd61d600700 1 -- 172.21.15.18:0/1297214701 <== mgr.24359 v2:172.21.15.186:6828/2316349902 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+310 (secure 0 0 0) 0x7fd62010f8e0 con 0x7fd60803d890 2024-04-12T20:55:05.212 INFO:teuthology.orchestra.run.smithi018.stdout:{ 2024-04-12T20:55:05.213 INFO:teuthology.orchestra.run.smithi018.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:f4452def3be4d350f4a8e5feaf7ebb1511be9298fe68478ba7a2d6fbd7d69a63", 2024-04-12T20:55:05.213 INFO:teuthology.orchestra.run.smithi018.stdout: "in_progress": true, 2024-04-12T20:55:05.213 INFO:teuthology.orchestra.run.smithi018.stdout: "which": "Upgrading daemons of type(s) mgr", 2024-04-12T20:55:05.213 INFO:teuthology.orchestra.run.smithi018.stdout: "services_complete": [], 2024-04-12T20:55:05.213 INFO:teuthology.orchestra.run.smithi018.stdout: "progress": "1/2 daemons upgraded", 2024-04-12T20:55:05.213 INFO:teuthology.orchestra.run.smithi018.stdout: "message": "", 2024-04-12T20:55:05.213 INFO:teuthology.orchestra.run.smithi018.stdout: "is_paused": false 2024-04-12T20:55:05.213 INFO:teuthology.orchestra.run.smithi018.stdout:} 2024-04-12T20:55:05.214 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:55:05.215+0000 7fd628200700 1 -- 172.21.15.18:0/1297214701 >> [v2:172.21.15.186:6828/2316349902,v1:172.21.15.186:6829/2316349902] conn(0x7fd60803d890 msgr2=0x7fd60803fd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:55:05.214 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:55:05.215+0000 7fd628200700 1 --2- 172.21.15.18:0/1297214701 >> [v2:172.21.15.186:6828/2316349902,v1:172.21.15.186:6829/2316349902] conn(0x7fd60803d890 0x7fd60803fd40 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fd60c009fd0 tx=0x7fd60c009e30 comp rx=0 tx=0).stop 2024-04-12T20:55:05.214 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:55:05.215+0000 7fd628200700 1 -- 172.21.15.18:0/1297214701 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd6200fcee0 msgr2=0x7fd62010da10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:55:05.214 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:55:05.215+0000 7fd628200700 1 --2- 172.21.15.18:0/1297214701 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd6200fcee0 0x7fd62010da10 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fd61400eb70 tx=0x7fd61400b040 comp rx=0 tx=0).stop 2024-04-12T20:55:05.214 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:55:05.216+0000 7fd628200700 1 -- 172.21.15.18:0/1297214701 shutdown_connections 2024-04-12T20:55:05.215 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:55:05.216+0000 7fd628200700 1 --2- 172.21.15.18:0/1297214701 >> [v2:172.21.15.186:6828/2316349902,v1:172.21.15.186:6829/2316349902] conn(0x7fd60803d890 0x7fd60803fd40 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:55:05.215 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:55:05.216+0000 7fd628200700 1 --2- 172.21.15.18:0/1297214701 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd6200fcee0 0x7fd62010da10 secure :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fd61400eb70 tx=0x7fd61400b040 comp rx=0 tx=0).stop 2024-04-12T20:55:05.215 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:55:05.216+0000 7fd628200700 1 --2- 172.21.15.18:0/1297214701 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd6200fe0d0 0x7fd62010df50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:55:05.215 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:55:05.216+0000 7fd628200700 1 -- 172.21.15.18:0/1297214701 >> 172.21.15.18:0/1297214701 conn(0x7fd6200f8560 msgr2=0x7fd6200f9c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T20:55:05.216 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:55:05.218+0000 7fd628200700 1 -- 172.21.15.18:0/1297214701 shutdown_connections 2024-04-12T20:55:05.216 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:55:05.218+0000 7fd628200700 1 -- 172.21.15.18:0/1297214701 wait complete. 2024-04-12T20:55:05.468 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:55:05 smithi186 ceph-mon[36597]: pgmap v5: 65 pgs: 65 active+clean; 72 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail 2024-04-12T20:55:05.468 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:55:05 smithi186 ceph-mon[36597]: from='client.24395 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T20:55:05.468 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:55:05 smithi186 ceph-mon[36597]: from='client.? 172.21.15.18:0/2046272903' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T20:55:05.601 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:55:05 smithi018 ceph-mon[28799]: pgmap v5: 65 pgs: 65 active+clean; 72 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail 2024-04-12T20:55:05.601 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:55:05 smithi018 ceph-mon[28799]: from='client.24395 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T20:55:05.601 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:55:05 smithi018 ceph-mon[28799]: from='client.? 172.21.15.18:0/2046272903' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T20:55:06.351 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:55:06 smithi018 ceph-mon[28799]: from='client.24403 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T20:55:06.530 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:55:06 smithi186 ceph-mon[36597]: from='client.24403 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T20:55:07.302 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:55:07 smithi018 ceph-mon[28799]: pgmap v6: 65 pgs: 65 active+clean; 78 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 41 KiB/s rd, 1.2 MiB/s wr, 137 op/s 2024-04-12T20:55:07.530 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:55:07 smithi186 ceph-mon[36597]: pgmap v6: 65 pgs: 65 active+clean; 78 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 41 KiB/s rd, 1.2 MiB/s wr, 137 op/s 2024-04-12T20:55:09.304 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:55:09 smithi018 ceph-mon[28799]: pgmap v7: 65 pgs: 65 active+clean; 84 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 29 KiB/s rd, 1.8 MiB/s wr, 258 op/s 2024-04-12T20:55:09.530 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:55:09 smithi186 ceph-mon[36597]: pgmap v7: 65 pgs: 65 active+clean; 84 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 29 KiB/s rd, 1.8 MiB/s wr, 258 op/s 2024-04-12T20:55:11.241 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:55:11 smithi018 ceph-mon[28799]: pgmap v8: 65 pgs: 65 active+clean; 84 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 23 KiB/s rd, 1.4 MiB/s wr, 200 op/s 2024-04-12T20:55:11.530 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:55:11 smithi186 ceph-mon[36597]: pgmap v8: 65 pgs: 65 active+clean; 84 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 23 KiB/s rd, 1.4 MiB/s wr, 200 op/s 2024-04-12T20:55:13.281 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:55:12 smithi186 ceph-mon[36597]: pgmap v9: 65 pgs: 65 active+clean; 99 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 19 KiB/s rd, 2.6 MiB/s wr, 296 op/s 2024-04-12T20:55:13.281 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:55:12 smithi186 ceph-mon[36597]: from='mgr.24359 172.21.15.186:0/1315906828' entity='mgr.smithi186.kqcfxk' 2024-04-12T20:55:13.281 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:55:12 smithi186 ceph-mon[36597]: Standby manager daemon smithi018.xjzwga started 2024-04-12T20:55:13.281 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:55:12 smithi186 ceph-mon[36597]: from='mgr.? 172.21.15.18:0/3475865256' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi018.xjzwga/crt"}]: dispatch 2024-04-12T20:55:13.281 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:55:12 smithi186 ceph-mon[36597]: from='mgr.24359 172.21.15.186:0/1315906828' entity='mgr.smithi186.kqcfxk' 2024-04-12T20:55:13.281 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:55:12 smithi186 ceph-mon[36597]: from='mgr.? 172.21.15.18:0/3475865256' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-12T20:55:13.281 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:55:12 smithi186 ceph-mon[36597]: from='mgr.? 172.21.15.18:0/3475865256' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi018.xjzwga/key"}]: dispatch 2024-04-12T20:55:13.281 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:55:12 smithi186 ceph-mon[36597]: from='mgr.? 172.21.15.18:0/3475865256' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-12T20:55:13.352 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:55:12 smithi018 ceph-mon[28799]: pgmap v9: 65 pgs: 65 active+clean; 99 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 19 KiB/s rd, 2.6 MiB/s wr, 296 op/s 2024-04-12T20:55:13.352 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:55:12 smithi018 ceph-mon[28799]: from='mgr.24359 172.21.15.186:0/1315906828' entity='mgr.smithi186.kqcfxk' 2024-04-12T20:55:13.352 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:55:12 smithi018 ceph-mon[28799]: Standby manager daemon smithi018.xjzwga started 2024-04-12T20:55:13.352 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:55:12 smithi018 ceph-mon[28799]: from='mgr.? 172.21.15.18:0/3475865256' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi018.xjzwga/crt"}]: dispatch 2024-04-12T20:55:13.352 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:55:12 smithi018 ceph-mon[28799]: from='mgr.24359 172.21.15.186:0/1315906828' entity='mgr.smithi186.kqcfxk' 2024-04-12T20:55:13.352 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:55:12 smithi018 ceph-mon[28799]: from='mgr.? 172.21.15.18:0/3475865256' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-12T20:55:13.352 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:55:12 smithi018 ceph-mon[28799]: from='mgr.? 172.21.15.18:0/3475865256' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi018.xjzwga/key"}]: dispatch 2024-04-12T20:55:13.352 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:55:12 smithi018 ceph-mon[28799]: from='mgr.? 172.21.15.18:0/3475865256' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-12T20:55:14.280 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:55:14 smithi186 ceph-mon[36597]: mgrmap e24: smithi186.kqcfxk(active, since 13s), standbys: smithi018.xjzwga 2024-04-12T20:55:14.280 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:55:14 smithi186 ceph-mon[36597]: from='mgr.24359 172.21.15.186:0/1315906828' entity='mgr.smithi186.kqcfxk' cmd=[{"prefix": "mgr metadata", "who": "smithi018.xjzwga", "id": "smithi018.xjzwga"}]: dispatch 2024-04-12T20:55:14.298 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:55:14 smithi018 ceph-mon[28799]: mgrmap e24: smithi186.kqcfxk(active, since 13s), standbys: smithi018.xjzwga 2024-04-12T20:55:14.298 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:55:14 smithi018 ceph-mon[28799]: from='mgr.24359 172.21.15.186:0/1315906828' entity='mgr.smithi186.kqcfxk' cmd=[{"prefix": "mgr metadata", "who": "smithi018.xjzwga", "id": "smithi018.xjzwga"}]: dispatch 2024-04-12T20:55:15.352 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:55:15 smithi018 ceph-mon[28799]: pgmap v10: 65 pgs: 65 active+clean; 99 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 17 KiB/s rd, 2.4 MiB/s wr, 271 op/s 2024-04-12T20:55:15.352 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:55:15 smithi018 ceph-mon[28799]: from='mgr.24359 172.21.15.186:0/1315906828' entity='mgr.smithi186.kqcfxk' 2024-04-12T20:55:15.352 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:55:15 smithi018 ceph-mon[28799]: from='mgr.24359 172.21.15.186:0/1315906828' entity='mgr.smithi186.kqcfxk' 2024-04-12T20:55:15.352 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:55:15 smithi018 ceph-mon[28799]: from='mgr.24359 172.21.15.186:0/1315906828' entity='mgr.smithi186.kqcfxk' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T20:55:15.455 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:55:15 smithi186 ceph-mon[36597]: pgmap v10: 65 pgs: 65 active+clean; 99 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 17 KiB/s rd, 2.4 MiB/s wr, 271 op/s 2024-04-12T20:55:15.456 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:55:15 smithi186 ceph-mon[36597]: from='mgr.24359 172.21.15.186:0/1315906828' entity='mgr.smithi186.kqcfxk' 2024-04-12T20:55:15.456 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:55:15 smithi186 ceph-mon[36597]: from='mgr.24359 172.21.15.186:0/1315906828' entity='mgr.smithi186.kqcfxk' 2024-04-12T20:55:15.456 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:55:15 smithi186 ceph-mon[36597]: from='mgr.24359 172.21.15.186:0/1315906828' entity='mgr.smithi186.kqcfxk' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T20:55:17.351 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:55:17 smithi018 ceph-mon[28799]: pgmap v11: 65 pgs: 65 active+clean; 107 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 17 KiB/s rd, 3.1 MiB/s wr, 311 op/s 2024-04-12T20:55:17.530 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:55:17 smithi186 ceph-mon[36597]: pgmap v11: 65 pgs: 65 active+clean; 107 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 17 KiB/s rd, 3.1 MiB/s wr, 311 op/s 2024-04-12T20:55:19.531 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:55:19 smithi186 ceph-mon[36597]: pgmap v12: 65 pgs: 65 active+clean; 110 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 2.9 MiB/s wr, 333 op/s 2024-04-12T20:55:19.531 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:55:19 smithi186 ceph-mon[36597]: from='mgr.24359 172.21.15.186:0/1315906828' entity='mgr.smithi186.kqcfxk' 2024-04-12T20:55:19.531 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:55:19 smithi186 ceph-mon[36597]: from='mgr.24359 172.21.15.186:0/1315906828' entity='mgr.smithi186.kqcfxk' 2024-04-12T20:55:19.601 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:55:19 smithi018 ceph-mon[28799]: pgmap v12: 65 pgs: 65 active+clean; 110 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 2.9 MiB/s wr, 333 op/s 2024-04-12T20:55:19.601 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:55:19 smithi018 ceph-mon[28799]: from='mgr.24359 172.21.15.186:0/1315906828' entity='mgr.smithi186.kqcfxk' 2024-04-12T20:55:19.601 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:55:19 smithi018 ceph-mon[28799]: from='mgr.24359 172.21.15.186:0/1315906828' entity='mgr.smithi186.kqcfxk' 2024-04-12T20:55:21.281 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:55:20 smithi186 ceph-mon[36597]: pgmap v13: 65 pgs: 65 active+clean; 110 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 2.4 MiB/s wr, 240 op/s 2024-04-12T20:55:21.281 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:55:20 smithi186 ceph-mon[36597]: from='mgr.24359 172.21.15.186:0/1315906828' entity='mgr.smithi186.kqcfxk' 2024-04-12T20:55:21.281 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:55:20 smithi186 ceph-mon[36597]: from='mgr.24359 172.21.15.186:0/1315906828' entity='mgr.smithi186.kqcfxk' 2024-04-12T20:55:21.282 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:55:20 smithi186 ceph-mon[36597]: from='mgr.24359 172.21.15.186:0/1315906828' entity='mgr.smithi186.kqcfxk' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-12T20:55:21.282 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:55:20 smithi186 ceph-mon[36597]: from='mgr.24359 172.21.15.186:0/1315906828' entity='mgr.smithi186.kqcfxk' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-12T20:55:21.282 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:55:20 smithi186 ceph-mon[36597]: from='mgr.24359 172.21.15.186:0/1315906828' entity='mgr.smithi186.kqcfxk' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-12T20:55:21.282 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:55:20 smithi186 ceph-mon[36597]: Adjusting osd_memory_target on smithi186 to 2581M 2024-04-12T20:55:21.282 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:55:20 smithi186 ceph-mon[36597]: from='mgr.24359 172.21.15.186:0/1315906828' entity='mgr.smithi186.kqcfxk' 2024-04-12T20:55:21.351 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:55:20 smithi018 ceph-mon[28799]: pgmap v13: 65 pgs: 65 active+clean; 110 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 2.4 MiB/s wr, 240 op/s 2024-04-12T20:55:21.352 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:55:20 smithi018 ceph-mon[28799]: from='mgr.24359 172.21.15.186:0/1315906828' entity='mgr.smithi186.kqcfxk' 2024-04-12T20:55:21.352 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:55:20 smithi018 ceph-mon[28799]: from='mgr.24359 172.21.15.186:0/1315906828' entity='mgr.smithi186.kqcfxk' 2024-04-12T20:55:21.352 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:55:20 smithi018 ceph-mon[28799]: from='mgr.24359 172.21.15.186:0/1315906828' entity='mgr.smithi186.kqcfxk' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-12T20:55:21.352 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:55:20 smithi018 ceph-mon[28799]: from='mgr.24359 172.21.15.186:0/1315906828' entity='mgr.smithi186.kqcfxk' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-12T20:55:21.352 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:55:20 smithi018 ceph-mon[28799]: from='mgr.24359 172.21.15.186:0/1315906828' entity='mgr.smithi186.kqcfxk' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-12T20:55:21.352 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:55:20 smithi018 ceph-mon[28799]: Adjusting osd_memory_target on smithi186 to 2581M 2024-04-12T20:55:21.352 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:55:20 smithi018 ceph-mon[28799]: from='mgr.24359 172.21.15.186:0/1315906828' entity='mgr.smithi186.kqcfxk' 2024-04-12T20:55:22.602 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:55:22 smithi018 ceph-mon[28799]: from='mgr.24359 172.21.15.186:0/1315906828' entity='mgr.smithi186.kqcfxk' 2024-04-12T20:55:22.602 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:55:22 smithi018 ceph-mon[28799]: from='mgr.24359 172.21.15.186:0/1315906828' entity='mgr.smithi186.kqcfxk' 2024-04-12T20:55:22.602 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:55:22 smithi018 ceph-mon[28799]: from='mgr.24359 172.21.15.186:0/1315906828' entity='mgr.smithi186.kqcfxk' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-12T20:55:22.602 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:55:22 smithi018 ceph-mon[28799]: from='mgr.24359 172.21.15.186:0/1315906828' entity='mgr.smithi186.kqcfxk' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-12T20:55:22.602 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:55:22 smithi018 ceph-mon[28799]: from='mgr.24359 172.21.15.186:0/1315906828' entity='mgr.smithi186.kqcfxk' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-12T20:55:22.602 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:55:22 smithi018 ceph-mon[28799]: Adjusting osd_memory_target on smithi018 to 1559M 2024-04-12T20:55:22.602 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:55:22 smithi018 ceph-mon[28799]: from='mgr.24359 172.21.15.186:0/1315906828' entity='mgr.smithi186.kqcfxk' 2024-04-12T20:55:22.602 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:55:22 smithi018 ceph-mon[28799]: from='mgr.24359 172.21.15.186:0/1315906828' entity='mgr.smithi186.kqcfxk' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-12T20:55:22.603 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:55:22 smithi018 ceph-mon[28799]: from='mgr.24359 172.21.15.186:0/1315906828' entity='mgr.smithi186.kqcfxk' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-12T20:55:22.603 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:55:22 smithi018 ceph-mon[28799]: Updating smithi018:/etc/ceph/ceph.conf 2024-04-12T20:55:22.603 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:55:22 smithi018 ceph-mon[28799]: Updating smithi186:/etc/ceph/ceph.conf 2024-04-12T20:55:22.603 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:55:22 smithi018 ceph-mon[28799]: pgmap v14: 65 pgs: 65 active+clean; 117 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 3.0 MiB/s wr, 354 op/s 2024-04-12T20:55:22.603 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:55:22 smithi018 ceph-mon[28799]: Updating smithi186:/var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/config/ceph.conf 2024-04-12T20:55:22.603 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:55:22 smithi018 ceph-mon[28799]: Updating smithi018:/var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/config/ceph.conf 2024-04-12T20:55:22.780 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:55:22 smithi186 ceph-mon[36597]: from='mgr.24359 172.21.15.186:0/1315906828' entity='mgr.smithi186.kqcfxk' 2024-04-12T20:55:22.780 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:55:22 smithi186 ceph-mon[36597]: from='mgr.24359 172.21.15.186:0/1315906828' entity='mgr.smithi186.kqcfxk' 2024-04-12T20:55:22.781 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:55:22 smithi186 ceph-mon[36597]: from='mgr.24359 172.21.15.186:0/1315906828' entity='mgr.smithi186.kqcfxk' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-12T20:55:22.781 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:55:22 smithi186 ceph-mon[36597]: from='mgr.24359 172.21.15.186:0/1315906828' entity='mgr.smithi186.kqcfxk' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-12T20:55:22.781 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:55:22 smithi186 ceph-mon[36597]: from='mgr.24359 172.21.15.186:0/1315906828' entity='mgr.smithi186.kqcfxk' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-12T20:55:22.781 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:55:22 smithi186 ceph-mon[36597]: Adjusting osd_memory_target on smithi018 to 1559M 2024-04-12T20:55:22.781 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:55:22 smithi186 ceph-mon[36597]: from='mgr.24359 172.21.15.186:0/1315906828' entity='mgr.smithi186.kqcfxk' 2024-04-12T20:55:22.781 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:55:22 smithi186 ceph-mon[36597]: from='mgr.24359 172.21.15.186:0/1315906828' entity='mgr.smithi186.kqcfxk' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-12T20:55:22.781 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:55:22 smithi186 ceph-mon[36597]: from='mgr.24359 172.21.15.186:0/1315906828' entity='mgr.smithi186.kqcfxk' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-12T20:55:22.781 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:55:22 smithi186 ceph-mon[36597]: Updating smithi018:/etc/ceph/ceph.conf 2024-04-12T20:55:22.781 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:55:22 smithi186 ceph-mon[36597]: Updating smithi186:/etc/ceph/ceph.conf 2024-04-12T20:55:22.781 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:55:22 smithi186 ceph-mon[36597]: pgmap v14: 65 pgs: 65 active+clean; 117 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 3.0 MiB/s wr, 354 op/s 2024-04-12T20:55:22.781 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:55:22 smithi186 ceph-mon[36597]: Updating smithi186:/var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/config/ceph.conf 2024-04-12T20:55:22.781 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:55:22 smithi186 ceph-mon[36597]: Updating smithi018:/var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/config/ceph.conf 2024-04-12T20:55:23.780 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:55:23 smithi186 ceph-mon[36597]: Updating smithi186:/etc/ceph/ceph.client.admin.keyring 2024-04-12T20:55:23.780 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:55:23 smithi186 ceph-mon[36597]: Updating smithi018:/etc/ceph/ceph.client.admin.keyring 2024-04-12T20:55:23.851 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:55:23 smithi018 ceph-mon[28799]: Updating smithi186:/etc/ceph/ceph.client.admin.keyring 2024-04-12T20:55:23.851 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:55:23 smithi018 ceph-mon[28799]: Updating smithi018:/etc/ceph/ceph.client.admin.keyring 2024-04-12T20:55:25.352 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:55:25 smithi018 ceph-mon[28799]: Updating smithi186:/var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/config/ceph.client.admin.keyring 2024-04-12T20:55:25.352 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:55:25 smithi018 ceph-mon[28799]: pgmap v15: 65 pgs: 65 active+clean; 117 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 1.7 MiB/s wr, 233 op/s 2024-04-12T20:55:25.352 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:55:25 smithi018 ceph-mon[28799]: Updating smithi018:/var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/config/ceph.client.admin.keyring 2024-04-12T20:55:25.352 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:55:25 smithi018 ceph-mon[28799]: from='mgr.24359 172.21.15.186:0/1315906828' entity='mgr.smithi186.kqcfxk' 2024-04-12T20:55:25.352 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:55:25 smithi018 ceph-mon[28799]: from='mgr.24359 172.21.15.186:0/1315906828' entity='mgr.smithi186.kqcfxk' 2024-04-12T20:55:25.352 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:55:25 smithi018 ceph-mon[28799]: from='mgr.24359 172.21.15.186:0/1315906828' entity='mgr.smithi186.kqcfxk' 2024-04-12T20:55:25.352 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:55:25 smithi018 ceph-mon[28799]: from='mgr.24359 172.21.15.186:0/1315906828' entity='mgr.smithi186.kqcfxk' 2024-04-12T20:55:25.352 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:55:25 smithi018 ceph-mon[28799]: from='mgr.24359 172.21.15.186:0/1315906828' entity='mgr.smithi186.kqcfxk' 2024-04-12T20:55:25.352 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:55:25 smithi018 ceph-mon[28799]: from='mgr.24359 172.21.15.186:0/1315906828' entity='mgr.smithi186.kqcfxk' 2024-04-12T20:55:25.352 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:55:25 smithi018 ceph-mon[28799]: from='mgr.24359 172.21.15.186:0/1315906828' entity='mgr.smithi186.kqcfxk' 2024-04-12T20:55:25.352 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:55:25 smithi018 ceph-mon[28799]: from='mgr.24359 172.21.15.186:0/1315906828' entity='mgr.smithi186.kqcfxk' 2024-04-12T20:55:25.352 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:55:25 smithi018 ceph-mon[28799]: from='mgr.24359 172.21.15.186:0/1315906828' entity='mgr.smithi186.kqcfxk' 2024-04-12T20:55:25.458 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:55:25 smithi186 ceph-mon[36597]: Updating smithi186:/var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/config/ceph.client.admin.keyring 2024-04-12T20:55:25.458 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:55:25 smithi186 ceph-mon[36597]: pgmap v15: 65 pgs: 65 active+clean; 117 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 1.7 MiB/s wr, 233 op/s 2024-04-12T20:55:25.459 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:55:25 smithi186 ceph-mon[36597]: Updating smithi018:/var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/config/ceph.client.admin.keyring 2024-04-12T20:55:25.459 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:55:25 smithi186 ceph-mon[36597]: from='mgr.24359 172.21.15.186:0/1315906828' entity='mgr.smithi186.kqcfxk' 2024-04-12T20:55:25.459 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:55:25 smithi186 ceph-mon[36597]: from='mgr.24359 172.21.15.186:0/1315906828' entity='mgr.smithi186.kqcfxk' 2024-04-12T20:55:25.459 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:55:25 smithi186 ceph-mon[36597]: from='mgr.24359 172.21.15.186:0/1315906828' entity='mgr.smithi186.kqcfxk' 2024-04-12T20:55:25.459 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:55:25 smithi186 ceph-mon[36597]: from='mgr.24359 172.21.15.186:0/1315906828' entity='mgr.smithi186.kqcfxk' 2024-04-12T20:55:25.459 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:55:25 smithi186 ceph-mon[36597]: from='mgr.24359 172.21.15.186:0/1315906828' entity='mgr.smithi186.kqcfxk' 2024-04-12T20:55:25.459 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:55:25 smithi186 ceph-mon[36597]: from='mgr.24359 172.21.15.186:0/1315906828' entity='mgr.smithi186.kqcfxk' 2024-04-12T20:55:25.459 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:55:25 smithi186 ceph-mon[36597]: from='mgr.24359 172.21.15.186:0/1315906828' entity='mgr.smithi186.kqcfxk' 2024-04-12T20:55:25.459 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:55:25 smithi186 ceph-mon[36597]: from='mgr.24359 172.21.15.186:0/1315906828' entity='mgr.smithi186.kqcfxk' 2024-04-12T20:55:25.459 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:55:25 smithi186 ceph-mon[36597]: from='mgr.24359 172.21.15.186:0/1315906828' entity='mgr.smithi186.kqcfxk' 2024-04-12T20:55:26.353 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:55:26 smithi018 ceph-mon[28799]: Reconfiguring node-exporter.smithi018 (dependencies changed)... 2024-04-12T20:55:26.353 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:55:26 smithi018 ceph-mon[28799]: Reconfiguring daemon node-exporter.smithi018 on smithi018 2024-04-12T20:55:26.530 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:55:26 smithi186 ceph-mon[36597]: Reconfiguring node-exporter.smithi018 (dependencies changed)... 2024-04-12T20:55:26.531 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:55:26 smithi186 ceph-mon[36597]: Reconfiguring daemon node-exporter.smithi018 on smithi018 2024-04-12T20:55:27.247 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:55:27 smithi018 ceph-mon[28799]: pgmap v16: 65 pgs: 65 active+clean; 126 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 2.4 MiB/s wr, 298 op/s 2024-04-12T20:55:27.530 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:55:27 smithi186 ceph-mon[36597]: pgmap v16: 65 pgs: 65 active+clean; 126 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 2.4 MiB/s wr, 298 op/s 2024-04-12T20:55:29.479 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:55:29 smithi018 ceph-mon[28799]: pgmap v17: 65 pgs: 65 active+clean; 133 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 2.4 MiB/s wr, 323 op/s 2024-04-12T20:55:29.479 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:55:29 smithi018 ceph-mon[28799]: from='mgr.24359 172.21.15.186:0/1315906828' entity='mgr.smithi186.kqcfxk' 2024-04-12T20:55:29.479 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:55:29 smithi018 ceph-mon[28799]: from='mgr.24359 172.21.15.186:0/1315906828' entity='mgr.smithi186.kqcfxk' 2024-04-12T20:55:29.581 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:55:29 smithi186 ceph-mon[36597]: pgmap v17: 65 pgs: 65 active+clean; 133 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 2.4 MiB/s wr, 323 op/s 2024-04-12T20:55:29.581 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:55:29 smithi186 ceph-mon[36597]: from='mgr.24359 172.21.15.186:0/1315906828' entity='mgr.smithi186.kqcfxk' 2024-04-12T20:55:29.581 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:55:29 smithi186 ceph-mon[36597]: from='mgr.24359 172.21.15.186:0/1315906828' entity='mgr.smithi186.kqcfxk' 2024-04-12T20:55:30.444 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:55:30 smithi018 ceph-mon[28799]: Reconfiguring alertmanager.smithi018 (dependencies changed)... 2024-04-12T20:55:30.444 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:55:30 smithi018 ceph-mon[28799]: Reconfiguring daemon alertmanager.smithi018 on smithi018 2024-04-12T20:55:30.445 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:55:30 smithi018 ceph-mon[28799]: from='mgr.24359 172.21.15.186:0/1315906828' entity='mgr.smithi186.kqcfxk' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T20:55:30.780 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:55:30 smithi186 ceph-mon[36597]: Reconfiguring alertmanager.smithi018 (dependencies changed)... 2024-04-12T20:55:30.780 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:55:30 smithi186 ceph-mon[36597]: Reconfiguring daemon alertmanager.smithi018 on smithi018 2024-04-12T20:55:30.780 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:55:30 smithi186 ceph-mon[36597]: from='mgr.24359 172.21.15.186:0/1315906828' entity='mgr.smithi186.kqcfxk' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T20:55:31.351 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:55:31 smithi018 ceph-mon[28799]: pgmap v18: 65 pgs: 65 active+clean; 133 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 2.0 MiB/s wr, 244 op/s 2024-04-12T20:55:31.780 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:55:31 smithi186 ceph-mon[36597]: pgmap v18: 65 pgs: 65 active+clean; 133 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 2.0 MiB/s wr, 244 op/s 2024-04-12T20:55:32.601 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:55:32 smithi018 ceph-mon[28799]: pgmap v19: 65 pgs: 65 active+clean; 144 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 3.0 MiB/s wr, 354 op/s 2024-04-12T20:55:32.780 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:55:32 smithi186 ceph-mon[36597]: pgmap v19: 65 pgs: 65 active+clean; 144 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 3.0 MiB/s wr, 354 op/s 2024-04-12T20:55:34.852 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:55:34 smithi018 ceph-mon[28799]: pgmap v20: 65 pgs: 65 active+clean; 144 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 2.3 MiB/s wr, 239 op/s 2024-04-12T20:55:35.030 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:55:34 smithi186 ceph-mon[36597]: pgmap v20: 65 pgs: 65 active+clean; 144 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 2.3 MiB/s wr, 239 op/s 2024-04-12T20:55:35.446 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:55:35.448+0000 7fcf22e00700 1 -- 172.21.15.18:0/1229747135 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fcf1c071f50 msgr2=0x7fcf1c070460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:55:35.446 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:55:35.448+0000 7fcf22e00700 1 --2- 172.21.15.18:0/1229747135 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fcf1c071f50 0x7fcf1c070460 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fcf0c004940 tx=0x7fcf0c009e30 comp rx=0 tx=0).stop 2024-04-12T20:55:35.447 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:55:35.449+0000 7fcf22e00700 1 -- 172.21.15.18:0/1229747135 shutdown_connections 2024-04-12T20:55:35.447 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:55:35.449+0000 7fcf22e00700 1 --2- 172.21.15.18:0/1229747135 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcf1c0709a0 0x7fcf1c070e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:55:35.447 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:55:35.449+0000 7fcf22e00700 1 --2- 172.21.15.18:0/1229747135 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fcf1c071f50 0x7fcf1c070460 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:55:35.447 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:55:35.449+0000 7fcf22e00700 1 -- 172.21.15.18:0/1229747135 >> 172.21.15.18:0/1229747135 conn(0x7fcf1c0f8560 msgr2=0x7fcf1c0fa970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T20:55:35.447 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:55:35.449+0000 7fcf22e00700 1 -- 172.21.15.18:0/1229747135 shutdown_connections 2024-04-12T20:55:35.447 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:55:35.449+0000 7fcf22e00700 1 -- 172.21.15.18:0/1229747135 wait complete. 2024-04-12T20:55:35.447 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:55:35.450+0000 7fcf22e00700 1 Processor -- start 2024-04-12T20:55:35.448 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:55:35.451+0000 7fcf22e00700 1 -- start start 2024-04-12T20:55:35.448 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:55:35.451+0000 7fcf22e00700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcf1c0709a0 0x7fcf1c10fd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T20:55:35.448 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:55:35.451+0000 7fcf22e00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fcf1c071f50 0x7fcf1c110270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T20:55:35.448 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:55:35.451+0000 7fcf22e00700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf1c110870 con 0x7fcf1c071f50 2024-04-12T20:55:35.448 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:55:35.451+0000 7fcf22e00700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf1c110fb0 con 0x7fcf1c0709a0 2024-04-12T20:55:35.449 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:55:35.451+0000 7fcf1b400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fcf1c071f50 0x7fcf1c110270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T20:55:35.449 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:55:35.452+0000 7fcf1b400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fcf1c071f50 0x7fcf1c110270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:53300/0 (socket says 172.21.15.18:53300) 2024-04-12T20:55:35.449 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:55:35.452+0000 7fcf1b400700 1 -- 172.21.15.18:0/1249018404 learned_addr learned my addr 172.21.15.18:0/1249018404 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T20:55:35.449 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:55:35.452+0000 7fcf1be00700 1 --2- 172.21.15.18:0/1249018404 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcf1c0709a0 0x7fcf1c10fd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T20:55:35.449 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:55:35.452+0000 7fcf1b400700 1 -- 172.21.15.18:0/1249018404 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcf1c0709a0 msgr2=0x7fcf1c10fd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:55:35.450 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:55:35.452+0000 7fcf1b400700 1 --2- 172.21.15.18:0/1249018404 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcf1c0709a0 0x7fcf1c10fd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:55:35.450 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:55:35.452+0000 7fcf1b400700 1 -- 172.21.15.18:0/1249018404 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf0c005040 con 0x7fcf1c071f50 2024-04-12T20:55:35.450 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:55:35.453+0000 7fcf1b400700 1 --2- 172.21.15.18:0/1249018404 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fcf1c071f50 0x7fcf1c110270 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fcf0800c8c0 tx=0x7fcf080044c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T20:55:35.451 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:55:35.453+0000 7fcf13e00700 1 -- 172.21.15.18:0/1249018404 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcf0800a980 con 0x7fcf1c071f50 2024-04-12T20:55:35.451 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:55:35.453+0000 7fcf22e00700 1 -- 172.21.15.18:0/1249018404 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcf1c111290 con 0x7fcf1c071f50 2024-04-12T20:55:35.451 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:55:35.453+0000 7fcf22e00700 1 -- 172.21.15.18:0/1249018404 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcf1c111850 con 0x7fcf1c071f50 2024-04-12T20:55:35.451 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:55:35.454+0000 7fcf13e00700 1 -- 172.21.15.18:0/1249018404 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fcf0800ce60 con 0x7fcf1c071f50 2024-04-12T20:55:35.451 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:55:35.454+0000 7fcf13e00700 1 -- 172.21.15.18:0/1249018404 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcf08012400 con 0x7fcf1c071f50 2024-04-12T20:55:35.454 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:55:35.455+0000 7fcf22e00700 1 -- 172.21.15.18:0/1249018404 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcefc0052e0 con 0x7fcf1c071f50 2024-04-12T20:55:35.454 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:55:35.457+0000 7fcf13e00700 1 -- 172.21.15.18:0/1249018404 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 24) v1 ==== 89644+0+0 (secure 0 0 0) 0x7fcf0800aae0 con 0x7fcf1c071f50 2024-04-12T20:55:35.455 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:55:35.458+0000 7fcf13e00700 1 --2- 172.21.15.18:0/1249018404 >> [v2:172.21.15.186:6828/2316349902,v1:172.21.15.186:6829/2316349902] conn(0x7fcf0006bcf0 0x7fcf0006e1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T20:55:35.455 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:55:35.458+0000 7fcf13e00700 1 -- 172.21.15.18:0/1249018404 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5808+0+0 (secure 0 0 0) 0x7fcf0808aa20 con 0x7fcf1c071f50 2024-04-12T20:55:35.456 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:55:35.459+0000 7fcf1be00700 1 --2- 172.21.15.18:0/1249018404 >> [v2:172.21.15.186:6828/2316349902,v1:172.21.15.186:6829/2316349902] conn(0x7fcf0006bcf0 0x7fcf0006e1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T20:55:35.458 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:55:35.461+0000 7fcf1be00700 1 --2- 172.21.15.18:0/1249018404 >> [v2:172.21.15.186:6828/2316349902,v1:172.21.15.186:6829/2316349902] conn(0x7fcf0006bcf0 0x7fcf0006e1a0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fcf0c00b3b0 tx=0x7fcf0c009e30 comp rx=0 tx=0).ready entity=mgr.24359 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T20:55:35.463 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:55:35.466+0000 7fcf13e00700 1 -- 172.21.15.18:0/1249018404 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186454 (secure 0 0 0) 0x7fcf08057c20 con 0x7fcf1c071f50 2024-04-12T20:55:35.841 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:55:35.843+0000 7fcf22e00700 1 -- 172.21.15.18:0/1249018404 --> [v2:172.21.15.186:6828/2316349902,v1:172.21.15.186:6829/2316349902] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fcefc000bf0 con 0x7fcf0006bcf0 2024-04-12T20:55:35.843 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:55:35.846+0000 7fcf13e00700 1 -- 172.21.15.18:0/1249018404 <== mgr.24359 v2:172.21.15.186:6828/2316349902 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+310 (secure 0 0 0) 0x7fcefc000bf0 con 0x7fcf0006bcf0 2024-04-12T20:55:35.847 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:55:35.850+0000 7fcf22e00700 1 -- 172.21.15.18:0/1249018404 >> [v2:172.21.15.186:6828/2316349902,v1:172.21.15.186:6829/2316349902] conn(0x7fcf0006bcf0 msgr2=0x7fcf0006e1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:55:35.847 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:55:35.850+0000 7fcf22e00700 1 --2- 172.21.15.18:0/1249018404 >> [v2:172.21.15.186:6828/2316349902,v1:172.21.15.186:6829/2316349902] conn(0x7fcf0006bcf0 0x7fcf0006e1a0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fcf0c00b3b0 tx=0x7fcf0c009e30 comp rx=0 tx=0).stop 2024-04-12T20:55:35.847 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:55:35.850+0000 7fcf22e00700 1 -- 172.21.15.18:0/1249018404 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fcf1c071f50 msgr2=0x7fcf1c110270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:55:35.847 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:55:35.850+0000 7fcf22e00700 1 --2- 172.21.15.18:0/1249018404 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fcf1c071f50 0x7fcf1c110270 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fcf0800c8c0 tx=0x7fcf080044c0 comp rx=0 tx=0).stop 2024-04-12T20:55:35.847 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:55:35.850+0000 7fcf22e00700 1 -- 172.21.15.18:0/1249018404 shutdown_connections 2024-04-12T20:55:35.848 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:55:35.850+0000 7fcf22e00700 1 --2- 172.21.15.18:0/1249018404 >> [v2:172.21.15.186:6828/2316349902,v1:172.21.15.186:6829/2316349902] conn(0x7fcf0006bcf0 0x7fcf0006e1a0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:55:35.848 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:55:35.850+0000 7fcf22e00700 1 --2- 172.21.15.18:0/1249018404 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcf1c0709a0 0x7fcf1c10fd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:55:35.848 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:55:35.850+0000 7fcf22e00700 1 --2- 172.21.15.18:0/1249018404 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fcf1c071f50 0x7fcf1c110270 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:55:35.848 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:55:35.850+0000 7fcf22e00700 1 -- 172.21.15.18:0/1249018404 >> 172.21.15.18:0/1249018404 conn(0x7fcf1c0f8560 msgr2=0x7fcf1c0f9cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T20:55:35.848 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:55:35.850+0000 7fcf22e00700 1 -- 172.21.15.18:0/1249018404 shutdown_connections 2024-04-12T20:55:35.848 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:55:35.850+0000 7fcf22e00700 1 -- 172.21.15.18:0/1249018404 wait complete. 2024-04-12T20:55:35.868 INFO:teuthology.orchestra.run.smithi018.stdout:true 2024-04-12T20:55:36.044 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:55:36.047+0000 7ff584400700 1 -- 172.21.15.18:0/2064986552 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff57c0709a0 msgr2=0x7ff57c070e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:55:36.044 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:55:36.047+0000 7ff584400700 1 --2- 172.21.15.18:0/2064986552 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff57c0709a0 0x7ff57c070e10 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7ff568004940 tx=0x7ff568009e30 comp rx=0 tx=0).stop 2024-04-12T20:55:36.045 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:55:36.047+0000 7ff584400700 1 -- 172.21.15.18:0/2064986552 shutdown_connections 2024-04-12T20:55:36.045 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:55:36.047+0000 7ff584400700 1 --2- 172.21.15.18:0/2064986552 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff57c0709a0 0x7ff57c070e10 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:55:36.045 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:55:36.047+0000 7ff584400700 1 --2- 172.21.15.18:0/2064986552 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff57c071f50 0x7ff57c070460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:55:36.045 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:55:36.047+0000 7ff584400700 1 -- 172.21.15.18:0/2064986552 >> 172.21.15.18:0/2064986552 conn(0x7ff57c0f8560 msgr2=0x7ff57c0fa970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T20:55:36.045 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:55:36.048+0000 7ff584400700 1 -- 172.21.15.18:0/2064986552 shutdown_connections 2024-04-12T20:55:36.045 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:55:36.048+0000 7ff584400700 1 -- 172.21.15.18:0/2064986552 wait complete. 2024-04-12T20:55:36.046 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:55:36.049+0000 7ff584400700 1 Processor -- start 2024-04-12T20:55:36.046 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:55:36.049+0000 7ff584400700 1 -- start start 2024-04-12T20:55:36.047 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:55:36.049+0000 7ff584400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff57c0709a0 0x7ff57c114000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T20:55:36.047 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:55:36.049+0000 7ff584400700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff57c071f50 0x7ff57c114540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T20:55:36.047 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:55:36.049+0000 7ff584400700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff57c114b40 con 0x7ff57c0709a0 2024-04-12T20:55:36.047 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:55:36.049+0000 7ff584400700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff57c115390 con 0x7ff57c071f50 2024-04-12T20:55:36.047 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:55:36.050+0000 7ff581a00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff57c0709a0 0x7ff57c114000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T20:55:36.048 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:55:36.050+0000 7ff581000700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff57c071f50 0x7ff57c114540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T20:55:36.048 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:55:36.050+0000 7ff581a00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff57c0709a0 0x7ff57c114000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:53304/0 (socket says 172.21.15.18:53304) 2024-04-12T20:55:36.048 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:55:36.050+0000 7ff581a00700 1 -- 172.21.15.18:0/1424068310 learned_addr learned my addr 172.21.15.18:0/1424068310 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T20:55:36.048 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:55:36.051+0000 7ff581000700 1 -- 172.21.15.18:0/1424068310 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff57c0709a0 msgr2=0x7ff57c114000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:55:36.048 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:55:36.051+0000 7ff581000700 1 --2- 172.21.15.18:0/1424068310 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff57c0709a0 0x7ff57c114000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:55:36.048 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:55:36.051+0000 7ff581000700 1 -- 172.21.15.18:0/1424068310 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff568005040 con 0x7ff57c071f50 2024-04-12T20:55:36.048 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:55:36.051+0000 7ff581a00700 1 --2- 172.21.15.18:0/1424068310 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff57c0709a0 0x7ff57c114000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-12T20:55:36.049 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:55:36.052+0000 7ff581000700 1 --2- 172.21.15.18:0/1424068310 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff57c071f50 0x7ff57c114540 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7ff56800b970 tx=0x7ff568009e30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T20:55:36.049 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:55:36.052+0000 7ff579600700 1 -- 172.21.15.18:0/1424068310 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff56800e040 con 0x7ff57c071f50 2024-04-12T20:55:36.050 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:55:36.052+0000 7ff579600700 1 -- 172.21.15.18:0/1424068310 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff56800bbf0 con 0x7ff57c071f50 2024-04-12T20:55:36.050 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:55:36.052+0000 7ff579600700 1 -- 172.21.15.18:0/1424068310 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff56801c3f0 con 0x7ff57c071f50 2024-04-12T20:55:36.051 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:55:36.053+0000 7ff584400700 1 -- 172.21.15.18:0/1424068310 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff57c115610 con 0x7ff57c071f50 2024-04-12T20:55:36.051 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:55:36.053+0000 7ff584400700 1 -- 172.21.15.18:0/1424068310 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff57c115b50 con 0x7ff57c071f50 2024-04-12T20:55:36.052 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:55:36.055+0000 7ff584400700 1 -- 172.21.15.18:0/1424068310 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff57c05f5f0 con 0x7ff57c071f50 2024-04-12T20:55:36.056 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:55:36.058+0000 7ff579600700 1 -- 172.21.15.18:0/1424068310 <== mon.1 v2:172.21.15.186:3300/0 4 ==== mgrmap(e 24) v1 ==== 89644+0+0 (secure 0 0 0) 0x7ff56800bd60 con 0x7ff57c071f50 2024-04-12T20:55:36.057 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:55:36.060+0000 7ff579600700 1 --2- 172.21.15.18:0/1424068310 >> [v2:172.21.15.186:6828/2316349902,v1:172.21.15.186:6829/2316349902] conn(0x7ff56406bcf0 0x7ff56406e1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T20:55:36.057 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:55:36.060+0000 7ff579600700 1 -- 172.21.15.18:0/1424068310 <== mon.1 v2:172.21.15.186:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5808+0+0 (secure 0 0 0) 0x7ff56808c800 con 0x7ff57c071f50 2024-04-12T20:55:36.058 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:55:36.061+0000 7ff581a00700 1 --2- 172.21.15.18:0/1424068310 >> [v2:172.21.15.186:6828/2316349902,v1:172.21.15.186:6829/2316349902] conn(0x7ff56406bcf0 0x7ff56406e1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T20:55:36.059 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:55:36.062+0000 7ff581a00700 1 --2- 172.21.15.18:0/1424068310 >> [v2:172.21.15.186:6828/2316349902,v1:172.21.15.186:6829/2316349902] conn(0x7ff56406bcf0 0x7ff56406e1a0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7ff57000e560 tx=0x7ff57000b040 comp rx=0 tx=0).ready entity=mgr.24359 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T20:55:36.075 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:55:36.078+0000 7ff579600700 1 -- 172.21.15.18:0/1424068310 <== mon.1 v2:172.21.15.186:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186454 (secure 0 0 0) 0x7ff5680598f0 con 0x7ff57c071f50 2024-04-12T20:55:36.413 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:55:36.416+0000 7ff584400700 1 -- 172.21.15.18:0/1424068310 --> [v2:172.21.15.186:6828/2316349902,v1:172.21.15.186:6829/2316349902] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7ff57c115f40 con 0x7ff56406bcf0 2024-04-12T20:55:36.416 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:55:36.419+0000 7ff579600700 1 -- 172.21.15.18:0/1424068310 <== mgr.24359 v2:172.21.15.186:6828/2316349902 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+310 (secure 0 0 0) 0x7ff57c115f40 con 0x7ff56406bcf0 2024-04-12T20:55:36.420 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:55:36.422+0000 7ff584400700 1 -- 172.21.15.18:0/1424068310 >> [v2:172.21.15.186:6828/2316349902,v1:172.21.15.186:6829/2316349902] conn(0x7ff56406bcf0 msgr2=0x7ff56406e1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:55:36.420 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:55:36.422+0000 7ff584400700 1 --2- 172.21.15.18:0/1424068310 >> [v2:172.21.15.186:6828/2316349902,v1:172.21.15.186:6829/2316349902] conn(0x7ff56406bcf0 0x7ff56406e1a0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7ff57000e560 tx=0x7ff57000b040 comp rx=0 tx=0).stop 2024-04-12T20:55:36.420 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:55:36.422+0000 7ff584400700 1 -- 172.21.15.18:0/1424068310 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff57c071f50 msgr2=0x7ff57c114540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:55:36.420 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:55:36.422+0000 7ff584400700 1 --2- 172.21.15.18:0/1424068310 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff57c071f50 0x7ff57c114540 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7ff56800b970 tx=0x7ff568009e30 comp rx=0 tx=0).stop 2024-04-12T20:55:36.420 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:55:36.423+0000 7ff584400700 1 -- 172.21.15.18:0/1424068310 shutdown_connections 2024-04-12T20:55:36.420 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:55:36.423+0000 7ff584400700 1 --2- 172.21.15.18:0/1424068310 >> [v2:172.21.15.186:6828/2316349902,v1:172.21.15.186:6829/2316349902] conn(0x7ff56406bcf0 0x7ff56406e1a0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:55:36.421 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:55:36.423+0000 7ff584400700 1 --2- 172.21.15.18:0/1424068310 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff57c0709a0 0x7ff57c114000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:55:36.421 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:55:36.423+0000 7ff584400700 1 --2- 172.21.15.18:0/1424068310 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff57c071f50 0x7ff57c114540 secure :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7ff56800b970 tx=0x7ff568009e30 comp rx=0 tx=0).stop 2024-04-12T20:55:36.421 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:55:36.423+0000 7ff584400700 1 -- 172.21.15.18:0/1424068310 >> 172.21.15.18:0/1424068310 conn(0x7ff57c0f8560 msgr2=0x7ff57c0f9d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T20:55:36.421 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:55:36.423+0000 7ff584400700 1 -- 172.21.15.18:0/1424068310 shutdown_connections 2024-04-12T20:55:36.421 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:55:36.424+0000 7ff584400700 1 -- 172.21.15.18:0/1424068310 wait complete. 2024-04-12T20:55:36.631 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:55:36.634+0000 7f0639800700 1 -- 172.21.15.18:0/156588965 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f06340fff00 msgr2=0x7f0634102320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:55:36.631 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:55:36.634+0000 7f0639800700 1 --2- 172.21.15.18:0/156588965 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f06340fff00 0x7f0634102320 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f0620004940 tx=0x7f0620009e30 comp rx=0 tx=0).stop 2024-04-12T20:55:36.632 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:55:36.634+0000 7f0639800700 1 -- 172.21.15.18:0/156588965 shutdown_connections 2024-04-12T20:55:36.632 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:55:36.634+0000 7f0639800700 1 --2- 172.21.15.18:0/156588965 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f06340fff00 0x7f0634102320 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:55:36.632 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:55:36.634+0000 7f0639800700 1 --2- 172.21.15.18:0/156588965 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f06340662e0 0x7f06340ff9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:55:36.632 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:55:36.634+0000 7f0639800700 1 -- 172.21.15.18:0/156588965 >> 172.21.15.18:0/156588965 conn(0x7f06340f7430 msgr2=0x7f06340f9860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T20:55:36.632 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:55:36.635+0000 7f0639800700 1 -- 172.21.15.18:0/156588965 shutdown_connections 2024-04-12T20:55:36.633 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:55:36.635+0000 7f0639800700 1 -- 172.21.15.18:0/156588965 wait complete. 2024-04-12T20:55:36.633 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:55:36.636+0000 7f0639800700 1 Processor -- start 2024-04-12T20:55:36.638 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:55:36.637+0000 7f0639800700 1 -- start start 2024-04-12T20:55:36.639 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:55:36.637+0000 7f0639800700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f06340662e0 0x7f06340fd3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T20:55:36.639 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:55:36.637+0000 7f0639800700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f06340fff00 0x7f06340fb8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T20:55:36.639 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:55:36.637+0000 7f0639800700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06341116f0 con 0x7f06340fff00 2024-04-12T20:55:36.639 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:55:36.637+0000 7f0639800700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0634111860 con 0x7f06340662e0 2024-04-12T20:55:36.639 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:55:36.638+0000 7f0632000700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f06340fff00 0x7f06340fb8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T20:55:36.639 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:55:36.638+0000 7f0632000700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f06340fff00 0x7f06340fb8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:53308/0 (socket says 172.21.15.18:53308) 2024-04-12T20:55:36.639 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:55:36.638+0000 7f0632000700 1 -- 172.21.15.18:0/3377959936 learned_addr learned my addr 172.21.15.18:0/3377959936 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T20:55:36.639 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:55:36.638+0000 7f0632a00700 1 --2- 172.21.15.18:0/3377959936 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f06340662e0 0x7f06340fd3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T20:55:36.639 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:55:36.639+0000 7f0632000700 1 -- 172.21.15.18:0/3377959936 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f06340662e0 msgr2=0x7f06340fd3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:55:36.639 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:55:36.639+0000 7f0632000700 1 --2- 172.21.15.18:0/3377959936 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f06340662e0 0x7f06340fd3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:55:36.639 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:55:36.639+0000 7f0632000700 1 -- 172.21.15.18:0/3377959936 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0620005040 con 0x7f06340fff00 2024-04-12T20:55:36.639 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:55:36.640+0000 7f0632000700 1 --2- 172.21.15.18:0/3377959936 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f06340fff00 0x7f06340fb8c0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f062000b3b0 tx=0x7f0620009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T20:55:36.639 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:55:36.640+0000 7f062aa00700 1 -- 172.21.15.18:0/3377959936 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f062000e040 con 0x7f06340fff00 2024-04-12T20:55:36.639 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:55:36.641+0000 7f062aa00700 1 -- 172.21.15.18:0/3377959936 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f0620004290 con 0x7f06340fff00 2024-04-12T20:55:36.640 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:55:36.641+0000 7f0639800700 1 -- 172.21.15.18:0/3377959936 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f06340fbec0 con 0x7f06340fff00 2024-04-12T20:55:36.640 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:55:36.641+0000 7f062aa00700 1 -- 172.21.15.18:0/3377959936 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f062001c630 con 0x7f06340fff00 2024-04-12T20:55:36.640 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:55:36.641+0000 7f0639800700 1 -- 172.21.15.18:0/3377959936 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f06340fc430 con 0x7f06340fff00 2024-04-12T20:55:36.644 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:55:36.644+0000 7f0639800700 1 -- 172.21.15.18:0/3377959936 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f063405f5f0 con 0x7f06340fff00 2024-04-12T20:55:36.644 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:55:36.647+0000 7f062aa00700 1 -- 172.21.15.18:0/3377959936 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 24) v1 ==== 89644+0+0 (secure 0 0 0) 0x7f062000bbf0 con 0x7f06340fff00 2024-04-12T20:55:36.647 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:55:36.649+0000 7f062aa00700 1 --2- 172.21.15.18:0/3377959936 >> [v2:172.21.15.186:6828/2316349902,v1:172.21.15.186:6829/2316349902] conn(0x7f061806bcf0 0x7f061806e1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T20:55:36.647 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:55:36.650+0000 7f062aa00700 1 -- 172.21.15.18:0/3377959936 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5808+0+0 (secure 0 0 0) 0x7f062008bef0 con 0x7f06340fff00 2024-04-12T20:55:36.648 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:55:36.650+0000 7f0632a00700 1 --2- 172.21.15.18:0/3377959936 >> [v2:172.21.15.186:6828/2316349902,v1:172.21.15.186:6829/2316349902] conn(0x7f061806bcf0 0x7f061806e1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T20:55:36.650 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:55:36.653+0000 7f0632a00700 1 --2- 172.21.15.18:0/3377959936 >> [v2:172.21.15.186:6828/2316349902,v1:172.21.15.186:6829/2316349902] conn(0x7f061806bcf0 0x7f061806e1a0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f06340fd180 tx=0x7f0624009ee0 comp rx=0 tx=0).ready entity=mgr.24359 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T20:55:36.651 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:55:36.654+0000 7f062aa00700 1 -- 172.21.15.18:0/3377959936 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186454 (secure 0 0 0) 0x7f06200554a0 con 0x7f06340fff00 2024-04-12T20:55:36.852 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:55:36 smithi018 ceph-mon[28799]: pgmap v21: 65 pgs: 65 active+clean; 150 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 2.9 MiB/s wr, 298 op/s 2024-04-12T20:55:36.852 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:55:36 smithi018 ceph-mon[28799]: from='client.14624 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T20:55:37.003 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:55:37.005+0000 7f0639800700 1 -- 172.21.15.18:0/3377959936 --> [v2:172.21.15.186:6828/2316349902,v1:172.21.15.186:6829/2316349902] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f0634105a80 con 0x7f061806bcf0 2024-04-12T20:55:37.030 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:55:36 smithi186 ceph-mon[36597]: pgmap v21: 65 pgs: 65 active+clean; 150 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 2.9 MiB/s wr, 298 op/s 2024-04-12T20:55:37.031 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:55:36 smithi186 ceph-mon[36597]: from='client.14624 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T20:55:37.038 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:55:37.040+0000 7f062aa00700 1 -- 172.21.15.18:0/3377959936 <== mgr.24359 v2:172.21.15.186:6828/2316349902 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3388 (secure 0 0 0) 0x7f0624004e30 con 0x7f061806bcf0 2024-04-12T20:55:37.038 INFO:teuthology.orchestra.run.smithi018.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-12T20:55:37.039 INFO:teuthology.orchestra.run.smithi018.stdout:alertmanager.smithi018 smithi018 *:9093,9094 running (8m) 18s ago 10m 13.5M - 0.25.0 c8568f914cd2 f43ec346f9e5 2024-04-12T20:55:37.039 INFO:teuthology.orchestra.run.smithi018.stdout:crash.smithi018 smithi018 running (11m) 18s ago 10m 6660k - 17.2.7-827-gd3b5060a 21de9b7369ad 1ac7603bddfd 2024-04-12T20:55:37.039 INFO:teuthology.orchestra.run.smithi018.stdout:crash.smithi186 smithi186 running (9m) 25s ago 9m 6669k - 17.2.7-827-gd3b5060a 21de9b7369ad 14bf8a1672eb 2024-04-12T20:55:37.039 INFO:teuthology.orchestra.run.smithi018.stdout:grafana.smithi018 smithi018 *:3000 running (8m) 18s ago 10m 74.1M - 9.4.7 954c08fa6188 a7c3fe73c4d7 2024-04-12T20:55:37.039 INFO:teuthology.orchestra.run.smithi018.stdout:mds.cephfs.smithi018.acqbsc smithi018 running (3m) 18s ago 3m 69.0M - 17.2.7-827-gd3b5060a 21de9b7369ad 5b5fce65f18a 2024-04-12T20:55:37.039 INFO:teuthology.orchestra.run.smithi018.stdout:mds.cephfs.smithi018.szjluo smithi018 running (3m) 18s ago 3m 65.4M - 17.2.7-827-gd3b5060a 21de9b7369ad cf9bf0c5031f 2024-04-12T20:55:37.039 INFO:teuthology.orchestra.run.smithi018.stdout:mds.cephfs.smithi186.jdllyn smithi186 running (3m) 25s ago 3m 11.8M - 17.2.7-827-gd3b5060a 21de9b7369ad 2b2bc46bd29a 2024-04-12T20:55:37.039 INFO:teuthology.orchestra.run.smithi018.stdout:mds.cephfs.smithi186.wjnafs smithi186 running (3m) 25s ago 3m 11.9M - 17.2.7-827-gd3b5060a 21de9b7369ad 74b115d6d6fd 2024-04-12T20:55:37.039 INFO:teuthology.orchestra.run.smithi018.stdout:mgr.smithi018.xjzwga smithi018 *:9283 running (12m) 18s ago 12m 331M - 17.2.7-827-gd3b5060a 21de9b7369ad c72f43857311 2024-04-12T20:55:37.039 INFO:teuthology.orchestra.run.smithi018.stdout:mgr.smithi186.kqcfxk smithi186 *:8443,9283 running (57s) 25s ago 9m 492M - 19.0.0-2879-g324baf8d 8717720dc5e5 bd4971d5260e 2024-04-12T20:55:37.040 INFO:teuthology.orchestra.run.smithi018.stdout:mon.smithi018 smithi018 running (12m) 18s ago 12m 47.1M 2048M 17.2.7-827-gd3b5060a 21de9b7369ad cd84caafa056 2024-04-12T20:55:37.040 INFO:teuthology.orchestra.run.smithi018.stdout:mon.smithi186 smithi186 running (9m) 25s ago 9m 37.0M 2048M 17.2.7-827-gd3b5060a 21de9b7369ad d7263c772b73 2024-04-12T20:55:37.040 INFO:teuthology.orchestra.run.smithi018.stdout:node-exporter.smithi018 smithi018 *:9100 starting - - - - 2024-04-12T20:55:37.040 INFO:teuthology.orchestra.run.smithi018.stdout:node-exporter.smithi186 smithi186 *:9100 running (9m) 25s ago 9m 8334k - 1.5.0 0da6a335fe13 57160d1060f2 2024-04-12T20:55:37.040 INFO:teuthology.orchestra.run.smithi018.stdout:osd.0 smithi018 running (8m) 18s ago 8m 83.2M 1559M 17.2.7-827-gd3b5060a 21de9b7369ad 0d83e027bf0a 2024-04-12T20:55:37.040 INFO:teuthology.orchestra.run.smithi018.stdout:osd.1 smithi018 running (7m) 18s ago 7m 90.2M 1559M 17.2.7-827-gd3b5060a 21de9b7369ad 6517c9cc7f87 2024-04-12T20:55:37.040 INFO:teuthology.orchestra.run.smithi018.stdout:osd.2 smithi018 running (7m) 18s ago 7m 75.7M 1559M 17.2.7-827-gd3b5060a 21de9b7369ad 8a83692acaa5 2024-04-12T20:55:37.040 INFO:teuthology.orchestra.run.smithi018.stdout:osd.3 smithi186 running (6m) 25s ago 6m 88.7M 2581M 17.2.7-827-gd3b5060a 21de9b7369ad c2def7a90d76 2024-04-12T20:55:37.041 INFO:teuthology.orchestra.run.smithi018.stdout:osd.4 smithi186 running (6m) 25s ago 6m 84.8M 2581M 17.2.7-827-gd3b5060a 21de9b7369ad 757e7c71b7b9 2024-04-12T20:55:37.041 INFO:teuthology.orchestra.run.smithi018.stdout:osd.5 smithi186 running (5m) 25s ago 5m 80.8M 2581M 17.2.7-827-gd3b5060a 21de9b7369ad 2ee072a436d0 2024-04-12T20:55:37.041 INFO:teuthology.orchestra.run.smithi018.stdout:prometheus.smithi018 smithi018 *:9095 running (8m) 18s ago 10m 51.1M - 2.43.0 a07b618ecd1d aecd046db448 2024-04-12T20:55:37.043 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:55:37.045+0000 7f0639800700 1 -- 172.21.15.18:0/3377959936 >> [v2:172.21.15.186:6828/2316349902,v1:172.21.15.186:6829/2316349902] conn(0x7f061806bcf0 msgr2=0x7f061806e1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:55:37.044 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:55:37.045+0000 7f0639800700 1 --2- 172.21.15.18:0/3377959936 >> [v2:172.21.15.186:6828/2316349902,v1:172.21.15.186:6829/2316349902] conn(0x7f061806bcf0 0x7f061806e1a0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f06340fd180 tx=0x7f0624009ee0 comp rx=0 tx=0).stop 2024-04-12T20:55:37.044 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:55:37.045+0000 7f0639800700 1 -- 172.21.15.18:0/3377959936 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f06340fff00 msgr2=0x7f06340fb8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:55:37.044 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:55:37.045+0000 7f0639800700 1 --2- 172.21.15.18:0/3377959936 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f06340fff00 0x7f06340fb8c0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f062000b3b0 tx=0x7f0620009e30 comp rx=0 tx=0).stop 2024-04-12T20:55:37.044 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:55:37.045+0000 7f0639800700 1 -- 172.21.15.18:0/3377959936 shutdown_connections 2024-04-12T20:55:37.044 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:55:37.045+0000 7f0639800700 1 --2- 172.21.15.18:0/3377959936 >> [v2:172.21.15.186:6828/2316349902,v1:172.21.15.186:6829/2316349902] conn(0x7f061806bcf0 0x7f061806e1a0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:55:37.044 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:55:37.045+0000 7f0639800700 1 --2- 172.21.15.18:0/3377959936 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f06340662e0 0x7f06340fd3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:55:37.045 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:55:37.045+0000 7f0639800700 1 --2- 172.21.15.18:0/3377959936 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f06340fff00 0x7f06340fb8c0 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:55:37.045 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:55:37.045+0000 7f0639800700 1 -- 172.21.15.18:0/3377959936 >> 172.21.15.18:0/3377959936 conn(0x7f06340f7430 msgr2=0x7f06340f7850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T20:55:37.045 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:55:37.045+0000 7f0639800700 1 -- 172.21.15.18:0/3377959936 shutdown_connections 2024-04-12T20:55:37.045 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:55:37.045+0000 7f0639800700 1 -- 172.21.15.18:0/3377959936 wait complete. 2024-04-12T20:55:37.240 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:55:37.243+0000 7f2834a00700 1 -- 172.21.15.18:0/1547223315 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f282800bd10 con 0x7f28380fcee0 2024-04-12T20:55:37.241 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:55:37.244+0000 7f283d200700 1 -- 172.21.15.18:0/1547223315 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f28380fcee0 msgr2=0x7f28380fd310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:55:37.241 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:55:37.244+0000 7f283d200700 1 --2- 172.21.15.18:0/1547223315 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f28380fcee0 0x7f28380fd310 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f2828004940 tx=0x7f2828009e30 comp rx=0 tx=0).stop 2024-04-12T20:55:37.241 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:55:37.244+0000 7f283d200700 1 -- 172.21.15.18:0/1547223315 shutdown_connections 2024-04-12T20:55:37.241 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:55:37.244+0000 7f283d200700 1 --2- 172.21.15.18:0/1547223315 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f28380fe0d0 0x7f28380fe540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:55:37.241 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:55:37.244+0000 7f283d200700 1 --2- 172.21.15.18:0/1547223315 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f28380fcee0 0x7f28380fd310 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:55:37.242 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:55:37.244+0000 7f283d200700 1 -- 172.21.15.18:0/1547223315 >> 172.21.15.18:0/1547223315 conn(0x7f28380f8560 msgr2=0x7f28380fa970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T20:55:37.244 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:55:37.244+0000 7f283d200700 1 -- 172.21.15.18:0/1547223315 shutdown_connections 2024-04-12T20:55:37.244 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:55:37.244+0000 7f283d200700 1 -- 172.21.15.18:0/1547223315 wait complete. 2024-04-12T20:55:37.244 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:55:37.245+0000 7f283d200700 1 Processor -- start 2024-04-12T20:55:37.244 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:55:37.246+0000 7f283d200700 1 -- start start 2024-04-12T20:55:37.244 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:55:37.247+0000 7f283d200700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f28380fcee0 0x7f283810b800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T20:55:37.244 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:55:37.247+0000 7f283d200700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f28380fe0d0 0x7f283810bd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T20:55:37.244 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:55:37.247+0000 7f283d200700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f283810c390 con 0x7f28380fcee0 2024-04-12T20:55:37.244 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:55:37.247+0000 7f283d200700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f283810cbe0 con 0x7f28380fe0d0 2024-04-12T20:55:37.245 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:55:37.247+0000 7f2835e00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f28380fcee0 0x7f283810b800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T20:55:37.245 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:55:37.247+0000 7f2835e00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f28380fcee0 0x7f283810b800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:53326/0 (socket says 172.21.15.18:53326) 2024-04-12T20:55:37.245 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:55:37.247+0000 7f2835e00700 1 -- 172.21.15.18:0/3565286243 learned_addr learned my addr 172.21.15.18:0/3565286243 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T20:55:37.245 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:55:37.248+0000 7f2835400700 1 --2- 172.21.15.18:0/3565286243 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f28380fe0d0 0x7f283810bd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T20:55:37.245 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:55:37.248+0000 7f2835e00700 1 -- 172.21.15.18:0/3565286243 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f28380fe0d0 msgr2=0x7f283810bd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:55:37.245 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:55:37.248+0000 7f2835e00700 1 --2- 172.21.15.18:0/3565286243 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f28380fe0d0 0x7f283810bd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:55:37.246 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:55:37.248+0000 7f2835e00700 1 -- 172.21.15.18:0/3565286243 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2828005040 con 0x7f28380fcee0 2024-04-12T20:55:37.246 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:55:37.248+0000 7f2835e00700 1 --2- 172.21.15.18:0/3565286243 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f28380fcee0 0x7f283810b800 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f2828000c00 tx=0x7f2828009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T20:55:37.246 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:55:37.249+0000 7f282e000700 1 -- 172.21.15.18:0/3565286243 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f282800e030 con 0x7f28380fcee0 2024-04-12T20:55:37.246 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:55:37.249+0000 7f282e000700 1 -- 172.21.15.18:0/3565286243 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f2828013420 con 0x7f28380fcee0 2024-04-12T20:55:37.247 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:55:37.249+0000 7f282e000700 1 -- 172.21.15.18:0/3565286243 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f282801c600 con 0x7f28380fcee0 2024-04-12T20:55:37.247 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:55:37.249+0000 7f283d200700 1 -- 172.21.15.18:0/3565286243 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f283810ce60 con 0x7f28380fcee0 2024-04-12T20:55:37.247 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:55:37.250+0000 7f283d200700 1 -- 172.21.15.18:0/3565286243 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f283810d420 con 0x7f28380fcee0 2024-04-12T20:55:37.250 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:55:37.253+0000 7f283d200700 1 -- 172.21.15.18:0/3565286243 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f283805f5f0 con 0x7f28380fcee0 2024-04-12T20:55:37.252 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:55:37.254+0000 7f282e000700 1 -- 172.21.15.18:0/3565286243 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 24) v1 ==== 89644+0+0 (secure 0 0 0) 0x7f2828013a70 con 0x7f28380fcee0 2024-04-12T20:55:37.252 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:55:37.255+0000 7f282e000700 1 --2- 172.21.15.18:0/3565286243 >> [v2:172.21.15.186:6828/2316349902,v1:172.21.15.186:6829/2316349902] conn(0x7f281c06bcf0 0x7f281c06e1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T20:55:37.252 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:55:37.255+0000 7f282e000700 1 -- 172.21.15.18:0/3565286243 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5808+0+0 (secure 0 0 0) 0x7f282808bc50 con 0x7f28380fcee0 2024-04-12T20:55:37.253 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:55:37.256+0000 7f2835400700 1 --2- 172.21.15.18:0/3565286243 >> [v2:172.21.15.186:6828/2316349902,v1:172.21.15.186:6829/2316349902] conn(0x7f281c06bcf0 0x7f281c06e1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T20:55:37.257 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:55:37.260+0000 7f2835400700 1 --2- 172.21.15.18:0/3565286243 >> [v2:172.21.15.186:6828/2316349902,v1:172.21.15.186:6829/2316349902] conn(0x7f281c06bcf0 0x7f281c06e1a0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f2824007fd0 tx=0x7f2824009040 comp rx=0 tx=0).ready entity=mgr.24359 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T20:55:37.259 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:55:37.262+0000 7f282e000700 1 -- 172.21.15.18:0/3565286243 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186454 (secure 0 0 0) 0x7f28280552d0 con 0x7f28380fcee0 2024-04-12T20:55:37.747 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:55:37.750+0000 7f283d200700 1 -- 172.21.15.18:0/3565286243 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f283810d750 con 0x7f28380fcee0 2024-04-12T20:55:37.748 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:55:37.751+0000 7f282e000700 1 -- 172.21.15.18:0/3565286243 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+846 (secure 0 0 0) 0x7f2828058990 con 0x7f28380fcee0 2024-04-12T20:55:37.749 INFO:teuthology.orchestra.run.smithi018.stdout:{ 2024-04-12T20:55:37.749 INFO:teuthology.orchestra.run.smithi018.stdout: "mon": { 2024-04-12T20:55:37.749 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 17.2.7-827-gd3b5060a (d3b5060a17449edfae9385cdf792afec109fca5d) quincy (stable)": 2 2024-04-12T20:55:37.749 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-12T20:55:37.749 INFO:teuthology.orchestra.run.smithi018.stdout: "mgr": { 2024-04-12T20:55:37.749 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 17.2.7-827-gd3b5060a (d3b5060a17449edfae9385cdf792afec109fca5d) quincy (stable)": 1, 2024-04-12T20:55:37.749 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 19.0.0-2879-g324baf8d (324baf8ddb91dea46cac50cb535c306115791355) squid (dev)": 1 2024-04-12T20:55:37.749 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-12T20:55:37.749 INFO:teuthology.orchestra.run.smithi018.stdout: "osd": { 2024-04-12T20:55:37.749 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 17.2.7-827-gd3b5060a (d3b5060a17449edfae9385cdf792afec109fca5d) quincy (stable)": 6 2024-04-12T20:55:37.750 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-12T20:55:37.750 INFO:teuthology.orchestra.run.smithi018.stdout: "mds": { 2024-04-12T20:55:37.750 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 17.2.7-827-gd3b5060a (d3b5060a17449edfae9385cdf792afec109fca5d) quincy (stable)": 4 2024-04-12T20:55:37.750 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-12T20:55:37.750 INFO:teuthology.orchestra.run.smithi018.stdout: "overall": { 2024-04-12T20:55:37.750 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 17.2.7-827-gd3b5060a (d3b5060a17449edfae9385cdf792afec109fca5d) quincy (stable)": 13, 2024-04-12T20:55:37.750 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 19.0.0-2879-g324baf8d (324baf8ddb91dea46cac50cb535c306115791355) squid (dev)": 1 2024-04-12T20:55:37.750 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-04-12T20:55:37.750 INFO:teuthology.orchestra.run.smithi018.stdout:} 2024-04-12T20:55:37.751 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:55:37.754+0000 7f283d200700 1 -- 172.21.15.18:0/3565286243 >> [v2:172.21.15.186:6828/2316349902,v1:172.21.15.186:6829/2316349902] conn(0x7f281c06bcf0 msgr2=0x7f281c06e1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:55:37.751 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:55:37.754+0000 7f283d200700 1 --2- 172.21.15.18:0/3565286243 >> [v2:172.21.15.186:6828/2316349902,v1:172.21.15.186:6829/2316349902] conn(0x7f281c06bcf0 0x7f281c06e1a0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f2824007fd0 tx=0x7f2824009040 comp rx=0 tx=0).stop 2024-04-12T20:55:37.751 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:55:37.754+0000 7f283d200700 1 -- 172.21.15.18:0/3565286243 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f28380fcee0 msgr2=0x7f283810b800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:55:37.752 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:55:37.754+0000 7f283d200700 1 --2- 172.21.15.18:0/3565286243 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f28380fcee0 0x7f283810b800 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f2828000c00 tx=0x7f2828009e30 comp rx=0 tx=0).stop 2024-04-12T20:55:37.752 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:55:37.754+0000 7f283d200700 1 -- 172.21.15.18:0/3565286243 shutdown_connections 2024-04-12T20:55:37.752 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:55:37.754+0000 7f283d200700 1 --2- 172.21.15.18:0/3565286243 >> [v2:172.21.15.186:6828/2316349902,v1:172.21.15.186:6829/2316349902] conn(0x7f281c06bcf0 0x7f281c06e1a0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:55:37.752 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:55:37.754+0000 7f283d200700 1 --2- 172.21.15.18:0/3565286243 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f28380fcee0 0x7f283810b800 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:55:37.752 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:55:37.754+0000 7f283d200700 1 --2- 172.21.15.18:0/3565286243 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f28380fe0d0 0x7f283810bd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:55:37.752 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:55:37.754+0000 7f283d200700 1 -- 172.21.15.18:0/3565286243 >> 172.21.15.18:0/3565286243 conn(0x7f28380f8560 msgr2=0x7f28380f9d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T20:55:37.752 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:55:37.755+0000 7f283d200700 1 -- 172.21.15.18:0/3565286243 shutdown_connections 2024-04-12T20:55:37.752 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:55:37.755+0000 7f283d200700 1 -- 172.21.15.18:0/3565286243 wait complete. 2024-04-12T20:55:37.851 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:55:37 smithi018 ceph-mon[28799]: from='client.24409 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T20:55:37.851 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:55:37 smithi018 ceph-mon[28799]: from='client.14632 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T20:55:37.939 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:55:37.941+0000 7f86c5800700 1 -- 172.21.15.18:0/3222892580 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f86c00fff00 msgr2=0x7f86c0102320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:55:37.939 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:55:37.941+0000 7f86c5800700 1 --2- 172.21.15.18:0/3222892580 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f86c00fff00 0x7f86c0102320 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f86ac004940 tx=0x7f86ac009e30 comp rx=0 tx=0).stop 2024-04-12T20:55:37.939 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:55:37.941+0000 7f86c5800700 1 -- 172.21.15.18:0/3222892580 shutdown_connections 2024-04-12T20:55:37.940 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:55:37.941+0000 7f86c5800700 1 --2- 172.21.15.18:0/3222892580 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f86c00fff00 0x7f86c0102320 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:55:37.940 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:55:37.941+0000 7f86c5800700 1 --2- 172.21.15.18:0/3222892580 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f86c00662e0 0x7f86c00ff9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:55:37.940 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:55:37.941+0000 7f86c5800700 1 -- 172.21.15.18:0/3222892580 >> 172.21.15.18:0/3222892580 conn(0x7f86c00f7430 msgr2=0x7f86c00f9860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T20:55:37.940 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:55:37.941+0000 7f86c5800700 1 -- 172.21.15.18:0/3222892580 shutdown_connections 2024-04-12T20:55:37.941 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:55:37.942+0000 7f86c5800700 1 -- 172.21.15.18:0/3222892580 wait complete. 2024-04-12T20:55:37.944 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:55:37.942+0000 7f86c5800700 1 Processor -- start 2024-04-12T20:55:37.944 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:55:37.943+0000 7f86c5800700 1 -- start start 2024-04-12T20:55:37.944 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:55:37.943+0000 7f86c5800700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f86c00662e0 0x7f86c010fd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T20:55:37.945 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:55:37.943+0000 7f86c5800700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f86c00fff00 0x7f86c0110250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T20:55:37.945 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:55:37.943+0000 7f86c5800700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86c0110850 con 0x7f86c00fff00 2024-04-12T20:55:37.945 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:55:37.943+0000 7f86c5800700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86c0110f90 con 0x7f86c00662e0 2024-04-12T20:55:37.945 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:55:37.944+0000 7f86be000700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f86c00fff00 0x7f86c0110250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T20:55:37.945 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:55:37.944+0000 7f86be000700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f86c00fff00 0x7f86c0110250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:53342/0 (socket says 172.21.15.18:53342) 2024-04-12T20:55:37.945 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:55:37.944+0000 7f86be000700 1 -- 172.21.15.18:0/3816615256 learned_addr learned my addr 172.21.15.18:0/3816615256 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T20:55:37.945 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:55:37.944+0000 7f86bea00700 1 --2- 172.21.15.18:0/3816615256 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f86c00662e0 0x7f86c010fd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T20:55:37.945 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:55:37.944+0000 7f86be000700 1 -- 172.21.15.18:0/3816615256 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f86c00662e0 msgr2=0x7f86c010fd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:55:37.945 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:55:37.944+0000 7f86be000700 1 --2- 172.21.15.18:0/3816615256 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f86c00662e0 0x7f86c010fd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:55:37.946 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:55:37.944+0000 7f86be000700 1 -- 172.21.15.18:0/3816615256 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86ac005040 con 0x7f86c00fff00 2024-04-12T20:55:37.946 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:55:37.944+0000 7f86bea00700 1 --2- 172.21.15.18:0/3816615256 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f86c00662e0 0x7f86c010fd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-12T20:55:37.946 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:55:37.945+0000 7f86be000700 1 --2- 172.21.15.18:0/3816615256 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f86c00fff00 0x7f86c0110250 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f86ac00b3b0 tx=0x7f86ac009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T20:55:37.946 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:55:37.945+0000 7f86b6a00700 1 -- 172.21.15.18:0/3816615256 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f86ac00e040 con 0x7f86c00fff00 2024-04-12T20:55:37.946 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:55:37.945+0000 7f86b6a00700 1 -- 172.21.15.18:0/3816615256 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f86ac004290 con 0x7f86c00fff00 2024-04-12T20:55:37.946 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:55:37.945+0000 7f86b6a00700 1 -- 172.21.15.18:0/3816615256 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f86ac01c630 con 0x7f86c00fff00 2024-04-12T20:55:37.946 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:55:37.946+0000 7f86c5800700 1 -- 172.21.15.18:0/3816615256 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f86c0111210 con 0x7f86c00fff00 2024-04-12T20:55:37.946 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:55:37.946+0000 7f86c5800700 1 -- 172.21.15.18:0/3816615256 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f86c01117d0 con 0x7f86c00fff00 2024-04-12T20:55:37.947 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:55:37.948+0000 7f86c5800700 1 -- 172.21.15.18:0/3816615256 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f86c00f8fb0 con 0x7f86c00fff00 2024-04-12T20:55:37.948 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:55:37.949+0000 7f86b6a00700 1 -- 172.21.15.18:0/3816615256 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 24) v1 ==== 89644+0+0 (secure 0 0 0) 0x7f86ac00bbf0 con 0x7f86c00fff00 2024-04-12T20:55:37.948 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:55:37.950+0000 7f86b6a00700 1 --2- 172.21.15.18:0/3816615256 >> [v2:172.21.15.186:6828/2316349902,v1:172.21.15.186:6829/2316349902] conn(0x7f86a406bcf0 0x7f86a406e1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T20:55:37.948 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:55:37.951+0000 7f86b6a00700 1 -- 172.21.15.18:0/3816615256 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5808+0+0 (secure 0 0 0) 0x7f86ac08bc40 con 0x7f86c00fff00 2024-04-12T20:55:37.949 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:55:37.951+0000 7f86bea00700 1 --2- 172.21.15.18:0/3816615256 >> [v2:172.21.15.186:6828/2316349902,v1:172.21.15.186:6829/2316349902] conn(0x7f86a406bcf0 0x7f86a406e1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T20:55:37.950 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:55:37.953+0000 7f86bea00700 1 --2- 172.21.15.18:0/3816615256 >> [v2:172.21.15.186:6828/2316349902,v1:172.21.15.186:6829/2316349902] conn(0x7f86a406bcf0 0x7f86a406e1a0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f86b8004550 tx=0x7f86b8009040 comp rx=0 tx=0).ready entity=mgr.24359 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T20:55:37.953 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:55:37.956+0000 7f86b6a00700 1 -- 172.21.15.18:0/3816615256 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186454 (secure 0 0 0) 0x7f86ac058d30 con 0x7f86c00fff00 2024-04-12T20:55:38.030 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:55:37 smithi186 ceph-mon[36597]: from='client.24409 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T20:55:38.030 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:55:37 smithi186 ceph-mon[36597]: from='client.14632 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T20:55:38.328 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:55:38.330+0000 7f86c5800700 1 -- 172.21.15.18:0/3816615256 --> [v2:172.21.15.186:6828/2316349902,v1:172.21.15.186:6829/2316349902] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f86c0111a10 con 0x7f86a406bcf0 2024-04-12T20:55:38.330 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:55:38.333+0000 7f86b6a00700 1 -- 172.21.15.18:0/3816615256 <== mgr.24359 v2:172.21.15.186:6828/2316349902 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+310 (secure 0 0 0) 0x7f86c0111a10 con 0x7f86a406bcf0 2024-04-12T20:55:38.332 INFO:teuthology.orchestra.run.smithi018.stdout:{ 2024-04-12T20:55:38.332 INFO:teuthology.orchestra.run.smithi018.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:f4452def3be4d350f4a8e5feaf7ebb1511be9298fe68478ba7a2d6fbd7d69a63", 2024-04-12T20:55:38.332 INFO:teuthology.orchestra.run.smithi018.stdout: "in_progress": true, 2024-04-12T20:55:38.332 INFO:teuthology.orchestra.run.smithi018.stdout: "which": "Upgrading daemons of type(s) mgr", 2024-04-12T20:55:38.332 INFO:teuthology.orchestra.run.smithi018.stdout: "services_complete": [], 2024-04-12T20:55:38.332 INFO:teuthology.orchestra.run.smithi018.stdout: "progress": "1/2 daemons upgraded", 2024-04-12T20:55:38.332 INFO:teuthology.orchestra.run.smithi018.stdout: "message": "", 2024-04-12T20:55:38.332 INFO:teuthology.orchestra.run.smithi018.stdout: "is_paused": false 2024-04-12T20:55:38.332 INFO:teuthology.orchestra.run.smithi018.stdout:} 2024-04-12T20:55:38.333 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:55:38.336+0000 7f86c5800700 1 -- 172.21.15.18:0/3816615256 >> [v2:172.21.15.186:6828/2316349902,v1:172.21.15.186:6829/2316349902] conn(0x7f86a406bcf0 msgr2=0x7f86a406e1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:55:38.333 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:55:38.336+0000 7f86c5800700 1 --2- 172.21.15.18:0/3816615256 >> [v2:172.21.15.186:6828/2316349902,v1:172.21.15.186:6829/2316349902] conn(0x7f86a406bcf0 0x7f86a406e1a0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f86b8004550 tx=0x7f86b8009040 comp rx=0 tx=0).stop 2024-04-12T20:55:38.333 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:55:38.336+0000 7f86c5800700 1 -- 172.21.15.18:0/3816615256 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f86c00fff00 msgr2=0x7f86c0110250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:55:38.333 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:55:38.336+0000 7f86c5800700 1 --2- 172.21.15.18:0/3816615256 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f86c00fff00 0x7f86c0110250 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f86ac00b3b0 tx=0x7f86ac009e30 comp rx=0 tx=0).stop 2024-04-12T20:55:38.334 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:55:38.336+0000 7f86c5800700 1 -- 172.21.15.18:0/3816615256 shutdown_connections 2024-04-12T20:55:38.334 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:55:38.337+0000 7f86c5800700 1 --2- 172.21.15.18:0/3816615256 >> [v2:172.21.15.186:6828/2316349902,v1:172.21.15.186:6829/2316349902] conn(0x7f86a406bcf0 0x7f86a406e1a0 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:55:38.334 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:55:38.337+0000 7f86c5800700 1 --2- 172.21.15.18:0/3816615256 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f86c00662e0 0x7f86c010fd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:55:38.334 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:55:38.337+0000 7f86c5800700 1 --2- 172.21.15.18:0/3816615256 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f86c00fff00 0x7f86c0110250 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:55:38.334 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:55:38.337+0000 7f86c5800700 1 -- 172.21.15.18:0/3816615256 >> 172.21.15.18:0/3816615256 conn(0x7f86c00f7430 msgr2=0x7f86c00fbb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T20:55:38.334 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:55:38.337+0000 7f86c5800700 1 -- 172.21.15.18:0/3816615256 shutdown_connections 2024-04-12T20:55:38.334 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:55:38.337+0000 7f86c5800700 1 -- 172.21.15.18:0/3816615256 wait complete. 2024-04-12T20:55:38.851 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:55:38 smithi018 ceph-mon[28799]: pgmap v22: 65 pgs: 65 active+clean; 151 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 2.2 MiB/s wr, 254 op/s 2024-04-12T20:55:38.851 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:55:38 smithi018 ceph-mon[28799]: from='client.? 172.21.15.18:0/3565286243' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T20:55:38.916 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:55:38 smithi186 ceph-mon[36597]: pgmap v22: 65 pgs: 65 active+clean; 151 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 2.2 MiB/s wr, 254 op/s 2024-04-12T20:55:38.917 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:55:38 smithi186 ceph-mon[36597]: from='client.? 172.21.15.18:0/3565286243' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T20:55:39.742 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:55:39 smithi018 ceph-mon[28799]: from='client.14640 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T20:55:40.030 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:55:39 smithi186 ceph-mon[36597]: from='client.14640 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T20:55:41.030 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:55:40 smithi186 ceph-mon[36597]: pgmap v23: 65 pgs: 65 active+clean; 151 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 1.6 MiB/s wr, 189 op/s 2024-04-12T20:55:41.101 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:55:40 smithi018 ceph-mon[28799]: pgmap v23: 65 pgs: 65 active+clean; 151 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 1.6 MiB/s wr, 189 op/s 2024-04-12T20:55:43.030 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:55:42 smithi186 ceph-mon[36597]: pgmap v24: 65 pgs: 65 active+clean; 161 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 2.4 MiB/s wr, 290 op/s 2024-04-12T20:55:43.030 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:55:42 smithi186 ceph-mon[36597]: from='mgr.24359 172.21.15.186:0/1315906828' entity='mgr.smithi186.kqcfxk' 2024-04-12T20:55:43.030 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:55:42 smithi186 ceph-mon[36597]: from='mgr.24359 172.21.15.186:0/1315906828' entity='mgr.smithi186.kqcfxk' 2024-04-12T20:55:43.106 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:55:42 smithi018 ceph-mon[28799]: pgmap v24: 65 pgs: 65 active+clean; 161 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 2.4 MiB/s wr, 290 op/s 2024-04-12T20:55:43.106 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:55:42 smithi018 ceph-mon[28799]: from='mgr.24359 172.21.15.186:0/1315906828' entity='mgr.smithi186.kqcfxk' 2024-04-12T20:55:43.106 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:55:42 smithi018 ceph-mon[28799]: from='mgr.24359 172.21.15.186:0/1315906828' entity='mgr.smithi186.kqcfxk' 2024-04-12T20:55:43.930 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:55:43 smithi018 ceph-mon[28799]: Reconfiguring grafana.smithi018 (dependencies changed)... 2024-04-12T20:55:43.931 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:55:43 smithi018 ceph-mon[28799]: Reconfiguring daemon grafana.smithi018 on smithi018 2024-04-12T20:55:44.030 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:55:43 smithi186 ceph-mon[36597]: Reconfiguring grafana.smithi018 (dependencies changed)... 2024-04-12T20:55:44.030 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:55:43 smithi186 ceph-mon[36597]: Reconfiguring daemon grafana.smithi018 on smithi018 2024-04-12T20:55:45.030 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:55:44 smithi186 ceph-mon[36597]: pgmap v25: 65 pgs: 65 active+clean; 161 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 1.5 MiB/s wr, 180 op/s 2024-04-12T20:55:45.102 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:55:44 smithi018 ceph-mon[28799]: pgmap v25: 65 pgs: 65 active+clean; 161 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 1.5 MiB/s wr, 180 op/s 2024-04-12T20:55:45.780 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:55:45 smithi186 ceph-mon[36597]: from='mgr.24359 172.21.15.186:0/1315906828' entity='mgr.smithi186.kqcfxk' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T20:55:45.854 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:55:45 smithi018 ceph-mon[28799]: from='mgr.24359 172.21.15.186:0/1315906828' entity='mgr.smithi186.kqcfxk' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T20:55:47.030 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:55:46 smithi186 ceph-mon[36597]: pgmap v26: 65 pgs: 65 active+clean; 174 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 2.7 MiB/s wr, 253 op/s 2024-04-12T20:55:47.101 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:55:46 smithi018 ceph-mon[28799]: pgmap v26: 65 pgs: 65 active+clean; 174 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 2.7 MiB/s wr, 253 op/s 2024-04-12T20:55:48.852 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:55:48 smithi018 ceph-mon[28799]: pgmap v27: 65 pgs: 65 active+clean; 178 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 2.5 MiB/s wr, 219 op/s 2024-04-12T20:55:49.030 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:55:48 smithi186 ceph-mon[36597]: pgmap v27: 65 pgs: 65 active+clean; 178 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 2.5 MiB/s wr, 219 op/s 2024-04-12T20:55:49.694 INFO:tasks.workunit.client.0.smithi018.stderr:+ pushd ltp-full-20091231/testcases/kernel/fs/fsstress 2024-04-12T20:55:49.707 INFO:tasks.workunit.client.0.smithi018.stdout:~/cephtest/mnt.0/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/fs/fsstress ~/cephtest/mnt.0/client.0/tmp/fsstress ~/cephtest/mnt.0/client.0/tmp 2024-04-12T20:55:49.708 INFO:tasks.workunit.client.0.smithi018.stderr:+ make 2024-04-12T20:55:50.006 INFO:tasks.workunit.client.0.smithi018.stdout:cc -DNO_XFS -I/home/ubuntu/cephtest/mnt.0/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/fs/fsstress -D_LARGEFILE64_SOURCE -D_GNU_SOURCE -I../../../../include -I../../../../include -L../../../../lib fsstress.c -o fsstress 2024-04-12T20:55:50.775 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:55:50 smithi018 ceph-mon[28799]: pgmap v28: 65 pgs: 65 active+clean; 178 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 2.3 MiB/s wr, 198 op/s 2024-04-12T20:55:51.030 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:55:50 smithi186 ceph-mon[36597]: pgmap v28: 65 pgs: 65 active+clean; 178 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 2.3 MiB/s wr, 198 op/s 2024-04-12T20:55:51.212 INFO:tasks.workunit.client.0.smithi018.stderr:++ readlink -f fsstress 2024-04-12T20:55:51.218 INFO:tasks.workunit.client.0.smithi018.stderr:+ BIN=/home/ubuntu/cephtest/mnt.0/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/fs/fsstress/fsstress 2024-04-12T20:55:51.218 INFO:tasks.workunit.client.0.smithi018.stderr:+ popd 2024-04-12T20:55:51.223 INFO:tasks.workunit.client.0.smithi018.stdout:~/cephtest/mnt.0/client.0/tmp/fsstress ~/cephtest/mnt.0/client.0/tmp 2024-04-12T20:55:51.223 INFO:tasks.workunit.client.0.smithi018.stderr:+ popd 2024-04-12T20:55:51.225 INFO:tasks.workunit.client.0.smithi018.stdout:~/cephtest/mnt.0/client.0/tmp 2024-04-12T20:55:51.227 INFO:tasks.workunit.client.0.smithi018.stderr:++ mktemp -d -p . 2024-04-12T20:55:51.232 INFO:tasks.workunit.client.0.smithi018.stderr:+ T=./tmp.00Ko3goaaa 2024-04-12T20:55:51.232 INFO:tasks.workunit.client.0.smithi018.stderr:+ /home/ubuntu/cephtest/mnt.0/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/fs/fsstress/fsstress -d ./tmp.00Ko3goaaa -l 1 -n 1000 -p 10 -v 2024-04-12T20:55:51.241 INFO:tasks.workunit.client.0.smithi018.stdout:seed = 1712988546 2024-04-12T20:55:51.249 INFO:tasks.workunit.client.0.smithi018.stdout:0/0: link - no file 2024-04-12T20:55:51.249 INFO:tasks.workunit.client.0.smithi018.stdout:0/1: link - no file 2024-04-12T20:55:51.251 INFO:tasks.workunit.client.0.smithi018.stdout:8/0: link - no file 2024-04-12T20:55:51.252 INFO:tasks.workunit.client.0.smithi018.stdout:8/1: dwrite - no filename 2024-04-12T20:55:51.253 INFO:tasks.workunit.client.0.smithi018.stdout:0/2: mknod c0 0 2024-04-12T20:55:51.262 INFO:tasks.workunit.client.0.smithi018.stdout:0/3: creat f1 x:0 0 0 2024-04-12T20:55:51.264 INFO:tasks.workunit.client.0.smithi018.stdout:1/0: mknod c0 0 2024-04-12T20:55:51.264 INFO:tasks.workunit.client.0.smithi018.stdout:1/1: dwrite - no filename 2024-04-12T20:55:51.265 INFO:tasks.workunit.client.0.smithi018.stdout:0/4: mknod c2 0 2024-04-12T20:55:51.267 INFO:tasks.workunit.client.0.smithi018.stdout:0/5: fdatasync f1 0 2024-04-12T20:55:51.269 INFO:tasks.workunit.client.0.smithi018.stdout:0/6: truncate f1 858955 0 2024-04-12T20:55:51.270 INFO:tasks.workunit.client.0.smithi018.stdout:3/0: dread - no filename 2024-04-12T20:55:51.271 INFO:tasks.workunit.client.0.smithi018.stdout:0/7: read f1 [178409,28411] 0 2024-04-12T20:55:51.276 INFO:tasks.workunit.client.0.smithi018.stdout:2/0: creat f0 x:0 0 0 2024-04-12T20:55:51.276 INFO:tasks.workunit.client.0.smithi018.stdout:4/0: rmdir - no directory 2024-04-12T20:55:51.276 INFO:tasks.workunit.client.0.smithi018.stdout:4/1: dwrite - no filename 2024-04-12T20:55:51.279 INFO:tasks.workunit.client.0.smithi018.stdout:3/1: creat f0 x:0 0 0 2024-04-12T20:55:51.280 INFO:tasks.workunit.client.0.smithi018.stdout:1/2: link c0 c1 0 2024-04-12T20:55:51.283 INFO:tasks.workunit.client.0.smithi018.stdout:0/8: rename f1 to f3 0 2024-04-12T20:55:51.285 INFO:tasks.workunit.client.0.smithi018.stdout:4/2: mkdir d0 0 2024-04-12T20:55:51.285 INFO:tasks.workunit.client.0.smithi018.stdout:4/3: dread - no filename 2024-04-12T20:55:51.285 INFO:tasks.workunit.client.0.smithi018.stdout:6/0: chown . 2549937 1 2024-04-12T20:55:51.286 INFO:tasks.workunit.client.0.smithi018.stdout:1/3: mknod c2 0 2024-04-12T20:55:51.287 INFO:tasks.workunit.client.0.smithi018.stdout:1/4: chown c2 65 1 2024-04-12T20:55:51.288 INFO:tasks.workunit.client.0.smithi018.stdout:1/5: dread - no filename 2024-04-12T20:55:51.288 INFO:tasks.workunit.client.0.smithi018.stdout:4/4: rename d0 to d0/d1 22 2024-04-12T20:55:51.288 INFO:tasks.workunit.client.0.smithi018.stdout:4/5: dwrite - no filename 2024-04-12T20:55:51.288 INFO:tasks.workunit.client.0.smithi018.stdout:4/6: readlink - no filename 2024-04-12T20:55:51.288 INFO:tasks.workunit.client.0.smithi018.stdout:0/9: mkdir d4 0 2024-04-12T20:55:51.293 INFO:tasks.workunit.client.0.smithi018.stdout:5/0: mkdir d0 0 2024-04-12T20:55:51.295 INFO:tasks.workunit.client.0.smithi018.stdout:7/0: dwrite - no filename 2024-04-12T20:55:51.295 INFO:tasks.workunit.client.0.smithi018.stdout:7/1: readlink - no filename 2024-04-12T20:55:51.295 INFO:tasks.workunit.client.0.smithi018.stdout:1/6: mkdir d3 0 2024-04-12T20:55:51.297 INFO:tasks.workunit.client.0.smithi018.stdout:5/1: rename d0 to d0/d1 22 2024-04-12T20:55:51.297 INFO:tasks.workunit.client.0.smithi018.stdout:5/2: dread - no filename 2024-04-12T20:55:51.297 INFO:tasks.workunit.client.0.smithi018.stdout:6/1: symlink l0 0 2024-04-12T20:55:51.298 INFO:tasks.workunit.client.0.smithi018.stdout:6/2: dwrite - no filename 2024-04-12T20:55:51.301 INFO:tasks.workunit.client.0.smithi018.stdout:6/3: readlink l0 0 2024-04-12T20:55:51.301 INFO:tasks.workunit.client.0.smithi018.stdout:9/0: fdatasync - no filename 2024-04-12T20:55:51.301 INFO:tasks.workunit.client.0.smithi018.stdout:9/1: fsync - no filename 2024-04-12T20:55:51.303 INFO:tasks.workunit.client.0.smithi018.stdout:4/7: rmdir d0 0 2024-04-12T20:55:51.304 INFO:tasks.workunit.client.0.smithi018.stdout:4/8: chown . 39 1 2024-04-12T20:55:51.305 INFO:tasks.workunit.client.0.smithi018.stdout:1/7: stat c1 0 2024-04-12T20:55:51.306 INFO:tasks.workunit.client.0.smithi018.stdout:7/2: mknod c0 0 2024-04-12T20:55:51.308 INFO:tasks.workunit.client.0.smithi018.stdout:5/3: mkdir d0/d2 0 2024-04-12T20:55:51.312 INFO:tasks.workunit.client.0.smithi018.stdout:9/2: creat f0 x:0 0 0 2024-04-12T20:55:51.312 INFO:tasks.workunit.client.0.smithi018.stdout:1/8: creat d3/f4 x:0 0 0 2024-04-12T20:55:51.315 INFO:tasks.workunit.client.0.smithi018.stdout:5/4: mknod d0/d2/c3 0 2024-04-12T20:55:51.315 INFO:tasks.workunit.client.0.smithi018.stdout:5/5: truncate - no filename 2024-04-12T20:55:51.317 INFO:tasks.workunit.client.0.smithi018.stdout:9/3: mknod c1 0 2024-04-12T20:55:51.322 INFO:tasks.workunit.client.0.smithi018.stdout:9/4: mknod c2 0 2024-04-12T20:55:51.324 INFO:tasks.workunit.client.0.smithi018.stdout:9/5: chown c1 13044 1 2024-04-12T20:55:51.325 INFO:tasks.workunit.client.0.smithi018.stdout:0/10: dwrite f3 [0,4194304] 0 2024-04-12T20:55:51.331 INFO:tasks.workunit.client.0.smithi018.stdout:9/6: mkdir d3 0 2024-04-12T20:55:51.334 INFO:tasks.workunit.client.0.smithi018.stdout:9/7: dread - f0 zero size 2024-04-12T20:55:51.343 INFO:tasks.workunit.client.0.smithi018.stdout:1/9: dwrite d3/f4 [0,4194304] 0 2024-04-12T20:55:51.355 INFO:tasks.workunit.client.0.smithi018.stdout:9/8: getdents d3 0 2024-04-12T20:55:51.358 INFO:tasks.workunit.client.0.smithi018.stdout:9/9: symlink d3/l4 0 2024-04-12T20:55:51.360 INFO:tasks.workunit.client.0.smithi018.stdout:2/1: getdents . 0 2024-04-12T20:55:51.360 INFO:tasks.workunit.client.0.smithi018.stdout:2/2: readlink - no filename 2024-04-12T20:55:51.363 INFO:tasks.workunit.client.0.smithi018.stdout:9/10: write f0 [535414,68075] 0 2024-04-12T20:55:51.364 INFO:tasks.workunit.client.0.smithi018.stdout:3/2: getdents . 0 2024-04-12T20:55:51.365 INFO:tasks.workunit.client.0.smithi018.stdout:2/3: symlink l1 0 2024-04-12T20:55:51.368 INFO:tasks.workunit.client.0.smithi018.stdout:2/4: write f0 [210016,98190] 0 2024-04-12T20:55:51.368 INFO:tasks.workunit.client.0.smithi018.stdout:3/3: dread - f0 zero size 2024-04-12T20:55:51.373 INFO:tasks.workunit.client.0.smithi018.stdout:3/4: dread - f0 zero size 2024-04-12T20:55:51.376 INFO:tasks.workunit.client.0.smithi018.stdout:3/5: rmdir - no directory 2024-04-12T20:55:51.382 INFO:tasks.workunit.client.0.smithi018.stdout:3/6: creat f1 x:0 0 0 2024-04-12T20:55:51.399 INFO:tasks.workunit.client.0.smithi018.stdout:2/5: dwrite f0 [0,4194304] 0 2024-04-12T20:55:51.402 INFO:tasks.workunit.client.0.smithi018.stdout:2/6: chown f0 11521 1 2024-04-12T20:55:51.404 INFO:tasks.workunit.client.0.smithi018.stdout:2/7: readlink l1 0 2024-04-12T20:55:51.407 INFO:tasks.workunit.client.0.smithi018.stdout:2/8: read f0 [3024508,71333] 0 2024-04-12T20:55:51.411 INFO:tasks.workunit.client.0.smithi018.stdout:2/9: rename f0 to f2 0 2024-04-12T20:55:51.412 INFO:tasks.workunit.client.0.smithi018.stdout:3/7: dwrite f1 [0,4194304] 0 2024-04-12T20:55:51.414 INFO:tasks.workunit.client.0.smithi018.stdout:2/10: creat f3 x:0 0 0 2024-04-12T20:55:51.418 INFO:tasks.workunit.client.0.smithi018.stdout:3/8: mkdir d2 0 2024-04-12T20:55:51.420 INFO:tasks.workunit.client.0.smithi018.stdout:3/9: write f1 [4123397,69087] 0 2024-04-12T20:55:51.422 INFO:tasks.workunit.client.0.smithi018.stdout:3/10: write f0 [712048,27631] 0 2024-04-12T20:55:51.424 INFO:tasks.workunit.client.0.smithi018.stdout:3/11: write f1 [2338840,117856] 0 2024-04-12T20:55:51.444 INFO:tasks.workunit.client.0.smithi018.stdout:3/12: dread f1 [0,4194304] 0 2024-04-12T20:55:51.445 INFO:tasks.workunit.client.0.smithi018.stdout:2/11: dwrite f2 [0,4194304] 0 2024-04-12T20:55:51.446 INFO:tasks.workunit.client.0.smithi018.stdout:2/12: stat f2 0 2024-04-12T20:55:51.446 INFO:tasks.workunit.client.0.smithi018.stdout:3/13: fsync f0 0 2024-04-12T20:55:51.447 INFO:tasks.workunit.client.0.smithi018.stdout:2/13: dread - f3 zero size 2024-04-12T20:55:51.449 INFO:tasks.workunit.client.0.smithi018.stdout:2/14: creat f4 x:0 0 0 2024-04-12T20:55:51.453 INFO:tasks.workunit.client.0.smithi018.stdout:3/14: dread f0 [0,4194304] 0 2024-04-12T20:55:51.455 INFO:tasks.workunit.client.0.smithi018.stdout:3/15: truncate f1 5033730 0 2024-04-12T20:55:51.472 INFO:tasks.workunit.client.0.smithi018.stdout:3/16: dread f1 [0,4194304] 0 2024-04-12T20:55:51.472 INFO:tasks.workunit.client.0.smithi018.stdout:2/15: dwrite f2 [0,4194304] 0 2024-04-12T20:55:51.475 INFO:tasks.workunit.client.0.smithi018.stdout:3/17: dread f0 [0,4194304] 0 2024-04-12T20:55:51.478 INFO:tasks.workunit.client.0.smithi018.stdout:3/18: symlink d2/l3 0 2024-04-12T20:55:51.480 INFO:tasks.workunit.client.0.smithi018.stdout:3/19: creat d2/f4 x:0 0 0 2024-04-12T20:55:51.480 INFO:tasks.workunit.client.0.smithi018.stdout:3/20: chown f0 1545 1 2024-04-12T20:55:51.482 INFO:tasks.workunit.client.0.smithi018.stdout:3/21: creat d2/f5 x:0 0 0 2024-04-12T20:55:51.503 INFO:tasks.workunit.client.0.smithi018.stdout:9/11: dread f0 [0,4194304] 0 2024-04-12T20:55:51.504 INFO:tasks.workunit.client.0.smithi018.stdout:9/12: rename d3 to d3/d5 22 2024-04-12T20:55:51.507 INFO:tasks.workunit.client.0.smithi018.stdout:9/13: mkdir d3/d6 0 2024-04-12T20:55:51.511 INFO:tasks.workunit.client.0.smithi018.stdout:9/14: write f0 [369139,72469] 0 2024-04-12T20:55:51.539 INFO:tasks.workunit.client.0.smithi018.stdout:6/4: getdents . 0 2024-04-12T20:55:51.542 INFO:tasks.workunit.client.0.smithi018.stdout:1/10: rmdir d3 39 2024-04-12T20:55:51.544 INFO:tasks.workunit.client.0.smithi018.stdout:0/11: dwrite f3 [0,4194304] 0 2024-04-12T20:55:51.544 INFO:tasks.workunit.client.0.smithi018.stdout:6/5: creat f1 x:0 0 0 2024-04-12T20:55:51.550 INFO:tasks.workunit.client.0.smithi018.stdout:9/15: dwrite f0 [0,4194304] 0 2024-04-12T20:55:51.553 INFO:tasks.workunit.client.0.smithi018.stdout:9/16: chown c2 155 1 2024-04-12T20:55:51.556 INFO:tasks.workunit.client.0.smithi018.stdout:1/11: mknod d3/c5 0 2024-04-12T20:55:51.561 INFO:tasks.workunit.client.0.smithi018.stdout:0/12: rmdir d4 0 2024-04-12T20:55:51.564 INFO:tasks.workunit.client.0.smithi018.stdout:0/13: write f3 [1233247,95091] 0 2024-04-12T20:55:51.566 INFO:tasks.workunit.client.0.smithi018.stdout:9/17: mknod d3/d6/c7 0 2024-04-12T20:55:51.589 INFO:tasks.workunit.client.0.smithi018.stdout:2/16: unlink f2 0 2024-04-12T20:55:51.591 INFO:tasks.workunit.client.0.smithi018.stdout:6/6: dwrite f1 [0,4194304] 0 2024-04-12T20:55:51.592 INFO:tasks.workunit.client.0.smithi018.stdout:1/12: dwrite d3/f4 [0,4194304] 0 2024-04-12T20:55:51.626 INFO:tasks.workunit.client.0.smithi018.stdout:1/13: dwrite d3/f4 [4194304,4194304] 0 2024-04-12T20:55:51.631 INFO:tasks.workunit.client.0.smithi018.stdout:3/22: fsync d2/f5 0 2024-04-12T20:55:51.633 INFO:tasks.workunit.client.0.smithi018.stdout:3/23: write f0 [1597026,44290] 0 2024-04-12T20:55:51.637 INFO:tasks.workunit.client.0.smithi018.stdout:3/24: fsync f0 0 2024-04-12T20:55:51.704 INFO:tasks.workunit.client.0.smithi018.stdout:0/14: creat f5 x:0 0 0 2024-04-12T20:55:51.704 INFO:tasks.workunit.client.0.smithi018.stdout:9/18: mkdir d3/d8 0 2024-04-12T20:55:51.709 INFO:tasks.workunit.client.0.smithi018.stdout:6/7: symlink l2 0 2024-04-12T20:55:51.710 INFO:tasks.workunit.client.0.smithi018.stdout:1/14: creat d3/f6 x:0 0 0 2024-04-12T20:55:51.712 INFO:tasks.workunit.client.0.smithi018.stdout:3/25: mkdir d2/d6 0 2024-04-12T20:55:51.715 INFO:tasks.workunit.client.0.smithi018.stdout:3/26: dread - d2/f4 zero size 2024-04-12T20:55:51.716 INFO:tasks.workunit.client.0.smithi018.stdout:6/8: rename l0 to l3 0 2024-04-12T20:55:51.718 INFO:tasks.workunit.client.0.smithi018.stdout:3/27: read - d2/f4 zero size 2024-04-12T20:55:51.728 INFO:tasks.workunit.client.0.smithi018.stdout:9/19: dread f0 [0,4194304] 0 2024-04-12T20:55:51.742 INFO:tasks.workunit.client.0.smithi018.stdout:0/15: dwrite f3 [0,4194304] 0 2024-04-12T20:55:51.746 INFO:tasks.workunit.client.0.smithi018.stdout:9/20: rename d3/d8 to d3/d6/d9 0 2024-04-12T20:55:51.759 INFO:tasks.workunit.client.0.smithi018.stdout:3/28: dwrite d2/f5 [0,4194304] 0 2024-04-12T20:55:51.765 INFO:tasks.workunit.client.0.smithi018.stdout:1/15: dwrite d3/f6 [0,4194304] 0 2024-04-12T20:55:51.767 INFO:tasks.workunit.client.0.smithi018.stdout:3/29: fsync d2/f4 0 2024-04-12T20:55:51.770 INFO:tasks.workunit.client.0.smithi018.stdout:3/30: readlink d2/l3 0 2024-04-12T20:55:51.771 INFO:tasks.workunit.client.0.smithi018.stdout:9/21: creat d3/d6/d9/fa x:0 0 0 2024-04-12T20:55:51.781 INFO:tasks.workunit.client.0.smithi018.stdout:9/22: read f0 [3407959,77166] 0 2024-04-12T20:55:51.783 INFO:tasks.workunit.client.0.smithi018.stdout:0/16: dwrite f3 [0,4194304] 0 2024-04-12T20:55:51.790 INFO:tasks.workunit.client.0.smithi018.stdout:9/23: mkdir d3/d6/db 0 2024-04-12T20:55:51.812 INFO:tasks.workunit.client.0.smithi018.stdout:3/31: dwrite f0 [0,4194304] 0 2024-04-12T20:55:51.814 INFO:tasks.workunit.client.0.smithi018.stdout:0/17: getdents . 0 2024-04-12T20:55:51.815 INFO:tasks.workunit.client.0.smithi018.stdout:3/32: write d2/f4 [260234,88508] 0 2024-04-12T20:55:51.817 INFO:tasks.workunit.client.0.smithi018.stdout:3/33: rename d2 to d2/d7 22 2024-04-12T20:55:51.821 INFO:tasks.workunit.client.0.smithi018.stdout:0/18: symlink l6 0 2024-04-12T20:55:51.827 INFO:tasks.workunit.client.0.smithi018.stdout:0/19: symlink l7 0 2024-04-12T20:55:51.830 INFO:tasks.workunit.client.0.smithi018.stdout:0/20: write f3 [4761914,92354] 0 2024-04-12T20:55:51.830 INFO:tasks.workunit.client.0.smithi018.stdout:3/34: creat d2/d6/f8 x:0 0 0 2024-04-12T20:55:51.832 INFO:tasks.workunit.client.0.smithi018.stdout:3/35: chown f0 100187301 1 2024-04-12T20:55:51.832 INFO:tasks.workunit.client.0.smithi018.stdout:0/21: dread - f5 zero size 2024-04-12T20:55:51.833 INFO:tasks.workunit.client.0.smithi018.stdout:0/22: dread - f5 zero size 2024-04-12T20:55:51.839 INFO:tasks.workunit.client.0.smithi018.stdout:3/36: fdatasync f1 0 2024-04-12T20:55:51.841 INFO:tasks.workunit.client.0.smithi018.stdout:0/23: mknod c8 0 2024-04-12T20:55:51.843 INFO:tasks.workunit.client.0.smithi018.stdout:6/9: unlink l3 0 2024-04-12T20:55:51.847 INFO:tasks.workunit.client.0.smithi018.stdout:0/24: rename f3 to f9 0 2024-04-12T20:55:51.848 INFO:tasks.workunit.client.0.smithi018.stdout:9/24: dwrite f0 [4194304,4194304] 0 2024-04-12T20:55:51.852 INFO:tasks.workunit.client.0.smithi018.stdout:6/10: mknod c4 0 2024-04-12T20:55:51.856 INFO:tasks.workunit.client.0.smithi018.stdout:0/25: mkdir da 0 2024-04-12T20:55:51.860 INFO:tasks.workunit.client.0.smithi018.stdout:6/11: mkdir d5 0 2024-04-12T20:55:51.872 INFO:tasks.workunit.client.0.smithi018.stdout:6/12: mkdir d5/d6 0 2024-04-12T20:55:51.873 INFO:tasks.workunit.client.0.smithi018.stdout:6/13: readlink l2 0 2024-04-12T20:55:51.873 INFO:tasks.workunit.client.0.smithi018.stdout:3/37: dwrite d2/d6/f8 [0,4194304] 0 2024-04-12T20:55:51.878 INFO:tasks.workunit.client.0.smithi018.stdout:3/38: truncate d2/f4 871960 0 2024-04-12T20:55:51.889 INFO:tasks.workunit.client.0.smithi018.stdout:3/39: creat d2/d6/f9 x:0 0 0 2024-04-12T20:55:51.892 INFO:tasks.workunit.client.0.smithi018.stdout:9/25: fsync f0 0 2024-04-12T20:55:51.898 INFO:tasks.workunit.client.0.smithi018.stdout:3/40: mknod d2/ca 0 2024-04-12T20:55:51.902 INFO:tasks.workunit.client.0.smithi018.stdout:9/26: mkdir d3/d6/d9/dc 0 2024-04-12T20:55:51.904 INFO:tasks.workunit.client.0.smithi018.stdout:3/41: symlink d2/d6/lb 0 2024-04-12T20:55:51.906 INFO:tasks.workunit.client.0.smithi018.stdout:0/26: dwrite f9 [4194304,4194304] 0 2024-04-12T20:55:51.906 INFO:tasks.workunit.client.0.smithi018.stdout:3/42: chown d2/f5 43 1 2024-04-12T20:55:51.911 INFO:tasks.workunit.client.0.smithi018.stdout:6/14: dwrite f1 [0,4194304] 0 2024-04-12T20:55:51.912 INFO:tasks.workunit.client.0.smithi018.stdout:0/27: stat c8 0 2024-04-12T20:55:51.913 INFO:tasks.workunit.client.0.smithi018.stdout:9/27: creat d3/d6/d9/fd x:0 0 0 2024-04-12T20:55:51.928 INFO:tasks.workunit.client.0.smithi018.stdout:6/15: creat d5/f7 x:0 0 0 2024-04-12T20:55:51.929 INFO:tasks.workunit.client.0.smithi018.stdout:3/43: truncate f1 4423746 0 2024-04-12T20:55:51.932 INFO:tasks.workunit.client.0.smithi018.stdout:3/44: creat d2/d6/fc x:0 0 0 2024-04-12T20:55:51.935 INFO:tasks.workunit.client.0.smithi018.stdout:6/16: link l2 d5/d6/l8 0 2024-04-12T20:55:51.941 INFO:tasks.workunit.client.0.smithi018.stdout:6/17: truncate d5/f7 590227 0 2024-04-12T20:55:51.943 INFO:tasks.workunit.client.0.smithi018.stdout:6/18: truncate f1 4250767 0 2024-04-12T20:55:51.945 INFO:tasks.workunit.client.0.smithi018.stdout:9/28: dwrite d3/d6/d9/fd [0,4194304] 0 2024-04-12T20:55:51.953 INFO:tasks.workunit.client.0.smithi018.stdout:9/29: unlink f0 0 2024-04-12T20:55:51.962 INFO:tasks.workunit.client.0.smithi018.stdout:9/30: symlink d3/d6/le 0 2024-04-12T20:55:51.963 INFO:tasks.workunit.client.0.smithi018.stdout:3/45: dwrite d2/d6/fc [0,4194304] 0 2024-04-12T20:55:51.963 INFO:tasks.workunit.client.0.smithi018.stdout:6/19: getdents d5 0 2024-04-12T20:55:51.969 INFO:tasks.workunit.client.0.smithi018.stdout:0/28: truncate f9 1935528 0 2024-04-12T20:55:51.975 INFO:tasks.workunit.client.0.smithi018.stdout:3/46: mknod d2/d6/cd 0 2024-04-12T20:55:51.979 INFO:tasks.workunit.client.0.smithi018.stdout:6/20: symlink d5/d6/l9 0 2024-04-12T20:55:51.984 INFO:tasks.workunit.client.0.smithi018.stdout:0/29: getdents da 0 2024-04-12T20:55:51.988 INFO:tasks.workunit.client.0.smithi018.stdout:0/30: fdatasync f9 0 2024-04-12T20:55:51.988 INFO:tasks.workunit.client.0.smithi018.stdout:9/31: dwrite d3/d6/d9/fd [0,4194304] 0 2024-04-12T20:55:51.994 INFO:tasks.workunit.client.0.smithi018.stdout:0/31: write f5 [857554,54096] 0 2024-04-12T20:55:51.994 INFO:tasks.workunit.client.0.smithi018.stdout:9/32: write d3/d6/d9/fd [3912757,105486] 0 2024-04-12T20:55:52.006 INFO:tasks.workunit.client.0.smithi018.stdout:9/33: unlink d3/d6/c7 0 2024-04-12T20:55:52.008 INFO:tasks.workunit.client.0.smithi018.stdout:9/34: dread - d3/d6/d9/fa zero size 2024-04-12T20:55:52.013 INFO:tasks.workunit.client.0.smithi018.stdout:3/47: rmdir d2/d6 39 2024-04-12T20:55:52.013 INFO:tasks.workunit.client.0.smithi018.stdout:0/32: mknod da/cb 0 2024-04-12T20:55:52.014 INFO:tasks.workunit.client.0.smithi018.stdout:6/21: dwrite d5/f7 [0,4194304] 0 2024-04-12T20:55:52.014 INFO:tasks.workunit.client.0.smithi018.stdout:0/33: fsync f5 0 2024-04-12T20:55:52.017 INFO:tasks.workunit.client.0.smithi018.stdout:9/35: mknod d3/d6/d9/cf 0 2024-04-12T20:55:52.025 INFO:tasks.workunit.client.0.smithi018.stdout:6/22: symlink d5/d6/la 0 2024-04-12T20:55:52.025 INFO:tasks.workunit.client.0.smithi018.stdout:9/36: creat d3/d6/f10 x:0 0 0 2024-04-12T20:55:52.031 INFO:tasks.workunit.client.0.smithi018.stdout:3/48: dread d2/f4 [0,4194304] 0 2024-04-12T20:55:52.037 INFO:tasks.workunit.client.0.smithi018.stdout:3/49: mknod d2/d6/ce 0 2024-04-12T20:55:52.044 INFO:tasks.workunit.client.0.smithi018.stdout:0/34: dwrite f5 [0,4194304] 0 2024-04-12T20:55:52.049 INFO:tasks.workunit.client.0.smithi018.stdout:0/35: write f9 [2867433,119754] 0 2024-04-12T20:55:52.057 INFO:tasks.workunit.client.0.smithi018.stdout:0/36: creat da/fc x:0 0 0 2024-04-12T20:55:52.061 INFO:tasks.workunit.client.0.smithi018.stdout:0/37: fdatasync f9 0 2024-04-12T20:55:52.063 INFO:tasks.workunit.client.0.smithi018.stdout:3/50: dread d2/d6/f8 [0,4194304] 0 2024-04-12T20:55:52.064 INFO:tasks.workunit.client.0.smithi018.stdout:6/23: truncate d5/f7 3132306 0 2024-04-12T20:55:52.071 INFO:tasks.workunit.client.0.smithi018.stdout:0/38: unlink f5 0 2024-04-12T20:55:52.077 INFO:tasks.workunit.client.0.smithi018.stdout:6/24: mknod d5/d6/cb 0 2024-04-12T20:55:52.080 INFO:tasks.workunit.client.0.smithi018.stdout:3/51: mkdir d2/df 0 2024-04-12T20:55:52.082 INFO:tasks.workunit.client.0.smithi018.stdout:3/52: chown d2/d6/f8 0 1 2024-04-12T20:55:52.093 INFO:tasks.workunit.client.0.smithi018.stdout:6/25: write d5/f7 [3771793,102351] 0 2024-04-12T20:55:52.103 INFO:tasks.workunit.client.0.smithi018.stdout:0/39: dwrite f9 [0,4194304] 0 2024-04-12T20:55:52.109 INFO:tasks.workunit.client.0.smithi018.stdout:0/40: symlink da/ld 0 2024-04-12T20:55:52.110 INFO:tasks.workunit.client.0.smithi018.stdout:3/53: dwrite f1 [0,4194304] 0 2024-04-12T20:55:52.114 INFO:tasks.workunit.client.0.smithi018.stdout:0/41: creat da/fe x:0 0 0 2024-04-12T20:55:52.118 INFO:tasks.workunit.client.0.smithi018.stdout:0/42: rename da to da/df 22 2024-04-12T20:55:52.118 INFO:tasks.workunit.client.0.smithi018.stdout:6/26: dread f1 [0,4194304] 0 2024-04-12T20:55:52.138 INFO:tasks.workunit.client.0.smithi018.stdout:3/54: dwrite f0 [0,4194304] 0 2024-04-12T20:55:52.151 INFO:tasks.workunit.client.0.smithi018.stdout:6/27: dwrite f1 [4194304,4194304] 0 2024-04-12T20:55:52.167 INFO:tasks.workunit.client.0.smithi018.stdout:3/55: truncate d2/d6/f8 3344384 0 2024-04-12T20:55:52.168 INFO:tasks.workunit.client.0.smithi018.stdout:6/28: truncate d5/f7 748083 0 2024-04-12T20:55:52.169 INFO:tasks.workunit.client.0.smithi018.stdout:3/56: fdatasync d2/f5 0 2024-04-12T20:55:52.171 INFO:tasks.workunit.client.0.smithi018.stdout:3/57: read f0 [76362,45343] 0 2024-04-12T20:55:52.173 INFO:tasks.workunit.client.0.smithi018.stdout:3/58: fsync d2/d6/f9 0 2024-04-12T20:55:52.185 INFO:tasks.workunit.client.0.smithi018.stdout:6/29: truncate d5/f7 1070099 0 2024-04-12T20:55:52.189 INFO:tasks.workunit.client.0.smithi018.stdout:3/59: dread d2/f5 [0,4194304] 0 2024-04-12T20:55:52.195 INFO:tasks.workunit.client.0.smithi018.stdout:3/60: rmdir d2/df 0 2024-04-12T20:55:52.199 INFO:tasks.workunit.client.0.smithi018.stdout:6/30: truncate f1 1127083 0 2024-04-12T20:55:52.209 INFO:tasks.workunit.client.0.smithi018.stdout:6/31: dread f1 [0,4194304] 0 2024-04-12T20:55:52.212 INFO:tasks.workunit.client.0.smithi018.stdout:6/32: write f1 [1741953,52720] 0 2024-04-12T20:55:52.216 INFO:tasks.workunit.client.0.smithi018.stdout:6/33: symlink d5/d6/lc 0 2024-04-12T20:55:52.221 INFO:tasks.workunit.client.0.smithi018.stdout:6/34: creat d5/fd x:0 0 0 2024-04-12T20:55:52.222 INFO:tasks.workunit.client.0.smithi018.stdout:6/35: readlink d5/d6/l9 0 2024-04-12T20:55:52.223 INFO:tasks.workunit.client.0.smithi018.stdout:6/36: write f1 [765832,63191] 0 2024-04-12T20:55:52.226 INFO:tasks.workunit.client.0.smithi018.stdout:6/37: mkdir d5/d6/de 0 2024-04-12T20:55:52.241 INFO:tasks.workunit.client.0.smithi018.stdout:6/38: dread d5/f7 [0,4194304] 0 2024-04-12T20:55:52.261 INFO:tasks.workunit.client.0.smithi018.stdout:6/39: dwrite d5/fd [0,4194304] 0 2024-04-12T20:55:52.262 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:55:51 smithi018 ceph-mon[28799]: from='mgr.24359 172.21.15.186:0/1315906828' entity='mgr.smithi186.kqcfxk' 2024-04-12T20:55:52.263 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:55:51 smithi018 ceph-mon[28799]: from='mgr.24359 172.21.15.186:0/1315906828' entity='mgr.smithi186.kqcfxk' 2024-04-12T20:55:52.263 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:55:51 smithi018 ceph-mon[28799]: Reconfiguring prometheus.smithi018 (dependencies changed)... 2024-04-12T20:55:52.280 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:55:51 smithi186 ceph-mon[36597]: from='mgr.24359 172.21.15.186:0/1315906828' entity='mgr.smithi186.kqcfxk' 2024-04-12T20:55:52.281 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:55:51 smithi186 ceph-mon[36597]: from='mgr.24359 172.21.15.186:0/1315906828' entity='mgr.smithi186.kqcfxk' 2024-04-12T20:55:52.281 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:55:51 smithi186 ceph-mon[36597]: Reconfiguring prometheus.smithi018 (dependencies changed)... 2024-04-12T20:55:52.292 INFO:tasks.workunit.client.0.smithi018.stdout:6/40: fdatasync d5/fd 0 2024-04-12T20:55:52.295 INFO:tasks.workunit.client.0.smithi018.stdout:6/41: dread d5/f7 [0,4194304] 0 2024-04-12T20:55:52.308 INFO:tasks.workunit.client.0.smithi018.stdout:6/42: dread d5/fd [0,4194304] 0 2024-04-12T20:55:52.309 INFO:tasks.workunit.client.0.smithi018.stdout:6/43: write f1 [2027282,63396] 0 2024-04-12T20:55:52.311 INFO:tasks.workunit.client.0.smithi018.stdout:6/44: read d5/fd [3246760,12273] 0 2024-04-12T20:55:52.313 INFO:tasks.workunit.client.0.smithi018.stdout:6/45: mkdir d5/df 0 2024-04-12T20:55:52.321 INFO:tasks.workunit.client.0.smithi018.stdout:6/46: dread f1 [0,4194304] 0 2024-04-12T20:55:52.323 INFO:tasks.workunit.client.0.smithi018.stdout:6/47: fsync d5/fd 0 2024-04-12T20:55:52.324 INFO:tasks.workunit.client.0.smithi018.stdout:6/48: chown d5/d6 0 1 2024-04-12T20:55:52.329 INFO:tasks.workunit.client.0.smithi018.stdout:6/49: rmdir d5/d6/de 0 2024-04-12T20:55:52.330 INFO:tasks.workunit.client.0.smithi018.stdout:6/50: creat d5/f10 x:0 0 0 2024-04-12T20:55:52.332 INFO:tasks.workunit.client.0.smithi018.stdout:6/51: write d5/fd [6949,31610] 0 2024-04-12T20:55:52.337 INFO:tasks.workunit.client.0.smithi018.stdout:6/52: rename f1 to d5/d6/f11 0 2024-04-12T20:55:52.339 INFO:tasks.workunit.client.0.smithi018.stdout:6/53: truncate d5/f10 565091 0 2024-04-12T20:55:52.342 INFO:tasks.workunit.client.0.smithi018.stdout:6/54: creat d5/d6/f12 x:0 0 0 2024-04-12T20:55:52.377 INFO:tasks.workunit.client.0.smithi018.stdout:6/55: link d5/d6/l8 d5/df/l13 0 2024-04-12T20:55:52.383 INFO:tasks.workunit.client.0.smithi018.stdout:6/56: fdatasync d5/f7 0 2024-04-12T20:55:52.385 INFO:tasks.workunit.client.0.smithi018.stdout:6/57: chown d5/d6/f12 0 1 2024-04-12T20:55:53.101 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:55:52 smithi018 ceph-mon[28799]: Reconfiguring daemon prometheus.smithi018 on smithi018 2024-04-12T20:55:53.102 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:55:52 smithi018 ceph-mon[28799]: pgmap v29: 65 pgs: 65 active+clean; 192 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 3.6 MiB/s wr, 284 op/s 2024-04-12T20:55:53.280 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:55:52 smithi186 ceph-mon[36597]: Reconfiguring daemon prometheus.smithi018 on smithi018 2024-04-12T20:55:53.281 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:55:53 smithi186 ceph-mon[36597]: pgmap v29: 65 pgs: 65 active+clean; 192 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 3.6 MiB/s wr, 284 op/s 2024-04-12T20:55:54.275 INFO:tasks.workunit.client.0.smithi018.stdout:2/17: sync 2024-04-12T20:55:54.275 INFO:tasks.workunit.client.0.smithi018.stdout:4/9: sync 2024-04-12T20:55:54.275 INFO:tasks.workunit.client.0.smithi018.stdout:4/10: truncate - no filename 2024-04-12T20:55:54.275 INFO:tasks.workunit.client.0.smithi018.stdout:8/2: sync 2024-04-12T20:55:54.275 INFO:tasks.workunit.client.0.smithi018.stdout:8/3: dread - no filename 2024-04-12T20:55:54.275 INFO:tasks.workunit.client.0.smithi018.stdout:8/4: read - no filename 2024-04-12T20:55:54.275 INFO:tasks.workunit.client.0.smithi018.stdout:8/5: dwrite - no filename 2024-04-12T20:55:54.275 INFO:tasks.workunit.client.0.smithi018.stdout:0/43: sync 2024-04-12T20:55:54.275 INFO:tasks.workunit.client.0.smithi018.stdout:1/16: sync 2024-04-12T20:55:54.275 INFO:tasks.workunit.client.0.smithi018.stdout:9/37: sync 2024-04-12T20:55:54.276 INFO:tasks.workunit.client.0.smithi018.stdout:5/6: sync 2024-04-12T20:55:54.276 INFO:tasks.workunit.client.0.smithi018.stdout:6/58: sync 2024-04-12T20:55:54.276 INFO:tasks.workunit.client.0.smithi018.stdout:3/61: sync 2024-04-12T20:55:54.276 INFO:tasks.workunit.client.0.smithi018.stdout:7/3: sync 2024-04-12T20:55:54.276 INFO:tasks.workunit.client.0.smithi018.stdout:7/4: rmdir - no directory 2024-04-12T20:55:54.276 INFO:tasks.workunit.client.0.smithi018.stdout:7/5: truncate - no filename 2024-04-12T20:55:54.282 INFO:tasks.workunit.client.0.smithi018.stdout:2/18: truncate f4 697420 0 2024-04-12T20:55:54.283 INFO:tasks.workunit.client.0.smithi018.stdout:7/6: mkdir d1 0 2024-04-12T20:55:54.283 INFO:tasks.workunit.client.0.smithi018.stdout:7/7: truncate - no filename 2024-04-12T20:55:54.285 INFO:tasks.workunit.client.0.smithi018.stdout:2/19: write f3 [1041425,83090] 0 2024-04-12T20:55:54.289 INFO:tasks.workunit.client.0.smithi018.stdout:0/44: symlink da/l10 0 2024-04-12T20:55:54.290 INFO:tasks.workunit.client.0.smithi018.stdout:6/59: mknod d5/d6/c14 0 2024-04-12T20:55:54.293 INFO:tasks.workunit.client.0.smithi018.stdout:0/45: rename da to da/d11 22 2024-04-12T20:55:54.293 INFO:tasks.workunit.client.0.smithi018.stdout:8/6: creat f0 x:0 0 0 2024-04-12T20:55:54.293 INFO:tasks.workunit.client.0.smithi018.stdout:5/7: mknod d0/c4 0 2024-04-12T20:55:54.293 INFO:tasks.workunit.client.0.smithi018.stdout:5/8: dwrite - no filename 2024-04-12T20:55:54.293 INFO:tasks.workunit.client.0.smithi018.stdout:8/7: rmdir - no directory 2024-04-12T20:55:54.295 INFO:tasks.workunit.client.0.smithi018.stdout:3/62: symlink d2/l10 0 2024-04-12T20:55:54.298 INFO:tasks.workunit.client.0.smithi018.stdout:8/8: truncate f0 266679 0 2024-04-12T20:55:54.300 INFO:tasks.workunit.client.0.smithi018.stdout:2/20: mkdir d5 0 2024-04-12T20:55:54.302 INFO:tasks.workunit.client.0.smithi018.stdout:6/60: mkdir d5/d15 0 2024-04-12T20:55:54.303 INFO:tasks.workunit.client.0.smithi018.stdout:5/9: chown d0/d2/c3 1255118 1 2024-04-12T20:55:54.303 INFO:tasks.workunit.client.0.smithi018.stdout:0/46: symlink da/l12 0 2024-04-12T20:55:54.304 INFO:tasks.workunit.client.0.smithi018.stdout:0/47: stat c0 0 2024-04-12T20:55:54.305 INFO:tasks.workunit.client.0.smithi018.stdout:8/9: read f0 [116087,80888] 0 2024-04-12T20:55:54.309 INFO:tasks.workunit.client.0.smithi018.stdout:7/8: creat d1/f2 x:0 0 0 2024-04-12T20:55:54.312 INFO:tasks.workunit.client.0.smithi018.stdout:8/10: write f0 [923328,115836] 0 2024-04-12T20:55:54.314 INFO:tasks.workunit.client.0.smithi018.stdout:9/38: dwrite d3/d6/d9/fa [0,4194304] 0 2024-04-12T20:55:54.314 INFO:tasks.workunit.client.0.smithi018.stdout:1/17: dread d3/f4 [4194304,4194304] 0 2024-04-12T20:55:54.317 INFO:tasks.workunit.client.0.smithi018.stdout:4/11: getdents . 0 2024-04-12T20:55:54.317 INFO:tasks.workunit.client.0.smithi018.stdout:4/12: dread - no filename 2024-04-12T20:55:54.318 INFO:tasks.workunit.client.0.smithi018.stdout:4/13: chown . 532918625 1 2024-04-12T20:55:54.318 INFO:tasks.workunit.client.0.smithi018.stdout:4/14: dread - no filename 2024-04-12T20:55:54.318 INFO:tasks.workunit.client.0.smithi018.stdout:4/15: truncate - no filename 2024-04-12T20:55:54.318 INFO:tasks.workunit.client.0.smithi018.stdout:4/16: stat - no entries 2024-04-12T20:55:54.318 INFO:tasks.workunit.client.0.smithi018.stdout:1/18: write d3/f6 [1542715,22429] 0 2024-04-12T20:55:54.321 INFO:tasks.workunit.client.0.smithi018.stdout:9/39: chown d3 5897 1 2024-04-12T20:55:54.323 INFO:tasks.workunit.client.0.smithi018.stdout:0/48: creat da/f13 x:0 0 0 2024-04-12T20:55:54.326 INFO:tasks.workunit.client.0.smithi018.stdout:6/61: readlink d5/df/l13 0 2024-04-12T20:55:54.326 INFO:tasks.workunit.client.0.smithi018.stdout:9/40: write d3/d6/d9/fd [1690871,112746] 0 2024-04-12T20:55:54.330 INFO:tasks.workunit.client.0.smithi018.stdout:7/9: mkdir d1/d3 0 2024-04-12T20:55:54.333 INFO:tasks.workunit.client.0.smithi018.stdout:8/11: write f0 [1566579,78794] 0 2024-04-12T20:55:54.341 INFO:tasks.workunit.client.0.smithi018.stdout:0/49: symlink da/l14 0 2024-04-12T20:55:54.342 INFO:tasks.workunit.client.0.smithi018.stdout:4/17: mkdir d2 0 2024-04-12T20:55:54.342 INFO:tasks.workunit.client.0.smithi018.stdout:4/18: dwrite - no filename 2024-04-12T20:55:54.344 INFO:tasks.workunit.client.0.smithi018.stdout:9/41: mknod d3/d6/c11 0 2024-04-12T20:55:54.346 INFO:tasks.workunit.client.0.smithi018.stdout:2/21: dwrite f3 [0,4194304] 0 2024-04-12T20:55:54.347 INFO:tasks.workunit.client.0.smithi018.stdout:2/22: write f4 [1015066,114646] 0 2024-04-12T20:55:54.350 INFO:tasks.workunit.client.0.smithi018.stdout:8/12: symlink l1 0 2024-04-12T20:55:54.354 INFO:tasks.workunit.client.0.smithi018.stdout:0/50: unlink f9 0 2024-04-12T20:55:54.356 INFO:tasks.workunit.client.0.smithi018.stdout:7/10: creat d1/d3/f4 x:0 0 0 2024-04-12T20:55:54.357 INFO:tasks.workunit.client.0.smithi018.stdout:9/42: creat d3/d6/f12 x:0 0 0 2024-04-12T20:55:54.358 INFO:tasks.workunit.client.0.smithi018.stdout:7/11: truncate d1/f2 262237 0 2024-04-12T20:55:54.361 INFO:tasks.workunit.client.0.smithi018.stdout:5/10: getdents d0/d2 0 2024-04-12T20:55:54.361 INFO:tasks.workunit.client.0.smithi018.stdout:5/11: dwrite - no filename 2024-04-12T20:55:54.367 INFO:tasks.workunit.client.0.smithi018.stdout:7/12: mknod d1/d3/c5 0 2024-04-12T20:55:54.368 INFO:tasks.workunit.client.0.smithi018.stdout:6/62: link d5/d6/la d5/df/l16 0 2024-04-12T20:55:54.370 INFO:tasks.workunit.client.0.smithi018.stdout:2/23: mknod d5/c6 0 2024-04-12T20:55:54.372 INFO:tasks.workunit.client.0.smithi018.stdout:5/12: mknod d0/d2/c5 0 2024-04-12T20:55:54.372 INFO:tasks.workunit.client.0.smithi018.stdout:5/13: dwrite - no filename 2024-04-12T20:55:54.372 INFO:tasks.workunit.client.0.smithi018.stdout:5/14: write - no filename 2024-04-12T20:55:54.374 INFO:tasks.workunit.client.0.smithi018.stdout:4/19: rmdir d2 0 2024-04-12T20:55:54.374 INFO:tasks.workunit.client.0.smithi018.stdout:4/20: truncate - no filename 2024-04-12T20:55:54.376 INFO:tasks.workunit.client.0.smithi018.stdout:9/43: mkdir d3/d6/d9/dc/d13 0 2024-04-12T20:55:54.377 INFO:tasks.workunit.client.0.smithi018.stdout:0/51: dwrite da/fc [0,4194304] 0 2024-04-12T20:55:54.379 INFO:tasks.workunit.client.0.smithi018.stdout:9/44: write d3/d6/f10 [765560,36726] 0 2024-04-12T20:55:54.382 INFO:tasks.workunit.client.0.smithi018.stdout:0/52: write da/fe [228475,84402] 0 2024-04-12T20:55:54.383 INFO:tasks.workunit.client.0.smithi018.stdout:2/24: mkdir d5/d7 0 2024-04-12T20:55:54.383 INFO:tasks.workunit.client.0.smithi018.stdout:9/45: truncate d3/d6/f12 466861 0 2024-04-12T20:55:54.509 INFO:tasks.workunit.client.0.smithi018.stdout:0/53: mkdir da/d15 0 2024-04-12T20:55:54.512 INFO:tasks.workunit.client.0.smithi018.stdout:0/54: fdatasync da/fe 0 2024-04-12T20:55:54.516 INFO:tasks.workunit.client.0.smithi018.stdout:2/25: dread f4 [0,4194304] 0 2024-04-12T20:55:54.519 INFO:tasks.workunit.client.0.smithi018.stdout:9/46: dwrite d3/d6/f12 [0,4194304] 0 2024-04-12T20:55:54.528 INFO:tasks.workunit.client.0.smithi018.stdout:8/13: getdents . 0 2024-04-12T20:55:54.529 INFO:tasks.workunit.client.0.smithi018.stdout:7/13: rmdir d1 39 2024-04-12T20:55:54.531 INFO:tasks.workunit.client.0.smithi018.stdout:8/14: write f0 [1663376,92739] 0 2024-04-12T20:55:54.531 INFO:tasks.workunit.client.0.smithi018.stdout:6/63: truncate d5/fd 3635093 0 2024-04-12T20:55:54.532 INFO:tasks.workunit.client.0.smithi018.stdout:2/26: mkdir d5/d8 0 2024-04-12T20:55:54.534 INFO:tasks.workunit.client.0.smithi018.stdout:9/47: rmdir d3/d6/d9/dc 39 2024-04-12T20:55:54.541 INFO:tasks.workunit.client.0.smithi018.stdout:3/63: dwrite d2/d6/f8 [0,4194304] 0 2024-04-12T20:55:54.543 INFO:tasks.workunit.client.0.smithi018.stdout:6/64: dread d5/d6/f11 [0,4194304] 0 2024-04-12T20:55:54.555 INFO:tasks.workunit.client.0.smithi018.stdout:4/21: getdents . 0 2024-04-12T20:55:54.555 INFO:tasks.workunit.client.0.smithi018.stdout:4/22: read - no filename 2024-04-12T20:55:54.555 INFO:tasks.workunit.client.0.smithi018.stdout:4/23: fdatasync - no filename 2024-04-12T20:55:54.555 INFO:tasks.workunit.client.0.smithi018.stdout:2/27: dwrite f3 [0,4194304] 0 2024-04-12T20:55:54.558 INFO:tasks.workunit.client.0.smithi018.stdout:4/24: chown . 493212890 1 2024-04-12T20:55:54.559 INFO:tasks.workunit.client.0.smithi018.stdout:7/14: dread - d1/d3/f4 zero size 2024-04-12T20:55:54.561 INFO:tasks.workunit.client.0.smithi018.stdout:8/15: rename l1 to l2 0 2024-04-12T20:55:54.563 INFO:tasks.workunit.client.0.smithi018.stdout:9/48: unlink d3/d6/le 0 2024-04-12T20:55:54.567 INFO:tasks.workunit.client.0.smithi018.stdout:7/15: mkdir d1/d6 0 2024-04-12T20:55:54.572 INFO:tasks.workunit.client.0.smithi018.stdout:0/55: getdents da 0 2024-04-12T20:55:54.575 INFO:tasks.workunit.client.0.smithi018.stdout:2/28: mkdir d5/d7/d9 0 2024-04-12T20:55:54.577 INFO:tasks.workunit.client.0.smithi018.stdout:3/64: dwrite d2/d6/fc [0,4194304] 0 2024-04-12T20:55:54.585 INFO:tasks.workunit.client.0.smithi018.stdout:4/25: getdents . 0 2024-04-12T20:55:54.585 INFO:tasks.workunit.client.0.smithi018.stdout:4/26: write - no filename 2024-04-12T20:55:54.585 INFO:tasks.workunit.client.0.smithi018.stdout:4/27: write - no filename 2024-04-12T20:55:54.586 INFO:tasks.workunit.client.0.smithi018.stdout:6/65: dwrite d5/d6/f12 [0,4194304] 0 2024-04-12T20:55:54.586 INFO:tasks.workunit.client.0.smithi018.stdout:7/16: creat d1/f7 x:0 0 0 2024-04-12T20:55:54.589 INFO:tasks.workunit.client.0.smithi018.stdout:2/29: read f4 [744852,60621] 0 2024-04-12T20:55:54.595 INFO:tasks.workunit.client.0.smithi018.stdout:7/17: stat d1/d6 0 2024-04-12T20:55:54.596 INFO:tasks.workunit.client.0.smithi018.stdout:3/65: chown d2/l3 99265 1 2024-04-12T20:55:54.599 INFO:tasks.workunit.client.0.smithi018.stdout:0/56: rename da/cb to da/d15/c16 0 2024-04-12T20:55:54.600 INFO:tasks.workunit.client.0.smithi018.stdout:6/66: mkdir d5/df/d17 0 2024-04-12T20:55:54.605 INFO:tasks.workunit.client.0.smithi018.stdout:3/66: unlink f0 0 2024-04-12T20:55:54.608 INFO:tasks.workunit.client.0.smithi018.stdout:3/67: readlink d2/l3 0 2024-04-12T20:55:54.608 INFO:tasks.workunit.client.0.smithi018.stdout:8/16: dwrite f0 [0,4194304] 0 2024-04-12T20:55:54.613 INFO:tasks.workunit.client.0.smithi018.stdout:9/49: dwrite d3/d6/f10 [0,4194304] 0 2024-04-12T20:55:54.613 INFO:tasks.workunit.client.0.smithi018.stdout:7/18: getdents d1/d6 0 2024-04-12T20:55:54.617 INFO:tasks.workunit.client.0.smithi018.stdout:8/17: chown l2 112 1 2024-04-12T20:55:54.620 INFO:tasks.workunit.client.0.smithi018.stdout:8/18: truncate f0 4501072 0 2024-04-12T20:55:54.624 INFO:tasks.workunit.client.0.smithi018.stdout:0/57: getdents da/d15 0 2024-04-12T20:55:54.625 INFO:tasks.workunit.client.0.smithi018.stdout:3/68: mkdir d2/d6/d11 0 2024-04-12T20:55:54.625 INFO:tasks.workunit.client.0.smithi018.stdout:8/19: write f0 [439731,74846] 0 2024-04-12T20:55:54.626 INFO:tasks.workunit.client.0.smithi018.stdout:0/58: stat c0 0 2024-04-12T20:55:54.643 INFO:tasks.workunit.client.0.smithi018.stdout:6/67: link d5/d6/f11 d5/df/f18 0 2024-04-12T20:55:54.644 INFO:tasks.workunit.client.0.smithi018.stdout:2/30: dread f3 [0,4194304] 0 2024-04-12T20:55:54.645 INFO:tasks.workunit.client.0.smithi018.stdout:0/59: dread - da/f13 zero size 2024-04-12T20:55:54.651 INFO:tasks.workunit.client.0.smithi018.stdout:8/20: write f0 [3283802,109783] 0 2024-04-12T20:55:54.651 INFO:tasks.workunit.client.0.smithi018.stdout:6/68: chown d5/d6/f11 0 1 2024-04-12T20:55:54.651 INFO:tasks.workunit.client.0.smithi018.stdout:0/60: chown c8 1409715 1 2024-04-12T20:55:54.652 INFO:tasks.workunit.client.0.smithi018.stdout:0/61: truncate da/f13 508237 0 2024-04-12T20:55:54.653 INFO:tasks.workunit.client.0.smithi018.stdout:9/50: dwrite d3/d6/f10 [0,4194304] 0 2024-04-12T20:55:54.670 INFO:tasks.workunit.client.0.smithi018.stdout:0/62: dwrite da/fe [0,4194304] 0 2024-04-12T20:55:54.672 INFO:tasks.workunit.client.0.smithi018.stdout:0/63: write da/fe [2209676,2512] 0 2024-04-12T20:55:54.753 INFO:tasks.workunit.client.0.smithi018.stdout:3/69: creat d2/d6/f12 x:0 0 0 2024-04-12T20:55:54.756 INFO:tasks.workunit.client.0.smithi018.stdout:8/21: symlink l3 0 2024-04-12T20:55:54.757 INFO:tasks.workunit.client.0.smithi018.stdout:6/69: mkdir d5/df/d19 0 2024-04-12T20:55:54.758 INFO:tasks.workunit.client.0.smithi018.stdout:9/51: mknod d3/c14 0 2024-04-12T20:55:54.761 INFO:tasks.workunit.client.0.smithi018.stdout:0/64: mkdir da/d15/d17 0 2024-04-12T20:55:54.764 INFO:tasks.workunit.client.0.smithi018.stdout:7/19: getdents d1 0 2024-04-12T20:55:54.766 INFO:tasks.workunit.client.0.smithi018.stdout:3/70: symlink d2/d6/l13 0 2024-04-12T20:55:54.766 INFO:tasks.workunit.client.0.smithi018.stdout:8/22: symlink l4 0 2024-04-12T20:55:54.771 INFO:tasks.workunit.client.0.smithi018.stdout:6/70: symlink d5/df/d17/l1a 0 2024-04-12T20:55:54.773 INFO:tasks.workunit.client.0.smithi018.stdout:8/23: creat f5 x:0 0 0 2024-04-12T20:55:54.775 INFO:tasks.workunit.client.0.smithi018.stdout:7/20: rename d1/d3 to d1/d6/d8 0 2024-04-12T20:55:54.775 INFO:tasks.workunit.client.0.smithi018.stdout:5/15: sync 2024-04-12T20:55:54.775 INFO:tasks.workunit.client.0.smithi018.stdout:5/16: dread - no filename 2024-04-12T20:55:54.775 INFO:tasks.workunit.client.0.smithi018.stdout:1/19: sync 2024-04-12T20:55:54.779 INFO:tasks.workunit.client.0.smithi018.stdout:3/71: unlink d2/d6/f8 0 2024-04-12T20:55:54.781 INFO:tasks.workunit.client.0.smithi018.stdout:6/71: dread d5/f10 [0,4194304] 0 2024-04-12T20:55:54.783 INFO:tasks.workunit.client.0.smithi018.stdout:8/24: creat f6 x:0 0 0 2024-04-12T20:55:54.784 INFO:tasks.workunit.client.0.smithi018.stdout:3/72: dread - d2/d6/f12 zero size 2024-04-12T20:55:54.787 INFO:tasks.workunit.client.0.smithi018.stdout:9/52: dwrite d3/d6/d9/fa [0,4194304] 0 2024-04-12T20:55:54.788 INFO:tasks.workunit.client.0.smithi018.stdout:7/21: mknod d1/c9 0 2024-04-12T20:55:54.788 INFO:tasks.workunit.client.0.smithi018.stdout:2/31: dwrite f3 [0,4194304] 0 2024-04-12T20:55:54.792 INFO:tasks.workunit.client.0.smithi018.stdout:1/20: creat d3/f7 x:0 0 0 2024-04-12T20:55:54.795 INFO:tasks.workunit.client.0.smithi018.stdout:1/21: write d3/f7 [258807,92575] 0 2024-04-12T20:55:54.796 INFO:tasks.workunit.client.0.smithi018.stdout:8/25: creat f7 x:0 0 0 2024-04-12T20:55:54.798 INFO:tasks.workunit.client.0.smithi018.stdout:5/17: rename d0/d2/c5 to d0/d2/c6 0 2024-04-12T20:55:54.799 INFO:tasks.workunit.client.0.smithi018.stdout:5/18: write - no filename 2024-04-12T20:55:54.808 INFO:tasks.workunit.client.0.smithi018.stdout:7/22: mknod d1/d6/d8/ca 0 2024-04-12T20:55:54.808 INFO:tasks.workunit.client.0.smithi018.stdout:7/23: readlink - no filename 2024-04-12T20:55:54.815 INFO:tasks.workunit.client.0.smithi018.stdout:8/26: rename l3 to l8 0 2024-04-12T20:55:54.817 INFO:tasks.workunit.client.0.smithi018.stdout:5/19: mkdir d0/d7 0 2024-04-12T20:55:54.817 INFO:tasks.workunit.client.0.smithi018.stdout:8/27: write f6 [114371,42734] 0 2024-04-12T20:55:54.819 INFO:tasks.workunit.client.0.smithi018.stdout:8/28: chown l2 6017 1 2024-04-12T20:55:54.821 INFO:tasks.workunit.client.0.smithi018.stdout:7/24: unlink c0 0 2024-04-12T20:55:54.824 INFO:tasks.workunit.client.0.smithi018.stdout:7/25: dread d1/f2 [0,4194304] 0 2024-04-12T20:55:54.825 INFO:tasks.workunit.client.0.smithi018.stdout:3/73: creat d2/f14 x:0 0 0 2024-04-12T20:55:54.826 INFO:tasks.workunit.client.0.smithi018.stdout:5/20: mkdir d0/d8 0 2024-04-12T20:55:54.827 INFO:tasks.workunit.client.0.smithi018.stdout:8/29: creat f9 x:0 0 0 2024-04-12T20:55:54.829 INFO:tasks.workunit.client.0.smithi018.stdout:9/53: getdents d3/d6/d9/dc/d13 0 2024-04-12T20:55:54.831 INFO:tasks.workunit.client.0.smithi018.stdout:9/54: read d3/d6/f10 [3372028,36210] 0 2024-04-12T20:55:54.832 INFO:tasks.workunit.client.0.smithi018.stdout:7/26: mkdir d1/db 0 2024-04-12T20:55:54.832 INFO:tasks.workunit.client.0.smithi018.stdout:9/55: readlink d3/l4 0 2024-04-12T20:55:54.834 INFO:tasks.workunit.client.0.smithi018.stdout:7/27: stat d1/c9 0 2024-04-12T20:55:54.834 INFO:tasks.workunit.client.0.smithi018.stdout:9/56: write d3/d6/f10 [3252407,127653] 0 2024-04-12T20:55:54.836 INFO:tasks.workunit.client.0.smithi018.stdout:5/21: mkdir d0/d2/d9 0 2024-04-12T20:55:54.836 INFO:tasks.workunit.client.0.smithi018.stdout:5/22: truncate - no filename 2024-04-12T20:55:54.836 INFO:tasks.workunit.client.0.smithi018.stdout:5/23: dread - no filename 2024-04-12T20:55:54.838 INFO:tasks.workunit.client.0.smithi018.stdout:8/30: creat fa x:0 0 0 2024-04-12T20:55:54.840 INFO:tasks.workunit.client.0.smithi018.stdout:1/22: link c1 d3/c8 0 2024-04-12T20:55:54.840 INFO:tasks.workunit.client.0.smithi018.stdout:1/23: readlink - no filename 2024-04-12T20:55:54.843 INFO:tasks.workunit.client.0.smithi018.stdout:1/24: truncate d3/f7 910770 0 2024-04-12T20:55:54.844 INFO:tasks.workunit.client.0.smithi018.stdout:7/28: write d1/f2 [1161094,30831] 0 2024-04-12T20:55:54.848 INFO:tasks.workunit.client.0.smithi018.stdout:7/29: chown d1/f7 3249737 1 2024-04-12T20:55:54.848 INFO:tasks.workunit.client.0.smithi018.stdout:4/28: sync 2024-04-12T20:55:54.848 INFO:tasks.workunit.client.0.smithi018.stdout:4/29: readlink - no filename 2024-04-12T20:55:54.849 INFO:tasks.workunit.client.0.smithi018.stdout:8/31: mkdir db 0 2024-04-12T20:55:54.849 INFO:tasks.workunit.client.0.smithi018.stdout:4/30: chown . 835078 1 2024-04-12T20:55:54.849 INFO:tasks.workunit.client.0.smithi018.stdout:4/31: dread - no filename 2024-04-12T20:55:54.849 INFO:tasks.workunit.client.0.smithi018.stdout:4/32: truncate - no filename 2024-04-12T20:55:54.849 INFO:tasks.workunit.client.0.smithi018.stdout:4/33: readlink - no filename 2024-04-12T20:55:54.849 INFO:tasks.workunit.client.0.smithi018.stdout:4/34: chown . 31 1 2024-04-12T20:55:54.850 INFO:tasks.workunit.client.0.smithi018.stdout:4/35: chown . 16855 1 2024-04-12T20:55:54.850 INFO:tasks.workunit.client.0.smithi018.stdout:4/36: read - no filename 2024-04-12T20:55:54.850 INFO:tasks.workunit.client.0.smithi018.stdout:4/37: chown . 5164 1 2024-04-12T20:55:54.850 INFO:tasks.workunit.client.0.smithi018.stdout:4/38: dread - no filename 2024-04-12T20:55:54.854 INFO:tasks.workunit.client.0.smithi018.stdout:3/74: rmdir d2/d6/d11 0 2024-04-12T20:55:54.854 INFO:tasks.workunit.client.0.smithi018.stdout:9/57: dread d3/d6/d9/fa [0,4194304] 0 2024-04-12T20:55:54.854 INFO:tasks.workunit.client.0.smithi018.stdout:3/75: dread - d2/d6/f9 zero size 2024-04-12T20:55:54.856 INFO:tasks.workunit.client.0.smithi018.stdout:4/39: mkdir d3 0 2024-04-12T20:55:54.856 INFO:tasks.workunit.client.0.smithi018.stdout:4/40: fdatasync - no filename 2024-04-12T20:55:54.858 INFO:tasks.workunit.client.0.smithi018.stdout:5/24: rmdir d0/d7 0 2024-04-12T20:55:54.859 INFO:tasks.workunit.client.0.smithi018.stdout:8/32: mkdir db/dc 0 2024-04-12T20:55:54.859 INFO:tasks.workunit.client.0.smithi018.stdout:8/33: readlink l2 0 2024-04-12T20:55:54.861 INFO:tasks.workunit.client.0.smithi018.stdout:3/76: mkdir d2/d6/d15 0 2024-04-12T20:55:54.863 INFO:tasks.workunit.client.0.smithi018.stdout:5/25: rename d0/d2/c3 to d0/d2/ca 0 2024-04-12T20:55:54.863 INFO:tasks.workunit.client.0.smithi018.stdout:5/26: dwrite - no filename 2024-04-12T20:55:54.964 INFO:tasks.workunit.client.0.smithi018.stdout:4/41: creat d3/f4 x:0 0 0 2024-04-12T20:55:54.972 INFO:tasks.workunit.client.0.smithi018.stdout:1/25: dwrite d3/f4 [0,4194304] 0 2024-04-12T20:55:54.979 INFO:tasks.workunit.client.0.smithi018.stdout:0/65: getdents da/d15 0 2024-04-12T20:55:54.983 INFO:tasks.workunit.client.0.smithi018.stdout:2/32: write f4 [183577,65075] 0 2024-04-12T20:55:54.989 INFO:tasks.workunit.client.0.smithi018.stdout:0/66: truncate da/fc 4547680 0 2024-04-12T20:55:54.989 INFO:tasks.workunit.client.0.smithi018.stdout:3/77: symlink d2/l16 0 2024-04-12T20:55:54.990 INFO:tasks.workunit.client.0.smithi018.stdout:2/33: fdatasync f3 0 2024-04-12T20:55:54.990 INFO:tasks.workunit.client.0.smithi018.stdout:4/42: rename d3/f4 to d3/f5 0 2024-04-12T20:55:54.993 INFO:tasks.workunit.client.0.smithi018.stdout:5/27: creat d0/d2/d9/fb x:0 0 0 2024-04-12T20:55:54.993 INFO:tasks.workunit.client.0.smithi018.stdout:1/26: stat c1 0 2024-04-12T20:55:54.998 INFO:tasks.workunit.client.0.smithi018.stdout:1/27: mkdir d3/d9 0 2024-04-12T20:55:54.999 INFO:tasks.workunit.client.0.smithi018.stdout:2/34: write f3 [367794,47356] 0 2024-04-12T20:55:54.999 INFO:tasks.workunit.client.0.smithi018.stdout:4/43: write d3/f5 [85569,119330] 0 2024-04-12T20:55:55.000 INFO:tasks.workunit.client.0.smithi018.stdout:5/28: mkdir d0/dc 0 2024-04-12T20:55:55.002 INFO:tasks.workunit.client.0.smithi018.stdout:2/35: chown d5/d7 6335056 1 2024-04-12T20:55:55.005 INFO:tasks.workunit.client.0.smithi018.stdout:4/44: mknod d3/c6 0 2024-04-12T20:55:55.007 INFO:tasks.workunit.client.0.smithi018.stdout:5/29: mknod d0/cd 0 2024-04-12T20:55:55.009 INFO:tasks.workunit.client.0.smithi018.stdout:4/45: mknod d3/c7 0 2024-04-12T20:55:55.009 INFO:tasks.workunit.client.0.smithi018.stdout:2/36: creat d5/d7/fa x:0 0 0 2024-04-12T20:55:55.010 INFO:tasks.workunit.client.0.smithi018.stdout:6/72: dwrite d5/f7 [0,4194304] 0 2024-04-12T20:55:55.012 INFO:tasks.workunit.client.0.smithi018.stdout:4/46: chown d3/c7 725157 1 2024-04-12T20:55:55.014 INFO:tasks.workunit.client.0.smithi018.stdout:6/73: mkdir d5/d6/d1b 0 2024-04-12T20:55:55.015 INFO:tasks.workunit.client.0.smithi018.stdout:5/30: creat d0/d2/d9/fe x:0 0 0 2024-04-12T20:55:55.015 INFO:tasks.workunit.client.0.smithi018.stdout:2/37: fsync f3 0 2024-04-12T20:55:55.016 INFO:tasks.workunit.client.0.smithi018.stdout:0/67: dread da/fe [0,4194304] 0 2024-04-12T20:55:55.017 INFO:tasks.workunit.client.0.smithi018.stdout:2/38: fsync f4 0 2024-04-12T20:55:55.018 INFO:tasks.workunit.client.0.smithi018.stdout:4/47: unlink d3/c6 0 2024-04-12T20:55:55.019 INFO:tasks.workunit.client.0.smithi018.stdout:0/68: readlink da/ld 0 2024-04-12T20:55:55.019 INFO:tasks.workunit.client.0.smithi018.stdout:5/31: chown d0/d8 211016420 1 2024-04-12T20:55:55.021 INFO:tasks.workunit.client.0.smithi018.stdout:3/78: dread f1 [0,4194304] 0 2024-04-12T20:55:55.022 INFO:tasks.workunit.client.0.smithi018.stdout:4/48: read d3/f5 [8822,42388] 0 2024-04-12T20:55:55.024 INFO:tasks.workunit.client.0.smithi018.stdout:5/32: creat d0/ff x:0 0 0 2024-04-12T20:55:55.024 INFO:tasks.workunit.client.0.smithi018.stdout:2/39: mknod d5/d8/cb 0 2024-04-12T20:55:55.026 INFO:tasks.workunit.client.0.smithi018.stdout:0/69: creat da/d15/f18 x:0 0 0 2024-04-12T20:55:55.028 INFO:tasks.workunit.client.0.smithi018.stdout:5/33: write d0/d2/d9/fe [867072,74607] 0 2024-04-12T20:55:55.028 INFO:tasks.workunit.client.0.smithi018.stdout:4/49: mknod d3/c8 0 2024-04-12T20:55:55.032 INFO:tasks.workunit.client.0.smithi018.stdout:0/70: unlink da/fe 0 2024-04-12T20:55:55.034 INFO:tasks.workunit.client.0.smithi018.stdout:6/74: dread d5/f7 [0,4194304] 0 2024-04-12T20:55:55.035 INFO:tasks.workunit.client.0.smithi018.stdout:3/79: rmdir d2/d6/d15 0 2024-04-12T20:55:55.035 INFO:tasks.workunit.client.0.smithi018.stdout:4/50: read d3/f5 [73113,86633] 0 2024-04-12T20:55:55.038 INFO:tasks.workunit.client.0.smithi018.stdout:6/75: write d5/d6/f12 [3191002,1651] 0 2024-04-12T20:55:55.038 INFO:tasks.workunit.client.0.smithi018.stdout:2/40: getdents d5 0 2024-04-12T20:55:55.040 INFO:tasks.workunit.client.0.smithi018.stdout:4/51: mkdir d3/d9 0 2024-04-12T20:55:55.045 INFO:tasks.workunit.client.0.smithi018.stdout:0/71: creat da/d15/d17/f19 x:0 0 0 2024-04-12T20:55:55.054 INFO:tasks.workunit.client.0.smithi018.stdout:4/52: creat d3/fa x:0 0 0 2024-04-12T20:55:55.062 INFO:tasks.workunit.client.0.smithi018.stdout:6/76: dread d5/d6/f12 [0,4194304] 0 2024-04-12T20:55:55.063 INFO:tasks.workunit.client.0.smithi018.stdout:5/34: dwrite d0/ff [0,4194304] 0 2024-04-12T20:55:55.069 INFO:tasks.workunit.client.0.smithi018.stdout:4/53: getdents d3/d9 0 2024-04-12T20:55:55.071 INFO:tasks.workunit.client.0.smithi018.stdout:5/35: read d0/ff [950623,78535] 0 2024-04-12T20:55:55.075 INFO:tasks.workunit.client.0.smithi018.stdout:4/54: mkdir d3/db 0 2024-04-12T20:55:55.080 INFO:tasks.workunit.client.0.smithi018.stdout:2/41: dwrite d5/d7/fa [0,4194304] 0 2024-04-12T20:55:55.080 INFO:tasks.workunit.client.0.smithi018.stdout:5/36: dread - d0/d2/d9/fb zero size 2024-04-12T20:55:55.082 INFO:tasks.workunit.client.0.smithi018.stdout:6/77: write d5/d6/f11 [81619,123945] 0 2024-04-12T20:55:55.085 INFO:tasks.workunit.client.0.smithi018.stdout:5/37: dread - d0/d2/d9/fb zero size 2024-04-12T20:55:55.090 INFO:tasks.workunit.client.0.smithi018.stdout:0/72: dwrite da/d15/f18 [0,4194304] 0 2024-04-12T20:55:55.093 INFO:tasks.workunit.client.0.smithi018.stdout:6/78: creat d5/d15/f1c x:0 0 0 2024-04-12T20:55:55.100 INFO:tasks.workunit.client.0.smithi018.stdout:6/79: rename d5/d6/la to d5/d6/l1d 0 2024-04-12T20:55:55.107 INFO:tasks.workunit.client.0.smithi018.stdout:4/55: getdents d3 0 2024-04-12T20:55:55.108 INFO:tasks.workunit.client.0.smithi018.stdout:4/56: readlink - no filename 2024-04-12T20:55:55.108 INFO:tasks.workunit.client.0.smithi018.stdout:6/80: mkdir d5/df/d1e 0 2024-04-12T20:55:55.110 INFO:tasks.workunit.client.0.smithi018.stdout:6/81: chown d5/df 1294346 1 2024-04-12T20:55:55.115 INFO:tasks.workunit.client.0.smithi018.stdout:6/82: rmdir d5/d15 39 2024-04-12T20:55:55.122 INFO:tasks.workunit.client.0.smithi018.stdout:6/83: creat d5/d6/d1b/f1f x:0 0 0 2024-04-12T20:55:55.126 INFO:tasks.workunit.client.0.smithi018.stdout:2/42: dwrite f4 [0,4194304] 0 2024-04-12T20:55:55.128 INFO:tasks.workunit.client.0.smithi018.stdout:5/38: dwrite d0/d2/d9/fb [0,4194304] 0 2024-04-12T20:55:55.131 INFO:tasks.workunit.client.0.smithi018.stdout:0/73: fdatasync da/d15/f18 0 2024-04-12T20:55:55.135 INFO:tasks.workunit.client.0.smithi018.stdout:6/84: dread - d5/d15/f1c zero size 2024-04-12T20:55:55.137 INFO:tasks.workunit.client.0.smithi018.stdout:2/43: symlink d5/d7/lc 0 2024-04-12T20:55:55.141 INFO:tasks.workunit.client.0.smithi018.stdout:6/85: read d5/d6/f11 [482099,45411] 0 2024-04-12T20:55:55.142 INFO:tasks.workunit.client.0.smithi018.stdout:5/39: mkdir d0/dc/d10 0 2024-04-12T20:55:55.145 INFO:tasks.workunit.client.0.smithi018.stdout:0/74: mknod da/d15/c1a 0 2024-04-12T20:55:55.152 INFO:tasks.workunit.client.0.smithi018.stdout:4/57: dwrite d3/f5 [0,4194304] 0 2024-04-12T20:55:55.156 INFO:tasks.workunit.client.0.smithi018.stdout:4/58: write d3/fa [510767,848] 0 2024-04-12T20:55:55.180 INFO:tasks.workunit.client.0.smithi018.stdout:4/59: write d3/f5 [4374719,65349] 0 2024-04-12T20:55:55.181 INFO:tasks.workunit.client.0.smithi018.stdout:2/44: truncate f4 3306595 0 2024-04-12T20:55:55.184 INFO:tasks.workunit.client.0.smithi018.stdout:2/45: readlink l1 0 2024-04-12T20:55:55.188 INFO:tasks.workunit.client.0.smithi018.stdout:0/75: dread da/d15/f18 [0,4194304] 0 2024-04-12T20:55:55.189 INFO:tasks.workunit.client.0.smithi018.stdout:6/86: dwrite d5/df/f18 [0,4194304] 0 2024-04-12T20:55:55.193 INFO:tasks.workunit.client.0.smithi018.stdout:4/60: mknod d3/d9/cc 0 2024-04-12T20:55:55.196 INFO:tasks.workunit.client.0.smithi018.stdout:5/40: dwrite d0/d2/d9/fe [0,4194304] 0 2024-04-12T20:55:55.200 INFO:tasks.workunit.client.0.smithi018.stdout:4/61: fsync d3/f5 0 2024-04-12T20:55:55.204 INFO:tasks.workunit.client.0.smithi018.stdout:0/76: mknod da/d15/d17/c1b 0 2024-04-12T20:55:55.205 INFO:tasks.workunit.client.0.smithi018.stdout:0/77: readlink l7 0 2024-04-12T20:55:55.207 INFO:tasks.workunit.client.0.smithi018.stdout:5/41: mkdir d0/d2/d11 0 2024-04-12T20:55:55.208 INFO:tasks.workunit.client.0.smithi018.stdout:0/78: dread - da/d15/d17/f19 zero size 2024-04-12T20:55:55.215 INFO:tasks.workunit.client.0.smithi018.stdout:4/62: creat d3/d9/fd x:0 0 0 2024-04-12T20:55:55.218 INFO:tasks.workunit.client.0.smithi018.stdout:6/87: creat d5/f20 x:0 0 0 2024-04-12T20:55:55.229 INFO:tasks.workunit.client.0.smithi018.stdout:0/79: link da/l14 da/d15/d17/l1c 0 2024-04-12T20:55:55.232 INFO:tasks.workunit.client.0.smithi018.stdout:2/46: truncate d5/d7/fa 3049393 0 2024-04-12T20:55:55.235 INFO:tasks.workunit.client.0.smithi018.stdout:5/42: dread d0/d2/d9/fb [0,4194304] 0 2024-04-12T20:55:55.237 INFO:tasks.workunit.client.0.smithi018.stdout:0/80: creat da/f1d x:0 0 0 2024-04-12T20:55:55.238 INFO:tasks.workunit.client.0.smithi018.stdout:0/81: fdatasync da/fc 0 2024-04-12T20:55:55.239 INFO:tasks.workunit.client.0.smithi018.stdout:4/63: truncate d3/fa 357696 0 2024-04-12T20:55:55.239 INFO:tasks.workunit.client.0.smithi018.stdout:5/43: write d0/d2/d9/fe [3967209,126737] 0 2024-04-12T20:55:55.242 INFO:tasks.workunit.client.0.smithi018.stdout:2/47: fdatasync d5/d7/fa 0 2024-04-12T20:55:55.244 INFO:tasks.workunit.client.0.smithi018.stdout:2/48: chown f4 52611 1 2024-04-12T20:55:55.244 INFO:tasks.workunit.client.0.smithi018.stdout:4/64: dread - d3/d9/fd zero size 2024-04-12T20:55:55.244 INFO:tasks.workunit.client.0.smithi018.stdout:6/88: dread d5/df/f18 [0,4194304] 0 2024-04-12T20:55:55.247 INFO:tasks.workunit.client.0.smithi018.stdout:4/65: write d3/f5 [2666665,21261] 0 2024-04-12T20:55:55.249 INFO:tasks.workunit.client.0.smithi018.stdout:5/44: creat d0/d8/f12 x:0 0 0 2024-04-12T20:55:55.258 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:55:55 smithi018 ceph-mon[28799]: pgmap v30: 65 pgs: 65 active+clean; 192 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 2.7 MiB/s wr, 183 op/s 2024-04-12T20:55:55.263 INFO:tasks.workunit.client.0.smithi018.stdout:0/82: dread da/d15/f18 [0,4194304] 0 2024-04-12T20:55:55.263 INFO:tasks.workunit.client.0.smithi018.stdout:2/49: creat d5/d7/d9/fd x:0 0 0 2024-04-12T20:55:55.264 INFO:tasks.workunit.client.0.smithi018.stdout:5/45: creat d0/d2/d9/f13 x:0 0 0 2024-04-12T20:55:55.271 INFO:tasks.workunit.client.0.smithi018.stdout:0/83: mknod da/d15/d17/c1e 0 2024-04-12T20:55:55.278 INFO:tasks.workunit.client.0.smithi018.stdout:2/50: rename d5/d7/fa to d5/d7/d9/fe 0 2024-04-12T20:55:55.278 INFO:tasks.workunit.client.0.smithi018.stdout:5/46: symlink d0/d2/d11/l14 0 2024-04-12T20:55:55.278 INFO:tasks.workunit.client.0.smithi018.stdout:6/89: getdents d5/df/d17 0 2024-04-12T20:55:55.280 INFO:tasks.workunit.client.0.smithi018.stdout:0/84: mknod da/d15/d17/c1f 0 2024-04-12T20:55:55.280 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:55:55 smithi186 ceph-mon[36597]: pgmap v30: 65 pgs: 65 active+clean; 192 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 2.7 MiB/s wr, 183 op/s 2024-04-12T20:55:55.280 INFO:tasks.workunit.client.0.smithi018.stdout:6/90: truncate d5/f20 416321 0 2024-04-12T20:55:55.283 INFO:tasks.workunit.client.0.smithi018.stdout:5/47: creat d0/dc/f15 x:0 0 0 2024-04-12T20:55:55.287 INFO:tasks.workunit.client.0.smithi018.stdout:0/85: dread da/f13 [0,4194304] 0 2024-04-12T20:55:55.289 INFO:tasks.workunit.client.0.smithi018.stdout:6/91: mknod d5/d6/d1b/c21 0 2024-04-12T20:55:55.292 INFO:tasks.workunit.client.0.smithi018.stdout:2/51: rmdir d5/d7/d9 39 2024-04-12T20:55:55.293 INFO:tasks.workunit.client.0.smithi018.stdout:6/92: write d5/d6/d1b/f1f [81368,9033] 0 2024-04-12T20:55:55.298 INFO:tasks.workunit.client.0.smithi018.stdout:4/66: truncate d3/fa 1284155 0 2024-04-12T20:55:55.298 INFO:tasks.workunit.client.0.smithi018.stdout:2/52: symlink d5/d7/d9/lf 0 2024-04-12T20:55:55.302 INFO:tasks.workunit.client.0.smithi018.stdout:6/93: mkdir d5/df/d1e/d22 0 2024-04-12T20:55:55.304 INFO:tasks.workunit.client.0.smithi018.stdout:2/53: mkdir d5/d8/d10 0 2024-04-12T20:55:55.308 INFO:tasks.workunit.client.0.smithi018.stdout:6/94: mkdir d5/df/d23 0 2024-04-12T20:55:55.310 INFO:tasks.workunit.client.0.smithi018.stdout:4/67: link d3/c7 d3/d9/ce 0 2024-04-12T20:55:55.311 INFO:tasks.workunit.client.0.smithi018.stdout:2/54: rename d5 to d5/d8/d10/d11 22 2024-04-12T20:55:55.312 INFO:tasks.workunit.client.0.smithi018.stdout:0/86: dwrite da/d15/d17/f19 [0,4194304] 0 2024-04-12T20:55:55.314 INFO:tasks.workunit.client.0.smithi018.stdout:0/87: dread - da/f1d zero size 2024-04-12T20:55:55.315 INFO:tasks.workunit.client.0.smithi018.stdout:6/95: unlink d5/d6/lc 0 2024-04-12T20:55:55.319 INFO:tasks.workunit.client.0.smithi018.stdout:4/68: symlink d3/lf 0 2024-04-12T20:55:55.319 INFO:tasks.workunit.client.0.smithi018.stdout:0/88: creat da/d15/f20 x:0 0 0 2024-04-12T20:55:55.319 INFO:tasks.workunit.client.0.smithi018.stdout:4/69: stat d3 0 2024-04-12T20:55:55.321 INFO:tasks.workunit.client.0.smithi018.stdout:0/89: read - da/f1d zero size 2024-04-12T20:55:55.321 INFO:tasks.workunit.client.0.smithi018.stdout:6/96: mknod d5/d15/c24 0 2024-04-12T20:55:55.326 INFO:tasks.workunit.client.0.smithi018.stdout:4/70: write d3/f5 [935702,81931] 0 2024-04-12T20:55:55.327 INFO:tasks.workunit.client.0.smithi018.stdout:0/90: mkdir da/d15/d17/d21 0 2024-04-12T20:55:55.329 INFO:tasks.workunit.client.0.smithi018.stdout:4/71: chown d3/f5 278 1 2024-04-12T20:55:55.329 INFO:tasks.workunit.client.0.smithi018.stdout:2/55: dread f4 [0,4194304] 0 2024-04-12T20:55:55.331 INFO:tasks.workunit.client.0.smithi018.stdout:2/56: write f3 [1433481,114403] 0 2024-04-12T20:55:55.331 INFO:tasks.workunit.client.0.smithi018.stdout:0/91: creat da/f22 x:0 0 0 2024-04-12T20:55:55.331 INFO:tasks.workunit.client.0.smithi018.stdout:6/97: rename d5/d6/f12 to d5/df/d19/f25 0 2024-04-12T20:55:55.332 INFO:tasks.workunit.client.0.smithi018.stdout:4/72: mkdir d3/d10 0 2024-04-12T20:55:55.335 INFO:tasks.workunit.client.0.smithi018.stdout:4/73: fdatasync d3/d9/fd 0 2024-04-12T20:55:55.335 INFO:tasks.workunit.client.0.smithi018.stdout:0/92: write da/fc [3947453,100918] 0 2024-04-12T20:55:55.338 INFO:tasks.workunit.client.0.smithi018.stdout:6/98: symlink d5/d15/l26 0 2024-04-12T20:55:55.342 INFO:tasks.workunit.client.0.smithi018.stdout:0/93: write da/d15/d17/f19 [4096175,3434] 0 2024-04-12T20:55:55.344 INFO:tasks.workunit.client.0.smithi018.stdout:4/74: link d3/d9/cc d3/d9/c11 0 2024-04-12T20:55:55.345 INFO:tasks.workunit.client.0.smithi018.stdout:0/94: fsync da/fc 0 2024-04-12T20:55:55.345 INFO:tasks.workunit.client.0.smithi018.stdout:6/99: getdents d5/df/d1e/d22 0 2024-04-12T20:55:55.350 INFO:tasks.workunit.client.0.smithi018.stdout:6/100: write d5/df/d19/f25 [4994224,57848] 0 2024-04-12T20:55:55.351 INFO:tasks.workunit.client.0.smithi018.stdout:2/57: dread f3 [0,4194304] 0 2024-04-12T20:55:55.351 INFO:tasks.workunit.client.0.smithi018.stdout:0/95: mknod da/c23 0 2024-04-12T20:55:55.352 INFO:tasks.workunit.client.0.smithi018.stdout:2/58: truncate f3 4711750 0 2024-04-12T20:55:55.353 INFO:tasks.workunit.client.0.smithi018.stdout:4/75: rename d3/fa to d3/db/f12 0 2024-04-12T20:55:55.356 INFO:tasks.workunit.client.0.smithi018.stdout:6/101: write d5/d6/d1b/f1f [918342,85221] 0 2024-04-12T20:55:55.358 INFO:tasks.workunit.client.0.smithi018.stdout:2/59: rename d5/d7/d9 to d5/d12 0 2024-04-12T20:55:55.364 INFO:tasks.workunit.client.0.smithi018.stdout:4/76: link d3/d9/cc d3/c13 0 2024-04-12T20:55:55.368 INFO:tasks.workunit.client.0.smithi018.stdout:4/77: creat d3/f14 x:0 0 0 2024-04-12T20:55:55.370 INFO:tasks.workunit.client.0.smithi018.stdout:0/96: dread da/fc [0,4194304] 0 2024-04-12T20:55:55.376 INFO:tasks.workunit.client.0.smithi018.stdout:0/97: symlink da/d15/l24 0 2024-04-12T20:55:55.378 INFO:tasks.workunit.client.0.smithi018.stdout:2/60: dread f4 [0,4194304] 0 2024-04-12T20:55:55.382 INFO:tasks.workunit.client.0.smithi018.stdout:2/61: symlink d5/l13 0 2024-04-12T20:55:55.392 INFO:tasks.workunit.client.0.smithi018.stdout:2/62: mkdir d5/d7/d14 0 2024-04-12T20:55:55.392 INFO:tasks.workunit.client.0.smithi018.stdout:4/78: read d3/db/f12 [884640,15510] 0 2024-04-12T20:55:55.394 INFO:tasks.workunit.client.0.smithi018.stdout:6/102: dwrite d5/f20 [0,4194304] 0 2024-04-12T20:55:55.397 INFO:tasks.workunit.client.0.smithi018.stdout:4/79: rmdir d3/d9 39 2024-04-12T20:55:55.399 INFO:tasks.workunit.client.0.smithi018.stdout:0/98: dread da/d15/f18 [0,4194304] 0 2024-04-12T20:55:55.400 INFO:tasks.workunit.client.0.smithi018.stdout:2/63: creat d5/d8/d10/f15 x:0 0 0 2024-04-12T20:55:55.410 INFO:tasks.workunit.client.0.smithi018.stdout:0/99: creat da/f25 x:0 0 0 2024-04-12T20:55:55.419 INFO:tasks.workunit.client.0.smithi018.stdout:6/103: dwrite d5/d6/f11 [0,4194304] 0 2024-04-12T20:55:55.421 INFO:tasks.workunit.client.0.smithi018.stdout:6/104: chown d5/df/d17 14614 1 2024-04-12T20:55:55.425 INFO:tasks.workunit.client.0.smithi018.stdout:2/64: dwrite f4 [0,4194304] 0 2024-04-12T20:55:55.431 INFO:tasks.workunit.client.0.smithi018.stdout:2/65: mknod d5/d7/c16 0 2024-04-12T20:55:55.431 INFO:tasks.workunit.client.0.smithi018.stdout:6/105: dread d5/d6/d1b/f1f [0,4194304] 0 2024-04-12T20:55:55.435 INFO:tasks.workunit.client.0.smithi018.stdout:2/66: unlink d5/d12/fd 0 2024-04-12T20:55:55.440 INFO:tasks.workunit.client.0.smithi018.stdout:2/67: mknod d5/d8/c17 0 2024-04-12T20:55:55.441 INFO:tasks.workunit.client.0.smithi018.stdout:2/68: rename d5 to d5/d7/d14/d18 22 2024-04-12T20:55:55.442 INFO:tasks.workunit.client.0.smithi018.stdout:2/69: chown d5/d8 802379 1 2024-04-12T20:55:55.445 INFO:tasks.workunit.client.0.smithi018.stdout:2/70: mkdir d5/d8/d10/d19 0 2024-04-12T20:55:55.465 INFO:tasks.workunit.client.0.smithi018.stdout:4/80: truncate d3/f5 3353317 0 2024-04-12T20:55:55.470 INFO:tasks.workunit.client.0.smithi018.stdout:4/81: write d3/f14 [814536,81786] 0 2024-04-12T20:55:55.473 INFO:tasks.workunit.client.0.smithi018.stdout:0/100: write da/f13 [188815,127351] 0 2024-04-12T20:55:55.476 INFO:tasks.workunit.client.0.smithi018.stdout:0/101: write da/d15/f18 [968701,84244] 0 2024-04-12T20:55:55.477 INFO:tasks.workunit.client.0.smithi018.stdout:4/82: link d3/c13 d3/db/c15 0 2024-04-12T20:55:55.480 INFO:tasks.workunit.client.0.smithi018.stdout:0/102: truncate da/f22 552039 0 2024-04-12T20:55:55.488 INFO:tasks.workunit.client.0.smithi018.stdout:9/58: sync 2024-04-12T20:55:55.488 INFO:tasks.workunit.client.0.smithi018.stdout:7/30: sync 2024-04-12T20:55:55.489 INFO:tasks.workunit.client.0.smithi018.stdout:6/106: dwrite d5/f10 [0,4194304] 0 2024-04-12T20:55:55.492 INFO:tasks.workunit.client.0.smithi018.stdout:4/83: write d3/f5 [2206793,2195] 0 2024-04-12T20:55:55.495 INFO:tasks.workunit.client.0.smithi018.stdout:7/31: mknod d1/cc 0 2024-04-12T20:55:55.497 INFO:tasks.workunit.client.0.smithi018.stdout:4/84: mknod d3/d10/c16 0 2024-04-12T20:55:55.525 INFO:tasks.workunit.client.0.smithi018.stdout:6/107: dread d5/d6/f11 [0,4194304] 0 2024-04-12T20:55:55.525 INFO:tasks.workunit.client.0.smithi018.stdout:9/59: dwrite d3/d6/f12 [4194304,4194304] 0 2024-04-12T20:55:55.528 INFO:tasks.workunit.client.0.smithi018.stdout:6/108: stat d5/d6/d1b 0 2024-04-12T20:55:55.529 INFO:tasks.workunit.client.0.smithi018.stdout:7/32: dwrite d1/f7 [0,4194304] 0 2024-04-12T20:55:55.530 INFO:tasks.workunit.client.0.smithi018.stdout:9/60: symlink d3/l15 0 2024-04-12T20:55:55.534 INFO:tasks.workunit.client.0.smithi018.stdout:7/33: creat d1/fd x:0 0 0 2024-04-12T20:55:55.536 INFO:tasks.workunit.client.0.smithi018.stdout:7/34: dread - d1/fd zero size 2024-04-12T20:55:55.540 INFO:tasks.workunit.client.0.smithi018.stdout:7/35: dread - d1/d6/d8/f4 zero size 2024-04-12T20:55:55.575 INFO:tasks.workunit.client.0.smithi018.stdout:6/109: dwrite d5/fd [0,4194304] 0 2024-04-12T20:55:55.578 INFO:tasks.workunit.client.0.smithi018.stdout:9/61: dwrite d3/d6/d9/fa [4194304,4194304] 0 2024-04-12T20:55:55.584 INFO:tasks.workunit.client.0.smithi018.stdout:9/62: rmdir d3/d6 39 2024-04-12T20:55:55.587 INFO:tasks.workunit.client.0.smithi018.stdout:7/36: dwrite d1/d6/d8/f4 [0,4194304] 0 2024-04-12T20:55:55.590 INFO:tasks.workunit.client.0.smithi018.stdout:7/37: chown d1/fd 2027121 1 2024-04-12T20:55:55.598 INFO:tasks.workunit.client.0.smithi018.stdout:9/63: rename d3/d6/c11 to d3/d6/c16 0 2024-04-12T20:55:55.601 INFO:tasks.workunit.client.0.smithi018.stdout:6/110: dread d5/f7 [0,4194304] 0 2024-04-12T20:55:55.603 INFO:tasks.workunit.client.0.smithi018.stdout:9/64: fdatasync d3/d6/f10 0 2024-04-12T20:55:55.607 INFO:tasks.workunit.client.0.smithi018.stdout:6/111: mkdir d5/df/d19/d27 0 2024-04-12T20:55:55.607 INFO:tasks.workunit.client.0.smithi018.stdout:7/38: dread d1/d6/d8/f4 [0,4194304] 0 2024-04-12T20:55:55.609 INFO:tasks.workunit.client.0.smithi018.stdout:9/65: rmdir d3/d6/d9 39 2024-04-12T20:55:55.612 INFO:tasks.workunit.client.0.smithi018.stdout:7/39: mknod d1/d6/d8/ce 0 2024-04-12T20:55:55.616 INFO:tasks.workunit.client.0.smithi018.stdout:7/40: write d1/fd [132842,73888] 0 2024-04-12T20:55:55.618 INFO:tasks.workunit.client.0.smithi018.stdout:9/66: creat d3/d6/d9/dc/d13/f17 x:0 0 0 2024-04-12T20:55:55.618 INFO:tasks.workunit.client.0.smithi018.stdout:7/41: chown d1/d6/d8/c5 3599358 1 2024-04-12T20:55:55.623 INFO:tasks.workunit.client.0.smithi018.stdout:7/42: symlink d1/db/lf 0 2024-04-12T20:55:55.624 INFO:tasks.workunit.client.0.smithi018.stdout:9/67: mknod d3/d6/db/c18 0 2024-04-12T20:55:55.626 INFO:tasks.workunit.client.0.smithi018.stdout:7/43: truncate d1/fd 700458 0 2024-04-12T20:55:55.626 INFO:tasks.workunit.client.0.smithi018.stdout:6/112: dread d5/df/f18 [0,4194304] 0 2024-04-12T20:55:55.631 INFO:tasks.workunit.client.0.smithi018.stdout:7/44: rename d1/c9 to d1/c10 0 2024-04-12T20:55:55.637 INFO:tasks.workunit.client.0.smithi018.stdout:7/45: symlink d1/d6/d8/l11 0 2024-04-12T20:55:55.642 INFO:tasks.workunit.client.0.smithi018.stdout:7/46: mknod d1/db/c12 0 2024-04-12T20:55:55.651 INFO:tasks.workunit.client.0.smithi018.stdout:6/113: dread d5/fd [0,4194304] 0 2024-04-12T20:55:55.654 INFO:tasks.workunit.client.0.smithi018.stdout:6/114: stat d5/df/d19/f25 0 2024-04-12T20:55:55.656 INFO:tasks.workunit.client.0.smithi018.stdout:9/68: dwrite d3/d6/d9/dc/d13/f17 [0,4194304] 0 2024-04-12T20:55:55.678 INFO:tasks.workunit.client.0.smithi018.stdout:7/47: dwrite d1/fd [0,4194304] 0 2024-04-12T20:55:55.680 INFO:tasks.workunit.client.0.smithi018.stdout:6/115: dread d5/f7 [0,4194304] 0 2024-04-12T20:55:55.684 INFO:tasks.workunit.client.0.smithi018.stdout:6/116: read d5/f20 [2101563,5662] 0 2024-04-12T20:55:55.691 INFO:tasks.workunit.client.0.smithi018.stdout:9/69: dwrite d3/d6/d9/dc/d13/f17 [0,4194304] 0 2024-04-12T20:55:55.697 INFO:tasks.workunit.client.0.smithi018.stdout:9/70: mknod d3/d6/d9/dc/d13/c19 0 2024-04-12T20:55:55.711 INFO:tasks.workunit.client.0.smithi018.stdout:7/48: truncate d1/d6/d8/f4 3730408 0 2024-04-12T20:55:55.713 INFO:tasks.workunit.client.0.smithi018.stdout:9/71: dread d3/d6/d9/fa [4194304,4194304] 0 2024-04-12T20:55:55.714 INFO:tasks.workunit.client.0.smithi018.stdout:6/117: truncate d5/df/d19/f25 3866135 0 2024-04-12T20:55:55.717 INFO:tasks.workunit.client.0.smithi018.stdout:7/49: creat d1/d6/d8/f13 x:0 0 0 2024-04-12T20:55:55.721 INFO:tasks.workunit.client.0.smithi018.stdout:9/72: write d3/d6/d9/fa [5614417,59342] 0 2024-04-12T20:55:55.721 INFO:tasks.workunit.client.0.smithi018.stdout:7/50: creat d1/f14 x:0 0 0 2024-04-12T20:55:55.722 INFO:tasks.workunit.client.0.smithi018.stdout:7/51: readlink d1/db/lf 0 2024-04-12T20:55:55.725 INFO:tasks.workunit.client.0.smithi018.stdout:7/52: fsync d1/fd 0 2024-04-12T20:55:55.728 INFO:tasks.workunit.client.0.smithi018.stdout:7/53: creat d1/f15 x:0 0 0 2024-04-12T20:55:55.730 INFO:tasks.workunit.client.0.smithi018.stdout:6/118: dread d5/df/f18 [0,4194304] 0 2024-04-12T20:55:55.730 INFO:tasks.workunit.client.0.smithi018.stdout:7/54: write d1/f14 [839887,10378] 0 2024-04-12T20:55:55.735 INFO:tasks.workunit.client.0.smithi018.stdout:9/73: dread d3/d6/f10 [0,4194304] 0 2024-04-12T20:55:55.735 INFO:tasks.workunit.client.0.smithi018.stdout:6/119: mknod d5/df/d17/c28 0 2024-04-12T20:55:55.738 INFO:tasks.workunit.client.0.smithi018.stdout:6/120: chown d5/df/l13 50148041 1 2024-04-12T20:55:55.743 INFO:tasks.workunit.client.0.smithi018.stdout:7/55: dread d1/f14 [0,4194304] 0 2024-04-12T20:55:55.755 INFO:tasks.workunit.client.0.smithi018.stdout:5/48: sync 2024-04-12T20:55:55.755 INFO:tasks.workunit.client.0.smithi018.stdout:3/80: sync 2024-04-12T20:55:55.755 INFO:tasks.workunit.client.0.smithi018.stdout:1/28: sync 2024-04-12T20:55:55.755 INFO:tasks.workunit.client.0.smithi018.stdout:8/34: sync 2024-04-12T20:55:55.757 INFO:tasks.workunit.client.0.smithi018.stdout:9/74: truncate d3/d6/d9/fa 639556 0 2024-04-12T20:55:55.758 INFO:tasks.workunit.client.0.smithi018.stdout:6/121: truncate d5/f10 3471513 0 2024-04-12T20:55:55.761 INFO:tasks.workunit.client.0.smithi018.stdout:3/81: write d2/d6/f9 [392462,49342] 0 2024-04-12T20:55:55.761 INFO:tasks.workunit.client.0.smithi018.stdout:9/75: stat d3/d6/d9/dc/d13 0 2024-04-12T20:55:55.762 INFO:tasks.workunit.client.0.smithi018.stdout:5/49: truncate d0/d8/f12 596298 0 2024-04-12T20:55:55.763 INFO:tasks.workunit.client.0.smithi018.stdout:3/82: readlink d2/d6/lb 0 2024-04-12T20:55:55.768 INFO:tasks.workunit.client.0.smithi018.stdout:6/122: creat d5/df/f29 x:0 0 0 2024-04-12T20:55:55.769 INFO:tasks.workunit.client.0.smithi018.stdout:3/83: fdatasync d2/d6/f12 0 2024-04-12T20:55:55.771 INFO:tasks.workunit.client.0.smithi018.stdout:7/56: dwrite d1/f14 [0,4194304] 0 2024-04-12T20:55:55.774 INFO:tasks.workunit.client.0.smithi018.stdout:3/84: write d2/d6/f12 [789448,109481] 0 2024-04-12T20:55:55.776 INFO:tasks.workunit.client.0.smithi018.stdout:6/123: dread d5/d6/d1b/f1f [0,4194304] 0 2024-04-12T20:55:55.786 INFO:tasks.workunit.client.0.smithi018.stdout:9/76: rename d3/d6/d9/dc/d13/c19 to d3/d6/c1a 0 2024-04-12T20:55:55.787 INFO:tasks.workunit.client.0.smithi018.stdout:3/85: symlink d2/l17 0 2024-04-12T20:55:55.787 INFO:tasks.workunit.client.0.smithi018.stdout:9/77: stat d3/d6/f10 0 2024-04-12T20:55:55.788 INFO:tasks.workunit.client.0.smithi018.stdout:3/86: chown d2/d6 424 1 2024-04-12T20:55:55.791 INFO:tasks.workunit.client.0.smithi018.stdout:7/57: dwrite d1/f14 [4194304,4194304] 0 2024-04-12T20:55:55.794 INFO:tasks.workunit.client.0.smithi018.stdout:7/58: mknod d1/d6/d8/c16 0 2024-04-12T20:55:55.799 INFO:tasks.workunit.client.0.smithi018.stdout:9/78: dread d3/d6/f12 [0,4194304] 0 2024-04-12T20:55:55.806 INFO:tasks.workunit.client.0.smithi018.stdout:3/87: dwrite d2/d6/f12 [0,4194304] 0 2024-04-12T20:55:55.810 INFO:tasks.workunit.client.0.smithi018.stdout:3/88: write d2/f14 [899529,62616] 0 2024-04-12T20:55:55.817 INFO:tasks.workunit.client.0.smithi018.stdout:7/59: dwrite d1/d6/d8/f13 [0,4194304] 0 2024-04-12T20:55:56.036 INFO:tasks.workunit.client.0.smithi018.stdout:2/71: sync 2024-04-12T20:55:56.040 INFO:tasks.workunit.client.0.smithi018.stdout:2/72: write f4 [4087050,1621] 0 2024-04-12T20:55:56.050 INFO:tasks.workunit.client.0.smithi018.stdout:2/73: rmdir d5/d7/d14 0 2024-04-12T20:55:56.052 INFO:tasks.workunit.client.0.smithi018.stdout:2/74: rename d5/d8 to d5/d8/d10/d1a 22 2024-04-12T20:55:56.057 INFO:tasks.workunit.client.0.smithi018.stdout:2/75: mknod d5/d7/c1b 0 2024-04-12T20:55:56.059 INFO:tasks.workunit.client.0.smithi018.stdout:2/76: read f3 [1558752,70479] 0 2024-04-12T20:55:56.070 INFO:tasks.workunit.client.0.smithi018.stdout:5/50: truncate d0/ff 1101547 0 2024-04-12T20:55:56.072 INFO:tasks.workunit.client.0.smithi018.stdout:9/79: dread d3/d6/d9/fa [0,4194304] 0 2024-04-12T20:55:56.072 INFO:tasks.workunit.client.0.smithi018.stdout:5/51: chown d0/dc/d10 102221 1 2024-04-12T20:55:56.078 INFO:tasks.workunit.client.0.smithi018.stdout:7/60: rmdir d1/d6/d8 39 2024-04-12T20:55:56.082 INFO:tasks.workunit.client.0.smithi018.stdout:5/52: mknod d0/d8/c16 0 2024-04-12T20:55:56.082 INFO:tasks.workunit.client.0.smithi018.stdout:9/80: mkdir d3/d6/d1b 0 2024-04-12T20:55:56.084 INFO:tasks.workunit.client.0.smithi018.stdout:2/77: dwrite d5/d8/d10/f15 [0,4194304] 0 2024-04-12T20:55:56.085 INFO:tasks.workunit.client.0.smithi018.stdout:9/81: chown c2 103119136 1 2024-04-12T20:55:56.088 INFO:tasks.workunit.client.0.smithi018.stdout:7/61: rename d1/db/lf to d1/d6/l17 0 2024-04-12T20:55:56.093 INFO:tasks.workunit.client.0.smithi018.stdout:7/62: chown d1/f7 918260 1 2024-04-12T20:55:56.098 INFO:tasks.workunit.client.0.smithi018.stdout:2/78: rename l1 to d5/d12/l1c 0 2024-04-12T20:55:56.098 INFO:tasks.workunit.client.0.smithi018.stdout:2/79: readlink d5/d12/lf 0 2024-04-12T20:55:56.100 INFO:tasks.workunit.client.0.smithi018.stdout:5/53: link d0/d8/f12 d0/dc/d10/f17 0 2024-04-12T20:55:56.101 INFO:tasks.workunit.client.0.smithi018.stdout:7/63: rename d1/d6/d8/ce to d1/db/c18 0 2024-04-12T20:55:56.102 INFO:tasks.workunit.client.0.smithi018.stdout:5/54: fdatasync d0/d2/d9/fe 0 2024-04-12T20:55:56.104 INFO:tasks.workunit.client.0.smithi018.stdout:2/80: creat d5/f1d x:0 0 0 2024-04-12T20:55:56.107 INFO:tasks.workunit.client.0.smithi018.stdout:7/64: symlink d1/d6/d8/l19 0 2024-04-12T20:55:56.107 INFO:tasks.workunit.client.0.smithi018.stdout:2/81: truncate d5/d8/d10/f15 4818280 0 2024-04-12T20:55:56.109 INFO:tasks.workunit.client.0.smithi018.stdout:5/55: mkdir d0/d2/d9/d18 0 2024-04-12T20:55:56.109 INFO:tasks.workunit.client.0.smithi018.stdout:7/65: write d1/f7 [4507788,76008] 0 2024-04-12T20:55:56.114 INFO:tasks.workunit.client.0.smithi018.stdout:7/66: symlink d1/l1a 0 2024-04-12T20:55:56.117 INFO:tasks.workunit.client.0.smithi018.stdout:2/82: link f4 d5/d8/f1e 0 2024-04-12T20:55:56.120 INFO:tasks.workunit.client.0.smithi018.stdout:7/67: creat d1/f1b x:0 0 0 2024-04-12T20:55:56.122 INFO:tasks.workunit.client.0.smithi018.stdout:2/83: unlink d5/d8/c17 0 2024-04-12T20:55:56.125 INFO:tasks.workunit.client.0.smithi018.stdout:9/82: dwrite d3/d6/f10 [0,4194304] 0 2024-04-12T20:55:56.129 INFO:tasks.workunit.client.0.smithi018.stdout:2/84: symlink d5/d8/d10/l1f 0 2024-04-12T20:55:56.131 INFO:tasks.workunit.client.0.smithi018.stdout:6/124: dwrite d5/f10 [0,4194304] 0 2024-04-12T20:55:56.132 INFO:tasks.workunit.client.0.smithi018.stdout:9/83: read d3/d6/d9/dc/d13/f17 [2833933,31658] 0 2024-04-12T20:55:56.133 INFO:tasks.workunit.client.0.smithi018.stdout:2/85: chown d5/d8/d10/l1f 185217316 1 2024-04-12T20:55:56.135 INFO:tasks.workunit.client.0.smithi018.stdout:6/125: readlink d5/df/l16 0 2024-04-12T20:55:56.140 INFO:tasks.workunit.client.0.smithi018.stdout:9/84: rename d3/d6/db to d3/d6/d9/dc/d1c 0 2024-04-12T20:55:56.140 INFO:tasks.workunit.client.0.smithi018.stdout:6/126: creat d5/df/f2a x:0 0 0 2024-04-12T20:55:56.141 INFO:tasks.workunit.client.0.smithi018.stdout:7/68: getdents d1 0 2024-04-12T20:55:56.142 INFO:tasks.workunit.client.0.smithi018.stdout:6/127: chown d5/d6/l8 3940 1 2024-04-12T20:55:56.143 INFO:tasks.workunit.client.0.smithi018.stdout:6/128: chown d5/fd 3 1 2024-04-12T20:55:56.147 INFO:tasks.workunit.client.0.smithi018.stdout:7/69: symlink d1/d6/l1c 0 2024-04-12T20:55:56.148 INFO:tasks.workunit.client.0.smithi018.stdout:6/129: mkdir d5/df/d19/d2b 0 2024-04-12T20:55:56.150 INFO:tasks.workunit.client.0.smithi018.stdout:7/70: mknod d1/c1d 0 2024-04-12T20:55:56.150 INFO:tasks.workunit.client.0.smithi018.stdout:2/86: write d5/d8/f1e [4229207,108334] 0 2024-04-12T20:55:56.156 INFO:tasks.workunit.client.0.smithi018.stdout:6/130: mkdir d5/df/d1e/d22/d2c 0 2024-04-12T20:55:56.158 INFO:tasks.workunit.client.0.smithi018.stdout:7/71: symlink d1/db/l1e 0 2024-04-12T20:55:56.162 INFO:tasks.workunit.client.0.smithi018.stdout:2/87: symlink d5/d7/l20 0 2024-04-12T20:55:56.163 INFO:tasks.workunit.client.0.smithi018.stdout:9/85: dwrite d3/d6/d9/fa [0,4194304] 0 2024-04-12T20:55:56.164 INFO:tasks.workunit.client.0.smithi018.stdout:2/88: fsync d5/d8/d10/f15 0 2024-04-12T20:55:56.165 INFO:tasks.workunit.client.0.smithi018.stdout:7/72: mknod d1/d6/d8/c1f 0 2024-04-12T20:55:56.167 INFO:tasks.workunit.client.0.smithi018.stdout:2/89: fdatasync d5/f1d 0 2024-04-12T20:55:56.170 INFO:tasks.workunit.client.0.smithi018.stdout:7/73: unlink d1/f7 0 2024-04-12T20:55:56.170 INFO:tasks.workunit.client.0.smithi018.stdout:6/131: link d5/d6/c14 d5/df/d19/d2b/c2d 0 2024-04-12T20:55:56.173 INFO:tasks.workunit.client.0.smithi018.stdout:2/90: fdatasync f3 0 2024-04-12T20:55:56.178 INFO:tasks.workunit.client.0.smithi018.stdout:7/74: dread d1/f2 [0,4194304] 0 2024-04-12T20:55:56.179 INFO:tasks.workunit.client.0.smithi018.stdout:6/132: mknod d5/df/c2e 0 2024-04-12T20:55:56.183 INFO:tasks.workunit.client.0.smithi018.stdout:2/91: mknod d5/d8/c21 0 2024-04-12T20:55:56.186 INFO:tasks.workunit.client.0.smithi018.stdout:7/75: symlink d1/db/l20 0 2024-04-12T20:55:56.188 INFO:tasks.workunit.client.0.smithi018.stdout:7/76: fsync d1/d6/d8/f13 0 2024-04-12T20:55:56.188 INFO:tasks.workunit.client.0.smithi018.stdout:9/86: getdents d3/d6/d9 0 2024-04-12T20:55:56.190 INFO:tasks.workunit.client.0.smithi018.stdout:2/92: mknod d5/c22 0 2024-04-12T20:55:56.191 INFO:tasks.workunit.client.0.smithi018.stdout:2/93: stat d5/d12/fe 0 2024-04-12T20:55:56.193 INFO:tasks.workunit.client.0.smithi018.stdout:2/94: write d5/f1d [522764,130935] 0 2024-04-12T20:55:56.196 INFO:tasks.workunit.client.0.smithi018.stdout:6/133: mkdir d5/df/d19/d27/d2f 0 2024-04-12T20:55:56.200 INFO:tasks.workunit.client.0.smithi018.stdout:2/95: mknod d5/d8/d10/c23 0 2024-04-12T20:55:56.201 INFO:tasks.workunit.client.0.smithi018.stdout:6/134: fsync d5/f20 0 2024-04-12T20:55:56.203 INFO:tasks.workunit.client.0.smithi018.stdout:7/77: getdents d1/d6 0 2024-04-12T20:55:56.204 INFO:tasks.workunit.client.0.smithi018.stdout:7/78: fsync d1/f15 0 2024-04-12T20:55:56.206 INFO:tasks.workunit.client.0.smithi018.stdout:9/87: dread d3/d6/f10 [0,4194304] 0 2024-04-12T20:55:56.207 INFO:tasks.workunit.client.0.smithi018.stdout:7/79: creat d1/f21 x:0 0 0 2024-04-12T20:55:56.213 INFO:tasks.workunit.client.0.smithi018.stdout:7/80: creat d1/db/f22 x:0 0 0 2024-04-12T20:55:56.223 INFO:tasks.workunit.client.0.smithi018.stdout:2/96: dread d5/d8/d10/f15 [0,4194304] 0 2024-04-12T20:55:56.223 INFO:tasks.workunit.client.0.smithi018.stdout:7/81: truncate d1/d6/d8/f4 479074 0 2024-04-12T20:55:56.224 INFO:tasks.workunit.client.0.smithi018.stdout:7/82: dread - d1/f1b zero size 2024-04-12T20:55:56.227 INFO:tasks.workunit.client.0.smithi018.stdout:6/135: dwrite d5/f7 [4194304,4194304] 0 2024-04-12T20:55:56.229 INFO:tasks.workunit.client.0.smithi018.stdout:2/97: mkdir d5/d8/d10/d19/d24 0 2024-04-12T20:55:56.230 INFO:tasks.workunit.client.0.smithi018.stdout:9/88: dwrite d3/d6/d9/dc/d13/f17 [0,4194304] 0 2024-04-12T20:55:56.232 INFO:tasks.workunit.client.0.smithi018.stdout:9/89: rename d3/d6 to d3/d6/d1b/d1d 22 2024-04-12T20:55:56.233 INFO:tasks.workunit.client.0.smithi018.stdout:6/136: symlink d5/d6/l30 0 2024-04-12T20:55:56.235 INFO:tasks.workunit.client.0.smithi018.stdout:6/137: readlink d5/d6/l8 0 2024-04-12T20:55:56.238 INFO:tasks.workunit.client.0.smithi018.stdout:9/90: mkdir d3/d6/d9/d1e 0 2024-04-12T20:55:56.238 INFO:tasks.workunit.client.0.smithi018.stdout:0/103: sync 2024-04-12T20:55:56.238 INFO:tasks.workunit.client.0.smithi018.stdout:4/85: sync 2024-04-12T20:55:56.241 INFO:tasks.workunit.client.0.smithi018.stdout:0/104: dread - da/f1d zero size 2024-04-12T20:55:56.241 INFO:tasks.workunit.client.0.smithi018.stdout:6/138: fdatasync d5/f20 0 2024-04-12T20:55:56.245 INFO:tasks.workunit.client.0.smithi018.stdout:4/86: write d3/f14 [729506,16755] 0 2024-04-12T20:55:56.250 INFO:tasks.workunit.client.0.smithi018.stdout:0/105: mkdir da/d15/d17/d26 0 2024-04-12T20:55:56.256 INFO:tasks.workunit.client.0.smithi018.stdout:9/91: unlink d3/d6/c16 0 2024-04-12T20:55:56.257 INFO:tasks.workunit.client.0.smithi018.stdout:6/139: rename d5/df/d23 to d5/df/d31 0 2024-04-12T20:55:56.261 INFO:tasks.workunit.client.0.smithi018.stdout:9/92: rmdir d3 39 2024-04-12T20:55:56.261 INFO:tasks.workunit.client.0.smithi018.stdout:7/83: dwrite d1/db/f22 [0,4194304] 0 2024-04-12T20:55:56.262 INFO:tasks.workunit.client.0.smithi018.stdout:7/84: stat d1 0 2024-04-12T20:55:56.266 INFO:tasks.workunit.client.0.smithi018.stdout:7/85: readlink d1/db/l1e 0 2024-04-12T20:55:56.268 INFO:tasks.workunit.client.0.smithi018.stdout:6/140: mknod d5/df/d17/c32 0 2024-04-12T20:55:56.272 INFO:tasks.workunit.client.0.smithi018.stdout:2/98: dwrite d5/d8/d10/f15 [4194304,4194304] 0 2024-04-12T20:55:56.276 INFO:tasks.workunit.client.0.smithi018.stdout:6/141: chown d5/df/l13 10 1 2024-04-12T20:55:56.283 INFO:tasks.workunit.client.0.smithi018.stdout:2/99: unlink d5/d12/fe 0 2024-04-12T20:55:56.285 INFO:tasks.workunit.client.0.smithi018.stdout:7/86: mknod d1/d6/d8/c23 0 2024-04-12T20:55:56.291 INFO:tasks.workunit.client.0.smithi018.stdout:6/142: creat d5/df/d1e/d22/d2c/f33 x:0 0 0 2024-04-12T20:55:56.293 INFO:tasks.workunit.client.0.smithi018.stdout:1/29: sync 2024-04-12T20:55:56.293 INFO:tasks.workunit.client.0.smithi018.stdout:3/89: sync 2024-04-12T20:55:56.293 INFO:tasks.workunit.client.0.smithi018.stdout:5/56: sync 2024-04-12T20:55:56.293 INFO:tasks.workunit.client.0.smithi018.stdout:2/100: read d5/d8/f1e [3351279,73492] 0 2024-04-12T20:55:56.293 INFO:tasks.workunit.client.0.smithi018.stdout:8/35: sync 2024-04-12T20:55:56.299 INFO:tasks.workunit.client.0.smithi018.stdout:0/106: dwrite da/f25 [0,4194304] 0 2024-04-12T20:55:56.300 INFO:tasks.workunit.client.0.smithi018.stdout:7/87: write d1/f1b [935248,64379] 0 2024-04-12T20:55:56.305 INFO:tasks.workunit.client.0.smithi018.stdout:8/36: write f5 [209068,103807] 0 2024-04-12T20:55:56.305 INFO:tasks.workunit.client.0.smithi018.stdout:5/57: read - d0/d2/d9/f13 zero size 2024-04-12T20:55:56.308 INFO:tasks.workunit.client.0.smithi018.stdout:1/30: read d3/f7 [515065,101160] 0 2024-04-12T20:55:56.311 INFO:tasks.workunit.client.0.smithi018.stdout:3/90: mknod d2/c18 0 2024-04-12T20:55:56.312 INFO:tasks.workunit.client.0.smithi018.stdout:2/101: read d5/f1d [111261,8483] 0 2024-04-12T20:55:56.315 INFO:tasks.workunit.client.0.smithi018.stdout:8/37: chown db 8 1 2024-04-12T20:55:56.317 INFO:tasks.workunit.client.0.smithi018.stdout:0/107: chown da/ld 1670249 1 2024-04-12T20:55:56.318 INFO:tasks.workunit.client.0.smithi018.stdout:7/88: readlink d1/d6/d8/l19 0 2024-04-12T20:55:56.319 INFO:tasks.workunit.client.0.smithi018.stdout:9/93: dread d3/d6/d9/dc/d13/f17 [0,4194304] 0 2024-04-12T20:55:56.320 INFO:tasks.workunit.client.0.smithi018.stdout:5/58: read d0/ff [492011,112956] 0 2024-04-12T20:55:56.321 INFO:tasks.workunit.client.0.smithi018.stdout:4/87: dwrite d3/db/f12 [0,4194304] 0 2024-04-12T20:55:56.324 INFO:tasks.workunit.client.0.smithi018.stdout:9/94: chown d3/d6/d1b 600718 1 2024-04-12T20:55:56.330 INFO:tasks.workunit.client.0.smithi018.stdout:8/38: rename l2 to db/dc/ld 0 2024-04-12T20:55:56.331 INFO:tasks.workunit.client.0.smithi018.stdout:0/108: dread da/f13 [0,4194304] 0 2024-04-12T20:55:56.337 INFO:tasks.workunit.client.0.smithi018.stdout:1/31: mkdir d3/d9/da 0 2024-04-12T20:55:56.338 INFO:tasks.workunit.client.0.smithi018.stdout:2/102: mkdir d5/d8/d25 0 2024-04-12T20:55:56.338 INFO:tasks.workunit.client.0.smithi018.stdout:0/109: write da/f22 [1260789,87888] 0 2024-04-12T20:55:56.342 INFO:tasks.workunit.client.0.smithi018.stdout:5/59: mknod d0/dc/c19 0 2024-04-12T20:55:56.344 INFO:tasks.workunit.client.0.smithi018.stdout:8/39: creat db/dc/fe x:0 0 0 2024-04-12T20:55:56.345 INFO:tasks.workunit.client.0.smithi018.stdout:0/110: dread - da/d15/f20 zero size 2024-04-12T20:55:56.346 INFO:tasks.workunit.client.0.smithi018.stdout:5/60: readlink d0/d2/d11/l14 0 2024-04-12T20:55:56.348 INFO:tasks.workunit.client.0.smithi018.stdout:4/88: stat d3/d9/ce 0 2024-04-12T20:55:56.348 INFO:tasks.workunit.client.0.smithi018.stdout:8/40: chown fa 1829 1 2024-04-12T20:55:56.348 INFO:tasks.workunit.client.0.smithi018.stdout:2/103: fsync d5/d8/d10/f15 0 2024-04-12T20:55:56.349 INFO:tasks.workunit.client.0.smithi018.stdout:3/91: dwrite d2/f5 [0,4194304] 0 2024-04-12T20:55:56.350 INFO:tasks.workunit.client.0.smithi018.stdout:6/143: getdents d5/df/d1e/d22/d2c 0 2024-04-12T20:55:56.353 INFO:tasks.workunit.client.0.smithi018.stdout:2/104: fdatasync d5/d8/d10/f15 0 2024-04-12T20:55:56.356 INFO:tasks.workunit.client.0.smithi018.stdout:5/61: chown d0 18485831 1 2024-04-12T20:55:56.357 INFO:tasks.workunit.client.0.smithi018.stdout:2/105: rename d5 to d5/d7/d26 22 2024-04-12T20:55:56.366 INFO:tasks.workunit.client.0.smithi018.stdout:5/62: creat d0/d2/f1a x:0 0 0 2024-04-12T20:55:56.366 INFO:tasks.workunit.client.0.smithi018.stdout:1/32: dread d3/f4 [4194304,4194304] 0 2024-04-12T20:55:56.369 INFO:tasks.workunit.client.0.smithi018.stdout:6/144: rename l2 to d5/df/d1e/d22/d2c/l34 0 2024-04-12T20:55:56.376 INFO:tasks.workunit.client.0.smithi018.stdout:3/92: creat d2/f19 x:0 0 0 2024-04-12T20:55:56.377 INFO:tasks.workunit.client.0.smithi018.stdout:7/89: write d1/d6/d8/f4 [888276,41144] 0 2024-04-12T20:55:56.382 INFO:tasks.workunit.client.0.smithi018.stdout:5/63: read d0/d8/f12 [533577,104548] 0 2024-04-12T20:55:56.384 INFO:tasks.workunit.client.0.smithi018.stdout:4/89: dread d3/db/f12 [0,4194304] 0 2024-04-12T20:55:56.385 INFO:tasks.workunit.client.0.smithi018.stdout:9/95: dread d3/d6/d9/fa [0,4194304] 0 2024-04-12T20:55:56.388 INFO:tasks.workunit.client.0.smithi018.stdout:2/106: truncate d5/d8/f1e 3103481 0 2024-04-12T20:55:56.391 INFO:tasks.workunit.client.0.smithi018.stdout:0/111: dwrite da/fc [0,4194304] 0 2024-04-12T20:55:56.396 INFO:tasks.workunit.client.0.smithi018.stdout:6/145: rename d5/df/d1e/d22/d2c to d5/df/d1e/d35 0 2024-04-12T20:55:56.400 INFO:tasks.workunit.client.0.smithi018.stdout:9/96: write d3/d6/d9/dc/d13/f17 [2923367,15852] 0 2024-04-12T20:55:56.402 INFO:tasks.workunit.client.0.smithi018.stdout:8/41: dwrite fa [0,4194304] 0 2024-04-12T20:55:56.403 INFO:tasks.workunit.client.0.smithi018.stdout:3/93: creat d2/f1a x:0 0 0 2024-04-12T20:55:56.404 INFO:tasks.workunit.client.0.smithi018.stdout:9/97: stat d3/d6/d9/d1e 0 2024-04-12T20:55:56.407 INFO:tasks.workunit.client.0.smithi018.stdout:8/42: fsync f6 0 2024-04-12T20:55:56.407 INFO:tasks.workunit.client.0.smithi018.stdout:6/146: symlink d5/d15/l36 0 2024-04-12T20:55:56.408 INFO:tasks.workunit.client.0.smithi018.stdout:3/94: chown d2 2 1 2024-04-12T20:55:56.410 INFO:tasks.workunit.client.0.smithi018.stdout:0/112: mkdir da/d15/d17/d21/d27 0 2024-04-12T20:55:56.417 INFO:tasks.workunit.client.0.smithi018.stdout:8/43: dread f6 [0,4194304] 0 2024-04-12T20:55:56.420 INFO:tasks.workunit.client.0.smithi018.stdout:4/90: dread d3/db/f12 [0,4194304] 0 2024-04-12T20:55:56.426 INFO:tasks.workunit.client.0.smithi018.stdout:9/98: creat d3/d6/f1f x:0 0 0 2024-04-12T20:55:56.427 INFO:tasks.workunit.client.0.smithi018.stdout:6/147: unlink d5/d6/d1b/f1f 0 2024-04-12T20:55:56.427 INFO:tasks.workunit.client.0.smithi018.stdout:8/44: chown f9 79 1 2024-04-12T20:55:56.427 INFO:tasks.workunit.client.0.smithi018.stdout:1/33: dwrite d3/f4 [0,4194304] 0 2024-04-12T20:55:56.429 INFO:tasks.workunit.client.0.smithi018.stdout:3/95: truncate d2/f4 471918 0 2024-04-12T20:55:56.433 INFO:tasks.workunit.client.0.smithi018.stdout:8/45: symlink db/dc/lf 0 2024-04-12T20:55:56.435 INFO:tasks.workunit.client.0.smithi018.stdout:5/64: dwrite d0/dc/d10/f17 [0,4194304] 0 2024-04-12T20:55:56.438 INFO:tasks.workunit.client.0.smithi018.stdout:3/96: fsync f1 0 2024-04-12T20:55:56.438 INFO:tasks.workunit.client.0.smithi018.stdout:6/148: fdatasync d5/fd 0 2024-04-12T20:55:56.439 INFO:tasks.workunit.client.0.smithi018.stdout:9/99: creat d3/d6/d9/d1e/f20 x:0 0 0 2024-04-12T20:55:56.441 INFO:tasks.workunit.client.0.smithi018.stdout:8/46: symlink db/dc/l10 0 2024-04-12T20:55:56.442 INFO:tasks.workunit.client.0.smithi018.stdout:1/34: symlink d3/d9/da/lb 0 2024-04-12T20:55:56.442 INFO:tasks.workunit.client.0.smithi018.stdout:3/97: dread - d2/f1a zero size 2024-04-12T20:55:56.444 INFO:tasks.workunit.client.0.smithi018.stdout:3/98: stat d2/d6/ce 0 2024-04-12T20:55:56.448 INFO:tasks.workunit.client.0.smithi018.stdout:8/47: creat db/f11 x:0 0 0 2024-04-12T20:55:56.450 INFO:tasks.workunit.client.0.smithi018.stdout:8/48: dread - f7 zero size 2024-04-12T20:55:56.454 INFO:tasks.workunit.client.0.smithi018.stdout:1/35: truncate d3/f7 982590 0 2024-04-12T20:55:56.456 INFO:tasks.workunit.client.0.smithi018.stdout:4/91: dread d3/f5 [0,4194304] 0 2024-04-12T20:55:56.461 INFO:tasks.workunit.client.0.smithi018.stdout:8/49: creat db/dc/f12 x:0 0 0 2024-04-12T20:55:56.461 INFO:tasks.workunit.client.0.smithi018.stdout:4/92: fdatasync d3/d9/fd 0 2024-04-12T20:55:56.461 INFO:tasks.workunit.client.0.smithi018.stdout:6/149: creat d5/f37 x:0 0 0 2024-04-12T20:55:56.463 INFO:tasks.workunit.client.0.smithi018.stdout:1/36: unlink d3/c8 0 2024-04-12T20:55:56.465 INFO:tasks.workunit.client.0.smithi018.stdout:2/107: dread d5/d8/f1e [0,4194304] 0 2024-04-12T20:55:56.466 INFO:tasks.workunit.client.0.smithi018.stdout:8/50: read - db/f11 zero size 2024-04-12T20:55:56.467 INFO:tasks.workunit.client.0.smithi018.stdout:0/113: truncate da/fc 2296588 0 2024-04-12T20:55:56.468 INFO:tasks.workunit.client.0.smithi018.stdout:1/37: stat d3/d9/da 0 2024-04-12T20:55:56.478 INFO:tasks.workunit.client.0.smithi018.stdout:6/150: creat d5/df/d17/f38 x:0 0 0 2024-04-12T20:55:56.482 INFO:tasks.workunit.client.0.smithi018.stdout:0/114: write da/f22 [1977280,84951] 0 2024-04-12T20:55:56.482 INFO:tasks.workunit.client.0.smithi018.stdout:8/51: dread f5 [0,4194304] 0 2024-04-12T20:55:56.482 INFO:tasks.workunit.client.0.smithi018.stdout:4/93: creat d3/d10/f17 x:0 0 0 2024-04-12T20:55:56.483 INFO:tasks.workunit.client.0.smithi018.stdout:9/100: dwrite d3/d6/d9/fa [0,4194304] 0 2024-04-12T20:55:56.484 INFO:tasks.workunit.client.0.smithi018.stdout:5/65: dwrite d0/d2/f1a [0,4194304] 0 2024-04-12T20:55:56.486 INFO:tasks.workunit.client.0.smithi018.stdout:3/99: truncate d2/f5 446336 0 2024-04-12T20:55:56.486 INFO:tasks.workunit.client.0.smithi018.stdout:2/108: symlink d5/d8/d10/d19/l27 0 2024-04-12T20:55:56.490 INFO:tasks.workunit.client.0.smithi018.stdout:2/109: chown d5/c22 18098 1 2024-04-12T20:55:56.490 INFO:tasks.workunit.client.0.smithi018.stdout:6/151: creat d5/d15/f39 x:0 0 0 2024-04-12T20:55:56.495 INFO:tasks.workunit.client.0.smithi018.stdout:8/52: unlink f9 0 2024-04-12T20:55:56.499 INFO:tasks.workunit.client.0.smithi018.stdout:3/100: creat d2/d6/f1b x:0 0 0 2024-04-12T20:55:56.503 INFO:tasks.workunit.client.0.smithi018.stdout:9/101: mkdir d3/d6/d9/dc/d1c/d21 0 2024-04-12T20:55:56.507 INFO:tasks.workunit.client.0.smithi018.stdout:8/53: creat db/dc/f13 x:0 0 0 2024-04-12T20:55:56.509 INFO:tasks.workunit.client.0.smithi018.stdout:2/110: mkdir d5/d7/d28 0 2024-04-12T20:55:56.509 INFO:tasks.workunit.client.0.smithi018.stdout:8/54: chown f0 1369 1 2024-04-12T20:55:56.509 INFO:tasks.workunit.client.0.smithi018.stdout:3/101: creat d2/d6/f1c x:0 0 0 2024-04-12T20:55:56.511 INFO:tasks.workunit.client.0.smithi018.stdout:1/38: getdents d3/d9/da 0 2024-04-12T20:55:56.513 INFO:tasks.workunit.client.0.smithi018.stdout:8/55: dread - db/dc/f12 zero size 2024-04-12T20:55:56.514 INFO:tasks.workunit.client.0.smithi018.stdout:5/66: dwrite d0/d2/d9/f13 [0,4194304] 0 2024-04-12T20:55:56.518 INFO:tasks.workunit.client.0.smithi018.stdout:9/102: creat d3/d6/d9/dc/f22 x:0 0 0 2024-04-12T20:55:56.525 INFO:tasks.workunit.client.0.smithi018.stdout:9/103: truncate d3/d6/d9/d1e/f20 318852 0 2024-04-12T20:55:56.526 INFO:tasks.workunit.client.0.smithi018.stdout:4/94: getdents d3/d10 0 2024-04-12T20:55:56.531 INFO:tasks.workunit.client.0.smithi018.stdout:4/95: chown d3/d10/c16 12175337 1 2024-04-12T20:55:56.531 INFO:tasks.workunit.client.0.smithi018.stdout:6/152: truncate d5/df/f18 1829087 0 2024-04-12T20:55:56.532 INFO:tasks.workunit.client.0.smithi018.stdout:9/104: write d3/d6/f1f [307910,10458] 0 2024-04-12T20:55:56.532 INFO:tasks.workunit.client.0.smithi018.stdout:4/96: chown d3 13088 1 2024-04-12T20:55:56.534 INFO:tasks.workunit.client.0.smithi018.stdout:2/111: getdents d5/d7/d28 0 2024-04-12T20:55:56.534 INFO:tasks.workunit.client.0.smithi018.stdout:3/102: mknod d2/c1d 0 2024-04-12T20:55:56.538 INFO:tasks.workunit.client.0.smithi018.stdout:4/97: unlink d3/d9/ce 0 2024-04-12T20:55:56.541 INFO:tasks.workunit.client.0.smithi018.stdout:8/56: dwrite f7 [0,4194304] 0 2024-04-12T20:55:56.541 INFO:tasks.workunit.client.0.smithi018.stdout:1/39: dread d3/f7 [0,4194304] 0 2024-04-12T20:55:56.541 INFO:tasks.workunit.client.0.smithi018.stdout:6/153: chown c4 2063056089 1 2024-04-12T20:55:56.541 INFO:tasks.workunit.client.0.smithi018.stdout:9/105: chown d3/d6/f1f 5094346 1 2024-04-12T20:55:56.548 INFO:tasks.workunit.client.0.smithi018.stdout:1/40: readlink d3/d9/da/lb 0 2024-04-12T20:55:56.548 INFO:tasks.workunit.client.0.smithi018.stdout:3/103: mknod d2/d6/c1e 0 2024-04-12T20:55:56.548 INFO:tasks.workunit.client.0.smithi018.stdout:8/57: read - db/f11 zero size 2024-04-12T20:55:56.549 INFO:tasks.workunit.client.0.smithi018.stdout:6/154: chown d5/df/d1e/d35 10765 1 2024-04-12T20:55:56.550 INFO:tasks.workunit.client.0.smithi018.stdout:4/98: rmdir d3 39 2024-04-12T20:55:56.551 INFO:tasks.workunit.client.0.smithi018.stdout:9/106: fsync d3/d6/d9/fa 0 2024-04-12T20:55:56.553 INFO:tasks.workunit.client.0.smithi018.stdout:3/104: stat d2/d6/f12 0 2024-04-12T20:55:56.553 INFO:tasks.workunit.client.0.smithi018.stdout:5/67: dwrite d0/d2/d9/fe [0,4194304] 0 2024-04-12T20:55:56.554 INFO:tasks.workunit.client.0.smithi018.stdout:8/58: dread f6 [0,4194304] 0 2024-04-12T20:55:56.561 INFO:tasks.workunit.client.0.smithi018.stdout:4/99: rmdir d3/d9 39 2024-04-12T20:55:56.562 INFO:tasks.workunit.client.0.smithi018.stdout:8/59: readlink l8 0 2024-04-12T20:55:56.562 INFO:tasks.workunit.client.0.smithi018.stdout:1/41: dread d3/f7 [0,4194304] 0 2024-04-12T20:55:56.565 INFO:tasks.workunit.client.0.smithi018.stdout:3/105: mknod d2/c1f 0 2024-04-12T20:55:56.566 INFO:tasks.workunit.client.0.smithi018.stdout:2/112: dread d5/d8/f1e [0,4194304] 0 2024-04-12T20:55:56.567 INFO:tasks.workunit.client.0.smithi018.stdout:0/115: dwrite da/fc [0,4194304] 0 2024-04-12T20:55:56.568 INFO:tasks.workunit.client.0.smithi018.stdout:8/60: write fa [2680732,102422] 0 2024-04-12T20:55:56.569 INFO:tasks.workunit.client.0.smithi018.stdout:9/107: read d3/d6/f1f [282056,42008] 0 2024-04-12T20:55:56.570 INFO:tasks.workunit.client.0.smithi018.stdout:1/42: mknod d3/cc 0 2024-04-12T20:55:56.570 INFO:tasks.workunit.client.0.smithi018.stdout:4/100: rename d3/db/f12 to d3/d10/f18 0 2024-04-12T20:55:56.572 INFO:tasks.workunit.client.0.smithi018.stdout:3/106: mknod d2/d6/c20 0 2024-04-12T20:55:56.582 INFO:tasks.workunit.client.0.smithi018.stdout:4/101: chown d3/db/c15 3 1 2024-04-12T20:55:56.582 INFO:tasks.workunit.client.0.smithi018.stdout:4/102: readlink d3/lf 0 2024-04-12T20:55:56.585 INFO:tasks.workunit.client.0.smithi018.stdout:5/68: dread d0/d2/d9/fb [0,4194304] 0 2024-04-12T20:55:56.587 INFO:tasks.workunit.client.0.smithi018.stdout:8/61: link f0 db/dc/f14 0 2024-04-12T20:55:56.588 INFO:tasks.workunit.client.0.smithi018.stdout:5/69: truncate d0/d2/d9/fe 4996456 0 2024-04-12T20:55:56.588 INFO:tasks.workunit.client.0.smithi018.stdout:9/108: creat d3/f23 x:0 0 0 2024-04-12T20:55:56.591 INFO:tasks.workunit.client.0.smithi018.stdout:9/109: chown d3/l4 595205546 1 2024-04-12T20:55:56.596 INFO:tasks.workunit.client.0.smithi018.stdout:8/62: rename f5 to db/f15 0 2024-04-12T20:55:56.597 INFO:tasks.workunit.client.0.smithi018.stdout:8/63: fsync f7 0 2024-04-12T20:55:56.604 INFO:tasks.workunit.client.0.smithi018.stdout:0/116: write da/f13 [961418,38136] 0 2024-04-12T20:55:56.607 INFO:tasks.workunit.client.0.smithi018.stdout:1/43: write d3/f7 [1971997,39678] 0 2024-04-12T20:55:56.607 INFO:tasks.workunit.client.0.smithi018.stdout:8/64: mknod db/c16 0 2024-04-12T20:55:56.608 INFO:tasks.workunit.client.0.smithi018.stdout:9/110: mknod d3/d6/d9/c24 0 2024-04-12T20:55:56.611 INFO:tasks.workunit.client.0.smithi018.stdout:3/107: dwrite d2/d6/f1c [0,4194304] 0 2024-04-12T20:55:56.612 INFO:tasks.workunit.client.0.smithi018.stdout:0/117: write da/fc [1717880,99462] 0 2024-04-12T20:55:56.614 INFO:tasks.workunit.client.0.smithi018.stdout:0/118: readlink l6 0 2024-04-12T20:55:56.615 INFO:tasks.workunit.client.0.smithi018.stdout:1/44: mkdir d3/d9/da/dd 0 2024-04-12T20:55:56.616 INFO:tasks.workunit.client.0.smithi018.stdout:8/65: write db/f11 [637695,80658] 0 2024-04-12T20:55:56.617 INFO:tasks.workunit.client.0.smithi018.stdout:9/111: write d3/d6/d9/fa [4624928,129679] 0 2024-04-12T20:55:56.618 INFO:tasks.workunit.client.0.smithi018.stdout:5/70: dread d0/d2/d9/f13 [0,4194304] 0 2024-04-12T20:55:56.622 INFO:tasks.workunit.client.0.smithi018.stdout:1/45: read d3/f6 [3176053,101196] 0 2024-04-12T20:55:56.623 INFO:tasks.workunit.client.0.smithi018.stdout:8/66: dread - db/dc/f13 zero size 2024-04-12T20:55:56.623 INFO:tasks.workunit.client.0.smithi018.stdout:0/119: read da/d15/d17/f19 [1509163,59272] 0 2024-04-12T20:55:56.625 INFO:tasks.workunit.client.0.smithi018.stdout:6/155: dwrite d5/d6/f11 [0,4194304] 0 2024-04-12T20:55:56.627 INFO:tasks.workunit.client.0.smithi018.stdout:8/67: chown f7 137856170 1 2024-04-12T20:55:56.629 INFO:tasks.workunit.client.0.smithi018.stdout:0/120: read - da/f1d zero size 2024-04-12T20:55:56.631 INFO:tasks.workunit.client.0.smithi018.stdout:5/71: symlink d0/d2/d11/l1b 0 2024-04-12T20:55:56.632 INFO:tasks.workunit.client.0.smithi018.stdout:5/72: stat d0/dc/d10 0 2024-04-12T20:55:56.638 INFO:tasks.workunit.client.0.smithi018.stdout:3/108: getdents d2 0 2024-04-12T20:55:56.638 INFO:tasks.workunit.client.0.smithi018.stdout:1/46: mkdir d3/d9/da/dd/de 0 2024-04-12T20:55:56.639 INFO:tasks.workunit.client.0.smithi018.stdout:0/121: readlink da/d15/d17/l1c 0 2024-04-12T20:55:56.639 INFO:tasks.workunit.client.0.smithi018.stdout:5/73: mkdir d0/d2/d1c 0 2024-04-12T20:55:56.647 INFO:tasks.workunit.client.0.smithi018.stdout:5/74: chown d0/d2/f1a 41055 1 2024-04-12T20:55:56.649 INFO:tasks.workunit.client.0.smithi018.stdout:8/68: getdents db 0 2024-04-12T20:55:56.650 INFO:tasks.workunit.client.0.smithi018.stdout:0/122: symlink da/d15/d17/l28 0 2024-04-12T20:55:56.653 INFO:tasks.workunit.client.0.smithi018.stdout:0/123: write da/f22 [1335325,79722] 0 2024-04-12T20:55:56.653 INFO:tasks.workunit.client.0.smithi018.stdout:5/75: creat d0/d2/d9/f1d x:0 0 0 2024-04-12T20:55:56.655 INFO:tasks.workunit.client.0.smithi018.stdout:8/69: mknod db/c17 0 2024-04-12T20:55:56.657 INFO:tasks.workunit.client.0.smithi018.stdout:9/112: dwrite d3/f23 [0,4194304] 0 2024-04-12T20:55:56.659 INFO:tasks.workunit.client.0.smithi018.stdout:8/70: dread - db/dc/fe zero size 2024-04-12T20:55:56.662 INFO:tasks.workunit.client.0.smithi018.stdout:6/156: fsync d5/df/f18 0 2024-04-12T20:55:56.662 INFO:tasks.workunit.client.0.smithi018.stdout:8/71: dread db/f15 [0,4194304] 0 2024-04-12T20:55:56.662 INFO:tasks.workunit.client.0.smithi018.stdout:5/76: mkdir d0/d2/d9/d1e 0 2024-04-12T20:55:56.668 INFO:tasks.workunit.client.0.smithi018.stdout:3/109: getdents d2/d6 0 2024-04-12T20:55:56.668 INFO:tasks.workunit.client.0.smithi018.stdout:6/157: stat d5/df/f18 0 2024-04-12T20:55:56.668 INFO:tasks.workunit.client.0.smithi018.stdout:8/72: write db/dc/f12 [310555,99488] 0 2024-04-12T20:55:56.670 INFO:tasks.workunit.client.0.smithi018.stdout:9/113: mknod d3/d6/d9/dc/d1c/d21/c25 0 2024-04-12T20:55:56.671 INFO:tasks.workunit.client.0.smithi018.stdout:3/110: read d2/d6/f9 [395673,81771] 0 2024-04-12T20:55:56.673 INFO:tasks.workunit.client.0.smithi018.stdout:6/158: chown d5/d6/d1b/c21 37620 1 2024-04-12T20:55:56.673 INFO:tasks.workunit.client.0.smithi018.stdout:1/47: dread d3/f4 [0,4194304] 0 2024-04-12T20:55:56.676 INFO:tasks.workunit.client.0.smithi018.stdout:5/77: rename d0/d2/d1c to d0/d2/d1f 0 2024-04-12T20:55:56.677 INFO:tasks.workunit.client.0.smithi018.stdout:8/73: mknod db/dc/c18 0 2024-04-12T20:55:56.680 INFO:tasks.workunit.client.0.smithi018.stdout:5/78: dread - d0/dc/f15 zero size 2024-04-12T20:55:56.681 INFO:tasks.workunit.client.0.smithi018.stdout:9/114: chown d3/d6/d9/d1e/f20 20556305 1 2024-04-12T20:55:56.681 INFO:tasks.workunit.client.0.smithi018.stdout:8/74: write db/dc/f13 [607878,58757] 0 2024-04-12T20:55:56.681 INFO:tasks.workunit.client.0.smithi018.stdout:0/124: dwrite da/f1d [0,4194304] 0 2024-04-12T20:55:56.683 INFO:tasks.workunit.client.0.smithi018.stdout:6/159: symlink d5/df/d19/d2b/l3a 0 2024-04-12T20:55:56.683 INFO:tasks.workunit.client.0.smithi018.stdout:0/125: stat l7 0 2024-04-12T20:55:56.683 INFO:tasks.workunit.client.0.smithi018.stdout:8/75: dread f6 [0,4194304] 0 2024-04-12T20:55:56.684 INFO:tasks.workunit.client.0.smithi018.stdout:1/48: mknod d3/d9/da/dd/de/cf 0 2024-04-12T20:55:56.685 INFO:tasks.workunit.client.0.smithi018.stdout:5/79: chown d0/d8/f12 1691240 1 2024-04-12T20:55:56.687 INFO:tasks.workunit.client.0.smithi018.stdout:8/76: chown db/dc 13718325 1 2024-04-12T20:55:56.689 INFO:tasks.workunit.client.0.smithi018.stdout:0/126: symlink da/d15/l29 0 2024-04-12T20:55:56.690 INFO:tasks.workunit.client.0.smithi018.stdout:6/160: write d5/f10 [2228250,39294] 0 2024-04-12T20:55:56.693 INFO:tasks.workunit.client.0.smithi018.stdout:0/127: fdatasync da/d15/f18 0 2024-04-12T20:55:56.694 INFO:tasks.workunit.client.0.smithi018.stdout:1/49: creat d3/f10 x:0 0 0 2024-04-12T20:55:56.694 INFO:tasks.workunit.client.0.smithi018.stdout:9/115: creat d3/d6/d1b/f26 x:0 0 0 2024-04-12T20:55:56.695 INFO:tasks.workunit.client.0.smithi018.stdout:8/77: mknod db/c19 0 2024-04-12T20:55:56.698 INFO:tasks.workunit.client.0.smithi018.stdout:6/161: stat d5/df/d19/d27/d2f 0 2024-04-12T20:55:56.698 INFO:tasks.workunit.client.0.smithi018.stdout:1/50: mkdir d3/d9/d11 0 2024-04-12T20:55:56.698 INFO:tasks.workunit.client.0.smithi018.stdout:8/78: chown db/dc/fe 2 1 2024-04-12T20:55:56.702 INFO:tasks.workunit.client.0.smithi018.stdout:6/162: rename d5/df/d1e to d5/d6/d1b/d3b 0 2024-04-12T20:55:56.703 INFO:tasks.workunit.client.0.smithi018.stdout:9/116: getdents d3/d6 0 2024-04-12T20:55:56.705 INFO:tasks.workunit.client.0.smithi018.stdout:1/51: creat d3/d9/da/dd/f12 x:0 0 0 2024-04-12T20:55:56.708 INFO:tasks.workunit.client.0.smithi018.stdout:8/79: creat db/dc/f1a x:0 0 0 2024-04-12T20:55:56.711 INFO:tasks.workunit.client.0.smithi018.stdout:6/163: rename d5/d15/c24 to d5/df/d17/c3c 0 2024-04-12T20:55:56.711 INFO:tasks.workunit.client.0.smithi018.stdout:8/80: write db/dc/f13 [677312,32577] 0 2024-04-12T20:55:56.711 INFO:tasks.workunit.client.0.smithi018.stdout:8/81: stat f0 0 2024-04-12T20:55:56.717 INFO:tasks.workunit.client.0.smithi018.stdout:6/164: creat d5/d6/d1b/d3b/f3d x:0 0 0 2024-04-12T20:55:56.720 INFO:tasks.workunit.client.0.smithi018.stdout:6/165: symlink d5/d15/l3e 0 2024-04-12T20:55:56.721 INFO:tasks.workunit.client.0.smithi018.stdout:5/80: dwrite d0/d2/d9/f13 [0,4194304] 0 2024-04-12T20:55:56.726 INFO:tasks.workunit.client.0.smithi018.stdout:6/166: mkdir d5/df/d3f 0 2024-04-12T20:55:56.728 INFO:tasks.workunit.client.0.smithi018.stdout:7/90: sync 2024-04-12T20:55:56.728 INFO:tasks.workunit.client.0.smithi018.stdout:5/81: write d0/d2/d9/f13 [3657460,7456] 0 2024-04-12T20:55:56.731 INFO:tasks.workunit.client.0.smithi018.stdout:6/167: stat d5/df/d17/f38 0 2024-04-12T20:55:56.735 INFO:tasks.workunit.client.0.smithi018.stdout:1/52: dwrite d3/f4 [0,4194304] 0 2024-04-12T20:55:56.736 INFO:tasks.workunit.client.0.smithi018.stdout:7/91: creat d1/d6/f24 x:0 0 0 2024-04-12T20:55:56.739 INFO:tasks.workunit.client.0.smithi018.stdout:9/117: dwrite d3/d6/f10 [0,4194304] 0 2024-04-12T20:55:56.740 INFO:tasks.workunit.client.0.smithi018.stdout:8/82: dwrite f7 [0,4194304] 0 2024-04-12T20:55:56.743 INFO:tasks.workunit.client.0.smithi018.stdout:1/53: mkdir d3/d9/da/dd/de/d13 0 2024-04-12T20:55:56.746 INFO:tasks.workunit.client.0.smithi018.stdout:5/82: link d0/d2/ca d0/dc/c20 0 2024-04-12T20:55:56.750 INFO:tasks.workunit.client.0.smithi018.stdout:8/83: creat db/f1b x:0 0 0 2024-04-12T20:55:56.751 INFO:tasks.workunit.client.0.smithi018.stdout:1/54: mkdir d3/d9/da/d14 0 2024-04-12T20:55:56.753 INFO:tasks.workunit.client.0.smithi018.stdout:5/83: symlink d0/d2/l21 0 2024-04-12T20:55:56.761 INFO:tasks.workunit.client.0.smithi018.stdout:1/55: symlink d3/l15 0 2024-04-12T20:55:56.761 INFO:tasks.workunit.client.0.smithi018.stdout:6/168: dread d5/df/d19/f25 [0,4194304] 0 2024-04-12T20:55:56.766 INFO:tasks.workunit.client.0.smithi018.stdout:1/56: dread - d3/d9/da/dd/f12 zero size 2024-04-12T20:55:56.768 INFO:tasks.workunit.client.0.smithi018.stdout:6/169: unlink d5/df/f29 0 2024-04-12T20:55:56.770 INFO:tasks.workunit.client.0.smithi018.stdout:9/118: dwrite d3/d6/d9/fa [0,4194304] 0 2024-04-12T20:55:56.774 INFO:tasks.workunit.client.0.smithi018.stdout:9/119: truncate d3/d6/f1f 411979 0 2024-04-12T20:55:56.774 INFO:tasks.workunit.client.0.smithi018.stdout:7/92: mkdir d1/d6/d25 0 2024-04-12T20:55:56.777 INFO:tasks.workunit.client.0.smithi018.stdout:5/84: getdents d0/d2/d11 0 2024-04-12T20:55:56.780 INFO:tasks.workunit.client.0.smithi018.stdout:1/57: symlink d3/d9/da/dd/l16 0 2024-04-12T20:55:56.780 INFO:tasks.workunit.client.0.smithi018.stdout:9/120: symlink d3/d6/d1b/l27 0 2024-04-12T20:55:56.786 INFO:tasks.workunit.client.0.smithi018.stdout:1/58: creat d3/d9/da/dd/f17 x:0 0 0 2024-04-12T20:55:56.786 INFO:tasks.workunit.client.0.smithi018.stdout:8/84: dwrite f0 [0,4194304] 0 2024-04-12T20:55:56.786 INFO:tasks.workunit.client.0.smithi018.stdout:9/121: chown d3/l4 0 1 2024-04-12T20:55:56.789 INFO:tasks.workunit.client.0.smithi018.stdout:6/170: rename d5/d6/c14 to d5/c40 0 2024-04-12T20:55:56.790 INFO:tasks.workunit.client.0.smithi018.stdout:8/85: chown db/dc/f12 51397954 1 2024-04-12T20:55:56.793 INFO:tasks.workunit.client.0.smithi018.stdout:6/171: write d5/f37 [257350,36170] 0 2024-04-12T20:55:56.793 INFO:tasks.workunit.client.0.smithi018.stdout:1/59: getdents d3/d9/da/dd/de/d13 0 2024-04-12T20:55:56.794 INFO:tasks.workunit.client.0.smithi018.stdout:9/122: mkdir d3/d6/d9/dc/d13/d28 0 2024-04-12T20:55:56.797 INFO:tasks.workunit.client.0.smithi018.stdout:8/86: write db/dc/f12 [1086093,127088] 0 2024-04-12T20:55:56.798 INFO:tasks.workunit.client.0.smithi018.stdout:7/93: rmdir d1 39 2024-04-12T20:55:56.801 INFO:tasks.workunit.client.0.smithi018.stdout:6/172: fdatasync d5/d6/d1b/d3b/d35/f33 0 2024-04-12T20:55:56.802 INFO:tasks.workunit.client.0.smithi018.stdout:9/123: stat d3/d6/d9/d1e/f20 0 2024-04-12T20:55:56.802 INFO:tasks.workunit.client.0.smithi018.stdout:1/60: write d3/f7 [1781502,108370] 0 2024-04-12T20:55:56.805 INFO:tasks.workunit.client.0.smithi018.stdout:9/124: readlink d3/l15 0 2024-04-12T20:55:56.806 INFO:tasks.workunit.client.0.smithi018.stdout:5/85: dwrite d0/d2/f1a [4194304,4194304] 0 2024-04-12T20:55:56.811 INFO:tasks.workunit.client.0.smithi018.stdout:7/94: write d1/fd [5062514,2909] 0 2024-04-12T20:55:56.813 INFO:tasks.workunit.client.0.smithi018.stdout:5/86: chown d0 624710242 1 2024-04-12T20:55:56.813 INFO:tasks.workunit.client.0.smithi018.stdout:9/125: chown d3/d6/d9/dc/d13/d28 88 1 2024-04-12T20:55:56.813 INFO:tasks.workunit.client.0.smithi018.stdout:8/87: chown db/dc/ld 1348806 1 2024-04-12T20:55:56.817 INFO:tasks.workunit.client.0.smithi018.stdout:9/126: write d3/d6/f10 [2901247,90824] 0 2024-04-12T20:55:56.818 INFO:tasks.workunit.client.0.smithi018.stdout:8/88: write db/f1b [319755,81517] 0 2024-04-12T20:55:56.823 INFO:tasks.workunit.client.0.smithi018.stdout:7/95: symlink d1/d6/l26 0 2024-04-12T20:55:56.823 INFO:tasks.workunit.client.0.smithi018.stdout:6/173: link d5/df/c2e d5/d6/d1b/c41 0 2024-04-12T20:55:56.824 INFO:tasks.workunit.client.0.smithi018.stdout:7/96: stat d1/l1a 0 2024-04-12T20:55:56.824 INFO:tasks.workunit.client.0.smithi018.stdout:1/61: dread d3/f7 [0,4194304] 0 2024-04-12T20:55:56.829 INFO:tasks.workunit.client.0.smithi018.stdout:9/127: rename d3/d6/d9/c24 to d3/d6/d9/dc/d13/d28/c29 0 2024-04-12T20:55:56.830 INFO:tasks.workunit.client.0.smithi018.stdout:7/97: write d1/f1b [611979,95189] 0 2024-04-12T20:55:56.834 INFO:tasks.workunit.client.0.smithi018.stdout:8/89: getdents db 0 2024-04-12T20:55:56.836 INFO:tasks.workunit.client.0.smithi018.stdout:1/62: creat d3/d9/f18 x:0 0 0 2024-04-12T20:55:56.837 INFO:tasks.workunit.client.0.smithi018.stdout:5/87: dwrite d0/d2/d9/f13 [0,4194304] 0 2024-04-12T20:55:56.839 INFO:tasks.workunit.client.0.smithi018.stdout:9/128: write d3/d6/f1f [983780,46327] 0 2024-04-12T20:55:56.842 INFO:tasks.workunit.client.0.smithi018.stdout:8/90: mknod db/c1c 0 2024-04-12T20:55:56.842 INFO:tasks.workunit.client.0.smithi018.stdout:9/129: readlink d3/d6/d1b/l27 0 2024-04-12T20:55:56.844 INFO:tasks.workunit.client.0.smithi018.stdout:5/88: chown d0/d2/c6 107820 1 2024-04-12T20:55:56.847 INFO:tasks.workunit.client.0.smithi018.stdout:6/174: getdents d5/df/d19/d27 0 2024-04-12T20:55:56.848 INFO:tasks.workunit.client.0.smithi018.stdout:8/91: symlink db/l1d 0 2024-04-12T20:55:56.849 INFO:tasks.workunit.client.0.smithi018.stdout:8/92: readlink db/dc/lf 0 2024-04-12T20:55:56.849 INFO:tasks.workunit.client.0.smithi018.stdout:7/98: link d1/f2 d1/d6/f27 0 2024-04-12T20:55:56.850 INFO:tasks.workunit.client.0.smithi018.stdout:9/130: symlink d3/d6/d9/dc/d13/l2a 0 2024-04-12T20:55:56.850 INFO:tasks.workunit.client.0.smithi018.stdout:1/63: getdents d3/d9/da/d14 0 2024-04-12T20:55:56.851 INFO:tasks.workunit.client.0.smithi018.stdout:6/175: symlink d5/d6/d1b/l42 0 2024-04-12T20:55:56.854 INFO:tasks.workunit.client.0.smithi018.stdout:8/93: unlink db/c17 0 2024-04-12T20:55:56.854 INFO:tasks.workunit.client.0.smithi018.stdout:6/176: write d5/f37 [996746,83175] 0 2024-04-12T20:55:56.860 INFO:tasks.workunit.client.0.smithi018.stdout:7/99: creat d1/d6/d8/f28 x:0 0 0 2024-04-12T20:55:56.861 INFO:tasks.workunit.client.0.smithi018.stdout:6/177: mkdir d5/df/d19/d43 0 2024-04-12T20:55:56.862 INFO:tasks.workunit.client.0.smithi018.stdout:9/131: getdents d3/d6 0 2024-04-12T20:55:56.866 INFO:tasks.workunit.client.0.smithi018.stdout:7/100: mknod d1/d6/d8/c29 0 2024-04-12T20:55:56.870 INFO:tasks.workunit.client.0.smithi018.stdout:7/101: write d1/fd [1458191,11885] 0 2024-04-12T20:55:56.874 INFO:tasks.workunit.client.0.smithi018.stdout:6/178: creat d5/df/d3f/f44 x:0 0 0 2024-04-12T20:55:56.876 INFO:tasks.workunit.client.0.smithi018.stdout:5/89: dwrite d0/dc/f15 [0,4194304] 0 2024-04-12T20:55:56.878 INFO:tasks.workunit.client.0.smithi018.stdout:7/102: write d1/f15 [306583,126755] 0 2024-04-12T20:55:56.882 INFO:tasks.workunit.client.0.smithi018.stdout:8/94: dread fa [0,4194304] 0 2024-04-12T20:55:56.890 INFO:tasks.workunit.client.0.smithi018.stdout:8/95: creat db/dc/f1e x:0 0 0 2024-04-12T20:55:56.890 INFO:tasks.workunit.client.0.smithi018.stdout:6/179: link d5/d6/d1b/l42 d5/df/l45 0 2024-04-12T20:55:56.894 INFO:tasks.workunit.client.0.smithi018.stdout:7/103: mknod d1/d6/d25/c2a 0 2024-04-12T20:55:56.896 INFO:tasks.workunit.client.0.smithi018.stdout:9/132: dwrite d3/d6/d9/dc/d13/f17 [4194304,4194304] 0 2024-04-12T20:55:56.897 INFO:tasks.workunit.client.0.smithi018.stdout:8/96: creat db/dc/f1f x:0 0 0 2024-04-12T20:55:56.902 INFO:tasks.workunit.client.0.smithi018.stdout:8/97: fsync db/dc/f12 0 2024-04-12T20:55:56.902 INFO:tasks.workunit.client.0.smithi018.stdout:6/180: creat d5/d6/d1b/d3b/d22/f46 x:0 0 0 2024-04-12T20:55:56.902 INFO:tasks.workunit.client.0.smithi018.stdout:6/181: rename d5 to d5/df/d47 22 2024-04-12T20:55:56.902 INFO:tasks.workunit.client.0.smithi018.stdout:7/104: creat d1/db/f2b x:0 0 0 2024-04-12T20:55:56.903 INFO:tasks.workunit.client.0.smithi018.stdout:9/133: symlink d3/d6/d9/d1e/l2b 0 2024-04-12T20:55:56.904 INFO:tasks.workunit.client.0.smithi018.stdout:5/90: dread d0/d8/f12 [0,4194304] 0 2024-04-12T20:55:56.907 INFO:tasks.workunit.client.0.smithi018.stdout:8/98: mknod db/dc/c20 0 2024-04-12T20:55:56.908 INFO:tasks.workunit.client.0.smithi018.stdout:9/134: fsync d3/d6/d9/dc/f22 0 2024-04-12T20:55:56.910 INFO:tasks.workunit.client.0.smithi018.stdout:5/91: creat d0/d8/f22 x:0 0 0 2024-04-12T20:55:56.910 INFO:tasks.workunit.client.0.smithi018.stdout:9/135: readlink d3/d6/d1b/l27 0 2024-04-12T20:55:56.911 INFO:tasks.workunit.client.0.smithi018.stdout:8/99: creat db/f21 x:0 0 0 2024-04-12T20:55:56.912 INFO:tasks.workunit.client.0.smithi018.stdout:6/182: creat d5/f48 x:0 0 0 2024-04-12T20:55:56.913 INFO:tasks.workunit.client.0.smithi018.stdout:5/92: fsync d0/dc/d10/f17 0 2024-04-12T20:55:56.919 INFO:tasks.workunit.client.0.smithi018.stdout:9/136: write d3/d6/d9/dc/f22 [10515,16576] 0 2024-04-12T20:55:56.925 INFO:tasks.workunit.client.0.smithi018.stdout:5/93: read d0/d2/d9/fe [1454880,41681] 0 2024-04-12T20:55:56.934 INFO:tasks.workunit.client.0.smithi018.stdout:9/137: dread d3/d6/d9/d1e/f20 [0,4194304] 0 2024-04-12T20:55:56.935 INFO:tasks.workunit.client.0.smithi018.stdout:5/94: mkdir d0/d2/d9/d23 0 2024-04-12T20:55:56.936 INFO:tasks.workunit.client.0.smithi018.stdout:7/105: dwrite d1/d6/d8/f13 [0,4194304] 0 2024-04-12T20:55:56.941 INFO:tasks.workunit.client.0.smithi018.stdout:9/138: mknod d3/d6/c2c 0 2024-04-12T20:55:56.942 INFO:tasks.workunit.client.0.smithi018.stdout:7/106: creat d1/d6/d8/f2c x:0 0 0 2024-04-12T20:55:56.944 INFO:tasks.workunit.client.0.smithi018.stdout:6/183: dwrite d5/d6/f11 [0,4194304] 0 2024-04-12T20:55:56.948 INFO:tasks.workunit.client.0.smithi018.stdout:5/95: rename d0/d2/ca to d0/d8/c24 0 2024-04-12T20:55:56.950 INFO:tasks.workunit.client.0.smithi018.stdout:7/107: fdatasync d1/d6/f24 0 2024-04-12T20:55:56.950 INFO:tasks.workunit.client.0.smithi018.stdout:8/100: dwrite db/f1b [0,4194304] 0 2024-04-12T20:55:56.952 INFO:tasks.workunit.client.0.smithi018.stdout:8/101: dread - db/dc/f1f zero size 2024-04-12T20:55:56.952 INFO:tasks.workunit.client.0.smithi018.stdout:6/184: creat d5/df/d3f/f49 x:0 0 0 2024-04-12T20:55:56.953 INFO:tasks.workunit.client.0.smithi018.stdout:7/108: write d1/fd [1123769,66126] 0 2024-04-12T20:55:56.954 INFO:tasks.workunit.client.0.smithi018.stdout:5/96: fsync d0/d2/d9/fb 0 2024-04-12T20:55:56.959 INFO:tasks.workunit.client.0.smithi018.stdout:5/97: read d0/ff [639536,78750] 0 2024-04-12T20:55:56.960 INFO:tasks.workunit.client.0.smithi018.stdout:6/185: fdatasync d5/df/d19/f25 0 2024-04-12T20:55:56.962 INFO:tasks.workunit.client.0.smithi018.stdout:8/102: unlink db/dc/fe 0 2024-04-12T20:55:56.962 INFO:tasks.workunit.client.0.smithi018.stdout:5/98: dread - d0/d8/f22 zero size 2024-04-12T20:55:56.966 INFO:tasks.workunit.client.0.smithi018.stdout:6/186: write d5/d6/f11 [1774756,4800] 0 2024-04-12T20:55:56.970 INFO:tasks.workunit.client.0.smithi018.stdout:8/103: symlink db/dc/l22 0 2024-04-12T20:55:56.970 INFO:tasks.workunit.client.0.smithi018.stdout:7/109: dread d1/f2 [0,4194304] 0 2024-04-12T20:55:56.971 INFO:tasks.workunit.client.0.smithi018.stdout:6/187: mknod d5/df/d3f/c4a 0 2024-04-12T20:55:56.972 INFO:tasks.workunit.client.0.smithi018.stdout:7/110: write d1/f1b [1025441,21027] 0 2024-04-12T20:55:56.973 INFO:tasks.workunit.client.0.smithi018.stdout:8/104: creat db/dc/f23 x:0 0 0 2024-04-12T20:55:56.974 INFO:tasks.workunit.client.0.smithi018.stdout:6/188: write d5/df/f2a [101698,99503] 0 2024-04-12T20:55:56.977 INFO:tasks.workunit.client.0.smithi018.stdout:8/105: mknod db/c24 0 2024-04-12T20:55:56.981 INFO:tasks.workunit.client.0.smithi018.stdout:8/106: mkdir db/d25 0 2024-04-12T20:55:56.981 INFO:tasks.workunit.client.0.smithi018.stdout:5/99: dread d0/d2/f1a [0,4194304] 0 2024-04-12T20:55:56.983 INFO:tasks.workunit.client.0.smithi018.stdout:6/189: symlink d5/df/d19/d27/d2f/l4b 0 2024-04-12T20:55:56.985 INFO:tasks.workunit.client.0.smithi018.stdout:7/111: creat d1/d6/f2d x:0 0 0 2024-04-12T20:55:56.987 INFO:tasks.workunit.client.0.smithi018.stdout:5/100: symlink d0/d2/d9/l25 0 2024-04-12T20:55:56.990 INFO:tasks.workunit.client.0.smithi018.stdout:6/190: mknod d5/d6/c4c 0 2024-04-12T20:55:56.991 INFO:tasks.workunit.client.0.smithi018.stdout:8/107: mkdir db/d25/d26 0 2024-04-12T20:55:56.998 INFO:tasks.workunit.client.0.smithi018.stdout:7/112: fdatasync d1/d6/f27 0 2024-04-12T20:55:56.998 INFO:tasks.workunit.client.0.smithi018.stdout:9/139: truncate d3/d6/d9/fa 417100 0 2024-04-12T20:55:56.998 INFO:tasks.workunit.client.0.smithi018.stdout:5/101: rename d0/d2/d9/d23 to d0/dc/d10/d26 0 2024-04-12T20:55:56.998 INFO:tasks.workunit.client.0.smithi018.stdout:9/140: dread - d3/d6/d1b/f26 zero size 2024-04-12T20:55:56.999 INFO:tasks.workunit.client.0.smithi018.stdout:6/191: symlink d5/d6/d1b/d3b/d22/l4d 0 2024-04-12T20:55:57.002 INFO:tasks.workunit.client.0.smithi018.stdout:7/113: rename d1/db/c12 to d1/db/c2e 0 2024-04-12T20:55:57.005 INFO:tasks.workunit.client.0.smithi018.stdout:9/141: truncate d3/d6/d1b/f26 426938 0 2024-04-12T20:55:57.007 INFO:tasks.workunit.client.0.smithi018.stdout:5/102: truncate d0/d8/f12 1956801 0 2024-04-12T20:55:57.011 INFO:tasks.workunit.client.0.smithi018.stdout:7/114: rename d1/f21 to d1/f2f 0 2024-04-12T20:55:57.013 INFO:tasks.workunit.client.0.smithi018.stdout:6/192: rename d5/df/d17 to d5/df/d31/d4e 0 2024-04-12T20:55:57.016 INFO:tasks.workunit.client.0.smithi018.stdout:9/142: creat d3/d6/d9/dc/f2d x:0 0 0 2024-04-12T20:55:57.016 INFO:tasks.workunit.client.0.smithi018.stdout:7/115: fsync d1/f1b 0 2024-04-12T20:55:57.016 INFO:tasks.workunit.client.0.smithi018.stdout:6/193: fsync d5/d6/d1b/d3b/d22/f46 0 2024-04-12T20:55:57.017 INFO:tasks.workunit.client.0.smithi018.stdout:8/108: dwrite db/f21 [0,4194304] 0 2024-04-12T20:55:57.027 INFO:tasks.workunit.client.0.smithi018.stdout:7/116: mkdir d1/d6/d25/d30 0 2024-04-12T20:55:57.028 INFO:tasks.workunit.client.0.smithi018.stdout:6/194: mkdir d5/d6/d1b/d3b/d35/d4f 0 2024-04-12T20:55:57.030 INFO:tasks.workunit.client.0.smithi018.stdout:9/143: creat d3/d6/d9/dc/d1c/f2e x:0 0 0 2024-04-12T20:55:57.033 INFO:tasks.workunit.client.0.smithi018.stdout:8/109: mkdir db/d25/d26/d27 0 2024-04-12T20:55:57.034 INFO:tasks.workunit.client.0.smithi018.stdout:8/110: stat db/dc/f23 0 2024-04-12T20:55:57.036 INFO:tasks.workunit.client.0.smithi018.stdout:6/195: rmdir d5/df/d19 39 2024-04-12T20:55:57.043 INFO:tasks.workunit.client.0.smithi018.stdout:6/196: symlink d5/df/l50 0 2024-04-12T20:55:57.044 INFO:tasks.workunit.client.0.smithi018.stdout:8/111: link db/dc/c20 db/d25/c28 0 2024-04-12T20:55:57.046 INFO:tasks.workunit.client.0.smithi018.stdout:9/144: getdents d3/d6/d9 0 2024-04-12T20:55:57.050 INFO:tasks.workunit.client.0.smithi018.stdout:7/117: truncate d1/fd 781045 0 2024-04-12T20:55:57.053 INFO:tasks.workunit.client.0.smithi018.stdout:6/197: symlink d5/df/d19/d43/l51 0 2024-04-12T20:55:57.056 INFO:tasks.workunit.client.0.smithi018.stdout:8/112: rmdir db 39 2024-04-12T20:55:57.060 INFO:tasks.workunit.client.0.smithi018.stdout:6/198: creat d5/d6/d1b/d3b/f52 x:0 0 0 2024-04-12T20:55:57.061 INFO:tasks.workunit.client.0.smithi018.stdout:6/199: stat d5/df/d3f/f49 0 2024-04-12T20:55:57.063 INFO:tasks.workunit.client.0.smithi018.stdout:8/113: write db/f21 [4652350,15046] 0 2024-04-12T20:55:57.066 INFO:tasks.workunit.client.0.smithi018.stdout:8/114: chown f7 118033 1 2024-04-12T20:55:57.069 INFO:tasks.workunit.client.0.smithi018.stdout:7/118: dread d1/db/f22 [0,4194304] 0 2024-04-12T20:55:57.077 INFO:tasks.workunit.client.0.smithi018.stdout:7/119: rename d1/d6/d25 to d1/db/d31 0 2024-04-12T20:55:57.079 INFO:tasks.workunit.client.0.smithi018.stdout:6/200: dread d5/f37 [0,4194304] 0 2024-04-12T20:55:57.079 INFO:tasks.workunit.client.0.smithi018.stdout:9/145: dwrite d3/d6/d9/d1e/f20 [0,4194304] 0 2024-04-12T20:55:57.081 INFO:tasks.workunit.client.0.smithi018.stdout:7/120: dread - d1/d6/d8/f2c zero size 2024-04-12T20:55:57.088 INFO:tasks.workunit.client.0.smithi018.stdout:9/146: creat d3/d6/d9/f2f x:0 0 0 2024-04-12T20:55:57.088 INFO:tasks.workunit.client.0.smithi018.stdout:6/201: creat d5/df/f53 x:0 0 0 2024-04-12T20:55:57.091 INFO:tasks.workunit.client.0.smithi018.stdout:8/115: dwrite db/dc/f1a [0,4194304] 0 2024-04-12T20:55:57.091 INFO:tasks.workunit.client.0.smithi018.stdout:6/202: chown d5/df/f53 78134 1 2024-04-12T20:55:57.095 INFO:tasks.workunit.client.0.smithi018.stdout:8/116: write db/dc/f13 [700704,72338] 0 2024-04-12T20:55:57.100 INFO:tasks.workunit.client.0.smithi018.stdout:8/117: write db/dc/f13 [1544097,54760] 0 2024-04-12T20:55:57.105 INFO:tasks.workunit.client.0.smithi018.stdout:6/203: truncate d5/df/d19/f25 4381162 0 2024-04-12T20:55:57.108 INFO:tasks.workunit.client.0.smithi018.stdout:7/121: dwrite d1/f14 [0,4194304] 0 2024-04-12T20:55:57.108 INFO:tasks.workunit.client.0.smithi018.stdout:8/118: write db/dc/f14 [3751216,38358] 0 2024-04-12T20:55:57.111 INFO:tasks.workunit.client.0.smithi018.stdout:9/147: dread - d3/d6/d9/f2f zero size 2024-04-12T20:55:57.112 INFO:tasks.workunit.client.0.smithi018.stdout:8/119: chown db/d25/d26/d27 2069671 1 2024-04-12T20:55:57.117 INFO:tasks.workunit.client.0.smithi018.stdout:6/204: creat d5/df/d19/f54 x:0 0 0 2024-04-12T20:55:57.118 INFO:tasks.workunit.client.0.smithi018.stdout:9/148: creat d3/d6/f30 x:0 0 0 2024-04-12T20:55:57.120 INFO:tasks.workunit.client.0.smithi018.stdout:6/205: mknod d5/d6/d1b/d3b/d35/d4f/c55 0 2024-04-12T20:55:57.124 INFO:tasks.workunit.client.0.smithi018.stdout:6/206: write d5/f10 [4379867,23782] 0 2024-04-12T20:55:57.128 INFO:tasks.workunit.client.0.smithi018.stdout:6/207: write d5/d15/f1c [307128,97644] 0 2024-04-12T20:55:57.130 INFO:tasks.workunit.client.0.smithi018.stdout:7/122: dread d1/d6/d8/f13 [0,4194304] 0 2024-04-12T20:55:57.131 INFO:tasks.workunit.client.0.smithi018.stdout:6/208: mknod d5/d6/c56 0 2024-04-12T20:55:57.133 INFO:tasks.workunit.client.0.smithi018.stdout:7/123: readlink d1/l1a 0 2024-04-12T20:55:57.135 INFO:tasks.workunit.client.0.smithi018.stdout:9/149: write d3/d6/d9/fa [1112646,90309] 0 2024-04-12T20:55:57.135 INFO:tasks.workunit.client.0.smithi018.stdout:7/124: chown d1/c1d 467725 1 2024-04-12T20:55:57.136 INFO:tasks.workunit.client.0.smithi018.stdout:8/120: dwrite db/f1b [0,4194304] 0 2024-04-12T20:55:57.137 INFO:tasks.workunit.client.0.smithi018.stdout:7/125: chown d1/db/f2b 2 1 2024-04-12T20:55:57.138 INFO:tasks.workunit.client.0.smithi018.stdout:6/209: creat d5/f57 x:0 0 0 2024-04-12T20:55:57.139 INFO:tasks.workunit.client.0.smithi018.stdout:8/121: truncate db/f15 907689 0 2024-04-12T20:55:57.139 INFO:tasks.workunit.client.0.smithi018.stdout:7/126: readlink d1/db/l1e 0 2024-04-12T20:55:57.141 INFO:tasks.workunit.client.0.smithi018.stdout:6/210: read d5/df/f18 [2662169,19620] 0 2024-04-12T20:55:57.142 INFO:tasks.workunit.client.0.smithi018.stdout:9/150: creat d3/d6/d9/dc/d13/d28/f31 x:0 0 0 2024-04-12T20:55:57.144 INFO:tasks.workunit.client.0.smithi018.stdout:8/122: symlink db/d25/d26/l29 0 2024-04-12T20:55:57.145 INFO:tasks.workunit.client.0.smithi018.stdout:8/123: stat db/f11 0 2024-04-12T20:55:57.147 INFO:tasks.workunit.client.0.smithi018.stdout:8/124: chown db/dc/ld 40703 1 2024-04-12T20:55:57.148 INFO:tasks.workunit.client.0.smithi018.stdout:6/211: mknod d5/d6/d1b/d3b/d35/d4f/c58 0 2024-04-12T20:55:57.151 INFO:tasks.workunit.client.0.smithi018.stdout:8/125: mknod db/d25/d26/c2a 0 2024-04-12T20:55:57.155 INFO:tasks.workunit.client.0.smithi018.stdout:6/212: symlink d5/d6/d1b/d3b/d35/l59 0 2024-04-12T20:55:57.159 INFO:tasks.workunit.client.0.smithi018.stdout:6/213: getdents d5/df/d31 0 2024-04-12T20:55:57.163 INFO:tasks.workunit.client.0.smithi018.stdout:7/127: dread d1/f14 [0,4194304] 0 2024-04-12T20:55:57.165 INFO:tasks.workunit.client.0.smithi018.stdout:7/128: fsync d1/f2 0 2024-04-12T20:55:57.166 INFO:tasks.workunit.client.0.smithi018.stdout:9/151: dwrite d3/d6/d9/dc/d1c/f2e [0,4194304] 0 2024-04-12T20:55:57.179 INFO:tasks.workunit.client.0.smithi018.stdout:8/126: dwrite db/dc/f1a [0,4194304] 0 2024-04-12T20:55:57.183 INFO:tasks.workunit.client.0.smithi018.stdout:8/127: chown f7 16781935 1 2024-04-12T20:55:57.189 INFO:tasks.workunit.client.0.smithi018.stdout:6/214: dwrite d5/d15/f39 [0,4194304] 0 2024-04-12T20:55:57.192 INFO:tasks.workunit.client.0.smithi018.stdout:8/128: creat db/d25/d26/d27/f2b x:0 0 0 2024-04-12T20:55:57.194 INFO:tasks.workunit.client.0.smithi018.stdout:6/215: fsync d5/df/f53 0 2024-04-12T20:55:57.195 INFO:tasks.workunit.client.0.smithi018.stdout:8/129: fdatasync db/dc/f13 0 2024-04-12T20:55:57.201 INFO:tasks.workunit.client.0.smithi018.stdout:7/129: dwrite d1/d6/f24 [0,4194304] 0 2024-04-12T20:55:57.201 INFO:tasks.workunit.client.0.smithi018.stdout:9/152: dread d3/d6/d9/d1e/f20 [0,4194304] 0 2024-04-12T20:55:57.202 INFO:tasks.workunit.client.0.smithi018.stdout:6/216: write d5/df/d3f/f44 [410813,6412] 0 2024-04-12T20:55:57.203 INFO:tasks.workunit.client.0.smithi018.stdout:7/130: stat d1/db/f2b 0 2024-04-12T20:55:57.204 INFO:tasks.workunit.client.0.smithi018.stdout:2/113: sync 2024-04-12T20:55:57.206 INFO:tasks.workunit.client.0.smithi018.stdout:7/131: dread - d1/db/f2b zero size 2024-04-12T20:55:57.207 INFO:tasks.workunit.client.0.smithi018.stdout:6/217: mknod d5/d6/d1b/d3b/d22/c5a 0 2024-04-12T20:55:57.209 INFO:tasks.workunit.client.0.smithi018.stdout:9/153: mknod d3/d6/d9/c32 0 2024-04-12T20:55:57.210 INFO:tasks.workunit.client.0.smithi018.stdout:6/218: write d5/d6/d1b/d3b/f3d [599669,19342] 0 2024-04-12T20:55:57.211 INFO:tasks.workunit.client.0.smithi018.stdout:7/132: write d1/d6/d8/f2c [373631,65093] 0 2024-04-12T20:55:57.214 INFO:tasks.workunit.client.0.smithi018.stdout:6/219: chown d5/d6/d1b/d3b/d35/f33 566013 1 2024-04-12T20:55:57.217 INFO:tasks.workunit.client.0.smithi018.stdout:7/133: creat d1/f32 x:0 0 0 2024-04-12T20:55:57.219 INFO:tasks.workunit.client.0.smithi018.stdout:6/220: unlink d5/df/f53 0 2024-04-12T20:55:57.226 INFO:tasks.workunit.client.0.smithi018.stdout:8/130: truncate fa 1836529 0 2024-04-12T20:55:57.229 INFO:tasks.workunit.client.0.smithi018.stdout:8/131: write db/dc/f12 [1454638,4814] 0 2024-04-12T20:55:57.232 INFO:tasks.workunit.client.0.smithi018.stdout:6/221: creat d5/d6/d1b/d3b/d35/d4f/f5b x:0 0 0 2024-04-12T20:55:57.234 INFO:tasks.workunit.client.0.smithi018.stdout:9/154: dwrite d3/d6/f1f [0,4194304] 0 2024-04-12T20:55:57.237 INFO:tasks.workunit.client.0.smithi018.stdout:8/132: unlink db/dc/f13 0 2024-04-12T20:55:57.240 INFO:tasks.workunit.client.0.smithi018.stdout:9/155: read d3/d6/f1f [2562097,62415] 0 2024-04-12T20:55:57.247 INFO:tasks.workunit.client.0.smithi018.stdout:9/156: symlink d3/d6/d9/dc/d1c/l33 0 2024-04-12T20:55:57.251 INFO:tasks.workunit.client.0.smithi018.stdout:7/134: dwrite d1/f14 [0,4194304] 0 2024-04-12T20:55:57.255 INFO:tasks.workunit.client.0.smithi018.stdout:6/222: dwrite d5/df/d19/f54 [0,4194304] 0 2024-04-12T20:55:57.259 INFO:tasks.workunit.client.0.smithi018.stdout:6/223: chown d5/f7 1608884 1 2024-04-12T20:55:57.261 INFO:tasks.workunit.client.0.smithi018.stdout:9/157: creat d3/d6/d9/dc/d1c/f34 x:0 0 0 2024-04-12T20:55:57.274 INFO:tasks.workunit.client.0.smithi018.stdout:7/135: link d1/db/d31/c2a d1/d6/d8/c33 0 2024-04-12T20:55:57.274 INFO:tasks.workunit.client.0.smithi018.stdout:8/133: dwrite db/f21 [4194304,4194304] 0 2024-04-12T20:55:57.277 INFO:tasks.workunit.client.0.smithi018.stdout:8/134: creat db/d25/f2c x:0 0 0 2024-04-12T20:55:57.280 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:55:57 smithi186 ceph-mon[36597]: pgmap v31: 65 pgs: 65 active+clean; 325 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 354 KiB/s rd, 23 MiB/s wr, 249 op/s 2024-04-12T20:55:57.285 INFO:tasks.workunit.client.0.smithi018.stdout:8/135: rename db/f21 to db/f2d 0 2024-04-12T20:55:57.289 INFO:tasks.workunit.client.0.smithi018.stdout:8/136: truncate db/d25/f2c 977786 0 2024-04-12T20:55:57.290 INFO:tasks.workunit.client.0.smithi018.stdout:7/136: fsync d1/f14 0 2024-04-12T20:55:57.293 INFO:tasks.workunit.client.0.smithi018.stdout:8/137: chown db/d25/d26/l29 493 1 2024-04-12T20:55:57.294 INFO:tasks.workunit.client.0.smithi018.stdout:6/224: dwrite d5/df/f18 [4194304,4194304] 0 2024-04-12T20:55:57.296 INFO:tasks.workunit.client.0.smithi018.stdout:7/137: mknod d1/db/d31/c34 0 2024-04-12T20:55:57.303 INFO:tasks.workunit.client.0.smithi018.stdout:7/138: rmdir d1/d6/d8 39 2024-04-12T20:55:57.305 INFO:tasks.workunit.client.0.smithi018.stdout:6/225: creat d5/d6/d1b/d3b/d35/d4f/f5c x:0 0 0 2024-04-12T20:55:57.316 INFO:tasks.workunit.client.0.smithi018.stdout:7/139: link d1/d6/f27 d1/db/f35 0 2024-04-12T20:55:57.321 INFO:tasks.workunit.client.0.smithi018.stdout:9/158: dread d3/d6/d9/fd [0,4194304] 0 2024-04-12T20:55:57.322 INFO:tasks.workunit.client.0.smithi018.stdout:7/140: rmdir d1 39 2024-04-12T20:55:57.328 INFO:tasks.workunit.client.0.smithi018.stdout:8/138: dwrite fa [0,4194304] 0 2024-04-12T20:55:57.328 INFO:tasks.workunit.client.0.smithi018.stdout:9/159: creat d3/d6/d1b/f35 x:0 0 0 2024-04-12T20:55:57.331 INFO:tasks.workunit.client.0.smithi018.stdout:6/226: dwrite d5/df/d3f/f49 [0,4194304] 0 2024-04-12T20:55:57.332 INFO:tasks.workunit.client.0.smithi018.stdout:8/139: write db/dc/f12 [1717306,84417] 0 2024-04-12T20:55:57.334 INFO:tasks.workunit.client.0.smithi018.stdout:8/140: rename db to db/d25/d26/d27/d2e 22 2024-04-12T20:55:57.337 INFO:tasks.workunit.client.0.smithi018.stdout:8/141: write db/dc/f1a [3896417,58438] 0 2024-04-12T20:55:57.338 INFO:tasks.workunit.client.0.smithi018.stdout:6/227: symlink d5/df/d19/l5d 0 2024-04-12T20:55:57.347 INFO:tasks.workunit.client.0.smithi018.stdout:8/142: creat db/d25/d26/f2f x:0 0 0 2024-04-12T20:55:57.347 INFO:tasks.workunit.client.0.smithi018.stdout:1/64: sync 2024-04-12T20:55:57.347 INFO:tasks.workunit.client.0.smithi018.stdout:0/128: sync 2024-04-12T20:55:57.347 INFO:tasks.workunit.client.0.smithi018.stdout:4/103: sync 2024-04-12T20:55:57.347 INFO:tasks.workunit.client.0.smithi018.stdout:3/111: sync 2024-04-12T20:55:57.347 INFO:tasks.workunit.client.0.smithi018.stdout:5/103: sync 2024-04-12T20:55:57.353 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:55:57 smithi018 ceph-mon[28799]: pgmap v31: 65 pgs: 65 active+clean; 325 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 354 KiB/s rd, 23 MiB/s wr, 249 op/s 2024-04-12T20:55:57.353 INFO:tasks.workunit.client.0.smithi018.stdout:5/104: readlink d0/d2/d11/l14 0 2024-04-12T20:55:57.354 INFO:tasks.workunit.client.0.smithi018.stdout:7/141: link d1/d6/d8/l11 d1/db/d31/l36 0 2024-04-12T20:55:57.355 INFO:tasks.workunit.client.0.smithi018.stdout:9/160: link c1 d3/d6/d9/c36 0 2024-04-12T20:55:57.357 INFO:tasks.workunit.client.0.smithi018.stdout:9/161: chown d3/f23 258515 1 2024-04-12T20:55:57.358 INFO:tasks.workunit.client.0.smithi018.stdout:9/162: dread - d3/d6/f30 zero size 2024-04-12T20:55:57.384 INFO:tasks.workunit.client.0.smithi018.stdout:8/143: rmdir db/dc 39 2024-04-12T20:55:57.390 INFO:tasks.workunit.client.0.smithi018.stdout:0/129: unlink da/d15/l29 0 2024-04-12T20:55:57.392 INFO:tasks.workunit.client.0.smithi018.stdout:5/105: creat d0/dc/d10/f27 x:0 0 0 2024-04-12T20:55:57.393 INFO:tasks.workunit.client.0.smithi018.stdout:6/228: dread d5/f20 [0,4194304] 0 2024-04-12T20:55:57.396 INFO:tasks.workunit.client.0.smithi018.stdout:7/142: rmdir d1/db 39 2024-04-12T20:55:57.403 INFO:tasks.workunit.client.0.smithi018.stdout:9/163: symlink d3/d6/d9/dc/l37 0 2024-04-12T20:55:57.405 INFO:tasks.workunit.client.0.smithi018.stdout:8/144: fsync db/dc/f1f 0 2024-04-12T20:55:57.411 INFO:tasks.workunit.client.0.smithi018.stdout:1/65: mkdir d3/d9/da/dd/de/d13/d19 0 2024-04-12T20:55:57.413 INFO:tasks.workunit.client.0.smithi018.stdout:1/66: dread - d3/d9/da/dd/f17 zero size 2024-04-12T20:55:57.417 INFO:tasks.workunit.client.0.smithi018.stdout:0/130: dread da/f25 [0,4194304] 0 2024-04-12T20:55:57.421 INFO:tasks.workunit.client.0.smithi018.stdout:9/164: write d3/d6/d1b/f35 [981019,17618] 0 2024-04-12T20:55:57.424 INFO:tasks.workunit.client.0.smithi018.stdout:0/131: write da/f1d [3631424,125022] 0 2024-04-12T20:55:57.425 INFO:tasks.workunit.client.0.smithi018.stdout:9/165: readlink d3/d6/d9/dc/l37 0 2024-04-12T20:55:57.426 INFO:tasks.workunit.client.0.smithi018.stdout:0/132: readlink da/d15/d17/l1c 0 2024-04-12T20:55:57.428 INFO:tasks.workunit.client.0.smithi018.stdout:9/166: chown d3/d6/d9/dc/l37 247835836 1 2024-04-12T20:55:57.429 INFO:tasks.workunit.client.0.smithi018.stdout:7/143: rename d1/d6/d8/f28 to d1/db/d31/f37 0 2024-04-12T20:55:57.430 INFO:tasks.workunit.client.0.smithi018.stdout:8/145: dwrite db/f1b [0,4194304] 0 2024-04-12T20:55:57.433 INFO:tasks.workunit.client.0.smithi018.stdout:8/146: stat db/d25/d26/l29 0 2024-04-12T20:55:57.443 INFO:tasks.workunit.client.0.smithi018.stdout:1/67: rename d3/cc to d3/c1a 0 2024-04-12T20:55:57.444 INFO:tasks.workunit.client.0.smithi018.stdout:9/167: dread d3/f23 [0,4194304] 0 2024-04-12T20:55:57.448 INFO:tasks.workunit.client.0.smithi018.stdout:9/168: dread d3/d6/d9/dc/f22 [0,4194304] 0 2024-04-12T20:55:57.449 INFO:tasks.workunit.client.0.smithi018.stdout:0/133: symlink da/d15/d17/d21/l2a 0 2024-04-12T20:55:57.452 INFO:tasks.workunit.client.0.smithi018.stdout:9/169: dread d3/d6/d9/dc/f22 [0,4194304] 0 2024-04-12T20:55:57.457 INFO:tasks.workunit.client.0.smithi018.stdout:7/144: write d1/d6/d8/f13 [2266995,1157] 0 2024-04-12T20:55:57.459 INFO:tasks.workunit.client.0.smithi018.stdout:4/104: getdents d3/db 0 2024-04-12T20:55:57.461 INFO:tasks.workunit.client.0.smithi018.stdout:1/68: creat d3/d9/da/dd/de/f1b x:0 0 0 2024-04-12T20:55:57.464 INFO:tasks.workunit.client.0.smithi018.stdout:5/106: write d0/d8/f12 [2801022,42065] 0 2024-04-12T20:55:57.469 INFO:tasks.workunit.client.0.smithi018.stdout:5/107: rename d0 to d0/dc/d10/d28 22 2024-04-12T20:55:57.470 INFO:tasks.workunit.client.0.smithi018.stdout:4/105: chown d3/d9/c11 43559 1 2024-04-12T20:55:57.471 INFO:tasks.workunit.client.0.smithi018.stdout:1/69: symlink d3/d9/da/l1c 0 2024-04-12T20:55:57.472 INFO:tasks.workunit.client.0.smithi018.stdout:5/108: write d0/d8/f12 [3728056,75920] 0 2024-04-12T20:55:57.473 INFO:tasks.workunit.client.0.smithi018.stdout:2/114: sync 2024-04-12T20:55:57.473 INFO:tasks.workunit.client.0.smithi018.stdout:0/134: truncate da/d15/f18 2560318 0 2024-04-12T20:55:57.475 INFO:tasks.workunit.client.0.smithi018.stdout:9/170: dwrite d3/d6/d9/dc/d1c/f34 [0,4194304] 0 2024-04-12T20:55:57.476 INFO:tasks.workunit.client.0.smithi018.stdout:5/109: read - d0/dc/d10/f27 zero size 2024-04-12T20:55:57.477 INFO:tasks.workunit.client.0.smithi018.stdout:7/145: mkdir d1/d38 0 2024-04-12T20:55:57.482 INFO:tasks.workunit.client.0.smithi018.stdout:4/106: creat d3/d9/f19 x:0 0 0 2024-04-12T20:55:57.483 INFO:tasks.workunit.client.0.smithi018.stdout:8/147: getdents db/d25 0 2024-04-12T20:55:57.486 INFO:tasks.workunit.client.0.smithi018.stdout:0/135: symlink da/d15/l2b 0 2024-04-12T20:55:57.490 INFO:tasks.workunit.client.0.smithi018.stdout:1/70: mknod d3/d9/da/dd/de/c1d 0 2024-04-12T20:55:57.491 INFO:tasks.workunit.client.0.smithi018.stdout:5/110: creat d0/d2/d11/f29 x:0 0 0 2024-04-12T20:55:57.491 INFO:tasks.workunit.client.0.smithi018.stdout:7/146: mknod d1/db/c39 0 2024-04-12T20:55:57.493 INFO:tasks.workunit.client.0.smithi018.stdout:4/107: creat d3/d9/f1a x:0 0 0 2024-04-12T20:55:57.495 INFO:tasks.workunit.client.0.smithi018.stdout:7/147: truncate d1/db/f35 1195207 0 2024-04-12T20:55:57.497 INFO:tasks.workunit.client.0.smithi018.stdout:8/148: mknod db/d25/d26/d27/c30 0 2024-04-12T20:55:57.498 INFO:tasks.workunit.client.0.smithi018.stdout:7/148: chown d1/d6/d8/f2c 132402620 1 2024-04-12T20:55:57.501 INFO:tasks.workunit.client.0.smithi018.stdout:8/149: read db/d25/f2c [264501,1796] 0 2024-04-12T20:55:57.502 INFO:tasks.workunit.client.0.smithi018.stdout:0/136: dread da/fc [0,4194304] 0 2024-04-12T20:55:57.510 INFO:tasks.workunit.client.0.smithi018.stdout:2/115: creat d5/d8/d10/d19/d24/f29 x:0 0 0 2024-04-12T20:55:57.514 INFO:tasks.workunit.client.0.smithi018.stdout:2/116: chown d5/d8/d10/l1f 412 1 2024-04-12T20:55:57.518 INFO:tasks.workunit.client.0.smithi018.stdout:1/71: rename d3/l15 to d3/d9/da/dd/de/d13/d19/l1e 0 2024-04-12T20:55:57.519 INFO:tasks.workunit.client.0.smithi018.stdout:4/108: mknod d3/db/c1b 0 2024-04-12T20:55:57.520 INFO:tasks.workunit.client.0.smithi018.stdout:7/149: dwrite d1/d6/f2d [0,4194304] 0 2024-04-12T20:55:57.524 INFO:tasks.workunit.client.0.smithi018.stdout:7/150: read d1/db/f22 [3497319,63966] 0 2024-04-12T20:55:57.525 INFO:tasks.workunit.client.0.smithi018.stdout:0/137: dread da/f25 [0,4194304] 0 2024-04-12T20:55:57.525 INFO:tasks.workunit.client.0.smithi018.stdout:4/109: chown d3/f14 6103444 1 2024-04-12T20:55:57.526 INFO:tasks.workunit.client.0.smithi018.stdout:1/72: write d3/d9/da/dd/de/f1b [50380,26236] 0 2024-04-12T20:55:57.529 INFO:tasks.workunit.client.0.smithi018.stdout:8/150: dwrite db/dc/f23 [0,4194304] 0 2024-04-12T20:55:57.529 INFO:tasks.workunit.client.0.smithi018.stdout:2/117: rmdir d5/d8/d10 39 2024-04-12T20:55:57.529 INFO:tasks.workunit.client.0.smithi018.stdout:1/73: write d3/f10 [17505,116541] 0 2024-04-12T20:55:57.534 INFO:tasks.workunit.client.0.smithi018.stdout:5/111: write d0/d2/d9/fe [2909788,68636] 0 2024-04-12T20:55:57.536 INFO:tasks.workunit.client.0.smithi018.stdout:1/74: write d3/f4 [2337227,117076] 0 2024-04-12T20:55:57.538 INFO:tasks.workunit.client.0.smithi018.stdout:5/112: write d0/d2/d9/f1d [830634,49902] 0 2024-04-12T20:55:57.539 INFO:tasks.workunit.client.0.smithi018.stdout:7/151: unlink d1/d6/d8/f2c 0 2024-04-12T20:55:57.541 INFO:tasks.workunit.client.0.smithi018.stdout:4/110: stat d3/c7 0 2024-04-12T20:55:57.543 INFO:tasks.workunit.client.0.smithi018.stdout:1/75: rename d3/d9 to d3/d9/da/d14/d1f 22 2024-04-12T20:55:57.546 INFO:tasks.workunit.client.0.smithi018.stdout:7/152: truncate d1/d6/f27 1210247 0 2024-04-12T20:55:57.546 INFO:tasks.workunit.client.0.smithi018.stdout:8/151: dread fa [0,4194304] 0 2024-04-12T20:55:57.547 INFO:tasks.workunit.client.0.smithi018.stdout:4/111: creat d3/d9/f1c x:0 0 0 2024-04-12T20:55:57.548 INFO:tasks.workunit.client.0.smithi018.stdout:2/118: fdatasync d5/d8/d10/f15 0 2024-04-12T20:55:57.554 INFO:tasks.workunit.client.0.smithi018.stdout:5/113: creat d0/d2/d11/f2a x:0 0 0 2024-04-12T20:55:57.554 INFO:tasks.workunit.client.0.smithi018.stdout:0/138: rmdir da/d15/d17/d26 0 2024-04-12T20:55:57.556 INFO:tasks.workunit.client.0.smithi018.stdout:4/112: symlink d3/db/l1d 0 2024-04-12T20:55:57.558 INFO:tasks.workunit.client.0.smithi018.stdout:2/119: rename d5/d7/l20 to d5/d8/d10/d19/l2a 0 2024-04-12T20:55:57.560 INFO:tasks.workunit.client.0.smithi018.stdout:8/152: read - db/d25/d26/d27/f2b zero size 2024-04-12T20:55:57.563 INFO:tasks.workunit.client.0.smithi018.stdout:5/114: unlink d0/ff 0 2024-04-12T20:55:57.569 INFO:tasks.workunit.client.0.smithi018.stdout:1/76: dwrite d3/d9/da/dd/f17 [0,4194304] 0 2024-04-12T20:55:57.573 INFO:tasks.workunit.client.0.smithi018.stdout:4/113: symlink d3/db/l1e 0 2024-04-12T20:55:57.574 INFO:tasks.workunit.client.0.smithi018.stdout:7/153: dwrite d1/f15 [0,4194304] 0 2024-04-12T20:55:57.576 INFO:tasks.workunit.client.0.smithi018.stdout:1/77: creat d3/d9/da/f20 x:0 0 0 2024-04-12T20:55:57.582 INFO:tasks.workunit.client.0.smithi018.stdout:4/114: mkdir d3/d10/d1f 0 2024-04-12T20:55:57.582 INFO:tasks.workunit.client.0.smithi018.stdout:2/120: mknod d5/d8/c2b 0 2024-04-12T20:55:57.588 INFO:tasks.workunit.client.0.smithi018.stdout:4/115: chown d3/f14 10489154 1 2024-04-12T20:55:57.589 INFO:tasks.workunit.client.0.smithi018.stdout:8/153: link db/c1c db/c31 0 2024-04-12T20:55:57.590 INFO:tasks.workunit.client.0.smithi018.stdout:4/116: readlink d3/lf 0 2024-04-12T20:55:57.593 INFO:tasks.workunit.client.0.smithi018.stdout:1/78: readlink d3/d9/da/dd/de/d13/d19/l1e 0 2024-04-12T20:55:57.594 INFO:tasks.workunit.client.0.smithi018.stdout:2/121: symlink d5/d8/d25/l2c 0 2024-04-12T20:55:57.597 INFO:tasks.workunit.client.0.smithi018.stdout:0/139: dwrite da/f13 [0,4194304] 0 2024-04-12T20:55:57.597 INFO:tasks.workunit.client.0.smithi018.stdout:4/117: unlink d3/d9/cc 0 2024-04-12T20:55:57.598 INFO:tasks.workunit.client.0.smithi018.stdout:0/140: stat da/d15/c16 0 2024-04-12T20:55:57.601 INFO:tasks.workunit.client.0.smithi018.stdout:8/154: symlink db/l32 0 2024-04-12T20:55:57.602 INFO:tasks.workunit.client.0.smithi018.stdout:8/155: truncate f6 1177086 0 2024-04-12T20:55:57.604 INFO:tasks.workunit.client.0.smithi018.stdout:1/79: mkdir d3/d9/da/dd/de/d13/d21 0 2024-04-12T20:55:57.606 INFO:tasks.workunit.client.0.smithi018.stdout:1/80: read d3/f10 [95681,65017] 0 2024-04-12T20:55:57.607 INFO:tasks.workunit.client.0.smithi018.stdout:4/118: unlink d3/d10/c16 0 2024-04-12T20:55:57.612 INFO:tasks.workunit.client.0.smithi018.stdout:4/119: write d3/f5 [1589590,37332] 0 2024-04-12T20:55:57.613 INFO:tasks.workunit.client.0.smithi018.stdout:1/81: symlink d3/d9/da/dd/de/d13/l22 0 2024-04-12T20:55:57.614 INFO:tasks.workunit.client.0.smithi018.stdout:3/112: sync 2024-04-12T20:55:57.614 INFO:tasks.workunit.client.0.smithi018.stdout:6/229: sync 2024-04-12T20:55:57.617 INFO:tasks.workunit.client.0.smithi018.stdout:1/82: readlink d3/d9/da/dd/l16 0 2024-04-12T20:55:57.619 INFO:tasks.workunit.client.0.smithi018.stdout:3/113: rmdir d2/d6 39 2024-04-12T20:55:57.621 INFO:tasks.workunit.client.0.smithi018.stdout:4/120: symlink d3/l20 0 2024-04-12T20:55:57.622 INFO:tasks.workunit.client.0.smithi018.stdout:5/115: fsync d0/d2/d11/f2a 0 2024-04-12T20:55:57.626 INFO:tasks.workunit.client.0.smithi018.stdout:0/141: dread da/fc [0,4194304] 0 2024-04-12T20:55:57.627 INFO:tasks.workunit.client.0.smithi018.stdout:3/114: chown d2/d6 245 1 2024-04-12T20:55:57.634 INFO:tasks.workunit.client.0.smithi018.stdout:5/116: rename d0/d2/d1f to d0/d2b 0 2024-04-12T20:55:57.634 INFO:tasks.workunit.client.0.smithi018.stdout:0/142: rename da to da/d15/d2c 22 2024-04-12T20:55:57.635 INFO:tasks.workunit.client.0.smithi018.stdout:8/156: dwrite db/f1b [0,4194304] 0 2024-04-12T20:55:57.654 INFO:tasks.workunit.client.0.smithi018.stdout:8/157: creat db/d25/d26/d27/f33 x:0 0 0 2024-04-12T20:55:57.654 INFO:tasks.workunit.client.0.smithi018.stdout:4/121: link d3/db/c15 d3/d10/d1f/c21 0 2024-04-12T20:55:57.655 INFO:tasks.workunit.client.0.smithi018.stdout:0/143: mknod da/d15/d17/d21/d27/c2d 0 2024-04-12T20:55:57.657 INFO:tasks.workunit.client.0.smithi018.stdout:8/158: chown db/c16 9023 1 2024-04-12T20:55:57.657 INFO:tasks.workunit.client.0.smithi018.stdout:7/154: fsync d1/db/f35 0 2024-04-12T20:55:57.660 INFO:tasks.workunit.client.0.smithi018.stdout:1/83: getdents d3/d9/da/dd/de/d13/d19 0 2024-04-12T20:55:57.664 INFO:tasks.workunit.client.0.smithi018.stdout:8/159: fdatasync db/dc/f1e 0 2024-04-12T20:55:57.665 INFO:tasks.workunit.client.0.smithi018.stdout:3/115: write d2/f5 [1491378,96145] 0 2024-04-12T20:55:57.667 INFO:tasks.workunit.client.0.smithi018.stdout:7/155: mkdir d1/db/d31/d3a 0 2024-04-12T20:55:57.667 INFO:tasks.workunit.client.0.smithi018.stdout:0/144: fdatasync da/f25 0 2024-04-12T20:55:57.677 INFO:tasks.workunit.client.0.smithi018.stdout:3/116: mknod d2/d6/c21 0 2024-04-12T20:55:57.681 INFO:tasks.workunit.client.0.smithi018.stdout:2/122: dwrite f4 [0,4194304] 0 2024-04-12T20:55:57.682 INFO:tasks.workunit.client.0.smithi018.stdout:5/117: dwrite d0/dc/d10/f27 [0,4194304] 0 2024-04-12T20:55:57.687 INFO:tasks.workunit.client.0.smithi018.stdout:4/122: dwrite d3/f5 [0,4194304] 0 2024-04-12T20:55:57.695 INFO:tasks.workunit.client.0.smithi018.stdout:4/123: dread - d3/d9/f19 zero size 2024-04-12T20:55:57.696 INFO:tasks.workunit.client.0.smithi018.stdout:4/124: stat d3/d9 0 2024-04-12T20:55:57.696 INFO:tasks.workunit.client.0.smithi018.stdout:2/123: rmdir d5/d8 39 2024-04-12T20:55:57.696 INFO:tasks.workunit.client.0.smithi018.stdout:0/145: dread da/fc [0,4194304] 0 2024-04-12T20:55:57.700 INFO:tasks.workunit.client.0.smithi018.stdout:5/118: symlink d0/dc/d10/d26/l2c 0 2024-04-12T20:55:57.700 INFO:tasks.workunit.client.0.smithi018.stdout:7/156: creat d1/db/d31/d30/f3b x:0 0 0 2024-04-12T20:55:57.701 INFO:tasks.workunit.client.0.smithi018.stdout:0/146: stat da/d15/c1a 0 2024-04-12T20:55:57.701 INFO:tasks.workunit.client.0.smithi018.stdout:8/160: dwrite db/dc/f1a [0,4194304] 0 2024-04-12T20:55:57.708 INFO:tasks.workunit.client.0.smithi018.stdout:1/84: rmdir d3/d9/da/dd/de/d13/d21 0 2024-04-12T20:55:57.708 INFO:tasks.workunit.client.0.smithi018.stdout:7/157: truncate d1/f15 5079228 0 2024-04-12T20:55:57.708 INFO:tasks.workunit.client.0.smithi018.stdout:0/147: chown da/d15/d17/d21/d27 2073334 1 2024-04-12T20:55:57.708 INFO:tasks.workunit.client.0.smithi018.stdout:2/124: creat d5/d7/d28/f2d x:0 0 0 2024-04-12T20:55:57.708 INFO:tasks.workunit.client.0.smithi018.stdout:3/117: link d2/d6/cd d2/c22 0 2024-04-12T20:55:57.710 INFO:tasks.workunit.client.0.smithi018.stdout:7/158: write d1/f2 [1130717,12426] 0 2024-04-12T20:55:57.712 INFO:tasks.workunit.client.0.smithi018.stdout:1/85: creat d3/d9/da/dd/de/d13/f23 x:0 0 0 2024-04-12T20:55:57.712 INFO:tasks.workunit.client.0.smithi018.stdout:8/161: link f6 db/d25/f34 0 2024-04-12T20:55:57.713 INFO:tasks.workunit.client.0.smithi018.stdout:3/118: fdatasync d2/f14 0 2024-04-12T20:55:57.715 INFO:tasks.workunit.client.0.smithi018.stdout:4/125: creat d3/db/f22 x:0 0 0 2024-04-12T20:55:57.715 INFO:tasks.workunit.client.0.smithi018.stdout:7/159: chown d1/l1a 197914 1 2024-04-12T20:55:57.716 INFO:tasks.workunit.client.0.smithi018.stdout:4/126: stat d3/d10 0 2024-04-12T20:55:57.718 INFO:tasks.workunit.client.0.smithi018.stdout:8/162: write db/f1b [1361332,73606] 0 2024-04-12T20:55:57.719 INFO:tasks.workunit.client.0.smithi018.stdout:4/127: write d3/d9/f19 [132098,8056] 0 2024-04-12T20:55:57.721 INFO:tasks.workunit.client.0.smithi018.stdout:0/148: rename da/d15/l2b to da/d15/d17/l2e 0 2024-04-12T20:55:57.729 INFO:tasks.workunit.client.0.smithi018.stdout:1/86: mknod d3/d9/da/dd/de/d13/c24 0 2024-04-12T20:55:57.731 INFO:tasks.workunit.client.0.smithi018.stdout:8/163: mknod db/d25/c35 0 2024-04-12T20:55:57.734 INFO:tasks.workunit.client.0.smithi018.stdout:7/160: mkdir d1/d6/d3c 0 2024-04-12T20:55:57.748 INFO:tasks.workunit.client.0.smithi018.stdout:0/149: mknod da/d15/d17/d21/d27/c2f 0 2024-04-12T20:55:57.752 INFO:tasks.workunit.client.0.smithi018.stdout:5/119: dwrite d0/dc/f15 [4194304,4194304] 0 2024-04-12T20:55:57.752 INFO:tasks.workunit.client.0.smithi018.stdout:8/164: creat db/d25/d26/d27/f36 x:0 0 0 2024-04-12T20:55:57.754 INFO:tasks.workunit.client.0.smithi018.stdout:0/150: readlink da/l10 0 2024-04-12T20:55:57.760 INFO:tasks.workunit.client.0.smithi018.stdout:2/125: dwrite d5/f1d [0,4194304] 0 2024-04-12T20:55:57.761 INFO:tasks.workunit.client.0.smithi018.stdout:8/165: mknod db/c37 0 2024-04-12T20:55:57.770 INFO:tasks.workunit.client.0.smithi018.stdout:4/128: dwrite d3/d9/f1c [0,4194304] 0 2024-04-12T20:55:57.775 INFO:tasks.workunit.client.0.smithi018.stdout:0/151: symlink da/d15/l30 0 2024-04-12T20:55:57.791 INFO:tasks.workunit.client.0.smithi018.stdout:0/152: mknod da/d15/d17/d21/d27/c31 0 2024-04-12T20:55:57.801 INFO:tasks.workunit.client.0.smithi018.stdout:0/153: fdatasync da/f25 0 2024-04-12T20:55:57.803 INFO:tasks.workunit.client.0.smithi018.stdout:1/87: write d3/f10 [260687,85828] 0 2024-04-12T20:55:57.804 INFO:tasks.workunit.client.0.smithi018.stdout:7/161: write d1/fd [1060807,9603] 0 2024-04-12T20:55:57.820 INFO:tasks.workunit.client.0.smithi018.stdout:5/120: dwrite d0/d8/f12 [4194304,4194304] 0 2024-04-12T20:55:57.820 INFO:tasks.workunit.client.0.smithi018.stdout:4/129: dwrite d3/d10/f17 [0,4194304] 0 2024-04-12T20:55:57.822 INFO:tasks.workunit.client.0.smithi018.stdout:1/88: read - d3/d9/f18 zero size 2024-04-12T20:55:57.823 INFO:tasks.workunit.client.0.smithi018.stdout:2/126: dwrite d5/d8/f1e [0,4194304] 0 2024-04-12T20:55:57.824 INFO:tasks.workunit.client.0.smithi018.stdout:8/166: dwrite db/dc/f1f [0,4194304] 0 2024-04-12T20:55:57.824 INFO:tasks.workunit.client.0.smithi018.stdout:7/162: rmdir d1 39 2024-04-12T20:55:57.828 INFO:tasks.workunit.client.0.smithi018.stdout:4/130: write d3/d9/fd [470019,86831] 0 2024-04-12T20:55:57.845 INFO:tasks.workunit.client.0.smithi018.stdout:8/167: dread - db/dc/f1e zero size 2024-04-12T20:55:57.846 INFO:tasks.workunit.client.0.smithi018.stdout:1/89: chown d3/f7 32 1 2024-04-12T20:55:57.848 INFO:tasks.workunit.client.0.smithi018.stdout:3/119: dwrite d2/f4 [0,4194304] 0 2024-04-12T20:55:57.851 INFO:tasks.workunit.client.0.smithi018.stdout:3/120: chown d2/f4 141431 1 2024-04-12T20:55:57.851 INFO:tasks.workunit.client.0.smithi018.stdout:8/168: write db/d25/d26/d27/f33 [404887,73412] 0 2024-04-12T20:55:57.854 INFO:tasks.workunit.client.0.smithi018.stdout:8/169: chown f0 120 1 2024-04-12T20:55:57.858 INFO:tasks.workunit.client.0.smithi018.stdout:3/121: write d2/d6/f12 [1253862,33258] 0 2024-04-12T20:55:57.858 INFO:tasks.workunit.client.0.smithi018.stdout:0/154: dread da/f13 [0,4194304] 0 2024-04-12T20:55:57.858 INFO:tasks.workunit.client.0.smithi018.stdout:8/170: chown db/d25/d26 2 1 2024-04-12T20:55:57.861 INFO:tasks.workunit.client.0.smithi018.stdout:0/155: read da/f13 [417962,22957] 0 2024-04-12T20:55:57.861 INFO:tasks.workunit.client.0.smithi018.stdout:8/171: chown db/f1b 79735 1 2024-04-12T20:55:57.880 INFO:tasks.workunit.client.0.smithi018.stdout:9/171: dwrite d3/d6/d9/fd [0,4194304] 0 2024-04-12T20:55:57.885 INFO:tasks.workunit.client.0.smithi018.stdout:2/127: mknod d5/d8/d10/d19/c2e 0 2024-04-12T20:55:57.890 INFO:tasks.workunit.client.0.smithi018.stdout:8/172: symlink db/dc/l38 0 2024-04-12T20:55:57.890 INFO:tasks.workunit.client.0.smithi018.stdout:7/163: dwrite d1/fd [0,4194304] 0 2024-04-12T20:55:57.896 INFO:tasks.workunit.client.0.smithi018.stdout:5/121: link d0/d8/f22 d0/d2b/f2d 0 2024-04-12T20:55:57.899 INFO:tasks.workunit.client.0.smithi018.stdout:5/122: dread - d0/d8/f22 zero size 2024-04-12T20:55:57.901 INFO:tasks.workunit.client.0.smithi018.stdout:6/230: rename d5/d6/c4c to d5/d6/c5e 0 2024-04-12T20:55:57.902 INFO:tasks.workunit.client.0.smithi018.stdout:4/131: stat d3/db/c1b 0 2024-04-12T20:55:57.902 INFO:tasks.workunit.client.0.smithi018.stdout:5/123: chown d0/d2/l21 456975877 1 2024-04-12T20:55:57.902 INFO:tasks.workunit.client.0.smithi018.stdout:1/90: dwrite d3/f7 [0,4194304] 0 2024-04-12T20:55:57.912 INFO:tasks.workunit.client.0.smithi018.stdout:7/164: rename d1/db/d31/d30/f3b to d1/d6/d8/f3d 0 2024-04-12T20:55:57.912 INFO:tasks.workunit.client.0.smithi018.stdout:8/173: write fa [412609,59398] 0 2024-04-12T20:55:57.922 INFO:tasks.workunit.client.0.smithi018.stdout:4/132: dread d3/f5 [0,4194304] 0 2024-04-12T20:55:57.922 INFO:tasks.workunit.client.0.smithi018.stdout:9/172: mknod d3/d6/d9/dc/d1c/c38 0 2024-04-12T20:55:57.924 INFO:tasks.workunit.client.0.smithi018.stdout:1/91: mknod d3/d9/da/dd/de/d13/c25 0 2024-04-12T20:55:57.926 INFO:tasks.workunit.client.0.smithi018.stdout:6/231: mkdir d5/df/d19/d43/d5f 0 2024-04-12T20:55:57.927 INFO:tasks.workunit.client.0.smithi018.stdout:6/232: stat d5/f7 0 2024-04-12T20:55:57.928 INFO:tasks.workunit.client.0.smithi018.stdout:1/92: chown d3/d9/da/l1c 25706 1 2024-04-12T20:55:57.929 INFO:tasks.workunit.client.0.smithi018.stdout:8/174: dread db/dc/f1a [0,4194304] 0 2024-04-12T20:55:57.930 INFO:tasks.workunit.client.0.smithi018.stdout:9/173: symlink d3/d6/d9/dc/l39 0 2024-04-12T20:55:57.931 INFO:tasks.workunit.client.0.smithi018.stdout:8/175: read db/f1b [3413747,15624] 0 2024-04-12T20:55:57.933 INFO:tasks.workunit.client.0.smithi018.stdout:8/176: chown db/d25/d26/c2a 46 1 2024-04-12T20:55:57.933 INFO:tasks.workunit.client.0.smithi018.stdout:0/156: dread da/d15/f18 [0,4194304] 0 2024-04-12T20:55:57.936 INFO:tasks.workunit.client.0.smithi018.stdout:3/122: getdents d2 0 2024-04-12T20:55:57.937 INFO:tasks.workunit.client.0.smithi018.stdout:0/157: read da/d15/d17/f19 [1988292,105347] 0 2024-04-12T20:55:57.937 INFO:tasks.workunit.client.0.smithi018.stdout:6/233: mkdir d5/d15/d60 0 2024-04-12T20:55:57.938 INFO:tasks.workunit.client.0.smithi018.stdout:5/124: rename d0/d8 to d0/d2/d9/d1e/d2e 0 2024-04-12T20:55:57.939 INFO:tasks.workunit.client.0.smithi018.stdout:6/234: fdatasync d5/f48 0 2024-04-12T20:55:57.941 INFO:tasks.workunit.client.0.smithi018.stdout:6/235: fsync d5/d15/f39 0 2024-04-12T20:55:57.945 INFO:tasks.workunit.client.0.smithi018.stdout:1/93: mkdir d3/d9/da/dd/d26 0 2024-04-12T20:55:57.946 INFO:tasks.workunit.client.0.smithi018.stdout:8/177: symlink db/d25/l39 0 2024-04-12T20:55:57.947 INFO:tasks.workunit.client.0.smithi018.stdout:3/123: chown d2/c22 43 1 2024-04-12T20:55:57.948 INFO:tasks.workunit.client.0.smithi018.stdout:0/158: write da/f13 [3077915,66827] 0 2024-04-12T20:55:57.951 INFO:tasks.workunit.client.0.smithi018.stdout:0/159: chown da/d15/d17/c1e 13134 1 2024-04-12T20:55:57.964 INFO:tasks.workunit.client.0.smithi018.stdout:0/160: dread da/f22 [0,4194304] 0 2024-04-12T20:55:57.976 INFO:tasks.workunit.client.0.smithi018.stdout:2/128: link d5/d8/c21 d5/c2f 0 2024-04-12T20:55:57.978 INFO:tasks.workunit.client.0.smithi018.stdout:0/161: dread da/f25 [0,4194304] 0 2024-04-12T20:55:57.979 INFO:tasks.workunit.client.0.smithi018.stdout:2/129: readlink d5/d8/d10/d19/l27 0 2024-04-12T20:55:57.980 INFO:tasks.workunit.client.0.smithi018.stdout:7/165: creat d1/f3e x:0 0 0 2024-04-12T20:55:57.980 INFO:tasks.workunit.client.0.smithi018.stdout:5/125: mkdir d0/dc/d10/d2f 0 2024-04-12T20:55:57.982 INFO:tasks.workunit.client.0.smithi018.stdout:2/130: write f3 [4066505,115697] 0 2024-04-12T20:55:57.984 INFO:tasks.workunit.client.0.smithi018.stdout:4/133: getdents d3/db 0 2024-04-12T20:55:57.985 INFO:tasks.workunit.client.0.smithi018.stdout:3/124: dread d2/d6/fc [0,4194304] 0 2024-04-12T20:55:57.987 INFO:tasks.workunit.client.0.smithi018.stdout:3/125: stat d2/f4 0 2024-04-12T20:55:57.988 INFO:tasks.workunit.client.0.smithi018.stdout:2/131: creat d5/d8/d10/f30 x:0 0 0 2024-04-12T20:55:57.989 INFO:tasks.workunit.client.0.smithi018.stdout:4/134: creat d3/db/f23 x:0 0 0 2024-04-12T20:55:57.989 INFO:tasks.workunit.client.0.smithi018.stdout:3/126: fdatasync d2/d6/f1c 0 2024-04-12T20:55:57.991 INFO:tasks.workunit.client.0.smithi018.stdout:9/174: rename d3/d6/d9/dc/d13/d28 to d3/d6/d9/d1e/d3a 0 2024-04-12T20:55:57.991 INFO:tasks.workunit.client.0.smithi018.stdout:0/162: dread da/d15/f18 [0,4194304] 0 2024-04-12T20:55:57.991 INFO:tasks.workunit.client.0.smithi018.stdout:4/135: stat d3/c7 0 2024-04-12T20:55:57.992 INFO:tasks.workunit.client.0.smithi018.stdout:7/166: symlink d1/l3f 0 2024-04-12T20:55:57.995 INFO:tasks.workunit.client.0.smithi018.stdout:4/136: write d3/f14 [483628,36831] 0 2024-04-12T20:55:57.997 INFO:tasks.workunit.client.0.smithi018.stdout:5/126: dread d0/d2/d9/f1d [0,4194304] 0 2024-04-12T20:55:57.997 INFO:tasks.workunit.client.0.smithi018.stdout:2/132: mkdir d5/d7/d31 0 2024-04-12T20:55:57.998 INFO:tasks.workunit.client.0.smithi018.stdout:0/163: creat da/d15/d17/f32 x:0 0 0 2024-04-12T20:55:58.000 INFO:tasks.workunit.client.0.smithi018.stdout:5/127: fsync d0/dc/d10/f17 0 2024-04-12T20:55:58.002 INFO:tasks.workunit.client.0.smithi018.stdout:7/167: rename d1/db/c39 to d1/db/d31/d3a/c40 0 2024-04-12T20:55:58.002 INFO:tasks.workunit.client.0.smithi018.stdout:5/128: chown d0/dc/d10/d26 108259 1 2024-04-12T20:55:58.008 INFO:tasks.workunit.client.0.smithi018.stdout:0/164: rename da/d15/f18 to da/d15/f33 0 2024-04-12T20:55:58.010 INFO:tasks.workunit.client.0.smithi018.stdout:7/168: creat d1/db/d31/d3a/f41 x:0 0 0 2024-04-12T20:55:58.011 INFO:tasks.workunit.client.0.smithi018.stdout:5/129: rmdir d0/d2/d9/d1e 39 2024-04-12T20:55:58.014 INFO:tasks.workunit.client.0.smithi018.stdout:0/165: creat da/d15/f34 x:0 0 0 2024-04-12T20:55:58.015 INFO:tasks.workunit.client.0.smithi018.stdout:3/127: dwrite d2/d6/f1b [0,4194304] 0 2024-04-12T20:55:58.017 INFO:tasks.workunit.client.0.smithi018.stdout:9/175: dwrite d3/d6/d9/fd [0,4194304] 0 2024-04-12T20:55:58.027 INFO:tasks.workunit.client.0.smithi018.stdout:7/169: getdents d1/db/d31/d30 0 2024-04-12T20:55:58.030 INFO:tasks.workunit.client.0.smithi018.stdout:4/137: creat d3/db/f24 x:0 0 0 2024-04-12T20:55:58.032 INFO:tasks.workunit.client.0.smithi018.stdout:0/166: dwrite da/f25 [0,4194304] 0 2024-04-12T20:55:58.032 INFO:tasks.workunit.client.0.smithi018.stdout:4/138: write d3/f14 [726865,57350] 0 2024-04-12T20:55:58.036 INFO:tasks.workunit.client.0.smithi018.stdout:7/170: rename d1/d6/d3c to d1/db/d31/d3a/d42 0 2024-04-12T20:55:58.037 INFO:tasks.workunit.client.0.smithi018.stdout:0/167: creat da/d15/f35 x:0 0 0 2024-04-12T20:55:58.039 INFO:tasks.workunit.client.0.smithi018.stdout:7/171: creat d1/db/f43 x:0 0 0 2024-04-12T20:55:58.042 INFO:tasks.workunit.client.0.smithi018.stdout:4/139: mknod d3/d10/d1f/c25 0 2024-04-12T20:55:58.045 INFO:tasks.workunit.client.0.smithi018.stdout:3/128: truncate d2/d6/fc 724549 0 2024-04-12T20:55:58.045 INFO:tasks.workunit.client.0.smithi018.stdout:0/168: mkdir da/d15/d17/d21/d36 0 2024-04-12T20:55:58.048 INFO:tasks.workunit.client.0.smithi018.stdout:7/172: rmdir d1/d38 0 2024-04-12T20:55:58.049 INFO:tasks.workunit.client.0.smithi018.stdout:0/169: mknod da/d15/d17/d21/d36/c37 0 2024-04-12T20:55:58.050 INFO:tasks.workunit.client.0.smithi018.stdout:7/173: fdatasync d1/db/f2b 0 2024-04-12T20:55:58.051 INFO:tasks.workunit.client.0.smithi018.stdout:0/170: stat da/d15/d17/d21/d36 0 2024-04-12T20:55:58.052 INFO:tasks.workunit.client.0.smithi018.stdout:7/174: unlink d1/d6/f2d 0 2024-04-12T20:55:58.054 INFO:tasks.workunit.client.0.smithi018.stdout:0/171: creat da/f38 x:0 0 0 2024-04-12T20:55:58.055 INFO:tasks.workunit.client.0.smithi018.stdout:7/175: creat d1/db/d31/d3a/d42/f44 x:0 0 0 2024-04-12T20:55:58.057 INFO:tasks.workunit.client.0.smithi018.stdout:7/176: rmdir d1/d6/d8 39 2024-04-12T20:55:58.060 INFO:tasks.workunit.client.0.smithi018.stdout:7/177: unlink d1/d6/d8/f3d 0 2024-04-12T20:55:58.062 INFO:tasks.workunit.client.0.smithi018.stdout:7/178: mkdir d1/db/d31/d45 0 2024-04-12T20:55:58.069 INFO:tasks.workunit.client.0.smithi018.stdout:4/140: truncate d3/f5 2861393 0 2024-04-12T20:55:58.073 INFO:tasks.workunit.client.0.smithi018.stdout:7/179: dread d1/db/f22 [0,4194304] 0 2024-04-12T20:55:58.079 INFO:tasks.workunit.client.0.smithi018.stdout:7/180: dread d1/d6/f27 [0,4194304] 0 2024-04-12T20:55:58.080 INFO:tasks.workunit.client.0.smithi018.stdout:7/181: truncate d1/f32 635619 0 2024-04-12T20:55:58.085 INFO:tasks.workunit.client.0.smithi018.stdout:7/182: rename d1/d6/d8/c16 to d1/d6/c46 0 2024-04-12T20:55:58.092 INFO:tasks.workunit.client.0.smithi018.stdout:4/141: dread d3/d10/f18 [0,4194304] 0 2024-04-12T20:55:58.094 INFO:tasks.workunit.client.0.smithi018.stdout:4/142: truncate d3/d9/f1a 1023636 0 2024-04-12T20:55:58.098 INFO:tasks.workunit.client.0.smithi018.stdout:4/143: creat d3/db/f26 x:0 0 0 2024-04-12T20:55:58.100 INFO:tasks.workunit.client.0.smithi018.stdout:7/183: dwrite d1/db/d31/d3a/d42/f44 [0,4194304] 0 2024-04-12T20:55:58.107 INFO:tasks.workunit.client.0.smithi018.stdout:4/144: creat d3/d10/d1f/f27 x:0 0 0 2024-04-12T20:55:58.119 INFO:tasks.workunit.client.0.smithi018.stdout:7/184: dwrite d1/f1b [0,4194304] 0 2024-04-12T20:55:58.123 INFO:tasks.workunit.client.0.smithi018.stdout:4/145: dwrite d3/f14 [0,4194304] 0 2024-04-12T20:55:58.127 INFO:tasks.workunit.client.0.smithi018.stdout:4/146: unlink d3/d9/f19 0 2024-04-12T20:55:58.134 INFO:tasks.workunit.client.0.smithi018.stdout:7/185: dwrite d1/f32 [0,4194304] 0 2024-04-12T20:55:58.137 INFO:tasks.workunit.client.0.smithi018.stdout:7/186: symlink d1/db/l47 0 2024-04-12T20:55:58.138 INFO:tasks.workunit.client.0.smithi018.stdout:7/187: stat d1/db/l20 0 2024-04-12T20:55:58.140 INFO:tasks.workunit.client.0.smithi018.stdout:7/188: rmdir d1/d6 39 2024-04-12T20:55:58.142 INFO:tasks.workunit.client.0.smithi018.stdout:7/189: chown d1/db/f22 1947 1 2024-04-12T20:55:58.143 INFO:tasks.workunit.client.0.smithi018.stdout:4/147: dwrite d3/d10/f18 [0,4194304] 0 2024-04-12T20:55:58.145 INFO:tasks.workunit.client.0.smithi018.stdout:7/190: symlink d1/db/d31/l48 0 2024-04-12T20:55:58.145 INFO:tasks.workunit.client.0.smithi018.stdout:4/148: chown d3/db/f22 165 1 2024-04-12T20:55:58.149 INFO:tasks.workunit.client.0.smithi018.stdout:7/191: write d1/d6/d8/f4 [943349,85004] 0 2024-04-12T20:55:58.151 INFO:tasks.workunit.client.0.smithi018.stdout:7/192: truncate d1/f14 8681019 0 2024-04-12T20:55:58.155 INFO:tasks.workunit.client.0.smithi018.stdout:7/193: symlink d1/db/d31/l49 0 2024-04-12T20:55:58.159 INFO:tasks.workunit.client.0.smithi018.stdout:7/194: creat d1/db/d31/d30/f4a x:0 0 0 2024-04-12T20:55:58.161 INFO:tasks.workunit.client.0.smithi018.stdout:7/195: symlink d1/db/d31/l4b 0 2024-04-12T20:55:58.164 INFO:tasks.workunit.client.0.smithi018.stdout:7/196: mkdir d1/db/d31/d3a/d42/d4c 0 2024-04-12T20:55:58.165 INFO:tasks.workunit.client.0.smithi018.stdout:7/197: fdatasync d1/f14 0 2024-04-12T20:55:58.168 INFO:tasks.workunit.client.0.smithi018.stdout:7/198: rename d1/f14 to d1/db/d31/f4d 0 2024-04-12T20:55:58.170 INFO:tasks.workunit.client.0.smithi018.stdout:7/199: stat d1/d6/d8/c23 0 2024-04-12T20:55:58.172 INFO:tasks.workunit.client.0.smithi018.stdout:7/200: symlink d1/l4e 0 2024-04-12T20:55:58.174 INFO:tasks.workunit.client.0.smithi018.stdout:7/201: chown d1/db/d31/l49 1344 1 2024-04-12T20:55:58.176 INFO:tasks.workunit.client.0.smithi018.stdout:7/202: mkdir d1/db/d31/d3a/d4f 0 2024-04-12T20:55:58.190 INFO:tasks.workunit.client.0.smithi018.stdout:4/149: truncate d3/d10/f17 1299817 0 2024-04-12T20:55:58.192 INFO:tasks.workunit.client.0.smithi018.stdout:4/150: symlink d3/d10/l28 0 2024-04-12T20:55:58.194 INFO:tasks.workunit.client.0.smithi018.stdout:7/203: dwrite d1/d6/d8/f4 [0,4194304] 0 2024-04-12T20:55:58.211 INFO:tasks.workunit.client.0.smithi018.stdout:7/204: dwrite d1/f2 [0,4194304] 0 2024-04-12T20:55:58.217 INFO:tasks.workunit.client.0.smithi018.stdout:4/151: dwrite d3/db/f24 [0,4194304] 0 2024-04-12T20:55:58.220 INFO:tasks.workunit.client.0.smithi018.stdout:4/152: creat d3/d10/f29 x:0 0 0 2024-04-12T20:55:58.225 INFO:tasks.workunit.client.0.smithi018.stdout:7/205: rmdir d1/db 39 2024-04-12T20:55:58.235 INFO:tasks.workunit.client.0.smithi018.stdout:7/206: symlink d1/db/d31/d30/l50 0 2024-04-12T20:55:58.235 INFO:tasks.workunit.client.0.smithi018.stdout:4/153: creat d3/d10/d1f/f2a x:0 0 0 2024-04-12T20:55:58.237 INFO:tasks.workunit.client.0.smithi018.stdout:4/154: dread - d3/db/f26 zero size 2024-04-12T20:55:58.239 INFO:tasks.workunit.client.0.smithi018.stdout:4/155: write d3/d10/d1f/f2a [77550,39764] 0 2024-04-12T20:55:58.239 INFO:tasks.workunit.client.0.smithi018.stdout:7/207: creat d1/db/d31/d3a/d42/d4c/f51 x:0 0 0 2024-04-12T20:55:58.242 INFO:tasks.workunit.client.0.smithi018.stdout:4/156: chown d3/db/f26 0 1 2024-04-12T20:55:58.244 INFO:tasks.workunit.client.0.smithi018.stdout:4/157: chown d3/d10/f18 287654 1 2024-04-12T20:55:58.245 INFO:tasks.workunit.client.0.smithi018.stdout:4/158: read d3/db/f24 [1843694,21858] 0 2024-04-12T20:55:58.247 INFO:tasks.workunit.client.0.smithi018.stdout:4/159: write d3/d10/d1f/f2a [330836,120352] 0 2024-04-12T20:55:58.252 INFO:tasks.workunit.client.0.smithi018.stdout:4/160: rename d3/d10/d1f/f27 to d3/db/f2b 0 2024-04-12T20:55:58.254 INFO:tasks.workunit.client.0.smithi018.stdout:7/208: truncate d1/d6/f27 2662479 0 2024-04-12T20:55:58.256 INFO:tasks.workunit.client.0.smithi018.stdout:4/161: creat d3/d10/d1f/f2c x:0 0 0 2024-04-12T20:55:58.257 INFO:tasks.workunit.client.0.smithi018.stdout:7/209: creat d1/d6/f52 x:0 0 0 2024-04-12T20:55:58.258 INFO:tasks.workunit.client.0.smithi018.stdout:4/162: truncate d3/d9/fd 1407307 0 2024-04-12T20:55:58.263 INFO:tasks.workunit.client.0.smithi018.stdout:4/163: rename d3/d9/f1c to d3/f2d 0 2024-04-12T20:55:58.266 INFO:tasks.workunit.client.0.smithi018.stdout:4/164: fdatasync d3/f5 0 2024-04-12T20:55:58.269 INFO:tasks.workunit.client.0.smithi018.stdout:7/210: dread d1/f15 [0,4194304] 0 2024-04-12T20:55:58.271 INFO:tasks.workunit.client.0.smithi018.stdout:7/211: fsync d1/d6/f52 0 2024-04-12T20:55:58.281 INFO:tasks.workunit.client.0.smithi018.stdout:9/176: truncate d3/d6/d9/dc/d1c/f2e 1807242 0 2024-04-12T20:55:58.281 INFO:tasks.workunit.client.0.smithi018.stdout:4/165: dread d3/f2d [0,4194304] 0 2024-04-12T20:55:58.284 INFO:tasks.workunit.client.0.smithi018.stdout:9/177: truncate d3/d6/d9/f2f 966391 0 2024-04-12T20:55:58.289 INFO:tasks.workunit.client.0.smithi018.stdout:9/178: mknod d3/d6/d9/dc/d13/c3b 0 2024-04-12T20:55:58.289 INFO:tasks.workunit.client.0.smithi018.stdout:7/212: dwrite d1/d6/f27 [0,4194304] 0 2024-04-12T20:55:58.290 INFO:tasks.workunit.client.0.smithi018.stdout:4/166: write d3/db/f24 [4328961,24098] 0 2024-04-12T20:55:58.293 INFO:tasks.workunit.client.0.smithi018.stdout:9/179: truncate d3/d6/d1b/f26 915297 0 2024-04-12T20:55:58.294 INFO:tasks.workunit.client.0.smithi018.stdout:7/213: write d1/db/d31/d3a/f41 [99339,42083] 0 2024-04-12T20:55:58.297 INFO:tasks.workunit.client.0.smithi018.stdout:9/180: creat d3/d6/f3c x:0 0 0 2024-04-12T20:55:58.300 INFO:tasks.workunit.client.0.smithi018.stdout:7/214: mkdir d1/db/d31/d45/d53 0 2024-04-12T20:55:58.301 INFO:tasks.workunit.client.0.smithi018.stdout:9/181: chown d3/d6/c1a 200379 1 2024-04-12T20:55:58.302 INFO:tasks.workunit.client.0.smithi018.stdout:7/215: readlink d1/db/l1e 0 2024-04-12T20:55:58.303 INFO:tasks.workunit.client.0.smithi018.stdout:9/182: readlink d3/d6/d9/dc/d13/l2a 0 2024-04-12T20:55:58.306 INFO:tasks.workunit.client.0.smithi018.stdout:9/183: truncate d3/d6/f30 880029 0 2024-04-12T20:55:58.309 INFO:tasks.workunit.client.0.smithi018.stdout:9/184: chown d3/d6/d9/dc/d13/l2a 11198 1 2024-04-12T20:55:58.310 INFO:tasks.workunit.client.0.smithi018.stdout:4/167: dwrite d3/f5 [0,4194304] 0 2024-04-12T20:55:58.312 INFO:tasks.workunit.client.0.smithi018.stdout:4/168: write d3/d10/f18 [3900061,45174] 0 2024-04-12T20:55:58.316 INFO:tasks.workunit.client.0.smithi018.stdout:7/216: dread d1/db/f35 [0,4194304] 0 2024-04-12T20:55:58.318 INFO:tasks.workunit.client.0.smithi018.stdout:9/185: mkdir d3/d6/d3d 0 2024-04-12T20:55:58.318 INFO:tasks.workunit.client.0.smithi018.stdout:4/169: getdents d3/d10/d1f 0 2024-04-12T20:55:58.322 INFO:tasks.workunit.client.0.smithi018.stdout:9/186: chown d3/d6/d9/dc/d1c/c38 155043 1 2024-04-12T20:55:58.323 INFO:tasks.workunit.client.0.smithi018.stdout:9/187: stat d3/d6/d9/d1e/d3a 0 2024-04-12T20:55:58.338 INFO:tasks.workunit.client.0.smithi018.stdout:9/188: dwrite d3/d6/d9/fd [0,4194304] 0 2024-04-12T20:55:58.340 INFO:tasks.workunit.client.0.smithi018.stdout:9/189: chown d3/d6/d1b/f35 118737814 1 2024-04-12T20:55:58.341 INFO:tasks.workunit.client.0.smithi018.stdout:9/190: chown d3/d6/f10 183 1 2024-04-12T20:55:58.344 INFO:tasks.workunit.client.0.smithi018.stdout:9/191: symlink d3/d6/d9/d1e/l3e 0 2024-04-12T20:55:58.348 INFO:tasks.workunit.client.0.smithi018.stdout:9/192: mkdir d3/d6/d9/d3f 0 2024-04-12T20:55:58.349 INFO:tasks.workunit.client.0.smithi018.stdout:9/193: fdatasync d3/d6/f30 0 2024-04-12T20:55:58.359 INFO:tasks.workunit.client.0.smithi018.stdout:9/194: dread d3/d6/d9/fa [0,4194304] 0 2024-04-12T20:55:58.361 INFO:tasks.workunit.client.0.smithi018.stdout:9/195: write d3/d6/f1f [1246946,59342] 0 2024-04-12T20:55:58.362 INFO:tasks.workunit.client.0.smithi018.stdout:9/196: stat d3/d6/d9/f2f 0 2024-04-12T20:55:58.365 INFO:tasks.workunit.client.0.smithi018.stdout:9/197: write d3/d6/d9/d1e/d3a/f31 [994721,17371] 0 2024-04-12T20:55:58.368 INFO:tasks.workunit.client.0.smithi018.stdout:9/198: creat d3/d6/d9/d1e/f40 x:0 0 0 2024-04-12T20:55:58.369 INFO:tasks.workunit.client.0.smithi018.stdout:9/199: chown d3/d6/d9/d3f 4182 1 2024-04-12T20:55:58.371 INFO:tasks.workunit.client.0.smithi018.stdout:9/200: mkdir d3/d6/d9/dc/d1c/d21/d41 0 2024-04-12T20:55:58.372 INFO:tasks.workunit.client.0.smithi018.stdout:9/201: chown d3/d6/d9/dc/d1c 129626150 1 2024-04-12T20:55:58.373 INFO:tasks.workunit.client.0.smithi018.stdout:9/202: stat d3/l15 0 2024-04-12T20:55:58.413 INFO:tasks.workunit.client.0.smithi018.stdout:9/203: dread d3/d6/f10 [0,4194304] 0 2024-04-12T20:55:58.489 INFO:tasks.workunit.client.0.smithi018.stdout:4/170: link d3/d10/f29 d3/d10/d1f/f2e 0 2024-04-12T20:55:58.490 INFO:tasks.workunit.client.0.smithi018.stdout:9/204: write d3/d6/d9/dc/d1c/f34 [3196783,86380] 0 2024-04-12T20:55:58.498 INFO:tasks.workunit.client.0.smithi018.stdout:9/205: creat d3/d6/d9/dc/d13/f42 x:0 0 0 2024-04-12T20:55:58.509 INFO:tasks.workunit.client.0.smithi018.stdout:4/171: dread d3/d10/f18 [0,4194304] 0 2024-04-12T20:55:58.510 INFO:tasks.workunit.client.0.smithi018.stdout:9/206: unlink d3/d6/d9/f2f 0 2024-04-12T20:55:58.511 INFO:tasks.workunit.client.0.smithi018.stdout:4/172: rmdir d3 39 2024-04-12T20:55:58.514 INFO:tasks.workunit.client.0.smithi018.stdout:4/173: read - d3/db/f26 zero size 2024-04-12T20:55:58.530 INFO:tasks.workunit.client.0.smithi018.stdout:4/174: rename d3/f5 to d3/d10/d1f/f2f 0 2024-04-12T20:55:58.535 INFO:tasks.workunit.client.0.smithi018.stdout:9/207: dwrite d3/f23 [4194304,4194304] 0 2024-04-12T20:55:58.541 INFO:tasks.workunit.client.0.smithi018.stdout:9/208: rmdir d3/d6/d9/dc/d1c 39 2024-04-12T20:55:58.542 INFO:tasks.workunit.client.0.smithi018.stdout:9/209: chown d3/d6 190811851 1 2024-04-12T20:55:58.543 INFO:tasks.workunit.client.0.smithi018.stdout:9/210: fsync d3/d6/d9/d1e/f40 0 2024-04-12T20:55:58.546 INFO:tasks.workunit.client.0.smithi018.stdout:4/175: dread d3/d10/f18 [0,4194304] 0 2024-04-12T20:55:58.546 INFO:tasks.workunit.client.0.smithi018.stdout:9/211: read d3/d6/d9/dc/d13/f17 [2829389,74722] 0 2024-04-12T20:55:58.548 INFO:tasks.workunit.client.0.smithi018.stdout:4/176: creat d3/d10/f30 x:0 0 0 2024-04-12T20:55:58.549 INFO:tasks.workunit.client.0.smithi018.stdout:9/212: write d3/d6/d9/d1e/f40 [14537,93402] 0 2024-04-12T20:55:58.551 INFO:tasks.workunit.client.0.smithi018.stdout:9/213: mkdir d3/d6/d43 0 2024-04-12T20:55:58.554 INFO:tasks.workunit.client.0.smithi018.stdout:9/214: readlink d3/d6/d9/dc/d1c/l33 0 2024-04-12T20:55:58.556 INFO:tasks.workunit.client.0.smithi018.stdout:9/215: write d3/d6/f12 [1718779,111541] 0 2024-04-12T20:55:58.561 INFO:tasks.workunit.client.0.smithi018.stdout:4/177: rmdir d3/d10 39 2024-04-12T20:55:58.567 INFO:tasks.workunit.client.0.smithi018.stdout:4/178: symlink d3/d10/d1f/l31 0 2024-04-12T20:55:58.569 INFO:tasks.workunit.client.0.smithi018.stdout:4/179: write d3/db/f24 [5077684,800] 0 2024-04-12T20:55:58.572 INFO:tasks.workunit.client.0.smithi018.stdout:4/180: truncate d3/db/f23 979429 0 2024-04-12T20:55:58.573 INFO:tasks.workunit.client.0.smithi018.stdout:4/181: stat d3/db/f24 0 2024-04-12T20:55:58.579 INFO:tasks.workunit.client.0.smithi018.stdout:9/216: dwrite d3/d6/d9/dc/d1c/f2e [0,4194304] 0 2024-04-12T20:55:58.580 INFO:tasks.workunit.client.0.smithi018.stdout:4/182: rename d3/d9 to d3/d32 0 2024-04-12T20:55:58.582 INFO:tasks.workunit.client.0.smithi018.stdout:4/183: dread - d3/db/f2b zero size 2024-04-12T20:55:58.587 INFO:tasks.workunit.client.0.smithi018.stdout:4/184: rename d3/d10 to d3/d32/d33 0 2024-04-12T20:55:58.588 INFO:tasks.workunit.client.0.smithi018.stdout:4/185: readlink d3/l20 0 2024-04-12T20:55:58.592 INFO:tasks.workunit.client.0.smithi018.stdout:4/186: unlink d3/d32/d33/d1f/c25 0 2024-04-12T20:55:58.603 INFO:tasks.workunit.client.0.smithi018.stdout:9/217: dread d3/d6/f12 [0,4194304] 0 2024-04-12T20:55:58.606 INFO:tasks.workunit.client.0.smithi018.stdout:4/187: dread d3/d32/d33/d1f/f2f [0,4194304] 0 2024-04-12T20:55:58.609 INFO:tasks.workunit.client.0.smithi018.stdout:4/188: mkdir d3/d34 0 2024-04-12T20:55:58.612 INFO:tasks.workunit.client.0.smithi018.stdout:4/189: creat d3/f35 x:0 0 0 2024-04-12T20:55:58.616 INFO:tasks.workunit.client.0.smithi018.stdout:9/218: dread d3/d6/f1f [0,4194304] 0 2024-04-12T20:55:58.618 INFO:tasks.workunit.client.0.smithi018.stdout:9/219: chown d3/d6/d9/d1e/l3e 3180 1 2024-04-12T20:55:58.622 INFO:tasks.workunit.client.0.smithi018.stdout:9/220: creat d3/d6/d9/d3f/f44 x:0 0 0 2024-04-12T20:55:58.628 INFO:tasks.workunit.client.0.smithi018.stdout:4/190: dwrite d3/d32/d33/f30 [0,4194304] 0 2024-04-12T20:55:58.629 INFO:tasks.workunit.client.0.smithi018.stdout:4/191: dread - d3/f35 zero size 2024-04-12T20:55:58.715 INFO:tasks.workunit.client.0.smithi018.stdout:4/192: write d3/d32/d33/d1f/f2e [715783,111368] 0 2024-04-12T20:55:58.723 INFO:tasks.workunit.client.0.smithi018.stdout:9/221: dwrite d3/d6/f3c [0,4194304] 0 2024-04-12T20:55:58.723 INFO:tasks.workunit.client.0.smithi018.stdout:4/193: write d3/d32/d33/f29 [437691,71278] 0 2024-04-12T20:55:58.727 INFO:tasks.workunit.client.0.smithi018.stdout:4/194: chown d3/db/f26 64505 1 2024-04-12T20:55:58.736 INFO:tasks.workunit.client.0.smithi018.stdout:4/195: chown d3/db/l1d 253758214 1 2024-04-12T20:55:58.740 INFO:tasks.workunit.client.0.smithi018.stdout:9/222: dread d3/d6/d1b/f35 [0,4194304] 0 2024-04-12T20:55:58.744 INFO:tasks.workunit.client.0.smithi018.stdout:9/223: readlink d3/d6/d9/dc/l37 0 2024-04-12T20:55:58.750 INFO:tasks.workunit.client.0.smithi018.stdout:9/224: creat d3/d6/d9/d1e/f45 x:0 0 0 2024-04-12T20:55:58.752 INFO:tasks.workunit.client.0.smithi018.stdout:9/225: fdatasync d3/d6/d9/d3f/f44 0 2024-04-12T20:55:58.759 INFO:tasks.workunit.client.0.smithi018.stdout:9/226: mkdir d3/d6/d9/dc/d46 0 2024-04-12T20:55:58.765 INFO:tasks.workunit.client.0.smithi018.stdout:9/227: symlink d3/d6/d9/d3f/l47 0 2024-04-12T20:55:58.768 INFO:tasks.workunit.client.0.smithi018.stdout:4/196: dwrite d3/d32/d33/d1f/f2c [0,4194304] 0 2024-04-12T20:55:58.772 INFO:tasks.workunit.client.0.smithi018.stdout:4/197: symlink d3/d32/d33/l36 0 2024-04-12T20:55:58.773 INFO:tasks.workunit.client.0.smithi018.stdout:9/228: unlink c1 0 2024-04-12T20:55:58.781 INFO:tasks.workunit.client.0.smithi018.stdout:4/198: rename d3/f14 to d3/db/f37 0 2024-04-12T20:55:58.784 INFO:tasks.workunit.client.0.smithi018.stdout:9/229: getdents d3/d6/d9/dc/d1c/d21 0 2024-04-12T20:55:58.787 INFO:tasks.workunit.client.0.smithi018.stdout:4/199: mknod d3/db/c38 0 2024-04-12T20:55:58.789 INFO:tasks.workunit.client.0.smithi018.stdout:9/230: creat d3/d6/d9/dc/d46/f48 x:0 0 0 2024-04-12T20:55:58.793 INFO:tasks.workunit.client.0.smithi018.stdout:9/231: chown d3/d6/d9/dc/d46/f48 2254 1 2024-04-12T20:55:58.794 INFO:tasks.workunit.client.0.smithi018.stdout:4/200: link d3/db/f26 d3/d32/f39 0 2024-04-12T20:55:58.813 INFO:tasks.workunit.client.0.smithi018.stdout:9/232: dwrite d3/d6/f12 [4194304,4194304] 0 2024-04-12T20:55:58.814 INFO:tasks.workunit.client.0.smithi018.stdout:6/236: sync 2024-04-12T20:55:58.824 INFO:tasks.workunit.client.0.smithi018.stdout:9/233: write d3/d6/d9/fa [1808246,125363] 0 2024-04-12T20:55:58.825 INFO:tasks.workunit.client.0.smithi018.stdout:6/237: link d5/df/d3f/f49 d5/d6/d1b/f61 0 2024-04-12T20:55:58.841 INFO:tasks.workunit.client.0.smithi018.stdout:9/234: dread d3/f23 [4194304,4194304] 0 2024-04-12T20:55:58.845 INFO:tasks.workunit.client.0.smithi018.stdout:9/235: unlink d3/d6/f12 0 2024-04-12T20:55:58.848 INFO:tasks.workunit.client.0.smithi018.stdout:6/238: dwrite d5/d6/d1b/d3b/d35/d4f/f5c [0,4194304] 0 2024-04-12T20:55:58.852 INFO:tasks.workunit.client.0.smithi018.stdout:6/239: truncate d5/df/f2a 793629 0 2024-04-12T20:55:58.852 INFO:tasks.workunit.client.0.smithi018.stdout:9/236: fdatasync d3/d6/d9/d1e/f20 0 2024-04-12T20:55:58.856 INFO:tasks.workunit.client.0.smithi018.stdout:9/237: write d3/d6/d9/dc/d1c/f34 [4426919,115586] 0 2024-04-12T20:55:58.860 INFO:tasks.workunit.client.0.smithi018.stdout:6/240: creat d5/d6/d1b/d3b/d22/f62 x:0 0 0 2024-04-12T20:55:58.881 INFO:tasks.workunit.client.0.smithi018.stdout:6/241: dwrite d5/d6/d1b/d3b/d22/f62 [0,4194304] 0 2024-04-12T20:55:58.902 INFO:tasks.workunit.client.0.smithi018.stdout:6/242: dwrite d5/df/d3f/f44 [0,4194304] 0 2024-04-12T20:55:58.904 INFO:tasks.workunit.client.0.smithi018.stdout:6/243: truncate d5/f37 1894484 0 2024-04-12T20:55:58.913 INFO:tasks.workunit.client.0.smithi018.stdout:6/244: symlink d5/df/d31/d4e/l63 0 2024-04-12T20:55:58.915 INFO:tasks.workunit.client.0.smithi018.stdout:6/245: dread - d5/d6/d1b/d3b/f52 zero size 2024-04-12T20:55:58.919 INFO:tasks.workunit.client.0.smithi018.stdout:6/246: creat d5/df/d3f/f64 x:0 0 0 2024-04-12T20:55:58.920 INFO:tasks.workunit.client.0.smithi018.stdout:6/247: read d5/f20 [3726836,42958] 0 2024-04-12T20:55:58.924 INFO:tasks.workunit.client.0.smithi018.stdout:6/248: creat d5/d15/d60/f65 x:0 0 0 2024-04-12T20:55:58.930 INFO:tasks.workunit.client.0.smithi018.stdout:6/249: link d5/df/d31/d4e/c28 d5/df/d19/d43/c66 0 2024-04-12T20:55:58.933 INFO:tasks.workunit.client.0.smithi018.stdout:6/250: symlink d5/df/l67 0 2024-04-12T20:55:58.938 INFO:tasks.workunit.client.0.smithi018.stdout:6/251: rename d5/df/d3f/f64 to d5/df/d19/f68 0 2024-04-12T20:55:58.941 INFO:tasks.workunit.client.0.smithi018.stdout:6/252: truncate d5/df/d3f/f49 4603492 0 2024-04-12T20:55:58.944 INFO:tasks.workunit.client.0.smithi018.stdout:6/253: creat d5/d6/d1b/d3b/f69 x:0 0 0 2024-04-12T20:55:58.962 INFO:tasks.workunit.client.0.smithi018.stdout:6/254: write d5/df/d3f/f49 [1638952,123061] 0 2024-04-12T20:55:58.979 INFO:tasks.workunit.client.0.smithi018.stdout:6/255: dwrite d5/d6/d1b/d3b/d22/f62 [0,4194304] 0 2024-04-12T20:55:58.986 INFO:tasks.workunit.client.0.smithi018.stdout:8/178: sync 2024-04-12T20:55:58.986 INFO:tasks.workunit.client.0.smithi018.stdout:1/94: sync 2024-04-12T20:55:58.986 INFO:tasks.workunit.client.0.smithi018.stdout:2/133: sync 2024-04-12T20:55:58.989 INFO:tasks.workunit.client.0.smithi018.stdout:8/179: readlink db/dc/l10 0 2024-04-12T20:55:58.991 INFO:tasks.workunit.client.0.smithi018.stdout:1/95: mknod d3/d9/c27 0 2024-04-12T20:55:58.992 INFO:tasks.workunit.client.0.smithi018.stdout:8/180: write f7 [3757124,78743] 0 2024-04-12T20:55:58.995 INFO:tasks.workunit.client.0.smithi018.stdout:2/134: creat d5/d8/d10/d19/d24/f32 x:0 0 0 2024-04-12T20:55:58.996 INFO:tasks.workunit.client.0.smithi018.stdout:6/256: dread d5/df/d3f/f44 [0,4194304] 0 2024-04-12T20:55:58.997 INFO:tasks.workunit.client.0.smithi018.stdout:1/96: write d3/d9/da/dd/f12 [541237,81508] 0 2024-04-12T20:55:58.998 INFO:tasks.workunit.client.0.smithi018.stdout:8/181: write db/dc/f14 [2819641,40619] 0 2024-04-12T20:55:59.001 INFO:tasks.workunit.client.0.smithi018.stdout:1/97: mknod d3/c28 0 2024-04-12T20:55:59.002 INFO:tasks.workunit.client.0.smithi018.stdout:2/135: mkdir d5/d8/d33 0 2024-04-12T20:55:59.004 INFO:tasks.workunit.client.0.smithi018.stdout:1/98: readlink d3/d9/da/dd/l16 0 2024-04-12T20:55:59.008 INFO:tasks.workunit.client.0.smithi018.stdout:8/182: rename db/d25/d26/d27/f36 to db/d25/d26/f3a 0 2024-04-12T20:55:59.009 INFO:tasks.workunit.client.0.smithi018.stdout:1/99: write d3/f4 [7272757,112323] 0 2024-04-12T20:55:59.013 INFO:tasks.workunit.client.0.smithi018.stdout:8/183: write db/d25/d26/d27/f33 [1409354,15432] 0 2024-04-12T20:55:59.014 INFO:tasks.workunit.client.0.smithi018.stdout:8/184: stat db/dc 0 2024-04-12T20:55:59.016 INFO:tasks.workunit.client.0.smithi018.stdout:8/185: stat db/d25/c35 0 2024-04-12T20:55:59.017 INFO:tasks.workunit.client.0.smithi018.stdout:8/186: dread - db/d25/d26/d27/f2b zero size 2024-04-12T20:55:59.019 INFO:tasks.workunit.client.0.smithi018.stdout:2/136: dread f4 [0,4194304] 0 2024-04-12T20:55:59.026 INFO:tasks.workunit.client.0.smithi018.stdout:8/187: dread f6 [0,4194304] 0 2024-04-12T20:55:59.027 INFO:tasks.workunit.client.0.smithi018.stdout:6/257: dread d5/df/d19/f54 [0,4194304] 0 2024-04-12T20:55:59.029 INFO:tasks.workunit.client.0.smithi018.stdout:6/258: chown d5/d15/f39 4926888 1 2024-04-12T20:55:59.035 INFO:tasks.workunit.client.0.smithi018.stdout:6/259: mkdir d5/d6/d1b/d3b/d6a 0 2024-04-12T20:55:59.037 INFO:tasks.workunit.client.0.smithi018.stdout:6/260: fdatasync d5/d6/f11 0 2024-04-12T20:55:59.040 INFO:tasks.workunit.client.0.smithi018.stdout:6/261: mkdir d5/d15/d6b 0 2024-04-12T20:55:59.041 INFO:tasks.workunit.client.0.smithi018.stdout:1/100: dread d3/f6 [0,4194304] 0 2024-04-12T20:55:59.046 INFO:tasks.workunit.client.0.smithi018.stdout:6/262: readlink d5/df/l13 0 2024-04-12T20:55:59.047 INFO:tasks.workunit.client.0.smithi018.stdout:8/188: write db/f1b [959432,130078] 0 2024-04-12T20:55:59.050 INFO:tasks.workunit.client.0.smithi018.stdout:8/189: unlink db/c24 0 2024-04-12T20:55:59.051 INFO:tasks.workunit.client.0.smithi018.stdout:8/190: chown db/d25/d26 3282 1 2024-04-12T20:55:59.052 INFO:tasks.workunit.client.0.smithi018.stdout:2/137: dread f3 [0,4194304] 0 2024-04-12T20:55:59.053 INFO:tasks.workunit.client.0.smithi018.stdout:2/138: chown d5 1 1 2024-04-12T20:55:59.056 INFO:tasks.workunit.client.0.smithi018.stdout:2/139: creat d5/d8/d25/f34 x:0 0 0 2024-04-12T20:55:59.058 INFO:tasks.workunit.client.0.smithi018.stdout:2/140: fsync f4 0 2024-04-12T20:55:59.063 INFO:tasks.workunit.client.0.smithi018.stdout:2/141: dread - d5/d8/d10/d19/d24/f29 zero size 2024-04-12T20:55:59.067 INFO:tasks.workunit.client.0.smithi018.stdout:6/263: dwrite d5/d15/d60/f65 [0,4194304] 0 2024-04-12T20:55:59.068 INFO:tasks.workunit.client.0.smithi018.stdout:1/101: dwrite d3/f6 [0,4194304] 0 2024-04-12T20:55:59.070 INFO:tasks.workunit.client.0.smithi018.stdout:2/142: creat d5/d7/f35 x:0 0 0 2024-04-12T20:55:59.071 INFO:tasks.workunit.client.0.smithi018.stdout:6/264: fdatasync d5/d6/d1b/d3b/d22/f62 0 2024-04-12T20:55:59.075 INFO:tasks.workunit.client.0.smithi018.stdout:8/191: dwrite db/dc/f12 [0,4194304] 0 2024-04-12T20:55:59.075 INFO:tasks.workunit.client.0.smithi018.stdout:6/265: write d5/d6/d1b/d3b/d35/f33 [490567,107967] 0 2024-04-12T20:55:59.081 INFO:tasks.workunit.client.0.smithi018.stdout:8/192: mkdir db/d3b 0 2024-04-12T20:55:59.082 INFO:tasks.workunit.client.0.smithi018.stdout:8/193: write db/f1b [1052427,104466] 0 2024-04-12T20:55:59.091 INFO:tasks.workunit.client.0.smithi018.stdout:8/194: creat db/d3b/f3c x:0 0 0 2024-04-12T20:55:59.093 INFO:tasks.workunit.client.0.smithi018.stdout:1/102: dwrite d3/f6 [0,4194304] 0 2024-04-12T20:55:59.098 INFO:tasks.workunit.client.0.smithi018.stdout:1/103: chown d3/d9/da/dd/de/d13/l22 251 1 2024-04-12T20:55:59.105 INFO:tasks.workunit.client.0.smithi018.stdout:1/104: mknod d3/d9/da/d14/c29 0 2024-04-12T20:55:59.116 INFO:tasks.workunit.client.0.smithi018.stdout:8/195: dwrite db/f2d [0,4194304] 0 2024-04-12T20:55:59.129 INFO:tasks.workunit.client.0.smithi018.stdout:1/105: dwrite d3/d9/f18 [0,4194304] 0 2024-04-12T20:55:59.133 INFO:tasks.workunit.client.0.smithi018.stdout:1/106: fsync d3/f10 0 2024-04-12T20:55:59.134 INFO:tasks.workunit.client.0.smithi018.stdout:1/107: readlink d3/d9/da/dd/l16 0 2024-04-12T20:55:59.135 INFO:tasks.workunit.client.0.smithi018.stdout:1/108: fdatasync d3/f10 0 2024-04-12T20:55:59.137 INFO:tasks.workunit.client.0.smithi018.stdout:8/196: dwrite db/dc/f1e [0,4194304] 0 2024-04-12T20:55:59.137 INFO:tasks.workunit.client.0.smithi018.stdout:1/109: fsync d3/f7 0 2024-04-12T20:55:59.138 INFO:tasks.workunit.client.0.smithi018.stdout:8/197: fdatasync db/f1b 0 2024-04-12T20:55:59.140 INFO:tasks.workunit.client.0.smithi018.stdout:8/198: stat db/c1c 0 2024-04-12T20:55:59.143 INFO:tasks.workunit.client.0.smithi018.stdout:1/110: rename c1 to d3/d9/da/dd/de/d13/c2a 0 2024-04-12T20:55:59.155 INFO:tasks.workunit.client.0.smithi018.stdout:8/199: dwrite db/f11 [0,4194304] 0 2024-04-12T20:55:59.157 INFO:tasks.workunit.client.0.smithi018.stdout:8/200: write db/f15 [1827084,99308] 0 2024-04-12T20:55:59.158 INFO:tasks.workunit.client.0.smithi018.stdout:8/201: fdatasync db/d25/d26/f2f 0 2024-04-12T20:55:59.159 INFO:tasks.workunit.client.0.smithi018.stdout:8/202: stat db/d25/d26 0 2024-04-12T20:55:59.163 INFO:tasks.workunit.client.0.smithi018.stdout:8/203: dread db/d25/f34 [0,4194304] 0 2024-04-12T20:55:59.164 INFO:tasks.workunit.client.0.smithi018.stdout:8/204: fdatasync db/f1b 0 2024-04-12T20:55:59.165 INFO:tasks.workunit.client.0.smithi018.stdout:1/111: dread d3/d9/da/dd/de/f1b [0,4194304] 0 2024-04-12T20:55:59.166 INFO:tasks.workunit.client.0.smithi018.stdout:1/112: chown d3/d9/da/dd 2276453 1 2024-04-12T20:55:59.169 INFO:tasks.workunit.client.0.smithi018.stdout:1/113: chown d3/c1a 141364 1 2024-04-12T20:55:59.177 INFO:tasks.workunit.client.0.smithi018.stdout:1/114: dread d3/f6 [0,4194304] 0 2024-04-12T20:55:59.179 INFO:tasks.workunit.client.0.smithi018.stdout:8/205: truncate db/dc/f12 4133083 0 2024-04-12T20:55:59.190 INFO:tasks.workunit.client.0.smithi018.stdout:8/206: truncate db/d25/d26/d27/f33 983584 0 2024-04-12T20:55:59.191 INFO:tasks.workunit.client.0.smithi018.stdout:8/207: fsync f7 0 2024-04-12T20:55:59.193 INFO:tasks.workunit.client.0.smithi018.stdout:8/208: creat db/d3b/f3d x:0 0 0 2024-04-12T20:55:59.195 INFO:tasks.workunit.client.0.smithi018.stdout:8/209: mknod db/d25/d26/c3e 0 2024-04-12T20:55:59.196 INFO:tasks.workunit.client.0.smithi018.stdout:8/210: stat db/c37 0 2024-04-12T20:55:59.198 INFO:tasks.workunit.client.0.smithi018.stdout:8/211: creat db/d25/d26/f3f x:0 0 0 2024-04-12T20:55:59.201 INFO:tasks.workunit.client.0.smithi018.stdout:8/212: rename db/dc/f23 to db/dc/f40 0 2024-04-12T20:55:59.202 INFO:tasks.workunit.client.0.smithi018.stdout:8/213: chown db/d25/d26/c3e 99 1 2024-04-12T20:55:59.205 INFO:tasks.workunit.client.0.smithi018.stdout:8/214: mknod db/d3b/c41 0 2024-04-12T20:55:59.205 INFO:tasks.workunit.client.0.smithi018.stdout:5/130: sync 2024-04-12T20:55:59.205 INFO:tasks.workunit.client.0.smithi018.stdout:3/129: sync 2024-04-12T20:55:59.207 INFO:tasks.workunit.client.0.smithi018.stdout:3/130: symlink d2/l23 0 2024-04-12T20:55:59.210 INFO:tasks.workunit.client.0.smithi018.stdout:5/131: mkdir d0/dc/d10/d26/d30 0 2024-04-12T20:55:59.213 INFO:tasks.workunit.client.0.smithi018.stdout:3/131: creat d2/d6/f24 x:0 0 0 2024-04-12T20:55:59.217 INFO:tasks.workunit.client.0.smithi018.stdout:3/132: chown f1 287384011 1 2024-04-12T20:55:59.227 INFO:tasks.workunit.client.0.smithi018.stdout:8/215: dwrite db/dc/f1f [0,4194304] 0 2024-04-12T20:55:59.231 INFO:tasks.workunit.client.0.smithi018.stdout:3/133: dread d2/d6/f9 [0,4194304] 0 2024-04-12T20:55:59.235 INFO:tasks.workunit.client.0.smithi018.stdout:3/134: truncate d2/f19 206554 0 2024-04-12T20:55:59.238 INFO:tasks.workunit.client.0.smithi018.stdout:3/135: write d2/f19 [396720,32723] 0 2024-04-12T20:55:59.246 INFO:tasks.workunit.client.0.smithi018.stdout:5/132: dwrite d0/d2b/f2d [0,4194304] 0 2024-04-12T20:55:59.258 INFO:tasks.workunit.client.0.smithi018.stdout:5/133: unlink d0/d2/d11/l1b 0 2024-04-12T20:55:59.263 INFO:tasks.workunit.client.0.smithi018.stdout:8/216: dwrite f6 [0,4194304] 0 2024-04-12T20:55:59.270 INFO:tasks.workunit.client.0.smithi018.stdout:3/136: dwrite d2/d6/f1c [0,4194304] 0 2024-04-12T20:55:59.281 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:55:59 smithi186 ceph-mon[36597]: pgmap v32: 65 pgs: 65 active+clean; 325 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 354 KiB/s rd, 22 MiB/s wr, 177 op/s 2024-04-12T20:55:59.285 INFO:tasks.workunit.client.0.smithi018.stdout:8/217: dread db/dc/f1f [0,4194304] 0 2024-04-12T20:55:59.287 INFO:tasks.workunit.client.0.smithi018.stdout:8/218: fsync db/d25/d26/f3a 0 2024-04-12T20:55:59.289 INFO:tasks.workunit.client.0.smithi018.stdout:8/219: write db/f1b [797760,107778] 0 2024-04-12T20:55:59.292 INFO:tasks.workunit.client.0.smithi018.stdout:3/137: write d2/d6/f9 [443790,86660] 0 2024-04-12T20:55:59.295 INFO:tasks.workunit.client.0.smithi018.stdout:3/138: read d2/f4 [226526,89737] 0 2024-04-12T20:55:59.302 INFO:tasks.workunit.client.0.smithi018.stdout:3/139: truncate d2/f19 17660 0 2024-04-12T20:55:59.304 INFO:tasks.workunit.client.0.smithi018.stdout:3/140: creat d2/d6/f25 x:0 0 0 2024-04-12T20:55:59.305 INFO:tasks.workunit.client.0.smithi018.stdout:3/141: chown d2/c22 566 1 2024-04-12T20:55:59.307 INFO:tasks.workunit.client.0.smithi018.stdout:8/220: dwrite db/dc/f40 [4194304,4194304] 0 2024-04-12T20:55:59.307 INFO:tasks.workunit.client.0.smithi018.stdout:3/142: symlink d2/l26 0 2024-04-12T20:55:59.317 INFO:tasks.workunit.client.0.smithi018.stdout:8/221: fsync db/d25/d26/d27/f33 0 2024-04-12T20:55:59.323 INFO:tasks.workunit.client.0.smithi018.stdout:8/222: link db/d25/d26/c3e db/d25/c42 0 2024-04-12T20:55:59.325 INFO:tasks.workunit.client.0.smithi018.stdout:3/143: dread d2/d6/f1c [0,4194304] 0 2024-04-12T20:55:59.329 INFO:tasks.workunit.client.0.smithi018.stdout:8/223: rename db/c37 to db/d25/d26/d27/c43 0 2024-04-12T20:55:59.330 INFO:tasks.workunit.client.0.smithi018.stdout:3/144: symlink d2/l27 0 2024-04-12T20:55:59.332 INFO:tasks.workunit.client.0.smithi018.stdout:8/224: getdents db/d25/d26 0 2024-04-12T20:55:59.336 INFO:tasks.workunit.client.0.smithi018.stdout:8/225: getdents db/d25/d26 0 2024-04-12T20:55:59.339 INFO:tasks.workunit.client.0.smithi018.stdout:3/145: getdents d2 0 2024-04-12T20:55:59.350 INFO:tasks.workunit.client.0.smithi018.stdout:8/226: dread db/dc/f1a [0,4194304] 0 2024-04-12T20:55:59.350 INFO:tasks.workunit.client.0.smithi018.stdout:3/146: dread d2/d6/f1c [0,4194304] 0 2024-04-12T20:55:59.351 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:55:59 smithi018 ceph-mon[28799]: pgmap v32: 65 pgs: 65 active+clean; 325 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 354 KiB/s rd, 22 MiB/s wr, 177 op/s 2024-04-12T20:55:59.352 INFO:tasks.workunit.client.0.smithi018.stdout:8/227: read f7 [2857552,34432] 0 2024-04-12T20:55:59.354 INFO:tasks.workunit.client.0.smithi018.stdout:3/147: link d2/d6/f24 d2/f28 0 2024-04-12T20:55:59.355 INFO:tasks.workunit.client.0.smithi018.stdout:8/228: creat db/d25/d26/d27/f44 x:0 0 0 2024-04-12T20:55:59.361 INFO:tasks.workunit.client.0.smithi018.stdout:8/229: rmdir db/d25/d26/d27 39 2024-04-12T20:55:59.362 INFO:tasks.workunit.client.0.smithi018.stdout:8/230: readlink db/d25/l39 0 2024-04-12T20:55:59.365 INFO:tasks.workunit.client.0.smithi018.stdout:8/231: mkdir db/dc/d45 0 2024-04-12T20:55:59.367 INFO:tasks.workunit.client.0.smithi018.stdout:8/232: read db/dc/f1e [1007094,128580] 0 2024-04-12T20:55:59.369 INFO:tasks.workunit.client.0.smithi018.stdout:8/233: chown db/dc/f40 1271 1 2024-04-12T20:55:59.370 INFO:tasks.workunit.client.0.smithi018.stdout:3/148: dwrite d2/f19 [0,4194304] 0 2024-04-12T20:55:59.371 INFO:tasks.workunit.client.0.smithi018.stdout:3/149: fdatasync d2/d6/f1b 0 2024-04-12T20:55:59.374 INFO:tasks.workunit.client.0.smithi018.stdout:8/234: rename db/d25/c35 to db/d25/d26/d27/c46 0 2024-04-12T20:55:59.375 INFO:tasks.workunit.client.0.smithi018.stdout:0/172: sync 2024-04-12T20:55:59.380 INFO:tasks.workunit.client.0.smithi018.stdout:3/150: unlink d2/f4 0 2024-04-12T20:55:59.382 INFO:tasks.workunit.client.0.smithi018.stdout:8/235: creat db/d3b/f47 x:0 0 0 2024-04-12T20:55:59.383 INFO:tasks.workunit.client.0.smithi018.stdout:0/173: creat da/d15/f39 x:0 0 0 2024-04-12T20:55:59.384 INFO:tasks.workunit.client.0.smithi018.stdout:3/151: mknod d2/c29 0 2024-04-12T20:55:59.387 INFO:tasks.workunit.client.0.smithi018.stdout:0/174: rename da/d15/f34 to da/d15/d17/f3a 0 2024-04-12T20:55:59.388 INFO:tasks.workunit.client.0.smithi018.stdout:8/236: creat db/f48 x:0 0 0 2024-04-12T20:55:59.389 INFO:tasks.workunit.client.0.smithi018.stdout:0/175: creat da/d15/d17/f3b x:0 0 0 2024-04-12T20:55:59.392 INFO:tasks.workunit.client.0.smithi018.stdout:0/176: symlink da/d15/d17/d21/d36/l3c 0 2024-04-12T20:55:59.402 INFO:tasks.workunit.client.0.smithi018.stdout:8/237: truncate f0 2041567 0 2024-04-12T20:55:59.403 INFO:tasks.workunit.client.0.smithi018.stdout:0/177: rmdir da/d15/d17 39 2024-04-12T20:55:59.467 INFO:tasks.workunit.client.0.smithi018.stdout:8/238: rename db/dc/f1f to db/d3b/f49 0 2024-04-12T20:55:59.471 INFO:tasks.workunit.client.0.smithi018.stdout:8/239: mknod db/d25/c4a 0 2024-04-12T20:55:59.473 INFO:tasks.workunit.client.0.smithi018.stdout:0/178: getdents da/d15/d17/d21 0 2024-04-12T20:55:59.479 INFO:tasks.workunit.client.0.smithi018.stdout:0/179: write da/d15/d17/f3a [580715,33820] 0 2024-04-12T20:55:59.481 INFO:tasks.workunit.client.0.smithi018.stdout:0/180: chown da/f25 175 1 2024-04-12T20:55:59.490 INFO:tasks.workunit.client.0.smithi018.stdout:0/181: rename c2 to da/d15/d17/d21/d36/c3d 0 2024-04-12T20:55:59.495 INFO:tasks.workunit.client.0.smithi018.stdout:0/182: mkdir da/d15/d3e 0 2024-04-12T20:55:59.500 INFO:tasks.workunit.client.0.smithi018.stdout:0/183: mkdir da/d15/d3e/d3f 0 2024-04-12T20:55:59.504 INFO:tasks.workunit.client.0.smithi018.stdout:0/184: chown da/d15/f35 170 1 2024-04-12T20:55:59.507 INFO:tasks.workunit.client.0.smithi018.stdout:8/240: dwrite db/d25/f2c [0,4194304] 0 2024-04-12T20:55:59.517 INFO:tasks.workunit.client.0.smithi018.stdout:0/185: getdents da/d15/d17 0 2024-04-12T20:55:59.521 INFO:tasks.workunit.client.0.smithi018.stdout:0/186: mknod da/d15/d17/c40 0 2024-04-12T20:55:59.524 INFO:tasks.workunit.client.0.smithi018.stdout:8/241: dread f6 [0,4194304] 0 2024-04-12T20:55:59.525 INFO:tasks.workunit.client.0.smithi018.stdout:8/242: read f6 [1302701,66742] 0 2024-04-12T20:55:59.526 INFO:tasks.workunit.client.0.smithi018.stdout:0/187: creat da/d15/f41 x:0 0 0 2024-04-12T20:55:59.527 INFO:tasks.workunit.client.0.smithi018.stdout:8/243: fdatasync db/f48 0 2024-04-12T20:55:59.547 INFO:tasks.workunit.client.0.smithi018.stdout:0/188: dwrite da/f1d [0,4194304] 0 2024-04-12T20:55:59.555 INFO:tasks.workunit.client.0.smithi018.stdout:7/217: sync 2024-04-12T20:55:59.555 INFO:tasks.workunit.client.0.smithi018.stdout:4/201: sync 2024-04-12T20:55:59.558 INFO:tasks.workunit.client.0.smithi018.stdout:7/218: read d1/db/d31/d3a/f41 [140315,121815] 0 2024-04-12T20:55:59.559 INFO:tasks.workunit.client.0.smithi018.stdout:4/202: write d3/d32/d33/f18 [1986017,120652] 0 2024-04-12T20:55:59.565 INFO:tasks.workunit.client.0.smithi018.stdout:4/203: symlink d3/db/l3a 0 2024-04-12T20:55:59.570 INFO:tasks.workunit.client.0.smithi018.stdout:4/204: creat d3/f3b x:0 0 0 2024-04-12T20:55:59.572 INFO:tasks.workunit.client.0.smithi018.stdout:7/219: truncate d1/db/f35 324151 0 2024-04-12T20:55:59.573 INFO:tasks.workunit.client.0.smithi018.stdout:4/205: write d3/f3b [880229,79182] 0 2024-04-12T20:55:59.581 INFO:tasks.workunit.client.0.smithi018.stdout:0/189: dwrite da/d15/d17/f3a [0,4194304] 0 2024-04-12T20:55:59.583 INFO:tasks.workunit.client.0.smithi018.stdout:0/190: chown da/f22 577 1 2024-04-12T20:55:59.588 INFO:tasks.workunit.client.0.smithi018.stdout:0/191: stat da/d15/d17/d21/d36/l3c 0 2024-04-12T20:55:59.588 INFO:tasks.workunit.client.0.smithi018.stdout:8/244: dread db/dc/f40 [0,4194304] 0 2024-04-12T20:55:59.594 INFO:tasks.workunit.client.0.smithi018.stdout:8/245: write db/d25/d26/f3f [328932,36536] 0 2024-04-12T20:55:59.595 INFO:tasks.workunit.client.0.smithi018.stdout:7/220: dwrite d1/d6/f24 [0,4194304] 0 2024-04-12T20:55:59.599 INFO:tasks.workunit.client.0.smithi018.stdout:0/192: write da/fc [1334349,104879] 0 2024-04-12T20:55:59.602 INFO:tasks.workunit.client.0.smithi018.stdout:0/193: fdatasync da/d15/d17/f32 0 2024-04-12T20:55:59.610 INFO:tasks.workunit.client.0.smithi018.stdout:4/206: dwrite d3/d32/d33/d1f/f2f [0,4194304] 0 2024-04-12T20:55:59.610 INFO:tasks.workunit.client.0.smithi018.stdout:7/221: getdents d1/db/d31/d3a/d42 0 2024-04-12T20:55:59.614 INFO:tasks.workunit.client.0.smithi018.stdout:4/207: write d3/db/f24 [990573,83365] 0 2024-04-12T20:55:59.619 INFO:tasks.workunit.client.0.smithi018.stdout:7/222: write d1/db/d31/d3a/d42/d4c/f51 [84929,127261] 0 2024-04-12T20:55:59.624 INFO:tasks.workunit.client.0.smithi018.stdout:4/208: fdatasync d3/db/f37 0 2024-04-12T20:55:59.624 INFO:tasks.workunit.client.0.smithi018.stdout:0/194: write da/f22 [589131,18313] 0 2024-04-12T20:55:59.625 INFO:tasks.workunit.client.0.smithi018.stdout:7/223: chown d1/f3e 26641 1 2024-04-12T20:55:59.629 INFO:tasks.workunit.client.0.smithi018.stdout:8/246: dwrite db/d3b/f3d [0,4194304] 0 2024-04-12T20:55:59.631 INFO:tasks.workunit.client.0.smithi018.stdout:0/195: symlink da/l42 0 2024-04-12T20:55:59.632 INFO:tasks.workunit.client.0.smithi018.stdout:7/224: mknod d1/c54 0 2024-04-12T20:55:59.633 INFO:tasks.workunit.client.0.smithi018.stdout:8/247: truncate db/f15 1936042 0 2024-04-12T20:55:59.636 INFO:tasks.workunit.client.0.smithi018.stdout:0/196: creat da/d15/d17/f43 x:0 0 0 2024-04-12T20:55:59.638 INFO:tasks.workunit.client.0.smithi018.stdout:4/209: link d3/lf d3/db/l3c 0 2024-04-12T20:55:59.638 INFO:tasks.workunit.client.0.smithi018.stdout:0/197: read da/d15/d17/f3a [2857711,388] 0 2024-04-12T20:55:59.640 INFO:tasks.workunit.client.0.smithi018.stdout:7/225: rename d1/db/c18 to d1/db/d31/d45/c55 0 2024-04-12T20:55:59.642 INFO:tasks.workunit.client.0.smithi018.stdout:8/248: truncate db/dc/f12 2960651 0 2024-04-12T20:55:59.643 INFO:tasks.workunit.client.0.smithi018.stdout:7/226: rmdir d1/d6 39 2024-04-12T20:55:59.647 INFO:tasks.workunit.client.0.smithi018.stdout:4/210: truncate d3/db/f37 350739 0 2024-04-12T20:55:59.649 INFO:tasks.workunit.client.0.smithi018.stdout:4/211: readlink d3/db/l3a 0 2024-04-12T20:55:59.649 INFO:tasks.workunit.client.0.smithi018.stdout:7/227: rename d1/fd to d1/db/d31/d45/d53/f56 0 2024-04-12T20:55:59.651 INFO:tasks.workunit.client.0.smithi018.stdout:4/212: chown d3/db 2697 1 2024-04-12T20:55:59.652 INFO:tasks.workunit.client.0.smithi018.stdout:4/213: readlink d3/db/l3a 0 2024-04-12T20:55:59.653 INFO:tasks.workunit.client.0.smithi018.stdout:7/228: creat d1/db/d31/d30/f57 x:0 0 0 2024-04-12T20:55:59.659 INFO:tasks.workunit.client.0.smithi018.stdout:0/198: dwrite da/d15/f39 [0,4194304] 0 2024-04-12T20:55:59.665 INFO:tasks.workunit.client.0.smithi018.stdout:8/249: write db/f11 [5100852,74722] 0 2024-04-12T20:55:59.667 INFO:tasks.workunit.client.0.smithi018.stdout:0/199: link l6 da/d15/d17/d21/d36/l44 0 2024-04-12T20:55:59.669 INFO:tasks.workunit.client.0.smithi018.stdout:9/238: sync 2024-04-12T20:55:59.669 INFO:tasks.workunit.client.0.smithi018.stdout:3/152: sync 2024-04-12T20:55:59.669 INFO:tasks.workunit.client.0.smithi018.stdout:6/266: sync 2024-04-12T20:55:59.669 INFO:tasks.workunit.client.0.smithi018.stdout:1/115: sync 2024-04-12T20:55:59.670 INFO:tasks.workunit.client.0.smithi018.stdout:5/134: sync 2024-04-12T20:55:59.670 INFO:tasks.workunit.client.0.smithi018.stdout:2/143: sync 2024-04-12T20:55:59.673 INFO:tasks.workunit.client.0.smithi018.stdout:4/214: truncate d3/d32/f1a 669101 0 2024-04-12T20:55:59.675 INFO:tasks.workunit.client.0.smithi018.stdout:3/153: mknod d2/c2a 0 2024-04-12T20:55:59.676 INFO:tasks.workunit.client.0.smithi018.stdout:6/267: creat d5/df/d19/f6c x:0 0 0 2024-04-12T20:55:59.681 INFO:tasks.workunit.client.0.smithi018.stdout:9/239: symlink d3/d6/d9/dc/d1c/d21/l49 0 2024-04-12T20:55:59.681 INFO:tasks.workunit.client.0.smithi018.stdout:4/215: mkdir d3/d34/d3d 0 2024-04-12T20:55:59.683 INFO:tasks.workunit.client.0.smithi018.stdout:9/240: chown d3/d6/d9/d1e/d3a 1 1 2024-04-12T20:55:59.685 INFO:tasks.workunit.client.0.smithi018.stdout:6/268: creat d5/d15/d60/f6d x:0 0 0 2024-04-12T20:55:59.690 INFO:tasks.workunit.client.0.smithi018.stdout:8/250: dread db/d25/f2c [0,4194304] 0 2024-04-12T20:55:59.691 INFO:tasks.workunit.client.0.smithi018.stdout:9/241: mknod d3/d6/d3d/c4a 0 2024-04-12T20:55:59.691 INFO:tasks.workunit.client.0.smithi018.stdout:6/269: write d5/f10 [2939985,79060] 0 2024-04-12T20:55:59.694 INFO:tasks.workunit.client.0.smithi018.stdout:8/251: readlink l8 0 2024-04-12T20:55:59.699 INFO:tasks.workunit.client.0.smithi018.stdout:0/200: dread da/d15/d17/f3a [0,4194304] 0 2024-04-12T20:55:59.704 INFO:tasks.workunit.client.0.smithi018.stdout:6/270: chown d5/d6/d1b/d3b/d22/f46 0 1 2024-04-12T20:55:59.708 INFO:tasks.workunit.client.0.smithi018.stdout:8/252: rename db/c1c to db/dc/c4b 0 2024-04-12T20:55:59.711 INFO:tasks.workunit.client.0.smithi018.stdout:2/144: dwrite d5/d8/d10/f30 [0,4194304] 0 2024-04-12T20:55:59.718 INFO:tasks.workunit.client.0.smithi018.stdout:1/116: dwrite d3/d9/da/dd/f12 [0,4194304] 0 2024-04-12T20:55:59.720 INFO:tasks.workunit.client.0.smithi018.stdout:8/253: fdatasync db/d3b/f47 0 2024-04-12T20:55:59.722 INFO:tasks.workunit.client.0.smithi018.stdout:3/154: dwrite d2/f14 [0,4194304] 0 2024-04-12T20:55:59.723 INFO:tasks.workunit.client.0.smithi018.stdout:6/271: symlink d5/d15/d60/l6e 0 2024-04-12T20:55:59.728 INFO:tasks.workunit.client.0.smithi018.stdout:2/145: creat d5/d8/d10/d19/d24/f36 x:0 0 0 2024-04-12T20:55:59.728 INFO:tasks.workunit.client.0.smithi018.stdout:4/216: dread d3/d32/d33/d1f/f2f [0,4194304] 0 2024-04-12T20:55:59.730 INFO:tasks.workunit.client.0.smithi018.stdout:3/155: write d2/f1a [824209,27928] 0 2024-04-12T20:55:59.731 INFO:tasks.workunit.client.0.smithi018.stdout:1/117: readlink d3/d9/da/dd/de/d13/l22 0 2024-04-12T20:55:59.732 INFO:tasks.workunit.client.0.smithi018.stdout:8/254: rename db/f48 to db/d25/f4c 0 2024-04-12T20:55:59.736 INFO:tasks.workunit.client.0.smithi018.stdout:9/242: dwrite d3/d6/d1b/f35 [0,4194304] 0 2024-04-12T20:55:59.738 INFO:tasks.workunit.client.0.smithi018.stdout:1/118: chown d3/d9/da/dd/de/d13/d19 71963 1 2024-04-12T20:55:59.742 INFO:tasks.workunit.client.0.smithi018.stdout:1/119: dread - d3/d9/da/f20 zero size 2024-04-12T20:55:59.748 INFO:tasks.workunit.client.0.smithi018.stdout:2/146: creat d5/d7/f37 x:0 0 0 2024-04-12T20:55:59.755 INFO:tasks.workunit.client.0.smithi018.stdout:4/217: getdents d3 0 2024-04-12T20:55:59.755 INFO:tasks.workunit.client.0.smithi018.stdout:9/243: truncate d3/d6/d9/dc/f22 222064 0 2024-04-12T20:55:59.775 INFO:tasks.workunit.client.0.smithi018.stdout:3/156: dwrite d2/f19 [4194304,4194304] 0 2024-04-12T20:55:59.779 INFO:tasks.workunit.client.0.smithi018.stdout:4/218: unlink d3/d32/d33/d1f/c21 0 2024-04-12T20:55:59.781 INFO:tasks.workunit.client.0.smithi018.stdout:8/255: dwrite f7 [0,4194304] 0 2024-04-12T20:55:59.782 INFO:tasks.workunit.client.0.smithi018.stdout:4/219: readlink d3/db/l3c 0 2024-04-12T20:55:59.788 INFO:tasks.workunit.client.0.smithi018.stdout:1/120: dwrite d3/d9/da/dd/de/f1b [0,4194304] 0 2024-04-12T20:55:59.790 INFO:tasks.workunit.client.0.smithi018.stdout:8/256: write db/f2d [368986,42829] 0 2024-04-12T20:55:59.796 INFO:tasks.workunit.client.0.smithi018.stdout:1/121: stat d3/f4 0 2024-04-12T20:55:59.797 INFO:tasks.workunit.client.0.smithi018.stdout:4/220: write d3/d32/d33/f18 [4779075,126159] 0 2024-04-12T20:55:59.802 INFO:tasks.workunit.client.0.smithi018.stdout:2/147: dread d5/f1d [0,4194304] 0 2024-04-12T20:55:59.803 INFO:tasks.workunit.client.0.smithi018.stdout:9/244: dwrite d3/d6/f30 [0,4194304] 0 2024-04-12T20:55:59.807 INFO:tasks.workunit.client.0.smithi018.stdout:2/148: dread - d5/d7/d28/f2d zero size 2024-04-12T20:55:59.807 INFO:tasks.workunit.client.0.smithi018.stdout:1/122: rmdir d3/d9/da/dd/de/d13 39 2024-04-12T20:55:59.811 INFO:tasks.workunit.client.0.smithi018.stdout:3/157: dread d2/f14 [0,4194304] 0 2024-04-12T20:55:59.812 INFO:tasks.workunit.client.0.smithi018.stdout:2/149: truncate d5/d7/f35 996560 0 2024-04-12T20:55:59.816 INFO:tasks.workunit.client.0.smithi018.stdout:4/221: getdents d3/db 0 2024-04-12T20:55:59.820 INFO:tasks.workunit.client.0.smithi018.stdout:8/257: link db/dc/f14 db/d3b/f4d 0 2024-04-12T20:55:59.822 INFO:tasks.workunit.client.0.smithi018.stdout:9/245: mknod d3/d6/d9/dc/c4b 0 2024-04-12T20:55:59.824 INFO:tasks.workunit.client.0.smithi018.stdout:3/158: rmdir d2/d6 39 2024-04-12T20:55:59.826 INFO:tasks.workunit.client.0.smithi018.stdout:4/222: unlink d3/d32/f39 0 2024-04-12T20:55:59.827 INFO:tasks.workunit.client.0.smithi018.stdout:9/246: write d3/d6/d9/d3f/f44 [818388,95311] 0 2024-04-12T20:55:59.832 INFO:tasks.workunit.client.0.smithi018.stdout:9/247: write d3/d6/d9/d1e/d3a/f31 [2013040,24051] 0 2024-04-12T20:55:59.833 INFO:tasks.workunit.client.0.smithi018.stdout:8/258: chown db/dc/c20 3477089 1 2024-04-12T20:55:59.837 INFO:tasks.workunit.client.0.smithi018.stdout:1/123: link d3/d9/da/dd/de/c1d d3/d9/da/d14/c2b 0 2024-04-12T20:55:59.838 INFO:tasks.workunit.client.0.smithi018.stdout:3/159: mknod d2/c2b 0 2024-04-12T20:55:59.840 INFO:tasks.workunit.client.0.smithi018.stdout:3/160: fdatasync d2/f19 0 2024-04-12T20:55:59.840 INFO:tasks.workunit.client.0.smithi018.stdout:4/223: unlink d3/d32/f1a 0 2024-04-12T20:55:59.841 INFO:tasks.workunit.client.0.smithi018.stdout:1/124: write d3/d9/da/f20 [380229,69856] 0 2024-04-12T20:55:59.843 INFO:tasks.workunit.client.0.smithi018.stdout:9/248: rename d3/d6/d9/dc/c4b to d3/d6/d1b/c4c 0 2024-04-12T20:55:59.845 INFO:tasks.workunit.client.0.smithi018.stdout:2/150: dwrite d5/d8/d10/d19/d24/f29 [0,4194304] 0 2024-04-12T20:55:59.846 INFO:tasks.workunit.client.0.smithi018.stdout:8/259: write db/d25/f34 [2288560,30065] 0 2024-04-12T20:55:59.846 INFO:tasks.workunit.client.0.smithi018.stdout:3/161: readlink d2/d6/l13 0 2024-04-12T20:55:59.849 INFO:tasks.workunit.client.0.smithi018.stdout:1/125: chown d3/d9/da/f20 3845 1 2024-04-12T20:55:59.851 INFO:tasks.workunit.client.0.smithi018.stdout:3/162: readlink d2/l16 0 2024-04-12T20:55:59.852 INFO:tasks.workunit.client.0.smithi018.stdout:3/163: chown d2/d6 1045142 1 2024-04-12T20:55:59.855 INFO:tasks.workunit.client.0.smithi018.stdout:3/164: fsync d2/d6/f9 0 2024-04-12T20:55:59.856 INFO:tasks.workunit.client.0.smithi018.stdout:8/260: mkdir db/d4e 0 2024-04-12T20:55:59.861 INFO:tasks.workunit.client.0.smithi018.stdout:8/261: creat db/d4e/f4f x:0 0 0 2024-04-12T20:55:59.870 INFO:tasks.workunit.client.0.smithi018.stdout:1/126: truncate d3/f4 1282515 0 2024-04-12T20:55:59.872 INFO:tasks.workunit.client.0.smithi018.stdout:3/165: rename d2/c1d to d2/c2c 0 2024-04-12T20:55:59.875 INFO:tasks.workunit.client.0.smithi018.stdout:2/151: dwrite d5/d7/d28/f2d [0,4194304] 0 2024-04-12T20:55:59.877 INFO:tasks.workunit.client.0.smithi018.stdout:3/166: read d2/d6/f1c [2562519,119884] 0 2024-04-12T20:55:59.878 INFO:tasks.workunit.client.0.smithi018.stdout:4/224: dwrite d3/db/f37 [0,4194304] 0 2024-04-12T20:55:59.878 INFO:tasks.workunit.client.0.smithi018.stdout:1/127: mknod d3/d9/c2c 0 2024-04-12T20:55:59.885 INFO:tasks.workunit.client.0.smithi018.stdout:1/128: chown d3/d9/da/lb 34 1 2024-04-12T20:55:59.885 INFO:tasks.workunit.client.0.smithi018.stdout:2/152: fsync d5/d8/d10/f15 0 2024-04-12T20:55:59.886 INFO:tasks.workunit.client.0.smithi018.stdout:3/167: symlink d2/d6/l2d 0 2024-04-12T20:55:59.890 INFO:tasks.workunit.client.0.smithi018.stdout:4/225: rename d3/d32/d33/d1f/l31 to d3/d32/l3e 0 2024-04-12T20:55:59.892 INFO:tasks.workunit.client.0.smithi018.stdout:2/153: readlink d5/d12/l1c 0 2024-04-12T20:55:59.895 INFO:tasks.workunit.client.0.smithi018.stdout:3/168: read d2/d6/f12 [3804457,29187] 0 2024-04-12T20:55:59.895 INFO:tasks.workunit.client.0.smithi018.stdout:4/226: write d3/d32/d33/d1f/f2c [1871924,27092] 0 2024-04-12T20:55:59.899 INFO:tasks.workunit.client.0.smithi018.stdout:3/169: fsync d2/d6/f1b 0 2024-04-12T20:55:59.900 INFO:tasks.workunit.client.0.smithi018.stdout:2/154: truncate d5/d8/d10/d19/d24/f32 115024 0 2024-04-12T20:55:59.901 INFO:tasks.workunit.client.0.smithi018.stdout:4/227: write d3/d32/d33/f30 [4529318,60502] 0 2024-04-12T20:55:59.903 INFO:tasks.workunit.client.0.smithi018.stdout:6/272: write d5/df/d19/f25 [3897477,38493] 0 2024-04-12T20:55:59.905 INFO:tasks.workunit.client.0.smithi018.stdout:1/129: mkdir d3/d9/da/dd/de/d13/d19/d2d 0 2024-04-12T20:55:59.906 INFO:tasks.workunit.client.0.smithi018.stdout:6/273: chown d5/df/d19/d43/d5f 198076 1 2024-04-12T20:55:59.912 INFO:tasks.workunit.client.0.smithi018.stdout:6/274: creat d5/d15/f6f x:0 0 0 2024-04-12T20:55:59.917 INFO:tasks.workunit.client.0.smithi018.stdout:6/275: truncate d5/d6/d1b/d3b/f69 382181 0 2024-04-12T20:55:59.922 INFO:tasks.workunit.client.0.smithi018.stdout:1/130: unlink d3/d9/da/dd/de/d13/f23 0 2024-04-12T20:55:59.925 INFO:tasks.workunit.client.0.smithi018.stdout:6/276: write d5/df/d19/f68 [271001,81429] 0 2024-04-12T20:55:59.929 INFO:tasks.workunit.client.0.smithi018.stdout:3/170: link d2/f5 d2/d6/f2e 0 2024-04-12T20:55:59.931 INFO:tasks.workunit.client.0.smithi018.stdout:4/228: dwrite d3/d32/fd [0,4194304] 0 2024-04-12T20:55:59.935 INFO:tasks.workunit.client.0.smithi018.stdout:6/277: symlink d5/df/d19/d43/l70 0 2024-04-12T20:55:59.936 INFO:tasks.workunit.client.0.smithi018.stdout:2/155: rename d5/d8/c2b to d5/c38 0 2024-04-12T20:55:59.938 INFO:tasks.workunit.client.0.smithi018.stdout:3/171: creat d2/d6/f2f x:0 0 0 2024-04-12T20:55:59.939 INFO:tasks.workunit.client.0.smithi018.stdout:9/249: write d3/d6/d9/dc/f22 [345957,28788] 0 2024-04-12T20:55:59.941 INFO:tasks.workunit.client.0.smithi018.stdout:1/131: symlink d3/d9/da/dd/de/l2e 0 2024-04-12T20:55:59.942 INFO:tasks.workunit.client.0.smithi018.stdout:9/250: dread - d3/d6/d9/dc/d46/f48 zero size 2024-04-12T20:55:59.942 INFO:tasks.workunit.client.0.smithi018.stdout:2/156: truncate d5/d8/d25/f34 247201 0 2024-04-12T20:55:59.942 INFO:tasks.workunit.client.0.smithi018.stdout:4/229: symlink d3/d32/l3f 0 2024-04-12T20:55:59.943 INFO:tasks.workunit.client.0.smithi018.stdout:3/172: mkdir d2/d6/d30 0 2024-04-12T20:55:59.946 INFO:tasks.workunit.client.0.smithi018.stdout:4/230: chown d3/db/l3c 3898 1 2024-04-12T20:55:59.947 INFO:tasks.workunit.client.0.smithi018.stdout:3/173: mkdir d2/d6/d30/d31 0 2024-04-12T20:55:59.947 INFO:tasks.workunit.client.0.smithi018.stdout:9/251: symlink d3/l4d 0 2024-04-12T20:55:59.949 INFO:tasks.workunit.client.0.smithi018.stdout:2/157: mkdir d5/d8/d10/d19/d24/d39 0 2024-04-12T20:55:59.950 INFO:tasks.workunit.client.0.smithi018.stdout:3/174: fdatasync d2/d6/f2f 0 2024-04-12T20:55:59.950 INFO:tasks.workunit.client.0.smithi018.stdout:2/158: stat d5/d7/d28 0 2024-04-12T20:55:59.954 INFO:tasks.workunit.client.0.smithi018.stdout:9/252: creat d3/d6/d9/dc/d1c/f4e x:0 0 0 2024-04-12T20:55:59.956 INFO:tasks.workunit.client.0.smithi018.stdout:2/159: dread d5/d8/d25/f34 [0,4194304] 0 2024-04-12T20:55:59.960 INFO:tasks.workunit.client.0.smithi018.stdout:9/253: mknod d3/d6/d9/dc/d1c/c4f 0 2024-04-12T20:55:59.960 INFO:tasks.workunit.client.0.smithi018.stdout:2/160: dread - d5/d8/d10/d19/d24/f36 zero size 2024-04-12T20:55:59.966 INFO:tasks.workunit.client.0.smithi018.stdout:1/132: dread d3/d9/da/dd/de/f1b [0,4194304] 0 2024-04-12T20:55:59.973 INFO:tasks.workunit.client.0.smithi018.stdout:6/278: mkdir d5/d71 0 2024-04-12T20:55:59.974 INFO:tasks.workunit.client.0.smithi018.stdout:3/175: dwrite d2/d6/f9 [0,4194304] 0 2024-04-12T20:55:59.978 INFO:tasks.workunit.client.0.smithi018.stdout:3/176: write d2/f14 [4001175,113825] 0 2024-04-12T20:55:59.978 INFO:tasks.workunit.client.0.smithi018.stdout:9/254: dread d3/d6/f30 [0,4194304] 0 2024-04-12T20:55:59.982 INFO:tasks.workunit.client.0.smithi018.stdout:3/177: dread - d2/f28 zero size 2024-04-12T20:55:59.984 INFO:tasks.workunit.client.0.smithi018.stdout:6/279: symlink d5/l72 0 2024-04-12T20:55:59.984 INFO:tasks.workunit.client.0.smithi018.stdout:3/178: readlink d2/l26 0 2024-04-12T20:55:59.985 INFO:tasks.workunit.client.0.smithi018.stdout:2/161: dwrite d5/d7/d28/f2d [0,4194304] 0 2024-04-12T20:55:59.989 INFO:tasks.workunit.client.0.smithi018.stdout:3/179: symlink d2/d6/d30/d31/l32 0 2024-04-12T20:55:59.992 INFO:tasks.workunit.client.0.smithi018.stdout:3/180: write d2/f19 [6707003,37810] 0 2024-04-12T20:55:59.997 INFO:tasks.workunit.client.0.smithi018.stdout:3/181: dread - d2/d6/f24 zero size 2024-04-12T20:55:59.997 INFO:tasks.workunit.client.0.smithi018.stdout:9/255: truncate d3/f23 8280509 0 2024-04-12T20:56:00.000 INFO:tasks.workunit.client.0.smithi018.stdout:3/182: write d2/d6/f9 [2059263,4267] 0 2024-04-12T20:56:00.001 INFO:tasks.workunit.client.0.smithi018.stdout:9/256: creat d3/d6/d9/d3f/f50 x:0 0 0 2024-04-12T20:56:00.002 INFO:tasks.workunit.client.0.smithi018.stdout:6/280: dread d5/d15/f1c [0,4194304] 0 2024-04-12T20:56:00.004 INFO:tasks.workunit.client.0.smithi018.stdout:9/257: dread - d3/d6/d9/dc/d46/f48 zero size 2024-04-12T20:56:00.005 INFO:tasks.workunit.client.0.smithi018.stdout:6/281: symlink d5/d15/d60/l73 0 2024-04-12T20:56:00.007 INFO:tasks.workunit.client.0.smithi018.stdout:9/258: fsync d3/d6/d9/dc/d1c/f4e 0 2024-04-12T20:56:00.011 INFO:tasks.workunit.client.0.smithi018.stdout:6/282: mkdir d5/df/d19/d74 0 2024-04-12T20:56:00.012 INFO:tasks.workunit.client.0.smithi018.stdout:9/259: symlink d3/d6/d9/dc/d13/l51 0 2024-04-12T20:56:00.014 INFO:tasks.workunit.client.0.smithi018.stdout:6/283: write d5/d15/d60/f6d [123839,35161] 0 2024-04-12T20:56:00.018 INFO:tasks.workunit.client.0.smithi018.stdout:6/284: chown d5/df/d19/d2b/c2d 1 1 2024-04-12T20:56:00.019 INFO:tasks.workunit.client.0.smithi018.stdout:9/260: link d3/d6/c2c d3/d6/d1b/c52 0 2024-04-12T20:56:00.021 INFO:tasks.workunit.client.0.smithi018.stdout:3/183: dwrite d2/d6/f1c [0,4194304] 0 2024-04-12T20:56:00.024 INFO:tasks.workunit.client.0.smithi018.stdout:3/184: write d2/d6/f25 [1043439,54703] 0 2024-04-12T20:56:00.029 INFO:tasks.workunit.client.0.smithi018.stdout:6/285: creat d5/df/d19/d74/f75 x:0 0 0 2024-04-12T20:56:00.029 INFO:tasks.workunit.client.0.smithi018.stdout:9/261: creat d3/d6/d9/d1e/f53 x:0 0 0 2024-04-12T20:56:00.037 INFO:tasks.workunit.client.0.smithi018.stdout:6/286: creat d5/d6/d1b/d3b/f76 x:0 0 0 2024-04-12T20:56:00.037 INFO:tasks.workunit.client.0.smithi018.stdout:9/262: link d3/l4d d3/d6/d9/d1e/l54 0 2024-04-12T20:56:00.040 INFO:tasks.workunit.client.0.smithi018.stdout:9/263: symlink d3/d6/d9/dc/d1c/d21/l55 0 2024-04-12T20:56:00.045 INFO:tasks.workunit.client.0.smithi018.stdout:3/185: dwrite d2/d6/f12 [0,4194304] 0 2024-04-12T20:56:00.053 INFO:tasks.workunit.client.0.smithi018.stdout:3/186: rename d2/l3 to d2/d6/d30/d31/l33 0 2024-04-12T20:56:00.065 INFO:tasks.workunit.client.0.smithi018.stdout:9/264: dwrite d3/d6/f3c [4194304,4194304] 0 2024-04-12T20:56:00.067 INFO:tasks.workunit.client.0.smithi018.stdout:9/265: fsync d3/d6/d9/d3f/f50 0 2024-04-12T20:56:00.069 INFO:tasks.workunit.client.0.smithi018.stdout:9/266: read d3/d6/d1b/f35 [2485256,26465] 0 2024-04-12T20:56:00.074 INFO:tasks.workunit.client.0.smithi018.stdout:9/267: symlink d3/d6/d9/dc/d13/l56 0 2024-04-12T20:56:00.077 INFO:tasks.workunit.client.0.smithi018.stdout:9/268: dread - d3/d6/d9/dc/d13/f42 zero size 2024-04-12T20:56:00.078 INFO:tasks.workunit.client.0.smithi018.stdout:9/269: stat d3/d6/d9/d1e/f45 0 2024-04-12T20:56:00.081 INFO:tasks.workunit.client.0.smithi018.stdout:9/270: fsync d3/d6/d9/dc/d13/f42 0 2024-04-12T20:56:00.082 INFO:tasks.workunit.client.0.smithi018.stdout:3/187: dread d2/f5 [0,4194304] 0 2024-04-12T20:56:00.083 INFO:tasks.workunit.client.0.smithi018.stdout:9/271: write d3/d6/d9/d3f/f44 [527552,70631] 0 2024-04-12T20:56:00.086 INFO:tasks.workunit.client.0.smithi018.stdout:3/188: write d2/d6/f1b [3478636,124362] 0 2024-04-12T20:56:00.086 INFO:tasks.workunit.client.0.smithi018.stdout:9/272: fsync d3/d6/d9/dc/d1c/f34 0 2024-04-12T20:56:00.090 INFO:tasks.workunit.client.0.smithi018.stdout:9/273: symlink d3/d6/d3d/l57 0 2024-04-12T20:56:00.090 INFO:tasks.workunit.client.0.smithi018.stdout:3/189: fsync d2/d6/fc 0 2024-04-12T20:56:00.091 INFO:tasks.workunit.client.0.smithi018.stdout:3/190: fdatasync d2/f19 0 2024-04-12T20:56:00.093 INFO:tasks.workunit.client.0.smithi018.stdout:9/274: symlink d3/d6/d9/d1e/d3a/l58 0 2024-04-12T20:56:00.093 INFO:tasks.workunit.client.0.smithi018.stdout:3/191: mkdir d2/d6/d34 0 2024-04-12T20:56:00.096 INFO:tasks.workunit.client.0.smithi018.stdout:9/275: write d3/d6/d9/dc/d1c/f2e [653299,123119] 0 2024-04-12T20:56:00.099 INFO:tasks.workunit.client.0.smithi018.stdout:9/276: fsync d3/d6/d9/d3f/f50 0 2024-04-12T20:56:00.101 INFO:tasks.workunit.client.0.smithi018.stdout:9/277: read - d3/d6/d9/dc/d46/f48 zero size 2024-04-12T20:56:00.103 INFO:tasks.workunit.client.0.smithi018.stdout:9/278: chown d3/d6/d9/fd 52523 1 2024-04-12T20:56:00.110 INFO:tasks.workunit.client.0.smithi018.stdout:3/192: dwrite d2/f14 [0,4194304] 0 2024-04-12T20:56:00.121 INFO:tasks.workunit.client.0.smithi018.stdout:9/279: dwrite d3/d6/d9/d1e/d3a/f31 [0,4194304] 0 2024-04-12T20:56:00.124 INFO:tasks.workunit.client.0.smithi018.stdout:9/280: creat d3/d6/d9/dc/d1c/d21/f59 x:0 0 0 2024-04-12T20:56:00.126 INFO:tasks.workunit.client.0.smithi018.stdout:3/193: dread d2/f1a [0,4194304] 0 2024-04-12T20:56:00.128 INFO:tasks.workunit.client.0.smithi018.stdout:3/194: write d2/d6/f1b [2842104,33995] 0 2024-04-12T20:56:00.134 INFO:tasks.workunit.client.0.smithi018.stdout:9/281: read d3/d6/f10 [3403784,113189] 0 2024-04-12T20:56:00.160 INFO:tasks.workunit.client.0.smithi018.stdout:3/195: dwrite d2/f28 [0,4194304] 0 2024-04-12T20:56:00.161 INFO:tasks.workunit.client.0.smithi018.stdout:9/282: dwrite d3/d6/d9/fa [0,4194304] 0 2024-04-12T20:56:00.163 INFO:tasks.workunit.client.0.smithi018.stdout:3/196: write d2/d6/f1c [3362555,113161] 0 2024-04-12T20:56:00.173 INFO:tasks.workunit.client.0.smithi018.stdout:9/283: mkdir d3/d6/d9/dc/d5a 0 2024-04-12T20:56:00.178 INFO:tasks.workunit.client.0.smithi018.stdout:9/284: chown d3/d6/d9/dc/d1c/d21/d41 3632743 1 2024-04-12T20:56:00.181 INFO:tasks.workunit.client.0.smithi018.stdout:3/197: dread d2/d6/f12 [0,4194304] 0 2024-04-12T20:56:00.182 INFO:tasks.workunit.client.0.smithi018.stdout:9/285: mkdir d3/d6/d9/dc/d13/d5b 0 2024-04-12T20:56:00.184 INFO:tasks.workunit.client.0.smithi018.stdout:3/198: chown d2/d6/f25 167 1 2024-04-12T20:56:00.184 INFO:tasks.workunit.client.0.smithi018.stdout:9/286: write d3/d6/d9/fd [2324283,102008] 0 2024-04-12T20:56:00.187 INFO:tasks.workunit.client.0.smithi018.stdout:9/287: chown d3/d6/d9/dc/d1c/c18 13178349 1 2024-04-12T20:56:00.188 INFO:tasks.workunit.client.0.smithi018.stdout:3/199: unlink f1 0 2024-04-12T20:56:00.188 INFO:tasks.workunit.client.0.smithi018.stdout:3/200: readlink d2/l26 0 2024-04-12T20:56:00.193 INFO:tasks.workunit.client.0.smithi018.stdout:3/201: rename d2/d6/c20 to d2/d6/c35 0 2024-04-12T20:56:00.197 INFO:tasks.workunit.client.0.smithi018.stdout:3/202: rename d2/l16 to d2/d6/l36 0 2024-04-12T20:56:00.203 INFO:tasks.workunit.client.0.smithi018.stdout:9/288: dwrite d3/d6/d9/d1e/d3a/f31 [0,4194304] 0 2024-04-12T20:56:00.216 INFO:tasks.workunit.client.0.smithi018.stdout:9/289: dread d3/d6/d9/d1e/d3a/f31 [0,4194304] 0 2024-04-12T20:56:00.220 INFO:tasks.workunit.client.0.smithi018.stdout:9/290: mknod d3/d6/d9/dc/d13/c5c 0 2024-04-12T20:56:00.221 INFO:tasks.workunit.client.0.smithi018.stdout:9/291: dread - d3/d6/d9/dc/d13/f42 zero size 2024-04-12T20:56:00.225 INFO:tasks.workunit.client.0.smithi018.stdout:9/292: mkdir d3/d6/d9/dc/d46/d5d 0 2024-04-12T20:56:00.233 INFO:tasks.workunit.client.0.smithi018.stdout:3/203: dread d2/d6/f1b [0,4194304] 0 2024-04-12T20:56:00.243 INFO:tasks.workunit.client.0.smithi018.stdout:9/293: dread d3/d6/d9/dc/d13/f17 [0,4194304] 0 2024-04-12T20:56:00.247 INFO:tasks.workunit.client.0.smithi018.stdout:3/204: dwrite d2/f19 [0,4194304] 0 2024-04-12T20:56:00.258 INFO:tasks.workunit.client.0.smithi018.stdout:3/205: creat d2/d6/d30/f37 x:0 0 0 2024-04-12T20:56:00.261 INFO:tasks.workunit.client.0.smithi018.stdout:9/294: dwrite d3/d6/d9/d3f/f44 [0,4194304] 0 2024-04-12T20:56:00.263 INFO:tasks.workunit.client.0.smithi018.stdout:9/295: dread - d3/d6/d9/dc/d13/f42 zero size 2024-04-12T20:56:00.264 INFO:tasks.workunit.client.0.smithi018.stdout:3/206: mkdir d2/d6/d30/d31/d38 0 2024-04-12T20:56:00.265 INFO:tasks.workunit.client.0.smithi018.stdout:9/296: chown d3/d6/d9/dc/l39 8983560 1 2024-04-12T20:56:00.267 INFO:tasks.workunit.client.0.smithi018.stdout:3/207: write d2/d6/f1b [1015436,91073] 0 2024-04-12T20:56:00.267 INFO:tasks.workunit.client.0.smithi018.stdout:9/297: read d3/d6/d1b/f26 [79497,53458] 0 2024-04-12T20:56:00.275 INFO:tasks.workunit.client.0.smithi018.stdout:9/298: creat d3/f5e x:0 0 0 2024-04-12T20:56:00.277 INFO:tasks.workunit.client.0.smithi018.stdout:9/299: write d3/d6/d9/d1e/f53 [919521,101880] 0 2024-04-12T20:56:00.278 INFO:tasks.workunit.client.0.smithi018.stdout:9/300: fsync d3/d6/d9/dc/f22 0 2024-04-12T20:56:00.280 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:56:00 smithi186 ceph-mon[36597]: from='mgr.24359 172.21.15.186:0/1315906828' entity='mgr.smithi186.kqcfxk' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T20:56:00.280 INFO:tasks.workunit.client.0.smithi018.stdout:3/208: dread d2/d6/f1c [0,4194304] 0 2024-04-12T20:56:00.282 INFO:tasks.workunit.client.0.smithi018.stdout:9/301: mknod d3/d6/d9/dc/d1c/d21/d41/c5f 0 2024-04-12T20:56:00.283 INFO:tasks.workunit.client.0.smithi018.stdout:3/209: mknod d2/d6/d30/d31/c39 0 2024-04-12T20:56:00.286 INFO:tasks.workunit.client.0.smithi018.stdout:9/302: creat d3/d6/d9/dc/d1c/f60 x:0 0 0 2024-04-12T20:56:00.287 INFO:tasks.workunit.client.0.smithi018.stdout:3/210: mkdir d2/d6/d30/d31/d38/d3a 0 2024-04-12T20:56:00.289 INFO:tasks.workunit.client.0.smithi018.stdout:9/303: write d3/d6/f30 [3139152,55628] 0 2024-04-12T20:56:00.291 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:56:00 smithi018 ceph-mon[28799]: from='mgr.24359 172.21.15.186:0/1315906828' entity='mgr.smithi186.kqcfxk' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T20:56:00.292 INFO:tasks.workunit.client.0.smithi018.stdout:9/304: fdatasync d3/d6/d9/dc/d13/f17 0 2024-04-12T20:56:00.298 INFO:tasks.workunit.client.0.smithi018.stdout:9/305: rename d3/d6/d9/dc/d1c/c4f to d3/d6/d3d/c61 0 2024-04-12T20:56:00.304 INFO:tasks.workunit.client.0.smithi018.stdout:9/306: rename d3/d6/f30 to d3/d6/d9/dc/d1c/f62 0 2024-04-12T20:56:00.304 INFO:tasks.workunit.client.0.smithi018.stdout:3/211: dwrite d2/d6/f12 [0,4194304] 0 2024-04-12T20:56:00.318 INFO:tasks.workunit.client.0.smithi018.stdout:9/307: dread d3/d6/d9/dc/d13/f17 [4194304,4194304] 0 2024-04-12T20:56:00.323 INFO:tasks.workunit.client.0.smithi018.stdout:9/308: mknod d3/d6/d9/dc/d1c/d21/c63 0 2024-04-12T20:56:00.326 INFO:tasks.workunit.client.0.smithi018.stdout:9/309: dread d3/d6/d1b/f26 [0,4194304] 0 2024-04-12T20:56:00.331 INFO:tasks.workunit.client.0.smithi018.stdout:9/310: mknod d3/d6/d9/dc/d13/d5b/c64 0 2024-04-12T20:56:00.403 INFO:tasks.workunit.client.0.smithi018.stdout:5/135: sync 2024-04-12T20:56:00.403 INFO:tasks.workunit.client.0.smithi018.stdout:7/229: sync 2024-04-12T20:56:00.403 INFO:tasks.workunit.client.0.smithi018.stdout:0/201: sync 2024-04-12T20:56:00.403 INFO:tasks.workunit.client.0.smithi018.stdout:8/262: sync 2024-04-12T20:56:00.407 INFO:tasks.workunit.client.0.smithi018.stdout:0/202: write da/d15/f35 [455686,51885] 0 2024-04-12T20:56:00.412 INFO:tasks.workunit.client.0.smithi018.stdout:8/263: creat db/d25/d26/d27/f50 x:0 0 0 2024-04-12T20:56:00.412 INFO:tasks.workunit.client.0.smithi018.stdout:0/203: write da/d15/f41 [1029091,113371] 0 2024-04-12T20:56:00.413 INFO:tasks.workunit.client.0.smithi018.stdout:7/230: dread d1/db/d31/d3a/f41 [0,4194304] 0 2024-04-12T20:56:00.414 INFO:tasks.workunit.client.0.smithi018.stdout:0/204: stat da/d15/d3e/d3f 0 2024-04-12T20:56:00.414 INFO:tasks.workunit.client.0.smithi018.stdout:8/264: chown fa 1917 1 2024-04-12T20:56:00.416 INFO:tasks.workunit.client.0.smithi018.stdout:5/136: truncate d0/d2/d9/fb 2480535 0 2024-04-12T20:56:00.420 INFO:tasks.workunit.client.0.smithi018.stdout:7/231: creat d1/db/d31/f58 x:0 0 0 2024-04-12T20:56:00.421 INFO:tasks.workunit.client.0.smithi018.stdout:0/205: mknod da/c45 0 2024-04-12T20:56:00.426 INFO:tasks.workunit.client.0.smithi018.stdout:7/232: write d1/d6/f52 [87039,121013] 0 2024-04-12T20:56:00.428 INFO:tasks.workunit.client.0.smithi018.stdout:7/233: dread - d1/db/d31/d30/f57 zero size 2024-04-12T20:56:00.429 INFO:tasks.workunit.client.0.smithi018.stdout:5/137: unlink d0/d2/d9/d1e/d2e/f22 0 2024-04-12T20:56:00.430 INFO:tasks.workunit.client.0.smithi018.stdout:5/138: read - d0/d2/d11/f2a zero size 2024-04-12T20:56:00.446 INFO:tasks.workunit.client.0.smithi018.stdout:8/265: write f0 [1699485,118243] 0 2024-04-12T20:56:00.449 INFO:tasks.workunit.client.0.smithi018.stdout:7/234: write d1/db/f35 [1266566,50432] 0 2024-04-12T20:56:00.450 INFO:tasks.workunit.client.0.smithi018.stdout:5/139: read d0/d2b/f2d [1348304,108745] 0 2024-04-12T20:56:00.451 INFO:tasks.workunit.client.0.smithi018.stdout:7/235: dread - d1/f3e zero size 2024-04-12T20:56:00.454 INFO:tasks.workunit.client.0.smithi018.stdout:5/140: write d0/d2/d9/d1e/d2e/f12 [3961712,13245] 0 2024-04-12T20:56:00.454 INFO:tasks.workunit.client.0.smithi018.stdout:9/311: dwrite d3/d6/d1b/f35 [0,4194304] 0 2024-04-12T20:56:00.455 INFO:tasks.workunit.client.0.smithi018.stdout:7/236: mkdir d1/db/d31/d59 0 2024-04-12T20:56:00.461 INFO:tasks.workunit.client.0.smithi018.stdout:9/312: creat d3/d6/d9/dc/d46/f65 x:0 0 0 2024-04-12T20:56:00.461 INFO:tasks.workunit.client.0.smithi018.stdout:7/237: stat d1/d6/l17 0 2024-04-12T20:56:00.461 INFO:tasks.workunit.client.0.smithi018.stdout:8/266: dread db/dc/f40 [0,4194304] 0 2024-04-12T20:56:00.464 INFO:tasks.workunit.client.0.smithi018.stdout:5/141: creat d0/dc/d10/d26/d30/f31 x:0 0 0 2024-04-12T20:56:00.464 INFO:tasks.workunit.client.0.smithi018.stdout:8/267: mknod db/dc/c51 0 2024-04-12T20:56:00.466 INFO:tasks.workunit.client.0.smithi018.stdout:7/238: write d1/db/d31/d45/d53/f56 [3027700,32557] 0 2024-04-12T20:56:00.467 INFO:tasks.workunit.client.0.smithi018.stdout:9/313: rename d3/d6/d9/c32 to d3/d6/d1b/c66 0 2024-04-12T20:56:00.470 INFO:tasks.workunit.client.0.smithi018.stdout:9/314: write d3/d6/d9/dc/d1c/f4e [885721,100400] 0 2024-04-12T20:56:00.473 INFO:tasks.workunit.client.0.smithi018.stdout:9/315: unlink d3/c14 0 2024-04-12T20:56:00.475 INFO:tasks.workunit.client.0.smithi018.stdout:9/316: write d3/d6/d1b/f35 [1644720,71519] 0 2024-04-12T20:56:00.476 INFO:tasks.workunit.client.0.smithi018.stdout:5/142: getdents d0/dc 0 2024-04-12T20:56:00.477 INFO:tasks.workunit.client.0.smithi018.stdout:7/239: creat d1/d6/f5a x:0 0 0 2024-04-12T20:56:00.479 INFO:tasks.workunit.client.0.smithi018.stdout:9/317: write d3/d6/d9/dc/d1c/f34 [5583128,123512] 0 2024-04-12T20:56:00.482 INFO:tasks.workunit.client.0.smithi018.stdout:7/240: creat d1/db/d31/d3a/d42/d4c/f5b x:0 0 0 2024-04-12T20:56:00.485 INFO:tasks.workunit.client.0.smithi018.stdout:8/268: dwrite db/dc/f1e [0,4194304] 0 2024-04-12T20:56:00.485 INFO:tasks.workunit.client.0.smithi018.stdout:5/143: write d0/d2/d9/fb [584089,99247] 0 2024-04-12T20:56:00.485 INFO:tasks.workunit.client.0.smithi018.stdout:8/269: stat db/l1d 0 2024-04-12T20:56:00.485 INFO:tasks.workunit.client.0.smithi018.stdout:7/241: rename d1/d6/d8 to d1/d6/d8/d5c 22 2024-04-12T20:56:00.490 INFO:tasks.workunit.client.0.smithi018.stdout:8/270: creat db/d25/f52 x:0 0 0 2024-04-12T20:56:00.491 INFO:tasks.workunit.client.0.smithi018.stdout:5/144: mknod d0/dc/d10/c32 0 2024-04-12T20:56:00.492 INFO:tasks.workunit.client.0.smithi018.stdout:7/242: fsync d1/db/d31/d3a/f41 0 2024-04-12T20:56:00.494 INFO:tasks.workunit.client.0.smithi018.stdout:7/243: truncate d1/db/f35 1927130 0 2024-04-12T20:56:00.496 INFO:tasks.workunit.client.0.smithi018.stdout:7/244: dread - d1/db/d31/f37 zero size 2024-04-12T20:56:00.496 INFO:tasks.workunit.client.0.smithi018.stdout:8/271: creat db/d25/d26/f53 x:0 0 0 2024-04-12T20:56:00.499 INFO:tasks.workunit.client.0.smithi018.stdout:7/245: symlink d1/db/d31/d3a/d42/d4c/l5d 0 2024-04-12T20:56:00.501 INFO:tasks.workunit.client.0.smithi018.stdout:7/246: chown d1/f2 0 1 2024-04-12T20:56:00.502 INFO:tasks.workunit.client.0.smithi018.stdout:5/145: rename d0/dc to d0/d2/d33 0 2024-04-12T20:56:00.503 INFO:tasks.workunit.client.0.smithi018.stdout:7/247: truncate d1/f32 4889488 0 2024-04-12T20:56:00.504 INFO:tasks.workunit.client.0.smithi018.stdout:8/272: write db/d25/d26/d27/f33 [318438,49077] 0 2024-04-12T20:56:00.505 INFO:tasks.workunit.client.0.smithi018.stdout:5/146: rename d0/d2 to d0/d2/d33/d10/d26/d30/d34 22 2024-04-12T20:56:00.507 INFO:tasks.workunit.client.0.smithi018.stdout:8/273: creat db/d25/d26/d27/f54 x:0 0 0 2024-04-12T20:56:00.508 INFO:tasks.workunit.client.0.smithi018.stdout:5/147: mkdir d0/d2/d9/d35 0 2024-04-12T20:56:00.520 INFO:tasks.workunit.client.0.smithi018.stdout:8/274: dread f7 [0,4194304] 0 2024-04-12T20:56:00.524 INFO:tasks.workunit.client.0.smithi018.stdout:7/248: dwrite d1/db/d31/d30/f4a [0,4194304] 0 2024-04-12T20:56:00.525 INFO:tasks.workunit.client.0.smithi018.stdout:8/275: fsync db/d25/d26/d27/f50 0 2024-04-12T20:56:00.525 INFO:tasks.workunit.client.0.smithi018.stdout:5/148: fsync d0/d2/d9/d1e/d2e/f12 0 2024-04-12T20:56:00.531 INFO:tasks.workunit.client.0.smithi018.stdout:5/149: rmdir d0/d2/d33/d10/d26 39 2024-04-12T20:56:00.533 INFO:tasks.workunit.client.0.smithi018.stdout:7/249: truncate d1/db/d31/d3a/d42/d4c/f51 599463 0 2024-04-12T20:56:00.540 INFO:tasks.workunit.client.0.smithi018.stdout:5/150: rmdir d0/d2/d9/d1e/d2e 39 2024-04-12T20:56:00.541 INFO:tasks.workunit.client.0.smithi018.stdout:7/250: mknod d1/d6/d8/c5e 0 2024-04-12T20:56:00.545 INFO:tasks.workunit.client.0.smithi018.stdout:5/151: chown d0/d2b/f2d 33 1 2024-04-12T20:56:00.547 INFO:tasks.workunit.client.0.smithi018.stdout:5/152: chown d0/d2/d33 2038940 1 2024-04-12T20:56:00.550 INFO:tasks.workunit.client.0.smithi018.stdout:8/276: dread f0 [0,4194304] 0 2024-04-12T20:56:00.554 INFO:tasks.workunit.client.0.smithi018.stdout:8/277: creat db/d3b/f55 x:0 0 0 2024-04-12T20:56:00.556 INFO:tasks.workunit.client.0.smithi018.stdout:5/153: link d0/d2/l21 d0/d2/d33/l36 0 2024-04-12T20:56:00.556 INFO:tasks.workunit.client.0.smithi018.stdout:7/251: truncate d1/db/f35 1331259 0 2024-04-12T20:56:00.558 INFO:tasks.workunit.client.0.smithi018.stdout:8/278: mkdir db/d4e/d56 0 2024-04-12T20:56:00.562 INFO:tasks.workunit.client.0.smithi018.stdout:7/252: write d1/f2f [701983,130028] 0 2024-04-12T20:56:00.564 INFO:tasks.workunit.client.0.smithi018.stdout:7/253: dread - d1/f3e zero size 2024-04-12T20:56:00.565 INFO:tasks.workunit.client.0.smithi018.stdout:8/279: mkdir db/d25/d26/d57 0 2024-04-12T20:56:00.567 INFO:tasks.workunit.client.0.smithi018.stdout:7/254: chown d1/db/d31/l49 213467 1 2024-04-12T20:56:00.570 INFO:tasks.workunit.client.0.smithi018.stdout:7/255: mkdir d1/d6/d8/d5f 0 2024-04-12T20:56:00.574 INFO:tasks.workunit.client.0.smithi018.stdout:7/256: write d1/db/d31/d30/f4a [4602197,121813] 0 2024-04-12T20:56:00.574 INFO:tasks.workunit.client.0.smithi018.stdout:5/154: dread d0/d2/d9/fb [0,4194304] 0 2024-04-12T20:56:00.576 INFO:tasks.workunit.client.0.smithi018.stdout:9/318: rmdir d3/d6 39 2024-04-12T20:56:00.577 INFO:tasks.workunit.client.0.smithi018.stdout:5/155: write d0/d2/d9/fe [5698946,68640] 0 2024-04-12T20:56:00.588 INFO:tasks.workunit.client.0.smithi018.stdout:9/319: link d3/d6/d9/dc/d13/l2a d3/d6/l67 0 2024-04-12T20:56:00.600 INFO:tasks.workunit.client.0.smithi018.stdout:5/156: dread d0/d2/d33/d10/f17 [4194304,4194304] 0 2024-04-12T20:56:00.603 INFO:tasks.workunit.client.0.smithi018.stdout:5/157: chown d0/d2/d33/d10/d2f 14 1 2024-04-12T20:56:00.604 INFO:tasks.workunit.client.0.smithi018.stdout:9/320: dwrite d3/d6/d9/d1e/f45 [0,4194304] 0 2024-04-12T20:56:00.605 INFO:tasks.workunit.client.0.smithi018.stdout:5/158: write d0/d2/f1a [4480535,67159] 0 2024-04-12T20:56:00.855 INFO:tasks.workunit.client.0.smithi018.stdout:4/231: sync 2024-04-12T20:56:00.877 INFO:tasks.workunit.client.0.smithi018.stdout:4/232: truncate d3/d32/d33/d1f/f2e 230465 0 2024-04-12T20:56:00.889 INFO:tasks.workunit.client.0.smithi018.stdout:4/233: mkdir d3/d34/d40 0 2024-04-12T20:56:00.896 INFO:tasks.workunit.client.0.smithi018.stdout:4/234: rmdir d3/db 39 2024-04-12T20:56:00.952 INFO:tasks.workunit.client.0.smithi018.stdout:1/133: sync 2024-04-12T20:56:00.953 INFO:tasks.workunit.client.0.smithi018.stdout:2/162: sync 2024-04-12T20:56:00.960 INFO:tasks.workunit.client.0.smithi018.stdout:1/134: mknod d3/c2f 0 2024-04-12T20:56:00.966 INFO:tasks.workunit.client.0.smithi018.stdout:1/135: truncate d3/f10 969635 0 2024-04-12T20:56:00.968 INFO:tasks.workunit.client.0.smithi018.stdout:2/163: truncate d5/d8/d25/f34 782340 0 2024-04-12T20:56:00.968 INFO:tasks.workunit.client.0.smithi018.stdout:1/136: stat d3/f7 0 2024-04-12T20:56:00.995 INFO:tasks.workunit.client.0.smithi018.stdout:1/137: dread d3/f4 [0,4194304] 0 2024-04-12T20:56:00.998 INFO:tasks.workunit.client.0.smithi018.stdout:2/164: dread d5/d8/d25/f34 [0,4194304] 0 2024-04-12T20:56:01.001 INFO:tasks.workunit.client.0.smithi018.stdout:2/165: read d5/d8/d10/f30 [2014308,33437] 0 2024-04-12T20:56:01.007 INFO:tasks.workunit.client.0.smithi018.stdout:2/166: mkdir d5/d3a 0 2024-04-12T20:56:01.014 INFO:tasks.workunit.client.0.smithi018.stdout:2/167: link d5/d8/cb d5/d8/d10/c3b 0 2024-04-12T20:56:01.019 INFO:tasks.workunit.client.0.smithi018.stdout:2/168: write d5/d8/d10/d19/d24/f32 [431580,105734] 0 2024-04-12T20:56:01.022 INFO:tasks.workunit.client.0.smithi018.stdout:2/169: fsync f3 0 2024-04-12T20:56:01.027 INFO:tasks.workunit.client.0.smithi018.stdout:1/138: dwrite d3/f4 [0,4194304] 0 2024-04-12T20:56:01.028 INFO:tasks.workunit.client.0.smithi018.stdout:2/170: mkdir d5/d7/d28/d3c 0 2024-04-12T20:56:01.029 INFO:tasks.workunit.client.0.smithi018.stdout:2/171: write f3 [2203586,98399] 0 2024-04-12T20:56:01.038 INFO:tasks.workunit.client.0.smithi018.stdout:1/139: rename c2 to d3/d9/da/dd/c30 0 2024-04-12T20:56:01.038 INFO:tasks.workunit.client.0.smithi018.stdout:2/172: mknod d5/d7/d28/d3c/c3d 0 2024-04-12T20:56:01.041 INFO:tasks.workunit.client.0.smithi018.stdout:1/140: getdents d3/d9/d11 0 2024-04-12T20:56:01.053 INFO:tasks.workunit.client.0.smithi018.stdout:2/173: dread d5/d8/d10/d19/d24/f29 [0,4194304] 0 2024-04-12T20:56:01.056 INFO:tasks.workunit.client.0.smithi018.stdout:2/174: mkdir d5/d7/d28/d3c/d3e 0 2024-04-12T20:56:01.058 INFO:tasks.workunit.client.0.smithi018.stdout:2/175: mknod d5/d8/d25/c3f 0 2024-04-12T20:56:01.060 INFO:tasks.workunit.client.0.smithi018.stdout:2/176: fsync d5/d7/d28/f2d 0 2024-04-12T20:56:01.061 INFO:tasks.workunit.client.0.smithi018.stdout:3/212: sync 2024-04-12T20:56:01.061 INFO:tasks.workunit.client.0.smithi018.stdout:0/206: sync 2024-04-12T20:56:01.061 INFO:tasks.workunit.client.0.smithi018.stdout:5/159: sync 2024-04-12T20:56:01.061 INFO:tasks.workunit.client.0.smithi018.stdout:6/287: sync 2024-04-12T20:56:01.061 INFO:tasks.workunit.client.0.smithi018.stdout:4/235: sync 2024-04-12T20:56:01.061 INFO:tasks.workunit.client.0.smithi018.stdout:8/280: sync 2024-04-12T20:56:01.061 INFO:tasks.workunit.client.0.smithi018.stdout:7/257: sync 2024-04-12T20:56:01.061 INFO:tasks.workunit.client.0.smithi018.stdout:9/321: sync 2024-04-12T20:56:01.066 INFO:tasks.workunit.client.0.smithi018.stdout:5/160: mkdir d0/d2/d11/d37 0 2024-04-12T20:56:01.067 INFO:tasks.workunit.client.0.smithi018.stdout:8/281: dread - db/d25/d26/d27/f50 zero size 2024-04-12T20:56:01.067 INFO:tasks.workunit.client.0.smithi018.stdout:7/258: stat d1/db/d31/d3a/d42/d4c/f5b 0 2024-04-12T20:56:01.068 INFO:tasks.workunit.client.0.smithi018.stdout:4/236: read d3/d32/d33/f17 [747785,51555] 0 2024-04-12T20:56:01.069 INFO:tasks.workunit.client.0.smithi018.stdout:0/207: creat da/d15/d17/d21/f46 x:0 0 0 2024-04-12T20:56:01.071 INFO:tasks.workunit.client.0.smithi018.stdout:7/259: chown d1/db/d31/d3a/d42/d4c 226 1 2024-04-12T20:56:01.072 INFO:tasks.workunit.client.0.smithi018.stdout:8/282: fdatasync db/d3b/f47 0 2024-04-12T20:56:01.072 INFO:tasks.workunit.client.0.smithi018.stdout:3/213: getdents d2/d6/d30/d31/d38/d3a 0 2024-04-12T20:56:01.073 INFO:tasks.workunit.client.0.smithi018.stdout:2/177: getdents d5/d8 0 2024-04-12T20:56:01.075 INFO:tasks.workunit.client.0.smithi018.stdout:6/288: rename d5/d15/d60/l6e to d5/df/d19/d43/d5f/l77 0 2024-04-12T20:56:01.075 INFO:tasks.workunit.client.0.smithi018.stdout:7/260: stat d1/f2 0 2024-04-12T20:56:01.077 INFO:tasks.workunit.client.0.smithi018.stdout:3/214: write d2/d6/f2f [300172,73853] 0 2024-04-12T20:56:01.077 INFO:tasks.workunit.client.0.smithi018.stdout:0/208: mkdir da/d15/d17/d21/d36/d47 0 2024-04-12T20:56:01.079 INFO:tasks.workunit.client.0.smithi018.stdout:8/283: mknod db/c58 0 2024-04-12T20:56:01.080 INFO:tasks.workunit.client.0.smithi018.stdout:9/322: truncate d3/d6/d9/d1e/f20 1212784 0 2024-04-12T20:56:01.086 INFO:tasks.workunit.client.0.smithi018.stdout:6/289: getdents d5/df/d31 0 2024-04-12T20:56:01.091 INFO:tasks.workunit.client.0.smithi018.stdout:4/237: link d3/d32/d33/l28 d3/l41 0 2024-04-12T20:56:01.092 INFO:tasks.workunit.client.0.smithi018.stdout:5/161: rename d0/d2/d33/d10/d26/l2c to d0/d2b/l38 0 2024-04-12T20:56:01.093 INFO:tasks.workunit.client.0.smithi018.stdout:8/284: mkdir db/d4e/d56/d59 0 2024-04-12T20:56:01.097 INFO:tasks.workunit.client.0.smithi018.stdout:6/290: chown d5/d6/d1b/c41 89339 1 2024-04-12T20:56:01.098 INFO:tasks.workunit.client.0.smithi018.stdout:4/238: symlink d3/d34/d3d/l42 0 2024-04-12T20:56:01.102 INFO:tasks.workunit.client.0.smithi018.stdout:3/215: getdents d2/d6/d30/d31/d38 0 2024-04-12T20:56:01.104 INFO:tasks.workunit.client.0.smithi018.stdout:7/261: dread d1/d6/f24 [0,4194304] 0 2024-04-12T20:56:01.112 INFO:tasks.workunit.client.0.smithi018.stdout:4/239: write d3/db/f37 [3905718,5834] 0 2024-04-12T20:56:01.115 INFO:tasks.workunit.client.0.smithi018.stdout:2/178: dwrite d5/d8/d10/d19/d24/f32 [0,4194304] 0 2024-04-12T20:56:01.116 INFO:tasks.workunit.client.0.smithi018.stdout:8/285: rename db/dc/lf to db/d25/d26/l5a 0 2024-04-12T20:56:01.117 INFO:tasks.workunit.client.0.smithi018.stdout:6/291: read d5/df/f2a [654917,50812] 0 2024-04-12T20:56:01.120 INFO:tasks.workunit.client.0.smithi018.stdout:9/323: dwrite d3/d6/d9/dc/d1c/d21/f59 [0,4194304] 0 2024-04-12T20:56:01.123 INFO:tasks.workunit.client.0.smithi018.stdout:6/292: rmdir d5/d6/d1b/d3b/d35 39 2024-04-12T20:56:01.127 INFO:tasks.workunit.client.0.smithi018.stdout:5/162: truncate d0/d2/d9/d1e/d2e/f12 167686 0 2024-04-12T20:56:01.127 INFO:tasks.workunit.client.0.smithi018.stdout:5/163: chown d0 701270 1 2024-04-12T20:56:01.127 INFO:tasks.workunit.client.0.smithi018.stdout:3/216: link d2/f19 d2/d6/d30/d31/d38/f3b 0 2024-04-12T20:56:01.128 INFO:tasks.workunit.client.0.smithi018.stdout:9/324: symlink d3/d6/d9/dc/d13/l68 0 2024-04-12T20:56:01.130 INFO:tasks.workunit.client.0.smithi018.stdout:0/209: dread da/fc [0,4194304] 0 2024-04-12T20:56:01.133 INFO:tasks.workunit.client.0.smithi018.stdout:7/262: creat d1/d6/f60 x:0 0 0 2024-04-12T20:56:01.135 INFO:tasks.workunit.client.0.smithi018.stdout:8/286: mknod db/dc/c5b 0 2024-04-12T20:56:01.136 INFO:tasks.workunit.client.0.smithi018.stdout:4/240: rename d3/d32/d33/d1f to d3/d34/d43 0 2024-04-12T20:56:01.137 INFO:tasks.workunit.client.0.smithi018.stdout:9/325: creat d3/d6/d9/d1e/d3a/f69 x:0 0 0 2024-04-12T20:56:01.139 INFO:tasks.workunit.client.0.smithi018.stdout:2/179: symlink d5/l40 0 2024-04-12T20:56:01.139 INFO:tasks.workunit.client.0.smithi018.stdout:6/293: write d5/d6/d1b/d3b/d35/f33 [381305,115088] 0 2024-04-12T20:56:01.143 INFO:tasks.workunit.client.0.smithi018.stdout:6/294: chown d5/df/d19/d2b 4333516 1 2024-04-12T20:56:01.149 INFO:tasks.workunit.client.0.smithi018.stdout:2/180: dread d5/d7/f35 [0,4194304] 0 2024-04-12T20:56:01.151 INFO:tasks.workunit.client.0.smithi018.stdout:6/295: chown d5/df/d19/d43/c66 229468723 1 2024-04-12T20:56:01.153 INFO:tasks.workunit.client.0.smithi018.stdout:2/181: fdatasync d5/d7/f35 0 2024-04-12T20:56:01.155 INFO:tasks.workunit.client.0.smithi018.stdout:8/287: fdatasync db/d25/f2c 0 2024-04-12T20:56:01.165 INFO:tasks.workunit.client.0.smithi018.stdout:3/217: dread d2/d6/f24 [0,4194304] 0 2024-04-12T20:56:01.165 INFO:tasks.workunit.client.0.smithi018.stdout:5/164: creat d0/d2/d33/d10/f39 x:0 0 0 2024-04-12T20:56:01.170 INFO:tasks.workunit.client.0.smithi018.stdout:4/241: dread d3/d34/d43/f2f [0,4194304] 0 2024-04-12T20:56:01.173 INFO:tasks.workunit.client.0.smithi018.stdout:3/218: write d2/d6/f1b [1245233,102201] 0 2024-04-12T20:56:01.173 INFO:tasks.workunit.client.0.smithi018.stdout:9/326: truncate d3/d6/f10 2758920 0 2024-04-12T20:56:01.179 INFO:tasks.workunit.client.0.smithi018.stdout:9/327: write d3/f5e [662366,5178] 0 2024-04-12T20:56:01.184 INFO:tasks.workunit.client.0.smithi018.stdout:7/263: getdents d1 0 2024-04-12T20:56:01.184 INFO:tasks.workunit.client.0.smithi018.stdout:0/210: dwrite da/d15/d17/f19 [0,4194304] 0 2024-04-12T20:56:01.188 INFO:tasks.workunit.client.0.smithi018.stdout:4/242: unlink d3/db/l3c 0 2024-04-12T20:56:01.190 INFO:tasks.workunit.client.0.smithi018.stdout:7/264: write d1/db/d31/d3a/d42/f44 [2596191,68071] 0 2024-04-12T20:56:01.191 INFO:tasks.workunit.client.0.smithi018.stdout:0/211: creat da/d15/d17/f48 x:0 0 0 2024-04-12T20:56:01.192 INFO:tasks.workunit.client.0.smithi018.stdout:3/219: getdents d2/d6/d34 0 2024-04-12T20:56:01.200 INFO:tasks.workunit.client.0.smithi018.stdout:2/182: dwrite d5/d8/d10/d19/d24/f36 [0,4194304] 0 2024-04-12T20:56:01.200 INFO:tasks.workunit.client.0.smithi018.stdout:6/296: link d5/df/d19/f25 d5/f78 0 2024-04-12T20:56:01.205 INFO:tasks.workunit.client.0.smithi018.stdout:7/265: write d1/f2f [228351,111353] 0 2024-04-12T20:56:01.208 INFO:tasks.workunit.client.0.smithi018.stdout:3/220: write d2/f1a [1472892,95626] 0 2024-04-12T20:56:01.213 INFO:tasks.workunit.client.0.smithi018.stdout:1/141: sync 2024-04-12T20:56:01.213 INFO:tasks.workunit.client.0.smithi018.stdout:4/243: creat d3/d34/d40/f44 x:0 0 0 2024-04-12T20:56:01.216 INFO:tasks.workunit.client.0.smithi018.stdout:6/297: symlink d5/df/d31/d4e/l79 0 2024-04-12T20:56:01.218 INFO:tasks.workunit.client.0.smithi018.stdout:5/165: dwrite d0/d2/d11/f29 [0,4194304] 0 2024-04-12T20:56:01.218 INFO:tasks.workunit.client.0.smithi018.stdout:7/266: chown d1/d6/f24 8581364 1 2024-04-12T20:56:01.222 INFO:tasks.workunit.client.0.smithi018.stdout:0/212: creat da/d15/d17/d21/d27/f49 x:0 0 0 2024-04-12T20:56:01.224 INFO:tasks.workunit.client.0.smithi018.stdout:8/288: dwrite db/dc/f1a [0,4194304] 0 2024-04-12T20:56:01.230 INFO:tasks.workunit.client.0.smithi018.stdout:1/142: truncate d3/d9/da/f20 785220 0 2024-04-12T20:56:01.231 INFO:tasks.workunit.client.0.smithi018.stdout:4/244: unlink d3/f35 0 2024-04-12T20:56:01.231 INFO:tasks.workunit.client.0.smithi018.stdout:0/213: read da/d15/d17/f19 [677943,33505] 0 2024-04-12T20:56:01.231 INFO:tasks.workunit.client.0.smithi018.stdout:5/166: write d0/d2/d33/d10/f39 [144625,115100] 0 2024-04-12T20:56:01.231 INFO:tasks.workunit.client.0.smithi018.stdout:3/221: mknod d2/d6/d30/c3c 0 2024-04-12T20:56:01.232 INFO:tasks.workunit.client.0.smithi018.stdout:8/289: read - db/d25/d26/f3a zero size 2024-04-12T20:56:01.236 INFO:tasks.workunit.client.0.smithi018.stdout:7/267: unlink d1/db/l20 0 2024-04-12T20:56:01.239 INFO:tasks.workunit.client.0.smithi018.stdout:4/245: readlink d3/lf 0 2024-04-12T20:56:01.240 INFO:tasks.workunit.client.0.smithi018.stdout:9/328: dwrite d3/d6/d9/d3f/f50 [0,4194304] 0 2024-04-12T20:56:01.244 INFO:tasks.workunit.client.0.smithi018.stdout:3/222: mkdir d2/d6/d3d 0 2024-04-12T20:56:01.245 INFO:tasks.workunit.client.0.smithi018.stdout:1/143: mkdir d3/d9/da/dd/d31 0 2024-04-12T20:56:01.246 INFO:tasks.workunit.client.0.smithi018.stdout:7/268: write d1/f1b [1339746,101453] 0 2024-04-12T20:56:01.258 INFO:tasks.workunit.client.0.smithi018.stdout:0/214: symlink da/d15/d17/d21/d36/l4a 0 2024-04-12T20:56:01.262 INFO:tasks.workunit.client.0.smithi018.stdout:8/290: getdents db/dc/d45 0 2024-04-12T20:56:01.262 INFO:tasks.workunit.client.0.smithi018.stdout:5/167: dread d0/d2/d9/f1d [0,4194304] 0 2024-04-12T20:56:01.264 INFO:tasks.workunit.client.0.smithi018.stdout:9/329: symlink d3/d6/d9/l6a 0 2024-04-12T20:56:01.267 INFO:tasks.workunit.client.0.smithi018.stdout:4/246: rename d3/d34/d3d/l42 to d3/d32/d33/l45 0 2024-04-12T20:56:01.267 INFO:tasks.workunit.client.0.smithi018.stdout:8/291: write db/d25/d26/f53 [716395,93068] 0 2024-04-12T20:56:01.271 INFO:tasks.workunit.client.0.smithi018.stdout:3/223: dread d2/d6/f2e [0,4194304] 0 2024-04-12T20:56:01.275 INFO:tasks.workunit.client.0.smithi018.stdout:4/247: chown d3/db/f22 3 1 2024-04-12T20:56:01.275 INFO:tasks.workunit.client.0.smithi018.stdout:8/292: chown db/dc/d45 0 1 2024-04-12T20:56:01.280 INFO:tasks.workunit.client.0.smithi018.stdout:1/144: rename d3/d9/da/dd/de to d3/d9/da/dd/d31/d32 0 2024-04-12T20:56:01.281 INFO:tasks.workunit.client.0.smithi018.stdout:2/183: dwrite d5/d8/d10/d19/d24/f29 [0,4194304] 0 2024-04-12T20:56:01.283 INFO:tasks.workunit.client.0.smithi018.stdout:7/269: link d1/db/l47 d1/db/d31/d3a/l61 0 2024-04-12T20:56:01.285 INFO:tasks.workunit.client.0.smithi018.stdout:6/298: dread d5/df/d3f/f49 [0,4194304] 0 2024-04-12T20:56:01.290 INFO:tasks.workunit.client.0.smithi018.stdout:7/270: truncate d1/f3e 307188 0 2024-04-12T20:56:01.291 INFO:tasks.workunit.client.0.smithi018.stdout:0/215: write da/d15/f33 [778816,16996] 0 2024-04-12T20:56:01.295 INFO:tasks.workunit.client.0.smithi018.stdout:6/299: symlink d5/df/d19/d74/l7a 0 2024-04-12T20:56:01.296 INFO:tasks.workunit.client.0.smithi018.stdout:4/248: symlink d3/d34/d40/l46 0 2024-04-12T20:56:01.298 INFO:tasks.workunit.client.0.smithi018.stdout:7/271: read d1/db/f35 [922476,95138] 0 2024-04-12T20:56:01.300 INFO:tasks.workunit.client.0.smithi018.stdout:4/249: fdatasync d3/d32/d33/f30 0 2024-04-12T20:56:01.301 INFO:tasks.workunit.client.0.smithi018.stdout:3/224: creat d2/d6/d30/f3e x:0 0 0 2024-04-12T20:56:01.302 INFO:tasks.workunit.client.0.smithi018.stdout:2/184: chown d5/d8/c21 63032301 1 2024-04-12T20:56:01.303 INFO:tasks.workunit.client.0.smithi018.stdout:5/168: rename d0/d2/d9/fb to d0/d2/d9/f3a 0 2024-04-12T20:56:01.306 INFO:tasks.workunit.client.0.smithi018.stdout:2/185: chown d5/c22 2341 1 2024-04-12T20:56:01.310 INFO:tasks.workunit.client.0.smithi018.stdout:0/216: symlink da/d15/d3e/l4b 0 2024-04-12T20:56:01.316 INFO:tasks.workunit.client.0.smithi018.stdout:9/330: dwrite d3/d6/f1f [0,4194304] 0 2024-04-12T20:56:01.318 INFO:tasks.workunit.client.0.smithi018.stdout:1/145: dwrite d3/d9/da/dd/f17 [0,4194304] 0 2024-04-12T20:56:01.322 INFO:tasks.workunit.client.0.smithi018.stdout:7/272: unlink d1/l1a 0 2024-04-12T20:56:01.323 INFO:tasks.workunit.client.0.smithi018.stdout:3/225: symlink d2/l3f 0 2024-04-12T20:56:01.323 INFO:tasks.workunit.client.0.smithi018.stdout:8/293: creat db/d25/d26/f5c x:0 0 0 2024-04-12T20:56:01.327 INFO:tasks.workunit.client.0.smithi018.stdout:0/217: rename da/d15/d17/f3a to da/d15/d17/d21/d36/f4c 0 2024-04-12T20:56:01.330 INFO:tasks.workunit.client.0.smithi018.stdout:9/331: mkdir d3/d6/d9/dc/d13/d5b/d6b 0 2024-04-12T20:56:01.339 INFO:tasks.workunit.client.0.smithi018.stdout:3/226: write d2/f5 [2312285,15813] 0 2024-04-12T20:56:01.350 INFO:tasks.workunit.client.0.smithi018.stdout:8/294: dread db/d25/d26/f53 [0,4194304] 0 2024-04-12T20:56:01.350 INFO:tasks.workunit.client.0.smithi018.stdout:6/300: dwrite d5/fd [0,4194304] 0 2024-04-12T20:56:01.350 INFO:tasks.workunit.client.0.smithi018.stdout:4/250: link d3/db/l3a d3/d32/l47 0 2024-04-12T20:56:01.350 INFO:tasks.workunit.client.0.smithi018.stdout:8/295: write db/f2d [9287102,42672] 0 2024-04-12T20:56:01.352 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:56:01 smithi018 ceph-mon[28799]: pgmap v33: 65 pgs: 65 active+clean; 325 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 354 KiB/s rd, 22 MiB/s wr, 152 op/s 2024-04-12T20:56:01.359 INFO:tasks.workunit.client.0.smithi018.stdout:8/296: chown db/d25/d26/l29 1 1 2024-04-12T20:56:01.364 INFO:tasks.workunit.client.0.smithi018.stdout:1/146: dwrite d3/f4 [4194304,4194304] 0 2024-04-12T20:56:01.364 INFO:tasks.workunit.client.0.smithi018.stdout:0/218: mkdir da/d15/d4d 0 2024-04-12T20:56:01.369 INFO:tasks.workunit.client.0.smithi018.stdout:8/297: truncate db/d3b/f55 33234 0 2024-04-12T20:56:01.369 INFO:tasks.workunit.client.0.smithi018.stdout:6/301: mknod d5/df/d19/d43/c7b 0 2024-04-12T20:56:01.370 INFO:tasks.workunit.client.0.smithi018.stdout:8/298: chown f6 2262 1 2024-04-12T20:56:01.379 INFO:tasks.workunit.client.0.smithi018.stdout:4/251: truncate d3/d32/d33/f17 843389 0 2024-04-12T20:56:01.381 INFO:tasks.workunit.client.0.smithi018.stdout:2/186: dread d5/d8/d10/f15 [4194304,4194304] 0 2024-04-12T20:56:01.384 INFO:tasks.workunit.client.0.smithi018.stdout:0/219: mknod da/c4e 0 2024-04-12T20:56:01.387 INFO:tasks.workunit.client.0.smithi018.stdout:4/252: write d3/db/f22 [947100,117961] 0 2024-04-12T20:56:01.390 INFO:tasks.workunit.client.0.smithi018.stdout:3/227: write d2/f19 [7565843,99124] 0 2024-04-12T20:56:01.390 INFO:tasks.workunit.client.0.smithi018.stdout:8/299: symlink db/d4e/d56/d59/l5d 0 2024-04-12T20:56:01.396 INFO:tasks.workunit.client.0.smithi018.stdout:5/169: dwrite d0/d2/d9/d1e/d2e/f12 [0,4194304] 0 2024-04-12T20:56:01.400 INFO:tasks.workunit.client.0.smithi018.stdout:0/220: rmdir da/d15/d3e 39 2024-04-12T20:56:01.405 INFO:tasks.workunit.client.0.smithi018.stdout:7/273: rmdir d1/db/d31/d59 0 2024-04-12T20:56:01.405 INFO:tasks.workunit.client.0.smithi018.stdout:1/147: dwrite d3/f10 [0,4194304] 0 2024-04-12T20:56:01.406 INFO:tasks.workunit.client.0.smithi018.stdout:6/302: dwrite d5/d15/f39 [0,4194304] 0 2024-04-12T20:56:01.412 INFO:tasks.workunit.client.0.smithi018.stdout:4/253: mkdir d3/d34/d40/d48 0 2024-04-12T20:56:01.412 INFO:tasks.workunit.client.0.smithi018.stdout:8/300: truncate db/d25/d26/f5c 118081 0 2024-04-12T20:56:01.418 INFO:tasks.workunit.client.0.smithi018.stdout:6/303: write d5/d6/d1b/d3b/f52 [891881,529] 0 2024-04-12T20:56:01.419 INFO:tasks.workunit.client.0.smithi018.stdout:1/148: unlink d3/d9/c27 0 2024-04-12T20:56:01.423 INFO:tasks.workunit.client.0.smithi018.stdout:2/187: dwrite d5/d8/d10/d19/d24/f32 [0,4194304] 0 2024-04-12T20:56:01.431 INFO:tasks.workunit.client.0.smithi018.stdout:6/304: dread d5/d6/d1b/d3b/f69 [0,4194304] 0 2024-04-12T20:56:01.435 INFO:tasks.workunit.client.0.smithi018.stdout:0/221: creat da/d15/d17/d21/d27/f4f x:0 0 0 2024-04-12T20:56:01.436 INFO:tasks.workunit.client.0.smithi018.stdout:8/301: fsync db/d3b/f4d 0 2024-04-12T20:56:01.437 INFO:tasks.workunit.client.0.smithi018.stdout:5/170: creat d0/d2/d9/d18/f3b x:0 0 0 2024-04-12T20:56:01.441 INFO:tasks.workunit.client.0.smithi018.stdout:2/188: symlink d5/l41 0 2024-04-12T20:56:01.446 INFO:tasks.workunit.client.0.smithi018.stdout:1/149: mkdir d3/d9/da/dd/d31/d32/d33 0 2024-04-12T20:56:01.446 INFO:tasks.workunit.client.0.smithi018.stdout:0/222: truncate da/d15/d17/d21/f46 638794 0 2024-04-12T20:56:01.448 INFO:tasks.workunit.client.0.smithi018.stdout:6/305: readlink d5/df/l45 0 2024-04-12T20:56:01.450 INFO:tasks.workunit.client.0.smithi018.stdout:7/274: rename d1/db/d31/d3a/l61 to d1/d6/d8/l62 0 2024-04-12T20:56:01.454 INFO:tasks.workunit.client.0.smithi018.stdout:3/228: dwrite d2/d6/fc [0,4194304] 0 2024-04-12T20:56:01.455 INFO:tasks.workunit.client.0.smithi018.stdout:7/275: readlink d1/db/d31/d30/l50 0 2024-04-12T20:56:01.458 INFO:tasks.workunit.client.0.smithi018.stdout:0/223: mkdir da/d15/d17/d50 0 2024-04-12T20:56:01.460 INFO:tasks.workunit.client.0.smithi018.stdout:6/306: symlink d5/d6/d1b/d3b/d22/l7c 0 2024-04-12T20:56:01.461 INFO:tasks.workunit.client.0.smithi018.stdout:4/254: link d3/d32/d33/l45 d3/d34/d43/l49 0 2024-04-12T20:56:01.463 INFO:tasks.workunit.client.0.smithi018.stdout:5/171: dwrite d0/d2/d9/f13 [0,4194304] 0 2024-04-12T20:56:01.466 INFO:tasks.workunit.client.0.smithi018.stdout:5/172: dread - d0/d2/d33/d10/d26/d30/f31 zero size 2024-04-12T20:56:01.471 INFO:tasks.workunit.client.0.smithi018.stdout:7/276: creat d1/db/d31/f63 x:0 0 0 2024-04-12T20:56:01.473 INFO:tasks.workunit.client.0.smithi018.stdout:7/277: readlink d1/db/d31/l49 0 2024-04-12T20:56:01.478 INFO:tasks.workunit.client.0.smithi018.stdout:7/278: dread - d1/db/d31/f63 zero size 2024-04-12T20:56:01.478 INFO:tasks.workunit.client.0.smithi018.stdout:1/150: creat d3/d9/da/dd/d31/d32/f34 x:0 0 0 2024-04-12T20:56:01.484 INFO:tasks.workunit.client.0.smithi018.stdout:9/332: truncate d3/d6/d9/dc/d1c/f2e 2402458 0 2024-04-12T20:56:01.489 INFO:tasks.workunit.client.0.smithi018.stdout:1/151: fsync d3/d9/f18 0 2024-04-12T20:56:01.491 INFO:tasks.workunit.client.0.smithi018.stdout:7/279: write d1/db/d31/d30/f57 [322872,96932] 0 2024-04-12T20:56:01.497 INFO:tasks.workunit.client.0.smithi018.stdout:3/229: dwrite d2/f19 [4194304,4194304] 0 2024-04-12T20:56:01.498 INFO:tasks.workunit.client.0.smithi018.stdout:0/224: dwrite da/d15/f39 [0,4194304] 0 2024-04-12T20:56:01.501 INFO:tasks.workunit.client.0.smithi018.stdout:9/333: write d3/d6/d1b/f35 [3417785,54857] 0 2024-04-12T20:56:01.504 INFO:tasks.workunit.client.0.smithi018.stdout:1/152: rename d3/f6 to d3/d9/da/dd/d31/f35 0 2024-04-12T20:56:01.508 INFO:tasks.workunit.client.0.smithi018.stdout:5/173: dwrite d0/d2/d33/d10/d26/d30/f31 [0,4194304] 0 2024-04-12T20:56:01.510 INFO:tasks.workunit.client.0.smithi018.stdout:6/307: dwrite d5/df/d3f/f49 [0,4194304] 0 2024-04-12T20:56:01.510 INFO:tasks.workunit.client.0.smithi018.stdout:3/230: mkdir d2/d6/d40 0 2024-04-12T20:56:01.515 INFO:tasks.workunit.client.0.smithi018.stdout:5/174: write d0/d2/f1a [4060672,36171] 0 2024-04-12T20:56:01.517 INFO:tasks.workunit.client.0.smithi018.stdout:6/308: mknod d5/df/d19/d2b/c7d 0 2024-04-12T20:56:01.521 INFO:tasks.workunit.client.0.smithi018.stdout:0/225: truncate da/d15/d17/f32 869116 0 2024-04-12T20:56:01.522 INFO:tasks.workunit.client.0.smithi018.stdout:9/334: truncate d3/d6/d9/dc/f2d 413089 0 2024-04-12T20:56:01.529 INFO:tasks.workunit.client.0.smithi018.stdout:5/175: fdatasync d0/d2/d9/fe 0 2024-04-12T20:56:01.529 INFO:tasks.workunit.client.0.smithi018.stdout:1/153: creat d3/d9/da/dd/d31/f36 x:0 0 0 2024-04-12T20:56:01.529 INFO:tasks.workunit.client.0.smithi018.stdout:3/231: symlink d2/d6/d34/l41 0 2024-04-12T20:56:01.529 INFO:tasks.workunit.client.0.smithi018.stdout:5/176: stat d0/d2/d33 0 2024-04-12T20:56:01.530 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:56:01 smithi186 ceph-mon[36597]: pgmap v33: 65 pgs: 65 active+clean; 325 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 354 KiB/s rd, 22 MiB/s wr, 152 op/s 2024-04-12T20:56:01.530 INFO:tasks.workunit.client.0.smithi018.stdout:7/280: dwrite d1/db/f43 [0,4194304] 0 2024-04-12T20:56:01.537 INFO:tasks.workunit.client.0.smithi018.stdout:9/335: unlink d3/d6/d9/dc/d13/l56 0 2024-04-12T20:56:01.538 INFO:tasks.workunit.client.0.smithi018.stdout:6/309: symlink d5/df/d19/d27/d2f/l7e 0 2024-04-12T20:56:01.542 INFO:tasks.workunit.client.0.smithi018.stdout:7/281: truncate d1/db/d31/d3a/d42/f44 4547893 0 2024-04-12T20:56:01.547 INFO:tasks.workunit.client.0.smithi018.stdout:6/310: creat d5/d71/f7f x:0 0 0 2024-04-12T20:56:01.547 INFO:tasks.workunit.client.0.smithi018.stdout:1/154: getdents d3/d9/da/dd/d31/d32/d13/d19/d2d 0 2024-04-12T20:56:01.556 INFO:tasks.workunit.client.0.smithi018.stdout:0/226: truncate da/f25 4870 0 2024-04-12T20:56:01.557 INFO:tasks.workunit.client.0.smithi018.stdout:5/177: dwrite d0/d2/d11/f2a [0,4194304] 0 2024-04-12T20:56:01.559 INFO:tasks.workunit.client.0.smithi018.stdout:6/311: rename d5/df/d31/d4e to d5/df/d31/d4e/d80 22 2024-04-12T20:56:01.559 INFO:tasks.workunit.client.0.smithi018.stdout:1/155: creat d3/d9/d11/f37 x:0 0 0 2024-04-12T20:56:01.561 INFO:tasks.workunit.client.0.smithi018.stdout:0/227: readlink l7 0 2024-04-12T20:56:01.571 INFO:tasks.workunit.client.0.smithi018.stdout:3/232: dwrite d2/d6/d30/f3e [0,4194304] 0 2024-04-12T20:56:01.571 INFO:tasks.workunit.client.0.smithi018.stdout:1/156: creat d3/d9/da/d14/f38 x:0 0 0 2024-04-12T20:56:01.574 INFO:tasks.workunit.client.0.smithi018.stdout:5/178: read d0/d2/d9/d1e/d2e/f12 [802038,15864] 0 2024-04-12T20:56:01.576 INFO:tasks.workunit.client.0.smithi018.stdout:1/157: write d3/d9/f18 [3785074,117663] 0 2024-04-12T20:56:01.577 INFO:tasks.workunit.client.0.smithi018.stdout:7/282: dwrite d1/d6/f52 [0,4194304] 0 2024-04-12T20:56:01.579 INFO:tasks.workunit.client.0.smithi018.stdout:5/179: chown d0/d2 0 1 2024-04-12T20:56:01.585 INFO:tasks.workunit.client.0.smithi018.stdout:5/180: fdatasync d0/d2/d9/fe 0 2024-04-12T20:56:01.586 INFO:tasks.workunit.client.0.smithi018.stdout:9/336: dwrite d3/d6/d9/dc/d13/f42 [0,4194304] 0 2024-04-12T20:56:01.591 INFO:tasks.workunit.client.0.smithi018.stdout:7/283: unlink d1/db/f22 0 2024-04-12T20:56:01.598 INFO:tasks.workunit.client.0.smithi018.stdout:3/233: truncate d2/f14 2124371 0 2024-04-12T20:56:01.601 INFO:tasks.workunit.client.0.smithi018.stdout:0/228: dwrite da/fc [0,4194304] 0 2024-04-12T20:56:01.604 INFO:tasks.workunit.client.0.smithi018.stdout:5/181: write d0/d2/d9/f1d [854909,125163] 0 2024-04-12T20:56:01.605 INFO:tasks.workunit.client.0.smithi018.stdout:7/284: mknod d1/db/d31/d45/d53/c64 0 2024-04-12T20:56:01.616 INFO:tasks.workunit.client.0.smithi018.stdout:1/158: getdents d3/d9/da/d14 0 2024-04-12T20:56:01.631 INFO:tasks.workunit.client.0.smithi018.stdout:3/234: truncate d2/d6/d30/d31/d38/f3b 2642456 0 2024-04-12T20:56:01.633 INFO:tasks.workunit.client.0.smithi018.stdout:6/312: rmdir d5/d6/d1b/d3b/d22 39 2024-04-12T20:56:01.643 INFO:tasks.workunit.client.0.smithi018.stdout:5/182: dwrite d0/d2/d11/f2a [0,4194304] 0 2024-04-12T20:56:01.646 INFO:tasks.workunit.client.0.smithi018.stdout:0/229: dwrite da/d15/f33 [0,4194304] 0 2024-04-12T20:56:01.651 INFO:tasks.workunit.client.0.smithi018.stdout:7/285: dwrite d1/db/d31/d3a/f41 [0,4194304] 0 2024-04-12T20:56:01.652 INFO:tasks.workunit.client.0.smithi018.stdout:3/235: rmdir d2/d6/d40 0 2024-04-12T20:56:01.654 INFO:tasks.workunit.client.0.smithi018.stdout:1/159: dwrite d3/d9/da/d14/f38 [0,4194304] 0 2024-04-12T20:56:01.655 INFO:tasks.workunit.client.0.smithi018.stdout:5/183: fsync d0/d2/d33/d10/f17 0 2024-04-12T20:56:01.659 INFO:tasks.workunit.client.0.smithi018.stdout:0/230: unlink da/d15/d17/l28 0 2024-04-12T20:56:01.664 INFO:tasks.workunit.client.0.smithi018.stdout:0/231: write da/d15/d17/f3b [551550,81610] 0 2024-04-12T20:56:01.665 INFO:tasks.workunit.client.0.smithi018.stdout:7/286: mknod d1/d6/d8/c65 0 2024-04-12T20:56:01.668 INFO:tasks.workunit.client.0.smithi018.stdout:3/236: creat d2/f42 x:0 0 0 2024-04-12T20:56:01.668 INFO:tasks.workunit.client.0.smithi018.stdout:5/184: mknod d0/d2/d9/d1e/c3c 0 2024-04-12T20:56:01.670 INFO:tasks.workunit.client.0.smithi018.stdout:6/313: dwrite d5/df/d19/d74/f75 [0,4194304] 0 2024-04-12T20:56:01.681 INFO:tasks.workunit.client.0.smithi018.stdout:5/185: write d0/d2/d33/d10/f27 [4152451,130049] 0 2024-04-12T20:56:01.682 INFO:tasks.workunit.client.0.smithi018.stdout:6/314: write d5/d6/d1b/f61 [686206,14662] 0 2024-04-12T20:56:01.685 INFO:tasks.workunit.client.0.smithi018.stdout:7/287: write d1/d6/f24 [151664,70266] 0 2024-04-12T20:56:01.688 INFO:tasks.workunit.client.0.smithi018.stdout:0/232: symlink da/d15/d3e/l51 0 2024-04-12T20:56:01.694 INFO:tasks.workunit.client.0.smithi018.stdout:1/160: dwrite d3/d9/da/d14/f38 [0,4194304] 0 2024-04-12T20:56:01.696 INFO:tasks.workunit.client.0.smithi018.stdout:9/337: write d3/d6/d9/dc/d13/f42 [5124574,60934] 0 2024-04-12T20:56:01.698 INFO:tasks.workunit.client.0.smithi018.stdout:0/233: creat da/f52 x:0 0 0 2024-04-12T20:56:01.705 INFO:tasks.workunit.client.0.smithi018.stdout:0/234: symlink da/d15/l53 0 2024-04-12T20:56:01.707 INFO:tasks.workunit.client.0.smithi018.stdout:3/237: dread d2/d6/f12 [0,4194304] 0 2024-04-12T20:56:01.712 INFO:tasks.workunit.client.0.smithi018.stdout:0/235: read da/d15/d17/f19 [2452876,55175] 0 2024-04-12T20:56:01.717 INFO:tasks.workunit.client.0.smithi018.stdout:9/338: symlink d3/d6/d9/dc/d46/l6c 0 2024-04-12T20:56:01.723 INFO:tasks.workunit.client.0.smithi018.stdout:0/236: creat da/d15/d3e/f54 x:0 0 0 2024-04-12T20:56:01.726 INFO:tasks.workunit.client.0.smithi018.stdout:5/186: dwrite d0/d2/d9/f3a [0,4194304] 0 2024-04-12T20:56:01.731 INFO:tasks.workunit.client.0.smithi018.stdout:0/237: mkdir da/d15/d17/d21/d36/d47/d55 0 2024-04-12T20:56:01.733 INFO:tasks.workunit.client.0.smithi018.stdout:5/187: creat d0/d2b/f3d x:0 0 0 2024-04-12T20:56:01.737 INFO:tasks.workunit.client.0.smithi018.stdout:6/315: rmdir d5/d15/d6b 0 2024-04-12T20:56:01.739 INFO:tasks.workunit.client.0.smithi018.stdout:9/339: rename d3/d6/d9/l6a to d3/d6/d9/dc/d5a/l6d 0 2024-04-12T20:56:01.740 INFO:tasks.workunit.client.0.smithi018.stdout:6/316: chown d5/df/d19/f54 8982 1 2024-04-12T20:56:01.741 INFO:tasks.workunit.client.0.smithi018.stdout:1/161: dread d3/d9/da/dd/d31/d32/f1b [0,4194304] 0 2024-04-12T20:56:01.741 INFO:tasks.workunit.client.0.smithi018.stdout:5/188: rename d0 to d0/d2/d33/d10/d2f/d3e 22 2024-04-12T20:56:01.741 INFO:tasks.workunit.client.0.smithi018.stdout:3/238: dread d2/d6/d30/f3e [0,4194304] 0 2024-04-12T20:56:01.747 INFO:tasks.workunit.client.0.smithi018.stdout:1/162: mknod d3/d9/da/dd/c39 0 2024-04-12T20:56:01.747 INFO:tasks.workunit.client.0.smithi018.stdout:5/189: symlink d0/d2/d33/d10/d26/l3f 0 2024-04-12T20:56:01.751 INFO:tasks.workunit.client.0.smithi018.stdout:5/190: write d0/d2/d33/d10/d26/d30/f31 [1011235,8029] 0 2024-04-12T20:56:01.752 INFO:tasks.workunit.client.0.smithi018.stdout:3/239: rmdir d2/d6/d30/d31 39 2024-04-12T20:56:01.757 INFO:tasks.workunit.client.0.smithi018.stdout:3/240: mkdir d2/d43 0 2024-04-12T20:56:01.758 INFO:tasks.workunit.client.0.smithi018.stdout:5/191: mkdir d0/d2b/d40 0 2024-04-12T20:56:01.762 INFO:tasks.workunit.client.0.smithi018.stdout:5/192: chown d0/d2/d9/d18/f3b 32010 1 2024-04-12T20:56:01.763 INFO:tasks.workunit.client.0.smithi018.stdout:1/163: rmdir d3/d9/da/dd 39 2024-04-12T20:56:01.764 INFO:tasks.workunit.client.0.smithi018.stdout:5/193: fdatasync d0/d2/d9/f1d 0 2024-04-12T20:56:01.770 INFO:tasks.workunit.client.0.smithi018.stdout:3/241: creat d2/d6/d30/d31/d38/f44 x:0 0 0 2024-04-12T20:56:01.775 INFO:tasks.workunit.client.0.smithi018.stdout:6/317: dwrite d5/f48 [0,4194304] 0 2024-04-12T20:56:01.778 INFO:tasks.workunit.client.0.smithi018.stdout:0/238: dread da/f13 [0,4194304] 0 2024-04-12T20:56:01.778 INFO:tasks.workunit.client.0.smithi018.stdout:6/318: chown d5/df/d31/d4e 0 1 2024-04-12T20:56:01.779 INFO:tasks.workunit.client.0.smithi018.stdout:1/164: symlink d3/d9/da/dd/d31/d32/d13/l3a 0 2024-04-12T20:56:01.786 INFO:tasks.workunit.client.0.smithi018.stdout:1/165: mknod d3/d9/da/dd/c3b 0 2024-04-12T20:56:01.787 INFO:tasks.workunit.client.0.smithi018.stdout:3/242: rmdir d2/d6/d30/d31/d38 39 2024-04-12T20:56:01.787 INFO:tasks.workunit.client.0.smithi018.stdout:9/340: dread d3/d6/d9/d1e/d3a/f31 [0,4194304] 0 2024-04-12T20:56:01.793 INFO:tasks.workunit.client.0.smithi018.stdout:1/166: truncate d3/d9/da/dd/d31/d32/f34 1018886 0 2024-04-12T20:56:01.795 INFO:tasks.workunit.client.0.smithi018.stdout:1/167: write d3/f10 [4351536,75544] 0 2024-04-12T20:56:01.798 INFO:tasks.workunit.client.0.smithi018.stdout:3/243: creat d2/d6/d30/d31/d38/d3a/f45 x:0 0 0 2024-04-12T20:56:01.804 INFO:tasks.workunit.client.0.smithi018.stdout:5/194: dwrite d0/d2/d9/d1e/d2e/f12 [0,4194304] 0 2024-04-12T20:56:01.804 INFO:tasks.workunit.client.0.smithi018.stdout:9/341: write d3/d6/d9/d1e/f20 [497336,42387] 0 2024-04-12T20:56:01.810 INFO:tasks.workunit.client.0.smithi018.stdout:3/244: link d2/d6/f25 d2/d6/d34/f46 0 2024-04-12T20:56:01.810 INFO:tasks.workunit.client.0.smithi018.stdout:5/195: symlink d0/l41 0 2024-04-12T20:56:01.815 INFO:tasks.workunit.client.0.smithi018.stdout:5/196: truncate d0/d2b/f3d 787705 0 2024-04-12T20:56:01.821 INFO:tasks.workunit.client.0.smithi018.stdout:0/239: dwrite da/f25 [0,4194304] 0 2024-04-12T20:56:01.824 INFO:tasks.workunit.client.0.smithi018.stdout:5/197: truncate d0/d2b/f2d 2592440 0 2024-04-12T20:56:01.828 INFO:tasks.workunit.client.0.smithi018.stdout:0/240: write da/d15/d17/d21/d27/f49 [506039,100013] 0 2024-04-12T20:56:01.833 INFO:tasks.workunit.client.0.smithi018.stdout:1/168: dwrite d3/f7 [4194304,4194304] 0 2024-04-12T20:56:01.836 INFO:tasks.workunit.client.0.smithi018.stdout:6/319: dread d5/d6/d1b/d3b/d35/d4f/f5c [0,4194304] 0 2024-04-12T20:56:01.837 INFO:tasks.workunit.client.0.smithi018.stdout:0/241: symlink da/d15/d17/l56 0 2024-04-12T20:56:01.847 INFO:tasks.workunit.client.0.smithi018.stdout:9/342: dwrite d3/d6/d9/dc/d1c/f4e [0,4194304] 0 2024-04-12T20:56:01.849 INFO:tasks.workunit.client.0.smithi018.stdout:0/242: mkdir da/d15/d3e/d3f/d57 0 2024-04-12T20:56:01.849 INFO:tasks.workunit.client.0.smithi018.stdout:6/320: rmdir d5/df/d19/d43/d5f 39 2024-04-12T20:56:01.850 INFO:tasks.workunit.client.0.smithi018.stdout:5/198: write d0/d2b/f2d [3625637,24598] 0 2024-04-12T20:56:01.852 INFO:tasks.workunit.client.0.smithi018.stdout:1/169: chown d3/d9/da/dd/d31/d32/d13/c2a 635577 1 2024-04-12T20:56:01.854 INFO:tasks.workunit.client.0.smithi018.stdout:3/245: dwrite d2/d6/f12 [0,4194304] 0 2024-04-12T20:56:01.856 INFO:tasks.workunit.client.0.smithi018.stdout:5/199: readlink d0/d2/d11/l14 0 2024-04-12T20:56:01.856 INFO:tasks.workunit.client.0.smithi018.stdout:0/243: read - da/d15/d3e/f54 zero size 2024-04-12T20:56:01.860 INFO:tasks.workunit.client.0.smithi018.stdout:1/170: readlink d3/d9/da/dd/d31/d32/l2e 0 2024-04-12T20:56:01.863 INFO:tasks.workunit.client.0.smithi018.stdout:0/244: chown da/d15/c16 4863 1 2024-04-12T20:56:01.866 INFO:tasks.workunit.client.0.smithi018.stdout:1/171: truncate d3/d9/da/f20 1695331 0 2024-04-12T20:56:01.869 INFO:tasks.workunit.client.0.smithi018.stdout:6/321: creat d5/d6/d1b/d3b/d22/f81 x:0 0 0 2024-04-12T20:56:01.872 INFO:tasks.workunit.client.0.smithi018.stdout:1/172: write d3/d9/da/dd/f17 [2470421,53695] 0 2024-04-12T20:56:01.874 INFO:tasks.workunit.client.0.smithi018.stdout:6/322: truncate d5/d6/d1b/f61 5312717 0 2024-04-12T20:56:01.878 INFO:tasks.workunit.client.0.smithi018.stdout:1/173: write d3/d9/da/d14/f38 [3212085,78270] 0 2024-04-12T20:56:01.880 INFO:tasks.workunit.client.0.smithi018.stdout:6/323: mkdir d5/df/d3f/d82 0 2024-04-12T20:56:01.885 INFO:tasks.workunit.client.0.smithi018.stdout:6/324: write d5/df/f18 [1223355,113482] 0 2024-04-12T20:56:01.886 INFO:tasks.workunit.client.0.smithi018.stdout:1/174: fdatasync d3/f4 0 2024-04-12T20:56:01.888 INFO:tasks.workunit.client.0.smithi018.stdout:9/343: read d3/f23 [7497431,27287] 0 2024-04-12T20:56:01.897 INFO:tasks.workunit.client.0.smithi018.stdout:1/175: creat d3/d9/f3c x:0 0 0 2024-04-12T20:56:01.898 INFO:tasks.workunit.client.0.smithi018.stdout:0/245: dwrite da/f25 [0,4194304] 0 2024-04-12T20:56:01.901 INFO:tasks.workunit.client.0.smithi018.stdout:5/200: dwrite d0/d2/d33/f15 [4194304,4194304] 0 2024-04-12T20:56:01.902 INFO:tasks.workunit.client.0.smithi018.stdout:0/246: dread - da/f52 zero size 2024-04-12T20:56:01.906 INFO:tasks.workunit.client.0.smithi018.stdout:5/201: chown d0/d2/d33 4342328 1 2024-04-12T20:56:01.906 INFO:tasks.workunit.client.0.smithi018.stdout:0/247: chown da/d15/d17/d21/f46 169 1 2024-04-12T20:56:01.908 INFO:tasks.workunit.client.0.smithi018.stdout:1/176: readlink d3/d9/da/dd/d31/d32/d13/d19/l1e 0 2024-04-12T20:56:01.911 INFO:tasks.workunit.client.0.smithi018.stdout:5/202: read d0/d2/d9/f13 [2533619,82672] 0 2024-04-12T20:56:01.915 INFO:tasks.workunit.client.0.smithi018.stdout:6/325: rmdir d5/df/d3f/d82 0 2024-04-12T20:56:01.918 INFO:tasks.workunit.client.0.smithi018.stdout:9/344: truncate d3/f23 2485346 0 2024-04-12T20:56:01.919 INFO:tasks.workunit.client.0.smithi018.stdout:1/177: link d3/d9/da/l1c d3/d9/d11/l3d 0 2024-04-12T20:56:01.924 INFO:tasks.workunit.client.0.smithi018.stdout:9/345: write d3/d6/d9/fd [2233621,13923] 0 2024-04-12T20:56:01.925 INFO:tasks.workunit.client.0.smithi018.stdout:3/246: dread d2/d6/f1c [0,4194304] 0 2024-04-12T20:56:01.925 INFO:tasks.workunit.client.0.smithi018.stdout:5/203: creat d0/d2/d9/d1e/f42 x:0 0 0 2024-04-12T20:56:01.929 INFO:tasks.workunit.client.0.smithi018.stdout:5/204: write d0/d2/d33/f15 [7271137,119402] 0 2024-04-12T20:56:01.932 INFO:tasks.workunit.client.0.smithi018.stdout:5/205: chown d0/d2/d33 33 1 2024-04-12T20:56:01.933 INFO:tasks.workunit.client.0.smithi018.stdout:9/346: unlink d3/d6/d9/dc/d1c/d21/l49 0 2024-04-12T20:56:01.934 INFO:tasks.workunit.client.0.smithi018.stdout:3/247: symlink d2/d6/d30/d31/l47 0 2024-04-12T20:56:01.936 INFO:tasks.workunit.client.0.smithi018.stdout:6/326: creat d5/f83 x:0 0 0 2024-04-12T20:56:01.937 INFO:tasks.workunit.client.0.smithi018.stdout:1/178: truncate d3/d9/da/dd/d31/f35 1167021 0 2024-04-12T20:56:01.938 INFO:tasks.workunit.client.0.smithi018.stdout:5/206: fsync d0/d2/d9/fe 0 2024-04-12T20:56:01.940 INFO:tasks.workunit.client.0.smithi018.stdout:3/248: write d2/d6/f2e [155323,78108] 0 2024-04-12T20:56:01.943 INFO:tasks.workunit.client.0.smithi018.stdout:0/248: dwrite da/d15/f20 [0,4194304] 0 2024-04-12T20:56:01.949 INFO:tasks.workunit.client.0.smithi018.stdout:3/249: rmdir d2 39 2024-04-12T20:56:01.954 INFO:tasks.workunit.client.0.smithi018.stdout:5/207: creat d0/d2/d33/d10/d26/f43 x:0 0 0 2024-04-12T20:56:01.955 INFO:tasks.workunit.client.0.smithi018.stdout:1/179: getdents d3/d9/da/d14 0 2024-04-12T20:56:01.960 INFO:tasks.workunit.client.0.smithi018.stdout:5/208: chown d0/d2/d33/d10/d26/l3f 1630 1 2024-04-12T20:56:01.962 INFO:tasks.workunit.client.0.smithi018.stdout:3/250: readlink d2/d6/d30/d31/l33 0 2024-04-12T20:56:01.964 INFO:tasks.workunit.client.0.smithi018.stdout:3/251: chown d2/f5 57963 1 2024-04-12T20:56:01.966 INFO:tasks.workunit.client.0.smithi018.stdout:0/249: dwrite da/d15/d17/f3b [0,4194304] 0 2024-04-12T20:56:01.973 INFO:tasks.workunit.client.0.smithi018.stdout:5/209: symlink d0/d2/d33/d10/d26/d30/l44 0 2024-04-12T20:56:01.979 INFO:tasks.workunit.client.0.smithi018.stdout:0/250: mkdir da/d15/d4d/d58 0 2024-04-12T20:56:01.981 INFO:tasks.workunit.client.0.smithi018.stdout:5/210: symlink d0/l45 0 2024-04-12T20:56:01.982 INFO:tasks.workunit.client.0.smithi018.stdout:0/251: fsync da/d15/f39 0 2024-04-12T20:56:01.987 INFO:tasks.workunit.client.0.smithi018.stdout:9/347: truncate d3/d6/d9/fa 275250 0 2024-04-12T20:56:01.988 INFO:tasks.workunit.client.0.smithi018.stdout:1/180: dwrite d3/d9/d11/f37 [0,4194304] 0 2024-04-12T20:56:01.989 INFO:tasks.workunit.client.0.smithi018.stdout:3/252: dread d2/f28 [0,4194304] 0 2024-04-12T20:56:01.997 INFO:tasks.workunit.client.0.smithi018.stdout:6/327: dread d5/d15/d60/f65 [0,4194304] 0 2024-04-12T20:56:02.002 INFO:tasks.workunit.client.0.smithi018.stdout:0/252: unlink da/ld 0 2024-04-12T20:56:02.002 INFO:tasks.workunit.client.0.smithi018.stdout:9/348: creat d3/d6/d9/d3f/f6e x:0 0 0 2024-04-12T20:56:02.002 INFO:tasks.workunit.client.0.smithi018.stdout:3/253: mkdir d2/d6/d30/d31/d38/d3a/d48 0 2024-04-12T20:56:02.003 INFO:tasks.workunit.client.0.smithi018.stdout:0/253: chown da/d15/d17/d21/d27/c31 1168 1 2024-04-12T20:56:02.008 INFO:tasks.workunit.client.0.smithi018.stdout:9/349: mkdir d3/d6/d9/dc/d6f 0 2024-04-12T20:56:02.013 INFO:tasks.workunit.client.0.smithi018.stdout:3/254: rmdir d2/d6/d30 39 2024-04-12T20:56:02.014 INFO:tasks.workunit.client.0.smithi018.stdout:0/254: creat da/d15/d3e/d3f/f59 x:0 0 0 2024-04-12T20:56:02.014 INFO:tasks.workunit.client.0.smithi018.stdout:9/350: write d3/d6/d9/dc/d13/f42 [1597201,67914] 0 2024-04-12T20:56:02.014 INFO:tasks.workunit.client.0.smithi018.stdout:5/211: truncate d0/d2/d9/f3a 2362876 0 2024-04-12T20:56:02.020 INFO:tasks.workunit.client.0.smithi018.stdout:1/181: dwrite d3/d9/da/dd/d31/d32/f34 [0,4194304] 0 2024-04-12T20:56:02.020 INFO:tasks.workunit.client.0.smithi018.stdout:9/351: chown d3/d6/d1b/f35 599 1 2024-04-12T20:56:02.026 INFO:tasks.workunit.client.0.smithi018.stdout:6/328: dwrite d5/df/d19/f68 [0,4194304] 0 2024-04-12T20:56:02.029 INFO:tasks.workunit.client.0.smithi018.stdout:0/255: symlink da/d15/d17/d21/d27/l5a 0 2024-04-12T20:56:02.030 INFO:tasks.workunit.client.0.smithi018.stdout:5/212: rename d0/d2/d9 to d0/d2/d33/d46 0 2024-04-12T20:56:02.038 INFO:tasks.workunit.client.0.smithi018.stdout:0/256: rmdir da/d15/d3e/d3f/d57 0 2024-04-12T20:56:02.042 INFO:tasks.workunit.client.0.smithi018.stdout:6/329: creat d5/df/d19/d43/d5f/f84 x:0 0 0 2024-04-12T20:56:02.042 INFO:tasks.workunit.client.0.smithi018.stdout:9/352: truncate d3/d6/d9/dc/d1c/f2e 24867 0 2024-04-12T20:56:02.046 INFO:tasks.workunit.client.0.smithi018.stdout:3/255: dwrite d2/d6/d34/f46 [0,4194304] 0 2024-04-12T20:56:02.058 INFO:tasks.workunit.client.0.smithi018.stdout:9/353: creat d3/d6/d9/dc/d1c/f70 x:0 0 0 2024-04-12T20:56:02.066 INFO:tasks.workunit.client.0.smithi018.stdout:0/257: dwrite da/d15/f20 [4194304,4194304] 0 2024-04-12T20:56:02.069 INFO:tasks.workunit.client.0.smithi018.stdout:3/256: dread d2/d6/f24 [0,4194304] 0 2024-04-12T20:56:02.071 INFO:tasks.workunit.client.0.smithi018.stdout:9/354: dread d3/f5e [0,4194304] 0 2024-04-12T20:56:02.072 INFO:tasks.workunit.client.0.smithi018.stdout:5/213: dread d0/d2/d33/d10/f27 [0,4194304] 0 2024-04-12T20:56:02.073 INFO:tasks.workunit.client.0.smithi018.stdout:6/330: dwrite d5/d6/d1b/d3b/f52 [0,4194304] 0 2024-04-12T20:56:02.076 INFO:tasks.workunit.client.0.smithi018.stdout:3/257: readlink d2/d6/l36 0 2024-04-12T20:56:02.079 INFO:tasks.workunit.client.0.smithi018.stdout:3/258: chown d2/d6/f25 720390 1 2024-04-12T20:56:02.084 INFO:tasks.workunit.client.0.smithi018.stdout:6/331: write d5/f57 [332846,94512] 0 2024-04-12T20:56:02.084 INFO:tasks.workunit.client.0.smithi018.stdout:0/258: rename da/d15/d17/f3b to da/d15/d3e/d3f/f5b 0 2024-04-12T20:56:02.084 INFO:tasks.workunit.client.0.smithi018.stdout:5/214: creat d0/d2/d11/d37/f47 x:0 0 0 2024-04-12T20:56:02.087 INFO:tasks.workunit.client.0.smithi018.stdout:3/259: creat d2/d6/d34/f49 x:0 0 0 2024-04-12T20:56:02.091 INFO:tasks.workunit.client.0.smithi018.stdout:3/260: read d2/d6/f12 [1085455,51998] 0 2024-04-12T20:56:02.092 INFO:tasks.workunit.client.0.smithi018.stdout:9/355: truncate d3/d6/d9/dc/d13/f17 7482717 0 2024-04-12T20:56:02.094 INFO:tasks.workunit.client.0.smithi018.stdout:6/332: mknod d5/d15/d60/c85 0 2024-04-12T20:56:02.097 INFO:tasks.workunit.client.0.smithi018.stdout:3/261: read - d2/d6/d30/d31/d38/d3a/f45 zero size 2024-04-12T20:56:02.098 INFO:tasks.workunit.client.0.smithi018.stdout:6/333: creat d5/df/d19/d74/f86 x:0 0 0 2024-04-12T20:56:02.100 INFO:tasks.workunit.client.0.smithi018.stdout:9/356: rename d3/d6/d9/dc/d1c/f4e to d3/d6/d9/d1e/f71 0 2024-04-12T20:56:02.102 INFO:tasks.workunit.client.0.smithi018.stdout:3/262: mknod d2/d6/c4a 0 2024-04-12T20:56:02.106 INFO:tasks.workunit.client.0.smithi018.stdout:3/263: fdatasync d2/d6/d34/f46 0 2024-04-12T20:56:02.108 INFO:tasks.workunit.client.0.smithi018.stdout:5/215: dwrite d0/d2b/f2d [0,4194304] 0 2024-04-12T20:56:02.118 INFO:tasks.workunit.client.0.smithi018.stdout:5/216: creat d0/d2/d33/d46/d1e/f48 x:0 0 0 2024-04-12T20:56:02.122 INFO:tasks.workunit.client.0.smithi018.stdout:9/357: getdents d3/d6/d9/dc/d1c/d21 0 2024-04-12T20:56:02.124 INFO:tasks.workunit.client.0.smithi018.stdout:5/217: readlink d0/d2/d33/l36 0 2024-04-12T20:56:02.129 INFO:tasks.workunit.client.0.smithi018.stdout:3/264: dwrite d2/f5 [0,4194304] 0 2024-04-12T20:56:02.133 INFO:tasks.workunit.client.0.smithi018.stdout:5/218: rename d0/c4 to d0/d2/d33/d10/d26/d30/c49 0 2024-04-12T20:56:02.135 INFO:tasks.workunit.client.0.smithi018.stdout:3/265: mknod d2/d6/d30/c4b 0 2024-04-12T20:56:02.135 INFO:tasks.workunit.client.0.smithi018.stdout:9/358: unlink d3/d6/d9/dc/d1c/f60 0 2024-04-12T20:56:02.136 INFO:tasks.workunit.client.0.smithi018.stdout:3/266: readlink d2/l23 0 2024-04-12T20:56:02.139 INFO:tasks.workunit.client.0.smithi018.stdout:9/359: chown d3/d6/d9/dc/d1c/l33 6 1 2024-04-12T20:56:02.142 INFO:tasks.workunit.client.0.smithi018.stdout:5/219: link d0/d2/d33/d46/fe d0/d2/d33/d10/d26/f4a 0 2024-04-12T20:56:02.145 INFO:tasks.workunit.client.0.smithi018.stdout:9/360: readlink d3/d6/l67 0 2024-04-12T20:56:02.146 INFO:tasks.workunit.client.0.smithi018.stdout:5/220: creat d0/d2/d11/f4b x:0 0 0 2024-04-12T20:56:02.158 INFO:tasks.workunit.client.0.smithi018.stdout:3/267: dwrite d2/d6/f24 [0,4194304] 0 2024-04-12T20:56:02.167 INFO:tasks.workunit.client.0.smithi018.stdout:9/361: truncate d3/d6/d9/dc/d13/f42 2015391 0 2024-04-12T20:56:02.173 INFO:tasks.workunit.client.0.smithi018.stdout:9/362: rmdir d3/d6/d9/dc/d1c/d21 39 2024-04-12T20:56:02.175 INFO:tasks.workunit.client.0.smithi018.stdout:5/221: dread d0/d2/f1a [4194304,4194304] 0 2024-04-12T20:56:02.177 INFO:tasks.workunit.client.0.smithi018.stdout:9/363: stat d3/d6/d9/dc/d1c/d21 0 2024-04-12T20:56:02.177 INFO:tasks.workunit.client.0.smithi018.stdout:5/222: dread - d0/d2/d11/f4b zero size 2024-04-12T20:56:02.178 INFO:tasks.workunit.client.0.smithi018.stdout:3/268: dwrite d2/d6/fc [0,4194304] 0 2024-04-12T20:56:02.179 INFO:tasks.workunit.client.0.smithi018.stdout:5/223: stat d0/d2/d33/d46/d1e/d2e 0 2024-04-12T20:56:02.184 INFO:tasks.workunit.client.0.smithi018.stdout:3/269: mkdir d2/d6/d30/d31/d38/d3a/d4c 0 2024-04-12T20:56:02.192 INFO:tasks.workunit.client.0.smithi018.stdout:5/224: dread d0/d2/d33/d46/f13 [0,4194304] 0 2024-04-12T20:56:02.193 INFO:tasks.workunit.client.0.smithi018.stdout:9/364: write d3/d6/d1b/f26 [622179,21354] 0 2024-04-12T20:56:02.203 INFO:tasks.workunit.client.0.smithi018.stdout:3/270: dwrite d2/d6/fc [0,4194304] 0 2024-04-12T20:56:02.214 INFO:tasks.workunit.client.0.smithi018.stdout:5/225: dwrite d0/d2/d33/d46/d1e/f48 [0,4194304] 0 2024-04-12T20:56:02.214 INFO:tasks.workunit.client.0.smithi018.stdout:3/271: getdents d2/d6/d30/d31/d38/d3a/d4c 0 2024-04-12T20:56:02.218 INFO:tasks.workunit.client.0.smithi018.stdout:5/226: dread - d0/d2/d11/f4b zero size 2024-04-12T20:56:02.220 INFO:tasks.workunit.client.0.smithi018.stdout:5/227: getdents d0 0 2024-04-12T20:56:02.227 INFO:tasks.workunit.client.0.smithi018.stdout:5/228: creat d0/d2/d11/d37/f4c x:0 0 0 2024-04-12T20:56:02.229 INFO:tasks.workunit.client.0.smithi018.stdout:5/229: fdatasync d0/d2/d33/d10/f17 0 2024-04-12T20:56:02.233 INFO:tasks.workunit.client.0.smithi018.stdout:5/230: mkdir d0/d2/d33/d46/d35/d4d 0 2024-04-12T20:56:02.234 INFO:tasks.workunit.client.0.smithi018.stdout:5/231: fsync d0/d2/d11/f2a 0 2024-04-12T20:56:02.236 INFO:tasks.workunit.client.0.smithi018.stdout:5/232: chown d0/d2/d11/f4b 861925 1 2024-04-12T20:56:02.240 INFO:tasks.workunit.client.0.smithi018.stdout:9/365: dread d3/d6/d1b/f35 [0,4194304] 0 2024-04-12T20:56:02.243 INFO:tasks.workunit.client.0.smithi018.stdout:9/366: symlink d3/d6/d43/l72 0 2024-04-12T20:56:02.250 INFO:tasks.workunit.client.0.smithi018.stdout:5/233: dwrite d0/d2/d33/d10/d26/d30/f31 [0,4194304] 0 2024-04-12T20:56:02.254 INFO:tasks.workunit.client.0.smithi018.stdout:5/234: write d0/d2/d11/d37/f47 [662678,85893] 0 2024-04-12T20:56:02.258 INFO:tasks.workunit.client.0.smithi018.stdout:9/367: write d3/d6/d9/dc/d1c/f2e [375694,40840] 0 2024-04-12T20:56:02.258 INFO:tasks.workunit.client.0.smithi018.stdout:5/235: mkdir d0/d2/d33/d10/d4e 0 2024-04-12T20:56:02.261 INFO:tasks.workunit.client.0.smithi018.stdout:9/368: truncate d3/d6/d9/d1e/f20 1360449 0 2024-04-12T20:56:02.263 INFO:tasks.workunit.client.0.smithi018.stdout:9/369: read d3/d6/d9/d1e/d3a/f31 [1541527,68088] 0 2024-04-12T20:56:02.265 INFO:tasks.workunit.client.0.smithi018.stdout:8/302: sync 2024-04-12T20:56:02.268 INFO:tasks.workunit.client.0.smithi018.stdout:8/303: creat db/dc/f5e x:0 0 0 2024-04-12T20:56:02.270 INFO:tasks.workunit.client.0.smithi018.stdout:8/304: stat db/dc/c20 0 2024-04-12T20:56:02.272 INFO:tasks.workunit.client.0.smithi018.stdout:8/305: write db/d3b/f3c [568057,19220] 0 2024-04-12T20:56:02.279 INFO:tasks.workunit.client.0.smithi018.stdout:5/236: dwrite d0/d2b/f3d [0,4194304] 0 2024-04-12T20:56:02.284 INFO:tasks.workunit.client.0.smithi018.stdout:9/370: dwrite d3/d6/d9/d3f/f44 [0,4194304] 0 2024-04-12T20:56:02.285 INFO:tasks.workunit.client.0.smithi018.stdout:8/306: link db/d25/d26/f3a db/d25/d26/d27/f5f 0 2024-04-12T20:56:02.286 INFO:tasks.workunit.client.0.smithi018.stdout:8/307: chown f6 147873996 1 2024-04-12T20:56:02.289 INFO:tasks.workunit.client.0.smithi018.stdout:5/237: creat d0/d2/d33/d46/d35/f4f x:0 0 0 2024-04-12T20:56:02.295 INFO:tasks.workunit.client.0.smithi018.stdout:9/371: creat d3/d6/d9/dc/d1c/f73 x:0 0 0 2024-04-12T20:56:02.298 INFO:tasks.workunit.client.0.smithi018.stdout:9/372: read d3/d6/d9/d1e/f20 [1322988,85880] 0 2024-04-12T20:56:02.307 INFO:tasks.workunit.client.0.smithi018.stdout:8/308: dwrite db/d25/f52 [0,4194304] 0 2024-04-12T20:56:02.314 INFO:tasks.workunit.client.0.smithi018.stdout:8/309: rename db/dc/f14 to db/d25/d26/d57/f60 0 2024-04-12T20:56:02.323 INFO:tasks.workunit.client.0.smithi018.stdout:5/238: truncate d0/d2/d33/d46/f13 4162932 0 2024-04-12T20:56:02.327 INFO:tasks.workunit.client.0.smithi018.stdout:9/373: write d3/d6/f10 [1874302,55582] 0 2024-04-12T20:56:02.329 INFO:tasks.workunit.client.0.smithi018.stdout:5/239: write d0/d2/d33/d46/d18/f3b [789453,26082] 0 2024-04-12T20:56:02.331 INFO:tasks.workunit.client.0.smithi018.stdout:5/240: chown d0/d2b 309 1 2024-04-12T20:56:02.335 INFO:tasks.workunit.client.0.smithi018.stdout:8/310: dwrite db/d25/d26/d27/f33 [0,4194304] 0 2024-04-12T20:56:02.338 INFO:tasks.workunit.client.0.smithi018.stdout:9/374: dread d3/d6/d9/fa [0,4194304] 0 2024-04-12T20:56:02.339 INFO:tasks.workunit.client.0.smithi018.stdout:8/311: unlink db/f11 0 2024-04-12T20:56:02.341 INFO:tasks.workunit.client.0.smithi018.stdout:9/375: readlink d3/d6/d9/dc/d1c/l33 0 2024-04-12T20:56:02.341 INFO:tasks.workunit.client.0.smithi018.stdout:5/241: link d0/d2/d33/d46/d1e/c3c d0/d2/d33/d10/d26/c50 0 2024-04-12T20:56:02.348 INFO:tasks.workunit.client.0.smithi018.stdout:5/242: getdents d0/d2/d33/d46/d35/d4d 0 2024-04-12T20:56:02.351 INFO:tasks.workunit.client.0.smithi018.stdout:5/243: chown d0/d2/d33/d46/d35/f4f 119 1 2024-04-12T20:56:02.375 INFO:tasks.workunit.client.0.smithi018.stdout:5/244: write d0/d2/f1a [2619702,86560] 0 2024-04-12T20:56:02.376 INFO:tasks.workunit.client.0.smithi018.stdout:8/312: write db/d25/d26/f3a [732318,61130] 0 2024-04-12T20:56:02.378 INFO:tasks.workunit.client.0.smithi018.stdout:8/313: read - db/d25/f4c zero size 2024-04-12T20:56:02.381 INFO:tasks.workunit.client.0.smithi018.stdout:8/314: dread - db/d4e/f4f zero size 2024-04-12T20:56:02.384 INFO:tasks.workunit.client.0.smithi018.stdout:8/315: truncate db/d25/f4c 988398 0 2024-04-12T20:56:02.393 INFO:tasks.workunit.client.0.smithi018.stdout:5/245: dread d0/d2b/f3d [0,4194304] 0 2024-04-12T20:56:02.398 INFO:tasks.workunit.client.0.smithi018.stdout:5/246: symlink d0/d2/d33/d10/d4e/l51 0 2024-04-12T20:56:02.399 INFO:tasks.workunit.client.0.smithi018.stdout:9/376: write d3/d6/d9/dc/d13/f17 [3475471,68189] 0 2024-04-12T20:56:02.403 INFO:tasks.workunit.client.0.smithi018.stdout:5/247: symlink d0/d2/d11/l52 0 2024-04-12T20:56:02.403 INFO:tasks.workunit.client.0.smithi018.stdout:9/377: write d3/d6/d9/d3f/f50 [947959,103344] 0 2024-04-12T20:56:02.407 INFO:tasks.workunit.client.0.smithi018.stdout:5/248: stat d0/d2/d33/d46/d1e/d2e/c24 0 2024-04-12T20:56:02.412 INFO:tasks.workunit.client.0.smithi018.stdout:8/316: symlink db/d25/d26/l61 0 2024-04-12T20:56:02.418 INFO:tasks.workunit.client.0.smithi018.stdout:5/249: creat d0/d2/d33/d10/d2f/f53 x:0 0 0 2024-04-12T20:56:02.430 INFO:tasks.workunit.client.0.smithi018.stdout:5/250: getdents d0/d2/d33/d46 0 2024-04-12T20:56:02.433 INFO:tasks.workunit.client.0.smithi018.stdout:9/378: dread d3/d6/d9/dc/d1c/d21/f59 [0,4194304] 0 2024-04-12T20:56:02.433 INFO:tasks.workunit.client.0.smithi018.stdout:5/251: fsync d0/d2/d33/d10/d26/d30/f31 0 2024-04-12T20:56:02.449 INFO:tasks.workunit.client.0.smithi018.stdout:9/379: rmdir d3/d6/d9/dc/d13 39 2024-04-12T20:56:02.451 INFO:tasks.workunit.client.0.smithi018.stdout:5/252: dwrite d0/d2/d33/f15 [4194304,4194304] 0 2024-04-12T20:56:02.463 INFO:tasks.workunit.client.0.smithi018.stdout:4/255: sync 2024-04-12T20:56:02.463 INFO:tasks.workunit.client.0.smithi018.stdout:2/189: sync 2024-04-12T20:56:02.466 INFO:tasks.workunit.client.0.smithi018.stdout:9/380: dread d3/d6/d9/dc/f22 [0,4194304] 0 2024-04-12T20:56:02.470 INFO:tasks.workunit.client.0.smithi018.stdout:2/190: symlink d5/l42 0 2024-04-12T20:56:02.471 INFO:tasks.workunit.client.0.smithi018.stdout:9/381: chown d3/d6/d9/d1e/l54 182285069 1 2024-04-12T20:56:02.472 INFO:tasks.workunit.client.0.smithi018.stdout:4/256: symlink d3/db/l4a 0 2024-04-12T20:56:02.475 INFO:tasks.workunit.client.0.smithi018.stdout:2/191: truncate d5/d8/d25/f34 1416732 0 2024-04-12T20:56:02.478 INFO:tasks.workunit.client.0.smithi018.stdout:9/382: creat d3/d6/d9/d1e/f74 x:0 0 0 2024-04-12T20:56:02.478 INFO:tasks.workunit.client.0.smithi018.stdout:4/257: mkdir d3/d34/d43/d4b 0 2024-04-12T20:56:02.478 INFO:tasks.workunit.client.0.smithi018.stdout:4/258: chown d3/d32 10945278 1 2024-04-12T20:56:02.484 INFO:tasks.workunit.client.0.smithi018.stdout:5/253: dwrite d0/d2/d33/d46/f13 [0,4194304] 0 2024-04-12T20:56:02.497 INFO:tasks.workunit.client.0.smithi018.stdout:4/259: dwrite d3/d34/d40/f44 [0,4194304] 0 2024-04-12T20:56:02.510 INFO:tasks.workunit.client.0.smithi018.stdout:9/383: dwrite d3/d6/d9/dc/d13/f17 [0,4194304] 0 2024-04-12T20:56:02.512 INFO:tasks.workunit.client.0.smithi018.stdout:5/254: dwrite d0/d2/d33/d10/f39 [0,4194304] 0 2024-04-12T20:56:02.517 INFO:tasks.workunit.client.0.smithi018.stdout:5/255: chown d0/d2/d33/d46/d1e/f42 99 1 2024-04-12T20:56:02.526 INFO:tasks.workunit.client.0.smithi018.stdout:4/260: link d3/d32/d33/f29 d3/db/f4c 0 2024-04-12T20:56:02.526 INFO:tasks.workunit.client.0.smithi018.stdout:9/384: symlink d3/d6/d9/dc/d13/d5b/l75 0 2024-04-12T20:56:02.528 INFO:tasks.workunit.client.0.smithi018.stdout:5/256: creat d0/d2b/d40/f54 x:0 0 0 2024-04-12T20:56:02.535 INFO:tasks.workunit.client.0.smithi018.stdout:4/261: symlink d3/d34/d43/d4b/l4d 0 2024-04-12T20:56:02.536 INFO:tasks.workunit.client.0.smithi018.stdout:4/262: chown d3/db/l4a 108 1 2024-04-12T20:56:02.540 INFO:tasks.workunit.client.0.smithi018.stdout:9/385: getdents d3/d6/d9/dc/d1c 0 2024-04-12T20:56:02.540 INFO:tasks.workunit.client.0.smithi018.stdout:4/263: read - d3/db/f26 zero size 2024-04-12T20:56:02.542 INFO:tasks.workunit.client.0.smithi018.stdout:4/264: fdatasync d3/d34/d43/f2c 0 2024-04-12T20:56:02.544 INFO:tasks.workunit.client.0.smithi018.stdout:9/386: chown d3/d6/d9/dc/d13/l2a 14570057 1 2024-04-12T20:56:02.546 INFO:tasks.workunit.client.0.smithi018.stdout:4/265: creat d3/d32/f4e x:0 0 0 2024-04-12T20:56:02.549 INFO:tasks.workunit.client.0.smithi018.stdout:5/257: dread d0/d2b/f2d [0,4194304] 0 2024-04-12T20:56:02.551 INFO:tasks.workunit.client.0.smithi018.stdout:5/258: write d0/d2/d11/f29 [1156298,38756] 0 2024-04-12T20:56:02.552 INFO:tasks.workunit.client.0.smithi018.stdout:9/387: creat d3/d6/d9/dc/d1c/d21/d41/f76 x:0 0 0 2024-04-12T20:56:02.555 INFO:tasks.workunit.client.0.smithi018.stdout:5/259: chown d0/d2/d33/d46/d1e/d2e/c24 102050 1 2024-04-12T20:56:02.555 INFO:tasks.workunit.client.0.smithi018.stdout:9/388: fdatasync d3/d6/d9/dc/d1c/d21/d41/f76 0 2024-04-12T20:56:02.558 INFO:tasks.workunit.client.0.smithi018.stdout:4/266: write d3/db/f4c [459482,88125] 0 2024-04-12T20:56:02.565 INFO:tasks.workunit.client.0.smithi018.stdout:5/260: mkdir d0/d2/d33/d46/d1e/d55 0 2024-04-12T20:56:02.566 INFO:tasks.workunit.client.0.smithi018.stdout:4/267: fdatasync d3/d32/d33/f17 0 2024-04-12T20:56:02.569 INFO:tasks.workunit.client.0.smithi018.stdout:5/261: write d0/d2/d33/d10/f39 [4609910,37849] 0 2024-04-12T20:56:02.571 INFO:tasks.workunit.client.0.smithi018.stdout:4/268: truncate d3/db/f26 242885 0 2024-04-12T20:56:02.578 INFO:tasks.workunit.client.0.smithi018.stdout:9/389: dwrite d3/d6/d9/dc/d1c/f34 [0,4194304] 0 2024-04-12T20:56:02.580 INFO:tasks.workunit.client.0.smithi018.stdout:4/269: getdents d3/d34/d40 0 2024-04-12T20:56:02.585 INFO:tasks.workunit.client.0.smithi018.stdout:4/270: rmdir d3/d34/d43 39 2024-04-12T20:56:02.592 INFO:tasks.workunit.client.0.smithi018.stdout:5/262: dwrite d0/d2/d33/d10/f39 [0,4194304] 0 2024-04-12T20:56:02.597 INFO:tasks.workunit.client.0.smithi018.stdout:5/263: mknod d0/d2/d11/d37/c56 0 2024-04-12T20:56:02.601 INFO:tasks.workunit.client.0.smithi018.stdout:5/264: symlink d0/d2/d33/d46/d35/d4d/l57 0 2024-04-12T20:56:02.605 INFO:tasks.workunit.client.0.smithi018.stdout:5/265: rmdir d0/d2b/d40 39 2024-04-12T20:56:02.612 INFO:tasks.workunit.client.0.smithi018.stdout:5/266: write d0/d2/d33/d46/f1d [230751,50194] 0 2024-04-12T20:56:02.617 INFO:tasks.workunit.client.0.smithi018.stdout:9/390: dread d3/d6/f1f [0,4194304] 0 2024-04-12T20:56:02.621 INFO:tasks.workunit.client.0.smithi018.stdout:9/391: mkdir d3/d6/d9/dc/d6f/d77 0 2024-04-12T20:56:02.629 INFO:tasks.workunit.client.0.smithi018.stdout:5/267: dwrite d0/d2/d33/d10/f39 [0,4194304] 0 2024-04-12T20:56:02.634 INFO:tasks.workunit.client.0.smithi018.stdout:5/268: dread - d0/d2/d33/d10/d2f/f53 zero size 2024-04-12T20:56:02.634 INFO:tasks.workunit.client.0.smithi018.stdout:9/392: dread d3/d6/d9/d1e/d3a/f31 [0,4194304] 0 2024-04-12T20:56:02.640 INFO:tasks.workunit.client.0.smithi018.stdout:5/269: getdents d0/d2/d11/d37 0 2024-04-12T20:56:02.645 INFO:tasks.workunit.client.0.smithi018.stdout:5/270: write d0/d2/d33/d10/d26/f43 [506802,100962] 0 2024-04-12T20:56:02.645 INFO:tasks.workunit.client.0.smithi018.stdout:9/393: dread d3/d6/d1b/f26 [0,4194304] 0 2024-04-12T20:56:02.649 INFO:tasks.workunit.client.0.smithi018.stdout:5/271: write d0/d2/d11/d37/f4c [841531,26296] 0 2024-04-12T20:56:02.650 INFO:tasks.workunit.client.0.smithi018.stdout:9/394: mkdir d3/d6/d9/dc/d1c/d21/d41/d78 0 2024-04-12T20:56:02.655 INFO:tasks.workunit.client.0.smithi018.stdout:5/272: getdents d0/d2/d33/d46/d1e 0 2024-04-12T20:56:02.657 INFO:tasks.workunit.client.0.smithi018.stdout:5/273: mkdir d0/d2/d33/d10/d2f/d58 0 2024-04-12T20:56:02.661 INFO:tasks.workunit.client.0.smithi018.stdout:5/274: creat d0/d2/d33/d46/d18/f59 x:0 0 0 2024-04-12T20:56:02.667 INFO:tasks.workunit.client.0.smithi018.stdout:9/395: dwrite d3/d6/d1b/f35 [0,4194304] 0 2024-04-12T20:56:02.670 INFO:tasks.workunit.client.0.smithi018.stdout:9/396: chown d3/d6/d9/dc/d6f 4265 1 2024-04-12T20:56:02.676 INFO:tasks.workunit.client.0.smithi018.stdout:9/397: rmdir d3/d6/d9/dc/d1c 39 2024-04-12T20:56:02.680 INFO:tasks.workunit.client.0.smithi018.stdout:5/275: dwrite d0/d2/d33/d46/f3a [0,4194304] 0 2024-04-12T20:56:02.683 INFO:tasks.workunit.client.0.smithi018.stdout:5/276: chown d0/d2/d33/d46/d1e/d2e/c24 124822743 1 2024-04-12T20:56:02.685 INFO:tasks.workunit.client.0.smithi018.stdout:9/398: mkdir d3/d6/d9/dc/d5a/d79 0 2024-04-12T20:56:02.689 INFO:tasks.workunit.client.0.smithi018.stdout:5/277: truncate d0/d2/d33/d46/d1e/f42 667664 0 2024-04-12T20:56:02.692 INFO:tasks.workunit.client.0.smithi018.stdout:5/278: dread - d0/d2b/d40/f54 zero size 2024-04-12T20:56:02.695 INFO:tasks.workunit.client.0.smithi018.stdout:5/279: stat d0/d2/d33/d10/d26/c50 0 2024-04-12T20:56:02.696 INFO:tasks.workunit.client.0.smithi018.stdout:9/399: getdents d3/d6/d9/d1e/d3a 0 2024-04-12T20:56:02.698 INFO:tasks.workunit.client.0.smithi018.stdout:6/334: sync 2024-04-12T20:56:02.698 INFO:tasks.workunit.client.0.smithi018.stdout:7/288: sync 2024-04-12T20:56:02.699 INFO:tasks.workunit.client.0.smithi018.stdout:0/259: sync 2024-04-12T20:56:02.699 INFO:tasks.workunit.client.0.smithi018.stdout:3/272: sync 2024-04-12T20:56:02.699 INFO:tasks.workunit.client.0.smithi018.stdout:1/182: sync 2024-04-12T20:56:02.699 INFO:tasks.workunit.client.0.smithi018.stdout:5/280: write d0/d2/d33/d46/f1d [1641703,31209] 0 2024-04-12T20:56:02.702 INFO:tasks.workunit.client.0.smithi018.stdout:9/400: write d3/d6/d9/d1e/f45 [3736067,57561] 0 2024-04-12T20:56:02.704 INFO:tasks.workunit.client.0.smithi018.stdout:9/401: read - d3/d6/d9/d3f/f6e zero size 2024-04-12T20:56:02.704 INFO:tasks.workunit.client.0.smithi018.stdout:7/289: write d1/db/f43 [3998829,8656] 0 2024-04-12T20:56:02.708 INFO:tasks.workunit.client.0.smithi018.stdout:9/402: fsync d3/d6/f10 0 2024-04-12T20:56:02.712 INFO:tasks.workunit.client.0.smithi018.stdout:7/290: dread - d1/db/d31/f37 zero size 2024-04-12T20:56:02.717 INFO:tasks.workunit.client.0.smithi018.stdout:3/273: getdents d2/d6/d30 0 2024-04-12T20:56:02.718 INFO:tasks.workunit.client.0.smithi018.stdout:1/183: getdents d3/d9/da/dd 0 2024-04-12T20:56:02.725 INFO:tasks.workunit.client.0.smithi018.stdout:1/184: rename d3/d9/d11 to d3/d9/d11/d3e 22 2024-04-12T20:56:02.726 INFO:tasks.workunit.client.0.smithi018.stdout:9/403: chown d3/d6/d9/dc/d1c/c38 4 1 2024-04-12T20:56:02.726 INFO:tasks.workunit.client.0.smithi018.stdout:7/291: write d1/db/d31/f4d [7359751,13751] 0 2024-04-12T20:56:02.726 INFO:tasks.workunit.client.0.smithi018.stdout:5/281: truncate d0/d2/f1a 3911344 0 2024-04-12T20:56:02.727 INFO:tasks.workunit.client.0.smithi018.stdout:0/260: dwrite da/f13 [0,4194304] 0 2024-04-12T20:56:02.727 INFO:tasks.workunit.client.0.smithi018.stdout:1/185: chown d3/d9 1857147158 1 2024-04-12T20:56:02.730 INFO:tasks.workunit.client.0.smithi018.stdout:3/274: mkdir d2/d4d 0 2024-04-12T20:56:02.730 INFO:tasks.workunit.client.0.smithi018.stdout:6/335: link d5/d6/d1b/d3b/f3d d5/df/f87 0 2024-04-12T20:56:02.733 INFO:tasks.workunit.client.0.smithi018.stdout:1/186: write d3/f7 [4873432,38119] 0 2024-04-12T20:56:02.733 INFO:tasks.workunit.client.0.smithi018.stdout:0/261: truncate da/d15/d3e/f54 832898 0 2024-04-12T20:56:02.736 INFO:tasks.workunit.client.0.smithi018.stdout:6/336: symlink d5/d6/l88 0 2024-04-12T20:56:02.737 INFO:tasks.workunit.client.0.smithi018.stdout:1/187: write d3/d9/da/dd/d31/f36 [48211,16518] 0 2024-04-12T20:56:02.740 INFO:tasks.workunit.client.0.smithi018.stdout:7/292: fsync d1/d6/d8/f13 0 2024-04-12T20:56:02.742 INFO:tasks.workunit.client.0.smithi018.stdout:7/293: read - d1/d6/f5a zero size 2024-04-12T20:56:02.742 INFO:tasks.workunit.client.0.smithi018.stdout:1/188: write d3/d9/da/d14/f38 [4113604,117409] 0 2024-04-12T20:56:02.745 INFO:tasks.workunit.client.0.smithi018.stdout:9/404: rename d3/d6/f10 to d3/d6/d9/dc/d1c/d21/d41/d78/f7a 0 2024-04-12T20:56:02.751 INFO:tasks.workunit.client.0.smithi018.stdout:9/405: write d3/d6/d9/dc/d46/f65 [77476,35977] 0 2024-04-12T20:56:02.757 INFO:tasks.workunit.client.0.smithi018.stdout:6/337: link d5/df/d19/d43/l70 d5/d71/l89 0 2024-04-12T20:56:02.758 INFO:tasks.workunit.client.0.smithi018.stdout:5/282: write d0/d2/d33/d10/d26/f4a [118001,87737] 0 2024-04-12T20:56:02.759 INFO:tasks.workunit.client.0.smithi018.stdout:9/406: mknod d3/d6/d9/dc/d13/d5b/c7b 0 2024-04-12T20:56:02.762 INFO:tasks.workunit.client.0.smithi018.stdout:5/283: fdatasync d0/d2/d33/d46/d1e/f48 0 2024-04-12T20:56:02.763 INFO:tasks.workunit.client.0.smithi018.stdout:7/294: stat d1/c1d 0 2024-04-12T20:56:02.767 INFO:tasks.workunit.client.0.smithi018.stdout:5/284: write d0/d2/d33/d46/d1e/f42 [456696,8343] 0 2024-04-12T20:56:02.767 INFO:tasks.workunit.client.0.smithi018.stdout:1/189: dread d3/d9/da/dd/d31/d32/f34 [0,4194304] 0 2024-04-12T20:56:02.768 INFO:tasks.workunit.client.0.smithi018.stdout:7/295: stat d1/d6/d8/c29 0 2024-04-12T20:56:02.770 INFO:tasks.workunit.client.0.smithi018.stdout:7/296: chown d1/d6/d8 1 1 2024-04-12T20:56:02.777 INFO:tasks.workunit.client.0.smithi018.stdout:9/407: dread d3/d6/f1f [0,4194304] 0 2024-04-12T20:56:02.779 INFO:tasks.workunit.client.0.smithi018.stdout:3/275: creat d2/d6/f4e x:0 0 0 2024-04-12T20:56:02.782 INFO:tasks.workunit.client.0.smithi018.stdout:5/285: creat d0/d2/d33/d46/d35/f5a x:0 0 0 2024-04-12T20:56:02.784 INFO:tasks.workunit.client.0.smithi018.stdout:1/190: creat d3/d9/da/dd/f3f x:0 0 0 2024-04-12T20:56:02.786 INFO:tasks.workunit.client.0.smithi018.stdout:0/262: rmdir da/d15/d17/d21/d36/d47/d55 0 2024-04-12T20:56:02.788 INFO:tasks.workunit.client.0.smithi018.stdout:5/286: write d0/d2/d33/d46/d35/f5a [510466,83719] 0 2024-04-12T20:56:02.791 INFO:tasks.workunit.client.0.smithi018.stdout:3/276: symlink d2/d6/d30/d31/d38/d3a/d4c/l4f 0 2024-04-12T20:56:02.796 INFO:tasks.workunit.client.0.smithi018.stdout:7/297: rename d1/f15 to d1/f66 0 2024-04-12T20:56:02.798 INFO:tasks.workunit.client.0.smithi018.stdout:1/191: creat d3/d9/da/dd/d31/d32/d13/d19/d2d/f40 x:0 0 0 2024-04-12T20:56:02.798 INFO:tasks.workunit.client.0.smithi018.stdout:6/338: dwrite d5/df/f87 [0,4194304] 0 2024-04-12T20:56:02.801 INFO:tasks.workunit.client.0.smithi018.stdout:0/263: dread da/d15/f41 [0,4194304] 0 2024-04-12T20:56:02.826 INFO:tasks.workunit.client.0.smithi018.stdout:9/408: rename d3/d6/d9/d1e/d3a to d3/d6/d9/dc/d13/d5b/d7c 0 2024-04-12T20:56:02.826 INFO:tasks.workunit.client.0.smithi018.stdout:3/277: creat d2/d6/d30/d31/f50 x:0 0 0 2024-04-12T20:56:02.827 INFO:tasks.workunit.client.0.smithi018.stdout:6/339: unlink d5/d15/l3e 0 2024-04-12T20:56:02.828 INFO:tasks.workunit.client.0.smithi018.stdout:0/264: mkdir da/d15/d17/d5c 0 2024-04-12T20:56:02.829 INFO:tasks.workunit.client.0.smithi018.stdout:9/409: mknod d3/d6/d43/c7d 0 2024-04-12T20:56:02.834 INFO:tasks.workunit.client.0.smithi018.stdout:6/340: write d5/df/d19/d74/f86 [924747,57468] 0 2024-04-12T20:56:02.834 INFO:tasks.workunit.client.0.smithi018.stdout:1/192: creat d3/d9/da/dd/d31/d32/f41 x:0 0 0 2024-04-12T20:56:02.835 INFO:tasks.workunit.client.0.smithi018.stdout:9/410: mkdir d3/d6/d9/dc/d5a/d79/d7e 0 2024-04-12T20:56:02.837 INFO:tasks.workunit.client.0.smithi018.stdout:0/265: rename da/d15/d17/f48 to da/d15/d17/d21/d36/d47/f5d 0 2024-04-12T20:56:02.839 INFO:tasks.workunit.client.0.smithi018.stdout:6/341: getdents d5/df/d19 0 2024-04-12T20:56:02.840 INFO:tasks.workunit.client.0.smithi018.stdout:7/298: creat d1/d6/f67 x:0 0 0 2024-04-12T20:56:02.840 INFO:tasks.workunit.client.0.smithi018.stdout:0/266: write da/f25 [5033833,108113] 0 2024-04-12T20:56:02.841 INFO:tasks.workunit.client.0.smithi018.stdout:1/193: symlink d3/d9/da/d14/l42 0 2024-04-12T20:56:02.841 INFO:tasks.workunit.client.0.smithi018.stdout:9/411: creat d3/d6/d43/f7f x:0 0 0 2024-04-12T20:56:02.842 INFO:tasks.workunit.client.0.smithi018.stdout:6/342: chown d5/df/d31/d4e 110758953 1 2024-04-12T20:56:02.843 INFO:tasks.workunit.client.0.smithi018.stdout:0/267: chown da/d15/f41 40808 1 2024-04-12T20:56:02.844 INFO:tasks.workunit.client.0.smithi018.stdout:9/412: chown d3/f5e 42 1 2024-04-12T20:56:02.844 INFO:tasks.workunit.client.0.smithi018.stdout:1/194: fdatasync d3/f10 0 2024-04-12T20:56:02.846 INFO:tasks.workunit.client.0.smithi018.stdout:6/343: rmdir d5/df/d19/d2b 39 2024-04-12T20:56:02.847 INFO:tasks.workunit.client.0.smithi018.stdout:0/268: creat da/d15/d3e/f5e x:0 0 0 2024-04-12T20:56:02.850 INFO:tasks.workunit.client.0.smithi018.stdout:1/195: mknod d3/d9/d11/c43 0 2024-04-12T20:56:02.850 INFO:tasks.workunit.client.0.smithi018.stdout:6/344: readlink d5/df/d19/d43/l51 0 2024-04-12T20:56:02.853 INFO:tasks.workunit.client.0.smithi018.stdout:6/345: chown d5/d6/d1b/d3b/d35/f33 5447604 1 2024-04-12T20:56:02.853 INFO:tasks.workunit.client.0.smithi018.stdout:0/269: creat da/d15/d17/d21/f5f x:0 0 0 2024-04-12T20:56:02.854 INFO:tasks.workunit.client.0.smithi018.stdout:1/196: chown d3/d9/da/l1c 27915 1 2024-04-12T20:56:02.856 INFO:tasks.workunit.client.0.smithi018.stdout:6/346: symlink d5/d6/d1b/l8a 0 2024-04-12T20:56:02.862 INFO:tasks.workunit.client.0.smithi018.stdout:6/347: getdents d5/d15 0 2024-04-12T20:56:02.864 INFO:tasks.workunit.client.0.smithi018.stdout:6/348: chown d5/f78 32 1 2024-04-12T20:56:02.864 INFO:tasks.workunit.client.0.smithi018.stdout:9/413: dread d3/d6/d1b/f35 [0,4194304] 0 2024-04-12T20:56:02.868 INFO:tasks.workunit.client.0.smithi018.stdout:6/349: chown d5/d6/l88 21588562 1 2024-04-12T20:56:02.873 INFO:tasks.workunit.client.0.smithi018.stdout:9/414: truncate d3/d6/d9/dc/d46/f65 482776 0 2024-04-12T20:56:02.874 INFO:tasks.workunit.client.0.smithi018.stdout:0/270: getdents da/d15/d17/d21 0 2024-04-12T20:56:02.874 INFO:tasks.workunit.client.0.smithi018.stdout:6/350: rename d5/d15/l36 to d5/d6/d1b/d3b/d22/l8b 0 2024-04-12T20:56:02.875 INFO:tasks.workunit.client.0.smithi018.stdout:0/271: chown da/fc 2 1 2024-04-12T20:56:02.878 INFO:tasks.workunit.client.0.smithi018.stdout:1/197: dwrite d3/d9/da/dd/d31/f36 [0,4194304] 0 2024-04-12T20:56:02.878 INFO:tasks.workunit.client.0.smithi018.stdout:9/415: fdatasync d3/f23 0 2024-04-12T20:56:02.879 INFO:tasks.workunit.client.0.smithi018.stdout:6/351: read d5/d15/d60/f65 [459222,54863] 0 2024-04-12T20:56:02.884 INFO:tasks.workunit.client.0.smithi018.stdout:6/352: mkdir d5/d6/d1b/d8c 0 2024-04-12T20:56:02.886 INFO:tasks.workunit.client.0.smithi018.stdout:1/198: mkdir d3/d9/da/dd/d31/d44 0 2024-04-12T20:56:02.887 INFO:tasks.workunit.client.0.smithi018.stdout:9/416: write d3/d6/d9/dc/d1c/f62 [258065,38980] 0 2024-04-12T20:56:02.889 INFO:tasks.workunit.client.0.smithi018.stdout:6/353: fdatasync d5/df/d31/d4e/f38 0 2024-04-12T20:56:02.890 INFO:tasks.workunit.client.0.smithi018.stdout:1/199: truncate d3/d9/f3c 592347 0 2024-04-12T20:56:02.891 INFO:tasks.workunit.client.0.smithi018.stdout:9/417: fsync d3/d6/d9/d1e/f74 0 2024-04-12T20:56:02.896 INFO:tasks.workunit.client.0.smithi018.stdout:0/272: dread da/d15/f20 [0,4194304] 0 2024-04-12T20:56:02.899 INFO:tasks.workunit.client.0.smithi018.stdout:6/354: creat d5/f8d x:0 0 0 2024-04-12T20:56:02.899 INFO:tasks.workunit.client.0.smithi018.stdout:0/273: symlink da/l60 0 2024-04-12T20:56:02.903 INFO:tasks.workunit.client.0.smithi018.stdout:0/274: mknod da/c61 0 2024-04-12T20:56:02.903 INFO:tasks.workunit.client.0.smithi018.stdout:6/355: truncate d5/d6/d1b/d3b/d22/f81 691417 0 2024-04-12T20:56:02.908 INFO:tasks.workunit.client.0.smithi018.stdout:0/275: symlink da/d15/d17/d21/l62 0 2024-04-12T20:56:02.909 INFO:tasks.workunit.client.0.smithi018.stdout:6/356: unlink d5/df/d19/f68 0 2024-04-12T20:56:02.916 INFO:tasks.workunit.client.0.smithi018.stdout:6/357: write d5/f10 [1109901,20994] 0 2024-04-12T20:56:02.916 INFO:tasks.workunit.client.0.smithi018.stdout:9/418: dwrite d3/d6/d9/dc/f2d [0,4194304] 0 2024-04-12T20:56:02.921 INFO:tasks.workunit.client.0.smithi018.stdout:1/200: truncate d3/d9/da/dd/d31/d32/f34 1596564 0 2024-04-12T20:56:02.921 INFO:tasks.workunit.client.0.smithi018.stdout:0/276: creat da/d15/d17/d21/d36/f63 x:0 0 0 2024-04-12T20:56:02.929 INFO:tasks.workunit.client.0.smithi018.stdout:0/277: stat da/d15/d17/d21/d36/l44 0 2024-04-12T20:56:02.929 INFO:tasks.workunit.client.0.smithi018.stdout:9/419: creat d3/f80 x:0 0 0 2024-04-12T20:56:02.935 INFO:tasks.workunit.client.0.smithi018.stdout:9/420: dread d3/d6/d9/d1e/f40 [0,4194304] 0 2024-04-12T20:56:02.936 INFO:tasks.workunit.client.0.smithi018.stdout:1/201: dread d3/d9/da/dd/d31/d32/f1b [0,4194304] 0 2024-04-12T20:56:02.939 INFO:tasks.workunit.client.0.smithi018.stdout:1/202: stat d3/d9/da/dd/l16 0 2024-04-12T20:56:02.939 INFO:tasks.workunit.client.0.smithi018.stdout:8/317: sync 2024-04-12T20:56:02.944 INFO:tasks.workunit.client.0.smithi018.stdout:1/203: chown d3/d9/da/lb 1148 1 2024-04-12T20:56:02.944 INFO:tasks.workunit.client.0.smithi018.stdout:9/421: rmdir d3/d6 39 2024-04-12T20:56:02.948 INFO:tasks.workunit.client.0.smithi018.stdout:1/204: read - d3/d9/da/dd/d31/d32/d13/d19/d2d/f40 zero size 2024-04-12T20:56:02.950 INFO:tasks.workunit.client.0.smithi018.stdout:0/278: dread da/f25 [0,4194304] 0 2024-04-12T20:56:02.950 INFO:tasks.workunit.client.0.smithi018.stdout:9/422: dread - d3/d6/d9/d3f/f6e zero size 2024-04-12T20:56:02.953 INFO:tasks.workunit.client.0.smithi018.stdout:1/205: fdatasync d3/d9/da/dd/d31/d32/f41 0 2024-04-12T20:56:02.954 INFO:tasks.workunit.client.0.smithi018.stdout:0/279: truncate da/d15/d3e/f5e 397874 0 2024-04-12T20:56:02.957 INFO:tasks.workunit.client.0.smithi018.stdout:1/206: write d3/f4 [7677103,78192] 0 2024-04-12T20:56:02.957 INFO:tasks.workunit.client.0.smithi018.stdout:6/358: dread d5/df/d19/f25 [0,4194304] 0 2024-04-12T20:56:02.958 INFO:tasks.workunit.client.0.smithi018.stdout:0/280: fdatasync da/d15/f20 0 2024-04-12T20:56:02.961 INFO:tasks.workunit.client.0.smithi018.stdout:9/423: link d3/d6/d9/dc/d46/f48 d3/d6/d9/d3f/f81 0 2024-04-12T20:56:02.965 INFO:tasks.workunit.client.0.smithi018.stdout:1/207: creat d3/d9/da/dd/d31/f45 x:0 0 0 2024-04-12T20:56:02.966 INFO:tasks.workunit.client.0.smithi018.stdout:9/424: chown d3/d6/d9/d1e/f71 187991 1 2024-04-12T20:56:02.966 INFO:tasks.workunit.client.0.smithi018.stdout:6/359: mkdir d5/d6/d1b/d3b/d8e 0 2024-04-12T20:56:02.968 INFO:tasks.workunit.client.0.smithi018.stdout:1/208: fsync d3/d9/f18 0 2024-04-12T20:56:02.969 INFO:tasks.workunit.client.0.smithi018.stdout:6/360: fdatasync d5/d6/d1b/d3b/f3d 0 2024-04-12T20:56:02.975 INFO:tasks.workunit.client.0.smithi018.stdout:9/425: getdents d3/d6/d3d 0 2024-04-12T20:56:02.975 INFO:tasks.workunit.client.0.smithi018.stdout:8/318: dread db/f2d [4194304,4194304] 0 2024-04-12T20:56:02.977 INFO:tasks.workunit.client.0.smithi018.stdout:1/209: creat d3/d9/da/dd/d31/d32/d13/f46 x:0 0 0 2024-04-12T20:56:02.980 INFO:tasks.workunit.client.0.smithi018.stdout:6/361: mkdir d5/d6/d1b/d3b/d8e/d8f 0 2024-04-12T20:56:02.981 INFO:tasks.workunit.client.0.smithi018.stdout:8/319: symlink db/dc/d45/l62 0 2024-04-12T20:56:02.983 INFO:tasks.workunit.client.0.smithi018.stdout:1/210: chown d3/d9/da/dd/d31/d32/c1d 916 1 2024-04-12T20:56:02.986 INFO:tasks.workunit.client.0.smithi018.stdout:8/320: creat db/d4e/d56/f63 x:0 0 0 2024-04-12T20:56:02.986 INFO:tasks.workunit.client.0.smithi018.stdout:6/362: mkdir d5/d6/d90 0 2024-04-12T20:56:02.988 INFO:tasks.workunit.client.0.smithi018.stdout:8/321: chown db/dc/l38 52496 1 2024-04-12T20:56:02.993 INFO:tasks.workunit.client.0.smithi018.stdout:8/322: creat db/f64 x:0 0 0 2024-04-12T20:56:02.997 INFO:tasks.workunit.client.0.smithi018.stdout:8/323: fdatasync db/d25/d26/f5c 0 2024-04-12T20:56:02.997 INFO:tasks.workunit.client.0.smithi018.stdout:9/426: truncate d3/d6/d9/d1e/f45 2643865 0 2024-04-12T20:56:03.001 INFO:tasks.workunit.client.0.smithi018.stdout:9/427: chown d3/d6/d9/dc/d1c/d21/d41/c5f 15732077 1 2024-04-12T20:56:03.005 INFO:tasks.workunit.client.0.smithi018.stdout:8/324: rename db/dc/f1e to db/dc/d45/f65 0 2024-04-12T20:56:03.007 INFO:tasks.workunit.client.0.smithi018.stdout:1/211: dwrite d3/f7 [4194304,4194304] 0 2024-04-12T20:56:03.007 INFO:tasks.workunit.client.0.smithi018.stdout:9/428: dread d3/d6/d1b/f26 [0,4194304] 0 2024-04-12T20:56:03.016 INFO:tasks.workunit.client.0.smithi018.stdout:1/212: read d3/d9/da/f20 [386587,95934] 0 2024-04-12T20:56:03.017 INFO:tasks.workunit.client.0.smithi018.stdout:9/429: creat d3/d6/d9/f82 x:0 0 0 2024-04-12T20:56:03.020 INFO:tasks.workunit.client.0.smithi018.stdout:1/213: write d3/d9/da/dd/f17 [4812907,638] 0 2024-04-12T20:56:03.020 INFO:tasks.workunit.client.0.smithi018.stdout:6/363: dread d5/df/f18 [4194304,4194304] 0 2024-04-12T20:56:03.027 INFO:tasks.workunit.client.0.smithi018.stdout:1/214: mkdir d3/d9/da/dd/d31/d32/d33/d47 0 2024-04-12T20:56:03.028 INFO:tasks.workunit.client.0.smithi018.stdout:9/430: creat d3/d6/d9/dc/d1c/d21/d41/d78/f83 x:0 0 0 2024-04-12T20:56:03.028 INFO:tasks.workunit.client.0.smithi018.stdout:8/325: getdents db/d4e/d56 0 2024-04-12T20:56:03.030 INFO:tasks.workunit.client.0.smithi018.stdout:9/431: read d3/f5e [146376,17973] 0 2024-04-12T20:56:03.032 INFO:tasks.workunit.client.0.smithi018.stdout:6/364: creat d5/d6/d1b/d3b/f91 x:0 0 0 2024-04-12T20:56:03.035 INFO:tasks.workunit.client.0.smithi018.stdout:8/326: mknod db/d25/d26/d27/c66 0 2024-04-12T20:56:03.035 INFO:tasks.workunit.client.0.smithi018.stdout:1/215: mkdir d3/d9/da/dd/d31/d32/d13/d19/d2d/d48 0 2024-04-12T20:56:03.038 INFO:tasks.workunit.client.0.smithi018.stdout:8/327: creat db/d4e/d56/d59/f67 x:0 0 0 2024-04-12T20:56:03.041 INFO:tasks.workunit.client.0.smithi018.stdout:9/432: rename d3/d6/d1b/c66 to d3/d6/d9/dc/d5a/d79/d7e/c84 0 2024-04-12T20:56:03.045 INFO:tasks.workunit.client.0.smithi018.stdout:9/433: symlink d3/d6/d43/l85 0 2024-04-12T20:56:03.048 INFO:tasks.workunit.client.0.smithi018.stdout:6/365: rename d5/d71/f7f to d5/d6/d1b/d3b/d8e/f92 0 2024-04-12T20:56:03.048 INFO:tasks.workunit.client.0.smithi018.stdout:8/328: dread db/d25/d26/f3a [0,4194304] 0 2024-04-12T20:56:03.051 INFO:tasks.workunit.client.0.smithi018.stdout:8/329: chown db/d25/d26/d27/f54 76135249 1 2024-04-12T20:56:03.052 INFO:tasks.workunit.client.0.smithi018.stdout:8/330: chown f6 1 1 2024-04-12T20:56:03.053 INFO:tasks.workunit.client.0.smithi018.stdout:8/331: chown db/dc/d45/l62 26400 1 2024-04-12T20:56:03.055 INFO:tasks.workunit.client.0.smithi018.stdout:6/366: rmdir d5/d15 39 2024-04-12T20:56:03.056 INFO:tasks.workunit.client.0.smithi018.stdout:9/434: write d3/d6/d9/dc/d1c/d21/d41/d78/f7a [1461085,93723] 0 2024-04-12T20:56:03.063 INFO:tasks.workunit.client.0.smithi018.stdout:9/435: getdents d3/d6/d9/dc/d13/d5b/d7c 0 2024-04-12T20:56:03.065 INFO:tasks.workunit.client.0.smithi018.stdout:6/367: creat d5/df/f93 x:0 0 0 2024-04-12T20:56:03.066 INFO:tasks.workunit.client.0.smithi018.stdout:9/436: chown d3/d6/d9/dc/l39 3520739 1 2024-04-12T20:56:03.070 INFO:tasks.workunit.client.0.smithi018.stdout:1/216: dread d3/d9/da/d14/f38 [0,4194304] 0 2024-04-12T20:56:03.070 INFO:tasks.workunit.client.0.smithi018.stdout:6/368: creat d5/d6/d1b/d3b/f94 x:0 0 0 2024-04-12T20:56:03.076 INFO:tasks.workunit.client.0.smithi018.stdout:6/369: creat d5/df/d19/d2b/f95 x:0 0 0 2024-04-12T20:56:03.079 INFO:tasks.workunit.client.0.smithi018.stdout:8/332: dwrite db/d25/d26/f53 [0,4194304] 0 2024-04-12T20:56:03.080 INFO:tasks.workunit.client.0.smithi018.stdout:6/370: read d5/d6/d1b/d3b/d35/d4f/f5c [3270238,100378] 0 2024-04-12T20:56:03.084 INFO:tasks.workunit.client.0.smithi018.stdout:6/371: chown d5/df/d31/d4e/l79 30799 1 2024-04-12T20:56:03.086 INFO:tasks.workunit.client.0.smithi018.stdout:8/333: creat db/f68 x:0 0 0 2024-04-12T20:56:03.090 INFO:tasks.workunit.client.0.smithi018.stdout:8/334: stat db/dc/d45/l62 0 2024-04-12T20:56:03.090 INFO:tasks.workunit.client.0.smithi018.stdout:9/437: dwrite d3/d6/d9/f82 [0,4194304] 0 2024-04-12T20:56:03.091 INFO:tasks.workunit.client.0.smithi018.stdout:6/372: mknod d5/d6/d1b/d3b/d22/c96 0 2024-04-12T20:56:03.101 INFO:tasks.workunit.client.0.smithi018.stdout:9/438: rename d3/l15 to d3/d6/d9/dc/d13/l86 0 2024-04-12T20:56:03.101 INFO:tasks.workunit.client.0.smithi018.stdout:6/373: write d5/d15/d60/f6d [880846,18245] 0 2024-04-12T20:56:03.102 INFO:tasks.workunit.client.0.smithi018.stdout:6/374: fsync d5/d6/d1b/f61 0 2024-04-12T20:56:03.107 INFO:tasks.workunit.client.0.smithi018.stdout:9/439: creat d3/d6/d9/dc/d5a/d79/f87 x:0 0 0 2024-04-12T20:56:03.111 INFO:tasks.workunit.client.0.smithi018.stdout:6/375: read d5/d6/d1b/f61 [4781434,28752] 0 2024-04-12T20:56:03.115 INFO:tasks.workunit.client.0.smithi018.stdout:6/376: readlink d5/df/d19/d27/d2f/l4b 0 2024-04-12T20:56:03.116 INFO:tasks.workunit.client.0.smithi018.stdout:9/440: chown d3/d6/d9/dc/d13/d5b/c64 2946 1 2024-04-12T20:56:03.119 INFO:tasks.workunit.client.0.smithi018.stdout:6/377: mknod d5/d6/c97 0 2024-04-12T20:56:03.122 INFO:tasks.workunit.client.0.smithi018.stdout:1/217: dread d3/d9/da/dd/f12 [0,4194304] 0 2024-04-12T20:56:03.122 INFO:tasks.workunit.client.0.smithi018.stdout:9/441: write d3/d6/d9/dc/d1c/d21/f59 [3266227,128378] 0 2024-04-12T20:56:03.122 INFO:tasks.workunit.client.0.smithi018.stdout:6/378: symlink d5/d6/d1b/d8c/l98 0 2024-04-12T20:56:03.123 INFO:tasks.workunit.client.0.smithi018.stdout:8/335: dread f0 [0,4194304] 0 2024-04-12T20:56:03.125 INFO:tasks.workunit.client.0.smithi018.stdout:8/336: readlink db/dc/l38 0 2024-04-12T20:56:03.127 INFO:tasks.workunit.client.0.smithi018.stdout:1/218: readlink d3/d9/da/dd/d31/d32/d13/d19/l1e 0 2024-04-12T20:56:03.128 INFO:tasks.workunit.client.0.smithi018.stdout:8/337: read - db/dc/f5e zero size 2024-04-12T20:56:03.129 INFO:tasks.workunit.client.0.smithi018.stdout:1/219: write d3/f7 [2466360,68137] 0 2024-04-12T20:56:03.129 INFO:tasks.workunit.client.0.smithi018.stdout:8/338: readlink db/d25/l39 0 2024-04-12T20:56:03.130 INFO:tasks.workunit.client.0.smithi018.stdout:9/442: symlink d3/d6/d9/dc/d46/d5d/l88 0 2024-04-12T20:56:03.131 INFO:tasks.workunit.client.0.smithi018.stdout:6/379: getdents d5/d71 0 2024-04-12T20:56:03.131 INFO:tasks.workunit.client.0.smithi018.stdout:1/220: chown d3/f10 335642 1 2024-04-12T20:56:03.132 INFO:tasks.workunit.client.0.smithi018.stdout:6/380: fsync d5/f10 0 2024-04-12T20:56:03.134 INFO:tasks.workunit.client.0.smithi018.stdout:1/221: fsync d3/d9/da/dd/f3f 0 2024-04-12T20:56:03.136 INFO:tasks.workunit.client.0.smithi018.stdout:8/339: truncate db/d25/d26/d27/f50 513720 0 2024-04-12T20:56:03.138 INFO:tasks.workunit.client.0.smithi018.stdout:1/222: mknod d3/d9/da/dd/c49 0 2024-04-12T20:56:03.141 INFO:tasks.workunit.client.0.smithi018.stdout:6/381: creat d5/df/d31/d4e/f99 x:0 0 0 2024-04-12T20:56:03.141 INFO:tasks.workunit.client.0.smithi018.stdout:9/443: symlink d3/d6/d9/dc/d1c/l89 0 2024-04-12T20:56:03.147 INFO:tasks.workunit.client.0.smithi018.stdout:8/340: getdents db 0 2024-04-12T20:56:03.149 INFO:tasks.workunit.client.0.smithi018.stdout:8/341: rmdir db 39 2024-04-12T20:56:03.149 INFO:tasks.workunit.client.0.smithi018.stdout:6/382: link d5/df/f2a d5/d15/d60/f9a 0 2024-04-12T20:56:03.153 INFO:tasks.workunit.client.0.smithi018.stdout:1/223: dread d3/f7 [4194304,4194304] 0 2024-04-12T20:56:03.155 INFO:tasks.workunit.client.0.smithi018.stdout:6/383: unlink d5/df/l16 0 2024-04-12T20:56:03.157 INFO:tasks.workunit.client.0.smithi018.stdout:8/342: mkdir db/d25/d26/d57/d69 0 2024-04-12T20:56:03.157 INFO:tasks.workunit.client.0.smithi018.stdout:6/384: chown d5/df/d19/d27/d2f/l4b 3186 1 2024-04-12T20:56:03.158 INFO:tasks.workunit.client.0.smithi018.stdout:1/224: creat d3/d9/da/dd/f4a x:0 0 0 2024-04-12T20:56:03.159 INFO:tasks.workunit.client.0.smithi018.stdout:1/225: chown d3/f7 37748 1 2024-04-12T20:56:03.159 INFO:tasks.workunit.client.0.smithi018.stdout:8/343: creat db/d4e/f6a x:0 0 0 2024-04-12T20:56:03.161 INFO:tasks.workunit.client.0.smithi018.stdout:6/385: mkdir d5/df/d9b 0 2024-04-12T20:56:03.162 INFO:tasks.workunit.client.0.smithi018.stdout:4/271: sync 2024-04-12T20:56:03.162 INFO:tasks.workunit.client.0.smithi018.stdout:2/192: sync 2024-04-12T20:56:03.163 INFO:tasks.workunit.client.0.smithi018.stdout:8/344: write db/d25/d26/d27/f33 [2067070,115055] 0 2024-04-12T20:56:03.164 INFO:tasks.workunit.client.0.smithi018.stdout:1/226: mknod d3/d9/da/dd/d31/d32/d13/d19/d2d/c4b 0 2024-04-12T20:56:03.165 INFO:tasks.workunit.client.0.smithi018.stdout:8/345: readlink db/l1d 0 2024-04-12T20:56:03.169 INFO:tasks.workunit.client.0.smithi018.stdout:8/346: write db/d25/d26/f53 [2862762,93649] 0 2024-04-12T20:56:03.174 INFO:tasks.workunit.client.0.smithi018.stdout:9/444: dread d3/d6/d9/dc/d13/f42 [0,4194304] 0 2024-04-12T20:56:03.174 INFO:tasks.workunit.client.0.smithi018.stdout:4/272: mkdir d3/d34/d40/d48/d4f 0 2024-04-12T20:56:03.176 INFO:tasks.workunit.client.0.smithi018.stdout:8/347: write db/d25/d26/f53 [391665,99507] 0 2024-04-12T20:56:03.180 INFO:tasks.workunit.client.0.smithi018.stdout:8/348: readlink db/dc/ld 0 2024-04-12T20:56:03.182 INFO:tasks.workunit.client.0.smithi018.stdout:9/445: rmdir d3/d6/d9/dc/d46 39 2024-04-12T20:56:03.184 INFO:tasks.workunit.client.0.smithi018.stdout:8/349: fsync db/dc/f1a 0 2024-04-12T20:56:03.186 INFO:tasks.workunit.client.0.smithi018.stdout:9/446: read d3/d6/d9/f82 [446002,59800] 0 2024-04-12T20:56:03.186 INFO:tasks.workunit.client.0.smithi018.stdout:1/227: creat d3/d9/da/f4c x:0 0 0 2024-04-12T20:56:03.187 INFO:tasks.workunit.client.0.smithi018.stdout:6/386: dwrite d5/f10 [0,4194304] 0 2024-04-12T20:56:03.189 INFO:tasks.workunit.client.0.smithi018.stdout:4/273: creat d3/d34/d40/d48/f50 x:0 0 0 2024-04-12T20:56:03.193 INFO:tasks.workunit.client.0.smithi018.stdout:8/350: write db/dc/d45/f65 [654868,103487] 0 2024-04-12T20:56:03.194 INFO:tasks.workunit.client.0.smithi018.stdout:4/274: mknod d3/d34/d40/c51 0 2024-04-12T20:56:03.197 INFO:tasks.workunit.client.0.smithi018.stdout:1/228: creat d3/d9/da/dd/d31/d32/d13/d19/d2d/f4d x:0 0 0 2024-04-12T20:56:03.198 INFO:tasks.workunit.client.0.smithi018.stdout:8/351: write db/d25/d26/f5c [1147738,36661] 0 2024-04-12T20:56:03.199 INFO:tasks.workunit.client.0.smithi018.stdout:6/387: unlink d5/d6/d1b/d3b/d22/f46 0 2024-04-12T20:56:03.203 INFO:tasks.workunit.client.0.smithi018.stdout:8/352: truncate db/d4e/d56/f63 198094 0 2024-04-12T20:56:03.203 INFO:tasks.workunit.client.0.smithi018.stdout:2/193: dread d5/d7/d28/f2d [0,4194304] 0 2024-04-12T20:56:03.208 INFO:tasks.workunit.client.0.smithi018.stdout:4/275: link d3/d32/fd d3/f52 0 2024-04-12T20:56:03.209 INFO:tasks.workunit.client.0.smithi018.stdout:8/353: creat db/d4e/f6b x:0 0 0 2024-04-12T20:56:03.212 INFO:tasks.workunit.client.0.smithi018.stdout:9/447: write d3/d6/d9/d3f/f81 [418108,87483] 0 2024-04-12T20:56:03.213 INFO:tasks.workunit.client.0.smithi018.stdout:4/276: mkdir d3/d34/d53 0 2024-04-12T20:56:03.216 INFO:tasks.workunit.client.0.smithi018.stdout:9/448: rename d3/d6/d9/dc/d5a to d3/d6/d9/dc/d5a/d79/d8a 22 2024-04-12T20:56:03.217 INFO:tasks.workunit.client.0.smithi018.stdout:8/354: write db/d3b/f49 [373768,62069] 0 2024-04-12T20:56:03.221 INFO:tasks.workunit.client.0.smithi018.stdout:4/277: chown d3/d34/d43/f2f 1336 1 2024-04-12T20:56:03.227 INFO:tasks.workunit.client.0.smithi018.stdout:8/355: mkdir db/d25/d26/d57/d69/d6c 0 2024-04-12T20:56:03.229 INFO:tasks.workunit.client.0.smithi018.stdout:6/388: dwrite d5/df/d19/d74/f86 [0,4194304] 0 2024-04-12T20:56:03.233 INFO:tasks.workunit.client.0.smithi018.stdout:8/356: creat db/d4e/f6d x:0 0 0 2024-04-12T20:56:03.234 INFO:tasks.workunit.client.0.smithi018.stdout:6/389: chown d5/d15/f1c 3953 1 2024-04-12T20:56:03.243 INFO:tasks.workunit.client.0.smithi018.stdout:9/449: dwrite d3/f5e [0,4194304] 0 2024-04-12T20:56:03.243 INFO:tasks.workunit.client.0.smithi018.stdout:8/357: unlink db/d25/d26/f3a 0 2024-04-12T20:56:03.247 INFO:tasks.workunit.client.0.smithi018.stdout:4/278: dwrite d3/d32/d33/f29 [0,4194304] 0 2024-04-12T20:56:03.250 INFO:tasks.workunit.client.0.smithi018.stdout:8/358: creat db/d4e/d56/f6e x:0 0 0 2024-04-12T20:56:03.251 INFO:tasks.workunit.client.0.smithi018.stdout:4/279: symlink d3/d34/d40/l54 0 2024-04-12T20:56:03.253 INFO:tasks.workunit.client.0.smithi018.stdout:6/390: rename d5/d15/f1c to d5/df/d19/d2b/f9c 0 2024-04-12T20:56:03.254 INFO:tasks.workunit.client.0.smithi018.stdout:9/450: unlink d3/d6/d9/fa 0 2024-04-12T20:56:03.255 INFO:tasks.workunit.client.0.smithi018.stdout:2/194: dread d5/d8/d10/d19/d24/f32 [0,4194304] 0 2024-04-12T20:56:03.257 INFO:tasks.workunit.client.0.smithi018.stdout:4/280: chown d3/d34/d43/f2f 6 1 2024-04-12T20:56:03.259 INFO:tasks.workunit.client.0.smithi018.stdout:4/281: chown d3/d34/d40/d48 38 1 2024-04-12T20:56:03.260 INFO:tasks.workunit.client.0.smithi018.stdout:9/451: chown d3/d6/d43/l72 634221 1 2024-04-12T20:56:03.262 INFO:tasks.workunit.client.0.smithi018.stdout:6/391: symlink d5/d71/l9d 0 2024-04-12T20:56:03.263 INFO:tasks.workunit.client.0.smithi018.stdout:4/282: symlink d3/d32/d33/l55 0 2024-04-12T20:56:03.265 INFO:tasks.workunit.client.0.smithi018.stdout:9/452: mknod d3/d6/d3d/c8b 0 2024-04-12T20:56:03.266 INFO:tasks.workunit.client.0.smithi018.stdout:4/283: rmdir d3/d34/d43 39 2024-04-12T20:56:03.266 INFO:tasks.workunit.client.0.smithi018.stdout:6/392: write d5/d6/d1b/f61 [1576474,25690] 0 2024-04-12T20:56:03.271 INFO:tasks.workunit.client.0.smithi018.stdout:4/284: chown d3/d32/d33/l55 16342 1 2024-04-12T20:56:03.272 INFO:tasks.workunit.client.0.smithi018.stdout:9/453: rename d3/l4 to d3/d6/d9/dc/d6f/l8c 0 2024-04-12T20:56:03.277 INFO:tasks.workunit.client.0.smithi018.stdout:9/454: chown d3/d6/d9/dc/d1c/d21/d41/d78 0 1 2024-04-12T20:56:03.281 INFO:tasks.workunit.client.0.smithi018.stdout:9/455: write d3/f23 [1630975,32723] 0 2024-04-12T20:56:03.296 INFO:tasks.workunit.client.0.smithi018.stdout:2/195: dwrite d5/d7/d28/f2d [0,4194304] 0 2024-04-12T20:56:03.300 INFO:tasks.workunit.client.0.smithi018.stdout:4/285: rename d3/db/l1d to d3/d34/l56 0 2024-04-12T20:56:03.304 INFO:tasks.workunit.client.0.smithi018.stdout:4/286: chown d3/d34/d40/f44 1580292 1 2024-04-12T20:56:03.305 INFO:tasks.workunit.client.0.smithi018.stdout:9/456: rename d3/d6/d9/d1e/l3e to d3/d6/d9/dc/d13/l8d 0 2024-04-12T20:56:03.312 INFO:tasks.workunit.client.0.smithi018.stdout:8/359: truncate db/d25/d26/f53 3313789 0 2024-04-12T20:56:03.313 INFO:tasks.workunit.client.0.smithi018.stdout:9/457: chown d3/d6/d9/dc/d46/d5d/l88 118 1 2024-04-12T20:56:03.316 INFO:tasks.workunit.client.0.smithi018.stdout:9/458: symlink d3/d6/d9/dc/l8e 0 2024-04-12T20:56:03.319 INFO:tasks.workunit.client.0.smithi018.stdout:6/393: dread d5/df/d3f/f49 [0,4194304] 0 2024-04-12T20:56:03.322 INFO:tasks.workunit.client.0.smithi018.stdout:9/459: write d3/d6/d9/dc/d1c/f73 [660215,44397] 0 2024-04-12T20:56:03.327 INFO:tasks.workunit.client.0.smithi018.stdout:2/196: dread f3 [0,4194304] 0 2024-04-12T20:56:03.331 INFO:tasks.workunit.client.0.smithi018.stdout:9/460: truncate d3/d6/d9/d3f/f6e 819879 0 2024-04-12T20:56:03.336 INFO:tasks.workunit.client.0.smithi018.stdout:9/461: creat d3/d6/d43/f8f x:0 0 0 2024-04-12T20:56:03.336 INFO:tasks.workunit.client.0.smithi018.stdout:2/197: mkdir d5/d8/d10/d43 0 2024-04-12T20:56:03.338 INFO:tasks.workunit.client.0.smithi018.stdout:9/462: chown d3/f5e 74078 1 2024-04-12T20:56:03.339 INFO:tasks.workunit.client.0.smithi018.stdout:2/198: truncate d5/d7/f37 873961 0 2024-04-12T20:56:03.340 INFO:tasks.workunit.client.0.smithi018.stdout:4/287: truncate d3/db/f4c 1677421 0 2024-04-12T20:56:03.344 INFO:tasks.workunit.client.0.smithi018.stdout:9/463: truncate d3/d6/d9/dc/f22 916154 0 2024-04-12T20:56:03.345 INFO:tasks.workunit.client.0.smithi018.stdout:8/360: symlink db/d25/d26/d27/l6f 0 2024-04-12T20:56:03.349 INFO:tasks.workunit.client.0.smithi018.stdout:9/464: symlink d3/d6/d9/dc/d5a/l90 0 2024-04-12T20:56:03.353 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:56:03 smithi018 ceph-mon[28799]: pgmap v34: 65 pgs: 65 active+clean; 675 MiB data, 3.7 GiB used, 533 GiB / 536 GiB avail; 6.5 MiB/s rd, 76 MiB/s wr, 273 op/s 2024-04-12T20:56:03.353 INFO:tasks.workunit.client.0.smithi018.stdout:4/288: creat d3/d34/d3d/f57 x:0 0 0 2024-04-12T20:56:03.354 INFO:tasks.workunit.client.0.smithi018.stdout:8/361: rmdir db/d4e/d56 39 2024-04-12T20:56:03.356 INFO:tasks.workunit.client.0.smithi018.stdout:9/465: creat d3/d6/d9/dc/d1c/d21/d41/f91 x:0 0 0 2024-04-12T20:56:03.359 INFO:tasks.workunit.client.0.smithi018.stdout:4/289: symlink d3/d34/d3d/l58 0 2024-04-12T20:56:03.360 INFO:tasks.workunit.client.0.smithi018.stdout:9/466: truncate d3/d6/d9/dc/d5a/d79/f87 890629 0 2024-04-12T20:56:03.363 INFO:tasks.workunit.client.0.smithi018.stdout:9/467: write d3/d6/d9/d1e/f53 [1194224,122118] 0 2024-04-12T20:56:03.368 INFO:tasks.workunit.client.0.smithi018.stdout:8/362: unlink db/d25/f2c 0 2024-04-12T20:56:03.371 INFO:tasks.workunit.client.0.smithi018.stdout:8/363: readlink db/d25/l39 0 2024-04-12T20:56:03.375 INFO:tasks.workunit.client.0.smithi018.stdout:6/394: dread d5/f7 [0,4194304] 0 2024-04-12T20:56:03.378 INFO:tasks.workunit.client.0.smithi018.stdout:9/468: rename d3/d6/d9/dc/d13/d5b/d7c/f31 to d3/d6/d9/dc/d13/d5b/d6b/f92 0 2024-04-12T20:56:03.378 INFO:tasks.workunit.client.0.smithi018.stdout:4/290: mknod d3/d34/c59 0 2024-04-12T20:56:03.385 INFO:tasks.workunit.client.0.smithi018.stdout:2/199: dread d5/d8/d10/d19/d24/f29 [0,4194304] 0 2024-04-12T20:56:03.389 INFO:tasks.workunit.client.0.smithi018.stdout:9/469: mkdir d3/d93 0 2024-04-12T20:56:03.395 INFO:tasks.workunit.client.0.smithi018.stdout:6/395: dwrite d5/df/f87 [0,4194304] 0 2024-04-12T20:56:03.398 INFO:tasks.workunit.client.0.smithi018.stdout:4/291: dwrite d3/db/f23 [0,4194304] 0 2024-04-12T20:56:03.400 INFO:tasks.workunit.client.0.smithi018.stdout:5/287: sync 2024-04-12T20:56:03.406 INFO:tasks.workunit.client.0.smithi018.stdout:8/364: rename db/d25/d26/d27/c66 to db/d4e/c70 0 2024-04-12T20:56:03.408 INFO:tasks.workunit.client.0.smithi018.stdout:6/396: mknod d5/d6/c9e 0 2024-04-12T20:56:03.415 INFO:tasks.workunit.client.0.smithi018.stdout:8/365: mkdir db/d25/d26/d57/d71 0 2024-04-12T20:56:03.415 INFO:tasks.workunit.client.0.smithi018.stdout:4/292: getdents d3/db 0 2024-04-12T20:56:03.426 INFO:tasks.workunit.client.0.smithi018.stdout:4/293: dread - d3/d34/d40/d48/f50 zero size 2024-04-12T20:56:03.429 INFO:tasks.workunit.client.0.smithi018.stdout:8/366: mkdir db/d72 0 2024-04-12T20:56:03.433 INFO:tasks.workunit.client.0.smithi018.stdout:2/200: dwrite d5/d8/d10/f30 [4194304,4194304] 0 2024-04-12T20:56:03.440 INFO:tasks.workunit.client.0.smithi018.stdout:5/288: dwrite d0/d2b/f2d [0,4194304] 0 2024-04-12T20:56:03.442 INFO:tasks.workunit.client.0.smithi018.stdout:0/281: sync 2024-04-12T20:56:03.442 INFO:tasks.workunit.client.0.smithi018.stdout:3/278: sync 2024-04-12T20:56:03.442 INFO:tasks.workunit.client.0.smithi018.stdout:1/229: sync 2024-04-12T20:56:03.442 INFO:tasks.workunit.client.0.smithi018.stdout:7/299: sync 2024-04-12T20:56:03.445 INFO:tasks.workunit.client.0.smithi018.stdout:4/294: rename d3/db/f22 to d3/d34/d43/d4b/f5a 0 2024-04-12T20:56:03.451 INFO:tasks.workunit.client.0.smithi018.stdout:5/289: mkdir d0/d5b 0 2024-04-12T20:56:03.455 INFO:tasks.workunit.client.0.smithi018.stdout:1/230: mkdir d3/d9/da/dd/d31/d32/d13/d19/d2d/d4e 0 2024-04-12T20:56:03.463 INFO:tasks.workunit.client.0.smithi018.stdout:7/300: write d1/db/d31/d3a/f41 [3018692,41079] 0 2024-04-12T20:56:03.463 INFO:tasks.workunit.client.0.smithi018.stdout:9/470: dread d3/d6/d9/dc/d1c/f62 [0,4194304] 0 2024-04-12T20:56:03.464 INFO:tasks.workunit.client.0.smithi018.stdout:4/295: rmdir d3/d32/d33 39 2024-04-12T20:56:03.468 INFO:tasks.workunit.client.0.smithi018.stdout:3/279: link d2/d6/f2e d2/d6/d30/d31/d38/d3a/d48/f51 0 2024-04-12T20:56:03.476 INFO:tasks.workunit.client.0.smithi018.stdout:9/471: creat d3/d6/d9/dc/d5a/d79/d7e/f94 x:0 0 0 2024-04-12T20:56:03.477 INFO:tasks.workunit.client.0.smithi018.stdout:2/201: link d5/d8/d10/c23 d5/d7/d28/d3c/c44 0 2024-04-12T20:56:03.478 INFO:tasks.workunit.client.0.smithi018.stdout:5/290: creat d0/d5b/f5c x:0 0 0 2024-04-12T20:56:03.484 INFO:tasks.workunit.client.0.smithi018.stdout:8/367: unlink db/d25/d26/d27/f2b 0 2024-04-12T20:56:03.486 INFO:tasks.workunit.client.0.smithi018.stdout:9/472: read d3/d6/d9/d1e/f53 [1307904,14113] 0 2024-04-12T20:56:03.487 INFO:tasks.workunit.client.0.smithi018.stdout:7/301: dwrite d1/d6/f52 [0,4194304] 0 2024-04-12T20:56:03.490 INFO:tasks.workunit.client.0.smithi018.stdout:5/291: creat d0/d5b/f5d x:0 0 0 2024-04-12T20:56:03.494 INFO:tasks.workunit.client.0.smithi018.stdout:0/282: dwrite da/d15/d17/f19 [0,4194304] 0 2024-04-12T20:56:03.498 INFO:tasks.workunit.client.0.smithi018.stdout:4/296: dread d3/d34/d43/f2e [0,4194304] 0 2024-04-12T20:56:03.505 INFO:tasks.workunit.client.0.smithi018.stdout:6/397: dwrite d5/d6/d1b/d3b/d35/d4f/f5b [0,4194304] 0 2024-04-12T20:56:03.506 INFO:tasks.workunit.client.0.smithi018.stdout:2/202: creat d5/d8/d10/d19/d24/d39/f45 x:0 0 0 2024-04-12T20:56:03.517 INFO:tasks.workunit.client.0.smithi018.stdout:2/203: symlink d5/d7/d28/l46 0 2024-04-12T20:56:03.521 INFO:tasks.workunit.client.0.smithi018.stdout:4/297: mknod d3/d32/c5b 0 2024-04-12T20:56:03.526 INFO:tasks.workunit.client.0.smithi018.stdout:8/368: mkdir db/d4e/d73 0 2024-04-12T20:56:03.530 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:56:03 smithi186 ceph-mon[36597]: pgmap v34: 65 pgs: 65 active+clean; 675 MiB data, 3.7 GiB used, 533 GiB / 536 GiB avail; 6.5 MiB/s rd, 76 MiB/s wr, 273 op/s 2024-04-12T20:56:03.532 INFO:tasks.workunit.client.0.smithi018.stdout:8/369: stat db/d25/d26/d57/d71 0 2024-04-12T20:56:03.536 INFO:tasks.workunit.client.0.smithi018.stdout:7/302: dwrite d1/db/d31/d30/f4a [4194304,4194304] 0 2024-04-12T20:56:03.537 INFO:tasks.workunit.client.0.smithi018.stdout:4/298: mknod d3/d32/c5c 0 2024-04-12T20:56:03.541 INFO:tasks.workunit.client.0.smithi018.stdout:7/303: chown d1/db/d31/l49 1774 1 2024-04-12T20:56:03.541 INFO:tasks.workunit.client.0.smithi018.stdout:0/283: dwrite da/d15/d17/d21/d27/f4f [0,4194304] 0 2024-04-12T20:56:03.547 INFO:tasks.workunit.client.0.smithi018.stdout:0/284: rmdir da/d15 39 2024-04-12T20:56:03.549 INFO:tasks.workunit.client.0.smithi018.stdout:6/398: write d5/d15/d60/f9a [246838,123644] 0 2024-04-12T20:56:03.550 INFO:tasks.workunit.client.0.smithi018.stdout:2/204: rmdir d5/d8/d10/d19/d24/d39 39 2024-04-12T20:56:03.553 INFO:tasks.workunit.client.0.smithi018.stdout:7/304: creat d1/d6/d8/f68 x:0 0 0 2024-04-12T20:56:03.554 INFO:tasks.workunit.client.0.smithi018.stdout:4/299: mkdir d3/d5d 0 2024-04-12T20:56:03.560 INFO:tasks.workunit.client.0.smithi018.stdout:0/285: truncate da/f13 4998839 0 2024-04-12T20:56:03.564 INFO:tasks.workunit.client.0.smithi018.stdout:7/305: mkdir d1/db/d31/d3a/d4f/d69 0 2024-04-12T20:56:03.565 INFO:tasks.workunit.client.0.smithi018.stdout:9/473: dwrite d3/d6/d9/d1e/f20 [0,4194304] 0 2024-04-12T20:56:03.572 INFO:tasks.workunit.client.0.smithi018.stdout:2/205: symlink d5/d8/d33/l47 0 2024-04-12T20:56:03.573 INFO:tasks.workunit.client.0.smithi018.stdout:4/300: creat d3/d34/d53/f5e x:0 0 0 2024-04-12T20:56:03.576 INFO:tasks.workunit.client.0.smithi018.stdout:8/370: dwrite db/d25/f52 [0,4194304] 0 2024-04-12T20:56:03.576 INFO:tasks.workunit.client.0.smithi018.stdout:9/474: creat d3/d6/d9/dc/d1c/d21/d41/d78/f95 x:0 0 0 2024-04-12T20:56:03.577 INFO:tasks.workunit.client.0.smithi018.stdout:4/301: dread - d3/d34/d53/f5e zero size 2024-04-12T20:56:03.581 INFO:tasks.workunit.client.0.smithi018.stdout:7/306: mknod d1/db/d31/c6a 0 2024-04-12T20:56:03.587 INFO:tasks.workunit.client.0.smithi018.stdout:6/399: dwrite d5/d6/d1b/d3b/d35/d4f/f5c [0,4194304] 0 2024-04-12T20:56:03.594 INFO:tasks.workunit.client.0.smithi018.stdout:6/400: creat d5/df/d19/d74/f9f x:0 0 0 2024-04-12T20:56:03.595 INFO:tasks.workunit.client.0.smithi018.stdout:7/307: chown d1/d6/c46 6820906 1 2024-04-12T20:56:03.599 INFO:tasks.workunit.client.0.smithi018.stdout:9/475: dwrite d3/f80 [0,4194304] 0 2024-04-12T20:56:03.601 INFO:tasks.workunit.client.0.smithi018.stdout:4/302: mknod d3/d34/c5f 0 2024-04-12T20:56:03.602 INFO:tasks.workunit.client.0.smithi018.stdout:2/206: link d5/d7/d28/f2d d5/d8/d10/f48 0 2024-04-12T20:56:03.605 INFO:tasks.workunit.client.0.smithi018.stdout:9/476: readlink d3/d6/d9/dc/d1c/l89 0 2024-04-12T20:56:03.606 INFO:tasks.workunit.client.0.smithi018.stdout:7/308: chown d1/f2 46 1 2024-04-12T20:56:03.617 INFO:tasks.workunit.client.0.smithi018.stdout:4/303: fsync d3/db/f2b 0 2024-04-12T20:56:03.621 INFO:tasks.workunit.client.0.smithi018.stdout:2/207: rename d5/d8/d10/l1f to d5/d8/d10/d19/d24/d39/l49 0 2024-04-12T20:56:03.624 INFO:tasks.workunit.client.0.smithi018.stdout:9/477: write d3/d6/f1f [646192,101945] 0 2024-04-12T20:56:03.629 INFO:tasks.workunit.client.0.smithi018.stdout:4/304: mkdir d3/d5d/d60 0 2024-04-12T20:56:03.630 INFO:tasks.workunit.client.0.smithi018.stdout:9/478: creat d3/d6/d9/d1e/f96 x:0 0 0 2024-04-12T20:56:03.631 INFO:tasks.workunit.client.0.smithi018.stdout:8/371: dwrite db/d3b/f4d [0,4194304] 0 2024-04-12T20:56:03.632 INFO:tasks.workunit.client.0.smithi018.stdout:2/208: fsync d5/d8/d10/d19/d24/f32 0 2024-04-12T20:56:03.635 INFO:tasks.workunit.client.0.smithi018.stdout:6/401: truncate d5/d6/d1b/d3b/f3d 1713306 0 2024-04-12T20:56:03.636 INFO:tasks.workunit.client.0.smithi018.stdout:7/309: dwrite d1/db/d31/d3a/d42/d4c/f51 [0,4194304] 0 2024-04-12T20:56:03.641 INFO:tasks.workunit.client.0.smithi018.stdout:9/479: creat d3/d6/d9/dc/f97 x:0 0 0 2024-04-12T20:56:03.643 INFO:tasks.workunit.client.0.smithi018.stdout:6/402: truncate d5/df/d31/d4e/f38 1002180 0 2024-04-12T20:56:03.645 INFO:tasks.workunit.client.0.smithi018.stdout:8/372: truncate db/f64 161194 0 2024-04-12T20:56:03.648 INFO:tasks.workunit.client.0.smithi018.stdout:9/480: readlink d3/d6/d9/dc/d13/l86 0 2024-04-12T20:56:03.650 INFO:tasks.workunit.client.0.smithi018.stdout:7/310: symlink d1/db/l6b 0 2024-04-12T20:56:03.651 INFO:tasks.workunit.client.0.smithi018.stdout:9/481: fsync d3/d6/d9/dc/d1c/f73 0 2024-04-12T20:56:03.652 INFO:tasks.workunit.client.0.smithi018.stdout:2/209: mknod d5/d8/d10/d19/d24/c4a 0 2024-04-12T20:56:03.653 INFO:tasks.workunit.client.0.smithi018.stdout:7/311: chown d1/db/d31/f63 16 1 2024-04-12T20:56:03.657 INFO:tasks.workunit.client.0.smithi018.stdout:9/482: write d3/d6/d9/dc/d1c/f2e [166287,48866] 0 2024-04-12T20:56:03.661 INFO:tasks.workunit.client.0.smithi018.stdout:9/483: unlink d3/d6/f1f 0 2024-04-12T20:56:03.666 INFO:tasks.workunit.client.0.smithi018.stdout:7/312: chown d1/d6/l1c 217102050 1 2024-04-12T20:56:03.670 INFO:tasks.workunit.client.0.smithi018.stdout:7/313: readlink d1/db/d31/d3a/d42/d4c/l5d 0 2024-04-12T20:56:03.672 INFO:tasks.workunit.client.0.smithi018.stdout:6/403: write d5/d6/d1b/d3b/f69 [296201,41881] 0 2024-04-12T20:56:03.672 INFO:tasks.workunit.client.0.smithi018.stdout:7/314: chown d1/d6/d8 105579201 1 2024-04-12T20:56:03.678 INFO:tasks.workunit.client.0.smithi018.stdout:2/210: fsync d5/d7/d28/f2d 0 2024-04-12T20:56:03.680 INFO:tasks.workunit.client.0.smithi018.stdout:9/484: dread d3/d6/d1b/f35 [0,4194304] 0 2024-04-12T20:56:03.683 INFO:tasks.workunit.client.0.smithi018.stdout:8/373: write db/d25/d26/d27/f5f [1417684,11292] 0 2024-04-12T20:56:03.685 INFO:tasks.workunit.client.0.smithi018.stdout:7/315: mkdir d1/db/d31/d3a/d42/d4c/d6c 0 2024-04-12T20:56:03.685 INFO:tasks.workunit.client.0.smithi018.stdout:2/211: dread - d5/d8/d10/d19/d24/d39/f45 zero size 2024-04-12T20:56:03.689 INFO:tasks.workunit.client.0.smithi018.stdout:4/305: dread d3/d34/d43/f2f [0,4194304] 0 2024-04-12T20:56:03.690 INFO:tasks.workunit.client.0.smithi018.stdout:6/404: dread d5/df/d31/d4e/f38 [0,4194304] 0 2024-04-12T20:56:03.699 INFO:tasks.workunit.client.0.smithi018.stdout:8/374: mknod db/dc/d45/c74 0 2024-04-12T20:56:03.702 INFO:tasks.workunit.client.0.smithi018.stdout:8/375: chown db/d25/d26/d27/l6f 5828 1 2024-04-12T20:56:03.706 INFO:tasks.workunit.client.0.smithi018.stdout:6/405: rmdir d5/df/d31/d4e 39 2024-04-12T20:56:03.706 INFO:tasks.workunit.client.0.smithi018.stdout:9/485: link d3/d6/d9/dc/d46/f65 d3/d6/d9/dc/d1c/d21/d41/d78/f98 0 2024-04-12T20:56:03.711 INFO:tasks.workunit.client.0.smithi018.stdout:9/486: symlink d3/d6/d9/l99 0 2024-04-12T20:56:03.711 INFO:tasks.workunit.client.0.smithi018.stdout:2/212: creat d5/d8/d10/d19/d24/f4b x:0 0 0 2024-04-12T20:56:03.711 INFO:tasks.workunit.client.0.smithi018.stdout:7/316: truncate d1/d6/d8/f13 2198363 0 2024-04-12T20:56:03.712 INFO:tasks.workunit.client.0.smithi018.stdout:8/376: fdatasync db/f1b 0 2024-04-12T20:56:03.715 INFO:tasks.workunit.client.0.smithi018.stdout:8/377: read - db/d4e/f4f zero size 2024-04-12T20:56:03.716 INFO:tasks.workunit.client.0.smithi018.stdout:7/317: dread - d1/db/d31/d3a/d42/d4c/f5b zero size 2024-04-12T20:56:03.716 INFO:tasks.workunit.client.0.smithi018.stdout:8/378: stat db 0 2024-04-12T20:56:03.720 INFO:tasks.workunit.client.0.smithi018.stdout:9/487: getdents d3/d6/d9/dc/d46 0 2024-04-12T20:56:03.721 INFO:tasks.workunit.client.0.smithi018.stdout:9/488: fsync d3/d6/d43/f8f 0 2024-04-12T20:56:03.729 INFO:tasks.workunit.client.0.smithi018.stdout:7/318: mknod d1/db/d31/d30/c6d 0 2024-04-12T20:56:03.733 INFO:tasks.workunit.client.0.smithi018.stdout:8/379: mknod db/d4e/d56/d59/c75 0 2024-04-12T20:56:03.738 INFO:tasks.workunit.client.0.smithi018.stdout:9/489: rmdir d3/d6/d9/dc/d1c 39 2024-04-12T20:56:03.740 INFO:tasks.workunit.client.0.smithi018.stdout:8/380: fsync db/d25/d26/d27/f54 0 2024-04-12T20:56:03.740 INFO:tasks.workunit.client.0.smithi018.stdout:4/306: dwrite d3/d34/d43/f2e [0,4194304] 0 2024-04-12T20:56:03.743 INFO:tasks.workunit.client.0.smithi018.stdout:4/307: chown d3/f3b 33 1 2024-04-12T20:56:03.744 INFO:tasks.workunit.client.0.smithi018.stdout:8/381: dread - db/d4e/f4f zero size 2024-04-12T20:56:03.744 INFO:tasks.workunit.client.0.smithi018.stdout:4/308: readlink d3/l20 0 2024-04-12T20:56:03.744 INFO:tasks.workunit.client.0.smithi018.stdout:2/213: dwrite d5/d7/f35 [0,4194304] 0 2024-04-12T20:56:03.747 INFO:tasks.workunit.client.0.smithi018.stdout:9/490: chown d3/d6/d9/dc/d5a/d79/f87 6 1 2024-04-12T20:56:03.752 INFO:tasks.workunit.client.0.smithi018.stdout:2/214: mknod d5/c4c 0 2024-04-12T20:56:03.754 INFO:tasks.workunit.client.0.smithi018.stdout:8/382: dread db/f64 [0,4194304] 0 2024-04-12T20:56:03.761 INFO:tasks.workunit.client.0.smithi018.stdout:4/309: creat d3/d34/d40/d48/d4f/f61 x:0 0 0 2024-04-12T20:56:03.763 INFO:tasks.workunit.client.0.smithi018.stdout:7/319: dwrite d1/db/f43 [0,4194304] 0 2024-04-12T20:56:03.777 INFO:tasks.workunit.client.0.smithi018.stdout:9/491: truncate d3/d6/d9/d1e/f20 3381313 0 2024-04-12T20:56:03.780 INFO:tasks.workunit.client.0.smithi018.stdout:4/310: creat d3/d34/d43/f62 x:0 0 0 2024-04-12T20:56:03.782 INFO:tasks.workunit.client.0.smithi018.stdout:9/492: stat d3/d6/d43/f8f 0 2024-04-12T20:56:03.782 INFO:tasks.workunit.client.0.smithi018.stdout:8/383: dwrite f6 [4194304,4194304] 0 2024-04-12T20:56:03.786 INFO:tasks.workunit.client.0.smithi018.stdout:8/384: fdatasync db/dc/d45/f65 0 2024-04-12T20:56:03.787 INFO:tasks.workunit.client.0.smithi018.stdout:2/215: dwrite d5/d8/d10/d19/d24/d39/f45 [0,4194304] 0 2024-04-12T20:56:03.790 INFO:tasks.workunit.client.0.smithi018.stdout:8/385: write db/dc/d45/f65 [5080669,19172] 0 2024-04-12T20:56:03.791 INFO:tasks.workunit.client.0.smithi018.stdout:9/493: creat d3/d6/d9/dc/d6f/f9a x:0 0 0 2024-04-12T20:56:03.795 INFO:tasks.workunit.client.0.smithi018.stdout:2/216: creat d5/d7/f4d x:0 0 0 2024-04-12T20:56:03.796 INFO:tasks.workunit.client.0.smithi018.stdout:8/386: mkdir db/d4e/d56/d59/d76 0 2024-04-12T20:56:03.801 INFO:tasks.workunit.client.0.smithi018.stdout:2/217: rename d5/d7/c1b to d5/d7/d31/c4e 0 2024-04-12T20:56:03.806 INFO:tasks.workunit.client.0.smithi018.stdout:9/494: dread d3/d6/d9/dc/f2d [0,4194304] 0 2024-04-12T20:56:03.806 INFO:tasks.workunit.client.0.smithi018.stdout:4/311: dwrite d3/d34/d43/d4b/f5a [0,4194304] 0 2024-04-12T20:56:03.806 INFO:tasks.workunit.client.0.smithi018.stdout:2/218: rename d5/d7/d28/f2d to d5/d7/d31/f4f 0 2024-04-12T20:56:03.808 INFO:tasks.workunit.client.0.smithi018.stdout:9/495: write d3/f5e [2541764,52577] 0 2024-04-12T20:56:03.810 INFO:tasks.workunit.client.0.smithi018.stdout:9/496: fdatasync d3/d6/f3c 0 2024-04-12T20:56:03.816 INFO:tasks.workunit.client.0.smithi018.stdout:9/497: unlink d3/d6/d9/dc/d1c/f70 0 2024-04-12T20:56:03.819 INFO:tasks.workunit.client.0.smithi018.stdout:4/312: dread d3/d34/d43/f2f [0,4194304] 0 2024-04-12T20:56:03.820 INFO:tasks.workunit.client.0.smithi018.stdout:2/219: dread d5/d8/d10/d19/d24/f29 [0,4194304] 0 2024-04-12T20:56:03.823 INFO:tasks.workunit.client.0.smithi018.stdout:4/313: symlink d3/d34/d40/d48/d4f/l63 0 2024-04-12T20:56:03.831 INFO:tasks.workunit.client.0.smithi018.stdout:2/220: dwrite d5/d8/f1e [0,4194304] 0 2024-04-12T20:56:03.844 INFO:tasks.workunit.client.0.smithi018.stdout:2/221: dread d5/d8/d10/d19/d24/d39/f45 [0,4194304] 0 2024-04-12T20:56:03.846 INFO:tasks.workunit.client.0.smithi018.stdout:2/222: chown d5/d8/d10/f15 3 1 2024-04-12T20:56:03.849 INFO:tasks.workunit.client.0.smithi018.stdout:2/223: creat d5/d7/d28/d3c/d3e/f50 x:0 0 0 2024-04-12T20:56:03.851 INFO:tasks.workunit.client.0.smithi018.stdout:2/224: unlink f3 0 2024-04-12T20:56:03.855 INFO:tasks.workunit.client.0.smithi018.stdout:2/225: creat d5/d8/d10/d19/d24/d39/f51 x:0 0 0 2024-04-12T20:56:03.855 INFO:tasks.workunit.client.0.smithi018.stdout:2/226: chown d5/d8/d10/d19 10756 1 2024-04-12T20:56:03.857 INFO:tasks.workunit.client.0.smithi018.stdout:2/227: creat d5/d7/d28/d3c/f52 x:0 0 0 2024-04-12T20:56:03.863 INFO:tasks.workunit.client.0.smithi018.stdout:2/228: readlink d5/d8/d10/d19/d24/d39/l49 0 2024-04-12T20:56:03.876 INFO:tasks.workunit.client.0.smithi018.stdout:2/229: dwrite d5/d8/d10/d19/d24/f36 [0,4194304] 0 2024-04-12T20:56:03.880 INFO:tasks.workunit.client.0.smithi018.stdout:2/230: link f4 d5/d7/d28/d3c/f53 0 2024-04-12T20:56:03.957 INFO:tasks.workunit.client.0.smithi018.stdout:7/320: dread d1/f1b [0,4194304] 0 2024-04-12T20:56:03.958 INFO:tasks.workunit.client.0.smithi018.stdout:8/387: truncate f6 5493467 0 2024-04-12T20:56:03.963 INFO:tasks.workunit.client.0.smithi018.stdout:9/498: read d3/d6/d9/d3f/f50 [3767550,30858] 0 2024-04-12T20:56:03.963 INFO:tasks.workunit.client.0.smithi018.stdout:4/314: rename d3/d34/d40/d48 to d3/d5d/d60/d64 0 2024-04-12T20:56:03.966 INFO:tasks.workunit.client.0.smithi018.stdout:4/315: write d3/db/f2b [109258,6766] 0 2024-04-12T20:56:03.967 INFO:tasks.workunit.client.0.smithi018.stdout:9/499: write d3/d6/d9/dc/d1c/f73 [1416924,65644] 0 2024-04-12T20:56:03.969 INFO:tasks.workunit.client.0.smithi018.stdout:8/388: dread db/d25/d26/f3f [0,4194304] 0 2024-04-12T20:56:03.969 INFO:tasks.workunit.client.0.smithi018.stdout:9/500: stat d3/d6/d43/f7f 0 2024-04-12T20:56:03.975 INFO:tasks.workunit.client.0.smithi018.stdout:8/389: dread db/f64 [0,4194304] 0 2024-04-12T20:56:03.987 INFO:tasks.workunit.client.0.smithi018.stdout:2/231: rename d5/d7/d28/d3c to d5/d12/d54 0 2024-04-12T20:56:03.990 INFO:tasks.workunit.client.0.smithi018.stdout:2/232: write d5/d12/d54/d3e/f50 [157829,52969] 0 2024-04-12T20:56:03.995 INFO:tasks.workunit.client.0.smithi018.stdout:7/321: rename d1/db/d31/d3a/d42/f44 to d1/f6e 0 2024-04-12T20:56:03.999 INFO:tasks.workunit.client.0.smithi018.stdout:4/316: dwrite d3/db/f4c [0,4194304] 0 2024-04-12T20:56:04.000 INFO:tasks.workunit.client.0.smithi018.stdout:7/322: fsync d1/d6/f5a 0 2024-04-12T20:56:04.000 INFO:tasks.workunit.client.0.smithi018.stdout:2/233: fsync d5/d8/d25/f34 0 2024-04-12T20:56:04.009 INFO:tasks.workunit.client.0.smithi018.stdout:4/317: dread d3/d34/d43/f2a [0,4194304] 0 2024-04-12T20:56:04.012 INFO:tasks.workunit.client.0.smithi018.stdout:7/323: mkdir d1/db/d6f 0 2024-04-12T20:56:04.015 INFO:tasks.workunit.client.0.smithi018.stdout:4/318: readlink d3/d32/d33/l55 0 2024-04-12T20:56:04.015 INFO:tasks.workunit.client.0.smithi018.stdout:9/501: rename d3/d6/d9/dc/d1c to d3/d6/d9/dc/d5a/d79/d7e/d9b 0 2024-04-12T20:56:04.018 INFO:tasks.workunit.client.0.smithi018.stdout:2/234: getdents d5/d12/d54/d3e 0 2024-04-12T20:56:04.021 INFO:tasks.workunit.client.0.smithi018.stdout:8/390: rename db/d3b/f47 to db/d25/d26/d57/d71/f77 0 2024-04-12T20:56:04.022 INFO:tasks.workunit.client.0.smithi018.stdout:9/502: truncate d3/d6/d9/d3f/f81 977489 0 2024-04-12T20:56:04.023 INFO:tasks.workunit.client.0.smithi018.stdout:2/235: mknod d5/d8/d10/d19/c55 0 2024-04-12T20:56:04.024 INFO:tasks.workunit.client.0.smithi018.stdout:8/391: rename db/d4e to db/d4e/d56/d59/d76/d78 22 2024-04-12T20:56:04.025 INFO:tasks.workunit.client.0.smithi018.stdout:7/324: read d1/f2f [182280,94574] 0 2024-04-12T20:56:04.026 INFO:tasks.workunit.client.0.smithi018.stdout:2/236: dread - d5/d12/d54/f52 zero size 2024-04-12T20:56:04.027 INFO:tasks.workunit.client.0.smithi018.stdout:7/325: stat d1/d6/d8/f4 0 2024-04-12T20:56:04.029 INFO:tasks.workunit.client.0.smithi018.stdout:8/392: read db/dc/d45/f65 [1019119,89179] 0 2024-04-12T20:56:04.030 INFO:tasks.workunit.client.0.smithi018.stdout:9/503: dread d3/d6/d1b/f26 [0,4194304] 0 2024-04-12T20:56:04.032 INFO:tasks.workunit.client.0.smithi018.stdout:2/237: creat d5/d8/d10/d43/f56 x:0 0 0 2024-04-12T20:56:04.034 INFO:tasks.workunit.client.0.smithi018.stdout:7/326: mkdir d1/db/d31/d3a/d4f/d70 0 2024-04-12T20:56:04.035 INFO:tasks.workunit.client.0.smithi018.stdout:8/393: chown db/d25/d26/c3e 25271 1 2024-04-12T20:56:04.035 INFO:tasks.workunit.client.0.smithi018.stdout:2/238: write d5/d7/f4d [292043,58426] 0 2024-04-12T20:56:04.037 INFO:tasks.workunit.client.0.smithi018.stdout:9/504: symlink d3/d6/d9/d3f/l9c 0 2024-04-12T20:56:04.040 INFO:tasks.workunit.client.0.smithi018.stdout:2/239: creat d5/d8/f57 x:0 0 0 2024-04-12T20:56:04.043 INFO:tasks.workunit.client.0.smithi018.stdout:8/394: truncate db/d25/d26/d27/f5f 2473663 0 2024-04-12T20:56:04.043 INFO:tasks.workunit.client.0.smithi018.stdout:2/240: rmdir d5/d12/d54 39 2024-04-12T20:56:04.045 INFO:tasks.workunit.client.0.smithi018.stdout:7/327: rename d1/db/d31/d3a/d42/d4c/l5d to d1/db/d31/d3a/d4f/d70/l71 0 2024-04-12T20:56:04.047 INFO:tasks.workunit.client.0.smithi018.stdout:8/395: creat db/dc/d45/f79 x:0 0 0 2024-04-12T20:56:04.048 INFO:tasks.workunit.client.0.smithi018.stdout:9/505: creat d3/d6/d9/dc/d6f/d77/f9d x:0 0 0 2024-04-12T20:56:04.051 INFO:tasks.workunit.client.0.smithi018.stdout:8/396: chown db/d3b/c41 5521384 1 2024-04-12T20:56:04.054 INFO:tasks.workunit.client.0.smithi018.stdout:4/319: dread d3/d34/d43/f2c [0,4194304] 0 2024-04-12T20:56:04.058 INFO:tasks.workunit.client.0.smithi018.stdout:4/320: creat d3/d34/d3d/f65 x:0 0 0 2024-04-12T20:56:04.062 INFO:tasks.workunit.client.0.smithi018.stdout:4/321: mkdir d3/d32/d66 0 2024-04-12T20:56:04.064 INFO:tasks.workunit.client.0.smithi018.stdout:9/506: rename d3/d6/d9/dc/d5a/d79/d7e/d9b/l89 to d3/d6/d9/dc/d6f/d77/l9e 0 2024-04-12T20:56:04.068 INFO:tasks.workunit.client.0.smithi018.stdout:8/397: dwrite db/d4e/f6b [0,4194304] 0 2024-04-12T20:56:04.076 INFO:tasks.workunit.client.0.smithi018.stdout:1/231: sync 2024-04-12T20:56:04.076 INFO:tasks.workunit.client.0.smithi018.stdout:9/507: rename d3/d6/d9/dc/d5a/d79/d7e/d9b to d3/d6/d9/dc/d5a/d79/d7e/d9b/d9f 22 2024-04-12T20:56:04.080 INFO:tasks.workunit.client.0.smithi018.stdout:9/508: chown d3/l4d 542158288 1 2024-04-12T20:56:04.081 INFO:tasks.workunit.client.0.smithi018.stdout:8/398: symlink db/d4e/d56/d59/d76/l7a 0 2024-04-12T20:56:04.081 INFO:tasks.workunit.client.0.smithi018.stdout:2/241: dwrite d5/d12/d54/f52 [0,4194304] 0 2024-04-12T20:56:04.082 INFO:tasks.workunit.client.0.smithi018.stdout:1/232: chown d3/d9/da/dd/d31/d32/d13 2 1 2024-04-12T20:56:04.086 INFO:tasks.workunit.client.0.smithi018.stdout:4/322: dread d3/d34/d43/f2e [0,4194304] 0 2024-04-12T20:56:04.087 INFO:tasks.workunit.client.0.smithi018.stdout:7/328: dread d1/f6e [0,4194304] 0 2024-04-12T20:56:04.089 INFO:tasks.workunit.client.0.smithi018.stdout:4/323: dread - d3/d34/d43/f62 zero size 2024-04-12T20:56:04.090 INFO:tasks.workunit.client.0.smithi018.stdout:2/242: chown d5/d8/d10/d19/d24/d39/f51 7 1 2024-04-12T20:56:04.094 INFO:tasks.workunit.client.0.smithi018.stdout:8/399: mkdir db/d25/d26/d57/d69/d6c/d7b 0 2024-04-12T20:56:04.097 INFO:tasks.workunit.client.0.smithi018.stdout:4/324: creat d3/d5d/d60/f67 x:0 0 0 2024-04-12T20:56:04.105 INFO:tasks.workunit.client.0.smithi018.stdout:1/233: creat d3/d9/da/dd/d31/d32/d13/d19/d2d/d4e/f4f x:0 0 0 2024-04-12T20:56:04.106 INFO:tasks.workunit.client.0.smithi018.stdout:9/509: dwrite d3/d6/d9/d1e/f53 [0,4194304] 0 2024-04-12T20:56:04.112 INFO:tasks.workunit.client.0.smithi018.stdout:2/243: creat d5/d12/f58 x:0 0 0 2024-04-12T20:56:04.113 INFO:tasks.workunit.client.0.smithi018.stdout:9/510: mknod d3/d6/d9/dc/d13/d5b/ca0 0 2024-04-12T20:56:04.115 INFO:tasks.workunit.client.0.smithi018.stdout:8/400: getdents db/d72 0 2024-04-12T20:56:04.125 INFO:tasks.workunit.client.0.smithi018.stdout:8/401: write db/d4e/d56/d59/f67 [362627,67236] 0 2024-04-12T20:56:04.125 INFO:tasks.workunit.client.0.smithi018.stdout:7/329: dwrite d1/db/d31/d30/f4a [4194304,4194304] 0 2024-04-12T20:56:04.125 INFO:tasks.workunit.client.0.smithi018.stdout:9/511: write d3/d6/d9/dc/d5a/d79/d7e/d9b/d21/d41/f76 [511428,11054] 0 2024-04-12T20:56:04.125 INFO:tasks.workunit.client.0.smithi018.stdout:4/325: creat d3/d34/d43/f68 x:0 0 0 2024-04-12T20:56:04.126 INFO:tasks.workunit.client.0.smithi018.stdout:6/406: sync 2024-04-12T20:56:04.127 INFO:tasks.workunit.client.0.smithi018.stdout:0/286: sync 2024-04-12T20:56:04.127 INFO:tasks.workunit.client.0.smithi018.stdout:3/280: sync 2024-04-12T20:56:04.127 INFO:tasks.workunit.client.0.smithi018.stdout:5/292: sync 2024-04-12T20:56:04.128 INFO:tasks.workunit.client.0.smithi018.stdout:1/234: link c0 d3/d9/c50 0 2024-04-12T20:56:04.131 INFO:tasks.workunit.client.0.smithi018.stdout:9/512: read d3/d6/d9/d1e/f53 [844701,84181] 0 2024-04-12T20:56:04.135 INFO:tasks.workunit.client.0.smithi018.stdout:8/402: fdatasync db/dc/f40 0 2024-04-12T20:56:04.136 INFO:tasks.workunit.client.0.smithi018.stdout:3/281: read d2/f28 [3525734,69468] 0 2024-04-12T20:56:04.140 INFO:tasks.workunit.client.0.smithi018.stdout:0/287: dread - da/d15/d17/d21/d36/f63 zero size 2024-04-12T20:56:04.140 INFO:tasks.workunit.client.0.smithi018.stdout:8/403: fsync db/d25/f52 0 2024-04-12T20:56:04.145 INFO:tasks.workunit.client.0.smithi018.stdout:4/326: symlink d3/d34/l69 0 2024-04-12T20:56:04.146 INFO:tasks.workunit.client.0.smithi018.stdout:1/235: symlink d3/d9/l51 0 2024-04-12T20:56:04.146 INFO:tasks.workunit.client.0.smithi018.stdout:6/407: mknod d5/df/d31/ca0 0 2024-04-12T20:56:04.148 INFO:tasks.workunit.client.0.smithi018.stdout:4/327: dread - d3/d5d/d60/f67 zero size 2024-04-12T20:56:04.149 INFO:tasks.workunit.client.0.smithi018.stdout:2/244: dwrite d5/f1d [0,4194304] 0 2024-04-12T20:56:04.151 INFO:tasks.workunit.client.0.smithi018.stdout:1/236: chown d3/d9/da/dd/d26 99099421 1 2024-04-12T20:56:04.154 INFO:tasks.workunit.client.0.smithi018.stdout:3/282: read d2/d6/f1c [193885,125305] 0 2024-04-12T20:56:04.165 INFO:tasks.workunit.client.0.smithi018.stdout:7/330: dwrite d1/db/d31/f4d [4194304,4194304] 0 2024-04-12T20:56:04.169 INFO:tasks.workunit.client.0.smithi018.stdout:2/245: creat d5/d7/d28/f59 x:0 0 0 2024-04-12T20:56:04.172 INFO:tasks.workunit.client.0.smithi018.stdout:5/293: mkdir d0/d2/d33/d10/d2f/d58/d5e 0 2024-04-12T20:56:04.174 INFO:tasks.workunit.client.0.smithi018.stdout:1/237: mknod d3/d9/da/dd/d31/c52 0 2024-04-12T20:56:04.176 INFO:tasks.workunit.client.0.smithi018.stdout:9/513: dwrite d3/d6/d9/dc/d13/d5b/d7c/f69 [0,4194304] 0 2024-04-12T20:56:04.176 INFO:tasks.workunit.client.0.smithi018.stdout:0/288: creat da/d15/d17/d5c/f64 x:0 0 0 2024-04-12T20:56:04.179 INFO:tasks.workunit.client.0.smithi018.stdout:3/283: creat d2/d43/f52 x:0 0 0 2024-04-12T20:56:04.181 INFO:tasks.workunit.client.0.smithi018.stdout:6/408: dwrite d5/df/d19/d74/f9f [0,4194304] 0 2024-04-12T20:56:04.182 INFO:tasks.workunit.client.0.smithi018.stdout:4/328: dread d3/d34/d43/f2f [0,4194304] 0 2024-04-12T20:56:04.184 INFO:tasks.workunit.client.0.smithi018.stdout:8/404: getdents db/d25/d26/d57/d69 0 2024-04-12T20:56:04.184 INFO:tasks.workunit.client.0.smithi018.stdout:3/284: fdatasync d2/d6/f24 0 2024-04-12T20:56:04.187 INFO:tasks.workunit.client.0.smithi018.stdout:9/514: truncate d3/d6/d9/dc/d6f/f9a 601496 0 2024-04-12T20:56:04.189 INFO:tasks.workunit.client.0.smithi018.stdout:1/238: symlink d3/d9/d11/l53 0 2024-04-12T20:56:04.191 INFO:tasks.workunit.client.0.smithi018.stdout:0/289: creat da/d15/d4d/d58/f65 x:0 0 0 2024-04-12T20:56:04.196 INFO:tasks.workunit.client.0.smithi018.stdout:0/290: write da/d15/d17/d5c/f64 [17363,56410] 0 2024-04-12T20:56:04.198 INFO:tasks.workunit.client.0.smithi018.stdout:2/246: getdents d5/d8/d10/d43 0 2024-04-12T20:56:04.201 INFO:tasks.workunit.client.0.smithi018.stdout:1/239: write d3/d9/da/dd/f17 [5232064,1333] 0 2024-04-12T20:56:04.204 INFO:tasks.workunit.client.0.smithi018.stdout:8/405: symlink db/d25/d26/d57/d69/l7c 0 2024-04-12T20:56:04.210 INFO:tasks.workunit.client.0.smithi018.stdout:8/406: write db/d3b/f55 [949079,76508] 0 2024-04-12T20:56:04.210 INFO:tasks.workunit.client.0.smithi018.stdout:5/294: dwrite d0/d2/d33/d10/d26/d30/f31 [4194304,4194304] 0 2024-04-12T20:56:04.212 INFO:tasks.workunit.client.0.smithi018.stdout:3/285: creat d2/d6/f53 x:0 0 0 2024-04-12T20:56:04.220 INFO:tasks.workunit.client.0.smithi018.stdout:6/409: creat d5/df/d19/fa1 x:0 0 0 2024-04-12T20:56:04.222 INFO:tasks.workunit.client.0.smithi018.stdout:6/410: dread - d5/d6/d1b/d3b/f91 zero size 2024-04-12T20:56:04.225 INFO:tasks.workunit.client.0.smithi018.stdout:6/411: fdatasync d5/f48 0 2024-04-12T20:56:04.227 INFO:tasks.workunit.client.0.smithi018.stdout:6/412: stat d5/d6/l88 0 2024-04-12T20:56:04.236 INFO:tasks.workunit.client.0.smithi018.stdout:1/240: fdatasync d3/f7 0 2024-04-12T20:56:04.243 INFO:tasks.workunit.client.0.smithi018.stdout:2/247: mknod d5/c5a 0 2024-04-12T20:56:04.243 INFO:tasks.workunit.client.0.smithi018.stdout:9/515: dwrite d3/d6/d9/dc/d6f/f9a [0,4194304] 0 2024-04-12T20:56:04.247 INFO:tasks.workunit.client.0.smithi018.stdout:7/331: dread d1/f66 [0,4194304] 0 2024-04-12T20:56:04.247 INFO:tasks.workunit.client.0.smithi018.stdout:6/413: creat d5/df/d19/d27/fa2 x:0 0 0 2024-04-12T20:56:04.251 INFO:tasks.workunit.client.0.smithi018.stdout:1/241: truncate d3/d9/da/dd/f4a 138781 0 2024-04-12T20:56:04.253 INFO:tasks.workunit.client.0.smithi018.stdout:5/295: dwrite d0/d2/d33/d10/f39 [4194304,4194304] 0 2024-04-12T20:56:04.253 INFO:tasks.workunit.client.0.smithi018.stdout:6/414: stat d5/d6/d90 0 2024-04-12T20:56:04.253 INFO:tasks.workunit.client.0.smithi018.stdout:3/286: mkdir d2/d6/d3d/d54 0 2024-04-12T20:56:04.258 INFO:tasks.workunit.client.0.smithi018.stdout:0/291: link da/d15/c16 da/d15/d17/d21/c66 0 2024-04-12T20:56:04.261 INFO:tasks.workunit.client.0.smithi018.stdout:4/329: dread d3/db/f37 [0,4194304] 0 2024-04-12T20:56:04.261 INFO:tasks.workunit.client.0.smithi018.stdout:7/332: read d1/d6/f27 [903229,122085] 0 2024-04-12T20:56:04.268 INFO:tasks.workunit.client.0.smithi018.stdout:2/248: creat d5/d12/f5b x:0 0 0 2024-04-12T20:56:04.268 INFO:tasks.workunit.client.0.smithi018.stdout:6/415: truncate d5/d6/d1b/d3b/f91 519035 0 2024-04-12T20:56:04.269 INFO:tasks.workunit.client.0.smithi018.stdout:1/242: read - d3/d9/da/dd/d31/d32/d13/d19/d2d/f4d zero size 2024-04-12T20:56:04.277 INFO:tasks.workunit.client.0.smithi018.stdout:7/333: creat d1/db/d31/d3a/d42/d4c/f72 x:0 0 0 2024-04-12T20:56:04.278 INFO:tasks.workunit.client.0.smithi018.stdout:2/249: fsync d5/d8/d10/d19/d24/f29 0 2024-04-12T20:56:04.287 INFO:tasks.workunit.client.0.smithi018.stdout:3/287: link d2/d6/d30/c4b d2/d6/d30/d31/d38/d3a/d4c/c55 0 2024-04-12T20:56:04.289 INFO:tasks.workunit.client.0.smithi018.stdout:9/516: truncate d3/d6/d9/fd 2922909 0 2024-04-12T20:56:04.293 INFO:tasks.workunit.client.0.smithi018.stdout:7/334: creat d1/db/d31/d3a/d4f/f73 x:0 0 0 2024-04-12T20:56:04.301 INFO:tasks.workunit.client.0.smithi018.stdout:4/330: dwrite d3/d34/d3d/f65 [0,4194304] 0 2024-04-12T20:56:04.303 INFO:tasks.workunit.client.0.smithi018.stdout:5/296: dwrite d0/d2b/f2d [0,4194304] 0 2024-04-12T20:56:04.308 INFO:tasks.workunit.client.0.smithi018.stdout:6/416: dwrite d5/df/d19/f54 [4194304,4194304] 0 2024-04-12T20:56:04.314 INFO:tasks.workunit.client.0.smithi018.stdout:9/517: truncate d3/d6/d9/dc/d5a/d79/d7e/d9b/f2e 809836 0 2024-04-12T20:56:04.316 INFO:tasks.workunit.client.0.smithi018.stdout:1/243: truncate d3/d9/da/dd/d31/d32/f34 1744608 0 2024-04-12T20:56:04.321 INFO:tasks.workunit.client.0.smithi018.stdout:3/288: fsync d2/d6/d30/d31/d38/f3b 0 2024-04-12T20:56:04.321 INFO:tasks.workunit.client.0.smithi018.stdout:0/292: dread da/d15/f20 [0,4194304] 0 2024-04-12T20:56:04.326 INFO:tasks.workunit.client.0.smithi018.stdout:6/417: creat d5/d71/fa3 x:0 0 0 2024-04-12T20:56:04.330 INFO:tasks.workunit.client.0.smithi018.stdout:6/418: creat d5/d6/d1b/d3b/fa4 x:0 0 0 2024-04-12T20:56:04.338 INFO:tasks.workunit.client.0.smithi018.stdout:5/297: mknod d0/d2/d33/c5f 0 2024-04-12T20:56:04.341 INFO:tasks.workunit.client.0.smithi018.stdout:9/518: dread d3/d6/d9/dc/d5a/d79/d7e/d9b/f2e [0,4194304] 0 2024-04-12T20:56:04.345 INFO:tasks.workunit.client.0.smithi018.stdout:4/331: symlink d3/d34/d43/l6a 0 2024-04-12T20:56:04.347 INFO:tasks.workunit.client.0.smithi018.stdout:9/519: chown d3/d6/d9/dc/d13 15878974 1 2024-04-12T20:56:04.350 INFO:tasks.workunit.client.0.smithi018.stdout:7/335: truncate d1/d6/f52 3908277 0 2024-04-12T20:56:04.351 INFO:tasks.workunit.client.0.smithi018.stdout:2/250: getdents d5/d8/d10/d19 0 2024-04-12T20:56:04.351 INFO:tasks.workunit.client.0.smithi018.stdout:5/298: rmdir d0/d2/d33/d46/d1e 39 2024-04-12T20:56:04.355 INFO:tasks.workunit.client.0.smithi018.stdout:6/419: creat d5/df/d19/d2b/fa5 x:0 0 0 2024-04-12T20:56:04.355 INFO:tasks.workunit.client.0.smithi018.stdout:1/244: truncate d3/d9/da/dd/d31/f35 853254 0 2024-04-12T20:56:04.358 INFO:tasks.workunit.client.0.smithi018.stdout:0/293: dwrite da/d15/d17/d21/f5f [0,4194304] 0 2024-04-12T20:56:04.359 INFO:tasks.workunit.client.0.smithi018.stdout:1/245: chown d3/d9/d11/f37 124556 1 2024-04-12T20:56:04.361 INFO:tasks.workunit.client.0.smithi018.stdout:7/336: write d1/db/d31/d30/f57 [966167,31109] 0 2024-04-12T20:56:04.361 INFO:tasks.workunit.client.0.smithi018.stdout:6/420: dread - d5/d71/fa3 zero size 2024-04-12T20:56:04.367 INFO:tasks.workunit.client.0.smithi018.stdout:3/289: getdents d2/d6/d30/d31/d38/d3a/d4c 0 2024-04-12T20:56:04.368 INFO:tasks.workunit.client.0.smithi018.stdout:0/294: truncate da/d15/f33 4462879 0 2024-04-12T20:56:04.368 INFO:tasks.workunit.client.0.smithi018.stdout:7/337: fsync d1/db/d31/d3a/d42/d4c/f5b 0 2024-04-12T20:56:04.371 INFO:tasks.workunit.client.0.smithi018.stdout:4/332: rename d3/f3b to d3/f6b 0 2024-04-12T20:56:04.371 INFO:tasks.workunit.client.0.smithi018.stdout:1/246: rename d3/d9/da/lb to d3/d9/d11/l54 0 2024-04-12T20:56:04.371 INFO:tasks.workunit.client.0.smithi018.stdout:9/520: creat d3/d6/d9/dc/d13/d5b/d6b/fa1 x:0 0 0 2024-04-12T20:56:04.371 INFO:tasks.workunit.client.0.smithi018.stdout:6/421: stat d5/d6/d1b/d3b/d22 0 2024-04-12T20:56:04.377 INFO:tasks.workunit.client.0.smithi018.stdout:9/521: readlink d3/l4d 0 2024-04-12T20:56:04.381 INFO:tasks.workunit.client.0.smithi018.stdout:4/333: read d3/f2d [3202042,50969] 0 2024-04-12T20:56:04.381 INFO:tasks.workunit.client.0.smithi018.stdout:7/338: write d1/db/d31/f63 [228669,13905] 0 2024-04-12T20:56:04.384 INFO:tasks.workunit.client.0.smithi018.stdout:1/247: creat d3/d9/da/dd/f55 x:0 0 0 2024-04-12T20:56:04.384 INFO:tasks.workunit.client.0.smithi018.stdout:6/422: write d5/d6/d1b/d3b/d22/f62 [3691235,125872] 0 2024-04-12T20:56:04.390 INFO:tasks.workunit.client.0.smithi018.stdout:3/290: link d2/l27 d2/d4d/l56 0 2024-04-12T20:56:04.390 INFO:tasks.workunit.client.0.smithi018.stdout:9/522: truncate d3/d6/d9/dc/d13/d5b/d6b/fa1 994469 0 2024-04-12T20:56:04.391 INFO:tasks.workunit.client.0.smithi018.stdout:2/251: truncate d5/d8/d10/d19/d24/d39/f45 4087 0 2024-04-12T20:56:04.395 INFO:tasks.workunit.client.0.smithi018.stdout:5/299: truncate d0/d2/d33/d46/d1e/f48 2468750 0 2024-04-12T20:56:04.395 INFO:tasks.workunit.client.0.smithi018.stdout:5/300: chown d0 215 1 2024-04-12T20:56:04.396 INFO:tasks.workunit.client.0.smithi018.stdout:6/423: creat d5/d15/fa6 x:0 0 0 2024-04-12T20:56:04.396 INFO:tasks.workunit.client.0.smithi018.stdout:2/252: chown d5/d8/d10/d19/d24/c4a 8 1 2024-04-12T20:56:04.397 INFO:tasks.workunit.client.0.smithi018.stdout:0/295: truncate da/f22 1722194 0 2024-04-12T20:56:04.401 INFO:tasks.workunit.client.0.smithi018.stdout:3/291: unlink d2/d6/d34/l41 0 2024-04-12T20:56:04.403 INFO:tasks.workunit.client.0.smithi018.stdout:0/296: dread - da/d15/d3e/d3f/f59 zero size 2024-04-12T20:56:04.404 INFO:tasks.workunit.client.0.smithi018.stdout:2/253: write d5/d8/d10/f30 [5536337,78261] 0 2024-04-12T20:56:04.407 INFO:tasks.workunit.client.0.smithi018.stdout:0/297: chown da/f38 142 1 2024-04-12T20:56:04.412 INFO:tasks.workunit.client.0.smithi018.stdout:5/301: mkdir d0/d2/d33/d10/d60 0 2024-04-12T20:56:04.412 INFO:tasks.workunit.client.0.smithi018.stdout:4/334: dwrite d3/d32/d33/f18 [0,4194304] 0 2024-04-12T20:56:04.418 INFO:tasks.workunit.client.0.smithi018.stdout:7/339: write d1/d6/d8/f4 [122257,45698] 0 2024-04-12T20:56:04.419 INFO:tasks.workunit.client.0.smithi018.stdout:3/292: unlink d2/d6/f2e 0 2024-04-12T20:56:04.424 INFO:tasks.workunit.client.0.smithi018.stdout:6/424: dread d5/d6/d1b/d3b/f3d [0,4194304] 0 2024-04-12T20:56:04.426 INFO:tasks.workunit.client.0.smithi018.stdout:9/523: rename d3/d6/d9/dc/d13/l8d to d3/d6/d9/dc/d46/la2 0 2024-04-12T20:56:04.433 INFO:tasks.workunit.client.0.smithi018.stdout:1/248: getdents d3/d9/da/dd/d31 0 2024-04-12T20:56:04.433 INFO:tasks.workunit.client.0.smithi018.stdout:7/340: rmdir d1/db/d31/d3a/d42 39 2024-04-12T20:56:04.436 INFO:tasks.workunit.client.0.smithi018.stdout:9/524: write d3/d6/d9/d3f/f81 [1179348,109717] 0 2024-04-12T20:56:04.438 INFO:tasks.workunit.client.0.smithi018.stdout:2/254: dwrite d5/d8/d10/d19/d24/d39/f51 [0,4194304] 0 2024-04-12T20:56:04.439 INFO:tasks.workunit.client.0.smithi018.stdout:5/302: rename d0/d2/d33/d10/d2f/d58/d5e to d0/d2/d33/d61 0 2024-04-12T20:56:04.442 INFO:tasks.workunit.client.0.smithi018.stdout:3/293: creat d2/d4d/f57 x:0 0 0 2024-04-12T20:56:04.442 INFO:tasks.workunit.client.0.smithi018.stdout:2/255: fdatasync d5/d8/d10/f15 0 2024-04-12T20:56:04.442 INFO:tasks.workunit.client.0.smithi018.stdout:2/256: readlink d5/l40 0 2024-04-12T20:56:04.445 INFO:tasks.workunit.client.0.smithi018.stdout:3/294: chown d2/d6/f9 65415022 1 2024-04-12T20:56:04.445 INFO:tasks.workunit.client.0.smithi018.stdout:4/335: dread d3/d34/d3d/f65 [0,4194304] 0 2024-04-12T20:56:04.449 INFO:tasks.workunit.client.0.smithi018.stdout:3/295: write d2/d6/f4e [430483,57565] 0 2024-04-12T20:56:04.450 INFO:tasks.workunit.client.0.smithi018.stdout:8/407: sync 2024-04-12T20:56:04.454 INFO:tasks.workunit.client.0.smithi018.stdout:9/525: creat d3/d6/d43/fa3 x:0 0 0 2024-04-12T20:56:04.456 INFO:tasks.workunit.client.0.smithi018.stdout:8/408: write db/d25/d26/d27/f54 [397780,24668] 0 2024-04-12T20:56:04.459 INFO:tasks.workunit.client.0.smithi018.stdout:7/341: mknod d1/db/d31/d45/d53/c74 0 2024-04-12T20:56:04.461 INFO:tasks.workunit.client.0.smithi018.stdout:1/249: dwrite d3/f7 [4194304,4194304] 0 2024-04-12T20:56:04.469 INFO:tasks.workunit.client.0.smithi018.stdout:7/342: write d1/db/d31/d3a/f41 [3850230,114974] 0 2024-04-12T20:56:04.472 INFO:tasks.workunit.client.0.smithi018.stdout:0/298: dread da/d15/f39 [0,4194304] 0 2024-04-12T20:56:04.479 INFO:tasks.workunit.client.0.smithi018.stdout:4/336: truncate d3/f2d 3752596 0 2024-04-12T20:56:04.482 INFO:tasks.workunit.client.0.smithi018.stdout:2/257: dwrite d5/d8/d10/f30 [4194304,4194304] 0 2024-04-12T20:56:04.482 INFO:tasks.workunit.client.0.smithi018.stdout:6/425: rmdir d5/d15 39 2024-04-12T20:56:04.485 INFO:tasks.workunit.client.0.smithi018.stdout:5/303: link d0/d2/d33/d10/f27 d0/d2/d33/d46/d35/f62 0 2024-04-12T20:56:04.485 INFO:tasks.workunit.client.0.smithi018.stdout:3/296: dwrite d2/f1a [0,4194304] 0 2024-04-12T20:56:04.486 INFO:tasks.workunit.client.0.smithi018.stdout:0/299: dread da/f13 [4194304,4194304] 0 2024-04-12T20:56:04.488 INFO:tasks.workunit.client.0.smithi018.stdout:9/526: rename d3/d6/d3d/l57 to d3/d6/d9/dc/la4 0 2024-04-12T20:56:04.491 INFO:tasks.workunit.client.0.smithi018.stdout:0/300: readlink da/l10 0 2024-04-12T20:56:04.494 INFO:tasks.workunit.client.0.smithi018.stdout:5/304: write d0/d2/d33/d46/d18/f59 [612054,106100] 0 2024-04-12T20:56:04.495 INFO:tasks.workunit.client.0.smithi018.stdout:2/258: chown d5/c38 69135208 1 2024-04-12T20:56:04.496 INFO:tasks.workunit.client.0.smithi018.stdout:3/297: read d2/d6/f4e [41905,21965] 0 2024-04-12T20:56:04.500 INFO:tasks.workunit.client.0.smithi018.stdout:6/426: write d5/d15/d60/f6d [783954,75500] 0 2024-04-12T20:56:04.500 INFO:tasks.workunit.client.0.smithi018.stdout:3/298: fdatasync d2/d43/f52 0 2024-04-12T20:56:04.501 INFO:tasks.workunit.client.0.smithi018.stdout:9/527: creat d3/d6/d9/dc/d5a/d79/d7e/d9b/d21/d41/fa5 x:0 0 0 2024-04-12T20:56:04.508 INFO:tasks.workunit.client.0.smithi018.stdout:6/427: creat d5/d6/d1b/d3b/d35/d4f/fa7 x:0 0 0 2024-04-12T20:56:04.510 INFO:tasks.workunit.client.0.smithi018.stdout:9/528: symlink d3/d6/d9/dc/d13/d5b/d6b/la6 0 2024-04-12T20:56:04.510 INFO:tasks.workunit.client.0.smithi018.stdout:0/301: mknod da/d15/d4d/c67 0 2024-04-12T20:56:04.514 INFO:tasks.workunit.client.0.smithi018.stdout:9/529: fsync d3/d6/d9/dc/d6f/d77/f9d 0 2024-04-12T20:56:04.518 INFO:tasks.workunit.client.0.smithi018.stdout:0/302: chown da/d15/d3e/d3f/f59 32855061 1 2024-04-12T20:56:04.524 INFO:tasks.workunit.client.0.smithi018.stdout:9/530: mkdir d3/d6/d9/dc/d13/d5b/d6b/da7 0 2024-04-12T20:56:04.525 INFO:tasks.workunit.client.0.smithi018.stdout:5/305: rename d0/d2/d11/d37/c56 to d0/d2/d33/d46/d1e/d2e/c63 0 2024-04-12T20:56:04.527 INFO:tasks.workunit.client.0.smithi018.stdout:6/428: link d5/d71/l9d d5/d6/d90/la8 0 2024-04-12T20:56:04.528 INFO:tasks.workunit.client.0.smithi018.stdout:4/337: write d3/d32/fd [1355101,68057] 0 2024-04-12T20:56:04.531 INFO:tasks.workunit.client.0.smithi018.stdout:5/306: write d0/d2b/f2d [2873648,60816] 0 2024-04-12T20:56:04.533 INFO:tasks.workunit.client.0.smithi018.stdout:1/250: link d3/d9/da/l1c d3/d9/da/dd/d31/d44/l56 0 2024-04-12T20:56:04.533 INFO:tasks.workunit.client.0.smithi018.stdout:9/531: dread d3/d6/d9/d1e/f40 [0,4194304] 0 2024-04-12T20:56:04.535 INFO:tasks.workunit.client.0.smithi018.stdout:6/429: unlink d5/d6/d1b/d3b/d22/f62 0 2024-04-12T20:56:04.537 INFO:tasks.workunit.client.0.smithi018.stdout:5/307: dread - d0/d2/d11/f4b zero size 2024-04-12T20:56:04.544 INFO:tasks.workunit.client.0.smithi018.stdout:8/409: getdents db/d25/d26/d57/d69 0 2024-04-12T20:56:04.544 INFO:tasks.workunit.client.0.smithi018.stdout:8/410: chown db/d3b 1 1 2024-04-12T20:56:04.549 INFO:tasks.workunit.client.0.smithi018.stdout:7/343: dread d1/f32 [0,4194304] 0 2024-04-12T20:56:04.551 INFO:tasks.workunit.client.0.smithi018.stdout:1/251: write d3/d9/da/dd/d31/d32/d13/d19/d2d/d4e/f4f [81509,16713] 0 2024-04-12T20:56:04.555 INFO:tasks.workunit.client.0.smithi018.stdout:1/252: chown d3 12 1 2024-04-12T20:56:04.556 INFO:tasks.workunit.client.0.smithi018.stdout:6/430: write d5/d6/d1b/d3b/d35/d4f/f5b [4922076,35713] 0 2024-04-12T20:56:04.566 INFO:tasks.workunit.client.0.smithi018.stdout:3/299: dread d2/f19 [0,4194304] 0 2024-04-12T20:56:04.566 INFO:tasks.workunit.client.0.smithi018.stdout:5/308: rmdir d0/d5b 39 2024-04-12T20:56:04.572 INFO:tasks.workunit.client.0.smithi018.stdout:2/259: dwrite d5/d8/d25/f34 [0,4194304] 0 2024-04-12T20:56:04.573 INFO:tasks.workunit.client.0.smithi018.stdout:7/344: fsync d1/f66 0 2024-04-12T20:56:04.577 INFO:tasks.workunit.client.0.smithi018.stdout:4/338: dwrite d3/d5d/d60/d64/d4f/f61 [0,4194304] 0 2024-04-12T20:56:04.579 INFO:tasks.workunit.client.0.smithi018.stdout:8/411: creat db/d72/f7d x:0 0 0 2024-04-12T20:56:04.584 INFO:tasks.workunit.client.0.smithi018.stdout:1/253: rmdir d3/d9/da/dd 39 2024-04-12T20:56:04.585 INFO:tasks.workunit.client.0.smithi018.stdout:9/532: dwrite d3/d6/d43/f8f [0,4194304] 0 2024-04-12T20:56:04.586 INFO:tasks.workunit.client.0.smithi018.stdout:4/339: dread - d3/d34/d43/f68 zero size 2024-04-12T20:56:04.590 INFO:tasks.workunit.client.0.smithi018.stdout:1/254: write d3/d9/da/f4c [183886,130568] 0 2024-04-12T20:56:04.599 INFO:tasks.workunit.client.0.smithi018.stdout:0/303: getdents da/d15/d17/d5c 0 2024-04-12T20:56:04.601 INFO:tasks.workunit.client.0.smithi018.stdout:3/300: creat d2/d6/d30/f58 x:0 0 0 2024-04-12T20:56:04.604 INFO:tasks.workunit.client.0.smithi018.stdout:0/304: chown da/f1d 605633699 1 2024-04-12T20:56:04.605 INFO:tasks.workunit.client.0.smithi018.stdout:4/340: unlink d3/d5d/d60/d64/f50 0 2024-04-12T20:56:04.609 INFO:tasks.workunit.client.0.smithi018.stdout:5/309: dread d0/d2/d33/d10/f39 [4194304,4194304] 0 2024-04-12T20:56:04.610 INFO:tasks.workunit.client.0.smithi018.stdout:1/255: write d3/d9/da/dd/f3f [969607,98090] 0 2024-04-12T20:56:04.614 INFO:tasks.workunit.client.0.smithi018.stdout:8/412: rename db/dc/c51 to db/d25/d26/d57/d71/c7e 0 2024-04-12T20:56:04.619 INFO:tasks.workunit.client.0.smithi018.stdout:8/413: dread - db/dc/f5e zero size 2024-04-12T20:56:04.619 INFO:tasks.workunit.client.0.smithi018.stdout:7/345: creat d1/db/d31/d3a/d42/f75 x:0 0 0 2024-04-12T20:56:04.619 INFO:tasks.workunit.client.0.smithi018.stdout:5/310: chown d0/d2/d33/d46/d1e/d55 7 1 2024-04-12T20:56:04.620 INFO:tasks.workunit.client.0.smithi018.stdout:9/533: mkdir d3/d93/da8 0 2024-04-12T20:56:04.620 INFO:tasks.workunit.client.0.smithi018.stdout:6/431: dwrite d5/df/d19/d2b/f9c [0,4194304] 0 2024-04-12T20:56:04.629 INFO:tasks.workunit.client.0.smithi018.stdout:0/305: creat da/d15/d17/d21/f68 x:0 0 0 2024-04-12T20:56:04.631 INFO:tasks.workunit.client.0.smithi018.stdout:0/306: read - da/f52 zero size 2024-04-12T20:56:04.632 INFO:tasks.workunit.client.0.smithi018.stdout:2/260: creat d5/d7/f5c x:0 0 0 2024-04-12T20:56:04.648 INFO:tasks.workunit.client.0.smithi018.stdout:8/414: mkdir db/d4e/d7f 0 2024-04-12T20:56:04.656 INFO:tasks.workunit.client.0.smithi018.stdout:4/341: link d3/d32/d33/l28 d3/d34/d43/d4b/l6c 0 2024-04-12T20:56:04.657 INFO:tasks.workunit.client.0.smithi018.stdout:3/301: dwrite d2/d6/f2f [0,4194304] 0 2024-04-12T20:56:04.658 INFO:tasks.workunit.client.0.smithi018.stdout:6/432: link d5/l72 d5/d15/la9 0 2024-04-12T20:56:04.662 INFO:tasks.workunit.client.0.smithi018.stdout:6/433: stat d5/d15/fa6 0 2024-04-12T20:56:04.664 INFO:tasks.workunit.client.0.smithi018.stdout:5/311: dwrite d0/d2/d11/d37/f4c [0,4194304] 0 2024-04-12T20:56:04.668 INFO:tasks.workunit.client.0.smithi018.stdout:3/302: chown d2/d6/l36 1185463863 1 2024-04-12T20:56:04.668 INFO:tasks.workunit.client.0.smithi018.stdout:7/346: dread d1/d6/f27 [0,4194304] 0 2024-04-12T20:56:04.670 INFO:tasks.workunit.client.0.smithi018.stdout:9/534: dwrite d3/d6/f3c [4194304,4194304] 0 2024-04-12T20:56:04.690 INFO:tasks.workunit.client.0.smithi018.stdout:4/342: truncate d3/db/f26 343571 0 2024-04-12T20:56:04.692 INFO:tasks.workunit.client.0.smithi018.stdout:4/343: chown d3/d5d 3507669 1 2024-04-12T20:56:04.698 INFO:tasks.workunit.client.0.smithi018.stdout:9/535: rename d3/d6/d9/dc/d5a/d79/d7e/d9b/d21/d41/d78/f95 to d3/d6/d9/dc/d13/fa9 0 2024-04-12T20:56:04.705 INFO:tasks.workunit.client.0.smithi018.stdout:3/303: creat d2/d6/d34/f59 x:0 0 0 2024-04-12T20:56:04.711 INFO:tasks.workunit.client.0.smithi018.stdout:0/307: getdents da/d15/d17/d21/d36/d47 0 2024-04-12T20:56:04.711 INFO:tasks.workunit.client.0.smithi018.stdout:6/434: dwrite d5/d6/d1b/d3b/d35/f33 [0,4194304] 0 2024-04-12T20:56:04.714 INFO:tasks.workunit.client.0.smithi018.stdout:5/312: creat d0/d5b/f64 x:0 0 0 2024-04-12T20:56:04.718 INFO:tasks.workunit.client.0.smithi018.stdout:9/536: truncate d3/d6/d9/dc/d5a/d79/d7e/d9b/d21/d41/f91 317403 0 2024-04-12T20:56:04.723 INFO:tasks.workunit.client.0.smithi018.stdout:4/344: creat d3/d34/d40/f6d x:0 0 0 2024-04-12T20:56:04.723 INFO:tasks.workunit.client.0.smithi018.stdout:9/537: fsync d3/d6/d9/dc/d6f/d77/f9d 0 2024-04-12T20:56:04.727 INFO:tasks.workunit.client.0.smithi018.stdout:0/308: dread da/d15/d3e/f5e [0,4194304] 0 2024-04-12T20:56:04.734 INFO:tasks.workunit.client.0.smithi018.stdout:4/345: stat d3/d34/d53 0 2024-04-12T20:56:04.737 INFO:tasks.workunit.client.0.smithi018.stdout:3/304: creat d2/d6/d3d/f5a x:0 0 0 2024-04-12T20:56:04.743 INFO:tasks.workunit.client.0.smithi018.stdout:7/347: dread d1/d6/f24 [0,4194304] 0 2024-04-12T20:56:04.745 INFO:tasks.workunit.client.0.smithi018.stdout:4/346: chown d3/d32/c5c 2 1 2024-04-12T20:56:04.746 INFO:tasks.workunit.client.0.smithi018.stdout:6/435: creat d5/df/d19/faa x:0 0 0 2024-04-12T20:56:04.748 INFO:tasks.workunit.client.0.smithi018.stdout:8/415: dwrite db/d25/d26/f53 [0,4194304] 0 2024-04-12T20:56:04.753 INFO:tasks.workunit.client.0.smithi018.stdout:9/538: rename d3/d6/d9/dc/l39 to d3/d6/d9/dc/d13/d5b/d6b/laa 0 2024-04-12T20:56:04.755 INFO:tasks.workunit.client.0.smithi018.stdout:4/347: write d3/db/f23 [2818440,33324] 0 2024-04-12T20:56:04.762 INFO:tasks.workunit.client.0.smithi018.stdout:3/305: mknod d2/d6/c5b 0 2024-04-12T20:56:04.762 INFO:tasks.workunit.client.0.smithi018.stdout:5/313: dwrite d0/d2/d33/d46/d35/f4f [0,4194304] 0 2024-04-12T20:56:04.766 INFO:tasks.workunit.client.0.smithi018.stdout:6/436: fsync d5/df/d19/fa1 0 2024-04-12T20:56:04.766 INFO:tasks.workunit.client.0.smithi018.stdout:1/256: dread d3/f4 [4194304,4194304] 0 2024-04-12T20:56:04.770 INFO:tasks.workunit.client.0.smithi018.stdout:1/257: chown d3/d9/da/dd/d31/f35 48805 1 2024-04-12T20:56:04.781 INFO:tasks.workunit.client.0.smithi018.stdout:1/258: write d3/d9/da/dd/d31/d32/f41 [786611,36462] 0 2024-04-12T20:56:04.783 INFO:tasks.workunit.client.0.smithi018.stdout:8/416: dread db/d25/d26/d27/f54 [0,4194304] 0 2024-04-12T20:56:04.788 INFO:tasks.workunit.client.0.smithi018.stdout:4/348: unlink d3/f2d 0 2024-04-12T20:56:04.792 INFO:tasks.workunit.client.0.smithi018.stdout:8/417: dread - db/d25/d26/d27/f44 zero size 2024-04-12T20:56:04.792 INFO:tasks.workunit.client.0.smithi018.stdout:3/306: truncate d2/d6/d30/d31/f50 954176 0 2024-04-12T20:56:04.795 INFO:tasks.workunit.client.0.smithi018.stdout:4/349: write d3/f52 [1064525,124501] 0 2024-04-12T20:56:04.797 INFO:tasks.workunit.client.0.smithi018.stdout:6/437: symlink d5/lab 0 2024-04-12T20:56:04.797 INFO:tasks.workunit.client.0.smithi018.stdout:8/418: stat db/d4e/d56/d59/c75 0 2024-04-12T20:56:04.802 INFO:tasks.workunit.client.0.smithi018.stdout:5/314: dwrite d0/d2/d33/d46/d18/f3b [0,4194304] 0 2024-04-12T20:56:04.802 INFO:tasks.workunit.client.0.smithi018.stdout:1/259: read d3/d9/da/dd/d31/d32/f34 [61135,9691] 0 2024-04-12T20:56:04.806 INFO:tasks.workunit.client.0.smithi018.stdout:3/307: creat d2/d6/d30/d31/d38/d3a/d48/f5c x:0 0 0 2024-04-12T20:56:04.807 INFO:tasks.workunit.client.0.smithi018.stdout:0/309: dread da/d15/f33 [0,4194304] 0 2024-04-12T20:56:04.809 INFO:tasks.workunit.client.0.smithi018.stdout:3/308: write d2/f42 [33820,8553] 0 2024-04-12T20:56:04.811 INFO:tasks.workunit.client.0.smithi018.stdout:6/438: write d5/df/d19/d27/fa2 [592380,47939] 0 2024-04-12T20:56:04.813 INFO:tasks.workunit.client.0.smithi018.stdout:7/348: dread d1/d6/d8/f4 [0,4194304] 0 2024-04-12T20:56:04.815 INFO:tasks.workunit.client.0.smithi018.stdout:1/260: fdatasync d3/d9/da/dd/d31/d32/d13/d19/d2d/f40 0 2024-04-12T20:56:04.816 INFO:tasks.workunit.client.0.smithi018.stdout:6/439: fdatasync d5/d6/d1b/d3b/d22/f81 0 2024-04-12T20:56:04.817 INFO:tasks.workunit.client.0.smithi018.stdout:3/309: write d2/d6/fc [893942,88482] 0 2024-04-12T20:56:04.819 INFO:tasks.workunit.client.0.smithi018.stdout:5/315: write d0/d2/d33/d46/d35/f4f [3092617,73299] 0 2024-04-12T20:56:04.819 INFO:tasks.workunit.client.0.smithi018.stdout:9/539: write d3/d6/d9/dc/d5a/d79/d7e/d9b/f2e [221301,36747] 0 2024-04-12T20:56:04.821 INFO:tasks.workunit.client.0.smithi018.stdout:1/261: read d3/f7 [7986155,18298] 0 2024-04-12T20:56:04.821 INFO:tasks.workunit.client.0.smithi018.stdout:6/440: chown d5/df/f87 4380795 1 2024-04-12T20:56:04.828 INFO:tasks.workunit.client.0.smithi018.stdout:7/349: write d1/db/d31/d3a/d4f/f73 [225212,85531] 0 2024-04-12T20:56:04.829 INFO:tasks.workunit.client.0.smithi018.stdout:0/310: creat da/d15/d3e/d3f/f69 x:0 0 0 2024-04-12T20:56:04.839 INFO:tasks.workunit.client.0.smithi018.stdout:9/540: dread d3/d6/d9/dc/d5a/d79/d7e/d9b/d21/d41/f91 [0,4194304] 0 2024-04-12T20:56:04.842 INFO:tasks.workunit.client.0.smithi018.stdout:4/350: dread d3/db/f23 [0,4194304] 0 2024-04-12T20:56:04.844 INFO:tasks.workunit.client.0.smithi018.stdout:6/441: rename d5/c40 to d5/df/d31/cac 0 2024-04-12T20:56:04.847 INFO:tasks.workunit.client.0.smithi018.stdout:5/316: link d0/d5b/f5d d0/d2/d33/d10/d26/d30/f65 0 2024-04-12T20:56:04.848 INFO:tasks.workunit.client.0.smithi018.stdout:9/541: chown d3/d6/d9/cf 1 1 2024-04-12T20:56:04.850 INFO:tasks.workunit.client.0.smithi018.stdout:1/262: mkdir d3/d9/da/dd/d31/d32/d13/d19/d2d/d48/d57 0 2024-04-12T20:56:04.853 INFO:tasks.workunit.client.0.smithi018.stdout:5/317: fsync d0/d2b/f3d 0 2024-04-12T20:56:04.854 INFO:tasks.workunit.client.0.smithi018.stdout:8/419: truncate db/d3b/f55 501758 0 2024-04-12T20:56:04.855 INFO:tasks.workunit.client.0.smithi018.stdout:6/442: creat d5/df/d9b/fad x:0 0 0 2024-04-12T20:56:04.866 INFO:tasks.workunit.client.0.smithi018.stdout:8/420: fdatasync db/d25/d26/d27/f50 0 2024-04-12T20:56:04.873 INFO:tasks.workunit.client.0.smithi018.stdout:4/351: dwrite d3/d32/d33/f18 [0,4194304] 0 2024-04-12T20:56:04.875 INFO:tasks.workunit.client.0.smithi018.stdout:7/350: dwrite d1/db/d31/d3a/d42/d4c/f5b [0,4194304] 0 2024-04-12T20:56:04.876 INFO:tasks.workunit.client.0.smithi018.stdout:3/310: dwrite d2/d6/f12 [4194304,4194304] 0 2024-04-12T20:56:04.880 INFO:tasks.workunit.client.0.smithi018.stdout:8/421: unlink db/d25/d26/d27/f44 0 2024-04-12T20:56:04.885 INFO:tasks.workunit.client.0.smithi018.stdout:5/318: write d0/d2/d33/d46/d35/f62 [2904472,8527] 0 2024-04-12T20:56:04.886 INFO:tasks.workunit.client.0.smithi018.stdout:3/311: dread - d2/d6/d30/f37 zero size 2024-04-12T20:56:04.891 INFO:tasks.workunit.client.0.smithi018.stdout:6/443: link d5/d6/d1b/d3b/d22/l8b d5/d15/d60/lae 0 2024-04-12T20:56:04.902 INFO:tasks.workunit.client.0.smithi018.stdout:1/263: dwrite d3/d9/da/dd/d31/d32/d13/d19/d2d/f40 [0,4194304] 0 2024-04-12T20:56:04.910 INFO:tasks.workunit.client.0.smithi018.stdout:6/444: symlink d5/df/d19/d2b/laf 0 2024-04-12T20:56:04.913 INFO:tasks.workunit.client.0.smithi018.stdout:9/542: truncate d3/d6/d9/dc/d13/d5b/d6b/f92 2222204 0 2024-04-12T20:56:04.920 INFO:tasks.workunit.client.0.smithi018.stdout:8/422: write db/f64 [931371,7681] 0 2024-04-12T20:56:04.920 INFO:tasks.workunit.client.0.smithi018.stdout:5/319: dwrite d0/d2/d33/d46/f13 [0,4194304] 0 2024-04-12T20:56:04.924 INFO:tasks.workunit.client.0.smithi018.stdout:9/543: mkdir d3/d6/d9/dc/dab 0 2024-04-12T20:56:04.924 INFO:tasks.workunit.client.0.smithi018.stdout:5/320: chown d0/d2/d33/d46/d1e/d2e/c24 0 1 2024-04-12T20:56:04.925 INFO:tasks.workunit.client.0.smithi018.stdout:9/544: fdatasync d3/d6/f3c 0 2024-04-12T20:56:04.930 INFO:tasks.workunit.client.0.smithi018.stdout:1/264: mkdir d3/d9/da/dd/d31/d32/d13/d19/d2d/d4e/d58 0 2024-04-12T20:56:04.931 INFO:tasks.workunit.client.0.smithi018.stdout:4/352: getdents d3/d5d/d60 0 2024-04-12T20:56:04.934 INFO:tasks.workunit.client.0.smithi018.stdout:8/423: symlink db/d3b/l80 0 2024-04-12T20:56:04.937 INFO:tasks.workunit.client.0.smithi018.stdout:6/445: write d5/df/d31/d4e/f38 [1190348,72787] 0 2024-04-12T20:56:04.941 INFO:tasks.workunit.client.0.smithi018.stdout:6/446: dread - d5/d15/f6f zero size 2024-04-12T20:56:04.942 INFO:tasks.workunit.client.0.smithi018.stdout:4/353: creat d3/d5d/f6e x:0 0 0 2024-04-12T20:56:04.951 INFO:tasks.workunit.client.0.smithi018.stdout:8/424: dread db/d4e/d56/d59/f67 [0,4194304] 0 2024-04-12T20:56:04.953 INFO:tasks.workunit.client.0.smithi018.stdout:1/265: dwrite d3/d9/f18 [0,4194304] 0 2024-04-12T20:56:04.953 INFO:tasks.workunit.client.0.smithi018.stdout:9/545: rmdir d3/d6/d9/dc/d5a/d79/d7e 39 2024-04-12T20:56:04.953 INFO:tasks.workunit.client.0.smithi018.stdout:6/447: creat d5/d6/d1b/d3b/d35/d4f/fb0 x:0 0 0 2024-04-12T20:56:04.962 INFO:tasks.workunit.client.0.smithi018.stdout:5/321: write d0/d2/f1a [3387209,104284] 0 2024-04-12T20:56:04.962 INFO:tasks.workunit.client.0.smithi018.stdout:8/425: dread db/d25/d26/d27/f54 [0,4194304] 0 2024-04-12T20:56:04.964 INFO:tasks.workunit.client.0.smithi018.stdout:6/448: symlink d5/df/d19/d2b/lb1 0 2024-04-12T20:56:04.969 INFO:tasks.workunit.client.0.smithi018.stdout:9/546: mkdir d3/d6/d9/dc/d5a/d79/dac 0 2024-04-12T20:56:04.971 INFO:tasks.workunit.client.0.smithi018.stdout:6/449: symlink d5/d15/d60/lb2 0 2024-04-12T20:56:04.972 INFO:tasks.workunit.client.0.smithi018.stdout:1/266: creat d3/d9/da/dd/d31/f59 x:0 0 0 2024-04-12T20:56:04.972 INFO:tasks.workunit.client.0.smithi018.stdout:3/312: dwrite d2/d6/d30/d31/d38/d3a/d48/f51 [4194304,4194304] 0 2024-04-12T20:56:04.980 INFO:tasks.workunit.client.0.smithi018.stdout:8/426: unlink db/c16 0 2024-04-12T20:56:04.981 INFO:tasks.workunit.client.0.smithi018.stdout:3/313: dread - d2/d6/d30/d31/d38/f44 zero size 2024-04-12T20:56:04.982 INFO:tasks.workunit.client.0.smithi018.stdout:6/450: creat d5/d6/d1b/d3b/d22/fb3 x:0 0 0 2024-04-12T20:56:04.985 INFO:tasks.workunit.client.0.smithi018.stdout:9/547: getdents d3/d6/d9/dc/dab 0 2024-04-12T20:56:04.986 INFO:tasks.workunit.client.0.smithi018.stdout:4/354: dwrite d3/d34/d43/f2f [0,4194304] 0 2024-04-12T20:56:04.988 INFO:tasks.workunit.client.0.smithi018.stdout:3/314: dread - d2/d6/f53 zero size 2024-04-12T20:56:04.989 INFO:tasks.workunit.client.0.smithi018.stdout:1/267: rename d3/d9/da/d14/c29 to d3/d9/da/dd/d31/d32/d13/d19/d2d/d4e/d58/c5a 0 2024-04-12T20:56:04.995 INFO:tasks.workunit.client.0.smithi018.stdout:1/268: write d3/d9/d11/f37 [656542,52327] 0 2024-04-12T20:56:04.997 INFO:tasks.workunit.client.0.smithi018.stdout:6/451: creat d5/df/d19/d43/d5f/fb4 x:0 0 0 2024-04-12T20:56:05.002 INFO:tasks.workunit.client.0.smithi018.stdout:9/548: symlink d3/d6/d9/dc/d13/d5b/d6b/da7/lad 0 2024-04-12T20:56:05.009 INFO:tasks.workunit.client.0.smithi018.stdout:6/452: link d5/df/d3f/f49 d5/df/d31/d4e/fb5 0 2024-04-12T20:56:05.009 INFO:tasks.workunit.client.0.smithi018.stdout:5/322: dread d0/d2/d11/f2a [0,4194304] 0 2024-04-12T20:56:05.017 INFO:tasks.workunit.client.0.smithi018.stdout:1/269: write d3/d9/da/dd/d31/f35 [1900200,43402] 0 2024-04-12T20:56:05.024 INFO:tasks.workunit.client.0.smithi018.stdout:4/355: rename d3/d32/c11 to d3/d34/c6f 0 2024-04-12T20:56:05.025 INFO:tasks.workunit.client.0.smithi018.stdout:4/356: fsync d3/db/f2b 0 2024-04-12T20:56:05.029 INFO:tasks.workunit.client.0.smithi018.stdout:5/323: mkdir d0/d2/d33/d46/d1e/d55/d66 0 2024-04-12T20:56:05.029 INFO:tasks.workunit.client.0.smithi018.stdout:3/315: dread d2/d6/f12 [0,4194304] 0 2024-04-12T20:56:05.034 INFO:tasks.workunit.client.0.smithi018.stdout:8/427: dwrite db/d25/d26/f2f [0,4194304] 0 2024-04-12T20:56:05.034 INFO:tasks.workunit.client.0.smithi018.stdout:9/549: rmdir d3/d6 39 2024-04-12T20:56:05.037 INFO:tasks.workunit.client.0.smithi018.stdout:4/357: symlink d3/d5d/d60/d64/l70 0 2024-04-12T20:56:05.037 INFO:tasks.workunit.client.0.smithi018.stdout:8/428: write f0 [4041755,29050] 0 2024-04-12T20:56:05.040 INFO:tasks.workunit.client.0.smithi018.stdout:5/324: symlink d0/d2/d11/d37/l67 0 2024-04-12T20:56:05.041 INFO:tasks.workunit.client.0.smithi018.stdout:6/453: dwrite d5/df/d9b/fad [0,4194304] 0 2024-04-12T20:56:05.043 INFO:tasks.workunit.client.0.smithi018.stdout:4/358: readlink d3/d34/d3d/l58 0 2024-04-12T20:56:05.045 INFO:tasks.workunit.client.0.smithi018.stdout:9/550: unlink d3/d6/d9/dc/d6f/d77/f9d 0 2024-04-12T20:56:05.047 INFO:tasks.workunit.client.0.smithi018.stdout:5/325: write d0/d2/d33/d10/d2f/f53 [742753,10624] 0 2024-04-12T20:56:05.047 INFO:tasks.workunit.client.0.smithi018.stdout:6/454: creat d5/df/d9b/fb6 x:0 0 0 2024-04-12T20:56:05.052 INFO:tasks.workunit.client.0.smithi018.stdout:1/270: truncate d3/d9/da/dd/d31/d32/f41 242170 0 2024-04-12T20:56:05.054 INFO:tasks.workunit.client.0.smithi018.stdout:6/455: rmdir d5/d6/d1b/d3b/d8e 39 2024-04-12T20:56:05.062 INFO:tasks.workunit.client.0.smithi018.stdout:4/359: rename d3/d32/d33/l55 to d3/d5d/d60/d64/l71 0 2024-04-12T20:56:05.064 INFO:tasks.workunit.client.0.smithi018.stdout:6/456: write d5/f48 [4788888,47235] 0 2024-04-12T20:56:05.069 INFO:tasks.workunit.client.0.smithi018.stdout:9/551: truncate d3/d6/d9/d1e/f45 1262643 0 2024-04-12T20:56:05.070 INFO:tasks.workunit.client.0.smithi018.stdout:5/326: dread d0/d2/d11/d37/f47 [0,4194304] 0 2024-04-12T20:56:05.072 INFO:tasks.workunit.client.0.smithi018.stdout:3/316: dwrite d2/d6/d30/f58 [0,4194304] 0 2024-04-12T20:56:05.077 INFO:tasks.workunit.client.0.smithi018.stdout:6/457: symlink d5/d6/d1b/d3b/d35/lb7 0 2024-04-12T20:56:05.085 INFO:tasks.workunit.client.0.smithi018.stdout:9/552: fdatasync d3/d6/d9/dc/d5a/d79/d7e/d9b/d21/f59 0 2024-04-12T20:56:05.087 INFO:tasks.workunit.client.0.smithi018.stdout:6/458: read d5/d15/d60/f6d [338485,82168] 0 2024-04-12T20:56:05.092 INFO:tasks.workunit.client.0.smithi018.stdout:9/553: chown d3/d6/f3c 112 1 2024-04-12T20:56:05.101 INFO:tasks.workunit.client.0.smithi018.stdout:6/459: mkdir d5/db8 0 2024-04-12T20:56:05.103 INFO:tasks.workunit.client.0.smithi018.stdout:1/271: dwrite d3/d9/da/dd/d31/d32/d13/d19/d2d/f4d [0,4194304] 0 2024-04-12T20:56:05.107 INFO:tasks.workunit.client.0.smithi018.stdout:9/554: rename d3/d6/d9/dc/d13/f17 to d3/d6/d9/dc/d5a/fae 0 2024-04-12T20:56:05.108 INFO:tasks.workunit.client.0.smithi018.stdout:4/360: dread d3/d34/d43/f2f [0,4194304] 0 2024-04-12T20:56:05.109 INFO:tasks.workunit.client.0.smithi018.stdout:8/429: dread db/f2d [4194304,4194304] 0 2024-04-12T20:56:05.110 INFO:tasks.workunit.client.0.smithi018.stdout:3/317: dwrite d2/d6/d34/f49 [0,4194304] 0 2024-04-12T20:56:05.113 INFO:tasks.workunit.client.0.smithi018.stdout:3/318: chown d2/c1f 4793994 1 2024-04-12T20:56:05.124 INFO:tasks.workunit.client.0.smithi018.stdout:1/272: creat d3/d9/da/dd/d31/d32/d13/d19/d2d/d4e/f5b x:0 0 0 2024-04-12T20:56:05.125 INFO:tasks.workunit.client.0.smithi018.stdout:8/430: mknod db/d25/d26/d57/c81 0 2024-04-12T20:56:05.127 INFO:tasks.workunit.client.0.smithi018.stdout:1/273: read d3/f4 [7204489,9039] 0 2024-04-12T20:56:05.137 INFO:tasks.workunit.client.0.smithi018.stdout:5/327: dread d0/d2/d33/f15 [0,4194304] 0 2024-04-12T20:56:05.138 INFO:tasks.workunit.client.0.smithi018.stdout:3/319: rename d2/d43/f52 to d2/f5d 0 2024-04-12T20:56:05.142 INFO:tasks.workunit.client.0.smithi018.stdout:6/460: dread d5/df/d19/d74/f9f [0,4194304] 0 2024-04-12T20:56:05.146 INFO:tasks.workunit.client.0.smithi018.stdout:9/555: dwrite d3/d6/d9/dc/d5a/d79/d7e/f94 [0,4194304] 0 2024-04-12T20:56:05.146 INFO:tasks.workunit.client.0.smithi018.stdout:3/320: write d2/d6/d34/f59 [702198,42637] 0 2024-04-12T20:56:05.156 INFO:tasks.workunit.client.0.smithi018.stdout:9/556: write d3/d6/d1b/f35 [5224527,82412] 0 2024-04-12T20:56:05.161 INFO:tasks.workunit.client.0.smithi018.stdout:8/431: creat db/d4e/d56/d59/f82 x:0 0 0 2024-04-12T20:56:05.161 INFO:tasks.workunit.client.0.smithi018.stdout:5/328: rmdir d0/d2/d11/d37 39 2024-04-12T20:56:05.165 INFO:tasks.workunit.client.0.smithi018.stdout:9/557: chown d3/d6/l67 35968068 1 2024-04-12T20:56:05.165 INFO:tasks.workunit.client.0.smithi018.stdout:1/274: dwrite d3/d9/da/dd/d31/d32/d13/f46 [0,4194304] 0 2024-04-12T20:56:05.166 INFO:tasks.workunit.client.0.smithi018.stdout:4/361: dread d3/d34/d40/f44 [0,4194304] 0 2024-04-12T20:56:05.168 INFO:tasks.workunit.client.0.smithi018.stdout:8/432: rmdir db/d25 39 2024-04-12T20:56:05.175 INFO:tasks.workunit.client.0.smithi018.stdout:9/558: mkdir d3/d93/daf 0 2024-04-12T20:56:05.176 INFO:tasks.workunit.client.0.smithi018.stdout:3/321: rename d2/d6/lb to d2/l5e 0 2024-04-12T20:56:05.176 INFO:tasks.workunit.client.0.smithi018.stdout:8/433: dread - db/d4e/f6a zero size 2024-04-12T20:56:05.178 INFO:tasks.workunit.client.0.smithi018.stdout:4/362: mkdir d3/d5d/d60/d64/d72 0 2024-04-12T20:56:05.183 INFO:tasks.workunit.client.0.smithi018.stdout:8/434: write db/f1b [2727234,122238] 0 2024-04-12T20:56:05.185 INFO:tasks.workunit.client.0.smithi018.stdout:6/461: dwrite d5/df/d31/d4e/f38 [0,4194304] 0 2024-04-12T20:56:05.185 INFO:tasks.workunit.client.0.smithi018.stdout:4/363: write d3/f6b [574979,5437] 0 2024-04-12T20:56:05.201 INFO:tasks.workunit.client.0.smithi018.stdout:2/261: sync 2024-04-12T20:56:05.201 INFO:tasks.workunit.client.0.smithi018.stdout:4/364: write d3/db/f37 [1027263,126320] 0 2024-04-12T20:56:05.207 INFO:tasks.workunit.client.0.smithi018.stdout:1/275: rename d3/d9/da/dd/d31/d32/d13/d19/d2d/d48/d57 to d3/d9/da/dd/d31/d32/d13/d19/d2d/d4e/d58/d5c 0 2024-04-12T20:56:05.207 INFO:tasks.workunit.client.0.smithi018.stdout:2/262: rmdir d5/d7/d31 39 2024-04-12T20:56:05.212 INFO:tasks.workunit.client.0.smithi018.stdout:8/435: mknod db/d25/d26/c83 0 2024-04-12T20:56:05.215 INFO:tasks.workunit.client.0.smithi018.stdout:2/263: symlink d5/d8/l5d 0 2024-04-12T20:56:05.224 INFO:tasks.workunit.client.0.smithi018.stdout:6/462: dwrite d5/d71/fa3 [0,4194304] 0 2024-04-12T20:56:05.225 INFO:tasks.workunit.client.0.smithi018.stdout:5/329: dread d0/d2/d33/d46/d1e/d2e/f12 [0,4194304] 0 2024-04-12T20:56:05.228 INFO:tasks.workunit.client.0.smithi018.stdout:8/436: unlink db/dc/ld 0 2024-04-12T20:56:05.231 INFO:tasks.workunit.client.0.smithi018.stdout:3/322: dwrite d2/d6/d34/f59 [0,4194304] 0 2024-04-12T20:56:05.236 INFO:tasks.workunit.client.0.smithi018.stdout:4/365: dwrite d3/d32/d33/f30 [0,4194304] 0 2024-04-12T20:56:05.237 INFO:tasks.workunit.client.0.smithi018.stdout:6/463: mknod d5/df/d9b/cb9 0 2024-04-12T20:56:05.250 INFO:tasks.workunit.client.0.smithi018.stdout:9/559: dread d3/f80 [0,4194304] 0 2024-04-12T20:56:05.250 INFO:tasks.workunit.client.0.smithi018.stdout:1/276: link d3/d9/da/dd/d31/d32/f34 d3/d9/da/dd/d31/d44/f5d 0 2024-04-12T20:56:05.252 INFO:tasks.workunit.client.0.smithi018.stdout:3/323: symlink d2/d6/l5f 0 2024-04-12T20:56:05.253 INFO:tasks.workunit.client.0.smithi018.stdout:8/437: dread db/d25/d26/f3f [0,4194304] 0 2024-04-12T20:56:05.254 INFO:tasks.workunit.client.0.smithi018.stdout:2/264: getdents d5/d8 0 2024-04-12T20:56:05.255 INFO:tasks.workunit.client.0.smithi018.stdout:4/366: symlink d3/d5d/d60/d64/d72/l73 0 2024-04-12T20:56:05.257 INFO:tasks.workunit.client.0.smithi018.stdout:3/324: dread - d2/d4d/f57 zero size 2024-04-12T20:56:05.258 INFO:tasks.workunit.client.0.smithi018.stdout:5/330: dwrite d0/d2/d11/f4b [0,4194304] 0 2024-04-12T20:56:05.264 INFO:tasks.workunit.client.0.smithi018.stdout:4/367: chown d3/d34/d43/f62 811 1 2024-04-12T20:56:05.265 INFO:tasks.workunit.client.0.smithi018.stdout:1/277: mkdir d3/d9/da/dd/d31/d32/d13/d19/d2d/d4e/d5e 0 2024-04-12T20:56:05.271 INFO:tasks.workunit.client.0.smithi018.stdout:2/265: fdatasync d5/d8/d10/d19/d24/f36 0 2024-04-12T20:56:05.275 INFO:tasks.workunit.client.0.smithi018.stdout:3/325: getdents d2 0 2024-04-12T20:56:05.277 INFO:tasks.workunit.client.0.smithi018.stdout:3/326: write d2/f5 [3442460,36766] 0 2024-04-12T20:56:05.280 INFO:tasks.workunit.client.0.smithi018.stdout:4/368: mknod d3/c74 0 2024-04-12T20:56:05.284 INFO:tasks.workunit.client.0.smithi018.stdout:6/464: dwrite d5/df/d9b/fb6 [0,4194304] 0 2024-04-12T20:56:05.295 INFO:tasks.workunit.client.0.smithi018.stdout:9/560: dwrite d3/d6/d1b/f26 [0,4194304] 0 2024-04-12T20:56:05.298 INFO:tasks.workunit.client.0.smithi018.stdout:1/278: rename d3/d9/d11/f37 to d3/d9/da/f5f 0 2024-04-12T20:56:05.302 INFO:tasks.workunit.client.0.smithi018.stdout:5/331: getdents d0/d2/d33/d10/d26/d30 0 2024-04-12T20:56:05.307 INFO:tasks.workunit.client.0.smithi018.stdout:1/279: fsync d3/d9/da/dd/d31/d32/f41 0 2024-04-12T20:56:05.309 INFO:tasks.workunit.client.0.smithi018.stdout:8/438: dread db/d25/f52 [0,4194304] 0 2024-04-12T20:56:05.311 INFO:tasks.workunit.client.0.smithi018.stdout:9/561: rename d3/d6/d3d/c4a to d3/d6/d9/dc/d46/d5d/cb0 0 2024-04-12T20:56:05.312 INFO:tasks.workunit.client.0.smithi018.stdout:1/280: dread - d3/d9/da/dd/f55 zero size 2024-04-12T20:56:05.313 INFO:tasks.workunit.client.0.smithi018.stdout:8/439: fdatasync db/d25/d26/d27/f54 0 2024-04-12T20:56:05.320 INFO:tasks.workunit.client.0.smithi018.stdout:7/351: sync 2024-04-12T20:56:05.320 INFO:tasks.workunit.client.0.smithi018.stdout:0/311: sync 2024-04-12T20:56:05.320 INFO:tasks.workunit.client.0.smithi018.stdout:1/281: chown d3/d9/da/f5f 10544569 1 2024-04-12T20:56:05.320 INFO:tasks.workunit.client.0.smithi018.stdout:6/465: creat d5/d6/d1b/d3b/d8e/d8f/fba x:0 0 0 2024-04-12T20:56:05.325 INFO:tasks.workunit.client.0.smithi018.stdout:7/352: chown d1/db/d31/l48 209 1 2024-04-12T20:56:05.325 INFO:tasks.workunit.client.0.smithi018.stdout:8/440: write db/d4e/d56/d59/f82 [628284,94271] 0 2024-04-12T20:56:05.325 INFO:tasks.workunit.client.0.smithi018.stdout:9/562: mknod d3/d93/daf/cb1 0 2024-04-12T20:56:05.335 INFO:tasks.workunit.client.0.smithi018.stdout:5/332: dwrite d0/d2/d33/d46/fe [0,4194304] 0 2024-04-12T20:56:05.336 INFO:tasks.workunit.client.0.smithi018.stdout:0/312: symlink da/d15/d3e/l6a 0 2024-04-12T20:56:05.341 INFO:tasks.workunit.client.0.smithi018.stdout:1/282: write d3/f7 [8647993,68345] 0 2024-04-12T20:56:05.342 INFO:tasks.workunit.client.0.smithi018.stdout:9/563: getdents d3/d6/d9/dc/d13/d5b/d6b 0 2024-04-12T20:56:05.344 INFO:tasks.workunit.client.0.smithi018.stdout:1/283: fsync d3/d9/da/f4c 0 2024-04-12T20:56:05.347 INFO:tasks.workunit.client.0.smithi018.stdout:4/369: dread d3/f52 [0,4194304] 0 2024-04-12T20:56:05.349 INFO:tasks.workunit.client.0.smithi018.stdout:7/353: rename d1/db/d31/d45/d53/c64 to d1/db/d31/d3a/c76 0 2024-04-12T20:56:05.351 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:56:05 smithi018 ceph-mon[28799]: pgmap v35: 65 pgs: 65 active+clean; 675 MiB data, 3.7 GiB used, 533 GiB / 536 GiB avail; 6.5 MiB/s rd, 75 MiB/s wr, 188 op/s 2024-04-12T20:56:05.353 INFO:tasks.workunit.client.0.smithi018.stdout:5/333: mknod d0/d2/d33/d46/d1e/d55/c68 0 2024-04-12T20:56:05.355 INFO:tasks.workunit.client.0.smithi018.stdout:1/284: creat d3/d9/da/dd/d31/d32/d13/d19/d2d/d4e/f60 x:0 0 0 2024-04-12T20:56:05.358 INFO:tasks.workunit.client.0.smithi018.stdout:5/334: stat d0/d2/d33/c19 0 2024-04-12T20:56:05.359 INFO:tasks.workunit.client.0.smithi018.stdout:7/354: mknod d1/c77 0 2024-04-12T20:56:05.363 INFO:tasks.workunit.client.0.smithi018.stdout:5/335: readlink d0/l45 0 2024-04-12T20:56:05.367 INFO:tasks.workunit.client.0.smithi018.stdout:8/441: dread db/f15 [0,4194304] 0 2024-04-12T20:56:05.368 INFO:tasks.workunit.client.0.smithi018.stdout:4/370: creat d3/d32/d66/f75 x:0 0 0 2024-04-12T20:56:05.372 INFO:tasks.workunit.client.0.smithi018.stdout:7/355: unlink d1/db/f2b 0 2024-04-12T20:56:05.376 INFO:tasks.workunit.client.0.smithi018.stdout:0/313: dwrite da/d15/d17/d21/d36/f63 [0,4194304] 0 2024-04-12T20:56:05.377 INFO:tasks.workunit.client.0.smithi018.stdout:4/371: unlink d3/d34/l69 0 2024-04-12T20:56:05.386 INFO:tasks.workunit.client.0.smithi018.stdout:0/314: mkdir da/d15/d4d/d6b 0 2024-04-12T20:56:05.390 INFO:tasks.workunit.client.0.smithi018.stdout:0/315: chown da/d15/d3e/f5e 43187328 1 2024-04-12T20:56:05.390 INFO:tasks.workunit.client.0.smithi018.stdout:9/564: dwrite d3/d6/d9/d1e/f40 [0,4194304] 0 2024-04-12T20:56:05.390 INFO:tasks.workunit.client.0.smithi018.stdout:6/466: dread d5/fd [0,4194304] 0 2024-04-12T20:56:05.394 INFO:tasks.workunit.client.0.smithi018.stdout:9/565: readlink d3/d6/l67 0 2024-04-12T20:56:05.396 INFO:tasks.workunit.client.0.smithi018.stdout:6/467: fsync d5/d6/d1b/d3b/f91 0 2024-04-12T20:56:05.404 INFO:tasks.workunit.client.0.smithi018.stdout:1/285: dwrite d3/d9/da/dd/d31/d32/d13/d19/d2d/f4d [4194304,4194304] 0 2024-04-12T20:56:05.410 INFO:tasks.workunit.client.0.smithi018.stdout:6/468: chown d5/d6/d1b/d8c/l98 91797 1 2024-04-12T20:56:05.410 INFO:tasks.workunit.client.0.smithi018.stdout:8/442: creat db/d4e/f84 x:0 0 0 2024-04-12T20:56:05.412 INFO:tasks.workunit.client.0.smithi018.stdout:5/336: truncate d0/d2/d11/d37/f4c 1853362 0 2024-04-12T20:56:05.414 INFO:tasks.workunit.client.0.smithi018.stdout:0/316: creat da/d15/d3e/d3f/f6c x:0 0 0 2024-04-12T20:56:05.416 INFO:tasks.workunit.client.0.smithi018.stdout:8/443: read - db/d4e/f4f zero size 2024-04-12T20:56:05.416 INFO:tasks.workunit.client.0.smithi018.stdout:7/356: link d1/d6/l26 d1/db/d31/d3a/d4f/d70/l78 0 2024-04-12T20:56:05.419 INFO:tasks.workunit.client.0.smithi018.stdout:8/444: dread - db/d4e/f4f zero size 2024-04-12T20:56:05.423 INFO:tasks.workunit.client.0.smithi018.stdout:4/372: dwrite d3/d5d/d60/f67 [0,4194304] 0 2024-04-12T20:56:05.426 INFO:tasks.workunit.client.0.smithi018.stdout:7/357: fdatasync d1/f6e 0 2024-04-12T20:56:05.428 INFO:tasks.workunit.client.0.smithi018.stdout:8/445: symlink db/d4e/d56/l85 0 2024-04-12T20:56:05.435 INFO:tasks.workunit.client.0.smithi018.stdout:5/337: link d0/d2/d33/d10/d2f/f53 d0/d2/d33/d46/d35/d4d/f69 0 2024-04-12T20:56:05.438 INFO:tasks.workunit.client.0.smithi018.stdout:9/566: link d3/d6/d43/c7d d3/d6/d9/dc/d5a/cb2 0 2024-04-12T20:56:05.438 INFO:tasks.workunit.client.0.smithi018.stdout:8/446: write db/d25/d26/d27/f5f [2695937,128525] 0 2024-04-12T20:56:05.439 INFO:tasks.workunit.client.0.smithi018.stdout:0/317: write da/f13 [5536824,76501] 0 2024-04-12T20:56:05.442 INFO:tasks.workunit.client.0.smithi018.stdout:4/373: creat d3/d34/d43/f76 x:0 0 0 2024-04-12T20:56:05.446 INFO:tasks.workunit.client.0.smithi018.stdout:4/374: fsync d3/d32/d33/f18 0 2024-04-12T20:56:05.448 INFO:tasks.workunit.client.0.smithi018.stdout:8/447: fdatasync db/dc/d45/f65 0 2024-04-12T20:56:05.450 INFO:tasks.workunit.client.0.smithi018.stdout:6/469: dwrite d5/df/d19/d2b/fa5 [0,4194304] 0 2024-04-12T20:56:05.451 INFO:tasks.workunit.client.0.smithi018.stdout:8/448: readlink l4 0 2024-04-12T20:56:05.451 INFO:tasks.workunit.client.0.smithi018.stdout:7/358: truncate d1/d6/d8/f4 3143012 0 2024-04-12T20:56:05.453 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:56:05 smithi186 ceph-mon[36597]: pgmap v35: 65 pgs: 65 active+clean; 675 MiB data, 3.7 GiB used, 533 GiB / 536 GiB avail; 6.5 MiB/s rd, 75 MiB/s wr, 188 op/s 2024-04-12T20:56:05.455 INFO:tasks.workunit.client.0.smithi018.stdout:6/470: dread - d5/d6/d1b/d3b/f94 zero size 2024-04-12T20:56:05.457 INFO:tasks.workunit.client.0.smithi018.stdout:4/375: write d3/db/f2b [517030,115728] 0 2024-04-12T20:56:05.459 INFO:tasks.workunit.client.0.smithi018.stdout:9/567: creat d3/d6/d9/dc/d5a/d79/fb3 x:0 0 0 2024-04-12T20:56:05.461 INFO:tasks.workunit.client.0.smithi018.stdout:0/318: creat da/d15/d4d/d6b/f6d x:0 0 0 2024-04-12T20:56:05.461 INFO:tasks.workunit.client.0.smithi018.stdout:6/471: fdatasync d5/d6/d1b/d3b/fa4 0 2024-04-12T20:56:05.461 INFO:tasks.workunit.client.0.smithi018.stdout:8/449: mkdir db/dc/d45/d86 0 2024-04-12T20:56:05.463 INFO:tasks.workunit.client.0.smithi018.stdout:5/338: dwrite d0/d2b/f2d [0,4194304] 0 2024-04-12T20:56:05.465 INFO:tasks.workunit.client.0.smithi018.stdout:0/319: stat da/d15/d17/f19 0 2024-04-12T20:56:05.466 INFO:tasks.workunit.client.0.smithi018.stdout:9/568: read - d3/d6/d9/d1e/f74 zero size 2024-04-12T20:56:05.467 INFO:tasks.workunit.client.0.smithi018.stdout:4/376: symlink d3/d5d/l77 0 2024-04-12T20:56:05.469 INFO:tasks.workunit.client.0.smithi018.stdout:6/472: fsync d5/df/d19/d2b/f9c 0 2024-04-12T20:56:05.473 INFO:tasks.workunit.client.0.smithi018.stdout:5/339: creat d0/d2/d33/d46/d1e/f6a x:0 0 0 2024-04-12T20:56:05.473 INFO:tasks.workunit.client.0.smithi018.stdout:9/569: write d3/d6/d9/dc/d13/d5b/d6b/fa1 [11790,22417] 0 2024-04-12T20:56:05.479 INFO:tasks.workunit.client.0.smithi018.stdout:0/320: mkdir da/d15/d17/d21/d36/d6e 0 2024-04-12T20:56:05.480 INFO:tasks.workunit.client.0.smithi018.stdout:4/377: mknod d3/d34/d43/c78 0 2024-04-12T20:56:05.480 INFO:tasks.workunit.client.0.smithi018.stdout:9/570: write d3/d6/d9/dc/d5a/d79/d7e/d9b/d21/f59 [3829793,122858] 0 2024-04-12T20:56:05.482 INFO:tasks.workunit.client.0.smithi018.stdout:5/340: symlink d0/d2/d33/d10/d4e/l6b 0 2024-04-12T20:56:05.485 INFO:tasks.workunit.client.0.smithi018.stdout:4/378: fsync d3/d34/d43/d4b/f5a 0 2024-04-12T20:56:05.485 INFO:tasks.workunit.client.0.smithi018.stdout:0/321: mknod da/d15/d17/c6f 0 2024-04-12T20:56:05.486 INFO:tasks.workunit.client.0.smithi018.stdout:6/473: mkdir d5/d15/d60/dbb 0 2024-04-12T20:56:05.487 INFO:tasks.workunit.client.0.smithi018.stdout:9/571: fsync d3/d6/d9/dc/d5a/d79/d7e/d9b/d21/d41/fa5 0 2024-04-12T20:56:05.492 INFO:tasks.workunit.client.0.smithi018.stdout:7/359: rename d1/d6/d8 to d1/db/d31/d3a/d79 0 2024-04-12T20:56:05.495 INFO:tasks.workunit.client.0.smithi018.stdout:6/474: creat d5/d15/d60/fbc x:0 0 0 2024-04-12T20:56:05.497 INFO:tasks.workunit.client.0.smithi018.stdout:0/322: dread da/d15/f41 [0,4194304] 0 2024-04-12T20:56:05.499 INFO:tasks.workunit.client.0.smithi018.stdout:4/379: mkdir d3/d5d/d60/d64/d79 0 2024-04-12T20:56:05.501 INFO:tasks.workunit.client.0.smithi018.stdout:4/380: chown d3/c7 731476 1 2024-04-12T20:56:05.502 INFO:tasks.workunit.client.0.smithi018.stdout:0/323: write da/d15/d3e/d3f/f6c [681800,107461] 0 2024-04-12T20:56:05.506 INFO:tasks.workunit.client.0.smithi018.stdout:8/450: dread db/dc/f40 [4194304,4194304] 0 2024-04-12T20:56:05.506 INFO:tasks.workunit.client.0.smithi018.stdout:6/475: rename d5/d15/f39 to d5/d6/fbd 0 2024-04-12T20:56:05.507 INFO:tasks.workunit.client.0.smithi018.stdout:5/341: creat d0/d2/d33/d46/d1e/f6c x:0 0 0 2024-04-12T20:56:05.508 INFO:tasks.workunit.client.0.smithi018.stdout:8/451: symlink db/d4e/d56/l87 0 2024-04-12T20:56:05.509 INFO:tasks.workunit.client.0.smithi018.stdout:0/324: mkdir da/d15/d17/d21/d70 0 2024-04-12T20:56:05.509 INFO:tasks.workunit.client.0.smithi018.stdout:9/572: creat d3/d6/d9/dc/fb4 x:0 0 0 2024-04-12T20:56:05.510 INFO:tasks.workunit.client.0.smithi018.stdout:4/381: rmdir d3/d5d 39 2024-04-12T20:56:05.512 INFO:tasks.workunit.client.0.smithi018.stdout:0/325: chown da/d15/d17/l1c 0 1 2024-04-12T20:56:05.512 INFO:tasks.workunit.client.0.smithi018.stdout:9/573: chown d3/d6/d9 51420 1 2024-04-12T20:56:05.514 INFO:tasks.workunit.client.0.smithi018.stdout:5/342: symlink d0/d2/d33/d46/d35/l6d 0 2024-04-12T20:56:05.515 INFO:tasks.workunit.client.0.smithi018.stdout:6/476: symlink d5/d6/d1b/d3b/d22/lbe 0 2024-04-12T20:56:05.517 INFO:tasks.workunit.client.0.smithi018.stdout:0/326: write da/d15/d3e/d3f/f5b [816449,127630] 0 2024-04-12T20:56:05.518 INFO:tasks.workunit.client.0.smithi018.stdout:0/327: chown da/d15/d17/d21 33 1 2024-04-12T20:56:05.519 INFO:tasks.workunit.client.0.smithi018.stdout:4/382: mknod d3/d5d/d60/d64/d72/c7a 0 2024-04-12T20:56:05.523 INFO:tasks.workunit.client.0.smithi018.stdout:0/328: write da/f13 [1933765,129436] 0 2024-04-12T20:56:05.523 INFO:tasks.workunit.client.0.smithi018.stdout:8/452: write db/d3b/f3d [3078474,100109] 0 2024-04-12T20:56:05.525 INFO:tasks.workunit.client.0.smithi018.stdout:6/477: creat d5/d15/fbf x:0 0 0 2024-04-12T20:56:05.525 INFO:tasks.workunit.client.0.smithi018.stdout:4/383: readlink d3/db/l4a 0 2024-04-12T20:56:05.530 INFO:tasks.workunit.client.0.smithi018.stdout:6/478: write d5/df/d19/d43/d5f/fb4 [905084,79031] 0 2024-04-12T20:56:05.531 INFO:tasks.workunit.client.0.smithi018.stdout:4/384: truncate d3/d32/d66/f75 471631 0 2024-04-12T20:56:05.531 INFO:tasks.workunit.client.0.smithi018.stdout:0/329: read da/d15/d17/f19 [326055,34421] 0 2024-04-12T20:56:05.535 INFO:tasks.workunit.client.0.smithi018.stdout:0/330: fsync da/d15/d17/d21/d27/f4f 0 2024-04-12T20:56:05.536 INFO:tasks.workunit.client.0.smithi018.stdout:8/453: creat db/d4e/f88 x:0 0 0 2024-04-12T20:56:05.539 INFO:tasks.workunit.client.0.smithi018.stdout:4/385: write d3/db/f23 [970021,28015] 0 2024-04-12T20:56:05.540 INFO:tasks.workunit.client.0.smithi018.stdout:8/454: chown db/d3b/l80 297570 1 2024-04-12T20:56:05.540 INFO:tasks.workunit.client.0.smithi018.stdout:0/331: mkdir da/d15/d17/d71 0 2024-04-12T20:56:05.542 INFO:tasks.workunit.client.0.smithi018.stdout:4/386: readlink d3/d5d/d60/d64/d4f/l63 0 2024-04-12T20:56:05.546 INFO:tasks.workunit.client.0.smithi018.stdout:6/479: rename d5/df/d19/d27/d2f to d5/d6/d1b/d8c/dc0 0 2024-04-12T20:56:05.548 INFO:tasks.workunit.client.0.smithi018.stdout:4/387: creat d3/d34/d43/f7b x:0 0 0 2024-04-12T20:56:05.549 INFO:tasks.workunit.client.0.smithi018.stdout:6/480: chown d5/df/l67 2234 1 2024-04-12T20:56:05.551 INFO:tasks.workunit.client.0.smithi018.stdout:0/332: rename da/d15/d17/f19 to da/d15/d17/d21/d36/d47/f72 0 2024-04-12T20:56:05.552 INFO:tasks.workunit.client.0.smithi018.stdout:4/388: fsync d3/d34/d43/f2f 0 2024-04-12T20:56:05.553 INFO:tasks.workunit.client.0.smithi018.stdout:6/481: mkdir d5/df/d3f/dc1 0 2024-04-12T20:56:05.555 INFO:tasks.workunit.client.0.smithi018.stdout:8/455: truncate db/d3b/f3c 396521 0 2024-04-12T20:56:05.555 INFO:tasks.workunit.client.0.smithi018.stdout:0/333: read da/d15/d3e/d3f/f6c [21148,3426] 0 2024-04-12T20:56:05.556 INFO:tasks.workunit.client.0.smithi018.stdout:6/482: dread - d5/d6/d1b/d3b/f76 zero size 2024-04-12T20:56:05.557 INFO:tasks.workunit.client.0.smithi018.stdout:8/456: chown db/d25/l39 695 1 2024-04-12T20:56:05.558 INFO:tasks.workunit.client.0.smithi018.stdout:0/334: readlink da/d15/d3e/l51 0 2024-04-12T20:56:05.559 INFO:tasks.workunit.client.0.smithi018.stdout:9/574: dread d3/d6/d9/dc/d5a/d79/d7e/d9b/f34 [0,4194304] 0 2024-04-12T20:56:05.562 INFO:tasks.workunit.client.0.smithi018.stdout:4/389: rename d3/db/f4c to d3/d34/d3d/f7c 0 2024-04-12T20:56:05.566 INFO:tasks.workunit.client.0.smithi018.stdout:6/483: mknod d5/db8/cc2 0 2024-04-12T20:56:05.571 INFO:tasks.workunit.client.0.smithi018.stdout:6/484: rmdir d5/d6/d1b/d3b/d8e 39 2024-04-12T20:56:05.571 INFO:tasks.workunit.client.0.smithi018.stdout:4/390: link d3/d34/d43/f68 d3/d32/d33/f7d 0 2024-04-12T20:56:05.573 INFO:tasks.workunit.client.0.smithi018.stdout:6/485: chown d5/df/d9b/fad 0 1 2024-04-12T20:56:05.577 INFO:tasks.workunit.client.0.smithi018.stdout:8/457: write db/d3b/f55 [183237,126617] 0 2024-04-12T20:56:05.578 INFO:tasks.workunit.client.0.smithi018.stdout:0/335: write da/d15/d17/d21/d36/f4c [2404836,129032] 0 2024-04-12T20:56:05.581 INFO:tasks.workunit.client.0.smithi018.stdout:4/391: link d3/d5d/d60/d64/d72/l73 d3/d34/d53/l7e 0 2024-04-12T20:56:05.581 INFO:tasks.workunit.client.0.smithi018.stdout:9/575: dwrite d3/f80 [0,4194304] 0 2024-04-12T20:56:05.582 INFO:tasks.workunit.client.0.smithi018.stdout:0/336: dread - da/d15/d3e/d3f/f59 zero size 2024-04-12T20:56:05.585 INFO:tasks.workunit.client.0.smithi018.stdout:9/576: chown d3/d6/d9/d3f/l47 69477347 1 2024-04-12T20:56:05.589 INFO:tasks.workunit.client.0.smithi018.stdout:4/392: creat d3/db/f7f x:0 0 0 2024-04-12T20:56:05.593 INFO:tasks.workunit.client.0.smithi018.stdout:6/486: dwrite d5/df/d19/d74/f75 [0,4194304] 0 2024-04-12T20:56:05.595 INFO:tasks.workunit.client.0.smithi018.stdout:6/487: chown d5/d6/d1b/d3b 918 1 2024-04-12T20:56:05.609 INFO:tasks.workunit.client.0.smithi018.stdout:6/488: rename d5/df/d9b/fad to d5/df/d19/d43/fc3 0 2024-04-12T20:56:05.612 INFO:tasks.workunit.client.0.smithi018.stdout:6/489: chown d5/d6/d1b/d3b/d35/l34 62387879 1 2024-04-12T20:56:05.613 INFO:tasks.workunit.client.0.smithi018.stdout:6/490: chown d5/d6/f11 24906247 1 2024-04-12T20:56:05.618 INFO:tasks.workunit.client.0.smithi018.stdout:6/491: creat d5/df/d9b/fc4 x:0 0 0 2024-04-12T20:56:05.631 INFO:tasks.workunit.client.0.smithi018.stdout:8/458: dread db/dc/d45/f65 [0,4194304] 0 2024-04-12T20:56:05.635 INFO:tasks.workunit.client.0.smithi018.stdout:8/459: creat db/d4e/d7f/f89 x:0 0 0 2024-04-12T20:56:05.637 INFO:tasks.workunit.client.0.smithi018.stdout:8/460: readlink db/l32 0 2024-04-12T20:56:05.640 INFO:tasks.workunit.client.0.smithi018.stdout:8/461: creat db/d25/f8a x:0 0 0 2024-04-12T20:56:05.640 INFO:tasks.workunit.client.0.smithi018.stdout:4/393: dread d3/d5d/d60/d64/d4f/f61 [0,4194304] 0 2024-04-12T20:56:05.643 INFO:tasks.workunit.client.0.smithi018.stdout:8/462: truncate db/d25/d26/d27/f50 1055343 0 2024-04-12T20:56:05.643 INFO:tasks.workunit.client.0.smithi018.stdout:4/394: mkdir d3/d34/d3d/d80 0 2024-04-12T20:56:05.645 INFO:tasks.workunit.client.0.smithi018.stdout:8/463: chown db/f15 26082 1 2024-04-12T20:56:05.645 INFO:tasks.workunit.client.0.smithi018.stdout:4/395: chown d3/db/f7f 1991148 1 2024-04-12T20:56:05.646 INFO:tasks.workunit.client.0.smithi018.stdout:8/464: dread - db/d4e/f4f zero size 2024-04-12T20:56:05.647 INFO:tasks.workunit.client.0.smithi018.stdout:6/492: dread d5/df/d19/f54 [4194304,4194304] 0 2024-04-12T20:56:05.650 INFO:tasks.workunit.client.0.smithi018.stdout:6/493: chown d5/df/d19/d2b/fa5 1178 1 2024-04-12T20:56:05.650 INFO:tasks.workunit.client.0.smithi018.stdout:4/396: truncate d3/d34/d43/f2f 1710325 0 2024-04-12T20:56:05.656 INFO:tasks.workunit.client.0.smithi018.stdout:6/494: mkdir d5/df/d19/dc5 0 2024-04-12T20:56:05.656 INFO:tasks.workunit.client.0.smithi018.stdout:4/397: truncate d3/db/f24 5215148 0 2024-04-12T20:56:05.657 INFO:tasks.workunit.client.0.smithi018.stdout:4/398: readlink d3/d5d/l77 0 2024-04-12T20:56:05.658 INFO:tasks.workunit.client.0.smithi018.stdout:4/399: chown d3/d34/c59 278 1 2024-04-12T20:56:05.663 INFO:tasks.workunit.client.0.smithi018.stdout:8/465: dwrite db/d4e/d56/f63 [0,4194304] 0 2024-04-12T20:56:05.665 INFO:tasks.workunit.client.0.smithi018.stdout:4/400: link d3/d34/d3d/f65 d3/d32/f81 0 2024-04-12T20:56:05.667 INFO:tasks.workunit.client.0.smithi018.stdout:4/401: fdatasync d3/d34/d53/f5e 0 2024-04-12T20:56:05.674 INFO:tasks.workunit.client.0.smithi018.stdout:6/495: dwrite d5/d6/d1b/d3b/d22/f81 [0,4194304] 0 2024-04-12T20:56:05.679 INFO:tasks.workunit.client.0.smithi018.stdout:8/466: rename db/d3b/f3c to db/d25/d26/d57/d69/d6c/d7b/f8b 0 2024-04-12T20:56:05.689 INFO:tasks.workunit.client.0.smithi018.stdout:8/467: creat db/d4e/d7f/f8c x:0 0 0 2024-04-12T20:56:05.695 INFO:tasks.workunit.client.0.smithi018.stdout:8/468: creat db/d4e/d73/f8d x:0 0 0 2024-04-12T20:56:05.698 INFO:tasks.workunit.client.0.smithi018.stdout:4/402: truncate d3/db/f37 1321143 0 2024-04-12T20:56:05.698 INFO:tasks.workunit.client.0.smithi018.stdout:8/469: dread - db/d25/d26/d57/d71/f77 zero size 2024-04-12T20:56:05.701 INFO:tasks.workunit.client.0.smithi018.stdout:8/470: fdatasync db/d4e/d56/d59/f82 0 2024-04-12T20:56:05.702 INFO:tasks.workunit.client.0.smithi018.stdout:4/403: creat d3/d34/d40/f82 x:0 0 0 2024-04-12T20:56:05.704 INFO:tasks.workunit.client.0.smithi018.stdout:4/404: chown d3/d34/d3d/f57 695334 1 2024-04-12T20:56:05.704 INFO:tasks.workunit.client.0.smithi018.stdout:6/496: dread d5/df/d19/f54 [0,4194304] 0 2024-04-12T20:56:05.706 INFO:tasks.workunit.client.0.smithi018.stdout:4/405: rmdir d3/d5d/d60 39 2024-04-12T20:56:05.707 INFO:tasks.workunit.client.0.smithi018.stdout:6/497: chown d5/df/d31/d4e/f38 4 1 2024-04-12T20:56:05.707 INFO:tasks.workunit.client.0.smithi018.stdout:4/406: chown d3/db/f2b 0 1 2024-04-12T20:56:05.710 INFO:tasks.workunit.client.0.smithi018.stdout:6/498: rmdir d5/df/d19/d2b 39 2024-04-12T20:56:05.714 INFO:tasks.workunit.client.0.smithi018.stdout:4/407: chown d3/d34/d43/d4b/l6c 28865768 1 2024-04-12T20:56:05.715 INFO:tasks.workunit.client.0.smithi018.stdout:4/408: truncate d3/d34/d40/f6d 195601 0 2024-04-12T20:56:05.716 INFO:tasks.workunit.client.0.smithi018.stdout:6/499: link d5/d15/la9 d5/df/d9b/lc6 0 2024-04-12T20:56:05.718 INFO:tasks.workunit.client.0.smithi018.stdout:4/409: rename d3/d34/d43/f68 to d3/d34/d43/d4b/f83 0 2024-04-12T20:56:05.740 INFO:tasks.workunit.client.0.smithi018.stdout:6/500: dwrite d5/df/d19/f6c [0,4194304] 0 2024-04-12T20:56:05.741 INFO:tasks.workunit.client.0.smithi018.stdout:8/471: dread f7 [0,4194304] 0 2024-04-12T20:56:05.741 INFO:tasks.workunit.client.0.smithi018.stdout:4/410: dwrite d3/d34/d43/f7b [0,4194304] 0 2024-04-12T20:56:05.746 INFO:tasks.workunit.client.0.smithi018.stdout:4/411: dread - d3/d32/f4e zero size 2024-04-12T20:56:05.746 INFO:tasks.workunit.client.0.smithi018.stdout:6/501: write d5/df/d19/d2b/f95 [146232,101296] 0 2024-04-12T20:56:05.748 INFO:tasks.workunit.client.0.smithi018.stdout:8/472: mknod db/d25/d26/d57/d71/c8e 0 2024-04-12T20:56:05.753 INFO:tasks.workunit.client.0.smithi018.stdout:6/502: creat d5/df/d31/d4e/fc7 x:0 0 0 2024-04-12T20:56:05.756 INFO:tasks.workunit.client.0.smithi018.stdout:6/503: rmdir d5/d6/d1b/d3b 39 2024-04-12T20:56:05.759 INFO:tasks.workunit.client.0.smithi018.stdout:6/504: mkdir d5/df/d19/d27/dc8 0 2024-04-12T20:56:05.763 INFO:tasks.workunit.client.0.smithi018.stdout:4/412: dwrite d3/d34/d40/f44 [0,4194304] 0 2024-04-12T20:56:05.766 INFO:tasks.workunit.client.0.smithi018.stdout:4/413: stat d3/db/c38 0 2024-04-12T20:56:05.770 INFO:tasks.workunit.client.0.smithi018.stdout:6/505: link d5/df/d19/d27/fa2 d5/d6/d1b/fc9 0 2024-04-12T20:56:05.776 INFO:tasks.workunit.client.0.smithi018.stdout:6/506: mkdir d5/df/d3f/dca 0 2024-04-12T20:56:05.777 INFO:tasks.workunit.client.0.smithi018.stdout:6/507: dread - d5/df/d9b/fc4 zero size 2024-04-12T20:56:05.780 INFO:tasks.workunit.client.0.smithi018.stdout:4/414: write d3/d34/d43/f2f [733912,51612] 0 2024-04-12T20:56:05.783 INFO:tasks.workunit.client.0.smithi018.stdout:6/508: symlink d5/d6/d1b/d3b/d8e/lcb 0 2024-04-12T20:56:05.784 INFO:tasks.workunit.client.0.smithi018.stdout:4/415: creat d3/d34/d43/d4b/f84 x:0 0 0 2024-04-12T20:56:05.790 INFO:tasks.workunit.client.0.smithi018.stdout:4/416: getdents d3/d34 0 2024-04-12T20:56:05.790 INFO:tasks.workunit.client.0.smithi018.stdout:6/509: link d5/d6/d1b/d3b/d35/l34 d5/df/d3f/dc1/lcc 0 2024-04-12T20:56:05.792 INFO:tasks.workunit.client.0.smithi018.stdout:4/417: creat d3/db/f85 x:0 0 0 2024-04-12T20:56:05.795 INFO:tasks.workunit.client.0.smithi018.stdout:6/510: getdents d5/df/d19/d43 0 2024-04-12T20:56:05.795 INFO:tasks.workunit.client.0.smithi018.stdout:4/418: readlink d3/d5d/d60/d64/d72/l73 0 2024-04-12T20:56:05.797 INFO:tasks.workunit.client.0.smithi018.stdout:4/419: chown d3/d32/d33/f29 281 1 2024-04-12T20:56:05.798 INFO:tasks.workunit.client.0.smithi018.stdout:4/420: read d3/f52 [1878283,71153] 0 2024-04-12T20:56:05.807 INFO:tasks.workunit.client.0.smithi018.stdout:4/421: truncate d3/d34/d3d/f7c 3017776 0 2024-04-12T20:56:05.809 INFO:tasks.workunit.client.0.smithi018.stdout:4/422: dread - d3/d34/d53/f5e zero size 2024-04-12T20:56:05.812 INFO:tasks.workunit.client.0.smithi018.stdout:6/511: dwrite d5/d6/d1b/d3b/d35/f33 [0,4194304] 0 2024-04-12T20:56:05.820 INFO:tasks.workunit.client.0.smithi018.stdout:6/512: getdents d5/d6/d1b/d8c/dc0 0 2024-04-12T20:56:05.823 INFO:tasks.workunit.client.0.smithi018.stdout:6/513: dread - d5/df/d19/faa zero size 2024-04-12T20:56:05.824 INFO:tasks.workunit.client.0.smithi018.stdout:6/514: dread - d5/d15/fbf zero size 2024-04-12T20:56:05.826 INFO:tasks.workunit.client.0.smithi018.stdout:6/515: read d5/df/d19/d43/fc3 [2398183,4740] 0 2024-04-12T20:56:05.826 INFO:tasks.workunit.client.0.smithi018.stdout:4/423: dread d3/d34/d43/f7b [0,4194304] 0 2024-04-12T20:56:05.828 INFO:tasks.workunit.client.0.smithi018.stdout:6/516: readlink d5/d6/d1b/d3b/d35/l59 0 2024-04-12T20:56:05.832 INFO:tasks.workunit.client.0.smithi018.stdout:6/517: rename d5/df/d19/d43/d5f/l77 to d5/df/d3f/lcd 0 2024-04-12T20:56:05.837 INFO:tasks.workunit.client.0.smithi018.stdout:6/518: rename d5/df/d19/d43/d5f/f84 to d5/df/d3f/dc1/fce 0 2024-04-12T20:56:05.838 INFO:tasks.workunit.client.0.smithi018.stdout:6/519: chown d5/df/d19/d27 1 1 2024-04-12T20:56:05.839 INFO:tasks.workunit.client.0.smithi018.stdout:6/520: read - d5/d15/fbf zero size 2024-04-12T20:56:05.842 INFO:tasks.workunit.client.0.smithi018.stdout:6/521: write d5/df/d3f/dc1/fce [200282,27715] 0 2024-04-12T20:56:05.846 INFO:tasks.workunit.client.0.smithi018.stdout:6/522: dread d5/d6/d1b/d3b/f91 [0,4194304] 0 2024-04-12T20:56:05.849 INFO:tasks.workunit.client.0.smithi018.stdout:6/523: truncate d5/d6/d1b/d3b/d8e/d8f/fba 175926 0 2024-04-12T20:56:05.863 INFO:tasks.workunit.client.0.smithi018.stdout:6/524: dwrite d5/df/d19/d2b/fa5 [0,4194304] 0 2024-04-12T20:56:05.866 INFO:tasks.workunit.client.0.smithi018.stdout:6/525: write d5/df/d19/fa1 [380784,3476] 0 2024-04-12T20:56:05.867 INFO:tasks.workunit.client.0.smithi018.stdout:6/526: chown d5/d15/l26 54116027 1 2024-04-12T20:56:05.869 INFO:tasks.workunit.client.0.smithi018.stdout:6/527: creat d5/df/d3f/dca/fcf x:0 0 0 2024-04-12T20:56:05.871 INFO:tasks.workunit.client.0.smithi018.stdout:6/528: chown d5/df/d31/d4e/c32 34 1 2024-04-12T20:56:05.884 INFO:tasks.workunit.client.0.smithi018.stdout:6/529: truncate d5/df/d31/d4e/f38 3022546 0 2024-04-12T20:56:05.889 INFO:tasks.workunit.client.0.smithi018.stdout:6/530: rename d5/df/d31/d4e/c28 to d5/df/d19/d43/d5f/cd0 0 2024-04-12T20:56:05.907 INFO:tasks.workunit.client.0.smithi018.stdout:6/531: dread d5/f78 [0,4194304] 0 2024-04-12T20:56:05.910 INFO:tasks.workunit.client.0.smithi018.stdout:6/532: write d5/d6/d1b/d3b/f52 [893452,74253] 0 2024-04-12T20:56:05.912 INFO:tasks.workunit.client.0.smithi018.stdout:6/533: read d5/d6/d1b/d3b/d35/f33 [1935324,17097] 0 2024-04-12T20:56:05.914 INFO:tasks.workunit.client.0.smithi018.stdout:6/534: write d5/df/d19/d2b/f95 [1204181,16298] 0 2024-04-12T20:56:05.919 INFO:tasks.workunit.client.0.smithi018.stdout:6/535: mkdir d5/d15/dd1 0 2024-04-12T20:56:05.922 INFO:tasks.workunit.client.0.smithi018.stdout:6/536: mknod d5/d6/d1b/d3b/d6a/cd2 0 2024-04-12T20:56:05.923 INFO:tasks.workunit.client.0.smithi018.stdout:6/537: fdatasync d5/df/d19/d74/f86 0 2024-04-12T20:56:05.926 INFO:tasks.workunit.client.0.smithi018.stdout:6/538: write d5/df/d19/d74/f75 [2472128,11891] 0 2024-04-12T20:56:05.929 INFO:tasks.workunit.client.0.smithi018.stdout:6/539: fsync d5/d15/d60/f65 0 2024-04-12T20:56:05.939 INFO:tasks.workunit.client.0.smithi018.stdout:6/540: truncate d5/df/d19/d74/f75 580529 0 2024-04-12T20:56:05.941 INFO:tasks.workunit.client.0.smithi018.stdout:6/541: fsync d5/df/d19/d43/fc3 0 2024-04-12T20:56:05.944 INFO:tasks.workunit.client.0.smithi018.stdout:6/542: truncate d5/f37 1814169 0 2024-04-12T20:56:05.960 INFO:tasks.workunit.client.0.smithi018.stdout:6/543: dwrite d5/fd [0,4194304] 0 2024-04-12T20:56:05.963 INFO:tasks.workunit.client.0.smithi018.stdout:6/544: mknod d5/df/d3f/dc1/cd3 0 2024-04-12T20:56:05.965 INFO:tasks.workunit.client.0.smithi018.stdout:6/545: mkdir d5/df/d3f/dd4 0 2024-04-12T20:56:05.971 INFO:tasks.workunit.client.0.smithi018.stdout:6/546: creat d5/d6/d1b/d3b/d35/fd5 x:0 0 0 2024-04-12T20:56:05.971 INFO:tasks.workunit.client.0.smithi018.stdout:6/547: chown d5/df/d3f/dca 1 1 2024-04-12T20:56:05.973 INFO:tasks.workunit.client.0.smithi018.stdout:6/548: stat d5/df/d9b/fb6 0 2024-04-12T20:56:05.989 INFO:tasks.workunit.client.0.smithi018.stdout:6/549: dwrite d5/d15/f6f [0,4194304] 0 2024-04-12T20:56:06.011 INFO:tasks.workunit.client.0.smithi018.stdout:6/550: dwrite d5/f78 [0,4194304] 0 2024-04-12T20:56:06.017 INFO:tasks.workunit.client.0.smithi018.stdout:6/551: creat d5/df/d31/fd6 x:0 0 0 2024-04-12T20:56:06.034 INFO:tasks.workunit.client.0.smithi018.stdout:6/552: dwrite d5/df/d31/d4e/f99 [0,4194304] 0 2024-04-12T20:56:06.037 INFO:tasks.workunit.client.0.smithi018.stdout:6/553: unlink d5/d15/d60/f6d 0 2024-04-12T20:56:06.042 INFO:tasks.workunit.client.0.smithi018.stdout:6/554: creat d5/d6/d1b/d3b/fd7 x:0 0 0 2024-04-12T20:56:06.044 INFO:tasks.workunit.client.0.smithi018.stdout:6/555: write d5/d6/d1b/d3b/d35/d4f/f5c [4979433,8145] 0 2024-04-12T20:56:06.057 INFO:tasks.workunit.client.0.smithi018.stdout:3/327: sync 2024-04-12T20:56:06.057 INFO:tasks.workunit.client.0.smithi018.stdout:2/266: sync 2024-04-12T20:56:06.057 INFO:tasks.workunit.client.0.smithi018.stdout:1/286: sync 2024-04-12T20:56:06.058 INFO:tasks.workunit.client.0.smithi018.stdout:6/556: mkdir d5/d6/d1b/d3b/d6a/dd8 0 2024-04-12T20:56:06.071 INFO:tasks.workunit.client.0.smithi018.stdout:6/557: write d5/f20 [5040789,34686] 0 2024-04-12T20:56:06.072 INFO:tasks.workunit.client.0.smithi018.stdout:1/287: write d3/d9/da/dd/d31/d32/f34 [473653,51558] 0 2024-04-12T20:56:06.079 INFO:tasks.workunit.client.0.smithi018.stdout:3/328: dwrite d2/f5 [4194304,4194304] 0 2024-04-12T20:56:06.080 INFO:tasks.workunit.client.0.smithi018.stdout:2/267: dread f4 [0,4194304] 0 2024-04-12T20:56:06.081 INFO:tasks.workunit.client.0.smithi018.stdout:3/329: chown d2/l3f 7913 1 2024-04-12T20:56:06.085 INFO:tasks.workunit.client.0.smithi018.stdout:2/268: write d5/f1d [2897211,13491] 0 2024-04-12T20:56:06.088 INFO:tasks.workunit.client.0.smithi018.stdout:3/330: creat d2/d6/d3d/f60 x:0 0 0 2024-04-12T20:56:06.092 INFO:tasks.workunit.client.0.smithi018.stdout:6/558: dwrite d5/df/d3f/f44 [0,4194304] 0 2024-04-12T20:56:06.100 INFO:tasks.workunit.client.0.smithi018.stdout:6/559: getdents d5/d6/d1b/d3b/d8e 0 2024-04-12T20:56:06.103 INFO:tasks.workunit.client.0.smithi018.stdout:6/560: write d5/d15/d60/f65 [648219,29125] 0 2024-04-12T20:56:06.104 INFO:tasks.workunit.client.0.smithi018.stdout:3/331: dread d2/d6/d30/f58 [0,4194304] 0 2024-04-12T20:56:06.109 INFO:tasks.workunit.client.0.smithi018.stdout:2/269: dwrite d5/d12/f58 [0,4194304] 0 2024-04-12T20:56:06.109 INFO:tasks.workunit.client.0.smithi018.stdout:6/561: creat d5/df/d19/fd9 x:0 0 0 2024-04-12T20:56:06.122 INFO:tasks.workunit.client.0.smithi018.stdout:2/270: chown d5/d8/d10/d19/d24/d39/f45 256848505 1 2024-04-12T20:56:06.126 INFO:tasks.workunit.client.0.smithi018.stdout:2/271: chown d5/d7/d28/f59 0 1 2024-04-12T20:56:06.130 INFO:tasks.workunit.client.0.smithi018.stdout:3/332: dwrite d2/d6/f53 [0,4194304] 0 2024-04-12T20:56:06.134 INFO:tasks.workunit.client.0.smithi018.stdout:6/562: dwrite d5/f83 [0,4194304] 0 2024-04-12T20:56:06.135 INFO:tasks.workunit.client.0.smithi018.stdout:3/333: truncate d2/d6/d30/f37 815586 0 2024-04-12T20:56:06.137 INFO:tasks.workunit.client.0.smithi018.stdout:2/272: symlink d5/d8/d33/l5e 0 2024-04-12T20:56:06.142 INFO:tasks.workunit.client.0.smithi018.stdout:2/273: unlink d5/c6 0 2024-04-12T20:56:06.149 INFO:tasks.workunit.client.0.smithi018.stdout:2/274: dread d5/d8/d10/d19/d24/d39/f45 [0,4194304] 0 2024-04-12T20:56:06.152 INFO:tasks.workunit.client.0.smithi018.stdout:2/275: dread - d5/d12/f5b zero size 2024-04-12T20:56:06.154 INFO:tasks.workunit.client.0.smithi018.stdout:3/334: dwrite d2/d6/d34/f49 [0,4194304] 0 2024-04-12T20:56:06.156 INFO:tasks.workunit.client.0.smithi018.stdout:3/335: chown d2/d6/d3d/d54 6662 1 2024-04-12T20:56:06.158 INFO:tasks.workunit.client.0.smithi018.stdout:3/336: dread - d2/d6/d30/d31/d38/d3a/d48/f5c zero size 2024-04-12T20:56:06.160 INFO:tasks.workunit.client.0.smithi018.stdout:2/276: rmdir d5/d8/d10/d19/d24 39 2024-04-12T20:56:06.163 INFO:tasks.workunit.client.0.smithi018.stdout:2/277: creat d5/d7/d28/f5f x:0 0 0 2024-04-12T20:56:06.165 INFO:tasks.workunit.client.0.smithi018.stdout:2/278: chown d5/d7/d28/l46 1557464588 1 2024-04-12T20:56:06.177 INFO:tasks.workunit.client.0.smithi018.stdout:3/337: dwrite d2/d6/f1b [0,4194304] 0 2024-04-12T20:56:06.178 INFO:tasks.workunit.client.0.smithi018.stdout:2/279: write d5/d7/d31/f4f [3538492,124124] 0 2024-04-12T20:56:06.182 INFO:tasks.workunit.client.0.smithi018.stdout:3/338: write d2/d6/d30/d31/d38/d3a/f45 [605795,93344] 0 2024-04-12T20:56:06.182 INFO:tasks.workunit.client.0.smithi018.stdout:2/280: read d5/d7/f4d [210722,111824] 0 2024-04-12T20:56:06.185 INFO:tasks.workunit.client.0.smithi018.stdout:3/339: mkdir d2/d43/d61 0 2024-04-12T20:56:06.189 INFO:tasks.workunit.client.0.smithi018.stdout:3/340: rename d2/d6/d34/f49 to d2/d43/d61/f62 0 2024-04-12T20:56:06.192 INFO:tasks.workunit.client.0.smithi018.stdout:3/341: write d2/d6/f24 [4472889,46254] 0 2024-04-12T20:56:06.194 INFO:tasks.workunit.client.0.smithi018.stdout:3/342: chown d2/d6/d34/f59 44 1 2024-04-12T20:56:06.200 INFO:tasks.workunit.client.0.smithi018.stdout:3/343: rmdir d2/d6 39 2024-04-12T20:56:06.202 INFO:tasks.workunit.client.0.smithi018.stdout:2/281: dwrite d5/d7/d28/f59 [0,4194304] 0 2024-04-12T20:56:06.206 INFO:tasks.workunit.client.0.smithi018.stdout:3/344: creat d2/d43/d61/f63 x:0 0 0 2024-04-12T20:56:06.224 INFO:tasks.workunit.client.0.smithi018.stdout:2/282: dwrite d5/d8/d10/f48 [4194304,4194304] 0 2024-04-12T20:56:06.230 INFO:tasks.workunit.client.0.smithi018.stdout:3/345: dread d2/f28 [0,4194304] 0 2024-04-12T20:56:06.233 INFO:tasks.workunit.client.0.smithi018.stdout:3/346: chown d2/f1a 0 1 2024-04-12T20:56:06.235 INFO:tasks.workunit.client.0.smithi018.stdout:3/347: fdatasync d2/d6/d34/f46 0 2024-04-12T20:56:06.239 INFO:tasks.workunit.client.0.smithi018.stdout:3/348: write d2/d6/d30/f58 [950124,107823] 0 2024-04-12T20:56:06.243 INFO:tasks.workunit.client.0.smithi018.stdout:3/349: mkdir d2/d43/d61/d64 0 2024-04-12T20:56:06.244 INFO:tasks.workunit.client.0.smithi018.stdout:2/283: dread d5/d7/f35 [0,4194304] 0 2024-04-12T20:56:06.244 INFO:tasks.workunit.client.0.smithi018.stdout:3/350: stat d2/d6/d30/f37 0 2024-04-12T20:56:06.247 INFO:tasks.workunit.client.0.smithi018.stdout:2/284: unlink d5/f1d 0 2024-04-12T20:56:06.248 INFO:tasks.workunit.client.0.smithi018.stdout:3/351: dread d2/f42 [0,4194304] 0 2024-04-12T20:56:06.249 INFO:tasks.workunit.client.0.smithi018.stdout:2/285: mknod d5/d8/c60 0 2024-04-12T20:56:06.252 INFO:tasks.workunit.client.0.smithi018.stdout:3/352: symlink d2/d6/d30/d31/d38/d3a/d4c/l65 0 2024-04-12T20:56:06.254 INFO:tasks.workunit.client.0.smithi018.stdout:3/353: fdatasync d2/d6/d30/f37 0 2024-04-12T20:56:06.257 INFO:tasks.workunit.client.0.smithi018.stdout:3/354: creat d2/d6/d30/d31/d38/f66 x:0 0 0 2024-04-12T20:56:06.266 INFO:tasks.workunit.client.0.smithi018.stdout:3/355: write d2/f19 [2893662,51572] 0 2024-04-12T20:56:06.268 INFO:tasks.workunit.client.0.smithi018.stdout:5/343: sync 2024-04-12T20:56:06.268 INFO:tasks.workunit.client.0.smithi018.stdout:7/360: sync 2024-04-12T20:56:06.269 INFO:tasks.workunit.client.0.smithi018.stdout:0/337: sync 2024-04-12T20:56:06.278 INFO:tasks.workunit.client.0.smithi018.stdout:3/356: rename d2/d6/f53 to d2/d6/d30/d31/f67 0 2024-04-12T20:56:06.287 INFO:tasks.workunit.client.0.smithi018.stdout:2/286: dread d5/d8/d10/d19/d24/f32 [0,4194304] 0 2024-04-12T20:56:06.287 INFO:tasks.workunit.client.0.smithi018.stdout:3/357: creat d2/d6/d30/d31/d38/d3a/f68 x:0 0 0 2024-04-12T20:56:06.291 INFO:tasks.workunit.client.0.smithi018.stdout:3/358: dread - d2/d43/d61/f63 zero size 2024-04-12T20:56:06.296 INFO:tasks.workunit.client.0.smithi018.stdout:7/361: dwrite d1/db/d31/d3a/d42/f75 [0,4194304] 0 2024-04-12T20:56:06.296 INFO:tasks.workunit.client.0.smithi018.stdout:2/287: write d5/d7/f35 [1867130,11450] 0 2024-04-12T20:56:06.298 INFO:tasks.workunit.client.0.smithi018.stdout:5/344: dwrite d0/d2/d11/f4b [0,4194304] 0 2024-04-12T20:56:06.299 INFO:tasks.workunit.client.0.smithi018.stdout:3/359: write d2/d43/d61/f63 [156775,22301] 0 2024-04-12T20:56:06.300 INFO:tasks.workunit.client.0.smithi018.stdout:2/288: chown d5/d8/d10/f48 432 1 2024-04-12T20:56:06.302 INFO:tasks.workunit.client.0.smithi018.stdout:0/338: dwrite da/d15/d17/f43 [0,4194304] 0 2024-04-12T20:56:06.305 INFO:tasks.workunit.client.0.smithi018.stdout:3/360: fdatasync d2/d6/d3d/f5a 0 2024-04-12T20:56:06.309 INFO:tasks.workunit.client.0.smithi018.stdout:0/339: rename da/d15/f35 to da/d15/d3e/f73 0 2024-04-12T20:56:06.317 INFO:tasks.workunit.client.0.smithi018.stdout:0/340: symlink da/d15/d3e/d3f/l74 0 2024-04-12T20:56:06.319 INFO:tasks.workunit.client.0.smithi018.stdout:3/361: write d2/d6/d30/f3e [2913078,90371] 0 2024-04-12T20:56:06.324 INFO:tasks.workunit.client.0.smithi018.stdout:0/341: creat da/f75 x:0 0 0 2024-04-12T20:56:06.329 INFO:tasks.workunit.client.0.smithi018.stdout:7/362: dread d1/d6/f27 [0,4194304] 0 2024-04-12T20:56:06.330 INFO:tasks.workunit.client.0.smithi018.stdout:0/342: truncate da/f75 685903 0 2024-04-12T20:56:06.330 INFO:tasks.workunit.client.0.smithi018.stdout:2/289: dwrite d5/d8/d10/d19/d24/d39/f51 [0,4194304] 0 2024-04-12T20:56:06.332 INFO:tasks.workunit.client.0.smithi018.stdout:3/362: creat d2/d6/d30/d31/f69 x:0 0 0 2024-04-12T20:56:06.334 INFO:tasks.workunit.client.0.smithi018.stdout:0/343: creat da/d15/d3e/f76 x:0 0 0 2024-04-12T20:56:06.335 INFO:tasks.workunit.client.0.smithi018.stdout:3/363: write d2/f19 [2790864,17395] 0 2024-04-12T20:56:06.335 INFO:tasks.workunit.client.0.smithi018.stdout:7/363: mknod d1/db/d31/d3a/d4f/c7a 0 2024-04-12T20:56:06.338 INFO:tasks.workunit.client.0.smithi018.stdout:5/345: write d0/d2/d11/d37/f4c [2275241,11742] 0 2024-04-12T20:56:06.340 INFO:tasks.workunit.client.0.smithi018.stdout:3/364: dread - d2/d6/d30/d31/f69 zero size 2024-04-12T20:56:06.341 INFO:tasks.workunit.client.0.smithi018.stdout:0/344: symlink da/d15/d3e/d3f/l77 0 2024-04-12T20:56:06.346 INFO:tasks.workunit.client.0.smithi018.stdout:3/365: write d2/d6/d30/d31/d38/d3a/f68 [645576,46804] 0 2024-04-12T20:56:06.348 INFO:tasks.workunit.client.0.smithi018.stdout:5/346: symlink d0/d2/d33/d46/d1e/d55/l6e 0 2024-04-12T20:56:06.351 INFO:tasks.workunit.client.0.smithi018.stdout:5/347: chown d0/d2/d33/c5f 0 1 2024-04-12T20:56:06.353 INFO:tasks.workunit.client.0.smithi018.stdout:7/364: link d1/l4e d1/db/d31/d3a/d4f/d70/l7b 0 2024-04-12T20:56:06.356 INFO:tasks.workunit.client.0.smithi018.stdout:0/345: link da/d15/d3e/d3f/f5b da/d15/d17/d21/d27/f78 0 2024-04-12T20:56:06.357 INFO:tasks.workunit.client.0.smithi018.stdout:7/365: chown d1/db/d31/d45/d53/f56 1459034 1 2024-04-12T20:56:06.357 INFO:tasks.workunit.client.0.smithi018.stdout:2/290: truncate d5/d8/d10/f30 3962462 0 2024-04-12T20:56:06.358 INFO:tasks.workunit.client.0.smithi018.stdout:3/366: dread d2/d6/d30/d31/f50 [0,4194304] 0 2024-04-12T20:56:06.359 INFO:tasks.workunit.client.0.smithi018.stdout:5/348: mknod d0/d2/d33/d61/c6f 0 2024-04-12T20:56:06.362 INFO:tasks.workunit.client.0.smithi018.stdout:7/366: fsync d1/d6/f24 0 2024-04-12T20:56:06.365 INFO:tasks.workunit.client.0.smithi018.stdout:0/346: dread da/d15/d3e/f5e [0,4194304] 0 2024-04-12T20:56:06.365 INFO:tasks.workunit.client.0.smithi018.stdout:4/424: sync 2024-04-12T20:56:06.365 INFO:tasks.workunit.client.0.smithi018.stdout:9/577: sync 2024-04-12T20:56:06.365 INFO:tasks.workunit.client.0.smithi018.stdout:8/473: sync 2024-04-12T20:56:06.367 INFO:tasks.workunit.client.0.smithi018.stdout:3/367: rmdir d2/d6/d30/d31/d38/d3a/d4c 39 2024-04-12T20:56:06.369 INFO:tasks.workunit.client.0.smithi018.stdout:2/291: creat d5/f61 x:0 0 0 2024-04-12T20:56:06.370 INFO:tasks.workunit.client.0.smithi018.stdout:5/349: write d0/d2/d33/d46/d1e/f42 [587921,117344] 0 2024-04-12T20:56:06.370 INFO:tasks.workunit.client.0.smithi018.stdout:7/367: readlink d1/db/d31/d3a/d79/l62 0 2024-04-12T20:56:06.372 INFO:tasks.workunit.client.0.smithi018.stdout:4/425: mknod d3/d5d/d60/d64/c86 0 2024-04-12T20:56:06.372 INFO:tasks.workunit.client.0.smithi018.stdout:2/292: fdatasync d5/d7/d28/f5f 0 2024-04-12T20:56:06.372 INFO:tasks.workunit.client.0.smithi018.stdout:3/368: chown d2/c29 92 1 2024-04-12T20:56:06.372 INFO:tasks.workunit.client.0.smithi018.stdout:8/474: symlink db/d25/d26/d57/d69/d6c/l8f 0 2024-04-12T20:56:06.376 INFO:tasks.workunit.client.0.smithi018.stdout:5/350: fdatasync d0/d2/d33/d46/d18/f59 0 2024-04-12T20:56:06.377 INFO:tasks.workunit.client.0.smithi018.stdout:2/293: mkdir d5/d7/d31/d62 0 2024-04-12T20:56:06.378 INFO:tasks.workunit.client.0.smithi018.stdout:9/578: fdatasync d3/d6/d9/dc/d5a/d79/f87 0 2024-04-12T20:56:06.381 INFO:tasks.workunit.client.0.smithi018.stdout:2/294: write d5/d12/d54/f52 [3791867,77170] 0 2024-04-12T20:56:06.382 INFO:tasks.workunit.client.0.smithi018.stdout:4/426: symlink d3/d34/d43/d4b/l87 0 2024-04-12T20:56:06.385 INFO:tasks.workunit.client.0.smithi018.stdout:9/579: write d3/d6/d9/dc/d6f/f9a [1172268,34768] 0 2024-04-12T20:56:06.385 INFO:tasks.workunit.client.0.smithi018.stdout:8/475: creat db/d4e/d56/d59/d76/f90 x:0 0 0 2024-04-12T20:56:06.389 INFO:tasks.workunit.client.0.smithi018.stdout:7/368: link d1/db/d31/f37 d1/db/d31/d45/f7c 0 2024-04-12T20:56:06.393 INFO:tasks.workunit.client.0.smithi018.stdout:5/351: mknod d0/d2/d33/d10/d26/c70 0 2024-04-12T20:56:06.395 INFO:tasks.workunit.client.0.smithi018.stdout:8/476: truncate db/d4e/f6b 5220328 0 2024-04-12T20:56:06.396 INFO:tasks.workunit.client.0.smithi018.stdout:4/427: rename d3/d32/c5c to d3/d34/d3d/d80/c88 0 2024-04-12T20:56:06.400 INFO:tasks.workunit.client.0.smithi018.stdout:5/352: stat d0/d2/d33/c19 0 2024-04-12T20:56:06.400 INFO:tasks.workunit.client.0.smithi018.stdout:9/580: truncate d3/d6/d9/f82 501775 0 2024-04-12T20:56:06.404 INFO:tasks.workunit.client.0.smithi018.stdout:9/581: fdatasync d3/d6/d9/d3f/f6e 0 2024-04-12T20:56:06.404 INFO:tasks.workunit.client.0.smithi018.stdout:7/369: creat d1/db/d31/d45/d53/f7d x:0 0 0 2024-04-12T20:56:06.407 INFO:tasks.workunit.client.0.smithi018.stdout:0/347: dwrite da/d15/d4d/d6b/f6d [0,4194304] 0 2024-04-12T20:56:06.407 INFO:tasks.workunit.client.0.smithi018.stdout:5/353: write d0/d2/d33/d46/d18/f3b [452300,93644] 0 2024-04-12T20:56:06.410 INFO:tasks.workunit.client.0.smithi018.stdout:7/370: stat d1/db/d31 0 2024-04-12T20:56:06.411 INFO:tasks.workunit.client.0.smithi018.stdout:7/371: readlink d1/db/l1e 0 2024-04-12T20:56:06.421 INFO:tasks.workunit.client.0.smithi018.stdout:4/428: truncate d3/db/f37 839754 0 2024-04-12T20:56:06.429 INFO:tasks.workunit.client.0.smithi018.stdout:7/372: truncate d1/db/d31/d3a/d42/d4c/f72 450965 0 2024-04-12T20:56:06.433 INFO:tasks.workunit.client.0.smithi018.stdout:3/369: dwrite d2/d6/f9 [0,4194304] 0 2024-04-12T20:56:06.433 INFO:tasks.workunit.client.0.smithi018.stdout:5/354: dread d0/d2/d33/d46/d1e/f42 [0,4194304] 0 2024-04-12T20:56:06.436 INFO:tasks.workunit.client.0.smithi018.stdout:7/373: write d1/d6/f67 [87866,73890] 0 2024-04-12T20:56:06.446 INFO:tasks.workunit.client.0.smithi018.stdout:8/477: dread db/d25/d26/d57/f60 [0,4194304] 0 2024-04-12T20:56:06.447 INFO:tasks.workunit.client.0.smithi018.stdout:5/355: unlink d0/d2b/f2d 0 2024-04-12T20:56:06.451 INFO:tasks.workunit.client.0.smithi018.stdout:0/348: truncate da/d15/f41 1035578 0 2024-04-12T20:56:06.455 INFO:tasks.workunit.client.0.smithi018.stdout:8/478: rmdir db/d25 39 2024-04-12T20:56:06.459 INFO:tasks.workunit.client.0.smithi018.stdout:2/295: truncate d5/d8/d10/f30 4448409 0 2024-04-12T20:56:06.461 INFO:tasks.workunit.client.0.smithi018.stdout:5/356: creat d0/d2/d33/d46/d18/f71 x:0 0 0 2024-04-12T20:56:06.464 INFO:tasks.workunit.client.0.smithi018.stdout:0/349: mkdir da/d15/d17/d5c/d79 0 2024-04-12T20:56:06.465 INFO:tasks.workunit.client.0.smithi018.stdout:7/374: creat d1/db/d31/d3a/d4f/d69/f7e x:0 0 0 2024-04-12T20:56:06.466 INFO:tasks.workunit.client.0.smithi018.stdout:4/429: getdents d3/d34/d3d/d80 0 2024-04-12T20:56:06.467 INFO:tasks.workunit.client.0.smithi018.stdout:8/479: symlink db/d3b/l91 0 2024-04-12T20:56:06.467 INFO:tasks.workunit.client.0.smithi018.stdout:5/357: write d0/d2/d33/d10/d26/f4a [3077621,124198] 0 2024-04-12T20:56:06.472 INFO:tasks.workunit.client.0.smithi018.stdout:3/370: dwrite d2/f42 [0,4194304] 0 2024-04-12T20:56:06.472 INFO:tasks.workunit.client.0.smithi018.stdout:0/350: mknod da/d15/d17/d21/c7a 0 2024-04-12T20:56:06.476 INFO:tasks.workunit.client.0.smithi018.stdout:2/296: mkdir d5/d7/d63 0 2024-04-12T20:56:06.479 INFO:tasks.workunit.client.0.smithi018.stdout:0/351: creat da/d15/d4d/d6b/f7b x:0 0 0 2024-04-12T20:56:06.479 INFO:tasks.workunit.client.0.smithi018.stdout:8/480: read fa [518877,60680] 0 2024-04-12T20:56:06.480 INFO:tasks.workunit.client.0.smithi018.stdout:2/297: write d5/d7/f37 [1787374,100097] 0 2024-04-12T20:56:06.480 INFO:tasks.workunit.client.0.smithi018.stdout:5/358: fdatasync d0/d2/d33/d46/d35/d4d/f69 0 2024-04-12T20:56:06.482 INFO:tasks.workunit.client.0.smithi018.stdout:3/371: mknod d2/d4d/c6a 0 2024-04-12T20:56:06.486 INFO:tasks.workunit.client.0.smithi018.stdout:5/359: chown d0/d2/d33/d46/d1e/f42 9428811 1 2024-04-12T20:56:06.488 INFO:tasks.workunit.client.0.smithi018.stdout:8/481: write db/d3b/f4d [3980532,47708] 0 2024-04-12T20:56:06.490 INFO:tasks.workunit.client.0.smithi018.stdout:0/352: creat da/d15/d17/d21/d36/f7c x:0 0 0 2024-04-12T20:56:06.492 INFO:tasks.workunit.client.0.smithi018.stdout:2/298: rmdir d5/d3a 0 2024-04-12T20:56:06.494 INFO:tasks.workunit.client.0.smithi018.stdout:3/372: mkdir d2/d6/d30/d31/d38/d3a/d4c/d6b 0 2024-04-12T20:56:06.495 INFO:tasks.workunit.client.0.smithi018.stdout:2/299: fdatasync d5/d7/d28/f5f 0 2024-04-12T20:56:06.496 INFO:tasks.workunit.client.0.smithi018.stdout:5/360: symlink d0/d2/d33/d46/d35/l72 0 2024-04-12T20:56:06.498 INFO:tasks.workunit.client.0.smithi018.stdout:4/430: write d3/d34/d43/d4b/f83 [296354,45178] 0 2024-04-12T20:56:06.500 INFO:tasks.workunit.client.0.smithi018.stdout:2/300: write d5/d7/d31/f4f [8732877,11458] 0 2024-04-12T20:56:06.501 INFO:tasks.workunit.client.0.smithi018.stdout:8/482: link db/d25/d26/d27/f50 db/d4e/d73/f92 0 2024-04-12T20:56:06.503 INFO:tasks.workunit.client.0.smithi018.stdout:3/373: truncate d2/d6/d30/d31/f50 790314 0 2024-04-12T20:56:06.505 INFO:tasks.workunit.client.0.smithi018.stdout:2/301: creat d5/d7/d31/f64 x:0 0 0 2024-04-12T20:56:06.506 INFO:tasks.workunit.client.0.smithi018.stdout:5/361: creat d0/d2/d33/d10/d60/f73 x:0 0 0 2024-04-12T20:56:06.507 INFO:tasks.workunit.client.0.smithi018.stdout:4/431: fdatasync d3/db/f26 0 2024-04-12T20:56:06.508 INFO:tasks.workunit.client.0.smithi018.stdout:8/483: creat db/d4e/d7f/f93 x:0 0 0 2024-04-12T20:56:06.510 INFO:tasks.workunit.client.0.smithi018.stdout:3/374: mknod d2/d4d/c6c 0 2024-04-12T20:56:06.518 INFO:tasks.workunit.client.0.smithi018.stdout:8/484: creat db/d25/d26/d57/d71/f94 x:0 0 0 2024-04-12T20:56:06.523 INFO:tasks.workunit.client.0.smithi018.stdout:0/353: dread da/d15/f20 [0,4194304] 0 2024-04-12T20:56:06.524 INFO:tasks.workunit.client.0.smithi018.stdout:2/302: creat d5/d7/d63/f65 x:0 0 0 2024-04-12T20:56:06.527 INFO:tasks.workunit.client.0.smithi018.stdout:3/375: fsync d2/d6/d30/d31/f50 0 2024-04-12T20:56:06.533 INFO:tasks.workunit.client.0.smithi018.stdout:8/485: mkdir db/d95 0 2024-04-12T20:56:06.534 INFO:tasks.workunit.client.0.smithi018.stdout:4/432: dwrite d3/d34/d40/f44 [0,4194304] 0 2024-04-12T20:56:06.534 INFO:tasks.workunit.client.0.smithi018.stdout:2/303: rename d5/f61 to d5/d7/d31/d62/f66 0 2024-04-12T20:56:06.538 INFO:tasks.workunit.client.0.smithi018.stdout:4/433: write d3/d5d/f6e [858099,104082] 0 2024-04-12T20:56:06.538 INFO:tasks.workunit.client.0.smithi018.stdout:5/362: dread d0/d2/d33/d10/d26/f4a [0,4194304] 0 2024-04-12T20:56:06.543 INFO:tasks.workunit.client.0.smithi018.stdout:5/363: write d0/d2/d33/d10/f27 [756471,46537] 0 2024-04-12T20:56:06.549 INFO:tasks.workunit.client.0.smithi018.stdout:4/434: dread d3/d34/d40/f6d [0,4194304] 0 2024-04-12T20:56:06.549 INFO:tasks.workunit.client.0.smithi018.stdout:8/486: rename db/d3b/c41 to db/d25/d26/d27/c96 0 2024-04-12T20:56:06.552 INFO:tasks.workunit.client.0.smithi018.stdout:2/304: creat d5/d8/d10/d19/d24/f67 x:0 0 0 2024-04-12T20:56:06.552 INFO:tasks.workunit.client.0.smithi018.stdout:5/364: truncate d0/d2/d33/d46/d18/f71 961542 0 2024-04-12T20:56:06.555 INFO:tasks.workunit.client.0.smithi018.stdout:4/435: creat d3/d5d/d60/d64/d72/f89 x:0 0 0 2024-04-12T20:56:06.557 INFO:tasks.workunit.client.0.smithi018.stdout:2/305: dread - d5/d7/d63/f65 zero size 2024-04-12T20:56:06.557 INFO:tasks.workunit.client.0.smithi018.stdout:3/376: dwrite d2/d6/d30/d31/d38/d3a/d48/f51 [4194304,4194304] 0 2024-04-12T20:56:06.558 INFO:tasks.workunit.client.0.smithi018.stdout:8/487: rmdir db/d4e 39 2024-04-12T20:56:06.561 INFO:tasks.workunit.client.0.smithi018.stdout:4/436: mknod d3/d5d/d60/d64/c8a 0 2024-04-12T20:56:06.561 INFO:tasks.workunit.client.0.smithi018.stdout:8/488: read db/dc/d45/f65 [4368599,73926] 0 2024-04-12T20:56:06.562 INFO:tasks.workunit.client.0.smithi018.stdout:5/365: read d0/d2/d33/f15 [2193895,15616] 0 2024-04-12T20:56:06.567 INFO:tasks.workunit.client.0.smithi018.stdout:5/366: read d0/d2/d33/d46/f3a [305253,29558] 0 2024-04-12T20:56:06.569 INFO:tasks.workunit.client.0.smithi018.stdout:2/306: mknod d5/d7/c68 0 2024-04-12T20:56:06.571 INFO:tasks.workunit.client.0.smithi018.stdout:2/307: readlink d5/d8/d10/d19/l27 0 2024-04-12T20:56:06.571 INFO:tasks.workunit.client.0.smithi018.stdout:0/354: dread da/d15/d17/d21/d27/f78 [0,4194304] 0 2024-04-12T20:56:06.574 INFO:tasks.workunit.client.0.smithi018.stdout:2/308: write d5/d7/f5c [374311,86044] 0 2024-04-12T20:56:06.575 INFO:tasks.workunit.client.0.smithi018.stdout:3/377: truncate d2/f28 3663011 0 2024-04-12T20:56:06.575 INFO:tasks.workunit.client.0.smithi018.stdout:0/355: write da/d15/d17/d21/f5f [742227,35703] 0 2024-04-12T20:56:06.578 INFO:tasks.workunit.client.0.smithi018.stdout:3/378: read d2/d6/d30/f58 [2219858,128676] 0 2024-04-12T20:56:06.579 INFO:tasks.workunit.client.0.smithi018.stdout:5/367: mkdir d0/d2/d11/d74 0 2024-04-12T20:56:06.579 INFO:tasks.workunit.client.0.smithi018.stdout:3/379: chown d2/d4d 10 1 2024-04-12T20:56:06.581 INFO:tasks.workunit.client.0.smithi018.stdout:4/437: link d3/d5d/d60/f67 d3/db/f8b 0 2024-04-12T20:56:06.582 INFO:tasks.workunit.client.0.smithi018.stdout:8/489: dread db/dc/f40 [4194304,4194304] 0 2024-04-12T20:56:06.583 INFO:tasks.workunit.client.0.smithi018.stdout:2/309: unlink d5/l41 0 2024-04-12T20:56:06.584 INFO:tasks.workunit.client.0.smithi018.stdout:3/380: write d2/d6/d30/d31/d38/f3b [963579,35574] 0 2024-04-12T20:56:06.587 INFO:tasks.workunit.client.0.smithi018.stdout:0/356: mknod da/d15/d17/d21/d36/c7d 0 2024-04-12T20:56:06.587 INFO:tasks.workunit.client.0.smithi018.stdout:5/368: rmdir d0/d2b 39 2024-04-12T20:56:06.589 INFO:tasks.workunit.client.0.smithi018.stdout:6/563: sync 2024-04-12T20:56:06.589 INFO:tasks.workunit.client.0.smithi018.stdout:1/288: sync 2024-04-12T20:56:06.593 INFO:tasks.workunit.client.0.smithi018.stdout:5/369: dread - d0/d5b/f64 zero size 2024-04-12T20:56:06.593 INFO:tasks.workunit.client.0.smithi018.stdout:6/564: fdatasync d5/df/d31/fd6 0 2024-04-12T20:56:06.596 INFO:tasks.workunit.client.0.smithi018.stdout:1/289: fsync d3/d9/da/dd/d31/d32/d13/d19/d2d/d4e/f4f 0 2024-04-12T20:56:06.596 INFO:tasks.workunit.client.0.smithi018.stdout:6/565: fdatasync d5/d6/d1b/d3b/f69 0 2024-04-12T20:56:06.596 INFO:tasks.workunit.client.0.smithi018.stdout:0/357: creat da/d15/d3e/d3f/f7e x:0 0 0 2024-04-12T20:56:06.598 INFO:tasks.workunit.client.0.smithi018.stdout:0/358: stat da/d15/d17/d21/d36 0 2024-04-12T20:56:06.599 INFO:tasks.workunit.client.0.smithi018.stdout:8/490: truncate db/dc/f40 1820828 0 2024-04-12T20:56:06.600 INFO:tasks.workunit.client.0.smithi018.stdout:6/566: mkdir d5/d6/d1b/d3b/d35/dda 0 2024-04-12T20:56:06.602 INFO:tasks.workunit.client.0.smithi018.stdout:4/438: rename d3/d34/d3d/f7c to d3/d34/d3d/d80/f8c 0 2024-04-12T20:56:06.602 INFO:tasks.workunit.client.0.smithi018.stdout:5/370: dread d0/d2/d33/d46/d1e/f42 [0,4194304] 0 2024-04-12T20:56:06.610 INFO:tasks.workunit.client.0.smithi018.stdout:6/567: mknod d5/df/d19/d27/dc8/cdb 0 2024-04-12T20:56:06.612 INFO:tasks.workunit.client.0.smithi018.stdout:2/310: rename d5/d8/d25/f34 to d5/d8/d10/f69 0 2024-04-12T20:56:06.613 INFO:tasks.workunit.client.0.smithi018.stdout:6/568: chown d5/df/f93 28018488 1 2024-04-12T20:56:06.614 INFO:tasks.workunit.client.0.smithi018.stdout:1/290: mkdir d3/d9/da/dd/d31/d32/d13/d19/d2d/d4e/d58/d61 0 2024-04-12T20:56:06.615 INFO:tasks.workunit.client.0.smithi018.stdout:3/381: link d2/d6/d30/d31/d38/d3a/d4c/l65 d2/d43/d61/d64/l6d 0 2024-04-12T20:56:06.618 INFO:tasks.workunit.client.0.smithi018.stdout:5/371: getdents d0/d2/d33/d46/d1e/d55/d66 0 2024-04-12T20:56:06.618 INFO:tasks.workunit.client.0.smithi018.stdout:1/291: fsync d3/d9/da/dd/d31/d32/d13/d19/d2d/f4d 0 2024-04-12T20:56:06.624 INFO:tasks.workunit.client.0.smithi018.stdout:6/569: creat d5/d6/d1b/d3b/fdc x:0 0 0 2024-04-12T20:56:06.624 INFO:tasks.workunit.client.0.smithi018.stdout:0/359: rmdir da/d15/d17/d50 0 2024-04-12T20:56:06.627 INFO:tasks.workunit.client.0.smithi018.stdout:2/311: symlink d5/d8/d10/d19/d24/l6a 0 2024-04-12T20:56:06.634 INFO:tasks.workunit.client.0.smithi018.stdout:3/382: rename d2/d4d/c6c to d2/d6/d30/d31/d38/d3a/d48/c6e 0 2024-04-12T20:56:06.634 INFO:tasks.workunit.client.0.smithi018.stdout:4/439: getdents d3/d34/d43 0 2024-04-12T20:56:06.640 INFO:tasks.workunit.client.0.smithi018.stdout:1/292: unlink d3/d9/da/dd/d31/d32/d13/d19/l1e 0 2024-04-12T20:56:06.641 INFO:tasks.workunit.client.0.smithi018.stdout:8/491: dread db/d4e/f6b [0,4194304] 0 2024-04-12T20:56:06.644 INFO:tasks.workunit.client.0.smithi018.stdout:2/312: symlink d5/d8/d10/d19/d24/d39/l6b 0 2024-04-12T20:56:06.645 INFO:tasks.workunit.client.0.smithi018.stdout:4/440: symlink d3/d34/d40/l8d 0 2024-04-12T20:56:06.645 INFO:tasks.workunit.client.0.smithi018.stdout:1/293: dread - d3/d9/da/dd/d31/f59 zero size 2024-04-12T20:56:06.648 INFO:tasks.workunit.client.0.smithi018.stdout:0/360: creat da/d15/d17/d21/d36/d6e/f7f x:0 0 0 2024-04-12T20:56:06.653 INFO:tasks.workunit.client.0.smithi018.stdout:8/492: dread db/d3b/f55 [0,4194304] 0 2024-04-12T20:56:06.653 INFO:tasks.workunit.client.0.smithi018.stdout:2/313: readlink d5/d8/d10/d19/l27 0 2024-04-12T20:56:06.654 INFO:tasks.workunit.client.0.smithi018.stdout:5/372: dread d0/d2/d33/d46/d35/f62 [0,4194304] 0 2024-04-12T20:56:06.655 INFO:tasks.workunit.client.0.smithi018.stdout:6/570: dwrite d5/d6/d1b/d3b/f52 [0,4194304] 0 2024-04-12T20:56:06.657 INFO:tasks.workunit.client.0.smithi018.stdout:3/383: creat d2/d6/d30/f6f x:0 0 0 2024-04-12T20:56:06.660 INFO:tasks.workunit.client.0.smithi018.stdout:0/361: write da/d15/f33 [2797992,127859] 0 2024-04-12T20:56:06.661 INFO:tasks.workunit.client.0.smithi018.stdout:8/493: chown db/d25/d26/d57/d69/d6c/l8f 3784 1 2024-04-12T20:56:06.662 INFO:tasks.workunit.client.0.smithi018.stdout:1/294: creat d3/d9/da/dd/d31/d32/d13/d19/f62 x:0 0 0 2024-04-12T20:56:06.665 INFO:tasks.workunit.client.0.smithi018.stdout:3/384: mkdir d2/d6/d3d/d70 0 2024-04-12T20:56:06.666 INFO:tasks.workunit.client.0.smithi018.stdout:2/314: mkdir d5/d8/d10/d19/d24/d6c 0 2024-04-12T20:56:06.667 INFO:tasks.workunit.client.0.smithi018.stdout:6/571: creat d5/d6/d1b/d3b/d8e/fdd x:0 0 0 2024-04-12T20:56:06.670 INFO:tasks.workunit.client.0.smithi018.stdout:5/373: creat d0/d2/d33/d46/d1e/d2e/f75 x:0 0 0 2024-04-12T20:56:06.672 INFO:tasks.workunit.client.0.smithi018.stdout:8/494: symlink db/d4e/l97 0 2024-04-12T20:56:06.674 INFO:tasks.workunit.client.0.smithi018.stdout:3/385: mkdir d2/d6/d34/d71 0 2024-04-12T20:56:06.675 INFO:tasks.workunit.client.0.smithi018.stdout:2/315: symlink d5/d8/d25/l6d 0 2024-04-12T20:56:06.676 INFO:tasks.workunit.client.0.smithi018.stdout:4/441: dwrite d3/db/f26 [0,4194304] 0 2024-04-12T20:56:06.678 INFO:tasks.workunit.client.0.smithi018.stdout:8/495: fdatasync db/d25/d26/d57/d71/f94 0 2024-04-12T20:56:06.679 INFO:tasks.workunit.client.0.smithi018.stdout:2/316: write d5/d8/d10/d43/f56 [983703,129512] 0 2024-04-12T20:56:06.680 INFO:tasks.workunit.client.0.smithi018.stdout:6/572: rename d5/df/f2a to d5/d6/d1b/d3b/d6a/dd8/fde 0 2024-04-12T20:56:06.681 INFO:tasks.workunit.client.0.smithi018.stdout:1/295: truncate d3/f10 4183358 0 2024-04-12T20:56:06.682 INFO:tasks.workunit.client.0.smithi018.stdout:6/573: chown d5/d15/d60/dbb 11 1 2024-04-12T20:56:06.683 INFO:tasks.workunit.client.0.smithi018.stdout:5/374: symlink d0/d2/d33/d10/d26/l76 0 2024-04-12T20:56:06.684 INFO:tasks.workunit.client.0.smithi018.stdout:4/442: mkdir d3/d32/d33/d8e 0 2024-04-12T20:56:06.685 INFO:tasks.workunit.client.0.smithi018.stdout:8/496: truncate db/f15 1410574 0 2024-04-12T20:56:06.687 INFO:tasks.workunit.client.0.smithi018.stdout:0/362: getdents da/d15/d3e 0 2024-04-12T20:56:06.688 INFO:tasks.workunit.client.0.smithi018.stdout:2/317: mknod d5/d8/c6e 0 2024-04-12T20:56:06.688 INFO:tasks.workunit.client.0.smithi018.stdout:6/574: symlink d5/d6/d1b/d3b/d22/ldf 0 2024-04-12T20:56:06.689 INFO:tasks.workunit.client.0.smithi018.stdout:5/375: dread - d0/d2/d33/d10/d26/d30/f65 zero size 2024-04-12T20:56:06.692 INFO:tasks.workunit.client.0.smithi018.stdout:0/363: chown da/d15/d3e/l4b 60258869 1 2024-04-12T20:56:06.693 INFO:tasks.workunit.client.0.smithi018.stdout:8/497: creat db/dc/d45/f98 x:0 0 0 2024-04-12T20:56:06.696 INFO:tasks.workunit.client.0.smithi018.stdout:4/443: unlink d3/d5d/d60/d64/d4f/l63 0 2024-04-12T20:56:06.701 INFO:tasks.workunit.client.0.smithi018.stdout:6/575: unlink d5/df/d3f/lcd 0 2024-04-12T20:56:06.701 INFO:tasks.workunit.client.0.smithi018.stdout:8/498: symlink db/d3b/l99 0 2024-04-12T20:56:06.707 INFO:tasks.workunit.client.0.smithi018.stdout:5/376: mknod d0/d2/d33/d46/c77 0 2024-04-12T20:56:06.708 INFO:tasks.workunit.client.0.smithi018.stdout:8/499: chown db/d72/f7d 15969 1 2024-04-12T20:56:06.710 INFO:tasks.workunit.client.0.smithi018.stdout:4/444: unlink d3/d5d/d60/d64/d72/c7a 0 2024-04-12T20:56:06.712 INFO:tasks.workunit.client.0.smithi018.stdout:6/576: chown d5/d6/d1b/d3b/d22/lbe 408 1 2024-04-12T20:56:06.712 INFO:tasks.workunit.client.0.smithi018.stdout:8/500: write db/f1b [53181,845] 0 2024-04-12T20:56:06.713 INFO:tasks.workunit.client.0.smithi018.stdout:3/386: dwrite d2/f5 [0,4194304] 0 2024-04-12T20:56:06.718 INFO:tasks.workunit.client.0.smithi018.stdout:5/377: symlink d0/d5b/l78 0 2024-04-12T20:56:06.725 INFO:tasks.workunit.client.0.smithi018.stdout:6/577: symlink d5/d71/le0 0 2024-04-12T20:56:06.725 INFO:tasks.workunit.client.0.smithi018.stdout:5/378: rmdir d0/d2/d11/d37 39 2024-04-12T20:56:06.725 INFO:tasks.workunit.client.0.smithi018.stdout:0/364: dread da/d15/f20 [4194304,4194304] 0 2024-04-12T20:56:06.726 INFO:tasks.workunit.client.0.smithi018.stdout:1/296: truncate d3/f10 3216199 0 2024-04-12T20:56:06.730 INFO:tasks.workunit.client.0.smithi018.stdout:2/318: dwrite d5/d7/d31/f64 [0,4194304] 0 2024-04-12T20:56:06.730 INFO:tasks.workunit.client.0.smithi018.stdout:1/297: symlink d3/d9/da/dd/d31/l63 0 2024-04-12T20:56:06.732 INFO:tasks.workunit.client.0.smithi018.stdout:6/578: chown d5/d6/d1b/l42 1164 1 2024-04-12T20:56:06.737 INFO:tasks.workunit.client.0.smithi018.stdout:5/379: chown d0/d2/d33/d10/d4e/l6b 462061744 1 2024-04-12T20:56:06.742 INFO:tasks.workunit.client.0.smithi018.stdout:0/365: creat da/d15/d17/d21/d36/f80 x:0 0 0 2024-04-12T20:56:06.745 INFO:tasks.workunit.client.0.smithi018.stdout:2/319: write d5/d8/d10/d19/d24/d39/f51 [4669396,4884] 0 2024-04-12T20:56:06.745 INFO:tasks.workunit.client.0.smithi018.stdout:1/298: write d3/d9/da/dd/d31/f45 [800495,11554] 0 2024-04-12T20:56:06.747 INFO:tasks.workunit.client.0.smithi018.stdout:8/501: dread db/f64 [0,4194304] 0 2024-04-12T20:56:06.752 INFO:tasks.workunit.client.0.smithi018.stdout:6/579: write d5/df/d19/d2b/f9c [2189293,809] 0 2024-04-12T20:56:06.753 INFO:tasks.workunit.client.0.smithi018.stdout:5/380: read d0/d2/d33/d10/f39 [1603103,10435] 0 2024-04-12T20:56:06.757 INFO:tasks.workunit.client.0.smithi018.stdout:1/299: fdatasync d3/d9/da/dd/d31/d32/d13/d19/f62 0 2024-04-12T20:56:06.760 INFO:tasks.workunit.client.0.smithi018.stdout:4/445: dwrite d3/db/f8b [0,4194304] 0 2024-04-12T20:56:06.762 INFO:tasks.workunit.client.0.smithi018.stdout:1/300: dread - d3/d9/da/dd/d31/f59 zero size 2024-04-12T20:56:06.767 INFO:tasks.workunit.client.0.smithi018.stdout:5/381: rename d0/d2/d33/d46/fe to d0/d2/d33/d46/d35/f79 0 2024-04-12T20:56:06.770 INFO:tasks.workunit.client.0.smithi018.stdout:8/502: dread db/dc/f40 [0,4194304] 0 2024-04-12T20:56:06.773 INFO:tasks.workunit.client.0.smithi018.stdout:0/366: truncate da/d15/f33 3430137 0 2024-04-12T20:56:06.773 INFO:tasks.workunit.client.0.smithi018.stdout:2/320: truncate f4 1505947 0 2024-04-12T20:56:06.779 INFO:tasks.workunit.client.0.smithi018.stdout:5/382: write d0/d2/d33/d46/d1e/f42 [1081402,95149] 0 2024-04-12T20:56:06.781 INFO:tasks.workunit.client.0.smithi018.stdout:2/321: mknod d5/d8/d10/c6f 0 2024-04-12T20:56:06.787 INFO:tasks.workunit.client.0.smithi018.stdout:6/580: dwrite d5/d6/d1b/d3b/fdc [0,4194304] 0 2024-04-12T20:56:06.796 INFO:tasks.workunit.client.0.smithi018.stdout:6/581: truncate d5/df/d19/fd9 145849 0 2024-04-12T20:56:06.801 INFO:tasks.workunit.client.0.smithi018.stdout:8/503: dwrite db/d25/f8a [0,4194304] 0 2024-04-12T20:56:06.805 INFO:tasks.workunit.client.0.smithi018.stdout:8/504: dread - db/dc/d45/f79 zero size 2024-04-12T20:56:06.813 INFO:tasks.workunit.client.0.smithi018.stdout:0/367: dwrite da/d15/d4d/d6b/f6d [0,4194304] 0 2024-04-12T20:56:06.819 INFO:tasks.workunit.client.0.smithi018.stdout:6/582: dread d5/df/d19/f6c [0,4194304] 0 2024-04-12T20:56:06.819 INFO:tasks.workunit.client.0.smithi018.stdout:2/322: truncate d5/d8/d10/d43/f56 525867 0 2024-04-12T20:56:06.821 INFO:tasks.workunit.client.0.smithi018.stdout:0/368: read da/d15/d3e/d3f/f6c [591104,47516] 0 2024-04-12T20:56:06.822 INFO:tasks.workunit.client.0.smithi018.stdout:2/323: chown d5/d8/d10/d19/d24/d39/f45 422830611 1 2024-04-12T20:56:06.826 INFO:tasks.workunit.client.0.smithi018.stdout:2/324: creat d5/d8/d10/f70 x:0 0 0 2024-04-12T20:56:06.842 INFO:tasks.workunit.client.0.smithi018.stdout:6/583: link d5/df/d31/ca0 d5/df/d3f/dc1/ce1 0 2024-04-12T20:56:06.844 INFO:tasks.workunit.client.0.smithi018.stdout:1/301: dread d3/f4 [0,4194304] 0 2024-04-12T20:56:06.844 INFO:tasks.workunit.client.0.smithi018.stdout:8/505: dread db/d4e/d56/f63 [0,4194304] 0 2024-04-12T20:56:06.844 INFO:tasks.workunit.client.0.smithi018.stdout:6/584: fsync d5/d71/fa3 0 2024-04-12T20:56:06.845 INFO:tasks.workunit.client.0.smithi018.stdout:2/325: dread d5/d7/d31/f4f [4194304,4194304] 0 2024-04-12T20:56:06.850 INFO:tasks.workunit.client.0.smithi018.stdout:8/506: write db/d4e/d73/f8d [1025386,111753] 0 2024-04-12T20:56:06.852 INFO:tasks.workunit.client.0.smithi018.stdout:2/326: read d5/d8/d10/f30 [1806052,94955] 0 2024-04-12T20:56:06.854 INFO:tasks.workunit.client.0.smithi018.stdout:5/383: dread d0/d2/d33/d46/d35/f4f [0,4194304] 0 2024-04-12T20:56:06.854 INFO:tasks.workunit.client.0.smithi018.stdout:8/507: chown db/d4e/d56/d59/d76/l7a 60720198 1 2024-04-12T20:56:06.859 INFO:tasks.workunit.client.0.smithi018.stdout:6/585: symlink d5/df/le2 0 2024-04-12T20:56:06.860 INFO:tasks.workunit.client.0.smithi018.stdout:2/327: symlink d5/d8/d10/d19/l71 0 2024-04-12T20:56:06.861 INFO:tasks.workunit.client.0.smithi018.stdout:5/384: creat d0/d2/d33/d46/d18/f7a x:0 0 0 2024-04-12T20:56:06.862 INFO:tasks.workunit.client.0.smithi018.stdout:8/508: truncate db/d25/d26/d27/f50 1533529 0 2024-04-12T20:56:06.865 INFO:tasks.workunit.client.0.smithi018.stdout:1/302: dread d3/d9/da/dd/d31/f45 [0,4194304] 0 2024-04-12T20:56:06.865 INFO:tasks.workunit.client.0.smithi018.stdout:6/586: mknod d5/d6/d1b/d3b/d35/ce3 0 2024-04-12T20:56:06.865 INFO:tasks.workunit.client.0.smithi018.stdout:2/328: creat d5/d7/d31/d62/f72 x:0 0 0 2024-04-12T20:56:06.868 INFO:tasks.workunit.client.0.smithi018.stdout:8/509: chown db/c58 1918 1 2024-04-12T20:56:06.871 INFO:tasks.workunit.client.0.smithi018.stdout:5/385: rmdir d0/d2/d11 39 2024-04-12T20:56:06.871 INFO:tasks.workunit.client.0.smithi018.stdout:8/510: read - db/d25/d26/d57/d71/f77 zero size 2024-04-12T20:56:06.874 INFO:tasks.workunit.client.0.smithi018.stdout:2/329: rename d5/d8/c60 to d5/d8/d10/d19/d24/d6c/c73 0 2024-04-12T20:56:06.880 INFO:tasks.workunit.client.0.smithi018.stdout:1/303: mknod d3/d9/da/dd/d31/d32/c64 0 2024-04-12T20:56:06.883 INFO:tasks.workunit.client.0.smithi018.stdout:5/386: mknod d0/d2/d33/d46/d35/c7b 0 2024-04-12T20:56:06.886 INFO:tasks.workunit.client.0.smithi018.stdout:1/304: truncate d3/d9/da/dd/f4a 654298 0 2024-04-12T20:56:06.888 INFO:tasks.workunit.client.0.smithi018.stdout:6/587: dwrite d5/d15/fbf [0,4194304] 0 2024-04-12T20:56:06.891 INFO:tasks.workunit.client.0.smithi018.stdout:6/588: dread - d5/d6/d1b/d3b/fd7 zero size 2024-04-12T20:56:06.895 INFO:tasks.workunit.client.0.smithi018.stdout:1/305: mkdir d3/d65 0 2024-04-12T20:56:06.898 INFO:tasks.workunit.client.0.smithi018.stdout:2/330: truncate d5/d12/d54/f52 3649208 0 2024-04-12T20:56:06.902 INFO:tasks.workunit.client.0.smithi018.stdout:6/589: creat d5/d15/dd1/fe4 x:0 0 0 2024-04-12T20:56:06.905 INFO:tasks.workunit.client.0.smithi018.stdout:5/387: mknod d0/d2/d33/d46/d1e/c7c 0 2024-04-12T20:56:06.910 INFO:tasks.workunit.client.0.smithi018.stdout:1/306: unlink d3/d9/da/dd/d31/d32/d13/d19/d2d/f40 0 2024-04-12T20:56:06.910 INFO:tasks.workunit.client.0.smithi018.stdout:2/331: rmdir d5/d7/d63 39 2024-04-12T20:56:06.912 INFO:tasks.workunit.client.0.smithi018.stdout:8/511: dwrite db/d25/d26/d27/f50 [0,4194304] 0 2024-04-12T20:56:06.913 INFO:tasks.workunit.client.0.smithi018.stdout:6/590: symlink d5/df/d3f/dd4/le5 0 2024-04-12T20:56:06.918 INFO:tasks.workunit.client.0.smithi018.stdout:1/307: mkdir d3/d9/da/dd/d31/d32/d13/d19/d2d/d48/d66 0 2024-04-12T20:56:06.937 INFO:tasks.workunit.client.0.smithi018.stdout:6/591: dwrite d5/d6/d1b/d3b/fa4 [0,4194304] 0 2024-04-12T20:56:06.944 INFO:tasks.workunit.client.0.smithi018.stdout:8/512: dwrite db/d25/f34 [4194304,4194304] 0 2024-04-12T20:56:06.946 INFO:tasks.workunit.client.0.smithi018.stdout:2/332: getdents d5/d8 0 2024-04-12T20:56:06.947 INFO:tasks.workunit.client.0.smithi018.stdout:1/308: rename d3/d9/da/dd/d31/c52 to d3/d9/da/dd/d31/d32/c67 0 2024-04-12T20:56:06.949 INFO:tasks.workunit.client.0.smithi018.stdout:5/388: dread d0/d2/d11/d37/f4c [0,4194304] 0 2024-04-12T20:56:06.951 INFO:tasks.workunit.client.0.smithi018.stdout:6/592: dread d5/df/d19/d27/fa2 [0,4194304] 0 2024-04-12T20:56:06.952 INFO:tasks.workunit.client.0.smithi018.stdout:1/309: write d3/d9/da/f5f [3947799,4032] 0 2024-04-12T20:56:06.953 INFO:tasks.workunit.client.0.smithi018.stdout:8/513: write db/d25/d26/d27/f33 [1294619,50908] 0 2024-04-12T20:56:06.954 INFO:tasks.workunit.client.0.smithi018.stdout:8/514: fdatasync f0 0 2024-04-12T20:56:06.956 INFO:tasks.workunit.client.0.smithi018.stdout:2/333: fdatasync f4 0 2024-04-12T20:56:06.958 INFO:tasks.workunit.client.0.smithi018.stdout:6/593: mkdir d5/df/d19/d43/de6 0 2024-04-12T20:56:06.961 INFO:tasks.workunit.client.0.smithi018.stdout:1/310: readlink d3/d9/d11/l54 0 2024-04-12T20:56:06.965 INFO:tasks.workunit.client.0.smithi018.stdout:6/594: creat d5/d6/d1b/d3b/d22/fe7 x:0 0 0 2024-04-12T20:56:06.968 INFO:tasks.workunit.client.0.smithi018.stdout:2/334: symlink d5/d12/l74 0 2024-04-12T20:56:06.968 INFO:tasks.workunit.client.0.smithi018.stdout:6/595: dread - d5/d15/fa6 zero size 2024-04-12T20:56:06.971 INFO:tasks.workunit.client.0.smithi018.stdout:1/311: mkdir d3/d9/da/dd/d31/d32/d13/d19/d2d/d4e/d58/d68 0 2024-04-12T20:56:06.973 INFO:tasks.workunit.client.0.smithi018.stdout:5/389: dwrite d0/d2/d33/d46/d1e/f6c [0,4194304] 0 2024-04-12T20:56:06.973 INFO:tasks.workunit.client.0.smithi018.stdout:6/596: write d5/d15/dd1/fe4 [197365,117396] 0 2024-04-12T20:56:06.976 INFO:tasks.workunit.client.0.smithi018.stdout:8/515: rename db/d25/d26/d57 to db/d4e/d9a 0 2024-04-12T20:56:06.977 INFO:tasks.workunit.client.0.smithi018.stdout:6/597: stat d5/d6/d1b/d3b/d8e/lcb 0 2024-04-12T20:56:06.977 INFO:tasks.workunit.client.0.smithi018.stdout:5/390: rmdir d0 39 2024-04-12T20:56:06.978 INFO:tasks.workunit.client.0.smithi018.stdout:6/598: chown d5/df/d19/d74 2435 1 2024-04-12T20:56:06.979 INFO:tasks.workunit.client.0.smithi018.stdout:2/335: rename d5/d8/c6e to d5/d8/d10/c75 0 2024-04-12T20:56:06.980 INFO:tasks.workunit.client.0.smithi018.stdout:6/599: fsync d5/df/d31/fd6 0 2024-04-12T20:56:06.983 INFO:tasks.workunit.client.0.smithi018.stdout:1/312: rename d3/d9/da/dd/c49 to d3/d9/da/dd/d31/d32/d33/d47/c69 0 2024-04-12T20:56:06.984 INFO:tasks.workunit.client.0.smithi018.stdout:5/391: read d0/d2/d33/d10/d26/d30/f31 [5799151,23841] 0 2024-04-12T20:56:06.986 INFO:tasks.workunit.client.0.smithi018.stdout:2/336: creat d5/d8/d33/f76 x:0 0 0 2024-04-12T20:56:06.989 INFO:tasks.workunit.client.0.smithi018.stdout:6/600: rename d5/d6/d1b/d3b/d35/d4f/f5b to d5/d15/fe8 0 2024-04-12T20:56:06.989 INFO:tasks.workunit.client.0.smithi018.stdout:2/337: fdatasync d5/d8/d10/f70 0 2024-04-12T20:56:06.994 INFO:tasks.workunit.client.0.smithi018.stdout:1/313: dread d3/d9/f3c [0,4194304] 0 2024-04-12T20:56:06.997 INFO:tasks.workunit.client.0.smithi018.stdout:2/338: rename d5/d7/f35 to d5/d8/d10/d19/d24/d39/f77 0 2024-04-12T20:56:06.997 INFO:tasks.workunit.client.0.smithi018.stdout:5/392: truncate d0/d2/d33/d46/d35/d4d/f69 264504 0 2024-04-12T20:56:06.999 INFO:tasks.workunit.client.0.smithi018.stdout:5/393: stat d0/d2b/d40 0 2024-04-12T20:56:07.000 INFO:tasks.workunit.client.0.smithi018.stdout:6/601: getdents d5/d6/d1b/d3b/d22 0 2024-04-12T20:56:07.019 INFO:tasks.workunit.client.0.smithi018.stdout:9/582: sync 2024-04-12T20:56:07.019 INFO:tasks.workunit.client.0.smithi018.stdout:7/375: sync 2024-04-12T20:56:07.021 INFO:tasks.workunit.client.0.smithi018.stdout:2/339: creat d5/d8/d10/f78 x:0 0 0 2024-04-12T20:56:07.025 INFO:tasks.workunit.client.0.smithi018.stdout:2/340: dread - d5/d7/d31/d62/f72 zero size 2024-04-12T20:56:07.026 INFO:tasks.workunit.client.0.smithi018.stdout:1/314: rename d3/d9/da/dd/d31/l63 to d3/d9/da/dd/d31/d44/l6a 0 2024-04-12T20:56:07.028 INFO:tasks.workunit.client.0.smithi018.stdout:5/394: creat d0/d2/d33/d10/d26/d30/f7d x:0 0 0 2024-04-12T20:56:07.029 INFO:tasks.workunit.client.0.smithi018.stdout:8/516: truncate db/d3b/f55 136494 0 2024-04-12T20:56:07.032 INFO:tasks.workunit.client.0.smithi018.stdout:6/602: dwrite d5/d6/d1b/d3b/d35/d4f/fb0 [0,4194304] 0 2024-04-12T20:56:07.035 INFO:tasks.workunit.client.0.smithi018.stdout:2/341: mkdir d5/d7/d79 0 2024-04-12T20:56:07.037 INFO:tasks.workunit.client.0.smithi018.stdout:7/376: write d1/f2f [921379,37558] 0 2024-04-12T20:56:07.037 INFO:tasks.workunit.client.0.smithi018.stdout:5/395: creat d0/d2/d33/d10/d26/f7e x:0 0 0 2024-04-12T20:56:07.038 INFO:tasks.workunit.client.0.smithi018.stdout:8/517: mknod db/d4e/d56/c9b 0 2024-04-12T20:56:07.044 INFO:tasks.workunit.client.0.smithi018.stdout:6/603: read d5/d15/f6f [1846483,124068] 0 2024-04-12T20:56:07.044 INFO:tasks.workunit.client.0.smithi018.stdout:1/315: creat d3/d9/da/dd/d31/d32/d13/d19/d2d/d4e/d58/d68/f6b x:0 0 0 2024-04-12T20:56:07.049 INFO:tasks.workunit.client.0.smithi018.stdout:8/518: symlink db/dc/l9c 0 2024-04-12T20:56:07.049 INFO:tasks.workunit.client.0.smithi018.stdout:5/396: mkdir d0/d2/d33/d46/d1e/d2e/d7f 0 2024-04-12T20:56:07.049 INFO:tasks.workunit.client.0.smithi018.stdout:2/342: getdents d5/d12/d54/d3e 0 2024-04-12T20:56:07.052 INFO:tasks.workunit.client.0.smithi018.stdout:9/583: dwrite d3/d6/d9/dc/d5a/d79/d7e/f94 [0,4194304] 0 2024-04-12T20:56:07.054 INFO:tasks.workunit.client.0.smithi018.stdout:5/397: chown d0/d2/d33/d10/d26/d30 2144114 1 2024-04-12T20:56:07.058 INFO:tasks.workunit.client.0.smithi018.stdout:2/343: fsync d5/d7/d31/d62/f66 0 2024-04-12T20:56:07.058 INFO:tasks.workunit.client.0.smithi018.stdout:9/584: write d3/d6/d9/dc/d5a/d79/d7e/d9b/d21/d41/d78/f7a [2257762,31636] 0 2024-04-12T20:56:07.063 INFO:tasks.workunit.client.0.smithi018.stdout:8/519: creat db/dc/d45/d86/f9d x:0 0 0 2024-04-12T20:56:07.069 INFO:tasks.workunit.client.0.smithi018.stdout:2/344: creat d5/d7/d31/f7a x:0 0 0 2024-04-12T20:56:07.071 INFO:tasks.workunit.client.0.smithi018.stdout:2/345: readlink d5/l40 0 2024-04-12T20:56:07.072 INFO:tasks.workunit.client.0.smithi018.stdout:6/604: dread d5/d6/d1b/d3b/fdc [0,4194304] 0 2024-04-12T20:56:07.077 INFO:tasks.workunit.client.0.smithi018.stdout:2/346: write d5/d7/f37 [634143,8677] 0 2024-04-12T20:56:07.081 INFO:tasks.workunit.client.0.smithi018.stdout:8/520: link db/d4e/d73/f8d db/d4e/d7f/f9e 0 2024-04-12T20:56:07.083 INFO:tasks.workunit.client.0.smithi018.stdout:6/605: unlink d5/df/d19/d74/f86 0 2024-04-12T20:56:07.086 INFO:tasks.workunit.client.0.smithi018.stdout:7/377: dwrite d1/d6/f27 [0,4194304] 0 2024-04-12T20:56:07.093 INFO:tasks.workunit.client.0.smithi018.stdout:9/585: dwrite d3/d6/d9/dc/d13/f42 [0,4194304] 0 2024-04-12T20:56:07.093 INFO:tasks.workunit.client.0.smithi018.stdout:2/347: rename d5/d8/d10/f30 to d5/d8/d10/d19/f7b 0 2024-04-12T20:56:07.094 INFO:tasks.workunit.client.0.smithi018.stdout:6/606: write d5/d6/d1b/d3b/f94 [86220,78335] 0 2024-04-12T20:56:07.097 INFO:tasks.workunit.client.0.smithi018.stdout:6/607: fdatasync d5/f10 0 2024-04-12T20:56:07.101 INFO:tasks.workunit.client.0.smithi018.stdout:8/521: link db/d25/d26/d27/f33 db/d4e/d9a/d71/f9f 0 2024-04-12T20:56:07.101 INFO:tasks.workunit.client.0.smithi018.stdout:9/586: fdatasync d3/d6/d9/dc/d5a/fae 0 2024-04-12T20:56:07.103 INFO:tasks.workunit.client.0.smithi018.stdout:7/378: creat d1/db/d31/d3a/d4f/d69/f7f x:0 0 0 2024-04-12T20:56:07.105 INFO:tasks.workunit.client.0.smithi018.stdout:5/398: dread d0/d2/d33/d46/f13 [0,4194304] 0 2024-04-12T20:56:07.106 INFO:tasks.workunit.client.0.smithi018.stdout:2/348: mkdir d5/d8/d10/d19/d7c 0 2024-04-12T20:56:07.111 INFO:tasks.workunit.client.0.smithi018.stdout:5/399: write d0/d2/d11/d37/f4c [2201593,58994] 0 2024-04-12T20:56:07.111 INFO:tasks.workunit.client.0.smithi018.stdout:2/349: creat d5/d8/d25/f7d x:0 0 0 2024-04-12T20:56:07.112 INFO:tasks.workunit.client.0.smithi018.stdout:9/587: mkdir d3/d6/d9/dc/d5a/d79/db5 0 2024-04-12T20:56:07.113 INFO:tasks.workunit.client.0.smithi018.stdout:7/379: symlink d1/db/d31/d3a/d4f/d69/l80 0 2024-04-12T20:56:07.116 INFO:tasks.workunit.client.0.smithi018.stdout:2/350: chown d5/d7/d31/f7a 11782 1 2024-04-12T20:56:07.118 INFO:tasks.workunit.client.0.smithi018.stdout:7/380: fdatasync d1/db/d31/d3a/d42/d4c/f5b 0 2024-04-12T20:56:07.124 INFO:tasks.workunit.client.0.smithi018.stdout:5/400: creat d0/d2/d33/d46/d35/d4d/f80 x:0 0 0 2024-04-12T20:56:07.129 INFO:tasks.workunit.client.0.smithi018.stdout:6/608: dwrite d5/d6/d1b/d3b/d22/fe7 [0,4194304] 0 2024-04-12T20:56:07.133 INFO:tasks.workunit.client.0.smithi018.stdout:2/351: creat d5/d7/d79/f7e x:0 0 0 2024-04-12T20:56:07.136 INFO:tasks.workunit.client.0.smithi018.stdout:5/401: mknod d0/d2/d33/d46/d18/c81 0 2024-04-12T20:56:07.137 INFO:tasks.workunit.client.0.smithi018.stdout:2/352: mknod d5/d8/d10/d43/c7f 0 2024-04-12T20:56:07.146 INFO:tasks.workunit.client.0.smithi018.stdout:9/588: dwrite d3/f5e [0,4194304] 0 2024-04-12T20:56:07.148 INFO:tasks.workunit.client.0.smithi018.stdout:5/402: chown d0/d2/d11/f4b 212041 1 2024-04-12T20:56:07.150 INFO:tasks.workunit.client.0.smithi018.stdout:2/353: unlink d5/d12/d54/d3e/f50 0 2024-04-12T20:56:07.153 INFO:tasks.workunit.client.0.smithi018.stdout:8/522: dread db/f2d [4194304,4194304] 0 2024-04-12T20:56:07.155 INFO:tasks.workunit.client.0.smithi018.stdout:5/403: creat d0/d2/d33/d46/d1e/d55/f82 x:0 0 0 2024-04-12T20:56:07.156 INFO:tasks.workunit.client.0.smithi018.stdout:9/589: mknod d3/d6/d9/dc/d46/cb6 0 2024-04-12T20:56:07.160 INFO:tasks.workunit.client.0.smithi018.stdout:7/381: dwrite d1/d6/f60 [0,4194304] 0 2024-04-12T20:56:07.163 INFO:tasks.workunit.client.0.smithi018.stdout:2/354: read d5/d12/d54/f53 [735912,127675] 0 2024-04-12T20:56:07.164 INFO:tasks.workunit.client.0.smithi018.stdout:7/382: stat d1/d6/c46 0 2024-04-12T20:56:07.165 INFO:tasks.workunit.client.0.smithi018.stdout:9/590: write d3/d6/d9/dc/d5a/d79/d7e/d9b/f2e [510514,18248] 0 2024-04-12T20:56:07.168 INFO:tasks.workunit.client.0.smithi018.stdout:6/609: dwrite d5/df/d3f/dc1/fce [0,4194304] 0 2024-04-12T20:56:07.168 INFO:tasks.workunit.client.0.smithi018.stdout:5/404: write d0/d2/d33/d46/d35/d4d/f80 [321007,48868] 0 2024-04-12T20:56:07.172 INFO:tasks.workunit.client.0.smithi018.stdout:6/610: dread - d5/d6/d1b/d3b/d35/d4f/fa7 zero size 2024-04-12T20:56:07.177 INFO:tasks.workunit.client.0.smithi018.stdout:2/355: symlink d5/d8/d10/d19/d24/d6c/l80 0 2024-04-12T20:56:07.178 INFO:tasks.workunit.client.0.smithi018.stdout:5/405: mkdir d0/d2/d33/d10/d26/d30/d83 0 2024-04-12T20:56:07.179 INFO:tasks.workunit.client.0.smithi018.stdout:7/383: symlink d1/db/d31/d3a/l81 0 2024-04-12T20:56:07.184 INFO:tasks.workunit.client.0.smithi018.stdout:9/591: getdents d3/d6/d9/dc/d5a/d79/d7e 0 2024-04-12T20:56:07.185 INFO:tasks.workunit.client.0.smithi018.stdout:2/356: mknod d5/d8/d10/d19/d24/c81 0 2024-04-12T20:56:07.188 INFO:tasks.workunit.client.0.smithi018.stdout:9/592: chown d3/d6/l67 62 1 2024-04-12T20:56:07.188 INFO:tasks.workunit.client.0.smithi018.stdout:2/357: fsync d5/d7/f5c 0 2024-04-12T20:56:07.193 INFO:tasks.workunit.client.0.smithi018.stdout:2/358: read - d5/d8/d25/f7d zero size 2024-04-12T20:56:07.194 INFO:tasks.workunit.client.0.smithi018.stdout:8/523: dwrite db/dc/f5e [0,4194304] 0 2024-04-12T20:56:07.201 INFO:tasks.workunit.client.0.smithi018.stdout:2/359: creat d5/d8/f82 x:0 0 0 2024-04-12T20:56:07.203 INFO:tasks.workunit.client.0.smithi018.stdout:9/593: getdents d3/d6/d43 0 2024-04-12T20:56:07.205 INFO:tasks.workunit.client.0.smithi018.stdout:2/360: write d5/d7/d31/d62/f66 [823254,113641] 0 2024-04-12T20:56:07.210 INFO:tasks.workunit.client.0.smithi018.stdout:6/611: dread d5/d6/d1b/d3b/f52 [0,4194304] 0 2024-04-12T20:56:07.210 INFO:tasks.workunit.client.0.smithi018.stdout:9/594: write d3/d6/d9/dc/d5a/d79/d7e/d9b/d21/d41/f76 [1054960,65927] 0 2024-04-12T20:56:07.211 INFO:tasks.workunit.client.0.smithi018.stdout:7/384: truncate d1/db/d31/d3a/d42/d4c/f72 366039 0 2024-04-12T20:56:07.214 INFO:tasks.workunit.client.0.smithi018.stdout:5/406: dread d0/d2/d33/d10/f39 [0,4194304] 0 2024-04-12T20:56:07.219 INFO:tasks.workunit.client.0.smithi018.stdout:8/524: write db/d3b/f55 [43076,5869] 0 2024-04-12T20:56:07.219 INFO:tasks.workunit.client.0.smithi018.stdout:6/612: mknod d5/d6/d1b/d3b/d35/ce9 0 2024-04-12T20:56:07.222 INFO:tasks.workunit.client.0.smithi018.stdout:8/525: chown db/d25/c4a 40150 1 2024-04-12T20:56:07.222 INFO:tasks.workunit.client.0.smithi018.stdout:6/613: fsync d5/df/d31/d4e/f99 0 2024-04-12T20:56:07.223 INFO:tasks.workunit.client.0.smithi018.stdout:2/361: dread d5/d8/d10/d19/d24/d39/f45 [0,4194304] 0 2024-04-12T20:56:07.226 INFO:tasks.workunit.client.0.smithi018.stdout:5/407: rename d0/d2/d33/d10/d26/d30 to d0/d5b/d84 0 2024-04-12T20:56:07.228 INFO:tasks.workunit.client.0.smithi018.stdout:9/595: link d3/d6/d9/dc/d6f/d77/l9e d3/d6/lb7 0 2024-04-12T20:56:07.231 INFO:tasks.workunit.client.0.smithi018.stdout:2/362: symlink d5/d8/d10/d19/d24/d39/l83 0 2024-04-12T20:56:07.232 INFO:tasks.workunit.client.0.smithi018.stdout:2/363: readlink d5/d8/d25/l2c 0 2024-04-12T20:56:07.236 INFO:tasks.workunit.client.0.smithi018.stdout:5/408: creat d0/d2/d33/d10/d26/f85 x:0 0 0 2024-04-12T20:56:07.237 INFO:tasks.workunit.client.0.smithi018.stdout:7/385: truncate d1/db/d31/f4d 1800830 0 2024-04-12T20:56:07.239 INFO:tasks.workunit.client.0.smithi018.stdout:7/386: write d1/d6/f60 [1845751,74115] 0 2024-04-12T20:56:07.242 INFO:tasks.workunit.client.0.smithi018.stdout:8/526: creat db/d25/d26/d27/fa0 x:0 0 0 2024-04-12T20:56:07.242 INFO:tasks.workunit.client.0.smithi018.stdout:2/364: symlink d5/d7/d79/l84 0 2024-04-12T20:56:07.242 INFO:tasks.workunit.client.0.smithi018.stdout:5/409: creat d0/d2/d33/d10/d2f/d58/f86 x:0 0 0 2024-04-12T20:56:07.246 INFO:tasks.workunit.client.0.smithi018.stdout:5/410: write d0/d2/d33/d46/d18/f71 [1752755,10927] 0 2024-04-12T20:56:07.258 INFO:tasks.workunit.client.0.smithi018.stdout:8/527: mkdir db/dc/d45/d86/da1 0 2024-04-12T20:56:07.264 INFO:tasks.workunit.client.0.smithi018.stdout:9/596: dwrite d3/d6/d9/dc/f97 [0,4194304] 0 2024-04-12T20:56:07.265 INFO:tasks.workunit.client.0.smithi018.stdout:8/528: chown fa 1002 1 2024-04-12T20:56:07.266 INFO:tasks.workunit.client.0.smithi018.stdout:7/387: dwrite d1/db/d31/d3a/d79/f68 [0,4194304] 0 2024-04-12T20:56:07.268 INFO:tasks.workunit.client.0.smithi018.stdout:2/365: symlink d5/d7/d63/l85 0 2024-04-12T20:56:07.269 INFO:tasks.workunit.client.0.smithi018.stdout:9/597: truncate d3/d6/d9/d1e/f74 249556 0 2024-04-12T20:56:07.269 INFO:tasks.workunit.client.0.smithi018.stdout:0/369: sync 2024-04-12T20:56:07.269 INFO:tasks.workunit.client.0.smithi018.stdout:1/316: sync 2024-04-12T20:56:07.269 INFO:tasks.workunit.client.0.smithi018.stdout:4/446: sync 2024-04-12T20:56:07.269 INFO:tasks.workunit.client.0.smithi018.stdout:3/387: sync 2024-04-12T20:56:07.272 INFO:tasks.workunit.client.0.smithi018.stdout:5/411: symlink d0/d2b/d40/l87 0 2024-04-12T20:56:07.273 INFO:tasks.workunit.client.0.smithi018.stdout:0/370: chown da/d15/d17/c1b 34203716 1 2024-04-12T20:56:07.273 INFO:tasks.workunit.client.0.smithi018.stdout:2/366: chown d5/l13 23154 1 2024-04-12T20:56:07.279 INFO:tasks.workunit.client.0.smithi018.stdout:9/598: truncate d3/d6/d43/fa3 235114 0 2024-04-12T20:56:07.280 INFO:tasks.workunit.client.0.smithi018.stdout:6/614: dread d5/d15/d60/f65 [0,4194304] 0 2024-04-12T20:56:07.283 INFO:tasks.workunit.client.0.smithi018.stdout:1/317: mknod d3/d9/da/dd/d31/c6c 0 2024-04-12T20:56:07.283 INFO:tasks.workunit.client.0.smithi018.stdout:7/388: mknod d1/db/d31/d3a/d42/c82 0 2024-04-12T20:56:07.285 INFO:tasks.workunit.client.0.smithi018.stdout:5/412: creat d0/d2/d33/d46/d1e/d55/f88 x:0 0 0 2024-04-12T20:56:07.289 INFO:tasks.workunit.client.0.smithi018.stdout:4/447: creat d3/db/f8f x:0 0 0 2024-04-12T20:56:07.294 INFO:tasks.workunit.client.0.smithi018.stdout:6/615: truncate d5/df/d31/d4e/fc7 646811 0 2024-04-12T20:56:07.296 INFO:tasks.workunit.client.0.smithi018.stdout:3/388: stat d2/d6/d30/d31/d38/d3a/d4c/c55 0 2024-04-12T20:56:07.299 INFO:tasks.workunit.client.0.smithi018.stdout:9/599: rename d3/d6/d9/dc/d46/f65 to d3/d6/d1b/fb8 0 2024-04-12T20:56:07.300 INFO:tasks.workunit.client.0.smithi018.stdout:8/529: dread db/d4e/f6b [0,4194304] 0 2024-04-12T20:56:07.302 INFO:tasks.workunit.client.0.smithi018.stdout:6/616: readlink d5/d15/la9 0 2024-04-12T20:56:07.302 INFO:tasks.workunit.client.0.smithi018.stdout:8/530: readlink db/d3b/l80 0 2024-04-12T20:56:07.305 INFO:tasks.workunit.client.0.smithi018.stdout:5/413: mkdir d0/d2/d33/d10/d2f/d58/d89 0 2024-04-12T20:56:07.305 INFO:tasks.workunit.client.0.smithi018.stdout:9/600: rmdir d3/d6/d9/dc/d5a/d79/d7e/d9b/d21/d41 39 2024-04-12T20:56:07.307 INFO:tasks.workunit.client.0.smithi018.stdout:3/389: symlink d2/d43/l72 0 2024-04-12T20:56:07.309 INFO:tasks.workunit.client.0.smithi018.stdout:3/390: chown d2/d6/f2f 889244 1 2024-04-12T20:56:07.313 INFO:tasks.workunit.client.0.smithi018.stdout:7/389: creat d1/db/d31/d3a/f83 x:0 0 0 2024-04-12T20:56:07.314 INFO:tasks.workunit.client.0.smithi018.stdout:4/448: getdents d3/d34/d43 0 2024-04-12T20:56:07.314 INFO:tasks.workunit.client.0.smithi018.stdout:0/371: write da/d15/d17/d21/f46 [700765,65838] 0 2024-04-12T20:56:07.314 INFO:tasks.workunit.client.0.smithi018.stdout:9/601: write d3/d6/d9/dc/d13/d5b/d7c/f69 [1608031,127417] 0 2024-04-12T20:56:07.316 INFO:tasks.workunit.client.0.smithi018.stdout:3/391: symlink d2/d43/d61/l73 0 2024-04-12T20:56:07.319 INFO:tasks.workunit.client.0.smithi018.stdout:1/318: truncate d3/d9/da/dd/f12 2410676 0 2024-04-12T20:56:07.321 INFO:tasks.workunit.client.0.smithi018.stdout:7/390: dread - d1/db/d31/d45/d53/f7d zero size 2024-04-12T20:56:07.325 INFO:tasks.workunit.client.0.smithi018.stdout:3/392: write d2/d6/f1b [4700689,70169] 0 2024-04-12T20:56:07.331 INFO:tasks.workunit.client.0.smithi018.stdout:9/602: read d3/d6/d9/d1e/f53 [3734678,126198] 0 2024-04-12T20:56:07.331 INFO:tasks.workunit.client.0.smithi018.stdout:5/414: dread d0/d2/d33/d46/f1d [0,4194304] 0 2024-04-12T20:56:07.331 INFO:tasks.workunit.client.0.smithi018.stdout:1/319: readlink d3/d9/da/dd/d31/d44/l56 0 2024-04-12T20:56:07.332 INFO:tasks.workunit.client.0.smithi018.stdout:7/391: symlink d1/db/l84 0 2024-04-12T20:56:07.335 INFO:tasks.workunit.client.0.smithi018.stdout:8/531: dwrite db/d4e/f6d [0,4194304] 0 2024-04-12T20:56:07.337 INFO:tasks.workunit.client.0.smithi018.stdout:3/393: truncate d2/d6/d30/d31/d38/d3a/d48/f5c 206551 0 2024-04-12T20:56:07.340 INFO:tasks.workunit.client.0.smithi018.stdout:9/603: dread d3/d6/d9/d1e/f74 [0,4194304] 0 2024-04-12T20:56:07.346 INFO:tasks.workunit.client.0.smithi018.stdout:8/532: fdatasync db/d4e/d56/f63 0 2024-04-12T20:56:07.350 INFO:tasks.workunit.client.0.smithi018.stdout:3/394: creat d2/d6/d30/d31/d38/f74 x:0 0 0 2024-04-12T20:56:07.350 INFO:tasks.workunit.client.0.smithi018.stdout:7/392: creat d1/db/d31/d3a/f85 x:0 0 0 2024-04-12T20:56:07.351 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:56:07 smithi018 ceph-mon[28799]: pgmap v36: 65 pgs: 65 active+clean; 934 MiB data, 4.2 GiB used, 532 GiB / 536 GiB avail; 20 MiB/s rd, 117 MiB/s wr, 295 op/s 2024-04-12T20:56:07.354 INFO:tasks.workunit.client.0.smithi018.stdout:3/395: dread - d2/d4d/f57 zero size 2024-04-12T20:56:07.357 INFO:tasks.workunit.client.0.smithi018.stdout:5/415: write d0/d5b/f5d [982170,98539] 0 2024-04-12T20:56:07.357 INFO:tasks.workunit.client.0.smithi018.stdout:8/533: write db/d25/f4c [783690,3331] 0 2024-04-12T20:56:07.367 INFO:tasks.workunit.client.0.smithi018.stdout:5/416: read d0/d2/d33/d46/d35/f4f [1872595,93857] 0 2024-04-12T20:56:07.372 INFO:tasks.workunit.client.0.smithi018.stdout:5/417: chown d0/d2/d33/d46/d35/l72 2811025 1 2024-04-12T20:56:07.373 INFO:tasks.workunit.client.0.smithi018.stdout:4/449: dread d3/d32/d33/f30 [0,4194304] 0 2024-04-12T20:56:07.374 INFO:tasks.workunit.client.0.smithi018.stdout:1/320: dwrite d3/f10 [0,4194304] 0 2024-04-12T20:56:07.375 INFO:tasks.workunit.client.0.smithi018.stdout:3/396: dread d2/d6/d30/d31/d38/d3a/d48/f5c [0,4194304] 0 2024-04-12T20:56:07.377 INFO:tasks.workunit.client.0.smithi018.stdout:9/604: dwrite d3/d6/d9/dc/d13/f42 [0,4194304] 0 2024-04-12T20:56:07.378 INFO:tasks.workunit.client.0.smithi018.stdout:5/418: write d0/d5b/f64 [311927,93090] 0 2024-04-12T20:56:07.382 INFO:tasks.workunit.client.0.smithi018.stdout:5/419: dread - d0/d2/d33/d10/d26/f7e zero size 2024-04-12T20:56:07.385 INFO:tasks.workunit.client.0.smithi018.stdout:9/605: mkdir d3/d6/d9/dc/d13/db9 0 2024-04-12T20:56:07.402 INFO:tasks.workunit.client.0.smithi018.stdout:8/534: write db/d25/d26/f3f [1351189,38387] 0 2024-04-12T20:56:07.406 INFO:tasks.workunit.client.0.smithi018.stdout:9/606: getdents d3/d6/d9/dc/d46 0 2024-04-12T20:56:07.417 INFO:tasks.workunit.client.0.smithi018.stdout:7/393: mkdir d1/d86 0 2024-04-12T20:56:07.418 INFO:tasks.workunit.client.0.smithi018.stdout:5/420: fsync d0/d2/d11/d37/f47 0 2024-04-12T20:56:07.419 INFO:tasks.workunit.client.0.smithi018.stdout:9/607: unlink d3/d6/d9/dc/d5a/d79/d7e/d9b/d21/d41/fa5 0 2024-04-12T20:56:07.420 INFO:tasks.workunit.client.0.smithi018.stdout:7/394: chown d1/db/d31/d3a/f83 1366687 1 2024-04-12T20:56:07.424 INFO:tasks.workunit.client.0.smithi018.stdout:7/395: chown d1/db/d31/d3a/d42/d4c/f5b 46182 1 2024-04-12T20:56:07.425 INFO:tasks.workunit.client.0.smithi018.stdout:4/450: unlink d3/d34/d43/l49 0 2024-04-12T20:56:07.426 INFO:tasks.workunit.client.0.smithi018.stdout:7/396: stat d1/f32 0 2024-04-12T20:56:07.433 INFO:tasks.workunit.client.0.smithi018.stdout:5/421: creat d0/d2/d33/d46/d1e/d2e/f8a x:0 0 0 2024-04-12T20:56:07.434 INFO:tasks.workunit.client.0.smithi018.stdout:4/451: mkdir d3/d34/d3d/d90 0 2024-04-12T20:56:07.434 INFO:tasks.workunit.client.0.smithi018.stdout:1/321: dread d3/d9/da/dd/d31/d32/f1b [0,4194304] 0 2024-04-12T20:56:07.438 INFO:tasks.workunit.client.0.smithi018.stdout:8/535: link db/d25/d26/l29 db/d4e/d73/la2 0 2024-04-12T20:56:07.447 INFO:tasks.workunit.client.0.smithi018.stdout:4/452: creat d3/d32/d33/f91 x:0 0 0 2024-04-12T20:56:07.450 INFO:tasks.workunit.client.0.smithi018.stdout:9/608: dwrite d3/d6/d9/d1e/f96 [0,4194304] 0 2024-04-12T20:56:07.453 INFO:tasks.workunit.client.0.smithi018.stdout:4/453: unlink d3/c8 0 2024-04-12T20:56:07.456 INFO:tasks.workunit.client.0.smithi018.stdout:5/422: rmdir d0/d5b 39 2024-04-12T20:56:07.458 INFO:tasks.workunit.client.0.smithi018.stdout:7/397: dwrite d1/db/d31/d30/f57 [0,4194304] 0 2024-04-12T20:56:07.462 INFO:tasks.workunit.client.0.smithi018.stdout:7/398: stat d1/db/f35 0 2024-04-12T20:56:07.462 INFO:tasks.workunit.client.0.smithi018.stdout:8/536: dread db/d25/d26/f3f [0,4194304] 0 2024-04-12T20:56:07.462 INFO:tasks.workunit.client.0.smithi018.stdout:3/397: symlink d2/l75 0 2024-04-12T20:56:07.466 INFO:tasks.workunit.client.0.smithi018.stdout:4/454: dread d3/d34/d40/f6d [0,4194304] 0 2024-04-12T20:56:07.466 INFO:tasks.workunit.client.0.smithi018.stdout:1/322: link d3/d9/da/dd/d31/d32/d13/d19/d2d/f4d d3/d9/da/dd/d31/d32/d33/f6d 0 2024-04-12T20:56:07.469 INFO:tasks.workunit.client.0.smithi018.stdout:7/399: dread - d1/db/d31/f58 zero size 2024-04-12T20:56:07.474 INFO:tasks.workunit.client.0.smithi018.stdout:1/323: fdatasync d3/d9/f3c 0 2024-04-12T20:56:07.478 INFO:tasks.workunit.client.0.smithi018.stdout:7/400: symlink d1/db/d31/d45/l87 0 2024-04-12T20:56:07.483 INFO:tasks.workunit.client.0.smithi018.stdout:7/401: write d1/db/d31/d30/f4a [7877941,114136] 0 2024-04-12T20:56:07.485 INFO:tasks.workunit.client.0.smithi018.stdout:5/423: mknod d0/d5b/d84/d83/c8b 0 2024-04-12T20:56:07.486 INFO:tasks.workunit.client.0.smithi018.stdout:3/398: mkdir d2/d6/d30/d31/d38/d3a/d76 0 2024-04-12T20:56:07.489 INFO:tasks.workunit.client.0.smithi018.stdout:9/609: dwrite d3/d6/d9/dc/d5a/d79/d7e/d9b/f2e [0,4194304] 0 2024-04-12T20:56:07.490 INFO:tasks.workunit.client.0.smithi018.stdout:5/424: write d0/d2/d11/f4b [4366768,41111] 0 2024-04-12T20:56:07.493 INFO:tasks.workunit.client.0.smithi018.stdout:3/399: write d2/d6/d3d/f60 [247056,16572] 0 2024-04-12T20:56:07.499 INFO:tasks.workunit.client.0.smithi018.stdout:8/537: rename db/d4e/f88 to db/d25/d26/fa3 0 2024-04-12T20:56:07.501 INFO:tasks.workunit.client.0.smithi018.stdout:3/400: chown d2/d6/d34/f59 222118 1 2024-04-12T20:56:07.503 INFO:tasks.workunit.client.0.smithi018.stdout:1/324: creat d3/d9/da/dd/d31/d32/d13/d19/f6e x:0 0 0 2024-04-12T20:56:07.503 INFO:tasks.workunit.client.0.smithi018.stdout:3/401: stat d2/d6/c1e 0 2024-04-12T20:56:07.511 INFO:tasks.workunit.client.0.smithi018.stdout:9/610: truncate d3/d6/d9/dc/d5a/d79/d7e/d9b/d21/d41/d78/f83 624891 0 2024-04-12T20:56:07.513 INFO:tasks.workunit.client.0.smithi018.stdout:5/425: rename d0/d5b/f64 to d0/d2/d33/d46/d1e/d2e/d7f/f8c 0 2024-04-12T20:56:07.516 INFO:tasks.workunit.client.0.smithi018.stdout:1/325: read d3/d9/da/f20 [903474,86046] 0 2024-04-12T20:56:07.516 INFO:tasks.workunit.client.0.smithi018.stdout:7/402: mknod d1/d6/c88 0 2024-04-12T20:56:07.518 INFO:tasks.workunit.client.0.smithi018.stdout:4/455: link d3/d5d/d60/d64/l71 d3/d34/d40/l92 0 2024-04-12T20:56:07.519 INFO:tasks.workunit.client.0.smithi018.stdout:5/426: read d0/d2/d33/d46/d35/f5a [65058,39454] 0 2024-04-12T20:56:07.522 INFO:tasks.workunit.client.0.smithi018.stdout:8/538: dread db/d4e/d56/f63 [0,4194304] 0 2024-04-12T20:56:07.526 INFO:tasks.workunit.client.0.smithi018.stdout:7/403: creat d1/db/d31/d3a/d4f/d69/f89 x:0 0 0 2024-04-12T20:56:07.527 INFO:tasks.workunit.client.0.smithi018.stdout:4/456: creat d3/d32/d66/f93 x:0 0 0 2024-04-12T20:56:07.528 INFO:tasks.workunit.client.0.smithi018.stdout:8/539: read db/d25/d26/f53 [2902351,5878] 0 2024-04-12T20:56:07.530 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:56:07 smithi186 ceph-mon[36597]: pgmap v36: 65 pgs: 65 active+clean; 934 MiB data, 4.2 GiB used, 532 GiB / 536 GiB avail; 20 MiB/s rd, 117 MiB/s wr, 295 op/s 2024-04-12T20:56:07.535 INFO:tasks.workunit.client.0.smithi018.stdout:3/402: rename d2/l17 to d2/d6/d30/d31/l77 0 2024-04-12T20:56:07.536 INFO:tasks.workunit.client.0.smithi018.stdout:1/326: dread d3/f4 [4194304,4194304] 0 2024-04-12T20:56:07.537 INFO:tasks.workunit.client.0.smithi018.stdout:4/457: mkdir d3/d32/d94 0 2024-04-12T20:56:07.539 INFO:tasks.workunit.client.0.smithi018.stdout:8/540: creat db/dc/d45/d86/da1/fa4 x:0 0 0 2024-04-12T20:56:07.543 INFO:tasks.workunit.client.0.smithi018.stdout:9/611: write d3/d6/d9/d1e/f53 [3410166,107206] 0 2024-04-12T20:56:07.543 INFO:tasks.workunit.client.0.smithi018.stdout:4/458: creat d3/d5d/d60/d64/d4f/f95 x:0 0 0 2024-04-12T20:56:07.546 INFO:tasks.workunit.client.0.smithi018.stdout:1/327: unlink d3/d9/da/dd/d31/d32/c64 0 2024-04-12T20:56:07.556 INFO:tasks.workunit.client.0.smithi018.stdout:5/427: dwrite d0/d2/d33/d46/d18/f71 [0,4194304] 0 2024-04-12T20:56:07.556 INFO:tasks.workunit.client.0.smithi018.stdout:4/459: creat d3/d32/d94/f96 x:0 0 0 2024-04-12T20:56:07.557 INFO:tasks.workunit.client.0.smithi018.stdout:7/404: dwrite d1/db/f35 [0,4194304] 0 2024-04-12T20:56:07.561 INFO:tasks.workunit.client.0.smithi018.stdout:1/328: creat d3/d9/da/f6f x:0 0 0 2024-04-12T20:56:07.565 INFO:tasks.workunit.client.0.smithi018.stdout:7/405: write d1/db/d31/d3a/d42/d4c/f51 [247269,128459] 0 2024-04-12T20:56:07.569 INFO:tasks.workunit.client.0.smithi018.stdout:5/428: dread d0/d2/d33/d46/f1d [0,4194304] 0 2024-04-12T20:56:07.569 INFO:tasks.workunit.client.0.smithi018.stdout:8/541: rename db/dc to db/d4e/d56/d59/d76/da5 0 2024-04-12T20:56:07.576 INFO:tasks.workunit.client.0.smithi018.stdout:4/460: rename d3/d34/d43/f2c to d3/d32/d94/f97 0 2024-04-12T20:56:07.577 INFO:tasks.workunit.client.0.smithi018.stdout:9/612: dwrite d3/d6/d9/dc/d5a/d79/d7e/d9b/d21/d41/f76 [0,4194304] 0 2024-04-12T20:56:07.578 INFO:tasks.workunit.client.0.smithi018.stdout:8/542: readlink db/d4e/d56/d59/d76/da5/l9c 0 2024-04-12T20:56:07.581 INFO:tasks.workunit.client.0.smithi018.stdout:1/329: rename d3/d9/da/dd/d31/d32/d13/d19/d2d/d4e/d58/d68/f6b to d3/d9/d11/f70 0 2024-04-12T20:56:07.585 INFO:tasks.workunit.client.0.smithi018.stdout:9/613: mkdir d3/d6/d3d/dba 0 2024-04-12T20:56:07.588 INFO:tasks.workunit.client.0.smithi018.stdout:7/406: unlink d1/db/d31/d3a/d42/d4c/f72 0 2024-04-12T20:56:07.588 INFO:tasks.workunit.client.0.smithi018.stdout:3/403: dwrite d2/f14 [0,4194304] 0 2024-04-12T20:56:07.589 INFO:tasks.workunit.client.0.smithi018.stdout:5/429: truncate d0/d2/d33/d10/d2f/d58/f86 508263 0 2024-04-12T20:56:07.590 INFO:tasks.workunit.client.0.smithi018.stdout:8/543: mknod db/d4e/d56/d59/d76/da5/ca6 0 2024-04-12T20:56:07.590 INFO:tasks.workunit.client.0.smithi018.stdout:9/614: mknod d3/d6/cbb 0 2024-04-12T20:56:07.593 INFO:tasks.workunit.client.0.smithi018.stdout:3/404: fsync d2/d6/f9 0 2024-04-12T20:56:07.594 INFO:tasks.workunit.client.0.smithi018.stdout:9/615: chown d3/d6/d9/dc/d13 1959806 1 2024-04-12T20:56:07.596 INFO:tasks.workunit.client.0.smithi018.stdout:4/461: link d3/d32/l47 d3/d32/d66/l98 0 2024-04-12T20:56:07.602 INFO:tasks.workunit.client.0.smithi018.stdout:1/330: mkdir d3/d9/da/dd/d31/d32/d13/d19/d2d/d4e/d58/d61/d71 0 2024-04-12T20:56:07.602 INFO:tasks.workunit.client.0.smithi018.stdout:8/544: write db/d4e/d56/d59/d76/da5/f1a [2206373,37857] 0 2024-04-12T20:56:07.602 INFO:tasks.workunit.client.0.smithi018.stdout:5/430: write d0/d2/d33/d46/d35/f79 [3823307,15050] 0 2024-04-12T20:56:07.606 INFO:tasks.workunit.client.0.smithi018.stdout:8/545: read - db/d4e/f84 zero size 2024-04-12T20:56:07.607 INFO:tasks.workunit.client.0.smithi018.stdout:4/462: fsync d3/d5d/d60/d64/d72/f89 0 2024-04-12T20:56:07.607 INFO:tasks.workunit.client.0.smithi018.stdout:3/405: rename d2/d6/d30/d31/d38/d3a/d48/f5c to d2/f78 0 2024-04-12T20:56:07.611 INFO:tasks.workunit.client.0.smithi018.stdout:5/431: write d0/d2/d33/d10/f27 [3409289,78602] 0 2024-04-12T20:56:07.613 INFO:tasks.workunit.client.0.smithi018.stdout:4/463: readlink d3/d32/d33/l36 0 2024-04-12T20:56:07.619 INFO:tasks.workunit.client.0.smithi018.stdout:7/407: dwrite d1/f2 [0,4194304] 0 2024-04-12T20:56:07.622 INFO:tasks.workunit.client.0.smithi018.stdout:8/546: creat db/d4e/d9a/d69/d6c/fa7 x:0 0 0 2024-04-12T20:56:07.622 INFO:tasks.workunit.client.0.smithi018.stdout:9/616: dwrite d3/d6/d9/dc/d5a/d79/f87 [0,4194304] 0 2024-04-12T20:56:07.624 INFO:tasks.workunit.client.0.smithi018.stdout:4/464: readlink d3/d32/l3e 0 2024-04-12T20:56:07.627 INFO:tasks.workunit.client.0.smithi018.stdout:4/465: write d3/d32/d66/f75 [1058621,5294] 0 2024-04-12T20:56:07.633 INFO:tasks.workunit.client.0.smithi018.stdout:3/406: rename d2/d6/f9 to d2/d6/d30/d31/d38/d3a/d4c/f79 0 2024-04-12T20:56:07.641 INFO:tasks.workunit.client.0.smithi018.stdout:1/331: rmdir d3/d9/da/dd/d31/d32 39 2024-04-12T20:56:07.645 INFO:tasks.workunit.client.0.smithi018.stdout:8/547: write db/f2d [700868,49469] 0 2024-04-12T20:56:07.645 INFO:tasks.workunit.client.0.smithi018.stdout:7/408: truncate d1/db/d31/f4d 2260079 0 2024-04-12T20:56:07.646 INFO:tasks.workunit.client.0.smithi018.stdout:4/466: symlink d3/d5d/l99 0 2024-04-12T20:56:07.651 INFO:tasks.workunit.client.0.smithi018.stdout:5/432: dwrite d0/d2/d33/d46/d35/d4d/f80 [0,4194304] 0 2024-04-12T20:56:07.655 INFO:tasks.workunit.client.0.smithi018.stdout:4/467: write d3/d32/fd [4259958,107079] 0 2024-04-12T20:56:07.656 INFO:tasks.workunit.client.0.smithi018.stdout:4/468: readlink d3/d32/l3f 0 2024-04-12T20:56:07.656 INFO:tasks.workunit.client.0.smithi018.stdout:7/409: mkdir d1/db/d6f/d8a 0 2024-04-12T20:56:07.659 INFO:tasks.workunit.client.0.smithi018.stdout:4/469: rmdir d3/d34/d3d/d80 39 2024-04-12T20:56:07.662 INFO:tasks.workunit.client.0.smithi018.stdout:7/410: write d1/f1b [4141903,85771] 0 2024-04-12T20:56:07.663 INFO:tasks.workunit.client.0.smithi018.stdout:5/433: mkdir d0/d2/d33/d10/d2f/d58/d89/d8d 0 2024-04-12T20:56:07.668 INFO:tasks.workunit.client.0.smithi018.stdout:3/407: dwrite d2/d6/d30/d31/f50 [0,4194304] 0 2024-04-12T20:56:07.668 INFO:tasks.workunit.client.0.smithi018.stdout:1/332: truncate d3/d9/f18 1746275 0 2024-04-12T20:56:07.671 INFO:tasks.workunit.client.0.smithi018.stdout:5/434: mknod d0/d2/d33/d10/d2f/c8e 0 2024-04-12T20:56:07.676 INFO:tasks.workunit.client.0.smithi018.stdout:3/408: dread d2/f78 [0,4194304] 0 2024-04-12T20:56:07.679 INFO:tasks.workunit.client.0.smithi018.stdout:1/333: symlink d3/d9/da/l72 0 2024-04-12T20:56:07.679 INFO:tasks.workunit.client.0.smithi018.stdout:3/409: stat d2/f78 0 2024-04-12T20:56:07.685 INFO:tasks.workunit.client.0.smithi018.stdout:1/334: stat d3/d9/da/dd/d31/d32/d13/d19/d2d/d4e/d58/d5c 0 2024-04-12T20:56:07.689 INFO:tasks.workunit.client.0.smithi018.stdout:1/335: mkdir d3/d9/da/d14/d73 0 2024-04-12T20:56:07.696 INFO:tasks.workunit.client.0.smithi018.stdout:4/470: truncate d3/d5d/d60/d64/d4f/f61 1617847 0 2024-04-12T20:56:07.699 INFO:tasks.workunit.client.0.smithi018.stdout:5/435: dwrite d0/d2/d33/d10/d26/f4a [0,4194304] 0 2024-04-12T20:56:07.700 INFO:tasks.workunit.client.0.smithi018.stdout:4/471: dread - d3/db/f7f zero size 2024-04-12T20:56:07.701 INFO:tasks.workunit.client.0.smithi018.stdout:8/548: dread db/d3b/f49 [0,4194304] 0 2024-04-12T20:56:07.706 INFO:tasks.workunit.client.0.smithi018.stdout:1/336: rename d3/d9/d11/l53 to d3/d9/da/dd/d31/d32/d33/l74 0 2024-04-12T20:56:07.706 INFO:tasks.workunit.client.0.smithi018.stdout:8/549: stat db/d25/c4a 0 2024-04-12T20:56:07.712 INFO:tasks.workunit.client.0.smithi018.stdout:3/410: dwrite d2/f5d [0,4194304] 0 2024-04-12T20:56:07.713 INFO:tasks.workunit.client.0.smithi018.stdout:7/411: dwrite d1/db/d31/d45/f7c [0,4194304] 0 2024-04-12T20:56:07.715 INFO:tasks.workunit.client.0.smithi018.stdout:3/411: dread - d2/d6/d30/f6f zero size 2024-04-12T20:56:07.718 INFO:tasks.workunit.client.0.smithi018.stdout:8/550: chown db/d25/d26/l29 60057091 1 2024-04-12T20:56:07.719 INFO:tasks.workunit.client.0.smithi018.stdout:1/337: mknod d3/d9/da/dd/d31/d32/d33/c75 0 2024-04-12T20:56:07.725 INFO:tasks.workunit.client.0.smithi018.stdout:1/338: write d3/d9/da/dd/d31/d32/d13/d19/f6e [639073,74806] 0 2024-04-12T20:56:07.728 INFO:tasks.workunit.client.0.smithi018.stdout:7/412: truncate d1/db/d31/f58 643964 0 2024-04-12T20:56:07.732 INFO:tasks.workunit.client.0.smithi018.stdout:3/412: symlink d2/d6/d30/d31/d38/d3a/d48/l7a 0 2024-04-12T20:56:07.733 INFO:tasks.workunit.client.0.smithi018.stdout:8/551: creat db/d4e/d56/d59/d76/da5/d45/d86/fa8 x:0 0 0 2024-04-12T20:56:07.737 INFO:tasks.workunit.client.0.smithi018.stdout:7/413: fdatasync d1/f66 0 2024-04-12T20:56:07.739 INFO:tasks.workunit.client.0.smithi018.stdout:1/339: creat d3/d9/da/dd/d31/d32/d13/d19/d2d/f76 x:0 0 0 2024-04-12T20:56:07.739 INFO:tasks.workunit.client.0.smithi018.stdout:5/436: write d0/d2/d33/d46/f3a [1733337,35899] 0 2024-04-12T20:56:07.749 INFO:tasks.workunit.client.0.smithi018.stdout:3/413: mkdir d2/d6/d34/d71/d7b 0 2024-04-12T20:56:07.751 INFO:tasks.workunit.client.0.smithi018.stdout:5/437: write d0/d2/d11/d37/f47 [1734385,67530] 0 2024-04-12T20:56:07.752 INFO:tasks.workunit.client.0.smithi018.stdout:8/552: chown db/d4e/d56/d59/d76/da5/f12 16623788 1 2024-04-12T20:56:07.757 INFO:tasks.workunit.client.0.smithi018.stdout:4/472: dwrite d3/d34/d3d/f65 [0,4194304] 0 2024-04-12T20:56:07.758 INFO:tasks.workunit.client.0.smithi018.stdout:3/414: mkdir d2/d6/d30/d31/d7c 0 2024-04-12T20:56:07.759 INFO:tasks.workunit.client.0.smithi018.stdout:5/438: dread - d0/d5b/d84/f7d zero size 2024-04-12T20:56:07.760 INFO:tasks.workunit.client.0.smithi018.stdout:4/473: write d3/d34/d43/f62 [467546,31245] 0 2024-04-12T20:56:07.762 INFO:tasks.workunit.client.0.smithi018.stdout:1/340: creat d3/d9/da/dd/d31/f77 x:0 0 0 2024-04-12T20:56:07.765 INFO:tasks.workunit.client.0.smithi018.stdout:8/553: mkdir db/d4e/d56/d59/d76/da5/d45/da9 0 2024-04-12T20:56:07.767 INFO:tasks.workunit.client.0.smithi018.stdout:8/554: read db/d4e/f6d [1159520,82276] 0 2024-04-12T20:56:07.768 INFO:tasks.workunit.client.0.smithi018.stdout:3/415: creat d2/d6/d34/d71/d7b/f7d x:0 0 0 2024-04-12T20:56:07.769 INFO:tasks.workunit.client.0.smithi018.stdout:4/474: symlink d3/d5d/d60/d64/d72/l9a 0 2024-04-12T20:56:07.769 INFO:tasks.workunit.client.0.smithi018.stdout:5/439: mkdir d0/d2/d33/d10/d2f/d8f 0 2024-04-12T20:56:07.773 INFO:tasks.workunit.client.0.smithi018.stdout:1/341: truncate d3/d9/da/dd/d31/f59 94780 0 2024-04-12T20:56:07.774 INFO:tasks.workunit.client.0.smithi018.stdout:4/475: mkdir d3/d32/d33/d9b 0 2024-04-12T20:56:07.776 INFO:tasks.workunit.client.0.smithi018.stdout:4/476: fsync d3/d32/d94/f96 0 2024-04-12T20:56:07.778 INFO:tasks.workunit.client.0.smithi018.stdout:3/416: mknod d2/d6/d3d/d54/c7e 0 2024-04-12T20:56:07.780 INFO:tasks.workunit.client.0.smithi018.stdout:8/555: truncate db/d4e/d56/d59/d76/da5/f12 1805385 0 2024-04-12T20:56:07.783 INFO:tasks.workunit.client.0.smithi018.stdout:3/417: creat d2/d4d/f7f x:0 0 0 2024-04-12T20:56:07.786 INFO:tasks.workunit.client.0.smithi018.stdout:1/342: symlink d3/d9/da/dd/d31/l78 0 2024-04-12T20:56:07.787 INFO:tasks.workunit.client.0.smithi018.stdout:7/414: dread d1/d6/f52 [0,4194304] 0 2024-04-12T20:56:07.788 INFO:tasks.workunit.client.0.smithi018.stdout:4/477: rename d3/d32/d33/f29 to d3/d34/d43/d4b/f9c 0 2024-04-12T20:56:07.790 INFO:tasks.workunit.client.0.smithi018.stdout:3/418: write d2/d6/d30/d31/d38/d3a/d48/f51 [5781205,86961] 0 2024-04-12T20:56:07.790 INFO:tasks.workunit.client.0.smithi018.stdout:5/440: dread d0/d2/d33/d46/d1e/f6c [0,4194304] 0 2024-04-12T20:56:07.799 INFO:tasks.workunit.client.0.smithi018.stdout:1/343: symlink d3/d9/da/dd/d31/d32/l79 0 2024-04-12T20:56:07.803 INFO:tasks.workunit.client.0.smithi018.stdout:5/441: mknod d0/d2/d33/d46/d1e/c90 0 2024-04-12T20:56:07.808 INFO:tasks.workunit.client.0.smithi018.stdout:8/556: dwrite f7 [0,4194304] 0 2024-04-12T20:56:07.808 INFO:tasks.workunit.client.0.smithi018.stdout:5/442: truncate d0/d5b/f5d 1841245 0 2024-04-12T20:56:07.809 INFO:tasks.workunit.client.0.smithi018.stdout:1/344: mkdir d3/d9/da/dd/d31/d32/d13/d19/d2d/d4e/d7a 0 2024-04-12T20:56:07.810 INFO:tasks.workunit.client.0.smithi018.stdout:8/557: write f0 [758708,14691] 0 2024-04-12T20:56:07.818 INFO:tasks.workunit.client.0.smithi018.stdout:5/443: fdatasync d0/d2/d33/d46/d35/f4f 0 2024-04-12T20:56:07.820 INFO:tasks.workunit.client.0.smithi018.stdout:8/558: mkdir db/d4e/d9a/daa 0 2024-04-12T20:56:07.821 INFO:tasks.workunit.client.0.smithi018.stdout:7/415: dwrite d1/db/d31/d30/f57 [0,4194304] 0 2024-04-12T20:56:07.825 INFO:tasks.workunit.client.0.smithi018.stdout:8/559: readlink db/d4e/d56/l85 0 2024-04-12T20:56:07.831 INFO:tasks.workunit.client.0.smithi018.stdout:3/419: dwrite d2/d6/d30/f58 [0,4194304] 0 2024-04-12T20:56:07.835 INFO:tasks.workunit.client.0.smithi018.stdout:4/478: dread d3/d34/d43/f2e [0,4194304] 0 2024-04-12T20:56:07.839 INFO:tasks.workunit.client.0.smithi018.stdout:5/444: link d0/d2/d33/d46/d1e/f42 d0/d2/d33/d46/d1e/d55/d66/f91 0 2024-04-12T20:56:07.844 INFO:tasks.workunit.client.0.smithi018.stdout:3/420: rename d2/f1a to d2/d6/d30/d31/d38/d3a/d4c/f80 0 2024-04-12T20:56:07.847 INFO:tasks.workunit.client.0.smithi018.stdout:7/416: link d1/db/d31/l36 d1/db/d31/d45/d53/l8b 0 2024-04-12T20:56:07.851 INFO:tasks.workunit.client.0.smithi018.stdout:3/421: stat d2/d6/d30/d31/d38/d3a 0 2024-04-12T20:56:07.853 INFO:tasks.workunit.client.0.smithi018.stdout:5/445: mknod d0/d2/c92 0 2024-04-12T20:56:07.853 INFO:tasks.workunit.client.0.smithi018.stdout:7/417: write d1/d6/f27 [300452,65138] 0 2024-04-12T20:56:07.855 INFO:tasks.workunit.client.0.smithi018.stdout:4/479: truncate d3/d5d/d60/d64/d4f/f61 2400987 0 2024-04-12T20:56:07.860 INFO:tasks.workunit.client.0.smithi018.stdout:8/560: dwrite db/d25/d26/d27/f5f [0,4194304] 0 2024-04-12T20:56:07.862 INFO:tasks.workunit.client.0.smithi018.stdout:4/480: readlink d3/d34/d43/d4b/l87 0 2024-04-12T20:56:07.868 INFO:tasks.workunit.client.0.smithi018.stdout:4/481: write d3/d34/d43/d4b/f84 [100400,70284] 0 2024-04-12T20:56:07.868 INFO:tasks.workunit.client.0.smithi018.stdout:7/418: getdents d1/db/d31/d3a/d4f/d70 0 2024-04-12T20:56:07.869 INFO:tasks.workunit.client.0.smithi018.stdout:1/345: dread d3/d9/da/dd/d31/d32/d13/d19/d2d/f4d [4194304,4194304] 0 2024-04-12T20:56:07.874 INFO:tasks.workunit.client.0.smithi018.stdout:3/422: dread d2/f5d [0,4194304] 0 2024-04-12T20:56:07.875 INFO:tasks.workunit.client.0.smithi018.stdout:4/482: unlink d3/d34/d40/c51 0 2024-04-12T20:56:07.875 INFO:tasks.workunit.client.0.smithi018.stdout:7/419: mkdir d1/db/d31/d45/d8c 0 2024-04-12T20:56:07.879 INFO:tasks.workunit.client.0.smithi018.stdout:4/483: chown d3/d34/d40/f6d 0 1 2024-04-12T20:56:07.880 INFO:tasks.workunit.client.0.smithi018.stdout:1/346: rename d3/d9/da/dd/d31/d32/d13/l3a to d3/d9/da/dd/l7b 0 2024-04-12T20:56:07.884 INFO:tasks.workunit.client.0.smithi018.stdout:3/423: mkdir d2/d6/d30/d31/d38/d3a/d4c/d6b/d81 0 2024-04-12T20:56:07.884 INFO:tasks.workunit.client.0.smithi018.stdout:6/617: sync 2024-04-12T20:56:07.884 INFO:tasks.workunit.client.0.smithi018.stdout:0/372: sync 2024-04-12T20:56:07.884 INFO:tasks.workunit.client.0.smithi018.stdout:2/367: sync 2024-04-12T20:56:07.887 INFO:tasks.workunit.client.0.smithi018.stdout:4/484: stat d3/db/f37 0 2024-04-12T20:56:07.887 INFO:tasks.workunit.client.0.smithi018.stdout:5/446: dwrite d0/d2/d33/d46/d18/f59 [0,4194304] 0 2024-04-12T20:56:07.890 INFO:tasks.workunit.client.0.smithi018.stdout:0/373: chown da/d15/l24 405981 1 2024-04-12T20:56:07.892 INFO:tasks.workunit.client.0.smithi018.stdout:4/485: chown d3/d32/d33/d9b 1721767 1 2024-04-12T20:56:07.893 INFO:tasks.workunit.client.0.smithi018.stdout:2/368: truncate d5/d7/f37 2504880 0 2024-04-12T20:56:07.897 INFO:tasks.workunit.client.0.smithi018.stdout:8/561: dwrite db/d4e/d9a/d69/d6c/d7b/f8b [0,4194304] 0 2024-04-12T20:56:07.901 INFO:tasks.workunit.client.0.smithi018.stdout:3/424: read d2/d6/d30/d31/d38/d3a/d4c/f80 [4177030,70839] 0 2024-04-12T20:56:07.901 INFO:tasks.workunit.client.0.smithi018.stdout:1/347: dread d3/d9/da/dd/d31/d32/f41 [0,4194304] 0 2024-04-12T20:56:07.905 INFO:tasks.workunit.client.0.smithi018.stdout:0/374: truncate da/d15/d3e/d3f/f59 899612 0 2024-04-12T20:56:07.909 INFO:tasks.workunit.client.0.smithi018.stdout:7/420: dwrite d1/d6/f60 [0,4194304] 0 2024-04-12T20:56:07.913 INFO:tasks.workunit.client.0.smithi018.stdout:6/618: creat d5/d6/d1b/d3b/d8e/fea x:0 0 0 2024-04-12T20:56:07.913 INFO:tasks.workunit.client.0.smithi018.stdout:8/562: mkdir db/d4e/dab 0 2024-04-12T20:56:07.916 INFO:tasks.workunit.client.0.smithi018.stdout:5/447: write d0/d2/d33/d46/d1e/f42 [233446,41589] 0 2024-04-12T20:56:07.918 INFO:tasks.workunit.client.0.smithi018.stdout:9/617: sync 2024-04-12T20:56:07.921 INFO:tasks.workunit.client.0.smithi018.stdout:3/425: mkdir d2/d6/d30/d31/d38/d3a/d48/d82 0 2024-04-12T20:56:07.922 INFO:tasks.workunit.client.0.smithi018.stdout:1/348: symlink d3/d9/da/dd/d31/d32/l7c 0 2024-04-12T20:56:07.922 INFO:tasks.workunit.client.0.smithi018.stdout:0/375: mkdir da/d15/d17/d21/d36/d47/d81 0 2024-04-12T20:56:07.923 INFO:tasks.workunit.client.0.smithi018.stdout:7/421: mknod d1/db/d31/d30/c8d 0 2024-04-12T20:56:07.924 INFO:tasks.workunit.client.0.smithi018.stdout:2/369: link d5/d8/d10/d19/l71 d5/d7/l86 0 2024-04-12T20:56:07.926 INFO:tasks.workunit.client.0.smithi018.stdout:6/619: symlink d5/d6/d1b/d3b/d35/dda/leb 0 2024-04-12T20:56:07.927 INFO:tasks.workunit.client.0.smithi018.stdout:1/349: chown d3/d9/d11/f70 787147478 1 2024-04-12T20:56:07.934 INFO:tasks.workunit.client.0.smithi018.stdout:7/422: chown d1/db/d31/d3a/d4f/d70/l71 114730 1 2024-04-12T20:56:07.935 INFO:tasks.workunit.client.0.smithi018.stdout:3/426: fdatasync d2/d6/f24 0 2024-04-12T20:56:07.936 INFO:tasks.workunit.client.0.smithi018.stdout:5/448: stat d0/d2/d33/d10/d2f/d58/d89/d8d 0 2024-04-12T20:56:07.937 INFO:tasks.workunit.client.0.smithi018.stdout:6/620: write d5/d15/fe8 [4352143,73336] 0 2024-04-12T20:56:07.938 INFO:tasks.workunit.client.0.smithi018.stdout:0/376: truncate da/d15/d17/d5c/f64 128516 0 2024-04-12T20:56:07.942 INFO:tasks.workunit.client.0.smithi018.stdout:0/377: stat da/d15/d4d/d6b/f7b 0 2024-04-12T20:56:07.942 INFO:tasks.workunit.client.0.smithi018.stdout:6/621: dread - d5/d6/d1b/d3b/d8e/f92 zero size 2024-04-12T20:56:07.942 INFO:tasks.workunit.client.0.smithi018.stdout:1/350: mknod d3/d9/da/dd/d31/d32/c7d 0 2024-04-12T20:56:07.943 INFO:tasks.workunit.client.0.smithi018.stdout:3/427: mkdir d2/d83 0 2024-04-12T20:56:07.943 INFO:tasks.workunit.client.0.smithi018.stdout:4/486: dread d3/f6b [0,4194304] 0 2024-04-12T20:56:07.946 INFO:tasks.workunit.client.0.smithi018.stdout:5/449: write d0/d2/d33/d10/d2f/d58/f86 [786464,20809] 0 2024-04-12T20:56:07.950 INFO:tasks.workunit.client.0.smithi018.stdout:4/487: symlink d3/d34/d40/l9d 0 2024-04-12T20:56:07.956 INFO:tasks.workunit.client.0.smithi018.stdout:4/488: read d3/d32/f81 [3284138,19982] 0 2024-04-12T20:56:07.957 INFO:tasks.workunit.client.0.smithi018.stdout:3/428: write d2/d6/d30/d31/f67 [5009948,94981] 0 2024-04-12T20:56:07.958 INFO:tasks.workunit.client.0.smithi018.stdout:6/622: creat d5/d6/d1b/fec x:0 0 0 2024-04-12T20:56:07.962 INFO:tasks.workunit.client.0.smithi018.stdout:0/378: truncate da/d15/d17/d21/d36/f7c 406470 0 2024-04-12T20:56:07.965 INFO:tasks.workunit.client.0.smithi018.stdout:5/450: creat d0/d2/d33/d46/d1e/d55/d66/f93 x:0 0 0 2024-04-12T20:56:07.965 INFO:tasks.workunit.client.0.smithi018.stdout:9/618: dwrite d3/d6/d9/dc/d5a/d79/d7e/d9b/d21/d41/d78/f7a [0,4194304] 0 2024-04-12T20:56:07.969 INFO:tasks.workunit.client.0.smithi018.stdout:1/351: link d3/d9/da/dd/c39 d3/d65/c7e 0 2024-04-12T20:56:07.970 INFO:tasks.workunit.client.0.smithi018.stdout:6/623: write d5/d6/f11 [8168323,66017] 0 2024-04-12T20:56:07.971 INFO:tasks.workunit.client.0.smithi018.stdout:8/563: dread db/d25/f52 [0,4194304] 0 2024-04-12T20:56:07.973 INFO:tasks.workunit.client.0.smithi018.stdout:2/370: dwrite d5/d7/d31/f4f [4194304,4194304] 0 2024-04-12T20:56:07.974 INFO:tasks.workunit.client.0.smithi018.stdout:0/379: truncate da/d15/d17/d21/d36/f7c 1043455 0 2024-04-12T20:56:07.976 INFO:tasks.workunit.client.0.smithi018.stdout:1/352: dread - d3/d9/da/dd/f55 zero size 2024-04-12T20:56:07.978 INFO:tasks.workunit.client.0.smithi018.stdout:5/451: creat d0/d2/d11/d37/f94 x:0 0 0 2024-04-12T20:56:07.982 INFO:tasks.workunit.client.0.smithi018.stdout:8/564: readlink db/d4e/d56/d59/d76/da5/l10 0 2024-04-12T20:56:07.982 INFO:tasks.workunit.client.0.smithi018.stdout:3/429: creat d2/d6/f84 x:0 0 0 2024-04-12T20:56:07.986 INFO:tasks.workunit.client.0.smithi018.stdout:2/371: creat d5/d8/d10/d43/f87 x:0 0 0 2024-04-12T20:56:07.988 INFO:tasks.workunit.client.0.smithi018.stdout:5/452: chown d0/d2/d33/d46/d1e/d2e 51 1 2024-04-12T20:56:07.990 INFO:tasks.workunit.client.0.smithi018.stdout:1/353: symlink d3/d9/d11/l7f 0 2024-04-12T20:56:07.990 INFO:tasks.workunit.client.0.smithi018.stdout:9/619: truncate d3/d6/d9/dc/d13/fa9 489923 0 2024-04-12T20:56:07.993 INFO:tasks.workunit.client.0.smithi018.stdout:6/624: rename d5/df/d31/cac to d5/d6/d1b/d3b/d8e/d8f/ced 0 2024-04-12T20:56:07.996 INFO:tasks.workunit.client.0.smithi018.stdout:8/565: fsync db/d25/d26/d27/f33 0 2024-04-12T20:56:07.997 INFO:tasks.workunit.client.0.smithi018.stdout:3/430: truncate d2/d43/d61/f63 780403 0 2024-04-12T20:56:08.002 INFO:tasks.workunit.client.0.smithi018.stdout:7/423: dwrite d1/f3e [0,4194304] 0 2024-04-12T20:56:08.012 INFO:tasks.workunit.client.0.smithi018.stdout:4/489: write d3/d34/d3d/d80/f8c [3023004,61454] 0 2024-04-12T20:56:08.015 INFO:tasks.workunit.client.0.smithi018.stdout:6/625: symlink d5/df/d31/d4e/lee 0 2024-04-12T20:56:08.023 INFO:tasks.workunit.client.0.smithi018.stdout:0/380: dwrite da/d15/d17/d21/d27/f78 [4194304,4194304] 0 2024-04-12T20:56:08.025 INFO:tasks.workunit.client.0.smithi018.stdout:6/626: symlink d5/df/d9b/lef 0 2024-04-12T20:56:08.025 INFO:tasks.workunit.client.0.smithi018.stdout:1/354: truncate d3/d9/da/f5f 1129067 0 2024-04-12T20:56:08.030 INFO:tasks.workunit.client.0.smithi018.stdout:8/566: dread db/d4e/d56/f63 [0,4194304] 0 2024-04-12T20:56:08.034 INFO:tasks.workunit.client.0.smithi018.stdout:6/627: creat d5/d6/d1b/d3b/d8e/ff0 x:0 0 0 2024-04-12T20:56:08.035 INFO:tasks.workunit.client.0.smithi018.stdout:5/453: link d0/d2/d33/d46/d1e/d2e/c63 d0/d2/d33/d46/d35/c95 0 2024-04-12T20:56:08.035 INFO:tasks.workunit.client.0.smithi018.stdout:7/424: creat d1/f8e x:0 0 0 2024-04-12T20:56:08.040 INFO:tasks.workunit.client.0.smithi018.stdout:6/628: creat d5/d6/d1b/ff1 x:0 0 0 2024-04-12T20:56:08.042 INFO:tasks.workunit.client.0.smithi018.stdout:1/355: mkdir d3/d9/da/dd/d31/d32/d33/d80 0 2024-04-12T20:56:08.049 INFO:tasks.workunit.client.0.smithi018.stdout:6/629: fsync d5/df/d3f/f49 0 2024-04-12T20:56:08.050 INFO:tasks.workunit.client.0.smithi018.stdout:0/381: dwrite da/d15/d17/d21/f46 [0,4194304] 0 2024-04-12T20:56:08.051 INFO:tasks.workunit.client.0.smithi018.stdout:1/356: mknod d3/d9/da/dd/d31/d32/d33/c81 0 2024-04-12T20:56:08.051 INFO:tasks.workunit.client.0.smithi018.stdout:3/431: dread d2/f5 [4194304,4194304] 0 2024-04-12T20:56:08.053 INFO:tasks.workunit.client.0.smithi018.stdout:8/567: rename db/d4e/d9a/d69/d6c/l8f to db/d4e/lac 0 2024-04-12T20:56:08.056 INFO:tasks.workunit.client.0.smithi018.stdout:1/357: write d3/f4 [9342229,107450] 0 2024-04-12T20:56:08.058 INFO:tasks.workunit.client.0.smithi018.stdout:0/382: creat da/d15/d17/d5c/f82 x:0 0 0 2024-04-12T20:56:08.063 INFO:tasks.workunit.client.0.smithi018.stdout:6/630: creat d5/df/d3f/ff2 x:0 0 0 2024-04-12T20:56:08.074 INFO:tasks.workunit.client.0.smithi018.stdout:6/631: mkdir d5/d6/d1b/d3b/d35/d4f/df3 0 2024-04-12T20:56:08.076 INFO:tasks.workunit.client.0.smithi018.stdout:5/454: rename d0/d2/d33/d46/d1e/d2e/c16 to d0/d2/d33/d46/c96 0 2024-04-12T20:56:08.080 INFO:tasks.workunit.client.0.smithi018.stdout:1/358: mkdir d3/d9/da/dd/d31/d32/d13/d19/d2d/d4e/d7a/d82 0 2024-04-12T20:56:08.080 INFO:tasks.workunit.client.0.smithi018.stdout:5/455: chown d0/d5b/f5c 933965815 1 2024-04-12T20:56:08.085 INFO:tasks.workunit.client.0.smithi018.stdout:5/456: chown d0/d2/d33/c5f 0 1 2024-04-12T20:56:08.085 INFO:tasks.workunit.client.0.smithi018.stdout:8/568: dwrite db/d25/d26/d27/f50 [0,4194304] 0 2024-04-12T20:56:08.089 INFO:tasks.workunit.client.0.smithi018.stdout:3/432: rename d2/d6/d30/d31/l47 to d2/l85 0 2024-04-12T20:56:08.090 INFO:tasks.workunit.client.0.smithi018.stdout:0/383: dwrite da/fc [0,4194304] 0 2024-04-12T20:56:08.093 INFO:tasks.workunit.client.0.smithi018.stdout:5/457: mkdir d0/d2/d11/d37/d97 0 2024-04-12T20:56:08.096 INFO:tasks.workunit.client.0.smithi018.stdout:8/569: unlink db/d72/f7d 0 2024-04-12T20:56:08.100 INFO:tasks.workunit.client.0.smithi018.stdout:0/384: fsync da/f25 0 2024-04-12T20:56:08.105 INFO:tasks.workunit.client.0.smithi018.stdout:9/620: truncate d3/d6/d9/dc/d5a/d79/d7e/d9b/d21/d41/d78/f7a 176533 0 2024-04-12T20:56:08.107 INFO:tasks.workunit.client.0.smithi018.stdout:5/458: read d0/d2/d33/d46/d1e/f48 [656027,10607] 0 2024-04-12T20:56:08.108 INFO:tasks.workunit.client.0.smithi018.stdout:6/632: dwrite d5/df/d19/faa [0,4194304] 0 2024-04-12T20:56:08.109 INFO:tasks.workunit.client.0.smithi018.stdout:3/433: rename d2/d4d/c6a to d2/d6/d3d/c86 0 2024-04-12T20:56:08.110 INFO:tasks.workunit.client.0.smithi018.stdout:9/621: write d3/f80 [2362872,102573] 0 2024-04-12T20:56:08.111 INFO:tasks.workunit.client.0.smithi018.stdout:0/385: mkdir da/d15/d3e/d83 0 2024-04-12T20:56:08.112 INFO:tasks.workunit.client.0.smithi018.stdout:3/434: read - d2/d6/d30/d31/f69 zero size 2024-04-12T20:56:08.115 INFO:tasks.workunit.client.0.smithi018.stdout:6/633: write d5/d15/dd1/fe4 [753126,77473] 0 2024-04-12T20:56:08.119 INFO:tasks.workunit.client.0.smithi018.stdout:3/435: write d2/d6/f2f [3261394,80392] 0 2024-04-12T20:56:08.126 INFO:tasks.workunit.client.0.smithi018.stdout:9/622: rmdir d3/d6/d1b 39 2024-04-12T20:56:08.133 INFO:tasks.workunit.client.0.smithi018.stdout:1/359: dwrite d3/d9/da/dd/d31/f45 [0,4194304] 0 2024-04-12T20:56:08.133 INFO:tasks.workunit.client.0.smithi018.stdout:5/459: rename d0/d2/d33/d46/d18/f59 to d0/d2/d33/d46/d1e/d2e/d7f/f98 0 2024-04-12T20:56:08.140 INFO:tasks.workunit.client.0.smithi018.stdout:8/570: dwrite db/d4e/d56/d59/d76/da5/d45/f79 [0,4194304] 0 2024-04-12T20:56:08.142 INFO:tasks.workunit.client.0.smithi018.stdout:3/436: creat d2/d6/d3d/f87 x:0 0 0 2024-04-12T20:56:08.145 INFO:tasks.workunit.client.0.smithi018.stdout:6/634: rename d5/d15/d60/f9a to d5/df/d19/dc5/ff4 0 2024-04-12T20:56:08.150 INFO:tasks.workunit.client.0.smithi018.stdout:1/360: readlink d3/d9/da/dd/l7b 0 2024-04-12T20:56:08.152 INFO:tasks.workunit.client.0.smithi018.stdout:3/437: truncate d2/f78 76333 0 2024-04-12T20:56:08.153 INFO:tasks.workunit.client.0.smithi018.stdout:9/623: rename d3/d6/d9/dc/d13/f42 to d3/d93/fbc 0 2024-04-12T20:56:08.155 INFO:tasks.workunit.client.0.smithi018.stdout:8/571: read db/d4e/d56/d59/d76/da5/f1a [2886857,62862] 0 2024-04-12T20:56:08.156 INFO:tasks.workunit.client.0.smithi018.stdout:0/386: dwrite da/d15/d17/d5c/f64 [0,4194304] 0 2024-04-12T20:56:08.160 INFO:tasks.workunit.client.0.smithi018.stdout:1/361: stat d3/d9/da/dd/d31/d32/d13/d19/f62 0 2024-04-12T20:56:08.166 INFO:tasks.workunit.client.0.smithi018.stdout:8/572: symlink db/d4e/d56/d59/d76/da5/lad 0 2024-04-12T20:56:08.168 INFO:tasks.workunit.client.0.smithi018.stdout:9/624: stat d3/d6/d9/dc/d6f/d77/l9e 0 2024-04-12T20:56:08.168 INFO:tasks.workunit.client.0.smithi018.stdout:0/387: rename da to da/d15/d17/d5c/d79/d84 22 2024-04-12T20:56:08.169 INFO:tasks.workunit.client.0.smithi018.stdout:8/573: read db/d4e/d73/f92 [1447737,24610] 0 2024-04-12T20:56:08.174 INFO:tasks.workunit.client.0.smithi018.stdout:5/460: dwrite d0/d5b/d84/f65 [0,4194304] 0 2024-04-12T20:56:08.174 INFO:tasks.workunit.client.0.smithi018.stdout:3/438: truncate d2/f5d 3984023 0 2024-04-12T20:56:08.181 INFO:tasks.workunit.client.0.smithi018.stdout:5/461: rmdir d0/d2/d11/d37 39 2024-04-12T20:56:08.181 INFO:tasks.workunit.client.0.smithi018.stdout:8/574: creat db/d95/fae x:0 0 0 2024-04-12T20:56:08.186 INFO:tasks.workunit.client.0.smithi018.stdout:3/439: getdents d2/d6/d34/d71/d7b 0 2024-04-12T20:56:08.188 INFO:tasks.workunit.client.0.smithi018.stdout:5/462: creat d0/d2/d33/d46/d1e/d55/f99 x:0 0 0 2024-04-12T20:56:08.192 INFO:tasks.workunit.client.0.smithi018.stdout:8/575: symlink db/d25/d26/d27/laf 0 2024-04-12T20:56:08.194 INFO:tasks.workunit.client.0.smithi018.stdout:8/576: fsync db/d4e/d56/d59/f82 0 2024-04-12T20:56:08.197 INFO:tasks.workunit.client.0.smithi018.stdout:8/577: chown db/d4e/d9a/d71 6997947 1 2024-04-12T20:56:08.199 INFO:tasks.workunit.client.0.smithi018.stdout:9/625: getdents d3/d6/d9/dc 0 2024-04-12T20:56:08.205 INFO:tasks.workunit.client.0.smithi018.stdout:9/626: stat d3/d6/d43/l85 0 2024-04-12T20:56:08.205 INFO:tasks.workunit.client.0.smithi018.stdout:6/635: dread d5/df/d19/d2b/f9c [0,4194304] 0 2024-04-12T20:56:08.208 INFO:tasks.workunit.client.0.smithi018.stdout:5/463: truncate d0/d2/d33/d10/f39 3927704 0 2024-04-12T20:56:08.210 INFO:tasks.workunit.client.0.smithi018.stdout:6/636: write d5/d6/d1b/d3b/d22/f81 [2105863,129664] 0 2024-04-12T20:56:08.213 INFO:tasks.workunit.client.0.smithi018.stdout:9/627: creat d3/d6/d9/dc/d13/d5b/d6b/fbd x:0 0 0 2024-04-12T20:56:08.214 INFO:tasks.workunit.client.0.smithi018.stdout:3/440: dwrite d2/f5 [0,4194304] 0 2024-04-12T20:56:08.215 INFO:tasks.workunit.client.0.smithi018.stdout:6/637: write d5/d15/fbf [3967298,58420] 0 2024-04-12T20:56:08.215 INFO:tasks.workunit.client.0.smithi018.stdout:5/464: dread - d0/d2/d33/d46/d1e/d2e/f8a zero size 2024-04-12T20:56:08.225 INFO:tasks.workunit.client.0.smithi018.stdout:6/638: write d5/df/d19/d27/fa2 [810790,34931] 0 2024-04-12T20:56:08.227 INFO:tasks.workunit.client.0.smithi018.stdout:5/465: mkdir d0/d2/d33/d46/d1e/d9a 0 2024-04-12T20:56:08.230 INFO:tasks.workunit.client.0.smithi018.stdout:3/441: getdents d2/d6/d34/d71 0 2024-04-12T20:56:08.231 INFO:tasks.workunit.client.0.smithi018.stdout:6/639: write d5/df/d19/faa [3428369,61448] 0 2024-04-12T20:56:08.232 INFO:tasks.workunit.client.0.smithi018.stdout:8/578: dwrite db/d4e/d56/d59/d76/da5/f5e [0,4194304] 0 2024-04-12T20:56:08.237 INFO:tasks.workunit.client.0.smithi018.stdout:1/362: dread d3/d9/da/d14/f38 [0,4194304] 0 2024-04-12T20:56:08.238 INFO:tasks.workunit.client.0.smithi018.stdout:3/442: readlink d2/d43/d61/l73 0 2024-04-12T20:56:08.238 INFO:tasks.workunit.client.0.smithi018.stdout:8/579: write db/d3b/f55 [996820,117947] 0 2024-04-12T20:56:08.240 INFO:tasks.workunit.client.0.smithi018.stdout:8/580: fdatasync db/d4e/d56/f63 0 2024-04-12T20:56:08.249 INFO:tasks.workunit.client.0.smithi018.stdout:8/581: creat db/d4e/d56/d59/d76/da5/fb0 x:0 0 0 2024-04-12T20:56:08.266 INFO:tasks.workunit.client.0.smithi018.stdout:9/628: dwrite d3/d6/d9/dc/d13/d5b/d6b/fbd [0,4194304] 0 2024-04-12T20:56:08.268 INFO:tasks.workunit.client.0.smithi018.stdout:5/466: dwrite d0/d2/d33/d10/d60/f73 [0,4194304] 0 2024-04-12T20:56:08.280 INFO:tasks.workunit.client.0.smithi018.stdout:8/582: link db/d4e/d56/d59/c75 db/d4e/d9a/daa/cb1 0 2024-04-12T20:56:08.283 INFO:tasks.workunit.client.0.smithi018.stdout:6/640: dwrite d5/fd [0,4194304] 0 2024-04-12T20:56:08.286 INFO:tasks.workunit.client.0.smithi018.stdout:3/443: dwrite d2/d6/d30/d31/d38/d3a/d48/f51 [4194304,4194304] 0 2024-04-12T20:56:08.287 INFO:tasks.workunit.client.0.smithi018.stdout:5/467: creat d0/d2/d33/d10/d26/f9b x:0 0 0 2024-04-12T20:56:08.293 INFO:tasks.workunit.client.0.smithi018.stdout:5/468: write d0/d2/d11/f2a [4360477,43118] 0 2024-04-12T20:56:08.294 INFO:tasks.workunit.client.0.smithi018.stdout:8/583: fsync db/d4e/d56/d59/d76/da5/f12 0 2024-04-12T20:56:08.301 INFO:tasks.workunit.client.0.smithi018.stdout:8/584: mkdir db/d4e/d73/db2 0 2024-04-12T20:56:08.302 INFO:tasks.workunit.client.0.smithi018.stdout:6/641: getdents d5/d6/d1b/d3b/d8e/d8f 0 2024-04-12T20:56:08.312 INFO:tasks.workunit.client.0.smithi018.stdout:9/629: dwrite d3/d6/d9/d1e/f74 [0,4194304] 0 2024-04-12T20:56:08.313 INFO:tasks.workunit.client.0.smithi018.stdout:8/585: link f7 db/fb3 0 2024-04-12T20:56:08.317 INFO:tasks.workunit.client.0.smithi018.stdout:3/444: dread d2/f14 [0,4194304] 0 2024-04-12T20:56:08.318 INFO:tasks.workunit.client.0.smithi018.stdout:8/586: creat db/d4e/d9a/d69/d6c/fb4 x:0 0 0 2024-04-12T20:56:08.319 INFO:tasks.workunit.client.0.smithi018.stdout:9/630: rmdir d3/d93/da8 0 2024-04-12T20:56:08.320 INFO:tasks.workunit.client.0.smithi018.stdout:3/445: fdatasync d2/d6/d30/d31/d38/f74 0 2024-04-12T20:56:08.321 INFO:tasks.workunit.client.0.smithi018.stdout:5/469: dread d0/d5b/d84/f65 [0,4194304] 0 2024-04-12T20:56:08.323 INFO:tasks.workunit.client.0.smithi018.stdout:9/631: fsync d3/d6/d9/d1e/f40 0 2024-04-12T20:56:08.325 INFO:tasks.workunit.client.0.smithi018.stdout:3/446: stat d2/d6/d3d/f5a 0 2024-04-12T20:56:08.330 INFO:tasks.workunit.client.0.smithi018.stdout:8/587: symlink db/d4e/d56/d59/d76/da5/d45/d86/lb5 0 2024-04-12T20:56:08.336 INFO:tasks.workunit.client.0.smithi018.stdout:6/642: dwrite d5/d6/d1b/d3b/fdc [0,4194304] 0 2024-04-12T20:56:08.336 INFO:tasks.workunit.client.0.smithi018.stdout:9/632: unlink d3/d6/d9/dc/d6f/d77/l9e 0 2024-04-12T20:56:08.342 INFO:tasks.workunit.client.0.smithi018.stdout:9/633: read d3/d6/d9/dc/d46/f48 [5714,78591] 0 2024-04-12T20:56:08.345 INFO:tasks.workunit.client.0.smithi018.stdout:5/470: dwrite d0/d2/d33/d46/f3a [0,4194304] 0 2024-04-12T20:56:08.346 INFO:tasks.workunit.client.0.smithi018.stdout:3/447: dread d2/d6/d30/f58 [0,4194304] 0 2024-04-12T20:56:08.362 INFO:tasks.workunit.client.0.smithi018.stdout:9/634: truncate d3/d6/d9/dc/d13/fa9 647716 0 2024-04-12T20:56:08.362 INFO:tasks.workunit.client.0.smithi018.stdout:3/448: mkdir d2/d4d/d88 0 2024-04-12T20:56:08.371 INFO:tasks.workunit.client.0.smithi018.stdout:5/471: link d0/d2/d33/d46/f3a d0/d2/d33/d46/d1e/d2e/f9c 0 2024-04-12T20:56:08.379 INFO:tasks.workunit.client.0.smithi018.stdout:5/472: mkdir d0/d2/d33/d10/d9d 0 2024-04-12T20:56:08.383 INFO:tasks.workunit.client.0.smithi018.stdout:6/643: dwrite d5/df/d19/d43/fc3 [0,4194304] 0 2024-04-12T20:56:08.394 INFO:tasks.workunit.client.0.smithi018.stdout:5/473: mkdir d0/d2/d33/d46/d9e 0 2024-04-12T20:56:08.398 INFO:tasks.workunit.client.0.smithi018.stdout:6/644: write d5/df/d19/d43/fc3 [4141228,128374] 0 2024-04-12T20:56:08.400 INFO:tasks.workunit.client.0.smithi018.stdout:5/474: fsync d0/d2/f1a 0 2024-04-12T20:56:08.402 INFO:tasks.workunit.client.0.smithi018.stdout:9/635: truncate d3/d6/d9/dc/f97 149678 0 2024-04-12T20:56:08.407 INFO:tasks.workunit.client.0.smithi018.stdout:3/449: dwrite d2/d6/d30/d31/d38/d3a/f45 [0,4194304] 0 2024-04-12T20:56:08.411 INFO:tasks.workunit.client.0.smithi018.stdout:1/363: truncate d3/d9/da/dd/d31/d32/d13/f46 294952 0 2024-04-12T20:56:08.413 INFO:tasks.workunit.client.0.smithi018.stdout:3/450: chown d2/d6/d30/d31 887789 1 2024-04-12T20:56:08.420 INFO:tasks.workunit.client.0.smithi018.stdout:1/364: chown d3/d9/da/dd/d31/d32/c1d 12355058 1 2024-04-12T20:56:08.420 INFO:tasks.workunit.client.0.smithi018.stdout:5/475: creat d0/d2/d11/d37/f9f x:0 0 0 2024-04-12T20:56:08.422 INFO:tasks.workunit.client.0.smithi018.stdout:6/645: mknod d5/d6/d1b/d8c/dc0/cf5 0 2024-04-12T20:56:08.428 INFO:tasks.workunit.client.0.smithi018.stdout:3/451: creat d2/d6/d30/d31/d38/d3a/d48/f89 x:0 0 0 2024-04-12T20:56:08.434 INFO:tasks.workunit.client.0.smithi018.stdout:1/365: mknod d3/d9/da/dd/d31/d32/d13/d19/d2d/d48/c83 0 2024-04-12T20:56:08.436 INFO:tasks.workunit.client.0.smithi018.stdout:8/588: write db/d4e/d7f/f9e [2045137,103736] 0 2024-04-12T20:56:08.439 INFO:tasks.workunit.client.0.smithi018.stdout:5/476: creat d0/d2b/fa0 x:0 0 0 2024-04-12T20:56:08.444 INFO:tasks.workunit.client.0.smithi018.stdout:1/366: write d3/d9/da/dd/d31/d32/f1b [718990,124007] 0 2024-04-12T20:56:08.444 INFO:tasks.workunit.client.0.smithi018.stdout:9/636: stat d3/d6/d9/d3f/f6e 0 2024-04-12T20:56:08.446 INFO:tasks.workunit.client.0.smithi018.stdout:8/589: creat db/d4e/d56/d59/d76/da5/d45/d86/da1/fb6 x:0 0 0 2024-04-12T20:56:08.449 INFO:tasks.workunit.client.0.smithi018.stdout:5/477: truncate d0/d2/d33/d46/d1e/d2e/f8a 669164 0 2024-04-12T20:56:08.450 INFO:tasks.workunit.client.0.smithi018.stdout:1/367: chown d3/d9/da/dd/d31/d32/f1b 23 1 2024-04-12T20:56:08.450 INFO:tasks.workunit.client.0.smithi018.stdout:8/590: unlink db/c58 0 2024-04-12T20:56:08.451 INFO:tasks.workunit.client.0.smithi018.stdout:5/478: fdatasync d0/d5b/d84/f7d 0 2024-04-12T20:56:08.453 INFO:tasks.workunit.client.0.smithi018.stdout:6/646: link d5/d6/d1b/d3b/d35/f33 d5/df/d9b/ff6 0 2024-04-12T20:56:08.455 INFO:tasks.workunit.client.0.smithi018.stdout:8/591: chown db/d25/d26/f3f 0 1 2024-04-12T20:56:08.459 INFO:tasks.workunit.client.0.smithi018.stdout:8/592: write db/d3b/f3d [1718289,60697] 0 2024-04-12T20:56:08.463 INFO:tasks.workunit.client.0.smithi018.stdout:5/479: mknod d0/ca1 0 2024-04-12T20:56:08.466 INFO:tasks.workunit.client.0.smithi018.stdout:9/637: unlink d3/d6/lb7 0 2024-04-12T20:56:08.469 INFO:tasks.workunit.client.0.smithi018.stdout:3/452: dwrite d2/d6/d30/d31/d38/d3a/d4c/f80 [0,4194304] 0 2024-04-12T20:56:08.470 INFO:tasks.workunit.client.0.smithi018.stdout:8/593: creat db/d4e/d56/d59/d76/da5/fb7 x:0 0 0 2024-04-12T20:56:08.479 INFO:tasks.workunit.client.0.smithi018.stdout:8/594: truncate db/d4e/d7f/f93 819071 0 2024-04-12T20:56:08.481 INFO:tasks.workunit.client.0.smithi018.stdout:3/453: write d2/d6/d30/f37 [1352839,82060] 0 2024-04-12T20:56:08.484 INFO:tasks.workunit.client.0.smithi018.stdout:9/638: dread d3/d6/d9/dc/f22 [0,4194304] 0 2024-04-12T20:56:08.485 INFO:tasks.workunit.client.0.smithi018.stdout:5/480: mkdir d0/d2/d33/d46/da2 0 2024-04-12T20:56:08.487 INFO:tasks.workunit.client.0.smithi018.stdout:8/595: mknod db/d4e/d7f/cb8 0 2024-04-12T20:56:08.489 INFO:tasks.workunit.client.0.smithi018.stdout:5/481: chown d0/d2/d33/d46/d1e/d9a 1392557 1 2024-04-12T20:56:08.492 INFO:tasks.workunit.client.0.smithi018.stdout:3/454: rename d2/f42 to d2/d4d/d88/f8a 0 2024-04-12T20:56:08.492 INFO:tasks.workunit.client.0.smithi018.stdout:5/482: readlink d0/d2/d33/d46/l25 0 2024-04-12T20:56:08.493 INFO:tasks.workunit.client.0.smithi018.stdout:6/647: dwrite d5/df/d31/d4e/f99 [0,4194304] 0 2024-04-12T20:56:08.494 INFO:tasks.workunit.client.0.smithi018.stdout:2/372: sync 2024-04-12T20:56:08.501 INFO:tasks.workunit.client.0.smithi018.stdout:8/596: creat db/d4e/d56/d59/fb9 x:0 0 0 2024-04-12T20:56:08.502 INFO:tasks.workunit.client.0.smithi018.stdout:1/368: dwrite d3/d9/da/dd/d31/d32/d13/f46 [0,4194304] 0 2024-04-12T20:56:08.506 INFO:tasks.workunit.client.0.smithi018.stdout:8/597: dread - db/d25/d26/d27/fa0 zero size 2024-04-12T20:56:08.506 INFO:tasks.workunit.client.0.smithi018.stdout:6/648: mknod d5/df/d19/d27/dc8/cf7 0 2024-04-12T20:56:08.506 INFO:tasks.workunit.client.0.smithi018.stdout:2/373: truncate d5/d7/d28/f59 4427162 0 2024-04-12T20:56:08.507 INFO:tasks.workunit.client.0.smithi018.stdout:5/483: fsync d0/d2/d33/d46/d1e/d2e/f9c 0 2024-04-12T20:56:08.516 INFO:tasks.workunit.client.0.smithi018.stdout:3/455: mknod d2/d6/d30/d31/d38/d3a/d76/c8b 0 2024-04-12T20:56:08.516 INFO:tasks.workunit.client.0.smithi018.stdout:1/369: chown d3/d9/da/dd/d31/d32/d13/l22 786055105 1 2024-04-12T20:56:08.517 INFO:tasks.workunit.client.0.smithi018.stdout:5/484: chown d0/d2/d33/d46/d18/f3b 8172595 1 2024-04-12T20:56:08.517 INFO:tasks.workunit.client.0.smithi018.stdout:8/598: rmdir db/d4e/d9a 39 2024-04-12T20:56:08.517 INFO:tasks.workunit.client.0.smithi018.stdout:6/649: write d5/d6/d1b/d3b/d22/fb3 [727377,69054] 0 2024-04-12T20:56:08.520 INFO:tasks.workunit.client.0.smithi018.stdout:9/639: dwrite d3/d6/d9/dc/d5a/d79/fb3 [0,4194304] 0 2024-04-12T20:56:08.520 INFO:tasks.workunit.client.0.smithi018.stdout:2/374: mkdir d5/d8/d10/d19/d24/d6c/d88 0 2024-04-12T20:56:08.529 INFO:tasks.workunit.client.0.smithi018.stdout:8/599: mkdir db/d4e/d7f/dba 0 2024-04-12T20:56:08.531 INFO:tasks.workunit.client.0.smithi018.stdout:3/456: mknod d2/d6/d30/d31/c8c 0 2024-04-12T20:56:08.532 INFO:tasks.workunit.client.0.smithi018.stdout:1/370: mkdir d3/d9/da/dd/d31/d32/d33/d84 0 2024-04-12T20:56:08.534 INFO:tasks.workunit.client.0.smithi018.stdout:2/375: creat d5/d12/f89 x:0 0 0 2024-04-12T20:56:08.541 INFO:tasks.workunit.client.0.smithi018.stdout:1/371: mknod d3/d9/da/dd/d31/d44/c85 0 2024-04-12T20:56:08.541 INFO:tasks.workunit.client.0.smithi018.stdout:6/650: rename d5/d6/c97 to d5/d6/d1b/cf8 0 2024-04-12T20:56:08.542 INFO:tasks.workunit.client.0.smithi018.stdout:8/600: mkdir db/d4e/d56/d59/d76/da5/d45/d86/dbb 0 2024-04-12T20:56:08.543 INFO:tasks.workunit.client.0.smithi018.stdout:6/651: chown d5/d71 3 1 2024-04-12T20:56:08.548 INFO:tasks.workunit.client.0.smithi018.stdout:2/376: mknod d5/d7/d79/c8a 0 2024-04-12T20:56:08.550 INFO:tasks.workunit.client.0.smithi018.stdout:1/372: write d3/d9/da/dd/d31/d32/d13/d19/f62 [46578,14036] 0 2024-04-12T20:56:08.556 INFO:tasks.workunit.client.0.smithi018.stdout:5/485: dread d0/d2/d33/d10/d60/f73 [0,4194304] 0 2024-04-12T20:56:08.556 INFO:tasks.workunit.client.0.smithi018.stdout:8/601: unlink db/d4e/lac 0 2024-04-12T20:56:08.556 INFO:tasks.workunit.client.0.smithi018.stdout:8/602: chown db/d4e/f6d 13893 1 2024-04-12T20:56:08.559 INFO:tasks.workunit.client.0.smithi018.stdout:2/377: rename d5/d8/d10/f69 to d5/d12/d54/d3e/f8b 0 2024-04-12T20:56:08.565 INFO:tasks.workunit.client.0.smithi018.stdout:1/373: mknod d3/d9/da/d14/d73/c86 0 2024-04-12T20:56:08.566 INFO:tasks.workunit.client.0.smithi018.stdout:5/486: mkdir d0/d2/d33/d46/da2/da3 0 2024-04-12T20:56:08.569 INFO:tasks.workunit.client.0.smithi018.stdout:2/378: link d5/d8/d33/f76 d5/d7/d79/f8c 0 2024-04-12T20:56:08.575 INFO:tasks.workunit.client.0.smithi018.stdout:5/487: mknod d0/ca4 0 2024-04-12T20:56:08.576 INFO:tasks.workunit.client.0.smithi018.stdout:2/379: rmdir d5/d8/d25 39 2024-04-12T20:56:08.578 INFO:tasks.workunit.client.0.smithi018.stdout:3/457: dwrite d2/d6/f24 [0,4194304] 0 2024-04-12T20:56:08.581 INFO:tasks.workunit.client.0.smithi018.stdout:5/488: unlink d0/d2b/d40/l87 0 2024-04-12T20:56:08.583 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:56:08.581+0000 7fa9ed200700 1 -- 172.21.15.18:0/2682322664 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa9e806e7a0 msgr2=0x7fa9e806ebb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:56:08.583 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:56:08.581+0000 7fa9ed200700 1 --2- 172.21.15.18:0/2682322664 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa9e806e7a0 0x7fa9e806ebb0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fa9d8009fa0 tx=0x7fa9d8009d20 comp rx=0 tx=0).stop 2024-04-12T20:56:08.583 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:56:08.582+0000 7fa9ed200700 1 -- 172.21.15.18:0/2682322664 shutdown_connections 2024-04-12T20:56:08.583 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:56:08.582+0000 7fa9ed200700 1 --2- 172.21.15.18:0/2682322664 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa9e806f0f0 0x7fa9e8108fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:56:08.583 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:56:08.582+0000 7fa9ed200700 1 --2- 172.21.15.18:0/2682322664 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa9e806e7a0 0x7fa9e806ebb0 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:56:08.583 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:56:08.582+0000 7fa9ed200700 1 -- 172.21.15.18:0/2682322664 >> 172.21.15.18:0/2682322664 conn(0x7fa9e806a380 msgr2=0x7fa9e806c7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T20:56:08.583 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:56:08.582+0000 7fa9ed200700 1 -- 172.21.15.18:0/2682322664 shutdown_connections 2024-04-12T20:56:08.583 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:56:08.582+0000 7fa9ed200700 1 -- 172.21.15.18:0/2682322664 wait complete. 2024-04-12T20:56:08.583 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:56:08.583+0000 7fa9ed200700 1 Processor -- start 2024-04-12T20:56:08.583 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:56:08.584+0000 7fa9ed200700 1 -- start start 2024-04-12T20:56:08.584 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:56:08.584+0000 7fa9ed200700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa9e806e7a0 0x7fa9e8118300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T20:56:08.585 INFO:tasks.workunit.client.0.smithi018.stdout:9/640: dread d3/d6/d9/dc/d5a/d79/d7e/d9b/f34 [0,4194304] 0 2024-04-12T20:56:08.587 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:56:08.585+0000 7fa9ed200700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa9e806f0f0 0x7fa9e8118840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T20:56:08.587 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:56:08.585+0000 7fa9ed200700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9e8118e40 con 0x7fa9e806e7a0 2024-04-12T20:56:08.587 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:56:08.585+0000 7fa9ed200700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9e8118fb0 con 0x7fa9e806f0f0 2024-04-12T20:56:08.587 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:56:08.585+0000 7fa9e5e00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa9e806e7a0 0x7fa9e8118300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T20:56:08.587 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:56:08.585+0000 7fa9e5e00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa9e806e7a0 0x7fa9e8118300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:55586/0 (socket says 172.21.15.18:55586) 2024-04-12T20:56:08.587 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:56:08.585+0000 7fa9e5e00700 1 -- 172.21.15.18:0/2591382772 learned_addr learned my addr 172.21.15.18:0/2591382772 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T20:56:08.587 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:56:08.586+0000 7fa9e5400700 1 --2- 172.21.15.18:0/2591382772 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa9e806f0f0 0x7fa9e8118840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T20:56:08.588 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:56:08.586+0000 7fa9e5e00700 1 -- 172.21.15.18:0/2591382772 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa9e806f0f0 msgr2=0x7fa9e8118840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:56:08.588 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:56:08.586+0000 7fa9e5e00700 1 --2- 172.21.15.18:0/2591382772 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa9e806f0f0 0x7fa9e8118840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:56:08.588 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:56:08.586+0000 7fa9e5e00700 1 -- 172.21.15.18:0/2591382772 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9d8005040 con 0x7fa9e806e7a0 2024-04-12T20:56:08.588 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:56:08.586+0000 7fa9e5400700 1 --2- 172.21.15.18:0/2591382772 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa9e806f0f0 0x7fa9e8118840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-12T20:56:08.588 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:56:08.587+0000 7fa9e5e00700 1 --2- 172.21.15.18:0/2591382772 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa9e806e7a0 0x7fa9e8118300 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fa9d8000c00 tx=0x7fa9d8009d20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T20:56:08.588 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:56:08.588+0000 7fa9de000700 1 -- 172.21.15.18:0/2591382772 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa9d800e070 con 0x7fa9e806e7a0 2024-04-12T20:56:08.588 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:56:08.588+0000 7fa9de000700 1 -- 172.21.15.18:0/2591382772 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa9d8011480 con 0x7fa9e806e7a0 2024-04-12T20:56:08.588 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:56:08.588+0000 7fa9de000700 1 -- 172.21.15.18:0/2591382772 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa9d8022640 con 0x7fa9e806e7a0 2024-04-12T20:56:08.588 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:56:08.588+0000 7fa9ed200700 1 -- 172.21.15.18:0/2591382772 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa9e811d9b0 con 0x7fa9e806e7a0 2024-04-12T20:56:08.588 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:56:08.588+0000 7fa9ed200700 1 -- 172.21.15.18:0/2591382772 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa9e811de90 con 0x7fa9e806e7a0 2024-04-12T20:56:08.589 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:56:08.590+0000 7fa9ed200700 1 -- 172.21.15.18:0/2591382772 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa9e8112110 con 0x7fa9e806e7a0 2024-04-12T20:56:08.589 INFO:tasks.workunit.client.0.smithi018.stdout:3/458: mkdir d2/d4d/d8d 0 2024-04-12T20:56:08.589 INFO:tasks.workunit.client.0.smithi018.stdout:2/380: chown d5/d12/d54/c44 2798 1 2024-04-12T20:56:08.594 INFO:tasks.workunit.client.0.smithi018.stdout:3/459: chown d2/d6/d34/d71/d7b/f7d 7875531 1 2024-04-12T20:56:08.597 INFO:tasks.workunit.client.0.smithi018.stdout:1/374: rename d3/d9/da/dd/d31/d32/c67 to d3/d9/da/dd/d31/d32/d13/d19/d2d/d4e/d58/c87 0 2024-04-12T20:56:08.599 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:56:08.595+0000 7fa9de000700 1 -- 172.21.15.18:0/2591382772 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 24) v1 ==== 89644+0+0 (secure 0 0 0) 0x7fa9d8011ad0 con 0x7fa9e806e7a0 2024-04-12T20:56:08.599 INFO:tasks.workunit.client.0.smithi018.stdout:9/641: truncate d3/d6/d9/d3f/f6e 1761467 0 2024-04-12T20:56:08.599 INFO:tasks.workunit.client.0.smithi018.stdout:2/381: getdents d5/d8/d33 0 2024-04-12T20:56:08.600 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:56:08.601+0000 7fa9de000700 1 --2- 172.21.15.18:0/2591382772 >> [v2:172.21.15.186:6828/2316349902,v1:172.21.15.186:6829/2316349902] conn(0x7fa9cc06bd00 0x7fa9cc06e1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T20:56:08.600 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:56:08.601+0000 7fa9de000700 1 -- 172.21.15.18:0/2591382772 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5808+0+0 (secure 0 0 0) 0x7fa9d808bf70 con 0x7fa9e806e7a0 2024-04-12T20:56:08.600 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:56:08.602+0000 7fa9e5400700 1 --2- 172.21.15.18:0/2591382772 >> [v2:172.21.15.186:6828/2316349902,v1:172.21.15.186:6829/2316349902] conn(0x7fa9cc06bd00 0x7fa9cc06e1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T20:56:08.604 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:56:08.605+0000 7fa9e5400700 1 --2- 172.21.15.18:0/2591382772 >> [v2:172.21.15.186:6828/2316349902,v1:172.21.15.186:6829/2316349902] conn(0x7fa9cc06bd00 0x7fa9cc06e1b0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fa9d4004710 tx=0x7fa9d400c040 comp rx=0 tx=0).ready entity=mgr.24359 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T20:56:08.605 INFO:tasks.workunit.client.0.smithi018.stdout:2/382: symlink d5/d8/d10/d43/l8d 0 2024-04-12T20:56:08.609 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:56:08.610+0000 7fa9de000700 1 -- 172.21.15.18:0/2591382772 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186454 (secure 0 0 0) 0x7fa9d8059120 con 0x7fa9e806e7a0 2024-04-12T20:56:08.609 INFO:tasks.workunit.client.0.smithi018.stdout:9/642: creat d3/d6/d9/dc/d5a/d79/d7e/fbe x:0 0 0 2024-04-12T20:56:08.611 INFO:tasks.workunit.client.0.smithi018.stdout:2/383: write d5/d12/f58 [5216195,116254] 0 2024-04-12T20:56:08.612 INFO:tasks.workunit.client.0.smithi018.stdout:8/603: dread fa [0,4194304] 0 2024-04-12T20:56:08.612 INFO:tasks.workunit.client.0.smithi018.stdout:1/375: fsync d3/d9/da/f5f 0 2024-04-12T20:56:08.620 INFO:tasks.workunit.client.0.smithi018.stdout:5/489: dwrite d0/d2/d33/d46/d18/f3b [0,4194304] 0 2024-04-12T20:56:08.621 INFO:tasks.workunit.client.0.smithi018.stdout:2/384: dread d5/d7/f4d [0,4194304] 0 2024-04-12T20:56:08.627 INFO:tasks.workunit.client.0.smithi018.stdout:1/376: write d3/d9/da/dd/d31/d32/d13/f46 [4226636,73910] 0 2024-04-12T20:56:08.627 INFO:tasks.workunit.client.0.smithi018.stdout:9/643: mknod d3/d6/d3d/dba/cbf 0 2024-04-12T20:56:08.629 INFO:tasks.workunit.client.0.smithi018.stdout:3/460: dwrite d2/d43/d61/f63 [0,4194304] 0 2024-04-12T20:56:08.636 INFO:tasks.workunit.client.0.smithi018.stdout:5/490: creat d0/d5b/d84/fa5 x:0 0 0 2024-04-12T20:56:08.637 INFO:tasks.workunit.client.0.smithi018.stdout:2/385: link d5/c22 d5/d12/d54/c8e 0 2024-04-12T20:56:08.638 INFO:tasks.workunit.client.0.smithi018.stdout:3/461: write d2/d6/f25 [3587809,76994] 0 2024-04-12T20:56:08.641 INFO:tasks.workunit.client.0.smithi018.stdout:2/386: readlink d5/d8/d10/d43/l8d 0 2024-04-12T20:56:08.644 INFO:tasks.workunit.client.0.smithi018.stdout:1/377: mknod d3/d9/da/dd/d26/c88 0 2024-04-12T20:56:08.646 INFO:tasks.workunit.client.0.smithi018.stdout:8/604: link db/d4e/d56/d59/d76/da5/d45/l62 db/d4e/d7f/dba/lbc 0 2024-04-12T20:56:08.653 INFO:tasks.workunit.client.0.smithi018.stdout:5/491: creat d0/d2/fa6 x:0 0 0 2024-04-12T20:56:08.657 INFO:tasks.workunit.client.0.smithi018.stdout:8/605: rename db/d4e/d56/d59/d76/l7a to db/d3b/lbd 0 2024-04-12T20:56:08.660 INFO:tasks.workunit.client.0.smithi018.stdout:2/387: truncate d5/d12/d54/f52 2628916 0 2024-04-12T20:56:08.664 INFO:tasks.workunit.client.0.smithi018.stdout:8/606: creat db/d4e/d56/d59/d76/fbe x:0 0 0 2024-04-12T20:56:08.666 INFO:tasks.workunit.client.0.smithi018.stdout:2/388: mkdir d5/d8/d10/d43/d8f 0 2024-04-12T20:56:08.669 INFO:tasks.workunit.client.0.smithi018.stdout:9/644: dread d3/d6/d9/d1e/f53 [0,4194304] 0 2024-04-12T20:56:08.669 INFO:tasks.workunit.client.0.smithi018.stdout:2/389: stat d5/d8/d10/d19/d24/d39/l49 0 2024-04-12T20:56:08.670 INFO:tasks.workunit.client.0.smithi018.stdout:5/492: dread d0/d2/d33/d10/d60/f73 [0,4194304] 0 2024-04-12T20:56:08.675 INFO:tasks.workunit.client.0.smithi018.stdout:9/645: read d3/d6/d9/dc/d13/d5b/d7c/f69 [1473666,92102] 0 2024-04-12T20:56:08.676 INFO:tasks.workunit.client.0.smithi018.stdout:9/646: chown d3/d6 294 1 2024-04-12T20:56:08.677 INFO:tasks.workunit.client.0.smithi018.stdout:8/607: mknod db/d4e/d9a/daa/cbf 0 2024-04-12T20:56:08.680 INFO:tasks.workunit.client.0.smithi018.stdout:8/608: write f6 [6656534,90990] 0 2024-04-12T20:56:08.682 INFO:tasks.workunit.client.0.smithi018.stdout:5/493: creat d0/fa7 x:0 0 0 2024-04-12T20:56:08.688 INFO:tasks.workunit.client.0.smithi018.stdout:1/378: truncate d3/d9/da/dd/d31/d32/f1b 559843 0 2024-04-12T20:56:08.693 INFO:tasks.workunit.client.0.smithi018.stdout:5/494: link d0/d2/d33/d46/d1e/c90 d0/d2/d33/d46/d1e/d9a/ca8 0 2024-04-12T20:56:08.695 INFO:tasks.workunit.client.0.smithi018.stdout:2/390: dread d5/d8/d10/d19/d24/f32 [0,4194304] 0 2024-04-12T20:56:08.695 INFO:tasks.workunit.client.0.smithi018.stdout:8/609: getdents db/d4e/d7f/dba 0 2024-04-12T20:56:08.698 INFO:tasks.workunit.client.0.smithi018.stdout:1/379: rename d3/d9/da/dd/d31/d32/l7c to d3/d9/da/dd/d31/d32/d33/l89 0 2024-04-12T20:56:08.699 INFO:tasks.workunit.client.0.smithi018.stdout:8/610: stat db/d4e/d56/d59/d76/da5/d45/da9 0 2024-04-12T20:56:08.699 INFO:tasks.workunit.client.0.smithi018.stdout:2/391: stat d5/d8/d25/l2c 0 2024-04-12T20:56:08.700 INFO:tasks.workunit.client.0.smithi018.stdout:5/495: symlink d0/d2/d33/d10/d26/la9 0 2024-04-12T20:56:08.704 INFO:tasks.workunit.client.0.smithi018.stdout:1/380: write d3/d9/d11/f70 [58521,83767] 0 2024-04-12T20:56:08.705 INFO:tasks.workunit.client.0.smithi018.stdout:1/381: stat d3/f4 0 2024-04-12T20:56:08.706 INFO:tasks.workunit.client.0.smithi018.stdout:8/611: mkdir db/d95/dc0 0 2024-04-12T20:56:08.711 INFO:tasks.workunit.client.0.smithi018.stdout:9/647: write d3/d6/d9/d1e/f45 [2169849,33924] 0 2024-04-12T20:56:08.711 INFO:tasks.workunit.client.0.smithi018.stdout:8/612: read db/d25/f34 [7984133,20302] 0 2024-04-12T20:56:08.713 INFO:tasks.workunit.client.0.smithi018.stdout:5/496: creat d0/d2/d33/d46/d1e/faa x:0 0 0 2024-04-12T20:56:08.714 INFO:tasks.workunit.client.0.smithi018.stdout:2/392: getdents d5/d8/d10/d19/d24/d39 0 2024-04-12T20:56:08.724 INFO:tasks.workunit.client.0.smithi018.stdout:5/497: mknod d0/d2/d33/cab 0 2024-04-12T20:56:08.725 INFO:tasks.workunit.client.0.smithi018.stdout:2/393: mkdir d5/d8/d10/d19/d24/d6c/d90 0 2024-04-12T20:56:08.729 INFO:tasks.workunit.client.0.smithi018.stdout:8/613: dread db/d3b/f55 [0,4194304] 0 2024-04-12T20:56:08.731 INFO:tasks.workunit.client.0.smithi018.stdout:1/382: dwrite d3/d9/da/dd/d31/d32/d13/d19/d2d/d4e/f4f [0,4194304] 0 2024-04-12T20:56:08.738 INFO:tasks.workunit.client.0.smithi018.stdout:8/614: creat db/d4e/d9a/d69/d6c/fc1 x:0 0 0 2024-04-12T20:56:08.741 INFO:tasks.workunit.client.0.smithi018.stdout:1/383: fsync d3/d9/da/dd/d31/d32/d13/d19/d2d/f4d 0 2024-04-12T20:56:08.742 INFO:tasks.workunit.client.0.smithi018.stdout:8/615: write db/d4e/d7f/f93 [1418908,120000] 0 2024-04-12T20:56:08.755 INFO:tasks.workunit.client.0.smithi018.stdout:2/394: truncate d5/d8/d10/d19/d24/d39/f51 260510 0 2024-04-12T20:56:08.756 INFO:tasks.workunit.client.0.smithi018.stdout:8/616: mknod db/d3b/cc2 0 2024-04-12T20:56:08.758 INFO:tasks.workunit.client.0.smithi018.stdout:9/648: dread d3/d6/d9/d1e/f71 [0,4194304] 0 2024-04-12T20:56:08.763 INFO:tasks.workunit.client.0.smithi018.stdout:4/490: sync 2024-04-12T20:56:08.763 INFO:tasks.workunit.client.0.smithi018.stdout:0/388: sync 2024-04-12T20:56:08.763 INFO:tasks.workunit.client.0.smithi018.stdout:1/384: mkdir d3/d9/da/dd/d31/d32/d13/d19/d2d/d4e/d58/d61/d8a 0 2024-04-12T20:56:08.763 INFO:tasks.workunit.client.0.smithi018.stdout:7/425: sync 2024-04-12T20:56:08.765 INFO:tasks.workunit.client.0.smithi018.stdout:7/426: stat d1/d6/c46 0 2024-04-12T20:56:08.766 INFO:tasks.workunit.client.0.smithi018.stdout:8/617: rename db/d25/d26/d27/l6f to db/d4e/d9a/d69/d6c/d7b/lc3 0 2024-04-12T20:56:08.770 INFO:tasks.workunit.client.0.smithi018.stdout:2/395: truncate d5/d7/f5c 31995 0 2024-04-12T20:56:08.774 INFO:tasks.workunit.client.0.smithi018.stdout:8/618: write db/d4e/d9a/d71/f94 [176841,39622] 0 2024-04-12T20:56:08.777 INFO:tasks.workunit.client.0.smithi018.stdout:9/649: getdents d3/d6/d1b 0 2024-04-12T20:56:08.781 INFO:tasks.workunit.client.0.smithi018.stdout:8/619: read f7 [77338,53872] 0 2024-04-12T20:56:08.782 INFO:tasks.workunit.client.0.smithi018.stdout:2/396: mkdir d5/d8/d10/d19/d24/d91 0 2024-04-12T20:56:08.785 INFO:tasks.workunit.client.0.smithi018.stdout:2/397: readlink d5/d12/l1c 0 2024-04-12T20:56:08.792 INFO:tasks.workunit.client.0.smithi018.stdout:1/385: dwrite d3/d9/da/f6f [0,4194304] 0 2024-04-12T20:56:08.801 INFO:tasks.workunit.client.0.smithi018.stdout:1/386: creat d3/d9/da/dd/d31/d32/d13/d19/d2d/f8b x:0 0 0 2024-04-12T20:56:08.804 INFO:tasks.workunit.client.0.smithi018.stdout:8/620: rmdir db/d4e/d73/db2 0 2024-04-12T20:56:08.811 INFO:tasks.workunit.client.0.smithi018.stdout:8/621: readlink db/d25/l39 0 2024-04-12T20:56:08.816 INFO:tasks.workunit.client.0.smithi018.stdout:9/650: dwrite d3/d6/d9/dc/d13/fa9 [0,4194304] 0 2024-04-12T20:56:08.817 INFO:tasks.workunit.client.0.smithi018.stdout:7/427: dread d1/f1b [0,4194304] 0 2024-04-12T20:56:08.818 INFO:tasks.workunit.client.0.smithi018.stdout:2/398: dwrite d5/d12/f89 [0,4194304] 0 2024-04-12T20:56:08.828 INFO:tasks.workunit.client.0.smithi018.stdout:0/389: dwrite da/f38 [0,4194304] 0 2024-04-12T20:56:08.828 INFO:tasks.workunit.client.0.smithi018.stdout:8/622: symlink db/d4e/d7f/dba/lc4 0 2024-04-12T20:56:08.832 INFO:tasks.workunit.client.0.smithi018.stdout:7/428: truncate d1/db/d31/d3a/d4f/f73 521222 0 2024-04-12T20:56:08.832 INFO:tasks.workunit.client.0.smithi018.stdout:2/399: write d5/d7/d31/f64 [4603419,71452] 0 2024-04-12T20:56:08.837 INFO:tasks.workunit.client.0.smithi018.stdout:1/387: link d3/d9/da/dd/d31/f35 d3/d9/da/dd/d31/d32/d13/d19/d2d/d4e/f8c 0 2024-04-12T20:56:08.838 INFO:tasks.workunit.client.0.smithi018.stdout:9/651: stat d3/d6/d9/dc/d5a/d79/d7e/d9b/c38 0 2024-04-12T20:56:08.841 INFO:tasks.workunit.client.0.smithi018.stdout:8/623: creat db/d4e/d56/d59/fc5 x:0 0 0 2024-04-12T20:56:08.846 INFO:tasks.workunit.client.0.smithi018.stdout:7/429: chown d1/db/d31/l36 86477 1 2024-04-12T20:56:08.849 INFO:tasks.workunit.client.0.smithi018.stdout:0/390: link da/d15/d3e/d3f/f6c da/d15/d4d/f85 0 2024-04-12T20:56:08.850 INFO:tasks.workunit.client.0.smithi018.stdout:8/624: chown db/d4e/d56/d59/d76/da5/d45/d86/da1/fa4 84981 1 2024-04-12T20:56:08.855 INFO:tasks.workunit.client.0.smithi018.stdout:2/400: mkdir d5/d8/d10/d19/d7c/d92 0 2024-04-12T20:56:08.858 INFO:tasks.workunit.client.0.smithi018.stdout:9/652: write d3/d6/d9/dc/f97 [421878,128694] 0 2024-04-12T20:56:08.861 INFO:tasks.workunit.client.0.smithi018.stdout:4/491: dread d3/d34/d43/d4b/f5a [0,4194304] 0 2024-04-12T20:56:08.863 INFO:tasks.workunit.client.0.smithi018.stdout:7/430: read d1/db/d31/d3a/d42/d4c/f5b [2284844,9331] 0 2024-04-12T20:56:08.864 INFO:tasks.workunit.client.0.smithi018.stdout:0/391: truncate da/d15/d3e/f73 297697 0 2024-04-12T20:56:08.865 INFO:tasks.workunit.client.0.smithi018.stdout:8/625: creat db/d4e/d56/d59/d76/da5/fc6 x:0 0 0 2024-04-12T20:56:08.869 INFO:tasks.workunit.client.0.smithi018.stdout:8/626: readlink db/d25/l39 0 2024-04-12T20:56:08.871 INFO:tasks.workunit.client.0.smithi018.stdout:9/653: mknod d3/d6/d9/dc/d13/db9/cc0 0 2024-04-12T20:56:08.878 INFO:tasks.workunit.client.0.smithi018.stdout:2/401: dread d5/d8/d10/d43/f56 [0,4194304] 0 2024-04-12T20:56:08.879 INFO:tasks.workunit.client.0.smithi018.stdout:4/492: rename d3/d34/d43/d4b/f5a to d3/db/f9e 0 2024-04-12T20:56:08.881 INFO:tasks.workunit.client.0.smithi018.stdout:1/388: dwrite d3/d9/da/dd/d31/d32/d13/d19/d2d/f76 [0,4194304] 0 2024-04-12T20:56:08.884 INFO:tasks.workunit.client.0.smithi018.stdout:9/654: fdatasync d3/d6/d9/dc/d5a/d79/d7e/d9b/d21/d41/d78/f98 0 2024-04-12T20:56:08.886 INFO:tasks.workunit.client.0.smithi018.stdout:7/431: link d1/db/f35 d1/db/d31/d3a/d42/f8f 0 2024-04-12T20:56:08.889 INFO:tasks.workunit.client.0.smithi018.stdout:8/627: rename db/d4e/d56/d59/fc5 to db/d4e/d56/fc7 0 2024-04-12T20:56:08.893 INFO:tasks.workunit.client.0.smithi018.stdout:9/655: write d3/d6/d9/d1e/f53 [4468244,121302] 0 2024-04-12T20:56:08.894 INFO:tasks.workunit.client.0.smithi018.stdout:8/628: write db/d25/d26/f2f [790683,102272] 0 2024-04-12T20:56:08.895 INFO:tasks.workunit.client.0.smithi018.stdout:2/402: unlink d5/d7/f37 0 2024-04-12T20:56:08.906 INFO:tasks.workunit.client.0.smithi018.stdout:8/629: creat db/d4e/d9a/fc8 x:0 0 0 2024-04-12T20:56:08.911 INFO:tasks.workunit.client.0.smithi018.stdout:7/432: mknod d1/db/d6f/d8a/c90 0 2024-04-12T20:56:08.911 INFO:tasks.workunit.client.0.smithi018.stdout:9/656: truncate d3/d6/d9/d3f/f81 1164485 0 2024-04-12T20:56:08.913 INFO:tasks.workunit.client.0.smithi018.stdout:2/403: creat d5/d12/d54/d3e/f93 x:0 0 0 2024-04-12T20:56:08.915 INFO:tasks.workunit.client.0.smithi018.stdout:9/657: readlink d3/d6/d9/d1e/l2b 0 2024-04-12T20:56:08.916 INFO:tasks.workunit.client.0.smithi018.stdout:2/404: write d5/d7/d31/d62/f72 [449057,47535] 0 2024-04-12T20:56:08.919 INFO:tasks.workunit.client.0.smithi018.stdout:8/630: getdents db/d4e/d56/d59/d76/da5/d45/d86/dbb 0 2024-04-12T20:56:08.921 INFO:tasks.workunit.client.0.smithi018.stdout:7/433: symlink d1/db/d31/d3a/d42/d4c/d6c/l91 0 2024-04-12T20:56:08.922 INFO:tasks.workunit.client.0.smithi018.stdout:8/631: chown db/d25/f52 1 1 2024-04-12T20:56:08.929 INFO:tasks.workunit.client.0.smithi018.stdout:4/493: write d3/db/f24 [1983504,100110] 0 2024-04-12T20:56:08.936 INFO:tasks.workunit.client.0.smithi018.stdout:4/494: mknod d3/d34/d43/d4b/c9f 0 2024-04-12T20:56:08.939 INFO:tasks.workunit.client.0.smithi018.stdout:7/434: write d1/db/d31/f58 [1074378,19018] 0 2024-04-12T20:56:08.943 INFO:tasks.workunit.client.0.smithi018.stdout:4/495: creat d3/d5d/d60/d64/d72/fa0 x:0 0 0 2024-04-12T20:56:08.944 INFO:tasks.workunit.client.0.smithi018.stdout:7/435: readlink d1/l3f 0 2024-04-12T20:56:08.952 INFO:tasks.workunit.client.0.smithi018.stdout:9/658: dwrite d3/d6/d9/dc/d5a/d79/d7e/d9b/f62 [0,4194304] 0 2024-04-12T20:56:08.954 INFO:tasks.workunit.client.0.smithi018.stdout:7/436: link d1/db/d31/d3a/d79/ca d1/db/d31/d30/c92 0 2024-04-12T20:56:08.959 INFO:tasks.workunit.client.0.smithi018.stdout:2/405: dread d5/d8/d10/d19/d24/d39/f77 [0,4194304] 0 2024-04-12T20:56:08.964 INFO:tasks.workunit.client.0.smithi018.stdout:2/406: fsync d5/d12/d54/d3e/f8b 0 2024-04-12T20:56:08.966 INFO:tasks.workunit.client.0.smithi018.stdout:4/496: dwrite d3/d32/fd [0,4194304] 0 2024-04-12T20:56:08.969 INFO:tasks.workunit.client.0.smithi018.stdout:9/659: truncate d3/d6/d9/d3f/f44 4969952 0 2024-04-12T20:56:08.973 INFO:tasks.workunit.client.0.smithi018.stdout:2/407: mkdir d5/d8/d10/d94 0 2024-04-12T20:56:08.985 INFO:tasks.workunit.client.0.smithi018.stdout:7/437: dwrite d1/f8e [0,4194304] 0 2024-04-12T20:56:08.990 INFO:tasks.workunit.client.0.smithi018.stdout:2/408: dread d5/d7/d31/d62/f72 [0,4194304] 0 2024-04-12T20:56:08.992 INFO:tasks.workunit.client.0.smithi018.stdout:9/660: creat d3/d6/d9/dc/d5a/d79/d7e/d9b/d21/d41/d78/fc1 x:0 0 0 2024-04-12T20:56:08.992 INFO:tasks.workunit.client.0.smithi018.stdout:7/438: write d1/db/d31/d3a/d42/d4c/f5b [1965254,57024] 0 2024-04-12T20:56:09.000 INFO:tasks.workunit.client.0.smithi018.stdout:7/439: symlink d1/db/d6f/d8a/l93 0 2024-04-12T20:56:09.002 INFO:tasks.workunit.client.0.smithi018.stdout:2/409: creat d5/f95 x:0 0 0 2024-04-12T20:56:09.004 INFO:tasks.workunit.client.0.smithi018.stdout:7/440: write d1/db/d31/d3a/f85 [765507,9688] 0 2024-04-12T20:56:09.004 INFO:tasks.workunit.client.0.smithi018.stdout:9/661: mkdir d3/d6/d9/dc/d5a/d79/d7e/d9b/d21/dc2 0 2024-04-12T20:56:09.005 INFO:tasks.workunit.client.0.smithi018.stdout:2/410: stat d5/d8/d10/d43/l8d 0 2024-04-12T20:56:09.009 INFO:tasks.workunit.client.0.smithi018.stdout:4/497: dread d3/d32/d33/f18 [0,4194304] 0 2024-04-12T20:56:09.010 INFO:tasks.workunit.client.0.smithi018.stdout:7/441: mkdir d1/db/d31/d3a/d4f/d94 0 2024-04-12T20:56:09.016 INFO:tasks.workunit.client.0.smithi018.stdout:9/662: mkdir d3/d6/d9/dc/d5a/d79/d7e/d9b/d21/dc3 0 2024-04-12T20:56:09.024 INFO:tasks.workunit.client.0.smithi018.stdout:9/663: creat d3/d6/d9/dc/d13/fc4 x:0 0 0 2024-04-12T20:56:09.029 INFO:tasks.workunit.client.0.smithi018.stdout:9/664: write d3/d6/d9/dc/d6f/f9a [4276246,16315] 0 2024-04-12T20:56:09.034 INFO:tasks.workunit.client.0.smithi018.stdout:4/498: dwrite d3/db/f26 [0,4194304] 0 2024-04-12T20:56:09.039 INFO:tasks.workunit.client.0.smithi018.stdout:2/411: dwrite d5/d8/d10/d19/d24/d39/f77 [0,4194304] 0 2024-04-12T20:56:09.039 INFO:tasks.workunit.client.0.smithi018.stdout:7/442: dwrite d1/db/f43 [0,4194304] 0 2024-04-12T20:56:09.045 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:56:09.046+0000 7fa9ed200700 1 -- 172.21.15.18:0/2591382772 --> [v2:172.21.15.186:6828/2316349902,v1:172.21.15.186:6829/2316349902] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fa9e802b3b0 con 0x7fa9cc06bd00 2024-04-12T20:56:09.045 INFO:tasks.workunit.client.0.smithi018.stdout:4/499: creat d3/d32/d33/d8e/fa1 x:0 0 0 2024-04-12T20:56:09.046 INFO:tasks.workunit.client.0.smithi018.stdout:7/443: creat d1/db/d31/d3a/d4f/d70/f95 x:0 0 0 2024-04-12T20:56:09.048 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:56:09.049+0000 7fa9de000700 1 -- 172.21.15.18:0/2591382772 <== mgr.24359 v2:172.21.15.186:6828/2316349902 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+310 (secure 0 0 0) 0x7fa9e802b3b0 con 0x7fa9cc06bd00 2024-04-12T20:56:09.051 INFO:tasks.workunit.client.0.smithi018.stdout:7/444: write d1/db/d31/d3a/d4f/f73 [169836,68062] 0 2024-04-12T20:56:09.052 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:56:09.054+0000 7fa9ed200700 1 -- 172.21.15.18:0/2591382772 >> [v2:172.21.15.186:6828/2316349902,v1:172.21.15.186:6829/2316349902] conn(0x7fa9cc06bd00 msgr2=0x7fa9cc06e1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:56:09.052 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:56:09.054+0000 7fa9ed200700 1 --2- 172.21.15.18:0/2591382772 >> [v2:172.21.15.186:6828/2316349902,v1:172.21.15.186:6829/2316349902] conn(0x7fa9cc06bd00 0x7fa9cc06e1b0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fa9d4004710 tx=0x7fa9d400c040 comp rx=0 tx=0).stop 2024-04-12T20:56:09.053 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:56:09.054+0000 7fa9ed200700 1 -- 172.21.15.18:0/2591382772 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa9e806e7a0 msgr2=0x7fa9e8118300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:56:09.053 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:56:09.054+0000 7fa9ed200700 1 --2- 172.21.15.18:0/2591382772 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa9e806e7a0 0x7fa9e8118300 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fa9d8000c00 tx=0x7fa9d8009d20 comp rx=0 tx=0).stop 2024-04-12T20:56:09.053 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:56:09.055+0000 7fa9ed200700 1 -- 172.21.15.18:0/2591382772 shutdown_connections 2024-04-12T20:56:09.053 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:56:09.055+0000 7fa9ed200700 1 --2- 172.21.15.18:0/2591382772 >> [v2:172.21.15.186:6828/2316349902,v1:172.21.15.186:6829/2316349902] conn(0x7fa9cc06bd00 0x7fa9cc06e1b0 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:56:09.053 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:56:09.055+0000 7fa9ed200700 1 --2- 172.21.15.18:0/2591382772 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa9e806e7a0 0x7fa9e8118300 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:56:09.053 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:56:09.055+0000 7fa9ed200700 1 --2- 172.21.15.18:0/2591382772 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa9e806f0f0 0x7fa9e8118840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:56:09.053 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:56:09.055+0000 7fa9ed200700 1 -- 172.21.15.18:0/2591382772 >> 172.21.15.18:0/2591382772 conn(0x7fa9e806a380 msgr2=0x7fa9e810f290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T20:56:09.054 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:56:09.055+0000 7fa9ed200700 1 -- 172.21.15.18:0/2591382772 shutdown_connections 2024-04-12T20:56:09.054 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:56:09.055+0000 7fa9ed200700 1 -- 172.21.15.18:0/2591382772 wait complete. 2024-04-12T20:56:09.056 INFO:tasks.workunit.client.0.smithi018.stdout:4/500: truncate d3/d5d/d60/d64/d72/f89 478248 0 2024-04-12T20:56:09.066 INFO:tasks.workunit.client.0.smithi018.stdout:2/412: dwrite d5/d12/f58 [0,4194304] 0 2024-04-12T20:56:09.070 INFO:tasks.workunit.client.0.smithi018.stdout:7/445: rename d1/db/d31/d3a/d79 to d1/db/d31/d3a/d4f/d70/d96 0 2024-04-12T20:56:09.071 INFO:tasks.workunit.client.0.smithi018.stdout:2/413: chown d5/l40 10 1 2024-04-12T20:56:09.071 INFO:tasks.workunit.client.0.smithi018.stdout:4/501: rmdir d3 39 2024-04-12T20:56:09.084 INFO:teuthology.orchestra.run.smithi018.stdout:true 2024-04-12T20:56:09.086 INFO:tasks.workunit.client.0.smithi018.stdout:2/414: truncate d5/d7/f4d 268062 0 2024-04-12T20:56:09.087 INFO:tasks.workunit.client.0.smithi018.stdout:4/502: dread d3/d5d/f6e [0,4194304] 0 2024-04-12T20:56:09.092 INFO:tasks.workunit.client.0.smithi018.stdout:4/503: write d3/d34/d43/d4b/f83 [215056,38176] 0 2024-04-12T20:56:09.096 INFO:tasks.workunit.client.0.smithi018.stdout:4/504: write d3/d34/d3d/d80/f8c [3627629,96712] 0 2024-04-12T20:56:09.097 INFO:tasks.workunit.client.0.smithi018.stdout:2/415: creat d5/d12/f96 x:0 0 0 2024-04-12T20:56:09.099 INFO:tasks.workunit.client.0.smithi018.stdout:7/446: dwrite d1/f2f [0,4194304] 0 2024-04-12T20:56:09.099 INFO:tasks.workunit.client.0.smithi018.stdout:4/505: fdatasync d3/db/f26 0 2024-04-12T20:56:09.102 INFO:tasks.workunit.client.0.smithi018.stdout:2/416: write d5/d7/d79/f7e [121236,122698] 0 2024-04-12T20:56:09.106 INFO:tasks.workunit.client.0.smithi018.stdout:4/506: chown d3/d34/d3d/l58 1587513 1 2024-04-12T20:56:09.108 INFO:tasks.workunit.client.0.smithi018.stdout:7/447: mkdir d1/db/d31/d3a/d97 0 2024-04-12T20:56:09.109 INFO:tasks.workunit.client.0.smithi018.stdout:2/417: readlink d5/d8/d10/d19/d24/d6c/l80 0 2024-04-12T20:56:09.112 INFO:tasks.workunit.client.0.smithi018.stdout:4/507: fsync d3/db/f26 0 2024-04-12T20:56:09.114 INFO:tasks.workunit.client.0.smithi018.stdout:4/508: fsync d3/f6b 0 2024-04-12T20:56:09.118 INFO:tasks.workunit.client.0.smithi018.stdout:7/448: creat d1/d6/f98 x:0 0 0 2024-04-12T20:56:09.119 INFO:tasks.workunit.client.0.smithi018.stdout:4/509: rmdir d3/d5d/d60 39 2024-04-12T20:56:09.122 INFO:tasks.workunit.client.0.smithi018.stdout:7/449: chown d1/db/d31/d3a/d4f/d69/f7f 15323813 1 2024-04-12T20:56:09.127 INFO:tasks.workunit.client.0.smithi018.stdout:7/450: write d1/db/d31/d3a/d4f/d69/f89 [797953,93825] 0 2024-04-12T20:56:09.135 INFO:tasks.workunit.client.0.smithi018.stdout:2/418: dwrite d5/d12/d54/d3e/f93 [0,4194304] 0 2024-04-12T20:56:09.137 INFO:tasks.workunit.client.0.smithi018.stdout:4/510: truncate d3/d32/d94/f97 431193 0 2024-04-12T20:56:09.139 INFO:tasks.workunit.client.0.smithi018.stdout:4/511: readlink d3/d34/d40/l9d 0 2024-04-12T20:56:09.152 INFO:tasks.workunit.client.0.smithi018.stdout:4/512: rename d3/d34/d43/f2f to d3/d34/d3d/d80/fa2 0 2024-04-12T20:56:09.170 INFO:tasks.workunit.client.0.smithi018.stdout:7/451: dread d1/db/d31/d3a/d42/d4c/f51 [0,4194304] 0 2024-04-12T20:56:09.174 INFO:tasks.workunit.client.0.smithi018.stdout:2/419: dread d5/d7/d28/f59 [0,4194304] 0 2024-04-12T20:56:09.176 INFO:tasks.workunit.client.0.smithi018.stdout:2/420: fdatasync d5/d8/f57 0 2024-04-12T20:56:09.177 INFO:tasks.workunit.client.0.smithi018.stdout:7/452: unlink d1/db/d31/d3a/d4f/d70/d96/l19 0 2024-04-12T20:56:09.177 INFO:tasks.workunit.client.0.smithi018.stdout:4/513: dwrite d3/d32/f81 [0,4194304] 0 2024-04-12T20:56:09.178 INFO:tasks.workunit.client.0.smithi018.stdout:2/421: fdatasync d5/d8/d10/d19/d24/f36 0 2024-04-12T20:56:09.179 INFO:tasks.workunit.client.0.smithi018.stdout:7/453: chown d1/db/d31/d30/f57 8 1 2024-04-12T20:56:09.181 INFO:tasks.workunit.client.0.smithi018.stdout:4/514: rmdir d3/db 39 2024-04-12T20:56:09.182 INFO:tasks.workunit.client.0.smithi018.stdout:2/422: dread d5/d8/d10/d19/d24/d39/f45 [0,4194304] 0 2024-04-12T20:56:09.185 INFO:tasks.workunit.client.0.smithi018.stdout:4/515: write d3/d34/d43/d4b/f9c [1656444,53656] 0 2024-04-12T20:56:09.186 INFO:tasks.workunit.client.0.smithi018.stdout:4/516: write d3/d32/fd [3353179,22960] 0 2024-04-12T20:56:09.195 INFO:tasks.workunit.client.0.smithi018.stdout:2/423: fdatasync d5/d12/d54/f52 0 2024-04-12T20:56:09.198 INFO:tasks.workunit.client.0.smithi018.stdout:2/424: chown d5/d12/d54/c3d 23194 1 2024-04-12T20:56:09.198 INFO:tasks.workunit.client.0.smithi018.stdout:4/517: dread d3/d32/d33/f17 [0,4194304] 0 2024-04-12T20:56:09.201 INFO:tasks.workunit.client.0.smithi018.stdout:2/425: mknod d5/d7/d31/c97 0 2024-04-12T20:56:09.204 INFO:tasks.workunit.client.0.smithi018.stdout:4/518: mkdir d3/d34/d3d/da3 0 2024-04-12T20:56:09.206 INFO:tasks.workunit.client.0.smithi018.stdout:7/454: truncate d1/db/d31/d3a/f41 3864497 0 2024-04-12T20:56:09.208 INFO:tasks.workunit.client.0.smithi018.stdout:2/426: mknod d5/d12/c98 0 2024-04-12T20:56:09.210 INFO:tasks.workunit.client.0.smithi018.stdout:4/519: creat d3/d34/fa4 x:0 0 0 2024-04-12T20:56:09.210 INFO:tasks.workunit.client.0.smithi018.stdout:7/455: creat d1/db/d31/d3a/d42/d4c/f99 x:0 0 0 2024-04-12T20:56:09.212 INFO:tasks.workunit.client.0.smithi018.stdout:4/520: write d3/f52 [3168601,115949] 0 2024-04-12T20:56:09.213 INFO:tasks.workunit.client.0.smithi018.stdout:2/427: unlink d5/d8/d10/f15 0 2024-04-12T20:56:09.214 INFO:tasks.workunit.client.0.smithi018.stdout:7/456: chown d1/db/d31/d30/c8d 2388 1 2024-04-12T20:56:09.217 INFO:tasks.workunit.client.0.smithi018.stdout:2/428: symlink d5/d7/d31/l99 0 2024-04-12T20:56:09.219 INFO:tasks.workunit.client.0.smithi018.stdout:4/521: rmdir d3/d34/d3d/d90 0 2024-04-12T20:56:09.221 INFO:tasks.workunit.client.0.smithi018.stdout:7/457: truncate d1/d6/f24 3202976 0 2024-04-12T20:56:09.222 INFO:tasks.workunit.client.0.smithi018.stdout:4/522: chown d3/d34/d43/f62 4308799 1 2024-04-12T20:56:09.223 INFO:tasks.workunit.client.0.smithi018.stdout:4/523: readlink d3/d5d/l77 0 2024-04-12T20:56:09.225 INFO:tasks.workunit.client.0.smithi018.stdout:2/429: creat d5/d8/d33/f9a x:0 0 0 2024-04-12T20:56:09.230 INFO:tasks.workunit.client.0.smithi018.stdout:4/524: creat d3/d32/d33/fa5 x:0 0 0 2024-04-12T20:56:09.231 INFO:tasks.workunit.client.0.smithi018.stdout:2/430: truncate d5/d12/d54/f52 1468020 0 2024-04-12T20:56:09.234 INFO:tasks.workunit.client.0.smithi018.stdout:4/525: truncate d3/db/f8f 968175 0 2024-04-12T20:56:09.234 INFO:tasks.workunit.client.0.smithi018.stdout:2/431: write d5/d8/d10/f78 [141131,1791] 0 2024-04-12T20:56:09.238 INFO:tasks.workunit.client.0.smithi018.stdout:4/526: write d3/d34/d40/f82 [85707,60664] 0 2024-04-12T20:56:09.241 INFO:tasks.workunit.client.0.smithi018.stdout:4/527: unlink d3/l41 0 2024-04-12T20:56:09.243 INFO:tasks.workunit.client.0.smithi018.stdout:2/432: mknod d5/d7/d31/d62/c9b 0 2024-04-12T20:56:09.245 INFO:tasks.workunit.client.0.smithi018.stdout:4/528: mknod d3/d5d/ca6 0 2024-04-12T20:56:09.249 INFO:tasks.workunit.client.0.smithi018.stdout:2/433: unlink d5/d12/d54/d3e/f93 0 2024-04-12T20:56:09.249 INFO:tasks.workunit.client.0.smithi018.stdout:4/529: write d3/d32/d66/f93 [944209,75152] 0 2024-04-12T20:56:09.250 INFO:tasks.workunit.client.0.smithi018.stdout:7/458: dwrite d1/db/d31/d3a/d42/f75 [4194304,4194304] 0 2024-04-12T20:56:09.253 INFO:tasks.workunit.client.0.smithi018.stdout:7/459: readlink d1/db/l47 0 2024-04-12T20:56:09.253 INFO:tasks.workunit.client.0.smithi018.stdout:2/434: stat d5/d8/d10/d19/d24/d39/f51 0 2024-04-12T20:56:09.256 INFO:tasks.workunit.client.0.smithi018.stdout:7/460: write d1/db/d31/d3a/d4f/f73 [428934,55735] 0 2024-04-12T20:56:09.261 INFO:tasks.workunit.client.0.smithi018.stdout:4/530: getdents d3/d32/d33/d8e 0 2024-04-12T20:56:09.263 INFO:tasks.workunit.client.0.smithi018.stdout:4/531: write d3/db/f8b [1181186,6526] 0 2024-04-12T20:56:09.265 INFO:tasks.workunit.client.0.smithi018.stdout:7/461: rmdir d1/db 39 2024-04-12T20:56:09.278 INFO:tasks.workunit.client.0.smithi018.stdout:3/462: sync 2024-04-12T20:56:09.279 INFO:tasks.workunit.client.0.smithi018.stdout:6/652: sync 2024-04-12T20:56:09.283 INFO:tasks.workunit.client.0.smithi018.stdout:2/435: truncate d5/d12/f58 725723 0 2024-04-12T20:56:09.287 INFO:tasks.workunit.client.0.smithi018.stdout:2/436: fdatasync d5/d8/d10/d19/d24/f29 0 2024-04-12T20:56:09.296 INFO:tasks.workunit.client.0.smithi018.stdout:2/437: mkdir d5/d8/d10/d19/d24/d39/d9c 0 2024-04-12T20:56:09.299 INFO:tasks.workunit.client.0.smithi018.stdout:4/532: dwrite d3/d34/d43/f7b [0,4194304] 0 2024-04-12T20:56:09.309 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:56:09.310+0000 7fc48b400700 1 -- 172.21.15.18:0/1718637684 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc4841046b0 msgr2=0x7fc484104b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:56:09.309 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:56:09.310+0000 7fc48b400700 1 --2- 172.21.15.18:0/1718637684 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc4841046b0 0x7fc484104b20 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fc46c004940 tx=0x7fc46c009e30 comp rx=0 tx=0).stop 2024-04-12T20:56:09.310 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:56:09 smithi018 ceph-mon[28799]: pgmap v37: 65 pgs: 65 active+clean; 1015 MiB data, 4.5 GiB used, 532 GiB / 536 GiB avail; 27 MiB/s rd, 114 MiB/s wr, 269 op/s 2024-04-12T20:56:09.311 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:56:09.311+0000 7fc48b400700 1 -- 172.21.15.18:0/1718637684 shutdown_connections 2024-04-12T20:56:09.311 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:56:09.311+0000 7fc48b400700 1 --2- 172.21.15.18:0/1718637684 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc4841046b0 0x7fc484104b20 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:56:09.311 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:56:09.311+0000 7fc48b400700 1 --2- 172.21.15.18:0/1718637684 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc484103400 0x7fc484103810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:56:09.311 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:56:09.311+0000 7fc48b400700 1 -- 172.21.15.18:0/1718637684 >> 172.21.15.18:0/1718637684 conn(0x7fc484072590 msgr2=0x7fc4840749c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T20:56:09.311 INFO:tasks.workunit.client.0.smithi018.stdout:2/438: creat d5/d8/d10/d19/d24/d6c/d90/f9d x:0 0 0 2024-04-12T20:56:09.311 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:56:09.313+0000 7fc48b400700 1 -- 172.21.15.18:0/1718637684 shutdown_connections 2024-04-12T20:56:09.311 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:56:09.313+0000 7fc48b400700 1 -- 172.21.15.18:0/1718637684 wait complete. 2024-04-12T20:56:09.312 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:56:09.314+0000 7fc48b400700 1 Processor -- start 2024-04-12T20:56:09.313 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:56:09.314+0000 7fc48b400700 1 -- start start 2024-04-12T20:56:09.313 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:56:09.315+0000 7fc48b400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc484103400 0x7fc484113f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T20:56:09.313 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:56:09.315+0000 7fc48b400700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc4841046b0 0x7fc4841144d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T20:56:09.314 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:56:09.315+0000 7fc48b400700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc484114ad0 con 0x7fc484103400 2024-04-12T20:56:09.314 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:56:09.315+0000 7fc48b400700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc484115320 con 0x7fc4841046b0 2024-04-12T20:56:09.314 INFO:tasks.workunit.client.0.smithi018.stdout:4/533: link d3/d32/fd d3/d32/d33/fa7 0 2024-04-12T20:56:09.314 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:56:09.316+0000 7fc488a00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc484103400 0x7fc484113f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T20:56:09.315 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:56:09.316+0000 7fc483e00700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc4841046b0 0x7fc4841144d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T20:56:09.315 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:56:09.316+0000 7fc483e00700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc4841046b0 0x7fc4841144d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.18:51236/0 (socket says 172.21.15.18:51236) 2024-04-12T20:56:09.315 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:56:09.316+0000 7fc483e00700 1 -- 172.21.15.18:0/1430457006 learned_addr learned my addr 172.21.15.18:0/1430457006 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T20:56:09.315 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:56:09.316+0000 7fc488a00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc484103400 0x7fc484113f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:55596/0 (socket says 172.21.15.18:55596) 2024-04-12T20:56:09.315 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:56:09.317+0000 7fc483e00700 1 -- 172.21.15.18:0/1430457006 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc484103400 msgr2=0x7fc484113f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:56:09.315 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:56:09.317+0000 7fc483e00700 1 --2- 172.21.15.18:0/1430457006 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc484103400 0x7fc484113f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:56:09.315 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:56:09.317+0000 7fc483e00700 1 -- 172.21.15.18:0/1430457006 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc46c005040 con 0x7fc4841046b0 2024-04-12T20:56:09.316 INFO:tasks.workunit.client.0.smithi018.stdout:3/463: dwrite d2/d6/f1c [0,4194304] 0 2024-04-12T20:56:09.316 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:56:09.318+0000 7fc483e00700 1 --2- 172.21.15.18:0/1430457006 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc4841046b0 0x7fc4841144d0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fc46c00b3b0 tx=0x7fc46c009e30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T20:56:09.320 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:56:09.318+0000 7fc480c00700 1 -- 172.21.15.18:0/1430457006 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc46c00e040 con 0x7fc4841046b0 2024-04-12T20:56:09.320 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:56:09.319+0000 7fc480c00700 1 -- 172.21.15.18:0/1430457006 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc46c004290 con 0x7fc4841046b0 2024-04-12T20:56:09.320 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:56:09.319+0000 7fc480c00700 1 -- 172.21.15.18:0/1430457006 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc46c01c3f0 con 0x7fc4841046b0 2024-04-12T20:56:09.320 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:56:09.319+0000 7fc48b400700 1 -- 172.21.15.18:0/1430457006 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc4841155a0 con 0x7fc4841046b0 2024-04-12T20:56:09.320 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:56:09.319+0000 7fc48b400700 1 -- 172.21.15.18:0/1430457006 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc484115ae0 con 0x7fc4841046b0 2024-04-12T20:56:09.320 INFO:tasks.workunit.client.0.smithi018.stdout:6/653: dread d5/d6/d1b/d3b/f52 [0,4194304] 0 2024-04-12T20:56:09.321 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:56:09.322+0000 7fc48b400700 1 -- 172.21.15.18:0/1430457006 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc4600052e0 con 0x7fc4841046b0 2024-04-12T20:56:09.321 INFO:tasks.workunit.client.0.smithi018.stdout:3/464: dread - d2/d6/d30/d31/d38/f44 zero size 2024-04-12T20:56:09.321 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:56:09.323+0000 7fc480c00700 1 -- 172.21.15.18:0/1430457006 <== mon.1 v2:172.21.15.186:3300/0 4 ==== mgrmap(e 24) v1 ==== 89644+0+0 (secure 0 0 0) 0x7fc46c01c550 con 0x7fc4841046b0 2024-04-12T20:56:09.321 INFO:tasks.workunit.client.0.smithi018.stdout:2/439: mknod d5/d8/d10/d19/d24/d39/c9e 0 2024-04-12T20:56:09.323 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:56:09.324+0000 7fc480c00700 1 --2- 172.21.15.18:0/1430457006 >> [v2:172.21.15.186:6828/2316349902,v1:172.21.15.186:6829/2316349902] conn(0x7fc46806bdb0 0x7fc46806e260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T20:56:09.323 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:56:09.324+0000 7fc480c00700 1 -- 172.21.15.18:0/1430457006 <== mon.1 v2:172.21.15.186:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5808+0+0 (secure 0 0 0) 0x7fc46c08bc60 con 0x7fc4841046b0 2024-04-12T20:56:09.323 INFO:tasks.workunit.client.0.smithi018.stdout:7/462: dwrite d1/db/d31/d3a/d4f/d70/d96/f13 [0,4194304] 0 2024-04-12T20:56:09.323 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:56:09.325+0000 7fc488a00700 1 --2- 172.21.15.18:0/1430457006 >> [v2:172.21.15.186:6828/2316349902,v1:172.21.15.186:6829/2316349902] conn(0x7fc46806bdb0 0x7fc46806e260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T20:56:09.324 INFO:tasks.workunit.client.0.smithi018.stdout:2/440: fdatasync d5/d8/d10/f70 0 2024-04-12T20:56:09.325 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:56:09.327+0000 7fc488a00700 1 --2- 172.21.15.18:0/1430457006 >> [v2:172.21.15.186:6828/2316349902,v1:172.21.15.186:6829/2316349902] conn(0x7fc46806bdb0 0x7fc46806e260 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fc47c009fd0 tx=0x7fc47c00b040 comp rx=0 tx=0).ready entity=mgr.24359 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T20:56:09.330 INFO:tasks.workunit.client.0.smithi018.stdout:4/534: rename d3/d34/d40/f44 to d3/fa8 0 2024-04-12T20:56:09.330 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:56:09.332+0000 7fc480c00700 1 -- 172.21.15.18:0/1430457006 <== mon.1 v2:172.21.15.186:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186454 (secure 0 0 0) 0x7fc46c058a00 con 0x7fc4841046b0 2024-04-12T20:56:09.335 INFO:tasks.workunit.client.0.smithi018.stdout:2/441: stat d5/d12/d54/c8e 0 2024-04-12T20:56:09.335 INFO:tasks.workunit.client.0.smithi018.stdout:7/463: dread d1/d6/f67 [0,4194304] 0 2024-04-12T20:56:09.343 INFO:tasks.workunit.client.0.smithi018.stdout:4/535: link d3/db/f24 d3/d34/fa9 0 2024-04-12T20:56:09.345 INFO:tasks.workunit.client.0.smithi018.stdout:4/536: readlink d3/d34/d40/l46 0 2024-04-12T20:56:09.345 INFO:tasks.workunit.client.0.smithi018.stdout:2/442: creat d5/d8/d10/d19/d7c/f9f x:0 0 0 2024-04-12T20:56:09.347 INFO:tasks.workunit.client.0.smithi018.stdout:5/498: sync 2024-04-12T20:56:09.353 INFO:tasks.workunit.client.0.smithi018.stdout:5/499: stat d0/fa7 0 2024-04-12T20:56:09.353 INFO:tasks.workunit.client.0.smithi018.stdout:3/465: truncate d2/d6/d34/f59 1378969 0 2024-04-12T20:56:09.356 INFO:tasks.workunit.client.0.smithi018.stdout:4/537: readlink d3/d32/d66/l98 0 2024-04-12T20:56:09.366 INFO:tasks.workunit.client.0.smithi018.stdout:4/538: truncate d3/d34/d43/d4b/f84 733871 0 2024-04-12T20:56:09.367 INFO:tasks.workunit.client.0.smithi018.stdout:5/500: rename d0/d2/d33/d46/d1e/d55/d66/f91 to d0/d2/d33/d10/d2f/d58/d89/fac 0 2024-04-12T20:56:09.375 INFO:tasks.workunit.client.0.smithi018.stdout:2/443: dread d5/d8/d10/d19/d24/f32 [0,4194304] 0 2024-04-12T20:56:09.375 INFO:tasks.workunit.client.0.smithi018.stdout:5/501: unlink d0/d2/d11/d37/f94 0 2024-04-12T20:56:09.377 INFO:tasks.workunit.client.0.smithi018.stdout:6/654: dread d5/d71/fa3 [0,4194304] 0 2024-04-12T20:56:09.377 INFO:tasks.workunit.client.0.smithi018.stdout:7/464: dread d1/db/d31/d3a/d42/d4c/f51 [0,4194304] 0 2024-04-12T20:56:09.379 INFO:tasks.workunit.client.0.smithi018.stdout:4/539: creat d3/faa x:0 0 0 2024-04-12T20:56:09.385 INFO:tasks.workunit.client.0.smithi018.stdout:3/466: truncate d2/d4d/d88/f8a 4035180 0 2024-04-12T20:56:09.388 INFO:tasks.workunit.client.0.smithi018.stdout:2/444: rename d5/d8/d10/d19/d24/d39/f77 to d5/d12/d54/fa0 0 2024-04-12T20:56:09.394 INFO:tasks.workunit.client.0.smithi018.stdout:4/540: creat d3/d34/d43/fab x:0 0 0 2024-04-12T20:56:09.400 INFO:tasks.workunit.client.0.smithi018.stdout:6/655: dread d5/df/d19/d43/d5f/fb4 [0,4194304] 0 2024-04-12T20:56:09.400 INFO:tasks.workunit.client.0.smithi018.stdout:3/467: chown d2/c1f 3639 1 2024-04-12T20:56:09.405 INFO:tasks.workunit.client.0.smithi018.stdout:7/465: rename d1/db/d31/f4d to d1/db/d31/d45/f9a 0 2024-04-12T20:56:09.408 INFO:tasks.workunit.client.0.smithi018.stdout:3/468: fdatasync d2/d6/d30/d31/d38/d3a/d4c/f79 0 2024-04-12T20:56:09.409 INFO:tasks.workunit.client.0.smithi018.stdout:4/541: creat d3/d32/fac x:0 0 0 2024-04-12T20:56:09.414 INFO:tasks.workunit.client.0.smithi018.stdout:4/542: chown d3/d34/d43/fab 15326192 1 2024-04-12T20:56:09.416 INFO:tasks.workunit.client.0.smithi018.stdout:4/543: chown d3/d34/d43/d4b/l87 2 1 2024-04-12T20:56:09.420 INFO:tasks.workunit.client.0.smithi018.stdout:6/656: creat d5/d6/d1b/d3b/d6a/dd8/ff9 x:0 0 0 2024-04-12T20:56:09.421 INFO:tasks.workunit.client.0.smithi018.stdout:3/469: mknod d2/d6/d3d/d70/c8e 0 2024-04-12T20:56:09.423 INFO:tasks.workunit.client.0.smithi018.stdout:6/657: chown d5/df/d19/d2b/f9c 212 1 2024-04-12T20:56:09.428 INFO:tasks.workunit.client.0.smithi018.stdout:3/470: creat d2/d43/f8f x:0 0 0 2024-04-12T20:56:09.434 INFO:tasks.workunit.client.0.smithi018.stdout:4/544: dread d3/d32/d66/f93 [0,4194304] 0 2024-04-12T20:56:09.435 INFO:tasks.workunit.client.0.smithi018.stdout:5/502: dread d0/d2/d33/d46/d35/f79 [0,4194304] 0 2024-04-12T20:56:09.436 INFO:tasks.workunit.client.0.smithi018.stdout:3/471: symlink d2/d43/d61/d64/l90 0 2024-04-12T20:56:09.440 INFO:tasks.workunit.client.0.smithi018.stdout:4/545: stat d3/d5d/d60/d64/c8a 0 2024-04-12T20:56:09.441 INFO:tasks.workunit.client.0.smithi018.stdout:7/466: dwrite d1/f66 [0,4194304] 0 2024-04-12T20:56:09.442 INFO:tasks.workunit.client.0.smithi018.stdout:4/546: stat d3/d32/l3f 0 2024-04-12T20:56:09.449 INFO:tasks.workunit.client.0.smithi018.stdout:3/472: creat d2/d6/d34/d71/d7b/f91 x:0 0 0 2024-04-12T20:56:09.452 INFO:tasks.workunit.client.0.smithi018.stdout:7/467: symlink d1/db/d31/d3a/d4f/d70/d96/d5f/l9b 0 2024-04-12T20:56:09.457 INFO:tasks.workunit.client.0.smithi018.stdout:5/503: read d0/d2/d33/d46/d35/d4d/f69 [143254,108241] 0 2024-04-12T20:56:09.459 INFO:tasks.workunit.client.0.smithi018.stdout:4/547: rmdir d3/d32/d33/d9b 0 2024-04-12T20:56:09.460 INFO:tasks.workunit.client.0.smithi018.stdout:7/468: truncate d1/db/d31/d45/d53/f7d 149418 0 2024-04-12T20:56:09.461 INFO:tasks.workunit.client.0.smithi018.stdout:5/504: stat d0/d2/d33/d46/d1e/d2e/c24 0 2024-04-12T20:56:09.464 INFO:tasks.workunit.client.0.smithi018.stdout:4/548: truncate d3/d34/d43/d4b/f83 1252566 0 2024-04-12T20:56:09.469 INFO:tasks.workunit.client.0.smithi018.stdout:7/469: rename d1/f2 to d1/db/d31/d45/f9c 0 2024-04-12T20:56:09.476 INFO:tasks.workunit.client.0.smithi018.stdout:4/549: creat d3/db/fad x:0 0 0 2024-04-12T20:56:09.479 INFO:tasks.workunit.client.0.smithi018.stdout:4/550: chown d3/d34/d53/l7e 79691 1 2024-04-12T20:56:09.480 INFO:tasks.workunit.client.0.smithi018.stdout:6/658: dread d5/d6/d1b/d3b/d35/d4f/f5c [0,4194304] 0 2024-04-12T20:56:09.487 INFO:tasks.workunit.client.0.smithi018.stdout:4/551: creat d3/d5d/fae x:0 0 0 2024-04-12T20:56:09.492 INFO:tasks.workunit.client.0.smithi018.stdout:6/659: getdents d5/df/d19/dc5 0 2024-04-12T20:56:09.495 INFO:tasks.workunit.client.0.smithi018.stdout:4/552: mknod d3/d5d/d60/d64/d72/caf 0 2024-04-12T20:56:09.498 INFO:tasks.workunit.client.0.smithi018.stdout:4/553: dread - d3/d34/d3d/f57 zero size 2024-04-12T20:56:09.501 INFO:tasks.workunit.client.0.smithi018.stdout:6/660: truncate d5/df/d31/d4e/fc7 1463753 0 2024-04-12T20:56:09.502 INFO:tasks.workunit.client.0.smithi018.stdout:5/505: dread d0/d2/d33/d10/f27 [0,4194304] 0 2024-04-12T20:56:09.503 INFO:tasks.workunit.client.0.smithi018.stdout:4/554: stat d3/db/c15 0 2024-04-12T20:56:09.506 INFO:tasks.workunit.client.0.smithi018.stdout:4/555: creat d3/d34/fb0 x:0 0 0 2024-04-12T20:56:09.509 INFO:tasks.workunit.client.0.smithi018.stdout:7/470: dread d1/db/f35 [0,4194304] 0 2024-04-12T20:56:09.509 INFO:tasks.workunit.client.0.smithi018.stdout:4/556: rmdir d3/d32/d33 39 2024-04-12T20:56:09.510 INFO:tasks.workunit.client.0.smithi018.stdout:7/471: fdatasync d1/f2f 0 2024-04-12T20:56:09.510 INFO:tasks.workunit.client.0.smithi018.stdout:6/661: getdents d5/d6/d1b/d3b/d8e 0 2024-04-12T20:56:09.512 INFO:tasks.workunit.client.0.smithi018.stdout:6/662: chown d5/d6/l9 1 1 2024-04-12T20:56:09.514 INFO:tasks.workunit.client.0.smithi018.stdout:4/557: truncate d3/faa 343900 0 2024-04-12T20:56:09.514 INFO:tasks.workunit.client.0.smithi018.stdout:7/472: write d1/db/d31/d3a/d42/d4c/f99 [171135,11881] 0 2024-04-12T20:56:09.515 INFO:tasks.workunit.client.0.smithi018.stdout:4/558: creat d3/d34/d43/fb1 x:0 0 0 2024-04-12T20:56:09.518 INFO:tasks.workunit.client.0.smithi018.stdout:4/559: chown d3/c13 8769809 1 2024-04-12T20:56:09.521 INFO:tasks.workunit.client.0.smithi018.stdout:6/663: creat d5/df/d19/ffa x:0 0 0 2024-04-12T20:56:09.524 INFO:tasks.workunit.client.0.smithi018.stdout:7/473: dread d1/db/d31/d3a/d4f/d69/f89 [0,4194304] 0 2024-04-12T20:56:09.524 INFO:tasks.workunit.client.0.smithi018.stdout:4/560: creat d3/d5d/fb2 x:0 0 0 2024-04-12T20:56:09.525 INFO:tasks.workunit.client.0.smithi018.stdout:6/664: truncate d5/d6/d1b/d3b/d8e/f92 405313 0 2024-04-12T20:56:09.528 INFO:tasks.workunit.client.0.smithi018.stdout:4/561: symlink d3/d5d/d60/d64/lb3 0 2024-04-12T20:56:09.530 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:56:09 smithi186 ceph-mon[36597]: pgmap v37: 65 pgs: 65 active+clean; 1015 MiB data, 4.5 GiB used, 532 GiB / 536 GiB avail; 27 MiB/s rd, 114 MiB/s wr, 269 op/s 2024-04-12T20:56:09.531 INFO:tasks.workunit.client.0.smithi018.stdout:7/474: rename d1/db/d31/d3a/d4f to d1/db/d31/d45/d8c/d9d 0 2024-04-12T20:56:09.531 INFO:tasks.workunit.client.0.smithi018.stdout:6/665: creat d5/d6/d90/ffb x:0 0 0 2024-04-12T20:56:09.531 INFO:tasks.workunit.client.0.smithi018.stdout:4/562: symlink d3/d34/d53/lb4 0 2024-04-12T20:56:09.549 INFO:tasks.workunit.client.0.smithi018.stdout:6/666: dread d5/d15/dd1/fe4 [0,4194304] 0 2024-04-12T20:56:09.549 INFO:tasks.workunit.client.0.smithi018.stdout:7/475: dread d1/db/d31/f58 [0,4194304] 0 2024-04-12T20:56:09.554 INFO:tasks.workunit.client.0.smithi018.stdout:7/476: rmdir d1/db/d31/d45/d8c 39 2024-04-12T20:56:09.556 INFO:tasks.workunit.client.0.smithi018.stdout:6/667: write d5/df/d19/d43/d5f/fb4 [1736944,16466] 0 2024-04-12T20:56:09.560 INFO:tasks.workunit.client.0.smithi018.stdout:4/563: dwrite d3/d34/d43/f2a [0,4194304] 0 2024-04-12T20:56:09.562 INFO:tasks.workunit.client.0.smithi018.stdout:7/477: creat d1/db/d31/d45/f9e x:0 0 0 2024-04-12T20:56:09.570 INFO:tasks.workunit.client.0.smithi018.stdout:6/668: creat d5/d6/d1b/d3b/ffc x:0 0 0 2024-04-12T20:56:09.572 INFO:tasks.workunit.client.0.smithi018.stdout:7/478: write d1/db/d31/d45/d8c/d9d/d70/d96/f68 [4686552,27554] 0 2024-04-12T20:56:09.583 INFO:tasks.workunit.client.0.smithi018.stdout:1/389: sync 2024-04-12T20:56:09.583 INFO:tasks.workunit.client.0.smithi018.stdout:9/665: sync 2024-04-12T20:56:09.584 INFO:tasks.workunit.client.0.smithi018.stdout:0/392: sync 2024-04-12T20:56:09.584 INFO:tasks.workunit.client.0.smithi018.stdout:8/632: sync 2024-04-12T20:56:09.585 INFO:tasks.workunit.client.0.smithi018.stdout:7/479: link d1/db/d31/d45/d8c/d9d/d69/f89 d1/db/f9f 0 2024-04-12T20:56:09.586 INFO:tasks.workunit.client.0.smithi018.stdout:6/669: readlink d5/df/d3f/dc1/lcc 0 2024-04-12T20:56:09.589 INFO:tasks.workunit.client.0.smithi018.stdout:4/564: dwrite d3/d5d/d60/d64/d4f/f95 [0,4194304] 0 2024-04-12T20:56:09.590 INFO:tasks.workunit.client.0.smithi018.stdout:6/670: chown d5/df/l67 442437 1 2024-04-12T20:56:09.592 INFO:tasks.workunit.client.0.smithi018.stdout:8/633: chown db/d4e/d56/d59/d76/da5/lad 186078 1 2024-04-12T20:56:09.592 INFO:tasks.workunit.client.0.smithi018.stdout:7/480: readlink d1/db/d31/d45/d8c/d9d/d70/d96/l62 0 2024-04-12T20:56:09.597 INFO:tasks.workunit.client.0.smithi018.stdout:4/565: rmdir d3/d34 39 2024-04-12T20:56:09.597 INFO:tasks.workunit.client.0.smithi018.stdout:1/390: symlink d3/d9/da/dd/d31/d32/d13/d19/d2d/d4e/d7a/l8d 0 2024-04-12T20:56:09.597 INFO:tasks.workunit.client.0.smithi018.stdout:8/634: mknod db/d25/cc9 0 2024-04-12T20:56:09.602 INFO:tasks.workunit.client.0.smithi018.stdout:4/566: readlink d3/d32/l3e 0 2024-04-12T20:56:09.729 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:56:09.730+0000 7fc48b400700 1 -- 172.21.15.18:0/1430457006 --> [v2:172.21.15.186:6828/2316349902,v1:172.21.15.186:6829/2316349902] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fc460000bf0 con 0x7fc46806bdb0 2024-04-12T20:56:09.732 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:56:09.734+0000 7fc480c00700 1 -- 172.21.15.18:0/1430457006 <== mgr.24359 v2:172.21.15.186:6828/2316349902 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+310 (secure 0 0 0) 0x7fc460000bf0 con 0x7fc46806bdb0 2024-04-12T20:56:09.735 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:56:09.737+0000 7fc48b400700 1 -- 172.21.15.18:0/1430457006 >> [v2:172.21.15.186:6828/2316349902,v1:172.21.15.186:6829/2316349902] conn(0x7fc46806bdb0 msgr2=0x7fc46806e260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:56:09.735 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:56:09.737+0000 7fc48b400700 1 --2- 172.21.15.18:0/1430457006 >> [v2:172.21.15.186:6828/2316349902,v1:172.21.15.186:6829/2316349902] conn(0x7fc46806bdb0 0x7fc46806e260 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fc47c009fd0 tx=0x7fc47c00b040 comp rx=0 tx=0).stop 2024-04-12T20:56:09.735 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:56:09.737+0000 7fc48b400700 1 -- 172.21.15.18:0/1430457006 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc4841046b0 msgr2=0x7fc4841144d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:56:09.736 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:56:09.737+0000 7fc48b400700 1 --2- 172.21.15.18:0/1430457006 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc4841046b0 0x7fc4841144d0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fc46c00b3b0 tx=0x7fc46c009e30 comp rx=0 tx=0).stop 2024-04-12T20:56:09.736 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:56:09.737+0000 7fc48b400700 1 -- 172.21.15.18:0/1430457006 shutdown_connections 2024-04-12T20:56:09.736 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:56:09.737+0000 7fc48b400700 1 --2- 172.21.15.18:0/1430457006 >> [v2:172.21.15.186:6828/2316349902,v1:172.21.15.186:6829/2316349902] conn(0x7fc46806bdb0 0x7fc46806e260 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:56:09.736 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:56:09.737+0000 7fc48b400700 1 --2- 172.21.15.18:0/1430457006 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc484103400 0x7fc484113f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:56:09.736 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:56:09.737+0000 7fc48b400700 1 --2- 172.21.15.18:0/1430457006 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc4841046b0 0x7fc4841144d0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:56:09.736 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:56:09.737+0000 7fc48b400700 1 -- 172.21.15.18:0/1430457006 >> 172.21.15.18:0/1430457006 conn(0x7fc484072590 msgr2=0x7fc484073bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T20:56:09.736 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:56:09.737+0000 7fc48b400700 1 -- 172.21.15.18:0/1430457006 shutdown_connections 2024-04-12T20:56:09.736 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:56:09.737+0000 7fc48b400700 1 -- 172.21.15.18:0/1430457006 wait complete. 2024-04-12T20:56:09.767 INFO:tasks.workunit.client.0.smithi018.stdout:6/671: truncate d5/d6/d1b/d3b/fdc 12082 0 2024-04-12T20:56:09.772 INFO:tasks.workunit.client.0.smithi018.stdout:1/391: dread d3/d9/da/dd/d31/d32/d13/d19/f62 [0,4194304] 0 2024-04-12T20:56:09.776 INFO:tasks.workunit.client.0.smithi018.stdout:0/393: dwrite da/f38 [4194304,4194304] 0 2024-04-12T20:56:09.777 INFO:tasks.workunit.client.0.smithi018.stdout:8/635: mknod db/d4e/d56/d59/d76/da5/d45/d86/dbb/cca 0 2024-04-12T20:56:09.778 INFO:tasks.workunit.client.0.smithi018.stdout:6/672: creat d5/d6/d1b/d3b/d22/ffd x:0 0 0 2024-04-12T20:56:09.782 INFO:tasks.workunit.client.0.smithi018.stdout:8/636: dread - db/d4e/d56/d59/d76/da5/d45/d86/f9d zero size 2024-04-12T20:56:09.783 INFO:tasks.workunit.client.0.smithi018.stdout:0/394: creat da/d15/d17/d5c/d79/f86 x:0 0 0 2024-04-12T20:56:09.787 INFO:tasks.workunit.client.0.smithi018.stdout:1/392: truncate d3/d9/da/dd/f12 1353593 0 2024-04-12T20:56:09.788 INFO:tasks.workunit.client.0.smithi018.stdout:7/481: dread d1/f3e [0,4194304] 0 2024-04-12T20:56:09.790 INFO:tasks.workunit.client.0.smithi018.stdout:0/395: write da/d15/d3e/d3f/f69 [380760,65313] 0 2024-04-12T20:56:09.791 INFO:tasks.workunit.client.0.smithi018.stdout:0/396: chown da/f75 49977 1 2024-04-12T20:56:09.792 INFO:tasks.workunit.client.0.smithi018.stdout:1/393: fdatasync d3/d9/da/dd/f3f 0 2024-04-12T20:56:09.794 INFO:tasks.workunit.client.0.smithi018.stdout:7/482: creat d1/d6/fa0 x:0 0 0 2024-04-12T20:56:09.798 INFO:tasks.workunit.client.0.smithi018.stdout:0/397: creat da/d15/d17/d21/d27/f87 x:0 0 0 2024-04-12T20:56:09.801 INFO:tasks.workunit.client.0.smithi018.stdout:7/483: creat d1/db/d31/d3a/d97/fa1 x:0 0 0 2024-04-12T20:56:09.806 INFO:tasks.workunit.client.0.smithi018.stdout:1/394: mknod d3/d9/da/dd/d31/d32/d33/d80/c8e 0 2024-04-12T20:56:09.807 INFO:tasks.workunit.client.0.smithi018.stdout:7/484: write d1/db/d31/f58 [284797,69029] 0 2024-04-12T20:56:09.812 INFO:tasks.workunit.client.0.smithi018.stdout:1/395: dread - d3/d9/da/dd/d31/d32/d13/d19/d2d/d4e/f60 zero size 2024-04-12T20:56:09.815 INFO:tasks.workunit.client.0.smithi018.stdout:8/637: dwrite db/fb3 [0,4194304] 0 2024-04-12T20:56:09.821 INFO:tasks.workunit.client.0.smithi018.stdout:8/638: write db/d4e/d56/d59/d76/fbe [74263,75643] 0 2024-04-12T20:56:09.825 INFO:tasks.workunit.client.0.smithi018.stdout:0/398: dwrite da/d15/d17/d21/d36/d47/f5d [0,4194304] 0 2024-04-12T20:56:09.830 INFO:tasks.workunit.client.0.smithi018.stdout:8/639: truncate db/d4e/d9a/d71/f9f 2421779 0 2024-04-12T20:56:09.831 INFO:tasks.workunit.client.0.smithi018.stdout:0/399: mkdir da/d15/d17/d21/d36/d88 0 2024-04-12T20:56:09.835 INFO:tasks.workunit.client.0.smithi018.stdout:7/485: dwrite d1/db/d31/d45/d53/f56 [0,4194304] 0 2024-04-12T20:56:09.843 INFO:tasks.workunit.client.0.smithi018.stdout:7/486: dread d1/db/d31/f58 [0,4194304] 0 2024-04-12T20:56:09.853 INFO:tasks.workunit.client.0.smithi018.stdout:8/640: dwrite db/d4e/d56/d59/d76/da5/fc6 [0,4194304] 0 2024-04-12T20:56:09.863 INFO:tasks.workunit.client.0.smithi018.stdout:0/400: dwrite da/d15/f41 [0,4194304] 0 2024-04-12T20:56:09.871 INFO:tasks.workunit.client.0.smithi018.stdout:0/401: truncate da/d15/d17/d21/d36/f80 70148 0 2024-04-12T20:56:09.880 INFO:tasks.workunit.client.0.smithi018.stdout:8/641: dwrite db/d4e/d73/f8d [0,4194304] 0 2024-04-12T20:56:09.884 INFO:tasks.workunit.client.0.smithi018.stdout:8/642: creat db/d4e/d9a/d69/d6c/fcb x:0 0 0 2024-04-12T20:56:09.889 INFO:tasks.workunit.client.0.smithi018.stdout:8/643: getdents db/d72 0 2024-04-12T20:56:09.895 INFO:tasks.workunit.client.0.smithi018.stdout:8/644: getdents db/d4e/d9a/daa 0 2024-04-12T20:56:09.938 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:56:09.939+0000 7ffb0de00700 1 -- 172.21.15.18:0/31386673 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ffb08070870 msgr2=0x7ffb08070ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:56:09.938 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:56:09.939+0000 7ffb0de00700 1 --2- 172.21.15.18:0/31386673 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ffb08070870 0x7ffb08070ce0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7ffaf4004940 tx=0x7ffaf4009e30 comp rx=0 tx=0).stop 2024-04-12T20:56:09.938 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:56:09.939+0000 7ffb0de00700 1 -- 172.21.15.18:0/31386673 shutdown_connections 2024-04-12T20:56:09.938 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:56:09.939+0000 7ffb0de00700 1 --2- 172.21.15.18:0/31386673 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ffb08070870 0x7ffb08070ce0 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:56:09.938 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:56:09.939+0000 7ffb0de00700 1 --2- 172.21.15.18:0/31386673 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ffb08071e20 0x7ffb08070330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:56:09.938 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:56:09.939+0000 7ffb0de00700 1 -- 172.21.15.18:0/31386673 >> 172.21.15.18:0/31386673 conn(0x7ffb080f8550 msgr2=0x7ffb080fa980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T20:56:09.938 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:56:09.939+0000 7ffb0de00700 1 -- 172.21.15.18:0/31386673 shutdown_connections 2024-04-12T20:56:09.938 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:56:09.939+0000 7ffb0de00700 1 -- 172.21.15.18:0/31386673 wait complete. 2024-04-12T20:56:09.939 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:56:09.940+0000 7ffb0de00700 1 Processor -- start 2024-04-12T20:56:09.940 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:56:09.940+0000 7ffb0de00700 1 -- start start 2024-04-12T20:56:09.940 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:56:09.941+0000 7ffb0de00700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ffb08070870 0x7ffb0810fc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T20:56:09.940 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:56:09.941+0000 7ffb0de00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ffb08071e20 0x7ffb08110170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T20:56:09.940 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:56:09.941+0000 7ffb0de00700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb08110770 con 0x7ffb08071e20 2024-04-12T20:56:09.940 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:56:09.941+0000 7ffb0de00700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb08110fc0 con 0x7ffb08070870 2024-04-12T20:56:09.940 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:56:09.941+0000 7ffb06000700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ffb08071e20 0x7ffb08110170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T20:56:09.940 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:56:09.941+0000 7ffb06000700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ffb08071e20 0x7ffb08110170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:55610/0 (socket says 172.21.15.18:55610) 2024-04-12T20:56:09.940 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:56:09.941+0000 7ffb06000700 1 -- 172.21.15.18:0/3292096487 learned_addr learned my addr 172.21.15.18:0/3292096487 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T20:56:09.940 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:56:09.941+0000 7ffb06a00700 1 --2- 172.21.15.18:0/3292096487 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ffb08070870 0x7ffb0810fc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T20:56:09.940 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:56:09.942+0000 7ffb06000700 1 -- 172.21.15.18:0/3292096487 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ffb08070870 msgr2=0x7ffb0810fc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:56:09.940 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:56:09.942+0000 7ffb06000700 1 --2- 172.21.15.18:0/3292096487 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ffb08070870 0x7ffb0810fc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:56:09.941 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:56:09.942+0000 7ffb06000700 1 -- 172.21.15.18:0/3292096487 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffaf4005040 con 0x7ffb08071e20 2024-04-12T20:56:09.941 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:56:09.942+0000 7ffb06000700 1 --2- 172.21.15.18:0/3292096487 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ffb08071e20 0x7ffb08110170 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7ffaf400b3b0 tx=0x7ffaf4009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T20:56:09.941 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:56:09.942+0000 7ffafea00700 1 -- 172.21.15.18:0/3292096487 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ffaf400e040 con 0x7ffb08071e20 2024-04-12T20:56:09.941 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:56:09.942+0000 7ffafea00700 1 -- 172.21.15.18:0/3292096487 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ffaf4004290 con 0x7ffb08071e20 2024-04-12T20:56:09.941 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:56:09.942+0000 7ffafea00700 1 -- 172.21.15.18:0/3292096487 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ffaf401c3f0 con 0x7ffb08071e20 2024-04-12T20:56:09.942 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:56:09.942+0000 7ffb0de00700 1 -- 172.21.15.18:0/3292096487 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffb08111240 con 0x7ffb08071e20 2024-04-12T20:56:09.942 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:56:09.943+0000 7ffb0de00700 1 -- 172.21.15.18:0/3292096487 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffb08111780 con 0x7ffb08071e20 2024-04-12T20:56:09.942 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:56:09.944+0000 7ffb0de00700 1 -- 172.21.15.18:0/3292096487 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffb0805f5f0 con 0x7ffb08071e20 2024-04-12T20:56:09.943 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:56:09.945+0000 7ffafea00700 1 -- 172.21.15.18:0/3292096487 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 24) v1 ==== 89644+0+0 (secure 0 0 0) 0x7ffaf400bbf0 con 0x7ffb08071e20 2024-04-12T20:56:09.944 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:56:09.946+0000 7ffafea00700 1 --2- 172.21.15.18:0/3292096487 >> [v2:172.21.15.186:6828/2316349902,v1:172.21.15.186:6829/2316349902] conn(0x7ffaec06bcf0 0x7ffaec06e1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T20:56:09.944 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:56:09.946+0000 7ffafea00700 1 -- 172.21.15.18:0/3292096487 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5808+0+0 (secure 0 0 0) 0x7ffaf408baf0 con 0x7ffb08071e20 2024-04-12T20:56:09.944 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:56:09.947+0000 7ffb06a00700 1 --2- 172.21.15.18:0/3292096487 >> [v2:172.21.15.186:6828/2316349902,v1:172.21.15.186:6829/2316349902] conn(0x7ffaec06bcf0 0x7ffaec06e1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T20:56:09.945 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:56:09.947+0000 7ffb06a00700 1 --2- 172.21.15.18:0/3292096487 >> [v2:172.21.15.186:6828/2316349902,v1:172.21.15.186:6829/2316349902] conn(0x7ffaec06bcf0 0x7ffaec06e1a0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7ffb08071b60 tx=0x7ffaf800b040 comp rx=0 tx=0).ready entity=mgr.24359 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T20:56:09.948 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:56:09.951+0000 7ffafea00700 1 -- 172.21.15.18:0/3292096487 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186454 (secure 0 0 0) 0x7ffaf4058ca0 con 0x7ffb08071e20 2024-04-12T20:56:10.000 INFO:tasks.workunit.client.0.smithi018.stdout:0/402: write da/d15/d3e/f54 [600507,24062] 0 2024-04-12T20:56:10.008 INFO:tasks.workunit.client.0.smithi018.stdout:0/403: rename da/d15/d17/d21/l2a to da/d15/d17/l89 0 2024-04-12T20:56:10.010 INFO:tasks.workunit.client.0.smithi018.stdout:0/404: mknod da/d15/d4d/c8a 0 2024-04-12T20:56:10.014 INFO:tasks.workunit.client.0.smithi018.stdout:0/405: mkdir da/d15/d3e/d83/d8b 0 2024-04-12T20:56:10.017 INFO:tasks.workunit.client.0.smithi018.stdout:0/406: creat da/d15/d17/d21/d36/d47/f8c x:0 0 0 2024-04-12T20:56:10.035 INFO:tasks.workunit.client.0.smithi018.stdout:0/407: dwrite da/d15/d4d/f85 [0,4194304] 0 2024-04-12T20:56:10.040 INFO:tasks.workunit.client.0.smithi018.stdout:0/408: creat da/d15/d17/d21/d36/d47/d81/f8d x:0 0 0 2024-04-12T20:56:10.042 INFO:tasks.workunit.client.0.smithi018.stdout:0/409: read - da/f52 zero size 2024-04-12T20:56:10.046 INFO:tasks.workunit.client.0.smithi018.stdout:0/410: unlink da/d15/d3e/d3f/l77 0 2024-04-12T20:56:10.048 INFO:tasks.workunit.client.0.smithi018.stdout:0/411: write da/d15/d17/d5c/f82 [316468,37059] 0 2024-04-12T20:56:10.060 INFO:tasks.workunit.client.0.smithi018.stdout:0/412: truncate da/d15/d17/f43 3311849 0 2024-04-12T20:56:10.071 INFO:tasks.workunit.client.0.smithi018.stdout:0/413: truncate da/d15/d3e/d3f/f5b 6410600 0 2024-04-12T20:56:10.074 INFO:tasks.workunit.client.0.smithi018.stdout:0/414: truncate da/d15/f39 626736 0 2024-04-12T20:56:10.079 INFO:tasks.workunit.client.0.smithi018.stdout:0/415: creat da/d15/f8e x:0 0 0 2024-04-12T20:56:10.083 INFO:tasks.workunit.client.0.smithi018.stdout:0/416: creat da/d15/d3e/d3f/f8f x:0 0 0 2024-04-12T20:56:10.087 INFO:tasks.workunit.client.0.smithi018.stdout:0/417: creat da/d15/f90 x:0 0 0 2024-04-12T20:56:10.104 INFO:tasks.workunit.client.0.smithi018.stdout:0/418: read da/d15/f39 [255933,37392] 0 2024-04-12T20:56:10.109 INFO:tasks.workunit.client.0.smithi018.stdout:0/419: mkdir da/d15/d17/d21/d36/d91 0 2024-04-12T20:56:10.113 INFO:tasks.workunit.client.0.smithi018.stdout:0/420: unlink da/d15/d3e/f54 0 2024-04-12T20:56:10.117 INFO:tasks.workunit.client.0.smithi018.stdout:0/421: getdents da/d15/d17/d21/d36/d47 0 2024-04-12T20:56:10.119 INFO:tasks.workunit.client.0.smithi018.stdout:0/422: truncate da/d15/f90 287012 0 2024-04-12T20:56:10.121 INFO:tasks.workunit.client.0.smithi018.stdout:0/423: read da/d15/d3e/f5e [168907,100118] 0 2024-04-12T20:56:10.127 INFO:tasks.workunit.client.0.smithi018.stdout:0/424: dread da/d15/d17/d21/d27/f49 [0,4194304] 0 2024-04-12T20:56:10.131 INFO:tasks.workunit.client.0.smithi018.stdout:0/425: truncate da/d15/f39 1455117 0 2024-04-12T20:56:10.149 INFO:tasks.workunit.client.0.smithi018.stdout:0/426: dwrite da/f38 [8388608,4194304] 0 2024-04-12T20:56:10.156 INFO:tasks.workunit.client.0.smithi018.stdout:0/427: truncate da/f22 727131 0 2024-04-12T20:56:10.170 INFO:tasks.workunit.client.0.smithi018.stdout:0/428: write da/d15/d17/d21/d27/f49 [792295,9412] 0 2024-04-12T20:56:10.175 INFO:tasks.workunit.client.0.smithi018.stdout:0/429: write da/d15/d17/d21/d36/f7c [2082743,45724] 0 2024-04-12T20:56:10.184 INFO:tasks.workunit.client.0.smithi018.stdout:0/430: write da/d15/f39 [1418576,104342] 0 2024-04-12T20:56:10.187 INFO:tasks.workunit.client.0.smithi018.stdout:0/431: dread da/d15/d3e/f5e [0,4194304] 0 2024-04-12T20:56:10.196 INFO:tasks.workunit.client.0.smithi018.stdout:0/432: link da/d15/d17/d21/d36/l4a da/d15/d17/d21/d36/l92 0 2024-04-12T20:56:10.216 INFO:tasks.workunit.client.0.smithi018.stdout:0/433: dwrite da/d15/d3e/f5e [0,4194304] 0 2024-04-12T20:56:10.222 INFO:tasks.workunit.client.0.smithi018.stdout:0/434: getdents da/d15 0 2024-04-12T20:56:10.223 INFO:tasks.workunit.client.0.smithi018.stdout:0/435: fdatasync da/d15/d17/d21/d27/f4f 0 2024-04-12T20:56:10.230 INFO:tasks.workunit.client.0.smithi018.stdout:0/436: link da/d15/d17/d21/d36/c3d da/d15/d17/d21/d36/d47/c93 0 2024-04-12T20:56:10.231 INFO:tasks.workunit.client.0.smithi018.stdout:0/437: read - da/d15/f8e zero size 2024-04-12T20:56:10.249 INFO:tasks.workunit.client.0.smithi018.stdout:0/438: dread da/fc [0,4194304] 0 2024-04-12T20:56:10.252 INFO:tasks.workunit.client.0.smithi018.stdout:0/439: mkdir da/d15/d17/d21/d94 0 2024-04-12T20:56:10.260 INFO:tasks.workunit.client.0.smithi018.stdout:0/440: link da/d15/d17/d21/l62 da/d15/d17/d21/d36/d47/l95 0 2024-04-12T20:56:10.263 INFO:tasks.workunit.client.0.smithi018.stdout:0/441: write da/d15/d17/d21/d36/d6e/f7f [140946,127939] 0 2024-04-12T20:56:10.267 INFO:tasks.workunit.client.0.smithi018.stdout:0/442: fdatasync da/d15/d3e/f73 0 2024-04-12T20:56:10.271 INFO:tasks.workunit.client.0.smithi018.stdout:0/443: creat da/d15/d4d/d6b/f96 x:0 0 0 2024-04-12T20:56:10.273 INFO:tasks.workunit.client.0.smithi018.stdout:2/445: sync 2024-04-12T20:56:10.283 INFO:tasks.workunit.client.0.smithi018.stdout:2/446: write d5/d8/d10/d19/f7b [795018,86640] 0 2024-04-12T20:56:10.288 INFO:tasks.workunit.client.0.smithi018.stdout:2/447: creat d5/d8/d10/d19/d24/d39/fa1 x:0 0 0 2024-04-12T20:56:10.288 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:56:10.290+0000 7ffb0de00700 1 -- 172.21.15.18:0/3292096487 --> [v2:172.21.15.186:6828/2316349902,v1:172.21.15.186:6829/2316349902] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7ffb08111b70 con 0x7ffaec06bcf0 2024-04-12T20:56:10.288 INFO:tasks.workunit.client.0.smithi018.stdout:0/444: dwrite da/f25 [0,4194304] 0 2024-04-12T20:56:10.291 INFO:tasks.workunit.client.0.smithi018.stdout:0/445: read - da/d15/f8e zero size 2024-04-12T20:56:10.309 INFO:tasks.workunit.client.0.smithi018.stdout:0/446: dread da/d15/f39 [0,4194304] 0 2024-04-12T20:56:10.310 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:56:10.312+0000 7ffafea00700 1 -- 172.21.15.18:0/3292096487 <== mgr.24359 v2:172.21.15.186:6828/2316349902 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3388 (secure 0 0 0) 0x7ffb08111b70 con 0x7ffaec06bcf0 2024-04-12T20:56:10.311 INFO:teuthology.orchestra.run.smithi018.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-12T20:56:10.311 INFO:teuthology.orchestra.run.smithi018.stdout:alertmanager.smithi018 smithi018 *:9093,9094 starting - - - - 2024-04-12T20:56:10.311 INFO:teuthology.orchestra.run.smithi018.stdout:crash.smithi018 smithi018 running (11m) 51s ago 11m 6660k - 17.2.7-827-gd3b5060a 21de9b7369ad 1ac7603bddfd 2024-04-12T20:56:10.311 INFO:teuthology.orchestra.run.smithi018.stdout:crash.smithi186 smithi186 running (10m) 58s ago 10m 6669k - 17.2.7-827-gd3b5060a 21de9b7369ad 14bf8a1672eb 2024-04-12T20:56:10.311 INFO:teuthology.orchestra.run.smithi018.stdout:grafana.smithi018 smithi018 *:3000 starting - - - - 2024-04-12T20:56:10.311 INFO:teuthology.orchestra.run.smithi018.stdout:mds.cephfs.smithi018.acqbsc smithi018 running (4m) 51s ago 4m 69.0M - 17.2.7-827-gd3b5060a 21de9b7369ad 5b5fce65f18a 2024-04-12T20:56:10.311 INFO:teuthology.orchestra.run.smithi018.stdout:mds.cephfs.smithi018.szjluo smithi018 running (4m) 51s ago 4m 65.4M - 17.2.7-827-gd3b5060a 21de9b7369ad cf9bf0c5031f 2024-04-12T20:56:10.311 INFO:teuthology.orchestra.run.smithi018.stdout:mds.cephfs.smithi186.jdllyn smithi186 running (4m) 58s ago 4m 11.8M - 17.2.7-827-gd3b5060a 21de9b7369ad 2b2bc46bd29a 2024-04-12T20:56:10.311 INFO:teuthology.orchestra.run.smithi018.stdout:mds.cephfs.smithi186.wjnafs smithi186 running (3m) 58s ago 3m 11.9M - 17.2.7-827-gd3b5060a 21de9b7369ad 74b115d6d6fd 2024-04-12T20:56:10.311 INFO:teuthology.orchestra.run.smithi018.stdout:mgr.smithi018.xjzwga smithi018 *:9283 running (13m) 51s ago 12m 331M - 17.2.7-827-gd3b5060a 21de9b7369ad c72f43857311 2024-04-12T20:56:10.311 INFO:teuthology.orchestra.run.smithi018.stdout:mgr.smithi186.kqcfxk smithi186 *:8443,9283 running (90s) 58s ago 9m 492M - 19.0.0-2879-g324baf8d 8717720dc5e5 bd4971d5260e 2024-04-12T20:56:10.312 INFO:teuthology.orchestra.run.smithi018.stdout:mon.smithi018 smithi018 running (13m) 51s ago 13m 47.1M 2048M 17.2.7-827-gd3b5060a 21de9b7369ad cd84caafa056 2024-04-12T20:56:10.312 INFO:teuthology.orchestra.run.smithi018.stdout:mon.smithi186 smithi186 running (9m) 58s ago 9m 37.0M 2048M 17.2.7-827-gd3b5060a 21de9b7369ad d7263c772b73 2024-04-12T20:56:10.312 INFO:teuthology.orchestra.run.smithi018.stdout:node-exporter.smithi018 smithi018 *:9100 starting - - - - 2024-04-12T20:56:10.312 INFO:teuthology.orchestra.run.smithi018.stdout:node-exporter.smithi186 smithi186 *:9100 running (10m) 58s ago 10m 8334k - 1.5.0 0da6a335fe13 57160d1060f2 2024-04-12T20:56:10.312 INFO:teuthology.orchestra.run.smithi018.stdout:osd.0 smithi018 running (8m) 51s ago 8m 83.2M 1559M 17.2.7-827-gd3b5060a 21de9b7369ad 0d83e027bf0a 2024-04-12T20:56:10.312 INFO:teuthology.orchestra.run.smithi018.stdout:osd.1 smithi018 running (8m) 51s ago 8m 90.2M 1559M 17.2.7-827-gd3b5060a 21de9b7369ad 6517c9cc7f87 2024-04-12T20:56:10.312 INFO:teuthology.orchestra.run.smithi018.stdout:osd.2 smithi018 running (7m) 51s ago 7m 75.7M 1559M 17.2.7-827-gd3b5060a 21de9b7369ad 8a83692acaa5 2024-04-12T20:56:10.312 INFO:teuthology.orchestra.run.smithi018.stdout:osd.3 smithi186 running (7m) 58s ago 7m 88.7M 2581M 17.2.7-827-gd3b5060a 21de9b7369ad c2def7a90d76 2024-04-12T20:56:10.312 INFO:teuthology.orchestra.run.smithi018.stdout:osd.4 smithi186 running (6m) 58s ago 6m 84.8M 2581M 17.2.7-827-gd3b5060a 21de9b7369ad 757e7c71b7b9 2024-04-12T20:56:10.313 INFO:teuthology.orchestra.run.smithi018.stdout:osd.5 smithi186 running (5m) 58s ago 5m 80.8M 2581M 17.2.7-827-gd3b5060a 21de9b7369ad 2ee072a436d0 2024-04-12T20:56:10.313 INFO:teuthology.orchestra.run.smithi018.stdout:prometheus.smithi018 smithi018 *:9095 running (9m) 51s ago 10m 51.1M - 2.43.0 a07b618ecd1d aecd046db448 2024-04-12T20:56:10.313 INFO:tasks.workunit.client.0.smithi018.stdout:2/448: dwrite d5/d8/d10/f78 [0,4194304] 0 2024-04-12T20:56:10.313 INFO:tasks.workunit.client.0.smithi018.stdout:0/447: symlink da/d15/d17/d5c/d79/l97 0 2024-04-12T20:56:10.315 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:56:10.316+0000 7ffb0de00700 1 -- 172.21.15.18:0/3292096487 >> [v2:172.21.15.186:6828/2316349902,v1:172.21.15.186:6829/2316349902] conn(0x7ffaec06bcf0 msgr2=0x7ffaec06e1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:56:10.315 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:56:10.316+0000 7ffb0de00700 1 --2- 172.21.15.18:0/3292096487 >> [v2:172.21.15.186:6828/2316349902,v1:172.21.15.186:6829/2316349902] conn(0x7ffaec06bcf0 0x7ffaec06e1a0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7ffb08071b60 tx=0x7ffaf800b040 comp rx=0 tx=0).stop 2024-04-12T20:56:10.315 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:56:10.316+0000 7ffb0de00700 1 -- 172.21.15.18:0/3292096487 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ffb08071e20 msgr2=0x7ffb08110170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:56:10.315 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:56:10.316+0000 7ffb0de00700 1 --2- 172.21.15.18:0/3292096487 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ffb08071e20 0x7ffb08110170 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7ffaf400b3b0 tx=0x7ffaf4009e30 comp rx=0 tx=0).stop 2024-04-12T20:56:10.316 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:56:10.316+0000 7ffb0de00700 1 -- 172.21.15.18:0/3292096487 shutdown_connections 2024-04-12T20:56:10.316 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:56:10.316+0000 7ffb0de00700 1 --2- 172.21.15.18:0/3292096487 >> [v2:172.21.15.186:6828/2316349902,v1:172.21.15.186:6829/2316349902] conn(0x7ffaec06bcf0 0x7ffaec06e1a0 secure :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7ffb08071b60 tx=0x7ffaf800b040 comp rx=0 tx=0).stop 2024-04-12T20:56:10.316 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:56:10.316+0000 7ffb0de00700 1 --2- 172.21.15.18:0/3292096487 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ffb08070870 0x7ffb0810fc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:56:10.316 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:56:10.316+0000 7ffb0de00700 1 --2- 172.21.15.18:0/3292096487 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ffb08071e20 0x7ffb08110170 secure :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7ffaf400b3b0 tx=0x7ffaf4009e30 comp rx=0 tx=0).stop 2024-04-12T20:56:10.316 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:56:10.316+0000 7ffb0de00700 1 -- 172.21.15.18:0/3292096487 >> 172.21.15.18:0/3292096487 conn(0x7ffb080f8550 msgr2=0x7ffb080f9c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T20:56:10.317 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:56:10.317+0000 7ffb0de00700 1 -- 172.21.15.18:0/3292096487 shutdown_connections 2024-04-12T20:56:10.317 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:56:10.317+0000 7ffb0de00700 1 -- 172.21.15.18:0/3292096487 wait complete. 2024-04-12T20:56:10.319 INFO:tasks.workunit.client.0.smithi018.stdout:2/449: creat d5/d8/d10/d19/d24/d6c/fa2 x:0 0 0 2024-04-12T20:56:10.319 INFO:tasks.workunit.client.0.smithi018.stdout:2/450: readlink d5/d12/l74 0 2024-04-12T20:56:10.321 INFO:tasks.workunit.client.0.smithi018.stdout:0/448: rename da/d15/d17/d21/d27/l5a to da/d15/d3e/d83/l98 0 2024-04-12T20:56:10.326 INFO:tasks.workunit.client.0.smithi018.stdout:2/451: mknod d5/d8/d10/d19/ca3 0 2024-04-12T20:56:10.329 INFO:tasks.workunit.client.0.smithi018.stdout:0/449: getdents da/d15 0 2024-04-12T20:56:10.330 INFO:tasks.workunit.client.0.smithi018.stdout:0/450: chown da/f1d 2133 1 2024-04-12T20:56:10.331 INFO:tasks.workunit.client.0.smithi018.stdout:2/452: truncate d5/d8/d10/d19/d24/d6c/d90/f9d 526408 0 2024-04-12T20:56:10.335 INFO:tasks.workunit.client.0.smithi018.stdout:2/453: dread d5/d12/f58 [0,4194304] 0 2024-04-12T20:56:10.337 INFO:tasks.workunit.client.0.smithi018.stdout:2/454: readlink d5/d8/d33/l5e 0 2024-04-12T20:56:10.341 INFO:tasks.workunit.client.0.smithi018.stdout:2/455: truncate d5/d7/f4d 1191179 0 2024-04-12T20:56:10.344 INFO:tasks.workunit.client.0.smithi018.stdout:2/456: chown d5/d8/d10/d19/d24/f29 4410 1 2024-04-12T20:56:10.345 INFO:tasks.workunit.client.0.smithi018.stdout:2/457: stat d5/d8/d10 0 2024-04-12T20:56:10.347 INFO:tasks.workunit.client.0.smithi018.stdout:0/451: dwrite da/fc [0,4194304] 0 2024-04-12T20:56:10.348 INFO:tasks.workunit.client.0.smithi018.stdout:2/458: fdatasync d5/d7/f4d 0 2024-04-12T20:56:10.350 INFO:tasks.workunit.client.0.smithi018.stdout:0/452: fsync da/d15/d17/d21/d27/f87 0 2024-04-12T20:56:10.351 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:56:10 smithi018 ceph-mon[28799]: from='client.14644 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T20:56:10.353 INFO:tasks.workunit.client.0.smithi018.stdout:2/459: truncate d5/d7/d79/f7e 1097217 0 2024-04-12T20:56:10.361 INFO:tasks.workunit.client.0.smithi018.stdout:2/460: getdents d5/d8/d10/d19 0 2024-04-12T20:56:10.365 INFO:tasks.workunit.client.0.smithi018.stdout:2/461: fdatasync d5/d8/d10/d43/f56 0 2024-04-12T20:56:10.365 INFO:tasks.workunit.client.0.smithi018.stdout:9/666: sync 2024-04-12T20:56:10.366 INFO:tasks.workunit.client.0.smithi018.stdout:1/396: sync 2024-04-12T20:56:10.366 INFO:tasks.workunit.client.0.smithi018.stdout:4/567: sync 2024-04-12T20:56:10.366 INFO:tasks.workunit.client.0.smithi018.stdout:6/673: sync 2024-04-12T20:56:10.366 INFO:tasks.workunit.client.0.smithi018.stdout:3/473: sync 2024-04-12T20:56:10.367 INFO:tasks.workunit.client.0.smithi018.stdout:8/645: sync 2024-04-12T20:56:10.367 INFO:tasks.workunit.client.0.smithi018.stdout:5/506: sync 2024-04-12T20:56:10.367 INFO:tasks.workunit.client.0.smithi018.stdout:7/487: sync 2024-04-12T20:56:10.373 INFO:tasks.workunit.client.0.smithi018.stdout:2/462: mkdir d5/d8/d10/d19/d24/d39/da4 0 2024-04-12T20:56:10.375 INFO:tasks.workunit.client.0.smithi018.stdout:8/646: mknod db/d4e/d56/d59/d76/ccc 0 2024-04-12T20:56:10.379 INFO:tasks.workunit.client.0.smithi018.stdout:1/397: dread d3/d9/f3c [0,4194304] 0 2024-04-12T20:56:10.384 INFO:tasks.workunit.client.0.smithi018.stdout:4/568: creat d3/d34/d43/d4b/fb5 x:0 0 0 2024-04-12T20:56:10.387 INFO:tasks.workunit.client.0.smithi018.stdout:9/667: symlink d3/d6/d9/dc/d13/d5b/d6b/lc5 0 2024-04-12T20:56:10.388 INFO:tasks.workunit.client.0.smithi018.stdout:3/474: rename d2/d4d/f7f to d2/d83/f92 0 2024-04-12T20:56:10.388 INFO:tasks.workunit.client.0.smithi018.stdout:2/463: unlink d5/d8/d10/d19/d24/f32 0 2024-04-12T20:56:10.388 INFO:tasks.workunit.client.0.smithi018.stdout:8/647: mkdir db/d4e/d9a/daa/dcd 0 2024-04-12T20:56:10.393 INFO:tasks.workunit.client.0.smithi018.stdout:3/475: creat d2/d6/d3d/d54/f93 x:0 0 0 2024-04-12T20:56:10.393 INFO:tasks.workunit.client.0.smithi018.stdout:0/453: dwrite da/d15/f33 [0,4194304] 0 2024-04-12T20:56:10.400 INFO:tasks.workunit.client.0.smithi018.stdout:8/648: truncate db/d4e/d56/d59/d76/f90 149456 0 2024-04-12T20:56:10.402 INFO:tasks.workunit.client.0.smithi018.stdout:1/398: write d3/d9/da/dd/d31/d32/d13/d19/d2d/f76 [1980564,76631] 0 2024-04-12T20:56:10.406 INFO:tasks.workunit.client.0.smithi018.stdout:2/464: rename d5/d12/f5b to d5/d8/d10/d43/fa5 0 2024-04-12T20:56:10.406 INFO:tasks.workunit.client.0.smithi018.stdout:3/476: write d2/d6/d30/d31/d38/f66 [927548,12699] 0 2024-04-12T20:56:10.419 INFO:tasks.workunit.client.0.smithi018.stdout:9/668: fdatasync d3/d6/d9/fd 0 2024-04-12T20:56:10.420 INFO:tasks.workunit.client.0.smithi018.stdout:7/488: dwrite d1/db/d31/d45/d8c/d9d/d69/f7e [0,4194304] 0 2024-04-12T20:56:10.421 INFO:tasks.workunit.client.0.smithi018.stdout:8/649: symlink db/d95/dc0/lce 0 2024-04-12T20:56:10.427 INFO:tasks.workunit.client.0.smithi018.stdout:2/465: mkdir d5/d7/d31/d62/da6 0 2024-04-12T20:56:10.430 INFO:tasks.workunit.client.0.smithi018.stdout:8/650: chown db/d25 13443 1 2024-04-12T20:56:10.434 INFO:tasks.workunit.client.0.smithi018.stdout:7/489: chown d1/db/d31/l4b 60434544 1 2024-04-12T20:56:10.436 INFO:tasks.workunit.client.0.smithi018.stdout:9/669: dread - d3/d6/d9/dc/d5a/d79/d7e/fbe zero size 2024-04-12T20:56:10.437 INFO:tasks.workunit.client.0.smithi018.stdout:5/507: dwrite d0/d2/d33/d46/d1e/d2e/f75 [0,4194304] 0 2024-04-12T20:56:10.437 INFO:tasks.workunit.client.0.smithi018.stdout:8/651: chown db/f1b 10077 1 2024-04-12T20:56:10.439 INFO:tasks.workunit.client.0.smithi018.stdout:1/399: mknod d3/d9/da/dd/d31/d32/d13/d19/d2d/d4e/d5e/c8f 0 2024-04-12T20:56:10.440 INFO:tasks.workunit.client.0.smithi018.stdout:3/477: truncate d2/f78 408834 0 2024-04-12T20:56:10.441 INFO:tasks.workunit.client.0.smithi018.stdout:0/454: dread da/d15/d17/d21/d36/f7c [0,4194304] 0 2024-04-12T20:56:10.444 INFO:tasks.workunit.client.0.smithi018.stdout:6/674: dwrite d5/d6/d1b/d3b/f3d [0,4194304] 0 2024-04-12T20:56:10.450 INFO:tasks.workunit.client.0.smithi018.stdout:5/508: rmdir d0/d2/d33/d10 39 2024-04-12T20:56:10.454 INFO:tasks.workunit.client.0.smithi018.stdout:2/466: fsync d5/d8/d10/d19/d24/d39/f45 0 2024-04-12T20:56:10.454 INFO:tasks.workunit.client.0.smithi018.stdout:8/652: truncate db/d4e/d56/d59/d76/da5/fb0 798462 0 2024-04-12T20:56:10.456 INFO:tasks.workunit.client.0.smithi018.stdout:0/455: unlink da/d15/d3e/d3f/f7e 0 2024-04-12T20:56:10.457 INFO:tasks.workunit.client.0.smithi018.stdout:1/400: unlink d3/d9/da/dd/d31/l78 0 2024-04-12T20:56:10.462 INFO:tasks.workunit.client.0.smithi018.stdout:7/490: link d1/cc d1/db/d31/d45/d8c/d9d/ca2 0 2024-04-12T20:56:10.470 INFO:tasks.workunit.client.0.smithi018.stdout:8/653: write db/d3b/f3d [1967393,124274] 0 2024-04-12T20:56:10.478 INFO:tasks.workunit.client.0.smithi018.stdout:5/509: truncate d0/d2/d33/d46/d35/f4f 1975060 0 2024-04-12T20:56:10.478 INFO:tasks.workunit.client.0.smithi018.stdout:2/467: creat d5/d8/fa7 x:0 0 0 2024-04-12T20:56:10.484 INFO:tasks.workunit.client.0.smithi018.stdout:3/478: dwrite d2/d6/f84 [0,4194304] 0 2024-04-12T20:56:10.485 INFO:tasks.workunit.client.0.smithi018.stdout:6/675: creat d5/d15/d60/dbb/ffe x:0 0 0 2024-04-12T20:56:10.487 INFO:tasks.workunit.client.0.smithi018.stdout:2/468: chown d5/d7/d31/f4f 133404 1 2024-04-12T20:56:10.493 INFO:tasks.workunit.client.0.smithi018.stdout:4/569: dwrite d3/d34/fa9 [0,4194304] 0 2024-04-12T20:56:10.498 INFO:tasks.workunit.client.0.smithi018.stdout:4/570: stat d3/d34/d43/f62 0 2024-04-12T20:56:10.504 INFO:tasks.workunit.client.0.smithi018.stdout:2/469: write d5/d8/d33/f9a [288133,120451] 0 2024-04-12T20:56:10.508 INFO:tasks.workunit.client.0.smithi018.stdout:0/456: dwrite da/d15/d17/d21/d36/f4c [0,4194304] 0 2024-04-12T20:56:10.512 INFO:tasks.workunit.client.0.smithi018.stdout:3/479: unlink d2/d6/f24 0 2024-04-12T20:56:10.520 INFO:tasks.workunit.client.0.smithi018.stdout:4/571: write d3/d34/d43/f7b [580787,41053] 0 2024-04-12T20:56:10.520 INFO:tasks.workunit.client.0.smithi018.stdout:5/510: chown d0/d2/d33/d10/d26/f85 80584 1 2024-04-12T20:56:10.529 INFO:tasks.workunit.client.0.smithi018.stdout:6/676: creat d5/d6/d1b/d3b/d35/dda/fff x:0 0 0 2024-04-12T20:56:10.530 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:56:10 smithi186 ceph-mon[36597]: from='client.14644 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T20:56:10.532 INFO:tasks.workunit.client.0.smithi018.stdout:7/491: dwrite d1/f6e [0,4194304] 0 2024-04-12T20:56:10.537 INFO:tasks.workunit.client.0.smithi018.stdout:8/654: getdents db/d25/d26 0 2024-04-12T20:56:10.540 INFO:tasks.workunit.client.0.smithi018.stdout:3/480: symlink d2/d43/d61/l94 0 2024-04-12T20:56:10.540 INFO:tasks.workunit.client.0.smithi018.stdout:1/401: dwrite d3/d9/da/f4c [0,4194304] 0 2024-04-12T20:56:10.546 INFO:tasks.workunit.client.0.smithi018.stdout:4/572: fdatasync d3/d32/d33/d8e/fa1 0 2024-04-12T20:56:10.548 INFO:tasks.workunit.client.0.smithi018.stdout:6/677: dread - d5/df/f93 zero size 2024-04-12T20:56:10.549 INFO:tasks.workunit.client.0.smithi018.stdout:2/470: truncate d5/d8/d10/d43/f56 977642 0 2024-04-12T20:56:10.554 INFO:tasks.workunit.client.0.smithi018.stdout:4/573: dread - d3/d5d/fae zero size 2024-04-12T20:56:10.557 INFO:tasks.workunit.client.0.smithi018.stdout:0/457: mknod da/d15/d3e/d83/d8b/c99 0 2024-04-12T20:56:10.558 INFO:tasks.workunit.client.0.smithi018.stdout:8/655: read db/d25/d26/f2f [3532147,10881] 0 2024-04-12T20:56:10.564 INFO:tasks.workunit.client.0.smithi018.stdout:9/670: dwrite d3/d6/d9/f82 [0,4194304] 0 2024-04-12T20:56:10.564 INFO:tasks.workunit.client.0.smithi018.stdout:5/511: creat d0/d2/d11/d74/fad x:0 0 0 2024-04-12T20:56:10.564 INFO:tasks.workunit.client.0.smithi018.stdout:6/678: fdatasync d5/d6/d1b/ff1 0 2024-04-12T20:56:10.566 INFO:tasks.workunit.client.0.smithi018.stdout:5/512: stat d0/fa7 0 2024-04-12T20:56:10.570 INFO:tasks.workunit.client.0.smithi018.stdout:4/574: chown d3/d34/d3d/f65 166 1 2024-04-12T20:56:10.573 INFO:tasks.workunit.client.0.smithi018.stdout:2/471: creat d5/d7/d79/fa8 x:0 0 0 2024-04-12T20:56:10.573 INFO:tasks.workunit.client.0.smithi018.stdout:3/481: mkdir d2/d6/d95 0 2024-04-12T20:56:10.574 INFO:tasks.workunit.client.0.smithi018.stdout:1/402: truncate d3/d9/da/dd/d31/d32/d13/d19/f62 882832 0 2024-04-12T20:56:10.575 INFO:tasks.workunit.client.0.smithi018.stdout:5/513: fdatasync d0/d2/d11/d74/fad 0 2024-04-12T20:56:10.575 INFO:tasks.workunit.client.0.smithi018.stdout:4/575: dread - d3/d34/d43/fab zero size 2024-04-12T20:56:10.583 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:56:10.585+0000 7fb62ec00700 1 -- 172.21.15.18:0/2242704079 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb6280fdff0 msgr2=0x7fb6280fe460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:56:10.583 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:56:10.585+0000 7fb62ec00700 1 --2- 172.21.15.18:0/2242704079 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb6280fdff0 0x7fb6280fe460 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fb610004940 tx=0x7fb610009e30 comp rx=0 tx=0).stop 2024-04-12T20:56:10.584 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:56:10.586+0000 7fb62ec00700 1 -- 172.21.15.18:0/2242704079 shutdown_connections 2024-04-12T20:56:10.584 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:56:10.586+0000 7fb62ec00700 1 --2- 172.21.15.18:0/2242704079 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb6280fdff0 0x7fb6280fe460 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:56:10.584 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:56:10.586+0000 7fb62ec00700 1 --2- 172.21.15.18:0/2242704079 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb6280fce80 0x7fb6280fd290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:56:10.584 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:56:10.586+0000 7fb62ec00700 1 -- 172.21.15.18:0/2242704079 >> 172.21.15.18:0/2242704079 conn(0x7fb6280f8540 msgr2=0x7fb6280fa970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T20:56:10.585 INFO:tasks.workunit.client.0.smithi018.stdout:0/458: mknod da/d15/d17/d21/d36/d47/c9a 0 2024-04-12T20:56:10.585 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:56:10.586+0000 7fb62ec00700 1 -- 172.21.15.18:0/2242704079 shutdown_connections 2024-04-12T20:56:10.586 INFO:tasks.workunit.client.0.smithi018.stdout:4/576: chown d3/d32/d33/f30 670969106 1 2024-04-12T20:56:10.587 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:56:10.587+0000 7fb62ec00700 1 -- 172.21.15.18:0/2242704079 wait complete. 2024-04-12T20:56:10.587 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:56:10.589+0000 7fb62ec00700 1 Processor -- start 2024-04-12T20:56:10.588 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:56:10.590+0000 7fb62ec00700 1 -- start start 2024-04-12T20:56:10.589 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:56:10.590+0000 7fb62ec00700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb6280fce80 0x7fb62810b810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T20:56:10.589 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:56:10.591+0000 7fb62ec00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb6280fdff0 0x7fb62810bd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T20:56:10.589 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:56:10.591+0000 7fb62ec00700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb62810c430 con 0x7fb6280fdff0 2024-04-12T20:56:10.589 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:56:10.591+0000 7fb62ec00700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb62810cbf0 con 0x7fb6280fce80 2024-04-12T20:56:10.589 INFO:tasks.workunit.client.0.smithi018.stdout:6/679: symlink d5/d15/d60/dbb/l100 0 2024-04-12T20:56:10.590 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:56:10.592+0000 7fb627e00700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb6280fce80 0x7fb62810b810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T20:56:10.590 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:56:10.592+0000 7fb627e00700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb6280fce80 0x7fb62810b810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.18:51274/0 (socket says 172.21.15.18:51274) 2024-04-12T20:56:10.590 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:56:10.592+0000 7fb627e00700 1 -- 172.21.15.18:0/3484154226 learned_addr learned my addr 172.21.15.18:0/3484154226 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T20:56:10.591 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:56:10.593+0000 7fb627e00700 1 -- 172.21.15.18:0/3484154226 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb6280fdff0 msgr2=0x7fb62810bd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:56:10.591 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:56:10.593+0000 7fb627e00700 1 --2- 172.21.15.18:0/3484154226 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb6280fdff0 0x7fb62810bd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:56:10.591 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:56:10.593+0000 7fb627e00700 1 -- 172.21.15.18:0/3484154226 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb610005040 con 0x7fb6280fce80 2024-04-12T20:56:10.593 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:56:10.595+0000 7fb627e00700 1 --2- 172.21.15.18:0/3484154226 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb6280fce80 0x7fb62810b810 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fb61800bbb0 tx=0x7fb618009840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T20:56:10.594 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:56:10.595+0000 7fb624c00700 1 -- 172.21.15.18:0/3484154226 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb6180092e0 con 0x7fb6280fce80 2024-04-12T20:56:10.594 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:56:10.595+0000 7fb62ec00700 1 -- 172.21.15.18:0/3484154226 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb62810ced0 con 0x7fb6280fce80 2024-04-12T20:56:10.595 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:56:10.595+0000 7fb62ec00700 1 -- 172.21.15.18:0/3484154226 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb62810d490 con 0x7fb6280fce80 2024-04-12T20:56:10.595 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:56:10.597+0000 7fb624c00700 1 -- 172.21.15.18:0/3484154226 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb61800cbd0 con 0x7fb6280fce80 2024-04-12T20:56:10.595 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:56:10.597+0000 7fb624c00700 1 -- 172.21.15.18:0/3484154226 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb618007400 con 0x7fb6280fce80 2024-04-12T20:56:10.597 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:56:10.598+0000 7fb62ec00700 1 -- 172.21.15.18:0/3484154226 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb6040052b0 con 0x7fb6280fce80 2024-04-12T20:56:10.600 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:56:10.601+0000 7fb624c00700 1 -- 172.21.15.18:0/3484154226 <== mon.1 v2:172.21.15.186:3300/0 4 ==== mgrmap(e 24) v1 ==== 89644+0+0 (secure 0 0 0) 0x7fb618012070 con 0x7fb6280fce80 2024-04-12T20:56:10.600 INFO:tasks.workunit.client.0.smithi018.stdout:2/472: rmdir d5/d8/d25 39 2024-04-12T20:56:10.601 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:56:10.603+0000 7fb624c00700 1 --2- 172.21.15.18:0/3484154226 >> [v2:172.21.15.186:6828/2316349902,v1:172.21.15.186:6829/2316349902] conn(0x7fb60c06bdc0 0x7fb60c06e270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T20:56:10.601 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:56:10.603+0000 7fb624c00700 1 -- 172.21.15.18:0/3484154226 <== mon.1 v2:172.21.15.186:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5808+0+0 (secure 0 0 0) 0x7fb61808a760 con 0x7fb6280fce80 2024-04-12T20:56:10.602 INFO:tasks.workunit.client.0.smithi018.stdout:1/403: readlink d3/d9/da/dd/d31/d44/l6a 0 2024-04-12T20:56:10.603 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:56:10.604+0000 7fb61f400700 1 --2- 172.21.15.18:0/3484154226 >> [v2:172.21.15.186:6828/2316349902,v1:172.21.15.186:6829/2316349902] conn(0x7fb60c06bdc0 0x7fb60c06e270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T20:56:10.603 INFO:tasks.workunit.client.0.smithi018.stdout:6/680: creat d5/d6/d1b/d3b/d8e/f101 x:0 0 0 2024-04-12T20:56:10.604 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:56:10.605+0000 7fb61f400700 1 --2- 172.21.15.18:0/3484154226 >> [v2:172.21.15.186:6828/2316349902,v1:172.21.15.186:6829/2316349902] conn(0x7fb60c06bdc0 0x7fb60c06e270 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fb610004f90 tx=0x7fb610009e30 comp rx=0 tx=0).ready entity=mgr.24359 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T20:56:10.604 INFO:tasks.workunit.client.0.smithi018.stdout:7/492: truncate d1/db/d31/d45/d8c/d9d/d69/f7e 2840215 0 2024-04-12T20:56:10.606 INFO:tasks.workunit.client.0.smithi018.stdout:2/473: dread d5/d12/f58 [0,4194304] 0 2024-04-12T20:56:10.606 INFO:tasks.workunit.client.0.smithi018.stdout:4/577: mkdir d3/d32/d33/db6 0 2024-04-12T20:56:10.607 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:56:10.609+0000 7fb624c00700 1 -- 172.21.15.18:0/3484154226 <== mon.1 v2:172.21.15.186:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186454 (secure 0 0 0) 0x7fb618053e30 con 0x7fb6280fce80 2024-04-12T20:56:10.610 INFO:tasks.workunit.client.0.smithi018.stdout:2/474: fdatasync d5/d8/d10/f48 0 2024-04-12T20:56:10.611 INFO:tasks.workunit.client.0.smithi018.stdout:5/514: mkdir d0/d2/d33/d46/da2/da3/dae 0 2024-04-12T20:56:10.622 INFO:tasks.workunit.client.0.smithi018.stdout:0/459: rename da/d15/d17/d21/l62 to da/d15/d4d/d6b/l9b 0 2024-04-12T20:56:10.628 INFO:tasks.workunit.client.0.smithi018.stdout:8/656: dwrite f7 [4194304,4194304] 0 2024-04-12T20:56:10.632 INFO:tasks.workunit.client.0.smithi018.stdout:6/681: truncate d5/d6/d1b/d3b/fdc 715093 0 2024-04-12T20:56:10.635 INFO:tasks.workunit.client.0.smithi018.stdout:0/460: chown da/d15/d17/d5c/f64 27682 1 2024-04-12T20:56:10.637 INFO:tasks.workunit.client.0.smithi018.stdout:8/657: fsync db/d25/f52 0 2024-04-12T20:56:10.639 INFO:tasks.workunit.client.0.smithi018.stdout:5/515: rmdir d0/d2 39 2024-04-12T20:56:10.640 INFO:tasks.workunit.client.0.smithi018.stdout:6/682: chown d5/d6 105766846 1 2024-04-12T20:56:10.650 INFO:tasks.workunit.client.0.smithi018.stdout:4/578: mkdir d3/db/db7 0 2024-04-12T20:56:10.651 INFO:tasks.workunit.client.0.smithi018.stdout:1/404: dwrite d3/d9/da/dd/f3f [0,4194304] 0 2024-04-12T20:56:10.652 INFO:tasks.workunit.client.0.smithi018.stdout:9/671: dwrite d3/d6/d9/dc/d5a/d79/d7e/d9b/d21/d41/d78/f98 [0,4194304] 0 2024-04-12T20:56:10.658 INFO:tasks.workunit.client.0.smithi018.stdout:4/579: dread - d3/d34/d3d/f57 zero size 2024-04-12T20:56:10.662 INFO:tasks.workunit.client.0.smithi018.stdout:7/493: dwrite d1/f2f [0,4194304] 0 2024-04-12T20:56:10.668 INFO:tasks.workunit.client.0.smithi018.stdout:3/482: getdents d2 0 2024-04-12T20:56:10.672 INFO:tasks.workunit.client.0.smithi018.stdout:4/580: dread - d3/d34/d43/f76 zero size 2024-04-12T20:56:10.674 INFO:tasks.workunit.client.0.smithi018.stdout:7/494: symlink d1/db/la3 0 2024-04-12T20:56:10.675 INFO:tasks.workunit.client.0.smithi018.stdout:5/516: rmdir d0/d2/d33/d46/d1e 39 2024-04-12T20:56:10.678 INFO:tasks.workunit.client.0.smithi018.stdout:7/495: read d1/db/f9f [109759,107065] 0 2024-04-12T20:56:10.695 INFO:tasks.workunit.client.0.smithi018.stdout:4/581: mknod d3/d32/d66/cb8 0 2024-04-12T20:56:10.696 INFO:tasks.workunit.client.0.smithi018.stdout:3/483: write d2/d6/d30/d31/d38/d3a/d48/f51 [5916037,3513] 0 2024-04-12T20:56:10.696 INFO:tasks.workunit.client.0.smithi018.stdout:2/475: link d5/d8/d10/d19/d24/c81 d5/d8/d25/ca9 0 2024-04-12T20:56:10.701 INFO:tasks.workunit.client.0.smithi018.stdout:8/658: rename db/d3b/f4d to db/d25/d26/d27/fcf 0 2024-04-12T20:56:10.702 INFO:tasks.workunit.client.0.smithi018.stdout:7/496: write d1/db/d31/d45/d8c/d9d/f73 [1324821,83469] 0 2024-04-12T20:56:10.705 INFO:tasks.workunit.client.0.smithi018.stdout:1/405: creat d3/d9/da/d14/d73/f90 x:0 0 0 2024-04-12T20:56:10.706 INFO:tasks.workunit.client.0.smithi018.stdout:0/461: link da/d15/d17/d21/d36/l44 da/d15/d4d/d6b/l9c 0 2024-04-12T20:56:10.707 INFO:tasks.workunit.client.0.smithi018.stdout:6/683: dwrite d5/df/d19/d2b/f95 [0,4194304] 0 2024-04-12T20:56:10.712 INFO:tasks.workunit.client.0.smithi018.stdout:2/476: unlink d5/d12/d54/fa0 0 2024-04-12T20:56:10.712 INFO:tasks.workunit.client.0.smithi018.stdout:7/497: dread d1/db/d31/d45/d53/f7d [0,4194304] 0 2024-04-12T20:56:10.719 INFO:tasks.workunit.client.0.smithi018.stdout:2/477: readlink d5/d8/d10/d19/d24/d39/l6b 0 2024-04-12T20:56:10.721 INFO:tasks.workunit.client.0.smithi018.stdout:2/478: fdatasync d5/d8/fa7 0 2024-04-12T20:56:10.727 INFO:tasks.workunit.client.0.smithi018.stdout:3/484: creat d2/d6/d30/d31/d7c/f96 x:0 0 0 2024-04-12T20:56:10.734 INFO:tasks.workunit.client.0.smithi018.stdout:3/485: write d2/d6/d30/f37 [1811322,77793] 0 2024-04-12T20:56:10.737 INFO:tasks.workunit.client.0.smithi018.stdout:1/406: rename d3/f10 to d3/d9/da/dd/d31/d32/d13/d19/d2d/d4e/d58/d61/d71/f91 0 2024-04-12T20:56:10.738 INFO:tasks.workunit.client.0.smithi018.stdout:0/462: mknod da/c9d 0 2024-04-12T20:56:10.738 INFO:tasks.workunit.client.0.smithi018.stdout:6/684: dwrite d5/d6/d1b/d3b/d35/dda/fff [0,4194304] 0 2024-04-12T20:56:10.740 INFO:tasks.workunit.client.0.smithi018.stdout:7/498: symlink d1/d86/la4 0 2024-04-12T20:56:10.742 INFO:tasks.workunit.client.0.smithi018.stdout:0/463: stat da/d15/d17/d21 0 2024-04-12T20:56:10.742 INFO:tasks.workunit.client.0.smithi018.stdout:5/517: getdents d0/d2b 0 2024-04-12T20:56:10.749 INFO:tasks.workunit.client.0.smithi018.stdout:2/479: dread d5/d12/d54/f53 [0,4194304] 0 2024-04-12T20:56:10.750 INFO:tasks.workunit.client.0.smithi018.stdout:2/480: readlink d5/d8/d10/d19/d24/l6a 0 2024-04-12T20:56:10.752 INFO:tasks.workunit.client.0.smithi018.stdout:6/685: link d5/df/d9b/fc4 d5/d6/d1b/d3b/d6a/dd8/f102 0 2024-04-12T20:56:10.755 INFO:tasks.workunit.client.0.smithi018.stdout:3/486: creat d2/d4d/d88/f97 x:0 0 0 2024-04-12T20:56:10.756 INFO:tasks.workunit.client.0.smithi018.stdout:6/686: fsync d5/d6/d1b/fec 0 2024-04-12T20:56:10.763 INFO:tasks.workunit.client.0.smithi018.stdout:6/687: symlink d5/df/d19/d27/dc8/l103 0 2024-04-12T20:56:10.763 INFO:tasks.workunit.client.0.smithi018.stdout:8/659: rename db/d25/d26/d27/c46 to db/d25/d26/d27/cd0 0 2024-04-12T20:56:10.765 INFO:tasks.workunit.client.0.smithi018.stdout:5/518: unlink d0/ca1 0 2024-04-12T20:56:10.770 INFO:tasks.workunit.client.0.smithi018.stdout:6/688: mkdir d5/df/d19/dc5/d104 0 2024-04-12T20:56:10.771 INFO:tasks.workunit.client.0.smithi018.stdout:0/464: mknod da/d15/d4d/d6b/c9e 0 2024-04-12T20:56:10.772 INFO:tasks.workunit.client.0.smithi018.stdout:6/689: write d5/fd [706295,75346] 0 2024-04-12T20:56:10.777 INFO:tasks.workunit.client.0.smithi018.stdout:1/407: write d3/d9/f3c [747020,113052] 0 2024-04-12T20:56:10.779 INFO:tasks.workunit.client.0.smithi018.stdout:8/660: unlink db/d4e/d56/d59/d76/da5/ca6 0 2024-04-12T20:56:10.783 INFO:tasks.workunit.client.0.smithi018.stdout:6/690: write d5/d6/d1b/d3b/f52 [4167573,86240] 0 2024-04-12T20:56:10.784 INFO:tasks.workunit.client.0.smithi018.stdout:1/408: stat d3/d9/da/dd/d31/d32/d13/d19/d2d/d4e/d7a/l8d 0 2024-04-12T20:56:10.790 INFO:tasks.workunit.client.0.smithi018.stdout:1/409: readlink d3/d9/da/dd/d31/d32/l2e 0 2024-04-12T20:56:10.792 INFO:tasks.workunit.client.0.smithi018.stdout:2/481: link d5/d8/d10/d19/d7c/f9f d5/d12/d54/d3e/faa 0 2024-04-12T20:56:10.794 INFO:tasks.workunit.client.0.smithi018.stdout:7/499: creat d1/db/d31/d3a/fa5 x:0 0 0 2024-04-12T20:56:10.795 INFO:tasks.workunit.client.0.smithi018.stdout:8/661: unlink db/d95/dc0/lce 0 2024-04-12T20:56:10.796 INFO:tasks.workunit.client.0.smithi018.stdout:6/691: dread d5/d6/d1b/fc9 [0,4194304] 0 2024-04-12T20:56:10.797 INFO:tasks.workunit.client.0.smithi018.stdout:8/662: readlink db/d4e/d56/l87 0 2024-04-12T20:56:10.799 INFO:tasks.workunit.client.0.smithi018.stdout:2/482: mkdir d5/d8/d10/d19/d24/d39/dab 0 2024-04-12T20:56:10.800 INFO:tasks.workunit.client.0.smithi018.stdout:0/465: dread da/d15/d17/d21/d36/f4c [0,4194304] 0 2024-04-12T20:56:10.801 INFO:tasks.workunit.client.0.smithi018.stdout:6/692: write d5/d6/d1b/d3b/fd7 [189767,39654] 0 2024-04-12T20:56:10.802 INFO:tasks.workunit.client.0.smithi018.stdout:1/410: truncate d3/d9/da/dd/f12 2075408 0 2024-04-12T20:56:10.810 INFO:tasks.workunit.client.0.smithi018.stdout:6/693: mkdir d5/d6/d1b/d3b/d35/d4f/df3/d105 0 2024-04-12T20:56:10.810 INFO:tasks.workunit.client.0.smithi018.stdout:1/411: chown d3/d9/da/dd/d31/d32/d13/d19 12487752 1 2024-04-12T20:56:10.811 INFO:tasks.workunit.client.0.smithi018.stdout:2/483: dread d5/d7/f5c [0,4194304] 0 2024-04-12T20:56:10.813 INFO:tasks.workunit.client.0.smithi018.stdout:0/466: truncate da/d15/d3e/f73 887144 0 2024-04-12T20:56:10.817 INFO:tasks.workunit.client.0.smithi018.stdout:2/484: chown d5/d8/d10/d43/fa5 3836171 1 2024-04-12T20:56:10.818 INFO:tasks.workunit.client.0.smithi018.stdout:5/519: dwrite d0/d2/d33/d10/d26/f4a [0,4194304] 0 2024-04-12T20:56:10.819 INFO:tasks.workunit.client.0.smithi018.stdout:0/467: truncate da/f75 1089133 0 2024-04-12T20:56:10.821 INFO:tasks.workunit.client.0.smithi018.stdout:2/485: dread - d5/d12/f96 zero size 2024-04-12T20:56:10.825 INFO:tasks.workunit.client.0.smithi018.stdout:7/500: dwrite d1/db/d31/d3a/d42/d4c/f51 [0,4194304] 0 2024-04-12T20:56:10.830 INFO:tasks.workunit.client.0.smithi018.stdout:1/412: fsync d3/d9/da/dd/d31/d32/d13/d19/d2d/d4e/f8c 0 2024-04-12T20:56:10.830 INFO:tasks.workunit.client.0.smithi018.stdout:5/520: stat d0/d2/d33/d46/c96 0 2024-04-12T20:56:10.831 INFO:tasks.workunit.client.0.smithi018.stdout:8/663: dwrite db/d4e/d9a/d69/d6c/fa7 [0,4194304] 0 2024-04-12T20:56:10.834 INFO:tasks.workunit.client.0.smithi018.stdout:0/468: dread - da/d15/d17/d21/f68 zero size 2024-04-12T20:56:10.836 INFO:tasks.workunit.client.0.smithi018.stdout:7/501: stat d1/d6/l26 0 2024-04-12T20:56:10.840 INFO:tasks.workunit.client.0.smithi018.stdout:2/486: write d5/d8/d10/d43/fa5 [764425,53772] 0 2024-04-12T20:56:10.849 INFO:tasks.workunit.client.0.smithi018.stdout:6/694: dwrite d5/d6/d90/ffb [0,4194304] 0 2024-04-12T20:56:10.857 INFO:tasks.workunit.client.0.smithi018.stdout:6/695: fsync d5/d6/d1b/d3b/fd7 0 2024-04-12T20:56:10.859 INFO:tasks.workunit.client.0.smithi018.stdout:5/521: rename d0/d2/d33/d46/d1e/d55/f99 to d0/d2/d33/d46/d18/faf 0 2024-04-12T20:56:10.867 INFO:tasks.workunit.client.0.smithi018.stdout:2/487: creat d5/d7/d63/fac x:0 0 0 2024-04-12T20:56:10.869 INFO:tasks.workunit.client.0.smithi018.stdout:6/696: creat d5/df/d3f/dca/f106 x:0 0 0 2024-04-12T20:56:10.873 INFO:tasks.workunit.client.0.smithi018.stdout:2/488: read d5/d12/d54/f53 [236055,62049] 0 2024-04-12T20:56:10.873 INFO:tasks.workunit.client.0.smithi018.stdout:8/664: truncate db/d25/d26/d27/f33 1855845 0 2024-04-12T20:56:10.876 INFO:tasks.workunit.client.0.smithi018.stdout:5/522: symlink d0/d2/d11/d37/lb0 0 2024-04-12T20:56:10.878 INFO:tasks.workunit.client.0.smithi018.stdout:2/489: dread - d5/d12/d54/d3e/faa zero size 2024-04-12T20:56:10.884 INFO:tasks.workunit.client.0.smithi018.stdout:0/469: write da/f22 [1678729,20212] 0 2024-04-12T20:56:10.887 INFO:tasks.workunit.client.0.smithi018.stdout:1/413: dwrite d3/d9/da/dd/d31/d32/d13/d19/d2d/f4d [0,4194304] 0 2024-04-12T20:56:10.897 INFO:tasks.workunit.client.0.smithi018.stdout:7/502: dread d1/d6/f52 [0,4194304] 0 2024-04-12T20:56:10.899 INFO:tasks.workunit.client.0.smithi018.stdout:2/490: symlink d5/d12/lad 0 2024-04-12T20:56:10.900 INFO:tasks.workunit.client.0.smithi018.stdout:8/665: unlink db/d4e/d9a/d69/d6c/fa7 0 2024-04-12T20:56:10.902 INFO:tasks.workunit.client.0.smithi018.stdout:1/414: creat d3/d9/da/dd/d31/d32/d13/d19/d2d/f92 x:0 0 0 2024-04-12T20:56:10.904 INFO:tasks.workunit.client.0.smithi018.stdout:0/470: symlink da/d15/d17/d21/d94/l9f 0 2024-04-12T20:56:10.904 INFO:tasks.workunit.client.0.smithi018.stdout:2/491: chown d5/d8/d10/d19/l2a 376232 1 2024-04-12T20:56:10.904 INFO:tasks.workunit.client.0.smithi018.stdout:7/503: dread - d1/db/d31/d45/d8c/d9d/d69/f7f zero size 2024-04-12T20:56:10.908 INFO:tasks.workunit.client.0.smithi018.stdout:5/523: write d0/d5b/d84/f31 [536312,128837] 0 2024-04-12T20:56:10.914 INFO:tasks.workunit.client.0.smithi018.stdout:2/492: chown d5/d8/d10/d19/d24/d39/l83 174121593 1 2024-04-12T20:56:10.917 INFO:tasks.workunit.client.0.smithi018.stdout:6/697: dwrite d5/f8d [0,4194304] 0 2024-04-12T20:56:10.919 INFO:tasks.workunit.client.0.smithi018.stdout:2/493: dread - d5/d7/d79/f8c zero size 2024-04-12T20:56:10.920 INFO:tasks.workunit.client.0.smithi018.stdout:8/666: creat db/d4e/d9a/d69/fd1 x:0 0 0 2024-04-12T20:56:10.920 INFO:tasks.workunit.client.0.smithi018.stdout:1/415: symlink d3/d9/da/l93 0 2024-04-12T20:56:10.924 INFO:tasks.workunit.client.0.smithi018.stdout:8/667: read db/d25/d26/f53 [2095007,113340] 0 2024-04-12T20:56:10.929 INFO:tasks.workunit.client.0.smithi018.stdout:0/471: readlink da/d15/d17/d21/d36/l92 0 2024-04-12T20:56:10.930 INFO:tasks.workunit.client.0.smithi018.stdout:6/698: creat d5/d6/d1b/d3b/d35/d4f/df3/d105/f107 x:0 0 0 2024-04-12T20:56:10.934 INFO:tasks.workunit.client.0.smithi018.stdout:0/472: chown da/d15/d17/d21/f68 671463585 1 2024-04-12T20:56:10.943 INFO:tasks.workunit.client.0.smithi018.stdout:6/699: creat d5/d6/f108 x:0 0 0 2024-04-12T20:56:10.948 INFO:tasks.workunit.client.0.smithi018.stdout:7/504: rename d1/db/d31/f58 to d1/db/d31/d3a/d42/d4c/d6c/fa6 0 2024-04-12T20:56:10.948 INFO:tasks.workunit.client.0.smithi018.stdout:6/700: fsync d5/df/d19/f6c 0 2024-04-12T20:56:10.948 INFO:tasks.workunit.client.0.smithi018.stdout:2/494: creat d5/d8/d10/d19/fae x:0 0 0 2024-04-12T20:56:10.953 INFO:tasks.workunit.client.0.smithi018.stdout:1/416: dwrite d3/d9/da/dd/d31/d32/d13/d19/d2d/f8b [0,4194304] 0 2024-04-12T20:56:10.958 INFO:tasks.workunit.client.0.smithi018.stdout:1/417: write d3/d9/da/dd/d31/d32/d13/d19/f6e [923619,51242] 0 2024-04-12T20:56:10.962 INFO:tasks.workunit.client.0.smithi018.stdout:0/473: symlink da/d15/d17/d21/d36/d6e/la0 0 2024-04-12T20:56:10.964 INFO:tasks.workunit.client.0.smithi018.stdout:8/668: symlink db/d4e/d56/d59/d76/da5/d45/da9/ld2 0 2024-04-12T20:56:10.971 INFO:tasks.workunit.client.0.smithi018.stdout:0/474: fsync da/d15/d17/d21/d27/f4f 0 2024-04-12T20:56:10.975 INFO:tasks.workunit.client.0.smithi018.stdout:8/669: creat db/d4e/d56/d59/fd3 x:0 0 0 2024-04-12T20:56:10.984 INFO:tasks.workunit.client.0.smithi018.stdout:0/475: symlink da/d15/d17/d21/d70/la1 0 2024-04-12T20:56:10.987 INFO:tasks.workunit.client.0.smithi018.stdout:8/670: mknod db/d95/dc0/cd4 0 2024-04-12T20:56:10.988 INFO:tasks.workunit.client.0.smithi018.stdout:7/505: link d1/d6/l17 d1/db/d31/d45/la7 0 2024-04-12T20:56:10.990 INFO:tasks.workunit.client.0.smithi018.stdout:1/418: dwrite d3/d9/da/dd/d31/d32/d13/d19/d2d/f4d [8388608,4194304] 0 2024-04-12T20:56:10.991 INFO:tasks.workunit.client.0.smithi018.stdout:0/476: unlink da/d15/l30 0 2024-04-12T20:56:10.991 INFO:tasks.workunit.client.0.smithi018.stdout:7/506: rmdir d1/db 39 2024-04-12T20:56:10.995 INFO:tasks.workunit.client.0.smithi018.stdout:7/507: fsync d1/d6/f60 0 2024-04-12T20:56:10.998 INFO:tasks.workunit.client.0.smithi018.stdout:5/524: truncate d0/d2/d11/f4b 3947779 0 2024-04-12T20:56:11.000 INFO:tasks.workunit.client.0.smithi018.stdout:6/701: dread d5/f83 [0,4194304] 0 2024-04-12T20:56:11.021 INFO:tasks.workunit.client.0.smithi018.stdout:0/477: dwrite da/f25 [0,4194304] 0 2024-04-12T20:56:11.038 INFO:tasks.workunit.client.0.smithi018.stdout:7/508: dread d1/db/d31/d45/d8c/d9d/f73 [0,4194304] 0 2024-04-12T20:56:11.047 INFO:tasks.workunit.client.0.smithi018.stdout:6/702: dwrite d5/df/d19/f25 [0,4194304] 0 2024-04-12T20:56:11.047 INFO:tasks.workunit.client.0.smithi018.stdout:5/525: dwrite d0/d2/d33/d10/d26/f4a [0,4194304] 0 2024-04-12T20:56:11.059 INFO:tasks.workunit.client.0.smithi018.stdout:2/495: dwrite d5/d8/d10/d19/d24/f36 [0,4194304] 0 2024-04-12T20:56:11.060 INFO:tasks.workunit.client.0.smithi018.stdout:1/419: dwrite d3/d9/da/dd/d31/f36 [0,4194304] 0 2024-04-12T20:56:11.061 INFO:tasks.workunit.client.0.smithi018.stdout:6/703: fdatasync d5/d6/d1b/d3b/f94 0 2024-04-12T20:56:11.062 INFO:tasks.workunit.client.0.smithi018.stdout:7/509: fdatasync d1/d6/f24 0 2024-04-12T20:56:11.062 INFO:tasks.workunit.client.0.smithi018.stdout:8/671: dread db/d4e/d56/d59/d76/da5/d45/f65 [0,4194304] 0 2024-04-12T20:56:11.066 INFO:tasks.workunit.client.0.smithi018.stdout:8/672: readlink l4 0 2024-04-12T20:56:11.067 INFO:tasks.workunit.client.0.smithi018.stdout:7/510: chown d1/db/d31/d30/l50 30167 1 2024-04-12T20:56:11.075 INFO:tasks.workunit.client.0.smithi018.stdout:1/420: read d3/d9/da/dd/d31/d32/d33/f6d [2390587,63595] 0 2024-04-12T20:56:11.075 INFO:tasks.workunit.client.0.smithi018.stdout:5/526: write d0/d2/d33/d46/d1e/d2e/d7f/f98 [1318756,24454] 0 2024-04-12T20:56:11.076 INFO:tasks.workunit.client.0.smithi018.stdout:8/673: chown db/f15 72 1 2024-04-12T20:56:11.078 INFO:tasks.workunit.client.0.smithi018.stdout:6/704: unlink d5/df/l13 0 2024-04-12T20:56:11.079 INFO:tasks.workunit.client.0.smithi018.stdout:7/511: unlink d1/d6/f27 0 2024-04-12T20:56:11.085 INFO:tasks.workunit.client.0.smithi018.stdout:5/527: rmdir d0/d2/d33 39 2024-04-12T20:56:11.092 INFO:tasks.workunit.client.0.smithi018.stdout:1/421: fsync d3/d9/da/dd/d31/d32/d33/f6d 0 2024-04-12T20:56:11.092 INFO:tasks.workunit.client.0.smithi018.stdout:8/674: truncate db/d4e/d56/d59/d76/da5/d45/d86/fa8 554618 0 2024-04-12T20:56:11.093 INFO:tasks.workunit.client.0.smithi018.stdout:2/496: rename d5/d8/d10/d19/d24/d6c to d5/d8/d10/d19/daf 0 2024-04-12T20:56:11.096 INFO:tasks.workunit.client.0.smithi018.stdout:5/528: creat d0/d5b/d84/d83/fb1 x:0 0 0 2024-04-12T20:56:11.100 INFO:tasks.workunit.client.0.smithi018.stdout:2/497: unlink d5/d7/c68 0 2024-04-12T20:56:11.102 INFO:tasks.workunit.client.0.smithi018.stdout:7/512: dread d1/f2f [0,4194304] 0 2024-04-12T20:56:11.102 INFO:tasks.workunit.client.0.smithi018.stdout:2/498: stat d5/d12/lad 0 2024-04-12T20:56:11.105 INFO:tasks.workunit.client.0.smithi018.stdout:2/499: write d5/d7/d31/f4f [2289081,24416] 0 2024-04-12T20:56:11.108 INFO:tasks.workunit.client.0.smithi018.stdout:7/513: mknod d1/db/d31/d3a/d42/d4c/d6c/ca8 0 2024-04-12T20:56:11.116 INFO:tasks.workunit.client.0.smithi018.stdout:5/529: rename d0/d2/d11/f29 to d0/d2/d33/d10/d2f/d8f/fb2 0 2024-04-12T20:56:11.117 INFO:tasks.workunit.client.0.smithi018.stdout:2/500: rmdir d5/d7/d79 39 2024-04-12T20:56:11.118 INFO:tasks.workunit.client.0.smithi018.stdout:5/530: readlink d0/l41 0 2024-04-12T20:56:11.119 INFO:tasks.workunit.client.0.smithi018.stdout:1/422: dwrite d3/d9/da/dd/d31/d32/f34 [0,4194304] 0 2024-04-12T20:56:11.123 INFO:tasks.workunit.client.0.smithi018.stdout:1/423: fsync d3/d9/f3c 0 2024-04-12T20:56:11.127 INFO:tasks.workunit.client.0.smithi018.stdout:6/705: dread d5/df/d19/d2b/fa5 [0,4194304] 0 2024-04-12T20:56:11.130 INFO:tasks.workunit.client.0.smithi018.stdout:7/514: dread d1/db/d31/d45/d53/f56 [0,4194304] 0 2024-04-12T20:56:11.131 INFO:tasks.workunit.client.0.smithi018.stdout:6/706: write d5/df/f87 [2409905,93435] 0 2024-04-12T20:56:11.135 INFO:tasks.workunit.client.0.smithi018.stdout:1/424: symlink d3/d9/da/dd/d31/d32/d13/d19/d2d/d4e/d58/d61/d71/l94 0 2024-04-12T20:56:11.146 INFO:tasks.workunit.client.0.smithi018.stdout:7/515: mkdir d1/db/d31/d45/d8c/d9d/d70/d96/d5f/da9 0 2024-04-12T20:56:11.148 INFO:tasks.workunit.client.0.smithi018.stdout:6/707: mknod d5/df/d19/d43/c109 0 2024-04-12T20:56:11.153 INFO:tasks.workunit.client.0.smithi018.stdout:7/516: truncate d1/f6e 4846339 0 2024-04-12T20:56:11.153 INFO:tasks.workunit.client.0.smithi018.stdout:2/501: dwrite d5/d7/d28/f5f [0,4194304] 0 2024-04-12T20:56:11.156 INFO:tasks.workunit.client.0.smithi018.stdout:8/675: dwrite db/f68 [0,4194304] 0 2024-04-12T20:56:11.161 INFO:tasks.workunit.client.0.smithi018.stdout:8/676: chown db/d4e/f6a 7015 1 2024-04-12T20:56:11.163 INFO:tasks.workunit.client.0.smithi018.stdout:7/517: rename d1/db/d31/d3a/l81 to d1/db/d31/d30/laa 0 2024-04-12T20:56:11.168 INFO:tasks.workunit.client.0.smithi018.stdout:8/677: creat db/d4e/d56/fd5 x:0 0 0 2024-04-12T20:56:11.171 INFO:tasks.workunit.client.0.smithi018.stdout:5/531: dwrite d0/d2/d33/d46/d1e/f6a [0,4194304] 0 2024-04-12T20:56:11.171 INFO:tasks.workunit.client.0.smithi018.stdout:6/708: dread d5/d6/d1b/d3b/f91 [0,4194304] 0 2024-04-12T20:56:11.174 INFO:tasks.workunit.client.0.smithi018.stdout:8/678: mkdir db/d95/dd6 0 2024-04-12T20:56:11.176 INFO:tasks.workunit.client.0.smithi018.stdout:5/532: write d0/d2/d33/d46/d18/f3b [4623660,100242] 0 2024-04-12T20:56:11.178 INFO:tasks.workunit.client.0.smithi018.stdout:2/502: getdents d5/d8/d10/d19/d7c 0 2024-04-12T20:56:11.179 INFO:tasks.workunit.client.0.smithi018.stdout:8/679: stat db/d4e/d73/la2 0 2024-04-12T20:56:11.184 INFO:tasks.workunit.client.0.smithi018.stdout:8/680: chown db/d25/d26/f53 63693 1 2024-04-12T20:56:11.192 INFO:tasks.workunit.client.0.smithi018.stdout:6/709: link d5/df/d3f/dc1/lcc d5/df/d3f/dca/l10a 0 2024-04-12T20:56:11.192 INFO:tasks.workunit.client.0.smithi018.stdout:7/518: dwrite d1/d6/f98 [0,4194304] 0 2024-04-12T20:56:11.195 INFO:tasks.workunit.client.0.smithi018.stdout:5/533: dread d0/d2/d33/d46/f1d [0,4194304] 0 2024-04-12T20:56:11.197 INFO:tasks.workunit.client.0.smithi018.stdout:7/519: write d1/db/d31/d3a/d42/f75 [3604476,57928] 0 2024-04-12T20:56:11.204 INFO:tasks.workunit.client.0.smithi018.stdout:6/710: mknod d5/df/d19/d2b/c10b 0 2024-04-12T20:56:11.204 INFO:tasks.workunit.client.0.smithi018.stdout:5/534: rmdir d0/d5b/d84 39 2024-04-12T20:56:11.207 INFO:tasks.workunit.client.0.smithi018.stdout:2/503: getdents d5/d8/d10/d19/d24 0 2024-04-12T20:56:11.207 INFO:tasks.workunit.client.0.smithi018.stdout:8/681: dread db/d4e/d56/d59/f82 [0,4194304] 0 2024-04-12T20:56:11.208 INFO:tasks.workunit.client.0.smithi018.stdout:6/711: mkdir d5/df/d31/d10c 0 2024-04-12T20:56:11.208 INFO:tasks.workunit.client.0.smithi018.stdout:5/535: readlink d0/d2/d33/d46/d35/l72 0 2024-04-12T20:56:11.211 INFO:tasks.workunit.client.0.smithi018.stdout:5/536: chown d0/d2/d33/d10/d2f 56372 1 2024-04-12T20:56:11.213 INFO:tasks.workunit.client.0.smithi018.stdout:7/520: mkdir d1/db/d31/d45/d8c/d9d/d70/dab 0 2024-04-12T20:56:11.217 INFO:tasks.workunit.client.0.smithi018.stdout:8/682: creat db/d72/fd7 x:0 0 0 2024-04-12T20:56:11.219 INFO:tasks.workunit.client.0.smithi018.stdout:7/521: write d1/d6/f60 [3600477,102459] 0 2024-04-12T20:56:11.221 INFO:tasks.workunit.client.0.smithi018.stdout:6/712: link d5/d6/d1b/d8c/dc0/cf5 d5/df/d19/c10d 0 2024-04-12T20:56:11.228 INFO:tasks.workunit.client.0.smithi018.stdout:6/713: truncate d5/f57 785736 0 2024-04-12T20:56:11.232 INFO:tasks.workunit.client.0.smithi018.stdout:5/537: dread d0/d2/d33/d46/d1e/d2e/f75 [0,4194304] 0 2024-04-12T20:56:11.232 INFO:tasks.workunit.client.0.smithi018.stdout:8/683: creat db/d25/d26/d27/fd8 x:0 0 0 2024-04-12T20:56:11.238 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:56:11.236+0000 7fb62ec00700 1 -- 172.21.15.18:0/3484154226 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fb604005830 con 0x7fb6280fce80 2024-04-12T20:56:11.238 INFO:tasks.workunit.client.0.smithi018.stdout:6/714: rename d5/df/d19/d2b/lb1 to d5/df/d19/d27/dc8/l10e 0 2024-04-12T20:56:11.238 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:56:11.238+0000 7fb624c00700 1 -- 172.21.15.18:0/3484154226 <== mon.1 v2:172.21.15.186:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+846 (secure 0 0 0) 0x7fb6180574f0 con 0x7fb6280fce80 2024-04-12T20:56:11.239 INFO:teuthology.orchestra.run.smithi018.stdout:{ 2024-04-12T20:56:11.239 INFO:teuthology.orchestra.run.smithi018.stdout: "mon": { 2024-04-12T20:56:11.239 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 17.2.7-827-gd3b5060a (d3b5060a17449edfae9385cdf792afec109fca5d) quincy (stable)": 2 2024-04-12T20:56:11.239 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-12T20:56:11.239 INFO:teuthology.orchestra.run.smithi018.stdout: "mgr": { 2024-04-12T20:56:11.239 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 17.2.7-827-gd3b5060a (d3b5060a17449edfae9385cdf792afec109fca5d) quincy (stable)": 1, 2024-04-12T20:56:11.239 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 19.0.0-2879-g324baf8d (324baf8ddb91dea46cac50cb535c306115791355) squid (dev)": 1 2024-04-12T20:56:11.239 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-12T20:56:11.239 INFO:teuthology.orchestra.run.smithi018.stdout: "osd": { 2024-04-12T20:56:11.239 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 17.2.7-827-gd3b5060a (d3b5060a17449edfae9385cdf792afec109fca5d) quincy (stable)": 6 2024-04-12T20:56:11.239 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-12T20:56:11.239 INFO:teuthology.orchestra.run.smithi018.stdout: "mds": { 2024-04-12T20:56:11.239 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 17.2.7-827-gd3b5060a (d3b5060a17449edfae9385cdf792afec109fca5d) quincy (stable)": 4 2024-04-12T20:56:11.240 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-12T20:56:11.240 INFO:teuthology.orchestra.run.smithi018.stdout: "overall": { 2024-04-12T20:56:11.240 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 17.2.7-827-gd3b5060a (d3b5060a17449edfae9385cdf792afec109fca5d) quincy (stable)": 13, 2024-04-12T20:56:11.240 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 19.0.0-2879-g324baf8d (324baf8ddb91dea46cac50cb535c306115791355) squid (dev)": 1 2024-04-12T20:56:11.240 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-04-12T20:56:11.240 INFO:teuthology.orchestra.run.smithi018.stdout:} 2024-04-12T20:56:11.240 INFO:tasks.workunit.client.0.smithi018.stdout:8/684: mkdir db/d4e/d73/dd9 0 2024-04-12T20:56:11.241 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:56:11.242+0000 7fb62ec00700 1 -- 172.21.15.18:0/3484154226 >> [v2:172.21.15.186:6828/2316349902,v1:172.21.15.186:6829/2316349902] conn(0x7fb60c06bdc0 msgr2=0x7fb60c06e270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:56:11.241 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:56:11.242+0000 7fb62ec00700 1 --2- 172.21.15.18:0/3484154226 >> [v2:172.21.15.186:6828/2316349902,v1:172.21.15.186:6829/2316349902] conn(0x7fb60c06bdc0 0x7fb60c06e270 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fb610004f90 tx=0x7fb610009e30 comp rx=0 tx=0).stop 2024-04-12T20:56:11.241 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:56:11.242+0000 7fb62ec00700 1 -- 172.21.15.18:0/3484154226 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb6280fce80 msgr2=0x7fb62810b810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:56:11.241 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:56:11.242+0000 7fb62ec00700 1 --2- 172.21.15.18:0/3484154226 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb6280fce80 0x7fb62810b810 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fb61800bbb0 tx=0x7fb618009840 comp rx=0 tx=0).stop 2024-04-12T20:56:11.241 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:56:11.242+0000 7fb62ec00700 1 -- 172.21.15.18:0/3484154226 shutdown_connections 2024-04-12T20:56:11.241 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:56:11.242+0000 7fb62ec00700 1 --2- 172.21.15.18:0/3484154226 >> [v2:172.21.15.186:6828/2316349902,v1:172.21.15.186:6829/2316349902] conn(0x7fb60c06bdc0 0x7fb60c06e270 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:56:11.242 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:56:11.242+0000 7fb62ec00700 1 --2- 172.21.15.18:0/3484154226 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb6280fce80 0x7fb62810b810 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:56:11.242 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:56:11.242+0000 7fb62ec00700 1 --2- 172.21.15.18:0/3484154226 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb6280fdff0 0x7fb62810bd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:56:11.242 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:56:11.242+0000 7fb62ec00700 1 -- 172.21.15.18:0/3484154226 >> 172.21.15.18:0/3484154226 conn(0x7fb6280f8540 msgr2=0x7fb6280f9a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T20:56:11.242 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:56:11.242+0000 7fb62ec00700 1 -- 172.21.15.18:0/3484154226 shutdown_connections 2024-04-12T20:56:11.242 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:56:11.243+0000 7fb62ec00700 1 -- 172.21.15.18:0/3484154226 wait complete. 2024-04-12T20:56:11.242 INFO:tasks.workunit.client.0.smithi018.stdout:5/538: mkdir d0/d2/d33/d46/d35/d4d/db3 0 2024-04-12T20:56:11.242 INFO:tasks.workunit.client.0.smithi018.stdout:2/504: dread d5/d7/d28/f59 [0,4194304] 0 2024-04-12T20:56:11.246 INFO:tasks.workunit.client.0.smithi018.stdout:8/685: chown db/d25/d26/l5a 1972693 1 2024-04-12T20:56:11.247 INFO:tasks.workunit.client.0.smithi018.stdout:2/505: chown d5/d7/d28/f5f 123 1 2024-04-12T20:56:11.249 INFO:tasks.workunit.client.0.smithi018.stdout:8/686: mkdir db/d4e/d9a/dda 0 2024-04-12T20:56:11.251 INFO:tasks.workunit.client.0.smithi018.stdout:7/522: dwrite d1/db/d31/d3a/f85 [0,4194304] 0 2024-04-12T20:56:11.256 INFO:tasks.workunit.client.0.smithi018.stdout:8/687: chown db/c31 2283780 1 2024-04-12T20:56:11.258 INFO:tasks.workunit.client.0.smithi018.stdout:2/506: creat d5/d8/d10/d19/d24/d39/d9c/fb0 x:0 0 0 2024-04-12T20:56:11.263 INFO:tasks.workunit.client.0.smithi018.stdout:8/688: dread - db/d4e/d56/fd5 zero size 2024-04-12T20:56:11.264 INFO:tasks.workunit.client.0.smithi018.stdout:7/523: rmdir d1/db/d31/d45/d8c/d9d/d70/dab 0 2024-04-12T20:56:11.270 INFO:tasks.workunit.client.0.smithi018.stdout:6/715: dwrite d5/df/d31/fd6 [0,4194304] 0 2024-04-12T20:56:11.272 INFO:tasks.workunit.client.0.smithi018.stdout:5/539: dwrite d0/d2/d33/d46/f3a [0,4194304] 0 2024-04-12T20:56:11.274 INFO:tasks.workunit.client.0.smithi018.stdout:8/689: dread - db/d4e/d9a/d69/d6c/fc1 zero size 2024-04-12T20:56:11.277 INFO:tasks.workunit.client.0.smithi018.stdout:7/524: creat d1/db/d31/d45/d8c/d9d/d70/d96/d5f/fac x:0 0 0 2024-04-12T20:56:11.282 INFO:tasks.workunit.client.0.smithi018.stdout:5/540: unlink d0/d2b/f3d 0 2024-04-12T20:56:11.283 INFO:tasks.workunit.client.0.smithi018.stdout:7/525: mkdir d1/db/d6f/dad 0 2024-04-12T20:56:11.286 INFO:tasks.workunit.client.0.smithi018.stdout:8/690: mkdir db/d4e/d56/d59/d76/da5/d45/da9/ddb 0 2024-04-12T20:56:11.290 INFO:tasks.workunit.client.0.smithi018.stdout:6/716: chown d5/df/d19/d43/l70 73 1 2024-04-12T20:56:11.295 INFO:tasks.workunit.client.0.smithi018.stdout:8/691: dread db/d4e/d56/d59/d76/f90 [0,4194304] 0 2024-04-12T20:56:11.296 INFO:tasks.workunit.client.0.smithi018.stdout:7/526: mknod d1/db/d31/d45/d8c/d9d/d70/cae 0 2024-04-12T20:56:11.299 INFO:tasks.workunit.client.0.smithi018.stdout:5/541: rename d0/d2/d33/d10/d4e/l6b to d0/d2/d33/d10/d4e/lb4 0 2024-04-12T20:56:11.302 INFO:tasks.workunit.client.0.smithi018.stdout:2/507: dwrite d5/d8/d10/d19/fae [0,4194304] 0 2024-04-12T20:56:11.304 INFO:tasks.workunit.client.0.smithi018.stdout:6/717: symlink d5/d6/d1b/d3b/d6a/dd8/l10f 0 2024-04-12T20:56:11.305 INFO:tasks.workunit.client.0.smithi018.stdout:8/692: write db/d4e/d9a/d71/f77 [932277,101743] 0 2024-04-12T20:56:11.307 INFO:tasks.workunit.client.0.smithi018.stdout:7/527: dread d1/db/d31/d45/d53/f7d [0,4194304] 0 2024-04-12T20:56:11.308 INFO:tasks.workunit.client.0.smithi018.stdout:6/718: fdatasync d5/d6/d1b/d3b/d8e/fdd 0 2024-04-12T20:56:11.318 INFO:tasks.workunit.client.0.smithi018.stdout:6/719: symlink d5/d6/d1b/d3b/d6a/dd8/l110 0 2024-04-12T20:56:11.318 INFO:tasks.workunit.client.0.smithi018.stdout:2/508: rmdir d5/d8/d10/d94 0 2024-04-12T20:56:11.325 INFO:tasks.workunit.client.0.smithi018.stdout:2/509: rmdir d5/d8/d10/d19/d24 39 2024-04-12T20:56:11.325 INFO:tasks.workunit.client.0.smithi018.stdout:7/528: link d1/d6/f24 d1/db/d31/d3a/d42/faf 0 2024-04-12T20:56:11.331 INFO:tasks.workunit.client.0.smithi018.stdout:2/510: write d5/f95 [258176,119163] 0 2024-04-12T20:56:11.332 INFO:tasks.workunit.client.0.smithi018.stdout:8/693: dwrite db/d4e/d9a/fc8 [0,4194304] 0 2024-04-12T20:56:11.332 INFO:tasks.workunit.client.0.smithi018.stdout:5/542: rename d0/d2/d33/d46/d1e/d2e/c24 to d0/d2b/cb5 0 2024-04-12T20:56:11.335 INFO:tasks.workunit.client.0.smithi018.stdout:7/529: symlink d1/db/d31/d45/d8c/d9d/lb0 0 2024-04-12T20:56:11.340 INFO:tasks.workunit.client.0.smithi018.stdout:2/511: readlink d5/d8/d25/l2c 0 2024-04-12T20:56:11.346 INFO:tasks.workunit.client.0.smithi018.stdout:8/694: getdents db 0 2024-04-12T20:56:11.351 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:56:11 smithi018 ceph-mon[28799]: pgmap v38: 65 pgs: 65 active+clean; 1015 MiB data, 4.5 GiB used, 532 GiB / 536 GiB avail; 27 MiB/s rd, 114 MiB/s wr, 269 op/s 2024-04-12T20:56:11.351 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:56:11 smithi018 ceph-mon[28799]: from='client.24425 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T20:56:11.351 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:56:11 smithi018 ceph-mon[28799]: from='client.14652 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T20:56:11.358 INFO:tasks.workunit.client.0.smithi018.stdout:6/720: dwrite d5/d6/d1b/d3b/fd7 [0,4194304] 0 2024-04-12T20:56:11.359 INFO:tasks.workunit.client.0.smithi018.stdout:5/543: dwrite d0/d2/d33/d46/d1e/f42 [0,4194304] 0 2024-04-12T20:56:11.363 INFO:tasks.workunit.client.0.smithi018.stdout:5/544: write d0/d2/d11/d37/f47 [1822120,96496] 0 2024-04-12T20:56:11.372 INFO:tasks.workunit.client.0.smithi018.stdout:6/721: mkdir d5/d15/dd1/d111 0 2024-04-12T20:56:11.374 INFO:tasks.workunit.client.0.smithi018.stdout:5/545: symlink d0/d2/d33/d46/d18/lb6 0 2024-04-12T20:56:11.378 INFO:tasks.workunit.client.0.smithi018.stdout:6/722: stat d5/d15/d60/fbc 0 2024-04-12T20:56:11.381 INFO:tasks.workunit.client.0.smithi018.stdout:2/512: dwrite d5/d12/d54/d3e/f8b [0,4194304] 0 2024-04-12T20:56:11.383 INFO:tasks.workunit.client.0.smithi018.stdout:6/723: readlink d5/d6/d1b/d3b/d22/ldf 0 2024-04-12T20:56:11.384 INFO:tasks.workunit.client.0.smithi018.stdout:7/530: dread d1/f3e [0,4194304] 0 2024-04-12T20:56:11.386 INFO:tasks.workunit.client.0.smithi018.stdout:2/513: dread - d5/d12/f96 zero size 2024-04-12T20:56:11.389 INFO:tasks.workunit.client.0.smithi018.stdout:4/582: sync 2024-04-12T20:56:11.389 INFO:tasks.workunit.client.0.smithi018.stdout:9/672: sync 2024-04-12T20:56:11.390 INFO:tasks.workunit.client.0.smithi018.stdout:8/695: dwrite db/d4e/d56/fc7 [0,4194304] 0 2024-04-12T20:56:11.398 INFO:tasks.workunit.client.0.smithi018.stdout:6/724: unlink d5/df/d3f/f49 0 2024-04-12T20:56:11.401 INFO:tasks.workunit.client.0.smithi018.stdout:5/546: rename d0/d2/d33/d46/d35/d4d/f69 to d0/d2/d33/d46/d1e/d2e/fb7 0 2024-04-12T20:56:11.403 INFO:tasks.workunit.client.0.smithi018.stdout:2/514: mknod d5/cb1 0 2024-04-12T20:56:11.405 INFO:tasks.workunit.client.0.smithi018.stdout:5/547: write d0/fa7 [598542,119712] 0 2024-04-12T20:56:11.411 INFO:tasks.workunit.client.0.smithi018.stdout:6/725: creat d5/df/d19/d27/dc8/f112 x:0 0 0 2024-04-12T20:56:11.411 INFO:tasks.workunit.client.0.smithi018.stdout:4/583: link d3/d34/d3d/f65 d3/d5d/d60/d64/fb9 0 2024-04-12T20:56:11.414 INFO:tasks.workunit.client.0.smithi018.stdout:9/673: symlink d3/d6/d9/dc/d46/lc6 0 2024-04-12T20:56:11.418 INFO:tasks.workunit.client.0.smithi018.stdout:8/696: rmdir db/d4e/d73/dd9 0 2024-04-12T20:56:11.427 INFO:tasks.workunit.client.0.smithi018.stdout:2/515: creat d5/d8/d10/d19/daf/d88/fb2 x:0 0 0 2024-04-12T20:56:11.427 INFO:tasks.workunit.client.0.smithi018.stdout:4/584: rename d3/d34/d43/d4b/f9c to d3/d34/fba 0 2024-04-12T20:56:11.428 INFO:tasks.workunit.client.0.smithi018.stdout:8/697: dread - db/d25/d26/d27/fd8 zero size 2024-04-12T20:56:11.429 INFO:tasks.workunit.client.0.smithi018.stdout:8/698: readlink db/l1d 0 2024-04-12T20:56:11.429 INFO:tasks.workunit.client.0.smithi018.stdout:7/531: write d1/db/d31/d45/d8c/d9d/d69/f7e [1728694,51387] 0 2024-04-12T20:56:11.433 INFO:tasks.workunit.client.0.smithi018.stdout:7/532: fsync d1/db/d31/d3a/d42/f75 0 2024-04-12T20:56:11.436 INFO:tasks.workunit.client.0.smithi018.stdout:7/533: stat d1/db/d31/d30/l50 0 2024-04-12T20:56:11.438 INFO:tasks.workunit.client.0.smithi018.stdout:8/699: symlink db/d25/d26/ldc 0 2024-04-12T20:56:11.440 INFO:tasks.workunit.client.0.smithi018.stdout:2/516: creat d5/d8/d10/d19/d24/d39/fb3 x:0 0 0 2024-04-12T20:56:11.441 INFO:tasks.workunit.client.0.smithi018.stdout:4/585: creat d3/d32/d33/db6/fbb x:0 0 0 2024-04-12T20:56:11.441 INFO:tasks.workunit.client.0.smithi018.stdout:7/534: fsync d1/db/f43 0 2024-04-12T20:56:11.444 INFO:tasks.workunit.client.0.smithi018.stdout:2/517: fdatasync f4 0 2024-04-12T20:56:11.445 INFO:tasks.workunit.client.0.smithi018.stdout:7/535: stat d1/d6/f5a 0 2024-04-12T20:56:11.445 INFO:tasks.workunit.client.0.smithi018.stdout:1/425: sync 2024-04-12T20:56:11.445 INFO:tasks.workunit.client.0.smithi018.stdout:3/487: sync 2024-04-12T20:56:11.445 INFO:tasks.workunit.client.0.smithi018.stdout:0/478: sync 2024-04-12T20:56:11.450 INFO:tasks.workunit.client.0.smithi018.stdout:5/548: dwrite d0/d2b/fa0 [0,4194304] 0 2024-04-12T20:56:11.450 INFO:tasks.workunit.client.0.smithi018.stdout:3/488: write d2/d43/f8f [473205,107116] 0 2024-04-12T20:56:11.454 INFO:tasks.workunit.client.0.smithi018.stdout:6/726: link d5/d6/d1b/d3b/d35/d4f/c55 d5/d6/d1b/d3b/d8e/c113 0 2024-04-12T20:56:11.454 INFO:tasks.workunit.client.0.smithi018.stdout:1/426: chown d3/d9/da/dd/d31/d32/d13/d19/d2d/d4e/d58 628181 1 2024-04-12T20:56:11.454 INFO:tasks.workunit.client.0.smithi018.stdout:9/674: rmdir d3/d6/d9/d1e 39 2024-04-12T20:56:11.456 INFO:tasks.workunit.client.0.smithi018.stdout:4/586: mkdir d3/d32/d33/dbc 0 2024-04-12T20:56:11.460 INFO:tasks.workunit.client.0.smithi018.stdout:6/727: creat d5/d71/f114 x:0 0 0 2024-04-12T20:56:11.463 INFO:tasks.workunit.client.0.smithi018.stdout:8/700: rename db/d3b/cc2 to db/d4e/d56/d59/d76/da5/d45/d86/cdd 0 2024-04-12T20:56:11.467 INFO:tasks.workunit.client.0.smithi018.stdout:7/536: creat d1/db/d31/d3a/d97/fb1 x:0 0 0 2024-04-12T20:56:11.468 INFO:tasks.workunit.client.0.smithi018.stdout:6/728: read d5/df/d19/dc5/ff4 [535014,25135] 0 2024-04-12T20:56:11.471 INFO:tasks.workunit.client.0.smithi018.stdout:8/701: creat db/d25/fde x:0 0 0 2024-04-12T20:56:11.473 INFO:tasks.workunit.client.0.smithi018.stdout:8/702: write db/d72/fd7 [576285,111886] 0 2024-04-12T20:56:11.477 INFO:tasks.workunit.client.0.smithi018.stdout:9/675: getdents d3/d6/d9/dc/d13/d5b/d6b/da7 0 2024-04-12T20:56:11.478 INFO:tasks.workunit.client.0.smithi018.stdout:5/549: mknod d0/d2/d33/d46/d1e/d9a/cb8 0 2024-04-12T20:56:11.480 INFO:tasks.workunit.client.0.smithi018.stdout:1/427: symlink d3/d9/da/dd/d31/d32/d13/d19/d2d/d48/d66/l95 0 2024-04-12T20:56:11.480 INFO:tasks.workunit.client.0.smithi018.stdout:7/537: rmdir d1/db/d31/d45 39 2024-04-12T20:56:11.483 INFO:tasks.workunit.client.0.smithi018.stdout:3/489: creat d2/d6/d30/d31/f98 x:0 0 0 2024-04-12T20:56:11.483 INFO:tasks.workunit.client.0.smithi018.stdout:4/587: creat d3/d5d/fbd x:0 0 0 2024-04-12T20:56:11.484 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:56:11.486+0000 7fe3eb000700 1 -- 172.21.15.18:0/2602910701 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe3e40708f0 msgr2=0x7fe3e4070d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:56:11.484 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:56:11.486+0000 7fe3eb000700 1 --2- 172.21.15.18:0/2602910701 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe3e40708f0 0x7fe3e4070d60 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fe3cc004940 tx=0x7fe3cc009e30 comp rx=0 tx=0).stop 2024-04-12T20:56:11.484 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:56:11.486+0000 7fe3eb000700 1 -- 172.21.15.18:0/2602910701 shutdown_connections 2024-04-12T20:56:11.484 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:56:11.486+0000 7fe3eb000700 1 --2- 172.21.15.18:0/2602910701 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe3e40708f0 0x7fe3e4070d60 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:56:11.485 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:56:11.486+0000 7fe3eb000700 1 --2- 172.21.15.18:0/2602910701 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe3e4071ea0 0x7fe3e40703b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:56:11.485 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:56:11.486+0000 7fe3eb000700 1 -- 172.21.15.18:0/2602910701 >> 172.21.15.18:0/2602910701 conn(0x7fe3e40f8560 msgr2=0x7fe3e40fa970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T20:56:11.485 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:56:11.486+0000 7fe3eb000700 1 -- 172.21.15.18:0/2602910701 shutdown_connections 2024-04-12T20:56:11.485 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:56:11.486+0000 7fe3eb000700 1 -- 172.21.15.18:0/2602910701 wait complete. 2024-04-12T20:56:11.485 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:56:11.487+0000 7fe3eb000700 1 Processor -- start 2024-04-12T20:56:11.485 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:56:11.487+0000 7fe3eb000700 1 -- start start 2024-04-12T20:56:11.486 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:56:11.487+0000 7fe3eb000700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe3e40708f0 0x7fe3e41140c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T20:56:11.486 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:56:11.488+0000 7fe3eb000700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe3e4071ea0 0x7fe3e4114600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T20:56:11.486 INFO:tasks.workunit.client.0.smithi018.stdout:0/479: getdents da/d15/d4d/d6b 0 2024-04-12T20:56:11.487 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:56:11.488+0000 7fe3eb000700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3e4114c00 con 0x7fe3e40708f0 2024-04-12T20:56:11.487 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:56:11.488+0000 7fe3eb000700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3e41153e0 con 0x7fe3e4071ea0 2024-04-12T20:56:11.487 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:56:11.488+0000 7fe3e3e00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe3e40708f0 0x7fe3e41140c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T20:56:11.487 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:56:11.488+0000 7fe3e3e00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe3e40708f0 0x7fe3e41140c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:48250/0 (socket says 172.21.15.18:48250) 2024-04-12T20:56:11.488 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:56:11.488+0000 7fe3e3e00700 1 -- 172.21.15.18:0/1255778741 learned_addr learned my addr 172.21.15.18:0/1255778741 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T20:56:11.488 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:56:11.489+0000 7fe3e3400700 1 --2- 172.21.15.18:0/1255778741 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe3e4071ea0 0x7fe3e4114600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T20:56:11.488 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:56:11.489+0000 7fe3e3e00700 1 -- 172.21.15.18:0/1255778741 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe3e4071ea0 msgr2=0x7fe3e4114600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:56:11.488 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:56:11.489+0000 7fe3e3e00700 1 --2- 172.21.15.18:0/1255778741 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe3e4071ea0 0x7fe3e4114600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:56:11.488 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:56:11.489+0000 7fe3e3e00700 1 -- 172.21.15.18:0/1255778741 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3cc005040 con 0x7fe3e40708f0 2024-04-12T20:56:11.488 INFO:tasks.workunit.client.0.smithi018.stdout:7/538: write d1/db/d31/d3a/f85 [110459,90050] 0 2024-04-12T20:56:11.488 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:56:11.489+0000 7fe3e3e00700 1 --2- 172.21.15.18:0/1255778741 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe3e40708f0 0x7fe3e41140c0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fe3dc001ce0 tx=0x7fe3dc009e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T20:56:11.490 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:56:11.490+0000 7fe3dbe00700 1 -- 172.21.15.18:0/1255778741 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe3dc00db60 con 0x7fe3e40708f0 2024-04-12T20:56:11.490 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:56:11.490+0000 7fe3dbe00700 1 -- 172.21.15.18:0/1255778741 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe3dc00f040 con 0x7fe3e40708f0 2024-04-12T20:56:11.490 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:56:11.490+0000 7fe3dbe00700 1 -- 172.21.15.18:0/1255778741 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe3dc009790 con 0x7fe3e40708f0 2024-04-12T20:56:11.490 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:56:11.490+0000 7fe3eb000700 1 -- 172.21.15.18:0/1255778741 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe3e41156c0 con 0x7fe3e40708f0 2024-04-12T20:56:11.491 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:56:11.491+0000 7fe3eb000700 1 -- 172.21.15.18:0/1255778741 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe3e4115bb0 con 0x7fe3e40708f0 2024-04-12T20:56:11.491 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:56:11.492+0000 7fe3eb000700 1 -- 172.21.15.18:0/1255778741 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe3e405f5f0 con 0x7fe3e40708f0 2024-04-12T20:56:11.498 INFO:tasks.workunit.client.0.smithi018.stdout:3/490: mknod d2/d4d/d88/c99 0 2024-04-12T20:56:11.498 INFO:tasks.workunit.client.0.smithi018.stdout:1/428: creat d3/d9/da/dd/d31/d32/d33/f96 x:0 0 0 2024-04-12T20:56:11.499 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:56:11.497+0000 7fe3dbe00700 1 -- 172.21.15.18:0/1255778741 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 24) v1 ==== 89644+0+0 (secure 0 0 0) 0x7fe3dc007920 con 0x7fe3e40708f0 2024-04-12T20:56:11.499 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:56:11.498+0000 7fe3dbe00700 1 --2- 172.21.15.18:0/1255778741 >> [v2:172.21.15.186:6828/2316349902,v1:172.21.15.186:6829/2316349902] conn(0x7fe3c806bcf0 0x7fe3c806e1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T20:56:11.499 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:56:11.499+0000 7fe3dbe00700 1 -- 172.21.15.18:0/1255778741 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5808+0+0 (secure 0 0 0) 0x7fe3dc05bdd0 con 0x7fe3e40708f0 2024-04-12T20:56:11.499 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:56:11.500+0000 7fe3e3400700 1 --2- 172.21.15.18:0/1255778741 >> [v2:172.21.15.186:6828/2316349902,v1:172.21.15.186:6829/2316349902] conn(0x7fe3c806bcf0 0x7fe3c806e1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T20:56:11.501 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:56:11.503+0000 7fe3e3400700 1 --2- 172.21.15.18:0/1255778741 >> [v2:172.21.15.186:6828/2316349902,v1:172.21.15.186:6829/2316349902] conn(0x7fe3c806bcf0 0x7fe3c806e1a0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fe3cc00b970 tx=0x7fe3cc009e30 comp rx=0 tx=0).ready entity=mgr.24359 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T20:56:11.501 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:56:11.503+0000 7fe3dbe00700 1 -- 172.21.15.18:0/1255778741 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186454 (secure 0 0 0) 0x7fe3dc054500 con 0x7fe3e40708f0 2024-04-12T20:56:11.507 INFO:tasks.workunit.client.0.smithi018.stdout:4/588: dread d3/d32/d94/f97 [0,4194304] 0 2024-04-12T20:56:11.515 INFO:tasks.workunit.client.0.smithi018.stdout:1/429: mkdir d3/d9/da/dd/d31/d32/d13/d19/d2d/d4e/d5e/d97 0 2024-04-12T20:56:11.516 INFO:tasks.workunit.client.0.smithi018.stdout:3/491: symlink d2/d6/d30/d31/d38/d3a/l9a 0 2024-04-12T20:56:11.524 INFO:tasks.workunit.client.0.smithi018.stdout:7/539: rename d1/db/d31/d45/d8c/d9d/d69/l80 to d1/db/d31/d45/d8c/d9d/d70/d96/d5f/da9/lb2 0 2024-04-12T20:56:11.528 INFO:tasks.workunit.client.0.smithi018.stdout:5/550: getdents d0/d5b/d84 0 2024-04-12T20:56:11.529 INFO:tasks.workunit.client.0.smithi018.stdout:0/480: dread da/d15/d17/d5c/f64 [0,4194304] 0 2024-04-12T20:56:11.530 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:56:11 smithi186 ceph-mon[36597]: pgmap v38: 65 pgs: 65 active+clean; 1015 MiB data, 4.5 GiB used, 532 GiB / 536 GiB avail; 27 MiB/s rd, 114 MiB/s wr, 269 op/s 2024-04-12T20:56:11.530 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:56:11 smithi186 ceph-mon[36597]: from='client.24425 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T20:56:11.531 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:56:11 smithi186 ceph-mon[36597]: from='client.14652 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T20:56:11.531 INFO:tasks.workunit.client.0.smithi018.stdout:1/430: rmdir d3/d9/da/d14/d73 39 2024-04-12T20:56:11.533 INFO:tasks.workunit.client.0.smithi018.stdout:7/540: mkdir d1/db/d6f/db3 0 2024-04-12T20:56:11.539 INFO:tasks.workunit.client.0.smithi018.stdout:0/481: fsync da/f52 0 2024-04-12T20:56:11.539 INFO:tasks.workunit.client.0.smithi018.stdout:5/551: mknod d0/d2b/cb9 0 2024-04-12T20:56:11.543 INFO:tasks.workunit.client.0.smithi018.stdout:4/589: dwrite d3/d5d/d60/f67 [0,4194304] 0 2024-04-12T20:56:11.551 INFO:tasks.workunit.client.0.smithi018.stdout:8/703: dwrite db/d25/d26/d27/f54 [0,4194304] 0 2024-04-12T20:56:11.555 INFO:tasks.workunit.client.0.smithi018.stdout:5/552: fsync d0/d2/d33/d46/d35/f62 0 2024-04-12T20:56:11.557 INFO:tasks.workunit.client.0.smithi018.stdout:3/492: link d2/d6/d30/d31/l77 d2/d4d/d8d/l9b 0 2024-04-12T20:56:11.557 INFO:tasks.workunit.client.0.smithi018.stdout:7/541: creat d1/d6/fb4 x:0 0 0 2024-04-12T20:56:11.562 INFO:tasks.workunit.client.0.smithi018.stdout:8/704: read - db/d4e/d56/d59/d76/da5/d45/d86/da1/fb6 zero size 2024-04-12T20:56:11.565 INFO:tasks.workunit.client.0.smithi018.stdout:8/705: chown db/d4e/d9a/fc8 52600961 1 2024-04-12T20:56:11.570 INFO:tasks.workunit.client.0.smithi018.stdout:7/542: mknod d1/db/d31/d45/d8c/d9d/d94/cb5 0 2024-04-12T20:56:11.571 INFO:tasks.workunit.client.0.smithi018.stdout:3/493: creat d2/d6/d30/d31/f9c x:0 0 0 2024-04-12T20:56:11.581 INFO:tasks.workunit.client.0.smithi018.stdout:7/543: unlink d1/db/d31/l36 0 2024-04-12T20:56:11.596 INFO:tasks.workunit.client.0.smithi018.stdout:0/482: dwrite da/d15/d17/d21/f68 [0,4194304] 0 2024-04-12T20:56:11.603 INFO:tasks.workunit.client.0.smithi018.stdout:0/483: stat da/d15/d17/d21/d36/f7c 0 2024-04-12T20:56:11.606 INFO:tasks.workunit.client.0.smithi018.stdout:8/706: dwrite db/d25/d26/d27/fa0 [0,4194304] 0 2024-04-12T20:56:11.609 INFO:tasks.workunit.client.0.smithi018.stdout:4/590: dread d3/fa8 [0,4194304] 0 2024-04-12T20:56:11.615 INFO:tasks.workunit.client.0.smithi018.stdout:1/431: dread d3/d9/da/dd/d31/d32/d13/f46 [0,4194304] 0 2024-04-12T20:56:11.615 INFO:tasks.workunit.client.0.smithi018.stdout:0/484: dread da/d15/d17/d21/d36/f80 [0,4194304] 0 2024-04-12T20:56:11.618 INFO:tasks.workunit.client.0.smithi018.stdout:3/494: dwrite d2/d6/d30/f3e [4194304,4194304] 0 2024-04-12T20:56:11.623 INFO:tasks.workunit.client.0.smithi018.stdout:5/553: dread d0/d2/d33/d10/f39 [0,4194304] 0 2024-04-12T20:56:11.630 INFO:tasks.workunit.client.0.smithi018.stdout:5/554: write d0/d2/d33/d46/d18/f71 [3791543,73435] 0 2024-04-12T20:56:11.631 INFO:tasks.workunit.client.0.smithi018.stdout:3/495: mknod d2/d83/c9d 0 2024-04-12T20:56:11.631 INFO:tasks.workunit.client.0.smithi018.stdout:4/591: rmdir d3/db/db7 0 2024-04-12T20:56:11.634 INFO:tasks.workunit.client.0.smithi018.stdout:5/555: dread - d0/d5b/d84/f7d zero size 2024-04-12T20:56:11.641 INFO:tasks.workunit.client.0.smithi018.stdout:9/676: dwrite d3/d6/d9/dc/f2d [0,4194304] 0 2024-04-12T20:56:11.642 INFO:tasks.workunit.client.0.smithi018.stdout:8/707: truncate db/d4e/d7f/f9e 1194882 0 2024-04-12T20:56:11.650 INFO:tasks.workunit.client.0.smithi018.stdout:6/729: write d5/f57 [1204433,50064] 0 2024-04-12T20:56:11.650 INFO:tasks.workunit.client.0.smithi018.stdout:3/496: rename d2/d6/l36 to d2/d6/d34/d71/l9e 0 2024-04-12T20:56:11.653 INFO:tasks.workunit.client.0.smithi018.stdout:0/485: dwrite da/d15/f39 [0,4194304] 0 2024-04-12T20:56:11.659 INFO:tasks.workunit.client.0.smithi018.stdout:5/556: getdents d0/d2/d33/d10/d9d 0 2024-04-12T20:56:11.663 INFO:tasks.workunit.client.0.smithi018.stdout:5/557: chown d0/d2/d33/d46/d1e/d55/c68 2110148340 1 2024-04-12T20:56:11.664 INFO:tasks.workunit.client.0.smithi018.stdout:7/544: dread d1/db/d31/d30/f57 [0,4194304] 0 2024-04-12T20:56:11.666 INFO:tasks.workunit.client.0.smithi018.stdout:8/708: creat db/d4e/dab/fdf x:0 0 0 2024-04-12T20:56:11.669 INFO:tasks.workunit.client.0.smithi018.stdout:0/486: mkdir da/d15/d17/d21/d36/da2 0 2024-04-12T20:56:11.672 INFO:tasks.workunit.client.0.smithi018.stdout:4/592: getdents d3/d5d/d60/d64 0 2024-04-12T20:56:11.673 INFO:tasks.workunit.client.0.smithi018.stdout:0/487: stat da/d15/d17/d5c/d79 0 2024-04-12T20:56:11.673 INFO:tasks.workunit.client.0.smithi018.stdout:1/432: getdents d3/d9/da/d14/d73 0 2024-04-12T20:56:11.679 INFO:tasks.workunit.client.0.smithi018.stdout:3/497: link d2/d6/d30/f37 d2/d6/d3d/d54/f9f 0 2024-04-12T20:56:11.686 INFO:tasks.workunit.client.0.smithi018.stdout:1/433: symlink d3/d9/da/dd/d31/d32/d33/d47/l98 0 2024-04-12T20:56:11.690 INFO:tasks.workunit.client.0.smithi018.stdout:3/498: creat d2/d4d/d88/fa0 x:0 0 0 2024-04-12T20:56:11.691 INFO:tasks.workunit.client.0.smithi018.stdout:6/730: rename d5/d6/d1b/d8c/dc0/cf5 to d5/df/c115 0 2024-04-12T20:56:11.694 INFO:tasks.workunit.client.0.smithi018.stdout:0/488: dread da/d15/d17/d21/f68 [0,4194304] 0 2024-04-12T20:56:11.694 INFO:tasks.workunit.client.0.smithi018.stdout:4/593: creat d3/d32/fbe x:0 0 0 2024-04-12T20:56:11.695 INFO:tasks.workunit.client.0.smithi018.stdout:3/499: unlink d2/d6/d3d/d70/c8e 0 2024-04-12T20:56:11.697 INFO:tasks.workunit.client.0.smithi018.stdout:9/677: dread d3/d6/d9/dc/d5a/d79/fb3 [0,4194304] 0 2024-04-12T20:56:11.700 INFO:tasks.workunit.client.0.smithi018.stdout:5/558: rename d0/d5b/d84/fa5 to d0/d2/fba 0 2024-04-12T20:56:11.701 INFO:tasks.workunit.client.0.smithi018.stdout:3/500: mkdir d2/d6/d34/da1 0 2024-04-12T20:56:11.704 INFO:tasks.workunit.client.0.smithi018.stdout:4/594: symlink d3/d5d/d60/d64/d79/lbf 0 2024-04-12T20:56:11.704 INFO:tasks.workunit.client.0.smithi018.stdout:3/501: stat d2/d6/d30/d31/f98 0 2024-04-12T20:56:11.707 INFO:tasks.workunit.client.0.smithi018.stdout:8/709: rename db/d25/d26 to db/d4e/d56/d59/d76/de0 0 2024-04-12T20:56:11.707 INFO:tasks.workunit.client.0.smithi018.stdout:3/502: chown d2/d6/d30/d31/d38/f74 104372032 1 2024-04-12T20:56:11.713 INFO:tasks.workunit.client.0.smithi018.stdout:7/545: rename d1/db/d31/d3a/d42/d4c/d6c/l91 to d1/db/d31/d45/d8c/d9d/d70/d96/d5f/da9/lb6 0 2024-04-12T20:56:11.713 INFO:tasks.workunit.client.0.smithi018.stdout:5/559: mknod d0/d2/d33/d46/da2/da3/dae/cbb 0 2024-04-12T20:56:11.714 INFO:tasks.workunit.client.0.smithi018.stdout:4/595: mkdir d3/d34/d3d/da3/dc0 0 2024-04-12T20:56:11.718 INFO:tasks.workunit.client.0.smithi018.stdout:7/546: write d1/db/d31/d45/d8c/d9d/d70/d96/d5f/fac [696335,8371] 0 2024-04-12T20:56:11.723 INFO:tasks.workunit.client.0.smithi018.stdout:5/560: creat d0/d2/d33/d46/d18/fbc x:0 0 0 2024-04-12T20:56:11.724 INFO:tasks.workunit.client.0.smithi018.stdout:7/547: readlink d1/db/d31/d45/d8c/d9d/d70/l71 0 2024-04-12T20:56:11.726 INFO:tasks.workunit.client.0.smithi018.stdout:0/489: rename da/d15/c16 to da/d15/d4d/ca3 0 2024-04-12T20:56:11.728 INFO:tasks.workunit.client.0.smithi018.stdout:4/596: truncate d3/f6b 527179 0 2024-04-12T20:56:11.731 INFO:tasks.workunit.client.0.smithi018.stdout:1/434: rename d3/d9/da/dd/d31/d32/d13/d19/d2d/c4b to d3/d9/da/dd/d31/d32/d13/d19/d2d/d4e/d58/d61/c99 0 2024-04-12T20:56:11.732 INFO:tasks.workunit.client.0.smithi018.stdout:5/561: creat d0/d2/d33/d46/d1e/d2e/d7f/fbd x:0 0 0 2024-04-12T20:56:11.732 INFO:tasks.workunit.client.0.smithi018.stdout:0/490: chown da/d15/d3e/l4b 18 1 2024-04-12T20:56:11.734 INFO:tasks.workunit.client.0.smithi018.stdout:8/710: getdents db/d4e/d73 0 2024-04-12T20:56:11.738 INFO:tasks.workunit.client.0.smithi018.stdout:4/597: rmdir d3/d32/d66 39 2024-04-12T20:56:11.739 INFO:tasks.workunit.client.0.smithi018.stdout:3/503: dwrite d2/d6/f84 [0,4194304] 0 2024-04-12T20:56:11.740 INFO:tasks.workunit.client.0.smithi018.stdout:3/504: fdatasync d2/d4d/f57 0 2024-04-12T20:56:11.745 INFO:tasks.workunit.client.0.smithi018.stdout:2/518: sync 2024-04-12T20:56:11.745 INFO:tasks.workunit.client.0.smithi018.stdout:6/731: rename d5/df/d19/dc5 to d5/d6/d1b/d3b/d6a/d116 0 2024-04-12T20:56:11.746 INFO:tasks.workunit.client.0.smithi018.stdout:4/598: creat d3/d34/fc1 x:0 0 0 2024-04-12T20:56:11.750 INFO:tasks.workunit.client.0.smithi018.stdout:8/711: mknod db/d95/dd6/ce1 0 2024-04-12T20:56:11.750 INFO:tasks.workunit.client.0.smithi018.stdout:3/505: getdents d2/d6/d30/d31/d38/d3a/d4c/d6b/d81 0 2024-04-12T20:56:11.753 INFO:tasks.workunit.client.0.smithi018.stdout:6/732: write d5/df/d19/f25 [2049656,111251] 0 2024-04-12T20:56:11.753 INFO:tasks.workunit.client.0.smithi018.stdout:9/678: dread d3/d6/d9/d1e/f53 [0,4194304] 0 2024-04-12T20:56:11.754 INFO:tasks.workunit.client.0.smithi018.stdout:7/548: rename d1/db/d31/l4b to d1/db/d31/d45/d8c/lb7 0 2024-04-12T20:56:11.755 INFO:tasks.workunit.client.0.smithi018.stdout:5/562: creat d0/d2/fbe x:0 0 0 2024-04-12T20:56:11.758 INFO:tasks.workunit.client.0.smithi018.stdout:2/519: creat d5/d8/d10/d19/daf/d90/fb4 x:0 0 0 2024-04-12T20:56:11.758 INFO:tasks.workunit.client.0.smithi018.stdout:3/506: rmdir d2/d6/d30/d31/d7c 39 2024-04-12T20:56:11.760 INFO:tasks.workunit.client.0.smithi018.stdout:1/435: getdents d3/d65 0 2024-04-12T20:56:11.763 INFO:tasks.workunit.client.0.smithi018.stdout:8/712: creat db/d4e/d9a/daa/dcd/fe2 x:0 0 0 2024-04-12T20:56:11.767 INFO:tasks.workunit.client.0.smithi018.stdout:8/713: chown db/d25/f52 1215857 1 2024-04-12T20:56:11.767 INFO:tasks.workunit.client.0.smithi018.stdout:6/733: rename d5/db8 to d5/d6/d1b/d3b/d22/d117 0 2024-04-12T20:56:11.770 INFO:tasks.workunit.client.0.smithi018.stdout:4/599: dread d3/db/f37 [0,4194304] 0 2024-04-12T20:56:11.781 INFO:tasks.workunit.client.0.smithi018.stdout:5/563: truncate d0/d2/d33/d46/d1e/d55/f88 74083 0 2024-04-12T20:56:11.784 INFO:tasks.workunit.client.0.smithi018.stdout:6/734: mknod d5/d15/dd1/c118 0 2024-04-12T20:56:11.785 INFO:tasks.workunit.client.0.smithi018.stdout:7/549: creat d1/db/d31/d45/d8c/d9d/d70/fb8 x:0 0 0 2024-04-12T20:56:11.785 INFO:tasks.workunit.client.0.smithi018.stdout:3/507: mkdir d2/d6/d30/d31/d38/d3a/d76/da2 0 2024-04-12T20:56:11.788 INFO:tasks.workunit.client.0.smithi018.stdout:7/550: write d1/db/d31/d3a/f85 [2172580,77428] 0 2024-04-12T20:56:11.790 INFO:tasks.workunit.client.0.smithi018.stdout:9/679: rename d3/d6/c1a to d3/d6/d9/dc/d13/db9/cc7 0 2024-04-12T20:56:11.793 INFO:tasks.workunit.client.0.smithi018.stdout:6/735: creat d5/d6/f119 x:0 0 0 2024-04-12T20:56:11.797 INFO:tasks.workunit.client.0.smithi018.stdout:4/600: mknod d3/d32/d33/d8e/cc2 0 2024-04-12T20:56:11.797 INFO:tasks.workunit.client.0.smithi018.stdout:2/520: fdatasync d5/d7/f4d 0 2024-04-12T20:56:11.798 INFO:tasks.workunit.client.0.smithi018.stdout:1/436: dread d3/d9/da/dd/d31/f35 [0,4194304] 0 2024-04-12T20:56:11.803 INFO:tasks.workunit.client.0.smithi018.stdout:4/601: write d3/d5d/fae [289894,124604] 0 2024-04-12T20:56:11.805 INFO:tasks.workunit.client.0.smithi018.stdout:5/564: rename d0/d2/d33/d46/da2 to d0/d2/d33/d10/d2f/dbf 0 2024-04-12T20:56:11.805 INFO:tasks.workunit.client.0.smithi018.stdout:7/551: chown d1/db/d31/d45/d8c/d9d/d70/d96/d5f/da9/lb2 0 1 2024-04-12T20:56:11.809 INFO:tasks.workunit.client.0.smithi018.stdout:9/680: getdents d3/d6/d9/dc/d5a 0 2024-04-12T20:56:11.811 INFO:tasks.workunit.client.0.smithi018.stdout:5/565: readlink d0/d2/d33/d10/d26/l3f 0 2024-04-12T20:56:11.813 INFO:tasks.workunit.client.0.smithi018.stdout:6/736: rename d5/df/d31/d4e/lee to d5/d6/d1b/d8c/l11a 0 2024-04-12T20:56:11.815 INFO:tasks.workunit.client.0.smithi018.stdout:2/521: creat d5/d8/d10/d19/daf/d88/fb5 x:0 0 0 2024-04-12T20:56:11.816 INFO:tasks.workunit.client.0.smithi018.stdout:8/714: creat db/d4e/d56/d59/d76/de0/fe3 x:0 0 0 2024-04-12T20:56:11.821 INFO:tasks.workunit.client.0.smithi018.stdout:5/566: symlink d0/d2/d33/d46/d35/d4d/lc0 0 2024-04-12T20:56:11.823 INFO:tasks.workunit.client.0.smithi018.stdout:1/437: dread d3/d9/da/dd/d31/d32/d13/d19/d2d/f4d [0,4194304] 0 2024-04-12T20:56:11.825 INFO:tasks.workunit.client.0.smithi018.stdout:2/522: mkdir d5/d7/db6 0 2024-04-12T20:56:11.827 INFO:tasks.workunit.client.0.smithi018.stdout:2/523: dread - d5/d8/d10/d19/d24/f4b zero size 2024-04-12T20:56:11.828 INFO:tasks.workunit.client.0.smithi018.stdout:5/567: symlink d0/d2/d33/d10/d2f/dbf/da3/dae/lc1 0 2024-04-12T20:56:11.829 INFO:tasks.workunit.client.0.smithi018.stdout:6/737: dread d5/d6/d1b/d3b/d6a/d116/ff4 [0,4194304] 0 2024-04-12T20:56:11.840 INFO:tasks.workunit.client.0.smithi018.stdout:8/715: dwrite f0 [0,4194304] 0 2024-04-12T20:56:11.841 INFO:tasks.workunit.client.0.smithi018.stdout:2/524: rename d5/d7/lc to d5/d8/d10/d19/d24/d39/da4/lb7 0 2024-04-12T20:56:11.845 INFO:tasks.workunit.client.0.smithi018.stdout:7/552: write d1/db/d31/f63 [1063679,47741] 0 2024-04-12T20:56:11.847 INFO:tasks.workunit.client.0.smithi018.stdout:3/508: dwrite d2/d6/f12 [0,4194304] 0 2024-04-12T20:56:11.848 INFO:tasks.workunit.client.0.smithi018.stdout:5/568: rename d0/d2/d33/d10/d2f/d58 to d0/d2/d11/d37/dc2 0 2024-04-12T20:56:11.853 INFO:tasks.workunit.client.0.smithi018.stdout:8/716: unlink db/l32 0 2024-04-12T20:56:11.854 INFO:tasks.workunit.client.0.smithi018.stdout:3/509: symlink d2/d6/d3d/d70/la3 0 2024-04-12T20:56:11.857 INFO:tasks.workunit.client.0.smithi018.stdout:6/738: getdents d5/d6/d1b/d3b/d22/d117 0 2024-04-12T20:56:11.859 INFO:tasks.workunit.client.0.smithi018.stdout:5/569: chown d0/d2/d11/d37/f4c 25535785 1 2024-04-12T20:56:11.860 INFO:tasks.workunit.client.0.smithi018.stdout:7/553: mkdir d1/db/d31/d45/d8c/d9d/d70/d96/d5f/da9/db9 0 2024-04-12T20:56:11.863 INFO:tasks.workunit.client.0.smithi018.stdout:6/739: fdatasync d5/df/d19/f6c 0 2024-04-12T20:56:11.878 INFO:tasks.workunit.client.0.smithi018.stdout:2/525: truncate d5/d7/d79/f7e 583096 0 2024-04-12T20:56:11.878 INFO:tasks.workunit.client.0.smithi018.stdout:8/717: write db/d4e/d56/d59/d76/fbe [786821,76775] 0 2024-04-12T20:56:11.881 INFO:tasks.workunit.client.0.smithi018.stdout:1/438: dwrite d3/d9/f18 [0,4194304] 0 2024-04-12T20:56:11.882 INFO:tasks.workunit.client.0.smithi018.stdout:6/740: fdatasync d5/df/d19/d74/f75 0 2024-04-12T20:56:11.896 INFO:tasks.workunit.client.0.smithi018.stdout:6/741: creat d5/df/d3f/f11b x:0 0 0 2024-04-12T20:56:11.896 INFO:tasks.workunit.client.0.smithi018.stdout:5/570: dwrite d0/d2/d33/d46/d35/f62 [0,4194304] 0 2024-04-12T20:56:11.899 INFO:tasks.workunit.client.0.smithi018.stdout:9/681: truncate d3/d6/d9/d1e/f74 1507291 0 2024-04-12T20:56:11.904 INFO:tasks.workunit.client.0.smithi018.stdout:6/742: fsync d5/d15/fa6 0 2024-04-12T20:56:11.907 INFO:tasks.workunit.client.0.smithi018.stdout:5/571: read - d0/d2/d33/d46/d1e/d55/d66/f93 zero size 2024-04-12T20:56:11.911 INFO:tasks.workunit.client.0.smithi018.stdout:1/439: dread d3/d9/f3c [0,4194304] 0 2024-04-12T20:56:11.911 INFO:tasks.workunit.client.0.smithi018.stdout:3/510: dwrite d2/d83/f92 [0,4194304] 0 2024-04-12T20:56:11.914 INFO:tasks.workunit.client.0.smithi018.stdout:5/572: readlink d0/d2/l21 0 2024-04-12T20:56:11.917 INFO:tasks.workunit.client.0.smithi018.stdout:2/526: link d5/d8/d10/d19/f7b d5/d7/d31/d62/fb8 0 2024-04-12T20:56:11.919 INFO:tasks.workunit.client.0.smithi018.stdout:8/718: dwrite f0 [0,4194304] 0 2024-04-12T20:56:11.921 INFO:tasks.workunit.client.0.smithi018.stdout:5/573: truncate d0/d2/d33/d46/d18/fbc 245643 0 2024-04-12T20:56:11.926 INFO:tasks.workunit.client.0.smithi018.stdout:5/574: write d0/d2/fa6 [813427,22256] 0 2024-04-12T20:56:11.926 INFO:tasks.workunit.client.0.smithi018.stdout:6/743: symlink d5/l11c 0 2024-04-12T20:56:11.931 INFO:tasks.workunit.client.0.smithi018.stdout:2/527: mknod d5/d12/d54/cb9 0 2024-04-12T20:56:11.938 INFO:tasks.workunit.client.0.smithi018.stdout:6/744: truncate d5/d6/d1b/d3b/d6a/dd8/ff9 707369 0 2024-04-12T20:56:11.939 INFO:tasks.workunit.client.0.smithi018.stdout:8/719: read db/d4e/d73/f92 [679216,9601] 0 2024-04-12T20:56:11.940 INFO:tasks.workunit.client.0.smithi018.stdout:9/682: dread d3/d6/d9/dc/d5a/d79/d7e/d9b/f34 [4194304,4194304] 0 2024-04-12T20:56:11.944 INFO:tasks.workunit.client.0.smithi018.stdout:9/683: mkdir d3/dc8 0 2024-04-12T20:56:11.947 INFO:tasks.workunit.client.0.smithi018.stdout:8/720: chown db/d4e/d56/d59/d76/da5/d45/d86/da1/fb6 3772 1 2024-04-12T20:56:11.948 INFO:tasks.workunit.client.0.smithi018.stdout:1/440: write d3/d9/da/dd/d31/f35 [738991,48854] 0 2024-04-12T20:56:11.949 INFO:tasks.workunit.client.0.smithi018.stdout:6/745: unlink d5/d6/d1b/d3b/d35/d4f/df3/d105/f107 0 2024-04-12T20:56:11.954 INFO:tasks.workunit.client.0.smithi018.stdout:1/441: fsync d3/d9/da/d14/d73/f90 0 2024-04-12T20:56:11.955 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:56:11.957+0000 7fe3eb000700 1 -- 172.21.15.18:0/1255778741 --> [v2:172.21.15.186:6828/2316349902,v1:172.21.15.186:6829/2316349902] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fe3e4116000 con 0x7fe3c806bcf0 2024-04-12T20:56:11.956 INFO:tasks.workunit.client.0.smithi018.stdout:2/528: link d5/d7/d79/f8c d5/d8/d25/fba 0 2024-04-12T20:56:11.957 INFO:tasks.workunit.client.0.smithi018.stdout:5/575: truncate d0/d2/d33/d46/d35/d4d/f80 1063793 0 2024-04-12T20:56:11.959 INFO:teuthology.orchestra.run.smithi018.stdout:{ 2024-04-12T20:56:11.959 INFO:teuthology.orchestra.run.smithi018.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:f4452def3be4d350f4a8e5feaf7ebb1511be9298fe68478ba7a2d6fbd7d69a63", 2024-04-12T20:56:11.959 INFO:teuthology.orchestra.run.smithi018.stdout: "in_progress": true, 2024-04-12T20:56:11.959 INFO:teuthology.orchestra.run.smithi018.stdout: "which": "Upgrading daemons of type(s) mgr", 2024-04-12T20:56:11.959 INFO:teuthology.orchestra.run.smithi018.stdout: "services_complete": [], 2024-04-12T20:56:11.959 INFO:teuthology.orchestra.run.smithi018.stdout: "progress": "1/2 daemons upgraded", 2024-04-12T20:56:11.959 INFO:teuthology.orchestra.run.smithi018.stdout: "message": "", 2024-04-12T20:56:11.959 INFO:teuthology.orchestra.run.smithi018.stdout: "is_paused": false 2024-04-12T20:56:11.959 INFO:teuthology.orchestra.run.smithi018.stdout:} 2024-04-12T20:56:11.960 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:56:11.959+0000 7fe3dbe00700 1 -- 172.21.15.18:0/1255778741 <== mgr.24359 v2:172.21.15.186:6828/2316349902 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+310 (secure 0 0 0) 0x7fe3e4116000 con 0x7fe3c806bcf0 2024-04-12T20:56:11.961 INFO:tasks.workunit.client.0.smithi018.stdout:8/721: mkdir db/d4e/d9a/d71/de4 0 2024-04-12T20:56:11.962 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:56:11.964+0000 7fe3eb000700 1 -- 172.21.15.18:0/1255778741 >> [v2:172.21.15.186:6828/2316349902,v1:172.21.15.186:6829/2316349902] conn(0x7fe3c806bcf0 msgr2=0x7fe3c806e1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:56:11.962 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:56:11.964+0000 7fe3eb000700 1 --2- 172.21.15.18:0/1255778741 >> [v2:172.21.15.186:6828/2316349902,v1:172.21.15.186:6829/2316349902] conn(0x7fe3c806bcf0 0x7fe3c806e1a0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fe3cc00b970 tx=0x7fe3cc009e30 comp rx=0 tx=0).stop 2024-04-12T20:56:11.962 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:56:11.964+0000 7fe3eb000700 1 -- 172.21.15.18:0/1255778741 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe3e40708f0 msgr2=0x7fe3e41140c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:56:11.962 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:56:11.964+0000 7fe3eb000700 1 --2- 172.21.15.18:0/1255778741 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe3e40708f0 0x7fe3e41140c0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fe3dc001ce0 tx=0x7fe3dc009e60 comp rx=0 tx=0).stop 2024-04-12T20:56:11.962 INFO:tasks.workunit.client.0.smithi018.stdout:3/511: dwrite d2/d6/f1c [4194304,4194304] 0 2024-04-12T20:56:11.964 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:56:11.964+0000 7fe3eb000700 1 -- 172.21.15.18:0/1255778741 shutdown_connections 2024-04-12T20:56:11.964 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:56:11.964+0000 7fe3eb000700 1 --2- 172.21.15.18:0/1255778741 >> [v2:172.21.15.186:6828/2316349902,v1:172.21.15.186:6829/2316349902] conn(0x7fe3c806bcf0 0x7fe3c806e1a0 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:56:11.964 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:56:11.964+0000 7fe3eb000700 1 --2- 172.21.15.18:0/1255778741 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe3e40708f0 0x7fe3e41140c0 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:56:11.964 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:56:11.964+0000 7fe3eb000700 1 --2- 172.21.15.18:0/1255778741 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe3e4071ea0 0x7fe3e4114600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:56:11.964 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:56:11.964+0000 7fe3eb000700 1 -- 172.21.15.18:0/1255778741 >> 172.21.15.18:0/1255778741 conn(0x7fe3e40f8560 msgr2=0x7fe3e40f9e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T20:56:11.964 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:56:11.965+0000 7fe3eb000700 1 -- 172.21.15.18:0/1255778741 shutdown_connections 2024-04-12T20:56:11.964 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:56:11.965+0000 7fe3eb000700 1 -- 172.21.15.18:0/1255778741 wait complete. 2024-04-12T20:56:11.966 INFO:tasks.workunit.client.0.smithi018.stdout:1/442: fsync d3/d9/da/dd/d31/d32/d13/d19/d2d/d4e/f60 0 2024-04-12T20:56:11.969 INFO:tasks.workunit.client.0.smithi018.stdout:3/512: rmdir d2 39 2024-04-12T20:56:11.969 INFO:tasks.workunit.client.0.smithi018.stdout:9/684: dread d3/d6/d9/dc/d13/d5b/d6b/fa1 [0,4194304] 0 2024-04-12T20:56:11.973 INFO:tasks.workunit.client.0.smithi018.stdout:6/746: mkdir d5/df/d3f/dca/d11d 0 2024-04-12T20:56:11.974 INFO:tasks.workunit.client.0.smithi018.stdout:9/685: readlink d3/d6/d9/dc/d46/l6c 0 2024-04-12T20:56:11.979 INFO:tasks.workunit.client.0.smithi018.stdout:3/513: fdatasync d2/d43/d61/f62 0 2024-04-12T20:56:11.984 INFO:tasks.workunit.client.0.smithi018.stdout:6/747: creat d5/d6/d1b/d3b/d35/f11e x:0 0 0 2024-04-12T20:56:11.985 INFO:tasks.workunit.client.0.smithi018.stdout:9/686: creat d3/d6/d3d/dba/fc9 x:0 0 0 2024-04-12T20:56:11.992 INFO:tasks.workunit.client.0.smithi018.stdout:8/722: link db/d4e/d56/d59/c75 db/d25/ce5 0 2024-04-12T20:56:11.992 INFO:tasks.workunit.client.0.smithi018.stdout:3/514: creat d2/d6/d30/d31/fa4 x:0 0 0 2024-04-12T20:56:11.995 INFO:tasks.workunit.client.0.smithi018.stdout:8/723: mkdir db/d72/de6 0 2024-04-12T20:56:11.995 INFO:tasks.workunit.client.0.smithi018.stdout:9/687: creat d3/d6/d9/dc/fca x:0 0 0 2024-04-12T20:56:12.003 INFO:tasks.workunit.client.0.smithi018.stdout:3/515: creat d2/d6/d30/d31/d38/d3a/d76/fa5 x:0 0 0 2024-04-12T20:56:12.008 INFO:tasks.workunit.client.0.smithi018.stdout:3/516: stat d2/d6/d30/d31/d38/f3b 0 2024-04-12T20:56:12.026 INFO:tasks.workunit.client.0.smithi018.stdout:5/576: dread d0/d2/d33/d46/d18/f3b [0,4194304] 0 2024-04-12T20:56:12.026 INFO:tasks.workunit.client.0.smithi018.stdout:2/529: dread d5/d8/d10/f78 [0,4194304] 0 2024-04-12T20:56:12.026 INFO:tasks.workunit.client.0.smithi018.stdout:2/530: chown d5/d12 2279993 1 2024-04-12T20:56:12.028 INFO:tasks.workunit.client.0.smithi018.stdout:5/577: write d0/d2/d33/d46/d18/f71 [1106918,16878] 0 2024-04-12T20:56:12.035 INFO:tasks.workunit.client.0.smithi018.stdout:8/724: dread f7 [4194304,4194304] 0 2024-04-12T20:56:12.036 INFO:tasks.workunit.client.0.smithi018.stdout:1/443: dread d3/f4 [4194304,4194304] 0 2024-04-12T20:56:12.038 INFO:tasks.workunit.client.0.smithi018.stdout:2/531: creat d5/d8/d10/d19/d24/d39/da4/fbb x:0 0 0 2024-04-12T20:56:12.040 INFO:tasks.workunit.client.0.smithi018.stdout:1/444: write d3/d9/da/dd/f55 [667429,126993] 0 2024-04-12T20:56:12.041 INFO:tasks.workunit.client.0.smithi018.stdout:8/725: truncate db/d4e/d9a/f60 5106146 0 2024-04-12T20:56:12.042 INFO:tasks.workunit.client.0.smithi018.stdout:6/748: dread d5/df/f18 [4194304,4194304] 0 2024-04-12T20:56:12.043 INFO:tasks.workunit.client.0.smithi018.stdout:2/532: dread - d5/d7/d31/f7a zero size 2024-04-12T20:56:12.049 INFO:tasks.workunit.client.0.smithi018.stdout:2/533: write d5/d8/d10/f48 [5801424,107991] 0 2024-04-12T20:56:12.054 INFO:tasks.workunit.client.0.smithi018.stdout:9/688: dread d3/d6/d9/dc/d5a/d79/d7e/d9b/d21/f59 [0,4194304] 0 2024-04-12T20:56:12.054 INFO:tasks.workunit.client.0.smithi018.stdout:6/749: creat d5/d6/d1b/d3b/d35/dda/f11f x:0 0 0 2024-04-12T20:56:12.058 INFO:tasks.workunit.client.0.smithi018.stdout:2/534: getdents d5/d8/d10/d19/daf/d88 0 2024-04-12T20:56:12.060 INFO:tasks.workunit.client.0.smithi018.stdout:8/726: dread db/d4e/d56/d59/d76/de0/f5c [0,4194304] 0 2024-04-12T20:56:12.061 INFO:tasks.workunit.client.0.smithi018.stdout:3/517: dread d2/d6/d30/d31/d38/d3a/d4c/f79 [0,4194304] 0 2024-04-12T20:56:12.062 INFO:tasks.workunit.client.0.smithi018.stdout:5/578: dwrite d0/d2/d33/f15 [8388608,4194304] 0 2024-04-12T20:56:12.062 INFO:tasks.workunit.client.0.smithi018.stdout:2/535: chown d5/c2f 3121 1 2024-04-12T20:56:12.062 INFO:tasks.workunit.client.0.smithi018.stdout:9/689: mknod d3/d6/d9/dc/d13/d5b/d7c/ccb 0 2024-04-12T20:56:12.068 INFO:tasks.workunit.client.0.smithi018.stdout:2/536: mkdir d5/d12/d54/dbc 0 2024-04-12T20:56:12.074 INFO:tasks.workunit.client.0.smithi018.stdout:5/579: mkdir d0/d2/d33/d46/d35/d4d/db3/dc3 0 2024-04-12T20:56:12.075 INFO:tasks.workunit.client.0.smithi018.stdout:3/518: write d2/d4d/f57 [536329,54962] 0 2024-04-12T20:56:12.078 INFO:tasks.workunit.client.0.smithi018.stdout:1/445: dwrite d3/d9/da/dd/d31/d32/d13/d19/d2d/f76 [0,4194304] 0 2024-04-12T20:56:12.078 INFO:tasks.workunit.client.0.smithi018.stdout:8/727: rename db/d4e/d56/d59/d76/da5/c5b to db/d95/dd6/ce7 0 2024-04-12T20:56:12.085 INFO:tasks.workunit.client.0.smithi018.stdout:3/519: mknod d2/d6/d30/d31/d38/d3a/d4c/d6b/ca6 0 2024-04-12T20:56:12.087 INFO:tasks.workunit.client.0.smithi018.stdout:5/580: mknod d0/d2b/cc4 0 2024-04-12T20:56:12.087 INFO:tasks.workunit.client.0.smithi018.stdout:9/690: getdents d3/d93 0 2024-04-12T20:56:12.089 INFO:tasks.workunit.client.0.smithi018.stdout:8/728: rmdir db/d4e/d56/d59/d76/de0 39 2024-04-12T20:56:12.090 INFO:tasks.workunit.client.0.smithi018.stdout:1/446: write d3/d9/da/dd/d31/d32/d13/d19/f6e [1121957,11398] 0 2024-04-12T20:56:12.091 INFO:tasks.workunit.client.0.smithi018.stdout:3/520: symlink d2/d43/d61/la7 0 2024-04-12T20:56:12.095 INFO:tasks.workunit.client.0.smithi018.stdout:8/729: creat db/d4e/d73/fe8 x:0 0 0 2024-04-12T20:56:12.096 INFO:tasks.workunit.client.0.smithi018.stdout:3/521: truncate d2/d43/f8f 627756 0 2024-04-12T20:56:12.097 INFO:tasks.workunit.client.0.smithi018.stdout:1/447: fdatasync d3/d9/da/dd/d31/d32/d13/d19/d2d/d4e/f60 0 2024-04-12T20:56:12.101 INFO:tasks.workunit.client.0.smithi018.stdout:6/750: write d5/f48 [5612617,121521] 0 2024-04-12T20:56:12.108 INFO:tasks.workunit.client.0.smithi018.stdout:9/691: chown d3/d6/d9/dc/d46/f48 746 1 2024-04-12T20:56:12.108 INFO:tasks.workunit.client.0.smithi018.stdout:2/537: truncate d5/d7/d28/f59 2304443 0 2024-04-12T20:56:12.110 INFO:tasks.workunit.client.0.smithi018.stdout:3/522: rmdir d2/d6/d30/d31/d38/d3a 39 2024-04-12T20:56:12.115 INFO:tasks.workunit.client.0.smithi018.stdout:6/751: fsync d5/d6/d1b/ff1 0 2024-04-12T20:56:12.118 INFO:tasks.workunit.client.0.smithi018.stdout:1/448: link d3/d9/da/dd/f3f d3/d9/da/dd/d31/d32/d13/d19/f9a 0 2024-04-12T20:56:12.118 INFO:tasks.workunit.client.0.smithi018.stdout:6/752: chown d5/d71/le0 718 1 2024-04-12T20:56:12.118 INFO:tasks.workunit.client.0.smithi018.stdout:9/692: truncate d3/d6/d9/dc/d13/fc4 454633 0 2024-04-12T20:56:12.122 INFO:tasks.workunit.client.0.smithi018.stdout:8/730: truncate db/d4e/d56/d59/d76/de0/f53 2922842 0 2024-04-12T20:56:12.123 INFO:tasks.workunit.client.0.smithi018.stdout:3/523: creat d2/d6/fa8 x:0 0 0 2024-04-12T20:56:12.125 INFO:tasks.workunit.client.0.smithi018.stdout:8/731: readlink db/d4e/d73/la2 0 2024-04-12T20:56:12.126 INFO:tasks.workunit.client.0.smithi018.stdout:6/753: mkdir d5/df/d19/d120 0 2024-04-12T20:56:12.128 INFO:tasks.workunit.client.0.smithi018.stdout:3/524: write d2/d6/d30/f6f [1046876,51834] 0 2024-04-12T20:56:12.131 INFO:tasks.workunit.client.0.smithi018.stdout:5/581: dread d0/d2/d33/d10/f17 [0,4194304] 0 2024-04-12T20:56:12.132 INFO:tasks.workunit.client.0.smithi018.stdout:6/754: fdatasync d5/df/d19/d2b/f95 0 2024-04-12T20:56:12.135 INFO:tasks.workunit.client.0.smithi018.stdout:8/732: read - db/d4e/d7f/f89 zero size 2024-04-12T20:56:12.137 INFO:tasks.workunit.client.0.smithi018.stdout:1/449: truncate d3/d9/da/dd/f55 628850 0 2024-04-12T20:56:12.139 INFO:tasks.workunit.client.0.smithi018.stdout:1/450: stat d3/d9/f18 0 2024-04-12T20:56:12.143 INFO:tasks.workunit.client.0.smithi018.stdout:8/733: chown db/d4e/d56/l85 726 1 2024-04-12T20:56:12.144 INFO:tasks.workunit.client.0.smithi018.stdout:1/451: readlink d3/d9/da/l1c 0 2024-04-12T20:56:12.145 INFO:tasks.workunit.client.0.smithi018.stdout:5/582: creat d0/d2/d11/fc5 x:0 0 0 2024-04-12T20:56:12.146 INFO:tasks.workunit.client.0.smithi018.stdout:2/538: dwrite d5/d8/d10/d19/d24/f29 [0,4194304] 0 2024-04-12T20:56:12.149 INFO:tasks.workunit.client.0.smithi018.stdout:3/525: truncate d2/d6/d30/f37 521071 0 2024-04-12T20:56:12.151 INFO:tasks.workunit.client.0.smithi018.stdout:9/693: dwrite d3/d6/d9/dc/d13/d5b/d6b/f92 [0,4194304] 0 2024-04-12T20:56:12.152 INFO:tasks.workunit.client.0.smithi018.stdout:5/583: write d0/d2/d33/d46/d1e/faa [1035967,124413] 0 2024-04-12T20:56:12.152 INFO:tasks.workunit.client.0.smithi018.stdout:1/452: stat d3/d9/c50 0 2024-04-12T20:56:12.162 INFO:tasks.workunit.client.0.smithi018.stdout:5/584: mknod d0/d2/d33/d10/d60/cc6 0 2024-04-12T20:56:12.163 INFO:tasks.workunit.client.0.smithi018.stdout:2/539: creat d5/fbd x:0 0 0 2024-04-12T20:56:12.169 INFO:tasks.workunit.client.0.smithi018.stdout:1/453: mknod d3/d9/da/dd/d31/d32/d33/d80/c9b 0 2024-04-12T20:56:12.176 INFO:tasks.workunit.client.0.smithi018.stdout:5/585: truncate d0/d2/d33/d46/d1e/d55/d66/f93 235814 0 2024-04-12T20:56:12.177 INFO:tasks.workunit.client.0.smithi018.stdout:1/454: fsync d3/d9/da/dd/d31/d44/f5d 0 2024-04-12T20:56:12.177 INFO:tasks.workunit.client.0.smithi018.stdout:6/755: dwrite d5/df/d9b/ff6 [0,4194304] 0 2024-04-12T20:56:12.180 INFO:tasks.workunit.client.0.smithi018.stdout:5/586: readlink d0/l41 0 2024-04-12T20:56:12.181 INFO:tasks.workunit.client.0.smithi018.stdout:2/540: link d5/d7/d31/d62/f72 d5/d7/d31/fbe 0 2024-04-12T20:56:12.188 INFO:tasks.workunit.client.0.smithi018.stdout:9/694: dwrite d3/d6/d9/d3f/f6e [0,4194304] 0 2024-04-12T20:56:12.189 INFO:tasks.workunit.client.0.smithi018.stdout:1/455: readlink d3/d9/da/d14/l42 0 2024-04-12T20:56:12.193 INFO:tasks.workunit.client.0.smithi018.stdout:8/734: dwrite db/d4e/d56/d59/d76/da5/d45/d86/f9d [0,4194304] 0 2024-04-12T20:56:12.194 INFO:tasks.workunit.client.0.smithi018.stdout:3/526: truncate d2/d6/d30/f6f 734406 0 2024-04-12T20:56:12.205 INFO:tasks.workunit.client.0.smithi018.stdout:8/735: chown db/d4e/d56/d59/d76/ccc 56115567 1 2024-04-12T20:56:12.207 INFO:tasks.workunit.client.0.smithi018.stdout:3/527: dread - d2/d6/d30/d31/f69 zero size 2024-04-12T20:56:12.208 INFO:tasks.workunit.client.0.smithi018.stdout:5/587: getdents d0/d2/d11/d37/dc2 0 2024-04-12T20:56:12.210 INFO:tasks.workunit.client.0.smithi018.stdout:1/456: symlink d3/d9/da/dd/d31/d32/d13/d19/d2d/d4e/d5e/d97/l9c 0 2024-04-12T20:56:12.212 INFO:tasks.workunit.client.0.smithi018.stdout:9/695: write d3/d6/d9/d3f/f81 [233686,28173] 0 2024-04-12T20:56:12.213 INFO:tasks.workunit.client.0.smithi018.stdout:5/588: chown d0/d2/d33/d10/d26 56844 1 2024-04-12T20:56:12.215 INFO:tasks.workunit.client.0.smithi018.stdout:3/528: symlink d2/d6/d30/d31/la9 0 2024-04-12T20:56:12.269 INFO:tasks.workunit.client.0.smithi018.stdout:6/756: dread d5/df/d19/d43/d5f/fb4 [0,4194304] 0 2024-04-12T20:56:12.270 INFO:tasks.workunit.client.0.smithi018.stdout:9/696: write d3/d6/d9/dc/f2d [3298278,109916] 0 2024-04-12T20:56:12.270 INFO:tasks.workunit.client.0.smithi018.stdout:2/541: dwrite d5/d8/d10/f78 [0,4194304] 0 2024-04-12T20:56:12.273 INFO:tasks.workunit.client.0.smithi018.stdout:5/589: rename d0/d2/d11/f2a to d0/d2/d11/d37/dc2/d89/fc7 0 2024-04-12T20:56:12.274 INFO:tasks.workunit.client.0.smithi018.stdout:1/457: write d3/d9/da/dd/d31/d32/d13/d19/d2d/d4e/d58/d61/d71/f91 [4689441,88243] 0 2024-04-12T20:56:12.280 INFO:tasks.workunit.client.0.smithi018.stdout:6/757: creat d5/d6/d1b/d3b/d35/dda/f121 x:0 0 0 2024-04-12T20:56:12.284 INFO:tasks.workunit.client.0.smithi018.stdout:3/529: dwrite d2/f5 [0,4194304] 0 2024-04-12T20:56:12.293 INFO:tasks.workunit.client.0.smithi018.stdout:3/530: unlink d2/f78 0 2024-04-12T20:56:12.298 INFO:tasks.workunit.client.0.smithi018.stdout:5/590: dread d0/d2/d33/d10/f27 [0,4194304] 0 2024-04-12T20:56:12.305 INFO:tasks.workunit.client.0.smithi018.stdout:5/591: mknod d0/d2/d11/d37/dc2/cc8 0 2024-04-12T20:56:12.307 INFO:tasks.workunit.client.0.smithi018.stdout:9/697: truncate d3/d6/d9/dc/d5a/d79/d7e/d9b/d21/d41/d78/f83 336609 0 2024-04-12T20:56:12.310 INFO:tasks.workunit.client.0.smithi018.stdout:9/698: chown d3/d6/d9/d1e/f40 6505017 1 2024-04-12T20:56:12.310 INFO:tasks.workunit.client.0.smithi018.stdout:3/531: fdatasync d2/d6/d30/d31/d38/d3a/d76/fa5 0 2024-04-12T20:56:12.312 INFO:tasks.workunit.client.0.smithi018.stdout:5/592: mknod d0/d2/d11/d37/dc2/d89/cc9 0 2024-04-12T20:56:12.315 INFO:tasks.workunit.client.0.smithi018.stdout:5/593: read d0/d2/d33/f15 [10988999,121009] 0 2024-04-12T20:56:12.315 INFO:tasks.workunit.client.0.smithi018.stdout:1/458: dread d3/d9/da/dd/d31/d32/d13/d19/d2d/d4e/f4f [0,4194304] 0 2024-04-12T20:56:12.317 INFO:tasks.workunit.client.0.smithi018.stdout:1/459: fdatasync d3/d9/d11/f70 0 2024-04-12T20:56:12.319 INFO:tasks.workunit.client.0.smithi018.stdout:3/532: dread d2/d6/d30/f37 [0,4194304] 0 2024-04-12T20:56:12.321 INFO:tasks.workunit.client.0.smithi018.stdout:1/460: read d3/d9/f18 [1021044,105902] 0 2024-04-12T20:56:12.321 INFO:tasks.workunit.client.0.smithi018.stdout:9/699: symlink d3/d6/d9/dc/d5a/d79/d7e/d9b/d21/dc3/lcc 0 2024-04-12T20:56:12.321 INFO:tasks.workunit.client.0.smithi018.stdout:5/594: mknod d0/d2b/cca 0 2024-04-12T20:56:12.327 INFO:tasks.workunit.client.0.smithi018.stdout:5/595: write d0/d2/d33/f15 [4667316,103351] 0 2024-04-12T20:56:12.327 INFO:tasks.workunit.client.0.smithi018.stdout:3/533: creat d2/d43/d61/d64/faa x:0 0 0 2024-04-12T20:56:12.329 INFO:tasks.workunit.client.0.smithi018.stdout:9/700: creat d3/d6/d9/dc/d5a/d79/d7e/d9b/d21/dc2/fcd x:0 0 0 2024-04-12T20:56:12.329 INFO:tasks.workunit.client.0.smithi018.stdout:5/596: fdatasync d0/d2b/fa0 0 2024-04-12T20:56:12.333 INFO:tasks.workunit.client.0.smithi018.stdout:9/701: chown d3/d6/d9/dc/d13/d5b/ca0 471314520 1 2024-04-12T20:56:12.333 INFO:tasks.workunit.client.0.smithi018.stdout:1/461: dread d3/d9/da/dd/f17 [4194304,4194304] 0 2024-04-12T20:56:12.335 INFO:tasks.workunit.client.0.smithi018.stdout:9/702: chown d3/d93/daf/cb1 324 1 2024-04-12T20:56:12.337 INFO:tasks.workunit.client.0.smithi018.stdout:3/534: chown d2/c2c 95581247 1 2024-04-12T20:56:12.337 INFO:tasks.workunit.client.0.smithi018.stdout:5/597: write d0/d2/d33/d46/d1e/d2e/d7f/f8c [908933,33610] 0 2024-04-12T20:56:12.341 INFO:tasks.workunit.client.0.smithi018.stdout:3/535: creat d2/d4d/d8d/fab x:0 0 0 2024-04-12T20:56:12.344 INFO:tasks.workunit.client.0.smithi018.stdout:1/462: link d3/d9/da/dd/d31/d32/d13/d19/f9a d3/d9/da/dd/f9d 0 2024-04-12T20:56:12.346 INFO:tasks.workunit.client.0.smithi018.stdout:3/536: mkdir d2/d6/d34/da1/dac 0 2024-04-12T20:56:12.348 INFO:tasks.workunit.client.0.smithi018.stdout:1/463: truncate d3/d9/d11/f70 237672 0 2024-04-12T20:56:12.350 INFO:tasks.workunit.client.0.smithi018.stdout:3/537: unlink d2/f14 0 2024-04-12T20:56:12.350 INFO:tasks.workunit.client.0.smithi018.stdout:1/464: fdatasync d3/d9/da/dd/d31/d32/f34 0 2024-04-12T20:56:12.351 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:56:12 smithi018 ceph-mon[28799]: from='client.? 172.21.15.18:0/3484154226' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T20:56:12.358 INFO:tasks.workunit.client.0.smithi018.stdout:5/598: dwrite d0/d2/d11/d37/dc2/d89/fc7 [0,4194304] 0 2024-04-12T20:56:12.373 INFO:tasks.workunit.client.0.smithi018.stdout:1/465: dwrite d3/d9/d11/f70 [0,4194304] 0 2024-04-12T20:56:12.381 INFO:tasks.workunit.client.0.smithi018.stdout:3/538: dwrite d2/d6/d30/d31/d38/d3a/f45 [0,4194304] 0 2024-04-12T20:56:12.395 INFO:tasks.workunit.client.0.smithi018.stdout:5/599: rename d0/d2/d33/d46/d18/f3b to d0/d2/d33/d10/d2f/dbf/da3/dae/fcb 0 2024-04-12T20:56:12.411 INFO:tasks.workunit.client.0.smithi018.stdout:5/600: dread d0/d2/d33/d10/f27 [0,4194304] 0 2024-04-12T20:56:12.420 INFO:tasks.workunit.client.0.smithi018.stdout:5/601: creat d0/d2/d33/d46/d35/fcc x:0 0 0 2024-04-12T20:56:12.427 INFO:tasks.workunit.client.0.smithi018.stdout:3/539: dwrite d2/d6/d30/f58 [4194304,4194304] 0 2024-04-12T20:56:12.435 INFO:tasks.workunit.client.0.smithi018.stdout:5/602: creat d0/d2/d33/d46/d35/d4d/db3/dc3/fcd x:0 0 0 2024-04-12T20:56:12.436 INFO:tasks.workunit.client.0.smithi018.stdout:3/540: chown d2/d6/d30/d31/d38/d3a/d48/c6e 7590 1 2024-04-12T20:56:12.436 INFO:tasks.workunit.client.0.smithi018.stdout:5/603: readlink d0/l45 0 2024-04-12T20:56:12.445 INFO:tasks.workunit.client.0.smithi018.stdout:3/541: rmdir d2/d6/d3d/d54 39 2024-04-12T20:56:12.445 INFO:tasks.workunit.client.0.smithi018.stdout:5/604: rmdir d0/d2/d33/d10/d26 39 2024-04-12T20:56:12.447 INFO:tasks.workunit.client.0.smithi018.stdout:5/605: readlink d0/d2/d11/d37/lb0 0 2024-04-12T20:56:12.451 INFO:tasks.workunit.client.0.smithi018.stdout:3/542: mkdir d2/d6/d30/d31/d38/d3a/d76/dad 0 2024-04-12T20:56:12.462 INFO:tasks.workunit.client.0.smithi018.stdout:3/543: getdents d2/d6/d3d 0 2024-04-12T20:56:12.463 INFO:tasks.workunit.client.0.smithi018.stdout:3/544: stat d2/d6/d34 0 2024-04-12T20:56:12.465 INFO:tasks.workunit.client.0.smithi018.stdout:5/606: creat d0/d2/fce x:0 0 0 2024-04-12T20:56:12.469 INFO:tasks.workunit.client.0.smithi018.stdout:3/545: truncate d2/d43/d61/d64/faa 892420 0 2024-04-12T20:56:12.471 INFO:tasks.workunit.client.0.smithi018.stdout:1/466: unlink d3/d9/da/dd/d31/d32/d13/d19/d2d/f92 0 2024-04-12T20:56:12.472 INFO:tasks.workunit.client.0.smithi018.stdout:3/546: write d2/d6/d30/f3e [5629156,59443] 0 2024-04-12T20:56:12.476 INFO:tasks.workunit.client.0.smithi018.stdout:3/547: creat d2/d6/d34/fae x:0 0 0 2024-04-12T20:56:12.479 INFO:tasks.workunit.client.0.smithi018.stdout:5/607: rename d0/d2/d33/d10/d26/f85 to d0/d2/d33/d46/d1e/fcf 0 2024-04-12T20:56:12.482 INFO:tasks.workunit.client.0.smithi018.stdout:1/467: rename d3/c2f to d3/d9/da/dd/d31/d32/d13/d19/d2d/d4e/d5e/c9e 0 2024-04-12T20:56:12.484 INFO:tasks.workunit.client.0.smithi018.stdout:3/548: creat d2/d6/d30/d31/faf x:0 0 0 2024-04-12T20:56:12.485 INFO:tasks.workunit.client.0.smithi018.stdout:1/468: chown d3/d9/da/dd/d31/d32/l2e 0 1 2024-04-12T20:56:12.499 INFO:tasks.workunit.client.0.smithi018.stdout:5/608: dwrite d0/d2/d33/d46/f1d [0,4194304] 0 2024-04-12T20:56:12.502 INFO:tasks.workunit.client.0.smithi018.stdout:3/549: dwrite d2/d6/f84 [0,4194304] 0 2024-04-12T20:56:12.506 INFO:tasks.workunit.client.0.smithi018.stdout:1/469: dwrite d3/d9/da/dd/d31/d32/d13/d19/d2d/d4e/f4f [0,4194304] 0 2024-04-12T20:56:12.507 INFO:tasks.workunit.client.0.smithi018.stdout:5/609: mknod d0/d2/d11/d37/dc2/cd0 0 2024-04-12T20:56:12.508 INFO:tasks.workunit.client.0.smithi018.stdout:3/550: symlink d2/d6/d30/d31/d38/d3a/d4c/d6b/lb0 0 2024-04-12T20:56:12.515 INFO:tasks.workunit.client.0.smithi018.stdout:1/470: creat d3/d9/da/dd/d31/d32/d13/d19/d2d/d4e/d58/d61/d71/f9f x:0 0 0 2024-04-12T20:56:12.519 INFO:tasks.workunit.client.0.smithi018.stdout:5/610: rename d0/d2/d11/d37/dc2/cc8 to d0/d2/d33/d46/d35/d4d/db3/dc3/cd1 0 2024-04-12T20:56:12.523 INFO:tasks.workunit.client.0.smithi018.stdout:1/471: mkdir d3/d9/da/dd/d31/d32/d13/d19/d2d/d4e/d5e/da0 0 2024-04-12T20:56:12.529 INFO:tasks.workunit.client.0.smithi018.stdout:3/551: dwrite d2/d6/d30/f3e [0,4194304] 0 2024-04-12T20:56:12.530 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:56:12 smithi186 ceph-mon[36597]: from='client.? 172.21.15.18:0/3484154226' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T20:56:12.538 INFO:tasks.workunit.client.0.smithi018.stdout:3/552: rename d2/d6/f12 to d2/d6/d30/d31/d38/d3a/d48/d82/fb1 0 2024-04-12T20:56:12.544 INFO:tasks.workunit.client.0.smithi018.stdout:5/611: dwrite d0/d2/d33/d46/d18/f71 [0,4194304] 0 2024-04-12T20:56:12.546 INFO:tasks.workunit.client.0.smithi018.stdout:1/472: truncate d3/d9/da/dd/d31/f35 1010323 0 2024-04-12T20:56:12.547 INFO:tasks.workunit.client.0.smithi018.stdout:5/612: dread - d0/d5b/d84/f7d zero size 2024-04-12T20:56:12.548 INFO:tasks.workunit.client.0.smithi018.stdout:1/473: write d3/d9/d11/f70 [4477214,42178] 0 2024-04-12T20:56:12.553 INFO:tasks.workunit.client.0.smithi018.stdout:5/613: mkdir d0/d2/d33/d46/d35/dd2 0 2024-04-12T20:56:12.556 INFO:tasks.workunit.client.0.smithi018.stdout:5/614: chown d0/d2/d33/d46/d18/lb6 714518659 1 2024-04-12T20:56:12.565 INFO:tasks.workunit.client.0.smithi018.stdout:3/553: dwrite d2/d6/d34/d71/d7b/f7d [0,4194304] 0 2024-04-12T20:56:12.565 INFO:tasks.workunit.client.0.smithi018.stdout:5/615: creat d0/d2/d33/d46/d1e/d2e/fd3 x:0 0 0 2024-04-12T20:56:12.569 INFO:tasks.workunit.client.0.smithi018.stdout:3/554: unlink d2/d6/d30/f37 0 2024-04-12T20:56:12.571 INFO:tasks.workunit.client.0.smithi018.stdout:5/616: getdents d0/d2/d33/d46/d9e 0 2024-04-12T20:56:12.574 INFO:tasks.workunit.client.0.smithi018.stdout:3/555: unlink d2/d6/d30/d31/f98 0 2024-04-12T20:56:12.575 INFO:tasks.workunit.client.0.smithi018.stdout:5/617: write d0/d2/d33/d46/d1e/d2e/f8a [401933,37442] 0 2024-04-12T20:56:12.576 INFO:tasks.workunit.client.0.smithi018.stdout:3/556: write d2/d6/d30/d31/fa4 [781886,55265] 0 2024-04-12T20:56:12.579 INFO:tasks.workunit.client.0.smithi018.stdout:5/618: symlink d0/d2/d33/d46/d35/ld4 0 2024-04-12T20:56:12.581 INFO:tasks.workunit.client.0.smithi018.stdout:5/619: chown d0/d2/d11/l14 14145 1 2024-04-12T20:56:12.586 INFO:tasks.workunit.client.0.smithi018.stdout:0/491: sync 2024-04-12T20:56:12.586 INFO:tasks.workunit.client.0.smithi018.stdout:7/554: sync 2024-04-12T20:56:12.586 INFO:tasks.workunit.client.0.smithi018.stdout:4/602: sync 2024-04-12T20:56:12.587 INFO:tasks.workunit.client.0.smithi018.stdout:5/620: truncate d0/d2/fbe 652265 0 2024-04-12T20:56:12.587 INFO:tasks.workunit.client.0.smithi018.stdout:3/557: rename d2/d6/d30/d31/d38/d3a/d4c/d6b/d81 to d2/d6/d34/d71/db2 0 2024-04-12T20:56:12.589 INFO:tasks.workunit.client.0.smithi018.stdout:0/492: fsync da/d15/d3e/f5e 0 2024-04-12T20:56:12.592 INFO:tasks.workunit.client.0.smithi018.stdout:0/493: fdatasync da/d15/d17/d21/d36/d47/d81/f8d 0 2024-04-12T20:56:12.594 INFO:tasks.workunit.client.0.smithi018.stdout:5/621: creat d0/d2/d33/d10/d2f/d8f/fd5 x:0 0 0 2024-04-12T20:56:12.598 INFO:tasks.workunit.client.0.smithi018.stdout:0/494: creat da/d15/d17/d21/d94/fa4 x:0 0 0 2024-04-12T20:56:12.601 INFO:tasks.workunit.client.0.smithi018.stdout:4/603: link d3/db/c38 d3/d34/cc3 0 2024-04-12T20:56:12.603 INFO:tasks.workunit.client.0.smithi018.stdout:7/555: rename d1/db/d31/d30/f4a to d1/fba 0 2024-04-12T20:56:12.606 INFO:tasks.workunit.client.0.smithi018.stdout:0/495: mknod da/d15/d3e/d3f/ca5 0 2024-04-12T20:56:12.612 INFO:tasks.workunit.client.0.smithi018.stdout:5/622: creat d0/d5b/fd6 x:0 0 0 2024-04-12T20:56:12.612 INFO:tasks.workunit.client.0.smithi018.stdout:0/496: readlink da/d15/d17/d21/d36/d47/l95 0 2024-04-12T20:56:12.617 INFO:tasks.workunit.client.0.smithi018.stdout:4/604: creat d3/db/fc4 x:0 0 0 2024-04-12T20:56:12.617 INFO:tasks.workunit.client.0.smithi018.stdout:0/497: dread - da/d15/d17/d21/d36/d47/f8c zero size 2024-04-12T20:56:12.619 INFO:tasks.workunit.client.0.smithi018.stdout:5/623: write d0/d2/d33/d10/d2f/d8f/fd5 [818862,124308] 0 2024-04-12T20:56:12.620 INFO:tasks.workunit.client.0.smithi018.stdout:3/558: dread d2/d6/fc [0,4194304] 0 2024-04-12T20:56:12.624 INFO:tasks.workunit.client.0.smithi018.stdout:3/559: chown d2/d6/cd 0 1 2024-04-12T20:56:12.624 INFO:tasks.workunit.client.0.smithi018.stdout:5/624: rmdir d0/d5b/d84/d83 39 2024-04-12T20:56:12.625 INFO:tasks.workunit.client.0.smithi018.stdout:3/560: stat d2/d6/d3d 0 2024-04-12T20:56:12.626 INFO:tasks.workunit.client.0.smithi018.stdout:5/625: readlink d0/d5b/l78 0 2024-04-12T20:56:12.627 INFO:tasks.workunit.client.0.smithi018.stdout:7/556: dwrite d1/db/d31/d3a/d97/fa1 [0,4194304] 0 2024-04-12T20:56:12.630 INFO:tasks.workunit.client.0.smithi018.stdout:5/626: readlink d0/d2/d33/d10/d2f/dbf/da3/dae/lc1 0 2024-04-12T20:56:12.630 INFO:tasks.workunit.client.0.smithi018.stdout:0/498: truncate da/d15/d17/d21/d27/f4f 2780142 0 2024-04-12T20:56:12.632 INFO:tasks.workunit.client.0.smithi018.stdout:7/557: chown d1/db/d31/d3a/d42/d4c/d6c/fa6 1 1 2024-04-12T20:56:12.646 INFO:tasks.workunit.client.0.smithi018.stdout:0/499: truncate da/f75 110058 0 2024-04-12T20:56:12.647 INFO:tasks.workunit.client.0.smithi018.stdout:3/561: write d2/d6/d3d/d54/f9f [76265,91496] 0 2024-04-12T20:56:12.651 INFO:tasks.workunit.client.0.smithi018.stdout:3/562: write d2/d6/d30/f3e [5820093,58884] 0 2024-04-12T20:56:12.651 INFO:tasks.workunit.client.0.smithi018.stdout:5/627: dread d0/d2/d33/d46/d35/f4f [0,4194304] 0 2024-04-12T20:56:12.654 INFO:tasks.workunit.client.0.smithi018.stdout:5/628: read d0/d2/d33/d46/d35/f4f [418172,107146] 0 2024-04-12T20:56:12.656 INFO:tasks.workunit.client.0.smithi018.stdout:0/500: rename l6 to da/d15/d17/d21/d36/da2/la6 0 2024-04-12T20:56:12.656 INFO:tasks.workunit.client.0.smithi018.stdout:5/629: stat d0/d2/fbe 0 2024-04-12T20:56:12.657 INFO:tasks.workunit.client.0.smithi018.stdout:3/563: creat d2/d43/fb3 x:0 0 0 2024-04-12T20:56:12.670 INFO:tasks.workunit.client.0.smithi018.stdout:5/630: dread d0/d2/d33/d46/d1e/d2e/f8a [0,4194304] 0 2024-04-12T20:56:12.670 INFO:tasks.workunit.client.0.smithi018.stdout:7/558: dread d1/db/f43 [0,4194304] 0 2024-04-12T20:56:12.675 INFO:tasks.workunit.client.0.smithi018.stdout:7/559: chown d1/db/d31/d45/d8c/d9d/d70/d96/ca 25927493 1 2024-04-12T20:56:12.679 INFO:tasks.workunit.client.0.smithi018.stdout:7/560: dread - d1/db/d31/d45/d8c/d9d/d70/fb8 zero size 2024-04-12T20:56:12.681 INFO:tasks.workunit.client.0.smithi018.stdout:5/631: link d0/d2/d33/d10/d26/f43 d0/d2/d33/d46/d1e/fd7 0 2024-04-12T20:56:12.682 INFO:tasks.workunit.client.0.smithi018.stdout:7/561: chown d1/db/d31 7736799 1 2024-04-12T20:56:12.682 INFO:tasks.workunit.client.0.smithi018.stdout:3/564: dwrite d2/d6/d30/d31/d7c/f96 [0,4194304] 0 2024-04-12T20:56:12.684 INFO:tasks.workunit.client.0.smithi018.stdout:5/632: fdatasync d0/d2/d11/d37/dc2/f86 0 2024-04-12T20:56:12.685 INFO:tasks.workunit.client.0.smithi018.stdout:0/501: dread da/d15/d3e/d3f/f6c [0,4194304] 0 2024-04-12T20:56:12.685 INFO:tasks.workunit.client.0.smithi018.stdout:7/562: readlink d1/db/d31/d30/l50 0 2024-04-12T20:56:12.685 INFO:tasks.workunit.client.0.smithi018.stdout:3/565: chown d2/d6/d3d/f60 3199940 1 2024-04-12T20:56:12.687 INFO:tasks.workunit.client.0.smithi018.stdout:3/566: stat d2/d6/d34/f46 0 2024-04-12T20:56:12.689 INFO:tasks.workunit.client.0.smithi018.stdout:5/633: write d0/d2/d11/d37/dc2/f86 [1663053,126806] 0 2024-04-12T20:56:12.691 INFO:tasks.workunit.client.0.smithi018.stdout:3/567: write d2/d6/d30/d31/d38/d3a/f45 [3503078,61836] 0 2024-04-12T20:56:12.692 INFO:tasks.workunit.client.0.smithi018.stdout:0/502: creat da/d15/d17/d21/d36/d47/d81/fa7 x:0 0 0 2024-04-12T20:56:12.696 INFO:tasks.workunit.client.0.smithi018.stdout:0/503: readlink da/d15/d17/d21/d70/la1 0 2024-04-12T20:56:12.701 INFO:tasks.workunit.client.0.smithi018.stdout:7/563: write d1/db/d31/d3a/d42/faf [2612860,8567] 0 2024-04-12T20:56:12.703 INFO:tasks.workunit.client.0.smithi018.stdout:0/504: symlink da/d15/d4d/la8 0 2024-04-12T20:56:12.706 INFO:tasks.workunit.client.0.smithi018.stdout:7/564: unlink d1/f8e 0 2024-04-12T20:56:12.709 INFO:tasks.workunit.client.0.smithi018.stdout:0/505: read da/d15/d17/d21/d36/f80 [18515,72881] 0 2024-04-12T20:56:12.712 INFO:tasks.workunit.client.0.smithi018.stdout:7/565: symlink d1/db/d31/d45/d8c/lbb 0 2024-04-12T20:56:12.712 INFO:tasks.workunit.client.0.smithi018.stdout:5/634: dwrite d0/d2/f1a [0,4194304] 0 2024-04-12T20:56:12.712 INFO:tasks.workunit.client.0.smithi018.stdout:0/506: chown da/d15/d17/l1c 0 1 2024-04-12T20:56:12.715 INFO:tasks.workunit.client.0.smithi018.stdout:5/635: fdatasync d0/d2/d11/fc5 0 2024-04-12T20:56:12.717 INFO:tasks.workunit.client.0.smithi018.stdout:7/566: mkdir d1/db/d31/d3a/d42/d4c/dbc 0 2024-04-12T20:56:12.719 INFO:tasks.workunit.client.0.smithi018.stdout:3/568: dwrite d2/d6/d30/d31/d38/d3a/f45 [0,4194304] 0 2024-04-12T20:56:12.726 INFO:tasks.workunit.client.0.smithi018.stdout:0/507: rename da/d15/d17/d21/d70 to da/d15/d3e/da9 0 2024-04-12T20:56:12.727 INFO:tasks.workunit.client.0.smithi018.stdout:7/567: creat d1/db/d31/d3a/d42/fbd x:0 0 0 2024-04-12T20:56:12.735 INFO:tasks.workunit.client.0.smithi018.stdout:7/568: dread d1/f66 [4194304,4194304] 0 2024-04-12T20:56:12.741 INFO:tasks.workunit.client.0.smithi018.stdout:5/636: dwrite d0/d5b/d84/f65 [0,4194304] 0 2024-04-12T20:56:12.744 INFO:tasks.workunit.client.0.smithi018.stdout:7/569: rename d1/db/d31/l49 to d1/db/d31/d45/d8c/d9d/lbe 0 2024-04-12T20:56:12.748 INFO:tasks.workunit.client.0.smithi018.stdout:5/637: fdatasync d0/d2/d33/d46/d1e/d2e/f12 0 2024-04-12T20:56:12.773 INFO:tasks.workunit.client.0.smithi018.stdout:5/638: dwrite d0/d2/d33/d46/d1e/d55/d66/f93 [0,4194304] 0 2024-04-12T20:56:12.774 INFO:tasks.workunit.client.0.smithi018.stdout:3/569: dread d2/d6/f25 [0,4194304] 0 2024-04-12T20:56:12.776 INFO:tasks.workunit.client.0.smithi018.stdout:5/639: dread - d0/d2/d33/d10/d26/f9b zero size 2024-04-12T20:56:12.777 INFO:tasks.workunit.client.0.smithi018.stdout:0/508: dread da/d15/f33 [0,4194304] 0 2024-04-12T20:56:12.787 INFO:tasks.workunit.client.0.smithi018.stdout:3/570: rename d2/d6/l2d to d2/d6/d30/d31/d38/d3a/d76/lb4 0 2024-04-12T20:56:12.787 INFO:tasks.workunit.client.0.smithi018.stdout:5/640: fsync d0/d2/d33/d10/f27 0 2024-04-12T20:56:12.796 INFO:tasks.workunit.client.0.smithi018.stdout:0/509: write da/d15/d17/d21/d36/f63 [4685139,33876] 0 2024-04-12T20:56:12.804 INFO:tasks.workunit.client.0.smithi018.stdout:7/570: dread d1/db/d31/d45/d8c/d9d/d70/d96/f68 [0,4194304] 0 2024-04-12T20:56:12.807 INFO:tasks.workunit.client.0.smithi018.stdout:3/571: mkdir d2/d6/d30/d31/d38/d3a/d4c/db5 0 2024-04-12T20:56:12.808 INFO:tasks.workunit.client.0.smithi018.stdout:7/571: creat d1/db/d31/d30/fbf x:0 0 0 2024-04-12T20:56:12.810 INFO:tasks.workunit.client.0.smithi018.stdout:3/572: stat d2/d6/d30/d31/d38/d3a/d76/lb4 0 2024-04-12T20:56:12.811 INFO:tasks.workunit.client.0.smithi018.stdout:0/510: creat da/faa x:0 0 0 2024-04-12T20:56:12.814 INFO:tasks.workunit.client.0.smithi018.stdout:5/641: rename d0/d2/d33/d10/d9d to d0/d2/d11/dd8 0 2024-04-12T20:56:12.820 INFO:tasks.workunit.client.0.smithi018.stdout:0/511: dread da/d15/d3e/f73 [0,4194304] 0 2024-04-12T20:56:12.823 INFO:tasks.workunit.client.0.smithi018.stdout:0/512: chown da/d15/d4d/la8 7468 1 2024-04-12T20:56:12.828 INFO:tasks.workunit.client.0.smithi018.stdout:0/513: mknod da/d15/d17/d21/d36/d88/cab 0 2024-04-12T20:56:12.830 INFO:tasks.workunit.client.0.smithi018.stdout:3/573: rename d2/d6/d30/d31/d38/d3a/d4c to d2/d43/db6 0 2024-04-12T20:56:12.833 INFO:tasks.workunit.client.0.smithi018.stdout:5/642: dwrite d0/d2/d33/d46/d35/f62 [4194304,4194304] 0 2024-04-12T20:56:12.835 INFO:tasks.workunit.client.0.smithi018.stdout:7/572: rename d1/db/d31/d45/d8c/d9d/d70/d96/c5e to d1/db/d31/d45/d8c/d9d/d70/d96/d5f/da9/cc0 0 2024-04-12T20:56:12.835 INFO:tasks.workunit.client.0.smithi018.stdout:0/514: mkdir da/d15/d4d/d58/dac 0 2024-04-12T20:56:12.840 INFO:tasks.workunit.client.0.smithi018.stdout:5/643: write d0/d2/f1a [2832770,19847] 0 2024-04-12T20:56:12.842 INFO:tasks.workunit.client.0.smithi018.stdout:7/573: symlink d1/db/d31/d3a/lc1 0 2024-04-12T20:56:12.842 INFO:tasks.workunit.client.0.smithi018.stdout:5/644: fsync d0/d2/d11/d37/f47 0 2024-04-12T20:56:12.843 INFO:tasks.workunit.client.0.smithi018.stdout:3/574: rename d2/d6/d3d/f60 to d2/d6/d3d/d54/fb7 0 2024-04-12T20:56:12.847 INFO:tasks.workunit.client.0.smithi018.stdout:3/575: readlink d2/d6/d30/d31/d38/d3a/d48/l7a 0 2024-04-12T20:56:12.850 INFO:tasks.workunit.client.0.smithi018.stdout:7/574: truncate d1/db/d31/d3a/d42/faf 4183503 0 2024-04-12T20:56:12.854 INFO:tasks.workunit.client.0.smithi018.stdout:0/515: dread da/d15/f33 [0,4194304] 0 2024-04-12T20:56:12.857 INFO:tasks.workunit.client.0.smithi018.stdout:7/575: chown d1/c10 0 1 2024-04-12T20:56:12.860 INFO:tasks.workunit.client.0.smithi018.stdout:5/645: write d0/d2/d33/d46/d1e/d55/f82 [529531,130671] 0 2024-04-12T20:56:12.862 INFO:tasks.workunit.client.0.smithi018.stdout:0/516: readlink da/d15/d17/d5c/d79/l97 0 2024-04-12T20:56:12.866 INFO:tasks.workunit.client.0.smithi018.stdout:5/646: unlink d0/d2/d33/d46/d35/d4d/lc0 0 2024-04-12T20:56:12.867 INFO:tasks.workunit.client.0.smithi018.stdout:0/517: symlink da/d15/d17/d21/d36/d6e/lad 0 2024-04-12T20:56:12.868 INFO:tasks.workunit.client.0.smithi018.stdout:3/576: dwrite d2/f5 [4194304,4194304] 0 2024-04-12T20:56:12.871 INFO:tasks.workunit.client.0.smithi018.stdout:5/647: creat d0/d2/d33/d61/fd9 x:0 0 0 2024-04-12T20:56:12.873 INFO:tasks.workunit.client.0.smithi018.stdout:0/518: write da/d15/d3e/d3f/f6c [5177780,74608] 0 2024-04-12T20:56:12.878 INFO:tasks.workunit.client.0.smithi018.stdout:0/519: stat da/d15/d17/d21 0 2024-04-12T20:56:12.878 INFO:tasks.workunit.client.0.smithi018.stdout:5/648: mkdir d0/d2/d33/d10/d26/dda 0 2024-04-12T20:56:12.879 INFO:tasks.workunit.client.0.smithi018.stdout:3/577: symlink d2/d6/d30/d31/d38/d3a/d76/da2/lb8 0 2024-04-12T20:56:12.883 INFO:tasks.workunit.client.0.smithi018.stdout:8/736: sync 2024-04-12T20:56:12.886 INFO:tasks.workunit.client.0.smithi018.stdout:3/578: dread d2/d43/f8f [0,4194304] 0 2024-04-12T20:56:12.887 INFO:tasks.workunit.client.0.smithi018.stdout:8/737: creat db/d4e/d56/d59/d76/da5/d45/d86/da1/fe9 x:0 0 0 2024-04-12T20:56:12.892 INFO:tasks.workunit.client.0.smithi018.stdout:8/738: dread - db/d4e/d56/d59/d76/da5/d45/d86/da1/fb6 zero size 2024-04-12T20:56:12.892 INFO:tasks.workunit.client.0.smithi018.stdout:0/520: getdents da/d15/d17/d21/d36/d6e 0 2024-04-12T20:56:12.895 INFO:tasks.workunit.client.0.smithi018.stdout:5/649: write d0/d2/d33/d46/d35/d4d/f80 [115873,31903] 0 2024-04-12T20:56:12.897 INFO:tasks.workunit.client.0.smithi018.stdout:0/521: mkdir da/d15/d17/d5c/dae 0 2024-04-12T20:56:12.897 INFO:tasks.workunit.client.0.smithi018.stdout:5/650: chown d0/d2/d33/d46/d1e/d9a 32476 1 2024-04-12T20:56:12.900 INFO:tasks.workunit.client.0.smithi018.stdout:0/522: fdatasync da/d15/d3e/d3f/f59 0 2024-04-12T20:56:12.900 INFO:tasks.workunit.client.0.smithi018.stdout:5/651: readlink d0/d2b/l38 0 2024-04-12T20:56:12.905 INFO:tasks.workunit.client.0.smithi018.stdout:5/652: symlink d0/d2/d33/d46/d35/d4d/ldb 0 2024-04-12T20:56:12.906 INFO:tasks.workunit.client.0.smithi018.stdout:8/739: write db/d4e/f4f [542050,53288] 0 2024-04-12T20:56:12.910 INFO:tasks.workunit.client.0.smithi018.stdout:3/579: dwrite d2/f19 [0,4194304] 0 2024-04-12T20:56:12.910 INFO:tasks.workunit.client.0.smithi018.stdout:8/740: write db/d4e/d7f/f93 [590218,92524] 0 2024-04-12T20:56:12.911 INFO:tasks.workunit.client.0.smithi018.stdout:5/653: mknod d0/d2/d11/d74/cdc 0 2024-04-12T20:56:12.912 INFO:tasks.workunit.client.0.smithi018.stdout:5/654: readlink d0/d2b/l38 0 2024-04-12T20:56:12.913 INFO:tasks.workunit.client.0.smithi018.stdout:3/580: fdatasync d2/d43/d61/d64/faa 0 2024-04-12T20:56:12.917 INFO:tasks.workunit.client.0.smithi018.stdout:8/741: mknod db/d4e/d56/d59/d76/da5/d45/da9/cea 0 2024-04-12T20:56:12.919 INFO:tasks.workunit.client.0.smithi018.stdout:0/523: dread da/d15/f33 [0,4194304] 0 2024-04-12T20:56:12.921 INFO:tasks.workunit.client.0.smithi018.stdout:3/581: write d2/d83/f92 [4459825,83411] 0 2024-04-12T20:56:12.921 INFO:tasks.workunit.client.0.smithi018.stdout:8/742: write db/f68 [738236,53411] 0 2024-04-12T20:56:12.928 INFO:tasks.workunit.client.0.smithi018.stdout:3/582: creat d2/d43/db6/d6b/fb9 x:0 0 0 2024-04-12T20:56:12.928 INFO:tasks.workunit.client.0.smithi018.stdout:8/743: symlink db/d95/dd6/leb 0 2024-04-12T20:56:12.931 INFO:tasks.workunit.client.0.smithi018.stdout:0/524: creat da/d15/d17/d21/d36/d47/faf x:0 0 0 2024-04-12T20:56:12.934 INFO:tasks.workunit.client.0.smithi018.stdout:0/525: write da/fc [2393427,83757] 0 2024-04-12T20:56:12.944 INFO:tasks.workunit.client.0.smithi018.stdout:0/526: creat da/d15/d17/d21/d36/fb0 x:0 0 0 2024-04-12T20:56:12.953 INFO:tasks.workunit.client.0.smithi018.stdout:8/744: dwrite db/d4e/d56/d59/d76/de0/d27/fd8 [0,4194304] 0 2024-04-12T20:56:12.954 INFO:tasks.workunit.client.0.smithi018.stdout:0/527: truncate da/d15/d17/d21/d27/f78 6133305 0 2024-04-12T20:56:12.957 INFO:tasks.workunit.client.0.smithi018.stdout:5/655: dread d0/d2/d33/d10/d26/f4a [0,4194304] 0 2024-04-12T20:56:12.957 INFO:tasks.workunit.client.0.smithi018.stdout:8/745: chown db/d4e/d56/d59/d76/da5/d45/d86/da1 7 1 2024-04-12T20:56:12.958 INFO:tasks.workunit.client.0.smithi018.stdout:3/583: truncate d2/d4d/f57 167785 0 2024-04-12T20:56:12.965 INFO:tasks.workunit.client.0.smithi018.stdout:3/584: link d2/d6/c35 d2/d83/cba 0 2024-04-12T20:56:12.968 INFO:tasks.workunit.client.0.smithi018.stdout:3/585: symlink d2/d6/d34/lbb 0 2024-04-12T20:56:12.974 INFO:tasks.workunit.client.0.smithi018.stdout:0/528: dwrite da/faa [0,4194304] 0 2024-04-12T20:56:12.994 INFO:tasks.workunit.client.0.smithi018.stdout:0/529: dwrite da/d15/d3e/d3f/f8f [0,4194304] 0 2024-04-12T20:56:13.000 INFO:tasks.workunit.client.0.smithi018.stdout:3/586: truncate d2/d4d/f57 302553 0 2024-04-12T20:56:13.005 INFO:tasks.workunit.client.0.smithi018.stdout:3/587: mkdir d2/d6/d34/d71/dbc 0 2024-04-12T20:56:13.009 INFO:tasks.workunit.client.0.smithi018.stdout:3/588: mknod d2/d43/db6/d6b/cbd 0 2024-04-12T20:56:13.025 INFO:tasks.workunit.client.0.smithi018.stdout:0/530: dread da/f38 [4194304,4194304] 0 2024-04-12T20:56:13.028 INFO:tasks.workunit.client.0.smithi018.stdout:0/531: mknod da/d15/d17/d21/d94/cb1 0 2024-04-12T20:56:13.039 INFO:tasks.workunit.client.0.smithi018.stdout:3/589: dread d2/d43/d61/f63 [0,4194304] 0 2024-04-12T20:56:13.043 INFO:tasks.workunit.client.0.smithi018.stdout:3/590: chown d2/d6/d30/d31/d38/d3a/d76/da2/lb8 1000497 1 2024-04-12T20:56:13.044 INFO:tasks.workunit.client.0.smithi018.stdout:0/532: dwrite da/d15/d3e/d3f/f69 [0,4194304] 0 2024-04-12T20:56:13.047 INFO:tasks.workunit.client.0.smithi018.stdout:3/591: chown d2/d6/d3d/d54/c7e 64863458 1 2024-04-12T20:56:13.048 INFO:tasks.workunit.client.0.smithi018.stdout:3/592: chown d2/d6/d30/f3e 34 1 2024-04-12T20:56:13.052 INFO:tasks.workunit.client.0.smithi018.stdout:3/593: mknod d2/d6/d3d/d54/cbe 0 2024-04-12T20:56:13.054 INFO:tasks.workunit.client.0.smithi018.stdout:3/594: write d2/d6/d3d/d54/f9f [955897,113864] 0 2024-04-12T20:56:13.059 INFO:tasks.workunit.client.0.smithi018.stdout:0/533: fdatasync da/d15/d17/d21/d27/f78 0 2024-04-12T20:56:13.062 INFO:tasks.workunit.client.0.smithi018.stdout:0/534: fsync da/d15/d17/d21/d36/d47/f5d 0 2024-04-12T20:56:13.065 INFO:tasks.workunit.client.0.smithi018.stdout:0/535: read da/d15/d17/d5c/f82 [7475,85172] 0 2024-04-12T20:56:13.068 INFO:tasks.workunit.client.0.smithi018.stdout:0/536: write da/d15/f8e [766391,92204] 0 2024-04-12T20:56:13.068 INFO:tasks.workunit.client.0.smithi018.stdout:3/595: rename d2/d6/d3d to d2/d43/d61/d64/dbf 0 2024-04-12T20:56:13.071 INFO:tasks.workunit.client.0.smithi018.stdout:0/537: rmdir da/d15/d17/d21/d36/d88 39 2024-04-12T20:56:13.071 INFO:tasks.workunit.client.0.smithi018.stdout:3/596: write d2/d43/d61/d64/dbf/f87 [1006595,33865] 0 2024-04-12T20:56:13.072 INFO:tasks.workunit.client.0.smithi018.stdout:0/538: chown da/d15/f90 8 1 2024-04-12T20:56:13.074 INFO:tasks.workunit.client.0.smithi018.stdout:3/597: write d2/d6/d34/d71/d7b/f7d [762302,59357] 0 2024-04-12T20:56:13.084 INFO:tasks.workunit.client.0.smithi018.stdout:0/539: dread da/d15/d17/d21/d36/f7c [0,4194304] 0 2024-04-12T20:56:13.087 INFO:tasks.workunit.client.0.smithi018.stdout:0/540: symlink da/d15/d4d/d58/dac/lb2 0 2024-04-12T20:56:13.092 INFO:tasks.workunit.client.0.smithi018.stdout:3/598: dwrite d2/d43/d61/f62 [0,4194304] 0 2024-04-12T20:56:13.095 INFO:tasks.workunit.client.0.smithi018.stdout:0/541: rename da/d15/d3e/d3f/ca5 to da/d15/d3e/d83/cb3 0 2024-04-12T20:56:13.096 INFO:tasks.workunit.client.0.smithi018.stdout:3/599: mknod d2/d43/d61/cc0 0 2024-04-12T20:56:13.098 INFO:tasks.workunit.client.0.smithi018.stdout:0/542: symlink da/d15/d3e/d83/lb4 0 2024-04-12T20:56:13.100 INFO:tasks.workunit.client.0.smithi018.stdout:3/600: write d2/d43/db6/f80 [2202376,112600] 0 2024-04-12T20:56:13.100 INFO:tasks.workunit.client.0.smithi018.stdout:0/543: chown da/d15/d4d/la8 0 1 2024-04-12T20:56:13.104 INFO:tasks.workunit.client.0.smithi018.stdout:0/544: write da/d15/d17/d21/d36/f4c [2348219,126606] 0 2024-04-12T20:56:13.107 INFO:tasks.workunit.client.0.smithi018.stdout:0/545: mkdir da/d15/d17/d21/d36/d47/db5 0 2024-04-12T20:56:13.107 INFO:tasks.workunit.client.0.smithi018.stdout:3/601: read d2/d6/f1b [3538673,41992] 0 2024-04-12T20:56:13.109 INFO:tasks.workunit.client.0.smithi018.stdout:3/602: readlink d2/d43/d61/d64/dbf/d70/la3 0 2024-04-12T20:56:13.110 INFO:tasks.workunit.client.0.smithi018.stdout:0/546: mknod da/d15/d4d/d58/cb6 0 2024-04-12T20:56:13.113 INFO:tasks.workunit.client.0.smithi018.stdout:3/603: fdatasync d2/d6/d30/d31/d38/d3a/f45 0 2024-04-12T20:56:13.115 INFO:tasks.workunit.client.0.smithi018.stdout:0/547: dread da/d15/d3e/d3f/f59 [0,4194304] 0 2024-04-12T20:56:13.115 INFO:tasks.workunit.client.0.smithi018.stdout:3/604: readlink d2/d6/d30/d31/d38/d3a/l9a 0 2024-04-12T20:56:13.116 INFO:tasks.workunit.client.0.smithi018.stdout:0/548: chown da/d15/d3e/d3f 21 1 2024-04-12T20:56:13.128 INFO:tasks.workunit.client.0.smithi018.stdout:0/549: dread da/d15/d3e/d3f/f6c [0,4194304] 0 2024-04-12T20:56:13.130 INFO:tasks.workunit.client.0.smithi018.stdout:3/605: dwrite d2/d43/d61/d64/faa [0,4194304] 0 2024-04-12T20:56:13.138 INFO:tasks.workunit.client.0.smithi018.stdout:3/606: mkdir d2/d6/d30/d31/d38/d3a/d76/dad/dc1 0 2024-04-12T20:56:13.141 INFO:tasks.workunit.client.0.smithi018.stdout:3/607: fsync d2/f5 0 2024-04-12T20:56:13.153 INFO:tasks.workunit.client.0.smithi018.stdout:3/608: truncate d2/d43/d61/f63 1637518 0 2024-04-12T20:56:13.155 INFO:tasks.workunit.client.0.smithi018.stdout:0/550: dwrite da/d15/d17/d21/f5f [0,4194304] 0 2024-04-12T20:56:13.166 INFO:tasks.workunit.client.0.smithi018.stdout:3/609: write d2/f5d [3482535,20184] 0 2024-04-12T20:56:13.166 INFO:tasks.workunit.client.0.smithi018.stdout:0/551: write da/d15/d4d/d6b/f6d [3121779,66665] 0 2024-04-12T20:56:13.167 INFO:tasks.workunit.client.0.smithi018.stdout:4/605: sync 2024-04-12T20:56:13.167 INFO:tasks.workunit.client.0.smithi018.stdout:6/758: sync 2024-04-12T20:56:13.167 INFO:tasks.workunit.client.0.smithi018.stdout:1/474: sync 2024-04-12T20:56:13.167 INFO:tasks.workunit.client.0.smithi018.stdout:2/542: sync 2024-04-12T20:56:13.168 INFO:tasks.workunit.client.0.smithi018.stdout:9/703: sync 2024-04-12T20:56:13.170 INFO:tasks.workunit.client.0.smithi018.stdout:3/610: fdatasync d2/d6/d30/d31/f67 0 2024-04-12T20:56:13.175 INFO:tasks.workunit.client.0.smithi018.stdout:0/552: write da/f38 [13428945,46101] 0 2024-04-12T20:56:13.175 INFO:tasks.workunit.client.0.smithi018.stdout:2/543: creat d5/d8/d10/d19/d7c/fbf x:0 0 0 2024-04-12T20:56:13.179 INFO:tasks.workunit.client.0.smithi018.stdout:4/606: rename d3/d34 to d3/d5d/d60/dc5 0 2024-04-12T20:56:13.181 INFO:tasks.workunit.client.0.smithi018.stdout:6/759: mkdir d5/d6/d1b/d3b/d35/d4f/df3/d105/d122 0 2024-04-12T20:56:13.181 INFO:tasks.workunit.client.0.smithi018.stdout:3/611: mknod d2/d6/d30/d31/d38/d3a/d48/cc2 0 2024-04-12T20:56:13.182 INFO:tasks.workunit.client.0.smithi018.stdout:4/607: readlink d3/d5d/d60/d64/l70 0 2024-04-12T20:56:13.182 INFO:tasks.workunit.client.0.smithi018.stdout:1/475: dread d3/d9/da/f20 [0,4194304] 0 2024-04-12T20:56:13.183 INFO:tasks.workunit.client.0.smithi018.stdout:9/704: creat d3/d6/d9/dc/d5a/d79/dac/fce x:0 0 0 2024-04-12T20:56:13.184 INFO:tasks.workunit.client.0.smithi018.stdout:2/544: mkdir d5/d12/d54/dc0 0 2024-04-12T20:56:13.188 INFO:tasks.workunit.client.0.smithi018.stdout:1/476: chown d3/d9/da/dd 6886 1 2024-04-12T20:56:13.188 INFO:tasks.workunit.client.0.smithi018.stdout:6/760: creat d5/df/d19/d2b/f123 x:0 0 0 2024-04-12T20:56:13.188 INFO:tasks.workunit.client.0.smithi018.stdout:0/553: symlink da/d15/d17/d21/d36/d47/db5/lb7 0 2024-04-12T20:56:13.190 INFO:tasks.workunit.client.0.smithi018.stdout:2/545: truncate d5/d7/f4d 138036 0 2024-04-12T20:56:13.191 INFO:tasks.workunit.client.0.smithi018.stdout:9/705: creat d3/d6/d9/dc/d5a/d79/d7e/d9b/d21/dc2/fcf x:0 0 0 2024-04-12T20:56:13.195 INFO:tasks.workunit.client.0.smithi018.stdout:9/706: symlink d3/d93/ld0 0 2024-04-12T20:56:13.195 INFO:tasks.workunit.client.0.smithi018.stdout:4/608: rename d3/d32/d33/f91 to d3/fc6 0 2024-04-12T20:56:13.196 INFO:tasks.workunit.client.0.smithi018.stdout:1/477: symlink d3/d9/da/dd/d31/la1 0 2024-04-12T20:56:13.198 INFO:tasks.workunit.client.0.smithi018.stdout:0/554: write da/d15/d3e/d3f/f6c [3979175,53376] 0 2024-04-12T20:56:13.201 INFO:tasks.workunit.client.0.smithi018.stdout:4/609: mkdir d3/d5d/d60/d64/d4f/dc7 0 2024-04-12T20:56:13.205 INFO:tasks.workunit.client.0.smithi018.stdout:1/478: mknod d3/d9/da/dd/d26/ca2 0 2024-04-12T20:56:13.207 INFO:tasks.workunit.client.0.smithi018.stdout:3/612: dwrite d2/d6/d30/d31/f69 [0,4194304] 0 2024-04-12T20:56:13.210 INFO:tasks.workunit.client.0.smithi018.stdout:0/555: write da/d15/d17/d21/d36/d6e/f7f [794418,61522] 0 2024-04-12T20:56:13.215 INFO:tasks.workunit.client.0.smithi018.stdout:4/610: symlink d3/db/lc8 0 2024-04-12T20:56:13.216 INFO:tasks.workunit.client.0.smithi018.stdout:2/546: link d5/d8/d10/d19/d24/d39/l49 d5/d8/d10/d19/d7c/lc1 0 2024-04-12T20:56:13.221 INFO:tasks.workunit.client.0.smithi018.stdout:6/761: dwrite d5/df/d19/d43/fc3 [0,4194304] 0 2024-04-12T20:56:13.223 INFO:tasks.workunit.client.0.smithi018.stdout:3/613: mknod d2/d43/cc3 0 2024-04-12T20:56:13.225 INFO:tasks.workunit.client.0.smithi018.stdout:0/556: truncate da/f13 369592 0 2024-04-12T20:56:13.227 INFO:tasks.workunit.client.0.smithi018.stdout:3/614: write d2/f5d [4387773,117400] 0 2024-04-12T20:56:13.228 INFO:tasks.workunit.client.0.smithi018.stdout:9/707: dwrite d3/d6/d9/dc/f2d [0,4194304] 0 2024-04-12T20:56:13.231 INFO:tasks.workunit.client.0.smithi018.stdout:4/611: creat d3/d5d/d60/dc5/d43/fc9 x:0 0 0 2024-04-12T20:56:13.235 INFO:tasks.workunit.client.0.smithi018.stdout:2/547: readlink d5/d8/d10/d19/l27 0 2024-04-12T20:56:13.236 INFO:tasks.workunit.client.0.smithi018.stdout:4/612: chown d3/d5d/d60/d64/d4f/f95 39273435 1 2024-04-12T20:56:13.236 INFO:tasks.workunit.client.0.smithi018.stdout:1/479: symlink d3/d9/da/dd/d31/d32/d13/d19/d2d/d4e/d7a/d82/la3 0 2024-04-12T20:56:13.237 INFO:tasks.workunit.client.0.smithi018.stdout:6/762: rmdir d5/d71 39 2024-04-12T20:56:13.240 INFO:tasks.workunit.client.0.smithi018.stdout:1/480: chown d3/d9/da/dd/d31/d32/d33 47208280 1 2024-04-12T20:56:13.241 INFO:tasks.workunit.client.0.smithi018.stdout:9/708: mknod d3/d6/d9/dc/d46/cd1 0 2024-04-12T20:56:13.242 INFO:tasks.workunit.client.0.smithi018.stdout:3/615: rmdir d2/d43/d61/d64/dbf/d54 39 2024-04-12T20:56:13.243 INFO:tasks.workunit.client.0.smithi018.stdout:2/548: creat d5/d7/d31/d62/fc2 x:0 0 0 2024-04-12T20:56:13.245 INFO:tasks.workunit.client.0.smithi018.stdout:2/549: stat d5/d12/lad 0 2024-04-12T20:56:13.249 INFO:tasks.workunit.client.0.smithi018.stdout:4/613: getdents d3/d5d/d60/dc5/d3d/d80 0 2024-04-12T20:56:13.253 INFO:tasks.workunit.client.0.smithi018.stdout:4/614: dread - d3/d32/f4e zero size 2024-04-12T20:56:13.257 INFO:tasks.workunit.client.0.smithi018.stdout:9/709: unlink d3/d6/d9/dc/d13/d5b/c64 0 2024-04-12T20:56:13.257 INFO:tasks.workunit.client.0.smithi018.stdout:1/481: rename d3/d9/da/dd/d31/d32/d13/d19/d2d/f8b to d3/d9/da/dd/d31/d32/d13/d19/d2d/d4e/d7a/fa4 0 2024-04-12T20:56:13.266 INFO:tasks.workunit.client.0.smithi018.stdout:4/615: read d3/d32/d33/f7d [322643,37291] 0 2024-04-12T20:56:13.270 INFO:tasks.workunit.client.0.smithi018.stdout:6/763: dwrite d5/d6/d1b/fec [0,4194304] 0 2024-04-12T20:56:13.274 INFO:tasks.workunit.client.0.smithi018.stdout:4/616: creat d3/d32/d33/db6/fca x:0 0 0 2024-04-12T20:56:13.274 INFO:tasks.workunit.client.0.smithi018.stdout:3/616: dwrite d2/d6/d30/f3e [4194304,4194304] 0 2024-04-12T20:56:13.275 INFO:tasks.workunit.client.0.smithi018.stdout:2/550: write d5/d8/d10/d19/f7b [2230971,82126] 0 2024-04-12T20:56:13.277 INFO:tasks.workunit.client.0.smithi018.stdout:4/617: readlink d3/d5d/l77 0 2024-04-12T20:56:13.277 INFO:tasks.workunit.client.0.smithi018.stdout:9/710: creat d3/d6/d9/dc/d13/fd2 x:0 0 0 2024-04-12T20:56:13.284 INFO:tasks.workunit.client.0.smithi018.stdout:2/551: chown d5/d8/d33/l47 16023 1 2024-04-12T20:56:13.284 INFO:tasks.workunit.client.0.smithi018.stdout:9/711: chown d3/d6/d3d/c61 79980159 1 2024-04-12T20:56:13.284 INFO:tasks.workunit.client.0.smithi018.stdout:6/764: mkdir d5/d15/d60/d124 0 2024-04-12T20:56:13.284 INFO:tasks.workunit.client.0.smithi018.stdout:4/618: truncate d3/db/fc4 390367 0 2024-04-12T20:56:13.288 INFO:tasks.workunit.client.0.smithi018.stdout:2/552: fdatasync d5/d8/d10/d19/d7c/f9f 0 2024-04-12T20:56:13.289 INFO:tasks.workunit.client.0.smithi018.stdout:6/765: unlink d5/f48 0 2024-04-12T20:56:13.295 INFO:tasks.workunit.client.0.smithi018.stdout:1/482: dwrite d3/d9/da/dd/f17 [4194304,4194304] 0 2024-04-12T20:56:13.299 INFO:tasks.workunit.client.0.smithi018.stdout:6/766: truncate d5/d6/d1b/d3b/d6a/dd8/f102 654166 0 2024-04-12T20:56:13.303 INFO:tasks.workunit.client.0.smithi018.stdout:2/553: getdents d5/d7/d31/d62 0 2024-04-12T20:56:13.303 INFO:tasks.workunit.client.0.smithi018.stdout:1/483: creat d3/d9/d11/fa5 x:0 0 0 2024-04-12T20:56:13.308 INFO:tasks.workunit.client.0.smithi018.stdout:4/619: write d3/d32/d94/f96 [1007832,114351] 0 2024-04-12T20:56:13.315 INFO:tasks.workunit.client.0.smithi018.stdout:1/484: creat d3/d9/da/dd/d26/fa6 x:0 0 0 2024-04-12T20:56:13.325 INFO:tasks.workunit.client.0.smithi018.stdout:1/485: creat d3/d9/da/dd/d31/d32/d13/d19/d2d/d4e/d5e/fa7 x:0 0 0 2024-04-12T20:56:13.330 INFO:tasks.workunit.client.0.smithi018.stdout:4/620: write d3/d32/d66/f75 [1741862,70317] 0 2024-04-12T20:56:13.336 INFO:tasks.workunit.client.0.smithi018.stdout:2/554: dwrite d5/d12/f58 [0,4194304] 0 2024-04-12T20:56:13.339 INFO:tasks.workunit.client.0.smithi018.stdout:4/621: rename d3/d32/d33/d8e/fa1 to d3/d32/d94/fcb 0 2024-04-12T20:56:13.341 INFO:tasks.workunit.client.0.smithi018.stdout:9/712: dread d3/d6/d9/dc/d13/d5b/d6b/f92 [0,4194304] 0 2024-04-12T20:56:13.341 INFO:tasks.workunit.client.0.smithi018.stdout:2/555: dread - d5/d8/f57 zero size 2024-04-12T20:56:13.346 INFO:tasks.workunit.client.0.smithi018.stdout:6/767: dread d5/df/d19/f6c [0,4194304] 0 2024-04-12T20:56:13.349 INFO:tasks.workunit.client.0.smithi018.stdout:9/713: chown d3/d6/d43/f8f 639602900 1 2024-04-12T20:56:13.355 INFO:tasks.workunit.client.0.smithi018.stdout:6/768: mkdir d5/d6/d90/d125 0 2024-04-12T20:56:13.358 INFO:tasks.workunit.client.0.smithi018.stdout:1/486: dwrite d3/d9/da/f5f [0,4194304] 0 2024-04-12T20:56:13.360 INFO:tasks.workunit.client.0.smithi018.stdout:2/556: truncate d5/d8/d10/d19/d24/d39/da4/fbb 199457 0 2024-04-12T20:56:13.361 INFO:tasks.workunit.client.0.smithi018.stdout:9/714: read d3/d6/d9/d3f/f50 [3102564,38678] 0 2024-04-12T20:56:13.366 INFO:tasks.workunit.client.0.smithi018.stdout:2/557: rename d5/d8/d10/f70 to d5/d12/d54/dbc/fc3 0 2024-04-12T20:56:13.369 INFO:tasks.workunit.client.0.smithi018.stdout:1/487: mkdir d3/d9/da/dd/d31/d32/d13/d19/d2d/d4e/d58/d61/d71/da8 0 2024-04-12T20:56:13.369 INFO:tasks.workunit.client.0.smithi018.stdout:9/715: truncate d3/d6/d9/d3f/f50 4065328 0 2024-04-12T20:56:13.369 INFO:tasks.workunit.client.0.smithi018.stdout:2/558: symlink d5/d12/d54/dbc/lc4 0 2024-04-12T20:56:13.372 INFO:tasks.workunit.client.0.smithi018.stdout:9/716: chown d3/d6/d9/dc/d46/cb6 111036223 1 2024-04-12T20:56:13.383 INFO:tasks.workunit.client.0.smithi018.stdout:1/488: write d3/d9/da/dd/d31/d32/f1b [1434201,86029] 0 2024-04-12T20:56:13.390 INFO:tasks.workunit.client.0.smithi018.stdout:1/489: creat d3/d9/da/dd/d31/d32/d13/d19/d2d/d4e/d58/d61/d71/fa9 x:0 0 0 2024-04-12T20:56:13.392 INFO:tasks.workunit.client.0.smithi018.stdout:9/717: link d3/d6/d9/dc/d5a/d79/d7e/d9b/f73 d3/d6/d9/dc/d6f/d77/fd3 0 2024-04-12T20:56:13.405 INFO:tasks.workunit.client.0.smithi018.stdout:2/559: dread d5/d7/d28/f5f [0,4194304] 0 2024-04-12T20:56:13.413 INFO:tasks.workunit.client.0.smithi018.stdout:2/560: truncate d5/d7/d79/f7e 140388 0 2024-04-12T20:56:13.414 INFO:tasks.workunit.client.0.smithi018.stdout:1/490: dwrite d3/d9/da/dd/d31/d32/d33/f96 [0,4194304] 0 2024-04-12T20:56:13.423 INFO:tasks.workunit.client.0.smithi018.stdout:2/561: dread d5/d8/f1e [0,4194304] 0 2024-04-12T20:56:13.426 INFO:tasks.workunit.client.0.smithi018.stdout:1/491: link d3/d9/da/dd/d31/d32/d13/d19/d2d/d4e/d7a/l8d d3/d9/da/dd/d31/d32/d13/d19/d2d/d4e/laa 0 2024-04-12T20:56:13.431 INFO:tasks.workunit.client.0.smithi018.stdout:2/562: symlink d5/d7/d31/d62/da6/lc5 0 2024-04-12T20:56:13.442 INFO:tasks.workunit.client.0.smithi018.stdout:1/492: truncate d3/d9/da/dd/d31/f45 4063680 0 2024-04-12T20:56:13.450 INFO:tasks.workunit.client.0.smithi018.stdout:2/563: dwrite d5/d7/d28/f5f [4194304,4194304] 0 2024-04-12T20:56:13.459 INFO:tasks.workunit.client.0.smithi018.stdout:2/564: truncate d5/d8/d25/f7d 917515 0 2024-04-12T20:56:13.461 INFO:tasks.workunit.client.0.smithi018.stdout:2/565: dread - d5/fbd zero size 2024-04-12T20:56:13.463 INFO:tasks.workunit.client.0.smithi018.stdout:2/566: chown d5/d8/d33 1 1 2024-04-12T20:56:13.467 INFO:tasks.workunit.client.0.smithi018.stdout:1/493: dread d3/d9/da/dd/f17 [0,4194304] 0 2024-04-12T20:56:13.467 INFO:tasks.workunit.client.0.smithi018.stdout:2/567: getdents d5/d7/db6 0 2024-04-12T20:56:13.474 INFO:tasks.workunit.client.0.smithi018.stdout:2/568: truncate d5/d7/d79/f7e 1091101 0 2024-04-12T20:56:13.476 INFO:tasks.workunit.client.0.smithi018.stdout:2/569: rmdir d5/d12/d54/dbc 39 2024-04-12T20:56:13.487 INFO:tasks.workunit.client.0.smithi018.stdout:1/494: dwrite d3/d9/da/dd/d31/d32/f41 [0,4194304] 0 2024-04-12T20:56:13.495 INFO:tasks.workunit.client.0.smithi018.stdout:1/495: mkdir d3/d9/da/dd/d31/d32/d33/d80/dab 0 2024-04-12T20:56:13.496 INFO:tasks.workunit.client.0.smithi018.stdout:2/570: dwrite d5/d7/d63/fac [0,4194304] 0 2024-04-12T20:56:13.517 INFO:tasks.workunit.client.0.smithi018.stdout:2/571: dread d5/d12/d54/f52 [0,4194304] 0 2024-04-12T20:56:13.519 INFO:tasks.workunit.client.0.smithi018.stdout:2/572: chown d5/d7/d63/l85 6523 1 2024-04-12T20:56:13.523 INFO:tasks.workunit.client.0.smithi018.stdout:2/573: creat d5/fc6 x:0 0 0 2024-04-12T20:56:13.530 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:56:13 smithi186 ceph-mon[36597]: pgmap v39: 65 pgs: 65 active+clean; 1.2 GiB data, 5.4 GiB used, 531 GiB / 536 GiB avail; 49 MiB/s rd, 165 MiB/s wr, 427 op/s 2024-04-12T20:56:13.530 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:56:13 smithi186 ceph-mon[36597]: from='client.14656 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T20:56:13.536 INFO:tasks.workunit.client.0.smithi018.stdout:5/656: sync 2024-04-12T20:56:13.536 INFO:tasks.workunit.client.0.smithi018.stdout:7/576: sync 2024-04-12T20:56:13.536 INFO:tasks.workunit.client.0.smithi018.stdout:8/746: sync 2024-04-12T20:56:13.540 INFO:tasks.workunit.client.0.smithi018.stdout:5/657: unlink d0/d2/d33/d10/d26/f9b 0 2024-04-12T20:56:13.541 INFO:tasks.workunit.client.0.smithi018.stdout:5/658: fdatasync d0/d5b/d84/f31 0 2024-04-12T20:56:13.543 INFO:tasks.workunit.client.0.smithi018.stdout:8/747: creat db/d4e/d56/d59/d76/de0/d27/fec x:0 0 0 2024-04-12T20:56:13.544 INFO:tasks.workunit.client.0.smithi018.stdout:8/748: stat db/d4e/d56/d59/d76/de0/ldc 0 2024-04-12T20:56:13.552 INFO:tasks.workunit.client.0.smithi018.stdout:5/659: mknod d0/d2/d33/d10/d2f/dbf/cdd 0 2024-04-12T20:56:13.555 INFO:tasks.workunit.client.0.smithi018.stdout:6/769: write d5/df/d3f/dca/fcf [155463,54918] 0 2024-04-12T20:56:13.558 INFO:tasks.workunit.client.0.smithi018.stdout:5/660: getdents d0/d2/d11/d74 0 2024-04-12T20:56:13.563 INFO:tasks.workunit.client.0.smithi018.stdout:5/661: unlink d0/d2/d33/c19 0 2024-04-12T20:56:13.568 INFO:tasks.workunit.client.0.smithi018.stdout:5/662: mknod d0/d2/d33/d46/d1e/d2e/d7f/cde 0 2024-04-12T20:56:13.577 INFO:tasks.workunit.client.0.smithi018.stdout:9/718: dwrite d3/d6/d9/dc/d5a/d79/d7e/d9b/f73 [0,4194304] 0 2024-04-12T20:56:13.577 INFO:tasks.workunit.client.0.smithi018.stdout:7/577: dread d1/db/d31/d3a/d42/d4c/f51 [0,4194304] 0 2024-04-12T20:56:13.581 INFO:tasks.workunit.client.0.smithi018.stdout:9/719: chown d3/d6/d9/dc/d6f/d77 1 1 2024-04-12T20:56:13.582 INFO:tasks.workunit.client.0.smithi018.stdout:5/663: link d0/d2/d33/d46/d35/l72 d0/d2/d33/d10/d2f/dbf/da3/dae/ldf 0 2024-04-12T20:56:13.585 INFO:tasks.workunit.client.0.smithi018.stdout:8/749: dread db/d25/f52 [0,4194304] 0 2024-04-12T20:56:13.587 INFO:tasks.workunit.client.0.smithi018.stdout:7/578: creat d1/db/d31/d45/fc2 x:0 0 0 2024-04-12T20:56:13.592 INFO:tasks.workunit.client.0.smithi018.stdout:5/664: creat d0/d2/fe0 x:0 0 0 2024-04-12T20:56:13.592 INFO:tasks.workunit.client.0.smithi018.stdout:8/750: creat db/d4e/d9a/daa/dcd/fed x:0 0 0 2024-04-12T20:56:13.594 INFO:tasks.workunit.client.0.smithi018.stdout:8/751: fdatasync db/d95/fae 0 2024-04-12T20:56:13.594 INFO:tasks.workunit.client.0.smithi018.stdout:9/720: creat d3/d6/d43/fd4 x:0 0 0 2024-04-12T20:56:13.601 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:56:13 smithi018 ceph-mon[28799]: pgmap v39: 65 pgs: 65 active+clean; 1.2 GiB data, 5.4 GiB used, 531 GiB / 536 GiB avail; 49 MiB/s rd, 165 MiB/s wr, 427 op/s 2024-04-12T20:56:13.601 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:56:13 smithi018 ceph-mon[28799]: from='client.14656 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T20:56:13.605 INFO:tasks.workunit.client.0.smithi018.stdout:5/665: dread d0/d2/d33/d46/d35/d4d/f80 [0,4194304] 0 2024-04-12T20:56:13.610 INFO:tasks.workunit.client.0.smithi018.stdout:7/579: dwrite d1/db/d31/d45/fc2 [0,4194304] 0 2024-04-12T20:56:13.611 INFO:tasks.workunit.client.0.smithi018.stdout:5/666: mkdir d0/d2/d33/d46/d35/d4d/de1 0 2024-04-12T20:56:13.612 INFO:tasks.workunit.client.0.smithi018.stdout:7/580: stat d1/db/d31/d45/d8c/d9d/d70/l71 0 2024-04-12T20:56:13.613 INFO:tasks.workunit.client.0.smithi018.stdout:5/667: chown d0/d2/l21 3 1 2024-04-12T20:56:13.613 INFO:tasks.workunit.client.0.smithi018.stdout:8/752: link db/d4e/d56/d59/d76/de0/d27/f33 db/d4e/d56/d59/d76/da5/d45/fee 0 2024-04-12T20:56:13.614 INFO:tasks.workunit.client.0.smithi018.stdout:7/581: truncate d1/f3e 4467220 0 2024-04-12T20:56:13.624 INFO:tasks.workunit.client.0.smithi018.stdout:8/753: mknod db/d3b/cef 0 2024-04-12T20:56:13.626 INFO:tasks.workunit.client.0.smithi018.stdout:7/582: unlink d1/db/d31/d45/d8c/d9d/d70/d96/c1f 0 2024-04-12T20:56:13.628 INFO:tasks.workunit.client.0.smithi018.stdout:7/583: write d1/d6/f24 [628746,86987] 0 2024-04-12T20:56:13.629 INFO:tasks.workunit.client.0.smithi018.stdout:8/754: unlink db/d4e/d56/d59/d76/de0/ldc 0 2024-04-12T20:56:13.633 INFO:tasks.workunit.client.0.smithi018.stdout:6/770: write d5/d6/d1b/f61 [2787842,15963] 0 2024-04-12T20:56:13.636 INFO:tasks.workunit.client.0.smithi018.stdout:7/584: rename d1/db/d6f/d8a to d1/db/d31/d3a/d42/d4c/d6c/dc3 0 2024-04-12T20:56:13.638 INFO:tasks.workunit.client.0.smithi018.stdout:8/755: symlink db/d4e/d56/d59/d76/de0/d27/lf0 0 2024-04-12T20:56:13.639 INFO:tasks.workunit.client.0.smithi018.stdout:7/585: readlink d1/db/d31/d45/d8c/d9d/lb0 0 2024-04-12T20:56:13.640 INFO:tasks.workunit.client.0.smithi018.stdout:8/756: dread - db/d25/fde zero size 2024-04-12T20:56:13.642 INFO:tasks.workunit.client.0.smithi018.stdout:5/668: truncate d0/d2/d33/d10/f27 3932362 0 2024-04-12T20:56:13.643 INFO:tasks.workunit.client.0.smithi018.stdout:7/586: chown d1/db/d31/d3a/d97/fa1 14 1 2024-04-12T20:56:13.644 INFO:tasks.workunit.client.0.smithi018.stdout:8/757: mkdir db/d4e/d9a/d69/d6c/df1 0 2024-04-12T20:56:13.648 INFO:tasks.workunit.client.0.smithi018.stdout:7/587: mkdir d1/db/d31/d45/d8c/d9d/d70/dc4 0 2024-04-12T20:56:13.651 INFO:tasks.workunit.client.0.smithi018.stdout:9/721: write d3/d6/d43/f7f [212736,111479] 0 2024-04-12T20:56:13.651 INFO:tasks.workunit.client.0.smithi018.stdout:5/669: symlink d0/d2/d33/d46/d1e/d55/d66/le2 0 2024-04-12T20:56:13.652 INFO:tasks.workunit.client.0.smithi018.stdout:8/758: unlink db/d4e/d56/d59/d76/da5/fc6 0 2024-04-12T20:56:13.657 INFO:tasks.workunit.client.0.smithi018.stdout:7/588: mknod d1/db/d31/d45/d8c/d9d/d70/d96/cc5 0 2024-04-12T20:56:13.660 INFO:tasks.workunit.client.0.smithi018.stdout:5/670: chown d0/d2/d33/d46/d35/d4d/db3/dc3/cd1 834 1 2024-04-12T20:56:13.662 INFO:tasks.workunit.client.0.smithi018.stdout:6/771: dwrite d5/d6/d1b/d3b/fdc [0,4194304] 0 2024-04-12T20:56:13.667 INFO:tasks.workunit.client.0.smithi018.stdout:6/772: mknod d5/d6/d1b/d3b/d8e/d8f/c126 0 2024-04-12T20:56:13.675 INFO:tasks.workunit.client.0.smithi018.stdout:6/773: mknod d5/df/d19/c127 0 2024-04-12T20:56:13.675 INFO:tasks.workunit.client.0.smithi018.stdout:9/722: dwrite d3/d6/d43/fa3 [0,4194304] 0 2024-04-12T20:56:13.683 INFO:tasks.workunit.client.0.smithi018.stdout:6/774: mkdir d5/d6/d1b/d3b/d6a/d116/d104/d128 0 2024-04-12T20:56:13.683 INFO:tasks.workunit.client.0.smithi018.stdout:7/589: dwrite d1/d6/f24 [0,4194304] 0 2024-04-12T20:56:13.690 INFO:tasks.workunit.client.0.smithi018.stdout:5/671: dread d0/d2/d33/d46/d1e/d55/f88 [0,4194304] 0 2024-04-12T20:56:13.702 INFO:tasks.workunit.client.0.smithi018.stdout:6/775: rename d5/df/d19/d2b/f9c to d5/df/d31/f129 0 2024-04-12T20:56:13.702 INFO:tasks.workunit.client.0.smithi018.stdout:9/723: dwrite d3/d6/d9/d3f/f81 [0,4194304] 0 2024-04-12T20:56:13.706 INFO:tasks.workunit.client.0.smithi018.stdout:9/724: readlink d3/d6/d9/dc/d5a/d79/d7e/d9b/d21/l55 0 2024-04-12T20:56:13.706 INFO:tasks.workunit.client.0.smithi018.stdout:6/776: unlink d5/d6/d1b/d3b/d6a/dd8/f102 0 2024-04-12T20:56:13.710 INFO:tasks.workunit.client.0.smithi018.stdout:9/725: symlink d3/d6/d3d/ld5 0 2024-04-12T20:56:13.714 INFO:tasks.workunit.client.0.smithi018.stdout:6/777: creat d5/d15/d60/d124/f12a x:0 0 0 2024-04-12T20:56:13.715 INFO:tasks.workunit.client.0.smithi018.stdout:9/726: chown d3/d6/d9/dc/d46/d5d/cb0 14798 1 2024-04-12T20:56:13.728 INFO:tasks.workunit.client.0.smithi018.stdout:7/590: dwrite d1/db/d31/d3a/f83 [0,4194304] 0 2024-04-12T20:56:13.740 INFO:tasks.workunit.client.0.smithi018.stdout:6/778: dwrite d5/d6/d1b/d3b/d6a/d116/ff4 [0,4194304] 0 2024-04-12T20:56:13.752 INFO:tasks.workunit.client.0.smithi018.stdout:6/779: symlink d5/d6/d1b/d3b/d6a/d116/d104/l12b 0 2024-04-12T20:56:13.755 INFO:tasks.workunit.client.0.smithi018.stdout:7/591: dwrite d1/d6/fb4 [0,4194304] 0 2024-04-12T20:56:13.758 INFO:tasks.workunit.client.0.smithi018.stdout:7/592: truncate d1/db/d31/d30/fbf 140366 0 2024-04-12T20:56:13.761 INFO:tasks.workunit.client.0.smithi018.stdout:6/780: rename d5/d6/d1b/d3b/d22/l7c to d5/d6/d1b/d3b/d22/l12c 0 2024-04-12T20:56:13.773 INFO:tasks.workunit.client.0.smithi018.stdout:6/781: getdents d5 0 2024-04-12T20:56:13.778 INFO:tasks.workunit.client.0.smithi018.stdout:7/593: dwrite d1/db/d31/f63 [0,4194304] 0 2024-04-12T20:56:13.781 INFO:tasks.workunit.client.0.smithi018.stdout:7/594: chown d1/db/d31/d45/d8c/d9d/d70/l78 19164598 1 2024-04-12T20:56:13.783 INFO:tasks.workunit.client.0.smithi018.stdout:7/595: read d1/f2f [3374245,74996] 0 2024-04-12T20:56:13.791 INFO:tasks.workunit.client.0.smithi018.stdout:7/596: rmdir d1/db/d31/d3a 39 2024-04-12T20:56:13.792 INFO:tasks.workunit.client.0.smithi018.stdout:7/597: write d1/db/f43 [2505555,944] 0 2024-04-12T20:56:13.805 INFO:tasks.workunit.client.0.smithi018.stdout:7/598: write d1/f32 [300308,107714] 0 2024-04-12T20:56:13.808 INFO:tasks.workunit.client.0.smithi018.stdout:7/599: mkdir d1/db/d31/d30/dc6 0 2024-04-12T20:56:13.869 INFO:tasks.workunit.client.0.smithi018.stdout:0/557: sync 2024-04-12T20:56:13.869 INFO:tasks.workunit.client.0.smithi018.stdout:4/622: sync 2024-04-12T20:56:13.869 INFO:tasks.workunit.client.0.smithi018.stdout:3/617: sync 2024-04-12T20:56:13.872 INFO:tasks.workunit.client.0.smithi018.stdout:0/558: stat da/d15/d3e/d83/d8b 0 2024-04-12T20:56:13.873 INFO:tasks.workunit.client.0.smithi018.stdout:3/618: mknod d2/d6/d34/d71/db2/cc4 0 2024-04-12T20:56:13.882 INFO:tasks.workunit.client.0.smithi018.stdout:3/619: symlink d2/d6/d30/d31/d38/d3a/d76/dad/lc5 0 2024-04-12T20:56:13.885 INFO:tasks.workunit.client.0.smithi018.stdout:3/620: rmdir d2/d43 39 2024-04-12T20:56:13.887 INFO:tasks.workunit.client.0.smithi018.stdout:3/621: chown d2/d6/d34/d71/dbc 59137237 1 2024-04-12T20:56:13.890 INFO:tasks.workunit.client.0.smithi018.stdout:3/622: read d2/d83/f92 [3817955,69434] 0 2024-04-12T20:56:13.892 INFO:tasks.workunit.client.0.smithi018.stdout:4/623: dwrite d3/d5d/d60/d64/d72/fa0 [0,4194304] 0 2024-04-12T20:56:13.895 INFO:tasks.workunit.client.0.smithi018.stdout:3/623: chown d2/d43/db6/d6b/cbd 70959 1 2024-04-12T20:56:13.898 INFO:tasks.workunit.client.0.smithi018.stdout:4/624: mkdir d3/d5d/d60/dc5/dcc 0 2024-04-12T20:56:13.907 INFO:tasks.workunit.client.0.smithi018.stdout:3/624: getdents d2/d6/d34/d71/db2 0 2024-04-12T20:56:13.907 INFO:tasks.workunit.client.0.smithi018.stdout:4/625: mknod d3/d5d/ccd 0 2024-04-12T20:56:13.907 INFO:tasks.workunit.client.0.smithi018.stdout:4/626: readlink d3/d32/d66/l98 0 2024-04-12T20:56:13.909 INFO:tasks.workunit.client.0.smithi018.stdout:3/625: creat d2/d43/d61/d64/dbf/fc6 x:0 0 0 2024-04-12T20:56:13.910 INFO:tasks.workunit.client.0.smithi018.stdout:0/559: dread da/d15/f41 [0,4194304] 0 2024-04-12T20:56:13.910 INFO:tasks.workunit.client.0.smithi018.stdout:4/627: chown d3/d5d/d60/dc5/d43/f76 0 1 2024-04-12T20:56:13.912 INFO:tasks.workunit.client.0.smithi018.stdout:4/628: fdatasync d3/d5d/d60/dc5/d43/f7b 0 2024-04-12T20:56:13.913 INFO:tasks.workunit.client.0.smithi018.stdout:3/626: unlink d2/d6/cd 0 2024-04-12T20:56:13.916 INFO:tasks.workunit.client.0.smithi018.stdout:4/629: chown d3/db/c38 3 1 2024-04-12T20:56:13.919 INFO:tasks.workunit.client.0.smithi018.stdout:4/630: mknod d3/d5d/d60/d64/d4f/cce 0 2024-04-12T20:56:13.921 INFO:tasks.workunit.client.0.smithi018.stdout:3/627: dread d2/d6/d30/f6f [0,4194304] 0 2024-04-12T20:56:13.924 INFO:tasks.workunit.client.0.smithi018.stdout:3/628: rmdir d2/d6/d30 39 2024-04-12T20:56:13.925 INFO:tasks.workunit.client.0.smithi018.stdout:4/631: truncate d3/db/f2b 680466 0 2024-04-12T20:56:13.926 INFO:tasks.workunit.client.0.smithi018.stdout:0/560: dwrite da/d15/d4d/f85 [0,4194304] 0 2024-04-12T20:56:13.928 INFO:tasks.workunit.client.0.smithi018.stdout:3/629: creat d2/d43/db6/d6b/fc7 x:0 0 0 2024-04-12T20:56:13.942 INFO:tasks.workunit.client.0.smithi018.stdout:3/630: readlink d2/l26 0 2024-04-12T20:56:13.946 INFO:tasks.workunit.client.0.smithi018.stdout:0/561: link da/d15/d17/d21/d36/d47/l95 da/d15/d3e/lb8 0 2024-04-12T20:56:13.948 INFO:tasks.workunit.client.0.smithi018.stdout:0/562: fdatasync da/d15/f41 0 2024-04-12T20:56:13.951 INFO:tasks.workunit.client.0.smithi018.stdout:0/563: chown da/d15/f33 549 1 2024-04-12T20:56:13.956 INFO:tasks.workunit.client.0.smithi018.stdout:0/564: mkdir da/d15/d17/db9 0 2024-04-12T20:56:13.958 INFO:tasks.workunit.client.0.smithi018.stdout:4/632: dwrite d3/d32/d94/fcb [0,4194304] 0 2024-04-12T20:56:13.962 INFO:tasks.workunit.client.0.smithi018.stdout:0/565: symlink da/d15/d17/d21/d94/lba 0 2024-04-12T20:56:13.970 INFO:tasks.workunit.client.0.smithi018.stdout:4/633: getdents d3/d5d/d60/dc5 0 2024-04-12T20:56:13.973 INFO:tasks.workunit.client.0.smithi018.stdout:3/631: rename d2/d83/c9d to d2/d6/d30/d31/cc8 0 2024-04-12T20:56:13.975 INFO:tasks.workunit.client.0.smithi018.stdout:4/634: fsync d3/d32/d33/fa7 0 2024-04-12T20:56:13.980 INFO:tasks.workunit.client.0.smithi018.stdout:4/635: creat d3/d5d/d60/dc5/d53/fcf x:0 0 0 2024-04-12T20:56:13.981 INFO:tasks.workunit.client.0.smithi018.stdout:3/632: mknod d2/d6/d30/d31/d38/d3a/d76/dad/dc1/cc9 0 2024-04-12T20:56:13.982 INFO:tasks.workunit.client.0.smithi018.stdout:4/636: chown d3/c7 55345 1 2024-04-12T20:56:13.984 INFO:tasks.workunit.client.0.smithi018.stdout:0/566: dread da/d15/d17/d21/f68 [0,4194304] 0 2024-04-12T20:56:13.987 INFO:tasks.workunit.client.0.smithi018.stdout:3/633: creat d2/d6/d30/d31/d38/d3a/d48/fca x:0 0 0 2024-04-12T20:56:13.988 INFO:tasks.workunit.client.0.smithi018.stdout:0/567: write da/d15/d17/d21/d36/d6e/f7f [1768612,34100] 0 2024-04-12T20:56:13.989 INFO:tasks.workunit.client.0.smithi018.stdout:3/634: chown d2/d6/f84 4 1 2024-04-12T20:56:13.995 INFO:tasks.workunit.client.0.smithi018.stdout:3/635: chown d2/d43/d61/cc0 246 1 2024-04-12T20:56:14.004 INFO:tasks.workunit.client.0.smithi018.stdout:3/636: rename d2/f5 to d2/d43/d61/d64/dbf/d54/fcb 0 2024-04-12T20:56:14.010 INFO:tasks.workunit.client.0.smithi018.stdout:0/568: dwrite da/d15/d17/f32 [0,4194304] 0 2024-04-12T20:56:14.010 INFO:tasks.workunit.client.0.smithi018.stdout:4/637: dread d3/d5d/d60/dc5/d43/f2a [0,4194304] 0 2024-04-12T20:56:14.026 INFO:tasks.workunit.client.0.smithi018.stdout:0/569: dwrite da/faa [0,4194304] 0 2024-04-12T20:56:14.035 INFO:tasks.workunit.client.0.smithi018.stdout:0/570: creat da/d15/d4d/fbb x:0 0 0 2024-04-12T20:56:14.035 INFO:tasks.workunit.client.0.smithi018.stdout:4/638: dwrite d3/d5d/fae [0,4194304] 0 2024-04-12T20:56:14.038 INFO:tasks.workunit.client.0.smithi018.stdout:0/571: dread - da/d15/d17/d5c/d79/f86 zero size 2024-04-12T20:56:14.041 INFO:tasks.workunit.client.0.smithi018.stdout:4/639: creat d3/db/fd0 x:0 0 0 2024-04-12T20:56:14.041 INFO:tasks.workunit.client.0.smithi018.stdout:0/572: chown da/d15/d3e/l51 25 1 2024-04-12T20:56:14.042 INFO:tasks.workunit.client.0.smithi018.stdout:4/640: fdatasync d3/d5d/d60/dc5/fc1 0 2024-04-12T20:56:14.050 INFO:tasks.workunit.client.0.smithi018.stdout:3/637: dread d2/d4d/d88/f8a [0,4194304] 0 2024-04-12T20:56:14.059 INFO:tasks.workunit.client.0.smithi018.stdout:0/573: dwrite da/d15/f33 [4194304,4194304] 0 2024-04-12T20:56:14.068 INFO:tasks.workunit.client.0.smithi018.stdout:0/574: creat da/d15/d17/d71/fbc x:0 0 0 2024-04-12T20:56:14.073 INFO:tasks.workunit.client.0.smithi018.stdout:0/575: symlink da/d15/d3e/d3f/lbd 0 2024-04-12T20:56:14.073 INFO:tasks.workunit.client.0.smithi018.stdout:3/638: dwrite d2/d4d/d88/fa0 [0,4194304] 0 2024-04-12T20:56:14.076 INFO:tasks.workunit.client.0.smithi018.stdout:0/576: write da/d15/d17/d21/d94/fa4 [756194,44480] 0 2024-04-12T20:56:14.076 INFO:tasks.workunit.client.0.smithi018.stdout:9/727: sync 2024-04-12T20:56:14.076 INFO:tasks.workunit.client.0.smithi018.stdout:2/574: sync 2024-04-12T20:56:14.077 INFO:tasks.workunit.client.0.smithi018.stdout:7/600: sync 2024-04-12T20:56:14.078 INFO:tasks.workunit.client.0.smithi018.stdout:1/496: sync 2024-04-12T20:56:14.078 INFO:tasks.workunit.client.0.smithi018.stdout:6/782: sync 2024-04-12T20:56:14.078 INFO:tasks.workunit.client.0.smithi018.stdout:8/759: sync 2024-04-12T20:56:14.078 INFO:tasks.workunit.client.0.smithi018.stdout:5/672: sync 2024-04-12T20:56:14.080 INFO:tasks.workunit.client.0.smithi018.stdout:4/641: dread d3/d32/fd [0,4194304] 0 2024-04-12T20:56:14.083 INFO:tasks.workunit.client.0.smithi018.stdout:3/639: mkdir d2/d6/d34/d71/dbc/dcc 0 2024-04-12T20:56:14.084 INFO:tasks.workunit.client.0.smithi018.stdout:0/577: write da/d15/d3e/d3f/f59 [1525965,33923] 0 2024-04-12T20:56:14.086 INFO:tasks.workunit.client.0.smithi018.stdout:9/728: creat d3/d6/d9/dc/d13/db9/fd6 x:0 0 0 2024-04-12T20:56:14.088 INFO:tasks.workunit.client.0.smithi018.stdout:0/578: fsync da/d15/f8e 0 2024-04-12T20:56:14.088 INFO:tasks.workunit.client.0.smithi018.stdout:6/783: write d5/d15/d60/d124/f12a [871173,92764] 0 2024-04-12T20:56:14.088 INFO:tasks.workunit.client.0.smithi018.stdout:7/601: mknod d1/db/d31/d3a/d42/cc7 0 2024-04-12T20:56:14.094 INFO:tasks.workunit.client.0.smithi018.stdout:5/673: dread - d0/d2b/d40/f54 zero size 2024-04-12T20:56:14.095 INFO:tasks.workunit.client.0.smithi018.stdout:7/602: rmdir d1/db/d31/d45 39 2024-04-12T20:56:14.097 INFO:tasks.workunit.client.0.smithi018.stdout:2/575: dread d5/d8/d10/d19/d24/d39/f51 [0,4194304] 0 2024-04-12T20:56:14.098 INFO:tasks.workunit.client.0.smithi018.stdout:9/729: mknod d3/d93/cd7 0 2024-04-12T20:56:14.098 INFO:tasks.workunit.client.0.smithi018.stdout:1/497: creat d3/d9/da/d14/fac x:0 0 0 2024-04-12T20:56:14.099 INFO:tasks.workunit.client.0.smithi018.stdout:6/784: symlink d5/d6/d1b/d3b/d22/d117/l12d 0 2024-04-12T20:56:14.100 INFO:tasks.workunit.client.0.smithi018.stdout:0/579: unlink da/d15/d3e/d3f/f59 0 2024-04-12T20:56:14.100 INFO:tasks.workunit.client.0.smithi018.stdout:4/642: truncate d3/d32/d94/f97 574859 0 2024-04-12T20:56:14.102 INFO:tasks.workunit.client.0.smithi018.stdout:8/760: rename db/d4e/d9a/d69/d6c/fc1 to db/d4e/d56/d59/d76/da5/d45/d86/da1/ff2 0 2024-04-12T20:56:14.111 INFO:tasks.workunit.client.0.smithi018.stdout:8/761: dread - db/d4e/d56/d59/d76/de0/d27/fec zero size 2024-04-12T20:56:14.111 INFO:tasks.workunit.client.0.smithi018.stdout:6/785: unlink d5/df/d19/d2b/c2d 0 2024-04-12T20:56:14.118 INFO:tasks.workunit.client.0.smithi018.stdout:1/498: mkdir d3/d9/da/dd/d31/dad 0 2024-04-12T20:56:14.118 INFO:tasks.workunit.client.0.smithi018.stdout:5/674: rmdir d0/d2/d11/d37/d97 0 2024-04-12T20:56:14.121 INFO:tasks.workunit.client.0.smithi018.stdout:8/762: mkdir db/d4e/d7f/df3 0 2024-04-12T20:56:14.126 INFO:tasks.workunit.client.0.smithi018.stdout:3/640: getdents d2/d6/d30/d31/d38 0 2024-04-12T20:56:14.126 INFO:tasks.workunit.client.0.smithi018.stdout:6/786: rmdir d5/d6/d1b/d3b/d8e 39 2024-04-12T20:56:14.132 INFO:tasks.workunit.client.0.smithi018.stdout:0/580: dwrite da/d15/f33 [4194304,4194304] 0 2024-04-12T20:56:14.133 INFO:tasks.workunit.client.0.smithi018.stdout:9/730: dwrite d3/d6/d9/dc/d5a/d79/d7e/d9b/d21/dc2/fcd [0,4194304] 0 2024-04-12T20:56:14.137 INFO:tasks.workunit.client.0.smithi018.stdout:6/787: dread - d5/d15/d60/dbb/ffe zero size 2024-04-12T20:56:14.138 INFO:tasks.workunit.client.0.smithi018.stdout:1/499: creat d3/d9/da/fae x:0 0 0 2024-04-12T20:56:14.141 INFO:tasks.workunit.client.0.smithi018.stdout:3/641: unlink d2/d4d/d8d/l9b 0 2024-04-12T20:56:14.142 INFO:tasks.workunit.client.0.smithi018.stdout:0/581: readlink da/d15/d17/l1c 0 2024-04-12T20:56:14.149 INFO:tasks.workunit.client.0.smithi018.stdout:2/576: dread d5/d7/d31/d62/fb8 [0,4194304] 0 2024-04-12T20:56:14.153 INFO:tasks.workunit.client.0.smithi018.stdout:3/642: creat d2/d6/d34/d71/dbc/fcd x:0 0 0 2024-04-12T20:56:14.158 INFO:tasks.workunit.client.0.smithi018.stdout:7/603: dread d1/db/d31/d3a/d42/f75 [4194304,4194304] 0 2024-04-12T20:56:14.160 INFO:tasks.workunit.client.0.smithi018.stdout:6/788: creat d5/d6/d90/d125/f12e x:0 0 0 2024-04-12T20:56:14.160 INFO:tasks.workunit.client.0.smithi018.stdout:4/643: dread d3/db/f8b [0,4194304] 0 2024-04-12T20:56:14.161 INFO:tasks.workunit.client.0.smithi018.stdout:3/643: stat d2/d6/d34/d71/d7b/f7d 0 2024-04-12T20:56:14.162 INFO:tasks.workunit.client.0.smithi018.stdout:0/582: dread - da/d15/d3e/f76 zero size 2024-04-12T20:56:14.165 INFO:tasks.workunit.client.0.smithi018.stdout:7/604: chown d1/db/d31/d3a/fa5 203857733 1 2024-04-12T20:56:14.167 INFO:tasks.workunit.client.0.smithi018.stdout:6/789: chown d5/d6/d1b/d3b/f94 543 1 2024-04-12T20:56:14.167 INFO:tasks.workunit.client.0.smithi018.stdout:3/644: write d2/d43/fb3 [30632,21432] 0 2024-04-12T20:56:14.171 INFO:tasks.workunit.client.0.smithi018.stdout:6/790: chown d5/d6/d1b/d3b/ffc 86 1 2024-04-12T20:56:14.173 INFO:tasks.workunit.client.0.smithi018.stdout:4/644: creat d3/d32/d94/fd1 x:0 0 0 2024-04-12T20:56:14.174 INFO:tasks.workunit.client.0.smithi018.stdout:6/791: chown d5/df/d31 3750 1 2024-04-12T20:56:14.177 INFO:tasks.workunit.client.0.smithi018.stdout:1/500: write d3/d9/da/dd/f55 [770490,76874] 0 2024-04-12T20:56:14.177 INFO:tasks.workunit.client.0.smithi018.stdout:5/675: write d0/d2/d33/d10/d2f/f53 [395363,126516] 0 2024-04-12T20:56:14.184 INFO:tasks.workunit.client.0.smithi018.stdout:2/577: link d5/d8/d10/c75 d5/d8/d25/cc7 0 2024-04-12T20:56:14.190 INFO:tasks.workunit.client.0.smithi018.stdout:7/605: rmdir d1/db/d6f/db3 0 2024-04-12T20:56:14.194 INFO:tasks.workunit.client.0.smithi018.stdout:4/645: dwrite d3/faa [0,4194304] 0 2024-04-12T20:56:14.194 INFO:tasks.workunit.client.0.smithi018.stdout:6/792: getdents d5/d6/d1b/d3b/d22 0 2024-04-12T20:56:14.198 INFO:tasks.workunit.client.0.smithi018.stdout:7/606: read d1/db/d31/d30/f57 [1862967,94894] 0 2024-04-12T20:56:14.198 INFO:tasks.workunit.client.0.smithi018.stdout:6/793: chown d5/d15/la9 3 1 2024-04-12T20:56:14.199 INFO:tasks.workunit.client.0.smithi018.stdout:3/645: dwrite d2/d4d/d88/fa0 [0,4194304] 0 2024-04-12T20:56:14.200 INFO:tasks.workunit.client.0.smithi018.stdout:9/731: rename d3/d6/d9/dc/d13 to d3/d6/d9/dc/d5a/d79/d7e/d9b/d21/dc3/dd8 0 2024-04-12T20:56:14.205 INFO:tasks.workunit.client.0.smithi018.stdout:4/646: unlink d3/d32/d94/f96 0 2024-04-12T20:56:14.205 INFO:tasks.workunit.client.0.smithi018.stdout:6/794: mknod d5/df/d31/d4e/c12f 0 2024-04-12T20:56:14.210 INFO:tasks.workunit.client.0.smithi018.stdout:0/583: rename da/d15/d4d/c67 to da/d15/d17/d21/cbe 0 2024-04-12T20:56:14.210 INFO:tasks.workunit.client.0.smithi018.stdout:7/607: chown d1/f6e 5 1 2024-04-12T20:56:14.211 INFO:tasks.workunit.client.0.smithi018.stdout:6/795: chown d5/d6/d1b/d3b/d6a/dd8/l110 25550 1 2024-04-12T20:56:14.212 INFO:tasks.workunit.client.0.smithi018.stdout:2/578: link d5/d8/d10/d19/daf/d88/fb5 d5/d8/d10/d19/d24/d39/da4/fc8 0 2024-04-12T20:56:14.213 INFO:tasks.workunit.client.0.smithi018.stdout:4/647: mknod d3/d5d/d60/d64/d79/cd2 0 2024-04-12T20:56:14.215 INFO:tasks.workunit.client.0.smithi018.stdout:6/796: mkdir d5/d71/d130 0 2024-04-12T20:56:14.216 INFO:tasks.workunit.client.0.smithi018.stdout:4/648: chown d3/d5d/d60/d64/d4f/dc7 1 1 2024-04-12T20:56:14.219 INFO:tasks.workunit.client.0.smithi018.stdout:5/676: rename d0/fa7 to d0/d2/d33/d10/d2f/dbf/da3/fe3 0 2024-04-12T20:56:14.220 INFO:tasks.workunit.client.0.smithi018.stdout:0/584: symlink da/d15/d4d/d6b/lbf 0 2024-04-12T20:56:14.224 INFO:tasks.workunit.client.0.smithi018.stdout:7/608: chown d1/db/d31/d45/f9a 17670 1 2024-04-12T20:56:14.226 INFO:tasks.workunit.client.0.smithi018.stdout:6/797: rename d5/df/d9b/lc6 to d5/df/d19/d43/de6/l131 0 2024-04-12T20:56:14.230 INFO:tasks.workunit.client.0.smithi018.stdout:5/677: mkdir d0/d2/de4 0 2024-04-12T20:56:14.232 INFO:tasks.workunit.client.0.smithi018.stdout:2/579: truncate d5/d8/d10/d43/f56 549719 0 2024-04-12T20:56:14.232 INFO:tasks.workunit.client.0.smithi018.stdout:7/609: chown d1/db/d31/d45/f9a 26295117 1 2024-04-12T20:56:14.236 INFO:tasks.workunit.client.0.smithi018.stdout:4/649: link d3/d5d/d60/dc5/d43/f2e d3/db/fd3 0 2024-04-12T20:56:14.236 INFO:tasks.workunit.client.0.smithi018.stdout:0/585: read da/d15/d17/d21/d36/d47/f72 [3442297,65832] 0 2024-04-12T20:56:14.237 INFO:tasks.workunit.client.0.smithi018.stdout:2/580: chown d5/d8/d10/c23 9330 1 2024-04-12T20:56:14.240 INFO:tasks.workunit.client.0.smithi018.stdout:6/798: truncate d5/d6/d1b/d3b/d22/fb3 1179454 0 2024-04-12T20:56:14.243 INFO:tasks.workunit.client.0.smithi018.stdout:3/646: dwrite d2/d43/d61/f62 [0,4194304] 0 2024-04-12T20:56:14.245 INFO:tasks.workunit.client.0.smithi018.stdout:2/581: symlink d5/d8/d10/d43/lc9 0 2024-04-12T20:56:14.248 INFO:tasks.workunit.client.0.smithi018.stdout:4/650: creat d3/d5d/d60/dc5/dcc/fd4 x:0 0 0 2024-04-12T20:56:14.248 INFO:tasks.workunit.client.0.smithi018.stdout:6/799: creat d5/df/d31/d10c/f132 x:0 0 0 2024-04-12T20:56:14.249 INFO:tasks.workunit.client.0.smithi018.stdout:9/732: dread d3/d6/d1b/f35 [0,4194304] 0 2024-04-12T20:56:14.249 INFO:tasks.workunit.client.0.smithi018.stdout:5/678: truncate d0/d2/d33/d10/d26/f7e 773879 0 2024-04-12T20:56:14.250 INFO:tasks.workunit.client.0.smithi018.stdout:3/647: fdatasync d2/d6/d30/d31/d38/d3a/d48/fca 0 2024-04-12T20:56:14.252 INFO:tasks.workunit.client.0.smithi018.stdout:7/610: link d1/db/d31/d45/d8c/d9d/d70/d96/f13 d1/db/d31/d30/dc6/fc8 0 2024-04-12T20:56:14.254 INFO:tasks.workunit.client.0.smithi018.stdout:2/582: mknod d5/d8/d10/d43/cca 0 2024-04-12T20:56:14.255 INFO:tasks.workunit.client.0.smithi018.stdout:5/679: readlink d0/d2/d33/d46/d35/d4d/l57 0 2024-04-12T20:56:14.257 INFO:tasks.workunit.client.0.smithi018.stdout:4/651: symlink d3/d5d/d60/dc5/d3d/ld5 0 2024-04-12T20:56:14.260 INFO:tasks.workunit.client.0.smithi018.stdout:3/648: chown d2/d6/d30/d31/d38/d3a/d76/fa5 3174 1 2024-04-12T20:56:14.261 INFO:tasks.workunit.client.0.smithi018.stdout:7/611: fdatasync d1/db/f35 0 2024-04-12T20:56:14.262 INFO:tasks.workunit.client.0.smithi018.stdout:6/800: creat d5/d6/d1b/d3b/d6a/d116/d104/d128/f133 x:0 0 0 2024-04-12T20:56:14.262 INFO:tasks.workunit.client.0.smithi018.stdout:9/733: symlink d3/d6/d9/dc/dab/ld9 0 2024-04-12T20:56:14.265 INFO:tasks.workunit.client.0.smithi018.stdout:2/583: creat d5/d12/d54/d3e/fcb x:0 0 0 2024-04-12T20:56:14.270 INFO:tasks.workunit.client.0.smithi018.stdout:2/584: chown d5/l13 1059 1 2024-04-12T20:56:14.270 INFO:tasks.workunit.client.0.smithi018.stdout:0/586: dwrite da/d15/d17/d21/d94/fa4 [0,4194304] 0 2024-04-12T20:56:14.275 INFO:tasks.workunit.client.0.smithi018.stdout:4/652: mknod d3/d5d/d60/d64/d4f/dc7/cd6 0 2024-04-12T20:56:14.277 INFO:tasks.workunit.client.0.smithi018.stdout:7/612: mkdir d1/d6/dc9 0 2024-04-12T20:56:14.280 INFO:tasks.workunit.client.0.smithi018.stdout:6/801: symlink d5/df/d19/d27/l134 0 2024-04-12T20:56:14.283 INFO:tasks.workunit.client.0.smithi018.stdout:9/734: creat d3/d6/fda x:0 0 0 2024-04-12T20:56:14.285 INFO:tasks.workunit.client.0.smithi018.stdout:0/587: unlink da/f75 0 2024-04-12T20:56:14.286 INFO:tasks.workunit.client.0.smithi018.stdout:2/585: link d5/d8/d10/d19/daf/fa2 d5/d8/d10/fcc 0 2024-04-12T20:56:14.287 INFO:tasks.workunit.client.0.smithi018.stdout:3/649: getdents d2/d6/d34/d71/d7b 0 2024-04-12T20:56:14.290 INFO:tasks.workunit.client.0.smithi018.stdout:9/735: creat d3/d6/d9/d1e/fdb x:0 0 0 2024-04-12T20:56:14.291 INFO:tasks.workunit.client.0.smithi018.stdout:0/588: symlink da/d15/d17/lc0 0 2024-04-12T20:56:14.292 INFO:tasks.workunit.client.0.smithi018.stdout:5/680: dwrite d0/d2/d33/d46/d1e/d2e/d7f/f98 [0,4194304] 0 2024-04-12T20:56:14.293 INFO:tasks.workunit.client.0.smithi018.stdout:7/613: write d1/db/d31/d45/d53/f56 [1558935,59825] 0 2024-04-12T20:56:14.297 INFO:tasks.workunit.client.0.smithi018.stdout:3/650: creat d2/d83/fce x:0 0 0 2024-04-12T20:56:14.298 INFO:tasks.workunit.client.0.smithi018.stdout:0/589: write da/f25 [607997,36940] 0 2024-04-12T20:56:14.299 INFO:tasks.workunit.client.0.smithi018.stdout:6/802: rename d5/df/d3f/dc1/cd3 to d5/d6/d1b/d3b/d6a/d116/c135 0 2024-04-12T20:56:14.315 INFO:tasks.workunit.client.0.smithi018.stdout:4/653: dwrite d3/d5d/d60/dc5/fb0 [0,4194304] 0 2024-04-12T20:56:14.315 INFO:tasks.workunit.client.0.smithi018.stdout:5/681: chown d0/d2/d11/f4b 69 1 2024-04-12T20:56:14.315 INFO:tasks.workunit.client.0.smithi018.stdout:0/590: mkdir da/d15/d17/d71/dc1 0 2024-04-12T20:56:14.316 INFO:tasks.workunit.client.0.smithi018.stdout:4/654: chown d3/d5d/d60/dc5/d43 10 1 2024-04-12T20:56:14.318 INFO:tasks.workunit.client.0.smithi018.stdout:6/803: creat d5/df/d19/f136 x:0 0 0 2024-04-12T20:56:14.319 INFO:tasks.workunit.client.0.smithi018.stdout:9/736: unlink d3/d6/d9/dc/d5a/d79/d7e/d9b/d21/dc3/dd8/db9/cc7 0 2024-04-12T20:56:14.324 INFO:tasks.workunit.client.0.smithi018.stdout:6/804: chown d5/d6/d1b/d3b/d35/ce9 574 1 2024-04-12T20:56:14.330 INFO:tasks.workunit.client.0.smithi018.stdout:3/651: dwrite d2/d6/d30/d31/d38/d3a/d76/fa5 [0,4194304] 0 2024-04-12T20:56:14.333 INFO:tasks.workunit.client.0.smithi018.stdout:4/655: rename d3/db/c15 to d3/d32/d33/dbc/cd7 0 2024-04-12T20:56:14.334 INFO:tasks.workunit.client.0.smithi018.stdout:9/737: rmdir d3/d6/d43 39 2024-04-12T20:56:14.334 INFO:tasks.workunit.client.0.smithi018.stdout:7/614: dwrite d1/f3e [0,4194304] 0 2024-04-12T20:56:14.336 INFO:tasks.workunit.client.0.smithi018.stdout:3/652: stat d2/d43/d61/la7 0 2024-04-12T20:56:14.338 INFO:tasks.workunit.client.0.smithi018.stdout:4/656: readlink d3/d5d/d60/d64/l70 0 2024-04-12T20:56:14.339 INFO:tasks.workunit.client.0.smithi018.stdout:9/738: write d3/d6/d9/d1e/fdb [744818,2554] 0 2024-04-12T20:56:14.343 INFO:tasks.workunit.client.0.smithi018.stdout:3/653: write d2/d6/d34/fae [585621,56232] 0 2024-04-12T20:56:14.348 INFO:tasks.workunit.client.0.smithi018.stdout:4/657: truncate d3/d5d/d60/dc5/d53/fcf 266742 0 2024-04-12T20:56:14.351 INFO:tasks.workunit.client.0.smithi018.stdout:4/658: chown d3/d32/d66/cb8 44952 1 2024-04-12T20:56:14.353 INFO:tasks.workunit.client.0.smithi018.stdout:3/654: mknod d2/ccf 0 2024-04-12T20:56:14.363 INFO:tasks.workunit.client.0.smithi018.stdout:3/655: truncate d2/d6/d34/d71/dbc/fcd 490283 0 2024-04-12T20:56:14.364 INFO:tasks.workunit.client.0.smithi018.stdout:0/591: rmdir da/d15/d17/d71 39 2024-04-12T20:56:14.365 INFO:tasks.workunit.client.0.smithi018.stdout:6/805: getdents d5/d6/d1b/d3b/d22 0 2024-04-12T20:56:14.367 INFO:tasks.workunit.client.0.smithi018.stdout:5/682: write d0/d2/d11/d37/f4c [64013,76094] 0 2024-04-12T20:56:14.370 INFO:tasks.workunit.client.0.smithi018.stdout:9/739: unlink d3/d6/d1b/c52 0 2024-04-12T20:56:14.373 INFO:tasks.workunit.client.0.smithi018.stdout:4/659: dread d3/d32/d94/f97 [0,4194304] 0 2024-04-12T20:56:14.375 INFO:tasks.workunit.client.0.smithi018.stdout:0/592: creat da/d15/d17/d71/fc2 x:0 0 0 2024-04-12T20:56:14.377 INFO:tasks.workunit.client.0.smithi018.stdout:7/615: dwrite d1/d6/fa0 [0,4194304] 0 2024-04-12T20:56:14.377 INFO:tasks.workunit.client.0.smithi018.stdout:6/806: symlink d5/d6/d1b/d3b/d6a/d116/d104/d128/l137 0 2024-04-12T20:56:14.380 INFO:tasks.workunit.client.0.smithi018.stdout:7/616: chown d1/db/d31/d30 4 1 2024-04-12T20:56:14.380 INFO:tasks.workunit.client.0.smithi018.stdout:9/740: read d3/d6/d9/dc/d5a/d79/d7e/d9b/d21/dc2/fcd [587943,13880] 0 2024-04-12T20:56:14.381 INFO:tasks.workunit.client.0.smithi018.stdout:3/656: write d2/d6/d30/d31/d38/f74 [614437,116446] 0 2024-04-12T20:56:14.388 INFO:tasks.workunit.client.0.smithi018.stdout:0/593: rename da/d15/d3e to da/d15/d17/d21/d36/da2/dc3 0 2024-04-12T20:56:14.388 INFO:tasks.workunit.client.0.smithi018.stdout:4/660: getdents d3/d5d/d60/dc5/d3d/d80 0 2024-04-12T20:56:14.390 INFO:tasks.workunit.client.0.smithi018.stdout:0/594: readlink da/l14 0 2024-04-12T20:56:14.392 INFO:tasks.workunit.client.0.smithi018.stdout:7/617: creat d1/d6/fca x:0 0 0 2024-04-12T20:56:14.393 INFO:tasks.workunit.client.0.smithi018.stdout:4/661: truncate d3/d5d/fbd 1033460 0 2024-04-12T20:56:14.395 INFO:tasks.workunit.client.0.smithi018.stdout:3/657: rename d2/d6/d30/d31/d7c to d2/d6/d30/d31/d38/d3a/dd0 0 2024-04-12T20:56:14.398 INFO:tasks.workunit.client.0.smithi018.stdout:4/662: creat d3/d32/d33/d8e/fd8 x:0 0 0 2024-04-12T20:56:14.399 INFO:tasks.workunit.client.0.smithi018.stdout:0/595: creat da/d15/d17/d21/d36/d47/d81/fc4 x:0 0 0 2024-04-12T20:56:14.401 INFO:tasks.workunit.client.0.smithi018.stdout:0/596: write da/d15/f39 [1929253,54039] 0 2024-04-12T20:56:14.402 INFO:tasks.workunit.client.0.smithi018.stdout:3/658: chown d2/d6/d34/f59 14983 1 2024-04-12T20:56:14.403 INFO:tasks.workunit.client.0.smithi018.stdout:7/618: creat d1/d6/fcb x:0 0 0 2024-04-12T20:56:14.403 INFO:tasks.workunit.client.0.smithi018.stdout:4/663: chown d3/d5d/d60/dc5/fa4 1874 1 2024-04-12T20:56:14.407 INFO:tasks.workunit.client.0.smithi018.stdout:4/664: write d3/faa [2644934,114550] 0 2024-04-12T20:56:14.411 INFO:tasks.workunit.client.0.smithi018.stdout:7/619: write d1/f1b [2160120,109549] 0 2024-04-12T20:56:14.413 INFO:tasks.workunit.client.0.smithi018.stdout:0/597: creat da/d15/d4d/d58/fc5 x:0 0 0 2024-04-12T20:56:14.415 INFO:tasks.workunit.client.0.smithi018.stdout:7/620: read d1/f3e [4048295,33413] 0 2024-04-12T20:56:14.416 INFO:tasks.workunit.client.0.smithi018.stdout:3/659: creat d2/d6/d34/d71/dbc/dcc/fd1 x:0 0 0 2024-04-12T20:56:14.419 INFO:tasks.workunit.client.0.smithi018.stdout:7/621: fdatasync d1/db/d31/d45/d8c/d9d/d70/f95 0 2024-04-12T20:56:14.424 INFO:tasks.workunit.client.0.smithi018.stdout:5/683: dwrite d0/d2/d33/d10/d2f/d8f/fb2 [0,4194304] 0 2024-04-12T20:56:14.426 INFO:tasks.workunit.client.0.smithi018.stdout:0/598: creat da/d15/d17/fc6 x:0 0 0 2024-04-12T20:56:14.427 INFO:tasks.workunit.client.0.smithi018.stdout:3/660: mknod d2/d6/d30/d31/d38/d3a/d76/dad/cd2 0 2024-04-12T20:56:14.438 INFO:tasks.workunit.client.0.smithi018.stdout:9/741: dwrite d3/d6/d9/d1e/f96 [0,4194304] 0 2024-04-12T20:56:14.439 INFO:tasks.workunit.client.0.smithi018.stdout:5/684: creat d0/d2/d11/d37/dc2/d89/d8d/fe5 x:0 0 0 2024-04-12T20:56:14.439 INFO:tasks.workunit.client.0.smithi018.stdout:0/599: fsync da/d15/d17/d21/d36/da2/dc3/d3f/f6c 0 2024-04-12T20:56:14.440 INFO:tasks.workunit.client.0.smithi018.stdout:7/622: rename d1/db/d31/d45/d8c/d9d/f73 to d1/db/d31/d45/d8c/d9d/d70/d96/d5f/fcc 0 2024-04-12T20:56:14.442 INFO:tasks.workunit.client.0.smithi018.stdout:7/623: fdatasync d1/d6/f24 0 2024-04-12T20:56:14.442 INFO:tasks.workunit.client.0.smithi018.stdout:6/807: dwrite d5/df/d31/f129 [0,4194304] 0 2024-04-12T20:56:14.444 INFO:tasks.workunit.client.0.smithi018.stdout:0/600: write da/d15/d17/d21/d36/d47/faf [1035072,123073] 0 2024-04-12T20:56:14.449 INFO:tasks.workunit.client.0.smithi018.stdout:3/661: rename d2/d6/d30/f3e to d2/d43/d61/d64/fd3 0 2024-04-12T20:56:14.450 INFO:tasks.workunit.client.0.smithi018.stdout:5/685: rmdir d0/d2/d11/d37/dc2 39 2024-04-12T20:56:14.455 INFO:tasks.workunit.client.0.smithi018.stdout:7/624: chown d1/db/d31/d45/d8c/d9d/lb0 1209413989 1 2024-04-12T20:56:14.462 INFO:tasks.workunit.client.0.smithi018.stdout:5/686: creat d0/d2/d33/d10/d2f/dbf/fe6 x:0 0 0 2024-04-12T20:56:14.463 INFO:tasks.workunit.client.0.smithi018.stdout:9/742: rename d3/d6/d9/dc/d5a/d79/d7e/d9b/d21/d41/f76 to d3/d6/d9/dc/d5a/d79/d7e/d9b/d21/dc3/dd8/fdc 0 2024-04-12T20:56:14.471 INFO:tasks.workunit.client.0.smithi018.stdout:5/687: symlink d0/d2/d33/d10/d2f/dbf/da3/dae/le7 0 2024-04-12T20:56:14.475 INFO:tasks.workunit.client.0.smithi018.stdout:9/743: creat d3/d6/d9/fdd x:0 0 0 2024-04-12T20:56:14.477 INFO:tasks.workunit.client.0.smithi018.stdout:3/662: dwrite d2/d43/db6/f80 [4194304,4194304] 0 2024-04-12T20:56:14.479 INFO:tasks.workunit.client.0.smithi018.stdout:9/744: fdatasync d3/d6/d9/dc/d6f/d77/fd3 0 2024-04-12T20:56:14.482 INFO:tasks.workunit.client.0.smithi018.stdout:5/688: creat d0/d2/d33/d46/d1e/d55/d66/fe8 x:0 0 0 2024-04-12T20:56:14.495 INFO:tasks.workunit.client.0.smithi018.stdout:3/663: link d2/d6/d30/c4b d2/d6/cd4 0 2024-04-12T20:56:14.497 INFO:tasks.workunit.client.0.smithi018.stdout:6/808: dread d5/df/d31/d4e/f38 [0,4194304] 0 2024-04-12T20:56:14.500 INFO:tasks.workunit.client.0.smithi018.stdout:6/809: readlink d5/df/d19/d2b/laf 0 2024-04-12T20:56:14.500 INFO:tasks.workunit.client.0.smithi018.stdout:9/745: dread d3/d6/d9/dc/d6f/d77/fd3 [0,4194304] 0 2024-04-12T20:56:14.510 INFO:tasks.workunit.client.0.smithi018.stdout:6/810: creat d5/d6/d1b/d3b/d6a/d116/f138 x:0 0 0 2024-04-12T20:56:14.512 INFO:tasks.workunit.client.0.smithi018.stdout:5/689: dwrite d0/d2/d33/d46/d1e/faa [0,4194304] 0 2024-04-12T20:56:14.514 INFO:tasks.workunit.client.0.smithi018.stdout:7/625: dread d1/db/d31/d45/f7c [0,4194304] 0 2024-04-12T20:56:14.606 INFO:tasks.workunit.client.0.smithi018.stdout:6/811: stat d5/d6/d1b/d3b/d6a/d116/d104/d128/l137 0 2024-04-12T20:56:14.613 INFO:tasks.workunit.client.0.smithi018.stdout:7/626: rmdir d1/db/d31/d30/dc6 39 2024-04-12T20:56:14.614 INFO:tasks.workunit.client.0.smithi018.stdout:3/664: dwrite d2/d6/f25 [0,4194304] 0 2024-04-12T20:56:14.619 INFO:tasks.workunit.client.0.smithi018.stdout:7/627: fdatasync d1/db/d31/d30/fbf 0 2024-04-12T20:56:14.619 INFO:tasks.workunit.client.0.smithi018.stdout:3/665: readlink d2/d6/d34/lbb 0 2024-04-12T20:56:14.624 INFO:tasks.workunit.client.0.smithi018.stdout:5/690: dwrite d0/d2/d33/f15 [12582912,4194304] 0 2024-04-12T20:56:14.624 INFO:tasks.workunit.client.0.smithi018.stdout:9/746: dwrite d3/d6/d9/dc/d5a/d79/d7e/d9b/f34 [0,4194304] 0 2024-04-12T20:56:14.633 INFO:tasks.workunit.client.0.smithi018.stdout:3/666: unlink d2/d6/d30/d31/l33 0 2024-04-12T20:56:14.633 INFO:tasks.workunit.client.0.smithi018.stdout:6/812: readlink d5/df/d3f/dca/l10a 0 2024-04-12T20:56:14.637 INFO:tasks.workunit.client.0.smithi018.stdout:5/691: unlink d0/d2/d33/d46/d1e/fd7 0 2024-04-12T20:56:14.640 INFO:tasks.workunit.client.0.smithi018.stdout:5/692: chown d0/d2/d33/d46/d1e/faa 2 1 2024-04-12T20:56:14.646 INFO:tasks.workunit.client.0.smithi018.stdout:6/813: fdatasync d5/f10 0 2024-04-12T20:56:14.648 INFO:tasks.workunit.client.0.smithi018.stdout:3/667: truncate d2/d6/d30/d31/d38/d3a/d48/f89 1021879 0 2024-04-12T20:56:14.651 INFO:tasks.workunit.client.0.smithi018.stdout:5/693: mkdir d0/d2/d11/d37/dc2/d89/d8d/de9 0 2024-04-12T20:56:14.653 INFO:tasks.workunit.client.0.smithi018.stdout:3/668: dread - d2/d6/d30/d31/f9c zero size 2024-04-12T20:56:14.655 INFO:tasks.workunit.client.0.smithi018.stdout:3/669: fsync d2/d6/f1c 0 2024-04-12T20:56:14.656 INFO:tasks.workunit.client.0.smithi018.stdout:5/694: creat d0/d2/de4/fea x:0 0 0 2024-04-12T20:56:14.660 INFO:tasks.workunit.client.0.smithi018.stdout:5/695: creat d0/d2/de4/feb x:0 0 0 2024-04-12T20:56:14.662 INFO:tasks.workunit.client.0.smithi018.stdout:3/670: symlink d2/d43/d61/d64/dbf/ld5 0 2024-04-12T20:56:14.662 INFO:tasks.workunit.client.0.smithi018.stdout:7/628: dread d1/db/d31/d45/f9c [0,4194304] 0 2024-04-12T20:56:14.666 INFO:tasks.workunit.client.0.smithi018.stdout:3/671: creat d2/d6/d34/fd6 x:0 0 0 2024-04-12T20:56:14.667 INFO:tasks.workunit.client.0.smithi018.stdout:3/672: chown d2/d6/f1b 51061333 1 2024-04-12T20:56:14.670 INFO:tasks.workunit.client.0.smithi018.stdout:7/629: truncate d1/d6/f24 2609366 0 2024-04-12T20:56:14.672 INFO:tasks.workunit.client.0.smithi018.stdout:3/673: symlink d2/d6/ld7 0 2024-04-12T20:56:14.675 INFO:tasks.workunit.client.0.smithi018.stdout:7/630: mkdir d1/db/d31/d3a/d42/dcd 0 2024-04-12T20:56:14.679 INFO:tasks.workunit.client.0.smithi018.stdout:6/814: dread d5/df/d19/d74/f9f [0,4194304] 0 2024-04-12T20:56:14.681 INFO:tasks.workunit.client.0.smithi018.stdout:7/631: symlink d1/lce 0 2024-04-12T20:56:14.685 INFO:tasks.workunit.client.0.smithi018.stdout:7/632: creat d1/db/d31/d3a/d97/fcf x:0 0 0 2024-04-12T20:56:14.690 INFO:tasks.workunit.client.0.smithi018.stdout:7/633: read d1/db/d31/d3a/d42/f75 [1825256,12077] 0 2024-04-12T20:56:14.692 INFO:tasks.workunit.client.0.smithi018.stdout:7/634: fsync d1/db/d31/d3a/d97/fa1 0 2024-04-12T20:56:14.695 INFO:tasks.workunit.client.0.smithi018.stdout:3/674: write d2/d6/d30/d31/f50 [2385842,28554] 0 2024-04-12T20:56:14.699 INFO:tasks.workunit.client.0.smithi018.stdout:7/635: symlink d1/db/d31/d45/d8c/d9d/d70/d96/d5f/da9/ld0 0 2024-04-12T20:56:14.703 INFO:tasks.workunit.client.0.smithi018.stdout:3/675: rename d2/d43/db6/d6b to d2/d43/db6/db5/dd8 0 2024-04-12T20:56:14.705 INFO:tasks.workunit.client.0.smithi018.stdout:5/696: dread d0/d2/d11/f4b [0,4194304] 0 2024-04-12T20:56:14.707 INFO:tasks.workunit.client.0.smithi018.stdout:3/676: truncate d2/d83/fce 869686 0 2024-04-12T20:56:14.708 INFO:tasks.workunit.client.0.smithi018.stdout:7/636: mknod d1/d6/dc9/cd1 0 2024-04-12T20:56:14.713 INFO:tasks.workunit.client.0.smithi018.stdout:3/677: unlink d2/d6/f2f 0 2024-04-12T20:56:14.715 INFO:tasks.workunit.client.0.smithi018.stdout:9/747: dwrite d3/d6/d9/dc/d5a/d79/d7e/d9b/d21/f59 [0,4194304] 0 2024-04-12T20:56:14.715 INFO:tasks.workunit.client.0.smithi018.stdout:3/678: chown d2/d6/d34/d71 516713 1 2024-04-12T20:56:14.717 INFO:tasks.workunit.client.0.smithi018.stdout:5/697: rename d0/d2/d33/d10/d2f to d0/d2/d33/d46/d1e/d9a/dec 0 2024-04-12T20:56:14.719 INFO:tasks.workunit.client.0.smithi018.stdout:7/637: creat d1/db/d31/d3a/d42/dcd/fd2 x:0 0 0 2024-04-12T20:56:14.720 INFO:tasks.workunit.client.0.smithi018.stdout:5/698: stat d0/d2/d33/d46/d1e/d55/d66 0 2024-04-12T20:56:14.722 INFO:tasks.workunit.client.0.smithi018.stdout:1/501: sync 2024-04-12T20:56:14.722 INFO:tasks.workunit.client.0.smithi018.stdout:8/763: sync 2024-04-12T20:56:14.727 INFO:tasks.workunit.client.0.smithi018.stdout:6/815: dwrite d5/d6/d1b/d3b/d22/fe7 [0,4194304] 0 2024-04-12T20:56:14.727 INFO:tasks.workunit.client.0.smithi018.stdout:5/699: mkdir d0/d2/d33/d10/d4e/ded 0 2024-04-12T20:56:14.728 INFO:tasks.workunit.client.0.smithi018.stdout:6/816: truncate d5/d6/d1b/f61 5878777 0 2024-04-12T20:56:14.738 INFO:tasks.workunit.client.0.smithi018.stdout:6/817: rename d5/df/d31/d10c to d5/d6/d90/d125/d139 0 2024-04-12T20:56:14.740 INFO:tasks.workunit.client.0.smithi018.stdout:6/818: write d5/df/d19/f25 [3699331,125282] 0 2024-04-12T20:56:14.753 INFO:tasks.workunit.client.0.smithi018.stdout:5/700: dwrite d0/d2/d11/d37/dc2/d89/d8d/fe5 [0,4194304] 0 2024-04-12T20:56:14.760 INFO:tasks.workunit.client.0.smithi018.stdout:1/502: dread d3/d9/da/dd/d31/d32/f1b [0,4194304] 0 2024-04-12T20:56:14.763 INFO:tasks.workunit.client.0.smithi018.stdout:8/764: dwrite db/d4e/d56/d59/d76/de0/d27/f5f [0,4194304] 0 2024-04-12T20:56:14.766 INFO:tasks.workunit.client.0.smithi018.stdout:1/503: creat d3/d9/faf x:0 0 0 2024-04-12T20:56:14.767 INFO:tasks.workunit.client.0.smithi018.stdout:6/819: dwrite d5/df/d19/d27/dc8/f112 [0,4194304] 0 2024-04-12T20:56:14.767 INFO:tasks.workunit.client.0.smithi018.stdout:5/701: truncate d0/d2/d33/d46/d35/d4d/db3/dc3/fcd 763552 0 2024-04-12T20:56:14.768 INFO:tasks.workunit.client.0.smithi018.stdout:8/765: creat db/d4e/d9a/d71/ff4 x:0 0 0 2024-04-12T20:56:14.771 INFO:tasks.workunit.client.0.smithi018.stdout:1/504: write d3/d9/da/dd/d31/d32/f34 [2041544,116602] 0 2024-04-12T20:56:14.771 INFO:tasks.workunit.client.0.smithi018.stdout:5/702: fdatasync d0/d2/d33/d46/f3a 0 2024-04-12T20:56:14.772 INFO:tasks.workunit.client.0.smithi018.stdout:8/766: chown db/d4e/d56/d59/d76/da5/d45/c74 377555 1 2024-04-12T20:56:14.774 INFO:tasks.workunit.client.0.smithi018.stdout:5/703: fdatasync d0/d2/d33/d46/d35/fcc 0 2024-04-12T20:56:14.775 INFO:tasks.workunit.client.0.smithi018.stdout:8/767: chown db/f64 7746 1 2024-04-12T20:56:14.778 INFO:tasks.workunit.client.0.smithi018.stdout:5/704: fsync d0/d2/d33/d46/d1e/d2e/f75 0 2024-04-12T20:56:14.780 INFO:tasks.workunit.client.0.smithi018.stdout:8/768: symlink db/d4e/d7f/lf5 0 2024-04-12T20:56:14.782 INFO:tasks.workunit.client.0.smithi018.stdout:8/769: readlink db/d4e/d7f/dba/lc4 0 2024-04-12T20:56:14.786 INFO:tasks.workunit.client.0.smithi018.stdout:8/770: link db/d4e/f84 db/d4e/dab/ff6 0 2024-04-12T20:56:14.794 INFO:tasks.workunit.client.0.smithi018.stdout:5/705: fdatasync d0/d2/d11/d37/dc2/d89/d8d/fe5 0 2024-04-12T20:56:14.795 INFO:tasks.workunit.client.0.smithi018.stdout:8/771: dread db/d4e/d56/d59/d76/da5/fb0 [0,4194304] 0 2024-04-12T20:56:14.802 INFO:tasks.workunit.client.0.smithi018.stdout:5/706: dread d0/d2/d33/d46/d1e/d9a/dec/f53 [0,4194304] 0 2024-04-12T20:56:14.805 INFO:tasks.workunit.client.0.smithi018.stdout:8/772: mkdir db/d4e/d73/df7 0 2024-04-12T20:56:14.806 INFO:tasks.workunit.client.0.smithi018.stdout:8/773: stat db/d25/cc9 0 2024-04-12T20:56:14.809 INFO:tasks.workunit.client.0.smithi018.stdout:5/707: truncate d0/d2/d11/d37/f47 500581 0 2024-04-12T20:56:14.813 INFO:tasks.workunit.client.0.smithi018.stdout:5/708: symlink d0/d2/d11/dd8/lee 0 2024-04-12T20:56:14.816 INFO:tasks.workunit.client.0.smithi018.stdout:8/774: rename db/d4e/d9a/d69/d6c to db/d4e/d56/d59/d76/de0/d27/df8 0 2024-04-12T20:56:14.823 INFO:tasks.workunit.client.0.smithi018.stdout:8/775: creat db/d4e/d56/d59/d76/de0/ff9 x:0 0 0 2024-04-12T20:56:14.833 INFO:tasks.workunit.client.0.smithi018.stdout:5/709: write d0/d2/d33/d46/d1e/d2e/f12 [3712731,1596] 0 2024-04-12T20:56:14.835 INFO:tasks.workunit.client.0.smithi018.stdout:5/710: chown d0/d2/d33/d46/d1e/d55/d66 5285910 1 2024-04-12T20:56:14.850 INFO:tasks.workunit.client.0.smithi018.stdout:5/711: dwrite d0/d2/d33/d46/d35/f79 [0,4194304] 0 2024-04-12T20:56:14.862 INFO:tasks.workunit.client.0.smithi018.stdout:8/776: dread db/d4e/d56/d59/d76/de0/f2f [0,4194304] 0 2024-04-12T20:56:14.864 INFO:tasks.workunit.client.0.smithi018.stdout:5/712: link d0/d2/d33/d10/d26/f43 d0/d2/d33/d46/d35/dd2/fef 0 2024-04-12T20:56:14.865 INFO:tasks.workunit.client.0.smithi018.stdout:9/748: dwrite d3/d6/d9/dc/d5a/d79/d7e/d9b/f2e [0,4194304] 0 2024-04-12T20:56:14.867 INFO:tasks.workunit.client.0.smithi018.stdout:8/777: dread - db/d4e/d56/d59/d76/da5/d45/f98 zero size 2024-04-12T20:56:14.869 INFO:tasks.workunit.client.0.smithi018.stdout:5/713: unlink d0/d2/d33/d46/d35/l6d 0 2024-04-12T20:56:14.875 INFO:tasks.workunit.client.0.smithi018.stdout:9/749: creat d3/d6/d9/dc/d6f/d77/fde x:0 0 0 2024-04-12T20:56:14.875 INFO:tasks.workunit.client.0.smithi018.stdout:5/714: readlink d0/d2/d33/d46/d1e/d9a/dec/dbf/da3/dae/lc1 0 2024-04-12T20:56:14.876 INFO:tasks.workunit.client.0.smithi018.stdout:5/715: readlink d0/d2/d33/d10/d4e/l51 0 2024-04-12T20:56:14.877 INFO:tasks.workunit.client.0.smithi018.stdout:9/750: creat d3/d93/daf/fdf x:0 0 0 2024-04-12T20:56:14.879 INFO:tasks.workunit.client.0.smithi018.stdout:9/751: dread - d3/d6/d9/fdd zero size 2024-04-12T20:56:14.880 INFO:tasks.workunit.client.0.smithi018.stdout:5/716: mknod d0/d2/d11/d37/dc2/cf0 0 2024-04-12T20:56:14.884 INFO:tasks.workunit.client.0.smithi018.stdout:5/717: chown d0/d2/d33/d46/d1e/d55/d66/f93 2970690 1 2024-04-12T20:56:14.888 INFO:tasks.workunit.client.0.smithi018.stdout:8/778: dwrite db/d25/f52 [0,4194304] 0 2024-04-12T20:56:14.892 INFO:tasks.workunit.client.0.smithi018.stdout:8/779: mkdir db/d4e/d7f/dfa 0 2024-04-12T20:56:14.898 INFO:tasks.workunit.client.0.smithi018.stdout:9/752: dwrite d3/d6/fda [0,4194304] 0 2024-04-12T20:56:14.902 INFO:tasks.workunit.client.0.smithi018.stdout:9/753: write d3/d6/d9/d1e/f96 [1782488,89191] 0 2024-04-12T20:56:14.904 INFO:tasks.workunit.client.0.smithi018.stdout:5/718: dwrite d0/d2/d33/d46/d18/fbc [0,4194304] 0 2024-04-12T20:56:14.906 INFO:tasks.workunit.client.0.smithi018.stdout:9/754: chown d3/d6/d9/l99 1451418 1 2024-04-12T20:56:14.915 INFO:tasks.workunit.client.0.smithi018.stdout:0/601: sync 2024-04-12T20:56:14.915 INFO:tasks.workunit.client.0.smithi018.stdout:2/586: sync 2024-04-12T20:56:14.915 INFO:tasks.workunit.client.0.smithi018.stdout:4/665: sync 2024-04-12T20:56:14.916 INFO:tasks.workunit.client.0.smithi018.stdout:5/719: getdents d0/d2/d33/d46/d1e/d55 0 2024-04-12T20:56:14.920 INFO:tasks.workunit.client.0.smithi018.stdout:4/666: dread - d3/fc6 zero size 2024-04-12T20:56:14.921 INFO:tasks.workunit.client.0.smithi018.stdout:5/720: mknod d0/d2/d33/d46/d35/d4d/cf1 0 2024-04-12T20:56:14.924 INFO:tasks.workunit.client.0.smithi018.stdout:4/667: stat d3/d32/d94/fcb 0 2024-04-12T20:56:14.934 INFO:tasks.workunit.client.0.smithi018.stdout:4/668: truncate d3/db/f7f 68629 0 2024-04-12T20:56:14.938 INFO:tasks.workunit.client.0.smithi018.stdout:2/587: write d5/d8/d10/d43/f56 [1419296,25905] 0 2024-04-12T20:56:14.940 INFO:tasks.workunit.client.0.smithi018.stdout:5/721: link d0/d2/d33/d46/d1e/d2e/f75 d0/d2/ff2 0 2024-04-12T20:56:14.943 INFO:tasks.workunit.client.0.smithi018.stdout:0/602: dwrite da/d15/d17/d71/fc2 [0,4194304] 0 2024-04-12T20:56:14.943 INFO:tasks.workunit.client.0.smithi018.stdout:9/755: dwrite d3/d6/d9/dc/d5a/d79/d7e/d9b/d21/dc3/dd8/d5b/d6b/f92 [0,4194304] 0 2024-04-12T20:56:14.943 INFO:tasks.workunit.client.0.smithi018.stdout:8/780: dread db/d4e/d56/d59/d76/de0/f3f [0,4194304] 0 2024-04-12T20:56:14.946 INFO:tasks.workunit.client.0.smithi018.stdout:2/588: write d5/d8/d10/f48 [8962165,82613] 0 2024-04-12T20:56:14.947 INFO:tasks.workunit.client.0.smithi018.stdout:5/722: chown d0/d2/d33/d46/d1e/d9a/dec/dbf/da3/fe3 28 1 2024-04-12T20:56:14.950 INFO:tasks.workunit.client.0.smithi018.stdout:8/781: chown db/d4e/d7f/dba/lc4 3861194 1 2024-04-12T20:56:14.950 INFO:tasks.workunit.client.0.smithi018.stdout:9/756: chown d3/d6/d43/fd4 25 1 2024-04-12T20:56:14.954 INFO:tasks.workunit.client.0.smithi018.stdout:4/669: symlink d3/db/ld9 0 2024-04-12T20:56:14.957 INFO:tasks.workunit.client.0.smithi018.stdout:8/782: creat db/d4e/d56/d59/d76/da5/d45/d86/ffb x:0 0 0 2024-04-12T20:56:14.959 INFO:tasks.workunit.client.0.smithi018.stdout:0/603: fdatasync da/d15/d17/f43 0 2024-04-12T20:56:14.963 INFO:tasks.workunit.client.0.smithi018.stdout:2/589: truncate d5/d8/d10/d19/d24/f4b 175307 0 2024-04-12T20:56:14.966 INFO:tasks.workunit.client.0.smithi018.stdout:9/757: truncate d3/d6/d9/dc/f22 831897 0 2024-04-12T20:56:14.966 INFO:tasks.workunit.client.0.smithi018.stdout:4/670: creat d3/d32/d33/dbc/fda x:0 0 0 2024-04-12T20:56:14.974 INFO:tasks.workunit.client.0.smithi018.stdout:9/758: truncate d3/d6/d9/dc/d6f/d77/fd3 4491445 0 2024-04-12T20:56:14.976 INFO:tasks.workunit.client.0.smithi018.stdout:4/671: unlink d3/d5d/fb2 0 2024-04-12T20:56:14.978 INFO:tasks.workunit.client.0.smithi018.stdout:8/783: creat db/d4e/d7f/dfa/ffc x:0 0 0 2024-04-12T20:56:14.979 INFO:tasks.workunit.client.0.smithi018.stdout:9/759: readlink d3/d6/d9/dc/d5a/d79/d7e/d9b/d21/dc3/dd8/d5b/l75 0 2024-04-12T20:56:14.985 INFO:tasks.workunit.client.0.smithi018.stdout:4/672: rmdir d3/db 39 2024-04-12T20:56:14.985 INFO:tasks.workunit.client.0.smithi018.stdout:5/723: dwrite d0/d2/d33/d46/d35/f62 [0,4194304] 0 2024-04-12T20:56:14.986 INFO:tasks.workunit.client.0.smithi018.stdout:9/760: write d3/d6/d9/dc/d5a/d79/fb3 [799714,126263] 0 2024-04-12T20:56:14.991 INFO:tasks.workunit.client.0.smithi018.stdout:0/604: getdents da/d15/d17/d21 0 2024-04-12T20:56:14.991 INFO:tasks.workunit.client.0.smithi018.stdout:8/784: getdents db/d4e/d73/df7 0 2024-04-12T20:56:14.993 INFO:tasks.workunit.client.0.smithi018.stdout:4/673: write d3/d32/d33/db6/fca [214342,82144] 0 2024-04-12T20:56:14.996 INFO:tasks.workunit.client.0.smithi018.stdout:9/761: write d3/d6/d3d/dba/fc9 [965195,68387] 0 2024-04-12T20:56:15.001 INFO:tasks.workunit.client.0.smithi018.stdout:2/590: dwrite d5/d8/d10/d19/d24/d39/d9c/fb0 [0,4194304] 0 2024-04-12T20:56:15.002 INFO:tasks.workunit.client.0.smithi018.stdout:5/724: creat d0/d2/d33/d46/d1e/d9a/dec/ff3 x:0 0 0 2024-04-12T20:56:15.004 INFO:tasks.workunit.client.0.smithi018.stdout:4/674: unlink d3/lf 0 2024-04-12T20:56:15.009 INFO:tasks.workunit.client.0.smithi018.stdout:4/675: chown d3/d5d/d60/d64/d4f/f95 96558507 1 2024-04-12T20:56:15.013 INFO:tasks.workunit.client.0.smithi018.stdout:8/785: rename db/d4e/d56/d59/fb9 to db/d4e/d9a/ffd 0 2024-04-12T20:56:15.013 INFO:tasks.workunit.client.0.smithi018.stdout:2/591: mkdir d5/d8/d10/d19/daf/d88/dcd 0 2024-04-12T20:56:15.017 INFO:tasks.workunit.client.0.smithi018.stdout:2/592: symlink d5/d7/d31/d62/lce 0 2024-04-12T20:56:15.018 INFO:tasks.workunit.client.0.smithi018.stdout:4/676: truncate d3/d5d/d60/dc5/d53/f5e 1011342 0 2024-04-12T20:56:15.019 INFO:tasks.workunit.client.0.smithi018.stdout:8/786: symlink db/d4e/d9a/d71/lfe 0 2024-04-12T20:56:15.023 INFO:tasks.workunit.client.0.smithi018.stdout:5/725: dread d0/d2/d33/d46/d1e/d9a/dec/d8f/fb2 [0,4194304] 0 2024-04-12T20:56:15.026 INFO:tasks.workunit.client.0.smithi018.stdout:8/787: creat db/fff x:0 0 0 2024-04-12T20:56:15.027 INFO:tasks.workunit.client.0.smithi018.stdout:9/762: write d3/d6/d9/dc/f22 [1461673,97937] 0 2024-04-12T20:56:15.030 INFO:tasks.workunit.client.0.smithi018.stdout:8/788: readlink db/d4e/d56/l85 0 2024-04-12T20:56:15.033 INFO:tasks.workunit.client.0.smithi018.stdout:5/726: rename d0/d2/d33/d10/d26/la9 to d0/d2/d33/d46/d1e/d55/lf4 0 2024-04-12T20:56:15.035 INFO:tasks.workunit.client.0.smithi018.stdout:4/677: getdents d3/d5d/d60 0 2024-04-12T20:56:15.036 INFO:tasks.workunit.client.0.smithi018.stdout:5/727: write d0/d5b/fd6 [483562,88726] 0 2024-04-12T20:56:15.038 INFO:tasks.workunit.client.0.smithi018.stdout:4/678: fdatasync d3/d5d/d60/dc5/fb0 0 2024-04-12T20:56:15.042 INFO:tasks.workunit.client.0.smithi018.stdout:2/593: write d5/d8/f1e [2357183,92295] 0 2024-04-12T20:56:15.049 INFO:tasks.workunit.client.0.smithi018.stdout:5/728: creat d0/d2/d33/d46/d9e/ff5 x:0 0 0 2024-04-12T20:56:15.050 INFO:tasks.workunit.client.0.smithi018.stdout:4/679: symlink d3/d5d/d60/dc5/d3d/da3/dc0/ldb 0 2024-04-12T20:56:15.059 INFO:tasks.workunit.client.0.smithi018.stdout:5/729: creat d0/d2/d33/d46/d35/dd2/ff6 x:0 0 0 2024-04-12T20:56:15.059 INFO:tasks.workunit.client.0.smithi018.stdout:4/680: mkdir d3/d5d/d60/dc5/d43/d4b/ddc 0 2024-04-12T20:56:15.060 INFO:tasks.workunit.client.0.smithi018.stdout:9/763: dwrite d3/d6/d9/dc/d5a/fae [0,4194304] 0 2024-04-12T20:56:15.061 INFO:tasks.workunit.client.0.smithi018.stdout:2/594: write d5/d8/d25/f7d [726275,109349] 0 2024-04-12T20:56:15.065 INFO:tasks.workunit.client.0.smithi018.stdout:9/764: fdatasync d3/d6/fda 0 2024-04-12T20:56:15.069 INFO:tasks.workunit.client.0.smithi018.stdout:2/595: symlink d5/d8/d10/lcf 0 2024-04-12T20:56:15.069 INFO:tasks.workunit.client.0.smithi018.stdout:8/789: dwrite db/d4e/d9a/ffd [0,4194304] 0 2024-04-12T20:56:15.074 INFO:tasks.workunit.client.0.smithi018.stdout:8/790: chown fa 408624 1 2024-04-12T20:56:15.079 INFO:tasks.workunit.client.0.smithi018.stdout:9/765: symlink d3/d6/d9/dc/d5a/d79/d7e/d9b/d21/dc3/le0 0 2024-04-12T20:56:15.079 INFO:tasks.workunit.client.0.smithi018.stdout:2/596: symlink d5/d8/d10/d19/d24/d39/da4/ld0 0 2024-04-12T20:56:15.085 INFO:tasks.workunit.client.0.smithi018.stdout:9/766: readlink d3/d6/d9/d3f/l9c 0 2024-04-12T20:56:15.085 INFO:tasks.workunit.client.0.smithi018.stdout:2/597: dread - d5/d8/d10/d19/daf/d90/fb4 zero size 2024-04-12T20:56:15.094 INFO:tasks.workunit.client.0.smithi018.stdout:5/730: dwrite d0/d2/d33/d46/d35/dd2/ff6 [0,4194304] 0 2024-04-12T20:56:15.094 INFO:tasks.workunit.client.0.smithi018.stdout:9/767: fsync d3/d6/d9/dc/d5a/d79/d7e/d9b/d21/d41/d78/f98 0 2024-04-12T20:56:15.094 INFO:tasks.workunit.client.0.smithi018.stdout:4/681: dwrite d3/d5d/d60/dc5/d43/fc9 [0,4194304] 0 2024-04-12T20:56:15.099 INFO:tasks.workunit.client.0.smithi018.stdout:5/731: write d0/d5b/f5d [780425,130211] 0 2024-04-12T20:56:15.105 INFO:tasks.workunit.client.0.smithi018.stdout:4/682: unlink d3/d5d/d60/dc5/d43/d4b/fb5 0 2024-04-12T20:56:15.112 INFO:tasks.workunit.client.0.smithi018.stdout:8/791: dwrite db/d4e/d56/f63 [0,4194304] 0 2024-04-12T20:56:15.115 INFO:tasks.workunit.client.0.smithi018.stdout:4/683: rename d3/d5d/d60/d64/c86 to d3/d5d/d60/d64/d72/cdd 0 2024-04-12T20:56:15.119 INFO:tasks.workunit.client.0.smithi018.stdout:2/598: truncate d5/d8/d10/d19/f7b 4255251 0 2024-04-12T20:56:15.119 INFO:tasks.workunit.client.0.smithi018.stdout:8/792: chown db/d3b/f49 64637342 1 2024-04-12T20:56:15.120 INFO:tasks.workunit.client.0.smithi018.stdout:2/599: stat d5/d12/d54/c8e 0 2024-04-12T20:56:15.125 INFO:tasks.workunit.client.0.smithi018.stdout:8/793: mkdir db/d25/d100 0 2024-04-12T20:56:15.129 INFO:tasks.workunit.client.0.smithi018.stdout:9/768: truncate d3/d93/fbc 268 0 2024-04-12T20:56:15.131 INFO:tasks.workunit.client.0.smithi018.stdout:3/679: sync 2024-04-12T20:56:15.138 INFO:tasks.workunit.client.0.smithi018.stdout:9/769: chown d3/d93/ld0 3 1 2024-04-12T20:56:15.138 INFO:tasks.workunit.client.0.smithi018.stdout:3/680: truncate d2/d43/fb3 215800 0 2024-04-12T20:56:15.140 INFO:tasks.workunit.client.0.smithi018.stdout:8/794: unlink db/d4e/d56/d59/d76/da5/d45/d86/da1/fa4 0 2024-04-12T20:56:15.141 INFO:tasks.workunit.client.0.smithi018.stdout:9/770: fdatasync d3/d6/d9/dc/d5a/d79/d7e/d9b/f73 0 2024-04-12T20:56:15.150 INFO:tasks.workunit.client.0.smithi018.stdout:4/684: dwrite d3/d32/d33/dbc/fda [0,4194304] 0 2024-04-12T20:56:15.165 INFO:tasks.workunit.client.0.smithi018.stdout:3/681: truncate d2/d6/d30/d31/d38/f44 645569 0 2024-04-12T20:56:15.165 INFO:tasks.workunit.client.0.smithi018.stdout:8/795: fsync db/d4e/d56/d59/d76/de0/f53 0 2024-04-12T20:56:15.166 INFO:tasks.workunit.client.0.smithi018.stdout:4/685: mknod d3/d5d/d60/d64/d79/cde 0 2024-04-12T20:56:15.166 INFO:tasks.workunit.client.0.smithi018.stdout:5/732: dread d0/d5b/d84/f31 [0,4194304] 0 2024-04-12T20:56:15.170 INFO:tasks.workunit.client.0.smithi018.stdout:3/682: readlink d2/l75 0 2024-04-12T20:56:15.174 INFO:tasks.workunit.client.0.smithi018.stdout:4/686: read d3/d32/d33/dbc/fda [627331,75578] 0 2024-04-12T20:56:15.176 INFO:tasks.workunit.client.0.smithi018.stdout:6/820: sync 2024-04-12T20:56:15.177 INFO:tasks.workunit.client.0.smithi018.stdout:1/505: sync 2024-04-12T20:56:15.177 INFO:tasks.workunit.client.0.smithi018.stdout:0/605: sync 2024-04-12T20:56:15.177 INFO:tasks.workunit.client.0.smithi018.stdout:7/638: sync 2024-04-12T20:56:15.179 INFO:tasks.workunit.client.0.smithi018.stdout:8/796: write db/d4e/d9a/daa/dcd/fe2 [955792,21085] 0 2024-04-12T20:56:15.180 INFO:tasks.workunit.client.0.smithi018.stdout:3/683: stat d2/c18 0 2024-04-12T20:56:15.189 INFO:tasks.workunit.client.0.smithi018.stdout:7/639: fsync d1/f66 0 2024-04-12T20:56:15.190 INFO:tasks.workunit.client.0.smithi018.stdout:2/600: dwrite d5/d7/d79/f8c [0,4194304] 0 2024-04-12T20:56:15.193 INFO:tasks.workunit.client.0.smithi018.stdout:4/687: rmdir d3/d32/d33 39 2024-04-12T20:56:15.193 INFO:tasks.workunit.client.0.smithi018.stdout:1/506: chown d3/d9/da/dd/d31/d32/d33/d80 416460 1 2024-04-12T20:56:15.197 INFO:tasks.workunit.client.0.smithi018.stdout:0/606: creat da/d15/d17/d21/d94/fc7 x:0 0 0 2024-04-12T20:56:15.199 INFO:tasks.workunit.client.0.smithi018.stdout:9/771: dwrite d3/d6/d9/dc/d5a/d79/d7e/d9b/d21/dc3/dd8/fd2 [0,4194304] 0 2024-04-12T20:56:15.203 INFO:tasks.workunit.client.0.smithi018.stdout:1/507: unlink d3/d9/da/l93 0 2024-04-12T20:56:15.204 INFO:tasks.workunit.client.0.smithi018.stdout:4/688: write d3/d32/d33/d8e/fd8 [430269,64526] 0 2024-04-12T20:56:15.204 INFO:tasks.workunit.client.0.smithi018.stdout:8/797: creat db/d4e/f101 x:0 0 0 2024-04-12T20:56:15.206 INFO:tasks.workunit.client.0.smithi018.stdout:2/601: write d5/d8/d10/d19/d24/d39/d9c/fb0 [2617207,56748] 0 2024-04-12T20:56:15.206 INFO:tasks.workunit.client.0.smithi018.stdout:7/640: creat d1/fd3 x:0 0 0 2024-04-12T20:56:15.214 INFO:tasks.workunit.client.0.smithi018.stdout:8/798: creat db/d4e/d73/f102 x:0 0 0 2024-04-12T20:56:15.216 INFO:tasks.workunit.client.0.smithi018.stdout:5/733: link d0/d2/d33/d46/d1e/d9a/dec/dbf/da3/dae/lc1 d0/d2/d11/d37/dc2/d89/d8d/lf7 0 2024-04-12T20:56:15.220 INFO:tasks.workunit.client.0.smithi018.stdout:5/734: readlink d0/d2/d33/d46/d35/ld4 0 2024-04-12T20:56:15.225 INFO:tasks.workunit.client.0.smithi018.stdout:1/508: mkdir d3/d9/da/dd/d31/d32/d33/d47/db0 0 2024-04-12T20:56:15.240 INFO:tasks.workunit.client.0.smithi018.stdout:9/772: creat d3/d93/fe1 x:0 0 0 2024-04-12T20:56:15.240 INFO:tasks.workunit.client.0.smithi018.stdout:6/821: truncate d5/df/d19/d27/dc8/f112 3345265 0 2024-04-12T20:56:15.247 INFO:tasks.workunit.client.0.smithi018.stdout:3/684: dread d2/d4d/d88/f8a [0,4194304] 0 2024-04-12T20:56:15.249 INFO:tasks.workunit.client.0.smithi018.stdout:2/602: write d5/d8/d10/d19/d24/d39/da4/fc8 [423766,91665] 0 2024-04-12T20:56:15.250 INFO:tasks.workunit.client.0.smithi018.stdout:7/641: link d1/db/d31/d45/d8c/d9d/d70/d96/c33 d1/db/d31/d45/d8c/d9d/d70/d96/d5f/da9/cd4 0 2024-04-12T20:56:15.252 INFO:tasks.workunit.client.0.smithi018.stdout:6/822: dread d5/df/d9b/fc4 [0,4194304] 0 2024-04-12T20:56:15.254 INFO:tasks.workunit.client.0.smithi018.stdout:1/509: rmdir d3/d9/da/dd/d31/d32/d13/d19/d2d/d4e/d58/d61/d8a 0 2024-04-12T20:56:15.254 INFO:tasks.workunit.client.0.smithi018.stdout:0/607: dread da/d15/d17/d21/d36/da2/dc3/f5e [0,4194304] 0 2024-04-12T20:56:15.255 INFO:tasks.workunit.client.0.smithi018.stdout:9/773: read d3/d6/d9/dc/d5a/d79/d7e/d9b/d21/d41/d78/f98 [3438196,91798] 0 2024-04-12T20:56:15.257 INFO:tasks.workunit.client.0.smithi018.stdout:3/685: readlink d2/d6/d34/lbb 0 2024-04-12T20:56:15.257 INFO:tasks.workunit.client.0.smithi018.stdout:5/735: write d0/d2/d33/d46/d1e/d2e/f75 [2794837,45584] 0 2024-04-12T20:56:15.264 INFO:tasks.workunit.client.0.smithi018.stdout:0/608: stat da/d15/d17/d71/fc2 0 2024-04-12T20:56:15.265 INFO:tasks.workunit.client.0.smithi018.stdout:5/736: fsync d0/d2/de4/feb 0 2024-04-12T20:56:15.265 INFO:tasks.workunit.client.0.smithi018.stdout:3/686: chown d2/d6/d34/d71/d7b/f7d 22518 1 2024-04-12T20:56:15.267 INFO:tasks.workunit.client.0.smithi018.stdout:2/603: stat d5/d8/d10/c3b 0 2024-04-12T20:56:15.270 INFO:tasks.workunit.client.0.smithi018.stdout:1/510: symlink d3/d9/da/dd/d31/d32/lb1 0 2024-04-12T20:56:15.270 INFO:tasks.workunit.client.0.smithi018.stdout:7/642: truncate d1/db/d31/d3a/d42/dcd/fd2 697608 0 2024-04-12T20:56:15.272 INFO:tasks.workunit.client.0.smithi018.stdout:4/689: dwrite d3/d32/f4e [0,4194304] 0 2024-04-12T20:56:15.274 INFO:tasks.workunit.client.0.smithi018.stdout:3/687: write d2/d6/d30/d31/f69 [2345534,86228] 0 2024-04-12T20:56:15.275 INFO:tasks.workunit.client.0.smithi018.stdout:6/823: rename d5/d15/d60/d124/f12a to d5/df/d19/f13a 0 2024-04-12T20:56:15.284 INFO:tasks.workunit.client.0.smithi018.stdout:3/688: fdatasync d2/d6/fa8 0 2024-04-12T20:56:15.284 INFO:tasks.workunit.client.0.smithi018.stdout:5/737: creat d0/d2/d33/d10/d26/dda/ff8 x:0 0 0 2024-04-12T20:56:15.286 INFO:tasks.workunit.client.0.smithi018.stdout:6/824: link d5/d6/d1b/d3b/d22/l4d d5/d6/d90/d125/l13b 0 2024-04-12T20:56:15.286 INFO:tasks.workunit.client.0.smithi018.stdout:0/609: rmdir da/d15/d17/d21/d36/da2/dc3/d83 39 2024-04-12T20:56:15.287 INFO:tasks.workunit.client.0.smithi018.stdout:8/799: dread db/d4e/f6b [0,4194304] 0 2024-04-12T20:56:15.292 INFO:tasks.workunit.client.0.smithi018.stdout:3/689: mkdir d2/d43/d61/d64/dd9 0 2024-04-12T20:56:15.292 INFO:tasks.workunit.client.0.smithi018.stdout:7/643: truncate d1/db/d31/d45/f9e 633083 0 2024-04-12T20:56:15.295 INFO:tasks.workunit.client.0.smithi018.stdout:9/774: dwrite d3/d6/d9/dc/f22 [0,4194304] 0 2024-04-12T20:56:15.296 INFO:tasks.workunit.client.0.smithi018.stdout:6/825: rmdir d5/d15/dd1/d111 0 2024-04-12T20:56:15.297 INFO:tasks.workunit.client.0.smithi018.stdout:8/800: chown db/d95/dd6/ce1 7832 1 2024-04-12T20:56:15.299 INFO:tasks.workunit.client.0.smithi018.stdout:0/610: creat da/d15/d17/d21/d36/d47/fc8 x:0 0 0 2024-04-12T20:56:15.301 INFO:tasks.workunit.client.0.smithi018.stdout:6/826: dread - d5/d15/fa6 zero size 2024-04-12T20:56:15.303 INFO:tasks.workunit.client.0.smithi018.stdout:2/604: getdents d5/d8 0 2024-04-12T20:56:15.304 INFO:tasks.workunit.client.0.smithi018.stdout:0/611: chown da/d15/d4d/d58/cb6 139088 1 2024-04-12T20:56:15.305 INFO:tasks.workunit.client.0.smithi018.stdout:8/801: creat db/d72/f103 x:0 0 0 2024-04-12T20:56:15.307 INFO:tasks.workunit.client.0.smithi018.stdout:6/827: fdatasync d5/df/d19/f6c 0 2024-04-12T20:56:15.308 INFO:tasks.workunit.client.0.smithi018.stdout:1/511: getdents d3/d9/da/dd/d31/d32/d13/d19 0 2024-04-12T20:56:15.309 INFO:tasks.workunit.client.0.smithi018.stdout:7/644: dread d1/db/d31/d45/d53/f7d [0,4194304] 0 2024-04-12T20:56:15.311 INFO:tasks.workunit.client.0.smithi018.stdout:8/802: creat db/d3b/f104 x:0 0 0 2024-04-12T20:56:15.315 INFO:tasks.workunit.client.0.smithi018.stdout:9/775: unlink d3/d6/d9/dc/d5a/d79/d7e/d9b/d21/dc3/dd8/fdc 0 2024-04-12T20:56:15.320 INFO:tasks.workunit.client.0.smithi018.stdout:0/612: creat da/d15/d17/d21/d36/d6e/fc9 x:0 0 0 2024-04-12T20:56:15.320 INFO:tasks.workunit.client.0.smithi018.stdout:7/645: fsync d1/db/d31/f63 0 2024-04-12T20:56:15.321 INFO:tasks.workunit.client.0.smithi018.stdout:2/605: mknod d5/d8/d10/d19/cd1 0 2024-04-12T20:56:15.325 INFO:tasks.workunit.client.0.smithi018.stdout:1/512: creat d3/d9/da/dd/d31/d32/d33/d47/fb2 x:0 0 0 2024-04-12T20:56:15.335 INFO:tasks.workunit.client.0.smithi018.stdout:9/776: creat d3/fe2 x:0 0 0 2024-04-12T20:56:15.335 INFO:tasks.workunit.client.0.smithi018.stdout:2/606: dread d5/d7/d31/fbe [0,4194304] 0 2024-04-12T20:56:15.338 INFO:tasks.workunit.client.0.smithi018.stdout:3/690: dwrite d2/d6/d30/d31/f67 [4194304,4194304] 0 2024-04-12T20:56:15.350 INFO:tasks.workunit.client.0.smithi018.stdout:9/777: creat d3/d6/d9/dc/d5a/d79/d7e/d9b/d21/dc2/fe3 x:0 0 0 2024-04-12T20:56:15.353 INFO:tasks.workunit.client.0.smithi018.stdout:6/828: dwrite d5/fd [0,4194304] 0 2024-04-12T20:56:15.355 INFO:tasks.workunit.client.0.smithi018.stdout:7/646: unlink d1/db/d31/d45/d8c/d9d/lbe 0 2024-04-12T20:56:15.357 INFO:tasks.workunit.client.0.smithi018.stdout:1/513: truncate d3/d9/da/dd/f17 4795058 0 2024-04-12T20:56:15.358 INFO:tasks.workunit.client.0.smithi018.stdout:0/613: dwrite da/d15/f39 [4194304,4194304] 0 2024-04-12T20:56:15.364 INFO:tasks.workunit.client.0.smithi018.stdout:8/803: dwrite db/fb3 [4194304,4194304] 0 2024-04-12T20:56:15.369 INFO:tasks.workunit.client.0.smithi018.stdout:1/514: chown d3/d9/da/dd/d31/d32/d33/d80/c9b 128764 1 2024-04-12T20:56:15.383 INFO:tasks.workunit.client.0.smithi018.stdout:1/515: creat d3/d9/da/dd/d31/d32/d33/d47/fb3 x:0 0 0 2024-04-12T20:56:15.386 INFO:tasks.workunit.client.0.smithi018.stdout:3/691: dwrite d2/d6/d30/d31/f50 [0,4194304] 0 2024-04-12T20:56:15.388 INFO:tasks.workunit.client.0.smithi018.stdout:2/607: dwrite d5/d8/d10/d19/d7c/fbf [0,4194304] 0 2024-04-12T20:56:15.391 INFO:tasks.workunit.client.0.smithi018.stdout:8/804: truncate db/f2d 4480166 0 2024-04-12T20:56:15.392 INFO:tasks.workunit.client.0.smithi018.stdout:1/516: dread - d3/d9/da/dd/d31/d32/d33/d47/fb2 zero size 2024-04-12T20:56:15.394 INFO:tasks.workunit.client.0.smithi018.stdout:2/608: chown d5/d12/d54/c44 99276524 1 2024-04-12T20:56:15.394 INFO:tasks.workunit.client.0.smithi018.stdout:6/829: write d5/df/d19/f54 [329918,110444] 0 2024-04-12T20:56:15.394 INFO:tasks.workunit.client.0.smithi018.stdout:0/614: getdents da/d15 0 2024-04-12T20:56:15.402 INFO:tasks.workunit.client.0.smithi018.stdout:8/805: mknod db/d4e/d9a/dda/c105 0 2024-04-12T20:56:15.409 INFO:tasks.workunit.client.0.smithi018.stdout:8/806: chown db/d4e/d9a/d71/ff4 0 1 2024-04-12T20:56:15.416 INFO:tasks.workunit.client.0.smithi018.stdout:8/807: mkdir db/d25/d106 0 2024-04-12T20:56:15.416 INFO:tasks.workunit.client.0.smithi018.stdout:9/778: read d3/d6/d9/dc/d5a/d79/d7e/d9b/d21/d41/d78/f98 [1358271,94519] 0 2024-04-12T20:56:15.420 INFO:tasks.workunit.client.0.smithi018.stdout:1/517: dwrite d3/d9/da/dd/d26/fa6 [0,4194304] 0 2024-04-12T20:56:15.427 INFO:tasks.workunit.client.0.smithi018.stdout:1/518: stat d3/d9/da/dd/d26/c88 0 2024-04-12T20:56:15.428 INFO:tasks.workunit.client.0.smithi018.stdout:9/779: unlink c2 0 2024-04-12T20:56:15.433 INFO:tasks.workunit.client.0.smithi018.stdout:6/830: write d5/d6/d1b/d3b/d8e/d8f/fba [243252,42071] 0 2024-04-12T20:56:15.441 INFO:tasks.workunit.client.0.smithi018.stdout:9/780: write d3/d6/d9/dc/d5a/d79/fb3 [2445358,7202] 0 2024-04-12T20:56:15.452 INFO:tasks.workunit.client.0.smithi018.stdout:8/808: dwrite db/d4e/d56/d59/fd3 [0,4194304] 0 2024-04-12T20:56:15.454 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:56:15 smithi186 ceph-mon[36597]: pgmap v40: 65 pgs: 65 active+clean; 1.2 GiB data, 5.4 GiB used, 531 GiB / 536 GiB avail; 42 MiB/s rd, 110 MiB/s wr, 305 op/s 2024-04-12T20:56:15.454 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:56:15 smithi186 ceph-mon[36597]: from='mgr.24359 172.21.15.186:0/1315906828' entity='mgr.smithi186.kqcfxk' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T20:56:15.454 INFO:tasks.workunit.client.0.smithi018.stdout:7/647: dread d1/f1b [0,4194304] 0 2024-04-12T20:56:15.458 INFO:tasks.workunit.client.0.smithi018.stdout:6/831: rename d5/d6/d1b/d3b/d35/dda to d5/d6/d1b/d3b/d22/d117/d13c 0 2024-04-12T20:56:15.470 INFO:tasks.workunit.client.0.smithi018.stdout:1/519: link d3/d9/da/dd/d31/d32/d13/d19/d2d/d4e/d58/d61/d71/fa9 d3/d9/da/dd/d31/d32/d13/d19/fb4 0 2024-04-12T20:56:15.473 INFO:tasks.workunit.client.0.smithi018.stdout:6/832: read d5/d6/d1b/d3b/fa4 [3544849,74603] 0 2024-04-12T20:56:15.479 INFO:tasks.workunit.client.0.smithi018.stdout:7/648: mkdir d1/db/d31/d3a/d42/d4c/dbc/dd5 0 2024-04-12T20:56:15.482 INFO:tasks.workunit.client.0.smithi018.stdout:9/781: link d3/d6/d9/dc/d5a/d79/d7e/d9b/d21/dc3/dd8/d5b/l75 d3/d6/d9/dc/d5a/d79/d7e/le4 0 2024-04-12T20:56:15.484 INFO:tasks.workunit.client.0.smithi018.stdout:0/615: dread da/d15/f33 [4194304,4194304] 0 2024-04-12T20:56:15.487 INFO:tasks.workunit.client.0.smithi018.stdout:2/609: dwrite d5/d8/d10/d19/d7c/f9f [0,4194304] 0 2024-04-12T20:56:15.488 INFO:tasks.workunit.client.0.smithi018.stdout:9/782: chown d3/d6/d9/dc/dab 313 1 2024-04-12T20:56:15.488 INFO:tasks.workunit.client.0.smithi018.stdout:3/692: dwrite d2/d43/d61/d64/dbf/d54/fb7 [0,4194304] 0 2024-04-12T20:56:15.495 INFO:tasks.workunit.client.0.smithi018.stdout:8/809: dwrite db/d4e/d56/d59/d76/da5/d45/d86/da1/ff2 [0,4194304] 0 2024-04-12T20:56:15.500 INFO:tasks.workunit.client.0.smithi018.stdout:2/610: unlink d5/d8/d10/d19/d24/d39/f45 0 2024-04-12T20:56:15.503 INFO:tasks.workunit.client.0.smithi018.stdout:1/520: creat d3/d9/da/dd/d31/d32/d13/d19/d2d/d4e/d7a/d82/fb5 x:0 0 0 2024-04-12T20:56:15.507 INFO:tasks.workunit.client.0.smithi018.stdout:8/810: creat db/d4e/f107 x:0 0 0 2024-04-12T20:56:15.515 INFO:tasks.workunit.client.0.smithi018.stdout:6/833: rmdir d5/df/d3f/dca/d11d 0 2024-04-12T20:56:15.516 INFO:tasks.workunit.client.0.smithi018.stdout:2/611: symlink d5/d8/d10/d19/d24/d39/da4/ld2 0 2024-04-12T20:56:15.521 INFO:tasks.workunit.client.0.smithi018.stdout:2/612: fsync d5/d12/f58 0 2024-04-12T20:56:15.523 INFO:tasks.workunit.client.0.smithi018.stdout:3/693: dread d2/d6/d30/d31/d38/f74 [0,4194304] 0 2024-04-12T20:56:15.525 INFO:tasks.workunit.client.0.smithi018.stdout:3/694: chown d2/d6/d34/d71 1830327 1 2024-04-12T20:56:15.528 INFO:tasks.workunit.client.0.smithi018.stdout:0/616: dwrite da/d15/d17/d21/d36/f4c [0,4194304] 0 2024-04-12T20:56:15.530 INFO:tasks.workunit.client.0.smithi018.stdout:2/613: write d5/d8/d10/d19/d24/d39/d9c/fb0 [1028740,67622] 0 2024-04-12T20:56:15.535 INFO:tasks.workunit.client.0.smithi018.stdout:7/649: write d1/db/d31/d3a/d42/d4c/f5b [971466,53007] 0 2024-04-12T20:56:15.542 INFO:tasks.workunit.client.0.smithi018.stdout:0/617: dread - da/d15/d17/d5c/d79/f86 zero size 2024-04-12T20:56:15.544 INFO:tasks.workunit.client.0.smithi018.stdout:1/521: dwrite d3/d9/da/dd/d31/d32/f41 [0,4194304] 0 2024-04-12T20:56:15.554 INFO:tasks.workunit.client.0.smithi018.stdout:3/695: link d2/d43/d61/d64/dbf/d54/fb7 d2/d4d/d88/fda 0 2024-04-12T20:56:15.563 INFO:tasks.workunit.client.0.smithi018.stdout:1/522: fsync d3/d9/da/d14/f38 0 2024-04-12T20:56:15.570 INFO:tasks.workunit.client.0.smithi018.stdout:2/614: dread d5/d7/d28/f5f [0,4194304] 0 2024-04-12T20:56:15.574 INFO:tasks.workunit.client.0.smithi018.stdout:2/615: readlink d5/d12/l1c 0 2024-04-12T20:56:15.578 INFO:tasks.workunit.client.0.smithi018.stdout:0/618: mknod da/d15/d17/d21/d36/d88/cca 0 2024-04-12T20:56:15.581 INFO:tasks.workunit.client.0.smithi018.stdout:3/696: unlink d2/f5d 0 2024-04-12T20:56:15.584 INFO:tasks.workunit.client.0.smithi018.stdout:0/619: chown da/d15/f8e 99653 1 2024-04-12T20:56:15.584 INFO:tasks.workunit.client.0.smithi018.stdout:8/811: dwrite db/f15 [0,4194304] 0 2024-04-12T20:56:15.586 INFO:tasks.workunit.client.0.smithi018.stdout:2/616: symlink d5/d8/d10/d19/daf/ld3 0 2024-04-12T20:56:15.589 INFO:tasks.workunit.client.0.smithi018.stdout:0/620: mkdir da/d15/d17/d21/d27/dcb 0 2024-04-12T20:56:15.591 INFO:tasks.workunit.client.0.smithi018.stdout:6/834: dwrite d5/d15/dd1/fe4 [0,4194304] 0 2024-04-12T20:56:15.592 INFO:tasks.workunit.client.0.smithi018.stdout:2/617: truncate d5/d7/f5c 589994 0 2024-04-12T20:56:15.599 INFO:tasks.workunit.client.0.smithi018.stdout:6/835: mkdir d5/d6/d1b/d3b/d35/d13d 0 2024-04-12T20:56:15.601 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:56:15 smithi018 ceph-mon[28799]: pgmap v40: 65 pgs: 65 active+clean; 1.2 GiB data, 5.4 GiB used, 531 GiB / 536 GiB avail; 42 MiB/s rd, 110 MiB/s wr, 305 op/s 2024-04-12T20:56:15.601 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:56:15 smithi018 ceph-mon[28799]: from='mgr.24359 172.21.15.186:0/1315906828' entity='mgr.smithi186.kqcfxk' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T20:56:15.601 INFO:tasks.workunit.client.0.smithi018.stdout:8/812: symlink db/d4e/d56/d59/d76/da5/d45/da9/ddb/l108 0 2024-04-12T20:56:15.606 INFO:tasks.workunit.client.0.smithi018.stdout:6/836: write d5/d6/d1b/d3b/f91 [95191,48655] 0 2024-04-12T20:56:15.615 INFO:tasks.workunit.client.0.smithi018.stdout:8/813: fsync db/d4e/d56/d59/d76/da5/d45/f79 0 2024-04-12T20:56:15.622 INFO:tasks.workunit.client.0.smithi018.stdout:2/618: getdents d5/d8/d10/d19/d24/d39/da4 0 2024-04-12T20:56:15.624 INFO:tasks.workunit.client.0.smithi018.stdout:7/650: dwrite d1/db/d31/d45/d8c/d9d/d70/d96/f4 [0,4194304] 0 2024-04-12T20:56:15.631 INFO:tasks.workunit.client.0.smithi018.stdout:2/619: rmdir d5/d8/d10/d19/d24 39 2024-04-12T20:56:15.638 INFO:tasks.workunit.client.0.smithi018.stdout:3/697: dread d2/d6/f1b [0,4194304] 0 2024-04-12T20:56:15.639 INFO:tasks.workunit.client.0.smithi018.stdout:1/523: dread d3/d9/da/dd/d31/d32/d13/d19/d2d/d4e/f4f [0,4194304] 0 2024-04-12T20:56:15.643 INFO:tasks.workunit.client.0.smithi018.stdout:8/814: rmdir db/d4e/d7f/df3 0 2024-04-12T20:56:15.644 INFO:tasks.workunit.client.0.smithi018.stdout:3/698: chown d2/d6/c5b 61082 1 2024-04-12T20:56:15.647 INFO:tasks.workunit.client.0.smithi018.stdout:0/621: rename da/d15/d17/d21/d27 to da/d15/d17/d71/dc1/dcc 0 2024-04-12T20:56:15.651 INFO:tasks.workunit.client.0.smithi018.stdout:7/651: link d1/db/c2e d1/db/d6f/dad/cd6 0 2024-04-12T20:56:15.653 INFO:tasks.workunit.client.0.smithi018.stdout:3/699: read d2/d43/d61/f63 [287008,106773] 0 2024-04-12T20:56:15.655 INFO:tasks.workunit.client.0.smithi018.stdout:6/837: rename d5/d6/l8 to d5/d15/d60/dbb/l13e 0 2024-04-12T20:56:15.658 INFO:tasks.workunit.client.0.smithi018.stdout:1/524: unlink d3/d9/da/dd/d31/d32/d13/d19/d2d/d4e/d58/d61/d71/f9f 0 2024-04-12T20:56:15.659 INFO:tasks.workunit.client.0.smithi018.stdout:7/652: read d1/db/d31/d3a/d42/dcd/fd2 [695067,129297] 0 2024-04-12T20:56:15.662 INFO:tasks.workunit.client.0.smithi018.stdout:3/700: write d2/d6/d30/d31/d38/d3a/f45 [444939,12369] 0 2024-04-12T20:56:15.674 INFO:tasks.workunit.client.0.smithi018.stdout:7/653: creat d1/db/d31/d45/d8c/d9d/d69/fd7 x:0 0 0 2024-04-12T20:56:15.674 INFO:tasks.workunit.client.0.smithi018.stdout:2/620: dwrite d5/d8/d10/d19/daf/fa2 [0,4194304] 0 2024-04-12T20:56:15.680 INFO:tasks.workunit.client.0.smithi018.stdout:3/701: stat d2/d83/cba 0 2024-04-12T20:56:15.681 INFO:tasks.workunit.client.0.smithi018.stdout:8/815: dread db/d4e/d56/d59/d76/da5/d45/d86/f9d [0,4194304] 0 2024-04-12T20:56:15.682 INFO:tasks.workunit.client.0.smithi018.stdout:0/622: link da/d15/d17/d21/d94/l9f da/d15/d17/d21/d36/da2/dc3/d3f/lcd 0 2024-04-12T20:56:15.688 INFO:tasks.workunit.client.0.smithi018.stdout:2/621: symlink d5/ld4 0 2024-04-12T20:56:15.688 INFO:tasks.workunit.client.0.smithi018.stdout:3/702: mkdir d2/d6/d34/da1/ddb 0 2024-04-12T20:56:15.689 INFO:tasks.workunit.client.0.smithi018.stdout:1/525: write d3/d9/da/dd/f3f [5033373,43947] 0 2024-04-12T20:56:15.692 INFO:tasks.workunit.client.0.smithi018.stdout:8/816: symlink db/d95/dc0/l109 0 2024-04-12T20:56:15.692 INFO:tasks.workunit.client.0.smithi018.stdout:7/654: getdents d1/db/d31/d3a/d42/d4c/d6c 0 2024-04-12T20:56:15.696 INFO:tasks.workunit.client.0.smithi018.stdout:2/622: symlink d5/d12/d54/dc0/ld5 0 2024-04-12T20:56:15.696 INFO:tasks.workunit.client.0.smithi018.stdout:8/817: dread - db/d4e/d7f/f89 zero size 2024-04-12T20:56:15.702 INFO:tasks.workunit.client.0.smithi018.stdout:8/818: chown db/d4e/d56/d59/d76/de0/l61 2 1 2024-04-12T20:56:15.703 INFO:tasks.workunit.client.0.smithi018.stdout:8/819: readlink l4 0 2024-04-12T20:56:15.703 INFO:tasks.workunit.client.0.smithi018.stdout:2/623: rmdir d5/d12/d54/dbc 39 2024-04-12T20:56:15.705 INFO:tasks.workunit.client.0.smithi018.stdout:0/623: truncate da/faa 1106392 0 2024-04-12T20:56:15.707 INFO:tasks.workunit.client.0.smithi018.stdout:7/655: getdents d1/db/d31/d45/d8c/d9d/d94 0 2024-04-12T20:56:15.708 INFO:tasks.workunit.client.0.smithi018.stdout:8/820: creat db/d4e/d56/d59/d76/da5/d45/d86/f10a x:0 0 0 2024-04-12T20:56:15.713 INFO:tasks.workunit.client.0.smithi018.stdout:7/656: chown d1/db/d31/d45/f9e 82626 1 2024-04-12T20:56:15.713 INFO:tasks.workunit.client.0.smithi018.stdout:0/624: mkdir da/d15/d17/d21/d36/dce 0 2024-04-12T20:56:15.716 INFO:tasks.workunit.client.0.smithi018.stdout:8/821: unlink db/d25/f52 0 2024-04-12T20:56:15.720 INFO:tasks.workunit.client.0.smithi018.stdout:0/625: mknod da/d15/d17/d71/ccf 0 2024-04-12T20:56:15.722 INFO:tasks.workunit.client.0.smithi018.stdout:7/657: creat d1/db/d31/d45/d8c/d9d/d70/fd8 x:0 0 0 2024-04-12T20:56:15.724 INFO:tasks.workunit.client.0.smithi018.stdout:8/822: dread - db/d4e/dab/ff6 zero size 2024-04-12T20:56:15.726 INFO:tasks.workunit.client.0.smithi018.stdout:0/626: mkdir da/d15/d4d/dd0 0 2024-04-12T20:56:15.729 INFO:tasks.workunit.client.0.smithi018.stdout:7/658: mknod d1/db/d31/d3a/d97/cd9 0 2024-04-12T20:56:15.730 INFO:tasks.workunit.client.0.smithi018.stdout:0/627: symlink da/d15/ld1 0 2024-04-12T20:56:15.731 INFO:tasks.workunit.client.0.smithi018.stdout:8/823: dread - db/d4e/d56/d59/d76/da5/d45/d86/da1/fe9 zero size 2024-04-12T20:56:15.731 INFO:tasks.workunit.client.0.smithi018.stdout:2/624: dwrite d5/d8/d10/d19/f7b [4194304,4194304] 0 2024-04-12T20:56:15.732 INFO:tasks.workunit.client.0.smithi018.stdout:1/526: dread d3/d9/da/dd/d31/d32/d13/f46 [0,4194304] 0 2024-04-12T20:56:15.734 INFO:tasks.workunit.client.0.smithi018.stdout:8/824: read - db/d4e/d73/fe8 zero size 2024-04-12T20:56:15.735 INFO:tasks.workunit.client.0.smithi018.stdout:0/628: mknod da/d15/d17/d21/d36/da2/cd2 0 2024-04-12T20:56:15.738 INFO:tasks.workunit.client.0.smithi018.stdout:1/527: mknod d3/d9/da/dd/d31/d32/d13/cb6 0 2024-04-12T20:56:15.745 INFO:tasks.workunit.client.0.smithi018.stdout:8/825: rmdir db 39 2024-04-12T20:56:15.750 INFO:tasks.workunit.client.0.smithi018.stdout:2/625: mknod d5/d8/d10/d19/d7c/cd6 0 2024-04-12T20:56:15.750 INFO:tasks.workunit.client.0.smithi018.stdout:0/629: mkdir da/d15/d17/d21/d36/da2/dc3/da9/dd3 0 2024-04-12T20:56:15.753 INFO:tasks.workunit.client.0.smithi018.stdout:2/626: rename d5/d12/lf to d5/d8/d10/d19/daf/d88/dcd/ld7 0 2024-04-12T20:56:15.758 INFO:tasks.workunit.client.0.smithi018.stdout:1/528: dread d3/d9/da/dd/d31/d32/d13/d19/d2d/d4e/f4f [0,4194304] 0 2024-04-12T20:56:15.761 INFO:tasks.workunit.client.0.smithi018.stdout:0/630: creat da/d15/d17/d21/d36/fd4 x:0 0 0 2024-04-12T20:56:15.763 INFO:tasks.workunit.client.0.smithi018.stdout:8/826: creat db/d4e/d56/d59/d76/f10b x:0 0 0 2024-04-12T20:56:15.764 INFO:tasks.workunit.client.0.smithi018.stdout:7/659: write d1/db/d31/d3a/d42/dcd/fd2 [955316,90801] 0 2024-04-12T20:56:15.767 INFO:tasks.workunit.client.0.smithi018.stdout:0/631: readlink da/d15/d17/d21/d36/da2/dc3/d3f/lbd 0 2024-04-12T20:56:15.769 INFO:tasks.workunit.client.0.smithi018.stdout:8/827: rmdir db/d4e/d56/d59/d76/da5/d45/d86/da1 39 2024-04-12T20:56:15.769 INFO:tasks.workunit.client.0.smithi018.stdout:0/632: dread - da/d15/d17/d21/d36/d47/d81/fc4 zero size 2024-04-12T20:56:15.770 INFO:tasks.workunit.client.0.smithi018.stdout:7/660: mknod d1/db/d31/d45/d8c/d9d/d70/cda 0 2024-04-12T20:56:15.771 INFO:tasks.workunit.client.0.smithi018.stdout:2/627: dread d5/d12/d54/f53 [0,4194304] 0 2024-04-12T20:56:15.773 INFO:tasks.workunit.client.0.smithi018.stdout:8/828: truncate db/d4e/f6a 590018 0 2024-04-12T20:56:15.773 INFO:tasks.workunit.client.0.smithi018.stdout:0/633: chown da/d15/d17/d21/d36/d47/d81/fc4 241364541 1 2024-04-12T20:56:15.777 INFO:tasks.workunit.client.0.smithi018.stdout:7/661: mknod d1/db/d31/d45/d8c/d9d/d94/cdb 0 2024-04-12T20:56:15.783 INFO:tasks.workunit.client.0.smithi018.stdout:8/829: link db/d4e/d56/d59/d76/de0/f2f db/d4e/d9a/daa/f10c 0 2024-04-12T20:56:15.784 INFO:tasks.workunit.client.0.smithi018.stdout:2/628: write d5/d8/f82 [567176,124716] 0 2024-04-12T20:56:15.787 INFO:tasks.workunit.client.0.smithi018.stdout:2/629: chown d5/fc6 1026 1 2024-04-12T20:56:15.788 INFO:tasks.workunit.client.0.smithi018.stdout:7/662: rmdir d1/db/d31/d3a/d42/d4c/dbc/dd5 0 2024-04-12T20:56:15.788 INFO:tasks.workunit.client.0.smithi018.stdout:8/830: fdatasync db/d25/f34 0 2024-04-12T20:56:15.790 INFO:tasks.workunit.client.0.smithi018.stdout:2/630: chown d5/d8/d25 29 1 2024-04-12T20:56:15.794 INFO:tasks.workunit.client.0.smithi018.stdout:0/634: dwrite da/d15/d4d/f85 [0,4194304] 0 2024-04-12T20:56:15.804 INFO:tasks.workunit.client.0.smithi018.stdout:2/631: creat d5/d8/d10/d43/fd8 x:0 0 0 2024-04-12T20:56:15.806 INFO:tasks.workunit.client.0.smithi018.stdout:0/635: truncate da/d15/d17/d21/d94/fc7 478302 0 2024-04-12T20:56:15.807 INFO:tasks.workunit.client.0.smithi018.stdout:8/831: rename db/d4e/f84 to db/d4e/d56/d59/d76/da5/d45/da9/f10d 0 2024-04-12T20:56:15.812 INFO:tasks.workunit.client.0.smithi018.stdout:1/529: dread d3/d9/da/dd/d31/d32/d13/d19/d2d/d4e/d7a/fa4 [0,4194304] 0 2024-04-12T20:56:15.812 INFO:tasks.workunit.client.0.smithi018.stdout:2/632: symlink d5/d8/d33/ld9 0 2024-04-12T20:56:15.813 INFO:tasks.workunit.client.0.smithi018.stdout:1/530: fsync d3/d9/faf 0 2024-04-12T20:56:15.818 INFO:tasks.workunit.client.0.smithi018.stdout:0/636: rename da/d15/d17/d5c/f82 to da/d15/d17/d71/dc1/dcc/dcb/fd5 0 2024-04-12T20:56:15.818 INFO:tasks.workunit.client.0.smithi018.stdout:2/633: rename d5 to d5/d8/d10/d19/daf/d88/dda 22 2024-04-12T20:56:15.819 INFO:tasks.workunit.client.0.smithi018.stdout:7/663: link d1/db/d31/d45/d8c/d9d/d69/f7e d1/db/d31/d45/fdc 0 2024-04-12T20:56:15.819 INFO:tasks.workunit.client.0.smithi018.stdout:8/832: mknod db/d4e/d73/df7/c10e 0 2024-04-12T20:56:15.821 INFO:tasks.workunit.client.0.smithi018.stdout:2/634: write d5/d8/d10/d19/daf/fa2 [2539629,115776] 0 2024-04-12T20:56:15.824 INFO:tasks.workunit.client.0.smithi018.stdout:0/637: write da/d15/d17/d21/d36/da2/dc3/d3f/f6c [2399064,117702] 0 2024-04-12T20:56:15.826 INFO:tasks.workunit.client.0.smithi018.stdout:2/635: write d5/d7/d31/d62/fb8 [6162925,25487] 0 2024-04-12T20:56:15.828 INFO:tasks.workunit.client.0.smithi018.stdout:1/531: mknod d3/d9/da/dd/d31/d32/d13/d19/cb7 0 2024-04-12T20:56:15.830 INFO:tasks.workunit.client.0.smithi018.stdout:8/833: mkdir db/d4e/d56/d59/d76/da5/d45/d86/dbb/d10f 0 2024-04-12T20:56:15.835 INFO:tasks.workunit.client.0.smithi018.stdout:0/638: mkdir da/d15/d17/d5c/d79/dd6 0 2024-04-12T20:56:15.838 INFO:tasks.workunit.client.0.smithi018.stdout:1/532: unlink d3/d9/da/dd/d31/d44/f5d 0 2024-04-12T20:56:15.839 INFO:tasks.workunit.client.0.smithi018.stdout:8/834: mkdir db/d72/de6/d110 0 2024-04-12T20:56:15.840 INFO:tasks.workunit.client.0.smithi018.stdout:8/835: readlink db/d3b/l99 0 2024-04-12T20:56:15.840 INFO:tasks.workunit.client.0.smithi018.stdout:4/690: sync 2024-04-12T20:56:15.841 INFO:tasks.workunit.client.0.smithi018.stdout:5/738: sync 2024-04-12T20:56:15.846 INFO:tasks.workunit.client.0.smithi018.stdout:0/639: mknod da/d15/d17/d21/d36/dce/cd7 0 2024-04-12T20:56:15.850 INFO:tasks.workunit.client.0.smithi018.stdout:0/640: chown da/d15/d17/d21/d36/da2/dc3/d3f/f6c 29935 1 2024-04-12T20:56:15.853 INFO:tasks.workunit.client.0.smithi018.stdout:4/691: mkdir d3/d5d/d60/dc5/dcc/ddf 0 2024-04-12T20:56:15.854 INFO:tasks.workunit.client.0.smithi018.stdout:0/641: read da/d15/d17/d21/d36/da2/dc3/f5e [336007,15145] 0 2024-04-12T20:56:15.854 INFO:tasks.workunit.client.0.smithi018.stdout:8/836: dread db/d4e/d56/d59/f82 [0,4194304] 0 2024-04-12T20:56:15.859 INFO:tasks.workunit.client.0.smithi018.stdout:2/636: dread d5/d7/d31/f64 [0,4194304] 0 2024-04-12T20:56:15.865 INFO:tasks.workunit.client.0.smithi018.stdout:8/837: creat db/d4e/d73/df7/f111 x:0 0 0 2024-04-12T20:56:15.866 INFO:tasks.workunit.client.0.smithi018.stdout:7/664: write d1/d6/f52 [2424966,4456] 0 2024-04-12T20:56:15.869 INFO:tasks.workunit.client.0.smithi018.stdout:2/637: rename d5/l42 to d5/d8/d25/ldb 0 2024-04-12T20:56:15.874 INFO:tasks.workunit.client.0.smithi018.stdout:7/665: fsync d1/db/d31/d45/f9e 0 2024-04-12T20:56:15.876 INFO:tasks.workunit.client.0.smithi018.stdout:0/642: dwrite da/d15/d17/d21/d36/f63 [0,4194304] 0 2024-04-12T20:56:15.880 INFO:tasks.workunit.client.0.smithi018.stdout:5/739: dread d0/d2/d33/d10/f17 [0,4194304] 0 2024-04-12T20:56:15.885 INFO:tasks.workunit.client.0.smithi018.stdout:5/740: stat d0/d2/d33/d46/d1e/d9a/dec/d8f 0 2024-04-12T20:56:15.887 INFO:tasks.workunit.client.0.smithi018.stdout:8/838: dread db/d4e/d56/d59/fd3 [0,4194304] 0 2024-04-12T20:56:15.888 INFO:tasks.workunit.client.0.smithi018.stdout:1/533: dread d3/d9/d11/f70 [0,4194304] 0 2024-04-12T20:56:15.894 INFO:tasks.workunit.client.0.smithi018.stdout:7/666: dread d1/db/f9f [0,4194304] 0 2024-04-12T20:56:15.903 INFO:tasks.workunit.client.0.smithi018.stdout:7/667: rmdir d1/d6/dc9 39 2024-04-12T20:56:15.904 INFO:tasks.workunit.client.0.smithi018.stdout:4/692: dwrite d3/d5d/d60/dc5/d43/fb1 [0,4194304] 0 2024-04-12T20:56:15.906 INFO:tasks.workunit.client.0.smithi018.stdout:2/638: dread d5/d8/d10/d19/f7b [0,4194304] 0 2024-04-12T20:56:15.908 INFO:tasks.workunit.client.0.smithi018.stdout:0/643: dwrite da/d15/f41 [0,4194304] 0 2024-04-12T20:56:15.910 INFO:tasks.workunit.client.0.smithi018.stdout:2/639: write d5/d8/d10/f78 [1896654,66727] 0 2024-04-12T20:56:15.911 INFO:tasks.workunit.client.0.smithi018.stdout:8/839: write db/d4e/d56/d59/d76/de0/f3f [1081992,98973] 0 2024-04-12T20:56:15.912 INFO:tasks.workunit.client.0.smithi018.stdout:7/668: chown d1/db/d31/d45/d8c/d9d/d70/fb8 82495 1 2024-04-12T20:56:15.915 INFO:tasks.workunit.client.0.smithi018.stdout:0/644: fsync da/d15/d17/d21/d36/d47/d81/fa7 0 2024-04-12T20:56:15.917 INFO:tasks.workunit.client.0.smithi018.stdout:5/741: dwrite d0/d2/d33/d46/f3a [0,4194304] 0 2024-04-12T20:56:15.917 INFO:tasks.workunit.client.0.smithi018.stdout:4/693: chown d3/db/f24 419953117 1 2024-04-12T20:56:15.919 INFO:tasks.workunit.client.0.smithi018.stdout:4/694: chown d3/d32/d66/f93 14156103 1 2024-04-12T20:56:15.927 INFO:tasks.workunit.client.0.smithi018.stdout:7/669: chown d1/db/d31/d45/d8c/d9d/d70/d96/ca 150 1 2024-04-12T20:56:15.934 INFO:tasks.workunit.client.0.smithi018.stdout:2/640: rename d5/d7/d79/l84 to d5/d8/d10/d19/d24/d39/ldc 0 2024-04-12T20:56:15.940 INFO:tasks.workunit.client.0.smithi018.stdout:5/742: mkdir d0/d2/d33/d46/d1e/d9a/dec/dbf/da3/df9 0 2024-04-12T20:56:15.941 INFO:tasks.workunit.client.0.smithi018.stdout:1/534: dread d3/f7 [4194304,4194304] 0 2024-04-12T20:56:15.944 INFO:tasks.workunit.client.0.smithi018.stdout:8/840: dwrite db/fff [0,4194304] 0 2024-04-12T20:56:15.946 INFO:tasks.workunit.client.0.smithi018.stdout:5/743: stat d0/d2/d11/d37/dc2/cf0 0 2024-04-12T20:56:15.950 INFO:tasks.workunit.client.0.smithi018.stdout:5/744: chown d0/d2/d33/d46/d1e/d2e/d7f 190873 1 2024-04-12T20:56:15.952 INFO:tasks.workunit.client.0.smithi018.stdout:1/535: mkdir d3/db8 0 2024-04-12T20:56:15.955 INFO:tasks.workunit.client.0.smithi018.stdout:8/841: stat db/d4e/d56/d59/d76/de0/d27/fa0 0 2024-04-12T20:56:15.960 INFO:tasks.workunit.client.0.smithi018.stdout:0/645: dread da/d15/d17/d21/d36/da2/dc3/d3f/f5b [4194304,4194304] 0 2024-04-12T20:56:15.961 INFO:tasks.workunit.client.0.smithi018.stdout:8/842: mkdir db/d72/de6/d112 0 2024-04-12T20:56:15.963 INFO:tasks.workunit.client.0.smithi018.stdout:2/641: dwrite d5/d8/d10/f48 [4194304,4194304] 0 2024-04-12T20:56:15.965 INFO:tasks.workunit.client.0.smithi018.stdout:2/642: chown d5/d7/d63 23 1 2024-04-12T20:56:15.968 INFO:tasks.workunit.client.0.smithi018.stdout:5/745: readlink d0/d2/d33/d46/d1e/d55/lf4 0 2024-04-12T20:56:15.972 INFO:tasks.workunit.client.0.smithi018.stdout:7/670: dread d1/db/d31/d3a/d42/f75 [0,4194304] 0 2024-04-12T20:56:15.975 INFO:tasks.workunit.client.0.smithi018.stdout:0/646: symlink da/d15/d17/d21/d36/d88/ld8 0 2024-04-12T20:56:15.975 INFO:tasks.workunit.client.0.smithi018.stdout:4/695: write d3/d32/d33/fa5 [149679,51184] 0 2024-04-12T20:56:15.977 INFO:tasks.workunit.client.0.smithi018.stdout:8/843: rename db/d25 to db/d4e/d56/d59/d76/da5/d45/d86/dbb/d113 0 2024-04-12T20:56:15.978 INFO:tasks.workunit.client.0.smithi018.stdout:0/647: chown da/d15/d17/d5c/f64 53 1 2024-04-12T20:56:15.983 INFO:tasks.workunit.client.0.smithi018.stdout:7/671: fdatasync d1/db/d31/d45/f9e 0 2024-04-12T20:56:15.987 INFO:tasks.workunit.client.0.smithi018.stdout:6/838: sync 2024-04-12T20:56:15.987 INFO:tasks.workunit.client.0.smithi018.stdout:9/783: sync 2024-04-12T20:56:15.987 INFO:tasks.workunit.client.0.smithi018.stdout:3/703: sync 2024-04-12T20:56:15.993 INFO:tasks.workunit.client.0.smithi018.stdout:5/746: rename d0/d2/d33/d46/d1e/d9a/dec to d0/d2/d33/d10/d60/dfa 0 2024-04-12T20:56:15.999 INFO:tasks.workunit.client.0.smithi018.stdout:0/648: mknod da/d15/d4d/d58/dac/cd9 0 2024-04-12T20:56:15.999 INFO:tasks.workunit.client.0.smithi018.stdout:2/643: link d5/c22 d5/d8/d10/d19/d24/d91/cdd 0 2024-04-12T20:56:16.003 INFO:tasks.workunit.client.0.smithi018.stdout:1/536: dwrite d3/d9/da/dd/d31/d32/f1b [0,4194304] 0 2024-04-12T20:56:16.003 INFO:tasks.workunit.client.0.smithi018.stdout:4/696: truncate d3/d5d/d60/dc5/d3d/d80/f8c 241411 0 2024-04-12T20:56:16.013 INFO:tasks.workunit.client.0.smithi018.stdout:9/784: creat d3/d6/d9/dc/d5a/d79/d7e/d9b/d21/dc3/dd8/fe5 x:0 0 0 2024-04-12T20:56:16.013 INFO:tasks.workunit.client.0.smithi018.stdout:7/672: read d1/d6/f60 [2685176,53713] 0 2024-04-12T20:56:16.016 INFO:tasks.workunit.client.0.smithi018.stdout:3/704: rename d2/d6/d30/d31/d38/d3a/d76/dad/cd2 to d2/d43/d61/d64/dbf/d70/cdc 0 2024-04-12T20:56:16.018 INFO:tasks.workunit.client.0.smithi018.stdout:7/673: fsync d1/d6/f52 0 2024-04-12T20:56:16.020 INFO:tasks.workunit.client.0.smithi018.stdout:2/644: creat d5/d8/d10/d19/d24/d39/da4/fde x:0 0 0 2024-04-12T20:56:16.020 INFO:tasks.workunit.client.0.smithi018.stdout:9/785: write d3/d6/d9/dc/d5a/d79/d7e/d9b/d21/d41/d78/f98 [3623241,121763] 0 2024-04-12T20:56:16.021 INFO:tasks.workunit.client.0.smithi018.stdout:0/649: rmdir da/d15/d17/d21/d36/d88 39 2024-04-12T20:56:16.025 INFO:tasks.workunit.client.0.smithi018.stdout:5/747: symlink d0/d2/d11/d37/dc2/d89/d8d/de9/lfb 0 2024-04-12T20:56:16.025 INFO:tasks.workunit.client.0.smithi018.stdout:2/645: chown d5/d8/d10/f78 54822 1 2024-04-12T20:56:16.028 INFO:tasks.workunit.client.0.smithi018.stdout:5/748: stat d0/d2/d11/d74/cdc 0 2024-04-12T20:56:16.028 INFO:tasks.workunit.client.0.smithi018.stdout:1/537: mkdir d3/d9/da/dd/d31/d32/d13/d19/d2d/d4e/db9 0 2024-04-12T20:56:16.030 INFO:tasks.workunit.client.0.smithi018.stdout:2/646: fsync d5/d8/d25/fba 0 2024-04-12T20:56:16.034 INFO:tasks.workunit.client.0.smithi018.stdout:3/705: chown d2/d6/d30/d31/cc8 7605 1 2024-04-12T20:56:16.035 INFO:tasks.workunit.client.0.smithi018.stdout:2/647: chown d5/d12/d54/dc0/ld5 57492 1 2024-04-12T20:56:16.040 INFO:tasks.workunit.client.0.smithi018.stdout:5/749: unlink d0/d2/d33/d10/d60/dfa/f53 0 2024-04-12T20:56:16.043 INFO:tasks.workunit.client.0.smithi018.stdout:9/786: rename d3/d6/d9/dc/d5a/d79/d7e/d9b/d21/dc3/dd8/c5c to d3/d6/d9/dc/d5a/d79/d7e/d9b/d21/dc2/ce6 0 2024-04-12T20:56:16.049 INFO:tasks.workunit.client.0.smithi018.stdout:8/844: dread db/d4e/d56/d59/d76/da5/f5e [0,4194304] 0 2024-04-12T20:56:16.062 INFO:tasks.workunit.client.0.smithi018.stdout:4/697: getdents d3/d32 0 2024-04-12T20:56:16.064 INFO:tasks.workunit.client.0.smithi018.stdout:5/750: mknod d0/d2/d33/d10/d4e/ded/cfc 0 2024-04-12T20:56:16.065 INFO:tasks.workunit.client.0.smithi018.stdout:6/839: dwrite d5/df/d3f/f44 [4194304,4194304] 0 2024-04-12T20:56:16.069 INFO:tasks.workunit.client.0.smithi018.stdout:9/787: write d3/d6/d9/dc/d5a/d79/d7e/d9b/d21/dc3/dd8/d5b/d6b/f92 [865072,114523] 0 2024-04-12T20:56:16.072 INFO:tasks.workunit.client.0.smithi018.stdout:3/706: creat d2/d43/d61/d64/dbf/fdd x:0 0 0 2024-04-12T20:56:16.076 INFO:tasks.workunit.client.0.smithi018.stdout:2/648: dwrite d5/d8/d10/d43/fd8 [0,4194304] 0 2024-04-12T20:56:16.077 INFO:tasks.workunit.client.0.smithi018.stdout:0/650: dwrite da/d15/d17/d21/d36/fd4 [0,4194304] 0 2024-04-12T20:56:16.078 INFO:tasks.workunit.client.0.smithi018.stdout:3/707: chown d2/d43/d61/d64/dbf/d70 2493 1 2024-04-12T20:56:16.079 INFO:tasks.workunit.client.0.smithi018.stdout:8/845: rename db/d4e/f107 to db/d72/f114 0 2024-04-12T20:56:16.085 INFO:tasks.workunit.client.0.smithi018.stdout:5/751: rmdir d0/d2/d33/d46/d1e/d9a 39 2024-04-12T20:56:16.086 INFO:tasks.workunit.client.0.smithi018.stdout:1/538: dread d3/d9/da/f5f [0,4194304] 0 2024-04-12T20:56:16.086 INFO:tasks.workunit.client.0.smithi018.stdout:7/674: dwrite d1/db/d31/d30/dc6/fc8 [0,4194304] 0 2024-04-12T20:56:16.087 INFO:tasks.workunit.client.0.smithi018.stdout:3/708: truncate d2/d6/d34/fae 1670130 0 2024-04-12T20:56:16.087 INFO:tasks.workunit.client.0.smithi018.stdout:4/698: symlink d3/d5d/d60/dc5/dcc/le0 0 2024-04-12T20:56:16.090 INFO:tasks.workunit.client.0.smithi018.stdout:2/649: chown d5/d8/d10/d19/daf/d88/dcd/ld7 170264 1 2024-04-12T20:56:16.090 INFO:tasks.workunit.client.0.smithi018.stdout:0/651: creat da/d15/d4d/fda x:0 0 0 2024-04-12T20:56:16.100 INFO:tasks.workunit.client.0.smithi018.stdout:6/840: mkdir d5/d6/d1b/d3b/d6a/d116/d13f 0 2024-04-12T20:56:16.102 INFO:tasks.workunit.client.0.smithi018.stdout:5/752: creat d0/d2/d33/d10/d60/dfa/dbf/da3/ffd x:0 0 0 2024-04-12T20:56:16.105 INFO:tasks.workunit.client.0.smithi018.stdout:9/788: mkdir d3/d6/d9/dc/d5a/d79/d7e/d9b/d21/dc3/dd8/d5b/d6b/da7/de7 0 2024-04-12T20:56:16.107 INFO:tasks.workunit.client.0.smithi018.stdout:2/650: write d5/d7/d28/f5f [5901598,51397] 0 2024-04-12T20:56:16.109 INFO:tasks.workunit.client.0.smithi018.stdout:7/675: mkdir d1/db/d31/d45/d8c/d9d/d70/ddd 0 2024-04-12T20:56:16.111 INFO:tasks.workunit.client.0.smithi018.stdout:3/709: creat d2/d6/d30/d31/d38/d3a/d76/dad/dc1/fde x:0 0 0 2024-04-12T20:56:16.111 INFO:tasks.workunit.client.0.smithi018.stdout:6/841: chown d5/d15/d60/f65 166402 1 2024-04-12T20:56:16.111 INFO:tasks.workunit.client.0.smithi018.stdout:0/652: mknod da/d15/d17/d21/d36/d47/d81/cdb 0 2024-04-12T20:56:16.113 INFO:tasks.workunit.client.0.smithi018.stdout:2/651: fdatasync d5/d7/d31/f4f 0 2024-04-12T20:56:16.119 INFO:tasks.workunit.client.0.smithi018.stdout:0/653: fsync da/d15/f39 0 2024-04-12T20:56:16.121 INFO:tasks.workunit.client.0.smithi018.stdout:9/789: fsync d3/d6/d9/dc/d5a/d79/d7e/d9b/f34 0 2024-04-12T20:56:16.123 INFO:tasks.workunit.client.0.smithi018.stdout:4/699: dwrite d3/fa8 [0,4194304] 0 2024-04-12T20:56:16.124 INFO:tasks.workunit.client.0.smithi018.stdout:1/539: mknod d3/d9/da/cba 0 2024-04-12T20:56:16.126 INFO:tasks.workunit.client.0.smithi018.stdout:0/654: truncate da/f25 5950851 0 2024-04-12T20:56:16.127 INFO:tasks.workunit.client.0.smithi018.stdout:9/790: fdatasync d3/d6/d9/dc/d5a/fae 0 2024-04-12T20:56:16.132 INFO:tasks.workunit.client.0.smithi018.stdout:5/753: truncate d0/d2/d33/d10/f39 3163453 0 2024-04-12T20:56:16.135 INFO:tasks.workunit.client.0.smithi018.stdout:5/754: fsync d0/d5b/d84/f31 0 2024-04-12T20:56:16.135 INFO:tasks.workunit.client.0.smithi018.stdout:0/655: chown da/d15/d17/d21/d36/d47/f5d 485095794 1 2024-04-12T20:56:16.142 INFO:tasks.workunit.client.0.smithi018.stdout:6/842: truncate d5/d15/d60/fbc 996571 0 2024-04-12T20:56:16.146 INFO:tasks.workunit.client.0.smithi018.stdout:9/791: rename d3/d6/d43/f7f to d3/dc8/fe8 0 2024-04-12T20:56:16.146 INFO:tasks.workunit.client.0.smithi018.stdout:2/652: mknod d5/cdf 0 2024-04-12T20:56:16.146 INFO:tasks.workunit.client.0.smithi018.stdout:5/755: write d0/d2/f1a [4643358,113916] 0 2024-04-12T20:56:16.148 INFO:tasks.workunit.client.0.smithi018.stdout:0/656: mknod da/d15/d17/d71/dc1/cdc 0 2024-04-12T20:56:16.152 INFO:tasks.workunit.client.0.smithi018.stdout:6/843: truncate d5/df/d19/d27/fa2 1076300 0 2024-04-12T20:56:16.153 INFO:tasks.workunit.client.0.smithi018.stdout:4/700: creat d3/d5d/d60/dc5/d43/d4b/ddc/fe1 x:0 0 0 2024-04-12T20:56:16.159 INFO:tasks.workunit.client.0.smithi018.stdout:4/701: dread - d3/d5d/d60/dc5/d3d/f57 zero size 2024-04-12T20:56:16.164 INFO:tasks.workunit.client.0.smithi018.stdout:5/756: creat d0/d2/d33/d10/d60/dfa/dbf/da3/df9/ffe x:0 0 0 2024-04-12T20:56:16.217 INFO:tasks.workunit.client.0.smithi018.stdout:8/846: dread db/d4e/d56/d59/d76/de0/d27/f54 [0,4194304] 0 2024-04-12T20:56:16.217 INFO:tasks.workunit.client.0.smithi018.stdout:2/653: write d5/d7/d31/d62/f66 [1919303,50630] 0 2024-04-12T20:56:16.218 INFO:tasks.workunit.client.0.smithi018.stdout:4/702: creat d3/d5d/d60/dc5/d3d/da3/fe2 x:0 0 0 2024-04-12T20:56:16.218 INFO:tasks.workunit.client.0.smithi018.stdout:5/757: mknod d0/d2/d11/d37/dc2/d89/d8d/cff 0 2024-04-12T20:56:16.218 INFO:tasks.workunit.client.0.smithi018.stdout:8/847: rmdir db/d4e/d9a/daa/dcd 39 2024-04-12T20:56:16.218 INFO:tasks.workunit.client.0.smithi018.stdout:7/676: dwrite d1/db/d31/d45/d8c/d9d/d69/f89 [0,4194304] 0 2024-04-12T20:56:16.218 INFO:tasks.workunit.client.0.smithi018.stdout:5/758: readlink d0/d2/d33/d10/d60/dfa/dbf/da3/dae/le7 0 2024-04-12T20:56:16.218 INFO:tasks.workunit.client.0.smithi018.stdout:4/703: mkdir d3/db/de3 0 2024-04-12T20:56:16.218 INFO:tasks.workunit.client.0.smithi018.stdout:2/654: creat d5/d7/fe0 x:0 0 0 2024-04-12T20:56:16.218 INFO:tasks.workunit.client.0.smithi018.stdout:9/792: write d3/d6/d9/f82 [5193238,27776] 0 2024-04-12T20:56:16.218 INFO:tasks.workunit.client.0.smithi018.stdout:8/848: fsync db/d4e/d56/d59/d76/de0/d27/f50 0 2024-04-12T20:56:16.218 INFO:tasks.workunit.client.0.smithi018.stdout:7/677: write d1/db/d31/f63 [1148704,112457] 0 2024-04-12T20:56:16.218 INFO:tasks.workunit.client.0.smithi018.stdout:0/657: dwrite da/faa [0,4194304] 0 2024-04-12T20:56:16.218 INFO:tasks.workunit.client.0.smithi018.stdout:4/704: dread - d3/db/f85 zero size 2024-04-12T20:56:16.218 INFO:tasks.workunit.client.0.smithi018.stdout:1/540: dread d3/d9/da/dd/d31/d32/d33/f6d [0,4194304] 0 2024-04-12T20:56:16.218 INFO:tasks.workunit.client.0.smithi018.stdout:5/759: symlink d0/d2/d33/d46/d35/d4d/de1/l100 0 2024-04-12T20:56:16.218 INFO:tasks.workunit.client.0.smithi018.stdout:6/844: dwrite d5/d6/d1b/d3b/d35/d4f/fa7 [0,4194304] 0 2024-04-12T20:56:16.219 INFO:tasks.workunit.client.0.smithi018.stdout:0/658: write da/d15/d17/d21/d36/d47/d81/f8d [878833,68733] 0 2024-04-12T20:56:16.223 INFO:tasks.workunit.client.0.smithi018.stdout:6/845: dread - d5/df/d19/d2b/f123 zero size 2024-04-12T20:56:16.226 INFO:tasks.workunit.client.0.smithi018.stdout:2/655: dread d5/d12/d54/d3e/faa [0,4194304] 0 2024-04-12T20:56:16.226 INFO:tasks.workunit.client.0.smithi018.stdout:0/659: read da/fc [1335344,1884] 0 2024-04-12T20:56:16.231 INFO:tasks.workunit.client.0.smithi018.stdout:8/849: creat db/d4e/d56/d59/d76/da5/d45/da9/ddb/f115 x:0 0 0 2024-04-12T20:56:16.231 INFO:tasks.workunit.client.0.smithi018.stdout:7/678: dwrite d1/db/d31/d3a/fa5 [0,4194304] 0 2024-04-12T20:56:16.236 INFO:tasks.workunit.client.0.smithi018.stdout:7/679: write d1/d6/fb4 [4148741,10485] 0 2024-04-12T20:56:16.242 INFO:tasks.workunit.client.0.smithi018.stdout:4/705: mkdir d3/d5d/d60/dc5/d3d/de4 0 2024-04-12T20:56:16.245 INFO:tasks.workunit.client.0.smithi018.stdout:8/850: symlink db/d4e/d73/l116 0 2024-04-12T20:56:16.248 INFO:tasks.workunit.client.0.smithi018.stdout:0/660: creat da/d15/d17/d21/d36/da2/dc3/d3f/fdd x:0 0 0 2024-04-12T20:56:16.249 INFO:tasks.workunit.client.0.smithi018.stdout:7/680: rename d1/db/d31/d3a/f83 to d1/db/d31/d3a/d42/d4c/dbc/fde 0 2024-04-12T20:56:16.250 INFO:tasks.workunit.client.0.smithi018.stdout:4/706: mknod d3/d32/d66/ce5 0 2024-04-12T20:56:16.257 INFO:tasks.workunit.client.0.smithi018.stdout:2/656: write d5/d8/d33/f9a [744167,81842] 0 2024-04-12T20:56:16.258 INFO:tasks.workunit.client.0.smithi018.stdout:1/541: truncate d3/f4 3648044 0 2024-04-12T20:56:16.261 INFO:tasks.workunit.client.0.smithi018.stdout:5/760: write d0/d2/d33/d10/f39 [1946274,76511] 0 2024-04-12T20:56:16.263 INFO:tasks.workunit.client.0.smithi018.stdout:8/851: getdents db/d4e/d56 0 2024-04-12T20:56:16.264 INFO:tasks.workunit.client.0.smithi018.stdout:2/657: creat d5/d8/d33/fe1 x:0 0 0 2024-04-12T20:56:16.268 INFO:tasks.workunit.client.0.smithi018.stdout:4/707: rename d3/d5d/d60/d64/d79 to d3/d5d/d60/dc5/d43/d4b/de6 0 2024-04-12T20:56:16.270 INFO:tasks.workunit.client.0.smithi018.stdout:5/761: mknod d0/d2/d33/d46/d35/d4d/c101 0 2024-04-12T20:56:16.274 INFO:tasks.workunit.client.0.smithi018.stdout:8/852: mkdir db/d95/dd6/d117 0 2024-04-12T20:56:16.276 INFO:tasks.workunit.client.0.smithi018.stdout:2/658: unlink d5/d8/d10/d19/d24/d39/da4/ld2 0 2024-04-12T20:56:16.277 INFO:tasks.workunit.client.0.smithi018.stdout:4/708: mkdir d3/d5d/d60/dc5/d53/de7 0 2024-04-12T20:56:16.284 INFO:tasks.workunit.client.0.smithi018.stdout:5/762: dread d0/d2/d33/d46/d1e/d55/f82 [0,4194304] 0 2024-04-12T20:56:16.287 INFO:tasks.workunit.client.0.smithi018.stdout:1/542: write d3/d9/da/dd/d31/d32/d13/d19/d2d/d4e/f5b [579927,4959] 0 2024-04-12T20:56:16.290 INFO:tasks.workunit.client.0.smithi018.stdout:9/793: dread d3/d6/d9/dc/d5a/d79/d7e/f94 [0,4194304] 0 2024-04-12T20:56:16.296 INFO:tasks.workunit.client.0.smithi018.stdout:2/659: rename d5/d8/d10/d19 to d5/d8/d10/d43/de2 0 2024-04-12T20:56:16.296 INFO:tasks.workunit.client.0.smithi018.stdout:0/661: dread da/d15/d17/d21/f5f [0,4194304] 0 2024-04-12T20:56:16.298 INFO:tasks.workunit.client.0.smithi018.stdout:9/794: write d3/d6/d9/fdd [632904,13557] 0 2024-04-12T20:56:16.298 INFO:tasks.workunit.client.0.smithi018.stdout:4/709: mkdir d3/d5d/d60/dc5/de8 0 2024-04-12T20:56:16.300 INFO:tasks.workunit.client.0.smithi018.stdout:1/543: truncate d3/d9/da/dd/f4a 1551371 0 2024-04-12T20:56:16.308 INFO:tasks.workunit.client.0.smithi018.stdout:9/795: chown d3/d6/d9/dc/d5a/d79/d7e/d9b/d21/c63 0 1 2024-04-12T20:56:16.311 INFO:tasks.workunit.client.0.smithi018.stdout:4/710: symlink d3/d32/d33/db6/le9 0 2024-04-12T20:56:16.315 INFO:tasks.workunit.client.0.smithi018.stdout:5/763: truncate d0/d2/d33/d46/f3a 1850330 0 2024-04-12T20:56:16.315 INFO:tasks.workunit.client.0.smithi018.stdout:1/544: mknod d3/d9/da/dd/d31/d32/d33/d84/cbb 0 2024-04-12T20:56:16.322 INFO:tasks.workunit.client.0.smithi018.stdout:4/711: read d3/d5d/d60/d64/d4f/f95 [3231905,109568] 0 2024-04-12T20:56:16.322 INFO:tasks.workunit.client.0.smithi018.stdout:8/853: dwrite db/d4e/d56/d59/d76/da5/d45/d86/dbb/d113/f4c [0,4194304] 0 2024-04-12T20:56:16.324 INFO:tasks.workunit.client.0.smithi018.stdout:1/545: creat d3/d9/da/dd/d26/fbc x:0 0 0 2024-04-12T20:56:16.335 INFO:tasks.workunit.client.0.smithi018.stdout:7/681: dread d1/d6/f60 [0,4194304] 0 2024-04-12T20:56:16.338 INFO:tasks.workunit.client.0.smithi018.stdout:2/660: truncate d5/d8/d10/d43/de2/d7c/f9f 3058817 0 2024-04-12T20:56:16.338 INFO:tasks.workunit.client.0.smithi018.stdout:0/662: dwrite da/d15/d4d/d6b/f6d [0,4194304] 0 2024-04-12T20:56:16.338 INFO:tasks.workunit.client.0.smithi018.stdout:8/854: read - db/d4e/d56/d59/d76/da5/d45/d86/da1/fe9 zero size 2024-04-12T20:56:16.343 INFO:tasks.workunit.client.0.smithi018.stdout:1/546: chown d3/d9/da/dd/d31/d32/d13/d19/d2d/d4e/d58/c5a 1654725 1 2024-04-12T20:56:16.348 INFO:tasks.workunit.client.0.smithi018.stdout:2/661: mknod d5/d8/d33/ce3 0 2024-04-12T20:56:16.352 INFO:tasks.workunit.client.0.smithi018.stdout:2/662: chown d5/d8/f1e 263810096 1 2024-04-12T20:56:16.356 INFO:tasks.workunit.client.0.smithi018.stdout:0/663: symlink da/d15/d17/d71/dc1/dcc/lde 0 2024-04-12T20:56:16.362 INFO:tasks.workunit.client.0.smithi018.stdout:7/682: getdents d1/db/d31/d3a/d42/dcd 0 2024-04-12T20:56:16.364 INFO:tasks.workunit.client.0.smithi018.stdout:5/764: truncate d0/d2/d33/d10/d60/dfa/dbf/da3/dae/fcb 3854573 0 2024-04-12T20:56:16.364 INFO:tasks.workunit.client.0.smithi018.stdout:2/663: read d5/d7/d79/f7e [288590,88114] 0 2024-04-12T20:56:16.368 INFO:tasks.workunit.client.0.smithi018.stdout:5/765: fsync d0/d2/d11/d37/dc2/d89/fac 0 2024-04-12T20:56:16.368 INFO:tasks.workunit.client.0.smithi018.stdout:1/547: getdents d3/d9/da/dd/d31/d32/d13/d19/d2d/d4e/d58/d61/d71/da8 0 2024-04-12T20:56:16.371 INFO:tasks.workunit.client.0.smithi018.stdout:9/796: dread d3/d6/d9/dc/d5a/d79/d7e/d9b/d21/dc3/dd8/fa9 [0,4194304] 0 2024-04-12T20:56:16.379 INFO:tasks.workunit.client.0.smithi018.stdout:2/664: creat d5/d12/d54/fe4 x:0 0 0 2024-04-12T20:56:16.384 INFO:tasks.workunit.client.0.smithi018.stdout:5/766: symlink d0/d2/d33/d10/d4e/ded/l102 0 2024-04-12T20:56:16.384 INFO:tasks.workunit.client.0.smithi018.stdout:4/712: dwrite d3/d5d/d60/dc5/d53/f5e [0,4194304] 0 2024-04-12T20:56:16.388 INFO:tasks.workunit.client.0.smithi018.stdout:0/664: dwrite da/d15/d17/fc6 [0,4194304] 0 2024-04-12T20:56:16.391 INFO:tasks.workunit.client.0.smithi018.stdout:2/665: rename d5/d7/d28 to d5/d8/d10/d43/de2/d7c/d92/de5 0 2024-04-12T20:56:16.391 INFO:tasks.workunit.client.0.smithi018.stdout:5/767: write d0/d2/d33/d46/d35/fcc [337892,117118] 0 2024-04-12T20:56:16.401 INFO:tasks.workunit.client.0.smithi018.stdout:2/666: chown d5/d7/d63/f65 33 1 2024-04-12T20:56:16.406 INFO:tasks.workunit.client.0.smithi018.stdout:8/855: dread db/d4e/d56/d59/d76/da5/d45/d86/dbb/d113/f34 [4194304,4194304] 0 2024-04-12T20:56:16.406 INFO:tasks.workunit.client.0.smithi018.stdout:0/665: read da/fc [3506709,127506] 0 2024-04-12T20:56:16.407 INFO:tasks.workunit.client.0.smithi018.stdout:1/548: dwrite d3/d9/da/dd/d31/d32/d13/d19/d2d/d4e/d5e/fa7 [0,4194304] 0 2024-04-12T20:56:16.408 INFO:tasks.workunit.client.0.smithi018.stdout:2/667: mkdir d5/d8/d10/d43/de2/d24/d39/de6 0 2024-04-12T20:56:16.414 INFO:tasks.workunit.client.0.smithi018.stdout:1/549: read - d3/d9/da/d14/d73/f90 zero size 2024-04-12T20:56:16.415 INFO:tasks.workunit.client.0.smithi018.stdout:0/666: read da/d15/d17/d21/f5f [2962053,76673] 0 2024-04-12T20:56:16.420 INFO:tasks.workunit.client.0.smithi018.stdout:8/856: mknod db/d4e/d56/d59/d76/da5/d45/da9/ddb/c118 0 2024-04-12T20:56:16.421 INFO:tasks.workunit.client.0.smithi018.stdout:5/768: dread d0/d5b/fd6 [0,4194304] 0 2024-04-12T20:56:16.423 INFO:tasks.workunit.client.0.smithi018.stdout:0/667: stat da/d15/d17/d21/d36/dce/cd7 0 2024-04-12T20:56:16.426 INFO:tasks.workunit.client.0.smithi018.stdout:8/857: write db/d4e/d9a/d71/f77 [772932,98090] 0 2024-04-12T20:56:16.432 INFO:tasks.workunit.client.0.smithi018.stdout:1/550: mkdir d3/d9/da/dd/d31/d32/d13/d19/d2d/d4e/db9/dbd 0 2024-04-12T20:56:16.434 INFO:tasks.workunit.client.0.smithi018.stdout:9/797: dwrite d3/d6/d9/dc/d5a/d79/d7e/d9b/d21/dc3/dd8/d5b/d7c/f69 [0,4194304] 0 2024-04-12T20:56:16.439 INFO:tasks.workunit.client.0.smithi018.stdout:2/668: dread d5/d7/d31/f64 [0,4194304] 0 2024-04-12T20:56:16.440 INFO:tasks.workunit.client.0.smithi018.stdout:9/798: write d3/d6/d9/dc/d6f/d77/fde [972896,23353] 0 2024-04-12T20:56:16.442 INFO:tasks.workunit.client.0.smithi018.stdout:9/799: fdatasync d3/d6/d43/fd4 0 2024-04-12T20:56:16.443 INFO:tasks.workunit.client.0.smithi018.stdout:0/668: mkdir da/d15/d17/d21/d36/d88/ddf 0 2024-04-12T20:56:16.444 INFO:tasks.workunit.client.0.smithi018.stdout:9/800: chown d3/f23 150412603 1 2024-04-12T20:56:16.445 INFO:tasks.workunit.client.0.smithi018.stdout:0/669: write da/f38 [7775332,71846] 0 2024-04-12T20:56:16.448 INFO:tasks.workunit.client.0.smithi018.stdout:5/769: link d0/d2/d33/d46/c96 d0/d2/d33/d10/d60/dfa/dbf/da3/dae/c103 0 2024-04-12T20:56:16.448 INFO:tasks.workunit.client.0.smithi018.stdout:1/551: mkdir d3/d9/da/dd/d31/d32/d13/d19/d2d/d4e/d58/dbe 0 2024-04-12T20:56:16.451 INFO:tasks.workunit.client.0.smithi018.stdout:2/669: symlink d5/d8/d33/le7 0 2024-04-12T20:56:16.453 INFO:tasks.workunit.client.0.smithi018.stdout:2/670: chown d5/c4c 408571083 1 2024-04-12T20:56:16.455 INFO:tasks.workunit.client.0.smithi018.stdout:2/671: stat d5/d8/d33/ce3 0 2024-04-12T20:56:16.458 INFO:tasks.workunit.client.0.smithi018.stdout:2/672: write d5/d12/d54/fe4 [934627,45768] 0 2024-04-12T20:56:16.463 INFO:tasks.workunit.client.0.smithi018.stdout:0/670: rename da/d15/d4d/d58/dac/lb2 to da/d15/d17/d21/le0 0 2024-04-12T20:56:16.463 INFO:tasks.workunit.client.0.smithi018.stdout:5/770: fsync d0/d2/d33/d46/d1e/f6c 0 2024-04-12T20:56:16.465 INFO:tasks.workunit.client.0.smithi018.stdout:5/771: fdatasync d0/d2/f1a 0 2024-04-12T20:56:16.466 INFO:tasks.workunit.client.0.smithi018.stdout:8/858: write db/d4e/d56/d59/d76/de0/f53 [2968585,39254] 0 2024-04-12T20:56:16.470 INFO:tasks.workunit.client.0.smithi018.stdout:1/552: rename d3/d9/da/dd/d31/d32/d13/cb6 to d3/d9/da/dd/d31/d32/d13/d19/d2d/d4e/d58/d61/cbf 0 2024-04-12T20:56:16.471 INFO:tasks.workunit.client.0.smithi018.stdout:0/671: mknod da/d15/d4d/d58/ce1 0 2024-04-12T20:56:16.479 INFO:tasks.workunit.client.0.smithi018.stdout:2/673: dwrite d5/d8/d25/fba [0,4194304] 0 2024-04-12T20:56:16.480 INFO:tasks.workunit.client.0.smithi018.stdout:5/772: read d0/d2/d33/d46/d1e/d2e/fb7 [419747,43354] 0 2024-04-12T20:56:16.489 INFO:tasks.workunit.client.0.smithi018.stdout:2/674: fdatasync d5/d8/d10/d43/de2/d24/d39/f51 0 2024-04-12T20:56:16.489 INFO:tasks.workunit.client.0.smithi018.stdout:5/773: creat d0/d2/d33/d46/d1e/f104 x:0 0 0 2024-04-12T20:56:16.499 INFO:tasks.workunit.client.0.smithi018.stdout:8/859: truncate db/d4e/d56/d59/d76/fbe 779662 0 2024-04-12T20:56:16.499 INFO:tasks.workunit.client.0.smithi018.stdout:9/801: dwrite d3/d6/d9/dc/f97 [0,4194304] 0 2024-04-12T20:56:16.500 INFO:tasks.workunit.client.0.smithi018.stdout:5/774: mknod d0/d2/d33/d46/d1e/d9a/c105 0 2024-04-12T20:56:16.506 INFO:tasks.workunit.client.0.smithi018.stdout:5/775: creat d0/d2/d33/d46/d35/f106 x:0 0 0 2024-04-12T20:56:16.506 INFO:tasks.workunit.client.0.smithi018.stdout:5/776: chown d0 277254 1 2024-04-12T20:56:16.509 INFO:tasks.workunit.client.0.smithi018.stdout:9/802: write d3/d6/d9/dc/d5a/d79/d7e/d9b/d21/dc3/dd8/fd2 [353071,48572] 0 2024-04-12T20:56:16.510 INFO:tasks.workunit.client.0.smithi018.stdout:1/553: dread d3/d9/da/dd/d31/d32/f41 [0,4194304] 0 2024-04-12T20:56:16.511 INFO:tasks.workunit.client.0.smithi018.stdout:2/675: dwrite d5/d8/d10/d43/de2/d24/f36 [0,4194304] 0 2024-04-12T20:56:16.513 INFO:tasks.workunit.client.0.smithi018.stdout:9/803: chown d3/d6/d9/fd 53 1 2024-04-12T20:56:16.519 INFO:tasks.workunit.client.0.smithi018.stdout:5/777: stat d0/d2/d33/d10/d60/dfa/dbf/da3/dae/ldf 0 2024-04-12T20:56:16.520 INFO:tasks.workunit.client.0.smithi018.stdout:2/676: creat d5/d12/fe8 x:0 0 0 2024-04-12T20:56:16.529 INFO:tasks.workunit.client.0.smithi018.stdout:9/804: stat d3/d6/d9/dc/d5a/d79/d7e/d9b/d21/d41/d78/f83 0 2024-04-12T20:56:16.529 INFO:tasks.workunit.client.0.smithi018.stdout:2/677: mkdir d5/d8/d10/d43/d8f/de9 0 2024-04-12T20:56:16.531 INFO:tasks.workunit.client.0.smithi018.stdout:1/554: rename d3/d9/da/dd/d31/d32/l79 to d3/d9/da/dd/d31/d32/lc0 0 2024-04-12T20:56:16.536 INFO:tasks.workunit.client.0.smithi018.stdout:2/678: unlink d5/d12/d54/dbc/fc3 0 2024-04-12T20:56:16.536 INFO:tasks.workunit.client.0.smithi018.stdout:9/805: read d3/d6/d9/dc/d5a/d79/d7e/d9b/f34 [1477139,46083] 0 2024-04-12T20:56:16.538 INFO:tasks.workunit.client.0.smithi018.stdout:1/555: dread - d3/d9/da/dd/d31/d32/d13/d19/fb4 zero size 2024-04-12T20:56:16.541 INFO:tasks.workunit.client.0.smithi018.stdout:5/778: dwrite d0/d2/d11/d37/dc2/f86 [0,4194304] 0 2024-04-12T20:56:16.551 INFO:tasks.workunit.client.0.smithi018.stdout:5/779: truncate d0/d2/d33/d61/fd9 212467 0 2024-04-12T20:56:16.555 INFO:tasks.workunit.client.0.smithi018.stdout:9/806: rename d3/d6/d9/dc/d5a/d79/d7e/fbe to d3/d6/d3d/dba/fe9 0 2024-04-12T20:56:16.555 INFO:tasks.workunit.client.0.smithi018.stdout:5/780: unlink d0/d2/d33/d10/d60/cc6 0 2024-04-12T20:56:16.561 INFO:tasks.workunit.client.0.smithi018.stdout:2/679: write d5/d12/f89 [2093325,82665] 0 2024-04-12T20:56:16.561 INFO:tasks.workunit.client.0.smithi018.stdout:1/556: write d3/f4 [3812523,124973] 0 2024-04-12T20:56:16.570 INFO:tasks.workunit.client.0.smithi018.stdout:5/781: symlink d0/d5b/d84/l107 0 2024-04-12T20:56:16.570 INFO:tasks.workunit.client.0.smithi018.stdout:1/557: symlink d3/d9/da/dd/d31/d32/d33/d47/lc1 0 2024-04-12T20:56:16.571 INFO:tasks.workunit.client.0.smithi018.stdout:8/860: dread db/fb3 [4194304,4194304] 0 2024-04-12T20:56:16.573 INFO:tasks.workunit.client.0.smithi018.stdout:2/680: symlink d5/d8/d10/d43/de2/d24/d39/de6/lea 0 2024-04-12T20:56:16.573 INFO:tasks.workunit.client.0.smithi018.stdout:9/807: mkdir d3/d6/d9/dc/d5a/d79/d7e/d9b/d21/dc3/dd8/d5b/d7c/dea 0 2024-04-12T20:56:16.575 INFO:tasks.workunit.client.0.smithi018.stdout:1/558: mkdir d3/d9/da/dd/d31/d32/d33/d84/dc2 0 2024-04-12T20:56:16.579 INFO:tasks.workunit.client.0.smithi018.stdout:8/861: write db/d4e/d56/d59/d76/da5/d45/d86/f10a [423800,1579] 0 2024-04-12T20:56:16.581 INFO:tasks.workunit.client.0.smithi018.stdout:1/559: dread - d3/d9/da/dd/d31/d32/d13/d19/fb4 zero size 2024-04-12T20:56:16.581 INFO:tasks.workunit.client.0.smithi018.stdout:8/862: chown db/d4e/dab/fdf 492 1 2024-04-12T20:56:16.585 INFO:tasks.workunit.client.0.smithi018.stdout:9/808: creat d3/d6/d3d/feb x:0 0 0 2024-04-12T20:56:16.586 INFO:tasks.workunit.client.0.smithi018.stdout:2/681: rename d5/d12/d54/c3d to d5/ceb 0 2024-04-12T20:56:16.587 INFO:tasks.workunit.client.0.smithi018.stdout:8/863: unlink db/d4e/dab/ff6 0 2024-04-12T20:56:16.590 INFO:tasks.workunit.client.0.smithi018.stdout:2/682: mkdir d5/d8/dec 0 2024-04-12T20:56:16.592 INFO:tasks.workunit.client.0.smithi018.stdout:8/864: unlink db/d4e/d73/df7/c10e 0 2024-04-12T20:56:16.593 INFO:tasks.workunit.client.0.smithi018.stdout:5/782: dwrite d0/d2/d33/d46/d1e/d2e/f75 [0,4194304] 0 2024-04-12T20:56:16.607 INFO:tasks.workunit.client.0.smithi018.stdout:8/865: fsync db/d4e/d56/d59/d76/de0/d27/df8/fb4 0 2024-04-12T20:56:16.607 INFO:tasks.workunit.client.0.smithi018.stdout:1/560: dwrite d3/d9/f3c [0,4194304] 0 2024-04-12T20:56:16.611 INFO:tasks.workunit.client.0.smithi018.stdout:1/561: chown d3/d9/da/dd/d31/d32/d33/d84/cbb 1944 1 2024-04-12T20:56:16.613 INFO:tasks.workunit.client.0.smithi018.stdout:8/866: mkdir db/d4e/d56/d59/d76/de0/d27/df8/d119 0 2024-04-12T20:56:16.616 INFO:tasks.workunit.client.0.smithi018.stdout:2/683: chown d5/d8/d10/d43/de2/l27 875 1 2024-04-12T20:56:16.616 INFO:tasks.workunit.client.0.smithi018.stdout:5/783: link d0/d2/d33/d46/d1e/f6a d0/d2/d11/f108 0 2024-04-12T20:56:16.620 INFO:tasks.workunit.client.0.smithi018.stdout:1/562: dread d3/d9/da/dd/d31/f59 [0,4194304] 0 2024-04-12T20:56:16.621 INFO:tasks.workunit.client.0.smithi018.stdout:8/867: fsync db/d4e/d56/d59/d76/de0/d27/f33 0 2024-04-12T20:56:16.622 INFO:tasks.workunit.client.0.smithi018.stdout:2/684: unlink d5/d8/d10/d43/de2/d7c/cd6 0 2024-04-12T20:56:16.625 INFO:tasks.workunit.client.0.smithi018.stdout:5/784: symlink d0/d2/d33/d10/d26/dda/l109 0 2024-04-12T20:56:16.627 INFO:tasks.workunit.client.0.smithi018.stdout:8/868: fdatasync db/d4e/d56/d59/d76/da5/d45/d86/da1/fb6 0 2024-04-12T20:56:16.628 INFO:tasks.workunit.client.0.smithi018.stdout:1/563: mkdir d3/d9/dc3 0 2024-04-12T20:56:16.629 INFO:tasks.workunit.client.0.smithi018.stdout:2/685: mkdir d5/d8/d10/ded 0 2024-04-12T20:56:16.632 INFO:tasks.workunit.client.0.smithi018.stdout:8/869: write db/d4e/d56/f63 [735015,33661] 0 2024-04-12T20:56:16.632 INFO:tasks.workunit.client.0.smithi018.stdout:9/809: dwrite d3/d6/d1b/f35 [0,4194304] 0 2024-04-12T20:56:16.640 INFO:tasks.workunit.client.0.smithi018.stdout:9/810: unlink d3/d6/d9/dc/d5a/fae 0 2024-04-12T20:56:16.640 INFO:tasks.workunit.client.0.smithi018.stdout:1/564: rename d3/d9/da/d14/d73/c86 to d3/d9/da/dd/d31/d32/d33/d47/cc4 0 2024-04-12T20:56:16.648 INFO:tasks.workunit.client.0.smithi018.stdout:9/811: rmdir d3/d6/d9/dc/d5a/d79/d7e/d9b/d21/dc3/dd8/db9 39 2024-04-12T20:56:16.657 INFO:tasks.workunit.client.0.smithi018.stdout:9/812: creat d3/d6/d9/dc/d5a/d79/d7e/d9b/d21/dc3/dd8/d5b/d6b/da7/de7/fec x:0 0 0 2024-04-12T20:56:16.663 INFO:tasks.workunit.client.0.smithi018.stdout:2/686: dread d5/d7/d31/f4f [0,4194304] 0 2024-04-12T20:56:16.664 INFO:tasks.workunit.client.0.smithi018.stdout:9/813: chown d3/d6/d9/dc/d5a/d79/d7e/le4 7557 1 2024-04-12T20:56:16.664 INFO:tasks.workunit.client.0.smithi018.stdout:1/565: dwrite d3/d9/da/dd/d31/d32/d13/d19/d2d/f76 [4194304,4194304] 0 2024-04-12T20:56:16.667 INFO:tasks.workunit.client.0.smithi018.stdout:1/566: stat d3/d9/da/dd/d31/d44 0 2024-04-12T20:56:16.670 INFO:tasks.workunit.client.0.smithi018.stdout:8/870: dread db/d4e/d9a/daa/f10c [0,4194304] 0 2024-04-12T20:56:16.675 INFO:tasks.workunit.client.0.smithi018.stdout:1/567: unlink d3/d9/da/dd/f17 0 2024-04-12T20:56:16.677 INFO:tasks.workunit.client.0.smithi018.stdout:9/814: symlink d3/d6/d9/dc/d5a/d79/d7e/d9b/d21/dc3/dd8/d5b/d7c/led 0 2024-04-12T20:56:16.678 INFO:tasks.workunit.client.0.smithi018.stdout:8/871: mkdir db/d4e/d56/d59/d76/da5/d11a 0 2024-04-12T20:56:16.682 INFO:tasks.workunit.client.0.smithi018.stdout:9/815: chown d3/d6/d9/dc/d46/lc6 18 1 2024-04-12T20:56:16.684 INFO:tasks.workunit.client.0.smithi018.stdout:9/816: dread - d3/fe2 zero size 2024-04-12T20:56:16.687 INFO:tasks.workunit.client.0.smithi018.stdout:1/568: unlink d3/d9/da/dd/d31/d32/d13/d19/d2d/d4e/d7a/l8d 0 2024-04-12T20:56:16.689 INFO:tasks.workunit.client.0.smithi018.stdout:8/872: mknod db/d4e/d56/d59/d76/de0/d27/df8/d119/c11b 0 2024-04-12T20:56:16.690 INFO:tasks.workunit.client.0.smithi018.stdout:2/687: dwrite d5/d8/d10/d43/de2/d7c/fbf [0,4194304] 0 2024-04-12T20:56:16.692 INFO:tasks.workunit.client.0.smithi018.stdout:8/873: chown db/d4e/d9a/daa/dcd 184284833 1 2024-04-12T20:56:16.694 INFO:tasks.workunit.client.0.smithi018.stdout:8/874: stat db/d4e/d9a/d71 0 2024-04-12T20:56:16.695 INFO:tasks.workunit.client.0.smithi018.stdout:1/569: creat d3/d9/da/dd/d31/d32/d13/d19/d2d/d4e/d5e/d97/fc5 x:0 0 0 2024-04-12T20:56:16.702 INFO:tasks.workunit.client.0.smithi018.stdout:1/570: mkdir d3/d9/da/dd/d31/dc6 0 2024-04-12T20:56:16.704 INFO:tasks.workunit.client.0.smithi018.stdout:8/875: truncate db/d4e/d73/f8d 705120 0 2024-04-12T20:56:16.707 INFO:tasks.workunit.client.0.smithi018.stdout:9/817: dwrite d3/d6/d3d/dba/fc9 [0,4194304] 0 2024-04-12T20:56:16.711 INFO:tasks.workunit.client.0.smithi018.stdout:2/688: dwrite d5/d7/d31/d62/fc2 [0,4194304] 0 2024-04-12T20:56:16.712 INFO:tasks.workunit.client.0.smithi018.stdout:8/876: creat db/d4e/d56/d59/d76/da5/d45/da9/f11c x:0 0 0 2024-04-12T20:56:16.714 INFO:tasks.workunit.client.0.smithi018.stdout:8/877: readlink db/d4e/d9a/d71/lfe 0 2024-04-12T20:56:16.721 INFO:tasks.workunit.client.0.smithi018.stdout:2/689: mknod d5/d8/cee 0 2024-04-12T20:56:16.721 INFO:tasks.workunit.client.0.smithi018.stdout:1/571: dread d3/d9/da/f5f [0,4194304] 0 2024-04-12T20:56:16.729 INFO:tasks.workunit.client.0.smithi018.stdout:1/572: write d3/d9/d11/fa5 [568261,496] 0 2024-04-12T20:56:16.737 INFO:tasks.workunit.client.0.smithi018.stdout:1/573: chown d3/d9/da/dd/d26/ca2 33 1 2024-04-12T20:56:16.738 INFO:tasks.workunit.client.0.smithi018.stdout:9/818: dwrite d3/dc8/fe8 [0,4194304] 0 2024-04-12T20:56:16.746 INFO:tasks.workunit.client.0.smithi018.stdout:9/819: mknod d3/d6/d9/dc/d46/cee 0 2024-04-12T20:56:16.748 INFO:tasks.workunit.client.0.smithi018.stdout:1/574: fdatasync d3/d9/da/dd/d31/d32/d13/d19/d2d/d4e/d5e/d97/fc5 0 2024-04-12T20:56:16.749 INFO:tasks.workunit.client.0.smithi018.stdout:2/690: truncate d5/d8/d10/d43/fa5 281946 0 2024-04-12T20:56:16.749 INFO:tasks.workunit.client.0.smithi018.stdout:1/575: stat d3/d9/da/l72 0 2024-04-12T20:56:16.757 INFO:tasks.workunit.client.0.smithi018.stdout:2/691: fsync d5/d8/d25/fba 0 2024-04-12T20:56:16.760 INFO:tasks.workunit.client.0.smithi018.stdout:9/820: creat d3/d6/d9/dc/d5a/d79/d7e/d9b/d21/d41/fef x:0 0 0 2024-04-12T20:56:16.761 INFO:tasks.workunit.client.0.smithi018.stdout:1/576: readlink d3/d9/da/dd/d31/d32/d13/d19/d2d/d4e/d58/d61/d71/l94 0 2024-04-12T20:56:16.764 INFO:tasks.workunit.client.0.smithi018.stdout:8/878: dread db/d4e/d9a/f60 [0,4194304] 0 2024-04-12T20:56:16.770 INFO:tasks.workunit.client.0.smithi018.stdout:8/879: fdatasync db/d4e/d73/df7/f111 0 2024-04-12T20:56:16.776 INFO:tasks.workunit.client.0.smithi018.stdout:8/880: truncate db/d4e/d56/d59/d76/da5/d45/d86/ffb 570574 0 2024-04-12T20:56:16.781 INFO:tasks.workunit.client.0.smithi018.stdout:9/821: dwrite d3/d6/d9/dc/fca [0,4194304] 0 2024-04-12T20:56:16.785 INFO:tasks.workunit.client.0.smithi018.stdout:1/577: dwrite d3/d9/da/f4c [0,4194304] 0 2024-04-12T20:56:16.787 INFO:tasks.workunit.client.0.smithi018.stdout:9/822: creat d3/d6/d9/dc/dab/ff0 x:0 0 0 2024-04-12T20:56:16.787 INFO:tasks.workunit.client.0.smithi018.stdout:8/881: truncate db/d4e/d73/f8d 983153 0 2024-04-12T20:56:16.794 INFO:tasks.workunit.client.0.smithi018.stdout:9/823: creat d3/d6/d9/dc/d5a/d79/d7e/d9b/d21/dc3/dd8/d5b/d7c/dea/ff1 x:0 0 0 2024-04-12T20:56:16.798 INFO:tasks.workunit.client.0.smithi018.stdout:1/578: truncate d3/d9/f3c 3732320 0 2024-04-12T20:56:16.814 INFO:tasks.workunit.client.0.smithi018.stdout:8/882: dwrite db/d72/fd7 [0,4194304] 0 2024-04-12T20:56:16.832 INFO:tasks.workunit.client.0.smithi018.stdout:9/824: dwrite d3/d6/d3d/dba/fe9 [0,4194304] 0 2024-04-12T20:56:16.836 INFO:tasks.workunit.client.0.smithi018.stdout:1/579: mknod d3/d9/da/dd/d26/cc7 0 2024-04-12T20:56:16.839 INFO:tasks.workunit.client.0.smithi018.stdout:8/883: getdents db/d4e/d7f/dba 0 2024-04-12T20:56:16.976 INFO:tasks.workunit.client.0.smithi018.stdout:8/884: readlink db/d4e/d56/d59/d76/da5/l22 0 2024-04-12T20:56:16.987 INFO:tasks.workunit.client.0.smithi018.stdout:8/885: rename db/d4e/d56/d59/d76/da5/d45/c74 to db/d3b/c11d 0 2024-04-12T20:56:16.987 INFO:tasks.workunit.client.0.smithi018.stdout:9/825: dwrite d3/d6/d9/dc/d5a/d79/fb3 [4194304,4194304] 0 2024-04-12T20:56:16.992 INFO:tasks.workunit.client.0.smithi018.stdout:9/826: write d3/d6/d9/dc/d5a/d79/d7e/d9b/f34 [3189419,77250] 0 2024-04-12T20:56:16.995 INFO:tasks.workunit.client.0.smithi018.stdout:9/827: write d3/d6/d9/dc/f97 [4654950,116527] 0 2024-04-12T20:56:16.999 INFO:tasks.workunit.client.0.smithi018.stdout:1/580: write d3/d9/da/dd/d31/f59 [241234,52979] 0 2024-04-12T20:56:17.002 INFO:tasks.workunit.client.0.smithi018.stdout:8/886: getdents db/d4e/d56/d59/d76/da5/d45/d86/dbb/d113 0 2024-04-12T20:56:17.003 INFO:tasks.workunit.client.0.smithi018.stdout:8/887: readlink l4 0 2024-04-12T20:56:17.003 INFO:tasks.workunit.client.0.smithi018.stdout:9/828: unlink d3/d6/d9/dc/d5a/d79/d7e/c84 0 2024-04-12T20:56:17.004 INFO:tasks.workunit.client.0.smithi018.stdout:1/581: chown d3/d9/da/dd/d31/d32/d13/d19/f62 80204487 1 2024-04-12T20:56:17.010 INFO:tasks.workunit.client.0.smithi018.stdout:9/829: dread - d3/d6/d9/dc/d5a/d79/d7e/d9b/d21/dc3/dd8/d5b/d6b/da7/de7/fec zero size 2024-04-12T20:56:17.011 INFO:tasks.workunit.client.0.smithi018.stdout:1/582: symlink d3/d9/da/dd/d31/dad/lc8 0 2024-04-12T20:56:17.018 INFO:tasks.workunit.client.0.smithi018.stdout:1/583: dread - d3/d9/da/dd/d31/d32/d13/d19/d2d/d4e/d7a/d82/fb5 zero size 2024-04-12T20:56:17.020 INFO:tasks.workunit.client.0.smithi018.stdout:9/830: rename d3/d6/d9/d3f/f50 to d3/d6/d9/dc/d6f/ff2 0 2024-04-12T20:56:17.022 INFO:tasks.workunit.client.0.smithi018.stdout:1/584: creat d3/d9/da/fc9 x:0 0 0 2024-04-12T20:56:17.023 INFO:tasks.workunit.client.0.smithi018.stdout:9/831: truncate d3/d6/d3d/feb 675168 0 2024-04-12T20:56:17.027 INFO:tasks.workunit.client.0.smithi018.stdout:9/832: unlink d3/d6/d3d/feb 0 2024-04-12T20:56:17.031 INFO:tasks.workunit.client.0.smithi018.stdout:8/888: dread db/f68 [0,4194304] 0 2024-04-12T20:56:17.037 INFO:tasks.workunit.client.0.smithi018.stdout:8/889: read db/d4e/d9a/fc8 [4120245,102185] 0 2024-04-12T20:56:17.039 INFO:tasks.workunit.client.0.smithi018.stdout:9/833: truncate d3/d6/d9/dc/d5a/d79/d7e/d9b/f62 1727833 0 2024-04-12T20:56:17.042 INFO:tasks.workunit.client.0.smithi018.stdout:8/890: mknod db/d4e/d56/d59/d76/de0/c11e 0 2024-04-12T20:56:17.045 INFO:tasks.workunit.client.0.smithi018.stdout:8/891: dread - db/d4e/d73/fe8 zero size 2024-04-12T20:56:17.045 INFO:tasks.workunit.client.0.smithi018.stdout:9/834: symlink d3/d6/d9/d1e/lf3 0 2024-04-12T20:56:17.047 INFO:tasks.workunit.client.0.smithi018.stdout:1/585: dwrite d3/d9/da/dd/d31/d32/d13/d19/d2d/d4e/f8c [0,4194304] 0 2024-04-12T20:56:17.067 INFO:tasks.workunit.client.0.smithi018.stdout:1/586: dwrite d3/d9/d11/fa5 [0,4194304] 0 2024-04-12T20:56:17.069 INFO:tasks.workunit.client.0.smithi018.stdout:8/892: dwrite db/d4e/d56/d59/d76/da5/d45/d86/dbb/d113/f4c [0,4194304] 0 2024-04-12T20:56:17.075 INFO:tasks.workunit.client.0.smithi018.stdout:1/587: write d3/d9/da/dd/d31/d32/d13/d19/d2d/d4e/d7a/d82/fb5 [929266,130180] 0 2024-04-12T20:56:17.078 INFO:tasks.workunit.client.0.smithi018.stdout:1/588: chown d3/d9/da/dd/d26/ca2 2699644 1 2024-04-12T20:56:17.080 INFO:tasks.workunit.client.0.smithi018.stdout:3/710: sync 2024-04-12T20:56:17.085 INFO:tasks.workunit.client.0.smithi018.stdout:9/835: dread d3/d6/d9/dc/d5a/d79/d7e/d9b/d21/dc2/fcd [0,4194304] 0 2024-04-12T20:56:17.090 INFO:tasks.workunit.client.0.smithi018.stdout:9/836: stat d3/d6/d9/dc/d5a/d79/d7e/d9b/d21/dc2 0 2024-04-12T20:56:17.092 INFO:tasks.workunit.client.0.smithi018.stdout:1/589: fdatasync d3/d9/da/dd/d31/d32/d13/d19/d2d/d4e/d58/d61/d71/fa9 0 2024-04-12T20:56:17.098 INFO:tasks.workunit.client.0.smithi018.stdout:9/837: rename d3/d6/d9/dc/d46/cd1 to d3/d6/d9/cf4 0 2024-04-12T20:56:17.102 INFO:tasks.workunit.client.0.smithi018.stdout:1/590: dread d3/d9/da/dd/d31/d32/d13/d19/f62 [0,4194304] 0 2024-04-12T20:56:17.105 INFO:tasks.workunit.client.0.smithi018.stdout:1/591: fdatasync d3/d9/da/d14/f38 0 2024-04-12T20:56:17.111 INFO:tasks.workunit.client.0.smithi018.stdout:9/838: rename d3/d6/d9/dc/d5a/d79/d7e/d9b/d21/dc3/dd8/db9/fd6 to d3/d6/d9/dc/d5a/d79/d7e/d9b/d21/dc3/dd8/d5b/d7c/dea/ff5 0 2024-04-12T20:56:17.114 INFO:tasks.workunit.client.0.smithi018.stdout:9/839: write d3/d6/d9/d1e/f96 [1479225,98968] 0 2024-04-12T20:56:17.115 INFO:tasks.workunit.client.0.smithi018.stdout:1/592: rename d3/d9/da/dd/d31/d32/d13/d19/d2d/d4e/d58/d61/d71/fa9 to d3/d9/da/dd/d31/d32/d33/d80/fca 0 2024-04-12T20:56:17.118 INFO:tasks.workunit.client.0.smithi018.stdout:3/711: dread d2/d6/d30/d31/d38/d3a/f45 [0,4194304] 0 2024-04-12T20:56:17.118 INFO:tasks.workunit.client.0.smithi018.stdout:9/840: mkdir d3/dc8/df6 0 2024-04-12T20:56:17.120 INFO:tasks.workunit.client.0.smithi018.stdout:1/593: stat d3/d9/da/dd/d31/d32/d13/d19/d2d/d4e/d7a/fa4 0 2024-04-12T20:56:17.120 INFO:tasks.workunit.client.0.smithi018.stdout:3/712: fdatasync d2/d6/d30/d31/f9c 0 2024-04-12T20:56:17.125 INFO:tasks.workunit.client.0.smithi018.stdout:1/594: mknod d3/d9/da/d14/ccb 0 2024-04-12T20:56:17.125 INFO:tasks.workunit.client.0.smithi018.stdout:3/713: getdents d2/d6/d34/d71/d7b 0 2024-04-12T20:56:17.126 INFO:tasks.workunit.client.0.smithi018.stdout:3/714: rmdir d2/d6/d34/d71/db2 39 2024-04-12T20:56:17.127 INFO:tasks.workunit.client.0.smithi018.stdout:9/841: rename d3/d6/d9/dc/d5a/d79/d7e/d9b/d21/dc3/dd8/fd2 to d3/d93/daf/ff7 0 2024-04-12T20:56:17.137 INFO:tasks.workunit.client.0.smithi018.stdout:1/595: link d3/d9/da/dd/f4a d3/d9/da/dd/d31/d32/d33/d80/dab/fcc 0 2024-04-12T20:56:17.138 INFO:tasks.workunit.client.0.smithi018.stdout:3/715: write d2/d4d/d88/f97 [488068,111525] 0 2024-04-12T20:56:17.143 INFO:tasks.workunit.client.0.smithi018.stdout:3/716: creat d2/d4d/d8d/fdf x:0 0 0 2024-04-12T20:56:17.150 INFO:tasks.workunit.client.0.smithi018.stdout:3/717: fdatasync d2/d6/d30/d31/d38/f44 0 2024-04-12T20:56:17.153 INFO:tasks.workunit.client.0.smithi018.stdout:3/718: write d2/f19 [2077631,63035] 0 2024-04-12T20:56:17.156 INFO:tasks.workunit.client.0.smithi018.stdout:1/596: dwrite d3/d9/da/f20 [0,4194304] 0 2024-04-12T20:56:17.160 INFO:tasks.workunit.client.0.smithi018.stdout:1/597: read d3/d9/da/f4c [3645471,41864] 0 2024-04-12T20:56:17.165 INFO:tasks.workunit.client.0.smithi018.stdout:1/598: symlink d3/d9/da/d14/d73/lcd 0 2024-04-12T20:56:17.174 INFO:tasks.workunit.client.0.smithi018.stdout:9/842: dwrite d3/d6/d9/dc/fb4 [0,4194304] 0 2024-04-12T20:56:17.179 INFO:tasks.workunit.client.0.smithi018.stdout:3/719: dwrite d2/d6/d34/fae [0,4194304] 0 2024-04-12T20:56:17.181 INFO:tasks.workunit.client.0.smithi018.stdout:1/599: rename d3/d9/da/dd/d31/dad to d3/d9/da/dd/d31/d32/d13/d19/d2d/d4e/d7a/d82/dce 0 2024-04-12T20:56:17.184 INFO:tasks.workunit.client.0.smithi018.stdout:3/720: creat d2/d6/d34/d71/dbc/fe0 x:0 0 0 2024-04-12T20:56:17.184 INFO:tasks.workunit.client.0.smithi018.stdout:1/600: dread - d3/d9/da/dd/d31/f77 zero size 2024-04-12T20:56:17.192 INFO:tasks.workunit.client.0.smithi018.stdout:1/601: creat d3/d9/da/dd/d31/d32/d13/d19/fcf x:0 0 0 2024-04-12T20:56:17.192 INFO:tasks.workunit.client.0.smithi018.stdout:3/721: mkdir d2/d6/d30/d31/d38/d3a/dd0/de1 0 2024-04-12T20:56:17.196 INFO:tasks.workunit.client.0.smithi018.stdout:3/722: write d2/d6/d30/f58 [611695,16078] 0 2024-04-12T20:56:17.199 INFO:tasks.workunit.client.0.smithi018.stdout:9/843: getdents d3/d6/d9/dc/d5a/d79 0 2024-04-12T20:56:17.204 INFO:tasks.workunit.client.0.smithi018.stdout:9/844: rmdir d3/d6/d9/dc/d5a/d79/d7e/d9b/d21 39 2024-04-12T20:56:17.206 INFO:tasks.workunit.client.0.smithi018.stdout:9/845: read d3/dc8/fe8 [4152080,72109] 0 2024-04-12T20:56:17.210 INFO:tasks.workunit.client.0.smithi018.stdout:3/723: write d2/d43/f8f [1256574,61043] 0 2024-04-12T20:56:17.211 INFO:tasks.workunit.client.0.smithi018.stdout:1/602: dread d3/d9/da/dd/d31/d32/d33/f6d [0,4194304] 0 2024-04-12T20:56:17.214 INFO:tasks.workunit.client.0.smithi018.stdout:9/846: creat d3/d6/d9/dc/d46/d5d/ff8 x:0 0 0 2024-04-12T20:56:17.215 INFO:tasks.workunit.client.0.smithi018.stdout:3/724: creat d2/d43/db6/db5/fe2 x:0 0 0 2024-04-12T20:56:17.216 INFO:tasks.workunit.client.0.smithi018.stdout:7/683: sync 2024-04-12T20:56:17.217 INFO:tasks.workunit.client.0.smithi018.stdout:6/846: sync 2024-04-12T20:56:17.219 INFO:tasks.workunit.client.0.smithi018.stdout:1/603: getdents d3/d9/da/dd/d31/d32/d33/d80/dab 0 2024-04-12T20:56:17.223 INFO:tasks.workunit.client.0.smithi018.stdout:9/847: creat d3/ff9 x:0 0 0 2024-04-12T20:56:17.223 INFO:tasks.workunit.client.0.smithi018.stdout:7/684: mkdir d1/db/d31/d45/d8c/d9d/d70/d96/d5f/da9/ddf 0 2024-04-12T20:56:17.225 INFO:tasks.workunit.client.0.smithi018.stdout:7/685: chown d1/d6/dc9 14 1 2024-04-12T20:56:17.228 INFO:tasks.workunit.client.0.smithi018.stdout:9/848: read d3/d6/d9/d1e/f40 [436944,69776] 0 2024-04-12T20:56:17.229 INFO:tasks.workunit.client.0.smithi018.stdout:7/686: creat d1/db/d6f/fe0 x:0 0 0 2024-04-12T20:56:17.231 INFO:tasks.workunit.client.0.smithi018.stdout:9/849: rmdir d3/d6/d9/dc/d46 39 2024-04-12T20:56:17.275 INFO:tasks.workunit.client.0.smithi018.stdout:9/850: read d3/d6/d9/dc/d5a/d79/d7e/d9b/f2e [407519,20207] 0 2024-04-12T20:56:17.277 INFO:tasks.workunit.client.0.smithi018.stdout:7/687: link d1/db/d31/d3a/d42/d4c/f5b d1/db/d31/d45/d8c/d9d/d70/d96/fe1 0 2024-04-12T20:56:17.280 INFO:tasks.workunit.client.0.smithi018.stdout:7/688: chown d1/db/d31/d45/d8c/d9d/d70/d96/d5f/fac 4082462 1 2024-04-12T20:56:17.281 INFO:tasks.workunit.client.0.smithi018.stdout:6/847: dread d5/d6/d1b/d3b/fd7 [0,4194304] 0 2024-04-12T20:56:17.283 INFO:tasks.workunit.client.0.smithi018.stdout:9/851: rename d3/d6/d9/d1e/f45 to d3/d6/d9/dc/d46/d5d/ffa 0 2024-04-12T20:56:17.284 INFO:tasks.workunit.client.0.smithi018.stdout:7/689: stat d1/db/d31/c2a 0 2024-04-12T20:56:17.285 INFO:tasks.workunit.client.0.smithi018.stdout:9/852: readlink d3/d6/d9/d3f/l47 0 2024-04-12T20:56:17.288 INFO:tasks.workunit.client.0.smithi018.stdout:7/690: chown d1/db/d31/d3a/d42/faf 210708 1 2024-04-12T20:56:17.290 INFO:tasks.workunit.client.0.smithi018.stdout:3/725: dread d2/d6/d34/d71/d7b/f7d [0,4194304] 0 2024-04-12T20:56:17.295 INFO:tasks.workunit.client.0.smithi018.stdout:3/726: symlink d2/d43/d61/d64/dd9/le3 0 2024-04-12T20:56:17.309 INFO:tasks.workunit.client.0.smithi018.stdout:6/848: write d5/d6/d1b/d3b/d22/f81 [851052,113094] 0 2024-04-12T20:56:17.312 INFO:tasks.workunit.client.0.smithi018.stdout:7/691: dread d1/db/f9f [0,4194304] 0 2024-04-12T20:56:17.313 INFO:tasks.workunit.client.0.smithi018.stdout:6/849: symlink d5/d15/dd1/l140 0 2024-04-12T20:56:17.316 INFO:tasks.workunit.client.0.smithi018.stdout:3/727: dwrite d2/d83/f92 [0,4194304] 0 2024-04-12T20:56:17.319 INFO:tasks.workunit.client.0.smithi018.stdout:9/853: dwrite d3/d6/d1b/f26 [0,4194304] 0 2024-04-12T20:56:17.323 INFO:tasks.workunit.client.0.smithi018.stdout:6/850: unlink d5/df/d19/d2b/c7d 0 2024-04-12T20:56:17.323 INFO:tasks.workunit.client.0.smithi018.stdout:3/728: mkdir d2/d6/d30/d31/d38/d3a/d76/da2/de4 0 2024-04-12T20:56:17.325 INFO:tasks.workunit.client.0.smithi018.stdout:9/854: chown d3/d6/d9/dc/d5a/d79/d7e/d9b/d21/l55 0 1 2024-04-12T20:56:17.327 INFO:tasks.workunit.client.0.smithi018.stdout:6/851: write d5/df/d3f/f11b [693734,105308] 0 2024-04-12T20:56:17.333 INFO:tasks.workunit.client.0.smithi018.stdout:3/729: write d2/d43/d61/d64/dbf/d54/fb7 [3531072,122904] 0 2024-04-12T20:56:17.334 INFO:tasks.workunit.client.0.smithi018.stdout:9/855: creat d3/d6/d9/dc/d6f/d77/ffb x:0 0 0 2024-04-12T20:56:17.336 INFO:tasks.workunit.client.0.smithi018.stdout:3/730: write d2/d6/f25 [1925717,123302] 0 2024-04-12T20:56:17.338 INFO:tasks.workunit.client.0.smithi018.stdout:6/852: rename d5/d6/d1b/d3b/d6a/cd2 to d5/d15/c141 0 2024-04-12T20:56:17.340 INFO:tasks.workunit.client.0.smithi018.stdout:9/856: write d3/d6/d9/dc/d46/f48 [1060076,83686] 0 2024-04-12T20:56:17.345 INFO:tasks.workunit.client.0.smithi018.stdout:6/853: creat d5/d15/f142 x:0 0 0 2024-04-12T20:56:17.349 INFO:tasks.workunit.client.0.smithi018.stdout:9/857: creat d3/d6/d9/dc/d46/d5d/ffc x:0 0 0 2024-04-12T20:56:17.349 INFO:tasks.workunit.client.0.smithi018.stdout:6/854: symlink d5/d6/d1b/d3b/d6a/d116/l143 0 2024-04-12T20:56:17.350 INFO:tasks.workunit.client.0.smithi018.stdout:7/692: dread d1/db/d31/d30/dc6/fc8 [0,4194304] 0 2024-04-12T20:56:17.357 INFO:tasks.workunit.client.0.smithi018.stdout:3/731: dwrite d2/d6/d30/f58 [8388608,4194304] 0 2024-04-12T20:56:17.363 INFO:tasks.workunit.client.0.smithi018.stdout:3/732: write d2/d6/d34/d71/dbc/fcd [1088984,49719] 0 2024-04-12T20:56:17.373 INFO:tasks.workunit.client.0.smithi018.stdout:6/855: dwrite d5/df/d19/f25 [4194304,4194304] 0 2024-04-12T20:56:17.373 INFO:tasks.workunit.client.0.smithi018.stdout:5/785: sync 2024-04-12T20:56:17.374 INFO:tasks.workunit.client.0.smithi018.stdout:8/893: sync 2024-04-12T20:56:17.374 INFO:tasks.workunit.client.0.smithi018.stdout:4/713: sync 2024-04-12T20:56:17.374 INFO:tasks.workunit.client.0.smithi018.stdout:0/672: sync 2024-04-12T20:56:17.374 INFO:tasks.workunit.client.0.smithi018.stdout:2/692: sync 2024-04-12T20:56:17.382 INFO:tasks.workunit.client.0.smithi018.stdout:8/894: write db/d4e/f101 [33982,87542] 0 2024-04-12T20:56:17.387 INFO:tasks.workunit.client.0.smithi018.stdout:8/895: dread - db/d4e/d56/d59/d76/de0/ff9 zero size 2024-04-12T20:56:17.390 INFO:tasks.workunit.client.0.smithi018.stdout:6/856: unlink d5/d6/d1b/d3b/d6a/d116/d104/d128/l137 0 2024-04-12T20:56:17.398 INFO:tasks.workunit.client.0.smithi018.stdout:6/857: creat d5/d6/d1b/d8c/dc0/f144 x:0 0 0 2024-04-12T20:56:17.409 INFO:tasks.workunit.client.0.smithi018.stdout:8/896: dwrite db/d72/f103 [0,4194304] 0 2024-04-12T20:56:17.422 INFO:tasks.workunit.client.0.smithi018.stdout:7/693: mknod d1/db/d31/d3a/d97/ce2 0 2024-04-12T20:56:17.422 INFO:tasks.workunit.client.0.smithi018.stdout:7/694: chown d1/f3e 23604952 1 2024-04-12T20:56:17.424 INFO:tasks.workunit.client.0.smithi018.stdout:6/858: dread d5/d6/d1b/d3b/f69 [0,4194304] 0 2024-04-12T20:56:17.429 INFO:tasks.workunit.client.0.smithi018.stdout:9/858: rename d3/d6/d9/dc/d6f/d77/ffb to d3/d6/d9/dc/d5a/d79/ffd 0 2024-04-12T20:56:17.432 INFO:tasks.workunit.client.0.smithi018.stdout:6/859: chown d5/df/d19/d43/c66 879172 1 2024-04-12T20:56:17.433 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:56:17 smithi018 ceph-mon[28799]: pgmap v41: 65 pgs: 65 active+clean; 1.4 GiB data, 5.8 GiB used, 531 GiB / 536 GiB avail; 58 MiB/s rd, 147 MiB/s wr, 406 op/s 2024-04-12T20:56:17.439 INFO:tasks.workunit.client.0.smithi018.stdout:6/860: mkdir d5/d6/d1b/d3b/d35/d4f/df3/d145 0 2024-04-12T20:56:17.442 INFO:tasks.workunit.client.0.smithi018.stdout:9/859: dread d3/d6/d9/dc/d5a/d79/d7e/d9b/d21/dc3/dd8/d5b/d6b/fa1 [0,4194304] 0 2024-04-12T20:56:17.447 INFO:tasks.workunit.client.0.smithi018.stdout:9/860: stat d3/d6/d9/dc/d5a/d79/d7e/d9b/d21/dc3/dd8/db9 0 2024-04-12T20:56:17.450 INFO:tasks.workunit.client.0.smithi018.stdout:2/693: dread d5/d8/d10/d43/de2/daf/d90/f9d [0,4194304] 0 2024-04-12T20:56:17.454 INFO:tasks.workunit.client.0.smithi018.stdout:0/673: dread da/f22 [0,4194304] 0 2024-04-12T20:56:17.455 INFO:tasks.workunit.client.0.smithi018.stdout:6/861: link d5/d6/d1b/d3b/d8e/f92 d5/df/d3f/f146 0 2024-04-12T20:56:17.456 INFO:tasks.workunit.client.0.smithi018.stdout:8/897: read db/d4e/d56/d59/d76/de0/d27/fa0 [392079,125038] 0 2024-04-12T20:56:17.456 INFO:tasks.workunit.client.0.smithi018.stdout:3/733: creat d2/d6/d34/da1/ddb/fe5 x:0 0 0 2024-04-12T20:56:17.464 INFO:tasks.workunit.client.0.smithi018.stdout:0/674: rename da/d15/f39 to da/d15/fe2 0 2024-04-12T20:56:17.464 INFO:tasks.workunit.client.0.smithi018.stdout:1/604: sync 2024-04-12T20:56:17.464 INFO:tasks.workunit.client.0.smithi018.stdout:3/734: dread d2/d4d/d88/f97 [0,4194304] 0 2024-04-12T20:56:17.465 INFO:tasks.workunit.client.0.smithi018.stdout:9/861: symlink d3/d6/d9/dc/d5a/d79/db5/lfe 0 2024-04-12T20:56:17.469 INFO:tasks.workunit.client.0.smithi018.stdout:2/694: unlink d5/d12/d54/d3e/faa 0 2024-04-12T20:56:17.470 INFO:tasks.workunit.client.0.smithi018.stdout:8/898: unlink db/fb3 0 2024-04-12T20:56:17.474 INFO:tasks.workunit.client.0.smithi018.stdout:6/862: creat d5/d6/d1b/d3b/d35/d13d/f147 x:0 0 0 2024-04-12T20:56:17.475 INFO:tasks.workunit.client.0.smithi018.stdout:2/695: write d5/d7/d31/d62/fc2 [292642,48995] 0 2024-04-12T20:56:17.476 INFO:tasks.workunit.client.0.smithi018.stdout:0/675: mkdir da/d15/d17/d21/d36/da2/dc3/d3f/de3 0 2024-04-12T20:56:17.477 INFO:tasks.workunit.client.0.smithi018.stdout:9/862: creat d3/d93/daf/fff x:0 0 0 2024-04-12T20:56:17.493 INFO:tasks.workunit.client.0.smithi018.stdout:1/605: link d3/d9/da/dd/d31/d32/d13/f46 d3/d9/da/d14/d73/fd0 0 2024-04-12T20:56:17.501 INFO:tasks.workunit.client.0.smithi018.stdout:8/899: write f7 [5114266,51832] 0 2024-04-12T20:56:17.502 INFO:tasks.workunit.client.0.smithi018.stdout:3/735: creat d2/d6/d30/d31/d38/d3a/d76/da2/de4/fe6 x:0 0 0 2024-04-12T20:56:17.505 INFO:tasks.workunit.client.0.smithi018.stdout:1/606: creat d3/fd1 x:0 0 0 2024-04-12T20:56:17.508 INFO:tasks.workunit.client.0.smithi018.stdout:6/863: dread d5/d6/fbd [0,4194304] 0 2024-04-12T20:56:17.510 INFO:tasks.workunit.client.0.smithi018.stdout:0/676: dread da/f38 [8388608,4194304] 0 2024-04-12T20:56:17.514 INFO:tasks.workunit.client.0.smithi018.stdout:2/696: link d5/d8/f82 d5/d7/db6/fef 0 2024-04-12T20:56:17.515 INFO:tasks.workunit.client.0.smithi018.stdout:8/900: read db/d4e/d56/fc7 [3599018,110422] 0 2024-04-12T20:56:17.521 INFO:tasks.workunit.client.0.smithi018.stdout:6/864: symlink d5/df/l148 0 2024-04-12T20:56:17.524 INFO:tasks.workunit.client.0.smithi018.stdout:0/677: rename da/d15/d4d/d58/dac/cd9 to da/d15/d17/d21/d36/d88/ce4 0 2024-04-12T20:56:17.528 INFO:tasks.workunit.client.0.smithi018.stdout:0/678: fdatasync da/d15/d17/d21/d36/da2/dc3/d3f/f69 0 2024-04-12T20:56:17.528 INFO:tasks.workunit.client.0.smithi018.stdout:2/697: mknod d5/d8/d10/d43/de2/daf/d88/cf0 0 2024-04-12T20:56:17.530 INFO:tasks.workunit.client.0.smithi018.stdout:1/607: rmdir d3/d9/da/dd/d31/d32/d13/d19/d2d/d4e/d58/d68 0 2024-04-12T20:56:17.530 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:56:17 smithi186 ceph-mon[36597]: pgmap v41: 65 pgs: 65 active+clean; 1.4 GiB data, 5.8 GiB used, 531 GiB / 536 GiB avail; 58 MiB/s rd, 147 MiB/s wr, 406 op/s 2024-04-12T20:56:17.531 INFO:tasks.workunit.client.0.smithi018.stdout:0/679: fsync da/d15/d17/d21/d94/fa4 0 2024-04-12T20:56:17.531 INFO:tasks.workunit.client.0.smithi018.stdout:2/698: readlink d5/d8/d25/l6d 0 2024-04-12T20:56:17.533 INFO:tasks.workunit.client.0.smithi018.stdout:6/865: creat d5/d6/d90/d125/d139/f149 x:0 0 0 2024-04-12T20:56:17.541 INFO:tasks.workunit.client.0.smithi018.stdout:2/699: creat d5/d7/d63/ff1 x:0 0 0 2024-04-12T20:56:17.541 INFO:tasks.workunit.client.0.smithi018.stdout:3/736: dread d2/d6/d30/d31/d38/f3b [0,4194304] 0 2024-04-12T20:56:17.546 INFO:tasks.workunit.client.0.smithi018.stdout:6/866: fdatasync d5/df/d31/d4e/f38 0 2024-04-12T20:56:17.549 INFO:tasks.workunit.client.0.smithi018.stdout:7/695: rmdir d1/db/d31/d3a 39 2024-04-12T20:56:17.553 INFO:tasks.workunit.client.0.smithi018.stdout:2/700: write d5/d8/f82 [123484,19121] 0 2024-04-12T20:56:17.554 INFO:tasks.workunit.client.0.smithi018.stdout:3/737: mknod d2/d6/d30/d31/d38/d3a/d76/da2/de4/ce7 0 2024-04-12T20:56:17.556 INFO:tasks.workunit.client.0.smithi018.stdout:8/901: dread db/f1b [0,4194304] 0 2024-04-12T20:56:17.558 INFO:tasks.workunit.client.0.smithi018.stdout:5/786: mknod d0/d5b/c10a 0 2024-04-12T20:56:17.560 INFO:tasks.workunit.client.0.smithi018.stdout:6/867: mkdir d5/df/d19/d43/d14a 0 2024-04-12T20:56:17.561 INFO:tasks.workunit.client.0.smithi018.stdout:8/902: write db/d4e/d7f/dfa/ffc [98756,106487] 0 2024-04-12T20:56:17.566 INFO:tasks.workunit.client.0.smithi018.stdout:3/738: unlink d2/d43/d61/d64/dbf/d54/fb7 0 2024-04-12T20:56:17.567 INFO:tasks.workunit.client.0.smithi018.stdout:5/787: chown d0/d2/d33/d46/d35/d4d/cf1 632313 1 2024-04-12T20:56:17.569 INFO:tasks.workunit.client.0.smithi018.stdout:4/714: rmdir d3/d32/d94 39 2024-04-12T20:56:17.569 INFO:tasks.workunit.client.0.smithi018.stdout:5/788: stat d0/d2/d33/d10/d60 0 2024-04-12T20:56:17.571 INFO:tasks.workunit.client.0.smithi018.stdout:3/739: chown d2/d6/d30/d31/d38/f3b 504179951 1 2024-04-12T20:56:17.579 INFO:tasks.workunit.client.0.smithi018.stdout:5/789: creat d0/d2/d33/d46/d35/f10b x:0 0 0 2024-04-12T20:56:17.580 INFO:tasks.workunit.client.0.smithi018.stdout:1/608: dread d3/d9/da/dd/d31/d32/d13/d19/d2d/d4e/d5e/fa7 [0,4194304] 0 2024-04-12T20:56:17.586 INFO:tasks.workunit.client.0.smithi018.stdout:4/715: rename d3/d5d/d60/dc5/d53/f5e to d3/d5d/d60/d64/fea 0 2024-04-12T20:56:17.586 INFO:tasks.workunit.client.0.smithi018.stdout:3/740: fdatasync d2/d4d/f57 0 2024-04-12T20:56:17.588 INFO:tasks.workunit.client.0.smithi018.stdout:3/741: dread - d2/d43/db6/db5/fe2 zero size 2024-04-12T20:56:17.593 INFO:tasks.workunit.client.0.smithi018.stdout:8/903: creat db/d4e/d56/d59/d76/f11f x:0 0 0 2024-04-12T20:56:17.596 INFO:tasks.workunit.client.0.smithi018.stdout:3/742: write d2/d6/d34/da1/ddb/fe5 [668419,85474] 0 2024-04-12T20:56:17.596 INFO:tasks.workunit.client.0.smithi018.stdout:1/609: creat d3/d9/da/dd/d31/d32/d13/d19/d2d/d4e/d7a/d82/dce/fd2 x:0 0 0 2024-04-12T20:56:17.599 INFO:tasks.workunit.client.0.smithi018.stdout:4/716: creat d3/d5d/d60/dc5/d3d/d80/feb x:0 0 0 2024-04-12T20:56:17.601 INFO:tasks.workunit.client.0.smithi018.stdout:9/863: write d3/d6/d9/dc/d5a/d79/d7e/d9b/d21/dc3/dd8/d5b/d6b/fa1 [1155289,7508] 0 2024-04-12T20:56:17.602 INFO:tasks.workunit.client.0.smithi018.stdout:6/868: dread d5/d6/d1b/d3b/d22/d117/d13c/fff [0,4194304] 0 2024-04-12T20:56:17.604 INFO:tasks.workunit.client.0.smithi018.stdout:8/904: chown db/d4e/c70 38686 1 2024-04-12T20:56:17.610 INFO:tasks.workunit.client.0.smithi018.stdout:2/701: creat d5/d8/d10/d43/de2/ff2 x:0 0 0 2024-04-12T20:56:17.612 INFO:tasks.workunit.client.0.smithi018.stdout:1/610: chown d3/d9/da/dd/d31/d32/d13/d19/d2d/d4e/f60 1333136 1 2024-04-12T20:56:17.613 INFO:tasks.workunit.client.0.smithi018.stdout:5/790: link d0/d2/d33/d46/c77 d0/d2/d33/d10/d26/dda/c10c 0 2024-04-12T20:56:17.615 INFO:tasks.workunit.client.0.smithi018.stdout:9/864: symlink d3/d6/d9/dc/d5a/d79/db5/l100 0 2024-04-12T20:56:17.618 INFO:tasks.workunit.client.0.smithi018.stdout:8/905: creat db/d4e/d56/d59/d76/da5/d45/d86/dbb/f120 x:0 0 0 2024-04-12T20:56:17.619 INFO:tasks.workunit.client.0.smithi018.stdout:1/611: symlink d3/d9/da/dd/d31/d32/d13/d19/d2d/d4e/d5e/ld3 0 2024-04-12T20:56:17.621 INFO:tasks.workunit.client.0.smithi018.stdout:6/869: rename d5/d6/d1b/d3b/d35/d4f/df3 to d5/d6/d1b/d3b/d22/d117/d14b 0 2024-04-12T20:56:17.622 INFO:tasks.workunit.client.0.smithi018.stdout:4/717: dread d3/d32/d33/db6/fca [0,4194304] 0 2024-04-12T20:56:17.626 INFO:tasks.workunit.client.0.smithi018.stdout:6/870: truncate d5/d15/f142 533892 0 2024-04-12T20:56:17.627 INFO:tasks.workunit.client.0.smithi018.stdout:2/702: symlink d5/lf3 0 2024-04-12T20:56:17.628 INFO:tasks.workunit.client.0.smithi018.stdout:8/906: mknod db/d95/dd6/c121 0 2024-04-12T20:56:17.631 INFO:tasks.workunit.client.0.smithi018.stdout:9/865: symlink d3/d6/d9/dc/d5a/d79/d7e/d9b/l101 0 2024-04-12T20:56:17.635 INFO:tasks.workunit.client.0.smithi018.stdout:4/718: creat d3/d5d/d60/d64/d4f/dc7/fec x:0 0 0 2024-04-12T20:56:17.635 INFO:tasks.workunit.client.0.smithi018.stdout:2/703: chown d5/d8/d10/d43/de2/d24/f36 34746990 1 2024-04-12T20:56:17.636 INFO:tasks.workunit.client.0.smithi018.stdout:6/871: truncate d5/d6/d1b/d3b/d22/d117/d13c/f11f 413561 0 2024-04-12T20:56:17.637 INFO:tasks.workunit.client.0.smithi018.stdout:0/680: write da/f22 [2712949,126210] 0 2024-04-12T20:56:17.640 INFO:tasks.workunit.client.0.smithi018.stdout:2/704: stat d5/d8/d25/fba 0 2024-04-12T20:56:17.642 INFO:tasks.workunit.client.0.smithi018.stdout:1/612: creat d3/d9/da/dd/d31/d32/fd4 x:0 0 0 2024-04-12T20:56:17.643 INFO:tasks.workunit.client.0.smithi018.stdout:3/743: rmdir d2/d4d 39 2024-04-12T20:56:17.644 INFO:tasks.workunit.client.0.smithi018.stdout:6/872: symlink d5/df/d31/d4e/l14c 0 2024-04-12T20:56:17.644 INFO:tasks.workunit.client.0.smithi018.stdout:8/907: mknod db/d72/de6/d112/c122 0 2024-04-12T20:56:17.645 INFO:tasks.workunit.client.0.smithi018.stdout:7/696: write d1/d6/f67 [831308,18946] 0 2024-04-12T20:56:17.649 INFO:tasks.workunit.client.0.smithi018.stdout:3/744: chown d2/d6/f25 289980 1 2024-04-12T20:56:17.657 INFO:tasks.workunit.client.0.smithi018.stdout:5/791: dwrite d0/d2/d33/d10/d26/dda/ff8 [0,4194304] 0 2024-04-12T20:56:17.659 INFO:tasks.workunit.client.0.smithi018.stdout:2/705: symlink d5/d8/d10/d43/de2/daf/lf4 0 2024-04-12T20:56:17.667 INFO:tasks.workunit.client.0.smithi018.stdout:6/873: creat d5/df/d19/d120/f14d x:0 0 0 2024-04-12T20:56:17.667 INFO:tasks.workunit.client.0.smithi018.stdout:8/908: creat db/d4e/d56/d59/d76/da5/f123 x:0 0 0 2024-04-12T20:56:17.668 INFO:tasks.workunit.client.0.smithi018.stdout:9/866: truncate d3/d6/d3d/dba/fe9 1284306 0 2024-04-12T20:56:17.672 INFO:tasks.workunit.client.0.smithi018.stdout:9/867: mknod d3/d6/d9/dc/d5a/c102 0 2024-04-12T20:56:17.674 INFO:tasks.workunit.client.0.smithi018.stdout:9/868: dread - d3/ff9 zero size 2024-04-12T20:56:17.676 INFO:tasks.workunit.client.0.smithi018.stdout:7/697: unlink d1/db/d31/d45/fdc 0 2024-04-12T20:56:17.680 INFO:tasks.workunit.client.0.smithi018.stdout:0/681: dwrite da/d15/d17/d21/d36/d6e/fc9 [0,4194304] 0 2024-04-12T20:56:17.683 INFO:tasks.workunit.client.0.smithi018.stdout:3/745: symlink d2/d6/d30/le8 0 2024-04-12T20:56:17.683 INFO:tasks.workunit.client.0.smithi018.stdout:9/869: creat d3/d6/d9/dc/d5a/d79/d7e/f103 x:0 0 0 2024-04-12T20:56:17.686 INFO:tasks.workunit.client.0.smithi018.stdout:8/909: fsync db/d4e/d56/d59/d76/de0/d27/f50 0 2024-04-12T20:56:17.686 INFO:tasks.workunit.client.0.smithi018.stdout:5/792: creat d0/d2/d33/d10/d60/dfa/d8f/f10d x:0 0 0 2024-04-12T20:56:17.694 INFO:tasks.workunit.client.0.smithi018.stdout:9/870: mknod d3/d6/d9/dc/d5a/d79/d7e/d9b/d21/dc2/c104 0 2024-04-12T20:56:17.697 INFO:tasks.workunit.client.0.smithi018.stdout:7/698: creat d1/db/d31/d45/d8c/d9d/d70/d96/d5f/da9/fe3 x:0 0 0 2024-04-12T20:56:17.697 INFO:tasks.workunit.client.0.smithi018.stdout:0/682: symlink da/d15/d17/d5c/d79/le5 0 2024-04-12T20:56:17.700 INFO:tasks.workunit.client.0.smithi018.stdout:0/683: stat da/d15/d4d/d58/cb6 0 2024-04-12T20:56:17.700 INFO:tasks.workunit.client.0.smithi018.stdout:9/871: mknod d3/d6/d9/dc/d5a/d79/d7e/d9b/d21/dc3/dd8/d5b/d6b/da7/c105 0 2024-04-12T20:56:17.700 INFO:tasks.workunit.client.0.smithi018.stdout:2/706: creat d5/d8/d10/d43/de2/d24/d91/ff5 x:0 0 0 2024-04-12T20:56:17.705 INFO:tasks.workunit.client.0.smithi018.stdout:4/719: write d3/d32/d33/f30 [1499739,33773] 0 2024-04-12T20:56:17.709 INFO:tasks.workunit.client.0.smithi018.stdout:3/746: rename d2/d6/d30/d31/f69 to d2/d6/d34/d71/dbc/dcc/fe9 0 2024-04-12T20:56:17.713 INFO:tasks.workunit.client.0.smithi018.stdout:1/613: link d3/d9/da/dd/d31/d32/c7d d3/d9/da/d14/d73/cd5 0 2024-04-12T20:56:17.715 INFO:tasks.workunit.client.0.smithi018.stdout:7/699: mkdir d1/db/d31/d45/d8c/d9d/d70/d96/d5f/de4 0 2024-04-12T20:56:17.718 INFO:tasks.workunit.client.0.smithi018.stdout:9/872: link d3/d6/d9/dc/d5a/d79/d7e/d9b/d21/dc3/dd8/d5b/d7c/ccb d3/d6/d9/dc/d5a/d79/d7e/d9b/d21/dc3/c106 0 2024-04-12T20:56:17.723 INFO:tasks.workunit.client.0.smithi018.stdout:9/873: chown d3/d6/d9/dc/d46/d5d/l88 83 1 2024-04-12T20:56:17.724 INFO:tasks.workunit.client.0.smithi018.stdout:5/793: unlink d0/d2/d33/d46/c96 0 2024-04-12T20:56:17.735 INFO:tasks.workunit.client.0.smithi018.stdout:9/874: chown d3/d6/d9/dc/d5a/d79/db5 14 1 2024-04-12T20:56:17.740 INFO:tasks.workunit.client.0.smithi018.stdout:1/614: getdents d3/d9/da/dd/d31/d32/d33/d84/dc2 0 2024-04-12T20:56:17.740 INFO:tasks.workunit.client.0.smithi018.stdout:3/747: dread d2/d6/d34/da1/ddb/fe5 [0,4194304] 0 2024-04-12T20:56:17.741 INFO:tasks.workunit.client.0.smithi018.stdout:8/910: link db/d4e/d56/d59/d76/de0/c3e db/d72/de6/c124 0 2024-04-12T20:56:17.756 INFO:tasks.workunit.client.0.smithi018.stdout:9/875: symlink d3/d6/d9/dc/d5a/d79/d7e/d9b/d21/dc3/dd8/d5b/d7c/dea/l107 0 2024-04-12T20:56:17.756 INFO:tasks.workunit.client.0.smithi018.stdout:0/684: dwrite da/d15/d17/d71/dc1/dcc/dcb/fd5 [0,4194304] 0 2024-04-12T20:56:17.756 INFO:tasks.workunit.client.0.smithi018.stdout:8/911: mknod db/c125 0 2024-04-12T20:56:17.761 INFO:tasks.workunit.client.0.smithi018.stdout:3/748: mknod d2/d6/d95/cea 0 2024-04-12T20:56:17.768 INFO:tasks.workunit.client.0.smithi018.stdout:0/685: chown da/l10 6855536 1 2024-04-12T20:56:17.768 INFO:tasks.workunit.client.0.smithi018.stdout:4/720: dread d3/d5d/d60/dc5/d43/fc9 [0,4194304] 0 2024-04-12T20:56:17.771 INFO:tasks.workunit.client.0.smithi018.stdout:0/686: chown da/d15/d17/d21/d36 1239 1 2024-04-12T20:56:17.774 INFO:tasks.workunit.client.0.smithi018.stdout:3/749: truncate d2/d43/d61/f63 1535098 0 2024-04-12T20:56:17.778 INFO:tasks.workunit.client.0.smithi018.stdout:1/615: dwrite d3/d9/da/dd/f55 [0,4194304] 0 2024-04-12T20:56:17.785 INFO:tasks.workunit.client.0.smithi018.stdout:0/687: chown da/f13 30288 1 2024-04-12T20:56:17.785 INFO:tasks.workunit.client.0.smithi018.stdout:8/912: write db/d4e/d56/d59/d76/de0/d27/f50 [683842,93930] 0 2024-04-12T20:56:17.785 INFO:tasks.workunit.client.0.smithi018.stdout:4/721: dread - d3/d32/d33/db6/fbb zero size 2024-04-12T20:56:17.790 INFO:tasks.workunit.client.0.smithi018.stdout:1/616: mkdir d3/d9/da/dd/d31/d32/d13/d19/dd6 0 2024-04-12T20:56:17.793 INFO:tasks.workunit.client.0.smithi018.stdout:1/617: chown d3/d9/da/d14/d73 205791 1 2024-04-12T20:56:17.797 INFO:tasks.workunit.client.0.smithi018.stdout:0/688: getdents da/d15/d17/d5c/dae 0 2024-04-12T20:56:17.800 INFO:tasks.workunit.client.0.smithi018.stdout:4/722: creat d3/d5d/d60/dc5/d3d/fed x:0 0 0 2024-04-12T20:56:17.801 INFO:tasks.workunit.client.0.smithi018.stdout:3/750: dread d2/d6/f1b [4194304,4194304] 0 2024-04-12T20:56:17.803 INFO:tasks.workunit.client.0.smithi018.stdout:1/618: mknod d3/d9/da/dd/d31/d32/d13/d19/d2d/d4e/d5e/cd7 0 2024-04-12T20:56:17.823 INFO:tasks.workunit.client.0.smithi018.stdout:4/723: mknod d3/d5d/cee 0 2024-04-12T20:56:17.828 INFO:tasks.workunit.client.0.smithi018.stdout:7/700: dread d1/db/d31/d3a/d42/d4c/f51 [0,4194304] 0 2024-04-12T20:56:17.835 INFO:tasks.workunit.client.0.smithi018.stdout:7/701: readlink d1/db/d31/d3a/lc1 0 2024-04-12T20:56:17.835 INFO:tasks.workunit.client.0.smithi018.stdout:5/794: dread d0/d2/d33/d10/f27 [0,4194304] 0 2024-04-12T20:56:17.836 INFO:tasks.workunit.client.0.smithi018.stdout:7/702: chown d1/f1b 0 1 2024-04-12T20:56:17.838 INFO:tasks.workunit.client.0.smithi018.stdout:9/876: dread d3/d6/d9/dc/d5a/d79/fb3 [0,4194304] 0 2024-04-12T20:56:17.842 INFO:tasks.workunit.client.0.smithi018.stdout:9/877: fsync d3/d6/d9/d3f/f6e 0 2024-04-12T20:56:17.843 INFO:tasks.workunit.client.0.smithi018.stdout:1/619: rename d3/c1a to d3/d9/da/dd/d31/d32/d33/d47/db0/cd8 0 2024-04-12T20:56:17.847 INFO:tasks.workunit.client.0.smithi018.stdout:9/878: unlink d3/d6/d9/dc/d6f/d77/fde 0 2024-04-12T20:56:17.847 INFO:tasks.workunit.client.0.smithi018.stdout:1/620: write d3/d9/da/dd/d31/d32/d33/d47/fb3 [623700,48925] 0 2024-04-12T20:56:17.850 INFO:tasks.workunit.client.0.smithi018.stdout:4/724: rmdir d3/d5d/d60/dc5/d43 39 2024-04-12T20:56:17.857 INFO:tasks.workunit.client.0.smithi018.stdout:7/703: creat d1/db/d31/d45/d8c/d9d/fe5 x:0 0 0 2024-04-12T20:56:17.858 INFO:tasks.workunit.client.0.smithi018.stdout:9/879: write d3/d6/d43/f8f [4350625,50618] 0 2024-04-12T20:56:17.860 INFO:tasks.workunit.client.0.smithi018.stdout:5/795: rename d0/d2/d33/d10/d60/dfa/dbf/cdd to d0/d2/d33/d46/d35/dd2/c10e 0 2024-04-12T20:56:17.864 INFO:tasks.workunit.client.0.smithi018.stdout:4/725: write d3/d32/d66/f75 [1944344,54328] 0 2024-04-12T20:56:17.867 INFO:tasks.workunit.client.0.smithi018.stdout:5/796: write d0/d2/d33/d10/d60/dfa/dbf/fe6 [654791,96889] 0 2024-04-12T20:56:17.870 INFO:tasks.workunit.client.0.smithi018.stdout:7/704: rename d1/db/l84 to d1/db/d31/d30/dc6/le6 0 2024-04-12T20:56:17.880 INFO:tasks.workunit.client.0.smithi018.stdout:9/880: dwrite d3/f80 [0,4194304] 0 2024-04-12T20:56:17.880 INFO:tasks.workunit.client.0.smithi018.stdout:5/797: getdents d0/d2b 0 2024-04-12T20:56:17.885 INFO:tasks.workunit.client.0.smithi018.stdout:3/751: write d2/d6/d30/d31/d38/d3a/d48/f89 [2049149,22320] 0 2024-04-12T20:56:17.885 INFO:tasks.workunit.client.0.smithi018.stdout:9/881: truncate d3/d6/d9/d1e/f40 967096 0 2024-04-12T20:56:17.890 INFO:tasks.workunit.client.0.smithi018.stdout:9/882: dread - d3/ff9 zero size 2024-04-12T20:56:17.890 INFO:tasks.workunit.client.0.smithi018.stdout:0/689: truncate da/d15/fe2 6423704 0 2024-04-12T20:56:17.896 INFO:tasks.workunit.client.0.smithi018.stdout:0/690: chown da/d15/d17/d21/d36/l92 3 1 2024-04-12T20:56:17.897 INFO:tasks.workunit.client.0.smithi018.stdout:3/752: rename d2/d6/f84 to d2/d43/d61/d64/dbf/d70/feb 0 2024-04-12T20:56:17.897 INFO:tasks.workunit.client.0.smithi018.stdout:7/705: write d1/db/d31/d45/f9e [1071309,86210] 0 2024-04-12T20:56:17.903 INFO:tasks.workunit.client.0.smithi018.stdout:0/691: unlink da/l12 0 2024-04-12T20:56:17.903 INFO:tasks.workunit.client.0.smithi018.stdout:3/753: mknod d2/d6/d34/d71/dbc/cec 0 2024-04-12T20:56:17.906 INFO:tasks.workunit.client.0.smithi018.stdout:7/706: write d1/db/d31/d45/d8c/d9d/fe5 [44923,2243] 0 2024-04-12T20:56:17.908 INFO:tasks.workunit.client.0.smithi018.stdout:9/883: link d3/d6/d9/dc/d6f/d77/fd3 d3/d6/f108 0 2024-04-12T20:56:17.910 INFO:tasks.workunit.client.0.smithi018.stdout:7/707: readlink d1/d6/l26 0 2024-04-12T20:56:17.913 INFO:tasks.workunit.client.0.smithi018.stdout:1/621: dwrite d3/d9/da/dd/d31/f77 [0,4194304] 0 2024-04-12T20:56:17.915 INFO:tasks.workunit.client.0.smithi018.stdout:9/884: write d3/d6/d9/dc/d46/d5d/ffc [442755,47751] 0 2024-04-12T20:56:17.916 INFO:tasks.workunit.client.0.smithi018.stdout:5/798: dwrite d0/d2/d33/d61/fd9 [0,4194304] 0 2024-04-12T20:56:17.923 INFO:tasks.workunit.client.0.smithi018.stdout:0/692: mknod da/d15/d17/d21/d36/da2/dc3/da9/dd3/ce6 0 2024-04-12T20:56:17.924 INFO:tasks.workunit.client.0.smithi018.stdout:7/708: symlink d1/db/d31/d3a/d42/d4c/le7 0 2024-04-12T20:56:17.925 INFO:tasks.workunit.client.0.smithi018.stdout:5/799: fdatasync d0/d2/d33/d10/f39 0 2024-04-12T20:56:17.930 INFO:tasks.workunit.client.0.smithi018.stdout:5/800: write d0/d2/d33/d10/d26/f4a [6090487,96002] 0 2024-04-12T20:56:17.933 INFO:tasks.workunit.client.0.smithi018.stdout:3/754: link d2/d43/d61/f62 d2/d43/d61/d64/dbf/fed 0 2024-04-12T20:56:17.935 INFO:tasks.workunit.client.0.smithi018.stdout:3/755: chown d2/d83 1880060 1 2024-04-12T20:56:17.936 INFO:tasks.workunit.client.0.smithi018.stdout:5/801: chown d0/d2/fbe 27 1 2024-04-12T20:56:17.937 INFO:tasks.workunit.client.0.smithi018.stdout:7/709: write d1/db/d31/d45/d8c/d9d/d69/f89 [5218714,16690] 0 2024-04-12T20:56:17.946 INFO:tasks.workunit.client.0.smithi018.stdout:5/802: unlink d0/d2/d33/d46/f1d 0 2024-04-12T20:56:17.951 INFO:tasks.workunit.client.0.smithi018.stdout:3/756: getdents d2/d6/d34/da1/dac 0 2024-04-12T20:56:17.952 INFO:tasks.workunit.client.0.smithi018.stdout:1/622: dwrite d3/d9/da/dd/f3f [0,4194304] 0 2024-04-12T20:56:17.954 INFO:tasks.workunit.client.0.smithi018.stdout:5/803: symlink d0/d2/d33/d46/d1e/d55/l10f 0 2024-04-12T20:56:17.959 INFO:tasks.workunit.client.0.smithi018.stdout:7/710: truncate d1/db/d31/d45/d8c/d9d/d69/f7e 3869782 0 2024-04-12T20:56:17.960 INFO:tasks.workunit.client.0.smithi018.stdout:0/693: dwrite da/d15/d4d/f85 [0,4194304] 0 2024-04-12T20:56:17.960 INFO:tasks.workunit.client.0.smithi018.stdout:3/757: mknod d2/d6/d30/d31/d38/d3a/d76/da2/cee 0 2024-04-12T20:56:17.964 INFO:tasks.workunit.client.0.smithi018.stdout:1/623: mknod d3/d9/da/d14/cd9 0 2024-04-12T20:56:17.964 INFO:tasks.workunit.client.0.smithi018.stdout:5/804: creat d0/d2/de4/f110 x:0 0 0 2024-04-12T20:56:17.969 INFO:tasks.workunit.client.0.smithi018.stdout:1/624: dread - d3/d9/faf zero size 2024-04-12T20:56:17.971 INFO:tasks.workunit.client.0.smithi018.stdout:5/805: read d0/d2/d33/d10/d60/dfa/dbf/fe6 [226847,112311] 0 2024-04-12T20:56:17.973 INFO:tasks.workunit.client.0.smithi018.stdout:3/758: creat d2/d43/d61/d64/dbf/d70/fef x:0 0 0 2024-04-12T20:56:17.977 INFO:tasks.workunit.client.0.smithi018.stdout:3/759: rmdir d2/d43/d61/d64/dd9 39 2024-04-12T20:56:17.980 INFO:tasks.workunit.client.0.smithi018.stdout:1/625: truncate d3/d9/da/dd/d31/d32/d33/d47/fb2 877025 0 2024-04-12T20:56:17.985 INFO:tasks.workunit.client.0.smithi018.stdout:1/626: fsync d3/d9/da/dd/d26/fa6 0 2024-04-12T20:56:17.989 INFO:tasks.workunit.client.0.smithi018.stdout:7/711: dwrite d1/db/d31/d3a/d42/d4c/dbc/fde [4194304,4194304] 0 2024-04-12T20:56:17.998 INFO:tasks.workunit.client.0.smithi018.stdout:9/885: dread d3/d6/d9/dc/f2d [0,4194304] 0 2024-04-12T20:56:18.000 INFO:tasks.workunit.client.0.smithi018.stdout:1/627: rmdir d3/d9/da/dd/d31/d32 39 2024-04-12T20:56:18.007 INFO:tasks.workunit.client.0.smithi018.stdout:7/712: symlink d1/db/d31/d45/d8c/d9d/d70/d96/d5f/da9/le8 0 2024-04-12T20:56:18.010 INFO:tasks.workunit.client.0.smithi018.stdout:9/886: truncate d3/d6/d9/dc/d5a/d79/d7e/d9b/d21/f59 5204937 0 2024-04-12T20:56:18.015 INFO:tasks.workunit.client.0.smithi018.stdout:0/694: write da/f1d [4631445,93435] 0 2024-04-12T20:56:18.032 INFO:tasks.workunit.client.0.smithi018.stdout:9/887: rename d3/d6/d9/dc/d6f to d3/d109 0 2024-04-12T20:56:18.035 INFO:tasks.workunit.client.0.smithi018.stdout:1/628: dwrite d3/d9/d11/fa5 [0,4194304] 0 2024-04-12T20:56:18.039 INFO:tasks.workunit.client.0.smithi018.stdout:3/760: dwrite d2/d6/d34/d71/d7b/f91 [0,4194304] 0 2024-04-12T20:56:18.041 INFO:tasks.workunit.client.0.smithi018.stdout:1/629: dread - d3/d9/da/fae zero size 2024-04-12T20:56:18.042 INFO:tasks.workunit.client.0.smithi018.stdout:9/888: write d3/d6/d9/dc/d5a/d79/d7e/d9b/d21/dc3/dd8/d5b/d6b/da7/de7/fec [98856,111557] 0 2024-04-12T20:56:18.052 INFO:tasks.workunit.client.0.smithi018.stdout:5/806: dread d0/d2/d33/d46/d18/f71 [0,4194304] 0 2024-04-12T20:56:18.061 INFO:tasks.workunit.client.0.smithi018.stdout:3/761: creat d2/d43/d61/d64/ff0 x:0 0 0 2024-04-12T20:56:18.062 INFO:tasks.workunit.client.0.smithi018.stdout:1/630: rename d3/d9/da/d14/l42 to d3/d9/da/dd/d31/d44/lda 0 2024-04-12T20:56:18.064 INFO:tasks.workunit.client.0.smithi018.stdout:3/762: chown d2/d6/d34/da1/ddb 3020907 1 2024-04-12T20:56:18.072 INFO:tasks.workunit.client.0.smithi018.stdout:7/713: dread d1/fba [4194304,4194304] 0 2024-04-12T20:56:18.077 INFO:tasks.workunit.client.0.smithi018.stdout:5/807: rename d0/d5b/c10a to d0/d2b/d40/c111 0 2024-04-12T20:56:18.082 INFO:tasks.workunit.client.0.smithi018.stdout:0/695: dread da/d15/d17/d21/f68 [0,4194304] 0 2024-04-12T20:56:18.083 INFO:tasks.workunit.client.0.smithi018.stdout:1/631: rmdir d3/d9/da/dd/d31/d32/d13/d19 39 2024-04-12T20:56:18.085 INFO:tasks.workunit.client.0.smithi018.stdout:9/889: write d3/d93/fbc [241030,120074] 0 2024-04-12T20:56:18.089 INFO:tasks.workunit.client.0.smithi018.stdout:7/714: getdents d1/d6/dc9 0 2024-04-12T20:56:18.089 INFO:tasks.workunit.client.0.smithi018.stdout:9/890: stat d3/d6/d9/dc/d5a/d79/d7e/d9b/d21/dc3/dd8/db9/cc0 0 2024-04-12T20:56:18.092 INFO:tasks.workunit.client.0.smithi018.stdout:3/763: getdents d2/d6/d34/d71 0 2024-04-12T20:56:18.095 INFO:tasks.workunit.client.0.smithi018.stdout:0/696: symlink da/d15/d17/d21/d36/da2/dc3/d83/le7 0 2024-04-12T20:56:18.095 INFO:tasks.workunit.client.0.smithi018.stdout:7/715: write d1/db/d31/d45/d8c/d9d/d70/f95 [513214,98104] 0 2024-04-12T20:56:18.097 INFO:tasks.workunit.client.0.smithi018.stdout:9/891: creat d3/d6/d9/f10a x:0 0 0 2024-04-12T20:56:18.098 INFO:tasks.workunit.client.0.smithi018.stdout:7/716: write d1/d6/fa0 [180992,12568] 0 2024-04-12T20:56:18.102 INFO:tasks.workunit.client.0.smithi018.stdout:9/892: mknod d3/d6/d9/d1e/c10b 0 2024-04-12T20:56:18.106 INFO:tasks.workunit.client.0.smithi018.stdout:1/632: dread d3/d9/da/dd/d31/d32/d13/d19/f6e [0,4194304] 0 2024-04-12T20:56:18.110 INFO:tasks.workunit.client.0.smithi018.stdout:3/764: rename d2/d43/d61/d64/dbf/d70/cdc to d2/d4d/d88/cf1 0 2024-04-12T20:56:18.114 INFO:tasks.workunit.client.0.smithi018.stdout:7/717: dread d1/d6/f67 [0,4194304] 0 2024-04-12T20:56:18.118 INFO:tasks.workunit.client.0.smithi018.stdout:3/765: truncate d2/d6/d34/d71/dbc/dcc/fd1 8248 0 2024-04-12T20:56:18.120 INFO:tasks.workunit.client.0.smithi018.stdout:9/893: rename d3/d6/d9/dc/d5a/d79/d7e/d9b/d21/dc3/dd8/d5b/d6b/lc5 to d3/d6/d9/dc/d5a/d79/l10c 0 2024-04-12T20:56:18.125 INFO:tasks.workunit.client.0.smithi018.stdout:7/718: dread d1/db/d31/d3a/d42/dcd/fd2 [0,4194304] 0 2024-04-12T20:56:18.126 INFO:tasks.workunit.client.0.smithi018.stdout:7/719: chown d1 631264 1 2024-04-12T20:56:18.130 INFO:tasks.workunit.client.0.smithi018.stdout:1/633: dread d3/d9/da/f4c [0,4194304] 0 2024-04-12T20:56:18.133 INFO:tasks.workunit.client.0.smithi018.stdout:7/720: mknod d1/db/d31/d3a/d42/d4c/ce9 0 2024-04-12T20:56:18.134 INFO:tasks.workunit.client.0.smithi018.stdout:3/766: write d2/d43/d61/f62 [2948198,33759] 0 2024-04-12T20:56:18.136 INFO:tasks.workunit.client.0.smithi018.stdout:3/767: readlink d2/d6/d34/lbb 0 2024-04-12T20:56:18.140 INFO:tasks.workunit.client.0.smithi018.stdout:1/634: getdents d3/d9/da 0 2024-04-12T20:56:18.143 INFO:tasks.workunit.client.0.smithi018.stdout:9/894: write d3/d6/d9/dc/d5a/d79/d7e/d9b/d21/dc2/fcd [248753,66727] 0 2024-04-12T20:56:18.143 INFO:tasks.workunit.client.0.smithi018.stdout:1/635: chown d3/d9/da/dd/d31/d32/d13/d19/d2d/d48 2301 1 2024-04-12T20:56:18.144 INFO:tasks.workunit.client.0.smithi018.stdout:3/768: mknod d2/d6/d34/da1/dac/cf2 0 2024-04-12T20:56:18.146 INFO:tasks.workunit.client.0.smithi018.stdout:9/895: dread - d3/d6/d43/fd4 zero size 2024-04-12T20:56:18.147 INFO:tasks.workunit.client.0.smithi018.stdout:3/769: mknod d2/d43/cf3 0 2024-04-12T20:56:18.151 INFO:tasks.workunit.client.0.smithi018.stdout:9/896: fsync d3/d6/d9/dc/d5a/d79/d7e/d9b/d21/d41/fef 0 2024-04-12T20:56:18.152 INFO:tasks.workunit.client.0.smithi018.stdout:3/770: creat d2/d6/d34/d71/dbc/ff4 x:0 0 0 2024-04-12T20:56:18.162 INFO:tasks.workunit.client.0.smithi018.stdout:3/771: truncate d2/d6/d30/d31/d38/f66 49313 0 2024-04-12T20:56:18.163 INFO:tasks.workunit.client.0.smithi018.stdout:1/636: dwrite d3/d9/da/dd/d31/d32/f41 [0,4194304] 0 2024-04-12T20:56:18.170 INFO:tasks.workunit.client.0.smithi018.stdout:9/897: dwrite d3/d6/d43/f8f [4194304,4194304] 0 2024-04-12T20:56:18.171 INFO:tasks.workunit.client.0.smithi018.stdout:1/637: write d3/d9/faf [1019760,101778] 0 2024-04-12T20:56:18.175 INFO:tasks.workunit.client.0.smithi018.stdout:7/721: dread d1/db/d31/d45/d53/f56 [0,4194304] 0 2024-04-12T20:56:18.175 INFO:tasks.workunit.client.0.smithi018.stdout:3/772: rename d2/d6/f1c to d2/d6/d30/d31/d38/d3a/d76/ff5 0 2024-04-12T20:56:18.179 INFO:tasks.workunit.client.0.smithi018.stdout:9/898: write d3/d6/d9/dc/d5a/d79/d7e/d9b/d21/dc2/fcd [86151,130123] 0 2024-04-12T20:56:18.190 INFO:tasks.workunit.client.0.smithi018.stdout:3/773: rename d2/d6/d34/d71/dbc/fcd to d2/d6/d34/da1/ddb/ff6 0 2024-04-12T20:56:18.191 INFO:tasks.workunit.client.0.smithi018.stdout:1/638: link d3/d9/da/dd/d31/d32/d13/d19/d2d/f4d d3/d9/da/d14/fdb 0 2024-04-12T20:56:18.198 INFO:tasks.workunit.client.0.smithi018.stdout:1/639: unlink d3/d9/da/dd/d31/d32/d13/d19/d2d/d48/c83 0 2024-04-12T20:56:18.198 INFO:tasks.workunit.client.0.smithi018.stdout:9/899: dwrite d3/d6/fda [0,4194304] 0 2024-04-12T20:56:18.199 INFO:tasks.workunit.client.0.smithi018.stdout:9/900: write d3/d6/d9/dc/d46/d5d/ffc [1480858,9467] 0 2024-04-12T20:56:18.254 INFO:tasks.workunit.client.0.smithi018.stdout:6/874: sync 2024-04-12T20:56:18.256 INFO:tasks.workunit.client.0.smithi018.stdout:1/640: truncate d3/d9/da/dd/f3f 1301929 0 2024-04-12T20:56:18.260 INFO:tasks.workunit.client.0.smithi018.stdout:6/875: dread - d5/d6/d1b/d3b/d8e/fdd zero size 2024-04-12T20:56:18.266 INFO:tasks.workunit.client.0.smithi018.stdout:7/722: dwrite d1/db/d31/d45/f9a [0,4194304] 0 2024-04-12T20:56:18.266 INFO:tasks.workunit.client.0.smithi018.stdout:6/876: getdents d5/d6/d1b/d3b/d6a/d116/d13f 0 2024-04-12T20:56:18.271 INFO:tasks.workunit.client.0.smithi018.stdout:6/877: truncate d5/f10 3560217 0 2024-04-12T20:56:18.280 INFO:tasks.workunit.client.0.smithi018.stdout:1/641: dwrite d3/d9/da/d14/d73/f90 [0,4194304] 0 2024-04-12T20:56:18.283 INFO:tasks.workunit.client.0.smithi018.stdout:7/723: dread d1/db/f9f [0,4194304] 0 2024-04-12T20:56:18.285 INFO:tasks.workunit.client.0.smithi018.stdout:9/901: dread d3/d6/d9/dc/d5a/d79/d7e/d9b/d21/dc3/dd8/fa9 [0,4194304] 0 2024-04-12T20:56:18.290 INFO:tasks.workunit.client.0.smithi018.stdout:7/724: read d1/db/d31/d45/fc2 [2591676,24615] 0 2024-04-12T20:56:18.290 INFO:tasks.workunit.client.0.smithi018.stdout:1/642: fsync d3/d9/da/dd/f9d 0 2024-04-12T20:56:18.294 INFO:tasks.workunit.client.0.smithi018.stdout:7/725: write d1/db/d31/d45/d8c/d9d/d69/fd7 [691587,5978] 0 2024-04-12T20:56:18.295 INFO:tasks.workunit.client.0.smithi018.stdout:9/902: symlink d3/d6/d9/dc/d5a/d79/d7e/d9b/d21/d41/d78/l10d 0 2024-04-12T20:56:18.299 INFO:tasks.workunit.client.0.smithi018.stdout:1/643: unlink d3/d9/da/dd/d31/d32/d13/d19/d2d/d48/d66/l95 0 2024-04-12T20:56:18.299 INFO:tasks.workunit.client.0.smithi018.stdout:9/903: stat d3/d6/d9/dc/d5a/d79/d7e/d9b/d21/dc3/dd8/l68 0 2024-04-12T20:56:18.301 INFO:tasks.workunit.client.0.smithi018.stdout:7/726: rename d1/c77 to d1/db/d31/d45/d8c/d9d/d70/d96/cea 0 2024-04-12T20:56:18.306 INFO:tasks.workunit.client.0.smithi018.stdout:9/904: symlink d3/d6/d9/dc/d5a/d79/db5/l10e 0 2024-04-12T20:56:18.310 INFO:tasks.workunit.client.0.smithi018.stdout:9/905: mknod d3/d6/d3d/dba/c10f 0 2024-04-12T20:56:18.311 INFO:tasks.workunit.client.0.smithi018.stdout:1/644: creat d3/d9/da/dd/d31/d32/d13/d19/d2d/d4e/db9/fdc x:0 0 0 2024-04-12T20:56:18.311 INFO:tasks.workunit.client.0.smithi018.stdout:7/727: symlink d1/db/d31/d45/leb 0 2024-04-12T20:56:18.314 INFO:tasks.workunit.client.0.smithi018.stdout:7/728: rmdir d1/db/d31/d3a 39 2024-04-12T20:56:18.315 INFO:tasks.workunit.client.0.smithi018.stdout:1/645: chown d3/d9/da/dd/d31/d32/d33/d80/c8e 128 1 2024-04-12T20:56:18.317 INFO:tasks.workunit.client.0.smithi018.stdout:7/729: write d1/d6/f52 [2334652,67731] 0 2024-04-12T20:56:18.318 INFO:tasks.workunit.client.0.smithi018.stdout:9/906: symlink d3/d6/d9/dc/d5a/d79/d7e/d9b/d21/d41/d78/l110 0 2024-04-12T20:56:18.320 INFO:tasks.workunit.client.0.smithi018.stdout:7/730: creat d1/d6/dc9/fec x:0 0 0 2024-04-12T20:56:18.321 INFO:tasks.workunit.client.0.smithi018.stdout:9/907: write d3/d93/fbc [710191,56732] 0 2024-04-12T20:56:18.322 INFO:tasks.workunit.client.0.smithi018.stdout:1/646: mknod d3/d9/da/dd/d31/d32/d13/d19/d2d/d4e/d7a/d82/dce/cdd 0 2024-04-12T20:56:18.323 INFO:tasks.workunit.client.0.smithi018.stdout:7/731: readlink d1/db/d31/d45/d8c/d9d/d70/l78 0 2024-04-12T20:56:18.324 INFO:tasks.workunit.client.0.smithi018.stdout:1/647: write d3/d9/da/f20 [1277501,109986] 0 2024-04-12T20:56:18.346 INFO:tasks.workunit.client.0.smithi018.stdout:7/732: dwrite d1/db/d31/d3a/d97/fb1 [0,4194304] 0 2024-04-12T20:56:18.347 INFO:tasks.workunit.client.0.smithi018.stdout:9/908: dread d3/d6/d9/d1e/f71 [0,4194304] 0 2024-04-12T20:56:18.352 INFO:tasks.workunit.client.0.smithi018.stdout:9/909: creat d3/d6/d9/dc/d5a/d79/d7e/d9b/d21/dc3/dd8/db9/f111 x:0 0 0 2024-04-12T20:56:18.357 INFO:tasks.workunit.client.0.smithi018.stdout:9/910: mknod d3/d6/d9/dc/d5a/d79/c112 0 2024-04-12T20:56:18.358 INFO:tasks.workunit.client.0.smithi018.stdout:9/911: readlink d3/d6/d1b/l27 0 2024-04-12T20:56:18.362 INFO:tasks.workunit.client.0.smithi018.stdout:9/912: symlink d3/d6/d9/d1e/l113 0 2024-04-12T20:56:18.363 INFO:tasks.workunit.client.0.smithi018.stdout:7/733: dwrite d1/db/d31/d45/d53/f7d [0,4194304] 0 2024-04-12T20:56:18.365 INFO:tasks.workunit.client.0.smithi018.stdout:7/734: readlink d1/db/d31/d30/l50 0 2024-04-12T20:56:18.366 INFO:tasks.workunit.client.0.smithi018.stdout:9/913: creat d3/d6/d3d/dba/f114 x:0 0 0 2024-04-12T20:56:18.371 INFO:tasks.workunit.client.0.smithi018.stdout:9/914: dread - d3/d6/d9/dc/d5a/d79/d7e/d9b/d21/dc3/dd8/db9/f111 zero size 2024-04-12T20:56:18.377 INFO:tasks.workunit.client.0.smithi018.stdout:9/915: getdents d3/d6/d9/dc 0 2024-04-12T20:56:18.391 INFO:tasks.workunit.client.0.smithi018.stdout:7/735: dwrite d1/db/d31/d45/d8c/d9d/d70/d96/d5f/fcc [0,4194304] 0 2024-04-12T20:56:18.408 INFO:tasks.workunit.client.0.smithi018.stdout:7/736: dwrite d1/d6/f60 [0,4194304] 0 2024-04-12T20:56:18.415 INFO:tasks.workunit.client.0.smithi018.stdout:7/737: write d1/db/d31/d3a/f85 [554491,877] 0 2024-04-12T20:56:18.418 INFO:tasks.workunit.client.0.smithi018.stdout:9/916: dwrite d3/d6/d9/d1e/f40 [0,4194304] 0 2024-04-12T20:56:18.423 INFO:tasks.workunit.client.0.smithi018.stdout:9/917: read d3/d6/d9/fd [985773,119984] 0 2024-04-12T20:56:18.425 INFO:tasks.workunit.client.0.smithi018.stdout:9/918: mknod d3/dc8/c115 0 2024-04-12T20:56:18.436 INFO:tasks.workunit.client.0.smithi018.stdout:9/919: link d3/d6/d43/l85 d3/d6/d9/dc/d5a/d79/d7e/d9b/d21/d41/l116 0 2024-04-12T20:56:18.455 INFO:tasks.workunit.client.0.smithi018.stdout:9/920: dread d3/d6/d3d/dba/fc9 [0,4194304] 0 2024-04-12T20:56:18.461 INFO:tasks.workunit.client.0.smithi018.stdout:9/921: link d3/d6/d9/f10a d3/d6/d9/dc/d46/d5d/f117 0 2024-04-12T20:56:18.465 INFO:tasks.workunit.client.0.smithi018.stdout:9/922: mknod d3/d6/d9/dc/c118 0 2024-04-12T20:56:18.467 INFO:tasks.workunit.client.0.smithi018.stdout:2/707: sync 2024-04-12T20:56:18.472 INFO:tasks.workunit.client.0.smithi018.stdout:2/708: dread d5/d8/d10/d43/de2/d24/d39/da4/fbb [0,4194304] 0 2024-04-12T20:56:18.477 INFO:tasks.workunit.client.0.smithi018.stdout:9/923: link d3/d109/f9a d3/d6/d9/dc/d5a/d79/d7e/d9b/d21/dc3/dd8/d5b/f119 0 2024-04-12T20:56:18.483 INFO:tasks.workunit.client.0.smithi018.stdout:9/924: creat d3/d6/d9/dc/d5a/d79/d7e/f11a x:0 0 0 2024-04-12T20:56:18.487 INFO:tasks.workunit.client.0.smithi018.stdout:2/709: dwrite d5/d8/d10/d43/fd8 [0,4194304] 0 2024-04-12T20:56:18.488 INFO:tasks.workunit.client.0.smithi018.stdout:2/710: readlink d5/d7/d31/l99 0 2024-04-12T20:56:18.491 INFO:tasks.workunit.client.0.smithi018.stdout:9/925: creat d3/d6/d43/f11b x:0 0 0 2024-04-12T20:56:18.491 INFO:tasks.workunit.client.0.smithi018.stdout:2/711: dread - d5/d8/d10/d43/de2/d24/d39/da4/fde zero size 2024-04-12T20:56:18.493 INFO:tasks.workunit.client.0.smithi018.stdout:9/926: fdatasync d3/d6/d1b/fb8 0 2024-04-12T20:56:18.498 INFO:tasks.workunit.client.0.smithi018.stdout:2/712: creat d5/d8/d10/d43/de2/d7c/ff6 x:0 0 0 2024-04-12T20:56:18.498 INFO:tasks.workunit.client.0.smithi018.stdout:9/927: symlink d3/d109/l11c 0 2024-04-12T20:56:18.506 INFO:tasks.workunit.client.0.smithi018.stdout:2/713: link d5/d7/d31/d62/fb8 d5/d7/d31/ff7 0 2024-04-12T20:56:18.509 INFO:tasks.workunit.client.0.smithi018.stdout:2/714: fdatasync d5/d7/d79/f7e 0 2024-04-12T20:56:18.516 INFO:tasks.workunit.client.0.smithi018.stdout:2/715: write d5/d7/d31/f7a [131978,97128] 0 2024-04-12T20:56:18.528 INFO:tasks.workunit.client.0.smithi018.stdout:2/716: dread d5/d8/d33/f9a [0,4194304] 0 2024-04-12T20:56:18.531 INFO:tasks.workunit.client.0.smithi018.stdout:2/717: write d5/d12/d54/d3e/fcb [5314,9598] 0 2024-04-12T20:56:18.610 INFO:tasks.workunit.client.0.smithi018.stdout:2/718: truncate d5/d8/d10/d43/de2/daf/d88/fb2 507029 0 2024-04-12T20:56:18.613 INFO:tasks.workunit.client.0.smithi018.stdout:9/928: dwrite d3/d6/d9/d1e/f71 [0,4194304] 0 2024-04-12T20:56:18.615 INFO:tasks.workunit.client.0.smithi018.stdout:9/929: write d3/d6/d43/f11b [546138,99854] 0 2024-04-12T20:56:18.616 INFO:tasks.workunit.client.0.smithi018.stdout:2/719: truncate d5/d8/d10/d43/de2/daf/d90/f9d 72945 0 2024-04-12T20:56:18.620 INFO:tasks.workunit.client.0.smithi018.stdout:2/720: mkdir d5/d7/d31/d62/da6/df8 0 2024-04-12T20:56:18.622 INFO:tasks.workunit.client.0.smithi018.stdout:9/930: mkdir d3/d6/d9/dc/d5a/d79/dac/d11d 0 2024-04-12T20:56:18.624 INFO:tasks.workunit.client.0.smithi018.stdout:2/721: creat d5/d7/d31/d62/da6/df8/ff9 x:0 0 0 2024-04-12T20:56:18.625 INFO:tasks.workunit.client.0.smithi018.stdout:9/931: mknod d3/d6/d9/c11e 0 2024-04-12T20:56:18.627 INFO:tasks.workunit.client.0.smithi018.stdout:2/722: write d5/d8/d33/f76 [1400885,61729] 0 2024-04-12T20:56:18.634 INFO:tasks.workunit.client.0.smithi018.stdout:9/932: unlink d3/d6/d9/dc/d5a/d79/d7e/d9b/d21/d41/c5f 0 2024-04-12T20:56:18.647 INFO:tasks.workunit.client.0.smithi018.stdout:9/933: rename d3/d6/d9/dc/d46/d5d/cb0 to d3/d6/d9/dc/d5a/d79/d7e/d9b/d21/dc3/dd8/d5b/d7c/dea/c11f 0 2024-04-12T20:56:18.647 INFO:tasks.workunit.client.0.smithi018.stdout:2/723: write d5/d7/d31/ff7 [5910982,13409] 0 2024-04-12T20:56:18.650 INFO:tasks.workunit.client.0.smithi018.stdout:2/724: fdatasync d5/d8/d10/d43/de2/d24/d39/da4/fde 0 2024-04-12T20:56:18.651 INFO:tasks.workunit.client.0.smithi018.stdout:9/934: creat d3/d6/d9/dc/d46/f120 x:0 0 0 2024-04-12T20:56:18.654 INFO:tasks.workunit.client.0.smithi018.stdout:9/935: chown d3/d6/d9/dc/d5a/d79/d7e/d9b/d21/l55 370 1 2024-04-12T20:56:18.656 INFO:tasks.workunit.client.0.smithi018.stdout:2/725: mkdir d5/d12/d54/dfa 0 2024-04-12T20:56:18.660 INFO:tasks.workunit.client.0.smithi018.stdout:2/726: creat d5/d7/ffb x:0 0 0 2024-04-12T20:56:18.665 INFO:tasks.workunit.client.0.smithi018.stdout:2/727: truncate d5/d8/d10/d43/de2/d24/d39/da4/fbb 1047762 0 2024-04-12T20:56:18.671 INFO:tasks.workunit.client.0.smithi018.stdout:8/913: sync 2024-04-12T20:56:18.684 INFO:tasks.workunit.client.0.smithi018.stdout:2/728: dwrite d5/d8/f1e [0,4194304] 0 2024-04-12T20:56:18.689 INFO:tasks.workunit.client.0.smithi018.stdout:2/729: chown d5/d8/d10/d43/de2/d7c/f9f 230471862 1 2024-04-12T20:56:18.692 INFO:tasks.workunit.client.0.smithi018.stdout:2/730: write d5/d12/d54/f53 [1782670,65969] 0 2024-04-12T20:56:18.697 INFO:tasks.workunit.client.0.smithi018.stdout:2/731: mkdir d5/d12/d54/dc0/dfc 0 2024-04-12T20:56:18.717 INFO:tasks.workunit.client.0.smithi018.stdout:2/732: dwrite d5/d7/f4d [0,4194304] 0 2024-04-12T20:56:18.719 INFO:tasks.workunit.client.0.smithi018.stdout:2/733: chown d5/d7/d31/f64 9 1 2024-04-12T20:56:18.725 INFO:tasks.workunit.client.0.smithi018.stdout:2/734: link d5/d7/d63/ff1 d5/d12/d54/dc0/ffd 0 2024-04-12T20:56:18.727 INFO:tasks.workunit.client.0.smithi018.stdout:9/936: truncate d3/d6/d9/dc/fb4 2712504 0 2024-04-12T20:56:18.739 INFO:tasks.workunit.client.0.smithi018.stdout:9/937: dread d3/d6/d9/d1e/f71 [0,4194304] 0 2024-04-12T20:56:18.748 INFO:tasks.workunit.client.0.smithi018.stdout:2/735: dread d5/d7/d31/f4f [0,4194304] 0 2024-04-12T20:56:18.750 INFO:tasks.workunit.client.0.smithi018.stdout:2/736: creat d5/d7/d31/d62/da6/ffe x:0 0 0 2024-04-12T20:56:18.754 INFO:tasks.workunit.client.0.smithi018.stdout:2/737: dread - d5/d8/d10/d43/de2/d24/d39/fb3 zero size 2024-04-12T20:56:18.764 INFO:tasks.workunit.client.0.smithi018.stdout:2/738: write d5/d8/d10/d43/de2/daf/d88/fb2 [802342,78591] 0 2024-04-12T20:56:18.766 INFO:tasks.workunit.client.0.smithi018.stdout:2/739: chown d5/d8/d10/d43/l8d 1717423 1 2024-04-12T20:56:18.773 INFO:tasks.workunit.client.0.smithi018.stdout:2/740: link d5/d8/d33/l5e d5/d7/lff 0 2024-04-12T20:56:18.774 INFO:tasks.workunit.client.0.smithi018.stdout:9/938: dread d3/d6/d9/dc/d5a/d79/d7e/d9b/d21/f59 [0,4194304] 0 2024-04-12T20:56:18.775 INFO:tasks.workunit.client.0.smithi018.stdout:9/939: chown d3/d109/l11c 4033592 1 2024-04-12T20:56:18.776 INFO:tasks.workunit.client.0.smithi018.stdout:2/741: creat d5/d7/d31/d62/da6/f100 x:0 0 0 2024-04-12T20:56:18.792 INFO:tasks.workunit.client.0.smithi018.stdout:9/940: dwrite d3/d6/d9/d1e/f40 [0,4194304] 0 2024-04-12T20:56:18.793 INFO:tasks.workunit.client.0.smithi018.stdout:4/726: sync 2024-04-12T20:56:18.798 INFO:tasks.workunit.client.0.smithi018.stdout:4/727: chown d3/d5d/d60/dc5/d43/d4b/de6 10690149 1 2024-04-12T20:56:18.799 INFO:tasks.workunit.client.0.smithi018.stdout:9/941: dread d3/d6/d43/f11b [0,4194304] 0 2024-04-12T20:56:18.800 INFO:tasks.workunit.client.0.smithi018.stdout:2/742: dread d5/d8/d10/f78 [0,4194304] 0 2024-04-12T20:56:18.805 INFO:tasks.workunit.client.0.smithi018.stdout:4/728: creat d3/d5d/d60/dc5/d53/de7/fef x:0 0 0 2024-04-12T20:56:18.811 INFO:tasks.workunit.client.0.smithi018.stdout:9/942: rename d3/d6/d9/dc/d5a/d79/d7e/d9b/d21/d41/d78/f98 to d3/d6/d9/dc/d5a/f121 0 2024-04-12T20:56:18.811 INFO:tasks.workunit.client.0.smithi018.stdout:4/729: creat d3/d5d/d60/ff0 x:0 0 0 2024-04-12T20:56:18.818 INFO:tasks.workunit.client.0.smithi018.stdout:2/743: dwrite d5/d8/d25/f7d [0,4194304] 0 2024-04-12T20:56:18.823 INFO:tasks.workunit.client.0.smithi018.stdout:1/648: sync 2024-04-12T20:56:18.823 INFO:tasks.workunit.client.0.smithi018.stdout:5/808: sync 2024-04-12T20:56:18.823 INFO:tasks.workunit.client.0.smithi018.stdout:0/697: sync 2024-04-12T20:56:18.823 INFO:tasks.workunit.client.0.smithi018.stdout:3/774: sync 2024-04-12T20:56:18.824 INFO:tasks.workunit.client.0.smithi018.stdout:6/878: sync 2024-04-12T20:56:18.824 INFO:tasks.workunit.client.0.smithi018.stdout:7/738: sync 2024-04-12T20:56:18.824 INFO:tasks.workunit.client.0.smithi018.stdout:8/914: sync 2024-04-12T20:56:18.834 INFO:tasks.workunit.client.0.smithi018.stdout:9/943: dread d3/d6/d9/dc/d46/d5d/ffa [0,4194304] 0 2024-04-12T20:56:18.838 INFO:tasks.workunit.client.0.smithi018.stdout:9/944: dread - d3/d6/d9/dc/d5a/d79/d7e/f103 zero size 2024-04-12T20:56:18.838 INFO:tasks.workunit.client.0.smithi018.stdout:6/879: creat d5/d6/d1b/d8c/dc0/f14e x:0 0 0 2024-04-12T20:56:18.845 INFO:tasks.workunit.client.0.smithi018.stdout:2/744: creat d5/d8/d10/d43/de2/daf/d90/f101 x:0 0 0 2024-04-12T20:56:18.850 INFO:tasks.workunit.client.0.smithi018.stdout:6/880: getdents d5/df/d3f/dca 0 2024-04-12T20:56:18.852 INFO:tasks.workunit.client.0.smithi018.stdout:5/809: truncate d0/d2/d33/d46/d1e/f6c 1673811 0 2024-04-12T20:56:18.855 INFO:tasks.workunit.client.0.smithi018.stdout:2/745: chown d5/d8/d10/d43/de2/daf/d88/fb2 4210 1 2024-04-12T20:56:18.863 INFO:tasks.workunit.client.0.smithi018.stdout:5/810: chown d0/d2/d33/d46/d35/fcc 63125 1 2024-04-12T20:56:18.863 INFO:tasks.workunit.client.0.smithi018.stdout:1/649: creat d3/d9/da/dd/d31/d32/d13/d19/d2d/d4e/fde x:0 0 0 2024-04-12T20:56:18.868 INFO:tasks.workunit.client.0.smithi018.stdout:2/746: creat d5/d12/d54/d3e/f102 x:0 0 0 2024-04-12T20:56:18.869 INFO:tasks.workunit.client.0.smithi018.stdout:1/650: chown d3/db8 671 1 2024-04-12T20:56:18.876 INFO:tasks.workunit.client.0.smithi018.stdout:9/945: dwrite d3/d6/d9/dc/d5a/d79/d7e/f103 [0,4194304] 0 2024-04-12T20:56:18.877 INFO:tasks.workunit.client.0.smithi018.stdout:6/881: fsync d5/d6/d1b/d3b/fdc 0 2024-04-12T20:56:18.878 INFO:tasks.workunit.client.0.smithi018.stdout:4/730: dread d3/d5d/d60/dc5/d43/f7b [0,4194304] 0 2024-04-12T20:56:18.879 INFO:tasks.workunit.client.0.smithi018.stdout:5/811: creat d0/d2/d33/d46/d18/f112 x:0 0 0 2024-04-12T20:56:18.880 INFO:tasks.workunit.client.0.smithi018.stdout:0/698: dwrite da/d15/d17/d21/d36/f4c [0,4194304] 0 2024-04-12T20:56:18.880 INFO:tasks.workunit.client.0.smithi018.stdout:2/747: readlink d5/d8/d10/d43/de2/d24/d39/da4/lb7 0 2024-04-12T20:56:18.881 INFO:tasks.workunit.client.0.smithi018.stdout:1/651: chown d3/d9/da/dd/d31/f36 54321077 1 2024-04-12T20:56:18.882 INFO:tasks.workunit.client.0.smithi018.stdout:7/739: write d1/db/d31/d45/d8c/d9d/d69/f7e [4622582,82866] 0 2024-04-12T20:56:18.883 INFO:tasks.workunit.client.0.smithi018.stdout:6/882: chown d5/d71/l89 1599328 1 2024-04-12T20:56:18.886 INFO:tasks.workunit.client.0.smithi018.stdout:8/915: dread db/d4e/d56/d59/d76/da5/d45/f79 [0,4194304] 0 2024-04-12T20:56:18.887 INFO:tasks.workunit.client.0.smithi018.stdout:5/812: rmdir d0/d2/d33/d46/d1e/d55 39 2024-04-12T20:56:18.891 INFO:tasks.workunit.client.0.smithi018.stdout:6/883: readlink d5/d6/l1d 0 2024-04-12T20:56:18.892 INFO:tasks.workunit.client.0.smithi018.stdout:0/699: chown da/d15/d17/d21/d36/da2/dc3/d3f/lbd 2537 1 2024-04-12T20:56:18.892 INFO:tasks.workunit.client.0.smithi018.stdout:7/740: creat d1/db/d31/d45/d53/fed x:0 0 0 2024-04-12T20:56:18.895 INFO:tasks.workunit.client.0.smithi018.stdout:5/813: chown d0/d2/d33/d10/d60/dfa/d8f 1740 1 2024-04-12T20:56:18.897 INFO:tasks.workunit.client.0.smithi018.stdout:4/731: write d3/d5d/d60/dc5/d43/fc9 [1351770,21682] 0 2024-04-12T20:56:18.902 INFO:tasks.workunit.client.0.smithi018.stdout:3/775: dwrite d2/d43/d61/d64/faa [0,4194304] 0 2024-04-12T20:56:18.905 INFO:tasks.workunit.client.0.smithi018.stdout:2/748: dread d5/d7/d31/f7a [0,4194304] 0 2024-04-12T20:56:18.905 INFO:tasks.workunit.client.0.smithi018.stdout:0/700: mkdir da/d15/de8 0 2024-04-12T20:56:18.905 INFO:tasks.workunit.client.0.smithi018.stdout:8/916: write db/d4e/d56/d59/d76/da5/d45/da9/ddb/f115 [406262,113181] 0 2024-04-12T20:56:18.914 INFO:tasks.workunit.client.0.smithi018.stdout:6/884: creat d5/d6/d1b/d3b/d6a/d116/d13f/f14f x:0 0 0 2024-04-12T20:56:18.915 INFO:tasks.workunit.client.0.smithi018.stdout:7/741: mknod d1/db/d31/d45/d8c/d9d/cee 0 2024-04-12T20:56:18.917 INFO:tasks.workunit.client.0.smithi018.stdout:0/701: creat da/d15/d4d/fe9 x:0 0 0 2024-04-12T20:56:18.926 INFO:tasks.workunit.client.0.smithi018.stdout:7/742: creat d1/db/fef x:0 0 0 2024-04-12T20:56:18.929 INFO:tasks.workunit.client.0.smithi018.stdout:2/749: symlink d5/d8/d10/d43/d8f/de9/l103 0 2024-04-12T20:56:18.929 INFO:tasks.workunit.client.0.smithi018.stdout:1/652: dwrite d3/d9/da/dd/d31/f35 [0,4194304] 0 2024-04-12T20:56:18.930 INFO:tasks.workunit.client.0.smithi018.stdout:8/917: write db/d4e/d56/d59/d76/de0/fa3 [74197,34176] 0 2024-04-12T20:56:18.930 INFO:tasks.workunit.client.0.smithi018.stdout:7/743: dread - d1/d6/fca zero size 2024-04-12T20:56:18.935 INFO:tasks.workunit.client.0.smithi018.stdout:0/702: truncate da/d15/d17/d21/d36/da2/dc3/d3f/fdd 939611 0 2024-04-12T20:56:18.945 INFO:tasks.workunit.client.0.smithi018.stdout:3/776: link d2/d6/d30/d31/l77 d2/d6/d30/d31/d38/d3a/dd0/de1/lf7 0 2024-04-12T20:56:18.946 INFO:tasks.workunit.client.0.smithi018.stdout:8/918: creat db/d4e/d73/df7/f126 x:0 0 0 2024-04-12T20:56:18.950 INFO:tasks.workunit.client.0.smithi018.stdout:9/946: truncate d3/d6/d9/dc/d5a/d79/d7e/d9b/d21/dc3/dd8/d5b/d7c/f69 2106455 0 2024-04-12T20:56:18.951 INFO:tasks.workunit.client.0.smithi018.stdout:0/703: mkdir da/d15/d17/db9/dea 0 2024-04-12T20:56:18.959 INFO:tasks.workunit.client.0.smithi018.stdout:6/885: dread d5/f37 [0,4194304] 0 2024-04-12T20:56:18.961 INFO:tasks.workunit.client.0.smithi018.stdout:3/777: dread d2/d6/d30/d31/d38/f44 [0,4194304] 0 2024-04-12T20:56:18.962 INFO:tasks.workunit.client.0.smithi018.stdout:6/886: chown d5/d6/d90/d125/d139 179655135 1 2024-04-12T20:56:18.965 INFO:tasks.workunit.client.0.smithi018.stdout:5/814: dwrite d0/d2/d11/f108 [4194304,4194304] 0 2024-04-12T20:56:18.966 INFO:tasks.workunit.client.0.smithi018.stdout:9/947: creat d3/d6/d9/dc/dab/f122 x:0 0 0 2024-04-12T20:56:18.970 INFO:tasks.workunit.client.0.smithi018.stdout:2/750: write d5/d7/d31/d62/f72 [1137153,24050] 0 2024-04-12T20:56:18.975 INFO:tasks.workunit.client.0.smithi018.stdout:1/653: dread d3/d9/da/dd/d31/f35 [0,4194304] 0 2024-04-12T20:56:18.976 INFO:tasks.workunit.client.0.smithi018.stdout:9/948: creat d3/d6/d9/dc/d46/f123 x:0 0 0 2024-04-12T20:56:18.977 INFO:tasks.workunit.client.0.smithi018.stdout:3/778: rename d2/d43/db6/db5/fe2 to d2/d6/d34/da1/ff8 0 2024-04-12T20:56:18.979 INFO:tasks.workunit.client.0.smithi018.stdout:8/919: write db/d3b/f55 [679601,47416] 0 2024-04-12T20:56:18.981 INFO:tasks.workunit.client.0.smithi018.stdout:6/887: truncate d5/d6/d1b/d3b/d22/d117/d13c/fff 4286419 0 2024-04-12T20:56:18.985 INFO:tasks.workunit.client.0.smithi018.stdout:9/949: symlink d3/d6/d9/dc/d5a/d79/dac/d11d/l124 0 2024-04-12T20:56:18.987 INFO:tasks.workunit.client.0.smithi018.stdout:3/779: unlink d2/d43/db6/db5/dd8/fc7 0 2024-04-12T20:56:18.989 INFO:tasks.workunit.client.0.smithi018.stdout:1/654: write d3/d9/da/dd/d31/d32/d13/d19/d2d/d4e/db9/fdc [358594,66690] 0 2024-04-12T20:56:18.990 INFO:tasks.workunit.client.0.smithi018.stdout:8/920: chown db/d4e/d56/d59/d76/da5/fb0 0 1 2024-04-12T20:56:18.997 INFO:tasks.workunit.client.0.smithi018.stdout:3/780: read d2/d43/d61/d64/faa [1071513,115347] 0 2024-04-12T20:56:18.998 INFO:tasks.workunit.client.0.smithi018.stdout:6/888: mkdir d5/df/d3f/dc1/d150 0 2024-04-12T20:56:18.999 INFO:tasks.workunit.client.0.smithi018.stdout:0/704: dwrite da/f52 [0,4194304] 0 2024-04-12T20:56:19.001 INFO:tasks.workunit.client.0.smithi018.stdout:7/744: dread d1/f32 [0,4194304] 0 2024-04-12T20:56:19.003 INFO:tasks.workunit.client.0.smithi018.stdout:8/921: fdatasync db/d4e/d56/d59/d76/da5/d45/d86/da1/fb6 0 2024-04-12T20:56:19.010 INFO:tasks.workunit.client.0.smithi018.stdout:6/889: mkdir d5/d6/d1b/d3b/d6a/d116/d104/d128/d151 0 2024-04-12T20:56:19.016 INFO:tasks.workunit.client.0.smithi018.stdout:7/745: symlink d1/db/d31/d3a/d42/dcd/lf0 0 2024-04-12T20:56:19.016 INFO:tasks.workunit.client.0.smithi018.stdout:3/781: creat d2/d6/ff9 x:0 0 0 2024-04-12T20:56:19.017 INFO:tasks.workunit.client.0.smithi018.stdout:8/922: fdatasync db/d4e/d56/d59/d76/da5/f1a 0 2024-04-12T20:56:19.018 INFO:tasks.workunit.client.0.smithi018.stdout:3/782: readlink d2/d6/d34/lbb 0 2024-04-12T20:56:19.018 INFO:tasks.workunit.client.0.smithi018.stdout:0/705: mknod da/d15/d17/d21/d36/da2/dc3/d3f/de3/ceb 0 2024-04-12T20:56:19.020 INFO:tasks.workunit.client.0.smithi018.stdout:9/950: rename d3/d6/d9/dc/d5a/d79/d7e/d9b/d21/dc3/dd8/d5b/d6b/laa to d3/d6/d9/dc/d5a/d79/dac/d11d/l125 0 2024-04-12T20:56:19.022 INFO:tasks.workunit.client.0.smithi018.stdout:8/923: write db/fff [3061679,8884] 0 2024-04-12T20:56:19.022 INFO:tasks.workunit.client.0.smithi018.stdout:7/746: symlink d1/db/d31/d45/d8c/d9d/d70/lf1 0 2024-04-12T20:56:19.022 INFO:tasks.workunit.client.0.smithi018.stdout:5/815: dread d0/d2/d33/d10/f39 [0,4194304] 0 2024-04-12T20:56:19.024 INFO:tasks.workunit.client.0.smithi018.stdout:3/783: fsync d2/d6/d30/d31/d38/d3a/d76/fa5 0 2024-04-12T20:56:19.027 INFO:tasks.workunit.client.0.smithi018.stdout:6/890: mknod d5/d6/d1b/c152 0 2024-04-12T20:56:19.028 INFO:tasks.workunit.client.0.smithi018.stdout:0/706: creat da/d15/d17/d21/d36/d47/d81/fec x:0 0 0 2024-04-12T20:56:19.028 INFO:tasks.workunit.client.0.smithi018.stdout:7/747: mkdir d1/db/d31/df2 0 2024-04-12T20:56:19.029 INFO:tasks.workunit.client.0.smithi018.stdout:3/784: chown d2/d6/d34/d71/d7b/f7d 46539 1 2024-04-12T20:56:19.030 INFO:tasks.workunit.client.0.smithi018.stdout:6/891: dread - d5/df/d19/f136 zero size 2024-04-12T20:56:19.031 INFO:tasks.workunit.client.0.smithi018.stdout:8/924: dread f0 [4194304,4194304] 0 2024-04-12T20:56:19.034 INFO:tasks.workunit.client.0.smithi018.stdout:5/816: read d0/d2/d33/d10/d26/dda/ff8 [3233512,20176] 0 2024-04-12T20:56:19.034 INFO:tasks.workunit.client.0.smithi018.stdout:0/707: chown da/d15/d17/d21/d36/d6e/fc9 2 1 2024-04-12T20:56:19.035 INFO:tasks.workunit.client.0.smithi018.stdout:6/892: chown d5/df/d19/c127 26688509 1 2024-04-12T20:56:19.036 INFO:tasks.workunit.client.0.smithi018.stdout:3/785: chown d2/d6/d34/d71/l9e 36696 1 2024-04-12T20:56:19.041 INFO:tasks.workunit.client.0.smithi018.stdout:3/786: mkdir d2/d43/d61/d64/dbf/d70/dfa 0 2024-04-12T20:56:19.044 INFO:tasks.workunit.client.0.smithi018.stdout:3/787: dread - d2/d6/d34/fd6 zero size 2024-04-12T20:56:19.045 INFO:tasks.workunit.client.0.smithi018.stdout:6/893: dread d5/df/d3f/dca/fcf [0,4194304] 0 2024-04-12T20:56:19.047 INFO:tasks.workunit.client.0.smithi018.stdout:5/817: mkdir d0/d2/d33/d46/d1e/d113 0 2024-04-12T20:56:19.051 INFO:tasks.workunit.client.0.smithi018.stdout:8/925: rename db/d4e/d56/d59/d76/da5/d45/d86/dbb/d113/f8a to db/d4e/d56/d59/d76/da5/f127 0 2024-04-12T20:56:19.051 INFO:tasks.workunit.client.0.smithi018.stdout:3/788: mkdir d2/d6/d30/d31/d38/dfb 0 2024-04-12T20:56:19.052 INFO:tasks.workunit.client.0.smithi018.stdout:6/894: creat d5/d6/d90/f153 x:0 0 0 2024-04-12T20:56:19.054 INFO:tasks.workunit.client.0.smithi018.stdout:5/818: creat d0/d2b/f114 x:0 0 0 2024-04-12T20:56:19.057 INFO:tasks.workunit.client.0.smithi018.stdout:3/789: stat d2/d6/d34/d71/dbc/dcc 0 2024-04-12T20:56:19.058 INFO:tasks.workunit.client.0.smithi018.stdout:8/926: chown db/f2d 170569813 1 2024-04-12T20:56:19.059 INFO:tasks.workunit.client.0.smithi018.stdout:0/708: truncate da/d15/d17/d71/dc1/dcc/f4f 1323662 0 2024-04-12T20:56:19.061 INFO:tasks.workunit.client.0.smithi018.stdout:9/951: write d3/d6/d9/dc/d5a/d79/d7e/d9b/f73 [4768473,36539] 0 2024-04-12T20:56:19.064 INFO:tasks.workunit.client.0.smithi018.stdout:7/748: dwrite d1/db/d31/d45/fc2 [0,4194304] 0 2024-04-12T20:56:19.067 INFO:tasks.workunit.client.0.smithi018.stdout:3/790: truncate d2/d43/d61/d64/dbf/fdd 390520 0 2024-04-12T20:56:19.067 INFO:tasks.workunit.client.0.smithi018.stdout:5/819: rename d0/d2/de4/fea to d0/d2/d33/d46/d1e/f115 0 2024-04-12T20:56:19.067 INFO:tasks.workunit.client.0.smithi018.stdout:0/709: truncate da/f13 268451 0 2024-04-12T20:56:19.069 INFO:tasks.workunit.client.0.smithi018.stdout:8/927: creat db/d4e/d56/d59/d76/de0/d27/df8/d7b/f128 x:0 0 0 2024-04-12T20:56:19.073 INFO:tasks.workunit.client.0.smithi018.stdout:6/895: rename d5/d6/d1b/d3b/d35 to d5/d6/d1b/d3b/d22/d117/d13c/d154 0 2024-04-12T20:56:19.077 INFO:tasks.workunit.client.0.smithi018.stdout:5/820: unlink d0/d2/d33/d61/c6f 0 2024-04-12T20:56:19.079 INFO:tasks.workunit.client.0.smithi018.stdout:7/749: creat d1/db/d31/d45/d8c/d9d/d70/d96/ff3 x:0 0 0 2024-04-12T20:56:19.081 INFO:tasks.workunit.client.0.smithi018.stdout:0/710: truncate da/d15/d17/d21/d36/d47/f72 4514517 0 2024-04-12T20:56:19.083 INFO:tasks.workunit.client.0.smithi018.stdout:8/928: mkdir db/d4e/d9a/daa/dcd/d129 0 2024-04-12T20:56:19.083 INFO:tasks.workunit.client.0.smithi018.stdout:6/896: link d5/df/d9b/cb9 d5/d15/d60/c155 0 2024-04-12T20:56:19.084 INFO:tasks.workunit.client.0.smithi018.stdout:8/929: chown db/d4e/d9a/d69 71 1 2024-04-12T20:56:19.088 INFO:tasks.workunit.client.0.smithi018.stdout:3/791: getdents d2/d6/d30/d31 0 2024-04-12T20:56:19.089 INFO:tasks.workunit.client.0.smithi018.stdout:5/821: mkdir d0/d2/d33/d46/d1e/d55/d66/d116 0 2024-04-12T20:56:19.092 INFO:tasks.workunit.client.0.smithi018.stdout:7/750: mkdir d1/db/d31/d45/df4 0 2024-04-12T20:56:19.093 INFO:tasks.workunit.client.0.smithi018.stdout:0/711: link da/l14 da/d15/d17/d5c/d79/dd6/led 0 2024-04-12T20:56:19.097 INFO:tasks.workunit.client.0.smithi018.stdout:8/930: rename db/d3b/f3d to db/d4e/d56/d59/d76/da5/d45/d86/dbb/d113/d100/f12a 0 2024-04-12T20:56:19.098 INFO:tasks.workunit.client.0.smithi018.stdout:5/822: creat d0/d2/d33/d46/d35/f117 x:0 0 0 2024-04-12T20:56:19.099 INFO:tasks.workunit.client.0.smithi018.stdout:6/897: write d5/d6/d1b/ff1 [653434,99068] 0 2024-04-12T20:56:19.103 INFO:tasks.workunit.client.0.smithi018.stdout:8/931: rmdir db/d4e 39 2024-04-12T20:56:19.104 INFO:tasks.workunit.client.0.smithi018.stdout:6/898: symlink d5/d6/d1b/l156 0 2024-04-12T20:56:19.107 INFO:tasks.workunit.client.0.smithi018.stdout:3/792: rename d2/d43/d61/d64/dd9 to d2/d6/d34/d71/db2/dfc 0 2024-04-12T20:56:19.110 INFO:tasks.workunit.client.0.smithi018.stdout:0/712: chown da/d15/d17/d5c/d79/dd6/led 1 1 2024-04-12T20:56:19.114 INFO:tasks.workunit.client.0.smithi018.stdout:6/899: dread d5/d6/d1b/d3b/d22/d117/d13c/f11f [0,4194304] 0 2024-04-12T20:56:19.117 INFO:tasks.workunit.client.0.smithi018.stdout:9/952: dwrite d3/d6/d9/dc/d5a/d79/d7e/d9b/d21/dc3/dd8/d5b/d7c/f69 [0,4194304] 0 2024-04-12T20:56:19.120 INFO:tasks.workunit.client.0.smithi018.stdout:0/713: mkdir da/d15/d17/d71/dc1/dee 0 2024-04-12T20:56:19.134 INFO:tasks.workunit.client.0.smithi018.stdout:8/932: symlink db/d4e/d56/d59/d76/da5/d45/da9/ddb/l12b 0 2024-04-12T20:56:19.135 INFO:tasks.workunit.client.0.smithi018.stdout:3/793: link d2/d43/d61/d64/dbf/d54/cbe d2/d6/d30/d31/d38/d3a/dd0/de1/cfd 0 2024-04-12T20:56:19.140 INFO:tasks.workunit.client.0.smithi018.stdout:7/751: write d1/db/d31/d3a/d42/fbd [781761,58710] 0 2024-04-12T20:56:19.141 INFO:tasks.workunit.client.0.smithi018.stdout:3/794: write d2/d6/d34/f46 [899099,87317] 0 2024-04-12T20:56:19.149 INFO:tasks.workunit.client.0.smithi018.stdout:9/953: dwrite d3/f80 [0,4194304] 0 2024-04-12T20:56:19.152 INFO:tasks.workunit.client.0.smithi018.stdout:0/714: dwrite da/d15/d4d/fda [0,4194304] 0 2024-04-12T20:56:19.155 INFO:tasks.workunit.client.0.smithi018.stdout:6/900: dwrite d5/d6/d1b/d3b/d6a/d116/f138 [0,4194304] 0 2024-04-12T20:56:19.164 INFO:tasks.workunit.client.0.smithi018.stdout:7/752: rename d1/db/d31/d45/d8c/d9d/d70/ddd to d1/db/d6f/dad/df5 0 2024-04-12T20:56:19.168 INFO:tasks.workunit.client.0.smithi018.stdout:5/823: dwrite d0/d2b/fa0 [0,4194304] 0 2024-04-12T20:56:19.173 INFO:tasks.workunit.client.0.smithi018.stdout:8/933: rename db/d4e/d56/d59/d76/da5/d45/d86/dbb/d113 to db/d4e/d56/d59/d76/da5/d11a/d12c 0 2024-04-12T20:56:19.178 INFO:tasks.workunit.client.0.smithi018.stdout:7/753: creat d1/ff6 x:0 0 0 2024-04-12T20:56:19.179 INFO:tasks.workunit.client.0.smithi018.stdout:5/824: write d0/d2/d33/d46/d1e/fcf [686778,47838] 0 2024-04-12T20:56:19.180 INFO:tasks.workunit.client.0.smithi018.stdout:8/934: unlink db/d4e/d56/d59/d76/de0/d27/fcf 0 2024-04-12T20:56:19.182 INFO:tasks.workunit.client.0.smithi018.stdout:5/825: chown d0/d2/d33/d46/d1e/f115 146308641 1 2024-04-12T20:56:19.183 INFO:tasks.workunit.client.0.smithi018.stdout:8/935: fsync db/d4e/d56/d59/d76/de0/fa3 0 2024-04-12T20:56:19.187 INFO:tasks.workunit.client.0.smithi018.stdout:7/754: symlink d1/db/d31/d45/d8c/lf7 0 2024-04-12T20:56:19.187 INFO:tasks.workunit.client.0.smithi018.stdout:5/826: symlink d0/d2/d33/d46/d35/l118 0 2024-04-12T20:56:19.188 INFO:tasks.workunit.client.0.smithi018.stdout:8/936: write db/d4e/d56/d59/d76/da5/d11a/d12c/f4c [5081787,101292] 0 2024-04-12T20:56:19.188 INFO:tasks.workunit.client.0.smithi018.stdout:7/755: fdatasync d1/fd3 0 2024-04-12T20:56:19.189 INFO:tasks.workunit.client.0.smithi018.stdout:9/954: link d3/d6/d9/dc/d5a/d79/dac/fce d3/d6/d9/dc/d5a/d79/d7e/d9b/d21/dc3/dd8/d5b/d6b/da7/de7/f126 0 2024-04-12T20:56:19.191 INFO:tasks.workunit.client.0.smithi018.stdout:5/827: chown d0/d2/d33/d46/d35/f79 48973700 1 2024-04-12T20:56:19.193 INFO:tasks.workunit.client.0.smithi018.stdout:7/756: chown d1/db/d31/d45/d8c/d9d/d70/d96/d5f 14504162 1 2024-04-12T20:56:19.197 INFO:tasks.workunit.client.0.smithi018.stdout:8/937: rename db/d4e/d73/df7/f111 to db/d4e/d56/d59/d76/da5/d45/f12d 0 2024-04-12T20:56:19.200 INFO:tasks.workunit.client.0.smithi018.stdout:7/757: creat d1/db/d31/d3a/d42/d4c/ff8 x:0 0 0 2024-04-12T20:56:19.201 INFO:tasks.workunit.client.0.smithi018.stdout:9/955: creat d3/d6/d1b/f127 x:0 0 0 2024-04-12T20:56:19.206 INFO:tasks.workunit.client.0.smithi018.stdout:6/901: write d5/d15/fe8 [5547889,37540] 0 2024-04-12T20:56:19.206 INFO:tasks.workunit.client.0.smithi018.stdout:8/938: truncate db/d4e/d56/d59/d76/da5/f123 816884 0 2024-04-12T20:56:19.208 INFO:tasks.workunit.client.0.smithi018.stdout:9/956: symlink d3/d6/l128 0 2024-04-12T20:56:19.213 INFO:tasks.workunit.client.0.smithi018.stdout:0/715: dread da/d15/d17/d21/d36/d47/f5d [0,4194304] 0 2024-04-12T20:56:19.218 INFO:tasks.workunit.client.0.smithi018.stdout:6/902: creat d5/df/d3f/f157 x:0 0 0 2024-04-12T20:56:19.219 INFO:tasks.workunit.client.0.smithi018.stdout:9/957: read d3/d6/d9/dc/d5a/d79/d7e/d9b/f73 [1211104,72146] 0 2024-04-12T20:56:19.219 INFO:tasks.workunit.client.0.smithi018.stdout:9/958: write d3/d6/d9/dc/d46/d5d/ff8 [884439,64139] 0 2024-04-12T20:56:19.223 INFO:tasks.workunit.client.0.smithi018.stdout:6/903: rename d5/df/d9b/fb6 to d5/d6/d1b/d3b/d6a/f158 0 2024-04-12T20:56:19.224 INFO:tasks.workunit.client.0.smithi018.stdout:9/959: fsync d3/d6/d3d/dba/fc9 0 2024-04-12T20:56:19.225 INFO:tasks.workunit.client.0.smithi018.stdout:8/939: link db/d4e/d56/d59/d76/da5/c20 db/d4e/d56/d59/d76/da5/c12e 0 2024-04-12T20:56:19.226 INFO:tasks.workunit.client.0.smithi018.stdout:5/828: write d0/d2/d33/d46/d1e/d55/d66/f93 [4371441,111673] 0 2024-04-12T20:56:19.226 INFO:tasks.workunit.client.0.smithi018.stdout:7/758: write d1/db/d31/f37 [2082881,4747] 0 2024-04-12T20:56:19.236 INFO:tasks.workunit.client.0.smithi018.stdout:5/829: unlink d0/d2/d33/d10/d26/f4a 0 2024-04-12T20:56:19.236 INFO:tasks.workunit.client.0.smithi018.stdout:6/904: truncate d5/d6/d1b/d3b/d6a/d116/d104/d128/f133 477939 0 2024-04-12T20:56:19.238 INFO:tasks.workunit.client.0.smithi018.stdout:8/940: fsync db/d4e/d56/d59/d76/de0/d27/df8/fb4 0 2024-04-12T20:56:19.242 INFO:tasks.workunit.client.0.smithi018.stdout:0/716: dwrite da/f38 [4194304,4194304] 0 2024-04-12T20:56:19.247 INFO:tasks.workunit.client.0.smithi018.stdout:5/830: write d0/d2/d33/d10/d60/dfa/dbf/da3/df9/ffe [682764,122218] 0 2024-04-12T20:56:19.247 INFO:tasks.workunit.client.0.smithi018.stdout:7/759: symlink d1/db/d31/d45/d8c/d9d/d70/d96/d5f/lf9 0 2024-04-12T20:56:19.249 INFO:tasks.workunit.client.0.smithi018.stdout:8/941: creat db/d4e/d7f/dfa/f12f x:0 0 0 2024-04-12T20:56:19.250 INFO:tasks.workunit.client.0.smithi018.stdout:7/760: chown d1/d6/f5a 4966139 1 2024-04-12T20:56:19.253 INFO:tasks.workunit.client.0.smithi018.stdout:0/717: symlink da/d15/d17/d21/d36/d47/lef 0 2024-04-12T20:56:19.254 INFO:tasks.workunit.client.0.smithi018.stdout:6/905: creat d5/d6/d1b/d3b/d22/d117/d14b/d105/d122/f159 x:0 0 0 2024-04-12T20:56:19.257 INFO:tasks.workunit.client.0.smithi018.stdout:5/831: creat d0/d2/d33/d10/d60/dfa/dbf/da3/f119 x:0 0 0 2024-04-12T20:56:19.260 INFO:tasks.workunit.client.0.smithi018.stdout:5/832: chown d0/d2/d33/d46/d1e/d55/d66/d116 15210 1 2024-04-12T20:56:19.264 INFO:tasks.workunit.client.0.smithi018.stdout:9/960: dread d3/d6/d9/fd [0,4194304] 0 2024-04-12T20:56:19.272 INFO:tasks.workunit.client.0.smithi018.stdout:5/833: write d0/d2/fe0 [50780,29801] 0 2024-04-12T20:56:19.274 INFO:tasks.workunit.client.0.smithi018.stdout:8/942: truncate db/d4e/d56/d59/d76/da5/d11a/d12c/f4c 1335915 0 2024-04-12T20:56:19.276 INFO:tasks.workunit.client.0.smithi018.stdout:9/961: rename d3/ff9 to d3/d6/d9/dc/d46/f129 0 2024-04-12T20:56:19.277 INFO:tasks.workunit.client.0.smithi018.stdout:5/834: chown d0/d2/d33/d46/d1e/f42 11351677 1 2024-04-12T20:56:19.277 INFO:tasks.workunit.client.0.smithi018.stdout:0/718: dwrite da/d15/d17/d21/d36/d47/fc8 [0,4194304] 0 2024-04-12T20:56:19.278 INFO:tasks.workunit.client.0.smithi018.stdout:6/906: getdents d5/df/d3f/dc1 0 2024-04-12T20:56:19.282 INFO:tasks.workunit.client.0.smithi018.stdout:0/719: write da/d15/d17/d21/d94/fa4 [3894521,15805] 0 2024-04-12T20:56:19.285 INFO:tasks.workunit.client.0.smithi018.stdout:6/907: chown d5/d6/d1b/d3b/d22/d117/d13c/d154/fd5 5 1 2024-04-12T20:56:19.287 INFO:tasks.workunit.client.0.smithi018.stdout:6/908: write d5/d6/d1b/ff1 [246713,107260] 0 2024-04-12T20:56:19.296 INFO:tasks.workunit.client.0.smithi018.stdout:5/835: rmdir d0/d2 39 2024-04-12T20:56:19.296 INFO:tasks.workunit.client.0.smithi018.stdout:6/909: read - d5/d6/d1b/d3b/d8e/fea zero size 2024-04-12T20:56:19.301 INFO:tasks.workunit.client.0.smithi018.stdout:0/720: symlink da/d15/d17/d21/d36/d91/lf0 0 2024-04-12T20:56:19.305 INFO:tasks.workunit.client.0.smithi018.stdout:6/910: read d5/d6/d1b/d3b/d6a/dd8/ff9 [35788,128166] 0 2024-04-12T20:56:19.305 INFO:tasks.workunit.client.0.smithi018.stdout:8/943: getdents db/d4e 0 2024-04-12T20:56:19.306 INFO:tasks.workunit.client.0.smithi018.stdout:5/836: chown d0/d2/d33/d46/d1e/f48 338842 1 2024-04-12T20:56:19.308 INFO:tasks.workunit.client.0.smithi018.stdout:8/944: readlink db/d4e/d56/l85 0 2024-04-12T20:56:19.311 INFO:tasks.workunit.client.0.smithi018.stdout:0/721: creat da/d15/d17/d71/dc1/ff1 x:0 0 0 2024-04-12T20:56:19.312 INFO:tasks.workunit.client.0.smithi018.stdout:0/722: chown l7 12954774 1 2024-04-12T20:56:19.313 INFO:tasks.workunit.client.0.smithi018.stdout:9/962: dwrite d3/d6/d9/dc/d5a/d79/d7e/d9b/d21/dc2/fcd [4194304,4194304] 0 2024-04-12T20:56:19.315 INFO:tasks.workunit.client.0.smithi018.stdout:5/837: stat d0/d2/d33/d46/d1e/c3c 0 2024-04-12T20:56:19.316 INFO:tasks.workunit.client.0.smithi018.stdout:0/723: readlink da/d15/d17/d21/d36/l3c 0 2024-04-12T20:56:19.318 INFO:tasks.workunit.client.0.smithi018.stdout:8/945: symlink db/d4e/d56/d59/d76/da5/d11a/d12c/d100/l130 0 2024-04-12T20:56:19.322 INFO:tasks.workunit.client.0.smithi018.stdout:9/963: chown d3/d6/d9/l99 1709380 1 2024-04-12T20:56:19.326 INFO:tasks.workunit.client.0.smithi018.stdout:5/838: mknod d0/d2/d11/d37/dc2/d89/d8d/c11a 0 2024-04-12T20:56:19.331 INFO:tasks.workunit.client.0.smithi018.stdout:5/839: chown d0/d2/d11/l14 1267117070 1 2024-04-12T20:56:19.331 INFO:tasks.workunit.client.0.smithi018.stdout:8/946: mkdir db/d4e/d56/d59/d76/da5/d45/da9/ddb/d131 0 2024-04-12T20:56:19.331 INFO:tasks.workunit.client.0.smithi018.stdout:9/964: getdents d3/dc8/df6 0 2024-04-12T20:56:19.333 INFO:tasks.workunit.client.0.smithi018.stdout:0/724: getdents da/d15/d17/d21/d36/da2/dc3/da9 0 2024-04-12T20:56:19.340 INFO:tasks.workunit.client.0.smithi018.stdout:8/947: dread db/d4e/d56/d59/d76/da5/fb0 [0,4194304] 0 2024-04-12T20:56:19.341 INFO:tasks.workunit.client.0.smithi018.stdout:0/725: symlink da/d15/d17/d21/d36/da2/dc3/lf2 0 2024-04-12T20:56:19.343 INFO:tasks.workunit.client.0.smithi018.stdout:9/965: creat d3/d6/d9/dc/d5a/d79/d7e/d9b/d21/d41/d78/f12a x:0 0 0 2024-04-12T20:56:19.352 INFO:tasks.workunit.client.0.smithi018.stdout:8/948: getdents db/d4e/d73/df7 0 2024-04-12T20:56:19.354 INFO:tasks.workunit.client.0.smithi018.stdout:9/966: symlink d3/d6/d9/dc/d46/d5d/l12b 0 2024-04-12T20:56:19.360 INFO:tasks.workunit.client.0.smithi018.stdout:9/967: fsync d3/d6/d9/dc/d5a/f121 0 2024-04-12T20:56:19.361 INFO:tasks.workunit.client.0.smithi018.stdout:0/726: dread da/d15/d17/d21/d36/d47/f5d [0,4194304] 0 2024-04-12T20:56:19.365 INFO:tasks.workunit.client.0.smithi018.stdout:8/949: write db/d4e/d9a/daa/f10c [4900485,46173] 0 2024-04-12T20:56:19.366 INFO:tasks.workunit.client.0.smithi018.stdout:9/968: write d3/d6/d9/d1e/f96 [793439,115270] 0 2024-04-12T20:56:19.367 INFO:tasks.workunit.client.0.smithi018.stdout:0/727: read da/d15/f8e [393212,45615] 0 2024-04-12T20:56:19.370 INFO:tasks.workunit.client.0.smithi018.stdout:6/911: dwrite d5/d6/d1b/d3b/f52 [0,4194304] 0 2024-04-12T20:56:19.372 INFO:tasks.workunit.client.0.smithi018.stdout:5/840: dread d0/d2/d11/d37/dc2/d89/d8d/fe5 [0,4194304] 0 2024-04-12T20:56:19.376 INFO:tasks.workunit.client.0.smithi018.stdout:5/841: write d0/d2/fe0 [103403,100336] 0 2024-04-12T20:56:19.377 INFO:tasks.workunit.client.0.smithi018.stdout:8/950: truncate db/d4e/d56/d59/d76/de0/d27/f50 4392103 0 2024-04-12T20:56:19.380 INFO:tasks.workunit.client.0.smithi018.stdout:0/728: creat da/d15/d17/d21/d36/d47/db5/ff3 x:0 0 0 2024-04-12T20:56:19.384 INFO:tasks.workunit.client.0.smithi018.stdout:8/951: stat db/d3b/lbd 0 2024-04-12T20:56:19.390 INFO:tasks.workunit.client.0.smithi018.stdout:5/842: rename d0/d2/d33/d10/d60/dfa/dbf/da3/df9/ffe to d0/d2/d33/d46/d1e/d113/f11b 0 2024-04-12T20:56:19.393 INFO:tasks.workunit.client.0.smithi018.stdout:9/969: dread d3/f80 [0,4194304] 0 2024-04-12T20:56:19.397 INFO:tasks.workunit.client.0.smithi018.stdout:0/729: link da/d15/d17/d21/d36/da2/dc3/d83/le7 da/d15/d17/db9/lf4 0 2024-04-12T20:56:19.400 INFO:tasks.workunit.client.0.smithi018.stdout:5/843: getdents d0/d2/d33/d46/d35/dd2 0 2024-04-12T20:56:19.405 INFO:tasks.workunit.client.0.smithi018.stdout:5/844: dread - d0/d2/d33/d10/d60/dfa/dbf/da3/f119 zero size 2024-04-12T20:56:19.406 INFO:tasks.workunit.client.0.smithi018.stdout:0/730: write da/d15/d17/d21/d36/da2/dc3/d3f/f69 [571450,31867] 0 2024-04-12T20:56:19.406 INFO:tasks.workunit.client.0.smithi018.stdout:6/912: dwrite d5/df/d19/d43/fc3 [0,4194304] 0 2024-04-12T20:56:19.413 INFO:tasks.workunit.client.0.smithi018.stdout:8/952: dread db/f2d [0,4194304] 0 2024-04-12T20:56:19.413 INFO:tasks.workunit.client.0.smithi018.stdout:6/913: rmdir d5/df/d3f/dd4 39 2024-04-12T20:56:19.413 INFO:tasks.workunit.client.0.smithi018.stdout:5/845: creat d0/d2/d11/dd8/f11c x:0 0 0 2024-04-12T20:56:19.422 INFO:tasks.workunit.client.0.smithi018.stdout:6/914: dread - d5/d6/d1b/d3b/d8e/fea zero size 2024-04-12T20:56:19.422 INFO:tasks.workunit.client.0.smithi018.stdout:5/846: symlink d0/d5b/d84/l11d 0 2024-04-12T20:56:19.425 INFO:tasks.workunit.client.0.smithi018.stdout:6/915: dread - d5/d6/d90/d125/d139/f149 zero size 2024-04-12T20:56:19.427 INFO:tasks.workunit.client.0.smithi018.stdout:5/847: write d0/d2/d33/d46/d1e/d2e/d7f/f98 [4437875,17025] 0 2024-04-12T20:56:19.432 INFO:tasks.workunit.client.0.smithi018.stdout:5/848: readlink d0/d2/d33/d10/d4e/lb4 0 2024-04-12T20:56:19.433 INFO:tasks.workunit.client.0.smithi018.stdout:6/916: unlink d5/d6/l9 0 2024-04-12T20:56:19.433 INFO:tasks.workunit.client.0.smithi018.stdout:9/970: dread d3/d6/f3c [4194304,4194304] 0 2024-04-12T20:56:19.438 INFO:tasks.workunit.client.0.smithi018.stdout:8/953: dwrite db/d72/f103 [0,4194304] 0 2024-04-12T20:56:19.439 INFO:tasks.workunit.client.0.smithi018.stdout:9/971: write d3/d6/d9/dc/f22 [3293673,4474] 0 2024-04-12T20:56:19.443 INFO:tasks.workunit.client.0.smithi018.stdout:9/972: readlink d3/d6/d9/d3f/l47 0 2024-04-12T20:56:19.444 INFO:tasks.workunit.client.0.smithi018.stdout:5/849: mkdir d0/d2/d33/d61/d11e 0 2024-04-12T20:56:19.448 INFO:tasks.workunit.client.0.smithi018.stdout:5/850: rmdir d0/d2/d11/d37/dc2 39 2024-04-12T20:56:19.456 INFO:tasks.workunit.client.0.smithi018.stdout:9/973: getdents d3/d6/d9/dc 0 2024-04-12T20:56:19.460 INFO:tasks.workunit.client.0.smithi018.stdout:8/954: dwrite db/d4e/d9a/d71/ff4 [0,4194304] 0 2024-04-12T20:56:19.480 INFO:tasks.workunit.client.0.smithi018.stdout:4/732: sync 2024-04-12T20:56:19.493 INFO:tasks.workunit.client.0.smithi018.stdout:6/917: dwrite d5/f20 [4194304,4194304] 0 2024-04-12T20:56:19.503 INFO:tasks.workunit.client.0.smithi018.stdout:9/974: dwrite d3/d6/d9/dc/d5a/d79/d7e/d9b/d21/dc3/dd8/d5b/d6b/f92 [0,4194304] 0 2024-04-12T20:56:19.511 INFO:tasks.workunit.client.0.smithi018.stdout:8/955: dwrite db/d4e/d56/d59/d76/da5/d11a/d12c/fde [0,4194304] 0 2024-04-12T20:56:19.516 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:56:19 smithi018 ceph-mon[28799]: pgmap v42: 65 pgs: 65 active+clean; 1.6 GiB data, 6.3 GiB used, 530 GiB / 536 GiB avail; 52 MiB/s rd, 127 MiB/s wr, 356 op/s 2024-04-12T20:56:19.516 INFO:tasks.workunit.client.0.smithi018.stdout:9/975: symlink d3/l12c 0 2024-04-12T20:56:19.518 INFO:tasks.workunit.client.0.smithi018.stdout:4/733: dwrite d3/d32/d33/d8e/fd8 [0,4194304] 0 2024-04-12T20:56:19.525 INFO:tasks.workunit.client.0.smithi018.stdout:4/734: creat d3/d5d/d60/dc5/d3d/da3/ff1 x:0 0 0 2024-04-12T20:56:19.528 INFO:tasks.workunit.client.0.smithi018.stdout:9/976: mkdir d3/d6/d43/d12d 0 2024-04-12T20:56:19.530 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:56:19 smithi186 ceph-mon[36597]: pgmap v42: 65 pgs: 65 active+clean; 1.6 GiB data, 6.3 GiB used, 530 GiB / 536 GiB avail; 52 MiB/s rd, 127 MiB/s wr, 356 op/s 2024-04-12T20:56:19.535 INFO:tasks.workunit.client.0.smithi018.stdout:8/956: dwrite db/f15 [0,4194304] 0 2024-04-12T20:56:19.538 INFO:tasks.workunit.client.0.smithi018.stdout:9/977: readlink d3/d6/d9/dc/d5a/d79/d7e/d9b/d21/dc3/dd8/l86 0 2024-04-12T20:56:19.541 INFO:tasks.workunit.client.0.smithi018.stdout:9/978: read d3/f80 [1601635,16125] 0 2024-04-12T20:56:19.542 INFO:tasks.workunit.client.0.smithi018.stdout:9/979: chown d3 897805 1 2024-04-12T20:56:19.544 INFO:tasks.workunit.client.0.smithi018.stdout:4/735: getdents d3/d5d/d60/dc5/d53 0 2024-04-12T20:56:19.547 INFO:tasks.workunit.client.0.smithi018.stdout:8/957: creat db/d4e/d56/d59/d76/de0/d27/df8/d7b/f132 x:0 0 0 2024-04-12T20:56:19.558 INFO:tasks.workunit.client.0.smithi018.stdout:9/980: rename d3/d6/d9/dc/d46/cee to d3/d6/d3d/dba/c12e 0 2024-04-12T20:56:19.559 INFO:tasks.workunit.client.0.smithi018.stdout:8/958: mknod db/d4e/d56/d59/d76/de0/d27/df8/d119/c133 0 2024-04-12T20:56:19.559 INFO:tasks.workunit.client.0.smithi018.stdout:6/918: write d5/d6/d1b/d3b/f76 [717651,80441] 0 2024-04-12T20:56:19.566 INFO:tasks.workunit.client.0.smithi018.stdout:8/959: symlink db/d4e/d56/d59/d76/de0/d27/df8/l134 0 2024-04-12T20:56:19.570 INFO:tasks.workunit.client.0.smithi018.stdout:8/960: symlink db/d4e/dab/l135 0 2024-04-12T20:56:19.571 INFO:tasks.workunit.client.0.smithi018.stdout:4/736: write d3/db/fc4 [143170,46400] 0 2024-04-12T20:56:19.575 INFO:tasks.workunit.client.0.smithi018.stdout:8/961: write db/d4e/d56/d59/d76/f10b [353553,121955] 0 2024-04-12T20:56:19.578 INFO:tasks.workunit.client.0.smithi018.stdout:8/962: stat db/d3b 0 2024-04-12T20:56:19.578 INFO:tasks.workunit.client.0.smithi018.stdout:4/737: mkdir d3/d5d/d60/dc5/d43/d4b/df2 0 2024-04-12T20:56:19.582 INFO:tasks.workunit.client.0.smithi018.stdout:6/919: dread d5/d15/f142 [0,4194304] 0 2024-04-12T20:56:19.586 INFO:tasks.workunit.client.0.smithi018.stdout:6/920: dread - d5/d6/d1b/d3b/d8e/fdd zero size 2024-04-12T20:56:19.586 INFO:tasks.workunit.client.0.smithi018.stdout:9/981: dwrite d3/d6/d9/dc/d46/f48 [4194304,4194304] 0 2024-04-12T20:56:19.589 INFO:tasks.workunit.client.0.smithi018.stdout:8/963: unlink db/d4e/d56/d59/d76/da5/d45/da9/ddb/l12b 0 2024-04-12T20:56:19.589 INFO:tasks.workunit.client.0.smithi018.stdout:6/921: fsync d5/d6/d1b/d3b/d22/f81 0 2024-04-12T20:56:19.596 INFO:tasks.workunit.client.0.smithi018.stdout:9/982: creat d3/d6/d3d/dba/f12f x:0 0 0 2024-04-12T20:56:19.598 INFO:tasks.workunit.client.0.smithi018.stdout:8/964: mknod db/d4e/d56/d59/d76/da5/d45/da9/c136 0 2024-04-12T20:56:19.603 INFO:tasks.workunit.client.0.smithi018.stdout:8/965: creat db/d95/dd6/f137 x:0 0 0 2024-04-12T20:56:19.607 INFO:tasks.workunit.client.0.smithi018.stdout:9/983: chown d3/d6/d9/dc/d5a/d79/d7e/d9b/f62 123242960 1 2024-04-12T20:56:19.608 INFO:tasks.workunit.client.0.smithi018.stdout:8/966: write db/d4e/d9a/ffd [2738122,22678] 0 2024-04-12T20:56:19.612 INFO:tasks.workunit.client.0.smithi018.stdout:8/967: fsync db/d4e/d73/f92 0 2024-04-12T20:56:19.619 INFO:tasks.workunit.client.0.smithi018.stdout:4/738: dread d3/db/f24 [0,4194304] 0 2024-04-12T20:56:19.619 INFO:tasks.workunit.client.0.smithi018.stdout:6/922: dwrite d5/d6/fbd [0,4194304] 0 2024-04-12T20:56:19.620 INFO:tasks.workunit.client.0.smithi018.stdout:8/968: rmdir db/d4e/d7f/dba 39 2024-04-12T20:56:19.624 INFO:tasks.workunit.client.0.smithi018.stdout:8/969: write db/d72/fd7 [2530109,44957] 0 2024-04-12T20:56:19.630 INFO:tasks.workunit.client.0.smithi018.stdout:6/923: fdatasync d5/d6/d1b/d3b/d22/d117/d13c/d154/d4f/fa7 0 2024-04-12T20:56:19.630 INFO:tasks.workunit.client.0.smithi018.stdout:8/970: rmdir db/d4e/d56/d59/d76/de0 39 2024-04-12T20:56:19.634 INFO:tasks.workunit.client.0.smithi018.stdout:8/971: write db/d4e/d56/d59/d76/de0/d27/fec [102402,34607] 0 2024-04-12T20:56:19.638 INFO:tasks.workunit.client.0.smithi018.stdout:8/972: symlink db/d95/dd6/d117/l138 0 2024-04-12T20:56:19.644 INFO:tasks.workunit.client.0.smithi018.stdout:4/739: write d3/d5d/d60/d64/fb9 [3706891,127075] 0 2024-04-12T20:56:19.648 INFO:tasks.workunit.client.0.smithi018.stdout:4/740: mknod d3/d5d/d60/dc5/d53/cf3 0 2024-04-12T20:56:19.651 INFO:tasks.workunit.client.0.smithi018.stdout:4/741: creat d3/d5d/d60/dc5/d43/d4b/df2/ff4 x:0 0 0 2024-04-12T20:56:19.662 INFO:tasks.workunit.client.0.smithi018.stdout:4/742: dwrite d3/faa [0,4194304] 0 2024-04-12T20:56:19.686 INFO:tasks.workunit.client.0.smithi018.stdout:2/751: sync 2024-04-12T20:56:19.686 INFO:tasks.workunit.client.0.smithi018.stdout:1/655: sync 2024-04-12T20:56:19.686 INFO:tasks.workunit.client.0.smithi018.stdout:3/795: sync 2024-04-12T20:56:19.686 INFO:tasks.workunit.client.0.smithi018.stdout:7/761: sync 2024-04-12T20:56:19.693 INFO:tasks.workunit.client.0.smithi018.stdout:2/752: fsync d5/f95 0 2024-04-12T20:56:19.694 INFO:tasks.workunit.client.0.smithi018.stdout:7/762: mkdir d1/d86/dfa 0 2024-04-12T20:56:19.694 INFO:tasks.workunit.client.0.smithi018.stdout:1/656: write d3/d9/da/f5f [3373396,12033] 0 2024-04-12T20:56:19.698 INFO:tasks.workunit.client.0.smithi018.stdout:3/796: creat d2/d6/d30/d31/d38/d3a/d76/da2/de4/ffe x:0 0 0 2024-04-12T20:56:19.700 INFO:tasks.workunit.client.0.smithi018.stdout:2/753: dread - d5/d8/d10/d43/de2/daf/d90/fb4 zero size 2024-04-12T20:56:19.702 INFO:tasks.workunit.client.0.smithi018.stdout:4/743: dwrite d3/d5d/d60/dc5/d3d/da3/fe2 [0,4194304] 0 2024-04-12T20:56:19.705 INFO:tasks.workunit.client.0.smithi018.stdout:7/763: mkdir d1/d6/dfb 0 2024-04-12T20:56:19.712 INFO:tasks.workunit.client.0.smithi018.stdout:2/754: creat d5/d8/d10/d43/de2/daf/d88/f104 x:0 0 0 2024-04-12T20:56:19.724 INFO:tasks.workunit.client.0.smithi018.stdout:6/924: dwrite d5/d15/d60/f65 [4194304,4194304] 0 2024-04-12T20:56:19.724 INFO:tasks.workunit.client.0.smithi018.stdout:9/984: link d3/d6/d9/c36 d3/d6/d9/dc/d46/c130 0 2024-04-12T20:56:19.726 INFO:tasks.workunit.client.0.smithi018.stdout:4/744: write d3/d32/fbe [72334,16958] 0 2024-04-12T20:56:19.733 INFO:tasks.workunit.client.0.smithi018.stdout:1/657: rename d3/d9/da/d14/fac to d3/d9/da/dd/d31/d32/d13/d19/d2d/d4e/d58/d61/d71/fdf 0 2024-04-12T20:56:19.737 INFO:tasks.workunit.client.0.smithi018.stdout:4/745: readlink d3/d5d/d60/dc5/d43/d4b/l87 0 2024-04-12T20:56:19.738 INFO:tasks.workunit.client.0.smithi018.stdout:9/985: stat d3/d6/d43/c7d 0 2024-04-12T20:56:19.742 INFO:tasks.workunit.client.0.smithi018.stdout:7/764: dread d1/fba [4194304,4194304] 0 2024-04-12T20:56:19.742 INFO:tasks.workunit.client.0.smithi018.stdout:3/797: dwrite d2/d6/d30/d31/f9c [0,4194304] 0 2024-04-12T20:56:19.743 INFO:tasks.workunit.client.0.smithi018.stdout:6/925: unlink d5/d6/d1b/d3b/d22/d117/d13c/d154/d13d/f147 0 2024-04-12T20:56:19.746 INFO:tasks.workunit.client.0.smithi018.stdout:1/658: read d3/d9/da/dd/d31/d32/d13/d19/d2d/f76 [1861182,78945] 0 2024-04-12T20:56:19.747 INFO:tasks.workunit.client.0.smithi018.stdout:6/926: chown d5/d6/d1b/d3b/d22 22160692 1 2024-04-12T20:56:19.751 INFO:tasks.workunit.client.0.smithi018.stdout:3/798: write d2/d43/fb3 [493562,103554] 0 2024-04-12T20:56:19.753 INFO:tasks.workunit.client.0.smithi018.stdout:3/799: readlink d2/d6/d34/lbb 0 2024-04-12T20:56:19.756 INFO:tasks.workunit.client.0.smithi018.stdout:9/986: unlink d3/d6/d9/dc/d5a/d79/d7e/d9b/d21/dc3/dd8/fe5 0 2024-04-12T20:56:19.759 INFO:tasks.workunit.client.0.smithi018.stdout:6/927: chown d5/d6/d1b/d3b/d22/d117/d13c/d154/d4f/c55 398700 1 2024-04-12T20:56:19.760 INFO:tasks.workunit.client.0.smithi018.stdout:9/987: mknod d3/d6/d1b/c131 0 2024-04-12T20:56:19.760 INFO:tasks.workunit.client.0.smithi018.stdout:7/765: rename d1/d86/dfa to d1/db/d31/d45/d8c/d9d/d70/d96/d5f/da9/dfc 0 2024-04-12T20:56:19.762 INFO:tasks.workunit.client.0.smithi018.stdout:3/800: creat d2/d43/db6/db5/fff x:0 0 0 2024-04-12T20:56:19.762 INFO:tasks.workunit.client.0.smithi018.stdout:6/928: chown d5/d6/fbd 902001 1 2024-04-12T20:56:19.764 INFO:tasks.workunit.client.0.smithi018.stdout:1/659: mknod d3/d9/da/dd/d31/d32/d33/d84/dc2/ce0 0 2024-04-12T20:56:19.766 INFO:tasks.workunit.client.0.smithi018.stdout:7/766: chown d1/db/d6f/fe0 1 1 2024-04-12T20:56:19.767 INFO:tasks.workunit.client.0.smithi018.stdout:9/988: write d3/d6/d9/dc/fca [1422784,92371] 0 2024-04-12T20:56:19.772 INFO:tasks.workunit.client.0.smithi018.stdout:6/929: readlink d5/df/d3f/dc1/lcc 0 2024-04-12T20:56:19.772 INFO:tasks.workunit.client.0.smithi018.stdout:7/767: chown d1/db/d31/d45/df4 7 1 2024-04-12T20:56:19.772 INFO:tasks.workunit.client.0.smithi018.stdout:3/801: rename d2/c1f to d2/c100 0 2024-04-12T20:56:19.774 INFO:tasks.workunit.client.0.smithi018.stdout:4/746: dwrite d3/db/fad [0,4194304] 0 2024-04-12T20:56:19.781 INFO:tasks.workunit.client.0.smithi018.stdout:6/930: symlink d5/d6/l15a 0 2024-04-12T20:56:19.782 INFO:tasks.workunit.client.0.smithi018.stdout:4/747: chown d3/d5d/d60/dc5/d43/d4b/de6/lbf 1799208 1 2024-04-12T20:56:19.782 INFO:tasks.workunit.client.0.smithi018.stdout:7/768: rmdir d1/db/d31/d45/d8c/d9d/d70/d96 39 2024-04-12T20:56:19.786 INFO:tasks.workunit.client.0.smithi018.stdout:6/931: truncate d5/f8d 2820324 0 2024-04-12T20:56:19.789 INFO:tasks.workunit.client.0.smithi018.stdout:3/802: fsync d2/d6/fc 0 2024-04-12T20:56:19.791 INFO:tasks.workunit.client.0.smithi018.stdout:7/769: creat d1/db/d31/d3a/d42/d4c/d6c/ffd x:0 0 0 2024-04-12T20:56:19.793 INFO:tasks.workunit.client.0.smithi018.stdout:9/989: write d3/f5e [2244051,65149] 0 2024-04-12T20:56:19.796 INFO:tasks.workunit.client.0.smithi018.stdout:4/748: rename d3/d32/f4e to d3/d5d/d60/dc5/d43/d4b/de6/ff5 0 2024-04-12T20:56:19.804 INFO:tasks.workunit.client.0.smithi018.stdout:1/660: dwrite d3/d9/da/dd/d31/d32/d13/f46 [0,4194304] 0 2024-04-12T20:56:19.806 INFO:tasks.workunit.client.0.smithi018.stdout:7/770: mknod d1/db/d31/d45/d8c/d9d/d70/d96/d5f/da9/db9/cfe 0 2024-04-12T20:56:19.811 INFO:tasks.workunit.client.0.smithi018.stdout:7/771: readlink d1/db/d31/d30/laa 0 2024-04-12T20:56:19.813 INFO:tasks.workunit.client.0.smithi018.stdout:1/661: fsync d3/d9/da/dd/d31/d32/f1b 0 2024-04-12T20:56:19.814 INFO:tasks.workunit.client.0.smithi018.stdout:9/990: dread d3/d6/d9/dc/d5a/d79/d7e/d9b/f62 [0,4194304] 0 2024-04-12T20:56:19.819 INFO:tasks.workunit.client.0.smithi018.stdout:9/991: creat d3/d6/d9/dc/d46/f132 x:0 0 0 2024-04-12T20:56:19.823 INFO:tasks.workunit.client.0.smithi018.stdout:3/803: dread d2/f28 [0,4194304] 0 2024-04-12T20:56:19.834 INFO:tasks.workunit.client.0.smithi018.stdout:9/992: chown d3/d6/d9/d1e/f20 4143828 1 2024-04-12T20:56:19.836 INFO:tasks.workunit.client.0.smithi018.stdout:3/804: write d2/d4d/d8d/fab [894494,10563] 0 2024-04-12T20:56:19.837 INFO:tasks.workunit.client.0.smithi018.stdout:9/993: readlink d3/d6/d9/dc/d5a/d79/db5/l100 0 2024-04-12T20:56:19.843 INFO:tasks.workunit.client.0.smithi018.stdout:1/662: dwrite d3/d9/da/dd/d31/d32/f34 [0,4194304] 0 2024-04-12T20:56:19.845 INFO:tasks.workunit.client.0.smithi018.stdout:9/994: creat d3/d6/d9/dc/d5a/d79/f133 x:0 0 0 2024-04-12T20:56:19.850 INFO:tasks.workunit.client.0.smithi018.stdout:9/995: symlink d3/d6/d9/dc/d5a/d79/d7e/d9b/d21/l134 0 2024-04-12T20:56:19.853 INFO:tasks.workunit.client.0.smithi018.stdout:1/663: unlink d3/d9/da/dd/d31/f77 0 2024-04-12T20:56:19.854 INFO:tasks.workunit.client.0.smithi018.stdout:9/996: mknod d3/d6/d9/dc/d5a/d79/dac/c135 0 2024-04-12T20:56:19.860 INFO:tasks.workunit.client.0.smithi018.stdout:1/664: stat d3/d9/da/dd/d31/d32/d33/d47/c69 0 2024-04-12T20:56:19.864 INFO:tasks.workunit.client.0.smithi018.stdout:7/772: dread d1/db/d31/d45/d8c/d9d/d70/d96/f68 [0,4194304] 0 2024-04-12T20:56:19.865 INFO:tasks.workunit.client.0.smithi018.stdout:1/665: creat d3/d9/da/dd/d31/d44/fe1 x:0 0 0 2024-04-12T20:56:19.865 INFO:tasks.workunit.client.0.smithi018.stdout:9/997: rename d3/d6/d9/dc/d5a/d79/d7e/d9b/d21/dc3/dd8/d5b/d6b/f92 to d3/d6/d9/dc/d46/f136 0 2024-04-12T20:56:19.866 INFO:tasks.workunit.client.0.smithi018.stdout:7/773: fdatasync d1/f1b 0 2024-04-12T20:56:19.870 INFO:tasks.workunit.client.0.smithi018.stdout:1/666: write d3/d9/da/dd/d31/d32/d13/d19/d2d/d4e/d7a/d82/fb5 [444336,82596] 0 2024-04-12T20:56:19.876 INFO:tasks.workunit.client.0.smithi018.stdout:1/667: read d3/d9/da/dd/d31/d32/d13/d19/f6e [880109,60071] 0 2024-04-12T20:56:19.878 INFO:tasks.workunit.client.0.smithi018.stdout:9/998: rename d3/d6/d9/dc/d5a/d79/d7e/d9b/c38 to d3/d6/d9/dc/d5a/d79/d7e/d9b/c137 0 2024-04-12T20:56:19.884 INFO:tasks.workunit.client.0.smithi018.stdout:1/668: mkdir d3/d9/da/dd/d31/d32/d13/d19/d2d/d4e/d5e/da0/de2 0 2024-04-12T20:56:19.887 INFO:tasks.workunit.client.0.smithi018.stdout:1/669: creat d3/fe3 x:0 0 0 2024-04-12T20:56:19.887 INFO:tasks.workunit.client.0.smithi018.stdout:7/774: dwrite d1/db/d31/d3a/d97/fb1 [0,4194304] 0 2024-04-12T20:56:19.888 INFO:tasks.workunit.client.0.smithi018.stdout:3/805: dwrite d2/d4d/d88/f8a [0,4194304] 0 2024-04-12T20:56:19.892 INFO:tasks.workunit.client.0.smithi018.stdout:7/775: dread - d1/db/d31/d3a/d42/d4c/ff8 zero size 2024-04-12T20:56:19.896 INFO:tasks.workunit.client.0.smithi018.stdout:3/806: creat d2/d4d/d88/f101 x:0 0 0 2024-04-12T20:56:19.897 INFO:tasks.workunit.client.0.smithi018.stdout:1/670: creat d3/d9/da/dd/d31/d32/d13/d19/d2d/d4e/d5e/d97/fe4 x:0 0 0 2024-04-12T20:56:19.902 INFO:tasks.workunit.client.0.smithi018.stdout:7/776: mknod d1/db/d31/d45/d53/cff 0 2024-04-12T20:56:19.906 INFO:tasks.workunit.client.0.smithi018.stdout:1/671: chown d3/d9/da/dd/d31/d32/d13/d19/d2d/d4e/d58/d61/d71/f91 0 1 2024-04-12T20:56:19.918 INFO:tasks.workunit.client.0.smithi018.stdout:9/999: dwrite d3/d6/d9/dc/d5a/d79/d7e/d9b/d21/d41/d78/f7a [0,4194304] 0 2024-04-12T20:56:19.931 INFO:tasks.workunit.client.0.smithi018.stdout:3/807: write d2/d43/db6/db5/dd8/fb9 [644664,22095] 0 2024-04-12T20:56:19.931 INFO:tasks.workunit.client.0.smithi018.stdout:7/777: write d1/db/d31/d3a/d42/faf [54674,91993] 0 2024-04-12T20:56:19.946 INFO:tasks.workunit.client.0.smithi018.stdout:0/731: sync 2024-04-12T20:56:19.946 INFO:tasks.workunit.client.0.smithi018.stdout:5/851: sync 2024-04-12T20:56:19.951 INFO:tasks.workunit.client.0.smithi018.stdout:0/732: mkdir da/d15/d17/d21/d36/dce/df5 0 2024-04-12T20:56:19.954 INFO:tasks.workunit.client.0.smithi018.stdout:5/852: symlink d0/d2/l11f 0 2024-04-12T20:56:19.960 INFO:tasks.workunit.client.0.smithi018.stdout:0/733: creat da/d15/d17/d21/d36/dce/ff6 x:0 0 0 2024-04-12T20:56:19.965 INFO:tasks.workunit.client.0.smithi018.stdout:3/808: dwrite d2/d6/d34/f59 [0,4194304] 0 2024-04-12T20:56:19.966 INFO:tasks.workunit.client.0.smithi018.stdout:7/778: dread d1/db/d31/d45/d53/f7d [0,4194304] 0 2024-04-12T20:56:19.968 INFO:tasks.workunit.client.0.smithi018.stdout:3/809: write d2/d4d/d88/f101 [362951,103328] 0 2024-04-12T20:56:19.976 INFO:tasks.workunit.client.0.smithi018.stdout:3/810: mkdir d2/d43/db6/d102 0 2024-04-12T20:56:19.977 INFO:tasks.workunit.client.0.smithi018.stdout:5/853: link d0/d2/c6 d0/d2/d33/d10/d60/dfa/d8f/c120 0 2024-04-12T20:56:19.979 INFO:tasks.workunit.client.0.smithi018.stdout:0/734: write da/d15/d17/d5c/f64 [4696144,12599] 0 2024-04-12T20:56:19.982 INFO:tasks.workunit.client.0.smithi018.stdout:0/735: fsync da/d15/f8e 0 2024-04-12T20:56:19.984 INFO:tasks.workunit.client.0.smithi018.stdout:7/779: dread d1/db/d31/d3a/d97/fb1 [0,4194304] 0 2024-04-12T20:56:19.984 INFO:tasks.workunit.client.0.smithi018.stdout:0/736: stat da/d15/d17/f43 0 2024-04-12T20:56:19.986 INFO:tasks.workunit.client.0.smithi018.stdout:3/811: dread d2/d6/f4e [0,4194304] 0 2024-04-12T20:56:19.989 INFO:tasks.workunit.client.0.smithi018.stdout:0/737: unlink da/d15/d4d/d58/cb6 0 2024-04-12T20:56:19.991 INFO:tasks.workunit.client.0.smithi018.stdout:5/854: truncate d0/d2/d33/d46/d1e/d2e/fd3 649462 0 2024-04-12T20:56:19.997 INFO:tasks.workunit.client.0.smithi018.stdout:3/812: rename d2/d43/db6/d102 to d2/d43/d61/d64/dbf/d103 0 2024-04-12T20:56:19.999 INFO:tasks.workunit.client.0.smithi018.stdout:5/855: creat d0/d2/d33/d46/d1e/d2e/d7f/f121 x:0 0 0 2024-04-12T20:56:20.001 INFO:tasks.workunit.client.0.smithi018.stdout:5/856: stat d0/d2/de4/f110 0 2024-04-12T20:56:20.002 INFO:tasks.workunit.client.0.smithi018.stdout:0/738: rename da/d15/d17/db9 to da/d15/d17/d71/dc1/dee/df7 0 2024-04-12T20:56:20.003 INFO:tasks.workunit.client.0.smithi018.stdout:5/857: chown d0/d2/d33/d46/d35/d4d/db3 24 1 2024-04-12T20:56:20.004 INFO:tasks.workunit.client.0.smithi018.stdout:0/739: fdatasync da/d15/d4d/d6b/f6d 0 2024-04-12T20:56:20.011 INFO:tasks.workunit.client.0.smithi018.stdout:7/780: dread d1/db/d31/d45/f9a [0,4194304] 0 2024-04-12T20:56:20.015 INFO:tasks.workunit.client.0.smithi018.stdout:0/740: getdents da/d15/d17/d71/dc1/dcc 0 2024-04-12T20:56:20.017 INFO:tasks.workunit.client.0.smithi018.stdout:7/781: unlink d1/db/d31/d45/d8c/d9d/d70/d96/c33 0 2024-04-12T20:56:20.019 INFO:tasks.workunit.client.0.smithi018.stdout:0/741: mknod da/d15/d17/d71/dc1/cf8 0 2024-04-12T20:56:20.021 INFO:tasks.workunit.client.0.smithi018.stdout:7/782: mkdir d1/db/d31/d30/dc6/d100 0 2024-04-12T20:56:20.023 INFO:tasks.workunit.client.0.smithi018.stdout:3/813: write d2/d43/d61/d64/dbf/d70/feb [1609958,75003] 0 2024-04-12T20:56:20.023 INFO:tasks.workunit.client.0.smithi018.stdout:5/858: write d0/d2/d33/d46/d35/f79 [7149335,107654] 0 2024-04-12T20:56:20.031 INFO:tasks.workunit.client.0.smithi018.stdout:5/859: symlink d0/d2/d33/d46/d1e/d113/l122 0 2024-04-12T20:56:20.031 INFO:tasks.workunit.client.0.smithi018.stdout:7/783: rename d1/db/d31/d3a/d42/d4c/d6c/ca8 to d1/db/d31/d30/dc6/c101 0 2024-04-12T20:56:20.038 INFO:tasks.workunit.client.0.smithi018.stdout:7/784: unlink d1/db/d31/d45/d53/l8b 0 2024-04-12T20:56:20.039 INFO:tasks.workunit.client.0.smithi018.stdout:5/860: creat d0/d2/d11/f123 x:0 0 0 2024-04-12T20:56:20.039 INFO:tasks.workunit.client.0.smithi018.stdout:7/785: chown d1/d6/f67 1 1 2024-04-12T20:56:20.042 INFO:tasks.workunit.client.0.smithi018.stdout:3/814: dwrite d2/d4d/d88/f8a [0,4194304] 0 2024-04-12T20:56:20.043 INFO:tasks.workunit.client.0.smithi018.stdout:7/786: write d1/db/d31/d45/f9e [518417,116491] 0 2024-04-12T20:56:20.047 INFO:tasks.workunit.client.0.smithi018.stdout:5/861: unlink d0/d2/d11/d37/dc2/d89/cc9 0 2024-04-12T20:56:20.051 INFO:tasks.workunit.client.0.smithi018.stdout:5/862: mkdir d0/d2/de4/d124 0 2024-04-12T20:56:20.059 INFO:tasks.workunit.client.0.smithi018.stdout:7/787: dread d1/db/d31/d45/d8c/d9d/d70/d96/f68 [0,4194304] 0 2024-04-12T20:56:20.060 INFO:tasks.workunit.client.0.smithi018.stdout:5/863: creat d0/d2/d33/d10/d60/dfa/dbf/da3/dae/f125 x:0 0 0 2024-04-12T20:56:20.064 INFO:tasks.workunit.client.0.smithi018.stdout:7/788: creat d1/d6/f102 x:0 0 0 2024-04-12T20:56:20.064 INFO:tasks.workunit.client.0.smithi018.stdout:5/864: mknod d0/d2/d33/d10/d60/dfa/d8f/c126 0 2024-04-12T20:56:20.065 INFO:tasks.workunit.client.0.smithi018.stdout:3/815: dwrite d2/d43/d61/d64/fd3 [0,4194304] 0 2024-04-12T20:56:20.069 INFO:tasks.workunit.client.0.smithi018.stdout:3/816: unlink d2/d83/cba 0 2024-04-12T20:56:20.071 INFO:tasks.workunit.client.0.smithi018.stdout:5/865: symlink d0/d2/d33/d10/d60/l127 0 2024-04-12T20:56:20.072 INFO:tasks.workunit.client.0.smithi018.stdout:7/789: getdents d1/db/d31/df2 0 2024-04-12T20:56:20.076 INFO:tasks.workunit.client.0.smithi018.stdout:3/817: rename d2/d4d/d88/f101 to d2/d6/d30/d31/d38/d3a/d48/f104 0 2024-04-12T20:56:20.078 INFO:tasks.workunit.client.0.smithi018.stdout:7/790: unlink d1/c54 0 2024-04-12T20:56:20.080 INFO:tasks.workunit.client.0.smithi018.stdout:3/818: rmdir d2/d6/d95 39 2024-04-12T20:56:20.082 INFO:tasks.workunit.client.0.smithi018.stdout:7/791: rmdir d1/db/d31/d3a/d97 39 2024-04-12T20:56:20.086 INFO:tasks.workunit.client.0.smithi018.stdout:7/792: mkdir d1/db/d31/d3a/d42/d4c/d103 0 2024-04-12T20:56:20.087 INFO:tasks.workunit.client.0.smithi018.stdout:3/819: mkdir d2/d6/d30/d31/d38/d3a/d76/da2/de4/d105 0 2024-04-12T20:56:20.089 INFO:tasks.workunit.client.0.smithi018.stdout:3/820: stat d2/d83/f92 0 2024-04-12T20:56:20.090 INFO:tasks.workunit.client.0.smithi018.stdout:5/866: dread d0/d2/d33/d10/d60/dfa/d8f/fd5 [0,4194304] 0 2024-04-12T20:56:20.090 INFO:tasks.workunit.client.0.smithi018.stdout:7/793: chown d1/db/d31/d45/d8c/d9d/d70/d96/c23 5637007 1 2024-04-12T20:56:20.096 INFO:tasks.workunit.client.0.smithi018.stdout:7/794: rmdir d1/db/d31/d30 39 2024-04-12T20:56:20.099 INFO:tasks.workunit.client.0.smithi018.stdout:5/867: mknod d0/d2/d11/d37/c128 0 2024-04-12T20:56:20.110 INFO:tasks.workunit.client.0.smithi018.stdout:3/821: dwrite d2/d43/d61/d64/dbf/fed [0,4194304] 0 2024-04-12T20:56:20.115 INFO:tasks.workunit.client.0.smithi018.stdout:5/868: link d0/d2/d33/d46/d1e/f6c d0/d2/de4/d124/f129 0 2024-04-12T20:56:20.116 INFO:tasks.workunit.client.0.smithi018.stdout:3/822: symlink d2/d43/d61/d64/dbf/d70/dfa/l106 0 2024-04-12T20:56:20.120 INFO:tasks.workunit.client.0.smithi018.stdout:7/795: dwrite d1/f3e [0,4194304] 0 2024-04-12T20:56:20.124 INFO:tasks.workunit.client.0.smithi018.stdout:3/823: fdatasync d2/d6/d30/d31/d38/d3a/f45 0 2024-04-12T20:56:20.127 INFO:tasks.workunit.client.0.smithi018.stdout:7/796: write d1/d6/fb4 [5023833,37207] 0 2024-04-12T20:56:20.128 INFO:tasks.workunit.client.0.smithi018.stdout:3/824: stat d2/l3f 0 2024-04-12T20:56:20.129 INFO:tasks.workunit.client.0.smithi018.stdout:7/797: chown d1/lce 2 1 2024-04-12T20:56:20.143 INFO:tasks.workunit.client.0.smithi018.stdout:5/869: dwrite d0/d2/d33/d46/d35/f4f [0,4194304] 0 2024-04-12T20:56:20.157 INFO:tasks.workunit.client.0.smithi018.stdout:3/825: dwrite d2/d6/d30/d31/d38/d3a/f45 [0,4194304] 0 2024-04-12T20:56:20.159 INFO:tasks.workunit.client.0.smithi018.stdout:7/798: dwrite d1/db/d31/d45/d8c/d9d/d70/fb8 [0,4194304] 0 2024-04-12T20:56:20.165 INFO:tasks.workunit.client.0.smithi018.stdout:7/799: rmdir d1/db/d31/d3a/d42/d4c/d6c 39 2024-04-12T20:56:20.168 INFO:tasks.workunit.client.0.smithi018.stdout:5/870: write d0/d2/d33/d10/d60/dfa/d8f/fd5 [407387,55182] 0 2024-04-12T20:56:20.173 INFO:tasks.workunit.client.0.smithi018.stdout:5/871: write d0/d2/d33/d10/f39 [4156753,81763] 0 2024-04-12T20:56:20.173 INFO:tasks.workunit.client.0.smithi018.stdout:3/826: link d2/d6/d30/d31/d38/d3a/d76/c8b d2/d43/d61/c107 0 2024-04-12T20:56:20.177 INFO:tasks.workunit.client.0.smithi018.stdout:7/800: rename d1/db/d31/d45/d8c/d9d/d70/d96/d5f/lf9 to d1/db/d31/df2/l104 0 2024-04-12T20:56:20.177 INFO:tasks.workunit.client.0.smithi018.stdout:3/827: chown d2/d43/d61/d64/dbf/d70/fef 178840657 1 2024-04-12T20:56:20.177 INFO:tasks.workunit.client.0.smithi018.stdout:8/973: sync 2024-04-12T20:56:20.180 INFO:tasks.workunit.client.0.smithi018.stdout:5/872: symlink d0/d2/d33/d10/d60/dfa/dbf/da3/l12a 0 2024-04-12T20:56:20.187 INFO:tasks.workunit.client.0.smithi018.stdout:8/974: read - db/d4e/d56/d59/d76/da5/d45/da9/f10d zero size 2024-04-12T20:56:20.190 INFO:tasks.workunit.client.0.smithi018.stdout:3/828: link d2/d6/d30/d31/l32 d2/d6/d30/d31/d38/dfb/l108 0 2024-04-12T20:56:20.190 INFO:tasks.workunit.client.0.smithi018.stdout:7/801: dread d1/d6/f67 [0,4194304] 0 2024-04-12T20:56:20.194 INFO:tasks.workunit.client.0.smithi018.stdout:3/829: rmdir d2/d6/d34/da1/dac 39 2024-04-12T20:56:20.199 INFO:tasks.workunit.client.0.smithi018.stdout:5/873: dread d0/d2/d33/d10/d60/dfa/dbf/da3/fe3 [0,4194304] 0 2024-04-12T20:56:20.203 INFO:tasks.workunit.client.0.smithi018.stdout:3/830: rename d2/d43/d61/d64/dbf to d2/d6/d30/d31/d38/d3a/dd0/d109 0 2024-04-12T20:56:20.206 INFO:tasks.workunit.client.0.smithi018.stdout:5/874: symlink d0/d2/d33/d10/d60/dfa/dbf/da3/l12b 0 2024-04-12T20:56:20.209 INFO:tasks.workunit.client.0.smithi018.stdout:8/975: write db/d4e/d56/d59/d76/da5/f12 [346710,33181] 0 2024-04-12T20:56:20.211 INFO:tasks.workunit.client.0.smithi018.stdout:5/875: chown d0/d2/d11/d37/dc2/d89/fc7 448043476 1 2024-04-12T20:56:20.217 INFO:tasks.workunit.client.0.smithi018.stdout:5/876: symlink d0/d2/d33/d10/d60/dfa/l12c 0 2024-04-12T20:56:20.223 INFO:tasks.workunit.client.0.smithi018.stdout:5/877: creat d0/d2/d33/d10/d60/dfa/dbf/da3/f12d x:0 0 0 2024-04-12T20:56:20.229 INFO:tasks.workunit.client.0.smithi018.stdout:3/831: dwrite d2/d4d/f57 [0,4194304] 0 2024-04-12T20:56:20.237 INFO:tasks.workunit.client.0.smithi018.stdout:8/976: dwrite db/d4e/d9a/f60 [4194304,4194304] 0 2024-04-12T20:56:20.242 INFO:tasks.workunit.client.0.smithi018.stdout:3/832: mkdir d2/d6/d30/d31/d38/d3a/d76/da2/d10a 0 2024-04-12T20:56:20.242 INFO:tasks.workunit.client.0.smithi018.stdout:8/977: chown db/d4e/d7f/f8c 169 1 2024-04-12T20:56:20.248 INFO:tasks.workunit.client.0.smithi018.stdout:3/833: creat d2/d83/f10b x:0 0 0 2024-04-12T20:56:20.255 INFO:tasks.workunit.client.0.smithi018.stdout:8/978: rename db/d4e/d56/d59/d76/da5/d45/d86/da1/fe9 to db/d4e/d56/d59/d76/de0/d27/df8/f139 0 2024-04-12T20:56:20.256 INFO:tasks.workunit.client.0.smithi018.stdout:3/834: fsync d2/d6/d30/d31/d38/d3a/d48/f104 0 2024-04-12T20:56:20.265 INFO:tasks.workunit.client.0.smithi018.stdout:5/878: dread d0/d2/d33/d10/f39 [0,4194304] 0 2024-04-12T20:56:20.265 INFO:tasks.workunit.client.0.smithi018.stdout:3/835: rmdir d2/d6/d30/d31/d38/d3a/d48 39 2024-04-12T20:56:20.278 INFO:tasks.workunit.client.0.smithi018.stdout:5/879: dread d0/d2/d33/d46/d1e/d2e/d7f/f8c [0,4194304] 0 2024-04-12T20:56:20.280 INFO:tasks.workunit.client.0.smithi018.stdout:8/979: dwrite db/d4e/d56/d59/d76/da5/d45/d86/dbb/f120 [0,4194304] 0 2024-04-12T20:56:20.281 INFO:tasks.workunit.client.0.smithi018.stdout:5/880: fsync d0/d2/d33/d10/d60/dfa/dbf/da3/ffd 0 2024-04-12T20:56:20.285 INFO:tasks.workunit.client.0.smithi018.stdout:8/980: write db/d4e/d56/d59/d76/f11f [813851,6584] 0 2024-04-12T20:56:20.288 INFO:tasks.workunit.client.0.smithi018.stdout:8/981: write f0 [5604756,79437] 0 2024-04-12T20:56:20.293 INFO:tasks.workunit.client.0.smithi018.stdout:5/881: creat d0/d2/d33/d10/d60/dfa/dbf/da3/f12e x:0 0 0 2024-04-12T20:56:20.297 INFO:tasks.workunit.client.0.smithi018.stdout:5/882: mkdir d0/d5b/d12f 0 2024-04-12T20:56:20.300 INFO:tasks.workunit.client.0.smithi018.stdout:3/836: dread d2/d6/d30/d31/f67 [0,4194304] 0 2024-04-12T20:56:20.305 INFO:tasks.workunit.client.0.smithi018.stdout:5/883: creat d0/d2/d11/f130 x:0 0 0 2024-04-12T20:56:20.305 INFO:tasks.workunit.client.0.smithi018.stdout:3/837: fdatasync d2/d6/d34/da1/ddb/fe5 0 2024-04-12T20:56:20.307 INFO:tasks.workunit.client.0.smithi018.stdout:5/884: chown d0/d2/d33/d10/d60/dfa/d8f/fb2 28404945 1 2024-04-12T20:56:20.310 INFO:tasks.workunit.client.0.smithi018.stdout:3/838: unlink d2/d6/f1b 0 2024-04-12T20:56:20.315 INFO:tasks.workunit.client.0.smithi018.stdout:3/839: symlink d2/d43/l10c 0 2024-04-12T20:56:20.317 INFO:tasks.workunit.client.0.smithi018.stdout:5/885: rename d0/d2/d11/d37/dc2/cf0 to d0/d2/de4/c131 0 2024-04-12T20:56:20.318 INFO:tasks.workunit.client.0.smithi018.stdout:8/982: dwrite db/d4e/d56/d59/d76/de0/d27/df8/d7b/f128 [0,4194304] 0 2024-04-12T20:56:20.321 INFO:tasks.workunit.client.0.smithi018.stdout:3/840: fdatasync d2/d83/fce 0 2024-04-12T20:56:20.322 INFO:tasks.workunit.client.0.smithi018.stdout:5/886: truncate d0/d2/d33/d46/d18/fbc 4564697 0 2024-04-12T20:56:20.326 INFO:tasks.workunit.client.0.smithi018.stdout:3/841: chown d2/d6/d30/d31/d38/d3a/f45 166741688 1 2024-04-12T20:56:20.331 INFO:tasks.workunit.client.0.smithi018.stdout:8/983: mkdir db/d4e/d56/d59/d76/da5/d45/da9/ddb/d131/d13a 0 2024-04-12T20:56:20.335 INFO:tasks.workunit.client.1.smithi186.stderr:+ pushd ltp-full-20091231/testcases/kernel/fs/fsstress 2024-04-12T20:56:20.335 INFO:tasks.workunit.client.0.smithi018.stdout:3/842: creat d2/d6/d30/d31/d38/d3a/d48/f10d x:0 0 0 2024-04-12T20:56:20.336 INFO:tasks.workunit.client.0.smithi018.stdout:8/984: write f0 [6535511,121082] 0 2024-04-12T20:56:20.339 INFO:tasks.workunit.client.0.smithi018.stdout:3/843: creat d2/d4d/d8d/f10e x:0 0 0 2024-04-12T20:56:20.343 INFO:tasks.workunit.client.0.smithi018.stdout:3/844: rmdir d2/d4d/d88 39 2024-04-12T20:56:20.348 INFO:tasks.workunit.client.0.smithi018.stdout:5/887: dwrite d0/d2/fa6 [0,4194304] 0 2024-04-12T20:56:20.349 INFO:tasks.workunit.client.0.smithi018.stdout:8/985: rename db/d4e/d7f/dba/lc4 to db/d4e/d56/d59/d76/da5/d45/da9/ddb/d131/d13a/l13b 0 2024-04-12T20:56:20.354 INFO:tasks.workunit.client.0.smithi018.stdout:3/845: creat d2/d6/d34/da1/f10f x:0 0 0 2024-04-12T20:56:20.356 INFO:tasks.workunit.client.0.smithi018.stdout:3/846: write d2/d6/d30/d31/f9c [4702684,91295] 0 2024-04-12T20:56:20.364 INFO:tasks.workunit.client.0.smithi018.stdout:3/847: rename d2/c2a to d2/d83/c110 0 2024-04-12T20:56:20.368 INFO:tasks.workunit.client.0.smithi018.stdout:3/848: creat d2/d6/d30/d31/d38/d3a/d76/da2/de4/f111 x:0 0 0 2024-04-12T20:56:20.371 INFO:tasks.workunit.client.1.smithi186.stdout:~/cephtest/mnt.1/client.1/tmp/fsstress/ltp-full-20091231/testcases/kernel/fs/fsstress ~/cephtest/mnt.1/client.1/tmp/fsstress ~/cephtest/mnt.1/client.1/tmp 2024-04-12T20:56:20.371 INFO:tasks.workunit.client.1.smithi186.stderr:+ make 2024-04-12T20:56:20.430 INFO:tasks.workunit.client.0.smithi018.stdout:3/849: chown d2/d6/d34/d71/db2/dfc/le3 250 1 2024-04-12T20:56:20.432 INFO:tasks.workunit.client.0.smithi018.stdout:5/888: dwrite d0/d2/d33/d46/f13 [0,4194304] 0 2024-04-12T20:56:20.435 INFO:tasks.workunit.client.0.smithi018.stdout:3/850: chown d2/d43/d61/f63 204221292 1 2024-04-12T20:56:20.437 INFO:tasks.workunit.client.0.smithi018.stdout:3/851: dread - d2/d43/db6/db5/fff zero size 2024-04-12T20:56:20.438 INFO:tasks.workunit.client.0.smithi018.stdout:5/889: getdents d0/d2b 0 2024-04-12T20:56:20.440 INFO:tasks.workunit.client.0.smithi018.stdout:3/852: write d2/d6/d30/d31/f50 [1313154,116301] 0 2024-04-12T20:56:20.445 INFO:tasks.workunit.client.0.smithi018.stdout:3/853: chown d2/d6/d30/d31/d38/d3a/dd0/d109/d54/f9f 2 1 2024-04-12T20:56:20.448 INFO:tasks.workunit.client.0.smithi018.stdout:3/854: write d2/d6/d34/f59 [3217568,123616] 0 2024-04-12T20:56:20.454 INFO:tasks.workunit.client.0.smithi018.stdout:3/855: creat d2/d6/d30/d31/d38/d3a/d76/da2/de4/f112 x:0 0 0 2024-04-12T20:56:20.458 INFO:tasks.workunit.client.0.smithi018.stdout:5/890: dwrite d0/d2/d33/d46/d18/faf [0,4194304] 0 2024-04-12T20:56:20.460 INFO:tasks.workunit.client.0.smithi018.stdout:3/856: unlink d2/d6/d30/d31/d38/d3a/d48/d82/fb1 0 2024-04-12T20:56:20.463 INFO:tasks.workunit.client.0.smithi018.stdout:5/891: mkdir d0/d2/d33/d10/d60/dfa/dbf/da3/d132 0 2024-04-12T20:56:20.464 INFO:tasks.workunit.client.0.smithi018.stdout:3/857: symlink d2/d6/d30/d31/d38/d3a/d76/da2/de4/l113 0 2024-04-12T20:56:20.469 INFO:tasks.workunit.client.0.smithi018.stdout:3/858: mknod d2/d6/d30/d31/d38/dfb/c114 0 2024-04-12T20:56:20.488 INFO:tasks.workunit.client.0.smithi018.stdout:5/892: write d0/d2/d33/d10/d60/dfa/dbf/da3/dae/fcb [2006553,48550] 0 2024-04-12T20:56:20.492 INFO:tasks.workunit.client.0.smithi018.stdout:5/893: creat d0/d2/d33/d10/d26/f133 x:0 0 0 2024-04-12T20:56:20.496 INFO:tasks.workunit.client.0.smithi018.stdout:3/859: dwrite d2/d4d/d88/f97 [0,4194304] 0 2024-04-12T20:56:20.496 INFO:tasks.workunit.client.1.smithi186.stdout:cc -DNO_XFS -I/home/ubuntu/cephtest/mnt.1/client.1/tmp/fsstress/ltp-full-20091231/testcases/kernel/fs/fsstress -D_LARGEFILE64_SOURCE -D_GNU_SOURCE -I../../../../include -I../../../../include -L../../../../lib fsstress.c -o fsstress 2024-04-12T20:56:20.500 INFO:tasks.workunit.client.0.smithi018.stdout:5/894: rename d0/d2/d33/d61/fd9 to d0/d2/d33/d46/d1e/d2e/d7f/f134 0 2024-04-12T20:56:20.504 INFO:tasks.workunit.client.0.smithi018.stdout:3/860: creat d2/d6/d30/d31/d38/d3a/d76/da2/d10a/f115 x:0 0 0 2024-04-12T20:56:20.506 INFO:tasks.workunit.client.0.smithi018.stdout:5/895: creat d0/d2/d33/d46/d35/f135 x:0 0 0 2024-04-12T20:56:20.512 INFO:tasks.workunit.client.0.smithi018.stdout:3/861: creat d2/d6/d30/d31/d38/d3a/d76/da2/de4/d105/f116 x:0 0 0 2024-04-12T20:56:20.513 INFO:tasks.workunit.client.0.smithi018.stdout:5/896: mkdir d0/d2/d11/d37/dc2/d89/d8d/d136 0 2024-04-12T20:56:20.520 INFO:tasks.workunit.client.0.smithi018.stdout:5/897: creat d0/d2/d33/d46/d1e/d55/d66/f137 x:0 0 0 2024-04-12T20:56:20.523 INFO:tasks.workunit.client.0.smithi018.stdout:5/898: write d0/d2/d33/d10/d60/dfa/ff3 [873328,69629] 0 2024-04-12T20:56:20.534 INFO:tasks.workunit.client.0.smithi018.stdout:3/862: dwrite d2/d6/f4e [0,4194304] 0 2024-04-12T20:56:20.534 INFO:tasks.workunit.client.0.smithi018.stdout:5/899: dread d0/d2/d33/d10/d26/f43 [0,4194304] 0 2024-04-12T20:56:20.538 INFO:tasks.workunit.client.0.smithi018.stdout:3/863: readlink d2/d6/d30/d31/l77 0 2024-04-12T20:56:20.541 INFO:tasks.workunit.client.0.smithi018.stdout:5/900: link d0/d2/d11/d37/dc2/d89/fc7 d0/d2/d33/d46/f138 0 2024-04-12T20:56:20.541 INFO:tasks.workunit.client.0.smithi018.stdout:3/864: dread - d2/d6/d30/d31/d38/d3a/d76/da2/de4/d105/f116 zero size 2024-04-12T20:56:20.549 INFO:tasks.workunit.client.0.smithi018.stdout:5/901: write d0/d2/d33/d46/d35/d4d/db3/dc3/fcd [909115,81275] 0 2024-04-12T20:56:20.553 INFO:tasks.workunit.client.0.smithi018.stdout:5/902: symlink d0/d2/d11/d37/dc2/d89/d8d/l139 0 2024-04-12T20:56:20.559 INFO:tasks.workunit.client.0.smithi018.stdout:5/903: truncate d0/d2/d33/d46/f3a 2767108 0 2024-04-12T20:56:20.559 INFO:tasks.workunit.client.0.smithi018.stdout:3/865: dwrite d2/d43/db6/db5/fff [0,4194304] 0 2024-04-12T20:56:20.561 INFO:tasks.workunit.client.0.smithi018.stdout:3/866: readlink d2/d43/l72 0 2024-04-12T20:56:20.561 INFO:tasks.workunit.client.0.smithi018.stdout:3/867: chown d2/d6/d34 1825661 1 2024-04-12T20:56:20.563 INFO:tasks.workunit.client.0.smithi018.stdout:5/904: read d0/d2/d11/d37/dc2/f86 [1461060,55641] 0 2024-04-12T20:56:20.565 INFO:tasks.workunit.client.0.smithi018.stdout:3/868: fdatasync d2/d4d/d88/fda 0 2024-04-12T20:56:20.570 INFO:tasks.workunit.client.0.smithi018.stdout:5/905: truncate d0/d2/d33/d46/d35/d4d/f80 939295 0 2024-04-12T20:56:20.578 INFO:tasks.workunit.client.0.smithi018.stdout:5/906: mkdir d0/d2/d33/d10/d4e/d13a 0 2024-04-12T20:56:20.579 INFO:tasks.workunit.client.0.smithi018.stdout:3/869: truncate d2/d43/db6/db5/fff 398617 0 2024-04-12T20:56:20.580 INFO:tasks.workunit.client.0.smithi018.stdout:6/932: sync 2024-04-12T20:56:20.580 INFO:tasks.workunit.client.0.smithi018.stdout:8/986: sync 2024-04-12T20:56:20.580 INFO:tasks.workunit.client.0.smithi018.stdout:0/742: sync 2024-04-12T20:56:20.580 INFO:tasks.workunit.client.0.smithi018.stdout:2/755: sync 2024-04-12T20:56:20.580 INFO:tasks.workunit.client.0.smithi018.stdout:1/672: sync 2024-04-12T20:56:20.581 INFO:tasks.workunit.client.0.smithi018.stdout:4/749: sync 2024-04-12T20:56:20.581 INFO:tasks.workunit.client.0.smithi018.stdout:7/802: sync 2024-04-12T20:56:20.582 INFO:tasks.workunit.client.0.smithi018.stdout:5/907: chown d0/d2/d33/d10/d60/dfa/dbf/da3/d132 7 1 2024-04-12T20:56:20.587 INFO:tasks.workunit.client.0.smithi018.stdout:1/673: chown d3/d9/da/dd/f3f 7417410 1 2024-04-12T20:56:20.588 INFO:tasks.workunit.client.0.smithi018.stdout:2/756: write d5/d8/d10/d43/f56 [2179295,59816] 0 2024-04-12T20:56:20.589 INFO:tasks.workunit.client.0.smithi018.stdout:0/743: readlink da/d15/d17/d21/d36/da2/dc3/d3f/lbd 0 2024-04-12T20:56:20.590 INFO:tasks.workunit.client.0.smithi018.stdout:4/750: unlink d3/d5d/d60/dc5/d43/d4b/f83 0 2024-04-12T20:56:20.590 INFO:tasks.workunit.client.0.smithi018.stdout:8/987: mkdir db/d4e/d56/d59/d76/da5/d45/d86/dbb/d13c 0 2024-04-12T20:56:20.593 INFO:tasks.workunit.client.0.smithi018.stdout:2/757: write d5/d8/d25/f7d [535337,24515] 0 2024-04-12T20:56:20.595 INFO:tasks.workunit.client.0.smithi018.stdout:6/933: getdents d5/d6/d1b/d3b/d6a 0 2024-04-12T20:56:20.595 INFO:tasks.workunit.client.0.smithi018.stdout:5/908: rename d0/d2b/cc4 to d0/d2b/c13b 0 2024-04-12T20:56:20.596 INFO:tasks.workunit.client.0.smithi018.stdout:7/803: truncate d1/db/d31/d3a/d42/f8f 652791 0 2024-04-12T20:56:20.596 INFO:tasks.workunit.client.0.smithi018.stdout:4/751: fdatasync d3/d5d/d60/dc5/d3d/f65 0 2024-04-12T20:56:20.598 INFO:tasks.workunit.client.0.smithi018.stdout:0/744: symlink da/d15/d17/d71/lf9 0 2024-04-12T20:56:20.602 INFO:tasks.workunit.client.0.smithi018.stdout:5/909: chown d0/d2/d33/d10/d60/dfa/c8e 777 1 2024-04-12T20:56:20.606 INFO:tasks.workunit.client.0.smithi018.stdout:2/758: creat d5/d8/d10/d43/de2/d24/d39/dab/f105 x:0 0 0 2024-04-12T20:56:20.607 INFO:tasks.workunit.client.0.smithi018.stdout:2/759: chown d5/c2f 11 1 2024-04-12T20:56:20.607 INFO:tasks.workunit.client.0.smithi018.stdout:8/988: truncate db/d4e/d56/d59/d76/de0/fa3 466590 0 2024-04-12T20:56:20.609 INFO:tasks.workunit.client.0.smithi018.stdout:5/910: fsync d0/d2/d33/d10/d60/dfa/dbf/da3/ffd 0 2024-04-12T20:56:20.609 INFO:tasks.workunit.client.0.smithi018.stdout:6/934: creat d5/d6/d1b/d3b/d22/d117/d13c/d154/d4f/f15b x:0 0 0 2024-04-12T20:56:20.612 INFO:tasks.workunit.client.0.smithi018.stdout:7/804: rmdir d1/db/d31/d45/d8c/d9d/d70/d96 39 2024-04-12T20:56:20.617 INFO:tasks.workunit.client.0.smithi018.stdout:3/870: dread d2/d83/f92 [0,4194304] 0 2024-04-12T20:56:20.618 INFO:tasks.workunit.client.0.smithi018.stdout:5/911: truncate d0/d2/d33/d10/d60/dfa/dbf/da3/f12d 438766 0 2024-04-12T20:56:20.620 INFO:tasks.workunit.client.0.smithi018.stdout:2/760: mknod d5/d8/d10/c106 0 2024-04-12T20:56:20.620 INFO:tasks.workunit.client.0.smithi018.stdout:3/871: fdatasync d2/d4d/d88/f97 0 2024-04-12T20:56:20.629 INFO:tasks.workunit.client.0.smithi018.stdout:3/872: write d2/d6/d30/d31/d38/d3a/d76/da2/de4/ffe [824316,72981] 0 2024-04-12T20:56:20.632 INFO:tasks.workunit.client.0.smithi018.stdout:8/989: mkdir db/d4e/d56/d59/d76/da5/d45/d86/d13d 0 2024-04-12T20:56:20.636 INFO:tasks.workunit.client.0.smithi018.stdout:1/674: dwrite d3/d9/da/dd/d31/d32/d13/d19/f62 [0,4194304] 0 2024-04-12T20:56:20.636 INFO:tasks.workunit.client.0.smithi018.stdout:2/761: mkdir d5/d7/d31/d107 0 2024-04-12T20:56:20.641 INFO:tasks.workunit.client.0.smithi018.stdout:7/805: symlink d1/db/d31/d45/d8c/d9d/d70/d96/l105 0 2024-04-12T20:56:20.646 INFO:tasks.workunit.client.0.smithi018.stdout:4/752: getdents d3/d5d/d60/dc5/d43/d4b 0 2024-04-12T20:56:20.649 INFO:tasks.workunit.client.0.smithi018.stdout:0/745: dwrite da/d15/d17/d21/d36/d47/d81/fec [0,4194304] 0 2024-04-12T20:56:20.649 INFO:tasks.workunit.client.0.smithi018.stdout:7/806: chown d1/db/d31/d3a/d42/d4c/d103 11195 1 2024-04-12T20:56:20.650 INFO:tasks.workunit.client.0.smithi018.stdout:2/762: creat d5/d8/d10/d43/de2/d24/d39/f108 x:0 0 0 2024-04-12T20:56:20.652 INFO:tasks.workunit.client.0.smithi018.stdout:5/912: rename d0/d2/d11/d37/dc2/d89/d8d/de9 to d0/d2/d11/d37/d13c 0 2024-04-12T20:56:20.653 INFO:tasks.workunit.client.0.smithi018.stdout:2/763: stat d5/d7/d79/f8c 0 2024-04-12T20:56:20.654 INFO:tasks.workunit.client.0.smithi018.stdout:4/753: mkdir d3/db/df6 0 2024-04-12T20:56:20.661 INFO:tasks.workunit.client.0.smithi018.stdout:1/675: rename d3/d9/da/dd/d31/d32/lb1 to d3/d9/da/dd/d31/d32/d33/le5 0 2024-04-12T20:56:20.661 INFO:tasks.workunit.client.0.smithi018.stdout:0/746: rmdir da/d15/d17/d21/d36/d6e 39 2024-04-12T20:56:20.665 INFO:tasks.workunit.client.0.smithi018.stdout:1/676: stat d3/d9/da/dd/d31/f35 0 2024-04-12T20:56:20.667 INFO:tasks.workunit.client.0.smithi018.stdout:0/747: write da/f22 [1754429,3160] 0 2024-04-12T20:56:20.670 INFO:tasks.workunit.client.0.smithi018.stdout:2/764: unlink d5/d8/f1e 0 2024-04-12T20:56:20.672 INFO:tasks.workunit.client.0.smithi018.stdout:3/873: dwrite d2/d6/d34/d71/dbc/ff4 [0,4194304] 0 2024-04-12T20:56:20.673 INFO:tasks.workunit.client.0.smithi018.stdout:4/754: creat d3/d5d/d60/dc5/d3d/d80/ff7 x:0 0 0 2024-04-12T20:56:20.676 INFO:tasks.workunit.client.0.smithi018.stdout:0/748: creat da/d15/d17/d71/ffa x:0 0 0 2024-04-12T20:56:20.680 INFO:tasks.workunit.client.0.smithi018.stdout:7/807: link d1/d6/dc9/fec d1/db/d31/d3a/d42/d4c/d6c/dc3/f106 0 2024-04-12T20:56:20.680 INFO:tasks.workunit.client.0.smithi018.stdout:1/677: chown d3/d9/da/dd/d31/d32/d33/f6d 0 1 2024-04-12T20:56:20.683 INFO:tasks.workunit.client.0.smithi018.stdout:6/935: dwrite d5/d6/d1b/d3b/ffc [0,4194304] 0 2024-04-12T20:56:20.692 INFO:tasks.workunit.client.0.smithi018.stdout:6/936: truncate d5/d6/d90/d125/d139/f149 947705 0 2024-04-12T20:56:20.696 INFO:tasks.workunit.client.0.smithi018.stdout:8/990: dread db/d4e/d56/d59/d76/da5/d11a/d12c/f34 [4194304,4194304] 0 2024-04-12T20:56:20.698 INFO:tasks.workunit.client.0.smithi018.stdout:1/678: mknod d3/d65/ce6 0 2024-04-12T20:56:20.698 INFO:tasks.workunit.client.0.smithi018.stdout:5/913: write d0/d2/fbe [1266888,90091] 0 2024-04-12T20:56:20.703 INFO:tasks.workunit.client.0.smithi018.stdout:3/874: mknod d2/d6/d34/da1/dac/c117 0 2024-04-12T20:56:20.704 INFO:tasks.workunit.client.0.smithi018.stdout:2/765: creat d5/d7/d31/d107/f109 x:0 0 0 2024-04-12T20:56:20.707 INFO:tasks.workunit.client.0.smithi018.stdout:1/679: fsync d3/d9/da/dd/d31/d32/f1b 0 2024-04-12T20:56:20.709 INFO:tasks.workunit.client.0.smithi018.stdout:4/755: write d3/db/f24 [2906151,43498] 0 2024-04-12T20:56:20.716 INFO:tasks.workunit.client.0.smithi018.stdout:2/766: unlink d5/d8/d10/d43/lc9 0 2024-04-12T20:56:20.718 INFO:tasks.workunit.client.0.smithi018.stdout:3/875: chown d2/l10 14144875 1 2024-04-12T20:56:20.719 INFO:tasks.workunit.client.0.smithi018.stdout:0/749: write da/d15/d17/d21/d36/f7c [1402206,82710] 0 2024-04-12T20:56:20.724 INFO:tasks.workunit.client.0.smithi018.stdout:5/914: truncate d0/d2/d11/d37/f47 802301 0 2024-04-12T20:56:20.724 INFO:tasks.workunit.client.0.smithi018.stdout:7/808: dwrite d1/db/d31/f63 [0,4194304] 0 2024-04-12T20:56:20.726 INFO:tasks.workunit.client.0.smithi018.stdout:3/876: symlink d2/d6/d34/l118 0 2024-04-12T20:56:20.730 INFO:tasks.workunit.client.0.smithi018.stdout:6/937: dwrite d5/d15/d60/f65 [8388608,4194304] 0 2024-04-12T20:56:20.735 INFO:tasks.workunit.client.0.smithi018.stdout:4/756: mknod d3/d5d/d60/dc5/d3d/cf8 0 2024-04-12T20:56:20.737 INFO:tasks.workunit.client.0.smithi018.stdout:0/750: mkdir da/d15/d17/d71/dc1/dfb 0 2024-04-12T20:56:20.737 INFO:tasks.workunit.client.0.smithi018.stdout:3/877: mknod d2/d6/d34/c119 0 2024-04-12T20:56:20.741 INFO:tasks.workunit.client.0.smithi018.stdout:5/915: rename d0/d2/l21 to d0/d2/d33/d46/d18/l13d 0 2024-04-12T20:56:20.742 INFO:tasks.workunit.client.0.smithi018.stdout:0/751: fsync da/d15/d17/d21/d94/fc7 0 2024-04-12T20:56:20.742 INFO:tasks.workunit.client.0.smithi018.stdout:7/809: chown d1/db/d31/d30/l50 208086422 1 2024-04-12T20:56:20.744 INFO:tasks.workunit.client.0.smithi018.stdout:3/878: read d2/d6/d34/d71/dbc/ff4 [770645,94191] 0 2024-04-12T20:56:20.745 INFO:tasks.workunit.client.0.smithi018.stdout:0/752: fdatasync da/faa 0 2024-04-12T20:56:20.746 INFO:tasks.workunit.client.0.smithi018.stdout:1/680: rmdir d3/d9/da/dd/d31/d32/d13/d19/d2d/d4e/d58/d61/d71/da8 0 2024-04-12T20:56:20.749 INFO:tasks.workunit.client.0.smithi018.stdout:6/938: link d5/df/d19/f136 d5/d6/d1b/d3b/d22/d117/f15c 0 2024-04-12T20:56:20.750 INFO:tasks.workunit.client.0.smithi018.stdout:7/810: dread - d1/d6/dc9/fec zero size 2024-04-12T20:56:20.759 INFO:tasks.workunit.client.0.smithi018.stdout:4/757: dread d3/db/f7f [0,4194304] 0 2024-04-12T20:56:20.760 INFO:tasks.workunit.client.0.smithi018.stdout:3/879: mkdir d2/d6/d34/d71/db2/dfc/d11a 0 2024-04-12T20:56:20.761 INFO:tasks.workunit.client.0.smithi018.stdout:0/753: chown da/d15/d17/d21/d36/d47/fc8 1929515 1 2024-04-12T20:56:20.761 INFO:tasks.workunit.client.0.smithi018.stdout:6/939: creat d5/d15/d60/f15d x:0 0 0 2024-04-12T20:56:20.762 INFO:tasks.workunit.client.0.smithi018.stdout:8/991: dread db/d4e/f6b [0,4194304] 0 2024-04-12T20:56:20.762 INFO:tasks.workunit.client.0.smithi018.stdout:2/767: dwrite d5/d8/d10/d43/fd8 [0,4194304] 0 2024-04-12T20:56:20.764 INFO:tasks.workunit.client.0.smithi018.stdout:6/940: fsync d5/f20 0 2024-04-12T20:56:20.768 INFO:tasks.workunit.client.0.smithi018.stdout:1/681: truncate d3/d9/da/dd/d26/fa6 4509757 0 2024-04-12T20:56:20.772 INFO:tasks.workunit.client.0.smithi018.stdout:2/768: write d5/d12/d54/d3e/f102 [196130,18598] 0 2024-04-12T20:56:20.776 INFO:tasks.workunit.client.0.smithi018.stdout:3/880: write d2/d6/d30/d31/d38/d3a/d76/da2/d10a/f115 [482097,24490] 0 2024-04-12T20:56:20.777 INFO:tasks.workunit.client.0.smithi018.stdout:2/769: chown d5/d7/d79/f8c 0 1 2024-04-12T20:56:20.780 INFO:tasks.workunit.client.0.smithi018.stdout:6/941: symlink d5/d6/d1b/d3b/l15e 0 2024-04-12T20:56:20.783 INFO:tasks.workunit.client.0.smithi018.stdout:2/770: write d5/d7/ffb [347746,127302] 0 2024-04-12T20:56:20.784 INFO:tasks.workunit.client.0.smithi018.stdout:5/916: truncate d0/d2/d11/d37/dc2/f86 3757945 0 2024-04-12T20:56:20.786 INFO:tasks.workunit.client.0.smithi018.stdout:0/754: write da/d15/f20 [6987799,30408] 0 2024-04-12T20:56:20.790 INFO:tasks.workunit.client.0.smithi018.stdout:1/682: creat d3/d9/da/dd/d31/d32/d13/d19/d2d/d48/fe7 x:0 0 0 2024-04-12T20:56:20.798 INFO:tasks.workunit.client.0.smithi018.stdout:6/942: fdatasync d5/d6/d1b/d3b/d22/d117/d13c/fff 0 2024-04-12T20:56:20.798 INFO:tasks.workunit.client.0.smithi018.stdout:4/758: dwrite d3/d32/fbe [0,4194304] 0 2024-04-12T20:56:20.799 INFO:tasks.workunit.client.0.smithi018.stdout:3/881: dread d2/d6/d30/d31/fa4 [0,4194304] 0 2024-04-12T20:56:20.803 INFO:tasks.workunit.client.0.smithi018.stdout:2/771: truncate d5/d8/fa7 397975 0 2024-04-12T20:56:20.803 INFO:tasks.workunit.client.0.smithi018.stdout:6/943: dread - d5/df/d19/d120/f14d zero size 2024-04-12T20:56:20.805 INFO:tasks.workunit.client.0.smithi018.stdout:5/917: fsync d0/d2/d33/d46/d1e/d2e/f9c 0 2024-04-12T20:56:20.812 INFO:tasks.workunit.client.0.smithi018.stdout:2/772: fsync d5/d7/d31/d62/da6/f100 0 2024-04-12T20:56:20.815 INFO:tasks.workunit.client.0.smithi018.stdout:5/918: dread - d0/d2/d33/d46/d35/f117 zero size 2024-04-12T20:56:20.816 INFO:tasks.workunit.client.0.smithi018.stdout:6/944: creat d5/d6/d1b/d3b/d22/d117/f15f x:0 0 0 2024-04-12T20:56:20.817 INFO:tasks.workunit.client.0.smithi018.stdout:2/773: stat f4 0 2024-04-12T20:56:20.817 INFO:tasks.workunit.client.0.smithi018.stdout:0/755: getdents da/d15/d17/d71/dc1/dcc/dcb 0 2024-04-12T20:56:20.818 INFO:tasks.workunit.client.0.smithi018.stdout:3/882: mknod d2/d6/d30/d31/d38/d3a/dd0/d109/d54/c11b 0 2024-04-12T20:56:20.820 INFO:tasks.workunit.client.0.smithi018.stdout:8/992: dwrite db/d4e/d56/d59/d76/fbe [0,4194304] 0 2024-04-12T20:56:20.825 INFO:tasks.workunit.client.0.smithi018.stdout:5/919: mknod d0/d2/d33/d46/d9e/c13e 0 2024-04-12T20:56:20.827 INFO:tasks.workunit.client.0.smithi018.stdout:6/945: dread - d5/d15/d60/dbb/ffe zero size 2024-04-12T20:56:20.827 INFO:tasks.workunit.client.0.smithi018.stdout:2/774: rename d5/ld4 to d5/d8/d10/d43/de2/d24/l10a 0 2024-04-12T20:56:20.828 INFO:tasks.workunit.client.0.smithi018.stdout:8/993: dread - db/d4e/d56/d59/d76/da5/d45/f12d zero size 2024-04-12T20:56:20.832 INFO:tasks.workunit.client.0.smithi018.stdout:3/883: link d2/d4d/d8d/fab d2/d6/d30/d31/d38/d3a/d76/da2/de4/f11c 0 2024-04-12T20:56:20.839 INFO:tasks.workunit.client.0.smithi018.stdout:5/920: symlink d0/d2/d33/d46/d1e/d2e/d7f/l13f 0 2024-04-12T20:56:20.845 INFO:tasks.workunit.client.0.smithi018.stdout:6/946: mkdir d5/df/d19/d43/de6/d160 0 2024-04-12T20:56:20.857 INFO:tasks.workunit.client.0.smithi018.stdout:3/884: dread d2/d6/d30/d31/d38/d3a/dd0/d109/fdd [0,4194304] 0 2024-04-12T20:56:20.859 INFO:tasks.workunit.client.0.smithi018.stdout:4/759: dwrite d3/d32/d33/d8e/fd8 [0,4194304] 0 2024-04-12T20:56:20.860 INFO:tasks.workunit.client.0.smithi018.stdout:2/775: dwrite d5/d8/d10/d43/de2/d7c/fbf [0,4194304] 0 2024-04-12T20:56:20.866 INFO:tasks.workunit.client.0.smithi018.stdout:0/756: dwrite da/d15/d4d/d58/fc5 [0,4194304] 0 2024-04-12T20:56:20.869 INFO:tasks.workunit.client.0.smithi018.stdout:6/947: write d5/d6/d1b/d3b/f3d [1866447,105332] 0 2024-04-12T20:56:20.869 INFO:tasks.workunit.client.0.smithi018.stdout:5/921: rename d0/d2/d33/d10/d60/dfa/dbf/fe6 to d0/d5b/d84/d83/f140 0 2024-04-12T20:56:20.871 INFO:tasks.workunit.client.0.smithi018.stdout:4/760: mknod d3/db/cf9 0 2024-04-12T20:56:20.875 INFO:tasks.workunit.client.0.smithi018.stdout:3/885: mknod d2/d6/d30/d31/d38/d3a/dd0/de1/c11d 0 2024-04-12T20:56:20.878 INFO:tasks.workunit.client.0.smithi018.stdout:8/994: dread db/f15 [0,4194304] 0 2024-04-12T20:56:20.880 INFO:tasks.workunit.client.0.smithi018.stdout:3/886: read - d2/d43/d61/d64/ff0 zero size 2024-04-12T20:56:20.880 INFO:tasks.workunit.client.0.smithi018.stdout:6/948: chown d5/d6/d1b/d3b/d22/d117/d13c/d154/f33 12793 1 2024-04-12T20:56:20.880 INFO:tasks.workunit.client.0.smithi018.stdout:2/776: write d5/d8/d10/d43/de2/daf/d88/fb5 [1426241,66875] 0 2024-04-12T20:56:20.882 INFO:tasks.workunit.client.0.smithi018.stdout:0/757: rename da/d15/d17/d21/d36/d47/fc8 to da/d15/d4d/dd0/ffc 0 2024-04-12T20:56:20.886 INFO:tasks.workunit.client.0.smithi018.stdout:8/995: mknod db/d4e/d56/d59/d76/da5/d45/c13e 0 2024-04-12T20:56:20.887 INFO:tasks.workunit.client.0.smithi018.stdout:6/949: stat d5/d6/d1b/d8c/l11a 0 2024-04-12T20:56:20.888 INFO:tasks.workunit.client.0.smithi018.stdout:8/996: chown db/d3b/f55 379421 1 2024-04-12T20:56:20.889 INFO:tasks.workunit.client.0.smithi018.stdout:2/777: stat d5/d8/cb 0 2024-04-12T20:56:20.894 INFO:tasks.workunit.client.0.smithi018.stdout:4/761: unlink d3/d32/d33/db6/fbb 0 2024-04-12T20:56:20.895 INFO:tasks.workunit.client.0.smithi018.stdout:8/997: chown db/d4e/d56/d59/d76/de0/d27/fd8 1610306 1 2024-04-12T20:56:20.898 INFO:tasks.workunit.client.0.smithi018.stdout:3/887: getdents d2/d6/d34/d71/d7b 0 2024-04-12T20:56:20.898 INFO:tasks.workunit.client.0.smithi018.stdout:5/922: dwrite d0/d2/d11/f108 [0,4194304] 0 2024-04-12T20:56:20.904 INFO:tasks.workunit.client.0.smithi018.stdout:4/762: write d3/d5d/d60/dc5/d43/d4b/ddc/fe1 [415557,90302] 0 2024-04-12T20:56:20.904 INFO:tasks.workunit.client.0.smithi018.stdout:6/950: link d5/d15/d60/dbb/l13e d5/l161 0 2024-04-12T20:56:20.906 INFO:tasks.workunit.client.0.smithi018.stdout:8/998: creat db/d95/dd6/d117/f13f x:0 0 0 2024-04-12T20:56:20.907 INFO:tasks.workunit.client.0.smithi018.stdout:5/923: rmdir d0/d2/d33/d46/d9e 39 2024-04-12T20:56:20.912 INFO:tasks.workunit.client.0.smithi018.stdout:6/951: readlink d5/d71/le0 0 2024-04-12T20:56:20.913 INFO:tasks.workunit.client.0.smithi018.stdout:0/758: getdents da/d15/d17/d21 0 2024-04-12T20:56:20.917 INFO:tasks.workunit.client.0.smithi018.stdout:4/763: symlink d3/d5d/d60/dc5/dcc/ddf/lfa 0 2024-04-12T20:56:20.919 INFO:tasks.workunit.client.0.smithi018.stdout:5/924: creat d0/d2/d33/d46/d1e/d9a/f141 x:0 0 0 2024-04-12T20:56:20.921 INFO:tasks.workunit.client.0.smithi018.stdout:6/952: mkdir d5/d6/d1b/d3b/d6a/d116/d13f/d162 0 2024-04-12T20:56:20.923 INFO:tasks.workunit.client.0.smithi018.stdout:3/888: rename d2/d6/d30/d31/d38/d3a/dd0/d109/d54/f93 to d2/d6/d95/f11e 0 2024-04-12T20:56:20.925 INFO:tasks.workunit.client.0.smithi018.stdout:0/759: symlink da/d15/d17/d21/d36/d47/d81/lfd 0 2024-04-12T20:56:20.928 INFO:tasks.workunit.client.0.smithi018.stdout:5/925: fdatasync d0/d2/d11/d37/f4c 0 2024-04-12T20:56:20.935 INFO:tasks.workunit.client.0.smithi018.stdout:3/889: creat d2/d6/d34/da1/dac/f11f x:0 0 0 2024-04-12T20:56:20.936 INFO:tasks.workunit.client.0.smithi018.stdout:4/764: fdatasync d3/d32/d33/dbc/fda 0 2024-04-12T20:56:20.938 INFO:tasks.workunit.client.0.smithi018.stdout:6/953: read d5/d15/f6f [55252,93087] 0 2024-04-12T20:56:20.940 INFO:tasks.workunit.client.0.smithi018.stdout:0/760: rename da/d15/d4d/d6b/l9c to da/d15/lfe 0 2024-04-12T20:56:20.945 INFO:tasks.workunit.client.0.smithi018.stdout:2/778: dwrite d5/d12/f58 [0,4194304] 0 2024-04-12T20:56:20.945 INFO:tasks.workunit.client.0.smithi018.stdout:0/761: chown da/d15/d17/d71/dc1/dcc/c31 30 1 2024-04-12T20:56:20.946 INFO:tasks.workunit.client.0.smithi018.stdout:8/999: truncate db/d4e/d56/d59/d76/de0/f53 2559428 0 2024-04-12T20:56:20.949 INFO:tasks.workunit.client.0.smithi018.stdout:3/890: mknod d2/d6/d30/c120 0 2024-04-12T20:56:20.950 INFO:tasks.workunit.client.0.smithi018.stdout:6/954: write d5/d6/d1b/d3b/d8e/ff0 [100400,103195] 0 2024-04-12T20:56:20.953 INFO:tasks.workunit.client.0.smithi018.stdout:0/762: unlink da/d15/d17/d21/d36/d47/c9a 0 2024-04-12T20:56:20.954 INFO:tasks.workunit.client.0.smithi018.stdout:2/779: creat d5/d8/d10/d43/de2/daf/d88/f10b x:0 0 0 2024-04-12T20:56:20.958 INFO:tasks.workunit.client.0.smithi018.stdout:5/926: write d0/d2/d33/d46/d9e/ff5 [998801,60885] 0 2024-04-12T20:56:20.960 INFO:tasks.workunit.client.0.smithi018.stdout:3/891: symlink d2/d6/d34/d71/db2/dfc/l121 0 2024-04-12T20:56:20.963 INFO:tasks.workunit.client.0.smithi018.stdout:2/780: rmdir d5/d7/d31 39 2024-04-12T20:56:20.964 INFO:tasks.workunit.client.0.smithi018.stdout:3/892: write d2/d6/d34/f46 [1418120,70819] 0 2024-04-12T20:56:20.969 INFO:tasks.workunit.client.0.smithi018.stdout:6/955: write d5/df/d19/f136 [108416,24034] 0 2024-04-12T20:56:20.972 INFO:tasks.workunit.client.0.smithi018.stdout:6/956: stat d5/df/d19/faa 0 2024-04-12T20:56:20.976 INFO:tasks.workunit.client.0.smithi018.stdout:4/765: dwrite d3/d5d/d60/dc5/d43/f76 [0,4194304] 0 2024-04-12T20:56:20.983 INFO:tasks.workunit.client.0.smithi018.stdout:2/781: rmdir d5/d12/d54/dbc 39 2024-04-12T20:56:20.983 INFO:tasks.workunit.client.0.smithi018.stdout:3/893: symlink d2/d6/d30/d31/d38/d3a/d76/da2/de4/d105/l122 0 2024-04-12T20:56:20.983 INFO:tasks.workunit.client.0.smithi018.stdout:6/957: truncate d5/df/d31/fd6 4729677 0 2024-04-12T20:56:20.986 INFO:tasks.workunit.client.0.smithi018.stdout:3/894: read d2/d43/f8f [1120730,113291] 0 2024-04-12T20:56:20.986 INFO:tasks.workunit.client.0.smithi018.stdout:5/927: getdents d0/d2/d33/d46/d9e 0 2024-04-12T20:56:20.989 INFO:tasks.workunit.client.0.smithi018.stdout:4/766: creat d3/db/ffb x:0 0 0 2024-04-12T20:56:20.991 INFO:tasks.workunit.client.0.smithi018.stdout:6/958: symlink d5/df/d19/d43/d14a/l163 0 2024-04-12T20:56:20.994 INFO:tasks.workunit.client.0.smithi018.stdout:3/895: creat d2/d6/d30/d31/d38/d3a/dd0/d109/d103/f123 x:0 0 0 2024-04-12T20:56:20.994 INFO:tasks.workunit.client.0.smithi018.stdout:4/767: dread d3/d5d/d60/dc5/d40/f6d [0,4194304] 0 2024-04-12T20:56:20.996 INFO:tasks.workunit.client.0.smithi018.stdout:0/763: dread da/d15/d4d/dd0/ffc [0,4194304] 0 2024-04-12T20:56:20.999 INFO:tasks.workunit.client.0.smithi018.stdout:3/896: unlink d2/d6/d30/d31/f67 0 2024-04-12T20:56:21.000 INFO:tasks.workunit.client.0.smithi018.stdout:4/768: mkdir d3/db/de3/dfc 0 2024-04-12T20:56:21.000 INFO:tasks.workunit.client.0.smithi018.stdout:3/897: stat d2/d43/db6/f80 0 2024-04-12T20:56:21.001 INFO:tasks.workunit.client.0.smithi018.stdout:0/764: truncate da/d15/d17/d21/d36/d47/d81/fa7 556224 0 2024-04-12T20:56:21.002 INFO:tasks.workunit.client.0.smithi018.stdout:4/769: dread - d3/d5d/d60/dc5/d3d/f57 zero size 2024-04-12T20:56:21.002 INFO:tasks.workunit.client.0.smithi018.stdout:3/898: fdatasync d2/d6/d34/d71/d7b/f91 0 2024-04-12T20:56:21.004 INFO:tasks.workunit.client.0.smithi018.stdout:0/765: write da/d15/f41 [4578144,63019] 0 2024-04-12T20:56:21.007 INFO:tasks.workunit.client.0.smithi018.stdout:4/770: creat d3/d32/d33/db6/ffd x:0 0 0 2024-04-12T20:56:21.014 INFO:tasks.workunit.client.0.smithi018.stdout:4/771: rename d3/db/f37 to d3/d32/d33/db6/ffe 0 2024-04-12T20:56:21.019 INFO:tasks.workunit.client.0.smithi018.stdout:4/772: read d3/d5d/d60/dc5/fba [94531,43922] 0 2024-04-12T20:56:21.019 INFO:tasks.workunit.client.0.smithi018.stdout:3/899: truncate d2/f19 1441342 0 2024-04-12T20:56:21.021 INFO:tasks.workunit.client.0.smithi018.stdout:4/773: stat d3/f6b 0 2024-04-12T20:56:21.026 INFO:tasks.workunit.client.0.smithi018.stdout:4/774: creat d3/fff x:0 0 0 2024-04-12T20:56:21.027 INFO:tasks.workunit.client.0.smithi018.stdout:4/775: chown d3/c7 1 1 2024-04-12T20:56:21.029 INFO:tasks.workunit.client.0.smithi018.stdout:3/900: write d2/d4d/d88/fda [3481004,87888] 0 2024-04-12T20:56:21.029 INFO:tasks.workunit.client.0.smithi018.stdout:4/776: stat d3/d5d/d60/dc5/d43/d4b/c9f 0 2024-04-12T20:56:21.031 INFO:tasks.workunit.client.0.smithi018.stdout:3/901: chown d2/d43/d61/d64 882 1 2024-04-12T20:56:21.032 INFO:tasks.workunit.client.0.smithi018.stdout:6/959: dread d5/d6/fbd [0,4194304] 0 2024-04-12T20:56:21.034 INFO:tasks.workunit.client.0.smithi018.stdout:4/777: mknod d3/d32/d33/dbc/c100 0 2024-04-12T20:56:21.034 INFO:tasks.workunit.client.0.smithi018.stdout:3/902: write d2/d6/f4e [1620092,97407] 0 2024-04-12T20:56:21.036 INFO:tasks.workunit.client.0.smithi018.stdout:0/766: dread da/d15/d17/d21/d36/f63 [0,4194304] 0 2024-04-12T20:56:21.039 INFO:tasks.workunit.client.0.smithi018.stdout:4/778: read d3/d5d/d60/dc5/d3d/d80/fa2 [191464,69242] 0 2024-04-12T20:56:21.042 INFO:tasks.workunit.client.0.smithi018.stdout:0/767: mknod da/d15/d4d/d6b/cff 0 2024-04-12T20:56:21.044 INFO:tasks.workunit.client.0.smithi018.stdout:4/779: chown d3/d5d/d60/d64/l71 614543 1 2024-04-12T20:56:21.047 INFO:tasks.workunit.client.0.smithi018.stdout:4/780: chown d3/d5d/d60/d64/d72/l73 106507703 1 2024-04-12T20:56:21.048 INFO:tasks.workunit.client.0.smithi018.stdout:0/768: creat da/d15/d17/d71/dc1/dee/df7/dea/f100 x:0 0 0 2024-04-12T20:56:21.049 INFO:tasks.workunit.client.0.smithi018.stdout:4/781: fsync d3/db/fc4 0 2024-04-12T20:56:21.051 INFO:tasks.workunit.client.0.smithi018.stdout:0/769: write da/f38 [13727745,104179] 0 2024-04-12T20:56:21.054 INFO:tasks.workunit.client.0.smithi018.stdout:4/782: mknod d3/d5d/d60/c101 0 2024-04-12T20:56:21.056 INFO:tasks.workunit.client.0.smithi018.stdout:3/903: dwrite d2/d6/fa8 [0,4194304] 0 2024-04-12T20:56:21.057 INFO:tasks.workunit.client.0.smithi018.stdout:6/960: dwrite d5/df/d19/d2b/fa5 [0,4194304] 0 2024-04-12T20:56:21.057 INFO:tasks.workunit.client.0.smithi018.stdout:6/961: stat d5 0 2024-04-12T20:56:21.059 INFO:tasks.workunit.client.0.smithi018.stdout:6/962: dread - d5/d71/f114 zero size 2024-04-12T20:56:21.060 INFO:tasks.workunit.client.0.smithi018.stdout:4/783: creat d3/d5d/d60/dc5/d43/d4b/ddc/f102 x:0 0 0 2024-04-12T20:56:21.062 INFO:tasks.workunit.client.0.smithi018.stdout:3/904: mknod d2/d6/d34/da1/dac/c124 0 2024-04-12T20:56:21.068 INFO:tasks.workunit.client.0.smithi018.stdout:4/784: truncate d3/d5d/d60/dc5/fa4 287336 0 2024-04-12T20:56:21.069 INFO:tasks.workunit.client.0.smithi018.stdout:0/770: getdents da/d15/d17/d71/dc1/dee/df7/dea 0 2024-04-12T20:56:21.069 INFO:tasks.workunit.client.0.smithi018.stdout:6/963: unlink d5/d6/d1b/d3b/d22/d117/d13c/d154/f33 0 2024-04-12T20:56:21.070 INFO:tasks.workunit.client.0.smithi018.stdout:3/905: unlink d2/d83/c110 0 2024-04-12T20:56:21.072 INFO:tasks.workunit.client.0.smithi018.stdout:4/785: fdatasync d3/d5d/d60/dc5/fa4 0 2024-04-12T20:56:21.072 INFO:tasks.workunit.client.0.smithi018.stdout:6/964: stat d5/d6/d1b/d3b/d22/ffd 0 2024-04-12T20:56:21.073 INFO:tasks.workunit.client.0.smithi018.stdout:3/906: chown d2/d43/d61/d64/fd3 88072298 1 2024-04-12T20:56:21.075 INFO:tasks.workunit.client.0.smithi018.stdout:0/771: truncate da/d15/d17/d21/d36/d47/db5/ff3 327324 0 2024-04-12T20:56:21.075 INFO:tasks.workunit.client.0.smithi018.stdout:3/907: read - d2/d6/d30/d31/d38/d3a/dd0/d109/f5a zero size 2024-04-12T20:56:21.079 INFO:tasks.workunit.client.0.smithi018.stdout:6/965: dread d5/df/d19/d74/f75 [0,4194304] 0 2024-04-12T20:56:21.080 INFO:tasks.workunit.client.0.smithi018.stdout:3/908: read - d2/d6/d30/d31/d38/d3a/d48/fca zero size 2024-04-12T20:56:21.080 INFO:tasks.workunit.client.0.smithi018.stdout:0/772: chown da/d15/d17/d21/d36/d6e/lad 5439959 1 2024-04-12T20:56:21.083 INFO:tasks.workunit.client.0.smithi018.stdout:6/966: creat d5/df/d19/f164 x:0 0 0 2024-04-12T20:56:21.086 INFO:tasks.workunit.client.0.smithi018.stdout:0/773: fsync da/d15/d17/d21/f68 0 2024-04-12T20:56:21.090 INFO:tasks.workunit.client.0.smithi018.stdout:0/774: rmdir da/d15/d17/d21/d36 39 2024-04-12T20:56:21.090 INFO:tasks.workunit.client.0.smithi018.stdout:6/967: rename d5/df/d3f/f146 to d5/d15/f165 0 2024-04-12T20:56:21.092 INFO:tasks.workunit.client.0.smithi018.stdout:3/909: write d2/d4d/d88/fa0 [4886433,88312] 0 2024-04-12T20:56:21.094 INFO:tasks.workunit.client.0.smithi018.stdout:0/775: readlink da/d15/d17/d21/d36/d88/ld8 0 2024-04-12T20:56:21.094 INFO:tasks.workunit.client.0.smithi018.stdout:6/968: creat d5/d6/d1b/d8c/dc0/f166 x:0 0 0 2024-04-12T20:56:21.097 INFO:tasks.workunit.client.0.smithi018.stdout:6/969: symlink d5/df/l167 0 2024-04-12T20:56:21.101 INFO:tasks.workunit.client.0.smithi018.stdout:3/910: creat d2/d6/d30/d31/d38/d3a/dd0/d109/f125 x:0 0 0 2024-04-12T20:56:21.101 INFO:tasks.workunit.client.0.smithi018.stdout:6/970: mknod d5/d6/d1b/d3b/d22/d117/d13c/c168 0 2024-04-12T20:56:21.102 INFO:tasks.workunit.client.0.smithi018.stdout:0/776: write da/d15/d17/d71/dc1/dcc/f49 [908327,13411] 0 2024-04-12T20:56:21.106 INFO:tasks.workunit.client.0.smithi018.stdout:0/777: mknod da/d15/d17/d5c/d79/c101 0 2024-04-12T20:56:21.106 INFO:tasks.workunit.client.0.smithi018.stdout:6/971: rename d5/df/d19/d43/d14a to d5/d6/d90/d169 0 2024-04-12T20:56:21.111 INFO:tasks.workunit.client.0.smithi018.stdout:3/911: dread d2/d6/d30/d31/d38/f74 [0,4194304] 0 2024-04-12T20:56:21.112 INFO:tasks.workunit.client.0.smithi018.stdout:0/778: truncate da/d15/d17/d21/d36/da2/dc3/f73 1079295 0 2024-04-12T20:56:21.113 INFO:tasks.workunit.client.0.smithi018.stdout:6/972: rename d5/d6/d1b/d3b/d8e/c113 to d5/d6/d1b/d3b/c16a 0 2024-04-12T20:56:21.118 INFO:tasks.workunit.client.0.smithi018.stdout:6/973: creat d5/d6/d1b/d3b/d22/d117/d14b/d105/f16b x:0 0 0 2024-04-12T20:56:21.119 INFO:tasks.workunit.client.0.smithi018.stdout:3/912: rename d2/d6/d30/d31/d38/d3a/dd0/d109/d70/la3 to d2/d6/d30/d31/d38/d3a/d76/l126 0 2024-04-12T20:56:21.121 INFO:tasks.workunit.client.0.smithi018.stdout:6/974: read d5/d6/d1b/d3b/d22/fb3 [605144,82694] 0 2024-04-12T20:56:21.125 INFO:tasks.workunit.client.0.smithi018.stdout:6/975: symlink d5/d6/d1b/d3b/d22/d117/l16c 0 2024-04-12T20:56:21.130 INFO:tasks.workunit.client.0.smithi018.stdout:0/779: dwrite da/d15/d17/d71/dc1/dcc/f78 [0,4194304] 0 2024-04-12T20:56:21.132 INFO:tasks.workunit.client.0.smithi018.stdout:6/976: dread d5/df/d19/f13a [0,4194304] 0 2024-04-12T20:56:21.135 INFO:tasks.workunit.client.0.smithi018.stdout:0/780: symlink da/d15/d17/d21/d36/d88/ddf/l102 0 2024-04-12T20:56:21.138 INFO:tasks.workunit.client.0.smithi018.stdout:6/977: unlink d5/df/d19/f136 0 2024-04-12T20:56:21.138 INFO:tasks.workunit.client.0.smithi018.stdout:3/913: dwrite d2/d6/d30/d31/d38/d3a/d76/da2/de4/f111 [0,4194304] 0 2024-04-12T20:56:21.148 INFO:tasks.workunit.client.0.smithi018.stdout:6/978: dread - d5/d6/d1b/d3b/d22/d117/d13c/d154/f11e zero size 2024-04-12T20:56:21.151 INFO:tasks.workunit.client.0.smithi018.stdout:3/914: read d2/d6/d30/d31/d38/d3a/dd0/d109/f87 [178499,35614] 0 2024-04-12T20:56:21.156 INFO:tasks.workunit.client.0.smithi018.stdout:6/979: symlink d5/d6/d1b/d3b/d22/d117/d13c/d154/l16d 0 2024-04-12T20:56:21.157 INFO:tasks.workunit.client.0.smithi018.stdout:0/781: dwrite da/f52 [4194304,4194304] 0 2024-04-12T20:56:21.157 INFO:tasks.workunit.client.0.smithi018.stdout:3/915: symlink d2/d6/d30/d31/d38/d3a/d76/da2/l127 0 2024-04-12T20:56:21.161 INFO:tasks.workunit.client.0.smithi018.stdout:6/980: write d5/df/d19/d2b/f123 [1021736,7903] 0 2024-04-12T20:56:21.161 INFO:tasks.workunit.client.0.smithi018.stdout:3/916: stat d2/d6/d30/d31/d38/d3a/dd0/f96 0 2024-04-12T20:56:21.164 INFO:tasks.workunit.client.0.smithi018.stdout:0/782: dread da/d15/f90 [0,4194304] 0 2024-04-12T20:56:21.166 INFO:tasks.workunit.client.0.smithi018.stdout:6/981: truncate d5/df/d9b/fc4 985935 0 2024-04-12T20:56:21.169 INFO:tasks.workunit.client.0.smithi018.stdout:3/917: truncate d2/d6/d30/d31/faf 639082 0 2024-04-12T20:56:21.170 INFO:tasks.workunit.client.0.smithi018.stdout:0/783: mkdir da/d15/d17/d21/d36/d47/d81/d103 0 2024-04-12T20:56:21.170 INFO:tasks.workunit.client.0.smithi018.stdout:3/918: chown d2/d6/ld7 137 1 2024-04-12T20:56:21.171 INFO:tasks.workunit.client.0.smithi018.stdout:6/982: creat d5/d6/d1b/f16e x:0 0 0 2024-04-12T20:56:21.175 INFO:tasks.workunit.client.0.smithi018.stdout:3/919: write d2/d6/d30/d31/d38/d3a/d76/da2/d10a/f115 [1484528,123945] 0 2024-04-12T20:56:21.176 INFO:tasks.workunit.client.0.smithi018.stdout:6/983: mkdir d5/d6/d1b/d3b/d22/d117/d13c/d154/d16f 0 2024-04-12T20:56:21.178 INFO:tasks.workunit.client.0.smithi018.stdout:3/920: dread - d2/d6/d30/d31/d38/d3a/d76/dad/dc1/fde zero size 2024-04-12T20:56:21.181 INFO:tasks.workunit.client.0.smithi018.stdout:0/784: write da/d15/d17/d21/d36/d6e/f7f [1178509,119933] 0 2024-04-12T20:56:21.185 INFO:tasks.workunit.client.0.smithi018.stdout:0/785: read da/d15/d17/f43 [1917885,68703] 0 2024-04-12T20:56:21.186 INFO:tasks.workunit.client.0.smithi018.stdout:3/921: write d2/d43/d61/f63 [418055,59872] 0 2024-04-12T20:56:21.191 INFO:tasks.workunit.client.0.smithi018.stdout:0/786: rename da/f22 to da/d15/d17/d71/dc1/dee/f104 0 2024-04-12T20:56:21.192 INFO:tasks.workunit.client.0.smithi018.stdout:0/787: stat da/faa 0 2024-04-12T20:56:21.195 INFO:tasks.workunit.client.0.smithi018.stdout:6/984: getdents d5/d6/d1b/d3b/d22/d117/d13c 0 2024-04-12T20:56:21.196 INFO:tasks.workunit.client.0.smithi018.stdout:3/922: rmdir d2/d6/d34/d71/db2/dfc/d11a 0 2024-04-12T20:56:21.198 INFO:tasks.workunit.client.0.smithi018.stdout:0/788: rename da/d15/d4d/d6b/f96 to da/d15/d4d/d58/f105 0 2024-04-12T20:56:21.199 INFO:tasks.workunit.client.0.smithi018.stdout:6/985: rmdir d5/df/d19/d43/d5f 39 2024-04-12T20:56:21.199 INFO:tasks.workunit.client.0.smithi018.stdout:3/923: mkdir d2/d6/d30/d31/d38/d128 0 2024-04-12T20:56:21.201 INFO:tasks.workunit.client.0.smithi018.stdout:6/986: readlink d5/d6/l1d 0 2024-04-12T20:56:21.202 INFO:tasks.workunit.client.0.smithi018.stdout:0/789: unlink da/f1d 0 2024-04-12T20:56:21.202 INFO:tasks.workunit.client.0.smithi018.stdout:3/924: read d2/d4d/d88/f97 [680837,22043] 0 2024-04-12T20:56:21.204 INFO:tasks.workunit.client.0.smithi018.stdout:6/987: creat d5/d15/d60/f170 x:0 0 0 2024-04-12T20:56:21.208 INFO:tasks.workunit.client.0.smithi018.stdout:3/925: truncate d2/d43/db6/f80 1857317 0 2024-04-12T20:56:21.216 INFO:tasks.workunit.client.0.smithi018.stdout:0/790: write da/d15/d17/d21/d36/d47/faf [1095506,68264] 0 2024-04-12T20:56:21.217 INFO:tasks.workunit.client.0.smithi018.stdout:0/791: stat da 0 2024-04-12T20:56:21.219 INFO:tasks.workunit.client.0.smithi018.stdout:6/988: dread d5/d6/d1b/d3b/d22/d117/f15c [0,4194304] 0 2024-04-12T20:56:21.220 INFO:tasks.workunit.client.0.smithi018.stdout:0/792: truncate da/d15/f90 845784 0 2024-04-12T20:56:21.223 INFO:tasks.workunit.client.0.smithi018.stdout:6/989: chown d5/d6/d1b/d3b/d22/d117/d14b 12634877 1 2024-04-12T20:56:21.228 INFO:tasks.workunit.client.0.smithi018.stdout:6/990: mknod d5/df/d19/c171 0 2024-04-12T20:56:21.232 INFO:tasks.workunit.client.0.smithi018.stdout:6/991: creat d5/d71/f172 x:0 0 0 2024-04-12T20:56:21.235 INFO:tasks.workunit.client.0.smithi018.stdout:6/992: fdatasync d5/d15/f6f 0 2024-04-12T20:56:21.237 INFO:tasks.workunit.client.0.smithi018.stdout:3/926: dread d2/d6/d34/d71/d7b/f7d [0,4194304] 0 2024-04-12T20:56:21.240 INFO:tasks.workunit.client.0.smithi018.stdout:0/793: dwrite da/d15/d17/d21/d36/fb0 [0,4194304] 0 2024-04-12T20:56:21.243 INFO:tasks.workunit.client.0.smithi018.stdout:6/993: dread d5/d6/d1b/d3b/f69 [0,4194304] 0 2024-04-12T20:56:21.249 INFO:tasks.workunit.client.0.smithi018.stdout:6/994: read d5/d6/d1b/d3b/d6a/d116/ff4 [3873295,40726] 0 2024-04-12T20:56:21.249 INFO:tasks.workunit.client.0.smithi018.stdout:0/794: link da/d15/d17/d21/d36/da2/dc3/f5e da/d15/d17/d21/d36/dce/f106 0 2024-04-12T20:56:21.252 INFO:tasks.workunit.client.0.smithi018.stdout:0/795: mkdir da/d15/d17/d21/d36/da2/dc3/da9/d107 0 2024-04-12T20:56:21.261 INFO:tasks.workunit.client.0.smithi018.stdout:0/796: mkdir da/d15/d17/d108 0 2024-04-12T20:56:21.261 INFO:tasks.workunit.client.0.smithi018.stdout:3/927: dwrite d2/d4d/d88/fda [0,4194304] 0 2024-04-12T20:56:21.266 INFO:tasks.workunit.client.0.smithi018.stdout:3/928: creat d2/d6/d30/d31/d38/d3a/dd0/d109/d70/f129 x:0 0 0 2024-04-12T20:56:21.268 INFO:tasks.workunit.client.0.smithi018.stdout:0/797: getdents da/d15/d17/d71/dc1/dcc 0 2024-04-12T20:56:21.270 INFO:tasks.workunit.client.0.smithi018.stdout:6/995: dread d5/d6/d1b/d3b/d6a/f158 [0,4194304] 0 2024-04-12T20:56:21.273 INFO:tasks.workunit.client.0.smithi018.stdout:3/929: rename d2/d6/d30/d31/d38/d3a/d48/d82 to d2/d43/db6/db5/d12a 0 2024-04-12T20:56:21.274 INFO:tasks.workunit.client.0.smithi018.stdout:0/798: mknod da/d15/d17/d21/d36/d47/d81/d103/c109 0 2024-04-12T20:56:21.279 INFO:tasks.workunit.client.0.smithi018.stdout:0/799: symlink da/d15/d17/d71/dc1/dee/l10a 0 2024-04-12T20:56:21.280 INFO:tasks.workunit.client.0.smithi018.stdout:6/996: fsync d5/f37 0 2024-04-12T20:56:21.282 INFO:tasks.workunit.client.0.smithi018.stdout:0/800: dread - da/d15/d4d/fe9 zero size 2024-04-12T20:56:21.286 INFO:tasks.workunit.client.0.smithi018.stdout:3/930: write d2/d43/db6/f80 [1062311,95053] 0 2024-04-12T20:56:21.286 INFO:tasks.workunit.client.0.smithi018.stdout:0/801: mknod da/d15/d17/d21/c10b 0 2024-04-12T20:56:21.288 INFO:tasks.workunit.client.0.smithi018.stdout:3/931: chown d2/d6/d30/d31/c8c 228463 1 2024-04-12T20:56:21.290 INFO:tasks.workunit.client.0.smithi018.stdout:0/802: creat da/d15/d17/d21/d36/d91/f10c x:0 0 0 2024-04-12T20:56:21.293 INFO:tasks.workunit.client.0.smithi018.stdout:3/932: creat d2/d6/d30/d31/d38/d3a/dd0/de1/f12b x:0 0 0 2024-04-12T20:56:21.294 INFO:tasks.workunit.client.0.smithi018.stdout:0/803: write da/d15/d17/d21/d36/dce/ff6 [982615,74662] 0 2024-04-12T20:56:21.299 INFO:tasks.workunit.client.0.smithi018.stdout:6/997: dwrite d5/d6/d1b/d3b/d22/d117/d14b/d105/f16b [0,4194304] 0 2024-04-12T20:56:21.302 INFO:tasks.workunit.client.0.smithi018.stdout:6/998: read - d5/d6/d90/d125/d139/f132 zero size 2024-04-12T20:56:21.310 INFO:tasks.workunit.client.0.smithi018.stdout:3/933: dwrite d2/d4d/d88/f8a [0,4194304] 0 2024-04-12T20:56:21.310 INFO:tasks.workunit.client.0.smithi018.stdout:0/804: truncate da/d15/d17/d21/d36/fd4 586271 0 2024-04-12T20:56:21.313 INFO:tasks.workunit.client.0.smithi018.stdout:0/805: chown da/d15/d17/d21/d36/c37 14 1 2024-04-12T20:56:21.313 INFO:tasks.workunit.client.0.smithi018.stdout:3/934: readlink d2/d6/d30/d31/d38/d3a/dd0/d109/ld5 0 2024-04-12T20:56:21.316 INFO:tasks.workunit.client.0.smithi018.stdout:3/935: creat d2/d6/d34/da1/dac/f12c x:0 0 0 2024-04-12T20:56:21.320 INFO:tasks.workunit.client.0.smithi018.stdout:3/936: chown d2/d6/d30/d31/d38/d3a/d76/c8b 1 1 2024-04-12T20:56:21.322 INFO:tasks.workunit.client.0.smithi018.stdout:3/937: chown d2/d43/db6/db5/dd8/ca6 1676902 1 2024-04-12T20:56:21.324 INFO:tasks.workunit.client.0.smithi018.stdout:3/938: write d2/d6/d34/da1/ddb/fe5 [1014135,62412] 0 2024-04-12T20:56:21.326 INFO:tasks.workunit.client.0.smithi018.stdout:0/806: write da/d15/d17/d71/fbc [658156,79424] 0 2024-04-12T20:56:21.329 INFO:tasks.workunit.client.0.smithi018.stdout:0/807: write da/d15/d17/d21/d36/f7c [2555468,120921] 0 2024-04-12T20:56:21.332 INFO:tasks.workunit.client.0.smithi018.stdout:0/808: read da/f38 [12003731,111182] 0 2024-04-12T20:56:21.334 INFO:tasks.workunit.client.0.smithi018.stdout:0/809: chown da/d15/d17/d21/d36/l92 63106299 1 2024-04-12T20:56:21.347 INFO:tasks.workunit.client.0.smithi018.stdout:3/939: dwrite d2/d6/d30/d31/d38/d3a/dd0/de1/f12b [0,4194304] 0 2024-04-12T20:56:21.354 INFO:tasks.workunit.client.0.smithi018.stdout:0/810: dwrite da/d15/d17/d21/f46 [0,4194304] 0 2024-04-12T20:56:21.357 INFO:tasks.workunit.client.0.smithi018.stdout:0/811: write da/d15/d17/d21/f68 [4613227,81905] 0 2024-04-12T20:56:21.361 INFO:tasks.workunit.client.0.smithi018.stdout:3/940: dread d2/d6/fa8 [0,4194304] 0 2024-04-12T20:56:21.368 INFO:tasks.workunit.client.0.smithi018.stdout:3/941: unlink d2/d6/d30/d31/d38/d3a/d48/c6e 0 2024-04-12T20:56:21.371 INFO:tasks.workunit.client.0.smithi018.stdout:3/942: write d2/d4d/d88/f97 [3294130,88752] 0 2024-04-12T20:56:21.375 INFO:tasks.workunit.client.0.smithi018.stdout:0/812: dwrite da/d15/d17/d21/d94/fc7 [0,4194304] 0 2024-04-12T20:56:21.376 INFO:tasks.workunit.client.0.smithi018.stdout:3/943: mkdir d2/d6/d34/d12d 0 2024-04-12T20:56:21.380 INFO:tasks.workunit.client.0.smithi018.stdout:3/944: stat d2/d6/d30/d31/d38/d3a/d76/da2/de4/f112 0 2024-04-12T20:56:21.384 INFO:tasks.workunit.client.0.smithi018.stdout:3/945: dread - d2/d6/d30/d31/d38/d3a/d48/f10d zero size 2024-04-12T20:56:21.386 INFO:tasks.workunit.client.0.smithi018.stdout:3/946: creat d2/d43/f12e x:0 0 0 2024-04-12T20:56:21.390 INFO:tasks.workunit.client.0.smithi018.stdout:3/947: dread d2/d6/d30/d31/d38/d3a/dd0/d109/fdd [0,4194304] 0 2024-04-12T20:56:21.394 INFO:tasks.workunit.client.0.smithi018.stdout:3/948: creat d2/d6/d34/da1/ddb/f12f x:0 0 0 2024-04-12T20:56:21.395 INFO:tasks.workunit.client.0.smithi018.stdout:0/813: dwrite da/d15/d17/d21/d36/f63 [0,4194304] 0 2024-04-12T20:56:21.398 INFO:tasks.workunit.client.0.smithi018.stdout:3/949: mkdir d2/d6/d30/d31/d38/d3a/dd0/d130 0 2024-04-12T20:56:21.399 INFO:tasks.workunit.client.0.smithi018.stdout:3/950: fsync d2/d6/d34/fd6 0 2024-04-12T20:56:21.407 INFO:tasks.workunit.client.0.smithi018.stdout:3/951: creat d2/d6/d30/d31/d38/d3a/d76/da2/d10a/f131 x:0 0 0 2024-04-12T20:56:21.407 INFO:tasks.workunit.client.0.smithi018.stdout:0/814: write da/d15/d17/d71/fc2 [2079726,107867] 0 2024-04-12T20:56:21.409 INFO:tasks.workunit.client.0.smithi018.stdout:0/815: chown da/d15/f33 169755128 1 2024-04-12T20:56:21.411 INFO:tasks.workunit.client.0.smithi018.stdout:3/952: unlink d2/d43/db6/db5/dd8/ca6 0 2024-04-12T20:56:21.413 INFO:tasks.workunit.client.0.smithi018.stdout:0/816: mkdir da/d15/d17/d10d 0 2024-04-12T20:56:21.414 INFO:tasks.workunit.client.0.smithi018.stdout:3/953: chown d2/d6/d30/d31/d38/f66 330444 1 2024-04-12T20:56:21.417 INFO:tasks.workunit.client.0.smithi018.stdout:3/954: dread d2/d6/d30/d31/d38/f66 [0,4194304] 0 2024-04-12T20:56:21.418 INFO:tasks.workunit.client.0.smithi018.stdout:0/817: truncate da/d15/d17/d21/d36/f80 484579 0 2024-04-12T20:56:21.420 INFO:tasks.workunit.client.0.smithi018.stdout:0/818: mknod da/d15/d17/d71/dc1/dcc/c10e 0 2024-04-12T20:56:21.420 INFO:tasks.workunit.client.0.smithi018.stdout:3/955: write d2/d6/d30/d31/d38/d3a/dd0/d109/d70/f129 [361693,113880] 0 2024-04-12T20:56:21.423 INFO:tasks.workunit.client.0.smithi018.stdout:0/819: stat da/d15/d17/d21/d36/da2/dc3/d83/le7 0 2024-04-12T20:56:21.424 INFO:tasks.workunit.client.0.smithi018.stdout:3/956: mknod d2/d6/d34/d71/dbc/c132 0 2024-04-12T20:56:21.426 INFO:tasks.workunit.client.0.smithi018.stdout:3/957: write d2/d6/d34/f46 [2410488,69729] 0 2024-04-12T20:56:21.428 INFO:tasks.workunit.client.0.smithi018.stdout:3/958: chown d2/l3f 302 1 2024-04-12T20:56:21.429 INFO:tasks.workunit.client.0.smithi018.stdout:0/820: truncate da/d15/d17/d21/d36/da2/dc3/d3f/f8f 1222939 0 2024-04-12T20:56:21.432 INFO:tasks.workunit.client.0.smithi018.stdout:3/959: creat d2/d6/d30/d31/d38/d3a/d76/da2/d10a/f133 x:0 0 0 2024-04-12T20:56:21.433 INFO:tasks.workunit.client.0.smithi018.stdout:0/821: chown da/d15/d17/d21/d36/da2/dc3/l6a 261 1 2024-04-12T20:56:21.435 INFO:tasks.workunit.client.0.smithi018.stdout:3/960: truncate d2/d83/fce 1362422 0 2024-04-12T20:56:21.438 INFO:tasks.workunit.client.0.smithi018.stdout:3/961: unlink d2/d43/cc3 0 2024-04-12T20:56:21.438 INFO:tasks.workunit.client.0.smithi018.stdout:0/822: getdents da/d15/d17/d21/d36/d6e 0 2024-04-12T20:56:21.440 INFO:tasks.workunit.client.0.smithi018.stdout:0/823: write da/d15/d4d/d6b/f6d [3810735,8326] 0 2024-04-12T20:56:21.440 INFO:tasks.workunit.client.0.smithi018.stdout:3/962: truncate d2/d4d/d8d/fab 1609325 0 2024-04-12T20:56:21.443 INFO:tasks.workunit.client.0.smithi018.stdout:0/824: write da/d15/d17/d21/d36/d47/d81/fec [1081414,3330] 0 2024-04-12T20:56:21.448 INFO:tasks.workunit.client.0.smithi018.stdout:0/825: creat da/d15/d17/d21/d36/da2/dc3/d3f/f10f x:0 0 0 2024-04-12T20:56:21.452 INFO:tasks.workunit.client.0.smithi018.stdout:3/963: dread d2/d6/fa8 [0,4194304] 0 2024-04-12T20:56:21.452 INFO:tasks.workunit.client.0.smithi018.stdout:0/826: mknod da/d15/d17/d108/c110 0 2024-04-12T20:56:21.454 INFO:tasks.workunit.client.0.smithi018.stdout:3/964: readlink d2/l27 0 2024-04-12T20:56:21.457 INFO:tasks.workunit.client.0.smithi018.stdout:3/965: write d2/d6/d34/f46 [1487830,96545] 0 2024-04-12T20:56:21.458 INFO:tasks.workunit.client.0.smithi018.stdout:0/827: read da/d15/d17/d21/d36/da2/dc3/f73 [1078207,35513] 0 2024-04-12T20:56:21.458 INFO:tasks.workunit.client.0.smithi018.stdout:3/966: readlink d2/l26 0 2024-04-12T20:56:21.460 INFO:tasks.workunit.client.0.smithi018.stdout:3/967: fdatasync d2/d6/d30/f58 0 2024-04-12T20:56:21.464 INFO:tasks.workunit.client.0.smithi018.stdout:0/828: creat da/d15/d17/d21/d36/d88/ddf/f111 x:0 0 0 2024-04-12T20:56:21.464 INFO:tasks.workunit.client.0.smithi018.stdout:3/968: creat d2/d43/db6/f134 x:0 0 0 2024-04-12T20:56:21.466 INFO:tasks.workunit.client.0.smithi018.stdout:3/969: chown d2/d6/d30/d31/d38/d3a/l9a 0 1 2024-04-12T20:56:21.468 INFO:tasks.workunit.client.0.smithi018.stdout:0/829: rename da/d15/d4d/dd0/ffc to da/d15/d17/d5c/d79/f112 0 2024-04-12T20:56:21.475 INFO:tasks.workunit.client.0.smithi018.stdout:3/970: truncate d2/d6/d30/d31/d38/d3a/dd0/d109/d54/fcb 7088740 0 2024-04-12T20:56:21.479 INFO:tasks.workunit.client.0.smithi018.stdout:3/971: dread d2/d6/d30/d31/d38/d3a/dd0/d109/fdd [0,4194304] 0 2024-04-12T20:56:21.485 INFO:tasks.workunit.client.0.smithi018.stdout:0/830: dwrite da/fc [0,4194304] 0 2024-04-12T20:56:21.485 INFO:tasks.workunit.client.0.smithi018.stdout:3/972: dread d2/d4d/d88/fa0 [4194304,4194304] 0 2024-04-12T20:56:21.488 INFO:tasks.workunit.client.0.smithi018.stdout:3/973: symlink d2/d43/db6/db5/l135 0 2024-04-12T20:56:21.498 INFO:tasks.workunit.client.0.smithi018.stdout:3/974: rename d2/d6/d30/d31/d38/d3a/dd0/d109/d54/cbe to d2/d6/d30/d31/d38/d3a/d76/da2/d10a/c136 0 2024-04-12T20:56:21.501 INFO:tasks.workunit.client.0.smithi018.stdout:3/975: dread - d2/d6/d30/d31/d38/d3a/d48/fca zero size 2024-04-12T20:56:21.504 INFO:tasks.workunit.client.0.smithi018.stdout:0/831: write da/d15/d17/d21/d36/dce/f106 [2264752,111573] 0 2024-04-12T20:56:21.506 INFO:tasks.workunit.client.0.smithi018.stdout:3/976: symlink d2/d6/d34/d71/db2/dfc/l137 0 2024-04-12T20:56:21.509 INFO:tasks.workunit.client.0.smithi018.stdout:7/811: sync 2024-04-12T20:56:21.509 INFO:tasks.workunit.client.0.smithi018.stdout:3/977: read d2/f28 [751660,7693] 0 2024-04-12T20:56:21.512 INFO:tasks.workunit.client.0.smithi018.stdout:0/832: rename da/d15/f8e to da/d15/d17/d21/d36/d47/db5/f113 0 2024-04-12T20:56:21.513 INFO:tasks.workunit.client.0.smithi018.stdout:7/812: mknod d1/db/d6f/c107 0 2024-04-12T20:56:21.515 INFO:tasks.workunit.client.0.smithi018.stdout:3/978: mknod d2/d6/d30/d31/d38/d3a/dd0/d109/d103/c138 0 2024-04-12T20:56:21.516 INFO:tasks.workunit.client.0.smithi018.stdout:0/833: mknod da/d15/d17/d5c/dae/c114 0 2024-04-12T20:56:21.517 INFO:tasks.workunit.client.0.smithi018.stdout:0/834: chown da 1 1 2024-04-12T20:56:21.517 INFO:tasks.workunit.client.0.smithi018.stdout:3/979: write d2/d6/d34/f59 [559928,105163] 0 2024-04-12T20:56:21.526 INFO:tasks.workunit.client.0.smithi018.stdout:0/835: creat da/f115 x:0 0 0 2024-04-12T20:56:21.529 INFO:tasks.workunit.client.0.smithi018.stdout:3/980: getdents d2/d6 0 2024-04-12T20:56:21.530 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:56:21 smithi186 ceph-mon[36597]: pgmap v43: 65 pgs: 65 active+clean; 1.6 GiB data, 6.3 GiB used, 530 GiB / 536 GiB avail; 45 MiB/s rd, 111 MiB/s wr, 315 op/s 2024-04-12T20:56:21.531 INFO:tasks.workunit.client.0.smithi018.stdout:0/836: creat da/d15/d17/d5c/f116 x:0 0 0 2024-04-12T20:56:21.536 INFO:tasks.workunit.client.1.smithi186.stderr:++ readlink -f fsstress 2024-04-12T20:56:21.536 INFO:tasks.workunit.client.0.smithi018.stdout:3/981: symlink d2/d6/d30/d31/d38/d3a/d76/da2/de4/l139 0 2024-04-12T20:56:21.537 INFO:tasks.workunit.client.0.smithi018.stdout:0/837: mkdir da/d15/d17/d21/d36/d88/d117 0 2024-04-12T20:56:21.538 INFO:tasks.workunit.client.0.smithi018.stdout:7/813: dwrite d1/db/d31/d3a/d42/d4c/f51 [0,4194304] 0 2024-04-12T20:56:21.544 INFO:tasks.workunit.client.0.smithi018.stdout:3/982: write d2/d6/d30/d31/d38/d3a/d76/da2/d10a/f133 [883151,35389] 0 2024-04-12T20:56:21.544 INFO:tasks.workunit.client.1.smithi186.stderr:+ BIN=/home/ubuntu/cephtest/mnt.1/client.1/tmp/fsstress/ltp-full-20091231/testcases/kernel/fs/fsstress/fsstress 2024-04-12T20:56:21.544 INFO:tasks.workunit.client.1.smithi186.stderr:+ popd 2024-04-12T20:56:21.548 INFO:tasks.workunit.client.0.smithi018.stdout:3/983: creat d2/d4d/f13a x:0 0 0 2024-04-12T20:56:21.550 INFO:tasks.workunit.client.0.smithi018.stdout:3/984: dread - d2/d6/ff9 zero size 2024-04-12T20:56:21.551 INFO:tasks.workunit.client.1.smithi186.stdout:~/cephtest/mnt.1/client.1/tmp/fsstress ~/cephtest/mnt.1/client.1/tmp 2024-04-12T20:56:21.552 INFO:tasks.workunit.client.1.smithi186.stderr:+ popd 2024-04-12T20:56:21.552 INFO:tasks.workunit.client.0.smithi018.stdout:3/985: write d2/d6/d34/f46 [1238684,4046] 0 2024-04-12T20:56:21.555 INFO:tasks.workunit.client.1.smithi186.stdout:~/cephtest/mnt.1/client.1/tmp 2024-04-12T20:56:21.557 INFO:tasks.workunit.client.1.smithi186.stderr:++ mktemp -d -p . 2024-04-12T20:56:21.558 INFO:tasks.workunit.client.0.smithi018.stdout:3/986: mkdir d2/d6/d34/d71/db2/d13b 0 2024-04-12T20:56:21.561 INFO:tasks.workunit.client.0.smithi018.stdout:3/987: write d2/d4d/d88/f8a [4922429,55589] 0 2024-04-12T20:56:21.562 INFO:tasks.workunit.client.0.smithi018.stdout:7/814: dread d1/db/d31/d3a/d42/f75 [0,4194304] 0 2024-04-12T20:56:21.563 INFO:tasks.workunit.client.0.smithi018.stdout:7/815: chown d1/d6/f98 128776 1 2024-04-12T20:56:21.564 INFO:tasks.workunit.client.0.smithi018.stdout:3/988: write d2/d4d/f57 [876858,46809] 0 2024-04-12T20:56:21.572 INFO:tasks.workunit.client.0.smithi018.stdout:7/816: mkdir d1/db/d31/d45/d8c/d9d/d70/d96/d5f/da9/ddf/d108 0 2024-04-12T20:56:21.577 INFO:tasks.workunit.client.1.smithi186.stderr:+ T=./tmp.gUD0e7KDu0 2024-04-12T20:56:21.577 INFO:tasks.workunit.client.1.smithi186.stderr:+ /home/ubuntu/cephtest/mnt.1/client.1/tmp/fsstress/ltp-full-20091231/testcases/kernel/fs/fsstress/fsstress -d ./tmp.gUD0e7KDu0 -l 1 -n 1000 -p 10 -v 2024-04-12T20:56:21.581 INFO:tasks.workunit.client.0.smithi018.stdout:7/817: truncate d1/d6/f24 682432 0 2024-04-12T20:56:21.584 INFO:tasks.workunit.client.0.smithi018.stdout:7/818: mknod d1/db/d6f/dad/c109 0 2024-04-12T20:56:21.588 INFO:tasks.workunit.client.0.smithi018.stdout:3/989: dwrite d2/d6/d30/d31/d38/d3a/d76/da2/d10a/f131 [0,4194304] 0 2024-04-12T20:56:21.588 INFO:tasks.workunit.client.0.smithi018.stdout:7/819: mkdir d1/d6/d10a 0 2024-04-12T20:56:21.589 INFO:tasks.workunit.client.1.smithi186.stdout:seed = 1712361678 2024-04-12T20:56:21.597 INFO:tasks.workunit.client.0.smithi018.stdout:7/820: dread d1/db/d31/d3a/d42/f8f [0,4194304] 0 2024-04-12T20:56:21.601 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:56:21 smithi018 ceph-mon[28799]: pgmap v43: 65 pgs: 65 active+clean; 1.6 GiB data, 6.3 GiB used, 530 GiB / 536 GiB avail; 45 MiB/s rd, 111 MiB/s wr, 315 op/s 2024-04-12T20:56:21.601 INFO:tasks.workunit.client.0.smithi018.stdout:3/990: write d2/d6/d30/d31/d38/d3a/dd0/d109/fc6 [261811,35427] 0 2024-04-12T20:56:21.602 INFO:tasks.workunit.client.1.smithi186.stdout:0/0: fdatasync - no filename 2024-04-12T20:56:21.602 INFO:tasks.workunit.client.1.smithi186.stdout:0/1: write - no filename 2024-04-12T20:56:21.603 INFO:tasks.workunit.client.0.smithi018.stdout:3/991: chown d2/d4d/d88/f8a 58439076 1 2024-04-12T20:56:21.604 INFO:tasks.workunit.client.1.smithi186.stdout:0/2: chown . 39398014 1 2024-04-12T20:56:21.605 INFO:tasks.workunit.client.0.smithi018.stdout:3/992: chown d2/d6/d34/d71/db2/dfc/l137 3282324 1 2024-04-12T20:56:21.605 INFO:tasks.workunit.client.1.smithi186.stdout:2/0: dread - no filename 2024-04-12T20:56:21.605 INFO:tasks.workunit.client.1.smithi186.stdout:2/1: dwrite - no filename 2024-04-12T20:56:21.608 INFO:tasks.workunit.client.1.smithi186.stdout:0/3: mkdir d0 0 2024-04-12T20:56:21.609 INFO:tasks.workunit.client.1.smithi186.stdout:2/2: getdents . 0 2024-04-12T20:56:21.610 INFO:tasks.workunit.client.1.smithi186.stdout:3/0: dwrite - no filename 2024-04-12T20:56:21.610 INFO:tasks.workunit.client.1.smithi186.stdout:3/1: dread - no filename 2024-04-12T20:56:21.610 INFO:tasks.workunit.client.1.smithi186.stdout:3/2: dread - no filename 2024-04-12T20:56:21.610 INFO:tasks.workunit.client.1.smithi186.stdout:3/3: fsync - no filename 2024-04-12T20:56:21.610 INFO:tasks.workunit.client.1.smithi186.stdout:3/4: write - no filename 2024-04-12T20:56:21.611 INFO:tasks.workunit.client.1.smithi186.stdout:0/4: mknod d0/c1 0 2024-04-12T20:56:21.611 INFO:tasks.workunit.client.1.smithi186.stdout:1/0: mknod c0 0 2024-04-12T20:56:21.613 INFO:tasks.workunit.client.1.smithi186.stdout:2/3: mkdir d0 0 2024-04-12T20:56:21.614 INFO:tasks.workunit.client.1.smithi186.stdout:5/0: dread - no filename 2024-04-12T20:56:21.614 INFO:tasks.workunit.client.1.smithi186.stdout:1/1: mknod c1 0 2024-04-12T20:56:21.615 INFO:tasks.workunit.client.1.smithi186.stdout:1/2: write - no filename 2024-04-12T20:56:21.615 INFO:tasks.workunit.client.1.smithi186.stdout:5/1: chown . 14075 1 2024-04-12T20:56:21.615 INFO:tasks.workunit.client.1.smithi186.stdout:3/5: getdents . 0 2024-04-12T20:56:21.615 INFO:tasks.workunit.client.1.smithi186.stdout:5/2: chown . 201888 1 2024-04-12T20:56:21.616 INFO:tasks.workunit.client.1.smithi186.stdout:6/0: write - no filename 2024-04-12T20:56:21.616 INFO:tasks.workunit.client.1.smithi186.stdout:6/1: dwrite - no filename 2024-04-12T20:56:21.616 INFO:tasks.workunit.client.1.smithi186.stdout:6/2: rename - no filename 2024-04-12T20:56:21.616 INFO:tasks.workunit.client.1.smithi186.stdout:6/3: stat - no entries 2024-04-12T20:56:21.617 INFO:tasks.workunit.client.1.smithi186.stdout:4/0: mkdir d0 0 2024-04-12T20:56:21.619 INFO:tasks.workunit.client.1.smithi186.stdout:2/4: creat d0/f1 x:0 0 0 2024-04-12T20:56:21.620 INFO:tasks.workunit.client.1.smithi186.stdout:2/5: chown d0 11304631 1 2024-04-12T20:56:21.621 INFO:tasks.workunit.client.1.smithi186.stdout:1/3: symlink l2 0 2024-04-12T20:56:21.621 INFO:tasks.workunit.client.1.smithi186.stdout:3/6: getdents . 0 2024-04-12T20:56:21.621 INFO:tasks.workunit.client.1.smithi186.stdout:3/7: dwrite - no filename 2024-04-12T20:56:21.621 INFO:tasks.workunit.client.1.smithi186.stdout:3/8: fdatasync - no filename 2024-04-12T20:56:21.621 INFO:tasks.workunit.client.1.smithi186.stdout:3/9: truncate - no filename 2024-04-12T20:56:21.622 INFO:tasks.workunit.client.1.smithi186.stdout:0/5: link d0/c1 d0/c2 0 2024-04-12T20:56:21.622 INFO:tasks.workunit.client.1.smithi186.stdout:0/6: dwrite - no filename 2024-04-12T20:56:21.622 INFO:tasks.workunit.client.1.smithi186.stdout:0/7: read - no filename 2024-04-12T20:56:21.622 INFO:tasks.workunit.client.1.smithi186.stdout:0/8: fsync - no filename 2024-04-12T20:56:21.622 INFO:tasks.workunit.client.1.smithi186.stdout:5/3: mkdir d0 0 2024-04-12T20:56:21.623 INFO:tasks.workunit.client.1.smithi186.stdout:4/1: symlink d0/l1 0 2024-04-12T20:56:21.624 INFO:tasks.workunit.client.1.smithi186.stdout:4/2: dread - no filename 2024-04-12T20:56:21.625 INFO:tasks.workunit.client.1.smithi186.stdout:1/4: symlink l3 0 2024-04-12T20:56:21.625 INFO:tasks.workunit.client.1.smithi186.stdout:1/5: fsync - no filename 2024-04-12T20:56:21.625 INFO:tasks.workunit.client.1.smithi186.stdout:1/6: dread - no filename 2024-04-12T20:56:21.625 INFO:tasks.workunit.client.1.smithi186.stdout:1/7: read - no filename 2024-04-12T20:56:21.625 INFO:tasks.workunit.client.1.smithi186.stdout:1/8: write - no filename 2024-04-12T20:56:21.626 INFO:tasks.workunit.client.1.smithi186.stdout:8/0: write - no filename 2024-04-12T20:56:21.626 INFO:tasks.workunit.client.1.smithi186.stdout:2/6: creat d0/f2 x:0 0 0 2024-04-12T20:56:21.627 INFO:tasks.workunit.client.1.smithi186.stdout:4/3: creat d0/f2 x:0 0 0 2024-04-12T20:56:21.628 INFO:tasks.workunit.client.1.smithi186.stdout:0/9: stat d0/c2 0 2024-04-12T20:56:21.628 INFO:tasks.workunit.client.1.smithi186.stdout:0/10: fdatasync - no filename 2024-04-12T20:56:21.628 INFO:tasks.workunit.client.1.smithi186.stdout:5/4: mkdir d0/d1 0 2024-04-12T20:56:21.628 INFO:tasks.workunit.client.1.smithi186.stdout:5/5: dwrite - no filename 2024-04-12T20:56:21.629 INFO:tasks.workunit.client.1.smithi186.stdout:7/0: mknod c0 0 2024-04-12T20:56:21.629 INFO:tasks.workunit.client.1.smithi186.stdout:7/1: dwrite - no filename 2024-04-12T20:56:21.629 INFO:tasks.workunit.client.1.smithi186.stdout:9/0: dread - no filename 2024-04-12T20:56:21.630 INFO:tasks.workunit.client.1.smithi186.stdout:4/4: chown d0/l1 790518017 1 2024-04-12T20:56:21.630 INFO:tasks.workunit.client.1.smithi186.stdout:3/10: mknod c0 0 2024-04-12T20:56:21.631 INFO:tasks.workunit.client.1.smithi186.stdout:7/2: stat c0 0 2024-04-12T20:56:21.631 INFO:tasks.workunit.client.1.smithi186.stdout:1/9: creat f4 x:0 0 0 2024-04-12T20:56:21.633 INFO:tasks.workunit.client.0.smithi018.stdout:7/821: dread d1/db/d31/d30/f57 [0,4194304] 0 2024-04-12T20:56:21.633 INFO:tasks.workunit.client.1.smithi186.stdout:7/3: chown c0 22800889 1 2024-04-12T20:56:21.633 INFO:tasks.workunit.client.1.smithi186.stdout:4/5: dread - d0/f2 zero size 2024-04-12T20:56:21.634 INFO:tasks.workunit.client.1.smithi186.stdout:8/1: getdents . 0 2024-04-12T20:56:21.634 INFO:tasks.workunit.client.1.smithi186.stdout:8/2: dread - no filename 2024-04-12T20:56:21.634 INFO:tasks.workunit.client.1.smithi186.stdout:8/3: rmdir - no directory 2024-04-12T20:56:21.635 INFO:tasks.workunit.client.1.smithi186.stdout:9/1: symlink l0 0 2024-04-12T20:56:21.635 INFO:tasks.workunit.client.1.smithi186.stdout:9/2: truncate - no filename 2024-04-12T20:56:21.635 INFO:tasks.workunit.client.1.smithi186.stdout:9/3: truncate - no filename 2024-04-12T20:56:21.635 INFO:tasks.workunit.client.1.smithi186.stdout:9/4: dread - no filename 2024-04-12T20:56:21.637 INFO:tasks.workunit.client.1.smithi186.stdout:5/6: rmdir d0/d1 0 2024-04-12T20:56:21.638 INFO:tasks.workunit.client.1.smithi186.stdout:7/4: mknod c1 0 2024-04-12T20:56:21.638 INFO:tasks.workunit.client.1.smithi186.stdout:7/5: truncate - no filename 2024-04-12T20:56:21.639 INFO:tasks.workunit.client.1.smithi186.stdout:3/11: chown c0 2 1 2024-04-12T20:56:21.640 INFO:tasks.workunit.client.1.smithi186.stdout:1/10: link c1 c5 0 2024-04-12T20:56:21.640 INFO:tasks.workunit.client.1.smithi186.stdout:0/11: rename d0/c1 to d0/c3 0 2024-04-12T20:56:21.641 INFO:tasks.workunit.client.1.smithi186.stdout:9/5: creat f1 x:0 0 0 2024-04-12T20:56:21.641 INFO:tasks.workunit.client.1.smithi186.stdout:8/4: symlink l0 0 2024-04-12T20:56:21.642 INFO:tasks.workunit.client.1.smithi186.stdout:8/5: write - no filename 2024-04-12T20:56:21.642 INFO:tasks.workunit.client.1.smithi186.stdout:7/6: rename c1 to c2 0 2024-04-12T20:56:21.645 INFO:tasks.workunit.client.1.smithi186.stdout:3/12: unlink c0 0 2024-04-12T20:56:21.645 INFO:tasks.workunit.client.1.smithi186.stdout:3/13: link - no file 2024-04-12T20:56:21.645 INFO:tasks.workunit.client.1.smithi186.stdout:3/14: dread - no filename 2024-04-12T20:56:21.645 INFO:tasks.workunit.client.1.smithi186.stdout:3/15: dread - no filename 2024-04-12T20:56:21.645 INFO:tasks.workunit.client.1.smithi186.stdout:3/16: write - no filename 2024-04-12T20:56:21.646 INFO:tasks.workunit.client.1.smithi186.stdout:1/11: read - f4 zero size 2024-04-12T20:56:21.646 INFO:tasks.workunit.client.1.smithi186.stdout:0/12: rmdir d0 39 2024-04-12T20:56:21.646 INFO:tasks.workunit.client.1.smithi186.stdout:0/13: dwrite - no filename 2024-04-12T20:56:21.646 INFO:tasks.workunit.client.1.smithi186.stdout:0/14: read - no filename 2024-04-12T20:56:21.646 INFO:tasks.workunit.client.1.smithi186.stdout:0/15: dread - no filename 2024-04-12T20:56:21.646 INFO:tasks.workunit.client.1.smithi186.stdout:0/16: dwrite - no filename 2024-04-12T20:56:21.647 INFO:tasks.workunit.client.1.smithi186.stdout:8/6: creat f1 x:0 0 0 2024-04-12T20:56:21.649 INFO:tasks.workunit.client.1.smithi186.stdout:7/7: unlink c0 0 2024-04-12T20:56:21.649 INFO:tasks.workunit.client.1.smithi186.stdout:7/8: write - no filename 2024-04-12T20:56:21.651 INFO:tasks.workunit.client.1.smithi186.stdout:9/6: link f1 f2 0 2024-04-12T20:56:21.651 INFO:tasks.workunit.client.1.smithi186.stdout:3/17: mkdir d1 0 2024-04-12T20:56:21.651 INFO:tasks.workunit.client.1.smithi186.stdout:3/18: truncate - no filename 2024-04-12T20:56:21.652 INFO:tasks.workunit.client.1.smithi186.stdout:3/19: dwrite - no filename 2024-04-12T20:56:21.652 INFO:tasks.workunit.client.1.smithi186.stdout:3/20: write - no filename 2024-04-12T20:56:21.653 INFO:tasks.workunit.client.0.smithi018.stdout:7/822: dread d1/db/d31/d3a/d42/d4c/f5b [0,4194304] 0 2024-04-12T20:56:21.653 INFO:tasks.workunit.client.1.smithi186.stdout:7/9: symlink l3 0 2024-04-12T20:56:21.653 INFO:tasks.workunit.client.1.smithi186.stdout:7/10: fdatasync - no filename 2024-04-12T20:56:21.653 INFO:tasks.workunit.client.1.smithi186.stdout:7/11: write - no filename 2024-04-12T20:56:21.654 INFO:tasks.workunit.client.1.smithi186.stdout:7/12: dread - no filename 2024-04-12T20:56:21.654 INFO:tasks.workunit.client.1.smithi186.stdout:7/13: truncate - no filename 2024-04-12T20:56:21.654 INFO:tasks.workunit.client.1.smithi186.stdout:3/21: chown d1 29 1 2024-04-12T20:56:21.654 INFO:tasks.workunit.client.1.smithi186.stdout:9/7: mkdir d3 0 2024-04-12T20:56:21.655 INFO:tasks.workunit.client.0.smithi018.stdout:7/823: symlink d1/db/l10b 0 2024-04-12T20:56:21.659 INFO:tasks.workunit.client.1.smithi186.stdout:7/14: creat f4 x:0 0 0 2024-04-12T20:56:21.659 INFO:tasks.workunit.client.0.smithi018.stdout:7/824: write d1/db/d31/d30/dc6/fc8 [1011173,101727] 0 2024-04-12T20:56:21.670 INFO:tasks.workunit.client.1.smithi186.stdout:2/7: dwrite d0/f2 [0,4194304] 0 2024-04-12T20:56:21.671 INFO:tasks.workunit.client.1.smithi186.stdout:3/22: mknod d1/c2 0 2024-04-12T20:56:21.677 INFO:tasks.workunit.client.1.smithi186.stdout:2/8: stat d0/f1 0 2024-04-12T20:56:21.682 INFO:tasks.workunit.client.0.smithi018.stdout:7/825: dread d1/db/d31/d3a/fa5 [0,4194304] 0 2024-04-12T20:56:21.683 INFO:tasks.workunit.client.1.smithi186.stdout:2/9: dread - d0/f1 zero size 2024-04-12T20:56:21.686 INFO:tasks.workunit.client.0.smithi018.stdout:7/826: fsync d1/db/d31/d3a/d42/f75 0 2024-04-12T20:56:21.688 INFO:tasks.workunit.client.0.smithi018.stdout:1/683: sync 2024-04-12T20:56:21.689 INFO:tasks.workunit.client.1.smithi186.stdout:2/10: chown d0 2283942 1 2024-04-12T20:56:21.689 INFO:tasks.workunit.client.1.smithi186.stdout:8/7: dwrite f1 [0,4194304] 0 2024-04-12T20:56:21.689 INFO:tasks.workunit.client.1.smithi186.stdout:1/12: dwrite f4 [0,4194304] 0 2024-04-12T20:56:21.691 INFO:tasks.workunit.client.0.smithi018.stdout:7/827: truncate d1/db/d31/d45/d53/fed 521707 0 2024-04-12T20:56:21.692 INFO:tasks.workunit.client.1.smithi186.stdout:8/8: stat f1 0 2024-04-12T20:56:21.693 INFO:tasks.workunit.client.1.smithi186.stdout:3/23: mkdir d1/d3 0 2024-04-12T20:56:21.693 INFO:tasks.workunit.client.1.smithi186.stdout:3/24: fsync - no filename 2024-04-12T20:56:21.696 INFO:tasks.workunit.client.1.smithi186.stdout:2/11: mkdir d0/d3 0 2024-04-12T20:56:21.697 INFO:tasks.workunit.client.1.smithi186.stdout:8/9: unlink f1 0 2024-04-12T20:56:21.697 INFO:tasks.workunit.client.1.smithi186.stdout:8/10: truncate - no filename 2024-04-12T20:56:21.698 INFO:tasks.workunit.client.1.smithi186.stdout:8/11: dread - no filename 2024-04-12T20:56:21.699 INFO:tasks.workunit.client.1.smithi186.stdout:1/13: write f4 [2274016,86685] 0 2024-04-12T20:56:21.700 INFO:tasks.workunit.client.0.smithi018.stdout:7/828: link d1/db/d31/d3a/d42/fbd d1/db/d31/d45/d8c/d9d/d70/d96/d5f/da9/ddf/f10c 0 2024-04-12T20:56:21.704 INFO:tasks.workunit.client.1.smithi186.stdout:9/8: dwrite f2 [0,4194304] 0 2024-04-12T20:56:21.708 INFO:tasks.workunit.client.1.smithi186.stdout:3/25: unlink d1/c2 0 2024-04-12T20:56:21.708 INFO:tasks.workunit.client.1.smithi186.stdout:3/26: write - no filename 2024-04-12T20:56:21.710 INFO:tasks.workunit.client.1.smithi186.stdout:9/9: chown l0 942 1 2024-04-12T20:56:21.712 INFO:tasks.workunit.client.1.smithi186.stdout:8/12: creat f2 x:0 0 0 2024-04-12T20:56:21.717 INFO:tasks.workunit.client.1.smithi186.stdout:8/13: readlink l0 0 2024-04-12T20:56:21.721 INFO:tasks.workunit.client.0.smithi018.stdout:1/684: dwrite d3/d9/da/dd/d31/d32/d13/d19/d2d/f76 [4194304,4194304] 0 2024-04-12T20:56:21.721 INFO:tasks.workunit.client.1.smithi186.stdout:9/10: unlink f2 0 2024-04-12T20:56:21.726 INFO:tasks.workunit.client.1.smithi186.stdout:2/12: dread d0/f2 [0,4194304] 0 2024-04-12T20:56:21.727 INFO:tasks.workunit.client.1.smithi186.stdout:3/27: mkdir d1/d3/d4 0 2024-04-12T20:56:21.729 INFO:tasks.workunit.client.0.smithi018.stdout:7/829: dwrite d1/db/d31/d3a/d42/d4c/f99 [0,4194304] 0 2024-04-12T20:56:21.731 INFO:tasks.workunit.client.1.smithi186.stdout:1/14: dread f4 [0,4194304] 0 2024-04-12T20:56:21.735 INFO:tasks.workunit.client.0.smithi018.stdout:7/830: symlink d1/db/d31/df2/l10d 0 2024-04-12T20:56:21.744 INFO:tasks.workunit.client.1.smithi186.stdout:4/6: fsync d0/f2 0 2024-04-12T20:56:21.747 INFO:tasks.workunit.client.0.smithi018.stdout:7/831: truncate d1/d6/f52 3722491 0 2024-04-12T20:56:21.750 INFO:tasks.workunit.client.1.smithi186.stdout:4/7: chown d0/f2 676315 1 2024-04-12T20:56:21.752 INFO:tasks.workunit.client.0.smithi018.stdout:7/832: creat d1/db/d31/d45/d8c/d9d/d70/d96/d5f/da9/ddf/f10e x:0 0 0 2024-04-12T20:56:21.752 INFO:tasks.workunit.client.1.smithi186.stdout:0/17: chown d0/c3 485306 1 2024-04-12T20:56:21.753 INFO:tasks.workunit.client.1.smithi186.stdout:0/18: truncate - no filename 2024-04-12T20:56:21.753 INFO:tasks.workunit.client.1.smithi186.stdout:0/19: truncate - no filename 2024-04-12T20:56:21.753 INFO:tasks.workunit.client.1.smithi186.stdout:0/20: dwrite - no filename 2024-04-12T20:56:21.753 INFO:tasks.workunit.client.1.smithi186.stdout:5/7: rmdir d0 0 2024-04-12T20:56:21.755 INFO:tasks.workunit.client.1.smithi186.stdout:7/15: chown c2 69 1 2024-04-12T20:56:21.758 INFO:tasks.workunit.client.1.smithi186.stdout:4/8: chown d0/l1 9 1 2024-04-12T20:56:21.761 INFO:tasks.workunit.client.0.smithi018.stdout:7/833: rename d1/db/d31/d45/d8c/lb7 to d1/db/d31/d3a/d42/d4c/d6c/l10f 0 2024-04-12T20:56:21.762 INFO:tasks.workunit.client.1.smithi186.stdout:8/14: dwrite f2 [0,4194304] 0 2024-04-12T20:56:21.763 INFO:tasks.workunit.client.1.smithi186.stdout:4/9: dread - d0/f2 zero size 2024-04-12T20:56:21.764 INFO:tasks.workunit.client.0.smithi018.stdout:7/834: read d1/db/d31/d45/f9e [147284,13465] 0 2024-04-12T20:56:21.766 INFO:tasks.workunit.client.0.smithi018.stdout:1/685: dread d3/d9/da/dd/d31/d32/d13/f46 [0,4194304] 0 2024-04-12T20:56:21.769 INFO:tasks.workunit.client.1.smithi186.stdout:5/8: creat f2 x:0 0 0 2024-04-12T20:56:21.769 INFO:tasks.workunit.client.1.smithi186.stdout:3/28: rmdir d1 39 2024-04-12T20:56:21.770 INFO:tasks.workunit.client.1.smithi186.stdout:3/29: write - no filename 2024-04-12T20:56:21.770 INFO:tasks.workunit.client.1.smithi186.stdout:3/30: truncate - no filename 2024-04-12T20:56:21.770 INFO:tasks.workunit.client.0.smithi018.stdout:7/835: mknod d1/c110 0 2024-04-12T20:56:21.770 INFO:tasks.workunit.client.0.smithi018.stdout:1/686: fdatasync d3/d9/da/dd/d31/d32/d33/d47/fb2 0 2024-04-12T20:56:21.774 INFO:tasks.workunit.client.1.smithi186.stdout:4/10: read - d0/f2 zero size 2024-04-12T20:56:21.774 INFO:tasks.workunit.client.1.smithi186.stdout:8/15: rename l0 to l3 0 2024-04-12T20:56:21.776 INFO:tasks.workunit.client.0.smithi018.stdout:1/687: rmdir d3/d9/da/dd/d26 39 2024-04-12T20:56:21.783 INFO:tasks.workunit.client.0.smithi018.stdout:1/688: rename d3/d9/da/dd/d31/d32/d13/d19/d2d/d4e/d58/c5a to d3/d9/ce8 0 2024-04-12T20:56:21.783 INFO:tasks.workunit.client.1.smithi186.stdout:0/21: link d0/c3 d0/c4 0 2024-04-12T20:56:21.783 INFO:tasks.workunit.client.1.smithi186.stdout:2/13: dwrite d0/f1 [0,4194304] 0 2024-04-12T20:56:21.784 INFO:tasks.workunit.client.0.smithi018.stdout:7/836: dread d1/db/d31/f63 [0,4194304] 0 2024-04-12T20:56:21.787 INFO:tasks.workunit.client.1.smithi186.stdout:5/9: mknod c3 0 2024-04-12T20:56:21.790 INFO:tasks.workunit.client.1.smithi186.stdout:4/11: dread - d0/f2 zero size 2024-04-12T20:56:21.790 INFO:tasks.workunit.client.1.smithi186.stdout:9/11: dwrite f1 [0,4194304] 0 2024-04-12T20:56:21.791 INFO:tasks.workunit.client.0.smithi018.stdout:1/689: dread d3/d9/da/dd/d31/f59 [0,4194304] 0 2024-04-12T20:56:21.797 INFO:tasks.workunit.client.0.smithi018.stdout:7/837: dread d1/db/f9f [4194304,4194304] 0 2024-04-12T20:56:21.797 INFO:tasks.workunit.client.1.smithi186.stdout:0/22: symlink d0/l5 0 2024-04-12T20:56:21.800 INFO:tasks.workunit.client.1.smithi186.stdout:3/31: symlink d1/d3/d4/l5 0 2024-04-12T20:56:21.800 INFO:tasks.workunit.client.1.smithi186.stdout:3/32: read - no filename 2024-04-12T20:56:21.800 INFO:tasks.workunit.client.1.smithi186.stdout:3/33: dread - no filename 2024-04-12T20:56:21.802 INFO:tasks.workunit.client.0.smithi018.stdout:1/690: mkdir d3/d9/da/dd/d31/d32/d13/d19/d2d/d4e/d5e/da0/de2/de9 0 2024-04-12T20:56:21.802 INFO:tasks.workunit.client.0.smithi018.stdout:7/838: rmdir d1/db/d31/d45/d53 39 2024-04-12T20:56:21.802 INFO:tasks.workunit.client.1.smithi186.stdout:2/14: chown d0/f1 10513 1 2024-04-12T20:56:21.804 INFO:tasks.workunit.client.1.smithi186.stdout:9/12: write f1 [640496,114524] 0 2024-04-12T20:56:21.807 INFO:tasks.workunit.client.1.smithi186.stdout:4/12: mkdir d0/d3 0 2024-04-12T20:56:21.807 INFO:tasks.workunit.client.0.smithi018.stdout:1/691: fdatasync d3/d9/da/dd/d31/d32/d13/d19/fb4 0 2024-04-12T20:56:21.808 INFO:tasks.workunit.client.1.smithi186.stdout:1/15: dwrite f4 [4194304,4194304] 0 2024-04-12T20:56:21.808 INFO:tasks.workunit.client.1.smithi186.stdout:0/23: creat d0/f6 x:0 0 0 2024-04-12T20:56:21.810 INFO:tasks.workunit.client.0.smithi018.stdout:7/839: unlink d1/l4e 0 2024-04-12T20:56:21.811 INFO:tasks.workunit.client.0.smithi018.stdout:1/692: creat d3/d65/fea x:0 0 0 2024-04-12T20:56:21.812 INFO:tasks.workunit.client.1.smithi186.stdout:5/10: link f2 f4 0 2024-04-12T20:56:21.812 INFO:tasks.workunit.client.1.smithi186.stdout:5/11: rmdir - no directory 2024-04-12T20:56:21.814 INFO:tasks.workunit.client.0.smithi018.stdout:7/840: fsync d1/db/d31/d3a/d42/d4c/d6c/dc3/f106 0 2024-04-12T20:56:21.815 INFO:tasks.workunit.client.0.smithi018.stdout:6/999: sync 2024-04-12T20:56:21.815 INFO:tasks.workunit.client.0.smithi018.stdout:5/928: sync 2024-04-12T20:56:21.815 INFO:tasks.workunit.client.1.smithi186.stdout:7/16: dwrite f4 [0,4194304] 0 2024-04-12T20:56:21.816 INFO:tasks.workunit.client.0.smithi018.stdout:4/786: sync 2024-04-12T20:56:21.816 INFO:tasks.workunit.client.0.smithi018.stdout:2/782: sync 2024-04-12T20:56:21.817 INFO:tasks.workunit.client.1.smithi186.stdout:7/17: rmdir - no directory 2024-04-12T20:56:21.821 INFO:tasks.workunit.client.0.smithi018.stdout:7/841: read - d1/db/d31/d3a/d42/d4c/d6c/ffd zero size 2024-04-12T20:56:21.821 INFO:tasks.workunit.client.0.smithi018.stdout:5/929: truncate d0/d2/d33/d46/d1e/f104 493407 0 2024-04-12T20:56:21.822 INFO:tasks.workunit.client.1.smithi186.stdout:4/13: creat d0/d3/f4 x:0 0 0 2024-04-12T20:56:21.824 INFO:tasks.workunit.client.0.smithi018.stdout:5/930: chown d0/d2/fa6 3 1 2024-04-12T20:56:21.824 INFO:tasks.workunit.client.1.smithi186.stdout:8/16: dread f2 [0,4194304] 0 2024-04-12T20:56:21.825 INFO:tasks.workunit.client.1.smithi186.stdout:8/17: rmdir - no directory 2024-04-12T20:56:21.829 INFO:tasks.workunit.client.1.smithi186.stdout:4/14: mknod d0/d3/c5 0 2024-04-12T20:56:21.834 INFO:tasks.workunit.client.1.smithi186.stdout:3/34: getdents d1/d3/d4 0 2024-04-12T20:56:21.834 INFO:tasks.workunit.client.1.smithi186.stdout:9/13: dread f1 [0,4194304] 0 2024-04-12T20:56:21.840 INFO:tasks.workunit.client.1.smithi186.stdout:3/35: readlink d1/d3/d4/l5 0 2024-04-12T20:56:21.840 INFO:tasks.workunit.client.1.smithi186.stdout:3/36: dwrite - no filename 2024-04-12T20:56:21.841 INFO:tasks.workunit.client.1.smithi186.stdout:7/18: truncate f4 640149 0 2024-04-12T20:56:21.842 INFO:tasks.workunit.client.1.smithi186.stdout:0/24: getdents d0 0 2024-04-12T20:56:21.843 INFO:tasks.workunit.client.1.smithi186.stdout:4/15: creat d0/f6 x:0 0 0 2024-04-12T20:56:21.844 INFO:tasks.workunit.client.1.smithi186.stdout:0/25: stat d0 0 2024-04-12T20:56:21.844 INFO:tasks.workunit.client.1.smithi186.stdout:2/15: dwrite d0/f2 [0,4194304] 0 2024-04-12T20:56:21.845 INFO:tasks.workunit.client.1.smithi186.stdout:3/37: rename d1/d3 to d1/d3/d4/d6 22 2024-04-12T20:56:21.846 INFO:tasks.workunit.client.1.smithi186.stdout:4/16: fdatasync d0/f6 0 2024-04-12T20:56:21.848 INFO:tasks.workunit.client.1.smithi186.stdout:9/14: mknod d3/c4 0 2024-04-12T20:56:21.849 INFO:tasks.workunit.client.1.smithi186.stdout:4/17: stat d0/d3 0 2024-04-12T20:56:21.850 INFO:tasks.workunit.client.1.smithi186.stdout:1/16: dwrite f4 [4194304,4194304] 0 2024-04-12T20:56:21.850 INFO:tasks.workunit.client.1.smithi186.stdout:9/15: chown f1 41946 1 2024-04-12T20:56:21.851 INFO:tasks.workunit.client.1.smithi186.stdout:9/16: chown d3 53328 1 2024-04-12T20:56:21.857 INFO:tasks.workunit.client.0.smithi018.stdout:2/783: creat d5/d8/d10/d43/de2/f10c x:0 0 0 2024-04-12T20:56:21.857 INFO:tasks.workunit.client.0.smithi018.stdout:2/784: write d5/d12/d54/d3e/fcb [190401,83500] 0 2024-04-12T20:56:21.858 INFO:tasks.workunit.client.0.smithi018.stdout:7/842: symlink d1/db/d31/d45/d8c/d9d/d70/dc4/l111 0 2024-04-12T20:56:21.858 INFO:tasks.workunit.client.0.smithi018.stdout:5/931: dwrite d0/d2/d33/d46/d35/f62 [0,4194304] 0 2024-04-12T20:56:21.858 INFO:tasks.workunit.client.1.smithi186.stdout:0/26: rename d0/c2 to d0/c7 0 2024-04-12T20:56:21.858 INFO:tasks.workunit.client.1.smithi186.stdout:9/17: write f1 [1064765,1302] 0 2024-04-12T20:56:21.859 INFO:tasks.workunit.client.1.smithi186.stdout:0/27: dread - d0/f6 zero size 2024-04-12T20:56:21.860 INFO:tasks.workunit.client.1.smithi186.stdout:7/19: fdatasync f4 0 2024-04-12T20:56:21.862 INFO:tasks.workunit.client.1.smithi186.stdout:0/28: dread - d0/f6 zero size 2024-04-12T20:56:21.862 INFO:tasks.workunit.client.1.smithi186.stdout:7/20: fsync f4 0 2024-04-12T20:56:21.863 INFO:tasks.workunit.client.0.smithi018.stdout:4/787: dread d3/d5d/d60/dc5/d43/f62 [0,4194304] 0 2024-04-12T20:56:21.865 INFO:tasks.workunit.client.1.smithi186.stdout:7/21: chown c2 16661654 1 2024-04-12T20:56:21.866 INFO:tasks.workunit.client.1.smithi186.stdout:2/16: dread d0/f1 [0,4194304] 0 2024-04-12T20:56:21.868 INFO:tasks.workunit.client.1.smithi186.stdout:1/17: unlink c5 0 2024-04-12T20:56:21.868 INFO:tasks.workunit.client.1.smithi186.stdout:3/38: link d1/d3/d4/l5 d1/l7 0 2024-04-12T20:56:21.869 INFO:tasks.workunit.client.1.smithi186.stdout:3/39: chown d1/d3 138586862 1 2024-04-12T20:56:21.871 INFO:tasks.workunit.client.1.smithi186.stdout:7/22: mknod c5 0 2024-04-12T20:56:21.876 INFO:tasks.workunit.client.0.smithi018.stdout:1/693: dread d3/d9/da/dd/d31/d32/d13/d19/d2d/d4e/d58/d61/d71/f91 [0,4194304] 0 2024-04-12T20:56:21.877 INFO:tasks.workunit.client.1.smithi186.stdout:1/18: mknod c6 0 2024-04-12T20:56:21.878 INFO:tasks.workunit.client.0.smithi018.stdout:1/694: stat d3/d9/da/dd/d31/la1 0 2024-04-12T20:56:21.880 INFO:tasks.workunit.client.1.smithi186.stdout:1/19: write f4 [2836894,38441] 0 2024-04-12T20:56:21.880 INFO:tasks.workunit.client.1.smithi186.stdout:2/17: dread d0/f1 [0,4194304] 0 2024-04-12T20:56:21.881 INFO:tasks.workunit.client.1.smithi186.stdout:4/18: getdents d0 0 2024-04-12T20:56:21.882 INFO:tasks.workunit.client.1.smithi186.stdout:7/23: creat f6 x:0 0 0 2024-04-12T20:56:21.886 INFO:tasks.workunit.client.1.smithi186.stdout:0/29: dwrite d0/f6 [0,4194304] 0 2024-04-12T20:56:21.891 INFO:tasks.workunit.client.1.smithi186.stdout:7/24: mkdir d7 0 2024-04-12T20:56:21.892 INFO:tasks.workunit.client.1.smithi186.stdout:3/40: link d1/l7 d1/d3/l8 0 2024-04-12T20:56:21.893 INFO:tasks.workunit.client.1.smithi186.stdout:3/41: write - no filename 2024-04-12T20:56:21.893 INFO:tasks.workunit.client.1.smithi186.stdout:3/42: truncate - no filename 2024-04-12T20:56:21.893 INFO:tasks.workunit.client.1.smithi186.stdout:4/19: rename d0/f2 to d0/d3/f7 0 2024-04-12T20:56:21.897 INFO:tasks.workunit.client.1.smithi186.stdout:7/25: dread f4 [0,4194304] 0 2024-04-12T20:56:21.901 INFO:tasks.workunit.client.0.smithi018.stdout:2/785: link d5/f95 d5/d12/d54/f10d 0 2024-04-12T20:56:21.902 INFO:tasks.workunit.client.0.smithi018.stdout:1/695: write d3/d9/da/dd/d31/d32/d13/d19/d2d/d4e/f8c [700080,78384] 0 2024-04-12T20:56:21.903 INFO:tasks.workunit.client.0.smithi018.stdout:7/843: creat d1/db/d31/d3a/f112 x:0 0 0 2024-04-12T20:56:21.906 INFO:tasks.workunit.client.0.smithi018.stdout:1/696: write d3/d9/da/dd/f9d [2099579,27150] 0 2024-04-12T20:56:21.906 INFO:tasks.workunit.client.0.smithi018.stdout:7/844: chown d1/db/d31/d45/d8c/d9d/d94/cdb 194 1 2024-04-12T20:56:21.906 INFO:tasks.workunit.client.0.smithi018.stdout:4/788: creat d3/d5d/d60/dc5/f103 x:0 0 0 2024-04-12T20:56:21.908 INFO:tasks.workunit.client.0.smithi018.stdout:1/697: fdatasync d3/d65/fea 0 2024-04-12T20:56:21.910 INFO:tasks.workunit.client.0.smithi018.stdout:2/786: creat d5/d8/f10e x:0 0 0 2024-04-12T20:56:21.913 INFO:tasks.workunit.client.0.smithi018.stdout:4/789: symlink d3/d5d/d60/dc5/d53/de7/l104 0 2024-04-12T20:56:21.916 INFO:tasks.workunit.client.0.smithi018.stdout:7/845: truncate d1/db/d31/d3a/d42/f75 4125698 0 2024-04-12T20:56:21.921 INFO:tasks.workunit.client.0.smithi018.stdout:4/790: rename d3/d5d/d60/dc5/d43/d4b/de6/ff5 to d3/d5d/d60/dc5/d43/d4b/f105 0 2024-04-12T20:56:21.924 INFO:tasks.workunit.client.0.smithi018.stdout:1/698: dwrite d3/f4 [0,4194304] 0 2024-04-12T20:56:21.925 INFO:tasks.workunit.client.0.smithi018.stdout:7/846: creat d1/db/d31/d3a/d42/f113 x:0 0 0 2024-04-12T20:56:21.927 INFO:tasks.workunit.client.0.smithi018.stdout:4/791: stat d3/d32/fd 0 2024-04-12T20:56:21.929 INFO:tasks.workunit.client.0.smithi018.stdout:4/792: chown d3/db/l4a 10700 1 2024-04-12T20:56:21.933 INFO:tasks.workunit.client.0.smithi018.stdout:2/787: rename d5/d8/d25/ca9 to d5/d8/d10/d43/de2/d24/d39/c10f 0 2024-04-12T20:56:21.933 INFO:tasks.workunit.client.0.smithi018.stdout:0/838: sync 2024-04-12T20:56:21.933 INFO:tasks.workunit.client.0.smithi018.stdout:3/993: sync 2024-04-12T20:56:21.934 INFO:tasks.workunit.client.0.smithi018.stdout:7/847: creat d1/db/d31/d30/dc6/f114 x:0 0 0 2024-04-12T20:56:21.935 INFO:tasks.workunit.client.0.smithi018.stdout:1/699: symlink d3/d9/da/dd/d31/d32/d13/d19/d2d/d4e/d5e/leb 0 2024-04-12T20:56:21.938 INFO:tasks.workunit.client.1.smithi186.stdout:4/20: rename d0/d3/c5 to d0/c8 0 2024-04-12T20:56:21.940 INFO:tasks.workunit.client.1.smithi186.stdout:4/21: dread - d0/d3/f4 zero size 2024-04-12T20:56:21.940 INFO:tasks.workunit.client.0.smithi018.stdout:0/839: fdatasync da/d15/d4d/fe9 0 2024-04-12T20:56:21.942 INFO:tasks.workunit.client.1.smithi186.stdout:7/26: creat d7/f8 x:0 0 0 2024-04-12T20:56:21.942 INFO:tasks.workunit.client.0.smithi018.stdout:4/793: symlink d3/db/de3/dfc/l106 0 2024-04-12T20:56:21.943 INFO:tasks.workunit.client.0.smithi018.stdout:1/700: creat d3/d9/da/fec x:0 0 0 2024-04-12T20:56:21.944 INFO:tasks.workunit.client.1.smithi186.stdout:9/18: getdents d3 0 2024-04-12T20:56:21.945 INFO:tasks.workunit.client.1.smithi186.stdout:9/19: write f1 [3087264,91277] 0 2024-04-12T20:56:21.945 INFO:tasks.workunit.client.1.smithi186.stdout:4/22: creat d0/d3/f9 x:0 0 0 2024-04-12T20:56:21.946 INFO:tasks.workunit.client.0.smithi018.stdout:7/848: creat d1/db/f115 x:0 0 0 2024-04-12T20:56:21.948 INFO:tasks.workunit.client.0.smithi018.stdout:2/788: readlink d5/d8/d10/d43/de2/d24/l6a 0 2024-04-12T20:56:21.949 INFO:tasks.workunit.client.0.smithi018.stdout:3/994: mkdir d2/d6/d30/d31/d38/d3a/d76/da2/d13c 0 2024-04-12T20:56:21.950 INFO:tasks.workunit.client.1.smithi186.stdout:3/43: rename d1/l7 to d1/d3/d4/l9 0 2024-04-12T20:56:21.950 INFO:tasks.workunit.client.1.smithi186.stdout:3/44: dwrite - no filename 2024-04-12T20:56:21.950 INFO:tasks.workunit.client.1.smithi186.stdout:3/45: dwrite - no filename 2024-04-12T20:56:21.950 INFO:tasks.workunit.client.1.smithi186.stdout:3/46: dread - no filename 2024-04-12T20:56:21.956 INFO:tasks.workunit.client.1.smithi186.stdout:4/23: creat d0/d3/fa x:0 0 0 2024-04-12T20:56:21.957 INFO:tasks.workunit.client.1.smithi186.stdout:4/24: stat d0 0 2024-04-12T20:56:21.958 INFO:tasks.workunit.client.1.smithi186.stdout:4/25: fsync d0/d3/f4 0 2024-04-12T20:56:21.962 INFO:tasks.workunit.client.0.smithi018.stdout:7/849: creat d1/db/d31/d45/d8c/d9d/d69/f116 x:0 0 0 2024-04-12T20:56:21.962 INFO:tasks.workunit.client.1.smithi186.stdout:9/20: dwrite f1 [0,4194304] 0 2024-04-12T20:56:21.962 INFO:tasks.workunit.client.1.smithi186.stdout:0/30: rename d0/c7 to d0/c8 0 2024-04-12T20:56:21.964 INFO:tasks.workunit.client.1.smithi186.stdout:3/47: chown d1/d3/l8 656929674 1 2024-04-12T20:56:21.964 INFO:tasks.workunit.client.0.smithi018.stdout:3/995: symlink d2/d6/d34/d71/l13d 0 2024-04-12T20:56:21.964 INFO:tasks.workunit.client.1.smithi186.stdout:3/48: chown d1/d3/d4 315784060 1 2024-04-12T20:56:21.965 INFO:tasks.workunit.client.0.smithi018.stdout:3/996: readlink d2/d6/d34/d71/db2/dfc/l137 0 2024-04-12T20:56:21.966 INFO:tasks.workunit.client.1.smithi186.stdout:4/26: creat d0/d3/fb x:0 0 0 2024-04-12T20:56:21.968 INFO:tasks.workunit.client.0.smithi018.stdout:2/789: creat d5/d7/d31/d62/f110 x:0 0 0 2024-04-12T20:56:21.969 INFO:tasks.workunit.client.1.smithi186.stdout:0/31: mknod d0/c9 0 2024-04-12T20:56:21.971 INFO:tasks.workunit.client.0.smithi018.stdout:0/840: rename da/d15/l53 to da/d15/d17/l118 0 2024-04-12T20:56:21.973 INFO:tasks.workunit.client.0.smithi018.stdout:0/841: stat da/d15/d17/d5c/f64 0 2024-04-12T20:56:21.973 INFO:tasks.workunit.client.1.smithi186.stdout:4/27: mkdir d0/dc 0 2024-04-12T20:56:21.973 INFO:tasks.workunit.client.1.smithi186.stdout:4/28: chown d0/d3/f7 39 1 2024-04-12T20:56:21.973 INFO:tasks.workunit.client.1.smithi186.stdout:1/20: truncate f4 252971 0 2024-04-12T20:56:21.975 INFO:tasks.workunit.client.1.smithi186.stdout:4/29: dread - d0/d3/f7 zero size 2024-04-12T20:56:21.977 INFO:tasks.workunit.client.1.smithi186.stdout:1/21: symlink l7 0 2024-04-12T20:56:21.977 INFO:tasks.workunit.client.1.smithi186.stdout:1/22: rmdir - no directory 2024-04-12T20:56:21.977 INFO:tasks.workunit.client.1.smithi186.stdout:7/27: rmdir d7 39 2024-04-12T20:56:21.978 INFO:tasks.workunit.client.1.smithi186.stdout:1/23: readlink l3 0 2024-04-12T20:56:21.980 INFO:tasks.workunit.client.0.smithi018.stdout:3/997: mkdir d2/d6/d34/d13e 0 2024-04-12T20:56:21.982 INFO:tasks.workunit.client.1.smithi186.stdout:9/21: dwrite f1 [0,4194304] 0 2024-04-12T20:56:21.984 INFO:tasks.workunit.client.0.smithi018.stdout:0/842: rmdir da/d15/d17/d21/d36/da2/dc3/d83/d8b 39 2024-04-12T20:56:21.986 INFO:tasks.workunit.client.0.smithi018.stdout:4/794: getdents d3/d5d/d60/dc5/d3d/d80 0 2024-04-12T20:56:21.986 INFO:tasks.workunit.client.1.smithi186.stdout:0/32: dread d0/f6 [0,4194304] 0 2024-04-12T20:56:21.987 INFO:tasks.workunit.client.1.smithi186.stdout:9/22: readlink l0 0 2024-04-12T20:56:21.991 INFO:tasks.workunit.client.0.smithi018.stdout:3/998: mkdir d2/d6/d30/d31/d38/d3a/dd0/de1/d13f 0 2024-04-12T20:56:21.992 INFO:tasks.workunit.client.1.smithi186.stdout:3/49: getdents d1/d3/d4 0 2024-04-12T20:56:21.993 INFO:tasks.workunit.client.0.smithi018.stdout:7/850: creat d1/db/d31/d45/d8c/d9d/d70/d96/d5f/f117 x:0 0 0 2024-04-12T20:56:21.994 INFO:tasks.workunit.client.1.smithi186.stdout:9/23: creat d3/f5 x:0 0 0 2024-04-12T20:56:21.994 INFO:tasks.workunit.client.0.smithi018.stdout:4/795: mknod d3/d5d/d60/dc5/d43/d4b/de6/c107 0 2024-04-12T20:56:22.000 INFO:tasks.workunit.client.1.smithi186.stdout:3/50: rename d1/d3/d4 to d1/d3/da 0 2024-04-12T20:56:22.001 INFO:tasks.workunit.client.1.smithi186.stdout:3/51: read - no filename 2024-04-12T20:56:22.001 INFO:tasks.workunit.client.1.smithi186.stdout:3/52: dwrite - no filename 2024-04-12T20:56:22.002 INFO:tasks.workunit.client.0.smithi018.stdout:4/796: mknod d3/d5d/d60/dc5/d53/de7/c108 0 2024-04-12T20:56:22.003 INFO:tasks.workunit.client.1.smithi186.stdout:9/24: symlink d3/l6 0 2024-04-12T20:56:22.004 INFO:tasks.workunit.client.0.smithi018.stdout:3/999: creat d2/d6/d30/f140 x:0 0 0 2024-04-12T20:56:22.004 INFO:tasks.workunit.client.1.smithi186.stdout:1/24: link c1 c8 0 2024-04-12T20:56:22.007 INFO:tasks.workunit.client.1.smithi186.stdout:3/53: stat d1/d3/da/l9 0 2024-04-12T20:56:22.007 INFO:tasks.workunit.client.1.smithi186.stdout:3/54: dread - no filename 2024-04-12T20:56:22.007 INFO:tasks.workunit.client.1.smithi186.stdout:3/55: write - no filename 2024-04-12T20:56:22.008 INFO:tasks.workunit.client.0.smithi018.stdout:5/932: write d0/d2/d11/d37/f4c [1977686,24905] 0 2024-04-12T20:56:22.009 INFO:tasks.workunit.client.1.smithi186.stdout:1/25: write f4 [542326,40595] 0 2024-04-12T20:56:22.012 INFO:tasks.workunit.client.0.smithi018.stdout:5/933: dread - d0/d2/d33/d46/d35/f135 zero size 2024-04-12T20:56:22.013 INFO:tasks.workunit.client.0.smithi018.stdout:4/797: link d3/d5d/d60/d64/d72/fa0 d3/d5d/d60/dc5/d43/d4b/f109 0 2024-04-12T20:56:22.013 INFO:tasks.workunit.client.1.smithi186.stdout:7/28: dwrite f4 [0,4194304] 0 2024-04-12T20:56:22.018 INFO:tasks.workunit.client.0.smithi018.stdout:5/934: fdatasync d0/d2/d33/d46/d1e/d9a/f141 0 2024-04-12T20:56:22.020 INFO:tasks.workunit.client.0.smithi018.stdout:7/851: dwrite d1/d6/fb4 [4194304,4194304] 0 2024-04-12T20:56:22.024 INFO:tasks.workunit.client.0.smithi018.stdout:4/798: rename d3/d5d/d60/dc5/l56 to d3/d5d/d60/l10a 0 2024-04-12T20:56:22.028 INFO:tasks.workunit.client.1.smithi186.stdout:4/30: dwrite d0/f6 [0,4194304] 0 2024-04-12T20:56:22.029 INFO:tasks.workunit.client.1.smithi186.stdout:3/56: rename d1/d3/l8 to d1/d3/da/lb 0 2024-04-12T20:56:22.029 INFO:tasks.workunit.client.1.smithi186.stdout:3/57: dwrite - no filename 2024-04-12T20:56:22.029 INFO:tasks.workunit.client.1.smithi186.stdout:3/58: fsync - no filename 2024-04-12T20:56:22.029 INFO:tasks.workunit.client.1.smithi186.stdout:3/59: fdatasync - no filename 2024-04-12T20:56:22.032 INFO:tasks.workunit.client.0.smithi018.stdout:4/799: dread d3/d5d/d60/dc5/d40/f6d [0,4194304] 0 2024-04-12T20:56:22.036 INFO:tasks.workunit.client.1.smithi186.stdout:0/33: dwrite d0/f6 [0,4194304] 0 2024-04-12T20:56:22.037 INFO:tasks.workunit.client.0.smithi018.stdout:7/852: unlink d1/d6/l26 0 2024-04-12T20:56:22.038 INFO:tasks.workunit.client.0.smithi018.stdout:5/935: dread d0/d2/d33/d46/d1e/d2e/f8a [0,4194304] 0 2024-04-12T20:56:22.039 INFO:tasks.workunit.client.0.smithi018.stdout:7/853: stat d1/db/d31/d45/d8c/d9d/d70/d96/ca 0 2024-04-12T20:56:22.042 INFO:tasks.workunit.client.1.smithi186.stdout:4/31: chown d0/d3/fa 93373975 1 2024-04-12T20:56:22.042 INFO:tasks.workunit.client.1.smithi186.stdout:3/60: creat d1/fc x:0 0 0 2024-04-12T20:56:22.044 INFO:tasks.workunit.client.1.smithi186.stdout:0/34: read d0/f6 [798952,35931] 0 2024-04-12T20:56:22.045 INFO:tasks.workunit.client.1.smithi186.stdout:7/29: rename f6 to d7/f9 0 2024-04-12T20:56:22.047 INFO:tasks.workunit.client.0.smithi018.stdout:2/790: write d5/d8/f57 [942970,79903] 0 2024-04-12T20:56:22.049 INFO:tasks.workunit.client.1.smithi186.stdout:9/25: dwrite d3/f5 [0,4194304] 0 2024-04-12T20:56:22.050 INFO:tasks.workunit.client.1.smithi186.stdout:0/35: chown d0/l5 574 1 2024-04-12T20:56:22.051 INFO:tasks.workunit.client.0.smithi018.stdout:5/936: symlink d0/d2/d11/d37/d13c/l142 0 2024-04-12T20:56:22.052 INFO:tasks.workunit.client.1.smithi186.stdout:7/30: creat d7/fa x:0 0 0 2024-04-12T20:56:22.054 INFO:tasks.workunit.client.0.smithi018.stdout:4/800: mkdir d3/d32/d94/d10b 0 2024-04-12T20:56:22.054 INFO:tasks.workunit.client.1.smithi186.stdout:0/36: readlink d0/l5 0 2024-04-12T20:56:22.055 INFO:tasks.workunit.client.0.smithi018.stdout:5/937: chown d0/d2/d33/d46/d1e/f48 3708695 1 2024-04-12T20:56:22.057 INFO:tasks.workunit.client.1.smithi186.stdout:1/26: dwrite f4 [0,4194304] 0 2024-04-12T20:56:22.059 INFO:tasks.workunit.client.1.smithi186.stdout:4/32: getdents d0 0 2024-04-12T20:56:22.060 INFO:tasks.workunit.client.0.smithi018.stdout:2/791: symlink d5/d8/d25/l111 0 2024-04-12T20:56:22.060 INFO:tasks.workunit.client.1.smithi186.stdout:4/33: readlink d0/l1 0 2024-04-12T20:56:22.061 INFO:tasks.workunit.client.0.smithi018.stdout:7/854: symlink d1/db/d31/l118 0 2024-04-12T20:56:22.066 INFO:tasks.workunit.client.0.smithi018.stdout:0/843: truncate da/d15/d17/d71/dc1/dcc/dcb/fd5 728258 0 2024-04-12T20:56:22.066 INFO:tasks.workunit.client.1.smithi186.stdout:1/27: creat f9 x:0 0 0 2024-04-12T20:56:22.066 INFO:tasks.workunit.client.1.smithi186.stdout:7/31: rename d7/fa to d7/fb 0 2024-04-12T20:56:22.066 INFO:tasks.workunit.client.1.smithi186.stdout:3/61: getdents d1 0 2024-04-12T20:56:22.068 INFO:tasks.workunit.client.1.smithi186.stdout:0/37: fsync d0/f6 0 2024-04-12T20:56:22.068 INFO:tasks.workunit.client.1.smithi186.stdout:3/62: write d1/fc [669760,51983] 0 2024-04-12T20:56:22.069 INFO:tasks.workunit.client.0.smithi018.stdout:4/801: rename d3/d32/d94/f97 to d3/d5d/d60/dc5/d43/d4b/df2/f10c 0 2024-04-12T20:56:22.069 INFO:tasks.workunit.client.0.smithi018.stdout:0/844: write da/d15/d17/d21/d36/d6e/f7f [1336038,63270] 0 2024-04-12T20:56:22.070 INFO:tasks.workunit.client.1.smithi186.stdout:0/38: fsync d0/f6 0 2024-04-12T20:56:22.071 INFO:tasks.workunit.client.1.smithi186.stdout:1/28: symlink la 0 2024-04-12T20:56:22.072 INFO:tasks.workunit.client.1.smithi186.stdout:9/26: getdents d3 0 2024-04-12T20:56:22.073 INFO:tasks.workunit.client.1.smithi186.stdout:9/27: chown d3/f5 122 1 2024-04-12T20:56:22.076 INFO:tasks.workunit.client.1.smithi186.stdout:4/34: dread d0/f6 [0,4194304] 0 2024-04-12T20:56:22.079 INFO:tasks.workunit.client.0.smithi018.stdout:2/792: creat d5/d12/d54/dc0/dfc/f112 x:0 0 0 2024-04-12T20:56:22.080 INFO:tasks.workunit.client.1.smithi186.stdout:4/35: chown d0/d3/fb 7 1 2024-04-12T20:56:22.080 INFO:tasks.workunit.client.1.smithi186.stdout:1/29: creat fb x:0 0 0 2024-04-12T20:56:22.081 INFO:tasks.workunit.client.0.smithi018.stdout:4/802: creat d3/d5d/d60/dc5/d43/d4b/de6/f10d x:0 0 0 2024-04-12T20:56:22.083 INFO:tasks.workunit.client.1.smithi186.stdout:1/30: chown l3 1 1 2024-04-12T20:56:22.083 INFO:tasks.workunit.client.1.smithi186.stdout:0/39: link d0/l5 d0/la 0 2024-04-12T20:56:22.083 INFO:tasks.workunit.client.0.smithi018.stdout:7/855: dwrite d1/d6/fa0 [4194304,4194304] 0 2024-04-12T20:56:22.084 INFO:tasks.workunit.client.1.smithi186.stdout:4/36: fdatasync d0/d3/f9 0 2024-04-12T20:56:22.084 INFO:tasks.workunit.client.1.smithi186.stdout:3/63: dread d1/fc [0,4194304] 0 2024-04-12T20:56:22.086 INFO:tasks.workunit.client.1.smithi186.stdout:1/31: mknod cc 0 2024-04-12T20:56:22.088 INFO:tasks.workunit.client.1.smithi186.stdout:4/37: creat d0/fd x:0 0 0 2024-04-12T20:56:22.093 INFO:tasks.workunit.client.1.smithi186.stdout:1/32: symlink ld 0 2024-04-12T20:56:22.095 INFO:tasks.workunit.client.1.smithi186.stdout:7/32: dwrite d7/f9 [0,4194304] 0 2024-04-12T20:56:22.097 INFO:tasks.workunit.client.1.smithi186.stdout:4/38: truncate d0/d3/f4 402824 0 2024-04-12T20:56:22.099 INFO:tasks.workunit.client.1.smithi186.stdout:4/39: dread - d0/d3/f9 zero size 2024-04-12T20:56:22.099 INFO:tasks.workunit.client.0.smithi018.stdout:1/701: dwrite d3/d9/da/dd/d31/d32/d13/d19/d2d/d4e/f5b [0,4194304] 0 2024-04-12T20:56:22.100 INFO:tasks.workunit.client.1.smithi186.stdout:7/33: creat d7/fc x:0 0 0 2024-04-12T20:56:22.104 INFO:tasks.workunit.client.0.smithi018.stdout:4/803: mknod d3/d5d/c10e 0 2024-04-12T20:56:22.105 INFO:tasks.workunit.client.1.smithi186.stdout:7/34: creat d7/fd x:0 0 0 2024-04-12T20:56:22.106 INFO:tasks.workunit.client.0.smithi018.stdout:0/845: link da/d15/d17/d71/dc1/dcc/f4f da/d15/d17/d21/d36/da2/dc3/da9/f119 0 2024-04-12T20:56:22.107 INFO:tasks.workunit.client.1.smithi186.stdout:0/40: dwrite d0/f6 [0,4194304] 0 2024-04-12T20:56:22.108 INFO:tasks.workunit.client.0.smithi018.stdout:5/938: truncate d0/d2/d33/d46/d35/f79 3690490 0 2024-04-12T20:56:22.109 INFO:tasks.workunit.client.1.smithi186.stdout:7/35: creat d7/fe x:0 0 0 2024-04-12T20:56:22.110 INFO:tasks.workunit.client.0.smithi018.stdout:7/856: symlink d1/db/d31/d45/d8c/d9d/d70/d96/d5f/da9/ddf/d108/l119 0 2024-04-12T20:56:22.110 INFO:tasks.workunit.client.1.smithi186.stdout:0/41: stat d0/c3 0 2024-04-12T20:56:22.112 INFO:tasks.workunit.client.0.smithi018.stdout:7/857: truncate d1/d6/f67 995519 0 2024-04-12T20:56:22.113 INFO:tasks.workunit.client.0.smithi018.stdout:4/804: rename d3/d5d/d60/dc5/d53/lb4 to d3/d5d/d60/dc5/d43/d4b/ddc/l10f 0 2024-04-12T20:56:22.114 INFO:tasks.workunit.client.1.smithi186.stdout:7/36: write f4 [2961208,21237] 0 2024-04-12T20:56:22.114 INFO:tasks.workunit.client.1.smithi186.stdout:0/42: mknod d0/cb 0 2024-04-12T20:56:22.114 INFO:tasks.workunit.client.0.smithi018.stdout:5/939: mknod d0/d2/d33/d46/d35/d4d/db3/c143 0 2024-04-12T20:56:22.117 INFO:tasks.workunit.client.0.smithi018.stdout:0/846: rename da/d15/d17/d21/d36/da2/dc3/f5e to da/d15/d17/d21/d36/d88/d117/f11a 0 2024-04-12T20:56:22.117 INFO:tasks.workunit.client.0.smithi018.stdout:4/805: chown d3/d5d/d60/d64/d4f/dc7/fec 10997991 1 2024-04-12T20:56:22.120 INFO:tasks.workunit.client.0.smithi018.stdout:0/847: stat da/d15/d17/d21/d36/da2/dc3/l4b 0 2024-04-12T20:56:22.121 INFO:tasks.workunit.client.1.smithi186.stdout:1/33: dwrite fb [0,4194304] 0 2024-04-12T20:56:22.123 INFO:tasks.workunit.client.1.smithi186.stdout:0/43: mknod d0/cc 0 2024-04-12T20:56:22.136 INFO:tasks.workunit.client.1.smithi186.stdout:4/40: dwrite d0/d3/fa [0,4194304] 0 2024-04-12T20:56:22.138 INFO:tasks.workunit.client.1.smithi186.stdout:0/44: creat d0/fd x:0 0 0 2024-04-12T20:56:22.139 INFO:tasks.workunit.client.1.smithi186.stdout:4/41: chown d0/dc 5037 1 2024-04-12T20:56:22.145 INFO:tasks.workunit.client.0.smithi018.stdout:7/858: chown d1/db/d31/d45/d8c/d9d/d70/lf1 0 1 2024-04-12T20:56:22.148 INFO:tasks.workunit.client.1.smithi186.stdout:7/37: getdents d7 0 2024-04-12T20:56:22.152 INFO:tasks.workunit.client.0.smithi018.stdout:5/940: rename d0/d2/d33/d46/d35/f79 to d0/d5b/f144 0 2024-04-12T20:56:22.156 INFO:tasks.workunit.client.1.smithi186.stdout:4/42: creat d0/dc/fe x:0 0 0 2024-04-12T20:56:22.157 INFO:tasks.workunit.client.0.smithi018.stdout:1/702: write d3/d9/da/dd/d31/d32/d13/d19/d2d/f4d [5487596,40920] 0 2024-04-12T20:56:22.162 INFO:tasks.workunit.client.1.smithi186.stdout:7/38: creat d7/ff x:0 0 0 2024-04-12T20:56:22.163 INFO:tasks.workunit.client.1.smithi186.stdout:4/43: mknod d0/cf 0 2024-04-12T20:56:22.164 INFO:tasks.workunit.client.1.smithi186.stdout:1/34: dwrite fb [0,4194304] 0 2024-04-12T20:56:22.164 INFO:tasks.workunit.client.1.smithi186.stdout:7/39: chown d7 1 1 2024-04-12T20:56:22.165 INFO:tasks.workunit.client.0.smithi018.stdout:5/941: rmdir d0/d2/d33/d46/d1e/d2e 39 2024-04-12T20:56:22.166 INFO:tasks.workunit.client.1.smithi186.stdout:1/35: dread - f9 zero size 2024-04-12T20:56:22.167 INFO:tasks.workunit.client.0.smithi018.stdout:1/703: write d3/d9/da/dd/d31/f35 [2805121,118709] 0 2024-04-12T20:56:22.167 INFO:tasks.workunit.client.0.smithi018.stdout:7/859: write d1/db/d31/d45/d8c/d9d/d70/d96/d5f/fcc [2702930,41499] 0 2024-04-12T20:56:22.168 INFO:tasks.workunit.client.1.smithi186.stdout:4/44: chown d0/d3/f4 300659 1 2024-04-12T20:56:22.168 INFO:tasks.workunit.client.0.smithi018.stdout:0/848: write da/d15/d17/d21/d36/d47/f8c [862854,85782] 0 2024-04-12T20:56:22.171 INFO:tasks.workunit.client.0.smithi018.stdout:2/793: dread d5/d8/d10/d43/de2/d24/f36 [0,4194304] 0 2024-04-12T20:56:22.173 INFO:tasks.workunit.client.1.smithi186.stdout:7/40: mkdir d7/d10 0 2024-04-12T20:56:22.173 INFO:tasks.workunit.client.1.smithi186.stdout:0/45: dwrite d0/fd [0,4194304] 0 2024-04-12T20:56:22.174 INFO:tasks.workunit.client.0.smithi018.stdout:5/942: chown d0/d2b/c13b 1209826488 1 2024-04-12T20:56:22.175 INFO:tasks.workunit.client.1.smithi186.stdout:4/45: truncate d0/dc/fe 992080 0 2024-04-12T20:56:22.176 INFO:tasks.workunit.client.1.smithi186.stdout:1/36: creat fe x:0 0 0 2024-04-12T20:56:22.176 INFO:tasks.workunit.client.1.smithi186.stdout:0/46: write d0/fd [520141,30495] 0 2024-04-12T20:56:22.179 INFO:tasks.workunit.client.1.smithi186.stdout:7/41: mkdir d7/d10/d11 0 2024-04-12T20:56:22.180 INFO:tasks.workunit.client.0.smithi018.stdout:2/794: dread - d5/d12/d54/dc0/dfc/f112 zero size 2024-04-12T20:56:22.181 INFO:tasks.workunit.client.0.smithi018.stdout:0/849: symlink da/d15/d17/d71/l11b 0 2024-04-12T20:56:22.185 INFO:tasks.workunit.client.0.smithi018.stdout:4/806: dwrite d3/d5d/d60/dc5/d43/f2a [0,4194304] 0 2024-04-12T20:56:22.187 INFO:tasks.workunit.client.1.smithi186.stdout:1/37: creat ff x:0 0 0 2024-04-12T20:56:22.188 INFO:tasks.workunit.client.0.smithi018.stdout:5/943: rmdir d0/d2/d11/dd8 39 2024-04-12T20:56:22.188 INFO:tasks.workunit.client.1.smithi186.stdout:7/42: creat d7/f12 x:0 0 0 2024-04-12T20:56:22.189 INFO:tasks.workunit.client.1.smithi186.stdout:1/38: write fe [415070,1155] 0 2024-04-12T20:56:22.189 INFO:tasks.workunit.client.1.smithi186.stdout:0/47: chown d0/la 216401109 1 2024-04-12T20:56:22.190 INFO:tasks.workunit.client.1.smithi186.stdout:7/43: dread - d7/fe zero size 2024-04-12T20:56:22.191 INFO:tasks.workunit.client.1.smithi186.stdout:1/39: dread - ff zero size 2024-04-12T20:56:22.192 INFO:tasks.workunit.client.1.smithi186.stdout:0/48: write d0/f6 [1434792,2893] 0 2024-04-12T20:56:22.193 INFO:tasks.workunit.client.1.smithi186.stdout:4/46: link d0/f6 d0/f10 0 2024-04-12T20:56:22.195 INFO:tasks.workunit.client.0.smithi018.stdout:1/704: creat d3/d9/da/dd/d31/d32/d13/d19/d2d/d4e/d7a/d82/fed x:0 0 0 2024-04-12T20:56:22.196 INFO:tasks.workunit.client.1.smithi186.stdout:7/44: creat d7/d10/f13 x:0 0 0 2024-04-12T20:56:22.200 INFO:tasks.workunit.client.1.smithi186.stdout:1/40: mknod c10 0 2024-04-12T20:56:22.202 INFO:tasks.workunit.client.1.smithi186.stdout:1/41: dread - ff zero size 2024-04-12T20:56:22.203 INFO:tasks.workunit.client.0.smithi018.stdout:2/795: creat d5/d7/d31/f113 x:0 0 0 2024-04-12T20:56:22.205 INFO:tasks.workunit.client.0.smithi018.stdout:4/807: rename d3/fc6 to d3/d5d/d60/dc5/dcc/ddf/f110 0 2024-04-12T20:56:22.205 INFO:tasks.workunit.client.0.smithi018.stdout:5/944: rmdir d0/d2/d11/d74 39 2024-04-12T20:56:22.205 INFO:tasks.workunit.client.1.smithi186.stdout:1/42: write fe [739169,24054] 0 2024-04-12T20:56:22.205 INFO:tasks.workunit.client.1.smithi186.stdout:4/47: rename d0/d3/f9 to d0/f11 0 2024-04-12T20:56:22.207 INFO:tasks.workunit.client.0.smithi018.stdout:7/860: dwrite d1/db/d31/d45/d8c/d9d/d70/fb8 [4194304,4194304] 0 2024-04-12T20:56:22.212 INFO:tasks.workunit.client.0.smithi018.stdout:7/861: write d1/db/d6f/fe0 [338932,127076] 0 2024-04-12T20:56:22.213 INFO:tasks.workunit.client.0.smithi018.stdout:0/850: creat da/d15/d17/d21/d36/da2/dc3/d83/f11c x:0 0 0 2024-04-12T20:56:22.216 INFO:tasks.workunit.client.1.smithi186.stdout:4/48: dread d0/d3/f4 [0,4194304] 0 2024-04-12T20:56:22.217 INFO:tasks.workunit.client.1.smithi186.stdout:1/43: symlink l11 0 2024-04-12T20:56:22.221 INFO:tasks.workunit.client.0.smithi018.stdout:2/796: creat d5/d7/db6/f114 x:0 0 0 2024-04-12T20:56:22.223 INFO:tasks.workunit.client.1.smithi186.stdout:0/49: dwrite d0/f6 [4194304,4194304] 0 2024-04-12T20:56:22.227 INFO:tasks.workunit.client.0.smithi018.stdout:4/808: rename d3/d5d/d60/dc5/d3d/da3/dc0 to d3/d5d/d60/dc5/d3d/d80/d111 0 2024-04-12T20:56:22.228 INFO:tasks.workunit.client.1.smithi186.stdout:7/45: dwrite d7/ff [0,4194304] 0 2024-04-12T20:56:22.228 INFO:tasks.workunit.client.0.smithi018.stdout:7/862: fsync d1/db/d31/d45/d8c/d9d/d70/d96/d5f/fac 0 2024-04-12T20:56:22.229 INFO:tasks.workunit.client.1.smithi186.stdout:1/44: rename ld to l12 0 2024-04-12T20:56:22.230 INFO:tasks.workunit.client.0.smithi018.stdout:0/851: dread da/d15/d17/d21/d36/d47/db5/f113 [0,4194304] 0 2024-04-12T20:56:22.235 INFO:tasks.workunit.client.0.smithi018.stdout:7/863: fsync d1/db/d31/d3a/d42/d4c/d6c/ffd 0 2024-04-12T20:56:22.237 INFO:tasks.workunit.client.0.smithi018.stdout:2/797: rename d5/d8/d10/d43/de2/d24/d39/fb3 to d5/d8/d10/d43/de2/d24/d39/de6/f115 0 2024-04-12T20:56:22.239 INFO:tasks.workunit.client.0.smithi018.stdout:2/798: chown d5/d7/d31/d62/f110 4 1 2024-04-12T20:56:22.240 INFO:tasks.workunit.client.0.smithi018.stdout:7/864: chown d1/db/d31/d45/d8c/d9d/d70/d96/ca 3471 1 2024-04-12T20:56:22.240 INFO:tasks.workunit.client.0.smithi018.stdout:5/945: getdents d0/d2/d33/d46/d1e/d55/d66 0 2024-04-12T20:56:22.242 INFO:tasks.workunit.client.1.smithi186.stdout:0/50: creat d0/fe x:0 0 0 2024-04-12T20:56:22.249 INFO:tasks.workunit.client.0.smithi018.stdout:0/852: unlink da/d15/d17/d71/fbc 0 2024-04-12T20:56:22.250 INFO:tasks.workunit.client.1.smithi186.stdout:7/46: creat d7/f14 x:0 0 0 2024-04-12T20:56:22.251 INFO:tasks.workunit.client.0.smithi018.stdout:1/705: dread d3/d9/da/f20 [0,4194304] 0 2024-04-12T20:56:22.251 INFO:tasks.workunit.client.1.smithi186.stdout:7/47: chown l3 1648593 1 2024-04-12T20:56:22.253 INFO:tasks.workunit.client.1.smithi186.stdout:1/45: rename l7 to l13 0 2024-04-12T20:56:22.257 INFO:tasks.workunit.client.1.smithi186.stdout:4/49: rename d0/d3 to d0/d12 0 2024-04-12T20:56:22.258 INFO:tasks.workunit.client.0.smithi018.stdout:4/809: creat d3/d5d/f112 x:0 0 0 2024-04-12T20:56:22.258 INFO:tasks.workunit.client.1.smithi186.stdout:4/50: write d0/d12/fa [4710072,16523] 0 2024-04-12T20:56:22.259 INFO:tasks.workunit.client.0.smithi018.stdout:7/865: stat d1/db/d6f/dad/cd6 0 2024-04-12T20:56:22.260 INFO:tasks.workunit.client.1.smithi186.stdout:7/48: rename d7/fd to d7/d10/f15 0 2024-04-12T20:56:22.261 INFO:tasks.workunit.client.1.smithi186.stdout:7/49: dread - d7/f14 zero size 2024-04-12T20:56:22.267 INFO:tasks.workunit.client.1.smithi186.stdout:1/46: unlink c6 0 2024-04-12T20:56:22.267 INFO:tasks.workunit.client.1.smithi186.stdout:1/47: dread - ff zero size 2024-04-12T20:56:22.269 INFO:tasks.workunit.client.1.smithi186.stdout:1/48: write fb [2210810,124244] 0 2024-04-12T20:56:22.276 INFO:tasks.workunit.client.0.smithi018.stdout:1/706: dwrite d3/d9/da/dd/d31/d32/d13/d19/f9a [0,4194304] 0 2024-04-12T20:56:22.282 INFO:tasks.workunit.client.1.smithi186.stdout:1/49: mkdir d14 0 2024-04-12T20:56:22.284 INFO:tasks.workunit.client.0.smithi018.stdout:2/799: getdents d5/d8/d10/d43/de2/d7c/d92 0 2024-04-12T20:56:22.289 INFO:tasks.workunit.client.0.smithi018.stdout:7/866: link d1/db/d31/d3a/d42/f113 d1/db/d31/d3a/d42/dcd/f11a 0 2024-04-12T20:56:22.289 INFO:tasks.workunit.client.0.smithi018.stdout:4/810: dread d3/d5d/d60/dc5/fa9 [0,4194304] 0 2024-04-12T20:56:22.289 INFO:tasks.workunit.client.0.smithi018.stdout:0/853: truncate da/d15/d17/d21/d36/d47/d81/fec 1190713 0 2024-04-12T20:56:22.293 INFO:tasks.workunit.client.1.smithi186.stdout:4/51: dwrite d0/d12/f7 [0,4194304] 0 2024-04-12T20:56:22.293 INFO:tasks.workunit.client.0.smithi018.stdout:5/946: getdents d0/d2b/d40 0 2024-04-12T20:56:22.294 INFO:tasks.workunit.client.1.smithi186.stdout:4/52: stat d0/d12/f7 0 2024-04-12T20:56:22.295 INFO:tasks.workunit.client.1.smithi186.stdout:4/53: stat d0/dc/fe 0 2024-04-12T20:56:22.296 INFO:tasks.workunit.client.1.smithi186.stdout:4/54: stat d0/d12/f4 0 2024-04-12T20:56:22.298 INFO:tasks.workunit.client.1.smithi186.stdout:0/51: truncate d0/fd 601390 0 2024-04-12T20:56:22.301 INFO:tasks.workunit.client.1.smithi186.stdout:7/50: rmdir d7/d10/d11 0 2024-04-12T20:56:22.306 INFO:tasks.workunit.client.1.smithi186.stdout:1/50: getdents d14 0 2024-04-12T20:56:22.307 INFO:tasks.workunit.client.0.smithi018.stdout:1/707: getdents d3/d9/da/dd 0 2024-04-12T20:56:22.309 INFO:tasks.workunit.client.1.smithi186.stdout:4/55: write d0/f11 [857926,106546] 0 2024-04-12T20:56:22.316 INFO:tasks.workunit.client.1.smithi186.stdout:7/51: write d7/d10/f15 [271651,33275] 0 2024-04-12T20:56:22.317 INFO:tasks.workunit.client.0.smithi018.stdout:4/811: dwrite d3/d5d/d60/dc5/d3d/d80/feb [0,4194304] 0 2024-04-12T20:56:22.317 INFO:tasks.workunit.client.0.smithi018.stdout:1/708: rmdir d3/d9/da/dd/d31/d32/d33/d47 39 2024-04-12T20:56:22.320 INFO:tasks.workunit.client.1.smithi186.stdout:0/52: rmdir d0 39 2024-04-12T20:56:22.323 INFO:tasks.workunit.client.0.smithi018.stdout:4/812: dread - d3/db/fd0 zero size 2024-04-12T20:56:22.325 INFO:tasks.workunit.client.1.smithi186.stdout:4/56: creat d0/d12/f13 x:0 0 0 2024-04-12T20:56:22.326 INFO:tasks.workunit.client.1.smithi186.stdout:7/52: unlink c5 0 2024-04-12T20:56:22.327 INFO:tasks.workunit.client.0.smithi018.stdout:7/867: truncate d1/db/d31/d3a/d42/d4c/f5b 1762314 0 2024-04-12T20:56:22.327 INFO:tasks.workunit.client.1.smithi186.stdout:7/53: chown d7/fe 73 1 2024-04-12T20:56:22.329 INFO:tasks.workunit.client.1.smithi186.stdout:0/53: dread - d0/fe zero size 2024-04-12T20:56:22.337 INFO:tasks.workunit.client.0.smithi018.stdout:2/800: dwrite d5/d8/d10/d43/de2/daf/d90/f9d [0,4194304] 0 2024-04-12T20:56:22.337 INFO:tasks.workunit.client.1.smithi186.stdout:4/57: mkdir d0/dc/d14 0 2024-04-12T20:56:22.337 INFO:tasks.workunit.client.1.smithi186.stdout:0/54: rmdir d0 39 2024-04-12T20:56:22.340 INFO:tasks.workunit.client.0.smithi018.stdout:5/947: dwrite d0/d2/d33/d10/d60/f73 [0,4194304] 0 2024-04-12T20:56:22.340 INFO:tasks.workunit.client.0.smithi018.stdout:0/854: dwrite da/d15/fe2 [4194304,4194304] 0 2024-04-12T20:56:22.340 INFO:tasks.workunit.client.0.smithi018.stdout:7/868: mkdir d1/db/d31/d45/d53/d11b 0 2024-04-12T20:56:22.341 INFO:tasks.workunit.client.0.smithi018.stdout:1/709: getdents d3/d9/d11 0 2024-04-12T20:56:22.346 INFO:tasks.workunit.client.0.smithi018.stdout:2/801: write d5/d7/db6/f114 [431760,16765] 0 2024-04-12T20:56:22.346 INFO:tasks.workunit.client.0.smithi018.stdout:4/813: write d3/db/f7f [1108409,126834] 0 2024-04-12T20:56:22.349 INFO:tasks.workunit.client.1.smithi186.stdout:4/58: dread d0/d12/f7 [0,4194304] 0 2024-04-12T20:56:22.349 INFO:tasks.workunit.client.1.smithi186.stdout:1/51: truncate fb 3932513 0 2024-04-12T20:56:22.351 INFO:tasks.workunit.client.0.smithi018.stdout:7/869: chown d1/db/d31/d3a/d42/cc7 2 1 2024-04-12T20:56:22.352 INFO:tasks.workunit.client.1.smithi186.stdout:4/59: dread - d0/fd zero size 2024-04-12T20:56:22.352 INFO:tasks.workunit.client.1.smithi186.stdout:4/60: read - d0/d12/fb zero size 2024-04-12T20:56:22.357 INFO:tasks.workunit.client.0.smithi018.stdout:0/855: mkdir da/d15/d17/d71/dc1/dee/d11d 0 2024-04-12T20:56:22.361 INFO:tasks.workunit.client.0.smithi018.stdout:4/814: chown d3/d5d/d60/dc5/d43/d4b/de6/cde 0 1 2024-04-12T20:56:22.367 INFO:tasks.workunit.client.1.smithi186.stdout:1/52: dread fe [0,4194304] 0 2024-04-12T20:56:22.369 INFO:tasks.workunit.client.1.smithi186.stdout:1/53: truncate ff 744731 0 2024-04-12T20:56:22.371 INFO:tasks.workunit.client.1.smithi186.stdout:0/55: dwrite d0/f6 [4194304,4194304] 0 2024-04-12T20:56:22.375 INFO:tasks.workunit.client.0.smithi018.stdout:4/815: truncate d3/d32/d33/db6/ffd 492610 0 2024-04-12T20:56:22.386 INFO:tasks.workunit.client.1.smithi186.stdout:7/54: truncate d7/f9 2154454 0 2024-04-12T20:56:22.387 INFO:tasks.workunit.client.0.smithi018.stdout:5/948: dwrite d0/d2/d33/d10/d60/f73 [4194304,4194304] 0 2024-04-12T20:56:22.392 INFO:tasks.workunit.client.0.smithi018.stdout:1/710: dwrite d3/d9/da/dd/d31/d32/d13/d19/d2d/d4e/d7a/d82/fed [0,4194304] 0 2024-04-12T20:56:22.392 INFO:tasks.workunit.client.0.smithi018.stdout:2/802: dwrite d5/d8/d10/d43/de2/daf/d90/f101 [0,4194304] 0 2024-04-12T20:56:22.395 INFO:tasks.workunit.client.1.smithi186.stdout:0/56: dwrite d0/f6 [4194304,4194304] 0 2024-04-12T20:56:22.402 INFO:tasks.workunit.client.1.smithi186.stdout:7/55: creat d7/f16 x:0 0 0 2024-04-12T20:56:22.405 INFO:tasks.workunit.client.1.smithi186.stdout:0/57: creat d0/ff x:0 0 0 2024-04-12T20:56:22.414 INFO:tasks.workunit.client.0.smithi018.stdout:7/870: dread d1/db/d31/d45/d8c/d9d/d69/f7e [0,4194304] 0 2024-04-12T20:56:22.415 INFO:tasks.workunit.client.1.smithi186.stdout:1/54: dwrite fe [0,4194304] 0 2024-04-12T20:56:22.419 INFO:tasks.workunit.client.1.smithi186.stdout:0/58: dread d0/fd [0,4194304] 0 2024-04-12T20:56:22.423 INFO:tasks.workunit.client.1.smithi186.stdout:0/59: chown d0 707 1 2024-04-12T20:56:22.425 INFO:tasks.workunit.client.1.smithi186.stdout:7/56: link d7/f16 d7/f17 0 2024-04-12T20:56:22.427 INFO:tasks.workunit.client.1.smithi186.stdout:0/60: mkdir d0/d10 0 2024-04-12T20:56:22.429 INFO:tasks.workunit.client.1.smithi186.stdout:4/61: dwrite d0/f10 [0,4194304] 0 2024-04-12T20:56:22.430 INFO:tasks.workunit.client.1.smithi186.stdout:7/57: mknod d7/d10/c18 0 2024-04-12T20:56:22.433 INFO:tasks.workunit.client.1.smithi186.stdout:0/61: write d0/ff [261156,40831] 0 2024-04-12T20:56:22.435 INFO:tasks.workunit.client.1.smithi186.stdout:7/58: write d7/fc [364064,128093] 0 2024-04-12T20:56:22.436 INFO:tasks.workunit.client.1.smithi186.stdout:0/62: read d0/f6 [3804327,119912] 0 2024-04-12T20:56:22.436 INFO:tasks.workunit.client.1.smithi186.stdout:7/59: dread - d7/f14 zero size 2024-04-12T20:56:22.441 INFO:tasks.workunit.client.1.smithi186.stdout:4/62: creat d0/dc/d14/f15 x:0 0 0 2024-04-12T20:56:22.443 INFO:tasks.workunit.client.1.smithi186.stdout:0/63: rename d0 to d0/d11 22 2024-04-12T20:56:22.443 INFO:tasks.workunit.client.1.smithi186.stdout:4/63: stat d0/dc/fe 0 2024-04-12T20:56:22.443 INFO:tasks.workunit.client.1.smithi186.stdout:7/60: creat d7/d10/f19 x:0 0 0 2024-04-12T20:56:22.444 INFO:tasks.workunit.client.1.smithi186.stdout:7/61: stat d7/f8 0 2024-04-12T20:56:22.451 INFO:tasks.workunit.client.1.smithi186.stdout:0/64: rename d0/ff to d0/d10/f12 0 2024-04-12T20:56:22.452 INFO:tasks.workunit.client.1.smithi186.stdout:7/62: mknod d7/d10/c1a 0 2024-04-12T20:56:22.454 INFO:tasks.workunit.client.1.smithi186.stdout:4/64: mknod d0/dc/d14/c16 0 2024-04-12T20:56:22.456 INFO:tasks.workunit.client.0.smithi018.stdout:0/856: creat da/d15/d17/d21/d36/da2/dc3/d3f/de3/f11e x:0 0 0 2024-04-12T20:56:22.456 INFO:tasks.workunit.client.1.smithi186.stdout:7/63: write d7/d10/f15 [1063152,83030] 0 2024-04-12T20:56:22.459 INFO:tasks.workunit.client.1.smithi186.stdout:7/64: write d7/ff [4911664,32765] 0 2024-04-12T20:56:22.462 INFO:tasks.workunit.client.1.smithi186.stdout:0/65: truncate d0/fd 596620 0 2024-04-12T20:56:22.464 INFO:tasks.workunit.client.1.smithi186.stdout:0/66: dread - d0/fe zero size 2024-04-12T20:56:22.465 INFO:tasks.workunit.client.1.smithi186.stdout:0/67: stat d0/la 0 2024-04-12T20:56:22.469 INFO:tasks.workunit.client.1.smithi186.stdout:0/68: fdatasync d0/f6 0 2024-04-12T20:56:22.472 INFO:tasks.workunit.client.1.smithi186.stdout:0/69: mkdir d0/d13 0 2024-04-12T20:56:22.475 INFO:tasks.workunit.client.1.smithi186.stdout:4/65: dread d0/d12/f7 [0,4194304] 0 2024-04-12T20:56:22.475 INFO:tasks.workunit.client.1.smithi186.stdout:0/70: truncate d0/d10/f12 681398 0 2024-04-12T20:56:22.478 INFO:tasks.workunit.client.0.smithi018.stdout:4/816: link d3/d5d/d60/ff0 d3/d5d/d60/dc5/d3d/de4/f113 0 2024-04-12T20:56:22.479 INFO:tasks.workunit.client.1.smithi186.stdout:4/66: creat d0/d12/f17 x:0 0 0 2024-04-12T20:56:22.482 INFO:tasks.workunit.client.1.smithi186.stdout:0/71: rename d0/d13 to d0/d14 0 2024-04-12T20:56:22.485 INFO:tasks.workunit.client.1.smithi186.stdout:0/72: truncate d0/d10/f12 1170184 0 2024-04-12T20:56:22.487 INFO:tasks.workunit.client.1.smithi186.stdout:4/67: rename d0/f11 to d0/f18 0 2024-04-12T20:56:22.488 INFO:tasks.workunit.client.1.smithi186.stdout:7/65: fsync d7/d10/f19 0 2024-04-12T20:56:22.489 INFO:tasks.workunit.client.1.smithi186.stdout:7/66: fsync f4 0 2024-04-12T20:56:22.490 INFO:tasks.workunit.client.1.smithi186.stdout:7/67: dread - d7/f12 zero size 2024-04-12T20:56:22.492 INFO:tasks.workunit.client.1.smithi186.stdout:4/68: dread d0/dc/fe [0,4194304] 0 2024-04-12T20:56:22.499 INFO:tasks.workunit.client.1.smithi186.stdout:1/55: link fb d14/f15 0 2024-04-12T20:56:22.501 INFO:tasks.workunit.client.0.smithi018.stdout:1/711: mkdir d3/d9/da/dd/d31/d32/d13/d19/d2d/d4e/d58/dee 0 2024-04-12T20:56:22.512 INFO:tasks.workunit.client.1.smithi186.stdout:7/68: creat d7/f1b x:0 0 0 2024-04-12T20:56:22.514 INFO:tasks.workunit.client.1.smithi186.stdout:1/56: dread ff [0,4194304] 0 2024-04-12T20:56:22.515 INFO:tasks.workunit.client.1.smithi186.stdout:1/57: truncate ff 1250286 0 2024-04-12T20:56:22.517 INFO:tasks.workunit.client.1.smithi186.stdout:4/69: dwrite d0/dc/d14/f15 [0,4194304] 0 2024-04-12T20:56:22.522 INFO:tasks.workunit.client.0.smithi018.stdout:2/803: link d5/d7/d63/l85 d5/d7/d31/d62/l116 0 2024-04-12T20:56:22.523 INFO:tasks.workunit.client.0.smithi018.stdout:2/804: readlink d5/d12/l1c 0 2024-04-12T20:56:22.532 INFO:tasks.workunit.client.0.smithi018.stdout:7/871: rmdir d1/d6/d10a 0 2024-04-12T20:56:22.537 INFO:tasks.workunit.client.0.smithi018.stdout:7/872: read d1/f3e [8252,58815] 0 2024-04-12T20:56:22.538 INFO:tasks.workunit.client.0.smithi018.stdout:2/805: dread d5/d12/d54/f10d [0,4194304] 0 2024-04-12T20:56:22.542 INFO:tasks.workunit.client.0.smithi018.stdout:2/806: fsync d5/d7/d31/d62/f110 0 2024-04-12T20:56:22.544 INFO:tasks.workunit.client.0.smithi018.stdout:0/857: write da/d15/d17/f43 [1432130,70067] 0 2024-04-12T20:56:22.546 INFO:tasks.workunit.client.0.smithi018.stdout:2/807: chown d5/d8/d10/d43/de2/daf/d88/fb2 0 1 2024-04-12T20:56:22.553 INFO:tasks.workunit.client.0.smithi018.stdout:0/858: symlink da/d15/d4d/d58/l11f 0 2024-04-12T20:56:22.555 INFO:tasks.workunit.client.0.smithi018.stdout:2/808: creat d5/d8/d10/d43/d8f/de9/f117 x:0 0 0 2024-04-12T20:56:22.558 INFO:tasks.workunit.client.0.smithi018.stdout:2/809: fsync d5/d7/d31/d62/fc2 0 2024-04-12T20:56:22.558 INFO:tasks.workunit.client.0.smithi018.stdout:0/859: write da/d15/d17/d21/d36/da2/dc3/d83/f11c [513340,30662] 0 2024-04-12T20:56:22.564 INFO:tasks.workunit.client.0.smithi018.stdout:0/860: read - da/d15/d17/d21/d36/da2/dc3/f76 zero size 2024-04-12T20:56:22.567 INFO:tasks.workunit.client.0.smithi018.stdout:4/817: dwrite d3/d5d/d60/dc5/d40/f82 [0,4194304] 0 2024-04-12T20:56:22.571 INFO:tasks.workunit.client.0.smithi018.stdout:0/861: mknod da/d15/c120 0 2024-04-12T20:56:22.572 INFO:tasks.workunit.client.0.smithi018.stdout:1/712: truncate d3/d9/da/d14/d73/fd0 1506228 0 2024-04-12T20:56:22.573 INFO:tasks.workunit.client.0.smithi018.stdout:7/873: rmdir d1/db/d31/d45/d8c/d9d/d70 39 2024-04-12T20:56:22.574 INFO:tasks.workunit.client.0.smithi018.stdout:4/818: readlink d3/d5d/d60/dc5/d43/d4b/l87 0 2024-04-12T20:56:22.581 INFO:tasks.workunit.client.0.smithi018.stdout:1/713: truncate d3/d9/da/dd/d31/d44/fe1 160513 0 2024-04-12T20:56:22.584 INFO:tasks.workunit.client.0.smithi018.stdout:0/862: creat da/d15/d17/d71/f121 x:0 0 0 2024-04-12T20:56:22.584 INFO:tasks.workunit.client.0.smithi018.stdout:2/810: dwrite d5/d8/d25/fba [0,4194304] 0 2024-04-12T20:56:22.588 INFO:tasks.workunit.client.0.smithi018.stdout:7/874: unlink d1/db/d31/d45/d8c/d9d/d70/d96/ff3 0 2024-04-12T20:56:22.589 INFO:tasks.workunit.client.0.smithi018.stdout:2/811: fsync d5/d8/d10/d43/de2/f7b 0 2024-04-12T20:56:22.593 INFO:tasks.workunit.client.0.smithi018.stdout:4/819: mkdir d3/db/d114 0 2024-04-12T20:56:22.595 INFO:tasks.workunit.client.0.smithi018.stdout:7/875: readlink d1/db/d31/d3a/d42/d4c/d6c/dc3/l93 0 2024-04-12T20:56:22.596 INFO:tasks.workunit.client.0.smithi018.stdout:4/820: fdatasync d3/d5d/f112 0 2024-04-12T20:56:22.602 INFO:tasks.workunit.client.0.smithi018.stdout:1/714: symlink d3/db8/lef 0 2024-04-12T20:56:22.606 INFO:tasks.workunit.client.0.smithi018.stdout:5/949: link d0/d2/d11/f123 d0/d2/f145 0 2024-04-12T20:56:22.608 INFO:tasks.workunit.client.0.smithi018.stdout:7/876: symlink d1/db/d31/d45/d8c/d9d/d94/l11c 0 2024-04-12T20:56:22.609 INFO:tasks.workunit.client.0.smithi018.stdout:5/950: chown d0/d2/d33/d46/d1e/d55/l10f 0 1 2024-04-12T20:56:22.613 INFO:tasks.workunit.client.0.smithi018.stdout:7/877: readlink d1/db/d31/d45/d8c/d9d/d70/lf1 0 2024-04-12T20:56:22.619 INFO:tasks.workunit.client.0.smithi018.stdout:4/821: dread d3/d5d/d60/dc5/d40/f82 [0,4194304] 0 2024-04-12T20:56:22.619 INFO:tasks.workunit.client.0.smithi018.stdout:5/951: creat d0/d2/d33/d46/d35/dd2/f146 x:0 0 0 2024-04-12T20:56:22.623 INFO:tasks.workunit.client.0.smithi018.stdout:5/952: chown d0/d2/d33/d10/d60/dfa/dbf/da3 1634101 1 2024-04-12T20:56:22.631 INFO:tasks.workunit.client.0.smithi018.stdout:0/863: getdents da/d15/d17/d21/d36 0 2024-04-12T20:56:22.638 INFO:tasks.workunit.client.0.smithi018.stdout:4/822: creat d3/d5d/d60/dc5/de8/f115 x:0 0 0 2024-04-12T20:56:22.639 INFO:tasks.workunit.client.1.smithi186.stdout:0/73: dread d0/fd [0,4194304] 0 2024-04-12T20:56:22.640 INFO:tasks.workunit.client.1.smithi186.stdout:7/69: mkdir d7/d10/d1c 0 2024-04-12T20:56:22.641 INFO:tasks.workunit.client.0.smithi018.stdout:1/715: dwrite d3/d9/da/dd/d31/f59 [0,4194304] 0 2024-04-12T20:56:22.644 INFO:tasks.workunit.client.1.smithi186.stdout:4/70: rmdir d0 39 2024-04-12T20:56:22.646 INFO:tasks.workunit.client.1.smithi186.stdout:0/74: write d0/d10/f12 [913744,42710] 0 2024-04-12T20:56:22.647 INFO:tasks.workunit.client.0.smithi018.stdout:2/812: truncate d5/d8/d10/d43/de2/daf/d88/fb2 178250 0 2024-04-12T20:56:22.647 INFO:tasks.workunit.client.0.smithi018.stdout:7/878: write d1/db/d31/d3a/d42/d4c/f5b [1125931,79005] 0 2024-04-12T20:56:22.650 INFO:tasks.workunit.client.0.smithi018.stdout:2/813: stat d5/d7/d79/f7e 0 2024-04-12T20:56:22.657 INFO:tasks.workunit.client.1.smithi186.stdout:7/70: rename d7/d10/c18 to d7/c1d 0 2024-04-12T20:56:22.658 INFO:tasks.workunit.client.1.smithi186.stdout:4/71: mknod d0/d12/c19 0 2024-04-12T20:56:22.658 INFO:tasks.workunit.client.1.smithi186.stdout:0/75: unlink d0/l5 0 2024-04-12T20:56:22.658 INFO:tasks.workunit.client.0.smithi018.stdout:1/716: rename d3/d9/da/dd/d31/d32/d13/d19/d2d/d48 to d3/d9/da/dd/d26/df0 0 2024-04-12T20:56:22.661 INFO:tasks.workunit.client.1.smithi186.stdout:1/58: dread fb [0,4194304] 0 2024-04-12T20:56:22.661 INFO:tasks.workunit.client.1.smithi186.stdout:7/71: truncate d7/f12 360958 0 2024-04-12T20:56:22.664 INFO:tasks.workunit.client.1.smithi186.stdout:0/76: creat d0/d10/f15 x:0 0 0 2024-04-12T20:56:22.665 INFO:tasks.workunit.client.0.smithi018.stdout:0/864: link da/d15/d17/d21/d36/c37 da/d15/d17/d21/d36/d88/c122 0 2024-04-12T20:56:22.671 INFO:tasks.workunit.client.0.smithi018.stdout:7/879: creat d1/db/d31/d45/d8c/d9d/d70/d96/d5f/da9/ddf/d108/f11d x:0 0 0 2024-04-12T20:56:22.671 INFO:tasks.workunit.client.0.smithi018.stdout:0/865: chown da/c9d 75 1 2024-04-12T20:56:22.671 INFO:tasks.workunit.client.1.smithi186.stdout:7/72: dread d7/d10/f15 [0,4194304] 0 2024-04-12T20:56:22.677 INFO:tasks.workunit.client.1.smithi186.stdout:0/77: mkdir d0/d14/d16 0 2024-04-12T20:56:22.683 INFO:tasks.workunit.client.1.smithi186.stdout:4/72: dwrite d0/dc/d14/f15 [0,4194304] 0 2024-04-12T20:56:22.685 INFO:tasks.workunit.client.0.smithi018.stdout:5/953: dwrite d0/d2/d33/d46/d35/d4d/f80 [0,4194304] 0 2024-04-12T20:56:22.685 INFO:tasks.workunit.client.1.smithi186.stdout:4/73: fsync d0/f10 0 2024-04-12T20:56:22.686 INFO:tasks.workunit.client.1.smithi186.stdout:0/78: creat d0/f17 x:0 0 0 2024-04-12T20:56:22.686 INFO:tasks.workunit.client.1.smithi186.stdout:1/59: dwrite f9 [0,4194304] 0 2024-04-12T20:56:22.687 INFO:tasks.workunit.client.0.smithi018.stdout:5/954: chown d0/d2/d33/d10/d26/dda 41002696 1 2024-04-12T20:56:22.688 INFO:tasks.workunit.client.1.smithi186.stdout:4/74: chown d0/dc/fe 703 1 2024-04-12T20:56:22.691 INFO:tasks.workunit.client.1.smithi186.stdout:0/79: dread d0/fd [0,4194304] 0 2024-04-12T20:56:22.691 INFO:tasks.workunit.client.1.smithi186.stdout:4/75: dread - d0/d12/fb zero size 2024-04-12T20:56:22.692 INFO:tasks.workunit.client.1.smithi186.stdout:7/73: getdents d7/d10/d1c 0 2024-04-12T20:56:22.693 INFO:tasks.workunit.client.1.smithi186.stdout:4/76: readlink d0/l1 0 2024-04-12T20:56:22.693 INFO:tasks.workunit.client.1.smithi186.stdout:0/80: chown d0/cc 14483 1 2024-04-12T20:56:22.696 INFO:tasks.workunit.client.1.smithi186.stdout:0/81: write d0/d10/f15 [932614,80377] 0 2024-04-12T20:56:22.697 INFO:tasks.workunit.client.0.smithi018.stdout:4/823: getdents d3/d5d/d60/d64/d72 0 2024-04-12T20:56:22.698 INFO:tasks.workunit.client.1.smithi186.stdout:1/60: creat d14/f16 x:0 0 0 2024-04-12T20:56:22.701 INFO:tasks.workunit.client.1.smithi186.stdout:4/77: write d0/f18 [846308,116087] 0 2024-04-12T20:56:22.704 INFO:tasks.workunit.client.1.smithi186.stdout:7/74: rename d7/fb to d7/d10/d1c/f1e 0 2024-04-12T20:56:22.704 INFO:tasks.workunit.client.0.smithi018.stdout:7/880: mkdir d1/db/d31/d30/dc6/d100/d11e 0 2024-04-12T20:56:22.704 INFO:tasks.workunit.client.0.smithi018.stdout:5/955: mknod d0/d2/d33/d10/d60/c147 0 2024-04-12T20:56:22.706 INFO:tasks.workunit.client.1.smithi186.stdout:7/75: write f4 [2055622,100178] 0 2024-04-12T20:56:22.706 INFO:tasks.workunit.client.1.smithi186.stdout:0/82: mknod d0/d14/c18 0 2024-04-12T20:56:22.707 INFO:tasks.workunit.client.0.smithi018.stdout:2/814: link d5/d8/d10/d43/de2/c2e d5/d8/d10/d43/de2/daf/c118 0 2024-04-12T20:56:22.707 INFO:tasks.workunit.client.0.smithi018.stdout:5/956: read - d0/d2/d11/f123 zero size 2024-04-12T20:56:22.709 INFO:tasks.workunit.client.1.smithi186.stdout:0/83: fdatasync d0/f17 0 2024-04-12T20:56:22.709 INFO:tasks.workunit.client.1.smithi186.stdout:7/76: chown d7/ff 477 1 2024-04-12T20:56:22.710 INFO:tasks.workunit.client.1.smithi186.stdout:7/77: dread - d7/f16 zero size 2024-04-12T20:56:22.711 INFO:tasks.workunit.client.0.smithi018.stdout:0/866: write da/d15/d17/d71/dc1/dcc/f4f [85535,37176] 0 2024-04-12T20:56:22.712 INFO:tasks.workunit.client.0.smithi018.stdout:4/824: truncate d3/d5d/d60/dc5/d3d/f57 578926 0 2024-04-12T20:56:22.717 INFO:tasks.workunit.client.1.smithi186.stdout:0/84: read d0/d10/f15 [557172,30260] 0 2024-04-12T20:56:22.717 INFO:tasks.workunit.client.0.smithi018.stdout:4/825: truncate d3/d5d/d60/dc5/de8/f115 695468 0 2024-04-12T20:56:22.719 INFO:tasks.workunit.client.1.smithi186.stdout:0/85: stat d0/f17 0 2024-04-12T20:56:22.719 INFO:tasks.workunit.client.1.smithi186.stdout:6/4: sync 2024-04-12T20:56:22.721 INFO:tasks.workunit.client.1.smithi186.stdout:7/78: rename d7/ff to d7/d10/d1c/f1f 0 2024-04-12T20:56:22.724 INFO:tasks.workunit.client.0.smithi018.stdout:0/867: chown da/d15/d17/d21/d36/da2/la6 8937 1 2024-04-12T20:56:22.725 INFO:tasks.workunit.client.1.smithi186.stdout:1/61: dwrite d14/f16 [0,4194304] 0 2024-04-12T20:56:22.726 INFO:tasks.workunit.client.0.smithi018.stdout:0/868: read da/d15/d17/d21/f46 [2164046,20762] 0 2024-04-12T20:56:22.731 INFO:tasks.workunit.client.1.smithi186.stdout:0/86: creat d0/d14/f19 x:0 0 0 2024-04-12T20:56:22.737 INFO:tasks.workunit.client.1.smithi186.stdout:6/5: mknod c0 0 2024-04-12T20:56:22.737 INFO:tasks.workunit.client.1.smithi186.stdout:6/6: fsync - no filename 2024-04-12T20:56:22.738 INFO:tasks.workunit.client.1.smithi186.stdout:0/87: creat d0/d14/f1a x:0 0 0 2024-04-12T20:56:22.739 INFO:tasks.workunit.client.0.smithi018.stdout:5/957: symlink d0/d2b/l148 0 2024-04-12T20:56:22.741 INFO:tasks.workunit.client.0.smithi018.stdout:5/958: readlink d0/d2/d11/d37/l67 0 2024-04-12T20:56:22.743 INFO:tasks.workunit.client.1.smithi186.stdout:6/7: rename c0 to c1 0 2024-04-12T20:56:22.743 INFO:tasks.workunit.client.1.smithi186.stdout:6/8: write - no filename 2024-04-12T20:56:22.743 INFO:tasks.workunit.client.1.smithi186.stdout:6/9: dwrite - no filename 2024-04-12T20:56:22.747 INFO:tasks.workunit.client.1.smithi186.stdout:6/10: mknod c2 0 2024-04-12T20:56:22.747 INFO:tasks.workunit.client.0.smithi018.stdout:0/869: creat da/d15/d17/d21/d36/da2/dc3/da9/f123 x:0 0 0 2024-04-12T20:56:22.750 INFO:tasks.workunit.client.0.smithi018.stdout:2/815: rename d5/d8/d25/f7d to d5/d8/d10/d43/de2/f119 0 2024-04-12T20:56:22.751 INFO:tasks.workunit.client.1.smithi186.stdout:6/11: rename c2 to c3 0 2024-04-12T20:56:22.751 INFO:tasks.workunit.client.1.smithi186.stdout:6/12: read - no filename 2024-04-12T20:56:22.752 INFO:tasks.workunit.client.1.smithi186.stdout:0/88: link d0/c3 d0/d10/c1b 0 2024-04-12T20:56:22.756 INFO:tasks.workunit.client.0.smithi018.stdout:5/959: unlink d0/d2/d33/d10/d60/dfa/dbf/da3/dae/c103 0 2024-04-12T20:56:22.756 INFO:tasks.workunit.client.1.smithi186.stdout:0/89: mkdir d0/d10/d1c 0 2024-04-12T20:56:22.758 INFO:tasks.workunit.client.0.smithi018.stdout:0/870: mknod da/d15/d17/d5c/d79/dd6/c124 0 2024-04-12T20:56:22.758 INFO:tasks.workunit.client.0.smithi018.stdout:7/881: dread d1/db/d31/d45/d8c/d9d/d70/d96/f4 [0,4194304] 0 2024-04-12T20:56:22.759 INFO:tasks.workunit.client.0.smithi018.stdout:2/816: mkdir d5/d7/d31/d11a 0 2024-04-12T20:56:22.760 INFO:tasks.workunit.client.1.smithi186.stdout:0/90: creat d0/f1d x:0 0 0 2024-04-12T20:56:22.762 INFO:tasks.workunit.client.0.smithi018.stdout:4/826: dwrite d3/d5d/d60/dc5/fb0 [4194304,4194304] 0 2024-04-12T20:56:22.763 INFO:tasks.workunit.client.0.smithi018.stdout:7/882: write d1/d6/f102 [369290,129479] 0 2024-04-12T20:56:22.763 INFO:tasks.workunit.client.0.smithi018.stdout:5/960: symlink d0/d2/d33/d10/d60/l149 0 2024-04-12T20:56:22.768 INFO:tasks.workunit.client.0.smithi018.stdout:0/871: creat da/d15/d17/d71/dc1/dee/df7/dea/f125 x:0 0 0 2024-04-12T20:56:22.771 INFO:tasks.workunit.client.0.smithi018.stdout:4/827: creat d3/d5d/d60/dc5/d40/f116 x:0 0 0 2024-04-12T20:56:22.771 INFO:tasks.workunit.client.1.smithi186.stdout:0/91: rename d0/d14/d16 to d0/d10/d1c/d1e 0 2024-04-12T20:56:22.772 INFO:tasks.workunit.client.1.smithi186.stdout:0/92: stat d0/c9 0 2024-04-12T20:56:22.774 INFO:tasks.workunit.client.1.smithi186.stdout:4/78: getdents d0/d12 0 2024-04-12T20:56:22.779 INFO:tasks.workunit.client.1.smithi186.stdout:4/79: dread d0/d12/f4 [0,4194304] 0 2024-04-12T20:56:22.780 INFO:tasks.workunit.client.1.smithi186.stdout:4/80: chown d0 2880 1 2024-04-12T20:56:22.781 INFO:tasks.workunit.client.0.smithi018.stdout:0/872: rmdir da/d15/d17/d21/d36/d47/db5 39 2024-04-12T20:56:22.787 INFO:tasks.workunit.client.0.smithi018.stdout:5/961: rename d0/d2/d11/d37/f47 to d0/d2/d33/d10/d60/f14a 0 2024-04-12T20:56:22.790 INFO:tasks.workunit.client.0.smithi018.stdout:5/962: chown d0/d2/d33/d46/d18/f7a 29491076 1 2024-04-12T20:56:22.792 INFO:tasks.workunit.client.0.smithi018.stdout:0/873: unlink da/d15/d17/d21/d36/d47/d81/lfd 0 2024-04-12T20:56:22.793 INFO:tasks.workunit.client.0.smithi018.stdout:7/883: write d1/db/d31/d45/d8c/d9d/d69/f7f [404008,63370] 0 2024-04-12T20:56:22.793 INFO:tasks.workunit.client.1.smithi186.stdout:6/13: getdents . 0 2024-04-12T20:56:22.793 INFO:tasks.workunit.client.1.smithi186.stdout:6/14: dread - no filename 2024-04-12T20:56:22.795 INFO:tasks.workunit.client.0.smithi018.stdout:4/828: write d3/d5d/d60/dc5/d40/f6d [758330,31078] 0 2024-04-12T20:56:22.797 INFO:tasks.workunit.client.1.smithi186.stdout:9/28: sync 2024-04-12T20:56:22.798 INFO:tasks.workunit.client.1.smithi186.stdout:8/18: sync 2024-04-12T20:56:22.798 INFO:tasks.workunit.client.1.smithi186.stdout:5/12: sync 2024-04-12T20:56:22.798 INFO:tasks.workunit.client.1.smithi186.stdout:2/18: sync 2024-04-12T20:56:22.799 INFO:tasks.workunit.client.1.smithi186.stdout:3/64: sync 2024-04-12T20:56:22.802 INFO:tasks.workunit.client.0.smithi018.stdout:4/829: fdatasync d3/d5d/f112 0 2024-04-12T20:56:22.802 INFO:tasks.workunit.client.0.smithi018.stdout:5/963: link d0/d5b/f5d d0/d2/d33/d46/d35/d4d/db3/dc3/f14b 0 2024-04-12T20:56:22.804 INFO:tasks.workunit.client.1.smithi186.stdout:7/79: dwrite d7/f17 [0,4194304] 0 2024-04-12T20:56:22.805 INFO:tasks.workunit.client.1.smithi186.stdout:4/81: write d0/d12/f7 [4627536,13255] 0 2024-04-12T20:56:22.805 INFO:tasks.workunit.client.1.smithi186.stdout:8/19: symlink l4 0 2024-04-12T20:56:22.805 INFO:tasks.workunit.client.1.smithi186.stdout:1/62: dwrite fb [0,4194304] 0 2024-04-12T20:56:22.807 INFO:tasks.workunit.client.1.smithi186.stdout:3/65: read d1/fc [385375,61112] 0 2024-04-12T20:56:22.810 INFO:tasks.workunit.client.1.smithi186.stdout:9/29: write f1 [5242866,44909] 0 2024-04-12T20:56:22.810 INFO:tasks.workunit.client.1.smithi186.stdout:6/15: unlink c3 0 2024-04-12T20:56:22.810 INFO:tasks.workunit.client.1.smithi186.stdout:6/16: dwrite - no filename 2024-04-12T20:56:22.811 INFO:tasks.workunit.client.0.smithi018.stdout:5/964: rename d0/d2/de4/feb to d0/d2/d33/d46/d1e/d113/f14c 0 2024-04-12T20:56:22.812 INFO:tasks.workunit.client.1.smithi186.stdout:5/13: write f2 [861209,100000] 0 2024-04-12T20:56:22.813 INFO:tasks.workunit.client.1.smithi186.stdout:5/14: stat f4 0 2024-04-12T20:56:22.815 INFO:tasks.workunit.client.1.smithi186.stdout:9/30: chown f1 2650 1 2024-04-12T20:56:22.815 INFO:tasks.workunit.client.0.smithi018.stdout:5/965: rmdir d0/d2/d33/d46/d9e 39 2024-04-12T20:56:22.817 INFO:tasks.workunit.client.1.smithi186.stdout:1/63: mknod d14/c17 0 2024-04-12T20:56:22.818 INFO:tasks.workunit.client.1.smithi186.stdout:1/64: read f4 [1975610,60335] 0 2024-04-12T20:56:22.819 INFO:tasks.workunit.client.1.smithi186.stdout:3/66: stat d1/d3/da/l9 0 2024-04-12T20:56:22.820 INFO:tasks.workunit.client.1.smithi186.stdout:0/93: dwrite d0/fd [0,4194304] 0 2024-04-12T20:56:22.821 INFO:tasks.workunit.client.1.smithi186.stdout:1/65: chown fb 44 1 2024-04-12T20:56:22.824 INFO:tasks.workunit.client.1.smithi186.stdout:3/67: dread d1/fc [0,4194304] 0 2024-04-12T20:56:22.825 INFO:tasks.workunit.client.0.smithi018.stdout:5/966: mknod d0/d2/d11/d37/c14d 0 2024-04-12T20:56:22.828 INFO:tasks.workunit.client.0.smithi018.stdout:7/884: dwrite d1/db/fef [0,4194304] 0 2024-04-12T20:56:22.830 INFO:tasks.workunit.client.1.smithi186.stdout:2/19: dread d0/f2 [0,4194304] 0 2024-04-12T20:56:22.835 INFO:tasks.workunit.client.1.smithi186.stdout:1/66: dread fe [0,4194304] 0 2024-04-12T20:56:22.838 INFO:tasks.workunit.client.0.smithi018.stdout:4/830: dwrite d3/d32/d33/f30 [0,4194304] 0 2024-04-12T20:56:22.838 INFO:tasks.workunit.client.0.smithi018.stdout:0/874: dwrite da/d15/d17/d21/d36/f4c [0,4194304] 0 2024-04-12T20:56:22.843 INFO:tasks.workunit.client.0.smithi018.stdout:7/885: fdatasync d1/db/d31/d3a/d42/f113 0 2024-04-12T20:56:22.858 INFO:tasks.workunit.client.1.smithi186.stdout:8/20: link f2 f5 0 2024-04-12T20:56:22.859 INFO:tasks.workunit.client.1.smithi186.stdout:1/67: dwrite d14/f16 [0,4194304] 0 2024-04-12T20:56:22.869 INFO:tasks.workunit.client.0.smithi018.stdout:0/875: truncate da/d15/d4d/d58/fc5 4413648 0 2024-04-12T20:56:22.872 INFO:tasks.workunit.client.1.smithi186.stdout:9/31: fdatasync f1 0 2024-04-12T20:56:22.874 INFO:tasks.workunit.client.1.smithi186.stdout:5/15: dread f2 [0,4194304] 0 2024-04-12T20:56:22.882 INFO:tasks.workunit.client.1.smithi186.stdout:9/32: dread f1 [0,4194304] 0 2024-04-12T20:56:22.891 INFO:tasks.workunit.client.1.smithi186.stdout:4/82: sync 2024-04-12T20:56:22.891 INFO:tasks.workunit.client.1.smithi186.stdout:3/68: symlink d1/d3/da/ld 0 2024-04-12T20:56:22.894 INFO:tasks.workunit.client.0.smithi018.stdout:7/886: dwrite d1/db/d31/d3a/d42/d4c/f51 [0,4194304] 0 2024-04-12T20:56:22.894 INFO:tasks.workunit.client.1.smithi186.stdout:3/69: read d1/fc [313340,90773] 0 2024-04-12T20:56:22.895 INFO:tasks.workunit.client.1.smithi186.stdout:4/83: chown d0/f18 154 1 2024-04-12T20:56:22.897 INFO:tasks.workunit.client.0.smithi018.stdout:4/831: dwrite d3/db/f26 [0,4194304] 0 2024-04-12T20:56:22.897 INFO:tasks.workunit.client.0.smithi018.stdout:7/887: stat d1/db/d31/d45/d8c/d9d/d70/d96/l62 0 2024-04-12T20:56:22.905 INFO:tasks.workunit.client.1.smithi186.stdout:7/80: dwrite d7/d10/f15 [0,4194304] 0 2024-04-12T20:56:22.906 INFO:tasks.workunit.client.1.smithi186.stdout:2/20: unlink d0/f1 0 2024-04-12T20:56:22.908 INFO:tasks.workunit.client.0.smithi018.stdout:0/876: mkdir da/d15/d17/d21/d36/dce/d126 0 2024-04-12T20:56:22.912 INFO:tasks.workunit.client.0.smithi018.stdout:7/888: truncate d1/db/f43 2403279 0 2024-04-12T20:56:22.914 INFO:tasks.workunit.client.0.smithi018.stdout:5/967: link d0/d2/d33/d46/f3a d0/d2/d33/d46/d1e/d2e/d7f/f14e 0 2024-04-12T20:56:22.917 INFO:tasks.workunit.client.0.smithi018.stdout:5/968: stat d0/d2/d33/cab 0 2024-04-12T20:56:22.920 INFO:tasks.workunit.client.1.smithi186.stdout:0/94: dread - d0/f1d zero size 2024-04-12T20:56:22.923 INFO:tasks.workunit.client.0.smithi018.stdout:0/877: mknod da/d15/d17/d10d/c127 0 2024-04-12T20:56:22.927 INFO:tasks.workunit.client.1.smithi186.stdout:3/70: creat d1/d3/fe x:0 0 0 2024-04-12T20:56:22.927 INFO:tasks.workunit.client.1.smithi186.stdout:4/84: rename d0/d12/f13 to d0/d12/f1a 0 2024-04-12T20:56:22.927 INFO:tasks.workunit.client.0.smithi018.stdout:5/969: creat d0/d2/d33/d10/d60/f14f x:0 0 0 2024-04-12T20:56:22.928 INFO:tasks.workunit.client.1.smithi186.stdout:4/85: dread - d0/d12/f17 zero size 2024-04-12T20:56:22.929 INFO:tasks.workunit.client.1.smithi186.stdout:4/86: readlink d0/l1 0 2024-04-12T20:56:22.930 INFO:tasks.workunit.client.0.smithi018.stdout:5/970: fsync d0/d2/de4/f110 0 2024-04-12T20:56:22.932 INFO:tasks.workunit.client.0.smithi018.stdout:5/971: stat d0/d5b/d84/f7d 0 2024-04-12T20:56:22.932 INFO:tasks.workunit.client.1.smithi186.stdout:7/81: creat d7/d10/d1c/f20 x:0 0 0 2024-04-12T20:56:22.932 INFO:tasks.workunit.client.1.smithi186.stdout:2/21: creat d0/f4 x:0 0 0 2024-04-12T20:56:22.933 INFO:tasks.workunit.client.1.smithi186.stdout:2/22: write d0/f4 [580796,96518] 0 2024-04-12T20:56:22.935 INFO:tasks.workunit.client.1.smithi186.stdout:2/23: chown d0/d3 28 1 2024-04-12T20:56:22.935 INFO:tasks.workunit.client.0.smithi018.stdout:7/889: creat d1/db/d31/d45/d8c/d9d/f11f x:0 0 0 2024-04-12T20:56:22.936 INFO:tasks.workunit.client.1.smithi186.stdout:1/68: rename l12 to d14/l18 0 2024-04-12T20:56:22.941 INFO:tasks.workunit.client.1.smithi186.stdout:6/17: getdents . 0 2024-04-12T20:56:22.942 INFO:tasks.workunit.client.0.smithi018.stdout:7/890: mknod d1/db/d31/d45/d8c/d9d/d70/d96/d5f/da9/c120 0 2024-04-12T20:56:22.943 INFO:tasks.workunit.client.1.smithi186.stdout:9/33: getdents d3 0 2024-04-12T20:56:22.946 INFO:tasks.workunit.client.0.smithi018.stdout:7/891: readlink d1/db/d31/d45/d8c/d9d/d70/d96/d5f/l9b 0 2024-04-12T20:56:22.949 INFO:tasks.workunit.client.0.smithi018.stdout:4/832: dread d3/d32/f81 [0,4194304] 0 2024-04-12T20:56:22.952 INFO:tasks.workunit.client.0.smithi018.stdout:5/972: dwrite d0/d2/de4/f110 [0,4194304] 0 2024-04-12T20:56:22.954 INFO:tasks.workunit.client.1.smithi186.stdout:2/24: mknod d0/c5 0 2024-04-12T20:56:22.954 INFO:tasks.workunit.client.1.smithi186.stdout:7/82: rmdir d7 39 2024-04-12T20:56:22.958 INFO:tasks.workunit.client.1.smithi186.stdout:8/21: dwrite f2 [0,4194304] 0 2024-04-12T20:56:22.960 INFO:tasks.workunit.client.1.smithi186.stdout:6/18: symlink l4 0 2024-04-12T20:56:22.960 INFO:tasks.workunit.client.1.smithi186.stdout:1/69: readlink l13 0 2024-04-12T20:56:22.961 INFO:tasks.workunit.client.1.smithi186.stdout:9/34: unlink d3/l6 0 2024-04-12T20:56:22.966 INFO:tasks.workunit.client.1.smithi186.stdout:5/16: getdents . 0 2024-04-12T20:56:22.966 INFO:tasks.workunit.client.1.smithi186.stdout:6/19: chown l4 53339671 1 2024-04-12T20:56:22.966 INFO:tasks.workunit.client.1.smithi186.stdout:6/20: read - no filename 2024-04-12T20:56:22.970 INFO:tasks.workunit.client.1.smithi186.stdout:0/95: link d0/d14/f1a d0/d10/d1c/d1e/f1f 0 2024-04-12T20:56:22.971 INFO:tasks.workunit.client.1.smithi186.stdout:4/87: link d0/d12/f4 d0/dc/d14/f1b 0 2024-04-12T20:56:22.974 INFO:tasks.workunit.client.1.smithi186.stdout:3/71: dwrite d1/fc [0,4194304] 0 2024-04-12T20:56:22.974 INFO:tasks.workunit.client.1.smithi186.stdout:4/88: stat d0/cf 0 2024-04-12T20:56:22.974 INFO:tasks.workunit.client.1.smithi186.stdout:2/25: link d0/f2 d0/f6 0 2024-04-12T20:56:22.977 INFO:tasks.workunit.client.1.smithi186.stdout:1/70: rename d14/l18 to d14/l19 0 2024-04-12T20:56:22.977 INFO:tasks.workunit.client.1.smithi186.stdout:5/17: dread f4 [0,4194304] 0 2024-04-12T20:56:22.980 INFO:tasks.workunit.client.1.smithi186.stdout:4/89: write d0/f10 [4822242,103523] 0 2024-04-12T20:56:22.984 INFO:tasks.workunit.client.1.smithi186.stdout:7/83: unlink d7/f9 0 2024-04-12T20:56:22.988 INFO:tasks.workunit.client.1.smithi186.stdout:2/26: fdatasync d0/f6 0 2024-04-12T20:56:22.991 INFO:tasks.workunit.client.1.smithi186.stdout:1/71: read ff [72730,78648] 0 2024-04-12T20:56:22.993 INFO:tasks.workunit.client.0.smithi018.stdout:4/833: write d3/d5d/d60/dc5/d3d/de4/f113 [148618,53767] 0 2024-04-12T20:56:22.995 INFO:tasks.workunit.client.1.smithi186.stdout:6/21: creat f5 x:0 0 0 2024-04-12T20:56:22.995 INFO:tasks.workunit.client.0.smithi018.stdout:5/973: write d0/d2/d33/d46/d1e/d2e/fd3 [1071900,7432] 0 2024-04-12T20:56:22.998 INFO:tasks.workunit.client.1.smithi186.stdout:9/35: dread f1 [0,4194304] 0 2024-04-12T20:56:23.002 INFO:tasks.workunit.client.1.smithi186.stdout:7/84: creat d7/d10/d1c/f21 x:0 0 0 2024-04-12T20:56:23.003 INFO:tasks.workunit.client.1.smithi186.stdout:5/18: write f4 [985905,1064] 0 2024-04-12T20:56:23.003 INFO:tasks.workunit.client.1.smithi186.stdout:8/22: dwrite f2 [0,4194304] 0 2024-04-12T20:56:23.003 INFO:tasks.workunit.client.1.smithi186.stdout:9/36: readlink l0 0 2024-04-12T20:56:23.007 INFO:tasks.workunit.client.0.smithi018.stdout:4/834: mknod d3/d5d/d60/dc5/d43/d4b/de6/c117 0 2024-04-12T20:56:23.008 INFO:tasks.workunit.client.1.smithi186.stdout:6/22: truncate f5 984543 0 2024-04-12T20:56:23.014 INFO:tasks.workunit.client.1.smithi186.stdout:1/72: fdatasync f4 0 2024-04-12T20:56:23.014 INFO:tasks.workunit.client.1.smithi186.stdout:2/27: write d0/f2 [184342,102582] 0 2024-04-12T20:56:23.014 INFO:tasks.workunit.client.1.smithi186.stdout:4/90: rename d0/d12/fa to d0/dc/d14/f1c 0 2024-04-12T20:56:23.014 INFO:tasks.workunit.client.1.smithi186.stdout:3/72: dread d1/fc [0,4194304] 0 2024-04-12T20:56:23.014 INFO:tasks.workunit.client.1.smithi186.stdout:9/37: mknod d3/c7 0 2024-04-12T20:56:23.015 INFO:tasks.workunit.client.1.smithi186.stdout:1/73: chown ff 3418 1 2024-04-12T20:56:23.015 INFO:tasks.workunit.client.1.smithi186.stdout:0/96: dwrite d0/f6 [4194304,4194304] 0 2024-04-12T20:56:23.016 INFO:tasks.workunit.client.1.smithi186.stdout:4/91: readlink d0/l1 0 2024-04-12T20:56:23.021 INFO:tasks.workunit.client.0.smithi018.stdout:4/835: mknod d3/d32/d33/dbc/c118 0 2024-04-12T20:56:23.021 INFO:tasks.workunit.client.1.smithi186.stdout:6/23: dread f5 [0,4194304] 0 2024-04-12T20:56:23.023 INFO:tasks.workunit.client.1.smithi186.stdout:1/74: mkdir d14/d1a 0 2024-04-12T20:56:23.024 INFO:tasks.workunit.client.1.smithi186.stdout:5/19: dread f4 [0,4194304] 0 2024-04-12T20:56:23.028 INFO:tasks.workunit.client.0.smithi018.stdout:4/836: symlink d3/d32/d94/d10b/l119 0 2024-04-12T20:56:23.030 INFO:tasks.workunit.client.1.smithi186.stdout:0/97: fsync d0/d10/d1c/d1e/f1f 0 2024-04-12T20:56:23.033 INFO:tasks.workunit.client.1.smithi186.stdout:0/98: stat d0/d14/c18 0 2024-04-12T20:56:23.033 INFO:tasks.workunit.client.1.smithi186.stdout:8/23: getdents . 0 2024-04-12T20:56:23.033 INFO:tasks.workunit.client.1.smithi186.stdout:8/24: rmdir - no directory 2024-04-12T20:56:23.037 INFO:tasks.workunit.client.0.smithi018.stdout:5/974: dwrite d0/d5b/d84/d83/f140 [0,4194304] 0 2024-04-12T20:56:23.039 INFO:tasks.workunit.client.1.smithi186.stdout:5/20: write f2 [804668,16960] 0 2024-04-12T20:56:23.040 INFO:tasks.workunit.client.1.smithi186.stdout:7/85: dwrite d7/d10/f13 [0,4194304] 0 2024-04-12T20:56:23.044 INFO:tasks.workunit.client.1.smithi186.stdout:8/25: mknod c6 0 2024-04-12T20:56:23.045 INFO:tasks.workunit.client.1.smithi186.stdout:5/21: write f4 [379310,12506] 0 2024-04-12T20:56:23.045 INFO:tasks.workunit.client.1.smithi186.stdout:2/28: getdents d0 0 2024-04-12T20:56:23.048 INFO:tasks.workunit.client.0.smithi018.stdout:5/975: link d0/d2/l11f d0/d2/d33/d46/d1e/l150 0 2024-04-12T20:56:23.050 INFO:tasks.workunit.client.0.smithi018.stdout:5/976: chown d0/d2/d33/d46/l25 596 1 2024-04-12T20:56:23.054 INFO:tasks.workunit.client.1.smithi186.stdout:7/86: mknod d7/c22 0 2024-04-12T20:56:23.056 INFO:tasks.workunit.client.0.smithi018.stdout:5/977: write d0/d2/d33/d46/d9e/ff5 [932754,18237] 0 2024-04-12T20:56:23.056 INFO:tasks.workunit.client.1.smithi186.stdout:2/29: read d0/f2 [1495114,130138] 0 2024-04-12T20:56:23.057 INFO:tasks.workunit.client.1.smithi186.stdout:1/75: creat d14/d1a/f1b x:0 0 0 2024-04-12T20:56:23.057 INFO:tasks.workunit.client.1.smithi186.stdout:6/24: creat f6 x:0 0 0 2024-04-12T20:56:23.057 INFO:tasks.workunit.client.0.smithi018.stdout:4/837: write d3/d32/fd [4195727,79196] 0 2024-04-12T20:56:23.060 INFO:tasks.workunit.client.1.smithi186.stdout:9/38: getdents d3 0 2024-04-12T20:56:23.061 INFO:tasks.workunit.client.1.smithi186.stdout:5/22: mkdir d5 0 2024-04-12T20:56:23.061 INFO:tasks.workunit.client.0.smithi018.stdout:5/978: symlink d0/d2/d33/d46/d1e/d9a/l151 0 2024-04-12T20:56:23.065 INFO:tasks.workunit.client.1.smithi186.stdout:4/92: dwrite d0/d12/f7 [0,4194304] 0 2024-04-12T20:56:23.065 INFO:tasks.workunit.client.0.smithi018.stdout:5/979: mknod d0/d2/d33/d10/d60/dfa/c152 0 2024-04-12T20:56:23.067 INFO:tasks.workunit.client.1.smithi186.stdout:7/87: write d7/d10/f13 [2504495,25995] 0 2024-04-12T20:56:23.072 INFO:tasks.workunit.client.0.smithi018.stdout:5/980: mkdir d0/d2/d33/d46/d35/dd2/d153 0 2024-04-12T20:56:23.077 INFO:tasks.workunit.client.1.smithi186.stdout:5/23: creat d5/f6 x:0 0 0 2024-04-12T20:56:23.081 INFO:tasks.workunit.client.1.smithi186.stdout:6/25: dread f5 [0,4194304] 0 2024-04-12T20:56:23.081 INFO:tasks.workunit.client.1.smithi186.stdout:4/93: creat d0/dc/f1d x:0 0 0 2024-04-12T20:56:23.084 INFO:tasks.workunit.client.1.smithi186.stdout:0/99: dwrite d0/d10/f15 [0,4194304] 0 2024-04-12T20:56:23.084 INFO:tasks.workunit.client.1.smithi186.stdout:7/88: symlink d7/d10/l23 0 2024-04-12T20:56:23.086 INFO:tasks.workunit.client.1.smithi186.stdout:6/26: write f6 [332621,35624] 0 2024-04-12T20:56:23.086 INFO:tasks.workunit.client.1.smithi186.stdout:5/24: mknod d5/c7 0 2024-04-12T20:56:23.088 INFO:tasks.workunit.client.1.smithi186.stdout:8/26: dwrite f2 [0,4194304] 0 2024-04-12T20:56:23.090 INFO:tasks.workunit.client.1.smithi186.stdout:4/94: write d0/fd [172265,49403] 0 2024-04-12T20:56:23.092 INFO:tasks.workunit.client.1.smithi186.stdout:7/89: mknod d7/d10/c24 0 2024-04-12T20:56:23.093 INFO:tasks.workunit.client.1.smithi186.stdout:1/76: dread d14/f16 [0,4194304] 0 2024-04-12T20:56:23.093 INFO:tasks.workunit.client.1.smithi186.stdout:9/39: dread f1 [0,4194304] 0 2024-04-12T20:56:23.093 INFO:tasks.workunit.client.1.smithi186.stdout:2/30: dread d0/f2 [0,4194304] 0 2024-04-12T20:56:23.095 INFO:tasks.workunit.client.1.smithi186.stdout:8/27: truncate f5 5018286 0 2024-04-12T20:56:23.098 INFO:tasks.workunit.client.1.smithi186.stdout:7/90: creat d7/f25 x:0 0 0 2024-04-12T20:56:23.098 INFO:tasks.workunit.client.1.smithi186.stdout:6/27: symlink l7 0 2024-04-12T20:56:23.098 INFO:tasks.workunit.client.1.smithi186.stdout:4/95: write d0/f10 [4066555,89335] 0 2024-04-12T20:56:23.100 INFO:tasks.workunit.client.1.smithi186.stdout:3/73: sync 2024-04-12T20:56:23.100 INFO:tasks.workunit.client.1.smithi186.stdout:0/100: symlink d0/d10/d1c/d1e/l20 0 2024-04-12T20:56:23.100 INFO:tasks.workunit.client.1.smithi186.stdout:5/25: dread f4 [0,4194304] 0 2024-04-12T20:56:23.103 INFO:tasks.workunit.client.1.smithi186.stdout:2/31: read d0/f4 [121076,11657] 0 2024-04-12T20:56:23.104 INFO:tasks.workunit.client.1.smithi186.stdout:9/40: dread f1 [4194304,4194304] 0 2024-04-12T20:56:23.105 INFO:tasks.workunit.client.0.smithi018.stdout:5/981: dwrite d0/d2/d33/d10/d60/dfa/dbf/da3/ffd [0,4194304] 0 2024-04-12T20:56:23.106 INFO:tasks.workunit.client.1.smithi186.stdout:5/26: dread - d5/f6 zero size 2024-04-12T20:56:23.107 INFO:tasks.workunit.client.0.smithi018.stdout:4/838: dread d3/d5d/d60/d64/d4f/f95 [0,4194304] 0 2024-04-12T20:56:23.107 INFO:tasks.workunit.client.1.smithi186.stdout:6/28: write f5 [1605568,88414] 0 2024-04-12T20:56:23.107 INFO:tasks.workunit.client.1.smithi186.stdout:8/28: dread f2 [4194304,4194304] 0 2024-04-12T20:56:23.108 INFO:tasks.workunit.client.1.smithi186.stdout:7/91: mknod d7/d10/c26 0 2024-04-12T20:56:23.110 INFO:tasks.workunit.client.1.smithi186.stdout:8/29: chown l3 227 1 2024-04-12T20:56:23.110 INFO:tasks.workunit.client.1.smithi186.stdout:4/96: mknod d0/dc/c1e 0 2024-04-12T20:56:23.111 INFO:tasks.workunit.client.1.smithi186.stdout:7/92: write d7/d10/d1c/f21 [271727,110707] 0 2024-04-12T20:56:23.115 INFO:tasks.workunit.client.1.smithi186.stdout:3/74: symlink d1/lf 0 2024-04-12T20:56:23.116 INFO:tasks.workunit.client.1.smithi186.stdout:0/101: read - d0/f17 zero size 2024-04-12T20:56:23.117 INFO:tasks.workunit.client.1.smithi186.stdout:3/75: chown d1/d3 211 1 2024-04-12T20:56:23.118 INFO:tasks.workunit.client.1.smithi186.stdout:1/77: dread fe [0,4194304] 0 2024-04-12T20:56:23.119 INFO:tasks.workunit.client.1.smithi186.stdout:9/41: creat d3/f8 x:0 0 0 2024-04-12T20:56:23.120 INFO:tasks.workunit.client.0.smithi018.stdout:5/982: truncate d0/d2/d33/f15 12151586 0 2024-04-12T20:56:23.122 INFO:tasks.workunit.client.1.smithi186.stdout:7/93: rmdir d7/d10/d1c 39 2024-04-12T20:56:23.122 INFO:tasks.workunit.client.1.smithi186.stdout:7/94: read - d7/f25 zero size 2024-04-12T20:56:23.123 INFO:tasks.workunit.client.1.smithi186.stdout:4/97: mkdir d0/dc/d1f 0 2024-04-12T20:56:23.125 INFO:tasks.workunit.client.0.smithi018.stdout:5/983: readlink d0/d2/d33/d46/d1e/d2e/d7f/l13f 0 2024-04-12T20:56:23.125 INFO:tasks.workunit.client.1.smithi186.stdout:6/29: mknod c8 0 2024-04-12T20:56:23.127 INFO:tasks.workunit.client.1.smithi186.stdout:8/30: dread f5 [0,4194304] 0 2024-04-12T20:56:23.129 INFO:tasks.workunit.client.0.smithi018.stdout:4/839: dread d3/db/f26 [0,4194304] 0 2024-04-12T20:56:23.129 INFO:tasks.workunit.client.1.smithi186.stdout:3/76: rmdir d1/d3/da 39 2024-04-12T20:56:23.131 INFO:tasks.workunit.client.0.smithi018.stdout:5/984: write d0/d2/d33/d10/d60/dfa/dbf/da3/dae/f125 [606716,117247] 0 2024-04-12T20:56:23.131 INFO:tasks.workunit.client.1.smithi186.stdout:1/78: mknod d14/c1c 0 2024-04-12T20:56:23.133 INFO:tasks.workunit.client.0.smithi018.stdout:4/840: chown d3/d5d/d60/dc5/d40/f6d 12396 1 2024-04-12T20:56:23.133 INFO:tasks.workunit.client.1.smithi186.stdout:7/95: unlink d7/f14 0 2024-04-12T20:56:23.134 INFO:tasks.workunit.client.1.smithi186.stdout:7/96: dread - d7/f8 zero size 2024-04-12T20:56:23.135 INFO:tasks.workunit.client.1.smithi186.stdout:4/98: creat d0/dc/d14/f20 x:0 0 0 2024-04-12T20:56:23.137 INFO:tasks.workunit.client.0.smithi018.stdout:4/841: read - d3/d32/d94/fd1 zero size 2024-04-12T20:56:23.137 INFO:tasks.workunit.client.1.smithi186.stdout:8/31: mkdir d7 0 2024-04-12T20:56:23.139 INFO:tasks.workunit.client.0.smithi018.stdout:5/985: rename d0/d2/d33/d46/d9e/c13e to d0/d2/d33/d46/d35/d4d/db3/c154 0 2024-04-12T20:56:23.139 INFO:tasks.workunit.client.0.smithi018.stdout:4/842: fdatasync d3/d5d/d60/d64/fb9 0 2024-04-12T20:56:23.142 INFO:tasks.workunit.client.1.smithi186.stdout:9/42: truncate f1 4082745 0 2024-04-12T20:56:23.143 INFO:tasks.workunit.client.0.smithi018.stdout:5/986: mknod d0/d2/d33/d46/d1e/d55/c155 0 2024-04-12T20:56:23.144 INFO:tasks.workunit.client.1.smithi186.stdout:1/79: mkdir d14/d1a/d1d 0 2024-04-12T20:56:23.146 INFO:tasks.workunit.client.1.smithi186.stdout:7/97: fdatasync d7/d10/d1c/f20 0 2024-04-12T20:56:23.147 INFO:tasks.workunit.client.1.smithi186.stdout:6/30: mkdir d9 0 2024-04-12T20:56:23.148 INFO:tasks.workunit.client.0.smithi018.stdout:4/843: rename d3/d5d/d60/dc5/fb0 to d3/d5d/f11a 0 2024-04-12T20:56:23.150 INFO:tasks.workunit.client.1.smithi186.stdout:8/32: fdatasync f2 0 2024-04-12T20:56:23.152 INFO:tasks.workunit.client.1.smithi186.stdout:2/32: truncate d0/f6 2222974 0 2024-04-12T20:56:23.153 INFO:tasks.workunit.client.1.smithi186.stdout:3/77: symlink d1/d3/l10 0 2024-04-12T20:56:23.159 INFO:tasks.workunit.client.1.smithi186.stdout:7/98: creat d7/f27 x:0 0 0 2024-04-12T20:56:23.161 INFO:tasks.workunit.client.0.smithi018.stdout:5/987: write d0/d5b/fd6 [47974,100240] 0 2024-04-12T20:56:23.164 INFO:tasks.workunit.client.1.smithi186.stdout:0/102: rename d0/fe to d0/f21 0 2024-04-12T20:56:23.166 INFO:tasks.workunit.client.1.smithi186.stdout:4/99: rename d0/dc/d14 to d0/d21 0 2024-04-12T20:56:23.167 INFO:tasks.workunit.client.1.smithi186.stdout:4/100: chown d0/cf 61 1 2024-04-12T20:56:23.169 INFO:tasks.workunit.client.0.smithi018.stdout:4/844: write d3/d5d/d60/dc5/d3d/d80/f8c [18877,33793] 0 2024-04-12T20:56:23.169 INFO:tasks.workunit.client.1.smithi186.stdout:4/101: write d0/d21/f15 [4623226,69386] 0 2024-04-12T20:56:23.171 INFO:tasks.workunit.client.0.smithi018.stdout:5/988: getdents d0/d2/d33/d10/d60/dfa/dbf 0 2024-04-12T20:56:23.172 INFO:tasks.workunit.client.1.smithi186.stdout:4/102: chown d0/d21/f1c 3705919 1 2024-04-12T20:56:23.173 INFO:tasks.workunit.client.1.smithi186.stdout:9/43: unlink d3/f8 0 2024-04-12T20:56:23.174 INFO:tasks.workunit.client.1.smithi186.stdout:4/103: write d0/d21/f15 [2268489,88778] 0 2024-04-12T20:56:23.175 INFO:tasks.workunit.client.1.smithi186.stdout:9/44: chown d3/c4 3063 1 2024-04-12T20:56:23.175 INFO:tasks.workunit.client.0.smithi018.stdout:4/845: fsync d3/d5d/f6e 0 2024-04-12T20:56:23.176 INFO:tasks.workunit.client.1.smithi186.stdout:4/104: write d0/fd [662632,17551] 0 2024-04-12T20:56:23.177 INFO:tasks.workunit.client.1.smithi186.stdout:6/31: unlink c1 0 2024-04-12T20:56:23.178 INFO:tasks.workunit.client.1.smithi186.stdout:6/32: stat l7 0 2024-04-12T20:56:23.179 INFO:tasks.workunit.client.1.smithi186.stdout:0/103: write d0/f21 [339649,46441] 0 2024-04-12T20:56:23.181 INFO:tasks.workunit.client.1.smithi186.stdout:0/104: write d0/d14/f19 [315353,101424] 0 2024-04-12T20:56:23.183 INFO:tasks.workunit.client.1.smithi186.stdout:0/105: fsync d0/f6 0 2024-04-12T20:56:23.184 INFO:tasks.workunit.client.1.smithi186.stdout:7/99: rename f4 to d7/f28 0 2024-04-12T20:56:23.184 INFO:tasks.workunit.client.1.smithi186.stdout:8/33: mkdir d7/d8 0 2024-04-12T20:56:23.185 INFO:tasks.workunit.client.1.smithi186.stdout:8/34: fsync f5 0 2024-04-12T20:56:23.186 INFO:tasks.workunit.client.1.smithi186.stdout:6/33: dread f6 [0,4194304] 0 2024-04-12T20:56:23.186 INFO:tasks.workunit.client.1.smithi186.stdout:7/100: chown d7/d10/d1c/f20 0 1 2024-04-12T20:56:23.195 INFO:tasks.workunit.client.1.smithi186.stdout:4/105: unlink d0/c8 0 2024-04-12T20:56:23.195 INFO:tasks.workunit.client.1.smithi186.stdout:8/35: dread f5 [0,4194304] 0 2024-04-12T20:56:23.196 INFO:tasks.workunit.client.1.smithi186.stdout:4/106: chown d0/d21/f15 76 1 2024-04-12T20:56:23.197 INFO:tasks.workunit.client.0.smithi018.stdout:4/846: dwrite d3/d5d/d60/dc5/fa9 [4194304,4194304] 0 2024-04-12T20:56:23.197 INFO:tasks.workunit.client.0.smithi018.stdout:5/989: dwrite d0/d2/d33/d46/d1e/d2e/f9c [0,4194304] 0 2024-04-12T20:56:23.201 INFO:tasks.workunit.client.0.smithi018.stdout:5/990: stat d0/d2/d33/d46/d18/c81 0 2024-04-12T20:56:23.202 INFO:tasks.workunit.client.1.smithi186.stdout:1/80: truncate fb 3733747 0 2024-04-12T20:56:23.206 INFO:tasks.workunit.client.1.smithi186.stdout:3/78: creat d1/d3/f11 x:0 0 0 2024-04-12T20:56:23.207 INFO:tasks.workunit.client.0.smithi018.stdout:5/991: mkdir d0/d2b/d156 0 2024-04-12T20:56:23.207 INFO:tasks.workunit.client.1.smithi186.stdout:1/81: dread ff [0,4194304] 0 2024-04-12T20:56:23.211 INFO:tasks.workunit.client.0.smithi018.stdout:5/992: chown d0/d5b/d84/l44 55 1 2024-04-12T20:56:23.212 INFO:tasks.workunit.client.0.smithi018.stdout:4/847: rename d3/db/fc4 to d3/d5d/d60/dc5/d3d/d80/d111/f11b 0 2024-04-12T20:56:23.215 INFO:tasks.workunit.client.0.smithi018.stdout:5/993: chown d0/d2/d11/d37/lb0 797537 1 2024-04-12T20:56:23.216 INFO:tasks.workunit.client.1.smithi186.stdout:6/34: unlink f5 0 2024-04-12T20:56:23.218 INFO:tasks.workunit.client.1.smithi186.stdout:5/27: sync 2024-04-12T20:56:23.218 INFO:tasks.workunit.client.1.smithi186.stdout:6/35: chown f6 101 1 2024-04-12T20:56:23.219 INFO:tasks.workunit.client.0.smithi018.stdout:5/994: read - d0/d2/d33/d46/d35/f106 zero size 2024-04-12T20:56:23.222 INFO:tasks.workunit.client.1.smithi186.stdout:3/79: rename d1 to d1/d3/d12 22 2024-04-12T20:56:23.223 INFO:tasks.workunit.client.1.smithi186.stdout:4/107: creat d0/d21/f22 x:0 0 0 2024-04-12T20:56:23.227 INFO:tasks.workunit.client.0.smithi018.stdout:4/848: write d3/d5d/d60/dc5/d3d/fed [37611,9333] 0 2024-04-12T20:56:23.230 INFO:tasks.workunit.client.0.smithi018.stdout:5/995: link d0/d5b/f5d d0/d2/d33/d46/d1e/d113/f157 0 2024-04-12T20:56:23.231 INFO:tasks.workunit.client.1.smithi186.stdout:8/36: getdents d7/d8 0 2024-04-12T20:56:23.232 INFO:tasks.workunit.client.1.smithi186.stdout:5/28: write f2 [14663,17675] 0 2024-04-12T20:56:23.232 INFO:tasks.workunit.client.0.smithi018.stdout:4/849: mkdir d3/db/de3/dfc/d11c 0 2024-04-12T20:56:23.234 INFO:tasks.workunit.client.0.smithi018.stdout:5/996: symlink d0/d2/d33/d46/d18/l158 0 2024-04-12T20:56:23.236 INFO:tasks.workunit.client.1.smithi186.stdout:5/29: write f2 [435062,7449] 0 2024-04-12T20:56:23.236 INFO:tasks.workunit.client.1.smithi186.stdout:7/101: dwrite d7/f28 [4194304,4194304] 0 2024-04-12T20:56:23.239 INFO:tasks.workunit.client.0.smithi018.stdout:4/850: rename d3/d32/d66/ce5 to d3/d5d/d60/dc5/d40/c11d 0 2024-04-12T20:56:23.240 INFO:tasks.workunit.client.1.smithi186.stdout:9/45: write f1 [4949500,82675] 0 2024-04-12T20:56:23.242 INFO:tasks.workunit.client.0.smithi018.stdout:5/997: truncate d0/d2/de4/d124/f129 2257449 0 2024-04-12T20:56:23.243 INFO:tasks.workunit.client.1.smithi186.stdout:6/36: mknod d9/ca 0 2024-04-12T20:56:23.245 INFO:tasks.workunit.client.1.smithi186.stdout:0/106: link d0/cc d0/d10/d1c/c22 0 2024-04-12T20:56:23.246 INFO:tasks.workunit.client.1.smithi186.stdout:8/37: creat d7/f9 x:0 0 0 2024-04-12T20:56:23.246 INFO:tasks.workunit.client.1.smithi186.stdout:4/108: link d0/d12/f17 d0/dc/f23 0 2024-04-12T20:56:23.247 INFO:tasks.workunit.client.1.smithi186.stdout:8/38: readlink l4 0 2024-04-12T20:56:23.249 INFO:tasks.workunit.client.0.smithi018.stdout:4/851: truncate d3/d5d/d60/dc5/d53/fcf 125706 0 2024-04-12T20:56:23.250 INFO:tasks.workunit.client.0.smithi018.stdout:5/998: creat d0/d2/d33/d10/d4e/d13a/f159 x:0 0 0 2024-04-12T20:56:23.252 INFO:tasks.workunit.client.1.smithi186.stdout:7/102: write d7/d10/d1c/f1f [1825752,77192] 0 2024-04-12T20:56:23.253 INFO:tasks.workunit.client.1.smithi186.stdout:6/37: creat d9/fb x:0 0 0 2024-04-12T20:56:23.256 INFO:tasks.workunit.client.1.smithi186.stdout:4/109: creat d0/f24 x:0 0 0 2024-04-12T20:56:23.257 INFO:tasks.workunit.client.1.smithi186.stdout:3/80: getdents d1/d3 0 2024-04-12T20:56:23.260 INFO:tasks.workunit.client.0.smithi018.stdout:5/999: rename d0/d2/d11/d74 to d0/d2/d33/d46/d35/d15a 0 2024-04-12T20:56:23.261 INFO:tasks.workunit.client.1.smithi186.stdout:4/110: stat d0/f24 0 2024-04-12T20:56:23.261 INFO:tasks.workunit.client.1.smithi186.stdout:7/103: mknod d7/c29 0 2024-04-12T20:56:23.264 INFO:tasks.workunit.client.1.smithi186.stdout:8/39: creat d7/d8/fa x:0 0 0 2024-04-12T20:56:23.265 INFO:tasks.workunit.client.1.smithi186.stdout:2/33: truncate d0/f2 3165857 0 2024-04-12T20:56:23.266 INFO:tasks.workunit.client.1.smithi186.stdout:4/111: readlink d0/l1 0 2024-04-12T20:56:23.267 INFO:tasks.workunit.client.0.smithi018.stdout:4/852: dwrite d3/d5d/d60/d64/fb9 [4194304,4194304] 0 2024-04-12T20:56:23.270 INFO:tasks.workunit.client.0.smithi018.stdout:4/853: read d3/d5d/d60/dc5/fa9 [6293029,106720] 0 2024-04-12T20:56:23.271 INFO:tasks.workunit.client.1.smithi186.stdout:5/30: dwrite f2 [0,4194304] 0 2024-04-12T20:56:23.272 INFO:tasks.workunit.client.1.smithi186.stdout:3/81: mknod d1/c13 0 2024-04-12T20:56:23.278 INFO:tasks.workunit.client.0.smithi018.stdout:4/854: mknod d3/db/de3/dfc/c11e 0 2024-04-12T20:56:23.279 INFO:tasks.workunit.client.1.smithi186.stdout:4/112: unlink d0/d21/f15 0 2024-04-12T20:56:23.285 INFO:tasks.workunit.client.1.smithi186.stdout:9/46: dwrite d3/f5 [0,4194304] 0 2024-04-12T20:56:23.293 INFO:tasks.workunit.client.1.smithi186.stdout:0/107: dwrite d0/d10/f12 [0,4194304] 0 2024-04-12T20:56:23.302 INFO:tasks.workunit.client.1.smithi186.stdout:8/40: link l4 d7/lb 0 2024-04-12T20:56:23.305 INFO:tasks.workunit.client.1.smithi186.stdout:9/47: mkdir d3/d9 0 2024-04-12T20:56:23.308 INFO:tasks.workunit.client.0.smithi018.stdout:4/855: write d3/d5d/d60/f67 [4039119,67097] 0 2024-04-12T20:56:23.308 INFO:tasks.workunit.client.1.smithi186.stdout:6/38: dwrite d9/fb [0,4194304] 0 2024-04-12T20:56:23.308 INFO:tasks.workunit.client.1.smithi186.stdout:0/108: rmdir d0/d10/d1c 39 2024-04-12T20:56:23.309 INFO:tasks.workunit.client.1.smithi186.stdout:1/82: dread fb [0,4194304] 0 2024-04-12T20:56:23.315 INFO:tasks.workunit.client.0.smithi018.stdout:4/856: rename d3/d5d/d60/dc5/dcc/ddf to d3/d32/d33/db6/d11f 0 2024-04-12T20:56:23.316 INFO:tasks.workunit.client.1.smithi186.stdout:0/109: dread - d0/f1d zero size 2024-04-12T20:56:23.320 INFO:tasks.workunit.client.1.smithi186.stdout:5/31: dwrite f4 [0,4194304] 0 2024-04-12T20:56:23.326 INFO:tasks.workunit.client.1.smithi186.stdout:6/39: write d9/fb [5086986,29980] 0 2024-04-12T20:56:23.326 INFO:tasks.workunit.client.1.smithi186.stdout:1/83: mknod d14/d1a/c1e 0 2024-04-12T20:56:23.327 INFO:tasks.workunit.client.1.smithi186.stdout:7/104: dwrite d7/d10/f15 [0,4194304] 0 2024-04-12T20:56:23.333 INFO:tasks.workunit.client.1.smithi186.stdout:4/113: truncate d0/f10 490411 0 2024-04-12T20:56:23.333 INFO:tasks.workunit.client.1.smithi186.stdout:5/32: read f2 [1095135,84752] 0 2024-04-12T20:56:23.339 INFO:tasks.workunit.client.0.smithi018.stdout:4/857: write d3/d5d/d60/dc5/d43/d4b/f109 [478734,126542] 0 2024-04-12T20:56:23.339 INFO:tasks.workunit.client.1.smithi186.stdout:7/105: dread d7/f12 [0,4194304] 0 2024-04-12T20:56:23.341 INFO:tasks.workunit.client.1.smithi186.stdout:3/82: dwrite d1/fc [0,4194304] 0 2024-04-12T20:56:23.341 INFO:tasks.workunit.client.1.smithi186.stdout:6/40: fsync d9/fb 0 2024-04-12T20:56:23.341 INFO:tasks.workunit.client.1.smithi186.stdout:5/33: creat d5/f8 x:0 0 0 2024-04-12T20:56:23.346 INFO:tasks.workunit.client.1.smithi186.stdout:7/106: write d7/d10/f15 [1171883,122779] 0 2024-04-12T20:56:23.346 INFO:tasks.workunit.client.0.smithi018.stdout:4/858: getdents d3/d5d/d60/dc5/d3d/d80/d111 0 2024-04-12T20:56:23.347 INFO:tasks.workunit.client.1.smithi186.stdout:4/114: write d0/d12/f1a [376610,46722] 0 2024-04-12T20:56:23.349 INFO:tasks.workunit.client.1.smithi186.stdout:1/84: truncate d14/f15 696947 0 2024-04-12T20:56:23.351 INFO:tasks.workunit.client.1.smithi186.stdout:2/34: dwrite d0/f6 [0,4194304] 0 2024-04-12T20:56:23.351 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:56:23 smithi018 ceph-mon[28799]: pgmap v44: 65 pgs: 65 active+clean; 1.9 GiB data, 7.1 GiB used, 529 GiB / 536 GiB avail; 73 MiB/s rd, 159 MiB/s wr, 484 op/s 2024-04-12T20:56:23.351 INFO:tasks.workunit.client.0.smithi018.stdout:4/859: unlink d3/d32/d33/db6/ffe 0 2024-04-12T20:56:23.352 INFO:tasks.workunit.client.1.smithi186.stdout:9/48: dread f1 [0,4194304] 0 2024-04-12T20:56:23.353 INFO:tasks.workunit.client.1.smithi186.stdout:3/83: fdatasync d1/d3/fe 0 2024-04-12T20:56:23.355 INFO:tasks.workunit.client.1.smithi186.stdout:6/41: symlink d9/lc 0 2024-04-12T20:56:23.356 INFO:tasks.workunit.client.1.smithi186.stdout:4/115: chown d0/d12/f1a 137689872 1 2024-04-12T20:56:23.358 INFO:tasks.workunit.client.1.smithi186.stdout:0/110: getdents d0/d10/d1c 0 2024-04-12T20:56:23.359 INFO:tasks.workunit.client.1.smithi186.stdout:4/116: write d0/fd [521424,30801] 0 2024-04-12T20:56:23.362 INFO:tasks.workunit.client.1.smithi186.stdout:1/85: write fe [1606280,47226] 0 2024-04-12T20:56:23.362 INFO:tasks.workunit.client.1.smithi186.stdout:5/34: write d5/f6 [586277,104277] 0 2024-04-12T20:56:23.367 INFO:tasks.workunit.client.1.smithi186.stdout:3/84: rmdir d1/d3 39 2024-04-12T20:56:23.368 INFO:tasks.workunit.client.1.smithi186.stdout:0/111: symlink d0/d14/l23 0 2024-04-12T20:56:23.370 INFO:tasks.workunit.client.1.smithi186.stdout:5/35: read d5/f6 [28133,101504] 0 2024-04-12T20:56:23.371 INFO:tasks.workunit.client.1.smithi186.stdout:5/36: chown d5 881551 1 2024-04-12T20:56:23.375 INFO:tasks.workunit.client.1.smithi186.stdout:0/112: creat d0/d14/f24 x:0 0 0 2024-04-12T20:56:23.377 INFO:tasks.workunit.client.1.smithi186.stdout:3/85: chown d1/d3/f11 5272394 1 2024-04-12T20:56:23.379 INFO:tasks.workunit.client.1.smithi186.stdout:4/117: dread d0/fd [0,4194304] 0 2024-04-12T20:56:23.381 INFO:tasks.workunit.client.1.smithi186.stdout:7/107: link d7/c1d d7/d10/c2a 0 2024-04-12T20:56:23.385 INFO:tasks.workunit.client.1.smithi186.stdout:7/108: readlink d7/d10/l23 0 2024-04-12T20:56:23.385 INFO:tasks.workunit.client.1.smithi186.stdout:4/118: dread - d0/d21/f20 zero size 2024-04-12T20:56:23.386 INFO:tasks.workunit.client.1.smithi186.stdout:3/86: dread - d1/d3/f11 zero size 2024-04-12T20:56:23.388 INFO:tasks.workunit.client.1.smithi186.stdout:9/49: truncate d3/f5 2470241 0 2024-04-12T20:56:23.390 INFO:tasks.workunit.client.1.smithi186.stdout:7/109: dread - d7/d10/f19 zero size 2024-04-12T20:56:23.390 INFO:tasks.workunit.client.1.smithi186.stdout:0/113: unlink d0/fd 0 2024-04-12T20:56:23.392 INFO:tasks.workunit.client.1.smithi186.stdout:5/37: link d5/f6 d5/f9 0 2024-04-12T20:56:23.393 INFO:tasks.workunit.client.1.smithi186.stdout:2/35: dwrite d0/f6 [4194304,4194304] 0 2024-04-12T20:56:23.393 INFO:tasks.workunit.client.1.smithi186.stdout:4/119: mknod d0/c25 0 2024-04-12T20:56:23.394 INFO:tasks.workunit.client.1.smithi186.stdout:6/42: dwrite d9/fb [0,4194304] 0 2024-04-12T20:56:23.394 INFO:tasks.workunit.client.1.smithi186.stdout:9/50: mknod d3/ca 0 2024-04-12T20:56:23.410 INFO:tasks.workunit.client.1.smithi186.stdout:0/114: mknod d0/d14/c25 0 2024-04-12T20:56:23.410 INFO:tasks.workunit.client.1.smithi186.stdout:9/51: chown d3 3469628 1 2024-04-12T20:56:23.410 INFO:tasks.workunit.client.1.smithi186.stdout:7/110: symlink d7/l2b 0 2024-04-12T20:56:23.411 INFO:tasks.workunit.client.1.smithi186.stdout:9/52: stat d3 0 2024-04-12T20:56:23.411 INFO:tasks.workunit.client.1.smithi186.stdout:2/36: creat d0/f7 x:0 0 0 2024-04-12T20:56:23.411 INFO:tasks.workunit.client.1.smithi186.stdout:1/86: dread d14/f16 [0,4194304] 0 2024-04-12T20:56:23.412 INFO:tasks.workunit.client.1.smithi186.stdout:6/43: mknod d9/cd 0 2024-04-12T20:56:23.413 INFO:tasks.workunit.client.1.smithi186.stdout:4/120: read d0/dc/fe [307483,112035] 0 2024-04-12T20:56:23.413 INFO:tasks.workunit.client.1.smithi186.stdout:2/37: read d0/f6 [713433,61185] 0 2024-04-12T20:56:23.416 INFO:tasks.workunit.client.1.smithi186.stdout:7/111: creat d7/d10/d1c/f2c x:0 0 0 2024-04-12T20:56:23.416 INFO:tasks.workunit.client.1.smithi186.stdout:0/115: symlink d0/l26 0 2024-04-12T20:56:23.419 INFO:tasks.workunit.client.1.smithi186.stdout:4/121: mkdir d0/d26 0 2024-04-12T20:56:23.421 INFO:tasks.workunit.client.1.smithi186.stdout:0/116: mknod d0/d14/c27 0 2024-04-12T20:56:23.421 INFO:tasks.workunit.client.1.smithi186.stdout:7/112: dread d7/fc [0,4194304] 0 2024-04-12T20:56:23.423 INFO:tasks.workunit.client.1.smithi186.stdout:2/38: creat d0/d3/f8 x:0 0 0 2024-04-12T20:56:23.426 INFO:tasks.workunit.client.1.smithi186.stdout:2/39: write d0/f7 [429630,82599] 0 2024-04-12T20:56:23.426 INFO:tasks.workunit.client.1.smithi186.stdout:4/122: write d0/fd [1079136,87370] 0 2024-04-12T20:56:23.431 INFO:tasks.workunit.client.1.smithi186.stdout:2/40: mknod d0/c9 0 2024-04-12T20:56:23.431 INFO:tasks.workunit.client.1.smithi186.stdout:7/113: write d7/d10/d1c/f2c [508610,69451] 0 2024-04-12T20:56:23.432 INFO:tasks.workunit.client.1.smithi186.stdout:1/87: truncate d14/f15 789978 0 2024-04-12T20:56:23.435 INFO:tasks.workunit.client.1.smithi186.stdout:5/38: dwrite f4 [0,4194304] 0 2024-04-12T20:56:23.435 INFO:tasks.workunit.client.1.smithi186.stdout:4/123: creat d0/d21/f27 x:0 0 0 2024-04-12T20:56:23.437 INFO:tasks.workunit.client.1.smithi186.stdout:0/117: truncate d0/d10/d1c/d1e/f1f 65573 0 2024-04-12T20:56:23.439 INFO:tasks.workunit.client.1.smithi186.stdout:4/124: fdatasync d0/f24 0 2024-04-12T20:56:23.442 INFO:tasks.workunit.client.1.smithi186.stdout:7/114: mkdir d7/d10/d2d 0 2024-04-12T20:56:23.445 INFO:tasks.workunit.client.1.smithi186.stdout:0/118: rename d0/f17 to d0/d10/d1c/f28 0 2024-04-12T20:56:23.449 INFO:tasks.workunit.client.1.smithi186.stdout:7/115: mknod d7/d10/d1c/c2e 0 2024-04-12T20:56:23.451 INFO:tasks.workunit.client.1.smithi186.stdout:5/39: getdents d5 0 2024-04-12T20:56:23.453 INFO:tasks.workunit.client.1.smithi186.stdout:0/119: symlink d0/d14/l29 0 2024-04-12T20:56:23.454 INFO:tasks.workunit.client.1.smithi186.stdout:6/44: dwrite f6 [0,4194304] 0 2024-04-12T20:56:23.454 INFO:tasks.workunit.client.1.smithi186.stdout:4/125: fdatasync d0/d12/f4 0 2024-04-12T20:56:23.457 INFO:tasks.workunit.client.1.smithi186.stdout:7/116: unlink d7/f1b 0 2024-04-12T20:56:23.458 INFO:tasks.workunit.client.1.smithi186.stdout:0/120: stat d0/f1d 0 2024-04-12T20:56:23.462 INFO:tasks.workunit.client.1.smithi186.stdout:5/40: fdatasync f2 0 2024-04-12T20:56:23.464 INFO:tasks.workunit.client.1.smithi186.stdout:9/53: dwrite d3/f5 [0,4194304] 0 2024-04-12T20:56:23.465 INFO:tasks.workunit.client.1.smithi186.stdout:6/45: creat d9/fe x:0 0 0 2024-04-12T20:56:23.469 INFO:tasks.workunit.client.1.smithi186.stdout:1/88: dwrite fe [0,4194304] 0 2024-04-12T20:56:23.469 INFO:tasks.workunit.client.1.smithi186.stdout:5/41: unlink d5/c7 0 2024-04-12T20:56:23.470 INFO:tasks.workunit.client.1.smithi186.stdout:1/89: stat f4 0 2024-04-12T20:56:23.471 INFO:tasks.workunit.client.1.smithi186.stdout:2/41: dwrite d0/f2 [4194304,4194304] 0 2024-04-12T20:56:23.475 INFO:tasks.workunit.client.1.smithi186.stdout:1/90: rename d14/c17 to d14/d1a/c1f 0 2024-04-12T20:56:23.476 INFO:tasks.workunit.client.1.smithi186.stdout:5/42: symlink d5/la 0 2024-04-12T20:56:23.478 INFO:tasks.workunit.client.1.smithi186.stdout:4/126: getdents d0 0 2024-04-12T20:56:23.478 INFO:tasks.workunit.client.1.smithi186.stdout:5/43: fsync f2 0 2024-04-12T20:56:23.478 INFO:tasks.workunit.client.1.smithi186.stdout:1/91: readlink d14/l19 0 2024-04-12T20:56:23.479 INFO:tasks.workunit.client.1.smithi186.stdout:7/117: link d7/d10/c1a d7/d10/d1c/c2f 0 2024-04-12T20:56:23.481 INFO:tasks.workunit.client.1.smithi186.stdout:6/46: creat d9/ff x:0 0 0 2024-04-12T20:56:23.482 INFO:tasks.workunit.client.1.smithi186.stdout:5/44: mkdir d5/db 0 2024-04-12T20:56:23.483 INFO:tasks.workunit.client.1.smithi186.stdout:4/127: mknod d0/d12/c28 0 2024-04-12T20:56:23.484 INFO:tasks.workunit.client.1.smithi186.stdout:4/128: chown d0/dc/d1f 490 1 2024-04-12T20:56:23.486 INFO:tasks.workunit.client.1.smithi186.stdout:4/129: chown d0/f18 724469 1 2024-04-12T20:56:23.486 INFO:tasks.workunit.client.1.smithi186.stdout:0/121: rmdir d0 39 2024-04-12T20:56:23.487 INFO:tasks.workunit.client.1.smithi186.stdout:6/47: chown d9/ff 114176 1 2024-04-12T20:56:23.491 INFO:tasks.workunit.client.1.smithi186.stdout:6/48: read f6 [374687,89463] 0 2024-04-12T20:56:23.493 INFO:tasks.workunit.client.1.smithi186.stdout:4/130: creat d0/dc/f29 x:0 0 0 2024-04-12T20:56:23.493 INFO:tasks.workunit.client.1.smithi186.stdout:1/92: link c10 d14/d1a/c20 0 2024-04-12T20:56:23.495 INFO:tasks.workunit.client.1.smithi186.stdout:2/42: dread d0/f4 [0,4194304] 0 2024-04-12T20:56:23.496 INFO:tasks.workunit.client.1.smithi186.stdout:4/131: stat d0/c25 0 2024-04-12T20:56:23.499 INFO:tasks.workunit.client.1.smithi186.stdout:0/122: chown d0/c8 1693481 1 2024-04-12T20:56:23.500 INFO:tasks.workunit.client.1.smithi186.stdout:2/43: chown d0/c5 102714064 1 2024-04-12T20:56:23.501 INFO:tasks.workunit.client.1.smithi186.stdout:4/132: symlink d0/l2a 0 2024-04-12T20:56:23.503 INFO:tasks.workunit.client.1.smithi186.stdout:7/118: rename l3 to d7/d10/l30 0 2024-04-12T20:56:23.507 INFO:tasks.workunit.client.1.smithi186.stdout:8/41: sync 2024-04-12T20:56:23.507 INFO:tasks.workunit.client.1.smithi186.stdout:9/54: dwrite f1 [0,4194304] 0 2024-04-12T20:56:23.507 INFO:tasks.workunit.client.1.smithi186.stdout:0/123: dread - d0/d14/f24 zero size 2024-04-12T20:56:23.507 INFO:tasks.workunit.client.1.smithi186.stdout:1/93: dread fb [0,4194304] 0 2024-04-12T20:56:23.508 INFO:tasks.workunit.client.1.smithi186.stdout:2/44: symlink d0/d3/la 0 2024-04-12T20:56:23.510 INFO:tasks.workunit.client.1.smithi186.stdout:4/133: creat d0/d26/f2b x:0 0 0 2024-04-12T20:56:23.514 INFO:tasks.workunit.client.1.smithi186.stdout:5/45: dwrite f2 [4194304,4194304] 0 2024-04-12T20:56:23.530 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:56:23 smithi186 ceph-mon[36597]: pgmap v44: 65 pgs: 65 active+clean; 1.9 GiB data, 7.1 GiB used, 529 GiB / 536 GiB avail; 73 MiB/s rd, 159 MiB/s wr, 484 op/s 2024-04-12T20:56:23.571 INFO:tasks.workunit.client.1.smithi186.stdout:1/94: unlink la 0 2024-04-12T20:56:23.574 INFO:tasks.workunit.client.1.smithi186.stdout:4/134: fsync d0/dc/f29 0 2024-04-12T20:56:23.576 INFO:tasks.workunit.client.1.smithi186.stdout:5/46: write f4 [2058695,28492] 0 2024-04-12T20:56:23.577 INFO:tasks.workunit.client.1.smithi186.stdout:4/135: dread - d0/d12/fb zero size 2024-04-12T20:56:23.579 INFO:tasks.workunit.client.1.smithi186.stdout:4/136: fdatasync d0/d12/fb 0 2024-04-12T20:56:23.580 INFO:tasks.workunit.client.1.smithi186.stdout:6/49: dread f6 [0,4194304] 0 2024-04-12T20:56:23.581 INFO:tasks.workunit.client.1.smithi186.stdout:6/50: readlink l7 0 2024-04-12T20:56:23.583 INFO:tasks.workunit.client.1.smithi186.stdout:7/119: dwrite d7/d10/f15 [0,4194304] 0 2024-04-12T20:56:23.585 INFO:tasks.workunit.client.1.smithi186.stdout:8/42: dread f2 [0,4194304] 0 2024-04-12T20:56:23.587 INFO:tasks.workunit.client.1.smithi186.stdout:4/137: dread d0/fd [0,4194304] 0 2024-04-12T20:56:23.590 INFO:tasks.workunit.client.1.smithi186.stdout:1/95: truncate ff 1239640 0 2024-04-12T20:56:23.592 INFO:tasks.workunit.client.1.smithi186.stdout:4/138: dread - d0/d12/f17 zero size 2024-04-12T20:56:23.593 INFO:tasks.workunit.client.1.smithi186.stdout:6/51: creat d9/f10 x:0 0 0 2024-04-12T20:56:23.595 INFO:tasks.workunit.client.1.smithi186.stdout:5/47: getdents d5/db 0 2024-04-12T20:56:23.596 INFO:tasks.workunit.client.1.smithi186.stdout:0/124: rename d0/f1d to d0/d10/f2a 0 2024-04-12T20:56:23.598 INFO:tasks.workunit.client.1.smithi186.stdout:2/45: dwrite d0/d3/f8 [0,4194304] 0 2024-04-12T20:56:23.599 INFO:tasks.workunit.client.1.smithi186.stdout:1/96: dread d14/f15 [0,4194304] 0 2024-04-12T20:56:23.608 INFO:tasks.workunit.client.1.smithi186.stdout:2/46: mknod d0/cb 0 2024-04-12T20:56:23.610 INFO:tasks.workunit.client.1.smithi186.stdout:7/120: dread d7/d10/d1c/f1f [0,4194304] 0 2024-04-12T20:56:23.610 INFO:tasks.workunit.client.1.smithi186.stdout:8/43: dread f5 [0,4194304] 0 2024-04-12T20:56:23.610 INFO:tasks.workunit.client.1.smithi186.stdout:3/87: sync 2024-04-12T20:56:23.611 INFO:tasks.workunit.client.1.smithi186.stdout:2/47: chown d0/f7 3 1 2024-04-12T20:56:23.614 INFO:tasks.workunit.client.1.smithi186.stdout:0/125: creat d0/f2b x:0 0 0 2024-04-12T20:56:23.614 INFO:tasks.workunit.client.1.smithi186.stdout:5/48: link d5/f9 d5/db/fc 0 2024-04-12T20:56:23.616 INFO:tasks.workunit.client.1.smithi186.stdout:8/44: mknod d7/cc 0 2024-04-12T20:56:23.621 INFO:tasks.workunit.client.1.smithi186.stdout:7/121: rename d7/l2b to d7/d10/d1c/l31 0 2024-04-12T20:56:23.625 INFO:tasks.workunit.client.1.smithi186.stdout:7/122: stat d7/f16 0 2024-04-12T20:56:23.625 INFO:tasks.workunit.client.1.smithi186.stdout:6/52: dwrite f6 [0,4194304] 0 2024-04-12T20:56:23.625 INFO:tasks.workunit.client.1.smithi186.stdout:8/45: creat d7/fd x:0 0 0 2024-04-12T20:56:23.626 INFO:tasks.workunit.client.1.smithi186.stdout:5/49: rename d5/f8 to d5/fd 0 2024-04-12T20:56:23.629 INFO:tasks.workunit.client.1.smithi186.stdout:4/139: dwrite d0/d21/f1c [0,4194304] 0 2024-04-12T20:56:23.630 INFO:tasks.workunit.client.1.smithi186.stdout:8/46: chown d7/d8/fa 162159922 1 2024-04-12T20:56:23.631 INFO:tasks.workunit.client.0.smithi018.stdout:1/717: sync 2024-04-12T20:56:23.631 INFO:tasks.workunit.client.0.smithi018.stdout:0/878: sync 2024-04-12T20:56:23.632 INFO:tasks.workunit.client.1.smithi186.stdout:7/123: mknod d7/d10/d1c/c32 0 2024-04-12T20:56:23.633 INFO:tasks.workunit.client.1.smithi186.stdout:3/88: dread d1/fc [0,4194304] 0 2024-04-12T20:56:23.634 INFO:tasks.workunit.client.1.smithi186.stdout:0/126: creat d0/d10/f2c x:0 0 0 2024-04-12T20:56:23.636 INFO:tasks.workunit.client.1.smithi186.stdout:4/140: truncate d0/d12/fb 48130 0 2024-04-12T20:56:23.636 INFO:tasks.workunit.client.1.smithi186.stdout:1/97: dwrite d14/f16 [0,4194304] 0 2024-04-12T20:56:23.636 INFO:tasks.workunit.client.1.smithi186.stdout:2/48: dread d0/d3/f8 [0,4194304] 0 2024-04-12T20:56:23.636 INFO:tasks.workunit.client.1.smithi186.stdout:3/89: chown d1/d3/fe 48154077 1 2024-04-12T20:56:23.636 INFO:tasks.workunit.client.1.smithi186.stdout:8/47: mkdir d7/de 0 2024-04-12T20:56:23.637 INFO:tasks.workunit.client.0.smithi018.stdout:0/879: mkdir da/d15/d17/d108/d128 0 2024-04-12T20:56:23.638 INFO:tasks.workunit.client.1.smithi186.stdout:1/98: stat d14/d1a/f1b 0 2024-04-12T20:56:23.638 INFO:tasks.workunit.client.0.smithi018.stdout:1/718: symlink d3/d9/da/dd/lf1 0 2024-04-12T20:56:23.639 INFO:tasks.workunit.client.1.smithi186.stdout:1/99: write d14/f16 [3347850,109370] 0 2024-04-12T20:56:23.643 INFO:tasks.workunit.client.1.smithi186.stdout:4/141: mknod d0/d12/c2c 0 2024-04-12T20:56:23.644 INFO:tasks.workunit.client.1.smithi186.stdout:9/55: sync 2024-04-12T20:56:23.645 INFO:tasks.workunit.client.1.smithi186.stdout:0/127: chown d0/la 109 1 2024-04-12T20:56:23.645 INFO:tasks.workunit.client.1.smithi186.stdout:6/53: dread f6 [0,4194304] 0 2024-04-12T20:56:23.646 INFO:tasks.workunit.client.1.smithi186.stdout:4/142: read d0/dc/fe [229983,129324] 0 2024-04-12T20:56:23.648 INFO:tasks.workunit.client.1.smithi186.stdout:9/56: truncate d3/f5 4370013 0 2024-04-12T20:56:23.649 INFO:tasks.workunit.client.1.smithi186.stdout:2/49: mkdir d0/dc 0 2024-04-12T20:56:23.650 INFO:tasks.workunit.client.1.smithi186.stdout:2/50: stat d0 0 2024-04-12T20:56:23.650 INFO:tasks.workunit.client.1.smithi186.stdout:9/57: chown d3/c7 16706 1 2024-04-12T20:56:23.650 INFO:tasks.workunit.client.1.smithi186.stdout:2/51: stat d0/d3/la 0 2024-04-12T20:56:23.657 INFO:tasks.workunit.client.1.smithi186.stdout:1/100: truncate fb 883447 0 2024-04-12T20:56:23.658 INFO:tasks.workunit.client.1.smithi186.stdout:8/48: truncate f5 3729369 0 2024-04-12T20:56:23.663 INFO:tasks.workunit.client.0.smithi018.stdout:0/880: dwrite da/d15/d17/d21/d36/da2/dc3/d3f/f10f [0,4194304] 0 2024-04-12T20:56:23.665 INFO:tasks.workunit.client.1.smithi186.stdout:2/52: mknod d0/cd 0 2024-04-12T20:56:23.666 INFO:tasks.workunit.client.1.smithi186.stdout:3/90: dwrite d1/fc [0,4194304] 0 2024-04-12T20:56:23.669 INFO:tasks.workunit.client.1.smithi186.stdout:5/50: truncate d5/f9 1043572 0 2024-04-12T20:56:23.669 INFO:tasks.workunit.client.0.smithi018.stdout:1/719: truncate d3/d9/da/dd/f55 1928541 0 2024-04-12T20:56:23.670 INFO:tasks.workunit.client.1.smithi186.stdout:3/91: read - d1/d3/f11 zero size 2024-04-12T20:56:23.671 INFO:tasks.workunit.client.0.smithi018.stdout:0/881: symlink da/d15/d17/d5c/d79/dd6/l129 0 2024-04-12T20:56:23.673 INFO:tasks.workunit.client.0.smithi018.stdout:1/720: write d3/d9/da/d14/fdb [13319392,128151] 0 2024-04-12T20:56:23.673 INFO:tasks.workunit.client.1.smithi186.stdout:0/128: mknod d0/c2d 0 2024-04-12T20:56:23.675 INFO:tasks.workunit.client.0.smithi018.stdout:0/882: read - da/d15/d17/d21/d36/d91/f10c zero size 2024-04-12T20:56:23.676 INFO:tasks.workunit.client.1.smithi186.stdout:4/143: link d0/d12/fb d0/dc/f2d 0 2024-04-12T20:56:23.677 INFO:tasks.workunit.client.0.smithi018.stdout:1/721: readlink d3/d9/da/dd/d31/d32/d13/l22 0 2024-04-12T20:56:23.678 INFO:tasks.workunit.client.0.smithi018.stdout:0/883: unlink da/f52 0 2024-04-12T20:56:23.679 INFO:tasks.workunit.client.1.smithi186.stdout:4/144: stat d0/dc/f29 0 2024-04-12T20:56:23.680 INFO:tasks.workunit.client.1.smithi186.stdout:0/129: chown d0/d10/d1c/f28 936 1 2024-04-12T20:56:23.683 INFO:tasks.workunit.client.1.smithi186.stdout:6/54: link c8 d9/c11 0 2024-04-12T20:56:23.684 INFO:tasks.workunit.client.1.smithi186.stdout:1/101: dwrite d14/d1a/f1b [0,4194304] 0 2024-04-12T20:56:23.686 INFO:tasks.workunit.client.1.smithi186.stdout:6/55: dread - d9/f10 zero size 2024-04-12T20:56:23.687 INFO:tasks.workunit.client.1.smithi186.stdout:9/58: rmdir d3/d9 0 2024-04-12T20:56:23.688 INFO:tasks.workunit.client.0.smithi018.stdout:0/884: getdents da/d15 0 2024-04-12T20:56:23.691 INFO:tasks.workunit.client.1.smithi186.stdout:5/51: symlink d5/db/le 0 2024-04-12T20:56:23.692 INFO:tasks.workunit.client.0.smithi018.stdout:0/885: read da/d15/d17/d21/d36/f80 [77277,46657] 0 2024-04-12T20:56:23.693 INFO:tasks.workunit.client.0.smithi018.stdout:0/886: readlink da/d15/l24 0 2024-04-12T20:56:23.695 INFO:tasks.workunit.client.1.smithi186.stdout:6/56: write d9/fb [3532676,11798] 0 2024-04-12T20:56:23.696 INFO:tasks.workunit.client.1.smithi186.stdout:4/145: rename d0/d12/c19 to d0/d21/c2e 0 2024-04-12T20:56:23.699 INFO:tasks.workunit.client.1.smithi186.stdout:2/53: dwrite d0/f7 [0,4194304] 0 2024-04-12T20:56:23.700 INFO:tasks.workunit.client.1.smithi186.stdout:3/92: link d1/lf d1/d3/da/l14 0 2024-04-12T20:56:23.701 INFO:tasks.workunit.client.1.smithi186.stdout:9/59: creat d3/fb x:0 0 0 2024-04-12T20:56:23.703 INFO:tasks.workunit.client.1.smithi186.stdout:9/60: fdatasync d3/fb 0 2024-04-12T20:56:23.703 INFO:tasks.workunit.client.0.smithi018.stdout:0/887: unlink da/d15/d17/d10d/c127 0 2024-04-12T20:56:23.704 INFO:tasks.workunit.client.0.smithi018.stdout:1/722: truncate d3/d9/da/dd/d31/f35 3505080 0 2024-04-12T20:56:23.706 INFO:tasks.workunit.client.0.smithi018.stdout:1/723: readlink d3/d9/da/dd/d31/d44/l6a 0 2024-04-12T20:56:23.708 INFO:tasks.workunit.client.1.smithi186.stdout:9/61: write d3/f5 [2102375,127829] 0 2024-04-12T20:56:23.709 INFO:tasks.workunit.client.1.smithi186.stdout:9/62: dread - d3/fb zero size 2024-04-12T20:56:23.710 INFO:tasks.workunit.client.1.smithi186.stdout:9/63: read - d3/fb zero size 2024-04-12T20:56:23.715 INFO:tasks.workunit.client.1.smithi186.stdout:2/54: creat d0/d3/fe x:0 0 0 2024-04-12T20:56:23.717 INFO:tasks.workunit.client.1.smithi186.stdout:3/93: mknod d1/d3/da/c15 0 2024-04-12T20:56:23.721 INFO:tasks.workunit.client.1.smithi186.stdout:9/64: dread d3/f5 [0,4194304] 0 2024-04-12T20:56:23.721 INFO:tasks.workunit.client.0.smithi018.stdout:1/724: dread d3/f4 [0,4194304] 0 2024-04-12T20:56:23.722 INFO:tasks.workunit.client.0.smithi018.stdout:0/888: dwrite da/faa [0,4194304] 0 2024-04-12T20:56:23.725 INFO:tasks.workunit.client.1.smithi186.stdout:2/55: fdatasync d0/f4 0 2024-04-12T20:56:23.730 INFO:tasks.workunit.client.1.smithi186.stdout:0/130: rename d0/d14/l23 to d0/l2e 0 2024-04-12T20:56:23.731 INFO:tasks.workunit.client.1.smithi186.stdout:5/52: link d5/db/le d5/db/lf 0 2024-04-12T20:56:23.735 INFO:tasks.workunit.client.0.smithi018.stdout:0/889: mknod da/d15/d17/d21/d36/da2/dc3/d83/d8b/c12a 0 2024-04-12T20:56:23.738 INFO:tasks.workunit.client.0.smithi018.stdout:1/725: rename d3/d9/da/dd/d31/d32/d33/le5 to d3/d9/da/dd/d31/d32/d13/d19/dd6/lf2 0 2024-04-12T20:56:23.739 INFO:tasks.workunit.client.0.smithi018.stdout:0/890: fsync da/faa 0 2024-04-12T20:56:23.740 INFO:tasks.workunit.client.1.smithi186.stdout:3/94: chown d1/lf 0 1 2024-04-12T20:56:23.744 INFO:tasks.workunit.client.1.smithi186.stdout:0/131: creat d0/d10/d1c/d1e/f2f x:0 0 0 2024-04-12T20:56:23.744 INFO:tasks.workunit.client.1.smithi186.stdout:0/132: chown d0/d10 0 1 2024-04-12T20:56:23.744 INFO:tasks.workunit.client.1.smithi186.stdout:0/133: fdatasync d0/f6 0 2024-04-12T20:56:23.744 INFO:tasks.workunit.client.1.smithi186.stdout:5/53: symlink d5/db/l10 0 2024-04-12T20:56:23.745 INFO:tasks.workunit.client.0.smithi018.stdout:0/891: mkdir da/d15/d17/d21/d36/d6e/d12b 0 2024-04-12T20:56:23.745 INFO:tasks.workunit.client.0.smithi018.stdout:1/726: symlink d3/d9/da/dd/d31/dc6/lf3 0 2024-04-12T20:56:23.745 INFO:tasks.workunit.client.1.smithi186.stdout:2/56: creat d0/dc/ff x:0 0 0 2024-04-12T20:56:23.745 INFO:tasks.workunit.client.1.smithi186.stdout:2/57: chown d0 51 1 2024-04-12T20:56:23.746 INFO:tasks.workunit.client.1.smithi186.stdout:3/95: unlink d1/d3/fe 0 2024-04-12T20:56:23.748 INFO:tasks.workunit.client.1.smithi186.stdout:5/54: creat d5/db/f11 x:0 0 0 2024-04-12T20:56:23.750 INFO:tasks.workunit.client.1.smithi186.stdout:0/134: rename d0/d14/c18 to d0/d14/c30 0 2024-04-12T20:56:23.750 INFO:tasks.workunit.client.1.smithi186.stdout:0/135: chown d0/d14 12523906 1 2024-04-12T20:56:23.751 INFO:tasks.workunit.client.1.smithi186.stdout:0/136: stat d0/d10/f2c 0 2024-04-12T20:56:23.753 INFO:tasks.workunit.client.1.smithi186.stdout:0/137: truncate d0/d14/f19 673356 0 2024-04-12T20:56:23.756 INFO:tasks.workunit.client.0.smithi018.stdout:0/892: dread da/d15/d17/d21/d36/da2/dc3/da9/f119 [0,4194304] 0 2024-04-12T20:56:23.757 INFO:tasks.workunit.client.1.smithi186.stdout:3/96: symlink d1/l16 0 2024-04-12T20:56:23.757 INFO:tasks.workunit.client.0.smithi018.stdout:1/727: dread d3/d9/da/dd/d31/d32/d13/d19/d2d/f76 [4194304,4194304] 0 2024-04-12T20:56:23.759 INFO:tasks.workunit.client.1.smithi186.stdout:3/97: write d1/fc [4864842,86945] 0 2024-04-12T20:56:23.759 INFO:tasks.workunit.client.0.smithi018.stdout:0/893: rmdir da/d15/d17/d21/d36/d6e 39 2024-04-12T20:56:23.764 INFO:tasks.workunit.client.0.smithi018.stdout:1/728: fdatasync d3/d9/da/d14/d73/fd0 0 2024-04-12T20:56:23.771 INFO:tasks.workunit.client.0.smithi018.stdout:1/729: mknod d3/d9/da/dd/d26/cf4 0 2024-04-12T20:56:23.772 INFO:tasks.workunit.client.1.smithi186.stdout:7/124: getdents d7/d10/d1c 0 2024-04-12T20:56:23.772 INFO:tasks.workunit.client.1.smithi186.stdout:7/125: read - d7/f8 zero size 2024-04-12T20:56:23.774 INFO:tasks.workunit.client.1.smithi186.stdout:1/102: truncate fb 829586 0 2024-04-12T20:56:23.774 INFO:tasks.workunit.client.0.smithi018.stdout:1/730: stat d3/d9/da/dd/f4a 0 2024-04-12T20:56:23.778 INFO:tasks.workunit.client.1.smithi186.stdout:1/103: mkdir d14/d1a/d21 0 2024-04-12T20:56:23.778 INFO:tasks.workunit.client.0.smithi018.stdout:0/894: truncate da/d15/d4d/fda 3458804 0 2024-04-12T20:56:23.781 INFO:tasks.workunit.client.1.smithi186.stdout:1/104: creat d14/f22 x:0 0 0 2024-04-12T20:56:23.781 INFO:tasks.workunit.client.1.smithi186.stdout:6/57: rmdir d9 39 2024-04-12T20:56:23.782 INFO:tasks.workunit.client.0.smithi018.stdout:1/731: mkdir d3/d9/da/dd/d31/d32/d13/d19/d2d/d4e/d7a/df5 0 2024-04-12T20:56:23.787 INFO:tasks.workunit.client.1.smithi186.stdout:7/126: dwrite d7/f16 [0,4194304] 0 2024-04-12T20:56:23.790 INFO:tasks.workunit.client.0.smithi018.stdout:2/817: sync 2024-04-12T20:56:23.792 INFO:tasks.workunit.client.1.smithi186.stdout:9/65: rmdir d3 39 2024-04-12T20:56:23.794 INFO:tasks.workunit.client.1.smithi186.stdout:4/146: write d0/dc/fe [1819757,46735] 0 2024-04-12T20:56:23.795 INFO:tasks.workunit.client.0.smithi018.stdout:1/732: link d3/d9/da/dd/d31/d32/d13/d19/d2d/d4e/d5e/d97/fe4 d3/d9/da/dd/d31/d32/d13/d19/d2d/d4e/d7a/d82/ff6 0 2024-04-12T20:56:23.799 INFO:tasks.workunit.client.1.smithi186.stdout:4/147: creat d0/dc/f2f x:0 0 0 2024-04-12T20:56:23.800 INFO:tasks.workunit.client.0.smithi018.stdout:0/895: dread da/d15/d17/d21/d36/f7c [0,4194304] 0 2024-04-12T20:56:23.800 INFO:tasks.workunit.client.0.smithi018.stdout:2/818: rename d5/d12/d54/c44 to d5/d8/d10/d43/de2/daf/d90/c11b 0 2024-04-12T20:56:23.806 INFO:tasks.workunit.client.1.smithi186.stdout:4/148: rmdir d0/d21 39 2024-04-12T20:56:23.806 INFO:tasks.workunit.client.1.smithi186.stdout:6/58: dwrite d9/ff [0,4194304] 0 2024-04-12T20:56:23.807 INFO:tasks.workunit.client.1.smithi186.stdout:4/149: read - d0/d12/f17 zero size 2024-04-12T20:56:23.807 INFO:tasks.workunit.client.1.smithi186.stdout:6/59: dread - d9/f10 zero size 2024-04-12T20:56:23.808 INFO:tasks.workunit.client.1.smithi186.stdout:0/138: getdents d0/d14 0 2024-04-12T20:56:23.810 INFO:tasks.workunit.client.1.smithi186.stdout:4/150: truncate d0/d26/f2b 452487 0 2024-04-12T20:56:23.811 INFO:tasks.workunit.client.0.smithi018.stdout:0/896: symlink da/d15/d17/d21/d36/d47/l12c 0 2024-04-12T20:56:23.811 INFO:tasks.workunit.client.1.smithi186.stdout:3/98: dread - d1/d3/f11 zero size 2024-04-12T20:56:23.811 INFO:tasks.workunit.client.1.smithi186.stdout:2/58: truncate d0/f2 7281230 0 2024-04-12T20:56:23.811 INFO:tasks.workunit.client.1.smithi186.stdout:0/139: write d0/f21 [996197,39972] 0 2024-04-12T20:56:23.812 INFO:tasks.workunit.client.1.smithi186.stdout:3/99: fdatasync d1/fc 0 2024-04-12T20:56:23.813 INFO:tasks.workunit.client.1.smithi186.stdout:0/140: readlink d0/d14/l29 0 2024-04-12T20:56:23.813 INFO:tasks.workunit.client.1.smithi186.stdout:6/60: rename d9/ff to d9/f12 0 2024-04-12T20:56:23.814 INFO:tasks.workunit.client.1.smithi186.stdout:3/100: dread - d1/d3/f11 zero size 2024-04-12T20:56:23.814 INFO:tasks.workunit.client.0.smithi018.stdout:1/733: rename d3/d9/da/dd/d31/d32/d33/d47/l98 to d3/d9/da/dd/d31/d32/d13/d19/d2d/d4e/d58/lf7 0 2024-04-12T20:56:23.817 INFO:tasks.workunit.client.1.smithi186.stdout:4/151: mknod d0/dc/c30 0 2024-04-12T20:56:23.817 INFO:tasks.workunit.client.0.smithi018.stdout:0/897: creat da/d15/d17/d5c/d79/dd6/f12d x:0 0 0 2024-04-12T20:56:23.817 INFO:tasks.workunit.client.0.smithi018.stdout:1/734: chown d3/d9/d11/l54 12387 1 2024-04-12T20:56:23.820 INFO:tasks.workunit.client.1.smithi186.stdout:2/59: creat d0/f10 x:0 0 0 2024-04-12T20:56:23.822 INFO:tasks.workunit.client.1.smithi186.stdout:8/49: write f5 [2390510,77637] 0 2024-04-12T20:56:23.824 INFO:tasks.workunit.client.1.smithi186.stdout:0/141: mknod d0/d14/c31 0 2024-04-12T20:56:23.826 INFO:tasks.workunit.client.0.smithi018.stdout:2/819: dwrite d5/d7/d79/f8c [4194304,4194304] 0 2024-04-12T20:56:23.829 INFO:tasks.workunit.client.1.smithi186.stdout:7/127: truncate d7/f28 674714 0 2024-04-12T20:56:23.829 INFO:tasks.workunit.client.1.smithi186.stdout:0/142: fdatasync d0/d10/d1c/d1e/f2f 0 2024-04-12T20:56:23.834 INFO:tasks.workunit.client.1.smithi186.stdout:1/105: dwrite fb [0,4194304] 0 2024-04-12T20:56:23.834 INFO:tasks.workunit.client.1.smithi186.stdout:0/143: write d0/d14/f19 [652765,59852] 0 2024-04-12T20:56:23.836 INFO:tasks.workunit.client.1.smithi186.stdout:6/61: rename d9/c11 to d9/c13 0 2024-04-12T20:56:23.837 INFO:tasks.workunit.client.1.smithi186.stdout:9/66: truncate f1 3417919 0 2024-04-12T20:56:23.838 INFO:tasks.workunit.client.1.smithi186.stdout:8/50: rmdir d7 39 2024-04-12T20:56:23.839 INFO:tasks.workunit.client.1.smithi186.stdout:6/62: readlink l4 0 2024-04-12T20:56:23.839 INFO:tasks.workunit.client.1.smithi186.stdout:5/55: dwrite d5/db/fc [0,4194304] 0 2024-04-12T20:56:23.840 INFO:tasks.workunit.client.0.smithi018.stdout:0/898: link da/d15/d17/d21/d36/d88/ddf/l102 da/d15/d17/d71/dc1/l12e 0 2024-04-12T20:56:23.841 INFO:tasks.workunit.client.1.smithi186.stdout:5/56: read d5/f6 [2979263,58621] 0 2024-04-12T20:56:23.842 INFO:tasks.workunit.client.1.smithi186.stdout:4/152: creat d0/d21/f31 x:0 0 0 2024-04-12T20:56:23.843 INFO:tasks.workunit.client.1.smithi186.stdout:3/101: unlink d1/d3/da/l9 0 2024-04-12T20:56:23.843 INFO:tasks.workunit.client.1.smithi186.stdout:5/57: read f4 [6580155,68796] 0 2024-04-12T20:56:23.845 INFO:tasks.workunit.client.1.smithi186.stdout:5/58: read f2 [6582283,57484] 0 2024-04-12T20:56:23.847 INFO:tasks.workunit.client.0.smithi018.stdout:0/899: mkdir da/d15/d4d/d12f 0 2024-04-12T20:56:23.847 INFO:tasks.workunit.client.0.smithi018.stdout:2/820: getdents d5/d8/d33 0 2024-04-12T20:56:23.849 INFO:tasks.workunit.client.0.smithi018.stdout:2/821: fsync d5/d7/fe0 0 2024-04-12T20:56:23.851 INFO:tasks.workunit.client.1.smithi186.stdout:8/51: read - d7/f9 zero size 2024-04-12T20:56:23.852 INFO:tasks.workunit.client.0.smithi018.stdout:2/822: chown d5/d8/d33/ce3 49991948 1 2024-04-12T20:56:23.855 INFO:tasks.workunit.client.1.smithi186.stdout:6/63: symlink d9/l14 0 2024-04-12T20:56:23.857 INFO:tasks.workunit.client.1.smithi186.stdout:3/102: mknod d1/d3/da/c17 0 2024-04-12T20:56:23.857 INFO:tasks.workunit.client.0.smithi018.stdout:2/823: truncate f4 1439221 0 2024-04-12T20:56:23.859 INFO:tasks.workunit.client.1.smithi186.stdout:5/59: chown d5/db/le 122215944 1 2024-04-12T20:56:23.860 INFO:tasks.workunit.client.1.smithi186.stdout:4/153: rename d0/d21/f1c to d0/d21/f32 0 2024-04-12T20:56:23.863 INFO:tasks.workunit.client.1.smithi186.stdout:4/154: chown d0/dc/f2d 4890121 1 2024-04-12T20:56:23.864 INFO:tasks.workunit.client.1.smithi186.stdout:7/128: symlink d7/d10/l33 0 2024-04-12T20:56:23.865 INFO:tasks.workunit.client.1.smithi186.stdout:2/60: write d0/f7 [4239936,42608] 0 2024-04-12T20:56:23.865 INFO:tasks.workunit.client.1.smithi186.stdout:4/155: fdatasync d0/f18 0 2024-04-12T20:56:23.865 INFO:tasks.workunit.client.1.smithi186.stdout:0/144: mkdir d0/d10/d32 0 2024-04-12T20:56:23.866 INFO:tasks.workunit.client.0.smithi018.stdout:0/900: symlink da/d15/d17/d21/d36/dce/d126/l130 0 2024-04-12T20:56:23.870 INFO:tasks.workunit.client.0.smithi018.stdout:0/901: chown da/d15/d17/d5c/d79/dd6/c124 1087476670 1 2024-04-12T20:56:23.874 INFO:tasks.workunit.client.1.smithi186.stdout:8/52: dwrite f2 [0,4194304] 0 2024-04-12T20:56:23.882 INFO:tasks.workunit.client.1.smithi186.stdout:7/129: dread d7/f16 [0,4194304] 0 2024-04-12T20:56:23.883 INFO:tasks.workunit.client.1.smithi186.stdout:4/156: dread d0/d12/f1a [0,4194304] 0 2024-04-12T20:56:23.884 INFO:tasks.workunit.client.1.smithi186.stdout:7/130: fsync d7/d10/d1c/f21 0 2024-04-12T20:56:23.891 INFO:tasks.workunit.client.0.smithi018.stdout:2/824: dwrite d5/d8/d10/d43/de2/f10c [0,4194304] 0 2024-04-12T20:56:23.895 INFO:tasks.workunit.client.1.smithi186.stdout:5/60: write d5/f6 [3233362,73250] 0 2024-04-12T20:56:23.899 INFO:tasks.workunit.client.0.smithi018.stdout:4/860: sync 2024-04-12T20:56:23.899 INFO:tasks.workunit.client.0.smithi018.stdout:7/892: sync 2024-04-12T20:56:23.901 INFO:tasks.workunit.client.1.smithi186.stdout:7/131: dwrite d7/d10/d1c/f2c [0,4194304] 0 2024-04-12T20:56:23.901 INFO:tasks.workunit.client.1.smithi186.stdout:2/61: mknod d0/dc/c11 0 2024-04-12T20:56:23.907 INFO:tasks.workunit.client.0.smithi018.stdout:0/902: chown da/f13 16558247 1 2024-04-12T20:56:23.908 INFO:tasks.workunit.client.0.smithi018.stdout:2/825: write d5/d8/d10/d43/de2/d24/d39/da4/fde [987683,32499] 0 2024-04-12T20:56:23.913 INFO:tasks.workunit.client.1.smithi186.stdout:0/145: creat d0/d10/d1c/f33 x:0 0 0 2024-04-12T20:56:23.914 INFO:tasks.workunit.client.0.smithi018.stdout:7/893: dread d1/db/d31/d30/fbf [0,4194304] 0 2024-04-12T20:56:23.914 INFO:tasks.workunit.client.1.smithi186.stdout:1/106: truncate fb 2286174 0 2024-04-12T20:56:23.916 INFO:tasks.workunit.client.1.smithi186.stdout:9/67: dwrite d3/f5 [0,4194304] 0 2024-04-12T20:56:23.917 INFO:tasks.workunit.client.1.smithi186.stdout:4/157: rename d0/d12 to d0/dc/d33 0 2024-04-12T20:56:23.918 INFO:tasks.workunit.client.1.smithi186.stdout:9/68: chown d3/c4 104 1 2024-04-12T20:56:23.919 INFO:tasks.workunit.client.1.smithi186.stdout:5/61: symlink d5/db/l12 0 2024-04-12T20:56:23.919 INFO:tasks.workunit.client.0.smithi018.stdout:2/826: creat d5/d8/d10/d43/f11c x:0 0 0 2024-04-12T20:56:23.920 INFO:tasks.workunit.client.1.smithi186.stdout:4/158: truncate d0/d21/f22 901550 0 2024-04-12T20:56:23.922 INFO:tasks.workunit.client.0.smithi018.stdout:0/903: symlink da/d15/d17/d21/d36/da2/dc3/d83/d8b/l131 0 2024-04-12T20:56:23.923 INFO:tasks.workunit.client.0.smithi018.stdout:7/894: symlink d1/db/d31/d3a/d97/l121 0 2024-04-12T20:56:23.923 INFO:tasks.workunit.client.1.smithi186.stdout:8/53: mknod d7/de/cf 0 2024-04-12T20:56:23.924 INFO:tasks.workunit.client.1.smithi186.stdout:0/146: chown d0/d10/d1c/d1e/f1f 254103 1 2024-04-12T20:56:23.925 INFO:tasks.workunit.client.0.smithi018.stdout:2/827: chown d5/d7/d31/d62/l116 35653161 1 2024-04-12T20:56:23.928 INFO:tasks.workunit.client.1.smithi186.stdout:5/62: symlink d5/db/l13 0 2024-04-12T20:56:23.928 INFO:tasks.workunit.client.1.smithi186.stdout:7/132: mkdir d7/d10/d2d/d34 0 2024-04-12T20:56:23.928 INFO:tasks.workunit.client.1.smithi186.stdout:4/159: creat d0/d26/f34 x:0 0 0 2024-04-12T20:56:23.928 INFO:tasks.workunit.client.1.smithi186.stdout:1/107: creat d14/d1a/d1d/f23 x:0 0 0 2024-04-12T20:56:23.929 INFO:tasks.workunit.client.1.smithi186.stdout:8/54: mkdir d7/de/d10 0 2024-04-12T20:56:23.929 INFO:tasks.workunit.client.1.smithi186.stdout:7/133: read - d7/f25 zero size 2024-04-12T20:56:23.931 INFO:tasks.workunit.client.1.smithi186.stdout:6/64: getdents d9 0 2024-04-12T20:56:23.932 INFO:tasks.workunit.client.0.smithi018.stdout:7/895: mkdir d1/db/d31/d3a/d42/d4c/d6c/dc3/d122 0 2024-04-12T20:56:23.934 INFO:tasks.workunit.client.0.smithi018.stdout:2/828: creat d5/d8/d10/d43/de2/d24/d39/dab/f11d x:0 0 0 2024-04-12T20:56:23.935 INFO:tasks.workunit.client.0.smithi018.stdout:7/896: chown d1/d6/fcb 3 1 2024-04-12T20:56:23.937 INFO:tasks.workunit.client.1.smithi186.stdout:5/63: mkdir d5/db/d14 0 2024-04-12T20:56:23.938 INFO:tasks.workunit.client.0.smithi018.stdout:7/897: stat d1/db/d31/d30/fbf 0 2024-04-12T20:56:23.938 INFO:tasks.workunit.client.1.smithi186.stdout:8/55: creat d7/de/f11 x:0 0 0 2024-04-12T20:56:23.939 INFO:tasks.workunit.client.1.smithi186.stdout:3/103: link d1/d3/l10 d1/l18 0 2024-04-12T20:56:23.940 INFO:tasks.workunit.client.0.smithi018.stdout:2/829: rmdir d5/d12 39 2024-04-12T20:56:23.941 INFO:tasks.workunit.client.1.smithi186.stdout:0/147: mknod d0/d10/d32/c34 0 2024-04-12T20:56:23.942 INFO:tasks.workunit.client.0.smithi018.stdout:2/830: chown d5/d7/d31/f7a 18 1 2024-04-12T20:56:23.943 INFO:tasks.workunit.client.1.smithi186.stdout:4/160: dread d0/d21/f32 [0,4194304] 0 2024-04-12T20:56:23.944 INFO:tasks.workunit.client.1.smithi186.stdout:4/161: fdatasync d0/dc/f2f 0 2024-04-12T20:56:23.945 INFO:tasks.workunit.client.0.smithi018.stdout:7/898: unlink d1/db/d31/d45/d8c/d9d/d69/f116 0 2024-04-12T20:56:23.945 INFO:tasks.workunit.client.1.smithi186.stdout:5/64: truncate f2 6032219 0 2024-04-12T20:56:23.947 INFO:tasks.workunit.client.1.smithi186.stdout:8/56: rename d7/f9 to d7/f12 0 2024-04-12T20:56:23.947 INFO:tasks.workunit.client.1.smithi186.stdout:1/108: dwrite d14/d1a/d1d/f23 [0,4194304] 0 2024-04-12T20:56:23.948 INFO:tasks.workunit.client.1.smithi186.stdout:5/65: write d5/f6 [3063923,113661] 0 2024-04-12T20:56:23.948 INFO:tasks.workunit.client.1.smithi186.stdout:8/57: fdatasync f5 0 2024-04-12T20:56:23.950 INFO:tasks.workunit.client.0.smithi018.stdout:4/861: write d3/d5d/fae [232562,38388] 0 2024-04-12T20:56:23.952 INFO:tasks.workunit.client.1.smithi186.stdout:0/148: chown d0/c9 2908712 1 2024-04-12T20:56:23.952 INFO:tasks.workunit.client.0.smithi018.stdout:2/831: fdatasync d5/d8/d10/d43/de2/d7c/f9f 0 2024-04-12T20:56:23.953 INFO:tasks.workunit.client.1.smithi186.stdout:7/134: rmdir d7/d10/d2d/d34 0 2024-04-12T20:56:23.954 INFO:tasks.workunit.client.1.smithi186.stdout:7/135: chown c2 30502 1 2024-04-12T20:56:23.955 INFO:tasks.workunit.client.0.smithi018.stdout:7/899: creat d1/db/d31/d45/d53/d11b/f123 x:0 0 0 2024-04-12T20:56:23.960 INFO:tasks.workunit.client.1.smithi186.stdout:4/162: symlink d0/dc/d33/l35 0 2024-04-12T20:56:23.961 INFO:tasks.workunit.client.0.smithi018.stdout:7/900: write d1/db/d31/d3a/d42/d4c/ff8 [644096,39920] 0 2024-04-12T20:56:23.964 INFO:tasks.workunit.client.0.smithi018.stdout:0/904: dread da/f25 [0,4194304] 0 2024-04-12T20:56:23.966 INFO:tasks.workunit.client.0.smithi018.stdout:2/832: rename d5/d7/d31/f4f to d5/d12/d54/d3e/f11e 0 2024-04-12T20:56:23.967 INFO:tasks.workunit.client.1.smithi186.stdout:5/66: mkdir d5/db/d15 0 2024-04-12T20:56:23.976 INFO:tasks.workunit.client.1.smithi186.stdout:3/104: creat d1/f19 x:0 0 0 2024-04-12T20:56:23.977 INFO:tasks.workunit.client.1.smithi186.stdout:3/105: dread - d1/d3/f11 zero size 2024-04-12T20:56:23.978 INFO:tasks.workunit.client.0.smithi018.stdout:2/833: mknod d5/d12/d54/dc0/c11f 0 2024-04-12T20:56:23.981 INFO:tasks.workunit.client.0.smithi018.stdout:4/862: dwrite d3/d5d/d60/dc5/d3d/d80/feb [4194304,4194304] 0 2024-04-12T20:56:23.982 INFO:tasks.workunit.client.1.smithi186.stdout:1/109: dread d14/f15 [0,4194304] 0 2024-04-12T20:56:23.984 INFO:tasks.workunit.client.1.smithi186.stdout:1/110: read - d14/f22 zero size 2024-04-12T20:56:23.987 INFO:tasks.workunit.client.0.smithi018.stdout:0/905: dread da/d15/d17/d21/d36/da2/dc3/d83/f11c [0,4194304] 0 2024-04-12T20:56:23.988 INFO:tasks.workunit.client.1.smithi186.stdout:5/67: stat f4 0 2024-04-12T20:56:23.988 INFO:tasks.workunit.client.0.smithi018.stdout:7/901: rename d1/db/d31/d45/d53 to d1/db/d31/d45/df4/d124 0 2024-04-12T20:56:23.989 INFO:tasks.workunit.client.1.smithi186.stdout:5/68: readlink d5/db/lf 0 2024-04-12T20:56:23.989 INFO:tasks.workunit.client.0.smithi018.stdout:4/863: chown d3/d32/d33/l28 75887 1 2024-04-12T20:56:23.990 INFO:tasks.workunit.client.1.smithi186.stdout:0/149: getdents d0/d10/d1c 0 2024-04-12T20:56:23.991 INFO:tasks.workunit.client.0.smithi018.stdout:0/906: dread - da/d15/d17/d5c/f116 zero size 2024-04-12T20:56:23.993 INFO:tasks.workunit.client.0.smithi018.stdout:2/834: getdents d5/d8 0 2024-04-12T20:56:23.994 INFO:tasks.workunit.client.0.smithi018.stdout:0/907: rename da/d15 to da/d15/d17/d21/d36/dce/d126/d132 22 2024-04-12T20:56:23.995 INFO:tasks.workunit.client.0.smithi018.stdout:7/902: write d1/db/d31/d45/df4/d124/f7d [1038365,6900] 0 2024-04-12T20:56:23.997 INFO:tasks.workunit.client.1.smithi186.stdout:3/106: mknod d1/c1a 0 2024-04-12T20:56:23.998 INFO:tasks.workunit.client.1.smithi186.stdout:3/107: stat d1/d3/da/ld 0 2024-04-12T20:56:23.999 INFO:tasks.workunit.client.1.smithi186.stdout:1/111: symlink d14/d1a/d1d/l24 0 2024-04-12T20:56:24.005 INFO:tasks.workunit.client.1.smithi186.stdout:9/69: truncate d3/f5 3389712 0 2024-04-12T20:56:24.009 INFO:tasks.workunit.client.0.smithi018.stdout:4/864: link d3/d5d/d60/dc5/d43/d4b/de6/c107 d3/d5d/d60/c120 0 2024-04-12T20:56:24.011 INFO:tasks.workunit.client.1.smithi186.stdout:9/70: unlink d3/c4 0 2024-04-12T20:56:24.011 INFO:tasks.workunit.client.1.smithi186.stdout:8/58: rmdir d7/de 39 2024-04-12T20:56:24.011 INFO:tasks.workunit.client.1.smithi186.stdout:2/62: sync 2024-04-12T20:56:24.011 INFO:tasks.workunit.client.0.smithi018.stdout:0/908: write da/d15/d17/fc6 [194319,126892] 0 2024-04-12T20:56:24.012 INFO:tasks.workunit.client.0.smithi018.stdout:2/835: truncate d5/d8/d10/d43/f56 456726 0 2024-04-12T20:56:24.012 INFO:tasks.workunit.client.1.smithi186.stdout:5/69: rename f4 to d5/db/d14/f16 0 2024-04-12T20:56:24.016 INFO:tasks.workunit.client.1.smithi186.stdout:2/63: write d0/f4 [161513,85324] 0 2024-04-12T20:56:24.017 INFO:tasks.workunit.client.0.smithi018.stdout:4/865: truncate d3/d5d/d60/dc5/fba 756591 0 2024-04-12T20:56:24.017 INFO:tasks.workunit.client.1.smithi186.stdout:3/108: dwrite d1/f19 [0,4194304] 0 2024-04-12T20:56:24.018 INFO:tasks.workunit.client.1.smithi186.stdout:6/65: truncate d9/f12 1895505 0 2024-04-12T20:56:24.019 INFO:tasks.workunit.client.1.smithi186.stdout:6/66: stat d9/lc 0 2024-04-12T20:56:24.020 INFO:tasks.workunit.client.0.smithi018.stdout:0/909: creat da/d15/d17/d71/dc1/dee/df7/dea/f133 x:0 0 0 2024-04-12T20:56:24.020 INFO:tasks.workunit.client.1.smithi186.stdout:9/71: symlink d3/lc 0 2024-04-12T20:56:24.021 INFO:tasks.workunit.client.1.smithi186.stdout:6/67: readlink d9/l14 0 2024-04-12T20:56:24.024 INFO:tasks.workunit.client.1.smithi186.stdout:7/136: rmdir d7 39 2024-04-12T20:56:24.024 INFO:tasks.workunit.client.1.smithi186.stdout:0/150: rename d0/f2b to d0/d14/f35 0 2024-04-12T20:56:24.024 INFO:tasks.workunit.client.0.smithi018.stdout:7/903: dwrite d1/db/d31/d45/d8c/d9d/d69/f7e [0,4194304] 0 2024-04-12T20:56:24.026 INFO:tasks.workunit.client.1.smithi186.stdout:8/59: creat d7/de/f13 x:0 0 0 2024-04-12T20:56:24.027 INFO:tasks.workunit.client.0.smithi018.stdout:4/866: getdents d3/db/df6 0 2024-04-12T20:56:24.028 INFO:tasks.workunit.client.1.smithi186.stdout:8/60: write f2 [2161446,79620] 0 2024-04-12T20:56:24.029 INFO:tasks.workunit.client.1.smithi186.stdout:8/61: fdatasync d7/de/f13 0 2024-04-12T20:56:24.030 INFO:tasks.workunit.client.0.smithi018.stdout:7/904: chown d1/db/d31/d45/df4/d124/f56 0 1 2024-04-12T20:56:24.034 INFO:tasks.workunit.client.0.smithi018.stdout:4/867: creat d3/d32/d94/d10b/f121 x:0 0 0 2024-04-12T20:56:24.035 INFO:tasks.workunit.client.1.smithi186.stdout:9/72: creat d3/fd x:0 0 0 2024-04-12T20:56:24.040 INFO:tasks.workunit.client.0.smithi018.stdout:7/905: dread d1/db/d31/d45/d8c/d9d/fe5 [0,4194304] 0 2024-04-12T20:56:24.041 INFO:tasks.workunit.client.0.smithi018.stdout:0/910: write da/d15/d17/d21/f46 [4414755,1847] 0 2024-04-12T20:56:24.043 INFO:tasks.workunit.client.1.smithi186.stdout:1/112: rename l11 to d14/d1a/d1d/l25 0 2024-04-12T20:56:24.045 INFO:tasks.workunit.client.0.smithi018.stdout:4/868: creat d3/d5d/d60/dc5/d43/f122 x:0 0 0 2024-04-12T20:56:24.046 INFO:tasks.workunit.client.1.smithi186.stdout:0/151: symlink d0/d10/d32/l36 0 2024-04-12T20:56:24.048 INFO:tasks.workunit.client.1.smithi186.stdout:0/152: chown d0/c8 5219806 1 2024-04-12T20:56:24.048 INFO:tasks.workunit.client.1.smithi186.stdout:4/163: dwrite d0/fd [0,4194304] 0 2024-04-12T20:56:24.048 INFO:tasks.workunit.client.1.smithi186.stdout:8/62: creat d7/de/f14 x:0 0 0 2024-04-12T20:56:24.048 INFO:tasks.workunit.client.0.smithi018.stdout:0/911: chown da/d15/d17/d21/d94/fa4 2771929 1 2024-04-12T20:56:24.049 INFO:tasks.workunit.client.1.smithi186.stdout:4/164: stat d0/d26/f2b 0 2024-04-12T20:56:24.055 INFO:tasks.workunit.client.0.smithi018.stdout:2/836: dwrite d5/d8/d10/d43/de2/daf/d90/f9d [0,4194304] 0 2024-04-12T20:56:24.056 INFO:tasks.workunit.client.1.smithi186.stdout:9/73: creat d3/fe x:0 0 0 2024-04-12T20:56:24.060 INFO:tasks.workunit.client.0.smithi018.stdout:2/837: creat d5/d8/f120 x:0 0 0 2024-04-12T20:56:24.063 INFO:tasks.workunit.client.1.smithi186.stdout:7/137: unlink d7/d10/l23 0 2024-04-12T20:56:24.064 INFO:tasks.workunit.client.1.smithi186.stdout:7/138: readlink d7/d10/l33 0 2024-04-12T20:56:24.065 INFO:tasks.workunit.client.1.smithi186.stdout:6/68: read d9/f12 [623781,127660] 0 2024-04-12T20:56:24.068 INFO:tasks.workunit.client.1.smithi186.stdout:7/139: write d7/d10/f15 [3282017,42840] 0 2024-04-12T20:56:24.070 INFO:tasks.workunit.client.1.smithi186.stdout:8/63: fdatasync d7/f12 0 2024-04-12T20:56:24.072 INFO:tasks.workunit.client.1.smithi186.stdout:8/64: write d7/fd [762009,22634] 0 2024-04-12T20:56:24.074 INFO:tasks.workunit.client.0.smithi018.stdout:4/869: dwrite d3/d5d/d60/dc5/d3d/d80/f8c [0,4194304] 0 2024-04-12T20:56:24.074 INFO:tasks.workunit.client.0.smithi018.stdout:7/906: write d1/db/d31/d45/df4/d124/f56 [4736688,87706] 0 2024-04-12T20:56:24.075 INFO:tasks.workunit.client.1.smithi186.stdout:9/74: creat d3/ff x:0 0 0 2024-04-12T20:56:24.076 INFO:tasks.workunit.client.1.smithi186.stdout:1/113: symlink d14/d1a/d21/l26 0 2024-04-12T20:56:24.078 INFO:tasks.workunit.client.1.smithi186.stdout:2/64: getdents d0/d3 0 2024-04-12T20:56:24.079 INFO:tasks.workunit.client.0.smithi018.stdout:4/870: readlink d3/d32/d33/l45 0 2024-04-12T20:56:24.083 INFO:tasks.workunit.client.1.smithi186.stdout:0/153: link d0/f21 d0/d10/d1c/d1e/f37 0 2024-04-12T20:56:24.083 INFO:tasks.workunit.client.1.smithi186.stdout:3/109: link d1/d3/da/l5 d1/d3/da/l1b 0 2024-04-12T20:56:24.083 INFO:tasks.workunit.client.0.smithi018.stdout:7/907: mknod d1/c125 0 2024-04-12T20:56:24.095 INFO:tasks.workunit.client.1.smithi186.stdout:2/65: getdents d0 0 2024-04-12T20:56:24.100 INFO:tasks.workunit.client.0.smithi018.stdout:0/912: truncate da/d15/d17/d21/d36/dce/f106 3916634 0 2024-04-12T20:56:24.101 INFO:tasks.workunit.client.0.smithi018.stdout:7/908: write d1/db/d31/d3a/d42/d4c/d6c/fa6 [1914582,47930] 0 2024-04-12T20:56:24.101 INFO:tasks.workunit.client.0.smithi018.stdout:4/871: dwrite d3/d5d/d60/dc5/d43/f7b [0,4194304] 0 2024-04-12T20:56:24.104 INFO:tasks.workunit.client.1.smithi186.stdout:4/165: write d0/dc/f23 [381837,87131] 0 2024-04-12T20:56:24.104 INFO:tasks.workunit.client.0.smithi018.stdout:2/838: symlink d5/d8/d10/d43/d8f/l121 0 2024-04-12T20:56:24.106 INFO:tasks.workunit.client.0.smithi018.stdout:4/872: write d3/d5d/d60/dc5/d3d/d80/ff7 [30461,105] 0 2024-04-12T20:56:24.106 INFO:tasks.workunit.client.0.smithi018.stdout:2/839: readlink d5/d12/l74 0 2024-04-12T20:56:24.107 INFO:tasks.workunit.client.1.smithi186.stdout:2/66: truncate d0/f2 1636154 0 2024-04-12T20:56:24.107 INFO:tasks.workunit.client.1.smithi186.stdout:7/140: write d7/f12 [699329,4730] 0 2024-04-12T20:56:24.109 INFO:tasks.workunit.client.1.smithi186.stdout:6/69: truncate d9/fb 1768933 0 2024-04-12T20:56:24.111 INFO:tasks.workunit.client.0.smithi018.stdout:0/913: mkdir da/d15/de8/d134 0 2024-04-12T20:56:24.113 INFO:tasks.workunit.client.1.smithi186.stdout:8/65: truncate f2 1756426 0 2024-04-12T20:56:24.114 INFO:tasks.workunit.client.1.smithi186.stdout:9/75: dwrite d3/f5 [0,4194304] 0 2024-04-12T20:56:24.114 INFO:tasks.workunit.client.1.smithi186.stdout:4/166: stat d0/dc/d33/fb 0 2024-04-12T20:56:24.114 INFO:tasks.workunit.client.1.smithi186.stdout:1/114: truncate d14/d1a/f1b 191943 0 2024-04-12T20:56:24.115 INFO:tasks.workunit.client.0.smithi018.stdout:7/909: mkdir d1/db/d31/d45/d126 0 2024-04-12T20:56:24.117 INFO:tasks.workunit.client.1.smithi186.stdout:3/110: dwrite d1/f19 [4194304,4194304] 0 2024-04-12T20:56:24.119 INFO:tasks.workunit.client.1.smithi186.stdout:8/66: dread - d7/de/f13 zero size 2024-04-12T20:56:24.121 INFO:tasks.workunit.client.1.smithi186.stdout:0/154: dwrite d0/d10/f2a [0,4194304] 0 2024-04-12T20:56:24.123 INFO:tasks.workunit.client.0.smithi018.stdout:0/914: symlink da/d15/d17/d21/d36/da2/dc3/da9/l135 0 2024-04-12T20:56:24.124 INFO:tasks.workunit.client.1.smithi186.stdout:0/155: dread - d0/d14/f24 zero size 2024-04-12T20:56:24.124 INFO:tasks.workunit.client.1.smithi186.stdout:2/67: rmdir d0 39 2024-04-12T20:56:24.131 INFO:tasks.workunit.client.1.smithi186.stdout:4/167: creat d0/dc/d33/f36 x:0 0 0 2024-04-12T20:56:24.135 INFO:tasks.workunit.client.1.smithi186.stdout:7/141: symlink d7/d10/l35 0 2024-04-12T20:56:24.139 INFO:tasks.workunit.client.0.smithi018.stdout:7/910: write d1/db/d31/d45/d8c/d9d/d69/fd7 [213335,95341] 0 2024-04-12T20:56:24.140 INFO:tasks.workunit.client.1.smithi186.stdout:0/156: mknod d0/d10/d32/c38 0 2024-04-12T20:56:24.142 INFO:tasks.workunit.client.0.smithi018.stdout:7/911: chown d1/c125 72028 1 2024-04-12T20:56:24.144 INFO:tasks.workunit.client.1.smithi186.stdout:3/111: mknod d1/c1c 0 2024-04-12T20:56:24.146 INFO:tasks.workunit.client.1.smithi186.stdout:5/70: dwrite d5/db/d14/f16 [0,4194304] 0 2024-04-12T20:56:24.148 INFO:tasks.workunit.client.0.smithi018.stdout:2/840: dwrite d5/d12/fe8 [0,4194304] 0 2024-04-12T20:56:24.151 INFO:tasks.workunit.client.0.smithi018.stdout:7/912: creat d1/db/d31/d45/d8c/d9d/f127 x:0 0 0 2024-04-12T20:56:24.153 INFO:tasks.workunit.client.1.smithi186.stdout:0/157: stat d0/d14/f35 0 2024-04-12T20:56:24.154 INFO:tasks.workunit.client.0.smithi018.stdout:2/841: fsync d5/d8/d10/d43/de2/d24/d39/da4/fc8 0 2024-04-12T20:56:24.154 INFO:tasks.workunit.client.0.smithi018.stdout:0/915: write da/d15/d4d/d58/f65 [517828,107331] 0 2024-04-12T20:56:24.155 INFO:tasks.workunit.client.1.smithi186.stdout:6/70: link c8 d9/c15 0 2024-04-12T20:56:24.158 INFO:tasks.workunit.client.0.smithi018.stdout:7/913: mknod d1/d6/dc9/c128 0 2024-04-12T20:56:24.164 INFO:tasks.workunit.client.1.smithi186.stdout:8/67: dwrite d7/d8/fa [0,4194304] 0 2024-04-12T20:56:24.164 INFO:tasks.workunit.client.1.smithi186.stdout:9/76: dwrite d3/fb [0,4194304] 0 2024-04-12T20:56:24.169 INFO:tasks.workunit.client.1.smithi186.stdout:8/68: write d7/d8/fa [3159675,56871] 0 2024-04-12T20:56:24.169 INFO:tasks.workunit.client.1.smithi186.stdout:7/142: dread d7/d10/f13 [0,4194304] 0 2024-04-12T20:56:24.171 INFO:tasks.workunit.client.1.smithi186.stdout:8/69: dread - d7/de/f11 zero size 2024-04-12T20:56:24.172 INFO:tasks.workunit.client.1.smithi186.stdout:1/115: dwrite f4 [0,4194304] 0 2024-04-12T20:56:24.172 INFO:tasks.workunit.client.1.smithi186.stdout:8/70: chown d7/de/d10 7578284 1 2024-04-12T20:56:24.176 INFO:tasks.workunit.client.1.smithi186.stdout:4/168: creat d0/dc/d1f/f37 x:0 0 0 2024-04-12T20:56:24.177 INFO:tasks.workunit.client.1.smithi186.stdout:5/71: symlink d5/db/d14/l17 0 2024-04-12T20:56:24.179 INFO:tasks.workunit.client.1.smithi186.stdout:4/169: write d0/f24 [893488,102059] 0 2024-04-12T20:56:24.181 INFO:tasks.workunit.client.1.smithi186.stdout:4/170: stat d0/dc/d1f/f37 0 2024-04-12T20:56:24.183 INFO:tasks.workunit.client.1.smithi186.stdout:3/112: mknod d1/d3/c1d 0 2024-04-12T20:56:24.184 INFO:tasks.workunit.client.0.smithi018.stdout:7/914: mknod d1/db/d31/d3a/d42/d4c/dbc/c129 0 2024-04-12T20:56:24.184 INFO:tasks.workunit.client.0.smithi018.stdout:2/842: truncate d5/d8/d10/d43/de2/d24/d39/de6/f115 861336 0 2024-04-12T20:56:24.184 INFO:tasks.workunit.client.1.smithi186.stdout:6/71: symlink d9/l16 0 2024-04-12T20:56:24.185 INFO:tasks.workunit.client.1.smithi186.stdout:9/77: creat d3/f10 x:0 0 0 2024-04-12T20:56:24.188 INFO:tasks.workunit.client.1.smithi186.stdout:9/78: write d3/f5 [1120277,61422] 0 2024-04-12T20:56:24.196 INFO:tasks.workunit.client.0.smithi018.stdout:0/916: dwrite da/d15/d17/d71/dc1/ff1 [0,4194304] 0 2024-04-12T20:56:24.196 INFO:tasks.workunit.client.0.smithi018.stdout:2/843: readlink d5/d8/d10/d43/de2/d24/l10a 0 2024-04-12T20:56:24.203 INFO:tasks.workunit.client.0.smithi018.stdout:2/844: creat d5/d7/d31/d62/f122 x:0 0 0 2024-04-12T20:56:24.206 INFO:tasks.workunit.client.1.smithi186.stdout:1/116: creat d14/d1a/d21/f27 x:0 0 0 2024-04-12T20:56:24.207 INFO:tasks.workunit.client.1.smithi186.stdout:7/143: rename d7/d10/d1c/l31 to d7/d10/d1c/l36 0 2024-04-12T20:56:24.208 INFO:tasks.workunit.client.1.smithi186.stdout:4/171: creat d0/d21/f38 x:0 0 0 2024-04-12T20:56:24.209 INFO:tasks.workunit.client.1.smithi186.stdout:7/144: fsync d7/f12 0 2024-04-12T20:56:24.209 INFO:tasks.workunit.client.1.smithi186.stdout:0/158: rename d0/c4 to d0/d14/c39 0 2024-04-12T20:56:24.209 INFO:tasks.workunit.client.1.smithi186.stdout:6/72: symlink d9/l17 0 2024-04-12T20:56:24.212 INFO:tasks.workunit.client.1.smithi186.stdout:9/79: creat d3/f11 x:0 0 0 2024-04-12T20:56:24.212 INFO:tasks.workunit.client.1.smithi186.stdout:4/172: write d0/d21/f20 [831156,72627] 0 2024-04-12T20:56:24.212 INFO:tasks.workunit.client.0.smithi018.stdout:2/845: rename d5/d7/d31/d62/da6/ffe to d5/d7/d79/f123 0 2024-04-12T20:56:24.213 INFO:tasks.workunit.client.1.smithi186.stdout:5/72: mkdir d5/db/d15/d18 0 2024-04-12T20:56:24.213 INFO:tasks.workunit.client.1.smithi186.stdout:9/80: fsync d3/f10 0 2024-04-12T20:56:24.214 INFO:tasks.workunit.client.1.smithi186.stdout:4/173: chown d0/d26/f34 3935 1 2024-04-12T20:56:24.214 INFO:tasks.workunit.client.0.smithi018.stdout:0/917: truncate da/d15/d17/d21/d36/d88/d117/f11a 3275005 0 2024-04-12T20:56:24.215 INFO:tasks.workunit.client.1.smithi186.stdout:5/73: write d5/f6 [1623724,70642] 0 2024-04-12T20:56:24.216 INFO:tasks.workunit.client.1.smithi186.stdout:4/174: fdatasync d0/d21/f22 0 2024-04-12T20:56:24.216 INFO:tasks.workunit.client.1.smithi186.stdout:5/74: write f2 [3319152,13742] 0 2024-04-12T20:56:24.218 INFO:tasks.workunit.client.0.smithi018.stdout:7/915: write d1/db/d31/d30/f57 [3714624,123238] 0 2024-04-12T20:56:24.218 INFO:tasks.workunit.client.1.smithi186.stdout:6/73: dread d9/f12 [0,4194304] 0 2024-04-12T20:56:24.219 INFO:tasks.workunit.client.1.smithi186.stdout:1/117: mknod d14/d1a/c28 0 2024-04-12T20:56:24.221 INFO:tasks.workunit.client.0.smithi018.stdout:0/918: creat da/d15/d17/d71/dc1/dee/df7/dea/f136 x:0 0 0 2024-04-12T20:56:24.221 INFO:tasks.workunit.client.1.smithi186.stdout:7/145: rmdir d7 39 2024-04-12T20:56:24.222 INFO:tasks.workunit.client.1.smithi186.stdout:0/159: rmdir d0/d14 39 2024-04-12T20:56:24.223 INFO:tasks.workunit.client.0.smithi018.stdout:7/916: write d1/db/d31/d3a/d42/d4c/d6c/fa6 [1347350,77683] 0 2024-04-12T20:56:24.224 INFO:tasks.workunit.client.1.smithi186.stdout:0/160: write d0/d10/d1c/d1e/f2f [1018860,62074] 0 2024-04-12T20:56:24.224 INFO:tasks.workunit.client.1.smithi186.stdout:5/75: fdatasync d5/db/d14/f16 0 2024-04-12T20:56:24.225 INFO:tasks.workunit.client.1.smithi186.stdout:9/81: mkdir d3/d12 0 2024-04-12T20:56:24.229 INFO:tasks.workunit.client.0.smithi018.stdout:7/917: dread d1/db/d6f/fe0 [0,4194304] 0 2024-04-12T20:56:24.230 INFO:tasks.workunit.client.1.smithi186.stdout:4/175: fdatasync d0/dc/f2d 0 2024-04-12T20:56:24.232 INFO:tasks.workunit.client.1.smithi186.stdout:6/74: rename d9/l17 to d9/l18 0 2024-04-12T20:56:24.233 INFO:tasks.workunit.client.1.smithi186.stdout:6/75: fdatasync d9/fe 0 2024-04-12T20:56:24.233 INFO:tasks.workunit.client.1.smithi186.stdout:3/113: creat d1/f1e x:0 0 0 2024-04-12T20:56:24.234 INFO:tasks.workunit.client.0.smithi018.stdout:0/919: write da/d15/f90 [567608,125501] 0 2024-04-12T20:56:24.238 INFO:tasks.workunit.client.1.smithi186.stdout:4/176: mkdir d0/dc/d39 0 2024-04-12T20:56:24.241 INFO:tasks.workunit.client.0.smithi018.stdout:0/920: getdents da/d15/d17/d21/d36/d88 0 2024-04-12T20:56:24.248 INFO:tasks.workunit.client.0.smithi018.stdout:0/921: creat da/d15/d17/d21/d36/d47/f137 x:0 0 0 2024-04-12T20:56:24.250 INFO:tasks.workunit.client.0.smithi018.stdout:7/918: dwrite d1/fba [8388608,4194304] 0 2024-04-12T20:56:24.251 INFO:tasks.workunit.client.0.smithi018.stdout:0/922: creat da/d15/d4d/d58/f138 x:0 0 0 2024-04-12T20:56:24.254 INFO:tasks.workunit.client.0.smithi018.stdout:7/919: fdatasync d1/db/d31/d45/d8c/d9d/d70/d96/f68 0 2024-04-12T20:56:24.257 INFO:tasks.workunit.client.0.smithi018.stdout:0/923: mknod da/d15/d17/d21/d36/d91/c139 0 2024-04-12T20:56:24.265 INFO:tasks.workunit.client.0.smithi018.stdout:0/924: write da/d15/d17/d5c/d79/f86 [147188,120975] 0 2024-04-12T20:56:24.269 INFO:tasks.workunit.client.0.smithi018.stdout:7/920: dread d1/db/fef [0,4194304] 0 2024-04-12T20:56:24.270 INFO:tasks.workunit.client.0.smithi018.stdout:7/921: fsync d1/db/f9f 0 2024-04-12T20:56:24.272 INFO:tasks.workunit.client.0.smithi018.stdout:7/922: dread d1/db/d6f/fe0 [0,4194304] 0 2024-04-12T20:56:24.273 INFO:tasks.workunit.client.1.smithi186.stdout:0/161: symlink d0/d10/l3a 0 2024-04-12T20:56:24.273 INFO:tasks.workunit.client.1.smithi186.stdout:3/114: readlink d1/d3/l10 0 2024-04-12T20:56:24.273 INFO:tasks.workunit.client.0.smithi018.stdout:7/923: readlink d1/d6/l1c 0 2024-04-12T20:56:24.275 INFO:tasks.workunit.client.1.smithi186.stdout:1/118: truncate f4 964944 0 2024-04-12T20:56:24.278 INFO:tasks.workunit.client.1.smithi186.stdout:2/68: dwrite d0/f2 [0,4194304] 0 2024-04-12T20:56:24.280 INFO:tasks.workunit.client.1.smithi186.stdout:3/115: write d1/f1e [223169,53803] 0 2024-04-12T20:56:24.282 INFO:tasks.workunit.client.1.smithi186.stdout:1/119: chown d14/d1a/c28 67219 1 2024-04-12T20:56:24.282 INFO:tasks.workunit.client.1.smithi186.stdout:7/146: getdents d7 0 2024-04-12T20:56:24.283 INFO:tasks.workunit.client.1.smithi186.stdout:1/120: readlink l13 0 2024-04-12T20:56:24.284 INFO:tasks.workunit.client.1.smithi186.stdout:7/147: chown d7/f17 1350091 1 2024-04-12T20:56:24.286 INFO:tasks.workunit.client.1.smithi186.stdout:2/69: mkdir d0/d3/d12 0 2024-04-12T20:56:24.287 INFO:tasks.workunit.client.1.smithi186.stdout:7/148: chown d7/fc 70113 1 2024-04-12T20:56:24.287 INFO:tasks.workunit.client.1.smithi186.stdout:4/177: truncate d0/dc/d33/f7 427054 0 2024-04-12T20:56:24.288 INFO:tasks.workunit.client.1.smithi186.stdout:6/76: dwrite f6 [4194304,4194304] 0 2024-04-12T20:56:24.290 INFO:tasks.workunit.client.0.smithi018.stdout:7/924: dread d1/db/d31/d45/d8c/d9d/d70/d96/d5f/da9/ddf/f10c [0,4194304] 0 2024-04-12T20:56:24.290 INFO:tasks.workunit.client.1.smithi186.stdout:4/178: dread - d0/dc/f1d zero size 2024-04-12T20:56:24.290 INFO:tasks.workunit.client.1.smithi186.stdout:1/121: dread d14/d1a/f1b [0,4194304] 0 2024-04-12T20:56:24.291 INFO:tasks.workunit.client.1.smithi186.stdout:1/122: chown f9 14 1 2024-04-12T20:56:24.292 INFO:tasks.workunit.client.1.smithi186.stdout:4/179: chown d0/dc/f2d 14533 1 2024-04-12T20:56:24.296 INFO:tasks.workunit.client.0.smithi018.stdout:0/925: dread da/d15/d17/d71/dc1/dcc/f78 [0,4194304] 0 2024-04-12T20:56:24.297 INFO:tasks.workunit.client.1.smithi186.stdout:2/70: rmdir d0 39 2024-04-12T20:56:24.297 INFO:tasks.workunit.client.1.smithi186.stdout:4/180: dread - d0/d21/f27 zero size 2024-04-12T20:56:24.297 INFO:tasks.workunit.client.1.smithi186.stdout:1/123: write d14/d1a/d21/f27 [730985,95150] 0 2024-04-12T20:56:24.297 INFO:tasks.workunit.client.1.smithi186.stdout:6/77: symlink d9/l19 0 2024-04-12T20:56:24.298 INFO:tasks.workunit.client.1.smithi186.stdout:1/124: chown l13 357 1 2024-04-12T20:56:24.299 INFO:tasks.workunit.client.1.smithi186.stdout:3/116: dread d1/f1e [0,4194304] 0 2024-04-12T20:56:24.300 INFO:tasks.workunit.client.0.smithi018.stdout:0/926: chown da/d15/d17/d21/d36/da2/dc3/d83 3780277 1 2024-04-12T20:56:24.300 INFO:tasks.workunit.client.1.smithi186.stdout:4/181: rename d0/f18 to d0/dc/d39/f3a 0 2024-04-12T20:56:24.301 INFO:tasks.workunit.client.0.smithi018.stdout:7/925: rename d1/db/d31/d3a/d42 to d1/db/d31/d30/dc6/d100/d12a 0 2024-04-12T20:56:24.302 INFO:tasks.workunit.client.1.smithi186.stdout:3/117: read d1/f19 [2166254,59327] 0 2024-04-12T20:56:24.302 INFO:tasks.workunit.client.1.smithi186.stdout:6/78: mknod d9/c1a 0 2024-04-12T20:56:24.302 INFO:tasks.workunit.client.0.smithi018.stdout:7/926: write d1/f1b [3878559,92547] 0 2024-04-12T20:56:24.303 INFO:tasks.workunit.client.1.smithi186.stdout:7/149: getdents d7/d10 0 2024-04-12T20:56:24.303 INFO:tasks.workunit.client.1.smithi186.stdout:2/71: truncate d0/d3/f8 4285063 0 2024-04-12T20:56:24.304 INFO:tasks.workunit.client.0.smithi018.stdout:0/927: mkdir da/d15/d17/d108/d128/d13a 0 2024-04-12T20:56:24.304 INFO:tasks.workunit.client.1.smithi186.stdout:8/71: sync 2024-04-12T20:56:24.306 INFO:tasks.workunit.client.1.smithi186.stdout:6/79: chown d9/lc 143 1 2024-04-12T20:56:24.307 INFO:tasks.workunit.client.1.smithi186.stdout:4/182: link d0/fd d0/dc/f3b 0 2024-04-12T20:56:24.308 INFO:tasks.workunit.client.0.smithi018.stdout:7/927: creat d1/db/d31/d30/dc6/d100/d12a/f12b x:0 0 0 2024-04-12T20:56:24.309 INFO:tasks.workunit.client.0.smithi018.stdout:7/928: fdatasync d1/d6/f102 0 2024-04-12T20:56:24.310 INFO:tasks.workunit.client.0.smithi018.stdout:7/929: chown d1/d86/la4 3 1 2024-04-12T20:56:24.311 INFO:tasks.workunit.client.1.smithi186.stdout:6/80: creat d9/f1b x:0 0 0 2024-04-12T20:56:24.316 INFO:tasks.workunit.client.1.smithi186.stdout:2/72: symlink d0/d3/d12/l13 0 2024-04-12T20:56:24.318 INFO:tasks.workunit.client.1.smithi186.stdout:4/183: symlink d0/dc/l3c 0 2024-04-12T20:56:24.322 INFO:tasks.workunit.client.1.smithi186.stdout:6/81: symlink d9/l1c 0 2024-04-12T20:56:24.322 INFO:tasks.workunit.client.1.smithi186.stdout:1/125: dwrite fb [0,4194304] 0 2024-04-12T20:56:24.322 INFO:tasks.workunit.client.1.smithi186.stdout:6/82: chown d9/f12 22563872 1 2024-04-12T20:56:24.324 INFO:tasks.workunit.client.0.smithi018.stdout:0/928: dwrite da/d15/d17/d21/d94/fa4 [0,4194304] 0 2024-04-12T20:56:24.324 INFO:tasks.workunit.client.1.smithi186.stdout:2/73: rename d0/dc/ff to d0/f14 0 2024-04-12T20:56:24.325 INFO:tasks.workunit.client.1.smithi186.stdout:4/184: mkdir d0/d21/d3d 0 2024-04-12T20:56:24.327 INFO:tasks.workunit.client.1.smithi186.stdout:1/126: creat d14/d1a/d1d/f29 x:0 0 0 2024-04-12T20:56:24.329 INFO:tasks.workunit.client.1.smithi186.stdout:6/83: creat d9/f1d x:0 0 0 2024-04-12T20:56:24.329 INFO:tasks.workunit.client.1.smithi186.stdout:6/84: chown l7 347327724 1 2024-04-12T20:56:24.331 INFO:tasks.workunit.client.1.smithi186.stdout:6/85: write d9/f10 [968620,8410] 0 2024-04-12T20:56:24.333 INFO:tasks.workunit.client.0.smithi018.stdout:0/929: mknod da/d15/d17/d21/d36/d47/c13b 0 2024-04-12T20:56:24.333 INFO:tasks.workunit.client.1.smithi186.stdout:8/72: write f5 [1923985,43912] 0 2024-04-12T20:56:24.336 INFO:tasks.workunit.client.1.smithi186.stdout:1/127: fdatasync ff 0 2024-04-12T20:56:24.336 INFO:tasks.workunit.client.1.smithi186.stdout:8/73: write d7/fd [300088,46537] 0 2024-04-12T20:56:24.339 INFO:tasks.workunit.client.1.smithi186.stdout:1/128: rename d14 to d14/d1a/d2a 22 2024-04-12T20:56:24.342 INFO:tasks.workunit.client.1.smithi186.stdout:1/129: dread d14/d1a/f1b [0,4194304] 0 2024-04-12T20:56:24.342 INFO:tasks.workunit.client.1.smithi186.stdout:8/74: creat d7/de/d10/f15 x:0 0 0 2024-04-12T20:56:24.343 INFO:tasks.workunit.client.1.smithi186.stdout:6/86: link d9/l1c d9/l1e 0 2024-04-12T20:56:24.344 INFO:tasks.workunit.client.0.smithi018.stdout:7/930: dread d1/db/d31/d3a/d97/fb1 [0,4194304] 0 2024-04-12T20:56:24.345 INFO:tasks.workunit.client.1.smithi186.stdout:8/75: chown d7/fd 95550515 1 2024-04-12T20:56:24.346 INFO:tasks.workunit.client.1.smithi186.stdout:4/185: dwrite d0/dc/f1d [0,4194304] 0 2024-04-12T20:56:24.346 INFO:tasks.workunit.client.1.smithi186.stdout:6/87: symlink d9/l1f 0 2024-04-12T20:56:24.349 INFO:tasks.workunit.client.1.smithi186.stdout:2/74: fsync d0/f14 0 2024-04-12T20:56:24.353 INFO:tasks.workunit.client.1.smithi186.stdout:2/75: symlink d0/dc/l15 0 2024-04-12T20:56:24.355 INFO:tasks.workunit.client.1.smithi186.stdout:2/76: chown d0/f6 142464264 1 2024-04-12T20:56:24.356 INFO:tasks.workunit.client.1.smithi186.stdout:8/76: getdents d7 0 2024-04-12T20:56:24.359 INFO:tasks.workunit.client.1.smithi186.stdout:8/77: write d7/de/f11 [998902,66862] 0 2024-04-12T20:56:24.361 INFO:tasks.workunit.client.0.smithi018.stdout:0/930: dread da/d15/d17/d5c/f64 [0,4194304] 0 2024-04-12T20:56:24.362 INFO:tasks.workunit.client.1.smithi186.stdout:2/77: rename d0/d3/d12/l13 to d0/d3/l16 0 2024-04-12T20:56:24.364 INFO:tasks.workunit.client.1.smithi186.stdout:4/186: truncate d0/dc/d39/f3a 183881 0 2024-04-12T20:56:24.364 INFO:tasks.workunit.client.1.smithi186.stdout:6/88: dwrite d9/f1b [0,4194304] 0 2024-04-12T20:56:24.365 INFO:tasks.workunit.client.1.smithi186.stdout:2/78: write d0/d3/fe [698839,44892] 0 2024-04-12T20:56:24.370 INFO:tasks.workunit.client.0.smithi018.stdout:0/931: creat da/d15/d17/d71/dc1/dcc/f13c x:0 0 0 2024-04-12T20:56:24.370 INFO:tasks.workunit.client.1.smithi186.stdout:4/187: symlink d0/dc/d1f/l3e 0 2024-04-12T20:56:24.371 INFO:tasks.workunit.client.0.smithi018.stdout:0/932: chown l7 209 1 2024-04-12T20:56:24.372 INFO:tasks.workunit.client.1.smithi186.stdout:2/79: mkdir d0/d3/d17 0 2024-04-12T20:56:24.372 INFO:tasks.workunit.client.0.smithi018.stdout:7/931: dwrite d1/db/d31/d45/d8c/d9d/d70/d96/d5f/f117 [0,4194304] 0 2024-04-12T20:56:24.375 INFO:tasks.workunit.client.1.smithi186.stdout:4/188: chown d0/dc/d33/f7 14321560 1 2024-04-12T20:56:24.378 INFO:tasks.workunit.client.0.smithi018.stdout:7/932: stat d1/db/d31/d30/dc6/d100/d12a/d4c 0 2024-04-12T20:56:24.378 INFO:tasks.workunit.client.0.smithi018.stdout:0/933: chown da/d15/d17/l89 500546576 1 2024-04-12T20:56:24.379 INFO:tasks.workunit.client.1.smithi186.stdout:2/80: mknod d0/c18 0 2024-04-12T20:56:24.381 INFO:tasks.workunit.client.0.smithi018.stdout:0/934: chown da/d15/d4d/la8 5 1 2024-04-12T20:56:24.381 INFO:tasks.workunit.client.1.smithi186.stdout:6/89: link d9/c15 d9/c20 0 2024-04-12T20:56:24.382 INFO:tasks.workunit.client.0.smithi018.stdout:7/933: write d1/d6/fa0 [408755,89951] 0 2024-04-12T20:56:24.386 INFO:tasks.workunit.client.1.smithi186.stdout:6/90: symlink d9/l21 0 2024-04-12T20:56:24.389 INFO:tasks.workunit.client.1.smithi186.stdout:6/91: truncate d9/f1d 975176 0 2024-04-12T20:56:24.389 INFO:tasks.workunit.client.0.smithi018.stdout:7/934: mkdir d1/db/d31/d45/d12c 0 2024-04-12T20:56:24.391 INFO:tasks.workunit.client.1.smithi186.stdout:8/78: dread d7/fd [0,4194304] 0 2024-04-12T20:56:24.400 INFO:tasks.workunit.client.0.smithi018.stdout:0/935: dwrite da/d15/f41 [0,4194304] 0 2024-04-12T20:56:24.401 INFO:tasks.workunit.client.1.smithi186.stdout:4/189: write d0/dc/d33/f1a [1306210,98978] 0 2024-04-12T20:56:24.403 INFO:tasks.workunit.client.1.smithi186.stdout:6/92: getdents d9 0 2024-04-12T20:56:24.405 INFO:tasks.workunit.client.1.smithi186.stdout:2/81: truncate d0/d3/f8 2913879 0 2024-04-12T20:56:24.405 INFO:tasks.workunit.client.0.smithi018.stdout:0/936: mkdir da/d15/d17/d71/dc1/dee/d13d 0 2024-04-12T20:56:24.409 INFO:tasks.workunit.client.1.smithi186.stdout:2/82: creat d0/dc/f19 x:0 0 0 2024-04-12T20:56:24.410 INFO:tasks.workunit.client.1.smithi186.stdout:6/93: dread d9/f10 [0,4194304] 0 2024-04-12T20:56:24.414 INFO:tasks.workunit.client.0.smithi018.stdout:0/937: getdents da/d15/d17/d21/d36/d47/db5 0 2024-04-12T20:56:24.417 INFO:tasks.workunit.client.0.smithi018.stdout:0/938: fsync da/d15/d17/d21/d36/da2/dc3/d3f/f5b 0 2024-04-12T20:56:24.421 INFO:tasks.workunit.client.0.smithi018.stdout:0/939: dread - da/d15/d17/d21/d36/d88/ddf/f111 zero size 2024-04-12T20:56:24.423 INFO:tasks.workunit.client.1.smithi186.stdout:2/83: getdents d0/dc 0 2024-04-12T20:56:24.424 INFO:tasks.workunit.client.1.smithi186.stdout:2/84: stat d0/c5 0 2024-04-12T20:56:24.424 INFO:tasks.workunit.client.0.smithi018.stdout:0/940: creat da/d15/d17/d71/dc1/dcc/f13e x:0 0 0 2024-04-12T20:56:24.429 INFO:tasks.workunit.client.1.smithi186.stdout:2/85: unlink d0/dc/f19 0 2024-04-12T20:56:24.433 INFO:tasks.workunit.client.1.smithi186.stdout:2/86: mkdir d0/d3/d1a 0 2024-04-12T20:56:24.435 INFO:tasks.workunit.client.1.smithi186.stdout:4/190: dwrite d0/dc/f2d [0,4194304] 0 2024-04-12T20:56:24.437 INFO:tasks.workunit.client.0.smithi018.stdout:0/941: dread da/d15/d17/d21/d36/dce/ff6 [0,4194304] 0 2024-04-12T20:56:24.437 INFO:tasks.workunit.client.1.smithi186.stdout:2/87: write d0/f7 [1229412,22953] 0 2024-04-12T20:56:24.441 INFO:tasks.workunit.client.0.smithi018.stdout:0/942: readlink da/d15/d17/d21/d94/l9f 0 2024-04-12T20:56:24.443 INFO:tasks.workunit.client.0.smithi018.stdout:0/943: write da/d15/d4d/d6b/f6d [1757878,13089] 0 2024-04-12T20:56:24.447 INFO:tasks.workunit.client.0.smithi018.stdout:0/944: chown da/d15/d17/d21/d36/d91/lf0 31837 1 2024-04-12T20:56:24.451 INFO:tasks.workunit.client.0.smithi018.stdout:1/735: sync 2024-04-12T20:56:24.451 INFO:tasks.workunit.client.0.smithi018.stdout:0/945: creat da/d15/d17/d71/dc1/dee/d13d/f13f x:0 0 0 2024-04-12T20:56:24.453 INFO:tasks.workunit.client.0.smithi018.stdout:0/946: chown da/d15/d17/d71/dc1/dcc/dcb 3190 1 2024-04-12T20:56:24.454 INFO:tasks.workunit.client.0.smithi018.stdout:1/736: rmdir d3/d9/da/dd/d31/d32/d33/d47/db0 39 2024-04-12T20:56:24.461 INFO:tasks.workunit.client.0.smithi018.stdout:0/947: write da/d15/d17/d21/d36/d47/f5d [468732,82585] 0 2024-04-12T20:56:24.462 INFO:tasks.workunit.client.0.smithi018.stdout:0/948: fdatasync da/fc 0 2024-04-12T20:56:24.463 INFO:tasks.workunit.client.1.smithi186.stdout:2/88: dwrite d0/f4 [0,4194304] 0 2024-04-12T20:56:24.464 INFO:tasks.workunit.client.0.smithi018.stdout:0/949: fsync da/d15/d17/d71/dc1/dee/df7/dea/f136 0 2024-04-12T20:56:24.464 INFO:tasks.workunit.client.1.smithi186.stdout:4/191: dwrite d0/dc/f2d [4194304,4194304] 0 2024-04-12T20:56:24.465 INFO:tasks.workunit.client.0.smithi018.stdout:0/950: fsync da/d15/d17/d21/d94/fa4 0 2024-04-12T20:56:24.470 INFO:tasks.workunit.client.1.smithi186.stdout:4/192: truncate d0/dc/d33/f4 710381 0 2024-04-12T20:56:24.475 INFO:tasks.workunit.client.1.smithi186.stdout:2/89: rmdir d0/d3/d17 0 2024-04-12T20:56:24.476 INFO:tasks.workunit.client.1.smithi186.stdout:4/193: mknod d0/dc/d39/c3f 0 2024-04-12T20:56:24.479 INFO:tasks.workunit.client.1.smithi186.stdout:4/194: creat d0/d21/f40 x:0 0 0 2024-04-12T20:56:24.484 INFO:tasks.workunit.client.0.smithi018.stdout:0/951: write da/d15/d17/d21/d36/d47/d81/fec [1183549,108336] 0 2024-04-12T20:56:24.490 INFO:tasks.workunit.client.1.smithi186.stdout:4/195: write d0/fd [4115395,74061] 0 2024-04-12T20:56:24.493 INFO:tasks.workunit.client.1.smithi186.stdout:4/196: rmdir d0/dc/d1f 39 2024-04-12T20:56:24.496 INFO:tasks.workunit.client.1.smithi186.stdout:4/197: truncate d0/dc/d1f/f37 878905 0 2024-04-12T20:56:24.502 INFO:tasks.workunit.client.0.smithi018.stdout:0/952: dwrite da/d15/d17/d21/d36/f80 [0,4194304] 0 2024-04-12T20:56:24.504 INFO:tasks.workunit.client.0.smithi018.stdout:0/953: chown da/d15/d4d/d6b/lbf 44483 1 2024-04-12T20:56:24.507 INFO:tasks.workunit.client.0.smithi018.stdout:0/954: dread - da/d15/d17/d71/dc1/dee/df7/dea/f100 zero size 2024-04-12T20:56:24.517 INFO:tasks.workunit.client.1.smithi186.stdout:4/198: dwrite d0/f6 [0,4194304] 0 2024-04-12T20:56:24.518 INFO:tasks.workunit.client.0.smithi018.stdout:0/955: dread da/d15/d4d/f85 [4194304,4194304] 0 2024-04-12T20:56:24.519 INFO:tasks.workunit.client.1.smithi186.stdout:4/199: creat d0/d21/f41 x:0 0 0 2024-04-12T20:56:24.521 INFO:tasks.workunit.client.0.smithi018.stdout:0/956: rmdir da/d15/d17/d21/d36/da2/dc3/da9 39 2024-04-12T20:56:24.523 INFO:tasks.workunit.client.1.smithi186.stdout:4/200: unlink d0/d21/f22 0 2024-04-12T20:56:24.523 INFO:tasks.workunit.client.1.smithi186.stdout:5/76: sync 2024-04-12T20:56:24.527 INFO:tasks.workunit.client.1.smithi186.stdout:5/77: symlink d5/db/l19 0 2024-04-12T20:56:24.528 INFO:tasks.workunit.client.1.smithi186.stdout:4/201: creat d0/f42 x:0 0 0 2024-04-12T20:56:24.528 INFO:tasks.workunit.client.0.smithi018.stdout:0/957: rename da/d15/d17/d21/d36/da2/dc3/d83 to da/d15/d17/d71/dc1/dee/df7/d140 0 2024-04-12T20:56:24.531 INFO:tasks.workunit.client.1.smithi186.stdout:4/202: creat d0/dc/d33/f43 x:0 0 0 2024-04-12T20:56:24.532 INFO:tasks.workunit.client.0.smithi018.stdout:0/958: creat da/d15/d17/d21/d36/da2/dc3/d3f/f141 x:0 0 0 2024-04-12T20:56:24.533 INFO:tasks.workunit.client.1.smithi186.stdout:4/203: readlink d0/l2a 0 2024-04-12T20:56:24.534 INFO:tasks.workunit.client.1.smithi186.stdout:4/204: write d0/dc/d1f/f37 [672384,58950] 0 2024-04-12T20:56:24.536 INFO:tasks.workunit.client.1.smithi186.stdout:4/205: read d0/fd [2124028,35367] 0 2024-04-12T20:56:24.537 INFO:tasks.workunit.client.0.smithi018.stdout:0/959: unlink da/d15/d17/d21/d36/da2/dc3/d3f/f69 0 2024-04-12T20:56:24.542 INFO:tasks.workunit.client.1.smithi186.stdout:5/78: dread d5/db/fc [0,4194304] 0 2024-04-12T20:56:24.546 INFO:tasks.workunit.client.1.smithi186.stdout:5/79: write d5/f6 [4775277,37429] 0 2024-04-12T20:56:24.549 INFO:tasks.workunit.client.1.smithi186.stdout:4/206: write d0/dc/d39/f3a [975036,60218] 0 2024-04-12T20:56:24.550 INFO:tasks.workunit.client.1.smithi186.stdout:5/80: creat d5/db/d14/f1a x:0 0 0 2024-04-12T20:56:24.552 INFO:tasks.workunit.client.1.smithi186.stdout:4/207: mknod d0/dc/d39/c44 0 2024-04-12T20:56:24.553 INFO:tasks.workunit.client.0.smithi018.stdout:0/960: dwrite da/d15/d17/d21/d36/fb0 [4194304,4194304] 0 2024-04-12T20:56:24.554 INFO:tasks.workunit.client.1.smithi186.stdout:4/208: write d0/d21/f27 [265092,26188] 0 2024-04-12T20:56:24.556 INFO:tasks.workunit.client.1.smithi186.stdout:5/81: getdents d5/db 0 2024-04-12T20:56:24.558 INFO:tasks.workunit.client.0.smithi018.stdout:0/961: truncate da/d15/d4d/d58/f65 1223111 0 2024-04-12T20:56:24.559 INFO:tasks.workunit.client.1.smithi186.stdout:5/82: symlink d5/db/d14/l1b 0 2024-04-12T20:56:24.560 INFO:tasks.workunit.client.1.smithi186.stdout:5/83: dread - d5/fd zero size 2024-04-12T20:56:24.561 INFO:tasks.workunit.client.0.smithi018.stdout:0/962: fsync da/d15/d17/d71/dc1/dee/f104 0 2024-04-12T20:56:24.562 INFO:tasks.workunit.client.1.smithi186.stdout:5/84: rmdir d5/db/d15 39 2024-04-12T20:56:24.563 INFO:tasks.workunit.client.0.smithi018.stdout:0/963: readlink da/d15/d17/d21/d36/da2/dc3/d3f/lbd 0 2024-04-12T20:56:24.564 INFO:tasks.workunit.client.1.smithi186.stdout:5/85: chown d5/db/d14/f1a 24928 1 2024-04-12T20:56:24.567 INFO:tasks.workunit.client.1.smithi186.stdout:5/86: mknod d5/c1c 0 2024-04-12T20:56:24.568 INFO:tasks.workunit.client.0.smithi018.stdout:0/964: getdents da/d15/d17/d10d 0 2024-04-12T20:56:24.577 INFO:tasks.workunit.client.0.smithi018.stdout:0/965: link da/d15/d17/d21/c7a da/d15/d17/d21/d36/d47/c142 0 2024-04-12T20:56:24.596 INFO:tasks.workunit.client.1.smithi186.stdout:4/209: dwrite d0/d21/f31 [0,4194304] 0 2024-04-12T20:56:24.599 INFO:tasks.workunit.client.1.smithi186.stdout:4/210: dread - d0/d21/f41 zero size 2024-04-12T20:56:24.612 INFO:tasks.workunit.client.1.smithi186.stdout:9/82: sync 2024-04-12T20:56:24.613 INFO:tasks.workunit.client.1.smithi186.stdout:9/83: stat d3/f11 0 2024-04-12T20:56:24.661 INFO:tasks.workunit.client.1.smithi186.stdout:0/162: sync 2024-04-12T20:56:24.661 INFO:tasks.workunit.client.1.smithi186.stdout:7/150: sync 2024-04-12T20:56:24.661 INFO:tasks.workunit.client.1.smithi186.stdout:1/130: sync 2024-04-12T20:56:24.662 INFO:tasks.workunit.client.1.smithi186.stdout:3/118: sync 2024-04-12T20:56:24.662 INFO:tasks.workunit.client.1.smithi186.stdout:2/90: sync 2024-04-12T20:56:24.662 INFO:tasks.workunit.client.1.smithi186.stdout:6/94: sync 2024-04-12T20:56:24.662 INFO:tasks.workunit.client.1.smithi186.stdout:8/79: sync 2024-04-12T20:56:24.667 INFO:tasks.workunit.client.1.smithi186.stdout:4/211: mknod d0/dc/d39/c45 0 2024-04-12T20:56:24.667 INFO:tasks.workunit.client.1.smithi186.stdout:2/91: rename d0 to d0/dc/d1b 22 2024-04-12T20:56:24.667 INFO:tasks.workunit.client.1.smithi186.stdout:9/84: mkdir d3/d13 0 2024-04-12T20:56:24.667 INFO:tasks.workunit.client.1.smithi186.stdout:8/80: dread - d7/de/d10/f15 zero size 2024-04-12T20:56:24.667 INFO:tasks.workunit.client.1.smithi186.stdout:0/163: fsync d0/d10/d1c/d1e/f2f 0 2024-04-12T20:56:24.670 INFO:tasks.workunit.client.1.smithi186.stdout:7/151: creat d7/d10/d2d/f37 x:0 0 0 2024-04-12T20:56:24.671 INFO:tasks.workunit.client.1.smithi186.stdout:5/87: link d5/db/l12 d5/db/l1d 0 2024-04-12T20:56:24.673 INFO:tasks.workunit.client.1.smithi186.stdout:2/92: mkdir d0/dc/d1c 0 2024-04-12T20:56:24.677 INFO:tasks.workunit.client.1.smithi186.stdout:4/212: truncate d0/dc/f3b 2295626 0 2024-04-12T20:56:24.686 INFO:tasks.workunit.client.1.smithi186.stdout:2/93: symlink d0/l1d 0 2024-04-12T20:56:24.687 INFO:tasks.workunit.client.1.smithi186.stdout:3/119: creat d1/d3/f1f x:0 0 0 2024-04-12T20:56:24.691 INFO:tasks.workunit.client.1.smithi186.stdout:2/94: dread - d0/f14 zero size 2024-04-12T20:56:24.694 INFO:tasks.workunit.client.1.smithi186.stdout:0/164: unlink d0/d14/c30 0 2024-04-12T20:56:24.697 INFO:tasks.workunit.client.1.smithi186.stdout:5/88: creat d5/db/d15/f1e x:0 0 0 2024-04-12T20:56:24.702 INFO:tasks.workunit.client.1.smithi186.stdout:1/131: dwrite fe [0,4194304] 0 2024-04-12T20:56:24.705 INFO:tasks.workunit.client.1.smithi186.stdout:9/85: dwrite d3/fd [0,4194304] 0 2024-04-12T20:56:24.705 INFO:tasks.workunit.client.1.smithi186.stdout:2/95: rename d0/dc/d1c to d0/d3/d1a/d1e 0 2024-04-12T20:56:24.705 INFO:tasks.workunit.client.1.smithi186.stdout:5/89: write d5/db/f11 [831795,111015] 0 2024-04-12T20:56:24.709 INFO:tasks.workunit.client.1.smithi186.stdout:3/120: mknod d1/d3/da/c20 0 2024-04-12T20:56:24.710 INFO:tasks.workunit.client.1.smithi186.stdout:4/213: dwrite d0/f6 [0,4194304] 0 2024-04-12T20:56:24.712 INFO:tasks.workunit.client.1.smithi186.stdout:0/165: symlink d0/l3b 0 2024-04-12T20:56:24.712 INFO:tasks.workunit.client.1.smithi186.stdout:5/90: write d5/db/d14/f1a [745443,14443] 0 2024-04-12T20:56:24.713 INFO:tasks.workunit.client.1.smithi186.stdout:4/214: read - d0/d21/f41 zero size 2024-04-12T20:56:24.714 INFO:tasks.workunit.client.1.smithi186.stdout:1/132: symlink d14/d1a/d21/l2b 0 2024-04-12T20:56:24.714 INFO:tasks.workunit.client.1.smithi186.stdout:0/166: dread - d0/d10/f2c zero size 2024-04-12T20:56:24.716 INFO:tasks.workunit.client.1.smithi186.stdout:5/91: read d5/f9 [3362472,34633] 0 2024-04-12T20:56:24.718 INFO:tasks.workunit.client.1.smithi186.stdout:3/121: write d1/f1e [981066,825] 0 2024-04-12T20:56:24.719 INFO:tasks.workunit.client.1.smithi186.stdout:0/167: write d0/d10/d1c/f33 [602922,127625] 0 2024-04-12T20:56:24.719 INFO:tasks.workunit.client.1.smithi186.stdout:4/215: creat d0/dc/d33/f46 x:0 0 0 2024-04-12T20:56:24.721 INFO:tasks.workunit.client.1.smithi186.stdout:5/92: rename d5/db to d5/db/d15/d1f 22 2024-04-12T20:56:24.726 INFO:tasks.workunit.client.1.smithi186.stdout:3/122: mkdir d1/d3/d21 0 2024-04-12T20:56:24.727 INFO:tasks.workunit.client.1.smithi186.stdout:5/93: creat d5/f20 x:0 0 0 2024-04-12T20:56:24.729 INFO:tasks.workunit.client.1.smithi186.stdout:4/216: creat d0/dc/d33/f47 x:0 0 0 2024-04-12T20:56:24.731 INFO:tasks.workunit.client.1.smithi186.stdout:5/94: chown d5/c1c 245496708 1 2024-04-12T20:56:24.731 INFO:tasks.workunit.client.1.smithi186.stdout:9/86: dread d3/fd [0,4194304] 0 2024-04-12T20:56:24.734 INFO:tasks.workunit.client.1.smithi186.stdout:9/87: chown d3/d12 791 1 2024-04-12T20:56:24.735 INFO:tasks.workunit.client.1.smithi186.stdout:3/123: mkdir d1/d3/d21/d22 0 2024-04-12T20:56:24.737 INFO:tasks.workunit.client.1.smithi186.stdout:9/88: fdatasync d3/f11 0 2024-04-12T20:56:24.739 INFO:tasks.workunit.client.1.smithi186.stdout:5/95: unlink d5/db/l19 0 2024-04-12T20:56:24.739 INFO:tasks.workunit.client.1.smithi186.stdout:0/168: dread d0/d10/d1c/d1e/f2f [0,4194304] 0 2024-04-12T20:56:24.740 INFO:tasks.workunit.client.1.smithi186.stdout:9/89: write d3/fe [536600,27496] 0 2024-04-12T20:56:24.741 INFO:tasks.workunit.client.1.smithi186.stdout:2/96: dwrite d0/f4 [0,4194304] 0 2024-04-12T20:56:24.742 INFO:tasks.workunit.client.1.smithi186.stdout:5/96: chown d5/f6 1335 1 2024-04-12T20:56:24.745 INFO:tasks.workunit.client.1.smithi186.stdout:0/169: write d0/d10/f12 [2220981,43460] 0 2024-04-12T20:56:24.747 INFO:tasks.workunit.client.1.smithi186.stdout:2/97: creat d0/d3/f1f x:0 0 0 2024-04-12T20:56:24.750 INFO:tasks.workunit.client.1.smithi186.stdout:0/170: mknod d0/d10/d1c/d1e/c3c 0 2024-04-12T20:56:24.756 INFO:tasks.workunit.client.1.smithi186.stdout:3/124: dread d1/f19 [4194304,4194304] 0 2024-04-12T20:56:24.757 INFO:tasks.workunit.client.1.smithi186.stdout:2/98: link d0/f6 d0/d3/d1a/f20 0 2024-04-12T20:56:24.760 INFO:tasks.workunit.client.1.smithi186.stdout:3/125: symlink d1/d3/d21/l23 0 2024-04-12T20:56:24.764 INFO:tasks.workunit.client.1.smithi186.stdout:2/99: mkdir d0/d3/d1a/d21 0 2024-04-12T20:56:24.768 INFO:tasks.workunit.client.1.smithi186.stdout:2/100: stat d0/cd 0 2024-04-12T20:56:24.771 INFO:tasks.workunit.client.1.smithi186.stdout:3/126: chown d1/d3/da/l1b 7771294 1 2024-04-12T20:56:24.772 INFO:tasks.workunit.client.1.smithi186.stdout:9/90: dwrite d3/f10 [0,4194304] 0 2024-04-12T20:56:24.772 INFO:tasks.workunit.client.1.smithi186.stdout:2/101: dread - d0/f10 zero size 2024-04-12T20:56:24.774 INFO:tasks.workunit.client.1.smithi186.stdout:5/97: getdents d5/db 0 2024-04-12T20:56:24.777 INFO:tasks.workunit.client.1.smithi186.stdout:4/217: write d0/fd [1870401,62337] 0 2024-04-12T20:56:24.781 INFO:tasks.workunit.client.1.smithi186.stdout:7/152: dwrite d7/f17 [4194304,4194304] 0 2024-04-12T20:56:24.781 INFO:tasks.workunit.client.1.smithi186.stdout:3/127: mkdir d1/d3/d21/d22/d24 0 2024-04-12T20:56:24.804 INFO:tasks.workunit.client.0.smithi018.stdout:2/846: sync 2024-04-12T20:56:24.805 INFO:tasks.workunit.client.0.smithi018.stdout:4/873: sync 2024-04-12T20:56:24.811 INFO:tasks.workunit.client.0.smithi018.stdout:2/847: fdatasync d5/fc6 0 2024-04-12T20:56:24.812 INFO:tasks.workunit.client.0.smithi018.stdout:4/874: mkdir d3/d32/d33/d8e/d123 0 2024-04-12T20:56:24.814 INFO:tasks.workunit.client.0.smithi018.stdout:2/848: symlink d5/d8/d10/d43/l124 0 2024-04-12T20:56:24.819 INFO:tasks.workunit.client.0.smithi018.stdout:2/849: chown d5/d8/d10/d43/de2/d24/d39/fa1 5 1 2024-04-12T20:56:24.821 INFO:tasks.workunit.client.1.smithi186.stdout:2/102: rename d0/d3/d1a/d1e to d0/d3/d1a/d21/d22 0 2024-04-12T20:56:24.821 INFO:tasks.workunit.client.1.smithi186.stdout:5/98: getdents d5/db 0 2024-04-12T20:56:24.824 INFO:tasks.workunit.client.1.smithi186.stdout:5/99: write f2 [5422088,49967] 0 2024-04-12T20:56:24.825 INFO:tasks.workunit.client.1.smithi186.stdout:2/103: symlink d0/l23 0 2024-04-12T20:56:24.827 INFO:tasks.workunit.client.1.smithi186.stdout:3/128: mknod d1/d3/d21/d22/d24/c25 0 2024-04-12T20:56:24.829 INFO:tasks.workunit.client.0.smithi018.stdout:2/850: unlink d5/d8/d10/d43/de2/l71 0 2024-04-12T20:56:24.829 INFO:tasks.workunit.client.1.smithi186.stdout:2/104: write d0/f10 [548838,35036] 0 2024-04-12T20:56:24.829 INFO:tasks.workunit.client.1.smithi186.stdout:0/171: dwrite d0/d10/d1c/d1e/f1f [0,4194304] 0 2024-04-12T20:56:24.830 INFO:tasks.workunit.client.1.smithi186.stdout:3/129: write d1/f1e [1819522,8483] 0 2024-04-12T20:56:24.832 INFO:tasks.workunit.client.1.smithi186.stdout:0/172: write d0/f6 [5925800,39691] 0 2024-04-12T20:56:24.832 INFO:tasks.workunit.client.1.smithi186.stdout:2/105: chown d0/d3/d1a/f20 27310016 1 2024-04-12T20:56:24.835 INFO:tasks.workunit.client.1.smithi186.stdout:3/130: read - d1/d3/f11 zero size 2024-04-12T20:56:24.837 INFO:tasks.workunit.client.0.smithi018.stdout:2/851: rename d5/d8/d10/d43/de2/d24/d39/de6 to d5/d7/d79/d125 0 2024-04-12T20:56:24.837 INFO:tasks.workunit.client.0.smithi018.stdout:4/875: dwrite d3/d5d/d60/d64/d72/f89 [0,4194304] 0 2024-04-12T20:56:24.839 INFO:tasks.workunit.client.1.smithi186.stdout:0/173: unlink d0/d10/c1b 0 2024-04-12T20:56:24.840 INFO:tasks.workunit.client.1.smithi186.stdout:2/106: mkdir d0/d3/d1a/d24 0 2024-04-12T20:56:24.840 INFO:tasks.workunit.client.1.smithi186.stdout:0/174: fsync d0/d10/f2a 0 2024-04-12T20:56:24.841 INFO:tasks.workunit.client.1.smithi186.stdout:0/175: chown d0/c2d 1 1 2024-04-12T20:56:24.844 INFO:tasks.workunit.client.0.smithi018.stdout:2/852: mkdir d5/d7/d31/d11a/d126 0 2024-04-12T20:56:24.846 INFO:tasks.workunit.client.0.smithi018.stdout:4/876: mknod d3/d5d/d60/d64/d4f/dc7/c124 0 2024-04-12T20:56:24.847 INFO:tasks.workunit.client.1.smithi186.stdout:0/176: chown d0/cb 5011 1 2024-04-12T20:56:24.848 INFO:tasks.workunit.client.1.smithi186.stdout:5/100: truncate d5/db/f11 363252 0 2024-04-12T20:56:24.850 INFO:tasks.workunit.client.0.smithi018.stdout:4/877: mknod d3/d5d/d60/dc5/d43/d4b/ddc/c125 0 2024-04-12T20:56:24.853 INFO:tasks.workunit.client.1.smithi186.stdout:0/177: write d0/d10/d1c/f28 [336127,13685] 0 2024-04-12T20:56:24.854 INFO:tasks.workunit.client.1.smithi186.stdout:5/101: mknod d5/db/d15/c21 0 2024-04-12T20:56:24.857 INFO:tasks.workunit.client.1.smithi186.stdout:0/178: stat d0/d14/f35 0 2024-04-12T20:56:24.860 INFO:tasks.workunit.client.1.smithi186.stdout:5/102: symlink d5/db/l22 0 2024-04-12T20:56:24.861 INFO:tasks.workunit.client.0.smithi018.stdout:2/853: mkdir d5/d7/d31/d107/d127 0 2024-04-12T20:56:24.863 INFO:tasks.workunit.client.0.smithi018.stdout:1/737: sync 2024-04-12T20:56:24.863 INFO:tasks.workunit.client.0.smithi018.stdout:0/966: sync 2024-04-12T20:56:24.863 INFO:tasks.workunit.client.0.smithi018.stdout:7/935: sync 2024-04-12T20:56:24.864 INFO:tasks.workunit.client.1.smithi186.stdout:5/103: stat d5/db/d14/f16 0 2024-04-12T20:56:24.865 INFO:tasks.workunit.client.1.smithi186.stdout:3/131: dwrite d1/f19 [4194304,4194304] 0 2024-04-12T20:56:24.868 INFO:tasks.workunit.client.1.smithi186.stdout:0/179: dread d0/d10/d1c/d1e/f2f [0,4194304] 0 2024-04-12T20:56:24.868 INFO:tasks.workunit.client.1.smithi186.stdout:0/180: readlink d0/la 0 2024-04-12T20:56:24.869 INFO:tasks.workunit.client.1.smithi186.stdout:2/107: dwrite d0/f2 [0,4194304] 0 2024-04-12T20:56:24.879 INFO:tasks.workunit.client.0.smithi018.stdout:2/854: mkdir d5/d8/d10/d43/de2/d24/d91/d128 0 2024-04-12T20:56:24.879 INFO:tasks.workunit.client.0.smithi018.stdout:7/936: readlink d1/db/d31/df2/l104 0 2024-04-12T20:56:24.882 INFO:tasks.workunit.client.0.smithi018.stdout:2/855: chown d5/d8/d10/d43/cca 3080 1 2024-04-12T20:56:24.884 INFO:tasks.workunit.client.1.smithi186.stdout:0/181: dread d0/f6 [4194304,4194304] 0 2024-04-12T20:56:24.885 INFO:tasks.workunit.client.0.smithi018.stdout:4/878: dwrite d3/db/fd0 [0,4194304] 0 2024-04-12T20:56:24.885 INFO:tasks.workunit.client.1.smithi186.stdout:3/132: dwrite d1/d3/f1f [0,4194304] 0 2024-04-12T20:56:24.887 INFO:tasks.workunit.client.1.smithi186.stdout:3/133: write d1/f1e [2457974,4292] 0 2024-04-12T20:56:24.904 INFO:tasks.workunit.client.1.smithi186.stdout:2/108: fsync d0/f2 0 2024-04-12T20:56:24.904 INFO:tasks.workunit.client.1.smithi186.stdout:5/104: getdents d5/db 0 2024-04-12T20:56:24.905 INFO:tasks.workunit.client.1.smithi186.stdout:2/109: write d0/f10 [36128,98183] 0 2024-04-12T20:56:24.906 INFO:tasks.workunit.client.1.smithi186.stdout:2/110: readlink d0/l1d 0 2024-04-12T20:56:24.908 INFO:tasks.workunit.client.1.smithi186.stdout:2/111: dread - d0/d3/f1f zero size 2024-04-12T20:56:24.909 INFO:tasks.workunit.client.1.smithi186.stdout:2/112: dread - d0/f14 zero size 2024-04-12T20:56:24.911 INFO:tasks.workunit.client.1.smithi186.stdout:3/134: link d1/d3/c1d d1/d3/da/c26 0 2024-04-12T20:56:24.912 INFO:tasks.workunit.client.1.smithi186.stdout:6/95: sync 2024-04-12T20:56:24.912 INFO:tasks.workunit.client.1.smithi186.stdout:8/81: sync 2024-04-12T20:56:24.912 INFO:tasks.workunit.client.1.smithi186.stdout:1/133: sync 2024-04-12T20:56:24.915 INFO:tasks.workunit.client.1.smithi186.stdout:0/182: getdents d0/d10 0 2024-04-12T20:56:24.918 INFO:tasks.workunit.client.1.smithi186.stdout:5/105: rename d5/db/d14/l17 to d5/db/l23 0 2024-04-12T20:56:24.920 INFO:tasks.workunit.client.1.smithi186.stdout:6/96: dread d9/fb [0,4194304] 0 2024-04-12T20:56:24.921 INFO:tasks.workunit.client.1.smithi186.stdout:1/134: dread d14/d1a/d21/f27 [0,4194304] 0 2024-04-12T20:56:24.921 INFO:tasks.workunit.client.1.smithi186.stdout:8/82: mkdir d7/d8/d16 0 2024-04-12T20:56:24.923 INFO:tasks.workunit.client.1.smithi186.stdout:1/135: readlink d14/d1a/d21/l2b 0 2024-04-12T20:56:24.926 INFO:tasks.workunit.client.1.smithi186.stdout:8/83: truncate d7/f12 678770 0 2024-04-12T20:56:24.928 INFO:tasks.workunit.client.1.smithi186.stdout:0/183: creat d0/d10/d32/f3d x:0 0 0 2024-04-12T20:56:24.929 INFO:tasks.workunit.client.1.smithi186.stdout:2/113: mkdir d0/d3/d1a/d24/d25 0 2024-04-12T20:56:24.933 INFO:tasks.workunit.client.1.smithi186.stdout:3/135: dwrite d1/d3/f11 [0,4194304] 0 2024-04-12T20:56:24.934 INFO:tasks.workunit.client.1.smithi186.stdout:6/97: creat d9/f22 x:0 0 0 2024-04-12T20:56:24.934 INFO:tasks.workunit.client.1.smithi186.stdout:6/98: fsync f6 0 2024-04-12T20:56:24.934 INFO:tasks.workunit.client.1.smithi186.stdout:3/136: readlink d1/l18 0 2024-04-12T20:56:24.940 INFO:tasks.workunit.client.0.smithi018.stdout:7/937: mkdir d1/db/d6f/dad/d12d 0 2024-04-12T20:56:24.941 INFO:tasks.workunit.client.0.smithi018.stdout:2/856: readlink d5/d8/d10/d43/de2/l2a 0 2024-04-12T20:56:24.944 INFO:tasks.workunit.client.0.smithi018.stdout:4/879: rename d3/d5d/d60/d64 to d3/d5d/d60/dc5/d53/de7/d126 0 2024-04-12T20:56:24.944 INFO:tasks.workunit.client.1.smithi186.stdout:0/184: creat d0/f3e x:0 0 0 2024-04-12T20:56:24.945 INFO:tasks.workunit.client.0.smithi018.stdout:0/967: mknod da/d15/d4d/d12f/c143 0 2024-04-12T20:56:24.946 INFO:tasks.workunit.client.1.smithi186.stdout:0/185: readlink d0/l3b 0 2024-04-12T20:56:24.947 INFO:tasks.workunit.client.1.smithi186.stdout:5/106: dread d5/db/d14/f16 [0,4194304] 0 2024-04-12T20:56:24.949 INFO:tasks.workunit.client.1.smithi186.stdout:3/137: getdents d1/d3/d21/d22/d24 0 2024-04-12T20:56:24.949 INFO:tasks.workunit.client.1.smithi186.stdout:8/84: dwrite d7/d8/fa [0,4194304] 0 2024-04-12T20:56:24.950 INFO:tasks.workunit.client.0.smithi018.stdout:4/880: symlink d3/d5d/d60/dc5/d40/l127 0 2024-04-12T20:56:24.951 INFO:tasks.workunit.client.0.smithi018.stdout:1/738: link d3/d9/da/dd/d31/d32/d33/l89 d3/d9/da/dd/d26/df0/lf8 0 2024-04-12T20:56:24.953 INFO:tasks.workunit.client.1.smithi186.stdout:6/99: getdents d9 0 2024-04-12T20:56:24.954 INFO:tasks.workunit.client.1.smithi186.stdout:0/186: mkdir d0/d10/d32/d3f 0 2024-04-12T20:56:24.954 INFO:tasks.workunit.client.0.smithi018.stdout:4/881: mkdir d3/d5d/d60/dc5/d40/d128 0 2024-04-12T20:56:24.955 INFO:tasks.workunit.client.1.smithi186.stdout:5/107: write d5/db/d14/f1a [521804,23974] 0 2024-04-12T20:56:24.958 INFO:tasks.workunit.client.1.smithi186.stdout:6/100: write d9/f1d [496584,29418] 0 2024-04-12T20:56:24.960 INFO:tasks.workunit.client.1.smithi186.stdout:8/85: dread d7/f12 [0,4194304] 0 2024-04-12T20:56:24.961 INFO:tasks.workunit.client.1.smithi186.stdout:2/114: dwrite d0/f4 [0,4194304] 0 2024-04-12T20:56:24.961 INFO:tasks.workunit.client.1.smithi186.stdout:6/101: write d9/f22 [532559,62900] 0 2024-04-12T20:56:24.973 INFO:tasks.workunit.client.1.smithi186.stdout:6/102: rmdir d9 39 2024-04-12T20:56:24.975 INFO:tasks.workunit.client.0.smithi018.stdout:0/968: creat da/d15/d17/d21/d36/da2/dc3/f144 x:0 0 0 2024-04-12T20:56:24.976 INFO:tasks.workunit.client.0.smithi018.stdout:4/882: rmdir d3/d5d/d60/dc5/d43/d4b/ddc 39 2024-04-12T20:56:24.977 INFO:tasks.workunit.client.1.smithi186.stdout:3/138: truncate d1/fc 4037016 0 2024-04-12T20:56:24.977 INFO:tasks.workunit.client.1.smithi186.stdout:0/187: dwrite d0/d14/f1a [0,4194304] 0 2024-04-12T20:56:24.978 INFO:tasks.workunit.client.1.smithi186.stdout:9/91: sync 2024-04-12T20:56:24.979 INFO:tasks.workunit.client.0.smithi018.stdout:1/739: creat d3/d9/da/dd/d31/d32/d13/d19/d2d/d4e/d7a/df5/ff9 x:0 0 0 2024-04-12T20:56:24.979 INFO:tasks.workunit.client.1.smithi186.stdout:3/139: fdatasync d1/f1e 0 2024-04-12T20:56:24.980 INFO:tasks.workunit.client.1.smithi186.stdout:0/188: readlink d0/l26 0 2024-04-12T20:56:24.983 INFO:tasks.workunit.client.1.smithi186.stdout:3/140: fsync d1/d3/f11 0 2024-04-12T20:56:24.983 INFO:tasks.workunit.client.0.smithi018.stdout:7/938: dread d1/db/d31/d45/df4/d124/f7d [0,4194304] 0 2024-04-12T20:56:24.984 INFO:tasks.workunit.client.1.smithi186.stdout:5/108: mknod d5/db/d15/d18/c24 0 2024-04-12T20:56:24.985 INFO:tasks.workunit.client.0.smithi018.stdout:0/969: symlink da/d15/d17/d5c/d79/dd6/l145 0 2024-04-12T20:56:24.986 INFO:tasks.workunit.client.1.smithi186.stdout:2/115: getdents d0/d3/d12 0 2024-04-12T20:56:24.989 INFO:tasks.workunit.client.1.smithi186.stdout:9/92: mknod d3/d12/c14 0 2024-04-12T20:56:24.991 INFO:tasks.workunit.client.0.smithi018.stdout:0/970: write da/d15/d17/d21/d36/d47/f137 [151518,108507] 0 2024-04-12T20:56:24.991 INFO:tasks.workunit.client.0.smithi018.stdout:7/939: write d1/db/d31/d45/d8c/d9d/d69/f7e [1900365,64025] 0 2024-04-12T20:56:24.995 INFO:tasks.workunit.client.1.smithi186.stdout:8/86: dwrite d7/de/f13 [0,4194304] 0 2024-04-12T20:56:24.995 INFO:tasks.workunit.client.1.smithi186.stdout:5/109: truncate d5/fd 334774 0 2024-04-12T20:56:24.995 INFO:tasks.workunit.client.1.smithi186.stdout:0/189: rename d0/d10/d32/l36 to d0/d10/d1c/l40 0 2024-04-12T20:56:24.996 INFO:tasks.workunit.client.0.smithi018.stdout:7/940: readlink d1/db/d31/d30/l50 0 2024-04-12T20:56:24.996 INFO:tasks.workunit.client.0.smithi018.stdout:1/740: truncate d3/d9/da/dd/d31/d32/f1b 464778 0 2024-04-12T20:56:24.997 INFO:tasks.workunit.client.0.smithi018.stdout:1/741: chown d3/d9/d11/c43 58626 1 2024-04-12T20:56:25.001 INFO:tasks.workunit.client.1.smithi186.stdout:9/93: write d3/fb [992257,64175] 0 2024-04-12T20:56:25.001 INFO:tasks.workunit.client.1.smithi186.stdout:3/141: mknod d1/d3/d21/d22/d24/c27 0 2024-04-12T20:56:25.002 INFO:tasks.workunit.client.0.smithi018.stdout:7/941: creat d1/db/d6f/f12e x:0 0 0 2024-04-12T20:56:25.003 INFO:tasks.workunit.client.1.smithi186.stdout:5/110: chown c3 718226972 1 2024-04-12T20:56:25.008 INFO:tasks.workunit.client.0.smithi018.stdout:0/971: getdents da/d15/d17/d71/dc1/dee/df7 0 2024-04-12T20:56:25.011 INFO:tasks.workunit.client.1.smithi186.stdout:0/190: symlink d0/d10/d1c/d1e/l41 0 2024-04-12T20:56:25.012 INFO:tasks.workunit.client.1.smithi186.stdout:6/103: dwrite d9/fe [0,4194304] 0 2024-04-12T20:56:25.013 INFO:tasks.workunit.client.1.smithi186.stdout:8/87: rename d7/d8/d16 to d7/d8/d17 0 2024-04-12T20:56:25.017 INFO:tasks.workunit.client.0.smithi018.stdout:1/742: rename d3/d9/da/dd/d31/d32/d13/d19/d2d/d4e/d58/d61/cbf to d3/d9/da/dd/d31/d32/d13/d19/d2d/cfa 0 2024-04-12T20:56:25.018 INFO:tasks.workunit.client.1.smithi186.stdout:0/191: chown d0/f6 3193 1 2024-04-12T20:56:25.019 INFO:tasks.workunit.client.0.smithi018.stdout:7/942: link d1/db/d31/d30/dc6/d100/d12a/d4c/d6c/dc3/c90 d1/db/d31/d45/d8c/d9d/d70/d96/d5f/da9/dfc/c12f 0 2024-04-12T20:56:25.023 INFO:tasks.workunit.client.0.smithi018.stdout:1/743: chown d3/d9/da/dd/d31/d32/d13/d19/d2d/d4e/d7a/df5 60 1 2024-04-12T20:56:25.023 INFO:tasks.workunit.client.1.smithi186.stdout:7/153: sync 2024-04-12T20:56:25.023 INFO:tasks.workunit.client.1.smithi186.stdout:4/218: sync 2024-04-12T20:56:25.025 INFO:tasks.workunit.client.0.smithi018.stdout:7/943: mknod d1/db/d31/d45/d8c/d9d/d94/c130 0 2024-04-12T20:56:25.026 INFO:tasks.workunit.client.1.smithi186.stdout:2/116: dread d0/d3/f8 [0,4194304] 0 2024-04-12T20:56:25.028 INFO:tasks.workunit.client.1.smithi186.stdout:8/88: truncate d7/f12 932985 0 2024-04-12T20:56:25.029 INFO:tasks.workunit.client.1.smithi186.stdout:6/104: dread d9/f22 [0,4194304] 0 2024-04-12T20:56:25.029 INFO:tasks.workunit.client.0.smithi018.stdout:7/944: chown d1/db/d31/d45/d8c/d9d/d70/d96/d5f/da9 0 1 2024-04-12T20:56:25.030 INFO:tasks.workunit.client.1.smithi186.stdout:2/117: stat d0/f14 0 2024-04-12T20:56:25.033 INFO:tasks.workunit.client.0.smithi018.stdout:1/744: truncate d3/d9/da/dd/d31/d32/d13/d19/d2d/d4e/d58/d61/d71/fdf 977401 0 2024-04-12T20:56:25.035 INFO:tasks.workunit.client.1.smithi186.stdout:6/105: write d9/f1b [2352966,47745] 0 2024-04-12T20:56:25.036 INFO:tasks.workunit.client.1.smithi186.stdout:3/142: dwrite d1/f1e [0,4194304] 0 2024-04-12T20:56:25.036 INFO:tasks.workunit.client.0.smithi018.stdout:7/945: getdents d1/db/d31/d45/d8c/d9d/d70/d96/d5f 0 2024-04-12T20:56:25.041 INFO:tasks.workunit.client.1.smithi186.stdout:0/192: getdents d0/d10 0 2024-04-12T20:56:25.042 INFO:tasks.workunit.client.1.smithi186.stdout:6/106: mknod d9/c23 0 2024-04-12T20:56:25.042 INFO:tasks.workunit.client.0.smithi018.stdout:7/946: fsync d1/db/d31/d30/dc6/d100/d12a/d4c/dbc/fde 0 2024-04-12T20:56:25.053 INFO:tasks.workunit.client.1.smithi186.stdout:6/107: creat d9/f24 x:0 0 0 2024-04-12T20:56:25.057 INFO:tasks.workunit.client.0.smithi018.stdout:7/947: dread d1/db/d31/d3a/d97/fb1 [0,4194304] 0 2024-04-12T20:56:25.059 INFO:tasks.workunit.client.1.smithi186.stdout:0/193: chown d0/cc 32983 1 2024-04-12T20:56:25.064 INFO:tasks.workunit.client.1.smithi186.stdout:7/154: dwrite d7/d10/d1c/f2c [0,4194304] 0 2024-04-12T20:56:25.064 INFO:tasks.workunit.client.1.smithi186.stdout:3/143: getdents d1/d3/d21/d22 0 2024-04-12T20:56:25.065 INFO:tasks.workunit.client.0.smithi018.stdout:1/745: dwrite d3/d9/da/dd/d26/fbc [0,4194304] 0 2024-04-12T20:56:25.068 INFO:tasks.workunit.client.0.smithi018.stdout:7/948: link d1/db/d31/d45/fc2 d1/db/d31/d30/dc6/d100/d12a/d4c/d103/f131 0 2024-04-12T20:56:25.071 INFO:tasks.workunit.client.1.smithi186.stdout:9/94: dwrite f1 [0,4194304] 0 2024-04-12T20:56:25.074 INFO:tasks.workunit.client.1.smithi186.stdout:7/155: dread - d7/d10/d2d/f37 zero size 2024-04-12T20:56:25.081 INFO:tasks.workunit.client.1.smithi186.stdout:6/108: truncate f6 3808640 0 2024-04-12T20:56:25.081 INFO:tasks.workunit.client.0.smithi018.stdout:7/949: rename d1/db/d31/d45/d8c/d9d/d94 to d1/db/d31/d132 0 2024-04-12T20:56:25.086 INFO:tasks.workunit.client.0.smithi018.stdout:1/746: rename d3/d9/da/dd/d26/cf4 to d3/d9/da/dd/d31/d32/d33/d47/db0/cfb 0 2024-04-12T20:56:25.086 INFO:tasks.workunit.client.1.smithi186.stdout:9/95: symlink d3/l15 0 2024-04-12T20:56:25.086 INFO:tasks.workunit.client.1.smithi186.stdout:5/111: dwrite d5/db/fc [4194304,4194304] 0 2024-04-12T20:56:25.087 INFO:tasks.workunit.client.1.smithi186.stdout:2/118: dwrite d0/f2 [0,4194304] 0 2024-04-12T20:56:25.093 INFO:tasks.workunit.client.1.smithi186.stdout:3/144: unlink d1/d3/da/l14 0 2024-04-12T20:56:25.094 INFO:tasks.workunit.client.1.smithi186.stdout:8/89: dwrite d7/fd [0,4194304] 0 2024-04-12T20:56:25.096 INFO:tasks.workunit.client.1.smithi186.stdout:7/156: symlink d7/d10/d1c/l38 0 2024-04-12T20:56:25.097 INFO:tasks.workunit.client.1.smithi186.stdout:9/96: unlink d3/lc 0 2024-04-12T20:56:25.097 INFO:tasks.workunit.client.1.smithi186.stdout:3/145: chown d1/c1c 6 1 2024-04-12T20:56:25.099 INFO:tasks.workunit.client.1.smithi186.stdout:9/97: chown d3/f5 3085 1 2024-04-12T20:56:25.102 INFO:tasks.workunit.client.1.smithi186.stdout:5/112: rename d5/c1c to d5/c25 0 2024-04-12T20:56:25.103 INFO:tasks.workunit.client.0.smithi018.stdout:1/747: link d3/d9/da/dd/d31/d32/d13/d19/d2d/d4e/laa d3/d9/da/dd/d31/dc6/lfc 0 2024-04-12T20:56:25.105 INFO:tasks.workunit.client.1.smithi186.stdout:7/157: mkdir d7/d10/d2d/d39 0 2024-04-12T20:56:25.105 INFO:tasks.workunit.client.1.smithi186.stdout:9/98: mkdir d3/d12/d16 0 2024-04-12T20:56:25.112 INFO:tasks.workunit.client.1.smithi186.stdout:0/194: dwrite d0/d10/f2c [0,4194304] 0 2024-04-12T20:56:25.112 INFO:tasks.workunit.client.1.smithi186.stdout:7/158: dread - d7/d10/f19 zero size 2024-04-12T20:56:25.112 INFO:tasks.workunit.client.1.smithi186.stdout:3/146: mkdir d1/d3/d21/d22/d24/d28 0 2024-04-12T20:56:25.113 INFO:tasks.workunit.client.1.smithi186.stdout:9/99: chown d3/f11 1547 1 2024-04-12T20:56:25.113 INFO:tasks.workunit.client.1.smithi186.stdout:1/136: sync 2024-04-12T20:56:25.121 INFO:tasks.workunit.client.1.smithi186.stdout:5/113: chown d5/db/l13 1614 1 2024-04-12T20:56:25.128 INFO:tasks.workunit.client.1.smithi186.stdout:1/137: write d14/d1a/d1d/f23 [803106,19124] 0 2024-04-12T20:56:25.129 INFO:tasks.workunit.client.1.smithi186.stdout:0/195: symlink d0/l42 0 2024-04-12T20:56:25.132 INFO:tasks.workunit.client.1.smithi186.stdout:3/147: rename d1/d3/da/l1b to d1/d3/d21/l29 0 2024-04-12T20:56:25.133 INFO:tasks.workunit.client.1.smithi186.stdout:5/114: creat d5/db/d15/f26 x:0 0 0 2024-04-12T20:56:25.134 INFO:tasks.workunit.client.1.smithi186.stdout:8/90: dread d7/fd [0,4194304] 0 2024-04-12T20:56:25.134 INFO:tasks.workunit.client.1.smithi186.stdout:7/159: rmdir d7/d10/d2d/d39 0 2024-04-12T20:56:25.135 INFO:tasks.workunit.client.1.smithi186.stdout:2/119: dwrite d0/f10 [0,4194304] 0 2024-04-12T20:56:25.136 INFO:tasks.workunit.client.1.smithi186.stdout:6/109: dwrite d9/f22 [0,4194304] 0 2024-04-12T20:56:25.139 INFO:tasks.workunit.client.1.smithi186.stdout:5/115: dread - d5/db/d15/f26 zero size 2024-04-12T20:56:25.139 INFO:tasks.workunit.client.0.smithi018.stdout:7/950: dwrite d1/db/d31/d30/dc6/d100/d12a/dcd/f11a [0,4194304] 0 2024-04-12T20:56:25.144 INFO:tasks.workunit.client.0.smithi018.stdout:1/748: dwrite d3/d9/da/dd/d31/d32/d13/d19/d2d/d4e/d5e/d97/fc5 [0,4194304] 0 2024-04-12T20:56:25.151 INFO:tasks.workunit.client.1.smithi186.stdout:1/138: dread f9 [0,4194304] 0 2024-04-12T20:56:25.152 INFO:tasks.workunit.client.0.smithi018.stdout:1/749: mkdir d3/d9/da/d14/dfd 0 2024-04-12T20:56:25.160 INFO:tasks.workunit.client.0.smithi018.stdout:1/750: symlink d3/d9/da/d14/d73/lfe 0 2024-04-12T20:56:25.164 INFO:tasks.workunit.client.0.smithi018.stdout:7/951: write d1/db/d31/f37 [3441344,17166] 0 2024-04-12T20:56:25.169 INFO:tasks.workunit.client.0.smithi018.stdout:7/952: chown d1/db/d31/d45/df4 48 1 2024-04-12T20:56:25.170 INFO:tasks.workunit.client.0.smithi018.stdout:1/751: truncate d3/d9/da/dd/f9d 4527084 0 2024-04-12T20:56:25.174 INFO:tasks.workunit.client.0.smithi018.stdout:7/953: read - d1/db/d31/d30/dc6/d100/d12a/d4c/d6c/dc3/f106 zero size 2024-04-12T20:56:25.180 INFO:tasks.workunit.client.0.smithi018.stdout:1/752: creat d3/d9/da/dd/d31/d32/d13/d19/d2d/d4e/d58/fff x:0 0 0 2024-04-12T20:56:25.185 INFO:tasks.workunit.client.0.smithi018.stdout:7/954: creat d1/db/d31/d3a/f133 x:0 0 0 2024-04-12T20:56:25.187 INFO:tasks.workunit.client.1.smithi186.stdout:4/219: sync 2024-04-12T20:56:25.190 INFO:tasks.workunit.client.1.smithi186.stdout:8/91: mkdir d7/d18 0 2024-04-12T20:56:25.190 INFO:tasks.workunit.client.0.smithi018.stdout:1/753: rename d3/d9/ce8 to d3/d9/da/dd/d31/d32/d13/d19/d2d/d4e/d5e/da0/c100 0 2024-04-12T20:56:25.192 INFO:tasks.workunit.client.0.smithi018.stdout:7/955: mknod d1/db/d31/d45/d8c/d9d/d70/d96/c134 0 2024-04-12T20:56:25.196 INFO:tasks.workunit.client.0.smithi018.stdout:7/956: mkdir d1/db/d31/d132/d135 0 2024-04-12T20:56:25.200 INFO:tasks.workunit.client.1.smithi186.stdout:3/148: unlink d1/lf 0 2024-04-12T20:56:25.201 INFO:tasks.workunit.client.1.smithi186.stdout:4/220: dwrite d0/d21/f20 [0,4194304] 0 2024-04-12T20:56:25.202 INFO:tasks.workunit.client.1.smithi186.stdout:3/149: read d1/f1e [1022517,5010] 0 2024-04-12T20:56:25.202 INFO:tasks.workunit.client.0.smithi018.stdout:7/957: write d1/db/d6f/f12e [693208,89110] 0 2024-04-12T20:56:25.205 INFO:tasks.workunit.client.1.smithi186.stdout:2/120: mkdir d0/d3/d1a/d21/d26 0 2024-04-12T20:56:25.206 INFO:tasks.workunit.client.1.smithi186.stdout:2/121: dread - d0/d3/f1f zero size 2024-04-12T20:56:25.209 INFO:tasks.workunit.client.0.smithi018.stdout:7/958: stat d1/db/d31/d45/d8c/d9d/d70 0 2024-04-12T20:56:25.209 INFO:tasks.workunit.client.1.smithi186.stdout:4/221: dread d0/dc/f3b [0,4194304] 0 2024-04-12T20:56:25.209 INFO:tasks.workunit.client.1.smithi186.stdout:5/116: mkdir d5/d27 0 2024-04-12T20:56:25.210 INFO:tasks.workunit.client.1.smithi186.stdout:4/222: stat d0/dc 0 2024-04-12T20:56:25.213 INFO:tasks.workunit.client.1.smithi186.stdout:9/100: sync 2024-04-12T20:56:25.214 INFO:tasks.workunit.client.1.smithi186.stdout:7/160: rename d7/d10/d1c/c2e to d7/d10/c3a 0 2024-04-12T20:56:25.218 INFO:tasks.workunit.client.1.smithi186.stdout:1/139: mkdir d14/d1a/d21/d2c 0 2024-04-12T20:56:25.221 INFO:tasks.workunit.client.0.smithi018.stdout:1/754: dwrite d3/d9/da/dd/d31/d32/d13/d19/d2d/d4e/fde [0,4194304] 0 2024-04-12T20:56:25.230 INFO:tasks.workunit.client.0.smithi018.stdout:7/959: write d1/db/d31/d3a/d97/fcf [826192,28070] 0 2024-04-12T20:56:25.230 INFO:tasks.workunit.client.1.smithi186.stdout:5/117: dwrite d5/db/d15/f1e [0,4194304] 0 2024-04-12T20:56:25.231 INFO:tasks.workunit.client.1.smithi186.stdout:9/101: dread d3/fb [0,4194304] 0 2024-04-12T20:56:25.233 INFO:tasks.workunit.client.1.smithi186.stdout:1/140: dread fb [0,4194304] 0 2024-04-12T20:56:25.245 INFO:tasks.workunit.client.1.smithi186.stdout:9/102: dread d3/fb [0,4194304] 0 2024-04-12T20:56:25.246 INFO:tasks.workunit.client.1.smithi186.stdout:9/103: fsync f1 0 2024-04-12T20:56:25.246 INFO:tasks.workunit.client.0.smithi018.stdout:1/755: rename d3/d9/da/dd/d31/d32/d13/d19/cb7 to d3/d9/da/dd/d31/d32/d13/d19/d2d/d4e/d58/d5c/c101 0 2024-04-12T20:56:25.248 INFO:tasks.workunit.client.1.smithi186.stdout:2/122: fdatasync d0/d3/f8 0 2024-04-12T20:56:25.249 INFO:tasks.workunit.client.1.smithi186.stdout:2/123: fdatasync d0/f6 0 2024-04-12T20:56:25.262 INFO:tasks.workunit.client.1.smithi186.stdout:9/104: dwrite d3/f10 [4194304,4194304] 0 2024-04-12T20:56:25.268 INFO:tasks.workunit.client.1.smithi186.stdout:5/118: rename d5/f20 to d5/db/d15/f28 0 2024-04-12T20:56:25.270 INFO:tasks.workunit.client.0.smithi018.stdout:7/960: truncate d1/db/d31/d45/df4/d124/f7d 2467709 0 2024-04-12T20:56:25.271 INFO:tasks.workunit.client.0.smithi018.stdout:1/756: write d3/d9/da/dd/f55 [2692302,104425] 0 2024-04-12T20:56:25.274 INFO:tasks.workunit.client.1.smithi186.stdout:3/150: mkdir d1/d3/d2a 0 2024-04-12T20:56:25.279 INFO:tasks.workunit.client.0.smithi018.stdout:1/757: write d3/d9/da/f20 [1945579,22982] 0 2024-04-12T20:56:25.281 INFO:tasks.workunit.client.0.smithi018.stdout:7/961: symlink d1/db/d31/d45/l136 0 2024-04-12T20:56:25.285 INFO:tasks.workunit.client.1.smithi186.stdout:3/151: dread d1/f19 [4194304,4194304] 0 2024-04-12T20:56:25.287 INFO:tasks.workunit.client.0.smithi018.stdout:7/962: mkdir d1/db/d31/d45/d8c/d9d/d70/d96/d5f/de4/d137 0 2024-04-12T20:56:25.287 INFO:tasks.workunit.client.1.smithi186.stdout:2/124: mkdir d0/d3/d1a/d27 0 2024-04-12T20:56:25.288 INFO:tasks.workunit.client.0.smithi018.stdout:7/963: chown d1/db 31 1 2024-04-12T20:56:25.289 INFO:tasks.workunit.client.0.smithi018.stdout:1/758: truncate d3/d9/da/dd/d31/d32/d13/d19/d2d/d4e/d58/d61/d71/fdf 1343410 0 2024-04-12T20:56:25.290 INFO:tasks.workunit.client.1.smithi186.stdout:4/223: mkdir d0/d21/d3d/d48 0 2024-04-12T20:56:25.291 INFO:tasks.workunit.client.1.smithi186.stdout:4/224: chown d0/dc/d33/f1a 834138 1 2024-04-12T20:56:25.292 INFO:tasks.workunit.client.0.smithi018.stdout:1/759: fdatasync d3/d9/da/dd/d26/fa6 0 2024-04-12T20:56:25.300 INFO:tasks.workunit.client.1.smithi186.stdout:5/119: chown d5/db/l10 3998379 1 2024-04-12T20:56:25.301 INFO:tasks.workunit.client.1.smithi186.stdout:5/120: chown d5/db/d14/f1a 7073 1 2024-04-12T20:56:25.301 INFO:tasks.workunit.client.0.smithi018.stdout:1/760: mknod d3/d9/da/dd/d26/df0/d66/c102 0 2024-04-12T20:56:25.305 INFO:tasks.workunit.client.0.smithi018.stdout:1/761: stat d3/d9/da/dd/d31/d32/c1d 0 2024-04-12T20:56:25.306 INFO:tasks.workunit.client.1.smithi186.stdout:3/152: rename d1/d3/l10 to d1/d3/da/l2b 0 2024-04-12T20:56:25.308 INFO:tasks.workunit.client.0.smithi018.stdout:7/964: dwrite d1/db/d31/d30/dc6/d100/d12a/dcd/f11a [0,4194304] 0 2024-04-12T20:56:25.309 INFO:tasks.workunit.client.1.smithi186.stdout:2/125: unlink d0/d3/fe 0 2024-04-12T20:56:25.311 INFO:tasks.workunit.client.0.smithi018.stdout:7/965: write d1/db/d6f/f12e [92093,81156] 0 2024-04-12T20:56:25.312 INFO:tasks.workunit.client.1.smithi186.stdout:2/126: write d0/d3/f1f [191698,86861] 0 2024-04-12T20:56:25.315 INFO:tasks.workunit.client.0.smithi018.stdout:7/966: dread - d1/db/d31/d45/d8c/d9d/f127 zero size 2024-04-12T20:56:25.315 INFO:tasks.workunit.client.1.smithi186.stdout:0/196: truncate d0/f6 5893324 0 2024-04-12T20:56:25.315 INFO:tasks.workunit.client.1.smithi186.stdout:2/127: readlink d0/d3/la 0 2024-04-12T20:56:25.316 INFO:tasks.workunit.client.1.smithi186.stdout:6/110: write f6 [3970310,7233] 0 2024-04-12T20:56:25.317 INFO:tasks.workunit.client.1.smithi186.stdout:2/128: chown d0/f6 4341346 1 2024-04-12T20:56:25.317 INFO:tasks.workunit.client.0.smithi018.stdout:7/967: symlink d1/db/d31/df2/l138 0 2024-04-12T20:56:25.318 INFO:tasks.workunit.client.0.smithi018.stdout:1/762: truncate d3/d9/da/dd/d31/d32/d13/d19/d2d/d4e/d58/d61/d71/f91 773178 0 2024-04-12T20:56:25.319 INFO:tasks.workunit.client.1.smithi186.stdout:5/121: chown d5/c25 5 1 2024-04-12T20:56:25.319 INFO:tasks.workunit.client.1.smithi186.stdout:2/129: chown d0/l23 1278 1 2024-04-12T20:56:25.320 INFO:tasks.workunit.client.0.smithi018.stdout:1/763: chown d3/d9/da/d14 66551774 1 2024-04-12T20:56:25.321 INFO:tasks.workunit.client.0.smithi018.stdout:7/968: readlink d1/db/d31/d30/dc6/le6 0 2024-04-12T20:56:25.322 INFO:tasks.workunit.client.1.smithi186.stdout:8/92: creat d7/de/d10/f19 x:0 0 0 2024-04-12T20:56:25.329 INFO:tasks.workunit.client.1.smithi186.stdout:7/161: symlink d7/d10/l3b 0 2024-04-12T20:56:25.330 INFO:tasks.workunit.client.0.smithi018.stdout:7/969: link d1/d6/fb4 d1/db/d31/d45/d8c/d9d/d70/d96/d5f/de4/f139 0 2024-04-12T20:56:25.334 INFO:tasks.workunit.client.1.smithi186.stdout:2/130: mknod d0/d3/d1a/d21/d22/c28 0 2024-04-12T20:56:25.334 INFO:tasks.workunit.client.1.smithi186.stdout:0/197: dread d0/d10/f2a [0,4194304] 0 2024-04-12T20:56:25.334 INFO:tasks.workunit.client.1.smithi186.stdout:4/225: dwrite d0/d21/f32 [0,4194304] 0 2024-04-12T20:56:25.336 INFO:tasks.workunit.client.1.smithi186.stdout:0/198: stat d0 0 2024-04-12T20:56:25.339 INFO:tasks.workunit.client.1.smithi186.stdout:9/105: getdents d3 0 2024-04-12T20:56:25.339 INFO:tasks.workunit.client.1.smithi186.stdout:8/93: rename d7/de/f13 to d7/d18/f1a 0 2024-04-12T20:56:25.339 INFO:tasks.workunit.client.1.smithi186.stdout:4/226: fdatasync d0/dc/f23 0 2024-04-12T20:56:25.341 INFO:tasks.workunit.client.1.smithi186.stdout:7/162: readlink d7/d10/l35 0 2024-04-12T20:56:25.342 INFO:tasks.workunit.client.1.smithi186.stdout:5/122: dread d5/db/f11 [0,4194304] 0 2024-04-12T20:56:25.345 INFO:tasks.workunit.client.1.smithi186.stdout:6/111: dwrite d9/f1d [0,4194304] 0 2024-04-12T20:56:25.347 INFO:tasks.workunit.client.1.smithi186.stdout:0/199: dread d0/d10/d1c/d1e/f2f [0,4194304] 0 2024-04-12T20:56:25.349 INFO:tasks.workunit.client.1.smithi186.stdout:0/200: fsync d0/d10/f15 0 2024-04-12T20:56:25.350 INFO:tasks.workunit.client.1.smithi186.stdout:5/123: dread f2 [4194304,4194304] 0 2024-04-12T20:56:25.354 INFO:tasks.workunit.client.0.smithi018.stdout:1/764: dread d3/d9/da/f20 [0,4194304] 0 2024-04-12T20:56:25.361 INFO:tasks.workunit.client.0.smithi018.stdout:7/970: dwrite d1/d6/fca [0,4194304] 0 2024-04-12T20:56:25.361 INFO:tasks.workunit.client.1.smithi186.stdout:6/112: mkdir d9/d25 0 2024-04-12T20:56:25.362 INFO:tasks.workunit.client.1.smithi186.stdout:1/141: dwrite d14/d1a/d21/f27 [0,4194304] 0 2024-04-12T20:56:25.369 INFO:tasks.workunit.client.1.smithi186.stdout:8/94: creat d7/d8/f1b x:0 0 0 2024-04-12T20:56:25.369 INFO:tasks.workunit.client.1.smithi186.stdout:0/201: dread d0/d10/d1c/d1e/f1f [0,4194304] 0 2024-04-12T20:56:25.375 INFO:tasks.workunit.client.1.smithi186.stdout:9/106: creat d3/d13/f17 x:0 0 0 2024-04-12T20:56:25.376 INFO:tasks.workunit.client.1.smithi186.stdout:7/163: rename d7/d10/l35 to d7/d10/d1c/l3c 0 2024-04-12T20:56:25.379 INFO:tasks.workunit.client.1.smithi186.stdout:6/113: rename d9 to d9/d26 22 2024-04-12T20:56:25.379 INFO:tasks.workunit.client.1.smithi186.stdout:9/107: read d3/fb [3191759,57497] 0 2024-04-12T20:56:25.379 INFO:tasks.workunit.client.0.smithi018.stdout:2/857: sync 2024-04-12T20:56:25.380 INFO:tasks.workunit.client.1.smithi186.stdout:4/227: creat d0/f49 x:0 0 0 2024-04-12T20:56:25.383 INFO:tasks.workunit.client.1.smithi186.stdout:3/153: write d1/fc [3352123,66107] 0 2024-04-12T20:56:25.384 INFO:tasks.workunit.client.0.smithi018.stdout:2/858: symlink d5/d8/d10/d43/de2/d7c/d92/l129 0 2024-04-12T20:56:25.385 INFO:tasks.workunit.client.1.smithi186.stdout:9/108: creat d3/d13/f18 x:0 0 0 2024-04-12T20:56:25.385 INFO:tasks.workunit.client.1.smithi186.stdout:4/228: write d0/d21/f27 [1241903,79724] 0 2024-04-12T20:56:25.386 INFO:tasks.workunit.client.1.smithi186.stdout:7/164: write d7/d10/d1c/f20 [823480,125337] 0 2024-04-12T20:56:25.388 INFO:tasks.workunit.client.1.smithi186.stdout:8/95: link d7/d8/f1b d7/d8/d17/f1c 0 2024-04-12T20:56:25.388 INFO:tasks.workunit.client.0.smithi018.stdout:2/859: unlink d5/d8/d10/d43/de2/d7c/d92/l129 0 2024-04-12T20:56:25.390 INFO:tasks.workunit.client.1.smithi186.stdout:5/124: getdents d5/db/d14 0 2024-04-12T20:56:25.395 INFO:tasks.workunit.client.1.smithi186.stdout:9/109: creat d3/f19 x:0 0 0 2024-04-12T20:56:25.395 INFO:tasks.workunit.client.1.smithi186.stdout:4/229: chown d0/f24 248378 1 2024-04-12T20:56:25.396 INFO:tasks.workunit.client.0.smithi018.stdout:2/860: mknod d5/d8/d10/d43/de2/d24/d39/d9c/c12a 0 2024-04-12T20:56:25.396 INFO:tasks.workunit.client.1.smithi186.stdout:8/96: dread - d7/d8/f1b zero size 2024-04-12T20:56:25.398 INFO:tasks.workunit.client.1.smithi186.stdout:7/165: stat d7/d10/d2d/f37 0 2024-04-12T20:56:25.400 INFO:tasks.workunit.client.0.smithi018.stdout:2/861: read d5/d8/d10/d43/fd8 [2074853,48259] 0 2024-04-12T20:56:25.400 INFO:tasks.workunit.client.1.smithi186.stdout:2/131: truncate d0/d3/f1f 183917 0 2024-04-12T20:56:25.402 INFO:tasks.workunit.client.1.smithi186.stdout:7/166: mknod d7/c3d 0 2024-04-12T20:56:25.403 INFO:tasks.workunit.client.1.smithi186.stdout:9/110: mknod d3/d12/c1a 0 2024-04-12T20:56:25.404 INFO:tasks.workunit.client.1.smithi186.stdout:4/230: unlink d0/dc/f1d 0 2024-04-12T20:56:25.406 INFO:tasks.workunit.client.1.smithi186.stdout:1/142: dread fb [0,4194304] 0 2024-04-12T20:56:25.407 INFO:tasks.workunit.client.1.smithi186.stdout:2/132: read d0/f10 [1513165,99564] 0 2024-04-12T20:56:25.407 INFO:tasks.workunit.client.1.smithi186.stdout:0/202: dwrite d0/d10/f15 [0,4194304] 0 2024-04-12T20:56:25.410 INFO:tasks.workunit.client.1.smithi186.stdout:8/97: write d7/d18/f1a [3568741,103903] 0 2024-04-12T20:56:25.410 INFO:tasks.workunit.client.1.smithi186.stdout:3/154: rename d1/d3/d2a to d1/d2c 0 2024-04-12T20:56:25.410 INFO:tasks.workunit.client.1.smithi186.stdout:9/111: truncate d3/f10 8509542 0 2024-04-12T20:56:25.411 INFO:tasks.workunit.client.0.smithi018.stdout:1/765: write d3/d9/da/dd/d31/d32/d13/d19/d2d/d4e/d58/d61/d71/f91 [366756,12303] 0 2024-04-12T20:56:25.412 INFO:tasks.workunit.client.1.smithi186.stdout:7/167: symlink d7/d10/d2d/l3e 0 2024-04-12T20:56:25.417 INFO:tasks.workunit.client.0.smithi018.stdout:2/862: truncate d5/d8/d25/fba 4233468 0 2024-04-12T20:56:25.417 INFO:tasks.workunit.client.1.smithi186.stdout:5/125: dread d5/f6 [0,4194304] 0 2024-04-12T20:56:25.417 INFO:tasks.workunit.client.1.smithi186.stdout:8/98: mkdir d7/d8/d1d 0 2024-04-12T20:56:25.418 INFO:tasks.workunit.client.0.smithi018.stdout:2/863: readlink d5/d12/l74 0 2024-04-12T20:56:25.421 INFO:tasks.workunit.client.1.smithi186.stdout:9/112: creat d3/f1b x:0 0 0 2024-04-12T20:56:25.423 INFO:tasks.workunit.client.0.smithi018.stdout:1/766: creat d3/d9/da/dd/d31/d32/d13/d19/d2d/d4e/d7a/df5/f103 x:0 0 0 2024-04-12T20:56:25.424 INFO:tasks.workunit.client.1.smithi186.stdout:7/168: creat d7/f3f x:0 0 0 2024-04-12T20:56:25.426 INFO:tasks.workunit.client.0.smithi018.stdout:2/864: mknod d5/d8/d10/d43/c12b 0 2024-04-12T20:56:25.426 INFO:tasks.workunit.client.1.smithi186.stdout:7/169: chown d7/f25 78245 1 2024-04-12T20:56:25.427 INFO:tasks.workunit.client.0.smithi018.stdout:0/972: sync 2024-04-12T20:56:25.427 INFO:tasks.workunit.client.0.smithi018.stdout:4/883: sync 2024-04-12T20:56:25.427 INFO:tasks.workunit.client.0.smithi018.stdout:7/971: dwrite d1/db/d31/d45/f9c [0,4194304] 0 2024-04-12T20:56:25.428 INFO:tasks.workunit.client.1.smithi186.stdout:2/133: rename d0/f7 to d0/dc/f29 0 2024-04-12T20:56:25.430 INFO:tasks.workunit.client.1.smithi186.stdout:3/155: symlink d1/d3/l2d 0 2024-04-12T20:56:25.433 INFO:tasks.workunit.client.1.smithi186.stdout:4/231: dwrite d0/dc/d33/f47 [0,4194304] 0 2024-04-12T20:56:25.434 INFO:tasks.workunit.client.1.smithi186.stdout:8/99: creat d7/d18/f1e x:0 0 0 2024-04-12T20:56:25.435 INFO:tasks.workunit.client.1.smithi186.stdout:4/232: write d0/dc/f23 [1369440,86281] 0 2024-04-12T20:56:25.436 INFO:tasks.workunit.client.0.smithi018.stdout:0/973: mkdir da/d15/d4d/d12f/d146 0 2024-04-12T20:56:25.436 INFO:tasks.workunit.client.1.smithi186.stdout:4/233: readlink d0/l1 0 2024-04-12T20:56:25.439 INFO:tasks.workunit.client.1.smithi186.stdout:4/234: truncate d0/dc/f23 1852976 0 2024-04-12T20:56:25.440 INFO:tasks.workunit.client.0.smithi018.stdout:1/767: creat d3/d9/da/dd/d31/d32/d13/d19/d2d/d4e/d7a/f104 x:0 0 0 2024-04-12T20:56:25.443 INFO:tasks.workunit.client.1.smithi186.stdout:2/134: dread d0/d3/f8 [0,4194304] 0 2024-04-12T20:56:25.444 INFO:tasks.workunit.client.1.smithi186.stdout:1/143: getdents d14/d1a/d21/d2c 0 2024-04-12T20:56:25.447 INFO:tasks.workunit.client.0.smithi018.stdout:4/884: rmdir d3/db/de3/dfc/d11c 0 2024-04-12T20:56:25.449 INFO:tasks.workunit.client.1.smithi186.stdout:7/170: mkdir d7/d10/d2d/d40 0 2024-04-12T20:56:25.450 INFO:tasks.workunit.client.0.smithi018.stdout:0/974: rename da/f13 to da/d15/d17/d21/d36/d47/d81/f147 0 2024-04-12T20:56:25.451 INFO:tasks.workunit.client.0.smithi018.stdout:1/768: symlink d3/d9/da/dd/d31/d32/d33/d80/l105 0 2024-04-12T20:56:25.452 INFO:tasks.workunit.client.1.smithi186.stdout:9/113: rename d3/f1b to d3/d12/f1c 0 2024-04-12T20:56:25.454 INFO:tasks.workunit.client.1.smithi186.stdout:3/156: fsync d1/f1e 0 2024-04-12T20:56:25.456 INFO:tasks.workunit.client.0.smithi018.stdout:7/972: dwrite d1/db/d31/d45/f9a [0,4194304] 0 2024-04-12T20:56:25.456 INFO:tasks.workunit.client.1.smithi186.stdout:8/100: chown d7/fd 1728282 1 2024-04-12T20:56:25.458 INFO:tasks.workunit.client.1.smithi186.stdout:4/235: symlink d0/d21/d3d/l4a 0 2024-04-12T20:56:25.461 INFO:tasks.workunit.client.0.smithi018.stdout:1/769: creat d3/d9/da/dd/d26/df0/f106 x:0 0 0 2024-04-12T20:56:25.462 INFO:tasks.workunit.client.1.smithi186.stdout:5/126: unlink d5/db/le 0 2024-04-12T20:56:25.462 INFO:tasks.workunit.client.1.smithi186.stdout:2/135: rmdir d0 39 2024-04-12T20:56:25.462 INFO:tasks.workunit.client.0.smithi018.stdout:7/973: write d1/fba [1240562,36269] 0 2024-04-12T20:56:25.468 INFO:tasks.workunit.client.1.smithi186.stdout:9/114: creat d3/f1d x:0 0 0 2024-04-12T20:56:25.469 INFO:tasks.workunit.client.0.smithi018.stdout:0/975: rename da/d15/d17/d71/ccf to da/d15/d17/d21/d36/d88/ddf/c148 0 2024-04-12T20:56:25.470 INFO:tasks.workunit.client.1.smithi186.stdout:9/115: dread - d3/ff zero size 2024-04-12T20:56:25.470 INFO:tasks.workunit.client.1.smithi186.stdout:3/157: mknod d1/d3/d21/c2e 0 2024-04-12T20:56:25.473 INFO:tasks.workunit.client.0.smithi018.stdout:2/865: dwrite d5/d12/f96 [0,4194304] 0 2024-04-12T20:56:25.476 INFO:tasks.workunit.client.1.smithi186.stdout:1/144: link d14/d1a/d1d/f29 d14/d1a/d21/d2c/f2d 0 2024-04-12T20:56:25.477 INFO:tasks.workunit.client.0.smithi018.stdout:4/885: dwrite d3/d32/fd [0,4194304] 0 2024-04-12T20:56:25.479 INFO:tasks.workunit.client.1.smithi186.stdout:5/127: rmdir d5/db/d15 39 2024-04-12T20:56:25.481 INFO:tasks.workunit.client.0.smithi018.stdout:4/886: read d3/d5d/d60/dc5/d3d/f65 [2147523,67116] 0 2024-04-12T20:56:25.481 INFO:tasks.workunit.client.0.smithi018.stdout:7/974: read d1/db/d31/d30/dc6/d100/d12a/d4c/dbc/fde [5022656,130769] 0 2024-04-12T20:56:25.482 INFO:tasks.workunit.client.1.smithi186.stdout:8/101: mkdir d7/d8/d1d/d1f 0 2024-04-12T20:56:25.483 INFO:tasks.workunit.client.0.smithi018.stdout:0/976: creat da/d15/d17/d21/d36/d88/d117/f149 x:0 0 0 2024-04-12T20:56:25.483 INFO:tasks.workunit.client.0.smithi018.stdout:1/770: mknod d3/d9/da/dd/d31/d32/d13/d19/d2d/d4e/d58/d61/c107 0 2024-04-12T20:56:25.489 INFO:tasks.workunit.client.1.smithi186.stdout:2/136: mkdir d0/d3/d12/d2a 0 2024-04-12T20:56:25.489 INFO:tasks.workunit.client.0.smithi018.stdout:2/866: write d5/d7/d31/d62/da6/f100 [522730,101892] 0 2024-04-12T20:56:25.494 INFO:tasks.workunit.client.1.smithi186.stdout:5/128: creat d5/db/d14/f29 x:0 0 0 2024-04-12T20:56:25.494 INFO:tasks.workunit.client.1.smithi186.stdout:7/171: creat d7/d10/f41 x:0 0 0 2024-04-12T20:56:25.494 INFO:tasks.workunit.client.0.smithi018.stdout:0/977: creat da/d15/d17/d21/d94/f14a x:0 0 0 2024-04-12T20:56:25.495 INFO:tasks.workunit.client.1.smithi186.stdout:8/102: chown d7/lb 0 1 2024-04-12T20:56:25.500 INFO:tasks.workunit.client.0.smithi018.stdout:4/887: truncate d3/d5d/d60/dc5/d43/d4b/ddc/f102 259348 0 2024-04-12T20:56:25.500 INFO:tasks.workunit.client.0.smithi018.stdout:7/975: rename d1/db/d31/d30/c92 to d1/d6/c13a 0 2024-04-12T20:56:25.501 INFO:tasks.workunit.client.1.smithi186.stdout:9/116: creat d3/d12/d16/f1e x:0 0 0 2024-04-12T20:56:25.502 INFO:tasks.workunit.client.1.smithi186.stdout:8/103: mkdir d7/d8/d20 0 2024-04-12T20:56:25.503 INFO:tasks.workunit.client.1.smithi186.stdout:5/129: chown d5/db/l1d 101449 1 2024-04-12T20:56:25.504 INFO:tasks.workunit.client.0.smithi018.stdout:0/978: write da/d15/d17/d21/d36/d47/d81/fa7 [1528219,103421] 0 2024-04-12T20:56:25.506 INFO:tasks.workunit.client.1.smithi186.stdout:6/114: sync 2024-04-12T20:56:25.506 INFO:tasks.workunit.client.1.smithi186.stdout:0/203: sync 2024-04-12T20:56:25.507 INFO:tasks.workunit.client.0.smithi018.stdout:7/976: truncate d1/f2f 1260210 0 2024-04-12T20:56:25.508 INFO:tasks.workunit.client.1.smithi186.stdout:2/137: dwrite d0/f14 [0,4194304] 0 2024-04-12T20:56:25.508 INFO:tasks.workunit.client.1.smithi186.stdout:8/104: unlink c6 0 2024-04-12T20:56:25.511 INFO:tasks.workunit.client.1.smithi186.stdout:2/138: read d0/d3/d1a/f20 [451677,84511] 0 2024-04-12T20:56:25.511 INFO:tasks.workunit.client.1.smithi186.stdout:7/172: creat d7/d10/f42 x:0 0 0 2024-04-12T20:56:25.513 INFO:tasks.workunit.client.1.smithi186.stdout:2/139: readlink d0/l23 0 2024-04-12T20:56:25.514 INFO:tasks.workunit.client.1.smithi186.stdout:0/204: write d0/d10/d1c/d1e/f1f [5189195,20255] 0 2024-04-12T20:56:25.514 INFO:tasks.workunit.client.1.smithi186.stdout:7/173: chown d7/fc 1 1 2024-04-12T20:56:25.515 INFO:tasks.workunit.client.1.smithi186.stdout:6/115: unlink d9/c13 0 2024-04-12T20:56:25.515 INFO:tasks.workunit.client.0.smithi018.stdout:2/867: link d5/d8/d10/d43/de2/d24/d39/l49 d5/d8/d10/d43/de2/d24/d39/d9c/l12c 0 2024-04-12T20:56:25.515 INFO:tasks.workunit.client.0.smithi018.stdout:0/979: creat da/d15/d17/d71/dc1/dee/f14b x:0 0 0 2024-04-12T20:56:25.516 INFO:tasks.workunit.client.1.smithi186.stdout:9/117: dread d3/fb [0,4194304] 0 2024-04-12T20:56:25.519 INFO:tasks.workunit.client.1.smithi186.stdout:5/130: getdents d5 0 2024-04-12T20:56:25.520 INFO:tasks.workunit.client.1.smithi186.stdout:7/174: truncate d7/d10/d1c/f20 1613510 0 2024-04-12T20:56:25.521 INFO:tasks.workunit.client.1.smithi186.stdout:1/145: write fb [3229273,25712] 0 2024-04-12T20:56:25.522 INFO:tasks.workunit.client.0.smithi018.stdout:4/888: rename d3/d5d/d60/dc5/d43/d4b/de6 to d3/d5d/d129 0 2024-04-12T20:56:25.523 INFO:tasks.workunit.client.0.smithi018.stdout:4/889: chown d3/db/de3 261 1 2024-04-12T20:56:25.528 INFO:tasks.workunit.client.0.smithi018.stdout:0/980: creat da/d15/d17/d21/d94/f14c x:0 0 0 2024-04-12T20:56:25.529 INFO:tasks.workunit.client.1.smithi186.stdout:9/118: rmdir d3 39 2024-04-12T20:56:25.530 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:56:25 smithi186 ceph-mon[36597]: pgmap v45: 65 pgs: 65 active+clean; 1.9 GiB data, 7.1 GiB used, 529 GiB / 536 GiB avail; 51 MiB/s rd, 108 MiB/s wr, 326 op/s 2024-04-12T20:56:25.532 INFO:tasks.workunit.client.1.smithi186.stdout:3/158: dwrite d1/f19 [4194304,4194304] 0 2024-04-12T20:56:25.532 INFO:tasks.workunit.client.0.smithi018.stdout:2/868: mknod d5/d7/d79/c12d 0 2024-04-12T20:56:25.533 INFO:tasks.workunit.client.0.smithi018.stdout:0/981: write da/d15/d17/d21/d36/da2/dc3/d3f/f141 [334362,72012] 0 2024-04-12T20:56:25.533 INFO:tasks.workunit.client.1.smithi186.stdout:6/116: link d9/ca d9/c27 0 2024-04-12T20:56:25.538 INFO:tasks.workunit.client.0.smithi018.stdout:2/869: write d5/d12/f96 [746255,82531] 0 2024-04-12T20:56:25.539 INFO:tasks.workunit.client.0.smithi018.stdout:7/977: rename d1/db/d31/d45/d8c/d9d/d70/d96/d5f/da9/dfc/c12f to d1/db/d31/d30/dc6/d100/d12a/d4c/d6c/c13b 0 2024-04-12T20:56:25.539 INFO:tasks.workunit.client.1.smithi186.stdout:8/105: dwrite d7/d18/f1e [0,4194304] 0 2024-04-12T20:56:25.539 INFO:tasks.workunit.client.1.smithi186.stdout:4/236: dwrite d0/dc/d33/f4 [0,4194304] 0 2024-04-12T20:56:25.540 INFO:tasks.workunit.client.0.smithi018.stdout:2/870: readlink d5/d8/d10/d43/de2/daf/ld3 0 2024-04-12T20:56:25.543 INFO:tasks.workunit.client.1.smithi186.stdout:6/117: chown d9/f1b 0 1 2024-04-12T20:56:25.543 INFO:tasks.workunit.client.0.smithi018.stdout:1/771: truncate d3/d9/da/dd/d31/d32/d13/d19/d2d/d4e/d7a/d82/fb5 113211 0 2024-04-12T20:56:25.543 INFO:tasks.workunit.client.1.smithi186.stdout:7/175: write d7/d10/f19 [583642,98368] 0 2024-04-12T20:56:25.544 INFO:tasks.workunit.client.1.smithi186.stdout:2/140: dread d0/f6 [0,4194304] 0 2024-04-12T20:56:25.545 INFO:tasks.workunit.client.1.smithi186.stdout:0/205: getdents d0/d10/d32 0 2024-04-12T20:56:25.548 INFO:tasks.workunit.client.0.smithi018.stdout:0/982: mkdir da/d15/d17/d5c/dae/d14d 0 2024-04-12T20:56:25.548 INFO:tasks.workunit.client.1.smithi186.stdout:4/237: write d0/dc/d33/f43 [57589,93394] 0 2024-04-12T20:56:25.548 INFO:tasks.workunit.client.1.smithi186.stdout:0/206: stat d0/d10/d1c/d1e/l20 0 2024-04-12T20:56:25.550 INFO:tasks.workunit.client.1.smithi186.stdout:1/146: symlink d14/d1a/d21/d2c/l2e 0 2024-04-12T20:56:25.551 INFO:tasks.workunit.client.0.smithi018.stdout:7/978: chown d1/db/d31/d45/d8c/d9d/d70/d96/cea 490941 1 2024-04-12T20:56:25.553 INFO:tasks.workunit.client.0.smithi018.stdout:1/772: write d3/d9/da/dd/d31/d32/d13/d19/d2d/d4e/d5e/d97/fc5 [1228113,80580] 0 2024-04-12T20:56:25.554 INFO:tasks.workunit.client.1.smithi186.stdout:3/159: mkdir d1/d3/d21/d22/d24/d2f 0 2024-04-12T20:56:25.554 INFO:tasks.workunit.client.1.smithi186.stdout:9/119: write d3/d12/f1c [436958,116647] 0 2024-04-12T20:56:25.556 INFO:tasks.workunit.client.1.smithi186.stdout:4/238: mknod d0/dc/d1f/c4b 0 2024-04-12T20:56:25.556 INFO:tasks.workunit.client.1.smithi186.stdout:7/176: truncate d7/f27 805444 0 2024-04-12T20:56:25.557 INFO:tasks.workunit.client.1.smithi186.stdout:0/207: creat d0/d10/f43 x:0 0 0 2024-04-12T20:56:25.565 INFO:tasks.workunit.client.0.smithi018.stdout:1/773: symlink d3/d9/da/dd/d31/d32/d13/d19/d2d/d4e/d5e/d97/l108 0 2024-04-12T20:56:25.566 INFO:tasks.workunit.client.0.smithi018.stdout:7/979: dread d1/db/d31/d45/d8c/d9d/d69/fd7 [0,4194304] 0 2024-04-12T20:56:25.567 INFO:tasks.workunit.client.1.smithi186.stdout:9/120: mkdir d3/d13/d1f 0 2024-04-12T20:56:25.569 INFO:tasks.workunit.client.1.smithi186.stdout:9/121: write d3/f10 [6930606,32957] 0 2024-04-12T20:56:25.570 INFO:tasks.workunit.client.1.smithi186.stdout:2/141: dwrite d0/d3/d1a/f20 [0,4194304] 0 2024-04-12T20:56:25.572 INFO:tasks.workunit.client.1.smithi186.stdout:7/177: creat d7/f43 x:0 0 0 2024-04-12T20:56:25.572 INFO:tasks.workunit.client.0.smithi018.stdout:4/890: write d3/d32/fac [671630,77680] 0 2024-04-12T20:56:25.572 INFO:tasks.workunit.client.0.smithi018.stdout:1/774: fsync d3/d9/da/dd/d31/f59 0 2024-04-12T20:56:25.572 INFO:tasks.workunit.client.0.smithi018.stdout:4/891: chown d3/f52 1725 1 2024-04-12T20:56:25.573 INFO:tasks.workunit.client.1.smithi186.stdout:9/122: write d3/d13/f18 [451770,28631] 0 2024-04-12T20:56:25.574 INFO:tasks.workunit.client.1.smithi186.stdout:3/160: rename d1/c1c to d1/d3/d21/d22/d24/d28/c30 0 2024-04-12T20:56:25.578 INFO:tasks.workunit.client.1.smithi186.stdout:3/161: stat d1/d3/d21/d22/d24/d28 0 2024-04-12T20:56:25.578 INFO:tasks.workunit.client.1.smithi186.stdout:4/239: symlink d0/d21/d3d/l4c 0 2024-04-12T20:56:25.578 INFO:tasks.workunit.client.1.smithi186.stdout:1/147: rmdir d14 39 2024-04-12T20:56:25.580 INFO:tasks.workunit.client.1.smithi186.stdout:0/208: unlink d0/d10/f15 0 2024-04-12T20:56:25.581 INFO:tasks.workunit.client.1.smithi186.stdout:8/106: link d7/d8/d17/f1c d7/de/f21 0 2024-04-12T20:56:25.582 INFO:tasks.workunit.client.0.smithi018.stdout:0/983: dwrite da/d15/d17/d71/dc1/dee/f104 [0,4194304] 0 2024-04-12T20:56:25.583 INFO:tasks.workunit.client.1.smithi186.stdout:6/118: dwrite d9/fb [0,4194304] 0 2024-04-12T20:56:25.584 INFO:tasks.workunit.client.0.smithi018.stdout:1/775: write d3/d9/da/dd/d31/d32/d13/d19/d2d/d4e/d7a/df5/ff9 [480455,118471] 0 2024-04-12T20:56:25.586 INFO:tasks.workunit.client.0.smithi018.stdout:7/980: creat d1/db/d31/d45/d8c/d9d/d70/d96/d5f/da9/ddf/f13c x:0 0 0 2024-04-12T20:56:25.587 INFO:tasks.workunit.client.0.smithi018.stdout:2/871: link d5/d12/d54/f53 d5/d7/d31/d62/da6/df8/f12e 0 2024-04-12T20:56:25.588 INFO:tasks.workunit.client.1.smithi186.stdout:0/209: read d0/f21 [228495,85634] 0 2024-04-12T20:56:25.589 INFO:tasks.workunit.client.1.smithi186.stdout:2/142: mknod d0/dc/c2b 0 2024-04-12T20:56:25.590 INFO:tasks.workunit.client.1.smithi186.stdout:7/178: creat d7/d10/d2d/f44 x:0 0 0 2024-04-12T20:56:25.593 INFO:tasks.workunit.client.0.smithi018.stdout:0/984: write da/d15/d17/d21/d36/f7c [1685491,88502] 0 2024-04-12T20:56:25.593 INFO:tasks.workunit.client.0.smithi018.stdout:2/872: truncate d5/d8/d10/d43/de2/d24/d39/f108 158579 0 2024-04-12T20:56:25.597 INFO:tasks.workunit.client.1.smithi186.stdout:9/123: write d3/fd [1032795,57595] 0 2024-04-12T20:56:25.603 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:56:25 smithi018 ceph-mon[28799]: pgmap v45: 65 pgs: 65 active+clean; 1.9 GiB data, 7.1 GiB used, 529 GiB / 536 GiB avail; 51 MiB/s rd, 108 MiB/s wr, 326 op/s 2024-04-12T20:56:25.603 INFO:tasks.workunit.client.0.smithi018.stdout:4/892: mknod d3/d5d/d60/dc5/d3d/de4/c12a 0 2024-04-12T20:56:25.604 INFO:tasks.workunit.client.0.smithi018.stdout:7/981: fdatasync d1/db/d31/d45/d8c/d9d/d70/d96/d5f/da9/fe3 0 2024-04-12T20:56:25.604 INFO:tasks.workunit.client.0.smithi018.stdout:7/982: stat d1/db/d31/d45/d8c/d9d/d70 0 2024-04-12T20:56:25.604 INFO:tasks.workunit.client.1.smithi186.stdout:8/107: dwrite d7/d18/f1a [4194304,4194304] 0 2024-04-12T20:56:25.605 INFO:tasks.workunit.client.0.smithi018.stdout:4/893: stat d3/d5d/d60/dc5/d3d/l58 0 2024-04-12T20:56:25.613 INFO:tasks.workunit.client.0.smithi018.stdout:1/776: getdents d3/d9/da/dd/d31/d32/d33/d84 0 2024-04-12T20:56:25.617 INFO:tasks.workunit.client.0.smithi018.stdout:1/777: write d3/d9/da/dd/f55 [1718920,62758] 0 2024-04-12T20:56:25.620 INFO:tasks.workunit.client.1.smithi186.stdout:2/143: dwrite d0/f14 [0,4194304] 0 2024-04-12T20:56:25.620 INFO:tasks.workunit.client.1.smithi186.stdout:7/179: dwrite d7/f12 [0,4194304] 0 2024-04-12T20:56:25.621 INFO:tasks.workunit.client.0.smithi018.stdout:4/894: creat d3/d5d/d60/dc5/d53/de7/d126/d4f/f12b x:0 0 0 2024-04-12T20:56:25.631 INFO:tasks.workunit.client.0.smithi018.stdout:0/985: dwrite da/d15/d17/d71/dc1/dee/df7/d140/f11c [0,4194304] 0 2024-04-12T20:56:25.646 INFO:tasks.workunit.client.0.smithi018.stdout:4/895: creat d3/d5d/d60/dc5/d40/f12c x:0 0 0 2024-04-12T20:56:25.648 INFO:tasks.workunit.client.0.smithi018.stdout:1/778: dwrite d3/d9/da/d14/fdb [8388608,4194304] 0 2024-04-12T20:56:25.651 INFO:tasks.workunit.client.1.smithi186.stdout:7/180: dread d7/f17 [0,4194304] 0 2024-04-12T20:56:25.652 INFO:tasks.workunit.client.0.smithi018.stdout:0/986: symlink da/d15/d17/d71/dc1/dee/df7/d140/l14e 0 2024-04-12T20:56:25.652 INFO:tasks.workunit.client.1.smithi186.stdout:5/131: sync 2024-04-12T20:56:25.652 INFO:tasks.workunit.client.0.smithi018.stdout:4/896: read d3/d32/f81 [5003241,88734] 0 2024-04-12T20:56:25.654 INFO:tasks.workunit.client.1.smithi186.stdout:7/181: chown d7/d10/d2d 1328449 1 2024-04-12T20:56:25.654 INFO:tasks.workunit.client.0.smithi018.stdout:0/987: chown da/f38 665409 1 2024-04-12T20:56:25.657 INFO:tasks.workunit.client.0.smithi018.stdout:7/983: write d1/db/d31/d30/dc6/d100/d12a/f75 [4108555,19084] 0 2024-04-12T20:56:25.658 INFO:tasks.workunit.client.1.smithi186.stdout:7/182: stat d7/d10/d1c/f21 0 2024-04-12T20:56:25.664 INFO:tasks.workunit.client.0.smithi018.stdout:1/779: creat d3/d9/da/dd/d31/d32/d13/d19/d2d/f109 x:0 0 0 2024-04-12T20:56:25.671 INFO:tasks.workunit.client.0.smithi018.stdout:1/780: creat d3/d9/da/dd/d31/d32/d33/d84/dc2/f10a x:0 0 0 2024-04-12T20:56:25.672 INFO:tasks.workunit.client.0.smithi018.stdout:7/984: link d1/db/d31/d45/d8c/d9d/d70/d96/d5f/da9/ddf/f13c d1/db/d31/d45/f13d 0 2024-04-12T20:56:25.676 INFO:tasks.workunit.client.0.smithi018.stdout:4/897: getdents d3/d5d/d60/dc5/dcc 0 2024-04-12T20:56:25.677 INFO:tasks.workunit.client.0.smithi018.stdout:0/988: dwrite da/d15/d17/d21/d36/d47/db5/f113 [0,4194304] 0 2024-04-12T20:56:25.685 INFO:tasks.workunit.client.0.smithi018.stdout:0/989: truncate da/d15/d17/d71/dc1/dcc/f87 534344 0 2024-04-12T20:56:25.697 INFO:tasks.workunit.client.0.smithi018.stdout:4/898: write d3/faa [4868991,60584] 0 2024-04-12T20:56:25.698 INFO:tasks.workunit.client.0.smithi018.stdout:7/985: truncate d1/d6/f67 812280 0 2024-04-12T20:56:25.698 INFO:tasks.workunit.client.1.smithi186.stdout:0/210: stat d0/d14/c39 0 2024-04-12T20:56:25.703 INFO:tasks.workunit.client.1.smithi186.stdout:9/124: rename d3/ca to d3/c20 0 2024-04-12T20:56:25.703 INFO:tasks.workunit.client.1.smithi186.stdout:1/148: mknod d14/d1a/d21/c2f 0 2024-04-12T20:56:25.704 INFO:tasks.workunit.client.0.smithi018.stdout:4/899: mkdir d3/d5d/d60/dc5/d3d/da3/d12d 0 2024-04-12T20:56:25.704 INFO:tasks.workunit.client.1.smithi186.stdout:8/108: creat d7/d8/d17/f22 x:0 0 0 2024-04-12T20:56:25.705 INFO:tasks.workunit.client.0.smithi018.stdout:1/781: dread d3/d9/da/dd/f9d [0,4194304] 0 2024-04-12T20:56:25.709 INFO:tasks.workunit.client.1.smithi186.stdout:5/132: fsync f2 0 2024-04-12T20:56:25.711 INFO:tasks.workunit.client.1.smithi186.stdout:7/183: mkdir d7/d10/d2d/d45 0 2024-04-12T20:56:25.711 INFO:tasks.workunit.client.0.smithi018.stdout:1/782: mkdir d3/d9/dc3/d10b 0 2024-04-12T20:56:25.713 INFO:tasks.workunit.client.1.smithi186.stdout:0/211: dwrite d0/f3e [0,4194304] 0 2024-04-12T20:56:25.715 INFO:tasks.workunit.client.1.smithi186.stdout:1/149: mknod d14/d1a/c30 0 2024-04-12T20:56:25.719 INFO:tasks.workunit.client.0.smithi018.stdout:7/986: write d1/db/f43 [671570,47457] 0 2024-04-12T20:56:25.721 INFO:tasks.workunit.client.0.smithi018.stdout:4/900: link d3/d5d/d60/dc5/d43/c78 d3/db/c12e 0 2024-04-12T20:56:25.721 INFO:tasks.workunit.client.0.smithi018.stdout:0/990: dread da/d15/d17/d21/f46 [0,4194304] 0 2024-04-12T20:56:25.723 INFO:tasks.workunit.client.1.smithi186.stdout:2/144: dwrite d0/dc/f29 [0,4194304] 0 2024-04-12T20:56:25.724 INFO:tasks.workunit.client.0.smithi018.stdout:7/987: stat d1/db/d31/d30/dc6/fc8 0 2024-04-12T20:56:25.729 INFO:tasks.workunit.client.1.smithi186.stdout:8/109: mkdir d7/d8/d20/d23 0 2024-04-12T20:56:25.733 INFO:tasks.workunit.client.1.smithi186.stdout:1/150: creat d14/d1a/d21/f31 x:0 0 0 2024-04-12T20:56:25.736 INFO:tasks.workunit.client.1.smithi186.stdout:1/151: dread - d14/d1a/d1d/f29 zero size 2024-04-12T20:56:25.736 INFO:tasks.workunit.client.0.smithi018.stdout:0/991: getdents da/d15/d17/d21/d36/da2/dc3/d3f/de3 0 2024-04-12T20:56:25.737 INFO:tasks.workunit.client.1.smithi186.stdout:8/110: truncate d7/de/d10/f19 857566 0 2024-04-12T20:56:25.737 INFO:tasks.workunit.client.1.smithi186.stdout:7/184: write d7/d10/f13 [859172,36814] 0 2024-04-12T20:56:25.739 INFO:tasks.workunit.client.1.smithi186.stdout:6/119: getdents d9 0 2024-04-12T20:56:25.739 INFO:tasks.workunit.client.1.smithi186.stdout:7/185: fdatasync d7/d10/f19 0 2024-04-12T20:56:25.740 INFO:tasks.workunit.client.0.smithi018.stdout:0/992: readlink da/d15/d17/d21/d36/da2/dc3/d3f/lcd 0 2024-04-12T20:56:25.740 INFO:tasks.workunit.client.1.smithi186.stdout:6/120: chown d9/f1b 16838139 1 2024-04-12T20:56:25.742 INFO:tasks.workunit.client.1.smithi186.stdout:0/212: symlink d0/l44 0 2024-04-12T20:56:25.745 INFO:tasks.workunit.client.0.smithi018.stdout:7/988: dread d1/db/d31/d3a/d97/fb1 [0,4194304] 0 2024-04-12T20:56:25.745 INFO:tasks.workunit.client.1.smithi186.stdout:5/133: mkdir d5/db/d15/d2a 0 2024-04-12T20:56:25.745 INFO:tasks.workunit.client.1.smithi186.stdout:9/125: dread d3/f5 [0,4194304] 0 2024-04-12T20:56:25.749 INFO:tasks.workunit.client.1.smithi186.stdout:9/126: dread - d3/f19 zero size 2024-04-12T20:56:25.749 INFO:tasks.workunit.client.1.smithi186.stdout:9/127: write d3/f1d [1002195,119047] 0 2024-04-12T20:56:25.750 INFO:tasks.workunit.client.0.smithi018.stdout:0/993: symlink da/d15/d17/d21/d36/d6e/d12b/l14f 0 2024-04-12T20:56:25.751 INFO:tasks.workunit.client.0.smithi018.stdout:7/989: write d1/db/d31/d30/dc6/d100/d12a/dcd/f11a [1009495,90593] 0 2024-04-12T20:56:25.752 INFO:tasks.workunit.client.0.smithi018.stdout:7/990: readlink d1/d86/la4 0 2024-04-12T20:56:25.754 INFO:tasks.workunit.client.0.smithi018.stdout:0/994: mkdir da/d15/d17/d21/d36/da2/dc3/d3f/d150 0 2024-04-12T20:56:25.755 INFO:tasks.workunit.client.1.smithi186.stdout:7/186: write d7/f16 [9107259,9035] 0 2024-04-12T20:56:25.759 INFO:tasks.workunit.client.1.smithi186.stdout:1/152: unlink c1 0 2024-04-12T20:56:25.760 INFO:tasks.workunit.client.0.smithi018.stdout:7/991: creat d1/db/d31/d30/dc6/d100/d12a/d4c/dbc/f13e x:0 0 0 2024-04-12T20:56:25.760 INFO:tasks.workunit.client.0.smithi018.stdout:0/995: symlink da/d15/d17/l151 0 2024-04-12T20:56:25.761 INFO:tasks.workunit.client.1.smithi186.stdout:1/153: read fe [1309157,5680] 0 2024-04-12T20:56:25.763 INFO:tasks.workunit.client.1.smithi186.stdout:5/134: dwrite d5/db/d15/f26 [0,4194304] 0 2024-04-12T20:56:25.764 INFO:tasks.workunit.client.1.smithi186.stdout:8/111: rename d7/cc to d7/d8/c24 0 2024-04-12T20:56:25.767 INFO:tasks.workunit.client.1.smithi186.stdout:6/121: rename d9/l1c to d9/l28 0 2024-04-12T20:56:25.767 INFO:tasks.workunit.client.0.smithi018.stdout:7/992: write d1/db/d31/f63 [1768439,48634] 0 2024-04-12T20:56:25.768 INFO:tasks.workunit.client.0.smithi018.stdout:1/783: dread d3/d9/da/d14/d73/f90 [0,4194304] 0 2024-04-12T20:56:25.768 INFO:tasks.workunit.client.0.smithi018.stdout:0/996: dread da/d15/d17/d21/d36/d47/db5/ff3 [0,4194304] 0 2024-04-12T20:56:25.770 INFO:tasks.workunit.client.0.smithi018.stdout:1/784: chown d3/c28 27 1 2024-04-12T20:56:25.772 INFO:tasks.workunit.client.1.smithi186.stdout:0/213: dwrite d0/d14/f35 [0,4194304] 0 2024-04-12T20:56:25.773 INFO:tasks.workunit.client.0.smithi018.stdout:4/901: dread d3/d32/d33/dbc/fda [0,4194304] 0 2024-04-12T20:56:25.775 INFO:tasks.workunit.client.0.smithi018.stdout:7/993: symlink d1/db/d31/d45/df4/d124/d11b/l13f 0 2024-04-12T20:56:25.777 INFO:tasks.workunit.client.0.smithi018.stdout:0/997: truncate da/d15/d17/d21/d94/f14c 763346 0 2024-04-12T20:56:25.779 INFO:tasks.workunit.client.0.smithi018.stdout:1/785: dread - d3/d9/da/dd/d31/d32/d13/d19/d2d/d4e/d7a/df5/f103 zero size 2024-04-12T20:56:25.783 INFO:tasks.workunit.client.0.smithi018.stdout:4/902: mkdir d3/db/de3/d12f 0 2024-04-12T20:56:25.789 INFO:tasks.workunit.client.0.smithi018.stdout:1/786: fdatasync d3/d9/da/dd/d31/d32/d13/d19/d2d/d4e/d7a/df5/ff9 0 2024-04-12T20:56:25.789 INFO:tasks.workunit.client.0.smithi018.stdout:0/998: mkdir da/d15/d4d/d12f/d152 0 2024-04-12T20:56:25.790 INFO:tasks.workunit.client.1.smithi186.stdout:6/122: symlink d9/d25/l29 0 2024-04-12T20:56:25.790 INFO:tasks.workunit.client.1.smithi186.stdout:9/128: getdents d3/d12 0 2024-04-12T20:56:25.790 INFO:tasks.workunit.client.0.smithi018.stdout:4/903: write d3/db/f26 [5157813,17726] 0 2024-04-12T20:56:25.794 INFO:tasks.workunit.client.0.smithi018.stdout:1/787: write d3/d9/da/dd/d26/df0/f106 [878187,95819] 0 2024-04-12T20:56:25.795 INFO:tasks.workunit.client.0.smithi018.stdout:0/999: mkdir da/d15/d17/d71/dc1/dee/df7/d140/d153 0 2024-04-12T20:56:25.796 INFO:tasks.workunit.client.1.smithi186.stdout:0/214: mknod d0/c45 0 2024-04-12T20:56:25.799 INFO:tasks.workunit.client.1.smithi186.stdout:5/135: creat d5/db/f2b x:0 0 0 2024-04-12T20:56:25.800 INFO:tasks.workunit.client.1.smithi186.stdout:5/136: stat d5/db/l1d 0 2024-04-12T20:56:25.802 INFO:tasks.workunit.client.1.smithi186.stdout:7/187: rename d7/f3f to d7/d10/f46 0 2024-04-12T20:56:25.803 INFO:tasks.workunit.client.0.smithi018.stdout:4/904: rename d3/d32/d33/db6/le9 to d3/d5d/d60/dc5/d43/d4b/l130 0 2024-04-12T20:56:25.804 INFO:tasks.workunit.client.1.smithi186.stdout:7/188: truncate d7/f27 1186508 0 2024-04-12T20:56:25.805 INFO:tasks.workunit.client.0.smithi018.stdout:7/994: dwrite d1/db/d31/d45/d8c/d9d/d70/d96/d5f/da9/ddf/d108/f11d [0,4194304] 0 2024-04-12T20:56:25.805 INFO:tasks.workunit.client.0.smithi018.stdout:1/788: mknod d3/d9/da/dd/d31/d32/d13/d19/d2d/d4e/d5e/da0/de2/c10c 0 2024-04-12T20:56:25.807 INFO:tasks.workunit.client.1.smithi186.stdout:0/215: mkdir d0/d10/d1c/d46 0 2024-04-12T20:56:25.808 INFO:tasks.workunit.client.1.smithi186.stdout:0/216: fsync d0/d10/f2a 0 2024-04-12T20:56:25.809 INFO:tasks.workunit.client.1.smithi186.stdout:5/137: fdatasync d5/db/d14/f29 0 2024-04-12T20:56:25.810 INFO:tasks.workunit.client.0.smithi018.stdout:7/995: mkdir d1/d6/d140 0 2024-04-12T20:56:25.816 INFO:tasks.workunit.client.1.smithi186.stdout:9/129: link f1 d3/d12/d16/f21 0 2024-04-12T20:56:25.816 INFO:tasks.workunit.client.1.smithi186.stdout:9/130: stat d3/d13/f18 0 2024-04-12T20:56:25.818 INFO:tasks.workunit.client.1.smithi186.stdout:0/217: mknod d0/d10/d1c/d1e/c47 0 2024-04-12T20:56:25.819 INFO:tasks.workunit.client.1.smithi186.stdout:5/138: fsync d5/f6 0 2024-04-12T20:56:25.822 INFO:tasks.workunit.client.1.smithi186.stdout:7/189: unlink d7/d10/c1a 0 2024-04-12T20:56:25.825 INFO:tasks.workunit.client.1.smithi186.stdout:9/131: creat d3/d13/f22 x:0 0 0 2024-04-12T20:56:25.826 INFO:tasks.workunit.client.0.smithi018.stdout:7/996: creat d1/db/d31/d45/d8c/d9d/d70/d96/d5f/de4/d137/f141 x:0 0 0 2024-04-12T20:56:25.827 INFO:tasks.workunit.client.1.smithi186.stdout:9/132: write d3/f10 [4673407,71794] 0 2024-04-12T20:56:25.827 INFO:tasks.workunit.client.1.smithi186.stdout:0/218: creat d0/d10/d32/f48 x:0 0 0 2024-04-12T20:56:25.828 INFO:tasks.workunit.client.0.smithi018.stdout:1/789: write d3/d9/da/dd/d31/d32/d13/d19/d2d/d4e/d7a/fa4 [1727992,98509] 0 2024-04-12T20:56:25.828 INFO:tasks.workunit.client.0.smithi018.stdout:4/905: dwrite d3/d5d/d60/dc5/d53/de7/d126/d72/fa0 [0,4194304] 0 2024-04-12T20:56:25.829 INFO:tasks.workunit.client.1.smithi186.stdout:7/190: symlink d7/d10/d1c/l47 0 2024-04-12T20:56:25.832 INFO:tasks.workunit.client.1.smithi186.stdout:7/191: write d7/d10/f41 [117228,122642] 0 2024-04-12T20:56:25.832 INFO:tasks.workunit.client.0.smithi018.stdout:7/997: fdatasync d1/db/d31/d3a/d97/fb1 0 2024-04-12T20:56:25.833 INFO:tasks.workunit.client.1.smithi186.stdout:5/139: mknod d5/db/d15/d2a/c2c 0 2024-04-12T20:56:25.836 INFO:tasks.workunit.client.0.smithi018.stdout:1/790: write d3/d9/da/d14/fdb [8980164,130745] 0 2024-04-12T20:56:25.842 INFO:tasks.workunit.client.1.smithi186.stdout:0/219: mknod d0/d14/c49 0 2024-04-12T20:56:25.846 INFO:tasks.workunit.client.1.smithi186.stdout:7/192: write d7/d10/d1c/f1f [5770278,94277] 0 2024-04-12T20:56:25.849 INFO:tasks.workunit.client.1.smithi186.stdout:5/140: mkdir d5/db/d15/d2a/d2d 0 2024-04-12T20:56:25.849 INFO:tasks.workunit.client.0.smithi018.stdout:4/906: rename d3/d5d/d60/dc5/d43/d4b/df2/f10c to d3/d5d/d60/dc5/d3d/d80/f131 0 2024-04-12T20:56:25.850 INFO:tasks.workunit.client.1.smithi186.stdout:5/141: stat d5/db/d14/f1a 0 2024-04-12T20:56:25.855 INFO:tasks.workunit.client.1.smithi186.stdout:0/220: unlink d0/f21 0 2024-04-12T20:56:25.860 INFO:tasks.workunit.client.1.smithi186.stdout:9/133: rmdir d3/d13/d1f 0 2024-04-12T20:56:25.863 INFO:tasks.workunit.client.0.smithi018.stdout:1/791: truncate d3/d9/da/dd/d31/d32/d13/d19/d2d/d4e/d7a/d82/fed 1807410 0 2024-04-12T20:56:25.866 INFO:tasks.workunit.client.1.smithi186.stdout:7/193: link d7/d10/d1c/c32 d7/d10/d2d/d45/c48 0 2024-04-12T20:56:25.867 INFO:tasks.workunit.client.1.smithi186.stdout:5/142: dwrite d5/db/d15/f1e [0,4194304] 0 2024-04-12T20:56:25.867 INFO:tasks.workunit.client.1.smithi186.stdout:7/194: chown d7/fc 15 1 2024-04-12T20:56:25.869 INFO:tasks.workunit.client.1.smithi186.stdout:0/221: rename d0/l42 to d0/l4a 0 2024-04-12T20:56:25.870 INFO:tasks.workunit.client.0.smithi018.stdout:7/998: dwrite d1/db/d31/d45/fc2 [0,4194304] 0 2024-04-12T20:56:25.872 INFO:tasks.workunit.client.0.smithi018.stdout:1/792: creat d3/d9/dc3/d10b/f10d x:0 0 0 2024-04-12T20:56:25.873 INFO:tasks.workunit.client.1.smithi186.stdout:3/162: truncate d1/fc 3846195 0 2024-04-12T20:56:25.876 INFO:tasks.workunit.client.1.smithi186.stdout:5/143: mkdir d5/db/d2e 0 2024-04-12T20:56:25.877 INFO:tasks.workunit.client.1.smithi186.stdout:4/240: truncate d0/d21/f1b 3694801 0 2024-04-12T20:56:25.879 INFO:tasks.workunit.client.1.smithi186.stdout:5/144: fdatasync d5/db/d15/f26 0 2024-04-12T20:56:25.880 INFO:tasks.workunit.client.1.smithi186.stdout:0/222: creat d0/d10/d1c/d1e/f4b x:0 0 0 2024-04-12T20:56:25.882 INFO:tasks.workunit.client.1.smithi186.stdout:9/134: dwrite d3/f1d [0,4194304] 0 2024-04-12T20:56:25.884 INFO:tasks.workunit.client.0.smithi018.stdout:2/873: sync 2024-04-12T20:56:25.884 INFO:tasks.workunit.client.1.smithi186.stdout:0/223: write d0/d10/f43 [652819,40826] 0 2024-04-12T20:56:25.886 INFO:tasks.workunit.client.0.smithi018.stdout:4/907: dwrite d3/d5d/d60/dc5/d3d/da3/fe2 [0,4194304] 0 2024-04-12T20:56:25.890 INFO:tasks.workunit.client.0.smithi018.stdout:4/908: write d3/d32/fac [1281629,39297] 0 2024-04-12T20:56:25.892 INFO:tasks.workunit.client.1.smithi186.stdout:7/195: symlink d7/d10/l49 0 2024-04-12T20:56:25.893 INFO:tasks.workunit.client.1.smithi186.stdout:3/163: dwrite d1/d3/f11 [0,4194304] 0 2024-04-12T20:56:25.909 INFO:tasks.workunit.client.1.smithi186.stdout:3/164: dread d1/f1e [0,4194304] 0 2024-04-12T20:56:25.909 INFO:tasks.workunit.client.1.smithi186.stdout:3/165: fsync d1/f19 0 2024-04-12T20:56:25.916 INFO:tasks.workunit.client.1.smithi186.stdout:9/135: mknod d3/d13/c23 0 2024-04-12T20:56:25.920 INFO:tasks.workunit.client.1.smithi186.stdout:2/145: truncate d0/dc/f29 2544190 0 2024-04-12T20:56:25.921 INFO:tasks.workunit.client.1.smithi186.stdout:7/196: rmdir d7/d10/d1c 39 2024-04-12T20:56:25.931 INFO:tasks.workunit.client.1.smithi186.stdout:1/154: write ff [1216591,33494] 0 2024-04-12T20:56:25.932 INFO:tasks.workunit.client.1.smithi186.stdout:1/155: chown d14/d1a/d21/d2c 72671 1 2024-04-12T20:56:25.932 INFO:tasks.workunit.client.1.smithi186.stdout:5/145: link d5/fd d5/db/d15/f2f 0 2024-04-12T20:56:25.935 INFO:tasks.workunit.client.1.smithi186.stdout:6/123: getdents d9/d25 0 2024-04-12T20:56:25.942 INFO:tasks.workunit.client.1.smithi186.stdout:9/136: mknod d3/d13/c24 0 2024-04-12T20:56:25.943 INFO:tasks.workunit.client.1.smithi186.stdout:1/156: dread f9 [0,4194304] 0 2024-04-12T20:56:25.943 INFO:tasks.workunit.client.1.smithi186.stdout:0/224: symlink d0/l4c 0 2024-04-12T20:56:25.944 INFO:tasks.workunit.client.1.smithi186.stdout:0/225: readlink d0/d10/d1c/d1e/l20 0 2024-04-12T20:56:25.946 INFO:tasks.workunit.client.0.smithi018.stdout:7/999: mkdir d1/db/d31/d30/dc6/d100/d12a/d4c/d6c/dc3/d142 0 2024-04-12T20:56:25.953 INFO:tasks.workunit.client.0.smithi018.stdout:4/909: link d3/d5d/d60/dc5/d3d/cf8 d3/d32/d94/d10b/c132 0 2024-04-12T20:56:25.961 INFO:tasks.workunit.client.0.smithi018.stdout:4/910: write d3/d5d/d60/dc5/d43/f2a [269041,108524] 0 2024-04-12T20:56:25.962 INFO:tasks.workunit.client.1.smithi186.stdout:5/146: mkdir d5/d30 0 2024-04-12T20:56:25.964 INFO:tasks.workunit.client.1.smithi186.stdout:6/124: rename d9/fb to d9/f2a 0 2024-04-12T20:56:25.969 INFO:tasks.workunit.client.1.smithi186.stdout:9/137: mkdir d3/d12/d25 0 2024-04-12T20:56:25.975 INFO:tasks.workunit.client.1.smithi186.stdout:7/197: mkdir d7/d10/d2d/d40/d4a 0 2024-04-12T20:56:25.975 INFO:tasks.workunit.client.1.smithi186.stdout:2/146: truncate d0/d3/f1f 615600 0 2024-04-12T20:56:25.986 INFO:tasks.workunit.client.0.smithi018.stdout:4/911: dread d3/d5d/d60/dc5/d3d/d80/fa2 [0,4194304] 0 2024-04-12T20:56:25.990 INFO:tasks.workunit.client.1.smithi186.stdout:0/226: dread d0/d10/d1c/d1e/f37 [0,4194304] 0 2024-04-12T20:56:25.992 INFO:tasks.workunit.client.1.smithi186.stdout:0/227: fdatasync d0/d10/d32/f48 0 2024-04-12T20:56:25.992 INFO:tasks.workunit.client.0.smithi018.stdout:4/912: creat d3/d5d/d129/f133 x:0 0 0 2024-04-12T20:56:25.994 INFO:tasks.workunit.client.1.smithi186.stdout:0/228: write d0/d10/f2a [1882112,12967] 0 2024-04-12T20:56:25.995 INFO:tasks.workunit.client.1.smithi186.stdout:0/229: stat d0/d10/f2c 0 2024-04-12T20:56:25.995 INFO:tasks.workunit.client.0.smithi018.stdout:4/913: fsync d3/d5d/d60/dc5/f103 0 2024-04-12T20:56:25.995 INFO:tasks.workunit.client.0.smithi018.stdout:2/874: write d5/d8/d10/d43/de2/daf/d88/fb2 [844050,24679] 0 2024-04-12T20:56:25.998 INFO:tasks.workunit.client.1.smithi186.stdout:3/166: rename d1/d3/d21/d22/d24/c27 to d1/d3/c31 0 2024-04-12T20:56:26.000 INFO:tasks.workunit.client.0.smithi018.stdout:1/793: dwrite d3/d9/da/dd/f12 [0,4194304] 0 2024-04-12T20:56:26.003 INFO:tasks.workunit.client.0.smithi018.stdout:4/914: creat d3/db/d114/f134 x:0 0 0 2024-04-12T20:56:26.007 INFO:tasks.workunit.client.1.smithi186.stdout:5/147: creat d5/db/d15/f31 x:0 0 0 2024-04-12T20:56:26.010 INFO:tasks.workunit.client.1.smithi186.stdout:6/125: mknod d9/d25/c2b 0 2024-04-12T20:56:26.012 INFO:tasks.workunit.client.1.smithi186.stdout:9/138: creat d3/d12/d16/f26 x:0 0 0 2024-04-12T20:56:26.012 INFO:tasks.workunit.client.0.smithi018.stdout:4/915: symlink d3/d32/l135 0 2024-04-12T20:56:26.014 INFO:tasks.workunit.client.1.smithi186.stdout:2/147: creat d0/d3/d1a/d21/f2c x:0 0 0 2024-04-12T20:56:26.014 INFO:tasks.workunit.client.1.smithi186.stdout:8/112: truncate d7/de/f21 676466 0 2024-04-12T20:56:26.014 INFO:tasks.workunit.client.1.smithi186.stdout:7/198: write d7/d10/f15 [4871151,14488] 0 2024-04-12T20:56:26.017 INFO:tasks.workunit.client.0.smithi018.stdout:4/916: write d3/d32/d94/d10b/f121 [463321,120955] 0 2024-04-12T20:56:26.017 INFO:tasks.workunit.client.1.smithi186.stdout:0/230: rename d0/d10/f12 to d0/d10/f4d 0 2024-04-12T20:56:26.018 INFO:tasks.workunit.client.1.smithi186.stdout:3/167: rmdir d1/d3/d21/d22/d24 39 2024-04-12T20:56:26.020 INFO:tasks.workunit.client.1.smithi186.stdout:6/126: symlink d9/l2c 0 2024-04-12T20:56:26.021 INFO:tasks.workunit.client.0.smithi018.stdout:1/794: dread d3/d9/da/dd/d31/d32/d13/d19/d2d/d4e/d58/d61/d71/f91 [0,4194304] 0 2024-04-12T20:56:26.026 INFO:tasks.workunit.client.0.smithi018.stdout:1/795: dread - d3/d9/da/dd/d31/d32/d13/d19/d2d/d4e/d7a/f104 zero size 2024-04-12T20:56:26.026 INFO:tasks.workunit.client.1.smithi186.stdout:7/199: rmdir d7/d10/d2d/d40 39 2024-04-12T20:56:26.027 INFO:tasks.workunit.client.1.smithi186.stdout:5/148: getdents d5/db/d15/d2a/d2d 0 2024-04-12T20:56:26.028 INFO:tasks.workunit.client.0.smithi018.stdout:1/796: read - d3/d65/fea zero size 2024-04-12T20:56:26.028 INFO:tasks.workunit.client.0.smithi018.stdout:4/917: rename d3/d5d/d60/dc5/d53/de7/fef to d3/d5d/d60/f136 0 2024-04-12T20:56:26.029 INFO:tasks.workunit.client.1.smithi186.stdout:9/139: dwrite d3/d13/f17 [0,4194304] 0 2024-04-12T20:56:26.030 INFO:tasks.workunit.client.0.smithi018.stdout:4/918: chown d3/d32/d33/dbc 0 1 2024-04-12T20:56:26.032 INFO:tasks.workunit.client.1.smithi186.stdout:0/231: rmdir d0/d14 39 2024-04-12T20:56:26.033 INFO:tasks.workunit.client.1.smithi186.stdout:6/127: mkdir d9/d25/d2d 0 2024-04-12T20:56:26.033 INFO:tasks.workunit.client.0.smithi018.stdout:1/797: write d3/d9/dc3/d10b/f10d [344274,38268] 0 2024-04-12T20:56:26.036 INFO:tasks.workunit.client.1.smithi186.stdout:1/157: write d14/d1a/f1b [1154800,88924] 0 2024-04-12T20:56:26.037 INFO:tasks.workunit.client.0.smithi018.stdout:4/919: symlink d3/d5d/d60/dc5/d43/d4b/ddc/l137 0 2024-04-12T20:56:26.040 INFO:tasks.workunit.client.1.smithi186.stdout:2/148: creat d0/d3/d1a/d21/d26/f2d x:0 0 0 2024-04-12T20:56:26.044 INFO:tasks.workunit.client.1.smithi186.stdout:4/241: link d0/d21/f1b d0/f4d 0 2024-04-12T20:56:26.046 INFO:tasks.workunit.client.1.smithi186.stdout:4/242: truncate d0/f42 678964 0 2024-04-12T20:56:26.046 INFO:tasks.workunit.client.0.smithi018.stdout:2/875: dwrite d5/d8/d10/d43/de2/d24/d39/dab/f105 [0,4194304] 0 2024-04-12T20:56:26.047 INFO:tasks.workunit.client.1.smithi186.stdout:9/140: fsync d3/fb 0 2024-04-12T20:56:26.048 INFO:tasks.workunit.client.1.smithi186.stdout:9/141: readlink l0 0 2024-04-12T20:56:26.048 INFO:tasks.workunit.client.0.smithi018.stdout:4/920: rmdir d3/d5d/d60/dc5/d3d/d80/d111 39 2024-04-12T20:56:26.049 INFO:tasks.workunit.client.1.smithi186.stdout:9/142: fdatasync d3/ff 0 2024-04-12T20:56:26.050 INFO:tasks.workunit.client.1.smithi186.stdout:8/113: mkdir d7/d8/d20/d23/d25 0 2024-04-12T20:56:26.054 INFO:tasks.workunit.client.1.smithi186.stdout:1/158: dwrite d14/f16 [4194304,4194304] 0 2024-04-12T20:56:26.054 INFO:tasks.workunit.client.1.smithi186.stdout:6/128: creat d9/f2e x:0 0 0 2024-04-12T20:56:26.056 INFO:tasks.workunit.client.1.smithi186.stdout:3/168: write d1/f1e [1449028,92629] 0 2024-04-12T20:56:26.057 INFO:tasks.workunit.client.1.smithi186.stdout:2/149: mkdir d0/dc/d2e 0 2024-04-12T20:56:26.058 INFO:tasks.workunit.client.1.smithi186.stdout:3/169: stat d1/f19 0 2024-04-12T20:56:26.059 INFO:tasks.workunit.client.0.smithi018.stdout:1/798: dread d3/d9/da/dd/d31/d32/d13/d19/d2d/d4e/d58/d61/d71/fdf [0,4194304] 0 2024-04-12T20:56:26.059 INFO:tasks.workunit.client.1.smithi186.stdout:1/159: truncate d14/d1a/d21/f31 231471 0 2024-04-12T20:56:26.060 INFO:tasks.workunit.client.0.smithi018.stdout:4/921: getdents d3/d5d/d60/dc5/d3d/da3 0 2024-04-12T20:56:26.061 INFO:tasks.workunit.client.1.smithi186.stdout:7/200: rename d7/f27 to d7/d10/d2d/d40/f4b 0 2024-04-12T20:56:26.064 INFO:tasks.workunit.client.0.smithi018.stdout:4/922: chown d3/d5d/d60/dc5/d43/d4b/df2 6317691 1 2024-04-12T20:56:26.064 INFO:tasks.workunit.client.0.smithi018.stdout:2/876: read d5/d8/d10/d43/de2/d24/d39/d9c/fb0 [1734064,49716] 0 2024-04-12T20:56:26.066 INFO:tasks.workunit.client.1.smithi186.stdout:5/149: truncate d5/f6 5453056 0 2024-04-12T20:56:26.067 INFO:tasks.workunit.client.0.smithi018.stdout:2/877: chown d5/c5a 7523200 1 2024-04-12T20:56:26.068 INFO:tasks.workunit.client.1.smithi186.stdout:9/143: mkdir d3/d12/d27 0 2024-04-12T20:56:26.070 INFO:tasks.workunit.client.1.smithi186.stdout:3/170: stat d1/d3/da/c26 0 2024-04-12T20:56:26.071 INFO:tasks.workunit.client.1.smithi186.stdout:9/144: write d3/f10 [6501347,62766] 0 2024-04-12T20:56:26.073 INFO:tasks.workunit.client.1.smithi186.stdout:0/232: rename d0/d10/d1c to d0/d14/d4e 0 2024-04-12T20:56:26.074 INFO:tasks.workunit.client.1.smithi186.stdout:4/243: symlink d0/dc/l4e 0 2024-04-12T20:56:26.074 INFO:tasks.workunit.client.0.smithi018.stdout:2/878: mkdir d5/d12/d54/dc0/d12f 0 2024-04-12T20:56:26.075 INFO:tasks.workunit.client.1.smithi186.stdout:8/114: mkdir d7/d8/d1d/d1f/d26 0 2024-04-12T20:56:26.076 INFO:tasks.workunit.client.1.smithi186.stdout:6/129: mknod d9/d25/d2d/c2f 0 2024-04-12T20:56:26.076 INFO:tasks.workunit.client.1.smithi186.stdout:4/244: fsync d0/dc/d33/f36 0 2024-04-12T20:56:26.077 INFO:tasks.workunit.client.1.smithi186.stdout:1/160: read f4 [951803,26899] 0 2024-04-12T20:56:26.079 INFO:tasks.workunit.client.1.smithi186.stdout:4/245: dread - d0/d21/f41 zero size 2024-04-12T20:56:26.080 INFO:tasks.workunit.client.1.smithi186.stdout:7/201: dwrite d7/d10/d2d/f37 [0,4194304] 0 2024-04-12T20:56:26.081 INFO:tasks.workunit.client.1.smithi186.stdout:4/246: write d0/dc/f2f [813902,114485] 0 2024-04-12T20:56:26.081 INFO:tasks.workunit.client.0.smithi018.stdout:1/799: write d3/d9/da/dd/d31/d32/d13/d19/d2d/d4e/d7a/d82/ff6 [179102,108923] 0 2024-04-12T20:56:26.082 INFO:tasks.workunit.client.0.smithi018.stdout:4/923: truncate d3/d32/d94/d10b/f121 531760 0 2024-04-12T20:56:26.083 INFO:tasks.workunit.client.1.smithi186.stdout:2/150: symlink d0/d3/d1a/d27/l2f 0 2024-04-12T20:56:26.084 INFO:tasks.workunit.client.0.smithi018.stdout:2/879: read d5/d8/d10/d43/de2/d24/f4b [85880,3299] 0 2024-04-12T20:56:26.085 INFO:tasks.workunit.client.0.smithi018.stdout:4/924: write d3/fff [920441,80198] 0 2024-04-12T20:56:26.088 INFO:tasks.workunit.client.0.smithi018.stdout:1/800: readlink d3/d9/da/dd/l16 0 2024-04-12T20:56:26.093 INFO:tasks.workunit.client.0.smithi018.stdout:1/801: creat d3/f10e x:0 0 0 2024-04-12T20:56:26.093 INFO:tasks.workunit.client.1.smithi186.stdout:6/130: mknod d9/d25/c30 0 2024-04-12T20:56:26.093 INFO:tasks.workunit.client.1.smithi186.stdout:5/150: mknod d5/db/d15/d2a/d2d/c32 0 2024-04-12T20:56:26.094 INFO:tasks.workunit.client.0.smithi018.stdout:2/880: mknod d5/d8/d10/d43/de2/daf/d88/c130 0 2024-04-12T20:56:26.096 INFO:tasks.workunit.client.0.smithi018.stdout:4/925: unlink d3/db/c12e 0 2024-04-12T20:56:26.097 INFO:tasks.workunit.client.1.smithi186.stdout:5/151: chown d5/db/d14/f1a 19725983 1 2024-04-12T20:56:26.097 INFO:tasks.workunit.client.1.smithi186.stdout:4/247: dread d0/dc/d33/f4 [0,4194304] 0 2024-04-12T20:56:26.097 INFO:tasks.workunit.client.0.smithi018.stdout:1/802: symlink d3/d65/l10f 0 2024-04-12T20:56:26.099 INFO:tasks.workunit.client.1.smithi186.stdout:7/202: dwrite d7/fe [0,4194304] 0 2024-04-12T20:56:26.099 INFO:tasks.workunit.client.1.smithi186.stdout:2/151: truncate d0/d3/f8 506504 0 2024-04-12T20:56:26.102 INFO:tasks.workunit.client.0.smithi018.stdout:1/803: stat d3/d9/da/dd/d26/df0/fe7 0 2024-04-12T20:56:26.103 INFO:tasks.workunit.client.0.smithi018.stdout:1/804: fsync d3/d65/fea 0 2024-04-12T20:56:26.105 INFO:tasks.workunit.client.0.smithi018.stdout:4/926: unlink d3/d5d/d60/dc5/d43/d4b/l87 0 2024-04-12T20:56:26.107 INFO:tasks.workunit.client.1.smithi186.stdout:8/115: write d7/fd [3849654,97951] 0 2024-04-12T20:56:26.111 INFO:tasks.workunit.client.0.smithi018.stdout:1/805: symlink d3/d9/da/dd/l110 0 2024-04-12T20:56:26.114 INFO:tasks.workunit.client.1.smithi186.stdout:6/131: rmdir d9/d25 39 2024-04-12T20:56:26.115 INFO:tasks.workunit.client.0.smithi018.stdout:4/927: creat d3/d5d/d60/dc5/d53/de7/d126/d72/f138 x:0 0 0 2024-04-12T20:56:26.115 INFO:tasks.workunit.client.0.smithi018.stdout:2/881: dread d5/d8/d10/d43/de2/d24/d39/dab/f105 [0,4194304] 0 2024-04-12T20:56:26.117 INFO:tasks.workunit.client.1.smithi186.stdout:8/116: dread d7/d18/f1e [0,4194304] 0 2024-04-12T20:56:26.119 INFO:tasks.workunit.client.1.smithi186.stdout:4/248: creat d0/d26/f4f x:0 0 0 2024-04-12T20:56:26.121 INFO:tasks.workunit.client.1.smithi186.stdout:3/171: write d1/fc [1920754,100183] 0 2024-04-12T20:56:26.122 INFO:tasks.workunit.client.1.smithi186.stdout:4/249: write d0/dc/d33/f46 [55772,19073] 0 2024-04-12T20:56:26.123 INFO:tasks.workunit.client.0.smithi018.stdout:4/928: write d3/d5d/d60/dc5/d53/de7/d126/d4f/f12b [592350,53025] 0 2024-04-12T20:56:26.124 INFO:tasks.workunit.client.1.smithi186.stdout:7/203: dwrite d7/d10/d2d/f37 [4194304,4194304] 0 2024-04-12T20:56:26.125 INFO:tasks.workunit.client.0.smithi018.stdout:2/882: truncate d5/fc6 783053 0 2024-04-12T20:56:26.129 INFO:tasks.workunit.client.1.smithi186.stdout:0/233: creat d0/f4f x:0 0 0 2024-04-12T20:56:26.134 INFO:tasks.workunit.client.1.smithi186.stdout:1/161: link d14/d1a/d21/l26 d14/l32 0 2024-04-12T20:56:26.134 INFO:tasks.workunit.client.0.smithi018.stdout:2/883: symlink d5/d8/d10/d43/l131 0 2024-04-12T20:56:26.137 INFO:tasks.workunit.client.1.smithi186.stdout:8/117: creat d7/de/d10/f27 x:0 0 0 2024-04-12T20:56:26.140 INFO:tasks.workunit.client.1.smithi186.stdout:4/250: dread d0/dc/d33/f1a [0,4194304] 0 2024-04-12T20:56:26.141 INFO:tasks.workunit.client.1.smithi186.stdout:9/145: getdents d3/d12 0 2024-04-12T20:56:26.142 INFO:tasks.workunit.client.1.smithi186.stdout:2/152: dwrite d0/f10 [0,4194304] 0 2024-04-12T20:56:26.143 INFO:tasks.workunit.client.1.smithi186.stdout:4/251: rename d0/dc/d39 to d0/dc/d39/d50 22 2024-04-12T20:56:26.143 INFO:tasks.workunit.client.0.smithi018.stdout:2/884: symlink d5/d12/l132 0 2024-04-12T20:56:26.143 INFO:tasks.workunit.client.0.smithi018.stdout:1/806: dwrite d3/d9/da/dd/d31/d32/d13/d19/d2d/d4e/d5e/d97/fe4 [0,4194304] 0 2024-04-12T20:56:26.144 INFO:tasks.workunit.client.1.smithi186.stdout:2/153: readlink d0/dc/l15 0 2024-04-12T20:56:26.145 INFO:tasks.workunit.client.1.smithi186.stdout:0/234: mkdir d0/d14/d4e/d50 0 2024-04-12T20:56:26.147 INFO:tasks.workunit.client.0.smithi018.stdout:4/929: write d3/d5d/d60/dc5/d43/fb1 [5049942,50580] 0 2024-04-12T20:56:26.150 INFO:tasks.workunit.client.1.smithi186.stdout:6/132: creat d9/d25/f31 x:0 0 0 2024-04-12T20:56:26.153 INFO:tasks.workunit.client.1.smithi186.stdout:1/162: mkdir d14/d1a/d33 0 2024-04-12T20:56:26.155 INFO:tasks.workunit.client.1.smithi186.stdout:1/163: chown d14/d1a/c30 508893 1 2024-04-12T20:56:26.156 INFO:tasks.workunit.client.1.smithi186.stdout:3/172: mknod d1/d3/c32 0 2024-04-12T20:56:26.157 INFO:tasks.workunit.client.0.smithi018.stdout:2/885: creat d5/d8/d10/f133 x:0 0 0 2024-04-12T20:56:26.161 INFO:tasks.workunit.client.0.smithi018.stdout:4/930: creat d3/d5d/d60/dc5/d53/de7/d126/d4f/f139 x:0 0 0 2024-04-12T20:56:26.164 INFO:tasks.workunit.client.1.smithi186.stdout:7/204: creat d7/d10/d2d/d40/d4a/f4c x:0 0 0 2024-04-12T20:56:26.164 INFO:tasks.workunit.client.0.smithi018.stdout:4/931: chown d3/d5d/d60/dc5/d53/de7/d126/d4f/dc7/cd6 826 1 2024-04-12T20:56:26.165 INFO:tasks.workunit.client.1.smithi186.stdout:4/252: rmdir d0/dc/d1f 39 2024-04-12T20:56:26.166 INFO:tasks.workunit.client.0.smithi018.stdout:1/807: rename d3/d9/da/dd/d31/d32/d13/d19/d2d/d4e/d58/dbe to d3/d9/da/dd/d31/d32/d13/d19/d111 0 2024-04-12T20:56:26.169 INFO:tasks.workunit.client.1.smithi186.stdout:5/152: getdents d5/db/d15/d2a 0 2024-04-12T20:56:26.175 INFO:tasks.workunit.client.0.smithi018.stdout:1/808: symlink d3/d9/da/dd/d31/d32/d13/d19/d2d/d4e/d5e/da0/de2/de9/l112 0 2024-04-12T20:56:26.179 INFO:tasks.workunit.client.0.smithi018.stdout:4/932: truncate d3/d32/fac 1302023 0 2024-04-12T20:56:26.182 INFO:tasks.workunit.client.1.smithi186.stdout:7/205: dwrite d7/d10/d2d/f44 [0,4194304] 0 2024-04-12T20:56:26.183 INFO:tasks.workunit.client.1.smithi186.stdout:4/253: dread d0/d21/f20 [0,4194304] 0 2024-04-12T20:56:26.183 INFO:tasks.workunit.client.1.smithi186.stdout:1/164: creat d14/d1a/d33/f34 x:0 0 0 2024-04-12T20:56:26.184 INFO:tasks.workunit.client.0.smithi018.stdout:4/933: write d3/d5d/d60/dc5/d53/de7/d126/d4f/f12b [146008,8507] 0 2024-04-12T20:56:26.186 INFO:tasks.workunit.client.1.smithi186.stdout:4/254: stat d0/dc/d39/f3a 0 2024-04-12T20:56:26.187 INFO:tasks.workunit.client.1.smithi186.stdout:1/165: write d14/d1a/f1b [953049,29542] 0 2024-04-12T20:56:26.188 INFO:tasks.workunit.client.1.smithi186.stdout:0/235: dwrite d0/d10/f4d [0,4194304] 0 2024-04-12T20:56:26.191 INFO:tasks.workunit.client.1.smithi186.stdout:4/255: dread d0/dc/d33/f46 [0,4194304] 0 2024-04-12T20:56:26.194 INFO:tasks.workunit.client.1.smithi186.stdout:4/256: truncate d0/dc/fe 2350746 0 2024-04-12T20:56:26.194 INFO:tasks.workunit.client.0.smithi018.stdout:2/886: dwrite d5/f95 [0,4194304] 0 2024-04-12T20:56:26.196 INFO:tasks.workunit.client.1.smithi186.stdout:5/153: fdatasync d5/db/fc 0 2024-04-12T20:56:26.199 INFO:tasks.workunit.client.1.smithi186.stdout:9/146: truncate d3/fe 495495 0 2024-04-12T20:56:26.200 INFO:tasks.workunit.client.0.smithi018.stdout:4/934: rename d3/d5d/c10e to d3/db/df6/c13a 0 2024-04-12T20:56:26.202 INFO:tasks.workunit.client.1.smithi186.stdout:5/154: write d5/db/d14/f1a [82747,79139] 0 2024-04-12T20:56:26.204 INFO:tasks.workunit.client.1.smithi186.stdout:1/166: mkdir d14/d1a/d21/d35 0 2024-04-12T20:56:26.208 INFO:tasks.workunit.client.1.smithi186.stdout:0/236: unlink d0/d14/c25 0 2024-04-12T20:56:26.210 INFO:tasks.workunit.client.1.smithi186.stdout:9/147: write d3/f10 [8987793,18334] 0 2024-04-12T20:56:26.210 INFO:tasks.workunit.client.1.smithi186.stdout:7/206: dwrite d7/f25 [0,4194304] 0 2024-04-12T20:56:26.211 INFO:tasks.workunit.client.1.smithi186.stdout:2/154: truncate d0/f6 1605636 0 2024-04-12T20:56:26.212 INFO:tasks.workunit.client.1.smithi186.stdout:3/173: symlink d1/d3/d21/d22/d24/d2f/l33 0 2024-04-12T20:56:26.213 INFO:tasks.workunit.client.1.smithi186.stdout:6/133: truncate d9/f2a 1701549 0 2024-04-12T20:56:26.213 INFO:tasks.workunit.client.1.smithi186.stdout:9/148: chown d3/d12/d16 78543410 1 2024-04-12T20:56:26.216 INFO:tasks.workunit.client.0.smithi018.stdout:2/887: read d5/d7/d31/d62/f72 [972147,102708] 0 2024-04-12T20:56:26.216 INFO:tasks.workunit.client.1.smithi186.stdout:2/155: fdatasync d0/f14 0 2024-04-12T20:56:26.218 INFO:tasks.workunit.client.1.smithi186.stdout:8/118: dwrite d7/f12 [0,4194304] 0 2024-04-12T20:56:26.221 INFO:tasks.workunit.client.0.smithi018.stdout:2/888: write d5/d8/d10/d43/d8f/de9/f117 [118773,95372] 0 2024-04-12T20:56:26.221 INFO:tasks.workunit.client.1.smithi186.stdout:3/174: chown d1/d3/l2d 91 1 2024-04-12T20:56:26.222 INFO:tasks.workunit.client.1.smithi186.stdout:8/119: readlink l3 0 2024-04-12T20:56:26.226 INFO:tasks.workunit.client.1.smithi186.stdout:4/257: creat d0/d21/d3d/d48/f51 x:0 0 0 2024-04-12T20:56:26.227 INFO:tasks.workunit.client.1.smithi186.stdout:1/167: truncate d14/d1a/d1d/f29 401003 0 2024-04-12T20:56:26.228 INFO:tasks.workunit.client.1.smithi186.stdout:6/134: mknod d9/d25/d2d/c32 0 2024-04-12T20:56:26.229 INFO:tasks.workunit.client.1.smithi186.stdout:8/120: mknod d7/d8/d1d/c28 0 2024-04-12T20:56:26.230 INFO:tasks.workunit.client.0.smithi018.stdout:2/889: unlink d5/d8/d10/d43/de2/d24/d39/d9c/c12a 0 2024-04-12T20:56:26.234 INFO:tasks.workunit.client.1.smithi186.stdout:6/135: chown d9/c23 25305 1 2024-04-12T20:56:26.234 INFO:tasks.workunit.client.1.smithi186.stdout:1/168: chown d14/d1a/d21/d2c/l2e 863363058 1 2024-04-12T20:56:26.234 INFO:tasks.workunit.client.1.smithi186.stdout:7/207: mkdir d7/d10/d2d/d45/d4d 0 2024-04-12T20:56:26.236 INFO:tasks.workunit.client.1.smithi186.stdout:4/258: rename d0/cf to d0/dc/d39/c52 0 2024-04-12T20:56:26.237 INFO:tasks.workunit.client.1.smithi186.stdout:3/175: mkdir d1/d3/d21/d22/d34 0 2024-04-12T20:56:26.241 INFO:tasks.workunit.client.1.smithi186.stdout:0/237: link d0/l2e d0/d14/d4e/d50/l51 0 2024-04-12T20:56:26.241 INFO:tasks.workunit.client.1.smithi186.stdout:1/169: stat d14/d1a/c1f 0 2024-04-12T20:56:26.245 INFO:tasks.workunit.client.1.smithi186.stdout:8/121: mkdir d7/d8/d20/d23/d25/d29 0 2024-04-12T20:56:26.246 INFO:tasks.workunit.client.0.smithi018.stdout:1/809: truncate d3/d9/da/dd/d31/d32/d13/d19/d2d/d4e/f5b 210289 0 2024-04-12T20:56:26.246 INFO:tasks.workunit.client.1.smithi186.stdout:4/259: rename d0/c25 to d0/d21/c53 0 2024-04-12T20:56:26.247 INFO:tasks.workunit.client.1.smithi186.stdout:8/122: truncate d7/f12 4878229 0 2024-04-12T20:56:26.249 INFO:tasks.workunit.client.0.smithi018.stdout:4/935: truncate d3/d5d/d60/dc5/de8/f115 167921 0 2024-04-12T20:56:26.251 INFO:tasks.workunit.client.1.smithi186.stdout:0/238: mknod d0/d14/c52 0 2024-04-12T20:56:26.251 INFO:tasks.workunit.client.0.smithi018.stdout:1/810: mkdir d3/d9/da/d14/d113 0 2024-04-12T20:56:26.253 INFO:tasks.workunit.client.0.smithi018.stdout:2/890: dread d5/d7/d31/d62/da6/df8/f12e [0,4194304] 0 2024-04-12T20:56:26.253 INFO:tasks.workunit.client.0.smithi018.stdout:4/936: unlink d3/d32/d33/f30 0 2024-04-12T20:56:26.258 INFO:tasks.workunit.client.0.smithi018.stdout:1/811: chown d3/d9/da/dd/d31/d32/d13/d19/d2d/d4e/f8c 11 1 2024-04-12T20:56:26.259 INFO:tasks.workunit.client.0.smithi018.stdout:2/891: unlink d5/d8/d10/d43/de2/daf/d88/dcd/ld7 0 2024-04-12T20:56:26.260 INFO:tasks.workunit.client.1.smithi186.stdout:5/155: truncate d5/f6 2525406 0 2024-04-12T20:56:26.260 INFO:tasks.workunit.client.1.smithi186.stdout:2/156: dread d0/f6 [0,4194304] 0 2024-04-12T20:56:26.261 INFO:tasks.workunit.client.1.smithi186.stdout:7/208: dread d7/d10/d1c/f2c [0,4194304] 0 2024-04-12T20:56:26.262 INFO:tasks.workunit.client.0.smithi018.stdout:1/812: chown d3/d9/da/dd/d31/d32/d33/d80/dab 10834183 1 2024-04-12T20:56:26.262 INFO:tasks.workunit.client.1.smithi186.stdout:5/156: chown d5/db/d15/d18 0 1 2024-04-12T20:56:26.263 INFO:tasks.workunit.client.0.smithi018.stdout:4/937: rename d3/d5d/d60/dc5/d3d/de4/f113 to d3/d32/d33/d8e/d123/f13b 0 2024-04-12T20:56:26.263 INFO:tasks.workunit.client.1.smithi186.stdout:9/149: getdents d3/d12 0 2024-04-12T20:56:26.265 INFO:tasks.workunit.client.1.smithi186.stdout:0/239: dread d0/d10/f4d [0,4194304] 0 2024-04-12T20:56:26.266 INFO:tasks.workunit.client.1.smithi186.stdout:7/209: chown d7/d10/d1c/f2c 104 1 2024-04-12T20:56:26.267 INFO:tasks.workunit.client.0.smithi018.stdout:2/892: creat d5/d7/db6/f134 x:0 0 0 2024-04-12T20:56:26.272 INFO:tasks.workunit.client.1.smithi186.stdout:6/136: getdents d9/d25/d2d 0 2024-04-12T20:56:26.276 INFO:tasks.workunit.client.1.smithi186.stdout:2/157: unlink d0/dc/l15 0 2024-04-12T20:56:26.277 INFO:tasks.workunit.client.1.smithi186.stdout:2/158: chown d0/c18 81155860 1 2024-04-12T20:56:26.278 INFO:tasks.workunit.client.1.smithi186.stdout:2/159: stat d0/l23 0 2024-04-12T20:56:26.283 INFO:tasks.workunit.client.0.smithi018.stdout:4/938: getdents d3/d32/d33 0 2024-04-12T20:56:26.291 INFO:tasks.workunit.client.1.smithi186.stdout:7/210: write d7/d10/d1c/f2c [4504020,72838] 0 2024-04-12T20:56:26.291 INFO:tasks.workunit.client.0.smithi018.stdout:4/939: rename d3/d32/fd to d3/d5d/d60/dc5/d3d/f13c 0 2024-04-12T20:56:26.293 INFO:tasks.workunit.client.1.smithi186.stdout:9/150: rename d3/d13/f17 to d3/d13/f28 0 2024-04-12T20:56:26.298 INFO:tasks.workunit.client.0.smithi018.stdout:4/940: chown d3/d5d/d60/dc5/d53/de7/d126/d72/l9a 7569 1 2024-04-12T20:56:26.299 INFO:tasks.workunit.client.1.smithi186.stdout:3/176: rename d1/d3/d21/c2e to d1/d3/d21/d22/c35 0 2024-04-12T20:56:26.300 INFO:tasks.workunit.client.0.smithi018.stdout:1/813: dwrite d3/d9/da/dd/d31/d32/d13/d19/d2d/d4e/d5e/fa7 [0,4194304] 0 2024-04-12T20:56:26.303 INFO:tasks.workunit.client.0.smithi018.stdout:2/893: dwrite d5/d12/f89 [4194304,4194304] 0 2024-04-12T20:56:26.304 INFO:tasks.workunit.client.0.smithi018.stdout:4/941: fdatasync d3/d5d/d60/dc5/fc1 0 2024-04-12T20:56:26.304 INFO:tasks.workunit.client.1.smithi186.stdout:0/240: symlink d0/d10/d32/d3f/l53 0 2024-04-12T20:56:26.305 INFO:tasks.workunit.client.1.smithi186.stdout:0/241: fdatasync d0/d14/f19 0 2024-04-12T20:56:26.309 INFO:tasks.workunit.client.1.smithi186.stdout:5/157: write d5/db/d15/f2f [1080805,46578] 0 2024-04-12T20:56:26.312 INFO:tasks.workunit.client.0.smithi018.stdout:2/894: unlink d5/d7/db6/f114 0 2024-04-12T20:56:26.313 INFO:tasks.workunit.client.1.smithi186.stdout:7/211: read - d7/d10/d1c/f1e zero size 2024-04-12T20:56:26.318 INFO:tasks.workunit.client.1.smithi186.stdout:1/170: getdents d14/d1a 0 2024-04-12T20:56:26.322 INFO:tasks.workunit.client.1.smithi186.stdout:8/123: dwrite d7/d8/d17/f1c [0,4194304] 0 2024-04-12T20:56:26.326 INFO:tasks.workunit.client.0.smithi018.stdout:2/895: rmdir d5/d8/d10/d43/de2 39 2024-04-12T20:56:26.326 INFO:tasks.workunit.client.1.smithi186.stdout:2/160: rename d0/d3/d1a/d27/l2f to d0/l30 0 2024-04-12T20:56:26.329 INFO:tasks.workunit.client.1.smithi186.stdout:2/161: dread - d0/d3/d1a/d21/d26/f2d zero size 2024-04-12T20:56:26.330 INFO:tasks.workunit.client.0.smithi018.stdout:4/942: dread d3/d5d/d60/dc5/d43/f2e [0,4194304] 0 2024-04-12T20:56:26.334 INFO:tasks.workunit.client.1.smithi186.stdout:4/260: link d0/d21/c2e d0/d21/d3d/c54 0 2024-04-12T20:56:26.337 INFO:tasks.workunit.client.0.smithi018.stdout:4/943: mkdir d3/d5d/d60/dc5/d53/de7/d126/d13d 0 2024-04-12T20:56:26.339 INFO:tasks.workunit.client.1.smithi186.stdout:0/242: creat d0/d10/d32/d3f/f54 x:0 0 0 2024-04-12T20:56:26.342 INFO:tasks.workunit.client.0.smithi018.stdout:4/944: dread - d3/d5d/d60/dc5/dcc/fd4 zero size 2024-04-12T20:56:26.342 INFO:tasks.workunit.client.0.smithi018.stdout:2/896: dread d5/d8/d33/f9a [0,4194304] 0 2024-04-12T20:56:26.344 INFO:tasks.workunit.client.0.smithi018.stdout:1/814: dwrite d3/d9/da/dd/d31/d32/d13/f46 [0,4194304] 0 2024-04-12T20:56:26.345 INFO:tasks.workunit.client.0.smithi018.stdout:2/897: fsync d5/d12/f58 0 2024-04-12T20:56:26.346 INFO:tasks.workunit.client.1.smithi186.stdout:0/243: dread d0/d14/d4e/f33 [0,4194304] 0 2024-04-12T20:56:26.346 INFO:tasks.workunit.client.1.smithi186.stdout:1/171: mkdir d14/d1a/d33/d36 0 2024-04-12T20:56:26.348 INFO:tasks.workunit.client.0.smithi018.stdout:4/945: write d3/d5d/d60/ff0 [27544,70322] 0 2024-04-12T20:56:26.352 INFO:tasks.workunit.client.0.smithi018.stdout:2/898: mknod d5/d8/d10/d43/de2/daf/d90/c135 0 2024-04-12T20:56:26.353 INFO:tasks.workunit.client.0.smithi018.stdout:1/815: write d3/d9/da/dd/d31/d32/d13/d19/d2d/d4e/d7a/d82/ff6 [5132165,87862] 0 2024-04-12T20:56:26.354 INFO:tasks.workunit.client.0.smithi018.stdout:4/946: mknod d3/db/de3/c13e 0 2024-04-12T20:56:26.360 INFO:tasks.workunit.client.1.smithi186.stdout:6/137: creat d9/f33 x:0 0 0 2024-04-12T20:56:26.360 INFO:tasks.workunit.client.1.smithi186.stdout:9/151: dwrite d3/fb [4194304,4194304] 0 2024-04-12T20:56:26.360 INFO:tasks.workunit.client.1.smithi186.stdout:6/138: dread - d9/f2e zero size 2024-04-12T20:56:26.360 INFO:tasks.workunit.client.1.smithi186.stdout:9/152: read d3/f5 [2627213,69511] 0 2024-04-12T20:56:26.362 INFO:tasks.workunit.client.1.smithi186.stdout:9/153: chown d3/d13/f18 12 1 2024-04-12T20:56:26.362 INFO:tasks.workunit.client.0.smithi018.stdout:1/816: creat d3/d9/da/dd/d31/d32/d13/d19/d2d/d4e/d7a/df5/f114 x:0 0 0 2024-04-12T20:56:26.362 INFO:tasks.workunit.client.1.smithi186.stdout:6/139: write d9/f1b [1897527,106807] 0 2024-04-12T20:56:26.364 INFO:tasks.workunit.client.1.smithi186.stdout:7/212: creat d7/d10/d2d/d45/d4d/f4e x:0 0 0 2024-04-12T20:56:26.364 INFO:tasks.workunit.client.1.smithi186.stdout:1/172: rmdir d14 39 2024-04-12T20:56:26.364 INFO:tasks.workunit.client.0.smithi018.stdout:4/947: truncate d3/d32/d94/d10b/f121 236910 0 2024-04-12T20:56:26.364 INFO:tasks.workunit.client.1.smithi186.stdout:9/154: readlink d3/l15 0 2024-04-12T20:56:26.369 INFO:tasks.workunit.client.1.smithi186.stdout:8/124: creat d7/d8/d20/d23/d25/d29/f2a x:0 0 0 2024-04-12T20:56:26.372 INFO:tasks.workunit.client.1.smithi186.stdout:8/125: dread - d7/de/d10/f15 zero size 2024-04-12T20:56:26.372 INFO:tasks.workunit.client.1.smithi186.stdout:0/244: dwrite d0/f4f [0,4194304] 0 2024-04-12T20:56:26.376 INFO:tasks.workunit.client.1.smithi186.stdout:2/162: write d0/f2 [2600891,94124] 0 2024-04-12T20:56:26.379 INFO:tasks.workunit.client.1.smithi186.stdout:2/163: chown d0/d3/d1a/d21/f2c 903011496 1 2024-04-12T20:56:26.380 INFO:tasks.workunit.client.1.smithi186.stdout:1/173: creat d14/d1a/f37 x:0 0 0 2024-04-12T20:56:26.381 INFO:tasks.workunit.client.1.smithi186.stdout:6/140: unlink d9/l28 0 2024-04-12T20:56:26.385 INFO:tasks.workunit.client.1.smithi186.stdout:2/164: dread - d0/d3/d1a/d21/f2c zero size 2024-04-12T20:56:26.385 INFO:tasks.workunit.client.1.smithi186.stdout:1/174: write d14/d1a/f1b [27310,17488] 0 2024-04-12T20:56:26.386 INFO:tasks.workunit.client.1.smithi186.stdout:4/261: getdents d0/dc/d33 0 2024-04-12T20:56:26.386 INFO:tasks.workunit.client.1.smithi186.stdout:6/141: stat d9/l16 0 2024-04-12T20:56:26.386 INFO:tasks.workunit.client.1.smithi186.stdout:2/165: stat d0/f6 0 2024-04-12T20:56:26.387 INFO:tasks.workunit.client.1.smithi186.stdout:3/177: truncate d1/f1e 1393105 0 2024-04-12T20:56:26.390 INFO:tasks.workunit.client.1.smithi186.stdout:1/175: write d14/f22 [50431,128558] 0 2024-04-12T20:56:26.392 INFO:tasks.workunit.client.1.smithi186.stdout:7/213: mknod d7/d10/d1c/c4f 0 2024-04-12T20:56:26.392 INFO:tasks.workunit.client.1.smithi186.stdout:5/158: truncate d5/f6 3386818 0 2024-04-12T20:56:26.393 INFO:tasks.workunit.client.1.smithi186.stdout:9/155: symlink d3/d12/d27/l29 0 2024-04-12T20:56:26.396 INFO:tasks.workunit.client.0.smithi018.stdout:1/817: dread d3/d9/da/d14/f38 [0,4194304] 0 2024-04-12T20:56:26.397 INFO:tasks.workunit.client.1.smithi186.stdout:6/142: mknod d9/c34 0 2024-04-12T20:56:26.399 INFO:tasks.workunit.client.0.smithi018.stdout:2/899: write d5/d8/d10/d43/de2/ff2 [539098,113785] 0 2024-04-12T20:56:26.400 INFO:tasks.workunit.client.1.smithi186.stdout:4/262: getdents d0/dc/d33 0 2024-04-12T20:56:26.401 INFO:tasks.workunit.client.1.smithi186.stdout:0/245: getdents d0/d10 0 2024-04-12T20:56:26.405 INFO:tasks.workunit.client.1.smithi186.stdout:1/176: creat d14/d1a/d21/d2c/f38 x:0 0 0 2024-04-12T20:56:26.409 INFO:tasks.workunit.client.0.smithi018.stdout:4/948: dwrite d3/d5d/d60/dc5/d53/de7/d126/d4f/f95 [0,4194304] 0 2024-04-12T20:56:26.410 INFO:tasks.workunit.client.0.smithi018.stdout:1/818: truncate d3/d9/da/dd/d31/d32/d33/d80/fca 633546 0 2024-04-12T20:56:26.411 INFO:tasks.workunit.client.1.smithi186.stdout:9/156: rename d3/f11 to d3/d12/d25/f2a 0 2024-04-12T20:56:26.422 INFO:tasks.workunit.client.0.smithi018.stdout:1/819: creat d3/d9/dc3/f115 x:0 0 0 2024-04-12T20:56:26.423 INFO:tasks.workunit.client.1.smithi186.stdout:4/263: dread d0/dc/f23 [0,4194304] 0 2024-04-12T20:56:26.424 INFO:tasks.workunit.client.1.smithi186.stdout:0/246: mknod d0/d14/d4e/c55 0 2024-04-12T20:56:26.425 INFO:tasks.workunit.client.1.smithi186.stdout:2/166: write d0/d3/f8 [1174580,108728] 0 2024-04-12T20:56:26.426 INFO:tasks.workunit.client.1.smithi186.stdout:5/159: write d5/db/f11 [987480,3599] 0 2024-04-12T20:56:26.429 INFO:tasks.workunit.client.1.smithi186.stdout:1/177: dwrite d14/f16 [4194304,4194304] 0 2024-04-12T20:56:26.432 INFO:tasks.workunit.client.1.smithi186.stdout:7/214: truncate d7/f16 8588908 0 2024-04-12T20:56:26.432 INFO:tasks.workunit.client.1.smithi186.stdout:0/247: dread - d0/d10/d32/f48 zero size 2024-04-12T20:56:26.433 INFO:tasks.workunit.client.0.smithi018.stdout:2/900: dwrite d5/d7/f5c [0,4194304] 0 2024-04-12T20:56:26.435 INFO:tasks.workunit.client.1.smithi186.stdout:7/215: dread - d7/d10/d1c/f1e zero size 2024-04-12T20:56:26.436 INFO:tasks.workunit.client.1.smithi186.stdout:8/126: dwrite d7/d18/f1e [0,4194304] 0 2024-04-12T20:56:26.438 INFO:tasks.workunit.client.1.smithi186.stdout:3/178: getdents d1/d3/d21/d22/d24 0 2024-04-12T20:56:26.441 INFO:tasks.workunit.client.0.smithi018.stdout:4/949: dwrite d3/db/f7f [0,4194304] 0 2024-04-12T20:56:26.443 INFO:tasks.workunit.client.1.smithi186.stdout:5/160: rmdir d5/db/d15/d18 39 2024-04-12T20:56:26.443 INFO:tasks.workunit.client.1.smithi186.stdout:4/264: write d0/dc/d33/f1a [1486087,36414] 0 2024-04-12T20:56:26.444 INFO:tasks.workunit.client.1.smithi186.stdout:8/127: fsync d7/de/f11 0 2024-04-12T20:56:26.444 INFO:tasks.workunit.client.1.smithi186.stdout:3/179: write d1/d3/f11 [196713,122395] 0 2024-04-12T20:56:26.448 INFO:tasks.workunit.client.1.smithi186.stdout:1/178: stat l13 0 2024-04-12T20:56:26.449 INFO:tasks.workunit.client.1.smithi186.stdout:8/128: chown d7/de/d10/f15 69306103 1 2024-04-12T20:56:26.450 INFO:tasks.workunit.client.0.smithi018.stdout:2/901: truncate d5/d8/d33/f76 4068550 0 2024-04-12T20:56:26.453 INFO:tasks.workunit.client.1.smithi186.stdout:6/143: dread d9/f1b [0,4194304] 0 2024-04-12T20:56:26.453 INFO:tasks.workunit.client.1.smithi186.stdout:7/216: creat d7/d10/d2d/d40/d4a/f50 x:0 0 0 2024-04-12T20:56:26.455 INFO:tasks.workunit.client.1.smithi186.stdout:2/167: symlink d0/dc/d2e/l31 0 2024-04-12T20:56:26.456 INFO:tasks.workunit.client.1.smithi186.stdout:8/129: mknod d7/d8/d1d/d1f/c2b 0 2024-04-12T20:56:26.458 INFO:tasks.workunit.client.1.smithi186.stdout:3/180: write d1/f19 [5084326,92353] 0 2024-04-12T20:56:26.458 INFO:tasks.workunit.client.1.smithi186.stdout:2/168: chown d0/d3/d12/d2a 1833468935 1 2024-04-12T20:56:26.458 INFO:tasks.workunit.client.0.smithi018.stdout:1/820: dwrite d3/d9/da/dd/d31/d32/f34 [0,4194304] 0 2024-04-12T20:56:26.458 INFO:tasks.workunit.client.0.smithi018.stdout:4/950: mkdir d3/d5d/d60/dc5/d40/d128/d13f 0 2024-04-12T20:56:26.459 INFO:tasks.workunit.client.1.smithi186.stdout:1/179: write d14/d1a/d21/f27 [1211273,6210] 0 2024-04-12T20:56:26.460 INFO:tasks.workunit.client.0.smithi018.stdout:2/902: rmdir d5/d8/d10/d43/de2/d7c/d92/de5 39 2024-04-12T20:56:26.460 INFO:tasks.workunit.client.1.smithi186.stdout:7/217: dread - d7/d10/f46 zero size 2024-04-12T20:56:26.462 INFO:tasks.workunit.client.1.smithi186.stdout:9/157: getdents d3/d12/d25 0 2024-04-12T20:56:26.468 INFO:tasks.workunit.client.0.smithi018.stdout:1/821: fdatasync d3/d9/da/dd/d31/d32/d13/d19/d2d/d4e/d7a/df5/f114 0 2024-04-12T20:56:26.470 INFO:tasks.workunit.client.0.smithi018.stdout:4/951: getdents d3/d5d/d60/dc5/d53/de7/d126/d72 0 2024-04-12T20:56:26.471 INFO:tasks.workunit.client.1.smithi186.stdout:7/218: write d7/fe [1126662,34276] 0 2024-04-12T20:56:26.472 INFO:tasks.workunit.client.0.smithi018.stdout:2/903: truncate d5/d8/d10/d43/de2/daf/d90/fb4 122853 0 2024-04-12T20:56:26.473 INFO:tasks.workunit.client.0.smithi018.stdout:2/904: stat d5/cb1 0 2024-04-12T20:56:26.477 INFO:tasks.workunit.client.1.smithi186.stdout:5/161: mkdir d5/db/d33 0 2024-04-12T20:56:26.479 INFO:tasks.workunit.client.0.smithi018.stdout:4/952: rename d3/d32/d94 to d3/d5d/d60/dc5/d53/de7/d126/d13d/d140 0 2024-04-12T20:56:26.479 INFO:tasks.workunit.client.1.smithi186.stdout:1/180: stat f9 0 2024-04-12T20:56:26.482 INFO:tasks.workunit.client.1.smithi186.stdout:9/158: symlink d3/d12/d16/l2b 0 2024-04-12T20:56:26.483 INFO:tasks.workunit.client.0.smithi018.stdout:2/905: creat d5/d8/d10/d43/de2/d24/f136 x:0 0 0 2024-04-12T20:56:26.483 INFO:tasks.workunit.client.1.smithi186.stdout:0/248: creat d0/d14/d4e/d1e/f56 x:0 0 0 2024-04-12T20:56:26.484 INFO:tasks.workunit.client.1.smithi186.stdout:3/181: mknod d1/c36 0 2024-04-12T20:56:26.491 INFO:tasks.workunit.client.0.smithi018.stdout:2/906: mkdir d5/d7/d31/d62/da6/df8/d137 0 2024-04-12T20:56:26.492 INFO:tasks.workunit.client.1.smithi186.stdout:3/182: write d1/f19 [7593877,53721] 0 2024-04-12T20:56:26.494 INFO:tasks.workunit.client.1.smithi186.stdout:4/265: dwrite d0/dc/f29 [0,4194304] 0 2024-04-12T20:56:26.499 INFO:tasks.workunit.client.0.smithi018.stdout:4/953: dwrite d3/d5d/d60/dc5/fba [0,4194304] 0 2024-04-12T20:56:26.501 INFO:tasks.workunit.client.1.smithi186.stdout:7/219: fdatasync d7/f16 0 2024-04-12T20:56:26.501 INFO:tasks.workunit.client.0.smithi018.stdout:1/822: write d3/d9/da/dd/d31/d32/f41 [4915565,36936] 0 2024-04-12T20:56:26.503 INFO:tasks.workunit.client.0.smithi018.stdout:2/907: link d5/d8/d10/d43/de2/l27 d5/d8/d10/d43/de2/d24/d91/l138 0 2024-04-12T20:56:26.503 INFO:tasks.workunit.client.1.smithi186.stdout:5/162: creat d5/db/f34 x:0 0 0 2024-04-12T20:56:26.504 INFO:tasks.workunit.client.1.smithi186.stdout:2/169: dwrite d0/f10 [0,4194304] 0 2024-04-12T20:56:26.504 INFO:tasks.workunit.client.0.smithi018.stdout:2/908: stat d5/d12/d54/dc0/ld5 0 2024-04-12T20:56:26.504 INFO:tasks.workunit.client.0.smithi018.stdout:4/954: symlink d3/d5d/d60/dc5/de8/l141 0 2024-04-12T20:56:26.507 INFO:tasks.workunit.client.0.smithi018.stdout:4/955: chown d3/d5d/d60/dc5/d3d/d80/feb 4538 1 2024-04-12T20:56:26.513 INFO:tasks.workunit.client.1.smithi186.stdout:4/266: mkdir d0/dc/d55 0 2024-04-12T20:56:26.518 INFO:tasks.workunit.client.0.smithi018.stdout:1/823: creat d3/d9/da/dd/d31/f116 x:0 0 0 2024-04-12T20:56:26.520 INFO:tasks.workunit.client.1.smithi186.stdout:1/181: dread fe [0,4194304] 0 2024-04-12T20:56:26.522 INFO:tasks.workunit.client.1.smithi186.stdout:6/144: dwrite d9/f2e [0,4194304] 0 2024-04-12T20:56:26.525 INFO:tasks.workunit.client.1.smithi186.stdout:9/159: dread d3/d13/f28 [0,4194304] 0 2024-04-12T20:56:26.525 INFO:tasks.workunit.client.1.smithi186.stdout:7/220: creat d7/d10/d2d/d40/d4a/f51 x:0 0 0 2024-04-12T20:56:26.525 INFO:tasks.workunit.client.1.smithi186.stdout:4/267: creat d0/dc/d33/f56 x:0 0 0 2024-04-12T20:56:26.525 INFO:tasks.workunit.client.0.smithi018.stdout:1/824: unlink d3/d9/da/dd/d31/d32/d13/d19/d2d/d4e/d5e/d97/l108 0 2024-04-12T20:56:26.529 INFO:tasks.workunit.client.1.smithi186.stdout:6/145: creat d9/d25/f35 x:0 0 0 2024-04-12T20:56:26.530 INFO:tasks.workunit.client.1.smithi186.stdout:0/249: truncate d0/d14/d4e/d1e/f4b 171572 0 2024-04-12T20:56:26.534 INFO:tasks.workunit.client.1.smithi186.stdout:4/268: creat d0/dc/d33/f57 x:0 0 0 2024-04-12T20:56:26.535 INFO:tasks.workunit.client.1.smithi186.stdout:9/160: creat d3/d13/f2c x:0 0 0 2024-04-12T20:56:26.538 INFO:tasks.workunit.client.1.smithi186.stdout:0/250: chown d0/d14/d4e/d1e/f2f 404 1 2024-04-12T20:56:26.538 INFO:tasks.workunit.client.1.smithi186.stdout:7/221: chown d7/d10/c3a 1 1 2024-04-12T20:56:26.539 INFO:tasks.workunit.client.1.smithi186.stdout:6/146: write d9/f33 [777906,4660] 0 2024-04-12T20:56:26.540 INFO:tasks.workunit.client.1.smithi186.stdout:4/269: mknod d0/c58 0 2024-04-12T20:56:26.541 INFO:tasks.workunit.client.1.smithi186.stdout:9/161: chown d3/c7 29 1 2024-04-12T20:56:26.541 INFO:tasks.workunit.client.1.smithi186.stdout:7/222: fdatasync d7/d10/f13 0 2024-04-12T20:56:26.543 INFO:tasks.workunit.client.1.smithi186.stdout:6/147: write d9/f22 [5146167,63345] 0 2024-04-12T20:56:26.544 INFO:tasks.workunit.client.1.smithi186.stdout:0/251: fsync d0/d14/d4e/d1e/f56 0 2024-04-12T20:56:26.547 INFO:tasks.workunit.client.1.smithi186.stdout:9/162: write d3/f5 [3428107,120935] 0 2024-04-12T20:56:26.549 INFO:tasks.workunit.client.1.smithi186.stdout:9/163: chown d3/l15 1181 1 2024-04-12T20:56:26.550 INFO:tasks.workunit.client.0.smithi018.stdout:4/956: write d3/d32/d33/f17 [569733,123403] 0 2024-04-12T20:56:26.550 INFO:tasks.workunit.client.0.smithi018.stdout:2/909: write d5/d7/d63/f65 [85636,70117] 0 2024-04-12T20:56:26.555 INFO:tasks.workunit.client.1.smithi186.stdout:7/223: creat d7/d10/d1c/f52 x:0 0 0 2024-04-12T20:56:26.557 INFO:tasks.workunit.client.0.smithi018.stdout:4/957: read - d3/d5d/d60/dc5/d43/d4b/df2/ff4 zero size 2024-04-12T20:56:26.558 INFO:tasks.workunit.client.1.smithi186.stdout:2/170: truncate d0/f4 1910647 0 2024-04-12T20:56:26.558 INFO:tasks.workunit.client.1.smithi186.stdout:9/164: creat d3/d12/d25/f2d x:0 0 0 2024-04-12T20:56:26.560 INFO:tasks.workunit.client.0.smithi018.stdout:1/825: dwrite d3/fd1 [0,4194304] 0 2024-04-12T20:56:26.561 INFO:tasks.workunit.client.1.smithi186.stdout:9/165: fdatasync d3/d12/d25/f2d 0 2024-04-12T20:56:26.563 INFO:tasks.workunit.client.1.smithi186.stdout:7/224: readlink d7/d10/l30 0 2024-04-12T20:56:26.564 INFO:tasks.workunit.client.1.smithi186.stdout:1/182: truncate d14/d1a/f1b 347884 0 2024-04-12T20:56:26.565 INFO:tasks.workunit.client.0.smithi018.stdout:4/958: creat d3/d5d/d60/dc5/d40/d128/f142 x:0 0 0 2024-04-12T20:56:26.566 INFO:tasks.workunit.client.1.smithi186.stdout:1/183: chown fe 59553 1 2024-04-12T20:56:26.567 INFO:tasks.workunit.client.0.smithi018.stdout:4/959: readlink d3/d32/l47 0 2024-04-12T20:56:26.568 INFO:tasks.workunit.client.1.smithi186.stdout:0/252: dread d0/d10/f4d [0,4194304] 0 2024-04-12T20:56:26.571 INFO:tasks.workunit.client.1.smithi186.stdout:2/171: rmdir d0/d3/d1a/d24/d25 0 2024-04-12T20:56:26.574 INFO:tasks.workunit.client.1.smithi186.stdout:3/183: dread d1/fc [0,4194304] 0 2024-04-12T20:56:26.576 INFO:tasks.workunit.client.0.smithi018.stdout:1/826: creat d3/d9/da/dd/f117 x:0 0 0 2024-04-12T20:56:26.576 INFO:tasks.workunit.client.0.smithi018.stdout:2/910: write d5/d8/d10/d43/f56 [167020,113105] 0 2024-04-12T20:56:26.579 INFO:tasks.workunit.client.1.smithi186.stdout:6/148: dwrite d9/f10 [0,4194304] 0 2024-04-12T20:56:26.582 INFO:tasks.workunit.client.0.smithi018.stdout:2/911: dread d5/d8/d10/d43/d8f/de9/f117 [0,4194304] 0 2024-04-12T20:56:26.584 INFO:tasks.workunit.client.0.smithi018.stdout:2/912: chown d5/d12/d54 1 1 2024-04-12T20:56:26.585 INFO:tasks.workunit.client.0.smithi018.stdout:2/913: readlink d5/d8/d33/l47 0 2024-04-12T20:56:26.592 INFO:tasks.workunit.client.1.smithi186.stdout:9/166: rename d3/d12/d25 to d3/d12/d16/d2e 0 2024-04-12T20:56:26.592 INFO:tasks.workunit.client.0.smithi018.stdout:2/914: creat d5/d7/d31/d62/da6/df8/d137/f139 x:0 0 0 2024-04-12T20:56:26.595 INFO:tasks.workunit.client.1.smithi186.stdout:7/225: dwrite d7/d10/d1c/f52 [0,4194304] 0 2024-04-12T20:56:26.595 INFO:tasks.workunit.client.1.smithi186.stdout:9/167: dread - d3/d13/f22 zero size 2024-04-12T20:56:26.598 INFO:tasks.workunit.client.0.smithi018.stdout:4/960: write d3/d5d/f6e [558156,77080] 0 2024-04-12T20:56:26.598 INFO:tasks.workunit.client.1.smithi186.stdout:2/172: dwrite d0/f6 [0,4194304] 0 2024-04-12T20:56:26.599 INFO:tasks.workunit.client.1.smithi186.stdout:6/149: rename d9/d25/d2d/c32 to d9/d25/d2d/c36 0 2024-04-12T20:56:26.599 INFO:tasks.workunit.client.0.smithi018.stdout:2/915: mknod d5/d7/d31/d11a/c13a 0 2024-04-12T20:56:26.603 INFO:tasks.workunit.client.1.smithi186.stdout:3/184: dread d1/d3/f11 [0,4194304] 0 2024-04-12T20:56:26.604 INFO:tasks.workunit.client.0.smithi018.stdout:2/916: mkdir d5/d7/d31/d62/d13b 0 2024-04-12T20:56:26.607 INFO:tasks.workunit.client.0.smithi018.stdout:4/961: getdents d3/d5d/d60 0 2024-04-12T20:56:26.609 INFO:tasks.workunit.client.0.smithi018.stdout:2/917: chown d5/d8/d10/d43/de2/d24/d39/f51 263974106 1 2024-04-12T20:56:26.609 INFO:tasks.workunit.client.1.smithi186.stdout:0/253: write d0/d14/d4e/d1e/f37 [1727265,48740] 0 2024-04-12T20:56:26.611 INFO:tasks.workunit.client.1.smithi186.stdout:9/168: rename d3/d12/d16 to d3/d2f 0 2024-04-12T20:56:26.614 INFO:tasks.workunit.client.0.smithi018.stdout:4/962: creat d3/d32/f143 x:0 0 0 2024-04-12T20:56:26.620 INFO:tasks.workunit.client.0.smithi018.stdout:4/963: creat d3/d32/d66/f144 x:0 0 0 2024-04-12T20:56:26.621 INFO:tasks.workunit.client.0.smithi018.stdout:2/918: link d5/d8/d33/ce3 d5/d8/d10/d43/de2/daf/d90/c13c 0 2024-04-12T20:56:26.631 INFO:tasks.workunit.client.1.smithi186.stdout:1/184: dwrite d14/d1a/d1d/f29 [0,4194304] 0 2024-04-12T20:56:26.631 INFO:tasks.workunit.client.0.smithi018.stdout:4/964: link d3/d32/c5b d3/d32/d33/d8e/d123/c145 0 2024-04-12T20:56:26.632 INFO:tasks.workunit.client.1.smithi186.stdout:7/226: dwrite d7/d10/d1c/f52 [0,4194304] 0 2024-04-12T20:56:26.635 INFO:tasks.workunit.client.1.smithi186.stdout:1/185: write d14/d1a/f37 [153383,27008] 0 2024-04-12T20:56:26.635 INFO:tasks.workunit.client.1.smithi186.stdout:0/254: link d0/d14/f35 d0/f57 0 2024-04-12T20:56:26.636 INFO:tasks.workunit.client.1.smithi186.stdout:6/150: dwrite d9/d25/f31 [0,4194304] 0 2024-04-12T20:56:26.636 INFO:tasks.workunit.client.1.smithi186.stdout:0/255: readlink d0/l3b 0 2024-04-12T20:56:26.640 INFO:tasks.workunit.client.1.smithi186.stdout:2/173: dread d0/f14 [0,4194304] 0 2024-04-12T20:56:26.643 INFO:tasks.workunit.client.1.smithi186.stdout:0/256: stat d0/c9 0 2024-04-12T20:56:26.644 INFO:tasks.workunit.client.1.smithi186.stdout:6/151: chown d9/f22 3880715 1 2024-04-12T20:56:26.653 INFO:tasks.workunit.client.1.smithi186.stdout:7/227: mkdir d7/d10/d2d/d40/d4a/d53 0 2024-04-12T20:56:26.654 INFO:tasks.workunit.client.1.smithi186.stdout:1/186: truncate f4 1363755 0 2024-04-12T20:56:26.659 INFO:tasks.workunit.client.1.smithi186.stdout:6/152: fsync d9/f12 0 2024-04-12T20:56:26.661 INFO:tasks.workunit.client.1.smithi186.stdout:1/187: write d14/d1a/f37 [807520,94526] 0 2024-04-12T20:56:26.661 INFO:tasks.workunit.client.1.smithi186.stdout:7/228: stat d7/d10/d1c/c32 0 2024-04-12T20:56:26.662 INFO:tasks.workunit.client.1.smithi186.stdout:7/229: readlink d7/d10/l3b 0 2024-04-12T20:56:26.664 INFO:tasks.workunit.client.1.smithi186.stdout:3/185: dread d1/d3/f1f [0,4194304] 0 2024-04-12T20:56:26.672 INFO:tasks.workunit.client.1.smithi186.stdout:0/257: dread d0/d14/d4e/f28 [0,4194304] 0 2024-04-12T20:56:26.672 INFO:tasks.workunit.client.1.smithi186.stdout:3/186: chown d1/c1a 41615 1 2024-04-12T20:56:26.672 INFO:tasks.workunit.client.1.smithi186.stdout:7/230: getdents d7 0 2024-04-12T20:56:26.675 INFO:tasks.workunit.client.1.smithi186.stdout:1/188: link d14/l32 d14/d1a/d33/l39 0 2024-04-12T20:56:26.677 INFO:tasks.workunit.client.1.smithi186.stdout:3/187: unlink d1/d3/d21/d22/c35 0 2024-04-12T20:56:26.681 INFO:tasks.workunit.client.1.smithi186.stdout:1/189: truncate d14/d1a/d33/f34 803837 0 2024-04-12T20:56:26.683 INFO:tasks.workunit.client.1.smithi186.stdout:9/169: dread d3/fd [0,4194304] 0 2024-04-12T20:56:26.687 INFO:tasks.workunit.client.1.smithi186.stdout:6/153: dwrite d9/f1b [0,4194304] 0 2024-04-12T20:56:26.687 INFO:tasks.workunit.client.1.smithi186.stdout:1/190: rmdir d14/d1a 39 2024-04-12T20:56:26.687 INFO:tasks.workunit.client.1.smithi186.stdout:3/188: symlink d1/d3/d21/d22/d24/d28/l37 0 2024-04-12T20:56:26.690 INFO:tasks.workunit.client.1.smithi186.stdout:2/174: dread d0/f4 [0,4194304] 0 2024-04-12T20:56:26.690 INFO:tasks.workunit.client.1.smithi186.stdout:0/258: dread d0/d14/d4e/d1e/f1f [0,4194304] 0 2024-04-12T20:56:26.691 INFO:tasks.workunit.client.1.smithi186.stdout:6/154: stat d9/f10 0 2024-04-12T20:56:26.692 INFO:tasks.workunit.client.1.smithi186.stdout:9/170: link d3/f1d d3/d12/f30 0 2024-04-12T20:56:26.693 INFO:tasks.workunit.client.1.smithi186.stdout:2/175: chown d0/d3/d1a/d21 96138 1 2024-04-12T20:56:26.697 INFO:tasks.workunit.client.1.smithi186.stdout:1/191: unlink d14/d1a/d21/c2f 0 2024-04-12T20:56:26.699 INFO:tasks.workunit.client.1.smithi186.stdout:0/259: getdents d0/d10 0 2024-04-12T20:56:26.700 INFO:tasks.workunit.client.1.smithi186.stdout:7/231: dwrite d7/d10/d2d/f37 [0,4194304] 0 2024-04-12T20:56:26.707 INFO:tasks.workunit.client.1.smithi186.stdout:7/232: unlink d7/f28 0 2024-04-12T20:56:26.710 INFO:tasks.workunit.client.1.smithi186.stdout:0/260: read - d0/d10/d32/f3d zero size 2024-04-12T20:56:26.711 INFO:tasks.workunit.client.1.smithi186.stdout:6/155: rename d9/d25/f31 to d9/f37 0 2024-04-12T20:56:26.715 INFO:tasks.workunit.client.1.smithi186.stdout:1/192: link c8 d14/c3a 0 2024-04-12T20:56:26.720 INFO:tasks.workunit.client.1.smithi186.stdout:3/189: dwrite d1/d3/f11 [0,4194304] 0 2024-04-12T20:56:26.726 INFO:tasks.workunit.client.1.smithi186.stdout:7/233: getdents d7/d10/d2d/d40 0 2024-04-12T20:56:26.728 INFO:tasks.workunit.client.1.smithi186.stdout:1/193: symlink d14/d1a/d21/d35/l3b 0 2024-04-12T20:56:26.728 INFO:tasks.workunit.client.1.smithi186.stdout:2/176: dread d0/f2 [0,4194304] 0 2024-04-12T20:56:26.732 INFO:tasks.workunit.client.1.smithi186.stdout:0/261: rename d0/c8 to d0/d10/d32/c58 0 2024-04-12T20:56:26.732 INFO:tasks.workunit.client.1.smithi186.stdout:9/171: dwrite d3/d2f/d2e/f2a [0,4194304] 0 2024-04-12T20:56:26.734 INFO:tasks.workunit.client.1.smithi186.stdout:1/194: stat c8 0 2024-04-12T20:56:26.736 INFO:tasks.workunit.client.1.smithi186.stdout:2/177: fsync d0/d3/d1a/d21/f2c 0 2024-04-12T20:56:26.739 INFO:tasks.workunit.client.1.smithi186.stdout:3/190: rename d1/c13 to d1/d3/d21/d22/d34/c38 0 2024-04-12T20:56:26.741 INFO:tasks.workunit.client.1.smithi186.stdout:7/234: rmdir d7/d10/d2d/d40/d4a/d53 0 2024-04-12T20:56:26.742 INFO:tasks.workunit.client.1.smithi186.stdout:9/172: symlink d3/l31 0 2024-04-12T20:56:26.746 INFO:tasks.workunit.client.1.smithi186.stdout:6/156: dwrite d9/f10 [0,4194304] 0 2024-04-12T20:56:26.749 INFO:tasks.workunit.client.1.smithi186.stdout:0/262: symlink d0/d14/d4e/d1e/l59 0 2024-04-12T20:56:26.751 INFO:tasks.workunit.client.1.smithi186.stdout:9/173: creat d3/d2f/f32 x:0 0 0 2024-04-12T20:56:26.753 INFO:tasks.workunit.client.1.smithi186.stdout:7/235: symlink d7/d10/d2d/d40/l54 0 2024-04-12T20:56:26.756 INFO:tasks.workunit.client.1.smithi186.stdout:6/157: mkdir d9/d25/d2d/d38 0 2024-04-12T20:56:26.756 INFO:tasks.workunit.client.1.smithi186.stdout:2/178: dread d0/f10 [0,4194304] 0 2024-04-12T20:56:26.757 INFO:tasks.workunit.client.1.smithi186.stdout:9/174: read d3/f10 [1593222,118562] 0 2024-04-12T20:56:26.757 INFO:tasks.workunit.client.1.smithi186.stdout:0/263: unlink d0/d10/d32/d3f/l53 0 2024-04-12T20:56:26.760 INFO:tasks.workunit.client.1.smithi186.stdout:0/264: chown d0 89586 1 2024-04-12T20:56:26.761 INFO:tasks.workunit.client.1.smithi186.stdout:7/236: mkdir d7/d55 0 2024-04-12T20:56:26.765 INFO:tasks.workunit.client.1.smithi186.stdout:0/265: write d0/d14/d4e/d1e/f2f [750922,88275] 0 2024-04-12T20:56:26.767 INFO:tasks.workunit.client.1.smithi186.stdout:6/158: rmdir d9/d25/d2d/d38 0 2024-04-12T20:56:26.769 INFO:tasks.workunit.client.1.smithi186.stdout:1/195: dwrite d14/f15 [0,4194304] 0 2024-04-12T20:56:26.769 INFO:tasks.workunit.client.1.smithi186.stdout:2/179: rename d0/d3/d12 to d0/d32 0 2024-04-12T20:56:26.769 INFO:tasks.workunit.client.1.smithi186.stdout:4/270: sync 2024-04-12T20:56:26.769 INFO:tasks.workunit.client.1.smithi186.stdout:5/163: sync 2024-04-12T20:56:26.769 INFO:tasks.workunit.client.1.smithi186.stdout:8/130: sync 2024-04-12T20:56:26.771 INFO:tasks.workunit.client.1.smithi186.stdout:9/175: mkdir d3/d2f/d2e/d33 0 2024-04-12T20:56:26.772 INFO:tasks.workunit.client.1.smithi186.stdout:2/180: write d0/f14 [4817390,100705] 0 2024-04-12T20:56:26.774 INFO:tasks.workunit.client.1.smithi186.stdout:0/266: fdatasync d0/d14/d4e/d1e/f37 0 2024-04-12T20:56:26.775 INFO:tasks.workunit.client.1.smithi186.stdout:1/196: mkdir d14/d1a/d3c 0 2024-04-12T20:56:26.775 INFO:tasks.workunit.client.1.smithi186.stdout:5/164: chown d5/db/d14/l1b 1227050 1 2024-04-12T20:56:26.778 INFO:tasks.workunit.client.1.smithi186.stdout:5/165: fsync d5/db/f2b 0 2024-04-12T20:56:26.779 INFO:tasks.workunit.client.1.smithi186.stdout:3/191: truncate d1/d3/f1f 3622734 0 2024-04-12T20:56:26.781 INFO:tasks.workunit.client.1.smithi186.stdout:6/159: rename d9/f24 to d9/d25/f39 0 2024-04-12T20:56:26.782 INFO:tasks.workunit.client.1.smithi186.stdout:3/192: read d1/d3/f11 [2712855,39281] 0 2024-04-12T20:56:26.786 INFO:tasks.workunit.client.1.smithi186.stdout:0/267: rmdir d0/d14 39 2024-04-12T20:56:26.792 INFO:tasks.workunit.client.1.smithi186.stdout:3/193: readlink d1/l18 0 2024-04-12T20:56:26.792 INFO:tasks.workunit.client.1.smithi186.stdout:5/166: mknod d5/db/d14/c35 0 2024-04-12T20:56:26.792 INFO:tasks.workunit.client.1.smithi186.stdout:7/237: dwrite d7/d10/d1c/f52 [0,4194304] 0 2024-04-12T20:56:26.793 INFO:tasks.workunit.client.1.smithi186.stdout:6/160: mknod d9/d25/d2d/c3a 0 2024-04-12T20:56:26.797 INFO:tasks.workunit.client.1.smithi186.stdout:8/131: dread f5 [0,4194304] 0 2024-04-12T20:56:26.798 INFO:tasks.workunit.client.1.smithi186.stdout:7/238: mknod d7/d10/d2d/d40/d4a/c56 0 2024-04-12T20:56:26.804 INFO:tasks.workunit.client.1.smithi186.stdout:1/197: dread fe [0,4194304] 0 2024-04-12T20:56:26.805 INFO:tasks.workunit.client.1.smithi186.stdout:8/132: symlink d7/d8/d20/d23/l2c 0 2024-04-12T20:56:26.807 INFO:tasks.workunit.client.1.smithi186.stdout:7/239: stat d7/d10/d1c/l38 0 2024-04-12T20:56:26.807 INFO:tasks.workunit.client.1.smithi186.stdout:6/161: getdents d9/d25/d2d 0 2024-04-12T20:56:26.812 INFO:tasks.workunit.client.1.smithi186.stdout:8/133: write d7/de/d10/f27 [901317,63621] 0 2024-04-12T20:56:26.813 INFO:tasks.workunit.client.1.smithi186.stdout:7/240: mkdir d7/d55/d57 0 2024-04-12T20:56:26.813 INFO:tasks.workunit.client.1.smithi186.stdout:2/181: dwrite d0/dc/f29 [0,4194304] 0 2024-04-12T20:56:26.815 INFO:tasks.workunit.client.1.smithi186.stdout:1/198: mknod d14/d1a/d21/d35/c3d 0 2024-04-12T20:56:26.816 INFO:tasks.workunit.client.1.smithi186.stdout:8/134: chown d7/d8/d20/d23/d25 0 1 2024-04-12T20:56:26.819 INFO:tasks.workunit.client.1.smithi186.stdout:9/176: dwrite d3/fb [0,4194304] 0 2024-04-12T20:56:26.828 INFO:tasks.workunit.client.1.smithi186.stdout:9/177: symlink d3/l34 0 2024-04-12T20:56:26.829 INFO:tasks.workunit.client.1.smithi186.stdout:1/199: write d14/d1a/d21/f31 [1050106,107351] 0 2024-04-12T20:56:26.833 INFO:tasks.workunit.client.1.smithi186.stdout:3/194: dwrite d1/f1e [0,4194304] 0 2024-04-12T20:56:26.835 INFO:tasks.workunit.client.1.smithi186.stdout:9/178: rmdir d3/d12 39 2024-04-12T20:56:26.842 INFO:tasks.workunit.client.1.smithi186.stdout:6/162: dwrite d9/f33 [0,4194304] 0 2024-04-12T20:56:26.842 INFO:tasks.workunit.client.1.smithi186.stdout:8/135: truncate d7/f12 4059807 0 2024-04-12T20:56:26.848 INFO:tasks.workunit.client.1.smithi186.stdout:3/195: rename d1/d3/da/c17 to d1/d3/d21/d22/d24/d28/c39 0 2024-04-12T20:56:26.849 INFO:tasks.workunit.client.1.smithi186.stdout:6/163: symlink d9/d25/l3b 0 2024-04-12T20:56:26.850 INFO:tasks.workunit.client.1.smithi186.stdout:2/182: dwrite d0/f14 [0,4194304] 0 2024-04-12T20:56:26.853 INFO:tasks.workunit.client.1.smithi186.stdout:9/179: creat d3/d12/d27/f35 x:0 0 0 2024-04-12T20:56:26.854 INFO:tasks.workunit.client.1.smithi186.stdout:8/136: write d7/de/d10/f27 [1909130,30548] 0 2024-04-12T20:56:26.856 INFO:tasks.workunit.client.1.smithi186.stdout:7/241: dread d7/d10/d2d/f37 [4194304,4194304] 0 2024-04-12T20:56:26.856 INFO:tasks.workunit.client.1.smithi186.stdout:3/196: write d1/fc [2886442,36249] 0 2024-04-12T20:56:26.856 INFO:tasks.workunit.client.1.smithi186.stdout:9/180: read - d3/d2f/f1e zero size 2024-04-12T20:56:26.860 INFO:tasks.workunit.client.1.smithi186.stdout:2/183: creat d0/d3/d1a/d24/f33 x:0 0 0 2024-04-12T20:56:26.860 INFO:tasks.workunit.client.1.smithi186.stdout:2/184: stat d0 0 2024-04-12T20:56:26.860 INFO:tasks.workunit.client.1.smithi186.stdout:3/197: chown d1/d3/d21/d22 17036 1 2024-04-12T20:56:26.863 INFO:tasks.workunit.client.1.smithi186.stdout:2/185: write d0/dc/f29 [4389926,92304] 0 2024-04-12T20:56:26.863 INFO:tasks.workunit.client.1.smithi186.stdout:1/200: dread d14/d1a/f1b [0,4194304] 0 2024-04-12T20:56:26.866 INFO:tasks.workunit.client.1.smithi186.stdout:8/137: symlink d7/d8/d20/d23/d25/l2d 0 2024-04-12T20:56:26.867 INFO:tasks.workunit.client.1.smithi186.stdout:2/186: chown d0/c5 45375535 1 2024-04-12T20:56:26.869 INFO:tasks.workunit.client.1.smithi186.stdout:7/242: creat d7/d10/d2d/f58 x:0 0 0 2024-04-12T20:56:26.869 INFO:tasks.workunit.client.1.smithi186.stdout:3/198: symlink d1/l3a 0 2024-04-12T20:56:26.877 INFO:tasks.workunit.client.1.smithi186.stdout:2/187: creat d0/d3/d1a/d24/f34 x:0 0 0 2024-04-12T20:56:26.881 INFO:tasks.workunit.client.1.smithi186.stdout:7/243: creat d7/d55/f59 x:0 0 0 2024-04-12T20:56:26.881 INFO:tasks.workunit.client.1.smithi186.stdout:6/164: dwrite d9/d25/f39 [0,4194304] 0 2024-04-12T20:56:26.883 INFO:tasks.workunit.client.1.smithi186.stdout:2/188: write d0/d3/d1a/f20 [4902865,95964] 0 2024-04-12T20:56:26.889 INFO:tasks.workunit.client.1.smithi186.stdout:2/189: dread - d0/d3/d1a/d21/f2c zero size 2024-04-12T20:56:26.890 INFO:tasks.workunit.client.1.smithi186.stdout:8/138: dwrite d7/d8/f1b [4194304,4194304] 0 2024-04-12T20:56:26.892 INFO:tasks.workunit.client.1.smithi186.stdout:7/244: mknod d7/d10/d2d/d40/d4a/c5a 0 2024-04-12T20:56:26.895 INFO:tasks.workunit.client.1.smithi186.stdout:7/245: write d7/f16 [5082381,29521] 0 2024-04-12T20:56:26.902 INFO:tasks.workunit.client.1.smithi186.stdout:6/165: unlink d9/fe 0 2024-04-12T20:56:26.904 INFO:tasks.workunit.client.1.smithi186.stdout:8/139: rmdir d7/d8/d1d/d1f 39 2024-04-12T20:56:26.905 INFO:tasks.workunit.client.1.smithi186.stdout:9/181: dread d3/f10 [4194304,4194304] 0 2024-04-12T20:56:26.910 INFO:tasks.workunit.client.1.smithi186.stdout:8/140: mkdir d7/d8/d20/d23/d25/d2e 0 2024-04-12T20:56:26.911 INFO:tasks.workunit.client.1.smithi186.stdout:3/199: dwrite d1/d3/f11 [0,4194304] 0 2024-04-12T20:56:26.911 INFO:tasks.workunit.client.1.smithi186.stdout:9/182: mkdir d3/d2f/d2e/d33/d36 0 2024-04-12T20:56:26.912 INFO:tasks.workunit.client.1.smithi186.stdout:8/141: chown d7/d8/d20/d23/d25 1 1 2024-04-12T20:56:26.913 INFO:tasks.workunit.client.1.smithi186.stdout:6/166: rename c8 to d9/d25/c3c 0 2024-04-12T20:56:26.919 INFO:tasks.workunit.client.1.smithi186.stdout:3/200: symlink d1/d3/d21/d22/l3b 0 2024-04-12T20:56:26.920 INFO:tasks.workunit.client.1.smithi186.stdout:6/167: symlink d9/l3d 0 2024-04-12T20:56:26.924 INFO:tasks.workunit.client.1.smithi186.stdout:2/190: truncate d0/f10 2954313 0 2024-04-12T20:56:26.924 INFO:tasks.workunit.client.1.smithi186.stdout:4/271: sync 2024-04-12T20:56:26.925 INFO:tasks.workunit.client.1.smithi186.stdout:6/168: mknod d9/c3e 0 2024-04-12T20:56:26.927 INFO:tasks.workunit.client.1.smithi186.stdout:9/183: dread d3/d12/f1c [0,4194304] 0 2024-04-12T20:56:26.928 INFO:tasks.workunit.client.1.smithi186.stdout:3/201: rename d1/d3/d21/d22/d24/d28/c30 to d1/d2c/c3c 0 2024-04-12T20:56:26.932 INFO:tasks.workunit.client.1.smithi186.stdout:2/191: symlink d0/d32/d2a/l35 0 2024-04-12T20:56:26.936 INFO:tasks.workunit.client.1.smithi186.stdout:7/246: dwrite d7/d10/f46 [0,4194304] 0 2024-04-12T20:56:26.941 INFO:tasks.workunit.client.1.smithi186.stdout:9/184: creat d3/d2f/d2e/d33/d36/f37 x:0 0 0 2024-04-12T20:56:26.942 INFO:tasks.workunit.client.1.smithi186.stdout:3/202: stat d1/d3/d21/d22/d24/d28/c39 0 2024-04-12T20:56:26.942 INFO:tasks.workunit.client.1.smithi186.stdout:7/247: read - d7/d10/d2d/d40/d4a/f4c zero size 2024-04-12T20:56:26.947 INFO:tasks.workunit.client.1.smithi186.stdout:8/142: dwrite d7/de/d10/f27 [0,4194304] 0 2024-04-12T20:56:26.947 INFO:tasks.workunit.client.1.smithi186.stdout:2/192: mknod d0/d32/c36 0 2024-04-12T20:56:26.951 INFO:tasks.workunit.client.1.smithi186.stdout:6/169: dread d9/d25/f39 [0,4194304] 0 2024-04-12T20:56:26.953 INFO:tasks.workunit.client.1.smithi186.stdout:3/203: write d1/fc [3673969,118598] 0 2024-04-12T20:56:26.955 INFO:tasks.workunit.client.1.smithi186.stdout:4/272: dread d0/dc/f3b [0,4194304] 0 2024-04-12T20:56:26.955 INFO:tasks.workunit.client.1.smithi186.stdout:7/248: chown d7/d10/d1c/f21 599 1 2024-04-12T20:56:26.956 INFO:tasks.workunit.client.1.smithi186.stdout:8/143: unlink l4 0 2024-04-12T20:56:26.957 INFO:tasks.workunit.client.1.smithi186.stdout:2/193: creat d0/dc/f37 x:0 0 0 2024-04-12T20:56:26.957 INFO:tasks.workunit.client.1.smithi186.stdout:3/204: read d1/fc [2880908,106703] 0 2024-04-12T20:56:26.959 INFO:tasks.workunit.client.1.smithi186.stdout:9/185: rename d3/d13/f2c to d3/d12/d27/f38 0 2024-04-12T20:56:26.960 INFO:tasks.workunit.client.1.smithi186.stdout:6/170: write d9/d25/f35 [495611,73795] 0 2024-04-12T20:56:26.962 INFO:tasks.workunit.client.1.smithi186.stdout:2/194: readlink d0/l30 0 2024-04-12T20:56:26.964 INFO:tasks.workunit.client.1.smithi186.stdout:4/273: mkdir d0/d59 0 2024-04-12T20:56:26.965 INFO:tasks.workunit.client.1.smithi186.stdout:8/144: dread d7/de/d10/f19 [0,4194304] 0 2024-04-12T20:56:26.966 INFO:tasks.workunit.client.1.smithi186.stdout:2/195: creat d0/d32/f38 x:0 0 0 2024-04-12T20:56:26.966 INFO:tasks.workunit.client.1.smithi186.stdout:9/186: write d3/d13/f28 [2530147,4351] 0 2024-04-12T20:56:26.967 INFO:tasks.workunit.client.1.smithi186.stdout:8/145: readlink l3 0 2024-04-12T20:56:26.968 INFO:tasks.workunit.client.1.smithi186.stdout:7/249: dread d7/d10/d2d/d40/f4b [0,4194304] 0 2024-04-12T20:56:26.968 INFO:tasks.workunit.client.1.smithi186.stdout:2/196: chown d0/f2 20665162 1 2024-04-12T20:56:26.970 INFO:tasks.workunit.client.1.smithi186.stdout:2/197: fdatasync d0/f2 0 2024-04-12T20:56:26.978 INFO:tasks.workunit.client.1.smithi186.stdout:6/171: dread d9/f10 [0,4194304] 0 2024-04-12T20:56:26.982 INFO:tasks.workunit.client.1.smithi186.stdout:7/250: rename d7/d10/d1c to d7/d10/d1c/d5b 22 2024-04-12T20:56:26.982 INFO:tasks.workunit.client.1.smithi186.stdout:7/251: read - d7/f8 zero size 2024-04-12T20:56:26.985 INFO:tasks.workunit.client.1.smithi186.stdout:4/274: dwrite d0/d21/f31 [4194304,4194304] 0 2024-04-12T20:56:26.985 INFO:tasks.workunit.client.1.smithi186.stdout:9/187: write d3/f1d [1250010,95150] 0 2024-04-12T20:56:26.987 INFO:tasks.workunit.client.1.smithi186.stdout:4/275: chown d0/d21/d3d/l4a 17106193 1 2024-04-12T20:56:26.988 INFO:tasks.workunit.client.1.smithi186.stdout:8/146: getdents d7/d8/d1d/d1f/d26 0 2024-04-12T20:56:26.989 INFO:tasks.workunit.client.1.smithi186.stdout:7/252: mkdir d7/d10/d2d/d40/d4a/d5c 0 2024-04-12T20:56:26.990 INFO:tasks.workunit.client.1.smithi186.stdout:6/172: mkdir d9/d3f 0 2024-04-12T20:56:26.991 INFO:tasks.workunit.client.1.smithi186.stdout:9/188: mknod d3/d2f/d2e/d33/d36/c39 0 2024-04-12T20:56:26.994 INFO:tasks.workunit.client.1.smithi186.stdout:7/253: write d7/d10/d2d/f37 [7490498,93279] 0 2024-04-12T20:56:26.999 INFO:tasks.workunit.client.1.smithi186.stdout:9/189: mknod d3/c3a 0 2024-04-12T20:56:27.000 INFO:tasks.workunit.client.1.smithi186.stdout:6/173: creat d9/f40 x:0 0 0 2024-04-12T20:56:27.000 INFO:tasks.workunit.client.1.smithi186.stdout:7/254: mkdir d7/d10/d1c/d5d 0 2024-04-12T20:56:27.005 INFO:tasks.workunit.client.1.smithi186.stdout:6/174: symlink d9/l41 0 2024-04-12T20:56:27.005 INFO:tasks.workunit.client.1.smithi186.stdout:2/198: dwrite d0/f10 [0,4194304] 0 2024-04-12T20:56:27.015 INFO:tasks.workunit.client.1.smithi186.stdout:5/167: sync 2024-04-12T20:56:27.016 INFO:tasks.workunit.client.1.smithi186.stdout:8/147: dwrite d7/de/d10/f15 [0,4194304] 0 2024-04-12T20:56:27.018 INFO:tasks.workunit.client.1.smithi186.stdout:0/268: sync 2024-04-12T20:56:27.024 INFO:tasks.workunit.client.1.smithi186.stdout:5/168: readlink d5/db/l13 0 2024-04-12T20:56:27.026 INFO:tasks.workunit.client.1.smithi186.stdout:7/255: dwrite d7/d10/d2d/f58 [0,4194304] 0 2024-04-12T20:56:27.026 INFO:tasks.workunit.client.1.smithi186.stdout:8/148: mkdir d7/de/d10/d2f 0 2024-04-12T20:56:27.034 INFO:tasks.workunit.client.1.smithi186.stdout:5/169: getdents d5/d30 0 2024-04-12T20:56:27.037 INFO:tasks.workunit.client.1.smithi186.stdout:2/199: dwrite d0/f6 [4194304,4194304] 0 2024-04-12T20:56:27.037 INFO:tasks.workunit.client.1.smithi186.stdout:6/175: dread d9/f12 [0,4194304] 0 2024-04-12T20:56:27.041 INFO:tasks.workunit.client.1.smithi186.stdout:7/256: rename d7/d10/d1c/l47 to d7/l5e 0 2024-04-12T20:56:27.049 INFO:tasks.workunit.client.1.smithi186.stdout:0/269: truncate d0/d14/f1a 4464265 0 2024-04-12T20:56:27.049 INFO:tasks.workunit.client.1.smithi186.stdout:2/200: creat d0/d3/d1a/d21/f39 x:0 0 0 2024-04-12T20:56:27.052 INFO:tasks.workunit.client.1.smithi186.stdout:7/257: rename d7/d10/d2d/d45 to d7/d55/d5f 0 2024-04-12T20:56:27.055 INFO:tasks.workunit.client.1.smithi186.stdout:2/201: rename d0/d3/d1a/d21/f2c to d0/d3/d1a/f3a 0 2024-04-12T20:56:27.058 INFO:tasks.workunit.client.1.smithi186.stdout:8/149: dwrite d7/d8/fa [0,4194304] 0 2024-04-12T20:56:27.059 INFO:tasks.workunit.client.1.smithi186.stdout:2/202: rename d0/d32 to d0/d32/d3b 22 2024-04-12T20:56:27.062 INFO:tasks.workunit.client.1.smithi186.stdout:0/270: creat d0/d14/d4e/d46/f5a x:0 0 0 2024-04-12T20:56:27.062 INFO:tasks.workunit.client.1.smithi186.stdout:7/258: mkdir d7/d10/d1c/d5d/d60 0 2024-04-12T20:56:27.064 INFO:tasks.workunit.client.1.smithi186.stdout:5/170: write d5/f9 [2600525,64118] 0 2024-04-12T20:56:27.064 INFO:tasks.workunit.client.1.smithi186.stdout:2/203: mknod d0/dc/d2e/c3c 0 2024-04-12T20:56:27.066 INFO:tasks.workunit.client.1.smithi186.stdout:8/150: dread - d7/d8/d17/f22 zero size 2024-04-12T20:56:27.069 INFO:tasks.workunit.client.1.smithi186.stdout:0/271: write d0/d14/d4e/d1e/f56 [804852,96911] 0 2024-04-12T20:56:27.070 INFO:tasks.workunit.client.1.smithi186.stdout:6/176: dwrite d9/d25/f39 [0,4194304] 0 2024-04-12T20:56:27.071 INFO:tasks.workunit.client.1.smithi186.stdout:5/171: write d5/db/fc [2699552,87111] 0 2024-04-12T20:56:27.074 INFO:tasks.workunit.client.1.smithi186.stdout:5/172: dread - d5/db/d15/f31 zero size 2024-04-12T20:56:27.075 INFO:tasks.workunit.client.1.smithi186.stdout:2/204: rename d0/d3/d1a/f20 to d0/d3/d1a/d21/d26/f3d 0 2024-04-12T20:56:27.076 INFO:tasks.workunit.client.1.smithi186.stdout:6/177: stat d9/f2a 0 2024-04-12T20:56:27.081 INFO:tasks.workunit.client.1.smithi186.stdout:1/201: sync 2024-04-12T20:56:27.082 INFO:tasks.workunit.client.1.smithi186.stdout:2/205: mkdir d0/d3e 0 2024-04-12T20:56:27.087 INFO:tasks.workunit.client.1.smithi186.stdout:5/173: rename d5/db/d14/f29 to d5/db/d33/f36 0 2024-04-12T20:56:27.089 INFO:tasks.workunit.client.1.smithi186.stdout:2/206: unlink d0/c18 0 2024-04-12T20:56:27.090 INFO:tasks.workunit.client.1.smithi186.stdout:5/174: dread - d5/db/f2b zero size 2024-04-12T20:56:27.090 INFO:tasks.workunit.client.1.smithi186.stdout:0/272: dread d0/d14/d4e/d1e/f56 [0,4194304] 0 2024-04-12T20:56:27.090 INFO:tasks.workunit.client.1.smithi186.stdout:1/202: unlink fb 0 2024-04-12T20:56:27.091 INFO:tasks.workunit.client.1.smithi186.stdout:8/151: dread d7/d8/fa [0,4194304] 0 2024-04-12T20:56:27.094 INFO:tasks.workunit.client.1.smithi186.stdout:6/178: dread d9/f2e [0,4194304] 0 2024-04-12T20:56:27.094 INFO:tasks.workunit.client.1.smithi186.stdout:1/203: fdatasync f9 0 2024-04-12T20:56:27.095 INFO:tasks.workunit.client.1.smithi186.stdout:2/207: symlink d0/d3e/l3f 0 2024-04-12T20:56:27.096 INFO:tasks.workunit.client.1.smithi186.stdout:8/152: write d7/de/f14 [372446,84579] 0 2024-04-12T20:56:27.100 INFO:tasks.workunit.client.1.smithi186.stdout:4/276: sync 2024-04-12T20:56:27.100 INFO:tasks.workunit.client.1.smithi186.stdout:9/190: sync 2024-04-12T20:56:27.101 INFO:tasks.workunit.client.1.smithi186.stdout:3/205: sync 2024-04-12T20:56:27.101 INFO:tasks.workunit.client.1.smithi186.stdout:1/204: mkdir d14/d1a/d21/d2c/d3e 0 2024-04-12T20:56:27.102 INFO:tasks.workunit.client.1.smithi186.stdout:7/259: dwrite d7/d10/f19 [0,4194304] 0 2024-04-12T20:56:27.103 INFO:tasks.workunit.client.1.smithi186.stdout:2/208: write d0/d3/d1a/d21/f39 [749949,124762] 0 2024-04-12T20:56:27.106 INFO:tasks.workunit.client.1.smithi186.stdout:3/206: readlink d1/d3/d21/d22/d24/d28/l37 0 2024-04-12T20:56:27.107 INFO:tasks.workunit.client.1.smithi186.stdout:5/175: getdents d5/db/d15 0 2024-04-12T20:56:27.108 INFO:tasks.workunit.client.1.smithi186.stdout:8/153: symlink d7/d18/l30 0 2024-04-12T20:56:27.111 INFO:tasks.workunit.client.1.smithi186.stdout:4/277: rmdir d0/d21/d3d 39 2024-04-12T20:56:27.113 INFO:tasks.workunit.client.1.smithi186.stdout:4/278: readlink d0/l2a 0 2024-04-12T20:56:27.113 INFO:tasks.workunit.client.1.smithi186.stdout:0/273: getdents d0/d14/d4e/d50 0 2024-04-12T20:56:27.115 INFO:tasks.workunit.client.1.smithi186.stdout:2/209: creat d0/dc/f40 x:0 0 0 2024-04-12T20:56:27.116 INFO:tasks.workunit.client.1.smithi186.stdout:4/279: write d0/d26/f34 [900275,83511] 0 2024-04-12T20:56:27.117 INFO:tasks.workunit.client.1.smithi186.stdout:0/274: write d0/d14/f19 [788997,101547] 0 2024-04-12T20:56:27.121 INFO:tasks.workunit.client.1.smithi186.stdout:5/176: creat d5/db/d15/d18/f37 x:0 0 0 2024-04-12T20:56:27.123 INFO:tasks.workunit.client.1.smithi186.stdout:3/207: mkdir d1/d3/d21/d22/d24/d28/d3d 0 2024-04-12T20:56:27.125 INFO:tasks.workunit.client.1.smithi186.stdout:8/154: creat d7/de/d10/d2f/f31 x:0 0 0 2024-04-12T20:56:27.125 INFO:tasks.workunit.client.1.smithi186.stdout:2/210: mkdir d0/d3/d1a/d41 0 2024-04-12T20:56:27.126 INFO:tasks.workunit.client.1.smithi186.stdout:5/177: write d5/f9 [2345193,16209] 0 2024-04-12T20:56:27.128 INFO:tasks.workunit.client.1.smithi186.stdout:6/179: truncate d9/f10 725426 0 2024-04-12T20:56:27.128 INFO:tasks.workunit.client.1.smithi186.stdout:9/191: creat d3/f3b x:0 0 0 2024-04-12T20:56:27.130 INFO:tasks.workunit.client.1.smithi186.stdout:1/205: dwrite d14/d1a/d1d/f29 [0,4194304] 0 2024-04-12T20:56:27.133 INFO:tasks.workunit.client.1.smithi186.stdout:0/275: symlink d0/d14/d4e/l5b 0 2024-04-12T20:56:27.133 INFO:tasks.workunit.client.1.smithi186.stdout:8/155: rmdir d7/d8/d1d 39 2024-04-12T20:56:27.134 INFO:tasks.workunit.client.1.smithi186.stdout:4/280: symlink d0/d59/l5a 0 2024-04-12T20:56:27.136 INFO:tasks.workunit.client.1.smithi186.stdout:6/180: creat d9/d25/d2d/f42 x:0 0 0 2024-04-12T20:56:27.140 INFO:tasks.workunit.client.1.smithi186.stdout:9/192: mknod d3/d2f/d2e/d33/c3c 0 2024-04-12T20:56:27.142 INFO:tasks.workunit.client.1.smithi186.stdout:8/156: write d7/d18/f1e [2223546,83929] 0 2024-04-12T20:56:27.145 INFO:tasks.workunit.client.1.smithi186.stdout:3/208: mknod d1/d3/d21/d22/d24/d28/d3d/c3e 0 2024-04-12T20:56:27.150 INFO:tasks.workunit.client.1.smithi186.stdout:3/209: fsync d1/f19 0 2024-04-12T20:56:27.152 INFO:tasks.workunit.client.1.smithi186.stdout:9/193: unlink d3/d2f/f32 0 2024-04-12T20:56:27.152 INFO:tasks.workunit.client.1.smithi186.stdout:8/157: readlink d7/d18/l30 0 2024-04-12T20:56:27.156 INFO:tasks.workunit.client.1.smithi186.stdout:1/206: creat d14/d1a/f3f x:0 0 0 2024-04-12T20:56:27.157 INFO:tasks.workunit.client.1.smithi186.stdout:5/178: link d5/db/d15/d18/c24 d5/db/d15/d18/c38 0 2024-04-12T20:56:27.158 INFO:tasks.workunit.client.1.smithi186.stdout:8/158: read d7/d8/fa [699228,76201] 0 2024-04-12T20:56:27.159 INFO:tasks.workunit.client.1.smithi186.stdout:7/260: dwrite d7/d10/d1c/f1e [0,4194304] 0 2024-04-12T20:56:27.160 INFO:tasks.workunit.client.1.smithi186.stdout:2/211: dwrite d0/dc/f40 [0,4194304] 0 2024-04-12T20:56:27.168 INFO:tasks.workunit.client.1.smithi186.stdout:6/181: getdents d9 0 2024-04-12T20:56:27.168 INFO:tasks.workunit.client.1.smithi186.stdout:9/194: write d3/d2f/d2e/d33/d36/f37 [527593,122859] 0 2024-04-12T20:56:27.171 INFO:tasks.workunit.client.1.smithi186.stdout:1/207: symlink d14/d1a/d1d/l40 0 2024-04-12T20:56:27.172 INFO:tasks.workunit.client.1.smithi186.stdout:8/159: readlink l3 0 2024-04-12T20:56:27.176 INFO:tasks.workunit.client.1.smithi186.stdout:7/261: unlink d7/fe 0 2024-04-12T20:56:27.177 INFO:tasks.workunit.client.1.smithi186.stdout:0/276: dwrite d0/f3e [4194304,4194304] 0 2024-04-12T20:56:27.179 INFO:tasks.workunit.client.1.smithi186.stdout:5/179: readlink d5/db/l12 0 2024-04-12T20:56:27.179 INFO:tasks.workunit.client.1.smithi186.stdout:6/182: chown d9/l1e 26472 1 2024-04-12T20:56:27.179 INFO:tasks.workunit.client.1.smithi186.stdout:8/160: readlink d7/d8/d20/d23/l2c 0 2024-04-12T20:56:27.181 INFO:tasks.workunit.client.1.smithi186.stdout:2/212: symlink d0/d3/d1a/d21/d22/l42 0 2024-04-12T20:56:27.185 INFO:tasks.workunit.client.1.smithi186.stdout:4/281: dwrite d0/d21/f1b [0,4194304] 0 2024-04-12T20:56:27.190 INFO:tasks.workunit.client.1.smithi186.stdout:9/195: creat d3/d2f/d2e/d33/d36/f3d x:0 0 0 2024-04-12T20:56:27.190 INFO:tasks.workunit.client.1.smithi186.stdout:6/183: rename d9/d25 to d9/d43 0 2024-04-12T20:56:27.193 INFO:tasks.workunit.client.1.smithi186.stdout:9/196: read - d3/d13/f22 zero size 2024-04-12T20:56:27.194 INFO:tasks.workunit.client.1.smithi186.stdout:9/197: readlink d3/l34 0 2024-04-12T20:56:27.195 INFO:tasks.workunit.client.1.smithi186.stdout:1/208: mknod d14/d1a/d21/d2c/d3e/c41 0 2024-04-12T20:56:27.200 INFO:tasks.workunit.client.1.smithi186.stdout:9/198: dread - d3/d2f/d2e/f2d zero size 2024-04-12T20:56:27.200 INFO:tasks.workunit.client.1.smithi186.stdout:4/282: link d0/dc/d33/f1a d0/dc/d39/f5b 0 2024-04-12T20:56:27.204 INFO:tasks.workunit.client.1.smithi186.stdout:5/180: unlink d5/db/l22 0 2024-04-12T20:56:27.205 INFO:tasks.workunit.client.1.smithi186.stdout:6/184: creat d9/d43/f44 x:0 0 0 2024-04-12T20:56:27.207 INFO:tasks.workunit.client.1.smithi186.stdout:9/199: dread - d3/d2f/f1e zero size 2024-04-12T20:56:27.210 INFO:tasks.workunit.client.0.smithi018.stdout:1/827: sync 2024-04-12T20:56:27.211 INFO:tasks.workunit.client.1.smithi186.stdout:4/283: mkdir d0/dc/d33/d5c 0 2024-04-12T20:56:27.212 INFO:tasks.workunit.client.1.smithi186.stdout:0/277: write d0/d14/f1a [1115698,119945] 0 2024-04-12T20:56:27.213 INFO:tasks.workunit.client.0.smithi018.stdout:1/828: readlink d3/d9/da/dd/d31/d32/d13/d19/d2d/d4e/d7a/d82/la3 0 2024-04-12T20:56:27.213 INFO:tasks.workunit.client.1.smithi186.stdout:2/213: dread d0/d3/f8 [0,4194304] 0 2024-04-12T20:56:27.216 INFO:tasks.workunit.client.0.smithi018.stdout:1/829: stat d3/d9/da/dd/d31/d32/d13/d19/d2d/d4e/d5e/fa7 0 2024-04-12T20:56:27.217 INFO:tasks.workunit.client.1.smithi186.stdout:6/185: dread d9/d43/f35 [0,4194304] 0 2024-04-12T20:56:27.217 INFO:tasks.workunit.client.1.smithi186.stdout:7/262: dwrite d7/d10/f19 [0,4194304] 0 2024-04-12T20:56:27.217 INFO:tasks.workunit.client.1.smithi186.stdout:8/161: link d7/d8/d1d/d1f/c2b d7/d8/d20/d23/d25/d2e/c32 0 2024-04-12T20:56:27.218 INFO:tasks.workunit.client.1.smithi186.stdout:9/200: creat d3/f3e x:0 0 0 2024-04-12T20:56:27.219 INFO:tasks.workunit.client.1.smithi186.stdout:0/278: fsync d0/d10/d32/f3d 0 2024-04-12T20:56:27.219 INFO:tasks.workunit.client.1.smithi186.stdout:2/214: dread - d0/d3/d1a/f3a zero size 2024-04-12T20:56:27.219 INFO:tasks.workunit.client.0.smithi018.stdout:1/830: write d3/d9/da/dd/f12 [32689,47722] 0 2024-04-12T20:56:27.221 INFO:tasks.workunit.client.1.smithi186.stdout:4/284: write d0/f24 [854439,50411] 0 2024-04-12T20:56:27.225 INFO:tasks.workunit.client.0.smithi018.stdout:1/831: read d3/d9/da/f4c [1471592,33270] 0 2024-04-12T20:56:27.226 INFO:tasks.workunit.client.1.smithi186.stdout:2/215: write d0/dc/f40 [181397,70040] 0 2024-04-12T20:56:27.227 INFO:tasks.workunit.client.1.smithi186.stdout:7/263: write d7/d10/d2d/d40/d4a/f51 [172479,53832] 0 2024-04-12T20:56:27.231 INFO:tasks.workunit.client.1.smithi186.stdout:3/210: sync 2024-04-12T20:56:27.232 INFO:tasks.workunit.client.1.smithi186.stdout:2/216: write d0/d3/d1a/d24/f34 [326148,78538] 0 2024-04-12T20:56:27.233 INFO:tasks.workunit.client.0.smithi018.stdout:1/832: creat d3/db8/f118 x:0 0 0 2024-04-12T20:56:27.234 INFO:tasks.workunit.client.1.smithi186.stdout:6/186: creat d9/d43/f45 x:0 0 0 2024-04-12T20:56:27.237 INFO:tasks.workunit.client.1.smithi186.stdout:0/279: mkdir d0/d5c 0 2024-04-12T20:56:27.237 INFO:tasks.workunit.client.1.smithi186.stdout:1/209: dwrite d14/d1a/d21/d2c/f2d [0,4194304] 0 2024-04-12T20:56:27.238 INFO:tasks.workunit.client.1.smithi186.stdout:3/211: mknod d1/d2c/c3f 0 2024-04-12T20:56:27.239 INFO:tasks.workunit.client.1.smithi186.stdout:6/187: fsync d9/d43/f39 0 2024-04-12T20:56:27.239 INFO:tasks.workunit.client.1.smithi186.stdout:2/217: write d0/dc/f37 [587339,86553] 0 2024-04-12T20:56:27.242 INFO:tasks.workunit.client.1.smithi186.stdout:9/201: rename d3/fe to d3/d2f/d2e/d33/f3f 0 2024-04-12T20:56:27.244 INFO:tasks.workunit.client.1.smithi186.stdout:3/212: truncate d1/f19 9163519 0 2024-04-12T20:56:27.248 INFO:tasks.workunit.client.1.smithi186.stdout:0/280: write d0/d14/d4e/d1e/f56 [655096,39181] 0 2024-04-12T20:56:27.248 INFO:tasks.workunit.client.1.smithi186.stdout:2/218: dread - d0/d3/d1a/d21/d26/f2d zero size 2024-04-12T20:56:27.248 INFO:tasks.workunit.client.1.smithi186.stdout:7/264: rmdir d7/d55/d57 0 2024-04-12T20:56:27.250 INFO:tasks.workunit.client.1.smithi186.stdout:9/202: dread d3/d12/f1c [0,4194304] 0 2024-04-12T20:56:27.253 INFO:tasks.workunit.client.0.smithi018.stdout:1/833: dwrite d3/d9/da/d14/d73/fd0 [0,4194304] 0 2024-04-12T20:56:27.255 INFO:tasks.workunit.client.0.smithi018.stdout:1/834: chown d3/d65/l10f 4680024 1 2024-04-12T20:56:27.261 INFO:tasks.workunit.client.0.smithi018.stdout:1/835: symlink d3/d9/da/dd/d31/d32/d13/d19/d2d/d4e/d7a/d82/l119 0 2024-04-12T20:56:27.264 INFO:tasks.workunit.client.0.smithi018.stdout:1/836: mknod d3/d9/da/dd/d31/d32/d13/c11a 0 2024-04-12T20:56:27.267 INFO:tasks.workunit.client.1.smithi186.stdout:4/285: dwrite d0/f42 [0,4194304] 0 2024-04-12T20:56:27.271 INFO:tasks.workunit.client.1.smithi186.stdout:8/162: getdents d7/d8/d20/d23/d25 0 2024-04-12T20:56:27.271 INFO:tasks.workunit.client.1.smithi186.stdout:5/181: truncate d5/db/f11 166832 0 2024-04-12T20:56:27.273 INFO:tasks.workunit.client.0.smithi018.stdout:1/837: mkdir d3/d9/da/dd/d31/d32/d13/d19/d2d/d4e/d58/d5c/d11b 0 2024-04-12T20:56:27.273 INFO:tasks.workunit.client.1.smithi186.stdout:1/210: write fe [5078008,53144] 0 2024-04-12T20:56:27.274 INFO:tasks.workunit.client.1.smithi186.stdout:5/182: chown d5/fd 0 1 2024-04-12T20:56:27.274 INFO:tasks.workunit.client.1.smithi186.stdout:2/219: creat d0/d32/d2a/f43 x:0 0 0 2024-04-12T20:56:27.274 INFO:tasks.workunit.client.0.smithi018.stdout:1/838: chown d3/f4 78745 1 2024-04-12T20:56:27.275 INFO:tasks.workunit.client.1.smithi186.stdout:3/213: creat d1/d3/d21/d22/d24/d28/d3d/f40 x:0 0 0 2024-04-12T20:56:27.276 INFO:tasks.workunit.client.0.smithi018.stdout:1/839: dread - d3/db8/f118 zero size 2024-04-12T20:56:27.276 INFO:tasks.workunit.client.1.smithi186.stdout:0/281: mkdir d0/d5d 0 2024-04-12T20:56:27.277 INFO:tasks.workunit.client.1.smithi186.stdout:8/163: symlink d7/de/d10/l33 0 2024-04-12T20:56:27.279 INFO:tasks.workunit.client.1.smithi186.stdout:3/214: stat d1/d3/d21/d22/d34 0 2024-04-12T20:56:27.280 INFO:tasks.workunit.client.0.smithi018.stdout:2/919: sync 2024-04-12T20:56:27.280 INFO:tasks.workunit.client.0.smithi018.stdout:4/965: sync 2024-04-12T20:56:27.280 INFO:tasks.workunit.client.1.smithi186.stdout:5/183: chown d5/db/d14/c35 7917 1 2024-04-12T20:56:27.282 INFO:tasks.workunit.client.1.smithi186.stdout:4/286: rename d0/dc/d1f/f37 to d0/dc/d33/f5d 0 2024-04-12T20:56:27.282 INFO:tasks.workunit.client.1.smithi186.stdout:1/211: read f4 [716998,54260] 0 2024-04-12T20:56:27.283 INFO:tasks.workunit.client.1.smithi186.stdout:0/282: rename d0/d14/d4e/d46 to d0/d14/d4e/d46/d5e 22 2024-04-12T20:56:27.284 INFO:tasks.workunit.client.0.smithi018.stdout:1/840: creat d3/d9/da/d14/d73/f11c x:0 0 0 2024-04-12T20:56:27.284 INFO:tasks.workunit.client.0.smithi018.stdout:4/966: write d3/d32/d66/f144 [292053,86183] 0 2024-04-12T20:56:27.284 INFO:tasks.workunit.client.0.smithi018.stdout:1/841: chown d3/d9 449 1 2024-04-12T20:56:27.284 INFO:tasks.workunit.client.0.smithi018.stdout:2/920: symlink d5/d7/d79/l13d 0 2024-04-12T20:56:27.285 INFO:tasks.workunit.client.1.smithi186.stdout:2/220: chown d0/d3/d1a/d21/d22/l42 65886 1 2024-04-12T20:56:27.285 INFO:tasks.workunit.client.0.smithi018.stdout:4/967: stat d3/d5d/d60/dc5/d40/l9d 0 2024-04-12T20:56:27.287 INFO:tasks.workunit.client.0.smithi018.stdout:2/921: dread - d5/d7/d31/d107/f109 zero size 2024-04-12T20:56:27.289 INFO:tasks.workunit.client.0.smithi018.stdout:1/842: readlink d3/d9/da/dd/d31/d32/lc0 0 2024-04-12T20:56:27.290 INFO:tasks.workunit.client.1.smithi186.stdout:1/212: write d14/d1a/d21/d2c/f38 [1039253,123973] 0 2024-04-12T20:56:27.291 INFO:tasks.workunit.client.0.smithi018.stdout:4/968: mkdir d3/d5d/d60/dc5/d3d/da3/d146 0 2024-04-12T20:56:27.293 INFO:tasks.workunit.client.1.smithi186.stdout:0/283: creat d0/d5c/f5f x:0 0 0 2024-04-12T20:56:27.293 INFO:tasks.workunit.client.0.smithi018.stdout:4/969: write d3/db/d114/f134 [935369,27550] 0 2024-04-12T20:56:27.294 INFO:tasks.workunit.client.1.smithi186.stdout:2/221: creat d0/d3/d1a/d41/f44 x:0 0 0 2024-04-12T20:56:27.300 INFO:tasks.workunit.client.0.smithi018.stdout:4/970: unlink d3/d32/d33/l36 0 2024-04-12T20:56:27.302 INFO:tasks.workunit.client.1.smithi186.stdout:1/213: write d14/d1a/d21/d2c/f38 [574226,97131] 0 2024-04-12T20:56:27.304 INFO:tasks.workunit.client.1.smithi186.stdout:3/215: write d1/d3/f1f [2138,16924] 0 2024-04-12T20:56:27.305 INFO:tasks.workunit.client.1.smithi186.stdout:8/164: dread d7/d18/f1e [0,4194304] 0 2024-04-12T20:56:27.308 INFO:tasks.workunit.client.1.smithi186.stdout:2/222: write d0/f6 [7321537,96647] 0 2024-04-12T20:56:27.309 INFO:tasks.workunit.client.1.smithi186.stdout:9/203: dwrite d3/d12/d27/f38 [0,4194304] 0 2024-04-12T20:56:27.310 INFO:tasks.workunit.client.0.smithi018.stdout:2/922: dwrite d5/d7/d31/d62/da6/df8/d137/f139 [0,4194304] 0 2024-04-12T20:56:27.314 INFO:tasks.workunit.client.1.smithi186.stdout:0/284: symlink d0/d5d/l60 0 2024-04-12T20:56:27.315 INFO:tasks.workunit.client.1.smithi186.stdout:5/184: dwrite d5/db/d15/f31 [0,4194304] 0 2024-04-12T20:56:27.316 INFO:tasks.workunit.client.0.smithi018.stdout:2/923: dread - d5/d8/f10e zero size 2024-04-12T20:56:27.320 INFO:tasks.workunit.client.1.smithi186.stdout:1/214: symlink d14/d1a/d21/d35/l42 0 2024-04-12T20:56:27.321 INFO:tasks.workunit.client.0.smithi018.stdout:2/924: creat d5/d8/d10/d43/de2/daf/d90/f13e x:0 0 0 2024-04-12T20:56:27.321 INFO:tasks.workunit.client.0.smithi018.stdout:1/843: dwrite d3/d9/da/dd/d31/d32/d13/d19/d2d/d4e/f60 [0,4194304] 0 2024-04-12T20:56:27.325 INFO:tasks.workunit.client.1.smithi186.stdout:1/215: write d14/d1a/d1d/f29 [4859284,22086] 0 2024-04-12T20:56:27.325 INFO:tasks.workunit.client.1.smithi186.stdout:3/216: rename d1/d3/f1f to d1/d3/d21/f41 0 2024-04-12T20:56:27.326 INFO:tasks.workunit.client.1.smithi186.stdout:2/223: truncate d0/d32/d2a/f43 45276 0 2024-04-12T20:56:27.327 INFO:tasks.workunit.client.0.smithi018.stdout:4/971: dwrite d3/d5d/d60/dc5/d3d/d80/f131 [0,4194304] 0 2024-04-12T20:56:27.327 INFO:tasks.workunit.client.0.smithi018.stdout:1/844: fsync d3/d9/da/fae 0 2024-04-12T20:56:27.328 INFO:tasks.workunit.client.1.smithi186.stdout:8/165: dread f2 [0,4194304] 0 2024-04-12T20:56:27.328 INFO:tasks.workunit.client.1.smithi186.stdout:9/204: mknod d3/d2f/d2e/c40 0 2024-04-12T20:56:27.330 INFO:tasks.workunit.client.0.smithi018.stdout:4/972: write d3/db/f8b [900636,111895] 0 2024-04-12T20:56:27.331 INFO:tasks.workunit.client.0.smithi018.stdout:2/925: creat d5/d7/d31/d107/d127/f13f x:0 0 0 2024-04-12T20:56:27.333 INFO:tasks.workunit.client.0.smithi018.stdout:2/926: stat d5/d7/d31/d62/da6/df8 0 2024-04-12T20:56:27.334 INFO:tasks.workunit.client.1.smithi186.stdout:5/185: creat d5/db/d15/f39 x:0 0 0 2024-04-12T20:56:27.338 INFO:tasks.workunit.client.1.smithi186.stdout:3/217: write d1/f19 [7271493,72198] 0 2024-04-12T20:56:27.338 INFO:tasks.workunit.client.1.smithi186.stdout:2/224: rmdir d0/d3/d1a/d24 39 2024-04-12T20:56:27.339 INFO:tasks.workunit.client.1.smithi186.stdout:8/166: readlink d7/d8/d20/d23/d25/l2d 0 2024-04-12T20:56:27.339 INFO:tasks.workunit.client.1.smithi186.stdout:4/287: dwrite d0/dc/f23 [0,4194304] 0 2024-04-12T20:56:27.340 INFO:tasks.workunit.client.0.smithi018.stdout:2/927: mknod d5/d12/d54/dc0/dfc/c140 0 2024-04-12T20:56:27.342 INFO:tasks.workunit.client.0.smithi018.stdout:4/973: symlink d3/d32/d33/dbc/l147 0 2024-04-12T20:56:27.346 INFO:tasks.workunit.client.1.smithi186.stdout:4/288: rmdir d0/dc/d39 39 2024-04-12T20:56:27.347 INFO:tasks.workunit.client.1.smithi186.stdout:2/225: stat d0/d3/d1a/d41 0 2024-04-12T20:56:27.348 INFO:tasks.workunit.client.0.smithi018.stdout:2/928: unlink d5/d8/d25/l6d 0 2024-04-12T20:56:27.350 INFO:tasks.workunit.client.0.smithi018.stdout:4/974: mknod d3/d5d/d60/dc5/d53/de7/d126/d72/c148 0 2024-04-12T20:56:27.354 INFO:tasks.workunit.client.0.smithi018.stdout:2/929: creat d5/d7/d31/d62/da6/df8/f141 x:0 0 0 2024-04-12T20:56:27.355 INFO:tasks.workunit.client.1.smithi186.stdout:1/216: rename d14/d1a/d21/d35/c3d to d14/d1a/d33/d36/c43 0 2024-04-12T20:56:27.357 INFO:tasks.workunit.client.1.smithi186.stdout:2/226: write d0/f2 [4752227,88187] 0 2024-04-12T20:56:27.358 INFO:tasks.workunit.client.1.smithi186.stdout:4/289: truncate d0/f42 4802086 0 2024-04-12T20:56:27.364 INFO:tasks.workunit.client.1.smithi186.stdout:1/217: mknod d14/d1a/d21/d2c/c44 0 2024-04-12T20:56:27.367 INFO:tasks.workunit.client.1.smithi186.stdout:4/290: chown d0/d26/f34 229573212 1 2024-04-12T20:56:27.385 INFO:tasks.workunit.client.1.smithi186.stdout:2/227: creat d0/d3/d1a/f45 x:0 0 0 2024-04-12T20:56:27.388 INFO:tasks.workunit.client.1.smithi186.stdout:4/291: rename d0/dc/d33/d5c to d0/dc/d33/d5c/d5e 22 2024-04-12T20:56:27.394 INFO:tasks.workunit.client.1.smithi186.stdout:2/228: chown d0/d32/d2a 0 1 2024-04-12T20:56:27.394 INFO:tasks.workunit.client.1.smithi186.stdout:9/205: dwrite d3/fd [0,4194304] 0 2024-04-12T20:56:27.395 INFO:tasks.workunit.client.1.smithi186.stdout:8/167: dwrite d7/fd [0,4194304] 0 2024-04-12T20:56:27.396 INFO:tasks.workunit.client.1.smithi186.stdout:5/186: dwrite d5/db/d14/f16 [0,4194304] 0 2024-04-12T20:56:27.396 INFO:tasks.workunit.client.1.smithi186.stdout:2/229: stat d0/dc/f40 0 2024-04-12T20:56:27.399 INFO:tasks.workunit.client.1.smithi186.stdout:3/218: dwrite d1/d3/d21/f41 [0,4194304] 0 2024-04-12T20:56:27.401 INFO:tasks.workunit.client.1.smithi186.stdout:4/292: rename d0/dc/l4e to d0/d26/l5f 0 2024-04-12T20:56:27.402 INFO:tasks.workunit.client.1.smithi186.stdout:9/206: write d3/d12/d27/f38 [2896834,66617] 0 2024-04-12T20:56:27.403 INFO:tasks.workunit.client.1.smithi186.stdout:8/168: fsync d7/de/d10/d2f/f31 0 2024-04-12T20:56:27.403 INFO:tasks.workunit.client.1.smithi186.stdout:4/293: chown d0/d26 932990 1 2024-04-12T20:56:27.404 INFO:tasks.workunit.client.1.smithi186.stdout:3/219: write d1/f19 [7691849,83326] 0 2024-04-12T20:56:27.407 INFO:tasks.workunit.client.1.smithi186.stdout:4/294: chown d0/d21/f41 1 1 2024-04-12T20:56:27.408 INFO:tasks.workunit.client.1.smithi186.stdout:9/207: dread - d3/f3e zero size 2024-04-12T20:56:27.416 INFO:tasks.workunit.client.1.smithi186.stdout:1/218: dread d14/d1a/f37 [0,4194304] 0 2024-04-12T20:56:27.417 INFO:tasks.workunit.client.1.smithi186.stdout:7/265: sync 2024-04-12T20:56:27.418 INFO:tasks.workunit.client.1.smithi186.stdout:6/188: sync 2024-04-12T20:56:27.420 INFO:tasks.workunit.client.1.smithi186.stdout:9/208: creat d3/d2f/d2e/f41 x:0 0 0 2024-04-12T20:56:27.428 INFO:tasks.workunit.client.0.smithi018.stdout:4/975: rename d3/d5d/d60/dc5/d43/f2a to d3/d5d/d60/dc5/d3d/da3/d12d/f149 0 2024-04-12T20:56:27.429 INFO:tasks.workunit.client.1.smithi186.stdout:2/230: dwrite d0/d3/d1a/f45 [0,4194304] 0 2024-04-12T20:56:27.429 INFO:tasks.workunit.client.1.smithi186.stdout:9/209: write d3/d2f/d2e/f2a [3692655,66643] 0 2024-04-12T20:56:27.429 INFO:tasks.workunit.client.1.smithi186.stdout:7/266: mknod d7/c61 0 2024-04-12T20:56:27.434 INFO:tasks.workunit.client.1.smithi186.stdout:5/187: dread d5/db/d14/f1a [0,4194304] 0 2024-04-12T20:56:27.434 INFO:tasks.workunit.client.1.smithi186.stdout:7/267: write d7/f17 [2470700,79711] 0 2024-04-12T20:56:27.438 INFO:tasks.workunit.client.1.smithi186.stdout:9/210: symlink d3/d12/l42 0 2024-04-12T20:56:27.442 INFO:tasks.workunit.client.1.smithi186.stdout:8/169: dread d7/de/d10/f15 [0,4194304] 0 2024-04-12T20:56:27.443 INFO:tasks.workunit.client.1.smithi186.stdout:4/295: dread d0/dc/d39/f3a [0,4194304] 0 2024-04-12T20:56:27.445 INFO:tasks.workunit.client.0.smithi018.stdout:2/930: getdents d5/d8/d10/d43/de2 0 2024-04-12T20:56:27.451 INFO:tasks.workunit.client.1.smithi186.stdout:7/268: symlink d7/d10/d2d/d40/d4a/l62 0 2024-04-12T20:56:27.452 INFO:tasks.workunit.client.1.smithi186.stdout:1/219: link d14/d1a/d21/d35/l42 d14/d1a/d21/l45 0 2024-04-12T20:56:27.453 INFO:tasks.workunit.client.1.smithi186.stdout:2/231: creat d0/d3e/f46 x:0 0 0 2024-04-12T20:56:27.454 INFO:tasks.workunit.client.1.smithi186.stdout:0/285: sync 2024-04-12T20:56:27.454 INFO:tasks.workunit.client.0.smithi018.stdout:1/845: dwrite d3/d9/da/dd/d31/d32/d33/f96 [0,4194304] 0 2024-04-12T20:56:27.455 INFO:tasks.workunit.client.1.smithi186.stdout:0/286: fdatasync d0/f4f 0 2024-04-12T20:56:27.456 INFO:tasks.workunit.client.0.smithi018.stdout:4/976: dwrite d3/d5d/f112 [0,4194304] 0 2024-04-12T20:56:27.457 INFO:tasks.workunit.client.1.smithi186.stdout:1/220: fdatasync d14/d1a/d1d/f23 0 2024-04-12T20:56:27.457 INFO:tasks.workunit.client.1.smithi186.stdout:9/211: mkdir d3/d2f/d2e/d43 0 2024-04-12T20:56:27.457 INFO:tasks.workunit.client.1.smithi186.stdout:2/232: read d0/d3/d1a/d21/d26/f3d [113968,44274] 0 2024-04-12T20:56:27.458 INFO:tasks.workunit.client.0.smithi018.stdout:2/931: unlink d5/d8/d10/d43/l131 0 2024-04-12T20:56:27.460 INFO:tasks.workunit.client.1.smithi186.stdout:8/170: creat d7/d8/d20/d23/d25/f34 x:0 0 0 2024-04-12T20:56:27.461 INFO:tasks.workunit.client.1.smithi186.stdout:1/221: readlink d14/d1a/d21/l2b 0 2024-04-12T20:56:27.461 INFO:tasks.workunit.client.1.smithi186.stdout:9/212: read - d3/ff zero size 2024-04-12T20:56:27.481 INFO:tasks.workunit.client.1.smithi186.stdout:2/233: dread d0/d3/d1a/f45 [0,4194304] 0 2024-04-12T20:56:27.482 INFO:tasks.workunit.client.0.smithi018.stdout:4/977: truncate d3/d5d/d60/dc5/fa4 628257 0 2024-04-12T20:56:27.484 INFO:tasks.workunit.client.0.smithi018.stdout:2/932: chown d5/d8/d10/d43/de2/daf/d90/c11b 12040 1 2024-04-12T20:56:27.489 INFO:tasks.workunit.client.1.smithi186.stdout:3/220: truncate d1/fc 2380087 0 2024-04-12T20:56:27.493 INFO:tasks.workunit.client.1.smithi186.stdout:9/213: creat d3/d2f/d2e/d33/f44 x:0 0 0 2024-04-12T20:56:27.496 INFO:tasks.workunit.client.0.smithi018.stdout:2/933: rename d5/d8/d10/d43/de2/d7c/d92 to d5/d8/d10/d43/de2/daf/d90/d142 0 2024-04-12T20:56:27.497 INFO:tasks.workunit.client.1.smithi186.stdout:3/221: chown d1/d3/d21/d22/d24/d28/d3d/f40 26527790 1 2024-04-12T20:56:27.498 INFO:tasks.workunit.client.0.smithi018.stdout:2/934: stat d5/d8/d10/d43/de2/daf/d90/f101 0 2024-04-12T20:56:27.500 INFO:tasks.workunit.client.1.smithi186.stdout:4/296: rename d0/c58 to d0/dc/c60 0 2024-04-12T20:56:27.501 INFO:tasks.workunit.client.1.smithi186.stdout:1/222: dwrite d14/d1a/d33/f34 [0,4194304] 0 2024-04-12T20:56:27.502 INFO:tasks.workunit.client.0.smithi018.stdout:2/935: rmdir d5/d12 39 2024-04-12T20:56:27.506 INFO:tasks.workunit.client.1.smithi186.stdout:6/189: truncate d9/f33 1783253 0 2024-04-12T20:56:27.507 INFO:tasks.workunit.client.1.smithi186.stdout:7/269: creat d7/d10/d1c/f63 x:0 0 0 2024-04-12T20:56:27.510 INFO:tasks.workunit.client.0.smithi018.stdout:2/936: symlink d5/d8/l143 0 2024-04-12T20:56:27.515 INFO:tasks.workunit.client.0.smithi018.stdout:2/937: chown d5/d7/d63/ff1 12 1 2024-04-12T20:56:27.518 INFO:tasks.workunit.client.1.smithi186.stdout:1/223: creat d14/d1a/d1d/f46 x:0 0 0 2024-04-12T20:56:27.522 INFO:tasks.workunit.client.1.smithi186.stdout:2/234: dread d0/d3/d1a/f45 [0,4194304] 0 2024-04-12T20:56:27.524 INFO:tasks.workunit.client.1.smithi186.stdout:7/270: creat d7/d10/d2d/d40/d4a/f64 x:0 0 0 2024-04-12T20:56:27.527 INFO:tasks.workunit.client.1.smithi186.stdout:6/190: creat d9/d3f/f46 x:0 0 0 2024-04-12T20:56:27.531 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:56:27 smithi186 ceph-mon[36597]: pgmap v46: 65 pgs: 65 active+clean; 2.1 GiB data, 7.6 GiB used, 529 GiB / 536 GiB avail; 58 MiB/s rd, 152 MiB/s wr, 438 op/s 2024-04-12T20:56:27.531 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:56:27 smithi186 ceph-mon[36597]: from='mgr.24359 172.21.15.186:0/1315906828' entity='mgr.smithi186.kqcfxk' 2024-04-12T20:56:27.531 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:56:27 smithi186 ceph-mon[36597]: from='mgr.24359 172.21.15.186:0/1315906828' entity='mgr.smithi186.kqcfxk' 2024-04-12T20:56:27.531 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:56:27 smithi186 ceph-mon[36597]: Reconfiguring node-exporter.smithi186 (dependencies changed)... 2024-04-12T20:56:27.531 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:56:27 smithi186 ceph-mon[36597]: Reconfiguring daemon node-exporter.smithi186 on smithi186 2024-04-12T20:56:27.532 INFO:tasks.workunit.client.1.smithi186.stdout:3/222: dwrite d1/f19 [0,4194304] 0 2024-04-12T20:56:27.539 INFO:tasks.workunit.client.1.smithi186.stdout:6/191: rename d9/d43/f35 to d9/f47 0 2024-04-12T20:56:27.542 INFO:tasks.workunit.client.1.smithi186.stdout:1/224: truncate d14/f22 469500 0 2024-04-12T20:56:27.544 INFO:tasks.workunit.client.1.smithi186.stdout:2/235: truncate d0/d32/d2a/f43 375633 0 2024-04-12T20:56:27.544 INFO:tasks.workunit.client.1.smithi186.stdout:0/287: dread d0/f6 [0,4194304] 0 2024-04-12T20:56:27.546 INFO:tasks.workunit.client.1.smithi186.stdout:7/271: mknod d7/d10/d1c/c65 0 2024-04-12T20:56:27.553 INFO:tasks.workunit.client.1.smithi186.stdout:6/192: creat d9/d3f/f48 x:0 0 0 2024-04-12T20:56:27.555 INFO:tasks.workunit.client.1.smithi186.stdout:2/236: symlink d0/d3/d1a/l47 0 2024-04-12T20:56:27.558 INFO:tasks.workunit.client.1.smithi186.stdout:8/171: dwrite d7/f12 [0,4194304] 0 2024-04-12T20:56:27.566 INFO:tasks.workunit.client.1.smithi186.stdout:5/188: dread d5/db/d15/f1e [0,4194304] 0 2024-04-12T20:56:27.566 INFO:tasks.workunit.client.1.smithi186.stdout:9/214: dwrite d3/d2f/d2e/d33/f3f [0,4194304] 0 2024-04-12T20:56:27.567 INFO:tasks.workunit.client.1.smithi186.stdout:8/172: readlink d7/d18/l30 0 2024-04-12T20:56:27.567 INFO:tasks.workunit.client.0.smithi018.stdout:1/846: dwrite d3/d9/da/dd/d31/d32/d33/d80/fca [0,4194304] 0 2024-04-12T20:56:27.570 INFO:tasks.workunit.client.1.smithi186.stdout:3/223: read d1/fc [899243,32379] 0 2024-04-12T20:56:27.574 INFO:tasks.workunit.client.0.smithi018.stdout:4/978: dwrite d3/d5d/fbd [0,4194304] 0 2024-04-12T20:56:27.576 INFO:tasks.workunit.client.1.smithi186.stdout:2/237: mkdir d0/d3/d1a/d48 0 2024-04-12T20:56:27.580 INFO:tasks.workunit.client.1.smithi186.stdout:7/272: link d7/c3d d7/d10/d2d/c66 0 2024-04-12T20:56:27.580 INFO:tasks.workunit.client.1.smithi186.stdout:9/215: creat d3/d2f/f45 x:0 0 0 2024-04-12T20:56:27.581 INFO:tasks.workunit.client.1.smithi186.stdout:1/225: dread fe [0,4194304] 0 2024-04-12T20:56:27.582 INFO:tasks.workunit.client.1.smithi186.stdout:6/193: getdents d9/d3f 0 2024-04-12T20:56:27.582 INFO:tasks.workunit.client.1.smithi186.stdout:2/238: chown d0/d3/d1a/d21/d22/c28 3833 1 2024-04-12T20:56:27.584 INFO:tasks.workunit.client.1.smithi186.stdout:6/194: readlink d9/l1e 0 2024-04-12T20:56:27.584 INFO:tasks.workunit.client.1.smithi186.stdout:9/216: write d3/d2f/d2e/f2d [1018261,125445] 0 2024-04-12T20:56:27.584 INFO:tasks.workunit.client.1.smithi186.stdout:8/173: mknod d7/de/d10/c35 0 2024-04-12T20:56:27.586 INFO:tasks.workunit.client.1.smithi186.stdout:3/224: truncate d1/fc 50098 0 2024-04-12T20:56:27.587 INFO:tasks.workunit.client.1.smithi186.stdout:5/189: creat d5/d30/f3a x:0 0 0 2024-04-12T20:56:27.588 INFO:tasks.workunit.client.0.smithi018.stdout:4/979: rename d3/d5d/d60/dc5/d43/fb1 to d3/d5d/d60/dc5/d3d/da3/d146/f14a 0 2024-04-12T20:56:27.591 INFO:tasks.workunit.client.1.smithi186.stdout:7/273: write d7/d10/d1c/f2c [4791610,21324] 0 2024-04-12T20:56:27.598 INFO:tasks.workunit.client.0.smithi018.stdout:1/847: mknod d3/d9/da/dd/d31/d32/d33/c11d 0 2024-04-12T20:56:27.600 INFO:tasks.workunit.client.0.smithi018.stdout:1/848: fdatasync d3/d9/dc3/f115 0 2024-04-12T20:56:27.600 INFO:tasks.workunit.client.1.smithi186.stdout:8/174: stat d7/de/d10/f15 0 2024-04-12T20:56:27.601 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:56:27 smithi018 ceph-mon[28799]: pgmap v46: 65 pgs: 65 active+clean; 2.1 GiB data, 7.6 GiB used, 529 GiB / 536 GiB avail; 58 MiB/s rd, 152 MiB/s wr, 438 op/s 2024-04-12T20:56:27.601 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:56:27 smithi018 ceph-mon[28799]: from='mgr.24359 172.21.15.186:0/1315906828' entity='mgr.smithi186.kqcfxk' 2024-04-12T20:56:27.602 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:56:27 smithi018 ceph-mon[28799]: from='mgr.24359 172.21.15.186:0/1315906828' entity='mgr.smithi186.kqcfxk' 2024-04-12T20:56:27.602 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:56:27 smithi018 ceph-mon[28799]: Reconfiguring node-exporter.smithi186 (dependencies changed)... 2024-04-12T20:56:27.602 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:56:27 smithi018 ceph-mon[28799]: Reconfiguring daemon node-exporter.smithi186 on smithi186 2024-04-12T20:56:27.602 INFO:tasks.workunit.client.1.smithi186.stdout:1/226: truncate d14/d1a/d21/f31 1176055 0 2024-04-12T20:56:27.604 INFO:tasks.workunit.client.0.smithi018.stdout:2/938: dwrite d5/d12/d54/d3e/f102 [0,4194304] 0 2024-04-12T20:56:27.611 INFO:tasks.workunit.client.1.smithi186.stdout:6/195: write d9/d3f/f48 [246432,63859] 0 2024-04-12T20:56:27.611 INFO:tasks.workunit.client.1.smithi186.stdout:8/175: chown d7/d8/d1d/d1f/d26 230800126 1 2024-04-12T20:56:27.612 INFO:tasks.workunit.client.0.smithi018.stdout:4/980: getdents d3/d5d 0 2024-04-12T20:56:27.613 INFO:tasks.workunit.client.1.smithi186.stdout:2/239: getdents d0 0 2024-04-12T20:56:27.613 INFO:tasks.workunit.client.1.smithi186.stdout:4/297: dread d0/f10 [0,4194304] 0 2024-04-12T20:56:27.617 INFO:tasks.workunit.client.1.smithi186.stdout:2/240: stat d0/dc/c2b 0 2024-04-12T20:56:27.618 INFO:tasks.workunit.client.0.smithi018.stdout:1/849: fdatasync d3/d9/da/dd/d31/d32/d13/d19/d2d/d4e/f5b 0 2024-04-12T20:56:27.620 INFO:tasks.workunit.client.1.smithi186.stdout:1/227: write d14/d1a/d1d/f23 [4739211,104524] 0 2024-04-12T20:56:27.620 INFO:tasks.workunit.client.1.smithi186.stdout:6/196: rmdir d9 39 2024-04-12T20:56:27.627 INFO:tasks.workunit.client.1.smithi186.stdout:5/190: rename d5/db/l10 to d5/db/d14/l3b 0 2024-04-12T20:56:27.627 INFO:tasks.workunit.client.0.smithi018.stdout:2/939: fsync d5/d8/d10/d43/de2/daf/d90/d142/de5/f5f 0 2024-04-12T20:56:27.629 INFO:tasks.workunit.client.0.smithi018.stdout:1/850: truncate d3/d9/da/dd/d31/f45 3574834 0 2024-04-12T20:56:27.630 INFO:tasks.workunit.client.1.smithi186.stdout:9/217: dwrite d3/d2f/d2e/d33/d36/f3d [0,4194304] 0 2024-04-12T20:56:27.635 INFO:tasks.workunit.client.1.smithi186.stdout:6/197: write d9/f22 [236140,42741] 0 2024-04-12T20:56:27.636 INFO:tasks.workunit.client.0.smithi018.stdout:1/851: creat d3/d9/f11e x:0 0 0 2024-04-12T20:56:27.637 INFO:tasks.workunit.client.0.smithi018.stdout:4/981: dwrite d3/d5d/d60/dc5/d43/d4b/f109 [4194304,4194304] 0 2024-04-12T20:56:27.638 INFO:tasks.workunit.client.1.smithi186.stdout:5/191: rmdir d5/db/d33 39 2024-04-12T20:56:27.639 INFO:tasks.workunit.client.0.smithi018.stdout:2/940: fsync d5/d8/d10/d43/de2/d24/f67 0 2024-04-12T20:56:27.642 INFO:tasks.workunit.client.1.smithi186.stdout:2/241: write d0/d3/d1a/f3a [217862,70665] 0 2024-04-12T20:56:27.644 INFO:tasks.workunit.client.0.smithi018.stdout:4/982: mknod d3/d5d/d129/c14b 0 2024-04-12T20:56:27.646 INFO:tasks.workunit.client.0.smithi018.stdout:2/941: symlink d5/d8/d10/d43/de2/daf/l144 0 2024-04-12T20:56:27.646 INFO:tasks.workunit.client.1.smithi186.stdout:9/218: mknod d3/d12/c46 0 2024-04-12T20:56:27.651 INFO:tasks.workunit.client.1.smithi186.stdout:6/198: readlink d9/d43/l29 0 2024-04-12T20:56:27.652 INFO:tasks.workunit.client.0.smithi018.stdout:4/983: symlink d3/d5d/d60/dc5/d53/de7/d126/d13d/d140/l14c 0 2024-04-12T20:56:27.652 INFO:tasks.workunit.client.1.smithi186.stdout:0/288: dread d0/d14/d4e/d1e/f1f [0,4194304] 0 2024-04-12T20:56:27.655 INFO:tasks.workunit.client.1.smithi186.stdout:5/192: truncate f2 6777259 0 2024-04-12T20:56:27.655 INFO:tasks.workunit.client.0.smithi018.stdout:2/942: truncate d5/d7/d31/f7a 297719 0 2024-04-12T20:56:27.659 INFO:tasks.workunit.client.0.smithi018.stdout:4/984: creat d3/d32/d33/db6/d11f/f14d x:0 0 0 2024-04-12T20:56:27.663 INFO:tasks.workunit.client.0.smithi018.stdout:2/943: truncate d5/d8/d10/d43/de2/d7c/ff6 323252 0 2024-04-12T20:56:27.664 INFO:tasks.workunit.client.1.smithi186.stdout:2/242: symlink d0/dc/l49 0 2024-04-12T20:56:27.665 INFO:tasks.workunit.client.0.smithi018.stdout:4/985: fsync d3/d32/d66/f93 0 2024-04-12T20:56:27.667 INFO:tasks.workunit.client.0.smithi018.stdout:4/986: stat d3/d5d/d60 0 2024-04-12T20:56:27.667 INFO:tasks.workunit.client.1.smithi186.stdout:3/225: dwrite d1/d3/d21/f41 [0,4194304] 0 2024-04-12T20:56:27.671 INFO:tasks.workunit.client.1.smithi186.stdout:9/219: dread d3/f10 [8388608,4194304] 0 2024-04-12T20:56:27.678 INFO:tasks.workunit.client.1.smithi186.stdout:4/298: dwrite d0/f6 [0,4194304] 0 2024-04-12T20:56:27.679 INFO:tasks.workunit.client.1.smithi186.stdout:8/176: dwrite d7/fd [0,4194304] 0 2024-04-12T20:56:27.685 INFO:tasks.workunit.client.1.smithi186.stdout:2/243: symlink d0/d3/d1a/d41/l4a 0 2024-04-12T20:56:27.686 INFO:tasks.workunit.client.1.smithi186.stdout:3/226: rename d1/d3/da/c20 to d1/d3/da/c42 0 2024-04-12T20:56:27.686 INFO:tasks.workunit.client.1.smithi186.stdout:1/228: dwrite d14/d1a/d21/f27 [0,4194304] 0 2024-04-12T20:56:27.687 INFO:tasks.workunit.client.1.smithi186.stdout:7/274: dread d7/f25 [0,4194304] 0 2024-04-12T20:56:27.691 INFO:tasks.workunit.client.1.smithi186.stdout:5/193: symlink d5/db/d15/d18/l3c 0 2024-04-12T20:56:27.691 INFO:tasks.workunit.client.0.smithi018.stdout:4/987: link d3/d5d/d60/dc5/de8/f115 d3/d5d/d60/dc5/d43/d4b/f14e 0 2024-04-12T20:56:27.699 INFO:tasks.workunit.client.0.smithi018.stdout:1/852: dread d3/d9/d11/f70 [0,4194304] 0 2024-04-12T20:56:27.699 INFO:tasks.workunit.client.1.smithi186.stdout:0/289: write d0/d10/f4d [5004987,15773] 0 2024-04-12T20:56:27.699 INFO:tasks.workunit.client.1.smithi186.stdout:3/227: symlink d1/d3/d21/d22/l43 0 2024-04-12T20:56:27.702 INFO:tasks.workunit.client.1.smithi186.stdout:8/177: write d7/d8/fa [4252485,115357] 0 2024-04-12T20:56:27.705 INFO:tasks.workunit.client.0.smithi018.stdout:2/944: dwrite d5/d8/d10/d43/de2/d24/f36 [4194304,4194304] 0 2024-04-12T20:56:27.706 INFO:tasks.workunit.client.1.smithi186.stdout:8/178: chown d7/de/d10/f27 452 1 2024-04-12T20:56:27.708 INFO:tasks.workunit.client.1.smithi186.stdout:4/299: mkdir d0/d21/d3d/d48/d61 0 2024-04-12T20:56:27.708 INFO:tasks.workunit.client.0.smithi018.stdout:2/945: dread - d5/d8/d10/f133 zero size 2024-04-12T20:56:27.709 INFO:tasks.workunit.client.1.smithi186.stdout:3/228: mknod d1/d3/d21/d22/d24/d28/d3d/c44 0 2024-04-12T20:56:27.711 INFO:tasks.workunit.client.1.smithi186.stdout:7/275: rename d7/d10/d1c/d5d/d60 to d7/d10/d2d/d67 0 2024-04-12T20:56:27.711 INFO:tasks.workunit.client.1.smithi186.stdout:0/290: truncate d0/d10/d32/f3d 640795 0 2024-04-12T20:56:27.711 INFO:tasks.workunit.client.0.smithi018.stdout:4/988: dread d3/d5d/d60/dc5/d43/d4b/f109 [0,4194304] 0 2024-04-12T20:56:27.714 INFO:tasks.workunit.client.0.smithi018.stdout:1/853: link d3/d65/l10f d3/d9/da/dd/d31/d32/d13/d19/l11f 0 2024-04-12T20:56:27.714 INFO:tasks.workunit.client.1.smithi186.stdout:3/229: stat d1/fc 0 2024-04-12T20:56:27.714 INFO:tasks.workunit.client.0.smithi018.stdout:2/946: creat d5/d8/dec/f145 x:0 0 0 2024-04-12T20:56:27.716 INFO:tasks.workunit.client.1.smithi186.stdout:0/291: fdatasync d0/d14/d4e/d1e/f2f 0 2024-04-12T20:56:27.718 INFO:tasks.workunit.client.0.smithi018.stdout:2/947: chown d5/d8/d10/d43/de2/daf/d88/f10b 5110519 1 2024-04-12T20:56:27.718 INFO:tasks.workunit.client.1.smithi186.stdout:6/199: dread d9/f22 [0,4194304] 0 2024-04-12T20:56:27.718 INFO:tasks.workunit.client.1.smithi186.stdout:4/300: chown d0/dc/c60 223387 1 2024-04-12T20:56:27.720 INFO:tasks.workunit.client.1.smithi186.stdout:8/179: unlink d7/d8/d20/d23/d25/d2e/c32 0 2024-04-12T20:56:27.721 INFO:tasks.workunit.client.1.smithi186.stdout:3/230: mknod d1/d3/c45 0 2024-04-12T20:56:27.721 INFO:tasks.workunit.client.1.smithi186.stdout:0/292: chown d0/d10/d32/c34 429476 1 2024-04-12T20:56:27.721 INFO:tasks.workunit.client.0.smithi018.stdout:1/854: rename d3/d9/da/dd/d31/d32/d13/d19/d2d/d4e/db9/fdc to d3/d65/f120 0 2024-04-12T20:56:27.723 INFO:tasks.workunit.client.0.smithi018.stdout:2/948: read - d5/d8/d10/d43/de2/d24/d39/fa1 zero size 2024-04-12T20:56:27.724 INFO:tasks.workunit.client.1.smithi186.stdout:0/293: read d0/d14/d4e/f28 [125444,15728] 0 2024-04-12T20:56:27.724 INFO:tasks.workunit.client.1.smithi186.stdout:3/231: symlink d1/d2c/l46 0 2024-04-12T20:56:27.725 INFO:tasks.workunit.client.0.smithi018.stdout:4/989: link d3/d5d/d60/dc5/d43/d4b/ddc/c125 d3/d5d/d60/dc5/d3d/da3/d12d/c14f 0 2024-04-12T20:56:27.727 INFO:tasks.workunit.client.1.smithi186.stdout:4/301: link d0/dc/d33/c28 d0/d59/c62 0 2024-04-12T20:56:27.728 INFO:tasks.workunit.client.1.smithi186.stdout:0/294: mknod d0/d14/d4e/d46/c61 0 2024-04-12T20:56:27.728 INFO:tasks.workunit.client.0.smithi018.stdout:2/949: unlink d5/c4c 0 2024-04-12T20:56:27.729 INFO:tasks.workunit.client.1.smithi186.stdout:8/180: unlink d7/de/d10/l33 0 2024-04-12T20:56:27.731 INFO:tasks.workunit.client.0.smithi018.stdout:2/950: write d5/d7/d31/f113 [601368,57339] 0 2024-04-12T20:56:27.734 INFO:tasks.workunit.client.1.smithi186.stdout:4/302: dread d0/d26/f2b [0,4194304] 0 2024-04-12T20:56:27.735 INFO:tasks.workunit.client.0.smithi018.stdout:2/951: getdents d5/d12/d54/dfa 0 2024-04-12T20:56:27.737 INFO:tasks.workunit.client.1.smithi186.stdout:8/181: dread - d7/d8/d20/d23/d25/d29/f2a zero size 2024-04-12T20:56:27.738 INFO:tasks.workunit.client.1.smithi186.stdout:9/220: truncate d3/d2f/d2e/f2a 870289 0 2024-04-12T20:56:27.738 INFO:tasks.workunit.client.0.smithi018.stdout:2/952: mknod d5/d8/d10/d43/de2/d24/d39/da4/c146 0 2024-04-12T20:56:27.739 INFO:tasks.workunit.client.1.smithi186.stdout:0/295: fsync d0/d14/d4e/d1e/f1f 0 2024-04-12T20:56:27.740 INFO:tasks.workunit.client.1.smithi186.stdout:3/232: getdents d1/d3/d21/d22/d24 0 2024-04-12T20:56:27.743 INFO:tasks.workunit.client.0.smithi018.stdout:4/990: dwrite d3/d5d/d60/dc5/d53/de7/d126/fea [0,4194304] 0 2024-04-12T20:56:27.744 INFO:tasks.workunit.client.1.smithi186.stdout:8/182: readlink d7/d8/d20/d23/d25/l2d 0 2024-04-12T20:56:27.744 INFO:tasks.workunit.client.1.smithi186.stdout:5/194: truncate d5/db/d14/f16 30635 0 2024-04-12T20:56:27.745 INFO:tasks.workunit.client.1.smithi186.stdout:8/183: fdatasync d7/de/d10/f27 0 2024-04-12T20:56:27.747 INFO:tasks.workunit.client.0.smithi018.stdout:4/991: rename d3/d5d to d3/d5d/d60/dc5/dcc/d150 22 2024-04-12T20:56:27.751 INFO:tasks.workunit.client.1.smithi186.stdout:0/296: readlink d0/l2e 0 2024-04-12T20:56:27.751 INFO:tasks.workunit.client.1.smithi186.stdout:7/276: dwrite d7/f8 [0,4194304] 0 2024-04-12T20:56:27.752 INFO:tasks.workunit.client.0.smithi018.stdout:4/992: unlink d3/db/lc8 0 2024-04-12T20:56:27.753 INFO:tasks.workunit.client.1.smithi186.stdout:2/244: truncate d0/dc/f40 1101257 0 2024-04-12T20:56:27.754 INFO:tasks.workunit.client.0.smithi018.stdout:4/993: readlink d3/d32/l135 0 2024-04-12T20:56:27.755 INFO:tasks.workunit.client.1.smithi186.stdout:3/233: creat d1/d3/da/f47 x:0 0 0 2024-04-12T20:56:27.757 INFO:tasks.workunit.client.1.smithi186.stdout:5/195: creat d5/db/d15/d18/f3d x:0 0 0 2024-04-12T20:56:27.765 INFO:tasks.workunit.client.0.smithi018.stdout:4/994: unlink d3/d5d/d60/dc5/d53/fcf 0 2024-04-12T20:56:27.766 INFO:tasks.workunit.client.0.smithi018.stdout:2/953: dwrite d5/d8/d10/d43/de2/daf/d90/f9d [4194304,4194304] 0 2024-04-12T20:56:27.768 INFO:tasks.workunit.client.1.smithi186.stdout:3/234: mkdir d1/d2c/d48 0 2024-04-12T20:56:27.768 INFO:tasks.workunit.client.1.smithi186.stdout:2/245: unlink d0/d3/f8 0 2024-04-12T20:56:27.770 INFO:tasks.workunit.client.1.smithi186.stdout:0/297: dread d0/d10/d32/f3d [0,4194304] 0 2024-04-12T20:56:27.770 INFO:tasks.workunit.client.1.smithi186.stdout:6/200: dread d9/f37 [0,4194304] 0 2024-04-12T20:56:27.775 INFO:tasks.workunit.client.1.smithi186.stdout:0/298: readlink d0/l4c 0 2024-04-12T20:56:27.780 INFO:tasks.workunit.client.1.smithi186.stdout:9/221: write d3/d2f/f21 [4859056,68241] 0 2024-04-12T20:56:27.781 INFO:tasks.workunit.client.0.smithi018.stdout:4/995: link d3/db/de3/c13e d3/d5d/d60/dc5/d3d/c151 0 2024-04-12T20:56:27.783 INFO:tasks.workunit.client.0.smithi018.stdout:4/996: chown d3/d32/d33/db6/ffd 16243368 1 2024-04-12T20:56:27.788 INFO:tasks.workunit.client.0.smithi018.stdout:1/855: dwrite d3/d9/d11/f70 [0,4194304] 0 2024-04-12T20:56:27.789 INFO:tasks.workunit.client.1.smithi186.stdout:3/235: mkdir d1/d3/d49 0 2024-04-12T20:56:27.789 INFO:tasks.workunit.client.0.smithi018.stdout:2/954: write d5/d8/d10/d43/f87 [325334,231] 0 2024-04-12T20:56:27.791 INFO:tasks.workunit.client.0.smithi018.stdout:4/997: mknod d3/d5d/d60/dc5/d43/d4b/c152 0 2024-04-12T20:56:27.796 INFO:tasks.workunit.client.1.smithi186.stdout:8/184: dwrite d7/d18/f1a [0,4194304] 0 2024-04-12T20:56:27.796 INFO:tasks.workunit.client.0.smithi018.stdout:4/998: fsync d3/d5d/d60/dc5/d53/de7/d126/d4f/f139 0 2024-04-12T20:56:27.796 INFO:tasks.workunit.client.0.smithi018.stdout:2/955: write d5/d7/d31/d107/d127/f13f [931436,74737] 0 2024-04-12T20:56:27.796 INFO:tasks.workunit.client.0.smithi018.stdout:1/856: creat d3/d9/da/dd/d31/d32/d13/d19/d2d/d4e/d58/f121 x:0 0 0 2024-04-12T20:56:27.801 INFO:tasks.workunit.client.1.smithi186.stdout:6/201: dread d9/f12 [0,4194304] 0 2024-04-12T20:56:27.802 INFO:tasks.workunit.client.0.smithi018.stdout:4/999: mknod d3/d5d/d60/dc5/d53/de7/d126/c153 0 2024-04-12T20:56:27.802 INFO:tasks.workunit.client.0.smithi018.stdout:1/857: creat d3/d9/da/dd/d31/d32/d13/d19/f122 x:0 0 0 2024-04-12T20:56:27.804 INFO:tasks.workunit.client.0.smithi018.stdout:2/956: rename d5/d8/d10/c3b to d5/d7/d31/d11a/d126/c147 0 2024-04-12T20:56:27.804 INFO:tasks.workunit.client.1.smithi186.stdout:9/222: mknod d3/c47 0 2024-04-12T20:56:27.804 INFO:tasks.workunit.client.1.smithi186.stdout:4/303: dwrite d0/dc/d33/f46 [0,4194304] 0 2024-04-12T20:56:27.807 INFO:tasks.workunit.client.1.smithi186.stdout:7/277: dwrite d7/fc [0,4194304] 0 2024-04-12T20:56:27.808 INFO:tasks.workunit.client.1.smithi186.stdout:6/202: fsync d9/f37 0 2024-04-12T20:56:27.808 INFO:tasks.workunit.client.1.smithi186.stdout:0/299: write d0/d10/d32/f3d [988645,21391] 0 2024-04-12T20:56:27.817 INFO:tasks.workunit.client.1.smithi186.stdout:5/196: dwrite d5/fd [0,4194304] 0 2024-04-12T20:56:27.823 INFO:tasks.workunit.client.0.smithi018.stdout:1/858: dwrite d3/d9/da/dd/d31/d32/d13/d19/d2d/d4e/d58/d61/d71/fdf [0,4194304] 0 2024-04-12T20:56:27.826 INFO:tasks.workunit.client.1.smithi186.stdout:8/185: truncate d7/de/d10/d2f/f31 360728 0 2024-04-12T20:56:27.829 INFO:tasks.workunit.client.1.smithi186.stdout:7/278: stat d7/d10/d2d/d40/d4a/f64 0 2024-04-12T20:56:27.834 INFO:tasks.workunit.client.1.smithi186.stdout:9/223: mkdir d3/d2f/d2e/d33/d36/d48 0 2024-04-12T20:56:27.834 INFO:tasks.workunit.client.1.smithi186.stdout:7/279: write d7/fc [3748556,115115] 0 2024-04-12T20:56:27.838 INFO:tasks.workunit.client.1.smithi186.stdout:5/197: dread - d5/db/d15/f28 zero size 2024-04-12T20:56:27.843 INFO:tasks.workunit.client.1.smithi186.stdout:0/300: rename d0/l4a to d0/d10/d32/l62 0 2024-04-12T20:56:27.845 INFO:tasks.workunit.client.1.smithi186.stdout:2/246: dread d0/f2 [0,4194304] 0 2024-04-12T20:56:27.847 INFO:tasks.workunit.client.1.smithi186.stdout:9/224: unlink d3/d2f/f45 0 2024-04-12T20:56:27.848 INFO:tasks.workunit.client.1.smithi186.stdout:0/301: stat d0/l2e 0 2024-04-12T20:56:27.849 INFO:tasks.workunit.client.1.smithi186.stdout:7/280: chown d7/d10/d1c/f20 0 1 2024-04-12T20:56:27.849 INFO:tasks.workunit.client.1.smithi186.stdout:3/236: dwrite d1/d3/f11 [0,4194304] 0 2024-04-12T20:56:27.849 INFO:tasks.workunit.client.1.smithi186.stdout:5/198: mknod d5/c3e 0 2024-04-12T20:56:27.851 INFO:tasks.workunit.client.1.smithi186.stdout:9/225: chown d3/d2f 675031 1 2024-04-12T20:56:27.858 INFO:tasks.workunit.client.1.smithi186.stdout:8/186: rename d7/d8 to d7/d18/d36 0 2024-04-12T20:56:27.859 INFO:tasks.workunit.client.1.smithi186.stdout:7/281: chown d7/c61 10 1 2024-04-12T20:56:27.863 INFO:tasks.workunit.client.1.smithi186.stdout:9/226: creat d3/d2f/d2e/d33/f49 x:0 0 0 2024-04-12T20:56:27.864 INFO:tasks.workunit.client.1.smithi186.stdout:4/304: dwrite d0/d21/f41 [0,4194304] 0 2024-04-12T20:56:27.864 INFO:tasks.workunit.client.1.smithi186.stdout:2/247: rename d0/f14 to d0/d3/d1a/d21/f4b 0 2024-04-12T20:56:27.866 INFO:tasks.workunit.client.1.smithi186.stdout:9/227: chown d3/f10 289 1 2024-04-12T20:56:27.871 INFO:tasks.workunit.client.1.smithi186.stdout:0/302: getdents d0/d5d 0 2024-04-12T20:56:27.871 INFO:tasks.workunit.client.1.smithi186.stdout:1/229: sync 2024-04-12T20:56:27.876 INFO:tasks.workunit.client.0.smithi018.stdout:2/957: dread d5/d7/d79/f8c [0,4194304] 0 2024-04-12T20:56:27.877 INFO:tasks.workunit.client.1.smithi186.stdout:3/237: mkdir d1/d3/d21/d22/d4a 0 2024-04-12T20:56:27.881 INFO:tasks.workunit.client.1.smithi186.stdout:2/248: rename d0/d3/d1a/f3a to d0/dc/f4c 0 2024-04-12T20:56:27.881 INFO:tasks.workunit.client.1.smithi186.stdout:4/305: mkdir d0/d21/d3d/d63 0 2024-04-12T20:56:27.881 INFO:tasks.workunit.client.1.smithi186.stdout:3/238: write d1/d3/da/f47 [942154,42105] 0 2024-04-12T20:56:27.888 INFO:tasks.workunit.client.1.smithi186.stdout:0/303: read d0/d14/d4e/d1e/f1f [1633014,78971] 0 2024-04-12T20:56:27.890 INFO:tasks.workunit.client.1.smithi186.stdout:5/199: link d5/db/d15/d18/c24 d5/db/d15/d18/c3f 0 2024-04-12T20:56:27.894 INFO:tasks.workunit.client.1.smithi186.stdout:6/203: dread d9/f1b [0,4194304] 0 2024-04-12T20:56:27.896 INFO:tasks.workunit.client.0.smithi018.stdout:1/859: dread d3/d9/da/dd/d31/d32/d33/f6d [4194304,4194304] 0 2024-04-12T20:56:27.896 INFO:tasks.workunit.client.1.smithi186.stdout:8/187: dwrite f2 [0,4194304] 0 2024-04-12T20:56:27.903 INFO:tasks.workunit.client.1.smithi186.stdout:0/304: mknod d0/d5d/c63 0 2024-04-12T20:56:27.909 INFO:tasks.workunit.client.1.smithi186.stdout:3/239: dread d1/d3/d21/f41 [0,4194304] 0 2024-04-12T20:56:27.909 INFO:tasks.workunit.client.1.smithi186.stdout:2/249: link d0/l23 d0/d3/d1a/d27/l4d 0 2024-04-12T20:56:27.910 INFO:tasks.workunit.client.1.smithi186.stdout:7/282: dread d7/d10/d2d/f44 [0,4194304] 0 2024-04-12T20:56:27.912 INFO:tasks.workunit.client.1.smithi186.stdout:2/250: fsync d0/dc/f29 0 2024-04-12T20:56:27.914 INFO:tasks.workunit.client.1.smithi186.stdout:7/283: write d7/d10/d1c/f20 [2593983,128085] 0 2024-04-12T20:56:27.925 INFO:tasks.workunit.client.1.smithi186.stdout:6/204: dwrite d9/f47 [0,4194304] 0 2024-04-12T20:56:27.936 INFO:tasks.workunit.client.1.smithi186.stdout:4/306: dread d0/dc/f2d [0,4194304] 0 2024-04-12T20:56:27.946 INFO:tasks.workunit.client.1.smithi186.stdout:9/228: dread d3/d2f/f21 [0,4194304] 0 2024-04-12T20:56:27.951 INFO:tasks.workunit.client.1.smithi186.stdout:7/284: dread d7/d10/d1c/f52 [0,4194304] 0 2024-04-12T20:56:27.951 INFO:tasks.workunit.client.1.smithi186.stdout:0/305: dread d0/d10/f2c [0,4194304] 0 2024-04-12T20:56:27.955 INFO:tasks.workunit.client.1.smithi186.stdout:4/307: dread d0/dc/d39/f5b [0,4194304] 0 2024-04-12T20:56:27.956 INFO:tasks.workunit.client.1.smithi186.stdout:4/308: chown d0/dc/f29 7504921 1 2024-04-12T20:56:27.957 INFO:tasks.workunit.client.1.smithi186.stdout:4/309: fsync d0/f10 0 2024-04-12T20:56:27.973 INFO:tasks.workunit.client.1.smithi186.stdout:4/310: dwrite d0/dc/d33/f43 [0,4194304] 0 2024-04-12T20:56:28.210 INFO:tasks.workunit.client.1.smithi186.stdout:1/230: write d14/f15 [4186028,122828] 0 2024-04-12T20:56:28.210 INFO:tasks.workunit.client.1.smithi186.stdout:5/200: creat d5/db/f40 x:0 0 0 2024-04-12T20:56:28.213 INFO:tasks.workunit.client.1.smithi186.stdout:3/240: write d1/d3/d21/f41 [2877202,27933] 0 2024-04-12T20:56:28.227 INFO:tasks.workunit.client.1.smithi186.stdout:9/229: creat d3/d2f/f4a x:0 0 0 2024-04-12T20:56:28.230 INFO:tasks.workunit.client.1.smithi186.stdout:0/306: creat d0/d10/d32/d3f/f64 x:0 0 0 2024-04-12T20:56:28.233 INFO:tasks.workunit.client.1.smithi186.stdout:9/230: write d3/d2f/f1e [18031,124202] 0 2024-04-12T20:56:28.242 INFO:tasks.workunit.client.1.smithi186.stdout:8/188: creat d7/d18/d36/f37 x:0 0 0 2024-04-12T20:56:28.248 INFO:tasks.workunit.client.1.smithi186.stdout:1/231: rename d14/d1a/d33/f34 to d14/d1a/d3c/f47 0 2024-04-12T20:56:28.251 INFO:tasks.workunit.client.1.smithi186.stdout:3/241: mknod d1/d3/da/c4b 0 2024-04-12T20:56:28.254 INFO:tasks.workunit.client.1.smithi186.stdout:0/307: dread d0/d14/d4e/d1e/f56 [0,4194304] 0 2024-04-12T20:56:28.254 INFO:tasks.workunit.client.1.smithi186.stdout:7/285: mkdir d7/d10/d68 0 2024-04-12T20:56:28.262 INFO:tasks.workunit.client.1.smithi186.stdout:4/311: unlink d0/d21/c53 0 2024-04-12T20:56:28.264 INFO:tasks.workunit.client.1.smithi186.stdout:8/189: symlink d7/de/d10/d2f/l38 0 2024-04-12T20:56:28.268 INFO:tasks.workunit.client.1.smithi186.stdout:3/242: fdatasync d1/d3/d21/d22/d24/d28/d3d/f40 0 2024-04-12T20:56:28.268 INFO:tasks.workunit.client.1.smithi186.stdout:2/251: dwrite d0/d3/d1a/d24/f34 [0,4194304] 0 2024-04-12T20:56:28.268 INFO:tasks.workunit.client.1.smithi186.stdout:1/232: mknod d14/d1a/d21/d2c/d3e/c48 0 2024-04-12T20:56:28.273 INFO:tasks.workunit.client.1.smithi186.stdout:6/205: getdents d9/d43 0 2024-04-12T20:56:28.276 INFO:tasks.workunit.client.1.smithi186.stdout:7/286: mkdir d7/d10/d1c/d5d/d69 0 2024-04-12T20:56:28.277 INFO:tasks.workunit.client.1.smithi186.stdout:2/252: dread - d0/d3/d1a/d24/f33 zero size 2024-04-12T20:56:28.279 INFO:tasks.workunit.client.1.smithi186.stdout:5/201: rename d5/db/d15/f26 to d5/db/d15/d2a/f41 0 2024-04-12T20:56:28.282 INFO:tasks.workunit.client.1.smithi186.stdout:9/231: getdents d3/d2f/d2e/d33/d36 0 2024-04-12T20:56:28.288 INFO:tasks.workunit.client.1.smithi186.stdout:4/312: mkdir d0/dc/d64 0 2024-04-12T20:56:28.291 INFO:tasks.workunit.client.1.smithi186.stdout:1/233: symlink d14/d1a/d21/l49 0 2024-04-12T20:56:28.292 INFO:tasks.workunit.client.1.smithi186.stdout:4/313: fdatasync d0/dc/d33/f56 0 2024-04-12T20:56:28.298 INFO:tasks.workunit.client.1.smithi186.stdout:8/190: rename l3 to d7/d18/d36/d20/d23/d25/d2e/l39 0 2024-04-12T20:56:28.304 INFO:tasks.workunit.client.1.smithi186.stdout:4/314: chown d0/d21/d3d/d48/f51 231 1 2024-04-12T20:56:28.311 INFO:tasks.workunit.client.1.smithi186.stdout:0/308: dwrite d0/d14/f19 [0,4194304] 0 2024-04-12T20:56:28.311 INFO:tasks.workunit.client.1.smithi186.stdout:8/191: write d7/d18/d36/d20/d23/d25/d29/f2a [885867,68949] 0 2024-04-12T20:56:28.315 INFO:tasks.workunit.client.1.smithi186.stdout:7/287: rename d7/d10/d68 to d7/d55/d5f/d4d/d6a 0 2024-04-12T20:56:28.315 INFO:tasks.workunit.client.1.smithi186.stdout:9/232: dwrite d3/f3e [0,4194304] 0 2024-04-12T20:56:28.316 INFO:tasks.workunit.client.1.smithi186.stdout:6/206: dwrite d9/d3f/f46 [0,4194304] 0 2024-04-12T20:56:28.318 INFO:tasks.workunit.client.1.smithi186.stdout:0/309: stat d0/d10 0 2024-04-12T20:56:28.321 INFO:tasks.workunit.client.1.smithi186.stdout:7/288: dread - d7/d55/d5f/d4d/f4e zero size 2024-04-12T20:56:28.322 INFO:tasks.workunit.client.1.smithi186.stdout:4/315: dread d0/dc/d39/f3a [0,4194304] 0 2024-04-12T20:56:28.325 INFO:tasks.workunit.client.1.smithi186.stdout:4/316: dread - d0/f49 zero size 2024-04-12T20:56:28.325 INFO:tasks.workunit.client.1.smithi186.stdout:9/233: write d3/d12/d27/f35 [618270,69311] 0 2024-04-12T20:56:28.326 INFO:tasks.workunit.client.1.smithi186.stdout:1/234: dwrite d14/d1a/d1d/f23 [0,4194304] 0 2024-04-12T20:56:28.329 INFO:tasks.workunit.client.1.smithi186.stdout:3/243: creat d1/d3/d21/d22/f4c x:0 0 0 2024-04-12T20:56:28.330 INFO:tasks.workunit.client.1.smithi186.stdout:6/207: symlink d9/d3f/l49 0 2024-04-12T20:56:28.332 INFO:tasks.workunit.client.1.smithi186.stdout:3/244: write d1/d3/d21/d22/f4c [930011,24290] 0 2024-04-12T20:56:28.334 INFO:tasks.workunit.client.1.smithi186.stdout:3/245: readlink d1/d2c/l46 0 2024-04-12T20:56:28.336 INFO:tasks.workunit.client.1.smithi186.stdout:2/253: truncate d0/d3/d1a/f45 2089012 0 2024-04-12T20:56:28.342 INFO:tasks.workunit.client.1.smithi186.stdout:0/310: unlink d0/d14/d4e/f28 0 2024-04-12T20:56:28.343 INFO:tasks.workunit.client.1.smithi186.stdout:9/234: dwrite d3/d2f/f1e [0,4194304] 0 2024-04-12T20:56:28.344 INFO:tasks.workunit.client.1.smithi186.stdout:0/311: dread - d0/d14/f24 zero size 2024-04-12T20:56:28.348 INFO:tasks.workunit.client.1.smithi186.stdout:1/235: mknod d14/d1a/d21/d2c/c4a 0 2024-04-12T20:56:28.349 INFO:tasks.workunit.client.1.smithi186.stdout:6/208: fsync d9/f1b 0 2024-04-12T20:56:28.353 INFO:tasks.workunit.client.1.smithi186.stdout:5/202: creat d5/db/d15/d2a/d2d/f42 x:0 0 0 2024-04-12T20:56:28.354 INFO:tasks.workunit.client.0.smithi018.stdout:1/860: sync 2024-04-12T20:56:28.355 INFO:tasks.workunit.client.0.smithi018.stdout:2/958: sync 2024-04-12T20:56:28.355 INFO:tasks.workunit.client.1.smithi186.stdout:7/289: symlink d7/d10/d1c/l6b 0 2024-04-12T20:56:28.356 INFO:tasks.workunit.client.1.smithi186.stdout:5/203: dread d5/db/d14/f1a [0,4194304] 0 2024-04-12T20:56:28.362 INFO:tasks.workunit.client.1.smithi186.stdout:8/192: link d7/d18/d36/f1b d7/d18/d36/d20/d23/d25/d2e/f3a 0 2024-04-12T20:56:28.363 INFO:tasks.workunit.client.1.smithi186.stdout:9/235: creat d3/d2f/d2e/d33/f4b x:0 0 0 2024-04-12T20:56:28.365 INFO:tasks.workunit.client.1.smithi186.stdout:1/236: rename d14/d1a/d21/d2c/f38 to d14/d1a/d21/f4b 0 2024-04-12T20:56:28.368 INFO:tasks.workunit.client.1.smithi186.stdout:6/209: dwrite d9/f1b [0,4194304] 0 2024-04-12T20:56:28.369 INFO:tasks.workunit.client.1.smithi186.stdout:7/290: mkdir d7/d10/d2d/d40/d4a/d6c 0 2024-04-12T20:56:28.372 INFO:tasks.workunit.client.0.smithi018.stdout:1/861: creat d3/d9/da/dd/d26/df0/d66/f123 x:0 0 0 2024-04-12T20:56:28.374 INFO:tasks.workunit.client.1.smithi186.stdout:7/291: chown d7/d10/d2d/d40/d4a/d6c 1641446 1 2024-04-12T20:56:28.374 INFO:tasks.workunit.client.1.smithi186.stdout:0/312: dread d0/d10/f43 [0,4194304] 0 2024-04-12T20:56:28.376 INFO:tasks.workunit.client.1.smithi186.stdout:8/193: unlink d7/d18/d36/d20/d23/d25/d2e/f3a 0 2024-04-12T20:56:28.376 INFO:tasks.workunit.client.1.smithi186.stdout:0/313: readlink d0/l2e 0 2024-04-12T20:56:28.378 INFO:tasks.workunit.client.1.smithi186.stdout:9/236: mkdir d3/d2f/d2e/d43/d4c 0 2024-04-12T20:56:28.381 INFO:tasks.workunit.client.1.smithi186.stdout:9/237: write d3/d12/f30 [309253,52460] 0 2024-04-12T20:56:28.387 INFO:tasks.workunit.client.1.smithi186.stdout:6/210: mknod d9/d43/d2d/c4a 0 2024-04-12T20:56:28.388 INFO:tasks.workunit.client.1.smithi186.stdout:1/237: rename l3 to d14/d1a/d33/d36/l4c 0 2024-04-12T20:56:28.388 INFO:tasks.workunit.client.1.smithi186.stdout:6/211: dread - d9/f40 zero size 2024-04-12T20:56:28.389 INFO:tasks.workunit.client.1.smithi186.stdout:1/238: chown d14/d1a/d21/d2c 2 1 2024-04-12T20:56:28.393 INFO:tasks.workunit.client.1.smithi186.stdout:7/292: dwrite d7/d10/d2d/d40/d4a/f4c [0,4194304] 0 2024-04-12T20:56:28.394 INFO:tasks.workunit.client.1.smithi186.stdout:8/194: dread f2 [0,4194304] 0 2024-04-12T20:56:28.397 INFO:tasks.workunit.client.0.smithi018.stdout:2/959: write d5/d8/d10/f48 [1795977,36458] 0 2024-04-12T20:56:28.400 INFO:tasks.workunit.client.1.smithi186.stdout:5/204: rename d5/db/d15/d18/c38 to d5/db/d15/d18/c43 0 2024-04-12T20:56:28.401 INFO:tasks.workunit.client.1.smithi186.stdout:4/317: truncate d0/dc/f2d 6194024 0 2024-04-12T20:56:28.401 INFO:tasks.workunit.client.0.smithi018.stdout:1/862: dwrite d3/d9/da/dd/d26/fbc [4194304,4194304] 0 2024-04-12T20:56:28.404 INFO:tasks.workunit.client.0.smithi018.stdout:1/863: fsync d3/d9/da/dd/d31/d32/f41 0 2024-04-12T20:56:28.404 INFO:tasks.workunit.client.0.smithi018.stdout:2/960: creat d5/d8/d10/d43/de2/f148 x:0 0 0 2024-04-12T20:56:28.405 INFO:tasks.workunit.client.1.smithi186.stdout:3/246: truncate d1/f19 7996085 0 2024-04-12T20:56:28.405 INFO:tasks.workunit.client.1.smithi186.stdout:4/318: stat d0/dc/d33/f47 0 2024-04-12T20:56:28.408 INFO:tasks.workunit.client.1.smithi186.stdout:1/239: write d14/d1a/d3c/f47 [1751929,50805] 0 2024-04-12T20:56:28.408 INFO:tasks.workunit.client.1.smithi186.stdout:7/293: chown d7/c1d 116 1 2024-04-12T20:56:28.408 INFO:tasks.workunit.client.0.smithi018.stdout:2/961: symlink d5/d7/d31/d62/da6/l149 0 2024-04-12T20:56:28.408 INFO:tasks.workunit.client.1.smithi186.stdout:3/247: chown d1/l18 46708992 1 2024-04-12T20:56:28.409 INFO:tasks.workunit.client.0.smithi018.stdout:1/864: dread - d3/d9/da/dd/d31/d32/d13/d19/fcf zero size 2024-04-12T20:56:28.418 INFO:tasks.workunit.client.1.smithi186.stdout:8/195: rename d7/f12 to d7/d18/d36/d20/d23/d25/d2e/f3b 0 2024-04-12T20:56:28.420 INFO:tasks.workunit.client.1.smithi186.stdout:5/205: fdatasync d5/db/d15/f2f 0 2024-04-12T20:56:28.424 INFO:tasks.workunit.client.1.smithi186.stdout:9/238: link d3/d2f/d2e/c40 d3/c4d 0 2024-04-12T20:56:28.424 INFO:tasks.workunit.client.0.smithi018.stdout:1/865: link d3/d9/da/dd/d31/d32/d13/d19/d2d/d4e/d7a/d82/fb5 d3/d9/da/dd/d31/d32/d13/d19/d2d/d4e/d7a/f124 0 2024-04-12T20:56:28.428 INFO:tasks.workunit.client.1.smithi186.stdout:7/294: creat d7/d55/d5f/d4d/f6d x:0 0 0 2024-04-12T20:56:28.429 INFO:tasks.workunit.client.0.smithi018.stdout:1/866: rmdir d3/d9/da/dd/d31/d44 39 2024-04-12T20:56:28.431 INFO:tasks.workunit.client.1.smithi186.stdout:1/240: creat d14/d1a/d21/d2c/f4d x:0 0 0 2024-04-12T20:56:28.433 INFO:tasks.workunit.client.1.smithi186.stdout:0/314: dwrite d0/d10/f2c [0,4194304] 0 2024-04-12T20:56:28.434 INFO:tasks.workunit.client.0.smithi018.stdout:1/867: unlink d3/d9/da/dd/d31/d32/f41 0 2024-04-12T20:56:28.435 INFO:tasks.workunit.client.1.smithi186.stdout:8/196: rmdir d7/d18/d36/d20/d23 39 2024-04-12T20:56:28.436 INFO:tasks.workunit.client.1.smithi186.stdout:2/254: dwrite d0/d3/f1f [0,4194304] 0 2024-04-12T20:56:28.438 INFO:tasks.workunit.client.1.smithi186.stdout:5/206: mkdir d5/d44 0 2024-04-12T20:56:28.438 INFO:tasks.workunit.client.0.smithi018.stdout:1/868: creat d3/d9/da/dd/d31/d32/d33/d80/f125 x:0 0 0 2024-04-12T20:56:28.439 INFO:tasks.workunit.client.1.smithi186.stdout:1/241: write d14/f22 [1165276,107654] 0 2024-04-12T20:56:28.440 INFO:tasks.workunit.client.1.smithi186.stdout:0/315: read d0/f6 [3406110,101816] 0 2024-04-12T20:56:28.441 INFO:tasks.workunit.client.0.smithi018.stdout:1/869: creat d3/d9/da/dd/f126 x:0 0 0 2024-04-12T20:56:28.445 INFO:tasks.workunit.client.1.smithi186.stdout:5/207: dread - d5/db/d15/d18/f3d zero size 2024-04-12T20:56:28.445 INFO:tasks.workunit.client.0.smithi018.stdout:1/870: mknod d3/d9/da/dd/d31/d32/d13/c127 0 2024-04-12T20:56:28.450 INFO:tasks.workunit.client.1.smithi186.stdout:7/295: truncate d7/d10/d1c/f52 334071 0 2024-04-12T20:56:28.451 INFO:tasks.workunit.client.1.smithi186.stdout:2/255: read d0/d32/d2a/f43 [147315,77191] 0 2024-04-12T20:56:28.451 INFO:tasks.workunit.client.0.smithi018.stdout:1/871: getdents d3/d9/da/dd/d31/d32/d13/d19/d2d/d4e/d5e/d97 0 2024-04-12T20:56:28.451 INFO:tasks.workunit.client.1.smithi186.stdout:9/239: chown d3/c4d 411864 1 2024-04-12T20:56:28.456 INFO:tasks.workunit.client.1.smithi186.stdout:3/248: dread d1/d3/d21/d22/f4c [0,4194304] 0 2024-04-12T20:56:28.456 INFO:tasks.workunit.client.1.smithi186.stdout:8/197: creat d7/de/d10/f3c x:0 0 0 2024-04-12T20:56:28.466 INFO:tasks.workunit.client.1.smithi186.stdout:7/296: truncate d7/d10/d2d/d40/d4a/f50 347221 0 2024-04-12T20:56:28.469 INFO:tasks.workunit.client.1.smithi186.stdout:4/319: dwrite d0/dc/d33/fb [0,4194304] 0 2024-04-12T20:56:28.470 INFO:tasks.workunit.client.1.smithi186.stdout:0/316: dwrite d0/d14/f1a [0,4194304] 0 2024-04-12T20:56:28.474 INFO:tasks.workunit.client.1.smithi186.stdout:4/320: dread - d0/dc/d33/f57 zero size 2024-04-12T20:56:28.476 INFO:tasks.workunit.client.1.smithi186.stdout:0/317: write d0/d5c/f5f [610408,43273] 0 2024-04-12T20:56:28.478 INFO:tasks.workunit.client.1.smithi186.stdout:7/297: mknod d7/d10/d2d/d40/d4a/d5c/c6e 0 2024-04-12T20:56:28.483 INFO:tasks.workunit.client.1.smithi186.stdout:0/318: chown d0/d14/d4e/d46/f5a 13199789 1 2024-04-12T20:56:28.486 INFO:tasks.workunit.client.1.smithi186.stdout:2/256: dwrite d0/dc/f4c [0,4194304] 0 2024-04-12T20:56:28.486 INFO:tasks.workunit.client.1.smithi186.stdout:7/298: creat d7/d55/f6f x:0 0 0 2024-04-12T20:56:28.492 INFO:tasks.workunit.client.1.smithi186.stdout:2/257: chown d0/dc 29505402 1 2024-04-12T20:56:28.493 INFO:tasks.workunit.client.1.smithi186.stdout:6/212: dread d9/f33 [0,4194304] 0 2024-04-12T20:56:28.495 INFO:tasks.workunit.client.1.smithi186.stdout:5/208: dread d5/db/d15/d2a/f41 [0,4194304] 0 2024-04-12T20:56:28.495 INFO:tasks.workunit.client.0.smithi018.stdout:2/962: link d5/d8/d10/d43/l124 d5/d8/d10/d43/de2/d7c/l14a 0 2024-04-12T20:56:28.497 INFO:tasks.workunit.client.1.smithi186.stdout:6/213: chown d9/f22 968877490 1 2024-04-12T20:56:28.499 INFO:tasks.workunit.client.1.smithi186.stdout:7/299: creat d7/d10/d2d/d67/f70 x:0 0 0 2024-04-12T20:56:28.499 INFO:tasks.workunit.client.1.smithi186.stdout:8/198: dwrite d7/d18/d36/d17/f22 [0,4194304] 0 2024-04-12T20:56:28.499 INFO:tasks.workunit.client.1.smithi186.stdout:6/214: rmdir d9 39 2024-04-12T20:56:28.502 INFO:tasks.workunit.client.1.smithi186.stdout:2/258: getdents d0/d3/d1a/d24 0 2024-04-12T20:56:28.508 INFO:tasks.workunit.client.1.smithi186.stdout:7/300: mkdir d7/d55/d71 0 2024-04-12T20:56:28.510 INFO:tasks.workunit.client.1.smithi186.stdout:2/259: rename d0/d3/d1a/d21/d22/l42 to d0/d32/l4e 0 2024-04-12T20:56:28.511 INFO:tasks.workunit.client.1.smithi186.stdout:5/209: getdents d5/d30 0 2024-04-12T20:56:28.512 INFO:tasks.workunit.client.1.smithi186.stdout:7/301: rmdir d7/d55/d5f/d4d 39 2024-04-12T20:56:28.514 INFO:tasks.workunit.client.1.smithi186.stdout:2/260: dread - d0/d3/d1a/d21/d26/f2d zero size 2024-04-12T20:56:28.518 INFO:tasks.workunit.client.1.smithi186.stdout:2/261: rmdir d0/dc/d2e 39 2024-04-12T20:56:28.519 INFO:tasks.workunit.client.1.smithi186.stdout:7/302: symlink d7/d10/d2d/d40/d4a/d5c/l72 0 2024-04-12T20:56:28.520 INFO:tasks.workunit.client.1.smithi186.stdout:8/199: getdents d7/d18/d36/d20/d23/d25/d2e 0 2024-04-12T20:56:28.522 INFO:tasks.workunit.client.1.smithi186.stdout:2/262: chown d0/cb 0 1 2024-04-12T20:56:28.523 INFO:tasks.workunit.client.1.smithi186.stdout:4/321: write d0/dc/d33/f7 [426702,95763] 0 2024-04-12T20:56:28.524 INFO:tasks.workunit.client.1.smithi186.stdout:9/240: write d3/d2f/d2e/f2a [725974,46714] 0 2024-04-12T20:56:28.526 INFO:tasks.workunit.client.1.smithi186.stdout:7/303: write d7/d10/f46 [2061924,124567] 0 2024-04-12T20:56:28.526 INFO:tasks.workunit.client.0.smithi018.stdout:1/872: write d3/d9/da/dd/d31/d32/d13/d19/d2d/d4e/d7a/d82/fb5 [1053014,53872] 0 2024-04-12T20:56:28.527 INFO:tasks.workunit.client.0.smithi018.stdout:2/963: write d5/d8/d10/d43/de2/daf/d90/d142/de5/f59 [3102826,68294] 0 2024-04-12T20:56:28.529 INFO:tasks.workunit.client.1.smithi186.stdout:2/263: chown d0/dc/l49 2 1 2024-04-12T20:56:28.531 INFO:tasks.workunit.client.0.smithi018.stdout:1/873: unlink d3/d9/f11e 0 2024-04-12T20:56:28.532 INFO:tasks.workunit.client.1.smithi186.stdout:4/322: chown d0/dc/d1f/l3e 9 1 2024-04-12T20:56:28.533 INFO:tasks.workunit.client.0.smithi018.stdout:1/874: stat d3/d9/da/dd/d31/d32/d33/f6d 0 2024-04-12T20:56:28.534 INFO:tasks.workunit.client.1.smithi186.stdout:8/200: rmdir d7/d18/d36/d20/d23/d25/d29 39 2024-04-12T20:56:28.537 INFO:tasks.workunit.client.1.smithi186.stdout:9/241: fdatasync d3/d2f/d2e/f2a 0 2024-04-12T20:56:28.540 INFO:tasks.workunit.client.0.smithi018.stdout:1/875: creat d3/d9/da/dd/d31/d32/d13/d19/d2d/d4e/d7a/d82/f128 x:0 0 0 2024-04-12T20:56:28.541 INFO:tasks.workunit.client.1.smithi186.stdout:4/323: dread - d0/dc/d33/f36 zero size 2024-04-12T20:56:28.543 INFO:tasks.workunit.client.1.smithi186.stdout:6/215: dwrite d9/f2e [0,4194304] 0 2024-04-12T20:56:28.543 INFO:tasks.workunit.client.0.smithi018.stdout:1/876: symlink d3/d9/da/dd/d31/d32/d13/d19/d2d/d4e/d7a/l129 0 2024-04-12T20:56:28.548 INFO:tasks.workunit.client.1.smithi186.stdout:9/242: dread - d3/d2f/d2e/d33/f49 zero size 2024-04-12T20:56:28.548 INFO:tasks.workunit.client.0.smithi018.stdout:2/964: dwrite d5/d7/fe0 [0,4194304] 0 2024-04-12T20:56:28.549 INFO:tasks.workunit.client.0.smithi018.stdout:1/877: symlink d3/d9/da/dd/d31/d32/d13/d19/d2d/d4e/d58/d61/d71/l12a 0 2024-04-12T20:56:28.551 INFO:tasks.workunit.client.1.smithi186.stdout:3/249: dread d1/f19 [0,4194304] 0 2024-04-12T20:56:28.551 INFO:tasks.workunit.client.1.smithi186.stdout:5/210: dwrite d5/db/fc [4194304,4194304] 0 2024-04-12T20:56:28.551 INFO:tasks.workunit.client.1.smithi186.stdout:4/324: write d0/dc/d33/f57 [540977,111635] 0 2024-04-12T20:56:28.551 INFO:tasks.workunit.client.1.smithi186.stdout:6/216: creat d9/f4b x:0 0 0 2024-04-12T20:56:28.552 INFO:tasks.workunit.client.1.smithi186.stdout:2/264: link d0/d3/d1a/d21/d22/c28 d0/d3/d1a/d27/c4f 0 2024-04-12T20:56:28.552 INFO:tasks.workunit.client.0.smithi018.stdout:2/965: read d5/d12/fe8 [1036394,90647] 0 2024-04-12T20:56:28.556 INFO:tasks.workunit.client.1.smithi186.stdout:9/243: dread - d3/ff zero size 2024-04-12T20:56:28.557 INFO:tasks.workunit.client.1.smithi186.stdout:5/211: readlink d5/db/lf 0 2024-04-12T20:56:28.560 INFO:tasks.workunit.client.1.smithi186.stdout:3/250: mkdir d1/d3/d49/d4d 0 2024-04-12T20:56:28.563 INFO:tasks.workunit.client.1.smithi186.stdout:9/244: creat d3/d13/f4e x:0 0 0 2024-04-12T20:56:28.565 INFO:tasks.workunit.client.1.smithi186.stdout:4/325: symlink d0/l65 0 2024-04-12T20:56:28.567 INFO:tasks.workunit.client.0.smithi018.stdout:2/966: dread d5/d7/ffb [0,4194304] 0 2024-04-12T20:56:28.568 INFO:tasks.workunit.client.1.smithi186.stdout:4/326: chown d0/d21/c16 49 1 2024-04-12T20:56:28.568 INFO:tasks.workunit.client.1.smithi186.stdout:2/265: rename d0/dc/f29 to d0/f50 0 2024-04-12T20:56:28.570 INFO:tasks.workunit.client.1.smithi186.stdout:8/201: dwrite d7/de/f11 [0,4194304] 0 2024-04-12T20:56:28.572 INFO:tasks.workunit.client.1.smithi186.stdout:4/327: chown d0/dc/d33/f1a 868002 1 2024-04-12T20:56:28.572 INFO:tasks.workunit.client.1.smithi186.stdout:5/212: symlink d5/db/d33/l45 0 2024-04-12T20:56:28.574 INFO:tasks.workunit.client.1.smithi186.stdout:3/251: rename d1/d3 to d1/d3/d21/d22/d4e 22 2024-04-12T20:56:28.574 INFO:tasks.workunit.client.1.smithi186.stdout:5/213: stat d5/la 0 2024-04-12T20:56:28.575 INFO:tasks.workunit.client.0.smithi018.stdout:2/967: creat d5/d8/d25/f14b x:0 0 0 2024-04-12T20:56:28.578 INFO:tasks.workunit.client.0.smithi018.stdout:1/878: dwrite d3/d9/da/dd/d31/d32/d13/d19/d2d/d4e/d7a/df5/ff9 [0,4194304] 0 2024-04-12T20:56:28.579 INFO:tasks.workunit.client.1.smithi186.stdout:5/214: dread - d5/db/f2b zero size 2024-04-12T20:56:28.579 INFO:tasks.workunit.client.1.smithi186.stdout:4/328: write d0/dc/d33/f57 [1681025,79493] 0 2024-04-12T20:56:28.587 INFO:tasks.workunit.client.1.smithi186.stdout:6/217: dwrite d9/d3f/f46 [4194304,4194304] 0 2024-04-12T20:56:28.587 INFO:tasks.workunit.client.1.smithi186.stdout:4/329: chown d0/d26/l5f 8531048 1 2024-04-12T20:56:28.592 INFO:tasks.workunit.client.1.smithi186.stdout:9/245: dwrite d3/ff [0,4194304] 0 2024-04-12T20:56:28.592 INFO:tasks.workunit.client.1.smithi186.stdout:3/252: dread d1/d3/d21/d22/f4c [0,4194304] 0 2024-04-12T20:56:28.593 INFO:tasks.workunit.client.1.smithi186.stdout:4/330: readlink d0/dc/l3c 0 2024-04-12T20:56:28.595 INFO:tasks.workunit.client.1.smithi186.stdout:5/215: mkdir d5/db/d15/d2a/d2d/d46 0 2024-04-12T20:56:28.596 INFO:tasks.workunit.client.0.smithi018.stdout:2/968: rename d5/d8/d10/d43/de2/d24/d91/l138 to d5/d12/d54/l14c 0 2024-04-12T20:56:28.598 INFO:tasks.workunit.client.0.smithi018.stdout:2/969: fdatasync d5/d8/d10/d43/de2/f148 0 2024-04-12T20:56:28.600 INFO:tasks.workunit.client.1.smithi186.stdout:3/253: chown d1/l16 6 1 2024-04-12T20:56:28.601 INFO:tasks.workunit.client.1.smithi186.stdout:4/331: creat d0/dc/d39/f66 x:0 0 0 2024-04-12T20:56:28.601 INFO:tasks.workunit.client.1.smithi186.stdout:8/202: write d7/d18/d36/f1b [8281470,107518] 0 2024-04-12T20:56:28.603 INFO:tasks.workunit.client.1.smithi186.stdout:5/216: symlink d5/db/d2e/l47 0 2024-04-12T20:56:28.604 INFO:tasks.workunit.client.0.smithi018.stdout:1/879: dwrite d3/d9/da/d14/fdb [8388608,4194304] 0 2024-04-12T20:56:28.608 INFO:tasks.workunit.client.0.smithi018.stdout:1/880: mkdir d3/d9/da/d12b 0 2024-04-12T20:56:28.611 INFO:tasks.workunit.client.1.smithi186.stdout:6/218: dread d9/f37 [0,4194304] 0 2024-04-12T20:56:28.613 INFO:tasks.workunit.client.1.smithi186.stdout:3/254: rename d1/f19 to d1/d2c/d48/f4f 0 2024-04-12T20:56:28.614 INFO:tasks.workunit.client.1.smithi186.stdout:4/332: creat d0/d21/d3d/d63/f67 x:0 0 0 2024-04-12T20:56:28.617 INFO:tasks.workunit.client.1.smithi186.stdout:6/219: mkdir d9/d4c 0 2024-04-12T20:56:28.617 INFO:tasks.workunit.client.0.smithi018.stdout:1/881: write d3/d9/da/dd/f3f [5147932,80853] 0 2024-04-12T20:56:28.618 INFO:tasks.workunit.client.1.smithi186.stdout:8/203: creat d7/d18/d36/d1d/d1f/d26/f3d x:0 0 0 2024-04-12T20:56:28.621 INFO:tasks.workunit.client.1.smithi186.stdout:5/217: rename d5/d30 to d5/db/d15/d2a/d2d/d48 0 2024-04-12T20:56:28.622 INFO:tasks.workunit.client.0.smithi018.stdout:1/882: readlink d3/d9/da/dd/d31/d32/d33/l89 0 2024-04-12T20:56:28.623 INFO:tasks.workunit.client.1.smithi186.stdout:3/255: fdatasync d1/d3/d21/d22/f4c 0 2024-04-12T20:56:28.625 INFO:tasks.workunit.client.1.smithi186.stdout:6/220: mkdir d9/d43/d2d/d4d 0 2024-04-12T20:56:28.626 INFO:tasks.workunit.client.1.smithi186.stdout:8/204: readlink d7/lb 0 2024-04-12T20:56:28.631 INFO:tasks.workunit.client.1.smithi186.stdout:9/246: fsync d3/ff 0 2024-04-12T20:56:28.634 INFO:tasks.workunit.client.1.smithi186.stdout:6/221: creat d9/d43/d2d/f4e x:0 0 0 2024-04-12T20:56:28.634 INFO:tasks.workunit.client.1.smithi186.stdout:5/218: unlink d5/db/d14/l3b 0 2024-04-12T20:56:28.638 INFO:tasks.workunit.client.1.smithi186.stdout:4/333: dwrite d0/d21/f1b [0,4194304] 0 2024-04-12T20:56:28.638 INFO:tasks.workunit.client.1.smithi186.stdout:9/247: rename d3/d12/c14 to d3/d2f/d2e/d43/d4c/c4f 0 2024-04-12T20:56:28.640 INFO:tasks.workunit.client.1.smithi186.stdout:6/222: creat d9/d43/f4f x:0 0 0 2024-04-12T20:56:28.641 INFO:tasks.workunit.client.1.smithi186.stdout:5/219: creat d5/db/d15/d2a/d2d/f49 x:0 0 0 2024-04-12T20:56:28.643 INFO:tasks.workunit.client.1.smithi186.stdout:4/334: creat d0/dc/d33/f68 x:0 0 0 2024-04-12T20:56:28.644 INFO:tasks.workunit.client.1.smithi186.stdout:9/248: rename d3/d2f/d2e to d3/d2f/d2e/d33/d50 22 2024-04-12T20:56:28.644 INFO:tasks.workunit.client.0.smithi018.stdout:2/970: dread d5/d8/d10/d43/de2/daf/d90/d142/de5/f5f [0,4194304] 0 2024-04-12T20:56:28.644 INFO:tasks.workunit.client.1.smithi186.stdout:6/223: mkdir d9/d43/d50 0 2024-04-12T20:56:28.647 INFO:tasks.workunit.client.1.smithi186.stdout:5/220: mknod d5/db/d2e/c4a 0 2024-04-12T20:56:28.647 INFO:tasks.workunit.client.1.smithi186.stdout:8/205: dread d7/fd [0,4194304] 0 2024-04-12T20:56:28.648 INFO:tasks.workunit.client.1.smithi186.stdout:3/256: dwrite d1/d3/d21/f41 [0,4194304] 0 2024-04-12T20:56:28.650 INFO:tasks.workunit.client.1.smithi186.stdout:8/206: write d7/d18/d36/d17/f1c [1919300,73750] 0 2024-04-12T20:56:28.650 INFO:tasks.workunit.client.0.smithi018.stdout:1/883: dwrite d3/d9/da/dd/d31/d32/d13/d19/f62 [0,4194304] 0 2024-04-12T20:56:28.651 INFO:tasks.workunit.client.1.smithi186.stdout:9/249: creat d3/d2f/d2e/d43/d4c/f51 x:0 0 0 2024-04-12T20:56:28.652 INFO:tasks.workunit.client.1.smithi186.stdout:6/224: mknod d9/d4c/c51 0 2024-04-12T20:56:28.653 INFO:tasks.workunit.client.1.smithi186.stdout:9/250: chown l0 135370103 1 2024-04-12T20:56:28.653 INFO:tasks.workunit.client.1.smithi186.stdout:5/221: creat d5/db/d15/d2a/d2d/d48/f4b x:0 0 0 2024-04-12T20:56:28.654 INFO:tasks.workunit.client.1.smithi186.stdout:4/335: creat d0/f69 x:0 0 0 2024-04-12T20:56:28.655 INFO:tasks.workunit.client.1.smithi186.stdout:5/222: dread - d5/db/f2b zero size 2024-04-12T20:56:28.656 INFO:tasks.workunit.client.1.smithi186.stdout:8/207: write d7/d18/d36/f1b [4485989,112587] 0 2024-04-12T20:56:28.656 INFO:tasks.workunit.client.1.smithi186.stdout:3/257: rename d1/l18 to d1/d3/d21/d22/d4a/l50 0 2024-04-12T20:56:28.657 INFO:tasks.workunit.client.1.smithi186.stdout:4/336: chown d0/dc/d33/f46 224 1 2024-04-12T20:56:28.657 INFO:tasks.workunit.client.0.smithi018.stdout:2/971: truncate d5/d8/d10/d43/de2/d24/d39/da4/fbb 1477007 0 2024-04-12T20:56:28.659 INFO:tasks.workunit.client.1.smithi186.stdout:9/251: mkdir d3/d2f/d2e/d33/d52 0 2024-04-12T20:56:28.659 INFO:tasks.workunit.client.1.smithi186.stdout:3/258: chown d1/d3/da/c15 93622848 1 2024-04-12T20:56:28.660 INFO:tasks.workunit.client.1.smithi186.stdout:8/208: readlink d7/d18/d36/d20/d23/d25/l2d 0 2024-04-12T20:56:28.660 INFO:tasks.workunit.client.1.smithi186.stdout:4/337: write d0/dc/d33/f4 [1510183,92700] 0 2024-04-12T20:56:28.660 INFO:tasks.workunit.client.1.smithi186.stdout:9/252: dread - d3/f19 zero size 2024-04-12T20:56:28.662 INFO:tasks.workunit.client.0.smithi018.stdout:2/972: readlink d5/d7/d79/d125/lea 0 2024-04-12T20:56:28.663 INFO:tasks.workunit.client.0.smithi018.stdout:2/973: chown d5/d8/d10 13234090 1 2024-04-12T20:56:28.667 INFO:tasks.workunit.client.1.smithi186.stdout:5/223: mkdir d5/db/d15/d2a/d4c 0 2024-04-12T20:56:28.669 INFO:tasks.workunit.client.1.smithi186.stdout:6/225: dwrite f6 [0,4194304] 0 2024-04-12T20:56:28.676 INFO:tasks.workunit.client.0.smithi018.stdout:1/884: dwrite d3/d9/da/dd/d31/d32/d13/d19/f122 [0,4194304] 0 2024-04-12T20:56:28.677 INFO:tasks.workunit.client.1.smithi186.stdout:8/209: link d7/d18/d36/d20/d23/d25/l2d d7/de/d10/l3e 0 2024-04-12T20:56:28.678 INFO:tasks.workunit.client.1.smithi186.stdout:4/338: creat d0/d21/f6a x:0 0 0 2024-04-12T20:56:28.682 INFO:tasks.workunit.client.1.smithi186.stdout:9/253: dwrite d3/d2f/d2e/d33/f4b [0,4194304] 0 2024-04-12T20:56:28.683 INFO:tasks.workunit.client.1.smithi186.stdout:6/226: mknod d9/d43/d2d/d4d/c52 0 2024-04-12T20:56:28.686 INFO:tasks.workunit.client.0.smithi018.stdout:1/885: rename d3/d9/da/d14 to d3/d9/d12c 0 2024-04-12T20:56:28.687 INFO:tasks.workunit.client.1.smithi186.stdout:8/210: chown d7/d18/d36/c24 32 1 2024-04-12T20:56:28.693 INFO:tasks.workunit.client.1.smithi186.stdout:9/254: creat d3/d2f/d2e/d33/d36/f53 x:0 0 0 2024-04-12T20:56:28.693 INFO:tasks.workunit.client.0.smithi018.stdout:2/974: dwrite d5/d8/d10/d43/de2/f119 [0,4194304] 0 2024-04-12T20:56:28.695 INFO:tasks.workunit.client.1.smithi186.stdout:5/224: dwrite d5/db/d15/d2a/d2d/d48/f4b [0,4194304] 0 2024-04-12T20:56:28.697 INFO:tasks.workunit.client.0.smithi018.stdout:1/886: creat d3/d9/da/dd/d31/d32/d13/d19/d2d/d4e/d58/dee/f12d x:0 0 0 2024-04-12T20:56:28.698 INFO:tasks.workunit.client.1.smithi186.stdout:4/339: rmdir d0/dc/d55 0 2024-04-12T20:56:28.701 INFO:tasks.workunit.client.1.smithi186.stdout:5/225: chown d5/db/d15/d18/c43 2834 1 2024-04-12T20:56:28.706 INFO:tasks.workunit.client.0.smithi018.stdout:1/887: rmdir d3/d9/da/dd/d31/d32/d13/d19/d2d/d4e 39 2024-04-12T20:56:28.706 INFO:tasks.workunit.client.0.smithi018.stdout:2/975: rename d5/d8/d10/f48 to d5/d8/d10/d43/de2/f14d 0 2024-04-12T20:56:28.707 INFO:tasks.workunit.client.1.smithi186.stdout:4/340: creat d0/dc/d33/f6b x:0 0 0 2024-04-12T20:56:28.709 INFO:tasks.workunit.client.1.smithi186.stdout:8/211: getdents d7/d18/d36/d20/d23/d25/d29 0 2024-04-12T20:56:28.713 INFO:tasks.workunit.client.0.smithi018.stdout:1/888: write d3/d9/da/dd/d31/d32/d13/d19/f62 [1042835,18159] 0 2024-04-12T20:56:28.716 INFO:tasks.workunit.client.1.smithi186.stdout:4/341: rename d0/d21/d3d/d48/f51 to d0/dc/d33/d5c/f6c 0 2024-04-12T20:56:28.719 INFO:tasks.workunit.client.1.smithi186.stdout:5/226: dread d5/db/d14/f16 [0,4194304] 0 2024-04-12T20:56:28.721 INFO:tasks.workunit.client.1.smithi186.stdout:4/342: symlink d0/dc/d1f/l6d 0 2024-04-12T20:56:28.723 INFO:tasks.workunit.client.1.smithi186.stdout:3/259: write d1/d2c/d48/f4f [2516972,60251] 0 2024-04-12T20:56:28.727 INFO:tasks.workunit.client.1.smithi186.stdout:3/260: dread - d1/d3/d21/d22/d24/d28/d3d/f40 zero size 2024-04-12T20:56:28.733 INFO:tasks.workunit.client.0.smithi018.stdout:2/976: write d5/d8/d33/f76 [1823924,113022] 0 2024-04-12T20:56:28.738 INFO:tasks.workunit.client.0.smithi018.stdout:2/977: creat d5/d8/d10/d43/de2/d24/d39/f14e x:0 0 0 2024-04-12T20:56:28.740 INFO:tasks.workunit.client.0.smithi018.stdout:1/889: dwrite d3/d9/da/dd/d31/d32/d13/d19/d2d/d4e/f5b [0,4194304] 0 2024-04-12T20:56:28.746 INFO:tasks.workunit.client.0.smithi018.stdout:1/890: creat d3/d9/da/dd/d31/d32/d13/d19/d2d/d4e/db9/f12e x:0 0 0 2024-04-12T20:56:28.752 INFO:tasks.workunit.client.0.smithi018.stdout:1/891: rename d3/d9/da/dd/d31/d44 to d3/d9/da/dd/d31/d32/d13/d19/d2d/d4e/d5e/da0/d12f 0 2024-04-12T20:56:28.761 INFO:tasks.workunit.client.0.smithi018.stdout:2/978: dwrite d5/d7/d31/d62/f72 [0,4194304] 0 2024-04-12T20:56:28.765 INFO:tasks.workunit.client.0.smithi018.stdout:2/979: symlink d5/d8/d10/l14f 0 2024-04-12T20:56:28.769 INFO:tasks.workunit.client.0.smithi018.stdout:1/892: dwrite d3/d9/da/fae [0,4194304] 0 2024-04-12T20:56:28.774 INFO:tasks.workunit.client.0.smithi018.stdout:1/893: write d3/d9/da/dd/d31/d32/d13/d19/d2d/f76 [3007759,44674] 0 2024-04-12T20:56:28.776 INFO:tasks.workunit.client.0.smithi018.stdout:2/980: truncate d5/d7/fe0 3875560 0 2024-04-12T20:56:28.778 INFO:tasks.workunit.client.0.smithi018.stdout:1/894: write d3/d9/da/dd/d31/d32/d33/d84/dc2/f10a [482790,483] 0 2024-04-12T20:56:28.780 INFO:tasks.workunit.client.0.smithi018.stdout:2/981: creat d5/d8/d10/d43/de2/d24/d39/f150 x:0 0 0 2024-04-12T20:56:28.783 INFO:tasks.workunit.client.0.smithi018.stdout:2/982: write d5/d8/d10/f133 [847799,63411] 0 2024-04-12T20:56:28.783 INFO:tasks.workunit.client.0.smithi018.stdout:1/895: mkdir d3/d9/da/dd/d31/d32/d13/d19/d2d/d4e/db9/d130 0 2024-04-12T20:56:28.786 INFO:tasks.workunit.client.0.smithi018.stdout:2/983: mknod d5/d12/c151 0 2024-04-12T20:56:28.787 INFO:tasks.workunit.client.0.smithi018.stdout:1/896: write d3/d9/da/dd/d31/d32/d33/f96 [4797439,82616] 0 2024-04-12T20:56:28.789 INFO:tasks.workunit.client.0.smithi018.stdout:2/984: write d5/d7/d31/d62/f122 [972964,21657] 0 2024-04-12T20:56:28.792 INFO:tasks.workunit.client.0.smithi018.stdout:2/985: creat d5/d7/d31/d62/f152 x:0 0 0 2024-04-12T20:56:28.792 INFO:tasks.workunit.client.0.smithi018.stdout:1/897: creat d3/d9/da/dd/d31/d32/d13/d19/f131 x:0 0 0 2024-04-12T20:56:28.794 INFO:tasks.workunit.client.0.smithi018.stdout:2/986: stat d5/d8/d10/d43/de2/d24/d39/l83 0 2024-04-12T20:56:28.796 INFO:tasks.workunit.client.0.smithi018.stdout:2/987: mknod d5/d7/d31/c153 0 2024-04-12T20:56:28.797 INFO:tasks.workunit.client.0.smithi018.stdout:1/898: unlink d3/d9/da/dd/d31/d32/d13/d19/d2d/d4e/d7a/l129 0 2024-04-12T20:56:28.800 INFO:tasks.workunit.client.0.smithi018.stdout:1/899: fdatasync d3/d9/da/dd/d31/d32/d13/d19/d2d/d4e/d7a/d82/fb5 0 2024-04-12T20:56:28.804 INFO:tasks.workunit.client.0.smithi018.stdout:1/900: truncate d3/d9/da/dd/d31/d32/d33/d80/f125 997303 0 2024-04-12T20:56:28.810 INFO:tasks.workunit.client.0.smithi018.stdout:1/901: creat d3/d9/da/dd/d31/d32/d13/d19/d2d/d4e/f132 x:0 0 0 2024-04-12T20:56:28.811 INFO:tasks.workunit.client.0.smithi018.stdout:2/988: dwrite d5/d8/d10/d43/f56 [0,4194304] 0 2024-04-12T20:56:28.811 INFO:tasks.workunit.client.0.smithi018.stdout:2/989: stat f4 0 2024-04-12T20:56:28.812 INFO:tasks.workunit.client.1.smithi186.stdout:5/227: rename d5/db/l1d to d5/db/l4d 0 2024-04-12T20:56:28.812 INFO:tasks.workunit.client.1.smithi186.stdout:4/343: link d0/l1 d0/dc/d64/l6e 0 2024-04-12T20:56:28.812 INFO:tasks.workunit.client.0.smithi018.stdout:1/902: write d3/d9/d12c/d73/f11c [734808,82426] 0 2024-04-12T20:56:28.814 INFO:tasks.workunit.client.0.smithi018.stdout:2/990: fsync d5/d8/d10/f78 0 2024-04-12T20:56:28.814 INFO:tasks.workunit.client.0.smithi018.stdout:1/903: stat d3/d9/da/dd/d31/d32/l2e 0 2024-04-12T20:56:28.816 INFO:tasks.workunit.client.1.smithi186.stdout:3/261: write d1/d3/d21/f41 [1984721,115349] 0 2024-04-12T20:56:28.818 INFO:tasks.workunit.client.1.smithi186.stdout:5/228: symlink d5/db/d14/l4e 0 2024-04-12T20:56:28.820 INFO:tasks.workunit.client.1.smithi186.stdout:9/255: dread d3/f5 [0,4194304] 0 2024-04-12T20:56:28.821 INFO:tasks.workunit.client.1.smithi186.stdout:4/344: symlink d0/d21/d3d/d48/d61/l6f 0 2024-04-12T20:56:28.826 INFO:tasks.workunit.client.1.smithi186.stdout:5/229: creat d5/db/d15/d2a/f4f x:0 0 0 2024-04-12T20:56:28.827 INFO:tasks.workunit.client.1.smithi186.stdout:5/230: stat d5/db/d15/d2a/d4c 0 2024-04-12T20:56:28.828 INFO:tasks.workunit.client.1.smithi186.stdout:4/345: mknod d0/dc/d33/c70 0 2024-04-12T20:56:28.829 INFO:tasks.workunit.client.1.smithi186.stdout:5/231: chown d5/f9 13 1 2024-04-12T20:56:28.829 INFO:tasks.workunit.client.0.smithi018.stdout:2/991: truncate d5/d7/d31/fbe 3102250 0 2024-04-12T20:56:28.830 INFO:tasks.workunit.client.1.smithi186.stdout:9/256: getdents d3/d13 0 2024-04-12T20:56:28.832 INFO:tasks.workunit.client.1.smithi186.stdout:3/262: dread d1/d3/d21/f41 [0,4194304] 0 2024-04-12T20:56:28.832 INFO:tasks.workunit.client.1.smithi186.stdout:4/346: mknod d0/d21/d3d/d48/d61/c71 0 2024-04-12T20:56:28.833 INFO:tasks.workunit.client.1.smithi186.stdout:4/347: stat d0/dc/d33/f7 0 2024-04-12T20:56:28.834 INFO:tasks.workunit.client.1.smithi186.stdout:5/232: rename d5/db/d15/d2a/c2c to d5/d44/c50 0 2024-04-12T20:56:28.835 INFO:tasks.workunit.client.1.smithi186.stdout:4/348: write d0/f24 [1563534,77629] 0 2024-04-12T20:56:28.836 INFO:tasks.workunit.client.0.smithi018.stdout:1/904: dwrite d3/d9/da/dd/d31/f59 [4194304,4194304] 0 2024-04-12T20:56:28.839 INFO:tasks.workunit.client.0.smithi018.stdout:2/992: write d5/d8/d10/d43/de2/fae [4939456,75196] 0 2024-04-12T20:56:28.841 INFO:tasks.workunit.client.1.smithi186.stdout:5/233: symlink d5/db/d15/d18/l51 0 2024-04-12T20:56:28.843 INFO:tasks.workunit.client.0.smithi018.stdout:1/905: write d3/d9/da/dd/d31/d32/d13/d19/d2d/d4e/d7a/f124 [1134843,129446] 0 2024-04-12T20:56:28.845 INFO:tasks.workunit.client.0.smithi018.stdout:2/993: mknod d5/d12/c154 0 2024-04-12T20:56:28.850 INFO:tasks.workunit.client.0.smithi018.stdout:2/994: creat d5/d8/d10/d43/de2/d24/d39/d9c/f155 x:0 0 0 2024-04-12T20:56:28.864 INFO:tasks.workunit.client.0.smithi018.stdout:1/906: dwrite d3/d9/da/dd/d31/d32/d13/d19/d2d/d4e/d7a/f104 [0,4194304] 0 2024-04-12T20:56:28.869 INFO:tasks.workunit.client.0.smithi018.stdout:2/995: dwrite d5/d8/f57 [0,4194304] 0 2024-04-12T20:56:28.870 INFO:tasks.workunit.client.1.smithi186.stdout:3/263: creat d1/d3/d49/d4d/f51 x:0 0 0 2024-04-12T20:56:28.870 INFO:tasks.workunit.client.1.smithi186.stdout:4/349: rename d0/dc/d1f/l6d to d0/d21/d3d/d63/l72 0 2024-04-12T20:56:28.871 INFO:tasks.workunit.client.1.smithi186.stdout:5/234: creat d5/db/d14/f52 x:0 0 0 2024-04-12T20:56:28.887 INFO:tasks.workunit.client.1.smithi186.stdout:4/350: dread d0/dc/d33/f57 [0,4194304] 0 2024-04-12T20:56:28.887 INFO:tasks.workunit.client.0.smithi018.stdout:1/907: dwrite d3/d65/f120 [0,4194304] 0 2024-04-12T20:56:28.891 INFO:tasks.workunit.client.1.smithi186.stdout:4/351: mkdir d0/dc/d1f/d73 0 2024-04-12T20:56:28.891 INFO:tasks.workunit.client.1.smithi186.stdout:3/264: dwrite d1/d3/d21/f41 [0,4194304] 0 2024-04-12T20:56:28.895 INFO:tasks.workunit.client.1.smithi186.stdout:4/352: symlink d0/dc/d33/d5c/l74 0 2024-04-12T20:56:28.896 INFO:tasks.workunit.client.1.smithi186.stdout:5/235: dwrite d5/fd [4194304,4194304] 0 2024-04-12T20:56:28.898 INFO:tasks.workunit.client.1.smithi186.stdout:3/265: mknod d1/d3/d21/d22/c52 0 2024-04-12T20:56:28.901 INFO:tasks.workunit.client.1.smithi186.stdout:3/266: truncate d1/d3/d21/d22/f4c 1748150 0 2024-04-12T20:56:28.904 INFO:tasks.workunit.client.1.smithi186.stdout:5/236: truncate d5/db/d14/f52 797373 0 2024-04-12T20:56:28.904 INFO:tasks.workunit.client.1.smithi186.stdout:3/267: write d1/d3/da/f47 [1743832,17918] 0 2024-04-12T20:56:28.904 INFO:tasks.workunit.client.1.smithi186.stdout:4/353: mknod d0/dc/d1f/d73/c75 0 2024-04-12T20:56:28.909 INFO:tasks.workunit.client.1.smithi186.stdout:4/354: creat d0/d21/d3d/d48/f76 x:0 0 0 2024-04-12T20:56:28.911 INFO:tasks.workunit.client.1.smithi186.stdout:4/355: write d0/dc/d33/fb [5921795,53605] 0 2024-04-12T20:56:28.914 INFO:tasks.workunit.client.1.smithi186.stdout:3/268: creat d1/d3/d21/d22/d24/f53 x:0 0 0 2024-04-12T20:56:28.916 INFO:tasks.workunit.client.1.smithi186.stdout:3/269: write d1/d3/da/f47 [120179,69852] 0 2024-04-12T20:56:28.916 INFO:tasks.workunit.client.1.smithi186.stdout:5/237: rename d5/db/lf to d5/db/d33/l53 0 2024-04-12T20:56:28.920 INFO:tasks.workunit.client.0.smithi018.stdout:1/908: creat d3/d9/da/f133 x:0 0 0 2024-04-12T20:56:28.926 INFO:tasks.workunit.client.1.smithi186.stdout:4/356: unlink d0/dc/d1f/c4b 0 2024-04-12T20:56:28.927 INFO:tasks.workunit.client.1.smithi186.stdout:5/238: creat d5/db/d14/f54 x:0 0 0 2024-04-12T20:56:28.932 INFO:tasks.workunit.client.1.smithi186.stdout:7/304: sync 2024-04-12T20:56:28.932 INFO:tasks.workunit.client.1.smithi186.stdout:1/242: sync 2024-04-12T20:56:28.932 INFO:tasks.workunit.client.1.smithi186.stdout:0/319: sync 2024-04-12T20:56:28.935 INFO:tasks.workunit.client.0.smithi018.stdout:1/909: write d3/d9/faf [1035423,26564] 0 2024-04-12T20:56:28.935 INFO:tasks.workunit.client.1.smithi186.stdout:5/239: write f2 [67119,976] 0 2024-04-12T20:56:28.936 INFO:tasks.workunit.client.1.smithi186.stdout:0/320: mkdir d0/d5c/d65 0 2024-04-12T20:56:28.938 INFO:tasks.workunit.client.1.smithi186.stdout:1/243: symlink d14/l4e 0 2024-04-12T20:56:28.942 INFO:tasks.workunit.client.1.smithi186.stdout:0/321: mkdir d0/d5c/d66 0 2024-04-12T20:56:28.944 INFO:tasks.workunit.client.1.smithi186.stdout:5/240: unlink d5/db/l13 0 2024-04-12T20:56:28.946 INFO:tasks.workunit.client.1.smithi186.stdout:7/305: rename d7/d10/d1c/f63 to d7/d55/d5f/d4d/f73 0 2024-04-12T20:56:28.947 INFO:tasks.workunit.client.1.smithi186.stdout:0/322: fdatasync d0/d14/f19 0 2024-04-12T20:56:28.947 INFO:tasks.workunit.client.1.smithi186.stdout:3/270: dwrite d1/fc [0,4194304] 0 2024-04-12T20:56:28.947 INFO:tasks.workunit.client.1.smithi186.stdout:1/244: rmdir d14/d1a/d21/d2c/d3e 39 2024-04-12T20:56:28.947 INFO:tasks.workunit.client.1.smithi186.stdout:4/357: dwrite d0/d21/f6a [0,4194304] 0 2024-04-12T20:56:28.948 INFO:tasks.workunit.client.0.smithi018.stdout:1/910: dread d3/d9/da/dd/d31/d32/d33/d80/fca [0,4194304] 0 2024-04-12T20:56:28.957 INFO:tasks.workunit.client.1.smithi186.stdout:3/271: mkdir d1/d2c/d54 0 2024-04-12T20:56:28.959 INFO:tasks.workunit.client.1.smithi186.stdout:0/323: getdents d0/d14 0 2024-04-12T20:56:28.962 INFO:tasks.workunit.client.1.smithi186.stdout:3/272: mknod d1/d2c/c55 0 2024-04-12T20:56:28.964 INFO:tasks.workunit.client.0.smithi018.stdout:1/911: truncate d3/d9/da/dd/d31/d32/d33/f6d 7807936 0 2024-04-12T20:56:28.964 INFO:tasks.workunit.client.1.smithi186.stdout:0/324: creat d0/d10/f67 x:0 0 0 2024-04-12T20:56:28.965 INFO:tasks.workunit.client.1.smithi186.stdout:3/273: write d1/d2c/d48/f4f [7708540,73673] 0 2024-04-12T20:56:28.967 INFO:tasks.workunit.client.0.smithi018.stdout:1/912: dread - d3/d9/da/f133 zero size 2024-04-12T20:56:28.969 INFO:tasks.workunit.client.1.smithi186.stdout:5/241: truncate d5/db/d15/d2a/d2d/d48/f4b 1700140 0 2024-04-12T20:56:28.969 INFO:tasks.workunit.client.1.smithi186.stdout:1/245: dread d14/d1a/f1b [0,4194304] 0 2024-04-12T20:56:28.973 INFO:tasks.workunit.client.1.smithi186.stdout:5/242: readlink d5/db/d33/l53 0 2024-04-12T20:56:28.975 INFO:tasks.workunit.client.1.smithi186.stdout:1/246: creat d14/f4f x:0 0 0 2024-04-12T20:56:28.976 INFO:tasks.workunit.client.1.smithi186.stdout:5/243: write d5/db/d14/f16 [784100,9618] 0 2024-04-12T20:56:28.978 INFO:tasks.workunit.client.1.smithi186.stdout:5/244: chown d5/la 1 1 2024-04-12T20:56:28.978 INFO:tasks.workunit.client.1.smithi186.stdout:4/358: dread d0/d21/f32 [0,4194304] 0 2024-04-12T20:56:28.980 INFO:tasks.workunit.client.1.smithi186.stdout:1/247: mknod d14/d1a/d3c/c50 0 2024-04-12T20:56:28.982 INFO:tasks.workunit.client.1.smithi186.stdout:5/245: fdatasync d5/db/d14/f16 0 2024-04-12T20:56:28.984 INFO:tasks.workunit.client.0.smithi018.stdout:1/913: dread d3/d9/da/dd/d31/d32/d13/d19/f122 [0,4194304] 0 2024-04-12T20:56:28.986 INFO:tasks.workunit.client.1.smithi186.stdout:0/325: write d0/d14/d4e/f33 [1260625,110642] 0 2024-04-12T20:56:28.988 INFO:tasks.workunit.client.1.smithi186.stdout:5/246: chown d5/db/d14 207447 1 2024-04-12T20:56:28.990 INFO:tasks.workunit.client.1.smithi186.stdout:1/248: creat d14/d1a/d3c/f51 x:0 0 0 2024-04-12T20:56:28.990 INFO:tasks.workunit.client.1.smithi186.stdout:4/359: link d0/dc/d39/c45 d0/d21/d3d/d48/c77 0 2024-04-12T20:56:28.990 INFO:tasks.workunit.client.1.smithi186.stdout:3/274: dwrite d1/d2c/d48/f4f [0,4194304] 0 2024-04-12T20:56:28.993 INFO:tasks.workunit.client.1.smithi186.stdout:0/326: fsync d0/d10/d32/d3f/f64 0 2024-04-12T20:56:28.993 INFO:tasks.workunit.client.1.smithi186.stdout:6/227: sync 2024-04-12T20:56:28.993 INFO:tasks.workunit.client.1.smithi186.stdout:8/212: sync 2024-04-12T20:56:28.993 INFO:tasks.workunit.client.1.smithi186.stdout:2/266: sync 2024-04-12T20:56:28.993 INFO:tasks.workunit.client.1.smithi186.stdout:1/249: fdatasync f4 0 2024-04-12T20:56:28.994 INFO:tasks.workunit.client.1.smithi186.stdout:5/247: write d5/db/d15/d2a/f4f [845754,103593] 0 2024-04-12T20:56:28.995 INFO:tasks.workunit.client.1.smithi186.stdout:7/306: dread d7/d10/f13 [0,4194304] 0 2024-04-12T20:56:28.999 INFO:tasks.workunit.client.1.smithi186.stdout:4/360: mknod d0/dc/d39/c78 0 2024-04-12T20:56:28.999 INFO:tasks.workunit.client.1.smithi186.stdout:2/267: rmdir d0/dc 39 2024-04-12T20:56:29.000 INFO:tasks.workunit.client.1.smithi186.stdout:3/275: write d1/d3/d21/d22/d24/f53 [608281,31714] 0 2024-04-12T20:56:29.002 INFO:tasks.workunit.client.1.smithi186.stdout:0/327: stat d0/d10/d32/l62 0 2024-04-12T20:56:29.005 INFO:tasks.workunit.client.1.smithi186.stdout:3/276: fsync d1/fc 0 2024-04-12T20:56:29.006 INFO:tasks.workunit.client.1.smithi186.stdout:5/248: write d5/db/d15/d2a/f4f [1178883,118940] 0 2024-04-12T20:56:29.006 INFO:tasks.workunit.client.1.smithi186.stdout:7/307: chown d7/d10/d2d/d40/d4a/d6c 8 1 2024-04-12T20:56:29.006 INFO:tasks.workunit.client.0.smithi018.stdout:1/914: dwrite d3/d9/da/dd/f4a [0,4194304] 0 2024-04-12T20:56:29.007 INFO:tasks.workunit.client.1.smithi186.stdout:4/361: creat d0/dc/d39/f79 x:0 0 0 2024-04-12T20:56:29.008 INFO:tasks.workunit.client.1.smithi186.stdout:2/268: symlink d0/l51 0 2024-04-12T20:56:29.009 INFO:tasks.workunit.client.1.smithi186.stdout:3/277: stat d1/d3/d21/d22/l3b 0 2024-04-12T20:56:29.010 INFO:tasks.workunit.client.1.smithi186.stdout:4/362: chown d0/f69 0 1 2024-04-12T20:56:29.011 INFO:tasks.workunit.client.0.smithi018.stdout:1/915: write d3/d9/da/dd/d31/d32/d13/f46 [74490,12122] 0 2024-04-12T20:56:29.013 INFO:tasks.workunit.client.1.smithi186.stdout:1/250: rename d14/f22 to d14/d1a/d21/d2c/d3e/f52 0 2024-04-12T20:56:29.016 INFO:tasks.workunit.client.1.smithi186.stdout:7/308: dread d7/d10/d1c/f52 [0,4194304] 0 2024-04-12T20:56:29.017 INFO:tasks.workunit.client.1.smithi186.stdout:6/228: getdents d9/d4c 0 2024-04-12T20:56:29.018 INFO:tasks.workunit.client.0.smithi018.stdout:1/916: creat d3/d65/f134 x:0 0 0 2024-04-12T20:56:29.019 INFO:tasks.workunit.client.1.smithi186.stdout:6/229: write f6 [612980,15641] 0 2024-04-12T20:56:29.019 INFO:tasks.workunit.client.1.smithi186.stdout:5/249: fsync d5/db/d15/d2a/f41 0 2024-04-12T20:56:29.021 INFO:tasks.workunit.client.1.smithi186.stdout:0/328: dread d0/d14/d4e/d1e/f37 [0,4194304] 0 2024-04-12T20:56:29.024 INFO:tasks.workunit.client.1.smithi186.stdout:0/329: fdatasync d0/d14/d4e/d1e/f2f 0 2024-04-12T20:56:29.024 INFO:tasks.workunit.client.1.smithi186.stdout:8/213: truncate d7/d18/d36/f1b 4492191 0 2024-04-12T20:56:29.025 INFO:tasks.workunit.client.0.smithi018.stdout:1/917: symlink d3/d9/da/dd/d31/d32/d33/d84/dc2/l135 0 2024-04-12T20:56:29.025 INFO:tasks.workunit.client.1.smithi186.stdout:4/363: dread d0/dc/d33/f7 [0,4194304] 0 2024-04-12T20:56:29.025 INFO:tasks.workunit.client.1.smithi186.stdout:1/251: dread d14/d1a/d21/f4b [0,4194304] 0 2024-04-12T20:56:29.026 INFO:tasks.workunit.client.1.smithi186.stdout:3/278: chown d1/d2c/c3c 448925794 1 2024-04-12T20:56:29.026 INFO:tasks.workunit.client.1.smithi186.stdout:7/309: creat d7/f74 x:0 0 0 2024-04-12T20:56:29.027 INFO:tasks.workunit.client.1.smithi186.stdout:6/230: creat d9/d3f/f53 x:0 0 0 2024-04-12T20:56:29.028 INFO:tasks.workunit.client.0.smithi018.stdout:1/918: chown d3/d9/da/dd/d31/d32/d13/d19/d2d/d4e/f8c 3 1 2024-04-12T20:56:29.030 INFO:tasks.workunit.client.1.smithi186.stdout:1/252: write d14/d1a/d1d/f29 [214278,47815] 0 2024-04-12T20:56:29.030 INFO:tasks.workunit.client.1.smithi186.stdout:7/310: stat d7/d10/d2d/f37 0 2024-04-12T20:56:29.031 INFO:tasks.workunit.client.1.smithi186.stdout:0/330: symlink d0/d5c/d65/l68 0 2024-04-12T20:56:29.032 INFO:tasks.workunit.client.1.smithi186.stdout:3/279: creat d1/d3/da/f56 x:0 0 0 2024-04-12T20:56:29.033 INFO:tasks.workunit.client.0.smithi018.stdout:1/919: symlink d3/d9/da/dd/d31/d32/d13/d19/d2d/d4e/d7a/l136 0 2024-04-12T20:56:29.033 INFO:tasks.workunit.client.1.smithi186.stdout:7/311: readlink d7/d10/d2d/l3e 0 2024-04-12T20:56:29.035 INFO:tasks.workunit.client.1.smithi186.stdout:8/214: creat d7/d18/d36/d1d/d1f/d26/f3f x:0 0 0 2024-04-12T20:56:29.037 INFO:tasks.workunit.client.0.smithi018.stdout:1/920: creat d3/d9/d12c/d113/f137 x:0 0 0 2024-04-12T20:56:29.038 INFO:tasks.workunit.client.1.smithi186.stdout:1/253: rmdir d14/d1a/d33/d36 39 2024-04-12T20:56:29.038 INFO:tasks.workunit.client.1.smithi186.stdout:4/364: dread d0/dc/d33/f1a [0,4194304] 0 2024-04-12T20:56:29.038 INFO:tasks.workunit.client.1.smithi186.stdout:3/280: write d1/d3/da/f47 [1099151,79629] 0 2024-04-12T20:56:29.041 INFO:tasks.workunit.client.1.smithi186.stdout:0/331: mknod d0/d10/c69 0 2024-04-12T20:56:29.043 INFO:tasks.workunit.client.1.smithi186.stdout:6/231: creat d9/d43/d50/f54 x:0 0 0 2024-04-12T20:56:29.044 INFO:tasks.workunit.client.1.smithi186.stdout:4/365: write d0/f6 [189927,44331] 0 2024-04-12T20:56:29.045 INFO:tasks.workunit.client.1.smithi186.stdout:1/254: fdatasync f4 0 2024-04-12T20:56:29.047 INFO:tasks.workunit.client.1.smithi186.stdout:3/281: write d1/d3/d21/d22/d24/d28/d3d/f40 [253370,68281] 0 2024-04-12T20:56:29.047 INFO:tasks.workunit.client.1.smithi186.stdout:7/312: symlink d7/d10/l75 0 2024-04-12T20:56:29.049 INFO:tasks.workunit.client.1.smithi186.stdout:8/215: symlink d7/de/d10/d2f/l40 0 2024-04-12T20:56:29.049 INFO:tasks.workunit.client.1.smithi186.stdout:0/332: symlink d0/d10/l6a 0 2024-04-12T20:56:29.051 INFO:tasks.workunit.client.1.smithi186.stdout:2/269: dread d0/f10 [0,4194304] 0 2024-04-12T20:56:29.051 INFO:tasks.workunit.client.1.smithi186.stdout:6/232: creat d9/d43/d50/f55 x:0 0 0 2024-04-12T20:56:29.052 INFO:tasks.workunit.client.1.smithi186.stdout:0/333: readlink d0/l26 0 2024-04-12T20:56:29.052 INFO:tasks.workunit.client.1.smithi186.stdout:5/250: dwrite d5/db/d15/d2a/d2d/d48/f3a [0,4194304] 0 2024-04-12T20:56:29.055 INFO:tasks.workunit.client.1.smithi186.stdout:1/255: unlink d14/d1a/d3c/f51 0 2024-04-12T20:56:29.058 INFO:tasks.workunit.client.1.smithi186.stdout:1/256: write d14/d1a/d1d/f46 [927840,39403] 0 2024-04-12T20:56:29.058 INFO:tasks.workunit.client.1.smithi186.stdout:8/216: mknod d7/d18/c41 0 2024-04-12T20:56:29.058 INFO:tasks.workunit.client.0.smithi018.stdout:1/921: dwrite d3/d9/d12c/d73/fd0 [4194304,4194304] 0 2024-04-12T20:56:29.061 INFO:tasks.workunit.client.1.smithi186.stdout:3/282: creat d1/d2c/d48/f57 x:0 0 0 2024-04-12T20:56:29.062 INFO:tasks.workunit.client.1.smithi186.stdout:2/270: symlink d0/d3/d1a/d41/l52 0 2024-04-12T20:56:29.068 INFO:tasks.workunit.client.0.smithi018.stdout:1/922: rename d3/d9/da/dd/d31/d32/d13/d19/d2d/d4e/d58/d61/d71 to d3/d9/da/dd/d138 0 2024-04-12T20:56:29.069 INFO:tasks.workunit.client.1.smithi186.stdout:3/283: chown d1/d3/d21/d22/d24/d2f/l33 76110 1 2024-04-12T20:56:29.069 INFO:tasks.workunit.client.1.smithi186.stdout:5/251: rename d5/db/d15/d18 to d5/db/d15/d2a/d2d/d46/d55 0 2024-04-12T20:56:29.072 INFO:tasks.workunit.client.0.smithi018.stdout:1/923: fdatasync d3/d9/d12c/f38 0 2024-04-12T20:56:29.075 INFO:tasks.workunit.client.1.smithi186.stdout:6/233: truncate d9/d43/f39 1679711 0 2024-04-12T20:56:29.075 INFO:tasks.workunit.client.1.smithi186.stdout:1/257: write d14/d1a/d21/d2c/d3e/f52 [1577900,66787] 0 2024-04-12T20:56:29.079 INFO:tasks.workunit.client.1.smithi186.stdout:7/313: dwrite d7/f8 [0,4194304] 0 2024-04-12T20:56:29.081 INFO:tasks.workunit.client.1.smithi186.stdout:1/258: creat d14/d1a/d21/d2c/d3e/f53 x:0 0 0 2024-04-12T20:56:29.082 INFO:tasks.workunit.client.1.smithi186.stdout:3/284: creat d1/d3/f58 x:0 0 0 2024-04-12T20:56:29.085 INFO:tasks.workunit.client.1.smithi186.stdout:1/259: chown d14/d1a/d21/f31 171360 1 2024-04-12T20:56:29.086 INFO:tasks.workunit.client.1.smithi186.stdout:3/285: truncate d1/d3/f58 147814 0 2024-04-12T20:56:29.087 INFO:tasks.workunit.client.1.smithi186.stdout:6/234: dread d9/f33 [0,4194304] 0 2024-04-12T20:56:29.089 INFO:tasks.workunit.client.1.smithi186.stdout:5/252: read - d5/db/d15/f28 zero size 2024-04-12T20:56:29.089 INFO:tasks.workunit.client.1.smithi186.stdout:2/271: dread d0/d3/d1a/d24/f34 [0,4194304] 0 2024-04-12T20:56:29.089 INFO:tasks.workunit.client.1.smithi186.stdout:3/286: stat d1/d3/d49/d4d 0 2024-04-12T20:56:29.090 INFO:tasks.workunit.client.1.smithi186.stdout:6/235: fdatasync d9/f2e 0 2024-04-12T20:56:29.095 INFO:tasks.workunit.client.1.smithi186.stdout:4/366: dread d0/d21/f20 [0,4194304] 0 2024-04-12T20:56:29.096 INFO:tasks.workunit.client.1.smithi186.stdout:5/253: dread d5/db/d14/f1a [0,4194304] 0 2024-04-12T20:56:29.097 INFO:tasks.workunit.client.1.smithi186.stdout:8/217: dwrite d7/d18/d36/d20/d23/d25/d2e/f3b [0,4194304] 0 2024-04-12T20:56:29.100 INFO:tasks.workunit.client.1.smithi186.stdout:6/236: rename d9/l3d to d9/d43/l56 0 2024-04-12T20:56:29.101 INFO:tasks.workunit.client.1.smithi186.stdout:4/367: symlink d0/d21/d3d/d63/l7a 0 2024-04-12T20:56:29.104 INFO:tasks.workunit.client.1.smithi186.stdout:6/237: write d9/d3f/f48 [504659,51650] 0 2024-04-12T20:56:29.106 INFO:tasks.workunit.client.1.smithi186.stdout:1/260: link d14/d1a/d33/l39 d14/d1a/d1d/l54 0 2024-04-12T20:56:29.109 INFO:tasks.workunit.client.1.smithi186.stdout:5/254: rename d5/db/d14/l4e to d5/db/d15/d2a/d4c/l56 0 2024-04-12T20:56:29.110 INFO:tasks.workunit.client.1.smithi186.stdout:1/261: chown d14/d1a/d21/d2c/f4d 35 1 2024-04-12T20:56:29.112 INFO:tasks.workunit.client.1.smithi186.stdout:2/272: getdents d0/dc 0 2024-04-12T20:56:29.117 INFO:tasks.workunit.client.1.smithi186.stdout:5/255: creat d5/db/f57 x:0 0 0 2024-04-12T20:56:29.117 INFO:tasks.workunit.client.1.smithi186.stdout:8/218: dread d7/de/f11 [0,4194304] 0 2024-04-12T20:56:29.118 INFO:tasks.workunit.client.1.smithi186.stdout:2/273: write d0/d3/d1a/d24/f34 [1014809,73261] 0 2024-04-12T20:56:29.120 INFO:tasks.workunit.client.1.smithi186.stdout:5/256: stat d5/db/d14/c35 0 2024-04-12T20:56:29.123 INFO:tasks.workunit.client.1.smithi186.stdout:4/368: dwrite d0/d21/f31 [0,4194304] 0 2024-04-12T20:56:29.123 INFO:tasks.workunit.client.1.smithi186.stdout:2/274: chown d0/d3/d1a/d21 1197 1 2024-04-12T20:56:29.124 INFO:tasks.workunit.client.1.smithi186.stdout:5/257: dread - d5/db/d14/f54 zero size 2024-04-12T20:56:29.124 INFO:tasks.workunit.client.1.smithi186.stdout:9/257: sync 2024-04-12T20:56:29.127 INFO:tasks.workunit.client.1.smithi186.stdout:4/369: symlink d0/d21/d3d/l7b 0 2024-04-12T20:56:29.128 INFO:tasks.workunit.client.1.smithi186.stdout:3/287: dread d1/f1e [0,4194304] 0 2024-04-12T20:56:29.130 INFO:tasks.workunit.client.1.smithi186.stdout:2/275: mknod d0/d32/d2a/c53 0 2024-04-12T20:56:29.131 INFO:tasks.workunit.client.1.smithi186.stdout:3/288: readlink d1/d3/da/ld 0 2024-04-12T20:56:29.131 INFO:tasks.workunit.client.1.smithi186.stdout:4/370: chown d0/d21/c16 102076278 1 2024-04-12T20:56:29.132 INFO:tasks.workunit.client.1.smithi186.stdout:5/258: truncate d5/db/d15/f39 201734 0 2024-04-12T20:56:29.132 INFO:tasks.workunit.client.1.smithi186.stdout:6/238: dwrite d9/d43/f4f [0,4194304] 0 2024-04-12T20:56:29.133 INFO:tasks.workunit.client.1.smithi186.stdout:9/258: unlink d3/d12/f30 0 2024-04-12T20:56:29.134 INFO:tasks.workunit.client.1.smithi186.stdout:4/371: chown d0/d21/f32 80 1 2024-04-12T20:56:29.135 INFO:tasks.workunit.client.1.smithi186.stdout:5/259: chown d5/db/d14 235 1 2024-04-12T20:56:29.139 INFO:tasks.workunit.client.1.smithi186.stdout:6/239: mknod d9/c57 0 2024-04-12T20:56:29.140 INFO:tasks.workunit.client.1.smithi186.stdout:9/259: mkdir d3/d2f/d54 0 2024-04-12T20:56:29.142 INFO:tasks.workunit.client.1.smithi186.stdout:4/372: unlink d0/dc/f29 0 2024-04-12T20:56:29.142 INFO:tasks.workunit.client.1.smithi186.stdout:6/240: dread - d9/f40 zero size 2024-04-12T20:56:29.143 INFO:tasks.workunit.client.1.smithi186.stdout:4/373: dread - d0/d21/f38 zero size 2024-04-12T20:56:29.146 INFO:tasks.workunit.client.1.smithi186.stdout:5/260: getdents d5/db/d33 0 2024-04-12T20:56:29.146 INFO:tasks.workunit.client.1.smithi186.stdout:4/374: write d0/f10 [1763450,53593] 0 2024-04-12T20:56:29.153 INFO:tasks.workunit.client.1.smithi186.stdout:4/375: dread - d0/dc/d33/f6b zero size 2024-04-12T20:56:29.154 INFO:tasks.workunit.client.1.smithi186.stdout:8/219: dwrite d7/d18/d36/d1d/d1f/d26/f3d [0,4194304] 0 2024-04-12T20:56:29.157 INFO:tasks.workunit.client.1.smithi186.stdout:4/376: read - d0/f69 zero size 2024-04-12T20:56:29.159 INFO:tasks.workunit.client.1.smithi186.stdout:8/220: creat d7/de/f42 x:0 0 0 2024-04-12T20:56:29.161 INFO:tasks.workunit.client.1.smithi186.stdout:4/377: creat d0/d26/f7c x:0 0 0 2024-04-12T20:56:29.163 INFO:tasks.workunit.client.1.smithi186.stdout:9/260: dwrite d3/d2f/f26 [0,4194304] 0 2024-04-12T20:56:29.163 INFO:tasks.workunit.client.1.smithi186.stdout:6/241: truncate d9/f47 3256534 0 2024-04-12T20:56:29.163 INFO:tasks.workunit.client.1.smithi186.stdout:1/262: dread d14/d1a/d3c/f47 [0,4194304] 0 2024-04-12T20:56:29.164 INFO:tasks.workunit.client.1.smithi186.stdout:8/221: creat d7/d18/f43 x:0 0 0 2024-04-12T20:56:29.165 INFO:tasks.workunit.client.1.smithi186.stdout:6/242: dread - d9/d3f/f53 zero size 2024-04-12T20:56:29.166 INFO:tasks.workunit.client.1.smithi186.stdout:4/378: chown d0/d21/d3d/l4c 61093031 1 2024-04-12T20:56:29.170 INFO:tasks.workunit.client.1.smithi186.stdout:4/379: stat d0/d26/f4f 0 2024-04-12T20:56:29.172 INFO:tasks.workunit.client.1.smithi186.stdout:5/261: dread d5/db/d15/d2a/f41 [0,4194304] 0 2024-04-12T20:56:29.172 INFO:tasks.workunit.client.1.smithi186.stdout:9/261: creat d3/d2f/f55 x:0 0 0 2024-04-12T20:56:29.173 INFO:tasks.workunit.client.1.smithi186.stdout:1/263: creat d14/d1a/d3c/f55 x:0 0 0 2024-04-12T20:56:29.178 INFO:tasks.workunit.client.1.smithi186.stdout:9/262: dread - d3/d2f/d2e/d33/d36/f53 zero size 2024-04-12T20:56:29.178 INFO:tasks.workunit.client.1.smithi186.stdout:5/262: fdatasync d5/db/d15/f31 0 2024-04-12T20:56:29.178 INFO:tasks.workunit.client.1.smithi186.stdout:4/380: unlink d0/d21/c2e 0 2024-04-12T20:56:29.183 INFO:tasks.workunit.client.1.smithi186.stdout:9/263: chown d3/d2f/d2e/d43 783911 1 2024-04-12T20:56:29.185 INFO:tasks.workunit.client.1.smithi186.stdout:1/264: link d14/d1a/d21/d2c/d3e/c41 d14/d1a/d21/c56 0 2024-04-12T20:56:29.185 INFO:tasks.workunit.client.1.smithi186.stdout:5/263: creat d5/db/d15/d2a/d2d/f58 x:0 0 0 2024-04-12T20:56:29.187 INFO:tasks.workunit.client.1.smithi186.stdout:9/264: symlink d3/d2f/d2e/l56 0 2024-04-12T20:56:29.187 INFO:tasks.workunit.client.1.smithi186.stdout:2/276: dread d0/d3/d1a/d21/d26/f3d [0,4194304] 0 2024-04-12T20:56:29.187 INFO:tasks.workunit.client.1.smithi186.stdout:1/265: fdatasync fe 0 2024-04-12T20:56:29.187 INFO:tasks.workunit.client.1.smithi186.stdout:5/264: chown d5/db/d15/d2a/d4c 16 1 2024-04-12T20:56:29.192 INFO:tasks.workunit.client.1.smithi186.stdout:5/265: write d5/db/d15/d2a/d2d/f42 [586771,99058] 0 2024-04-12T20:56:29.193 INFO:tasks.workunit.client.1.smithi186.stdout:5/266: fsync f2 0 2024-04-12T20:56:29.196 INFO:tasks.workunit.client.1.smithi186.stdout:5/267: dread - d5/db/f34 zero size 2024-04-12T20:56:29.197 INFO:tasks.workunit.client.1.smithi186.stdout:9/265: rename d3/d2f/d2e/f2a to d3/d2f/d2e/d33/d52/f57 0 2024-04-12T20:56:29.198 INFO:tasks.workunit.client.1.smithi186.stdout:0/334: sync 2024-04-12T20:56:29.202 INFO:tasks.workunit.client.1.smithi186.stdout:8/222: write d7/d18/d36/d17/f1c [2861533,7852] 0 2024-04-12T20:56:29.205 INFO:tasks.workunit.client.1.smithi186.stdout:4/381: dwrite d0/dc/d33/f68 [0,4194304] 0 2024-04-12T20:56:29.205 INFO:tasks.workunit.client.1.smithi186.stdout:9/266: write d3/f3b [427463,129428] 0 2024-04-12T20:56:29.205 INFO:tasks.workunit.client.1.smithi186.stdout:6/243: truncate d9/f47 1254396 0 2024-04-12T20:56:29.205 INFO:tasks.workunit.client.1.smithi186.stdout:0/335: mknod d0/d5c/c6b 0 2024-04-12T20:56:29.206 INFO:tasks.workunit.client.1.smithi186.stdout:5/268: creat d5/f59 x:0 0 0 2024-04-12T20:56:29.207 INFO:tasks.workunit.client.1.smithi186.stdout:8/223: write d7/d18/d36/d1d/d1f/d26/f3f [767735,20747] 0 2024-04-12T20:56:29.209 INFO:tasks.workunit.client.1.smithi186.stdout:4/382: fsync d0/d21/f1b 0 2024-04-12T20:56:29.213 INFO:tasks.workunit.client.1.smithi186.stdout:2/277: write d0/d32/d2a/f43 [699482,60141] 0 2024-04-12T20:56:29.214 INFO:tasks.workunit.client.1.smithi186.stdout:6/244: rmdir d9/d4c 39 2024-04-12T20:56:29.219 INFO:tasks.workunit.client.1.smithi186.stdout:0/336: creat d0/d5d/f6c x:0 0 0 2024-04-12T20:56:29.219 INFO:tasks.workunit.client.1.smithi186.stdout:8/224: mknod d7/de/c44 0 2024-04-12T20:56:29.221 INFO:tasks.workunit.client.1.smithi186.stdout:9/267: rename d3/f3b to d3/d2f/d2e/d43/d4c/f58 0 2024-04-12T20:56:29.224 INFO:tasks.workunit.client.1.smithi186.stdout:0/337: creat d0/d5c/f6d x:0 0 0 2024-04-12T20:56:29.226 INFO:tasks.workunit.client.1.smithi186.stdout:1/266: dread d14/f15 [0,4194304] 0 2024-04-12T20:56:29.229 INFO:tasks.workunit.client.1.smithi186.stdout:0/338: mkdir d0/d14/d4e/d50/d6e 0 2024-04-12T20:56:29.230 INFO:tasks.workunit.client.1.smithi186.stdout:9/268: mknod d3/d2f/d2e/d43/c59 0 2024-04-12T20:56:29.231 INFO:tasks.workunit.client.1.smithi186.stdout:5/269: dwrite d5/db/d15/d2a/d2d/f58 [0,4194304] 0 2024-04-12T20:56:29.231 INFO:tasks.workunit.client.1.smithi186.stdout:7/314: sync 2024-04-12T20:56:29.233 INFO:tasks.workunit.client.1.smithi186.stdout:7/315: dread - d7/f43 zero size 2024-04-12T20:56:29.237 INFO:tasks.workunit.client.1.smithi186.stdout:7/316: stat d7 0 2024-04-12T20:56:29.237 INFO:tasks.workunit.client.1.smithi186.stdout:4/383: rename d0/d21/d3d/d63/f67 to d0/d21/d3d/f7d 0 2024-04-12T20:56:29.238 INFO:tasks.workunit.client.1.smithi186.stdout:5/270: mknod d5/db/d15/d2a/c5a 0 2024-04-12T20:56:29.241 INFO:tasks.workunit.client.1.smithi186.stdout:6/245: dwrite d9/f40 [0,4194304] 0 2024-04-12T20:56:29.244 INFO:tasks.workunit.client.1.smithi186.stdout:8/225: truncate d7/fd 2701179 0 2024-04-12T20:56:29.246 INFO:tasks.workunit.client.1.smithi186.stdout:0/339: rename d0/d10/l3a to d0/d5c/d66/l6f 0 2024-04-12T20:56:29.247 INFO:tasks.workunit.client.1.smithi186.stdout:5/271: truncate d5/db/d14/f16 1030254 0 2024-04-12T20:56:29.248 INFO:tasks.workunit.client.1.smithi186.stdout:9/269: link d3/fd d3/d2f/d2e/d33/d52/f5a 0 2024-04-12T20:56:29.248 INFO:tasks.workunit.client.1.smithi186.stdout:6/246: write d9/d3f/f48 [521421,102395] 0 2024-04-12T20:56:29.249 INFO:tasks.workunit.client.1.smithi186.stdout:8/226: stat d7/d18/d36/d1d/d1f/d26/f3d 0 2024-04-12T20:56:29.250 INFO:tasks.workunit.client.1.smithi186.stdout:2/278: dwrite d0/f4 [0,4194304] 0 2024-04-12T20:56:29.255 INFO:tasks.workunit.client.1.smithi186.stdout:8/227: write d7/d18/d36/d20/d23/d25/d29/f2a [1791250,104358] 0 2024-04-12T20:56:29.256 INFO:tasks.workunit.client.1.smithi186.stdout:5/272: mkdir d5/db/d15/d2a/d4c/d5b 0 2024-04-12T20:56:29.258 INFO:tasks.workunit.client.1.smithi186.stdout:4/384: mknod d0/d21/d3d/c7e 0 2024-04-12T20:56:29.259 INFO:tasks.workunit.client.1.smithi186.stdout:8/228: dread - d7/d18/f43 zero size 2024-04-12T20:56:29.260 INFO:tasks.workunit.client.1.smithi186.stdout:9/270: creat d3/d2f/d54/f5b x:0 0 0 2024-04-12T20:56:29.260 INFO:tasks.workunit.client.1.smithi186.stdout:5/273: write d5/db/fc [5125434,46147] 0 2024-04-12T20:56:29.265 INFO:tasks.workunit.client.1.smithi186.stdout:2/279: symlink d0/l54 0 2024-04-12T20:56:29.266 INFO:tasks.workunit.client.1.smithi186.stdout:5/274: write d5/db/f57 [978733,122431] 0 2024-04-12T20:56:29.267 INFO:tasks.workunit.client.1.smithi186.stdout:9/271: fdatasync d3/d2f/f26 0 2024-04-12T20:56:29.271 INFO:tasks.workunit.client.1.smithi186.stdout:5/275: write d5/db/d15/d2a/f41 [292918,12679] 0 2024-04-12T20:56:29.271 INFO:tasks.workunit.client.1.smithi186.stdout:9/272: truncate d3/d2f/f55 771087 0 2024-04-12T20:56:29.272 INFO:tasks.workunit.client.1.smithi186.stdout:7/317: truncate d7/d10/f15 562349 0 2024-04-12T20:56:29.273 INFO:tasks.workunit.client.1.smithi186.stdout:5/276: fdatasync d5/db/d15/d2a/d2d/f58 0 2024-04-12T20:56:29.274 INFO:tasks.workunit.client.1.smithi186.stdout:0/340: truncate d0/d10/f4d 1684965 0 2024-04-12T20:56:29.276 INFO:tasks.workunit.client.1.smithi186.stdout:4/385: write d0/dc/d33/f1a [2116389,123143] 0 2024-04-12T20:56:29.276 INFO:tasks.workunit.client.1.smithi186.stdout:9/273: write d3/d2f/d2e/d33/d36/f3d [1456472,107326] 0 2024-04-12T20:56:29.278 INFO:tasks.workunit.client.1.smithi186.stdout:4/386: readlink d0/dc/l3c 0 2024-04-12T20:56:29.280 INFO:tasks.workunit.client.1.smithi186.stdout:4/387: stat d0/d21/f1b 0 2024-04-12T20:56:29.280 INFO:tasks.workunit.client.1.smithi186.stdout:8/229: dread d7/d18/d36/d17/f22 [0,4194304] 0 2024-04-12T20:56:29.281 INFO:tasks.workunit.client.1.smithi186.stdout:5/277: mknod d5/db/d15/d2a/d4c/c5c 0 2024-04-12T20:56:29.286 INFO:tasks.workunit.client.1.smithi186.stdout:5/278: chown d5/f59 50061803 1 2024-04-12T20:56:29.287 INFO:tasks.workunit.client.1.smithi186.stdout:5/279: dread - d5/f59 zero size 2024-04-12T20:56:29.291 INFO:tasks.workunit.client.1.smithi186.stdout:8/230: write d7/d18/d36/d17/f22 [3336334,91765] 0 2024-04-12T20:56:29.291 INFO:tasks.workunit.client.1.smithi186.stdout:0/341: truncate d0/d10/f2c 3353617 0 2024-04-12T20:56:29.293 INFO:tasks.workunit.client.1.smithi186.stdout:5/280: mkdir d5/d44/d5d 0 2024-04-12T20:56:29.296 INFO:tasks.workunit.client.1.smithi186.stdout:8/231: creat d7/d18/d36/d20/d23/f45 x:0 0 0 2024-04-12T20:56:29.298 INFO:tasks.workunit.client.1.smithi186.stdout:0/342: creat d0/f70 x:0 0 0 2024-04-12T20:56:29.301 INFO:tasks.workunit.client.1.smithi186.stdout:7/318: dwrite d7/d10/d1c/f2c [4194304,4194304] 0 2024-04-12T20:56:29.302 INFO:tasks.workunit.client.0.smithi018.stdout:2/996: sync 2024-04-12T20:56:29.303 INFO:tasks.workunit.client.1.smithi186.stdout:8/232: dread - d7/d18/d36/d20/d23/d25/f34 zero size 2024-04-12T20:56:29.304 INFO:tasks.workunit.client.1.smithi186.stdout:9/274: dwrite d3/d2f/f1e [0,4194304] 0 2024-04-12T20:56:29.305 INFO:tasks.workunit.client.1.smithi186.stdout:4/388: dread d0/dc/f2f [0,4194304] 0 2024-04-12T20:56:29.306 INFO:tasks.workunit.client.0.smithi018.stdout:2/997: truncate d5/d7/d63/fac 3967070 0 2024-04-12T20:56:29.309 INFO:tasks.workunit.client.0.smithi018.stdout:2/998: read d5/d12/d54/d3e/f102 [125693,43546] 0 2024-04-12T20:56:29.315 INFO:tasks.workunit.client.1.smithi186.stdout:2/280: dread d0/f50 [0,4194304] 0 2024-04-12T20:56:29.316 INFO:tasks.workunit.client.1.smithi186.stdout:7/319: getdents d7/d10/d2d 0 2024-04-12T20:56:29.318 INFO:tasks.workunit.client.0.smithi018.stdout:2/999: rename d5/d8/d10/d43/de2/d24/d39/d9c/f155 to d5/d7/d31/f156 0 2024-04-12T20:56:29.319 INFO:tasks.workunit.client.1.smithi186.stdout:2/281: chown d0/d3/d1a 0 1 2024-04-12T20:56:29.324 INFO:tasks.workunit.client.1.smithi186.stdout:8/233: creat d7/d18/d36/d20/d23/d25/f46 x:0 0 0 2024-04-12T20:56:29.325 INFO:tasks.workunit.client.1.smithi186.stdout:7/320: write d7/d10/d2d/d40/d4a/f51 [170801,45182] 0 2024-04-12T20:56:29.328 INFO:tasks.workunit.client.1.smithi186.stdout:4/389: dread d0/dc/d33/f1a [0,4194304] 0 2024-04-12T20:56:29.328 INFO:tasks.workunit.client.1.smithi186.stdout:8/234: rmdir d7 39 2024-04-12T20:56:29.330 INFO:tasks.workunit.client.1.smithi186.stdout:2/282: dread d0/d32/d2a/f43 [0,4194304] 0 2024-04-12T20:56:29.331 INFO:tasks.workunit.client.1.smithi186.stdout:5/281: dwrite d5/db/d15/d2a/d2d/d46/d55/f37 [0,4194304] 0 2024-04-12T20:56:29.333 INFO:tasks.workunit.client.1.smithi186.stdout:3/289: sync 2024-04-12T20:56:29.336 INFO:tasks.workunit.client.1.smithi186.stdout:5/282: stat d5/db/d15/d2a/d2d/d46/d55/l3c 0 2024-04-12T20:56:29.341 INFO:tasks.workunit.client.1.smithi186.stdout:7/321: rename d7/d10/l49 to d7/d55/d71/l76 0 2024-04-12T20:56:29.342 INFO:tasks.workunit.client.1.smithi186.stdout:9/275: dwrite d3/fd [4194304,4194304] 0 2024-04-12T20:56:29.344 INFO:tasks.workunit.client.1.smithi186.stdout:3/290: creat d1/d3/d49/f59 x:0 0 0 2024-04-12T20:56:29.348 INFO:tasks.workunit.client.1.smithi186.stdout:3/291: read d1/d2c/d48/f4f [1747531,99463] 0 2024-04-12T20:56:29.360 INFO:tasks.workunit.client.0.smithi018.stdout:1/924: sync 2024-04-12T20:56:29.366 INFO:tasks.workunit.client.1.smithi186.stdout:3/292: creat d1/d2c/d48/f5a x:0 0 0 2024-04-12T20:56:29.371 INFO:tasks.workunit.client.0.smithi018.stdout:1/925: dread d3/d9/da/dd/d138/fdf [0,4194304] 0 2024-04-12T20:56:29.375 INFO:tasks.workunit.client.0.smithi018.stdout:1/926: fsync d3/d9/d12c/fdb 0 2024-04-12T20:56:29.376 INFO:tasks.workunit.client.1.smithi186.stdout:6/247: sync 2024-04-12T20:56:29.378 INFO:tasks.workunit.client.1.smithi186.stdout:1/267: sync 2024-04-12T20:56:29.379 INFO:tasks.workunit.client.1.smithi186.stdout:4/390: dwrite d0/dc/f2f [0,4194304] 0 2024-04-12T20:56:29.381 INFO:tasks.workunit.client.1.smithi186.stdout:8/235: dwrite d7/d18/d36/f1b [0,4194304] 0 2024-04-12T20:56:29.381 INFO:tasks.workunit.client.1.smithi186.stdout:2/283: dwrite d0/d3/d1a/d21/f39 [0,4194304] 0 2024-04-12T20:56:29.381 INFO:tasks.workunit.client.0.smithi018.stdout:1/927: symlink d3/d9/da/dd/d31/d32/d13/d19/d2d/d4e/d5e/d97/l139 0 2024-04-12T20:56:29.385 INFO:tasks.workunit.client.0.smithi018.stdout:1/928: mkdir d3/d9/da/dd/d31/d32/d13/d19/d13a 0 2024-04-12T20:56:29.389 INFO:tasks.workunit.client.1.smithi186.stdout:0/343: dread d0/d10/f2c [0,4194304] 0 2024-04-12T20:56:29.390 INFO:tasks.workunit.client.0.smithi018.stdout:1/929: truncate d3/d9/da/dd/d31/d32/d13/d19/f9a 5642942 0 2024-04-12T20:56:29.391 INFO:tasks.workunit.client.1.smithi186.stdout:5/283: dwrite d5/db/d15/f28 [0,4194304] 0 2024-04-12T20:56:29.392 INFO:tasks.workunit.client.1.smithi186.stdout:1/268: readlink d14/l32 0 2024-04-12T20:56:29.393 INFO:tasks.workunit.client.1.smithi186.stdout:3/293: link d1/l16 d1/d3/d49/l5b 0 2024-04-12T20:56:29.395 INFO:tasks.workunit.client.1.smithi186.stdout:7/322: dwrite d7/d10/d1c/f2c [0,4194304] 0 2024-04-12T20:56:29.398 INFO:tasks.workunit.client.1.smithi186.stdout:6/248: mknod d9/d43/d2d/d4d/c58 0 2024-04-12T20:56:29.402 INFO:tasks.workunit.client.1.smithi186.stdout:6/249: dread - d9/f4b zero size 2024-04-12T20:56:29.402 INFO:tasks.workunit.client.1.smithi186.stdout:4/391: mknod d0/c7f 0 2024-04-12T20:56:29.406 INFO:tasks.workunit.client.1.smithi186.stdout:8/236: rename d7/d18/d36/d20/d23 to d7/d18/d36/d20/d23/d25/d47 22 2024-04-12T20:56:29.407 INFO:tasks.workunit.client.0.smithi018.stdout:1/930: dwrite d3/d9/da/dd/d26/fa6 [4194304,4194304] 0 2024-04-12T20:56:29.413 INFO:tasks.workunit.client.1.smithi186.stdout:9/276: dwrite d3/d2f/d2e/d33/d36/f37 [0,4194304] 0 2024-04-12T20:56:29.413 INFO:tasks.workunit.client.1.smithi186.stdout:3/294: getdents d1/d3/d21/d22/d34 0 2024-04-12T20:56:29.413 INFO:tasks.workunit.client.1.smithi186.stdout:9/277: stat d3/d12 0 2024-04-12T20:56:29.413 INFO:tasks.workunit.client.0.smithi018.stdout:1/931: write d3/d9/da/dd/d31/d32/d13/d19/d2d/d4e/d5e/d97/fe4 [5188019,46077] 0 2024-04-12T20:56:29.416 INFO:tasks.workunit.client.1.smithi186.stdout:6/250: dread - d9/d43/d2d/f4e zero size 2024-04-12T20:56:29.418 INFO:tasks.workunit.client.1.smithi186.stdout:7/323: rename d7/d55/f6f to d7/d10/d2d/d40/f77 0 2024-04-12T20:56:29.419 INFO:tasks.workunit.client.0.smithi018.stdout:1/932: read d3/d9/da/dd/f55 [1940924,38586] 0 2024-04-12T20:56:29.426 INFO:tasks.workunit.client.1.smithi186.stdout:8/237: creat d7/d18/d36/d17/f48 x:0 0 0 2024-04-12T20:56:29.430 INFO:tasks.workunit.client.0.smithi018.stdout:1/933: fsync d3/d9/da/dd/d31/f36 0 2024-04-12T20:56:29.430 INFO:tasks.workunit.client.1.smithi186.stdout:6/251: mkdir d9/d43/d59 0 2024-04-12T20:56:29.431 INFO:tasks.workunit.client.0.smithi018.stdout:1/934: dread - d3/d65/fea zero size 2024-04-12T20:56:29.432 INFO:tasks.workunit.client.1.smithi186.stdout:9/278: symlink d3/d2f/d2e/d43/l5c 0 2024-04-12T20:56:29.433 INFO:tasks.workunit.client.1.smithi186.stdout:5/284: truncate d5/fd 4027101 0 2024-04-12T20:56:29.434 INFO:tasks.workunit.client.0.smithi018.stdout:1/935: mkdir d3/db8/d13b 0 2024-04-12T20:56:29.436 INFO:tasks.workunit.client.0.smithi018.stdout:1/936: chown d3/d9/da/dd/d26 8284544 1 2024-04-12T20:56:29.436 INFO:tasks.workunit.client.1.smithi186.stdout:3/295: rename d1/d3/d21/d22/d24/d28/d3d to d1/d3/d21/d5c 0 2024-04-12T20:56:29.437 INFO:tasks.workunit.client.1.smithi186.stdout:7/324: unlink d7/d10/d2d/d40/f4b 0 2024-04-12T20:56:29.438 INFO:tasks.workunit.client.1.smithi186.stdout:9/279: unlink f1 0 2024-04-12T20:56:29.438 INFO:tasks.workunit.client.1.smithi186.stdout:1/269: dwrite d14/d1a/d1d/f29 [0,4194304] 0 2024-04-12T20:56:29.441 INFO:tasks.workunit.client.0.smithi018.stdout:1/937: dread - d3/d9/da/dd/d31/d32/d13/d19/d2d/d4e/d58/f121 zero size 2024-04-12T20:56:29.442 INFO:tasks.workunit.client.1.smithi186.stdout:6/252: creat d9/d3f/f5a x:0 0 0 2024-04-12T20:56:29.445 INFO:tasks.workunit.client.1.smithi186.stdout:8/238: truncate d7/de/d10/f3c 186981 0 2024-04-12T20:56:29.450 INFO:tasks.workunit.client.1.smithi186.stdout:4/392: dwrite d0/dc/d33/f56 [0,4194304] 0 2024-04-12T20:56:29.452 INFO:tasks.workunit.client.1.smithi186.stdout:9/280: symlink d3/d2f/l5d 0 2024-04-12T20:56:29.455 INFO:tasks.workunit.client.1.smithi186.stdout:7/325: write d7/d10/f19 [2140577,80112] 0 2024-04-12T20:56:29.455 INFO:tasks.workunit.client.1.smithi186.stdout:2/284: dwrite d0/d3/d1a/f45 [0,4194304] 0 2024-04-12T20:56:29.457 INFO:tasks.workunit.client.1.smithi186.stdout:6/253: creat d9/d43/d50/f5b x:0 0 0 2024-04-12T20:56:29.458 INFO:tasks.workunit.client.0.smithi018.stdout:1/938: dwrite d3/d9/da/dd/d31/d32/d13/d19/d2d/d4e/d7a/d82/f128 [0,4194304] 0 2024-04-12T20:56:29.459 INFO:tasks.workunit.client.1.smithi186.stdout:0/344: dwrite d0/d14/d4e/d1e/f56 [0,4194304] 0 2024-04-12T20:56:29.462 INFO:tasks.workunit.client.0.smithi018.stdout:1/939: stat d3/d9/da/dd/d31/d32/d13/d19/d2d/d4e/d7a/d82/la3 0 2024-04-12T20:56:29.463 INFO:tasks.workunit.client.1.smithi186.stdout:2/285: chown d0/dc/f4c 436 1 2024-04-12T20:56:29.464 INFO:tasks.workunit.client.1.smithi186.stdout:9/281: creat d3/d2f/f5e x:0 0 0 2024-04-12T20:56:29.465 INFO:tasks.workunit.client.1.smithi186.stdout:8/239: creat d7/d18/d36/d1d/d1f/f49 x:0 0 0 2024-04-12T20:56:29.467 INFO:tasks.workunit.client.1.smithi186.stdout:2/286: stat d0/cd 0 2024-04-12T20:56:29.467 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:56:29 smithi186 ceph-mon[36597]: pgmap v47: 65 pgs: 65 active+clean; 2.3 GiB data, 8.2 GiB used, 528 GiB / 536 GiB avail; 49 MiB/s rd, 150 MiB/s wr, 377 op/s 2024-04-12T20:56:29.470 INFO:tasks.workunit.client.0.smithi018.stdout:1/940: mknod d3/d9/d12c/d73/c13c 0 2024-04-12T20:56:29.471 INFO:tasks.workunit.client.1.smithi186.stdout:6/254: mknod d9/d43/d2d/c5c 0 2024-04-12T20:56:29.473 INFO:tasks.workunit.client.1.smithi186.stdout:4/393: mkdir d0/dc/d33/d5c/d80 0 2024-04-12T20:56:29.480 INFO:tasks.workunit.client.0.smithi018.stdout:1/941: rmdir d3/d9/da/dd/d31/d32/d13/d19/d2d/d4e/d5e 39 2024-04-12T20:56:29.482 INFO:tasks.workunit.client.0.smithi018.stdout:1/942: dread - d3/d9/da/dd/f117 zero size 2024-04-12T20:56:29.483 INFO:tasks.workunit.client.1.smithi186.stdout:2/287: creat d0/dc/f55 x:0 0 0 2024-04-12T20:56:29.484 INFO:tasks.workunit.client.0.smithi018.stdout:1/943: fsync d3/d9/dc3/f115 0 2024-04-12T20:56:29.484 INFO:tasks.workunit.client.1.smithi186.stdout:5/285: dread d5/db/f57 [0,4194304] 0 2024-04-12T20:56:29.484 INFO:tasks.workunit.client.1.smithi186.stdout:9/282: creat d3/d2f/d2e/d33/d52/f5f x:0 0 0 2024-04-12T20:56:29.488 INFO:tasks.workunit.client.0.smithi018.stdout:1/944: mkdir d3/d9/da/dd/d31/d32/d13/d19/d111/d13d 0 2024-04-12T20:56:29.490 INFO:tasks.workunit.client.1.smithi186.stdout:3/296: dwrite d1/f1e [0,4194304] 0 2024-04-12T20:56:29.495 INFO:tasks.workunit.client.1.smithi186.stdout:3/297: write d1/d3/f58 [501532,23472] 0 2024-04-12T20:56:29.495 INFO:tasks.workunit.client.1.smithi186.stdout:7/326: creat d7/d55/d5f/f78 x:0 0 0 2024-04-12T20:56:29.500 INFO:tasks.workunit.client.0.smithi018.stdout:1/945: rename d3/d9/da/dd/d31/d32/d13/d19/d2d/d4e/laa to d3/d9/da/dd/d31/d32/d13/d19/d2d/d4e/d58/d5c/l13e 0 2024-04-12T20:56:29.507 INFO:tasks.workunit.client.0.smithi018.stdout:1/946: creat d3/d9/da/dd/d31/d32/d13/d19/d2d/d4e/d7a/d82/dce/f13f x:0 0 0 2024-04-12T20:56:29.510 INFO:tasks.workunit.client.1.smithi186.stdout:4/394: dread d0/dc/d33/f1a [0,4194304] 0 2024-04-12T20:56:29.510 INFO:tasks.workunit.client.1.smithi186.stdout:1/270: dread d14/d1a/d1d/f23 [0,4194304] 0 2024-04-12T20:56:29.511 INFO:tasks.workunit.client.0.smithi018.stdout:1/947: rmdir d3/d9/da/dd/d31/d32/d13/d19/d2d/d4e/d58/d5c 39 2024-04-12T20:56:29.514 INFO:tasks.workunit.client.1.smithi186.stdout:8/240: getdents d7/d18/d36/d20 0 2024-04-12T20:56:29.519 INFO:tasks.workunit.client.1.smithi186.stdout:1/271: rmdir d14/d1a/d21/d35 39 2024-04-12T20:56:29.522 INFO:tasks.workunit.client.1.smithi186.stdout:6/255: dwrite d9/d43/f45 [0,4194304] 0 2024-04-12T20:56:29.526 INFO:tasks.workunit.client.1.smithi186.stdout:1/272: write d14/d1a/d1d/f46 [288954,79465] 0 2024-04-12T20:56:29.528 INFO:tasks.workunit.client.1.smithi186.stdout:2/288: symlink d0/dc/d2e/l56 0 2024-04-12T20:56:29.531 INFO:tasks.workunit.client.1.smithi186.stdout:8/241: creat d7/d18/d36/d20/f4a x:0 0 0 2024-04-12T20:56:29.531 INFO:tasks.workunit.client.0.smithi018.stdout:1/948: write d3/d9/da/dd/d31/d32/d13/d19/fcf [650096,43494] 0 2024-04-12T20:56:29.532 INFO:tasks.workunit.client.1.smithi186.stdout:5/286: dwrite d5/db/f40 [0,4194304] 0 2024-04-12T20:56:29.534 INFO:tasks.workunit.client.1.smithi186.stdout:2/289: chown d0/d3/d1a/d21/f4b 63647 1 2024-04-12T20:56:29.535 INFO:tasks.workunit.client.1.smithi186.stdout:3/298: truncate d1/d3/da/f47 672325 0 2024-04-12T20:56:29.535 INFO:tasks.workunit.client.1.smithi186.stdout:8/242: readlink d7/lb 0 2024-04-12T20:56:29.538 INFO:tasks.workunit.client.1.smithi186.stdout:6/256: unlink d9/d43/d2d/c36 0 2024-04-12T20:56:29.544 INFO:tasks.workunit.client.1.smithi186.stdout:2/290: mknod d0/d3/d1a/d41/c57 0 2024-04-12T20:56:29.544 INFO:tasks.workunit.client.1.smithi186.stdout:0/345: dread d0/d14/d4e/d1e/f1f [0,4194304] 0 2024-04-12T20:56:29.547 INFO:tasks.workunit.client.1.smithi186.stdout:3/299: creat d1/d3/d21/f5d x:0 0 0 2024-04-12T20:56:29.550 INFO:tasks.workunit.client.1.smithi186.stdout:1/273: creat d14/d1a/d21/f57 x:0 0 0 2024-04-12T20:56:29.553 INFO:tasks.workunit.client.1.smithi186.stdout:7/327: dread d7/f12 [0,4194304] 0 2024-04-12T20:56:29.553 INFO:tasks.workunit.client.1.smithi186.stdout:8/243: rename d7/de/c44 to d7/d18/d36/d1d/d1f/c4b 0 2024-04-12T20:56:29.553 INFO:tasks.workunit.client.1.smithi186.stdout:4/395: getdents d0/dc/d1f/d73 0 2024-04-12T20:56:29.554 INFO:tasks.workunit.client.1.smithi186.stdout:1/274: chown d14/d1a/d21/f31 1 1 2024-04-12T20:56:29.555 INFO:tasks.workunit.client.0.smithi018.stdout:1/949: dwrite d3/d9/d12c/fdb [0,4194304] 0 2024-04-12T20:56:29.555 INFO:tasks.workunit.client.1.smithi186.stdout:2/291: creat d0/d3/f58 x:0 0 0 2024-04-12T20:56:29.558 INFO:tasks.workunit.client.1.smithi186.stdout:0/346: mkdir d0/d10/d71 0 2024-04-12T20:56:29.559 INFO:tasks.workunit.client.0.smithi018.stdout:1/950: creat d3/db8/d13b/f140 x:0 0 0 2024-04-12T20:56:29.560 INFO:tasks.workunit.client.1.smithi186.stdout:3/300: mkdir d1/d3/d21/d5e 0 2024-04-12T20:56:29.561 INFO:tasks.workunit.client.1.smithi186.stdout:8/244: rename d7/d18/d36/d17/f22 to d7/d18/f4c 0 2024-04-12T20:56:29.561 INFO:tasks.workunit.client.1.smithi186.stdout:7/328: readlink d7/d55/d71/l76 0 2024-04-12T20:56:29.562 INFO:tasks.workunit.client.1.smithi186.stdout:1/275: read d14/d1a/f37 [5207,90771] 0 2024-04-12T20:56:29.562 INFO:tasks.workunit.client.1.smithi186.stdout:0/347: mkdir d0/d10/d72 0 2024-04-12T20:56:29.566 INFO:tasks.workunit.client.1.smithi186.stdout:7/329: fsync d7/d10/d2d/d67/f70 0 2024-04-12T20:56:29.570 INFO:tasks.workunit.client.1.smithi186.stdout:7/330: dread d7/d10/d1c/f52 [0,4194304] 0 2024-04-12T20:56:29.572 INFO:tasks.workunit.client.1.smithi186.stdout:0/348: mknod d0/d10/c73 0 2024-04-12T20:56:29.577 INFO:tasks.workunit.client.0.smithi018.stdout:1/951: dwrite d3/d9/d12c/d73/fd0 [4194304,4194304] 0 2024-04-12T20:56:29.577 INFO:tasks.workunit.client.1.smithi186.stdout:0/349: rmdir d0/d5c 39 2024-04-12T20:56:29.580 INFO:tasks.workunit.client.1.smithi186.stdout:8/245: link d7/de/d10/f3c d7/d18/d36/d1d/f4d 0 2024-04-12T20:56:29.581 INFO:tasks.workunit.client.1.smithi186.stdout:0/350: write d0/d10/d32/f3d [1728585,31742] 0 2024-04-12T20:56:29.583 INFO:tasks.workunit.client.0.smithi018.stdout:1/952: mknod d3/d9/da/c141 0 2024-04-12T20:56:29.583 INFO:tasks.workunit.client.1.smithi186.stdout:1/276: rmdir d14 39 2024-04-12T20:56:29.584 INFO:tasks.workunit.client.0.smithi018.stdout:1/953: chown d3/d9/da/dd/d31/d32 234 1 2024-04-12T20:56:29.586 INFO:tasks.workunit.client.1.smithi186.stdout:5/287: write d5/db/d15/d2a/d2d/d48/f4b [2343686,5025] 0 2024-04-12T20:56:29.588 INFO:tasks.workunit.client.1.smithi186.stdout:5/288: fsync d5/db/f34 0 2024-04-12T20:56:29.590 INFO:tasks.workunit.client.1.smithi186.stdout:5/289: readlink d5/db/d33/l45 0 2024-04-12T20:56:29.592 INFO:tasks.workunit.client.1.smithi186.stdout:9/283: dread d3/d2f/f21 [0,4194304] 0 2024-04-12T20:56:29.599 INFO:tasks.workunit.client.0.smithi018.stdout:1/954: dread d3/d9/da/dd/d31/d32/d13/d19/d2d/d4e/d7a/f104 [0,4194304] 0 2024-04-12T20:56:29.599 INFO:tasks.workunit.client.1.smithi186.stdout:0/351: mknod d0/d5c/d65/c74 0 2024-04-12T20:56:29.600 INFO:tasks.workunit.client.1.smithi186.stdout:1/277: mkdir d14/d1a/d58 0 2024-04-12T20:56:29.601 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:56:29 smithi018 ceph-mon[28799]: pgmap v47: 65 pgs: 65 active+clean; 2.3 GiB data, 8.2 GiB used, 528 GiB / 536 GiB avail; 49 MiB/s rd, 150 MiB/s wr, 377 op/s 2024-04-12T20:56:29.601 INFO:tasks.workunit.client.0.smithi018.stdout:1/955: stat d3/d9/f3c 0 2024-04-12T20:56:29.603 INFO:tasks.workunit.client.0.smithi018.stdout:1/956: chown d3/d65/l10f 122724182 1 2024-04-12T20:56:29.605 INFO:tasks.workunit.client.1.smithi186.stdout:8/246: truncate d7/de/d10/f3c 151523 0 2024-04-12T20:56:29.605 INFO:tasks.workunit.client.1.smithi186.stdout:0/352: dread - d0/d5c/f6d zero size 2024-04-12T20:56:29.606 INFO:tasks.workunit.client.0.smithi018.stdout:1/957: creat d3/d9/d12c/d113/f142 x:0 0 0 2024-04-12T20:56:29.606 INFO:tasks.workunit.client.1.smithi186.stdout:7/331: truncate d7/d10/d2d/d40/d4a/f50 321465 0 2024-04-12T20:56:29.607 INFO:tasks.workunit.client.0.smithi018.stdout:1/958: dread - d3/d65/f134 zero size 2024-04-12T20:56:29.608 INFO:tasks.workunit.client.1.smithi186.stdout:5/290: symlink d5/db/d15/l5e 0 2024-04-12T20:56:29.608 INFO:tasks.workunit.client.1.smithi186.stdout:0/353: dread - d0/d10/d32/f48 zero size 2024-04-12T20:56:29.609 INFO:tasks.workunit.client.0.smithi018.stdout:1/959: chown d3/d9/da/dd/d26/df0/d66/f123 20363 1 2024-04-12T20:56:29.612 INFO:tasks.workunit.client.1.smithi186.stdout:1/278: mkdir d14/d1a/d21/d35/d59 0 2024-04-12T20:56:29.612 INFO:tasks.workunit.client.1.smithi186.stdout:6/257: dread d9/f37 [0,4194304] 0 2024-04-12T20:56:29.613 INFO:tasks.workunit.client.1.smithi186.stdout:9/284: creat d3/d2f/d54/f60 x:0 0 0 2024-04-12T20:56:29.614 INFO:tasks.workunit.client.1.smithi186.stdout:2/292: dwrite d0/dc/f40 [0,4194304] 0 2024-04-12T20:56:29.614 INFO:tasks.workunit.client.0.smithi018.stdout:1/960: rename d3/d9/d12c/d73/cd5 to d3/d9/dc3/d10b/c143 0 2024-04-12T20:56:29.617 INFO:tasks.workunit.client.1.smithi186.stdout:7/332: mknod d7/d10/d2d/d40/c79 0 2024-04-12T20:56:29.618 INFO:tasks.workunit.client.0.smithi018.stdout:1/961: fsync d3/d9/da/dd/d31/d32/d13/d19/d2d/d4e/d7a/df5/f114 0 2024-04-12T20:56:29.620 INFO:tasks.workunit.client.1.smithi186.stdout:1/279: chown cc 63810433 1 2024-04-12T20:56:29.627 INFO:tasks.workunit.client.1.smithi186.stdout:9/285: read d3/d2f/d2e/d43/d4c/f58 [429738,57271] 0 2024-04-12T20:56:29.628 INFO:tasks.workunit.client.1.smithi186.stdout:6/258: read d9/f10 [190265,38127] 0 2024-04-12T20:56:29.628 INFO:tasks.workunit.client.1.smithi186.stdout:7/333: mknod d7/d10/d2d/c7a 0 2024-04-12T20:56:29.631 INFO:tasks.workunit.client.1.smithi186.stdout:6/259: fdatasync d9/d3f/f53 0 2024-04-12T20:56:29.631 INFO:tasks.workunit.client.1.smithi186.stdout:0/354: truncate d0/f4f 95886 0 2024-04-12T20:56:29.633 INFO:tasks.workunit.client.1.smithi186.stdout:5/291: creat d5/db/f5f x:0 0 0 2024-04-12T20:56:29.634 INFO:tasks.workunit.client.1.smithi186.stdout:7/334: mkdir d7/d10/d1c/d5d/d7b 0 2024-04-12T20:56:29.635 INFO:tasks.workunit.client.1.smithi186.stdout:0/355: fsync d0/d14/d4e/d1e/f2f 0 2024-04-12T20:56:29.639 INFO:tasks.workunit.client.0.smithi018.stdout:1/962: dwrite d3/d9/da/dd/d31/d32/d13/d19/d2d/d4e/f8c [0,4194304] 0 2024-04-12T20:56:29.641 INFO:tasks.workunit.client.1.smithi186.stdout:2/293: dread d0/d3/d1a/f45 [0,4194304] 0 2024-04-12T20:56:29.645 INFO:tasks.workunit.client.1.smithi186.stdout:9/286: dread d3/d2f/d2e/d33/d52/f5a [4194304,4194304] 0 2024-04-12T20:56:29.645 INFO:tasks.workunit.client.0.smithi018.stdout:1/963: mknod d3/d9/da/dd/d31/d32/d13/d19/d2d/d4e/d7a/d82/dce/c144 0 2024-04-12T20:56:29.647 INFO:tasks.workunit.client.1.smithi186.stdout:5/292: creat d5/d27/f60 x:0 0 0 2024-04-12T20:56:29.650 INFO:tasks.workunit.client.1.smithi186.stdout:6/260: truncate d9/f22 4107284 0 2024-04-12T20:56:29.651 INFO:tasks.workunit.client.0.smithi018.stdout:1/964: write d3/d9/da/dd/d31/d32/d33/d80/fca [2710907,15043] 0 2024-04-12T20:56:29.652 INFO:tasks.workunit.client.1.smithi186.stdout:1/280: dwrite d14/f16 [4194304,4194304] 0 2024-04-12T20:56:29.654 INFO:tasks.workunit.client.1.smithi186.stdout:7/335: rmdir d7/d10/d2d/d40/d4a 39 2024-04-12T20:56:29.656 INFO:tasks.workunit.client.0.smithi018.stdout:1/965: symlink d3/d9/da/dd/d31/d32/d33/d47/db0/l145 0 2024-04-12T20:56:29.658 INFO:tasks.workunit.client.1.smithi186.stdout:8/247: dread d7/de/d10/f27 [0,4194304] 0 2024-04-12T20:56:29.661 INFO:tasks.workunit.client.1.smithi186.stdout:6/261: rename d9/d3f to d9/d43/d59/d5d 0 2024-04-12T20:56:29.664 INFO:tasks.workunit.client.1.smithi186.stdout:0/356: dwrite d0/d5d/f6c [0,4194304] 0 2024-04-12T20:56:29.668 INFO:tasks.workunit.client.1.smithi186.stdout:8/248: write d7/d18/f1e [1124079,115068] 0 2024-04-12T20:56:29.672 INFO:tasks.workunit.client.1.smithi186.stdout:0/357: chown d0/cb 38 1 2024-04-12T20:56:29.673 INFO:tasks.workunit.client.1.smithi186.stdout:7/336: fdatasync d7/d10/d2d/d40/d4a/f50 0 2024-04-12T20:56:29.673 INFO:tasks.workunit.client.1.smithi186.stdout:8/249: stat d7/de/d10/f3c 0 2024-04-12T20:56:29.675 INFO:tasks.workunit.client.1.smithi186.stdout:2/294: dwrite d0/dc/f55 [0,4194304] 0 2024-04-12T20:56:29.678 INFO:tasks.workunit.client.1.smithi186.stdout:0/358: chown d0/d14/d4e/c55 929975 1 2024-04-12T20:56:29.683 INFO:tasks.workunit.client.1.smithi186.stdout:0/359: mknod d0/d14/c75 0 2024-04-12T20:56:29.683 INFO:tasks.workunit.client.1.smithi186.stdout:7/337: write d7/d10/d2d/f37 [5907181,10136] 0 2024-04-12T20:56:29.684 INFO:tasks.workunit.client.1.smithi186.stdout:2/295: dread - d0/d3/d1a/d24/f33 zero size 2024-04-12T20:56:29.684 INFO:tasks.workunit.client.1.smithi186.stdout:8/250: truncate d7/d18/d36/d1d/f4d 1012788 0 2024-04-12T20:56:29.686 INFO:tasks.workunit.client.1.smithi186.stdout:0/360: fsync d0/f57 0 2024-04-12T20:56:29.688 INFO:tasks.workunit.client.1.smithi186.stdout:8/251: chown d7/de/d10/l3e 5633269 1 2024-04-12T20:56:29.690 INFO:tasks.workunit.client.1.smithi186.stdout:2/296: write d0/f10 [3613693,50542] 0 2024-04-12T20:56:29.694 INFO:tasks.workunit.client.1.smithi186.stdout:7/338: truncate d7/d10/d2d/d67/f70 161789 0 2024-04-12T20:56:29.697 INFO:tasks.workunit.client.1.smithi186.stdout:7/339: chown d7/d10/d1c/c4f 4 1 2024-04-12T20:56:29.698 INFO:tasks.workunit.client.1.smithi186.stdout:8/252: creat d7/de/d10/d2f/f4e x:0 0 0 2024-04-12T20:56:29.701 INFO:tasks.workunit.client.1.smithi186.stdout:2/297: link d0/f2 d0/d3/d1a/d27/f59 0 2024-04-12T20:56:29.702 INFO:tasks.workunit.client.1.smithi186.stdout:8/253: creat d7/de/d10/d2f/f4f x:0 0 0 2024-04-12T20:56:29.704 INFO:tasks.workunit.client.1.smithi186.stdout:2/298: dread - d0/d3/d1a/d24/f33 zero size 2024-04-12T20:56:29.705 INFO:tasks.workunit.client.1.smithi186.stdout:7/340: write d7/d10/d2d/f44 [4369813,65723] 0 2024-04-12T20:56:29.707 INFO:tasks.workunit.client.1.smithi186.stdout:9/287: dread d3/d13/f28 [0,4194304] 0 2024-04-12T20:56:29.707 INFO:tasks.workunit.client.1.smithi186.stdout:1/281: dread f4 [0,4194304] 0 2024-04-12T20:56:29.708 INFO:tasks.workunit.client.1.smithi186.stdout:8/254: write d7/d18/d36/d20/f4a [333414,37286] 0 2024-04-12T20:56:29.710 INFO:tasks.workunit.client.1.smithi186.stdout:7/341: write d7/d10/f13 [2837650,28700] 0 2024-04-12T20:56:29.710 INFO:tasks.workunit.client.1.smithi186.stdout:2/299: symlink d0/d3/d1a/d21/d22/l5a 0 2024-04-12T20:56:29.711 INFO:tasks.workunit.client.0.smithi018.stdout:1/966: dread d3/d9/f18 [0,4194304] 0 2024-04-12T20:56:29.714 INFO:tasks.workunit.client.1.smithi186.stdout:7/342: stat d7/d10/d2d/c7a 0 2024-04-12T20:56:29.716 INFO:tasks.workunit.client.1.smithi186.stdout:9/288: unlink d3/d2f/l5d 0 2024-04-12T20:56:29.718 INFO:tasks.workunit.client.1.smithi186.stdout:9/289: chown d3/d12/c1a 63 1 2024-04-12T20:56:29.719 INFO:tasks.workunit.client.1.smithi186.stdout:7/343: symlink d7/d55/d5f/l7c 0 2024-04-12T20:56:29.722 INFO:tasks.workunit.client.1.smithi186.stdout:9/290: read - d3/d2f/d2e/d33/d36/f53 zero size 2024-04-12T20:56:29.723 INFO:tasks.workunit.client.1.smithi186.stdout:6/262: dread d9/f2e [0,4194304] 0 2024-04-12T20:56:29.727 INFO:tasks.workunit.client.1.smithi186.stdout:8/255: dread d7/de/f21 [0,4194304] 0 2024-04-12T20:56:29.727 INFO:tasks.workunit.client.1.smithi186.stdout:9/291: mkdir d3/d2f/d2e/d33/d61 0 2024-04-12T20:56:29.727 INFO:tasks.workunit.client.0.smithi018.stdout:1/967: dwrite d3/d9/da/dd/d26/fa6 [4194304,4194304] 0 2024-04-12T20:56:29.729 INFO:tasks.workunit.client.1.smithi186.stdout:6/263: mkdir d9/d5e 0 2024-04-12T20:56:29.731 INFO:tasks.workunit.client.1.smithi186.stdout:1/282: dwrite d14/d1a/d3c/f47 [0,4194304] 0 2024-04-12T20:56:29.732 INFO:tasks.workunit.client.1.smithi186.stdout:9/292: read d3/d2f/f1e [92323,113973] 0 2024-04-12T20:56:29.733 INFO:tasks.workunit.client.0.smithi018.stdout:1/968: creat d3/d9/da/dd/f146 x:0 0 0 2024-04-12T20:56:29.734 INFO:tasks.workunit.client.0.smithi018.stdout:1/969: fdatasync d3/d9/da/f133 0 2024-04-12T20:56:29.735 INFO:tasks.workunit.client.1.smithi186.stdout:6/264: creat d9/d43/d2d/f5f x:0 0 0 2024-04-12T20:56:29.736 INFO:tasks.workunit.client.1.smithi186.stdout:1/283: write d14/d1a/d21/f4b [1032542,43579] 0 2024-04-12T20:56:29.749 INFO:tasks.workunit.client.0.smithi018.stdout:1/970: dread d3/d9/da/dd/d31/f35 [0,4194304] 0 2024-04-12T20:56:29.751 INFO:tasks.workunit.client.1.smithi186.stdout:8/256: dwrite d7/de/f14 [0,4194304] 0 2024-04-12T20:56:29.754 INFO:tasks.workunit.client.1.smithi186.stdout:6/265: dread d9/f2a [0,4194304] 0 2024-04-12T20:56:29.755 INFO:tasks.workunit.client.0.smithi018.stdout:1/971: readlink d3/d9/da/dd/d31/d32/d13/d19/d2d/d4e/d5e/da0/d12f/lda 0 2024-04-12T20:56:29.756 INFO:tasks.workunit.client.1.smithi186.stdout:9/293: mknod d3/d2f/d2e/d33/d61/c62 0 2024-04-12T20:56:29.757 INFO:tasks.workunit.client.0.smithi018.stdout:1/972: write d3/d65/f120 [1127280,46713] 0 2024-04-12T20:56:29.759 INFO:tasks.workunit.client.1.smithi186.stdout:7/344: dread d7/f16 [0,4194304] 0 2024-04-12T20:56:29.760 INFO:tasks.workunit.client.1.smithi186.stdout:9/294: write d3/d2f/d2e/d33/d36/f37 [2396501,107990] 0 2024-04-12T20:56:29.765 INFO:tasks.workunit.client.1.smithi186.stdout:6/266: symlink d9/l60 0 2024-04-12T20:56:29.768 INFO:tasks.workunit.client.1.smithi186.stdout:9/295: truncate d3/d2f/f55 1360059 0 2024-04-12T20:56:29.768 INFO:tasks.workunit.client.1.smithi186.stdout:1/284: dwrite d14/d1a/d3c/f47 [4194304,4194304] 0 2024-04-12T20:56:29.769 INFO:tasks.workunit.client.0.smithi018.stdout:1/973: rename d3/d9/da/dd/d26/c88 to d3/d9/da/dd/c147 0 2024-04-12T20:56:29.769 INFO:tasks.workunit.client.1.smithi186.stdout:6/267: creat d9/d4c/f61 x:0 0 0 2024-04-12T20:56:29.772 INFO:tasks.workunit.client.1.smithi186.stdout:1/285: creat d14/d1a/f5a x:0 0 0 2024-04-12T20:56:29.774 INFO:tasks.workunit.client.1.smithi186.stdout:6/268: symlink d9/d43/d59/l62 0 2024-04-12T20:56:29.775 INFO:tasks.workunit.client.1.smithi186.stdout:9/296: fsync d3/d2f/d2e/d33/d36/f53 0 2024-04-12T20:56:29.782 INFO:tasks.workunit.client.0.smithi018.stdout:1/974: write d3/d9/f18 [479983,38632] 0 2024-04-12T20:56:29.784 INFO:tasks.workunit.client.1.smithi186.stdout:6/269: truncate d9/f2e 14482 0 2024-04-12T20:56:29.787 INFO:tasks.workunit.client.0.smithi018.stdout:1/975: chown d3/d9/da/dd/d31/d32/d33/l74 34643 1 2024-04-12T20:56:29.787 INFO:tasks.workunit.client.1.smithi186.stdout:6/270: truncate d9/f33 775228 0 2024-04-12T20:56:29.788 INFO:tasks.workunit.client.0.smithi018.stdout:1/976: dread - d3/f10e zero size 2024-04-12T20:56:29.789 INFO:tasks.workunit.client.1.smithi186.stdout:1/286: dread ff [0,4194304] 0 2024-04-12T20:56:29.790 INFO:tasks.workunit.client.1.smithi186.stdout:1/287: chown f4 898183 1 2024-04-12T20:56:29.791 INFO:tasks.workunit.client.0.smithi018.stdout:1/977: fdatasync d3/d9/da/dd/d31/d32/d13/d19/f62 0 2024-04-12T20:56:29.792 INFO:tasks.workunit.client.1.smithi186.stdout:6/271: rename d9/l1e to d9/d4c/l63 0 2024-04-12T20:56:29.792 INFO:tasks.workunit.client.1.smithi186.stdout:8/257: truncate d7/d18/d36/d20/d23/d25/d2e/f3b 977602 0 2024-04-12T20:56:29.794 INFO:tasks.workunit.client.0.smithi018.stdout:1/978: mkdir d3/d9/da/d12b/d148 0 2024-04-12T20:56:29.795 INFO:tasks.workunit.client.0.smithi018.stdout:1/979: dread - d3/d9/da/dd/f146 zero size 2024-04-12T20:56:29.796 INFO:tasks.workunit.client.1.smithi186.stdout:6/272: truncate d9/d43/d50/f55 300878 0 2024-04-12T20:56:29.796 INFO:tasks.workunit.client.1.smithi186.stdout:1/288: rename d14/d1a/d21/d2c to d14/d1a/d33/d36/d5b 0 2024-04-12T20:56:29.797 INFO:tasks.workunit.client.1.smithi186.stdout:9/297: dwrite d3/ff [0,4194304] 0 2024-04-12T20:56:29.797 INFO:tasks.workunit.client.1.smithi186.stdout:6/273: write d9/f1b [1369818,1874] 0 2024-04-12T20:56:29.799 INFO:tasks.workunit.client.1.smithi186.stdout:8/258: link d7/d18/d36/d20/d23/f45 d7/d18/d36/d1d/d1f/d26/f50 0 2024-04-12T20:56:29.803 INFO:tasks.workunit.client.1.smithi186.stdout:6/274: unlink d9/f2a 0 2024-04-12T20:56:29.805 INFO:tasks.workunit.client.1.smithi186.stdout:8/259: creat d7/de/d10/f51 x:0 0 0 2024-04-12T20:56:29.808 INFO:tasks.workunit.client.0.smithi018.stdout:1/980: dread d3/d9/da/dd/d31/d32/d13/d19/f62 [0,4194304] 0 2024-04-12T20:56:29.812 INFO:tasks.workunit.client.1.smithi186.stdout:3/301: sync 2024-04-12T20:56:29.814 INFO:tasks.workunit.client.1.smithi186.stdout:1/289: dwrite f9 [0,4194304] 0 2024-04-12T20:56:29.818 INFO:tasks.workunit.client.1.smithi186.stdout:3/302: write d1/d3/d21/d5c/f40 [1103358,82109] 0 2024-04-12T20:56:29.822 INFO:tasks.workunit.client.1.smithi186.stdout:6/275: write d9/f12 [2447928,120590] 0 2024-04-12T20:56:29.825 INFO:tasks.workunit.client.0.smithi018.stdout:1/981: dwrite d3/d9/da/fae [0,4194304] 0 2024-04-12T20:56:29.826 INFO:tasks.workunit.client.1.smithi186.stdout:3/303: creat d1/d2c/d48/f5f x:0 0 0 2024-04-12T20:56:29.827 INFO:tasks.workunit.client.1.smithi186.stdout:9/298: dwrite d3/f10 [8388608,4194304] 0 2024-04-12T20:56:29.831 INFO:tasks.workunit.client.0.smithi018.stdout:1/982: creat d3/d9/da/dd/d31/d32/d33/d80/f149 x:0 0 0 2024-04-12T20:56:29.833 INFO:tasks.workunit.client.1.smithi186.stdout:1/290: getdents d14/d1a/d21/d35/d59 0 2024-04-12T20:56:29.833 INFO:tasks.workunit.client.1.smithi186.stdout:6/276: creat d9/d43/d2d/d4d/f64 x:0 0 0 2024-04-12T20:56:29.833 INFO:tasks.workunit.client.1.smithi186.stdout:8/260: dwrite d7/d18/f4c [0,4194304] 0 2024-04-12T20:56:29.833 INFO:tasks.workunit.client.1.smithi186.stdout:6/277: readlink d9/lc 0 2024-04-12T20:56:29.835 INFO:tasks.workunit.client.1.smithi186.stdout:9/299: fsync d3/d13/f28 0 2024-04-12T20:56:29.835 INFO:tasks.workunit.client.1.smithi186.stdout:3/304: unlink d1/d3/d21/d22/d24/d28/c39 0 2024-04-12T20:56:29.836 INFO:tasks.workunit.client.0.smithi018.stdout:1/983: link d3/d9/da/dd/d26/df0/f106 d3/db8/f14a 0 2024-04-12T20:56:29.836 INFO:tasks.workunit.client.1.smithi186.stdout:6/278: dread - d9/d43/d50/f54 zero size 2024-04-12T20:56:29.837 INFO:tasks.workunit.client.1.smithi186.stdout:1/291: mknod d14/d1a/d3c/c5c 0 2024-04-12T20:56:29.842 INFO:tasks.workunit.client.1.smithi186.stdout:8/261: mknod d7/d18/d36/d1d/d1f/d26/c52 0 2024-04-12T20:56:29.846 INFO:tasks.workunit.client.1.smithi186.stdout:8/262: dread - d7/d18/d36/d1d/d1f/d26/f50 zero size 2024-04-12T20:56:29.850 INFO:tasks.workunit.client.1.smithi186.stdout:8/263: mkdir d7/d18/d53 0 2024-04-12T20:56:29.851 INFO:tasks.workunit.client.1.smithi186.stdout:9/300: link d3/l34 d3/d12/l63 0 2024-04-12T20:56:29.854 INFO:tasks.workunit.client.1.smithi186.stdout:8/264: stat d7/d18/d36/d1d/d1f/c4b 0 2024-04-12T20:56:29.854 INFO:tasks.workunit.client.1.smithi186.stdout:9/301: chown d3/d2f/l2b 178138403 1 2024-04-12T20:56:29.856 INFO:tasks.workunit.client.1.smithi186.stdout:9/302: stat d3/d13/c24 0 2024-04-12T20:56:29.859 INFO:tasks.workunit.client.1.smithi186.stdout:9/303: mknod d3/d12/c64 0 2024-04-12T20:56:29.860 INFO:tasks.workunit.client.1.smithi186.stdout:8/265: truncate d7/de/d10/d2f/f4f 497803 0 2024-04-12T20:56:29.860 INFO:tasks.workunit.client.1.smithi186.stdout:3/305: dwrite d1/d3/d49/f59 [0,4194304] 0 2024-04-12T20:56:29.866 INFO:tasks.workunit.client.1.smithi186.stdout:6/279: dwrite d9/d43/f4f [0,4194304] 0 2024-04-12T20:56:29.869 INFO:tasks.workunit.client.1.smithi186.stdout:6/280: truncate d9/f4b 204484 0 2024-04-12T20:56:29.872 INFO:tasks.workunit.client.1.smithi186.stdout:6/281: truncate d9/d43/d59/d5d/f5a 28501 0 2024-04-12T20:56:29.872 INFO:tasks.workunit.client.1.smithi186.stdout:9/304: link d3/d2f/d2e/d33/d52/f5f d3/d13/f65 0 2024-04-12T20:56:29.874 INFO:tasks.workunit.client.1.smithi186.stdout:6/282: chown d9/d43/d59/l62 11099 1 2024-04-12T20:56:29.880 INFO:tasks.workunit.client.1.smithi186.stdout:1/292: truncate d14/d1a/d1d/f23 2324657 0 2024-04-12T20:56:29.882 INFO:tasks.workunit.client.1.smithi186.stdout:9/305: rename d3/d2f/d2e/d33/d52 to d3/d2f/d2e/d33/d66 0 2024-04-12T20:56:29.885 INFO:tasks.workunit.client.1.smithi186.stdout:6/283: dread d9/f10 [0,4194304] 0 2024-04-12T20:56:29.885 INFO:tasks.workunit.client.1.smithi186.stdout:1/293: mknod d14/d1a/d58/c5d 0 2024-04-12T20:56:29.886 INFO:tasks.workunit.client.1.smithi186.stdout:9/306: creat d3/d12/d27/f67 x:0 0 0 2024-04-12T20:56:29.887 INFO:tasks.workunit.client.1.smithi186.stdout:6/284: chown d9/d43/c2b 881494 1 2024-04-12T20:56:29.889 INFO:tasks.workunit.client.1.smithi186.stdout:6/285: write d9/d43/d50/f55 [1164877,86798] 0 2024-04-12T20:56:29.890 INFO:tasks.workunit.client.1.smithi186.stdout:9/307: unlink d3/d13/c24 0 2024-04-12T20:56:29.891 INFO:tasks.workunit.client.1.smithi186.stdout:1/294: rename d14/d1a/d33/l39 to d14/d1a/d33/d36/l5e 0 2024-04-12T20:56:29.893 INFO:tasks.workunit.client.1.smithi186.stdout:8/266: dread d7/de/f11 [0,4194304] 0 2024-04-12T20:56:29.893 INFO:tasks.workunit.client.1.smithi186.stdout:9/308: dread - d3/d2f/d2e/d43/d4c/f51 zero size 2024-04-12T20:56:29.897 INFO:tasks.workunit.client.1.smithi186.stdout:6/286: dread d9/d43/d50/f55 [0,4194304] 0 2024-04-12T20:56:29.899 INFO:tasks.workunit.client.1.smithi186.stdout:8/267: link f5 d7/de/d10/f54 0 2024-04-12T20:56:29.900 INFO:tasks.workunit.client.1.smithi186.stdout:9/309: creat d3/d2f/f68 x:0 0 0 2024-04-12T20:56:29.902 INFO:tasks.workunit.client.1.smithi186.stdout:8/268: mknod d7/de/c55 0 2024-04-12T20:56:29.905 INFO:tasks.workunit.client.1.smithi186.stdout:1/295: dwrite d14/d1a/f3f [0,4194304] 0 2024-04-12T20:56:29.906 INFO:tasks.workunit.client.1.smithi186.stdout:9/310: unlink d3/d2f/d2e/d33/c3c 0 2024-04-12T20:56:29.907 INFO:tasks.workunit.client.1.smithi186.stdout:8/269: unlink d7/de/d10/d2f/l38 0 2024-04-12T20:56:29.907 INFO:tasks.workunit.client.1.smithi186.stdout:1/296: truncate ff 2212380 0 2024-04-12T20:56:29.908 INFO:tasks.workunit.client.1.smithi186.stdout:9/311: fsync d3/ff 0 2024-04-12T20:56:29.909 INFO:tasks.workunit.client.1.smithi186.stdout:1/297: rmdir d14 39 2024-04-12T20:56:29.910 INFO:tasks.workunit.client.1.smithi186.stdout:8/270: write d7/d18/d36/fa [3566903,54603] 0 2024-04-12T20:56:29.913 INFO:tasks.workunit.client.1.smithi186.stdout:9/312: write d3/d2f/d2e/f41 [335079,99010] 0 2024-04-12T20:56:29.913 INFO:tasks.workunit.client.1.smithi186.stdout:1/298: dread - d14/f4f zero size 2024-04-12T20:56:29.915 INFO:tasks.workunit.client.1.smithi186.stdout:9/313: chown d3/c47 503780 1 2024-04-12T20:56:29.920 INFO:tasks.workunit.client.1.smithi186.stdout:8/271: unlink d7/d18/d36/f37 0 2024-04-12T20:56:29.924 INFO:tasks.workunit.client.1.smithi186.stdout:6/287: write d9/f47 [992736,52032] 0 2024-04-12T20:56:29.925 INFO:tasks.workunit.client.1.smithi186.stdout:9/314: rename d3/c20 to d3/d2f/d2e/c69 0 2024-04-12T20:56:29.926 INFO:tasks.workunit.client.1.smithi186.stdout:9/315: readlink l0 0 2024-04-12T20:56:29.927 INFO:tasks.workunit.client.1.smithi186.stdout:9/316: chown d3/d12/d27 446104 1 2024-04-12T20:56:29.927 INFO:tasks.workunit.client.1.smithi186.stdout:6/288: mknod d9/d43/d59/c65 0 2024-04-12T20:56:29.928 INFO:tasks.workunit.client.1.smithi186.stdout:8/272: creat d7/d18/d36/d20/d23/d25/f56 x:0 0 0 2024-04-12T20:56:29.932 INFO:tasks.workunit.client.1.smithi186.stdout:9/317: rename d3/d12/f1c to d3/d2f/d54/f6a 0 2024-04-12T20:56:29.933 INFO:tasks.workunit.client.1.smithi186.stdout:8/273: creat d7/d18/d36/d20/d23/d25/d29/f57 x:0 0 0 2024-04-12T20:56:29.935 INFO:tasks.workunit.client.1.smithi186.stdout:9/318: mkdir d3/d2f/d2e/d33/d66/d6b 0 2024-04-12T20:56:29.935 INFO:tasks.workunit.client.0.smithi018.stdout:1/984: dwrite d3/d9/da/dd/d31/d32/d13/d19/d2d/d4e/d5e/da0/d12f/fe1 [0,4194304] 0 2024-04-12T20:56:29.944 INFO:tasks.workunit.client.1.smithi186.stdout:6/289: dwrite d9/d4c/f61 [0,4194304] 0 2024-04-12T20:56:29.957 INFO:tasks.workunit.client.0.smithi018.stdout:1/985: dwrite d3/d9/da/dd/d31/d32/d13/d19/d2d/f109 [0,4194304] 0 2024-04-12T20:56:29.960 INFO:tasks.workunit.client.1.smithi186.stdout:9/319: dwrite d3/d2f/d2e/d33/d36/f53 [0,4194304] 0 2024-04-12T20:56:29.963 INFO:tasks.workunit.client.1.smithi186.stdout:8/274: dwrite d7/de/d10/f15 [4194304,4194304] 0 2024-04-12T20:56:29.966 INFO:tasks.workunit.client.1.smithi186.stdout:9/320: write d3/d2f/d54/f6a [450610,85841] 0 2024-04-12T20:56:29.966 INFO:tasks.workunit.client.0.smithi018.stdout:1/986: mknod d3/d9/da/dd/d31/d32/c14b 0 2024-04-12T20:56:29.967 INFO:tasks.workunit.client.1.smithi186.stdout:6/290: dread d9/d43/f45 [0,4194304] 0 2024-04-12T20:56:29.970 INFO:tasks.workunit.client.0.smithi018.stdout:1/987: unlink d3/d9/da/fae 0 2024-04-12T20:56:29.971 INFO:tasks.workunit.client.1.smithi186.stdout:1/299: dwrite d14/d1a/d1d/f23 [0,4194304] 0 2024-04-12T20:56:29.976 INFO:tasks.workunit.client.1.smithi186.stdout:9/321: creat d3/d2f/d2e/d33/d66/d6b/f6c x:0 0 0 2024-04-12T20:56:29.979 INFO:tasks.workunit.client.0.smithi018.stdout:1/988: mknod d3/d9/da/dd/d31/d32/d33/d47/db0/c14c 0 2024-04-12T20:56:29.980 INFO:tasks.workunit.client.1.smithi186.stdout:6/291: link d9/d43/d2d/c3a d9/d43/d2d/d4d/c66 0 2024-04-12T20:56:29.981 INFO:tasks.workunit.client.1.smithi186.stdout:1/300: mkdir d14/d1a/d33/d36/d5b/d3e/d5f 0 2024-04-12T20:56:29.981 INFO:tasks.workunit.client.0.smithi018.stdout:1/989: dread - d3/d65/fea zero size 2024-04-12T20:56:29.986 INFO:tasks.workunit.client.0.smithi018.stdout:1/990: creat d3/db8/d13b/f14d x:0 0 0 2024-04-12T20:56:29.990 INFO:tasks.workunit.client.1.smithi186.stdout:9/322: creat d3/d2f/f6d x:0 0 0 2024-04-12T20:56:29.993 INFO:tasks.workunit.client.1.smithi186.stdout:9/323: write d3/d2f/d2e/d33/d66/f57 [1092042,4254] 0 2024-04-12T20:56:29.996 INFO:tasks.workunit.client.1.smithi186.stdout:4/396: sync 2024-04-12T20:56:30.002 INFO:tasks.workunit.client.0.smithi018.stdout:1/991: link d3/d9/da/dd/d31/d32/d13/d19/d2d/d4e/d5e/da0/de2/c10c d3/d9/da/dd/d138/c14e 0 2024-04-12T20:56:30.004 INFO:tasks.workunit.client.1.smithi186.stdout:4/397: fdatasync d0/dc/d33/f57 0 2024-04-12T20:56:30.004 INFO:tasks.workunit.client.1.smithi186.stdout:6/292: dwrite d9/d43/d59/d5d/f46 [4194304,4194304] 0 2024-04-12T20:56:30.005 INFO:tasks.workunit.client.1.smithi186.stdout:8/275: dread d7/d18/f1a [0,4194304] 0 2024-04-12T20:56:30.014 INFO:tasks.workunit.client.1.smithi186.stdout:1/301: dwrite d14/d1a/d33/d36/d5b/d3e/f52 [0,4194304] 0 2024-04-12T20:56:30.014 INFO:tasks.workunit.client.1.smithi186.stdout:9/324: getdents d3 0 2024-04-12T20:56:30.014 INFO:tasks.workunit.client.1.smithi186.stdout:4/398: fdatasync d0/d21/d3d/f7d 0 2024-04-12T20:56:30.014 INFO:tasks.workunit.client.1.smithi186.stdout:6/293: mknod d9/d43/d2d/d4d/c67 0 2024-04-12T20:56:30.014 INFO:tasks.workunit.client.1.smithi186.stdout:1/302: dread - d14/d1a/f5a zero size 2024-04-12T20:56:30.018 INFO:tasks.workunit.client.0.smithi018.stdout:1/992: dread d3/d9/d11/f70 [0,4194304] 0 2024-04-12T20:56:30.018 INFO:tasks.workunit.client.1.smithi186.stdout:8/276: getdents d7 0 2024-04-12T20:56:30.020 INFO:tasks.workunit.client.1.smithi186.stdout:9/325: rename d3/f5 to d3/d2f/d2e/d43/d4c/f6e 0 2024-04-12T20:56:30.023 INFO:tasks.workunit.client.1.smithi186.stdout:6/294: link d9/d43/d59/d5d/f48 d9/d43/f68 0 2024-04-12T20:56:30.025 INFO:tasks.workunit.client.1.smithi186.stdout:4/399: dread d0/d21/f27 [0,4194304] 0 2024-04-12T20:56:30.031 INFO:tasks.workunit.client.1.smithi186.stdout:8/277: rename d7/d18/d36/d1d/d1f/c2b to d7/d18/c58 0 2024-04-12T20:56:30.032 INFO:tasks.workunit.client.1.smithi186.stdout:1/303: dread d14/d1a/d21/f4b [0,4194304] 0 2024-04-12T20:56:30.034 INFO:tasks.workunit.client.0.smithi018.stdout:1/993: truncate d3/d9/da/dd/d31/d32/d13/d19/d2d/d4e/d58/f121 548202 0 2024-04-12T20:56:30.035 INFO:tasks.workunit.client.1.smithi186.stdout:1/304: truncate d14/d1a/f37 1328360 0 2024-04-12T20:56:30.035 INFO:tasks.workunit.client.1.smithi186.stdout:6/295: fdatasync d9/f2e 0 2024-04-12T20:56:30.036 INFO:tasks.workunit.client.1.smithi186.stdout:8/278: creat d7/f59 x:0 0 0 2024-04-12T20:56:30.041 INFO:tasks.workunit.client.1.smithi186.stdout:6/296: rename d9/cd to d9/d43/d50/c69 0 2024-04-12T20:56:30.041 INFO:tasks.workunit.client.1.smithi186.stdout:1/305: mknod d14/d1a/d33/c60 0 2024-04-12T20:56:30.041 INFO:tasks.workunit.client.1.smithi186.stdout:6/297: fsync d9/f12 0 2024-04-12T20:56:30.045 INFO:tasks.workunit.client.1.smithi186.stdout:4/400: write d0/d26/f2b [1249872,2781] 0 2024-04-12T20:56:30.047 INFO:tasks.workunit.client.1.smithi186.stdout:9/326: dread d3/d2f/d2e/f41 [0,4194304] 0 2024-04-12T20:56:30.051 INFO:tasks.workunit.client.1.smithi186.stdout:9/327: read d3/ff [1991696,2338] 0 2024-04-12T20:56:30.052 INFO:tasks.workunit.client.0.smithi018.stdout:1/994: dwrite d3/d9/f18 [0,4194304] 0 2024-04-12T20:56:30.054 INFO:tasks.workunit.client.1.smithi186.stdout:6/298: rename d9/d43/f4f to d9/f6a 0 2024-04-12T20:56:30.054 INFO:tasks.workunit.client.1.smithi186.stdout:9/328: readlink d3/l34 0 2024-04-12T20:56:30.055 INFO:tasks.workunit.client.1.smithi186.stdout:6/299: readlink d9/l21 0 2024-04-12T20:56:30.061 INFO:tasks.workunit.client.1.smithi186.stdout:1/306: dwrite d14/d1a/f37 [0,4194304] 0 2024-04-12T20:56:30.061 INFO:tasks.workunit.client.1.smithi186.stdout:9/329: mkdir d3/d12/d6f 0 2024-04-12T20:56:30.062 INFO:tasks.workunit.client.0.smithi018.stdout:1/995: creat d3/d9/da/dd/d31/d32/d13/d19/d2d/d4e/d5e/da0/d12f/f14f x:0 0 0 2024-04-12T20:56:30.067 INFO:tasks.workunit.client.0.smithi018.stdout:1/996: creat d3/d9/da/dd/d31/d32/d13/d19/dd6/f150 x:0 0 0 2024-04-12T20:56:30.067 INFO:tasks.workunit.client.1.smithi186.stdout:6/300: getdents d9/d4c 0 2024-04-12T20:56:30.072 INFO:tasks.workunit.client.1.smithi186.stdout:4/401: dwrite d0/dc/d33/f1a [0,4194304] 0 2024-04-12T20:56:30.072 INFO:tasks.workunit.client.1.smithi186.stdout:9/330: fsync d3/d2f/d2e/f41 0 2024-04-12T20:56:30.075 INFO:tasks.workunit.client.1.smithi186.stdout:8/279: dwrite d7/d18/d36/d17/f1c [4194304,4194304] 0 2024-04-12T20:56:30.078 INFO:tasks.workunit.client.1.smithi186.stdout:6/301: truncate d9/f37 105314 0 2024-04-12T20:56:30.084 INFO:tasks.workunit.client.1.smithi186.stdout:9/331: mkdir d3/d2f/d2e/d33/d36/d70 0 2024-04-12T20:56:30.085 INFO:tasks.workunit.client.1.smithi186.stdout:6/302: creat d9/d43/d2d/f6b x:0 0 0 2024-04-12T20:56:30.087 INFO:tasks.workunit.client.1.smithi186.stdout:1/307: fdatasync d14/d1a/f37 0 2024-04-12T20:56:30.087 INFO:tasks.workunit.client.1.smithi186.stdout:4/402: creat d0/f81 x:0 0 0 2024-04-12T20:56:30.095 INFO:tasks.workunit.client.1.smithi186.stdout:8/280: rename d7/d18/d36/d20/d23/d25/d2e/f3b to d7/d18/d36/d20/d23/d25/f5a 0 2024-04-12T20:56:30.098 INFO:tasks.workunit.client.1.smithi186.stdout:8/281: dread - d7/d18/d36/d20/d23/d25/f46 zero size 2024-04-12T20:56:30.101 INFO:tasks.workunit.client.0.smithi018.stdout:1/997: dwrite d3/d9/da/dd/d31/d32/f1b [0,4194304] 0 2024-04-12T20:56:30.104 INFO:tasks.workunit.client.1.smithi186.stdout:6/303: dwrite d9/d43/f45 [4194304,4194304] 0 2024-04-12T20:56:30.107 INFO:tasks.workunit.client.1.smithi186.stdout:6/304: creat d9/d4c/f6c x:0 0 0 2024-04-12T20:56:30.108 INFO:tasks.workunit.client.1.smithi186.stdout:6/305: stat d9/d43/c2b 0 2024-04-12T20:56:30.109 INFO:tasks.workunit.client.1.smithi186.stdout:6/306: readlink d9/l41 0 2024-04-12T20:56:30.110 INFO:tasks.workunit.client.0.smithi018.stdout:1/998: getdents d3/d9/da/dd/d31/d32/d13/d19/d2d/d4e/d7a/df5 0 2024-04-12T20:56:30.114 INFO:tasks.workunit.client.0.smithi018.stdout:1/999: mkdir d3/d9/da/dd/d26/d151 0 2024-04-12T20:56:30.115 INFO:tasks.workunit.client.1.smithi186.stdout:6/307: rename f6 to d9/d5e/f6d 0 2024-04-12T20:56:30.118 INFO:tasks.workunit.client.1.smithi186.stdout:6/308: mkdir d9/d43/d59/d5d/d6e 0 2024-04-12T20:56:30.125 INFO:tasks.workunit.client.0.smithi018.stderr:+ rm -rf -- ./tmp.00Ko3goaaa 2024-04-12T20:56:30.131 INFO:tasks.workunit.client.1.smithi186.stdout:6/309: dwrite d9/d43/d2d/f5f [0,4194304] 0 2024-04-12T20:56:30.135 INFO:tasks.workunit.client.1.smithi186.stdout:6/310: mkdir d9/d4c/d6f 0 2024-04-12T20:56:30.135 INFO:tasks.workunit.client.1.smithi186.stdout:5/293: sync 2024-04-12T20:56:30.135 INFO:tasks.workunit.client.1.smithi186.stdout:0/361: sync 2024-04-12T20:56:30.138 INFO:tasks.workunit.client.1.smithi186.stdout:8/282: fsync d7/d18/d36/f1b 0 2024-04-12T20:56:30.138 INFO:tasks.workunit.client.1.smithi186.stdout:0/362: rename d0 to d0/d5c/d65/d76 22 2024-04-12T20:56:30.139 INFO:tasks.workunit.client.1.smithi186.stdout:8/283: stat d7/f59 0 2024-04-12T20:56:30.141 INFO:tasks.workunit.client.1.smithi186.stdout:8/284: dread - d7/d18/d36/d20/d23/d25/f46 zero size 2024-04-12T20:56:30.143 INFO:tasks.workunit.client.1.smithi186.stdout:8/285: fsync d7/d18/d36/d20/d23/d25/f46 0 2024-04-12T20:56:30.168 INFO:tasks.workunit.client.1.smithi186.stdout:6/311: write d9/d5e/f6d [242075,80083] 0 2024-04-12T20:56:30.170 INFO:tasks.workunit.client.1.smithi186.stdout:5/294: creat d5/db/d14/f61 x:0 0 0 2024-04-12T20:56:30.171 INFO:tasks.workunit.client.1.smithi186.stdout:6/312: write d9/d43/d2d/d4d/f64 [121450,70974] 0 2024-04-12T20:56:30.183 INFO:tasks.workunit.client.1.smithi186.stdout:6/313: mkdir d9/d4c/d70 0 2024-04-12T20:56:30.184 INFO:tasks.workunit.client.1.smithi186.stdout:5/295: creat d5/db/d15/d2a/d4c/f62 x:0 0 0 2024-04-12T20:56:30.185 INFO:tasks.workunit.client.1.smithi186.stdout:5/296: chown d5/db/d33 0 1 2024-04-12T20:56:30.185 INFO:tasks.workunit.client.1.smithi186.stdout:6/314: creat d9/d5e/f71 x:0 0 0 2024-04-12T20:56:30.187 INFO:tasks.workunit.client.1.smithi186.stdout:0/363: getdents d0/d14 0 2024-04-12T20:56:30.188 INFO:tasks.workunit.client.1.smithi186.stdout:5/297: write d5/db/fc [7602963,85219] 0 2024-04-12T20:56:30.188 INFO:tasks.workunit.client.1.smithi186.stdout:6/315: fsync d9/f10 0 2024-04-12T20:56:30.193 INFO:tasks.workunit.client.1.smithi186.stdout:0/364: truncate d0/d14/f19 1621266 0 2024-04-12T20:56:30.194 INFO:tasks.workunit.client.1.smithi186.stdout:6/316: truncate d9/d43/f39 442677 0 2024-04-12T20:56:30.198 INFO:tasks.workunit.client.1.smithi186.stdout:0/365: creat d0/d10/d32/d3f/f77 x:0 0 0 2024-04-12T20:56:30.198 INFO:tasks.workunit.client.1.smithi186.stdout:6/317: symlink d9/d43/d2d/d4d/l72 0 2024-04-12T20:56:30.203 INFO:tasks.workunit.client.1.smithi186.stdout:6/318: write d9/d43/d59/d5d/f46 [5626014,20590] 0 2024-04-12T20:56:30.210 INFO:tasks.workunit.client.1.smithi186.stdout:6/319: mkdir d9/d5e/d73 0 2024-04-12T20:56:30.234 INFO:tasks.workunit.client.1.smithi186.stdout:0/366: dread d0/f57 [0,4194304] 0 2024-04-12T20:56:30.251 INFO:tasks.workunit.client.1.smithi186.stdout:1/308: dread d14/f16 [0,4194304] 0 2024-04-12T20:56:30.253 INFO:tasks.workunit.client.1.smithi186.stdout:1/309: read ff [260859,110905] 0 2024-04-12T20:56:30.259 INFO:tasks.workunit.client.1.smithi186.stdout:0/367: dwrite d0/d14/d4e/d1e/f1f [0,4194304] 0 2024-04-12T20:56:30.263 INFO:tasks.workunit.client.1.smithi186.stdout:0/368: write d0/d10/d32/f48 [432822,37460] 0 2024-04-12T20:56:30.263 INFO:tasks.workunit.client.1.smithi186.stdout:1/310: creat d14/d1a/d33/d36/d5b/f61 x:0 0 0 2024-04-12T20:56:30.266 INFO:tasks.workunit.client.1.smithi186.stdout:1/311: rename d14/d1a/d58 to d14/d1a/d58/d62 22 2024-04-12T20:56:30.270 INFO:tasks.workunit.client.1.smithi186.stdout:1/312: mknod d14/d1a/d3c/c63 0 2024-04-12T20:56:30.276 INFO:tasks.workunit.client.1.smithi186.stdout:1/313: mknod d14/d1a/d21/c64 0 2024-04-12T20:56:30.279 INFO:tasks.workunit.client.1.smithi186.stdout:1/314: write d14/d1a/d1d/f46 [1798375,51128] 0 2024-04-12T20:56:30.279 INFO:tasks.workunit.client.1.smithi186.stdout:5/298: truncate d5/db/d15/d2a/d2d/d46/d55/f37 2009738 0 2024-04-12T20:56:30.281 INFO:tasks.workunit.client.1.smithi186.stdout:1/315: stat d14/d1a/d33/d36/d5b/d3e/c48 0 2024-04-12T20:56:30.284 INFO:tasks.workunit.client.1.smithi186.stdout:1/316: write d14/d1a/d33/d36/d5b/d3e/f52 [2994987,119473] 0 2024-04-12T20:56:30.289 INFO:tasks.workunit.client.1.smithi186.stdout:1/317: truncate d14/d1a/d33/d36/d5b/f4d 416353 0 2024-04-12T20:56:30.295 INFO:tasks.workunit.client.1.smithi186.stdout:5/299: dwrite d5/d27/f60 [0,4194304] 0 2024-04-12T20:56:30.303 INFO:tasks.workunit.client.1.smithi186.stdout:0/369: dwrite d0/f6 [0,4194304] 0 2024-04-12T20:56:30.307 INFO:tasks.workunit.client.1.smithi186.stdout:0/370: write d0/d10/d32/f48 [594110,108468] 0 2024-04-12T20:56:30.309 INFO:tasks.workunit.client.1.smithi186.stdout:1/318: dwrite d14/f16 [4194304,4194304] 0 2024-04-12T20:56:30.312 INFO:tasks.workunit.client.1.smithi186.stdout:0/371: chown d0/c3 7434392 1 2024-04-12T20:56:30.312 INFO:tasks.workunit.client.1.smithi186.stdout:1/319: truncate d14/d1a/f5a 931140 0 2024-04-12T20:56:30.319 INFO:tasks.workunit.client.1.smithi186.stdout:1/320: mknod d14/d1a/d33/d36/d5b/d3e/c65 0 2024-04-12T20:56:30.320 INFO:tasks.workunit.client.1.smithi186.stdout:1/321: chown f9 422 1 2024-04-12T20:56:30.325 INFO:tasks.workunit.client.1.smithi186.stdout:1/322: fdatasync ff 0 2024-04-12T20:56:30.328 INFO:tasks.workunit.client.1.smithi186.stdout:1/323: mknod d14/d1a/d33/d36/d5b/d3e/c66 0 2024-04-12T20:56:30.329 INFO:tasks.workunit.client.1.smithi186.stdout:5/300: dread d5/db/d15/f28 [0,4194304] 0 2024-04-12T20:56:30.331 INFO:tasks.workunit.client.1.smithi186.stdout:1/324: dread f4 [0,4194304] 0 2024-04-12T20:56:30.332 INFO:tasks.workunit.client.1.smithi186.stdout:5/301: mknod d5/db/d15/d2a/d2d/d46/d55/c63 0 2024-04-12T20:56:30.334 INFO:tasks.workunit.client.1.smithi186.stdout:5/302: creat d5/db/d2e/f64 x:0 0 0 2024-04-12T20:56:30.336 INFO:tasks.workunit.client.1.smithi186.stdout:1/325: creat d14/d1a/d33/d36/d5b/f67 x:0 0 0 2024-04-12T20:56:30.337 INFO:tasks.workunit.client.1.smithi186.stdout:1/326: fdatasync d14/d1a/f37 0 2024-04-12T20:56:30.338 INFO:tasks.workunit.client.1.smithi186.stdout:5/303: mkdir d5/db/d15/d2a/d65 0 2024-04-12T20:56:30.338 INFO:tasks.workunit.client.1.smithi186.stdout:1/327: read fe [4262402,10927] 0 2024-04-12T20:56:30.340 INFO:tasks.workunit.client.1.smithi186.stdout:0/372: write d0/f57 [1213118,110858] 0 2024-04-12T20:56:30.341 INFO:tasks.workunit.client.1.smithi186.stdout:5/304: chown d5/db/d15/d2a/d4c/d5b 542803025 1 2024-04-12T20:56:30.342 INFO:tasks.workunit.client.1.smithi186.stdout:1/328: write d14/d1a/d21/f27 [4526790,31197] 0 2024-04-12T20:56:30.342 INFO:tasks.workunit.client.1.smithi186.stdout:0/373: chown d0/d10/d32/f3d 593467556 1 2024-04-12T20:56:30.345 INFO:tasks.workunit.client.1.smithi186.stdout:5/305: unlink d5/db/d14/f54 0 2024-04-12T20:56:30.351 INFO:tasks.workunit.client.1.smithi186.stdout:5/306: mknod d5/db/d15/d2a/d2d/d46/c66 0 2024-04-12T20:56:30.354 INFO:tasks.workunit.client.1.smithi186.stdout:1/329: link d14/d1a/d21/c64 d14/c68 0 2024-04-12T20:56:30.357 INFO:tasks.workunit.client.1.smithi186.stdout:1/330: fsync d14/d1a/d21/f4b 0 2024-04-12T20:56:30.359 INFO:tasks.workunit.client.1.smithi186.stdout:1/331: mkdir d14/d1a/d58/d69 0 2024-04-12T20:56:30.360 INFO:tasks.workunit.client.1.smithi186.stdout:0/374: write d0/d14/f19 [126918,54059] 0 2024-04-12T20:56:30.362 INFO:tasks.workunit.client.1.smithi186.stdout:1/332: write d14/d1a/d1d/f29 [1415654,25820] 0 2024-04-12T20:56:30.365 INFO:tasks.workunit.client.1.smithi186.stdout:0/375: chown d0/d5c/d66/l6f 381 1 2024-04-12T20:56:30.367 INFO:tasks.workunit.client.1.smithi186.stdout:1/333: mkdir d14/d1a/d33/d36/d5b/d6a 0 2024-04-12T20:56:30.372 INFO:tasks.workunit.client.1.smithi186.stdout:1/334: dread d14/d1a/d21/f4b [0,4194304] 0 2024-04-12T20:56:30.374 INFO:tasks.workunit.client.1.smithi186.stdout:5/307: dread d5/db/d15/d2a/d2d/d48/f3a [0,4194304] 0 2024-04-12T20:56:30.376 INFO:tasks.workunit.client.1.smithi186.stdout:1/335: unlink d14/d1a/d21/c64 0 2024-04-12T20:56:30.377 INFO:tasks.workunit.client.1.smithi186.stdout:1/336: chown d14/d1a/d3c/f47 58 1 2024-04-12T20:56:30.382 INFO:tasks.workunit.client.1.smithi186.stdout:0/376: dwrite d0/d10/f4d [0,4194304] 0 2024-04-12T20:56:30.382 INFO:tasks.workunit.client.1.smithi186.stdout:0/377: readlink d0/l26 0 2024-04-12T20:56:30.384 INFO:tasks.workunit.client.1.smithi186.stdout:5/308: write d5/db/f11 [432075,109703] 0 2024-04-12T20:56:30.387 INFO:tasks.workunit.client.1.smithi186.stdout:0/378: truncate d0/d10/d32/d3f/f64 277845 0 2024-04-12T20:56:30.387 INFO:tasks.workunit.client.1.smithi186.stdout:5/309: write d5/db/fc [215198,77872] 0 2024-04-12T20:56:30.402 INFO:tasks.workunit.client.1.smithi186.stdout:0/379: truncate d0/d14/d4e/d1e/f1f 2339138 0 2024-04-12T20:56:30.406 INFO:tasks.workunit.client.1.smithi186.stdout:5/310: dwrite d5/db/d14/f61 [0,4194304] 0 2024-04-12T20:56:30.409 INFO:tasks.workunit.client.1.smithi186.stdout:0/380: link d0/d10/d32/d3f/f64 d0/d10/d32/d3f/f78 0 2024-04-12T20:56:30.409 INFO:tasks.workunit.client.1.smithi186.stdout:5/311: creat d5/d44/f67 x:0 0 0 2024-04-12T20:56:30.410 INFO:tasks.workunit.client.1.smithi186.stdout:5/312: chown d5 7603 1 2024-04-12T20:56:30.415 INFO:tasks.workunit.client.1.smithi186.stdout:0/381: mkdir d0/d10/d72/d79 0 2024-04-12T20:56:30.416 INFO:tasks.workunit.client.1.smithi186.stdout:5/313: mknod d5/db/d15/d2a/d4c/d5b/c68 0 2024-04-12T20:56:30.417 INFO:tasks.workunit.client.1.smithi186.stdout:5/314: readlink d5/db/d2e/l47 0 2024-04-12T20:56:30.419 INFO:tasks.workunit.client.1.smithi186.stdout:5/315: chown d5/db/d15/d2a/d2d/f49 201542867 1 2024-04-12T20:56:30.421 INFO:tasks.workunit.client.1.smithi186.stdout:5/316: read d5/d27/f60 [1034964,40857] 0 2024-04-12T20:56:30.422 INFO:tasks.workunit.client.1.smithi186.stdout:5/317: stat d5/d44 0 2024-04-12T20:56:30.427 INFO:tasks.workunit.client.1.smithi186.stdout:5/318: read d5/db/d15/d2a/f4f [359807,85792] 0 2024-04-12T20:56:30.428 INFO:tasks.workunit.client.1.smithi186.stdout:5/319: chown d5/d27 105947 1 2024-04-12T20:56:30.432 INFO:tasks.workunit.client.1.smithi186.stdout:5/320: mknod d5/db/c69 0 2024-04-12T20:56:30.433 INFO:tasks.workunit.client.1.smithi186.stdout:0/382: dwrite d0/d5c/f5f [0,4194304] 0 2024-04-12T20:56:30.437 INFO:tasks.workunit.client.1.smithi186.stdout:5/321: mkdir d5/db/d15/d6a 0 2024-04-12T20:56:30.437 INFO:tasks.workunit.client.1.smithi186.stdout:0/383: write d0/d14/f24 [279331,36680] 0 2024-04-12T20:56:30.440 INFO:tasks.workunit.client.1.smithi186.stdout:0/384: dread - d0/d10/d32/d3f/f77 zero size 2024-04-12T20:56:30.441 INFO:tasks.workunit.client.1.smithi186.stdout:5/322: link d5/f59 d5/d27/f6b 0 2024-04-12T20:56:30.443 INFO:tasks.workunit.client.1.smithi186.stdout:0/385: write d0/d5c/f5f [189904,120706] 0 2024-04-12T20:56:30.444 INFO:tasks.workunit.client.1.smithi186.stdout:5/323: chown d5/db/d14/f52 7621 1 2024-04-12T20:56:30.450 INFO:tasks.workunit.client.1.smithi186.stdout:0/386: creat d0/d10/d71/f7a x:0 0 0 2024-04-12T20:56:30.453 INFO:tasks.workunit.client.1.smithi186.stdout:0/387: creat d0/d5c/d65/f7b x:0 0 0 2024-04-12T20:56:30.457 INFO:tasks.workunit.client.1.smithi186.stdout:0/388: rmdir d0/d14/d4e/d46 39 2024-04-12T20:56:30.461 INFO:tasks.workunit.client.1.smithi186.stdout:0/389: rmdir d0 39 2024-04-12T20:56:30.462 INFO:tasks.workunit.client.1.smithi186.stdout:5/324: dwrite d5/db/d15/d2a/f4f [0,4194304] 0 2024-04-12T20:56:30.466 INFO:tasks.workunit.client.1.smithi186.stdout:0/390: getdents d0/d5d 0 2024-04-12T20:56:30.468 INFO:tasks.workunit.client.1.smithi186.stdout:0/391: symlink d0/d10/d72/l7c 0 2024-04-12T20:56:30.470 INFO:tasks.workunit.client.1.smithi186.stdout:3/306: sync 2024-04-12T20:56:30.471 INFO:tasks.workunit.client.1.smithi186.stdout:7/345: sync 2024-04-12T20:56:30.471 INFO:tasks.workunit.client.1.smithi186.stdout:2/300: sync 2024-04-12T20:56:30.471 INFO:tasks.workunit.client.1.smithi186.stdout:0/392: creat d0/d10/d72/f7d x:0 0 0 2024-04-12T20:56:30.476 INFO:tasks.workunit.client.1.smithi186.stdout:3/307: write d1/d3/d49/d4d/f51 [613067,127154] 0 2024-04-12T20:56:30.477 INFO:tasks.workunit.client.1.smithi186.stdout:5/325: dread d5/db/d15/f28 [0,4194304] 0 2024-04-12T20:56:30.479 INFO:tasks.workunit.client.1.smithi186.stdout:0/393: symlink d0/d10/d32/d3f/l7e 0 2024-04-12T20:56:30.481 INFO:tasks.workunit.client.1.smithi186.stdout:7/346: rmdir d7/d10/d1c/d5d/d7b 0 2024-04-12T20:56:30.481 INFO:tasks.workunit.client.1.smithi186.stdout:0/394: chown d0/d14/d4e/c55 5568683 1 2024-04-12T20:56:30.483 INFO:tasks.workunit.client.1.smithi186.stdout:7/347: readlink d7/d10/d2d/d40/l54 0 2024-04-12T20:56:30.488 INFO:tasks.workunit.client.1.smithi186.stdout:2/301: truncate d0/dc/f4c 2934536 0 2024-04-12T20:56:30.488 INFO:tasks.workunit.client.1.smithi186.stdout:7/348: creat d7/d10/d2d/d40/d4a/d6c/f7d x:0 0 0 2024-04-12T20:56:30.489 INFO:tasks.workunit.client.1.smithi186.stdout:5/326: read d5/db/d15/f2f [2147099,42365] 0 2024-04-12T20:56:30.491 INFO:tasks.workunit.client.1.smithi186.stdout:0/395: getdents d0/d5c 0 2024-04-12T20:56:30.492 INFO:tasks.workunit.client.1.smithi186.stdout:3/308: dread d1/d3/d21/d22/f4c [0,4194304] 0 2024-04-12T20:56:30.497 INFO:tasks.workunit.client.1.smithi186.stdout:5/327: creat d5/db/d15/d2a/d2d/d46/f6c x:0 0 0 2024-04-12T20:56:30.497 INFO:tasks.workunit.client.1.smithi186.stdout:3/309: creat d1/d3/d21/d22/d24/f60 x:0 0 0 2024-04-12T20:56:30.498 INFO:tasks.workunit.client.1.smithi186.stdout:5/328: mkdir d5/db/d15/d2a/d6d 0 2024-04-12T20:56:30.499 INFO:tasks.workunit.client.1.smithi186.stdout:3/310: stat d1/d3/d21/d22/d24 0 2024-04-12T20:56:30.508 INFO:tasks.workunit.client.1.smithi186.stdout:6/320: sync 2024-04-12T20:56:30.508 INFO:tasks.workunit.client.1.smithi186.stdout:9/332: sync 2024-04-12T20:56:30.508 INFO:tasks.workunit.client.1.smithi186.stdout:8/286: sync 2024-04-12T20:56:30.508 INFO:tasks.workunit.client.1.smithi186.stdout:4/403: sync 2024-04-12T20:56:30.513 INFO:tasks.workunit.client.1.smithi186.stdout:4/404: dread - d0/d21/f40 zero size 2024-04-12T20:56:30.514 INFO:tasks.workunit.client.1.smithi186.stdout:2/302: dread d0/d3/d1a/d21/d26/f3d [0,4194304] 0 2024-04-12T20:56:30.517 INFO:tasks.workunit.client.1.smithi186.stdout:9/333: mkdir d3/d12/d27/d71 0 2024-04-12T20:56:30.518 INFO:tasks.workunit.client.1.smithi186.stdout:4/405: truncate d0/d21/f40 180657 0 2024-04-12T20:56:30.519 INFO:tasks.workunit.client.1.smithi186.stdout:6/321: mknod d9/d5e/d73/c74 0 2024-04-12T20:56:30.528 INFO:tasks.workunit.client.1.smithi186.stdout:0/396: dread d0/f3e [4194304,4194304] 0 2024-04-12T20:56:30.530 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:56:30 smithi186 ceph-mon[36597]: from='mgr.24359 172.21.15.186:0/1315906828' entity='mgr.smithi186.kqcfxk' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T20:56:30.531 INFO:tasks.workunit.client.1.smithi186.stdout:5/329: dwrite d5/d27/f6b [0,4194304] 0 2024-04-12T20:56:30.534 INFO:tasks.workunit.client.1.smithi186.stdout:3/311: dwrite d1/d2c/d48/f4f [0,4194304] 0 2024-04-12T20:56:30.534 INFO:tasks.workunit.client.1.smithi186.stdout:0/397: unlink d0/d14/d4e/d1e/f37 0 2024-04-12T20:56:30.542 INFO:tasks.workunit.client.1.smithi186.stdout:6/322: symlink d9/d43/d59/d5d/d6e/l75 0 2024-04-12T20:56:30.543 INFO:tasks.workunit.client.1.smithi186.stdout:9/334: getdents d3/d12 0 2024-04-12T20:56:30.543 INFO:tasks.workunit.client.1.smithi186.stdout:5/330: unlink d5/db/d15/d2a/d2d/d46/d55/c24 0 2024-04-12T20:56:30.543 INFO:tasks.workunit.client.1.smithi186.stdout:0/398: creat d0/d5c/f7f x:0 0 0 2024-04-12T20:56:30.551 INFO:tasks.workunit.client.1.smithi186.stdout:3/312: chown d1/d3/da/lb 0 1 2024-04-12T20:56:30.551 INFO:tasks.workunit.client.1.smithi186.stdout:0/399: dread - d0/f70 zero size 2024-04-12T20:56:30.553 INFO:tasks.workunit.client.1.smithi186.stdout:9/335: stat d3/fd 0 2024-04-12T20:56:30.556 INFO:tasks.workunit.client.1.smithi186.stdout:5/331: chown d5/db/d14/f52 2183 1 2024-04-12T20:56:30.561 INFO:tasks.workunit.client.1.smithi186.stdout:6/323: truncate d9/d43/d50/f54 668410 0 2024-04-12T20:56:30.564 INFO:tasks.workunit.client.1.smithi186.stdout:8/287: dwrite d7/de/d10/f15 [4194304,4194304] 0 2024-04-12T20:56:30.566 INFO:tasks.workunit.client.1.smithi186.stdout:9/336: chown d3/d2f/d2e/d43/l5c 7175720 1 2024-04-12T20:56:30.566 INFO:tasks.workunit.client.1.smithi186.stdout:4/406: dwrite d0/dc/d33/f7 [0,4194304] 0 2024-04-12T20:56:30.572 INFO:tasks.workunit.client.1.smithi186.stdout:5/332: stat d5/db/d33/l45 0 2024-04-12T20:56:30.572 INFO:tasks.workunit.client.1.smithi186.stdout:2/303: dwrite d0/d3/d1a/d24/f33 [0,4194304] 0 2024-04-12T20:56:30.575 INFO:tasks.workunit.client.1.smithi186.stdout:4/407: creat d0/dc/d39/f82 x:0 0 0 2024-04-12T20:56:30.575 INFO:tasks.workunit.client.1.smithi186.stdout:2/304: readlink d0/d3/d1a/d41/l4a 0 2024-04-12T20:56:30.578 INFO:tasks.workunit.client.1.smithi186.stdout:5/333: rename d5/db/d15/d2a/f4f to d5/d44/f6e 0 2024-04-12T20:56:30.578 INFO:tasks.workunit.client.1.smithi186.stdout:6/324: symlink d9/d43/d59/d5d/d6e/l76 0 2024-04-12T20:56:30.579 INFO:tasks.workunit.client.1.smithi186.stdout:3/313: link d1/d3/da/lb d1/d3/d21/d22/d34/l61 0 2024-04-12T20:56:30.584 INFO:tasks.workunit.client.1.smithi186.stdout:6/325: chown d9/d43/d59 2 1 2024-04-12T20:56:30.586 INFO:tasks.workunit.client.1.smithi186.stdout:2/305: creat d0/d3/d1a/d48/f5b x:0 0 0 2024-04-12T20:56:30.587 INFO:tasks.workunit.client.1.smithi186.stdout:8/288: getdents d7/d18/d36/d17 0 2024-04-12T20:56:30.589 INFO:tasks.workunit.client.1.smithi186.stdout:0/400: dwrite d0/d10/d32/f3d [0,4194304] 0 2024-04-12T20:56:30.589 INFO:tasks.workunit.client.1.smithi186.stdout:5/334: mknod d5/db/d15/d2a/d2d/d46/c6f 0 2024-04-12T20:56:30.591 INFO:tasks.workunit.client.1.smithi186.stdout:4/408: creat d0/d21/f83 x:0 0 0 2024-04-12T20:56:30.593 INFO:tasks.workunit.client.1.smithi186.stdout:2/306: rmdir d0/d3/d1a/d24 39 2024-04-12T20:56:30.594 INFO:tasks.workunit.client.1.smithi186.stdout:5/335: mkdir d5/db/d2e/d70 0 2024-04-12T20:56:30.596 INFO:tasks.workunit.client.1.smithi186.stdout:2/307: fdatasync d0/dc/f40 0 2024-04-12T20:56:30.600 INFO:tasks.workunit.client.1.smithi186.stdout:0/401: creat d0/d14/d4e/d46/f80 x:0 0 0 2024-04-12T20:56:30.601 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:56:30 smithi018 ceph-mon[28799]: from='mgr.24359 172.21.15.186:0/1315906828' entity='mgr.smithi186.kqcfxk' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T20:56:30.609 INFO:tasks.workunit.client.1.smithi186.stdout:4/409: truncate d0/dc/d33/f57 959201 0 2024-04-12T20:56:30.611 INFO:tasks.workunit.client.1.smithi186.stdout:0/402: write d0/d10/d32/d3f/f77 [385419,30966] 0 2024-04-12T20:56:30.614 INFO:tasks.workunit.client.1.smithi186.stdout:3/314: dread d1/d3/d21/d5c/f40 [0,4194304] 0 2024-04-12T20:56:30.616 INFO:tasks.workunit.client.1.smithi186.stdout:5/336: rename d5/db/d14 to d5/db/d71 0 2024-04-12T20:56:30.619 INFO:tasks.workunit.client.1.smithi186.stdout:0/403: dread d0/d10/f43 [0,4194304] 0 2024-04-12T20:56:30.620 INFO:tasks.workunit.client.1.smithi186.stdout:3/315: creat d1/d3/d21/d5c/f62 x:0 0 0 2024-04-12T20:56:30.622 INFO:tasks.workunit.client.1.smithi186.stdout:3/316: readlink d1/d3/d21/d22/d24/d2f/l33 0 2024-04-12T20:56:30.623 INFO:tasks.workunit.client.1.smithi186.stdout:2/308: dread d0/d3/d1a/d24/f33 [0,4194304] 0 2024-04-12T20:56:30.625 INFO:tasks.workunit.client.1.smithi186.stdout:4/410: creat d0/d21/d3d/f84 x:0 0 0 2024-04-12T20:56:30.626 INFO:tasks.workunit.client.1.smithi186.stdout:5/337: creat d5/d44/d5d/f72 x:0 0 0 2024-04-12T20:56:30.630 INFO:tasks.workunit.client.1.smithi186.stdout:2/309: creat d0/d3/d1a/d41/f5c x:0 0 0 2024-04-12T20:56:30.631 INFO:tasks.workunit.client.1.smithi186.stdout:2/310: readlink d0/l30 0 2024-04-12T20:56:30.634 INFO:tasks.workunit.client.1.smithi186.stdout:3/317: creat d1/f63 x:0 0 0 2024-04-12T20:56:30.637 INFO:tasks.workunit.client.1.smithi186.stdout:0/404: write d0/f4f [887976,122656] 0 2024-04-12T20:56:30.639 INFO:tasks.workunit.client.1.smithi186.stdout:3/318: readlink d1/d3/d21/d22/l3b 0 2024-04-12T20:56:30.641 INFO:tasks.workunit.client.1.smithi186.stdout:3/319: stat d1/d3/d21/d22/d24 0 2024-04-12T20:56:30.645 INFO:tasks.workunit.client.1.smithi186.stdout:0/405: rename d0/d14/d4e/d46 to d0/d81 0 2024-04-12T20:56:30.646 INFO:tasks.workunit.client.1.smithi186.stdout:3/320: mkdir d1/d2c/d64 0 2024-04-12T20:56:30.647 INFO:tasks.workunit.client.1.smithi186.stdout:2/311: dwrite d0/dc/f40 [0,4194304] 0 2024-04-12T20:56:30.647 INFO:tasks.workunit.client.1.smithi186.stdout:5/338: dwrite d5/db/d15/d2a/d2d/d48/f3a [0,4194304] 0 2024-04-12T20:56:30.650 INFO:tasks.workunit.client.1.smithi186.stdout:2/312: fsync d0/d3/d1a/d24/f34 0 2024-04-12T20:56:30.651 INFO:tasks.workunit.client.1.smithi186.stdout:2/313: chown d0/c5 10 1 2024-04-12T20:56:30.657 INFO:tasks.workunit.client.1.smithi186.stdout:3/321: creat d1/d3/d21/f65 x:0 0 0 2024-04-12T20:56:30.657 INFO:tasks.workunit.client.1.smithi186.stdout:5/339: mknod d5/db/d33/c73 0 2024-04-12T20:56:30.666 INFO:tasks.workunit.client.1.smithi186.stdout:3/322: creat d1/d3/d21/d22/d34/f66 x:0 0 0 2024-04-12T20:56:30.666 INFO:tasks.workunit.client.1.smithi186.stdout:2/314: rename d0/f4 to d0/dc/f5d 0 2024-04-12T20:56:30.666 INFO:tasks.workunit.client.1.smithi186.stdout:0/406: dread d0/d5c/f5f [0,4194304] 0 2024-04-12T20:56:30.666 INFO:tasks.workunit.client.1.smithi186.stdout:2/315: chown d0/f10 9686292 1 2024-04-12T20:56:30.667 INFO:tasks.workunit.client.1.smithi186.stdout:3/323: stat d1/d3/d21/d22/d4a/l50 0 2024-04-12T20:56:30.667 INFO:tasks.workunit.client.1.smithi186.stdout:0/407: write d0/d14/f24 [358992,114734] 0 2024-04-12T20:56:30.667 INFO:tasks.workunit.client.1.smithi186.stdout:0/408: chown d0 1 1 2024-04-12T20:56:30.670 INFO:tasks.workunit.client.1.smithi186.stdout:3/324: creat d1/d2c/f67 x:0 0 0 2024-04-12T20:56:30.674 INFO:tasks.workunit.client.1.smithi186.stdout:5/340: rename d5/db/d15/d2a/c5a to d5/db/d15/d2a/d2d/d46/d55/c74 0 2024-04-12T20:56:30.675 INFO:tasks.workunit.client.1.smithi186.stdout:5/341: chown d5/db/f57 51440189 1 2024-04-12T20:56:30.680 INFO:tasks.workunit.client.1.smithi186.stdout:0/409: creat d0/d14/d4e/d1e/f82 x:0 0 0 2024-04-12T20:56:30.685 INFO:tasks.workunit.client.1.smithi186.stdout:0/410: getdents d0/d5c/d66 0 2024-04-12T20:56:30.687 INFO:tasks.workunit.client.1.smithi186.stdout:0/411: chown d0/d14/c49 52 1 2024-04-12T20:56:30.687 INFO:tasks.workunit.client.1.smithi186.stdout:1/337: sync 2024-04-12T20:56:30.688 INFO:tasks.workunit.client.1.smithi186.stdout:2/316: truncate d0/dc/f40 3386202 0 2024-04-12T20:56:30.691 INFO:tasks.workunit.client.1.smithi186.stdout:0/412: rename d0/d10/f67 to d0/d81/f83 0 2024-04-12T20:56:30.695 INFO:tasks.workunit.client.1.smithi186.stdout:0/413: write d0/d14/f35 [2264713,53424] 0 2024-04-12T20:56:30.696 INFO:tasks.workunit.client.1.smithi186.stdout:2/317: rename d0/d3/d1a/d48/f5b to d0/d3/d1a/d41/f5e 0 2024-04-12T20:56:30.696 INFO:tasks.workunit.client.1.smithi186.stdout:3/325: dread d1/d3/d21/f41 [0,4194304] 0 2024-04-12T20:56:30.697 INFO:tasks.workunit.client.1.smithi186.stdout:0/414: dread - d0/d5c/d65/f7b zero size 2024-04-12T20:56:30.699 INFO:tasks.workunit.client.1.smithi186.stdout:0/415: rmdir d0 39 2024-04-12T20:56:30.702 INFO:tasks.workunit.client.1.smithi186.stdout:2/318: truncate d0/d3/d1a/d27/f59 3732737 0 2024-04-12T20:56:30.702 INFO:tasks.workunit.client.1.smithi186.stdout:5/342: dread d5/db/d15/f31 [0,4194304] 0 2024-04-12T20:56:30.706 INFO:tasks.workunit.client.1.smithi186.stdout:0/416: dread d0/d14/d4e/d1e/f4b [0,4194304] 0 2024-04-12T20:56:30.709 INFO:tasks.workunit.client.1.smithi186.stdout:2/319: creat d0/dc/d2e/f5f x:0 0 0 2024-04-12T20:56:30.710 INFO:tasks.workunit.client.1.smithi186.stdout:2/320: stat d0/c9 0 2024-04-12T20:56:30.713 INFO:tasks.workunit.client.1.smithi186.stdout:1/338: dwrite d14/d1a/d33/d36/d5b/f61 [0,4194304] 0 2024-04-12T20:56:30.717 INFO:tasks.workunit.client.1.smithi186.stdout:3/326: dwrite d1/d3/d21/d22/d24/f60 [0,4194304] 0 2024-04-12T20:56:30.721 INFO:tasks.workunit.client.1.smithi186.stdout:5/343: write d5/d27/f60 [4091344,32776] 0 2024-04-12T20:56:30.724 INFO:tasks.workunit.client.1.smithi186.stdout:3/327: write d1/d2c/d48/f4f [1415569,25753] 0 2024-04-12T20:56:30.727 INFO:tasks.workunit.client.1.smithi186.stdout:5/344: chown d5/db/d15/d2a/f41 6 1 2024-04-12T20:56:30.729 INFO:tasks.workunit.client.1.smithi186.stdout:1/339: creat d14/d1a/d33/f6b x:0 0 0 2024-04-12T20:56:30.729 INFO:tasks.workunit.client.1.smithi186.stdout:5/345: write d5/db/f40 [4366703,35618] 0 2024-04-12T20:56:30.729 INFO:tasks.workunit.client.1.smithi186.stdout:2/321: getdents d0/d3/d1a/d48 0 2024-04-12T20:56:30.730 INFO:tasks.workunit.client.1.smithi186.stdout:3/328: symlink d1/d3/da/l68 0 2024-04-12T20:56:30.731 INFO:tasks.workunit.client.1.smithi186.stdout:3/329: fsync d1/fc 0 2024-04-12T20:56:30.737 INFO:tasks.workunit.client.1.smithi186.stdout:2/322: creat d0/d3/d1a/d24/f60 x:0 0 0 2024-04-12T20:56:30.738 INFO:tasks.workunit.client.1.smithi186.stdout:3/330: creat d1/d3/d21/d5c/f69 x:0 0 0 2024-04-12T20:56:30.739 INFO:tasks.workunit.client.1.smithi186.stdout:1/340: mkdir d14/d6c 0 2024-04-12T20:56:30.743 INFO:tasks.workunit.client.1.smithi186.stdout:2/323: unlink d0/d3/d1a/d21/f39 0 2024-04-12T20:56:30.746 INFO:tasks.workunit.client.1.smithi186.stdout:1/341: rename c8 to d14/d1a/d58/d69/c6d 0 2024-04-12T20:56:30.746 INFO:tasks.workunit.client.1.smithi186.stdout:2/324: read d0/d3/d1a/d21/f4b [3447256,26928] 0 2024-04-12T20:56:30.747 INFO:tasks.workunit.client.1.smithi186.stdout:0/417: truncate d0/d14/d4e/f33 1214834 0 2024-04-12T20:56:30.748 INFO:tasks.workunit.client.1.smithi186.stdout:5/346: dwrite d5/db/f11 [0,4194304] 0 2024-04-12T20:56:30.749 INFO:tasks.workunit.client.1.smithi186.stdout:3/331: creat d1/f6a x:0 0 0 2024-04-12T20:56:30.751 INFO:tasks.workunit.client.1.smithi186.stdout:1/342: dread - d14/d1a/d21/f57 zero size 2024-04-12T20:56:30.752 INFO:tasks.workunit.client.1.smithi186.stdout:0/418: read d0/f4f [713990,113437] 0 2024-04-12T20:56:30.755 INFO:tasks.workunit.client.1.smithi186.stdout:3/332: symlink d1/d3/d21/l6b 0 2024-04-12T20:56:30.755 INFO:tasks.workunit.client.1.smithi186.stdout:1/343: stat d14/d1a/d21/f4b 0 2024-04-12T20:56:30.755 INFO:tasks.workunit.client.1.smithi186.stdout:5/347: creat d5/db/d71/f75 x:0 0 0 2024-04-12T20:56:30.757 INFO:tasks.workunit.client.1.smithi186.stdout:1/344: stat d14/d1a/d21/f27 0 2024-04-12T20:56:30.760 INFO:tasks.workunit.client.1.smithi186.stdout:0/419: write d0/d10/d32/f48 [1088058,51138] 0 2024-04-12T20:56:30.762 INFO:tasks.workunit.client.1.smithi186.stdout:2/325: truncate d0/d3/d1a/d24/f34 2231833 0 2024-04-12T20:56:30.765 INFO:tasks.workunit.client.1.smithi186.stdout:1/345: creat d14/d1a/d21/d35/f6e x:0 0 0 2024-04-12T20:56:30.770 INFO:tasks.workunit.client.1.smithi186.stdout:5/348: creat d5/db/f76 x:0 0 0 2024-04-12T20:56:30.772 INFO:tasks.workunit.client.1.smithi186.stdout:3/333: dread d1/d3/da/f47 [0,4194304] 0 2024-04-12T20:56:30.772 INFO:tasks.workunit.client.1.smithi186.stdout:1/346: link d14/d1a/d33/f6b d14/d1a/d1d/f6f 0 2024-04-12T20:56:30.775 INFO:tasks.workunit.client.1.smithi186.stdout:5/349: write d5/db/d15/d2a/d2d/f49 [188203,88659] 0 2024-04-12T20:56:30.778 INFO:tasks.workunit.client.1.smithi186.stdout:5/350: chown d5/c25 52464434 1 2024-04-12T20:56:30.778 INFO:tasks.workunit.client.1.smithi186.stdout:3/334: creat d1/d3/da/f6c x:0 0 0 2024-04-12T20:56:30.779 INFO:tasks.workunit.client.1.smithi186.stdout:0/420: dread d0/f6 [4194304,4194304] 0 2024-04-12T20:56:30.784 INFO:tasks.workunit.client.1.smithi186.stdout:7/349: sync 2024-04-12T20:56:30.786 INFO:tasks.workunit.client.1.smithi186.stdout:3/335: rename d1/c36 to d1/d2c/d64/c6d 0 2024-04-12T20:56:30.787 INFO:tasks.workunit.client.1.smithi186.stdout:2/326: write d0/f6 [4642221,66913] 0 2024-04-12T20:56:30.788 INFO:tasks.workunit.client.1.smithi186.stdout:0/421: mkdir d0/d14/d4e/d1e/d84 0 2024-04-12T20:56:30.788 INFO:tasks.workunit.client.1.smithi186.stdout:7/350: fdatasync d7/d55/d5f/f78 0 2024-04-12T20:56:30.790 INFO:tasks.workunit.client.1.smithi186.stdout:3/336: chown d1/d3/f58 3 1 2024-04-12T20:56:30.795 INFO:tasks.workunit.client.1.smithi186.stdout:2/327: symlink d0/d3/d1a/d41/l61 0 2024-04-12T20:56:30.796 INFO:tasks.workunit.client.1.smithi186.stdout:3/337: readlink d1/d3/d21/d22/d24/d28/l37 0 2024-04-12T20:56:30.797 INFO:tasks.workunit.client.1.smithi186.stdout:7/351: rmdir d7/d10/d1c/d5d/d69 0 2024-04-12T20:56:30.797 INFO:tasks.workunit.client.1.smithi186.stdout:1/347: dwrite d14/d1a/d21/f27 [0,4194304] 0 2024-04-12T20:56:30.798 INFO:tasks.workunit.client.1.smithi186.stdout:2/328: readlink d0/dc/l49 0 2024-04-12T20:56:30.801 INFO:tasks.workunit.client.1.smithi186.stdout:5/351: dwrite d5/d44/f6e [4194304,4194304] 0 2024-04-12T20:56:30.802 INFO:tasks.workunit.client.1.smithi186.stdout:3/338: mkdir d1/d3/d21/d22/d34/d6e 0 2024-04-12T20:56:30.805 INFO:tasks.workunit.client.1.smithi186.stdout:5/352: readlink d5/db/d2e/l47 0 2024-04-12T20:56:30.807 INFO:tasks.workunit.client.1.smithi186.stdout:1/348: stat d14/d1a/d1d/l54 0 2024-04-12T20:56:30.808 INFO:tasks.workunit.client.1.smithi186.stdout:3/339: mkdir d1/d3/d49/d4d/d6f 0 2024-04-12T20:56:30.810 INFO:tasks.workunit.client.1.smithi186.stdout:2/329: write d0/d3/d1a/d24/f33 [2465223,102705] 0 2024-04-12T20:56:30.811 INFO:tasks.workunit.client.1.smithi186.stdout:3/340: fsync d1/d3/f11 0 2024-04-12T20:56:30.812 INFO:tasks.workunit.client.1.smithi186.stdout:2/330: chown d0/d3/d1a 1155801 1 2024-04-12T20:56:30.815 INFO:tasks.workunit.client.1.smithi186.stdout:1/349: mknod d14/d1a/d1d/c70 0 2024-04-12T20:56:30.816 INFO:tasks.workunit.client.1.smithi186.stdout:3/341: fsync d1/f1e 0 2024-04-12T20:56:30.819 INFO:tasks.workunit.client.1.smithi186.stdout:5/353: mkdir d5/db/d15/d2a/d2d/d77 0 2024-04-12T20:56:30.821 INFO:tasks.workunit.client.1.smithi186.stdout:0/422: dwrite d0/d5c/f5f [4194304,4194304] 0 2024-04-12T20:56:30.822 INFO:tasks.workunit.client.1.smithi186.stdout:3/342: write d1/d2c/d48/f5a [7631,53161] 0 2024-04-12T20:56:30.827 INFO:tasks.workunit.client.1.smithi186.stdout:2/331: link d0/d3/d1a/d21/f4b d0/d32/d2a/f62 0 2024-04-12T20:56:30.828 INFO:tasks.workunit.client.1.smithi186.stdout:2/332: chown d0/f50 862459 1 2024-04-12T20:56:30.829 INFO:tasks.workunit.client.1.smithi186.stdout:5/354: mknod d5/d44/d5d/c78 0 2024-04-12T20:56:30.831 INFO:tasks.workunit.client.1.smithi186.stdout:0/423: chown d0/d14/d4e/d1e/c3c 125999984 1 2024-04-12T20:56:30.831 INFO:tasks.workunit.client.1.smithi186.stdout:5/355: unlink c3 0 2024-04-12T20:56:30.835 INFO:tasks.workunit.client.1.smithi186.stdout:2/333: dread - d0/d3/d1a/d41/f5c zero size 2024-04-12T20:56:30.835 INFO:tasks.workunit.client.1.smithi186.stdout:3/343: getdents d1/d3/d21/d5e 0 2024-04-12T20:56:30.835 INFO:tasks.workunit.client.1.smithi186.stdout:5/356: dread - d5/db/f2b zero size 2024-04-12T20:56:30.835 INFO:tasks.workunit.client.1.smithi186.stdout:0/424: mkdir d0/d85 0 2024-04-12T20:56:30.835 INFO:tasks.workunit.client.1.smithi186.stdout:3/344: chown d1/d2c/f67 640128666 1 2024-04-12T20:56:30.836 INFO:tasks.workunit.client.1.smithi186.stdout:5/357: readlink d5/db/d71/l1b 0 2024-04-12T20:56:30.837 INFO:tasks.workunit.client.1.smithi186.stdout:2/334: chown d0/d3/d1a/d21/d22/l5a 54360089 1 2024-04-12T20:56:30.838 INFO:tasks.workunit.client.1.smithi186.stdout:1/350: truncate f9 2180828 0 2024-04-12T20:56:30.838 INFO:tasks.workunit.client.1.smithi186.stdout:5/358: chown d5/db/d15/d2a/d65 1697525 1 2024-04-12T20:56:30.839 INFO:tasks.workunit.client.1.smithi186.stdout:3/345: creat d1/d3/da/f70 x:0 0 0 2024-04-12T20:56:30.841 INFO:tasks.workunit.client.1.smithi186.stdout:7/352: dread d7/f12 [0,4194304] 0 2024-04-12T20:56:30.842 INFO:tasks.workunit.client.1.smithi186.stdout:3/346: dread - d1/d3/da/f56 zero size 2024-04-12T20:56:30.843 INFO:tasks.workunit.client.1.smithi186.stdout:7/353: write d7/f74 [106856,128260] 0 2024-04-12T20:56:30.843 INFO:tasks.workunit.client.1.smithi186.stdout:1/351: rmdir d14/d1a/d33 39 2024-04-12T20:56:30.847 INFO:tasks.workunit.client.1.smithi186.stdout:0/425: rename d0/l4c to d0/d14/d4e/d1e/l86 0 2024-04-12T20:56:30.850 INFO:tasks.workunit.client.1.smithi186.stdout:3/347: rename d1/d3/d21/d5c/f40 to d1/d3/d49/d4d/f71 0 2024-04-12T20:56:30.851 INFO:tasks.workunit.client.1.smithi186.stdout:0/426: write d0/d10/d32/d3f/f77 [373568,4588] 0 2024-04-12T20:56:30.855 INFO:tasks.workunit.client.1.smithi186.stdout:0/427: fdatasync d0/d10/d32/d3f/f77 0 2024-04-12T20:56:30.858 INFO:tasks.workunit.client.1.smithi186.stdout:0/428: fsync d0/d5c/f7f 0 2024-04-12T20:56:30.859 INFO:tasks.workunit.client.1.smithi186.stdout:5/359: dwrite f2 [0,4194304] 0 2024-04-12T20:56:30.862 INFO:tasks.workunit.client.1.smithi186.stdout:3/348: rename d1/d2c/c55 to d1/d3/d21/d5e/c72 0 2024-04-12T20:56:30.863 INFO:tasks.workunit.client.1.smithi186.stdout:2/335: write d0/dc/f4c [2254977,23231] 0 2024-04-12T20:56:30.863 INFO:tasks.workunit.client.1.smithi186.stdout:7/354: link d7/d10/d1c/f52 d7/d10/d1c/f7e 0 2024-04-12T20:56:30.870 INFO:tasks.workunit.client.1.smithi186.stdout:2/336: mkdir d0/d3/d63 0 2024-04-12T20:56:30.871 INFO:tasks.workunit.client.1.smithi186.stdout:5/360: symlink d5/db/d15/d6a/l79 0 2024-04-12T20:56:30.874 INFO:tasks.workunit.client.1.smithi186.stdout:0/429: creat d0/f87 x:0 0 0 2024-04-12T20:56:30.875 INFO:tasks.workunit.client.1.smithi186.stdout:2/337: rmdir d0/dc/d2e 39 2024-04-12T20:56:30.876 INFO:tasks.workunit.client.1.smithi186.stdout:7/355: mknod d7/d10/d2d/d40/c7f 0 2024-04-12T20:56:30.879 INFO:tasks.workunit.client.1.smithi186.stdout:5/361: rmdir d5/db/d71 39 2024-04-12T20:56:30.879 INFO:tasks.workunit.client.1.smithi186.stdout:3/349: dwrite d1/fc [0,4194304] 0 2024-04-12T20:56:30.880 INFO:tasks.workunit.client.1.smithi186.stdout:2/338: write d0/d3/d1a/d21/d26/f3d [2614328,121261] 0 2024-04-12T20:56:30.886 INFO:tasks.workunit.client.1.smithi186.stdout:5/362: mkdir d5/db/d15/d2a/d4c/d5b/d7a 0 2024-04-12T20:56:30.886 INFO:tasks.workunit.client.1.smithi186.stdout:7/356: truncate d7/d10/d2d/d40/d4a/f51 537456 0 2024-04-12T20:56:30.886 INFO:tasks.workunit.client.1.smithi186.stdout:2/339: write d0/d3/d1a/d21/d26/f2d [286859,11903] 0 2024-04-12T20:56:30.886 INFO:tasks.workunit.client.1.smithi186.stdout:1/352: dread d14/d1a/d33/d36/d5b/f2d [0,4194304] 0 2024-04-12T20:56:30.887 INFO:tasks.workunit.client.1.smithi186.stdout:0/430: creat d0/f88 x:0 0 0 2024-04-12T20:56:30.887 INFO:tasks.workunit.client.1.smithi186.stdout:3/350: creat d1/d3/d49/d4d/f73 x:0 0 0 2024-04-12T20:56:30.888 INFO:tasks.workunit.client.1.smithi186.stdout:0/431: dread - d0/d5c/f6d zero size 2024-04-12T20:56:30.889 INFO:tasks.workunit.client.1.smithi186.stdout:1/353: write ff [2732200,21499] 0 2024-04-12T20:56:30.892 INFO:tasks.workunit.client.1.smithi186.stdout:3/351: symlink d1/d2c/l74 0 2024-04-12T20:56:30.893 INFO:tasks.workunit.client.1.smithi186.stdout:5/363: dread - d5/db/d15/d2a/d2d/d46/f6c zero size 2024-04-12T20:56:30.896 INFO:tasks.workunit.client.1.smithi186.stdout:0/432: write d0/d14/f19 [1154122,12578] 0 2024-04-12T20:56:30.897 INFO:tasks.workunit.client.1.smithi186.stdout:9/337: sync 2024-04-12T20:56:30.897 INFO:tasks.workunit.client.1.smithi186.stdout:6/326: sync 2024-04-12T20:56:30.897 INFO:tasks.workunit.client.1.smithi186.stdout:8/289: sync 2024-04-12T20:56:30.904 INFO:tasks.workunit.client.1.smithi186.stdout:2/340: dread d0/d32/d2a/f43 [0,4194304] 0 2024-04-12T20:56:30.904 INFO:tasks.workunit.client.1.smithi186.stdout:3/352: rename d1/d3/d21/f41 to d1/d3/d49/d4d/f75 0 2024-04-12T20:56:30.906 INFO:tasks.workunit.client.1.smithi186.stdout:2/341: chown d0/d3 7985118 1 2024-04-12T20:56:30.906 INFO:tasks.workunit.client.1.smithi186.stdout:5/364: creat d5/db/d15/d2a/d4c/f7b x:0 0 0 2024-04-12T20:56:30.908 INFO:tasks.workunit.client.1.smithi186.stdout:8/290: symlink d7/d18/d36/d20/l5b 0 2024-04-12T20:56:30.909 INFO:tasks.workunit.client.1.smithi186.stdout:9/338: mkdir d3/d12/d27/d71/d72 0 2024-04-12T20:56:30.910 INFO:tasks.workunit.client.1.smithi186.stdout:0/433: rename d0/d14/d4e/d1e/f2f to d0/d10/d71/f89 0 2024-04-12T20:56:30.913 INFO:tasks.workunit.client.1.smithi186.stdout:7/357: truncate d7/d10/d1c/f1f 136387 0 2024-04-12T20:56:30.913 INFO:tasks.workunit.client.1.smithi186.stdout:1/354: dread d14/d1a/d33/d36/d5b/f61 [0,4194304] 0 2024-04-12T20:56:30.914 INFO:tasks.workunit.client.1.smithi186.stdout:3/353: mknod d1/d3/c76 0 2024-04-12T20:56:30.915 INFO:tasks.workunit.client.1.smithi186.stdout:7/358: chown d7/f25 33477 1 2024-04-12T20:56:30.915 INFO:tasks.workunit.client.1.smithi186.stdout:0/434: rename d0/d10/d72/l7c to d0/d14/l8a 0 2024-04-12T20:56:30.916 INFO:tasks.workunit.client.1.smithi186.stdout:7/359: fdatasync d7/fc 0 2024-04-12T20:56:30.917 INFO:tasks.workunit.client.1.smithi186.stdout:9/339: creat d3/f73 x:0 0 0 2024-04-12T20:56:30.917 INFO:tasks.workunit.client.1.smithi186.stdout:8/291: mkdir d7/d18/d36/d20/d23/d25/d2e/d5c 0 2024-04-12T20:56:30.917 INFO:tasks.workunit.client.1.smithi186.stdout:3/354: chown d1/d3/d21/d22/d24 246836 1 2024-04-12T20:56:30.918 INFO:tasks.workunit.client.1.smithi186.stdout:1/355: mkdir d14/d1a/d33/d36/d5b/d3e/d71 0 2024-04-12T20:56:30.920 INFO:tasks.workunit.client.1.smithi186.stdout:0/435: unlink d0/d81/f80 0 2024-04-12T20:56:30.920 INFO:tasks.workunit.client.1.smithi186.stdout:8/292: stat d7/de/d10/d2f/f4f 0 2024-04-12T20:56:30.922 INFO:tasks.workunit.client.1.smithi186.stdout:9/340: dread - d3/d2f/d2e/d33/d66/f5f zero size 2024-04-12T20:56:30.923 INFO:tasks.workunit.client.1.smithi186.stdout:7/360: creat d7/d10/d1c/d5d/f80 x:0 0 0 2024-04-12T20:56:30.926 INFO:tasks.workunit.client.1.smithi186.stdout:0/436: mknod d0/d10/d32/c8b 0 2024-04-12T20:56:30.926 INFO:tasks.workunit.client.1.smithi186.stdout:2/342: dread d0/d3/d1a/d24/f33 [0,4194304] 0 2024-04-12T20:56:30.927 INFO:tasks.workunit.client.1.smithi186.stdout:8/293: symlink d7/d18/d36/d1d/d1f/l5d 0 2024-04-12T20:56:30.929 INFO:tasks.workunit.client.1.smithi186.stdout:2/343: fsync d0/f6 0 2024-04-12T20:56:30.931 INFO:tasks.workunit.client.1.smithi186.stdout:8/294: chown d7/de/d10 14320 1 2024-04-12T20:56:30.934 INFO:tasks.workunit.client.1.smithi186.stdout:1/356: getdents d14 0 2024-04-12T20:56:30.935 INFO:tasks.workunit.client.1.smithi186.stdout:0/437: mknod d0/d14/d4e/d1e/c8c 0 2024-04-12T20:56:30.937 INFO:tasks.workunit.client.1.smithi186.stdout:9/341: dread d3/d2f/d2e/d43/d4c/f58 [0,4194304] 0 2024-04-12T20:56:30.941 INFO:tasks.workunit.client.1.smithi186.stdout:3/355: dwrite d1/d2c/d48/f57 [0,4194304] 0 2024-04-12T20:56:30.942 INFO:tasks.workunit.client.1.smithi186.stdout:0/438: creat d0/d10/f8d x:0 0 0 2024-04-12T20:56:30.949 INFO:tasks.workunit.client.1.smithi186.stdout:9/342: write d3/d2f/d54/f6a [1414722,73295] 0 2024-04-12T20:56:30.950 INFO:tasks.workunit.client.1.smithi186.stdout:7/361: write d7/d10/f42 [78106,129308] 0 2024-04-12T20:56:30.950 INFO:tasks.workunit.client.1.smithi186.stdout:3/356: dread - d1/d3/d49/d4d/f73 zero size 2024-04-12T20:56:30.952 INFO:tasks.workunit.client.1.smithi186.stdout:1/357: dread ff [0,4194304] 0 2024-04-12T20:56:30.964 INFO:tasks.workunit.client.1.smithi186.stdout:3/357: creat d1/d3/d21/d5c/f77 x:0 0 0 2024-04-12T20:56:30.971 INFO:tasks.workunit.client.1.smithi186.stdout:8/295: dwrite d7/d18/d36/d20/d23/d25/d29/f2a [0,4194304] 0 2024-04-12T20:56:30.975 INFO:tasks.workunit.client.1.smithi186.stdout:3/358: write d1/d3/d21/d5c/f77 [953907,54831] 0 2024-04-12T20:56:30.979 INFO:tasks.workunit.client.1.smithi186.stdout:0/439: dwrite d0/f88 [0,4194304] 0 2024-04-12T20:56:30.980 INFO:tasks.workunit.client.1.smithi186.stdout:2/344: dwrite d0/d3/d1a/d24/f34 [0,4194304] 0 2024-04-12T20:56:30.982 INFO:tasks.workunit.client.1.smithi186.stdout:8/296: truncate d7/f59 746767 0 2024-04-12T20:56:30.986 INFO:tasks.workunit.client.1.smithi186.stdout:7/362: dread d7/f12 [0,4194304] 0 2024-04-12T20:56:30.991 INFO:tasks.workunit.client.1.smithi186.stdout:9/343: dwrite d3/d2f/d2e/d33/d66/f5f [0,4194304] 0 2024-04-12T20:56:30.991 INFO:tasks.workunit.client.1.smithi186.stdout:0/440: mknod d0/d10/d71/c8e 0 2024-04-12T20:56:30.998 INFO:tasks.workunit.client.1.smithi186.stdout:0/441: creat d0/d10/f8f x:0 0 0 2024-04-12T20:56:31.010 INFO:tasks.workunit.client.1.smithi186.stdout:0/442: write d0/f3e [6696058,71119] 0 2024-04-12T20:56:31.022 INFO:tasks.workunit.client.1.smithi186.stdout:4/411: sync 2024-04-12T20:56:31.024 INFO:tasks.workunit.client.1.smithi186.stdout:0/443: write d0/d81/f5a [713224,90486] 0 2024-04-12T20:56:31.026 INFO:tasks.workunit.client.1.smithi186.stdout:3/359: dwrite d1/d3/f11 [0,4194304] 0 2024-04-12T20:56:31.031 INFO:tasks.workunit.client.1.smithi186.stdout:9/344: dwrite d3/d2f/d2e/d33/d36/f53 [0,4194304] 0 2024-04-12T20:56:31.035 INFO:tasks.workunit.client.1.smithi186.stdout:4/412: creat d0/d59/f85 x:0 0 0 2024-04-12T20:56:31.036 INFO:tasks.workunit.client.1.smithi186.stdout:0/444: fdatasync d0/d10/f8d 0 2024-04-12T20:56:31.041 INFO:tasks.workunit.client.1.smithi186.stdout:2/345: dwrite d0/d32/f38 [0,4194304] 0 2024-04-12T20:56:31.041 INFO:tasks.workunit.client.1.smithi186.stdout:0/445: fsync d0/d14/f35 0 2024-04-12T20:56:31.043 INFO:tasks.workunit.client.1.smithi186.stdout:8/297: dwrite d7/d18/d36/d20/d23/d25/d29/f2a [0,4194304] 0 2024-04-12T20:56:31.045 INFO:tasks.workunit.client.1.smithi186.stdout:7/363: dread d7/fc [0,4194304] 0 2024-04-12T20:56:31.048 INFO:tasks.workunit.client.1.smithi186.stdout:0/446: write d0/d5c/f7f [586752,111569] 0 2024-04-12T20:56:31.050 INFO:tasks.workunit.client.1.smithi186.stdout:2/346: mknod d0/d3/d1a/d21/d22/c64 0 2024-04-12T20:56:31.051 INFO:tasks.workunit.client.1.smithi186.stdout:4/413: rmdir d0/d59 39 2024-04-12T20:56:31.051 INFO:tasks.workunit.client.1.smithi186.stdout:7/364: chown d7/d10/d2d/d40/d4a/c5a 730 1 2024-04-12T20:56:31.054 INFO:tasks.workunit.client.1.smithi186.stdout:8/298: rename d7/d18/d36/d17 to d7/de/d5e 0 2024-04-12T20:56:31.057 INFO:tasks.workunit.client.1.smithi186.stdout:4/414: rmdir d0/dc/d33/d5c 39 2024-04-12T20:56:31.057 INFO:tasks.workunit.client.1.smithi186.stdout:2/347: write d0/d3/d1a/d41/f5e [585512,15854] 0 2024-04-12T20:56:31.060 INFO:tasks.workunit.client.1.smithi186.stdout:3/360: link d1/d3/da/c15 d1/c78 0 2024-04-12T20:56:31.062 INFO:tasks.workunit.client.1.smithi186.stdout:2/348: rename d0/c5 to d0/d3/d1a/d27/c65 0 2024-04-12T20:56:31.063 INFO:tasks.workunit.client.1.smithi186.stdout:7/365: dread d7/d10/f41 [0,4194304] 0 2024-04-12T20:56:31.065 INFO:tasks.workunit.client.1.smithi186.stdout:8/299: getdents d7/d18/d36/d20 0 2024-04-12T20:56:31.066 INFO:tasks.workunit.client.1.smithi186.stdout:4/415: getdents d0/dc/d33/d5c 0 2024-04-12T20:56:31.067 INFO:tasks.workunit.client.1.smithi186.stdout:8/300: stat d7/d18/d36/d20/f4a 0 2024-04-12T20:56:31.069 INFO:tasks.workunit.client.1.smithi186.stdout:3/361: creat d1/d3/d21/d22/f79 x:0 0 0 2024-04-12T20:56:31.070 INFO:tasks.workunit.client.1.smithi186.stdout:2/349: rename d0/dc/l49 to d0/dc/d2e/l66 0 2024-04-12T20:56:31.074 INFO:tasks.workunit.client.1.smithi186.stdout:8/301: creat d7/d18/d36/d20/d23/f5f x:0 0 0 2024-04-12T20:56:31.074 INFO:tasks.workunit.client.1.smithi186.stdout:3/362: truncate d1/d3/da/f6c 770110 0 2024-04-12T20:56:31.074 INFO:tasks.workunit.client.1.smithi186.stdout:7/366: rename d7/d10/d2d/d40/c79 to d7/d10/d2d/d40/d4a/c81 0 2024-04-12T20:56:31.074 INFO:tasks.workunit.client.1.smithi186.stdout:2/350: creat d0/d3/d1a/d24/f67 x:0 0 0 2024-04-12T20:56:31.074 INFO:tasks.workunit.client.1.smithi186.stdout:4/416: symlink d0/d21/d3d/d48/d61/l86 0 2024-04-12T20:56:31.075 INFO:tasks.workunit.client.1.smithi186.stdout:2/351: chown d0/f6 7797 1 2024-04-12T20:56:31.075 INFO:tasks.workunit.client.1.smithi186.stdout:8/302: rmdir d7/d18/d36/d1d/d1f/d26 39 2024-04-12T20:56:31.076 INFO:tasks.workunit.client.1.smithi186.stdout:2/352: stat d0/d32/d2a/f43 0 2024-04-12T20:56:31.076 INFO:tasks.workunit.client.1.smithi186.stdout:3/363: creat d1/d3/d49/d4d/f7a x:0 0 0 2024-04-12T20:56:31.078 INFO:tasks.workunit.client.1.smithi186.stdout:7/367: symlink d7/d10/d2d/d67/l82 0 2024-04-12T20:56:31.079 INFO:tasks.workunit.client.1.smithi186.stdout:4/417: dread - d0/d59/f85 zero size 2024-04-12T20:56:31.080 INFO:tasks.workunit.client.1.smithi186.stdout:7/368: chown d7/d10/d1c/f20 913 1 2024-04-12T20:56:31.082 INFO:tasks.workunit.client.1.smithi186.stdout:3/364: dread d1/d3/da/f6c [0,4194304] 0 2024-04-12T20:56:31.082 INFO:tasks.workunit.client.1.smithi186.stdout:8/303: fdatasync d7/d18/d36/d1d/f4d 0 2024-04-12T20:56:31.086 INFO:tasks.workunit.client.1.smithi186.stdout:7/369: chown d7/d10/d2d/l3e 20 1 2024-04-12T20:56:31.087 INFO:tasks.workunit.client.1.smithi186.stdout:7/370: chown d7/d10/d2d/d40/d4a/l62 367 1 2024-04-12T20:56:31.088 INFO:tasks.workunit.client.1.smithi186.stdout:2/353: dread d0/dc/f40 [0,4194304] 0 2024-04-12T20:56:31.088 INFO:tasks.workunit.client.1.smithi186.stdout:4/418: mkdir d0/dc/d64/d87 0 2024-04-12T20:56:31.091 INFO:tasks.workunit.client.1.smithi186.stdout:7/371: mknod d7/d10/d2d/d67/c83 0 2024-04-12T20:56:31.092 INFO:tasks.workunit.client.1.smithi186.stdout:8/304: mknod d7/d18/d53/c60 0 2024-04-12T20:56:31.093 INFO:tasks.workunit.client.1.smithi186.stdout:8/305: chown d7/de/d10/d2f 7178 1 2024-04-12T20:56:31.094 INFO:tasks.workunit.client.1.smithi186.stdout:2/354: mknod d0/d3/d1a/d27/c68 0 2024-04-12T20:56:31.094 INFO:tasks.workunit.client.1.smithi186.stdout:4/419: rmdir d0/d21/d3d/d48/d61 39 2024-04-12T20:56:31.096 INFO:tasks.workunit.client.1.smithi186.stdout:4/420: dread - d0/dc/d33/d5c/f6c zero size 2024-04-12T20:56:31.096 INFO:tasks.workunit.client.1.smithi186.stdout:7/372: mkdir d7/d55/d5f/d4d/d84 0 2024-04-12T20:56:31.100 INFO:tasks.workunit.client.1.smithi186.stdout:2/355: mkdir d0/d32/d69 0 2024-04-12T20:56:31.101 INFO:tasks.workunit.client.1.smithi186.stdout:3/365: rename d1/d3/f11 to d1/d3/d21/d22/f7b 0 2024-04-12T20:56:31.104 INFO:tasks.workunit.client.1.smithi186.stdout:4/421: readlink d0/dc/d64/l6e 0 2024-04-12T20:56:31.105 INFO:tasks.workunit.client.1.smithi186.stdout:2/356: creat d0/d3/d1a/d21/d26/f6a x:0 0 0 2024-04-12T20:56:31.107 INFO:tasks.workunit.client.1.smithi186.stdout:3/366: mkdir d1/d2c/d54/d7c 0 2024-04-12T20:56:31.112 INFO:tasks.workunit.client.1.smithi186.stdout:3/367: symlink d1/d3/d21/d22/d24/d28/l7d 0 2024-04-12T20:56:31.113 INFO:tasks.workunit.client.1.smithi186.stdout:2/357: mknod d0/d3/d1a/d27/c6b 0 2024-04-12T20:56:31.115 INFO:tasks.workunit.client.1.smithi186.stdout:7/373: dwrite d7/d10/d1c/d5d/f80 [0,4194304] 0 2024-04-12T20:56:31.120 INFO:tasks.workunit.client.1.smithi186.stdout:7/374: readlink d7/d10/l75 0 2024-04-12T20:56:31.123 INFO:tasks.workunit.client.1.smithi186.stdout:3/368: getdents d1/d3/d49 0 2024-04-12T20:56:31.124 INFO:tasks.workunit.client.1.smithi186.stdout:8/306: dread f2 [0,4194304] 0 2024-04-12T20:56:31.124 INFO:tasks.workunit.client.1.smithi186.stdout:7/375: stat d7/d10/f13 0 2024-04-12T20:56:31.125 INFO:tasks.workunit.client.1.smithi186.stdout:2/358: creat d0/d3/d1a/d21/d22/f6c x:0 0 0 2024-04-12T20:56:31.130 INFO:tasks.workunit.client.1.smithi186.stdout:8/307: mknod d7/de/d5e/c61 0 2024-04-12T20:56:31.131 INFO:tasks.workunit.client.1.smithi186.stdout:8/308: chown d7/d18/d36 153 1 2024-04-12T20:56:31.131 INFO:tasks.workunit.client.1.smithi186.stdout:4/422: dwrite d0/dc/d33/f6b [0,4194304] 0 2024-04-12T20:56:31.132 INFO:tasks.workunit.client.1.smithi186.stdout:7/376: creat d7/d55/d71/f85 x:0 0 0 2024-04-12T20:56:31.133 INFO:tasks.workunit.client.1.smithi186.stdout:7/377: stat d7/d10/l3b 0 2024-04-12T20:56:31.134 INFO:tasks.workunit.client.1.smithi186.stdout:2/359: rename d0/dc/d2e/l31 to d0/d3e/l6d 0 2024-04-12T20:56:31.135 INFO:tasks.workunit.client.1.smithi186.stdout:4/423: chown d0/dc/d39/c3f 409514 1 2024-04-12T20:56:31.135 INFO:tasks.workunit.client.1.smithi186.stdout:8/309: chown d7/d18/d36/f1b 13476196 1 2024-04-12T20:56:31.136 INFO:tasks.workunit.client.1.smithi186.stdout:3/369: creat d1/d3/f7e x:0 0 0 2024-04-12T20:56:31.137 INFO:tasks.workunit.client.1.smithi186.stdout:2/360: chown d0/dc/d2e/l56 23005 1 2024-04-12T20:56:31.138 INFO:tasks.workunit.client.1.smithi186.stdout:7/378: mkdir d7/d10/d2d/d86 0 2024-04-12T20:56:31.142 INFO:tasks.workunit.client.1.smithi186.stdout:3/370: creat d1/d3/d21/d22/d24/d2f/f7f x:0 0 0 2024-04-12T20:56:31.147 INFO:tasks.workunit.client.1.smithi186.stdout:4/424: rename d0/dc/d33/d5c to d0/dc/d88 0 2024-04-12T20:56:31.148 INFO:tasks.workunit.client.1.smithi186.stdout:7/379: readlink d7/d10/d2d/d40/d4a/d5c/l72 0 2024-04-12T20:56:31.153 INFO:tasks.workunit.client.1.smithi186.stdout:7/380: unlink d7/d10/f46 0 2024-04-12T20:56:31.155 INFO:tasks.workunit.client.1.smithi186.stdout:2/361: dread d0/d3/d1a/d27/f59 [0,4194304] 0 2024-04-12T20:56:31.156 INFO:tasks.workunit.client.1.smithi186.stdout:7/381: symlink d7/d10/d2d/d40/d4a/d6c/l87 0 2024-04-12T20:56:31.157 INFO:tasks.workunit.client.1.smithi186.stdout:2/362: read d0/dc/f40 [2697277,17487] 0 2024-04-12T20:56:31.159 INFO:tasks.workunit.client.1.smithi186.stdout:8/310: dread d7/d18/d36/d20/d23/d25/f5a [0,4194304] 0 2024-04-12T20:56:31.159 INFO:tasks.workunit.client.1.smithi186.stdout:7/382: fsync d7/d10/f42 0 2024-04-12T20:56:31.161 INFO:tasks.workunit.client.1.smithi186.stdout:8/311: creat d7/de/f62 x:0 0 0 2024-04-12T20:56:31.164 INFO:tasks.workunit.client.1.smithi186.stdout:2/363: mknod d0/d3/d1a/d48/c6e 0 2024-04-12T20:56:31.164 INFO:tasks.workunit.client.1.smithi186.stdout:4/425: dwrite d0/dc/d33/f1a [0,4194304] 0 2024-04-12T20:56:31.166 INFO:tasks.workunit.client.1.smithi186.stdout:2/364: dread - d0/d3/d1a/d41/f44 zero size 2024-04-12T20:56:31.166 INFO:tasks.workunit.client.1.smithi186.stdout:7/383: mknod d7/d10/d2d/c88 0 2024-04-12T20:56:31.171 INFO:tasks.workunit.client.1.smithi186.stdout:7/384: read d7/d10/d1c/f21 [133237,119768] 0 2024-04-12T20:56:31.173 INFO:tasks.workunit.client.1.smithi186.stdout:2/365: chown d0/d3e/l6d 71 1 2024-04-12T20:56:31.175 INFO:tasks.workunit.client.1.smithi186.stdout:2/366: write d0/dc/f4c [983286,85573] 0 2024-04-12T20:56:31.181 INFO:tasks.workunit.client.1.smithi186.stdout:7/385: creat d7/d10/d2d/f89 x:0 0 0 2024-04-12T20:56:31.183 INFO:tasks.workunit.client.1.smithi186.stdout:4/426: truncate d0/dc/d39/f3a 541555 0 2024-04-12T20:56:31.183 INFO:tasks.workunit.client.1.smithi186.stdout:8/312: dwrite d7/d18/d36/d20/d23/d25/f56 [0,4194304] 0 2024-04-12T20:56:31.185 INFO:tasks.workunit.client.1.smithi186.stdout:7/386: mkdir d7/d55/d5f/d4d/d8a 0 2024-04-12T20:56:31.194 INFO:tasks.workunit.client.1.smithi186.stdout:7/387: symlink d7/d55/d5f/d4d/l8b 0 2024-04-12T20:56:31.196 INFO:tasks.workunit.client.1.smithi186.stdout:8/313: creat d7/d18/d36/d1d/d1f/d26/f63 x:0 0 0 2024-04-12T20:56:31.201 INFO:tasks.workunit.client.1.smithi186.stdout:2/367: dwrite d0/d3/d1a/d21/d26/f6a [0,4194304] 0 2024-04-12T20:56:31.204 INFO:tasks.workunit.client.1.smithi186.stdout:8/314: symlink d7/de/l64 0 2024-04-12T20:56:31.206 INFO:tasks.workunit.client.1.smithi186.stdout:7/388: mkdir d7/d10/d1c/d8c 0 2024-04-12T20:56:31.212 INFO:tasks.workunit.client.1.smithi186.stdout:8/315: symlink d7/d18/d36/d1d/l65 0 2024-04-12T20:56:31.213 INFO:tasks.workunit.client.1.smithi186.stdout:4/427: dwrite d0/fd [0,4194304] 0 2024-04-12T20:56:31.220 INFO:tasks.workunit.client.1.smithi186.stdout:4/428: creat d0/dc/d64/f89 x:0 0 0 2024-04-12T20:56:31.222 INFO:tasks.workunit.client.1.smithi186.stdout:4/429: stat d0/dc/d33/f17 0 2024-04-12T20:56:31.222 INFO:tasks.workunit.client.1.smithi186.stdout:7/389: link d7/d10/d2d/d40/d4a/d6c/f7d d7/d10/d2d/d40/d4a/d5c/f8d 0 2024-04-12T20:56:31.223 INFO:tasks.workunit.client.1.smithi186.stdout:4/430: readlink d0/dc/d1f/l3e 0 2024-04-12T20:56:31.224 INFO:tasks.workunit.client.1.smithi186.stdout:4/431: chown d0/dc/d39/c44 63295 1 2024-04-12T20:56:31.226 INFO:tasks.workunit.client.1.smithi186.stdout:8/316: link d7/de/d10/d2f/f31 d7/d18/d36/d20/d23/d25/f66 0 2024-04-12T20:56:31.228 INFO:tasks.workunit.client.1.smithi186.stdout:7/390: creat d7/d10/d1c/f8e x:0 0 0 2024-04-12T20:56:31.229 INFO:tasks.workunit.client.1.smithi186.stdout:4/432: creat d0/dc/d33/f8a x:0 0 0 2024-04-12T20:56:31.231 INFO:tasks.workunit.client.1.smithi186.stdout:8/317: mkdir d7/d18/d36/d20/d23/d25/d2e/d67 0 2024-04-12T20:56:31.231 INFO:tasks.workunit.client.1.smithi186.stdout:4/433: fdatasync d0/dc/d33/f36 0 2024-04-12T20:56:31.232 INFO:tasks.workunit.client.1.smithi186.stdout:7/391: symlink d7/d55/d5f/d4d/l8f 0 2024-04-12T20:56:31.234 INFO:tasks.workunit.client.1.smithi186.stdout:8/318: unlink d7/d18/c41 0 2024-04-12T20:56:31.238 INFO:tasks.workunit.client.1.smithi186.stdout:4/434: getdents d0/dc/d39 0 2024-04-12T20:56:31.239 INFO:tasks.workunit.client.1.smithi186.stdout:5/365: sync 2024-04-12T20:56:31.239 INFO:tasks.workunit.client.1.smithi186.stdout:6/327: sync 2024-04-12T20:56:31.241 INFO:tasks.workunit.client.1.smithi186.stdout:2/368: dread d0/d3/d1a/d21/f4b [0,4194304] 0 2024-04-12T20:56:31.243 INFO:tasks.workunit.client.1.smithi186.stdout:6/328: write d9/d43/d59/d5d/f46 [2802461,110723] 0 2024-04-12T20:56:31.244 INFO:tasks.workunit.client.1.smithi186.stdout:4/435: truncate d0/dc/d33/f47 5037680 0 2024-04-12T20:56:31.246 INFO:tasks.workunit.client.1.smithi186.stdout:5/366: creat d5/db/d15/d2a/d2d/d48/f7c x:0 0 0 2024-04-12T20:56:31.248 INFO:tasks.workunit.client.1.smithi186.stdout:2/369: creat d0/d32/f6f x:0 0 0 2024-04-12T20:56:31.255 INFO:tasks.workunit.client.1.smithi186.stdout:4/436: fdatasync d0/d21/f40 0 2024-04-12T20:56:31.256 INFO:tasks.workunit.client.1.smithi186.stdout:5/367: dread d5/db/d71/f52 [0,4194304] 0 2024-04-12T20:56:31.257 INFO:tasks.workunit.client.1.smithi186.stdout:7/392: dwrite d7/d55/d5f/d4d/f6d [0,4194304] 0 2024-04-12T20:56:31.261 INFO:tasks.workunit.client.1.smithi186.stdout:2/370: creat d0/d3/d1a/d27/f70 x:0 0 0 2024-04-12T20:56:31.264 INFO:tasks.workunit.client.1.smithi186.stdout:2/371: write d0/d3/d1a/d27/f59 [2049403,112280] 0 2024-04-12T20:56:31.264 INFO:tasks.workunit.client.1.smithi186.stdout:4/437: mknod d0/d26/c8b 0 2024-04-12T20:56:31.266 INFO:tasks.workunit.client.1.smithi186.stdout:5/368: truncate d5/db/d15/d2a/d2d/d46/f6c 104381 0 2024-04-12T20:56:31.270 INFO:tasks.workunit.client.1.smithi186.stdout:6/329: truncate d9/d43/d59/d5d/f46 7389838 0 2024-04-12T20:56:31.271 INFO:tasks.workunit.client.1.smithi186.stdout:2/372: creat d0/d32/d2a/f71 x:0 0 0 2024-04-12T20:56:31.271 INFO:tasks.workunit.client.1.smithi186.stdout:8/319: dwrite d7/de/d10/d2f/f31 [0,4194304] 0 2024-04-12T20:56:31.277 INFO:tasks.workunit.client.1.smithi186.stdout:5/369: write d5/d44/f67 [114334,32552] 0 2024-04-12T20:56:31.278 INFO:tasks.workunit.client.1.smithi186.stdout:6/330: read d9/f2e [13395,50731] 0 2024-04-12T20:56:31.280 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:56:31 smithi186 ceph-mon[36597]: pgmap v48: 65 pgs: 65 active+clean; 2.3 GiB data, 8.2 GiB used, 528 GiB / 536 GiB avail; 40 MiB/s rd, 127 MiB/s wr, 321 op/s 2024-04-12T20:56:31.280 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:56:31 smithi186 ceph-mon[36597]: mgrmap e25: smithi186.kqcfxk(active, since 90s), standbys: smithi018.xjzwga 2024-04-12T20:56:31.282 INFO:tasks.workunit.client.1.smithi186.stdout:8/320: dread - d7/d18/d36/d20/d23/f45 zero size 2024-04-12T20:56:31.285 INFO:tasks.workunit.client.1.smithi186.stdout:7/393: dwrite d7/d10/f13 [0,4194304] 0 2024-04-12T20:56:31.290 INFO:tasks.workunit.client.1.smithi186.stdout:5/370: mkdir d5/db/d15/d2a/d2d/d46/d7d 0 2024-04-12T20:56:31.290 INFO:tasks.workunit.client.1.smithi186.stdout:6/331: rmdir d9 39 2024-04-12T20:56:31.291 INFO:tasks.workunit.client.1.smithi186.stdout:8/321: mkdir d7/de/d68 0 2024-04-12T20:56:31.297 INFO:tasks.workunit.client.1.smithi186.stdout:4/438: dwrite d0/f24 [0,4194304] 0 2024-04-12T20:56:31.298 INFO:tasks.workunit.client.1.smithi186.stdout:1/358: sync 2024-04-12T20:56:31.298 INFO:tasks.workunit.client.1.smithi186.stdout:3/371: sync 2024-04-12T20:56:31.298 INFO:tasks.workunit.client.1.smithi186.stdout:9/345: sync 2024-04-12T20:56:31.298 INFO:tasks.workunit.client.1.smithi186.stdout:0/447: sync 2024-04-12T20:56:31.299 INFO:tasks.workunit.client.1.smithi186.stdout:5/371: write d5/db/f11 [2936708,93450] 0 2024-04-12T20:56:31.305 INFO:tasks.workunit.client.1.smithi186.stdout:0/448: chown d0/d5c/f6d 126948943 1 2024-04-12T20:56:31.305 INFO:tasks.workunit.client.1.smithi186.stdout:3/372: dread - d1/d3/d49/d4d/f7a zero size 2024-04-12T20:56:31.308 INFO:tasks.workunit.client.1.smithi186.stdout:6/332: rename d9/f33 to d9/d43/f77 0 2024-04-12T20:56:31.316 INFO:tasks.workunit.client.1.smithi186.stdout:4/439: chown d0/d21/d3d/f84 83370 1 2024-04-12T20:56:31.317 INFO:tasks.workunit.client.1.smithi186.stdout:3/373: fdatasync d1/d2c/d48/f5a 0 2024-04-12T20:56:31.320 INFO:tasks.workunit.client.1.smithi186.stdout:2/373: dread d0/d3/d1a/d24/f34 [0,4194304] 0 2024-04-12T20:56:31.321 INFO:tasks.workunit.client.1.smithi186.stdout:5/372: unlink d5/db/d15/d2a/d2d/d46/d55/c63 0 2024-04-12T20:56:31.322 INFO:tasks.workunit.client.1.smithi186.stdout:7/394: rename d7/d10/d2d/d40/d4a/c56 to d7/d10/d2d/d86/c90 0 2024-04-12T20:56:31.322 INFO:tasks.workunit.client.1.smithi186.stdout:8/322: getdents d7/d18/d36/d20/d23/d25/d2e 0 2024-04-12T20:56:31.323 INFO:tasks.workunit.client.1.smithi186.stdout:3/374: dread - d1/d3/d21/d5c/f69 zero size 2024-04-12T20:56:31.324 INFO:tasks.workunit.client.1.smithi186.stdout:6/333: symlink d9/d43/d50/l78 0 2024-04-12T20:56:31.326 INFO:tasks.workunit.client.1.smithi186.stdout:5/373: write d5/db/d71/f61 [4313755,78365] 0 2024-04-12T20:56:31.329 INFO:tasks.workunit.client.1.smithi186.stdout:2/374: rmdir d0/d3/d1a/d21/d26 39 2024-04-12T20:56:31.329 INFO:tasks.workunit.client.1.smithi186.stdout:7/395: fdatasync d7/d10/d2d/d40/d4a/f51 0 2024-04-12T20:56:31.330 INFO:tasks.workunit.client.1.smithi186.stdout:4/440: mknod d0/dc/d64/d87/c8c 0 2024-04-12T20:56:31.335 INFO:tasks.workunit.client.1.smithi186.stdout:2/375: stat d0/d32/d69 0 2024-04-12T20:56:31.335 INFO:tasks.workunit.client.1.smithi186.stdout:7/396: write d7/d10/f19 [1144800,94865] 0 2024-04-12T20:56:31.337 INFO:tasks.workunit.client.1.smithi186.stdout:1/359: dwrite d14/d1a/d33/d36/d5b/d3e/f52 [0,4194304] 0 2024-04-12T20:56:31.342 INFO:tasks.workunit.client.1.smithi186.stdout:5/374: symlink d5/db/d15/d2a/d2d/l7e 0 2024-04-12T20:56:31.344 INFO:tasks.workunit.client.1.smithi186.stdout:1/360: read d14/d1a/d33/d36/d5b/f2d [1803284,105137] 0 2024-04-12T20:56:31.345 INFO:tasks.workunit.client.1.smithi186.stdout:2/376: rmdir d0/d3/d1a/d41 39 2024-04-12T20:56:31.346 INFO:tasks.workunit.client.1.smithi186.stdout:4/441: mknod d0/d21/d3d/d48/d61/c8d 0 2024-04-12T20:56:31.347 INFO:tasks.workunit.client.1.smithi186.stdout:1/361: chown d14/d1a/d21/l2b 1627150 1 2024-04-12T20:56:31.347 INFO:tasks.workunit.client.1.smithi186.stdout:6/334: rmdir d9/d4c/d70 0 2024-04-12T20:56:31.351 INFO:tasks.workunit.client.1.smithi186.stdout:5/375: chown d5/db/d15/d2a/d2d/f58 0 1 2024-04-12T20:56:31.351 INFO:tasks.workunit.client.1.smithi186.stdout:7/397: getdents d7/d55/d5f/d4d/d6a 0 2024-04-12T20:56:31.355 INFO:tasks.workunit.client.1.smithi186.stdout:5/376: stat d5/db/c69 0 2024-04-12T20:56:31.357 INFO:tasks.workunit.client.1.smithi186.stdout:2/377: write d0/d32/d2a/f62 [5691734,116954] 0 2024-04-12T20:56:31.358 INFO:tasks.workunit.client.1.smithi186.stdout:6/335: fsync d9/f10 0 2024-04-12T20:56:31.360 INFO:tasks.workunit.client.1.smithi186.stdout:1/362: creat d14/d1a/d1d/f72 x:0 0 0 2024-04-12T20:56:31.362 INFO:tasks.workunit.client.1.smithi186.stdout:5/377: read - d5/db/d15/d2a/d4c/f7b zero size 2024-04-12T20:56:31.363 INFO:tasks.workunit.client.1.smithi186.stdout:8/323: dwrite d7/de/f11 [0,4194304] 0 2024-04-12T20:56:31.366 INFO:tasks.workunit.client.1.smithi186.stdout:7/398: chown d7/d10/d1c/c2f 5320 1 2024-04-12T20:56:31.368 INFO:tasks.workunit.client.1.smithi186.stdout:2/378: write d0/dc/f4c [1770439,10164] 0 2024-04-12T20:56:31.380 INFO:tasks.workunit.client.1.smithi186.stdout:5/378: mkdir d5/db/d15/d2a/d6d/d7f 0 2024-04-12T20:56:31.381 INFO:tasks.workunit.client.1.smithi186.stdout:8/324: fdatasync d7/d18/d36/d20/d23/d25/d29/f57 0 2024-04-12T20:56:31.381 INFO:tasks.workunit.client.1.smithi186.stdout:0/449: dread d0/d10/f2a [0,4194304] 0 2024-04-12T20:56:31.385 INFO:tasks.workunit.client.1.smithi186.stdout:6/336: unlink d9/d43/d2d/d4d/c66 0 2024-04-12T20:56:31.387 INFO:tasks.workunit.client.1.smithi186.stdout:2/379: write d0/dc/f5d [86898,31548] 0 2024-04-12T20:56:31.387 INFO:tasks.workunit.client.1.smithi186.stdout:3/375: dwrite d1/d3/da/f56 [0,4194304] 0 2024-04-12T20:56:31.389 INFO:tasks.workunit.client.1.smithi186.stdout:7/399: rmdir d7/d10/d2d/d40/d4a 39 2024-04-12T20:56:31.391 INFO:tasks.workunit.client.1.smithi186.stdout:8/325: write d7/d18/d36/fa [2630644,14923] 0 2024-04-12T20:56:31.394 INFO:tasks.workunit.client.1.smithi186.stdout:4/442: dwrite d0/dc/d33/f4 [0,4194304] 0 2024-04-12T20:56:31.400 INFO:tasks.workunit.client.1.smithi186.stdout:6/337: creat d9/d43/d50/f79 x:0 0 0 2024-04-12T20:56:31.400 INFO:tasks.workunit.client.1.smithi186.stdout:3/376: read d1/d3/f58 [59799,76600] 0 2024-04-12T20:56:31.400 INFO:tasks.workunit.client.1.smithi186.stdout:2/380: truncate d0/dc/f40 3416148 0 2024-04-12T20:56:31.406 INFO:tasks.workunit.client.1.smithi186.stdout:8/326: write d7/de/d10/f27 [2837506,101016] 0 2024-04-12T20:56:31.409 INFO:tasks.workunit.client.1.smithi186.stdout:3/377: chown d1/d2c/d48/f5a 3 1 2024-04-12T20:56:31.409 INFO:tasks.workunit.client.1.smithi186.stdout:7/400: mknod d7/d55/d5f/d4d/c91 0 2024-04-12T20:56:31.411 INFO:tasks.workunit.client.1.smithi186.stdout:8/327: rmdir d7/d18/d36/d20/d23/d25/d29 39 2024-04-12T20:56:31.411 INFO:tasks.workunit.client.1.smithi186.stdout:9/346: sync 2024-04-12T20:56:31.417 INFO:tasks.workunit.client.1.smithi186.stdout:3/378: fdatasync d1/d3/d21/f65 0 2024-04-12T20:56:31.417 INFO:tasks.workunit.client.1.smithi186.stdout:2/381: creat d0/f72 x:0 0 0 2024-04-12T20:56:31.420 INFO:tasks.workunit.client.1.smithi186.stdout:9/347: unlink d3/d2f/f55 0 2024-04-12T20:56:31.422 INFO:tasks.workunit.client.1.smithi186.stdout:7/401: creat d7/d55/d5f/f92 x:0 0 0 2024-04-12T20:56:31.423 INFO:tasks.workunit.client.1.smithi186.stdout:2/382: mkdir d0/dc/d2e/d73 0 2024-04-12T20:56:31.426 INFO:tasks.workunit.client.1.smithi186.stdout:8/328: unlink d7/fd 0 2024-04-12T20:56:31.437 INFO:tasks.workunit.client.1.smithi186.stdout:2/383: chown d0/d3/d1a/d27/c6b 820582 1 2024-04-12T20:56:31.437 INFO:tasks.workunit.client.1.smithi186.stdout:9/348: write d3/fb [2719926,71963] 0 2024-04-12T20:56:31.438 INFO:tasks.workunit.client.1.smithi186.stdout:6/338: write d9/f2e [362668,79174] 0 2024-04-12T20:56:31.443 INFO:tasks.workunit.client.1.smithi186.stdout:5/379: dwrite d5/db/d15/d2a/d2d/d46/d55/f3d [0,4194304] 0 2024-04-12T20:56:31.443 INFO:tasks.workunit.client.1.smithi186.stdout:3/379: link d1/d3/d21/d22/l3b d1/d2c/d54/d7c/l80 0 2024-04-12T20:56:31.443 INFO:tasks.workunit.client.1.smithi186.stdout:1/363: dread d14/d1a/f37 [0,4194304] 0 2024-04-12T20:56:31.447 INFO:tasks.workunit.client.1.smithi186.stdout:2/384: read d0/dc/f40 [2303860,81654] 0 2024-04-12T20:56:31.449 INFO:tasks.workunit.client.1.smithi186.stdout:7/402: creat d7/d10/d2d/d86/f93 x:0 0 0 2024-04-12T20:56:31.450 INFO:tasks.workunit.client.1.smithi186.stdout:6/339: mkdir d9/d5e/d7a 0 2024-04-12T20:56:31.454 INFO:tasks.workunit.client.1.smithi186.stdout:2/385: stat d0/d32/d2a/f43 0 2024-04-12T20:56:31.458 INFO:tasks.workunit.client.1.smithi186.stdout:4/443: dwrite d0/d21/f32 [0,4194304] 0 2024-04-12T20:56:31.467 INFO:tasks.workunit.client.1.smithi186.stdout:0/450: dwrite d0/d14/d4e/f33 [0,4194304] 0 2024-04-12T20:56:31.467 INFO:tasks.workunit.client.1.smithi186.stdout:5/380: mkdir d5/db/d15/d2a/d6d/d7f/d80 0 2024-04-12T20:56:31.469 INFO:tasks.workunit.client.1.smithi186.stdout:9/349: truncate d3/fd 5228754 0 2024-04-12T20:56:31.471 INFO:tasks.workunit.client.1.smithi186.stdout:4/444: write d0/dc/d39/f79 [189694,60025] 0 2024-04-12T20:56:31.480 INFO:tasks.workunit.client.1.smithi186.stdout:0/451: symlink d0/d14/d4e/d50/l90 0 2024-04-12T20:56:31.481 INFO:tasks.workunit.client.1.smithi186.stdout:2/386: unlink d0/d3/d1a/d41/f5e 0 2024-04-12T20:56:31.482 INFO:tasks.workunit.client.1.smithi186.stdout:7/403: creat d7/d10/d2d/d40/d4a/d5c/f94 x:0 0 0 2024-04-12T20:56:31.482 INFO:tasks.workunit.client.1.smithi186.stdout:5/381: creat d5/db/d71/f81 x:0 0 0 2024-04-12T20:56:31.482 INFO:tasks.workunit.client.1.smithi186.stdout:6/340: truncate d9/d43/d59/d5d/f48 1145910 0 2024-04-12T20:56:31.482 INFO:tasks.workunit.client.1.smithi186.stdout:9/350: stat d3/d2f/d2e/d33/d36 0 2024-04-12T20:56:31.485 INFO:tasks.workunit.client.1.smithi186.stdout:1/364: getdents d14/d1a/d33/d36/d5b 0 2024-04-12T20:56:31.492 INFO:tasks.workunit.client.1.smithi186.stdout:6/341: chown d9/d43/d50/f55 25 1 2024-04-12T20:56:31.492 INFO:tasks.workunit.client.1.smithi186.stdout:0/452: dread - d0/d5c/d65/f7b zero size 2024-04-12T20:56:31.494 INFO:tasks.workunit.client.1.smithi186.stdout:9/351: write d3/d2f/d2e/f2d [1175428,47708] 0 2024-04-12T20:56:31.498 INFO:tasks.workunit.client.1.smithi186.stdout:5/382: mkdir d5/db/d15/d2a/d2d/d82 0 2024-04-12T20:56:31.500 INFO:tasks.workunit.client.1.smithi186.stdout:7/404: creat d7/d10/f95 x:0 0 0 2024-04-12T20:56:31.506 INFO:tasks.workunit.client.1.smithi186.stdout:2/387: mknod d0/d3/d1a/d21/d22/c74 0 2024-04-12T20:56:31.506 INFO:tasks.workunit.client.1.smithi186.stdout:3/380: dwrite d1/d3/d49/d4d/f73 [0,4194304] 0 2024-04-12T20:56:31.515 INFO:tasks.workunit.client.1.smithi186.stdout:4/445: dwrite d0/d59/f85 [0,4194304] 0 2024-04-12T20:56:31.517 INFO:tasks.workunit.client.1.smithi186.stdout:3/381: mkdir d1/d2c/d54/d81 0 2024-04-12T20:56:31.517 INFO:tasks.workunit.client.1.smithi186.stdout:1/365: link d14/d1a/f1b d14/d1a/d33/d36/d5b/d3e/f73 0 2024-04-12T20:56:31.520 INFO:tasks.workunit.client.1.smithi186.stdout:8/329: dread d7/d18/f4c [0,4194304] 0 2024-04-12T20:56:31.521 INFO:tasks.workunit.client.1.smithi186.stdout:9/352: mknod d3/d12/d6f/c74 0 2024-04-12T20:56:31.523 INFO:tasks.workunit.client.1.smithi186.stdout:5/383: dread d5/db/d71/f52 [0,4194304] 0 2024-04-12T20:56:31.523 INFO:tasks.workunit.client.1.smithi186.stdout:0/453: rename d0/d10/l6a to d0/d14/d4e/l91 0 2024-04-12T20:56:31.524 INFO:tasks.workunit.client.1.smithi186.stdout:3/382: chown d1/d3/d21/d22/d4a 835 1 2024-04-12T20:56:31.524 INFO:tasks.workunit.client.1.smithi186.stdout:1/366: mkdir d14/d74 0 2024-04-12T20:56:31.526 INFO:tasks.workunit.client.1.smithi186.stdout:7/405: mknod d7/d10/d1c/d8c/c96 0 2024-04-12T20:56:31.526 INFO:tasks.workunit.client.1.smithi186.stdout:3/383: fsync d1/f1e 0 2024-04-12T20:56:31.530 INFO:tasks.workunit.client.1.smithi186.stdout:7/406: dread - d7/d55/d5f/f92 zero size 2024-04-12T20:56:31.533 INFO:tasks.workunit.client.1.smithi186.stdout:3/384: read d1/f1e [3989310,69358] 0 2024-04-12T20:56:31.534 INFO:tasks.workunit.client.1.smithi186.stdout:6/342: dwrite d9/f12 [0,4194304] 0 2024-04-12T20:56:31.534 INFO:tasks.workunit.client.1.smithi186.stdout:5/384: write d5/db/d15/d2a/d2d/f49 [842988,85241] 0 2024-04-12T20:56:31.535 INFO:tasks.workunit.client.1.smithi186.stdout:9/353: rename d3/d13/f22 to d3/d12/d6f/f75 0 2024-04-12T20:56:31.542 INFO:tasks.workunit.client.1.smithi186.stdout:3/385: write d1/d3/da/f56 [4187782,37609] 0 2024-04-12T20:56:31.546 INFO:tasks.workunit.client.1.smithi186.stdout:8/330: write d7/d18/d36/d1d/f4d [1593570,10847] 0 2024-04-12T20:56:31.549 INFO:tasks.workunit.client.1.smithi186.stdout:2/388: dwrite d0/d32/f6f [0,4194304] 0 2024-04-12T20:56:31.554 INFO:tasks.workunit.client.1.smithi186.stdout:7/407: rename d7/d55/d5f to d7/d10/d2d/d40/d4a/d6c/d97 0 2024-04-12T20:56:31.559 INFO:tasks.workunit.client.1.smithi186.stdout:5/385: creat d5/db/d15/d2a/d2d/d46/f83 x:0 0 0 2024-04-12T20:56:31.559 INFO:tasks.workunit.client.1.smithi186.stdout:4/446: dwrite d0/dc/d33/fb [4194304,4194304] 0 2024-04-12T20:56:31.569 INFO:tasks.workunit.client.1.smithi186.stdout:4/447: write d0/f49 [721347,41157] 0 2024-04-12T20:56:31.572 INFO:tasks.workunit.client.1.smithi186.stdout:4/448: write d0/d26/f2b [1051448,96812] 0 2024-04-12T20:56:31.572 INFO:tasks.workunit.client.1.smithi186.stdout:9/354: link d3/d2f/d2e/d43/d4c/f6e d3/d12/d27/d71/f76 0 2024-04-12T20:56:31.573 INFO:tasks.workunit.client.1.smithi186.stdout:7/408: mknod d7/d10/d2d/d40/d4a/d6c/d97/d4d/d84/c98 0 2024-04-12T20:56:31.573 INFO:tasks.workunit.client.1.smithi186.stdout:5/386: mkdir d5/db/d15/d2a/d2d/d82/d84 0 2024-04-12T20:56:31.584 INFO:tasks.workunit.client.1.smithi186.stdout:3/386: getdents d1/d2c 0 2024-04-12T20:56:31.585 INFO:tasks.workunit.client.1.smithi186.stdout:9/355: chown d3/d2f/d2e/f2d 23483 1 2024-04-12T20:56:31.590 INFO:tasks.workunit.client.1.smithi186.stdout:6/343: dwrite d9/d43/f77 [0,4194304] 0 2024-04-12T20:56:31.590 INFO:tasks.workunit.client.1.smithi186.stdout:4/449: symlink d0/dc/d88/l8e 0 2024-04-12T20:56:31.591 INFO:tasks.workunit.client.1.smithi186.stdout:2/389: link d0/d32/l4e d0/d3/d1a/d21/d26/l75 0 2024-04-12T20:56:31.593 INFO:tasks.workunit.client.1.smithi186.stdout:9/356: mkdir d3/d2f/d2e/d33/d66/d77 0 2024-04-12T20:56:31.598 INFO:tasks.workunit.client.1.smithi186.stdout:4/450: write d0/dc/f2f [4589962,14438] 0 2024-04-12T20:56:31.599 INFO:tasks.workunit.client.1.smithi186.stdout:1/367: dwrite d14/d1a/d33/f6b [0,4194304] 0 2024-04-12T20:56:31.600 INFO:tasks.workunit.client.1.smithi186.stdout:6/344: creat d9/d4c/d6f/f7b x:0 0 0 2024-04-12T20:56:31.600 INFO:tasks.workunit.client.1.smithi186.stdout:3/387: mknod d1/d3/d21/d22/d24/d28/c82 0 2024-04-12T20:56:31.601 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:56:31 smithi018 ceph-mon[28799]: pgmap v48: 65 pgs: 65 active+clean; 2.3 GiB data, 8.2 GiB used, 528 GiB / 536 GiB avail; 40 MiB/s rd, 127 MiB/s wr, 321 op/s 2024-04-12T20:56:31.601 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:56:31 smithi018 ceph-mon[28799]: mgrmap e25: smithi186.kqcfxk(active, since 90s), standbys: smithi018.xjzwga 2024-04-12T20:56:31.601 INFO:tasks.workunit.client.1.smithi186.stdout:4/451: fdatasync d0/d26/f4f 0 2024-04-12T20:56:31.603 INFO:tasks.workunit.client.1.smithi186.stdout:2/390: chown d0/dc/d2e/l56 398168494 1 2024-04-12T20:56:31.606 INFO:tasks.workunit.client.1.smithi186.stdout:9/357: mknod d3/d12/d27/d71/d72/c78 0 2024-04-12T20:56:31.619 INFO:tasks.workunit.client.1.smithi186.stdout:2/391: write d0/dc/f4c [2294022,29262] 0 2024-04-12T20:56:31.620 INFO:tasks.workunit.client.1.smithi186.stdout:9/358: write d3/d2f/d2e/f41 [1438797,106266] 0 2024-04-12T20:56:31.621 INFO:tasks.workunit.client.1.smithi186.stdout:5/387: dwrite d5/db/d15/d2a/d2d/f58 [0,4194304] 0 2024-04-12T20:56:31.622 INFO:tasks.workunit.client.1.smithi186.stdout:1/368: truncate ff 1522410 0 2024-04-12T20:56:31.628 INFO:tasks.workunit.client.1.smithi186.stdout:4/452: getdents d0/dc/d88 0 2024-04-12T20:56:31.630 INFO:tasks.workunit.client.1.smithi186.stdout:5/388: symlink d5/d27/l85 0 2024-04-12T20:56:31.631 INFO:tasks.workunit.client.1.smithi186.stdout:2/392: symlink d0/d32/d2a/l76 0 2024-04-12T20:56:31.641 INFO:tasks.workunit.client.1.smithi186.stdout:0/454: sync 2024-04-12T20:56:31.655 INFO:tasks.workunit.client.1.smithi186.stdout:6/345: getdents d9/d43/d2d/d4d 0 2024-04-12T20:56:31.656 INFO:tasks.workunit.client.1.smithi186.stdout:3/388: dwrite d1/d3/d21/d5c/f69 [0,4194304] 0 2024-04-12T20:56:31.656 INFO:tasks.workunit.client.1.smithi186.stdout:9/359: write d3/d12/d27/d71/f76 [372082,116755] 0 2024-04-12T20:56:31.656 INFO:tasks.workunit.client.1.smithi186.stdout:7/409: dread d7/d10/d2d/d40/d4a/f4c [0,4194304] 0 2024-04-12T20:56:31.659 INFO:tasks.workunit.client.1.smithi186.stdout:6/346: fdatasync d9/f6a 0 2024-04-12T20:56:31.662 INFO:tasks.workunit.client.1.smithi186.stdout:0/455: rmdir d0/d14/d4e/d1e 39 2024-04-12T20:56:31.663 INFO:tasks.workunit.client.1.smithi186.stdout:9/360: mknod d3/d2f/d2e/d33/c79 0 2024-04-12T20:56:31.665 INFO:tasks.workunit.client.1.smithi186.stdout:7/410: mkdir d7/d55/d71/d99 0 2024-04-12T20:56:31.666 INFO:tasks.workunit.client.1.smithi186.stdout:2/393: creat d0/d3/d1a/d21/f77 x:0 0 0 2024-04-12T20:56:31.668 INFO:tasks.workunit.client.1.smithi186.stdout:6/347: symlink d9/d43/d2d/d4d/l7c 0 2024-04-12T20:56:31.669 INFO:tasks.workunit.client.1.smithi186.stdout:3/389: dread d1/d3/da/f47 [0,4194304] 0 2024-04-12T20:56:31.672 INFO:tasks.workunit.client.1.smithi186.stdout:7/411: truncate d7/d10/f41 1159850 0 2024-04-12T20:56:31.675 INFO:tasks.workunit.client.1.smithi186.stdout:2/394: mknod d0/d3/d1a/d27/c78 0 2024-04-12T20:56:31.675 INFO:tasks.workunit.client.1.smithi186.stdout:6/348: symlink d9/d43/d50/l7d 0 2024-04-12T20:56:31.676 INFO:tasks.workunit.client.1.smithi186.stdout:5/389: dread d5/db/d71/f16 [0,4194304] 0 2024-04-12T20:56:31.677 INFO:tasks.workunit.client.1.smithi186.stdout:0/456: mknod d0/d14/d4e/d1e/d84/c92 0 2024-04-12T20:56:31.682 INFO:tasks.workunit.client.1.smithi186.stdout:9/361: read d3/d2f/f1e [1209051,13818] 0 2024-04-12T20:56:31.683 INFO:tasks.workunit.client.1.smithi186.stdout:0/457: symlink d0/d10/d71/l93 0 2024-04-12T20:56:31.683 INFO:tasks.workunit.client.1.smithi186.stdout:3/390: link d1/d3/d21/d22/f4c d1/d3/d49/d4d/d6f/f83 0 2024-04-12T20:56:31.684 INFO:tasks.workunit.client.1.smithi186.stdout:7/412: readlink d7/d10/d2d/d40/d4a/d6c/d97/d4d/l8b 0 2024-04-12T20:56:31.691 INFO:tasks.workunit.client.1.smithi186.stdout:2/395: rename d0/d3/d1a/d27/f70 to d0/f79 0 2024-04-12T20:56:31.692 INFO:tasks.workunit.client.1.smithi186.stdout:4/453: dread d0/dc/d33/fb [0,4194304] 0 2024-04-12T20:56:31.693 INFO:tasks.workunit.client.1.smithi186.stdout:3/391: chown d1/d3/d21/d22/f4c 886 1 2024-04-12T20:56:31.695 INFO:tasks.workunit.client.1.smithi186.stdout:4/454: chown d0/d21/f20 5204 1 2024-04-12T20:56:31.701 INFO:tasks.workunit.client.1.smithi186.stdout:4/455: rmdir d0 39 2024-04-12T20:56:31.721 INFO:tasks.workunit.client.1.smithi186.stdout:4/456: write d0/dc/d33/f46 [5107509,103411] 0 2024-04-12T20:56:31.724 INFO:tasks.workunit.client.1.smithi186.stdout:5/390: dread d5/f9 [4194304,4194304] 0 2024-04-12T20:56:31.730 INFO:tasks.workunit.client.1.smithi186.stdout:2/396: dwrite d0/f10 [0,4194304] 0 2024-04-12T20:56:31.733 INFO:tasks.workunit.client.1.smithi186.stdout:6/349: dwrite d9/f12 [4194304,4194304] 0 2024-04-12T20:56:31.739 INFO:tasks.workunit.client.1.smithi186.stdout:2/397: mkdir d0/d3/d7a 0 2024-04-12T20:56:31.741 INFO:tasks.workunit.client.1.smithi186.stdout:7/413: dwrite d7/d10/f95 [0,4194304] 0 2024-04-12T20:56:31.743 INFO:tasks.workunit.client.1.smithi186.stdout:9/362: dwrite d3/d2f/f4a [0,4194304] 0 2024-04-12T20:56:31.743 INFO:tasks.workunit.client.1.smithi186.stdout:5/391: mkdir d5/db/d15/d2a/d6d/d7f/d86 0 2024-04-12T20:56:31.749 INFO:tasks.workunit.client.1.smithi186.stdout:4/457: rename d0/d21/d3d/d48/f76 to d0/d59/f8f 0 2024-04-12T20:56:31.749 INFO:tasks.workunit.client.1.smithi186.stdout:2/398: symlink d0/d3e/l7b 0 2024-04-12T20:56:31.750 INFO:tasks.workunit.client.1.smithi186.stdout:3/392: dwrite d1/f63 [0,4194304] 0 2024-04-12T20:56:31.751 INFO:tasks.workunit.client.1.smithi186.stdout:2/399: fsync d0/d32/f38 0 2024-04-12T20:56:31.752 INFO:tasks.workunit.client.1.smithi186.stdout:6/350: creat d9/d43/d2d/f7e x:0 0 0 2024-04-12T20:56:31.755 INFO:tasks.workunit.client.1.smithi186.stdout:3/393: fsync d1/d3/d49/d4d/f51 0 2024-04-12T20:56:31.759 INFO:tasks.workunit.client.1.smithi186.stdout:2/400: rename d0/f2 to d0/d32/f7c 0 2024-04-12T20:56:31.761 INFO:tasks.workunit.client.1.smithi186.stdout:2/401: chown d0/d32/f6f 477 1 2024-04-12T20:56:31.765 INFO:tasks.workunit.client.1.smithi186.stdout:8/331: sync 2024-04-12T20:56:31.766 INFO:tasks.workunit.client.1.smithi186.stdout:6/351: creat d9/d5e/d7a/f7f x:0 0 0 2024-04-12T20:56:31.770 INFO:tasks.workunit.client.1.smithi186.stdout:2/402: rmdir d0/dc 39 2024-04-12T20:56:31.771 INFO:tasks.workunit.client.1.smithi186.stdout:4/458: getdents d0/dc/d33 0 2024-04-12T20:56:31.772 INFO:tasks.workunit.client.1.smithi186.stdout:9/363: dread d3/d2f/d2e/f41 [0,4194304] 0 2024-04-12T20:56:31.773 INFO:tasks.workunit.client.1.smithi186.stdout:8/332: dread - d7/d18/d36/d20/d23/d25/f34 zero size 2024-04-12T20:56:31.775 INFO:tasks.workunit.client.1.smithi186.stdout:6/352: symlink d9/d5e/d73/l80 0 2024-04-12T20:56:31.775 INFO:tasks.workunit.client.1.smithi186.stdout:4/459: stat d0/dc/d33/fb 0 2024-04-12T20:56:31.777 INFO:tasks.workunit.client.1.smithi186.stdout:9/364: dread - d3/d2f/f5e zero size 2024-04-12T20:56:31.779 INFO:tasks.workunit.client.1.smithi186.stdout:5/392: truncate d5/db/d15/d2a/d2d/d48/f4b 1952760 0 2024-04-12T20:56:31.782 INFO:tasks.workunit.client.1.smithi186.stdout:7/414: truncate d7/d10/d2d/f44 635723 0 2024-04-12T20:56:31.783 INFO:tasks.workunit.client.1.smithi186.stdout:3/394: truncate d1/d2c/d48/f57 1680443 0 2024-04-12T20:56:31.785 INFO:tasks.workunit.client.1.smithi186.stdout:9/365: fsync d3/d2f/d2e/f2d 0 2024-04-12T20:56:31.785 INFO:tasks.workunit.client.1.smithi186.stdout:6/353: mkdir d9/d5e/d7a/d81 0 2024-04-12T20:56:31.786 INFO:tasks.workunit.client.1.smithi186.stdout:7/415: dread - d7/d10/d1c/f8e zero size 2024-04-12T20:56:31.787 INFO:tasks.workunit.client.1.smithi186.stdout:4/460: rename d0/fd to d0/d21/d3d/d63/f90 0 2024-04-12T20:56:31.792 INFO:tasks.workunit.client.1.smithi186.stdout:6/354: mknod d9/d43/d2d/c82 0 2024-04-12T20:56:31.795 INFO:tasks.workunit.client.1.smithi186.stdout:6/355: fdatasync d9/d43/d50/f54 0 2024-04-12T20:56:31.799 INFO:tasks.workunit.client.1.smithi186.stdout:1/369: sync 2024-04-12T20:56:31.799 INFO:tasks.workunit.client.1.smithi186.stdout:9/366: mknod d3/d2f/d2e/d33/d36/d48/c7a 0 2024-04-12T20:56:31.799 INFO:tasks.workunit.client.1.smithi186.stdout:0/458: sync 2024-04-12T20:56:31.801 INFO:tasks.workunit.client.1.smithi186.stdout:1/370: fdatasync fe 0 2024-04-12T20:56:31.802 INFO:tasks.workunit.client.1.smithi186.stdout:7/416: creat d7/d10/d2d/d40/d4a/d5c/f9a x:0 0 0 2024-04-12T20:56:31.803 INFO:tasks.workunit.client.1.smithi186.stdout:3/395: link d1/d3/d21/d5c/f69 d1/d2c/d54/d81/f84 0 2024-04-12T20:56:31.804 INFO:tasks.workunit.client.1.smithi186.stdout:1/371: fdatasync d14/d1a/d33/f6b 0 2024-04-12T20:56:31.805 INFO:tasks.workunit.client.1.smithi186.stdout:6/356: rename d9/d43/d2d/f6b to d9/d43/d59/f83 0 2024-04-12T20:56:31.805 INFO:tasks.workunit.client.1.smithi186.stdout:8/333: dread d7/d18/f4c [0,4194304] 0 2024-04-12T20:56:31.806 INFO:tasks.workunit.client.1.smithi186.stdout:7/417: chown d7/d10/d2d/d86/f93 1606 1 2024-04-12T20:56:31.806 INFO:tasks.workunit.client.1.smithi186.stdout:0/459: rmdir d0/d5c/d65 39 2024-04-12T20:56:31.809 INFO:tasks.workunit.client.1.smithi186.stdout:4/461: link d0/d21/d3d/l4a d0/d21/d3d/l91 0 2024-04-12T20:56:31.809 INFO:tasks.workunit.client.1.smithi186.stdout:1/372: fdatasync d14/f16 0 2024-04-12T20:56:31.809 INFO:tasks.workunit.client.1.smithi186.stdout:5/393: dread d5/db/f57 [0,4194304] 0 2024-04-12T20:56:31.812 INFO:tasks.workunit.client.1.smithi186.stdout:6/357: creat d9/d5e/d73/f84 x:0 0 0 2024-04-12T20:56:31.813 INFO:tasks.workunit.client.1.smithi186.stdout:8/334: mknod d7/d18/d36/d20/d23/d25/c69 0 2024-04-12T20:56:31.815 INFO:tasks.workunit.client.1.smithi186.stdout:5/394: dread - d5/db/d71/f75 zero size 2024-04-12T20:56:31.816 INFO:tasks.workunit.client.1.smithi186.stdout:0/460: creat d0/d10/d71/f94 x:0 0 0 2024-04-12T20:56:31.817 INFO:tasks.workunit.client.1.smithi186.stdout:3/396: symlink d1/d2c/d48/l85 0 2024-04-12T20:56:31.817 INFO:tasks.workunit.client.1.smithi186.stdout:4/462: write d0/d59/f8f [782244,57267] 0 2024-04-12T20:56:31.822 INFO:tasks.workunit.client.1.smithi186.stdout:1/373: symlink d14/d1a/d33/d36/d5b/l75 0 2024-04-12T20:56:31.823 INFO:tasks.workunit.client.1.smithi186.stdout:6/358: symlink d9/d43/d2d/l85 0 2024-04-12T20:56:31.823 INFO:tasks.workunit.client.1.smithi186.stdout:3/397: readlink d1/d3/d21/d22/d24/d2f/l33 0 2024-04-12T20:56:31.825 INFO:tasks.workunit.client.1.smithi186.stdout:4/463: mknod d0/dc/d88/c92 0 2024-04-12T20:56:31.826 INFO:tasks.workunit.client.1.smithi186.stdout:3/398: dread - d1/d3/f7e zero size 2024-04-12T20:56:31.827 INFO:tasks.workunit.client.1.smithi186.stdout:5/395: creat d5/db/d15/d2a/d2d/d48/f87 x:0 0 0 2024-04-12T20:56:31.828 INFO:tasks.workunit.client.1.smithi186.stdout:2/403: dread d0/d3/d1a/f45 [0,4194304] 0 2024-04-12T20:56:31.831 INFO:tasks.workunit.client.1.smithi186.stdout:4/464: rmdir d0/dc/d1f/d73 39 2024-04-12T20:56:31.833 INFO:tasks.workunit.client.1.smithi186.stdout:6/359: mknod d9/d5e/d7a/d81/c86 0 2024-04-12T20:56:31.834 INFO:tasks.workunit.client.1.smithi186.stdout:7/418: dread d7/d10/d1c/d5d/f80 [0,4194304] 0 2024-04-12T20:56:31.837 INFO:tasks.workunit.client.1.smithi186.stdout:2/404: stat d0/dc/c11 0 2024-04-12T20:56:31.840 INFO:tasks.workunit.client.1.smithi186.stdout:3/399: truncate d1/d3/d49/d4d/f7a 330275 0 2024-04-12T20:56:31.840 INFO:tasks.workunit.client.1.smithi186.stdout:7/419: truncate d7/d10/d1c/f1e 4662658 0 2024-04-12T20:56:31.849 INFO:tasks.workunit.client.1.smithi186.stdout:2/405: rename d0/f6 to d0/d32/d69/f7d 0 2024-04-12T20:56:31.862 INFO:tasks.workunit.client.1.smithi186.stdout:2/406: write d0/d32/f38 [1246650,107461] 0 2024-04-12T20:56:31.863 INFO:tasks.workunit.client.1.smithi186.stdout:1/374: creat d14/d1a/d33/d36/d5b/d3e/d71/f76 x:0 0 0 2024-04-12T20:56:31.864 INFO:tasks.workunit.client.1.smithi186.stdout:3/400: link d1/l16 d1/d3/d21/d22/d4a/l86 0 2024-04-12T20:56:31.866 INFO:tasks.workunit.client.1.smithi186.stdout:5/396: dwrite d5/db/d15/d2a/d4c/f62 [0,4194304] 0 2024-04-12T20:56:31.866 INFO:tasks.workunit.client.1.smithi186.stdout:0/461: dread d0/d5c/f5f [0,4194304] 0 2024-04-12T20:56:31.867 INFO:tasks.workunit.client.1.smithi186.stdout:9/367: dread d3/d2f/f1e [0,4194304] 0 2024-04-12T20:56:31.876 INFO:tasks.workunit.client.1.smithi186.stdout:2/407: unlink d0/d3/d1a/l47 0 2024-04-12T20:56:31.876 INFO:tasks.workunit.client.1.smithi186.stdout:5/397: chown d5/db/d15/d2a/d2d/d77 1671037 1 2024-04-12T20:56:31.877 INFO:tasks.workunit.client.1.smithi186.stdout:9/368: dread - d3/d2f/d2e/d43/d4c/f51 zero size 2024-04-12T20:56:31.879 INFO:tasks.workunit.client.1.smithi186.stdout:0/462: symlink d0/d14/d4e/d50/l95 0 2024-04-12T20:56:31.879 INFO:tasks.workunit.client.1.smithi186.stdout:4/465: dread d0/dc/d33/fb [4194304,4194304] 0 2024-04-12T20:56:31.881 INFO:tasks.workunit.client.1.smithi186.stdout:1/375: creat d14/f77 x:0 0 0 2024-04-12T20:56:31.881 INFO:tasks.workunit.client.1.smithi186.stdout:7/420: dwrite d7/f16 [4194304,4194304] 0 2024-04-12T20:56:31.885 INFO:tasks.workunit.client.1.smithi186.stdout:7/421: chown d7/d10/d1c/f21 164185325 1 2024-04-12T20:56:31.885 INFO:tasks.workunit.client.1.smithi186.stdout:3/401: creat d1/d3/d21/d22/f87 x:0 0 0 2024-04-12T20:56:31.886 INFO:tasks.workunit.client.1.smithi186.stdout:6/360: dwrite d9/d4c/f6c [0,4194304] 0 2024-04-12T20:56:31.888 INFO:tasks.workunit.client.1.smithi186.stdout:5/398: fdatasync d5/db/d15/f28 0 2024-04-12T20:56:31.888 INFO:tasks.workunit.client.1.smithi186.stdout:7/422: fsync d7/d10/d1c/f2c 0 2024-04-12T20:56:31.891 INFO:tasks.workunit.client.1.smithi186.stdout:9/369: symlink d3/d2f/d2e/d43/d4c/l7b 0 2024-04-12T20:56:31.896 INFO:tasks.workunit.client.1.smithi186.stdout:0/463: mkdir d0/d14/d4e/d50/d6e/d96 0 2024-04-12T20:56:31.901 INFO:tasks.workunit.client.1.smithi186.stdout:6/361: mknod d9/d5e/d7a/d81/c87 0 2024-04-12T20:56:31.902 INFO:tasks.workunit.client.1.smithi186.stdout:2/408: creat d0/f7e x:0 0 0 2024-04-12T20:56:31.904 INFO:tasks.workunit.client.1.smithi186.stdout:5/399: fsync d5/db/d33/f36 0 2024-04-12T20:56:31.907 INFO:tasks.workunit.client.1.smithi186.stdout:1/376: creat d14/d1a/d33/d36/d5b/d3e/d71/f78 x:0 0 0 2024-04-12T20:56:31.907 INFO:tasks.workunit.client.1.smithi186.stdout:9/370: creat d3/d2f/d2e/d33/d36/d70/f7c x:0 0 0 2024-04-12T20:56:31.908 INFO:tasks.workunit.client.1.smithi186.stdout:8/335: sync 2024-04-12T20:56:31.909 INFO:tasks.workunit.client.1.smithi186.stdout:6/362: symlink d9/d4c/d6f/l88 0 2024-04-12T20:56:31.913 INFO:tasks.workunit.client.1.smithi186.stdout:5/400: mkdir d5/db/d71/d88 0 2024-04-12T20:56:31.913 INFO:tasks.workunit.client.1.smithi186.stdout:2/409: truncate d0/d3/f58 148880 0 2024-04-12T20:56:31.915 INFO:tasks.workunit.client.1.smithi186.stdout:0/464: link d0/d10/d32/d3f/f77 d0/d14/d4e/d1e/d84/f97 0 2024-04-12T20:56:31.917 INFO:tasks.workunit.client.1.smithi186.stdout:8/336: readlink d7/d18/d36/d20/d23/l2c 0 2024-04-12T20:56:31.918 INFO:tasks.workunit.client.1.smithi186.stdout:9/371: mkdir d3/d2f/d54/d7d 0 2024-04-12T20:56:31.922 INFO:tasks.workunit.client.1.smithi186.stdout:2/410: fsync d0/dc/f40 0 2024-04-12T20:56:31.925 INFO:tasks.workunit.client.1.smithi186.stdout:9/372: mknod d3/d12/c7e 0 2024-04-12T20:56:31.927 INFO:tasks.workunit.client.1.smithi186.stdout:1/377: creat d14/d1a/d33/d36/d5b/f79 x:0 0 0 2024-04-12T20:56:31.927 INFO:tasks.workunit.client.1.smithi186.stdout:6/363: truncate d9/d43/d50/f54 1275271 0 2024-04-12T20:56:31.931 INFO:tasks.workunit.client.1.smithi186.stdout:2/411: chown d0/d32/d2a 1494 1 2024-04-12T20:56:31.931 INFO:tasks.workunit.client.1.smithi186.stdout:3/402: dread d1/d3/d21/d5c/f77 [0,4194304] 0 2024-04-12T20:56:31.933 INFO:tasks.workunit.client.1.smithi186.stdout:6/364: unlink d9/f6a 0 2024-04-12T20:56:31.940 INFO:tasks.workunit.client.1.smithi186.stdout:7/423: dwrite d7/d10/d2d/d40/d4a/d6c/d97/f92 [0,4194304] 0 2024-04-12T20:56:31.947 INFO:tasks.workunit.client.1.smithi186.stdout:6/365: mkdir d9/d4c/d6f/d89 0 2024-04-12T20:56:31.948 INFO:tasks.workunit.client.1.smithi186.stdout:1/378: fsync d14/d1a/d33/d36/d5b/d3e/f53 0 2024-04-12T20:56:31.951 INFO:tasks.workunit.client.1.smithi186.stdout:6/366: dread - d9/d5e/f71 zero size 2024-04-12T20:56:31.951 INFO:tasks.workunit.client.1.smithi186.stdout:7/424: symlink d7/l9b 0 2024-04-12T20:56:31.955 INFO:tasks.workunit.client.1.smithi186.stdout:2/412: write d0/d32/d2a/f43 [1236868,17487] 0 2024-04-12T20:56:31.955 INFO:tasks.workunit.client.1.smithi186.stdout:5/401: dwrite d5/d44/f67 [0,4194304] 0 2024-04-12T20:56:31.956 INFO:tasks.workunit.client.1.smithi186.stdout:4/466: dread d0/f6 [0,4194304] 0 2024-04-12T20:56:31.957 INFO:tasks.workunit.client.1.smithi186.stdout:6/367: dread - d9/d43/d2d/f4e zero size 2024-04-12T20:56:31.961 INFO:tasks.workunit.client.1.smithi186.stdout:1/379: write d14/d1a/d33/d36/d5b/f79 [440490,85037] 0 2024-04-12T20:56:31.962 INFO:tasks.workunit.client.1.smithi186.stdout:6/368: write d9/d43/d2d/f7e [566729,93539] 0 2024-04-12T20:56:31.964 INFO:tasks.workunit.client.1.smithi186.stdout:1/380: chown d14/d1a/d21 8469923 1 2024-04-12T20:56:31.965 INFO:tasks.workunit.client.1.smithi186.stdout:2/413: write d0/d3/d1a/d24/f60 [926111,69401] 0 2024-04-12T20:56:31.971 INFO:tasks.workunit.client.1.smithi186.stdout:8/337: dwrite d7/d18/d36/d1d/d1f/d26/f3d [0,4194304] 0 2024-04-12T20:56:31.973 INFO:tasks.workunit.client.1.smithi186.stdout:0/465: truncate d0/d10/d32/f48 837811 0 2024-04-12T20:56:31.975 INFO:tasks.workunit.client.1.smithi186.stdout:6/369: truncate d9/f1d 3482538 0 2024-04-12T20:56:31.975 INFO:tasks.workunit.client.1.smithi186.stdout:2/414: symlink d0/d3/d1a/d21/d26/l7f 0 2024-04-12T20:56:31.980 INFO:tasks.workunit.client.1.smithi186.stdout:2/415: read d0/d3/d1a/d24/f34 [2694788,46260] 0 2024-04-12T20:56:31.980 INFO:tasks.workunit.client.1.smithi186.stdout:4/467: creat d0/f93 x:0 0 0 2024-04-12T20:56:31.982 INFO:tasks.workunit.client.1.smithi186.stdout:1/381: symlink d14/d1a/d33/d36/d5b/d3e/d5f/l7a 0 2024-04-12T20:56:31.984 INFO:tasks.workunit.client.1.smithi186.stdout:4/468: read d0/f49 [384952,45996] 0 2024-04-12T20:56:31.984 INFO:tasks.workunit.client.1.smithi186.stdout:6/370: mknod d9/d43/d59/c8a 0 2024-04-12T20:56:31.986 INFO:tasks.workunit.client.1.smithi186.stdout:0/466: fsync d0/d81/f83 0 2024-04-12T20:56:31.990 INFO:tasks.workunit.client.1.smithi186.stdout:0/467: readlink d0/l44 0 2024-04-12T20:56:31.990 INFO:tasks.workunit.client.1.smithi186.stdout:2/416: creat d0/d3/f80 x:0 0 0 2024-04-12T20:56:31.990 INFO:tasks.workunit.client.1.smithi186.stdout:1/382: truncate d14/d1a/f5a 1683604 0 2024-04-12T20:56:31.992 INFO:tasks.workunit.client.1.smithi186.stdout:8/338: read d7/de/d10/d2f/f31 [825566,112136] 0 2024-04-12T20:56:31.993 INFO:tasks.workunit.client.1.smithi186.stdout:6/371: unlink d9/d43/d2d/d4d/c58 0 2024-04-12T20:56:31.994 INFO:tasks.workunit.client.1.smithi186.stdout:3/403: dwrite d1/d2c/d54/d81/f84 [0,4194304] 0 2024-04-12T20:56:32.001 INFO:tasks.workunit.client.1.smithi186.stdout:5/402: truncate d5/db/d15/d2a/d2d/d48/f3a 3509061 0 2024-04-12T20:56:32.001 INFO:tasks.workunit.client.1.smithi186.stdout:1/383: rmdir d14/d1a/d3c 39 2024-04-12T20:56:32.004 INFO:tasks.workunit.client.1.smithi186.stdout:4/469: mknod d0/d21/c94 0 2024-04-12T20:56:32.008 INFO:tasks.workunit.client.1.smithi186.stdout:2/417: creat d0/d3/d1a/d48/f81 x:0 0 0 2024-04-12T20:56:32.010 INFO:tasks.workunit.client.1.smithi186.stdout:5/403: write d5/db/d15/d2a/d2d/d48/f87 [354257,37778] 0 2024-04-12T20:56:32.013 INFO:tasks.workunit.client.1.smithi186.stdout:6/372: dread d9/d43/d59/d5d/f48 [0,4194304] 0 2024-04-12T20:56:32.014 INFO:tasks.workunit.client.1.smithi186.stdout:5/404: read d5/fd [1010312,92082] 0 2024-04-12T20:56:32.015 INFO:tasks.workunit.client.1.smithi186.stdout:0/468: creat d0/d14/d4e/d1e/f98 x:0 0 0 2024-04-12T20:56:32.017 INFO:tasks.workunit.client.1.smithi186.stdout:3/404: link d1/d3/d21/d5c/c3e d1/d3/d49/d4d/d6f/c88 0 2024-04-12T20:56:32.024 INFO:tasks.workunit.client.1.smithi186.stdout:1/384: dread d14/d1a/d21/f4b [0,4194304] 0 2024-04-12T20:56:32.024 INFO:tasks.workunit.client.1.smithi186.stdout:3/405: write d1/d2c/d48/f5f [1043693,10358] 0 2024-04-12T20:56:32.027 INFO:tasks.workunit.client.1.smithi186.stdout:6/373: unlink d9/d43/f39 0 2024-04-12T20:56:32.027 INFO:tasks.workunit.client.1.smithi186.stdout:0/469: chown d0/d14/d4e/d1e/l20 3 1 2024-04-12T20:56:32.031 INFO:tasks.workunit.client.1.smithi186.stdout:4/470: creat d0/d21/d3d/d48/f95 x:0 0 0 2024-04-12T20:56:32.039 INFO:tasks.workunit.client.1.smithi186.stdout:8/339: write d7/d18/d36/d1d/d1f/d26/f50 [289569,122461] 0 2024-04-12T20:56:32.039 INFO:tasks.workunit.client.1.smithi186.stdout:1/385: readlink d14/d1a/d33/d36/d5b/l2e 0 2024-04-12T20:56:32.041 INFO:tasks.workunit.client.1.smithi186.stdout:0/470: mknod d0/d5c/c99 0 2024-04-12T20:56:32.044 INFO:tasks.workunit.client.1.smithi186.stdout:4/471: chown d0/d21/d3d/c54 2198 1 2024-04-12T20:56:32.047 INFO:tasks.workunit.client.1.smithi186.stdout:4/472: fdatasync d0/f6 0 2024-04-12T20:56:32.053 INFO:tasks.workunit.client.1.smithi186.stdout:0/471: getdents d0/d81 0 2024-04-12T20:56:32.055 INFO:tasks.workunit.client.1.smithi186.stdout:1/386: getdents d14/d1a/d21/d35/d59 0 2024-04-12T20:56:32.058 INFO:tasks.workunit.client.1.smithi186.stdout:5/405: dwrite f2 [4194304,4194304] 0 2024-04-12T20:56:32.059 INFO:tasks.workunit.client.1.smithi186.stdout:8/340: link d7/d18/d36/d1d/f4d d7/d18/d36/d20/d23/d25/d29/f6a 0 2024-04-12T20:56:32.059 INFO:tasks.workunit.client.1.smithi186.stdout:9/373: sync 2024-04-12T20:56:32.060 INFO:tasks.workunit.client.1.smithi186.stdout:2/418: dread d0/d3/d1a/d21/d26/f3d [0,4194304] 0 2024-04-12T20:56:32.064 INFO:tasks.workunit.client.1.smithi186.stdout:3/406: dwrite d1/d2c/d48/f5f [0,4194304] 0 2024-04-12T20:56:32.067 INFO:tasks.workunit.client.1.smithi186.stdout:4/473: rename d0/dc/d33/c2c to d0/d21/d3d/d63/c96 0 2024-04-12T20:56:32.068 INFO:tasks.workunit.client.1.smithi186.stdout:2/419: write d0/d3/d1a/d24/f33 [3905918,47592] 0 2024-04-12T20:56:32.069 INFO:tasks.workunit.client.1.smithi186.stdout:9/374: creat d3/d12/f7f x:0 0 0 2024-04-12T20:56:32.070 INFO:tasks.workunit.client.1.smithi186.stdout:4/474: write d0/dc/d39/f79 [326197,31637] 0 2024-04-12T20:56:32.070 INFO:tasks.workunit.client.1.smithi186.stdout:3/407: mknod d1/d2c/d64/c89 0 2024-04-12T20:56:32.074 INFO:tasks.workunit.client.1.smithi186.stdout:1/387: creat d14/d1a/d33/d36/f7b x:0 0 0 2024-04-12T20:56:32.075 INFO:tasks.workunit.client.1.smithi186.stdout:9/375: fdatasync d3/d12/d27/f38 0 2024-04-12T20:56:32.077 INFO:tasks.workunit.client.1.smithi186.stdout:8/341: rename d7/de/f14 to d7/d18/d36/d1d/d1f/d26/f6b 0 2024-04-12T20:56:32.079 INFO:tasks.workunit.client.1.smithi186.stdout:2/420: creat d0/d3/d1a/d24/f82 x:0 0 0 2024-04-12T20:56:32.088 INFO:tasks.workunit.client.1.smithi186.stdout:0/472: dwrite d0/d5c/d65/f7b [0,4194304] 0 2024-04-12T20:56:32.088 INFO:tasks.workunit.client.1.smithi186.stdout:8/342: write d7/de/d5e/f1c [4469484,63463] 0 2024-04-12T20:56:32.088 INFO:tasks.workunit.client.1.smithi186.stdout:3/408: creat d1/f8a x:0 0 0 2024-04-12T20:56:32.094 INFO:tasks.workunit.client.1.smithi186.stdout:3/409: truncate d1/f8a 247182 0 2024-04-12T20:56:32.095 INFO:tasks.workunit.client.1.smithi186.stdout:8/343: chown d7/lb 14 1 2024-04-12T20:56:32.095 INFO:tasks.workunit.client.1.smithi186.stdout:0/473: read d0/d10/d32/d3f/f64 [34381,63500] 0 2024-04-12T20:56:32.097 INFO:tasks.workunit.client.1.smithi186.stdout:6/374: truncate d9/d43/f77 3845574 0 2024-04-12T20:56:32.097 INFO:tasks.workunit.client.1.smithi186.stdout:6/375: fsync d9/f4b 0 2024-04-12T20:56:32.099 INFO:tasks.workunit.client.1.smithi186.stdout:8/344: stat d7/d18/d36/d1d/c28 0 2024-04-12T20:56:32.120 INFO:tasks.workunit.client.1.smithi186.stdout:6/376: dwrite d9/d4c/f6c [0,4194304] 0 2024-04-12T20:56:32.125 INFO:tasks.workunit.client.1.smithi186.stdout:0/474: dwrite d0/d10/f8d [0,4194304] 0 2024-04-12T20:56:32.128 INFO:tasks.workunit.client.1.smithi186.stdout:4/475: getdents d0/d21/d3d/d63 0 2024-04-12T20:56:32.131 INFO:tasks.workunit.client.1.smithi186.stdout:6/377: creat d9/d43/d2d/f8b x:0 0 0 2024-04-12T20:56:32.137 INFO:tasks.workunit.client.1.smithi186.stdout:3/410: write d1/d3/d49/d4d/d6f/f83 [1232150,56635] 0 2024-04-12T20:56:32.139 INFO:tasks.workunit.client.1.smithi186.stdout:0/475: dread d0/d14/d4e/d1e/d84/f97 [0,4194304] 0 2024-04-12T20:56:32.140 INFO:tasks.workunit.client.1.smithi186.stdout:4/476: read - d0/dc/d33/f8a zero size 2024-04-12T20:56:32.148 INFO:tasks.workunit.client.1.smithi186.stdout:6/378: dread d9/f2e [0,4194304] 0 2024-04-12T20:56:32.148 INFO:tasks.workunit.client.1.smithi186.stdout:0/476: fsync d0/d14/d4e/d1e/f98 0 2024-04-12T20:56:32.149 INFO:tasks.workunit.client.1.smithi186.stdout:1/388: dwrite f9 [0,4194304] 0 2024-04-12T20:56:32.156 INFO:tasks.workunit.client.1.smithi186.stdout:8/345: truncate d7/de/f11 4156471 0 2024-04-12T20:56:32.164 INFO:tasks.workunit.client.1.smithi186.stdout:7/425: sync 2024-04-12T20:56:32.164 INFO:tasks.workunit.client.1.smithi186.stdout:0/477: mknod d0/d14/d4e/c9a 0 2024-04-12T20:56:32.164 INFO:tasks.workunit.client.1.smithi186.stdout:8/346: read d7/de/d10/f19 [247349,57419] 0 2024-04-12T20:56:32.165 INFO:tasks.workunit.client.1.smithi186.stdout:4/477: getdents d0/dc/d88 0 2024-04-12T20:56:32.166 INFO:tasks.workunit.client.1.smithi186.stdout:6/379: link d9/d43/d2d/c82 d9/d5e/c8c 0 2024-04-12T20:56:32.178 INFO:tasks.workunit.client.1.smithi186.stdout:0/478: symlink d0/d14/d4e/d1e/l9b 0 2024-04-12T20:56:32.193 INFO:tasks.workunit.client.1.smithi186.stdout:0/479: creat d0/d14/d4e/d50/d6e/f9c x:0 0 0 2024-04-12T20:56:32.199 INFO:tasks.workunit.client.1.smithi186.stdout:0/480: readlink d0/l26 0 2024-04-12T20:56:32.206 INFO:tasks.workunit.client.1.smithi186.stdout:1/389: dwrite d14/d1a/d3c/f47 [4194304,4194304] 0 2024-04-12T20:56:32.207 INFO:tasks.workunit.client.1.smithi186.stdout:4/478: dwrite d0/dc/d39/f66 [0,4194304] 0 2024-04-12T20:56:32.216 INFO:tasks.workunit.client.1.smithi186.stdout:7/426: dwrite d7/d10/d2d/d40/d4a/d6c/d97/d4d/f4e [0,4194304] 0 2024-04-12T20:56:32.216 INFO:tasks.workunit.client.1.smithi186.stdout:8/347: dwrite d7/d18/d36/fa [0,4194304] 0 2024-04-12T20:56:32.217 INFO:tasks.workunit.client.1.smithi186.stdout:0/481: mknod d0/d14/d4e/d50/d6e/d96/c9d 0 2024-04-12T20:56:32.219 INFO:tasks.workunit.client.1.smithi186.stdout:7/427: readlink d7/d10/d2d/d67/l82 0 2024-04-12T20:56:32.222 INFO:tasks.workunit.client.1.smithi186.stdout:6/380: dread d9/f12 [0,4194304] 0 2024-04-12T20:56:32.223 INFO:tasks.workunit.client.1.smithi186.stdout:0/482: fsync d0/d14/d4e/d1e/d84/f97 0 2024-04-12T20:56:32.226 INFO:tasks.workunit.client.1.smithi186.stdout:7/428: readlink d7/d10/d2d/d40/d4a/d6c/l87 0 2024-04-12T20:56:32.228 INFO:tasks.workunit.client.1.smithi186.stdout:0/483: dread - d0/d10/d71/f94 zero size 2024-04-12T20:56:32.228 INFO:tasks.workunit.client.1.smithi186.stdout:1/390: chown d14/d1a/d33/d36/d5b/d3e/f52 1011 1 2024-04-12T20:56:32.232 INFO:tasks.workunit.client.1.smithi186.stdout:7/429: creat d7/d10/d2d/d86/f9c x:0 0 0 2024-04-12T20:56:32.234 INFO:tasks.workunit.client.1.smithi186.stdout:0/484: chown d0/d10/d32/d3f/f54 3959458 1 2024-04-12T20:56:32.234 INFO:tasks.workunit.client.1.smithi186.stdout:5/406: sync 2024-04-12T20:56:32.238 INFO:tasks.workunit.client.1.smithi186.stdout:1/391: rename d14/d1a/d21/d35/l3b to d14/d6c/l7c 0 2024-04-12T20:56:32.238 INFO:tasks.workunit.client.1.smithi186.stdout:5/407: chown d5/db/d15/d2a/d6d 26473 1 2024-04-12T20:56:32.240 INFO:tasks.workunit.client.1.smithi186.stdout:0/485: creat d0/d10/d32/f9e x:0 0 0 2024-04-12T20:56:32.241 INFO:tasks.workunit.client.1.smithi186.stdout:5/408: dread - d5/db/d2e/f64 zero size 2024-04-12T20:56:32.243 INFO:tasks.workunit.client.1.smithi186.stdout:7/430: creat d7/d55/d71/d99/f9d x:0 0 0 2024-04-12T20:56:32.244 INFO:tasks.workunit.client.1.smithi186.stdout:5/409: chown d5/d44/d5d/c78 32872981 1 2024-04-12T20:56:32.246 INFO:tasks.workunit.client.1.smithi186.stdout:8/348: dread d7/de/d10/d2f/f31 [0,4194304] 0 2024-04-12T20:56:32.246 INFO:tasks.workunit.client.1.smithi186.stdout:7/431: read d7/f16 [7197814,105262] 0 2024-04-12T20:56:32.246 INFO:tasks.workunit.client.1.smithi186.stdout:0/486: rmdir d0/d5c 39 2024-04-12T20:56:32.250 INFO:tasks.workunit.client.1.smithi186.stdout:8/349: chown d7/de/d10/f51 63936503 1 2024-04-12T20:56:32.251 INFO:tasks.workunit.client.1.smithi186.stdout:4/479: dread d0/f6 [0,4194304] 0 2024-04-12T20:56:32.252 INFO:tasks.workunit.client.1.smithi186.stdout:0/487: write d0/f57 [872765,92005] 0 2024-04-12T20:56:32.252 INFO:tasks.workunit.client.1.smithi186.stdout:1/392: creat d14/f7d x:0 0 0 2024-04-12T20:56:32.260 INFO:tasks.workunit.client.1.smithi186.stdout:8/350: write d7/d18/d36/d20/d23/d25/f46 [695990,18989] 0 2024-04-12T20:56:32.261 INFO:tasks.workunit.client.1.smithi186.stdout:4/480: truncate d0/dc/d88/f6c 532219 0 2024-04-12T20:56:32.262 INFO:tasks.workunit.client.1.smithi186.stdout:1/393: getdents d14/d74 0 2024-04-12T20:56:32.265 INFO:tasks.workunit.client.1.smithi186.stdout:0/488: write d0/f3e [3878377,107209] 0 2024-04-12T20:56:32.266 INFO:tasks.workunit.client.1.smithi186.stdout:6/381: write d9/d43/d59/d5d/f48 [565657,12242] 0 2024-04-12T20:56:32.269 INFO:tasks.workunit.client.1.smithi186.stdout:4/481: rename d0/f49 to d0/dc/d64/d87/f97 0 2024-04-12T20:56:32.275 INFO:tasks.workunit.client.1.smithi186.stdout:4/482: chown d0/d26 4044944 1 2024-04-12T20:56:32.289 INFO:tasks.workunit.client.1.smithi186.stdout:5/410: dwrite d5/d27/f6b [0,4194304] 0 2024-04-12T20:56:32.298 INFO:tasks.workunit.client.1.smithi186.stdout:7/432: dread d7/f12 [0,4194304] 0 2024-04-12T20:56:32.312 INFO:tasks.workunit.client.1.smithi186.stdout:6/382: dwrite d9/d43/d50/f5b [0,4194304] 0 2024-04-12T20:56:32.312 INFO:tasks.workunit.client.1.smithi186.stdout:5/411: write d5/db/d15/d2a/d2d/d46/d55/f3d [3806618,40034] 0 2024-04-12T20:56:32.315 INFO:tasks.workunit.client.1.smithi186.stdout:1/394: dwrite d14/d1a/d1d/f23 [4194304,4194304] 0 2024-04-12T20:56:32.318 INFO:tasks.workunit.client.1.smithi186.stdout:7/433: fsync d7/d10/d1c/f21 0 2024-04-12T20:56:32.321 INFO:tasks.workunit.client.1.smithi186.stdout:0/489: dwrite d0/d10/f43 [0,4194304] 0 2024-04-12T20:56:32.324 INFO:tasks.workunit.client.1.smithi186.stdout:9/376: sync 2024-04-12T20:56:32.325 INFO:tasks.workunit.client.1.smithi186.stdout:2/421: sync 2024-04-12T20:56:32.325 INFO:tasks.workunit.client.1.smithi186.stdout:3/411: sync 2024-04-12T20:56:32.331 INFO:tasks.workunit.client.1.smithi186.stdout:9/377: write d3/d12/d27/f35 [1354383,107211] 0 2024-04-12T20:56:32.332 INFO:tasks.workunit.client.1.smithi186.stdout:5/412: truncate d5/db/d71/f52 1754936 0 2024-04-12T20:56:32.333 INFO:tasks.workunit.client.1.smithi186.stdout:7/434: creat d7/d10/d2d/d67/f9e x:0 0 0 2024-04-12T20:56:32.334 INFO:tasks.workunit.client.1.smithi186.stdout:3/412: readlink d1/d3/d21/d22/l43 0 2024-04-12T20:56:32.337 INFO:tasks.workunit.client.1.smithi186.stdout:2/422: unlink d0/d3/d1a/d24/f82 0 2024-04-12T20:56:32.347 INFO:tasks.workunit.client.1.smithi186.stdout:9/378: symlink d3/d2f/d2e/d33/l80 0 2024-04-12T20:56:32.348 INFO:tasks.workunit.client.1.smithi186.stdout:0/490: dread d0/f4f [0,4194304] 0 2024-04-12T20:56:32.350 INFO:tasks.workunit.client.1.smithi186.stdout:5/413: creat d5/db/d15/f89 x:0 0 0 2024-04-12T20:56:32.351 INFO:tasks.workunit.client.1.smithi186.stdout:2/423: symlink d0/d3/d1a/l83 0 2024-04-12T20:56:32.355 INFO:tasks.workunit.client.1.smithi186.stdout:4/483: dread d0/dc/d33/f17 [0,4194304] 0 2024-04-12T20:56:32.357 INFO:tasks.workunit.client.1.smithi186.stdout:9/379: mknod d3/d2f/d2e/c81 0 2024-04-12T20:56:32.359 INFO:tasks.workunit.client.1.smithi186.stdout:1/395: dread d14/d1a/d1d/f6f [0,4194304] 0 2024-04-12T20:56:32.360 INFO:tasks.workunit.client.1.smithi186.stdout:7/435: link d7/d10/d1c/f52 d7/d55/f9f 0 2024-04-12T20:56:32.364 INFO:tasks.workunit.client.1.smithi186.stdout:3/413: link d1/d3/c1d d1/d3/da/c8b 0 2024-04-12T20:56:32.368 INFO:tasks.workunit.client.1.smithi186.stdout:4/484: mkdir d0/dc/d64/d87/d98 0 2024-04-12T20:56:32.368 INFO:tasks.workunit.client.1.smithi186.stdout:9/380: chown d3/d2f/d2e/d43/c59 0 1 2024-04-12T20:56:32.373 INFO:tasks.workunit.client.1.smithi186.stdout:7/436: creat d7/d10/d2d/d40/d4a/fa0 x:0 0 0 2024-04-12T20:56:32.374 INFO:tasks.workunit.client.1.smithi186.stdout:8/351: sync 2024-04-12T20:56:32.377 INFO:tasks.workunit.client.1.smithi186.stdout:1/396: chown d14/d1a/d33/d36/d5b/d3e/d5f 73491947 1 2024-04-12T20:56:32.380 INFO:tasks.workunit.client.1.smithi186.stdout:3/414: fdatasync d1/d3/d21/d22/d34/f66 0 2024-04-12T20:56:32.384 INFO:tasks.workunit.client.1.smithi186.stdout:6/383: dwrite d9/f1d [0,4194304] 0 2024-04-12T20:56:32.384 INFO:tasks.workunit.client.1.smithi186.stdout:7/437: creat d7/fa1 x:0 0 0 2024-04-12T20:56:32.386 INFO:tasks.workunit.client.1.smithi186.stdout:9/381: mkdir d3/d2f/d2e/d43/d4c/d82 0 2024-04-12T20:56:32.388 INFO:tasks.workunit.client.1.smithi186.stdout:0/491: dwrite d0/d10/f43 [4194304,4194304] 0 2024-04-12T20:56:32.389 INFO:tasks.workunit.client.1.smithi186.stdout:7/438: chown d7/d55 442 1 2024-04-12T20:56:32.390 INFO:tasks.workunit.client.1.smithi186.stdout:8/352: rename d7/lb to d7/de/l6c 0 2024-04-12T20:56:32.393 INFO:tasks.workunit.client.1.smithi186.stdout:2/424: dread d0/d32/f7c [0,4194304] 0 2024-04-12T20:56:32.393 INFO:tasks.workunit.client.1.smithi186.stdout:3/415: truncate d1/d3/d21/d22/f79 1028357 0 2024-04-12T20:56:32.394 INFO:tasks.workunit.client.1.smithi186.stdout:1/397: link d14/d1a/d3c/f47 d14/d1a/d33/d36/d5b/d3e/f7e 0 2024-04-12T20:56:32.397 INFO:tasks.workunit.client.1.smithi186.stdout:6/384: fdatasync d9/d43/d59/f83 0 2024-04-12T20:56:32.399 INFO:tasks.workunit.client.1.smithi186.stdout:9/382: truncate d3/d2f/d2e/d33/f44 333451 0 2024-04-12T20:56:32.399 INFO:tasks.workunit.client.1.smithi186.stdout:1/398: fdatasync d14/f16 0 2024-04-12T20:56:32.399 INFO:tasks.workunit.client.1.smithi186.stdout:0/492: rename d0/d14/d4e/l5b to d0/d85/l9f 0 2024-04-12T20:56:32.406 INFO:tasks.workunit.client.1.smithi186.stdout:0/493: mknod d0/d14/d4e/d50/d6e/d96/ca0 0 2024-04-12T20:56:32.406 INFO:tasks.workunit.client.1.smithi186.stdout:6/385: truncate d9/d4c/d6f/f7b 929987 0 2024-04-12T20:56:32.406 INFO:tasks.workunit.client.1.smithi186.stdout:2/425: creat d0/d3/d1a/d48/f84 x:0 0 0 2024-04-12T20:56:32.407 INFO:tasks.workunit.client.1.smithi186.stdout:8/353: symlink d7/d18/d36/d20/d23/d25/l6d 0 2024-04-12T20:56:32.411 INFO:tasks.workunit.client.1.smithi186.stdout:2/426: stat d0/d3/f58 0 2024-04-12T20:56:32.411 INFO:tasks.workunit.client.1.smithi186.stdout:1/399: mknod d14/d1a/d33/d36/d5b/d3e/d5f/c7f 0 2024-04-12T20:56:32.412 INFO:tasks.workunit.client.1.smithi186.stdout:6/386: rmdir d9/d4c 39 2024-04-12T20:56:32.415 INFO:tasks.workunit.client.1.smithi186.stdout:5/414: truncate d5/d27/f6b 3129138 0 2024-04-12T20:56:32.423 INFO:tasks.workunit.client.1.smithi186.stdout:2/427: symlink d0/d32/l85 0 2024-04-12T20:56:32.424 INFO:tasks.workunit.client.1.smithi186.stdout:5/415: mknod d5/db/d15/d2a/d4c/c8a 0 2024-04-12T20:56:32.425 INFO:tasks.workunit.client.1.smithi186.stdout:4/485: write d0/dc/d33/f57 [453719,48421] 0 2024-04-12T20:56:32.428 INFO:tasks.workunit.client.1.smithi186.stdout:7/439: dwrite d7/d10/d1c/f20 [0,4194304] 0 2024-04-12T20:56:32.432 INFO:tasks.workunit.client.1.smithi186.stdout:4/486: write d0/dc/d33/f68 [3216356,9426] 0 2024-04-12T20:56:32.437 INFO:tasks.workunit.client.1.smithi186.stdout:2/428: chown d0/f79 1 1 2024-04-12T20:56:32.437 INFO:tasks.workunit.client.1.smithi186.stdout:7/440: unlink d7/d10/l75 0 2024-04-12T20:56:32.438 INFO:tasks.workunit.client.1.smithi186.stdout:6/387: dread d9/d4c/d6f/f7b [0,4194304] 0 2024-04-12T20:56:32.438 INFO:tasks.workunit.client.1.smithi186.stdout:4/487: fdatasync d0/d21/f40 0 2024-04-12T20:56:32.449 INFO:tasks.workunit.client.1.smithi186.stdout:4/488: symlink d0/dc/d39/l99 0 2024-04-12T20:56:32.454 INFO:tasks.workunit.client.1.smithi186.stdout:1/400: dwrite d14/d1a/d33/d36/d5b/f79 [0,4194304] 0 2024-04-12T20:56:32.466 INFO:tasks.workunit.client.1.smithi186.stdout:4/489: mkdir d0/dc/d33/d9a 0 2024-04-12T20:56:32.474 INFO:tasks.workunit.client.1.smithi186.stdout:4/490: stat d0/dc/d39 0 2024-04-12T20:56:32.477 INFO:tasks.workunit.client.1.smithi186.stdout:2/429: dwrite d0/d32/f6f [0,4194304] 0 2024-04-12T20:56:32.479 INFO:tasks.workunit.client.1.smithi186.stdout:1/401: link d14/d1a/d33/d36/c43 d14/d1a/d1d/c80 0 2024-04-12T20:56:32.480 INFO:tasks.workunit.client.1.smithi186.stdout:5/416: dwrite d5/db/d71/f75 [0,4194304] 0 2024-04-12T20:56:32.487 INFO:tasks.workunit.client.1.smithi186.stdout:3/416: dwrite d1/d3/da/f6c [0,4194304] 0 2024-04-12T20:56:32.487 INFO:tasks.workunit.client.1.smithi186.stdout:1/402: readlink d14/d1a/d1d/l54 0 2024-04-12T20:56:32.488 INFO:tasks.workunit.client.1.smithi186.stdout:7/441: dread d7/d10/f95 [0,4194304] 0 2024-04-12T20:56:32.493 INFO:tasks.workunit.client.1.smithi186.stdout:5/417: unlink d5/db/d71/f16 0 2024-04-12T20:56:32.493 INFO:tasks.workunit.client.1.smithi186.stdout:1/403: fdatasync d14/d1a/d33/d36/d5b/d3e/d71/f76 0 2024-04-12T20:56:32.500 INFO:tasks.workunit.client.1.smithi186.stdout:3/417: symlink d1/d3/d49/d4d/d6f/l8c 0 2024-04-12T20:56:32.506 INFO:tasks.workunit.client.1.smithi186.stdout:7/442: rename d7/d10/d2d/d40/d4a/d5c/f94 to d7/d10/d2d/d40/d4a/d6c/d97/d4d/fa2 0 2024-04-12T20:56:32.510 INFO:tasks.workunit.client.1.smithi186.stdout:3/418: mkdir d1/d2c/d8d 0 2024-04-12T20:56:32.510 INFO:tasks.workunit.client.1.smithi186.stdout:0/494: write d0/d14/d4e/d1e/f4b [548174,129279] 0 2024-04-12T20:56:32.517 INFO:tasks.workunit.client.1.smithi186.stdout:0/495: write d0/d10/d71/f94 [1019441,7272] 0 2024-04-12T20:56:32.518 INFO:tasks.workunit.client.1.smithi186.stdout:1/404: read d14/d1a/f3f [3751852,7515] 0 2024-04-12T20:56:32.522 INFO:tasks.workunit.client.1.smithi186.stdout:4/491: dwrite d0/d21/f20 [4194304,4194304] 0 2024-04-12T20:56:32.522 INFO:tasks.workunit.client.1.smithi186.stdout:7/443: mkdir d7/d10/d2d/d40/d4a/d6c/d97/da3 0 2024-04-12T20:56:32.522 INFO:tasks.workunit.client.1.smithi186.stdout:2/430: dwrite d0/d32/f7c [0,4194304] 0 2024-04-12T20:56:32.524 INFO:tasks.workunit.client.1.smithi186.stdout:7/444: readlink d7/l9b 0 2024-04-12T20:56:32.524 INFO:tasks.workunit.client.1.smithi186.stdout:0/496: fsync d0/d14/d4e/d1e/f98 0 2024-04-12T20:56:32.526 INFO:tasks.workunit.client.1.smithi186.stdout:7/445: readlink d7/d10/d2d/d40/l54 0 2024-04-12T20:56:32.527 INFO:tasks.workunit.client.1.smithi186.stdout:2/431: dread - d0/d3/f80 zero size 2024-04-12T20:56:32.529 INFO:tasks.workunit.client.1.smithi186.stdout:0/497: symlink d0/la1 0 2024-04-12T20:56:32.530 INFO:tasks.workunit.client.1.smithi186.stdout:4/492: truncate d0/dc/d39/f3a 1110658 0 2024-04-12T20:56:32.532 INFO:tasks.workunit.client.1.smithi186.stdout:8/354: sync 2024-04-12T20:56:32.533 INFO:tasks.workunit.client.1.smithi186.stdout:2/432: mkdir d0/d3/d1a/d21/d26/d86 0 2024-04-12T20:56:32.535 INFO:tasks.workunit.client.1.smithi186.stdout:4/493: mkdir d0/d21/d3d/d63/d9b 0 2024-04-12T20:56:32.536 INFO:tasks.workunit.client.1.smithi186.stdout:0/498: rmdir d0/d14/d4e/d50/d6e/d96 39 2024-04-12T20:56:32.537 INFO:tasks.workunit.client.1.smithi186.stdout:7/446: link d7/d10/f15 d7/d55/fa4 0 2024-04-12T20:56:32.545 INFO:tasks.workunit.client.1.smithi186.stdout:8/355: creat d7/d18/d36/d20/d23/d25/d2e/d67/f6e x:0 0 0 2024-04-12T20:56:32.546 INFO:tasks.workunit.client.1.smithi186.stdout:4/494: mkdir d0/dc/d88/d80/d9c 0 2024-04-12T20:56:32.547 INFO:tasks.workunit.client.1.smithi186.stdout:7/447: read d7/f74 [98097,65026] 0 2024-04-12T20:56:32.547 INFO:tasks.workunit.client.1.smithi186.stdout:2/433: dread d0/d32/d2a/f43 [0,4194304] 0 2024-04-12T20:56:32.552 INFO:tasks.workunit.client.1.smithi186.stdout:4/495: chown d0/dc/d33/f68 815459 1 2024-04-12T20:56:32.557 INFO:tasks.workunit.client.1.smithi186.stdout:2/434: truncate d0/d3/d1a/d24/f60 1362917 0 2024-04-12T20:56:32.558 INFO:tasks.workunit.client.1.smithi186.stdout:4/496: mknod d0/d21/d3d/d63/c9d 0 2024-04-12T20:56:32.560 INFO:tasks.workunit.client.1.smithi186.stdout:2/435: truncate d0/f72 342099 0 2024-04-12T20:56:32.567 INFO:tasks.workunit.client.1.smithi186.stdout:4/497: mkdir d0/d21/d3d/d9e 0 2024-04-12T20:56:32.572 INFO:tasks.workunit.client.1.smithi186.stdout:9/383: dread d3/d12/d27/f38 [0,4194304] 0 2024-04-12T20:56:32.575 INFO:tasks.workunit.client.1.smithi186.stdout:1/405: dread d14/d1a/d33/d36/d5b/d3e/f7e [0,4194304] 0 2024-04-12T20:56:32.582 INFO:tasks.workunit.client.1.smithi186.stdout:8/356: dwrite d7/de/d10/f54 [0,4194304] 0 2024-04-12T20:56:32.583 INFO:tasks.workunit.client.1.smithi186.stdout:4/498: creat d0/d21/d3d/d63/f9f x:0 0 0 2024-04-12T20:56:32.587 INFO:tasks.workunit.client.1.smithi186.stdout:1/406: chown d14/d1a/d21/f27 30 1 2024-04-12T20:56:32.589 INFO:tasks.workunit.client.1.smithi186.stdout:6/388: dwrite d9/d43/d50/f55 [0,4194304] 0 2024-04-12T20:56:32.589 INFO:tasks.workunit.client.1.smithi186.stdout:9/384: fsync d3/fd 0 2024-04-12T20:56:32.598 INFO:tasks.workunit.client.1.smithi186.stdout:4/499: write d0/d21/f32 [371588,38539] 0 2024-04-12T20:56:32.598 INFO:tasks.workunit.client.1.smithi186.stdout:9/385: stat d3/d2f/d2e/d33/d36/d48/c7a 0 2024-04-12T20:56:32.598 INFO:tasks.workunit.client.1.smithi186.stdout:1/407: rmdir d14 39 2024-04-12T20:56:32.600 INFO:tasks.workunit.client.1.smithi186.stdout:2/436: dwrite d0/d3/f58 [0,4194304] 0 2024-04-12T20:56:32.601 INFO:tasks.workunit.client.1.smithi186.stdout:8/357: symlink d7/de/d5e/l6f 0 2024-04-12T20:56:32.607 INFO:tasks.workunit.client.1.smithi186.stdout:6/389: creat d9/d5e/d7a/d81/f8d x:0 0 0 2024-04-12T20:56:32.609 INFO:tasks.workunit.client.1.smithi186.stdout:4/500: write d0/d21/f27 [2175036,101370] 0 2024-04-12T20:56:32.609 INFO:tasks.workunit.client.1.smithi186.stdout:9/386: rename d3/d12/d27 to d3/d12/d6f/d83 0 2024-04-12T20:56:32.612 INFO:tasks.workunit.client.1.smithi186.stdout:8/358: creat d7/d18/d36/d20/d23/d25/d2e/f70 x:0 0 0 2024-04-12T20:56:32.625 INFO:tasks.workunit.client.1.smithi186.stdout:1/408: write d14/f4f [242912,53860] 0 2024-04-12T20:56:32.628 INFO:tasks.workunit.client.1.smithi186.stdout:5/418: dwrite d5/db/d71/f52 [0,4194304] 0 2024-04-12T20:56:32.632 INFO:tasks.workunit.client.1.smithi186.stdout:6/390: write d9/f12 [6415674,95900] 0 2024-04-12T20:56:32.636 INFO:tasks.workunit.client.1.smithi186.stdout:6/391: readlink d9/l14 0 2024-04-12T20:56:32.637 INFO:tasks.workunit.client.1.smithi186.stdout:8/359: mkdir d7/d18/d36/d20/d71 0 2024-04-12T20:56:32.638 INFO:tasks.workunit.client.1.smithi186.stdout:4/501: truncate d0/d21/f40 278744 0 2024-04-12T20:56:32.639 INFO:tasks.workunit.client.1.smithi186.stdout:3/419: dwrite d1/d3/da/f47 [0,4194304] 0 2024-04-12T20:56:32.651 INFO:tasks.workunit.client.1.smithi186.stdout:1/409: chown d14/d1a/c20 0 1 2024-04-12T20:56:32.652 INFO:tasks.workunit.client.1.smithi186.stdout:8/360: symlink d7/de/d10/l72 0 2024-04-12T20:56:32.656 INFO:tasks.workunit.client.1.smithi186.stdout:4/502: read d0/d21/f6a [97712,34200] 0 2024-04-12T20:56:32.662 INFO:tasks.workunit.client.1.smithi186.stdout:9/387: link d3/l15 d3/d2f/d54/l84 0 2024-04-12T20:56:32.663 INFO:tasks.workunit.client.1.smithi186.stdout:2/437: dwrite d0/d3/d1a/d21/d22/f6c [0,4194304] 0 2024-04-12T20:56:32.665 INFO:tasks.workunit.client.1.smithi186.stdout:1/410: symlink d14/d74/l81 0 2024-04-12T20:56:32.676 INFO:tasks.workunit.client.1.smithi186.stdout:4/503: symlink d0/dc/d64/la0 0 2024-04-12T20:56:32.676 INFO:tasks.workunit.client.1.smithi186.stdout:9/388: fsync d3/d2f/f21 0 2024-04-12T20:56:32.677 INFO:tasks.workunit.client.1.smithi186.stdout:0/499: dwrite d0/d10/d32/d3f/f77 [0,4194304] 0 2024-04-12T20:56:32.681 INFO:tasks.workunit.client.1.smithi186.stdout:3/420: link d1/d2c/c3c d1/c8e 0 2024-04-12T20:56:32.687 INFO:tasks.workunit.client.1.smithi186.stdout:7/448: dwrite d7/d10/f15 [0,4194304] 0 2024-04-12T20:56:32.688 INFO:tasks.workunit.client.1.smithi186.stdout:9/389: mknod d3/d2f/d54/c85 0 2024-04-12T20:56:32.690 INFO:tasks.workunit.client.1.smithi186.stdout:6/392: dwrite d9/f1d [0,4194304] 0 2024-04-12T20:56:32.696 INFO:tasks.workunit.client.1.smithi186.stdout:3/421: write d1/d3/d21/f65 [826754,77611] 0 2024-04-12T20:56:32.699 INFO:tasks.workunit.client.1.smithi186.stdout:0/500: getdents d0/d10/d72/d79 0 2024-04-12T20:56:32.701 INFO:tasks.workunit.client.1.smithi186.stdout:8/361: symlink d7/d18/d36/d20/d23/d25/d2e/l73 0 2024-04-12T20:56:32.706 INFO:tasks.workunit.client.1.smithi186.stdout:6/393: mknod d9/c8e 0 2024-04-12T20:56:32.706 INFO:tasks.workunit.client.1.smithi186.stdout:4/504: read d0/dc/d33/f4 [1318452,88343] 0 2024-04-12T20:56:32.711 INFO:tasks.workunit.client.1.smithi186.stdout:8/362: read d7/de/d10/f19 [192535,127053] 0 2024-04-12T20:56:32.711 INFO:tasks.workunit.client.1.smithi186.stdout:2/438: dwrite d0/d3/d1a/d41/f5c [0,4194304] 0 2024-04-12T20:56:32.712 INFO:tasks.workunit.client.1.smithi186.stdout:0/501: creat d0/d14/d4e/d1e/d84/fa2 x:0 0 0 2024-04-12T20:56:32.713 INFO:tasks.workunit.client.1.smithi186.stdout:4/505: rename d0/d21 to d0/d21/d3d/d63/da1 22 2024-04-12T20:56:32.713 INFO:tasks.workunit.client.1.smithi186.stdout:7/449: mknod d7/d10/d2d/d40/d4a/d6c/d97/d4d/d8a/ca5 0 2024-04-12T20:56:32.715 INFO:tasks.workunit.client.1.smithi186.stdout:6/394: rmdir d9/d43/d59/d5d 39 2024-04-12T20:56:32.719 INFO:tasks.workunit.client.1.smithi186.stdout:6/395: write d9/d4c/f61 [780213,18098] 0 2024-04-12T20:56:32.723 INFO:tasks.workunit.client.1.smithi186.stdout:8/363: symlink d7/d18/d53/l74 0 2024-04-12T20:56:32.723 INFO:tasks.workunit.client.1.smithi186.stdout:7/450: read - d7/d10/d2d/d40/d4a/d6c/d97/f78 zero size 2024-04-12T20:56:32.725 INFO:tasks.workunit.client.1.smithi186.stdout:6/396: creat d9/d43/f8f x:0 0 0 2024-04-12T20:56:32.727 INFO:tasks.workunit.client.1.smithi186.stdout:7/451: chown d7/d55/fa4 98549 1 2024-04-12T20:56:32.728 INFO:tasks.workunit.client.1.smithi186.stdout:7/452: chown d7/f16 1504796 1 2024-04-12T20:56:32.728 INFO:tasks.workunit.client.1.smithi186.stdout:2/439: creat d0/d3/d1a/f87 x:0 0 0 2024-04-12T20:56:32.731 INFO:tasks.workunit.client.1.smithi186.stdout:0/502: creat d0/d14/d4e/d1e/fa3 x:0 0 0 2024-04-12T20:56:32.732 INFO:tasks.workunit.client.1.smithi186.stdout:7/453: creat d7/d10/d2d/d40/d4a/fa6 x:0 0 0 2024-04-12T20:56:32.738 INFO:tasks.workunit.client.1.smithi186.stdout:0/503: creat d0/d5c/d65/fa4 x:0 0 0 2024-04-12T20:56:32.742 INFO:tasks.workunit.client.1.smithi186.stdout:0/504: read d0/d14/f19 [1293255,34294] 0 2024-04-12T20:56:32.743 INFO:tasks.workunit.client.1.smithi186.stdout:4/506: dwrite d0/d21/f83 [0,4194304] 0 2024-04-12T20:56:32.745 INFO:tasks.workunit.client.1.smithi186.stdout:8/364: dwrite f2 [0,4194304] 0 2024-04-12T20:56:32.750 INFO:tasks.workunit.client.1.smithi186.stdout:0/505: write d0/d81/f5a [446162,58554] 0 2024-04-12T20:56:32.754 INFO:tasks.workunit.client.1.smithi186.stdout:7/454: dread d7/f12 [0,4194304] 0 2024-04-12T20:56:32.758 INFO:tasks.workunit.client.1.smithi186.stdout:0/506: fsync d0/d10/f43 0 2024-04-12T20:56:32.763 INFO:tasks.workunit.client.1.smithi186.stdout:2/440: dread d0/dc/f40 [0,4194304] 0 2024-04-12T20:56:32.767 INFO:tasks.workunit.client.1.smithi186.stdout:7/455: symlink d7/d10/d2d/d86/la7 0 2024-04-12T20:56:32.768 INFO:tasks.workunit.client.1.smithi186.stdout:0/507: dread - d0/d5c/d65/fa4 zero size 2024-04-12T20:56:32.772 INFO:tasks.workunit.client.1.smithi186.stdout:7/456: write d7/d55/fa4 [540122,51385] 0 2024-04-12T20:56:32.773 INFO:tasks.workunit.client.1.smithi186.stdout:2/441: stat d0/d3/d1a/d48/c6e 0 2024-04-12T20:56:32.776 INFO:tasks.workunit.client.1.smithi186.stdout:1/411: truncate d14/d1a/d33/d36/d5b/f79 2336130 0 2024-04-12T20:56:32.776 INFO:tasks.workunit.client.1.smithi186.stdout:0/508: symlink d0/d14/la5 0 2024-04-12T20:56:32.776 INFO:tasks.workunit.client.1.smithi186.stdout:7/457: readlink d7/d10/l3b 0 2024-04-12T20:56:32.780 INFO:tasks.workunit.client.1.smithi186.stdout:2/442: dread - d0/d3/d1a/d41/f44 zero size 2024-04-12T20:56:32.786 INFO:tasks.workunit.client.1.smithi186.stdout:8/365: dwrite d7/d18/d36/d20/f4a [0,4194304] 0 2024-04-12T20:56:32.787 INFO:tasks.workunit.client.1.smithi186.stdout:3/422: dread d1/d3/d49/d4d/f51 [0,4194304] 0 2024-04-12T20:56:32.791 INFO:tasks.workunit.client.1.smithi186.stdout:0/509: rename d0/d81/f5a to d0/d14/fa6 0 2024-04-12T20:56:32.797 INFO:tasks.workunit.client.1.smithi186.stdout:2/443: creat d0/d3/d1a/d21/d26/f88 x:0 0 0 2024-04-12T20:56:32.802 INFO:tasks.workunit.client.1.smithi186.stdout:2/444: stat d0/d3/d1a/d41/l61 0 2024-04-12T20:56:32.802 INFO:tasks.workunit.client.1.smithi186.stdout:4/507: truncate d0/dc/d33/fb 3101349 0 2024-04-12T20:56:32.804 INFO:tasks.workunit.client.1.smithi186.stdout:1/412: getdents d14/d1a/d33 0 2024-04-12T20:56:32.806 INFO:tasks.workunit.client.1.smithi186.stdout:0/510: unlink d0/d10/d32/c34 0 2024-04-12T20:56:32.808 INFO:tasks.workunit.client.1.smithi186.stdout:8/366: dread d7/d18/d36/d1d/d1f/d26/f3f [0,4194304] 0 2024-04-12T20:56:32.810 INFO:tasks.workunit.client.1.smithi186.stdout:7/458: dwrite d7/d10/d2d/d67/f70 [0,4194304] 0 2024-04-12T20:56:32.815 INFO:tasks.workunit.client.1.smithi186.stdout:0/511: mkdir d0/d5c/d66/da7 0 2024-04-12T20:56:32.816 INFO:tasks.workunit.client.1.smithi186.stdout:4/508: creat d0/dc/fa2 x:0 0 0 2024-04-12T20:56:32.819 INFO:tasks.workunit.client.1.smithi186.stdout:1/413: rename d14/f16 to d14/d1a/d33/d36/d5b/d3e/d71/f82 0 2024-04-12T20:56:32.830 INFO:tasks.workunit.client.1.smithi186.stdout:8/367: getdents d7/de/d68 0 2024-04-12T20:56:32.831 INFO:tasks.workunit.client.1.smithi186.stdout:3/423: dwrite d1/d3/d21/d5c/f77 [0,4194304] 0 2024-04-12T20:56:32.836 INFO:tasks.workunit.client.1.smithi186.stdout:8/368: chown d7/d18/d53/l74 957838493 1 2024-04-12T20:56:32.836 INFO:tasks.workunit.client.1.smithi186.stdout:4/509: unlink d0/dc/d33/f57 0 2024-04-12T20:56:32.840 INFO:tasks.workunit.client.1.smithi186.stdout:2/445: dwrite d0/d3/d1a/d41/f44 [0,4194304] 0 2024-04-12T20:56:32.845 INFO:tasks.workunit.client.1.smithi186.stdout:7/459: dread d7/d10/d1c/f20 [0,4194304] 0 2024-04-12T20:56:32.845 INFO:tasks.workunit.client.1.smithi186.stdout:4/510: mknod d0/d21/d3d/d48/ca3 0 2024-04-12T20:56:32.848 INFO:tasks.workunit.client.1.smithi186.stdout:0/512: dwrite d0/d10/d71/f7a [0,4194304] 0 2024-04-12T20:56:32.851 INFO:tasks.workunit.client.1.smithi186.stdout:4/511: write d0/d21/f83 [3133395,126329] 0 2024-04-12T20:56:32.851 INFO:tasks.workunit.client.1.smithi186.stdout:8/369: symlink d7/d18/d36/d20/d23/d25/l75 0 2024-04-12T20:56:32.851 INFO:tasks.workunit.client.1.smithi186.stdout:0/513: chown d0/f87 167164 1 2024-04-12T20:56:32.852 INFO:tasks.workunit.client.1.smithi186.stdout:5/419: sync 2024-04-12T20:56:32.854 INFO:tasks.workunit.client.1.smithi186.stdout:4/512: read - d0/d21/d3d/d48/f95 zero size 2024-04-12T20:56:32.855 INFO:tasks.workunit.client.1.smithi186.stdout:1/414: dwrite fe [0,4194304] 0 2024-04-12T20:56:32.855 INFO:tasks.workunit.client.1.smithi186.stdout:2/446: unlink d0/d3/d1a/d27/c65 0 2024-04-12T20:56:32.856 INFO:tasks.workunit.client.1.smithi186.stdout:7/460: symlink d7/d10/d1c/d5d/la8 0 2024-04-12T20:56:32.859 INFO:tasks.workunit.client.1.smithi186.stdout:8/370: chown d7/d18/d36/d20/d23/d25/d2e/l39 163363996 1 2024-04-12T20:56:32.859 INFO:tasks.workunit.client.1.smithi186.stdout:0/514: write d0/f87 [14441,39338] 0 2024-04-12T20:56:32.859 INFO:tasks.workunit.client.1.smithi186.stdout:5/420: unlink d5/db/d71/c35 0 2024-04-12T20:56:32.871 INFO:tasks.workunit.client.1.smithi186.stdout:5/421: write d5/db/d71/f52 [1118783,2242] 0 2024-04-12T20:56:32.872 INFO:tasks.workunit.client.1.smithi186.stdout:0/515: chown d0/d14/d4e/d1e/d84/fa2 97891 1 2024-04-12T20:56:32.874 INFO:tasks.workunit.client.1.smithi186.stdout:4/513: truncate d0/dc/d33/f47 1574810 0 2024-04-12T20:56:32.875 INFO:tasks.workunit.client.1.smithi186.stdout:2/447: creat d0/dc/f89 x:0 0 0 2024-04-12T20:56:32.878 INFO:tasks.workunit.client.1.smithi186.stdout:5/422: dread - d5/db/d15/d2a/d2d/d48/f7c zero size 2024-04-12T20:56:32.880 INFO:tasks.workunit.client.1.smithi186.stdout:4/514: symlink d0/d26/la4 0 2024-04-12T20:56:32.881 INFO:tasks.workunit.client.1.smithi186.stdout:2/448: dread - d0/d3/d1a/d48/f84 zero size 2024-04-12T20:56:32.885 INFO:tasks.workunit.client.1.smithi186.stdout:4/515: creat d0/dc/d88/d80/fa5 x:0 0 0 2024-04-12T20:56:32.892 INFO:tasks.workunit.client.1.smithi186.stdout:5/423: mknod d5/db/d15/d2a/d2d/d46/d7d/c8b 0 2024-04-12T20:56:32.899 INFO:tasks.workunit.client.1.smithi186.stdout:1/415: dread d14/d1a/d3c/f47 [0,4194304] 0 2024-04-12T20:56:32.905 INFO:tasks.workunit.client.1.smithi186.stdout:8/371: dwrite d7/de/f42 [0,4194304] 0 2024-04-12T20:56:32.907 INFO:tasks.workunit.client.1.smithi186.stdout:3/424: dread d1/d3/d49/d4d/f75 [0,4194304] 0 2024-04-12T20:56:32.913 INFO:tasks.workunit.client.1.smithi186.stdout:7/461: dwrite d7/d10/d2d/d40/d4a/fa6 [0,4194304] 0 2024-04-12T20:56:32.916 INFO:tasks.workunit.client.1.smithi186.stdout:8/372: write d7/d18/d36/d20/d23/d25/f56 [584326,108362] 0 2024-04-12T20:56:32.916 INFO:tasks.workunit.client.1.smithi186.stdout:6/397: sync 2024-04-12T20:56:32.916 INFO:tasks.workunit.client.1.smithi186.stdout:9/390: sync 2024-04-12T20:56:32.922 INFO:tasks.workunit.client.1.smithi186.stdout:4/516: write d0/dc/fe [3177438,119132] 0 2024-04-12T20:56:32.922 INFO:tasks.workunit.client.1.smithi186.stdout:0/516: dwrite d0/d10/d32/f3d [0,4194304] 0 2024-04-12T20:56:32.923 INFO:tasks.workunit.client.1.smithi186.stdout:6/398: mknod d9/d5e/d73/c90 0 2024-04-12T20:56:32.932 INFO:tasks.workunit.client.1.smithi186.stdout:3/425: write d1/d3/d21/d22/d24/d2f/f7f [401992,96342] 0 2024-04-12T20:56:32.935 INFO:tasks.workunit.client.1.smithi186.stdout:7/462: symlink d7/d10/d2d/d40/d4a/d6c/d97/d4d/d84/la9 0 2024-04-12T20:56:32.936 INFO:tasks.workunit.client.1.smithi186.stdout:2/449: dwrite d0/d3/d1a/f45 [0,4194304] 0 2024-04-12T20:56:32.939 INFO:tasks.workunit.client.1.smithi186.stdout:8/373: creat d7/d18/d53/f76 x:0 0 0 2024-04-12T20:56:32.940 INFO:tasks.workunit.client.1.smithi186.stdout:4/517: mkdir d0/d21/d3d/d63/da6 0 2024-04-12T20:56:32.945 INFO:tasks.workunit.client.1.smithi186.stdout:6/399: mkdir d9/d43/d59/d5d/d6e/d91 0 2024-04-12T20:56:32.946 INFO:tasks.workunit.client.1.smithi186.stdout:8/374: readlink d7/d18/d36/d20/d23/d25/l6d 0 2024-04-12T20:56:32.953 INFO:tasks.workunit.client.1.smithi186.stdout:1/416: dwrite d14/d1a/d33/d36/d5b/d3e/d71/f76 [0,4194304] 0 2024-04-12T20:56:32.953 INFO:tasks.workunit.client.1.smithi186.stdout:3/426: chown d1/d3/d21/f65 31211 1 2024-04-12T20:56:32.954 INFO:tasks.workunit.client.1.smithi186.stdout:7/463: mkdir d7/d10/d2d/d40/d4a/d5c/daa 0 2024-04-12T20:56:32.961 INFO:tasks.workunit.client.1.smithi186.stdout:9/391: dwrite d3/d12/d6f/d83/f35 [0,4194304] 0 2024-04-12T20:56:32.967 INFO:tasks.workunit.client.1.smithi186.stdout:7/464: stat d7/d10/d2d/d40/d4a/d6c/d97 0 2024-04-12T20:56:32.970 INFO:tasks.workunit.client.1.smithi186.stdout:4/518: link d0/dc/d33/f4 d0/dc/d64/d87/d98/fa7 0 2024-04-12T20:56:32.971 INFO:tasks.workunit.client.1.smithi186.stdout:3/427: stat d1/d3/da/lb 0 2024-04-12T20:56:32.976 INFO:tasks.workunit.client.1.smithi186.stdout:1/417: write d14/d1a/d33/d36/d5b/f61 [5076266,33344] 0 2024-04-12T20:56:32.981 INFO:tasks.workunit.client.1.smithi186.stdout:5/424: dread d5/f9 [4194304,4194304] 0 2024-04-12T20:56:32.989 INFO:tasks.workunit.client.1.smithi186.stdout:4/519: write d0/dc/d33/f56 [1984328,87365] 0 2024-04-12T20:56:32.989 INFO:tasks.workunit.client.1.smithi186.stdout:9/392: link d3/d2f/d54/f5b d3/d12/f86 0 2024-04-12T20:56:32.989 INFO:tasks.workunit.client.1.smithi186.stdout:0/517: dwrite d0/d14/d4e/d50/d6e/f9c [0,4194304] 0 2024-04-12T20:56:32.992 INFO:tasks.workunit.client.1.smithi186.stdout:6/400: dwrite d9/d43/d59/f83 [0,4194304] 0 2024-04-12T20:56:33.001 INFO:tasks.workunit.client.1.smithi186.stdout:0/518: fsync d0/d14/d4e/d1e/d84/f97 0 2024-04-12T20:56:33.001 INFO:tasks.workunit.client.1.smithi186.stdout:8/375: dwrite d7/d18/f4c [4194304,4194304] 0 2024-04-12T20:56:33.003 INFO:tasks.workunit.client.1.smithi186.stdout:0/519: fdatasync d0/d10/f8f 0 2024-04-12T20:56:33.006 INFO:tasks.workunit.client.1.smithi186.stdout:8/376: chown d7/d18/d36/c24 401139 1 2024-04-12T20:56:33.009 INFO:tasks.workunit.client.1.smithi186.stdout:2/450: truncate d0/d3/d1a/d21/d26/f6a 2632358 0 2024-04-12T20:56:33.012 INFO:tasks.workunit.client.1.smithi186.stdout:6/401: mknod d9/d5e/d7a/d81/c92 0 2024-04-12T20:56:33.012 INFO:tasks.workunit.client.1.smithi186.stdout:2/451: truncate d0/d3/d1a/d48/f81 1037619 0 2024-04-12T20:56:33.016 INFO:tasks.workunit.client.1.smithi186.stdout:9/393: rename d3/d2f/d2e/d33/l80 to d3/d2f/d2e/d33/l87 0 2024-04-12T20:56:33.024 INFO:tasks.workunit.client.1.smithi186.stdout:0/520: creat d0/d10/fa8 x:0 0 0 2024-04-12T20:56:33.024 INFO:tasks.workunit.client.1.smithi186.stdout:7/465: dwrite d7/d10/d1c/f7e [0,4194304] 0 2024-04-12T20:56:33.028 INFO:tasks.workunit.client.1.smithi186.stdout:8/377: unlink d7/d18/d36/d1d/d1f/l5d 0 2024-04-12T20:56:33.029 INFO:tasks.workunit.client.1.smithi186.stdout:8/378: fsync d7/de/f42 0 2024-04-12T20:56:33.030 INFO:tasks.workunit.client.1.smithi186.stdout:7/466: chown d7/d10/d2d/d40/d4a/d6c/d97/d4d/d84/la9 897535 1 2024-04-12T20:56:33.032 INFO:tasks.workunit.client.1.smithi186.stdout:5/425: rename d5/db/d15/d2a/d2d/d46/d55/c74 to d5/db/d15/d2a/d2d/d46/d7d/c8c 0 2024-04-12T20:56:33.033 INFO:tasks.workunit.client.1.smithi186.stdout:6/402: creat d9/d43/f93 x:0 0 0 2024-04-12T20:56:33.034 INFO:tasks.workunit.client.1.smithi186.stdout:1/418: write d14/d1a/d21/f31 [448610,123236] 0 2024-04-12T20:56:33.036 INFO:tasks.workunit.client.1.smithi186.stdout:2/452: fdatasync d0/d3/d1a/d24/f34 0 2024-04-12T20:56:33.036 INFO:tasks.workunit.client.1.smithi186.stdout:9/394: symlink d3/d2f/d2e/d43/d4c/l88 0 2024-04-12T20:56:33.039 INFO:tasks.workunit.client.1.smithi186.stdout:2/453: chown d0/d32/d2a 29 1 2024-04-12T20:56:33.042 INFO:tasks.workunit.client.1.smithi186.stdout:2/454: dread - d0/d3/d1a/f87 zero size 2024-04-12T20:56:33.043 INFO:tasks.workunit.client.1.smithi186.stdout:4/520: rename d0/d26/c8b to d0/d21/d3d/d63/d9b/ca8 0 2024-04-12T20:56:33.044 INFO:tasks.workunit.client.1.smithi186.stdout:5/426: mknod d5/db/d15/d6a/c8d 0 2024-04-12T20:56:33.047 INFO:tasks.workunit.client.1.smithi186.stdout:3/428: dread d1/d2c/d54/d81/f84 [0,4194304] 0 2024-04-12T20:56:33.050 INFO:tasks.workunit.client.1.smithi186.stdout:1/419: chown d14/d1a/d58/d69/c6d 2326446 1 2024-04-12T20:56:33.051 INFO:tasks.workunit.client.1.smithi186.stdout:8/379: mknod d7/d18/d36/d1d/d1f/d26/c77 0 2024-04-12T20:56:33.051 INFO:tasks.workunit.client.1.smithi186.stdout:9/395: creat d3/d2f/d2e/d43/d4c/f89 x:0 0 0 2024-04-12T20:56:33.053 INFO:tasks.workunit.client.1.smithi186.stdout:9/396: fsync d3/d2f/f68 0 2024-04-12T20:56:33.054 INFO:tasks.workunit.client.1.smithi186.stdout:3/429: readlink d1/d3/d21/d22/d4a/l86 0 2024-04-12T20:56:33.057 INFO:tasks.workunit.client.1.smithi186.stdout:3/430: write d1/d3/da/f70 [323999,67668] 0 2024-04-12T20:56:33.057 INFO:tasks.workunit.client.1.smithi186.stdout:4/521: creat d0/dc/d1f/fa9 x:0 0 0 2024-04-12T20:56:33.060 INFO:tasks.workunit.client.1.smithi186.stdout:8/380: symlink d7/d18/d36/d20/d23/d25/d2e/d5c/l78 0 2024-04-12T20:56:33.062 INFO:tasks.workunit.client.1.smithi186.stdout:1/420: mknod d14/d1a/d33/d36/d5b/d6a/c83 0 2024-04-12T20:56:33.062 INFO:tasks.workunit.client.1.smithi186.stdout:3/431: symlink d1/d3/d21/d22/d24/l8f 0 2024-04-12T20:56:33.064 INFO:tasks.workunit.client.1.smithi186.stdout:1/421: stat d14/d1a/d33/d36/l5e 0 2024-04-12T20:56:33.066 INFO:tasks.workunit.client.1.smithi186.stdout:8/381: dread d7/d18/d36/d1d/d1f/d26/f3f [0,4194304] 0 2024-04-12T20:56:33.068 INFO:tasks.workunit.client.1.smithi186.stdout:8/382: chown d7/d18/d36/c24 1678201 1 2024-04-12T20:56:33.069 INFO:tasks.workunit.client.1.smithi186.stdout:1/422: unlink d14/d1a/c30 0 2024-04-12T20:56:33.071 INFO:tasks.workunit.client.1.smithi186.stdout:2/455: dread d0/d32/f38 [0,4194304] 0 2024-04-12T20:56:33.074 INFO:tasks.workunit.client.1.smithi186.stdout:4/522: dwrite d0/f93 [0,4194304] 0 2024-04-12T20:56:33.076 INFO:tasks.workunit.client.1.smithi186.stdout:3/432: rename d1/d3/d49/d4d/f73 to d1/d3/d21/d22/f90 0 2024-04-12T20:56:33.077 INFO:tasks.workunit.client.1.smithi186.stdout:4/523: fsync d0/f81 0 2024-04-12T20:56:33.080 INFO:tasks.workunit.client.1.smithi186.stdout:2/456: mknod d0/d3/d1a/d24/c8a 0 2024-04-12T20:56:33.082 INFO:tasks.workunit.client.1.smithi186.stdout:9/397: dread d3/fb [4194304,4194304] 0 2024-04-12T20:56:33.083 INFO:tasks.workunit.client.1.smithi186.stdout:2/457: fdatasync d0/d3/d1a/d24/f60 0 2024-04-12T20:56:33.090 INFO:tasks.workunit.client.1.smithi186.stdout:9/398: dread - d3/f19 zero size 2024-04-12T20:56:33.096 INFO:tasks.workunit.client.1.smithi186.stdout:7/467: write d7/f17 [7643458,13151] 0 2024-04-12T20:56:33.096 INFO:tasks.workunit.client.1.smithi186.stdout:2/458: read d0/d32/d2a/f62 [4561836,23630] 0 2024-04-12T20:56:33.101 INFO:tasks.workunit.client.1.smithi186.stdout:0/521: write d0/d10/d32/f48 [592583,90575] 0 2024-04-12T20:56:33.102 INFO:tasks.workunit.client.1.smithi186.stdout:2/459: stat d0/d32 0 2024-04-12T20:56:33.102 INFO:tasks.workunit.client.1.smithi186.stdout:1/423: dwrite f9 [4194304,4194304] 0 2024-04-12T20:56:33.108 INFO:tasks.workunit.client.1.smithi186.stdout:9/399: rmdir d3/d2f/d2e/d33 39 2024-04-12T20:56:33.110 INFO:tasks.workunit.client.1.smithi186.stdout:1/424: write d14/d1a/d21/f4b [391398,32326] 0 2024-04-12T20:56:33.115 INFO:tasks.workunit.client.1.smithi186.stdout:8/383: truncate d7/d18/f4c 2447011 0 2024-04-12T20:56:33.117 INFO:tasks.workunit.client.1.smithi186.stdout:2/460: rename d0/d3/d63 to d0/d32/d8b 0 2024-04-12T20:56:33.119 INFO:tasks.workunit.client.1.smithi186.stdout:7/468: truncate d7/d10/d2d/d40/d4a/d5c/f9a 420980 0 2024-04-12T20:56:33.122 INFO:tasks.workunit.client.1.smithi186.stdout:3/433: dwrite d1/d3/d21/d5c/f77 [4194304,4194304] 0 2024-04-12T20:56:33.124 INFO:tasks.workunit.client.1.smithi186.stdout:9/400: write d3/d2f/d2e/d33/f4b [1592446,54432] 0 2024-04-12T20:56:33.125 INFO:tasks.workunit.client.1.smithi186.stdout:6/403: dwrite d9/f10 [0,4194304] 0 2024-04-12T20:56:33.126 INFO:tasks.workunit.client.1.smithi186.stdout:7/469: fsync d7/d10/d2d/d67/f9e 0 2024-04-12T20:56:33.131 INFO:tasks.workunit.client.1.smithi186.stdout:7/470: chown d7/d10/d2d/d40/d4a/f64 63238757 1 2024-04-12T20:56:33.134 INFO:tasks.workunit.client.1.smithi186.stdout:8/384: dread - d7/de/d10/d2f/f4e zero size 2024-04-12T20:56:33.134 INFO:tasks.workunit.client.1.smithi186.stdout:1/425: creat d14/d1a/d33/f84 x:0 0 0 2024-04-12T20:56:33.137 INFO:tasks.workunit.client.1.smithi186.stdout:6/404: creat d9/d43/d59/d5d/f94 x:0 0 0 2024-04-12T20:56:33.139 INFO:tasks.workunit.client.1.smithi186.stdout:4/524: dread d0/dc/d39/f66 [0,4194304] 0 2024-04-12T20:56:33.140 INFO:tasks.workunit.client.1.smithi186.stdout:0/522: dread d0/d5c/f7f [0,4194304] 0 2024-04-12T20:56:33.141 INFO:tasks.workunit.client.1.smithi186.stdout:9/401: symlink d3/d12/d6f/d83/d71/d72/l8a 0 2024-04-12T20:56:33.143 INFO:tasks.workunit.client.1.smithi186.stdout:1/426: mknod d14/d1a/d58/c85 0 2024-04-12T20:56:33.145 INFO:tasks.workunit.client.1.smithi186.stdout:8/385: fsync d7/d18/d36/d20/d23/f45 0 2024-04-12T20:56:33.146 INFO:tasks.workunit.client.1.smithi186.stdout:3/434: getdents d1/d3/d21/d22/d34/d6e 0 2024-04-12T20:56:33.146 INFO:tasks.workunit.client.1.smithi186.stdout:6/405: unlink d9/d43/d2d/f42 0 2024-04-12T20:56:33.151 INFO:tasks.workunit.client.1.smithi186.stdout:9/402: dread - d3/d2f/d54/f5b zero size 2024-04-12T20:56:33.152 INFO:tasks.workunit.client.1.smithi186.stdout:2/461: dread d0/d3/f58 [0,4194304] 0 2024-04-12T20:56:33.158 INFO:tasks.workunit.client.1.smithi186.stdout:4/525: rmdir d0/d21/d3d/d9e 0 2024-04-12T20:56:33.158 INFO:tasks.workunit.client.1.smithi186.stdout:8/386: unlink d7/d18/d36/d20/d23/d25/f5a 0 2024-04-12T20:56:33.167 INFO:tasks.workunit.client.1.smithi186.stdout:0/523: creat d0/d14/d4e/d1e/fa9 x:0 0 0 2024-04-12T20:56:33.168 INFO:tasks.workunit.client.1.smithi186.stdout:6/406: rename d9/l18 to d9/d43/d59/l95 0 2024-04-12T20:56:33.174 INFO:tasks.workunit.client.1.smithi186.stdout:7/471: dread d7/f25 [0,4194304] 0 2024-04-12T20:56:33.175 INFO:tasks.workunit.client.1.smithi186.stdout:1/427: dread - d14/d1a/d33/d36/f7b zero size 2024-04-12T20:56:33.175 INFO:tasks.workunit.client.1.smithi186.stdout:9/403: getdents d3/d2f/d2e/d33/d66 0 2024-04-12T20:56:33.176 INFO:tasks.workunit.client.1.smithi186.stdout:2/462: dread d0/dc/f37 [0,4194304] 0 2024-04-12T20:56:33.177 INFO:tasks.workunit.client.1.smithi186.stdout:6/407: mkdir d9/d5e/d7a/d81/d96 0 2024-04-12T20:56:33.180 INFO:tasks.workunit.client.1.smithi186.stdout:3/435: dwrite d1/f8a [0,4194304] 0 2024-04-12T20:56:33.195 INFO:tasks.workunit.client.1.smithi186.stdout:2/463: write d0/d3/d1a/d41/f44 [3463938,109893] 0 2024-04-12T20:56:33.195 INFO:tasks.workunit.client.1.smithi186.stdout:6/408: mkdir d9/d4c/d6f/d97 0 2024-04-12T20:56:33.198 INFO:tasks.workunit.client.1.smithi186.stdout:5/427: sync 2024-04-12T20:56:33.198 INFO:tasks.workunit.client.1.smithi186.stdout:3/436: symlink d1/d2c/d54/d81/l91 0 2024-04-12T20:56:33.202 INFO:tasks.workunit.client.1.smithi186.stdout:1/428: rename d14/d1a/d33/d36/d5b/d3e/f52 to d14/d1a/d33/d36/d5b/d3e/d71/f86 0 2024-04-12T20:56:33.204 INFO:tasks.workunit.client.1.smithi186.stdout:7/472: write d7/d10/d2d/d40/d4a/d6c/d97/d4d/f73 [719081,94369] 0 2024-04-12T20:56:33.204 INFO:tasks.workunit.client.1.smithi186.stdout:2/464: readlink d0/d3/d1a/l83 0 2024-04-12T20:56:33.216 INFO:tasks.workunit.client.1.smithi186.stdout:3/437: creat d1/d3/d21/d22/d24/d2f/f92 x:0 0 0 2024-04-12T20:56:33.217 INFO:tasks.workunit.client.1.smithi186.stdout:0/524: dwrite d0/d14/d4e/d1e/fa3 [0,4194304] 0 2024-04-12T20:56:33.218 INFO:tasks.workunit.client.1.smithi186.stdout:4/526: dwrite d0/d21/f1b [4194304,4194304] 0 2024-04-12T20:56:33.218 INFO:tasks.workunit.client.1.smithi186.stdout:1/429: write d14/d1a/d21/d35/f6e [957226,67186] 0 2024-04-12T20:56:33.221 INFO:tasks.workunit.client.1.smithi186.stdout:2/465: mknod d0/d3/d1a/d21/d22/c8c 0 2024-04-12T20:56:33.222 INFO:tasks.workunit.client.1.smithi186.stdout:1/430: write d14/f7d [558872,52812] 0 2024-04-12T20:56:33.222 INFO:tasks.workunit.client.1.smithi186.stdout:7/473: symlink d7/d10/d2d/d86/lab 0 2024-04-12T20:56:33.227 INFO:tasks.workunit.client.1.smithi186.stdout:1/431: chown d14/d1a/f5a 3116 1 2024-04-12T20:56:33.233 INFO:tasks.workunit.client.1.smithi186.stdout:2/466: unlink d0/dc/d2e/l56 0 2024-04-12T20:56:33.233 INFO:tasks.workunit.client.1.smithi186.stdout:0/525: creat d0/d14/d4e/d50/d6e/d96/faa x:0 0 0 2024-04-12T20:56:33.235 INFO:tasks.workunit.client.1.smithi186.stdout:1/432: unlink d14/d1a/d3c/f47 0 2024-04-12T20:56:33.241 INFO:tasks.workunit.client.1.smithi186.stdout:8/387: dread d7/d18/f4c [0,4194304] 0 2024-04-12T20:56:33.246 INFO:tasks.workunit.client.1.smithi186.stdout:1/433: stat d14/d1a/c20 0 2024-04-12T20:56:33.247 INFO:tasks.workunit.client.1.smithi186.stdout:4/527: link d0/dc/d39/c45 d0/d21/d3d/d63/caa 0 2024-04-12T20:56:33.249 INFO:tasks.workunit.client.1.smithi186.stdout:2/467: mkdir d0/d3/d7a/d8d 0 2024-04-12T20:56:33.250 INFO:tasks.workunit.client.1.smithi186.stdout:9/404: dread d3/ff [0,4194304] 0 2024-04-12T20:56:33.253 INFO:tasks.workunit.client.1.smithi186.stdout:1/434: read d14/d1a/d33/f6b [2873830,5840] 0 2024-04-12T20:56:33.255 INFO:tasks.workunit.client.1.smithi186.stdout:5/428: truncate d5/db/d71/f52 1372990 0 2024-04-12T20:56:33.257 INFO:tasks.workunit.client.1.smithi186.stdout:8/388: dread - d7/d18/d36/d1d/d1f/d26/f63 zero size 2024-04-12T20:56:33.258 INFO:tasks.workunit.client.1.smithi186.stdout:2/468: read d0/d3/d1a/d41/f44 [625378,114536] 0 2024-04-12T20:56:33.269 INFO:tasks.workunit.client.1.smithi186.stdout:7/474: dwrite d7/d10/d1c/f7e [4194304,4194304] 0 2024-04-12T20:56:33.271 INFO:tasks.workunit.client.1.smithi186.stdout:9/405: rename d3/d12/d6f/d83/d71/f76 to d3/d2f/d2e/d33/d66/d77/f8b 0 2024-04-12T20:56:33.280 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:56:33 smithi186 ceph-mon[36597]: pgmap v49: 65 pgs: 65 active+clean; 2.8 GiB data, 9.7 GiB used, 527 GiB / 536 GiB avail; 66 MiB/s rd, 214 MiB/s wr, 495 op/s 2024-04-12T20:56:33.285 INFO:tasks.workunit.client.1.smithi186.stdout:9/406: creat d3/d12/d6f/f8c x:0 0 0 2024-04-12T20:56:33.286 INFO:tasks.workunit.client.1.smithi186.stdout:1/435: getdents d14 0 2024-04-12T20:56:33.293 INFO:tasks.workunit.client.1.smithi186.stdout:0/526: dwrite d0/d10/d32/d3f/f78 [0,4194304] 0 2024-04-12T20:56:33.297 INFO:tasks.workunit.client.1.smithi186.stdout:8/389: getdents d7/de/d5e 0 2024-04-12T20:56:33.298 INFO:tasks.workunit.client.1.smithi186.stdout:2/469: dwrite d0/d3/f1f [0,4194304] 0 2024-04-12T20:56:33.299 INFO:tasks.workunit.client.1.smithi186.stdout:7/475: truncate d7/d10/d2d/d40/d4a/f64 48571 0 2024-04-12T20:56:33.302 INFO:tasks.workunit.client.1.smithi186.stdout:4/528: dwrite d0/d21/f1b [0,4194304] 0 2024-04-12T20:56:33.305 INFO:tasks.workunit.client.1.smithi186.stdout:1/436: write d14/d1a/d21/f27 [1266032,70446] 0 2024-04-12T20:56:33.305 INFO:tasks.workunit.client.1.smithi186.stdout:5/429: dwrite d5/db/d15/f31 [0,4194304] 0 2024-04-12T20:56:33.306 INFO:tasks.workunit.client.1.smithi186.stdout:7/476: fsync d7/d55/f9f 0 2024-04-12T20:56:33.310 INFO:tasks.workunit.client.1.smithi186.stdout:7/477: fsync d7/f8 0 2024-04-12T20:56:33.310 INFO:tasks.workunit.client.1.smithi186.stdout:9/407: read d3/d2f/d2e/f41 [1422524,79713] 0 2024-04-12T20:56:33.313 INFO:tasks.workunit.client.1.smithi186.stdout:2/470: write d0/d32/f38 [1897157,23728] 0 2024-04-12T20:56:33.314 INFO:tasks.workunit.client.1.smithi186.stdout:0/527: creat d0/d14/d4e/fab x:0 0 0 2024-04-12T20:56:33.314 INFO:tasks.workunit.client.1.smithi186.stdout:4/529: mknod d0/dc/d64/cab 0 2024-04-12T20:56:33.316 INFO:tasks.workunit.client.1.smithi186.stdout:9/408: chown d3/d13/f4e 1 1 2024-04-12T20:56:33.322 INFO:tasks.workunit.client.1.smithi186.stdout:4/530: write d0/dc/fe [1894248,117436] 0 2024-04-12T20:56:33.323 INFO:tasks.workunit.client.1.smithi186.stdout:7/478: mknod d7/d55/cac 0 2024-04-12T20:56:33.324 INFO:tasks.workunit.client.1.smithi186.stdout:5/430: mknod d5/db/d15/d2a/d4c/c8e 0 2024-04-12T20:56:33.326 INFO:tasks.workunit.client.1.smithi186.stdout:2/471: creat d0/d32/d8b/f8e x:0 0 0 2024-04-12T20:56:33.329 INFO:tasks.workunit.client.1.smithi186.stdout:8/390: dread f5 [0,4194304] 0 2024-04-12T20:56:33.333 INFO:tasks.workunit.client.1.smithi186.stdout:1/437: dread d14/d1a/d33/d36/d5b/d3e/f73 [0,4194304] 0 2024-04-12T20:56:33.335 INFO:tasks.workunit.client.1.smithi186.stdout:5/431: write d5/db/d15/d2a/d2d/f42 [1628066,37690] 0 2024-04-12T20:56:33.336 INFO:tasks.workunit.client.1.smithi186.stdout:7/479: chown d7/d10/d1c/l3c 0 1 2024-04-12T20:56:33.339 INFO:tasks.workunit.client.1.smithi186.stdout:1/438: symlink d14/d74/l87 0 2024-04-12T20:56:33.340 INFO:tasks.workunit.client.1.smithi186.stdout:4/531: link d0/dc/d88/c92 d0/cac 0 2024-04-12T20:56:33.342 INFO:tasks.workunit.client.1.smithi186.stdout:8/391: creat d7/d18/d36/d20/d23/d25/d2e/f79 x:0 0 0 2024-04-12T20:56:33.343 INFO:tasks.workunit.client.1.smithi186.stdout:7/480: mkdir d7/d10/d2d/d40/d4a/d6c/d97/d4d/d8a/dad 0 2024-04-12T20:56:33.348 INFO:tasks.workunit.client.1.smithi186.stdout:7/481: rename d7/c22 to d7/cae 0 2024-04-12T20:56:33.352 INFO:tasks.workunit.client.1.smithi186.stdout:8/392: rename d7/de/d10/d2f to d7/de/d7a 0 2024-04-12T20:56:33.356 INFO:tasks.workunit.client.1.smithi186.stdout:7/482: mknod d7/d10/d2d/d40/d4a/d5c/caf 0 2024-04-12T20:56:33.357 INFO:tasks.workunit.client.1.smithi186.stdout:9/409: dwrite d3/fb [0,4194304] 0 2024-04-12T20:56:33.359 INFO:tasks.workunit.client.1.smithi186.stdout:0/528: dread d0/d10/f8d [0,4194304] 0 2024-04-12T20:56:33.365 INFO:tasks.workunit.client.1.smithi186.stdout:8/393: getdents d7 0 2024-04-12T20:56:33.366 INFO:tasks.workunit.client.1.smithi186.stdout:7/483: creat d7/d55/d71/d99/fb0 x:0 0 0 2024-04-12T20:56:33.367 INFO:tasks.workunit.client.1.smithi186.stdout:1/439: write d14/d1a/f1b [852720,88513] 0 2024-04-12T20:56:33.368 INFO:tasks.workunit.client.1.smithi186.stdout:0/529: mkdir d0/d10/d32/dac 0 2024-04-12T20:56:33.372 INFO:tasks.workunit.client.1.smithi186.stdout:0/530: write d0/f3e [4089084,20738] 0 2024-04-12T20:56:33.373 INFO:tasks.workunit.client.1.smithi186.stdout:8/394: chown d7/d18/d36/d1d/d1f/c4b 91153917 1 2024-04-12T20:56:33.374 INFO:tasks.workunit.client.1.smithi186.stdout:8/395: readlink d7/de/l64 0 2024-04-12T20:56:33.378 INFO:tasks.workunit.client.1.smithi186.stdout:7/484: creat d7/d10/d2d/d40/fb1 x:0 0 0 2024-04-12T20:56:33.381 INFO:tasks.workunit.client.1.smithi186.stdout:8/396: fsync d7/d18/d36/d1d/d1f/d26/f6b 0 2024-04-12T20:56:33.381 INFO:tasks.workunit.client.1.smithi186.stdout:5/432: dread d5/f59 [0,4194304] 0 2024-04-12T20:56:33.381 INFO:tasks.workunit.client.1.smithi186.stdout:1/440: link d14/d1a/d1d/l25 d14/d74/l88 0 2024-04-12T20:56:33.386 INFO:tasks.workunit.client.1.smithi186.stdout:8/397: fdatasync d7/de/d10/f54 0 2024-04-12T20:56:33.387 INFO:tasks.workunit.client.1.smithi186.stdout:4/532: dread d0/dc/d33/f68 [0,4194304] 0 2024-04-12T20:56:33.392 INFO:tasks.workunit.client.1.smithi186.stdout:1/441: dread d14/d1a/f5a [0,4194304] 0 2024-04-12T20:56:33.397 INFO:tasks.workunit.client.1.smithi186.stdout:1/442: symlink d14/d6c/l89 0 2024-04-12T20:56:33.400 INFO:tasks.workunit.client.1.smithi186.stdout:0/531: dwrite d0/d10/f8d [0,4194304] 0 2024-04-12T20:56:33.402 INFO:tasks.workunit.client.1.smithi186.stdout:1/443: write d14/d1a/d1d/f29 [1175946,60806] 0 2024-04-12T20:56:33.404 INFO:tasks.workunit.client.1.smithi186.stdout:4/533: truncate d0/d21/f27 388096 0 2024-04-12T20:56:33.407 INFO:tasks.workunit.client.1.smithi186.stdout:1/444: fsync f9 0 2024-04-12T20:56:33.410 INFO:tasks.workunit.client.1.smithi186.stdout:8/398: dread d7/d18/d36/d20/d23/d25/d29/f6a [0,4194304] 0 2024-04-12T20:56:33.414 INFO:tasks.workunit.client.1.smithi186.stdout:4/534: link d0/dc/fa2 d0/d21/d3d/fad 0 2024-04-12T20:56:33.418 INFO:tasks.workunit.client.1.smithi186.stdout:4/535: chown d0/d26/l5f 260041591 1 2024-04-12T20:56:33.421 INFO:tasks.workunit.client.1.smithi186.stdout:8/399: rename d7/d18/f1e to d7/d18/d36/d20/d23/d25/d2e/d5c/f7b 0 2024-04-12T20:56:33.424 INFO:tasks.workunit.client.1.smithi186.stdout:5/433: dwrite d5/d27/f6b [0,4194304] 0 2024-04-12T20:56:33.426 INFO:tasks.workunit.client.1.smithi186.stdout:4/536: unlink d0/dc/l3c 0 2024-04-12T20:56:33.429 INFO:tasks.workunit.client.1.smithi186.stdout:6/409: sync 2024-04-12T20:56:33.430 INFO:tasks.workunit.client.1.smithi186.stdout:0/532: dread d0/d10/d71/f89 [0,4194304] 0 2024-04-12T20:56:33.433 INFO:tasks.workunit.client.1.smithi186.stdout:6/410: stat d9/d43/d2d/c5c 0 2024-04-12T20:56:33.434 INFO:tasks.workunit.client.1.smithi186.stdout:9/410: dread d3/d2f/f21 [0,4194304] 0 2024-04-12T20:56:33.434 INFO:tasks.workunit.client.1.smithi186.stdout:7/485: dread d7/d10/d2d/d40/d4a/d6c/d97/d4d/f6d [0,4194304] 0 2024-04-12T20:56:33.436 INFO:tasks.workunit.client.1.smithi186.stdout:8/400: rename d7/d18/f43 to d7/d18/d36/d20/d23/d25/d2e/d67/f7c 0 2024-04-12T20:56:33.436 INFO:tasks.workunit.client.1.smithi186.stdout:4/537: truncate d0/dc/d88/d80/fa5 357606 0 2024-04-12T20:56:33.441 INFO:tasks.workunit.client.1.smithi186.stdout:6/411: mkdir d9/d5e/d7a/d81/d96/d98 0 2024-04-12T20:56:33.441 INFO:tasks.workunit.client.1.smithi186.stdout:7/486: dread - d7/d10/d2d/d40/d4a/d5c/f8d zero size 2024-04-12T20:56:33.441 INFO:tasks.workunit.client.1.smithi186.stdout:4/538: chown d0/f6 0 1 2024-04-12T20:56:33.442 INFO:tasks.workunit.client.1.smithi186.stdout:8/401: dread - d7/d18/d36/d20/d23/d25/f34 zero size 2024-04-12T20:56:33.446 INFO:tasks.workunit.client.1.smithi186.stdout:8/402: chown d7/d18/d36/d20/d23/d25/d2e/d5c/l78 8 1 2024-04-12T20:56:33.447 INFO:tasks.workunit.client.1.smithi186.stdout:5/434: dwrite d5/db/d33/f36 [0,4194304] 0 2024-04-12T20:56:33.449 INFO:tasks.workunit.client.1.smithi186.stdout:6/412: chown d9/d43/d2d/c3a 30 1 2024-04-12T20:56:33.453 INFO:tasks.workunit.client.1.smithi186.stdout:5/435: chown d5/db/d71/f1a 1100933 1 2024-04-12T20:56:33.462 INFO:tasks.workunit.client.1.smithi186.stdout:6/413: fsync d9/d43/f44 0 2024-04-12T20:56:33.463 INFO:tasks.workunit.client.1.smithi186.stdout:5/436: symlink d5/d27/l8f 0 2024-04-12T20:56:33.464 INFO:tasks.workunit.client.1.smithi186.stdout:9/411: dwrite d3/d2f/d2e/d33/f4b [0,4194304] 0 2024-04-12T20:56:33.465 INFO:tasks.workunit.client.1.smithi186.stdout:4/539: creat d0/fae x:0 0 0 2024-04-12T20:56:33.466 INFO:tasks.workunit.client.1.smithi186.stdout:8/403: mkdir d7/d18/d36/d1d/d7d 0 2024-04-12T20:56:33.466 INFO:tasks.workunit.client.1.smithi186.stdout:6/414: dread - d9/d43/d59/d5d/f53 zero size 2024-04-12T20:56:33.468 INFO:tasks.workunit.client.1.smithi186.stdout:7/487: dwrite d7/d10/f19 [4194304,4194304] 0 2024-04-12T20:56:33.470 INFO:tasks.workunit.client.1.smithi186.stdout:5/437: write d5/f59 [43144,118006] 0 2024-04-12T20:56:33.475 INFO:tasks.workunit.client.1.smithi186.stdout:7/488: fdatasync d7/d55/d71/d99/f9d 0 2024-04-12T20:56:33.475 INFO:tasks.workunit.client.1.smithi186.stdout:0/533: dwrite d0/d10/d32/d3f/f54 [0,4194304] 0 2024-04-12T20:56:33.488 INFO:tasks.workunit.client.1.smithi186.stdout:9/412: creat d3/d2f/d2e/d33/f8d x:0 0 0 2024-04-12T20:56:33.488 INFO:tasks.workunit.client.1.smithi186.stdout:8/404: creat d7/d18/d36/d20/d23/f7e x:0 0 0 2024-04-12T20:56:33.489 INFO:tasks.workunit.client.1.smithi186.stdout:9/413: dread - d3/d2f/d2e/d43/d4c/f89 zero size 2024-04-12T20:56:33.491 INFO:tasks.workunit.client.1.smithi186.stdout:6/415: symlink d9/d5e/d73/l99 0 2024-04-12T20:56:33.492 INFO:tasks.workunit.client.1.smithi186.stdout:5/438: rename d5/db/d15/d2a/d2d/f42 to d5/db/d15/d2a/d2d/f90 0 2024-04-12T20:56:33.493 INFO:tasks.workunit.client.1.smithi186.stdout:4/540: mkdir d0/d21/d3d/d63/da6/daf 0 2024-04-12T20:56:33.493 INFO:tasks.workunit.client.1.smithi186.stdout:5/439: fdatasync d5/db/f76 0 2024-04-12T20:56:33.493 INFO:tasks.workunit.client.1.smithi186.stdout:6/416: chown d9/d43/d50/f54 2 1 2024-04-12T20:56:33.494 INFO:tasks.workunit.client.1.smithi186.stdout:8/405: mkdir d7/de/d10/d7f 0 2024-04-12T20:56:33.496 INFO:tasks.workunit.client.1.smithi186.stdout:5/440: creat d5/d27/f91 x:0 0 0 2024-04-12T20:56:33.497 INFO:tasks.workunit.client.1.smithi186.stdout:6/417: creat d9/d4c/d6f/f9a x:0 0 0 2024-04-12T20:56:33.498 INFO:tasks.workunit.client.1.smithi186.stdout:6/418: readlink d9/d43/l3b 0 2024-04-12T20:56:33.500 INFO:tasks.workunit.client.1.smithi186.stdout:4/541: symlink d0/dc/d33/lb0 0 2024-04-12T20:56:33.500 INFO:tasks.workunit.client.1.smithi186.stdout:7/489: link d7/d10/c2a d7/d10/d2d/d86/cb2 0 2024-04-12T20:56:33.501 INFO:tasks.workunit.client.1.smithi186.stdout:9/414: link d3/d2f/d2e/l56 d3/d2f/d2e/l8e 0 2024-04-12T20:56:33.503 INFO:tasks.workunit.client.1.smithi186.stdout:4/542: mkdir d0/d26/db1 0 2024-04-12T20:56:33.506 INFO:tasks.workunit.client.1.smithi186.stdout:7/490: unlink d7/c61 0 2024-04-12T20:56:33.506 INFO:tasks.workunit.client.1.smithi186.stdout:6/419: creat d9/d4c/d6f/d89/f9b x:0 0 0 2024-04-12T20:56:33.508 INFO:tasks.workunit.client.1.smithi186.stdout:9/415: creat d3/d2f/d2e/d43/f8f x:0 0 0 2024-04-12T20:56:33.510 INFO:tasks.workunit.client.1.smithi186.stdout:6/420: chown d9/d43/d50/l7d 14371 1 2024-04-12T20:56:33.514 INFO:tasks.workunit.client.1.smithi186.stdout:8/406: dwrite d7/d18/d36/d1d/d1f/f49 [0,4194304] 0 2024-04-12T20:56:33.515 INFO:tasks.workunit.client.1.smithi186.stdout:5/441: dwrite d5/db/d15/f89 [0,4194304] 0 2024-04-12T20:56:33.518 INFO:tasks.workunit.client.1.smithi186.stdout:8/407: chown d7/d18/d36/d20/d23/d25/l75 267009672 1 2024-04-12T20:56:33.519 INFO:tasks.workunit.client.1.smithi186.stdout:7/491: rename d7/cae to d7/d10/d2d/d40/d4a/cb3 0 2024-04-12T20:56:33.531 INFO:tasks.workunit.client.1.smithi186.stdout:9/416: dwrite d3/d2f/d2e/d33/f4b [0,4194304] 0 2024-04-12T20:56:33.531 INFO:tasks.workunit.client.1.smithi186.stdout:8/408: rename d7/d18/d53/f76 to d7/de/d7a/f80 0 2024-04-12T20:56:33.538 INFO:tasks.workunit.client.1.smithi186.stdout:4/543: truncate d0/dc/fe 1987446 0 2024-04-12T20:56:33.542 INFO:tasks.workunit.client.1.smithi186.stdout:4/544: mkdir d0/d21/d3d/d63/d9b/db2 0 2024-04-12T20:56:33.545 INFO:tasks.workunit.client.1.smithi186.stdout:7/492: dwrite d7/d10/d2d/d40/f77 [0,4194304] 0 2024-04-12T20:56:33.549 INFO:tasks.workunit.client.1.smithi186.stdout:6/421: dread d9/f1b [0,4194304] 0 2024-04-12T20:56:33.553 INFO:tasks.workunit.client.1.smithi186.stdout:7/493: fdatasync d7/d10/f95 0 2024-04-12T20:56:33.555 INFO:tasks.workunit.client.1.smithi186.stdout:4/545: rename d0/dc/d39/c45 to d0/dc/d88/d80/cb3 0 2024-04-12T20:56:33.563 INFO:tasks.workunit.client.1.smithi186.stdout:4/546: mkdir d0/db4 0 2024-04-12T20:56:33.566 INFO:tasks.workunit.client.1.smithi186.stdout:5/442: dread d5/d27/f60 [0,4194304] 0 2024-04-12T20:56:33.566 INFO:tasks.workunit.client.1.smithi186.stdout:4/547: write d0/d21/f32 [4908067,107317] 0 2024-04-12T20:56:33.572 INFO:tasks.workunit.client.1.smithi186.stdout:4/548: symlink d0/d21/d3d/d63/d9b/lb5 0 2024-04-12T20:56:33.574 INFO:tasks.workunit.client.1.smithi186.stdout:6/422: dwrite d9/d43/d50/f5b [0,4194304] 0 2024-04-12T20:56:33.576 INFO:tasks.workunit.client.1.smithi186.stdout:8/409: dread d7/de/f11 [0,4194304] 0 2024-04-12T20:56:33.579 INFO:tasks.workunit.client.1.smithi186.stdout:4/549: creat d0/dc/d1f/d73/fb6 x:0 0 0 2024-04-12T20:56:33.582 INFO:tasks.workunit.client.1.smithi186.stdout:6/423: mkdir d9/d4c/d6f/d89/d9c 0 2024-04-12T20:56:33.584 INFO:tasks.workunit.client.1.smithi186.stdout:6/424: truncate d9/d43/d2d/f8b 632733 0 2024-04-12T20:56:33.590 INFO:tasks.workunit.client.1.smithi186.stdout:6/425: creat d9/d5e/f9d x:0 0 0 2024-04-12T20:56:33.593 INFO:tasks.workunit.client.1.smithi186.stdout:6/426: dread - d9/d43/d2d/f4e zero size 2024-04-12T20:56:33.595 INFO:tasks.workunit.client.1.smithi186.stdout:2/472: sync 2024-04-12T20:56:33.595 INFO:tasks.workunit.client.1.smithi186.stdout:3/438: sync 2024-04-12T20:56:33.597 INFO:tasks.workunit.client.1.smithi186.stdout:5/443: dread d5/db/d15/f1e [0,4194304] 0 2024-04-12T20:56:33.601 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:56:33 smithi018 ceph-mon[28799]: pgmap v49: 65 pgs: 65 active+clean; 2.8 GiB data, 9.7 GiB used, 527 GiB / 536 GiB avail; 66 MiB/s rd, 214 MiB/s wr, 495 op/s 2024-04-12T20:56:33.602 INFO:tasks.workunit.client.1.smithi186.stdout:5/444: rmdir d5/d27 39 2024-04-12T20:56:33.602 INFO:tasks.workunit.client.1.smithi186.stdout:7/494: dwrite d7/d10/d2d/d40/d4a/f64 [0,4194304] 0 2024-04-12T20:56:33.605 INFO:tasks.workunit.client.1.smithi186.stdout:2/473: mkdir d0/d3/d1a/d41/d8f 0 2024-04-12T20:56:33.609 INFO:tasks.workunit.client.1.smithi186.stdout:7/495: fdatasync d7/d10/d2d/d67/f70 0 2024-04-12T20:56:33.614 INFO:tasks.workunit.client.1.smithi186.stdout:8/410: dread d7/de/d10/f27 [0,4194304] 0 2024-04-12T20:56:33.614 INFO:tasks.workunit.client.1.smithi186.stdout:3/439: link d1/d3/d21/l29 d1/d3/da/l93 0 2024-04-12T20:56:33.615 INFO:tasks.workunit.client.1.smithi186.stdout:2/474: truncate d0/d3/d1a/d48/f84 1045871 0 2024-04-12T20:56:33.617 INFO:tasks.workunit.client.1.smithi186.stdout:4/550: dwrite d0/dc/d33/f46 [4194304,4194304] 0 2024-04-12T20:56:33.619 INFO:tasks.workunit.client.1.smithi186.stdout:2/475: symlink d0/l90 0 2024-04-12T20:56:33.620 INFO:tasks.workunit.client.1.smithi186.stdout:6/427: dwrite d9/f1d [4194304,4194304] 0 2024-04-12T20:56:33.626 INFO:tasks.workunit.client.1.smithi186.stdout:4/551: write d0/dc/d39/f79 [970922,59934] 0 2024-04-12T20:56:33.627 INFO:tasks.workunit.client.1.smithi186.stdout:3/440: mkdir d1/d3/d21/d22/d24/d2f/d94 0 2024-04-12T20:56:33.630 INFO:tasks.workunit.client.1.smithi186.stdout:2/476: mkdir d0/d32/d2a/d91 0 2024-04-12T20:56:33.631 INFO:tasks.workunit.client.1.smithi186.stdout:4/552: read - d0/dc/fa2 zero size 2024-04-12T20:56:33.633 INFO:tasks.workunit.client.1.smithi186.stdout:5/445: dread d5/db/d15/f89 [0,4194304] 0 2024-04-12T20:56:33.634 INFO:tasks.workunit.client.1.smithi186.stdout:8/411: mkdir d7/d18/d36/d1d/d7d/d81 0 2024-04-12T20:56:33.636 INFO:tasks.workunit.client.1.smithi186.stdout:2/477: stat d0/d3/d1a/d41/f44 0 2024-04-12T20:56:33.636 INFO:tasks.workunit.client.1.smithi186.stdout:4/553: readlink d0/d21/d3d/d63/l72 0 2024-04-12T20:56:33.637 INFO:tasks.workunit.client.1.smithi186.stdout:3/441: chown d1/d3/d21/d22/f87 129745349 1 2024-04-12T20:56:33.638 INFO:tasks.workunit.client.1.smithi186.stdout:6/428: truncate d9/d43/d59/d5d/f5a 360764 0 2024-04-12T20:56:33.639 INFO:tasks.workunit.client.1.smithi186.stdout:5/446: stat d5/d27/f91 0 2024-04-12T20:56:33.640 INFO:tasks.workunit.client.1.smithi186.stdout:4/554: chown d0/dc/d33/f8a 40397061 1 2024-04-12T20:56:33.642 INFO:tasks.workunit.client.1.smithi186.stdout:2/478: rmdir d0/d3e 39 2024-04-12T20:56:33.642 INFO:tasks.workunit.client.1.smithi186.stdout:7/496: dwrite d7/f17 [0,4194304] 0 2024-04-12T20:56:33.644 INFO:tasks.workunit.client.1.smithi186.stdout:6/429: truncate d9/d43/d2d/d4d/f64 689412 0 2024-04-12T20:56:33.645 INFO:tasks.workunit.client.1.smithi186.stdout:5/447: symlink d5/db/d15/d6a/l92 0 2024-04-12T20:56:33.649 INFO:tasks.workunit.client.1.smithi186.stdout:2/479: unlink d0/d32/d2a/l35 0 2024-04-12T20:56:33.652 INFO:tasks.workunit.client.1.smithi186.stdout:5/448: unlink d5/la 0 2024-04-12T20:56:33.656 INFO:tasks.workunit.client.1.smithi186.stdout:6/430: rmdir d9/d5e/d7a/d81 39 2024-04-12T20:56:33.656 INFO:tasks.workunit.client.1.smithi186.stdout:5/449: readlink d5/db/d71/l1b 0 2024-04-12T20:56:33.659 INFO:tasks.workunit.client.1.smithi186.stdout:6/431: creat d9/d43/d50/f9e x:0 0 0 2024-04-12T20:56:33.660 INFO:tasks.workunit.client.1.smithi186.stdout:5/450: mkdir d5/db/d15/d2a/d4c/d93 0 2024-04-12T20:56:33.662 INFO:tasks.workunit.client.1.smithi186.stdout:3/442: dwrite d1/d3/d21/f65 [0,4194304] 0 2024-04-12T20:56:33.665 INFO:tasks.workunit.client.1.smithi186.stdout:3/443: chown d1 13 1 2024-04-12T20:56:33.667 INFO:tasks.workunit.client.1.smithi186.stdout:8/412: dwrite d7/d18/d36/d1d/d1f/d26/f3f [0,4194304] 0 2024-04-12T20:56:33.667 INFO:tasks.workunit.client.1.smithi186.stdout:6/432: dread d9/f2e [0,4194304] 0 2024-04-12T20:56:33.669 INFO:tasks.workunit.client.1.smithi186.stdout:7/497: fsync d7/f17 0 2024-04-12T20:56:33.671 INFO:tasks.workunit.client.1.smithi186.stdout:7/498: chown d7/d55/f9f 25313 1 2024-04-12T20:56:33.673 INFO:tasks.workunit.client.1.smithi186.stdout:2/480: dwrite d0/d3/d1a/d48/f81 [0,4194304] 0 2024-04-12T20:56:33.673 INFO:tasks.workunit.client.1.smithi186.stdout:3/444: rmdir d1 39 2024-04-12T20:56:33.674 INFO:tasks.workunit.client.1.smithi186.stdout:7/499: write d7/d55/d71/d99/f9d [203256,5502] 0 2024-04-12T20:56:33.675 INFO:tasks.workunit.client.1.smithi186.stdout:2/481: readlink d0/l30 0 2024-04-12T20:56:33.681 INFO:tasks.workunit.client.1.smithi186.stdout:8/413: symlink d7/d18/d36/d20/d23/d25/d2e/d67/l82 0 2024-04-12T20:56:33.682 INFO:tasks.workunit.client.1.smithi186.stdout:8/414: chown d7/d18/d36 0 1 2024-04-12T20:56:33.683 INFO:tasks.workunit.client.1.smithi186.stdout:7/500: symlink d7/d10/d1c/d8c/lb4 0 2024-04-12T20:56:33.684 INFO:tasks.workunit.client.1.smithi186.stdout:5/451: mkdir d5/db/d15/d2a/d4c/d5b/d7a/d94 0 2024-04-12T20:56:33.686 INFO:tasks.workunit.client.1.smithi186.stdout:6/433: creat d9/d43/d59/f9f x:0 0 0 2024-04-12T20:56:33.687 INFO:tasks.workunit.client.1.smithi186.stdout:5/452: chown d5/db/d15/f2f 29521 1 2024-04-12T20:56:33.688 INFO:tasks.workunit.client.1.smithi186.stdout:6/434: read - d9/d43/d59/d5d/f94 zero size 2024-04-12T20:56:33.688 INFO:tasks.workunit.client.1.smithi186.stdout:7/501: chown d7/d10/d2d/d86/cb2 62 1 2024-04-12T20:56:33.688 INFO:tasks.workunit.client.1.smithi186.stdout:3/445: mkdir d1/d3/d21/d22/d24/d2f/d95 0 2024-04-12T20:56:33.689 INFO:tasks.workunit.client.1.smithi186.stdout:8/415: symlink d7/de/d10/d7f/l83 0 2024-04-12T20:56:33.690 INFO:tasks.workunit.client.1.smithi186.stdout:6/435: write d9/d5e/f71 [681276,13647] 0 2024-04-12T20:56:33.691 INFO:tasks.workunit.client.1.smithi186.stdout:5/453: symlink d5/db/d15/d2a/d6d/d7f/d86/l95 0 2024-04-12T20:56:33.693 INFO:tasks.workunit.client.1.smithi186.stdout:3/446: truncate d1/d3/d21/d5c/f62 1043961 0 2024-04-12T20:56:33.695 INFO:tasks.workunit.client.1.smithi186.stdout:5/454: write d5/db/f57 [337031,49856] 0 2024-04-12T20:56:33.695 INFO:tasks.workunit.client.1.smithi186.stdout:7/502: rmdir d7/d10/d2d/d40/d4a/d6c/d97/da3 0 2024-04-12T20:56:33.697 INFO:tasks.workunit.client.1.smithi186.stdout:3/447: symlink d1/d2c/d54/l96 0 2024-04-12T20:56:33.701 INFO:tasks.workunit.client.1.smithi186.stdout:8/416: rename d7/d18/d36/d20/d23/d25/d2e/l73 to d7/d18/d36/d20/d23/d25/d2e/d5c/l84 0 2024-04-12T20:56:33.702 INFO:tasks.workunit.client.1.smithi186.stdout:7/503: creat d7/d10/d1c/d8c/fb5 x:0 0 0 2024-04-12T20:56:33.704 INFO:tasks.workunit.client.1.smithi186.stdout:8/417: read d7/de/d7a/f31 [1543016,49421] 0 2024-04-12T20:56:33.705 INFO:tasks.workunit.client.1.smithi186.stdout:3/448: mknod d1/d3/d49/d4d/d6f/c97 0 2024-04-12T20:56:33.706 INFO:tasks.workunit.client.1.smithi186.stdout:5/455: mknod d5/db/d15/d2a/d4c/d5b/d7a/d94/c96 0 2024-04-12T20:56:33.708 INFO:tasks.workunit.client.1.smithi186.stdout:8/418: dread - d7/d18/d36/d20/d23/d25/d2e/d67/f7c zero size 2024-04-12T20:56:33.708 INFO:tasks.workunit.client.1.smithi186.stdout:5/456: chown d5/db/d15/d2a/d2d/d48/f87 204869270 1 2024-04-12T20:56:33.710 INFO:tasks.workunit.client.1.smithi186.stdout:1/445: sync 2024-04-12T20:56:33.712 INFO:tasks.workunit.client.1.smithi186.stdout:5/457: write d5/db/d71/f61 [723501,15250] 0 2024-04-12T20:56:33.713 INFO:tasks.workunit.client.1.smithi186.stdout:6/436: truncate d9/d43/d2d/f5f 1812906 0 2024-04-12T20:56:33.713 INFO:tasks.workunit.client.1.smithi186.stdout:2/482: dread d0/f10 [0,4194304] 0 2024-04-12T20:56:33.716 INFO:tasks.workunit.client.1.smithi186.stdout:8/419: rename d7/d18/d36/d20/d23/d25/f66 to d7/d18/f85 0 2024-04-12T20:56:33.720 INFO:tasks.workunit.client.1.smithi186.stdout:3/449: truncate d1/f63 2800113 0 2024-04-12T20:56:33.720 INFO:tasks.workunit.client.1.smithi186.stdout:7/504: dwrite d7/f16 [4194304,4194304] 0 2024-04-12T20:56:33.723 INFO:tasks.workunit.client.1.smithi186.stdout:3/450: write d1/d3/d21/d22/d24/d2f/f7f [1157545,11264] 0 2024-04-12T20:56:33.727 INFO:tasks.workunit.client.1.smithi186.stdout:5/458: truncate d5/db/d15/f28 1992231 0 2024-04-12T20:56:33.730 INFO:tasks.workunit.client.1.smithi186.stdout:7/505: creat d7/d55/d71/fb6 x:0 0 0 2024-04-12T20:56:33.730 INFO:tasks.workunit.client.1.smithi186.stdout:1/446: link d14/d1a/d21/d35/l42 d14/d1a/l8a 0 2024-04-12T20:56:33.734 INFO:tasks.workunit.client.1.smithi186.stdout:7/506: chown d7/d10/d2d/d86/cb2 1 1 2024-04-12T20:56:33.740 INFO:tasks.workunit.client.1.smithi186.stdout:2/483: truncate d0/d32/f38 3018402 0 2024-04-12T20:56:33.741 INFO:tasks.workunit.client.1.smithi186.stdout:8/420: dwrite d7/d18/d36/d20/d23/d25/f34 [0,4194304] 0 2024-04-12T20:56:33.746 INFO:tasks.workunit.client.1.smithi186.stdout:2/484: fdatasync d0/d3/d1a/d21/d26/f88 0 2024-04-12T20:56:33.758 INFO:tasks.workunit.client.1.smithi186.stdout:8/421: mknod d7/c86 0 2024-04-12T20:56:33.759 INFO:tasks.workunit.client.1.smithi186.stdout:5/459: link d5/db/d15/d2a/d2d/d46/d7d/c8b d5/db/d15/d2a/d6d/d7f/c97 0 2024-04-12T20:56:33.761 INFO:tasks.workunit.client.1.smithi186.stdout:1/447: write d14/d1a/d33/d36/d5b/d3e/f7e [234765,127966] 0 2024-04-12T20:56:33.761 INFO:tasks.workunit.client.1.smithi186.stdout:6/437: dread d9/d5e/f6d [0,4194304] 0 2024-04-12T20:56:33.762 INFO:tasks.workunit.client.1.smithi186.stdout:3/451: dwrite d1/d3/d21/d5c/f62 [0,4194304] 0 2024-04-12T20:56:33.763 INFO:tasks.workunit.client.1.smithi186.stdout:5/460: truncate d5/fd 1432870 0 2024-04-12T20:56:33.765 INFO:tasks.workunit.client.1.smithi186.stdout:5/461: chown d5/d44 908711 1 2024-04-12T20:56:33.766 INFO:tasks.workunit.client.1.smithi186.stdout:3/452: dread - d1/d3/d21/d22/f87 zero size 2024-04-12T20:56:33.769 INFO:tasks.workunit.client.1.smithi186.stdout:8/422: write d7/d18/d36/d20/d23/f7e [513934,15656] 0 2024-04-12T20:56:33.772 INFO:tasks.workunit.client.1.smithi186.stdout:3/453: creat d1/d3/d21/d22/d34/f98 x:0 0 0 2024-04-12T20:56:33.775 INFO:tasks.workunit.client.1.smithi186.stdout:8/423: mkdir d7/d18/d36/d1d/d7d/d87 0 2024-04-12T20:56:33.777 INFO:tasks.workunit.client.1.smithi186.stdout:3/454: creat d1/d3/d49/d4d/d6f/f99 x:0 0 0 2024-04-12T20:56:33.780 INFO:tasks.workunit.client.1.smithi186.stdout:3/455: write d1/d3/d21/f5d [142102,12811] 0 2024-04-12T20:56:33.781 INFO:tasks.workunit.client.1.smithi186.stdout:6/438: dread d9/d43/d50/f5b [0,4194304] 0 2024-04-12T20:56:33.781 INFO:tasks.workunit.client.1.smithi186.stdout:8/424: symlink d7/de/d68/l88 0 2024-04-12T20:56:33.785 INFO:tasks.workunit.client.1.smithi186.stdout:3/456: chown d1/d3/d49/d4d/f7a 819449460 1 2024-04-12T20:56:33.786 INFO:tasks.workunit.client.1.smithi186.stdout:5/462: dread d5/db/d15/d2a/d2d/f49 [0,4194304] 0 2024-04-12T20:56:33.787 INFO:tasks.workunit.client.1.smithi186.stdout:1/448: dwrite d14/d1a/d33/d36/d5b/d3e/d71/f76 [0,4194304] 0 2024-04-12T20:56:33.787 INFO:tasks.workunit.client.1.smithi186.stdout:6/439: dread d9/f2e [0,4194304] 0 2024-04-12T20:56:33.788 INFO:tasks.workunit.client.1.smithi186.stdout:8/425: write d7/d18/d36/d1d/d1f/d26/f3d [3210633,53263] 0 2024-04-12T20:56:33.794 INFO:tasks.workunit.client.1.smithi186.stdout:5/463: fsync d5/db/d15/f2f 0 2024-04-12T20:56:33.796 INFO:tasks.workunit.client.1.smithi186.stdout:8/426: write d7/d18/d36/d20/d23/d25/d29/f57 [665654,85099] 0 2024-04-12T20:56:33.797 INFO:tasks.workunit.client.1.smithi186.stdout:1/449: getdents d14/d1a/d21/d35/d59 0 2024-04-12T20:56:33.797 INFO:tasks.workunit.client.1.smithi186.stdout:9/417: sync 2024-04-12T20:56:33.798 INFO:tasks.workunit.client.1.smithi186.stdout:4/555: sync 2024-04-12T20:56:33.798 INFO:tasks.workunit.client.1.smithi186.stdout:0/534: sync 2024-04-12T20:56:33.801 INFO:tasks.workunit.client.1.smithi186.stdout:6/440: rename d9/d43/f77 to d9/d43/d59/d5d/d6e/fa0 0 2024-04-12T20:56:33.801 INFO:tasks.workunit.client.1.smithi186.stdout:4/556: read d0/dc/f3b [2828589,7227] 0 2024-04-12T20:56:33.801 INFO:tasks.workunit.client.1.smithi186.stdout:1/450: read d14/f4f [135989,124632] 0 2024-04-12T20:56:33.804 INFO:tasks.workunit.client.1.smithi186.stdout:0/535: readlink d0/d14/d4e/d1e/l41 0 2024-04-12T20:56:33.805 INFO:tasks.workunit.client.1.smithi186.stdout:1/451: stat ff 0 2024-04-12T20:56:33.808 INFO:tasks.workunit.client.1.smithi186.stdout:6/441: symlink d9/d43/d59/la1 0 2024-04-12T20:56:33.809 INFO:tasks.workunit.client.1.smithi186.stdout:9/418: dread d3/d2f/d2e/f2d [0,4194304] 0 2024-04-12T20:56:33.810 INFO:tasks.workunit.client.1.smithi186.stdout:5/464: link d5/db/d15/d2a/d2d/d48/f87 d5/db/d15/d2a/d4c/d93/f98 0 2024-04-12T20:56:33.811 INFO:tasks.workunit.client.1.smithi186.stdout:6/442: write d9/d5e/f71 [272394,121121] 0 2024-04-12T20:56:33.816 INFO:tasks.workunit.client.1.smithi186.stdout:1/452: mkdir d14/d1a/d33/d36/d5b/d3e/d8b 0 2024-04-12T20:56:33.816 INFO:tasks.workunit.client.1.smithi186.stdout:8/427: dread d7/d18/d36/d20/d23/d25/d29/f57 [0,4194304] 0 2024-04-12T20:56:33.816 INFO:tasks.workunit.client.1.smithi186.stdout:6/443: creat d9/d4c/d6f/fa2 x:0 0 0 2024-04-12T20:56:33.818 INFO:tasks.workunit.client.1.smithi186.stdout:6/444: fdatasync d9/d43/d50/f9e 0 2024-04-12T20:56:33.818 INFO:tasks.workunit.client.1.smithi186.stdout:5/465: rename d5/db/d15/d2a/d2d/f49 to d5/db/d2e/d70/f99 0 2024-04-12T20:56:33.820 INFO:tasks.workunit.client.1.smithi186.stdout:1/453: mkdir d14/d1a/d33/d36/d5b/d3e/d71/d8c 0 2024-04-12T20:56:33.821 INFO:tasks.workunit.client.1.smithi186.stdout:6/445: dread d9/f2e [0,4194304] 0 2024-04-12T20:56:33.822 INFO:tasks.workunit.client.1.smithi186.stdout:4/557: truncate d0/dc/d88/d80/fa5 220727 0 2024-04-12T20:56:33.822 INFO:tasks.workunit.client.1.smithi186.stdout:1/454: readlink d14/d74/l81 0 2024-04-12T20:56:33.823 INFO:tasks.workunit.client.1.smithi186.stdout:6/446: read - d9/d43/d50/f9e zero size 2024-04-12T20:56:33.826 INFO:tasks.workunit.client.1.smithi186.stdout:3/457: dread d1/d3/da/f56 [0,4194304] 0 2024-04-12T20:56:33.828 INFO:tasks.workunit.client.1.smithi186.stdout:6/447: symlink d9/d4c/d6f/d97/la3 0 2024-04-12T20:56:33.829 INFO:tasks.workunit.client.1.smithi186.stdout:3/458: chown d1/d3/da/f47 5870725 1 2024-04-12T20:56:33.830 INFO:tasks.workunit.client.1.smithi186.stdout:1/455: dread d14/d1a/d33/d36/d5b/d3e/f73 [0,4194304] 0 2024-04-12T20:56:33.837 INFO:tasks.workunit.client.1.smithi186.stdout:8/428: dwrite f2 [4194304,4194304] 0 2024-04-12T20:56:33.844 INFO:tasks.workunit.client.1.smithi186.stdout:5/466: dwrite d5/db/f5f [0,4194304] 0 2024-04-12T20:56:33.847 INFO:tasks.workunit.client.1.smithi186.stdout:4/558: dwrite d0/dc/d33/f4 [0,4194304] 0 2024-04-12T20:56:33.860 INFO:tasks.workunit.client.1.smithi186.stdout:5/467: dread d5/db/d2e/d70/f99 [0,4194304] 0 2024-04-12T20:56:33.864 INFO:tasks.workunit.client.1.smithi186.stdout:4/559: getdents d0/d21/d3d/d63 0 2024-04-12T20:56:33.866 INFO:tasks.workunit.client.1.smithi186.stdout:8/429: dread d7/d18/d36/d1d/d1f/d26/f3f [0,4194304] 0 2024-04-12T20:56:33.868 INFO:tasks.workunit.client.1.smithi186.stdout:4/560: creat d0/dc/d1f/d73/fb7 x:0 0 0 2024-04-12T20:56:33.871 INFO:tasks.workunit.client.1.smithi186.stdout:8/430: write d7/d18/d36/d20/d23/d25/f46 [720876,2793] 0 2024-04-12T20:56:33.872 INFO:tasks.workunit.client.1.smithi186.stdout:5/468: rename d5/db/d2e/c4a to d5/db/d15/d2a/d6d/d7f/d86/c9a 0 2024-04-12T20:56:33.875 INFO:tasks.workunit.client.1.smithi186.stdout:9/419: dread d3/f1d [0,4194304] 0 2024-04-12T20:56:33.875 INFO:tasks.workunit.client.1.smithi186.stdout:4/561: fsync d0/dc/d64/d87/f97 0 2024-04-12T20:56:33.878 INFO:tasks.workunit.client.1.smithi186.stdout:8/431: rename d7/de/cf to d7/d18/d53/c89 0 2024-04-12T20:56:33.879 INFO:tasks.workunit.client.1.smithi186.stdout:5/469: mknod d5/db/d15/d2a/d6d/d7f/d86/c9b 0 2024-04-12T20:56:33.880 INFO:tasks.workunit.client.1.smithi186.stdout:4/562: symlink d0/dc/d88/d80/lb8 0 2024-04-12T20:56:33.882 INFO:tasks.workunit.client.1.smithi186.stdout:3/459: dread d1/d3/d21/d5c/f77 [0,4194304] 0 2024-04-12T20:56:33.883 INFO:tasks.workunit.client.1.smithi186.stdout:9/420: creat d3/d2f/d2e/d33/d61/f90 x:0 0 0 2024-04-12T20:56:33.884 INFO:tasks.workunit.client.1.smithi186.stdout:4/563: fsync d0/dc/d33/f36 0 2024-04-12T20:56:33.892 INFO:tasks.workunit.client.1.smithi186.stdout:5/470: write d5/db/d15/f1e [2307972,40309] 0 2024-04-12T20:56:33.894 INFO:tasks.workunit.client.1.smithi186.stdout:4/564: mkdir d0/d21/db9 0 2024-04-12T20:56:33.897 INFO:tasks.workunit.client.1.smithi186.stdout:9/421: link d3/d12/d6f/d83/d71/d72/c78 d3/d2f/d2e/d33/d66/c91 0 2024-04-12T20:56:33.897 INFO:tasks.workunit.client.1.smithi186.stdout:3/460: read d1/d3/d21/d5c/f77 [4108126,120434] 0 2024-04-12T20:56:33.897 INFO:tasks.workunit.client.1.smithi186.stdout:1/456: dread f9 [4194304,4194304] 0 2024-04-12T20:56:33.898 INFO:tasks.workunit.client.1.smithi186.stdout:4/565: fsync d0/d21/d3d/d63/f90 0 2024-04-12T20:56:33.898 INFO:tasks.workunit.client.1.smithi186.stdout:5/471: read d5/db/f40 [3954296,37724] 0 2024-04-12T20:56:33.899 INFO:tasks.workunit.client.1.smithi186.stdout:9/422: readlink d3/d12/d6f/d83/l29 0 2024-04-12T20:56:33.899 INFO:tasks.workunit.client.1.smithi186.stdout:8/432: dwrite f2 [4194304,4194304] 0 2024-04-12T20:56:33.904 INFO:tasks.workunit.client.1.smithi186.stdout:4/566: mknod d0/d59/cba 0 2024-04-12T20:56:33.904 INFO:tasks.workunit.client.1.smithi186.stdout:3/461: dread d1/d3/d21/d22/f79 [0,4194304] 0 2024-04-12T20:56:33.906 INFO:tasks.workunit.client.1.smithi186.stdout:1/457: mknod d14/d1a/d21/d35/c8d 0 2024-04-12T20:56:33.907 INFO:tasks.workunit.client.1.smithi186.stdout:9/423: mkdir d3/d2f/d54/d92 0 2024-04-12T20:56:33.909 INFO:tasks.workunit.client.1.smithi186.stdout:8/433: fsync d7/d18/d36/d1d/d1f/d26/f3f 0 2024-04-12T20:56:33.910 INFO:tasks.workunit.client.1.smithi186.stdout:8/434: read f2 [8245577,112388] 0 2024-04-12T20:56:33.911 INFO:tasks.workunit.client.1.smithi186.stdout:8/435: write f2 [8580987,54668] 0 2024-04-12T20:56:33.912 INFO:tasks.workunit.client.1.smithi186.stdout:3/462: creat d1/d3/d21/f9a x:0 0 0 2024-04-12T20:56:33.913 INFO:tasks.workunit.client.1.smithi186.stdout:1/458: readlink d14/d1a/d21/l45 0 2024-04-12T20:56:33.914 INFO:tasks.workunit.client.1.smithi186.stdout:9/424: mkdir d3/d2f/d2e/d33/d36/d93 0 2024-04-12T20:56:33.921 INFO:tasks.workunit.client.1.smithi186.stdout:4/567: creat d0/dc/d33/fbb x:0 0 0 2024-04-12T20:56:33.922 INFO:tasks.workunit.client.1.smithi186.stdout:3/463: rename d1/d2c/d54/d81/f84 to d1/d2c/d64/f9b 0 2024-04-12T20:56:33.924 INFO:tasks.workunit.client.1.smithi186.stdout:9/425: read d3/d2f/f26 [3008836,59254] 0 2024-04-12T20:56:33.924 INFO:tasks.workunit.client.1.smithi186.stdout:5/472: dread d5/db/d15/d2a/d2d/d48/f4b [0,4194304] 0 2024-04-12T20:56:33.925 INFO:tasks.workunit.client.1.smithi186.stdout:1/459: creat d14/d1a/d33/d36/d5b/d3e/d8b/f8e x:0 0 0 2024-04-12T20:56:33.925 INFO:tasks.workunit.client.1.smithi186.stdout:5/473: dread - d5/d27/f91 zero size 2024-04-12T20:56:33.927 INFO:tasks.workunit.client.1.smithi186.stdout:5/474: read - d5/d27/f91 zero size 2024-04-12T20:56:33.928 INFO:tasks.workunit.client.1.smithi186.stdout:4/568: read d0/dc/d33/f6b [3136203,119883] 0 2024-04-12T20:56:33.932 INFO:tasks.workunit.client.1.smithi186.stdout:9/426: getdents d3/d2f/d2e/d33/d36/d48 0 2024-04-12T20:56:33.933 INFO:tasks.workunit.client.1.smithi186.stdout:7/507: sync 2024-04-12T20:56:33.933 INFO:tasks.workunit.client.1.smithi186.stdout:2/485: sync 2024-04-12T20:56:33.934 INFO:tasks.workunit.client.1.smithi186.stdout:1/460: creat d14/d1a/d33/d36/d5b/d3e/d5f/f8f x:0 0 0 2024-04-12T20:56:33.937 INFO:tasks.workunit.client.1.smithi186.stdout:2/486: chown d0/d3/d1a 1929357578 1 2024-04-12T20:56:33.937 INFO:tasks.workunit.client.1.smithi186.stdout:5/475: symlink d5/db/d15/d2a/d2d/d46/d7d/l9c 0 2024-04-12T20:56:33.940 INFO:tasks.workunit.client.1.smithi186.stdout:2/487: write d0/d3/d1a/f45 [1151257,75884] 0 2024-04-12T20:56:33.942 INFO:tasks.workunit.client.1.smithi186.stdout:3/464: dwrite d1/d3/d21/f65 [0,4194304] 0 2024-04-12T20:56:33.942 INFO:tasks.workunit.client.1.smithi186.stdout:8/436: dwrite d7/d18/d36/d20/d23/d25/d2e/d67/f7c [0,4194304] 0 2024-04-12T20:56:33.942 INFO:tasks.workunit.client.1.smithi186.stdout:7/508: mkdir d7/d10/d1c/d5d/db7 0 2024-04-12T20:56:33.942 INFO:tasks.workunit.client.1.smithi186.stdout:2/488: fdatasync d0/f72 0 2024-04-12T20:56:33.944 INFO:tasks.workunit.client.1.smithi186.stdout:1/461: mknod d14/d1a/d33/d36/c90 0 2024-04-12T20:56:33.948 INFO:tasks.workunit.client.1.smithi186.stdout:4/569: creat d0/dc/d64/d87/d98/fbc x:0 0 0 2024-04-12T20:56:33.949 INFO:tasks.workunit.client.1.smithi186.stdout:5/476: creat d5/db/d15/d2a/d2d/f9d x:0 0 0 2024-04-12T20:56:33.951 INFO:tasks.workunit.client.1.smithi186.stdout:2/489: write d0/d32/d69/f7d [3009923,128870] 0 2024-04-12T20:56:33.952 INFO:tasks.workunit.client.1.smithi186.stdout:4/570: chown d0/d21/d3d/d48/d61/c8d 10216571 1 2024-04-12T20:56:33.955 INFO:tasks.workunit.client.1.smithi186.stdout:2/490: truncate d0/dc/f89 247573 0 2024-04-12T20:56:33.956 INFO:tasks.workunit.client.1.smithi186.stdout:1/462: mkdir d14/d91 0 2024-04-12T20:56:33.958 INFO:tasks.workunit.client.1.smithi186.stdout:8/437: mknod d7/d18/d36/d20/d23/d25/d29/c8a 0 2024-04-12T20:56:33.959 INFO:tasks.workunit.client.1.smithi186.stdout:5/477: mknod d5/db/d15/d2a/d2d/d82/c9e 0 2024-04-12T20:56:33.966 INFO:tasks.workunit.client.1.smithi186.stdout:7/509: creat d7/d10/d2d/d40/d4a/d6c/d97/d4d/d8a/dad/fb8 x:0 0 0 2024-04-12T20:56:33.971 INFO:tasks.workunit.client.1.smithi186.stdout:3/465: dwrite d1/d3/d21/d5c/f69 [4194304,4194304] 0 2024-04-12T20:56:33.971 INFO:tasks.workunit.client.1.smithi186.stdout:7/510: write d7/d10/f95 [1294620,5376] 0 2024-04-12T20:56:33.973 INFO:tasks.workunit.client.1.smithi186.stdout:6/448: sync 2024-04-12T20:56:33.974 INFO:tasks.workunit.client.1.smithi186.stdout:0/536: sync 2024-04-12T20:56:33.975 INFO:tasks.workunit.client.1.smithi186.stdout:4/571: dwrite d0/d21/d3d/f7d [0,4194304] 0 2024-04-12T20:56:33.977 INFO:tasks.workunit.client.1.smithi186.stdout:0/537: chown d0/c45 1095 1 2024-04-12T20:56:33.979 INFO:tasks.workunit.client.1.smithi186.stdout:7/511: write d7/fa1 [996768,64038] 0 2024-04-12T20:56:33.980 INFO:tasks.workunit.client.1.smithi186.stdout:8/438: truncate d7/d18/d36/f1b 2022436 0 2024-04-12T20:56:33.983 INFO:tasks.workunit.client.1.smithi186.stdout:9/427: dwrite d3/f3e [0,4194304] 0 2024-04-12T20:56:33.983 INFO:tasks.workunit.client.1.smithi186.stdout:3/466: mkdir d1/d3/d49/d4d/d9c 0 2024-04-12T20:56:33.986 INFO:tasks.workunit.client.1.smithi186.stdout:7/512: fdatasync d7/d10/d1c/f7e 0 2024-04-12T20:56:33.989 INFO:tasks.workunit.client.1.smithi186.stdout:5/478: getdents d5/db/d15/d2a/d2d/d46/d7d 0 2024-04-12T20:56:33.989 INFO:tasks.workunit.client.1.smithi186.stdout:6/449: chown d9/c20 1939 1 2024-04-12T20:56:33.989 INFO:tasks.workunit.client.1.smithi186.stdout:3/467: chown d1/d2c/c3f 1020820649 1 2024-04-12T20:56:33.991 INFO:tasks.workunit.client.1.smithi186.stdout:4/572: creat d0/dc/d88/fbd x:0 0 0 2024-04-12T20:56:33.991 INFO:tasks.workunit.client.1.smithi186.stdout:2/491: link d0/dc/c2b d0/c92 0 2024-04-12T20:56:33.991 INFO:tasks.workunit.client.1.smithi186.stdout:8/439: rmdir d7 39 2024-04-12T20:56:33.993 INFO:tasks.workunit.client.1.smithi186.stdout:3/468: write d1/f1e [125360,3711] 0 2024-04-12T20:56:33.993 INFO:tasks.workunit.client.1.smithi186.stdout:0/538: creat d0/d5c/d66/da7/fad x:0 0 0 2024-04-12T20:56:33.993 INFO:tasks.workunit.client.1.smithi186.stdout:6/450: mknod d9/d43/ca4 0 2024-04-12T20:56:33.995 INFO:tasks.workunit.client.1.smithi186.stdout:4/573: chown d0/dc/d88 2684 1 2024-04-12T20:56:33.996 INFO:tasks.workunit.client.1.smithi186.stdout:0/539: chown d0/la1 879156 1 2024-04-12T20:56:33.996 INFO:tasks.workunit.client.1.smithi186.stdout:7/513: truncate d7/d10/d2d/d40/d4a/fa0 245569 0 2024-04-12T20:56:33.998 INFO:tasks.workunit.client.1.smithi186.stdout:1/463: symlink d14/d1a/d33/d36/d5b/d3e/l92 0 2024-04-12T20:56:33.999 INFO:tasks.workunit.client.1.smithi186.stdout:6/451: write d9/d43/d2d/d4d/f64 [1231140,11445] 0 2024-04-12T20:56:34.007 INFO:tasks.workunit.client.1.smithi186.stdout:6/452: fsync d9/d43/d59/f83 0 2024-04-12T20:56:34.011 INFO:tasks.workunit.client.1.smithi186.stdout:9/428: link d3/d2f/d54/f5b d3/d2f/d2e/d33/f94 0 2024-04-12T20:56:34.014 INFO:tasks.workunit.client.1.smithi186.stdout:3/469: link d1/f8a d1/d2c/d8d/f9d 0 2024-04-12T20:56:34.020 INFO:tasks.workunit.client.1.smithi186.stdout:8/440: rename d7/d18/d36/d1d/f4d to d7/d18/d36/d20/f8b 0 2024-04-12T20:56:34.023 INFO:tasks.workunit.client.1.smithi186.stdout:9/429: fdatasync d3/d2f/d2e/d33/f8d 0 2024-04-12T20:56:34.026 INFO:tasks.workunit.client.1.smithi186.stdout:6/453: truncate d9/f37 552817 0 2024-04-12T20:56:34.031 INFO:tasks.workunit.client.1.smithi186.stdout:3/470: chown d1/d3/d49/d4d/f75 89507 1 2024-04-12T20:56:34.032 INFO:tasks.workunit.client.1.smithi186.stdout:5/479: dwrite d5/db/d71/f75 [4194304,4194304] 0 2024-04-12T20:56:34.033 INFO:tasks.workunit.client.1.smithi186.stdout:4/574: write d0/d26/f34 [236828,35052] 0 2024-04-12T20:56:34.036 INFO:tasks.workunit.client.1.smithi186.stdout:1/464: dread f4 [0,4194304] 0 2024-04-12T20:56:34.039 INFO:tasks.workunit.client.1.smithi186.stdout:8/441: read d7/de/d10/f15 [7576750,60001] 0 2024-04-12T20:56:34.040 INFO:tasks.workunit.client.1.smithi186.stdout:9/430: truncate d3/d2f/d2e/f2d 1606478 0 2024-04-12T20:56:34.043 INFO:tasks.workunit.client.1.smithi186.stdout:2/492: dwrite d0/d32/f7c [0,4194304] 0 2024-04-12T20:56:34.046 INFO:tasks.workunit.client.1.smithi186.stdout:3/471: creat d1/d3/da/f9e x:0 0 0 2024-04-12T20:56:34.046 INFO:tasks.workunit.client.1.smithi186.stdout:9/431: stat d3/d2f/l2b 0 2024-04-12T20:56:34.048 INFO:tasks.workunit.client.1.smithi186.stdout:0/540: dwrite d0/d10/d32/d3f/f54 [0,4194304] 0 2024-04-12T20:56:34.052 INFO:tasks.workunit.client.1.smithi186.stdout:2/493: readlink d0/d3/la 0 2024-04-12T20:56:34.053 INFO:tasks.workunit.client.1.smithi186.stdout:7/514: dwrite d7/d10/d1c/d8c/fb5 [0,4194304] 0 2024-04-12T20:56:34.053 INFO:tasks.workunit.client.1.smithi186.stdout:8/442: chown d7/f59 14741057 1 2024-04-12T20:56:34.054 INFO:tasks.workunit.client.1.smithi186.stdout:6/454: write d9/d43/d59/d5d/f48 [260859,62985] 0 2024-04-12T20:56:34.060 INFO:tasks.workunit.client.1.smithi186.stdout:4/575: link d0/d21/f31 d0/d26/fbe 0 2024-04-12T20:56:34.064 INFO:tasks.workunit.client.1.smithi186.stdout:3/472: mknod d1/d3/d49/d4d/c9f 0 2024-04-12T20:56:34.064 INFO:tasks.workunit.client.1.smithi186.stdout:9/432: readlink d3/d2f/d2e/l8e 0 2024-04-12T20:56:34.065 INFO:tasks.workunit.client.1.smithi186.stdout:8/443: chown d7/d18/d36/d20/d71 112516175 1 2024-04-12T20:56:34.069 INFO:tasks.workunit.client.1.smithi186.stdout:5/480: creat d5/db/d15/d2a/d2d/f9f x:0 0 0 2024-04-12T20:56:34.069 INFO:tasks.workunit.client.1.smithi186.stdout:7/515: write d7/d10/d2d/f37 [508819,87847] 0 2024-04-12T20:56:34.070 INFO:tasks.workunit.client.1.smithi186.stdout:2/494: mknod d0/dc/d2e/c93 0 2024-04-12T20:56:34.074 INFO:tasks.workunit.client.1.smithi186.stdout:5/481: write d5/db/f5f [4299859,75715] 0 2024-04-12T20:56:34.075 INFO:tasks.workunit.client.1.smithi186.stdout:0/541: mknod d0/d10/d72/d79/cae 0 2024-04-12T20:56:34.076 INFO:tasks.workunit.client.1.smithi186.stdout:2/495: truncate d0/d3/d1a/d48/f81 5135969 0 2024-04-12T20:56:34.077 INFO:tasks.workunit.client.1.smithi186.stdout:1/465: dread d14/d1a/d21/f31 [0,4194304] 0 2024-04-12T20:56:34.078 INFO:tasks.workunit.client.1.smithi186.stdout:3/473: symlink d1/d3/d21/d22/d4a/la0 0 2024-04-12T20:56:34.078 INFO:tasks.workunit.client.1.smithi186.stdout:8/444: truncate d7/d18/d36/d20/d23/f45 957005 0 2024-04-12T20:56:34.079 INFO:tasks.workunit.client.1.smithi186.stdout:9/433: mknod d3/d2f/d2e/d33/d66/d77/c95 0 2024-04-12T20:56:34.080 INFO:tasks.workunit.client.1.smithi186.stdout:0/542: write d0/d5d/f6c [1601414,20714] 0 2024-04-12T20:56:34.082 INFO:tasks.workunit.client.1.smithi186.stdout:8/445: readlink d7/de/l64 0 2024-04-12T20:56:34.083 INFO:tasks.workunit.client.1.smithi186.stdout:6/455: mknod d9/d43/d59/d5d/d6e/d91/ca5 0 2024-04-12T20:56:34.087 INFO:tasks.workunit.client.1.smithi186.stdout:2/496: write d0/d3/d1a/d48/f81 [272868,93750] 0 2024-04-12T20:56:34.089 INFO:tasks.workunit.client.1.smithi186.stdout:7/516: mknod d7/d10/d2d/d40/d4a/d5c/daa/cb9 0 2024-04-12T20:56:34.092 INFO:tasks.workunit.client.1.smithi186.stdout:6/456: creat d9/d43/d2d/fa6 x:0 0 0 2024-04-12T20:56:34.093 INFO:tasks.workunit.client.1.smithi186.stdout:2/497: write d0/d32/d2a/f71 [986088,63776] 0 2024-04-12T20:56:34.095 INFO:tasks.workunit.client.1.smithi186.stdout:5/482: link d5/db/d15/d6a/l79 d5/db/d15/d2a/d2d/la0 0 2024-04-12T20:56:34.098 INFO:tasks.workunit.client.1.smithi186.stdout:3/474: dread d1/d3/f58 [0,4194304] 0 2024-04-12T20:56:34.100 INFO:tasks.workunit.client.1.smithi186.stdout:1/466: mknod d14/d1a/d33/d36/d5b/c93 0 2024-04-12T20:56:34.102 INFO:tasks.workunit.client.1.smithi186.stdout:4/576: link d0/d59/c62 d0/cbf 0 2024-04-12T20:56:34.103 INFO:tasks.workunit.client.1.smithi186.stdout:8/446: mkdir d7/d18/d36/d20/d8c 0 2024-04-12T20:56:34.104 INFO:tasks.workunit.client.1.smithi186.stdout:3/475: write d1/f6a [414151,25514] 0 2024-04-12T20:56:34.105 INFO:tasks.workunit.client.1.smithi186.stdout:4/577: readlink d0/l2a 0 2024-04-12T20:56:34.108 INFO:tasks.workunit.client.1.smithi186.stdout:6/457: mknod d9/d5e/d73/ca7 0 2024-04-12T20:56:34.110 INFO:tasks.workunit.client.1.smithi186.stdout:1/467: read d14/d1a/d21/f31 [155778,21356] 0 2024-04-12T20:56:34.115 INFO:tasks.workunit.client.1.smithi186.stdout:6/458: chown d9/d43/d59/d5d/f48 0 1 2024-04-12T20:56:34.118 INFO:tasks.workunit.client.1.smithi186.stdout:8/447: mknod d7/d18/d36/d20/d23/c8d 0 2024-04-12T20:56:34.119 INFO:tasks.workunit.client.1.smithi186.stdout:3/476: mkdir d1/d3/d49/d4d/da1 0 2024-04-12T20:56:34.121 INFO:tasks.workunit.client.1.smithi186.stdout:3/477: chown d1/l16 873 1 2024-04-12T20:56:34.124 INFO:tasks.workunit.client.1.smithi186.stdout:1/468: rmdir d14/d1a/d33/d36 39 2024-04-12T20:56:34.125 INFO:tasks.workunit.client.1.smithi186.stdout:2/498: dwrite d0/d3/d1a/d21/d22/f6c [4194304,4194304] 0 2024-04-12T20:56:34.125 INFO:tasks.workunit.client.1.smithi186.stdout:5/483: truncate d5/db/d71/f52 41147 0 2024-04-12T20:56:34.130 INFO:tasks.workunit.client.1.smithi186.stdout:7/517: creat d7/d10/d2d/d40/d4a/d6c/d97/fba x:0 0 0 2024-04-12T20:56:34.131 INFO:tasks.workunit.client.1.smithi186.stdout:9/434: link d3/c3a d3/d2f/d2e/d33/d36/d48/c96 0 2024-04-12T20:56:34.132 INFO:tasks.workunit.client.1.smithi186.stdout:6/459: dread - d9/d5e/d7a/d81/f8d zero size 2024-04-12T20:56:34.134 INFO:tasks.workunit.client.1.smithi186.stdout:1/469: creat d14/d1a/d1d/f94 x:0 0 0 2024-04-12T20:56:34.135 INFO:tasks.workunit.client.1.smithi186.stdout:0/543: truncate d0/d10/d71/f7a 3577450 0 2024-04-12T20:56:34.136 INFO:tasks.workunit.client.1.smithi186.stdout:3/478: dread d1/d3/d49/d4d/f51 [0,4194304] 0 2024-04-12T20:56:34.138 INFO:tasks.workunit.client.1.smithi186.stdout:9/435: chown d3/d2f/d2e/d43/d4c/f89 8103583 1 2024-04-12T20:56:34.139 INFO:tasks.workunit.client.1.smithi186.stdout:5/484: creat d5/db/d15/d2a/d6d/d7f/fa1 x:0 0 0 2024-04-12T20:56:34.141 INFO:tasks.workunit.client.1.smithi186.stdout:7/518: symlink d7/lbb 0 2024-04-12T20:56:34.143 INFO:tasks.workunit.client.1.smithi186.stdout:5/485: stat d5/db/d33/l53 0 2024-04-12T20:56:34.144 INFO:tasks.workunit.client.1.smithi186.stdout:0/544: chown d0/d14/d4e/d1e/f4b 16278 1 2024-04-12T20:56:34.146 INFO:tasks.workunit.client.1.smithi186.stdout:3/479: rename d1/d2c/d8d to d1/d3/da2 0 2024-04-12T20:56:34.146 INFO:tasks.workunit.client.1.smithi186.stdout:9/436: write d3/d2f/d2e/d33/f8d [823691,54417] 0 2024-04-12T20:56:34.149 INFO:tasks.workunit.client.1.smithi186.stdout:5/486: mknod d5/db/d15/d2a/d6d/ca2 0 2024-04-12T20:56:34.150 INFO:tasks.workunit.client.1.smithi186.stdout:4/578: dwrite d0/dc/d33/f17 [0,4194304] 0 2024-04-12T20:56:34.156 INFO:tasks.workunit.client.1.smithi186.stdout:6/460: rename d9/c20 to d9/d43/d59/d5d/d6e/d91/ca8 0 2024-04-12T20:56:34.162 INFO:tasks.workunit.client.1.smithi186.stdout:0/545: creat d0/d14/d4e/d50/d6e/faf x:0 0 0 2024-04-12T20:56:34.165 INFO:tasks.workunit.client.1.smithi186.stdout:3/480: symlink d1/d3/d49/d4d/la3 0 2024-04-12T20:56:34.169 INFO:tasks.workunit.client.1.smithi186.stdout:2/499: dwrite d0/f72 [0,4194304] 0 2024-04-12T20:56:34.171 INFO:tasks.workunit.client.1.smithi186.stdout:0/546: chown d0/d10/d72/d79/cae 9227152 1 2024-04-12T20:56:34.173 INFO:tasks.workunit.client.1.smithi186.stdout:0/547: chown d0/d14/d4e 164948 1 2024-04-12T20:56:34.176 INFO:tasks.workunit.client.1.smithi186.stdout:8/448: dread d7/d18/d36/d1d/d1f/d26/f3d [0,4194304] 0 2024-04-12T20:56:34.180 INFO:tasks.workunit.client.1.smithi186.stdout:8/449: fdatasync d7/d18/d36/d1d/d1f/d26/f63 0 2024-04-12T20:56:34.182 INFO:tasks.workunit.client.1.smithi186.stdout:3/481: truncate d1/f8a 2888954 0 2024-04-12T20:56:34.185 INFO:tasks.workunit.client.1.smithi186.stdout:8/450: fsync d7/d18/d36/d20/d23/d25/d2e/d67/f6e 0 2024-04-12T20:56:34.190 INFO:tasks.workunit.client.1.smithi186.stdout:5/487: getdents d5/db/d15/d2a/d4c/d5b/d7a 0 2024-04-12T20:56:34.193 INFO:tasks.workunit.client.1.smithi186.stdout:7/519: dread d7/d10/d1c/d8c/fb5 [0,4194304] 0 2024-04-12T20:56:34.195 INFO:tasks.workunit.client.1.smithi186.stdout:8/451: chown d7/d18/d36/f1b 16848 1 2024-04-12T20:56:34.199 INFO:tasks.workunit.client.1.smithi186.stdout:1/470: dread d14/d1a/d1d/f6f [0,4194304] 0 2024-04-12T20:56:34.199 INFO:tasks.workunit.client.1.smithi186.stdout:0/548: link d0/d10/d32/c38 d0/d5c/cb0 0 2024-04-12T20:56:34.206 INFO:tasks.workunit.client.1.smithi186.stdout:2/500: dwrite d0/dc/f55 [4194304,4194304] 0 2024-04-12T20:56:34.208 INFO:tasks.workunit.client.1.smithi186.stdout:3/482: mkdir d1/d3/d21/d22/da4 0 2024-04-12T20:56:34.209 INFO:tasks.workunit.client.1.smithi186.stdout:8/452: truncate d7/d18/d36/fa 5406645 0 2024-04-12T20:56:34.210 INFO:tasks.workunit.client.1.smithi186.stdout:5/488: creat d5/db/d15/d2a/d4c/d5b/d7a/d94/fa3 x:0 0 0 2024-04-12T20:56:34.213 INFO:tasks.workunit.client.1.smithi186.stdout:0/549: truncate d0/d10/d32/f9e 523862 0 2024-04-12T20:56:34.214 INFO:tasks.workunit.client.1.smithi186.stdout:9/437: dwrite d3/d13/f4e [0,4194304] 0 2024-04-12T20:56:34.218 INFO:tasks.workunit.client.1.smithi186.stdout:3/483: chown d1/d2c/d48/f5a 8250602 1 2024-04-12T20:56:34.220 INFO:tasks.workunit.client.1.smithi186.stdout:1/471: read d14/f15 [1702960,56044] 0 2024-04-12T20:56:34.220 INFO:tasks.workunit.client.1.smithi186.stdout:8/453: rename d7/d18/d53 to d7/d18/d8e 0 2024-04-12T20:56:34.228 INFO:tasks.workunit.client.1.smithi186.stdout:4/579: dwrite d0/f42 [4194304,4194304] 0 2024-04-12T20:56:34.229 INFO:tasks.workunit.client.1.smithi186.stdout:2/501: rename d0/d32/d2a/f43 to d0/d3/d7a/f94 0 2024-04-12T20:56:34.230 INFO:tasks.workunit.client.1.smithi186.stdout:0/550: creat d0/d10/d71/fb1 x:0 0 0 2024-04-12T20:56:34.231 INFO:tasks.workunit.client.1.smithi186.stdout:2/502: chown d0/d3/d7a 344 1 2024-04-12T20:56:34.231 INFO:tasks.workunit.client.1.smithi186.stdout:5/489: mkdir d5/db/d15/d2a/d2d/d48/da4 0 2024-04-12T20:56:34.232 INFO:tasks.workunit.client.1.smithi186.stdout:6/461: dwrite d9/f37 [0,4194304] 0 2024-04-12T20:56:34.235 INFO:tasks.workunit.client.1.smithi186.stdout:8/454: write d7/d18/d36/d1d/d1f/f49 [3593149,17190] 0 2024-04-12T20:56:34.238 INFO:tasks.workunit.client.1.smithi186.stdout:5/490: fdatasync d5/db/d15/f2f 0 2024-04-12T20:56:34.238 INFO:tasks.workunit.client.1.smithi186.stdout:4/580: write d0/dc/d33/f17 [1963568,8227] 0 2024-04-12T20:56:34.239 INFO:tasks.workunit.client.1.smithi186.stdout:6/462: fdatasync d9/d43/d50/f54 0 2024-04-12T20:56:34.241 INFO:tasks.workunit.client.1.smithi186.stdout:2/503: symlink d0/d32/d8b/l95 0 2024-04-12T20:56:34.242 INFO:tasks.workunit.client.1.smithi186.stdout:8/455: stat d7/d18/d36/d1d 0 2024-04-12T20:56:34.243 INFO:tasks.workunit.client.1.smithi186.stdout:0/551: write d0/d14/d4e/d1e/fa9 [685135,113540] 0 2024-04-12T20:56:34.246 INFO:tasks.workunit.client.1.smithi186.stdout:4/581: dread - d0/dc/fa2 zero size 2024-04-12T20:56:34.248 INFO:tasks.workunit.client.1.smithi186.stdout:8/456: readlink d7/d18/l30 0 2024-04-12T20:56:34.250 INFO:tasks.workunit.client.1.smithi186.stdout:5/491: chown d5/db/d15/d2a/d6d/d7f 2032 1 2024-04-12T20:56:34.250 INFO:tasks.workunit.client.1.smithi186.stdout:2/504: write d0/dc/d2e/f5f [63845,84330] 0 2024-04-12T20:56:34.251 INFO:tasks.workunit.client.1.smithi186.stdout:6/463: dread d9/f4b [0,4194304] 0 2024-04-12T20:56:34.252 INFO:tasks.workunit.client.1.smithi186.stdout:9/438: getdents d3/d2f/d2e/d33/d36/d70 0 2024-04-12T20:56:34.255 INFO:tasks.workunit.client.1.smithi186.stdout:7/520: dread d7/d10/d2d/d40/d4a/fa6 [0,4194304] 0 2024-04-12T20:56:34.256 INFO:tasks.workunit.client.1.smithi186.stdout:1/472: mkdir d14/d1a/d33/d36/d5b/d3e/d71/d8c/d95 0 2024-04-12T20:56:34.256 INFO:tasks.workunit.client.1.smithi186.stdout:0/552: chown d0/d14/d4e/f33 4589 1 2024-04-12T20:56:34.258 INFO:tasks.workunit.client.1.smithi186.stdout:2/505: mknod d0/d3/c96 0 2024-04-12T20:56:34.258 INFO:tasks.workunit.client.1.smithi186.stdout:9/439: rmdir d3/d12/d6f/d83 39 2024-04-12T20:56:34.261 INFO:tasks.workunit.client.1.smithi186.stdout:5/492: symlink d5/db/d2e/d70/la5 0 2024-04-12T20:56:34.262 INFO:tasks.workunit.client.1.smithi186.stdout:9/440: chown d3/d2f/f1e 239284 1 2024-04-12T20:56:34.262 INFO:tasks.workunit.client.1.smithi186.stdout:3/484: dread d1/d3/da/f56 [0,4194304] 0 2024-04-12T20:56:34.267 INFO:tasks.workunit.client.1.smithi186.stdout:2/506: write d0/d3/d1a/d21/d26/f2d [652972,118599] 0 2024-04-12T20:56:34.268 INFO:tasks.workunit.client.1.smithi186.stdout:3/485: fdatasync d1/d3/d49/f59 0 2024-04-12T20:56:34.270 INFO:tasks.workunit.client.1.smithi186.stdout:6/464: dread d9/d5e/f71 [0,4194304] 0 2024-04-12T20:56:34.273 INFO:tasks.workunit.client.1.smithi186.stdout:4/582: getdents d0/d21/d3d/d63/da6 0 2024-04-12T20:56:34.273 INFO:tasks.workunit.client.1.smithi186.stdout:2/507: creat d0/d32/f97 x:0 0 0 2024-04-12T20:56:34.275 INFO:tasks.workunit.client.1.smithi186.stdout:8/457: getdents d7/de/d10/d7f 0 2024-04-12T20:56:34.276 INFO:tasks.workunit.client.1.smithi186.stdout:5/493: rename d5/db/d15/d2a/d2d/d46/c66 to d5/db/d15/d2a/d4c/d5b/d7a/d94/ca6 0 2024-04-12T20:56:34.280 INFO:tasks.workunit.client.1.smithi186.stdout:6/465: mkdir d9/da9 0 2024-04-12T20:56:34.281 INFO:tasks.workunit.client.1.smithi186.stdout:2/508: fsync d0/d3/d1a/d21/d22/f6c 0 2024-04-12T20:56:34.283 INFO:tasks.workunit.client.1.smithi186.stdout:8/458: mknod d7/de/d10/c8f 0 2024-04-12T20:56:34.284 INFO:tasks.workunit.client.1.smithi186.stdout:0/553: write d0/d14/d4e/d1e/f1f [3279096,63882] 0 2024-04-12T20:56:34.284 INFO:tasks.workunit.client.1.smithi186.stdout:1/473: truncate d14/d1a/f1b 76866 0 2024-04-12T20:56:34.285 INFO:tasks.workunit.client.1.smithi186.stdout:9/441: write d3/f1d [3212122,73963] 0 2024-04-12T20:56:34.290 INFO:tasks.workunit.client.1.smithi186.stdout:2/509: truncate d0/d32/f97 925798 0 2024-04-12T20:56:34.292 INFO:tasks.workunit.client.1.smithi186.stdout:5/494: creat d5/db/d15/d2a/d2d/d48/da4/fa7 x:0 0 0 2024-04-12T20:56:34.293 INFO:tasks.workunit.client.1.smithi186.stdout:9/442: chown d3/d12/c7e 201464 1 2024-04-12T20:56:34.293 INFO:tasks.workunit.client.1.smithi186.stdout:8/459: fsync d7/d18/f1a 0 2024-04-12T20:56:34.295 INFO:tasks.workunit.client.1.smithi186.stdout:1/474: symlink d14/d91/l96 0 2024-04-12T20:56:34.296 INFO:tasks.workunit.client.1.smithi186.stdout:8/460: chown d7/de/d5e 12489 1 2024-04-12T20:56:34.300 INFO:tasks.workunit.client.1.smithi186.stdout:5/495: stat d5/db/d15/d2a/d6d/d7f/d86/c9b 0 2024-04-12T20:56:34.301 INFO:tasks.workunit.client.1.smithi186.stdout:8/461: dread - d7/d18/d36/d1d/d1f/d26/f63 zero size 2024-04-12T20:56:34.301 INFO:tasks.workunit.client.1.smithi186.stdout:6/466: dread d9/d43/d50/f54 [0,4194304] 0 2024-04-12T20:56:34.303 INFO:tasks.workunit.client.1.smithi186.stdout:1/475: rename ff to d14/d1a/d58/f97 0 2024-04-12T20:56:34.303 INFO:tasks.workunit.client.1.smithi186.stdout:3/486: dwrite d1/d3/d21/d22/d24/d2f/f7f [0,4194304] 0 2024-04-12T20:56:34.308 INFO:tasks.workunit.client.1.smithi186.stdout:9/443: creat d3/d2f/d54/d92/f97 x:0 0 0 2024-04-12T20:56:34.309 INFO:tasks.workunit.client.1.smithi186.stdout:9/444: stat d3/d2f/d2e/d33/d66/d77/f8b 0 2024-04-12T20:56:34.311 INFO:tasks.workunit.client.1.smithi186.stdout:4/583: dread d0/f24 [0,4194304] 0 2024-04-12T20:56:34.312 INFO:tasks.workunit.client.1.smithi186.stdout:3/487: write d1/d3/d49/d4d/d6f/f99 [256730,127177] 0 2024-04-12T20:56:34.314 INFO:tasks.workunit.client.1.smithi186.stdout:6/467: creat d9/d4c/d6f/d89/faa x:0 0 0 2024-04-12T20:56:34.318 INFO:tasks.workunit.client.1.smithi186.stdout:5/496: link d5/d44/c50 d5/db/d15/d2a/d4c/d5b/d7a/ca8 0 2024-04-12T20:56:34.319 INFO:tasks.workunit.client.1.smithi186.stdout:8/462: dread d7/d18/d36/d20/d23/f45 [0,4194304] 0 2024-04-12T20:56:34.319 INFO:tasks.workunit.client.1.smithi186.stdout:0/554: dwrite d0/d10/d32/f3d [0,4194304] 0 2024-04-12T20:56:34.321 INFO:tasks.workunit.client.1.smithi186.stdout:6/468: fdatasync d9/d43/d59/d5d/f48 0 2024-04-12T20:56:34.321 INFO:tasks.workunit.client.1.smithi186.stdout:3/488: write d1/d3/da/f6c [919733,26701] 0 2024-04-12T20:56:34.325 INFO:tasks.workunit.client.1.smithi186.stdout:4/584: read d0/dc/d64/d87/d98/fa7 [2581145,25679] 0 2024-04-12T20:56:34.326 INFO:tasks.workunit.client.1.smithi186.stdout:9/445: write d3/d2f/d2e/d33/d36/f3d [4609508,96695] 0 2024-04-12T20:56:34.330 INFO:tasks.workunit.client.1.smithi186.stdout:0/555: truncate d0/d10/d71/fb1 8531 0 2024-04-12T20:56:34.333 INFO:tasks.workunit.client.1.smithi186.stdout:2/510: dwrite d0/dc/f37 [0,4194304] 0 2024-04-12T20:56:34.339 INFO:tasks.workunit.client.1.smithi186.stdout:9/446: mknod d3/d2f/d2e/d33/d36/c98 0 2024-04-12T20:56:34.343 INFO:tasks.workunit.client.1.smithi186.stdout:0/556: symlink d0/d5c/d66/da7/lb2 0 2024-04-12T20:56:34.346 INFO:tasks.workunit.client.1.smithi186.stdout:9/447: unlink d3/d13/f28 0 2024-04-12T20:56:34.351 INFO:tasks.workunit.client.1.smithi186.stdout:2/511: chown d0/dc/c2b 3912 1 2024-04-12T20:56:34.354 INFO:tasks.workunit.client.1.smithi186.stdout:0/557: mknod d0/d14/d4e/d50/cb3 0 2024-04-12T20:56:34.355 INFO:tasks.workunit.client.1.smithi186.stdout:2/512: readlink d0/d3/d1a/d41/l4a 0 2024-04-12T20:56:34.358 INFO:tasks.workunit.client.1.smithi186.stdout:9/448: creat d3/d2f/d2e/d33/d36/d70/f99 x:0 0 0 2024-04-12T20:56:34.361 INFO:tasks.workunit.client.1.smithi186.stdout:8/463: dwrite d7/de/f42 [0,4194304] 0 2024-04-12T20:56:34.366 INFO:tasks.workunit.client.1.smithi186.stdout:1/476: dread d14/d1a/f3f [0,4194304] 0 2024-04-12T20:56:34.369 INFO:tasks.workunit.client.1.smithi186.stdout:0/558: rename d0/d14/d4e/d50/d6e/d96/faa to d0/d5c/d66/da7/fb4 0 2024-04-12T20:56:34.371 INFO:tasks.workunit.client.1.smithi186.stdout:4/585: dwrite d0/d21/f32 [0,4194304] 0 2024-04-12T20:56:34.372 INFO:tasks.workunit.client.1.smithi186.stdout:2/513: getdents d0/d32/d8b 0 2024-04-12T20:56:34.376 INFO:tasks.workunit.client.1.smithi186.stdout:1/477: symlink d14/d1a/d33/d36/d5b/d3e/d71/d8c/l98 0 2024-04-12T20:56:34.377 INFO:tasks.workunit.client.1.smithi186.stdout:2/514: fsync d0/d32/f6f 0 2024-04-12T20:56:34.379 INFO:tasks.workunit.client.1.smithi186.stdout:0/559: read d0/d14/d4e/d1e/fa9 [223076,69947] 0 2024-04-12T20:56:34.385 INFO:tasks.workunit.client.1.smithi186.stdout:8/464: rename d7/d18/d36/f1b to d7/f90 0 2024-04-12T20:56:34.386 INFO:tasks.workunit.client.1.smithi186.stdout:3/489: dread d1/d3/d21/d22/f90 [0,4194304] 0 2024-04-12T20:56:34.387 INFO:tasks.workunit.client.1.smithi186.stdout:0/560: write d0/d5c/d66/da7/fad [734791,50015] 0 2024-04-12T20:56:34.388 INFO:tasks.workunit.client.1.smithi186.stdout:2/515: write d0/f50 [1445881,64381] 0 2024-04-12T20:56:34.390 INFO:tasks.workunit.client.1.smithi186.stdout:6/469: dwrite d9/d43/f44 [0,4194304] 0 2024-04-12T20:56:34.390 INFO:tasks.workunit.client.1.smithi186.stdout:3/490: fsync d1/d3/d21/d22/d34/f98 0 2024-04-12T20:56:34.392 INFO:tasks.workunit.client.1.smithi186.stdout:4/586: rename d0/d21/d3d to d0/d59/dc0 0 2024-04-12T20:56:34.395 INFO:tasks.workunit.client.1.smithi186.stdout:3/491: write d1/d3/d21/d5c/f62 [3868407,127030] 0 2024-04-12T20:56:34.398 INFO:tasks.workunit.client.1.smithi186.stdout:2/516: unlink d0/dc/f89 0 2024-04-12T20:56:34.399 INFO:tasks.workunit.client.1.smithi186.stdout:9/449: write d3/fd [2327411,13063] 0 2024-04-12T20:56:34.403 INFO:tasks.workunit.client.1.smithi186.stdout:0/561: rename d0/la1 to d0/d10/d32/d3f/lb5 0 2024-04-12T20:56:34.403 INFO:tasks.workunit.client.1.smithi186.stdout:5/497: dread d5/db/d71/f75 [0,4194304] 0 2024-04-12T20:56:34.408 INFO:tasks.workunit.client.1.smithi186.stdout:9/450: mknod d3/d2f/d2e/d33/d36/d48/c9a 0 2024-04-12T20:56:34.409 INFO:tasks.workunit.client.1.smithi186.stdout:3/492: truncate d1/f8a 539977 0 2024-04-12T20:56:34.412 INFO:tasks.workunit.client.1.smithi186.stdout:9/451: chown d3/d2f/f6d 97919360 1 2024-04-12T20:56:34.412 INFO:tasks.workunit.client.1.smithi186.stdout:6/470: creat d9/d43/d59/fab x:0 0 0 2024-04-12T20:56:34.415 INFO:tasks.workunit.client.1.smithi186.stdout:1/478: dwrite d14/d1a/d33/d36/d5b/d3e/d5f/f8f [0,4194304] 0 2024-04-12T20:56:34.415 INFO:tasks.workunit.client.1.smithi186.stdout:0/562: creat d0/d10/d32/dac/fb6 x:0 0 0 2024-04-12T20:56:34.417 INFO:tasks.workunit.client.1.smithi186.stdout:3/493: creat d1/d2c/d54/d81/fa5 x:0 0 0 2024-04-12T20:56:34.417 INFO:tasks.workunit.client.1.smithi186.stdout:2/517: truncate d0/d3/d1a/d21/d22/f6c 3852246 0 2024-04-12T20:56:34.419 INFO:tasks.workunit.client.1.smithi186.stdout:0/563: readlink d0/d5c/d66/l6f 0 2024-04-12T20:56:34.420 INFO:tasks.workunit.client.1.smithi186.stdout:9/452: rmdir d3/d13 39 2024-04-12T20:56:34.420 INFO:tasks.workunit.client.1.smithi186.stdout:1/479: fdatasync d14/d1a/d21/f57 0 2024-04-12T20:56:34.421 INFO:tasks.workunit.client.1.smithi186.stdout:2/518: symlink d0/d32/l98 0 2024-04-12T20:56:34.422 INFO:tasks.workunit.client.1.smithi186.stdout:1/480: chown d14/d1a/d21/l2b 4026 1 2024-04-12T20:56:34.423 INFO:tasks.workunit.client.1.smithi186.stdout:6/471: creat d9/d43/d59/d5d/d6e/d91/fac x:0 0 0 2024-04-12T20:56:34.426 INFO:tasks.workunit.client.1.smithi186.stdout:0/564: write d0/d10/f2a [1300336,93477] 0 2024-04-12T20:56:34.428 INFO:tasks.workunit.client.1.smithi186.stdout:2/519: rename d0/d3/d1a/d41/l61 to d0/d32/d2a/l99 0 2024-04-12T20:56:34.429 INFO:tasks.workunit.client.1.smithi186.stdout:0/565: truncate d0/d10/d72/f7d 108096 0 2024-04-12T20:56:34.430 INFO:tasks.workunit.client.1.smithi186.stdout:9/453: symlink d3/d2f/d2e/d43/d4c/d82/l9b 0 2024-04-12T20:56:34.431 INFO:tasks.workunit.client.1.smithi186.stdout:2/520: dread - d0/d3/d1a/d21/d26/f88 zero size 2024-04-12T20:56:34.433 INFO:tasks.workunit.client.1.smithi186.stdout:0/566: symlink d0/d10/lb7 0 2024-04-12T20:56:34.434 INFO:tasks.workunit.client.1.smithi186.stdout:9/454: write d3/d2f/f68 [605976,16925] 0 2024-04-12T20:56:34.434 INFO:tasks.workunit.client.1.smithi186.stdout:1/481: link d14/f77 d14/d1a/d33/d36/d5b/d6a/f99 0 2024-04-12T20:56:34.441 INFO:tasks.workunit.client.1.smithi186.stdout:2/521: creat d0/d3/d1a/d21/d26/f9a x:0 0 0 2024-04-12T20:56:34.441 INFO:tasks.workunit.client.1.smithi186.stdout:4/587: dwrite d0/dc/d88/d80/fa5 [0,4194304] 0 2024-04-12T20:56:34.449 INFO:tasks.workunit.client.1.smithi186.stdout:0/567: creat d0/fb8 x:0 0 0 2024-04-12T20:56:34.456 INFO:tasks.workunit.client.1.smithi186.stdout:6/472: dwrite d9/d43/d50/f5b [0,4194304] 0 2024-04-12T20:56:34.462 INFO:tasks.workunit.client.1.smithi186.stdout:6/473: read - d9/d5e/f9d zero size 2024-04-12T20:56:34.464 INFO:tasks.workunit.client.1.smithi186.stdout:9/455: truncate d3/fd 3411477 0 2024-04-12T20:56:34.465 INFO:tasks.workunit.client.1.smithi186.stdout:6/474: dread - d9/d4c/d6f/f9a zero size 2024-04-12T20:56:34.468 INFO:tasks.workunit.client.1.smithi186.stdout:6/475: dread d9/f4b [0,4194304] 0 2024-04-12T20:56:34.468 INFO:tasks.workunit.client.1.smithi186.stdout:9/456: dread - d3/d12/d6f/f75 zero size 2024-04-12T20:56:34.472 INFO:tasks.workunit.client.1.smithi186.stdout:2/522: dwrite d0/d3/f80 [0,4194304] 0 2024-04-12T20:56:34.478 INFO:tasks.workunit.client.1.smithi186.stdout:6/476: creat d9/d43/d50/fad x:0 0 0 2024-04-12T20:56:34.478 INFO:tasks.workunit.client.1.smithi186.stdout:2/523: fdatasync d0/dc/f55 0 2024-04-12T20:56:34.479 INFO:tasks.workunit.client.1.smithi186.stdout:9/457: chown d3/d13/c23 3658693 1 2024-04-12T20:56:34.480 INFO:tasks.workunit.client.1.smithi186.stdout:0/568: dwrite d0/f4f [0,4194304] 0 2024-04-12T20:56:34.484 INFO:tasks.workunit.client.1.smithi186.stdout:0/569: stat d0/d10/d32/d3f/f54 0 2024-04-12T20:56:34.486 INFO:tasks.workunit.client.1.smithi186.stdout:9/458: readlink d3/d2f/d2e/l8e 0 2024-04-12T20:56:34.487 INFO:tasks.workunit.client.1.smithi186.stdout:2/524: write d0/d3/d1a/d48/f81 [1683836,61349] 0 2024-04-12T20:56:34.488 INFO:tasks.workunit.client.1.smithi186.stdout:4/588: dread d0/dc/d33/f47 [0,4194304] 0 2024-04-12T20:56:34.488 INFO:tasks.workunit.client.1.smithi186.stdout:1/482: dread d14/d1a/d1d/f29 [0,4194304] 0 2024-04-12T20:56:34.491 INFO:tasks.workunit.client.1.smithi186.stdout:2/525: dread - d0/f7e zero size 2024-04-12T20:56:34.491 INFO:tasks.workunit.client.1.smithi186.stdout:6/477: rmdir d9/d5e/d7a/d81/d96/d98 0 2024-04-12T20:56:34.491 INFO:tasks.workunit.client.1.smithi186.stdout:7/521: sync 2024-04-12T20:56:34.491 INFO:tasks.workunit.client.1.smithi186.stdout:1/483: read f9 [6065809,51103] 0 2024-04-12T20:56:34.498 INFO:tasks.workunit.client.1.smithi186.stdout:4/589: truncate d0/dc/f3b 2201081 0 2024-04-12T20:56:34.500 INFO:tasks.workunit.client.1.smithi186.stdout:1/484: truncate d14/d1a/d33/d36/d5b/d3e/d71/f82 8684761 0 2024-04-12T20:56:34.502 INFO:tasks.workunit.client.1.smithi186.stdout:4/590: symlink d0/dc/lc1 0 2024-04-12T20:56:34.505 INFO:tasks.workunit.client.1.smithi186.stdout:1/485: creat d14/d74/f9a x:0 0 0 2024-04-12T20:56:34.509 INFO:tasks.workunit.client.1.smithi186.stdout:9/459: dread d3/d13/f4e [0,4194304] 0 2024-04-12T20:56:34.605 INFO:tasks.workunit.client.1.smithi186.stdout:6/478: dwrite d9/d5e/d7a/d81/f8d [0,4194304] 0 2024-04-12T20:56:34.605 INFO:tasks.workunit.client.1.smithi186.stdout:2/526: dwrite d0/d3/d1a/d41/f44 [0,4194304] 0 2024-04-12T20:56:34.610 INFO:tasks.workunit.client.1.smithi186.stdout:2/527: chown d0/d3/d1a 26 1 2024-04-12T20:56:34.615 INFO:tasks.workunit.client.1.smithi186.stdout:6/479: symlink d9/d4c/d6f/d97/lae 0 2024-04-12T20:56:34.615 INFO:tasks.workunit.client.1.smithi186.stdout:2/528: creat d0/d32/f9b x:0 0 0 2024-04-12T20:56:34.622 INFO:tasks.workunit.client.1.smithi186.stdout:4/591: dwrite d0/dc/d33/f43 [4194304,4194304] 0 2024-04-12T20:56:34.628 INFO:tasks.workunit.client.1.smithi186.stdout:2/529: symlink d0/d3/d1a/d24/l9c 0 2024-04-12T20:56:34.632 INFO:tasks.workunit.client.1.smithi186.stdout:9/460: dwrite d3/d2f/d2e/d33/d66/f5f [4194304,4194304] 0 2024-04-12T20:56:34.647 INFO:tasks.workunit.client.1.smithi186.stdout:2/530: dwrite d0/f50 [4194304,4194304] 0 2024-04-12T20:56:34.653 INFO:tasks.workunit.client.1.smithi186.stdout:6/480: getdents d9/d4c/d6f/d89/d9c 0 2024-04-12T20:56:34.654 INFO:tasks.workunit.client.1.smithi186.stdout:2/531: dread - d0/d3/d1a/d24/f67 zero size 2024-04-12T20:56:34.662 INFO:tasks.workunit.client.1.smithi186.stdout:2/532: chown d0/d3/d1a/d21/d22/c28 935529425 1 2024-04-12T20:56:34.669 INFO:tasks.workunit.client.1.smithi186.stdout:6/481: dread d9/d43/f44 [0,4194304] 0 2024-04-12T20:56:34.673 INFO:tasks.workunit.client.1.smithi186.stdout:2/533: dread d0/d3/d1a/d41/f44 [0,4194304] 0 2024-04-12T20:56:34.678 INFO:tasks.workunit.client.1.smithi186.stdout:5/498: sync 2024-04-12T20:56:34.679 INFO:tasks.workunit.client.1.smithi186.stdout:8/465: sync 2024-04-12T20:56:34.679 INFO:tasks.workunit.client.1.smithi186.stdout:3/494: sync 2024-04-12T20:56:34.681 INFO:tasks.workunit.client.1.smithi186.stdout:3/495: chown d1/l3a 411740897 1 2024-04-12T20:56:34.682 INFO:tasks.workunit.client.1.smithi186.stdout:5/499: write d5/db/d15/d2a/d2d/d46/f6c [410235,91920] 0 2024-04-12T20:56:34.682 INFO:tasks.workunit.client.1.smithi186.stdout:8/466: chown d7/d18/d36/d20/d23/d25/f56 1636477 1 2024-04-12T20:56:34.686 INFO:tasks.workunit.client.1.smithi186.stdout:2/534: dread d0/f50 [4194304,4194304] 0 2024-04-12T20:56:34.689 INFO:tasks.workunit.client.1.smithi186.stdout:8/467: creat d7/d18/d36/d20/d23/d25/d2e/d5c/f91 x:0 0 0 2024-04-12T20:56:34.689 INFO:tasks.workunit.client.1.smithi186.stdout:4/592: write d0/d21/f40 [815899,33593] 0 2024-04-12T20:56:34.690 INFO:tasks.workunit.client.1.smithi186.stdout:6/482: dread d9/f10 [0,4194304] 0 2024-04-12T20:56:34.691 INFO:tasks.workunit.client.1.smithi186.stdout:5/500: link d5/db/d15/d2a/d2d/f58 d5/d44/fa9 0 2024-04-12T20:56:34.695 INFO:tasks.workunit.client.1.smithi186.stdout:8/468: write d7/d18/f4c [2882921,5532] 0 2024-04-12T20:56:34.695 INFO:tasks.workunit.client.1.smithi186.stdout:2/535: getdents d0/dc/d2e/d73 0 2024-04-12T20:56:34.696 INFO:tasks.workunit.client.1.smithi186.stdout:6/483: symlink d9/d43/d59/d5d/d6e/laf 0 2024-04-12T20:56:34.700 INFO:tasks.workunit.client.1.smithi186.stdout:6/484: fdatasync d9/d43/d2d/d4d/f64 0 2024-04-12T20:56:34.701 INFO:tasks.workunit.client.1.smithi186.stdout:8/469: creat d7/d18/d36/d20/d8c/f92 x:0 0 0 2024-04-12T20:56:34.704 INFO:tasks.workunit.client.1.smithi186.stdout:6/485: creat d9/d5e/fb0 x:0 0 0 2024-04-12T20:56:34.705 INFO:tasks.workunit.client.1.smithi186.stdout:5/501: truncate d5/db/f11 1823000 0 2024-04-12T20:56:34.705 INFO:tasks.workunit.client.1.smithi186.stdout:9/461: dwrite d3/d2f/d2e/d33/f94 [0,4194304] 0 2024-04-12T20:56:34.708 INFO:tasks.workunit.client.1.smithi186.stdout:5/502: chown d5/db/d15/d2a/d4c/d93 0 1 2024-04-12T20:56:34.711 INFO:tasks.workunit.client.1.smithi186.stdout:5/503: mknod d5/db/d33/caa 0 2024-04-12T20:56:34.711 INFO:tasks.workunit.client.1.smithi186.stdout:6/486: chown d9/d43/d50/c69 5063 1 2024-04-12T20:56:34.713 INFO:tasks.workunit.client.1.smithi186.stdout:9/462: unlink d3/c47 0 2024-04-12T20:56:34.713 INFO:tasks.workunit.client.1.smithi186.stdout:5/504: fdatasync d5/db/d15/f1e 0 2024-04-12T20:56:34.715 INFO:tasks.workunit.client.1.smithi186.stdout:2/536: dwrite d0/d3/d1a/d24/f33 [0,4194304] 0 2024-04-12T20:56:34.716 INFO:tasks.workunit.client.1.smithi186.stdout:4/593: dread d0/d21/f6a [0,4194304] 0 2024-04-12T20:56:34.717 INFO:tasks.workunit.client.1.smithi186.stdout:8/470: rename d7/d18/d36/d1d/d1f/c4b to d7/d18/d36/d1d/c93 0 2024-04-12T20:56:34.718 INFO:tasks.workunit.client.1.smithi186.stdout:6/487: symlink d9/d43/d59/d5d/d6e/lb1 0 2024-04-12T20:56:34.724 INFO:tasks.workunit.client.1.smithi186.stdout:5/505: mknod d5/db/d15/d2a/d65/cab 0 2024-04-12T20:56:34.725 INFO:tasks.workunit.client.1.smithi186.stdout:9/463: write d3/d13/f4e [3475746,121218] 0 2024-04-12T20:56:34.727 INFO:tasks.workunit.client.1.smithi186.stdout:4/594: rename d0/dc/d33/f56 to d0/dc/d1f/d73/fc2 0 2024-04-12T20:56:34.727 INFO:tasks.workunit.client.1.smithi186.stdout:5/506: stat d5/db/c69 0 2024-04-12T20:56:34.728 INFO:tasks.workunit.client.1.smithi186.stdout:8/471: readlink d7/d18/d36/d20/d23/d25/d2e/d5c/l84 0 2024-04-12T20:56:34.729 INFO:tasks.workunit.client.1.smithi186.stdout:2/537: creat d0/d3/d1a/f9d x:0 0 0 2024-04-12T20:56:34.733 INFO:tasks.workunit.client.1.smithi186.stdout:8/472: creat d7/d18/d36/d20/d23/d25/d2e/f94 x:0 0 0 2024-04-12T20:56:34.734 INFO:tasks.workunit.client.1.smithi186.stdout:5/507: truncate f2 7324491 0 2024-04-12T20:56:34.736 INFO:tasks.workunit.client.1.smithi186.stdout:4/595: mknod d0/d59/dc0/d48/cc3 0 2024-04-12T20:56:34.737 INFO:tasks.workunit.client.1.smithi186.stdout:6/488: dread d9/d43/d2d/d4d/f64 [0,4194304] 0 2024-04-12T20:56:34.738 INFO:tasks.workunit.client.1.smithi186.stdout:4/596: readlink d0/l1 0 2024-04-12T20:56:34.739 INFO:tasks.workunit.client.1.smithi186.stdout:5/508: chown d5/d44/f67 1784907152 1 2024-04-12T20:56:34.744 INFO:tasks.workunit.client.1.smithi186.stdout:5/509: truncate d5/d44/f67 4467933 0 2024-04-12T20:56:34.745 INFO:tasks.workunit.client.1.smithi186.stdout:2/538: getdents d0/d3/d1a/d21/d22 0 2024-04-12T20:56:34.747 INFO:tasks.workunit.client.1.smithi186.stdout:4/597: getdents d0/d59/dc0/d63/da6/daf 0 2024-04-12T20:56:34.748 INFO:tasks.workunit.client.1.smithi186.stdout:9/464: dwrite d3/d2f/d54/f5b [0,4194304] 0 2024-04-12T20:56:34.750 INFO:tasks.workunit.client.1.smithi186.stdout:4/598: readlink d0/d59/dc0/d63/d9b/lb5 0 2024-04-12T20:56:34.750 INFO:tasks.workunit.client.1.smithi186.stdout:0/570: sync 2024-04-12T20:56:34.750 INFO:tasks.workunit.client.1.smithi186.stdout:1/486: sync 2024-04-12T20:56:34.751 INFO:tasks.workunit.client.1.smithi186.stdout:7/522: sync 2024-04-12T20:56:34.756 INFO:tasks.workunit.client.1.smithi186.stdout:1/487: write d14/d1a/d33/d36/d5b/d3e/f7e [2722653,125933] 0 2024-04-12T20:56:34.757 INFO:tasks.workunit.client.1.smithi186.stdout:1/488: stat f4 0 2024-04-12T20:56:34.762 INFO:tasks.workunit.client.1.smithi186.stdout:0/571: mknod d0/d14/d4e/d1e/cb9 0 2024-04-12T20:56:34.762 INFO:tasks.workunit.client.1.smithi186.stdout:7/523: creat d7/d10/d2d/d40/d4a/d6c/fbc x:0 0 0 2024-04-12T20:56:34.765 INFO:tasks.workunit.client.1.smithi186.stdout:0/572: unlink d0/d10/d32/f9e 0 2024-04-12T20:56:34.771 INFO:tasks.workunit.client.1.smithi186.stdout:7/524: symlink d7/d10/d2d/d40/d4a/d5c/daa/lbd 0 2024-04-12T20:56:34.775 INFO:tasks.workunit.client.1.smithi186.stdout:1/489: getdents d14/d91 0 2024-04-12T20:56:34.776 INFO:tasks.workunit.client.1.smithi186.stdout:4/599: dwrite d0/dc/d88/f6c [0,4194304] 0 2024-04-12T20:56:34.777 INFO:tasks.workunit.client.1.smithi186.stdout:7/525: fsync d7/d10/f13 0 2024-04-12T20:56:34.777 INFO:tasks.workunit.client.1.smithi186.stdout:6/489: dread d9/d4c/f61 [0,4194304] 0 2024-04-12T20:56:34.779 INFO:tasks.workunit.client.1.smithi186.stdout:4/600: fsync d0/dc/d39/f79 0 2024-04-12T20:56:34.783 INFO:tasks.workunit.client.1.smithi186.stdout:5/510: dwrite d5/db/d71/f52 [0,4194304] 0 2024-04-12T20:56:34.783 INFO:tasks.workunit.client.1.smithi186.stdout:1/490: unlink d14/d1a/c28 0 2024-04-12T20:56:34.789 INFO:tasks.workunit.client.1.smithi186.stdout:6/490: creat d9/d4c/d6f/fb2 x:0 0 0 2024-04-12T20:56:34.790 INFO:tasks.workunit.client.1.smithi186.stdout:7/526: rename d7/d10/d2d/d40/d4a/d5c/daa to d7/d55/d71/dbe 0 2024-04-12T20:56:34.792 INFO:tasks.workunit.client.1.smithi186.stdout:1/491: read d14/f7d [432064,75288] 0 2024-04-12T20:56:34.794 INFO:tasks.workunit.client.1.smithi186.stdout:8/473: dread d7/d18/d36/d20/d23/d25/d29/f2a [0,4194304] 0 2024-04-12T20:56:34.795 INFO:tasks.workunit.client.1.smithi186.stdout:6/491: write d9/d5e/fb0 [619774,72431] 0 2024-04-12T20:56:34.796 INFO:tasks.workunit.client.1.smithi186.stdout:0/573: dwrite d0/d10/d71/f94 [0,4194304] 0 2024-04-12T20:56:34.799 INFO:tasks.workunit.client.1.smithi186.stdout:8/474: chown d7/d18/d36/d1d/d1f/d26 10705 1 2024-04-12T20:56:34.801 INFO:tasks.workunit.client.1.smithi186.stdout:5/511: getdents d5/db/d15/d2a/d6d 0 2024-04-12T20:56:34.803 INFO:tasks.workunit.client.1.smithi186.stdout:0/574: creat d0/d10/d72/d79/fba x:0 0 0 2024-04-12T20:56:34.810 INFO:tasks.workunit.client.1.smithi186.stdout:0/575: chown d0/d14/d4e/d1e/l59 0 1 2024-04-12T20:56:34.811 INFO:tasks.workunit.client.1.smithi186.stdout:4/601: dwrite d0/d59/f8f [0,4194304] 0 2024-04-12T20:56:34.815 INFO:tasks.workunit.client.1.smithi186.stdout:8/475: creat d7/d18/d36/d1d/d1f/d26/f95 x:0 0 0 2024-04-12T20:56:34.820 INFO:tasks.workunit.client.1.smithi186.stdout:0/576: creat d0/d5c/d65/fbb x:0 0 0 2024-04-12T20:56:34.820 INFO:tasks.workunit.client.1.smithi186.stdout:4/602: creat d0/d59/dc0/d63/d9b/fc4 x:0 0 0 2024-04-12T20:56:34.822 INFO:tasks.workunit.client.1.smithi186.stdout:5/512: creat d5/db/d15/d2a/d2d/d77/fac x:0 0 0 2024-04-12T20:56:34.828 INFO:tasks.workunit.client.1.smithi186.stdout:4/603: creat d0/dc/d33/fc5 x:0 0 0 2024-04-12T20:56:34.830 INFO:tasks.workunit.client.1.smithi186.stdout:4/604: read - d0/dc/fa2 zero size 2024-04-12T20:56:34.832 INFO:tasks.workunit.client.1.smithi186.stdout:0/577: link d0/d10/d32/c38 d0/d14/d4e/d1e/d84/cbc 0 2024-04-12T20:56:34.842 INFO:tasks.workunit.client.1.smithi186.stdout:0/578: read d0/d10/d72/f7d [1863,46726] 0 2024-04-12T20:56:34.843 INFO:tasks.workunit.client.1.smithi186.stdout:4/605: write d0/d21/f31 [3407016,43443] 0 2024-04-12T20:56:34.843 INFO:tasks.workunit.client.1.smithi186.stdout:3/496: sync 2024-04-12T20:56:34.845 INFO:tasks.workunit.client.1.smithi186.stdout:8/476: dwrite d7/d18/d36/d20/d23/d25/d29/f6a [0,4194304] 0 2024-04-12T20:56:34.846 INFO:tasks.workunit.client.1.smithi186.stdout:7/527: dread d7/d10/d1c/d5d/f80 [0,4194304] 0 2024-04-12T20:56:34.851 INFO:tasks.workunit.client.1.smithi186.stdout:8/477: truncate d7/de/d7a/f4f 794966 0 2024-04-12T20:56:34.853 INFO:tasks.workunit.client.1.smithi186.stdout:5/513: dread d5/db/d15/f28 [0,4194304] 0 2024-04-12T20:56:34.853 INFO:tasks.workunit.client.1.smithi186.stdout:7/528: creat d7/d10/fbf x:0 0 0 2024-04-12T20:56:34.857 INFO:tasks.workunit.client.1.smithi186.stdout:1/492: dread d14/d1a/d21/f27 [0,4194304] 0 2024-04-12T20:56:34.858 INFO:tasks.workunit.client.1.smithi186.stdout:3/497: dread d1/d3/d21/d22/d24/f53 [0,4194304] 0 2024-04-12T20:56:34.858 INFO:tasks.workunit.client.1.smithi186.stdout:6/492: dread d9/d43/d59/d5d/d6e/fa0 [0,4194304] 0 2024-04-12T20:56:34.861 INFO:tasks.workunit.client.1.smithi186.stdout:7/529: symlink d7/d10/d2d/d40/lc0 0 2024-04-12T20:56:34.862 INFO:tasks.workunit.client.1.smithi186.stdout:8/478: getdents d7 0 2024-04-12T20:56:34.863 INFO:tasks.workunit.client.1.smithi186.stdout:1/493: chown d14/f4f 120115 1 2024-04-12T20:56:34.864 INFO:tasks.workunit.client.1.smithi186.stdout:6/493: creat d9/d4c/fb3 x:0 0 0 2024-04-12T20:56:34.864 INFO:tasks.workunit.client.1.smithi186.stdout:5/514: read d5/db/d71/f1a [346128,120420] 0 2024-04-12T20:56:34.868 INFO:tasks.workunit.client.1.smithi186.stdout:7/530: truncate d7/d55/f9f 9432970 0 2024-04-12T20:56:34.869 INFO:tasks.workunit.client.1.smithi186.stdout:5/515: chown d5/db/d71/d88 3 1 2024-04-12T20:56:34.869 INFO:tasks.workunit.client.1.smithi186.stdout:0/579: dwrite d0/d14/d4e/d1e/fa3 [0,4194304] 0 2024-04-12T20:56:34.869 INFO:tasks.workunit.client.1.smithi186.stdout:1/494: creat d14/d1a/d1d/f9b x:0 0 0 2024-04-12T20:56:34.870 INFO:tasks.workunit.client.1.smithi186.stdout:6/494: symlink d9/d43/d59/d5d/d6e/lb4 0 2024-04-12T20:56:34.872 INFO:tasks.workunit.client.1.smithi186.stdout:7/531: creat d7/d55/d71/fc1 x:0 0 0 2024-04-12T20:56:34.874 INFO:tasks.workunit.client.1.smithi186.stdout:5/516: mknod d5/db/d15/d6a/cad 0 2024-04-12T20:56:34.874 INFO:tasks.workunit.client.1.smithi186.stdout:0/580: chown d0/d14/d4e/d50/l95 240 1 2024-04-12T20:56:34.875 INFO:tasks.workunit.client.1.smithi186.stdout:6/495: fsync d9/d43/f45 0 2024-04-12T20:56:34.875 INFO:tasks.workunit.client.1.smithi186.stdout:7/532: readlink d7/d55/d71/l76 0 2024-04-12T20:56:34.876 INFO:tasks.workunit.client.1.smithi186.stdout:0/581: fsync d0/f57 0 2024-04-12T20:56:34.878 INFO:tasks.workunit.client.1.smithi186.stdout:1/495: mknod d14/d1a/d33/d36/d5b/d6a/c9c 0 2024-04-12T20:56:34.882 INFO:tasks.workunit.client.1.smithi186.stdout:7/533: read - d7/d10/d2d/d40/d4a/d6c/fbc zero size 2024-04-12T20:56:34.883 INFO:tasks.workunit.client.1.smithi186.stdout:1/496: dread - d14/d1a/d33/d36/f7b zero size 2024-04-12T20:56:34.885 INFO:tasks.workunit.client.1.smithi186.stdout:3/498: dwrite d1/d3/d21/d22/d24/d2f/f92 [0,4194304] 0 2024-04-12T20:56:34.886 INFO:tasks.workunit.client.1.smithi186.stdout:0/582: truncate d0/d10/d32/f48 1290890 0 2024-04-12T20:56:34.886 INFO:tasks.workunit.client.1.smithi186.stdout:1/497: read - d14/d1a/d21/f57 zero size 2024-04-12T20:56:34.892 INFO:tasks.workunit.client.1.smithi186.stdout:0/583: fdatasync d0/d5c/d66/da7/fad 0 2024-04-12T20:56:34.892 INFO:tasks.workunit.client.1.smithi186.stdout:6/496: rename d9/d43/d2d/f4e to d9/d43/d59/d5d/d6e/d91/fb5 0 2024-04-12T20:56:34.893 INFO:tasks.workunit.client.1.smithi186.stdout:1/498: stat d14/d1a/d33/d36/d5b/l75 0 2024-04-12T20:56:34.895 INFO:tasks.workunit.client.1.smithi186.stdout:3/499: mkdir d1/d3/d49/d4d/d6f/da6 0 2024-04-12T20:56:34.897 INFO:tasks.workunit.client.1.smithi186.stdout:0/584: unlink d0/d14/f1a 0 2024-04-12T20:56:34.897 INFO:tasks.workunit.client.1.smithi186.stdout:7/534: mkdir d7/d10/d2d/d40/d4a/d6c/d97/dc2 0 2024-04-12T20:56:34.898 INFO:tasks.workunit.client.1.smithi186.stdout:6/497: truncate d9/d43/d50/f79 281421 0 2024-04-12T20:56:34.899 INFO:tasks.workunit.client.1.smithi186.stdout:5/517: rename d5/db/d15/d2a/d4c/d5b/d7a/ca8 to d5/db/d15/d2a/d2d/d48/da4/cae 0 2024-04-12T20:56:34.901 INFO:tasks.workunit.client.1.smithi186.stdout:1/499: creat d14/d1a/d33/d36/f9d x:0 0 0 2024-04-12T20:56:34.901 INFO:tasks.workunit.client.1.smithi186.stdout:7/535: fdatasync d7/f43 0 2024-04-12T20:56:34.906 INFO:tasks.workunit.client.1.smithi186.stdout:7/536: fdatasync d7/d10/d2d/d86/f93 0 2024-04-12T20:56:34.906 INFO:tasks.workunit.client.1.smithi186.stdout:7/537: fdatasync d7/f17 0 2024-04-12T20:56:34.906 INFO:tasks.workunit.client.1.smithi186.stdout:6/498: write d9/d4c/d6f/fa2 [597115,117925] 0 2024-04-12T20:56:34.908 INFO:tasks.workunit.client.1.smithi186.stdout:6/499: readlink d9/d4c/d6f/l88 0 2024-04-12T20:56:34.908 INFO:tasks.workunit.client.1.smithi186.stdout:5/518: mkdir d5/db/d15/d2a/d6d/d7f/daf 0 2024-04-12T20:56:34.909 INFO:tasks.workunit.client.1.smithi186.stdout:5/519: fdatasync d5/db/f76 0 2024-04-12T20:56:34.911 INFO:tasks.workunit.client.1.smithi186.stdout:6/500: dread - d9/d5e/f9d zero size 2024-04-12T20:56:34.912 INFO:tasks.workunit.client.1.smithi186.stdout:1/500: creat d14/d1a/d33/d36/d5b/d3e/f9e x:0 0 0 2024-04-12T20:56:34.914 INFO:tasks.workunit.client.1.smithi186.stdout:5/520: stat d5/db/d15/d2a/d2d/d46/d55 0 2024-04-12T20:56:34.918 INFO:tasks.workunit.client.1.smithi186.stdout:8/479: dread d7/de/d10/f15 [0,4194304] 0 2024-04-12T20:56:34.919 INFO:tasks.workunit.client.1.smithi186.stdout:5/521: stat d5/db/d15/f39 0 2024-04-12T20:56:34.921 INFO:tasks.workunit.client.1.smithi186.stdout:1/501: unlink d14/l32 0 2024-04-12T20:56:34.927 INFO:tasks.workunit.client.1.smithi186.stdout:5/522: write d5/d44/f6e [6011449,13108] 0 2024-04-12T20:56:34.928 INFO:tasks.workunit.client.1.smithi186.stdout:3/500: dwrite d1/d3/d21/d5c/f77 [4194304,4194304] 0 2024-04-12T20:56:34.930 INFO:tasks.workunit.client.1.smithi186.stdout:8/480: mknod d7/de/d10/d7f/c96 0 2024-04-12T20:56:34.931 INFO:tasks.workunit.client.1.smithi186.stdout:1/502: write d14/d1a/f37 [281615,40612] 0 2024-04-12T20:56:34.932 INFO:tasks.workunit.client.1.smithi186.stdout:0/585: dwrite d0/d5c/f6d [0,4194304] 0 2024-04-12T20:56:34.935 INFO:tasks.workunit.client.1.smithi186.stdout:3/501: readlink d1/d3/d21/d22/d24/l8f 0 2024-04-12T20:56:34.936 INFO:tasks.workunit.client.1.smithi186.stdout:7/538: dwrite d7/d10/fbf [0,4194304] 0 2024-04-12T20:56:34.937 INFO:tasks.workunit.client.1.smithi186.stdout:9/465: sync 2024-04-12T20:56:34.937 INFO:tasks.workunit.client.1.smithi186.stdout:2/539: sync 2024-04-12T20:56:34.940 INFO:tasks.workunit.client.1.smithi186.stdout:3/502: write d1/d3/f7e [211374,71220] 0 2024-04-12T20:56:34.943 INFO:tasks.workunit.client.1.smithi186.stdout:1/503: write fe [352373,90391] 0 2024-04-12T20:56:34.946 INFO:tasks.workunit.client.1.smithi186.stdout:7/539: dread - d7/d10/d2d/f89 zero size 2024-04-12T20:56:34.952 INFO:tasks.workunit.client.1.smithi186.stdout:3/503: readlink d1/d2c/d54/l96 0 2024-04-12T20:56:34.954 INFO:tasks.workunit.client.1.smithi186.stdout:4/606: sync 2024-04-12T20:56:34.958 INFO:tasks.workunit.client.1.smithi186.stdout:9/466: creat d3/d12/d6f/f9c x:0 0 0 2024-04-12T20:56:34.959 INFO:tasks.workunit.client.1.smithi186.stdout:8/481: write d7/d18/d36/d20/d23/d25/d2e/d5c/f7b [1968418,127824] 0 2024-04-12T20:56:34.960 INFO:tasks.workunit.client.1.smithi186.stdout:7/540: fdatasync d7/d10/d1c/d5d/f80 0 2024-04-12T20:56:34.962 INFO:tasks.workunit.client.1.smithi186.stdout:6/501: dwrite d9/d4c/d6f/f7b [0,4194304] 0 2024-04-12T20:56:34.966 INFO:tasks.workunit.client.1.smithi186.stdout:4/607: write d0/dc/d33/f6b [3760555,10456] 0 2024-04-12T20:56:34.969 INFO:tasks.workunit.client.1.smithi186.stdout:1/504: symlink d14/d1a/d33/d36/d5b/d3e/d71/d8c/d95/l9f 0 2024-04-12T20:56:34.970 INFO:tasks.workunit.client.1.smithi186.stdout:7/541: unlink d7/d10/c2a 0 2024-04-12T20:56:34.973 INFO:tasks.workunit.client.1.smithi186.stdout:8/482: symlink d7/d18/d8e/l97 0 2024-04-12T20:56:34.976 INFO:tasks.workunit.client.1.smithi186.stdout:6/502: creat d9/d4c/fb6 x:0 0 0 2024-04-12T20:56:34.976 INFO:tasks.workunit.client.1.smithi186.stdout:3/504: creat d1/d3/d49/d4d/d9c/fa7 x:0 0 0 2024-04-12T20:56:34.977 INFO:tasks.workunit.client.1.smithi186.stdout:4/608: rmdir d0/dc/d33 39 2024-04-12T20:56:34.978 INFO:tasks.workunit.client.1.smithi186.stdout:0/586: dwrite d0/d10/d72/d79/fba [0,4194304] 0 2024-04-12T20:56:34.981 INFO:tasks.workunit.client.1.smithi186.stdout:8/483: creat d7/de/d5e/f98 x:0 0 0 2024-04-12T20:56:34.983 INFO:tasks.workunit.client.1.smithi186.stdout:1/505: chown d14/d1a/d33/d36/d5b/d3e/f53 4294 1 2024-04-12T20:56:34.989 INFO:tasks.workunit.client.1.smithi186.stdout:3/505: write d1/d2c/d48/f5f [1655175,113821] 0 2024-04-12T20:56:34.992 INFO:tasks.workunit.client.1.smithi186.stdout:2/540: dwrite d0/d3/d1a/d21/d26/f2d [0,4194304] 0 2024-04-12T20:56:34.995 INFO:tasks.workunit.client.1.smithi186.stdout:7/542: truncate d7/d10/d2d/d40/d4a/d6c/d97/d4d/fa2 907727 0 2024-04-12T20:56:34.996 INFO:tasks.workunit.client.1.smithi186.stdout:4/609: rename d0/dc/d39 to d0/d59/dc0/d63/da6/dc6 0 2024-04-12T20:56:34.999 INFO:tasks.workunit.client.1.smithi186.stdout:3/506: symlink d1/d3/d49/d4d/d9c/la8 0 2024-04-12T20:56:34.999 INFO:tasks.workunit.client.1.smithi186.stdout:1/506: write d14/d1a/d33/d36/d5b/d3e/d8b/f8e [458874,127126] 0 2024-04-12T20:56:35.001 INFO:tasks.workunit.client.1.smithi186.stdout:0/587: symlink d0/d14/d4e/d1e/lbd 0 2024-04-12T20:56:35.008 INFO:tasks.workunit.client.1.smithi186.stdout:2/541: mknod d0/dc/d2e/d73/c9e 0 2024-04-12T20:56:35.008 INFO:tasks.workunit.client.1.smithi186.stdout:6/503: dwrite d9/d4c/f6c [4194304,4194304] 0 2024-04-12T20:56:35.009 INFO:tasks.workunit.client.1.smithi186.stdout:8/484: rename d7/d18/d36/d1d/d1f/d26/f63 to d7/d18/d36/d20/d23/d25/d2e/d5c/f99 0 2024-04-12T20:56:35.011 INFO:tasks.workunit.client.1.smithi186.stdout:2/542: readlink d0/l1d 0 2024-04-12T20:56:35.013 INFO:tasks.workunit.client.1.smithi186.stdout:0/588: stat d0/d5c/d65/f7b 0 2024-04-12T20:56:35.013 INFO:tasks.workunit.client.1.smithi186.stdout:1/507: chown d14/d1a/d21 959 1 2024-04-12T20:56:35.020 INFO:tasks.workunit.client.1.smithi186.stdout:8/485: write d7/de/d10/f3c [3537978,105928] 0 2024-04-12T20:56:35.021 INFO:tasks.workunit.client.1.smithi186.stdout:3/507: creat d1/d2c/d54/d7c/fa9 x:0 0 0 2024-04-12T20:56:35.027 INFO:tasks.workunit.client.1.smithi186.stdout:7/543: creat d7/d10/d1c/d5d/fc3 x:0 0 0 2024-04-12T20:56:35.027 INFO:tasks.workunit.client.1.smithi186.stdout:4/610: dwrite d0/dc/d64/d87/d98/fbc [0,4194304] 0 2024-04-12T20:56:35.027 INFO:tasks.workunit.client.1.smithi186.stdout:3/508: write d1/d3/da/f6c [41715,64565] 0 2024-04-12T20:56:35.027 INFO:tasks.workunit.client.1.smithi186.stdout:5/523: dread d5/f59 [0,4194304] 0 2024-04-12T20:56:35.035 INFO:tasks.workunit.client.1.smithi186.stdout:0/589: unlink d0/d14/f24 0 2024-04-12T20:56:35.036 INFO:tasks.workunit.client.1.smithi186.stdout:7/544: write d7/d10/d2d/d40/d4a/d6c/fbc [574181,11250] 0 2024-04-12T20:56:35.043 INFO:tasks.workunit.client.1.smithi186.stdout:6/504: dwrite d9/d5e/fb0 [0,4194304] 0 2024-04-12T20:56:35.046 INFO:tasks.workunit.client.1.smithi186.stdout:7/545: dread - d7/d10/d2d/d40/d4a/d6c/d97/f78 zero size 2024-04-12T20:56:35.047 INFO:tasks.workunit.client.1.smithi186.stdout:1/508: write d14/f15 [3011635,127647] 0 2024-04-12T20:56:35.049 INFO:tasks.workunit.client.1.smithi186.stdout:2/543: chown d0/d3/d1a/d21/d26/f6a 6 1 2024-04-12T20:56:35.049 INFO:tasks.workunit.client.1.smithi186.stdout:0/590: creat d0/d5c/d65/fbe x:0 0 0 2024-04-12T20:56:35.050 INFO:tasks.workunit.client.1.smithi186.stdout:1/509: stat d14/d91/l96 0 2024-04-12T20:56:35.052 INFO:tasks.workunit.client.1.smithi186.stdout:9/467: dread d3/f10 [8388608,4194304] 0 2024-04-12T20:56:35.056 INFO:tasks.workunit.client.1.smithi186.stdout:1/510: chown d14/d1a/c20 386 1 2024-04-12T20:56:35.057 INFO:tasks.workunit.client.1.smithi186.stdout:8/486: link d7/d18/d36/d20/d23/d25/d2e/d5c/l78 d7/d18/d36/d20/d23/l9a 0 2024-04-12T20:56:35.061 INFO:tasks.workunit.client.1.smithi186.stdout:2/544: dread d0/d32/f97 [0,4194304] 0 2024-04-12T20:56:35.061 INFO:tasks.workunit.client.1.smithi186.stdout:9/468: mkdir d3/d2f/d2e/d33/d9d 0 2024-04-12T20:56:35.064 INFO:tasks.workunit.client.1.smithi186.stdout:8/487: read d7/d18/f1a [6275929,98207] 0 2024-04-12T20:56:35.070 INFO:tasks.workunit.client.1.smithi186.stdout:2/545: mknod d0/d32/c9f 0 2024-04-12T20:56:35.070 INFO:tasks.workunit.client.1.smithi186.stdout:0/591: creat d0/fbf x:0 0 0 2024-04-12T20:56:35.072 INFO:tasks.workunit.client.1.smithi186.stdout:7/546: rename d7/f8 to d7/d10/d2d/d40/fc4 0 2024-04-12T20:56:35.074 INFO:tasks.workunit.client.1.smithi186.stdout:8/488: mknod d7/de/d7a/c9b 0 2024-04-12T20:56:35.075 INFO:tasks.workunit.client.1.smithi186.stdout:3/509: dwrite d1/d2c/d48/f4f [4194304,4194304] 0 2024-04-12T20:56:35.077 INFO:tasks.workunit.client.1.smithi186.stdout:0/592: dread - d0/d5c/d65/fbb zero size 2024-04-12T20:56:35.081 INFO:tasks.workunit.client.1.smithi186.stdout:7/547: chown d7/d10/d1c/f20 253971713 1 2024-04-12T20:56:35.083 INFO:tasks.workunit.client.1.smithi186.stdout:6/505: dwrite d9/d4c/f61 [0,4194304] 0 2024-04-12T20:56:35.088 INFO:tasks.workunit.client.1.smithi186.stdout:4/611: dwrite d0/dc/d33/f43 [0,4194304] 0 2024-04-12T20:56:35.097 INFO:tasks.workunit.client.1.smithi186.stdout:0/593: mkdir d0/dc0 0 2024-04-12T20:56:35.099 INFO:tasks.workunit.client.1.smithi186.stdout:4/612: symlink d0/d26/lc7 0 2024-04-12T20:56:35.105 INFO:tasks.workunit.client.1.smithi186.stdout:2/546: getdents d0/d3/d7a 0 2024-04-12T20:56:35.105 INFO:tasks.workunit.client.1.smithi186.stdout:7/548: creat d7/d55/d71/d99/fc5 x:0 0 0 2024-04-12T20:56:35.106 INFO:tasks.workunit.client.1.smithi186.stdout:6/506: write d9/d4c/fb3 [686803,87031] 0 2024-04-12T20:56:35.109 INFO:tasks.workunit.client.1.smithi186.stdout:1/511: dwrite d14/d1a/d33/d36/d5b/d3e/d71/f78 [0,4194304] 0 2024-04-12T20:56:35.112 INFO:tasks.workunit.client.1.smithi186.stdout:0/594: mkdir d0/dc0/dc1 0 2024-04-12T20:56:35.119 INFO:tasks.workunit.client.1.smithi186.stdout:7/549: fsync d7/d10/d2d/f44 0 2024-04-12T20:56:35.122 INFO:tasks.workunit.client.1.smithi186.stdout:9/469: dwrite d3/d2f/d2e/d43/d4c/f89 [0,4194304] 0 2024-04-12T20:56:35.125 INFO:tasks.workunit.client.1.smithi186.stdout:2/547: symlink d0/d3/d1a/d24/la0 0 2024-04-12T20:56:35.126 INFO:tasks.workunit.client.1.smithi186.stdout:5/524: dwrite f2 [0,4194304] 0 2024-04-12T20:56:35.129 INFO:tasks.workunit.client.1.smithi186.stdout:7/550: stat d7/d10/d2d/d86/lab 0 2024-04-12T20:56:35.134 INFO:tasks.workunit.client.1.smithi186.stdout:0/595: read d0/d10/d32/d3f/f64 [1394338,33444] 0 2024-04-12T20:56:35.136 INFO:tasks.workunit.client.1.smithi186.stdout:5/525: mknod d5/db/d2e/cb0 0 2024-04-12T20:56:35.136 INFO:tasks.workunit.client.1.smithi186.stdout:8/489: dread d7/d18/f4c [0,4194304] 0 2024-04-12T20:56:35.139 INFO:tasks.workunit.client.1.smithi186.stdout:2/548: mknod d0/d3/d1a/d21/d22/ca1 0 2024-04-12T20:56:35.143 INFO:tasks.workunit.client.1.smithi186.stdout:2/549: chown d0/d32/f97 85 1 2024-04-12T20:56:35.146 INFO:tasks.workunit.client.1.smithi186.stdout:1/512: dread d14/d1a/d33/d36/d5b/f2d [0,4194304] 0 2024-04-12T20:56:35.152 INFO:tasks.workunit.client.1.smithi186.stdout:8/490: dread d7/de/d7a/f4f [0,4194304] 0 2024-04-12T20:56:35.153 INFO:tasks.workunit.client.1.smithi186.stdout:1/513: rename d14/d1a/d33/d36/d5b/d3e to d14/d1a/d33/d36/d5b/d3e/d5f/da0 22 2024-04-12T20:56:35.155 INFO:tasks.workunit.client.1.smithi186.stdout:1/514: chown d14/d1a/d21 43 1 2024-04-12T20:56:35.163 INFO:tasks.workunit.client.1.smithi186.stdout:8/491: read - d7/d18/d36/d20/d23/d25/d2e/f94 zero size 2024-04-12T20:56:35.172 INFO:tasks.workunit.client.1.smithi186.stdout:1/515: dwrite d14/d1a/d21/d35/f6e [0,4194304] 0 2024-04-12T20:56:35.176 INFO:tasks.workunit.client.1.smithi186.stdout:3/510: dread d1/fc [0,4194304] 0 2024-04-12T20:56:35.179 INFO:tasks.workunit.client.1.smithi186.stdout:3/511: fsync d1/d3/d49/d4d/d9c/fa7 0 2024-04-12T20:56:35.183 INFO:tasks.workunit.client.1.smithi186.stdout:9/470: fdatasync d3/d2f/d2e/d43/d4c/f89 0 2024-04-12T20:56:35.185 INFO:tasks.workunit.client.1.smithi186.stdout:7/551: creat d7/d55/d71/dbe/fc6 x:0 0 0 2024-04-12T20:56:35.196 INFO:tasks.workunit.client.1.smithi186.stdout:6/507: dread d9/f40 [0,4194304] 0 2024-04-12T20:56:35.200 INFO:tasks.workunit.client.1.smithi186.stdout:2/550: symlink d0/d3/d1a/d27/la2 0 2024-04-12T20:56:35.201 INFO:tasks.workunit.client.1.smithi186.stdout:2/551: stat d0/d3/d1a/f45 0 2024-04-12T20:56:35.223 INFO:tasks.workunit.client.1.smithi186.stdout:9/471: mknod d3/d2f/d2e/d43/d4c/d82/c9e 0 2024-04-12T20:56:35.228 INFO:tasks.workunit.client.1.smithi186.stdout:4/613: dwrite d0/d21/f38 [0,4194304] 0 2024-04-12T20:56:35.235 INFO:tasks.workunit.client.1.smithi186.stdout:2/552: rename d0/d3/d1a/l83 to d0/d3/la3 0 2024-04-12T20:56:35.235 INFO:tasks.workunit.client.1.smithi186.stdout:6/508: write d9/f10 [4994240,43379] 0 2024-04-12T20:56:35.238 INFO:tasks.workunit.client.1.smithi186.stdout:8/492: link d7/d18/f1a d7/d18/d36/d20/d23/d25/d2e/d67/f9c 0 2024-04-12T20:56:35.250 INFO:tasks.workunit.client.1.smithi186.stdout:9/472: mkdir d3/d2f/d54/d9f 0 2024-04-12T20:56:35.254 INFO:tasks.workunit.client.1.smithi186.stdout:6/509: creat d9/d43/d59/d5d/d6e/fb7 x:0 0 0 2024-04-12T20:56:35.256 INFO:tasks.workunit.client.1.smithi186.stdout:1/516: creat d14/d1a/fa1 x:0 0 0 2024-04-12T20:56:35.256 INFO:tasks.workunit.client.1.smithi186.stdout:6/510: chown d9/f37 51419 1 2024-04-12T20:56:35.258 INFO:tasks.workunit.client.1.smithi186.stdout:9/473: dread - d3/d2f/d54/f60 zero size 2024-04-12T20:56:35.259 INFO:tasks.workunit.client.1.smithi186.stdout:9/474: chown d3/d12/c64 2945117 1 2024-04-12T20:56:35.259 INFO:tasks.workunit.client.1.smithi186.stdout:2/553: creat d0/d3/d7a/d8d/fa4 x:0 0 0 2024-04-12T20:56:35.260 INFO:tasks.workunit.client.1.smithi186.stdout:9/475: dread - d3/f19 zero size 2024-04-12T20:56:35.261 INFO:tasks.workunit.client.1.smithi186.stdout:6/511: write d9/f2e [754791,66643] 0 2024-04-12T20:56:35.262 INFO:tasks.workunit.client.1.smithi186.stdout:1/517: mkdir d14/d1a/d33/d36/d5b/d6a/da2 0 2024-04-12T20:56:35.265 INFO:tasks.workunit.client.1.smithi186.stdout:5/526: sync 2024-04-12T20:56:35.266 INFO:tasks.workunit.client.1.smithi186.stdout:0/596: mknod d0/d14/cc2 0 2024-04-12T20:56:35.267 INFO:tasks.workunit.client.1.smithi186.stdout:9/476: chown d3/l31 567 1 2024-04-12T20:56:35.268 INFO:tasks.workunit.client.1.smithi186.stdout:7/552: symlink d7/d10/d1c/lc7 0 2024-04-12T20:56:35.269 INFO:tasks.workunit.client.1.smithi186.stdout:2/554: creat d0/d3/d1a/fa5 x:0 0 0 2024-04-12T20:56:35.270 INFO:tasks.workunit.client.1.smithi186.stdout:0/597: read d0/d14/f35 [3061911,43740] 0 2024-04-12T20:56:35.272 INFO:tasks.workunit.client.1.smithi186.stdout:5/527: creat d5/db/d15/d2a/d4c/d5b/d7a/fb1 x:0 0 0 2024-04-12T20:56:35.273 INFO:tasks.workunit.client.1.smithi186.stdout:2/555: readlink d0/d3/d1a/d27/la2 0 2024-04-12T20:56:35.274 INFO:tasks.workunit.client.1.smithi186.stdout:0/598: read - d0/d10/f8f zero size 2024-04-12T20:56:35.276 INFO:tasks.workunit.client.1.smithi186.stdout:1/518: creat d14/d1a/d33/d36/d5b/fa3 x:0 0 0 2024-04-12T20:56:35.277 INFO:tasks.workunit.client.1.smithi186.stdout:5/528: chown d5/db/d15/d2a/d6d/d7f/d80 151944 1 2024-04-12T20:56:35.280 INFO:tasks.workunit.client.1.smithi186.stdout:4/614: dread d0/dc/f2f [0,4194304] 0 2024-04-12T20:56:35.280 INFO:tasks.workunit.client.1.smithi186.stdout:2/556: mknod d0/d3/d7a/d8d/ca6 0 2024-04-12T20:56:35.286 INFO:tasks.workunit.client.1.smithi186.stdout:2/557: creat d0/d3/d1a/d41/fa7 x:0 0 0 2024-04-12T20:56:35.288 INFO:tasks.workunit.client.1.smithi186.stdout:2/558: chown d0/d32/f6f 1427 1 2024-04-12T20:56:35.289 INFO:tasks.workunit.client.1.smithi186.stdout:4/615: mknod d0/dc/cc8 0 2024-04-12T20:56:35.290 INFO:tasks.workunit.client.1.smithi186.stdout:6/512: dwrite d9/d43/d50/f54 [0,4194304] 0 2024-04-12T20:56:35.291 INFO:tasks.workunit.client.1.smithi186.stdout:5/529: creat d5/db/d15/d2a/d4c/d93/fb2 x:0 0 0 2024-04-12T20:56:35.292 INFO:tasks.workunit.client.1.smithi186.stdout:1/519: link d14/d1a/fa1 d14/d1a/d58/d69/fa4 0 2024-04-12T20:56:35.295 INFO:tasks.workunit.client.1.smithi186.stdout:2/559: chown d0/d32/l4e 973970 1 2024-04-12T20:56:35.297 INFO:tasks.workunit.client.1.smithi186.stdout:1/520: mkdir d14/d1a/d58/da5 0 2024-04-12T20:56:35.298 INFO:tasks.workunit.client.1.smithi186.stdout:2/560: fdatasync d0/dc/f4c 0 2024-04-12T20:56:35.298 INFO:tasks.workunit.client.1.smithi186.stdout:4/616: mknod d0/d21/db9/cc9 0 2024-04-12T20:56:35.301 INFO:tasks.workunit.client.1.smithi186.stdout:5/530: mknod d5/db/d15/d2a/d6d/d7f/cb3 0 2024-04-12T20:56:35.306 INFO:tasks.workunit.client.1.smithi186.stdout:4/617: mknod d0/d59/dc0/d63/cca 0 2024-04-12T20:56:35.314 INFO:tasks.workunit.client.1.smithi186.stdout:6/513: creat d9/d43/d59/fb8 x:0 0 0 2024-04-12T20:56:35.316 INFO:tasks.workunit.client.1.smithi186.stdout:4/618: chown d0/d59/dc0/l7b 0 1 2024-04-12T20:56:35.317 INFO:tasks.workunit.client.1.smithi186.stdout:4/619: chown d0/dc/d33/f6b 234485611 1 2024-04-12T20:56:35.321 INFO:tasks.workunit.client.1.smithi186.stdout:3/512: truncate d1/d3/d21/d22/d24/f60 1515651 0 2024-04-12T20:56:35.322 INFO:tasks.workunit.client.1.smithi186.stdout:2/561: link d0/d3/d1a/f45 d0/dc/fa8 0 2024-04-12T20:56:35.324 INFO:tasks.workunit.client.1.smithi186.stdout:8/493: rename d7/d18/d36/d20/f8b to d7/de/d68/f9d 0 2024-04-12T20:56:35.325 INFO:tasks.workunit.client.1.smithi186.stdout:2/562: write d0/d3/d1a/f87 [933533,118909] 0 2024-04-12T20:56:35.325 INFO:tasks.workunit.client.1.smithi186.stdout:4/620: chown d0/d59/dc0/d48/c77 14354 1 2024-04-12T20:56:35.328 INFO:tasks.workunit.client.1.smithi186.stdout:4/621: stat d0/dc/d64 0 2024-04-12T20:56:35.328 INFO:tasks.workunit.client.1.smithi186.stdout:5/531: dread d5/d44/f67 [0,4194304] 0 2024-04-12T20:56:35.329 INFO:tasks.workunit.client.1.smithi186.stdout:3/513: rmdir d1/d3/d21/d22/d24/d2f 39 2024-04-12T20:56:35.331 INFO:tasks.workunit.client.1.smithi186.stdout:2/563: chown d0/d3/d1a/d41/fa7 610 1 2024-04-12T20:56:35.331 INFO:tasks.workunit.client.1.smithi186.stdout:1/521: dread d14/d1a/d1d/f23 [0,4194304] 0 2024-04-12T20:56:35.338 INFO:tasks.workunit.client.1.smithi186.stdout:4/622: chown d0/d59/dc0/c7e 950 1 2024-04-12T20:56:35.338 INFO:tasks.workunit.client.1.smithi186.stdout:3/514: chown d1/d3/da2/f9d 46671983 1 2024-04-12T20:56:35.338 INFO:tasks.workunit.client.1.smithi186.stdout:5/532: mknod d5/db/d15/d2a/d2d/d46/cb4 0 2024-04-12T20:56:35.341 INFO:tasks.workunit.client.1.smithi186.stdout:2/564: mkdir d0/dc/d2e/da9 0 2024-04-12T20:56:35.342 INFO:tasks.workunit.client.1.smithi186.stdout:6/514: symlink d9/d43/lb9 0 2024-04-12T20:56:35.343 INFO:tasks.workunit.client.1.smithi186.stdout:0/599: getdents d0/d14 0 2024-04-12T20:56:35.346 INFO:tasks.workunit.client.1.smithi186.stdout:3/515: symlink d1/d3/d49/d4d/d6f/laa 0 2024-04-12T20:56:35.346 INFO:tasks.workunit.client.1.smithi186.stdout:6/515: chown d9/d43/d2d/d4d/f64 50767 1 2024-04-12T20:56:35.347 INFO:tasks.workunit.client.1.smithi186.stdout:8/494: dwrite d7/d18/d36/d1d/d1f/d26/f3d [4194304,4194304] 0 2024-04-12T20:56:35.347 INFO:tasks.workunit.client.1.smithi186.stdout:1/522: creat d14/d1a/d33/d36/d5b/d6a/da2/fa6 x:0 0 0 2024-04-12T20:56:35.352 INFO:tasks.workunit.client.1.smithi186.stdout:7/553: mknod d7/d10/d2d/d40/cc8 0 2024-04-12T20:56:35.354 INFO:tasks.workunit.client.1.smithi186.stdout:6/516: dread d9/d43/d50/f79 [0,4194304] 0 2024-04-12T20:56:35.357 INFO:tasks.workunit.client.1.smithi186.stdout:3/516: creat d1/d2c/d48/fab x:0 0 0 2024-04-12T20:56:35.358 INFO:tasks.workunit.client.1.smithi186.stdout:6/517: fdatasync d9/d4c/d6f/fb2 0 2024-04-12T20:56:35.360 INFO:tasks.workunit.client.1.smithi186.stdout:5/533: dwrite d5/fd [0,4194304] 0 2024-04-12T20:56:35.363 INFO:tasks.workunit.client.1.smithi186.stdout:6/518: fsync d9/d43/d2d/f7e 0 2024-04-12T20:56:35.369 INFO:tasks.workunit.client.1.smithi186.stdout:7/554: fsync d7/d10/d1c/f20 0 2024-04-12T20:56:35.373 INFO:tasks.workunit.client.1.smithi186.stdout:4/623: dwrite d0/f24 [0,4194304] 0 2024-04-12T20:56:35.376 INFO:tasks.workunit.client.1.smithi186.stdout:7/555: write d7/d10/fbf [1916542,76329] 0 2024-04-12T20:56:35.376 INFO:tasks.workunit.client.1.smithi186.stdout:3/517: unlink d1/d3/d21/d22/d34/f98 0 2024-04-12T20:56:35.378 INFO:tasks.workunit.client.1.smithi186.stdout:5/534: creat d5/db/d15/d2a/d2d/d48/fb5 x:0 0 0 2024-04-12T20:56:35.383 INFO:tasks.workunit.client.1.smithi186.stdout:6/519: rmdir d9/d43/d59/d5d/d6e 39 2024-04-12T20:56:35.384 INFO:tasks.workunit.client.1.smithi186.stdout:6/520: readlink d9/l14 0 2024-04-12T20:56:35.387 INFO:tasks.workunit.client.1.smithi186.stdout:4/624: fsync d0/d59/dc0/d63/da6/dc6/f66 0 2024-04-12T20:56:35.388 INFO:tasks.workunit.client.1.smithi186.stdout:7/556: mknod d7/d10/d2d/d86/cc9 0 2024-04-12T20:56:35.394 INFO:tasks.workunit.client.1.smithi186.stdout:5/535: rename d5/db/d15/d2a/d2d/d46/d7d to d5/db/d15/d2a/d6d/d7f/d86/db6 0 2024-04-12T20:56:35.395 INFO:tasks.workunit.client.1.smithi186.stdout:6/521: symlink d9/d5e/d7a/d81/d96/lba 0 2024-04-12T20:56:35.395 INFO:tasks.workunit.client.1.smithi186.stdout:7/557: dread d7/d10/d2d/d40/d4a/f51 [0,4194304] 0 2024-04-12T20:56:35.396 INFO:tasks.workunit.client.1.smithi186.stdout:5/536: dread - d5/db/f76 zero size 2024-04-12T20:56:35.397 INFO:tasks.workunit.client.1.smithi186.stdout:9/477: sync 2024-04-12T20:56:35.401 INFO:tasks.workunit.client.1.smithi186.stdout:0/600: link d0/d14/d4e/d50/l95 d0/d14/d4e/d1e/d84/lc3 0 2024-04-12T20:56:35.403 INFO:tasks.workunit.client.1.smithi186.stdout:1/523: rmdir d14/d1a 39 2024-04-12T20:56:35.406 INFO:tasks.workunit.client.1.smithi186.stdout:2/565: truncate d0/f72 3987909 0 2024-04-12T20:56:35.406 INFO:tasks.workunit.client.1.smithi186.stdout:8/495: write d7/de/d5e/f48 [970603,130925] 0 2024-04-12T20:56:35.407 INFO:tasks.workunit.client.1.smithi186.stdout:7/558: rmdir d7/d10/d2d/d40/d4a/d6c/d97 39 2024-04-12T20:56:35.408 INFO:tasks.workunit.client.1.smithi186.stdout:5/537: creat d5/db/d15/d2a/d6d/d7f/fb7 x:0 0 0 2024-04-12T20:56:35.411 INFO:tasks.workunit.client.1.smithi186.stdout:7/559: dread - d7/d55/d71/dbe/fc6 zero size 2024-04-12T20:56:35.411 INFO:tasks.workunit.client.1.smithi186.stdout:8/496: symlink d7/l9e 0 2024-04-12T20:56:35.412 INFO:tasks.workunit.client.1.smithi186.stdout:9/478: dread - d3/d2f/d2e/d33/d36/d70/f99 zero size 2024-04-12T20:56:35.412 INFO:tasks.workunit.client.1.smithi186.stdout:3/518: dread d1/d2c/d48/f57 [0,4194304] 0 2024-04-12T20:56:35.413 INFO:tasks.workunit.client.1.smithi186.stdout:5/538: dread - d5/db/d15/d2a/d2d/d48/da4/fa7 zero size 2024-04-12T20:56:35.415 INFO:tasks.workunit.client.1.smithi186.stdout:3/519: readlink d1/d3/d21/l6b 0 2024-04-12T20:56:35.417 INFO:tasks.workunit.client.1.smithi186.stdout:1/524: creat d14/d1a/d33/d36/d5b/d3e/d71/d8c/d95/fa7 x:0 0 0 2024-04-12T20:56:35.417 INFO:tasks.workunit.client.1.smithi186.stdout:6/522: dread d9/f40 [0,4194304] 0 2024-04-12T20:56:35.418 INFO:tasks.workunit.client.1.smithi186.stdout:9/479: creat d3/d12/fa0 x:0 0 0 2024-04-12T20:56:35.419 INFO:tasks.workunit.client.1.smithi186.stdout:6/523: chown d9/l16 31907430 1 2024-04-12T20:56:35.420 INFO:tasks.workunit.client.1.smithi186.stdout:7/560: symlink d7/d10/d2d/d40/d4a/d5c/lca 0 2024-04-12T20:56:35.423 INFO:tasks.workunit.client.1.smithi186.stdout:5/539: symlink d5/db/d15/d2a/d2d/lb8 0 2024-04-12T20:56:35.426 INFO:tasks.workunit.client.1.smithi186.stdout:1/525: dread fe [4194304,4194304] 0 2024-04-12T20:56:35.429 INFO:tasks.workunit.client.1.smithi186.stdout:2/566: dwrite d0/d3/d1a/d24/f33 [0,4194304] 0 2024-04-12T20:56:35.433 INFO:tasks.workunit.client.1.smithi186.stdout:6/524: symlink d9/d43/d2d/d4d/lbb 0 2024-04-12T20:56:35.434 INFO:tasks.workunit.client.1.smithi186.stdout:0/601: dwrite d0/d14/d4e/d1e/f4b [0,4194304] 0 2024-04-12T20:56:35.435 INFO:tasks.workunit.client.1.smithi186.stdout:7/561: write d7/d10/f41 [1703866,82667] 0 2024-04-12T20:56:35.440 INFO:tasks.workunit.client.1.smithi186.stdout:2/567: readlink d0/d3/d1a/d21/d26/l75 0 2024-04-12T20:56:35.440 INFO:tasks.workunit.client.1.smithi186.stdout:1/526: fsync d14/d1a/d1d/f29 0 2024-04-12T20:56:35.442 INFO:tasks.workunit.client.1.smithi186.stdout:6/525: creat d9/d4c/d6f/d97/fbc x:0 0 0 2024-04-12T20:56:35.447 INFO:tasks.workunit.client.1.smithi186.stdout:0/602: chown d0/d14/d4e/d50/d6e/d96 12 1 2024-04-12T20:56:35.449 INFO:tasks.workunit.client.1.smithi186.stdout:4/625: dwrite d0/f69 [0,4194304] 0 2024-04-12T20:56:35.450 INFO:tasks.workunit.client.1.smithi186.stdout:7/562: mkdir d7/d10/d2d/d86/dcb 0 2024-04-12T20:56:35.452 INFO:tasks.workunit.client.1.smithi186.stdout:9/480: link d3/d12/d6f/c74 d3/d2f/d2e/ca1 0 2024-04-12T20:56:35.457 INFO:tasks.workunit.client.1.smithi186.stdout:0/603: rename d0/dc0 to d0/d85/dc4 0 2024-04-12T20:56:35.457 INFO:tasks.workunit.client.1.smithi186.stdout:8/497: write d7/de/d5e/f98 [952311,21711] 0 2024-04-12T20:56:35.458 INFO:tasks.workunit.client.1.smithi186.stdout:4/626: symlink d0/d26/lcb 0 2024-04-12T20:56:35.459 INFO:tasks.workunit.client.1.smithi186.stdout:2/568: write d0/d3/d1a/d24/f60 [276118,20946] 0 2024-04-12T20:56:35.462 INFO:tasks.workunit.client.1.smithi186.stdout:6/526: mkdir d9/da9/dbd 0 2024-04-12T20:56:35.462 INFO:tasks.workunit.client.1.smithi186.stdout:3/520: dread d1/f63 [0,4194304] 0 2024-04-12T20:56:35.463 INFO:tasks.workunit.client.1.smithi186.stdout:9/481: stat d3/d2f/d2e/d43/d4c 0 2024-04-12T20:56:35.466 INFO:tasks.workunit.client.1.smithi186.stdout:5/540: dwrite d5/db/d15/d2a/d4c/d5b/d7a/fb1 [0,4194304] 0 2024-04-12T20:56:35.469 INFO:tasks.workunit.client.1.smithi186.stdout:6/527: mkdir d9/d4c/dbe 0 2024-04-12T20:56:35.473 INFO:tasks.workunit.client.1.smithi186.stdout:4/627: symlink d0/d21/lcc 0 2024-04-12T20:56:35.474 INFO:tasks.workunit.client.1.smithi186.stdout:2/569: rename d0/d3/d1a/d21/d26/f88 to d0/d3/d7a/d8d/faa 0 2024-04-12T20:56:35.477 INFO:tasks.workunit.client.1.smithi186.stdout:5/541: creat d5/db/d15/d2a/d6d/fb9 x:0 0 0 2024-04-12T20:56:35.478 INFO:tasks.workunit.client.1.smithi186.stdout:9/482: symlink d3/d2f/d2e/d33/d66/d77/la2 0 2024-04-12T20:56:35.478 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:56:35 smithi018 ceph-mon[28799]: pgmap v50: 65 pgs: 65 active+clean; 2.8 GiB data, 9.7 GiB used, 527 GiB / 536 GiB avail; 39 MiB/s rd, 166 MiB/s wr, 327 op/s 2024-04-12T20:56:35.478 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:56:35 smithi018 ceph-mon[28799]: from='mgr.24359 172.21.15.186:0/1315906828' entity='mgr.smithi186.kqcfxk' 2024-04-12T20:56:35.479 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:56:35 smithi018 ceph-mon[28799]: from='mgr.24359 172.21.15.186:0/1315906828' entity='mgr.smithi186.kqcfxk' 2024-04-12T20:56:35.479 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:56:35 smithi018 ceph-mon[28799]: from='mgr.24359 172.21.15.186:0/1315906828' entity='mgr.smithi186.kqcfxk' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-04-12T20:56:35.479 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:56:35 smithi018 ceph-mon[28799]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-04-12T20:56:35.479 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:56:35 smithi018 ceph-mon[28799]: from='mgr.24359 172.21.15.186:0/1315906828' entity='mgr.smithi186.kqcfxk' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-04-12T20:56:35.479 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:56:35 smithi018 ceph-mon[28799]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-04-12T20:56:35.479 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:56:35 smithi018 ceph-mon[28799]: from='mgr.24359 172.21.15.186:0/1315906828' entity='mgr.smithi186.kqcfxk' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-04-12T20:56:35.480 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:56:35 smithi018 ceph-mon[28799]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-04-12T20:56:35.480 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:56:35 smithi018 ceph-mon[28799]: from='mgr.24359 172.21.15.186:0/1315906828' entity='mgr.smithi186.kqcfxk' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-12T20:56:35.480 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:56:35 smithi018 ceph-mon[28799]: Upgrade: Need to upgrade myself (mgr.smithi186.kqcfxk) 2024-04-12T20:56:35.480 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:56:35 smithi018 ceph-mon[28799]: Upgrade: Need to upgrade myself (mgr.smithi186.kqcfxk) 2024-04-12T20:56:35.483 INFO:tasks.workunit.client.1.smithi186.stdout:4/628: creat d0/dc/d64/d87/d98/fcd x:0 0 0 2024-04-12T20:56:35.485 INFO:tasks.workunit.client.1.smithi186.stdout:9/483: readlink d3/d2f/d2e/d33/l87 0 2024-04-12T20:56:35.489 INFO:tasks.workunit.client.1.smithi186.stdout:7/563: dwrite d7/f25 [4194304,4194304] 0 2024-04-12T20:56:35.489 INFO:tasks.workunit.client.1.smithi186.stdout:4/629: readlink d0/dc/d64/la0 0 2024-04-12T20:56:35.491 INFO:tasks.workunit.client.1.smithi186.stdout:1/527: dwrite d14/d1a/d33/d36/d5b/d3e/d71/f82 [4194304,4194304] 0 2024-04-12T20:56:35.491 INFO:tasks.workunit.client.1.smithi186.stdout:6/528: creat d9/d43/d59/d5d/fbf x:0 0 0 2024-04-12T20:56:35.492 INFO:tasks.workunit.client.1.smithi186.stdout:5/542: rename d5/db/d15/d2a/d2d/lb8 to d5/d44/lba 0 2024-04-12T20:56:35.497 INFO:tasks.workunit.client.1.smithi186.stdout:4/630: write d0/d59/f8f [2688358,11413] 0 2024-04-12T20:56:35.499 INFO:tasks.workunit.client.1.smithi186.stdout:9/484: symlink d3/d2f/d2e/d43/d4c/d82/la3 0 2024-04-12T20:56:35.499 INFO:tasks.workunit.client.1.smithi186.stdout:1/528: stat d14/d1a/d33/d36/d5b/d3e/f9e 0 2024-04-12T20:56:35.499 INFO:tasks.workunit.client.1.smithi186.stdout:6/529: write d9/d43/d50/fad [58906,51625] 0 2024-04-12T20:56:35.507 INFO:tasks.workunit.client.1.smithi186.stdout:0/604: dread d0/d14/d4e/d1e/f1f [0,4194304] 0 2024-04-12T20:56:35.508 INFO:tasks.workunit.client.1.smithi186.stdout:4/631: chown d0/dc/d1f/d73/c75 77899 1 2024-04-12T20:56:35.509 INFO:tasks.workunit.client.1.smithi186.stdout:8/498: dwrite d7/d18/d36/d1d/d1f/d26/f6b [0,4194304] 0 2024-04-12T20:56:35.509 INFO:tasks.workunit.client.1.smithi186.stdout:6/530: write d9/f1d [786716,25713] 0 2024-04-12T20:56:35.513 INFO:tasks.workunit.client.1.smithi186.stdout:6/531: chown d9/d5e/d73/c90 102269 1 2024-04-12T20:56:35.514 INFO:tasks.workunit.client.1.smithi186.stdout:1/529: rmdir d14/d1a/d58/d69 39 2024-04-12T20:56:35.519 INFO:tasks.workunit.client.1.smithi186.stdout:7/564: mkdir d7/d10/d2d/d40/d4a/d6c/d97/d4d/d6a/dcc 0 2024-04-12T20:56:35.519 INFO:tasks.workunit.client.1.smithi186.stdout:3/521: dread d1/d3/d21/d22/d24/f60 [0,4194304] 0 2024-04-12T20:56:35.521 INFO:tasks.workunit.client.1.smithi186.stdout:8/499: creat d7/de/d10/d7f/f9f x:0 0 0 2024-04-12T20:56:35.522 INFO:tasks.workunit.client.1.smithi186.stdout:9/485: write d3/d12/d6f/d83/f35 [1348877,8845] 0 2024-04-12T20:56:35.526 INFO:tasks.workunit.client.1.smithi186.stdout:6/532: unlink d9/d43/d59/d5d/f94 0 2024-04-12T20:56:35.530 INFO:tasks.workunit.client.1.smithi186.stdout:2/570: dread d0/d3/d1a/d27/f59 [0,4194304] 0 2024-04-12T20:56:35.530 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:56:35 smithi186 ceph-mon[36597]: pgmap v50: 65 pgs: 65 active+clean; 2.8 GiB data, 9.7 GiB used, 527 GiB / 536 GiB avail; 39 MiB/s rd, 166 MiB/s wr, 327 op/s 2024-04-12T20:56:35.531 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:56:35 smithi186 ceph-mon[36597]: from='mgr.24359 172.21.15.186:0/1315906828' entity='mgr.smithi186.kqcfxk' 2024-04-12T20:56:35.531 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:56:35 smithi186 ceph-mon[36597]: from='mgr.24359 172.21.15.186:0/1315906828' entity='mgr.smithi186.kqcfxk' 2024-04-12T20:56:35.531 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:56:35 smithi186 ceph-mon[36597]: from='mgr.24359 172.21.15.186:0/1315906828' entity='mgr.smithi186.kqcfxk' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-04-12T20:56:35.531 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:56:35 smithi186 ceph-mon[36597]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-04-12T20:56:35.531 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:56:35 smithi186 ceph-mon[36597]: from='mgr.24359 172.21.15.186:0/1315906828' entity='mgr.smithi186.kqcfxk' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-04-12T20:56:35.531 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:56:35 smithi186 ceph-mon[36597]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-04-12T20:56:35.531 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:56:35 smithi186 ceph-mon[36597]: from='mgr.24359 172.21.15.186:0/1315906828' entity='mgr.smithi186.kqcfxk' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-04-12T20:56:35.531 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:56:35 smithi186 ceph-mon[36597]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-04-12T20:56:35.531 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:56:35 smithi186 ceph-mon[36597]: from='mgr.24359 172.21.15.186:0/1315906828' entity='mgr.smithi186.kqcfxk' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-12T20:56:35.531 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:56:35 smithi186 ceph-mon[36597]: Upgrade: Need to upgrade myself (mgr.smithi186.kqcfxk) 2024-04-12T20:56:35.531 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:56:35 smithi186 ceph-mon[36597]: Upgrade: Need to upgrade myself (mgr.smithi186.kqcfxk) 2024-04-12T20:56:35.532 INFO:tasks.workunit.client.1.smithi186.stdout:0/605: creat d0/d85/dc4/dc1/fc5 x:0 0 0 2024-04-12T20:56:35.532 INFO:tasks.workunit.client.1.smithi186.stdout:4/632: mkdir d0/d59/dc0/d48/dce 0 2024-04-12T20:56:35.534 INFO:tasks.workunit.client.1.smithi186.stdout:4/633: dread - d0/dc/d1f/d73/fb7 zero size 2024-04-12T20:56:35.535 INFO:tasks.workunit.client.1.smithi186.stdout:7/565: mknod d7/ccd 0 2024-04-12T20:56:35.539 INFO:tasks.workunit.client.1.smithi186.stdout:5/543: dread d5/db/d15/d2a/d2d/d46/d55/f3d [0,4194304] 0 2024-04-12T20:56:35.540 INFO:tasks.workunit.client.1.smithi186.stdout:9/486: mknod d3/d2f/d54/ca4 0 2024-04-12T20:56:35.541 INFO:tasks.workunit.client.1.smithi186.stdout:9/487: fsync d3/d12/fa0 0 2024-04-12T20:56:35.546 INFO:tasks.workunit.client.1.smithi186.stdout:6/533: rmdir d9/d4c/d6f 39 2024-04-12T20:56:35.548 INFO:tasks.workunit.client.1.smithi186.stdout:6/534: readlink d9/d43/l29 0 2024-04-12T20:56:35.550 INFO:tasks.workunit.client.1.smithi186.stdout:7/566: dwrite d7/d10/d2d/d86/f9c [0,4194304] 0 2024-04-12T20:56:35.551 INFO:tasks.workunit.client.1.smithi186.stdout:5/544: dread d5/fd [0,4194304] 0 2024-04-12T20:56:35.554 INFO:tasks.workunit.client.1.smithi186.stdout:5/545: dread - d5/db/d15/d2a/d4c/d93/fb2 zero size 2024-04-12T20:56:35.555 INFO:tasks.workunit.client.1.smithi186.stdout:8/500: truncate d7/de/f21 1699684 0 2024-04-12T20:56:35.555 INFO:tasks.workunit.client.1.smithi186.stdout:5/546: chown d5/db/d15 401 1 2024-04-12T20:56:35.558 INFO:tasks.workunit.client.1.smithi186.stdout:1/530: creat d14/fa8 x:0 0 0 2024-04-12T20:56:35.561 INFO:tasks.workunit.client.1.smithi186.stdout:9/488: rmdir d3/d2f/d2e/d33/d36/d70 39 2024-04-12T20:56:35.563 INFO:tasks.workunit.client.1.smithi186.stdout:3/522: link d1/d3/f58 d1/d2c/d54/d7c/fac 0 2024-04-12T20:56:35.566 INFO:tasks.workunit.client.1.smithi186.stdout:9/489: fsync d3/d2f/d54/f5b 0 2024-04-12T20:56:35.567 INFO:tasks.workunit.client.1.smithi186.stdout:7/567: symlink d7/d10/d1c/d5d/lce 0 2024-04-12T20:56:35.567 INFO:tasks.workunit.client.1.smithi186.stdout:2/571: rename d0/d32/d2a/l99 to d0/d32/d69/lab 0 2024-04-12T20:56:35.569 INFO:tasks.workunit.client.1.smithi186.stdout:4/634: mknod d0/d21/ccf 0 2024-04-12T20:56:35.570 INFO:tasks.workunit.client.1.smithi186.stdout:2/572: stat d0/d3/d1a/d27/c78 0 2024-04-12T20:56:35.570 INFO:tasks.workunit.client.1.smithi186.stdout:5/547: mkdir d5/db/d15/d2a/d6d/dbb 0 2024-04-12T20:56:35.572 INFO:tasks.workunit.client.1.smithi186.stdout:9/490: dread - d3/d12/f7f zero size 2024-04-12T20:56:35.573 INFO:tasks.workunit.client.1.smithi186.stdout:1/531: rmdir d14/d1a/d33/d36/d5b/d3e/d71/d8c/d95 39 2024-04-12T20:56:35.578 INFO:tasks.workunit.client.1.smithi186.stdout:9/491: dread - d3/f73 zero size 2024-04-12T20:56:35.578 INFO:tasks.workunit.client.1.smithi186.stdout:2/573: chown d0/d3/d1a/d21/d26/f9a 143 1 2024-04-12T20:56:35.578 INFO:tasks.workunit.client.1.smithi186.stdout:0/606: dwrite d0/d5c/f5f [4194304,4194304] 0 2024-04-12T20:56:35.578 INFO:tasks.workunit.client.1.smithi186.stdout:6/535: dread d9/d5e/f6d [0,4194304] 0 2024-04-12T20:56:35.580 INFO:tasks.workunit.client.1.smithi186.stdout:2/574: fsync d0/d3/d1a/d21/d26/f2d 0 2024-04-12T20:56:35.581 INFO:tasks.workunit.client.1.smithi186.stdout:6/536: stat d9/d43/d59/d5d/f5a 0 2024-04-12T20:56:35.581 INFO:tasks.workunit.client.1.smithi186.stdout:3/523: mkdir d1/d3/d49/d4d/dad 0 2024-04-12T20:56:35.582 INFO:tasks.workunit.client.1.smithi186.stdout:4/635: unlink d0/dc/d33/f4 0 2024-04-12T20:56:35.583 INFO:tasks.workunit.client.1.smithi186.stdout:2/575: dread - d0/d3/d1a/d24/f67 zero size 2024-04-12T20:56:35.584 INFO:tasks.workunit.client.1.smithi186.stdout:1/532: write fe [3337132,51869] 0 2024-04-12T20:56:35.585 INFO:tasks.workunit.client.1.smithi186.stdout:5/548: mknod d5/db/d15/d2a/d2d/d48/da4/cbc 0 2024-04-12T20:56:35.589 INFO:tasks.workunit.client.1.smithi186.stdout:1/533: chown d14/d1a/d33/d36/f7b 28603 1 2024-04-12T20:56:35.591 INFO:tasks.workunit.client.1.smithi186.stdout:1/534: write d14/d1a/d1d/f94 [837454,6802] 0 2024-04-12T20:56:35.593 INFO:tasks.workunit.client.1.smithi186.stdout:8/501: rename d7/d18/d36/d1d/d1f/d26/f50 to d7/d18/d36/d1d/fa0 0 2024-04-12T20:56:35.593 INFO:tasks.workunit.client.1.smithi186.stdout:4/636: mknod d0/dc/d1f/d73/cd0 0 2024-04-12T20:56:35.595 INFO:tasks.workunit.client.1.smithi186.stdout:1/535: write d14/d1a/d33/d36/d5b/f2d [3054020,79722] 0 2024-04-12T20:56:35.596 INFO:tasks.workunit.client.1.smithi186.stdout:4/637: dread - d0/d59/dc0/fad zero size 2024-04-12T20:56:35.597 INFO:tasks.workunit.client.1.smithi186.stdout:9/492: creat d3/d2f/d54/d9f/fa5 x:0 0 0 2024-04-12T20:56:35.598 INFO:tasks.workunit.client.1.smithi186.stdout:6/537: mknod d9/d5e/d7a/d81/d96/cc0 0 2024-04-12T20:56:35.604 INFO:tasks.workunit.client.1.smithi186.stdout:8/502: mknod d7/d18/d36/d20/d23/d25/d2e/ca1 0 2024-04-12T20:56:35.606 INFO:tasks.workunit.client.1.smithi186.stdout:0/607: creat d0/d5c/fc6 x:0 0 0 2024-04-12T20:56:35.608 INFO:tasks.workunit.client.1.smithi186.stdout:8/503: write d7/d18/d36/d20/d8c/f92 [324077,128079] 0 2024-04-12T20:56:35.608 INFO:tasks.workunit.client.1.smithi186.stdout:7/568: getdents d7/d10/d2d 0 2024-04-12T20:56:35.615 INFO:tasks.workunit.client.1.smithi186.stdout:9/493: chown d3/d2f/d2e/d33/d36/d70/f7c 857272458 1 2024-04-12T20:56:35.615 INFO:tasks.workunit.client.1.smithi186.stdout:6/538: dwrite d9/f4b [0,4194304] 0 2024-04-12T20:56:35.617 INFO:tasks.workunit.client.1.smithi186.stdout:1/536: dwrite d14/d1a/d33/d36/d5b/d3e/d8b/f8e [0,4194304] 0 2024-04-12T20:56:35.619 INFO:tasks.workunit.client.1.smithi186.stdout:6/539: write d9/d43/d50/f9e [873878,80922] 0 2024-04-12T20:56:35.621 INFO:tasks.workunit.client.1.smithi186.stdout:0/608: mknod d0/d5d/cc7 0 2024-04-12T20:56:35.622 INFO:tasks.workunit.client.1.smithi186.stdout:6/540: dread - d9/d4c/fb6 zero size 2024-04-12T20:56:35.624 INFO:tasks.workunit.client.1.smithi186.stdout:7/569: creat d7/d10/d2d/fcf x:0 0 0 2024-04-12T20:56:35.626 INFO:tasks.workunit.client.1.smithi186.stdout:2/576: link d0/d3/d7a/d8d/ca6 d0/d3/d7a/cac 0 2024-04-12T20:56:35.636 INFO:tasks.workunit.client.1.smithi186.stdout:8/504: dread d7/de/d7a/f4f [0,4194304] 0 2024-04-12T20:56:35.639 INFO:tasks.workunit.client.1.smithi186.stdout:9/494: rename d3/c7 to d3/d2f/d2e/d33/ca6 0 2024-04-12T20:56:35.639 INFO:tasks.workunit.client.1.smithi186.stdout:6/541: creat d9/da9/fc1 x:0 0 0 2024-04-12T20:56:35.639 INFO:tasks.workunit.client.1.smithi186.stdout:8/505: readlink d7/de/d68/l88 0 2024-04-12T20:56:35.639 INFO:tasks.workunit.client.1.smithi186.stdout:5/549: dwrite d5/db/d15/d2a/d4c/d93/f98 [0,4194304] 0 2024-04-12T20:56:35.641 INFO:tasks.workunit.client.1.smithi186.stdout:9/495: chown d3/d2f/l2b 4164559 1 2024-04-12T20:56:35.644 INFO:tasks.workunit.client.1.smithi186.stdout:8/506: write d7/d18/d36/d20/d23/d25/d2e/f79 [359559,72321] 0 2024-04-12T20:56:35.652 INFO:tasks.workunit.client.1.smithi186.stdout:0/609: dwrite d0/d5c/f7f [0,4194304] 0 2024-04-12T20:56:35.660 INFO:tasks.workunit.client.1.smithi186.stdout:1/537: mkdir d14/d6c/da9 0 2024-04-12T20:56:35.662 INFO:tasks.workunit.client.1.smithi186.stdout:4/638: getdents d0/dc/d64/d87 0 2024-04-12T20:56:35.666 INFO:tasks.workunit.client.1.smithi186.stdout:6/542: unlink d9/c34 0 2024-04-12T20:56:35.669 INFO:tasks.workunit.client.1.smithi186.stdout:9/496: write d3/d2f/f6d [628467,67271] 0 2024-04-12T20:56:35.676 INFO:tasks.workunit.client.1.smithi186.stdout:2/577: creat d0/d3/d1a/d41/d8f/fad x:0 0 0 2024-04-12T20:56:35.677 INFO:tasks.workunit.client.1.smithi186.stdout:4/639: mknod d0/d59/dc0/d63/d9b/cd1 0 2024-04-12T20:56:35.679 INFO:tasks.workunit.client.1.smithi186.stdout:4/640: write d0/dc/d64/d87/d98/fcd [388416,104043] 0 2024-04-12T20:56:35.681 INFO:tasks.workunit.client.1.smithi186.stdout:1/538: dread - d14/d1a/d33/d36/d5b/f67 zero size 2024-04-12T20:56:35.683 INFO:tasks.workunit.client.1.smithi186.stdout:9/497: creat d3/d2f/d2e/d43/fa7 x:0 0 0 2024-04-12T20:56:35.685 INFO:tasks.workunit.client.1.smithi186.stdout:9/498: write d3/d2f/d54/d9f/fa5 [411407,28289] 0 2024-04-12T20:56:35.686 INFO:tasks.workunit.client.1.smithi186.stdout:7/570: unlink d7/d10/d2d/d40/d4a/d6c/d97/d4d/d84/c98 0 2024-04-12T20:56:35.687 INFO:tasks.workunit.client.1.smithi186.stdout:9/499: write d3/d2f/d54/f5b [1343885,54998] 0 2024-04-12T20:56:35.688 INFO:tasks.workunit.client.1.smithi186.stdout:5/550: creat d5/db/fbd x:0 0 0 2024-04-12T20:56:35.690 INFO:tasks.workunit.client.1.smithi186.stdout:9/500: fdatasync d3/d12/f86 0 2024-04-12T20:56:35.691 INFO:tasks.workunit.client.1.smithi186.stdout:6/543: mknod d9/d43/d59/cc2 0 2024-04-12T20:56:35.695 INFO:tasks.workunit.client.1.smithi186.stdout:2/578: dread d0/d3/f58 [0,4194304] 0 2024-04-12T20:56:35.696 INFO:tasks.workunit.client.1.smithi186.stdout:8/507: rename d7/d18/d36/d20/d23/d25/d2e to d7/d18/d36/d1d/d1f/da2 0 2024-04-12T20:56:35.699 INFO:tasks.workunit.client.1.smithi186.stdout:0/610: creat d0/d5c/fc8 x:0 0 0 2024-04-12T20:56:35.699 INFO:tasks.workunit.client.1.smithi186.stdout:6/544: dread d9/d43/d50/f9e [0,4194304] 0 2024-04-12T20:56:35.701 INFO:tasks.workunit.client.1.smithi186.stdout:0/611: chown d0/d14/d4e/fab 3197445 1 2024-04-12T20:56:35.706 INFO:tasks.workunit.client.1.smithi186.stdout:4/641: fdatasync d0/d21/f40 0 2024-04-12T20:56:35.711 INFO:tasks.workunit.client.1.smithi186.stdout:1/539: mknod d14/d1a/d33/d36/d5b/d3e/d71/d8c/caa 0 2024-04-12T20:56:35.714 INFO:tasks.workunit.client.1.smithi186.stdout:2/579: fsync d0/dc/fa8 0 2024-04-12T20:56:35.714 INFO:tasks.workunit.client.1.smithi186.stdout:9/501: dread d3/d2f/d2e/d33/d36/f53 [0,4194304] 0 2024-04-12T20:56:35.715 INFO:tasks.workunit.client.1.smithi186.stdout:3/524: sync 2024-04-12T20:56:35.715 INFO:tasks.workunit.client.1.smithi186.stdout:7/571: mkdir d7/d10/d2d/d40/d4a/d6c/d97/dd0 0 2024-04-12T20:56:35.719 INFO:tasks.workunit.client.1.smithi186.stdout:4/642: mkdir d0/dc/d64/d87/dd2 0 2024-04-12T20:56:35.720 INFO:tasks.workunit.client.1.smithi186.stdout:2/580: stat d0/d32/d8b/l95 0 2024-04-12T20:56:35.721 INFO:tasks.workunit.client.1.smithi186.stdout:3/525: dread - d1/d3/d21/d22/d34/f66 zero size 2024-04-12T20:56:35.722 INFO:tasks.workunit.client.1.smithi186.stdout:0/612: dwrite d0/d10/f8f [0,4194304] 0 2024-04-12T20:56:35.726 INFO:tasks.workunit.client.1.smithi186.stdout:8/508: link d7/de/d7a/f80 d7/d18/d36/d1d/fa3 0 2024-04-12T20:56:35.734 INFO:tasks.workunit.client.1.smithi186.stdout:2/581: write d0/d32/d8b/f8e [205241,53011] 0 2024-04-12T20:56:35.735 INFO:tasks.workunit.client.1.smithi186.stdout:0/613: chown d0/d5d/c63 4515210 1 2024-04-12T20:56:35.741 INFO:tasks.workunit.client.1.smithi186.stdout:5/551: dwrite d5/db/d15/f28 [0,4194304] 0 2024-04-12T20:56:35.742 INFO:tasks.workunit.client.1.smithi186.stdout:0/614: fsync d0/d5c/fc6 0 2024-04-12T20:56:35.742 INFO:tasks.workunit.client.1.smithi186.stdout:6/545: dread d9/d43/f45 [0,4194304] 0 2024-04-12T20:56:35.744 INFO:tasks.workunit.client.1.smithi186.stdout:0/615: write d0/d10/f43 [5837178,80736] 0 2024-04-12T20:56:35.745 INFO:tasks.workunit.client.1.smithi186.stdout:1/540: dwrite d14/d1a/d33/f84 [0,4194304] 0 2024-04-12T20:56:35.745 INFO:tasks.workunit.client.1.smithi186.stdout:8/509: mkdir d7/d18/d36/d20/d23/d25/da4 0 2024-04-12T20:56:35.751 INFO:tasks.workunit.client.1.smithi186.stdout:2/582: symlink d0/d3/d1a/d27/lae 0 2024-04-12T20:56:35.753 INFO:tasks.workunit.client.1.smithi186.stdout:0/616: write d0/d10/f2a [3095456,3187] 0 2024-04-12T20:56:35.759 INFO:tasks.workunit.client.1.smithi186.stdout:6/546: read - d9/d4c/d6f/d97/fbc zero size 2024-04-12T20:56:35.761 INFO:tasks.workunit.client.1.smithi186.stdout:3/526: dwrite d1/d3/da/f70 [0,4194304] 0 2024-04-12T20:56:35.764 INFO:tasks.workunit.client.1.smithi186.stdout:0/617: chown d0/la 1008567 1 2024-04-12T20:56:35.765 INFO:tasks.workunit.client.1.smithi186.stdout:1/541: getdents d14 0 2024-04-12T20:56:35.766 INFO:tasks.workunit.client.1.smithi186.stdout:8/510: rename d7/d18/d36/d1d/d1f/da2/d67 to d7/d18/d36/d1d/d7d/da5 0 2024-04-12T20:56:35.771 INFO:tasks.workunit.client.1.smithi186.stdout:0/618: fsync d0/d14/d4e/d1e/f98 0 2024-04-12T20:56:35.775 INFO:tasks.workunit.client.1.smithi186.stdout:5/552: dread d5/db/d15/d2a/d2d/d46/d55/f3d [0,4194304] 0 2024-04-12T20:56:35.781 INFO:tasks.workunit.client.1.smithi186.stdout:7/572: truncate d7/d10/d2d/d40/d4a/f64 3525344 0 2024-04-12T20:56:35.781 INFO:tasks.workunit.client.1.smithi186.stdout:9/502: dread d3/ff [0,4194304] 0 2024-04-12T20:56:35.782 INFO:tasks.workunit.client.1.smithi186.stdout:3/527: readlink d1/d3/d21/d22/d34/l61 0 2024-04-12T20:56:35.782 INFO:tasks.workunit.client.1.smithi186.stdout:2/583: dread d0/d3/d1a/d27/f59 [0,4194304] 0 2024-04-12T20:56:35.782 INFO:tasks.workunit.client.1.smithi186.stdout:1/542: chown d14/d1a/d33/f6b 1312105 1 2024-04-12T20:56:35.786 INFO:tasks.workunit.client.1.smithi186.stdout:4/643: dread d0/d59/f8f [0,4194304] 0 2024-04-12T20:56:35.799 INFO:tasks.workunit.client.1.smithi186.stdout:4/644: symlink d0/d59/dc0/d63/da6/ld3 0 2024-04-12T20:56:35.801 INFO:tasks.workunit.client.1.smithi186.stdout:0/619: dwrite d0/d14/d4e/d1e/f98 [0,4194304] 0 2024-04-12T20:56:35.806 INFO:tasks.workunit.client.1.smithi186.stdout:5/553: dwrite d5/db/d15/f31 [0,4194304] 0 2024-04-12T20:56:35.807 INFO:tasks.workunit.client.1.smithi186.stdout:8/511: write d7/d18/d36/d20/d23/d25/d29/f6a [184073,82397] 0 2024-04-12T20:56:35.816 INFO:tasks.workunit.client.1.smithi186.stdout:9/503: dwrite d3/d2f/d2e/d43/fa7 [0,4194304] 0 2024-04-12T20:56:35.818 INFO:tasks.workunit.client.1.smithi186.stdout:4/645: symlink d0/d59/dc0/d63/da6/ld4 0 2024-04-12T20:56:35.821 INFO:tasks.workunit.client.1.smithi186.stdout:0/620: rename d0/d14/d4e/c9a to d0/d5c/d65/cc9 0 2024-04-12T20:56:35.823 INFO:tasks.workunit.client.1.smithi186.stdout:2/584: dwrite d0/d32/d8b/f8e [0,4194304] 0 2024-04-12T20:56:35.825 INFO:tasks.workunit.client.1.smithi186.stdout:5/554: mknod d5/db/d2e/cbe 0 2024-04-12T20:56:35.834 INFO:tasks.workunit.client.1.smithi186.stdout:2/585: stat d0/d3/d1a/d48 0 2024-04-12T20:56:35.834 INFO:tasks.workunit.client.1.smithi186.stdout:0/621: symlink d0/d10/lca 0 2024-04-12T20:56:35.834 INFO:tasks.workunit.client.1.smithi186.stdout:4/646: creat d0/db4/fd5 x:0 0 0 2024-04-12T20:56:35.837 INFO:tasks.workunit.client.1.smithi186.stdout:7/573: dread d7/d10/d2d/d40/d4a/d6c/d97/d4d/f4e [0,4194304] 0 2024-04-12T20:56:35.838 INFO:tasks.workunit.client.1.smithi186.stdout:0/622: chown d0/d5d/c63 16754746 1 2024-04-12T20:56:35.839 INFO:tasks.workunit.client.1.smithi186.stdout:1/543: dwrite d14/d1a/d58/d69/fa4 [0,4194304] 0 2024-04-12T20:56:35.840 INFO:tasks.workunit.client.1.smithi186.stdout:4/647: readlink d0/dc/d64/la0 0 2024-04-12T20:56:35.840 INFO:tasks.workunit.client.1.smithi186.stdout:3/528: dwrite d1/d3/d49/d4d/f71 [0,4194304] 0 2024-04-12T20:56:35.842 INFO:tasks.workunit.client.1.smithi186.stdout:0/623: fsync d0/d10/d32/d3f/f54 0 2024-04-12T20:56:35.848 INFO:tasks.workunit.client.1.smithi186.stdout:2/586: creat d0/d3/faf x:0 0 0 2024-04-12T20:56:35.852 INFO:tasks.workunit.client.1.smithi186.stdout:8/512: truncate d7/d18/f4c 178507 0 2024-04-12T20:56:35.857 INFO:tasks.workunit.client.1.smithi186.stdout:8/513: read d7/d18/d36/d1d/d1f/d26/f3f [599945,31935] 0 2024-04-12T20:56:35.858 INFO:tasks.workunit.client.1.smithi186.stdout:7/574: symlink d7/d10/d1c/d8c/ld1 0 2024-04-12T20:56:35.859 INFO:tasks.workunit.client.1.smithi186.stdout:9/504: dwrite d3/d2f/d2e/d33/d66/f57 [0,4194304] 0 2024-04-12T20:56:35.864 INFO:tasks.workunit.client.1.smithi186.stdout:9/505: truncate d3/d2f/d54/f5b 4759598 0 2024-04-12T20:56:35.871 INFO:tasks.workunit.client.1.smithi186.stdout:6/547: sync 2024-04-12T20:56:35.874 INFO:tasks.workunit.client.1.smithi186.stdout:6/548: chown d9/d4c/d6f/d89/faa 127 1 2024-04-12T20:56:35.876 INFO:tasks.workunit.client.1.smithi186.stdout:1/544: unlink d14/d1a/d21/d35/c8d 0 2024-04-12T20:56:35.878 INFO:tasks.workunit.client.1.smithi186.stdout:2/587: unlink d0/d3/d1a/d27/c6b 0 2024-04-12T20:56:35.885 INFO:tasks.workunit.client.1.smithi186.stdout:7/575: creat d7/d10/d2d/d40/d4a/d5c/fd2 x:0 0 0 2024-04-12T20:56:35.886 INFO:tasks.workunit.client.1.smithi186.stdout:9/506: truncate d3/d2f/d2e/d33/f49 884092 0 2024-04-12T20:56:35.887 INFO:tasks.workunit.client.1.smithi186.stdout:4/648: symlink d0/d21/ld6 0 2024-04-12T20:56:35.889 INFO:tasks.workunit.client.1.smithi186.stdout:4/649: write d0/f24 [4591911,109472] 0 2024-04-12T20:56:35.889 INFO:tasks.workunit.client.1.smithi186.stdout:0/624: dread d0/d10/f8d [0,4194304] 0 2024-04-12T20:56:35.890 INFO:tasks.workunit.client.1.smithi186.stdout:3/529: mknod d1/d3/cae 0 2024-04-12T20:56:35.892 INFO:tasks.workunit.client.1.smithi186.stdout:4/650: truncate d0/d26/f2b 1472608 0 2024-04-12T20:56:35.893 INFO:tasks.workunit.client.1.smithi186.stdout:4/651: stat d0/d59/dc0/d48/c77 0 2024-04-12T20:56:35.894 INFO:tasks.workunit.client.1.smithi186.stdout:1/545: symlink d14/d1a/d33/d36/lab 0 2024-04-12T20:56:35.895 INFO:tasks.workunit.client.1.smithi186.stdout:1/546: fdatasync d14/d1a/d21/f4b 0 2024-04-12T20:56:35.895 INFO:tasks.workunit.client.1.smithi186.stdout:5/555: rename d5/db/d15/d2a/d65 to d5/db/d15/d2a/d2d/d46/d55/dbf 0 2024-04-12T20:56:35.897 INFO:tasks.workunit.client.1.smithi186.stdout:1/547: write d14/f15 [2180723,7839] 0 2024-04-12T20:56:35.902 INFO:tasks.workunit.client.1.smithi186.stdout:9/507: creat d3/d2f/d2e/d33/d66/d77/fa8 x:0 0 0 2024-04-12T20:56:35.903 INFO:tasks.workunit.client.1.smithi186.stdout:0/625: fsync d0/d14/f19 0 2024-04-12T20:56:35.904 INFO:tasks.workunit.client.1.smithi186.stdout:0/626: chown d0/d10/d71/c8e 20020 1 2024-04-12T20:56:35.906 INFO:tasks.workunit.client.1.smithi186.stdout:0/627: stat d0/d10/c73 0 2024-04-12T20:56:35.907 INFO:tasks.workunit.client.1.smithi186.stdout:2/588: creat d0/dc/d2e/da9/fb0 x:0 0 0 2024-04-12T20:56:35.908 INFO:tasks.workunit.client.1.smithi186.stdout:2/589: stat d0/d3 0 2024-04-12T20:56:35.910 INFO:tasks.workunit.client.1.smithi186.stdout:7/576: dread d7/f16 [4194304,4194304] 0 2024-04-12T20:56:35.910 INFO:tasks.workunit.client.1.smithi186.stdout:6/549: rename d9/d43/d2d/d4d/l72 to d9/d5e/lc3 0 2024-04-12T20:56:35.910 INFO:tasks.workunit.client.1.smithi186.stdout:2/590: dread - d0/d32/f9b zero size 2024-04-12T20:56:35.914 INFO:tasks.workunit.client.1.smithi186.stdout:9/508: mknod d3/d2f/d54/ca9 0 2024-04-12T20:56:35.914 INFO:tasks.workunit.client.1.smithi186.stdout:2/591: write d0/d3/d1a/fa5 [512670,66584] 0 2024-04-12T20:56:35.916 INFO:tasks.workunit.client.1.smithi186.stdout:5/556: unlink d5/d44/c50 0 2024-04-12T20:56:35.916 INFO:tasks.workunit.client.1.smithi186.stdout:1/548: dwrite d14/d74/f9a [0,4194304] 0 2024-04-12T20:56:35.917 INFO:tasks.workunit.client.1.smithi186.stdout:2/592: stat d0/d3/d1a/d41/f5c 0 2024-04-12T20:56:35.918 INFO:tasks.workunit.client.1.smithi186.stdout:9/509: write d3/d12/f7f [946805,92814] 0 2024-04-12T20:56:35.918 INFO:tasks.workunit.client.1.smithi186.stdout:0/628: symlink d0/d10/d72/d79/lcb 0 2024-04-12T20:56:35.921 INFO:tasks.workunit.client.1.smithi186.stdout:3/530: creat d1/d3/d21/d22/d24/faf x:0 0 0 2024-04-12T20:56:35.923 INFO:tasks.workunit.client.1.smithi186.stdout:2/593: fsync d0/d3/d1a/d48/f84 0 2024-04-12T20:56:35.923 INFO:tasks.workunit.client.1.smithi186.stdout:7/577: unlink d7/d10/d1c/f52 0 2024-04-12T20:56:35.927 INFO:tasks.workunit.client.1.smithi186.stdout:8/514: dwrite d7/de/f21 [0,4194304] 0 2024-04-12T20:56:35.932 INFO:tasks.workunit.client.1.smithi186.stdout:4/652: link d0/d21/f31 d0/dc/d64/d87/dd2/fd7 0 2024-04-12T20:56:35.938 INFO:tasks.workunit.client.1.smithi186.stdout:4/653: truncate d0/dc/d1f/fa9 443603 0 2024-04-12T20:56:35.940 INFO:tasks.workunit.client.1.smithi186.stdout:1/549: mknod d14/d1a/d33/d36/d5b/d6a/da2/cac 0 2024-04-12T20:56:35.940 INFO:tasks.workunit.client.1.smithi186.stdout:6/550: symlink d9/d43/d59/d5d/d6e/d91/lc4 0 2024-04-12T20:56:35.946 INFO:tasks.workunit.client.1.smithi186.stdout:3/531: unlink d1/d2c/d54/d81/fa5 0 2024-04-12T20:56:35.948 INFO:tasks.workunit.client.1.smithi186.stdout:8/515: creat d7/de/d10/d7f/fa6 x:0 0 0 2024-04-12T20:56:35.948 INFO:tasks.workunit.client.1.smithi186.stdout:6/551: readlink d9/d43/d2d/d4d/l7c 0 2024-04-12T20:56:35.949 INFO:tasks.workunit.client.1.smithi186.stdout:7/578: dread d7/fa1 [0,4194304] 0 2024-04-12T20:56:35.949 INFO:tasks.workunit.client.1.smithi186.stdout:5/557: dwrite d5/db/d15/d2a/d2d/f9d [0,4194304] 0 2024-04-12T20:56:35.953 INFO:tasks.workunit.client.1.smithi186.stdout:5/558: stat d5/db/d33 0 2024-04-12T20:56:35.962 INFO:tasks.workunit.client.1.smithi186.stdout:0/629: dwrite d0/f6 [0,4194304] 0 2024-04-12T20:56:35.963 INFO:tasks.workunit.client.1.smithi186.stdout:3/532: rmdir d1/d3/d21 39 2024-04-12T20:56:35.968 INFO:tasks.workunit.client.1.smithi186.stdout:8/516: mkdir d7/de/d68/da7 0 2024-04-12T20:56:35.973 INFO:tasks.workunit.client.1.smithi186.stdout:3/533: mkdir d1/d3/d49/d4d/d6f/db0 0 2024-04-12T20:56:35.974 INFO:tasks.workunit.client.1.smithi186.stdout:4/654: rmdir d0/d26/db1 0 2024-04-12T20:56:35.978 INFO:tasks.workunit.client.1.smithi186.stdout:4/655: readlink d0/dc/d33/lb0 0 2024-04-12T20:56:35.979 INFO:tasks.workunit.client.1.smithi186.stdout:6/552: dread d9/d43/d59/d5d/f48 [0,4194304] 0 2024-04-12T20:56:35.980 INFO:tasks.workunit.client.1.smithi186.stdout:9/510: dread d3/d2f/d2e/d33/d36/f3d [0,4194304] 0 2024-04-12T20:56:35.982 INFO:tasks.workunit.client.1.smithi186.stdout:7/579: rename d7/f43 to d7/d10/d2d/d40/fd3 0 2024-04-12T20:56:35.986 INFO:tasks.workunit.client.1.smithi186.stdout:1/550: dwrite d14/d1a/d33/d36/d5b/d3e/d5f/f8f [0,4194304] 0 2024-04-12T20:56:35.986 INFO:tasks.workunit.client.1.smithi186.stdout:4/656: write d0/dc/f23 [4056426,25943] 0 2024-04-12T20:56:35.986 INFO:tasks.workunit.client.1.smithi186.stdout:3/534: creat d1/d3/d21/fb1 x:0 0 0 2024-04-12T20:56:35.988 INFO:tasks.workunit.client.1.smithi186.stdout:5/559: getdents d5/db/d15 0 2024-04-12T20:56:35.992 INFO:tasks.workunit.client.1.smithi186.stdout:7/580: unlink d7/d10/f41 0 2024-04-12T20:56:35.993 INFO:tasks.workunit.client.1.smithi186.stdout:9/511: fdatasync d3/d2f/d2e/d33/d36/f37 0 2024-04-12T20:56:35.994 INFO:tasks.workunit.client.1.smithi186.stdout:0/630: dwrite d0/d85/dc4/dc1/fc5 [0,4194304] 0 2024-04-12T20:56:35.998 INFO:tasks.workunit.client.1.smithi186.stdout:6/553: unlink d9/d43/d59/d5d/d6e/fa0 0 2024-04-12T20:56:36.002 INFO:tasks.workunit.client.1.smithi186.stdout:4/657: mkdir d0/db4/dd8 0 2024-04-12T20:56:36.003 INFO:tasks.workunit.client.1.smithi186.stdout:1/551: chown d14/d1a/d1d/c80 31045 1 2024-04-12T20:56:36.005 INFO:tasks.workunit.client.1.smithi186.stdout:2/594: sync 2024-04-12T20:56:36.009 INFO:tasks.workunit.client.1.smithi186.stdout:9/512: symlink d3/d12/d6f/d83/laa 0 2024-04-12T20:56:36.012 INFO:tasks.workunit.client.1.smithi186.stdout:2/595: write d0/d3/d1a/d24/f33 [3095895,81377] 0 2024-04-12T20:56:36.014 INFO:tasks.workunit.client.1.smithi186.stdout:8/517: rename d7/d18/d36/d1d/d1f/da2/f70 to d7/d18/d36/fa8 0 2024-04-12T20:56:36.018 INFO:tasks.workunit.client.1.smithi186.stdout:1/552: unlink d14/d1a/d21/l45 0 2024-04-12T20:56:36.021 INFO:tasks.workunit.client.1.smithi186.stdout:2/596: chown d0/d3/d1a/d27/la2 70204063 1 2024-04-12T20:56:36.021 INFO:tasks.workunit.client.1.smithi186.stdout:7/581: mknod d7/d10/d2d/d40/d4a/d6c/d97/cd4 0 2024-04-12T20:56:36.023 INFO:tasks.workunit.client.1.smithi186.stdout:6/554: dread d9/d43/f45 [4194304,4194304] 0 2024-04-12T20:56:36.025 INFO:tasks.workunit.client.1.smithi186.stdout:3/535: dwrite d1/d3/d21/d5c/f69 [8388608,4194304] 0 2024-04-12T20:56:36.026 INFO:tasks.workunit.client.1.smithi186.stdout:4/658: symlink d0/dc/d33/ld9 0 2024-04-12T20:56:36.026 INFO:tasks.workunit.client.1.smithi186.stdout:7/582: read d7/f25 [1576266,83465] 0 2024-04-12T20:56:36.028 INFO:tasks.workunit.client.1.smithi186.stdout:0/631: dwrite d0/d10/d71/fb1 [0,4194304] 0 2024-04-12T20:56:36.031 INFO:tasks.workunit.client.1.smithi186.stdout:8/518: unlink d7/de/d10/f51 0 2024-04-12T20:56:36.037 INFO:tasks.workunit.client.1.smithi186.stdout:0/632: write d0/d10/d32/f3d [2636981,13567] 0 2024-04-12T20:56:36.039 INFO:tasks.workunit.client.1.smithi186.stdout:1/553: dwrite fe [0,4194304] 0 2024-04-12T20:56:36.039 INFO:tasks.workunit.client.1.smithi186.stdout:9/513: symlink d3/d2f/d2e/d33/d66/d77/lab 0 2024-04-12T20:56:36.045 INFO:tasks.workunit.client.1.smithi186.stdout:9/514: chown d3/f1d 3019943 1 2024-04-12T20:56:36.045 INFO:tasks.workunit.client.1.smithi186.stdout:0/633: write d0/d10/f8f [2263724,80595] 0 2024-04-12T20:56:36.045 INFO:tasks.workunit.client.1.smithi186.stdout:2/597: unlink d0/d3/d1a/d21/d26/f3d 0 2024-04-12T20:56:36.045 INFO:tasks.workunit.client.1.smithi186.stdout:6/555: mkdir d9/d5e/d7a/d81/dc5 0 2024-04-12T20:56:36.054 INFO:tasks.workunit.client.1.smithi186.stdout:4/659: creat d0/dc/d1f/d73/fda x:0 0 0 2024-04-12T20:56:36.056 INFO:tasks.workunit.client.1.smithi186.stdout:5/560: getdents d5/db/d15/d2a/d4c/d93 0 2024-04-12T20:56:36.057 INFO:tasks.workunit.client.1.smithi186.stdout:4/660: stat d0/d59 0 2024-04-12T20:56:36.060 INFO:tasks.workunit.client.1.smithi186.stdout:3/536: dwrite d1/d3/da/f6c [4194304,4194304] 0 2024-04-12T20:56:36.071 INFO:tasks.workunit.client.1.smithi186.stdout:9/515: dwrite d3/f1d [0,4194304] 0 2024-04-12T20:56:36.072 INFO:tasks.workunit.client.1.smithi186.stdout:9/516: readlink l0 0 2024-04-12T20:56:36.073 INFO:tasks.workunit.client.1.smithi186.stdout:0/634: fsync d0/d10/f4d 0 2024-04-12T20:56:36.085 INFO:tasks.workunit.client.1.smithi186.stdout:1/554: mknod d14/d1a/cad 0 2024-04-12T20:56:36.085 INFO:tasks.workunit.client.1.smithi186.stdout:3/537: dwrite d1/d2c/f67 [0,4194304] 0 2024-04-12T20:56:36.087 INFO:tasks.workunit.client.1.smithi186.stdout:5/561: stat d5/db/l23 0 2024-04-12T20:56:36.092 INFO:tasks.workunit.client.1.smithi186.stdout:2/598: creat d0/d3e/fb1 x:0 0 0 2024-04-12T20:56:36.092 INFO:tasks.workunit.client.1.smithi186.stdout:4/661: mkdir d0/dc/d64/d87/d98/ddb 0 2024-04-12T20:56:36.105 INFO:tasks.workunit.client.1.smithi186.stdout:8/519: write d7/d18/d36/d1d/fa0 [610602,71710] 0 2024-04-12T20:56:36.109 INFO:tasks.workunit.client.1.smithi186.stdout:9/517: unlink d3/f19 0 2024-04-12T20:56:36.123 INFO:tasks.workunit.client.1.smithi186.stdout:3/538: write d1/fc [3892191,98198] 0 2024-04-12T20:56:36.124 INFO:tasks.workunit.client.1.smithi186.stdout:5/562: mkdir d5/db/d15/d2a/d4c/dc0 0 2024-04-12T20:56:36.126 INFO:tasks.workunit.client.1.smithi186.stdout:2/599: unlink d0/d32/d8b/l95 0 2024-04-12T20:56:36.130 INFO:tasks.workunit.client.1.smithi186.stdout:2/600: dread - d0/d3/d1a/d41/d8f/fad zero size 2024-04-12T20:56:36.131 INFO:tasks.workunit.client.1.smithi186.stdout:4/662: creat d0/dc/d88/d80/fdc x:0 0 0 2024-04-12T20:56:36.133 INFO:tasks.workunit.client.1.smithi186.stdout:2/601: readlink d0/l51 0 2024-04-12T20:56:36.137 INFO:tasks.workunit.client.1.smithi186.stdout:8/520: symlink d7/d18/d36/d20/d23/d25/d29/la9 0 2024-04-12T20:56:36.138 INFO:tasks.workunit.client.1.smithi186.stdout:0/635: dwrite d0/d14/d4e/d1e/fa3 [0,4194304] 0 2024-04-12T20:56:36.140 INFO:tasks.workunit.client.1.smithi186.stdout:0/636: fdatasync d0/f88 0 2024-04-12T20:56:36.143 INFO:tasks.workunit.client.1.smithi186.stdout:8/521: readlink d7/d18/d36/d20/d23/d25/d29/la9 0 2024-04-12T20:56:36.143 INFO:tasks.workunit.client.1.smithi186.stdout:6/556: rename d9/d43/d2d/d4d/c52 to d9/d43/d59/d5d/cc6 0 2024-04-12T20:56:36.144 INFO:tasks.workunit.client.1.smithi186.stdout:9/518: rename d3/d2f to d3/d2f/d2e/d33/d61/dac 22 2024-04-12T20:56:36.150 INFO:tasks.workunit.client.1.smithi186.stdout:7/583: link d7/l5e d7/d10/d2d/d40/d4a/d6c/d97/d4d/d8a/ld5 0 2024-04-12T20:56:36.152 INFO:tasks.workunit.client.1.smithi186.stdout:1/555: creat d14/d6c/da9/fae x:0 0 0 2024-04-12T20:56:36.157 INFO:tasks.workunit.client.1.smithi186.stdout:3/539: creat d1/d3/d21/fb2 x:0 0 0 2024-04-12T20:56:36.160 INFO:tasks.workunit.client.1.smithi186.stdout:4/663: mkdir d0/dc/d88/ddd 0 2024-04-12T20:56:36.164 INFO:tasks.workunit.client.1.smithi186.stdout:4/664: write d0/dc/d33/f8a [356961,117782] 0 2024-04-12T20:56:36.165 INFO:tasks.workunit.client.1.smithi186.stdout:8/522: dwrite d7/d18/d36/d20/f4a [0,4194304] 0 2024-04-12T20:56:36.168 INFO:tasks.workunit.client.1.smithi186.stdout:2/602: symlink d0/d3/d1a/d41/lb2 0 2024-04-12T20:56:36.168 INFO:tasks.workunit.client.1.smithi186.stdout:4/665: chown d0/d59/dc0/d63/d9b/fc4 171681 1 2024-04-12T20:56:36.179 INFO:tasks.workunit.client.1.smithi186.stdout:8/523: dread d7/de/d10/f19 [0,4194304] 0 2024-04-12T20:56:36.179 INFO:tasks.workunit.client.1.smithi186.stdout:7/584: symlink d7/d10/d2d/d40/d4a/d6c/d97/d4d/d84/ld6 0 2024-04-12T20:56:36.182 INFO:tasks.workunit.client.1.smithi186.stdout:9/519: mknod d3/d12/d6f/cad 0 2024-04-12T20:56:36.186 INFO:tasks.workunit.client.1.smithi186.stdout:0/637: dread d0/d10/d32/d3f/f64 [0,4194304] 0 2024-04-12T20:56:36.191 INFO:tasks.workunit.client.1.smithi186.stdout:0/638: fsync d0/d14/f19 0 2024-04-12T20:56:36.191 INFO:tasks.workunit.client.1.smithi186.stdout:3/540: rename d1/d3/d21/d5e/c72 to d1/d3/da/cb3 0 2024-04-12T20:56:36.192 INFO:tasks.workunit.client.1.smithi186.stdout:8/524: mknod d7/de/d10/caa 0 2024-04-12T20:56:36.195 INFO:tasks.workunit.client.1.smithi186.stdout:6/557: link d9/d43/d2d/d4d/l7c d9/d43/d2d/lc7 0 2024-04-12T20:56:36.197 INFO:tasks.workunit.client.1.smithi186.stdout:2/603: dwrite d0/dc/f40 [4194304,4194304] 0 2024-04-12T20:56:36.198 INFO:tasks.workunit.client.1.smithi186.stdout:9/520: unlink d3/d2f/d2e/d33/d36/c98 0 2024-04-12T20:56:36.198 INFO:tasks.workunit.client.1.smithi186.stdout:7/585: mknod d7/d10/d2d/d86/cd7 0 2024-04-12T20:56:36.199 INFO:tasks.workunit.client.1.smithi186.stdout:8/525: stat d7/d18/d36/d1d/d1f/da2/d5c/f99 0 2024-04-12T20:56:36.203 INFO:tasks.workunit.client.1.smithi186.stdout:9/521: stat d3/d2f/d2e/d33/f49 0 2024-04-12T20:56:36.203 INFO:tasks.workunit.client.1.smithi186.stdout:5/563: truncate d5/db/d71/f52 4132949 0 2024-04-12T20:56:36.211 INFO:tasks.workunit.client.1.smithi186.stdout:1/556: rename d14/d1a/d33/d36/d5b/d6a/da2/cac to d14/d1a/d33/d36/d5b/d3e/d71/d8c/caf 0 2024-04-12T20:56:36.212 INFO:tasks.workunit.client.1.smithi186.stdout:1/557: fdatasync d14/d1a/d33/f84 0 2024-04-12T20:56:36.212 INFO:tasks.workunit.client.1.smithi186.stdout:4/666: dread d0/dc/f2d [0,4194304] 0 2024-04-12T20:56:36.213 INFO:tasks.workunit.client.1.smithi186.stdout:3/541: creat d1/d3/d21/d22/d24/fb4 x:0 0 0 2024-04-12T20:56:36.214 INFO:tasks.workunit.client.1.smithi186.stdout:3/542: chown d1/d2c/c3f 15589811 1 2024-04-12T20:56:36.214 INFO:tasks.workunit.client.1.smithi186.stdout:6/558: creat d9/d4c/fc8 x:0 0 0 2024-04-12T20:56:36.215 INFO:tasks.workunit.client.1.smithi186.stdout:9/522: rmdir d3/d2f/d2e/d33 39 2024-04-12T20:56:36.217 INFO:tasks.workunit.client.1.smithi186.stdout:3/543: read d1/d3/da/f70 [1764642,83461] 0 2024-04-12T20:56:36.218 INFO:tasks.workunit.client.1.smithi186.stdout:2/604: write d0/d3/d1a/d21/d22/f6c [165605,66394] 0 2024-04-12T20:56:36.218 INFO:tasks.workunit.client.1.smithi186.stdout:9/523: stat d3/d2f/d2e/d43/d4c/d82/la3 0 2024-04-12T20:56:36.218 INFO:tasks.workunit.client.1.smithi186.stdout:8/526: mkdir d7/d18/d36/d1d/d1f/d26/dab 0 2024-04-12T20:56:36.222 INFO:tasks.workunit.client.1.smithi186.stdout:8/527: truncate d7/d18/d36/fa8 522711 0 2024-04-12T20:56:36.223 INFO:tasks.workunit.client.1.smithi186.stdout:1/558: unlink d14/d1a/d3c/c63 0 2024-04-12T20:56:36.225 INFO:tasks.workunit.client.1.smithi186.stdout:4/667: symlink d0/dc/d64/d87/d98/lde 0 2024-04-12T20:56:36.226 INFO:tasks.workunit.client.1.smithi186.stdout:8/528: write d7/d18/d36/d1d/d1f/d26/f95 [715912,21470] 0 2024-04-12T20:56:36.227 INFO:tasks.workunit.client.1.smithi186.stdout:5/564: fdatasync d5/db/d15/d2a/d2d/d46/d55/f37 0 2024-04-12T20:56:36.230 INFO:tasks.workunit.client.1.smithi186.stdout:3/544: fsync d1/d3/d21/d22/f7b 0 2024-04-12T20:56:36.232 INFO:tasks.workunit.client.1.smithi186.stdout:2/605: mkdir d0/d3/d1a/d24/db3 0 2024-04-12T20:56:36.233 INFO:tasks.workunit.client.1.smithi186.stdout:1/559: symlink d14/d1a/d3c/lb0 0 2024-04-12T20:56:36.237 INFO:tasks.workunit.client.1.smithi186.stdout:8/529: symlink d7/d18/d36/d20/d23/d25/d29/lac 0 2024-04-12T20:56:36.238 INFO:tasks.workunit.client.1.smithi186.stdout:2/606: symlink d0/d3/d1a/d41/d8f/lb4 0 2024-04-12T20:56:36.244 INFO:tasks.workunit.client.1.smithi186.stdout:2/607: write d0/d3/d1a/d21/d26/f9a [565227,56437] 0 2024-04-12T20:56:36.244 INFO:tasks.workunit.client.1.smithi186.stdout:8/530: unlink d7/d18/d8e/l74 0 2024-04-12T20:56:36.249 INFO:tasks.workunit.client.1.smithi186.stdout:1/560: fdatasync d14/d1a/d33/d36/d5b/f4d 0 2024-04-12T20:56:36.249 INFO:tasks.workunit.client.1.smithi186.stdout:6/559: dwrite d9/d5e/f71 [0,4194304] 0 2024-04-12T20:56:36.253 INFO:tasks.workunit.client.1.smithi186.stdout:6/560: fsync d9/d4c/d6f/f9a 0 2024-04-12T20:56:36.259 INFO:tasks.workunit.client.1.smithi186.stdout:5/565: dwrite d5/db/d15/d2a/d4c/d5b/d7a/d94/fa3 [0,4194304] 0 2024-04-12T20:56:36.260 INFO:tasks.workunit.client.1.smithi186.stdout:9/524: dwrite d3/d2f/d2e/d33/d66/d77/fa8 [0,4194304] 0 2024-04-12T20:56:36.262 INFO:tasks.workunit.client.1.smithi186.stdout:4/668: dwrite d0/d21/f32 [0,4194304] 0 2024-04-12T20:56:36.265 INFO:tasks.workunit.client.1.smithi186.stdout:4/669: chown d0/dc/d88/d80/fa5 31 1 2024-04-12T20:56:36.267 INFO:tasks.workunit.client.1.smithi186.stdout:9/525: rmdir d3/d2f/d54 39 2024-04-12T20:56:36.269 INFO:tasks.workunit.client.1.smithi186.stdout:6/561: rename d9/d43/d59/f83 to d9/d5e/d7a/d81/d96/fc9 0 2024-04-12T20:56:36.273 INFO:tasks.workunit.client.1.smithi186.stdout:4/670: write d0/dc/d33/f6b [2474969,47574] 0 2024-04-12T20:56:36.276 INFO:tasks.workunit.client.1.smithi186.stdout:6/562: write d9/d4c/fc8 [808465,122994] 0 2024-04-12T20:56:36.277 INFO:tasks.workunit.client.1.smithi186.stdout:8/531: dwrite d7/f59 [0,4194304] 0 2024-04-12T20:56:36.292 INFO:tasks.workunit.client.1.smithi186.stdout:8/532: rename d7/d18/d36/d20/d23/d25/da4 to d7/d18/d36/dad 0 2024-04-12T20:56:36.297 INFO:tasks.workunit.client.1.smithi186.stdout:5/566: dwrite d5/f59 [0,4194304] 0 2024-04-12T20:56:36.309 INFO:tasks.workunit.client.1.smithi186.stdout:5/567: creat d5/d44/fc1 x:0 0 0 2024-04-12T20:56:36.313 INFO:tasks.workunit.client.1.smithi186.stdout:2/608: dread d0/d3/d1a/d21/d26/f6a [0,4194304] 0 2024-04-12T20:56:36.313 INFO:tasks.workunit.client.1.smithi186.stdout:6/563: dwrite d9/d43/d50/f5b [0,4194304] 0 2024-04-12T20:56:36.315 INFO:tasks.workunit.client.1.smithi186.stdout:9/526: dwrite d3/d2f/f26 [0,4194304] 0 2024-04-12T20:56:36.328 INFO:tasks.workunit.client.1.smithi186.stdout:6/564: creat d9/d4c/d6f/d89/d9c/fca x:0 0 0 2024-04-12T20:56:36.330 INFO:tasks.workunit.client.1.smithi186.stdout:2/609: creat d0/dc/d2e/d73/fb5 x:0 0 0 2024-04-12T20:56:36.330 INFO:tasks.workunit.client.1.smithi186.stdout:4/671: dwrite d0/d59/dc0/d63/da6/dc6/f3a [0,4194304] 0 2024-04-12T20:56:36.331 INFO:tasks.workunit.client.1.smithi186.stdout:9/527: symlink d3/d2f/d2e/d33/d61/lae 0 2024-04-12T20:56:36.336 INFO:tasks.workunit.client.1.smithi186.stdout:5/568: rename d5/c25 to d5/db/d15/d2a/d2d/d48/da4/cc2 0 2024-04-12T20:56:36.338 INFO:tasks.workunit.client.1.smithi186.stdout:9/528: mkdir d3/d2f/d2e/d33/d36/d48/daf 0 2024-04-12T20:56:36.340 INFO:tasks.workunit.client.1.smithi186.stdout:8/533: dwrite d7/d18/d36/d1d/fa3 [0,4194304] 0 2024-04-12T20:56:36.341 INFO:tasks.workunit.client.1.smithi186.stdout:6/565: mknod d9/d43/d59/ccb 0 2024-04-12T20:56:36.343 INFO:tasks.workunit.client.1.smithi186.stdout:6/566: fdatasync d9/d5e/f71 0 2024-04-12T20:56:36.345 INFO:tasks.workunit.client.1.smithi186.stdout:5/569: creat d5/d44/fc3 x:0 0 0 2024-04-12T20:56:36.352 INFO:tasks.workunit.client.1.smithi186.stdout:0/639: sync 2024-04-12T20:56:36.354 INFO:tasks.workunit.client.1.smithi186.stdout:8/534: creat d7/d18/d36/d1d/d1f/d26/dab/fae x:0 0 0 2024-04-12T20:56:36.367 INFO:tasks.workunit.client.1.smithi186.stdout:4/672: write d0/dc/f2d [1344178,71123] 0 2024-04-12T20:56:36.369 INFO:tasks.workunit.client.1.smithi186.stdout:2/610: dwrite d0/d3e/fb1 [0,4194304] 0 2024-04-12T20:56:36.372 INFO:tasks.workunit.client.1.smithi186.stdout:8/535: write d7/d18/d36/d1d/d1f/da2/f79 [572438,85715] 0 2024-04-12T20:56:36.374 INFO:tasks.workunit.client.1.smithi186.stdout:6/567: dwrite d9/d5e/f71 [4194304,4194304] 0 2024-04-12T20:56:36.380 INFO:tasks.workunit.client.1.smithi186.stdout:2/611: chown d0/d3/d7a 67 1 2024-04-12T20:56:36.381 INFO:tasks.workunit.client.1.smithi186.stdout:4/673: chown d0/f4d 39 1 2024-04-12T20:56:36.383 INFO:tasks.workunit.client.1.smithi186.stdout:9/529: dwrite d3/d2f/d2e/d33/d66/d77/fa8 [4194304,4194304] 0 2024-04-12T20:56:36.389 INFO:tasks.workunit.client.1.smithi186.stdout:9/530: chown d3/f1d 239028962 1 2024-04-12T20:56:36.389 INFO:tasks.workunit.client.1.smithi186.stdout:8/536: creat d7/d18/d36/d1d/d1f/d26/faf x:0 0 0 2024-04-12T20:56:36.390 INFO:tasks.workunit.client.1.smithi186.stdout:2/612: symlink d0/lb6 0 2024-04-12T20:56:36.397 INFO:tasks.workunit.client.1.smithi186.stdout:5/570: dread d5/db/f40 [0,4194304] 0 2024-04-12T20:56:36.399 INFO:tasks.workunit.client.1.smithi186.stdout:5/571: chown d5/db/d71/f1a 1129978509 1 2024-04-12T20:56:36.401 INFO:tasks.workunit.client.1.smithi186.stdout:2/613: link d0/d32/d8b/f8e d0/d32/d2a/fb7 0 2024-04-12T20:56:36.402 INFO:tasks.workunit.client.1.smithi186.stdout:0/640: dwrite d0/d14/d4e/d1e/d84/f97 [0,4194304] 0 2024-04-12T20:56:36.407 INFO:tasks.workunit.client.1.smithi186.stdout:1/561: sync 2024-04-12T20:56:36.407 INFO:tasks.workunit.client.1.smithi186.stdout:7/586: sync 2024-04-12T20:56:36.407 INFO:tasks.workunit.client.1.smithi186.stdout:2/614: chown d0/d3/d1a/f9d 7172465 1 2024-04-12T20:56:36.409 INFO:tasks.workunit.client.1.smithi186.stdout:3/545: sync 2024-04-12T20:56:36.413 INFO:tasks.workunit.client.1.smithi186.stdout:7/587: write d7/d10/d2d/fcf [123737,100489] 0 2024-04-12T20:56:36.414 INFO:tasks.workunit.client.1.smithi186.stdout:3/546: dread - d1/d3/d49/d4d/d9c/fa7 zero size 2024-04-12T20:56:36.414 INFO:tasks.workunit.client.1.smithi186.stdout:5/572: rename d5/d44 to d5/db/d15/d2a/d6d/dbb/dc4 0 2024-04-12T20:56:36.417 INFO:tasks.workunit.client.1.smithi186.stdout:6/568: dwrite d9/d43/d59/d5d/d6e/d91/fb5 [0,4194304] 0 2024-04-12T20:56:36.417 INFO:tasks.workunit.client.1.smithi186.stdout:0/641: mknod d0/d14/d4e/d50/d6e/d96/ccc 0 2024-04-12T20:56:36.419 INFO:tasks.workunit.client.1.smithi186.stdout:7/588: readlink d7/d55/d71/l76 0 2024-04-12T20:56:36.419 INFO:tasks.workunit.client.1.smithi186.stdout:1/562: write d14/d1a/d21/f27 [1680790,82788] 0 2024-04-12T20:56:36.425 INFO:tasks.workunit.client.1.smithi186.stdout:3/547: unlink d1/d3/d49/l5b 0 2024-04-12T20:56:36.429 INFO:tasks.workunit.client.1.smithi186.stdout:0/642: creat d0/d14/d4e/d1e/d84/fcd x:0 0 0 2024-04-12T20:56:36.432 INFO:tasks.workunit.client.1.smithi186.stdout:8/537: dwrite d7/d18/d36/fa [0,4194304] 0 2024-04-12T20:56:36.433 INFO:tasks.workunit.client.1.smithi186.stdout:5/573: symlink d5/db/d15/d2a/d6d/d7f/daf/lc5 0 2024-04-12T20:56:36.434 INFO:tasks.workunit.client.1.smithi186.stdout:6/569: mknod d9/d43/d59/ccc 0 2024-04-12T20:56:36.437 INFO:tasks.workunit.client.1.smithi186.stdout:0/643: rmdir d0/d85/dc4 39 2024-04-12T20:56:36.441 INFO:tasks.workunit.client.1.smithi186.stdout:2/615: dwrite d0/d3/d7a/f94 [0,4194304] 0 2024-04-12T20:56:36.441 INFO:tasks.workunit.client.1.smithi186.stdout:5/574: read - d5/db/d15/d2a/d2d/d48/f7c zero size 2024-04-12T20:56:36.443 INFO:tasks.workunit.client.1.smithi186.stdout:3/548: mknod d1/d3/d49/d4d/da1/cb5 0 2024-04-12T20:56:36.447 INFO:tasks.workunit.client.1.smithi186.stdout:4/674: dread d0/f42 [4194304,4194304] 0 2024-04-12T20:56:36.447 INFO:tasks.workunit.client.1.smithi186.stdout:5/575: chown d5/d27/l8f 0 1 2024-04-12T20:56:36.450 INFO:tasks.workunit.client.1.smithi186.stdout:3/549: fdatasync d1/d3/d21/d5c/f77 0 2024-04-12T20:56:36.452 INFO:tasks.workunit.client.1.smithi186.stdout:2/616: stat d0/d3/d1a/d27/f59 0 2024-04-12T20:56:36.454 INFO:tasks.workunit.client.1.smithi186.stdout:6/570: creat d9/d4c/dbe/fcd x:0 0 0 2024-04-12T20:56:36.455 INFO:tasks.workunit.client.1.smithi186.stdout:2/617: dread - d0/d3/d1a/d21/f77 zero size 2024-04-12T20:56:36.456 INFO:tasks.workunit.client.1.smithi186.stdout:9/531: dwrite d3/d2f/d2e/d33/f3f [4194304,4194304] 0 2024-04-12T20:56:36.460 INFO:tasks.workunit.client.1.smithi186.stdout:6/571: write d9/d4c/f6c [6005165,59598] 0 2024-04-12T20:56:36.464 INFO:tasks.workunit.client.1.smithi186.stdout:1/563: getdents d14/d6c/da9 0 2024-04-12T20:56:36.467 INFO:tasks.workunit.client.1.smithi186.stdout:1/564: chown d14/d1a/d58/c5d 814961741 1 2024-04-12T20:56:36.470 INFO:tasks.workunit.client.1.smithi186.stdout:0/644: dwrite d0/f87 [0,4194304] 0 2024-04-12T20:56:36.474 INFO:tasks.workunit.client.1.smithi186.stdout:4/675: unlink d0/dc/d88/f6c 0 2024-04-12T20:56:36.478 INFO:tasks.workunit.client.1.smithi186.stdout:6/572: mkdir d9/d4c/d6f/d89/d9c/dce 0 2024-04-12T20:56:36.479 INFO:tasks.workunit.client.1.smithi186.stdout:1/565: mknod d14/d74/cb1 0 2024-04-12T20:56:36.484 INFO:tasks.workunit.client.1.smithi186.stdout:4/676: write d0/d21/f38 [2666593,111766] 0 2024-04-12T20:56:36.487 INFO:tasks.workunit.client.1.smithi186.stdout:3/550: rmdir d1/d3/d21/d5e 0 2024-04-12T20:56:36.490 INFO:tasks.workunit.client.1.smithi186.stdout:5/576: creat d5/db/d15/d2a/d2d/d46/d55/fc6 x:0 0 0 2024-04-12T20:56:36.491 INFO:tasks.workunit.client.1.smithi186.stdout:0/645: unlink d0/d14/d4e/l40 0 2024-04-12T20:56:36.492 INFO:tasks.workunit.client.1.smithi186.stdout:6/573: write d9/d4c/d6f/d89/d9c/fca [111572,28928] 0 2024-04-12T20:56:36.497 INFO:tasks.workunit.client.1.smithi186.stdout:7/589: sync 2024-04-12T20:56:36.498 INFO:tasks.workunit.client.1.smithi186.stdout:9/532: dwrite d3/d12/d6f/f8c [0,4194304] 0 2024-04-12T20:56:36.500 INFO:tasks.workunit.client.1.smithi186.stdout:6/574: unlink d9/d43/d50/l7d 0 2024-04-12T20:56:36.503 INFO:tasks.workunit.client.1.smithi186.stdout:4/677: getdents d0/d59/dc0/d63/d9b 0 2024-04-12T20:56:36.505 INFO:tasks.workunit.client.1.smithi186.stdout:5/577: truncate d5/db/d15/d2a/d2d/d48/f87 4510739 0 2024-04-12T20:56:36.509 INFO:tasks.workunit.client.1.smithi186.stdout:3/551: getdents d1/d3/d49 0 2024-04-12T20:56:36.511 INFO:tasks.workunit.client.1.smithi186.stdout:4/678: mkdir d0/d59/ddf 0 2024-04-12T20:56:36.513 INFO:tasks.workunit.client.1.smithi186.stdout:1/566: dwrite d14/d1a/d21/f27 [0,4194304] 0 2024-04-12T20:56:36.514 INFO:tasks.workunit.client.1.smithi186.stdout:7/590: write d7/d10/d1c/d8c/fb5 [4266338,63203] 0 2024-04-12T20:56:36.517 INFO:tasks.workunit.client.1.smithi186.stdout:8/538: truncate d7/de/d10/f3c 3407045 0 2024-04-12T20:56:36.519 INFO:tasks.workunit.client.1.smithi186.stdout:2/618: write d0/d3/d1a/f45 [562880,15984] 0 2024-04-12T20:56:36.521 INFO:tasks.workunit.client.1.smithi186.stdout:3/552: symlink d1/d3/d21/d22/d24/lb6 0 2024-04-12T20:56:36.525 INFO:tasks.workunit.client.1.smithi186.stdout:1/567: dread d14/d1a/f1b [0,4194304] 0 2024-04-12T20:56:36.532 INFO:tasks.workunit.client.1.smithi186.stdout:8/539: symlink d7/d18/d36/d1d/lb0 0 2024-04-12T20:56:36.532 INFO:tasks.workunit.client.1.smithi186.stdout:2/619: write d0/d32/d2a/f71 [566576,112039] 0 2024-04-12T20:56:36.541 INFO:tasks.workunit.client.1.smithi186.stdout:0/646: truncate d0/d10/d32/f3d 3916771 0 2024-04-12T20:56:36.543 INFO:tasks.workunit.client.1.smithi186.stdout:4/679: dread d0/d21/f32 [0,4194304] 0 2024-04-12T20:56:36.547 INFO:tasks.workunit.client.1.smithi186.stdout:0/647: write d0/d81/f83 [901287,9653] 0 2024-04-12T20:56:36.547 INFO:tasks.workunit.client.1.smithi186.stdout:1/568: symlink d14/d1a/d33/d36/d5b/d3e/d8b/lb2 0 2024-04-12T20:56:36.548 INFO:tasks.workunit.client.1.smithi186.stdout:2/620: mknod d0/d32/d2a/d91/cb8 0 2024-04-12T20:56:36.550 INFO:tasks.workunit.client.1.smithi186.stdout:5/578: write d5/db/f34 [675410,111179] 0 2024-04-12T20:56:36.554 INFO:tasks.workunit.client.1.smithi186.stdout:4/680: write d0/dc/d88/d80/fdc [707464,23530] 0 2024-04-12T20:56:36.558 INFO:tasks.workunit.client.1.smithi186.stdout:9/533: dread d3/d12/f86 [0,4194304] 0 2024-04-12T20:56:36.564 INFO:tasks.workunit.client.1.smithi186.stdout:3/553: dwrite d1/d3/da/f6c [0,4194304] 0 2024-04-12T20:56:36.565 INFO:tasks.workunit.client.1.smithi186.stdout:2/621: rename d0/f10 to d0/d3/d1a/d24/fb9 0 2024-04-12T20:56:36.568 INFO:tasks.workunit.client.1.smithi186.stdout:8/540: dwrite d7/d18/d36/d1d/d1f/da2/f79 [0,4194304] 0 2024-04-12T20:56:36.571 INFO:tasks.workunit.client.1.smithi186.stdout:6/575: dread d9/d43/d50/f55 [0,4194304] 0 2024-04-12T20:56:36.571 INFO:tasks.workunit.client.1.smithi186.stdout:5/579: creat d5/db/d15/d2a/d6d/dbb/dc4/d5d/fc7 x:0 0 0 2024-04-12T20:56:36.573 INFO:tasks.workunit.client.1.smithi186.stdout:2/622: truncate d0/d3/faf 331352 0 2024-04-12T20:56:36.583 INFO:tasks.workunit.client.1.smithi186.stdout:7/591: dwrite d7/d10/d2d/f58 [0,4194304] 0 2024-04-12T20:56:36.583 INFO:tasks.workunit.client.1.smithi186.stdout:6/576: fdatasync d9/d5e/f71 0 2024-04-12T20:56:36.585 INFO:tasks.workunit.client.1.smithi186.stdout:1/569: truncate d14/d1a/d33/d36/d5b/f4d 347105 0 2024-04-12T20:56:36.591 INFO:tasks.workunit.client.1.smithi186.stdout:2/623: unlink d0/d3/f1f 0 2024-04-12T20:56:36.592 INFO:tasks.workunit.client.1.smithi186.stdout:8/541: read d7/d18/d36/d20/d23/d25/d29/f2a [1354694,51269] 0 2024-04-12T20:56:36.597 INFO:tasks.workunit.client.1.smithi186.stdout:1/570: rename d14/d1a/d1d/f23 to d14/d1a/d58/d69/fb3 0 2024-04-12T20:56:36.597 INFO:tasks.workunit.client.1.smithi186.stdout:5/580: mknod d5/db/d15/d2a/d2d/d82/d84/cc8 0 2024-04-12T20:56:36.602 INFO:tasks.workunit.client.1.smithi186.stdout:2/624: write d0/dc/f4c [2612972,74527] 0 2024-04-12T20:56:36.602 INFO:tasks.workunit.client.1.smithi186.stdout:0/648: dwrite d0/d10/f43 [0,4194304] 0 2024-04-12T20:56:36.610 INFO:tasks.workunit.client.1.smithi186.stdout:2/625: write d0/f7e [290878,63488] 0 2024-04-12T20:56:36.613 INFO:tasks.workunit.client.1.smithi186.stdout:3/554: dwrite d1/d3/d49/d4d/d6f/f83 [0,4194304] 0 2024-04-12T20:56:36.614 INFO:tasks.workunit.client.1.smithi186.stdout:9/534: dwrite d3/d2f/d2e/d33/d66/f57 [0,4194304] 0 2024-04-12T20:56:36.618 INFO:tasks.workunit.client.1.smithi186.stdout:1/571: symlink d14/d1a/d33/d36/d5b/d3e/d8b/lb4 0 2024-04-12T20:56:36.621 INFO:tasks.workunit.client.1.smithi186.stdout:8/542: link d7/d18/d36/d20/d23/d25/l2d d7/de/d68/da7/lb1 0 2024-04-12T20:56:36.643 INFO:tasks.workunit.client.1.smithi186.stdout:3/555: dread d1/d2c/d54/d7c/fac [0,4194304] 0 2024-04-12T20:56:36.735 INFO:tasks.workunit.client.1.smithi186.stdout:5/581: dwrite d5/db/f5f [4194304,4194304] 0 2024-04-12T20:56:36.735 INFO:tasks.workunit.client.1.smithi186.stdout:1/572: write d14/d1a/d33/d36/d5b/d6a/da2/fa6 [398387,40125] 0 2024-04-12T20:56:36.735 INFO:tasks.workunit.client.1.smithi186.stdout:6/577: dread d9/d4c/f61 [0,4194304] 0 2024-04-12T20:56:36.735 INFO:tasks.workunit.client.1.smithi186.stdout:8/543: truncate d7/d18/d36/d1d/d1f/da2/f94 1006157 0 2024-04-12T20:56:36.735 INFO:tasks.workunit.client.1.smithi186.stdout:1/573: readlink d14/l4e 0 2024-04-12T20:56:36.735 INFO:tasks.workunit.client.1.smithi186.stdout:5/582: readlink d5/d27/l8f 0 2024-04-12T20:56:36.736 INFO:tasks.workunit.client.1.smithi186.stdout:9/535: rename d3/d12/d6f/f9c to d3/d2f/d2e/d43/d4c/fb0 0 2024-04-12T20:56:36.736 INFO:tasks.workunit.client.1.smithi186.stdout:8/544: readlink d7/d18/d36/d20/d23/d25/l6d 0 2024-04-12T20:56:36.736 INFO:tasks.workunit.client.1.smithi186.stdout:3/556: symlink d1/d3/d21/d22/d34/lb7 0 2024-04-12T20:56:36.736 INFO:tasks.workunit.client.1.smithi186.stdout:0/649: dwrite d0/d5c/d65/fa4 [0,4194304] 0 2024-04-12T20:56:36.736 INFO:tasks.workunit.client.1.smithi186.stdout:5/583: chown d5/f9 1508 1 2024-04-12T20:56:36.736 INFO:tasks.workunit.client.1.smithi186.stdout:2/626: dwrite d0/d3e/fb1 [0,4194304] 0 2024-04-12T20:56:36.736 INFO:tasks.workunit.client.1.smithi186.stdout:2/627: write d0/d3/f80 [3470067,17212] 0 2024-04-12T20:56:36.736 INFO:tasks.workunit.client.1.smithi186.stdout:9/536: mknod d3/d12/d6f/d83/d71/d72/cb1 0 2024-04-12T20:56:36.736 INFO:tasks.workunit.client.1.smithi186.stdout:2/628: mknod d0/d3/cba 0 2024-04-12T20:56:36.736 INFO:tasks.workunit.client.1.smithi186.stdout:4/681: dread d0/dc/f23 [0,4194304] 0 2024-04-12T20:56:36.736 INFO:tasks.workunit.client.1.smithi186.stdout:9/537: rename d3/d12/d6f/d83/laa to d3/d12/d6f/d83/d71/lb2 0 2024-04-12T20:56:36.736 INFO:tasks.workunit.client.1.smithi186.stdout:4/682: chown d0 37405990 1 2024-04-12T20:56:36.737 INFO:tasks.workunit.client.1.smithi186.stdout:2/629: rmdir d0/d3e 39 2024-04-12T20:56:36.737 INFO:tasks.workunit.client.1.smithi186.stdout:5/584: truncate d5/db/d15/d2a/f41 3166208 0 2024-04-12T20:56:36.737 INFO:tasks.workunit.client.1.smithi186.stdout:3/557: rename d1/d3/da/l68 to d1/d3/d21/d22/d24/d28/lb8 0 2024-04-12T20:56:36.737 INFO:tasks.workunit.client.1.smithi186.stdout:4/683: chown d0/dc/d64/d87/d98/fcd 36580 1 2024-04-12T20:56:36.737 INFO:tasks.workunit.client.1.smithi186.stdout:6/578: dwrite d9/d43/d59/d5d/d6e/d91/fb5 [0,4194304] 0 2024-04-12T20:56:36.737 INFO:tasks.workunit.client.1.smithi186.stdout:2/630: chown d0/d3/d1a/d21/d22 185859 1 2024-04-12T20:56:36.737 INFO:tasks.workunit.client.1.smithi186.stdout:9/538: mkdir d3/d2f/d2e/db3 0 2024-04-12T20:56:36.737 INFO:tasks.workunit.client.1.smithi186.stdout:0/650: dwrite d0/fb8 [0,4194304] 0 2024-04-12T20:56:36.737 INFO:tasks.workunit.client.1.smithi186.stdout:3/558: write d1/d3/d21/d5c/f77 [3729480,123227] 0 2024-04-12T20:56:36.737 INFO:tasks.workunit.client.1.smithi186.stdout:5/585: mknod d5/db/d15/d2a/d2d/d46/cc9 0 2024-04-12T20:56:36.737 INFO:tasks.workunit.client.1.smithi186.stdout:6/579: mknod d9/d4c/d6f/ccf 0 2024-04-12T20:56:36.737 INFO:tasks.workunit.client.1.smithi186.stdout:2/631: creat d0/d3/d1a/d41/fbb x:0 0 0 2024-04-12T20:56:36.737 INFO:tasks.workunit.client.1.smithi186.stdout:6/580: fdatasync d9/f2e 0 2024-04-12T20:56:36.738 INFO:tasks.workunit.client.1.smithi186.stdout:0/651: dread - d0/f70 zero size 2024-04-12T20:56:36.738 INFO:tasks.workunit.client.1.smithi186.stdout:9/539: unlink d3/d2f/f4a 0 2024-04-12T20:56:36.738 INFO:tasks.workunit.client.1.smithi186.stdout:5/586: fsync d5/db/d15/d2a/d6d/fb9 0 2024-04-12T20:56:36.738 INFO:tasks.workunit.client.1.smithi186.stdout:0/652: fsync d0/d14/d4e/d50/d6e/f9c 0 2024-04-12T20:56:36.738 INFO:tasks.workunit.client.1.smithi186.stdout:6/581: fsync d9/d5e/fb0 0 2024-04-12T20:56:36.738 INFO:tasks.workunit.client.1.smithi186.stdout:2/632: symlink d0/d3/d1a/d41/d8f/lbc 0 2024-04-12T20:56:36.738 INFO:tasks.workunit.client.1.smithi186.stdout:7/592: sync 2024-04-12T20:56:36.738 INFO:tasks.workunit.client.1.smithi186.stdout:9/540: mknod d3/d13/cb4 0 2024-04-12T20:56:36.738 INFO:tasks.workunit.client.1.smithi186.stdout:5/587: mkdir d5/db/d15/d2a/d6d/d7f/d86/dca 0 2024-04-12T20:56:36.738 INFO:tasks.workunit.client.1.smithi186.stdout:2/633: creat d0/d3/d7a/d8d/fbd x:0 0 0 2024-04-12T20:56:36.738 INFO:tasks.workunit.client.1.smithi186.stdout:0/653: chown d0/d5c/d65/cc9 114794860 1 2024-04-12T20:56:36.738 INFO:tasks.workunit.client.1.smithi186.stdout:9/541: mknod d3/d12/d6f/d83/cb5 0 2024-04-12T20:56:36.739 INFO:tasks.workunit.client.1.smithi186.stdout:2/634: readlink d0/d3e/l6d 0 2024-04-12T20:56:36.739 INFO:tasks.workunit.client.1.smithi186.stdout:5/588: readlink d5/db/d15/d6a/l79 0 2024-04-12T20:56:36.739 INFO:tasks.workunit.client.1.smithi186.stdout:0/654: creat d0/d14/d4e/d50/fce x:0 0 0 2024-04-12T20:56:36.739 INFO:tasks.workunit.client.1.smithi186.stdout:9/542: dread - d3/d12/d6f/f75 zero size 2024-04-12T20:56:36.739 INFO:tasks.workunit.client.1.smithi186.stdout:2/635: fsync d0/d3/d7a/f94 0 2024-04-12T20:56:36.739 INFO:tasks.workunit.client.1.smithi186.stdout:1/574: dwrite d14/d1a/d33/d36/d5b/f79 [0,4194304] 0 2024-04-12T20:56:36.739 INFO:tasks.workunit.client.1.smithi186.stdout:4/684: truncate d0/dc/d33/f6b 424694 0 2024-04-12T20:56:36.743 INFO:tasks.workunit.client.1.smithi186.stdout:5/589: write d5/db/d15/f31 [1791813,85636] 0 2024-04-12T20:56:36.743 INFO:tasks.workunit.client.1.smithi186.stdout:3/559: dwrite d1/d3/d21/d22/d24/d2f/f7f [4194304,4194304] 0 2024-04-12T20:56:36.747 INFO:tasks.workunit.client.1.smithi186.stdout:6/582: dwrite d9/d4c/d6f/fb2 [0,4194304] 0 2024-04-12T20:56:36.748 INFO:tasks.workunit.client.1.smithi186.stdout:0/655: creat d0/d5c/d66/fcf x:0 0 0 2024-04-12T20:56:36.748 INFO:tasks.workunit.client.1.smithi186.stdout:3/560: fsync d1/d3/d49/f59 0 2024-04-12T20:56:36.755 INFO:tasks.workunit.client.1.smithi186.stdout:9/543: mknod d3/d2f/d2e/d33/cb6 0 2024-04-12T20:56:36.756 INFO:tasks.workunit.client.1.smithi186.stdout:8/545: sync 2024-04-12T20:56:36.760 INFO:tasks.workunit.client.1.smithi186.stdout:4/685: read d0/dc/d33/f47 [622890,96220] 0 2024-04-12T20:56:36.761 INFO:tasks.workunit.client.1.smithi186.stdout:7/593: dwrite d7/d10/d2d/fcf [0,4194304] 0 2024-04-12T20:56:36.766 INFO:tasks.workunit.client.1.smithi186.stdout:1/575: dread d14/d1a/d1d/f94 [0,4194304] 0 2024-04-12T20:56:36.766 INFO:tasks.workunit.client.1.smithi186.stdout:5/590: getdents d5/db/d15/d2a/d2d/d77 0 2024-04-12T20:56:36.766 INFO:tasks.workunit.client.1.smithi186.stdout:2/636: getdents d0/d3/d1a/d21/d26/d86 0 2024-04-12T20:56:36.767 INFO:tasks.workunit.client.1.smithi186.stdout:6/583: symlink d9/d43/d59/d5d/ld0 0 2024-04-12T20:56:36.771 INFO:tasks.workunit.client.1.smithi186.stdout:0/656: link d0/d85/dc4/dc1/fc5 d0/d10/d72/d79/fd0 0 2024-04-12T20:56:36.772 INFO:tasks.workunit.client.1.smithi186.stdout:1/576: creat d14/d6c/fb5 x:0 0 0 2024-04-12T20:56:36.773 INFO:tasks.workunit.client.1.smithi186.stdout:2/637: write d0/d3/d1a/d21/d26/f2d [1392076,52137] 0 2024-04-12T20:56:36.774 INFO:tasks.workunit.client.1.smithi186.stdout:0/657: fdatasync d0/d5c/d65/fbe 0 2024-04-12T20:56:36.777 INFO:tasks.workunit.client.1.smithi186.stdout:1/577: write d14/d1a/d33/f6b [844296,86763] 0 2024-04-12T20:56:36.779 INFO:tasks.workunit.client.1.smithi186.stdout:0/658: dread - d0/d5c/d66/fcf zero size 2024-04-12T20:56:36.780 INFO:tasks.workunit.client.1.smithi186.stdout:6/584: write d9/d5e/f6d [2492064,89251] 0 2024-04-12T20:56:36.783 INFO:tasks.workunit.client.1.smithi186.stdout:8/546: link d7/c86 d7/de/d68/da7/cb2 0 2024-04-12T20:56:36.786 INFO:tasks.workunit.client.1.smithi186.stdout:4/686: rmdir d0/dc/d88/ddd 0 2024-04-12T20:56:36.787 INFO:tasks.workunit.client.1.smithi186.stdout:0/659: creat d0/d10/d32/dac/fd1 x:0 0 0 2024-04-12T20:56:36.789 INFO:tasks.workunit.client.1.smithi186.stdout:2/638: rename d0/dc/fa8 to d0/fbe 0 2024-04-12T20:56:36.791 INFO:tasks.workunit.client.1.smithi186.stdout:0/660: fsync d0/d5c/fc8 0 2024-04-12T20:56:36.791 INFO:tasks.workunit.client.1.smithi186.stdout:6/585: creat d9/da9/dbd/fd1 x:0 0 0 2024-04-12T20:56:36.792 INFO:tasks.workunit.client.1.smithi186.stdout:4/687: mknod d0/d59/dc0/d48/d61/ce0 0 2024-04-12T20:56:36.794 INFO:tasks.workunit.client.1.smithi186.stdout:2/639: chown d0/d3/d1a/d48/c6e 1 1 2024-04-12T20:56:36.794 INFO:tasks.workunit.client.1.smithi186.stdout:0/661: unlink d0/f57 0 2024-04-12T20:56:36.798 INFO:tasks.workunit.client.1.smithi186.stdout:9/544: write d3/d2f/f1e [1802609,63601] 0 2024-04-12T20:56:36.799 INFO:tasks.workunit.client.1.smithi186.stdout:0/662: chown d0/d85/l9f 10378 1 2024-04-12T20:56:36.801 INFO:tasks.workunit.client.1.smithi186.stdout:4/688: creat d0/dc/d33/fe1 x:0 0 0 2024-04-12T20:56:36.802 INFO:tasks.workunit.client.1.smithi186.stdout:3/561: dread d1/d3/d21/d22/d24/d2f/f7f [0,4194304] 0 2024-04-12T20:56:36.804 INFO:tasks.workunit.client.1.smithi186.stdout:5/591: truncate d5/d27/f60 279942 0 2024-04-12T20:56:36.806 INFO:tasks.workunit.client.1.smithi186.stdout:7/594: truncate d7/d10/d2d/d40/d4a/d6c/d97/d4d/f4e 1809216 0 2024-04-12T20:56:36.806 INFO:tasks.workunit.client.1.smithi186.stdout:3/562: write d1/d3/f7e [852134,129050] 0 2024-04-12T20:56:36.807 INFO:tasks.workunit.client.1.smithi186.stdout:9/545: dread - d3/d2f/d54/d92/f97 zero size 2024-04-12T20:56:36.810 INFO:tasks.workunit.client.1.smithi186.stdout:3/563: stat d1/d3/d49/d4d/d6f/laa 0 2024-04-12T20:56:36.812 INFO:tasks.workunit.client.1.smithi186.stdout:6/586: dwrite d9/d4c/fb6 [0,4194304] 0 2024-04-12T20:56:36.816 INFO:tasks.workunit.client.1.smithi186.stdout:2/640: truncate d0/d32/f6f 592540 0 2024-04-12T20:56:36.823 INFO:tasks.workunit.client.1.smithi186.stdout:1/578: dread d14/d1a/d1d/f29 [0,4194304] 0 2024-04-12T20:56:36.823 INFO:tasks.workunit.client.1.smithi186.stdout:8/547: dwrite d7/d18/d36/d1d/d7d/da5/f9c [0,4194304] 0 2024-04-12T20:56:36.826 INFO:tasks.workunit.client.1.smithi186.stdout:7/595: mknod d7/d55/cd8 0 2024-04-12T20:56:36.827 INFO:tasks.workunit.client.1.smithi186.stdout:5/592: fdatasync d5/db/d15/d2a/d6d/dbb/dc4/f67 0 2024-04-12T20:56:36.828 INFO:tasks.workunit.client.1.smithi186.stdout:9/546: unlink d3/d2f/d2e/d33/d66/d77/f8b 0 2024-04-12T20:56:36.829 INFO:tasks.workunit.client.1.smithi186.stdout:8/548: dread - d7/d18/d36/d1d/d1f/da2/d5c/f99 zero size 2024-04-12T20:56:36.832 INFO:tasks.workunit.client.1.smithi186.stdout:1/579: truncate d14/d1a/d33/d36/d5b/d6a/f99 590954 0 2024-04-12T20:56:36.834 INFO:tasks.workunit.client.1.smithi186.stdout:3/564: truncate d1/f8a 1323613 0 2024-04-12T20:56:36.835 INFO:tasks.workunit.client.1.smithi186.stdout:5/593: dread d5/db/d15/f39 [0,4194304] 0 2024-04-12T20:56:36.836 INFO:tasks.workunit.client.1.smithi186.stdout:2/641: unlink d0/d3/d1a/d41/l52 0 2024-04-12T20:56:36.845 INFO:tasks.workunit.client.1.smithi186.stdout:9/547: creat d3/d13/fb7 x:0 0 0 2024-04-12T20:56:36.848 INFO:tasks.workunit.client.1.smithi186.stdout:6/587: symlink d9/d4c/d6f/d89/d9c/dce/ld2 0 2024-04-12T20:56:36.849 INFO:tasks.workunit.client.1.smithi186.stdout:1/580: creat d14/d1a/d33/d36/fb6 x:0 0 0 2024-04-12T20:56:36.850 INFO:tasks.workunit.client.1.smithi186.stdout:1/581: chown d14/d1a/d33/d36/c90 13762400 1 2024-04-12T20:56:36.851 INFO:tasks.workunit.client.1.smithi186.stdout:3/565: mknod d1/d3/d21/d22/d24/d28/cb9 0 2024-04-12T20:56:36.852 INFO:tasks.workunit.client.1.smithi186.stdout:3/566: fsync d1/d2c/d48/f4f 0 2024-04-12T20:56:36.854 INFO:tasks.workunit.client.1.smithi186.stdout:0/663: getdents d0/d10/d72/d79 0 2024-04-12T20:56:36.857 INFO:tasks.workunit.client.1.smithi186.stdout:5/594: rename d5/db/d15/d2a/d6d/d7f/fa1 to d5/db/d15/d2a/d2d/d82/d84/fcb 0 2024-04-12T20:56:36.858 INFO:tasks.workunit.client.1.smithi186.stdout:5/595: chown d5/db/d33/c73 19036416 1 2024-04-12T20:56:36.861 INFO:tasks.workunit.client.1.smithi186.stdout:4/689: truncate d0/d21/f1b 4638786 0 2024-04-12T20:56:36.865 INFO:tasks.workunit.client.1.smithi186.stdout:2/642: dread d0/d3/d1a/d24/f34 [0,4194304] 0 2024-04-12T20:56:36.867 INFO:tasks.workunit.client.1.smithi186.stdout:2/643: write d0/d3/d1a/d41/fbb [351470,57421] 0 2024-04-12T20:56:36.868 INFO:tasks.workunit.client.1.smithi186.stdout:6/588: write d9/d43/f44 [1127047,96786] 0 2024-04-12T20:56:36.877 INFO:tasks.workunit.client.1.smithi186.stdout:3/567: dread d1/d3/d49/d4d/f71 [0,4194304] 0 2024-04-12T20:56:36.881 INFO:tasks.workunit.client.1.smithi186.stdout:7/596: dwrite d7/d10/d1c/f1f [0,4194304] 0 2024-04-12T20:56:36.884 INFO:tasks.workunit.client.1.smithi186.stdout:0/664: creat d0/d5c/d65/fd2 x:0 0 0 2024-04-12T20:56:36.891 INFO:tasks.workunit.client.1.smithi186.stdout:8/549: creat d7/fb3 x:0 0 0 2024-04-12T20:56:36.895 INFO:tasks.workunit.client.1.smithi186.stdout:2/644: write d0/d3/d1a/f45 [595548,67108] 0 2024-04-12T20:56:36.897 INFO:tasks.workunit.client.1.smithi186.stdout:6/589: rename d9/l14 to d9/d5e/d73/ld3 0 2024-04-12T20:56:36.901 INFO:tasks.workunit.client.1.smithi186.stdout:0/665: dwrite d0/f88 [0,4194304] 0 2024-04-12T20:56:36.901 INFO:tasks.workunit.client.1.smithi186.stdout:3/568: rmdir d1/d3/d21/d22/d34 39 2024-04-12T20:56:36.903 INFO:tasks.workunit.client.1.smithi186.stdout:5/596: unlink d5/db/l12 0 2024-04-12T20:56:36.907 INFO:tasks.workunit.client.1.smithi186.stdout:4/690: creat d0/d59/dc0/d63/da6/daf/fe2 x:0 0 0 2024-04-12T20:56:36.907 INFO:tasks.workunit.client.1.smithi186.stdout:9/548: link d3/d12/d6f/d83/cb5 d3/d2f/d2e/d33/d66/d6b/cb8 0 2024-04-12T20:56:36.914 INFO:tasks.workunit.client.1.smithi186.stdout:1/582: link d14/d1a/d33/d36/d5b/d6a/f99 d14/d1a/d58/fb7 0 2024-04-12T20:56:36.915 INFO:tasks.workunit.client.1.smithi186.stdout:6/590: rmdir d9/d4c/d6f/d97 39 2024-04-12T20:56:36.917 INFO:tasks.workunit.client.1.smithi186.stdout:7/597: creat d7/d10/d2d/d67/fd9 x:0 0 0 2024-04-12T20:56:36.920 INFO:tasks.workunit.client.1.smithi186.stdout:4/691: mkdir d0/d59/dc0/d63/da6/dc6/de3 0 2024-04-12T20:56:36.921 INFO:tasks.workunit.client.1.smithi186.stdout:9/549: chown d3/d2f/d2e/d43/d4c/f6e 183557 1 2024-04-12T20:56:36.921 INFO:tasks.workunit.client.1.smithi186.stdout:1/583: mkdir d14/d1a/d1d/db8 0 2024-04-12T20:56:36.923 INFO:tasks.workunit.client.1.smithi186.stdout:3/569: symlink d1/d3/d21/d22/da4/lba 0 2024-04-12T20:56:36.924 INFO:tasks.workunit.client.1.smithi186.stdout:9/550: stat d3/d2f/d54/d92/f97 0 2024-04-12T20:56:36.924 INFO:tasks.workunit.client.1.smithi186.stdout:8/550: dwrite d7/de/d10/d7f/f9f [0,4194304] 0 2024-04-12T20:56:36.926 INFO:tasks.workunit.client.1.smithi186.stdout:9/551: write d3/fb [4463252,18826] 0 2024-04-12T20:56:36.926 INFO:tasks.workunit.client.1.smithi186.stdout:2/645: dread d0/d3/f80 [0,4194304] 0 2024-04-12T20:56:36.927 INFO:tasks.workunit.client.1.smithi186.stdout:8/551: chown d7/de/d5e/c61 6144337 1 2024-04-12T20:56:36.929 INFO:tasks.workunit.client.1.smithi186.stdout:9/552: chown d3/d2f/f6d 0 1 2024-04-12T20:56:36.930 INFO:tasks.workunit.client.1.smithi186.stdout:4/692: symlink d0/dc/d88/d80/le4 0 2024-04-12T20:56:36.932 INFO:tasks.workunit.client.1.smithi186.stdout:2/646: truncate d0/d3/d1a/d41/fa7 861362 0 2024-04-12T20:56:36.932 INFO:tasks.workunit.client.1.smithi186.stdout:4/693: stat d0/dc/d33 0 2024-04-12T20:56:36.932 INFO:tasks.workunit.client.1.smithi186.stdout:9/553: write d3/d2f/d54/d92/f97 [174397,110172] 0 2024-04-12T20:56:36.934 INFO:tasks.workunit.client.1.smithi186.stdout:9/554: chown d3/d12/d6f/d83/d71 9 1 2024-04-12T20:56:36.938 INFO:tasks.workunit.client.1.smithi186.stdout:3/570: mkdir d1/d2c/d54/d7c/dbb 0 2024-04-12T20:56:36.942 INFO:tasks.workunit.client.1.smithi186.stdout:0/666: getdents d0/d10/d32/d3f 0 2024-04-12T20:56:36.946 INFO:tasks.workunit.client.1.smithi186.stdout:2/647: symlink d0/d3e/lbf 0 2024-04-12T20:56:36.946 INFO:tasks.workunit.client.1.smithi186.stdout:1/584: mkdir d14/d1a/d33/d36/d5b/db9 0 2024-04-12T20:56:36.947 INFO:tasks.workunit.client.1.smithi186.stdout:9/555: dread d3/d2f/d54/f5b [0,4194304] 0 2024-04-12T20:56:36.953 INFO:tasks.workunit.client.1.smithi186.stdout:6/591: link d9/d43/d59/d5d/ld0 d9/d43/d59/d5d/d6e/ld4 0 2024-04-12T20:56:36.956 INFO:tasks.workunit.client.1.smithi186.stdout:6/592: write d9/d4c/fc8 [1345623,40390] 0 2024-04-12T20:56:36.957 INFO:tasks.workunit.client.1.smithi186.stdout:6/593: stat d9/l2c 0 2024-04-12T20:56:36.958 INFO:tasks.workunit.client.1.smithi186.stdout:7/598: link d7/d10/d1c/d5d/fc3 d7/d10/d2d/d40/d4a/d5c/fda 0 2024-04-12T20:56:36.961 INFO:tasks.workunit.client.1.smithi186.stdout:0/667: dwrite d0/f6 [0,4194304] 0 2024-04-12T20:56:36.961 INFO:tasks.workunit.client.1.smithi186.stdout:6/594: write d9/d43/d59/d5d/d6e/fb7 [592308,74864] 0 2024-04-12T20:56:36.962 INFO:tasks.workunit.client.1.smithi186.stdout:8/552: creat d7/d18/d36/d1d/d7d/d87/fb4 x:0 0 0 2024-04-12T20:56:36.967 INFO:tasks.workunit.client.1.smithi186.stdout:2/648: rename d0/d3/d1a/f45 to d0/d3/d1a/d24/db3/fc0 0 2024-04-12T20:56:36.971 INFO:tasks.workunit.client.1.smithi186.stdout:8/553: chown d7/d18/d36/d20 42565452 1 2024-04-12T20:56:36.971 INFO:tasks.workunit.client.1.smithi186.stdout:3/571: dread d1/fc [0,4194304] 0 2024-04-12T20:56:36.975 INFO:tasks.workunit.client.1.smithi186.stdout:6/595: creat d9/d43/fd5 x:0 0 0 2024-04-12T20:56:36.976 INFO:tasks.workunit.client.1.smithi186.stdout:4/694: dread d0/d59/dc0/d63/da6/dc6/f5b [0,4194304] 0 2024-04-12T20:56:36.977 INFO:tasks.workunit.client.1.smithi186.stdout:0/668: rename d0/d5c/d66/da7/fad to d0/d85/dc4/dc1/fd3 0 2024-04-12T20:56:36.978 INFO:tasks.workunit.client.1.smithi186.stdout:2/649: write d0/d3/f58 [1346196,53387] 0 2024-04-12T20:56:36.983 INFO:tasks.workunit.client.1.smithi186.stdout:0/669: mknod d0/d5d/cd4 0 2024-04-12T20:56:36.984 INFO:tasks.workunit.client.1.smithi186.stdout:2/650: write d0/dc/d2e/f5f [952358,7583] 0 2024-04-12T20:56:36.986 INFO:tasks.workunit.client.1.smithi186.stdout:7/599: dread d7/d10/d2d/fcf [0,4194304] 0 2024-04-12T20:56:36.987 INFO:tasks.workunit.client.1.smithi186.stdout:4/695: unlink d0/dc/d1f/d73/cd0 0 2024-04-12T20:56:36.988 INFO:tasks.workunit.client.1.smithi186.stdout:2/651: read d0/d32/f7c [88641,78505] 0 2024-04-12T20:56:36.991 INFO:tasks.workunit.client.1.smithi186.stdout:4/696: chown d0/d59/dc0/d48/dce 286 1 2024-04-12T20:56:36.991 INFO:tasks.workunit.client.1.smithi186.stdout:8/554: dread d7/de/d10/d7f/f9f [0,4194304] 0 2024-04-12T20:56:36.992 INFO:tasks.workunit.client.1.smithi186.stdout:6/596: link d9/d5e/d73/f84 d9/d4c/d6f/d89/d9c/fd6 0 2024-04-12T20:56:36.993 INFO:tasks.workunit.client.1.smithi186.stdout:4/697: chown d0/d21/f40 7 1 2024-04-12T20:56:36.993 INFO:tasks.workunit.client.1.smithi186.stdout:9/556: dwrite d3/d2f/d2e/d33/d66/d6b/f6c [0,4194304] 0 2024-04-12T20:56:36.997 INFO:tasks.workunit.client.1.smithi186.stdout:3/572: write d1/d3/d21/d22/d24/f60 [623794,117293] 0 2024-04-12T20:56:37.001 INFO:tasks.workunit.client.1.smithi186.stdout:0/670: getdents d0/d10/d32 0 2024-04-12T20:56:37.002 INFO:tasks.workunit.client.1.smithi186.stdout:8/555: creat d7/d18/d36/d1d/d7d/fb5 x:0 0 0 2024-04-12T20:56:37.002 INFO:tasks.workunit.client.1.smithi186.stdout:4/698: mknod d0/d59/dc0/d63/ce5 0 2024-04-12T20:56:37.003 INFO:tasks.workunit.client.1.smithi186.stdout:6/597: dread d9/d43/d59/d5d/f48 [0,4194304] 0 2024-04-12T20:56:37.006 INFO:tasks.workunit.client.1.smithi186.stdout:0/671: creat d0/d14/d4e/d50/fd5 x:0 0 0 2024-04-12T20:56:37.009 INFO:tasks.workunit.client.1.smithi186.stdout:7/600: rename d7/d10/d2d/d40/d4a/f50 to d7/d10/d1c/fdb 0 2024-04-12T20:56:37.010 INFO:tasks.workunit.client.1.smithi186.stdout:4/699: creat d0/d59/fe6 x:0 0 0 2024-04-12T20:56:37.014 INFO:tasks.workunit.client.1.smithi186.stdout:0/672: fdatasync d0/d5c/d66/fcf 0 2024-04-12T20:56:37.017 INFO:tasks.workunit.client.1.smithi186.stdout:2/652: dwrite d0/d3/d1a/d21/d26/f9a [0,4194304] 0 2024-04-12T20:56:37.019 INFO:tasks.workunit.client.1.smithi186.stdout:7/601: creat d7/d10/d1c/d8c/fdc x:0 0 0 2024-04-12T20:56:37.019 INFO:tasks.workunit.client.1.smithi186.stdout:4/700: fsync d0/dc/f2f 0 2024-04-12T20:56:37.020 INFO:tasks.workunit.client.1.smithi186.stdout:6/598: dread d9/d43/d2d/f7e [0,4194304] 0 2024-04-12T20:56:37.024 INFO:tasks.workunit.client.1.smithi186.stdout:3/573: dread d1/f8a [0,4194304] 0 2024-04-12T20:56:37.025 INFO:tasks.workunit.client.1.smithi186.stdout:9/557: dwrite d3/d2f/d2e/d33/f3f [0,4194304] 0 2024-04-12T20:56:37.029 INFO:tasks.workunit.client.1.smithi186.stdout:2/653: symlink d0/d32/d2a/d91/lc1 0 2024-04-12T20:56:37.030 INFO:tasks.workunit.client.1.smithi186.stdout:3/574: readlink d1/d3/d21/d22/l43 0 2024-04-12T20:56:37.030 INFO:tasks.workunit.client.1.smithi186.stdout:4/701: creat d0/d59/dc0/d63/da6/dc6/fe7 x:0 0 0 2024-04-12T20:56:37.031 INFO:tasks.workunit.client.1.smithi186.stdout:6/599: chown d9/d4c/d6f/d97/la3 2 1 2024-04-12T20:56:37.033 INFO:tasks.workunit.client.1.smithi186.stdout:3/575: write d1/d3/d21/fb2 [489402,7033] 0 2024-04-12T20:56:37.033 INFO:tasks.workunit.client.1.smithi186.stdout:0/673: rename d0/d14/d4e/d1e/c3c to d0/d5c/d65/cd6 0 2024-04-12T20:56:37.035 INFO:tasks.workunit.client.1.smithi186.stdout:4/702: creat d0/d59/dc0/d48/d61/fe8 x:0 0 0 2024-04-12T20:56:37.036 INFO:tasks.workunit.client.1.smithi186.stdout:7/602: mknod d7/d10/d2d/d40/d4a/d6c/d97/d4d/cdd 0 2024-04-12T20:56:37.039 INFO:tasks.workunit.client.1.smithi186.stdout:9/558: creat d3/d2f/d2e/d33/fb9 x:0 0 0 2024-04-12T20:56:37.039 INFO:tasks.workunit.client.1.smithi186.stdout:5/597: sync 2024-04-12T20:56:37.041 INFO:tasks.workunit.client.1.smithi186.stdout:8/556: dread d7/de/f11 [0,4194304] 0 2024-04-12T20:56:37.042 INFO:tasks.workunit.client.1.smithi186.stdout:9/559: chown d3/d2f/f21 25183207 1 2024-04-12T20:56:37.043 INFO:tasks.workunit.client.1.smithi186.stdout:3/576: write d1/d3/da/f56 [2321335,61379] 0 2024-04-12T20:56:37.045 INFO:tasks.workunit.client.1.smithi186.stdout:7/603: mknod d7/d10/d2d/d40/d4a/cde 0 2024-04-12T20:56:37.045 INFO:tasks.workunit.client.1.smithi186.stdout:0/674: mknod d0/d85/dc4/dc1/cd7 0 2024-04-12T20:56:37.049 INFO:tasks.workunit.client.1.smithi186.stdout:7/604: write d7/d10/d2d/d40/d4a/f51 [118973,44159] 0 2024-04-12T20:56:37.051 INFO:tasks.workunit.client.1.smithi186.stdout:6/600: rename d9/d43/d59/c8a to d9/cd7 0 2024-04-12T20:56:37.051 INFO:tasks.workunit.client.1.smithi186.stdout:8/557: truncate d7/de/d10/f19 1840840 0 2024-04-12T20:56:37.053 INFO:tasks.workunit.client.1.smithi186.stdout:2/654: link d0/d3/d1a/d21/d26/l75 d0/dc/d2e/d73/lc2 0 2024-04-12T20:56:37.053 INFO:tasks.workunit.client.1.smithi186.stdout:9/560: truncate d3/d2f/d2e/d43/d4c/f58 599796 0 2024-04-12T20:56:37.057 INFO:tasks.workunit.client.1.smithi186.stdout:7/605: mkdir d7/d10/d2d/d40/d4a/d6c/ddf 0 2024-04-12T20:56:37.059 INFO:tasks.workunit.client.1.smithi186.stdout:2/655: symlink d0/d32/d69/lc3 0 2024-04-12T20:56:37.059 INFO:tasks.workunit.client.1.smithi186.stdout:6/601: dread d9/d43/d2d/f7e [0,4194304] 0 2024-04-12T20:56:37.061 INFO:tasks.workunit.client.1.smithi186.stdout:9/561: mkdir d3/d2f/d2e/d33/d61/dba 0 2024-04-12T20:56:37.063 INFO:tasks.workunit.client.1.smithi186.stdout:2/656: creat d0/dc/d2e/d73/fc4 x:0 0 0 2024-04-12T20:56:37.063 INFO:tasks.workunit.client.1.smithi186.stdout:0/675: dread d0/f88 [0,4194304] 0 2024-04-12T20:56:37.063 INFO:tasks.workunit.client.1.smithi186.stdout:1/585: sync 2024-04-12T20:56:37.064 INFO:tasks.workunit.client.1.smithi186.stdout:5/598: getdents d5/db/d33 0 2024-04-12T20:56:37.066 INFO:tasks.workunit.client.1.smithi186.stdout:8/558: link d7/de/d5e/f1c d7/d18/d36/d1d/fb6 0 2024-04-12T20:56:37.066 INFO:tasks.workunit.client.1.smithi186.stdout:9/562: write d3/d12/d6f/f75 [714300,26038] 0 2024-04-12T20:56:37.067 INFO:tasks.workunit.client.1.smithi186.stdout:3/577: dwrite d1/d3/da/f6c [0,4194304] 0 2024-04-12T20:56:37.072 INFO:tasks.workunit.client.1.smithi186.stdout:2/657: creat d0/fc5 x:0 0 0 2024-04-12T20:56:37.072 INFO:tasks.workunit.client.1.smithi186.stdout:4/703: dread d0/f4d [0,4194304] 0 2024-04-12T20:56:37.074 INFO:tasks.workunit.client.1.smithi186.stdout:8/559: symlink d7/d18/d36/d1d/d7d/lb7 0 2024-04-12T20:56:37.077 INFO:tasks.workunit.client.1.smithi186.stdout:9/563: mknod d3/cbb 0 2024-04-12T20:56:37.081 INFO:tasks.workunit.client.1.smithi186.stdout:1/586: creat d14/d1a/d33/d36/d5b/db9/fba x:0 0 0 2024-04-12T20:56:37.081 INFO:tasks.workunit.client.1.smithi186.stdout:4/704: dread d0/dc/d1f/fa9 [0,4194304] 0 2024-04-12T20:56:37.084 INFO:tasks.workunit.client.1.smithi186.stdout:8/560: chown d7/d18/d36/d20/d23/d25/c69 0 1 2024-04-12T20:56:37.086 INFO:tasks.workunit.client.1.smithi186.stdout:0/676: dwrite d0/d14/f19 [0,4194304] 0 2024-04-12T20:56:37.140 INFO:tasks.workunit.client.1.smithi186.stdout:2/658: dread d0/dc/d2e/f5f [0,4194304] 0 2024-04-12T20:56:37.141 INFO:tasks.workunit.client.1.smithi186.stdout:8/561: write d7/d18/d36/d20/f4a [5161468,36374] 0 2024-04-12T20:56:37.145 INFO:tasks.workunit.client.1.smithi186.stdout:4/705: creat d0/d21/fe9 x:0 0 0 2024-04-12T20:56:37.146 INFO:tasks.workunit.client.1.smithi186.stdout:8/562: readlink d7/d18/d8e/l97 0 2024-04-12T20:56:37.147 INFO:tasks.workunit.client.1.smithi186.stdout:2/659: creat d0/d3/d1a/d21/d26/fc6 x:0 0 0 2024-04-12T20:56:37.147 INFO:tasks.workunit.client.1.smithi186.stdout:0/677: read d0/d10/d32/d3f/f77 [694139,101530] 0 2024-04-12T20:56:37.148 INFO:tasks.workunit.client.1.smithi186.stdout:3/578: dwrite d1/d3/d49/d4d/f71 [0,4194304] 0 2024-04-12T20:56:37.153 INFO:tasks.workunit.client.1.smithi186.stdout:8/563: unlink d7/d18/d36/d20/d8c/f92 0 2024-04-12T20:56:37.154 INFO:tasks.workunit.client.1.smithi186.stdout:0/678: mkdir d0/d10/d32/dd8 0 2024-04-12T20:56:37.156 INFO:tasks.workunit.client.1.smithi186.stdout:4/706: mknod d0/dc/d64/d87/dd2/cea 0 2024-04-12T20:56:37.158 INFO:tasks.workunit.client.1.smithi186.stdout:4/707: fsync d0/d59/dc0/d63/da6/dc6/fe7 0 2024-04-12T20:56:37.160 INFO:tasks.workunit.client.1.smithi186.stdout:2/660: rename d0/d3/d1a/d24/f67 to d0/dc/d2e/da9/fc7 0 2024-04-12T20:56:37.160 INFO:tasks.workunit.client.1.smithi186.stdout:3/579: mkdir d1/d3/d21/d22/d4a/dbc 0 2024-04-12T20:56:37.161 INFO:tasks.workunit.client.1.smithi186.stdout:0/679: fdatasync d0/d10/d32/d3f/f78 0 2024-04-12T20:56:37.164 INFO:tasks.workunit.client.1.smithi186.stdout:2/661: read - d0/d3/d1a/d21/f77 zero size 2024-04-12T20:56:37.165 INFO:tasks.workunit.client.1.smithi186.stdout:3/580: chown d1/d3/d49/d4d/dad 91867990 1 2024-04-12T20:56:37.168 INFO:tasks.workunit.client.1.smithi186.stdout:2/662: write d0/d3/d1a/d21/d26/f9a [2094308,18642] 0 2024-04-12T20:56:37.171 INFO:tasks.workunit.client.1.smithi186.stdout:3/581: fsync d1/d3/d49/d4d/d6f/f83 0 2024-04-12T20:56:37.172 INFO:tasks.workunit.client.1.smithi186.stdout:4/708: creat d0/d59/dc0/d63/da6/feb x:0 0 0 2024-04-12T20:56:37.173 INFO:tasks.workunit.client.1.smithi186.stdout:0/680: mkdir d0/d85/dd9 0 2024-04-12T20:56:37.174 INFO:tasks.workunit.client.1.smithi186.stdout:5/599: dread d5/db/d15/d2a/d4c/d93/f98 [0,4194304] 0 2024-04-12T20:56:37.175 INFO:tasks.workunit.client.1.smithi186.stdout:1/587: dread d14/d1a/d33/d36/d5b/d3e/f7e [4194304,4194304] 0 2024-04-12T20:56:37.179 INFO:tasks.workunit.client.1.smithi186.stdout:0/681: readlink d0/d14/d4e/d1e/d84/lc3 0 2024-04-12T20:56:37.182 INFO:tasks.workunit.client.1.smithi186.stdout:4/709: fdatasync d0/dc/fe 0 2024-04-12T20:56:37.186 INFO:tasks.workunit.client.1.smithi186.stdout:8/564: dread d7/f59 [0,4194304] 0 2024-04-12T20:56:37.191 INFO:tasks.workunit.client.1.smithi186.stdout:8/565: write d7/d18/d36/d1d/d1f/f49 [1045171,15276] 0 2024-04-12T20:56:37.192 INFO:tasks.workunit.client.1.smithi186.stdout:2/663: dwrite d0/dc/f4c [0,4194304] 0 2024-04-12T20:56:37.195 INFO:tasks.workunit.client.1.smithi186.stdout:1/588: truncate f9 5350361 0 2024-04-12T20:56:37.195 INFO:tasks.workunit.client.1.smithi186.stdout:3/582: dwrite d1/d3/d21/d5c/f77 [0,4194304] 0 2024-04-12T20:56:37.197 INFO:tasks.workunit.client.1.smithi186.stdout:2/664: chown d0/d3/faf 1775 1 2024-04-12T20:56:37.198 INFO:tasks.workunit.client.1.smithi186.stdout:6/602: sync 2024-04-12T20:56:37.198 INFO:tasks.workunit.client.1.smithi186.stdout:7/606: sync 2024-04-12T20:56:37.205 INFO:tasks.workunit.client.1.smithi186.stdout:6/603: truncate d9/d43/d2d/f8b 1140190 0 2024-04-12T20:56:37.205 INFO:tasks.workunit.client.1.smithi186.stdout:5/600: dwrite d5/db/f11 [0,4194304] 0 2024-04-12T20:56:37.213 INFO:tasks.workunit.client.1.smithi186.stdout:5/601: chown d5/db/d15/c21 1686 1 2024-04-12T20:56:37.219 INFO:tasks.workunit.client.1.smithi186.stdout:4/710: symlink d0/db4/dd8/lec 0 2024-04-12T20:56:37.219 INFO:tasks.workunit.client.1.smithi186.stdout:3/583: symlink d1/d2c/d54/d81/lbd 0 2024-04-12T20:56:37.219 INFO:tasks.workunit.client.1.smithi186.stdout:2/665: creat d0/d3/fc8 x:0 0 0 2024-04-12T20:56:37.222 INFO:tasks.workunit.client.1.smithi186.stdout:2/666: write d0/d3/d1a/f87 [2065883,78634] 0 2024-04-12T20:56:37.222 INFO:tasks.workunit.client.1.smithi186.stdout:8/566: rename d7/d18/d36/d1d/d1f/d26/f3d to d7/d18/d36/d1d/d1f/d26/fb8 0 2024-04-12T20:56:37.224 INFO:tasks.workunit.client.1.smithi186.stdout:7/607: creat d7/d10/d2d/d40/d4a/d6c/d97/d4d/d8a/dad/fe0 x:0 0 0 2024-04-12T20:56:37.226 INFO:tasks.workunit.client.1.smithi186.stdout:0/682: getdents d0/d10/d72 0 2024-04-12T20:56:37.226 INFO:tasks.workunit.client.1.smithi186.stdout:5/602: dread - d5/db/fbd zero size 2024-04-12T20:56:37.230 INFO:tasks.workunit.client.1.smithi186.stdout:1/589: mkdir d14/d1a/d33/d36/d5b/d3e/dbb 0 2024-04-12T20:56:37.230 INFO:tasks.workunit.client.1.smithi186.stdout:0/683: write d0/d10/d32/d3f/f78 [1682726,130272] 0 2024-04-12T20:56:37.231 INFO:tasks.workunit.client.1.smithi186.stdout:3/584: fdatasync d1/d3/d49/d4d/f75 0 2024-04-12T20:56:37.232 INFO:tasks.workunit.client.1.smithi186.stdout:2/667: symlink d0/d3/d1a/d24/db3/lc9 0 2024-04-12T20:56:37.235 INFO:tasks.workunit.client.1.smithi186.stdout:0/684: chown d0/d14/d4e/d50/d6e/faf 8386 1 2024-04-12T20:56:37.238 INFO:tasks.workunit.client.1.smithi186.stdout:1/590: truncate d14/d1a/d33/d36/d5b/f2d 5903930 0 2024-04-12T20:56:37.238 INFO:tasks.workunit.client.1.smithi186.stdout:8/567: dread d7/d18/d36/d1d/d1f/da2/f79 [0,4194304] 0 2024-04-12T20:56:37.239 INFO:tasks.workunit.client.1.smithi186.stdout:1/591: chown d14/d1a/d1d/db8 2 1 2024-04-12T20:56:37.240 INFO:tasks.workunit.client.1.smithi186.stdout:7/608: rename d7/d55/d71/d99 to d7/de1 0 2024-04-12T20:56:37.243 INFO:tasks.workunit.client.1.smithi186.stdout:3/585: symlink d1/d2c/d54/d81/lbe 0 2024-04-12T20:56:37.243 INFO:tasks.workunit.client.1.smithi186.stdout:4/711: write d0/dc/d64/d87/dd2/fd7 [3808762,53877] 0 2024-04-12T20:56:37.244 INFO:tasks.workunit.client.1.smithi186.stdout:2/668: mknod d0/d3/d1a/d27/cca 0 2024-04-12T20:56:37.244 INFO:tasks.workunit.client.1.smithi186.stdout:7/609: truncate d7/d55/d71/dbe/fc6 523615 0 2024-04-12T20:56:37.245 INFO:tasks.workunit.client.1.smithi186.stdout:4/712: fdatasync d0/f69 0 2024-04-12T20:56:37.246 INFO:tasks.workunit.client.1.smithi186.stdout:0/685: mkdir d0/d10/dda 0 2024-04-12T20:56:37.246 INFO:tasks.workunit.client.1.smithi186.stdout:8/568: symlink d7/de/d68/lb9 0 2024-04-12T20:56:37.252 INFO:tasks.workunit.client.1.smithi186.stdout:4/713: dread - d0/d59/dc0/d63/da6/daf/fe2 zero size 2024-04-12T20:56:37.255 INFO:tasks.workunit.client.1.smithi186.stdout:7/610: write d7/d10/d1c/f20 [5185570,65144] 0 2024-04-12T20:56:37.256 INFO:tasks.workunit.client.1.smithi186.stdout:8/569: write d7/d18/d36/d1d/d1f/d26/fb8 [5473685,115372] 0 2024-04-12T20:56:37.259 INFO:tasks.workunit.client.1.smithi186.stdout:0/686: rename d0/d85/dc4/dc1/fc5 to d0/d10/d71/fdb 0 2024-04-12T20:56:37.262 INFO:tasks.workunit.client.1.smithi186.stdout:3/586: truncate d1/d3/d21/d22/f7b 1539188 0 2024-04-12T20:56:37.264 INFO:tasks.workunit.client.1.smithi186.stdout:5/603: dread d5/d27/f6b [0,4194304] 0 2024-04-12T20:56:37.265 INFO:tasks.workunit.client.1.smithi186.stdout:1/592: dwrite d14/d1a/d33/d36/d5b/f61 [0,4194304] 0 2024-04-12T20:56:37.270 INFO:tasks.workunit.client.1.smithi186.stdout:7/611: mkdir d7/d10/d2d/d40/d4a/d6c/d97/dc2/de2 0 2024-04-12T20:56:37.277 INFO:tasks.workunit.client.1.smithi186.stdout:2/669: dwrite d0/d32/d2a/fb7 [0,4194304] 0 2024-04-12T20:56:37.279 INFO:tasks.workunit.client.1.smithi186.stdout:4/714: rename d0/d21/lcc to d0/d59/dc0/led 0 2024-04-12T20:56:37.283 INFO:tasks.workunit.client.1.smithi186.stdout:5/604: creat d5/db/d2e/fcc x:0 0 0 2024-04-12T20:56:37.286 INFO:tasks.workunit.client.1.smithi186.stdout:1/593: dwrite d14/fa8 [0,4194304] 0 2024-04-12T20:56:37.296 INFO:tasks.workunit.client.1.smithi186.stdout:0/687: write d0/d14/f35 [364050,91082] 0 2024-04-12T20:56:37.300 INFO:tasks.workunit.client.1.smithi186.stdout:8/570: rename d7/d18/d36/d20/d23/d25/d29/lac to d7/d18/d36/d1d/d1f/d26/lba 0 2024-04-12T20:56:37.300 INFO:tasks.workunit.client.1.smithi186.stdout:5/605: dwrite d5/db/d33/f36 [0,4194304] 0 2024-04-12T20:56:37.301 INFO:tasks.workunit.client.1.smithi186.stdout:9/564: sync 2024-04-12T20:56:37.303 INFO:tasks.workunit.client.1.smithi186.stdout:1/594: mkdir d14/d1a/d21/d35/dbc 0 2024-04-12T20:56:37.305 INFO:tasks.workunit.client.1.smithi186.stdout:0/688: fsync d0/d10/d32/f3d 0 2024-04-12T20:56:37.306 INFO:tasks.workunit.client.1.smithi186.stdout:3/587: dread d1/d3/d21/d5c/f69 [4194304,4194304] 0 2024-04-12T20:56:37.309 INFO:tasks.workunit.client.1.smithi186.stdout:7/612: getdents d7/d10/d1c/d5d 0 2024-04-12T20:56:37.312 INFO:tasks.workunit.client.1.smithi186.stdout:1/595: write d14/d1a/d33/d36/d5b/d3e/d71/f82 [2903417,129220] 0 2024-04-12T20:56:37.321 INFO:tasks.workunit.client.1.smithi186.stdout:5/606: creat d5/db/d15/d2a/d6d/d7f/d80/fcd x:0 0 0 2024-04-12T20:56:37.322 INFO:tasks.workunit.client.1.smithi186.stdout:9/565: rename d3/d2f/d2e/d33/d61/dba to d3/d2f/d2e/d33/d61/dbc 0 2024-04-12T20:56:37.322 INFO:tasks.workunit.client.1.smithi186.stdout:4/715: write d0/d59/dc0/d63/f90 [283039,34428] 0 2024-04-12T20:56:37.323 INFO:tasks.workunit.client.1.smithi186.stdout:2/670: rename d0/d3/d1a/d21/d26/l7f to d0/d3/d1a/lcb 0 2024-04-12T20:56:37.324 INFO:tasks.workunit.client.1.smithi186.stdout:9/566: stat d3/d13/f18 0 2024-04-12T20:56:37.325 INFO:tasks.workunit.client.1.smithi186.stdout:6/604: sync 2024-04-12T20:56:37.326 INFO:tasks.workunit.client.1.smithi186.stdout:8/571: mknod d7/d18/d36/d20/d71/cbb 0 2024-04-12T20:56:37.328 INFO:tasks.workunit.client.1.smithi186.stdout:4/716: write d0/d59/dc0/d63/d9b/fc4 [858133,97484] 0 2024-04-12T20:56:37.328 INFO:tasks.workunit.client.1.smithi186.stdout:7/613: symlink d7/d10/d1c/d5d/db7/le3 0 2024-04-12T20:56:37.330 INFO:tasks.workunit.client.1.smithi186.stdout:0/689: rename d0/d85/dd9 to d0/d5d/ddc 0 2024-04-12T20:56:37.330 INFO:tasks.workunit.client.1.smithi186.stdout:7/614: fdatasync d7/d10/fbf 0 2024-04-12T20:56:37.331 INFO:tasks.workunit.client.1.smithi186.stdout:6/605: write d9/d43/d59/d5d/f53 [962779,108896] 0 2024-04-12T20:56:37.334 INFO:tasks.workunit.client.1.smithi186.stdout:3/588: dwrite d1/d3/d49/d4d/d9c/fa7 [0,4194304] 0 2024-04-12T20:56:37.335 INFO:tasks.workunit.client.1.smithi186.stdout:1/596: fdatasync f9 0 2024-04-12T20:56:37.336 INFO:tasks.workunit.client.1.smithi186.stdout:5/607: mknod d5/db/d15/d2a/d4c/d5b/d7a/cce 0 2024-04-12T20:56:37.339 INFO:tasks.workunit.client.1.smithi186.stdout:5/608: fsync d5/db/f11 0 2024-04-12T20:56:37.341 INFO:tasks.workunit.client.1.smithi186.stdout:9/567: creat d3/d2f/d2e/d43/fbd x:0 0 0 2024-04-12T20:56:37.344 INFO:tasks.workunit.client.1.smithi186.stdout:6/606: creat d9/d4c/d6f/d89/d9c/dce/fd8 x:0 0 0 2024-04-12T20:56:37.345 INFO:tasks.workunit.client.1.smithi186.stdout:9/568: fsync d3/d2f/f5e 0 2024-04-12T20:56:37.345 INFO:tasks.workunit.client.1.smithi186.stdout:7/615: rmdir d7/d10/d1c/d8c 39 2024-04-12T20:56:37.346 INFO:tasks.workunit.client.1.smithi186.stdout:2/671: dwrite d0/fc5 [0,4194304] 0 2024-04-12T20:56:37.350 INFO:tasks.workunit.client.1.smithi186.stdout:1/597: mkdir d14/d91/dbd 0 2024-04-12T20:56:37.363 INFO:tasks.workunit.client.1.smithi186.stdout:3/589: dread d1/d2c/d48/f57 [0,4194304] 0 2024-04-12T20:56:37.366 INFO:tasks.workunit.client.1.smithi186.stdout:8/572: truncate d7/de/d10/f19 834789 0 2024-04-12T20:56:37.370 INFO:tasks.workunit.client.1.smithi186.stdout:6/607: dwrite d9/d5e/d7a/f7f [0,4194304] 0 2024-04-12T20:56:37.370 INFO:tasks.workunit.client.1.smithi186.stdout:9/569: creat d3/d12/d6f/d83/fbe x:0 0 0 2024-04-12T20:56:37.371 INFO:tasks.workunit.client.1.smithi186.stdout:7/616: chown d7/d10/c26 338 1 2024-04-12T20:56:37.375 INFO:tasks.workunit.client.1.smithi186.stdout:8/573: chown d7/d18/d36/d1d/d1f/d26/c52 1 1 2024-04-12T20:56:37.376 INFO:tasks.workunit.client.1.smithi186.stdout:2/672: creat d0/d3/d1a/d24/db3/fcc x:0 0 0 2024-04-12T20:56:37.379 INFO:tasks.workunit.client.1.smithi186.stdout:4/717: creat d0/fee x:0 0 0 2024-04-12T20:56:37.381 INFO:tasks.workunit.client.1.smithi186.stdout:2/673: write d0/d3/d1a/d21/d26/f9a [3090553,114340] 0 2024-04-12T20:56:37.386 INFO:tasks.workunit.client.1.smithi186.stdout:5/609: getdents d5/db/d15/d2a/d4c/d93 0 2024-04-12T20:56:37.388 INFO:tasks.workunit.client.1.smithi186.stdout:1/598: rename d14/d1a/d21/l26 to d14/d1a/d33/d36/d5b/d6a/da2/lbe 0 2024-04-12T20:56:37.389 INFO:tasks.workunit.client.1.smithi186.stdout:7/617: stat d7/d10/d1c/d8c/fb5 0 2024-04-12T20:56:37.391 INFO:tasks.workunit.client.1.smithi186.stdout:8/574: creat d7/d18/d36/d1d/d7d/da5/fbc x:0 0 0 2024-04-12T20:56:37.391 INFO:tasks.workunit.client.1.smithi186.stdout:6/608: mkdir d9/d43/d59/d5d/d6e/d91/dd9 0 2024-04-12T20:56:37.391 INFO:tasks.workunit.client.1.smithi186.stdout:2/674: rmdir d0/d3/d1a/d21/d26 39 2024-04-12T20:56:37.392 INFO:tasks.workunit.client.1.smithi186.stdout:8/575: dread - d7/fb3 zero size 2024-04-12T20:56:37.393 INFO:tasks.workunit.client.1.smithi186.stdout:0/690: link d0/d14/la5 d0/d14/d4e/d50/d6e/ldd 0 2024-04-12T20:56:37.400 INFO:tasks.workunit.client.1.smithi186.stdout:6/609: unlink d9/d4c/dbe/fcd 0 2024-04-12T20:56:37.401 INFO:tasks.workunit.client.1.smithi186.stdout:8/576: stat d7/f90 0 2024-04-12T20:56:37.401 INFO:tasks.workunit.client.1.smithi186.stdout:9/570: link d3/d2f/d2e/d33/d36/f37 d3/d12/d6f/d83/d71/d72/fbf 0 2024-04-12T20:56:37.403 INFO:tasks.workunit.client.1.smithi186.stdout:1/599: creat d14/d1a/d21/d35/d59/fbf x:0 0 0 2024-04-12T20:56:37.409 INFO:tasks.workunit.client.1.smithi186.stdout:9/571: unlink d3/d12/c46 0 2024-04-12T20:56:37.410 INFO:tasks.workunit.client.1.smithi186.stdout:0/691: creat d0/d85/fde x:0 0 0 2024-04-12T20:56:37.412 INFO:tasks.workunit.client.1.smithi186.stdout:5/610: getdents d5/db/d15 0 2024-04-12T20:56:37.412 INFO:tasks.workunit.client.1.smithi186.stdout:7/618: getdents d7/d10/d2d/d86 0 2024-04-12T20:56:37.414 INFO:tasks.workunit.client.1.smithi186.stdout:9/572: chown d3/d2f/d2e/d33/d9d 242694 1 2024-04-12T20:56:37.421 INFO:tasks.workunit.client.1.smithi186.stdout:0/692: fdatasync d0/d10/d72/f7d 0 2024-04-12T20:56:37.422 INFO:tasks.workunit.client.1.smithi186.stdout:5/611: fdatasync d5/db/d2e/f64 0 2024-04-12T20:56:37.422 INFO:tasks.workunit.client.1.smithi186.stdout:8/577: dread d7/d18/f4c [0,4194304] 0 2024-04-12T20:56:37.422 INFO:tasks.workunit.client.1.smithi186.stdout:2/675: dread d0/d3/d1a/d24/f34 [0,4194304] 0 2024-04-12T20:56:37.422 INFO:tasks.workunit.client.1.smithi186.stdout:1/600: rename d14/d1a/d33/d36/fb6 to d14/d1a/d33/d36/d5b/d3e/d71/d8c/d95/fc0 0 2024-04-12T20:56:37.423 INFO:tasks.workunit.client.1.smithi186.stdout:4/718: dwrite d0/d21/f20 [4194304,4194304] 0 2024-04-12T20:56:37.425 INFO:tasks.workunit.client.1.smithi186.stdout:0/693: stat d0/d10/fa8 0 2024-04-12T20:56:37.427 INFO:tasks.workunit.client.1.smithi186.stdout:9/573: symlink d3/d2f/d54/d9f/lc0 0 2024-04-12T20:56:37.427 INFO:tasks.workunit.client.1.smithi186.stdout:5/612: write d5/db/d15/d2a/d4c/f62 [3023844,24068] 0 2024-04-12T20:56:37.429 INFO:tasks.workunit.client.1.smithi186.stdout:1/601: creat d14/d91/fc1 x:0 0 0 2024-04-12T20:56:37.430 INFO:tasks.workunit.client.1.smithi186.stdout:8/578: chown d7/d18/d36/d1d 380 1 2024-04-12T20:56:37.440 INFO:tasks.workunit.client.1.smithi186.stdout:2/676: mknod d0/dc/ccd 0 2024-04-12T20:56:37.443 INFO:tasks.workunit.client.1.smithi186.stdout:9/574: rename d3/d12/d6f/d83/f67 to d3/d2f/d2e/d33/d66/fc1 0 2024-04-12T20:56:37.445 INFO:tasks.workunit.client.1.smithi186.stdout:8/579: symlink d7/de/d10/d7f/lbd 0 2024-04-12T20:56:37.446 INFO:tasks.workunit.client.1.smithi186.stdout:2/677: mkdir d0/d3/dce 0 2024-04-12T20:56:37.448 INFO:tasks.workunit.client.1.smithi186.stdout:1/602: dread d14/d1a/d33/d36/d5b/f2d [4194304,4194304] 0 2024-04-12T20:56:37.448 INFO:tasks.workunit.client.1.smithi186.stdout:4/719: read d0/d21/f83 [3520419,89625] 0 2024-04-12T20:56:37.448 INFO:tasks.workunit.client.1.smithi186.stdout:9/575: mknod d3/d13/cc2 0 2024-04-12T20:56:37.453 INFO:tasks.workunit.client.1.smithi186.stdout:2/678: write d0/d3/d1a/fa5 [764937,97860] 0 2024-04-12T20:56:37.453 INFO:tasks.workunit.client.1.smithi186.stdout:1/603: unlink d14/d1a/c20 0 2024-04-12T20:56:37.456 INFO:tasks.workunit.client.1.smithi186.stdout:8/580: creat d7/d18/d36/d1d/d1f/da2/fbe x:0 0 0 2024-04-12T20:56:37.457 INFO:tasks.workunit.client.1.smithi186.stdout:9/576: write d3/f1d [4750646,28125] 0 2024-04-12T20:56:37.462 INFO:tasks.workunit.client.1.smithi186.stdout:4/720: getdents d0/dc/d64/d87/dd2 0 2024-04-12T20:56:37.467 INFO:tasks.workunit.client.1.smithi186.stdout:0/694: dwrite d0/d10/d32/f48 [0,4194304] 0 2024-04-12T20:56:37.473 INFO:tasks.workunit.client.1.smithi186.stdout:2/679: dread d0/d32/f7c [4194304,4194304] 0 2024-04-12T20:56:37.473 INFO:tasks.workunit.client.1.smithi186.stdout:4/721: mkdir d0/dc/d88/d80/def 0 2024-04-12T20:56:37.474 INFO:tasks.workunit.client.1.smithi186.stdout:1/604: creat d14/d1a/d33/d36/d5b/fc2 x:0 0 0 2024-04-12T20:56:37.474 INFO:tasks.workunit.client.1.smithi186.stdout:5/613: dwrite d5/db/f11 [0,4194304] 0 2024-04-12T20:56:37.475 INFO:tasks.workunit.client.1.smithi186.stdout:8/581: truncate d7/d18/d36/d1d/d1f/d26/faf 298086 0 2024-04-12T20:56:37.478 INFO:tasks.workunit.client.1.smithi186.stdout:0/695: rmdir d0/d85/dc4/dc1 39 2024-04-12T20:56:37.479 INFO:tasks.workunit.client.1.smithi186.stdout:2/680: creat d0/d32/fcf x:0 0 0 2024-04-12T20:56:37.481 INFO:tasks.workunit.client.1.smithi186.stdout:1/605: truncate d14/d1a/f5a 221176 0 2024-04-12T20:56:37.482 INFO:tasks.workunit.client.1.smithi186.stdout:4/722: creat d0/d59/ddf/ff0 x:0 0 0 2024-04-12T20:56:37.486 INFO:tasks.workunit.client.1.smithi186.stdout:5/614: creat d5/db/d15/d2a/d4c/d93/fcf x:0 0 0 2024-04-12T20:56:37.488 INFO:tasks.workunit.client.1.smithi186.stdout:8/582: getdents d7/de/d68/da7 0 2024-04-12T20:56:37.490 INFO:tasks.workunit.client.1.smithi186.stdout:5/615: dread - d5/db/d15/d2a/d2d/d46/d55/fc6 zero size 2024-04-12T20:56:37.490 INFO:tasks.workunit.client.1.smithi186.stdout:2/681: write d0/d32/d2a/f62 [307882,123331] 0 2024-04-12T20:56:37.496 INFO:tasks.workunit.client.1.smithi186.stdout:4/723: rename d0/dc/d64/d87/f97 to d0/dc/ff1 0 2024-04-12T20:56:37.497 INFO:tasks.workunit.client.1.smithi186.stdout:6/610: write d9/d5e/d7a/f7f [4947270,2476] 0 2024-04-12T20:56:37.498 INFO:tasks.workunit.client.1.smithi186.stdout:2/682: creat d0/d3/d1a/d27/fd0 x:0 0 0 2024-04-12T20:56:37.499 INFO:tasks.workunit.client.1.smithi186.stdout:7/619: dread d7/d10/f95 [0,4194304] 0 2024-04-12T20:56:37.504 INFO:tasks.workunit.client.1.smithi186.stdout:5/616: truncate d5/db/d15/d2a/d6d/dbb/dc4/fc3 199926 0 2024-04-12T20:56:37.504 INFO:tasks.workunit.client.1.smithi186.stdout:4/724: read d0/dc/d33/f68 [306691,20710] 0 2024-04-12T20:56:37.506 INFO:tasks.workunit.client.1.smithi186.stdout:5/617: chown d5/db/d15/d2a/d4c/d93/fb2 7 1 2024-04-12T20:56:37.507 INFO:tasks.workunit.client.1.smithi186.stdout:6/611: mknod d9/d43/d2d/d4d/cda 0 2024-04-12T20:56:37.508 INFO:tasks.workunit.client.1.smithi186.stdout:2/683: mknod d0/d3/d1a/d21/d22/cd1 0 2024-04-12T20:56:37.512 INFO:tasks.workunit.client.1.smithi186.stdout:1/606: rename d14/d1a/d33/d36/d5b/c4a to d14/d1a/d33/d36/d5b/d3e/d71/d8c/d95/cc3 0 2024-04-12T20:56:37.522 INFO:tasks.workunit.client.1.smithi186.stdout:6/612: mkdir d9/d5e/d7a/d81/d96/ddb 0 2024-04-12T20:56:37.525 INFO:tasks.workunit.client.1.smithi186.stdout:3/590: dwrite d1/d3/da/f70 [0,4194304] 0 2024-04-12T20:56:37.531 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:56:37 smithi186 ceph-mon[36597]: pgmap v51: 65 pgs: 65 active+clean; 3.0 GiB data, 10 GiB used, 526 GiB / 536 GiB avail; 55 MiB/s rd, 210 MiB/s wr, 439 op/s 2024-04-12T20:56:37.531 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:56:37 smithi186 ceph-mon[36597]: from='mgr.24359 172.21.15.186:0/1315906828' entity='mgr.smithi186.kqcfxk' 2024-04-12T20:56:37.531 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:56:37 smithi186 ceph-mon[36597]: from='mgr.24359 172.21.15.186:0/1315906828' entity='mgr.smithi186.kqcfxk' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi018.xjzwga", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-12T20:56:37.531 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:56:37 smithi186 ceph-mon[36597]: from='mgr.24359 172.21.15.186:0/1315906828' entity='mgr.smithi186.kqcfxk' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-12T20:56:37.531 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:56:37 smithi186 ceph-mon[36597]: from='mgr.24359 172.21.15.186:0/1315906828' entity='mgr.smithi186.kqcfxk' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-12T20:56:37.531 INFO:tasks.workunit.client.1.smithi186.stdout:1/607: creat d14/d1a/d21/d35/fc4 x:0 0 0 2024-04-12T20:56:37.534 INFO:tasks.workunit.client.1.smithi186.stdout:7/620: dwrite d7/d10/d2d/d40/d4a/d6c/d97/f78 [0,4194304] 0 2024-04-12T20:56:37.538 INFO:tasks.workunit.client.1.smithi186.stdout:8/583: dread d7/de/d7a/f31 [0,4194304] 0 2024-04-12T20:56:37.544 INFO:tasks.workunit.client.1.smithi186.stdout:7/621: read d7/d10/d2d/d40/d4a/fa0 [85175,87790] 0 2024-04-12T20:56:37.546 INFO:tasks.workunit.client.1.smithi186.stdout:7/622: write d7/d55/d71/dbe/fc6 [7694,3452] 0 2024-04-12T20:56:37.549 INFO:tasks.workunit.client.1.smithi186.stdout:0/696: dread d0/d10/d72/d79/fba [0,4194304] 0 2024-04-12T20:56:37.554 INFO:tasks.workunit.client.1.smithi186.stdout:2/684: rename d0/d3/d1a/d21/d26/l75 to d0/d3/ld2 0 2024-04-12T20:56:37.554 INFO:tasks.workunit.client.1.smithi186.stdout:1/608: unlink d14/d1a/d21/f57 0 2024-04-12T20:56:37.558 INFO:tasks.workunit.client.1.smithi186.stdout:8/584: symlink d7/de/d10/d7f/lbf 0 2024-04-12T20:56:37.561 INFO:tasks.workunit.client.1.smithi186.stdout:8/585: stat d7/d18/d36/c24 0 2024-04-12T20:56:37.562 INFO:tasks.workunit.client.1.smithi186.stdout:0/697: readlink d0/d14/d4e/d1e/l9b 0 2024-04-12T20:56:37.567 INFO:tasks.workunit.client.1.smithi186.stdout:7/623: fdatasync d7/d10/d2d/f44 0 2024-04-12T20:56:37.567 INFO:tasks.workunit.client.1.smithi186.stdout:6/613: dread d9/d5e/f71 [0,4194304] 0 2024-04-12T20:56:37.570 INFO:tasks.workunit.client.1.smithi186.stdout:2/685: mkdir d0/dd3 0 2024-04-12T20:56:37.570 INFO:tasks.workunit.client.1.smithi186.stdout:6/614: read d9/d43/d59/d5d/f53 [457596,31733] 0 2024-04-12T20:56:37.572 INFO:tasks.workunit.client.1.smithi186.stdout:0/698: rmdir d0/d10/d72/d79 39 2024-04-12T20:56:37.574 INFO:tasks.workunit.client.1.smithi186.stdout:1/609: dwrite fe [0,4194304] 0 2024-04-12T20:56:37.579 INFO:tasks.workunit.client.1.smithi186.stdout:2/686: write d0/d3/f80 [3172691,25068] 0 2024-04-12T20:56:37.580 INFO:tasks.workunit.client.1.smithi186.stdout:6/615: creat d9/d43/d59/d5d/d6e/d91/fdc x:0 0 0 2024-04-12T20:56:37.581 INFO:tasks.workunit.client.1.smithi186.stdout:0/699: creat d0/d14/d4e/d50/fdf x:0 0 0 2024-04-12T20:56:37.584 INFO:tasks.workunit.client.1.smithi186.stdout:6/616: mknod d9/d5e/cdd 0 2024-04-12T20:56:37.588 INFO:tasks.workunit.client.1.smithi186.stdout:0/700: chown d0/d10/d72/d79/cae 921690 1 2024-04-12T20:56:37.591 INFO:tasks.workunit.client.1.smithi186.stdout:2/687: creat d0/d3/d1a/d21/d26/d86/fd4 x:0 0 0 2024-04-12T20:56:37.593 INFO:tasks.workunit.client.1.smithi186.stdout:6/617: creat d9/d4c/d6f/d89/fde x:0 0 0 2024-04-12T20:56:37.595 INFO:tasks.workunit.client.1.smithi186.stdout:9/577: write d3/d2f/d54/f60 [852224,70268] 0 2024-04-12T20:56:37.595 INFO:tasks.workunit.client.1.smithi186.stdout:6/618: write d9/da9/dbd/fd1 [445690,80186] 0 2024-04-12T20:56:37.596 INFO:tasks.workunit.client.1.smithi186.stdout:2/688: stat d0/f72 0 2024-04-12T20:56:37.596 INFO:tasks.workunit.client.1.smithi186.stdout:0/701: rmdir d0/d85/dc4/dc1 39 2024-04-12T20:56:37.597 INFO:tasks.workunit.client.1.smithi186.stdout:9/578: write d3/f1d [4584939,54071] 0 2024-04-12T20:56:37.600 INFO:tasks.workunit.client.1.smithi186.stdout:0/702: truncate d0/d5c/d65/fbe 468591 0 2024-04-12T20:56:37.600 INFO:tasks.workunit.client.1.smithi186.stdout:2/689: fsync d0/d3/d1a/d24/db3/fc0 0 2024-04-12T20:56:37.601 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:56:37 smithi018 ceph-mon[28799]: pgmap v51: 65 pgs: 65 active+clean; 3.0 GiB data, 10 GiB used, 526 GiB / 536 GiB avail; 55 MiB/s rd, 210 MiB/s wr, 439 op/s 2024-04-12T20:56:37.601 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:56:37 smithi018 ceph-mon[28799]: from='mgr.24359 172.21.15.186:0/1315906828' entity='mgr.smithi186.kqcfxk' 2024-04-12T20:56:37.601 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:56:37 smithi018 ceph-mon[28799]: from='mgr.24359 172.21.15.186:0/1315906828' entity='mgr.smithi186.kqcfxk' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi018.xjzwga", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-12T20:56:37.601 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:56:37 smithi018 ceph-mon[28799]: from='mgr.24359 172.21.15.186:0/1315906828' entity='mgr.smithi186.kqcfxk' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-12T20:56:37.601 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:56:37 smithi018 ceph-mon[28799]: from='mgr.24359 172.21.15.186:0/1315906828' entity='mgr.smithi186.kqcfxk' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-12T20:56:37.602 INFO:tasks.workunit.client.1.smithi186.stdout:0/703: chown d0/d5c/fc8 47563 1 2024-04-12T20:56:37.606 INFO:tasks.workunit.client.1.smithi186.stdout:9/579: rename d3/d2f/d2e/d33/d36/d48 to d3/d2f/d2e/d33/d66/d77/dc3 0 2024-04-12T20:56:37.610 INFO:tasks.workunit.client.1.smithi186.stdout:0/704: creat d0/d5d/ddc/fe0 x:0 0 0 2024-04-12T20:56:37.612 INFO:tasks.workunit.client.1.smithi186.stdout:9/580: chown d3/l34 15 1 2024-04-12T20:56:37.615 INFO:tasks.workunit.client.1.smithi186.stdout:0/705: creat d0/d10/d32/dd8/fe1 x:0 0 0 2024-04-12T20:56:37.619 INFO:tasks.workunit.client.1.smithi186.stdout:2/690: dwrite d0/d3/d1a/d41/f44 [0,4194304] 0 2024-04-12T20:56:37.619 INFO:tasks.workunit.client.1.smithi186.stdout:0/706: mkdir d0/d10/d32/de2 0 2024-04-12T20:56:37.623 INFO:tasks.workunit.client.1.smithi186.stdout:5/618: truncate d5/db/d15/d2a/d4c/f62 1747202 0 2024-04-12T20:56:37.625 INFO:tasks.workunit.client.1.smithi186.stdout:0/707: symlink d0/d10/d72/le3 0 2024-04-12T20:56:37.625 INFO:tasks.workunit.client.1.smithi186.stdout:2/691: dread d0/d3/d1a/d48/f84 [0,4194304] 0 2024-04-12T20:56:37.626 INFO:tasks.workunit.client.1.smithi186.stdout:4/725: dwrite d0/d26/f4f [0,4194304] 0 2024-04-12T20:56:37.629 INFO:tasks.workunit.client.1.smithi186.stdout:0/708: symlink d0/d81/le4 0 2024-04-12T20:56:37.634 INFO:tasks.workunit.client.1.smithi186.stdout:2/692: chown d0/d3/d7a/d8d/ca6 9 1 2024-04-12T20:56:37.635 INFO:tasks.workunit.client.1.smithi186.stdout:0/709: symlink d0/d10/d32/d3f/le5 0 2024-04-12T20:56:37.637 INFO:tasks.workunit.client.1.smithi186.stdout:4/726: mkdir d0/dc/df2 0 2024-04-12T20:56:37.638 INFO:tasks.workunit.client.1.smithi186.stdout:4/727: fsync d0/db4/fd5 0 2024-04-12T20:56:37.639 INFO:tasks.workunit.client.1.smithi186.stdout:5/619: getdents d5/db 0 2024-04-12T20:56:37.646 INFO:tasks.workunit.client.1.smithi186.stdout:2/693: mknod d0/d3/cd5 0 2024-04-12T20:56:37.647 INFO:tasks.workunit.client.1.smithi186.stdout:5/620: mkdir d5/db/d2e/dd0 0 2024-04-12T20:56:37.647 INFO:tasks.workunit.client.1.smithi186.stdout:0/710: readlink d0/d14/d4e/d1e/l20 0 2024-04-12T20:56:37.647 INFO:tasks.workunit.client.1.smithi186.stdout:8/586: write d7/de/d5e/f1c [4706501,45153] 0 2024-04-12T20:56:37.649 INFO:tasks.workunit.client.1.smithi186.stdout:0/711: dread - d0/d14/d4e/d50/fce zero size 2024-04-12T20:56:37.651 INFO:tasks.workunit.client.1.smithi186.stdout:8/587: truncate d7/d18/d36/d1d/fa0 1232277 0 2024-04-12T20:56:37.651 INFO:tasks.workunit.client.1.smithi186.stdout:1/610: truncate d14/d1a/d1d/f29 559953 0 2024-04-12T20:56:37.664 INFO:tasks.workunit.client.1.smithi186.stdout:4/728: symlink d0/d21/lf3 0 2024-04-12T20:56:37.666 INFO:tasks.workunit.client.1.smithi186.stdout:2/694: creat d0/d3/d1a/d41/fd6 x:0 0 0 2024-04-12T20:56:37.671 INFO:tasks.workunit.client.1.smithi186.stdout:8/588: getdents d7/d18/d36/dad 0 2024-04-12T20:56:37.672 INFO:tasks.workunit.client.1.smithi186.stdout:1/611: mkdir d14/d91/dbd/dc5 0 2024-04-12T20:56:37.674 INFO:tasks.workunit.client.1.smithi186.stdout:5/621: creat d5/fd1 x:0 0 0 2024-04-12T20:56:37.675 INFO:tasks.workunit.client.1.smithi186.stdout:8/589: fdatasync d7/d18/d36/d1d/d1f/d26/fb8 0 2024-04-12T20:56:37.677 INFO:tasks.workunit.client.1.smithi186.stdout:1/612: write d14/d1a/d58/d69/fb3 [5394058,94079] 0 2024-04-12T20:56:37.681 INFO:tasks.workunit.client.1.smithi186.stdout:5/622: write d5/db/d15/d2a/d6d/dbb/dc4/fc3 [323374,1978] 0 2024-04-12T20:56:37.682 INFO:tasks.workunit.client.1.smithi186.stdout:8/590: creat d7/d18/d36/d20/d71/fc0 x:0 0 0 2024-04-12T20:56:37.682 INFO:tasks.workunit.client.1.smithi186.stdout:4/729: dwrite d0/d59/dc0/d63/da6/daf/fe2 [0,4194304] 0 2024-04-12T20:56:37.683 INFO:tasks.workunit.client.1.smithi186.stdout:1/613: chown d14/d1a/d33/d36/d5b/d3e/d8b/lb2 10889647 1 2024-04-12T20:56:37.685 INFO:tasks.workunit.client.1.smithi186.stdout:1/614: chown d14/d1a/d3c/c50 809498 1 2024-04-12T20:56:37.687 INFO:tasks.workunit.client.1.smithi186.stdout:5/623: fdatasync d5/db/d15/d2a/d2d/d46/d55/f3d 0 2024-04-12T20:56:37.687 INFO:tasks.workunit.client.1.smithi186.stdout:2/695: dwrite d0/d3/d1a/d41/d8f/fad [0,4194304] 0 2024-04-12T20:56:37.688 INFO:tasks.workunit.client.1.smithi186.stdout:4/730: read d0/dc/d1f/d73/fc2 [533213,42255] 0 2024-04-12T20:56:37.692 INFO:tasks.workunit.client.1.smithi186.stdout:2/696: symlink d0/d3/d7a/ld7 0 2024-04-12T20:56:37.693 INFO:tasks.workunit.client.1.smithi186.stdout:4/731: mkdir d0/db4/dd8/df4 0 2024-04-12T20:56:37.697 INFO:tasks.workunit.client.1.smithi186.stdout:5/624: symlink d5/db/d2e/dd0/ld2 0 2024-04-12T20:56:37.698 INFO:tasks.workunit.client.1.smithi186.stdout:4/732: creat d0/d59/dc0/d63/ff5 x:0 0 0 2024-04-12T20:56:37.703 INFO:tasks.workunit.client.1.smithi186.stdout:2/697: getdents d0/d3/d1a/d24/db3 0 2024-04-12T20:56:37.710 INFO:tasks.workunit.client.1.smithi186.stdout:5/625: dwrite d5/db/d15/d2a/d6d/dbb/dc4/fc3 [0,4194304] 0 2024-04-12T20:56:37.716 INFO:tasks.workunit.client.1.smithi186.stdout:5/626: dread - d5/db/d15/d2a/d2d/d46/f83 zero size 2024-04-12T20:56:37.735 INFO:tasks.workunit.client.1.smithi186.stdout:9/581: rename d3/d2f/d2e/d33/d66 to d3/d2f/d54/d9f/dc4 0 2024-04-12T20:56:37.739 INFO:tasks.workunit.client.1.smithi186.stdout:9/582: mknod d3/d12/d6f/d83/cc5 0 2024-04-12T20:56:37.741 INFO:tasks.workunit.client.1.smithi186.stdout:9/583: creat d3/d13/fc6 x:0 0 0 2024-04-12T20:56:37.744 INFO:tasks.workunit.client.1.smithi186.stdout:9/584: rmdir d3/d2f/d2e 39 2024-04-12T20:56:37.748 INFO:tasks.workunit.client.1.smithi186.stdout:1/615: truncate d14/d1a/d33/d36/d5b/f2d 1269181 0 2024-04-12T20:56:37.750 INFO:tasks.workunit.client.1.smithi186.stdout:1/616: dread - d14/d1a/d33/d36/f7b zero size 2024-04-12T20:56:37.751 INFO:tasks.workunit.client.1.smithi186.stdout:1/617: chown d14/d1a/d33/d36/d5b/db9 3 1 2024-04-12T20:56:37.759 INFO:tasks.workunit.client.1.smithi186.stdout:8/591: write d7/d18/f85 [1320583,24230] 0 2024-04-12T20:56:37.761 INFO:tasks.workunit.client.1.smithi186.stdout:0/712: dwrite d0/d14/d4e/d1e/f1f [0,4194304] 0 2024-04-12T20:56:37.762 INFO:tasks.workunit.client.1.smithi186.stdout:3/591: sync 2024-04-12T20:56:37.762 INFO:tasks.workunit.client.1.smithi186.stdout:7/624: sync 2024-04-12T20:56:37.763 INFO:tasks.workunit.client.1.smithi186.stdout:4/733: truncate d0/dc/f23 38691 0 2024-04-12T20:56:37.764 INFO:tasks.workunit.client.1.smithi186.stdout:0/713: chown d0/d14/c39 20324 1 2024-04-12T20:56:37.768 INFO:tasks.workunit.client.1.smithi186.stdout:5/627: write d5/fd [3649163,102782] 0 2024-04-12T20:56:37.769 INFO:tasks.workunit.client.1.smithi186.stdout:0/714: fdatasync d0/d5c/d65/fd2 0 2024-04-12T20:56:37.773 INFO:tasks.workunit.client.1.smithi186.stdout:7/625: creat d7/d10/d2d/d67/fe4 x:0 0 0 2024-04-12T20:56:37.773 INFO:tasks.workunit.client.1.smithi186.stdout:5/628: fdatasync d5/db/d15/d2a/d4c/d93/fcf 0 2024-04-12T20:56:37.777 INFO:tasks.workunit.client.1.smithi186.stdout:0/715: creat d0/d5d/ddc/fe6 x:0 0 0 2024-04-12T20:56:37.779 INFO:tasks.workunit.client.1.smithi186.stdout:1/618: dwrite d14/d1a/d1d/f94 [0,4194304] 0 2024-04-12T20:56:37.780 INFO:tasks.workunit.client.1.smithi186.stdout:2/698: dwrite d0/d3/d1a/d21/d26/f6a [0,4194304] 0 2024-04-12T20:56:37.788 INFO:tasks.workunit.client.1.smithi186.stdout:5/629: dread d5/db/d71/f1a [0,4194304] 0 2024-04-12T20:56:37.796 INFO:tasks.workunit.client.1.smithi186.stdout:0/716: rename d0/cc to d0/d10/d32/de2/ce7 0 2024-04-12T20:56:37.796 INFO:tasks.workunit.client.1.smithi186.stdout:3/592: write d1/d3/d21/d22/f7b [307299,120473] 0 2024-04-12T20:56:37.798 INFO:tasks.workunit.client.1.smithi186.stdout:4/734: dwrite d0/dc/d33/f47 [0,4194304] 0 2024-04-12T20:56:37.799 INFO:tasks.workunit.client.1.smithi186.stdout:0/717: dread - d0/d14/d4e/d50/d6e/faf zero size 2024-04-12T20:56:37.800 INFO:tasks.workunit.client.1.smithi186.stdout:3/593: write d1/d3/d21/d22/d24/faf [435808,21420] 0 2024-04-12T20:56:37.801 INFO:tasks.workunit.client.1.smithi186.stdout:2/699: readlink d0/d3/d1a/lcb 0 2024-04-12T20:56:37.802 INFO:tasks.workunit.client.1.smithi186.stdout:8/592: truncate d7/d18/d36/d20/d23/d25/f46 60897 0 2024-04-12T20:56:37.808 INFO:tasks.workunit.client.1.smithi186.stdout:5/630: rmdir d5/db/d15/d2a/d6d/d7f/daf 39 2024-04-12T20:56:37.810 INFO:tasks.workunit.client.1.smithi186.stdout:7/626: getdents d7/d10/d2d/d86/dcb 0 2024-04-12T20:56:37.811 INFO:tasks.workunit.client.1.smithi186.stdout:5/631: chown d5/db/d15/d6a/l92 0 1 2024-04-12T20:56:37.816 INFO:tasks.workunit.client.1.smithi186.stdout:0/718: mkdir d0/d10/d71/de8 0 2024-04-12T20:56:37.816 INFO:tasks.workunit.client.1.smithi186.stdout:8/593: dread d7/d18/d36/d20/d23/f45 [0,4194304] 0 2024-04-12T20:56:37.818 INFO:tasks.workunit.client.1.smithi186.stdout:2/700: creat d0/dc/d2e/d73/fd8 x:0 0 0 2024-04-12T20:56:37.819 INFO:tasks.workunit.client.1.smithi186.stdout:0/719: write d0/d10/d32/dac/fd1 [630455,51034] 0 2024-04-12T20:56:37.820 INFO:tasks.workunit.client.1.smithi186.stdout:2/701: chown d0/d32/d69/lc3 33850 1 2024-04-12T20:56:37.822 INFO:tasks.workunit.client.1.smithi186.stdout:1/619: creat d14/d1a/d21/fc6 x:0 0 0 2024-04-12T20:56:37.825 INFO:tasks.workunit.client.1.smithi186.stdout:1/620: chown d14/d1a/d33/d36/d5b/d3e/d8b/f8e 1444 1 2024-04-12T20:56:37.829 INFO:tasks.workunit.client.1.smithi186.stdout:8/594: rmdir d7/d18/d36/d1d 39 2024-04-12T20:56:37.830 INFO:tasks.workunit.client.1.smithi186.stdout:3/594: creat d1/d3/d49/d4d/d6f/db0/fbf x:0 0 0 2024-04-12T20:56:37.831 INFO:tasks.workunit.client.1.smithi186.stdout:0/720: creat d0/d10/d32/fe9 x:0 0 0 2024-04-12T20:56:37.834 INFO:tasks.workunit.client.1.smithi186.stdout:2/702: dwrite d0/d3/d1a/f9d [0,4194304] 0 2024-04-12T20:56:37.835 INFO:tasks.workunit.client.1.smithi186.stdout:5/632: fdatasync d5/db/d15/d2a/d2d/d48/da4/fa7 0 2024-04-12T20:56:37.839 INFO:tasks.workunit.client.1.smithi186.stdout:1/621: unlink d14/d1a/d33/d36/f9d 0 2024-04-12T20:56:37.843 INFO:tasks.workunit.client.1.smithi186.stdout:8/595: fdatasync d7/d18/d36/d1d/d7d/fb5 0 2024-04-12T20:56:37.843 INFO:tasks.workunit.client.1.smithi186.stdout:0/721: creat d0/d5c/d65/fea x:0 0 0 2024-04-12T20:56:37.845 INFO:tasks.workunit.client.1.smithi186.stdout:0/722: read - d0/d5c/fc8 zero size 2024-04-12T20:56:37.845 INFO:tasks.workunit.client.1.smithi186.stdout:7/627: rename d7/d10/d1c/d5d/fc3 to d7/d10/d2d/fe5 0 2024-04-12T20:56:37.847 INFO:tasks.workunit.client.1.smithi186.stdout:2/703: creat d0/dc/d2e/d73/fd9 x:0 0 0 2024-04-12T20:56:37.851 INFO:tasks.workunit.client.1.smithi186.stdout:1/622: creat d14/fc7 x:0 0 0 2024-04-12T20:56:37.851 INFO:tasks.workunit.client.1.smithi186.stdout:4/735: read d0/dc/d33/f17 [279,63011] 0 2024-04-12T20:56:37.852 INFO:tasks.workunit.client.1.smithi186.stdout:8/596: mkdir d7/d18/d36/d20/d23/dc1 0 2024-04-12T20:56:37.853 INFO:tasks.workunit.client.1.smithi186.stdout:0/723: symlink d0/d5c/d65/leb 0 2024-04-12T20:56:37.854 INFO:tasks.workunit.client.1.smithi186.stdout:4/736: chown d0/d59/dc0/d63/da6/dc6/f82 6243007 1 2024-04-12T20:56:37.854 INFO:tasks.workunit.client.1.smithi186.stdout:8/597: readlink d7/d18/d36/d1d/d7d/da5/l82 0 2024-04-12T20:56:37.857 INFO:tasks.workunit.client.1.smithi186.stdout:8/598: truncate d7/de/d10/d7f/fa6 167329 0 2024-04-12T20:56:37.857 INFO:tasks.workunit.client.1.smithi186.stdout:2/704: rmdir d0/d3 39 2024-04-12T20:56:37.859 INFO:tasks.workunit.client.1.smithi186.stdout:8/599: write d7/d18/d36/d20/f4a [2519322,62848] 0 2024-04-12T20:56:37.872 INFO:tasks.workunit.client.1.smithi186.stdout:4/737: dwrite d0/d59/dc0/d63/da6/dc6/fe7 [0,4194304] 0 2024-04-12T20:56:37.873 INFO:tasks.workunit.client.1.smithi186.stdout:7/628: truncate d7/d10/d2d/fcf 1403813 0 2024-04-12T20:56:37.875 INFO:tasks.workunit.client.1.smithi186.stdout:5/633: link d5/db/d15/d2a/d4c/l56 d5/db/d15/d2a/d6d/ld3 0 2024-04-12T20:56:37.876 INFO:tasks.workunit.client.1.smithi186.stdout:2/705: mknod d0/d3/d1a/cda 0 2024-04-12T20:56:37.877 INFO:tasks.workunit.client.1.smithi186.stdout:4/738: stat d0/dc/d64/d87/dd2/cea 0 2024-04-12T20:56:37.877 INFO:tasks.workunit.client.1.smithi186.stdout:0/724: dread d0/d14/d4e/d1e/f98 [0,4194304] 0 2024-04-12T20:56:37.881 INFO:tasks.workunit.client.1.smithi186.stdout:7/629: readlink d7/d10/d2d/d40/lc0 0 2024-04-12T20:56:37.881 INFO:tasks.workunit.client.1.smithi186.stdout:8/600: dwrite d7/d18/d36/d20/d23/d25/f56 [4194304,4194304] 0 2024-04-12T20:56:37.885 INFO:tasks.workunit.client.1.smithi186.stdout:8/601: write d7/d18/d36/d1d/d7d/da5/f6e [437096,32110] 0 2024-04-12T20:56:37.887 INFO:tasks.workunit.client.1.smithi186.stdout:2/706: mknod d0/d3/d1a/d21/d26/d86/cdb 0 2024-04-12T20:56:37.888 INFO:tasks.workunit.client.1.smithi186.stdout:1/623: rename d14/d1a/d58/da5 to d14/d91/dbd/dc5/dc8 0 2024-04-12T20:56:37.890 INFO:tasks.workunit.client.1.smithi186.stdout:5/634: creat d5/db/d71/d88/fd4 x:0 0 0 2024-04-12T20:56:37.892 INFO:tasks.workunit.client.1.smithi186.stdout:5/635: fsync d5/fd 0 2024-04-12T20:56:37.892 INFO:tasks.workunit.client.1.smithi186.stdout:4/739: mknod d0/dc/cf6 0 2024-04-12T20:56:37.895 INFO:tasks.workunit.client.1.smithi186.stdout:8/602: chown d7/d18/d36/d1d/d1f/d26/lba 914309 1 2024-04-12T20:56:37.897 INFO:tasks.workunit.client.1.smithi186.stdout:0/725: dwrite d0/d10/d32/dac/fb6 [0,4194304] 0 2024-04-12T20:56:37.900 INFO:tasks.workunit.client.1.smithi186.stdout:7/630: fdatasync d7/d10/d1c/fdb 0 2024-04-12T20:56:37.900 INFO:tasks.workunit.client.1.smithi186.stdout:1/624: truncate d14/d1a/d1d/f46 2699874 0 2024-04-12T20:56:37.902 INFO:tasks.workunit.client.1.smithi186.stdout:4/740: read d0/d26/f34 [854917,53936] 0 2024-04-12T20:56:37.902 INFO:tasks.workunit.client.1.smithi186.stdout:8/603: creat d7/d18/d36/d20/d71/fc2 x:0 0 0 2024-04-12T20:56:37.907 INFO:tasks.workunit.client.1.smithi186.stdout:0/726: write d0/d85/dc4/dc1/fd3 [1373779,118359] 0 2024-04-12T20:56:37.908 INFO:tasks.workunit.client.1.smithi186.stdout:7/631: unlink d7/d10/d1c/d5d/db7/le3 0 2024-04-12T20:56:37.911 INFO:tasks.workunit.client.1.smithi186.stdout:4/741: fsync d0/dc/fa2 0 2024-04-12T20:56:37.916 INFO:tasks.workunit.client.1.smithi186.stdout:1/625: fdatasync d14/f77 0 2024-04-12T20:56:37.917 INFO:tasks.workunit.client.1.smithi186.stdout:5/636: dwrite d5/f6 [4194304,4194304] 0 2024-04-12T20:56:37.920 INFO:tasks.workunit.client.1.smithi186.stdout:2/707: getdents d0/d3/d1a/d21/d26 0 2024-04-12T20:56:37.921 INFO:tasks.workunit.client.1.smithi186.stdout:1/626: chown d14/d1a/d58/c5d 30923551 1 2024-04-12T20:56:37.921 INFO:tasks.workunit.client.1.smithi186.stdout:0/727: mkdir d0/d14/d4e/d1e/d84/dec 0 2024-04-12T20:56:37.924 INFO:tasks.workunit.client.1.smithi186.stdout:4/742: creat d0/d59/dc0/d48/d61/ff7 x:0 0 0 2024-04-12T20:56:37.925 INFO:tasks.workunit.client.1.smithi186.stdout:8/604: mkdir d7/d18/d36/d1d/dc3 0 2024-04-12T20:56:37.937 INFO:tasks.workunit.client.1.smithi186.stdout:1/627: write d14/d1a/d58/fb7 [872782,97387] 0 2024-04-12T20:56:37.939 INFO:tasks.workunit.client.1.smithi186.stdout:4/743: write d0/dc/fa2 [79749,37132] 0 2024-04-12T20:56:37.941 INFO:tasks.workunit.client.1.smithi186.stdout:1/628: fdatasync d14/d1a/d33/d36/d5b/d3e/d71/f78 0 2024-04-12T20:56:37.942 INFO:tasks.workunit.client.1.smithi186.stdout:4/744: chown d0/dc/d1f/l3e 474139161 1 2024-04-12T20:56:37.949 INFO:tasks.workunit.client.1.smithi186.stdout:7/632: link d7/d10/d2d/d67/fe4 d7/d10/d1c/d5d/db7/fe6 0 2024-04-12T20:56:37.954 INFO:tasks.workunit.client.1.smithi186.stdout:5/637: creat d5/db/d15/d2a/d2d/d46/d55/fd5 x:0 0 0 2024-04-12T20:56:37.958 INFO:tasks.workunit.client.1.smithi186.stdout:5/638: read d5/db/d15/d2a/d6d/dbb/dc4/fa9 [3637439,54132] 0 2024-04-12T20:56:37.959 INFO:tasks.workunit.client.1.smithi186.stdout:5/639: stat d5/db/d15/d2a/d6d/fb9 0 2024-04-12T20:56:37.960 INFO:tasks.workunit.client.1.smithi186.stdout:4/745: mkdir d0/dc/d88/d80/df8 0 2024-04-12T20:56:37.961 INFO:tasks.workunit.client.1.smithi186.stdout:2/708: creat d0/d3/d1a/d21/fdc x:0 0 0 2024-04-12T20:56:37.962 INFO:tasks.workunit.client.1.smithi186.stdout:1/629: write f4 [1971079,62717] 0 2024-04-12T20:56:37.963 INFO:tasks.workunit.client.1.smithi186.stdout:5/640: write d5/db/d15/d2a/d4c/d5b/d7a/fb1 [3060118,124133] 0 2024-04-12T20:56:37.964 INFO:tasks.workunit.client.1.smithi186.stdout:2/709: fdatasync d0/d3/d7a/f94 0 2024-04-12T20:56:37.965 INFO:tasks.workunit.client.1.smithi186.stdout:7/633: chown d7/d10/d1c/f2c 437 1 2024-04-12T20:56:37.968 INFO:tasks.workunit.client.1.smithi186.stdout:7/634: chown d7/d10/d2d/d40/f77 185709284 1 2024-04-12T20:56:37.970 INFO:tasks.workunit.client.1.smithi186.stdout:2/710: dread d0/d3/d1a/d48/f84 [0,4194304] 0 2024-04-12T20:56:37.972 INFO:tasks.workunit.client.1.smithi186.stdout:1/630: mkdir d14/d1a/d33/d36/d5b/db9/dc9 0 2024-04-12T20:56:37.974 INFO:tasks.workunit.client.1.smithi186.stdout:8/605: getdents d7/de 0 2024-04-12T20:56:37.975 INFO:tasks.workunit.client.1.smithi186.stdout:5/641: mkdir d5/db/d2e/d70/dd6 0 2024-04-12T20:56:37.975 INFO:tasks.workunit.client.1.smithi186.stdout:1/631: truncate d14/d1a/d33/d36/d5b/fc2 739487 0 2024-04-12T20:56:37.977 INFO:tasks.workunit.client.1.smithi186.stdout:7/635: fdatasync d7/fc 0 2024-04-12T20:56:37.981 INFO:tasks.workunit.client.1.smithi186.stdout:8/606: symlink d7/d18/d36/d1d/d1f/d26/dab/lc4 0 2024-04-12T20:56:37.983 INFO:tasks.workunit.client.1.smithi186.stdout:8/607: fdatasync d7/d18/d36/d1d/d7d/da5/f9c 0 2024-04-12T20:56:37.984 INFO:tasks.workunit.client.1.smithi186.stdout:5/642: symlink d5/db/d15/d2a/d2d/d46/ld7 0 2024-04-12T20:56:37.986 INFO:tasks.workunit.client.1.smithi186.stdout:1/632: symlink d14/d1a/d33/d36/d5b/d3e/d71/lca 0 2024-04-12T20:56:37.986 INFO:tasks.workunit.client.1.smithi186.stdout:4/746: link d0/dc/d88/d80/le4 d0/d59/dc0/d63/da6/daf/lf9 0 2024-04-12T20:56:37.988 INFO:tasks.workunit.client.1.smithi186.stdout:2/711: getdents d0/d3/dce 0 2024-04-12T20:56:37.989 INFO:tasks.workunit.client.1.smithi186.stdout:4/747: read d0/dc/f23 [21765,80376] 0 2024-04-12T20:56:37.990 INFO:tasks.workunit.client.1.smithi186.stdout:7/636: mknod d7/d10/d2d/d40/d4a/d6c/d97/dc2/de2/ce7 0 2024-04-12T20:56:37.991 INFO:tasks.workunit.client.1.smithi186.stdout:2/712: write d0/dc/f40 [721195,125643] 0 2024-04-12T20:56:37.993 INFO:tasks.workunit.client.1.smithi186.stdout:4/748: fdatasync d0/dc/d64/d87/dd2/fd7 0 2024-04-12T20:56:37.996 INFO:tasks.workunit.client.1.smithi186.stdout:2/713: write d0/d32/f7c [5215046,58743] 0 2024-04-12T20:56:38.000 INFO:tasks.workunit.client.1.smithi186.stdout:2/714: write d0/d3/f58 [3027290,124593] 0 2024-04-12T20:56:38.000 INFO:tasks.workunit.client.1.smithi186.stdout:4/749: unlink d0/d59/dc0/c7e 0 2024-04-12T20:56:38.004 INFO:tasks.workunit.client.1.smithi186.stdout:7/637: creat d7/d10/d2d/d40/d4a/d6c/d97/d4d/d84/fe8 x:0 0 0 2024-04-12T20:56:38.005 INFO:tasks.workunit.client.1.smithi186.stdout:2/715: rmdir d0/d3/d1a/d21 39 2024-04-12T20:56:38.008 INFO:tasks.workunit.client.1.smithi186.stdout:1/633: dwrite d14/d1a/d33/d36/d5b/fa3 [0,4194304] 0 2024-04-12T20:56:38.008 INFO:tasks.workunit.client.1.smithi186.stdout:8/608: dwrite d7/d18/d36/d1d/d1f/da2/d5c/f7b [0,4194304] 0 2024-04-12T20:56:38.011 INFO:tasks.workunit.client.1.smithi186.stdout:8/609: dread - d7/d18/d36/d1d/d1f/da2/fbe zero size 2024-04-12T20:56:38.015 INFO:tasks.workunit.client.1.smithi186.stdout:4/750: creat d0/dc/d88/d80/df8/ffa x:0 0 0 2024-04-12T20:56:38.015 INFO:tasks.workunit.client.1.smithi186.stdout:5/643: dread d5/db/d71/f61 [0,4194304] 0 2024-04-12T20:56:38.015 INFO:tasks.workunit.client.1.smithi186.stdout:3/595: sync 2024-04-12T20:56:38.015 INFO:tasks.workunit.client.1.smithi186.stdout:9/585: sync 2024-04-12T20:56:38.016 INFO:tasks.workunit.client.1.smithi186.stdout:6/619: sync 2024-04-12T20:56:38.019 INFO:tasks.workunit.client.1.smithi186.stdout:8/610: write d7/d18/d36/d1d/d7d/da5/f9c [2982976,56106] 0 2024-04-12T20:56:38.021 INFO:tasks.workunit.client.1.smithi186.stdout:4/751: rmdir d0/dc/d1f/d73 39 2024-04-12T20:56:38.023 INFO:tasks.workunit.client.1.smithi186.stdout:9/586: rmdir d3/d2f/d54/d9f/dc4/d77/dc3 39 2024-04-12T20:56:38.026 INFO:tasks.workunit.client.1.smithi186.stdout:1/634: dwrite d14/d1a/d33/d36/d5b/f79 [0,4194304] 0 2024-04-12T20:56:38.029 INFO:tasks.workunit.client.1.smithi186.stdout:1/635: write d14/d1a/d21/d35/d59/fbf [775640,102982] 0 2024-04-12T20:56:38.030 INFO:tasks.workunit.client.1.smithi186.stdout:3/596: mknod d1/d3/d21/d22/da4/cc0 0 2024-04-12T20:56:38.039 INFO:tasks.workunit.client.1.smithi186.stdout:2/716: dread d0/d32/f97 [0,4194304] 0 2024-04-12T20:56:38.039 INFO:tasks.workunit.client.1.smithi186.stdout:6/620: mknod d9/d5e/d7a/d81/d96/ddb/cdf 0 2024-04-12T20:56:38.039 INFO:tasks.workunit.client.1.smithi186.stdout:3/597: write d1/d3/d49/d4d/f75 [2486706,101894] 0 2024-04-12T20:56:38.039 INFO:tasks.workunit.client.1.smithi186.stdout:6/621: write d9/f2e [174729,75966] 0 2024-04-12T20:56:38.039 INFO:tasks.workunit.client.1.smithi186.stdout:5/644: rmdir d5/db/d2e/d70/dd6 0 2024-04-12T20:56:38.045 INFO:tasks.workunit.client.1.smithi186.stdout:1/636: creat d14/d6c/fcb x:0 0 0 2024-04-12T20:56:38.046 INFO:tasks.workunit.client.1.smithi186.stdout:5/645: unlink d5/db/d15/d2a/d2d/f58 0 2024-04-12T20:56:38.051 INFO:tasks.workunit.client.1.smithi186.stdout:9/587: dwrite d3/d2f/f1e [0,4194304] 0 2024-04-12T20:56:38.054 INFO:tasks.workunit.client.1.smithi186.stdout:5/646: symlink d5/db/d15/d2a/d6d/ld8 0 2024-04-12T20:56:38.057 INFO:tasks.workunit.client.1.smithi186.stdout:1/637: read d14/d1a/d33/d36/d5b/d3e/d71/f78 [889890,63575] 0 2024-04-12T20:56:38.059 INFO:tasks.workunit.client.1.smithi186.stdout:3/598: dread d1/d3/d21/d22/f79 [0,4194304] 0 2024-04-12T20:56:38.062 INFO:tasks.workunit.client.1.smithi186.stdout:5/647: read d5/db/f5f [2777848,120698] 0 2024-04-12T20:56:38.070 INFO:tasks.workunit.client.1.smithi186.stdout:8/611: dwrite d7/d18/d36/d1d/d1f/da2/f79 [4194304,4194304] 0 2024-04-12T20:56:38.071 INFO:tasks.workunit.client.1.smithi186.stdout:5/648: chown d5/db/f57 4014592 1 2024-04-12T20:56:38.072 INFO:tasks.workunit.client.1.smithi186.stdout:4/752: dwrite d0/dc/d33/f5d [0,4194304] 0 2024-04-12T20:56:38.077 INFO:tasks.workunit.client.1.smithi186.stdout:2/717: dwrite d0/dc/d2e/d73/fc4 [0,4194304] 0 2024-04-12T20:56:38.078 INFO:tasks.workunit.client.1.smithi186.stdout:5/649: chown d5/db/d15/d2a/d6d/d7f/d80/fcd 63058 1 2024-04-12T20:56:38.080 INFO:tasks.workunit.client.1.smithi186.stdout:6/622: dread d9/d4c/f61 [0,4194304] 0 2024-04-12T20:56:38.080 INFO:tasks.workunit.client.1.smithi186.stdout:3/599: rename d1/d3/d21/l23 to d1/d3/d49/d4d/dad/lc1 0 2024-04-12T20:56:38.085 INFO:tasks.workunit.client.1.smithi186.stdout:8/612: creat d7/d18/d36/d1d/d7d/fc5 x:0 0 0 2024-04-12T20:56:38.087 INFO:tasks.workunit.client.1.smithi186.stdout:1/638: write d14/d1a/d33/d36/d5b/f4d [744488,118305] 0 2024-04-12T20:56:38.091 INFO:tasks.workunit.client.1.smithi186.stdout:2/718: creat d0/dc/d2e/fdd x:0 0 0 2024-04-12T20:56:38.098 INFO:tasks.workunit.client.1.smithi186.stdout:8/613: rename d7/d18/d36/d20/f4a to d7/de/d5e/fc6 0 2024-04-12T20:56:38.104 INFO:tasks.workunit.client.1.smithi186.stdout:8/614: write d7/d18/d36/d1d/d7d/d87/fb4 [222152,40430] 0 2024-04-12T20:56:38.106 INFO:tasks.workunit.client.1.smithi186.stdout:1/639: creat d14/d1a/fcc x:0 0 0 2024-04-12T20:56:38.106 INFO:tasks.workunit.client.1.smithi186.stdout:2/719: truncate d0/d3/d7a/f94 2416774 0 2024-04-12T20:56:38.108 INFO:tasks.workunit.client.1.smithi186.stdout:3/600: link d1/d3/c45 d1/d3/d21/d22/d34/d6e/cc2 0 2024-04-12T20:56:38.109 INFO:tasks.workunit.client.1.smithi186.stdout:2/720: readlink d0/d3/d1a/d41/l4a 0 2024-04-12T20:56:38.110 INFO:tasks.workunit.client.1.smithi186.stdout:6/623: dwrite d9/f37 [0,4194304] 0 2024-04-12T20:56:38.113 INFO:tasks.workunit.client.1.smithi186.stdout:1/640: truncate d14/d1a/d58/fb7 1860005 0 2024-04-12T20:56:38.113 INFO:tasks.workunit.client.1.smithi186.stdout:2/721: fdatasync d0/d3/d7a/d8d/fbd 0 2024-04-12T20:56:38.116 INFO:tasks.workunit.client.1.smithi186.stdout:8/615: unlink d7/d18/d36/d20/d23/d25/l2d 0 2024-04-12T20:56:38.119 INFO:tasks.workunit.client.1.smithi186.stdout:8/616: chown d7/d18/d36/d20/d23/dc1 47593482 1 2024-04-12T20:56:38.123 INFO:tasks.workunit.client.1.smithi186.stdout:4/753: dwrite d0/d59/dc0/f84 [0,4194304] 0 2024-04-12T20:56:38.123 INFO:tasks.workunit.client.1.smithi186.stdout:3/601: chown d1/d3/d21/d22/d24/d28/lb8 29 1 2024-04-12T20:56:38.124 INFO:tasks.workunit.client.1.smithi186.stdout:6/624: creat d9/d43/d2d/d4d/fe0 x:0 0 0 2024-04-12T20:56:38.126 INFO:tasks.workunit.client.1.smithi186.stdout:6/625: chown d9/d5e/f71 1 1 2024-04-12T20:56:38.128 INFO:tasks.workunit.client.1.smithi186.stdout:1/641: creat d14/d91/dbd/dc5/fcd x:0 0 0 2024-04-12T20:56:38.130 INFO:tasks.workunit.client.1.smithi186.stdout:1/642: write d14/d1a/d33/d36/d5b/d6a/da2/fa6 [908027,84459] 0 2024-04-12T20:56:38.132 INFO:tasks.workunit.client.1.smithi186.stdout:0/728: sync 2024-04-12T20:56:38.133 INFO:tasks.workunit.client.1.smithi186.stdout:8/617: rename d7/d18/d36/d20/d23/l2c to d7/d18/d36/dad/lc7 0 2024-04-12T20:56:38.136 INFO:tasks.workunit.client.1.smithi186.stdout:8/618: readlink d7/d18/d36/d20/d23/d25/l6d 0 2024-04-12T20:56:38.141 INFO:tasks.workunit.client.1.smithi186.stdout:5/650: link d5/db/d15/d2a/d6d/dbb/dc4/fa9 d5/db/d15/d2a/d6d/d7f/d86/dca/fd9 0 2024-04-12T20:56:38.153 INFO:tasks.workunit.client.1.smithi186.stdout:3/602: truncate d1/d3/d49/f59 2581224 0 2024-04-12T20:56:38.154 INFO:tasks.workunit.client.1.smithi186.stdout:4/754: truncate d0/d21/f27 1236145 0 2024-04-12T20:56:38.157 INFO:tasks.workunit.client.1.smithi186.stdout:8/619: unlink d7/d18/d36/d1d/d1f/d26/f95 0 2024-04-12T20:56:38.158 INFO:tasks.workunit.client.1.smithi186.stdout:8/620: readlink d7/d18/d36/d1d/d1f/da2/l39 0 2024-04-12T20:56:38.161 INFO:tasks.workunit.client.1.smithi186.stdout:8/621: write d7/d18/d36/d1d/d1f/d26/f6b [2560025,129181] 0 2024-04-12T20:56:38.162 INFO:tasks.workunit.client.1.smithi186.stdout:4/755: dread d0/d26/f34 [0,4194304] 0 2024-04-12T20:56:38.168 INFO:tasks.workunit.client.1.smithi186.stdout:3/603: dwrite d1/d3/d21/fb1 [0,4194304] 0 2024-04-12T20:56:38.169 INFO:tasks.workunit.client.1.smithi186.stdout:1/643: mknod d14/d1a/d33/d36/d5b/db9/dc9/cce 0 2024-04-12T20:56:38.170 INFO:tasks.workunit.client.1.smithi186.stdout:5/651: write d5/db/d71/f61 [862702,23069] 0 2024-04-12T20:56:38.173 INFO:tasks.workunit.client.1.smithi186.stdout:5/652: write d5/db/fbd [786495,116829] 0 2024-04-12T20:56:38.173 INFO:tasks.workunit.client.1.smithi186.stdout:2/722: link d0/d3/d7a/f94 d0/d3/d1a/d48/fde 0 2024-04-12T20:56:38.176 INFO:tasks.workunit.client.1.smithi186.stdout:5/653: write d5/db/d15/d2a/d2d/d77/fac [606377,23455] 0 2024-04-12T20:56:38.179 INFO:tasks.workunit.client.1.smithi186.stdout:6/626: link d9/d43/d2d/d4d/cda d9/d43/d2d/ce1 0 2024-04-12T20:56:38.180 INFO:tasks.workunit.client.1.smithi186.stdout:8/622: mknod d7/de/d68/cc8 0 2024-04-12T20:56:38.182 INFO:tasks.workunit.client.1.smithi186.stdout:4/756: rmdir d0/d59/ddf 39 2024-04-12T20:56:38.184 INFO:tasks.workunit.client.1.smithi186.stdout:9/588: sync 2024-04-12T20:56:38.184 INFO:tasks.workunit.client.1.smithi186.stdout:7/638: sync 2024-04-12T20:56:38.185 INFO:tasks.workunit.client.1.smithi186.stdout:0/729: creat d0/d10/d72/fed x:0 0 0 2024-04-12T20:56:38.187 INFO:tasks.workunit.client.1.smithi186.stdout:0/730: write d0/d14/d4e/d50/fce [861938,64752] 0 2024-04-12T20:56:38.191 INFO:tasks.workunit.client.1.smithi186.stdout:5/654: creat d5/db/d15/d2a/d2d/d82/fda x:0 0 0 2024-04-12T20:56:38.194 INFO:tasks.workunit.client.1.smithi186.stdout:8/623: rmdir d7/d18/d36/d1d/d1f/da2/d5c 39 2024-04-12T20:56:38.204 INFO:tasks.workunit.client.1.smithi186.stdout:0/731: unlink d0/d10/d32/d3f/f77 0 2024-04-12T20:56:38.208 INFO:tasks.workunit.client.1.smithi186.stdout:2/723: mkdir d0/d3/dce/ddf 0 2024-04-12T20:56:38.213 INFO:tasks.workunit.client.1.smithi186.stdout:6/627: creat d9/d43/d59/d5d/d6e/d91/dd9/fe2 x:0 0 0 2024-04-12T20:56:38.215 INFO:tasks.workunit.client.1.smithi186.stdout:8/624: dread - d7/d18/d36/d1d/d1f/da2/d5c/f91 zero size 2024-04-12T20:56:38.218 INFO:tasks.workunit.client.1.smithi186.stdout:4/757: write d0/d21/f6a [3006008,69591] 0 2024-04-12T20:56:38.218 INFO:tasks.workunit.client.1.smithi186.stdout:5/655: write d5/db/d15/f39 [183525,126545] 0 2024-04-12T20:56:38.219 INFO:tasks.workunit.client.1.smithi186.stdout:1/644: creat d14/fcf x:0 0 0 2024-04-12T20:56:38.221 INFO:tasks.workunit.client.1.smithi186.stdout:3/604: rename d1/f6a to d1/d3/fc3 0 2024-04-12T20:56:38.223 INFO:tasks.workunit.client.1.smithi186.stdout:7/639: dwrite d7/d10/d1c/f2c [8388608,4194304] 0 2024-04-12T20:56:38.224 INFO:tasks.workunit.client.1.smithi186.stdout:0/732: creat d0/d10/d32/dd8/fee x:0 0 0 2024-04-12T20:56:38.224 INFO:tasks.workunit.client.1.smithi186.stdout:3/605: chown d1/d3/c32 922675 1 2024-04-12T20:56:38.226 INFO:tasks.workunit.client.1.smithi186.stdout:2/724: creat d0/d3/d7a/fe0 x:0 0 0 2024-04-12T20:56:38.226 INFO:tasks.workunit.client.1.smithi186.stdout:0/733: write d0/d5d/ddc/fe6 [684175,72993] 0 2024-04-12T20:56:38.230 INFO:tasks.workunit.client.1.smithi186.stdout:7/640: write d7/d10/d2d/d40/d4a/d6c/d97/d4d/d8a/dad/fe0 [668112,110725] 0 2024-04-12T20:56:38.232 INFO:tasks.workunit.client.1.smithi186.stdout:0/734: truncate d0/d5c/d66/da7/fb4 523665 0 2024-04-12T20:56:38.236 INFO:tasks.workunit.client.1.smithi186.stdout:1/645: fdatasync d14/d1a/f37 0 2024-04-12T20:56:38.238 INFO:tasks.workunit.client.1.smithi186.stdout:1/646: chown d14/d1a/d33/d36/d5b/d3e/d71/d8c/l98 1 1 2024-04-12T20:56:38.241 INFO:tasks.workunit.client.1.smithi186.stdout:6/628: mkdir d9/d4c/d6f/de3 0 2024-04-12T20:56:38.246 INFO:tasks.workunit.client.1.smithi186.stdout:3/606: mknod d1/d3/d21/d22/d4a/cc4 0 2024-04-12T20:56:38.246 INFO:tasks.workunit.client.1.smithi186.stdout:3/607: chown d1/d3/d21/d22/d24 565 1 2024-04-12T20:56:38.246 INFO:tasks.workunit.client.1.smithi186.stdout:2/725: creat d0/d3/d1a/d48/fe1 x:0 0 0 2024-04-12T20:56:38.250 INFO:tasks.workunit.client.1.smithi186.stdout:0/735: creat d0/d10/d32/dac/fef x:0 0 0 2024-04-12T20:56:38.251 INFO:tasks.workunit.client.1.smithi186.stdout:9/589: creat d3/d2f/d2e/fc7 x:0 0 0 2024-04-12T20:56:38.254 INFO:tasks.workunit.client.1.smithi186.stdout:0/736: truncate d0/d10/d32/fe9 55937 0 2024-04-12T20:56:38.254 INFO:tasks.workunit.client.1.smithi186.stdout:4/758: dread d0/d59/dc0/d63/da6/dc6/f66 [0,4194304] 0 2024-04-12T20:56:38.257 INFO:tasks.workunit.client.1.smithi186.stdout:3/608: rename d1/d3/da/f9e to d1/d3/d49/fc5 0 2024-04-12T20:56:38.260 INFO:tasks.workunit.client.1.smithi186.stdout:9/590: symlink d3/d12/d6f/d83/d71/d72/lc8 0 2024-04-12T20:56:38.263 INFO:tasks.workunit.client.1.smithi186.stdout:5/656: truncate d5/db/f11 1741971 0 2024-04-12T20:56:38.263 INFO:tasks.workunit.client.1.smithi186.stdout:8/625: truncate d7/d18/d36/d1d/d1f/da2/d5c/f7b 1831925 0 2024-04-12T20:56:38.264 INFO:tasks.workunit.client.1.smithi186.stdout:6/629: dread d9/d43/d50/f9e [0,4194304] 0 2024-04-12T20:56:38.264 INFO:tasks.workunit.client.1.smithi186.stdout:1/647: dread d14/d1a/d33/d36/d5b/f2d [0,4194304] 0 2024-04-12T20:56:38.266 INFO:tasks.workunit.client.1.smithi186.stdout:2/726: write d0/d3/d7a/f94 [615183,124477] 0 2024-04-12T20:56:38.268 INFO:tasks.workunit.client.1.smithi186.stdout:0/737: mkdir d0/d85/df0 0 2024-04-12T20:56:38.271 INFO:tasks.workunit.client.1.smithi186.stdout:7/641: dread d7/d10/d2d/d40/d4a/f4c [0,4194304] 0 2024-04-12T20:56:38.275 INFO:tasks.workunit.client.1.smithi186.stdout:3/609: mkdir d1/d3/d21/d22/dc6 0 2024-04-12T20:56:38.276 INFO:tasks.workunit.client.1.smithi186.stdout:7/642: truncate d7/d10/d2d/d40/d4a/d6c/d97/d4d/d84/fe8 598619 0 2024-04-12T20:56:38.277 INFO:tasks.workunit.client.1.smithi186.stdout:3/610: write d1/d3/d21/d22/f7b [2548109,44792] 0 2024-04-12T20:56:38.282 INFO:tasks.workunit.client.1.smithi186.stdout:6/630: mknod d9/d5e/d7a/ce4 0 2024-04-12T20:56:38.282 INFO:tasks.workunit.client.1.smithi186.stdout:1/648: write d14/f7d [186491,13800] 0 2024-04-12T20:56:38.283 INFO:tasks.workunit.client.1.smithi186.stdout:4/759: link d0/dc/c60 d0/d59/dc0/d63/da6/cfb 0 2024-04-12T20:56:38.283 INFO:tasks.workunit.client.1.smithi186.stdout:8/626: getdents d7/de/d68/da7 0 2024-04-12T20:56:38.286 INFO:tasks.workunit.client.1.smithi186.stdout:3/611: dread d1/d3/d21/d22/d24/faf [0,4194304] 0 2024-04-12T20:56:38.286 INFO:tasks.workunit.client.1.smithi186.stdout:6/631: dread - d9/d43/d2d/fa6 zero size 2024-04-12T20:56:38.287 INFO:tasks.workunit.client.1.smithi186.stdout:7/643: unlink d7/d10/d2d/d40/d4a/f51 0 2024-04-12T20:56:38.287 INFO:tasks.workunit.client.1.smithi186.stdout:5/657: dread d5/db/d15/f89 [0,4194304] 0 2024-04-12T20:56:38.289 INFO:tasks.workunit.client.1.smithi186.stdout:4/760: read d0/dc/d64/d87/d98/fbc [1530034,110000] 0 2024-04-12T20:56:38.290 INFO:tasks.workunit.client.1.smithi186.stdout:0/738: unlink d0/d14/d4e/d1e/l86 0 2024-04-12T20:56:38.291 INFO:tasks.workunit.client.1.smithi186.stdout:9/591: link d3/d12/c1a d3/d2f/d2e/d33/d9d/cc9 0 2024-04-12T20:56:38.295 INFO:tasks.workunit.client.1.smithi186.stdout:0/739: read d0/d10/f8d [3619692,72292] 0 2024-04-12T20:56:38.295 INFO:tasks.workunit.client.1.smithi186.stdout:2/727: creat d0/fe2 x:0 0 0 2024-04-12T20:56:38.297 INFO:tasks.workunit.client.1.smithi186.stdout:0/740: write d0/d81/f83 [1795541,84004] 0 2024-04-12T20:56:38.303 INFO:tasks.workunit.client.1.smithi186.stdout:3/612: dread d1/d3/d49/d4d/d9c/fa7 [0,4194304] 0 2024-04-12T20:56:38.305 INFO:tasks.workunit.client.1.smithi186.stdout:7/644: creat d7/de1/fe9 x:0 0 0 2024-04-12T20:56:38.306 INFO:tasks.workunit.client.1.smithi186.stdout:7/645: fdatasync d7/d10/fbf 0 2024-04-12T20:56:38.307 INFO:tasks.workunit.client.1.smithi186.stdout:2/728: dread d0/d3/d1a/f87 [0,4194304] 0 2024-04-12T20:56:38.315 INFO:tasks.workunit.client.1.smithi186.stdout:9/592: write d3/d12/f86 [2536920,37090] 0 2024-04-12T20:56:38.317 INFO:tasks.workunit.client.1.smithi186.stdout:9/593: chown d3/d12/d6f/d83/d71/d72/l8a 0 1 2024-04-12T20:56:38.318 INFO:tasks.workunit.client.1.smithi186.stdout:1/649: mknod d14/d1a/d21/d35/dbc/cd0 0 2024-04-12T20:56:38.319 INFO:tasks.workunit.client.1.smithi186.stdout:7/646: dwrite d7/d10/d1c/f20 [0,4194304] 0 2024-04-12T20:56:38.319 INFO:tasks.workunit.client.1.smithi186.stdout:1/650: stat d14/d91/dbd/dc5 0 2024-04-12T20:56:38.322 INFO:tasks.workunit.client.1.smithi186.stdout:1/651: write d14/d1a/d33/d36/d5b/db9/fba [162020,2766] 0 2024-04-12T20:56:38.329 INFO:tasks.workunit.client.1.smithi186.stdout:5/658: readlink d5/db/d15/d2a/d6d/dbb/dc4/lba 0 2024-04-12T20:56:38.329 INFO:tasks.workunit.client.1.smithi186.stdout:2/729: mknod d0/d3/d7a/ce3 0 2024-04-12T20:56:38.330 INFO:tasks.workunit.client.1.smithi186.stdout:2/730: fdatasync d0/d32/f9b 0 2024-04-12T20:56:38.333 INFO:tasks.workunit.client.1.smithi186.stdout:6/632: link d9/d43/d59/d5d/d6e/d91/fb5 d9/d43/d59/d5d/d6e/d91/fe5 0 2024-04-12T20:56:38.339 INFO:tasks.workunit.client.1.smithi186.stdout:8/627: rename d7/d18/d36/d1d/d7d/d87 to d7/d18/d36/dc9 0 2024-04-12T20:56:38.341 INFO:tasks.workunit.client.1.smithi186.stdout:1/652: creat d14/d1a/d33/d36/d5b/d6a/fd1 x:0 0 0 2024-04-12T20:56:38.344 INFO:tasks.workunit.client.1.smithi186.stdout:0/741: dwrite d0/d10/d32/f3d [0,4194304] 0 2024-04-12T20:56:38.346 INFO:tasks.workunit.client.1.smithi186.stdout:9/594: fdatasync d3/d12/d6f/d83/d71/d72/fbf 0 2024-04-12T20:56:38.346 INFO:tasks.workunit.client.1.smithi186.stdout:1/653: write d14/d91/dbd/dc5/fcd [145728,95820] 0 2024-04-12T20:56:38.350 INFO:tasks.workunit.client.1.smithi186.stdout:2/731: dwrite d0/dc/d2e/fdd [0,4194304] 0 2024-04-12T20:56:38.350 INFO:tasks.workunit.client.1.smithi186.stdout:3/613: creat d1/d3/d21/d22/d24/fc7 x:0 0 0 2024-04-12T20:56:38.352 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:56:38 smithi018 ceph-mon[28799]: Upgrade: Updating mgr.smithi018.xjzwga 2024-04-12T20:56:38.352 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:56:38 smithi018 ceph-mon[28799]: Deploying daemon mgr.smithi018.xjzwga on smithi018 2024-04-12T20:56:38.352 INFO:tasks.workunit.client.1.smithi186.stdout:4/761: rename d0/d26/l5f to d0/dc/d88/d80/lfc 0 2024-04-12T20:56:38.355 INFO:tasks.workunit.client.1.smithi186.stdout:3/614: chown d1/d3/d49/d4d/dad 27312234 1 2024-04-12T20:56:38.358 INFO:tasks.workunit.client.1.smithi186.stdout:4/762: stat d0/d59/dc0/d63/da6/dc6/f82 0 2024-04-12T20:56:38.364 INFO:tasks.workunit.client.1.smithi186.stdout:8/628: dwrite d7/de/d5e/f98 [0,4194304] 0 2024-04-12T20:56:38.364 INFO:tasks.workunit.client.1.smithi186.stdout:6/633: mkdir d9/d43/d59/d5d/de6 0 2024-04-12T20:56:38.368 INFO:tasks.workunit.client.1.smithi186.stdout:6/634: stat d9/c8e 0 2024-04-12T20:56:38.373 INFO:tasks.workunit.client.1.smithi186.stdout:2/732: mkdir d0/d32/de4 0 2024-04-12T20:56:38.374 INFO:tasks.workunit.client.1.smithi186.stdout:5/659: dwrite d5/db/d15/d2a/d6d/dbb/dc4/fc1 [0,4194304] 0 2024-04-12T20:56:38.376 INFO:tasks.workunit.client.1.smithi186.stdout:6/635: truncate d9/da9/dbd/fd1 1432969 0 2024-04-12T20:56:38.376 INFO:tasks.workunit.client.1.smithi186.stdout:3/615: mknod d1/d2c/d54/d7c/cc8 0 2024-04-12T20:56:38.382 INFO:tasks.workunit.client.1.smithi186.stdout:1/654: chown d14/d6c/l7c 5351954 1 2024-04-12T20:56:38.383 INFO:tasks.workunit.client.1.smithi186.stdout:7/647: getdents d7/d10/d2d/d40/d4a/d6c/d97/dc2 0 2024-04-12T20:56:38.385 INFO:tasks.workunit.client.1.smithi186.stdout:0/742: link d0/d10/d32/dac/fd1 d0/d10/d32/ff1 0 2024-04-12T20:56:38.385 INFO:tasks.workunit.client.1.smithi186.stdout:7/648: dread - d7/d10/d1c/d5d/db7/fe6 zero size 2024-04-12T20:56:38.390 INFO:tasks.workunit.client.1.smithi186.stdout:3/616: creat d1/d3/d21/d22/d24/d28/fc9 x:0 0 0 2024-04-12T20:56:38.392 INFO:tasks.workunit.client.1.smithi186.stdout:7/649: dread - d7/d10/d1c/d8c/fdc zero size 2024-04-12T20:56:38.392 INFO:tasks.workunit.client.1.smithi186.stdout:4/763: creat d0/dc/d1f/d73/ffd x:0 0 0 2024-04-12T20:56:38.392 INFO:tasks.workunit.client.1.smithi186.stdout:8/629: symlink d7/d18/d36/d1d/dc3/lca 0 2024-04-12T20:56:38.393 INFO:tasks.workunit.client.1.smithi186.stdout:3/617: write d1/d3/da/f56 [4212621,72189] 0 2024-04-12T20:56:38.398 INFO:tasks.workunit.client.1.smithi186.stdout:6/636: symlink d9/d5e/d7a/le7 0 2024-04-12T20:56:38.403 INFO:tasks.workunit.client.1.smithi186.stdout:3/618: chown d1/d3/d21/d22/d24 12258411 1 2024-04-12T20:56:38.405 INFO:tasks.workunit.client.1.smithi186.stdout:4/764: mknod d0/d59/dc0/d48/dce/cfe 0 2024-04-12T20:56:38.410 INFO:tasks.workunit.client.1.smithi186.stdout:6/637: mknod d9/ce8 0 2024-04-12T20:56:38.410 INFO:tasks.workunit.client.1.smithi186.stdout:3/619: stat d1/d2c/d64/c89 0 2024-04-12T20:56:38.412 INFO:tasks.workunit.client.1.smithi186.stdout:8/630: getdents d7/de/d68/da7 0 2024-04-12T20:56:38.414 INFO:tasks.workunit.client.1.smithi186.stdout:1/655: dwrite d14/d1a/d33/d36/d5b/d3e/d5f/f8f [0,4194304] 0 2024-04-12T20:56:38.414 INFO:tasks.workunit.client.1.smithi186.stdout:4/765: write d0/d21/f6a [997372,13859] 0 2024-04-12T20:56:38.415 INFO:tasks.workunit.client.1.smithi186.stdout:9/595: dwrite d3/d2f/d2e/d33/f44 [0,4194304] 0 2024-04-12T20:56:38.416 INFO:tasks.workunit.client.1.smithi186.stdout:6/638: chown d9/d4c/fb3 35881557 1 2024-04-12T20:56:38.419 INFO:tasks.workunit.client.1.smithi186.stdout:2/733: dwrite d0/d3/d1a/d21/d26/d86/fd4 [0,4194304] 0 2024-04-12T20:56:38.421 INFO:tasks.workunit.client.1.smithi186.stdout:0/743: link d0/d14/d4e/d50/d6e/d96/c9d d0/d10/d72/cf2 0 2024-04-12T20:56:38.425 INFO:tasks.workunit.client.1.smithi186.stdout:8/631: read d7/de/d7a/f80 [464002,60558] 0 2024-04-12T20:56:38.433 INFO:tasks.workunit.client.1.smithi186.stdout:4/766: rename d0/fae to d0/dc/d64/d87/dd2/fff 0 2024-04-12T20:56:38.435 INFO:tasks.workunit.client.1.smithi186.stdout:0/744: read d0/d10/d72/d79/fd0 [2407774,93515] 0 2024-04-12T20:56:38.443 INFO:tasks.workunit.client.1.smithi186.stdout:7/650: link d7/d10/d2d/d40/d4a/d6c/d97/d4d/d8a/ld5 d7/d10/d1c/d5d/lea 0 2024-04-12T20:56:38.449 INFO:tasks.workunit.client.1.smithi186.stdout:8/632: dwrite d7/d18/f1a [4194304,4194304] 0 2024-04-12T20:56:38.454 INFO:tasks.workunit.client.1.smithi186.stdout:1/656: rename fe to d14/d1a/d21/d35/d59/fd2 0 2024-04-12T20:56:38.455 INFO:tasks.workunit.client.1.smithi186.stdout:4/767: unlink d0/c7f 0 2024-04-12T20:56:38.457 INFO:tasks.workunit.client.1.smithi186.stdout:7/651: mkdir d7/deb 0 2024-04-12T20:56:38.462 INFO:tasks.workunit.client.1.smithi186.stdout:4/768: write d0/dc/d88/d80/fdc [896886,64017] 0 2024-04-12T20:56:38.466 INFO:tasks.workunit.client.1.smithi186.stdout:0/745: rename d0/c45 to d0/d14/d4e/d50/d6e/cf3 0 2024-04-12T20:56:38.471 INFO:tasks.workunit.client.1.smithi186.stdout:7/652: symlink d7/d10/d1c/d5d/lec 0 2024-04-12T20:56:38.471 INFO:tasks.workunit.client.1.smithi186.stdout:6/639: dread d9/f4b [0,4194304] 0 2024-04-12T20:56:38.473 INFO:tasks.workunit.client.1.smithi186.stdout:5/660: dwrite d5/db/d15/d2a/d2d/d48/f3a [0,4194304] 0 2024-04-12T20:56:38.477 INFO:tasks.workunit.client.1.smithi186.stdout:3/620: write d1/d3/da2/f9d [2039470,78071] 0 2024-04-12T20:56:38.479 INFO:tasks.workunit.client.1.smithi186.stdout:9/596: dread d3/f10 [8388608,4194304] 0 2024-04-12T20:56:38.479 INFO:tasks.workunit.client.1.smithi186.stdout:0/746: mknod d0/d5c/d66/da7/cf4 0 2024-04-12T20:56:38.480 INFO:tasks.workunit.client.1.smithi186.stdout:3/621: stat d1/d2c/d48/fab 0 2024-04-12T20:56:38.483 INFO:tasks.workunit.client.1.smithi186.stdout:3/622: write d1/d3/d49/d4d/f51 [1313198,77132] 0 2024-04-12T20:56:38.483 INFO:tasks.workunit.client.1.smithi186.stdout:8/633: rename d7/d18/d36/d1d/d7d/fc5 to d7/de/d68/fcb 0 2024-04-12T20:56:38.487 INFO:tasks.workunit.client.1.smithi186.stdout:4/769: unlink d0/d59/dc0/d48/d61/fe8 0 2024-04-12T20:56:38.493 INFO:tasks.workunit.client.1.smithi186.stdout:9/597: readlink d3/d12/d6f/d83/d71/d72/lc8 0 2024-04-12T20:56:38.493 INFO:tasks.workunit.client.1.smithi186.stdout:6/640: unlink d9/l41 0 2024-04-12T20:56:38.498 INFO:tasks.workunit.client.1.smithi186.stdout:5/661: creat d5/db/d2e/dd0/fdb x:0 0 0 2024-04-12T20:56:38.499 INFO:tasks.workunit.client.1.smithi186.stdout:7/653: creat d7/d10/d2d/d40/d4a/d6c/ddf/fed x:0 0 0 2024-04-12T20:56:38.499 INFO:tasks.workunit.client.1.smithi186.stdout:8/634: symlink d7/d18/d36/dc9/lcc 0 2024-04-12T20:56:38.502 INFO:tasks.workunit.client.1.smithi186.stdout:9/598: mknod d3/d2f/d54/d9f/cca 0 2024-04-12T20:56:38.503 INFO:tasks.workunit.client.1.smithi186.stdout:1/657: dwrite d14/d1a/d33/d36/d5b/d3e/d8b/f8e [0,4194304] 0 2024-04-12T20:56:38.505 INFO:tasks.workunit.client.1.smithi186.stdout:8/635: write d7/de/d7a/f31 [3039044,52910] 0 2024-04-12T20:56:38.506 INFO:tasks.workunit.client.1.smithi186.stdout:0/747: dwrite d0/d14/d4e/d1e/f4b [0,4194304] 0 2024-04-12T20:56:38.508 INFO:tasks.workunit.client.1.smithi186.stdout:7/654: chown d7/d10/d2d/d67/fd9 736579 1 2024-04-12T20:56:38.509 INFO:tasks.workunit.client.1.smithi186.stdout:2/734: truncate d0/d3/d1a/d27/f59 3334733 0 2024-04-12T20:56:38.522 INFO:tasks.workunit.client.1.smithi186.stdout:5/662: creat d5/db/d15/d2a/d2d/d77/fdc x:0 0 0 2024-04-12T20:56:38.522 INFO:tasks.workunit.client.1.smithi186.stdout:3/623: dwrite d1/f1e [4194304,4194304] 0 2024-04-12T20:56:38.525 INFO:tasks.workunit.client.1.smithi186.stdout:7/655: truncate d7/d10/d1c/d8c/fb5 5242190 0 2024-04-12T20:56:38.531 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:56:38 smithi186 ceph-mon[36597]: Upgrade: Updating mgr.smithi018.xjzwga 2024-04-12T20:56:38.531 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:56:38 smithi186 ceph-mon[36597]: Deploying daemon mgr.smithi018.xjzwga on smithi018 2024-04-12T20:56:38.541 INFO:tasks.workunit.client.1.smithi186.stdout:1/658: rename d14/fa8 to d14/d1a/d33/fd3 0 2024-04-12T20:56:38.546 INFO:tasks.workunit.client.1.smithi186.stdout:6/641: write d9/d43/d2d/f5f [2179361,33754] 0 2024-04-12T20:56:38.554 INFO:tasks.workunit.client.1.smithi186.stdout:8/636: rename d7/d18/d36/d20/d23/f45 to d7/de/d7a/fcd 0 2024-04-12T20:56:38.563 INFO:tasks.workunit.client.1.smithi186.stdout:0/748: dwrite d0/d10/d32/d3f/f64 [0,4194304] 0 2024-04-12T20:56:38.565 INFO:tasks.workunit.client.1.smithi186.stdout:6/642: dread - d9/d43/d59/d5d/fbf zero size 2024-04-12T20:56:38.565 INFO:tasks.workunit.client.1.smithi186.stdout:7/656: write d7/d10/d1c/fdb [441630,110423] 0 2024-04-12T20:56:38.566 INFO:tasks.workunit.client.1.smithi186.stdout:8/637: chown d7/d18/d36/d1d 959 1 2024-04-12T20:56:38.570 INFO:tasks.workunit.client.1.smithi186.stdout:2/735: dwrite d0/dc/d2e/d73/fd8 [0,4194304] 0 2024-04-12T20:56:38.570 INFO:tasks.workunit.client.1.smithi186.stdout:9/599: dwrite d3/d2f/d2e/d43/d4c/f89 [0,4194304] 0 2024-04-12T20:56:38.576 INFO:tasks.workunit.client.1.smithi186.stdout:3/624: dwrite d1/d3/da2/f9d [0,4194304] 0 2024-04-12T20:56:38.579 INFO:tasks.workunit.client.1.smithi186.stdout:5/663: dwrite d5/db/d33/f36 [0,4194304] 0 2024-04-12T20:56:38.579 INFO:tasks.workunit.client.1.smithi186.stdout:2/736: dread - d0/f79 zero size 2024-04-12T20:56:38.585 INFO:tasks.workunit.client.1.smithi186.stdout:5/664: write d5/db/d15/f1e [4947068,35068] 0 2024-04-12T20:56:38.589 INFO:tasks.workunit.client.1.smithi186.stdout:7/657: rename d7/de1/fb0 to d7/d10/d1c/d5d/fee 0 2024-04-12T20:56:38.593 INFO:tasks.workunit.client.1.smithi186.stdout:2/737: creat d0/d32/fe5 x:0 0 0 2024-04-12T20:56:38.596 INFO:tasks.workunit.client.1.smithi186.stdout:6/643: mkdir d9/d4c/de9 0 2024-04-12T20:56:38.596 INFO:tasks.workunit.client.1.smithi186.stdout:1/659: truncate f4 760717 0 2024-04-12T20:56:38.600 INFO:tasks.workunit.client.1.smithi186.stdout:0/749: rename d0/d14/c75 to d0/d14/d4e/d50/d6e/d96/cf5 0 2024-04-12T20:56:38.602 INFO:tasks.workunit.client.1.smithi186.stdout:7/658: symlink d7/d10/d2d/d40/d4a/d6c/d97/d4d/lef 0 2024-04-12T20:56:38.603 INFO:tasks.workunit.client.1.smithi186.stdout:2/738: rmdir d0/d32/d2a 39 2024-04-12T20:56:38.608 INFO:tasks.workunit.client.1.smithi186.stdout:2/739: write d0/d3/faf [422423,102396] 0 2024-04-12T20:56:38.609 INFO:tasks.workunit.client.1.smithi186.stdout:6/644: mknod d9/d4c/cea 0 2024-04-12T20:56:38.611 INFO:tasks.workunit.client.1.smithi186.stdout:0/750: write d0/d5c/d66/da7/fb4 [200755,8853] 0 2024-04-12T20:56:38.614 INFO:tasks.workunit.client.1.smithi186.stdout:5/665: dwrite d5/db/d15/d2a/d2d/d48/fb5 [0,4194304] 0 2024-04-12T20:56:38.614 INFO:tasks.workunit.client.1.smithi186.stdout:3/625: rename d1/d3/d49/d4d/d6f/c88 to d1/d3/da/cca 0 2024-04-12T20:56:38.616 INFO:tasks.workunit.client.1.smithi186.stdout:6/645: chown d9/d43/d59/d5d/d6e/lb1 539969 1 2024-04-12T20:56:38.616 INFO:tasks.workunit.client.1.smithi186.stdout:1/660: unlink d14/d1a/d33/d36/d5b/d3e/c48 0 2024-04-12T20:56:38.620 INFO:tasks.workunit.client.1.smithi186.stdout:2/740: creat d0/d3e/fe6 x:0 0 0 2024-04-12T20:56:38.620 INFO:tasks.workunit.client.1.smithi186.stdout:3/626: fdatasync d1/d3/d21/fb2 0 2024-04-12T20:56:38.623 INFO:tasks.workunit.client.1.smithi186.stdout:1/661: truncate d14/d6c/fcb 249646 0 2024-04-12T20:56:38.625 INFO:tasks.workunit.client.1.smithi186.stdout:8/638: dread d7/d18/d36/d1d/d1f/d26/fb8 [0,4194304] 0 2024-04-12T20:56:38.630 INFO:tasks.workunit.client.1.smithi186.stdout:2/741: symlink d0/d3/le7 0 2024-04-12T20:56:38.633 INFO:tasks.workunit.client.1.smithi186.stdout:9/600: dread d3/d2f/d2e/d33/f4b [0,4194304] 0 2024-04-12T20:56:38.633 INFO:tasks.workunit.client.1.smithi186.stdout:7/659: rename d7/c1d to d7/d10/d2d/d40/d4a/d6c/d97/d4d/d8a/cf0 0 2024-04-12T20:56:38.635 INFO:tasks.workunit.client.1.smithi186.stdout:7/660: chown d7/d10 60 1 2024-04-12T20:56:38.637 INFO:tasks.workunit.client.1.smithi186.stdout:4/770: sync 2024-04-12T20:56:38.637 INFO:tasks.workunit.client.1.smithi186.stdout:5/666: mknod d5/db/d15/d2a/d6d/d7f/d86/db6/cdd 0 2024-04-12T20:56:38.639 INFO:tasks.workunit.client.1.smithi186.stdout:4/771: write d0/d21/f31 [5534987,62340] 0 2024-04-12T20:56:38.640 INFO:tasks.workunit.client.1.smithi186.stdout:5/667: chown d5/db/d15/d2a/d2d/f9d 486678 1 2024-04-12T20:56:38.641 INFO:tasks.workunit.client.1.smithi186.stdout:6/646: getdents d9/d5e/d7a/d81/dc5 0 2024-04-12T20:56:38.644 INFO:tasks.workunit.client.1.smithi186.stdout:1/662: rmdir d14/d1a/d33/d36 39 2024-04-12T20:56:38.644 INFO:tasks.workunit.client.1.smithi186.stdout:6/647: chown d9/d4c/d6f/ccf 9774 1 2024-04-12T20:56:38.651 INFO:tasks.workunit.client.1.smithi186.stdout:0/751: write d0/d10/f2c [233093,59425] 0 2024-04-12T20:56:38.653 INFO:tasks.workunit.client.1.smithi186.stdout:8/639: symlink d7/d18/d36/d1d/d7d/da5/lce 0 2024-04-12T20:56:38.654 INFO:tasks.workunit.client.1.smithi186.stdout:2/742: mkdir d0/dc/d2e/de8 0 2024-04-12T20:56:38.656 INFO:tasks.workunit.client.1.smithi186.stdout:2/743: write d0/dc/f40 [9139887,73300] 0 2024-04-12T20:56:38.656 INFO:tasks.workunit.client.1.smithi186.stdout:4/772: symlink d0/dc/d64/d87/l100 0 2024-04-12T20:56:38.659 INFO:tasks.workunit.client.1.smithi186.stdout:2/744: write d0/dc/f40 [9023955,63403] 0 2024-04-12T20:56:38.659 INFO:tasks.workunit.client.1.smithi186.stdout:3/627: dread d1/d3/da/f6c [0,4194304] 0 2024-04-12T20:56:38.660 INFO:tasks.workunit.client.1.smithi186.stdout:6/648: rmdir d9/d43/d2d/d4d 39 2024-04-12T20:56:38.669 INFO:tasks.workunit.client.1.smithi186.stdout:7/661: dread d7/f17 [4194304,4194304] 0 2024-04-12T20:56:38.678 INFO:tasks.workunit.client.1.smithi186.stdout:7/662: dread d7/d10/d1c/f2c [8388608,4194304] 0 2024-04-12T20:56:38.700 INFO:tasks.workunit.client.1.smithi186.stdout:8/640: symlink d7/d18/d36/d1d/d7d/da5/lcf 0 2024-04-12T20:56:38.702 INFO:tasks.workunit.client.1.smithi186.stdout:4/773: readlink d0/d59/dc0/led 0 2024-04-12T20:56:38.704 INFO:tasks.workunit.client.1.smithi186.stdout:5/668: fsync d5/db/d15/d2a/d6d/dbb/dc4/fa9 0 2024-04-12T20:56:38.704 INFO:tasks.workunit.client.1.smithi186.stdout:4/774: read d0/dc/d33/fb [2480489,106141] 0 2024-04-12T20:56:38.706 INFO:tasks.workunit.client.1.smithi186.stdout:3/628: write d1/fc [577956,93151] 0 2024-04-12T20:56:38.706 INFO:tasks.workunit.client.1.smithi186.stdout:6/649: symlink d9/da9/dbd/leb 0 2024-04-12T20:56:38.707 INFO:tasks.workunit.client.1.smithi186.stdout:4/775: chown d0/d59/dc0/d48/f95 4 1 2024-04-12T20:56:38.708 INFO:tasks.workunit.client.1.smithi186.stdout:2/745: mkdir d0/d3/d1a/d41/d8f/de9 0 2024-04-12T20:56:38.711 INFO:tasks.workunit.client.1.smithi186.stdout:7/663: write d7/d10/d2d/d40/f77 [3877692,94272] 0 2024-04-12T20:56:38.715 INFO:tasks.workunit.client.1.smithi186.stdout:7/664: chown d7/d10/d2d/d40/d4a/d6c/d97/l7c 0 1 2024-04-12T20:56:38.715 INFO:tasks.workunit.client.1.smithi186.stdout:5/669: mknod d5/d27/cde 0 2024-04-12T20:56:38.717 INFO:tasks.workunit.client.1.smithi186.stdout:5/670: fsync d5/db/fbd 0 2024-04-12T20:56:38.719 INFO:tasks.workunit.client.1.smithi186.stdout:5/671: write d5/db/fc [5843617,54747] 0 2024-04-12T20:56:38.720 INFO:tasks.workunit.client.1.smithi186.stdout:8/641: symlink d7/d18/d36/d1d/d7d/d81/ld0 0 2024-04-12T20:56:38.723 INFO:tasks.workunit.client.1.smithi186.stdout:3/629: dwrite d1/d2c/d48/fab [0,4194304] 0 2024-04-12T20:56:38.724 INFO:tasks.workunit.client.1.smithi186.stdout:2/746: symlink d0/lea 0 2024-04-12T20:56:38.724 INFO:tasks.workunit.client.1.smithi186.stdout:3/630: fsync d1/d3/da/f70 0 2024-04-12T20:56:38.724 INFO:tasks.workunit.client.1.smithi186.stdout:4/776: dread d0/dc/f3b [0,4194304] 0 2024-04-12T20:56:38.728 INFO:tasks.workunit.client.1.smithi186.stdout:5/672: truncate d5/db/f2b 425520 0 2024-04-12T20:56:38.731 INFO:tasks.workunit.client.1.smithi186.stdout:2/747: readlink d0/d3/l16 0 2024-04-12T20:56:38.732 INFO:tasks.workunit.client.1.smithi186.stdout:0/752: link d0/d14/cc2 d0/d14/d4e/d1e/d84/dec/cf6 0 2024-04-12T20:56:38.732 INFO:tasks.workunit.client.1.smithi186.stdout:4/777: dread - d0/d59/dc0/d48/f95 zero size 2024-04-12T20:56:38.735 INFO:tasks.workunit.client.1.smithi186.stdout:3/631: creat d1/d3/d49/d4d/d9c/fcb x:0 0 0 2024-04-12T20:56:38.736 INFO:tasks.workunit.client.1.smithi186.stdout:7/665: creat d7/d10/d2d/d40/ff1 x:0 0 0 2024-04-12T20:56:38.737 INFO:tasks.workunit.client.1.smithi186.stdout:0/753: truncate d0/d10/d32/fe9 761089 0 2024-04-12T20:56:38.738 INFO:tasks.workunit.client.1.smithi186.stdout:2/748: dread d0/d3/faf [0,4194304] 0 2024-04-12T20:56:38.738 INFO:tasks.workunit.client.1.smithi186.stdout:6/650: rmdir d9/da9/dbd 39 2024-04-12T20:56:38.742 INFO:tasks.workunit.client.1.smithi186.stdout:3/632: creat d1/d2c/d54/fcc x:0 0 0 2024-04-12T20:56:38.743 INFO:tasks.workunit.client.1.smithi186.stdout:9/601: dwrite d3/d2f/d54/d9f/dc4/f5a [0,4194304] 0 2024-04-12T20:56:38.743 INFO:tasks.workunit.client.1.smithi186.stdout:1/663: sync 2024-04-12T20:56:38.744 INFO:tasks.workunit.client.1.smithi186.stdout:1/664: fsync d14/d91/fc1 0 2024-04-12T20:56:38.746 INFO:tasks.workunit.client.1.smithi186.stdout:9/602: truncate d3/d13/fb7 171518 0 2024-04-12T20:56:38.747 INFO:tasks.workunit.client.1.smithi186.stdout:5/673: symlink d5/db/d15/d2a/d4c/dc0/ldf 0 2024-04-12T20:56:38.752 INFO:tasks.workunit.client.1.smithi186.stdout:0/754: creat d0/d5c/d65/ff7 x:0 0 0 2024-04-12T20:56:38.755 INFO:tasks.workunit.client.1.smithi186.stdout:4/778: creat d0/d59/ddf/f101 x:0 0 0 2024-04-12T20:56:38.758 INFO:tasks.workunit.client.1.smithi186.stdout:7/666: truncate d7/d10/d2d/d67/fe4 1021707 0 2024-04-12T20:56:38.759 INFO:tasks.workunit.client.1.smithi186.stdout:2/749: dwrite d0/d3/d1a/d48/f84 [0,4194304] 0 2024-04-12T20:56:38.760 INFO:tasks.workunit.client.1.smithi186.stdout:7/667: dread - d7/d10/d1c/d5d/fee zero size 2024-04-12T20:56:38.762 INFO:tasks.workunit.client.1.smithi186.stdout:8/642: rename d7/de/l6c to d7/d18/d36/d20/ld1 0 2024-04-12T20:56:38.764 INFO:tasks.workunit.client.1.smithi186.stdout:9/603: mknod d3/d2f/d2e/d33/d36/d70/ccb 0 2024-04-12T20:56:38.766 INFO:tasks.workunit.client.1.smithi186.stdout:9/604: chown d3 0 1 2024-04-12T20:56:38.770 INFO:tasks.workunit.client.1.smithi186.stdout:6/651: rename d9/d43/l3b to d9/d5e/d7a/d81/lec 0 2024-04-12T20:56:38.772 INFO:tasks.workunit.client.1.smithi186.stdout:0/755: dwrite d0/d5c/d65/fd2 [0,4194304] 0 2024-04-12T20:56:38.772 INFO:tasks.workunit.client.1.smithi186.stdout:6/652: read - d9/d5e/d73/f84 zero size 2024-04-12T20:56:38.775 INFO:tasks.workunit.client.1.smithi186.stdout:4/779: mkdir d0/dc/d88/d80/d9c/d102 0 2024-04-12T20:56:38.776 INFO:tasks.workunit.client.1.smithi186.stdout:4/780: chown d0 298315062 1 2024-04-12T20:56:38.777 INFO:tasks.workunit.client.1.smithi186.stdout:2/750: mknod d0/dc/d2e/de8/ceb 0 2024-04-12T20:56:38.778 INFO:tasks.workunit.client.1.smithi186.stdout:7/668: dwrite d7/d10/d1c/f20 [4194304,4194304] 0 2024-04-12T20:56:38.780 INFO:tasks.workunit.client.1.smithi186.stdout:9/605: read d3/d12/d6f/d83/f38 [3212609,91578] 0 2024-04-12T20:56:38.783 INFO:tasks.workunit.client.1.smithi186.stdout:4/781: dread d0/dc/d33/f8a [0,4194304] 0 2024-04-12T20:56:38.785 INFO:tasks.workunit.client.1.smithi186.stdout:0/756: rmdir d0/d14/d4e/d50/d6e/d96 39 2024-04-12T20:56:38.785 INFO:tasks.workunit.client.1.smithi186.stdout:9/606: truncate d3/d2f/d2e/d33/d36/f53 5198450 0 2024-04-12T20:56:38.786 INFO:tasks.workunit.client.1.smithi186.stdout:0/757: fsync d0/d5c/fc8 0 2024-04-12T20:56:38.789 INFO:tasks.workunit.client.1.smithi186.stdout:2/751: symlink d0/d3/lec 0 2024-04-12T20:56:38.791 INFO:tasks.workunit.client.1.smithi186.stdout:8/643: creat d7/d18/d36/d1d/d1f/fd2 x:0 0 0 2024-04-12T20:56:38.793 INFO:tasks.workunit.client.1.smithi186.stdout:8/644: read d7/d18/d36/fa8 [235419,88784] 0 2024-04-12T20:56:38.794 INFO:tasks.workunit.client.1.smithi186.stdout:8/645: fsync d7/fb3 0 2024-04-12T20:56:38.801 INFO:tasks.workunit.client.1.smithi186.stdout:4/782: unlink d0/d59/dc0/fad 0 2024-04-12T20:56:38.801 INFO:tasks.workunit.client.1.smithi186.stdout:5/674: write d5/db/d15/d2a/d2d/d46/d55/f37 [601347,70417] 0 2024-04-12T20:56:38.804 INFO:tasks.workunit.client.1.smithi186.stdout:0/758: mkdir d0/d5c/d66/df8 0 2024-04-12T20:56:38.805 INFO:tasks.workunit.client.1.smithi186.stdout:5/675: fsync d5/db/d15/d2a/d2d/d46/d55/fd5 0 2024-04-12T20:56:38.805 INFO:tasks.workunit.client.1.smithi186.stdout:6/653: getdents d9/d43/d59/d5d/de6 0 2024-04-12T20:56:38.808 INFO:tasks.workunit.client.1.smithi186.stdout:2/752: mknod d0/d3/d1a/d21/d26/ced 0 2024-04-12T20:56:38.809 INFO:tasks.workunit.client.1.smithi186.stdout:0/759: read d0/d14/f19 [434634,76387] 0 2024-04-12T20:56:38.811 INFO:tasks.workunit.client.1.smithi186.stdout:3/633: rename d1/d3/d21/d22/d24/f60 to d1/d3/d21/d22/fcd 0 2024-04-12T20:56:38.812 INFO:tasks.workunit.client.1.smithi186.stdout:7/669: link d7/d10/d1c/d8c/fb5 d7/d10/d2d/d40/d4a/ff2 0 2024-04-12T20:56:38.814 INFO:tasks.workunit.client.1.smithi186.stdout:9/607: mknod d3/d2f/d54/d7d/ccc 0 2024-04-12T20:56:38.814 INFO:tasks.workunit.client.1.smithi186.stdout:0/760: truncate d0/d14/d4e/d50/fd5 600472 0 2024-04-12T20:56:38.816 INFO:tasks.workunit.client.1.smithi186.stdout:9/608: chown d3/d12/l42 6596 1 2024-04-12T20:56:38.817 INFO:tasks.workunit.client.1.smithi186.stdout:5/676: creat d5/db/d15/d2a/d6d/fe0 x:0 0 0 2024-04-12T20:56:38.817 INFO:tasks.workunit.client.1.smithi186.stdout:6/654: creat d9/d5e/d7a/d81/d96/ddb/fed x:0 0 0 2024-04-12T20:56:38.819 INFO:tasks.workunit.client.1.smithi186.stdout:8/646: dread d7/de/d7a/f80 [0,4194304] 0 2024-04-12T20:56:38.819 INFO:tasks.workunit.client.1.smithi186.stdout:4/783: dwrite d0/dc/d33/f43 [4194304,4194304] 0 2024-04-12T20:56:38.826 INFO:tasks.workunit.client.1.smithi186.stdout:1/665: rename d14/d1a/d33/d36/d5b/d3e/c65 to d14/d1a/d33/d36/d5b/d6a/cd4 0 2024-04-12T20:56:38.827 INFO:tasks.workunit.client.1.smithi186.stdout:1/666: chown d14/d1a/d3c 88 1 2024-04-12T20:56:38.828 INFO:tasks.workunit.client.1.smithi186.stdout:1/667: dread - d14/fcf zero size 2024-04-12T20:56:38.828 INFO:tasks.workunit.client.1.smithi186.stdout:4/784: creat d0/d59/dc0/d48/d61/f103 x:0 0 0 2024-04-12T20:56:38.829 INFO:tasks.workunit.client.1.smithi186.stdout:1/668: chown d14/d1a/d3c/c5c 6688 1 2024-04-12T20:56:38.832 INFO:tasks.workunit.client.1.smithi186.stdout:0/761: mknod d0/d5c/d66/df8/cf9 0 2024-04-12T20:56:38.833 INFO:tasks.workunit.client.1.smithi186.stdout:2/753: rename d0/d3/d1a/d21/f77 to d0/d3/d1a/d41/fee 0 2024-04-12T20:56:38.835 INFO:tasks.workunit.client.1.smithi186.stdout:5/677: mkdir d5/db/d15/d2a/d6d/dbb/dc4/d5d/de1 0 2024-04-12T20:56:38.835 INFO:tasks.workunit.client.1.smithi186.stdout:2/754: dread - d0/d3/d1a/d48/fe1 zero size 2024-04-12T20:56:38.838 INFO:tasks.workunit.client.1.smithi186.stdout:5/678: chown d5/db/d15/d2a/d4c/dc0/ldf 9505 1 2024-04-12T20:56:38.841 INFO:tasks.workunit.client.1.smithi186.stdout:1/669: fdatasync d14/d1a/d33/d36/d5b/f67 0 2024-04-12T20:56:38.842 INFO:tasks.workunit.client.1.smithi186.stdout:3/634: rename d1/d3/d49/d4d/f71 to d1/d3/d49/d4d/d6f/da6/fce 0 2024-04-12T20:56:38.859 INFO:tasks.workunit.client.1.smithi186.stdout:5/679: mkdir d5/db/d2e/d70/de2 0 2024-04-12T20:56:38.861 INFO:tasks.workunit.client.1.smithi186.stdout:1/670: mkdir d14/d1a/d21/d35/dbc/dd5 0 2024-04-12T20:56:38.861 INFO:tasks.workunit.client.1.smithi186.stdout:8/647: getdents d7/d18/d36/d1d/d7d 0 2024-04-12T20:56:38.863 INFO:tasks.workunit.client.1.smithi186.stdout:9/609: rename d3/d2f/f26 to d3/d12/fcd 0 2024-04-12T20:56:38.865 INFO:tasks.workunit.client.1.smithi186.stdout:1/671: write d14/d1a/d33/d36/d5b/d3e/d71/f82 [1593946,30537] 0 2024-04-12T20:56:38.866 INFO:tasks.workunit.client.1.smithi186.stdout:3/635: creat d1/d3/d49/d4d/dad/fcf x:0 0 0 2024-04-12T20:56:38.867 INFO:tasks.workunit.client.1.smithi186.stdout:2/755: getdents d0/d32/d8b 0 2024-04-12T20:56:38.867 INFO:tasks.workunit.client.1.smithi186.stdout:9/610: fdatasync d3/d12/d6f/d83/f35 0 2024-04-12T20:56:38.870 INFO:tasks.workunit.client.1.smithi186.stdout:9/611: chown d3/d2f/d2e/d33/d9d 59678018 1 2024-04-12T20:56:38.871 INFO:tasks.workunit.client.1.smithi186.stdout:0/762: dread d0/d10/d32/d3f/f54 [0,4194304] 0 2024-04-12T20:56:38.873 INFO:tasks.workunit.client.1.smithi186.stdout:8/648: readlink d7/d18/d36/d1d/d1f/da2/d5c/l78 0 2024-04-12T20:56:38.874 INFO:tasks.workunit.client.1.smithi186.stdout:8/649: chown d7/d18/d36/d1d/d7d/da5 19328533 1 2024-04-12T20:56:38.875 INFO:tasks.workunit.client.1.smithi186.stdout:5/680: truncate d5/db/d15/d2a/d2d/d46/f83 624783 0 2024-04-12T20:56:38.881 INFO:tasks.workunit.client.1.smithi186.stdout:9/612: chown d3/d2f/d2e/c69 0 1 2024-04-12T20:56:38.881 INFO:tasks.workunit.client.1.smithi186.stdout:9/613: chown d3/f3e 24449 1 2024-04-12T20:56:38.883 INFO:tasks.workunit.client.1.smithi186.stdout:2/756: dwrite d0/d3/d1a/d27/fd0 [0,4194304] 0 2024-04-12T20:56:38.884 INFO:tasks.workunit.client.1.smithi186.stdout:8/650: unlink d7/d18/d36/d1d/d1f/d26/lba 0 2024-04-12T20:56:38.885 INFO:tasks.workunit.client.1.smithi186.stdout:6/655: rename d9/d43/d59/d5d/d6e/fb7 to d9/da9/fee 0 2024-04-12T20:56:38.887 INFO:tasks.workunit.client.1.smithi186.stdout:6/656: write d9/d43/d50/f5b [2631750,922] 0 2024-04-12T20:56:38.888 INFO:tasks.workunit.client.1.smithi186.stdout:1/672: link d14/d1a/d33/d36/d5b/d3e/d71/f76 d14/d1a/d33/d36/fd6 0 2024-04-12T20:56:38.890 INFO:tasks.workunit.client.1.smithi186.stdout:5/681: mkdir d5/db/d15/d2a/d2d/de3 0 2024-04-12T20:56:38.892 INFO:tasks.workunit.client.1.smithi186.stdout:6/657: write d9/d4c/d6f/fa2 [118353,38429] 0 2024-04-12T20:56:38.894 INFO:tasks.workunit.client.1.smithi186.stdout:6/658: truncate d9/d43/d59/fb8 390388 0 2024-04-12T20:56:38.896 INFO:tasks.workunit.client.1.smithi186.stdout:2/757: symlink d0/dc/d2e/de8/lef 0 2024-04-12T20:56:38.901 INFO:tasks.workunit.client.1.smithi186.stdout:7/670: rename d7/d10/d2d/d86/cc9 to d7/d10/d2d/d40/d4a/d6c/d97/dd0/cf3 0 2024-04-12T20:56:38.909 INFO:tasks.workunit.client.1.smithi186.stdout:0/763: creat d0/d14/d4e/d1e/ffa x:0 0 0 2024-04-12T20:56:38.911 INFO:tasks.workunit.client.1.smithi186.stdout:2/758: symlink d0/d3/d1a/d24/lf0 0 2024-04-12T20:56:38.911 INFO:tasks.workunit.client.1.smithi186.stdout:7/671: rmdir d7/d10/d2d/d40/d4a/d6c/d97/dd0 39 2024-04-12T20:56:38.913 INFO:tasks.workunit.client.1.smithi186.stdout:9/614: creat d3/fce x:0 0 0 2024-04-12T20:56:38.914 INFO:tasks.workunit.client.1.smithi186.stdout:8/651: dwrite d7/de/d5e/fc6 [4194304,4194304] 0 2024-04-12T20:56:38.915 INFO:tasks.workunit.client.1.smithi186.stdout:5/682: fdatasync d5/d27/f60 0 2024-04-12T20:56:38.917 INFO:tasks.workunit.client.1.smithi186.stdout:1/673: creat d14/d1a/d33/d36/d5b/d3e/fd7 x:0 0 0 2024-04-12T20:56:38.918 INFO:tasks.workunit.client.1.smithi186.stdout:6/659: dread d9/d5e/f6d [0,4194304] 0 2024-04-12T20:56:38.918 INFO:tasks.workunit.client.1.smithi186.stdout:8/652: write d7/fb3 [563224,73962] 0 2024-04-12T20:56:38.920 INFO:tasks.workunit.client.1.smithi186.stdout:2/759: creat d0/d3/d7a/ff1 x:0 0 0 2024-04-12T20:56:38.930 INFO:tasks.workunit.client.1.smithi186.stdout:2/760: fdatasync d0/dc/d2e/da9/fc7 0 2024-04-12T20:56:38.931 INFO:tasks.workunit.client.1.smithi186.stdout:1/674: mknod d14/d1a/d33/d36/d5b/d3e/dbb/cd8 0 2024-04-12T20:56:38.934 INFO:tasks.workunit.client.1.smithi186.stdout:2/761: dread - d0/d3/d7a/d8d/fa4 zero size 2024-04-12T20:56:38.935 INFO:tasks.workunit.client.1.smithi186.stdout:7/672: truncate d7/d10/d2d/d40/fc4 1870890 0 2024-04-12T20:56:38.938 INFO:tasks.workunit.client.1.smithi186.stdout:1/675: symlink d14/d1a/d58/ld9 0 2024-04-12T20:56:38.940 INFO:tasks.workunit.client.1.smithi186.stdout:2/762: read d0/dc/f55 [6895809,59075] 0 2024-04-12T20:56:38.941 INFO:tasks.workunit.client.1.smithi186.stdout:4/785: sync 2024-04-12T20:56:38.942 INFO:tasks.workunit.client.1.smithi186.stdout:1/676: chown d14/d1a/d21/d35/d59 1608 1 2024-04-12T20:56:38.952 INFO:tasks.workunit.client.1.smithi186.stdout:4/786: fdatasync d0/d21/f40 0 2024-04-12T20:56:38.952 INFO:tasks.workunit.client.1.smithi186.stdout:8/653: dwrite d7/d18/d36/d20/d71/fc0 [0,4194304] 0 2024-04-12T20:56:38.953 INFO:tasks.workunit.client.1.smithi186.stdout:6/660: dwrite d9/d43/d2d/d4d/f64 [0,4194304] 0 2024-04-12T20:56:38.955 INFO:tasks.workunit.client.1.smithi186.stdout:5/683: dwrite d5/db/d15/d2a/d2d/d48/f4b [0,4194304] 0 2024-04-12T20:56:38.961 INFO:tasks.workunit.client.1.smithi186.stdout:3/636: sync 2024-04-12T20:56:38.964 INFO:tasks.workunit.client.1.smithi186.stdout:8/654: write d7/d18/d36/d1d/d7d/da5/f7c [3968369,77803] 0 2024-04-12T20:56:38.964 INFO:tasks.workunit.client.1.smithi186.stdout:0/764: dread d0/d5c/d65/f7b [0,4194304] 0 2024-04-12T20:56:38.966 INFO:tasks.workunit.client.1.smithi186.stdout:2/763: rename d0/dc/d2e/d73/c9e to d0/d3/d1a/d41/d8f/de9/cf2 0 2024-04-12T20:56:38.970 INFO:tasks.workunit.client.1.smithi186.stdout:9/615: dread d3/d12/fcd [0,4194304] 0 2024-04-12T20:56:38.974 INFO:tasks.workunit.client.1.smithi186.stdout:3/637: mkdir d1/d3/d21/d22/d24/d2f/dd0 0 2024-04-12T20:56:38.977 INFO:tasks.workunit.client.1.smithi186.stdout:8/655: chown d7/d18/d36/d1d/d1f/da2/d5c/f7b 257119316 1 2024-04-12T20:56:38.978 INFO:tasks.workunit.client.1.smithi186.stdout:5/684: creat d5/db/d15/d2a/d4c/fe4 x:0 0 0 2024-04-12T20:56:38.980 INFO:tasks.workunit.client.1.smithi186.stdout:2/764: mknod d0/d3/d1a/d21/d22/cf3 0 2024-04-12T20:56:38.980 INFO:tasks.workunit.client.1.smithi186.stdout:3/638: truncate d1/d3/d49/fc5 140521 0 2024-04-12T20:56:38.982 INFO:tasks.workunit.client.1.smithi186.stdout:7/673: dread d7/fc [0,4194304] 0 2024-04-12T20:56:38.983 INFO:tasks.workunit.client.1.smithi186.stdout:4/787: write d0/dc/ff1 [721066,87896] 0 2024-04-12T20:56:38.986 INFO:tasks.workunit.client.1.smithi186.stdout:9/616: creat d3/d12/fcf x:0 0 0 2024-04-12T20:56:38.990 INFO:tasks.workunit.client.1.smithi186.stdout:2/765: dread - d0/d3/d7a/d8d/fbd zero size 2024-04-12T20:56:38.991 INFO:tasks.workunit.client.1.smithi186.stdout:0/765: symlink d0/d85/dc4/lfb 0 2024-04-12T20:56:38.991 INFO:tasks.workunit.client.1.smithi186.stdout:3/639: mknod d1/d2c/d48/cd1 0 2024-04-12T20:56:38.996 INFO:tasks.workunit.client.1.smithi186.stdout:2/766: dread - d0/d3/d7a/ff1 zero size 2024-04-12T20:56:38.998 INFO:tasks.workunit.client.1.smithi186.stdout:6/661: dwrite d9/d43/fd5 [0,4194304] 0 2024-04-12T20:56:38.999 INFO:tasks.workunit.client.1.smithi186.stdout:4/788: truncate d0/d59/dc0/d48/d61/ff7 763720 0 2024-04-12T20:56:39.004 INFO:tasks.workunit.client.1.smithi186.stdout:8/656: mkdir d7/de/d10/d7f/dd3 0 2024-04-12T20:56:39.008 INFO:tasks.workunit.client.1.smithi186.stdout:9/617: creat d3/fd0 x:0 0 0 2024-04-12T20:56:39.009 INFO:tasks.workunit.client.1.smithi186.stdout:6/662: truncate d9/d43/f93 871942 0 2024-04-12T20:56:39.010 INFO:tasks.workunit.client.1.smithi186.stdout:4/789: fdatasync d0/dc/d64/d87/d98/fbc 0 2024-04-12T20:56:39.011 INFO:tasks.workunit.client.1.smithi186.stdout:1/677: dread d14/d1a/d33/d36/d5b/d3e/d71/f76 [0,4194304] 0 2024-04-12T20:56:39.013 INFO:tasks.workunit.client.1.smithi186.stdout:5/685: truncate d5/db/d15/f31 2330784 0 2024-04-12T20:56:39.018 INFO:tasks.workunit.client.1.smithi186.stdout:3/640: write d1/d3/d49/f59 [3261112,63773] 0 2024-04-12T20:56:39.019 INFO:tasks.workunit.client.1.smithi186.stdout:2/767: getdents d0/d3/d1a/d21/d26/d86 0 2024-04-12T20:56:39.022 INFO:tasks.workunit.client.1.smithi186.stdout:4/790: rmdir d0/dc/d88/d80/d9c 39 2024-04-12T20:56:39.023 INFO:tasks.workunit.client.1.smithi186.stdout:5/686: fsync d5/db/d15/f28 0 2024-04-12T20:56:39.025 INFO:tasks.workunit.client.1.smithi186.stdout:7/674: dwrite d7/d55/d71/fb6 [0,4194304] 0 2024-04-12T20:56:39.026 INFO:tasks.workunit.client.1.smithi186.stdout:6/663: mknod d9/d4c/d6f/d89/d9c/cef 0 2024-04-12T20:56:39.027 INFO:tasks.workunit.client.1.smithi186.stdout:0/766: dwrite d0/d5d/ddc/fe0 [0,4194304] 0 2024-04-12T20:56:39.028 INFO:tasks.workunit.client.1.smithi186.stdout:1/678: fsync d14/d1a/d33/d36/d5b/d3e/d5f/f8f 0 2024-04-12T20:56:39.032 INFO:tasks.workunit.client.1.smithi186.stdout:4/791: mkdir d0/db4/dd8/d104 0 2024-04-12T20:56:39.033 INFO:tasks.workunit.client.1.smithi186.stdout:6/664: fdatasync d9/d43/f45 0 2024-04-12T20:56:39.040 INFO:tasks.workunit.client.1.smithi186.stdout:8/657: link d7/de/d68/fcb d7/de/d10/d7f/dd3/fd4 0 2024-04-12T20:56:39.040 INFO:tasks.workunit.client.1.smithi186.stdout:1/679: stat d14/d1a/d33/d36/d5b/d3e/d71/d8c/caa 0 2024-04-12T20:56:39.045 INFO:tasks.workunit.client.1.smithi186.stdout:9/618: dwrite d3/d2f/f68 [0,4194304] 0 2024-04-12T20:56:39.048 INFO:tasks.workunit.client.1.smithi186.stdout:0/767: write d0/d14/d4e/d1e/d84/fcd [35305,11478] 0 2024-04-12T20:56:39.048 INFO:tasks.workunit.client.1.smithi186.stdout:8/658: chown d7/fb3 295158 1 2024-04-12T20:56:39.048 INFO:tasks.workunit.client.1.smithi186.stdout:7/675: creat d7/d10/d2d/d40/d4a/d6c/d97/d4d/d6a/ff4 x:0 0 0 2024-04-12T20:56:39.048 INFO:tasks.workunit.client.1.smithi186.stdout:1/680: write d14/d1a/f37 [842079,103035] 0 2024-04-12T20:56:39.050 INFO:tasks.workunit.client.1.smithi186.stdout:6/665: chown d9/d43/d59/d5d/d6e/d91/ca8 3434 1 2024-04-12T20:56:39.054 INFO:tasks.workunit.client.1.smithi186.stdout:1/681: fsync d14/f7d 0 2024-04-12T20:56:39.054 INFO:tasks.workunit.client.1.smithi186.stdout:8/659: fsync d7/d18/d36/d1d/d1f/fd2 0 2024-04-12T20:56:39.055 INFO:tasks.workunit.client.1.smithi186.stdout:2/768: link d0/dc/d2e/de8/lef d0/dc/lf4 0 2024-04-12T20:56:39.058 INFO:tasks.workunit.client.1.smithi186.stdout:6/666: rmdir d9/d43/d50 39 2024-04-12T20:56:39.062 INFO:tasks.workunit.client.1.smithi186.stdout:8/660: write d7/de/d7a/f31 [1321047,105570] 0 2024-04-12T20:56:39.065 INFO:tasks.workunit.client.1.smithi186.stdout:2/769: write d0/dc/d2e/d73/fd9 [164713,19008] 0 2024-04-12T20:56:39.066 INFO:tasks.workunit.client.1.smithi186.stdout:1/682: read - d14/d1a/d33/d36/d5b/d3e/fd7 zero size 2024-04-12T20:56:39.066 INFO:tasks.workunit.client.1.smithi186.stdout:3/641: dwrite d1/d3/d21/d22/f79 [0,4194304] 0 2024-04-12T20:56:39.068 INFO:tasks.workunit.client.1.smithi186.stdout:4/792: truncate d0/dc/d64/d87/dd2/fd7 3564391 0 2024-04-12T20:56:39.070 INFO:tasks.workunit.client.1.smithi186.stdout:1/683: mknod d14/d1a/d1d/cda 0 2024-04-12T20:56:39.071 INFO:tasks.workunit.client.1.smithi186.stdout:7/676: write d7/d10/d2d/d40/d4a/d6c/f7d [663754,121356] 0 2024-04-12T20:56:39.072 INFO:tasks.workunit.client.1.smithi186.stdout:6/667: rename d9/d43/c2b to d9/d4c/d6f/d89/d9c/cf0 0 2024-04-12T20:56:39.073 INFO:tasks.workunit.client.1.smithi186.stdout:3/642: write d1/d3/da/f56 [5328480,62153] 0 2024-04-12T20:56:39.074 INFO:tasks.workunit.client.1.smithi186.stdout:4/793: dread - d0/d59/dc0/d48/f95 zero size 2024-04-12T20:56:39.079 INFO:tasks.workunit.client.1.smithi186.stdout:1/684: symlink d14/d1a/d33/d36/d5b/d6a/da2/ldb 0 2024-04-12T20:56:39.080 INFO:tasks.workunit.client.1.smithi186.stdout:8/661: write d7/de/d10/f3c [1149060,7652] 0 2024-04-12T20:56:39.083 INFO:tasks.workunit.client.1.smithi186.stdout:8/662: readlink d7/d18/d36/d20/d23/d25/l75 0 2024-04-12T20:56:39.085 INFO:tasks.workunit.client.1.smithi186.stdout:4/794: rmdir d0/d59/dc0/d63/da6/daf 39 2024-04-12T20:56:39.085 INFO:tasks.workunit.client.1.smithi186.stdout:3/643: read d1/d3/d21/fb1 [2477382,54729] 0 2024-04-12T20:56:39.087 INFO:tasks.workunit.client.1.smithi186.stdout:6/668: mknod d9/d4c/d6f/de3/cf1 0 2024-04-12T20:56:39.089 INFO:tasks.workunit.client.1.smithi186.stdout:1/685: mknod d14/d6c/cdc 0 2024-04-12T20:56:39.090 INFO:tasks.workunit.client.1.smithi186.stdout:8/663: dread - d7/d18/d36/d20/d23/f5f zero size 2024-04-12T20:56:39.091 INFO:tasks.workunit.client.1.smithi186.stdout:0/768: dread d0/f88 [0,4194304] 0 2024-04-12T20:56:39.095 INFO:tasks.workunit.client.1.smithi186.stdout:3/644: creat d1/d3/d49/d4d/fd2 x:0 0 0 2024-04-12T20:56:39.095 INFO:tasks.workunit.client.1.smithi186.stdout:0/769: write d0/d85/dc4/dc1/fd3 [2300596,128528] 0 2024-04-12T20:56:39.095 INFO:tasks.workunit.client.1.smithi186.stdout:8/664: mkdir d7/de/d68/da7/dd5 0 2024-04-12T20:56:39.096 INFO:tasks.workunit.client.1.smithi186.stdout:3/645: creat d1/d2c/d54/fd3 x:0 0 0 2024-04-12T20:56:39.100 INFO:tasks.workunit.client.1.smithi186.stdout:1/686: getdents d14/d1a/d21/d35/d59 0 2024-04-12T20:56:39.103 INFO:tasks.workunit.client.1.smithi186.stdout:8/665: fdatasync d7/d18/d36/d20/d23/d25/f34 0 2024-04-12T20:56:39.104 INFO:tasks.workunit.client.1.smithi186.stdout:0/770: mkdir d0/d5c/d66/da7/dfc 0 2024-04-12T20:56:39.105 INFO:tasks.workunit.client.1.smithi186.stdout:8/666: chown d7 659 1 2024-04-12T20:56:39.110 INFO:tasks.workunit.client.1.smithi186.stdout:2/770: dread d0/d32/f7c [0,4194304] 0 2024-04-12T20:56:39.113 INFO:tasks.workunit.client.1.smithi186.stdout:4/795: dwrite d0/d59/dc0/d63/ff5 [0,4194304] 0 2024-04-12T20:56:39.113 INFO:tasks.workunit.client.1.smithi186.stdout:8/667: dread - d7/d18/d36/d1d/d1f/da2/fbe zero size 2024-04-12T20:56:39.115 INFO:tasks.workunit.client.1.smithi186.stdout:1/687: rename d14/d74/l81 to d14/d1a/d21/d35/dbc/ldd 0 2024-04-12T20:56:39.120 INFO:tasks.workunit.client.1.smithi186.stdout:0/771: write d0/d14/d4e/d1e/f98 [301462,26528] 0 2024-04-12T20:56:39.120 INFO:tasks.workunit.client.1.smithi186.stdout:6/669: dwrite d9/da9/fee [0,4194304] 0 2024-04-12T20:56:39.120 INFO:tasks.workunit.client.1.smithi186.stdout:1/688: dread - d14/d6c/da9/fae zero size 2024-04-12T20:56:39.124 INFO:tasks.workunit.client.1.smithi186.stdout:2/771: mknod d0/d3/d1a/d41/cf5 0 2024-04-12T20:56:39.127 INFO:tasks.workunit.client.1.smithi186.stdout:4/796: write d0/dc/d88/fbd [376529,59426] 0 2024-04-12T20:56:39.133 INFO:tasks.workunit.client.1.smithi186.stdout:0/772: creat d0/d10/ffd x:0 0 0 2024-04-12T20:56:39.134 INFO:tasks.workunit.client.1.smithi186.stdout:4/797: symlink d0/dc/l105 0 2024-04-12T20:56:39.138 INFO:tasks.workunit.client.1.smithi186.stdout:6/670: rename d9/d4c/d6f/de3/cf1 to d9/d43/d50/cf2 0 2024-04-12T20:56:39.138 INFO:tasks.workunit.client.1.smithi186.stdout:0/773: dread - d0/d5c/d65/ff7 zero size 2024-04-12T20:56:39.141 INFO:tasks.workunit.client.1.smithi186.stdout:6/671: chown d9/d5e/d73/l80 4567830 1 2024-04-12T20:56:39.142 INFO:tasks.workunit.client.1.smithi186.stdout:0/774: chown d0/d10/d32/d3f/f54 679296 1 2024-04-12T20:56:39.144 INFO:tasks.workunit.client.1.smithi186.stdout:3/646: dread d1/fc [0,4194304] 0 2024-04-12T20:56:39.145 INFO:tasks.workunit.client.1.smithi186.stdout:8/668: dwrite d7/de/d68/f9d [0,4194304] 0 2024-04-12T20:56:39.145 INFO:tasks.workunit.client.1.smithi186.stdout:0/775: chown d0/d5c/d65/c74 7400 1 2024-04-12T20:56:39.152 INFO:tasks.workunit.client.1.smithi186.stdout:0/776: write d0/f88 [3044683,94100] 0 2024-04-12T20:56:39.161 INFO:tasks.workunit.client.1.smithi186.stdout:6/672: rename d9/d4c/l63 to d9/d4c/lf3 0 2024-04-12T20:56:39.162 INFO:tasks.workunit.client.1.smithi186.stdout:0/777: dread - d0/d85/fde zero size 2024-04-12T20:56:39.162 INFO:tasks.workunit.client.1.smithi186.stdout:8/669: truncate d7/d18/d36/d1d/d1f/da2/f94 1718131 0 2024-04-12T20:56:39.164 INFO:tasks.workunit.client.1.smithi186.stdout:5/687: sync 2024-04-12T20:56:39.166 INFO:tasks.workunit.client.1.smithi186.stdout:1/689: dwrite d14/d1a/d21/d35/d59/fbf [0,4194304] 0 2024-04-12T20:56:39.168 INFO:tasks.workunit.client.1.smithi186.stdout:6/673: chown d9/d43/d59/la1 60758310 1 2024-04-12T20:56:39.174 INFO:tasks.workunit.client.1.smithi186.stdout:1/690: chown d14/d6c/fb5 6 1 2024-04-12T20:56:39.175 INFO:tasks.workunit.client.1.smithi186.stdout:4/798: dwrite d0/dc/d33/f7 [0,4194304] 0 2024-04-12T20:56:39.177 INFO:tasks.workunit.client.1.smithi186.stdout:0/778: creat d0/d14/d4e/d1e/d84/dec/ffe x:0 0 0 2024-04-12T20:56:39.178 INFO:tasks.workunit.client.1.smithi186.stdout:8/670: read d7/d18/d36/d1d/d1f/d26/f3f [2652812,118379] 0 2024-04-12T20:56:39.180 INFO:tasks.workunit.client.1.smithi186.stdout:0/779: chown d0/d10/d72/d79/fba 801 1 2024-04-12T20:56:39.180 INFO:tasks.workunit.client.1.smithi186.stdout:1/691: rmdir d14/d74 39 2024-04-12T20:56:39.181 INFO:tasks.workunit.client.1.smithi186.stdout:3/647: dwrite d1/d3/d21/d22/d24/d2f/f92 [0,4194304] 0 2024-04-12T20:56:39.183 INFO:tasks.workunit.client.1.smithi186.stdout:6/674: mknod d9/d5e/d7a/d81/cf4 0 2024-04-12T20:56:39.184 INFO:tasks.workunit.client.1.smithi186.stdout:2/772: dread d0/fc5 [0,4194304] 0 2024-04-12T20:56:39.185 INFO:tasks.workunit.client.1.smithi186.stdout:1/692: chown d14/d1a/d33/d36/d5b/db9/dc9 26 1 2024-04-12T20:56:39.187 INFO:tasks.workunit.client.1.smithi186.stdout:4/799: write d0/f6 [1431058,71704] 0 2024-04-12T20:56:39.195 INFO:tasks.workunit.client.1.smithi186.stdout:8/671: dread d7/de/d5e/f48 [0,4194304] 0 2024-04-12T20:56:39.199 INFO:tasks.workunit.client.1.smithi186.stdout:5/688: dwrite d5/db/f11 [0,4194304] 0 2024-04-12T20:56:39.201 INFO:tasks.workunit.client.1.smithi186.stdout:1/693: creat d14/d1a/d33/d36/d5b/d3e/d8b/fde x:0 0 0 2024-04-12T20:56:39.203 INFO:tasks.workunit.client.1.smithi186.stdout:4/800: link d0/dc/d33/f68 d0/db4/f106 0 2024-04-12T20:56:39.206 INFO:tasks.workunit.client.1.smithi186.stdout:8/672: stat d7/d18/d36/d20/d23/d25/c69 0 2024-04-12T20:56:39.209 INFO:tasks.workunit.client.1.smithi186.stdout:4/801: readlink d0/dc/d88/d80/lfc 0 2024-04-12T20:56:39.212 INFO:tasks.workunit.client.1.smithi186.stdout:4/802: chown d0/dc/d1f/d73/fb7 559 1 2024-04-12T20:56:39.216 INFO:tasks.workunit.client.1.smithi186.stdout:1/694: unlink d14/d91/l96 0 2024-04-12T20:56:39.218 INFO:tasks.workunit.client.1.smithi186.stdout:1/695: readlink l13 0 2024-04-12T20:56:39.227 INFO:tasks.workunit.client.1.smithi186.stdout:2/773: dwrite d0/d3/d1a/d41/fbb [0,4194304] 0 2024-04-12T20:56:39.232 INFO:tasks.workunit.client.1.smithi186.stdout:0/780: dwrite d0/d5c/d65/ff7 [0,4194304] 0 2024-04-12T20:56:39.232 INFO:tasks.workunit.client.1.smithi186.stdout:1/696: creat d14/d1a/d58/d69/fdf x:0 0 0 2024-04-12T20:56:39.234 INFO:tasks.workunit.client.1.smithi186.stdout:4/803: dread d0/d59/dc0/d63/f90 [0,4194304] 0 2024-04-12T20:56:39.238 INFO:tasks.workunit.client.1.smithi186.stdout:0/781: dread - d0/d10/fa8 zero size 2024-04-12T20:56:39.238 INFO:tasks.workunit.client.1.smithi186.stdout:1/697: stat d14/d1a/d33/d36/d5b/d3e 0 2024-04-12T20:56:39.242 INFO:tasks.workunit.client.1.smithi186.stdout:6/675: dwrite d9/d4c/d6f/d89/d9c/fd6 [0,4194304] 0 2024-04-12T20:56:39.250 INFO:tasks.workunit.client.1.smithi186.stdout:1/698: write d14/d1a/d1d/f9b [614972,39521] 0 2024-04-12T20:56:39.250 INFO:tasks.workunit.client.1.smithi186.stdout:8/673: dwrite d7/de/d7a/f80 [0,4194304] 0 2024-04-12T20:56:39.254 INFO:tasks.workunit.client.1.smithi186.stdout:4/804: truncate d0/dc/f2f 3695560 0 2024-04-12T20:56:39.256 INFO:tasks.workunit.client.1.smithi186.stdout:2/774: read d0/d3/d1a/d48/fde [1041036,8031] 0 2024-04-12T20:56:39.257 INFO:tasks.workunit.client.1.smithi186.stdout:3/648: dwrite d1/d3/d21/d22/d24/d2f/f7f [0,4194304] 0 2024-04-12T20:56:39.257 INFO:tasks.workunit.client.1.smithi186.stdout:1/699: stat d14/d1a/d21/d35/dbc/dd5 0 2024-04-12T20:56:39.267 INFO:tasks.workunit.client.1.smithi186.stdout:0/782: creat d0/d5c/d65/fff x:0 0 0 2024-04-12T20:56:39.272 INFO:tasks.workunit.client.1.smithi186.stdout:3/649: fsync d1/d3/da2/f9d 0 2024-04-12T20:56:39.276 INFO:tasks.workunit.client.1.smithi186.stdout:2/775: creat d0/d3/ff6 x:0 0 0 2024-04-12T20:56:39.276 INFO:tasks.workunit.client.1.smithi186.stdout:4/805: symlink d0/dc/d88/d80/df8/l107 0 2024-04-12T20:56:39.279 INFO:tasks.workunit.client.1.smithi186.stdout:1/700: unlink d14/d1a/d33/d36/d5b/d6a/da2/ldb 0 2024-04-12T20:56:39.283 INFO:tasks.workunit.client.1.smithi186.stdout:0/783: dread - d0/d5c/fc6 zero size 2024-04-12T20:56:39.283 INFO:tasks.workunit.client.1.smithi186.stdout:5/689: dread d5/db/f5f [4194304,4194304] 0 2024-04-12T20:56:39.284 INFO:tasks.workunit.client.1.smithi186.stdout:3/650: creat d1/d2c/d54/fd4 x:0 0 0 2024-04-12T20:56:39.288 INFO:tasks.workunit.client.1.smithi186.stdout:6/676: dread d9/d5e/f6d [0,4194304] 0 2024-04-12T20:56:39.288 INFO:tasks.workunit.client.1.smithi186.stdout:4/806: creat d0/dc/d1f/d73/f108 x:0 0 0 2024-04-12T20:56:39.289 INFO:tasks.workunit.client.1.smithi186.stdout:8/674: dwrite d7/de/d7a/f80 [4194304,4194304] 0 2024-04-12T20:56:39.289 INFO:tasks.workunit.client.1.smithi186.stdout:1/701: mknod d14/d1a/d33/ce0 0 2024-04-12T20:56:39.289 INFO:tasks.workunit.client.1.smithi186.stdout:3/651: write d1/d2c/f67 [800038,4434] 0 2024-04-12T20:56:39.292 INFO:tasks.workunit.client.1.smithi186.stdout:2/776: rename d0/d3/d7a/d8d to d0/d3/d1a/d48/df7 0 2024-04-12T20:56:39.292 INFO:tasks.workunit.client.1.smithi186.stdout:0/784: rmdir d0/d85/dc4/dc1 39 2024-04-12T20:56:39.293 INFO:tasks.workunit.client.1.smithi186.stdout:9/619: sync 2024-04-12T20:56:39.294 INFO:tasks.workunit.client.1.smithi186.stdout:3/652: write d1/d3/d21/f65 [1752316,97765] 0 2024-04-12T20:56:39.296 INFO:tasks.workunit.client.1.smithi186.stdout:5/690: truncate d5/db/d15/d2a/d2d/f90 2312996 0 2024-04-12T20:56:39.298 INFO:tasks.workunit.client.1.smithi186.stdout:2/777: read - d0/d3/d7a/fe0 zero size 2024-04-12T20:56:39.299 INFO:tasks.workunit.client.1.smithi186.stdout:0/785: creat d0/d5c/d66/da7/f100 x:0 0 0 2024-04-12T20:56:39.301 INFO:tasks.workunit.client.1.smithi186.stdout:8/675: mkdir d7/d18/d36/d20/d23/dd6 0 2024-04-12T20:56:39.301 INFO:tasks.workunit.client.1.smithi186.stdout:9/620: readlink d3/d12/d6f/d83/d71/d72/l8a 0 2024-04-12T20:56:39.302 INFO:tasks.workunit.client.1.smithi186.stdout:9/621: chown d3/ff 131108867 1 2024-04-12T20:56:39.304 INFO:tasks.workunit.client.1.smithi186.stdout:5/691: mknod d5/db/d15/ce5 0 2024-04-12T20:56:39.306 INFO:tasks.workunit.client.1.smithi186.stdout:3/653: readlink d1/d2c/d54/d7c/l80 0 2024-04-12T20:56:39.309 INFO:tasks.workunit.client.1.smithi186.stdout:5/692: chown d5/db/d15/d2a/d2d/d82/c9e 509648400 1 2024-04-12T20:56:39.310 INFO:tasks.workunit.client.1.smithi186.stdout:4/807: rmdir d0/dc/d88/d80/def 0 2024-04-12T20:56:39.313 INFO:tasks.workunit.client.1.smithi186.stdout:4/808: dread - d0/dc/d33/fbb zero size 2024-04-12T20:56:39.313 INFO:tasks.workunit.client.1.smithi186.stdout:5/693: chown d5/db/d15/d2a/d4c/c8e 17481123 1 2024-04-12T20:56:39.315 INFO:tasks.workunit.client.1.smithi186.stdout:0/786: dread d0/d5d/ddc/fe0 [0,4194304] 0 2024-04-12T20:56:39.317 INFO:tasks.workunit.client.1.smithi186.stdout:8/676: mkdir d7/d18/d36/d20/d23/dd7 0 2024-04-12T20:56:39.318 INFO:tasks.workunit.client.1.smithi186.stdout:1/702: link d14/d1a/l8a d14/d6c/le1 0 2024-04-12T20:56:39.319 INFO:tasks.workunit.client.1.smithi186.stdout:6/677: dread d9/f37 [0,4194304] 0 2024-04-12T20:56:39.322 INFO:tasks.workunit.client.1.smithi186.stdout:5/694: unlink d5/db/d71/l1b 0 2024-04-12T20:56:39.324 INFO:tasks.workunit.client.1.smithi186.stdout:4/809: rename d0/dc/f23 to d0/dc/d64/f109 0 2024-04-12T20:56:39.324 INFO:tasks.workunit.client.1.smithi186.stdout:5/695: chown d5/db/d15/d2a/d4c/d93/fcf 48696 1 2024-04-12T20:56:39.325 INFO:tasks.workunit.client.1.smithi186.stdout:2/778: creat d0/d3/d1a/ff8 x:0 0 0 2024-04-12T20:56:39.325 INFO:tasks.workunit.client.1.smithi186.stdout:4/810: chown d0/d59/dc0/d63/da6 961 1 2024-04-12T20:56:39.326 INFO:tasks.workunit.client.1.smithi186.stdout:9/622: creat d3/d2f/d2e/db3/fd1 x:0 0 0 2024-04-12T20:56:39.327 INFO:tasks.workunit.client.1.smithi186.stdout:3/654: symlink d1/ld5 0 2024-04-12T20:56:39.329 INFO:tasks.workunit.client.1.smithi186.stdout:0/787: creat d0/d5d/f101 x:0 0 0 2024-04-12T20:56:39.330 INFO:tasks.workunit.client.1.smithi186.stdout:5/696: truncate d5/db/d15/d2a/d6d/dbb/dc4/d5d/fc7 249378 0 2024-04-12T20:56:39.331 INFO:tasks.workunit.client.1.smithi186.stdout:6/678: read d9/d43/d59/d5d/f5a [118254,113613] 0 2024-04-12T20:56:39.332 INFO:tasks.workunit.client.1.smithi186.stdout:7/677: sync 2024-04-12T20:56:39.332 INFO:tasks.workunit.client.1.smithi186.stdout:8/677: fdatasync d7/d18/d36/d1d/fa0 0 2024-04-12T20:56:39.333 INFO:tasks.workunit.client.1.smithi186.stdout:1/703: creat d14/d74/fe2 x:0 0 0 2024-04-12T20:56:39.334 INFO:tasks.workunit.client.1.smithi186.stdout:2/779: symlink d0/d3/dce/ddf/lf9 0 2024-04-12T20:56:39.335 INFO:tasks.workunit.client.1.smithi186.stdout:3/655: creat d1/d3/d21/d22/d34/fd6 x:0 0 0 2024-04-12T20:56:39.336 INFO:tasks.workunit.client.1.smithi186.stdout:7/678: fdatasync d7/d10/d2d/d40/d4a/d6c/ddf/fed 0 2024-04-12T20:56:39.337 INFO:tasks.workunit.client.1.smithi186.stdout:0/788: creat d0/d14/d4e/d1e/d84/dec/f102 x:0 0 0 2024-04-12T20:56:39.338 INFO:tasks.workunit.client.1.smithi186.stdout:0/789: readlink d0/d85/l9f 0 2024-04-12T20:56:39.339 INFO:tasks.workunit.client.1.smithi186.stdout:9/623: mkdir d3/d2f/d54/d9f/dc4/d77/dc3/dd2 0 2024-04-12T20:56:39.342 INFO:tasks.workunit.client.1.smithi186.stdout:9/624: read d3/d2f/d2e/d33/f94 [3490607,123676] 0 2024-04-12T20:56:39.343 INFO:tasks.workunit.client.1.smithi186.stdout:4/811: dwrite d0/d59/dc0/f84 [4194304,4194304] 0 2024-04-12T20:56:39.344 INFO:tasks.workunit.client.1.smithi186.stdout:5/697: symlink d5/db/d15/d2a/d6d/d7f/d80/le6 0 2024-04-12T20:56:39.345 INFO:tasks.workunit.client.1.smithi186.stdout:4/812: readlink d0/dc/lc1 0 2024-04-12T20:56:39.349 INFO:tasks.workunit.client.1.smithi186.stdout:5/698: read d5/db/d15/f89 [1810559,108578] 0 2024-04-12T20:56:39.349 INFO:tasks.workunit.client.1.smithi186.stdout:6/679: unlink d9/f2e 0 2024-04-12T20:56:39.350 INFO:tasks.workunit.client.1.smithi186.stdout:2/780: chown d0/l23 8 1 2024-04-12T20:56:39.351 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:56:39 smithi018 ceph-mon[28799]: pgmap v52: 65 pgs: 65 active+clean; 3.1 GiB data, 11 GiB used, 526 GiB / 536 GiB avail; 62 MiB/s rd, 197 MiB/s wr, 392 op/s 2024-04-12T20:56:39.352 INFO:tasks.workunit.client.1.smithi186.stdout:1/704: mkdir d14/d1a/d21/d35/dbc/de3 0 2024-04-12T20:56:39.352 INFO:tasks.workunit.client.1.smithi186.stdout:6/680: stat d9/d5e/d73/c90 0 2024-04-12T20:56:39.355 INFO:tasks.workunit.client.1.smithi186.stdout:8/678: mkdir d7/d18/d36/d20/dd8 0 2024-04-12T20:56:39.357 INFO:tasks.workunit.client.1.smithi186.stdout:7/679: getdents d7/deb 0 2024-04-12T20:56:39.359 INFO:tasks.workunit.client.1.smithi186.stdout:2/781: mknod d0/dc/d2e/cfa 0 2024-04-12T20:56:39.359 INFO:tasks.workunit.client.1.smithi186.stdout:5/699: write d5/db/d2e/d70/f99 [1178939,21122] 0 2024-04-12T20:56:39.360 INFO:tasks.workunit.client.1.smithi186.stdout:3/656: dwrite d1/d3/d49/d4d/dad/fcf [0,4194304] 0 2024-04-12T20:56:39.361 INFO:tasks.workunit.client.1.smithi186.stdout:0/790: dwrite d0/d5c/d65/fbe [0,4194304] 0 2024-04-12T20:56:39.363 INFO:tasks.workunit.client.1.smithi186.stdout:8/679: chown d7/de/d5e/l6f 7 1 2024-04-12T20:56:39.364 INFO:tasks.workunit.client.1.smithi186.stdout:6/681: mknod d9/d43/d50/cf5 0 2024-04-12T20:56:39.368 INFO:tasks.workunit.client.1.smithi186.stdout:5/700: chown d5/db/d15/d2a/d2d/d82/c9e 189984914 1 2024-04-12T20:56:39.370 INFO:tasks.workunit.client.1.smithi186.stdout:5/701: chown d5/db/d71/f81 7 1 2024-04-12T20:56:39.377 INFO:tasks.workunit.client.1.smithi186.stdout:9/625: getdents d3/d2f/d54/d9f/dc4 0 2024-04-12T20:56:39.379 INFO:tasks.workunit.client.1.smithi186.stdout:1/705: mkdir d14/d1a/d21/d35/dbc/de3/de4 0 2024-04-12T20:56:39.383 INFO:tasks.workunit.client.1.smithi186.stdout:6/682: mknod d9/d4c/dbe/cf6 0 2024-04-12T20:56:39.383 INFO:tasks.workunit.client.1.smithi186.stdout:8/680: creat d7/de/d7a/fd9 x:0 0 0 2024-04-12T20:56:39.384 INFO:tasks.workunit.client.1.smithi186.stdout:6/683: write d9/d43/d2d/d4d/fe0 [161750,16776] 0 2024-04-12T20:56:39.386 INFO:tasks.workunit.client.1.smithi186.stdout:5/702: rmdir d5/db/d2e 39 2024-04-12T20:56:39.389 INFO:tasks.workunit.client.1.smithi186.stdout:2/782: getdents d0/dd3 0 2024-04-12T20:56:39.391 INFO:tasks.workunit.client.1.smithi186.stdout:9/626: creat d3/d2f/d2e/d33/d61/dbc/fd3 x:0 0 0 2024-04-12T20:56:39.393 INFO:tasks.workunit.client.1.smithi186.stdout:1/706: mkdir d14/d1a/d1d/de5 0 2024-04-12T20:56:39.394 INFO:tasks.workunit.client.1.smithi186.stdout:7/680: mknod d7/d10/d2d/d40/d4a/d6c/d97/d4d/d6a/dcc/cf5 0 2024-04-12T20:56:39.395 INFO:tasks.workunit.client.1.smithi186.stdout:4/813: write d0/dc/d64/d87/dd2/fd7 [584116,65560] 0 2024-04-12T20:56:39.396 INFO:tasks.workunit.client.1.smithi186.stdout:5/703: symlink d5/db/d15/d2a/le7 0 2024-04-12T20:56:39.398 INFO:tasks.workunit.client.1.smithi186.stdout:0/791: link d0/d5d/c63 d0/d14/c103 0 2024-04-12T20:56:39.400 INFO:tasks.workunit.client.1.smithi186.stdout:9/627: mkdir d3/d2f/d2e/d43/d4c/d82/dd4 0 2024-04-12T20:56:39.401 INFO:tasks.workunit.client.1.smithi186.stdout:7/681: symlink d7/d10/d2d/d40/d4a/d6c/d97/d4d/d84/lf6 0 2024-04-12T20:56:39.402 INFO:tasks.workunit.client.1.smithi186.stdout:8/681: mkdir d7/d18/d36/d20/d23/dc1/dda 0 2024-04-12T20:56:39.404 INFO:tasks.workunit.client.1.smithi186.stdout:4/814: mkdir d0/d59/dc0/d48/dce/d10a 0 2024-04-12T20:56:39.406 INFO:tasks.workunit.client.1.smithi186.stdout:9/628: mkdir d3/d12/d6f/d83/dd5 0 2024-04-12T20:56:39.408 INFO:tasks.workunit.client.1.smithi186.stdout:2/783: getdents d0/dc/d2e/da9 0 2024-04-12T20:56:39.409 INFO:tasks.workunit.client.1.smithi186.stdout:7/682: chown d7/d10/d1c/d8c/fb5 99894 1 2024-04-12T20:56:39.410 INFO:tasks.workunit.client.1.smithi186.stdout:0/792: rename d0/f87 to d0/d85/dc4/dc1/f104 0 2024-04-12T20:56:39.415 INFO:tasks.workunit.client.1.smithi186.stdout:9/629: dread - d3/d2f/d2e/d33/d61/f90 zero size 2024-04-12T20:56:39.415 INFO:tasks.workunit.client.1.smithi186.stdout:1/707: dwrite d14/d74/fe2 [0,4194304] 0 2024-04-12T20:56:39.417 INFO:tasks.workunit.client.1.smithi186.stdout:0/793: write d0/d10/f8f [343251,65992] 0 2024-04-12T20:56:39.468 INFO:tasks.workunit.client.1.smithi186.stdout:6/684: dwrite d9/d5e/f6d [4194304,4194304] 0 2024-04-12T20:56:39.469 INFO:tasks.workunit.client.1.smithi186.stdout:0/794: mkdir d0/d14/d105 0 2024-04-12T20:56:39.474 INFO:tasks.workunit.client.1.smithi186.stdout:6/685: readlink d9/l19 0 2024-04-12T20:56:39.475 INFO:tasks.workunit.client.1.smithi186.stdout:5/704: dwrite d5/db/fbd [0,4194304] 0 2024-04-12T20:56:39.481 INFO:tasks.workunit.client.1.smithi186.stdout:8/682: dwrite d7/de/d7a/f31 [0,4194304] 0 2024-04-12T20:56:39.484 INFO:tasks.workunit.client.1.smithi186.stdout:5/705: mkdir d5/db/d15/de8 0 2024-04-12T20:56:39.484 INFO:tasks.workunit.client.1.smithi186.stdout:6/686: write d9/d43/d50/f55 [4712181,74167] 0 2024-04-12T20:56:39.485 INFO:tasks.workunit.client.1.smithi186.stdout:2/784: dread d0/fc5 [0,4194304] 0 2024-04-12T20:56:39.486 INFO:tasks.workunit.client.1.smithi186.stdout:9/630: rename d3/d12/d6f/d83/d71 to d3/d2f/d54/d9f/dc4/d77/dd6 0 2024-04-12T20:56:39.493 INFO:tasks.workunit.client.1.smithi186.stdout:2/785: dread - d0/d32/f9b zero size 2024-04-12T20:56:39.493 INFO:tasks.workunit.client.1.smithi186.stdout:0/795: dread d0/d10/d72/f7d [0,4194304] 0 2024-04-12T20:56:39.493 INFO:tasks.workunit.client.1.smithi186.stdout:4/815: dread d0/f24 [0,4194304] 0 2024-04-12T20:56:39.494 INFO:tasks.workunit.client.1.smithi186.stdout:3/657: sync 2024-04-12T20:56:39.496 INFO:tasks.workunit.client.1.smithi186.stdout:0/796: fdatasync d0/d10/d32/f48 0 2024-04-12T20:56:39.501 INFO:tasks.workunit.client.1.smithi186.stdout:6/687: symlink d9/d5e/d7a/d81/d96/lf7 0 2024-04-12T20:56:39.501 INFO:tasks.workunit.client.1.smithi186.stdout:1/708: dwrite d14/d1a/fcc [0,4194304] 0 2024-04-12T20:56:39.513 INFO:tasks.workunit.client.1.smithi186.stdout:6/688: read - d9/d43/d59/d5d/d6e/d91/fdc zero size 2024-04-12T20:56:39.515 INFO:tasks.workunit.client.1.smithi186.stdout:1/709: write d14/d1a/d21/f31 [378967,24849] 0 2024-04-12T20:56:39.517 INFO:tasks.workunit.client.1.smithi186.stdout:9/631: link d3/d2f/d2e/d33/d36/f3d d3/d12/d6f/d83/fd7 0 2024-04-12T20:56:39.519 INFO:tasks.workunit.client.1.smithi186.stdout:8/683: write d7/de/d10/d7f/dd3/fd4 [7298,117589] 0 2024-04-12T20:56:39.524 INFO:tasks.workunit.client.1.smithi186.stdout:7/683: sync 2024-04-12T20:56:39.527 INFO:tasks.workunit.client.1.smithi186.stdout:0/797: rename d0/d10/d32/d3f/le5 to d0/d5c/d66/da7/dfc/l106 0 2024-04-12T20:56:39.527 INFO:tasks.workunit.client.1.smithi186.stdout:2/786: dwrite d0/d32/fcf [0,4194304] 0 2024-04-12T20:56:39.530 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:56:39 smithi186 ceph-mon[36597]: pgmap v52: 65 pgs: 65 active+clean; 3.1 GiB data, 11 GiB used, 526 GiB / 536 GiB avail; 62 MiB/s rd, 197 MiB/s wr, 392 op/s 2024-04-12T20:56:39.532 INFO:tasks.workunit.client.1.smithi186.stdout:6/689: unlink d9/d4c/d6f/fa2 0 2024-04-12T20:56:39.532 INFO:tasks.workunit.client.1.smithi186.stdout:3/658: creat d1/d2c/d54/d7c/dbb/fd7 x:0 0 0 2024-04-12T20:56:39.546 INFO:tasks.workunit.client.1.smithi186.stdout:2/787: rename d0/d3/d1a/d41/fee to d0/d32/d69/ffb 0 2024-04-12T20:56:39.547 INFO:tasks.workunit.client.1.smithi186.stdout:5/706: truncate d5/db/d15/d2a/d2d/d48/f4b 2209330 0 2024-04-12T20:56:39.548 INFO:tasks.workunit.client.1.smithi186.stdout:7/684: mkdir d7/d10/d2d/d40/df7 0 2024-04-12T20:56:39.548 INFO:tasks.workunit.client.1.smithi186.stdout:3/659: dread d1/d2c/d54/d7c/fac [0,4194304] 0 2024-04-12T20:56:39.552 INFO:tasks.workunit.client.1.smithi186.stdout:8/684: dread d7/d18/d36/d20/d23/d25/d29/f6a [0,4194304] 0 2024-04-12T20:56:39.552 INFO:tasks.workunit.client.1.smithi186.stdout:4/816: write d0/dc/d33/f8a [696061,102607] 0 2024-04-12T20:56:39.555 INFO:tasks.workunit.client.1.smithi186.stdout:8/685: fsync d7/de/d68/fcb 0 2024-04-12T20:56:39.556 INFO:tasks.workunit.client.1.smithi186.stdout:2/788: link d0/d3/d7a/fe0 d0/d3/d7a/ffc 0 2024-04-12T20:56:39.561 INFO:tasks.workunit.client.1.smithi186.stdout:1/710: truncate d14/d1a/d1d/f9b 143668 0 2024-04-12T20:56:39.562 INFO:tasks.workunit.client.1.smithi186.stdout:2/789: write d0/d3/d1a/d21/d26/f2d [266135,37223] 0 2024-04-12T20:56:39.563 INFO:tasks.workunit.client.1.smithi186.stdout:7/685: fdatasync d7/d10/d2d/d40/d4a/f64 0 2024-04-12T20:56:39.567 INFO:tasks.workunit.client.1.smithi186.stdout:8/686: truncate d7/d18/d36/d1d/d7d/da5/f6e 1440502 0 2024-04-12T20:56:39.567 INFO:tasks.workunit.client.1.smithi186.stdout:4/817: dread d0/dc/d33/f17 [0,4194304] 0 2024-04-12T20:56:39.569 INFO:tasks.workunit.client.1.smithi186.stdout:5/707: link d5/db/f34 d5/db/d15/d2a/d2d/d48/fe9 0 2024-04-12T20:56:39.573 INFO:tasks.workunit.client.1.smithi186.stdout:6/690: truncate d9/d4c/d6f/d89/d9c/fd6 973278 0 2024-04-12T20:56:39.575 INFO:tasks.workunit.client.1.smithi186.stdout:1/711: creat d14/d1a/d21/d35/fe6 x:0 0 0 2024-04-12T20:56:39.577 INFO:tasks.workunit.client.1.smithi186.stdout:2/790: creat d0/d3/d1a/d41/d8f/de9/ffd x:0 0 0 2024-04-12T20:56:39.577 INFO:tasks.workunit.client.1.smithi186.stdout:3/660: dwrite d1/d3/d21/f9a [0,4194304] 0 2024-04-12T20:56:39.583 INFO:tasks.workunit.client.1.smithi186.stdout:5/708: rename d5/db/d15/d2a/d6d/dbb/dc4/fc1 to d5/db/d15/fea 0 2024-04-12T20:56:39.583 INFO:tasks.workunit.client.1.smithi186.stdout:4/818: symlink d0/d21/l10b 0 2024-04-12T20:56:39.588 INFO:tasks.workunit.client.1.smithi186.stdout:6/691: creat d9/d43/d59/d5d/d6e/d91/dd9/ff8 x:0 0 0 2024-04-12T20:56:39.588 INFO:tasks.workunit.client.1.smithi186.stdout:7/686: getdents d7/d10/d2d 0 2024-04-12T20:56:39.588 INFO:tasks.workunit.client.1.smithi186.stdout:4/819: chown d0/d59/dc0/d48/d61/l6f 264820045 1 2024-04-12T20:56:39.590 INFO:tasks.workunit.client.1.smithi186.stdout:2/791: truncate d0/d3/d1a/d21/d26/fc6 589445 0 2024-04-12T20:56:39.594 INFO:tasks.workunit.client.1.smithi186.stdout:8/687: dread d7/de/d10/f3c [0,4194304] 0 2024-04-12T20:56:39.595 INFO:tasks.workunit.client.1.smithi186.stdout:2/792: rmdir d0/d32 39 2024-04-12T20:56:39.598 INFO:tasks.workunit.client.1.smithi186.stdout:8/688: dread - d7/d18/d36/d1d/d1f/da2/d5c/f99 zero size 2024-04-12T20:56:39.599 INFO:tasks.workunit.client.1.smithi186.stdout:8/689: fdatasync d7/d18/d36/d20/d71/fc0 0 2024-04-12T20:56:39.600 INFO:tasks.workunit.client.1.smithi186.stdout:6/692: unlink d9/d5e/d73/l99 0 2024-04-12T20:56:39.601 INFO:tasks.workunit.client.1.smithi186.stdout:7/687: unlink d7/d10/d2d/d40/d4a/d5c/f8d 0 2024-04-12T20:56:39.605 INFO:tasks.workunit.client.1.smithi186.stdout:4/820: symlink d0/dc/d33/d9a/l10c 0 2024-04-12T20:56:39.606 INFO:tasks.workunit.client.1.smithi186.stdout:2/793: mknod d0/d3/cfe 0 2024-04-12T20:56:39.606 INFO:tasks.workunit.client.1.smithi186.stdout:4/821: readlink d0/dc/d1f/l3e 0 2024-04-12T20:56:39.607 INFO:tasks.workunit.client.1.smithi186.stdout:2/794: readlink d0/d3/l16 0 2024-04-12T20:56:39.608 INFO:tasks.workunit.client.1.smithi186.stdout:4/822: chown d0/dc/d64/d87/c8c 5283417 1 2024-04-12T20:56:39.610 INFO:tasks.workunit.client.1.smithi186.stdout:8/690: rmdir d7/d18/d36 39 2024-04-12T20:56:39.612 INFO:tasks.workunit.client.1.smithi186.stdout:8/691: truncate d7/fb3 1557514 0 2024-04-12T20:56:39.612 INFO:tasks.workunit.client.1.smithi186.stdout:6/693: mknod d9/d4c/d6f/d97/cf9 0 2024-04-12T20:56:39.613 INFO:tasks.workunit.client.1.smithi186.stdout:6/694: readlink d9/l19 0 2024-04-12T20:56:39.614 INFO:tasks.workunit.client.1.smithi186.stdout:5/709: link d5/db/d15/d2a/d2d/f90 d5/db/d2e/d70/feb 0 2024-04-12T20:56:39.616 INFO:tasks.workunit.client.1.smithi186.stdout:3/661: getdents d1/d3/d21/d22 0 2024-04-12T20:56:39.618 INFO:tasks.workunit.client.1.smithi186.stdout:1/712: getdents d14/d1a/d33/d36/d5b/d3e/d71 0 2024-04-12T20:56:39.618 INFO:tasks.workunit.client.1.smithi186.stdout:3/662: stat d1/d2c/d48/fab 0 2024-04-12T20:56:39.620 INFO:tasks.workunit.client.1.smithi186.stdout:1/713: write d14/d1a/d33/d36/d5b/fc2 [832952,47515] 0 2024-04-12T20:56:39.621 INFO:tasks.workunit.client.1.smithi186.stdout:0/798: sync 2024-04-12T20:56:39.621 INFO:tasks.workunit.client.1.smithi186.stdout:9/632: sync 2024-04-12T20:56:39.628 INFO:tasks.workunit.client.1.smithi186.stdout:4/823: mknod d0/d59/dc0/d48/dce/d10a/c10d 0 2024-04-12T20:56:39.633 INFO:tasks.workunit.client.1.smithi186.stdout:3/663: mknod d1/d3/d21/d22/d24/cd8 0 2024-04-12T20:56:39.635 INFO:tasks.workunit.client.1.smithi186.stdout:7/688: truncate d7/d10/d1c/f1f 1689258 0 2024-04-12T20:56:39.636 INFO:tasks.workunit.client.1.smithi186.stdout:4/824: fdatasync d0/d21/f32 0 2024-04-12T20:56:39.636 INFO:tasks.workunit.client.1.smithi186.stdout:8/692: dwrite d7/d18/d36/d1d/d7d/da5/f6e [0,4194304] 0 2024-04-12T20:56:39.637 INFO:tasks.workunit.client.1.smithi186.stdout:7/689: chown d7/ccd 6112 1 2024-04-12T20:56:39.637 INFO:tasks.workunit.client.1.smithi186.stdout:2/795: write d0/d32/f97 [1599014,79873] 0 2024-04-12T20:56:39.639 INFO:tasks.workunit.client.1.smithi186.stdout:4/825: chown d0/d59/dc0/f84 12 1 2024-04-12T20:56:39.641 INFO:tasks.workunit.client.1.smithi186.stdout:6/695: getdents d9/d43/d2d/d4d 0 2024-04-12T20:56:39.646 INFO:tasks.workunit.client.1.smithi186.stdout:4/826: write d0/f69 [3305254,60899] 0 2024-04-12T20:56:39.646 INFO:tasks.workunit.client.1.smithi186.stdout:0/799: creat d0/d5c/f107 x:0 0 0 2024-04-12T20:56:39.646 INFO:tasks.workunit.client.1.smithi186.stdout:0/800: chown d0/f70 7 1 2024-04-12T20:56:39.651 INFO:tasks.workunit.client.1.smithi186.stdout:5/710: rmdir d5/db/d15/d2a/d2d/de3 0 2024-04-12T20:56:39.651 INFO:tasks.workunit.client.1.smithi186.stdout:0/801: stat d0/d10/d71/de8 0 2024-04-12T20:56:39.651 INFO:tasks.workunit.client.1.smithi186.stdout:2/796: truncate d0/d3e/f46 349460 0 2024-04-12T20:56:39.651 INFO:tasks.workunit.client.1.smithi186.stdout:2/797: fdatasync d0/fbe 0 2024-04-12T20:56:39.651 INFO:tasks.workunit.client.1.smithi186.stdout:4/827: creat d0/d59/dc0/d63/da6/dc6/f10e x:0 0 0 2024-04-12T20:56:39.656 INFO:tasks.workunit.client.1.smithi186.stdout:8/693: mknod d7/d18/d36/d20/cdb 0 2024-04-12T20:56:39.658 INFO:tasks.workunit.client.1.smithi186.stdout:3/664: creat d1/d3/d21/d22/d24/fd9 x:0 0 0 2024-04-12T20:56:39.659 INFO:tasks.workunit.client.1.smithi186.stdout:6/696: getdents d9/d43/d2d/d4d 0 2024-04-12T20:56:39.660 INFO:tasks.workunit.client.1.smithi186.stdout:0/802: chown d0/d14/d4e/d1e/l9b 1 1 2024-04-12T20:56:39.663 INFO:tasks.workunit.client.1.smithi186.stdout:8/694: write d7/de/f21 [1957698,51973] 0 2024-04-12T20:56:39.667 INFO:tasks.workunit.client.1.smithi186.stdout:0/803: chown d0/f6 114441 1 2024-04-12T20:56:39.667 INFO:tasks.workunit.client.1.smithi186.stdout:5/711: dread d5/db/f34 [0,4194304] 0 2024-04-12T20:56:39.667 INFO:tasks.workunit.client.1.smithi186.stdout:9/633: dwrite d3/d2f/d2e/d43/d4c/f58 [0,4194304] 0 2024-04-12T20:56:39.673 INFO:tasks.workunit.client.1.smithi186.stdout:1/714: dwrite d14/d1a/d1d/f9b [0,4194304] 0 2024-04-12T20:56:39.680 INFO:tasks.workunit.client.1.smithi186.stdout:8/695: truncate d7/d18/d36/d1d/d1f/da2/d5c/f91 370567 0 2024-04-12T20:56:39.681 INFO:tasks.workunit.client.1.smithi186.stdout:9/634: truncate d3/d2f/d2e/d33/d36/d70/f99 651867 0 2024-04-12T20:56:39.683 INFO:tasks.workunit.client.1.smithi186.stdout:8/696: chown d7/de/c55 20907231 1 2024-04-12T20:56:39.690 INFO:tasks.workunit.client.1.smithi186.stdout:7/690: getdents d7/d10/d2d/d40/d4a 0 2024-04-12T20:56:39.691 INFO:tasks.workunit.client.1.smithi186.stdout:4/828: dread d0/d21/f31 [0,4194304] 0 2024-04-12T20:56:39.694 INFO:tasks.workunit.client.1.smithi186.stdout:6/697: rename l7 to d9/d5e/d73/lfa 0 2024-04-12T20:56:39.697 INFO:tasks.workunit.client.1.smithi186.stdout:1/715: mknod d14/d1a/d33/d36/d5b/d6a/da2/ce7 0 2024-04-12T20:56:39.699 INFO:tasks.workunit.client.1.smithi186.stdout:1/716: fsync d14/fcf 0 2024-04-12T20:56:39.701 INFO:tasks.workunit.client.1.smithi186.stdout:8/697: mkdir d7/de/ddc 0 2024-04-12T20:56:39.701 INFO:tasks.workunit.client.1.smithi186.stdout:0/804: dread d0/d10/d32/d3f/f78 [0,4194304] 0 2024-04-12T20:56:39.703 INFO:tasks.workunit.client.1.smithi186.stdout:6/698: mknod d9/d4c/d6f/d89/d9c/dce/cfb 0 2024-04-12T20:56:39.705 INFO:tasks.workunit.client.1.smithi186.stdout:7/691: mknod d7/d10/d2d/d40/cf8 0 2024-04-12T20:56:39.708 INFO:tasks.workunit.client.1.smithi186.stdout:3/665: rename d1/d2c/d64/c6d to d1/d2c/cda 0 2024-04-12T20:56:39.710 INFO:tasks.workunit.client.1.smithi186.stdout:9/635: creat d3/d2f/d2e/fd8 x:0 0 0 2024-04-12T20:56:39.710 INFO:tasks.workunit.client.1.smithi186.stdout:3/666: write d1/f1e [4502148,76305] 0 2024-04-12T20:56:39.711 INFO:tasks.workunit.client.1.smithi186.stdout:4/829: dwrite d0/dc/ff1 [0,4194304] 0 2024-04-12T20:56:39.715 INFO:tasks.workunit.client.1.smithi186.stdout:7/692: creat d7/d10/d1c/d5d/ff9 x:0 0 0 2024-04-12T20:56:39.716 INFO:tasks.workunit.client.1.smithi186.stdout:8/698: mknod d7/de/ddc/cdd 0 2024-04-12T20:56:39.718 INFO:tasks.workunit.client.1.smithi186.stdout:2/798: rename d0/d32/d2a/fb7 to d0/d32/d8b/fff 0 2024-04-12T20:56:39.718 INFO:tasks.workunit.client.1.smithi186.stdout:7/693: write d7/d10/d2d/d40/d4a/d6c/d97/f78 [485974,117720] 0 2024-04-12T20:56:39.721 INFO:tasks.workunit.client.1.smithi186.stdout:2/799: read - d0/d3/d1a/d48/df7/faa zero size 2024-04-12T20:56:39.722 INFO:tasks.workunit.client.1.smithi186.stdout:0/805: creat d0/d14/d4e/d50/d6e/d96/f108 x:0 0 0 2024-04-12T20:56:39.722 INFO:tasks.workunit.client.1.smithi186.stdout:6/699: dread d9/d43/d2d/f5f [0,4194304] 0 2024-04-12T20:56:39.726 INFO:tasks.workunit.client.1.smithi186.stdout:9/636: creat d3/d12/d6f/d83/fd9 x:0 0 0 2024-04-12T20:56:39.727 INFO:tasks.workunit.client.1.smithi186.stdout:4/830: rmdir d0/d26 39 2024-04-12T20:56:39.735 INFO:tasks.workunit.client.1.smithi186.stdout:3/667: rename d1/d3/d49 to d1/d3/d21/d22/d34/ddb 0 2024-04-12T20:56:39.736 INFO:tasks.workunit.client.1.smithi186.stdout:7/694: truncate d7/d10/d1c/d8c/fb5 331847 0 2024-04-12T20:56:39.744 INFO:tasks.workunit.client.1.smithi186.stdout:8/699: creat d7/d18/d36/d20/dd8/fde x:0 0 0 2024-04-12T20:56:39.744 INFO:tasks.workunit.client.1.smithi186.stdout:9/637: rename d3/cbb to d3/d2f/d2e/d33/d36/d93/cda 0 2024-04-12T20:56:39.745 INFO:tasks.workunit.client.1.smithi186.stdout:8/700: chown d7/d18/d36/d20 8938 1 2024-04-12T20:56:39.745 INFO:tasks.workunit.client.1.smithi186.stdout:4/831: unlink d0/d21/fe9 0 2024-04-12T20:56:39.746 INFO:tasks.workunit.client.1.smithi186.stdout:8/701: chown d7 19803613 1 2024-04-12T20:56:39.747 INFO:tasks.workunit.client.1.smithi186.stdout:3/668: mkdir d1/d3/d21/d22/d24/d2f/ddc 0 2024-04-12T20:56:39.748 INFO:tasks.workunit.client.1.smithi186.stdout:3/669: chown d1/d2c/d64 225062 1 2024-04-12T20:56:39.749 INFO:tasks.workunit.client.1.smithi186.stdout:0/806: mknod d0/d14/d105/c109 0 2024-04-12T20:56:39.750 INFO:tasks.workunit.client.1.smithi186.stdout:2/800: dwrite d0/d32/d8b/fff [0,4194304] 0 2024-04-12T20:56:39.751 INFO:tasks.workunit.client.1.smithi186.stdout:0/807: readlink d0/d10/d32/l62 0 2024-04-12T20:56:39.755 INFO:tasks.workunit.client.1.smithi186.stdout:7/695: unlink d7/c3d 0 2024-04-12T20:56:39.759 INFO:tasks.workunit.client.1.smithi186.stdout:4/832: creat d0/d59/dc0/d63/d9b/db2/f10f x:0 0 0 2024-04-12T20:56:39.760 INFO:tasks.workunit.client.1.smithi186.stdout:7/696: chown d7/d10/d2d/d40/d4a/c5a 20 1 2024-04-12T20:56:39.760 INFO:tasks.workunit.client.1.smithi186.stdout:5/712: dwrite d5/db/f57 [0,4194304] 0 2024-04-12T20:56:39.766 INFO:tasks.workunit.client.1.smithi186.stdout:3/670: symlink d1/d3/d21/d22/d34/ddb/d4d/d6f/db0/ldd 0 2024-04-12T20:56:39.770 INFO:tasks.workunit.client.1.smithi186.stdout:9/638: dwrite d3/d2f/d2e/d43/d4c/f58 [4194304,4194304] 0 2024-04-12T20:56:39.770 INFO:tasks.workunit.client.1.smithi186.stdout:8/702: creat d7/d18/d36/d20/d23/d25/fdf x:0 0 0 2024-04-12T20:56:39.775 INFO:tasks.workunit.client.1.smithi186.stdout:9/639: fsync d3/fce 0 2024-04-12T20:56:39.780 INFO:tasks.workunit.client.1.smithi186.stdout:1/717: dwrite d14/d1a/d58/f97 [0,4194304] 0 2024-04-12T20:56:39.782 INFO:tasks.workunit.client.1.smithi186.stdout:2/801: dwrite d0/d3/d1a/d41/d8f/de9/ffd [0,4194304] 0 2024-04-12T20:56:39.783 INFO:tasks.workunit.client.1.smithi186.stdout:7/697: rename d7/d10/d2d/d40/cf8 to d7/d10/d2d/d40/d4a/d6c/d97/d4d/cfa 0 2024-04-12T20:56:39.784 INFO:tasks.workunit.client.1.smithi186.stdout:3/671: write d1/d3/d21/d22/d34/ddb/d4d/f51 [1580098,33245] 0 2024-04-12T20:56:39.788 INFO:tasks.workunit.client.1.smithi186.stdout:1/718: readlink d14/d1a/d33/d36/d5b/d3e/d8b/lb2 0 2024-04-12T20:56:39.792 INFO:tasks.workunit.client.1.smithi186.stdout:0/808: dwrite d0/d10/f8d [0,4194304] 0 2024-04-12T20:56:39.795 INFO:tasks.workunit.client.1.smithi186.stdout:3/672: truncate d1/d3/d21/d22/d34/fd6 904033 0 2024-04-12T20:56:39.798 INFO:tasks.workunit.client.1.smithi186.stdout:9/640: rename d3/d12/d6f/d83/fd9 to d3/d2f/d2e/d43/d4c/d82/dd4/fdb 0 2024-04-12T20:56:39.798 INFO:tasks.workunit.client.1.smithi186.stdout:3/673: fsync d1/d3/d21/d22/f79 0 2024-04-12T20:56:39.799 INFO:tasks.workunit.client.1.smithi186.stdout:4/833: dwrite d0/f42 [0,4194304] 0 2024-04-12T20:56:39.801 INFO:tasks.workunit.client.1.smithi186.stdout:4/834: stat d0/db4/f106 0 2024-04-12T20:56:39.801 INFO:tasks.workunit.client.1.smithi186.stdout:9/641: truncate d3/d13/fb7 595020 0 2024-04-12T20:56:39.802 INFO:tasks.workunit.client.1.smithi186.stdout:4/835: fdatasync d0/d59/dc0/d63/da6/dc6/f66 0 2024-04-12T20:56:39.810 INFO:tasks.workunit.client.1.smithi186.stdout:5/713: getdents d5/db/d15/d2a/d2d/d82 0 2024-04-12T20:56:39.817 INFO:tasks.workunit.client.1.smithi186.stdout:8/703: rename d7/d18/d36/d20/dd8/fde to d7/de/ddc/fe0 0 2024-04-12T20:56:39.821 INFO:tasks.workunit.client.1.smithi186.stdout:3/674: mknod d1/d3/d21/d22/d34/ddb/d4d/dad/cde 0 2024-04-12T20:56:39.822 INFO:tasks.workunit.client.1.smithi186.stdout:3/675: fdatasync d1/d3/da2/f9d 0 2024-04-12T20:56:39.822 INFO:tasks.workunit.client.1.smithi186.stdout:9/642: dread d3/d2f/d2e/d33/f44 [0,4194304] 0 2024-04-12T20:56:39.825 INFO:tasks.workunit.client.1.smithi186.stdout:3/676: write d1/d3/d21/d22/d34/ddb/d4d/dad/fcf [1002313,47613] 0 2024-04-12T20:56:39.826 INFO:tasks.workunit.client.1.smithi186.stdout:4/836: dread d0/dc/d33/f47 [0,4194304] 0 2024-04-12T20:56:39.832 INFO:tasks.workunit.client.1.smithi186.stdout:1/719: symlink d14/d1a/d1d/db8/le8 0 2024-04-12T20:56:39.841 INFO:tasks.workunit.client.1.smithi186.stdout:0/809: mknod d0/d85/dc4/c10a 0 2024-04-12T20:56:39.841 INFO:tasks.workunit.client.1.smithi186.stdout:2/802: creat d0/f100 x:0 0 0 2024-04-12T20:56:39.842 INFO:tasks.workunit.client.1.smithi186.stdout:9/643: creat d3/d2f/d2e/d43/d4c/fdc x:0 0 0 2024-04-12T20:56:39.844 INFO:tasks.workunit.client.1.smithi186.stdout:2/803: write d0/dc/d2e/d73/fb5 [510849,110234] 0 2024-04-12T20:56:39.845 INFO:tasks.workunit.client.1.smithi186.stdout:3/677: creat d1/d3/d21/d22/d34/ddb/d4d/d6f/db0/fdf x:0 0 0 2024-04-12T20:56:39.846 INFO:tasks.workunit.client.1.smithi186.stdout:3/678: fsync d1/d3/da/f70 0 2024-04-12T20:56:39.848 INFO:tasks.workunit.client.1.smithi186.stdout:6/700: sync 2024-04-12T20:56:39.849 INFO:tasks.workunit.client.1.smithi186.stdout:4/837: rmdir d0/dc/d64 39 2024-04-12T20:56:39.854 INFO:tasks.workunit.client.1.smithi186.stdout:7/698: unlink d7/d10/d2d/d40/d4a/d6c/d97/dd0/cf3 0 2024-04-12T20:56:39.855 INFO:tasks.workunit.client.1.smithi186.stdout:1/720: fsync d14/d1a/d33/d36/d5b/d3e/d71/f76 0 2024-04-12T20:56:39.856 INFO:tasks.workunit.client.1.smithi186.stdout:7/699: dread - d7/d55/d71/fc1 zero size 2024-04-12T20:56:39.859 INFO:tasks.workunit.client.1.smithi186.stdout:2/804: dwrite d0/d32/d2a/f62 [4194304,4194304] 0 2024-04-12T20:56:39.859 INFO:tasks.workunit.client.1.smithi186.stdout:8/704: symlink d7/d18/le1 0 2024-04-12T20:56:39.865 INFO:tasks.workunit.client.1.smithi186.stdout:9/644: creat d3/d2f/d2e/d33/fdd x:0 0 0 2024-04-12T20:56:39.871 INFO:tasks.workunit.client.1.smithi186.stdout:3/679: creat d1/d3/d21/d22/da4/fe0 x:0 0 0 2024-04-12T20:56:39.873 INFO:tasks.workunit.client.1.smithi186.stdout:4/838: unlink d0/d59/dc0/d48/dce/cfe 0 2024-04-12T20:56:39.877 INFO:tasks.workunit.client.1.smithi186.stdout:7/700: write d7/f74 [1164662,47993] 0 2024-04-12T20:56:39.879 INFO:tasks.workunit.client.1.smithi186.stdout:2/805: creat d0/d3/d7a/f101 x:0 0 0 2024-04-12T20:56:39.879 INFO:tasks.workunit.client.1.smithi186.stdout:8/705: creat d7/d18/d36/d1d/dc3/fe2 x:0 0 0 2024-04-12T20:56:39.881 INFO:tasks.workunit.client.1.smithi186.stdout:9/645: rename d3/d2f/d2e/l56 to d3/d2f/d2e/d33/d61/dbc/lde 0 2024-04-12T20:56:39.882 INFO:tasks.workunit.client.1.smithi186.stdout:2/806: readlink d0/d32/d2a/l76 0 2024-04-12T20:56:39.883 INFO:tasks.workunit.client.1.smithi186.stdout:4/839: chown d0/d21/ccf 36213 1 2024-04-12T20:56:39.885 INFO:tasks.workunit.client.1.smithi186.stdout:3/680: unlink d1/d3/d21/d22/d34/ddb/d4d/f75 0 2024-04-12T20:56:39.890 INFO:tasks.workunit.client.1.smithi186.stdout:1/721: unlink d14/d1a/d1d/l25 0 2024-04-12T20:56:39.891 INFO:tasks.workunit.client.1.smithi186.stdout:5/714: getdents d5/db/d15/d2a/d4c/d5b/d7a/d94 0 2024-04-12T20:56:39.893 INFO:tasks.workunit.client.1.smithi186.stdout:9/646: mknod d3/d2f/d2e/d43/d4c/d82/cdf 0 2024-04-12T20:56:39.893 INFO:tasks.workunit.client.1.smithi186.stdout:5/715: write d5/db/d15/d2a/d4c/d93/fcf [995660,25300] 0 2024-04-12T20:56:39.898 INFO:tasks.workunit.client.1.smithi186.stdout:3/681: rename d1/d2c/f67 to d1/d3/da/fe1 0 2024-04-12T20:56:39.900 INFO:tasks.workunit.client.1.smithi186.stdout:8/706: creat d7/de/d68/da7/dd5/fe3 x:0 0 0 2024-04-12T20:56:39.904 INFO:tasks.workunit.client.1.smithi186.stdout:4/840: mknod d0/dc/d88/d80/d9c/c110 0 2024-04-12T20:56:39.906 INFO:tasks.workunit.client.1.smithi186.stdout:0/810: dread d0/d14/d4e/d1e/d84/f97 [0,4194304] 0 2024-04-12T20:56:39.907 INFO:tasks.workunit.client.1.smithi186.stdout:8/707: read d7/d18/d36/d1d/d7d/da5/f7c [3869062,54020] 0 2024-04-12T20:56:39.907 INFO:tasks.workunit.client.1.smithi186.stdout:1/722: dread d14/d1a/d58/fb7 [0,4194304] 0 2024-04-12T20:56:39.908 INFO:tasks.workunit.client.1.smithi186.stdout:9/647: symlink d3/d12/d6f/le0 0 2024-04-12T20:56:39.909 INFO:tasks.workunit.client.1.smithi186.stdout:3/682: mknod d1/d3/d21/d22/d4a/dbc/ce2 0 2024-04-12T20:56:39.910 INFO:tasks.workunit.client.1.smithi186.stdout:8/708: dread - d7/de/ddc/fe0 zero size 2024-04-12T20:56:39.912 INFO:tasks.workunit.client.1.smithi186.stdout:6/701: dread d9/f22 [0,4194304] 0 2024-04-12T20:56:39.914 INFO:tasks.workunit.client.1.smithi186.stdout:1/723: rmdir d14/d1a/d21 39 2024-04-12T20:56:39.914 INFO:tasks.workunit.client.1.smithi186.stdout:0/811: truncate d0/d14/d4e/d1e/d84/dec/ffe 771803 0 2024-04-12T20:56:39.915 INFO:tasks.workunit.client.1.smithi186.stdout:6/702: truncate d9/f4b 4499004 0 2024-04-12T20:56:39.916 INFO:tasks.workunit.client.1.smithi186.stdout:6/703: stat d9/d5e/d73/c74 0 2024-04-12T20:56:39.917 INFO:tasks.workunit.client.1.smithi186.stdout:4/841: creat d0/dc/d1f/f111 x:0 0 0 2024-04-12T20:56:39.920 INFO:tasks.workunit.client.1.smithi186.stdout:6/704: chown d9/d5e/d7a/d81/c87 127916 1 2024-04-12T20:56:39.920 INFO:tasks.workunit.client.1.smithi186.stdout:4/842: write d0/dc/d1f/d73/f108 [963166,42944] 0 2024-04-12T20:56:39.921 INFO:tasks.workunit.client.1.smithi186.stdout:9/648: creat d3/d2f/d54/d9f/dc4/d77/dd6/d72/fe1 x:0 0 0 2024-04-12T20:56:39.923 INFO:tasks.workunit.client.1.smithi186.stdout:6/705: write d9/d43/f68 [1049937,70563] 0 2024-04-12T20:56:39.935 INFO:tasks.workunit.client.1.smithi186.stdout:8/709: creat d7/d18/d36/d20/d23/fe4 x:0 0 0 2024-04-12T20:56:39.936 INFO:tasks.workunit.client.1.smithi186.stdout:7/701: dwrite d7/d10/d2d/f89 [0,4194304] 0 2024-04-12T20:56:39.939 INFO:tasks.workunit.client.1.smithi186.stdout:2/807: truncate d0/f7e 239111 0 2024-04-12T20:56:39.945 INFO:tasks.workunit.client.1.smithi186.stdout:1/724: chown d14/d1a/d21/d35/dbc/de3/de4 155 1 2024-04-12T20:56:39.947 INFO:tasks.workunit.client.1.smithi186.stdout:3/683: dread d1/f8a [0,4194304] 0 2024-04-12T20:56:39.948 INFO:tasks.workunit.client.1.smithi186.stdout:9/649: mknod d3/d2f/d54/d9f/dc4/d77/dc3/dd2/ce2 0 2024-04-12T20:56:39.954 INFO:tasks.workunit.client.1.smithi186.stdout:4/843: dwrite d0/d59/ddf/ff0 [0,4194304] 0 2024-04-12T20:56:39.954 INFO:tasks.workunit.client.1.smithi186.stdout:5/716: dwrite d5/db/d71/f1a [0,4194304] 0 2024-04-12T20:56:39.954 INFO:tasks.workunit.client.1.smithi186.stdout:3/684: stat d1/d3/d21/d22/d24/d2f/l33 0 2024-04-12T20:56:39.956 INFO:tasks.workunit.client.1.smithi186.stdout:6/706: write d9/d43/d59/d5d/f5a [552254,52418] 0 2024-04-12T20:56:39.957 INFO:tasks.workunit.client.1.smithi186.stdout:8/710: write d7/de/f42 [3376936,51734] 0 2024-04-12T20:56:39.960 INFO:tasks.workunit.client.1.smithi186.stdout:7/702: truncate d7/d10/f13 3829864 0 2024-04-12T20:56:39.963 INFO:tasks.workunit.client.1.smithi186.stdout:9/650: symlink d3/d2f/d54/d9f/dc4/d77/dc3/le3 0 2024-04-12T20:56:39.963 INFO:tasks.workunit.client.1.smithi186.stdout:1/725: creat d14/d91/dbd/fe9 x:0 0 0 2024-04-12T20:56:39.965 INFO:tasks.workunit.client.1.smithi186.stdout:2/808: symlink d0/d3/d1a/d21/l102 0 2024-04-12T20:56:39.970 INFO:tasks.workunit.client.1.smithi186.stdout:2/809: write d0/d3/d1a/d24/db3/fc0 [418038,92676] 0 2024-04-12T20:56:39.975 INFO:tasks.workunit.client.1.smithi186.stdout:3/685: creat d1/d3/d21/d22/d24/d28/fe3 x:0 0 0 2024-04-12T20:56:39.975 INFO:tasks.workunit.client.1.smithi186.stdout:5/717: dwrite d5/db/d15/d2a/d2d/d48/fb5 [0,4194304] 0 2024-04-12T20:56:39.976 INFO:tasks.workunit.client.1.smithi186.stdout:8/711: readlink d7/de/d10/l3e 0 2024-04-12T20:56:39.980 INFO:tasks.workunit.client.1.smithi186.stdout:9/651: creat d3/d2f/d2e/d43/d4c/d82/dd4/fe4 x:0 0 0 2024-04-12T20:56:39.980 INFO:tasks.workunit.client.1.smithi186.stdout:6/707: dread d9/d43/f93 [0,4194304] 0 2024-04-12T20:56:39.982 INFO:tasks.workunit.client.1.smithi186.stdout:2/810: rename d0/d3/d1a/d48/f84 to d0/d32/d2a/f103 0 2024-04-12T20:56:39.988 INFO:tasks.workunit.client.1.smithi186.stdout:8/712: mknod d7/de/d10/d7f/ce5 0 2024-04-12T20:56:39.997 INFO:tasks.workunit.client.1.smithi186.stdout:2/811: truncate d0/dc/f37 2320513 0 2024-04-12T20:56:39.999 INFO:tasks.workunit.client.1.smithi186.stdout:3/686: mkdir d1/d3/d21/d22/d24/d2f/dd0/de4 0 2024-04-12T20:56:40.001 INFO:tasks.workunit.client.1.smithi186.stdout:6/708: symlink d9/d4c/d6f/lfc 0 2024-04-12T20:56:40.001 INFO:tasks.workunit.client.1.smithi186.stdout:3/687: chown d1/d3/d21/d22/d4a/l50 532228247 1 2024-04-12T20:56:40.002 INFO:tasks.workunit.client.1.smithi186.stdout:0/812: sync 2024-04-12T20:56:40.007 INFO:tasks.workunit.client.1.smithi186.stdout:7/703: link d7/d10/d2d/d40/d4a/d6c/d97/d4d/d84/fe8 d7/d10/d1c/d5d/db7/ffb 0 2024-04-12T20:56:40.007 INFO:tasks.workunit.client.1.smithi186.stdout:0/813: write d0/d10/d32/dac/fb6 [1780474,91096] 0 2024-04-12T20:56:40.007 INFO:tasks.workunit.client.1.smithi186.stdout:4/844: dwrite d0/dc/d64/d87/d98/fa7 [4194304,4194304] 0 2024-04-12T20:56:40.014 INFO:tasks.workunit.client.1.smithi186.stdout:1/726: getdents d14/d91/dbd/dc5 0 2024-04-12T20:56:40.014 INFO:tasks.workunit.client.1.smithi186.stdout:5/718: link d5/db/f11 d5/db/fec 0 2024-04-12T20:56:40.015 INFO:tasks.workunit.client.1.smithi186.stdout:6/709: dread - d9/d43/d59/f9f zero size 2024-04-12T20:56:40.016 INFO:tasks.workunit.client.1.smithi186.stdout:9/652: rename d3/d2f/d54/d9f/dc4/d77/dc3/dd2 to d3/d2f/d2e/d33/d36/de5 0 2024-04-12T20:56:40.017 INFO:tasks.workunit.client.1.smithi186.stdout:7/704: creat d7/ffc x:0 0 0 2024-04-12T20:56:40.018 INFO:tasks.workunit.client.1.smithi186.stdout:0/814: mknod d0/d10/d32/dac/c10b 0 2024-04-12T20:56:40.020 INFO:tasks.workunit.client.1.smithi186.stdout:5/719: write d5/db/d2e/fcc [361476,62486] 0 2024-04-12T20:56:40.020 INFO:tasks.workunit.client.1.smithi186.stdout:8/713: getdents d7/de/d10/d7f/dd3 0 2024-04-12T20:56:40.021 INFO:tasks.workunit.client.1.smithi186.stdout:2/812: dwrite d0/d3/d1a/d41/fd6 [0,4194304] 0 2024-04-12T20:56:40.024 INFO:tasks.workunit.client.1.smithi186.stdout:0/815: readlink d0/d14/d4e/d1e/lbd 0 2024-04-12T20:56:40.025 INFO:tasks.workunit.client.1.smithi186.stdout:7/705: stat d7/d10/d2d/d40/d4a/d5c/fda 0 2024-04-12T20:56:40.027 INFO:tasks.workunit.client.1.smithi186.stdout:3/688: getdents d1/d3/d21/d22/d24/d2f 0 2024-04-12T20:56:40.028 INFO:tasks.workunit.client.1.smithi186.stdout:5/720: write d5/db/d2e/dd0/fdb [993962,102805] 0 2024-04-12T20:56:40.028 INFO:tasks.workunit.client.1.smithi186.stdout:1/727: mknod d14/d1a/d33/d36/d5b/d3e/d71/d8c/d95/cea 0 2024-04-12T20:56:40.028 INFO:tasks.workunit.client.1.smithi186.stdout:0/816: chown d0/d14/d4e/d50/d6e/faf 0 1 2024-04-12T20:56:40.030 INFO:tasks.workunit.client.1.smithi186.stdout:9/653: symlink d3/d2f/d2e/db3/le6 0 2024-04-12T20:56:40.032 INFO:tasks.workunit.client.1.smithi186.stdout:4/845: fdatasync d0/f4d 0 2024-04-12T20:56:40.036 INFO:tasks.workunit.client.1.smithi186.stdout:6/710: dread d9/d4c/d6f/d89/d9c/fca [0,4194304] 0 2024-04-12T20:56:40.037 INFO:tasks.workunit.client.1.smithi186.stdout:2/813: creat d0/d3/f104 x:0 0 0 2024-04-12T20:56:40.037 INFO:tasks.workunit.client.1.smithi186.stdout:8/714: mkdir d7/d18/d36/d1d/d1f/d26/de6 0 2024-04-12T20:56:40.038 INFO:tasks.workunit.client.1.smithi186.stdout:7/706: truncate d7/d10/f15 3309802 0 2024-04-12T20:56:40.038 INFO:tasks.workunit.client.1.smithi186.stdout:4/846: write d0/d59/dc0/d63/da6/dc6/f66 [4212081,21308] 0 2024-04-12T20:56:40.039 INFO:tasks.workunit.client.1.smithi186.stdout:1/728: creat d14/d6c/feb x:0 0 0 2024-04-12T20:56:40.042 INFO:tasks.workunit.client.1.smithi186.stdout:0/817: creat d0/d10/d72/d79/f10c x:0 0 0 2024-04-12T20:56:40.043 INFO:tasks.workunit.client.1.smithi186.stdout:3/689: dread d1/d2c/d48/f57 [0,4194304] 0 2024-04-12T20:56:40.045 INFO:tasks.workunit.client.1.smithi186.stdout:5/721: rename d5/db/d15/d2a/d6d/d7f/d86/l95 to d5/db/d15/d2a/d4c/dc0/led 0 2024-04-12T20:56:40.046 INFO:tasks.workunit.client.1.smithi186.stdout:8/715: write d7/d18/d36/d1d/d1f/d26/dab/fae [44837,101524] 0 2024-04-12T20:56:40.048 INFO:tasks.workunit.client.1.smithi186.stdout:2/814: unlink d0/d3/d1a/d21/f4b 0 2024-04-12T20:56:40.051 INFO:tasks.workunit.client.1.smithi186.stdout:6/711: symlink d9/d5e/d7a/d81/d96/ddb/lfd 0 2024-04-12T20:56:40.054 INFO:tasks.workunit.client.1.smithi186.stdout:1/729: dread d14/d1a/d33/d36/d5b/d6a/da2/fa6 [0,4194304] 0 2024-04-12T20:56:40.055 INFO:tasks.workunit.client.1.smithi186.stdout:6/712: read d9/d4c/d6f/f7b [166645,84698] 0 2024-04-12T20:56:40.055 INFO:tasks.workunit.client.1.smithi186.stdout:4/847: dwrite d0/f42 [4194304,4194304] 0 2024-04-12T20:56:40.056 INFO:tasks.workunit.client.1.smithi186.stdout:0/818: rename d0/f88 to d0/d5d/ddc/f10d 0 2024-04-12T20:56:40.057 INFO:tasks.workunit.client.1.smithi186.stdout:8/716: rmdir d7/d18/d36/d1d/d7d/d81 39 2024-04-12T20:56:40.058 INFO:tasks.workunit.client.1.smithi186.stdout:0/819: write d0/d5c/f107 [270360,73038] 0 2024-04-12T20:56:40.060 INFO:tasks.workunit.client.1.smithi186.stdout:2/815: creat d0/d3/d1a/d24/f105 x:0 0 0 2024-04-12T20:56:40.063 INFO:tasks.workunit.client.1.smithi186.stdout:3/690: symlink d1/d3/d21/d22/dc6/le5 0 2024-04-12T20:56:40.066 INFO:tasks.workunit.client.1.smithi186.stdout:5/722: mknod d5/db/d15/d2a/d2d/d46/d55/cee 0 2024-04-12T20:56:40.067 INFO:tasks.workunit.client.1.smithi186.stdout:1/730: mknod d14/d1a/d33/d36/d5b/d3e/d71/cec 0 2024-04-12T20:56:40.068 INFO:tasks.workunit.client.1.smithi186.stdout:9/654: getdents d3/d2f/d2e/d33/d36/d93 0 2024-04-12T20:56:40.068 INFO:tasks.workunit.client.1.smithi186.stdout:1/731: chown l13 23066003 1 2024-04-12T20:56:40.070 INFO:tasks.workunit.client.1.smithi186.stdout:0/820: symlink d0/d5d/ddc/l10e 0 2024-04-12T20:56:40.073 INFO:tasks.workunit.client.1.smithi186.stdout:8/717: rename d7/d18/d36/d1d/fa3 to d7/de/d7a/fe7 0 2024-04-12T20:56:40.074 INFO:tasks.workunit.client.1.smithi186.stdout:4/848: truncate d0/d59/dc0/d63/da6/dc6/f82 34663 0 2024-04-12T20:56:40.076 INFO:tasks.workunit.client.1.smithi186.stdout:4/849: write d0/dc/d64/d87/d98/fa7 [4159572,5950] 0 2024-04-12T20:56:40.078 INFO:tasks.workunit.client.1.smithi186.stdout:2/816: symlink d0/d3/d1a/d21/d26/l106 0 2024-04-12T20:56:40.079 INFO:tasks.workunit.client.1.smithi186.stdout:3/691: symlink d1/d3/d21/d22/d34/ddb/d4d/d6f/da6/le6 0 2024-04-12T20:56:40.083 INFO:tasks.workunit.client.1.smithi186.stdout:5/723: unlink f2 0 2024-04-12T20:56:40.133 INFO:tasks.workunit.client.1.smithi186.stdout:3/692: unlink d1/d3/d21/d22/d34/lb7 0 2024-04-12T20:56:40.134 INFO:tasks.workunit.client.1.smithi186.stdout:1/732: rmdir d14/d1a/d33/d36/d5b/d3e 39 2024-04-12T20:56:40.138 INFO:tasks.workunit.client.1.smithi186.stdout:8/718: mkdir d7/d18/d36/d20/d23/dd6/de8 0 2024-04-12T20:56:40.138 INFO:tasks.workunit.client.1.smithi186.stdout:5/724: creat d5/db/d15/d2a/d6d/dbb/dc4/d5d/fef x:0 0 0 2024-04-12T20:56:40.142 INFO:tasks.workunit.client.1.smithi186.stdout:9/655: dread d3/d2f/d54/d9f/dc4/f57 [0,4194304] 0 2024-04-12T20:56:40.143 INFO:tasks.workunit.client.1.smithi186.stdout:4/850: creat d0/d21/db9/f112 x:0 0 0 2024-04-12T20:56:40.147 INFO:tasks.workunit.client.1.smithi186.stdout:6/713: dwrite d9/d43/f93 [0,4194304] 0 2024-04-12T20:56:40.151 INFO:tasks.workunit.client.1.smithi186.stdout:6/714: dread - d9/d43/d59/d5d/d6e/d91/dd9/ff8 zero size 2024-04-12T20:56:40.163 INFO:tasks.workunit.client.1.smithi186.stdout:7/707: sync 2024-04-12T20:56:40.212 INFO:tasks.workunit.client.1.smithi186.stdout:0/821: sync 2024-04-12T20:56:40.223 INFO:tasks.workunit.client.1.smithi186.stdout:3/693: symlink d1/d3/d21/d22/d4a/dbc/le7 0 2024-04-12T20:56:40.227 INFO:tasks.workunit.client.1.smithi186.stdout:2/817: link d0/d3/d1a/d21/d26/ced d0/d3e/c107 0 2024-04-12T20:56:40.228 INFO:tasks.workunit.client.1.smithi186.stdout:3/694: fdatasync d1/d3/d21/d22/d24/fd9 0 2024-04-12T20:56:40.231 INFO:tasks.workunit.client.1.smithi186.stdout:2/818: truncate d0/d32/f97 1693135 0 2024-04-12T20:56:40.251 INFO:tasks.workunit.client.1.smithi186.stdout:9/656: fdatasync d3/d12/fcd 0 2024-04-12T20:56:40.254 INFO:tasks.workunit.client.1.smithi186.stdout:4/851: chown d0/d59/dc0/d63/da6/daf 833988 1 2024-04-12T20:56:40.255 INFO:tasks.workunit.client.1.smithi186.stdout:1/733: dwrite d14/d1a/d21/d35/d59/fd2 [4194304,4194304] 0 2024-04-12T20:56:40.256 INFO:tasks.workunit.client.1.smithi186.stdout:4/852: fdatasync d0/d59/dc0/d63/ff5 0 2024-04-12T20:56:40.262 INFO:tasks.workunit.client.1.smithi186.stdout:1/734: stat d14/d1a/d21/d35/d59/fd2 0 2024-04-12T20:56:40.263 INFO:tasks.workunit.client.1.smithi186.stdout:7/708: creat d7/d55/ffd x:0 0 0 2024-04-12T20:56:40.267 INFO:tasks.workunit.client.1.smithi186.stdout:3/695: creat d1/d3/da2/fe8 x:0 0 0 2024-04-12T20:56:40.271 INFO:tasks.workunit.client.1.smithi186.stdout:8/719: creat d7/de/fe9 x:0 0 0 2024-04-12T20:56:40.272 INFO:tasks.workunit.client.1.smithi186.stdout:2/819: rmdir d0/d32/d8b 39 2024-04-12T20:56:40.279 INFO:tasks.workunit.client.1.smithi186.stdout:4/853: creat d0/dc/d64/d87/d98/f113 x:0 0 0 2024-04-12T20:56:40.284 INFO:tasks.workunit.client.1.smithi186.stdout:1/735: symlink d14/d6c/da9/led 0 2024-04-12T20:56:40.290 INFO:tasks.workunit.client.1.smithi186.stdout:3/696: creat d1/d3/da2/fe9 x:0 0 0 2024-04-12T20:56:40.294 INFO:tasks.workunit.client.1.smithi186.stdout:5/725: mknod d5/db/d2e/cf0 0 2024-04-12T20:56:40.296 INFO:tasks.workunit.client.1.smithi186.stdout:5/726: write d5/db/d15/f39 [466926,123592] 0 2024-04-12T20:56:40.299 INFO:tasks.workunit.client.1.smithi186.stdout:1/736: mknod d14/d1a/d21/d35/dbc/de3/cee 0 2024-04-12T20:56:40.302 INFO:tasks.workunit.client.1.smithi186.stdout:8/720: getdents d7/d18/d36/d20/d23/dd6/de8 0 2024-04-12T20:56:40.306 INFO:tasks.workunit.client.1.smithi186.stdout:6/715: getdents d9/d4c/d6f/de3 0 2024-04-12T20:56:40.308 INFO:tasks.workunit.client.1.smithi186.stdout:6/716: chown d9/c8e 993304 1 2024-04-12T20:56:40.313 INFO:tasks.workunit.client.1.smithi186.stdout:4/854: dread d0/d59/dc0/d63/da6/daf/fe2 [0,4194304] 0 2024-04-12T20:56:40.320 INFO:tasks.workunit.client.1.smithi186.stdout:1/737: dwrite d14/d1a/d21/d35/f6e [0,4194304] 0 2024-04-12T20:56:40.323 INFO:tasks.workunit.client.1.smithi186.stdout:8/721: write d7/de/ddc/fe0 [324056,45260] 0 2024-04-12T20:56:40.327 INFO:tasks.workunit.client.1.smithi186.stdout:8/722: dread - d7/d18/d36/d1d/d7d/fb5 zero size 2024-04-12T20:56:40.327 INFO:tasks.workunit.client.1.smithi186.stdout:6/717: rename d9/d4c/d6f/d89/d9c/dce to d9/d4c/dbe/dfe 0 2024-04-12T20:56:40.329 INFO:tasks.workunit.client.1.smithi186.stdout:8/723: chown d7/d18/d36/dc9 26 1 2024-04-12T20:56:40.348 INFO:tasks.workunit.client.1.smithi186.stdout:1/738: rename d14/d1a/d33/d36/d5b/d3e/d71/d8c/d95 to d14/d6c/def 0 2024-04-12T20:56:40.352 INFO:tasks.workunit.client.1.smithi186.stdout:3/697: getdents d1/d3/d21/d22/dc6 0 2024-04-12T20:56:40.355 INFO:tasks.workunit.client.1.smithi186.stdout:3/698: chown d1/d3/d21/d22/d24/fc7 7 1 2024-04-12T20:56:40.356 INFO:tasks.workunit.client.1.smithi186.stdout:6/718: mknod d9/d43/d59/cff 0 2024-04-12T20:56:40.360 INFO:tasks.workunit.client.1.smithi186.stdout:4/855: getdents d0/d59/dc0 0 2024-04-12T20:56:40.362 INFO:tasks.workunit.client.1.smithi186.stdout:9/657: sync 2024-04-12T20:56:40.363 INFO:tasks.workunit.client.1.smithi186.stdout:4/856: read d0/f6 [1964258,30336] 0 2024-04-12T20:56:40.364 INFO:tasks.workunit.client.1.smithi186.stdout:1/739: fsync d14/d1a/f5a 0 2024-04-12T20:56:40.374 INFO:tasks.workunit.client.1.smithi186.stdout:1/740: unlink d14/d1a/d1d/f9b 0 2024-04-12T20:56:40.380 INFO:tasks.workunit.client.1.smithi186.stdout:4/857: creat d0/db4/dd8/f114 x:0 0 0 2024-04-12T20:56:40.383 INFO:tasks.workunit.client.1.smithi186.stdout:8/724: link d7/de/d10/f19 d7/d18/d36/d1d/fea 0 2024-04-12T20:56:40.384 INFO:tasks.workunit.client.1.smithi186.stdout:3/699: dread d1/d3/da/f56 [0,4194304] 0 2024-04-12T20:56:40.387 INFO:tasks.workunit.client.1.smithi186.stdout:1/741: fdatasync d14/f15 0 2024-04-12T20:56:40.388 INFO:tasks.workunit.client.1.smithi186.stdout:4/858: creat d0/d59/ddf/f115 x:0 0 0 2024-04-12T20:56:40.391 INFO:tasks.workunit.client.1.smithi186.stdout:6/719: getdents d9/d5e 0 2024-04-12T20:56:40.391 INFO:tasks.workunit.client.1.smithi186.stdout:9/658: rename d3/d2f/d54/f60 to d3/d2f/d2e/fe7 0 2024-04-12T20:56:40.394 INFO:tasks.workunit.client.1.smithi186.stdout:6/720: read d9/f10 [2451487,50573] 0 2024-04-12T20:56:40.395 INFO:tasks.workunit.client.1.smithi186.stdout:9/659: chown d3/d2f/d2e/d33/fb9 314 1 2024-04-12T20:56:40.397 INFO:tasks.workunit.client.1.smithi186.stdout:9/660: fsync d3/d12/fcf 0 2024-04-12T20:56:40.405 INFO:tasks.workunit.client.1.smithi186.stdout:2/820: sync 2024-04-12T20:56:40.406 INFO:tasks.workunit.client.1.smithi186.stdout:4/859: unlink d0/dc/d64/d87/dd2/fd7 0 2024-04-12T20:56:40.408 INFO:tasks.workunit.client.1.smithi186.stdout:6/721: fdatasync d9/d43/d59/fab 0 2024-04-12T20:56:40.409 INFO:tasks.workunit.client.1.smithi186.stdout:8/725: dwrite d7/de/f21 [4194304,4194304] 0 2024-04-12T20:56:40.414 INFO:tasks.workunit.client.1.smithi186.stdout:9/661: mknod d3/d2f/d2e/d33/d36/ce8 0 2024-04-12T20:56:40.417 INFO:tasks.workunit.client.1.smithi186.stdout:3/700: dwrite d1/fc [4194304,4194304] 0 2024-04-12T20:56:40.419 INFO:tasks.workunit.client.1.smithi186.stdout:2/821: symlink d0/d32/l108 0 2024-04-12T20:56:40.423 INFO:tasks.workunit.client.1.smithi186.stdout:8/726: fdatasync d7/de/d7a/f4f 0 2024-04-12T20:56:40.426 INFO:tasks.workunit.client.1.smithi186.stdout:6/722: mkdir d9/da9/dbd/d100 0 2024-04-12T20:56:40.427 INFO:tasks.workunit.client.1.smithi186.stdout:8/727: write d7/d18/d36/d20/d23/d25/f56 [2877211,105092] 0 2024-04-12T20:56:40.428 INFO:tasks.workunit.client.1.smithi186.stdout:0/822: write d0/d14/d4e/d1e/d84/fa2 [219847,121487] 0 2024-04-12T20:56:40.428 INFO:tasks.workunit.client.1.smithi186.stdout:7/709: write d7/d10/d1c/d5d/db7/ffb [1023075,9800] 0 2024-04-12T20:56:40.431 INFO:tasks.workunit.client.1.smithi186.stdout:9/662: mknod d3/d2f/d54/d9f/dc4/ce9 0 2024-04-12T20:56:40.443 INFO:tasks.workunit.client.1.smithi186.stdout:3/701: dread d1/d3/d21/d22/d34/ddb/d4d/d6f/f99 [0,4194304] 0 2024-04-12T20:56:40.443 INFO:tasks.workunit.client.1.smithi186.stdout:7/710: unlink d7/d10/f95 0 2024-04-12T20:56:40.449 INFO:tasks.workunit.client.1.smithi186.stdout:0/823: dwrite d0/d10/d32/dac/fef [0,4194304] 0 2024-04-12T20:56:40.452 INFO:tasks.workunit.client.1.smithi186.stdout:6/723: rename d9/d5e/d7a/d81/d96 to d9/d43/d59/d5d/d101 0 2024-04-12T20:56:40.452 INFO:tasks.workunit.client.1.smithi186.stdout:5/727: dwrite d5/db/d71/f75 [8388608,4194304] 0 2024-04-12T20:56:40.453 INFO:tasks.workunit.client.1.smithi186.stdout:6/724: stat d9/da9/dbd/d100 0 2024-04-12T20:56:40.455 INFO:tasks.workunit.client.1.smithi186.stdout:8/728: rename d7/d18/d36/d1d/d1f/da2/d5c/f91 to d7/d18/d36/d20/d71/feb 0 2024-04-12T20:56:40.462 INFO:tasks.workunit.client.1.smithi186.stdout:4/860: symlink d0/d59/dc0/d63/l116 0 2024-04-12T20:56:40.466 INFO:tasks.workunit.client.1.smithi186.stdout:4/861: write d0/dc/d1f/d73/fb7 [112630,69200] 0 2024-04-12T20:56:40.468 INFO:tasks.workunit.client.1.smithi186.stdout:5/728: rename d5/d27/cde to d5/d27/cf1 0 2024-04-12T20:56:40.471 INFO:tasks.workunit.client.1.smithi186.stdout:0/824: rename d0/d14/d4e/d50/fd5 to d0/d5d/ddc/f10f 0 2024-04-12T20:56:40.471 INFO:tasks.workunit.client.1.smithi186.stdout:5/729: read - d5/db/d15/d2a/d2d/d46/d55/fd5 zero size 2024-04-12T20:56:40.472 INFO:tasks.workunit.client.1.smithi186.stdout:0/825: dread - d0/d5c/d66/fcf zero size 2024-04-12T20:56:40.473 INFO:tasks.workunit.client.1.smithi186.stdout:5/730: readlink d5/d27/l85 0 2024-04-12T20:56:40.476 INFO:tasks.workunit.client.1.smithi186.stdout:5/731: write d5/db/d71/d88/fd4 [401064,30056] 0 2024-04-12T20:56:40.479 INFO:tasks.workunit.client.1.smithi186.stdout:5/732: chown d5/db/d15/d2a/d6d/d7f/d86/db6/l9c 12 1 2024-04-12T20:56:40.480 INFO:tasks.workunit.client.1.smithi186.stdout:0/826: creat d0/d14/d4e/d50/f110 x:0 0 0 2024-04-12T20:56:40.487 INFO:tasks.workunit.client.1.smithi186.stdout:0/827: read d0/d5c/d66/da7/fb4 [173274,48852] 0 2024-04-12T20:56:40.487 INFO:tasks.workunit.client.1.smithi186.stdout:4/862: dwrite d0/dc/d88/d80/fdc [0,4194304] 0 2024-04-12T20:56:40.490 INFO:tasks.workunit.client.1.smithi186.stdout:7/711: dread d7/d55/f9f [0,4194304] 0 2024-04-12T20:56:40.494 INFO:tasks.workunit.client.1.smithi186.stdout:3/702: dread d1/d3/d21/d22/d24/d2f/f92 [0,4194304] 0 2024-04-12T20:56:40.498 INFO:tasks.workunit.client.1.smithi186.stdout:0/828: creat d0/d85/f111 x:0 0 0 2024-04-12T20:56:40.501 INFO:tasks.workunit.client.1.smithi186.stdout:1/742: dwrite d14/d1a/d33/d36/d5b/f67 [0,4194304] 0 2024-04-12T20:56:40.502 INFO:tasks.workunit.client.1.smithi186.stdout:7/712: chown d7/d10/d2d/d40/d4a/d6c/d97/f78 1445 1 2024-04-12T20:56:40.503 INFO:tasks.workunit.client.1.smithi186.stdout:0/829: chown d0/d14/d4e/d1e/f1f 7 1 2024-04-12T20:56:40.503 INFO:tasks.workunit.client.1.smithi186.stdout:4/863: getdents d0/dc/d88/d80/df8 0 2024-04-12T20:56:40.507 INFO:tasks.workunit.client.1.smithi186.stdout:6/725: write d9/f1b [930422,16446] 0 2024-04-12T20:56:40.515 INFO:tasks.workunit.client.1.smithi186.stdout:3/703: link d1/c1a d1/d3/d21/d22/d24/d2f/d95/cea 0 2024-04-12T20:56:40.517 INFO:tasks.workunit.client.1.smithi186.stdout:5/733: dread d5/db/f40 [0,4194304] 0 2024-04-12T20:56:40.517 INFO:tasks.workunit.client.1.smithi186.stdout:7/713: creat d7/d10/d2d/d40/d4a/d6c/ddf/ffe x:0 0 0 2024-04-12T20:56:40.517 INFO:tasks.workunit.client.1.smithi186.stdout:9/663: dwrite d3/d2f/d2e/d33/f44 [0,4194304] 0 2024-04-12T20:56:40.517 INFO:tasks.workunit.client.1.smithi186.stdout:6/726: creat d9/d4c/dbe/f102 x:0 0 0 2024-04-12T20:56:40.521 INFO:tasks.workunit.client.1.smithi186.stdout:0/830: write d0/d10/f4d [2504062,6581] 0 2024-04-12T20:56:40.521 INFO:tasks.workunit.client.1.smithi186.stdout:4/864: dread d0/dc/fa2 [0,4194304] 0 2024-04-12T20:56:40.525 INFO:tasks.workunit.client.1.smithi186.stdout:4/865: readlink d0/dc/d64/d87/l100 0 2024-04-12T20:56:40.529 INFO:tasks.workunit.client.1.smithi186.stdout:9/664: write d3/d2f/d2e/d43/fa7 [2424529,51667] 0 2024-04-12T20:56:40.531 INFO:tasks.workunit.client.1.smithi186.stdout:5/734: fsync d5/db/d15/d2a/d6d/dbb/dc4/d5d/f72 0 2024-04-12T20:56:40.531 INFO:tasks.workunit.client.1.smithi186.stdout:2/822: sync 2024-04-12T20:56:40.532 INFO:tasks.workunit.client.1.smithi186.stdout:0/831: chown d0/d14/cc2 29763 1 2024-04-12T20:56:40.532 INFO:tasks.workunit.client.1.smithi186.stdout:7/714: mknod d7/d10/d2d/d40/d4a/d6c/ddf/cff 0 2024-04-12T20:56:40.538 INFO:tasks.workunit.client.1.smithi186.stdout:0/832: chown d0/d14/d4e/d1e/fa9 138 1 2024-04-12T20:56:40.539 INFO:tasks.workunit.client.1.smithi186.stdout:4/866: creat d0/db4/dd8/d104/f117 x:0 0 0 2024-04-12T20:56:40.539 INFO:tasks.workunit.client.1.smithi186.stdout:1/743: dwrite d14/d1a/d21/d35/fc4 [0,4194304] 0 2024-04-12T20:56:40.541 INFO:tasks.workunit.client.1.smithi186.stdout:9/665: symlink d3/d2f/d54/d9f/dc4/d77/dc3/daf/lea 0 2024-04-12T20:56:40.543 INFO:tasks.workunit.client.1.smithi186.stdout:7/715: chown d7/d10/d2d/d40/d4a/ff2 0 1 2024-04-12T20:56:40.544 INFO:tasks.workunit.client.1.smithi186.stdout:2/823: chown d0/d3/d1a/d41/d8f/lb4 16213961 1 2024-04-12T20:56:40.545 INFO:tasks.workunit.client.1.smithi186.stdout:3/704: dwrite d1/d3/d21/fb2 [0,4194304] 0 2024-04-12T20:56:40.550 INFO:tasks.workunit.client.1.smithi186.stdout:7/716: chown d7/d10/c3a 289674 1 2024-04-12T20:56:40.551 INFO:tasks.workunit.client.1.smithi186.stdout:0/833: mknod d0/d85/dc4/dc1/c112 0 2024-04-12T20:56:40.551 INFO:tasks.workunit.client.1.smithi186.stdout:6/727: link d9/d43/c30 d9/d43/d59/d5d/d6e/d91/dd9/c103 0 2024-04-12T20:56:40.552 INFO:tasks.workunit.client.1.smithi186.stdout:5/735: rename d5/db/d15/d2a/d6d/d7f/cb3 to d5/db/d15/d2a/d6d/cf2 0 2024-04-12T20:56:40.552 INFO:tasks.workunit.client.1.smithi186.stdout:9/666: mknod d3/d2f/d2e/d43/ceb 0 2024-04-12T20:56:40.557 INFO:tasks.workunit.client.1.smithi186.stdout:5/736: write d5/db/d15/fea [2589324,26263] 0 2024-04-12T20:56:40.561 INFO:tasks.workunit.client.1.smithi186.stdout:9/667: chown d3/d12/d6f/d83/f38 416 1 2024-04-12T20:56:40.565 INFO:tasks.workunit.client.1.smithi186.stdout:5/737: write d5/db/d71/f1a [849003,79016] 0 2024-04-12T20:56:40.565 INFO:tasks.workunit.client.1.smithi186.stdout:1/744: dread d14/d1a/d21/d35/fc4 [0,4194304] 0 2024-04-12T20:56:40.570 INFO:tasks.workunit.client.1.smithi186.stdout:7/717: rename d7/d10/d2d/d40/d4a/cb3 to d7/d10/d2d/d40/d4a/d6c/d97/d4d/d8a/dad/c100 0 2024-04-12T20:56:40.578 INFO:tasks.workunit.client.1.smithi186.stdout:5/738: stat d5/db/d15/d2a/d2d/d46/f6c 0 2024-04-12T20:56:40.578 INFO:tasks.workunit.client.1.smithi186.stdout:7/718: readlink d7/d10/d1c/d8c/ld1 0 2024-04-12T20:56:40.579 INFO:tasks.workunit.client.1.smithi186.stdout:2/824: dwrite d0/d3/f58 [0,4194304] 0 2024-04-12T20:56:40.579 INFO:tasks.workunit.client.1.smithi186.stdout:9/668: truncate d3/d2f/d2e/d33/f94 178455 0 2024-04-12T20:56:40.579 INFO:tasks.workunit.client.1.smithi186.stdout:1/745: creat d14/d1a/d33/d36/d5b/d3e/d8b/ff0 x:0 0 0 2024-04-12T20:56:40.580 INFO:tasks.workunit.client.1.smithi186.stdout:4/867: dwrite d0/dc/f2d [0,4194304] 0 2024-04-12T20:56:40.583 INFO:tasks.workunit.client.1.smithi186.stdout:5/739: read d5/db/d15/f1e [4293382,41378] 0 2024-04-12T20:56:40.583 INFO:tasks.workunit.client.1.smithi186.stdout:0/834: dwrite d0/d10/d32/d3f/f54 [0,4194304] 0 2024-04-12T20:56:40.586 INFO:tasks.workunit.client.1.smithi186.stdout:4/868: write d0/dc/d64/d87/d98/fcd [912937,2798] 0 2024-04-12T20:56:40.588 INFO:tasks.workunit.client.1.smithi186.stdout:3/705: dread d1/f1e [0,4194304] 0 2024-04-12T20:56:40.589 INFO:tasks.workunit.client.1.smithi186.stdout:8/729: sync 2024-04-12T20:56:40.589 INFO:tasks.workunit.client.1.smithi186.stdout:6/728: truncate d9/d5e/f6d 6135740 0 2024-04-12T20:56:40.592 INFO:tasks.workunit.client.1.smithi186.stdout:3/706: stat d1/d2c/d64/c89 0 2024-04-12T20:56:40.594 INFO:tasks.workunit.client.1.smithi186.stdout:8/730: write d7/de/d7a/fd9 [78422,42572] 0 2024-04-12T20:56:40.597 INFO:tasks.workunit.client.1.smithi186.stdout:7/719: write d7/fc [4010544,72785] 0 2024-04-12T20:56:40.599 INFO:tasks.workunit.client.1.smithi186.stdout:1/746: creat d14/d1a/d33/d36/d5b/d6a/da2/ff1 x:0 0 0 2024-04-12T20:56:40.601 INFO:tasks.workunit.client.1.smithi186.stdout:1/747: chown d14/d1a/d21 146 1 2024-04-12T20:56:40.603 INFO:tasks.workunit.client.1.smithi186.stdout:9/669: rename l0 to d3/d2f/d2e/d43/d4c/d82/lec 0 2024-04-12T20:56:40.604 INFO:tasks.workunit.client.1.smithi186.stdout:5/740: mknod d5/db/d71/cf3 0 2024-04-12T20:56:40.616 INFO:tasks.workunit.client.1.smithi186.stdout:8/731: rmdir d7/d18/d36/dad 39 2024-04-12T20:56:40.617 INFO:tasks.workunit.client.1.smithi186.stdout:2/825: dread d0/d3e/f46 [0,4194304] 0 2024-04-12T20:56:40.618 INFO:tasks.workunit.client.1.smithi186.stdout:9/670: write d3/d2f/d2e/d43/fa7 [1729396,32793] 0 2024-04-12T20:56:40.618 INFO:tasks.workunit.client.1.smithi186.stdout:1/748: fsync d14/d6c/def/fc0 0 2024-04-12T20:56:40.620 INFO:tasks.workunit.client.1.smithi186.stdout:3/707: rename d1/d2c/d54/d7c/fa9 to d1/d3/d21/d22/d4a/feb 0 2024-04-12T20:56:40.620 INFO:tasks.workunit.client.1.smithi186.stdout:5/741: chown d5/db/d15/d2a/d6d/dbb/dc4/d5d/de1 146874 1 2024-04-12T20:56:40.621 INFO:tasks.workunit.client.1.smithi186.stdout:2/826: dread - d0/d3e/fe6 zero size 2024-04-12T20:56:40.622 INFO:tasks.workunit.client.1.smithi186.stdout:0/835: dwrite d0/d14/f35 [0,4194304] 0 2024-04-12T20:56:40.623 INFO:tasks.workunit.client.1.smithi186.stdout:7/720: mknod d7/d10/d2d/c101 0 2024-04-12T20:56:40.624 INFO:tasks.workunit.client.1.smithi186.stdout:3/708: write d1/d3/f7e [525522,28076] 0 2024-04-12T20:56:40.624 INFO:tasks.workunit.client.1.smithi186.stdout:4/869: write d0/d59/dc0/f7d [4423332,32662] 0 2024-04-12T20:56:40.627 INFO:tasks.workunit.client.1.smithi186.stdout:8/732: write d7/d18/d36/d1d/d1f/d26/f3f [2436699,95187] 0 2024-04-12T20:56:40.628 INFO:tasks.workunit.client.1.smithi186.stdout:0/836: read d0/f6 [4251486,80264] 0 2024-04-12T20:56:40.629 INFO:tasks.workunit.client.1.smithi186.stdout:1/749: creat d14/d1a/d21/d35/dbc/ff2 x:0 0 0 2024-04-12T20:56:40.632 INFO:tasks.workunit.client.1.smithi186.stdout:5/742: fsync d5/db/d71/f81 0 2024-04-12T20:56:40.635 INFO:tasks.workunit.client.1.smithi186.stdout:8/733: write d7/de/d68/f9d [5013999,75841] 0 2024-04-12T20:56:40.639 INFO:tasks.workunit.client.1.smithi186.stdout:0/837: mknod d0/d81/c113 0 2024-04-12T20:56:40.641 INFO:tasks.workunit.client.1.smithi186.stdout:2/827: creat d0/d32/d8b/f109 x:0 0 0 2024-04-12T20:56:40.642 INFO:tasks.workunit.client.1.smithi186.stdout:4/870: mkdir d0/dc/df2/d118 0 2024-04-12T20:56:40.643 INFO:tasks.workunit.client.1.smithi186.stdout:9/671: dwrite d3/d13/f4e [0,4194304] 0 2024-04-12T20:56:40.645 INFO:tasks.workunit.client.1.smithi186.stdout:3/709: creat d1/d3/d21/d22/d24/d2f/ddc/fec x:0 0 0 2024-04-12T20:56:40.645 INFO:tasks.workunit.client.1.smithi186.stdout:5/743: write d5/db/d15/d2a/d2d/d48/f87 [4142784,50748] 0 2024-04-12T20:56:40.647 INFO:tasks.workunit.client.1.smithi186.stdout:7/721: dwrite d7/de1/fc5 [0,4194304] 0 2024-04-12T20:56:40.649 INFO:tasks.workunit.client.1.smithi186.stdout:8/734: creat d7/de/d68/da7/fec x:0 0 0 2024-04-12T20:56:40.649 INFO:tasks.workunit.client.1.smithi186.stdout:4/871: chown d0/dc/d64/d87/d98/fa7 5 1 2024-04-12T20:56:40.650 INFO:tasks.workunit.client.1.smithi186.stdout:4/872: stat d0/f6 0 2024-04-12T20:56:40.651 INFO:tasks.workunit.client.1.smithi186.stdout:3/710: mkdir d1/d2c/d48/ded 0 2024-04-12T20:56:40.653 INFO:tasks.workunit.client.1.smithi186.stdout:2/828: write d0/d3/d1a/d21/d26/fc6 [682382,39065] 0 2024-04-12T20:56:40.670 INFO:tasks.workunit.client.1.smithi186.stdout:2/829: dread - d0/d3/d1a/d48/df7/fa4 zero size 2024-04-12T20:56:40.676 INFO:tasks.workunit.client.1.smithi186.stdout:2/830: write d0/d3/d1a/d21/d26/f9a [1600938,54457] 0 2024-04-12T20:56:40.678 INFO:tasks.workunit.client.1.smithi186.stdout:0/838: dwrite d0/d14/d4e/d1e/d84/dec/f102 [0,4194304] 0 2024-04-12T20:56:40.679 INFO:tasks.workunit.client.1.smithi186.stdout:9/672: dwrite d3/d13/fc6 [0,4194304] 0 2024-04-12T20:56:40.682 INFO:tasks.workunit.client.1.smithi186.stdout:2/831: chown d0/d3/d1a/d21/d22/c64 3898360 1 2024-04-12T20:56:40.684 INFO:tasks.workunit.client.1.smithi186.stdout:1/750: write d14/d1a/d1d/f46 [3023024,125899] 0 2024-04-12T20:56:40.686 INFO:tasks.workunit.client.1.smithi186.stdout:0/839: stat d0/d5d/cc7 0 2024-04-12T20:56:40.691 INFO:tasks.workunit.client.1.smithi186.stdout:3/711: dread d1/fc [0,4194304] 0 2024-04-12T20:56:40.692 INFO:tasks.workunit.client.1.smithi186.stdout:5/744: readlink d5/db/d15/d2a/d2d/d46/d55/l3c 0 2024-04-12T20:56:40.699 INFO:tasks.workunit.client.1.smithi186.stdout:2/832: mkdir d0/d3/d1a/d41/d10a 0 2024-04-12T20:56:40.699 INFO:tasks.workunit.client.1.smithi186.stdout:0/840: truncate d0/d5c/d65/f7b 4013708 0 2024-04-12T20:56:40.699 INFO:tasks.workunit.client.1.smithi186.stdout:3/712: creat d1/d3/d21/d22/d24/d28/fee x:0 0 0 2024-04-12T20:56:40.700 INFO:tasks.workunit.client.1.smithi186.stdout:1/751: link d14/d1a/d33/d36/d5b/d3e/fd7 d14/d1a/d1d/db8/ff3 0 2024-04-12T20:56:40.702 INFO:tasks.workunit.client.1.smithi186.stdout:4/873: dread d0/f10 [0,4194304] 0 2024-04-12T20:56:40.703 INFO:tasks.workunit.client.1.smithi186.stdout:7/722: getdents d7/d10/d1c/d5d/db7 0 2024-04-12T20:56:40.706 INFO:tasks.workunit.client.1.smithi186.stdout:1/752: chown d14/d1a/d33/f6b 41720 1 2024-04-12T20:56:40.706 INFO:tasks.workunit.client.1.smithi186.stdout:9/673: dwrite d3/d2f/f68 [4194304,4194304] 0 2024-04-12T20:56:40.707 INFO:tasks.workunit.client.1.smithi186.stdout:1/753: readlink d14/d1a/d33/d36/lab 0 2024-04-12T20:56:40.709 INFO:tasks.workunit.client.1.smithi186.stdout:8/735: getdents d7/d18/d36/d1d/d1f/d26/dab 0 2024-04-12T20:56:40.710 INFO:tasks.workunit.client.1.smithi186.stdout:4/874: symlink d0/dc/d64/d87/dd2/l119 0 2024-04-12T20:56:40.713 INFO:tasks.workunit.client.1.smithi186.stdout:8/736: read f2 [5256981,34356] 0 2024-04-12T20:56:40.713 INFO:tasks.workunit.client.1.smithi186.stdout:5/745: creat d5/db/d15/d2a/d6d/d7f/ff4 x:0 0 0 2024-04-12T20:56:40.714 INFO:tasks.workunit.client.1.smithi186.stdout:7/723: dread - d7/d55/d71/f85 zero size 2024-04-12T20:56:40.714 INFO:tasks.workunit.client.1.smithi186.stdout:6/729: sync 2024-04-12T20:56:40.716 INFO:tasks.workunit.client.1.smithi186.stdout:0/841: dwrite d0/d10/d32/d3f/f54 [0,4194304] 0 2024-04-12T20:56:40.722 INFO:tasks.workunit.client.1.smithi186.stdout:3/713: mknod d1/d3/d21/d22/d24/d2f/d95/cef 0 2024-04-12T20:56:40.724 INFO:tasks.workunit.client.1.smithi186.stdout:1/754: creat d14/d1a/d33/d36/d5b/d3e/d5f/ff4 x:0 0 0 2024-04-12T20:56:40.724 INFO:tasks.workunit.client.1.smithi186.stdout:2/833: truncate d0/d32/d2a/f62 4351537 0 2024-04-12T20:56:40.725 INFO:tasks.workunit.client.1.smithi186.stdout:2/834: readlink d0/d3e/l3f 0 2024-04-12T20:56:40.726 INFO:tasks.workunit.client.1.smithi186.stdout:8/737: mknod d7/d18/d36/d1d/d1f/d26/dab/ced 0 2024-04-12T20:56:40.733 INFO:tasks.workunit.client.1.smithi186.stdout:5/746: rename d5/db/d15/d2a/d2d/d82/c9e to d5/db/d15/d2a/d6d/d7f/d80/cf5 0 2024-04-12T20:56:40.736 INFO:tasks.workunit.client.1.smithi186.stdout:4/875: truncate d0/dc/d33/f68 1675717 0 2024-04-12T20:56:40.738 INFO:tasks.workunit.client.1.smithi186.stdout:5/747: truncate d5/db/d15/d2a/d6d/d7f/ff4 115837 0 2024-04-12T20:56:40.738 INFO:tasks.workunit.client.1.smithi186.stdout:6/730: symlink d9/d43/d59/d5d/l104 0 2024-04-12T20:56:40.741 INFO:tasks.workunit.client.1.smithi186.stdout:3/714: truncate d1/f8a 4164950 0 2024-04-12T20:56:40.744 INFO:tasks.workunit.client.1.smithi186.stdout:0/842: dread d0/d10/d71/f89 [0,4194304] 0 2024-04-12T20:56:40.744 INFO:tasks.workunit.client.1.smithi186.stdout:4/876: creat d0/d59/dc0/d63/f11a x:0 0 0 2024-04-12T20:56:40.747 INFO:tasks.workunit.client.1.smithi186.stdout:8/738: rename d7/de/d68/da7/cb2 to d7/de/d10/d7f/cee 0 2024-04-12T20:56:40.750 INFO:tasks.workunit.client.1.smithi186.stdout:6/731: creat d9/d4c/d6f/d89/d9c/f105 x:0 0 0 2024-04-12T20:56:40.752 INFO:tasks.workunit.client.1.smithi186.stdout:3/715: stat d1/d2c/cda 0 2024-04-12T20:56:40.756 INFO:tasks.workunit.client.1.smithi186.stdout:4/877: creat d0/dc/d64/d87/d98/f11b x:0 0 0 2024-04-12T20:56:40.757 INFO:tasks.workunit.client.1.smithi186.stdout:1/755: dread d14/d1a/d33/d36/d5b/d3e/d5f/f8f [0,4194304] 0 2024-04-12T20:56:40.761 INFO:tasks.workunit.client.1.smithi186.stdout:4/878: rmdir d0/dc/d1f/d73 39 2024-04-12T20:56:40.765 INFO:tasks.workunit.client.1.smithi186.stdout:8/739: dwrite d7/de/ddc/fe0 [0,4194304] 0 2024-04-12T20:56:40.767 INFO:tasks.workunit.client.1.smithi186.stdout:1/756: write d14/d1a/d21/d35/fc4 [1752763,2335] 0 2024-04-12T20:56:40.768 INFO:tasks.workunit.client.1.smithi186.stdout:8/740: fsync d7/d18/d36/fa 0 2024-04-12T20:56:40.769 INFO:tasks.workunit.client.1.smithi186.stdout:4/879: creat d0/dc/d64/d87/d98/f11c x:0 0 0 2024-04-12T20:56:40.772 INFO:tasks.workunit.client.1.smithi186.stdout:5/748: dread d5/db/d15/d2a/d4c/d5b/d7a/d94/fa3 [0,4194304] 0 2024-04-12T20:56:40.774 INFO:tasks.workunit.client.1.smithi186.stdout:0/843: getdents d0 0 2024-04-12T20:56:40.777 INFO:tasks.workunit.client.1.smithi186.stdout:8/741: mkdir d7/d18/def 0 2024-04-12T20:56:40.778 INFO:tasks.workunit.client.1.smithi186.stdout:1/757: chown d14/d1a/d33/d36/d5b/db9/fba 3 1 2024-04-12T20:56:40.779 INFO:tasks.workunit.client.1.smithi186.stdout:9/674: dwrite d3/d12/d6f/d83/f38 [4194304,4194304] 0 2024-04-12T20:56:40.779 INFO:tasks.workunit.client.1.smithi186.stdout:4/880: read d0/d21/f6a [1632390,93838] 0 2024-04-12T20:56:40.786 INFO:tasks.workunit.client.1.smithi186.stdout:2/835: truncate d0/fc5 187034 0 2024-04-12T20:56:40.787 INFO:tasks.workunit.client.1.smithi186.stdout:3/716: dread d1/d2c/d48/fab [0,4194304] 0 2024-04-12T20:56:40.789 INFO:tasks.workunit.client.1.smithi186.stdout:3/717: fdatasync d1/d3/d21/f65 0 2024-04-12T20:56:40.791 INFO:tasks.workunit.client.1.smithi186.stdout:7/724: dwrite d7/d10/f42 [0,4194304] 0 2024-04-12T20:56:40.791 INFO:tasks.workunit.client.1.smithi186.stdout:6/732: write d9/f10 [5824183,27171] 0 2024-04-12T20:56:40.795 INFO:tasks.workunit.client.1.smithi186.stdout:8/742: fsync d7/d18/d36/d1d/d1f/da2/d5c/f99 0 2024-04-12T20:56:40.797 INFO:tasks.workunit.client.1.smithi186.stdout:8/743: chown d7/d18/d36/d20/d23 136490 1 2024-04-12T20:56:40.799 INFO:tasks.workunit.client.1.smithi186.stdout:2/836: fsync d0/d32/d2a/f71 0 2024-04-12T20:56:40.804 INFO:tasks.workunit.client.1.smithi186.stdout:6/733: dread d9/da9/dbd/fd1 [0,4194304] 0 2024-04-12T20:56:40.810 INFO:tasks.workunit.client.1.smithi186.stdout:0/844: write d0/d5d/f6c [4561779,30860] 0 2024-04-12T20:56:40.811 INFO:tasks.workunit.client.1.smithi186.stdout:9/675: fsync d3/d2f/d54/f5b 0 2024-04-12T20:56:40.814 INFO:tasks.workunit.client.1.smithi186.stdout:7/725: symlink d7/d10/d2d/d40/d4a/l102 0 2024-04-12T20:56:40.816 INFO:tasks.workunit.client.1.smithi186.stdout:2/837: dwrite d0/dc/f4c [0,4194304] 0 2024-04-12T20:56:40.818 INFO:tasks.workunit.client.1.smithi186.stdout:4/881: creat d0/dc/d64/d87/d98/ddb/f11d x:0 0 0 2024-04-12T20:56:40.820 INFO:tasks.workunit.client.1.smithi186.stdout:4/882: chown d0/d59/dc0/d48/ca3 52317 1 2024-04-12T20:56:40.823 INFO:tasks.workunit.client.1.smithi186.stdout:6/734: rmdir d9/d43/d2d/d4d 39 2024-04-12T20:56:40.824 INFO:tasks.workunit.client.1.smithi186.stdout:5/749: link d5/db/d33/c73 d5/db/d15/d2a/d2d/d82/d84/cf6 0 2024-04-12T20:56:40.826 INFO:tasks.workunit.client.1.smithi186.stdout:1/758: creat d14/d1a/ff5 x:0 0 0 2024-04-12T20:56:40.829 INFO:tasks.workunit.client.1.smithi186.stdout:1/759: readlink d14/d6c/da9/led 0 2024-04-12T20:56:40.830 INFO:tasks.workunit.client.1.smithi186.stdout:9/676: dwrite d3/d2f/d2e/d43/d4c/fdc [0,4194304] 0 2024-04-12T20:56:40.832 INFO:tasks.workunit.client.1.smithi186.stdout:7/726: dwrite d7/d10/d1c/d5d/ff9 [0,4194304] 0 2024-04-12T20:56:40.835 INFO:tasks.workunit.client.1.smithi186.stdout:6/735: symlink d9/d4c/dbe/dfe/l106 0 2024-04-12T20:56:40.841 INFO:tasks.workunit.client.1.smithi186.stdout:2/838: mkdir d0/d3/d1a/d10b 0 2024-04-12T20:56:40.841 INFO:tasks.workunit.client.1.smithi186.stdout:5/750: creat d5/db/d15/d2a/ff7 x:0 0 0 2024-04-12T20:56:40.841 INFO:tasks.workunit.client.1.smithi186.stdout:9/677: symlink d3/d12/d6f/d83/led 0 2024-04-12T20:56:40.843 INFO:tasks.workunit.client.1.smithi186.stdout:5/751: chown d5/db/d2e/f64 353 1 2024-04-12T20:56:40.843 INFO:tasks.workunit.client.1.smithi186.stdout:9/678: chown d3/d12/d6f/le0 72 1 2024-04-12T20:56:40.846 INFO:tasks.workunit.client.1.smithi186.stdout:6/736: mknod d9/d5e/d73/c107 0 2024-04-12T20:56:40.848 INFO:tasks.workunit.client.1.smithi186.stdout:6/737: chown d9/d43/d2d/fa6 351850742 1 2024-04-12T20:56:40.849 INFO:tasks.workunit.client.1.smithi186.stdout:8/744: truncate d7/de/ddc/fe0 698414 0 2024-04-12T20:56:40.849 INFO:tasks.workunit.client.1.smithi186.stdout:0/845: creat d0/d85/f114 x:0 0 0 2024-04-12T20:56:40.851 INFO:tasks.workunit.client.1.smithi186.stdout:6/738: write d9/d4c/d6f/d97/fbc [559637,105715] 0 2024-04-12T20:56:40.852 INFO:tasks.workunit.client.1.smithi186.stdout:9/679: rename d3/d2f/d2e/d43/d4c/d82/l9b to d3/d2f/d2e/db3/lee 0 2024-04-12T20:56:40.853 INFO:tasks.workunit.client.1.smithi186.stdout:8/745: mkdir d7/de/d5e/df0 0 2024-04-12T20:56:40.859 INFO:tasks.workunit.client.1.smithi186.stdout:5/752: getdents d5/db 0 2024-04-12T20:56:40.859 INFO:tasks.workunit.client.1.smithi186.stdout:9/680: write d3/d2f/d2e/d33/f94 [121995,9218] 0 2024-04-12T20:56:40.860 INFO:tasks.workunit.client.1.smithi186.stdout:5/753: stat d5/c3e 0 2024-04-12T20:56:40.860 INFO:tasks.workunit.client.1.smithi186.stdout:8/746: mknod d7/d18/d36/d1d/cf1 0 2024-04-12T20:56:40.864 INFO:tasks.workunit.client.1.smithi186.stdout:5/754: truncate d5/db/d15/d2a/d2d/d46/d55/f37 2113656 0 2024-04-12T20:56:40.865 INFO:tasks.workunit.client.1.smithi186.stdout:8/747: symlink d7/d18/d36/d20/lf2 0 2024-04-12T20:56:40.866 INFO:tasks.workunit.client.1.smithi186.stdout:8/748: chown d7/de/c55 98108618 1 2024-04-12T20:56:40.868 INFO:tasks.workunit.client.1.smithi186.stdout:7/727: write d7/d10/d2d/d40/d4a/d6c/d97/d4d/d8a/dad/fb8 [352851,1600] 0 2024-04-12T20:56:40.870 INFO:tasks.workunit.client.1.smithi186.stdout:5/755: creat d5/db/ff8 x:0 0 0 2024-04-12T20:56:40.874 INFO:tasks.workunit.client.1.smithi186.stdout:9/681: dwrite d3/fce [0,4194304] 0 2024-04-12T20:56:40.909 INFO:tasks.workunit.client.1.smithi186.stdout:3/718: sync 2024-04-12T20:56:40.912 INFO:tasks.workunit.client.1.smithi186.stdout:0/846: write d0/d14/d4e/d1e/fa9 [326097,127767] 0 2024-04-12T20:56:40.912 INFO:tasks.workunit.client.1.smithi186.stdout:7/728: write d7/de1/f9d [512976,61510] 0 2024-04-12T20:56:40.913 INFO:tasks.workunit.client.1.smithi186.stdout:8/749: truncate d7/d18/f1a 8272281 0 2024-04-12T20:56:40.913 INFO:tasks.workunit.client.1.smithi186.stdout:4/883: dread d0/f93 [0,4194304] 0 2024-04-12T20:56:40.915 INFO:tasks.workunit.client.1.smithi186.stdout:3/719: dread - d1/d3/d21/d22/d4a/feb zero size 2024-04-12T20:56:40.917 INFO:tasks.workunit.client.1.smithi186.stdout:7/729: read - d7/d10/d2d/d40/d4a/d5c/fda zero size 2024-04-12T20:56:40.919 INFO:tasks.workunit.client.1.smithi186.stdout:8/750: mknod d7/d18/d36/d1d/d1f/cf3 0 2024-04-12T20:56:40.923 INFO:tasks.workunit.client.1.smithi186.stdout:4/884: link d0/f42 d0/dc/d64/d87/dd2/f11e 0 2024-04-12T20:56:40.932 INFO:tasks.workunit.client.1.smithi186.stdout:8/751: write d7/de/f11 [4883789,32897] 0 2024-04-12T20:56:40.932 INFO:tasks.workunit.client.1.smithi186.stdout:0/847: dwrite d0/d10/f8d [0,4194304] 0 2024-04-12T20:56:40.935 INFO:tasks.workunit.client.1.smithi186.stdout:8/752: chown d7/de/d68 4086505 1 2024-04-12T20:56:40.935 INFO:tasks.workunit.client.1.smithi186.stdout:3/720: dwrite d1/fc [4194304,4194304] 0 2024-04-12T20:56:40.936 INFO:tasks.workunit.client.1.smithi186.stdout:3/721: stat d1/d2c/d54/l96 0 2024-04-12T20:56:40.942 INFO:tasks.workunit.client.1.smithi186.stdout:4/885: dwrite d0/d59/ddf/f101 [0,4194304] 0 2024-04-12T20:56:40.942 INFO:tasks.workunit.client.1.smithi186.stdout:8/753: fdatasync d7/d18/d36/d20/d23/fe4 0 2024-04-12T20:56:40.949 INFO:tasks.workunit.client.1.smithi186.stdout:7/730: dread d7/d10/d2d/d40/d4a/d6c/d97/f92 [0,4194304] 0 2024-04-12T20:56:40.949 INFO:tasks.workunit.client.1.smithi186.stdout:8/754: creat d7/de/d10/d7f/ff4 x:0 0 0 2024-04-12T20:56:40.951 INFO:tasks.workunit.client.1.smithi186.stdout:8/755: rmdir d7/d18/d8e 39 2024-04-12T20:56:40.957 INFO:tasks.workunit.client.1.smithi186.stdout:8/756: read d7/d18/f85 [4150713,130293] 0 2024-04-12T20:56:40.962 INFO:tasks.workunit.client.1.smithi186.stdout:0/848: dread d0/f4f [0,4194304] 0 2024-04-12T20:56:40.965 INFO:tasks.workunit.client.1.smithi186.stdout:3/722: truncate d1/d2c/d48/fab 1328854 0 2024-04-12T20:56:40.971 INFO:tasks.workunit.client.1.smithi186.stdout:0/849: symlink d0/d10/dda/l115 0 2024-04-12T20:56:40.971 INFO:tasks.workunit.client.1.smithi186.stdout:3/723: mknod d1/d3/d21/d22/d34/ddb/d4d/dad/cf0 0 2024-04-12T20:56:40.973 INFO:tasks.workunit.client.1.smithi186.stdout:1/760: sync 2024-04-12T20:56:40.974 INFO:tasks.workunit.client.1.smithi186.stdout:7/731: dwrite d7/d10/d2d/d40/d4a/d6c/d97/d4d/d8a/dad/fb8 [0,4194304] 0 2024-04-12T20:56:40.978 INFO:tasks.workunit.client.1.smithi186.stdout:3/724: write d1/d3/d21/fb2 [4649634,68960] 0 2024-04-12T20:56:40.978 INFO:tasks.workunit.client.1.smithi186.stdout:1/761: write d14/d6c/fb5 [295209,44898] 0 2024-04-12T20:56:40.980 INFO:tasks.workunit.client.1.smithi186.stdout:4/886: dwrite d0/d21/f41 [0,4194304] 0 2024-04-12T20:56:40.981 INFO:tasks.workunit.client.1.smithi186.stdout:0/850: getdents d0/d10/d72 0 2024-04-12T20:56:40.984 INFO:tasks.workunit.client.1.smithi186.stdout:3/725: chown d1/d3/da/f6c 7039988 1 2024-04-12T20:56:40.988 INFO:tasks.workunit.client.1.smithi186.stdout:8/757: dwrite d7/d18/d36/d20/d71/feb [0,4194304] 0 2024-04-12T20:56:40.990 INFO:tasks.workunit.client.1.smithi186.stdout:7/732: rename d7/d55/f59 to d7/d55/d71/dbe/f103 0 2024-04-12T20:56:40.991 INFO:tasks.workunit.client.1.smithi186.stdout:1/762: write d14/d1a/d33/d36/d5b/fa3 [1415948,84838] 0 2024-04-12T20:56:40.991 INFO:tasks.workunit.client.1.smithi186.stdout:0/851: creat d0/d10/d71/f116 x:0 0 0 2024-04-12T20:56:40.994 INFO:tasks.workunit.client.1.smithi186.stdout:1/763: stat d14/d1a/d21/d35/dbc/cd0 0 2024-04-12T20:56:40.998 INFO:tasks.workunit.client.1.smithi186.stdout:0/852: creat d0/d5c/d65/f117 x:0 0 0 2024-04-12T20:56:41.000 INFO:tasks.workunit.client.1.smithi186.stdout:1/764: creat d14/d1a/d21/d35/ff6 x:0 0 0 2024-04-12T20:56:41.002 INFO:tasks.workunit.client.1.smithi186.stdout:7/733: truncate d7/d10/d2d/d40/d4a/d6c/fbc 1563282 0 2024-04-12T20:56:41.005 INFO:tasks.workunit.client.1.smithi186.stdout:0/853: creat d0/d5d/f118 x:0 0 0 2024-04-12T20:56:41.006 INFO:tasks.workunit.client.1.smithi186.stdout:4/887: write d0/db4/f106 [1301357,87357] 0 2024-04-12T20:56:41.007 INFO:tasks.workunit.client.1.smithi186.stdout:9/682: sync 2024-04-12T20:56:41.007 INFO:tasks.workunit.client.1.smithi186.stdout:2/839: sync 2024-04-12T20:56:41.007 INFO:tasks.workunit.client.1.smithi186.stdout:6/739: sync 2024-04-12T20:56:41.007 INFO:tasks.workunit.client.1.smithi186.stdout:5/756: sync 2024-04-12T20:56:41.009 INFO:tasks.workunit.client.1.smithi186.stdout:8/758: write d7/de/d7a/f4f [16688,101693] 0 2024-04-12T20:56:41.010 INFO:tasks.workunit.client.1.smithi186.stdout:7/734: rename d7/d10/d2d/d40/d4a/d5c/f9a to d7/d55/d71/f104 0 2024-04-12T20:56:41.015 INFO:tasks.workunit.client.1.smithi186.stdout:4/888: readlink d0/d59/dc0/d63/da6/dc6/l99 0 2024-04-12T20:56:41.016 INFO:tasks.workunit.client.1.smithi186.stdout:6/740: write d9/d43/d59/d5d/d6e/d91/dd9/ff8 [192448,24549] 0 2024-04-12T20:56:41.017 INFO:tasks.workunit.client.1.smithi186.stdout:4/889: chown d0/d21/db9 312610 1 2024-04-12T20:56:41.018 INFO:tasks.workunit.client.1.smithi186.stdout:9/683: creat d3/d2f/d54/d92/fef x:0 0 0 2024-04-12T20:56:41.019 INFO:tasks.workunit.client.1.smithi186.stdout:6/741: chown d9/f12 6 1 2024-04-12T20:56:41.022 INFO:tasks.workunit.client.1.smithi186.stdout:2/840: symlink d0/l10c 0 2024-04-12T20:56:41.022 INFO:tasks.workunit.client.1.smithi186.stdout:7/735: chown d7/d10/d1c/f8e 49328357 1 2024-04-12T20:56:41.024 INFO:tasks.workunit.client.1.smithi186.stdout:8/759: mkdir d7/d18/d36/d20/d23/dd7/df5 0 2024-04-12T20:56:41.029 INFO:tasks.workunit.client.1.smithi186.stdout:6/742: symlink d9/d5e/d7a/l108 0 2024-04-12T20:56:41.029 INFO:tasks.workunit.client.1.smithi186.stdout:5/757: symlink d5/db/d15/d2a/d4c/d5b/d7a/d94/lf9 0 2024-04-12T20:56:41.031 INFO:tasks.workunit.client.1.smithi186.stdout:7/736: write d7/d10/d2d/d40/d4a/d6c/d97/f78 [5056747,8143] 0 2024-04-12T20:56:41.031 INFO:tasks.workunit.client.1.smithi186.stdout:9/684: dread d3/d2f/d54/d92/f97 [0,4194304] 0 2024-04-12T20:56:41.032 INFO:tasks.workunit.client.1.smithi186.stdout:1/765: dwrite d14/d74/f9a [0,4194304] 0 2024-04-12T20:56:41.033 INFO:tasks.workunit.client.1.smithi186.stdout:8/760: write d7/d18/d36/fa [55978,20263] 0 2024-04-12T20:56:41.033 INFO:tasks.workunit.client.1.smithi186.stdout:2/841: rename d0/d3/le7 to d0/d3/d1a/d21/d26/l10d 0 2024-04-12T20:56:41.039 INFO:tasks.workunit.client.1.smithi186.stdout:2/842: fsync d0/f50 0 2024-04-12T20:56:41.044 INFO:tasks.workunit.client.1.smithi186.stdout:3/726: dread d1/d2c/d48/f4f [4194304,4194304] 0 2024-04-12T20:56:41.045 INFO:tasks.workunit.client.1.smithi186.stdout:0/854: dwrite d0/d5c/d65/fbb [0,4194304] 0 2024-04-12T20:56:41.048 INFO:tasks.workunit.client.1.smithi186.stdout:3/727: dread - d1/d3/d21/d22/da4/fe0 zero size 2024-04-12T20:56:41.050 INFO:tasks.workunit.client.1.smithi186.stdout:5/758: rename d5/db/d15/d2a/d2d/d48 to d5/db/d15/d2a/d4c/d5b/d7a/d94/dfa 0 2024-04-12T20:56:41.051 INFO:tasks.workunit.client.1.smithi186.stdout:6/743: dread d9/d43/d2d/f7e [0,4194304] 0 2024-04-12T20:56:41.056 INFO:tasks.workunit.client.1.smithi186.stdout:9/685: truncate d3/d2f/d2e/d33/d36/f3d 4684065 0 2024-04-12T20:56:41.056 INFO:tasks.workunit.client.1.smithi186.stdout:2/843: fsync d0/d3/d1a/d24/fb9 0 2024-04-12T20:56:41.057 INFO:tasks.workunit.client.1.smithi186.stdout:4/890: dwrite d0/d59/dc0/d63/da6/dc6/f66 [0,4194304] 0 2024-04-12T20:56:41.059 INFO:tasks.workunit.client.1.smithi186.stdout:0/855: chown d0/d14/d4e/d50/d6e/d96/ca0 36002 1 2024-04-12T20:56:41.061 INFO:tasks.workunit.client.1.smithi186.stdout:3/728: mknod d1/d2c/d54/d7c/cf1 0 2024-04-12T20:56:41.064 INFO:tasks.workunit.client.1.smithi186.stdout:8/761: truncate d7/d18/d36/d1d/d1f/da2/d5c/f7b 2582290 0 2024-04-12T20:56:41.066 INFO:tasks.workunit.client.1.smithi186.stdout:0/856: fsync d0/d10/d32/dd8/fe1 0 2024-04-12T20:56:41.066 INFO:tasks.workunit.client.1.smithi186.stdout:5/759: rmdir d5/db/d15/d2a/d6d/d7f/d80 39 2024-04-12T20:56:41.069 INFO:tasks.workunit.client.1.smithi186.stdout:0/857: truncate d0/d85/f114 334452 0 2024-04-12T20:56:41.071 INFO:tasks.workunit.client.1.smithi186.stdout:9/686: mknod d3/d2f/d54/d9f/cf0 0 2024-04-12T20:56:41.074 INFO:tasks.workunit.client.1.smithi186.stdout:6/744: rename d9/l2c to d9/d43/d2d/d4d/l109 0 2024-04-12T20:56:41.078 INFO:tasks.workunit.client.1.smithi186.stdout:0/858: dread - d0/d14/d4e/d50/d6e/faf zero size 2024-04-12T20:56:41.079 INFO:tasks.workunit.client.1.smithi186.stdout:5/760: mknod d5/db/d15/de8/cfb 0 2024-04-12T20:56:41.082 INFO:tasks.workunit.client.1.smithi186.stdout:7/737: write d7/d10/d2d/d40/d4a/d5c/fda [404925,94589] 0 2024-04-12T20:56:41.082 INFO:tasks.workunit.client.1.smithi186.stdout:9/687: rmdir d3/d2f/d2e/d43/d4c 39 2024-04-12T20:56:41.082 INFO:tasks.workunit.client.1.smithi186.stdout:1/766: dwrite d14/d1a/d1d/f94 [0,4194304] 0 2024-04-12T20:56:41.085 INFO:tasks.workunit.client.1.smithi186.stdout:0/859: chown d0/d5c/d65/cc9 178 1 2024-04-12T20:56:41.088 INFO:tasks.workunit.client.1.smithi186.stdout:4/891: write d0/dc/d88/d80/fa5 [1299244,4697] 0 2024-04-12T20:56:41.090 INFO:tasks.workunit.client.1.smithi186.stdout:1/767: write d14/d6c/feb [48950,53513] 0 2024-04-12T20:56:41.095 INFO:tasks.workunit.client.1.smithi186.stdout:8/762: truncate d7/d18/d36/d20/d23/d25/f46 842523 0 2024-04-12T20:56:41.095 INFO:tasks.workunit.client.1.smithi186.stdout:7/738: unlink d7/fa1 0 2024-04-12T20:56:41.096 INFO:tasks.workunit.client.1.smithi186.stdout:9/688: dread d3/d13/f18 [0,4194304] 0 2024-04-12T20:56:41.096 INFO:tasks.workunit.client.1.smithi186.stdout:6/745: getdents d9/da9/dbd/d100 0 2024-04-12T20:56:41.097 INFO:tasks.workunit.client.1.smithi186.stdout:0/860: fdatasync d0/d5c/d66/da7/fb4 0 2024-04-12T20:56:41.097 INFO:tasks.workunit.client.1.smithi186.stdout:3/729: write d1/d3/da2/f9d [3771238,97838] 0 2024-04-12T20:56:41.101 INFO:tasks.workunit.client.1.smithi186.stdout:1/768: write d14/d1a/d33/fd3 [3956695,64095] 0 2024-04-12T20:56:41.103 INFO:tasks.workunit.client.1.smithi186.stdout:5/761: creat d5/db/d15/d2a/d6d/d7f/ffc x:0 0 0 2024-04-12T20:56:41.105 INFO:tasks.workunit.client.1.smithi186.stdout:0/861: rmdir d0/d14/d105 39 2024-04-12T20:56:41.110 INFO:tasks.workunit.client.1.smithi186.stdout:8/763: mknod d7/d18/d36/d1d/d7d/da5/cf6 0 2024-04-12T20:56:41.110 INFO:tasks.workunit.client.1.smithi186.stdout:9/689: dread - d3/d2f/d54/d9f/dc4/fc1 zero size 2024-04-12T20:56:41.111 INFO:tasks.workunit.client.1.smithi186.stdout:9/690: chown d3/d12 24 1 2024-04-12T20:56:41.112 INFO:tasks.workunit.client.1.smithi186.stdout:4/892: link d0/d59/dc0/l7b d0/d59/dc0/d63/da6/l11f 0 2024-04-12T20:56:41.113 INFO:tasks.workunit.client.1.smithi186.stdout:1/769: rename d14/d1a/d1d/l24 to d14/d1a/d21/d35/dbc/de3/de4/lf7 0 2024-04-12T20:56:41.123 INFO:tasks.workunit.client.1.smithi186.stdout:7/739: link d7/l9b d7/d10/d2d/d40/d4a/d6c/l105 0 2024-04-12T20:56:41.126 INFO:tasks.workunit.client.1.smithi186.stdout:8/764: mkdir d7/d18/d36/d20/d23/dd6/de8/df7 0 2024-04-12T20:56:41.127 INFO:tasks.workunit.client.1.smithi186.stdout:9/691: symlink d3/d2f/d54/d9f/dc4/d77/dc3/daf/lf1 0 2024-04-12T20:56:41.129 INFO:tasks.workunit.client.1.smithi186.stdout:7/740: stat d7/d10/d1c/f20 0 2024-04-12T20:56:41.129 INFO:tasks.workunit.client.1.smithi186.stdout:1/770: mknod d14/d1a/d21/d35/dbc/dd5/cf8 0 2024-04-12T20:56:41.132 INFO:tasks.workunit.client.1.smithi186.stdout:2/844: dread d0/d3/d1a/d24/f34 [0,4194304] 0 2024-04-12T20:56:41.137 INFO:tasks.workunit.client.1.smithi186.stdout:6/746: dwrite d9/d43/d2d/d4d/fe0 [0,4194304] 0 2024-04-12T20:56:41.140 INFO:tasks.workunit.client.1.smithi186.stdout:0/862: write d0/d10/d32/ff1 [1663177,64924] 0 2024-04-12T20:56:41.140 INFO:tasks.workunit.client.1.smithi186.stdout:8/765: truncate d7/de/d10/f19 279336 0 2024-04-12T20:56:41.143 INFO:tasks.workunit.client.1.smithi186.stdout:8/766: fsync d7/de/f11 0 2024-04-12T20:56:41.146 INFO:tasks.workunit.client.1.smithi186.stdout:2/845: rename d0/d3/d1a/d21/d22/cf3 to d0/d3/d1a/d10b/c10e 0 2024-04-12T20:56:41.153 INFO:tasks.workunit.client.1.smithi186.stdout:8/767: fdatasync d7/d18/d36/d20/d23/fe4 0 2024-04-12T20:56:41.155 INFO:tasks.workunit.client.1.smithi186.stdout:5/762: dwrite d5/db/d15/d2a/d4c/d5b/d7a/d94/dfa/fb5 [4194304,4194304] 0 2024-04-12T20:56:41.159 INFO:tasks.workunit.client.1.smithi186.stdout:0/863: mkdir d0/d14/d4e/d50/d6e/d119 0 2024-04-12T20:56:41.166 INFO:tasks.workunit.client.1.smithi186.stdout:1/771: truncate d14/d74/f9a 2807567 0 2024-04-12T20:56:41.166 INFO:tasks.workunit.client.1.smithi186.stdout:0/864: stat d0/d5c/d66/da7/lb2 0 2024-04-12T20:56:41.171 INFO:tasks.workunit.client.1.smithi186.stdout:5/763: unlink d5/db/d15/d2a/d2d/c32 0 2024-04-12T20:56:41.176 INFO:tasks.workunit.client.1.smithi186.stdout:1/772: fsync d14/d1a/d1d/db8/ff3 0 2024-04-12T20:56:41.179 INFO:tasks.workunit.client.1.smithi186.stdout:8/768: truncate d7/de/d10/d7f/dd3/fd4 642628 0 2024-04-12T20:56:41.181 INFO:tasks.workunit.client.1.smithi186.stdout:9/692: dwrite d3/d2f/d2e/d33/f44 [0,4194304] 0 2024-04-12T20:56:41.185 INFO:tasks.workunit.client.1.smithi186.stdout:0/865: creat d0/d14/d4e/d50/d6e/d119/f11a x:0 0 0 2024-04-12T20:56:41.185 INFO:tasks.workunit.client.1.smithi186.stdout:2/846: dwrite d0/dc/d2e/d73/fd9 [0,4194304] 0 2024-04-12T20:56:41.190 INFO:tasks.workunit.client.1.smithi186.stdout:5/764: rmdir d5/db/d15/d2a/d2d 39 2024-04-12T20:56:41.191 INFO:tasks.workunit.client.1.smithi186.stdout:3/730: sync 2024-04-12T20:56:41.191 INFO:tasks.workunit.client.1.smithi186.stdout:4/893: sync 2024-04-12T20:56:41.193 INFO:tasks.workunit.client.1.smithi186.stdout:1/773: creat d14/d1a/d1d/de5/ff9 x:0 0 0 2024-04-12T20:56:41.195 INFO:tasks.workunit.client.1.smithi186.stdout:7/741: dwrite d7/d10/d2d/d40/d4a/d6c/d97/fba [0,4194304] 0 2024-04-12T20:56:41.195 INFO:tasks.workunit.client.1.smithi186.stdout:2/847: fsync d0/d3/d7a/f94 0 2024-04-12T20:56:41.201 INFO:tasks.workunit.client.1.smithi186.stdout:5/765: stat d5/db/f57 0 2024-04-12T20:56:41.201 INFO:tasks.workunit.client.1.smithi186.stdout:6/747: dread d9/da9/fee [0,4194304] 0 2024-04-12T20:56:41.206 INFO:tasks.workunit.client.1.smithi186.stdout:4/894: read - d0/dc/d1f/d73/ffd zero size 2024-04-12T20:56:41.207 INFO:tasks.workunit.client.1.smithi186.stdout:3/731: symlink d1/d3/d21/d22/d4a/dbc/lf2 0 2024-04-12T20:56:41.207 INFO:tasks.workunit.client.1.smithi186.stdout:2/848: dread - d0/d3/f104 zero size 2024-04-12T20:56:41.207 INFO:tasks.workunit.client.1.smithi186.stdout:1/774: mkdir d14/d1a/d33/d36/dfa 0 2024-04-12T20:56:41.211 INFO:tasks.workunit.client.1.smithi186.stdout:4/895: stat d0/dc/d33/f17 0 2024-04-12T20:56:41.214 INFO:tasks.workunit.client.1.smithi186.stdout:2/849: unlink d0/d32/fe5 0 2024-04-12T20:56:41.214 INFO:tasks.workunit.client.1.smithi186.stdout:5/766: write d5/db/d15/d2a/d4c/d93/fcf [816525,18415] 0 2024-04-12T20:56:41.215 INFO:tasks.workunit.client.1.smithi186.stdout:1/775: write d14/d91/fc1 [191182,63352] 0 2024-04-12T20:56:41.215 INFO:tasks.workunit.client.1.smithi186.stdout:5/767: chown d5 33306 1 2024-04-12T20:56:41.217 INFO:tasks.workunit.client.1.smithi186.stdout:9/693: dwrite d3/d2f/d2e/fd8 [0,4194304] 0 2024-04-12T20:56:41.218 INFO:tasks.workunit.client.1.smithi186.stdout:8/769: rename d7/d18/d36/d20/d23/d25 to d7/d18/d36/d1d/d1f/da2/df8 0 2024-04-12T20:56:41.221 INFO:tasks.workunit.client.1.smithi186.stdout:7/742: mkdir d7/d10/d2d/d40/d4a/d6c/d97/d106 0 2024-04-12T20:56:41.222 INFO:tasks.workunit.client.1.smithi186.stdout:1/776: dread - d14/d1a/d1d/de5/ff9 zero size 2024-04-12T20:56:41.224 INFO:tasks.workunit.client.1.smithi186.stdout:3/732: fdatasync d1/d3/da/f56 0 2024-04-12T20:56:41.228 INFO:tasks.workunit.client.1.smithi186.stdout:3/733: chown d1/d3/d21/d22/d24 197375 1 2024-04-12T20:56:41.228 INFO:tasks.workunit.client.1.smithi186.stdout:0/866: dwrite d0/d81/f83 [0,4194304] 0 2024-04-12T20:56:41.228 INFO:tasks.workunit.client.1.smithi186.stdout:2/850: symlink d0/d3/d1a/d41/d8f/l10f 0 2024-04-12T20:56:41.229 INFO:tasks.workunit.client.1.smithi186.stdout:6/748: truncate d9/d43/d59/d5d/f46 6555641 0 2024-04-12T20:56:41.231 INFO:tasks.workunit.client.1.smithi186.stdout:8/770: creat d7/d18/d36/d20/d23/ff9 x:0 0 0 2024-04-12T20:56:41.232 INFO:tasks.workunit.client.1.smithi186.stdout:6/749: write d9/d4c/d6f/d89/fde [527686,112044] 0 2024-04-12T20:56:41.233 INFO:tasks.workunit.client.1.smithi186.stdout:8/771: stat d7/d18/d36/d1d/d1f/da2/f79 0 2024-04-12T20:56:41.235 INFO:tasks.workunit.client.1.smithi186.stdout:4/896: rename d0/d59/dc0/d48/dce to d0/dc/d64/d87/d120 0 2024-04-12T20:56:41.239 INFO:tasks.workunit.client.1.smithi186.stdout:7/743: unlink d7/d55/cac 0 2024-04-12T20:56:41.241 INFO:tasks.workunit.client.1.smithi186.stdout:9/694: creat d3/d2f/d54/d9f/dc4/ff2 x:0 0 0 2024-04-12T20:56:41.241 INFO:tasks.workunit.client.1.smithi186.stdout:7/744: chown d7/d10/d1c/f20 653133 1 2024-04-12T20:56:41.243 INFO:tasks.workunit.client.1.smithi186.stdout:1/777: creat d14/d1a/d33/d36/ffb x:0 0 0 2024-04-12T20:56:41.247 INFO:tasks.workunit.client.1.smithi186.stdout:1/778: truncate d14/d1a/d1d/de5/ff9 201787 0 2024-04-12T20:56:41.249 INFO:tasks.workunit.client.1.smithi186.stdout:8/772: dwrite d7/de/d68/f9d [4194304,4194304] 0 2024-04-12T20:56:41.251 INFO:tasks.workunit.client.1.smithi186.stdout:6/750: chown d9/d43/d50/cf2 74280 1 2024-04-12T20:56:41.252 INFO:tasks.workunit.client.1.smithi186.stdout:5/768: write d5/db/d15/d2a/d4c/f7b [609088,90273] 0 2024-04-12T20:56:41.258 INFO:tasks.workunit.client.1.smithi186.stdout:0/867: symlink d0/d85/dc4/l11b 0 2024-04-12T20:56:41.259 INFO:tasks.workunit.client.1.smithi186.stdout:8/773: write d7/d18/d36/d1d/d1f/d26/fb8 [6087844,10254] 0 2024-04-12T20:56:41.261 INFO:tasks.workunit.client.1.smithi186.stdout:2/851: link d0/d3/d1a/d48/df7/fbd d0/d3/d1a/d27/f110 0 2024-04-12T20:56:41.261 INFO:tasks.workunit.client.1.smithi186.stdout:1/779: rmdir d14/d1a/d1d/db8 39 2024-04-12T20:56:41.263 INFO:tasks.workunit.client.1.smithi186.stdout:5/769: write d5/db/d15/d2a/d4c/f7b [1674534,79315] 0 2024-04-12T20:56:41.267 INFO:tasks.workunit.client.1.smithi186.stdout:7/745: dwrite d7/d10/f42 [4194304,4194304] 0 2024-04-12T20:56:41.270 INFO:tasks.workunit.client.1.smithi186.stdout:1/780: mkdir d14/d6c/dfc 0 2024-04-12T20:56:41.270 INFO:tasks.workunit.client.1.smithi186.stdout:3/734: dwrite d1/d3/d21/d22/d34/ddb/fc5 [0,4194304] 0 2024-04-12T20:56:41.274 INFO:tasks.workunit.client.1.smithi186.stdout:8/774: creat d7/de/d68/da7/dd5/ffa x:0 0 0 2024-04-12T20:56:41.278 INFO:tasks.workunit.client.1.smithi186.stdout:4/897: getdents d0/dc/d64/d87 0 2024-04-12T20:56:41.278 INFO:tasks.workunit.client.1.smithi186.stdout:2/852: mkdir d0/d3/d1a/d48/df7/d111 0 2024-04-12T20:56:41.281 INFO:tasks.workunit.client.1.smithi186.stdout:8/775: write d7/de/d7a/fd9 [329726,69377] 0 2024-04-12T20:56:41.284 INFO:tasks.workunit.client.1.smithi186.stdout:1/781: chown c10 0 1 2024-04-12T20:56:41.291 INFO:tasks.workunit.client.1.smithi186.stdout:3/735: rename d1/d3/d21/d22/d34/fd6 to d1/d3/d21/d22/d24/d2f/d95/ff3 0 2024-04-12T20:56:41.292 INFO:tasks.workunit.client.1.smithi186.stdout:3/736: chown d1/d3/da/l2b 12 1 2024-04-12T20:56:41.293 INFO:tasks.workunit.client.1.smithi186.stdout:9/695: dread d3/d2f/d2e/d43/d4c/f58 [4194304,4194304] 0 2024-04-12T20:56:41.294 INFO:tasks.workunit.client.1.smithi186.stdout:2/853: mknod d0/d32/d69/c112 0 2024-04-12T20:56:41.295 INFO:tasks.workunit.client.1.smithi186.stdout:6/751: link d9/d43/d2d/d4d/lbb d9/d43/l10a 0 2024-04-12T20:56:41.297 INFO:tasks.workunit.client.1.smithi186.stdout:5/770: chown d5/db/d15/d2a/d2d/d46/d55/c3f 125587 1 2024-04-12T20:56:41.299 INFO:tasks.workunit.client.1.smithi186.stdout:8/776: mkdir d7/de/d10/d7f/dfb 0 2024-04-12T20:56:41.299 INFO:tasks.workunit.client.1.smithi186.stdout:0/868: write d0/d5c/d65/f7b [2816555,45534] 0 2024-04-12T20:56:41.300 INFO:tasks.workunit.client.1.smithi186.stdout:1/782: creat d14/d1a/d33/d36/d5b/d3e/d71/ffd x:0 0 0 2024-04-12T20:56:41.301 INFO:tasks.workunit.client.1.smithi186.stdout:0/869: fdatasync d0/d10/f43 0 2024-04-12T20:56:41.303 INFO:tasks.workunit.client.1.smithi186.stdout:4/898: mkdir d0/d26/d121 0 2024-04-12T20:56:41.304 INFO:tasks.workunit.client.1.smithi186.stdout:8/777: write d7/d18/d36/d1d/d1f/da2/df8/d29/f6a [2569686,17629] 0 2024-04-12T20:56:41.304 INFO:tasks.workunit.client.1.smithi186.stdout:3/737: mknod d1/d2c/d54/d7c/cf4 0 2024-04-12T20:56:41.305 INFO:tasks.workunit.client.1.smithi186.stdout:7/746: write d7/d10/d1c/d8c/fb5 [210498,63261] 0 2024-04-12T20:56:41.306 INFO:tasks.workunit.client.1.smithi186.stdout:2/854: creat d0/d3e/f113 x:0 0 0 2024-04-12T20:56:41.309 INFO:tasks.workunit.client.1.smithi186.stdout:8/778: chown d7/de/d5e/c61 398 1 2024-04-12T20:56:41.314 INFO:tasks.workunit.client.1.smithi186.stdout:2/855: write d0/d3/d1a/d48/df7/fa4 [216901,51005] 0 2024-04-12T20:56:41.314 INFO:tasks.workunit.client.1.smithi186.stdout:9/696: dwrite d3/d2f/d2e/d43/d4c/d82/dd4/fe4 [0,4194304] 0 2024-04-12T20:56:41.314 INFO:tasks.workunit.client.1.smithi186.stdout:1/783: mknod d14/d1a/d33/d36/d5b/d6a/cfe 0 2024-04-12T20:56:41.315 INFO:tasks.workunit.client.1.smithi186.stdout:5/771: creat d5/db/d15/d2a/d6d/d7f/d86/dca/ffd x:0 0 0 2024-04-12T20:56:41.317 INFO:tasks.workunit.client.1.smithi186.stdout:5/772: readlink d5/db/d15/d2a/d6d/dbb/dc4/lba 0 2024-04-12T20:56:41.319 INFO:tasks.workunit.client.1.smithi186.stdout:3/738: creat d1/d3/d21/d22/d34/ddb/d4d/d6f/ff5 x:0 0 0 2024-04-12T20:56:41.321 INFO:tasks.workunit.client.1.smithi186.stdout:5/773: dread - d5/db/d15/d2a/d4c/d5b/d7a/d94/dfa/da4/fa7 zero size 2024-04-12T20:56:41.322 INFO:tasks.workunit.client.1.smithi186.stdout:0/870: symlink d0/d14/d105/l11c 0 2024-04-12T20:56:41.325 INFO:tasks.workunit.client.1.smithi186.stdout:7/747: creat d7/d10/d2d/d40/d4a/d6c/d97/d4d/d6a/dcc/f107 x:0 0 0 2024-04-12T20:56:41.329 INFO:tasks.workunit.client.1.smithi186.stdout:1/784: creat d14/d1a/d33/d36/d5b/d3e/d8b/fff x:0 0 0 2024-04-12T20:56:41.330 INFO:tasks.workunit.client.1.smithi186.stdout:9/697: mknod d3/d2f/d54/d9f/dc4/d77/dd6/d72/cf3 0 2024-04-12T20:56:41.331 INFO:tasks.workunit.client.1.smithi186.stdout:4/899: creat d0/dc/df2/d118/f122 x:0 0 0 2024-04-12T20:56:41.334 INFO:tasks.workunit.client.1.smithi186.stdout:3/739: symlink d1/d3/d21/d22/d34/ddb/d4d/d9c/lf6 0 2024-04-12T20:56:41.335 INFO:tasks.workunit.client.1.smithi186.stdout:6/752: truncate d9/d43/d59/d5d/f48 539187 0 2024-04-12T20:56:41.337 INFO:tasks.workunit.client.1.smithi186.stdout:7/748: write d7/d10/d2d/d67/fd9 [802276,65217] 0 2024-04-12T20:56:41.338 INFO:tasks.workunit.client.1.smithi186.stdout:1/785: unlink d14/d1a/d33/d36/d5b/d6a/c83 0 2024-04-12T20:56:41.342 INFO:tasks.workunit.client.1.smithi186.stdout:3/740: rmdir d1/d3/d21/d22/d24/d2f 39 2024-04-12T20:56:41.347 INFO:tasks.workunit.client.1.smithi186.stdout:8/779: link d7/d18/c58 d7/d18/d36/d1d/d1f/d26/dab/cfc 0 2024-04-12T20:56:41.350 INFO:tasks.workunit.client.1.smithi186.stdout:9/698: write d3/d2f/d2e/d33/f49 [297607,67667] 0 2024-04-12T20:56:41.354 INFO:tasks.workunit.client.1.smithi186.stdout:2/856: link d0/d32/d2a/f62 d0/d3/d1a/d41/d8f/f114 0 2024-04-12T20:56:41.355 INFO:tasks.workunit.client.1.smithi186.stdout:7/749: mkdir d7/d10/d1c/d108 0 2024-04-12T20:56:41.356 INFO:tasks.workunit.client.1.smithi186.stdout:8/780: mkdir d7/d18/d36/d20/d23/dfd 0 2024-04-12T20:56:41.357 INFO:tasks.workunit.client.1.smithi186.stdout:4/900: creat d0/dc/d88/d80/f123 x:0 0 0 2024-04-12T20:56:41.358 INFO:tasks.workunit.client.1.smithi186.stdout:0/871: getdents d0/d14/d105 0 2024-04-12T20:56:41.359 INFO:tasks.workunit.client.1.smithi186.stdout:0/872: fdatasync d0/d10/d32/dac/fef 0 2024-04-12T20:56:41.360 INFO:tasks.workunit.client.1.smithi186.stdout:9/699: rename d3/d12 to d3/d2f/d54/d9f/dc4/d77/dd6/d72/df4 0 2024-04-12T20:56:41.361 INFO:tasks.workunit.client.1.smithi186.stdout:0/873: write d0/d10/d72/fed [965311,30104] 0 2024-04-12T20:56:41.362 INFO:tasks.workunit.client.1.smithi186.stdout:6/753: creat d9/d5e/d7a/f10b x:0 0 0 2024-04-12T20:56:41.363 INFO:tasks.workunit.client.1.smithi186.stdout:0/874: readlink d0/l2e 0 2024-04-12T20:56:41.364 INFO:tasks.workunit.client.1.smithi186.stdout:2/857: mkdir d0/d3/d1a/d48/d115 0 2024-04-12T20:56:41.365 INFO:tasks.workunit.client.1.smithi186.stdout:1/786: link d14/d1a/d3c/c50 d14/d1a/d33/d36/d5b/d3e/d5f/c100 0 2024-04-12T20:56:41.367 INFO:tasks.workunit.client.1.smithi186.stdout:8/781: write d7/de/d10/d7f/f9f [1205111,98588] 0 2024-04-12T20:56:41.371 INFO:tasks.workunit.client.1.smithi186.stdout:0/875: mkdir d0/d5c/d66/da7/dfc/d11d 0 2024-04-12T20:56:41.373 INFO:tasks.workunit.client.1.smithi186.stdout:7/750: mkdir d7/d10/d2d/d40/d4a/d6c/d97/d109 0 2024-04-12T20:56:41.374 INFO:tasks.workunit.client.1.smithi186.stdout:7/751: stat d7/d10/d2d/d40/l54 0 2024-04-12T20:56:41.375 INFO:tasks.workunit.client.1.smithi186.stdout:7/752: fdatasync d7/f17 0 2024-04-12T20:56:41.377 INFO:tasks.workunit.client.1.smithi186.stdout:3/741: getdents d1/d2c 0 2024-04-12T20:56:41.379 INFO:tasks.workunit.client.1.smithi186.stdout:8/782: mknod d7/d18/d36/d20/d23/dd6/cfe 0 2024-04-12T20:56:41.384 INFO:tasks.workunit.client.1.smithi186.stdout:8/783: write d7/de/d10/d7f/fa6 [879567,83863] 0 2024-04-12T20:56:41.384 INFO:tasks.workunit.client.1.smithi186.stdout:4/901: dread d0/d59/dc0/f84 [0,4194304] 0 2024-04-12T20:56:41.387 INFO:tasks.workunit.client.1.smithi186.stdout:9/700: dwrite d3/d2f/d2e/d43/d4c/fb0 [0,4194304] 0 2024-04-12T20:56:41.394 INFO:tasks.workunit.client.1.smithi186.stdout:3/742: rename d1/d2c/d54/d81 to d1/d3/d21/d22/d34/df7 0 2024-04-12T20:56:41.396 INFO:tasks.workunit.client.1.smithi186.stdout:1/787: write d14/d1a/d58/fb7 [2144538,81346] 0 2024-04-12T20:56:41.399 INFO:tasks.workunit.client.1.smithi186.stdout:2/858: creat d0/d3/d1a/d21/f116 x:0 0 0 2024-04-12T20:56:41.399 INFO:tasks.workunit.client.1.smithi186.stdout:4/902: creat d0/d59/ddf/f124 x:0 0 0 2024-04-12T20:56:41.403 INFO:tasks.workunit.client.1.smithi186.stdout:8/784: creat d7/d18/d36/d20/d23/dfd/fff x:0 0 0 2024-04-12T20:56:41.408 INFO:tasks.workunit.client.1.smithi186.stdout:1/788: rmdir d14/d6c/dfc 0 2024-04-12T20:56:41.409 INFO:tasks.workunit.client.1.smithi186.stdout:8/785: chown d7/d18/d36/d20/d23/c8d 0 1 2024-04-12T20:56:41.415 INFO:tasks.workunit.client.1.smithi186.stdout:8/786: stat d7/d18/d36/d20/d23/ff9 0 2024-04-12T20:56:41.415 INFO:tasks.workunit.client.1.smithi186.stdout:7/753: dwrite d7/d10/d2d/d40/d4a/fa0 [0,4194304] 0 2024-04-12T20:56:41.415 INFO:tasks.workunit.client.1.smithi186.stdout:9/701: write d3/d2f/d54/f6a [854457,44418] 0 2024-04-12T20:56:41.420 INFO:tasks.workunit.client.1.smithi186.stdout:1/789: creat d14/d1a/d33/d36/dfa/f101 x:0 0 0 2024-04-12T20:56:41.420 INFO:tasks.workunit.client.1.smithi186.stdout:6/754: dwrite d9/d5e/d73/f84 [0,4194304] 0 2024-04-12T20:56:41.427 INFO:tasks.workunit.client.1.smithi186.stdout:8/787: write d7/de/d10/f3c [3606888,81531] 0 2024-04-12T20:56:41.431 INFO:tasks.workunit.client.1.smithi186.stdout:7/754: mkdir d7/d55/d71/d10a 0 2024-04-12T20:56:41.436 INFO:tasks.workunit.client.1.smithi186.stdout:3/743: dwrite d1/d3/f58 [0,4194304] 0 2024-04-12T20:56:41.436 INFO:tasks.workunit.client.1.smithi186.stdout:1/790: rename d14/d91/fc1 to d14/d1a/d3c/f102 0 2024-04-12T20:56:41.437 INFO:tasks.workunit.client.1.smithi186.stdout:2/859: dwrite d0/dc/d2e/f5f [0,4194304] 0 2024-04-12T20:56:41.439 INFO:tasks.workunit.client.1.smithi186.stdout:4/903: dwrite d0/dc/d64/f109 [0,4194304] 0 2024-04-12T20:56:41.447 INFO:tasks.workunit.client.1.smithi186.stdout:1/791: chown d14/d1a/d33/d36/d5b/d3e/d5f/l7a 7 1 2024-04-12T20:56:41.447 INFO:tasks.workunit.client.1.smithi186.stdout:3/744: chown d1/d3/d21/f65 129965600 1 2024-04-12T20:56:41.447 INFO:tasks.workunit.client.1.smithi186.stdout:6/755: rename d9/d43/lb9 to d9/d43/d59/d5d/d6e/d91/l10c 0 2024-04-12T20:56:41.450 INFO:tasks.workunit.client.1.smithi186.stdout:8/788: readlink d7/d18/d36/dad/lc7 0 2024-04-12T20:56:41.454 INFO:tasks.workunit.client.1.smithi186.stdout:1/792: mknod d14/d1a/d33/d36/d5b/d3e/d8b/c103 0 2024-04-12T20:56:41.459 INFO:tasks.workunit.client.1.smithi186.stdout:7/755: creat d7/d10/d2d/d86/dcb/f10b x:0 0 0 2024-04-12T20:56:41.462 INFO:tasks.workunit.client.1.smithi186.stdout:1/793: mkdir d14/d1a/d33/d36/d5b/d6a/da2/d104 0 2024-04-12T20:56:41.463 INFO:tasks.workunit.client.1.smithi186.stdout:1/794: chown l13 10376918 1 2024-04-12T20:56:41.466 INFO:tasks.workunit.client.1.smithi186.stdout:8/789: rmdir d7/d18/d36/d20/dd8 0 2024-04-12T20:56:41.466 INFO:tasks.workunit.client.1.smithi186.stdout:1/795: write d14/d1a/d33/d36/fd6 [3062724,36865] 0 2024-04-12T20:56:41.469 INFO:tasks.workunit.client.1.smithi186.stdout:8/790: chown d7/d18/l30 44068688 1 2024-04-12T20:56:41.471 INFO:tasks.workunit.client.1.smithi186.stdout:3/745: rename d1/d3/d21/d22/d24/d2f to d1/d3/df8 0 2024-04-12T20:56:41.471 INFO:tasks.workunit.client.1.smithi186.stdout:8/791: chown f5 0 1 2024-04-12T20:56:41.472 INFO:tasks.workunit.client.1.smithi186.stdout:9/702: dread d3/d2f/d54/d9f/dc4/d77/dd6/d72/df4/d6f/d83/fd7 [0,4194304] 0 2024-04-12T20:56:41.475 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:56:41 smithi018 ceph-mon[28799]: pgmap v53: 65 pgs: 65 active+clean; 3.1 GiB data, 11 GiB used, 526 GiB / 536 GiB avail; 56 MiB/s rd, 163 MiB/s wr, 352 op/s 2024-04-12T20:56:41.475 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:56:41 smithi018 ceph-mon[28799]: fsmap cephfs:2 {0=cephfs.smithi018.acqbsc=up:active,1=cephfs.smithi018.szjluo=up:active} 2 up:standby 2024-04-12T20:56:41.476 INFO:tasks.workunit.client.1.smithi186.stdout:2/860: dread d0/d32/d69/f7d [0,4194304] 0 2024-04-12T20:56:41.477 INFO:tasks.workunit.client.1.smithi186.stdout:9/703: dread - d3/d2f/d2e/d43/d4c/d82/dd4/fdb zero size 2024-04-12T20:56:41.478 INFO:tasks.workunit.client.1.smithi186.stdout:4/904: dwrite d0/f10 [0,4194304] 0 2024-04-12T20:56:41.479 INFO:tasks.workunit.client.1.smithi186.stdout:7/756: link d7/de1/f9d d7/d10/d2d/d40/f10c 0 2024-04-12T20:56:41.480 INFO:tasks.workunit.client.1.smithi186.stdout:8/792: creat d7/d18/d36/d20/d23/dd6/de8/f100 x:0 0 0 2024-04-12T20:56:41.482 INFO:tasks.workunit.client.1.smithi186.stdout:1/796: rename d14/d1a/d21/d35/dbc/de3/de4 to d14/d1a/d33/d36/d5b/d105 0 2024-04-12T20:56:41.483 INFO:tasks.workunit.client.1.smithi186.stdout:9/704: chown d3/d2f/d54/d9f/dc4/d77/dd6/d72/df4/f86 53638919 1 2024-04-12T20:56:41.485 INFO:tasks.workunit.client.1.smithi186.stdout:8/793: write d7/d18/d36/d1d/d7d/da5/f6e [1472434,83053] 0 2024-04-12T20:56:41.488 INFO:tasks.workunit.client.1.smithi186.stdout:7/757: read d7/d10/d2d/d40/d4a/d6c/d97/f92 [287648,89970] 0 2024-04-12T20:56:41.489 INFO:tasks.workunit.client.1.smithi186.stdout:2/861: link d0/dc/d2e/d73/fd8 d0/d3/d1a/d21/d26/d86/f117 0 2024-04-12T20:56:41.496 INFO:tasks.workunit.client.1.smithi186.stdout:2/862: rmdir d0/d3/d1a/d21/d22 39 2024-04-12T20:56:41.497 INFO:tasks.workunit.client.1.smithi186.stdout:7/758: mknod d7/d10/d1c/d8c/c10d 0 2024-04-12T20:56:41.498 INFO:tasks.workunit.client.1.smithi186.stdout:2/863: stat d0/d3/d1a/d48/df7/fa4 0 2024-04-12T20:56:41.500 INFO:tasks.workunit.client.1.smithi186.stdout:5/774: sync 2024-04-12T20:56:41.501 INFO:tasks.workunit.client.1.smithi186.stdout:9/705: dread d3/d2f/d2e/fe7 [0,4194304] 0 2024-04-12T20:56:41.503 INFO:tasks.workunit.client.1.smithi186.stdout:8/794: rename d7/d18/d36/d20/d71/feb to d7/d18/d36/f101 0 2024-04-12T20:56:41.503 INFO:tasks.workunit.client.1.smithi186.stdout:7/759: creat d7/d55/f10e x:0 0 0 2024-04-12T20:56:41.506 INFO:tasks.workunit.client.1.smithi186.stdout:2/864: mkdir d0/d3/d1a/d21/d118 0 2024-04-12T20:56:41.509 INFO:tasks.workunit.client.1.smithi186.stdout:3/746: dread d1/d3/df8/f92 [0,4194304] 0 2024-04-12T20:56:41.509 INFO:tasks.workunit.client.1.smithi186.stdout:8/795: symlink d7/d18/d36/dad/l102 0 2024-04-12T20:56:41.509 INFO:tasks.workunit.client.1.smithi186.stdout:1/797: dwrite d14/d1a/d21/f27 [0,4194304] 0 2024-04-12T20:56:41.510 INFO:tasks.workunit.client.1.smithi186.stdout:9/706: creat d3/d2f/d54/d9f/dc4/d77/dc3/daf/ff5 x:0 0 0 2024-04-12T20:56:41.511 INFO:tasks.workunit.client.1.smithi186.stdout:2/865: dread - d0/d32/d8b/f109 zero size 2024-04-12T20:56:41.512 INFO:tasks.workunit.client.1.smithi186.stdout:5/775: getdents d5/db/d15/de8 0 2024-04-12T20:56:41.512 INFO:tasks.workunit.client.1.smithi186.stdout:4/905: dread d0/f69 [0,4194304] 0 2024-04-12T20:56:41.517 INFO:tasks.workunit.client.1.smithi186.stdout:7/760: unlink d7/d10/d2d/d40/d4a/d6c/d97/d4d/d8a/dad/c100 0 2024-04-12T20:56:41.523 INFO:tasks.workunit.client.1.smithi186.stdout:8/796: mknod d7/d18/d36/d1d/d1f/da2/c103 0 2024-04-12T20:56:41.523 INFO:tasks.workunit.client.1.smithi186.stdout:4/906: fsync d0/dc/d64/f109 0 2024-04-12T20:56:41.523 INFO:tasks.workunit.client.1.smithi186.stdout:1/798: creat d14/d1a/d1d/de5/f106 x:0 0 0 2024-04-12T20:56:41.523 INFO:tasks.workunit.client.1.smithi186.stdout:9/707: creat d3/d2f/d54/d9f/dc4/d77/dd6/d72/df4/d6f/d83/ff6 x:0 0 0 2024-04-12T20:56:41.524 INFO:tasks.workunit.client.1.smithi186.stdout:3/747: truncate d1/d2c/d48/f5a 350304 0 2024-04-12T20:56:41.528 INFO:tasks.workunit.client.1.smithi186.stdout:8/797: mkdir d7/d18/def/d104 0 2024-04-12T20:56:41.530 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:56:41 smithi186 ceph-mon[36597]: pgmap v53: 65 pgs: 65 active+clean; 3.1 GiB data, 11 GiB used, 526 GiB / 536 GiB avail; 56 MiB/s rd, 163 MiB/s wr, 352 op/s 2024-04-12T20:56:41.530 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:56:41 smithi186 ceph-mon[36597]: fsmap cephfs:2 {0=cephfs.smithi018.acqbsc=up:active,1=cephfs.smithi018.szjluo=up:active} 2 up:standby 2024-04-12T20:56:41.540 INFO:tasks.workunit.client.1.smithi186.stdout:8/798: creat d7/d18/d36/d20/d23/dd6/de8/df7/f105 x:0 0 0 2024-04-12T20:56:41.542 INFO:tasks.workunit.client.1.smithi186.stdout:5/776: dwrite d5/db/d15/fea [4194304,4194304] 0 2024-04-12T20:56:41.542 INFO:tasks.workunit.client.1.smithi186.stdout:4/907: dwrite d0/f4d [0,4194304] 0 2024-04-12T20:56:41.548 INFO:tasks.workunit.client.1.smithi186.stdout:7/761: dwrite d7/d55/d71/dbe/fc6 [0,4194304] 0 2024-04-12T20:56:41.551 INFO:tasks.workunit.client.1.smithi186.stdout:0/876: sync 2024-04-12T20:56:41.551 INFO:tasks.workunit.client.1.smithi186.stdout:9/708: dwrite d3/d2f/d54/d9f/dc4/d77/dd6/d72/df4/d6f/f8c [0,4194304] 0 2024-04-12T20:56:41.554 INFO:tasks.workunit.client.1.smithi186.stdout:2/866: dread d0/d3/d1a/d48/f81 [0,4194304] 0 2024-04-12T20:56:41.564 INFO:tasks.workunit.client.1.smithi186.stdout:5/777: rename d5/d27/l8f to d5/db/d2e/dd0/lfe 0 2024-04-12T20:56:41.567 INFO:tasks.workunit.client.1.smithi186.stdout:2/867: creat d0/d3/d7a/f119 x:0 0 0 2024-04-12T20:56:41.568 INFO:tasks.workunit.client.1.smithi186.stdout:1/799: dread d14/d1a/d33/d36/d5b/d3e/f7e [0,4194304] 0 2024-04-12T20:56:41.568 INFO:tasks.workunit.client.1.smithi186.stdout:2/868: fsync d0/dc/f4c 0 2024-04-12T20:56:41.568 INFO:tasks.workunit.client.1.smithi186.stdout:9/709: mknod d3/d2f/d54/d9f/dc4/d77/dd6/d72/df4/d6f/d83/dd5/cf7 0 2024-04-12T20:56:41.572 INFO:tasks.workunit.client.1.smithi186.stdout:8/799: dwrite d7/de/d68/fcb [0,4194304] 0 2024-04-12T20:56:41.572 INFO:tasks.workunit.client.1.smithi186.stdout:5/778: truncate d5/db/ff8 1048046 0 2024-04-12T20:56:41.576 INFO:tasks.workunit.client.1.smithi186.stdout:5/779: fdatasync d5/db/d2e/f64 0 2024-04-12T20:56:41.576 INFO:tasks.workunit.client.1.smithi186.stdout:0/877: dread d0/d5c/d66/da7/fb4 [0,4194304] 0 2024-04-12T20:56:41.578 INFO:tasks.workunit.client.1.smithi186.stdout:1/800: rmdir d14/d1a/d33/d36/d5b/db9/dc9 39 2024-04-12T20:56:41.580 INFO:tasks.workunit.client.1.smithi186.stdout:8/800: mknod d7/d18/d36/d20/d23/dd6/de8/c106 0 2024-04-12T20:56:41.581 INFO:tasks.workunit.client.1.smithi186.stdout:4/908: write d0/d59/dc0/d48/f95 [942007,50509] 0 2024-04-12T20:56:41.582 INFO:tasks.workunit.client.1.smithi186.stdout:2/869: dread d0/dc/d2e/d73/fb5 [0,4194304] 0 2024-04-12T20:56:41.584 INFO:tasks.workunit.client.1.smithi186.stdout:1/801: rmdir d14/d6c 39 2024-04-12T20:56:41.584 INFO:tasks.workunit.client.1.smithi186.stdout:8/801: dread - d7/d18/d36/d1d/d1f/fd2 zero size 2024-04-12T20:56:41.586 INFO:tasks.workunit.client.1.smithi186.stdout:9/710: rename d3/d2f/d54/ca4 to d3/d2f/d2e/cf8 0 2024-04-12T20:56:41.587 INFO:tasks.workunit.client.1.smithi186.stdout:8/802: fsync d7/d18/d36/d1d/d7d/da5/f6e 0 2024-04-12T20:56:41.590 INFO:tasks.workunit.client.1.smithi186.stdout:1/802: mkdir d14/d1a/d21/d35/d107 0 2024-04-12T20:56:41.592 INFO:tasks.workunit.client.1.smithi186.stdout:0/878: link d0/d14/d4e/d50/l90 d0/d81/l11e 0 2024-04-12T20:56:41.596 INFO:tasks.workunit.client.1.smithi186.stdout:0/879: write d0/d10/d72/fed [146357,65323] 0 2024-04-12T20:56:41.597 INFO:tasks.workunit.client.1.smithi186.stdout:8/803: creat d7/d18/d36/d1d/d1f/d26/de6/f107 x:0 0 0 2024-04-12T20:56:41.600 INFO:tasks.workunit.client.1.smithi186.stdout:0/880: write d0/d10/f4d [1084954,113943] 0 2024-04-12T20:56:41.601 INFO:tasks.workunit.client.1.smithi186.stdout:8/804: chown d7/de/d7a/f80 58 1 2024-04-12T20:56:41.602 INFO:tasks.workunit.client.1.smithi186.stdout:5/780: truncate d5/db/f40 169762 0 2024-04-12T20:56:41.604 INFO:tasks.workunit.client.1.smithi186.stdout:8/805: write d7/de/d7a/fd9 [149575,39546] 0 2024-04-12T20:56:41.605 INFO:tasks.workunit.client.1.smithi186.stdout:4/909: dwrite d0/d59/ddf/ff0 [0,4194304] 0 2024-04-12T20:56:41.636 INFO:tasks.workunit.client.1.smithi186.stdout:6/756: sync 2024-04-12T20:56:41.636 INFO:tasks.workunit.client.1.smithi186.stdout:9/711: dread d3/d2f/d2e/f2d [0,4194304] 0 2024-04-12T20:56:41.636 INFO:tasks.workunit.client.1.smithi186.stdout:6/757: stat d9/c3e 0 2024-04-12T20:56:41.639 INFO:tasks.workunit.client.1.smithi186.stdout:0/881: symlink d0/d14/d4e/l11f 0 2024-04-12T20:56:41.640 INFO:tasks.workunit.client.1.smithi186.stdout:8/806: truncate d7/d18/d36/d20/d23/ff9 416764 0 2024-04-12T20:56:41.640 INFO:tasks.workunit.client.1.smithi186.stdout:6/758: truncate d9/d4c/d6f/f9a 616512 0 2024-04-12T20:56:41.641 INFO:tasks.workunit.client.1.smithi186.stdout:9/712: write d3/d2f/d54/d9f/dc4/d77/dd6/d72/df4/f86 [66440,46327] 0 2024-04-12T20:56:41.642 INFO:tasks.workunit.client.1.smithi186.stdout:5/781: dread d5/db/d15/d2a/d2d/d46/f6c [0,4194304] 0 2024-04-12T20:56:41.642 INFO:tasks.workunit.client.1.smithi186.stdout:6/759: fsync d9/d43/f44 0 2024-04-12T20:56:41.644 INFO:tasks.workunit.client.1.smithi186.stdout:0/882: mkdir d0/d10/d32/de2/d120 0 2024-04-12T20:56:41.647 INFO:tasks.workunit.client.1.smithi186.stdout:5/782: write d5/db/d33/f36 [4721309,117385] 0 2024-04-12T20:56:41.647 INFO:tasks.workunit.client.1.smithi186.stdout:9/713: mkdir d3/d2f/d54/d9f/dc4/d77/dd6/d72/df4/df9 0 2024-04-12T20:56:41.648 INFO:tasks.workunit.client.1.smithi186.stdout:0/883: write d0/d10/d32/dac/fb6 [4383621,1551] 0 2024-04-12T20:56:41.648 INFO:tasks.workunit.client.1.smithi186.stdout:6/760: symlink d9/d43/d59/d5d/de6/l10d 0 2024-04-12T20:56:41.648 INFO:tasks.workunit.client.1.smithi186.stdout:1/803: dread d14/d1a/d58/f97 [0,4194304] 0 2024-04-12T20:56:41.650 INFO:tasks.workunit.client.1.smithi186.stdout:8/807: dread d7/d18/d36/d1d/fa0 [0,4194304] 0 2024-04-12T20:56:41.650 INFO:tasks.workunit.client.1.smithi186.stdout:5/783: write d5/db/d15/fea [2742862,62023] 0 2024-04-12T20:56:41.653 INFO:tasks.workunit.client.1.smithi186.stdout:0/884: mkdir d0/d10/d71/d121 0 2024-04-12T20:56:41.653 INFO:tasks.workunit.client.1.smithi186.stdout:9/714: chown d3/d2f/d2e/d33/d36/d70/f7c 78847990 1 2024-04-12T20:56:41.658 INFO:tasks.workunit.client.1.smithi186.stdout:1/804: link d14/d1a/d21/d35/dbc/ff2 d14/d91/f108 0 2024-04-12T20:56:41.660 INFO:tasks.workunit.client.1.smithi186.stdout:0/885: symlink d0/d5c/d66/df8/l122 0 2024-04-12T20:56:41.661 INFO:tasks.workunit.client.1.smithi186.stdout:6/761: getdents d9/d4c/dbe 0 2024-04-12T20:56:41.665 INFO:tasks.workunit.client.1.smithi186.stdout:0/886: write d0/d14/f35 [4682466,50667] 0 2024-04-12T20:56:41.666 INFO:tasks.workunit.client.1.smithi186.stdout:8/808: mkdir d7/d18/d36/d20/d23/dd7/df5/d108 0 2024-04-12T20:56:41.671 INFO:tasks.workunit.client.1.smithi186.stdout:5/784: link d5/fd d5/db/d2e/d70/de2/fff 0 2024-04-12T20:56:41.672 INFO:tasks.workunit.client.1.smithi186.stdout:0/887: write d0/d14/d4e/d50/d6e/faf [501463,57199] 0 2024-04-12T20:56:41.673 INFO:tasks.workunit.client.1.smithi186.stdout:1/805: truncate d14/d6c/fcb 268921 0 2024-04-12T20:56:41.673 INFO:tasks.workunit.client.1.smithi186.stdout:8/809: write d7/d18/d36/d1d/d1f/da2/f79 [6295835,48063] 0 2024-04-12T20:56:41.674 INFO:tasks.workunit.client.1.smithi186.stdout:9/715: dwrite d3/fb [0,4194304] 0 2024-04-12T20:56:41.696 INFO:tasks.workunit.client.1.smithi186.stdout:6/762: mkdir d9/d43/d59/d5d/d6e/d10e 0 2024-04-12T20:56:41.697 INFO:tasks.workunit.client.1.smithi186.stdout:1/806: dread d14/d1a/d33/d36/d5b/d3e/d71/f78 [0,4194304] 0 2024-04-12T20:56:41.699 INFO:tasks.workunit.client.1.smithi186.stdout:5/785: creat d5/db/d71/f100 x:0 0 0 2024-04-12T20:56:41.700 INFO:tasks.workunit.client.1.smithi186.stdout:5/786: write d5/f6 [2875487,37425] 0 2024-04-12T20:56:41.700 INFO:tasks.workunit.client.1.smithi186.stdout:6/763: creat d9/d4c/d6f/d97/f10f x:0 0 0 2024-04-12T20:56:41.701 INFO:tasks.workunit.client.1.smithi186.stdout:8/810: mknod d7/d18/d36/d1d/d7d/d81/c109 0 2024-04-12T20:56:41.703 INFO:tasks.workunit.client.1.smithi186.stdout:8/811: readlink d7/de/d10/d7f/l83 0 2024-04-12T20:56:41.704 INFO:tasks.workunit.client.1.smithi186.stdout:0/888: creat d0/d14/f123 x:0 0 0 2024-04-12T20:56:41.704 INFO:tasks.workunit.client.1.smithi186.stdout:9/716: link d3/f73 d3/d2f/d54/d9f/dc4/d6b/ffa 0 2024-04-12T20:56:41.705 INFO:tasks.workunit.client.1.smithi186.stdout:0/889: chown d0/d14/d4e/c55 515086028 1 2024-04-12T20:56:41.706 INFO:tasks.workunit.client.1.smithi186.stdout:9/717: write d3/fb [9004298,8114] 0 2024-04-12T20:56:41.707 INFO:tasks.workunit.client.1.smithi186.stdout:6/764: creat d9/d43/d59/d5d/de6/f110 x:0 0 0 2024-04-12T20:56:41.710 INFO:tasks.workunit.client.1.smithi186.stdout:6/765: read d9/d43/d2d/f7e [81117,6353] 0 2024-04-12T20:56:41.710 INFO:tasks.workunit.client.1.smithi186.stdout:1/807: creat d14/d1a/d33/d36/d5b/d3e/f109 x:0 0 0 2024-04-12T20:56:41.711 INFO:tasks.workunit.client.1.smithi186.stdout:8/812: write d7/de/d10/f54 [7184027,39955] 0 2024-04-12T20:56:41.712 INFO:tasks.workunit.client.1.smithi186.stdout:5/787: dread d5/db/f34 [0,4194304] 0 2024-04-12T20:56:41.713 INFO:tasks.workunit.client.1.smithi186.stdout:1/808: dread - d14/d1a/d21/fc6 zero size 2024-04-12T20:56:41.715 INFO:tasks.workunit.client.1.smithi186.stdout:6/766: write d9/d43/d59/d5d/d6e/d91/dd9/ff8 [703917,4430] 0 2024-04-12T20:56:41.715 INFO:tasks.workunit.client.1.smithi186.stdout:5/788: fdatasync d5/db/d2e/f64 0 2024-04-12T20:56:41.717 INFO:tasks.workunit.client.1.smithi186.stdout:1/809: stat d14/d1a/d33/d36/d5b/d6a/cfe 0 2024-04-12T20:56:41.718 INFO:tasks.workunit.client.1.smithi186.stdout:8/813: write d7/d18/d36/d20/d23/dd6/de8/f100 [1033186,79951] 0 2024-04-12T20:56:41.718 INFO:tasks.workunit.client.1.smithi186.stdout:9/718: rename d3/d2f/d54/d9f/dc4/f57 to d3/d2f/d54/d9f/dc4/d77/dd6/d72/df4/df9/ffb 0 2024-04-12T20:56:41.720 INFO:tasks.workunit.client.1.smithi186.stdout:6/767: write d9/d4c/d6f/fb2 [2878638,17845] 0 2024-04-12T20:56:41.723 INFO:tasks.workunit.client.1.smithi186.stdout:8/814: fdatasync d7/d18/d36/d1d/fb6 0 2024-04-12T20:56:41.726 INFO:tasks.workunit.client.1.smithi186.stdout:0/890: dwrite d0/d85/fde [0,4194304] 0 2024-04-12T20:56:41.726 INFO:tasks.workunit.client.1.smithi186.stdout:6/768: truncate d9/d4c/d6f/fb2 4614833 0 2024-04-12T20:56:41.726 INFO:tasks.workunit.client.1.smithi186.stdout:5/789: rename d5/db/d15/d2a/d6d to d5/db/d2e/d70/d101 0 2024-04-12T20:56:41.728 INFO:tasks.workunit.client.1.smithi186.stdout:0/891: stat d0/d5c/d65/fbe 0 2024-04-12T20:56:41.728 INFO:tasks.workunit.client.1.smithi186.stdout:8/815: write d7/d18/d36/d1d/d1f/da2/df8/f34 [2803033,73399] 0 2024-04-12T20:56:41.729 INFO:tasks.workunit.client.1.smithi186.stdout:9/719: truncate d3/d2f/d2e/d33/d36/d70/f99 720803 0 2024-04-12T20:56:41.730 INFO:tasks.workunit.client.1.smithi186.stdout:5/790: write d5/db/d2e/d70/f99 [1744011,74917] 0 2024-04-12T20:56:41.730 INFO:tasks.workunit.client.1.smithi186.stdout:0/892: chown d0/d10/d72/d79 341832 1 2024-04-12T20:56:41.733 INFO:tasks.workunit.client.1.smithi186.stdout:0/893: stat d0/d5d/ddc/fe0 0 2024-04-12T20:56:41.735 INFO:tasks.workunit.client.1.smithi186.stdout:9/720: write d3/d2f/d54/d9f/dc4/d77/dd6/d72/df4/f86 [703637,91702] 0 2024-04-12T20:56:41.736 INFO:tasks.workunit.client.1.smithi186.stdout:7/762: sync 2024-04-12T20:56:41.736 INFO:tasks.workunit.client.1.smithi186.stdout:4/910: sync 2024-04-12T20:56:41.736 INFO:tasks.workunit.client.1.smithi186.stdout:2/870: sync 2024-04-12T20:56:41.736 INFO:tasks.workunit.client.1.smithi186.stdout:3/748: sync 2024-04-12T20:56:41.736 INFO:tasks.workunit.client.1.smithi186.stdout:9/721: read - d3/d2f/d2e/d33/fb9 zero size 2024-04-12T20:56:41.738 INFO:tasks.workunit.client.1.smithi186.stdout:7/763: write d7/f74 [1446046,86764] 0 2024-04-12T20:56:41.741 INFO:tasks.workunit.client.1.smithi186.stdout:3/749: truncate d1/d2c/d54/d7c/dbb/fd7 910116 0 2024-04-12T20:56:41.742 INFO:tasks.workunit.client.1.smithi186.stdout:0/894: symlink d0/d10/d72/l124 0 2024-04-12T20:56:41.744 INFO:tasks.workunit.client.1.smithi186.stdout:0/895: write d0/d14/d4e/d1e/fa9 [1816127,91607] 0 2024-04-12T20:56:41.747 INFO:tasks.workunit.client.1.smithi186.stdout:2/871: rmdir d0/dc/d2e/de8 39 2024-04-12T20:56:41.750 INFO:tasks.workunit.client.1.smithi186.stdout:4/911: creat d0/dc/d64/f125 x:0 0 0 2024-04-12T20:56:41.751 INFO:tasks.workunit.client.1.smithi186.stdout:4/912: fsync d0/d21/f41 0 2024-04-12T20:56:41.752 INFO:tasks.workunit.client.1.smithi186.stdout:9/722: symlink d3/d2f/d2e/d33/d36/d70/lfc 0 2024-04-12T20:56:41.753 INFO:tasks.workunit.client.1.smithi186.stdout:9/723: dread - d3/fd0 zero size 2024-04-12T20:56:41.755 INFO:tasks.workunit.client.1.smithi186.stdout:6/769: getdents d9/d43/d59/d5d/d101/ddb 0 2024-04-12T20:56:41.757 INFO:tasks.workunit.client.1.smithi186.stdout:2/872: creat d0/dc/d2e/d73/f11a x:0 0 0 2024-04-12T20:56:41.757 INFO:tasks.workunit.client.1.smithi186.stdout:7/764: dwrite d7/d10/d2d/d40/d4a/d6c/d97/d4d/d6a/ff4 [0,4194304] 0 2024-04-12T20:56:41.759 INFO:tasks.workunit.client.1.smithi186.stdout:6/770: truncate d9/d4c/d6f/d89/d9c/f105 109244 0 2024-04-12T20:56:41.762 INFO:tasks.workunit.client.1.smithi186.stdout:9/724: mkdir d3/d2f/d2e/d43/d4c/d82/dfd 0 2024-04-12T20:56:41.763 INFO:tasks.workunit.client.1.smithi186.stdout:7/765: chown d7/d10/f13 672875 1 2024-04-12T20:56:41.768 INFO:tasks.workunit.client.1.smithi186.stdout:4/913: write d0/d21/f31 [3355647,68540] 0 2024-04-12T20:56:41.770 INFO:tasks.workunit.client.1.smithi186.stdout:7/766: mknod d7/de1/c10f 0 2024-04-12T20:56:41.773 INFO:tasks.workunit.client.1.smithi186.stdout:0/896: rename d0/d10/d32/d3f/f64 to d0/d14/d4e/d50/d6e/f125 0 2024-04-12T20:56:41.774 INFO:tasks.workunit.client.1.smithi186.stdout:3/750: getdents d1/d2c/d54/d7c 0 2024-04-12T20:56:41.775 INFO:tasks.workunit.client.1.smithi186.stdout:0/897: dread - d0/d14/d4e/d50/f110 zero size 2024-04-12T20:56:41.776 INFO:tasks.workunit.client.1.smithi186.stdout:6/771: creat d9/d4c/d6f/f111 x:0 0 0 2024-04-12T20:56:41.779 INFO:tasks.workunit.client.1.smithi186.stdout:6/772: chown d9/d43/d59/d5d/d6e/d91/dd9/ff8 4904217 1 2024-04-12T20:56:41.780 INFO:tasks.workunit.client.1.smithi186.stdout:0/898: unlink d0/d10/f8f 0 2024-04-12T20:56:41.786 INFO:tasks.workunit.client.1.smithi186.stdout:6/773: creat d9/d43/d50/f112 x:0 0 0 2024-04-12T20:56:41.787 INFO:tasks.workunit.client.1.smithi186.stdout:6/774: stat d9/d43/d59/d5d/fbf 0 2024-04-12T20:56:41.789 INFO:tasks.workunit.client.1.smithi186.stdout:0/899: creat d0/d5c/d66/f126 x:0 0 0 2024-04-12T20:56:41.790 INFO:tasks.workunit.client.1.smithi186.stdout:4/914: dread d0/d21/f20 [0,4194304] 0 2024-04-12T20:56:41.791 INFO:tasks.workunit.client.1.smithi186.stdout:9/725: rename d3/fd0 to d3/d2f/d2e/ffe 0 2024-04-12T20:56:41.792 INFO:tasks.workunit.client.1.smithi186.stdout:0/900: truncate d0/d10/d32/fe9 1176639 0 2024-04-12T20:56:41.793 INFO:tasks.workunit.client.1.smithi186.stdout:6/775: mknod d9/da9/dbd/c113 0 2024-04-12T20:56:41.794 INFO:tasks.workunit.client.1.smithi186.stdout:1/810: write d14/d74/f9a [2851122,107516] 0 2024-04-12T20:56:41.795 INFO:tasks.workunit.client.1.smithi186.stdout:6/776: chown d9/d43/d59/d5d/d6e/l76 5390344 1 2024-04-12T20:56:41.796 INFO:tasks.workunit.client.1.smithi186.stdout:8/816: truncate d7/f90 204411 0 2024-04-12T20:56:41.796 INFO:tasks.workunit.client.1.smithi186.stdout:6/777: readlink d9/l21 0 2024-04-12T20:56:41.798 INFO:tasks.workunit.client.1.smithi186.stdout:9/726: chown d3/d2f/d2e/db3/lee 3126 1 2024-04-12T20:56:41.799 INFO:tasks.workunit.client.1.smithi186.stdout:0/901: unlink d0/d5d/cc7 0 2024-04-12T20:56:41.801 INFO:tasks.workunit.client.1.smithi186.stdout:5/791: sync 2024-04-12T20:56:41.803 INFO:tasks.workunit.client.1.smithi186.stdout:7/767: rename d7/d10/d2d/d86/cb2 to d7/d10/d1c/d108/c110 0 2024-04-12T20:56:41.803 INFO:tasks.workunit.client.1.smithi186.stdout:5/792: fdatasync d5/db/d15/d2a/d2d/d46/d55/fc6 0 2024-04-12T20:56:41.804 INFO:tasks.workunit.client.1.smithi186.stdout:6/778: creat d9/d43/d59/d5d/d6e/d91/dd9/f114 x:0 0 0 2024-04-12T20:56:41.808 INFO:tasks.workunit.client.1.smithi186.stdout:4/915: rename d0/d59/dc0/d63/l72 to d0/dc/d88/d80/d9c/d102/l126 0 2024-04-12T20:56:41.812 INFO:tasks.workunit.client.1.smithi186.stdout:8/817: truncate d7/d18/d36/d20/d71/fc0 231576 0 2024-04-12T20:56:41.813 INFO:tasks.workunit.client.1.smithi186.stdout:2/873: write d0/d3/d1a/d21/d26/d86/f117 [1617683,23441] 0 2024-04-12T20:56:41.815 INFO:tasks.workunit.client.1.smithi186.stdout:8/818: stat d7/de/d68/cc8 0 2024-04-12T20:56:41.815 INFO:tasks.workunit.client.1.smithi186.stdout:6/779: symlink d9/d43/d59/l115 0 2024-04-12T20:56:41.816 INFO:tasks.workunit.client.1.smithi186.stdout:1/811: dwrite d14/d1a/d33/d36/d5b/db9/fba [0,4194304] 0 2024-04-12T20:56:41.817 INFO:tasks.workunit.client.1.smithi186.stdout:0/902: dwrite d0/d5c/d65/f7b [0,4194304] 0 2024-04-12T20:56:41.820 INFO:tasks.workunit.client.1.smithi186.stdout:6/780: chown d9/d4c/d6f/ccf 1855765 1 2024-04-12T20:56:41.824 INFO:tasks.workunit.client.1.smithi186.stdout:9/727: rename d3/d13/f4e to d3/d2f/d54/d9f/dc4/d77/dd6/d72/df4/fff 0 2024-04-12T20:56:41.824 INFO:tasks.workunit.client.1.smithi186.stdout:3/751: truncate d1/d3/df8/f92 950194 0 2024-04-12T20:56:41.826 INFO:tasks.workunit.client.1.smithi186.stdout:2/874: creat d0/d3/d1a/d10b/f11b x:0 0 0 2024-04-12T20:56:41.833 INFO:tasks.workunit.client.1.smithi186.stdout:1/812: creat d14/d1a/d33/d36/d5b/d6a/da2/f10a x:0 0 0 2024-04-12T20:56:41.841 INFO:tasks.workunit.client.1.smithi186.stdout:5/793: dread d5/db/d15/d2a/d4c/d5b/d7a/d94/fa3 [0,4194304] 0 2024-04-12T20:56:41.843 INFO:tasks.workunit.client.1.smithi186.stdout:9/728: dwrite d3/d2f/d54/d9f/dc4/d77/dc3/daf/ff5 [0,4194304] 0 2024-04-12T20:56:41.844 INFO:tasks.workunit.client.1.smithi186.stdout:4/916: mknod d0/d21/c127 0 2024-04-12T20:56:41.845 INFO:tasks.workunit.client.1.smithi186.stdout:2/875: dwrite d0/d3/d1a/d27/fd0 [0,4194304] 0 2024-04-12T20:56:41.846 INFO:tasks.workunit.client.1.smithi186.stdout:7/768: rename d7/d10/d2d/d40/d4a/d6c/d97/d4d/c91 to d7/d10/d1c/d5d/c111 0 2024-04-12T20:56:41.848 INFO:tasks.workunit.client.1.smithi186.stdout:7/769: dread - d7/d55/d71/f85 zero size 2024-04-12T20:56:41.851 INFO:tasks.workunit.client.1.smithi186.stdout:8/819: mkdir d7/d18/d36/d1d/d10a 0 2024-04-12T20:56:41.856 INFO:tasks.workunit.client.1.smithi186.stdout:3/752: fsync d1/d3/df8/d95/ff3 0 2024-04-12T20:56:41.861 INFO:tasks.workunit.client.1.smithi186.stdout:2/876: dread d0/d3/d1a/d48/f81 [0,4194304] 0 2024-04-12T20:56:41.861 INFO:tasks.workunit.client.1.smithi186.stdout:0/903: write d0/f4f [3767525,93588] 0 2024-04-12T20:56:41.861 INFO:tasks.workunit.client.1.smithi186.stdout:1/813: creat d14/d1a/d33/d36/d5b/d6a/da2/f10b x:0 0 0 2024-04-12T20:56:41.866 INFO:tasks.workunit.client.1.smithi186.stdout:5/794: write d5/db/d15/f89 [2150345,16010] 0 2024-04-12T20:56:41.868 INFO:tasks.workunit.client.1.smithi186.stdout:7/770: creat d7/d10/d1c/d8c/f112 x:0 0 0 2024-04-12T20:56:41.874 INFO:tasks.workunit.client.1.smithi186.stdout:0/904: mknod d0/d14/d105/c127 0 2024-04-12T20:56:41.879 INFO:tasks.workunit.client.1.smithi186.stdout:5/795: fsync d5/db/d15/d2a/d2d/f9f 0 2024-04-12T20:56:41.881 INFO:tasks.workunit.client.1.smithi186.stdout:2/877: dread d0/d3/d1a/d27/fd0 [0,4194304] 0 2024-04-12T20:56:41.883 INFO:tasks.workunit.client.1.smithi186.stdout:7/771: creat d7/d10/d2d/d40/d4a/d6c/d97/dc2/f113 x:0 0 0 2024-04-12T20:56:41.887 INFO:tasks.workunit.client.1.smithi186.stdout:1/814: dwrite d14/d1a/d33/d36/d5b/d3e/d71/f78 [0,4194304] 0 2024-04-12T20:56:41.888 INFO:tasks.workunit.client.1.smithi186.stdout:2/878: write d0/d3e/fe6 [757406,106037] 0 2024-04-12T20:56:41.888 INFO:tasks.workunit.client.1.smithi186.stdout:9/729: getdents d3/d13 0 2024-04-12T20:56:41.888 INFO:tasks.workunit.client.1.smithi186.stdout:5/796: mknod d5/db/d2e/d70/c102 0 2024-04-12T20:56:41.893 INFO:tasks.workunit.client.1.smithi186.stdout:8/820: write d7/de/d7a/f31 [1147988,100944] 0 2024-04-12T20:56:41.893 INFO:tasks.workunit.client.1.smithi186.stdout:6/781: truncate d9/d43/fd5 981608 0 2024-04-12T20:56:41.894 INFO:tasks.workunit.client.1.smithi186.stdout:9/730: readlink d3/d2f/d54/d9f/dc4/d77/dd6/d72/df4/l42 0 2024-04-12T20:56:41.899 INFO:tasks.workunit.client.1.smithi186.stdout:2/879: creat d0/d32/d8b/f11c x:0 0 0 2024-04-12T20:56:41.899 INFO:tasks.workunit.client.1.smithi186.stdout:3/753: dread d1/d3/d21/fb1 [0,4194304] 0 2024-04-12T20:56:41.899 INFO:tasks.workunit.client.1.smithi186.stdout:0/905: dwrite d0/d5c/d65/ff7 [0,4194304] 0 2024-04-12T20:56:41.900 INFO:tasks.workunit.client.1.smithi186.stdout:5/797: symlink d5/db/d33/l103 0 2024-04-12T20:56:41.902 INFO:tasks.workunit.client.1.smithi186.stdout:4/917: dread d0/f4d [4194304,4194304] 0 2024-04-12T20:56:41.903 INFO:tasks.workunit.client.1.smithi186.stdout:9/731: chown d3/d2f/d54/l84 3926 1 2024-04-12T20:56:41.906 INFO:tasks.workunit.client.1.smithi186.stdout:8/821: truncate d7/de/f21 382545 0 2024-04-12T20:56:41.907 INFO:tasks.workunit.client.1.smithi186.stdout:4/918: chown d0/d59/dc0/d63/da6/dc6/f66 30148 1 2024-04-12T20:56:41.907 INFO:tasks.workunit.client.1.smithi186.stdout:5/798: truncate d5/db/f34 1165582 0 2024-04-12T20:56:41.909 INFO:tasks.workunit.client.1.smithi186.stdout:3/754: creat d1/d3/ff9 x:0 0 0 2024-04-12T20:56:41.911 INFO:tasks.workunit.client.1.smithi186.stdout:0/906: rename d0/d10/c69 to d0/d14/c128 0 2024-04-12T20:56:41.911 INFO:tasks.workunit.client.1.smithi186.stdout:4/919: symlink d0/dc/d88/l128 0 2024-04-12T20:56:41.914 INFO:tasks.workunit.client.1.smithi186.stdout:4/920: read - d0/dc/d88/d80/f123 zero size 2024-04-12T20:56:41.915 INFO:tasks.workunit.client.1.smithi186.stdout:6/782: dwrite d9/d43/d59/d5d/d6e/d91/dd9/f114 [0,4194304] 0 2024-04-12T20:56:41.916 INFO:tasks.workunit.client.1.smithi186.stdout:5/799: creat d5/db/d15/d2a/d4c/d5b/d7a/f104 x:0 0 0 2024-04-12T20:56:41.917 INFO:tasks.workunit.client.1.smithi186.stdout:3/755: chown d1/d3/da/c26 4 1 2024-04-12T20:56:41.921 INFO:tasks.workunit.client.1.smithi186.stdout:9/732: rename d3/d2f/d54/d9f/dc4/d77/dd6/d72/c78 to d3/d2f/d2e/d33/d61/c100 0 2024-04-12T20:56:41.921 INFO:tasks.workunit.client.1.smithi186.stdout:3/756: chown d1/d3/d21/d22/d24/cd8 17794 1 2024-04-12T20:56:41.924 INFO:tasks.workunit.client.1.smithi186.stdout:6/783: write d9/d43/d59/d5d/d101/fc9 [1156261,23882] 0 2024-04-12T20:56:41.927 INFO:tasks.workunit.client.1.smithi186.stdout:9/733: chown d3/d2f/d2e/d43/d4c/d82/la3 5 1 2024-04-12T20:56:41.930 INFO:tasks.workunit.client.1.smithi186.stdout:4/921: link d0/d59/dc0/d48/d61/ff7 d0/dc/d33/d9a/f129 0 2024-04-12T20:56:41.933 INFO:tasks.workunit.client.1.smithi186.stdout:7/772: write d7/d10/d2d/d40/d4a/d6c/d97/d4d/f73 [1208870,82254] 0 2024-04-12T20:56:41.934 INFO:tasks.workunit.client.1.smithi186.stdout:6/784: chown d9/d43/f45 129750 1 2024-04-12T20:56:41.935 INFO:tasks.workunit.client.1.smithi186.stdout:5/800: rename d5/db/d2e/d70/d101/dbb/dc4/f67 to d5/db/d2e/d70/d101/f105 0 2024-04-12T20:56:41.937 INFO:tasks.workunit.client.1.smithi186.stdout:6/785: chown d9/d4c/fb6 18892909 1 2024-04-12T20:56:41.938 INFO:tasks.workunit.client.1.smithi186.stdout:9/734: fdatasync d3/d2f/d54/d9f/dc4/d77/dd6/d72/df4/d6f/d83/f38 0 2024-04-12T20:56:41.941 INFO:tasks.workunit.client.1.smithi186.stdout:2/880: write d0/d3/d1a/d27/f59 [2139276,98613] 0 2024-04-12T20:56:41.947 INFO:tasks.workunit.client.1.smithi186.stdout:8/822: truncate f5 2243171 0 2024-04-12T20:56:41.947 INFO:tasks.workunit.client.1.smithi186.stdout:4/922: truncate d0/d59/dc0/d63/da6/feb 677706 0 2024-04-12T20:56:41.948 INFO:tasks.workunit.client.1.smithi186.stdout:9/735: write d3/fce [4445496,65457] 0 2024-04-12T20:56:41.949 INFO:tasks.workunit.client.1.smithi186.stdout:2/881: chown d0/d3/dce/ddf/lf9 482970 1 2024-04-12T20:56:41.951 INFO:tasks.workunit.client.1.smithi186.stdout:7/773: mkdir d7/d10/d2d/d40/d4a/d6c/d97/d4d/d84/d114 0 2024-04-12T20:56:41.954 INFO:tasks.workunit.client.1.smithi186.stdout:7/774: fdatasync d7/f74 0 2024-04-12T20:56:41.955 INFO:tasks.workunit.client.1.smithi186.stdout:4/923: readlink d0/d59/dc0/led 0 2024-04-12T20:56:41.955 INFO:tasks.workunit.client.1.smithi186.stdout:3/757: dwrite d1/d3/d21/d22/d34/ddb/d4d/fd2 [0,4194304] 0 2024-04-12T20:56:41.958 INFO:tasks.workunit.client.1.smithi186.stdout:1/815: dwrite f9 [0,4194304] 0 2024-04-12T20:56:41.961 INFO:tasks.workunit.client.1.smithi186.stdout:0/907: dread d0/d5c/d65/fbe [0,4194304] 0 2024-04-12T20:56:41.962 INFO:tasks.workunit.client.1.smithi186.stdout:3/758: read d1/d3/d21/f9a [3781515,69465] 0 2024-04-12T20:56:41.963 INFO:tasks.workunit.client.1.smithi186.stdout:1/816: chown d14/d1a/d58/d69/fb3 9542 1 2024-04-12T20:56:41.967 INFO:tasks.workunit.client.1.smithi186.stdout:6/786: mkdir d9/d43/d59/d116 0 2024-04-12T20:56:41.970 INFO:tasks.workunit.client.1.smithi186.stdout:4/924: creat d0/d26/f12a x:0 0 0 2024-04-12T20:56:41.979 INFO:tasks.workunit.client.1.smithi186.stdout:9/736: symlink d3/d2f/d2e/d43/l101 0 2024-04-12T20:56:41.985 INFO:tasks.workunit.client.1.smithi186.stdout:0/908: creat d0/d5c/d66/da7/dfc/d11d/f129 x:0 0 0 2024-04-12T20:56:41.987 INFO:tasks.workunit.client.1.smithi186.stdout:9/737: unlink d3/d2f/d54/d92/f97 0 2024-04-12T20:56:41.989 INFO:tasks.workunit.client.1.smithi186.stdout:3/759: dwrite d1/fc [4194304,4194304] 0 2024-04-12T20:56:41.990 INFO:tasks.workunit.client.1.smithi186.stdout:1/817: getdents d14/d1a/d33/d36/d5b/d6a/da2 0 2024-04-12T20:56:41.991 INFO:tasks.workunit.client.1.smithi186.stdout:2/882: dwrite d0/d3/d1a/f87 [0,4194304] 0 2024-04-12T20:56:41.993 INFO:tasks.workunit.client.1.smithi186.stdout:5/801: dread d5/f6 [4194304,4194304] 0 2024-04-12T20:56:41.996 INFO:tasks.workunit.client.1.smithi186.stdout:9/738: chown d3/d2f/d2e/d33/d36/f37 355784633 1 2024-04-12T20:56:42.001 INFO:tasks.workunit.client.1.smithi186.stdout:9/739: fdatasync d3/d2f/d2e/d43/fa7 0 2024-04-12T20:56:42.005 INFO:tasks.workunit.client.1.smithi186.stdout:9/740: fdatasync d3/d2f/d2e/d33/fdd 0 2024-04-12T20:56:42.008 INFO:tasks.workunit.client.1.smithi186.stdout:9/741: fdatasync d3/d2f/d2e/d33/d61/f90 0 2024-04-12T20:56:42.012 INFO:tasks.workunit.client.1.smithi186.stdout:0/909: write d0/d10/d71/fb1 [3256411,105384] 0 2024-04-12T20:56:42.013 INFO:tasks.workunit.client.1.smithi186.stdout:5/802: mkdir d5/db/d15/d2a/d4c/d5b/d7a/d94/d106 0 2024-04-12T20:56:42.016 INFO:tasks.workunit.client.1.smithi186.stdout:1/818: getdents d14/d1a/d33 0 2024-04-12T20:56:42.024 INFO:tasks.workunit.client.1.smithi186.stdout:6/787: dread d9/d43/d59/d5d/f53 [0,4194304] 0 2024-04-12T20:56:42.024 INFO:tasks.workunit.client.1.smithi186.stdout:3/760: dwrite d1/d3/d21/d22/d34/ddb/f59 [0,4194304] 0 2024-04-12T20:56:42.024 INFO:tasks.workunit.client.1.smithi186.stdout:3/761: chown d1/d3/df8/l33 113522 1 2024-04-12T20:56:42.025 INFO:tasks.workunit.client.1.smithi186.stdout:8/823: dwrite d7/de/d10/f15 [0,4194304] 0 2024-04-12T20:56:42.028 INFO:tasks.workunit.client.1.smithi186.stdout:1/819: mknod d14/d1a/d1d/de5/c10c 0 2024-04-12T20:56:42.029 INFO:tasks.workunit.client.1.smithi186.stdout:9/742: chown d3/d2f/d54/d9f/dc4/d6b/cb8 1745 1 2024-04-12T20:56:42.030 INFO:tasks.workunit.client.1.smithi186.stdout:2/883: write d0/f7e [212914,8382] 0 2024-04-12T20:56:42.030 INFO:tasks.workunit.client.1.smithi186.stdout:6/788: unlink d9/d43/d59/d5d/d101/cc0 0 2024-04-12T20:56:42.033 INFO:tasks.workunit.client.1.smithi186.stdout:8/824: chown d7/c86 4356721 1 2024-04-12T20:56:42.040 INFO:tasks.workunit.client.1.smithi186.stdout:1/820: dread d14/d1a/d3c/f102 [0,4194304] 0 2024-04-12T20:56:42.041 INFO:tasks.workunit.client.1.smithi186.stdout:2/884: stat d0/d3/d1a/d21 0 2024-04-12T20:56:42.042 INFO:tasks.workunit.client.1.smithi186.stdout:6/789: creat d9/d5e/d7a/f117 x:0 0 0 2024-04-12T20:56:42.043 INFO:tasks.workunit.client.1.smithi186.stdout:8/825: readlink d7/d18/d36/d20/d23/l9a 0 2024-04-12T20:56:42.044 INFO:tasks.workunit.client.1.smithi186.stdout:7/775: dread d7/d10/d2d/f58 [0,4194304] 0 2024-04-12T20:56:42.047 INFO:tasks.workunit.client.1.smithi186.stdout:5/803: dwrite d5/db/ff8 [0,4194304] 0 2024-04-12T20:56:42.050 INFO:tasks.workunit.client.1.smithi186.stdout:9/743: symlink d3/d2f/d54/d9f/dc4/d77/dd6/l102 0 2024-04-12T20:56:42.050 INFO:tasks.workunit.client.1.smithi186.stdout:2/885: read d0/dc/f55 [87202,37039] 0 2024-04-12T20:56:42.054 INFO:tasks.workunit.client.1.smithi186.stdout:0/910: dwrite d0/d10/f43 [8388608,4194304] 0 2024-04-12T20:56:42.055 INFO:tasks.workunit.client.1.smithi186.stdout:7/776: chown d7/d10/d2d/d40/d4a/d6c/d97/d4d/d84/fe8 17767 1 2024-04-12T20:56:42.058 INFO:tasks.workunit.client.1.smithi186.stdout:3/762: dwrite d1/d3/d21/d22/d34/ddb/d4d/dad/fcf [0,4194304] 0 2024-04-12T20:56:42.062 INFO:tasks.workunit.client.1.smithi186.stdout:5/804: chown d5/db/d15/d2a/d4c/d5b/d7a/d94/dfa/fb5 154290 1 2024-04-12T20:56:42.066 INFO:tasks.workunit.client.1.smithi186.stdout:1/821: rename d14/d1a/d33/d36/d5b/d3e/c66 to d14/d1a/d33/d36/d5b/d105/c10d 0 2024-04-12T20:56:42.072 INFO:tasks.workunit.client.1.smithi186.stdout:3/763: rmdir d1/d3/df8/dd0 39 2024-04-12T20:56:42.072 INFO:tasks.workunit.client.1.smithi186.stdout:6/790: dwrite d9/da9/fee [0,4194304] 0 2024-04-12T20:56:42.073 INFO:tasks.workunit.client.1.smithi186.stdout:5/805: dread - d5/db/d15/d2a/d4c/d5b/d7a/d94/dfa/f7c zero size 2024-04-12T20:56:42.078 INFO:tasks.workunit.client.1.smithi186.stdout:2/886: write d0/d3/d7a/ffc [737033,116920] 0 2024-04-12T20:56:42.079 INFO:tasks.workunit.client.1.smithi186.stdout:3/764: fdatasync d1/d2c/d64/f9b 0 2024-04-12T20:56:42.086 INFO:tasks.workunit.client.1.smithi186.stdout:9/744: rename d3/d2f/d54/d9f/dc4/d77/dd6/d72/df4/d6f/d83 to d3/d2f/d103 0 2024-04-12T20:56:42.094 INFO:tasks.workunit.client.1.smithi186.stdout:9/745: write d3/d2f/d2e/d43/d4c/d82/dd4/fe4 [2087491,27341] 0 2024-04-12T20:56:42.096 INFO:tasks.workunit.client.1.smithi186.stdout:8/826: dwrite f5 [0,4194304] 0 2024-04-12T20:56:42.096 INFO:tasks.workunit.client.1.smithi186.stdout:3/765: mknod d1/d3/df8/ddc/cfa 0 2024-04-12T20:56:42.101 INFO:tasks.workunit.client.1.smithi186.stdout:9/746: chown d3/d2f/d2e/d43/ceb 15 1 2024-04-12T20:56:42.101 INFO:tasks.workunit.client.1.smithi186.stdout:7/777: rename d7/d55/d71/dbe/f103 to d7/d10/d2d/d40/df7/f115 0 2024-04-12T20:56:42.102 INFO:tasks.workunit.client.1.smithi186.stdout:2/887: getdents d0/d3/dce/ddf 0 2024-04-12T20:56:42.108 INFO:tasks.workunit.client.1.smithi186.stdout:8/827: symlink d7/d18/d36/d20/d23/dd6/de8/df7/l10b 0 2024-04-12T20:56:42.111 INFO:tasks.workunit.client.1.smithi186.stdout:9/747: mknod d3/d2f/d2e/d33/d61/c104 0 2024-04-12T20:56:42.121 INFO:tasks.workunit.client.1.smithi186.stdout:1/822: dread d14/d1a/d33/d36/d5b/d3e/d71/f82 [0,4194304] 0 2024-04-12T20:56:42.128 INFO:tasks.workunit.client.1.smithi186.stdout:5/806: write d5/db/f76 [1007655,38236] 0 2024-04-12T20:56:42.129 INFO:tasks.workunit.client.1.smithi186.stdout:9/748: mkdir d3/d2f/d2e/d33/d61/dbc/d105 0 2024-04-12T20:56:42.139 INFO:tasks.workunit.client.1.smithi186.stdout:7/778: mknod d7/d10/d2d/d40/d4a/d6c/d97/d109/c116 0 2024-04-12T20:56:42.139 INFO:tasks.workunit.client.1.smithi186.stdout:7/779: stat d7/f17 0 2024-04-12T20:56:42.139 INFO:tasks.workunit.client.1.smithi186.stdout:4/925: sync 2024-04-12T20:56:42.139 INFO:tasks.workunit.client.1.smithi186.stdout:6/791: rename d9/d5e/f71 to d9/da9/f118 0 2024-04-12T20:56:42.139 INFO:tasks.workunit.client.1.smithi186.stdout:8/828: mkdir d7/d18/d8e/d10c 0 2024-04-12T20:56:42.146 INFO:tasks.workunit.client.1.smithi186.stdout:6/792: fsync d9/d43/d59/d5d/d6e/d91/dd9/ff8 0 2024-04-12T20:56:42.147 INFO:tasks.workunit.client.1.smithi186.stdout:7/780: creat d7/d10/d2d/d40/d4a/d6c/d97/d109/f117 x:0 0 0 2024-04-12T20:56:42.149 INFO:tasks.workunit.client.1.smithi186.stdout:4/926: rename d0/dc/d88/l74 to d0/d59/dc0/d63/da6/dc6/l12b 0 2024-04-12T20:56:42.152 INFO:tasks.workunit.client.1.smithi186.stdout:6/793: read - d9/d43/d59/f9f zero size 2024-04-12T20:56:42.156 INFO:tasks.workunit.client.1.smithi186.stdout:1/823: dwrite d14/d1a/d33/d36/d5b/d6a/da2/f10a [0,4194304] 0 2024-04-12T20:56:42.156 INFO:tasks.workunit.client.1.smithi186.stdout:3/766: dread d1/d3/da/fe1 [0,4194304] 0 2024-04-12T20:56:42.157 INFO:tasks.workunit.client.1.smithi186.stdout:2/888: dread d0/dc/f5d [0,4194304] 0 2024-04-12T20:56:42.160 INFO:tasks.workunit.client.1.smithi186.stdout:9/749: dwrite d3/d2f/f68 [4194304,4194304] 0 2024-04-12T20:56:42.175 INFO:tasks.workunit.client.1.smithi186.stdout:4/927: link d0/d59/dc0/d63/c96 d0/dc/d33/d9a/c12c 0 2024-04-12T20:56:42.178 INFO:tasks.workunit.client.1.smithi186.stdout:7/781: dread d7/d10/d2d/d40/d4a/d6c/d97/d4d/f73 [0,4194304] 0 2024-04-12T20:56:42.179 INFO:tasks.workunit.client.1.smithi186.stdout:9/750: link d3/d2f/d54/d9f/dc4/d77/dd6/d72/fe1 d3/d2f/d54/d7d/f106 0 2024-04-12T20:56:42.186 INFO:tasks.workunit.client.1.smithi186.stdout:8/829: dread d7/de/d68/f9d [4194304,4194304] 0 2024-04-12T20:56:42.188 INFO:tasks.workunit.client.1.smithi186.stdout:7/782: creat d7/deb/f118 x:0 0 0 2024-04-12T20:56:42.189 INFO:tasks.workunit.client.1.smithi186.stdout:1/824: write d14/d1a/d3c/f55 [179304,52742] 0 2024-04-12T20:56:42.194 INFO:tasks.workunit.client.1.smithi186.stdout:8/830: unlink d7/d18/d36/d1d/d1f/d26/c77 0 2024-04-12T20:56:42.196 INFO:tasks.workunit.client.1.smithi186.stdout:7/783: symlink d7/d55/d71/dbe/l119 0 2024-04-12T20:56:42.198 INFO:tasks.workunit.client.1.smithi186.stdout:1/825: mkdir d14/d1a/d21/d35/d107/d10e 0 2024-04-12T20:56:42.199 INFO:tasks.workunit.client.1.smithi186.stdout:3/767: truncate d1/d3/d21/d22/f7b 800905 0 2024-04-12T20:56:42.204 INFO:tasks.workunit.client.1.smithi186.stdout:8/831: write d7/d18/d36/d1d/d1f/da2/df8/d29/f6a [3193096,13884] 0 2024-04-12T20:56:42.207 INFO:tasks.workunit.client.1.smithi186.stdout:8/832: write d7/de/d68/da7/dd5/ffa [964617,43469] 0 2024-04-12T20:56:42.207 INFO:tasks.workunit.client.1.smithi186.stdout:4/928: truncate d0/dc/d88/d80/fdc 3303647 0 2024-04-12T20:56:42.208 INFO:tasks.workunit.client.1.smithi186.stdout:9/751: dread d3/d2f/d103/f38 [4194304,4194304] 0 2024-04-12T20:56:42.209 INFO:tasks.workunit.client.1.smithi186.stdout:9/752: readlink d3/d2f/l2b 0 2024-04-12T20:56:42.211 INFO:tasks.workunit.client.1.smithi186.stdout:9/753: write d3/d2f/d2e/fd8 [1750907,120289] 0 2024-04-12T20:56:42.214 INFO:tasks.workunit.client.1.smithi186.stdout:8/833: chown d7/de/d10/f19 58731573 1 2024-04-12T20:56:42.216 INFO:tasks.workunit.client.1.smithi186.stdout:6/794: dread d9/f37 [0,4194304] 0 2024-04-12T20:56:42.216 INFO:tasks.workunit.client.1.smithi186.stdout:9/754: truncate d3/d2f/d54/d9f/dc4/ff2 939071 0 2024-04-12T20:56:42.220 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:56:42.216+0000 7f7dbe000700 1 -- 172.21.15.18:0/4095873283 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7db81033f0 msgr2=0x7f7db8103800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:56:42.220 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:56:42.216+0000 7f7dbe000700 1 --2- 172.21.15.18:0/4095873283 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7db81033f0 0x7f7db8103800 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f7da8004940 tx=0x7f7da8009e30 comp rx=0 tx=0).stop 2024-04-12T20:56:42.220 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:56:42.217+0000 7f7dbe000700 1 -- 172.21.15.18:0/4095873283 shutdown_connections 2024-04-12T20:56:42.220 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:56:42.217+0000 7f7dbe000700 1 --2- 172.21.15.18:0/4095873283 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7db81046a0 0x7f7db8104b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:56:42.220 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:56:42.217+0000 7f7dbe000700 1 --2- 172.21.15.18:0/4095873283 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7db81033f0 0x7f7db8103800 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:56:42.220 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:56:42.217+0000 7f7dbe000700 1 -- 172.21.15.18:0/4095873283 >> 172.21.15.18:0/4095873283 conn(0x7f7db8066d00 msgr2=0x7f7db8067120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T20:56:42.220 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:56:42.217+0000 7f7dbe000700 1 -- 172.21.15.18:0/4095873283 shutdown_connections 2024-04-12T20:56:42.220 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:56:42.217+0000 7f7dbe000700 1 -- 172.21.15.18:0/4095873283 wait complete. 2024-04-12T20:56:42.220 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:56:42.218+0000 7f7dbe000700 1 Processor -- start 2024-04-12T20:56:42.221 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:56:42.219+0000 7f7dbe000700 1 -- start start 2024-04-12T20:56:42.221 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:56:42.219+0000 7f7dbe000700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7db81033f0 0x7f7db806e3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T20:56:42.221 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:56:42.219+0000 7f7dbe000700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7db81046a0 0x7f7db806e8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T20:56:42.221 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:56:42.219+0000 7f7dbe000700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7db806fff0 con 0x7f7db81033f0 2024-04-12T20:56:42.221 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:56:42.219+0000 7f7dbe000700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7db806ee30 con 0x7f7db81046a0 2024-04-12T20:56:42.221 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:56:42.219+0000 7f7db7400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7db81033f0 0x7f7db806e3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T20:56:42.221 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:56:42.219+0000 7f7db6a00700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7db81046a0 0x7f7db806e8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T20:56:42.221 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:56:42.220+0000 7f7db6a00700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7db81046a0 0x7f7db806e8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.18:33138/0 (socket says 172.21.15.18:33138) 2024-04-12T20:56:42.221 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:56:42.220+0000 7f7db6a00700 1 -- 172.21.15.18:0/1345185769 learned_addr learned my addr 172.21.15.18:0/1345185769 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T20:56:42.221 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:56:42.220+0000 7f7db6a00700 1 -- 172.21.15.18:0/1345185769 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7db81033f0 msgr2=0x7f7db806e3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:56:42.221 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:56:42.220+0000 7f7db6a00700 1 --2- 172.21.15.18:0/1345185769 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7db81033f0 0x7f7db806e3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:56:42.221 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:56:42.220+0000 7f7db6a00700 1 -- 172.21.15.18:0/1345185769 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7da8005040 con 0x7f7db81046a0 2024-04-12T20:56:42.221 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:56:42.220+0000 7f7db7400700 1 --2- 172.21.15.18:0/1345185769 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7db81033f0 0x7f7db806e3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-12T20:56:42.221 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:56:42.220+0000 7f7db6a00700 1 --2- 172.21.15.18:0/1345185769 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7db81046a0 0x7f7db806e8f0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f7da400eb40 tx=0x7f7da400b040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T20:56:42.222 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:56:42.221+0000 7f7daf400700 1 -- 172.21.15.18:0/1345185769 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7da4007cc0 con 0x7f7db81046a0 2024-04-12T20:56:42.222 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:56:42.221+0000 7f7dbe000700 1 -- 172.21.15.18:0/1345185769 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7db806f0b0 con 0x7f7db81046a0 2024-04-12T20:56:42.222 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:56:42.221+0000 7f7dbe000700 1 -- 172.21.15.18:0/1345185769 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7db8122590 con 0x7f7db81046a0 2024-04-12T20:56:42.222 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:56:42.222+0000 7f7daf400700 1 -- 172.21.15.18:0/1345185769 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f7da400f040 con 0x7f7db81046a0 2024-04-12T20:56:42.222 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:56:42.222+0000 7f7daf400700 1 -- 172.21.15.18:0/1345185769 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7da400ca10 con 0x7f7db81046a0 2024-04-12T20:56:42.222 INFO:tasks.workunit.client.1.smithi186.stdout:3/768: creat d1/d3/d21/d22/d34/d6e/ffb x:0 0 0 2024-04-12T20:56:42.224 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:56:42.226+0000 7f7dbe000700 1 -- 172.21.15.18:0/1345185769 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7d980052e0 con 0x7f7db81046a0 2024-04-12T20:56:42.225 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:56:42.227+0000 7f7daf400700 1 -- 172.21.15.18:0/1345185769 <== mon.1 v2:172.21.15.186:3300/0 4 ==== mgrmap(e 25) v1 ==== 89542+0+0 (secure 0 0 0) 0x7f7da400cc80 con 0x7f7db81046a0 2024-04-12T20:56:42.226 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:56:42.228+0000 7f7daf400700 1 --2- 172.21.15.18:0/1345185769 >> [v2:172.21.15.186:6828/2316349902,v1:172.21.15.186:6829/2316349902] conn(0x7f7d9c06bc30 0x7f7d9c06e0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T20:56:42.226 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:56:42.228+0000 7f7daf400700 1 -- 172.21.15.18:0/1345185769 <== mon.1 v2:172.21.15.186:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5808+0+0 (secure 0 0 0) 0x7f7da408a440 con 0x7f7db81046a0 2024-04-12T20:56:42.226 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:56:42.229+0000 7f7db7400700 1 --2- 172.21.15.18:0/1345185769 >> [v2:172.21.15.186:6828/2316349902,v1:172.21.15.186:6829/2316349902] conn(0x7f7d9c06bc30 0x7f7d9c06e0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T20:56:42.228 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:56:42.230+0000 7f7db7400700 1 --2- 172.21.15.18:0/1345185769 >> [v2:172.21.15.186:6828/2316349902,v1:172.21.15.186:6829/2316349902] conn(0x7f7d9c06bc30 0x7f7d9c06e0e0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f7da8013870 tx=0x7f7da8009e30 comp rx=0 tx=0).ready entity=mgr.24359 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T20:56:42.231 INFO:tasks.workunit.client.1.smithi186.stdout:1/826: getdents d14/d1a/d33/d36/d5b/d105 0 2024-04-12T20:56:42.233 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:56:42.235+0000 7f7daf400700 1 -- 172.21.15.18:0/1345185769 <== mon.1 v2:172.21.15.186:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186454 (secure 0 0 0) 0x7f7da4050870 con 0x7f7db81046a0 2024-04-12T20:56:42.233 INFO:tasks.workunit.client.1.smithi186.stdout:7/784: dwrite d7/de1/f9d [0,4194304] 0 2024-04-12T20:56:42.234 INFO:tasks.workunit.client.1.smithi186.stdout:1/827: fsync d14/d1a/d1d/de5/ff9 0 2024-04-12T20:56:42.235 INFO:tasks.workunit.client.1.smithi186.stdout:6/795: mknod d9/c119 0 2024-04-12T20:56:42.237 INFO:tasks.workunit.client.1.smithi186.stdout:8/834: creat d7/d18/d36/d20/d23/dc1/dda/f10d x:0 0 0 2024-04-12T20:56:42.240 INFO:tasks.workunit.client.1.smithi186.stdout:8/835: chown d7/d18/d36/d20/d71/fc2 244 1 2024-04-12T20:56:42.255 INFO:tasks.workunit.client.1.smithi186.stdout:3/769: rename d1/d2c/d48/f5a to d1/d3/df8/ddc/ffc 0 2024-04-12T20:56:42.258 INFO:tasks.workunit.client.1.smithi186.stdout:1/828: fdatasync d14/d1a/d21/d35/dbc/ff2 0 2024-04-12T20:56:42.259 INFO:tasks.workunit.client.1.smithi186.stdout:6/796: dwrite d9/d4c/dbe/dfe/fd8 [0,4194304] 0 2024-04-12T20:56:42.261 INFO:tasks.workunit.client.1.smithi186.stdout:8/836: creat d7/de/d68/f10e x:0 0 0 2024-04-12T20:56:42.262 INFO:tasks.workunit.client.1.smithi186.stdout:9/755: link d3/d2f/d103/cc5 d3/d2f/d103/dd5/c107 0 2024-04-12T20:56:42.266 INFO:tasks.workunit.client.1.smithi186.stdout:8/837: chown d7/d18/d36/d1d/d1f/da2/df8/d29 0 1 2024-04-12T20:56:42.273 INFO:tasks.workunit.client.1.smithi186.stdout:1/829: mkdir d14/d1a/d33/d36/d5b/d3e/dbb/d10f 0 2024-04-12T20:56:42.275 INFO:tasks.workunit.client.1.smithi186.stdout:1/830: chown d14/d1a/d33/d36/d5b/d6a/f99 7613 1 2024-04-12T20:56:42.277 INFO:tasks.workunit.client.1.smithi186.stdout:7/785: creat d7/d10/d2d/d40/d4a/d6c/d97/f11a x:0 0 0 2024-04-12T20:56:42.278 INFO:tasks.workunit.client.1.smithi186.stdout:7/786: write d7/d55/ffd [57269,98726] 0 2024-04-12T20:56:42.281 INFO:tasks.workunit.client.1.smithi186.stdout:9/756: symlink d3/d2f/d2e/d33/l108 0 2024-04-12T20:56:42.283 INFO:tasks.workunit.client.1.smithi186.stdout:9/757: chown d3/d2f/d2e/d33/d36/f37 1 1 2024-04-12T20:56:42.284 INFO:tasks.workunit.client.1.smithi186.stdout:9/758: fdatasync d3/d2f/d2e/d43/fbd 0 2024-04-12T20:56:42.285 INFO:tasks.workunit.client.1.smithi186.stdout:6/797: mkdir d9/d43/d59/d116/d11a 0 2024-04-12T20:56:42.287 INFO:tasks.workunit.client.1.smithi186.stdout:1/831: write d14/d1a/d33/d36/d5b/d3e/d71/f82 [7785078,93915] 0 2024-04-12T20:56:42.288 INFO:tasks.workunit.client.1.smithi186.stdout:7/787: mkdir d7/d55/d71/d11b 0 2024-04-12T20:56:42.293 INFO:tasks.workunit.client.1.smithi186.stdout:8/838: link d7/d18/d36/d20/d23/dfd/fff d7/d18/d8e/d10c/f10f 0 2024-04-12T20:56:42.295 INFO:tasks.workunit.client.1.smithi186.stdout:9/759: creat d3/d2f/d54/d7d/f109 x:0 0 0 2024-04-12T20:56:42.299 INFO:tasks.workunit.client.1.smithi186.stdout:3/770: link d1/d3/d21/d22/da4/lba d1/d3/d21/d22/d34/df7/lfd 0 2024-04-12T20:56:42.301 INFO:tasks.workunit.client.1.smithi186.stdout:1/832: rename d14/f4f to d14/d1a/d58/d69/f110 0 2024-04-12T20:56:42.303 INFO:tasks.workunit.client.1.smithi186.stdout:7/788: dwrite d7/d10/d2d/d40/d4a/fa0 [0,4194304] 0 2024-04-12T20:56:42.304 INFO:tasks.workunit.client.1.smithi186.stdout:1/833: dread - d14/d1a/d33/d36/d5b/d3e/fd7 zero size 2024-04-12T20:56:42.304 INFO:tasks.workunit.client.1.smithi186.stdout:3/771: creat d1/d3/d21/d22/d24/d28/ffe x:0 0 0 2024-04-12T20:56:42.308 INFO:tasks.workunit.client.1.smithi186.stdout:7/789: mknod d7/d10/d1c/d108/c11c 0 2024-04-12T20:56:42.316 INFO:tasks.workunit.client.1.smithi186.stdout:3/772: creat d1/d3/d21/d22/d24/fff x:0 0 0 2024-04-12T20:56:42.322 INFO:tasks.workunit.client.1.smithi186.stdout:6/798: dwrite d9/d4c/d6f/f7b [0,4194304] 0 2024-04-12T20:56:42.322 INFO:tasks.workunit.client.1.smithi186.stdout:9/760: dread d3/d2f/d2e/d33/f3f [4194304,4194304] 0 2024-04-12T20:56:42.325 INFO:tasks.workunit.client.1.smithi186.stdout:5/807: sync 2024-04-12T20:56:42.325 INFO:tasks.workunit.client.1.smithi186.stdout:0/911: sync 2024-04-12T20:56:42.332 INFO:tasks.workunit.client.1.smithi186.stdout:6/799: rmdir d9/d43/d59/d5d/de6 39 2024-04-12T20:56:42.332 INFO:tasks.workunit.client.1.smithi186.stdout:7/790: dwrite d7/d10/d1c/d5d/ff9 [0,4194304] 0 2024-04-12T20:56:42.333 INFO:tasks.workunit.client.1.smithi186.stdout:9/761: write d3/d2f/d54/d9f/dc4/f5a [1504740,115021] 0 2024-04-12T20:56:42.337 INFO:tasks.workunit.client.1.smithi186.stdout:8/839: dwrite d7/f90 [0,4194304] 0 2024-04-12T20:56:42.339 INFO:tasks.workunit.client.1.smithi186.stdout:1/834: dread d14/d1a/d21/d35/fc4 [0,4194304] 0 2024-04-12T20:56:42.343 INFO:tasks.workunit.client.1.smithi186.stdout:6/800: rmdir d9/d43/d59 39 2024-04-12T20:56:42.350 INFO:tasks.workunit.client.1.smithi186.stdout:9/762: unlink d3/d2f/d2e/d33/d36/de5/ce2 0 2024-04-12T20:56:42.351 INFO:tasks.workunit.client.1.smithi186.stdout:7/791: mkdir d7/d10/d2d/d40/d4a/d6c/d97/d4d/d6a/d11d 0 2024-04-12T20:56:42.352 INFO:tasks.workunit.client.1.smithi186.stdout:9/763: chown d3/d2f/d54/d9f/lc0 14434077 1 2024-04-12T20:56:42.354 INFO:tasks.workunit.client.1.smithi186.stdout:1/835: symlink d14/d1a/d58/d69/l111 0 2024-04-12T20:56:42.354 INFO:tasks.workunit.client.1.smithi186.stdout:6/801: unlink d9/lc 0 2024-04-12T20:56:42.355 INFO:tasks.workunit.client.1.smithi186.stdout:5/808: truncate d5/db/d15/d2a/d4c/d5b/d7a/d94/dfa/f4b 583633 0 2024-04-12T20:56:42.359 INFO:tasks.workunit.client.1.smithi186.stdout:7/792: creat d7/d10/d2d/d40/d4a/d6c/d97/d4d/f11e x:0 0 0 2024-04-12T20:56:42.362 INFO:tasks.workunit.client.1.smithi186.stdout:1/836: symlink d14/d1a/d21/d35/dbc/de3/l112 0 2024-04-12T20:56:42.365 INFO:tasks.workunit.client.1.smithi186.stdout:1/837: write d14/d1a/d33/d36/d5b/d3e/d5f/f8f [3660575,93090] 0 2024-04-12T20:56:42.368 INFO:tasks.workunit.client.1.smithi186.stdout:9/764: dwrite d3/d2f/d54/d92/fef [0,4194304] 0 2024-04-12T20:56:42.368 INFO:tasks.workunit.client.1.smithi186.stdout:1/838: dread - d14/d1a/d33/d36/dfa/f101 zero size 2024-04-12T20:56:42.380 INFO:tasks.workunit.client.1.smithi186.stdout:8/840: getdents d7/d18/d36 0 2024-04-12T20:56:42.387 INFO:tasks.workunit.client.1.smithi186.stdout:5/809: dwrite d5/db/d15/f1e [0,4194304] 0 2024-04-12T20:56:42.402 INFO:tasks.workunit.client.1.smithi186.stdout:8/841: dread d7/d18/d36/d1d/d1f/d26/fb8 [4194304,4194304] 0 2024-04-12T20:56:42.404 INFO:tasks.workunit.client.1.smithi186.stdout:8/842: write d7/d18/d36/dc9/fb4 [746650,61325] 0 2024-04-12T20:56:42.406 INFO:tasks.workunit.client.1.smithi186.stdout:8/843: dread - d7/de/d10/d7f/ff4 zero size 2024-04-12T20:56:42.412 INFO:tasks.workunit.client.1.smithi186.stdout:9/765: dread d3/d2f/d54/d9f/dc4/d77/dd6/d72/df4/d6f/f75 [0,4194304] 0 2024-04-12T20:56:42.414 INFO:tasks.workunit.client.1.smithi186.stdout:7/793: symlink d7/d10/d2d/d40/d4a/d6c/d97/d4d/d6a/l11f 0 2024-04-12T20:56:42.416 INFO:tasks.workunit.client.1.smithi186.stdout:0/912: write d0/d10/d72/d79/fd0 [1226619,115542] 0 2024-04-12T20:56:42.429 INFO:tasks.workunit.client.1.smithi186.stdout:1/839: truncate d14/d1a/d21/f4b 400417 0 2024-04-12T20:56:42.434 INFO:tasks.workunit.client.1.smithi186.stdout:4/929: sync 2024-04-12T20:56:42.435 INFO:tasks.workunit.client.1.smithi186.stdout:2/889: sync 2024-04-12T20:56:42.436 INFO:tasks.workunit.client.1.smithi186.stdout:2/890: dread - d0/d3e/f113 zero size 2024-04-12T20:56:42.439 INFO:tasks.workunit.client.1.smithi186.stdout:8/844: creat d7/d18/d36/d20/d23/dc1/f110 x:0 0 0 2024-04-12T20:56:42.444 INFO:tasks.workunit.client.1.smithi186.stdout:8/845: write d7/de/d7a/f31 [4650190,9673] 0 2024-04-12T20:56:42.447 INFO:tasks.workunit.client.1.smithi186.stdout:1/840: dwrite d14/d1a/d33/d36/d5b/fc2 [0,4194304] 0 2024-04-12T20:56:42.450 INFO:tasks.workunit.client.1.smithi186.stdout:0/913: fsync d0/d5c/d65/fbe 0 2024-04-12T20:56:42.454 INFO:tasks.workunit.client.1.smithi186.stdout:6/802: dwrite d9/da9/f118 [8388608,4194304] 0 2024-04-12T20:56:42.460 INFO:tasks.workunit.client.1.smithi186.stdout:2/891: mknod d0/dc/d2e/d73/c11d 0 2024-04-12T20:56:42.460 INFO:tasks.workunit.client.1.smithi186.stdout:4/930: truncate d0/dc/d64/f89 825142 0 2024-04-12T20:56:42.471 INFO:tasks.workunit.client.1.smithi186.stdout:1/841: rename d14/d74/f9a to d14/d1a/d33/d36/d5b/d3e/dbb/f113 0 2024-04-12T20:56:42.473 INFO:tasks.workunit.client.1.smithi186.stdout:5/810: dread d5/db/d71/f52 [0,4194304] 0 2024-04-12T20:56:42.476 INFO:tasks.workunit.client.1.smithi186.stdout:9/766: creat d3/d2f/f10a x:0 0 0 2024-04-12T20:56:42.476 INFO:tasks.workunit.client.1.smithi186.stdout:7/794: write d7/d10/d2d/d67/f70 [3539737,28815] 0 2024-04-12T20:56:42.477 INFO:tasks.workunit.client.1.smithi186.stdout:2/892: rmdir d0/d3 39 2024-04-12T20:56:42.477 INFO:tasks.workunit.client.1.smithi186.stdout:4/931: truncate d0/d26/f7c 502261 0 2024-04-12T20:56:42.479 INFO:tasks.workunit.client.1.smithi186.stdout:1/842: truncate d14/d1a/d33/d36/dfa/f101 522301 0 2024-04-12T20:56:42.485 INFO:tasks.workunit.client.1.smithi186.stdout:8/846: link d7/d18/d36/d1d/d7d/fb5 d7/d18/d36/d1d/d7d/f111 0 2024-04-12T20:56:42.486 INFO:tasks.workunit.client.1.smithi186.stdout:7/795: chown d7/d10/c24 222 1 2024-04-12T20:56:42.488 INFO:tasks.workunit.client.1.smithi186.stdout:1/843: dread - d14/d1a/d33/d36/d5b/d3e/d8b/ff0 zero size 2024-04-12T20:56:42.491 INFO:tasks.workunit.client.1.smithi186.stdout:5/811: truncate d5/db/d15/d2a/d2d/d46/f6c 154384 0 2024-04-12T20:56:42.492 INFO:tasks.workunit.client.1.smithi186.stdout:0/914: dwrite d0/d14/d4e/d1e/d84/f97 [4194304,4194304] 0 2024-04-12T20:56:42.493 INFO:tasks.workunit.client.1.smithi186.stdout:2/893: dread d0/d3/d1a/d41/fa7 [0,4194304] 0 2024-04-12T20:56:42.494 INFO:tasks.workunit.client.1.smithi186.stdout:5/812: readlink d5/db/d15/d6a/l79 0 2024-04-12T20:56:42.502 INFO:tasks.workunit.client.1.smithi186.stdout:9/767: rename d3/d2f/d54/d9f/dc4/d77/dc3/daf to d3/d13/d10b 0 2024-04-12T20:56:42.503 INFO:tasks.workunit.client.1.smithi186.stdout:8/847: symlink d7/de/d10/d7f/dd3/l112 0 2024-04-12T20:56:42.505 INFO:tasks.workunit.client.1.smithi186.stdout:8/848: fsync d7/d18/d36/d1d/d1f/d26/de6/f107 0 2024-04-12T20:56:42.514 INFO:tasks.workunit.client.1.smithi186.stdout:2/894: dread d0/d32/f97 [0,4194304] 0 2024-04-12T20:56:42.516 INFO:tasks.workunit.client.1.smithi186.stdout:0/915: mkdir d0/d14/d4e/d50/d6e/d96/d12a 0 2024-04-12T20:56:42.517 INFO:tasks.workunit.client.1.smithi186.stdout:0/916: dread - d0/d14/d4e/d50/fdf zero size 2024-04-12T20:56:42.519 INFO:tasks.workunit.client.1.smithi186.stdout:0/917: write d0/d10/d32/dac/fb6 [2092291,113530] 0 2024-04-12T20:56:42.522 INFO:tasks.workunit.client.1.smithi186.stdout:9/768: mknod d3/d2f/d2e/d33/d36/de5/c10c 0 2024-04-12T20:56:42.523 INFO:tasks.workunit.client.1.smithi186.stdout:4/932: write d0/d21/f27 [1622428,40496] 0 2024-04-12T20:56:42.524 INFO:tasks.workunit.client.1.smithi186.stdout:9/769: readlink d3/d2f/d54/d9f/dc4/d77/dd6/l102 0 2024-04-12T20:56:42.526 INFO:tasks.workunit.client.1.smithi186.stdout:8/849: fsync d7/d18/d36/d20/d23/f7e 0 2024-04-12T20:56:42.526 INFO:tasks.workunit.client.1.smithi186.stdout:4/933: truncate d0/db4/dd8/d104/f117 265643 0 2024-04-12T20:56:42.527 INFO:tasks.workunit.client.1.smithi186.stdout:7/796: mkdir d7/d10/d2d/d40/d4a/d6c/d97/d4d/d6a/d11d/d120 0 2024-04-12T20:56:42.528 INFO:tasks.workunit.client.1.smithi186.stdout:2/895: fdatasync d0/d3/d1a/d41/fa7 0 2024-04-12T20:56:42.530 INFO:tasks.workunit.client.1.smithi186.stdout:2/896: readlink d0/d3/d1a/d24/l9c 0 2024-04-12T20:56:42.536 INFO:tasks.workunit.client.1.smithi186.stdout:3/773: sync 2024-04-12T20:56:42.537 INFO:tasks.workunit.client.1.smithi186.stdout:0/918: symlink d0/d10/d32/dac/l12b 0 2024-04-12T20:56:42.539 INFO:tasks.workunit.client.1.smithi186.stdout:0/919: chown d0/d5c/d66/da7/dfc 1585 1 2024-04-12T20:56:42.541 INFO:tasks.workunit.client.1.smithi186.stdout:7/797: dwrite d7/d10/d2d/d40/d4a/d6c/d97/d4d/f11e [0,4194304] 0 2024-04-12T20:56:42.545 INFO:tasks.workunit.client.1.smithi186.stdout:4/934: rmdir d0/d59 39 2024-04-12T20:56:42.553 INFO:tasks.workunit.client.1.smithi186.stdout:8/850: read d7/d18/d36/d1d/d1f/da2/d5c/f7b [2304610,130935] 0 2024-04-12T20:56:42.555 INFO:tasks.workunit.client.1.smithi186.stdout:7/798: dwrite d7/de1/f9d [0,4194304] 0 2024-04-12T20:56:42.557 INFO:tasks.workunit.client.1.smithi186.stdout:7/799: stat d7/d10/d2d/d40/d4a/d6c/d97/dc2 0 2024-04-12T20:56:42.560 INFO:tasks.workunit.client.1.smithi186.stdout:5/813: rename d5/db/d15/d2a/le7 to d5/l107 0 2024-04-12T20:56:42.566 INFO:tasks.workunit.client.1.smithi186.stdout:1/844: getdents d14/d74 0 2024-04-12T20:56:42.574 INFO:tasks.workunit.client.1.smithi186.stdout:2/897: dwrite d0/d3/d1a/d24/f33 [4194304,4194304] 0 2024-04-12T20:56:42.576 INFO:tasks.workunit.client.1.smithi186.stdout:7/800: chown d7/d10/d1c/c4f 13711 1 2024-04-12T20:56:42.578 INFO:tasks.workunit.client.1.smithi186.stdout:0/920: rename d0/d14/d105/c109 to d0/d5c/d66/c12c 0 2024-04-12T20:56:42.581 INFO:tasks.workunit.client.1.smithi186.stdout:6/803: sync 2024-04-12T20:56:42.581 INFO:tasks.workunit.client.1.smithi186.stdout:1/845: creat d14/d1a/d33/f114 x:0 0 0 2024-04-12T20:56:42.585 INFO:tasks.workunit.client.1.smithi186.stdout:8/851: creat d7/d18/d8e/d10c/f113 x:0 0 0 2024-04-12T20:56:42.587 INFO:tasks.workunit.client.1.smithi186.stdout:2/898: unlink d0/d3/c96 0 2024-04-12T20:56:42.587 INFO:tasks.workunit.client.1.smithi186.stdout:8/852: stat d7/d18/d36/d1d/d7d/da5/cf6 0 2024-04-12T20:56:42.587 INFO:tasks.workunit.client.1.smithi186.stdout:0/921: read d0/d10/d32/dac/fef [2525994,130124] 0 2024-04-12T20:56:42.588 INFO:tasks.workunit.client.1.smithi186.stdout:5/814: creat d5/db/d2e/d70/de2/f108 x:0 0 0 2024-04-12T20:56:42.589 INFO:tasks.workunit.client.1.smithi186.stdout:7/801: creat d7/d10/d1c/d5d/db7/f121 x:0 0 0 2024-04-12T20:56:42.591 INFO:tasks.workunit.client.1.smithi186.stdout:8/853: chown d7/d18/d36/c24 100205255 1 2024-04-12T20:56:42.591 INFO:tasks.workunit.client.1.smithi186.stdout:9/770: getdents d3/d2f/d54/d92 0 2024-04-12T20:56:42.592 INFO:tasks.workunit.client.1.smithi186.stdout:4/935: creat d0/dc/f12d x:0 0 0 2024-04-12T20:56:42.593 INFO:tasks.workunit.client.1.smithi186.stdout:2/899: write d0/d3/d1a/d41/d8f/de9/ffd [5102798,65063] 0 2024-04-12T20:56:42.596 INFO:tasks.workunit.client.1.smithi186.stdout:8/854: read - d7/d18/d36/d20/d23/fe4 zero size 2024-04-12T20:56:42.597 INFO:tasks.workunit.client.1.smithi186.stdout:1/846: creat d14/d1a/d21/d35/dbc/de3/f115 x:0 0 0 2024-04-12T20:56:42.597 INFO:tasks.workunit.client.1.smithi186.stdout:0/922: fsync d0/d14/d4e/f33 0 2024-04-12T20:56:42.599 INFO:tasks.workunit.client.1.smithi186.stdout:5/815: fsync d5/db/fbd 0 2024-04-12T20:56:42.599 INFO:tasks.workunit.client.1.smithi186.stdout:3/774: dread d1/d3/d21/f9a [0,4194304] 0 2024-04-12T20:56:42.601 INFO:tasks.workunit.client.1.smithi186.stdout:2/900: creat d0/d32/d8b/f11e x:0 0 0 2024-04-12T20:56:42.602 INFO:tasks.workunit.client.1.smithi186.stdout:5/816: chown d5/db/d2e/d70/d101 22065092 1 2024-04-12T20:56:42.602 INFO:tasks.workunit.client.1.smithi186.stdout:9/771: write d3/d2f/d54/d9f/dc4/d77/dd6/d72/df4/fcf [67480,105577] 0 2024-04-12T20:56:42.607 INFO:tasks.workunit.client.1.smithi186.stdout:3/775: fdatasync d1/d3/d21/d22/d34/ddb/d4d/dad/fcf 0 2024-04-12T20:56:42.607 INFO:tasks.workunit.client.1.smithi186.stdout:9/772: dread - d3/d2f/d2e/d33/d61/dbc/fd3 zero size 2024-04-12T20:56:42.610 INFO:tasks.workunit.client.1.smithi186.stdout:9/773: chown d3/d2f/d2e/db3/le6 29 1 2024-04-12T20:56:42.612 INFO:tasks.workunit.client.1.smithi186.stdout:7/802: dwrite d7/d10/d2d/d40/d4a/ff2 [0,4194304] 0 2024-04-12T20:56:42.613 INFO:tasks.workunit.client.1.smithi186.stdout:4/936: truncate d0/dc/d33/f6b 994773 0 2024-04-12T20:56:42.614 INFO:tasks.workunit.client.1.smithi186.stdout:7/803: chown d7/d10/d2d/f89 465 1 2024-04-12T20:56:42.619 INFO:tasks.workunit.client.1.smithi186.stdout:6/804: dread d9/d43/d2d/d4d/fe0 [0,4194304] 0 2024-04-12T20:56:42.621 INFO:tasks.workunit.client.1.smithi186.stdout:2/901: creat d0/d3/d1a/d21/d26/d86/f11f x:0 0 0 2024-04-12T20:56:42.622 INFO:tasks.workunit.client.1.smithi186.stdout:8/855: fsync d7/d18/d36/d20/d71/fc0 0 2024-04-12T20:56:42.626 INFO:tasks.workunit.client.1.smithi186.stdout:0/923: symlink d0/d14/d4e/d50/d6e/d96/d12a/l12d 0 2024-04-12T20:56:42.628 INFO:tasks.workunit.client.1.smithi186.stdout:3/776: creat d1/d3/d21/d22/d34/ddb/d4d/d6f/da6/f100 x:0 0 0 2024-04-12T20:56:42.629 INFO:tasks.workunit.client.1.smithi186.stdout:3/777: readlink d1/d2c/d48/l85 0 2024-04-12T20:56:42.632 INFO:tasks.workunit.client.1.smithi186.stdout:9/774: creat d3/d2f/d2e/d33/d61/f10d x:0 0 0 2024-04-12T20:56:42.635 INFO:tasks.workunit.client.1.smithi186.stdout:4/937: creat d0/dc/d64/d87/d98/f12e x:0 0 0 2024-04-12T20:56:42.637 INFO:tasks.workunit.client.1.smithi186.stdout:5/817: mknod d5/db/d15/d2a/d2d/d46/d55/c109 0 2024-04-12T20:56:42.642 INFO:tasks.workunit.client.1.smithi186.stdout:7/804: mknod d7/d10/d2d/c122 0 2024-04-12T20:56:42.644 INFO:tasks.workunit.client.1.smithi186.stdout:4/938: write d0/f24 [4348120,9495] 0 2024-04-12T20:56:42.644 INFO:tasks.workunit.client.1.smithi186.stdout:9/775: truncate d3/d2f/d54/d9f/dc4/d77/dd6/d72/fbf 3885522 0 2024-04-12T20:56:42.647 INFO:tasks.workunit.client.1.smithi186.stdout:7/805: dread - d7/d10/d2d/d40/df7/f115 zero size 2024-04-12T20:56:42.647 INFO:tasks.workunit.client.1.smithi186.stdout:9/776: fdatasync d3/d2f/d2e/d43/d4c/f58 0 2024-04-12T20:56:42.650 INFO:tasks.workunit.client.1.smithi186.stdout:0/924: creat d0/d85/dc4/f12e x:0 0 0 2024-04-12T20:56:42.652 INFO:tasks.workunit.client.1.smithi186.stdout:0/925: write d0/d14/f35 [1199856,130036] 0 2024-04-12T20:56:42.688 INFO:tasks.workunit.client.1.smithi186.stdout:9/777: write d3/d2f/d54/d9f/dc4/d77/dd6/d72/df4/d6f/f8c [1944255,118549] 0 2024-04-12T20:56:42.689 INFO:tasks.workunit.client.1.smithi186.stdout:1/847: truncate d14/d1a/d21/d35/fc4 1439205 0 2024-04-12T20:56:42.692 INFO:tasks.workunit.client.1.smithi186.stdout:5/818: dread d5/db/d15/d2a/d4c/f62 [0,4194304] 0 2024-04-12T20:56:42.693 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:56:42.695+0000 7f7dbe000700 1 -- 172.21.15.18:0/1345185769 --> [v2:172.21.15.186:6828/2316349902,v1:172.21.15.186:6829/2316349902] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f7d98000bf0 con 0x7f7d9c06bc30 2024-04-12T20:56:42.695 INFO:tasks.workunit.client.1.smithi186.stdout:1/848: write d14/d1a/d21/f31 [1142983,51633] 0 2024-04-12T20:56:42.695 INFO:tasks.workunit.client.1.smithi186.stdout:6/805: dread d9/d43/d2d/d4d/f64 [0,4194304] 0 2024-04-12T20:56:42.696 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:56:42.698+0000 7f7daf400700 1 -- 172.21.15.18:0/1345185769 <== mgr.24359 v2:172.21.15.186:6828/2316349902 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+341 (secure 0 0 0) 0x7f7d98000bf0 con 0x7f7d9c06bc30 2024-04-12T20:56:42.697 INFO:tasks.workunit.client.1.smithi186.stdout:2/902: dread d0/dc/d2e/d73/fd9 [0,4194304] 0 2024-04-12T20:56:42.698 INFO:tasks.workunit.client.1.smithi186.stdout:5/819: chown d5/db/d15/d2a/d4c/d5b/d7a/d94/lf9 10751 1 2024-04-12T20:56:42.701 INFO:tasks.workunit.client.1.smithi186.stdout:8/856: sync 2024-04-12T20:56:42.701 INFO:tasks.workunit.client.1.smithi186.stdout:3/778: sync 2024-04-12T20:56:42.701 INFO:tasks.workunit.client.1.smithi186.stdout:7/806: creat d7/d10/d2d/d40/d4a/d6c/d97/f123 x:0 0 0 2024-04-12T20:56:42.702 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:56:42.702+0000 7f7dbe000700 1 -- 172.21.15.18:0/1345185769 >> [v2:172.21.15.186:6828/2316349902,v1:172.21.15.186:6829/2316349902] conn(0x7f7d9c06bc30 msgr2=0x7f7d9c06e0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:56:42.702 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:56:42.702+0000 7f7dbe000700 1 --2- 172.21.15.18:0/1345185769 >> [v2:172.21.15.186:6828/2316349902,v1:172.21.15.186:6829/2316349902] conn(0x7f7d9c06bc30 0x7f7d9c06e0e0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f7da8013870 tx=0x7f7da8009e30 comp rx=0 tx=0).stop 2024-04-12T20:56:42.702 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:56:42.702+0000 7f7dbe000700 1 -- 172.21.15.18:0/1345185769 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7db81046a0 msgr2=0x7f7db806e8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:56:42.702 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:56:42.702+0000 7f7dbe000700 1 --2- 172.21.15.18:0/1345185769 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7db81046a0 0x7f7db806e8f0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f7da400eb40 tx=0x7f7da400b040 comp rx=0 tx=0).stop 2024-04-12T20:56:42.702 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:56:42.703+0000 7f7dbe000700 1 -- 172.21.15.18:0/1345185769 shutdown_connections 2024-04-12T20:56:42.702 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:56:42.703+0000 7f7dbe000700 1 --2- 172.21.15.18:0/1345185769 >> [v2:172.21.15.186:6828/2316349902,v1:172.21.15.186:6829/2316349902] conn(0x7f7d9c06bc30 0x7f7d9c06e0e0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:56:42.702 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:56:42.703+0000 7f7dbe000700 1 --2- 172.21.15.18:0/1345185769 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7db81033f0 0x7f7db806e3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:56:42.702 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:56:42.703+0000 7f7dbe000700 1 --2- 172.21.15.18:0/1345185769 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7db81046a0 0x7f7db806e8f0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:56:42.702 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:56:42.703+0000 7f7dbe000700 1 -- 172.21.15.18:0/1345185769 >> 172.21.15.18:0/1345185769 conn(0x7f7db8066d00 msgr2=0x7f7db8075130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T20:56:42.702 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:56:42.703+0000 7f7dbe000700 1 -- 172.21.15.18:0/1345185769 shutdown_connections 2024-04-12T20:56:42.702 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:56:42.703+0000 7f7dbe000700 1 -- 172.21.15.18:0/1345185769 wait complete. 2024-04-12T20:56:42.703 INFO:tasks.workunit.client.1.smithi186.stdout:8/857: write d7/de/d7a/fd9 [821280,110779] 0 2024-04-12T20:56:42.705 INFO:tasks.workunit.client.1.smithi186.stdout:6/806: symlink d9/d5e/d73/l11b 0 2024-04-12T20:56:42.706 INFO:tasks.workunit.client.1.smithi186.stdout:8/858: chown d7/d18/d36/d1d/dc3 1 1 2024-04-12T20:56:42.707 INFO:tasks.workunit.client.1.smithi186.stdout:6/807: chown d9/d43/l29 13416 1 2024-04-12T20:56:42.708 INFO:tasks.workunit.client.1.smithi186.stdout:5/820: chown d5/db/d15/d2a/d4c/l56 1184772110 1 2024-04-12T20:56:42.710 INFO:tasks.workunit.client.1.smithi186.stdout:6/808: fsync d9/d43/d2d/f7e 0 2024-04-12T20:56:42.713 INFO:tasks.workunit.client.1.smithi186.stdout:0/926: dread d0/d10/f2a [0,4194304] 0 2024-04-12T20:56:42.727 INFO:tasks.workunit.client.1.smithi186.stdout:5/821: symlink d5/db/d15/d2a/d4c/l10a 0 2024-04-12T20:56:42.728 INFO:teuthology.orchestra.run.smithi018.stdout:true 2024-04-12T20:56:42.730 INFO:tasks.workunit.client.1.smithi186.stdout:0/927: creat d0/d14/d105/f12f x:0 0 0 2024-04-12T20:56:42.731 INFO:tasks.workunit.client.1.smithi186.stdout:7/807: dread d7/fc [0,4194304] 0 2024-04-12T20:56:42.735 INFO:tasks.workunit.client.1.smithi186.stdout:2/903: creat d0/dc/f120 x:0 0 0 2024-04-12T20:56:42.740 INFO:tasks.workunit.client.1.smithi186.stdout:1/849: mknod d14/d1a/d33/d36/d5b/c116 0 2024-04-12T20:56:42.743 INFO:tasks.workunit.client.1.smithi186.stdout:7/808: mkdir d7/d10/d2d/d40/d4a/d6c/d97/d109/d124 0 2024-04-12T20:56:42.744 INFO:tasks.workunit.client.1.smithi186.stdout:6/809: symlink d9/d43/d59/d5d/l11c 0 2024-04-12T20:56:42.746 INFO:tasks.workunit.client.1.smithi186.stdout:2/904: mkdir d0/d3/d1a/d48/df7/d121 0 2024-04-12T20:56:42.748 INFO:tasks.workunit.client.1.smithi186.stdout:5/822: creat d5/db/d2e/d70/d101/d7f/d80/f10b x:0 0 0 2024-04-12T20:56:42.749 INFO:tasks.workunit.client.1.smithi186.stdout:0/928: symlink d0/d10/d71/d121/l130 0 2024-04-12T20:56:42.750 INFO:tasks.workunit.client.1.smithi186.stdout:0/929: fdatasync d0/d5c/d65/f7b 0 2024-04-12T20:56:42.751 INFO:tasks.workunit.client.1.smithi186.stdout:1/850: mknod d14/d1a/d21/d35/dbc/c117 0 2024-04-12T20:56:42.753 INFO:tasks.workunit.client.1.smithi186.stdout:4/939: sync 2024-04-12T20:56:42.754 INFO:tasks.workunit.client.1.smithi186.stdout:1/851: write d14/d1a/d33/d36/d5b/f79 [4885332,79360] 0 2024-04-12T20:56:42.757 INFO:tasks.workunit.client.1.smithi186.stdout:6/810: rename d9/d4c/d6f/d97/fbc to d9/d4c/de9/f11d 0 2024-04-12T20:56:42.758 INFO:tasks.workunit.client.1.smithi186.stdout:1/852: write d14/f15 [653069,824] 0 2024-04-12T20:56:42.759 INFO:tasks.workunit.client.1.smithi186.stdout:2/905: creat d0/d3/d1a/d41/d8f/f122 x:0 0 0 2024-04-12T20:56:42.761 INFO:tasks.workunit.client.1.smithi186.stdout:2/906: chown d0/d3/d1a/d48/df7/d111 2278 1 2024-04-12T20:56:42.765 INFO:tasks.workunit.client.1.smithi186.stdout:0/930: readlink d0/d14/d4e/d1e/l59 0 2024-04-12T20:56:42.767 INFO:tasks.workunit.client.1.smithi186.stdout:0/931: write d0/d10/f4d [3623901,52999] 0 2024-04-12T20:56:42.772 INFO:tasks.workunit.client.1.smithi186.stdout:1/853: readlink d14/d1a/d33/d36/l5e 0 2024-04-12T20:56:42.774 INFO:tasks.workunit.client.1.smithi186.stdout:5/823: symlink d5/db/d2e/d70/d101/d7f/l10c 0 2024-04-12T20:56:42.776 INFO:tasks.workunit.client.1.smithi186.stdout:2/907: fdatasync d0/d3/faf 0 2024-04-12T20:56:42.778 INFO:tasks.workunit.client.1.smithi186.stdout:7/809: creat d7/d10/f125 x:0 0 0 2024-04-12T20:56:42.782 INFO:tasks.workunit.client.1.smithi186.stdout:3/779: write d1/d2c/d48/fab [1327924,63496] 0 2024-04-12T20:56:42.783 INFO:tasks.workunit.client.1.smithi186.stdout:2/908: truncate d0/d3/d1a/d21/fdc 695250 0 2024-04-12T20:56:42.789 INFO:tasks.workunit.client.1.smithi186.stdout:5/824: unlink d5/d27/l85 0 2024-04-12T20:56:42.790 INFO:tasks.workunit.client.1.smithi186.stdout:8/859: dwrite d7/d18/d8e/d10c/f10f [0,4194304] 0 2024-04-12T20:56:42.791 INFO:tasks.workunit.client.1.smithi186.stdout:1/854: rmdir d14/d1a/d21/d35 39 2024-04-12T20:56:42.794 INFO:tasks.workunit.client.1.smithi186.stdout:9/778: sync 2024-04-12T20:56:42.800 INFO:tasks.workunit.client.1.smithi186.stdout:7/810: symlink d7/d55/d71/d10a/l126 0 2024-04-12T20:56:42.802 INFO:tasks.workunit.client.1.smithi186.stdout:8/860: readlink d7/d18/d36/d20/ld1 0 2024-04-12T20:56:42.802 INFO:tasks.workunit.client.1.smithi186.stdout:6/811: getdents d9/d4c/dbe/dfe 0 2024-04-12T20:56:42.804 INFO:tasks.workunit.client.1.smithi186.stdout:8/861: chown d7/d18/d36/c24 7789 1 2024-04-12T20:56:42.806 INFO:tasks.workunit.client.1.smithi186.stdout:5/825: symlink d5/db/d2e/d70/d101/d7f/d80/l10d 0 2024-04-12T20:56:42.807 INFO:tasks.workunit.client.1.smithi186.stdout:3/780: mkdir d1/d101 0 2024-04-12T20:56:42.808 INFO:tasks.workunit.client.1.smithi186.stdout:8/862: chown d7/d18/d36/d1d/d7d/lb7 0 1 2024-04-12T20:56:42.810 INFO:tasks.workunit.client.1.smithi186.stdout:9/779: unlink d3/d2f/d2e/d43/fa7 0 2024-04-12T20:56:42.811 INFO:tasks.workunit.client.1.smithi186.stdout:4/940: truncate d0/dc/ff1 1685410 0 2024-04-12T20:56:42.812 INFO:tasks.workunit.client.1.smithi186.stdout:2/909: symlink d0/dc/d2e/de8/l123 0 2024-04-12T20:56:42.813 INFO:tasks.workunit.client.1.smithi186.stdout:9/780: chown d3/d2f/d2e/d33/d36/de5 0 1 2024-04-12T20:56:42.814 INFO:tasks.workunit.client.1.smithi186.stdout:7/811: mkdir d7/d127 0 2024-04-12T20:56:42.818 INFO:tasks.workunit.client.1.smithi186.stdout:5/826: mkdir d5/db/d33/d10e 0 2024-04-12T20:56:42.820 INFO:tasks.workunit.client.1.smithi186.stdout:0/932: truncate d0/d10/d72/d79/fd0 3118639 0 2024-04-12T20:56:42.825 INFO:tasks.workunit.client.1.smithi186.stdout:9/781: chown d3/f73 24 1 2024-04-12T20:56:42.827 INFO:tasks.workunit.client.1.smithi186.stdout:4/941: creat d0/dc/df2/d118/f12f x:0 0 0 2024-04-12T20:56:42.830 INFO:tasks.workunit.client.1.smithi186.stdout:7/812: symlink d7/d10/d2d/d40/d4a/l128 0 2024-04-12T20:56:42.832 INFO:tasks.workunit.client.1.smithi186.stdout:0/933: write d0/d5d/ddc/fe0 [958308,23027] 0 2024-04-12T20:56:42.833 INFO:tasks.workunit.client.1.smithi186.stdout:3/781: link d1/d3/da2/fe9 d1/d2c/d48/f102 0 2024-04-12T20:56:42.836 INFO:tasks.workunit.client.1.smithi186.stdout:9/782: rmdir d3/d2f/d2e/d43/d4c 39 2024-04-12T20:56:42.837 INFO:tasks.workunit.client.1.smithi186.stdout:0/934: chown d0/d14/l8a 3429861 1 2024-04-12T20:56:42.838 INFO:tasks.workunit.client.1.smithi186.stdout:4/942: creat d0/d21/db9/f130 x:0 0 0 2024-04-12T20:56:42.839 INFO:tasks.workunit.client.1.smithi186.stdout:9/783: fsync d3/d2f/d2e/d33/fdd 0 2024-04-12T20:56:42.839 INFO:tasks.workunit.client.1.smithi186.stdout:1/855: dwrite d14/d91/f108 [0,4194304] 0 2024-04-12T20:56:42.840 INFO:tasks.workunit.client.1.smithi186.stdout:0/935: fsync d0/d14/d4e/d1e/f4b 0 2024-04-12T20:56:42.845 INFO:tasks.workunit.client.1.smithi186.stdout:5/827: fsync d5/db/fec 0 2024-04-12T20:56:42.845 INFO:tasks.workunit.client.1.smithi186.stdout:7/813: mkdir d7/deb/d129 0 2024-04-12T20:56:42.855 INFO:tasks.workunit.client.1.smithi186.stdout:3/782: dwrite d1/d3/da2/f9d [0,4194304] 0 2024-04-12T20:56:42.858 INFO:tasks.workunit.client.1.smithi186.stdout:0/936: unlink d0/d85/dc4/dc1/c112 0 2024-04-12T20:56:42.861 INFO:tasks.workunit.client.1.smithi186.stdout:1/856: write d14/d1a/d21/d35/ff6 [467479,5557] 0 2024-04-12T20:56:42.862 INFO:tasks.workunit.client.1.smithi186.stdout:6/812: dwrite d9/d43/f68 [0,4194304] 0 2024-04-12T20:56:42.862 INFO:tasks.workunit.client.1.smithi186.stdout:8/863: link d7/d18/d36/d1d/d1f/d26/c52 d7/d18/d36/d1d/d1f/c114 0 2024-04-12T20:56:42.867 INFO:tasks.workunit.client.1.smithi186.stdout:7/814: symlink d7/d10/d2d/d40/d4a/d6c/d97/dc2/l12a 0 2024-04-12T20:56:42.867 INFO:tasks.workunit.client.1.smithi186.stdout:5/828: dwrite d5/db/d15/d2a/d4c/d93/f98 [0,4194304] 0 2024-04-12T20:56:42.868 INFO:tasks.workunit.client.1.smithi186.stdout:9/784: mkdir d3/d2f/d10e 0 2024-04-12T20:56:42.874 INFO:tasks.workunit.client.1.smithi186.stdout:4/943: rename d0/d26/f2b to d0/d59/dc0/d63/d9b/db2/f131 0 2024-04-12T20:56:42.874 INFO:tasks.workunit.client.1.smithi186.stdout:7/815: truncate d7/d10/d2d/d40/d4a/d6c/d97/d4d/d84/fe8 1924670 0 2024-04-12T20:56:42.875 INFO:tasks.workunit.client.1.smithi186.stdout:8/864: read - d7/de/d7a/f4e zero size 2024-04-12T20:56:42.879 INFO:tasks.workunit.client.1.smithi186.stdout:1/857: creat d14/d1a/d33/d36/dfa/f118 x:0 0 0 2024-04-12T20:56:42.881 INFO:tasks.workunit.client.1.smithi186.stdout:6/813: write d9/d43/d2d/f5f [372326,11271] 0 2024-04-12T20:56:42.881 INFO:tasks.workunit.client.1.smithi186.stdout:9/785: truncate d3/d2f/d54/d9f/dc4/d77/fa8 734698 0 2024-04-12T20:56:42.882 INFO:tasks.workunit.client.1.smithi186.stdout:4/944: mknod d0/d59/dc0/d48/d61/c132 0 2024-04-12T20:56:42.883 INFO:tasks.workunit.client.1.smithi186.stdout:7/816: creat d7/d10/d2d/d40/d4a/d6c/d97/d4d/f12b x:0 0 0 2024-04-12T20:56:42.883 INFO:tasks.workunit.client.1.smithi186.stdout:9/786: chown d3/d2f/d54/d9f/dc4/d6b/ffa 656772 1 2024-04-12T20:56:42.888 INFO:tasks.workunit.client.1.smithi186.stdout:0/937: creat d0/d14/d4e/d1e/d84/f131 x:0 0 0 2024-04-12T20:56:42.893 INFO:tasks.workunit.client.1.smithi186.stdout:3/783: getdents d1/d3/df8/ddc 0 2024-04-12T20:56:42.899 INFO:tasks.workunit.client.1.smithi186.stdout:4/945: fdatasync d0/dc/d64/d87/dd2/f11e 0 2024-04-12T20:56:42.904 INFO:tasks.workunit.client.1.smithi186.stdout:0/938: readlink d0/d14/d4e/l91 0 2024-04-12T20:56:42.904 INFO:tasks.workunit.client.1.smithi186.stdout:6/814: mknod d9/da9/c11e 0 2024-04-12T20:56:42.905 INFO:tasks.workunit.client.1.smithi186.stdout:6/815: readlink d9/l21 0 2024-04-12T20:56:42.914 INFO:tasks.workunit.client.1.smithi186.stdout:4/946: creat d0/d59/dc0/d63/f133 x:0 0 0 2024-04-12T20:56:42.917 INFO:tasks.workunit.client.1.smithi186.stdout:9/787: truncate d3/d2f/d54/d9f/dc4/d77/dd6/d72/df4/fcd 2033567 0 2024-04-12T20:56:42.919 INFO:tasks.workunit.client.1.smithi186.stdout:8/865: truncate d7/d18/f85 2182157 0 2024-04-12T20:56:42.920 INFO:tasks.workunit.client.1.smithi186.stdout:0/939: fsync d0/d10/f2a 0 2024-04-12T20:56:42.922 INFO:tasks.workunit.client.1.smithi186.stdout:8/866: write d7/d18/d36/d1d/d1f/da2/f79 [6533481,102042] 0 2024-04-12T20:56:42.924 INFO:tasks.workunit.client.1.smithi186.stdout:6/816: creat d9/d4c/dbe/f11f x:0 0 0 2024-04-12T20:56:42.925 INFO:tasks.workunit.client.1.smithi186.stdout:3/784: write d1/d3/d21/d22/d34/ddb/d4d/d6f/f99 [1041815,104040] 0 2024-04-12T20:56:42.929 INFO:tasks.workunit.client.1.smithi186.stdout:4/947: rmdir d0/d59/dc0/d63/d9b 39 2024-04-12T20:56:42.930 INFO:tasks.workunit.client.1.smithi186.stdout:9/788: dwrite d3/d13/fc6 [0,4194304] 0 2024-04-12T20:56:42.932 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:56:42.933+0000 7f0c54e00700 1 -- 172.21.15.18:0/2142328463 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0c500ffd40 msgr2=0x7f0c50102160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:56:42.932 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:56:42.933+0000 7f0c54e00700 1 --2- 172.21.15.18:0/2142328463 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0c500ffd40 0x7f0c50102160 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f0c38004880 tx=0x7f0c38009e30 comp rx=0 tx=0).stop 2024-04-12T20:56:42.932 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:56:42.933+0000 7f0c54e00700 1 -- 172.21.15.18:0/2142328463 shutdown_connections 2024-04-12T20:56:42.932 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:56:42.933+0000 7f0c54e00700 1 --2- 172.21.15.18:0/2142328463 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0c500ffd40 0x7f0c50102160 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:56:42.932 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:56:42.933+0000 7f0c54e00700 1 --2- 172.21.15.18:0/2142328463 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0c500662e0 0x7f0c500ff800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:56:42.932 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:56:42.933+0000 7f0c54e00700 1 -- 172.21.15.18:0/2142328463 >> 172.21.15.18:0/2142328463 conn(0x7f0c500f7270 msgr2=0x7f0c500f96a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T20:56:42.932 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:56:42.933+0000 7f0c54e00700 1 -- 172.21.15.18:0/2142328463 shutdown_connections 2024-04-12T20:56:42.933 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:56:42.933+0000 7f0c54e00700 1 -- 172.21.15.18:0/2142328463 wait complete. 2024-04-12T20:56:42.933 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:56:42.934+0000 7f0c54e00700 1 Processor -- start 2024-04-12T20:56:42.933 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:56:42.935+0000 7f0c54e00700 1 -- start start 2024-04-12T20:56:42.936 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:56:42.938+0000 7f0c54e00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0c500662e0 0x7f0c5010b660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T20:56:42.936 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:56:42.939+0000 7f0c54e00700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0c500ffd40 0x7f0c5010bba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T20:56:42.937 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:56:42.939+0000 7f0c4f400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0c500662e0 0x7f0c5010b660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T20:56:42.938 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:56:42.939+0000 7f0c54e00700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c5010c1f0 con 0x7f0c500662e0 2024-04-12T20:56:42.938 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:56:42.939+0000 7f0c54e00700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c5010ca40 con 0x7f0c500ffd40 2024-04-12T20:56:42.938 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:56:42.940+0000 7f0c4f400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0c500662e0 0x7f0c5010b660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:42460/0 (socket says 172.21.15.18:42460) 2024-04-12T20:56:42.938 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:56:42.941+0000 7f0c4f400700 1 -- 172.21.15.18:0/570993045 learned_addr learned my addr 172.21.15.18:0/570993045 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T20:56:42.940 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:56:42.942+0000 7f0c4f400700 1 -- 172.21.15.18:0/570993045 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0c500ffd40 msgr2=0x7f0c5010bba0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-12T20:56:42.940 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:56:42.942+0000 7f0c47e00700 1 --2- 172.21.15.18:0/570993045 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0c500ffd40 0x7f0c5010bba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T20:56:42.940 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:56:42.942+0000 7f0c4f400700 1 --2- 172.21.15.18:0/570993045 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0c500ffd40 0x7f0c5010bba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:56:42.940 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:56:42.942+0000 7f0c4f400700 1 -- 172.21.15.18:0/570993045 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c40009940 con 0x7f0c500662e0 2024-04-12T20:56:42.940 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:56:42.942+0000 7f0c47e00700 1 --2- 172.21.15.18:0/570993045 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0c500ffd40 0x7f0c5010bba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-12T20:56:42.941 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:56:42.943+0000 7f0c4f400700 1 --2- 172.21.15.18:0/570993045 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0c500662e0 0x7f0c5010b660 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f0c40001f00 tx=0x7f0c40004d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T20:56:42.941 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:56:42.943+0000 7f0c47400700 1 -- 172.21.15.18:0/570993045 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0c40007920 con 0x7f0c500662e0 2024-04-12T20:56:42.941 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:56:42.943+0000 7f0c54e00700 1 -- 172.21.15.18:0/570993045 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0c38005040 con 0x7f0c500662e0 2024-04-12T20:56:42.942 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:56:42.943+0000 7f0c47400700 1 -- 172.21.15.18:0/570993045 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f0c400090d0 con 0x7f0c500662e0 2024-04-12T20:56:42.942 INFO:tasks.workunit.client.1.smithi186.stdout:5/829: link d5/db/d15/d2a/d2d/d46/d55/c43 d5/db/d2e/d70/d101/d7f/c10f 0 2024-04-12T20:56:42.942 INFO:tasks.workunit.client.1.smithi186.stdout:0/940: creat d0/d10/f132 x:0 0 0 2024-04-12T20:56:42.943 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:56:42.945+0000 7f0c47400700 1 -- 172.21.15.18:0/570993045 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0c400157d0 con 0x7f0c500662e0 2024-04-12T20:56:42.943 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:56:42.945+0000 7f0c54e00700 1 -- 172.21.15.18:0/570993045 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0c5010d150 con 0x7f0c500662e0 2024-04-12T20:56:42.944 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:56:42.946+0000 7f0c54e00700 1 -- 172.21.15.18:0/570993045 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0c5005f5f0 con 0x7f0c500662e0 2024-04-12T20:56:42.944 INFO:tasks.workunit.client.1.smithi186.stdout:5/830: fdatasync d5/db/d15/d2a/d4c/d93/f98 0 2024-04-12T20:56:42.949 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:56:42.951+0000 7f0c47400700 1 -- 172.21.15.18:0/570993045 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 25) v1 ==== 89542+0+0 (secure 0 0 0) 0x7f0c40009240 con 0x7f0c500662e0 2024-04-12T20:56:42.952 INFO:tasks.workunit.client.1.smithi186.stdout:9/789: symlink d3/d2f/d103/dd5/l10f 0 2024-04-12T20:56:42.952 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:56:42.953+0000 7f0c47400700 1 --2- 172.21.15.18:0/570993045 >> [v2:172.21.15.186:6828/2316349902,v1:172.21.15.186:6829/2316349902] conn(0x7f0c3406bca0 0x7f0c3406e150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T20:56:42.952 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:56:42.953+0000 7f0c47400700 1 -- 172.21.15.18:0/570993045 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5808+0+0 (secure 0 0 0) 0x7f0c4008a830 con 0x7f0c500662e0 2024-04-12T20:56:42.953 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:56:42.954+0000 7f0c47e00700 1 --2- 172.21.15.18:0/570993045 >> [v2:172.21.15.186:6828/2316349902,v1:172.21.15.186:6829/2316349902] conn(0x7f0c3406bca0 0x7f0c3406e150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T20:56:42.953 INFO:tasks.workunit.client.1.smithi186.stdout:2/910: sync 2024-04-12T20:56:42.955 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:56:42.956+0000 7f0c47e00700 1 --2- 172.21.15.18:0/570993045 >> [v2:172.21.15.186:6828/2316349902,v1:172.21.15.186:6829/2316349902] conn(0x7f0c3406bca0 0x7f0c3406e150 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f0c38005010 tx=0x7f0c38009e30 comp rx=0 tx=0).ready entity=mgr.24359 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T20:56:42.956 INFO:tasks.workunit.client.1.smithi186.stdout:8/867: creat d7/d18/d36/d20/d23/dd7/df5/d108/f115 x:0 0 0 2024-04-12T20:56:42.956 INFO:tasks.workunit.client.1.smithi186.stdout:3/785: symlink d1/d2c/d48/ded/l103 0 2024-04-12T20:56:42.958 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:56:42.960+0000 7f0c47400700 1 -- 172.21.15.18:0/570993045 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186454 (secure 0 0 0) 0x7f0c40050870 con 0x7f0c500662e0 2024-04-12T20:56:42.962 INFO:tasks.workunit.client.1.smithi186.stdout:3/786: read d1/d3/da/f6c [1961068,50560] 0 2024-04-12T20:56:42.962 INFO:tasks.workunit.client.1.smithi186.stdout:2/911: rmdir d0/d3/d1a/d41/d8f 39 2024-04-12T20:56:42.962 INFO:tasks.workunit.client.1.smithi186.stdout:2/912: write d0/d3/f58 [4071917,22409] 0 2024-04-12T20:56:42.963 INFO:tasks.workunit.client.1.smithi186.stdout:6/817: dread d9/f1d [0,4194304] 0 2024-04-12T20:56:42.964 INFO:tasks.workunit.client.1.smithi186.stdout:8/868: mknod d7/d18/d36/d20/d23/dd7/c116 0 2024-04-12T20:56:42.966 INFO:tasks.workunit.client.1.smithi186.stdout:6/818: readlink d9/d43/d59/d5d/d101/lf7 0 2024-04-12T20:56:42.971 INFO:tasks.workunit.client.1.smithi186.stdout:0/941: getdents d0/d14 0 2024-04-12T20:56:42.973 INFO:tasks.workunit.client.1.smithi186.stdout:0/942: write d0/d14/f35 [3546694,70246] 0 2024-04-12T20:56:42.973 INFO:tasks.workunit.client.1.smithi186.stdout:3/787: dread d1/f8a [0,4194304] 0 2024-04-12T20:56:42.975 INFO:tasks.workunit.client.1.smithi186.stdout:9/790: getdents d3/d2f/d2e/d33/d61 0 2024-04-12T20:56:42.977 INFO:tasks.workunit.client.1.smithi186.stdout:4/948: truncate d0/dc/d33/f43 6095801 0 2024-04-12T20:56:42.980 INFO:tasks.workunit.client.1.smithi186.stdout:3/788: mknod d1/d3/da/c104 0 2024-04-12T20:56:42.982 INFO:tasks.workunit.client.1.smithi186.stdout:3/789: dread - d1/d3/da2/fe8 zero size 2024-04-12T20:56:42.982 INFO:tasks.workunit.client.1.smithi186.stdout:0/943: creat d0/d14/d105/f133 x:0 0 0 2024-04-12T20:56:42.983 INFO:tasks.workunit.client.1.smithi186.stdout:0/944: readlink d0/d10/d72/l124 0 2024-04-12T20:56:42.988 INFO:tasks.workunit.client.1.smithi186.stdout:4/949: mkdir d0/d26/d121/d134 0 2024-04-12T20:56:42.990 INFO:tasks.workunit.client.1.smithi186.stdout:9/791: link d3/d2f/d2e/d33/f44 d3/d2f/d54/d9f/f110 0 2024-04-12T20:56:42.991 INFO:tasks.workunit.client.1.smithi186.stdout:2/913: dwrite d0/d3/d1a/d41/f5c [0,4194304] 0 2024-04-12T20:56:42.992 INFO:tasks.workunit.client.1.smithi186.stdout:6/819: truncate d9/f4b 4474308 0 2024-04-12T20:56:42.993 INFO:tasks.workunit.client.1.smithi186.stdout:3/790: mknod d1/d3/d21/d22/d34/ddb/d4d/dad/c105 0 2024-04-12T20:56:42.996 INFO:tasks.workunit.client.1.smithi186.stdout:0/945: mkdir d0/d10/d32/d3f/d134 0 2024-04-12T20:56:42.997 INFO:tasks.workunit.client.1.smithi186.stdout:8/869: dread d7/d18/d36/d1d/d1f/da2/df8/d29/f2a [0,4194304] 0 2024-04-12T20:56:43.000 INFO:tasks.workunit.client.1.smithi186.stdout:9/792: creat d3/d2f/d54/d9f/f111 x:0 0 0 2024-04-12T20:56:43.003 INFO:tasks.workunit.client.1.smithi186.stdout:6/820: rename d9/d5e/d7a/l108 to d9/d5e/d7a/d81/l120 0 2024-04-12T20:56:43.004 INFO:tasks.workunit.client.1.smithi186.stdout:3/791: symlink d1/d3/d21/d22/d34/l106 0 2024-04-12T20:56:43.005 INFO:tasks.workunit.client.1.smithi186.stdout:0/946: chown d0/d14/d4e/d1e/c8c 9 1 2024-04-12T20:56:43.007 INFO:tasks.workunit.client.1.smithi186.stdout:4/950: symlink d0/d59/dc0/d63/d9b/l135 0 2024-04-12T20:56:43.009 INFO:tasks.workunit.client.1.smithi186.stdout:2/914: getdents d0/d3/dce/ddf 0 2024-04-12T20:56:43.011 INFO:tasks.workunit.client.1.smithi186.stdout:6/821: creat d9/d4c/d6f/d89/f121 x:0 0 0 2024-04-12T20:56:43.014 INFO:tasks.workunit.client.1.smithi186.stdout:0/947: read - d0/d14/d4e/d1e/f82 zero size 2024-04-12T20:56:43.028 INFO:tasks.workunit.client.1.smithi186.stdout:4/951: rename d0/dc/df2 to d0/d136 0 2024-04-12T20:56:43.032 INFO:tasks.workunit.client.1.smithi186.stdout:4/952: dread - d0/d59/dc0/d63/d9b/db2/f10f zero size 2024-04-12T20:56:43.035 INFO:tasks.workunit.client.1.smithi186.stdout:4/953: fsync d0/d59/dc0/d63/f133 0 2024-04-12T20:56:43.037 INFO:tasks.workunit.client.1.smithi186.stdout:4/954: fdatasync d0/dc/f2d 0 2024-04-12T20:56:43.037 INFO:tasks.workunit.client.1.smithi186.stdout:0/948: dwrite d0/d10/d32/fe9 [0,4194304] 0 2024-04-12T20:56:43.038 INFO:tasks.workunit.client.1.smithi186.stdout:3/792: dread d1/d3/f58 [0,4194304] 0 2024-04-12T20:56:43.043 INFO:tasks.workunit.client.1.smithi186.stdout:7/817: sync 2024-04-12T20:56:43.043 INFO:tasks.workunit.client.1.smithi186.stdout:1/858: sync 2024-04-12T20:56:43.045 INFO:tasks.workunit.client.1.smithi186.stdout:0/949: creat d0/d5c/d65/f135 x:0 0 0 2024-04-12T20:56:43.048 INFO:tasks.workunit.client.1.smithi186.stdout:7/818: creat d7/d10/d2d/d67/f12c x:0 0 0 2024-04-12T20:56:43.050 INFO:tasks.workunit.client.1.smithi186.stdout:9/793: truncate d3/d13/fc6 417590 0 2024-04-12T20:56:43.050 INFO:tasks.workunit.client.1.smithi186.stdout:1/859: read - d14/d1a/d33/d36/d5b/d3e/d8b/fff zero size 2024-04-12T20:56:43.051 INFO:tasks.workunit.client.1.smithi186.stdout:2/915: dread d0/d32/d2a/f103 [0,4194304] 0 2024-04-12T20:56:43.051 INFO:tasks.workunit.client.1.smithi186.stdout:6/822: write d9/f40 [1953320,90495] 0 2024-04-12T20:56:43.061 INFO:tasks.workunit.client.1.smithi186.stdout:9/794: mkdir d3/d2f/d54/d7d/d112 0 2024-04-12T20:56:43.069 INFO:tasks.workunit.client.1.smithi186.stdout:7/819: write d7/d10/d2d/d40/d4a/d6c/ddf/fed [29211,70382] 0 2024-04-12T20:56:43.069 INFO:tasks.workunit.client.1.smithi186.stdout:1/860: fsync d14/d1a/d33/d36/dfa/f118 0 2024-04-12T20:56:43.069 INFO:tasks.workunit.client.1.smithi186.stdout:3/793: dread d1/f63 [0,4194304] 0 2024-04-12T20:56:43.071 INFO:tasks.workunit.client.1.smithi186.stdout:4/955: dwrite d0/d21/f40 [0,4194304] 0 2024-04-12T20:56:43.078 INFO:tasks.workunit.client.1.smithi186.stdout:8/870: dwrite d7/d18/f1a [4194304,4194304] 0 2024-04-12T20:56:43.086 INFO:tasks.workunit.client.1.smithi186.stdout:8/871: write d7/de/f21 [5018562,95963] 0 2024-04-12T20:56:43.093 INFO:tasks.workunit.client.1.smithi186.stdout:4/956: creat d0/d59/dc0/d63/d9b/db2/f137 x:0 0 0 2024-04-12T20:56:43.096 INFO:tasks.workunit.client.1.smithi186.stdout:5/831: sync 2024-04-12T20:56:43.098 INFO:tasks.workunit.client.1.smithi186.stdout:9/795: link d3/d2f/d2e/d33/d36/d70/f7c d3/d2f/d2e/d33/d36/d93/f113 0 2024-04-12T20:56:43.100 INFO:tasks.workunit.client.1.smithi186.stdout:6/823: dwrite d9/d4c/d6f/fb2 [4194304,4194304] 0 2024-04-12T20:56:43.103 INFO:tasks.workunit.client.1.smithi186.stdout:8/872: fsync d7/d18/d36/d1d/d1f/d26/dab/fae 0 2024-04-12T20:56:43.103 INFO:tasks.workunit.client.1.smithi186.stdout:3/794: symlink d1/d3/d21/d22/d34/ddb/d4d/da1/l107 0 2024-04-12T20:56:43.105 INFO:tasks.workunit.client.1.smithi186.stdout:4/957: stat d0/d21/l10b 0 2024-04-12T20:56:43.106 INFO:tasks.workunit.client.1.smithi186.stdout:1/861: creat d14/d1a/d1d/f119 x:0 0 0 2024-04-12T20:56:43.107 INFO:tasks.workunit.client.1.smithi186.stdout:0/950: dwrite d0/d85/dc4/dc1/fd3 [0,4194304] 0 2024-04-12T20:56:43.111 INFO:tasks.workunit.client.1.smithi186.stdout:2/916: dwrite d0/d32/d8b/f11c [0,4194304] 0 2024-04-12T20:56:43.111 INFO:tasks.workunit.client.1.smithi186.stdout:5/832: fsync d5/d27/f91 0 2024-04-12T20:56:43.116 INFO:tasks.workunit.client.1.smithi186.stdout:4/958: rmdir d0/dc/d88/d80 39 2024-04-12T20:56:43.117 INFO:tasks.workunit.client.1.smithi186.stdout:2/917: truncate d0/f79 786376 0 2024-04-12T20:56:43.123 INFO:tasks.workunit.client.1.smithi186.stdout:6/824: creat d9/d4c/dbe/f122 x:0 0 0 2024-04-12T20:56:43.125 INFO:tasks.workunit.client.1.smithi186.stdout:8/873: symlink d7/d18/d36/d20/d71/l117 0 2024-04-12T20:56:43.125 INFO:tasks.workunit.client.1.smithi186.stdout:2/918: readlink d0/d3/d1a/d21/l102 0 2024-04-12T20:56:43.127 INFO:tasks.workunit.client.1.smithi186.stdout:5/833: symlink d5/db/d71/d88/l110 0 2024-04-12T20:56:43.128 INFO:tasks.workunit.client.1.smithi186.stdout:0/951: write d0/d85/dc4/f12e [6931,27690] 0 2024-04-12T20:56:43.134 INFO:tasks.workunit.client.1.smithi186.stdout:7/820: dwrite d7/d10/d2d/f44 [0,4194304] 0 2024-04-12T20:56:43.137 INFO:tasks.workunit.client.1.smithi186.stdout:1/862: truncate d14/d1a/d58/d69/fdf 145465 0 2024-04-12T20:56:43.137 INFO:tasks.workunit.client.1.smithi186.stdout:4/959: truncate d0/dc/fa2 336681 0 2024-04-12T20:56:43.139 INFO:tasks.workunit.client.1.smithi186.stdout:6/825: chown d9/d43/d2d/d4d/fe0 415 1 2024-04-12T20:56:43.142 INFO:tasks.workunit.client.1.smithi186.stdout:8/874: rename d7/de/d7a/c9b to d7/de/d10/d7f/c118 0 2024-04-12T20:56:43.146 INFO:tasks.workunit.client.1.smithi186.stdout:1/863: fdatasync d14/d1a/d33/d36/d5b/d3e/d5f/f8f 0 2024-04-12T20:56:43.147 INFO:tasks.workunit.client.1.smithi186.stdout:5/834: truncate d5/db/d2e/d70/d101/d7f/ffc 284381 0 2024-04-12T20:56:43.147 INFO:tasks.workunit.client.1.smithi186.stdout:7/821: unlink d7/d10/d1c/f8e 0 2024-04-12T20:56:43.150 INFO:tasks.workunit.client.1.smithi186.stdout:6/826: truncate d9/d43/d50/f5b 5154884 0 2024-04-12T20:56:43.151 INFO:tasks.workunit.client.1.smithi186.stdout:2/919: creat d0/d3/d1a/d24/db3/f124 x:0 0 0 2024-04-12T20:56:43.157 INFO:tasks.workunit.client.1.smithi186.stdout:0/952: rename d0/d10/f4d to d0/d14/d4e/d50/d6e/d96/d12a/f136 0 2024-04-12T20:56:43.157 INFO:tasks.workunit.client.1.smithi186.stdout:9/796: dwrite d3/d2f/d2e/d33/fdd [0,4194304] 0 2024-04-12T20:56:43.165 INFO:tasks.workunit.client.1.smithi186.stdout:1/864: write d14/d1a/d3c/f102 [1056489,17154] 0 2024-04-12T20:56:43.170 INFO:tasks.workunit.client.1.smithi186.stdout:3/795: dread d1/d3/d21/d22/d34/ddb/d4d/d6f/f99 [0,4194304] 0 2024-04-12T20:56:43.171 INFO:tasks.workunit.client.1.smithi186.stdout:5/835: write d5/db/d2e/d70/d101/dbb/dc4/fc3 [2695791,116555] 0 2024-04-12T20:56:43.174 INFO:tasks.workunit.client.1.smithi186.stdout:8/875: dread - d7/d18/d36/d1d/d7d/da5/fbc zero size 2024-04-12T20:56:43.178 INFO:tasks.workunit.client.1.smithi186.stdout:7/822: dwrite d7/f17 [0,4194304] 0 2024-04-12T20:56:43.181 INFO:tasks.workunit.client.1.smithi186.stdout:4/960: dwrite d0/d59/dc0/d63/da6/dc6/f66 [0,4194304] 0 2024-04-12T20:56:43.184 INFO:tasks.workunit.client.1.smithi186.stdout:1/865: mknod d14/d1a/d21/d35/dbc/de3/c11a 0 2024-04-12T20:56:43.184 INFO:tasks.workunit.client.1.smithi186.stdout:2/920: mkdir d0/d3/d1a/d48/df7/d111/d125 0 2024-04-12T20:56:43.185 INFO:tasks.workunit.client.1.smithi186.stdout:3/796: fsync d1/d2c/d48/f57 0 2024-04-12T20:56:43.186 INFO:tasks.workunit.client.1.smithi186.stdout:0/953: mkdir d0/d10/d32/de2/d120/d137 0 2024-04-12T20:56:43.190 INFO:tasks.workunit.client.1.smithi186.stdout:8/876: creat d7/d18/d8e/d10c/f119 x:0 0 0 2024-04-12T20:56:43.192 INFO:tasks.workunit.client.1.smithi186.stdout:9/797: mknod d3/d2f/d2e/d43/d4c/c114 0 2024-04-12T20:56:43.193 INFO:tasks.workunit.client.1.smithi186.stdout:7/823: creat d7/d10/d2d/d40/d4a/d6c/d97/d109/f12d x:0 0 0 2024-04-12T20:56:43.196 INFO:tasks.workunit.client.1.smithi186.stdout:7/824: write d7/d10/d2d/d67/f70 [175520,86793] 0 2024-04-12T20:56:43.197 INFO:tasks.workunit.client.1.smithi186.stdout:9/798: fsync d3/d2f/d2e/d43/d4c/fdc 0 2024-04-12T20:56:43.198 INFO:tasks.workunit.client.1.smithi186.stdout:3/797: mknod d1/d3/d21/d22/da4/c108 0 2024-04-12T20:56:43.200 INFO:tasks.workunit.client.1.smithi186.stdout:0/954: creat d0/d14/d4e/d1e/d84/dec/f138 x:0 0 0 2024-04-12T20:56:43.204 INFO:tasks.workunit.client.1.smithi186.stdout:8/877: truncate d7/d18/d36/d20/d23/dc1/dda/f10d 740139 0 2024-04-12T20:56:43.205 INFO:tasks.workunit.client.1.smithi186.stdout:3/798: readlink d1/d3/d21/d22/l3b 0 2024-04-12T20:56:43.205 INFO:tasks.workunit.client.1.smithi186.stdout:6/827: getdents d9/d43/d59/d5d/d6e/d91/dd9 0 2024-04-12T20:56:43.208 INFO:tasks.workunit.client.1.smithi186.stdout:0/955: truncate d0/d14/d4e/d50/d6e/d119/f11a 117271 0 2024-04-12T20:56:43.209 INFO:tasks.workunit.client.1.smithi186.stdout:4/961: mkdir d0/dc/d88/d80/d9c/d138 0 2024-04-12T20:56:43.212 INFO:tasks.workunit.client.1.smithi186.stdout:0/956: truncate d0/d14/d4e/d1e/d84/f97 8799360 0 2024-04-12T20:56:43.214 INFO:tasks.workunit.client.1.smithi186.stdout:0/957: chown d0/d5c/d65/fff 191311 1 2024-04-12T20:56:43.216 INFO:tasks.workunit.client.1.smithi186.stdout:0/958: dread - d0/d5c/d66/fcf zero size 2024-04-12T20:56:43.223 INFO:tasks.workunit.client.1.smithi186.stdout:8/878: mknod d7/d18/d36/d20/d23/dd7/c11a 0 2024-04-12T20:56:43.228 INFO:tasks.workunit.client.1.smithi186.stdout:6/828: fsync d9/d43/d59/d5d/d6e/d91/fac 0 2024-04-12T20:56:43.229 INFO:tasks.workunit.client.1.smithi186.stdout:4/962: rmdir d0/dc/d88/d80/df8 39 2024-04-12T20:56:43.231 INFO:tasks.workunit.client.1.smithi186.stdout:4/963: chown d0/d21/lf3 16568261 1 2024-04-12T20:56:43.232 INFO:tasks.workunit.client.1.smithi186.stdout:0/959: mknod d0/d5d/ddc/c139 0 2024-04-12T20:56:43.233 INFO:tasks.workunit.client.1.smithi186.stdout:7/825: truncate d7/d10/d2d/f37 9240856 0 2024-04-12T20:56:43.233 INFO:tasks.workunit.client.1.smithi186.stdout:0/960: stat d0/d5d/ddc 0 2024-04-12T20:56:43.237 INFO:tasks.workunit.client.1.smithi186.stdout:1/866: write d14/d1a/d58/d69/fb3 [3257768,33300] 0 2024-04-12T20:56:43.237 INFO:tasks.workunit.client.1.smithi186.stdout:4/964: chown d0/d59/dc0/d63/cca 45 1 2024-04-12T20:56:43.240 INFO:tasks.workunit.client.1.smithi186.stdout:4/965: fsync d0/d136/d118/f122 0 2024-04-12T20:56:43.246 INFO:tasks.workunit.client.1.smithi186.stdout:8/879: link d7/d18/d36/d20/d23/dd6/de8/df7/f105 d7/de/d5e/df0/f11b 0 2024-04-12T20:56:43.248 INFO:tasks.workunit.client.1.smithi186.stdout:5/836: dwrite d5/db/f40 [0,4194304] 0 2024-04-12T20:56:43.248 INFO:tasks.workunit.client.1.smithi186.stdout:2/921: truncate d0/d3e/fe6 10481 0 2024-04-12T20:56:43.251 INFO:tasks.workunit.client.1.smithi186.stdout:1/867: symlink d14/d1a/d33/d36/d5b/d3e/d5f/l11b 0 2024-04-12T20:56:43.252 INFO:tasks.workunit.client.1.smithi186.stdout:7/826: creat d7/d10/d2d/d40/d4a/d6c/d97/d109/f12e x:0 0 0 2024-04-12T20:56:43.253 INFO:tasks.workunit.client.1.smithi186.stdout:0/961: mkdir d0/d5c/d13a 0 2024-04-12T20:56:43.254 INFO:tasks.workunit.client.1.smithi186.stdout:1/868: write d14/d1a/d58/fb7 [1554025,97461] 0 2024-04-12T20:56:43.259 INFO:tasks.workunit.client.1.smithi186.stdout:4/966: truncate d0/d59/dc0/d63/da6/dc6/f3a 3879061 0 2024-04-12T20:56:43.263 INFO:tasks.workunit.client.1.smithi186.stdout:2/922: creat d0/dc/d2e/d73/f126 x:0 0 0 2024-04-12T20:56:43.263 INFO:tasks.workunit.client.1.smithi186.stdout:3/799: write d1/d2c/d48/f5f [2334737,64200] 0 2024-04-12T20:56:43.263 INFO:tasks.workunit.client.1.smithi186.stdout:0/962: truncate d0/d85/dc4/f12e 392455 0 2024-04-12T20:56:43.265 INFO:tasks.workunit.client.1.smithi186.stdout:8/880: mkdir d7/d18/d36/d1d/d1f/d11c 0 2024-04-12T20:56:43.267 INFO:tasks.workunit.client.1.smithi186.stdout:7/827: readlink d7/d10/d2d/l3e 0 2024-04-12T20:56:43.268 INFO:tasks.workunit.client.1.smithi186.stdout:4/967: dread - d0/dc/d64/d87/d98/f12e zero size 2024-04-12T20:56:43.270 INFO:tasks.workunit.client.1.smithi186.stdout:6/829: dread d9/d43/d50/f55 [0,4194304] 0 2024-04-12T20:56:43.275 INFO:tasks.workunit.client.1.smithi186.stdout:1/869: read d14/d1a/d58/d69/f110 [197146,100271] 0 2024-04-12T20:56:43.282 INFO:tasks.workunit.client.1.smithi186.stdout:3/800: creat d1/d3/df8/ddc/f109 x:0 0 0 2024-04-12T20:56:43.283 INFO:tasks.workunit.client.1.smithi186.stdout:0/963: symlink d0/d14/d4e/d50/l13b 0 2024-04-12T20:56:43.283 INFO:tasks.workunit.client.1.smithi186.stdout:5/837: dwrite d5/db/d2e/d70/d101/dbb/dc4/fc3 [0,4194304] 0 2024-04-12T20:56:43.284 INFO:tasks.workunit.client.1.smithi186.stdout:4/968: creat d0/d136/d118/f139 x:0 0 0 2024-04-12T20:56:43.286 INFO:tasks.workunit.client.1.smithi186.stdout:4/969: readlink d0/l2a 0 2024-04-12T20:56:43.286 INFO:tasks.workunit.client.1.smithi186.stdout:9/799: dwrite d3/d2f/f21 [0,4194304] 0 2024-04-12T20:56:43.287 INFO:tasks.workunit.client.1.smithi186.stdout:6/830: creat d9/d4c/d6f/de3/f123 x:0 0 0 2024-04-12T20:56:43.288 INFO:tasks.workunit.client.1.smithi186.stdout:0/964: write d0/d5c/d65/fd2 [2487028,56405] 0 2024-04-12T20:56:43.291 INFO:tasks.workunit.client.1.smithi186.stdout:4/970: write d0/d59/dc0/d63/da6/feb [1268189,69016] 0 2024-04-12T20:56:43.293 INFO:tasks.workunit.client.1.smithi186.stdout:3/801: chown d1/c78 45705883 1 2024-04-12T20:56:43.295 INFO:tasks.workunit.client.1.smithi186.stdout:2/923: mknod d0/d3/d1a/d48/df7/d121/c127 0 2024-04-12T20:56:43.299 INFO:tasks.workunit.client.1.smithi186.stdout:2/924: dread - d0/d3/d1a/d24/f105 zero size 2024-04-12T20:56:43.307 INFO:tasks.workunit.client.1.smithi186.stdout:9/800: mknod d3/d2f/d54/c115 0 2024-04-12T20:56:43.307 INFO:tasks.workunit.client.1.smithi186.stdout:7/828: creat d7/d10/d2d/d40/d4a/d6c/d97/d4d/d6a/d11d/d120/f12f x:0 0 0 2024-04-12T20:56:43.309 INFO:tasks.workunit.client.1.smithi186.stdout:8/881: rename d7/d18/d36/dc9 to d7/de/d10/d11d 0 2024-04-12T20:56:43.310 INFO:tasks.workunit.client.1.smithi186.stdout:3/802: creat d1/d3/da/f10a x:0 0 0 2024-04-12T20:56:43.312 INFO:tasks.workunit.client.1.smithi186.stdout:7/829: dread d7/d55/ffd [0,4194304] 0 2024-04-12T20:56:43.313 INFO:tasks.workunit.client.1.smithi186.stdout:6/831: unlink d9/d43/d2d/d4d/l109 0 2024-04-12T20:56:43.314 INFO:tasks.workunit.client.1.smithi186.stdout:8/882: read d7/de/d68/f9d [4027412,43046] 0 2024-04-12T20:56:43.316 INFO:tasks.workunit.client.1.smithi186.stdout:0/965: dwrite d0/d14/d4e/d50/f110 [0,4194304] 0 2024-04-12T20:56:43.318 INFO:tasks.workunit.client.1.smithi186.stdout:9/801: symlink d3/d2f/d54/d7d/l116 0 2024-04-12T20:56:43.326 INFO:tasks.workunit.client.1.smithi186.stdout:4/971: creat d0/dc/d88/d80/df8/f13a x:0 0 0 2024-04-12T20:56:43.327 INFO:tasks.workunit.client.1.smithi186.stdout:2/925: creat d0/d3/d1a/d41/d8f/de9/f128 x:0 0 0 2024-04-12T20:56:43.336 INFO:tasks.workunit.client.1.smithi186.stdout:5/838: rename d5/db/d2e/d70/d101/d7f/d80/cf5 to d5/db/d2e/d70/d101/dbb/c111 0 2024-04-12T20:56:43.336 INFO:tasks.workunit.client.1.smithi186.stdout:1/870: dwrite d14/d1a/d33/d36/d5b/d3e/fd7 [0,4194304] 0 2024-04-12T20:56:43.339 INFO:tasks.workunit.client.1.smithi186.stdout:9/802: creat d3/d13/d10b/f117 x:0 0 0 2024-04-12T20:56:43.343 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:56:43.344+0000 7f0c54e00700 1 -- 172.21.15.18:0/570993045 --> [v2:172.21.15.186:6828/2316349902,v1:172.21.15.186:6829/2316349902] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f0c5010d4d0 con 0x7f0c3406bca0 2024-04-12T20:56:43.345 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:56:43.346+0000 7f0c47400700 1 -- 172.21.15.18:0/570993045 <== mgr.24359 v2:172.21.15.186:6828/2316349902 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+341 (secure 0 0 0) 0x7f0c5010d4d0 con 0x7f0c3406bca0 2024-04-12T20:56:43.347 INFO:tasks.workunit.client.1.smithi186.stdout:2/926: unlink d0/dc/d2e/de8/l123 0 2024-04-12T20:56:43.347 INFO:tasks.workunit.client.1.smithi186.stdout:4/972: read d0/d59/f85 [2467625,17420] 0 2024-04-12T20:56:43.347 INFO:tasks.workunit.client.1.smithi186.stdout:6/832: dread d9/d4c/f61 [0,4194304] 0 2024-04-12T20:56:43.348 INFO:tasks.workunit.client.1.smithi186.stdout:2/927: stat d0/dc/d2e/fdd 0 2024-04-12T20:56:43.349 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:56:43.351+0000 7f0c54e00700 1 -- 172.21.15.18:0/570993045 >> [v2:172.21.15.186:6828/2316349902,v1:172.21.15.186:6829/2316349902] conn(0x7f0c3406bca0 msgr2=0x7f0c3406e150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:56:43.349 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:56:43.351+0000 7f0c54e00700 1 --2- 172.21.15.18:0/570993045 >> [v2:172.21.15.186:6828/2316349902,v1:172.21.15.186:6829/2316349902] conn(0x7f0c3406bca0 0x7f0c3406e150 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f0c38005010 tx=0x7f0c38009e30 comp rx=0 tx=0).stop 2024-04-12T20:56:43.349 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:56:43.351+0000 7f0c54e00700 1 -- 172.21.15.18:0/570993045 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0c500662e0 msgr2=0x7f0c5010b660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:56:43.349 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:56:43.351+0000 7f0c54e00700 1 --2- 172.21.15.18:0/570993045 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0c500662e0 0x7f0c5010b660 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f0c40001f00 tx=0x7f0c40004d10 comp rx=0 tx=0).stop 2024-04-12T20:56:43.349 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:56:43.351+0000 7f0c54e00700 1 -- 172.21.15.18:0/570993045 shutdown_connections 2024-04-12T20:56:43.349 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:56:43.351+0000 7f0c54e00700 1 --2- 172.21.15.18:0/570993045 >> [v2:172.21.15.186:6828/2316349902,v1:172.21.15.186:6829/2316349902] conn(0x7f0c3406bca0 0x7f0c3406e150 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:56:43.349 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:56:43.351+0000 7f0c54e00700 1 --2- 172.21.15.18:0/570993045 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0c500662e0 0x7f0c5010b660 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:56:43.350 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:56:43.351+0000 7f0c54e00700 1 --2- 172.21.15.18:0/570993045 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0c500ffd40 0x7f0c5010bba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:56:43.350 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:56:43.351+0000 7f0c54e00700 1 -- 172.21.15.18:0/570993045 >> 172.21.15.18:0/570993045 conn(0x7f0c500f7270 msgr2=0x7f0c500fb900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T20:56:43.351 INFO:tasks.workunit.client.1.smithi186.stdout:8/883: truncate d7/de/d7a/f31 2884511 0 2024-04-12T20:56:43.351 INFO:tasks.workunit.client.1.smithi186.stdout:2/928: chown d0/d3/fc8 15 1 2024-04-12T20:56:43.351 INFO:tasks.workunit.client.1.smithi186.stdout:4/973: truncate d0/d136/d118/f122 499236 0 2024-04-12T20:56:43.351 INFO:tasks.workunit.client.1.smithi186.stdout:0/966: symlink d0/d14/d4e/d1e/d84/l13c 0 2024-04-12T20:56:43.352 INFO:tasks.workunit.client.1.smithi186.stdout:2/929: read d0/d32/d8b/f11c [1678992,80281] 0 2024-04-12T20:56:43.352 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:56:43.353+0000 7f0c54e00700 1 -- 172.21.15.18:0/570993045 shutdown_connections 2024-04-12T20:56:43.352 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:56:43.353+0000 7f0c54e00700 1 -- 172.21.15.18:0/570993045 wait complete. 2024-04-12T20:56:43.355 INFO:tasks.workunit.client.1.smithi186.stdout:3/803: rename d1/d2c/d54 to d1/d3/df8/dd0/d10b 0 2024-04-12T20:56:43.355 INFO:tasks.workunit.client.1.smithi186.stdout:1/871: chown d14/d1a/d33/fd3 3570769 1 2024-04-12T20:56:43.355 INFO:tasks.workunit.client.1.smithi186.stdout:9/803: creat d3/d2f/d2e/db3/f118 x:0 0 0 2024-04-12T20:56:43.357 INFO:tasks.workunit.client.1.smithi186.stdout:1/872: fsync d14/d1a/fcc 0 2024-04-12T20:56:43.359 INFO:tasks.workunit.client.1.smithi186.stdout:4/974: mkdir d0/d59/dc0/d48/d61/d13b 0 2024-04-12T20:56:43.360 INFO:tasks.workunit.client.1.smithi186.stdout:5/839: symlink d5/db/d2e/d70/d101/d7f/l112 0 2024-04-12T20:56:43.363 INFO:tasks.workunit.client.1.smithi186.stdout:5/840: chown d5/db/d15/d2a/d2d/la0 18 1 2024-04-12T20:56:43.364 INFO:tasks.workunit.client.1.smithi186.stdout:7/830: getdents d7/deb 0 2024-04-12T20:56:43.365 INFO:tasks.workunit.client.1.smithi186.stdout:7/831: dread - d7/d10/d1c/d8c/f112 zero size 2024-04-12T20:56:43.365 INFO:tasks.workunit.client.1.smithi186.stdout:6/833: mknod d9/d43/d59/c124 0 2024-04-12T20:56:43.398 INFO:tasks.workunit.client.1.smithi186.stdout:9/804: mknod d3/d2f/d2e/d33/d61/dbc/d105/c119 0 2024-04-12T20:56:43.401 INFO:tasks.workunit.client.1.smithi186.stdout:3/804: truncate d1/d3/d21/d22/d34/ddb/d4d/dad/fcf 1193903 0 2024-04-12T20:56:43.404 INFO:tasks.workunit.client.1.smithi186.stdout:6/834: symlink d9/d5e/d7a/l125 0 2024-04-12T20:56:43.404 INFO:tasks.workunit.client.1.smithi186.stdout:3/805: write d1/d3/da2/fe8 [413520,58682] 0 2024-04-12T20:56:43.404 INFO:tasks.workunit.client.1.smithi186.stdout:7/832: rename d7/d10/d2d/d40/d4a/d6c/f7d to d7/d55/d71/f130 0 2024-04-12T20:56:43.405 INFO:tasks.workunit.client.1.smithi186.stdout:8/884: creat d7/d18/d36/d20/d23/dd7/df5/d108/f11e x:0 0 0 2024-04-12T20:56:43.406 INFO:tasks.workunit.client.1.smithi186.stdout:7/833: chown d7/deb 2703 1 2024-04-12T20:56:43.407 INFO:tasks.workunit.client.1.smithi186.stdout:9/805: creat d3/d2f/d54/d9f/dc4/d77/dd6/d72/df4/df9/f11a x:0 0 0 2024-04-12T20:56:43.407 INFO:tasks.workunit.client.1.smithi186.stdout:2/930: dread d0/d3/d1a/d27/f59 [0,4194304] 0 2024-04-12T20:56:43.409 INFO:tasks.workunit.client.1.smithi186.stdout:6/835: symlink d9/d5e/d73/l126 0 2024-04-12T20:56:43.410 INFO:tasks.workunit.client.1.smithi186.stdout:5/841: getdents d5/db/d15/d2a 0 2024-04-12T20:56:43.413 INFO:tasks.workunit.client.1.smithi186.stdout:7/834: fdatasync d7/d10/d2d/fe5 0 2024-04-12T20:56:43.416 INFO:tasks.workunit.client.1.smithi186.stdout:4/975: dread d0/d21/f32 [0,4194304] 0 2024-04-12T20:56:43.418 INFO:tasks.workunit.client.1.smithi186.stdout:4/976: truncate d0/d59/ddf/f124 408894 0 2024-04-12T20:56:43.419 INFO:tasks.workunit.client.1.smithi186.stdout:8/885: rename d7/de/d5e/df0 to d7/de/d68/d11f 0 2024-04-12T20:56:43.419 INFO:tasks.workunit.client.1.smithi186.stdout:6/836: symlink d9/d5e/d73/l127 0 2024-04-12T20:56:43.433 INFO:tasks.workunit.client.1.smithi186.stdout:7/835: rmdir d7 39 2024-04-12T20:56:43.435 INFO:tasks.workunit.client.1.smithi186.stdout:3/806: dread d1/d3/d21/d22/d34/ddb/d4d/d6f/da6/fce [0,4194304] 0 2024-04-12T20:56:43.436 INFO:tasks.workunit.client.1.smithi186.stdout:8/886: dwrite d7/d18/d36/d20/d71/fc2 [0,4194304] 0 2024-04-12T20:56:43.437 INFO:tasks.workunit.client.1.smithi186.stdout:2/931: rename d0/cb to d0/d32/d8b/c129 0 2024-04-12T20:56:43.442 INFO:tasks.workunit.client.1.smithi186.stdout:3/807: mknod d1/d3/d21/d22/d34/ddb/d4d/dad/c10c 0 2024-04-12T20:56:43.447 INFO:tasks.workunit.client.1.smithi186.stdout:7/836: fsync d7/d10/d2d/d40/d4a/d6c/d97/d4d/d84/fe8 0 2024-04-12T20:56:43.449 INFO:tasks.workunit.client.1.smithi186.stdout:3/808: dread - d1/d3/d21/d22/d34/ddb/d4d/d9c/fcb zero size 2024-04-12T20:56:43.452 INFO:tasks.workunit.client.1.smithi186.stdout:4/977: dread d0/d59/dc0/d63/d9b/fc4 [0,4194304] 0 2024-04-12T20:56:43.453 INFO:tasks.workunit.client.1.smithi186.stdout:8/887: symlink d7/d18/d36/d1d/d1f/d26/de6/l120 0 2024-04-12T20:56:43.455 INFO:tasks.workunit.client.1.smithi186.stdout:5/842: dwrite d5/db/d15/d2a/d2d/f9f [0,4194304] 0 2024-04-12T20:56:43.456 INFO:tasks.workunit.client.1.smithi186.stdout:7/837: write d7/d10/d2d/d40/d4a/d6c/d97/d109/f12e [144188,81089] 0 2024-04-12T20:56:43.457 INFO:tasks.workunit.client.1.smithi186.stdout:3/809: write d1/d3/da/f70 [4296997,79618] 0 2024-04-12T20:56:43.461 INFO:tasks.workunit.client.1.smithi186.stdout:8/888: chown d7/d18/d36/d1d/d7d/da5/l82 137309387 1 2024-04-12T20:56:43.468 INFO:tasks.workunit.client.1.smithi186.stdout:2/932: dwrite d0/f100 [0,4194304] 0 2024-04-12T20:56:43.468 INFO:tasks.workunit.client.1.smithi186.stdout:6/837: dwrite d9/d43/d50/f55 [4194304,4194304] 0 2024-04-12T20:56:43.469 INFO:tasks.workunit.client.1.smithi186.stdout:8/889: mkdir d7/d18/d36/d20/d71/d121 0 2024-04-12T20:56:43.471 INFO:tasks.workunit.client.1.smithi186.stdout:3/810: truncate d1/d3/d21/d22/f87 641627 0 2024-04-12T20:56:43.475 INFO:tasks.workunit.client.1.smithi186.stdout:2/933: symlink d0/dc/d2e/da9/l12a 0 2024-04-12T20:56:43.475 INFO:tasks.workunit.client.1.smithi186.stdout:5/843: symlink d5/db/d2e/d70/d101/dbb/dc4/d5d/de1/l113 0 2024-04-12T20:56:43.480 INFO:tasks.workunit.client.1.smithi186.stdout:7/838: dread d7/d10/d2d/d40/f10c [0,4194304] 0 2024-04-12T20:56:43.482 INFO:tasks.workunit.client.1.smithi186.stdout:3/811: unlink d1/d3/d21/d22/d34/ddb/d4d/d6f/da6/fce 0 2024-04-12T20:56:43.484 INFO:tasks.workunit.client.1.smithi186.stdout:8/890: read d7/d18/d36/d1d/d1f/da2/df8/d29/f6a [7777403,27633] 0 2024-04-12T20:56:43.485 INFO:tasks.workunit.client.1.smithi186.stdout:2/934: unlink d0/d3/d1a/d48/df7/ca6 0 2024-04-12T20:56:43.485 INFO:tasks.workunit.client.1.smithi186.stdout:6/838: fsync d9/d5e/f6d 0 2024-04-12T20:56:43.489 INFO:tasks.workunit.client.1.smithi186.stdout:5/844: truncate d5/db/d15/d2a/d2d/d82/d84/fcb 842308 0 2024-04-12T20:56:43.490 INFO:tasks.workunit.client.1.smithi186.stdout:7/839: symlink d7/d55/d71/d10a/l131 0 2024-04-12T20:56:43.493 INFO:tasks.workunit.client.1.smithi186.stdout:8/891: write d7/d18/d36/d1d/d7d/da5/f9c [2749451,81372] 0 2024-04-12T20:56:43.494 INFO:tasks.workunit.client.1.smithi186.stdout:4/978: dwrite d0/d59/dc0/d63/da6/dc6/f82 [0,4194304] 0 2024-04-12T20:56:43.497 INFO:tasks.workunit.client.1.smithi186.stdout:2/935: truncate d0/d3/d1a/d27/f110 756684 0 2024-04-12T20:56:43.498 INFO:tasks.workunit.client.1.smithi186.stdout:8/892: stat d7/d18/d36/d1d/d7d/da5 0 2024-04-12T20:56:43.498 INFO:tasks.workunit.client.1.smithi186.stdout:6/839: creat d9/d4c/d6f/f128 x:0 0 0 2024-04-12T20:56:43.504 INFO:tasks.workunit.client.1.smithi186.stdout:2/936: mknod d0/d3/d7a/c12b 0 2024-04-12T20:56:43.504 INFO:tasks.workunit.client.1.smithi186.stdout:7/840: creat d7/d10/d2d/d40/d4a/d6c/d97/d109/d124/f132 x:0 0 0 2024-04-12T20:56:43.506 INFO:tasks.workunit.client.1.smithi186.stdout:3/812: write d1/d2c/d48/f57 [321391,63059] 0 2024-04-12T20:56:43.508 INFO:tasks.workunit.client.1.smithi186.stdout:8/893: mknod d7/d18/d36/d1d/d1f/d26/de6/c122 0 2024-04-12T20:56:43.511 INFO:tasks.workunit.client.1.smithi186.stdout:8/894: dread - d7/d18/d8e/d10c/f113 zero size 2024-04-12T20:56:43.512 INFO:tasks.workunit.client.1.smithi186.stdout:7/841: rename d7/d55/d71/dbe/lbd to d7/d10/d2d/d86/l133 0 2024-04-12T20:56:43.514 INFO:tasks.workunit.client.1.smithi186.stdout:3/813: mknod d1/d3/d21/d22/d4a/dbc/c10d 0 2024-04-12T20:56:43.515 INFO:tasks.workunit.client.1.smithi186.stdout:4/979: write d0/dc/d33/f47 [4551451,75365] 0 2024-04-12T20:56:43.518 INFO:tasks.workunit.client.1.smithi186.stdout:5/845: dread d5/db/d15/d2a/d4c/d93/f98 [0,4194304] 0 2024-04-12T20:56:43.524 INFO:tasks.workunit.client.1.smithi186.stdout:2/937: dwrite d0/d3/d1a/d41/f5c [0,4194304] 0 2024-04-12T20:56:43.525 INFO:tasks.workunit.client.1.smithi186.stdout:4/980: mkdir d0/d21/d13c 0 2024-04-12T20:56:43.527 INFO:tasks.workunit.client.1.smithi186.stdout:8/895: rename d7/d18/d36/d1d/d7d/lb7 to d7/de/d10/d11d/l123 0 2024-04-12T20:56:43.530 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:56:43 smithi186 ceph-mon[36597]: pgmap v54: 65 pgs: 65 active+clean; 3.2 GiB data, 11 GiB used, 526 GiB / 536 GiB avail; 86 MiB/s rd, 233 MiB/s wr, 529 op/s 2024-04-12T20:56:43.555 INFO:tasks.workunit.client.1.smithi186.stdout:1/873: sync 2024-04-12T20:56:43.556 INFO:tasks.workunit.client.1.smithi186.stdout:0/967: sync 2024-04-12T20:56:43.564 INFO:tasks.workunit.client.1.smithi186.stdout:4/981: read - d0/dc/d1f/d73/fb6 zero size 2024-04-12T20:56:43.565 INFO:tasks.workunit.client.1.smithi186.stdout:2/938: mkdir d0/d3/d1a/d21/d22/d12c 0 2024-04-12T20:56:43.565 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:56:43.567+0000 7f288d600700 1 -- 172.21.15.18:0/2821650228 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f28880fe0d0 msgr2=0x7f28880fe540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:56:43.565 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:56:43.567+0000 7f288d600700 1 --2- 172.21.15.18:0/2821650228 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f28880fe0d0 0x7f28880fe540 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f2874004880 tx=0x7f2874009e30 comp rx=0 tx=0).stop 2024-04-12T20:56:43.566 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:56:43 smithi018 ceph-mon[28799]: pgmap v54: 65 pgs: 65 active+clean; 3.2 GiB data, 11 GiB used, 526 GiB / 536 GiB avail; 86 MiB/s rd, 233 MiB/s wr, 529 op/s 2024-04-12T20:56:43.566 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:56:43.567+0000 7f288d600700 1 -- 172.21.15.18:0/2821650228 shutdown_connections 2024-04-12T20:56:43.566 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:56:43.567+0000 7f288d600700 1 --2- 172.21.15.18:0/2821650228 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f28880fe0d0 0x7f28880fe540 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:56:43.566 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:56:43.568+0000 7f288d600700 1 --2- 172.21.15.18:0/2821650228 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f28880fcee0 0x7f28880fd310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:56:43.566 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:56:43.568+0000 7f288d600700 1 -- 172.21.15.18:0/2821650228 >> 172.21.15.18:0/2821650228 conn(0x7f28880f8560 msgr2=0x7f28880fa970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T20:56:43.567 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:56:43.569+0000 7f288d600700 1 -- 172.21.15.18:0/2821650228 shutdown_connections 2024-04-12T20:56:43.567 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:56:43.569+0000 7f288d600700 1 -- 172.21.15.18:0/2821650228 wait complete. 2024-04-12T20:56:43.568 INFO:tasks.workunit.client.1.smithi186.stdout:7/842: dwrite d7/d10/d2d/d40/d4a/d6c/d97/d4d/f12b [0,4194304] 0 2024-04-12T20:56:43.568 INFO:tasks.workunit.client.1.smithi186.stdout:8/896: symlink d7/d18/d36/d1d/d10a/l124 0 2024-04-12T20:56:43.569 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:56:43.570+0000 7f288d600700 1 Processor -- start 2024-04-12T20:56:43.569 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:56:43.570+0000 7f288d600700 1 -- start start 2024-04-12T20:56:43.569 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:56:43.571+0000 7f288d600700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f28880fcee0 0x7f28881141b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T20:56:43.569 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:56:43.571+0000 7f288d600700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f28881146f0 0x7f2888115540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T20:56:43.570 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:56:43.571+0000 7f288d600700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2888114bb0 con 0x7f28880fcee0 2024-04-12T20:56:43.570 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:56:43.571+0000 7f288d600700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2888114cf0 con 0x7f28881146f0 2024-04-12T20:56:43.570 INFO:tasks.workunit.client.1.smithi186.stdout:0/968: symlink d0/d10/d32/dac/l13d 0 2024-04-12T20:56:43.571 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:56:43.571+0000 7f2886a00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f28880fcee0 0x7f28881141b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T20:56:43.571 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:56:43.572+0000 7f2886a00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f28880fcee0 0x7f28881141b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:42488/0 (socket says 172.21.15.18:42488) 2024-04-12T20:56:43.571 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:56:43.572+0000 7f2886a00700 1 -- 172.21.15.18:0/822084671 learned_addr learned my addr 172.21.15.18:0/822084671 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T20:56:43.571 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:56:43.572+0000 7f2886000700 1 --2- 172.21.15.18:0/822084671 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f28881146f0 0x7f2888115540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T20:56:43.571 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:56:43.572+0000 7f2886a00700 1 -- 172.21.15.18:0/822084671 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f28881146f0 msgr2=0x7f2888115540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:56:43.571 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:56:43.572+0000 7f2886a00700 1 --2- 172.21.15.18:0/822084671 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f28881146f0 0x7f2888115540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:56:43.571 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:56:43.572+0000 7f2886a00700 1 -- 172.21.15.18:0/822084671 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2874005040 con 0x7f28880fcee0 2024-04-12T20:56:43.571 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:56:43.573+0000 7f2886000700 1 --2- 172.21.15.18:0/822084671 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f28881146f0 0x7f2888115540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-12T20:56:43.571 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:56:43.573+0000 7f2886a00700 1 --2- 172.21.15.18:0/822084671 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f28880fcee0 0x7f28881141b0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f287800eb40 tx=0x7f287800b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T20:56:43.572 INFO:tasks.workunit.client.1.smithi186.stdout:6/840: dread d9/d43/f44 [0,4194304] 0 2024-04-12T20:56:43.572 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:56:43.574+0000 7f287ea00700 1 -- 172.21.15.18:0/822084671 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2878007cc0 con 0x7f28880fcee0 2024-04-12T20:56:43.573 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:56:43.574+0000 7f287ea00700 1 -- 172.21.15.18:0/822084671 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f287800f040 con 0x7f28880fcee0 2024-04-12T20:56:43.573 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:56:43.574+0000 7f287ea00700 1 -- 172.21.15.18:0/822084671 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2878009640 con 0x7f28880fcee0 2024-04-12T20:56:43.573 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:56:43.575+0000 7f288d600700 1 -- 172.21.15.18:0/822084671 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2888115ba0 con 0x7f28880fcee0 2024-04-12T20:56:43.573 INFO:tasks.workunit.client.1.smithi186.stdout:1/874: creat d14/d1a/d21/d35/dbc/de3/f11c x:0 0 0 2024-04-12T20:56:43.573 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:56:43.575+0000 7f288d600700 1 -- 172.21.15.18:0/822084671 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2888116130 con 0x7f28880fcee0 2024-04-12T20:56:43.578 INFO:tasks.workunit.client.1.smithi186.stdout:2/939: symlink d0/d3/d1a/d24/db3/l12d 0 2024-04-12T20:56:43.578 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:56:43.579+0000 7f287ea00700 1 -- 172.21.15.18:0/822084671 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 25) v1 ==== 89542+0+0 (secure 0 0 0) 0x7f28780097a0 con 0x7f28880fcee0 2024-04-12T20:56:43.578 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:56:43.579+0000 7f288d600700 1 -- 172.21.15.18:0/822084671 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f288805f5f0 con 0x7f28880fcee0 2024-04-12T20:56:43.579 INFO:tasks.workunit.client.1.smithi186.stdout:2/940: chown d0/d3/d1a/d24/fb9 3580 1 2024-04-12T20:56:43.579 INFO:tasks.workunit.client.1.smithi186.stdout:7/843: symlink d7/d10/d2d/d40/d4a/d6c/d97/dd0/l134 0 2024-04-12T20:56:43.579 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:56:43.581+0000 7f287ea00700 1 --2- 172.21.15.18:0/822084671 >> [v2:172.21.15.186:6828/2316349902,v1:172.21.15.186:6829/2316349902] conn(0x7f286c06bc90 0x7f286c06e140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T20:56:43.579 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:56:43.581+0000 7f287ea00700 1 -- 172.21.15.18:0/822084671 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5808+0+0 (secure 0 0 0) 0x7f2878089eb0 con 0x7f28880fcee0 2024-04-12T20:56:43.580 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:56:43.581+0000 7f2886000700 1 --2- 172.21.15.18:0/822084671 >> [v2:172.21.15.186:6828/2316349902,v1:172.21.15.186:6829/2316349902] conn(0x7f286c06bc90 0x7f286c06e140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T20:56:43.581 INFO:tasks.workunit.client.1.smithi186.stdout:1/875: unlink d14/d1a/d3c/f102 0 2024-04-12T20:56:43.581 INFO:tasks.workunit.client.1.smithi186.stdout:4/982: mknod d0/d59/dc0/d48/d61/d13b/c13d 0 2024-04-12T20:56:43.582 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:56:43.584+0000 7f2886000700 1 --2- 172.21.15.18:0/822084671 >> [v2:172.21.15.186:6828/2316349902,v1:172.21.15.186:6829/2316349902] conn(0x7f286c06bc90 0x7f286c06e140 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f2874009040 tx=0x7f2874009e30 comp rx=0 tx=0).ready entity=mgr.24359 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T20:56:43.583 INFO:tasks.workunit.client.1.smithi186.stdout:8/897: dread d7/d18/d36/d1d/fa0 [0,4194304] 0 2024-04-12T20:56:43.586 INFO:tasks.workunit.client.1.smithi186.stdout:7/844: read - d7/d10/d2d/d40/d4a/d6c/d97/d4d/d6a/dcc/f107 zero size 2024-04-12T20:56:43.591 INFO:tasks.workunit.client.1.smithi186.stdout:0/969: dwrite d0/d10/d32/d3f/f54 [0,4194304] 0 2024-04-12T20:56:43.593 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:56:43.595+0000 7f287ea00700 1 -- 172.21.15.18:0/822084671 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186454 (secure 0 0 0) 0x7f2878057280 con 0x7f28880fcee0 2024-04-12T20:56:43.594 INFO:tasks.workunit.client.1.smithi186.stdout:4/983: dread d0/d59/dc0/d63/da6/feb [0,4194304] 0 2024-04-12T20:56:43.594 INFO:tasks.workunit.client.1.smithi186.stdout:3/814: dread d1/d3/d21/d22/f90 [0,4194304] 0 2024-04-12T20:56:43.595 INFO:tasks.workunit.client.1.smithi186.stdout:7/845: stat d7/d10/d1c/f1f 0 2024-04-12T20:56:43.595 INFO:tasks.workunit.client.1.smithi186.stdout:6/841: write d9/d5e/d7a/f7f [5535470,107768] 0 2024-04-12T20:56:43.598 INFO:tasks.workunit.client.1.smithi186.stdout:6/842: chown d9/d43/d2d/d4d/cda 18 1 2024-04-12T20:56:43.598 INFO:tasks.workunit.client.1.smithi186.stdout:0/970: unlink d0/d10/lb7 0 2024-04-12T20:56:43.600 INFO:tasks.workunit.client.1.smithi186.stdout:1/876: link d14/d1a/d1d/f72 d14/d1a/d33/f11d 0 2024-04-12T20:56:43.601 INFO:tasks.workunit.client.1.smithi186.stdout:6/843: read d9/da9/f118 [5988525,73095] 0 2024-04-12T20:56:43.602 INFO:tasks.workunit.client.1.smithi186.stdout:4/984: creat d0/d21/db9/f13e x:0 0 0 2024-04-12T20:56:43.608 INFO:tasks.workunit.client.1.smithi186.stdout:3/815: unlink d1/d3/d21/d22/d34/df7/lbe 0 2024-04-12T20:56:43.609 INFO:tasks.workunit.client.1.smithi186.stdout:2/941: dwrite d0/d3/faf [0,4194304] 0 2024-04-12T20:56:43.610 INFO:tasks.workunit.client.1.smithi186.stdout:1/877: mkdir d14/d6c/da9/d11e 0 2024-04-12T20:56:43.613 INFO:tasks.workunit.client.1.smithi186.stdout:8/898: getdents d7/d18/d36/d1d/d7d 0 2024-04-12T20:56:43.614 INFO:tasks.workunit.client.1.smithi186.stdout:4/985: rename d0/d59/dc0/d63/d9b/l135 to d0/d26/d121/d134/l13f 0 2024-04-12T20:56:43.618 INFO:tasks.workunit.client.1.smithi186.stdout:4/986: write d0/dc/d88/d80/df8/ffa [367456,72775] 0 2024-04-12T20:56:43.619 INFO:tasks.workunit.client.1.smithi186.stdout:3/816: creat d1/d3/da2/f10e x:0 0 0 2024-04-12T20:56:43.625 INFO:tasks.workunit.client.1.smithi186.stdout:8/899: dread d7/d18/d36/d1d/d1f/d26/faf [0,4194304] 0 2024-04-12T20:56:43.628 INFO:tasks.workunit.client.1.smithi186.stdout:6/844: link d9/d43/d59/d5d/d101/ddb/fed d9/d43/d59/f129 0 2024-04-12T20:56:43.630 INFO:tasks.workunit.client.1.smithi186.stdout:1/878: getdents d14/d1a/d21/d35/d59 0 2024-04-12T20:56:43.630 INFO:tasks.workunit.client.1.smithi186.stdout:4/987: dread d0/d26/f34 [0,4194304] 0 2024-04-12T20:56:43.632 INFO:tasks.workunit.client.1.smithi186.stdout:3/817: chown d1/d3/d21/d22/da4/lba 5 1 2024-04-12T20:56:43.635 INFO:tasks.workunit.client.1.smithi186.stdout:6/845: truncate d9/d4c/d6f/d89/d9c/f105 711850 0 2024-04-12T20:56:43.636 INFO:tasks.workunit.client.1.smithi186.stdout:4/988: write d0/d59/dc0/d63/da6/dc6/f82 [1225830,59667] 0 2024-04-12T20:56:43.638 INFO:tasks.workunit.client.1.smithi186.stdout:8/900: rename d7/d18/d36/d20/d23/dd6/de8/df7/f105 to d7/d18/d36/d1d/d1f/da2/df8/d29/f125 0 2024-04-12T20:56:43.640 INFO:tasks.workunit.client.1.smithi186.stdout:7/846: dwrite d7/f12 [0,4194304] 0 2024-04-12T20:56:43.640 INFO:tasks.workunit.client.1.smithi186.stdout:4/989: dread - d0/db4/dd8/f114 zero size 2024-04-12T20:56:43.640 INFO:tasks.workunit.client.1.smithi186.stdout:0/971: dread d0/d14/d4e/d50/d6e/d96/d12a/f136 [0,4194304] 0 2024-04-12T20:56:43.643 INFO:tasks.workunit.client.1.smithi186.stdout:6/846: write d9/d43/d59/d5d/d6e/d91/dd9/ff8 [359545,28960] 0 2024-04-12T20:56:43.644 INFO:tasks.workunit.client.1.smithi186.stdout:4/990: readlink d0/d59/dc0/d48/d61/l6f 0 2024-04-12T20:56:43.647 INFO:tasks.workunit.client.1.smithi186.stdout:7/847: read d7/d55/fa4 [2945859,90110] 0 2024-04-12T20:56:43.652 INFO:tasks.workunit.client.1.smithi186.stdout:2/942: dread d0/d32/fcf [0,4194304] 0 2024-04-12T20:56:43.655 INFO:tasks.workunit.client.1.smithi186.stdout:4/991: rmdir d0/d59/dc0/d48/d61/d13b 39 2024-04-12T20:56:43.655 INFO:tasks.workunit.client.1.smithi186.stdout:0/972: mknod d0/d5c/c13e 0 2024-04-12T20:56:43.658 INFO:tasks.workunit.client.1.smithi186.stdout:7/848: mknod d7/d10/d2d/d67/c135 0 2024-04-12T20:56:43.659 INFO:tasks.workunit.client.1.smithi186.stdout:6/847: mknod d9/da9/dbd/d100/c12a 0 2024-04-12T20:56:43.661 INFO:tasks.workunit.client.1.smithi186.stdout:2/943: rmdir d0/dc/d2e/d73 39 2024-04-12T20:56:43.663 INFO:tasks.workunit.client.1.smithi186.stdout:4/992: mknod d0/dc/d64/d87/c140 0 2024-04-12T20:56:43.666 INFO:tasks.workunit.client.1.smithi186.stdout:9/806: sync 2024-04-12T20:56:43.667 INFO:tasks.workunit.client.1.smithi186.stdout:6/848: fsync d9/d4c/d6f/f9a 0 2024-04-12T20:56:43.669 INFO:tasks.workunit.client.1.smithi186.stdout:7/849: symlink d7/d55/d71/dbe/l136 0 2024-04-12T20:56:43.669 INFO:tasks.workunit.client.1.smithi186.stdout:8/901: dwrite d7/d18/d36/d20/d23/dd7/df5/d108/f115 [0,4194304] 0 2024-04-12T20:56:43.669 INFO:tasks.workunit.client.1.smithi186.stdout:0/973: creat d0/d10/d32/de2/d120/d137/f13f x:0 0 0 2024-04-12T20:56:43.672 INFO:tasks.workunit.client.1.smithi186.stdout:1/879: dwrite d14/d1a/d33/d36/d5b/f2d [0,4194304] 0 2024-04-12T20:56:43.675 INFO:tasks.workunit.client.1.smithi186.stdout:9/807: fsync d3/d2f/d54/d9f/dc4/f5f 0 2024-04-12T20:56:43.675 INFO:tasks.workunit.client.1.smithi186.stdout:3/818: dwrite d1/d3/d21/f9a [4194304,4194304] 0 2024-04-12T20:56:43.675 INFO:tasks.workunit.client.1.smithi186.stdout:6/849: fsync d9/d43/d2d/fa6 0 2024-04-12T20:56:43.682 INFO:tasks.workunit.client.1.smithi186.stdout:0/974: fdatasync d0/d5c/f6d 0 2024-04-12T20:56:43.687 INFO:tasks.workunit.client.1.smithi186.stdout:8/902: write d7/d18/d36/d20/d23/ff9 [659848,42239] 0 2024-04-12T20:56:43.687 INFO:tasks.workunit.client.1.smithi186.stdout:3/819: write d1/d3/d21/d22/da4/fe0 [475131,40995] 0 2024-04-12T20:56:43.688 INFO:tasks.workunit.client.1.smithi186.stdout:0/975: write d0/d10/d71/fb1 [1316453,87656] 0 2024-04-12T20:56:43.696 INFO:tasks.workunit.client.1.smithi186.stdout:6/850: mkdir d9/d4c/d6f/d89/d9c/d12b 0 2024-04-12T20:56:43.697 INFO:tasks.workunit.client.1.smithi186.stdout:2/944: dwrite d0/d3/f58 [0,4194304] 0 2024-04-12T20:56:43.699 INFO:tasks.workunit.client.1.smithi186.stdout:6/851: stat d9/d43/f68 0 2024-04-12T20:56:43.706 INFO:tasks.workunit.client.1.smithi186.stdout:4/993: write d0/dc/d33/f43 [4737038,50290] 0 2024-04-12T20:56:43.713 INFO:tasks.workunit.client.1.smithi186.stdout:4/994: chown d0/dc/d64/f125 0 1 2024-04-12T20:56:43.719 INFO:tasks.workunit.client.1.smithi186.stdout:8/903: dwrite d7/de/d68/da7/fec [0,4194304] 0 2024-04-12T20:56:43.719 INFO:tasks.workunit.client.1.smithi186.stdout:9/808: dwrite d3/d2f/d54/d9f/dc4/d77/dd6/d72/df4/df9/f11a [0,4194304] 0 2024-04-12T20:56:43.723 INFO:tasks.workunit.client.1.smithi186.stdout:6/852: dwrite d9/d43/d50/f5b [4194304,4194304] 0 2024-04-12T20:56:43.731 INFO:tasks.workunit.client.1.smithi186.stdout:5/846: sync 2024-04-12T20:56:43.739 INFO:tasks.workunit.client.1.smithi186.stdout:7/850: rename d7/d10/d2d/d86/dcb/f10b to d7/d10/d2d/d40/d4a/d6c/d97/d4d/d6a/f137 0 2024-04-12T20:56:43.742 INFO:tasks.workunit.client.1.smithi186.stdout:3/820: unlink d1/d2c/d48/f102 0 2024-04-12T20:56:43.745 INFO:tasks.workunit.client.1.smithi186.stdout:0/976: creat d0/d5d/ddc/f140 x:0 0 0 2024-04-12T20:56:43.748 INFO:tasks.workunit.client.1.smithi186.stdout:0/977: write d0/d10/d71/fb1 [1453651,130498] 0 2024-04-12T20:56:43.749 INFO:tasks.workunit.client.1.smithi186.stdout:4/995: mkdir d0/d59/dc0/d63/da6/dc6/d141 0 2024-04-12T20:56:43.755 INFO:tasks.workunit.client.1.smithi186.stdout:9/809: unlink d3/d2f/d2e/fc7 0 2024-04-12T20:56:43.756 INFO:tasks.workunit.client.1.smithi186.stdout:3/821: dread d1/d3/d21/d22/da4/fe0 [0,4194304] 0 2024-04-12T20:56:43.759 INFO:tasks.workunit.client.1.smithi186.stdout:2/945: rename d0/dd3 to d0/d32/d2a/d91/d12e 0 2024-04-12T20:56:43.761 INFO:tasks.workunit.client.1.smithi186.stdout:0/978: creat d0/d10/d32/f141 x:0 0 0 2024-04-12T20:56:43.762 INFO:tasks.workunit.client.1.smithi186.stdout:9/810: truncate d3/d2f/d2e/d33/d36/f53 5429251 0 2024-04-12T20:56:43.763 INFO:tasks.workunit.client.1.smithi186.stdout:1/880: link d14/d1a/d33/d36/d5b/d3e/l92 d14/d91/dbd/dc5/dc8/l11f 0 2024-04-12T20:56:43.765 INFO:tasks.workunit.client.1.smithi186.stdout:3/822: write d1/f1e [1364227,22862] 0 2024-04-12T20:56:43.767 INFO:tasks.workunit.client.1.smithi186.stdout:4/996: dwrite d0/d21/f41 [0,4194304] 0 2024-04-12T20:56:43.770 INFO:tasks.workunit.client.1.smithi186.stdout:8/904: creat d7/d18/d36/d1d/d1f/f126 x:0 0 0 2024-04-12T20:56:43.773 INFO:tasks.workunit.client.1.smithi186.stdout:4/997: write d0/dc/d33/f17 [295425,83077] 0 2024-04-12T20:56:43.775 INFO:tasks.workunit.client.1.smithi186.stdout:6/853: creat d9/d43/d2d/f12c x:0 0 0 2024-04-12T20:56:43.777 INFO:tasks.workunit.client.1.smithi186.stdout:1/881: readlink d14/d74/l88 0 2024-04-12T20:56:43.778 INFO:tasks.workunit.client.1.smithi186.stdout:8/905: creat d7/d18/d36/d20/d23/dd6/f127 x:0 0 0 2024-04-12T20:56:43.779 INFO:tasks.workunit.client.1.smithi186.stdout:9/811: creat d3/d2f/d10e/f11b x:0 0 0 2024-04-12T20:56:43.780 INFO:tasks.workunit.client.1.smithi186.stdout:6/854: write d9/d43/d2d/f7e [1294623,65996] 0 2024-04-12T20:56:43.780 INFO:tasks.workunit.client.1.smithi186.stdout:4/998: creat d0/dc/d64/d87/d120/d10a/f142 x:0 0 0 2024-04-12T20:56:43.786 INFO:tasks.workunit.client.1.smithi186.stdout:9/812: write d3/d2f/d103/ff6 [454492,45222] 0 2024-04-12T20:56:43.786 INFO:tasks.workunit.client.1.smithi186.stdout:8/906: chown d7/de/d7a/f31 16735639 1 2024-04-12T20:56:43.786 INFO:tasks.workunit.client.1.smithi186.stdout:3/823: creat d1/d3/d21/d22/d24/f10f x:0 0 0 2024-04-12T20:56:43.788 INFO:tasks.workunit.client.1.smithi186.stdout:4/999: symlink d0/d59/dc0/l143 0 2024-04-12T20:56:43.792 INFO:tasks.workunit.client.1.smithi186.stdout:8/907: symlink d7/de/d5e/l128 0 2024-04-12T20:56:43.793 INFO:tasks.workunit.client.1.smithi186.stdout:6/855: mknod d9/d4c/d6f/d89/d9c/d12b/c12d 0 2024-04-12T20:56:43.795 INFO:tasks.workunit.client.1.smithi186.stdout:6/856: dread - d9/d43/d59/d5d/d101/ddb/fed zero size 2024-04-12T20:56:43.795 INFO:tasks.workunit.client.1.smithi186.stdout:0/979: dread d0/d14/f35 [0,4194304] 0 2024-04-12T20:56:43.797 INFO:tasks.workunit.client.1.smithi186.stdout:3/824: symlink d1/d3/d21/d22/d34/ddb/d4d/d6f/da6/l110 0 2024-04-12T20:56:43.800 INFO:tasks.workunit.client.1.smithi186.stdout:9/813: symlink d3/d2f/d2e/d43/d4c/d82/dfd/l11c 0 2024-04-12T20:56:43.801 INFO:tasks.workunit.client.1.smithi186.stdout:9/814: readlink d3/d2f/d54/l84 0 2024-04-12T20:56:43.803 INFO:tasks.workunit.client.1.smithi186.stdout:8/908: mkdir d7/de/d10/d11d/d129 0 2024-04-12T20:56:43.806 INFO:tasks.workunit.client.1.smithi186.stdout:0/980: rmdir d0/d14/d4e/d1e 39 2024-04-12T20:56:43.812 INFO:tasks.workunit.client.1.smithi186.stdout:9/815: fsync d3/d2f/d2e/d43/f8f 0 2024-04-12T20:56:43.814 INFO:tasks.workunit.client.1.smithi186.stdout:8/909: symlink d7/d18/def/l12a 0 2024-04-12T20:56:43.819 INFO:tasks.workunit.client.1.smithi186.stdout:2/946: dwrite d0/d3e/fb1 [0,4194304] 0 2024-04-12T20:56:43.821 INFO:tasks.workunit.client.1.smithi186.stdout:3/825: truncate d1/d2c/d64/f9b 3944131 0 2024-04-12T20:56:43.824 INFO:tasks.workunit.client.1.smithi186.stdout:9/816: chown d3/c3a 204154 1 2024-04-12T20:56:43.827 INFO:tasks.workunit.client.1.smithi186.stdout:8/910: creat d7/d18/d8e/f12b x:0 0 0 2024-04-12T20:56:43.828 INFO:tasks.workunit.client.1.smithi186.stdout:1/882: truncate d14/d1a/d21/d35/d59/fd2 6667284 0 2024-04-12T20:56:43.829 INFO:tasks.workunit.client.1.smithi186.stdout:1/883: truncate d14/d6c/fcb 682036 0 2024-04-12T20:56:43.831 INFO:tasks.workunit.client.1.smithi186.stdout:6/857: link d9/d43/d59/d5d/d101/ddb/cdf d9/d5e/d7a/d81/dc5/c12e 0 2024-04-12T20:56:43.831 INFO:tasks.workunit.client.1.smithi186.stdout:2/947: fdatasync d0/d3/d1a/d27/fd0 0 2024-04-12T20:56:43.833 INFO:tasks.workunit.client.1.smithi186.stdout:3/826: dread - d1/d3/d21/d22/d24/fb4 zero size 2024-04-12T20:56:43.834 INFO:tasks.workunit.client.1.smithi186.stdout:3/827: truncate d1/d3/da/f10a 351482 0 2024-04-12T20:56:43.839 INFO:tasks.workunit.client.1.smithi186.stdout:1/884: creat d14/d1a/d21/d35/d107/f120 x:0 0 0 2024-04-12T20:56:43.842 INFO:tasks.workunit.client.1.smithi186.stdout:1/885: chown d14/d74/cb1 216575 1 2024-04-12T20:56:43.842 INFO:tasks.workunit.client.1.smithi186.stdout:0/981: getdents d0/d10/d32/d3f 0 2024-04-12T20:56:43.843 INFO:tasks.workunit.client.1.smithi186.stdout:2/948: unlink d0/dc/d2e/de8/ceb 0 2024-04-12T20:56:43.843 INFO:tasks.workunit.client.1.smithi186.stdout:8/911: dwrite d7/d18/d36/d20/d23/dd6/de8/f100 [0,4194304] 0 2024-04-12T20:56:43.844 INFO:tasks.workunit.client.1.smithi186.stdout:3/828: mkdir d1/d3/d21/d22/d34/ddb/d4d/d6f/db0/d111 0 2024-04-12T20:56:43.847 INFO:tasks.workunit.client.1.smithi186.stdout:0/982: write d0/d85/dc4/dc1/fd3 [1532520,118287] 0 2024-04-12T20:56:43.848 INFO:tasks.workunit.client.1.smithi186.stdout:3/829: dread - d1/d3/df8/dd0/d10b/fd4 zero size 2024-04-12T20:56:43.848 INFO:tasks.workunit.client.1.smithi186.stdout:5/847: sync 2024-04-12T20:56:43.849 INFO:tasks.workunit.client.1.smithi186.stdout:2/949: rename d0/d3 to d0/d3/d1a/d24/db3/d12f 22 2024-04-12T20:56:43.852 INFO:tasks.workunit.client.1.smithi186.stdout:2/950: dread - d0/d32/d8b/f109 zero size 2024-04-12T20:56:43.859 INFO:tasks.workunit.client.1.smithi186.stdout:3/830: rmdir d1/d2c/d48/ded 39 2024-04-12T20:56:43.860 INFO:tasks.workunit.client.1.smithi186.stdout:2/951: creat d0/d3/d1a/d24/db3/f130 x:0 0 0 2024-04-12T20:56:43.865 INFO:tasks.workunit.client.1.smithi186.stdout:3/831: readlink d1/d3/d21/d22/d34/ddb/d4d/d6f/laa 0 2024-04-12T20:56:43.866 INFO:tasks.workunit.client.1.smithi186.stdout:9/817: dwrite d3/d2f/d54/d9f/dc4/d77/dd6/d72/fe1 [0,4194304] 0 2024-04-12T20:56:43.868 INFO:tasks.workunit.client.1.smithi186.stdout:8/912: dwrite d7/d18/d36/d20/d23/fe4 [0,4194304] 0 2024-04-12T20:56:43.870 INFO:tasks.workunit.client.1.smithi186.stdout:3/832: stat d1/d3/d21/d22/d24/cd8 0 2024-04-12T20:56:43.876 INFO:tasks.workunit.client.1.smithi186.stdout:3/833: creat d1/d3/d21/d5c/f112 x:0 0 0 2024-04-12T20:56:43.877 INFO:tasks.workunit.client.1.smithi186.stdout:2/952: write d0/dc/d2e/d73/fd8 [2173867,69619] 0 2024-04-12T20:56:43.881 INFO:tasks.workunit.client.1.smithi186.stdout:6/858: dwrite d9/d4c/fb3 [0,4194304] 0 2024-04-12T20:56:43.883 INFO:tasks.workunit.client.1.smithi186.stdout:9/818: getdents d3/d2f/d2e/db3 0 2024-04-12T20:56:43.886 INFO:tasks.workunit.client.1.smithi186.stdout:1/886: dwrite d14/d1a/d58/d69/fa4 [4194304,4194304] 0 2024-04-12T20:56:43.887 INFO:tasks.workunit.client.1.smithi186.stdout:6/859: mknod d9/da9/dbd/d100/c12f 0 2024-04-12T20:56:43.888 INFO:tasks.workunit.client.1.smithi186.stdout:8/913: getdents d7/de/d10/d11d 0 2024-04-12T20:56:43.891 INFO:tasks.workunit.client.1.smithi186.stdout:8/914: read - d7/d18/d8e/f12b zero size 2024-04-12T20:56:43.893 INFO:tasks.workunit.client.1.smithi186.stdout:1/887: write d14/d1a/d33/d36/dfa/f101 [574695,73838] 0 2024-04-12T20:56:43.897 INFO:tasks.workunit.client.1.smithi186.stdout:7/851: sync 2024-04-12T20:56:43.904 INFO:tasks.workunit.client.1.smithi186.stdout:6/860: creat d9/d5e/d7a/d81/f130 x:0 0 0 2024-04-12T20:56:43.904 INFO:tasks.workunit.client.1.smithi186.stdout:2/953: rename d0/d3/d7a to d0/dc/d2e/d131 0 2024-04-12T20:56:43.905 INFO:tasks.workunit.client.1.smithi186.stdout:8/915: creat d7/d18/d36/d1d/dc3/f12c x:0 0 0 2024-04-12T20:56:43.907 INFO:tasks.workunit.client.1.smithi186.stdout:7/852: rmdir d7/d10/d1c/d8c 39 2024-04-12T20:56:43.914 INFO:tasks.workunit.client.1.smithi186.stdout:8/916: creat d7/de/d10/d7f/dd3/f12d x:0 0 0 2024-04-12T20:56:43.915 INFO:tasks.workunit.client.1.smithi186.stdout:2/954: truncate d0/d32/f97 1444303 0 2024-04-12T20:56:43.917 INFO:tasks.workunit.client.1.smithi186.stdout:2/955: readlink d0/d3/d1a/d24/lf0 0 2024-04-12T20:56:43.918 INFO:tasks.workunit.client.1.smithi186.stdout:6/861: rename d9/d43/f45 to d9/d43/d59/d5d/d6e/d91/f131 0 2024-04-12T20:56:43.920 INFO:tasks.workunit.client.1.smithi186.stdout:6/862: write d9/d43/d50/f55 [3681466,8585] 0 2024-04-12T20:56:43.926 INFO:tasks.workunit.client.1.smithi186.stdout:2/956: dread d0/d32/d69/f7d [0,4194304] 0 2024-04-12T20:56:43.929 INFO:tasks.workunit.client.1.smithi186.stdout:8/917: dwrite d7/de/d68/da7/fec [0,4194304] 0 2024-04-12T20:56:43.930 INFO:tasks.workunit.client.1.smithi186.stdout:7/853: mknod d7/d10/d1c/d8c/c138 0 2024-04-12T20:56:43.934 INFO:tasks.workunit.client.1.smithi186.stdout:0/983: write d0/d14/d4e/d1e/f56 [3256975,116229] 0 2024-04-12T20:56:43.936 INFO:tasks.workunit.client.1.smithi186.stdout:0/984: chown d0/f6 7275 1 2024-04-12T20:56:43.936 INFO:tasks.workunit.client.1.smithi186.stdout:3/834: dread d1/d2c/d64/f9b [0,4194304] 0 2024-04-12T20:56:43.942 INFO:tasks.workunit.client.1.smithi186.stdout:8/918: unlink d7/de/d7a/fd9 0 2024-04-12T20:56:43.945 INFO:tasks.workunit.client.1.smithi186.stdout:8/919: fsync d7/d18/d36/d1d/d1f/da2/f94 0 2024-04-12T20:56:43.946 INFO:tasks.workunit.client.1.smithi186.stdout:7/854: symlink d7/d10/d2d/d40/d4a/d6c/d97/dc2/de2/l139 0 2024-04-12T20:56:43.947 INFO:tasks.workunit.client.1.smithi186.stdout:5/848: dwrite d5/db/d15/d2a/d4c/d5b/d7a/d94/dfa/f4b [0,4194304] 0 2024-04-12T20:56:43.952 INFO:tasks.workunit.client.1.smithi186.stdout:5/849: chown d5/db/d2e/d70/d101/d7f/fb7 63892824 1 2024-04-12T20:56:43.954 INFO:tasks.workunit.client.1.smithi186.stdout:1/888: link d14/d1a/d33/d36/d5b/d3e/d5f/c100 d14/d1a/d33/d36/d5b/d3e/d8b/c121 0 2024-04-12T20:56:43.957 INFO:tasks.workunit.client.1.smithi186.stdout:7/855: creat d7/d55/d71/d10a/f13a x:0 0 0 2024-04-12T20:56:43.958 INFO:tasks.workunit.client.1.smithi186.stdout:2/957: dwrite d0/d3/d1a/d48/df7/faa [0,4194304] 0 2024-04-12T20:56:43.959 INFO:tasks.workunit.client.1.smithi186.stdout:8/920: creat d7/d18/d8e/d10c/f12e x:0 0 0 2024-04-12T20:56:43.962 INFO:tasks.workunit.client.1.smithi186.stdout:0/985: creat d0/d5c/d13a/f142 x:0 0 0 2024-04-12T20:56:43.966 INFO:tasks.workunit.client.1.smithi186.stdout:5/850: rmdir d5/db/d2e/d70/d101/dbb/dc4/d5d/de1 39 2024-04-12T20:56:43.971 INFO:tasks.workunit.client.1.smithi186.stdout:2/958: symlink d0/d3/d1a/d24/db3/l132 0 2024-04-12T20:56:43.973 INFO:tasks.workunit.client.1.smithi186.stdout:9/819: dwrite d3/d2f/f5e [0,4194304] 0 2024-04-12T20:56:43.979 INFO:tasks.workunit.client.1.smithi186.stdout:8/921: mkdir d7/de/d7a/d12f 0 2024-04-12T20:56:43.981 INFO:tasks.workunit.client.1.smithi186.stdout:8/922: stat d7/d18/d36/d1d/d1f/da2/f79 0 2024-04-12T20:56:43.983 INFO:tasks.workunit.client.1.smithi186.stdout:8/923: stat d7/de/d7a/f80 0 2024-04-12T20:56:43.989 INFO:tasks.workunit.client.1.smithi186.stdout:0/986: link d0/d14/d4e/d50/f110 d0/d14/d105/f143 0 2024-04-12T20:56:43.990 INFO:tasks.workunit.client.1.smithi186.stdout:6/863: dwrite d9/f22 [0,4194304] 0 2024-04-12T20:56:43.994 INFO:tasks.workunit.client.1.smithi186.stdout:3/835: write d1/d3/d21/d22/f4c [2376035,100000] 0 2024-04-12T20:56:43.995 INFO:tasks.workunit.client.1.smithi186.stdout:9/820: read d3/d2f/d54/d9f/fa5 [22577,43691] 0 2024-04-12T20:56:43.999 INFO:tasks.workunit.client.1.smithi186.stdout:6/864: mkdir d9/d43/d2d/d4d/d132 0 2024-04-12T20:56:44.005 INFO:tasks.workunit.client.1.smithi186.stdout:2/959: rename d0/d3/d1a/d21/d22/c8c to d0/d3/d1a/d21/c133 0 2024-04-12T20:56:44.006 INFO:tasks.workunit.client.1.smithi186.stdout:3/836: creat d1/d2c/d64/f113 x:0 0 0 2024-04-12T20:56:44.006 INFO:tasks.workunit.client.1.smithi186.stdout:0/987: symlink d0/d14/d4e/d1e/l144 0 2024-04-12T20:56:44.012 INFO:tasks.workunit.client.1.smithi186.stdout:2/960: write d0/dc/d2e/d131/f94 [10541,108588] 0 2024-04-12T20:56:44.015 INFO:tasks.workunit.client.1.smithi186.stdout:7/856: dwrite d7/d10/d2d/f37 [0,4194304] 0 2024-04-12T20:56:44.016 INFO:tasks.workunit.client.1.smithi186.stdout:2/961: chown d0/d3/d1a/d21/d26/f9a 0 1 2024-04-12T20:56:44.019 INFO:tasks.workunit.client.1.smithi186.stdout:9/821: rename d3/f1d to d3/d2f/f11d 0 2024-04-12T20:56:44.021 INFO:tasks.workunit.client.1.smithi186.stdout:2/962: truncate d0/d3/d1a/d24/db3/f130 437045 0 2024-04-12T20:56:44.024 INFO:tasks.workunit.client.1.smithi186.stdout:2/963: chown d0/d3/d1a/cda 11809754 1 2024-04-12T20:56:44.026 INFO:tasks.workunit.client.1.smithi186.stdout:2/964: chown d0/d3/d1a/d48/df7/d121/c127 4185278 1 2024-04-12T20:56:44.026 INFO:tasks.workunit.client.1.smithi186.stdout:3/837: dwrite d1/d3/df8/dd0/d10b/fd3 [0,4194304] 0 2024-04-12T20:56:44.027 INFO:tasks.workunit.client.1.smithi186.stdout:9/822: creat d3/d2f/d2e/d43/d4c/d82/dfd/f11e x:0 0 0 2024-04-12T20:56:44.028 INFO:tasks.workunit.client.1.smithi186.stdout:3/838: readlink d1/d2c/d48/l85 0 2024-04-12T20:56:44.030 INFO:tasks.workunit.client.1.smithi186.stdout:7/857: getdents d7/d10/d2d/d40 0 2024-04-12T20:56:44.031 INFO:tasks.workunit.client.1.smithi186.stdout:3/839: dread - d1/d3/df8/ddc/fec zero size 2024-04-12T20:56:44.032 INFO:tasks.workunit.client.1.smithi186.stdout:5/851: write d5/db/d15/d2a/d2d/d46/f6c [465792,40112] 0 2024-04-12T20:56:44.034 INFO:tasks.workunit.client.1.smithi186.stdout:2/965: creat d0/d3/d1a/d21/d118/f134 x:0 0 0 2024-04-12T20:56:44.036 INFO:tasks.workunit.client.1.smithi186.stdout:3/840: truncate d1/d3/d21/d22/d4a/feb 282958 0 2024-04-12T20:56:44.037 INFO:tasks.workunit.client.1.smithi186.stdout:9/823: mkdir d3/d2f/d54/d9f/dc4/d77/d11f 0 2024-04-12T20:56:44.038 INFO:tasks.workunit.client.1.smithi186.stdout:9/824: fsync d3/d13/d10b/ff5 0 2024-04-12T20:56:44.039 INFO:tasks.workunit.client.1.smithi186.stdout:7/858: mkdir d7/d10/d2d/d40/d4a/d5c/d13b 0 2024-04-12T20:56:44.040 INFO:tasks.workunit.client.1.smithi186.stdout:9/825: stat d3/d2f/d2e/d33/l108 0 2024-04-12T20:56:44.042 INFO:tasks.workunit.client.1.smithi186.stdout:3/841: dread d1/d3/df8/d95/ff3 [0,4194304] 0 2024-04-12T20:56:44.043 INFO:tasks.workunit.client.1.smithi186.stdout:0/988: write d0/d10/fa8 [600008,123290] 0 2024-04-12T20:56:44.044 INFO:tasks.workunit.client.1.smithi186.stdout:6/865: write d9/d43/d50/f79 [358209,53180] 0 2024-04-12T20:56:44.045 INFO:tasks.workunit.client.1.smithi186.stdout:7/859: mknod d7/d10/d2d/d40/d4a/d5c/c13c 0 2024-04-12T20:56:44.049 INFO:tasks.workunit.client.1.smithi186.stdout:2/966: dread d0/d32/d69/f7d [0,4194304] 0 2024-04-12T20:56:44.050 INFO:tasks.workunit.client.1.smithi186.stdout:3/842: write d1/d3/d21/f9a [131723,28852] 0 2024-04-12T20:56:44.050 INFO:tasks.workunit.client.1.smithi186.stdout:6/866: chown d9/f12 510620 1 2024-04-12T20:56:44.052 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:56:44.052+0000 7f288d600700 1 -- 172.21.15.18:0/822084671 --> [v2:172.21.15.186:6828/2316349902,v1:172.21.15.186:6829/2316349902] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f2888071470 con 0x7f286c06bc90 2024-04-12T20:56:44.053 INFO:tasks.workunit.client.1.smithi186.stdout:2/967: stat d0/d3/d1a/d48/df7/d121 0 2024-04-12T20:56:44.055 INFO:tasks.workunit.client.1.smithi186.stdout:5/852: getdents d5/db/d15 0 2024-04-12T20:56:44.055 INFO:tasks.workunit.client.1.smithi186.stdout:3/843: dread - d1/d3/df8/dd0/d10b/fd4 zero size 2024-04-12T20:56:44.057 INFO:tasks.workunit.client.1.smithi186.stdout:6/867: mkdir d9/d4c/de9/d133 0 2024-04-12T20:56:44.060 INFO:tasks.workunit.client.1.smithi186.stdout:2/968: mkdir d0/d3/dce/ddf/d135 0 2024-04-12T20:56:44.062 INFO:tasks.workunit.client.1.smithi186.stdout:2/969: write d0/d3/d1a/d21/d26/d86/f11f [301541,28795] 0 2024-04-12T20:56:44.062 INFO:tasks.workunit.client.1.smithi186.stdout:0/989: dwrite d0/d14/d4e/d1e/d84/fcd [0,4194304] 0 2024-04-12T20:56:44.066 INFO:tasks.workunit.client.1.smithi186.stdout:1/889: sync 2024-04-12T20:56:44.067 INFO:teuthology.orchestra.run.smithi018.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-12T20:56:44.067 INFO:teuthology.orchestra.run.smithi018.stdout:alertmanager.smithi018 smithi018 *:9093,9094 starting - - - - 2024-04-12T20:56:44.067 INFO:teuthology.orchestra.run.smithi018.stdout:crash.smithi018 smithi018 running (12m) 85s ago 12m 6660k - 17.2.7-827-gd3b5060a 21de9b7369ad 1ac7603bddfd 2024-04-12T20:56:44.068 INFO:teuthology.orchestra.run.smithi018.stdout:crash.smithi186 smithi186 running (10m) 92s ago 10m 6669k - 17.2.7-827-gd3b5060a 21de9b7369ad 14bf8a1672eb 2024-04-12T20:56:44.068 INFO:teuthology.orchestra.run.smithi018.stdout:grafana.smithi018 smithi018 *:3000 starting - - - - 2024-04-12T20:56:44.068 INFO:teuthology.orchestra.run.smithi018.stdout:mds.cephfs.smithi018.acqbsc smithi018 running (4m) 85s ago 4m 69.0M - 17.2.7-827-gd3b5060a 21de9b7369ad 5b5fce65f18a 2024-04-12T20:56:44.068 INFO:teuthology.orchestra.run.smithi018.stdout:mds.cephfs.smithi018.szjluo smithi018 running (4m) 85s ago 4m 65.4M - 17.2.7-827-gd3b5060a 21de9b7369ad cf9bf0c5031f 2024-04-12T20:56:44.068 INFO:teuthology.orchestra.run.smithi018.stdout:mds.cephfs.smithi186.jdllyn smithi186 running (4m) 92s ago 4m 11.8M - 17.2.7-827-gd3b5060a 21de9b7369ad 2b2bc46bd29a 2024-04-12T20:56:44.068 INFO:teuthology.orchestra.run.smithi018.stdout:mds.cephfs.smithi186.wjnafs smithi186 running (4m) 92s ago 4m 11.9M - 17.2.7-827-gd3b5060a 21de9b7369ad 74b115d6d6fd 2024-04-12T20:56:44.068 INFO:teuthology.orchestra.run.smithi018.stdout:mgr.smithi018.xjzwga smithi018 *:8443,9283,8765 starting - - - - 2024-04-12T20:56:44.068 INFO:teuthology.orchestra.run.smithi018.stdout:mgr.smithi186.kqcfxk smithi186 *:8443,9283 running (2m) 92s ago 10m 492M - 19.0.0-2879-g324baf8d 8717720dc5e5 bd4971d5260e 2024-04-12T20:56:44.068 INFO:teuthology.orchestra.run.smithi018.stdout:mon.smithi018 smithi018 running (13m) 85s ago 13m 47.1M 2048M 17.2.7-827-gd3b5060a 21de9b7369ad cd84caafa056 2024-04-12T20:56:44.068 INFO:teuthology.orchestra.run.smithi018.stdout:mon.smithi186 smithi186 running (10m) 92s ago 10m 37.0M 2048M 17.2.7-827-gd3b5060a 21de9b7369ad d7263c772b73 2024-04-12T20:56:44.068 INFO:teuthology.orchestra.run.smithi018.stdout:node-exporter.smithi018 smithi018 *:9100 starting - - - - 2024-04-12T20:56:44.068 INFO:teuthology.orchestra.run.smithi018.stdout:node-exporter.smithi186 smithi186 *:9100 starting - - - - 2024-04-12T20:56:44.068 INFO:teuthology.orchestra.run.smithi018.stdout:osd.0 smithi018 running (9m) 85s ago 9m 83.2M 1559M 17.2.7-827-gd3b5060a 21de9b7369ad 0d83e027bf0a 2024-04-12T20:56:44.068 INFO:teuthology.orchestra.run.smithi018.stdout:osd.1 smithi018 running (8m) 85s ago 8m 90.2M 1559M 17.2.7-827-gd3b5060a 21de9b7369ad 6517c9cc7f87 2024-04-12T20:56:44.068 INFO:teuthology.orchestra.run.smithi018.stdout:osd.2 smithi018 running (8m) 85s ago 8m 75.7M 1559M 17.2.7-827-gd3b5060a 21de9b7369ad 8a83692acaa5 2024-04-12T20:56:44.069 INFO:teuthology.orchestra.run.smithi018.stdout:osd.3 smithi186 running (7m) 92s ago 7m 88.7M 2581M 17.2.7-827-gd3b5060a 21de9b7369ad c2def7a90d76 2024-04-12T20:56:44.069 INFO:teuthology.orchestra.run.smithi018.stdout:osd.4 smithi186 running (7m) 92s ago 7m 84.8M 2581M 17.2.7-827-gd3b5060a 21de9b7369ad 757e7c71b7b9 2024-04-12T20:56:44.069 INFO:teuthology.orchestra.run.smithi018.stdout:osd.5 smithi186 running (6m) 92s ago 6m 80.8M 2581M 17.2.7-827-gd3b5060a 21de9b7369ad 2ee072a436d0 2024-04-12T20:56:44.069 INFO:teuthology.orchestra.run.smithi018.stdout:prometheus.smithi018 smithi018 *:9095 starting - - - - 2024-04-12T20:56:44.069 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:56:44.067+0000 7f287ea00700 1 -- 172.21.15.18:0/822084671 <== mgr.24359 v2:172.21.15.186:6828/2316349902 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7f2888071470 con 0x7f286c06bc90 2024-04-12T20:56:44.071 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:56:44.072+0000 7f288d600700 1 -- 172.21.15.18:0/822084671 >> [v2:172.21.15.186:6828/2316349902,v1:172.21.15.186:6829/2316349902] conn(0x7f286c06bc90 msgr2=0x7f286c06e140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:56:44.071 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:56:44.072+0000 7f288d600700 1 --2- 172.21.15.18:0/822084671 >> [v2:172.21.15.186:6828/2316349902,v1:172.21.15.186:6829/2316349902] conn(0x7f286c06bc90 0x7f286c06e140 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f2874009040 tx=0x7f2874009e30 comp rx=0 tx=0).stop 2024-04-12T20:56:44.071 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:56:44.072+0000 7f288d600700 1 -- 172.21.15.18:0/822084671 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f28880fcee0 msgr2=0x7f28881141b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:56:44.071 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:56:44.072+0000 7f288d600700 1 --2- 172.21.15.18:0/822084671 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f28880fcee0 0x7f28881141b0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f287800eb40 tx=0x7f287800b040 comp rx=0 tx=0).stop 2024-04-12T20:56:44.071 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:56:44.072+0000 7f288d600700 1 -- 172.21.15.18:0/822084671 shutdown_connections 2024-04-12T20:56:44.071 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:56:44.072+0000 7f288d600700 1 --2- 172.21.15.18:0/822084671 >> [v2:172.21.15.186:6828/2316349902,v1:172.21.15.186:6829/2316349902] conn(0x7f286c06bc90 0x7f286c06e140 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:56:44.071 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:56:44.072+0000 7f288d600700 1 --2- 172.21.15.18:0/822084671 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f28880fcee0 0x7f28881141b0 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:56:44.071 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:56:44.072+0000 7f288d600700 1 --2- 172.21.15.18:0/822084671 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f28881146f0 0x7f2888115540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:56:44.071 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:56:44.072+0000 7f288d600700 1 -- 172.21.15.18:0/822084671 >> 172.21.15.18:0/822084671 conn(0x7f28880f8560 msgr2=0x7f28880f9dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T20:56:44.071 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:56:44.073+0000 7f288d600700 1 -- 172.21.15.18:0/822084671 shutdown_connections 2024-04-12T20:56:44.072 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:56:44.073+0000 7f288d600700 1 -- 172.21.15.18:0/822084671 wait complete. 2024-04-12T20:56:44.079 INFO:tasks.workunit.client.1.smithi186.stdout:5/853: fdatasync d5/db/d2e/d70/d101/dbb/dc4/fa9 0 2024-04-12T20:56:44.079 INFO:tasks.workunit.client.1.smithi186.stdout:9/826: truncate d3/fd 3191994 0 2024-04-12T20:56:44.079 INFO:tasks.workunit.client.1.smithi186.stdout:6/868: mkdir d9/d5e/d73/d134 0 2024-04-12T20:56:44.082 INFO:tasks.workunit.client.1.smithi186.stdout:9/827: chown d3/d2f/d54/d9f/dc4/d77/la2 0 1 2024-04-12T20:56:44.086 INFO:tasks.workunit.client.1.smithi186.stdout:5/854: read d5/db/d2e/d70/d101/dbb/dc4/fa9 [1940341,36804] 0 2024-04-12T20:56:44.090 INFO:tasks.workunit.client.1.smithi186.stdout:7/860: link d7/d10/d2d/d40/d4a/d6c/d97/d4d/d8a/dad/fe0 d7/d10/d1c/d5d/f13d 0 2024-04-12T20:56:44.092 INFO:tasks.workunit.client.1.smithi186.stdout:1/890: mkdir d14/d91/dbd/d122 0 2024-04-12T20:56:44.095 INFO:tasks.workunit.client.1.smithi186.stdout:7/861: write d7/d10/d2d/d40/d4a/d6c/d97/d4d/d6a/dcc/f107 [462268,128247] 0 2024-04-12T20:56:44.105 INFO:tasks.workunit.client.1.smithi186.stdout:2/970: dwrite d0/dc/d2e/fdd [0,4194304] 0 2024-04-12T20:56:44.107 INFO:tasks.workunit.client.1.smithi186.stdout:7/862: readlink d7/d10/d1c/d5d/lea 0 2024-04-12T20:56:44.108 INFO:tasks.workunit.client.1.smithi186.stdout:0/990: getdents d0/d10/dda 0 2024-04-12T20:56:44.110 INFO:tasks.workunit.client.1.smithi186.stdout:5/855: dwrite d5/db/d15/d2a/d4c/f7b [0,4194304] 0 2024-04-12T20:56:44.119 INFO:tasks.workunit.client.1.smithi186.stdout:7/863: fsync d7/d10/d2d/f58 0 2024-04-12T20:56:44.129 INFO:tasks.workunit.client.1.smithi186.stdout:1/891: dwrite d14/d1a/d21/fc6 [0,4194304] 0 2024-04-12T20:56:44.132 INFO:tasks.workunit.client.1.smithi186.stdout:6/869: dwrite d9/da9/fc1 [0,4194304] 0 2024-04-12T20:56:44.134 INFO:tasks.workunit.client.1.smithi186.stdout:0/991: getdents d0/d14/d4e/d50/d6e 0 2024-04-12T20:56:44.146 INFO:tasks.workunit.client.1.smithi186.stdout:0/992: read d0/d10/f2a [3883015,86685] 0 2024-04-12T20:56:44.146 INFO:tasks.workunit.client.1.smithi186.stdout:1/892: mknod d14/d91/c123 0 2024-04-12T20:56:44.146 INFO:tasks.workunit.client.1.smithi186.stdout:5/856: dwrite d5/db/d2e/d70/f99 [0,4194304] 0 2024-04-12T20:56:44.146 INFO:tasks.workunit.client.1.smithi186.stdout:2/971: dwrite d0/dc/f4c [0,4194304] 0 2024-04-12T20:56:44.146 INFO:tasks.workunit.client.1.smithi186.stdout:6/870: mknod d9/d5e/d7a/d81/dc5/c135 0 2024-04-12T20:56:44.150 INFO:tasks.workunit.client.1.smithi186.stdout:1/893: truncate d14/d1a/d33/d36/ffb 884621 0 2024-04-12T20:56:44.150 INFO:tasks.workunit.client.1.smithi186.stdout:0/993: unlink d0/d10/d71/f89 0 2024-04-12T20:56:44.162 INFO:tasks.workunit.client.1.smithi186.stdout:7/864: link d7/d10/d1c/lc7 d7/d10/d1c/l13e 0 2024-04-12T20:56:44.163 INFO:tasks.workunit.client.1.smithi186.stdout:6/871: creat d9/d5e/d73/d134/f136 x:0 0 0 2024-04-12T20:56:44.166 INFO:tasks.workunit.client.1.smithi186.stdout:1/894: read - d14/d6c/def/fa7 zero size 2024-04-12T20:56:44.175 INFO:tasks.workunit.client.1.smithi186.stdout:1/895: creat d14/d1a/d1d/de5/f124 x:0 0 0 2024-04-12T20:56:44.177 INFO:tasks.workunit.client.1.smithi186.stdout:6/872: link d9/d43/d59/c65 d9/da9/dbd/c137 0 2024-04-12T20:56:44.182 INFO:tasks.workunit.client.1.smithi186.stdout:1/896: read d14/d1a/d33/d36/d5b/d3e/d71/f78 [3833657,50094] 0 2024-04-12T20:56:44.183 INFO:tasks.workunit.client.1.smithi186.stdout:6/873: mknod d9/d43/d2d/c138 0 2024-04-12T20:56:44.187 INFO:tasks.workunit.client.1.smithi186.stdout:1/897: mknod d14/d1a/d58/c125 0 2024-04-12T20:56:44.189 INFO:tasks.workunit.client.1.smithi186.stdout:1/898: write d14/d1a/d1d/f119 [863377,38126] 0 2024-04-12T20:56:44.194 INFO:tasks.workunit.client.1.smithi186.stdout:1/899: write d14/d1a/d33/d36/d5b/d3e/d8b/ff0 [341518,110456] 0 2024-04-12T20:56:44.196 INFO:tasks.workunit.client.1.smithi186.stdout:0/994: truncate d0/d14/d4e/d1e/d84/fcd 3714087 0 2024-04-12T20:56:44.197 INFO:tasks.workunit.client.1.smithi186.stdout:7/865: write d7/d10/f19 [6183428,75505] 0 2024-04-12T20:56:44.201 INFO:tasks.workunit.client.1.smithi186.stdout:0/995: creat d0/f145 x:0 0 0 2024-04-12T20:56:44.205 INFO:tasks.workunit.client.1.smithi186.stdout:1/900: getdents d14/d1a/d21/d35/dbc/dd5 0 2024-04-12T20:56:44.206 INFO:tasks.workunit.client.1.smithi186.stdout:0/996: symlink d0/d5c/d65/l146 0 2024-04-12T20:56:44.233 INFO:tasks.workunit.client.1.smithi186.stdout:8/924: sync 2024-04-12T20:56:44.239 INFO:tasks.workunit.client.1.smithi186.stdout:8/925: stat d7/de/d7a/f4e 0 2024-04-12T20:56:44.240 INFO:tasks.workunit.client.1.smithi186.stdout:7/866: write d7/d10/d1c/f1f [460257,95118] 0 2024-04-12T20:56:44.240 INFO:tasks.workunit.client.1.smithi186.stdout:1/901: mkdir d14/d1a/d21/d126 0 2024-04-12T20:56:44.247 INFO:tasks.workunit.client.1.smithi186.stdout:0/997: mkdir d0/d10/dda/d147 0 2024-04-12T20:56:44.250 INFO:tasks.workunit.client.1.smithi186.stdout:0/998: chown d0/d10/d32/de2/d120/d137 3713613 1 2024-04-12T20:56:44.251 INFO:tasks.workunit.client.1.smithi186.stdout:8/926: creat d7/d18/d36/d20/d23/f130 x:0 0 0 2024-04-12T20:56:44.255 INFO:tasks.workunit.client.1.smithi186.stdout:6/874: dread d9/f47 [0,4194304] 0 2024-04-12T20:56:44.256 INFO:tasks.workunit.client.1.smithi186.stdout:7/867: truncate d7/d10/d1c/f7e 914835 0 2024-04-12T20:56:44.260 INFO:tasks.workunit.client.1.smithi186.stdout:8/927: mkdir d7/de/d68/d11f/d131 0 2024-04-12T20:56:44.260 INFO:tasks.workunit.client.1.smithi186.stdout:0/999: fsync d0/d14/d4e/d50/fce 0 2024-04-12T20:56:44.260 INFO:tasks.workunit.client.1.smithi186.stdout:2/972: dread d0/dc/f55 [4194304,4194304] 0 2024-04-12T20:56:44.262 INFO:tasks.workunit.client.1.smithi186.stdout:6/875: rename d9/d4c/d6f/d89/d9c/d12b/c12d to d9/d4c/de9/d133/c139 0 2024-04-12T20:56:44.262 INFO:tasks.workunit.client.1.smithi186.stdout:8/928: dread - d7/d18/d36/d20/d23/dc1/f110 zero size 2024-04-12T20:56:44.264 INFO:tasks.workunit.client.1.smithi186.stdout:2/973: fdatasync d0/d3/d1a/d24/f60 0 2024-04-12T20:56:44.267 INFO:tasks.workunit.client.1.smithi186.stdout:6/876: write d9/d43/d59/d5d/de6/f110 [628138,106134] 0 2024-04-12T20:56:44.269 INFO:tasks.workunit.client.1.smithi186.stdout:2/974: rmdir d0/d3/d1a/d21/d22 39 2024-04-12T20:56:44.278 INFO:tasks.workunit.client.1.smithi186.stdout:8/929: link d7/d18/d36/d1d/d1f/f49 d7/d18/d8e/f132 0 2024-04-12T20:56:44.279 INFO:tasks.workunit.client.1.smithi186.stdout:1/902: dwrite d14/d1a/d33/d36/d5b/d3e/f7e [8388608,4194304] 0 2024-04-12T20:56:44.280 INFO:tasks.workunit.client.1.smithi186.stdout:2/975: symlink d0/l136 0 2024-04-12T20:56:44.290 INFO:tasks.workunit.client.1.smithi186.stdout:1/903: symlink d14/d1a/d58/l127 0 2024-04-12T20:56:44.293 INFO:tasks.workunit.client.1.smithi186.stdout:2/976: mkdir d0/d3/d1a/d48/d115/d137 0 2024-04-12T20:56:44.296 INFO:tasks.workunit.client.1.smithi186.stdout:1/904: readlink l2 0 2024-04-12T20:56:44.298 INFO:tasks.workunit.client.1.smithi186.stdout:8/930: dwrite d7/d18/d36/d20/d23/dd6/f127 [0,4194304] 0 2024-04-12T20:56:44.304 INFO:tasks.workunit.client.1.smithi186.stdout:2/977: mkdir d0/dc/d2e/d138 0 2024-04-12T20:56:44.304 INFO:tasks.workunit.client.1.smithi186.stdout:8/931: chown d7/d18/d36/c24 176 1 2024-04-12T20:56:44.308 INFO:tasks.workunit.client.1.smithi186.stdout:6/877: dwrite d9/d43/d59/d5d/f46 [4194304,4194304] 0 2024-04-12T20:56:44.317 INFO:tasks.workunit.client.1.smithi186.stdout:8/932: fdatasync d7/d18/d36/d1d/d1f/d26/f6b 0 2024-04-12T20:56:44.321 INFO:tasks.workunit.client.1.smithi186.stdout:2/978: creat d0/d32/de4/f139 x:0 0 0 2024-04-12T20:56:44.324 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:56:44.326+0000 7fae13200700 1 -- 172.21.15.18:0/593015162 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fae0c0662e0 msgr2=0x7fae0c0ff9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:56:44.324 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:56:44.326+0000 7fae13200700 1 --2- 172.21.15.18:0/593015162 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fae0c0662e0 0x7fae0c0ff9c0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fadf8004940 tx=0x7fadf8009e30 comp rx=0 tx=0).stop 2024-04-12T20:56:44.324 INFO:tasks.workunit.client.1.smithi186.stdout:9/828: sync 2024-04-12T20:56:44.325 INFO:tasks.workunit.client.1.smithi186.stdout:8/933: mknod d7/d18/d36/d1d/d7d/c133 0 2024-04-12T20:56:44.325 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:56:44.327+0000 7fae13200700 1 -- 172.21.15.18:0/593015162 shutdown_connections 2024-04-12T20:56:44.325 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:56:44.327+0000 7fae13200700 1 --2- 172.21.15.18:0/593015162 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fae0c0fff00 0x7fae0c102320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:56:44.325 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:56:44.327+0000 7fae13200700 1 --2- 172.21.15.18:0/593015162 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fae0c0662e0 0x7fae0c0ff9c0 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:56:44.325 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:56:44.327+0000 7fae13200700 1 -- 172.21.15.18:0/593015162 >> 172.21.15.18:0/593015162 conn(0x7fae0c0f7430 msgr2=0x7fae0c0f9860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T20:56:44.326 INFO:tasks.workunit.client.1.smithi186.stdout:1/905: dwrite d14/d1a/d21/d35/d59/fd2 [4194304,4194304] 0 2024-04-12T20:56:44.326 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:56:44.328+0000 7fae13200700 1 -- 172.21.15.18:0/593015162 shutdown_connections 2024-04-12T20:56:44.327 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:56:44.328+0000 7fae13200700 1 -- 172.21.15.18:0/593015162 wait complete. 2024-04-12T20:56:44.327 INFO:tasks.workunit.client.1.smithi186.stdout:9/829: fdatasync d3/d2f/d2e/d43/fbd 0 2024-04-12T20:56:44.327 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:56:44.329+0000 7fae13200700 1 Processor -- start 2024-04-12T20:56:44.329 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:56:44.329+0000 7fae13200700 1 -- start start 2024-04-12T20:56:44.329 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:56:44.330+0000 7fae13200700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fae0c0662e0 0x7fae0c06e3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T20:56:44.329 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:56:44.330+0000 7fae13200700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fae0c0fff00 0x7fae0c06e930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T20:56:44.329 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:56:44.330+0000 7fae13200700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae0c06ffe0 con 0x7fae0c0662e0 2024-04-12T20:56:44.329 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:56:44.330+0000 7fae13200700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae0c06ee70 con 0x7fae0c0fff00 2024-04-12T20:56:44.329 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:56:44.331+0000 7fae0be00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fae0c0662e0 0x7fae0c06e3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T20:56:44.329 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:56:44.331+0000 7fae0be00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fae0c0662e0 0x7fae0c06e3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:42542/0 (socket says 172.21.15.18:42542) 2024-04-12T20:56:44.329 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:56:44.331+0000 7fae0be00700 1 -- 172.21.15.18:0/1560534785 learned_addr learned my addr 172.21.15.18:0/1560534785 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T20:56:44.329 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:56:44.331+0000 7fae0b400700 1 --2- 172.21.15.18:0/1560534785 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fae0c0fff00 0x7fae0c06e930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T20:56:44.330 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:56:44.331+0000 7fae0be00700 1 -- 172.21.15.18:0/1560534785 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fae0c0fff00 msgr2=0x7fae0c06e930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:56:44.330 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:56:44.331+0000 7fae0be00700 1 --2- 172.21.15.18:0/1560534785 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fae0c0fff00 0x7fae0c06e930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:56:44.330 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:56:44.331+0000 7fae0be00700 1 -- 172.21.15.18:0/1560534785 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fadf8005040 con 0x7fae0c0662e0 2024-04-12T20:56:44.331 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:56:44.333+0000 7fae0be00700 1 --2- 172.21.15.18:0/1560534785 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fae0c0662e0 0x7fae0c06e3f0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fadf800b3b0 tx=0x7fadf8009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T20:56:44.332 INFO:tasks.workunit.client.1.smithi186.stdout:1/906: mknod d14/d1a/d33/d36/d5b/d3e/dbb/c128 0 2024-04-12T20:56:44.333 INFO:tasks.workunit.client.1.smithi186.stdout:9/830: dread d3/d2f/d54/d9f/dc4/d77/dd6/d72/df4/fcf [0,4194304] 0 2024-04-12T20:56:44.334 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:56:44.335+0000 7fae03e00700 1 -- 172.21.15.18:0/1560534785 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fadf800e030 con 0x7fae0c0662e0 2024-04-12T20:56:44.334 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:56:44.335+0000 7fae03e00700 1 -- 172.21.15.18:0/1560534785 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fadf800bbf0 con 0x7fae0c0662e0 2024-04-12T20:56:44.335 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:56:44.335+0000 7fae03e00700 1 -- 172.21.15.18:0/1560534785 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fadf8013e60 con 0x7fae0c0662e0 2024-04-12T20:56:44.335 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:56:44.336+0000 7fae13200700 1 -- 172.21.15.18:0/1560534785 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fae0c06f0f0 con 0x7fae0c0662e0 2024-04-12T20:56:44.338 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:56:44.340+0000 7fae13200700 1 -- 172.21.15.18:0/1560534785 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fae0c11e0e0 con 0x7fae0c0662e0 2024-04-12T20:56:44.340 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:56:44.342+0000 7fae13200700 1 -- 172.21.15.18:0/1560534785 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fae0c05f5f0 con 0x7fae0c0662e0 2024-04-12T20:56:44.343 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:56:44.345+0000 7fae03e00700 1 -- 172.21.15.18:0/1560534785 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 25) v1 ==== 89542+0+0 (secure 0 0 0) 0x7fadf800bd60 con 0x7fae0c0662e0 2024-04-12T20:56:44.345 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:56:44.347+0000 7fae03e00700 1 --2- 172.21.15.18:0/1560534785 >> [v2:172.21.15.186:6828/2316349902,v1:172.21.15.186:6829/2316349902] conn(0x7fadf006bcb0 0x7fadf006e160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T20:56:44.345 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:56:44.347+0000 7fae03e00700 1 -- 172.21.15.18:0/1560534785 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5808+0+0 (secure 0 0 0) 0x7fadf808c0b0 con 0x7fae0c0662e0 2024-04-12T20:56:44.346 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:56:44.347+0000 7fae0b400700 1 --2- 172.21.15.18:0/1560534785 >> [v2:172.21.15.186:6828/2316349902,v1:172.21.15.186:6829/2316349902] conn(0x7fadf006bcb0 0x7fadf006e160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T20:56:44.349 INFO:tasks.workunit.client.1.smithi186.stdout:1/907: link c10 d14/d1a/d33/d36/c129 0 2024-04-12T20:56:44.353 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:56:44.353+0000 7fae0b400700 1 --2- 172.21.15.18:0/1560534785 >> [v2:172.21.15.186:6828/2316349902,v1:172.21.15.186:6829/2316349902] conn(0x7fadf006bcb0 0x7fadf006e160 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fadfc004f40 tx=0x7fadfc00b040 comp rx=0 tx=0).ready entity=mgr.24359 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T20:56:44.353 INFO:tasks.workunit.client.1.smithi186.stdout:9/831: rename d3/d2f/d54/d9f/dc4/d77/lab to d3/d2f/d2e/db3/l120 0 2024-04-12T20:56:44.354 INFO:tasks.workunit.client.1.smithi186.stdout:7/868: sync 2024-04-12T20:56:44.354 INFO:tasks.workunit.client.1.smithi186.stdout:5/857: sync 2024-04-12T20:56:44.354 INFO:tasks.workunit.client.1.smithi186.stdout:6/878: dread d9/d4c/fb6 [0,4194304] 0 2024-04-12T20:56:44.354 INFO:tasks.workunit.client.1.smithi186.stdout:3/844: sync 2024-04-12T20:56:44.356 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:56:44.358+0000 7fae03e00700 1 -- 172.21.15.18:0/1560534785 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186454 (secure 0 0 0) 0x7fadf8055890 con 0x7fae0c0662e0 2024-04-12T20:56:44.357 INFO:tasks.workunit.client.1.smithi186.stdout:9/832: chown d3/d2f/d54/ca9 4543 1 2024-04-12T20:56:44.358 INFO:tasks.workunit.client.1.smithi186.stdout:6/879: stat d9/d43/l56 0 2024-04-12T20:56:44.358 INFO:tasks.workunit.client.1.smithi186.stdout:2/979: write d0/dc/d2e/d73/fc4 [3316987,100534] 0 2024-04-12T20:56:44.360 INFO:tasks.workunit.client.1.smithi186.stdout:6/880: chown d9/d43/d2d/f8b 1 1 2024-04-12T20:56:44.361 INFO:tasks.workunit.client.1.smithi186.stdout:5/858: mkdir d5/db/d15/d2a/d4c/d114 0 2024-04-12T20:56:44.361 INFO:tasks.workunit.client.1.smithi186.stdout:3/845: creat d1/d3/d21/d5c/f114 x:0 0 0 2024-04-12T20:56:44.362 INFO:tasks.workunit.client.1.smithi186.stdout:7/869: fsync d7/d10/d1c/d5d/f80 0 2024-04-12T20:56:44.368 INFO:tasks.workunit.client.1.smithi186.stdout:6/881: mkdir d9/d43/d59/d5d/d101/d13a 0 2024-04-12T20:56:44.374 INFO:tasks.workunit.client.1.smithi186.stdout:1/908: link d14/d1a/d33/d36/d5b/d105/c10d d14/d1a/d33/c12a 0 2024-04-12T20:56:44.377 INFO:tasks.workunit.client.1.smithi186.stdout:9/833: getdents d3/d2f/d54/d9f/dc4/d77/d11f 0 2024-04-12T20:56:44.377 INFO:tasks.workunit.client.1.smithi186.stdout:1/909: write d14/d1a/fa1 [1144592,126983] 0 2024-04-12T20:56:44.379 INFO:tasks.workunit.client.1.smithi186.stdout:9/834: read - d3/d2f/d2e/d43/fbd zero size 2024-04-12T20:56:44.379 INFO:tasks.workunit.client.1.smithi186.stdout:7/870: dwrite d7/d10/d1c/d5d/ff9 [0,4194304] 0 2024-04-12T20:56:44.380 INFO:tasks.workunit.client.1.smithi186.stdout:6/882: symlink d9/d43/d50/l13b 0 2024-04-12T20:56:44.382 INFO:tasks.workunit.client.1.smithi186.stdout:6/883: chown d9/d4c/d6f/ccf 65 1 2024-04-12T20:56:44.384 INFO:tasks.workunit.client.1.smithi186.stdout:5/859: mkdir d5/db/d2e/d70/d101/dbb/dc4/d5d/d115 0 2024-04-12T20:56:44.389 INFO:tasks.workunit.client.1.smithi186.stdout:1/910: creat d14/d1a/d33/d36/d5b/d3e/d5f/f12b x:0 0 0 2024-04-12T20:56:44.398 INFO:tasks.workunit.client.1.smithi186.stdout:8/934: link d7/d18/d36/d1d/d7d/f111 d7/d18/d36/d1d/d1f/d26/dab/f134 0 2024-04-12T20:56:44.398 INFO:tasks.workunit.client.1.smithi186.stdout:3/846: getdents d1/d2c/d48/ded 0 2024-04-12T20:56:44.400 INFO:tasks.workunit.client.1.smithi186.stdout:5/860: dread d5/db/d2e/d70/f99 [0,4194304] 0 2024-04-12T20:56:44.403 INFO:tasks.workunit.client.1.smithi186.stdout:6/884: dwrite d9/d5e/d7a/f7f [0,4194304] 0 2024-04-12T20:56:44.405 INFO:tasks.workunit.client.1.smithi186.stdout:9/835: dread d3/d2f/d54/d9f/dc4/d77/fa8 [0,4194304] 0 2024-04-12T20:56:44.406 INFO:tasks.workunit.client.1.smithi186.stdout:5/861: dread - d5/fd1 zero size 2024-04-12T20:56:44.407 INFO:tasks.workunit.client.1.smithi186.stdout:3/847: fsync d1/d3/d21/d22/d24/f53 0 2024-04-12T20:56:44.411 INFO:tasks.workunit.client.1.smithi186.stdout:1/911: dwrite d14/d1a/d33/d36/dfa/f118 [0,4194304] 0 2024-04-12T20:56:44.414 INFO:tasks.workunit.client.1.smithi186.stdout:5/862: read d5/db/d2e/d70/d101/dbb/dc4/d5d/fc7 [10267,41479] 0 2024-04-12T20:56:44.415 INFO:tasks.workunit.client.1.smithi186.stdout:5/863: chown d5/db/d15/f28 2961865 1 2024-04-12T20:56:44.417 INFO:tasks.workunit.client.1.smithi186.stdout:5/864: stat d5/db/d2e/d70/f99 0 2024-04-12T20:56:44.420 INFO:tasks.workunit.client.1.smithi186.stdout:8/935: write d7/de/d7a/fcd [1527977,27365] 0 2024-04-12T20:56:44.425 INFO:tasks.workunit.client.1.smithi186.stdout:3/848: mknod d1/d3/d21/d22/d34/ddb/d4d/da1/c115 0 2024-04-12T20:56:44.433 INFO:tasks.workunit.client.1.smithi186.stdout:1/912: creat d14/d1a/d21/d126/f12c x:0 0 0 2024-04-12T20:56:44.434 INFO:tasks.workunit.client.1.smithi186.stdout:7/871: dwrite d7/d10/d2d/d40/d4a/d6c/d97/f92 [0,4194304] 0 2024-04-12T20:56:44.436 INFO:tasks.workunit.client.1.smithi186.stdout:3/849: creat d1/d2c/f116 x:0 0 0 2024-04-12T20:56:44.442 INFO:tasks.workunit.client.1.smithi186.stdout:5/865: unlink d5/db/l23 0 2024-04-12T20:56:44.450 INFO:tasks.workunit.client.1.smithi186.stdout:6/885: dread d9/d43/d50/f54 [0,4194304] 0 2024-04-12T20:56:44.451 INFO:tasks.workunit.client.1.smithi186.stdout:9/836: write d3/d13/fc6 [1287896,8496] 0 2024-04-12T20:56:44.457 INFO:tasks.workunit.client.1.smithi186.stdout:1/913: dwrite d14/d1a/d1d/f29 [0,4194304] 0 2024-04-12T20:56:44.464 INFO:tasks.workunit.client.1.smithi186.stdout:7/872: dwrite d7/d10/d2d/d40/d4a/d6c/d97/d4d/d6a/dcc/f107 [0,4194304] 0 2024-04-12T20:56:44.466 INFO:tasks.workunit.client.1.smithi186.stdout:8/936: dread d7/de/d68/f9d [0,4194304] 0 2024-04-12T20:56:44.468 INFO:tasks.workunit.client.1.smithi186.stdout:2/980: sync 2024-04-12T20:56:44.468 INFO:tasks.workunit.client.1.smithi186.stdout:9/837: rename d3/d2f/d54/d7d/f109 to d3/d2f/d54/d7d/f121 0 2024-04-12T20:56:44.470 INFO:tasks.workunit.client.1.smithi186.stdout:1/914: mknod d14/d1a/d33/d36/d5b/d3e/d71/c12d 0 2024-04-12T20:56:44.479 INFO:tasks.workunit.client.1.smithi186.stdout:5/866: rename d5/db/d71/d88/fd4 to d5/db/d15/d2a/d4c/dc0/f116 0 2024-04-12T20:56:44.481 INFO:tasks.workunit.client.1.smithi186.stdout:9/838: symlink d3/d13/l122 0 2024-04-12T20:56:44.482 INFO:tasks.workunit.client.1.smithi186.stdout:8/937: chown d7/d18/d36/d1d/d7d/f111 7386415 1 2024-04-12T20:56:44.484 INFO:tasks.workunit.client.1.smithi186.stdout:5/867: chown d5/db/d15/d2a/d4c/f62 43400 1 2024-04-12T20:56:44.484 INFO:tasks.workunit.client.1.smithi186.stdout:8/938: readlink d7/d18/le1 0 2024-04-12T20:56:44.488 INFO:tasks.workunit.client.1.smithi186.stdout:8/939: fdatasync d7/d18/d36/d1d/fa0 0 2024-04-12T20:56:44.488 INFO:tasks.workunit.client.1.smithi186.stdout:3/850: write d1/d3/d21/d22/d34/ddb/d4d/dad/fcf [2219277,1864] 0 2024-04-12T20:56:44.490 INFO:tasks.workunit.client.1.smithi186.stdout:6/886: truncate d9/d5e/d7a/f7f 3455799 0 2024-04-12T20:56:44.492 INFO:tasks.workunit.client.1.smithi186.stdout:1/915: dwrite f9 [0,4194304] 0 2024-04-12T20:56:44.493 INFO:tasks.workunit.client.1.smithi186.stdout:7/873: dread d7/d10/d1c/f7e [0,4194304] 0 2024-04-12T20:56:44.497 INFO:tasks.workunit.client.1.smithi186.stdout:7/874: stat d7/d10/d1c/f7e 0 2024-04-12T20:56:44.498 INFO:tasks.workunit.client.1.smithi186.stdout:6/887: truncate d9/d4c/d6f/de3/f123 805738 0 2024-04-12T20:56:44.498 INFO:tasks.workunit.client.1.smithi186.stdout:9/839: fsync d3/d2f/d2e/d33/f8d 0 2024-04-12T20:56:44.500 INFO:tasks.workunit.client.1.smithi186.stdout:3/851: mkdir d1/d3/df8/d95/d117 0 2024-04-12T20:56:44.501 INFO:tasks.workunit.client.1.smithi186.stdout:8/940: mkdir d7/d18/d36/d20/d23/dd7/d135 0 2024-04-12T20:56:44.503 INFO:tasks.workunit.client.1.smithi186.stdout:2/981: dread d0/dc/d2e/d131/f94 [0,4194304] 0 2024-04-12T20:56:44.504 INFO:tasks.workunit.client.1.smithi186.stdout:7/875: write d7/d10/d1c/d8c/fb5 [2310215,88205] 0 2024-04-12T20:56:44.506 INFO:tasks.workunit.client.1.smithi186.stdout:9/840: rmdir d3/d2f/d2e/db3 39 2024-04-12T20:56:44.507 INFO:tasks.workunit.client.1.smithi186.stdout:3/852: stat d1/d3/d21/d22/d34/ddb/d4d/dad/cf0 0 2024-04-12T20:56:44.509 INFO:tasks.workunit.client.1.smithi186.stdout:8/941: chown d7/d18/d36/d1d/fea 701591 1 2024-04-12T20:56:44.512 INFO:tasks.workunit.client.1.smithi186.stdout:1/916: mkdir d14/d1a/d1d/d12e 0 2024-04-12T20:56:44.514 INFO:tasks.workunit.client.1.smithi186.stdout:6/888: fdatasync d9/d43/d59/d5d/d101/ddb/fed 0 2024-04-12T20:56:44.517 INFO:tasks.workunit.client.1.smithi186.stdout:2/982: write d0/dc/d2e/d131/fe0 [482864,88079] 0 2024-04-12T20:56:44.529 INFO:tasks.workunit.client.1.smithi186.stdout:1/917: symlink d14/d1a/d21/d35/d59/l12f 0 2024-04-12T20:56:44.529 INFO:tasks.workunit.client.1.smithi186.stdout:5/868: write d5/db/d2e/d70/feb [1237842,81379] 0 2024-04-12T20:56:44.530 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:56:44 smithi186 ceph-mon[36597]: from='client.24439 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T20:56:44.530 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:56:44 smithi186 ceph-mon[36597]: from='mgr.24359 172.21.15.186:0/1315906828' entity='mgr.smithi186.kqcfxk' 2024-04-12T20:56:44.530 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:56:44 smithi186 ceph-mon[36597]: from='mgr.24359 172.21.15.186:0/1315906828' entity='mgr.smithi186.kqcfxk' 2024-04-12T20:56:44.530 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:56:44 smithi186 ceph-mon[36597]: from='mgr.24359 172.21.15.186:0/1315906828' entity='mgr.smithi186.kqcfxk' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-12T20:56:44.538 INFO:tasks.workunit.client.1.smithi186.stdout:6/889: creat d9/d43/d59/d5d/de6/f13c x:0 0 0 2024-04-12T20:56:44.539 INFO:tasks.workunit.client.1.smithi186.stdout:9/841: write d3/d2f/d54/d9f/fa5 [157128,29007] 0 2024-04-12T20:56:44.546 INFO:tasks.workunit.client.1.smithi186.stdout:8/942: dwrite d7/d18/d36/d1d/d1f/da2/df8/d29/f6a [8388608,4194304] 0 2024-04-12T20:56:44.547 INFO:tasks.workunit.client.1.smithi186.stdout:8/943: fdatasync d7/de/fe9 0 2024-04-12T20:56:44.548 INFO:tasks.workunit.client.1.smithi186.stdout:1/918: symlink d14/d1a/d33/d36/d5b/d3e/dbb/d10f/l130 0 2024-04-12T20:56:44.551 INFO:tasks.workunit.client.1.smithi186.stdout:9/842: mkdir d3/d2f/d2e/d33/d36/d93/d123 0 2024-04-12T20:56:44.551 INFO:tasks.workunit.client.1.smithi186.stdout:3/853: link d1/d3/d21/d22/d34/ddb/d4d/dad/cde d1/c118 0 2024-04-12T20:56:44.552 INFO:tasks.workunit.client.1.smithi186.stdout:1/919: read - d14/d1a/d21/d35/dbc/de3/f11c zero size 2024-04-12T20:56:44.552 INFO:tasks.workunit.client.1.smithi186.stdout:7/876: link d7/d10/d2d/d40/d4a/d6c/d97/d4d/d6a/dcc/cf5 d7/d10/d2d/d40/d4a/d5c/d13b/c13f 0 2024-04-12T20:56:44.556 INFO:tasks.workunit.client.1.smithi186.stdout:5/869: link d5/db/d15/d2a/d4c/d5b/d7a/d94/fa3 d5/db/d2e/d70/d101/d7f/d86/db6/f117 0 2024-04-12T20:56:44.557 INFO:tasks.workunit.client.1.smithi186.stdout:6/890: mkdir d9/d43/d59/d116/d11a/d13d 0 2024-04-12T20:56:44.558 INFO:tasks.workunit.client.1.smithi186.stdout:2/983: dwrite d0/d32/d2a/f62 [4194304,4194304] 0 2024-04-12T20:56:44.559 INFO:tasks.workunit.client.1.smithi186.stdout:8/944: mkdir d7/d18/d36/d20/d23/dd7/df5/d108/d136 0 2024-04-12T20:56:44.561 INFO:tasks.workunit.client.1.smithi186.stdout:7/877: mknod d7/d10/d2d/d67/c140 0 2024-04-12T20:56:44.563 INFO:tasks.workunit.client.1.smithi186.stdout:1/920: rmdir d14/d1a/d21/d35/dbc/de3 39 2024-04-12T20:56:44.563 INFO:tasks.workunit.client.1.smithi186.stdout:8/945: fsync d7/d18/d36/d1d/d1f/da2/d5c/f99 0 2024-04-12T20:56:44.566 INFO:tasks.workunit.client.1.smithi186.stdout:8/946: stat d7/de/d68/fcb 0 2024-04-12T20:56:44.571 INFO:tasks.workunit.client.1.smithi186.stdout:5/870: mknod d5/db/d2e/d70/d101/dbb/dc4/c118 0 2024-04-12T20:56:44.577 INFO:tasks.workunit.client.1.smithi186.stdout:3/854: getdents d1/d3/d21/d22/d4a 0 2024-04-12T20:56:44.577 INFO:tasks.workunit.client.1.smithi186.stdout:6/891: getdents d9/d4c/d6f/d89 0 2024-04-12T20:56:44.579 INFO:tasks.workunit.client.1.smithi186.stdout:3/855: dread - d1/d3/da2/f10e zero size 2024-04-12T20:56:44.580 INFO:tasks.workunit.client.1.smithi186.stdout:2/984: dwrite d0/d3e/f113 [0,4194304] 0 2024-04-12T20:56:44.580 INFO:tasks.workunit.client.1.smithi186.stdout:1/921: truncate d14/d1a/d33/d36/d5b/d3e/f53 123698 0 2024-04-12T20:56:44.582 INFO:tasks.workunit.client.1.smithi186.stdout:3/856: chown d1/d3/da/l2b 15 1 2024-04-12T20:56:44.584 INFO:tasks.workunit.client.1.smithi186.stdout:8/947: link d7/de/d10/d7f/f9f d7/de/d10/d11d/f137 0 2024-04-12T20:56:44.586 INFO:tasks.workunit.client.1.smithi186.stdout:5/871: truncate d5/db/d15/d2a/d2d/d46/d55/fc6 826564 0 2024-04-12T20:56:44.590 INFO:tasks.workunit.client.1.smithi186.stdout:6/892: creat d9/d5e/f13e x:0 0 0 2024-04-12T20:56:44.594 INFO:tasks.workunit.client.1.smithi186.stdout:7/878: dwrite d7/d10/d2d/d40/d4a/d6c/d97/d4d/d84/fe8 [0,4194304] 0 2024-04-12T20:56:44.596 INFO:tasks.workunit.client.1.smithi186.stdout:7/879: chown d7/l5e 1345 1 2024-04-12T20:56:44.603 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:56:44 smithi018 ceph-mon[28799]: from='client.24439 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T20:56:44.603 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:56:44 smithi018 ceph-mon[28799]: from='mgr.24359 172.21.15.186:0/1315906828' entity='mgr.smithi186.kqcfxk' 2024-04-12T20:56:44.603 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:56:44 smithi018 ceph-mon[28799]: from='mgr.24359 172.21.15.186:0/1315906828' entity='mgr.smithi186.kqcfxk' 2024-04-12T20:56:44.603 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:56:44 smithi018 ceph-mon[28799]: from='mgr.24359 172.21.15.186:0/1315906828' entity='mgr.smithi186.kqcfxk' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-12T20:56:44.603 INFO:tasks.workunit.client.1.smithi186.stdout:2/985: mknod d0/dc/d2e/d138/c13a 0 2024-04-12T20:56:44.604 INFO:tasks.workunit.client.1.smithi186.stdout:3/857: mkdir d1/d3/df8/dd0/de4/d119 0 2024-04-12T20:56:44.604 INFO:tasks.workunit.client.1.smithi186.stdout:7/880: chown d7/d10/d1c/d5d/c111 25 1 2024-04-12T20:56:44.604 INFO:tasks.workunit.client.1.smithi186.stdout:8/948: dwrite d7/fb3 [0,4194304] 0 2024-04-12T20:56:44.607 INFO:tasks.workunit.client.1.smithi186.stdout:6/893: mknod d9/d43/d59/d5d/c13f 0 2024-04-12T20:56:44.612 INFO:tasks.workunit.client.1.smithi186.stdout:9/843: write d3/d2f/d54/d9f/dc4/f5a [73508,117169] 0 2024-04-12T20:56:44.616 INFO:tasks.workunit.client.1.smithi186.stdout:6/894: creat d9/d4c/de9/f140 x:0 0 0 2024-04-12T20:56:44.624 INFO:tasks.workunit.client.1.smithi186.stdout:8/949: creat d7/d18/d36/d1d/d7d/d81/f138 x:0 0 0 2024-04-12T20:56:44.626 INFO:tasks.workunit.client.1.smithi186.stdout:6/895: unlink d9/d43/c3c 0 2024-04-12T20:56:44.627 INFO:tasks.workunit.client.1.smithi186.stdout:8/950: stat d7/d18/d36/c24 0 2024-04-12T20:56:44.630 INFO:tasks.workunit.client.1.smithi186.stdout:7/881: dwrite d7/d10/d2d/d40/d4a/d6c/d97/f11a [0,4194304] 0 2024-04-12T20:56:44.631 INFO:tasks.workunit.client.1.smithi186.stdout:1/922: truncate d14/d1a/d33/d36/dfa/f118 1529687 0 2024-04-12T20:56:44.635 INFO:tasks.workunit.client.1.smithi186.stdout:8/951: mknod d7/de/d68/da7/c139 0 2024-04-12T20:56:44.638 INFO:tasks.workunit.client.1.smithi186.stdout:3/858: write d1/d3/d21/d5c/f77 [7002827,70127] 0 2024-04-12T20:56:44.638 INFO:tasks.workunit.client.1.smithi186.stdout:5/872: dwrite d5/db/d15/f2f [0,4194304] 0 2024-04-12T20:56:44.645 INFO:tasks.workunit.client.1.smithi186.stdout:6/896: symlink d9/da9/l141 0 2024-04-12T20:56:44.648 INFO:tasks.workunit.client.1.smithi186.stdout:1/923: readlink d14/l19 0 2024-04-12T20:56:44.649 INFO:tasks.workunit.client.1.smithi186.stdout:7/882: readlink d7/d10/d1c/l13e 0 2024-04-12T20:56:44.652 INFO:tasks.workunit.client.1.smithi186.stdout:3/859: symlink d1/d3/d21/d22/d4a/dbc/l11a 0 2024-04-12T20:56:44.653 INFO:tasks.workunit.client.1.smithi186.stdout:6/897: chown d9/d4c/d6f/d97/f10f 50 1 2024-04-12T20:56:44.654 INFO:tasks.workunit.client.1.smithi186.stdout:7/883: chown d7/d55/d71/fc1 125137043 1 2024-04-12T20:56:44.655 INFO:tasks.workunit.client.1.smithi186.stdout:2/986: dwrite d0/d3e/f46 [0,4194304] 0 2024-04-12T20:56:44.657 INFO:tasks.workunit.client.1.smithi186.stdout:1/924: read - d14/d1a/d33/d36/d5b/d3e/d8b/fff zero size 2024-04-12T20:56:44.662 INFO:tasks.workunit.client.1.smithi186.stdout:1/925: write d14/d1a/d58/d69/fdf [240297,75990] 0 2024-04-12T20:56:44.669 INFO:tasks.workunit.client.1.smithi186.stdout:5/873: dread d5/fd [0,4194304] 0 2024-04-12T20:56:44.670 INFO:tasks.workunit.client.1.smithi186.stdout:9/844: dread d3/d2f/d54/d9f/dc4/d77/dd6/d72/df4/df9/ffb [0,4194304] 0 2024-04-12T20:56:44.671 INFO:tasks.workunit.client.1.smithi186.stdout:8/952: creat d7/d18/d36/d20/f13a x:0 0 0 2024-04-12T20:56:44.675 INFO:tasks.workunit.client.1.smithi186.stdout:2/987: creat d0/d3/dce/f13b x:0 0 0 2024-04-12T20:56:44.684 INFO:tasks.workunit.client.1.smithi186.stdout:7/884: dread d7/d10/d2d/f58 [0,4194304] 0 2024-04-12T20:56:44.704 INFO:tasks.workunit.client.1.smithi186.stdout:3/860: rename d1/d3/d21/d22/d34/ddb/d4d/dad/cde to d1/d3/da2/c11b 0 2024-04-12T20:56:44.704 INFO:tasks.workunit.client.1.smithi186.stdout:9/845: fdatasync d3/d2f/d2e/d33/d36/f37 0 2024-04-12T20:56:44.707 INFO:tasks.workunit.client.1.smithi186.stdout:3/861: chown d1/d3/d21/d22/d34/ddb/d4d/d6f/ff5 26 1 2024-04-12T20:56:44.755 INFO:tasks.workunit.client.1.smithi186.stdout:2/988: rename d0/d3/d1a/d21/fdc to d0/d3/d1a/d24/db3/f13c 0 2024-04-12T20:56:44.759 INFO:tasks.workunit.client.1.smithi186.stdout:6/898: stat d9/d43/d2d/d4d/fe0 0 2024-04-12T20:56:44.760 INFO:tasks.workunit.client.1.smithi186.stdout:3/862: symlink d1/d3/df8/dd0/d10b/d7c/dbb/l11c 0 2024-04-12T20:56:44.762 INFO:tasks.workunit.client.1.smithi186.stdout:7/885: link d7/d10/c26 d7/d10/d2d/d40/d4a/d6c/d97/dc2/de2/c141 0 2024-04-12T20:56:44.762 INFO:tasks.workunit.client.1.smithi186.stdout:6/899: read - d9/d43/d59/d5d/d101/ddb/fed zero size 2024-04-12T20:56:44.762 INFO:tasks.workunit.client.1.smithi186.stdout:3/863: write d1/d3/da/f56 [5740234,81070] 0 2024-04-12T20:56:44.766 INFO:tasks.workunit.client.1.smithi186.stdout:6/900: write d9/d43/d59/d5d/de6/f13c [11682,117188] 0 2024-04-12T20:56:44.773 INFO:tasks.workunit.client.1.smithi186.stdout:1/926: sync 2024-04-12T20:56:44.773 INFO:tasks.workunit.client.1.smithi186.stdout:6/901: rmdir d9/d5e/d7a/d81 39 2024-04-12T20:56:44.775 INFO:tasks.workunit.client.1.smithi186.stdout:2/989: link d0/dc/lf4 d0/d3/d1a/d27/l13d 0 2024-04-12T20:56:44.781 INFO:tasks.workunit.client.1.smithi186.stdout:7/886: dwrite d7/d10/d2d/f37 [0,4194304] 0 2024-04-12T20:56:44.786 INFO:tasks.workunit.client.1.smithi186.stdout:2/990: write d0/d3/d1a/ff8 [811898,96533] 0 2024-04-12T20:56:44.792 INFO:tasks.workunit.client.1.smithi186.stdout:7/887: symlink d7/d10/d2d/l142 0 2024-04-12T20:56:44.795 INFO:tasks.workunit.client.1.smithi186.stdout:1/927: truncate d14/d74/fe2 1275717 0 2024-04-12T20:56:44.797 INFO:tasks.workunit.client.1.smithi186.stdout:6/902: dwrite d9/d43/d59/d5d/d101/fc9 [4194304,4194304] 0 2024-04-12T20:56:44.797 INFO:tasks.workunit.client.1.smithi186.stdout:2/991: rename d0/d32/d69/lab to d0/d3/dce/l13e 0 2024-04-12T20:56:44.798 INFO:tasks.workunit.client.1.smithi186.stdout:1/928: write d14/d1a/d33/d36/d5b/f67 [1529785,121814] 0 2024-04-12T20:56:44.806 INFO:tasks.workunit.client.1.smithi186.stdout:8/953: truncate d7/d18/d36/d20/d23/dfd/fff 1390415 0 2024-04-12T20:56:44.806 INFO:tasks.workunit.client.1.smithi186.stdout:8/954: stat d7/de/d10/f54 0 2024-04-12T20:56:44.806 INFO:tasks.workunit.client.1.smithi186.stdout:1/929: readlink d14/d1a/d21/d35/dbc/de3/l112 0 2024-04-12T20:56:44.806 INFO:tasks.workunit.client.1.smithi186.stdout:2/992: dread d0/d3e/fe6 [0,4194304] 0 2024-04-12T20:56:44.807 INFO:tasks.workunit.client.1.smithi186.stdout:8/955: read d7/de/d7a/f4f [248923,100020] 0 2024-04-12T20:56:44.811 INFO:tasks.workunit.client.1.smithi186.stdout:9/846: write d3/ff [777206,97042] 0 2024-04-12T20:56:44.819 INFO:tasks.workunit.client.1.smithi186.stdout:8/956: rmdir d7/d18/d36/d1d/d1f/d26/de6 39 2024-04-12T20:56:44.824 INFO:tasks.workunit.client.1.smithi186.stdout:8/957: fsync d7/d18/d8e/d10c/f113 0 2024-04-12T20:56:44.825 INFO:tasks.workunit.client.1.smithi186.stdout:1/930: dwrite d14/d1a/d1d/f119 [0,4194304] 0 2024-04-12T20:56:44.831 INFO:tasks.workunit.client.1.smithi186.stdout:9/847: truncate d3/d13/f65 4423089 0 2024-04-12T20:56:44.832 INFO:tasks.workunit.client.1.smithi186.stdout:8/958: readlink d7/d18/d36/d1d/lb0 0 2024-04-12T20:56:44.833 INFO:tasks.workunit.client.1.smithi186.stdout:2/993: dwrite d0/d3e/f113 [0,4194304] 0 2024-04-12T20:56:44.838 INFO:tasks.workunit.client.1.smithi186.stdout:6/903: truncate d9/d43/d2d/d4d/fe0 1928606 0 2024-04-12T20:56:44.840 INFO:tasks.workunit.client.1.smithi186.stdout:9/848: write d3/d2f/d2e/d33/d61/dbc/fd3 [571366,14329] 0 2024-04-12T20:56:44.843 INFO:tasks.workunit.client.1.smithi186.stdout:8/959: unlink d7/d18/d36/d1d/d7d/d81/c109 0 2024-04-12T20:56:44.849 INFO:tasks.workunit.client.1.smithi186.stdout:6/904: truncate d9/d4c/de9/f140 507532 0 2024-04-12T20:56:44.855 INFO:tasks.workunit.client.1.smithi186.stdout:9/849: rename d3/d13/l122 to d3/d2f/d54/d7d/l124 0 2024-04-12T20:56:44.860 INFO:tasks.workunit.client.1.smithi186.stdout:8/960: read d7/f59 [3130388,14997] 0 2024-04-12T20:56:44.867 INFO:tasks.workunit.client.1.smithi186.stdout:6/905: dwrite d9/d4c/d6f/d89/d9c/fd6 [4194304,4194304] 0 2024-04-12T20:56:44.870 INFO:tasks.workunit.client.1.smithi186.stdout:1/931: dwrite d14/d91/dbd/dc5/fcd [0,4194304] 0 2024-04-12T20:56:44.877 INFO:tasks.workunit.client.1.smithi186.stdout:5/874: sync 2024-04-12T20:56:44.879 INFO:tasks.workunit.client.1.smithi186.stdout:5/875: chown d5/db/d2e/d70/d101/ld3 7358289 1 2024-04-12T20:56:44.890 INFO:tasks.workunit.client.1.smithi186.stdout:2/994: getdents d0/d3/d1a/d27 0 2024-04-12T20:56:44.890 INFO:tasks.workunit.client.1.smithi186.stdout:6/906: dread d9/d43/d2d/f7e [0,4194304] 0 2024-04-12T20:56:44.892 INFO:tasks.workunit.client.1.smithi186.stdout:7/888: sync 2024-04-12T20:56:44.892 INFO:tasks.workunit.client.1.smithi186.stdout:3/864: sync 2024-04-12T20:56:44.893 INFO:tasks.workunit.client.1.smithi186.stdout:6/907: chown d9/d43/d59/d5d/d6e/d91/f131 3 1 2024-04-12T20:56:44.896 INFO:tasks.workunit.client.1.smithi186.stdout:1/932: symlink d14/d1a/d33/d36/d5b/d3e/dbb/l131 0 2024-04-12T20:56:44.896 INFO:tasks.workunit.client.1.smithi186.stdout:6/908: fdatasync d9/d43/d59/d5d/de6/f13c 0 2024-04-12T20:56:44.903 INFO:tasks.workunit.client.1.smithi186.stdout:5/876: mknod d5/db/d15/d2a/d4c/d5b/d7a/c119 0 2024-04-12T20:56:44.907 INFO:tasks.workunit.client.1.smithi186.stdout:8/961: creat d7/d18/d36/d20/d23/dd7/d135/f13b x:0 0 0 2024-04-12T20:56:44.907 INFO:tasks.workunit.client.1.smithi186.stdout:3/865: dwrite d1/d3/da/f70 [0,4194304] 0 2024-04-12T20:56:44.911 INFO:tasks.workunit.client.1.smithi186.stdout:8/962: fsync d7/de/f42 0 2024-04-12T20:56:44.913 INFO:tasks.workunit.client.1.smithi186.stdout:2/995: fdatasync d0/d32/f7c 0 2024-04-12T20:56:44.914 INFO:tasks.workunit.client.1.smithi186.stdout:8/963: chown d7/d18/d36/d1d/d1f/da2/df8/d29/f2a 12 1 2024-04-12T20:56:44.915 INFO:tasks.workunit.client.1.smithi186.stdout:2/996: write d0/d32/d8b/f109 [886471,9424] 0 2024-04-12T20:56:44.917 INFO:tasks.workunit.client.1.smithi186.stdout:6/909: creat d9/d4c/dbe/dfe/f142 x:0 0 0 2024-04-12T20:56:44.917 INFO:tasks.workunit.client.1.smithi186.stdout:1/933: symlink d14/d1a/d21/d35/dbc/de3/l132 0 2024-04-12T20:56:44.920 INFO:tasks.workunit.client.1.smithi186.stdout:7/889: rename d7/d10/d2d/d40/d4a/d6c/d97/d4d/d6a/d11d to d7/d10/d2d/d40/d4a/d6c/d97/d4d/d8a/dad/d143 0 2024-04-12T20:56:44.921 INFO:tasks.workunit.client.1.smithi186.stdout:9/850: dread d3/d2f/f11d [0,4194304] 0 2024-04-12T20:56:44.923 INFO:tasks.workunit.client.1.smithi186.stdout:3/866: fdatasync d1/d3/d21/d22/fcd 0 2024-04-12T20:56:44.926 INFO:tasks.workunit.client.1.smithi186.stdout:8/964: mknod d7/de/c13c 0 2024-04-12T20:56:44.931 INFO:tasks.workunit.client.1.smithi186.stdout:1/934: mknod d14/d1a/d33/d36/d5b/d6a/da2/c133 0 2024-04-12T20:56:44.934 INFO:tasks.workunit.client.1.smithi186.stdout:2/997: rename d0/dc/c11 to d0/d3/d1a/d21/d26/d86/c13f 0 2024-04-12T20:56:44.937 INFO:tasks.workunit.client.1.smithi186.stdout:9/851: dwrite d3/d2f/d54/f5b [0,4194304] 0 2024-04-12T20:56:44.943 INFO:tasks.workunit.client.1.smithi186.stdout:3/867: fsync d1/d3/df8/f7f 0 2024-04-12T20:56:44.946 INFO:tasks.workunit.client.1.smithi186.stdout:5/877: write d5/db/d15/d2a/d4c/d5b/d7a/fb1 [4855578,73458] 0 2024-04-12T20:56:44.947 INFO:tasks.workunit.client.1.smithi186.stdout:8/965: mknod d7/c13d 0 2024-04-12T20:56:44.948 INFO:tasks.workunit.client.1.smithi186.stdout:8/966: stat d7/de/f11 0 2024-04-12T20:56:44.950 INFO:tasks.workunit.client.1.smithi186.stdout:9/852: getdents d3/d2f/d54/d9f/dc4/d77/d11f 0 2024-04-12T20:56:44.952 INFO:tasks.workunit.client.1.smithi186.stdout:5/878: stat d5/l107 0 2024-04-12T20:56:44.955 INFO:tasks.workunit.client.1.smithi186.stdout:7/890: getdents d7/d10/d2d/d40/d4a/d5c 0 2024-04-12T20:56:44.957 INFO:tasks.workunit.client.1.smithi186.stdout:9/853: mknod d3/d2f/d2e/d33/c125 0 2024-04-12T20:56:44.960 INFO:tasks.workunit.client.1.smithi186.stdout:9/854: write d3/d2f/d54/d9f/fa5 [1329126,17848] 0 2024-04-12T20:56:44.961 INFO:tasks.workunit.client.1.smithi186.stdout:2/998: dread d0/d3/d1a/d24/f34 [0,4194304] 0 2024-04-12T20:56:44.969 INFO:tasks.workunit.client.1.smithi186.stdout:1/935: link d14/c1c d14/d1a/d21/d35/dbc/c134 0 2024-04-12T20:56:44.971 INFO:tasks.workunit.client.1.smithi186.stdout:2/999: symlink d0/d3/dce/l140 0 2024-04-12T20:56:44.972 INFO:tasks.workunit.client.1.smithi186.stdout:1/936: write d14/d6c/fb5 [1053471,104932] 0 2024-04-12T20:56:44.975 INFO:tasks.workunit.client.1.smithi186.stdout:9/855: mkdir d3/d2f/d126 0 2024-04-12T20:56:44.978 INFO:tasks.workunit.client.1.smithi186.stdout:3/868: dwrite d1/d2c/d64/f9b [4194304,4194304] 0 2024-04-12T20:56:44.982 INFO:tasks.workunit.client.1.smithi186.stdout:9/856: mknod d3/d2f/d54/d9f/dc4/d6b/c127 0 2024-04-12T20:56:44.983 INFO:tasks.workunit.client.1.smithi186.stdout:1/937: getdents d14/d1a/d21/d35/d107 0 2024-04-12T20:56:44.984 INFO:tasks.workunit.client.1.smithi186.stdout:6/910: sync 2024-04-12T20:56:44.984 INFO:tasks.workunit.client.1.smithi186.stdout:3/869: symlink d1/d3/df8/dd0/d10b/d7c/dbb/l11d 0 2024-04-12T20:56:44.985 INFO:tasks.workunit.client.1.smithi186.stdout:1/938: write d14/d1a/d21/d35/dbc/ff2 [2647209,125126] 0 2024-04-12T20:56:44.989 INFO:tasks.workunit.client.1.smithi186.stdout:8/967: truncate d7/de/d68/da7/dd5/ffa 483299 0 2024-04-12T20:56:44.995 INFO:tasks.workunit.client.1.smithi186.stdout:1/939: rmdir d14/d1a/d33/d36 39 2024-04-12T20:56:44.998 INFO:tasks.workunit.client.1.smithi186.stdout:3/870: mkdir d1/d3/df8/d95/d117/d11e 0 2024-04-12T20:56:45.000 INFO:tasks.workunit.client.1.smithi186.stdout:8/968: chown d7/d18/d36/d1d/d1f/d26/dab/cfc 14 1 2024-04-12T20:56:45.002 INFO:tasks.workunit.client.1.smithi186.stdout:5/879: dwrite d5/db/d15/d2a/d2d/d46/f83 [0,4194304] 0 2024-04-12T20:56:45.003 INFO:tasks.workunit.client.1.smithi186.stdout:7/891: dwrite d7/d55/d71/f85 [0,4194304] 0 2024-04-12T20:56:45.010 INFO:tasks.workunit.client.1.smithi186.stdout:3/871: mknod d1/d3/d21/d22/d24/d28/c11f 0 2024-04-12T20:56:45.010 INFO:tasks.workunit.client.1.smithi186.stdout:6/911: link d9/d43/d50/cf2 d9/d4c/d6f/d89/d9c/c143 0 2024-04-12T20:56:45.010 INFO:tasks.workunit.client.1.smithi186.stdout:8/969: getdents d7/d18/def/d104 0 2024-04-12T20:56:45.016 INFO:tasks.workunit.client.1.smithi186.stdout:7/892: mkdir d7/d10/d2d/d40/d4a/d6c/d97/d4d/d6a/d144 0 2024-04-12T20:56:45.017 INFO:tasks.workunit.client.1.smithi186.stdout:3/872: readlink d1/d3/d21/d22/d34/ddb/d4d/da1/l107 0 2024-04-12T20:56:45.017 INFO:tasks.workunit.client.1.smithi186.stdout:3/873: chown d1/d3/c1d 804148 1 2024-04-12T20:56:45.018 INFO:tasks.workunit.client.1.smithi186.stdout:3/874: fdatasync d1/d3/df8/d95/ff3 0 2024-04-12T20:56:45.019 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:56:45.022+0000 7fae13200700 1 -- 172.21.15.18:0/1560534785 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fae0c11e430 con 0x7fae0c0662e0 2024-04-12T20:56:45.020 INFO:tasks.workunit.client.1.smithi186.stdout:5/880: link d5/db/d15/f1e d5/db/d15/d2a/d2d/d82/d84/f11a 0 2024-04-12T20:56:45.022 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:56:45.025+0000 7fae03e00700 1 -- 172.21.15.18:0/1560534785 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+846 (secure 0 0 0) 0x7fadf8058f50 con 0x7fae0c0662e0 2024-04-12T20:56:45.024 INFO:teuthology.orchestra.run.smithi018.stdout:{ 2024-04-12T20:56:45.024 INFO:teuthology.orchestra.run.smithi018.stdout: "mon": { 2024-04-12T20:56:45.024 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 17.2.7-827-gd3b5060a (d3b5060a17449edfae9385cdf792afec109fca5d) quincy (stable)": 2 2024-04-12T20:56:45.024 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-12T20:56:45.024 INFO:teuthology.orchestra.run.smithi018.stdout: "mgr": { 2024-04-12T20:56:45.024 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 17.2.7-827-gd3b5060a (d3b5060a17449edfae9385cdf792afec109fca5d) quincy (stable)": 1, 2024-04-12T20:56:45.024 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 19.0.0-2879-g324baf8d (324baf8ddb91dea46cac50cb535c306115791355) squid (dev)": 1 2024-04-12T20:56:45.024 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-12T20:56:45.024 INFO:teuthology.orchestra.run.smithi018.stdout: "osd": { 2024-04-12T20:56:45.025 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 17.2.7-827-gd3b5060a (d3b5060a17449edfae9385cdf792afec109fca5d) quincy (stable)": 6 2024-04-12T20:56:45.025 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-12T20:56:45.025 INFO:teuthology.orchestra.run.smithi018.stdout: "mds": { 2024-04-12T20:56:45.025 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 17.2.7-827-gd3b5060a (d3b5060a17449edfae9385cdf792afec109fca5d) quincy (stable)": 4 2024-04-12T20:56:45.025 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-12T20:56:45.025 INFO:teuthology.orchestra.run.smithi018.stdout: "overall": { 2024-04-12T20:56:45.025 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 17.2.7-827-gd3b5060a (d3b5060a17449edfae9385cdf792afec109fca5d) quincy (stable)": 13, 2024-04-12T20:56:45.025 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 19.0.0-2879-g324baf8d (324baf8ddb91dea46cac50cb535c306115791355) squid (dev)": 1 2024-04-12T20:56:45.025 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-04-12T20:56:45.025 INFO:teuthology.orchestra.run.smithi018.stdout:} 2024-04-12T20:56:45.027 INFO:tasks.workunit.client.1.smithi186.stdout:6/912: dwrite d9/d43/d59/d5d/d101/fc9 [0,4194304] 0 2024-04-12T20:56:45.029 INFO:tasks.workunit.client.1.smithi186.stdout:8/970: rename d7/de/d68/da7/dd5/fe3 to d7/de/f13e 0 2024-04-12T20:56:45.030 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:56:45.032+0000 7fae13200700 1 -- 172.21.15.18:0/1560534785 >> [v2:172.21.15.186:6828/2316349902,v1:172.21.15.186:6829/2316349902] conn(0x7fadf006bcb0 msgr2=0x7fadf006e160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:56:45.030 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:56:45.032+0000 7fae13200700 1 --2- 172.21.15.18:0/1560534785 >> [v2:172.21.15.186:6828/2316349902,v1:172.21.15.186:6829/2316349902] conn(0x7fadf006bcb0 0x7fadf006e160 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fadfc004f40 tx=0x7fadfc00b040 comp rx=0 tx=0).stop 2024-04-12T20:56:45.030 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:56:45.032+0000 7fae13200700 1 -- 172.21.15.18:0/1560534785 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fae0c0662e0 msgr2=0x7fae0c06e3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:56:45.030 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:56:45.032+0000 7fae13200700 1 --2- 172.21.15.18:0/1560534785 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fae0c0662e0 0x7fae0c06e3f0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fadf800b3b0 tx=0x7fadf8009e30 comp rx=0 tx=0).stop 2024-04-12T20:56:45.031 INFO:tasks.workunit.client.1.smithi186.stdout:7/893: truncate d7/d10/d1c/f20 1237975 0 2024-04-12T20:56:45.032 INFO:tasks.workunit.client.1.smithi186.stdout:7/894: fdatasync d7/f17 0 2024-04-12T20:56:45.032 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:56:45.034+0000 7fae13200700 1 -- 172.21.15.18:0/1560534785 shutdown_connections 2024-04-12T20:56:45.032 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:56:45.034+0000 7fae13200700 1 --2- 172.21.15.18:0/1560534785 >> [v2:172.21.15.186:6828/2316349902,v1:172.21.15.186:6829/2316349902] conn(0x7fadf006bcb0 0x7fadf006e160 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:56:45.032 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:56:45.034+0000 7fae13200700 1 --2- 172.21.15.18:0/1560534785 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fae0c0662e0 0x7fae0c06e3f0 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:56:45.033 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:56:45.034+0000 7fae13200700 1 --2- 172.21.15.18:0/1560534785 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fae0c0fff00 0x7fae0c06e930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:56:45.033 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:56:45.034+0000 7fae13200700 1 -- 172.21.15.18:0/1560534785 >> 172.21.15.18:0/1560534785 conn(0x7fae0c0f7430 msgr2=0x7fae0c0f7850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T20:56:45.034 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:56:45.037+0000 7fae13200700 1 -- 172.21.15.18:0/1560534785 shutdown_connections 2024-04-12T20:56:45.034 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:56:45.037+0000 7fae13200700 1 -- 172.21.15.18:0/1560534785 wait complete. 2024-04-12T20:56:45.048 INFO:tasks.workunit.client.1.smithi186.stdout:3/875: rename d1/d2c to d1/d3/df8/dd0/d10b/d7c/dbb/d120 0 2024-04-12T20:56:45.050 INFO:tasks.workunit.client.1.smithi186.stdout:6/913: dwrite d9/da9/fee [0,4194304] 0 2024-04-12T20:56:45.053 INFO:tasks.workunit.client.1.smithi186.stdout:3/876: read d1/d3/d21/d22/d34/ddb/f59 [369835,101517] 0 2024-04-12T20:56:45.055 INFO:tasks.workunit.client.1.smithi186.stdout:6/914: read d9/d4c/d6f/f9a [115974,59843] 0 2024-04-12T20:56:45.064 INFO:tasks.workunit.client.1.smithi186.stdout:3/877: read - d1/d3/d21/d22/d24/d28/fc9 zero size 2024-04-12T20:56:45.070 INFO:tasks.workunit.client.1.smithi186.stdout:1/940: dwrite d14/f7d [0,4194304] 0 2024-04-12T20:56:45.070 INFO:tasks.workunit.client.1.smithi186.stdout:3/878: dread - d1/d3/d21/d22/d34/ddb/d4d/d6f/ff5 zero size 2024-04-12T20:56:45.073 INFO:tasks.workunit.client.1.smithi186.stdout:6/915: link d9/f22 d9/d43/d2d/d4d/d132/f144 0 2024-04-12T20:56:45.074 INFO:tasks.workunit.client.1.smithi186.stdout:9/857: sync 2024-04-12T20:56:45.077 INFO:tasks.workunit.client.1.smithi186.stdout:6/916: mkdir d9/d4c/de9/d133/d145 0 2024-04-12T20:56:45.082 INFO:tasks.workunit.client.1.smithi186.stdout:9/858: rmdir d3/d2f/d54/d9f/dc4/d77/dd6/d72/df4/d6f 39 2024-04-12T20:56:45.084 INFO:tasks.workunit.client.1.smithi186.stdout:6/917: write d9/d43/d59/d5d/f5a [707248,62899] 0 2024-04-12T20:56:45.086 INFO:tasks.workunit.client.1.smithi186.stdout:6/918: stat d9/d43/d59/d5d/f5a 0 2024-04-12T20:56:45.087 INFO:tasks.workunit.client.1.smithi186.stdout:5/881: dwrite d5/db/d15/d2a/d4c/d5b/d7a/d94/dfa/f3a [0,4194304] 0 2024-04-12T20:56:45.090 INFO:tasks.workunit.client.1.smithi186.stdout:7/895: dread d7/d10/f42 [4194304,4194304] 0 2024-04-12T20:56:45.090 INFO:tasks.workunit.client.1.smithi186.stdout:9/859: truncate d3/d2f/d2e/d43/d4c/d82/dfd/f11e 615838 0 2024-04-12T20:56:45.102 INFO:tasks.workunit.client.1.smithi186.stdout:5/882: unlink d5/db/d2e/d70/d101/d7f/d86/db6/c8c 0 2024-04-12T20:56:45.104 INFO:tasks.workunit.client.1.smithi186.stdout:8/971: sync 2024-04-12T20:56:45.106 INFO:tasks.workunit.client.1.smithi186.stdout:9/860: dwrite d3/d13/d10b/ff5 [4194304,4194304] 0 2024-04-12T20:56:45.107 INFO:tasks.workunit.client.1.smithi186.stdout:9/861: readlink d3/l31 0 2024-04-12T20:56:45.109 INFO:tasks.workunit.client.1.smithi186.stdout:9/862: chown d3/d2f/d2e/d33/f3f 167680809 1 2024-04-12T20:56:45.113 INFO:tasks.workunit.client.1.smithi186.stdout:9/863: readlink d3/d2f/d54/d9f/dc4/d77/dc3/le3 0 2024-04-12T20:56:45.114 INFO:tasks.workunit.client.1.smithi186.stdout:6/919: rename d9/d43/d59 to d9/d5e/d7a/d146 0 2024-04-12T20:56:45.133 INFO:tasks.workunit.client.1.smithi186.stdout:7/896: rename d7/d10/d2d/d40/fc4 to d7/d127/f145 0 2024-04-12T20:56:45.136 INFO:tasks.workunit.client.1.smithi186.stdout:1/941: write d14/d1a/d33/f84 [1629412,105088] 0 2024-04-12T20:56:45.138 INFO:tasks.workunit.client.1.smithi186.stdout:9/864: creat d3/d2f/d54/d9f/dc4/d77/dd6/f128 x:0 0 0 2024-04-12T20:56:45.138 INFO:tasks.workunit.client.1.smithi186.stdout:6/920: creat d9/d4c/de9/d133/f147 x:0 0 0 2024-04-12T20:56:45.138 INFO:tasks.workunit.client.1.smithi186.stdout:8/972: dread d7/d18/d36/d1d/d1f/da2/f79 [4194304,4194304] 0 2024-04-12T20:56:45.142 INFO:tasks.workunit.client.1.smithi186.stdout:8/973: write d7/d18/d36/d1d/d1f/da2/df8/fdf [111220,38720] 0 2024-04-12T20:56:45.148 INFO:tasks.workunit.client.1.smithi186.stdout:3/879: truncate d1/d3/df8/dd0/d10b/fd3 1685084 0 2024-04-12T20:56:45.155 INFO:tasks.workunit.client.1.smithi186.stdout:6/921: dwrite d9/d5e/d7a/d146/d5d/de6/f110 [0,4194304] 0 2024-04-12T20:56:45.160 INFO:tasks.workunit.client.1.smithi186.stdout:6/922: stat d9/d43/d2d/d4d/cda 0 2024-04-12T20:56:45.163 INFO:tasks.workunit.client.1.smithi186.stdout:6/923: chown d9/d5e/d7a/d146/d5d/d6e/d10e 356 1 2024-04-12T20:56:45.281 INFO:tasks.workunit.client.1.smithi186.stdout:5/883: write d5/db/d15/d2a/f41 [974164,123986] 0 2024-04-12T20:56:45.283 INFO:tasks.workunit.client.1.smithi186.stdout:5/884: dread - d5/db/d2e/f64 zero size 2024-04-12T20:56:45.303 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:56:45.304+0000 7f8ed3e00700 1 -- 172.21.15.18:0/3289009729 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8ecc097fc0 msgr2=0x7f8ecc098430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:56:45.303 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:56:45.304+0000 7f8ed3e00700 1 --2- 172.21.15.18:0/3289009729 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8ecc097fc0 0x7f8ecc098430 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f8ec0004880 tx=0x7f8ec0009e30 comp rx=0 tx=0).stop 2024-04-12T20:56:45.303 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:56:45.305+0000 7f8ed3e00700 1 -- 172.21.15.18:0/3289009729 shutdown_connections 2024-04-12T20:56:45.303 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:56:45.305+0000 7f8ed3e00700 1 --2- 172.21.15.18:0/3289009729 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8ecc097fc0 0x7f8ecc098430 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:56:45.303 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:56:45.305+0000 7f8ed3e00700 1 --2- 172.21.15.18:0/3289009729 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8ecc096e50 0x7f8ecc097260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:56:45.303 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:56:45.305+0000 7f8ed3e00700 1 -- 172.21.15.18:0/3289009729 >> 172.21.15.18:0/3289009729 conn(0x7f8ecc092530 msgr2=0x7f8ecc094940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T20:56:45.304 INFO:tasks.workunit.client.1.smithi186.stdout:3/880: creat d1/d3/df8/dd0/d10b/d7c/dbb/d120/d48/ded/f121 x:0 0 0 2024-04-12T20:56:45.305 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:56:45.307+0000 7f8ed3e00700 1 -- 172.21.15.18:0/3289009729 shutdown_connections 2024-04-12T20:56:45.305 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:56:45.307+0000 7f8ed3e00700 1 -- 172.21.15.18:0/3289009729 wait complete. 2024-04-12T20:56:45.307 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:56:45.309+0000 7f8ed3e00700 1 Processor -- start 2024-04-12T20:56:45.308 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:56:45.310+0000 7f8ed3e00700 1 -- start start 2024-04-12T20:56:45.308 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:56:45.311+0000 7f8ed3e00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8ecc096e50 0x7f8ecc0a7960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T20:56:45.309 INFO:tasks.workunit.client.1.smithi186.stdout:8/974: dwrite d7/d18/d36/d1d/d7d/da5/f9c [0,4194304] 0 2024-04-12T20:56:45.309 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:56:45.311+0000 7f8ed3e00700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8ecc097fc0 0x7f8ecc0a7ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T20:56:45.309 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:56:45.311+0000 7f8ed3e00700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ecc0a84a0 con 0x7f8ecc096e50 2024-04-12T20:56:45.310 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:56:45.311+0000 7f8ed3e00700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ecc0a8610 con 0x7f8ecc097fc0 2024-04-12T20:56:45.310 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:56:45.312+0000 7f8ed2000700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8ecc097fc0 0x7f8ecc0a7ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T20:56:45.310 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:56:45.312+0000 7f8ed2000700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8ecc097fc0 0x7f8ecc0a7ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.18:33224/0 (socket says 172.21.15.18:33224) 2024-04-12T20:56:45.310 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:56:45.312+0000 7f8ed2000700 1 -- 172.21.15.18:0/1443077612 learned_addr learned my addr 172.21.15.18:0/1443077612 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T20:56:45.311 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:56:45.313+0000 7f8ed2000700 1 -- 172.21.15.18:0/1443077612 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8ecc096e50 msgr2=0x7f8ecc0a7960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:56:45.311 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:56:45.313+0000 7f8ed2000700 1 --2- 172.21.15.18:0/1443077612 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8ecc096e50 0x7f8ecc0a7960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:56:45.311 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:56:45.313+0000 7f8ed2000700 1 -- 172.21.15.18:0/1443077612 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ec0005040 con 0x7f8ecc097fc0 2024-04-12T20:56:45.311 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:56:45.313+0000 7f8ed2000700 1 --2- 172.21.15.18:0/1443077612 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8ecc097fc0 0x7f8ecc0a7ea0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f8ec0008cd0 tx=0x7f8ec0009e30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T20:56:45.312 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:56:45.315+0000 7f8ecaa00700 1 -- 172.21.15.18:0/1443077612 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8ec000e030 con 0x7f8ecc097fc0 2024-04-12T20:56:45.312 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:56:45.315+0000 7f8ecaa00700 1 -- 172.21.15.18:0/1443077612 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f8ec0013e60 con 0x7f8ecc097fc0 2024-04-12T20:56:45.312 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:56:45.315+0000 7f8ecaa00700 1 -- 172.21.15.18:0/1443077612 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8ec00123c0 con 0x7f8ecc097fc0 2024-04-12T20:56:45.313 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:56:45.315+0000 7f8ed3e00700 1 -- 172.21.15.18:0/1443077612 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8ecc0ad010 con 0x7f8ecc097fc0 2024-04-12T20:56:45.313 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:56:45.316+0000 7f8ed3e00700 1 -- 172.21.15.18:0/1443077612 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8ecc0ad5d0 con 0x7f8ecc097fc0 2024-04-12T20:56:45.314 INFO:tasks.workunit.client.1.smithi186.stdout:6/924: chown d9/d43/d2d/c82 7712203 1 2024-04-12T20:56:45.317 INFO:tasks.workunit.client.1.smithi186.stdout:6/925: chown d9/d5e/d7a/d146/d5d/d6e 49 1 2024-04-12T20:56:45.317 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:56:45.318+0000 7f8ebfe00700 1 -- 172.21.15.18:0/1443077612 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8eac0052b0 con 0x7f8ecc097fc0 2024-04-12T20:56:45.320 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:56:45.320+0000 7f8ecaa00700 1 -- 172.21.15.18:0/1443077612 <== mon.1 v2:172.21.15.186:3300/0 4 ==== mgrmap(e 25) v1 ==== 89542+0+0 (secure 0 0 0) 0x7f8ec001c400 con 0x7f8ecc097fc0 2024-04-12T20:56:45.320 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:56:45.322+0000 7f8ecaa00700 1 --2- 172.21.15.18:0/1443077612 >> [v2:172.21.15.186:6828/2316349902,v1:172.21.15.186:6829/2316349902] conn(0x7f8eb806bd70 0x7f8eb806e220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T20:56:45.320 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:56:45.322+0000 7f8ecaa00700 1 -- 172.21.15.18:0/1443077612 <== mon.1 v2:172.21.15.186:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5808+0+0 (secure 0 0 0) 0x7f8ec008b670 con 0x7f8ecc097fc0 2024-04-12T20:56:45.320 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:56:45.324+0000 7f8ed2a00700 1 --2- 172.21.15.18:0/1443077612 >> [v2:172.21.15.186:6828/2316349902,v1:172.21.15.186:6829/2316349902] conn(0x7f8eb806bd70 0x7f8eb806e220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T20:56:45.323 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:56:45.325+0000 7f8ed2a00700 1 --2- 172.21.15.18:0/1443077612 >> [v2:172.21.15.186:6828/2316349902,v1:172.21.15.186:6829/2316349902] conn(0x7f8eb806bd70 0x7f8eb806e220 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f8ed4063600 tx=0x7f8ed4077040 comp rx=0 tx=0).ready entity=mgr.24359 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T20:56:45.325 INFO:tasks.workunit.client.1.smithi186.stdout:9/865: creat d3/d2f/d54/d7d/d112/f129 x:0 0 0 2024-04-12T20:56:45.326 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:56:45.329+0000 7f8ecaa00700 1 -- 172.21.15.18:0/1443077612 <== mon.1 v2:172.21.15.186:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186454 (secure 0 0 0) 0x7f8ec0058aa0 con 0x7f8ecc097fc0 2024-04-12T20:56:45.346 INFO:tasks.workunit.client.1.smithi186.stdout:5/885: link d5/db/d2e/fcc d5/db/d2e/dd0/f11b 0 2024-04-12T20:56:45.353 INFO:tasks.workunit.client.1.smithi186.stdout:7/897: fsync d7/d10/d2d/d40/d4a/d6c/d97/d4d/d6a/f137 0 2024-04-12T20:56:45.361 INFO:tasks.workunit.client.1.smithi186.stdout:5/886: chown d5/db/d2e/d70/d101/d7f/daf 396738 1 2024-04-12T20:56:45.364 INFO:tasks.workunit.client.1.smithi186.stdout:5/887: write d5/db/d15/f39 [113876,97709] 0 2024-04-12T20:56:45.378 INFO:tasks.workunit.client.1.smithi186.stdout:3/881: getdents d1/d3/df8/d95/d117 0 2024-04-12T20:56:45.378 INFO:tasks.workunit.client.1.smithi186.stdout:7/898: truncate d7/d10/d2d/d40/d4a/d6c/fbc 990299 0 2024-04-12T20:56:45.379 INFO:tasks.workunit.client.1.smithi186.stdout:3/882: symlink d1/d3/da/l122 0 2024-04-12T20:56:45.380 INFO:tasks.workunit.client.1.smithi186.stdout:3/883: fdatasync d1/d3/d21/fb2 0 2024-04-12T20:56:45.382 INFO:tasks.workunit.client.1.smithi186.stdout:7/899: mknod d7/d10/d2d/d40/d4a/d6c/d97/d4d/d84/c146 0 2024-04-12T20:56:45.386 INFO:tasks.workunit.client.1.smithi186.stdout:3/884: creat d1/d3/df8/d95/f123 x:0 0 0 2024-04-12T20:56:45.394 INFO:tasks.workunit.client.1.smithi186.stdout:3/885: rmdir d1/d3/d21/d22/d24 39 2024-04-12T20:56:45.398 INFO:tasks.workunit.client.1.smithi186.stdout:5/888: dread d5/f6 [4194304,4194304] 0 2024-04-12T20:56:45.408 INFO:tasks.workunit.client.1.smithi186.stdout:5/889: unlink d5/db/f34 0 2024-04-12T20:56:45.469 INFO:tasks.workunit.client.1.smithi186.stdout:1/942: truncate d14/d1a/d33/d36/d5b/fa3 1011315 0 2024-04-12T20:56:45.477 INFO:tasks.workunit.client.1.smithi186.stdout:8/975: dwrite d7/d18/d36/d1d/d1f/da2/df8/d29/f2a [0,4194304] 0 2024-04-12T20:56:45.479 INFO:tasks.workunit.client.1.smithi186.stdout:8/976: fsync d7/d18/d36/d1d/d1f/da2/df8/d29/f6a 0 2024-04-12T20:56:45.488 INFO:tasks.workunit.client.1.smithi186.stdout:8/977: mkdir d7/de/d68/d11f/d13f 0 2024-04-12T20:56:45.498 INFO:tasks.workunit.client.1.smithi186.stdout:8/978: write d7/d18/d36/d1d/d1f/da2/f79 [868161,8768] 0 2024-04-12T20:56:45.501 INFO:tasks.workunit.client.1.smithi186.stdout:8/979: fdatasync d7/de/d10/d7f/ff4 0 2024-04-12T20:56:45.505 INFO:tasks.workunit.client.1.smithi186.stdout:8/980: symlink d7/de/d10/d7f/l140 0 2024-04-12T20:56:45.508 INFO:tasks.workunit.client.1.smithi186.stdout:7/900: rename d7/d10/d2d/d40/f77 to d7/d55/d71/dbe/f147 0 2024-04-12T20:56:45.510 INFO:tasks.workunit.client.1.smithi186.stdout:8/981: creat d7/d18/d36/d20/d71/f141 x:0 0 0 2024-04-12T20:56:45.510 INFO:tasks.workunit.client.1.smithi186.stdout:7/901: stat d7/d10/d2d/d67/l82 0 2024-04-12T20:56:45.517 INFO:tasks.workunit.client.1.smithi186.stdout:1/943: rename d14/d1a/d33/d36/ffb to d14/d91/dbd/dc5/f135 0 2024-04-12T20:56:45.526 INFO:tasks.workunit.client.1.smithi186.stdout:1/944: mknod d14/d91/c136 0 2024-04-12T20:56:45.527 INFO:tasks.workunit.client.1.smithi186.stdout:6/926: sync 2024-04-12T20:56:45.527 INFO:tasks.workunit.client.1.smithi186.stdout:9/866: sync 2024-04-12T20:56:45.528 INFO:tasks.workunit.client.1.smithi186.stdout:8/982: link d7/de/f62 d7/de/d5e/f142 0 2024-04-12T20:56:45.529 INFO:tasks.workunit.client.1.smithi186.stdout:1/945: mkdir d14/d1a/d21/d126/d137 0 2024-04-12T20:56:45.542 INFO:tasks.workunit.client.1.smithi186.stdout:6/927: rename d9/d4c/fb3 to d9/d4c/d6f/de3/f148 0 2024-04-12T20:56:45.543 INFO:tasks.workunit.client.1.smithi186.stdout:6/928: fsync d9/d4c/dbe/f122 0 2024-04-12T20:56:45.544 INFO:tasks.workunit.client.1.smithi186.stdout:1/946: dread d14/f7d [0,4194304] 0 2024-04-12T20:56:45.561 INFO:tasks.workunit.client.1.smithi186.stdout:6/929: dwrite d9/d4c/dbe/dfe/fd8 [0,4194304] 0 2024-04-12T20:56:45.561 INFO:tasks.workunit.client.1.smithi186.stdout:7/902: mkdir d7/d10/d2d/d40/d4a/d5c/d148 0 2024-04-12T20:56:45.564 INFO:tasks.workunit.client.1.smithi186.stdout:8/983: write d7/d18/d36/d1d/d1f/da2/df8/d29/f125 [305314,65689] 0 2024-04-12T20:56:45.564 INFO:tasks.workunit.client.1.smithi186.stdout:1/947: dwrite d14/d1a/d58/fb7 [0,4194304] 0 2024-04-12T20:56:45.568 INFO:tasks.workunit.client.1.smithi186.stdout:7/903: mknod d7/d10/d2d/d40/d4a/d5c/c149 0 2024-04-12T20:56:45.572 INFO:tasks.workunit.client.1.smithi186.stdout:9/867: dwrite d3/d2f/d2e/d33/f4b [0,4194304] 0 2024-04-12T20:56:45.577 INFO:tasks.workunit.client.1.smithi186.stdout:9/868: truncate d3/d2f/d103/ff6 773590 0 2024-04-12T20:56:45.577 INFO:tasks.workunit.client.1.smithi186.stdout:6/930: mknod d9/d43/d2d/c149 0 2024-04-12T20:56:45.579 INFO:tasks.workunit.client.1.smithi186.stdout:8/984: creat d7/d18/d8e/f143 x:0 0 0 2024-04-12T20:56:45.581 INFO:tasks.workunit.client.1.smithi186.stdout:6/931: chown d9/d43/d2d/d4d/d132 258705 1 2024-04-12T20:56:45.591 INFO:tasks.workunit.client.1.smithi186.stdout:9/869: mknod d3/d2f/d54/d9f/dc4/d77/dd6/c12a 0 2024-04-12T20:56:45.597 INFO:tasks.workunit.client.1.smithi186.stdout:7/904: rmdir d7/d10/d2d/d40/d4a/d6c/d97/d4d/d84/d114 0 2024-04-12T20:56:45.598 INFO:tasks.workunit.client.1.smithi186.stdout:9/870: symlink d3/d2f/d54/d7d/l12b 0 2024-04-12T20:56:45.598 INFO:tasks.workunit.client.1.smithi186.stdout:1/948: dwrite d14/d1a/d58/d69/f110 [0,4194304] 0 2024-04-12T20:56:45.601 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:56:45 smithi018 ceph-mon[28799]: from='client.14664 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T20:56:45.601 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:56:45 smithi018 ceph-mon[28799]: pgmap v55: 65 pgs: 65 active+clean; 3.2 GiB data, 11 GiB used, 526 GiB / 536 GiB avail; 60 MiB/s rd, 146 MiB/s wr, 355 op/s 2024-04-12T20:56:45.601 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:56:45 smithi018 ceph-mon[28799]: from='client.14668 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T20:56:45.601 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:56:45 smithi018 ceph-mon[28799]: from='mgr.24359 172.21.15.186:0/1315906828' entity='mgr.smithi186.kqcfxk' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T20:56:45.601 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:56:45 smithi018 ceph-mon[28799]: from='client.? 172.21.15.18:0/1560534785' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T20:56:45.601 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:56:45 smithi018 ceph-mon[28799]: from='mgr.24359 172.21.15.186:0/1315906828' entity='mgr.smithi186.kqcfxk' 2024-04-12T20:56:45.602 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:56:45 smithi018 ceph-mon[28799]: from='mgr.24359 172.21.15.186:0/1315906828' entity='mgr.smithi186.kqcfxk' 2024-04-12T20:56:45.604 INFO:tasks.workunit.client.1.smithi186.stdout:6/932: dwrite d9/da9/fc1 [0,4194304] 0 2024-04-12T20:56:45.607 INFO:tasks.workunit.client.1.smithi186.stdout:6/933: stat d9/d5e/d7a/d146/d5d/d6e/d91/lc4 0 2024-04-12T20:56:45.608 INFO:tasks.workunit.client.1.smithi186.stdout:1/949: mknod d14/d1a/d33/d36/d5b/d3e/dbb/d10f/c138 0 2024-04-12T20:56:45.616 INFO:tasks.workunit.client.1.smithi186.stdout:7/905: dread d7/d10/d2d/d40/d4a/d6c/d97/d4d/d84/fe8 [0,4194304] 0 2024-04-12T20:56:45.616 INFO:tasks.workunit.client.1.smithi186.stdout:3/886: sync 2024-04-12T20:56:45.616 INFO:tasks.workunit.client.1.smithi186.stdout:5/890: sync 2024-04-12T20:56:45.622 INFO:tasks.workunit.client.1.smithi186.stdout:6/934: unlink d9/d5e/d7a/d146/c124 0 2024-04-12T20:56:45.623 INFO:tasks.workunit.client.1.smithi186.stdout:1/950: fsync d14/d1a/d33/d36/d5b/d3e/d8b/f8e 0 2024-04-12T20:56:45.623 INFO:tasks.workunit.client.1.smithi186.stdout:8/985: dwrite d7/de/d10/d7f/f9f [4194304,4194304] 0 2024-04-12T20:56:45.627 INFO:tasks.workunit.client.1.smithi186.stdout:6/935: readlink d9/da9/dbd/leb 0 2024-04-12T20:56:45.628 INFO:tasks.workunit.client.1.smithi186.stdout:3/887: chown d1/d3/d21/d22/d24/d28/l37 68139 1 2024-04-12T20:56:45.628 INFO:tasks.workunit.client.1.smithi186.stdout:9/871: dwrite d3/d2f/d54/d9f/fa5 [0,4194304] 0 2024-04-12T20:56:45.631 INFO:tasks.workunit.client.1.smithi186.stdout:6/936: chown d9/d5e/d7a/d146/d5d/d101/lf7 780 1 2024-04-12T20:56:45.631 INFO:tasks.workunit.client.1.smithi186.stdout:3/888: chown d1/d3/d21/d5c 0 1 2024-04-12T20:56:45.633 INFO:tasks.workunit.client.1.smithi186.stdout:9/872: chown d3/d2f/l2b 1226 1 2024-04-12T20:56:45.633 INFO:tasks.workunit.client.1.smithi186.stdout:7/906: truncate d7/d127/f145 791788 0 2024-04-12T20:56:45.640 INFO:tasks.workunit.client.1.smithi186.stdout:9/873: read d3/d2f/d54/d9f/dc4/ff2 [448245,76436] 0 2024-04-12T20:56:45.643 INFO:tasks.workunit.client.1.smithi186.stdout:6/937: rename d9/d5e/d7a to d9/d5e/d7a/d81/d14a 22 2024-04-12T20:56:45.647 INFO:tasks.workunit.client.1.smithi186.stdout:1/951: dread d14/d1a/d58/fb7 [0,4194304] 0 2024-04-12T20:56:45.651 INFO:tasks.workunit.client.1.smithi186.stdout:7/907: truncate d7/f16 1120689 0 2024-04-12T20:56:45.654 INFO:tasks.workunit.client.1.smithi186.stdout:9/874: unlink d3/d2f/d54/d9f/dc4/d77/dd6/d72/df4/l63 0 2024-04-12T20:56:45.655 INFO:tasks.workunit.client.1.smithi186.stdout:5/891: write d5/db/d15/d2a/d2d/d77/fdc [428491,50465] 0 2024-04-12T20:56:45.657 INFO:tasks.workunit.client.1.smithi186.stdout:1/952: symlink d14/d74/l139 0 2024-04-12T20:56:45.659 INFO:tasks.workunit.client.1.smithi186.stdout:1/953: chown d14/d1a/d58/d69/fa4 2499 1 2024-04-12T20:56:45.660 INFO:tasks.workunit.client.1.smithi186.stdout:8/986: write d7/d18/f85 [2220935,11682] 0 2024-04-12T20:56:45.662 INFO:tasks.workunit.client.1.smithi186.stdout:7/908: creat d7/de1/f14a x:0 0 0 2024-04-12T20:56:45.672 INFO:tasks.workunit.client.1.smithi186.stdout:9/875: dwrite d3/d2f/d54/d7d/f106 [0,4194304] 0 2024-04-12T20:56:45.672 INFO:tasks.workunit.client.1.smithi186.stdout:7/909: rename d7/d10/d2d/d40/d4a/d6c/l105 to d7/d10/l14b 0 2024-04-12T20:56:45.682 INFO:tasks.workunit.client.1.smithi186.stdout:8/987: dwrite d7/de/d10/d7f/fa6 [0,4194304] 0 2024-04-12T20:56:45.687 INFO:tasks.workunit.client.1.smithi186.stdout:8/988: readlink d7/de/d10/d7f/lbd 0 2024-04-12T20:56:45.690 INFO:tasks.workunit.client.1.smithi186.stdout:8/989: chown d7/d18/d36/d1d/d1f/da2/df8/d29 28 1 2024-04-12T20:56:45.690 INFO:tasks.workunit.client.1.smithi186.stdout:3/889: dread d1/f8a [0,4194304] 0 2024-04-12T20:56:45.692 INFO:tasks.workunit.client.1.smithi186.stdout:8/990: readlink d7/d18/d36/d1d/d10a/l124 0 2024-04-12T20:56:45.693 INFO:tasks.workunit.client.1.smithi186.stdout:9/876: dread d3/d2f/d2e/d33/f4b [0,4194304] 0 2024-04-12T20:56:45.695 INFO:tasks.workunit.client.1.smithi186.stdout:3/890: write d1/d3/df8/dd0/d10b/d7c/dbb/fd7 [848007,25696] 0 2024-04-12T20:56:45.696 INFO:tasks.workunit.client.1.smithi186.stdout:7/910: write d7/d55/ffd [505637,2947] 0 2024-04-12T20:56:45.702 INFO:tasks.workunit.client.1.smithi186.stdout:7/911: creat d7/d55/d71/f14c x:0 0 0 2024-04-12T20:56:45.702 INFO:tasks.workunit.client.1.smithi186.stdout:7/912: chown c2 446 1 2024-04-12T20:56:45.706 INFO:tasks.workunit.client.1.smithi186.stdout:3/891: mkdir d1/d3/d21/d22/d34/ddb/d4d/da1/d124 0 2024-04-12T20:56:45.707 INFO:tasks.workunit.client.1.smithi186.stdout:9/877: creat d3/d2f/f12c x:0 0 0 2024-04-12T20:56:45.721 INFO:tasks.workunit.client.1.smithi186.stdout:3/892: dread d1/d3/d21/d22/fcd [0,4194304] 0 2024-04-12T20:56:45.728 INFO:tasks.workunit.client.1.smithi186.stdout:5/892: dread d5/db/d15/d2a/d4c/d5b/d7a/fb1 [0,4194304] 0 2024-04-12T20:56:45.735 INFO:tasks.workunit.client.1.smithi186.stdout:8/991: dread d7/de/d68/da7/fec [0,4194304] 0 2024-04-12T20:56:45.736 INFO:tasks.workunit.client.1.smithi186.stdout:3/893: dread d1/d3/d21/d22/d24/f53 [0,4194304] 0 2024-04-12T20:56:45.737 INFO:tasks.workunit.client.1.smithi186.stdout:3/894: chown d1/d3/da/c15 14 1 2024-04-12T20:56:45.749 INFO:tasks.workunit.client.1.smithi186.stdout:3/895: dwrite d1/d3/df8/ddc/f109 [0,4194304] 0 2024-04-12T20:56:45.752 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:56:45.753+0000 7f8ebfe00700 1 -- 172.21.15.18:0/1443077612 --> [v2:172.21.15.186:6828/2316349902,v1:172.21.15.186:6829/2316349902] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f8eac000bc0 con 0x7f8eb806bd70 2024-04-12T20:56:45.754 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:56:45.756+0000 7f8ecaa00700 1 -- 172.21.15.18:0/1443077612 <== mgr.24359 v2:172.21.15.186:6828/2316349902 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+321 (secure 0 0 0) 0x7f8eac000bc0 con 0x7f8eb806bd70 2024-04-12T20:56:45.755 INFO:teuthology.orchestra.run.smithi018.stdout:{ 2024-04-12T20:56:45.755 INFO:teuthology.orchestra.run.smithi018.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:f4452def3be4d350f4a8e5feaf7ebb1511be9298fe68478ba7a2d6fbd7d69a63", 2024-04-12T20:56:45.756 INFO:teuthology.orchestra.run.smithi018.stdout: "in_progress": true, 2024-04-12T20:56:45.756 INFO:teuthology.orchestra.run.smithi018.stdout: "which": "Upgrading daemons of type(s) mgr", 2024-04-12T20:56:45.756 INFO:teuthology.orchestra.run.smithi018.stdout: "services_complete": [], 2024-04-12T20:56:45.756 INFO:teuthology.orchestra.run.smithi018.stdout: "progress": "", 2024-04-12T20:56:45.756 INFO:teuthology.orchestra.run.smithi018.stdout: "message": "Currently upgrading mgr daemons", 2024-04-12T20:56:45.756 INFO:teuthology.orchestra.run.smithi018.stdout: "is_paused": false 2024-04-12T20:56:45.756 INFO:teuthology.orchestra.run.smithi018.stdout:} 2024-04-12T20:56:45.756 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:56:45.759+0000 7f8ebfe00700 1 -- 172.21.15.18:0/1443077612 >> [v2:172.21.15.186:6828/2316349902,v1:172.21.15.186:6829/2316349902] conn(0x7f8eb806bd70 msgr2=0x7f8eb806e220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:56:45.757 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:56:45.759+0000 7f8ebfe00700 1 --2- 172.21.15.18:0/1443077612 >> [v2:172.21.15.186:6828/2316349902,v1:172.21.15.186:6829/2316349902] conn(0x7f8eb806bd70 0x7f8eb806e220 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f8ed4063600 tx=0x7f8ed4077040 comp rx=0 tx=0).stop 2024-04-12T20:56:45.757 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:56:45.759+0000 7f8ebfe00700 1 -- 172.21.15.18:0/1443077612 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8ecc097fc0 msgr2=0x7f8ecc0a7ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:56:45.757 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:56:45.759+0000 7f8ebfe00700 1 --2- 172.21.15.18:0/1443077612 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8ecc097fc0 0x7f8ecc0a7ea0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f8ec0008cd0 tx=0x7f8ec0009e30 comp rx=0 tx=0).stop 2024-04-12T20:56:45.757 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:56:45.760+0000 7f8ebfe00700 1 -- 172.21.15.18:0/1443077612 shutdown_connections 2024-04-12T20:56:45.757 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:56:45.760+0000 7f8ebfe00700 1 --2- 172.21.15.18:0/1443077612 >> [v2:172.21.15.186:6828/2316349902,v1:172.21.15.186:6829/2316349902] conn(0x7f8eb806bd70 0x7f8eb806e220 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:56:45.757 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:56:45.760+0000 7f8ebfe00700 1 --2- 172.21.15.18:0/1443077612 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8ecc096e50 0x7f8ecc0a7960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:56:45.757 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:56:45.760+0000 7f8ebfe00700 1 --2- 172.21.15.18:0/1443077612 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8ecc097fc0 0x7f8ecc0a7ea0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:56:45.757 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:56:45.760+0000 7f8ebfe00700 1 -- 172.21.15.18:0/1443077612 >> 172.21.15.18:0/1443077612 conn(0x7f8ecc092530 msgr2=0x7f8ecc093f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T20:56:45.757 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:56:45.760+0000 7f8ebfe00700 1 -- 172.21.15.18:0/1443077612 shutdown_connections 2024-04-12T20:56:45.757 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:56:45.760+0000 7f8ebfe00700 1 -- 172.21.15.18:0/1443077612 wait complete. 2024-04-12T20:56:45.762 INFO:tasks.workunit.client.1.smithi186.stdout:7/913: rmdir d7/d10/d2d/d40/d4a/d6c/d97/d109/d124 39 2024-04-12T20:56:45.765 INFO:tasks.workunit.client.1.smithi186.stdout:9/878: rename d3/d2f/d103/dd5 to d3/d2f/d2e/d33/d12d 0 2024-04-12T20:56:45.771 INFO:tasks.workunit.client.1.smithi186.stdout:7/914: mknod d7/d10/d2d/d40/d4a/d6c/d97/dc2/c14d 0 2024-04-12T20:56:45.776 INFO:tasks.workunit.client.1.smithi186.stdout:9/879: rename d3/d2f/d2e/d33/d36/f3d to d3/d13/f12e 0 2024-04-12T20:56:45.776 INFO:tasks.workunit.client.1.smithi186.stdout:5/893: link d5/db/d15/d6a/c8d d5/db/d15/d2a/d4c/c11c 0 2024-04-12T20:56:45.778 INFO:tasks.workunit.client.1.smithi186.stdout:7/915: mkdir d7/d10/d2d/d40/d4a/d14e 0 2024-04-12T20:56:45.780 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:56:45 smithi186 ceph-mon[36597]: from='client.14664 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T20:56:45.780 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:56:45 smithi186 ceph-mon[36597]: pgmap v55: 65 pgs: 65 active+clean; 3.2 GiB data, 11 GiB used, 526 GiB / 536 GiB avail; 60 MiB/s rd, 146 MiB/s wr, 355 op/s 2024-04-12T20:56:45.780 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:56:45 smithi186 ceph-mon[36597]: from='client.14668 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T20:56:45.780 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:56:45 smithi186 ceph-mon[36597]: from='mgr.24359 172.21.15.186:0/1315906828' entity='mgr.smithi186.kqcfxk' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T20:56:45.780 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:56:45 smithi186 ceph-mon[36597]: from='client.? 172.21.15.18:0/1560534785' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T20:56:45.780 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:56:45 smithi186 ceph-mon[36597]: from='mgr.24359 172.21.15.186:0/1315906828' entity='mgr.smithi186.kqcfxk' 2024-04-12T20:56:45.780 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:56:45 smithi186 ceph-mon[36597]: from='mgr.24359 172.21.15.186:0/1315906828' entity='mgr.smithi186.kqcfxk' 2024-04-12T20:56:45.781 INFO:tasks.workunit.client.1.smithi186.stdout:8/992: getdents d7/d18/d36/d20/d23/dfd 0 2024-04-12T20:56:45.781 INFO:tasks.workunit.client.1.smithi186.stdout:8/993: stat d7/de/d5e/fc6 0 2024-04-12T20:56:45.783 INFO:tasks.workunit.client.1.smithi186.stdout:8/994: fdatasync d7/d18/d36/d20/d71/fc2 0 2024-04-12T20:56:45.786 INFO:tasks.workunit.client.1.smithi186.stdout:5/894: mkdir d5/db/d71/d88/d11d 0 2024-04-12T20:56:45.787 INFO:tasks.workunit.client.1.smithi186.stdout:9/880: write d3/d13/fb7 [381800,83238] 0 2024-04-12T20:56:45.789 INFO:tasks.workunit.client.1.smithi186.stdout:6/938: sync 2024-04-12T20:56:45.789 INFO:tasks.workunit.client.1.smithi186.stdout:1/954: sync 2024-04-12T20:56:45.798 INFO:tasks.workunit.client.1.smithi186.stdout:9/881: fdatasync d3/d2f/d54/d9f/dc4/fc1 0 2024-04-12T20:56:45.804 INFO:tasks.workunit.client.1.smithi186.stdout:8/995: link d7/de/d10/d11d/l123 d7/de/d7a/d12f/l144 0 2024-04-12T20:56:45.806 INFO:tasks.workunit.client.1.smithi186.stdout:8/996: stat d7/de/d10/d7f/dd3/l112 0 2024-04-12T20:56:45.806 INFO:tasks.workunit.client.1.smithi186.stdout:6/939: dwrite d9/d5e/d7a/d146/d5d/f48 [0,4194304] 0 2024-04-12T20:56:45.813 INFO:tasks.workunit.client.1.smithi186.stdout:1/955: dread d14/d1a/d1d/f29 [0,4194304] 0 2024-04-12T20:56:45.823 INFO:tasks.workunit.client.1.smithi186.stdout:6/940: dwrite d9/d5e/d7a/d146/d5d/f48 [0,4194304] 0 2024-04-12T20:56:45.827 INFO:tasks.workunit.client.1.smithi186.stdout:9/882: dwrite d3/d2f/d54/d7d/f121 [0,4194304] 0 2024-04-12T20:56:45.828 INFO:tasks.workunit.client.1.smithi186.stdout:1/956: getdents d14/d6c/def 0 2024-04-12T20:56:45.830 INFO:tasks.workunit.client.1.smithi186.stdout:1/957: stat d14/d1a/d33/d36/d5b/f67 0 2024-04-12T20:56:45.831 INFO:tasks.workunit.client.1.smithi186.stdout:3/896: write d1/d3/da/f47 [3996143,129833] 0 2024-04-12T20:56:45.832 INFO:tasks.workunit.client.1.smithi186.stdout:9/883: truncate d3/f73 941815 0 2024-04-12T20:56:45.832 INFO:tasks.workunit.client.1.smithi186.stdout:1/958: fdatasync d14/d1a/d1d/f119 0 2024-04-12T20:56:45.836 INFO:tasks.workunit.client.1.smithi186.stdout:9/884: mkdir d3/d2f/d103/d12f 0 2024-04-12T20:56:45.838 INFO:tasks.workunit.client.1.smithi186.stdout:9/885: readlink d3/d2f/l2b 0 2024-04-12T20:56:45.838 INFO:tasks.workunit.client.1.smithi186.stdout:3/897: creat d1/d3/d21/d22/d34/df7/f125 x:0 0 0 2024-04-12T20:56:45.840 INFO:tasks.workunit.client.1.smithi186.stdout:7/916: write d7/d10/d2d/d40/d4a/d6c/d97/d4d/f4e [1970595,81393] 0 2024-04-12T20:56:45.842 INFO:tasks.workunit.client.1.smithi186.stdout:5/895: write d5/db/d2e/d70/d101/d7f/fb7 [894056,28675] 0 2024-04-12T20:56:45.848 INFO:tasks.workunit.client.1.smithi186.stdout:9/886: mknod d3/d2f/d2e/d43/d4c/c130 0 2024-04-12T20:56:45.850 INFO:tasks.workunit.client.1.smithi186.stdout:8/997: write d7/d18/d36/d1d/d7d/f111 [631375,13874] 0 2024-04-12T20:56:45.853 INFO:tasks.workunit.client.1.smithi186.stdout:7/917: unlink d7/d10/c24 0 2024-04-12T20:56:45.855 INFO:tasks.workunit.client.1.smithi186.stdout:3/898: fdatasync d1/d3/df8/dd0/d10b/fd3 0 2024-04-12T20:56:45.855 INFO:tasks.workunit.client.1.smithi186.stdout:7/918: dread - d7/d10/d2d/d40/d4a/d5c/fd2 zero size 2024-04-12T20:56:45.862 INFO:tasks.workunit.client.1.smithi186.stdout:1/959: rename d14/l19 to d14/d1a/l13a 0 2024-04-12T20:56:45.863 INFO:tasks.workunit.client.1.smithi186.stdout:9/887: dwrite d3/d2f/d2e/d33/d61/dbc/fd3 [0,4194304] 0 2024-04-12T20:56:45.865 INFO:tasks.workunit.client.1.smithi186.stdout:1/960: read d14/d1a/d3c/f55 [69835,35619] 0 2024-04-12T20:56:45.866 INFO:tasks.workunit.client.1.smithi186.stdout:3/899: creat d1/d3/d21/d22/da4/f126 x:0 0 0 2024-04-12T20:56:45.867 INFO:tasks.workunit.client.1.smithi186.stdout:9/888: readlink d3/d2f/d103/led 0 2024-04-12T20:56:45.870 INFO:tasks.workunit.client.1.smithi186.stdout:7/919: mkdir d7/d10/d2d/d14f 0 2024-04-12T20:56:45.875 INFO:tasks.workunit.client.1.smithi186.stdout:1/961: creat d14/d6c/da9/d11e/f13b x:0 0 0 2024-04-12T20:56:45.877 INFO:tasks.workunit.client.1.smithi186.stdout:7/920: dread - d7/d55/d71/d10a/f13a zero size 2024-04-12T20:56:45.879 INFO:tasks.workunit.client.1.smithi186.stdout:1/962: chown d14/d1a/fcc 100385061 1 2024-04-12T20:56:45.880 INFO:tasks.workunit.client.1.smithi186.stdout:5/896: rename d5/db/d15/d2a/d4c/dc0/f116 to d5/db/d2e/d70/d101/dbb/dc4/f11e 0 2024-04-12T20:56:45.882 INFO:tasks.workunit.client.1.smithi186.stdout:3/900: write d1/d3/df8/dd0/d10b/d7c/fac [1790954,13776] 0 2024-04-12T20:56:45.885 INFO:tasks.workunit.client.1.smithi186.stdout:8/998: dwrite d7/de/d7a/fe7 [0,4194304] 0 2024-04-12T20:56:45.886 INFO:tasks.workunit.client.1.smithi186.stdout:6/941: dwrite d9/d5e/d7a/d146/d5d/d6e/d91/dd9/fe2 [0,4194304] 0 2024-04-12T20:56:45.889 INFO:tasks.workunit.client.1.smithi186.stdout:6/942: dread - d9/d5e/d7a/d146/f129 zero size 2024-04-12T20:56:45.895 INFO:tasks.workunit.client.1.smithi186.stdout:3/901: unlink d1/d3/d21/d22/d34/ddb/d4d/d6f/f99 0 2024-04-12T20:56:45.902 INFO:tasks.workunit.client.1.smithi186.stdout:8/999: dread d7/d18/d36/d1d/fa0 [0,4194304] 0 2024-04-12T20:56:45.906 INFO:tasks.workunit.client.1.smithi186.stdout:7/921: getdents d7 0 2024-04-12T20:56:45.909 INFO:tasks.workunit.client.1.smithi186.stdout:1/963: rename d14/c68 to d14/d1a/c13c 0 2024-04-12T20:56:45.909 INFO:tasks.workunit.client.1.smithi186.stdout:3/902: getdents d1/d3/d21/d22/d34/ddb/d4d/d6f/db0/d111 0 2024-04-12T20:56:45.909 INFO:tasks.workunit.client.1.smithi186.stdout:5/897: getdents d5/db/d2e/d70/d101/dbb/dc4/d5d 0 2024-04-12T20:56:45.913 INFO:tasks.workunit.client.1.smithi186.stdout:3/903: rmdir d1/d3/d21/d22/d34/ddb 39 2024-04-12T20:56:45.915 INFO:tasks.workunit.client.1.smithi186.stdout:1/964: mkdir d14/d91/dbd/dc5/dc8/d13d 0 2024-04-12T20:56:45.917 INFO:tasks.workunit.client.1.smithi186.stdout:5/898: creat d5/db/d15/d2a/d2d/d82/f11f x:0 0 0 2024-04-12T20:56:45.919 INFO:tasks.workunit.client.1.smithi186.stdout:1/965: write d14/d1a/d21/d35/ff6 [449606,42911] 0 2024-04-12T20:56:45.921 INFO:tasks.workunit.client.1.smithi186.stdout:7/922: write d7/f25 [3746366,92802] 0 2024-04-12T20:56:45.922 INFO:tasks.workunit.client.1.smithi186.stdout:5/899: chown d5/db/d15/d2a/d2d/d46/d55/c109 53405672 1 2024-04-12T20:56:45.923 INFO:tasks.workunit.client.1.smithi186.stdout:3/904: symlink d1/l127 0 2024-04-12T20:56:45.924 INFO:tasks.workunit.client.1.smithi186.stdout:1/966: rmdir d14/d91/dbd/dc5 39 2024-04-12T20:56:45.926 INFO:tasks.workunit.client.1.smithi186.stdout:6/943: dread d9/f1d [0,4194304] 0 2024-04-12T20:56:45.926 INFO:tasks.workunit.client.1.smithi186.stdout:1/967: fdatasync d14/d1a/d21/f31 0 2024-04-12T20:56:45.926 INFO:tasks.workunit.client.1.smithi186.stdout:7/923: mknod d7/d10/d2d/c150 0 2024-04-12T20:56:45.928 INFO:tasks.workunit.client.1.smithi186.stdout:6/944: fdatasync d9/d43/f68 0 2024-04-12T20:56:45.928 INFO:tasks.workunit.client.1.smithi186.stdout:3/905: creat d1/d3/d21/d22/d34/d6e/f128 x:0 0 0 2024-04-12T20:56:45.929 INFO:tasks.workunit.client.1.smithi186.stdout:5/900: truncate d5/db/d15/d2a/d2d/d82/fda 813101 0 2024-04-12T20:56:45.930 INFO:tasks.workunit.client.1.smithi186.stdout:6/945: dread - d9/d4c/d6f/f111 zero size 2024-04-12T20:56:45.930 INFO:tasks.workunit.client.1.smithi186.stdout:7/924: chown d7/d10/l33 235 1 2024-04-12T20:56:45.931 INFO:tasks.workunit.client.1.smithi186.stdout:1/968: rmdir d14/d1a/d21/d35/d107 39 2024-04-12T20:56:45.936 INFO:tasks.workunit.client.1.smithi186.stdout:1/969: mknod d14/c13e 0 2024-04-12T20:56:45.938 INFO:tasks.workunit.client.1.smithi186.stdout:7/925: creat d7/d10/d2d/d40/d4a/d6c/d97/dc2/f151 x:0 0 0 2024-04-12T20:56:45.941 INFO:tasks.workunit.client.1.smithi186.stdout:1/970: chown d14/d1a/d33/d36/d5b/d6a/cd4 1043070 1 2024-04-12T20:56:45.944 INFO:tasks.workunit.client.1.smithi186.stdout:3/906: write d1/d3/d21/d22/f90 [4928663,53441] 0 2024-04-12T20:56:45.952 INFO:tasks.workunit.client.1.smithi186.stdout:3/907: creat d1/d3/d21/d22/d34/ddb/f129 x:0 0 0 2024-04-12T20:56:45.958 INFO:tasks.workunit.client.1.smithi186.stdout:7/926: dwrite d7/d10/d1c/f1f [0,4194304] 0 2024-04-12T20:56:45.958 INFO:tasks.workunit.client.1.smithi186.stdout:5/901: dread d5/db/d71/f1a [0,4194304] 0 2024-04-12T20:56:45.960 INFO:tasks.workunit.client.1.smithi186.stdout:6/946: dread d9/d5e/d7a/d146/d5d/d6e/d91/f131 [0,4194304] 0 2024-04-12T20:56:45.962 INFO:tasks.workunit.client.1.smithi186.stdout:1/971: dwrite d14/d1a/d1d/de5/f106 [0,4194304] 0 2024-04-12T20:56:45.963 INFO:tasks.workunit.client.1.smithi186.stdout:6/947: write d9/d4c/d6f/f111 [333171,401] 0 2024-04-12T20:56:45.965 INFO:tasks.workunit.client.1.smithi186.stdout:6/948: write d9/d4c/de9/f140 [600739,1447] 0 2024-04-12T20:56:45.968 INFO:tasks.workunit.client.1.smithi186.stdout:7/927: rename d7/fc to d7/deb/f152 0 2024-04-12T20:56:45.969 INFO:tasks.workunit.client.1.smithi186.stdout:5/902: rmdir d5/db/d71 39 2024-04-12T20:56:45.970 INFO:tasks.workunit.client.1.smithi186.stdout:6/949: chown d9/d4c/d6f/f7b 3873192 1 2024-04-12T20:56:45.974 INFO:tasks.workunit.client.1.smithi186.stdout:9/889: sync 2024-04-12T20:56:45.976 INFO:tasks.workunit.client.1.smithi186.stdout:3/908: link d1/d3/da2/f9d d1/d3/d21/d22/d34/ddb/d4d/d6f/da6/f12a 0 2024-04-12T20:56:45.980 INFO:tasks.workunit.client.1.smithi186.stdout:7/928: mkdir d7/d10/d1c/d5d/db7/d153 0 2024-04-12T20:56:45.980 INFO:tasks.workunit.client.1.smithi186.stdout:1/972: mknod d14/d1a/d21/d35/d107/c13f 0 2024-04-12T20:56:45.981 INFO:tasks.workunit.client.1.smithi186.stdout:9/890: write d3/fce [4069327,6726] 0 2024-04-12T20:56:45.989 INFO:tasks.workunit.client.1.smithi186.stdout:7/929: truncate d7/d10/d1c/fdb 1181821 0 2024-04-12T20:56:45.993 INFO:tasks.workunit.client.1.smithi186.stdout:9/891: getdents d3/d2f/d54/d7d/d112 0 2024-04-12T20:56:45.993 INFO:tasks.workunit.client.1.smithi186.stdout:7/930: chown d7/d55/f9f 4017 1 2024-04-12T20:56:45.995 INFO:tasks.workunit.client.1.smithi186.stdout:1/973: write d14/d1a/d1d/f6f [3424218,40926] 0 2024-04-12T20:56:45.995 INFO:tasks.workunit.client.1.smithi186.stdout:6/950: dwrite d9/d4c/de9/d133/f147 [0,4194304] 0 2024-04-12T20:56:45.997 INFO:tasks.workunit.client.1.smithi186.stdout:7/931: write d7/d55/f10e [293392,55761] 0 2024-04-12T20:56:46.001 INFO:tasks.workunit.client.1.smithi186.stdout:9/892: mkdir d3/d2f/d54/d9f/dc4/d77/dd6/d131 0 2024-04-12T20:56:46.002 INFO:tasks.workunit.client.1.smithi186.stdout:1/974: symlink d14/d1a/d21/d35/l140 0 2024-04-12T20:56:46.005 INFO:tasks.workunit.client.1.smithi186.stdout:5/903: dwrite d5/db/d2e/d70/d101/d7f/d80/fcd [0,4194304] 0 2024-04-12T20:56:46.006 INFO:tasks.workunit.client.1.smithi186.stdout:1/975: dread - d14/d1a/d33/f114 zero size 2024-04-12T20:56:46.008 INFO:tasks.workunit.client.1.smithi186.stdout:7/932: mkdir d7/d10/d2d/d40/d4a/d5c/d13b/d154 0 2024-04-12T20:56:46.012 INFO:tasks.workunit.client.1.smithi186.stdout:1/976: stat d14/d91/dbd/dc5/f135 0 2024-04-12T20:56:46.026 INFO:tasks.workunit.client.1.smithi186.stdout:6/951: truncate d9/d5e/d73/f84 7799900 0 2024-04-12T20:56:46.026 INFO:tasks.workunit.client.1.smithi186.stdout:5/904: write d5/db/f11 [4107832,93804] 0 2024-04-12T20:56:46.026 INFO:tasks.workunit.client.1.smithi186.stdout:1/977: write d14/d1a/d21/d35/d59/fbf [2458550,27284] 0 2024-04-12T20:56:46.028 INFO:tasks.workunit.client.1.smithi186.stdout:1/978: read - d14/fcf zero size 2024-04-12T20:56:46.030 INFO:tasks.workunit.client.1.smithi186.stdout:1/979: fdatasync d14/f15 0 2024-04-12T20:56:46.033 INFO:tasks.workunit.client.1.smithi186.stdout:6/952: rename d9/d5e/d7a/d146/ccc to d9/d5e/d7a/d146/d5d/d101/d13a/c14b 0 2024-04-12T20:56:46.038 INFO:tasks.workunit.client.1.smithi186.stdout:5/905: rename d5/db/d2e/d70/d101/d7f to d5/db/d15/d2a/d4c/d5b/d7a/d94/d106/d120 0 2024-04-12T20:56:46.038 INFO:tasks.workunit.client.1.smithi186.stdout:7/933: dwrite d7/d10/d2d/d40/f10c [0,4194304] 0 2024-04-12T20:56:46.039 INFO:tasks.workunit.client.1.smithi186.stdout:9/893: dread d3/d13/f65 [0,4194304] 0 2024-04-12T20:56:46.044 INFO:tasks.workunit.client.1.smithi186.stdout:6/953: mknod d9/d43/d2d/d4d/d132/c14c 0 2024-04-12T20:56:46.047 INFO:tasks.workunit.client.1.smithi186.stdout:5/906: truncate d5/db/d15/d2a/d4c/f62 692831 0 2024-04-12T20:56:46.048 INFO:tasks.workunit.client.1.smithi186.stdout:9/894: mkdir d3/d2f/d54/d7d/d112/d132 0 2024-04-12T20:56:46.050 INFO:tasks.workunit.client.1.smithi186.stdout:6/954: symlink d9/d4c/d6f/de3/l14d 0 2024-04-12T20:56:46.051 INFO:tasks.workunit.client.1.smithi186.stdout:5/907: fsync d5/db/d15/d2a/d4c/d5b/d7a/d94/d106/d120/fb7 0 2024-04-12T20:56:46.051 INFO:tasks.workunit.client.1.smithi186.stdout:9/895: write d3/d2f/d54/d9f/fa5 [3334481,70451] 0 2024-04-12T20:56:46.055 INFO:tasks.workunit.client.1.smithi186.stdout:9/896: chown d3/d2f/d2e/f2d 248915 1 2024-04-12T20:56:46.056 INFO:tasks.workunit.client.1.smithi186.stdout:1/980: rename d14/d1a/d33/d36/d5b/d3e/d8b/fde to d14/d1a/d33/f141 0 2024-04-12T20:56:46.057 INFO:tasks.workunit.client.1.smithi186.stdout:9/897: chown d3/d2f/d2e/f2d 5725205 1 2024-04-12T20:56:46.058 INFO:tasks.workunit.client.1.smithi186.stdout:9/898: fdatasync d3/fb 0 2024-04-12T20:56:46.062 INFO:tasks.workunit.client.1.smithi186.stdout:5/908: getdents d5/db 0 2024-04-12T20:56:46.065 INFO:tasks.workunit.client.1.smithi186.stdout:1/981: chown d14/d1a/d33/d36/d5b/d3e/d71/cec 136 1 2024-04-12T20:56:46.066 INFO:tasks.workunit.client.1.smithi186.stdout:6/955: getdents d9/d5e/d7a/d146/d5d/d6e/d91 0 2024-04-12T20:56:46.066 INFO:tasks.workunit.client.1.smithi186.stdout:9/899: chown d3/d2f/d103/cc5 5913 1 2024-04-12T20:56:46.067 INFO:tasks.workunit.client.1.smithi186.stdout:5/909: write d5/db/d15/d2a/d4c/d93/fcf [951935,100613] 0 2024-04-12T20:56:46.070 INFO:tasks.workunit.client.1.smithi186.stdout:5/910: stat d5/db/d2e/d70/d101/dbb/dc4/d5d/c78 0 2024-04-12T20:56:46.078 INFO:tasks.workunit.client.1.smithi186.stdout:6/956: unlink d9/l21 0 2024-04-12T20:56:46.079 INFO:tasks.workunit.client.1.smithi186.stdout:1/982: mknod d14/d1a/d1d/c142 0 2024-04-12T20:56:46.080 INFO:tasks.workunit.client.1.smithi186.stdout:7/934: dwrite d7/d10/d2d/d40/ff1 [0,4194304] 0 2024-04-12T20:56:46.082 INFO:tasks.workunit.client.1.smithi186.stdout:5/911: rename d5/db/d33/c73 to d5/db/d2e/d70/d101/dbb/c121 0 2024-04-12T20:56:46.085 INFO:tasks.workunit.client.1.smithi186.stdout:5/912: write d5/db/d15/f89 [4263060,4318] 0 2024-04-12T20:56:46.086 INFO:tasks.workunit.client.1.smithi186.stdout:1/983: creat d14/d1a/d3c/f143 x:0 0 0 2024-04-12T20:56:46.086 INFO:tasks.workunit.client.1.smithi186.stdout:6/957: mkdir d9/d5e/d7a/d146/d5d/d6e/d91/dd9/d14e 0 2024-04-12T20:56:46.088 INFO:tasks.workunit.client.1.smithi186.stdout:1/984: chown d14/d1a/d33/d36/d5b/d3e/d5f 1 1 2024-04-12T20:56:46.089 INFO:tasks.workunit.client.1.smithi186.stdout:7/935: mkdir d7/d10/d1c/d5d/db7/d155 0 2024-04-12T20:56:46.091 INFO:tasks.workunit.client.1.smithi186.stdout:9/900: dread d3/d2f/d54/d92/fef [0,4194304] 0 2024-04-12T20:56:46.092 INFO:tasks.workunit.client.1.smithi186.stdout:9/901: stat d3/d2f/d2e/d33/d36/c39 0 2024-04-12T20:56:46.095 INFO:tasks.workunit.client.1.smithi186.stdout:5/913: creat d5/db/d15/f122 x:0 0 0 2024-04-12T20:56:46.098 INFO:tasks.workunit.client.1.smithi186.stdout:3/909: sync 2024-04-12T20:56:46.099 INFO:tasks.workunit.client.1.smithi186.stdout:6/958: creat d9/d4c/de9/d133/f14f x:0 0 0 2024-04-12T20:56:46.103 INFO:tasks.workunit.client.1.smithi186.stdout:7/936: rmdir d7/d10/d1c/d8c 39 2024-04-12T20:56:46.104 INFO:tasks.workunit.client.1.smithi186.stdout:1/985: creat d14/d1a/d21/d35/d59/f144 x:0 0 0 2024-04-12T20:56:46.106 INFO:tasks.workunit.client.1.smithi186.stdout:5/914: chown d5/d27/cf1 1 1 2024-04-12T20:56:46.107 INFO:tasks.workunit.client.1.smithi186.stdout:3/910: symlink d1/d3/df8/d95/d117/l12b 0 2024-04-12T20:56:46.109 INFO:tasks.workunit.client.1.smithi186.stdout:7/937: mknod d7/d55/c156 0 2024-04-12T20:56:46.111 INFO:tasks.workunit.client.1.smithi186.stdout:1/986: mknod d14/d1a/d33/d36/dfa/c145 0 2024-04-12T20:56:46.120 INFO:tasks.workunit.client.1.smithi186.stdout:3/911: rename d1/d3/d21/d22/d34/ddb/d4d/la3 to d1/d3/d21/d22/d34/df7/l12c 0 2024-04-12T20:56:46.120 INFO:tasks.workunit.client.1.smithi186.stdout:1/987: chown d14/d1a/d33/d36/d5b/db9/dc9 24 1 2024-04-12T20:56:46.121 INFO:tasks.workunit.client.1.smithi186.stdout:9/902: link d3/d2f/d2e/d43/d4c/d82/dfd/l11c d3/d2f/d2e/d33/d61/dbc/d105/l133 0 2024-04-12T20:56:46.121 INFO:tasks.workunit.client.1.smithi186.stdout:6/959: link d9/d5e/fb0 d9/d4c/dbe/f150 0 2024-04-12T20:56:46.124 INFO:tasks.workunit.client.1.smithi186.stdout:3/912: fsync d1/d3/df8/dd0/d10b/d7c/dbb/d120/d48/ded/f121 0 2024-04-12T20:56:46.125 INFO:tasks.workunit.client.1.smithi186.stdout:1/988: unlink d14/d1a/d58/l127 0 2024-04-12T20:56:46.131 INFO:tasks.workunit.client.1.smithi186.stdout:3/913: creat d1/d3/d21/d22/da4/f12d x:0 0 0 2024-04-12T20:56:46.133 INFO:tasks.workunit.client.1.smithi186.stdout:1/989: unlink d14/d1a/d1d/c142 0 2024-04-12T20:56:46.140 INFO:tasks.workunit.client.1.smithi186.stdout:5/915: mknod d5/db/d15/d2a/d4c/d5b/d7a/d94/dfa/da4/c123 0 2024-04-12T20:56:46.140 INFO:tasks.workunit.client.1.smithi186.stdout:7/938: link d7/d55/f9f d7/d10/f157 0 2024-04-12T20:56:46.141 INFO:tasks.workunit.client.1.smithi186.stdout:3/914: symlink d1/d3/d21/d22/d34/d6e/l12e 0 2024-04-12T20:56:46.143 INFO:tasks.workunit.client.1.smithi186.stdout:7/939: stat d7/d10/d2d/d40/d4a/d6c/d97/d4d/d6a 0 2024-04-12T20:56:46.144 INFO:tasks.workunit.client.1.smithi186.stdout:3/915: chown d1/d3/d21/d22/da4/lba 3420 1 2024-04-12T20:56:46.150 INFO:tasks.workunit.client.1.smithi186.stdout:7/940: mkdir d7/d10/d2d/d67/d158 0 2024-04-12T20:56:46.151 INFO:tasks.workunit.client.1.smithi186.stdout:3/916: mkdir d1/d3/df8/dd0/d10b/d12f 0 2024-04-12T20:56:46.157 INFO:tasks.workunit.client.1.smithi186.stdout:7/941: rmdir d7/d55/d71 39 2024-04-12T20:56:46.159 INFO:tasks.workunit.client.1.smithi186.stdout:7/942: stat d7/d10/d2d/d40/d4a/d5c/c13c 0 2024-04-12T20:56:46.164 INFO:tasks.workunit.client.1.smithi186.stdout:9/903: write d3/d2f/f6d [428620,66669] 0 2024-04-12T20:56:46.167 INFO:tasks.workunit.client.1.smithi186.stdout:3/917: getdents d1 0 2024-04-12T20:56:46.170 INFO:tasks.workunit.client.1.smithi186.stdout:6/960: write d9/f4b [3099635,122226] 0 2024-04-12T20:56:46.171 INFO:tasks.workunit.client.1.smithi186.stdout:1/990: write d14/d1a/d21/d35/fe6 [660335,13987] 0 2024-04-12T20:56:46.172 INFO:tasks.workunit.client.1.smithi186.stdout:3/918: fdatasync d1/d3/d21/fb1 0 2024-04-12T20:56:46.173 INFO:tasks.workunit.client.1.smithi186.stdout:5/916: write d5/db/d2e/d70/d101/dbb/dc4/d5d/fef [114471,35891] 0 2024-04-12T20:56:46.173 INFO:tasks.workunit.client.1.smithi186.stdout:1/991: fdatasync d14/d6c/feb 0 2024-04-12T20:56:46.176 INFO:tasks.workunit.client.1.smithi186.stdout:3/919: chown d1/d3/d21/d22/d34 525235 1 2024-04-12T20:56:46.176 INFO:tasks.workunit.client.1.smithi186.stdout:1/992: write d14/d6c/da9/d11e/f13b [200415,18873] 0 2024-04-12T20:56:46.179 INFO:tasks.workunit.client.1.smithi186.stdout:6/961: creat d9/d43/f151 x:0 0 0 2024-04-12T20:56:46.181 INFO:tasks.workunit.client.1.smithi186.stdout:5/917: symlink d5/db/d2e/d70/d101/dbb/dc4/d5d/l124 0 2024-04-12T20:56:46.183 INFO:tasks.workunit.client.1.smithi186.stdout:3/920: unlink d1/d3/d21/d22/d4a/dbc/l11a 0 2024-04-12T20:56:46.186 INFO:tasks.workunit.client.1.smithi186.stdout:3/921: chown d1/d3/d21/d22/d34/ddb/d4d/d9c/fa7 245086484 1 2024-04-12T20:56:46.187 INFO:tasks.workunit.client.1.smithi186.stdout:6/962: unlink l4 0 2024-04-12T20:56:46.188 INFO:tasks.workunit.client.1.smithi186.stdout:5/918: getdents d5/db/d71/d88/d11d 0 2024-04-12T20:56:46.194 INFO:tasks.workunit.client.1.smithi186.stdout:1/993: link d14/d1a/d33/d36/d5b/c44 d14/d6c/da9/c146 0 2024-04-12T20:56:46.195 INFO:tasks.workunit.client.1.smithi186.stdout:3/922: truncate d1/d3/d21/d22/d34/ddb/d4d/d6f/ff5 99822 0 2024-04-12T20:56:46.195 INFO:tasks.workunit.client.1.smithi186.stdout:5/919: dread - d5/db/d15/d2a/d4c/d5b/d7a/d94/dfa/da4/fa7 zero size 2024-04-12T20:56:46.198 INFO:tasks.workunit.client.1.smithi186.stdout:6/963: rename d9/d4c/d6f/f9a to d9/da9/f152 0 2024-04-12T20:56:46.200 INFO:tasks.workunit.client.1.smithi186.stdout:1/994: mkdir d14/d91/dbd/dc5/d147 0 2024-04-12T20:56:46.202 INFO:tasks.workunit.client.1.smithi186.stdout:7/943: dread d7/d10/d1c/f2c [4194304,4194304] 0 2024-04-12T20:56:46.202 INFO:tasks.workunit.client.1.smithi186.stdout:6/964: chown d9/d4c/d6f/d89/d9c/cef 248 1 2024-04-12T20:56:46.202 INFO:tasks.workunit.client.1.smithi186.stdout:3/923: readlink d1/d3/d21/d22/d4a/dbc/le7 0 2024-04-12T20:56:46.212 INFO:tasks.workunit.client.1.smithi186.stdout:1/995: chown d14/d1a/d33/d36/d5b/d3e/d8b/c121 203670 1 2024-04-12T20:56:46.213 INFO:tasks.workunit.client.1.smithi186.stdout:1/996: readlink d14/d74/l87 0 2024-04-12T20:56:46.215 INFO:tasks.workunit.client.1.smithi186.stdout:7/944: dread - d7/de1/fe9 zero size 2024-04-12T20:56:46.217 INFO:tasks.workunit.client.1.smithi186.stdout:3/924: mknod d1/d101/c130 0 2024-04-12T20:56:46.218 INFO:tasks.workunit.client.1.smithi186.stdout:1/997: mknod d14/d91/dbd/dc5/c148 0 2024-04-12T20:56:46.234 INFO:tasks.workunit.client.1.smithi186.stdout:1/998: creat d14/d74/f149 x:0 0 0 2024-04-12T20:56:46.234 INFO:tasks.workunit.client.1.smithi186.stdout:1/999: dread - d14/d1a/d21/d126/f12c zero size 2024-04-12T20:56:46.235 INFO:tasks.workunit.client.1.smithi186.stdout:3/925: readlink d1/d3/d21/d22/d4a/l86 0 2024-04-12T20:56:46.235 INFO:tasks.workunit.client.1.smithi186.stdout:7/945: mkdir d7/d10/d2d/d40/d4a/d6c/d97/d4d/d84/d159 0 2024-04-12T20:56:46.238 INFO:tasks.workunit.client.1.smithi186.stdout:3/926: rmdir d1/d3/d21/d22/d34/ddb/d4d/d6f/da6 39 2024-04-12T20:56:46.240 INFO:tasks.workunit.client.1.smithi186.stdout:5/920: creat d5/db/d2e/d70/d101/dbb/dc4/d5d/de1/f125 x:0 0 0 2024-04-12T20:56:46.244 INFO:tasks.workunit.client.1.smithi186.stdout:3/927: dread d1/d3/d21/d22/d34/ddb/d4d/d6f/ff5 [0,4194304] 0 2024-04-12T20:56:46.245 INFO:tasks.workunit.client.1.smithi186.stdout:7/946: dread d7/d55/d71/f104 [0,4194304] 0 2024-04-12T20:56:46.245 INFO:tasks.workunit.client.1.smithi186.stdout:6/965: dwrite d9/d5e/f9d [0,4194304] 0 2024-04-12T20:56:46.247 INFO:tasks.workunit.client.1.smithi186.stdout:9/904: sync 2024-04-12T20:56:46.249 INFO:tasks.workunit.client.1.smithi186.stdout:3/928: creat d1/d3/df8/d94/f131 x:0 0 0 2024-04-12T20:56:46.251 INFO:tasks.workunit.client.1.smithi186.stdout:6/966: rmdir d9/d43/d2d/d4d 39 2024-04-12T20:56:46.254 INFO:tasks.workunit.client.1.smithi186.stdout:5/921: fdatasync d5/db/d2e/d70/d101/f105 0 2024-04-12T20:56:46.257 INFO:tasks.workunit.client.1.smithi186.stdout:9/905: unlink d3/d2f/d2e/d43/d4c/f58 0 2024-04-12T20:56:46.262 INFO:tasks.workunit.client.1.smithi186.stdout:6/967: rmdir d9/d4c/d6f/d89/d9c 39 2024-04-12T20:56:46.264 INFO:tasks.workunit.client.1.smithi186.stdout:7/947: symlink d7/d10/d2d/d40/d4a/d5c/d13b/d154/l15a 0 2024-04-12T20:56:46.264 INFO:tasks.workunit.client.1.smithi186.stdout:6/968: fsync d9/d43/d2d/f12c 0 2024-04-12T20:56:46.267 INFO:tasks.workunit.client.1.smithi186.stdout:3/929: truncate d1/d3/df8/d95/ff3 728552 0 2024-04-12T20:56:46.269 INFO:tasks.workunit.client.1.smithi186.stdout:9/906: symlink d3/d2f/d54/d7d/d112/l134 0 2024-04-12T20:56:46.272 INFO:tasks.workunit.client.1.smithi186.stdout:7/948: rename d7/d10/d2d/d40/d4a/d6c/d97/d4d/d84 to d7/d10/d1c/d5d/db7/d15b 0 2024-04-12T20:56:46.276 INFO:tasks.workunit.client.1.smithi186.stdout:6/969: fdatasync d9/f47 0 2024-04-12T20:56:46.278 INFO:tasks.workunit.client.1.smithi186.stdout:5/922: creat d5/db/d2e/d70/d101/dbb/dc4/d5d/f126 x:0 0 0 2024-04-12T20:56:46.280 INFO:tasks.workunit.client.1.smithi186.stdout:7/949: creat d7/d10/d1c/d5d/db7/d15b/f15c x:0 0 0 2024-04-12T20:56:46.282 INFO:tasks.workunit.client.1.smithi186.stdout:5/923: fdatasync d5/db/d15/d2a/d2d/d46/f6c 0 2024-04-12T20:56:46.284 INFO:tasks.workunit.client.1.smithi186.stdout:6/970: write d9/d4c/f61 [3590581,36027] 0 2024-04-12T20:56:46.284 INFO:tasks.workunit.client.1.smithi186.stdout:9/907: dwrite d3/ff [0,4194304] 0 2024-04-12T20:56:46.285 INFO:tasks.workunit.client.1.smithi186.stdout:7/950: fsync d7/d10/d2d/d86/f9c 0 2024-04-12T20:56:46.292 INFO:tasks.workunit.client.1.smithi186.stdout:7/951: chown d7/d10/d1c/d8c/c10d 31315936 1 2024-04-12T20:56:46.297 INFO:tasks.workunit.client.1.smithi186.stdout:9/908: rmdir d3/d2f/d54/d9f/dc4/d77/dd6/d72/df4/d6f 39 2024-04-12T20:56:46.298 INFO:tasks.workunit.client.1.smithi186.stdout:3/930: write d1/d3/da2/fe9 [968445,86383] 0 2024-04-12T20:56:46.302 INFO:tasks.workunit.client.1.smithi186.stdout:5/924: link d5/db/d2e/d70/d101/fb9 d5/db/d15/d2a/d4c/d5b/d7a/d94/d106/d120/d80/f127 0 2024-04-12T20:56:46.303 INFO:tasks.workunit.client.1.smithi186.stdout:9/909: write d3/d2f/d54/d9f/dc4/d77/dd6/d72/df4/df9/f11a [3717933,15288] 0 2024-04-12T20:56:46.305 INFO:tasks.workunit.client.1.smithi186.stdout:7/952: creat d7/d10/d2d/d40/d4a/d5c/d148/f15d x:0 0 0 2024-04-12T20:56:46.309 INFO:tasks.workunit.client.1.smithi186.stdout:9/910: symlink d3/d2f/l135 0 2024-04-12T20:56:46.310 INFO:tasks.workunit.client.1.smithi186.stdout:5/925: mknod d5/db/d15/d2a/d4c/d5b/d7a/d94/d106/c128 0 2024-04-12T20:56:46.313 INFO:tasks.workunit.client.1.smithi186.stdout:5/926: mknod d5/db/d2e/d70/c129 0 2024-04-12T20:56:46.313 INFO:tasks.workunit.client.1.smithi186.stdout:9/911: mkdir d3/d2f/d54/d9f/dc4/d6b/d136 0 2024-04-12T20:56:46.317 INFO:tasks.workunit.client.1.smithi186.stdout:3/931: dwrite d1/d3/d21/d22/d34/ddb/f129 [0,4194304] 0 2024-04-12T20:56:46.320 INFO:tasks.workunit.client.1.smithi186.stdout:9/912: unlink d3/d2f/d54/d9f/dc4/d77/dd6/f128 0 2024-04-12T20:56:46.325 INFO:tasks.workunit.client.1.smithi186.stdout:5/927: mkdir d5/db/d15/d2a/d4c/d5b/d7a/d94/dfa/da4/d12a 0 2024-04-12T20:56:46.327 INFO:tasks.workunit.client.1.smithi186.stdout:5/928: chown d5/db/d15/d2a/d4c 502697153 1 2024-04-12T20:56:46.329 INFO:tasks.workunit.client.1.smithi186.stdout:9/913: creat d3/d2f/d54/d9f/dc4/d77/dd6/f137 x:0 0 0 2024-04-12T20:56:46.329 INFO:tasks.workunit.client.1.smithi186.stdout:3/932: creat d1/d3/d21/d22/d34/ddb/d4d/d6f/db0/d111/f132 x:0 0 0 2024-04-12T20:56:46.333 INFO:tasks.workunit.client.1.smithi186.stdout:3/933: rmdir d1/d3/da 39 2024-04-12T20:56:46.337 INFO:tasks.workunit.client.1.smithi186.stdout:3/934: mknod d1/d3/df8/ddc/c133 0 2024-04-12T20:56:46.339 INFO:tasks.workunit.client.1.smithi186.stdout:9/914: write d3/d2f/d2e/f41 [1343669,75937] 0 2024-04-12T20:56:46.343 INFO:tasks.workunit.client.1.smithi186.stdout:3/935: mknod d1/d3/df8/dd0/d10b/d7c/dbb/d120/d48/ded/c134 0 2024-04-12T20:56:46.343 INFO:tasks.workunit.client.1.smithi186.stdout:5/929: dwrite d5/db/d15/d2a/d4c/d5b/d7a/d94/dfa/f87 [0,4194304] 0 2024-04-12T20:56:46.351 INFO:tasks.workunit.client.1.smithi186.stdout:5/930: mkdir d5/db/d33/d10e/d12b 0 2024-04-12T20:56:46.352 INFO:tasks.workunit.client.1.smithi186.stdout:3/936: rename d1/d3/da/c26 to d1/d3/d21/d22/d4a/c135 0 2024-04-12T20:56:46.353 INFO:tasks.workunit.client.1.smithi186.stdout:5/931: rmdir d5 39 2024-04-12T20:56:46.358 INFO:tasks.workunit.client.1.smithi186.stdout:9/915: dwrite d3/d2f/f10a [0,4194304] 0 2024-04-12T20:56:46.358 INFO:tasks.workunit.client.1.smithi186.stdout:6/971: sync 2024-04-12T20:56:46.359 INFO:tasks.workunit.client.1.smithi186.stdout:7/953: sync 2024-04-12T20:56:46.360 INFO:tasks.workunit.client.1.smithi186.stdout:6/972: fdatasync d9/d43/f44 0 2024-04-12T20:56:46.367 INFO:tasks.workunit.client.1.smithi186.stdout:6/973: fsync d9/d4c/dbe/dfe/fd8 0 2024-04-12T20:56:46.367 INFO:tasks.workunit.client.1.smithi186.stdout:9/916: creat d3/d2f/d2e/d33/d36/d93/d123/f138 x:0 0 0 2024-04-12T20:56:46.377 INFO:tasks.workunit.client.1.smithi186.stdout:7/954: getdents d7/d10/d2d/d40/d4a/d5c/d13b 0 2024-04-12T20:56:46.380 INFO:tasks.workunit.client.1.smithi186.stdout:9/917: mkdir d3/d2f/d2e/d33/d61/dbc/d139 0 2024-04-12T20:56:46.380 INFO:tasks.workunit.client.1.smithi186.stdout:6/974: symlink d9/l153 0 2024-04-12T20:56:46.382 INFO:tasks.workunit.client.1.smithi186.stdout:7/955: read - d7/d10/d2d/d40/d4a/d6c/ddf/ffe zero size 2024-04-12T20:56:46.383 INFO:tasks.workunit.client.1.smithi186.stdout:3/937: dwrite d1/d3/df8/ddc/ffc [0,4194304] 0 2024-04-12T20:56:46.387 INFO:tasks.workunit.client.1.smithi186.stdout:6/975: symlink d9/d4c/dbe/dfe/l154 0 2024-04-12T20:56:46.390 INFO:tasks.workunit.client.1.smithi186.stdout:3/938: fsync d1/d3/d21/d22/d34/ddb/d4d/d9c/fcb 0 2024-04-12T20:56:46.390 INFO:tasks.workunit.client.1.smithi186.stdout:5/932: dwrite d5/db/f5f [0,4194304] 0 2024-04-12T20:56:46.394 INFO:tasks.workunit.client.1.smithi186.stdout:6/976: creat d9/d5e/d7a/d146/d116/d11a/d13d/f155 x:0 0 0 2024-04-12T20:56:46.401 INFO:tasks.workunit.client.1.smithi186.stdout:5/933: mknod d5/db/d15/d2a/d2d/d82/c12c 0 2024-04-12T20:56:46.402 INFO:tasks.workunit.client.1.smithi186.stdout:6/977: write d9/d43/f151 [722600,117930] 0 2024-04-12T20:56:46.409 INFO:tasks.workunit.client.1.smithi186.stdout:7/956: dwrite d7/d10/d2d/f37 [0,4194304] 0 2024-04-12T20:56:46.410 INFO:tasks.workunit.client.1.smithi186.stdout:3/939: link d1/d3/da/l2b d1/d3/d21/d22/d34/ddb/d4d/d6f/l136 0 2024-04-12T20:56:46.412 INFO:tasks.workunit.client.1.smithi186.stdout:6/978: mknod d9/d5e/d7a/d146/d5d/d6e/d91/dd9/c156 0 2024-04-12T20:56:46.415 INFO:tasks.workunit.client.1.smithi186.stdout:9/918: dwrite d3/d2f/d54/d9f/fa5 [0,4194304] 0 2024-04-12T20:56:46.420 INFO:tasks.workunit.client.1.smithi186.stdout:9/919: fsync d3/d2f/d2e/d33/fb9 0 2024-04-12T20:56:46.423 INFO:tasks.workunit.client.1.smithi186.stdout:7/957: readlink d7/d10/d1c/d5d/la8 0 2024-04-12T20:56:46.423 INFO:tasks.workunit.client.1.smithi186.stdout:3/940: fsync d1/d3/df8/dd0/d10b/d7c/dbb/d120/d48/f5f 0 2024-04-12T20:56:46.433 INFO:tasks.workunit.client.1.smithi186.stdout:6/979: creat d9/d5e/d7a/d146/d5d/d6e/d10e/f157 x:0 0 0 2024-04-12T20:56:46.440 INFO:tasks.workunit.client.1.smithi186.stdout:5/934: dwrite d5/db/d15/d2a/d2d/d82/d84/fcb [0,4194304] 0 2024-04-12T20:56:46.445 INFO:tasks.workunit.client.1.smithi186.stdout:6/980: read d9/f1b [589744,1424] 0 2024-04-12T20:56:46.446 INFO:tasks.workunit.client.1.smithi186.stdout:5/935: truncate d5/db/d2e/d70/de2/f108 501764 0 2024-04-12T20:56:46.455 INFO:tasks.workunit.client.1.smithi186.stdout:6/981: truncate d9/d5e/d7a/d146/f129 368550 0 2024-04-12T20:56:46.456 INFO:tasks.workunit.client.1.smithi186.stdout:3/941: dwrite d1/d3/d21/d22/d34/ddb/d4d/f51 [0,4194304] 0 2024-04-12T20:56:46.464 INFO:tasks.workunit.client.1.smithi186.stdout:9/920: dread d3/d2f/d54/d9f/dc4/d77/dd6/d72/df4/fcd [0,4194304] 0 2024-04-12T20:56:46.464 INFO:tasks.workunit.client.1.smithi186.stdout:9/921: fsync d3/d2f/d2e/d33/d61/f10d 0 2024-04-12T20:56:46.469 INFO:tasks.workunit.client.1.smithi186.stdout:5/936: write d5/db/d15/d2a/d4c/d93/fb2 [166667,43290] 0 2024-04-12T20:56:46.477 INFO:tasks.workunit.client.1.smithi186.stdout:5/937: creat d5/db/d15/de8/f12d x:0 0 0 2024-04-12T20:56:46.486 INFO:tasks.workunit.client.1.smithi186.stdout:7/958: dread d7/d10/d1c/f2c [0,4194304] 0 2024-04-12T20:56:46.487 INFO:tasks.workunit.client.1.smithi186.stdout:6/982: dwrite d9/d43/d50/f9e [0,4194304] 0 2024-04-12T20:56:46.487 INFO:tasks.workunit.client.1.smithi186.stdout:3/942: dwrite d1/d3/d21/d22/d34/ddb/fc5 [0,4194304] 0 2024-04-12T20:56:46.489 INFO:tasks.workunit.client.1.smithi186.stdout:5/938: truncate d5/db/d2e/d70/d101/dbb/dc4/d5d/de1/f125 571745 0 2024-04-12T20:56:46.496 INFO:tasks.workunit.client.1.smithi186.stdout:3/943: creat d1/f137 x:0 0 0 2024-04-12T20:56:46.498 INFO:tasks.workunit.client.1.smithi186.stdout:6/983: mkdir d9/d4c/de9/d133/d145/d158 0 2024-04-12T20:56:46.501 INFO:tasks.workunit.client.1.smithi186.stdout:9/922: truncate d3/d2f/d2e/d33/d36/f53 1550613 0 2024-04-12T20:56:46.501 INFO:tasks.workunit.client.1.smithi186.stdout:7/959: getdents d7/d10/d2d/d40/d4a/d6c/d97/d106 0 2024-04-12T20:56:46.504 INFO:tasks.workunit.client.1.smithi186.stdout:6/984: creat d9/d5e/d7a/d81/dc5/f159 x:0 0 0 2024-04-12T20:56:46.505 INFO:tasks.workunit.client.1.smithi186.stdout:7/960: symlink d7/d55/d71/dbe/l15e 0 2024-04-12T20:56:46.508 INFO:tasks.workunit.client.1.smithi186.stdout:6/985: truncate d9/f1d 2192225 0 2024-04-12T20:56:46.510 INFO:tasks.workunit.client.1.smithi186.stdout:6/986: chown d9/f1b 38 1 2024-04-12T20:56:46.510 INFO:tasks.workunit.client.1.smithi186.stdout:7/961: write d7/d10/d2d/d40/d4a/d6c/d97/fba [2340702,33147] 0 2024-04-12T20:56:46.514 INFO:tasks.workunit.client.1.smithi186.stdout:6/987: creat d9/d4c/de9/d133/f15a x:0 0 0 2024-04-12T20:56:46.519 INFO:tasks.workunit.client.1.smithi186.stdout:5/939: write d5/db/d2e/d70/d101/dbb/dc4/d5d/fc7 [432073,83400] 0 2024-04-12T20:56:46.520 INFO:tasks.workunit.client.1.smithi186.stdout:6/988: symlink d9/d5e/d7a/d146/d5d/d6e/l15b 0 2024-04-12T20:56:46.521 INFO:tasks.workunit.client.1.smithi186.stdout:3/944: truncate d1/fc 8469356 0 2024-04-12T20:56:46.523 INFO:tasks.workunit.client.1.smithi186.stdout:7/962: unlink d7/d10/d2d/d40/l54 0 2024-04-12T20:56:46.531 INFO:tasks.workunit.client.1.smithi186.stdout:9/923: dread d3/d2f/d2e/d43/d4c/f89 [0,4194304] 0 2024-04-12T20:56:46.533 INFO:tasks.workunit.client.1.smithi186.stdout:6/989: creat d9/d5e/d7a/d146/d5d/d6e/d91/dd9/d14e/f15c x:0 0 0 2024-04-12T20:56:46.538 INFO:tasks.workunit.client.1.smithi186.stdout:7/963: write d7/d10/d1c/d8c/fdc [636772,42633] 0 2024-04-12T20:56:46.538 INFO:tasks.workunit.client.1.smithi186.stdout:3/945: write d1/d3/df8/dd0/d10b/fcc [3792,55919] 0 2024-04-12T20:56:46.540 INFO:tasks.workunit.client.1.smithi186.stdout:7/964: dread - d7/d10/f125 zero size 2024-04-12T20:56:46.544 INFO:tasks.workunit.client.1.smithi186.stdout:7/965: symlink d7/d10/d2d/d67/d158/l15f 0 2024-04-12T20:56:46.544 INFO:tasks.workunit.client.1.smithi186.stdout:5/940: dwrite d5/db/d2e/d70/de2/fff [0,4194304] 0 2024-04-12T20:56:46.545 INFO:tasks.workunit.client.1.smithi186.stdout:3/946: mknod d1/d3/d21/d22/d24/c138 0 2024-04-12T20:56:46.548 INFO:tasks.workunit.client.1.smithi186.stdout:7/966: mkdir d7/d160 0 2024-04-12T20:56:46.548 INFO:tasks.workunit.client.1.smithi186.stdout:5/941: read - d5/db/d15/de8/f12d zero size 2024-04-12T20:56:46.552 INFO:tasks.workunit.client.1.smithi186.stdout:7/967: write d7/d10/d2d/fe5 [46279,39673] 0 2024-04-12T20:56:46.557 INFO:tasks.workunit.client.1.smithi186.stdout:5/942: creat d5/db/d15/d2a/d4c/d5b/d7a/d94/d106/d120/d86/dca/f12e x:0 0 0 2024-04-12T20:56:46.559 INFO:tasks.workunit.client.1.smithi186.stdout:7/968: unlink d7/d55/d71/dbe/l119 0 2024-04-12T20:56:46.563 INFO:tasks.workunit.client.1.smithi186.stdout:3/947: rename d1/d3/d21/d22/d34/ddb/d4d/dad/lc1 to d1/d3/d21/d22/d24/l139 0 2024-04-12T20:56:46.563 INFO:tasks.workunit.client.1.smithi186.stdout:5/943: creat d5/db/d71/d88/f12f x:0 0 0 2024-04-12T20:56:46.571 INFO:tasks.workunit.client.1.smithi186.stdout:5/944: rename d5/db/d15/d2a/d4c/c8e to d5/db/d2e/dd0/c130 0 2024-04-12T20:56:46.572 INFO:tasks.workunit.client.1.smithi186.stdout:3/948: creat d1/d3/d21/d22/d34/f13a x:0 0 0 2024-04-12T20:56:46.576 INFO:tasks.workunit.client.1.smithi186.stdout:9/924: dread d3/d2f/d54/d9f/dc4/d77/dd6/d72/df4/d6f/f8c [0,4194304] 0 2024-04-12T20:56:46.580 INFO:tasks.workunit.client.1.smithi186.stdout:3/949: rename d1/d3/da/l2b to d1/d3/df8/d95/d117/d11e/l13b 0 2024-04-12T20:56:46.580 INFO:tasks.workunit.client.1.smithi186.stdout:5/945: rename d5/db/d15/d2a/d4c/d5b/d7a/d94/d106 to d5/db/d15/d2a/d4c/d5b/d7a/d94/d106/d120/d86/d131 22 2024-04-12T20:56:46.595 INFO:tasks.workunit.client.1.smithi186.stdout:5/946: write d5/d27/f60 [786978,129945] 0 2024-04-12T20:56:46.595 INFO:tasks.workunit.client.1.smithi186.stdout:7/969: dwrite d7/d10/f15 [0,4194304] 0 2024-04-12T20:56:46.595 INFO:tasks.workunit.client.1.smithi186.stdout:9/925: dwrite d3/d2f/d2e/d33/d61/f90 [0,4194304] 0 2024-04-12T20:56:46.595 INFO:tasks.workunit.client.1.smithi186.stdout:3/950: write d1/f63 [2253420,123128] 0 2024-04-12T20:56:46.605 INFO:tasks.workunit.client.1.smithi186.stdout:5/947: creat d5/db/d33/d10e/d12b/f132 x:0 0 0 2024-04-12T20:56:46.611 INFO:tasks.workunit.client.1.smithi186.stdout:7/970: rename d7/d10/d2d/d40/d4a/d6c/d97/d4d/f6d to d7/d10/d2d/d40/d4a/d5c/f161 0 2024-04-12T20:56:46.612 INFO:tasks.workunit.client.1.smithi186.stdout:9/926: read d3/d2f/d54/d9f/dc4/d6b/ffa [176019,32410] 0 2024-04-12T20:56:46.624 INFO:tasks.workunit.client.1.smithi186.stdout:6/990: sync 2024-04-12T20:56:46.624 INFO:tasks.workunit.client.1.smithi186.stdout:5/948: dwrite d5/db/d15/f31 [0,4194304] 0 2024-04-12T20:56:46.628 INFO:tasks.workunit.client.1.smithi186.stdout:6/991: write d9/d4c/d6f/de3/f123 [355450,55407] 0 2024-04-12T20:56:46.635 INFO:tasks.workunit.client.1.smithi186.stdout:3/951: dread d1/f63 [0,4194304] 0 2024-04-12T20:56:46.638 INFO:tasks.workunit.client.1.smithi186.stdout:6/992: symlink d9/d5e/d7a/d146/d5d/d6e/d91/dd9/l15d 0 2024-04-12T20:56:46.642 INFO:tasks.workunit.client.1.smithi186.stdout:6/993: dread - d9/d43/d2d/f12c zero size 2024-04-12T20:56:46.642 INFO:tasks.workunit.client.1.smithi186.stdout:9/927: dwrite d3/d2f/d2e/d33/d36/d93/d123/f138 [0,4194304] 0 2024-04-12T20:56:46.650 INFO:tasks.workunit.client.1.smithi186.stdout:3/952: unlink d1/c118 0 2024-04-12T20:56:46.650 INFO:tasks.workunit.client.1.smithi186.stdout:6/994: write d9/d4c/d6f/de3/f123 [608879,127089] 0 2024-04-12T20:56:46.651 INFO:tasks.workunit.client.1.smithi186.stdout:7/971: dread d7/d55/d71/dbe/fc6 [0,4194304] 0 2024-04-12T20:56:46.651 INFO:tasks.workunit.client.1.smithi186.stdout:7/972: dread - d7/d55/d71/f14c zero size 2024-04-12T20:56:46.655 INFO:tasks.workunit.client.1.smithi186.stdout:6/995: dread d9/d5e/d7a/d146/fb8 [0,4194304] 0 2024-04-12T20:56:46.655 INFO:tasks.workunit.client.1.smithi186.stdout:5/949: dwrite d5/db/d15/f31 [4194304,4194304] 0 2024-04-12T20:56:46.655 INFO:tasks.workunit.client.1.smithi186.stdout:9/928: creat d3/d2f/d54/d9f/dc4/d77/f13a x:0 0 0 2024-04-12T20:56:46.659 INFO:tasks.workunit.client.1.smithi186.stdout:9/929: readlink d3/d2f/d2e/d43/d4c/l7b 0 2024-04-12T20:56:46.660 INFO:tasks.workunit.client.1.smithi186.stdout:7/973: mkdir d7/d127/d162 0 2024-04-12T20:56:46.661 INFO:tasks.workunit.client.1.smithi186.stdout:9/930: readlink d3/d2f/d2e/d33/d12d/l10f 0 2024-04-12T20:56:46.663 INFO:tasks.workunit.client.1.smithi186.stdout:7/974: dread - d7/d10/d2d/d40/d4a/d6c/d97/d109/f117 zero size 2024-04-12T20:56:46.664 INFO:tasks.workunit.client.1.smithi186.stdout:6/996: rmdir d9/d5e/d73/d134 39 2024-04-12T20:56:46.670 INFO:tasks.workunit.client.1.smithi186.stdout:9/931: truncate d3/d2f/d54/d9f/dc4/fc1 822865 0 2024-04-12T20:56:46.674 INFO:tasks.workunit.client.1.smithi186.stdout:6/997: unlink d9/d4c/dbe/cf6 0 2024-04-12T20:56:46.677 INFO:tasks.workunit.client.1.smithi186.stdout:5/950: mkdir d5/db/d2e/d70/d101/dbb/dc4/d133 0 2024-04-12T20:56:46.679 INFO:tasks.workunit.client.1.smithi186.stdout:9/932: mknod d3/d2f/d2e/d33/d61/dbc/c13b 0 2024-04-12T20:56:46.682 INFO:tasks.workunit.client.1.smithi186.stdout:7/975: getdents d7/d10/d2d/d40/d4a/d6c/d97/dc2/de2 0 2024-04-12T20:56:46.684 INFO:tasks.workunit.client.1.smithi186.stdout:6/998: dread d9/d43/f151 [0,4194304] 0 2024-04-12T20:56:46.687 INFO:tasks.workunit.client.1.smithi186.stdout:7/976: mkdir d7/d10/d2d/d40/d4a/d5c/d13b/d154/d163 0 2024-04-12T20:56:46.691 INFO:tasks.workunit.client.1.smithi186.stdout:9/933: dread d3/d2f/d2e/f2d [0,4194304] 0 2024-04-12T20:56:46.692 INFO:tasks.workunit.client.1.smithi186.stdout:3/953: dread d1/d3/df8/dd0/d10b/d7c/fac [0,4194304] 0 2024-04-12T20:56:46.692 INFO:tasks.workunit.client.1.smithi186.stdout:7/977: fsync d7/d10/d2d/d40/d4a/d6c/ddf/ffe 0 2024-04-12T20:56:46.695 INFO:tasks.workunit.client.1.smithi186.stdout:7/978: fsync d7/d10/d2d/d40/d4a/d5c/d148/f15d 0 2024-04-12T20:56:46.696 INFO:tasks.workunit.client.1.smithi186.stdout:7/979: dread - d7/ffc zero size 2024-04-12T20:56:46.702 INFO:tasks.workunit.client.1.smithi186.stdout:5/951: truncate d5/db/d15/fea 1673752 0 2024-04-12T20:56:46.704 INFO:tasks.workunit.client.1.smithi186.stdout:9/934: dwrite d3/d2f/f12c [0,4194304] 0 2024-04-12T20:56:46.715 INFO:tasks.workunit.client.1.smithi186.stdout:6/999: write d9/d43/d50/fad [990293,74753] 0 2024-04-12T20:56:46.727 INFO:tasks.workunit.client.1.smithi186.stdout:7/980: dwrite d7/d10/d1c/f7e [0,4194304] 0 2024-04-12T20:56:46.738 INFO:tasks.workunit.client.1.smithi186.stdout:9/935: dread d3/d2f/d54/d9f/dc4/d77/dd6/d72/df4/df9/f11a [0,4194304] 0 2024-04-12T20:56:46.772 INFO:tasks.workunit.client.1.smithi186.stdout:3/954: getdents d1/d3/d21/d22/d34/ddb/d4d/dad 0 2024-04-12T20:56:46.785 INFO:tasks.workunit.client.1.smithi186.stdout:9/936: creat d3/d2f/d2e/d33/d9d/f13c x:0 0 0 2024-04-12T20:56:46.786 INFO:tasks.workunit.client.1.smithi186.stdout:9/937: chown d3/f73 184222830 1 2024-04-12T20:56:46.796 INFO:tasks.workunit.client.1.smithi186.stdout:5/952: truncate d5/db/d15/d2a/d4c/d5b/d7a/d94/dfa/fb5 1922564 0 2024-04-12T20:56:46.802 INFO:tasks.workunit.client.1.smithi186.stdout:9/938: dread d3/d2f/f12c [0,4194304] 0 2024-04-12T20:56:46.804 INFO:tasks.workunit.client.1.smithi186.stdout:7/981: dread d7/d10/d2d/d40/d4a/d6c/d97/f78 [0,4194304] 0 2024-04-12T20:56:46.812 INFO:tasks.workunit.client.1.smithi186.stdout:5/953: getdents d5/db/d15/d2a/d4c/d5b/d7a/d94/dfa 0 2024-04-12T20:56:46.816 INFO:tasks.workunit.client.1.smithi186.stdout:3/955: write d1/d3/d21/d22/d34/ddb/d4d/d6f/ff5 [262819,89574] 0 2024-04-12T20:56:46.823 INFO:tasks.workunit.client.1.smithi186.stdout:3/956: creat d1/d3/d21/d22/d34/d6e/f13c x:0 0 0 2024-04-12T20:56:46.828 INFO:tasks.workunit.client.1.smithi186.stdout:3/957: fsync d1/d3/d21/d22/f79 0 2024-04-12T20:56:46.832 INFO:tasks.workunit.client.1.smithi186.stdout:9/939: dwrite d3/f10 [8388608,4194304] 0 2024-04-12T20:56:46.832 INFO:tasks.workunit.client.1.smithi186.stdout:7/982: dread d7/d10/d2d/d67/f70 [0,4194304] 0 2024-04-12T20:56:46.836 INFO:tasks.workunit.client.1.smithi186.stdout:5/954: dwrite d5/db/d15/d2a/d2d/d77/fdc [0,4194304] 0 2024-04-12T20:56:46.837 INFO:tasks.workunit.client.1.smithi186.stdout:7/983: write d7/d10/d2d/d40/d4a/d6c/d97/d109/f12e [404764,65662] 0 2024-04-12T20:56:46.849 INFO:tasks.workunit.client.1.smithi186.stdout:9/940: rmdir d3/d2f/d54/d9f/dc4/d77/d11f 0 2024-04-12T20:56:46.850 INFO:tasks.workunit.client.1.smithi186.stdout:7/984: symlink d7/d10/d2d/d40/df7/l164 0 2024-04-12T20:56:46.850 INFO:tasks.workunit.client.1.smithi186.stdout:3/958: getdents d1/d3/d21/d22/d34/ddb/d4d/da1 0 2024-04-12T20:56:46.853 INFO:tasks.workunit.client.1.smithi186.stdout:9/941: creat d3/d2f/d2e/d43/d4c/d82/f13d x:0 0 0 2024-04-12T20:56:46.854 INFO:tasks.workunit.client.1.smithi186.stdout:3/959: creat d1/d3/d21/d22/d4a/f13d x:0 0 0 2024-04-12T20:56:46.861 INFO:tasks.workunit.client.1.smithi186.stdout:5/955: dwrite d5/db/d15/d2a/d4c/d5b/d7a/d94/d106/d120/d86/dca/ffd [0,4194304] 0 2024-04-12T20:56:46.863 INFO:tasks.workunit.client.1.smithi186.stdout:9/942: link d3/d2f/f1e d3/d2f/d2e/d33/d9d/f13e 0 2024-04-12T20:56:46.865 INFO:tasks.workunit.client.1.smithi186.stdout:3/960: getdents d1/d3/da 0 2024-04-12T20:56:46.872 INFO:tasks.workunit.client.1.smithi186.stdout:5/956: creat d5/db/d2e/d70/d101/dbb/dc4/d133/f134 x:0 0 0 2024-04-12T20:56:46.874 INFO:tasks.workunit.client.1.smithi186.stdout:5/957: stat d5/db/d15/d2a/d4c/d114 0 2024-04-12T20:56:46.876 INFO:tasks.workunit.client.1.smithi186.stdout:7/985: write d7/d55/d71/f130 [319695,28405] 0 2024-04-12T20:56:46.880 INFO:tasks.workunit.client.1.smithi186.stdout:3/961: write d1/d3/d21/d22/f7b [29588,100342] 0 2024-04-12T20:56:46.884 INFO:tasks.workunit.client.1.smithi186.stdout:7/986: truncate d7/d10/d1c/f21 238005 0 2024-04-12T20:56:46.886 INFO:tasks.workunit.client.1.smithi186.stdout:7/987: readlink d7/d10/d1c/d5d/db7/d15b/ld6 0 2024-04-12T20:56:46.888 INFO:tasks.workunit.client.1.smithi186.stdout:3/962: mkdir d1/d3/df8/dd0/de4/d119/d13e 0 2024-04-12T20:56:46.889 INFO:tasks.workunit.client.1.smithi186.stdout:5/958: dread d5/fd [0,4194304] 0 2024-04-12T20:56:46.891 INFO:tasks.workunit.client.1.smithi186.stdout:7/988: unlink d7/d55/f10e 0 2024-04-12T20:56:46.893 INFO:tasks.workunit.client.1.smithi186.stdout:3/963: write d1/d3/df8/dd0/d10b/fd4 [956818,96590] 0 2024-04-12T20:56:46.898 INFO:tasks.workunit.client.1.smithi186.stdout:9/943: dread d3/d13/f12e [0,4194304] 0 2024-04-12T20:56:46.898 INFO:tasks.workunit.client.1.smithi186.stdout:3/964: creat d1/d3/d21/d22/d4a/f13f x:0 0 0 2024-04-12T20:56:46.899 INFO:tasks.workunit.client.1.smithi186.stdout:7/989: creat d7/d10/d2d/d40/d4a/d6c/d97/dc2/de2/f165 x:0 0 0 2024-04-12T20:56:46.900 INFO:tasks.workunit.client.1.smithi186.stdout:5/959: mknod d5/db/d15/d2a/d4c/d5b/d7a/d94/dfa/da4/c135 0 2024-04-12T20:56:46.901 INFO:tasks.workunit.client.1.smithi186.stdout:3/965: dread - d1/d3/d21/d22/d24/d28/fee zero size 2024-04-12T20:56:46.903 INFO:tasks.workunit.client.1.smithi186.stdout:7/990: fdatasync d7/d10/d2d/d40/d4a/d6c/ddf/ffe 0 2024-04-12T20:56:46.903 INFO:tasks.workunit.client.1.smithi186.stdout:5/960: chown d5/db/d15/d2a/d2d/d82/d84/cc8 4410406 1 2024-04-12T20:56:46.906 INFO:tasks.workunit.client.1.smithi186.stdout:3/966: dread - d1/d3/d21/d22/d24/d28/fe3 zero size 2024-04-12T20:56:46.909 INFO:tasks.workunit.client.1.smithi186.stdout:5/961: creat d5/db/d2e/d70/f136 x:0 0 0 2024-04-12T20:56:46.918 INFO:tasks.workunit.client.1.smithi186.stdout:3/967: creat d1/d3/d21/f140 x:0 0 0 2024-04-12T20:56:46.918 INFO:tasks.workunit.client.1.smithi186.stdout:7/991: dwrite d7/d10/d2d/d40/d4a/d6c/d97/f92 [4194304,4194304] 0 2024-04-12T20:56:46.921 INFO:tasks.workunit.client.1.smithi186.stdout:5/962: symlink d5/db/d2e/dd0/l137 0 2024-04-12T20:56:46.926 INFO:tasks.workunit.client.1.smithi186.stdout:3/968: rename d1/d3/df8/dd0/de4/d119 to d1/d3/df8/dd0/d10b/d12f/d141 0 2024-04-12T20:56:46.929 INFO:tasks.workunit.client.1.smithi186.stdout:7/992: rmdir d7/d10/d2d/d40/d4a/d6c/d97/d109/d124 39 2024-04-12T20:56:46.936 INFO:tasks.workunit.client.1.smithi186.stdout:5/963: mkdir d5/db/d2e/d70/d101/dbb/dc4/d5d/d115/d138 0 2024-04-12T20:56:46.940 INFO:tasks.workunit.client.1.smithi186.stdout:5/964: mkdir d5/db/d15/d2a/d2d/d82/d139 0 2024-04-12T20:56:46.943 INFO:tasks.workunit.client.1.smithi186.stdout:7/993: dwrite d7/d10/d2d/d40/d4a/d6c/d97/f123 [0,4194304] 0 2024-04-12T20:56:46.950 INFO:tasks.workunit.client.1.smithi186.stdout:7/994: symlink d7/d55/d71/d11b/l166 0 2024-04-12T20:56:46.951 INFO:tasks.workunit.client.1.smithi186.stdout:7/995: dread - d7/d10/f125 zero size 2024-04-12T20:56:46.953 INFO:tasks.workunit.client.1.smithi186.stdout:7/996: truncate d7/d55/d71/f130 1171926 0 2024-04-12T20:56:46.958 INFO:tasks.workunit.client.1.smithi186.stdout:7/997: mknod d7/d10/d2d/d40/df7/c167 0 2024-04-12T20:56:46.960 INFO:tasks.workunit.client.1.smithi186.stdout:7/998: read d7/d55/ffd [325161,120206] 0 2024-04-12T20:56:46.964 INFO:tasks.workunit.client.1.smithi186.stdout:7/999: creat d7/d10/d2d/d67/f168 x:0 0 0 2024-04-12T20:56:47.106 INFO:tasks.workunit.client.1.smithi186.stdout:3/969: sync 2024-04-12T20:56:47.106 INFO:tasks.workunit.client.1.smithi186.stdout:9/944: sync 2024-04-12T20:56:47.106 INFO:tasks.workunit.client.1.smithi186.stdout:5/965: sync 2024-04-12T20:56:47.109 INFO:tasks.workunit.client.1.smithi186.stdout:3/970: stat d1/d3/d21/d22/d4a/cc4 0 2024-04-12T20:56:47.114 INFO:tasks.workunit.client.1.smithi186.stdout:9/945: truncate d3/d2f/d54/d9f/dc4/f5f 1843856 0 2024-04-12T20:56:47.119 INFO:tasks.workunit.client.1.smithi186.stdout:3/971: readlink d1/d3/df8/dd0/d10b/d7c/dbb/d120/l74 0 2024-04-12T20:56:47.119 INFO:tasks.workunit.client.1.smithi186.stdout:5/966: dread d5/db/d15/d2a/d4c/d93/fcf [0,4194304] 0 2024-04-12T20:56:47.125 INFO:tasks.workunit.client.1.smithi186.stdout:5/967: write d5/db/d2e/d70/de2/f108 [518332,5935] 0 2024-04-12T20:56:47.140 INFO:tasks.workunit.client.1.smithi186.stdout:3/972: rename d1/d3/c31 to d1/d3/d21/d22/d34/ddb/d4d/da1/c142 0 2024-04-12T20:56:47.145 INFO:tasks.workunit.client.1.smithi186.stdout:3/973: write d1/d3/df8/dd0/d10b/d7c/dbb/d120/d48/fab [1544581,64169] 0 2024-04-12T20:56:47.152 INFO:tasks.workunit.client.1.smithi186.stdout:3/974: truncate d1/d3/d21/d22/d34/ddb/d4d/d9c/fcb 88453 0 2024-04-12T20:56:47.154 INFO:tasks.workunit.client.1.smithi186.stdout:3/975: fdatasync d1/d3/df8/dd0/d10b/fd4 0 2024-04-12T20:56:47.163 INFO:tasks.workunit.client.1.smithi186.stdout:5/968: write d5/db/d15/d2a/d4c/d5b/d7a/d94/dfa/f7c [598613,75379] 0 2024-04-12T20:56:47.166 INFO:tasks.workunit.client.1.smithi186.stdout:9/946: dwrite d3/d2f/d2e/db3/fd1 [0,4194304] 0 2024-04-12T20:56:47.167 INFO:tasks.workunit.client.1.smithi186.stdout:9/947: readlink d3/l15 0 2024-04-12T20:56:47.171 INFO:tasks.workunit.client.1.smithi186.stdout:5/969: mkdir d5/db/d15/d2a/d2d/d13a 0 2024-04-12T20:56:47.173 INFO:tasks.workunit.client.1.smithi186.stdout:9/948: truncate d3/d13/f12e 803732 0 2024-04-12T20:56:47.174 INFO:tasks.workunit.client.1.smithi186.stdout:5/970: fsync d5/db/d15/d2a/d4c/d93/fb2 0 2024-04-12T20:56:47.178 INFO:tasks.workunit.client.1.smithi186.stdout:5/971: mkdir d5/db/d33/d10e/d12b/d13b 0 2024-04-12T20:56:47.184 INFO:tasks.workunit.client.1.smithi186.stdout:5/972: write d5/db/d15/d2a/d4c/d5b/d7a/d94/d106/d120/d86/dca/f12e [503043,120286] 0 2024-04-12T20:56:47.186 INFO:tasks.workunit.client.1.smithi186.stdout:5/973: fdatasync d5/db/f11 0 2024-04-12T20:56:47.190 INFO:tasks.workunit.client.1.smithi186.stdout:9/949: dwrite d3/d2f/d2e/d33/fdd [0,4194304] 0 2024-04-12T20:56:47.209 INFO:tasks.workunit.client.1.smithi186.stdout:5/974: dwrite d5/fd [0,4194304] 0 2024-04-12T20:56:47.223 INFO:tasks.workunit.client.1.smithi186.stdout:9/950: dwrite d3/d2f/d2e/d33/d36/f53 [0,4194304] 0 2024-04-12T20:56:47.226 INFO:tasks.workunit.client.1.smithi186.stdout:9/951: fsync d3/d13/d10b/ff5 0 2024-04-12T20:56:47.234 INFO:tasks.workunit.client.1.smithi186.stdout:9/952: rename d3/d2f/d2e/d33/d36/d70/f99 to d3/d2f/d2e/d33/d61/dbc/f13f 0 2024-04-12T20:56:47.238 INFO:tasks.workunit.client.1.smithi186.stdout:9/953: dread - d3/d2f/d54/d9f/dc4/d77/dd6/d72/df4/fa0 zero size 2024-04-12T20:56:47.243 INFO:tasks.workunit.client.1.smithi186.stdout:9/954: unlink d3/d2f/d2e/d33/f49 0 2024-04-12T20:56:47.247 INFO:tasks.workunit.client.1.smithi186.stdout:9/955: symlink d3/d2f/d2e/d33/d36/de5/l140 0 2024-04-12T20:56:47.249 INFO:tasks.workunit.client.1.smithi186.stdout:9/956: fsync d3/d2f/d2e/d33/d36/d93/d123/f138 0 2024-04-12T20:56:47.258 INFO:tasks.workunit.client.1.smithi186.stdout:9/957: write d3/d2f/d2e/d33/d36/f37 [3583162,103078] 0 2024-04-12T20:56:47.265 INFO:tasks.workunit.client.1.smithi186.stdout:9/958: symlink d3/d2f/d54/d9f/dc4/d77/dd6/d72/df4/d6f/l141 0 2024-04-12T20:56:47.267 INFO:tasks.workunit.client.1.smithi186.stdout:9/959: stat d3/d2f/d103/led 0 2024-04-12T20:56:47.268 INFO:tasks.workunit.client.1.smithi186.stdout:9/960: readlink d3/d2f/d103/l29 0 2024-04-12T20:56:47.272 INFO:tasks.workunit.client.1.smithi186.stdout:9/961: creat d3/d2f/d54/d92/f142 x:0 0 0 2024-04-12T20:56:47.277 INFO:tasks.workunit.client.1.smithi186.stdout:9/962: creat d3/d2f/d2e/db3/f143 x:0 0 0 2024-04-12T20:56:47.282 INFO:tasks.workunit.client.1.smithi186.stdout:9/963: creat d3/d2f/d2e/d33/d9d/f144 x:0 0 0 2024-04-12T20:56:47.295 INFO:tasks.workunit.client.1.smithi186.stdout:9/964: rmdir d3/d2f/d54/d7d/d112/d132 0 2024-04-12T20:56:47.319 INFO:tasks.workunit.client.1.smithi186.stdout:5/975: sync 2024-04-12T20:56:47.319 INFO:tasks.workunit.client.1.smithi186.stdout:3/976: sync 2024-04-12T20:56:47.327 INFO:tasks.workunit.client.1.smithi186.stdout:9/965: dread d3/d2f/d2e/d43/d4c/fdc [0,4194304] 0 2024-04-12T20:56:47.330 INFO:tasks.workunit.client.1.smithi186.stdout:9/966: stat d3/d2f/d103/l29 0 2024-04-12T20:56:47.331 INFO:tasks.workunit.client.1.smithi186.stdout:3/977: dread d1/d3/d21/d22/f7b [0,4194304] 0 2024-04-12T20:56:47.334 INFO:tasks.workunit.client.1.smithi186.stdout:9/967: chown d3/d2f/d2e/ca1 50 1 2024-04-12T20:56:47.335 INFO:tasks.workunit.client.1.smithi186.stdout:3/978: truncate d1/fc 8581695 0 2024-04-12T20:56:47.341 INFO:tasks.workunit.client.1.smithi186.stdout:3/979: mkdir d1/d3/df8/ddc/d143 0 2024-04-12T20:56:47.347 INFO:tasks.workunit.client.1.smithi186.stdout:3/980: mknod d1/d3/d21/c144 0 2024-04-12T20:56:47.349 INFO:tasks.workunit.client.1.smithi186.stdout:5/976: dwrite d5/db/d15/f122 [0,4194304] 0 2024-04-12T20:56:47.355 INFO:tasks.workunit.client.1.smithi186.stdout:3/981: read d1/d3/d21/d22/d34/ddb/d4d/d9c/fa7 [753899,46754] 0 2024-04-12T20:56:47.359 INFO:tasks.workunit.client.1.smithi186.stdout:3/982: chown d1/d3/d21/d22/d24/cd8 664691 1 2024-04-12T20:56:47.370 INFO:tasks.workunit.client.1.smithi186.stdout:3/983: mknod d1/d3/d21/d22/d34/ddb/d4d/da1/d124/c145 0 2024-04-12T20:56:47.371 INFO:tasks.workunit.client.1.smithi186.stdout:5/977: dwrite d5/db/d15/d2a/d2d/d77/fdc [0,4194304] 0 2024-04-12T20:56:47.372 INFO:tasks.workunit.client.1.smithi186.stdout:9/968: dread d3/d2f/f1e [0,4194304] 0 2024-04-12T20:56:47.377 INFO:tasks.workunit.client.1.smithi186.stdout:9/969: truncate d3/f3e 5097430 0 2024-04-12T20:56:47.380 INFO:tasks.workunit.client.1.smithi186.stdout:9/970: read d3/d2f/f11d [4197769,125191] 0 2024-04-12T20:56:47.385 INFO:tasks.workunit.client.1.smithi186.stdout:9/971: read d3/d2f/d2e/d33/d61/dbc/f13f [110907,128445] 0 2024-04-12T20:56:47.387 INFO:tasks.workunit.client.1.smithi186.stdout:9/972: write d3/d2f/f5e [585529,107063] 0 2024-04-12T20:56:47.388 INFO:tasks.workunit.client.1.smithi186.stdout:9/973: read - d3/d2f/d2e/d43/fbd zero size 2024-04-12T20:56:47.392 INFO:tasks.workunit.client.1.smithi186.stdout:5/978: write d5/db/d15/d2a/d2d/d46/d55/f37 [534478,79621] 0 2024-04-12T20:56:47.393 INFO:tasks.workunit.client.1.smithi186.stdout:3/984: link d1/d3/d21/d22/d4a/c135 d1/d3/d21/d22/da4/c146 0 2024-04-12T20:56:47.415 INFO:tasks.workunit.client.1.smithi186.stdout:9/974: creat d3/d2f/d2e/d33/d36/d70/f145 x:0 0 0 2024-04-12T20:56:47.417 INFO:tasks.workunit.client.1.smithi186.stdout:5/979: symlink d5/db/d15/d2a/d4c/d5b/d7a/l13c 0 2024-04-12T20:56:47.418 INFO:tasks.workunit.client.1.smithi186.stdout:9/975: chown d3/d2f/d2e/d33/d61/dbc/f13f 13026 1 2024-04-12T20:56:47.422 INFO:tasks.workunit.client.1.smithi186.stdout:5/980: stat d5/db/d2e/d70/d101/dbb 0 2024-04-12T20:56:47.423 INFO:tasks.workunit.client.1.smithi186.stdout:9/976: write d3/d13/d10b/ff5 [4838820,77986] 0 2024-04-12T20:56:47.426 INFO:tasks.workunit.client.1.smithi186.stdout:5/981: creat d5/db/d33/d10e/d12b/f13d x:0 0 0 2024-04-12T20:56:47.428 INFO:tasks.workunit.client.1.smithi186.stdout:9/977: symlink d3/d2f/d2e/l146 0 2024-04-12T20:56:47.429 INFO:tasks.workunit.client.1.smithi186.stdout:3/985: truncate d1/d3/d21/d22/d34/ddb/d4d/d6f/f83 92081 0 2024-04-12T20:56:47.430 INFO:tasks.workunit.client.1.smithi186.stdout:5/982: truncate d5/db/d33/d10e/d12b/f132 678228 0 2024-04-12T20:56:47.434 INFO:tasks.workunit.client.1.smithi186.stdout:3/986: mknod d1/d3/df8/dd0/d10b/d7c/dbb/d120/d64/c147 0 2024-04-12T20:56:47.438 INFO:tasks.workunit.client.1.smithi186.stdout:5/983: rename d5/db/d15/d2a/d4c/d5b/c68 to d5/db/d2e/d70/d101/dbb/dc4/d5d/de1/c13e 0 2024-04-12T20:56:47.440 INFO:tasks.workunit.client.1.smithi186.stdout:3/987: mknod d1/d3/df8/dd0/d10b/d12f/c148 0 2024-04-12T20:56:47.445 INFO:tasks.workunit.client.1.smithi186.stdout:5/984: getdents d5/db/d71 0 2024-04-12T20:56:47.445 INFO:tasks.workunit.client.1.smithi186.stdout:3/988: rmdir d1/d3/d21/d22/d34/ddb/d4d/da1 39 2024-04-12T20:56:47.448 INFO:tasks.workunit.client.1.smithi186.stdout:3/989: chown d1/d3/df8/f92 5342745 1 2024-04-12T20:56:47.451 INFO:tasks.workunit.client.1.smithi186.stdout:9/978: dwrite d3/d2f/d54/d9f/dc4/d77/dd6/d72/df4/fcd [0,4194304] 0 2024-04-12T20:56:47.453 INFO:tasks.workunit.client.1.smithi186.stdout:3/990: unlink d1/ld5 0 2024-04-12T20:56:47.466 INFO:tasks.workunit.client.1.smithi186.stdout:5/985: dwrite d5/db/d71/d88/f12f [0,4194304] 0 2024-04-12T20:56:47.469 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:56:47 smithi018 ceph-mon[28799]: pgmap v56: 65 pgs: 65 active+clean; 3.1 GiB data, 11 GiB used, 525 GiB / 536 GiB avail; 77 MiB/s rd, 182 MiB/s wr, 450 op/s 2024-04-12T20:56:47.469 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:56:47 smithi018 ceph-mon[28799]: from='client.24453 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T20:56:47.476 INFO:tasks.workunit.client.1.smithi186.stdout:3/991: write d1/d3/d21/d22/d24/faf [279540,58967] 0 2024-04-12T20:56:47.476 INFO:tasks.workunit.client.1.smithi186.stdout:9/979: dwrite d3/d2f/d2e/db3/fd1 [0,4194304] 0 2024-04-12T20:56:47.481 INFO:tasks.workunit.client.1.smithi186.stdout:9/980: read d3/d2f/d2e/f41 [845753,102000] 0 2024-04-12T20:56:47.576 INFO:tasks.workunit.client.1.smithi186.stdout:3/992: symlink d1/d3/df8/dd0/d10b/d7c/dbb/d120/d48/ded/l149 0 2024-04-12T20:56:47.583 INFO:tasks.workunit.client.1.smithi186.stdout:5/986: write d5/db/d15/d2a/d4c/d5b/d7a/d94/dfa/fb5 [2277738,56388] 0 2024-04-12T20:56:47.587 INFO:tasks.workunit.client.1.smithi186.stdout:9/981: write d3/d2f/d2e/d43/d4c/f89 [2429286,130935] 0 2024-04-12T20:56:47.595 INFO:tasks.workunit.client.1.smithi186.stdout:9/982: write d3/fd [2227500,113164] 0 2024-04-12T20:56:47.625 INFO:tasks.workunit.client.1.smithi186.stdout:3/993: dwrite d1/d3/df8/dd0/d10b/d7c/dbb/d120/d48/f5f [4194304,4194304] 0 2024-04-12T20:56:47.636 INFO:tasks.workunit.client.1.smithi186.stdout:9/983: rename d3/d2f/d54/d9f/dc4/d77/dd6/d72/df4/f7f to d3/d2f/d54/d9f/dc4/d77/dd6/d72/df4/f147 0 2024-04-12T20:56:47.643 INFO:tasks.workunit.client.1.smithi186.stdout:9/984: fsync d3/d2f/d2e/fd8 0 2024-04-12T20:56:47.652 INFO:tasks.workunit.client.1.smithi186.stdout:9/985: truncate d3/d2f/d2e/d43/fbd 507077 0 2024-04-12T20:56:47.655 INFO:tasks.workunit.client.1.smithi186.stdout:9/986: chown d3/d2f/d2e/d43 59088 1 2024-04-12T20:56:47.656 INFO:tasks.workunit.client.1.smithi186.stdout:3/994: dread d1/d3/df8/ddc/ffc [0,4194304] 0 2024-04-12T20:56:47.661 INFO:tasks.workunit.client.1.smithi186.stdout:9/987: fdatasync d3/d2f/d54/d9f/dc4/d77/dd6/d72/fe1 0 2024-04-12T20:56:47.662 INFO:tasks.workunit.client.1.smithi186.stdout:3/995: rmdir d1/d3/d21/d22/d34/ddb/d4d 39 2024-04-12T20:56:47.663 INFO:tasks.workunit.client.1.smithi186.stdout:9/988: readlink d3/d2f/l2b 0 2024-04-12T20:56:47.670 INFO:tasks.workunit.client.1.smithi186.stdout:3/996: fdatasync d1/d3/d21/d22/d4a/f13d 0 2024-04-12T20:56:47.672 INFO:tasks.workunit.client.1.smithi186.stdout:9/989: symlink d3/d2f/d54/d9f/dc4/d6b/d136/l148 0 2024-04-12T20:56:47.676 INFO:tasks.workunit.client.1.smithi186.stdout:3/997: creat d1/d3/d21/d22/d34/f14a x:0 0 0 2024-04-12T20:56:47.680 INFO:tasks.workunit.client.1.smithi186.stdout:3/998: creat d1/d3/da/f14b x:0 0 0 2024-04-12T20:56:47.683 INFO:tasks.workunit.client.1.smithi186.stdout:3/999: stat d1/d3/df8/dd0/d10b/fcc 0 2024-04-12T20:56:47.684 INFO:tasks.workunit.client.1.smithi186.stdout:9/990: write d3/d2f/d54/d9f/dc4/ff2 [450810,39399] 0 2024-04-12T20:56:47.694 INFO:tasks.workunit.client.1.smithi186.stdout:9/991: unlink d3/d2f/d2e/d43/d4c/fdc 0 2024-04-12T20:56:47.700 INFO:tasks.workunit.client.1.smithi186.stdout:9/992: write d3/d2f/d2e/d43/d4c/f6e [890938,22579] 0 2024-04-12T20:56:47.780 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:56:47 smithi186 ceph-mon[36597]: pgmap v56: 65 pgs: 65 active+clean; 3.1 GiB data, 11 GiB used, 525 GiB / 536 GiB avail; 77 MiB/s rd, 182 MiB/s wr, 450 op/s 2024-04-12T20:56:47.780 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:56:47 smithi186 ceph-mon[36597]: from='client.24453 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T20:56:47.899 INFO:tasks.workunit.client.1.smithi186.stdout:5/987: dread - d5/d27/f91 zero size 2024-04-12T20:56:47.903 INFO:tasks.workunit.client.1.smithi186.stdout:9/993: write d3/f3e [4618338,26124] 0 2024-04-12T20:56:47.907 INFO:tasks.workunit.client.1.smithi186.stdout:5/988: dread d5/db/d2e/d70/d101/dbb/dc4/f11e [0,4194304] 0 2024-04-12T20:56:47.910 INFO:tasks.workunit.client.1.smithi186.stdout:5/989: mknod d5/db/d15/c13f 0 2024-04-12T20:56:47.926 INFO:tasks.workunit.client.1.smithi186.stdout:5/990: dwrite d5/db/f5f [4194304,4194304] 0 2024-04-12T20:56:47.951 INFO:tasks.workunit.client.1.smithi186.stdout:5/991: dread d5/db/d71/f75 [0,4194304] 0 2024-04-12T20:56:48.001 INFO:tasks.workunit.client.1.smithi186.stdout:5/992: write d5/db/d15/f28 [3767556,119225] 0 2024-04-12T20:56:48.034 INFO:tasks.workunit.client.1.smithi186.stdout:5/993: mkdir d5/db/d2e/d140 0 2024-04-12T20:56:48.041 INFO:tasks.workunit.client.1.smithi186.stdout:5/994: creat d5/db/d71/d88/d11d/f141 x:0 0 0 2024-04-12T20:56:48.047 INFO:tasks.workunit.client.1.smithi186.stdout:5/995: mknod d5/db/d2e/d70/d101/dbb/c142 0 2024-04-12T20:56:48.050 INFO:tasks.workunit.client.1.smithi186.stdout:9/994: creat d3/d2f/d54/d9f/dc4/d77/dc3/f149 x:0 0 0 2024-04-12T20:56:48.058 INFO:tasks.workunit.client.1.smithi186.stdout:9/995: rename d3/fd to d3/d2f/d2e/d33/d61/f14a 0 2024-04-12T20:56:48.067 INFO:tasks.workunit.client.1.smithi186.stdout:5/996: write d5/db/d15/d2a/d4c/f7b [5053344,48342] 0 2024-04-12T20:56:48.083 INFO:tasks.workunit.client.1.smithi186.stdout:9/996: write d3/f3e [4157213,60021] 0 2024-04-12T20:56:48.104 INFO:tasks.workunit.client.1.smithi186.stdout:5/997: dwrite d5/db/d15/d2a/d2d/d77/fac [0,4194304] 0 2024-04-12T20:56:48.114 INFO:tasks.workunit.client.1.smithi186.stdout:9/997: dread d3/d2f/d2e/d33/d36/d93/d123/f138 [0,4194304] 0 2024-04-12T20:56:48.118 INFO:tasks.workunit.client.1.smithi186.stdout:9/998: dread - d3/d2f/d54/d9f/dc4/d77/f13a zero size 2024-04-12T20:56:48.121 INFO:tasks.workunit.client.1.smithi186.stdout:5/998: symlink d5/db/d15/d2a/d4c/d5b/d7a/d94/dfa/da4/l143 0 2024-04-12T20:56:48.127 INFO:tasks.workunit.client.1.smithi186.stdout:5/999: write d5/db/d2e/f64 [297307,40975] 0 2024-04-12T20:56:48.127 INFO:tasks.workunit.client.1.smithi186.stdout:9/999: write d3/d13/fb7 [777087,511] 0 2024-04-12T20:56:48.150 INFO:tasks.workunit.client.1.smithi186.stderr:+ rm -rf -- ./tmp.gUD0e7KDu0 2024-04-12T20:56:49.351 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:56:49 smithi018 ceph-mon[28799]: pgmap v57: 65 pgs: 65 active+clean; 3.2 GiB data, 11 GiB used, 525 GiB / 536 GiB avail; 73 MiB/s rd, 168 MiB/s wr, 412 op/s 2024-04-12T20:56:49.530 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:56:49 smithi186 ceph-mon[36597]: pgmap v57: 65 pgs: 65 active+clean; 3.2 GiB data, 11 GiB used, 525 GiB / 536 GiB avail; 73 MiB/s rd, 168 MiB/s wr, 412 op/s 2024-04-12T20:56:51.589 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:56:51 smithi018 ceph-mon[28799]: pgmap v58: 65 pgs: 65 active+clean; 3.2 GiB data, 11 GiB used, 525 GiB / 536 GiB avail; 59 MiB/s rd, 136 MiB/s wr, 347 op/s 2024-04-12T20:56:51.589 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:56:51 smithi018 ceph-mon[28799]: from='mgr.24359 172.21.15.186:0/1315906828' entity='mgr.smithi186.kqcfxk' 2024-04-12T20:56:51.589 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:56:51 smithi018 ceph-mon[28799]: from='mgr.24359 172.21.15.186:0/1315906828' entity='mgr.smithi186.kqcfxk' 2024-04-12T20:56:51.780 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:56:51 smithi186 ceph-mon[36597]: pgmap v58: 65 pgs: 65 active+clean; 3.2 GiB data, 11 GiB used, 525 GiB / 536 GiB avail; 59 MiB/s rd, 136 MiB/s wr, 347 op/s 2024-04-12T20:56:51.780 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:56:51 smithi186 ceph-mon[36597]: from='mgr.24359 172.21.15.186:0/1315906828' entity='mgr.smithi186.kqcfxk' 2024-04-12T20:56:51.780 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:56:51 smithi186 ceph-mon[36597]: from='mgr.24359 172.21.15.186:0/1315906828' entity='mgr.smithi186.kqcfxk' 2024-04-12T20:56:53.351 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:56:53 smithi018 ceph-mon[28799]: pgmap v59: 65 pgs: 65 active+clean; 2.9 GiB data, 10 GiB used, 526 GiB / 536 GiB avail; 68 MiB/s rd, 157 MiB/s wr, 435 op/s 2024-04-12T20:56:53.780 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:56:53 smithi186 ceph-mon[36597]: pgmap v59: 65 pgs: 65 active+clean; 2.9 GiB data, 10 GiB used, 526 GiB / 536 GiB avail; 68 MiB/s rd, 157 MiB/s wr, 435 op/s 2024-04-12T20:56:55.601 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:56:55 smithi018 ceph-mon[28799]: pgmap v60: 65 pgs: 65 active+clean; 2.9 GiB data, 10 GiB used, 526 GiB / 536 GiB avail; 38 MiB/s rd, 86 MiB/s wr, 258 op/s 2024-04-12T20:56:55.780 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:56:55 smithi186 ceph-mon[36597]: pgmap v60: 65 pgs: 65 active+clean; 2.9 GiB data, 10 GiB used, 526 GiB / 536 GiB avail; 38 MiB/s rd, 86 MiB/s wr, 258 op/s 2024-04-12T20:56:57.097 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-04-12T20:56:57.097 DEBUG:teuthology.orchestra.run.smithi018:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-04-12T20:56:57.601 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:56:57 smithi018 ceph-mon[28799]: pgmap v61: 65 pgs: 65 active+clean; 2.5 GiB data, 9.4 GiB used, 527 GiB / 536 GiB avail; 38 MiB/s rd, 86 MiB/s wr, 287 op/s 2024-04-12T20:56:57.601 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:56:57 smithi018 ceph-mon[28799]: Standby manager daemon smithi018.xjzwga restarted 2024-04-12T20:56:57.601 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:56:57 smithi018 ceph-mon[28799]: Standby manager daemon smithi018.xjzwga started 2024-04-12T20:56:57.602 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:56:57 smithi018 ceph-mon[28799]: from='mgr.? 172.21.15.18:0/2209228970' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi018.xjzwga/crt"}]: dispatch 2024-04-12T20:56:57.602 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:56:57 smithi018 ceph-mon[28799]: from='mgr.? 172.21.15.18:0/2209228970' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-12T20:56:57.602 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:56:57 smithi018 ceph-mon[28799]: from='mgr.? 172.21.15.18:0/2209228970' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi018.xjzwga/key"}]: dispatch 2024-04-12T20:56:57.602 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:56:57 smithi018 ceph-mon[28799]: from='mgr.? 172.21.15.18:0/2209228970' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-12T20:56:57.780 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:56:57 smithi186 ceph-mon[36597]: pgmap v61: 65 pgs: 65 active+clean; 2.5 GiB data, 9.4 GiB used, 527 GiB / 536 GiB avail; 38 MiB/s rd, 86 MiB/s wr, 287 op/s 2024-04-12T20:56:57.780 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:56:57 smithi186 ceph-mon[36597]: Standby manager daemon smithi018.xjzwga restarted 2024-04-12T20:56:57.781 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:56:57 smithi186 ceph-mon[36597]: Standby manager daemon smithi018.xjzwga started 2024-04-12T20:56:57.781 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:56:57 smithi186 ceph-mon[36597]: from='mgr.? 172.21.15.18:0/2209228970' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi018.xjzwga/crt"}]: dispatch 2024-04-12T20:56:57.781 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:56:57 smithi186 ceph-mon[36597]: from='mgr.? 172.21.15.18:0/2209228970' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-12T20:56:57.781 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:56:57 smithi186 ceph-mon[36597]: from='mgr.? 172.21.15.18:0/2209228970' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi018.xjzwga/key"}]: dispatch 2024-04-12T20:56:57.781 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:56:57 smithi186 ceph-mon[36597]: from='mgr.? 172.21.15.18:0/2209228970' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-12T20:56:58.601 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:56:58 smithi018 ceph-mon[28799]: mgrmap e26: smithi186.kqcfxk(active, since 117s), standbys: smithi018.xjzwga 2024-04-12T20:56:58.601 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:56:58 smithi018 ceph-mon[28799]: pgmap v62: 65 pgs: 65 active+clean; 2.2 GiB data, 8.5 GiB used, 528 GiB / 536 GiB avail; 22 MiB/s rd, 50 MiB/s wr, 212 op/s 2024-04-12T20:56:58.780 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:56:58 smithi186 ceph-mon[36597]: mgrmap e26: smithi186.kqcfxk(active, since 117s), standbys: smithi018.xjzwga 2024-04-12T20:56:58.780 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:56:58 smithi186 ceph-mon[36597]: pgmap v62: 65 pgs: 65 active+clean; 2.2 GiB data, 8.5 GiB used, 528 GiB / 536 GiB avail; 22 MiB/s rd, 50 MiB/s wr, 212 op/s 2024-04-12T20:57:01.031 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:57:00 smithi186 ceph-mon[36597]: pgmap v63: 65 pgs: 65 active+clean; 2.2 GiB data, 8.5 GiB used, 528 GiB / 536 GiB avail; 9.9 MiB/s rd, 21 MiB/s wr, 137 op/s 2024-04-12T20:57:01.031 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:57:00 smithi186 ceph-mon[36597]: from='mgr.24359 172.21.15.186:0/1315906828' entity='mgr.smithi186.kqcfxk' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T20:57:01.104 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:57:00 smithi018 ceph-mon[28799]: pgmap v63: 65 pgs: 65 active+clean; 2.2 GiB data, 8.5 GiB used, 528 GiB / 536 GiB avail; 9.9 MiB/s rd, 21 MiB/s wr, 137 op/s 2024-04-12T20:57:01.104 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:57:00 smithi018 ceph-mon[28799]: from='mgr.24359 172.21.15.186:0/1315906828' entity='mgr.smithi186.kqcfxk' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T20:57:02.780 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:57:02 smithi186 ceph-mon[36597]: pgmap v64: 65 pgs: 65 active+clean; 1.7 GiB data, 6.7 GiB used, 530 GiB / 536 GiB avail; 9.9 MiB/s rd, 22 MiB/s wr, 204 op/s 2024-04-12T20:57:02.780 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:57:02 smithi186 ceph-mon[36597]: from='mgr.24359 172.21.15.186:0/1315906828' entity='mgr.smithi186.kqcfxk' 2024-04-12T20:57:02.780 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:57:02 smithi186 ceph-mon[36597]: from='mgr.24359 172.21.15.186:0/1315906828' entity='mgr.smithi186.kqcfxk' 2024-04-12T20:57:03.101 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:57:02 smithi018 ceph-mon[28799]: pgmap v64: 65 pgs: 65 active+clean; 1.7 GiB data, 6.7 GiB used, 530 GiB / 536 GiB avail; 9.9 MiB/s rd, 22 MiB/s wr, 204 op/s 2024-04-12T20:57:03.101 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:57:02 smithi018 ceph-mon[28799]: from='mgr.24359 172.21.15.186:0/1315906828' entity='mgr.smithi186.kqcfxk' 2024-04-12T20:57:03.101 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:57:02 smithi018 ceph-mon[28799]: from='mgr.24359 172.21.15.186:0/1315906828' entity='mgr.smithi186.kqcfxk' 2024-04-12T20:57:05.530 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:57:05 smithi186 ceph-mon[36597]: pgmap v65: 65 pgs: 65 active+clean; 1.7 GiB data, 6.7 GiB used, 530 GiB / 536 GiB avail; 17 KiB/s rd, 778 KiB/s wr, 116 op/s 2024-04-12T20:57:05.531 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:57:05 smithi186 ceph-mon[36597]: from='mgr.24359 172.21.15.186:0/1315906828' entity='mgr.smithi186.kqcfxk' 2024-04-12T20:57:05.531 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:57:05 smithi186 ceph-mon[36597]: from='mgr.24359 172.21.15.186:0/1315906828' entity='mgr.smithi186.kqcfxk' 2024-04-12T20:57:05.581 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:57:05 smithi018 ceph-mon[28799]: pgmap v65: 65 pgs: 65 active+clean; 1.7 GiB data, 6.7 GiB used, 530 GiB / 536 GiB avail; 17 KiB/s rd, 778 KiB/s wr, 116 op/s 2024-04-12T20:57:05.581 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:57:05 smithi018 ceph-mon[28799]: from='mgr.24359 172.21.15.186:0/1315906828' entity='mgr.smithi186.kqcfxk' 2024-04-12T20:57:05.581 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:57:05 smithi018 ceph-mon[28799]: from='mgr.24359 172.21.15.186:0/1315906828' entity='mgr.smithi186.kqcfxk' 2024-04-12T20:57:07.505 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:57:07 smithi018 ceph-mon[28799]: pgmap v66: 65 pgs: 65 active+clean; 1.4 GiB data, 6.1 GiB used, 530 GiB / 536 GiB avail; 25 KiB/s rd, 874 KiB/s wr, 163 op/s 2024-04-12T20:57:07.530 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:57:07 smithi186 ceph-mon[36597]: pgmap v66: 65 pgs: 65 active+clean; 1.4 GiB data, 6.1 GiB used, 530 GiB / 536 GiB avail; 25 KiB/s rd, 874 KiB/s wr, 163 op/s 2024-04-12T20:57:09.530 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:57:09 smithi186 ceph-mon[36597]: pgmap v67: 65 pgs: 65 active+clean; 1.2 GiB data, 5.4 GiB used, 531 GiB / 536 GiB avail; 23 KiB/s rd, 1021 KiB/s wr, 182 op/s 2024-04-12T20:57:09.601 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:57:09 smithi018 ceph-mon[28799]: pgmap v67: 65 pgs: 65 active+clean; 1.2 GiB data, 5.4 GiB used, 531 GiB / 536 GiB avail; 23 KiB/s rd, 1021 KiB/s wr, 182 op/s 2024-04-12T20:57:11.530 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:57:11 smithi186 ceph-mon[36597]: pgmap v68: 65 pgs: 65 active+clean; 1.2 GiB data, 5.4 GiB used, 531 GiB / 536 GiB avail; 21 KiB/s rd, 843 KiB/s wr, 162 op/s 2024-04-12T20:57:11.601 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:57:11 smithi018 ceph-mon[28799]: pgmap v68: 65 pgs: 65 active+clean; 1.2 GiB data, 5.4 GiB used, 531 GiB / 536 GiB avail; 21 KiB/s rd, 843 KiB/s wr, 162 op/s 2024-04-12T20:57:13.506 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:57:13 smithi018 ceph-mon[28799]: pgmap v69: 65 pgs: 65 active+clean; 980 MiB data, 4.8 GiB used, 532 GiB / 536 GiB avail; 30 KiB/s rd, 1.2 MiB/s wr, 241 op/s 2024-04-12T20:57:13.506 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:57:13 smithi018 ceph-mon[28799]: from='mgr.24359 172.21.15.186:0/1315906828' entity='mgr.smithi186.kqcfxk' 2024-04-12T20:57:13.506 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:57:13 smithi018 ceph-mon[28799]: from='mgr.24359 172.21.15.186:0/1315906828' entity='mgr.smithi186.kqcfxk' 2024-04-12T20:57:13.506 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:57:13 smithi018 ceph-mon[28799]: from='mgr.24359 172.21.15.186:0/1315906828' entity='mgr.smithi186.kqcfxk' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-12T20:57:13.506 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:57:13 smithi018 ceph-mon[28799]: from='mgr.24359 172.21.15.186:0/1315906828' entity='mgr.smithi186.kqcfxk' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-12T20:57:13.506 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:57:13 smithi018 ceph-mon[28799]: from='mgr.24359 172.21.15.186:0/1315906828' entity='mgr.smithi186.kqcfxk' 2024-04-12T20:57:13.506 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:57:13 smithi018 ceph-mon[28799]: from='mgr.24359 172.21.15.186:0/1315906828' entity='mgr.smithi186.kqcfxk' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-12T20:57:13.506 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:57:13 smithi018 ceph-mon[28799]: from='mgr.24359 172.21.15.186:0/1315906828' entity='mgr.smithi186.kqcfxk' cmd=[{"prefix": "mgr fail", "who": "smithi186.kqcfxk"}]: dispatch 2024-04-12T20:57:13.506 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:57:13 smithi018 ceph-mon[28799]: osdmap e48: 6 total, 6 up, 6 in 2024-04-12T20:57:13.507 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:57:13 smithi018 ceph-mon[28799]: from='mgr.24359 172.21.15.186:0/1315906828' entity='mgr.smithi186.kqcfxk' cmd='[{"prefix": "mgr fail", "who": "smithi186.kqcfxk"}]': finished 2024-04-12T20:57:13.507 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:57:13 smithi018 ceph-mon[28799]: mgrmap e27: smithi018.xjzwga(active, starting, since 0.00937556s) 2024-04-12T20:57:13.507 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:57:13 smithi018 ceph-mon[28799]: from='mgr.14672 172.21.15.18:0/2209228970' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "mon metadata", "id": "smithi018"}]: dispatch 2024-04-12T20:57:13.507 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:57:13 smithi018 ceph-mon[28799]: from='mgr.14672 172.21.15.18:0/2209228970' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "mon metadata", "id": "smithi186"}]: dispatch 2024-04-12T20:57:13.507 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:57:13 smithi018 ceph-mon[28799]: from='mgr.14672 172.21.15.18:0/2209228970' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi018.acqbsc"}]: dispatch 2024-04-12T20:57:13.507 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:57:13 smithi018 ceph-mon[28799]: from='mgr.14672 172.21.15.18:0/2209228970' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi018.szjluo"}]: dispatch 2024-04-12T20:57:13.507 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:57:13 smithi018 ceph-mon[28799]: from='mgr.14672 172.21.15.18:0/2209228970' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi186.wjnafs"}]: dispatch 2024-04-12T20:57:13.507 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:57:13 smithi018 ceph-mon[28799]: from='mgr.14672 172.21.15.18:0/2209228970' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi186.jdllyn"}]: dispatch 2024-04-12T20:57:13.507 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:57:13 smithi018 ceph-mon[28799]: from='mgr.14672 172.21.15.18:0/2209228970' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "mgr metadata", "who": "smithi018.xjzwga", "id": "smithi018.xjzwga"}]: dispatch 2024-04-12T20:57:13.507 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:57:13 smithi018 ceph-mon[28799]: from='mgr.14672 172.21.15.18:0/2209228970' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-12T20:57:13.507 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:57:13 smithi018 ceph-mon[28799]: from='mgr.14672 172.21.15.18:0/2209228970' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-12T20:57:13.507 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:57:13 smithi018 ceph-mon[28799]: from='mgr.14672 172.21.15.18:0/2209228970' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-12T20:57:13.507 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:57:13 smithi018 ceph-mon[28799]: from='mgr.14672 172.21.15.18:0/2209228970' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-12T20:57:13.507 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:57:13 smithi018 ceph-mon[28799]: from='mgr.14672 172.21.15.18:0/2209228970' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-12T20:57:13.508 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:57:13 smithi018 ceph-mon[28799]: from='mgr.14672 172.21.15.18:0/2209228970' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-12T20:57:13.508 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:57:13 smithi018 ceph-mon[28799]: from='mgr.14672 172.21.15.18:0/2209228970' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-12T20:57:13.508 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:57:13 smithi018 ceph-mon[28799]: from='mgr.14672 172.21.15.18:0/2209228970' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-12T20:57:13.508 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:57:13 smithi018 ceph-mon[28799]: from='mgr.14672 172.21.15.18:0/2209228970' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-12T20:57:13.508 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:57:13 smithi018 ceph-mon[28799]: Manager daemon smithi018.xjzwga is now available 2024-04-12T20:57:13.508 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:57:13 smithi018 ceph-mon[28799]: from='mgr.14672 172.21.15.18:0/2209228970' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-12T20:57:13.508 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:57:13 smithi018 ceph-mon[28799]: from='mgr.14672 172.21.15.18:0/2209228970' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T20:57:13.508 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:57:13 smithi018 ceph-mon[28799]: from='mgr.14672 172.21.15.18:0/2209228970' entity='mgr.smithi018.xjzwga' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi018.xjzwga/mirror_snapshot_schedule"}]: dispatch 2024-04-12T20:57:13.508 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:57:13 smithi018 ceph-mon[28799]: from='mgr.14672 172.21.15.18:0/2209228970' entity='mgr.smithi018.xjzwga' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi018.xjzwga/trash_purge_schedule"}]: dispatch 2024-04-12T20:57:13.531 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:57:13 smithi186 ceph-mon[36597]: pgmap v69: 65 pgs: 65 active+clean; 980 MiB data, 4.8 GiB used, 532 GiB / 536 GiB avail; 30 KiB/s rd, 1.2 MiB/s wr, 241 op/s 2024-04-12T20:57:13.531 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:57:13 smithi186 ceph-mon[36597]: from='mgr.24359 172.21.15.186:0/1315906828' entity='mgr.smithi186.kqcfxk' 2024-04-12T20:57:13.531 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:57:13 smithi186 ceph-mon[36597]: from='mgr.24359 172.21.15.186:0/1315906828' entity='mgr.smithi186.kqcfxk' 2024-04-12T20:57:13.531 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:57:13 smithi186 ceph-mon[36597]: from='mgr.24359 172.21.15.186:0/1315906828' entity='mgr.smithi186.kqcfxk' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-12T20:57:13.532 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:57:13 smithi186 ceph-mon[36597]: from='mgr.24359 172.21.15.186:0/1315906828' entity='mgr.smithi186.kqcfxk' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-12T20:57:13.532 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:57:13 smithi186 ceph-mon[36597]: from='mgr.24359 172.21.15.186:0/1315906828' entity='mgr.smithi186.kqcfxk' 2024-04-12T20:57:13.532 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:57:13 smithi186 ceph-mon[36597]: from='mgr.24359 172.21.15.186:0/1315906828' entity='mgr.smithi186.kqcfxk' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-12T20:57:13.532 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:57:13 smithi186 ceph-mon[36597]: from='mgr.24359 172.21.15.186:0/1315906828' entity='mgr.smithi186.kqcfxk' cmd=[{"prefix": "mgr fail", "who": "smithi186.kqcfxk"}]: dispatch 2024-04-12T20:57:13.532 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:57:13 smithi186 ceph-mon[36597]: osdmap e48: 6 total, 6 up, 6 in 2024-04-12T20:57:13.532 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:57:13 smithi186 ceph-mon[36597]: from='mgr.24359 172.21.15.186:0/1315906828' entity='mgr.smithi186.kqcfxk' cmd='[{"prefix": "mgr fail", "who": "smithi186.kqcfxk"}]': finished 2024-04-12T20:57:13.532 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:57:13 smithi186 ceph-mon[36597]: mgrmap e27: smithi018.xjzwga(active, starting, since 0.00937556s) 2024-04-12T20:57:13.532 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:57:13 smithi186 ceph-mon[36597]: from='mgr.14672 172.21.15.18:0/2209228970' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "mon metadata", "id": "smithi018"}]: dispatch 2024-04-12T20:57:13.532 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:57:13 smithi186 ceph-mon[36597]: from='mgr.14672 172.21.15.18:0/2209228970' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "mon metadata", "id": "smithi186"}]: dispatch 2024-04-12T20:57:13.532 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:57:13 smithi186 ceph-mon[36597]: from='mgr.14672 172.21.15.18:0/2209228970' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi018.acqbsc"}]: dispatch 2024-04-12T20:57:13.532 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:57:13 smithi186 ceph-mon[36597]: from='mgr.14672 172.21.15.18:0/2209228970' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi018.szjluo"}]: dispatch 2024-04-12T20:57:13.532 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:57:13 smithi186 ceph-mon[36597]: from='mgr.14672 172.21.15.18:0/2209228970' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi186.wjnafs"}]: dispatch 2024-04-12T20:57:13.533 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:57:13 smithi186 ceph-mon[36597]: from='mgr.14672 172.21.15.18:0/2209228970' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi186.jdllyn"}]: dispatch 2024-04-12T20:57:13.533 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:57:13 smithi186 ceph-mon[36597]: from='mgr.14672 172.21.15.18:0/2209228970' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "mgr metadata", "who": "smithi018.xjzwga", "id": "smithi018.xjzwga"}]: dispatch 2024-04-12T20:57:13.533 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:57:13 smithi186 ceph-mon[36597]: from='mgr.14672 172.21.15.18:0/2209228970' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-12T20:57:13.533 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:57:13 smithi186 ceph-mon[36597]: from='mgr.14672 172.21.15.18:0/2209228970' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-12T20:57:13.533 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:57:13 smithi186 ceph-mon[36597]: from='mgr.14672 172.21.15.18:0/2209228970' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-12T20:57:13.533 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:57:13 smithi186 ceph-mon[36597]: from='mgr.14672 172.21.15.18:0/2209228970' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-12T20:57:13.533 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:57:13 smithi186 ceph-mon[36597]: from='mgr.14672 172.21.15.18:0/2209228970' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-12T20:57:13.533 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:57:13 smithi186 ceph-mon[36597]: from='mgr.14672 172.21.15.18:0/2209228970' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-12T20:57:13.533 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:57:13 smithi186 ceph-mon[36597]: from='mgr.14672 172.21.15.18:0/2209228970' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-12T20:57:13.533 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:57:13 smithi186 ceph-mon[36597]: from='mgr.14672 172.21.15.18:0/2209228970' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-12T20:57:13.533 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:57:13 smithi186 ceph-mon[36597]: from='mgr.14672 172.21.15.18:0/2209228970' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-12T20:57:13.533 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:57:13 smithi186 ceph-mon[36597]: Manager daemon smithi018.xjzwga is now available 2024-04-12T20:57:13.533 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:57:13 smithi186 ceph-mon[36597]: from='mgr.14672 172.21.15.18:0/2209228970' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-12T20:57:13.533 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:57:13 smithi186 ceph-mon[36597]: from='mgr.14672 172.21.15.18:0/2209228970' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T20:57:13.533 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:57:13 smithi186 ceph-mon[36597]: from='mgr.14672 172.21.15.18:0/2209228970' entity='mgr.smithi018.xjzwga' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi018.xjzwga/mirror_snapshot_schedule"}]: dispatch 2024-04-12T20:57:13.534 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:57:13 smithi186 ceph-mon[36597]: from='mgr.14672 172.21.15.18:0/2209228970' entity='mgr.smithi018.xjzwga' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi018.xjzwga/trash_purge_schedule"}]: dispatch 2024-04-12T20:57:14.780 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:57:14 smithi186 ceph-mon[36597]: mgrmap e28: smithi018.xjzwga(active, since 1.01638s) 2024-04-12T20:57:14.780 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:57:14 smithi186 ceph-mon[36597]: [12/Apr/2024:20:57:13] ENGINE Bus STARTING 2024-04-12T20:57:14.780 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:57:14 smithi186 ceph-mon[36597]: [12/Apr/2024:20:57:13] ENGINE Serving on http://172.21.15.18:8765 2024-04-12T20:57:14.781 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:57:14 smithi186 ceph-mon[36597]: [12/Apr/2024:20:57:13] ENGINE Serving on https://172.21.15.18:7150 2024-04-12T20:57:14.781 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:57:14 smithi186 ceph-mon[36597]: [12/Apr/2024:20:57:13] ENGINE Bus STARTED 2024-04-12T20:57:14.836 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:57:14 smithi018 ceph-mon[28799]: mgrmap e28: smithi018.xjzwga(active, since 1.01638s) 2024-04-12T20:57:14.836 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:57:14 smithi018 ceph-mon[28799]: [12/Apr/2024:20:57:13] ENGINE Bus STARTING 2024-04-12T20:57:14.836 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:57:14 smithi018 ceph-mon[28799]: [12/Apr/2024:20:57:13] ENGINE Serving on http://172.21.15.18:8765 2024-04-12T20:57:14.836 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:57:14 smithi018 ceph-mon[28799]: [12/Apr/2024:20:57:13] ENGINE Serving on https://172.21.15.18:7150 2024-04-12T20:57:14.837 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:57:14 smithi018 ceph-mon[28799]: [12/Apr/2024:20:57:13] ENGINE Bus STARTED 2024-04-12T20:57:15.799 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:57:15 smithi018 ceph-mon[28799]: pgmap v4: 65 pgs: 65 active+clean; 707 MiB data, 4.1 GiB used, 532 GiB / 536 GiB avail 2024-04-12T20:57:15.821 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:57:15 smithi186 ceph-mon[36597]: pgmap v4: 65 pgs: 65 active+clean; 707 MiB data, 4.1 GiB used, 532 GiB / 536 GiB avail 2024-04-12T20:57:15.948 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:57:15.951+0000 7f02fe000700 1 -- 172.21.15.18:0/3154117546 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f02f80fcee0 msgr2=0x7f02f80fd310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:57:15.949 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:57:15.951+0000 7f02fe000700 1 --2- 172.21.15.18:0/3154117546 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f02f80fcee0 0x7f02f80fd310 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f02e8004940 tx=0x7f02e8009e30 comp rx=0 tx=0).stop 2024-04-12T20:57:15.949 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:57:15.951+0000 7f02fe000700 1 -- 172.21.15.18:0/3154117546 shutdown_connections 2024-04-12T20:57:15.949 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:57:15.951+0000 7f02fe000700 1 --2- 172.21.15.18:0/3154117546 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f02f80fe0d0 0x7f02f80fe540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:57:15.950 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:57:15.951+0000 7f02fe000700 1 --2- 172.21.15.18:0/3154117546 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f02f80fcee0 0x7f02f80fd310 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:57:15.950 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:57:15.951+0000 7f02fe000700 1 -- 172.21.15.18:0/3154117546 >> 172.21.15.18:0/3154117546 conn(0x7f02f80f8560 msgr2=0x7f02f80fa970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T20:57:15.950 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:57:15.951+0000 7f02fe000700 1 -- 172.21.15.18:0/3154117546 shutdown_connections 2024-04-12T20:57:15.950 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:57:15.951+0000 7f02fe000700 1 -- 172.21.15.18:0/3154117546 wait complete. 2024-04-12T20:57:15.950 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:57:15.952+0000 7f02fe000700 1 Processor -- start 2024-04-12T20:57:15.950 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:57:15.953+0000 7f02fe000700 1 -- start start 2024-04-12T20:57:15.951 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:57:15.953+0000 7f02fe000700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f02f80fcee0 0x7f02f8114000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T20:57:15.951 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:57:15.953+0000 7f02fe000700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f02f80fe0d0 0x7f02f8114540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T20:57:15.951 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:57:15.953+0000 7f02fe000700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02f8114b40 con 0x7f02f80fe0d0 2024-04-12T20:57:15.951 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:57:15.953+0000 7f02fe000700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02f8115390 con 0x7f02f80fcee0 2024-04-12T20:57:15.951 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:57:15.954+0000 7f02f6a00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f02f80fe0d0 0x7f02f8114540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T20:57:15.951 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:57:15.954+0000 7f02f6a00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f02f80fe0d0 0x7f02f8114540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:34332/0 (socket says 172.21.15.18:34332) 2024-04-12T20:57:15.951 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:57:15.954+0000 7f02f6a00700 1 -- 172.21.15.18:0/3310335226 learned_addr learned my addr 172.21.15.18:0/3310335226 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T20:57:15.951 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:57:15.954+0000 7f02f7400700 1 --2- 172.21.15.18:0/3310335226 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f02f80fcee0 0x7f02f8114000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T20:57:15.952 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:57:15.954+0000 7f02f6a00700 1 -- 172.21.15.18:0/3310335226 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f02f80fcee0 msgr2=0x7f02f8114000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:57:15.952 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:57:15.954+0000 7f02f6a00700 1 --2- 172.21.15.18:0/3310335226 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f02f80fcee0 0x7f02f8114000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:57:15.952 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:57:15.954+0000 7f02f6a00700 1 -- 172.21.15.18:0/3310335226 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02e8005040 con 0x7f02f80fe0d0 2024-04-12T20:57:15.952 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:57:15.955+0000 7f02f6a00700 1 --2- 172.21.15.18:0/3310335226 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f02f80fe0d0 0x7f02f8114540 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f02e400eb70 tx=0x7f02e400b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T20:57:15.952 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:57:15.955+0000 7f02ef400700 1 -- 172.21.15.18:0/3310335226 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f02e4007cc0 con 0x7f02f80fe0d0 2024-04-12T20:57:15.952 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:57:15.955+0000 7f02ef400700 1 -- 172.21.15.18:0/3310335226 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f02e400f040 con 0x7f02f80fe0d0 2024-04-12T20:57:15.953 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:57:15.955+0000 7f02fe000700 1 -- 172.21.15.18:0/3310335226 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f02f8115610 con 0x7f02f80fe0d0 2024-04-12T20:57:15.953 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:57:15.955+0000 7f02ef400700 1 -- 172.21.15.18:0/3310335226 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f02e400ca10 con 0x7f02f80fe0d0 2024-04-12T20:57:15.953 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:57:15.955+0000 7f02fe000700 1 -- 172.21.15.18:0/3310335226 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f02f8115b50 con 0x7f02f80fe0d0 2024-04-12T20:57:15.954 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:57:15.957+0000 7f02ecc00700 1 -- 172.21.15.18:0/3310335226 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f02f805f5f0 con 0x7f02f80fe0d0 2024-04-12T20:57:15.954 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:57:15.957+0000 7f02ef400700 1 -- 172.21.15.18:0/3310335226 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 29) v1 ==== 46558+0+0 (secure 0 0 0) 0x7f02e400cb70 con 0x7f02f80fe0d0 2024-04-12T20:57:15.955 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:57:15.958+0000 7f02ef400700 1 --2- 172.21.15.18:0/3310335226 >> [v2:172.21.15.18:6800/1605950634,v1:172.21.15.18:6801/1605950634] conn(0x7f02dc0394d0 0x7f02dc03b980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T20:57:15.955 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:57:15.958+0000 7f02ef400700 1 -- 172.21.15.18:0/3310335226 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6066+0+0 (secure 0 0 0) 0x7f02e404ce20 con 0x7f02f80fe0d0 2024-04-12T20:57:15.955 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:57:15.958+0000 7f02f7400700 1 --2- 172.21.15.18:0/3310335226 >> [v2:172.21.15.18:6800/1605950634,v1:172.21.15.18:6801/1605950634] conn(0x7f02dc0394d0 0x7f02dc03b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T20:57:15.956 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:57:15.959+0000 7f02f7400700 1 --2- 172.21.15.18:0/3310335226 >> [v2:172.21.15.18:6800/1605950634,v1:172.21.15.18:6801/1605950634] conn(0x7f02dc0394d0 0x7f02dc03b980 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f02e8013870 tx=0x7f02e8009e30 comp rx=0 tx=0).ready entity=mgr.14672 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T20:57:15.961 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:57:15.964+0000 7f02ef400700 1 -- 172.21.15.18:0/3310335226 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186454 (secure 0 0 0) 0x7f02e4012df0 con 0x7f02f80fe0d0 2024-04-12T20:57:16.324 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:57:16.326+0000 7f02ecc00700 1 -- 172.21.15.18:0/3310335226 --> [v2:172.21.15.18:6800/1605950634,v1:172.21.15.18:6801/1605950634] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f02f81059b0 con 0x7f02dc0394d0 2024-04-12T20:57:16.326 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:57:16.329+0000 7f02ef400700 1 -- 172.21.15.18:0/3310335226 <== mgr.14672 v2:172.21.15.18:6800/1605950634 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+329 (secure 0 0 0) 0x7f02f81059b0 con 0x7f02dc0394d0 2024-04-12T20:57:16.329 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:57:16.332+0000 7f02ecc00700 1 -- 172.21.15.18:0/3310335226 >> [v2:172.21.15.18:6800/1605950634,v1:172.21.15.18:6801/1605950634] conn(0x7f02dc0394d0 msgr2=0x7f02dc03b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:57:16.329 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:57:16.332+0000 7f02ecc00700 1 --2- 172.21.15.18:0/3310335226 >> [v2:172.21.15.18:6800/1605950634,v1:172.21.15.18:6801/1605950634] conn(0x7f02dc0394d0 0x7f02dc03b980 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f02e8013870 tx=0x7f02e8009e30 comp rx=0 tx=0).stop 2024-04-12T20:57:16.329 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:57:16.332+0000 7f02ecc00700 1 -- 172.21.15.18:0/3310335226 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f02f80fe0d0 msgr2=0x7f02f8114540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:57:16.329 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:57:16.332+0000 7f02ecc00700 1 --2- 172.21.15.18:0/3310335226 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f02f80fe0d0 0x7f02f8114540 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f02e400eb70 tx=0x7f02e400b040 comp rx=0 tx=0).stop 2024-04-12T20:57:16.330 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:57:16.332+0000 7f02ecc00700 1 -- 172.21.15.18:0/3310335226 shutdown_connections 2024-04-12T20:57:16.330 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:57:16.333+0000 7f02ecc00700 1 --2- 172.21.15.18:0/3310335226 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f02f80fcee0 0x7f02f8114000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:57:16.330 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:57:16.333+0000 7f02ecc00700 1 --2- 172.21.15.18:0/3310335226 >> [v2:172.21.15.18:6800/1605950634,v1:172.21.15.18:6801/1605950634] conn(0x7f02dc0394d0 0x7f02dc03b980 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:57:16.330 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:57:16.333+0000 7f02ecc00700 1 --2- 172.21.15.18:0/3310335226 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f02f80fe0d0 0x7f02f8114540 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:57:16.330 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:57:16.333+0000 7f02ecc00700 1 -- 172.21.15.18:0/3310335226 >> 172.21.15.18:0/3310335226 conn(0x7f02f80f8560 msgr2=0x7f02f80f9c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T20:57:16.330 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:57:16.333+0000 7f02ecc00700 1 -- 172.21.15.18:0/3310335226 shutdown_connections 2024-04-12T20:57:16.331 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:57:16.333+0000 7f02ecc00700 1 -- 172.21.15.18:0/3310335226 wait complete. 2024-04-12T20:57:16.350 INFO:teuthology.orchestra.run.smithi018.stdout:true 2024-04-12T20:57:16.547 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:57:16.550+0000 7f9c2ec00700 1 -- 172.21.15.18:0/1271549021 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9c28070890 msgr2=0x7f9c28070d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:57:16.547 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:57:16.550+0000 7f9c2ec00700 1 --2- 172.21.15.18:0/1271549021 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9c28070890 0x7f9c28070d00 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f9c14004970 tx=0x7f9c14009e30 comp rx=0 tx=0).stop 2024-04-12T20:57:16.547 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:57:16.550+0000 7f9c2ec00700 1 -- 172.21.15.18:0/1271549021 shutdown_connections 2024-04-12T20:57:16.548 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:57:16.550+0000 7f9c2ec00700 1 --2- 172.21.15.18:0/1271549021 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9c28070890 0x7f9c28070d00 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:57:16.548 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:57:16.550+0000 7f9c2ec00700 1 --2- 172.21.15.18:0/1271549021 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9c28071e40 0x7f9c28070350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:57:16.548 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:57:16.550+0000 7f9c2ec00700 1 -- 172.21.15.18:0/1271549021 >> 172.21.15.18:0/1271549021 conn(0x7f9c280f8390 msgr2=0x7f9c280fa7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T20:57:16.548 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:57:16.550+0000 7f9c2ec00700 1 -- 172.21.15.18:0/1271549021 shutdown_connections 2024-04-12T20:57:16.548 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:57:16.550+0000 7f9c2ec00700 1 -- 172.21.15.18:0/1271549021 wait complete. 2024-04-12T20:57:16.548 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:57:16.551+0000 7f9c2ec00700 1 Processor -- start 2024-04-12T20:57:16.548 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:57:16.551+0000 7f9c2ec00700 1 -- start start 2024-04-12T20:57:16.549 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:57:16.552+0000 7f9c2ec00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9c28070890 0x7f9c28113e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T20:57:16.549 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:57:16.552+0000 7f9c2ec00700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9c28071e40 0x7f9c28114340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T20:57:16.549 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:57:16.552+0000 7f9c2ec00700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c281150e0 con 0x7f9c28070890 2024-04-12T20:57:16.549 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:57:16.552+0000 7f9c2ec00700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c28115250 con 0x7f9c28071e40 2024-04-12T20:57:16.550 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:57:16.552+0000 7f9c2ce00700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9c28071e40 0x7f9c28114340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T20:57:16.550 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:57:16.552+0000 7f9c2ce00700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9c28071e40 0x7f9c28114340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.18:57682/0 (socket says 172.21.15.18:57682) 2024-04-12T20:57:16.550 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:57:16.552+0000 7f9c2ce00700 1 -- 172.21.15.18:0/404314110 learned_addr learned my addr 172.21.15.18:0/404314110 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T20:57:16.550 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:57:16.553+0000 7f9c2d800700 1 --2- 172.21.15.18:0/404314110 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9c28070890 0x7f9c28113e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T20:57:16.550 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:57:16.553+0000 7f9c2ce00700 1 -- 172.21.15.18:0/404314110 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9c28070890 msgr2=0x7f9c28113e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:57:16.550 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:57:16.553+0000 7f9c2ce00700 1 --2- 172.21.15.18:0/404314110 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9c28070890 0x7f9c28113e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:57:16.550 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:57:16.553+0000 7f9c2ce00700 1 -- 172.21.15.18:0/404314110 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c14005040 con 0x7f9c28071e40 2024-04-12T20:57:16.551 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:57:16.553+0000 7f9c2ce00700 1 --2- 172.21.15.18:0/404314110 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9c28071e40 0x7f9c28114340 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f9c14004f50 tx=0x7f9c14009e30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T20:57:16.552 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:57:16.554+0000 7f9c25600700 1 -- 172.21.15.18:0/404314110 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9c1400e040 con 0x7f9c28071e40 2024-04-12T20:57:16.552 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:57:16.554+0000 7f9c25600700 1 -- 172.21.15.18:0/404314110 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f9c1400bce0 con 0x7f9c28071e40 2024-04-12T20:57:16.553 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:57:16.554+0000 7f9c2ec00700 1 -- 172.21.15.18:0/404314110 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9c281154d0 con 0x7f9c28071e40 2024-04-12T20:57:16.553 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:57:16.554+0000 7f9c25600700 1 -- 172.21.15.18:0/404314110 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9c1401c630 con 0x7f9c28071e40 2024-04-12T20:57:16.553 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:57:16.554+0000 7f9c2ec00700 1 -- 172.21.15.18:0/404314110 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9c28115a40 con 0x7f9c28071e40 2024-04-12T20:57:16.553 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:57:16.555+0000 7f9c25600700 1 -- 172.21.15.18:0/404314110 <== mon.1 v2:172.21.15.186:3300/0 4 ==== mgrmap(e 29) v1 ==== 46558+0+0 (secure 0 0 0) 0x7f9c14013420 con 0x7f9c28071e40 2024-04-12T20:57:16.553 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:57:16.555+0000 7f9c2ec00700 1 -- 172.21.15.18:0/404314110 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9c281034c0 con 0x7f9c28071e40 2024-04-12T20:57:16.553 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:57:16.556+0000 7f9c25600700 1 --2- 172.21.15.18:0/404314110 >> [v2:172.21.15.18:6800/1605950634,v1:172.21.15.18:6801/1605950634] conn(0x7f9c10039470 0x7f9c1003b920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T20:57:16.554 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:57:16.556+0000 7f9c25600700 1 -- 172.21.15.18:0/404314110 <== mon.1 v2:172.21.15.186:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6066+0+0 (secure 0 0 0) 0x7f9c1404e6e0 con 0x7f9c28071e40 2024-04-12T20:57:16.554 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:57:16.557+0000 7f9c2d800700 1 --2- 172.21.15.18:0/404314110 >> [v2:172.21.15.18:6800/1605950634,v1:172.21.15.18:6801/1605950634] conn(0x7f9c10039470 0x7f9c1003b920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T20:57:16.554 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:57:16.557+0000 7f9c2d800700 1 --2- 172.21.15.18:0/404314110 >> [v2:172.21.15.18:6800/1605950634,v1:172.21.15.18:6801/1605950634] conn(0x7f9c10039470 0x7f9c1003b920 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f9c280fade0 tx=0x7f9c1c00b040 comp rx=0 tx=0).ready entity=mgr.14672 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T20:57:16.559 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:57:16.562+0000 7f9c25600700 1 -- 172.21.15.18:0/404314110 <== mon.1 v2:172.21.15.186:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186454 (secure 0 0 0) 0x7f9c14011550 con 0x7f9c28071e40 2024-04-12T20:57:16.585 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-04-12T20:57:16.585 DEBUG:teuthology.orchestra.run.smithi186:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.1/client.1/tmp 2024-04-12T20:57:16.601 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:57:16 smithi018 ceph-mon[28799]: mgrmap e29: smithi018.xjzwga(active, since 3s) 2024-04-12T20:57:16.900 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:57:16.903+0000 7f9c2ec00700 1 -- 172.21.15.18:0/404314110 --> [v2:172.21.15.18:6800/1605950634,v1:172.21.15.18:6801/1605950634] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f9c28115cb0 con 0x7f9c10039470 2024-04-12T20:57:16.901 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:57:16.904+0000 7f9c25600700 1 -- 172.21.15.18:0/404314110 <== mgr.14672 v2:172.21.15.18:6800/1605950634 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+329 (secure 0 0 0) 0x7f9c28115cb0 con 0x7f9c10039470 2024-04-12T20:57:16.904 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:57:16.907+0000 7f9c2ec00700 1 -- 172.21.15.18:0/404314110 >> [v2:172.21.15.18:6800/1605950634,v1:172.21.15.18:6801/1605950634] conn(0x7f9c10039470 msgr2=0x7f9c1003b920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:57:16.905 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:57:16.907+0000 7f9c2ec00700 1 --2- 172.21.15.18:0/404314110 >> [v2:172.21.15.18:6800/1605950634,v1:172.21.15.18:6801/1605950634] conn(0x7f9c10039470 0x7f9c1003b920 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f9c280fade0 tx=0x7f9c1c00b040 comp rx=0 tx=0).stop 2024-04-12T20:57:16.905 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:57:16.907+0000 7f9c2ec00700 1 -- 172.21.15.18:0/404314110 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9c28071e40 msgr2=0x7f9c28114340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:57:16.905 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:57:16.907+0000 7f9c2ec00700 1 --2- 172.21.15.18:0/404314110 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9c28071e40 0x7f9c28114340 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f9c14004f50 tx=0x7f9c14009e30 comp rx=0 tx=0).stop 2024-04-12T20:57:16.905 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:57:16.907+0000 7f9c2ec00700 1 -- 172.21.15.18:0/404314110 shutdown_connections 2024-04-12T20:57:16.905 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:57:16.907+0000 7f9c2ec00700 1 --2- 172.21.15.18:0/404314110 >> [v2:172.21.15.18:6800/1605950634,v1:172.21.15.18:6801/1605950634] conn(0x7f9c10039470 0x7f9c1003b920 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:57:16.905 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:57:16.907+0000 7f9c2ec00700 1 --2- 172.21.15.18:0/404314110 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9c28070890 0x7f9c28113e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:57:16.905 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:57:16.908+0000 7f9c2ec00700 1 --2- 172.21.15.18:0/404314110 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9c28071e40 0x7f9c28114340 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:57:16.905 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:57:16.908+0000 7f9c2ec00700 1 -- 172.21.15.18:0/404314110 >> 172.21.15.18:0/404314110 conn(0x7f9c280f8390 msgr2=0x7f9c280f9980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T20:57:16.905 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:57:16.908+0000 7f9c2ec00700 1 -- 172.21.15.18:0/404314110 shutdown_connections 2024-04-12T20:57:16.905 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:57:16.908+0000 7f9c2ec00700 1 -- 172.21.15.18:0/404314110 wait complete. 2024-04-12T20:57:17.030 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:57:16 smithi186 ceph-mon[36597]: mgrmap e29: smithi018.xjzwga(active, since 3s) 2024-04-12T20:57:17.098 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:57:17.100+0000 7f0d89e00700 1 -- 172.21.15.18:0/1671572504 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0d840feff0 msgr2=0x7f0d840ff400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:57:17.098 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:57:17.100+0000 7f0d89e00700 1 --2- 172.21.15.18:0/1671572504 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0d840feff0 0x7f0d840ff400 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f0d70004940 tx=0x7f0d70009e30 comp rx=0 tx=0).stop 2024-04-12T20:57:17.099 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:57:17.101+0000 7f0d89e00700 1 -- 172.21.15.18:0/1671572504 shutdown_connections 2024-04-12T20:57:17.099 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:57:17.101+0000 7f0d89e00700 1 --2- 172.21.15.18:0/1671572504 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0d841002a0 0x7f0d84100710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:57:17.099 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:57:17.101+0000 7f0d89e00700 1 --2- 172.21.15.18:0/1671572504 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0d840feff0 0x7f0d840ff400 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:57:17.099 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:57:17.101+0000 7f0d89e00700 1 -- 172.21.15.18:0/1671572504 >> 172.21.15.18:0/1671572504 conn(0x7f0d840fa860 msgr2=0x7f0d840fcc40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T20:57:17.099 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:57:17.102+0000 7f0d89e00700 1 -- 172.21.15.18:0/1671572504 shutdown_connections 2024-04-12T20:57:17.099 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:57:17.102+0000 7f0d89e00700 1 -- 172.21.15.18:0/1671572504 wait complete. 2024-04-12T20:57:17.101 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:57:17.104+0000 7f0d89e00700 1 Processor -- start 2024-04-12T20:57:17.101 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:57:17.105+0000 7f0d89e00700 1 -- start start 2024-04-12T20:57:17.102 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:57:17.105+0000 7f0d89e00700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0d840feff0 0x7f0d8410fc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T20:57:17.103 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:57:17.106+0000 7f0d89e00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0d841002a0 0x7f0d84110160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T20:57:17.103 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:57:17.106+0000 7f0d89e00700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d84110760 con 0x7f0d841002a0 2024-04-12T20:57:17.104 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:57:17.106+0000 7f0d89e00700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d84110fb0 con 0x7f0d840feff0 2024-04-12T20:57:17.104 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:57:17.107+0000 7f0d82000700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0d841002a0 0x7f0d84110160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T20:57:17.104 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:57:17.107+0000 7f0d82a00700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0d840feff0 0x7f0d8410fc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T20:57:17.104 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:57:17.107+0000 7f0d82a00700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0d840feff0 0x7f0d8410fc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.18:57698/0 (socket says 172.21.15.18:57698) 2024-04-12T20:57:17.104 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:57:17.107+0000 7f0d82a00700 1 -- 172.21.15.18:0/211985896 learned_addr learned my addr 172.21.15.18:0/211985896 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T20:57:17.105 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:57:17.108+0000 7f0d82000700 1 -- 172.21.15.18:0/211985896 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0d840feff0 msgr2=0x7f0d8410fc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:57:17.105 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:57:17.108+0000 7f0d82000700 1 --2- 172.21.15.18:0/211985896 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0d840feff0 0x7f0d8410fc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:57:17.105 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:57:17.108+0000 7f0d82000700 1 -- 172.21.15.18:0/211985896 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d70005040 con 0x7f0d841002a0 2024-04-12T20:57:17.106 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:57:17.109+0000 7f0d82000700 1 --2- 172.21.15.18:0/211985896 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0d841002a0 0x7f0d84110160 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f0d74001c00 tx=0x7f0d74009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T20:57:17.106 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:57:17.109+0000 7f0d7aa00700 1 -- 172.21.15.18:0/211985896 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0d74007de0 con 0x7f0d841002a0 2024-04-12T20:57:17.107 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:57:17.109+0000 7f0d89e00700 1 -- 172.21.15.18:0/211985896 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0d84111290 con 0x7f0d841002a0 2024-04-12T20:57:17.107 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:57:17.109+0000 7f0d89e00700 1 -- 172.21.15.18:0/211985896 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0d841117d0 con 0x7f0d841002a0 2024-04-12T20:57:17.107 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:57:17.109+0000 7f0d7aa00700 1 -- 172.21.15.18:0/211985896 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f0d7400ebd0 con 0x7f0d841002a0 2024-04-12T20:57:17.107 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:57:17.109+0000 7f0d7aa00700 1 -- 172.21.15.18:0/211985896 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0d74014400 con 0x7f0d841002a0 2024-04-12T20:57:17.108 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:57:17.111+0000 7f0d7aa00700 1 -- 172.21.15.18:0/211985896 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 29) v1 ==== 46558+0+0 (secure 0 0 0) 0x7f0d74014560 con 0x7f0d841002a0 2024-04-12T20:57:17.111 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:57:17.113+0000 7f0d7aa00700 1 --2- 172.21.15.18:0/211985896 >> [v2:172.21.15.18:6800/1605950634,v1:172.21.15.18:6801/1605950634] conn(0x7f0d68039520 0x7f0d6803b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T20:57:17.111 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:57:17.113+0000 7f0d7aa00700 1 -- 172.21.15.18:0/211985896 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6066+0+0 (secure 0 0 0) 0x7f0d7404e020 con 0x7f0d841002a0 2024-04-12T20:57:17.112 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:57:17.114+0000 7f0d89e00700 1 -- 172.21.15.18:0/211985896 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0d640052e0 con 0x7f0d841002a0 2024-04-12T20:57:17.112 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:57:17.115+0000 7f0d82a00700 1 --2- 172.21.15.18:0/211985896 >> [v2:172.21.15.18:6800/1605950634,v1:172.21.15.18:6801/1605950634] conn(0x7f0d68039520 0x7f0d6803b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T20:57:17.114 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:57:17.117+0000 7f0d82a00700 1 --2- 172.21.15.18:0/211985896 >> [v2:172.21.15.18:6800/1605950634,v1:172.21.15.18:6801/1605950634] conn(0x7f0d68039520 0x7f0d6803b9d0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f0d70004f90 tx=0x7f0d70009e30 comp rx=0 tx=0).ready entity=mgr.14672 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T20:57:17.123 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:57:17.126+0000 7f0d7aa00700 1 -- 172.21.15.18:0/211985896 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186454 (secure 0 0 0) 0x7f0d7401a850 con 0x7f0d841002a0 2024-04-12T20:57:17.514 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:57:17.517+0000 7f0d89e00700 1 -- 172.21.15.18:0/211985896 --> [v2:172.21.15.18:6800/1605950634,v1:172.21.15.18:6801/1605950634] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f0d64000bf0 con 0x7f0d68039520 2024-04-12T20:57:17.534 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:57:17.536+0000 7f0d7aa00700 1 -- 172.21.15.18:0/211985896 <== mgr.14672 v2:172.21.15.18:6800/1605950634 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3520 (secure 0 0 0) 0x7f0d64000bf0 con 0x7f0d68039520 2024-04-12T20:57:17.534 INFO:teuthology.orchestra.run.smithi018.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-12T20:57:17.534 INFO:teuthology.orchestra.run.smithi018.stdout:alertmanager.smithi018 smithi018 *:9093,9094 running (96s) 15s ago 12m 15.1M - 0.25.0 c8568f914cd2 b4ed63d191d4 2024-04-12T20:57:17.534 INFO:teuthology.orchestra.run.smithi018.stdout:crash.smithi018 smithi018 running (12m) 15s ago 12m 6660k - 17.2.7-827-gd3b5060a 21de9b7369ad 1ac7603bddfd 2024-04-12T20:57:17.534 INFO:teuthology.orchestra.run.smithi018.stdout:crash.smithi186 smithi186 running (11m) 27s ago 11m 6669k - 17.2.7-827-gd3b5060a 21de9b7369ad 14bf8a1672eb 2024-04-12T20:57:17.535 INFO:teuthology.orchestra.run.smithi018.stdout:grafana.smithi018 smithi018 *:3000 running (87s) 15s ago 12m 75.2M - 9.4.7 954c08fa6188 d6251dcdaec0 2024-04-12T20:57:17.535 INFO:teuthology.orchestra.run.smithi018.stdout:mds.cephfs.smithi018.acqbsc smithi018 running (5m) 15s ago 5m 133M - 17.2.7-827-gd3b5060a 21de9b7369ad 5b5fce65f18a 2024-04-12T20:57:17.535 INFO:teuthology.orchestra.run.smithi018.stdout:mds.cephfs.smithi018.szjluo smithi018 running (5m) 15s ago 5m 142M - 17.2.7-827-gd3b5060a 21de9b7369ad cf9bf0c5031f 2024-04-12T20:57:17.535 INFO:teuthology.orchestra.run.smithi018.stdout:mds.cephfs.smithi186.jdllyn smithi186 running (5m) 27s ago 5m 12.3M - 17.2.7-827-gd3b5060a 21de9b7369ad 2b2bc46bd29a 2024-04-12T20:57:17.535 INFO:teuthology.orchestra.run.smithi018.stdout:mds.cephfs.smithi186.wjnafs smithi186 running (5m) 27s ago 5m 12.3M - 17.2.7-827-gd3b5060a 21de9b7369ad 74b115d6d6fd 2024-04-12T20:57:17.535 INFO:teuthology.orchestra.run.smithi018.stdout:mgr.smithi018.xjzwga smithi018 *:8443,9283,8765 running (34s) 15s ago 14m 64.7M - 19.0.0-2879-g324baf8d 8717720dc5e5 56088305eab7 2024-04-12T20:57:17.535 INFO:teuthology.orchestra.run.smithi018.stdout:mgr.smithi186.kqcfxk smithi186 *:8443,9283 running (2m) 27s ago 11m 505M - 19.0.0-2879-g324baf8d 8717720dc5e5 bd4971d5260e 2024-04-12T20:57:17.535 INFO:teuthology.orchestra.run.smithi018.stdout:mon.smithi018 smithi018 running (14m) 15s ago 14m 50.6M 2048M 17.2.7-827-gd3b5060a 21de9b7369ad cd84caafa056 2024-04-12T20:57:17.535 INFO:teuthology.orchestra.run.smithi018.stdout:mon.smithi186 smithi186 running (10m) 27s ago 10m 39.2M 2048M 17.2.7-827-gd3b5060a 21de9b7369ad d7263c772b73 2024-04-12T20:57:17.535 INFO:teuthology.orchestra.run.smithi018.stdout:node-exporter.smithi018 smithi018 *:9100 running (109s) 15s ago 12m 7922k - 1.5.0 0da6a335fe13 2d717b2cd3c2 2024-04-12T20:57:17.535 INFO:teuthology.orchestra.run.smithi018.stdout:node-exporter.smithi186 smithi186 *:9100 running (45s) 27s ago 11m 6651k - 1.5.0 0da6a335fe13 4ac3111e3ae2 2024-04-12T20:57:17.535 INFO:teuthology.orchestra.run.smithi018.stdout:osd.0 smithi018 running (10m) 15s ago 10m 216M 1559M 17.2.7-827-gd3b5060a 21de9b7369ad 0d83e027bf0a 2024-04-12T20:57:17.535 INFO:teuthology.orchestra.run.smithi018.stdout:osd.1 smithi018 running (9m) 15s ago 9m 230M 1559M 17.2.7-827-gd3b5060a 21de9b7369ad 6517c9cc7f87 2024-04-12T20:57:17.535 INFO:teuthology.orchestra.run.smithi018.stdout:osd.2 smithi018 running (8m) 15s ago 8m 181M 1559M 17.2.7-827-gd3b5060a 21de9b7369ad 8a83692acaa5 2024-04-12T20:57:17.536 INFO:teuthology.orchestra.run.smithi018.stdout:osd.3 smithi186 running (8m) 27s ago 8m 253M 2581M 17.2.7-827-gd3b5060a 21de9b7369ad c2def7a90d76 2024-04-12T20:57:17.536 INFO:teuthology.orchestra.run.smithi018.stdout:osd.4 smithi186 running (7m) 27s ago 7m 214M 2581M 17.2.7-827-gd3b5060a 21de9b7369ad 757e7c71b7b9 2024-04-12T20:57:17.536 INFO:teuthology.orchestra.run.smithi018.stdout:osd.5 smithi186 running (7m) 27s ago 7m 250M 2581M 17.2.7-827-gd3b5060a 21de9b7369ad 2ee072a436d0 2024-04-12T20:57:17.536 INFO:teuthology.orchestra.run.smithi018.stdout:prometheus.smithi018 smithi018 *:9095 running (54s) 15s ago 11m 43.3M - 2.43.0 a07b618ecd1d 34ccc01df66b 2024-04-12T20:57:17.538 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:57:17.541+0000 7f0d89e00700 1 -- 172.21.15.18:0/211985896 >> [v2:172.21.15.18:6800/1605950634,v1:172.21.15.18:6801/1605950634] conn(0x7f0d68039520 msgr2=0x7f0d6803b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:57:17.538 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:57:17.541+0000 7f0d89e00700 1 --2- 172.21.15.18:0/211985896 >> [v2:172.21.15.18:6800/1605950634,v1:172.21.15.18:6801/1605950634] conn(0x7f0d68039520 0x7f0d6803b9d0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f0d70004f90 tx=0x7f0d70009e30 comp rx=0 tx=0).stop 2024-04-12T20:57:17.538 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:57:17.541+0000 7f0d89e00700 1 -- 172.21.15.18:0/211985896 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0d841002a0 msgr2=0x7f0d84110160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:57:17.538 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:57:17.541+0000 7f0d89e00700 1 --2- 172.21.15.18:0/211985896 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0d841002a0 0x7f0d84110160 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f0d74001c00 tx=0x7f0d74009da0 comp rx=0 tx=0).stop 2024-04-12T20:57:17.539 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:57:17.541+0000 7f0d89e00700 1 -- 172.21.15.18:0/211985896 shutdown_connections 2024-04-12T20:57:17.539 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:57:17.541+0000 7f0d89e00700 1 --2- 172.21.15.18:0/211985896 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0d840feff0 0x7f0d8410fc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:57:17.539 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:57:17.541+0000 7f0d89e00700 1 --2- 172.21.15.18:0/211985896 >> [v2:172.21.15.18:6800/1605950634,v1:172.21.15.18:6801/1605950634] conn(0x7f0d68039520 0x7f0d6803b9d0 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:57:17.539 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:57:17.541+0000 7f0d89e00700 1 --2- 172.21.15.18:0/211985896 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0d841002a0 0x7f0d84110160 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:57:17.539 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:57:17.541+0000 7f0d89e00700 1 -- 172.21.15.18:0/211985896 >> 172.21.15.18:0/211985896 conn(0x7f0d840fa860 msgr2=0x7f0d840fcf60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T20:57:17.539 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:57:17.541+0000 7f0d89e00700 1 -- 172.21.15.18:0/211985896 shutdown_connections 2024-04-12T20:57:17.539 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:57:17.541+0000 7f0d89e00700 1 -- 172.21.15.18:0/211985896 wait complete. 2024-04-12T20:57:17.726 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:57:17.728+0000 7f8119400700 1 -- 172.21.15.18:0/2159676373 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f81140fcee0 msgr2=0x7f81140fd310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:57:17.726 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:57:17.728+0000 7f8119400700 1 --2- 172.21.15.18:0/2159676373 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f81140fcee0 0x7f81140fd310 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f8104004940 tx=0x7f8104009e30 comp rx=0 tx=0).stop 2024-04-12T20:57:17.727 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:57:17.728+0000 7f8119400700 1 -- 172.21.15.18:0/2159676373 shutdown_connections 2024-04-12T20:57:17.727 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:57:17.728+0000 7f8119400700 1 --2- 172.21.15.18:0/2159676373 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f81140fe0d0 0x7f81140fe540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:57:17.727 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:57:17.728+0000 7f8119400700 1 --2- 172.21.15.18:0/2159676373 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f81140fcee0 0x7f81140fd310 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:57:17.727 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:57:17.728+0000 7f8119400700 1 -- 172.21.15.18:0/2159676373 >> 172.21.15.18:0/2159676373 conn(0x7f81140f8560 msgr2=0x7f81140fa970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T20:57:17.727 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:57:17.729+0000 7f8119400700 1 -- 172.21.15.18:0/2159676373 shutdown_connections 2024-04-12T20:57:17.727 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:57:17.729+0000 7f8119400700 1 -- 172.21.15.18:0/2159676373 wait complete. 2024-04-12T20:57:17.727 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:57:17.729+0000 7f8119400700 1 Processor -- start 2024-04-12T20:57:17.727 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:57:17.730+0000 7f8119400700 1 -- start start 2024-04-12T20:57:17.728 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:57:17.730+0000 7f8119400700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f81140fcee0 0x7f8114114080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T20:57:17.728 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:57:17.730+0000 7f8119400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f81140fe0d0 0x7f81141145c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T20:57:17.728 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:57:17.730+0000 7f8119400700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8114114bc0 con 0x7f81140fe0d0 2024-04-12T20:57:17.728 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:57:17.730+0000 7f8119400700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8114115410 con 0x7f81140fcee0 2024-04-12T20:57:17.728 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:57:17.731+0000 7f8112800700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f81140fcee0 0x7f8114114080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T20:57:17.728 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:57:17.731+0000 7f8112800700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f81140fcee0 0x7f8114114080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.18:57710/0 (socket says 172.21.15.18:57710) 2024-04-12T20:57:17.729 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:57:17.731+0000 7f8112800700 1 -- 172.21.15.18:0/2842379134 learned_addr learned my addr 172.21.15.18:0/2842379134 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T20:57:17.729 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:57:17.731+0000 7f8111e00700 1 --2- 172.21.15.18:0/2842379134 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f81140fe0d0 0x7f81141145c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T20:57:17.729 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:57:17.731+0000 7f8112800700 1 -- 172.21.15.18:0/2842379134 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f81140fe0d0 msgr2=0x7f81141145c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:57:17.729 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:57:17.731+0000 7f8112800700 1 --2- 172.21.15.18:0/2842379134 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f81140fe0d0 0x7f81141145c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:57:17.729 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:57:17.731+0000 7f8112800700 1 -- 172.21.15.18:0/2842379134 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8104005040 con 0x7f81140fcee0 2024-04-12T20:57:17.729 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:57:17.732+0000 7f8112800700 1 --2- 172.21.15.18:0/2842379134 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f81140fcee0 0x7f8114114080 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f810400b970 tx=0x7f8104009e30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T20:57:17.730 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:57:17.733+0000 7f810aa00700 1 -- 172.21.15.18:0/2842379134 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f810400e040 con 0x7f81140fcee0 2024-04-12T20:57:17.730 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:57:17.733+0000 7f810aa00700 1 -- 172.21.15.18:0/2842379134 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f810400bbf0 con 0x7f81140fcee0 2024-04-12T20:57:17.730 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:57:17.733+0000 7f8119400700 1 -- 172.21.15.18:0/2842379134 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8114115690 con 0x7f81140fcee0 2024-04-12T20:57:17.730 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:57:17.733+0000 7f810aa00700 1 -- 172.21.15.18:0/2842379134 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f810401c3f0 con 0x7f81140fcee0 2024-04-12T20:57:17.730 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:57:17.733+0000 7f8119400700 1 -- 172.21.15.18:0/2842379134 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8114115bd0 con 0x7f81140fcee0 2024-04-12T20:57:17.733 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:57:17.735+0000 7f8119400700 1 -- 172.21.15.18:0/2842379134 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8114101a60 con 0x7f81140fcee0 2024-04-12T20:57:17.733 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:57:17.735+0000 7f810aa00700 1 -- 172.21.15.18:0/2842379134 <== mon.1 v2:172.21.15.186:3300/0 4 ==== mgrmap(e 29) v1 ==== 46558+0+0 (secure 0 0 0) 0x7f810400bd60 con 0x7f81140fcee0 2024-04-12T20:57:17.733 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:57:17.736+0000 7f810aa00700 1 --2- 172.21.15.18:0/2842379134 >> [v2:172.21.15.18:6800/1605950634,v1:172.21.15.18:6801/1605950634] conn(0x7f80f8039480 0x7f80f803b930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T20:57:17.733 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:57:17.736+0000 7f810aa00700 1 -- 172.21.15.18:0/2842379134 <== mon.1 v2:172.21.15.186:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6066+0+0 (secure 0 0 0) 0x7f810404e720 con 0x7f81140fcee0 2024-04-12T20:57:17.733 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:57:17.736+0000 7f8111e00700 1 --2- 172.21.15.18:0/2842379134 >> [v2:172.21.15.18:6800/1605950634,v1:172.21.15.18:6801/1605950634] conn(0x7f80f8039480 0x7f80f803b930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T20:57:17.734 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:57:17.737+0000 7f8111e00700 1 --2- 172.21.15.18:0/2842379134 >> [v2:172.21.15.18:6800/1605950634,v1:172.21.15.18:6801/1605950634] conn(0x7f80f8039480 0x7f80f803b930 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f8100004f20 tx=0x7f810000b040 comp rx=0 tx=0).ready entity=mgr.14672 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T20:57:17.750 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:57:17.753+0000 7f810aa00700 1 -- 172.21.15.18:0/2842379134 <== mon.1 v2:172.21.15.186:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186454 (secure 0 0 0) 0x7f8104011940 con 0x7f81140fcee0 2024-04-12T20:57:17.851 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:57:17 smithi018 ceph-mon[28799]: from='client.14704 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T20:57:17.851 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:57:17 smithi018 ceph-mon[28799]: pgmap v5: 65 pgs: 65 active+clean; 707 MiB data, 4.1 GiB used, 532 GiB / 536 GiB avail 2024-04-12T20:57:18.030 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:57:17 smithi186 ceph-mon[36597]: from='client.14704 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T20:57:18.030 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:57:17 smithi186 ceph-mon[36597]: pgmap v5: 65 pgs: 65 active+clean; 707 MiB data, 4.1 GiB used, 532 GiB / 536 GiB avail 2024-04-12T20:57:18.240 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:57:18.243+0000 7f8119400700 1 -- 172.21.15.18:0/2842379134 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f811404bed0 con 0x7f81140fcee0 2024-04-12T20:57:18.241 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:57:18.244+0000 7f810aa00700 1 -- 172.21.15.18:0/2842379134 <== mon.1 v2:172.21.15.186:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+739 (secure 0 0 0) 0x7f8104011370 con 0x7f81140fcee0 2024-04-12T20:57:18.241 INFO:teuthology.orchestra.run.smithi018.stdout:{ 2024-04-12T20:57:18.241 INFO:teuthology.orchestra.run.smithi018.stdout: "mon": { 2024-04-12T20:57:18.241 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 17.2.7-827-gd3b5060a (d3b5060a17449edfae9385cdf792afec109fca5d) quincy (stable)": 2 2024-04-12T20:57:18.242 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-12T20:57:18.242 INFO:teuthology.orchestra.run.smithi018.stdout: "mgr": { 2024-04-12T20:57:18.242 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 19.0.0-2879-g324baf8d (324baf8ddb91dea46cac50cb535c306115791355) squid (dev)": 1 2024-04-12T20:57:18.242 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-12T20:57:18.242 INFO:teuthology.orchestra.run.smithi018.stdout: "osd": { 2024-04-12T20:57:18.242 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 17.2.7-827-gd3b5060a (d3b5060a17449edfae9385cdf792afec109fca5d) quincy (stable)": 6 2024-04-12T20:57:18.242 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-12T20:57:18.242 INFO:teuthology.orchestra.run.smithi018.stdout: "mds": { 2024-04-12T20:57:18.242 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 17.2.7-827-gd3b5060a (d3b5060a17449edfae9385cdf792afec109fca5d) quincy (stable)": 4 2024-04-12T20:57:18.242 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-12T20:57:18.242 INFO:teuthology.orchestra.run.smithi018.stdout: "overall": { 2024-04-12T20:57:18.242 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 17.2.7-827-gd3b5060a (d3b5060a17449edfae9385cdf792afec109fca5d) quincy (stable)": 12, 2024-04-12T20:57:18.242 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 19.0.0-2879-g324baf8d (324baf8ddb91dea46cac50cb535c306115791355) squid (dev)": 1 2024-04-12T20:57:18.242 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-04-12T20:57:18.242 INFO:teuthology.orchestra.run.smithi018.stdout:} 2024-04-12T20:57:18.244 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:57:18.248+0000 7f8119400700 1 -- 172.21.15.18:0/2842379134 >> [v2:172.21.15.18:6800/1605950634,v1:172.21.15.18:6801/1605950634] conn(0x7f80f8039480 msgr2=0x7f80f803b930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:57:18.245 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:57:18.248+0000 7f8119400700 1 --2- 172.21.15.18:0/2842379134 >> [v2:172.21.15.18:6800/1605950634,v1:172.21.15.18:6801/1605950634] conn(0x7f80f8039480 0x7f80f803b930 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f8100004f20 tx=0x7f810000b040 comp rx=0 tx=0).stop 2024-04-12T20:57:18.245 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:57:18.248+0000 7f8119400700 1 -- 172.21.15.18:0/2842379134 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f81140fcee0 msgr2=0x7f8114114080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:57:18.245 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:57:18.248+0000 7f8119400700 1 --2- 172.21.15.18:0/2842379134 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f81140fcee0 0x7f8114114080 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f810400b970 tx=0x7f8104009e30 comp rx=0 tx=0).stop 2024-04-12T20:57:18.246 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:57:18.248+0000 7f8119400700 1 -- 172.21.15.18:0/2842379134 shutdown_connections 2024-04-12T20:57:18.246 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:57:18.248+0000 7f8119400700 1 --2- 172.21.15.18:0/2842379134 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f81140fcee0 0x7f8114114080 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:57:18.246 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:57:18.248+0000 7f8119400700 1 --2- 172.21.15.18:0/2842379134 >> [v2:172.21.15.18:6800/1605950634,v1:172.21.15.18:6801/1605950634] conn(0x7f80f8039480 0x7f80f803b930 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:57:18.246 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:57:18.248+0000 7f8119400700 1 --2- 172.21.15.18:0/2842379134 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f81140fe0d0 0x7f81141145c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:57:18.246 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:57:18.248+0000 7f8119400700 1 -- 172.21.15.18:0/2842379134 >> 172.21.15.18:0/2842379134 conn(0x7f81140f8560 msgr2=0x7f81140f9d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T20:57:18.246 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:57:18.249+0000 7f8119400700 1 -- 172.21.15.18:0/2842379134 shutdown_connections 2024-04-12T20:57:18.247 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:57:18.249+0000 7f8119400700 1 -- 172.21.15.18:0/2842379134 wait complete. 2024-04-12T20:57:18.443 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:57:18.446+0000 7fd364400700 1 -- 172.21.15.18:0/4050486121 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd35c105600 msgr2=0x7fd35c105a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:57:18.443 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:57:18.446+0000 7fd364400700 1 --2- 172.21.15.18:0/4050486121 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd35c105600 0x7fd35c105a10 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fd350004940 tx=0x7fd350009e30 comp rx=0 tx=0).stop 2024-04-12T20:57:18.444 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:57:18.446+0000 7fd364400700 1 -- 172.21.15.18:0/4050486121 shutdown_connections 2024-04-12T20:57:18.444 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:57:18.446+0000 7fd364400700 1 --2- 172.21.15.18:0/4050486121 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd35c1068b0 0x7fd35c106d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:57:18.444 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:57:18.446+0000 7fd364400700 1 --2- 172.21.15.18:0/4050486121 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd35c105600 0x7fd35c105a10 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:57:18.444 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:57:18.446+0000 7fd364400700 1 -- 172.21.15.18:0/4050486121 >> 172.21.15.18:0/4050486121 conn(0x7fd35c06bf80 msgr2=0x7fd35c06e3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T20:57:18.444 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:57:18.447+0000 7fd364400700 1 -- 172.21.15.18:0/4050486121 shutdown_connections 2024-04-12T20:57:18.444 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:57:18.447+0000 7fd364400700 1 -- 172.21.15.18:0/4050486121 wait complete. 2024-04-12T20:57:18.445 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:57:18.448+0000 7fd364400700 1 Processor -- start 2024-04-12T20:57:18.445 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:57:18.448+0000 7fd364400700 1 -- start start 2024-04-12T20:57:18.446 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:57:18.449+0000 7fd364400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd35c105600 0x7fd35c068df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T20:57:18.446 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:57:18.449+0000 7fd364400700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd35c1068b0 0x7fd35c0674a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T20:57:18.446 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:57:18.449+0000 7fd364400700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd35c119e30 con 0x7fd35c105600 2024-04-12T20:57:18.446 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:57:18.449+0000 7fd364400700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd35c119fa0 con 0x7fd35c1068b0 2024-04-12T20:57:18.446 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:57:18.449+0000 7fd361a00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd35c105600 0x7fd35c068df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T20:57:18.446 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:57:18.449+0000 7fd361a00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd35c105600 0x7fd35c068df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:34406/0 (socket says 172.21.15.18:34406) 2024-04-12T20:57:18.446 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:57:18.449+0000 7fd361a00700 1 -- 172.21.15.18:0/3086903551 learned_addr learned my addr 172.21.15.18:0/3086903551 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T20:57:18.446 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:57:18.449+0000 7fd361000700 1 --2- 172.21.15.18:0/3086903551 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd35c1068b0 0x7fd35c0674a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T20:57:18.447 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:57:18.450+0000 7fd361a00700 1 -- 172.21.15.18:0/3086903551 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd35c1068b0 msgr2=0x7fd35c0674a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:57:18.447 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:57:18.450+0000 7fd361a00700 1 --2- 172.21.15.18:0/3086903551 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd35c1068b0 0x7fd35c0674a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:57:18.447 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:57:18.450+0000 7fd361a00700 1 -- 172.21.15.18:0/3086903551 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd350005040 con 0x7fd35c105600 2024-04-12T20:57:18.447 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:57:18.450+0000 7fd361a00700 1 --2- 172.21.15.18:0/3086903551 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd35c105600 0x7fd35c068df0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fd35000b3b0 tx=0x7fd350009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T20:57:18.447 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:57:18.450+0000 7fd361000700 1 --2- 172.21.15.18:0/3086903551 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd35c1068b0 0x7fd35c0674a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-12T20:57:18.448 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:57:18.450+0000 7fd359600700 1 -- 172.21.15.18:0/3086903551 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd35000e040 con 0x7fd35c105600 2024-04-12T20:57:18.448 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:57:18.450+0000 7fd359600700 1 -- 172.21.15.18:0/3086903551 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd350004290 con 0x7fd35c105600 2024-04-12T20:57:18.448 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:57:18.450+0000 7fd359600700 1 -- 172.21.15.18:0/3086903551 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd35001c630 con 0x7fd35c105600 2024-04-12T20:57:18.448 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:57:18.451+0000 7fd364400700 1 -- 172.21.15.18:0/3086903551 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd35c0679e0 con 0x7fd35c105600 2024-04-12T20:57:18.448 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:57:18.451+0000 7fd364400700 1 -- 172.21.15.18:0/3086903551 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd35c067fa0 con 0x7fd35c105600 2024-04-12T20:57:18.450 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:57:18.452+0000 7fd359600700 1 -- 172.21.15.18:0/3086903551 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 29) v1 ==== 46558+0+0 (secure 0 0 0) 0x7fd35000bbf0 con 0x7fd35c105600 2024-04-12T20:57:18.450 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:57:18.452+0000 7fd359600700 1 --2- 172.21.15.18:0/3086903551 >> [v2:172.21.15.18:6800/1605950634,v1:172.21.15.18:6801/1605950634] conn(0x7fd344039470 0x7fd34403b920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T20:57:18.450 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:57:18.453+0000 7fd359600700 1 -- 172.21.15.18:0/3086903551 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6066+0+0 (secure 0 0 0) 0x7fd35004e7a0 con 0x7fd35c105600 2024-04-12T20:57:18.450 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:57:18.453+0000 7fd364400700 1 -- 172.21.15.18:0/3086903551 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd35c04c710 con 0x7fd35c105600 2024-04-12T20:57:18.453 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:57:18.456+0000 7fd361000700 1 --2- 172.21.15.18:0/3086903551 >> [v2:172.21.15.18:6800/1605950634,v1:172.21.15.18:6801/1605950634] conn(0x7fd344039470 0x7fd34403b920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T20:57:18.454 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:57:18.457+0000 7fd361000700 1 --2- 172.21.15.18:0/3086903551 >> [v2:172.21.15.18:6800/1605950634,v1:172.21.15.18:6801/1605950634] conn(0x7fd344039470 0x7fd34403b920 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fd348009b10 tx=0x7fd348008040 comp rx=0 tx=0).ready entity=mgr.14672 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T20:57:18.457 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:57:18.460+0000 7fd359600700 1 -- 172.21.15.18:0/3086903551 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186454 (secure 0 0 0) 0x7fd350011940 con 0x7fd35c105600 2024-04-12T20:57:18.916 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:57:18.919+0000 7fd364400700 1 -- 172.21.15.18:0/3086903551 --> [v2:172.21.15.18:6800/1605950634,v1:172.21.15.18:6801/1605950634] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fd35c10e300 con 0x7fd344039470 2024-04-12T20:57:18.918 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:57:18.920+0000 7fd359600700 1 -- 172.21.15.18:0/3086903551 <== mgr.14672 v2:172.21.15.18:6800/1605950634 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+329 (secure 0 0 0) 0x7fd35c10e300 con 0x7fd344039470 2024-04-12T20:57:18.918 INFO:teuthology.orchestra.run.smithi018.stdout:{ 2024-04-12T20:57:18.918 INFO:teuthology.orchestra.run.smithi018.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:f4452def3be4d350f4a8e5feaf7ebb1511be9298fe68478ba7a2d6fbd7d69a63", 2024-04-12T20:57:18.918 INFO:teuthology.orchestra.run.smithi018.stdout: "in_progress": true, 2024-04-12T20:57:18.918 INFO:teuthology.orchestra.run.smithi018.stdout: "which": "Upgrading daemons of type(s) mgr", 2024-04-12T20:57:18.918 INFO:teuthology.orchestra.run.smithi018.stdout: "services_complete": [ 2024-04-12T20:57:18.918 INFO:teuthology.orchestra.run.smithi018.stdout: "mgr" 2024-04-12T20:57:18.919 INFO:teuthology.orchestra.run.smithi018.stdout: ], 2024-04-12T20:57:18.919 INFO:teuthology.orchestra.run.smithi018.stdout: "progress": "2/2 daemons upgraded", 2024-04-12T20:57:18.919 INFO:teuthology.orchestra.run.smithi018.stdout: "message": "", 2024-04-12T20:57:18.919 INFO:teuthology.orchestra.run.smithi018.stdout: "is_paused": false 2024-04-12T20:57:18.919 INFO:teuthology.orchestra.run.smithi018.stdout:} 2024-04-12T20:57:18.921 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:57:18.924+0000 7fd364400700 1 -- 172.21.15.18:0/3086903551 >> [v2:172.21.15.18:6800/1605950634,v1:172.21.15.18:6801/1605950634] conn(0x7fd344039470 msgr2=0x7fd34403b920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:57:18.921 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:57:18.924+0000 7fd364400700 1 --2- 172.21.15.18:0/3086903551 >> [v2:172.21.15.18:6800/1605950634,v1:172.21.15.18:6801/1605950634] conn(0x7fd344039470 0x7fd34403b920 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fd348009b10 tx=0x7fd348008040 comp rx=0 tx=0).stop 2024-04-12T20:57:18.921 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:57:18.924+0000 7fd364400700 1 -- 172.21.15.18:0/3086903551 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd35c105600 msgr2=0x7fd35c068df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:57:18.921 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:57:18.924+0000 7fd364400700 1 --2- 172.21.15.18:0/3086903551 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd35c105600 0x7fd35c068df0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fd35000b3b0 tx=0x7fd350009e30 comp rx=0 tx=0).stop 2024-04-12T20:57:18.921 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:57:18.924+0000 7fd364400700 1 -- 172.21.15.18:0/3086903551 shutdown_connections 2024-04-12T20:57:18.922 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:57:18.924+0000 7fd364400700 1 --2- 172.21.15.18:0/3086903551 >> [v2:172.21.15.18:6800/1605950634,v1:172.21.15.18:6801/1605950634] conn(0x7fd344039470 0x7fd34403b920 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:57:18.922 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:57:18.924+0000 7fd364400700 1 --2- 172.21.15.18:0/3086903551 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd35c105600 0x7fd35c068df0 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:57:18.922 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:57:18.924+0000 7fd364400700 1 --2- 172.21.15.18:0/3086903551 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd35c1068b0 0x7fd35c0674a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:57:18.922 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:57:18.924+0000 7fd364400700 1 -- 172.21.15.18:0/3086903551 >> 172.21.15.18:0/3086903551 conn(0x7fd35c06bf80 msgr2=0x7fd35c06d5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T20:57:18.922 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:57:18.925+0000 7fd364400700 1 -- 172.21.15.18:0/3086903551 shutdown_connections 2024-04-12T20:57:18.922 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:57:18.925+0000 7fd364400700 1 -- 172.21.15.18:0/3086903551 wait complete. 2024-04-12T20:57:19.030 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:57:18 smithi186 ceph-mon[36597]: from='client.24487 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T20:57:19.030 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:57:18 smithi186 ceph-mon[36597]: from='client.14712 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T20:57:19.030 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:57:18 smithi186 ceph-mon[36597]: from='mgr.14672 172.21.15.18:0/2209228970' entity='mgr.smithi018.xjzwga' 2024-04-12T20:57:19.030 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:57:18 smithi186 ceph-mon[36597]: from='mgr.14672 172.21.15.18:0/2209228970' entity='mgr.smithi018.xjzwga' 2024-04-12T20:57:19.031 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:57:18 smithi186 ceph-mon[36597]: from='client.? 172.21.15.18:0/2842379134' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T20:57:19.103 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:57:18 smithi018 ceph-mon[28799]: from='client.24487 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T20:57:19.104 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:57:18 smithi018 ceph-mon[28799]: from='client.14712 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T20:57:19.104 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:57:18 smithi018 ceph-mon[28799]: from='mgr.14672 172.21.15.18:0/2209228970' entity='mgr.smithi018.xjzwga' 2024-04-12T20:57:19.104 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:57:18 smithi018 ceph-mon[28799]: from='mgr.14672 172.21.15.18:0/2209228970' entity='mgr.smithi018.xjzwga' 2024-04-12T20:57:19.104 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:57:18 smithi018 ceph-mon[28799]: from='client.? 172.21.15.18:0/2842379134' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T20:57:19.930 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:57:19 smithi186 ceph-mon[36597]: pgmap v6: 65 pgs: 65 active+clean; 707 MiB data, 4.1 GiB used, 532 GiB / 536 GiB avail 2024-04-12T20:57:20.101 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:57:19 smithi018 ceph-mon[28799]: pgmap v6: 65 pgs: 65 active+clean; 707 MiB data, 4.1 GiB used, 532 GiB / 536 GiB avail 2024-04-12T20:57:21.030 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:57:20 smithi186 ceph-mon[36597]: from='client.14720 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T20:57:21.030 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:57:20 smithi186 ceph-mon[36597]: from='mgr.14672 172.21.15.18:0/2209228970' entity='mgr.smithi018.xjzwga' 2024-04-12T20:57:21.030 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:57:20 smithi186 ceph-mon[36597]: from='mgr.14672 172.21.15.18:0/2209228970' entity='mgr.smithi018.xjzwga' 2024-04-12T20:57:21.101 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:57:20 smithi018 ceph-mon[28799]: from='client.14720 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T20:57:21.101 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:57:20 smithi018 ceph-mon[28799]: from='mgr.14672 172.21.15.18:0/2209228970' entity='mgr.smithi018.xjzwga' 2024-04-12T20:57:21.102 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:57:20 smithi018 ceph-mon[28799]: from='mgr.14672 172.21.15.18:0/2209228970' entity='mgr.smithi018.xjzwga' 2024-04-12T20:57:22.030 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:57:21 smithi186 ceph-mon[36597]: pgmap v7: 65 pgs: 65 active+clean; 285 MiB data, 2.9 GiB used, 534 GiB / 536 GiB avail; 54 KiB/s rd, 952 KiB/s wr, 177 op/s 2024-04-12T20:57:22.030 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:57:21 smithi186 ceph-mon[36597]: from='mgr.14672 172.21.15.18:0/2209228970' entity='mgr.smithi018.xjzwga' 2024-04-12T20:57:22.030 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:57:21 smithi186 ceph-mon[36597]: from='mgr.14672 172.21.15.18:0/2209228970' entity='mgr.smithi018.xjzwga' 2024-04-12T20:57:22.101 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:57:21 smithi018 ceph-mon[28799]: pgmap v7: 65 pgs: 65 active+clean; 285 MiB data, 2.9 GiB used, 534 GiB / 536 GiB avail; 54 KiB/s rd, 952 KiB/s wr, 177 op/s 2024-04-12T20:57:22.101 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:57:21 smithi018 ceph-mon[28799]: from='mgr.14672 172.21.15.18:0/2209228970' entity='mgr.smithi018.xjzwga' 2024-04-12T20:57:22.101 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:57:21 smithi018 ceph-mon[28799]: from='mgr.14672 172.21.15.18:0/2209228970' entity='mgr.smithi018.xjzwga' 2024-04-12T20:57:24.101 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:57:23 smithi018 ceph-mon[28799]: pgmap v8: 65 pgs: 65 active+clean; 285 MiB data, 2.9 GiB used, 534 GiB / 536 GiB avail; 42 KiB/s rd, 740 KiB/s wr, 137 op/s 2024-04-12T20:57:24.158 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:57:23 smithi186 ceph-mon[36597]: pgmap v8: 65 pgs: 65 active+clean; 285 MiB data, 2.9 GiB used, 534 GiB / 536 GiB avail; 42 KiB/s rd, 740 KiB/s wr, 137 op/s 2024-04-12T20:57:25.101 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:57:24 smithi018 ceph-mon[28799]: Standby manager daemon smithi186.kqcfxk started 2024-04-12T20:57:25.101 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:57:24 smithi018 ceph-mon[28799]: from='mgr.? 172.21.15.186:0/4120757040' entity='mgr.smithi186.kqcfxk' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi186.kqcfxk/crt"}]: dispatch 2024-04-12T20:57:25.101 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:57:24 smithi018 ceph-mon[28799]: from='mgr.? 172.21.15.186:0/4120757040' entity='mgr.smithi186.kqcfxk' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-12T20:57:25.102 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:57:24 smithi018 ceph-mon[28799]: from='mgr.? 172.21.15.186:0/4120757040' entity='mgr.smithi186.kqcfxk' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi186.kqcfxk/key"}]: dispatch 2024-04-12T20:57:25.102 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:57:24 smithi018 ceph-mon[28799]: from='mgr.? 172.21.15.186:0/4120757040' entity='mgr.smithi186.kqcfxk' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-12T20:57:25.102 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:57:24 smithi018 ceph-mon[28799]: from='mgr.14672 172.21.15.18:0/2209228970' entity='mgr.smithi018.xjzwga' 2024-04-12T20:57:25.102 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:57:24 smithi018 ceph-mon[28799]: from='mgr.14672 172.21.15.18:0/2209228970' entity='mgr.smithi018.xjzwga' 2024-04-12T20:57:25.280 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:57:24 smithi186 ceph-mon[36597]: Standby manager daemon smithi186.kqcfxk started 2024-04-12T20:57:25.280 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:57:24 smithi186 ceph-mon[36597]: from='mgr.? 172.21.15.186:0/4120757040' entity='mgr.smithi186.kqcfxk' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi186.kqcfxk/crt"}]: dispatch 2024-04-12T20:57:25.280 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:57:24 smithi186 ceph-mon[36597]: from='mgr.? 172.21.15.186:0/4120757040' entity='mgr.smithi186.kqcfxk' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-12T20:57:25.280 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:57:24 smithi186 ceph-mon[36597]: from='mgr.? 172.21.15.186:0/4120757040' entity='mgr.smithi186.kqcfxk' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi186.kqcfxk/key"}]: dispatch 2024-04-12T20:57:25.280 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:57:24 smithi186 ceph-mon[36597]: from='mgr.? 172.21.15.186:0/4120757040' entity='mgr.smithi186.kqcfxk' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-12T20:57:25.280 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:57:24 smithi186 ceph-mon[36597]: from='mgr.14672 172.21.15.18:0/2209228970' entity='mgr.smithi018.xjzwga' 2024-04-12T20:57:25.281 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:57:24 smithi186 ceph-mon[36597]: from='mgr.14672 172.21.15.18:0/2209228970' entity='mgr.smithi018.xjzwga' 2024-04-12T20:57:25.795 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:57:25 smithi018 ceph-mon[28799]: pgmap v9: 65 pgs: 65 active+clean; 287 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 50 KiB/s rd, 1.2 MiB/s wr, 289 op/s 2024-04-12T20:57:25.795 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:57:25 smithi018 ceph-mon[28799]: mgrmap e30: smithi018.xjzwga(active, since 12s), standbys: smithi186.kqcfxk 2024-04-12T20:57:25.795 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:57:25 smithi018 ceph-mon[28799]: from='mgr.14672 172.21.15.18:0/2209228970' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "mgr metadata", "who": "smithi186.kqcfxk", "id": "smithi186.kqcfxk"}]: dispatch 2024-04-12T20:57:26.280 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:57:25 smithi186 ceph-mon[36597]: pgmap v9: 65 pgs: 65 active+clean; 287 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 50 KiB/s rd, 1.2 MiB/s wr, 289 op/s 2024-04-12T20:57:26.280 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:57:25 smithi186 ceph-mon[36597]: mgrmap e30: smithi018.xjzwga(active, since 12s), standbys: smithi186.kqcfxk 2024-04-12T20:57:26.280 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:57:25 smithi186 ceph-mon[36597]: from='mgr.14672 172.21.15.18:0/2209228970' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "mgr metadata", "who": "smithi186.kqcfxk", "id": "smithi186.kqcfxk"}]: dispatch 2024-04-12T20:57:28.101 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:57:27 smithi018 ceph-mon[28799]: pgmap v10: 65 pgs: 65 active+clean; 287 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 45 KiB/s rd, 1.1 MiB/s wr, 265 op/s 2024-04-12T20:57:28.101 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:57:27 smithi018 ceph-mon[28799]: from='mgr.14672 172.21.15.18:0/2209228970' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T20:57:28.280 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:57:27 smithi186 ceph-mon[36597]: pgmap v10: 65 pgs: 65 active+clean; 287 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 45 KiB/s rd, 1.1 MiB/s wr, 265 op/s 2024-04-12T20:57:28.280 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:57:27 smithi186 ceph-mon[36597]: from='mgr.14672 172.21.15.18:0/2209228970' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T20:57:30.101 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:57:29 smithi018 ceph-mon[28799]: pgmap v11: 65 pgs: 65 active+clean; 287 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 45 KiB/s rd, 1.1 MiB/s wr, 265 op/s 2024-04-12T20:57:30.280 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:57:29 smithi186 ceph-mon[36597]: pgmap v11: 65 pgs: 65 active+clean; 287 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 45 KiB/s rd, 1.1 MiB/s wr, 265 op/s 2024-04-12T20:57:32.083 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:57:31 smithi186 ceph-mon[36597]: pgmap v12: 65 pgs: 65 active+clean; 282 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 60 KiB/s rd, 1.6 MiB/s wr, 419 op/s 2024-04-12T20:57:32.101 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:57:31 smithi018 ceph-mon[28799]: pgmap v12: 65 pgs: 65 active+clean; 282 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 60 KiB/s rd, 1.6 MiB/s wr, 419 op/s 2024-04-12T20:57:33.811 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:57:33 smithi018 ceph-mon[28799]: pgmap v13: 65 pgs: 65 active+clean; 282 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 29 KiB/s rd, 1.1 MiB/s wr, 316 op/s 2024-04-12T20:57:34.155 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:57:33 smithi186 ceph-mon[36597]: pgmap v13: 65 pgs: 65 active+clean; 282 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 29 KiB/s rd, 1.1 MiB/s wr, 316 op/s 2024-04-12T20:57:36.101 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:57:35 smithi018 ceph-mon[28799]: pgmap v14: 65 pgs: 65 active+clean; 281 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 45 KiB/s rd, 1.7 MiB/s wr, 482 op/s 2024-04-12T20:57:36.281 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:57:35 smithi186 ceph-mon[36597]: pgmap v14: 65 pgs: 65 active+clean; 281 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 45 KiB/s rd, 1.7 MiB/s wr, 482 op/s 2024-04-12T20:57:37.280 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:57:36 smithi186 ceph-mon[36597]: pgmap v15: 65 pgs: 65 active+clean; 281 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 31 KiB/s rd, 1.1 MiB/s wr, 320 op/s 2024-04-12T20:57:37.351 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:57:36 smithi018 ceph-mon[28799]: pgmap v15: 65 pgs: 65 active+clean; 281 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 31 KiB/s rd, 1.1 MiB/s wr, 320 op/s 2024-04-12T20:57:39.101 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:57:38 smithi018 ceph-mon[28799]: from='mgr.14672 172.21.15.18:0/2209228970' entity='mgr.smithi018.xjzwga' 2024-04-12T20:57:39.101 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:57:38 smithi018 ceph-mon[28799]: from='mgr.14672 172.21.15.18:0/2209228970' entity='mgr.smithi018.xjzwga' 2024-04-12T20:57:39.101 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:57:38 smithi018 ceph-mon[28799]: from='mgr.14672 172.21.15.18:0/2209228970' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-12T20:57:39.101 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:57:38 smithi018 ceph-mon[28799]: from='mgr.14672 172.21.15.18:0/2209228970' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-12T20:57:39.102 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:57:38 smithi018 ceph-mon[28799]: Updating smithi018:/etc/ceph/ceph.conf 2024-04-12T20:57:39.102 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:57:38 smithi018 ceph-mon[28799]: Updating smithi186:/etc/ceph/ceph.conf 2024-04-12T20:57:39.102 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:57:38 smithi018 ceph-mon[28799]: pgmap v16: 65 pgs: 65 active+clean; 281 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 31 KiB/s rd, 1.1 MiB/s wr, 320 op/s 2024-04-12T20:57:39.102 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:57:38 smithi018 ceph-mon[28799]: Updating smithi186:/var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/config/ceph.conf 2024-04-12T20:57:39.102 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:57:38 smithi018 ceph-mon[28799]: Updating smithi018:/var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/config/ceph.conf 2024-04-12T20:57:39.280 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:57:38 smithi186 ceph-mon[36597]: from='mgr.14672 172.21.15.18:0/2209228970' entity='mgr.smithi018.xjzwga' 2024-04-12T20:57:39.280 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:57:38 smithi186 ceph-mon[36597]: from='mgr.14672 172.21.15.18:0/2209228970' entity='mgr.smithi018.xjzwga' 2024-04-12T20:57:39.280 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:57:38 smithi186 ceph-mon[36597]: from='mgr.14672 172.21.15.18:0/2209228970' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-12T20:57:39.280 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:57:38 smithi186 ceph-mon[36597]: from='mgr.14672 172.21.15.18:0/2209228970' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-12T20:57:39.280 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:57:38 smithi186 ceph-mon[36597]: Updating smithi018:/etc/ceph/ceph.conf 2024-04-12T20:57:39.280 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:57:38 smithi186 ceph-mon[36597]: Updating smithi186:/etc/ceph/ceph.conf 2024-04-12T20:57:39.281 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:57:38 smithi186 ceph-mon[36597]: pgmap v16: 65 pgs: 65 active+clean; 281 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 31 KiB/s rd, 1.1 MiB/s wr, 320 op/s 2024-04-12T20:57:39.281 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:57:38 smithi186 ceph-mon[36597]: Updating smithi186:/var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/config/ceph.conf 2024-04-12T20:57:39.281 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:57:38 smithi186 ceph-mon[36597]: Updating smithi018:/var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/config/ceph.conf 2024-04-12T20:57:40.280 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:57:39 smithi186 ceph-mon[36597]: Updating smithi186:/etc/ceph/ceph.client.admin.keyring 2024-04-12T20:57:40.351 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:57:39 smithi018 ceph-mon[28799]: Updating smithi186:/etc/ceph/ceph.client.admin.keyring 2024-04-12T20:57:41.280 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:57:40 smithi186 ceph-mon[36597]: Updating smithi186:/var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/config/ceph.client.admin.keyring 2024-04-12T20:57:41.280 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:57:40 smithi186 ceph-mon[36597]: Updating smithi018:/etc/ceph/ceph.client.admin.keyring 2024-04-12T20:57:41.280 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:57:40 smithi186 ceph-mon[36597]: pgmap v17: 65 pgs: 65 active+clean; 281 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 43 KiB/s rd, 1.7 MiB/s wr, 445 op/s 2024-04-12T20:57:41.280 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:57:40 smithi186 ceph-mon[36597]: from='mgr.14672 172.21.15.18:0/2209228970' entity='mgr.smithi018.xjzwga' 2024-04-12T20:57:41.280 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:57:40 smithi186 ceph-mon[36597]: from='mgr.14672 172.21.15.18:0/2209228970' entity='mgr.smithi018.xjzwga' 2024-04-12T20:57:41.351 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:57:40 smithi018 ceph-mon[28799]: Updating smithi186:/var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/config/ceph.client.admin.keyring 2024-04-12T20:57:41.351 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:57:40 smithi018 ceph-mon[28799]: Updating smithi018:/etc/ceph/ceph.client.admin.keyring 2024-04-12T20:57:41.351 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:57:40 smithi018 ceph-mon[28799]: pgmap v17: 65 pgs: 65 active+clean; 281 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 43 KiB/s rd, 1.7 MiB/s wr, 445 op/s 2024-04-12T20:57:41.351 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:57:40 smithi018 ceph-mon[28799]: from='mgr.14672 172.21.15.18:0/2209228970' entity='mgr.smithi018.xjzwga' 2024-04-12T20:57:41.351 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:57:40 smithi018 ceph-mon[28799]: from='mgr.14672 172.21.15.18:0/2209228970' entity='mgr.smithi018.xjzwga' 2024-04-12T20:57:42.280 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:57:41 smithi186 ceph-mon[36597]: Updating smithi018:/var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/config/ceph.client.admin.keyring 2024-04-12T20:57:42.280 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:57:41 smithi186 ceph-mon[36597]: from='mgr.14672 172.21.15.18:0/2209228970' entity='mgr.smithi018.xjzwga' 2024-04-12T20:57:42.281 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:57:41 smithi186 ceph-mon[36597]: from='mgr.14672 172.21.15.18:0/2209228970' entity='mgr.smithi018.xjzwga' 2024-04-12T20:57:42.281 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:57:41 smithi186 ceph-mon[36597]: from='mgr.14672 172.21.15.18:0/2209228970' entity='mgr.smithi018.xjzwga' 2024-04-12T20:57:42.351 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:57:41 smithi018 ceph-mon[28799]: Updating smithi018:/var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/config/ceph.client.admin.keyring 2024-04-12T20:57:42.351 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:57:41 smithi018 ceph-mon[28799]: from='mgr.14672 172.21.15.18:0/2209228970' entity='mgr.smithi018.xjzwga' 2024-04-12T20:57:42.351 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:57:41 smithi018 ceph-mon[28799]: from='mgr.14672 172.21.15.18:0/2209228970' entity='mgr.smithi018.xjzwga' 2024-04-12T20:57:42.351 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:57:41 smithi018 ceph-mon[28799]: from='mgr.14672 172.21.15.18:0/2209228970' entity='mgr.smithi018.xjzwga' 2024-04-12T20:57:43.280 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:57:42 smithi186 ceph-mon[36597]: Reconfiguring prometheus.smithi018 (dependencies changed)... 2024-04-12T20:57:43.281 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:57:42 smithi186 ceph-mon[36597]: pgmap v18: 65 pgs: 65 active+clean; 281 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 29 KiB/s rd, 1.1 MiB/s wr, 290 op/s 2024-04-12T20:57:43.281 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:57:42 smithi186 ceph-mon[36597]: Reconfiguring daemon prometheus.smithi018 on smithi018 2024-04-12T20:57:43.281 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:57:42 smithi186 ceph-mon[36597]: from='mgr.14672 172.21.15.18:0/2209228970' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T20:57:43.352 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:57:42 smithi018 ceph-mon[28799]: Reconfiguring prometheus.smithi018 (dependencies changed)... 2024-04-12T20:57:43.352 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:57:42 smithi018 ceph-mon[28799]: pgmap v18: 65 pgs: 65 active+clean; 281 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 29 KiB/s rd, 1.1 MiB/s wr, 290 op/s 2024-04-12T20:57:43.353 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:57:42 smithi018 ceph-mon[28799]: Reconfiguring daemon prometheus.smithi018 on smithi018 2024-04-12T20:57:43.353 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:57:42 smithi018 ceph-mon[28799]: from='mgr.14672 172.21.15.18:0/2209228970' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T20:57:45.690 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:57:45 smithi018 ceph-mon[28799]: pgmap v19: 65 pgs: 65 active+clean; 282 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 45 KiB/s rd, 1.5 MiB/s wr, 449 op/s 2024-04-12T20:57:46.030 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:57:45 smithi186 ceph-mon[36597]: pgmap v19: 65 pgs: 65 active+clean; 282 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 45 KiB/s rd, 1.5 MiB/s wr, 449 op/s 2024-04-12T20:57:47.601 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:57:47 smithi018 ceph-mon[28799]: pgmap v20: 65 pgs: 65 active+clean; 282 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 28 KiB/s rd, 982 KiB/s wr, 283 op/s 2024-04-12T20:57:48.030 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:57:47 smithi186 ceph-mon[36597]: pgmap v20: 65 pgs: 65 active+clean; 282 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 28 KiB/s rd, 982 KiB/s wr, 283 op/s 2024-04-12T20:57:49.145 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:57:49.145+0000 7fe7f5000700 1 -- 172.21.15.18:0/604392778 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe7f00708f0 msgr2=0x7fe7f0070d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:57:49.145 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:57:49.145+0000 7fe7f5000700 1 --2- 172.21.15.18:0/604392778 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe7f00708f0 0x7fe7f0070d60 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fe7dc004940 tx=0x7fe7dc009e30 comp rx=0 tx=0).stop 2024-04-12T20:57:49.145 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:57:49.147+0000 7fe7f5000700 1 -- 172.21.15.18:0/604392778 shutdown_connections 2024-04-12T20:57:49.145 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:57:49.147+0000 7fe7f5000700 1 --2- 172.21.15.18:0/604392778 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe7f00708f0 0x7fe7f0070d60 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:57:49.145 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:57:49.147+0000 7fe7f5000700 1 --2- 172.21.15.18:0/604392778 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe7f0071ea0 0x7fe7f00703b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:57:49.145 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:57:49.147+0000 7fe7f5000700 1 -- 172.21.15.18:0/604392778 >> 172.21.15.18:0/604392778 conn(0x7fe7f00f8560 msgr2=0x7fe7f00fa970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T20:57:49.146 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:57:49.147+0000 7fe7f5000700 1 -- 172.21.15.18:0/604392778 shutdown_connections 2024-04-12T20:57:49.146 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:57:49.147+0000 7fe7f5000700 1 -- 172.21.15.18:0/604392778 wait complete. 2024-04-12T20:57:49.146 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:57:49.148+0000 7fe7f5000700 1 Processor -- start 2024-04-12T20:57:49.146 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:57:49.148+0000 7fe7f5000700 1 -- start start 2024-04-12T20:57:49.147 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:57:49.149+0000 7fe7f5000700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe7f00708f0 0x7fe7f010fc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T20:57:49.147 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:57:49.149+0000 7fe7f5000700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe7f0071ea0 0x7fe7f0110150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T20:57:49.147 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:57:49.149+0000 7fe7f5000700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7f0110750 con 0x7fe7f0071ea0 2024-04-12T20:57:49.147 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:57:49.149+0000 7fe7f5000700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7f0110fa0 con 0x7fe7f00708f0 2024-04-12T20:57:49.147 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:57:49.149+0000 7fe7ed400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe7f0071ea0 0x7fe7f0110150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T20:57:49.148 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:57:49.149+0000 7fe7ed400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe7f0071ea0 0x7fe7f0110150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:52088/0 (socket says 172.21.15.18:52088) 2024-04-12T20:57:49.148 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:57:49.149+0000 7fe7ed400700 1 -- 172.21.15.18:0/245155134 learned_addr learned my addr 172.21.15.18:0/245155134 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T20:57:49.148 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:57:49.150+0000 7fe7ede00700 1 --2- 172.21.15.18:0/245155134 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe7f00708f0 0x7fe7f010fc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T20:57:49.148 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:57:49.150+0000 7fe7ed400700 1 -- 172.21.15.18:0/245155134 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe7f00708f0 msgr2=0x7fe7f010fc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:57:49.148 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:57:49.150+0000 7fe7ed400700 1 --2- 172.21.15.18:0/245155134 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe7f00708f0 0x7fe7f010fc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:57:49.148 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:57:49.150+0000 7fe7ed400700 1 -- 172.21.15.18:0/245155134 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7dc005040 con 0x7fe7f0071ea0 2024-04-12T20:57:49.148 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:57:49.151+0000 7fe7ed400700 1 --2- 172.21.15.18:0/245155134 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe7f0071ea0 0x7fe7f0110150 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fe7dc00b970 tx=0x7fe7dc009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T20:57:49.150 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:57:49.151+0000 7fe7e6000700 1 -- 172.21.15.18:0/245155134 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe7dc00e030 con 0x7fe7f0071ea0 2024-04-12T20:57:49.150 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:57:49.151+0000 7fe7e6000700 1 -- 172.21.15.18:0/245155134 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe7dc004290 con 0x7fe7f0071ea0 2024-04-12T20:57:49.150 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:57:49.151+0000 7fe7e6000700 1 -- 172.21.15.18:0/245155134 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe7dc01c640 con 0x7fe7f0071ea0 2024-04-12T20:57:49.150 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:57:49.151+0000 7fe7f5000700 1 -- 172.21.15.18:0/245155134 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe7f0111220 con 0x7fe7f0071ea0 2024-04-12T20:57:49.150 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:57:49.151+0000 7fe7f5000700 1 -- 172.21.15.18:0/245155134 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe7f0111700 con 0x7fe7f0071ea0 2024-04-12T20:57:49.150 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:57:49.153+0000 7fe7f5000700 1 -- 172.21.15.18:0/245155134 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe7f005f5f0 con 0x7fe7f0071ea0 2024-04-12T20:57:49.152 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:57:49.154+0000 7fe7e6000700 1 -- 172.21.15.18:0/245155134 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 30) v1 ==== 92561+0+0 (secure 0 0 0) 0x7fe7dc00bbf0 con 0x7fe7f0071ea0 2024-04-12T20:57:49.154 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:57:49.156+0000 7fe7e6000700 1 --2- 172.21.15.18:0/245155134 >> [v2:172.21.15.18:6800/1605950634,v1:172.21.15.18:6801/1605950634] conn(0x7fe7d406e670 0x7fe7d4070b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T20:57:49.154 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:57:49.156+0000 7fe7e6000700 1 -- 172.21.15.18:0/245155134 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6066+0+0 (secure 0 0 0) 0x7fe7dc08f910 con 0x7fe7f0071ea0 2024-04-12T20:57:49.154 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:57:49.156+0000 7fe7ede00700 1 --2- 172.21.15.18:0/245155134 >> [v2:172.21.15.18:6800/1605950634,v1:172.21.15.18:6801/1605950634] conn(0x7fe7d406e670 0x7fe7d4070b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T20:57:49.156 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:57:49.158+0000 7fe7ede00700 1 --2- 172.21.15.18:0/245155134 >> [v2:172.21.15.18:6800/1605950634,v1:172.21.15.18:6801/1605950634] conn(0x7fe7d406e670 0x7fe7d4070b20 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fe7f0071be0 tx=0x7fe7e000b040 comp rx=0 tx=0).ready entity=mgr.14672 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T20:57:49.159 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:57:49.161+0000 7fe7e6000700 1 -- 172.21.15.18:0/245155134 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186454 (secure 0 0 0) 0x7fe7dc05c8d0 con 0x7fe7f0071ea0 2024-04-12T20:57:49.499 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:57:49.501+0000 7fe7f5000700 1 -- 172.21.15.18:0/245155134 --> [v2:172.21.15.18:6800/1605950634,v1:172.21.15.18:6801/1605950634] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fe7f0111b40 con 0x7fe7d406e670 2024-04-12T20:57:49.501 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:57:49.503+0000 7fe7e6000700 1 -- 172.21.15.18:0/245155134 <== mgr.14672 v2:172.21.15.18:6800/1605950634 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+329 (secure 0 0 0) 0x7fe7f0111b40 con 0x7fe7d406e670 2024-04-12T20:57:49.505 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:57:49.506+0000 7fe7f5000700 1 -- 172.21.15.18:0/245155134 >> [v2:172.21.15.18:6800/1605950634,v1:172.21.15.18:6801/1605950634] conn(0x7fe7d406e670 msgr2=0x7fe7d4070b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:57:49.505 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:57:49.506+0000 7fe7f5000700 1 --2- 172.21.15.18:0/245155134 >> [v2:172.21.15.18:6800/1605950634,v1:172.21.15.18:6801/1605950634] conn(0x7fe7d406e670 0x7fe7d4070b20 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fe7f0071be0 tx=0x7fe7e000b040 comp rx=0 tx=0).stop 2024-04-12T20:57:49.505 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:57:49.506+0000 7fe7f5000700 1 -- 172.21.15.18:0/245155134 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe7f0071ea0 msgr2=0x7fe7f0110150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:57:49.505 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:57:49.506+0000 7fe7f5000700 1 --2- 172.21.15.18:0/245155134 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe7f0071ea0 0x7fe7f0110150 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fe7dc00b970 tx=0x7fe7dc009e30 comp rx=0 tx=0).stop 2024-04-12T20:57:49.505 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:57:49.507+0000 7fe7f5000700 1 -- 172.21.15.18:0/245155134 shutdown_connections 2024-04-12T20:57:49.505 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:57:49.507+0000 7fe7f5000700 1 --2- 172.21.15.18:0/245155134 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe7f00708f0 0x7fe7f010fc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:57:49.505 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:57:49.507+0000 7fe7f5000700 1 --2- 172.21.15.18:0/245155134 >> [v2:172.21.15.18:6800/1605950634,v1:172.21.15.18:6801/1605950634] conn(0x7fe7d406e670 0x7fe7d4070b20 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:57:49.505 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:57:49.507+0000 7fe7f5000700 1 --2- 172.21.15.18:0/245155134 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe7f0071ea0 0x7fe7f0110150 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:57:49.505 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:57:49.507+0000 7fe7f5000700 1 -- 172.21.15.18:0/245155134 >> 172.21.15.18:0/245155134 conn(0x7fe7f00f8560 msgr2=0x7fe7f00f9c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T20:57:49.506 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:57:49.507+0000 7fe7f5000700 1 -- 172.21.15.18:0/245155134 shutdown_connections 2024-04-12T20:57:49.506 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:57:49.507+0000 7fe7f5000700 1 -- 172.21.15.18:0/245155134 wait complete. 2024-04-12T20:57:49.526 INFO:teuthology.orchestra.run.smithi018.stdout:true 2024-04-12T20:57:49.698 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:57:49.699+0000 7f948b200700 1 -- 172.21.15.18:0/1321709445 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f94840fce70 msgr2=0x7f94840fd280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:57:49.698 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:57:49.699+0000 7f948b200700 1 --2- 172.21.15.18:0/1321709445 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f94840fce70 0x7f94840fd280 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f9474004940 tx=0x7f9474009e30 comp rx=0 tx=0).stop 2024-04-12T20:57:49.698 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:57:49.699+0000 7f948b200700 1 -- 172.21.15.18:0/1321709445 shutdown_connections 2024-04-12T20:57:49.698 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:57:49.699+0000 7f948b200700 1 --2- 172.21.15.18:0/1321709445 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f94840fdfe0 0x7f94840fe450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:57:49.698 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:57:49.699+0000 7f948b200700 1 --2- 172.21.15.18:0/1321709445 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f94840fce70 0x7f94840fd280 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:57:49.698 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:57:49.699+0000 7f948b200700 1 -- 172.21.15.18:0/1321709445 >> 172.21.15.18:0/1321709445 conn(0x7f94840f8550 msgr2=0x7f94840fa960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T20:57:49.698 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:57:49.699+0000 7f948b200700 1 -- 172.21.15.18:0/1321709445 shutdown_connections 2024-04-12T20:57:49.698 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:57:49.700+0000 7f948b200700 1 -- 172.21.15.18:0/1321709445 wait complete. 2024-04-12T20:57:49.700 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:57:49.700+0000 7f948b200700 1 Processor -- start 2024-04-12T20:57:49.700 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:57:49.701+0000 7f948b200700 1 -- start start 2024-04-12T20:57:49.700 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:57:49.701+0000 7f948b200700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f94840fce70 0x7f948410d9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T20:57:49.700 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:57:49.701+0000 7f948b200700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f94840fdfe0 0x7f948410dee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T20:57:49.700 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:57:49.701+0000 7f948b200700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f948410e4e0 con 0x7f94840fdfe0 2024-04-12T20:57:49.700 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:57:49.701+0000 7f948b200700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f948410e650 con 0x7f94840fce70 2024-04-12T20:57:49.700 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:57:49.702+0000 7f9483400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f94840fdfe0 0x7f948410dee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T20:57:49.700 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:57:49.702+0000 7f9483400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f94840fdfe0 0x7f948410dee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:52106/0 (socket says 172.21.15.18:52106) 2024-04-12T20:57:49.700 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:57:49.702+0000 7f9483400700 1 -- 172.21.15.18:0/1130076559 learned_addr learned my addr 172.21.15.18:0/1130076559 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T20:57:49.700 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:57:49.702+0000 7f9483e00700 1 --2- 172.21.15.18:0/1130076559 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f94840fce70 0x7f948410d9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T20:57:49.701 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:57:49.702+0000 7f9483400700 1 -- 172.21.15.18:0/1130076559 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f94840fce70 msgr2=0x7f948410d9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:57:49.701 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:57:49.702+0000 7f9483400700 1 --2- 172.21.15.18:0/1130076559 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f94840fce70 0x7f948410d9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:57:49.701 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:57:49.702+0000 7f9483400700 1 -- 172.21.15.18:0/1130076559 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9474005040 con 0x7f94840fdfe0 2024-04-12T20:57:49.702 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:57:49.702+0000 7f9483400700 1 --2- 172.21.15.18:0/1130076559 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f94840fdfe0 0x7f948410dee0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f946c002a50 tx=0x7f946c0046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T20:57:49.702 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:57:49.703+0000 7f947be00700 1 -- 172.21.15.18:0/1130076559 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f946c007cc0 con 0x7f94840fdfe0 2024-04-12T20:57:49.702 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:57:49.703+0000 7f947be00700 1 -- 172.21.15.18:0/1130076559 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f946c00f040 con 0x7f94840fdfe0 2024-04-12T20:57:49.702 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:57:49.703+0000 7f948b200700 1 -- 172.21.15.18:0/1130076559 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f94841130b0 con 0x7f94840fdfe0 2024-04-12T20:57:49.702 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:57:49.703+0000 7f948b200700 1 -- 172.21.15.18:0/1130076559 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9484113670 con 0x7f94840fdfe0 2024-04-12T20:57:49.702 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:57:49.703+0000 7f947be00700 1 -- 172.21.15.18:0/1130076559 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f946c00daf0 con 0x7f94840fdfe0 2024-04-12T20:57:49.702 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:57:49.704+0000 7f948b200700 1 -- 172.21.15.18:0/1130076559 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f948404c710 con 0x7f94840fdfe0 2024-04-12T20:57:49.704 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:57:49.706+0000 7f947be00700 1 -- 172.21.15.18:0/1130076559 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 30) v1 ==== 92561+0+0 (secure 0 0 0) 0x7f946c00dc90 con 0x7f94840fdfe0 2024-04-12T20:57:49.705 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:57:49.707+0000 7f947be00700 1 --2- 172.21.15.18:0/1130076559 >> [v2:172.21.15.18:6800/1605950634,v1:172.21.15.18:6801/1605950634] conn(0x7f9468072a70 0x7f9468074f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T20:57:49.705 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:57:49.707+0000 7f947be00700 1 -- 172.21.15.18:0/1130076559 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6066+0+0 (secure 0 0 0) 0x7f946c08d7e0 con 0x7f94840fdfe0 2024-04-12T20:57:49.705 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:57:49.708+0000 7f9483e00700 1 --2- 172.21.15.18:0/1130076559 >> [v2:172.21.15.18:6800/1605950634,v1:172.21.15.18:6801/1605950634] conn(0x7f9468072a70 0x7f9468074f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T20:57:49.706 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:57:49.708+0000 7f9483e00700 1 --2- 172.21.15.18:0/1130076559 >> [v2:172.21.15.18:6800/1605950634,v1:172.21.15.18:6801/1605950634] conn(0x7f9468072a70 0x7f9468074f20 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f947400b3b0 tx=0x7f9474009e30 comp rx=0 tx=0).ready entity=mgr.14672 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T20:57:49.712 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:57:49.714+0000 7f947be00700 1 -- 172.21.15.18:0/1130076559 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186454 (secure 0 0 0) 0x7f946c05a720 con 0x7f94840fdfe0 2024-04-12T20:57:49.787 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:57:49 smithi018 ceph-mon[28799]: pgmap v21: 65 pgs: 65 active+clean; 282 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 28 KiB/s rd, 982 KiB/s wr, 283 op/s 2024-04-12T20:57:50.030 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:57:49 smithi186 ceph-mon[36597]: pgmap v21: 65 pgs: 65 active+clean; 282 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 28 KiB/s rd, 982 KiB/s wr, 283 op/s 2024-04-12T20:57:50.106 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:57:50.108+0000 7f948b200700 1 -- 172.21.15.18:0/1130076559 --> [v2:172.21.15.18:6800/1605950634,v1:172.21.15.18:6801/1605950634] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f94841023c0 con 0x7f9468072a70 2024-04-12T20:57:50.107 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:57:50.110+0000 7f947be00700 1 -- 172.21.15.18:0/1130076559 <== mgr.14672 v2:172.21.15.18:6800/1605950634 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+329 (secure 0 0 0) 0x7f94841023c0 con 0x7f9468072a70 2024-04-12T20:57:50.111 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:57:50.113+0000 7f948b200700 1 -- 172.21.15.18:0/1130076559 >> [v2:172.21.15.18:6800/1605950634,v1:172.21.15.18:6801/1605950634] conn(0x7f9468072a70 msgr2=0x7f9468074f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:57:50.111 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:57:50.113+0000 7f948b200700 1 --2- 172.21.15.18:0/1130076559 >> [v2:172.21.15.18:6800/1605950634,v1:172.21.15.18:6801/1605950634] conn(0x7f9468072a70 0x7f9468074f20 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f947400b3b0 tx=0x7f9474009e30 comp rx=0 tx=0).stop 2024-04-12T20:57:50.111 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:57:50.113+0000 7f948b200700 1 -- 172.21.15.18:0/1130076559 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f94840fdfe0 msgr2=0x7f948410dee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:57:50.111 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:57:50.113+0000 7f948b200700 1 --2- 172.21.15.18:0/1130076559 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f94840fdfe0 0x7f948410dee0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f946c002a50 tx=0x7f946c0046b0 comp rx=0 tx=0).stop 2024-04-12T20:57:50.112 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:57:50.113+0000 7f948b200700 1 -- 172.21.15.18:0/1130076559 shutdown_connections 2024-04-12T20:57:50.112 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:57:50.113+0000 7f948b200700 1 --2- 172.21.15.18:0/1130076559 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f94840fce70 0x7f948410d9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:57:50.112 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:57:50.113+0000 7f948b200700 1 --2- 172.21.15.18:0/1130076559 >> [v2:172.21.15.18:6800/1605950634,v1:172.21.15.18:6801/1605950634] conn(0x7f9468072a70 0x7f9468074f20 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:57:50.112 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:57:50.113+0000 7f948b200700 1 --2- 172.21.15.18:0/1130076559 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f94840fdfe0 0x7f948410dee0 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:57:50.112 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:57:50.113+0000 7f948b200700 1 -- 172.21.15.18:0/1130076559 >> 172.21.15.18:0/1130076559 conn(0x7f94840f8550 msgr2=0x7f94840f9c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T20:57:50.112 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:57:50.113+0000 7f948b200700 1 -- 172.21.15.18:0/1130076559 shutdown_connections 2024-04-12T20:57:50.112 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:57:50.113+0000 7f948b200700 1 -- 172.21.15.18:0/1130076559 wait complete. 2024-04-12T20:57:50.291 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:57:50.293+0000 7fe1bf400700 1 -- 172.21.15.18:0/1396747613 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe1b80fbcd0 msgr2=0x7fe1b80fc0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:57:50.291 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:57:50.293+0000 7fe1bf400700 1 --2- 172.21.15.18:0/1396747613 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe1b80fbcd0 0x7fe1b80fc0e0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fe1b0009ce0 tx=0x7fe1b0004d10 comp rx=0 tx=0).stop 2024-04-12T20:57:50.292 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:57:50.294+0000 7fe1bf400700 1 -- 172.21.15.18:0/1396747613 shutdown_connections 2024-04-12T20:57:50.292 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:57:50.294+0000 7fe1bf400700 1 --2- 172.21.15.18:0/1396747613 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe1b80fcf80 0x7fe1b80fd3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:57:50.292 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:57:50.294+0000 7fe1bf400700 1 --2- 172.21.15.18:0/1396747613 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe1b80fbcd0 0x7fe1b80fc0e0 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:57:50.292 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:57:50.294+0000 7fe1bf400700 1 -- 172.21.15.18:0/1396747613 >> 172.21.15.18:0/1396747613 conn(0x7fe1b80f7430 msgr2=0x7fe1b80f9860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T20:57:50.292 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:57:50.294+0000 7fe1bf400700 1 -- 172.21.15.18:0/1396747613 shutdown_connections 2024-04-12T20:57:50.292 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:57:50.294+0000 7fe1bf400700 1 -- 172.21.15.18:0/1396747613 wait complete. 2024-04-12T20:57:50.293 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:57:50.295+0000 7fe1bf400700 1 Processor -- start 2024-04-12T20:57:50.293 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:57:50.295+0000 7fe1bf400700 1 -- start start 2024-04-12T20:57:50.293 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:57:50.296+0000 7fe1bf400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe1b80fbcd0 0x7fe1b810fe90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T20:57:50.294 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:57:50.296+0000 7fe1bf400700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe1b80fcf80 0x7fe1b81103d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T20:57:50.294 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:57:50.296+0000 7fe1bf400700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1b8111140 con 0x7fe1b80fbcd0 2024-04-12T20:57:50.294 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:57:50.296+0000 7fe1bf400700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1b81112b0 con 0x7fe1b80fcf80 2024-04-12T20:57:50.294 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:57:50.296+0000 7fe1bca00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe1b80fbcd0 0x7fe1b810fe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T20:57:50.295 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:57:50.296+0000 7fe1bca00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe1b80fbcd0 0x7fe1b810fe90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:52126/0 (socket says 172.21.15.18:52126) 2024-04-12T20:57:50.295 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:57:50.296+0000 7fe1bca00700 1 -- 172.21.15.18:0/2607211737 learned_addr learned my addr 172.21.15.18:0/2607211737 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T20:57:50.295 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:57:50.297+0000 7fe1b7e00700 1 --2- 172.21.15.18:0/2607211737 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe1b80fcf80 0x7fe1b81103d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T20:57:50.295 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:57:50.297+0000 7fe1bca00700 1 -- 172.21.15.18:0/2607211737 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe1b80fcf80 msgr2=0x7fe1b81103d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:57:50.295 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:57:50.297+0000 7fe1bca00700 1 --2- 172.21.15.18:0/2607211737 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe1b80fcf80 0x7fe1b81103d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:57:50.295 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:57:50.297+0000 7fe1bca00700 1 -- 172.21.15.18:0/2607211737 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1b0009940 con 0x7fe1b80fbcd0 2024-04-12T20:57:50.295 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:57:50.297+0000 7fe1bca00700 1 --2- 172.21.15.18:0/2607211737 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe1b80fbcd0 0x7fe1b810fe90 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fe1b0005010 tx=0x7fe1b0004d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T20:57:50.296 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:57:50.298+0000 7fe1b4c00700 1 -- 172.21.15.18:0/2607211737 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe1b000e040 con 0x7fe1b80fbcd0 2024-04-12T20:57:50.296 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:57:50.298+0000 7fe1b4c00700 1 -- 172.21.15.18:0/2607211737 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe1b0008ed0 con 0x7fe1b80fbcd0 2024-04-12T20:57:50.296 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:57:50.298+0000 7fe1b4c00700 1 -- 172.21.15.18:0/2607211737 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe1b001c870 con 0x7fe1b80fbcd0 2024-04-12T20:57:50.297 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:57:50.298+0000 7fe1bf400700 1 -- 172.21.15.18:0/2607211737 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe1b81114d0 con 0x7fe1b80fbcd0 2024-04-12T20:57:50.297 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:57:50.298+0000 7fe1bf400700 1 -- 172.21.15.18:0/2607211737 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe1b8111a90 con 0x7fe1b80fbcd0 2024-04-12T20:57:50.298 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:57:50.300+0000 7fe1bf400700 1 -- 172.21.15.18:0/2607211737 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe1b8101910 con 0x7fe1b80fbcd0 2024-04-12T20:57:50.302 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:57:50.302+0000 7fe1b4c00700 1 -- 172.21.15.18:0/2607211737 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 30) v1 ==== 92561+0+0 (secure 0 0 0) 0x7fe1b0012c70 con 0x7fe1b80fbcd0 2024-04-12T20:57:50.302 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:57:50.303+0000 7fe1b4c00700 1 --2- 172.21.15.18:0/2607211737 >> [v2:172.21.15.18:6800/1605950634,v1:172.21.15.18:6801/1605950634] conn(0x7fe19c06e7b0 0x7fe19c070c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T20:57:50.302 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:57:50.304+0000 7fe1b4c00700 1 -- 172.21.15.18:0/2607211737 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6066+0+0 (secure 0 0 0) 0x7fe1b008ff30 con 0x7fe1b80fbcd0 2024-04-12T20:57:50.302 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:57:50.304+0000 7fe1b7e00700 1 --2- 172.21.15.18:0/2607211737 >> [v2:172.21.15.18:6800/1605950634,v1:172.21.15.18:6801/1605950634] conn(0x7fe19c06e7b0 0x7fe19c070c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T20:57:50.302 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:57:50.305+0000 7fe1b7e00700 1 --2- 172.21.15.18:0/2607211737 >> [v2:172.21.15.18:6800/1605950634,v1:172.21.15.18:6801/1605950634] conn(0x7fe19c06e7b0 0x7fe19c070c60 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fe1a8004d60 tx=0x7fe1a800b040 comp rx=0 tx=0).ready entity=mgr.14672 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T20:57:50.307 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:57:50.309+0000 7fe1b4c00700 1 -- 172.21.15.18:0/2607211737 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186454 (secure 0 0 0) 0x7fe1b00593c0 con 0x7fe1b80fbcd0 2024-04-12T20:57:50.648 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:57:50.650+0000 7fe1bf400700 1 -- 172.21.15.18:0/2607211737 --> [v2:172.21.15.18:6800/1605950634,v1:172.21.15.18:6801/1605950634] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fe1b81048a0 con 0x7fe19c06e7b0 2024-04-12T20:57:50.664 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:57:50.666+0000 7fe1b4c00700 1 -- 172.21.15.18:0/2607211737 <== mgr.14672 v2:172.21.15.18:6800/1605950634 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7fe1b81048a0 con 0x7fe19c06e7b0 2024-04-12T20:57:50.664 INFO:teuthology.orchestra.run.smithi018.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-12T20:57:50.664 INFO:teuthology.orchestra.run.smithi018.stdout:alertmanager.smithi018 smithi018 *:9093,9094 running (2m) 29s ago 12m 15.2M - 0.25.0 c8568f914cd2 b4ed63d191d4 2024-04-12T20:57:50.664 INFO:teuthology.orchestra.run.smithi018.stdout:crash.smithi018 smithi018 running (13m) 29s ago 13m 6660k - 17.2.7-827-gd3b5060a 21de9b7369ad 1ac7603bddfd 2024-04-12T20:57:50.664 INFO:teuthology.orchestra.run.smithi018.stdout:crash.smithi186 smithi186 running (11m) 32s ago 11m 6669k - 17.2.7-827-gd3b5060a 21de9b7369ad 14bf8a1672eb 2024-04-12T20:57:50.664 INFO:teuthology.orchestra.run.smithi018.stdout:grafana.smithi018 smithi018 *:3000 running (2m) 29s ago 12m 75.6M - 9.4.7 954c08fa6188 d6251dcdaec0 2024-04-12T20:57:50.664 INFO:teuthology.orchestra.run.smithi018.stdout:mds.cephfs.smithi018.acqbsc smithi018 running (5m) 29s ago 5m 143M - 17.2.7-827-gd3b5060a 21de9b7369ad 5b5fce65f18a 2024-04-12T20:57:50.664 INFO:teuthology.orchestra.run.smithi018.stdout:mds.cephfs.smithi018.szjluo smithi018 running (5m) 29s ago 5m 145M - 17.2.7-827-gd3b5060a 21de9b7369ad cf9bf0c5031f 2024-04-12T20:57:50.665 INFO:teuthology.orchestra.run.smithi018.stdout:mds.cephfs.smithi186.jdllyn smithi186 running (5m) 32s ago 5m 12.5M - 17.2.7-827-gd3b5060a 21de9b7369ad 2b2bc46bd29a 2024-04-12T20:57:50.665 INFO:teuthology.orchestra.run.smithi018.stdout:mds.cephfs.smithi186.wjnafs smithi186 running (5m) 32s ago 5m 12.6M - 17.2.7-827-gd3b5060a 21de9b7369ad 74b115d6d6fd 2024-04-12T20:57:50.665 INFO:teuthology.orchestra.run.smithi018.stdout:mgr.smithi018.xjzwga smithi018 *:8443,9283,8765 running (67s) 29s ago 14m 478M - 19.0.0-2879-g324baf8d 8717720dc5e5 56088305eab7 2024-04-12T20:57:50.665 INFO:teuthology.orchestra.run.smithi018.stdout:mgr.smithi186.kqcfxk smithi186 *:8443,9283 running (3m) 32s ago 11m 75.1M - 19.0.0-2879-g324baf8d 8717720dc5e5 bd4971d5260e 2024-04-12T20:57:50.665 INFO:teuthology.orchestra.run.smithi018.stdout:mon.smithi018 smithi018 running (14m) 29s ago 14m 52.6M 2048M 17.2.7-827-gd3b5060a 21de9b7369ad cd84caafa056 2024-04-12T20:57:50.665 INFO:teuthology.orchestra.run.smithi018.stdout:mon.smithi186 smithi186 running (11m) 32s ago 11m 41.4M 2048M 17.2.7-827-gd3b5060a 21de9b7369ad d7263c772b73 2024-04-12T20:57:50.665 INFO:teuthology.orchestra.run.smithi018.stdout:node-exporter.smithi018 smithi018 *:9100 running (2m) 29s ago 13m 7243k - 1.5.0 0da6a335fe13 2d717b2cd3c2 2024-04-12T20:57:50.665 INFO:teuthology.orchestra.run.smithi018.stdout:node-exporter.smithi186 smithi186 *:9100 running (78s) 32s ago 11m 7822k - 1.5.0 0da6a335fe13 4ac3111e3ae2 2024-04-12T20:57:50.665 INFO:teuthology.orchestra.run.smithi018.stdout:osd.0 smithi018 running (10m) 29s ago 10m 193M 1559M 17.2.7-827-gd3b5060a 21de9b7369ad 0d83e027bf0a 2024-04-12T20:57:50.665 INFO:teuthology.orchestra.run.smithi018.stdout:osd.1 smithi018 running (9m) 29s ago 9m 221M 1559M 17.2.7-827-gd3b5060a 21de9b7369ad 6517c9cc7f87 2024-04-12T20:57:50.665 INFO:teuthology.orchestra.run.smithi018.stdout:osd.2 smithi018 running (9m) 29s ago 9m 168M 1559M 17.2.7-827-gd3b5060a 21de9b7369ad 8a83692acaa5 2024-04-12T20:57:50.665 INFO:teuthology.orchestra.run.smithi018.stdout:osd.3 smithi186 running (8m) 32s ago 8m 231M 2581M 17.2.7-827-gd3b5060a 21de9b7369ad c2def7a90d76 2024-04-12T20:57:50.665 INFO:teuthology.orchestra.run.smithi018.stdout:osd.4 smithi186 running (8m) 32s ago 8m 203M 2581M 17.2.7-827-gd3b5060a 21de9b7369ad 757e7c71b7b9 2024-04-12T20:57:50.665 INFO:teuthology.orchestra.run.smithi018.stdout:osd.5 smithi186 running (7m) 32s ago 7m 209M 2581M 17.2.7-827-gd3b5060a 21de9b7369ad 2ee072a436d0 2024-04-12T20:57:50.666 INFO:teuthology.orchestra.run.smithi018.stdout:prometheus.smithi018 smithi018 *:9095 running (87s) 29s ago 12m 44.5M - 2.43.0 a07b618ecd1d 34ccc01df66b 2024-04-12T20:57:50.667 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:57:50.669+0000 7fe1bf400700 1 -- 172.21.15.18:0/2607211737 >> [v2:172.21.15.18:6800/1605950634,v1:172.21.15.18:6801/1605950634] conn(0x7fe19c06e7b0 msgr2=0x7fe19c070c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:57:50.667 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:57:50.669+0000 7fe1bf400700 1 --2- 172.21.15.18:0/2607211737 >> [v2:172.21.15.18:6800/1605950634,v1:172.21.15.18:6801/1605950634] conn(0x7fe19c06e7b0 0x7fe19c070c60 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fe1a8004d60 tx=0x7fe1a800b040 comp rx=0 tx=0).stop 2024-04-12T20:57:50.667 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:57:50.669+0000 7fe1bf400700 1 -- 172.21.15.18:0/2607211737 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe1b80fbcd0 msgr2=0x7fe1b810fe90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:57:50.667 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:57:50.669+0000 7fe1bf400700 1 --2- 172.21.15.18:0/2607211737 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe1b80fbcd0 0x7fe1b810fe90 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fe1b0005010 tx=0x7fe1b0004d10 comp rx=0 tx=0).stop 2024-04-12T20:57:50.668 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:57:50.670+0000 7fe1bf400700 1 -- 172.21.15.18:0/2607211737 shutdown_connections 2024-04-12T20:57:50.668 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:57:50.670+0000 7fe1bf400700 1 --2- 172.21.15.18:0/2607211737 >> [v2:172.21.15.18:6800/1605950634,v1:172.21.15.18:6801/1605950634] conn(0x7fe19c06e7b0 0x7fe19c070c60 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:57:50.668 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:57:50.670+0000 7fe1bf400700 1 --2- 172.21.15.18:0/2607211737 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe1b80fbcd0 0x7fe1b810fe90 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:57:50.668 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:57:50.670+0000 7fe1bf400700 1 --2- 172.21.15.18:0/2607211737 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe1b80fcf80 0x7fe1b81103d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:57:50.668 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:57:50.670+0000 7fe1bf400700 1 -- 172.21.15.18:0/2607211737 >> 172.21.15.18:0/2607211737 conn(0x7fe1b80f7430 msgr2=0x7fe1b8065c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T20:57:50.668 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:57:50.670+0000 7fe1bf400700 1 -- 172.21.15.18:0/2607211737 shutdown_connections 2024-04-12T20:57:50.668 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:57:50.670+0000 7fe1bf400700 1 -- 172.21.15.18:0/2607211737 wait complete. 2024-04-12T20:57:50.748 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:57:50 smithi018 ceph-mon[28799]: from='client.14724 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T20:57:50.843 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:57:50.845+0000 7f41f2a00700 1 -- 172.21.15.18:0/2185212110 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f41ec1002a0 msgr2=0x7f41ec100710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:57:50.844 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:57:50.845+0000 7f41f2a00700 1 --2- 172.21.15.18:0/2185212110 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f41ec1002a0 0x7f41ec100710 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f41d8004940 tx=0x7f41d8009e30 comp rx=0 tx=0).stop 2024-04-12T20:57:50.844 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:57:50.845+0000 7f41f2a00700 1 -- 172.21.15.18:0/2185212110 shutdown_connections 2024-04-12T20:57:50.844 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:57:50.845+0000 7f41f2a00700 1 --2- 172.21.15.18:0/2185212110 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f41ec1002a0 0x7f41ec100710 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:57:50.844 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:57:50.845+0000 7f41f2a00700 1 --2- 172.21.15.18:0/2185212110 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f41ec0feff0 0x7f41ec0ff400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:57:50.844 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:57:50.845+0000 7f41f2a00700 1 -- 172.21.15.18:0/2185212110 >> 172.21.15.18:0/2185212110 conn(0x7f41ec0fa950 msgr2=0x7f41ec0fcd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T20:57:50.844 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:57:50.846+0000 7f41f2a00700 1 -- 172.21.15.18:0/2185212110 shutdown_connections 2024-04-12T20:57:50.844 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:57:50.846+0000 7f41f2a00700 1 -- 172.21.15.18:0/2185212110 wait complete. 2024-04-12T20:57:50.845 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:57:50.846+0000 7f41f2a00700 1 Processor -- start 2024-04-12T20:57:50.845 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:57:50.847+0000 7f41f2a00700 1 -- start start 2024-04-12T20:57:50.845 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:57:50.847+0000 7f41f2a00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f41ec0feff0 0x7f41ec10fc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T20:57:50.845 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:57:50.847+0000 7f41f2a00700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f41ec1002a0 0x7f41ec1101c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T20:57:50.845 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:57:50.847+0000 7f41f2a00700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41ec110850 con 0x7f41ec0feff0 2024-04-12T20:57:50.845 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:57:50.847+0000 7f41f2a00700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41ec110ff0 con 0x7f41ec1002a0 2024-04-12T20:57:50.846 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:57:50.847+0000 7f41ebe00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f41ec0feff0 0x7f41ec10fc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T20:57:50.846 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:57:50.848+0000 7f41ebe00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f41ec0feff0 0x7f41ec10fc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:50132/0 (socket says 172.21.15.18:50132) 2024-04-12T20:57:50.846 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:57:50.848+0000 7f41ebe00700 1 -- 172.21.15.18:0/1948640541 learned_addr learned my addr 172.21.15.18:0/1948640541 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T20:57:50.846 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:57:50.848+0000 7f41ebe00700 1 -- 172.21.15.18:0/1948640541 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f41ec1002a0 msgr2=0x7f41ec1101c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:57:50.846 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:57:50.848+0000 7f41ebe00700 1 --2- 172.21.15.18:0/1948640541 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f41ec1002a0 0x7f41ec1101c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:57:50.847 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:57:50.848+0000 7f41ebe00700 1 -- 172.21.15.18:0/1948640541 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41d8005040 con 0x7f41ec0feff0 2024-04-12T20:57:50.847 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:57:50.849+0000 7f41ebe00700 1 --2- 172.21.15.18:0/1948640541 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f41ec0feff0 0x7f41ec10fc80 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f41dc00b790 tx=0x7f41dc009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T20:57:50.847 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:57:50.849+0000 7f41e3e00700 1 -- 172.21.15.18:0/1948640541 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f41dc00f770 con 0x7f41ec0feff0 2024-04-12T20:57:50.847 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:57:50.849+0000 7f41e3e00700 1 -- 172.21.15.18:0/1948640541 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f41dc00fda0 con 0x7f41ec0feff0 2024-04-12T20:57:50.847 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:57:50.849+0000 7f41e3e00700 1 -- 172.21.15.18:0/1948640541 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f41dc00d310 con 0x7f41ec0feff0 2024-04-12T20:57:50.847 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:57:50.849+0000 7f41f2a00700 1 -- 172.21.15.18:0/1948640541 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f41ec1112d0 con 0x7f41ec0feff0 2024-04-12T20:57:50.847 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:57:50.849+0000 7f41f2a00700 1 -- 172.21.15.18:0/1948640541 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f41ec111890 con 0x7f41ec0feff0 2024-04-12T20:57:50.848 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:57:50.851+0000 7f41f2a00700 1 -- 172.21.15.18:0/1948640541 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f41ec05f5f0 con 0x7f41ec0feff0 2024-04-12T20:57:50.850 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:57:50.852+0000 7f41e3e00700 1 -- 172.21.15.18:0/1948640541 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 30) v1 ==== 92561+0+0 (secure 0 0 0) 0x7f41dc010420 con 0x7f41ec0feff0 2024-04-12T20:57:50.852 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:57:50.854+0000 7f41e3e00700 1 --2- 172.21.15.18:0/1948640541 >> [v2:172.21.15.18:6800/1605950634,v1:172.21.15.18:6801/1605950634] conn(0x7f41d006e7b0 0x7f41d0070c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T20:57:50.852 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:57:50.855+0000 7f41e3e00700 1 -- 172.21.15.18:0/1948640541 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6066+0+0 (secure 0 0 0) 0x7f41dc08e030 con 0x7f41ec0feff0 2024-04-12T20:57:50.853 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:57:50.855+0000 7f41eb400700 1 --2- 172.21.15.18:0/1948640541 >> [v2:172.21.15.18:6800/1605950634,v1:172.21.15.18:6801/1605950634] conn(0x7f41d006e7b0 0x7f41d0070c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T20:57:50.854 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:57:50.856+0000 7f41eb400700 1 --2- 172.21.15.18:0/1948640541 >> [v2:172.21.15.18:6800/1605950634,v1:172.21.15.18:6801/1605950634] conn(0x7f41d006e7b0 0x7f41d0070c60 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f41d8004f90 tx=0x7f41d8009e30 comp rx=0 tx=0).ready entity=mgr.14672 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T20:57:50.858 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:57:50.861+0000 7f41e3e00700 1 -- 172.21.15.18:0/1948640541 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186454 (secure 0 0 0) 0x7f41dc0c2870 con 0x7f41ec0feff0 2024-04-12T20:57:51.030 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:57:50 smithi186 ceph-mon[36597]: from='client.14724 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T20:57:51.350 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:57:51.352+0000 7f41f2a00700 1 -- 172.21.15.18:0/1948640541 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f41ec107b60 con 0x7f41ec0feff0 2024-04-12T20:57:51.351 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:57:51.352+0000 7f41e3e00700 1 -- 172.21.15.18:0/1948640541 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+739 (secure 0 0 0) 0x7f41dc05b100 con 0x7f41ec0feff0 2024-04-12T20:57:51.351 INFO:teuthology.orchestra.run.smithi018.stdout:{ 2024-04-12T20:57:51.351 INFO:teuthology.orchestra.run.smithi018.stdout: "mon": { 2024-04-12T20:57:51.351 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 17.2.7-827-gd3b5060a (d3b5060a17449edfae9385cdf792afec109fca5d) quincy (stable)": 2 2024-04-12T20:57:51.352 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-12T20:57:51.352 INFO:teuthology.orchestra.run.smithi018.stdout: "mgr": { 2024-04-12T20:57:51.352 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 19.0.0-2879-g324baf8d (324baf8ddb91dea46cac50cb535c306115791355) squid (dev)": 2 2024-04-12T20:57:51.352 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-12T20:57:51.352 INFO:teuthology.orchestra.run.smithi018.stdout: "osd": { 2024-04-12T20:57:51.352 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 17.2.7-827-gd3b5060a (d3b5060a17449edfae9385cdf792afec109fca5d) quincy (stable)": 6 2024-04-12T20:57:51.352 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-12T20:57:51.352 INFO:teuthology.orchestra.run.smithi018.stdout: "mds": { 2024-04-12T20:57:51.352 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 17.2.7-827-gd3b5060a (d3b5060a17449edfae9385cdf792afec109fca5d) quincy (stable)": 4 2024-04-12T20:57:51.352 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-12T20:57:51.352 INFO:teuthology.orchestra.run.smithi018.stdout: "overall": { 2024-04-12T20:57:51.352 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 17.2.7-827-gd3b5060a (d3b5060a17449edfae9385cdf792afec109fca5d) quincy (stable)": 12, 2024-04-12T20:57:51.352 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 19.0.0-2879-g324baf8d (324baf8ddb91dea46cac50cb535c306115791355) squid (dev)": 2 2024-04-12T20:57:51.353 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-04-12T20:57:51.353 INFO:teuthology.orchestra.run.smithi018.stdout:} 2024-04-12T20:57:51.353 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:57:51.355+0000 7f41f2a00700 1 -- 172.21.15.18:0/1948640541 >> [v2:172.21.15.18:6800/1605950634,v1:172.21.15.18:6801/1605950634] conn(0x7f41d006e7b0 msgr2=0x7f41d0070c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:57:51.353 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:57:51.355+0000 7f41f2a00700 1 --2- 172.21.15.18:0/1948640541 >> [v2:172.21.15.18:6800/1605950634,v1:172.21.15.18:6801/1605950634] conn(0x7f41d006e7b0 0x7f41d0070c60 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f41d8004f90 tx=0x7f41d8009e30 comp rx=0 tx=0).stop 2024-04-12T20:57:51.353 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:57:51.355+0000 7f41f2a00700 1 -- 172.21.15.18:0/1948640541 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f41ec0feff0 msgr2=0x7f41ec10fc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:57:51.353 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:57:51.355+0000 7f41f2a00700 1 --2- 172.21.15.18:0/1948640541 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f41ec0feff0 0x7f41ec10fc80 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f41dc00b790 tx=0x7f41dc009250 comp rx=0 tx=0).stop 2024-04-12T20:57:51.354 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:57:51.355+0000 7f41f2a00700 1 -- 172.21.15.18:0/1948640541 shutdown_connections 2024-04-12T20:57:51.354 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:57:51.355+0000 7f41f2a00700 1 --2- 172.21.15.18:0/1948640541 >> [v2:172.21.15.18:6800/1605950634,v1:172.21.15.18:6801/1605950634] conn(0x7f41d006e7b0 0x7f41d0070c60 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:57:51.354 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:57:51.355+0000 7f41f2a00700 1 --2- 172.21.15.18:0/1948640541 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f41ec0feff0 0x7f41ec10fc80 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:57:51.354 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:57:51.355+0000 7f41f2a00700 1 --2- 172.21.15.18:0/1948640541 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f41ec1002a0 0x7f41ec1101c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:57:51.354 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:57:51.355+0000 7f41f2a00700 1 -- 172.21.15.18:0/1948640541 >> 172.21.15.18:0/1948640541 conn(0x7f41ec0fa950 msgr2=0x7f41ec0fcf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T20:57:51.354 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:57:51.356+0000 7f41f2a00700 1 -- 172.21.15.18:0/1948640541 shutdown_connections 2024-04-12T20:57:51.354 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:57:51.356+0000 7f41f2a00700 1 -- 172.21.15.18:0/1948640541 wait complete. 2024-04-12T20:57:51.555 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:57:51.557+0000 7ff3e8800700 1 -- 172.21.15.18:0/2651000244 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff3e006f9d0 msgr2=0x7ff3e006e460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:57:51.555 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:57:51.557+0000 7ff3e8800700 1 --2- 172.21.15.18:0/2651000244 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff3e006f9d0 0x7ff3e006e460 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7ff3d4004880 tx=0x7ff3d4009e30 comp rx=0 tx=0).stop 2024-04-12T20:57:51.555 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:57:51.557+0000 7ff3e8800700 1 -- 172.21.15.18:0/2651000244 shutdown_connections 2024-04-12T20:57:51.555 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:57:51.557+0000 7ff3e8800700 1 --2- 172.21.15.18:0/2651000244 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff3e006e9a0 0x7ff3e006ee10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:57:51.555 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:57:51.557+0000 7ff3e8800700 1 --2- 172.21.15.18:0/2651000244 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff3e006f9d0 0x7ff3e006e460 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:57:51.555 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:57:51.557+0000 7ff3e8800700 1 -- 172.21.15.18:0/2651000244 >> 172.21.15.18:0/2651000244 conn(0x7ff3e0069de0 msgr2=0x7ff3e006c210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T20:57:51.556 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:57:51.557+0000 7ff3e8800700 1 -- 172.21.15.18:0/2651000244 shutdown_connections 2024-04-12T20:57:51.556 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:57:51.557+0000 7ff3e8800700 1 -- 172.21.15.18:0/2651000244 wait complete. 2024-04-12T20:57:51.557 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:57:51.558+0000 7ff3e8800700 1 Processor -- start 2024-04-12T20:57:51.557 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:57:51.559+0000 7ff3e8800700 1 -- start start 2024-04-12T20:57:51.558 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:57:51.560+0000 7ff3e8800700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff3e006e9a0 0x7ff3e0118340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T20:57:51.558 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:57:51.560+0000 7ff3e8800700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff3e006f9d0 0x7ff3e0118880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T20:57:51.558 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:57:51.560+0000 7ff3e8800700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3e0118e80 con 0x7ff3e006e9a0 2024-04-12T20:57:51.558 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:57:51.560+0000 7ff3e8800700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3e0118ff0 con 0x7ff3e006f9d0 2024-04-12T20:57:51.558 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:57:51.561+0000 7ff3e5e00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff3e006e9a0 0x7ff3e0118340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T20:57:51.560 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:57:51.561+0000 7ff3e5e00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff3e006e9a0 0x7ff3e0118340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:50150/0 (socket says 172.21.15.18:50150) 2024-04-12T20:57:51.560 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:57:51.561+0000 7ff3e5e00700 1 -- 172.21.15.18:0/943749989 learned_addr learned my addr 172.21.15.18:0/943749989 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T20:57:51.560 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:57:51.561+0000 7ff3e5400700 1 --2- 172.21.15.18:0/943749989 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff3e006f9d0 0x7ff3e0118880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T20:57:51.560 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:57:51.562+0000 7ff3e5400700 1 -- 172.21.15.18:0/943749989 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff3e006e9a0 msgr2=0x7ff3e0118340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:57:51.560 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:57:51.562+0000 7ff3e5400700 1 --2- 172.21.15.18:0/943749989 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff3e006e9a0 0x7ff3e0118340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:57:51.560 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:57:51.562+0000 7ff3e5400700 1 -- 172.21.15.18:0/943749989 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3d4005040 con 0x7ff3e006f9d0 2024-04-12T20:57:51.560 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:57:51.562+0000 7ff3e5400700 1 --2- 172.21.15.18:0/943749989 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff3e006f9d0 0x7ff3e0118880 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7ff3d0009fd0 tx=0x7ff3d000b040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T20:57:51.561 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:57:51.563+0000 7ff3da000700 1 -- 172.21.15.18:0/943749989 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff3d0007c90 con 0x7ff3e006f9d0 2024-04-12T20:57:51.561 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:57:51.563+0000 7ff3da000700 1 -- 172.21.15.18:0/943749989 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff3d0007df0 con 0x7ff3e006f9d0 2024-04-12T20:57:51.561 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:57:51.563+0000 7ff3da000700 1 -- 172.21.15.18:0/943749989 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff3d001da10 con 0x7ff3e006f9d0 2024-04-12T20:57:51.562 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:57:51.564+0000 7ff3e8800700 1 -- 172.21.15.18:0/943749989 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff3e010db50 con 0x7ff3e006f9d0 2024-04-12T20:57:51.563 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:57:51.565+0000 7ff3e8800700 1 -- 172.21.15.18:0/943749989 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff3e010e0e0 con 0x7ff3e006f9d0 2024-04-12T20:57:51.567 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:57:51.567+0000 7ff3e8800700 1 -- 172.21.15.18:0/943749989 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff3e004c710 con 0x7ff3e006f9d0 2024-04-12T20:57:51.568 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:57:51.570+0000 7ff3da000700 1 -- 172.21.15.18:0/943749989 <== mon.1 v2:172.21.15.186:3300/0 4 ==== mgrmap(e 30) v1 ==== 92561+0+0 (secure 0 0 0) 0x7ff3d001db70 con 0x7ff3e006f9d0 2024-04-12T20:57:51.571 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:57:51.573+0000 7ff3da000700 1 --2- 172.21.15.18:0/943749989 >> [v2:172.21.15.18:6800/1605950634,v1:172.21.15.18:6801/1605950634] conn(0x7ff3c806e7b0 0x7ff3c8070c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T20:57:51.571 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:57:51.573+0000 7ff3da000700 1 -- 172.21.15.18:0/943749989 <== mon.1 v2:172.21.15.186:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6066+0+0 (secure 0 0 0) 0x7ff3d0096b80 con 0x7ff3e006f9d0 2024-04-12T20:57:51.571 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:57:51.574+0000 7ff3e5e00700 1 --2- 172.21.15.18:0/943749989 >> [v2:172.21.15.18:6800/1605950634,v1:172.21.15.18:6801/1605950634] conn(0x7ff3c806e7b0 0x7ff3c8070c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T20:57:51.574 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:57:51.576+0000 7ff3e5e00700 1 --2- 172.21.15.18:0/943749989 >> [v2:172.21.15.18:6800/1605950634,v1:172.21.15.18:6801/1605950634] conn(0x7ff3c806e7b0 0x7ff3c8070c60 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7ff3d4009fd0 tx=0x7ff3d4009e30 comp rx=0 tx=0).ready entity=mgr.14672 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T20:57:51.575 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:57:51.577+0000 7ff3da000700 1 -- 172.21.15.18:0/943749989 <== mon.1 v2:172.21.15.186:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186454 (secure 0 0 0) 0x7ff3d0063b40 con 0x7ff3e006f9d0 2024-04-12T20:57:51.851 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:57:51 smithi018 ceph-mon[28799]: from='client.14728 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T20:57:51.852 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:57:51 smithi018 ceph-mon[28799]: pgmap v22: 65 pgs: 65 active+clean; 279 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 43 KiB/s rd, 1.5 MiB/s wr, 431 op/s 2024-04-12T20:57:51.852 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:57:51 smithi018 ceph-mon[28799]: from='client.14732 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T20:57:51.852 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:57:51 smithi018 ceph-mon[28799]: from='client.? 172.21.15.18:0/1948640541' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T20:57:51.920 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:57:51.922+0000 7ff3e8800700 1 -- 172.21.15.18:0/943749989 --> [v2:172.21.15.18:6800/1605950634,v1:172.21.15.18:6801/1605950634] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7ff3e0109b20 con 0x7ff3c806e7b0 2024-04-12T20:57:51.921 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:57:51.923+0000 7ff3da000700 1 -- 172.21.15.18:0/943749989 <== mgr.14672 v2:172.21.15.18:6800/1605950634 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+329 (secure 0 0 0) 0x7ff3e0109b20 con 0x7ff3c806e7b0 2024-04-12T20:57:51.923 INFO:teuthology.orchestra.run.smithi018.stdout:{ 2024-04-12T20:57:51.923 INFO:teuthology.orchestra.run.smithi018.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:f4452def3be4d350f4a8e5feaf7ebb1511be9298fe68478ba7a2d6fbd7d69a63", 2024-04-12T20:57:51.924 INFO:teuthology.orchestra.run.smithi018.stdout: "in_progress": true, 2024-04-12T20:57:51.924 INFO:teuthology.orchestra.run.smithi018.stdout: "which": "Upgrading daemons of type(s) mgr", 2024-04-12T20:57:51.924 INFO:teuthology.orchestra.run.smithi018.stdout: "services_complete": [ 2024-04-12T20:57:51.924 INFO:teuthology.orchestra.run.smithi018.stdout: "mgr" 2024-04-12T20:57:51.924 INFO:teuthology.orchestra.run.smithi018.stdout: ], 2024-04-12T20:57:51.924 INFO:teuthology.orchestra.run.smithi018.stdout: "progress": "2/2 daemons upgraded", 2024-04-12T20:57:51.924 INFO:teuthology.orchestra.run.smithi018.stdout: "message": "", 2024-04-12T20:57:51.924 INFO:teuthology.orchestra.run.smithi018.stdout: "is_paused": false 2024-04-12T20:57:51.924 INFO:teuthology.orchestra.run.smithi018.stdout:} 2024-04-12T20:57:51.925 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:57:51.926+0000 7ff3e8800700 1 -- 172.21.15.18:0/943749989 >> [v2:172.21.15.18:6800/1605950634,v1:172.21.15.18:6801/1605950634] conn(0x7ff3c806e7b0 msgr2=0x7ff3c8070c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:57:51.925 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:57:51.926+0000 7ff3e8800700 1 --2- 172.21.15.18:0/943749989 >> [v2:172.21.15.18:6800/1605950634,v1:172.21.15.18:6801/1605950634] conn(0x7ff3c806e7b0 0x7ff3c8070c60 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7ff3d4009fd0 tx=0x7ff3d4009e30 comp rx=0 tx=0).stop 2024-04-12T20:57:51.925 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:57:51.926+0000 7ff3e8800700 1 -- 172.21.15.18:0/943749989 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff3e006f9d0 msgr2=0x7ff3e0118880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:57:51.925 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:57:51.926+0000 7ff3e8800700 1 --2- 172.21.15.18:0/943749989 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff3e006f9d0 0x7ff3e0118880 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7ff3d0009fd0 tx=0x7ff3d000b040 comp rx=0 tx=0).stop 2024-04-12T20:57:51.925 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:57:51.927+0000 7ff3e8800700 1 -- 172.21.15.18:0/943749989 shutdown_connections 2024-04-12T20:57:51.925 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:57:51.927+0000 7ff3e8800700 1 --2- 172.21.15.18:0/943749989 >> [v2:172.21.15.18:6800/1605950634,v1:172.21.15.18:6801/1605950634] conn(0x7ff3c806e7b0 0x7ff3c8070c60 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:57:51.925 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:57:51.927+0000 7ff3e8800700 1 --2- 172.21.15.18:0/943749989 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff3e006e9a0 0x7ff3e0118340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:57:51.926 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:57:51.927+0000 7ff3e8800700 1 --2- 172.21.15.18:0/943749989 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff3e006f9d0 0x7ff3e0118880 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:57:51.926 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:57:51.927+0000 7ff3e8800700 1 -- 172.21.15.18:0/943749989 >> 172.21.15.18:0/943749989 conn(0x7ff3e0069de0 msgr2=0x7ff3e006b4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T20:57:51.926 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:57:51.927+0000 7ff3e8800700 1 -- 172.21.15.18:0/943749989 shutdown_connections 2024-04-12T20:57:51.926 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:57:51.928+0000 7ff3e8800700 1 -- 172.21.15.18:0/943749989 wait complete. 2024-04-12T20:57:52.030 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:57:51 smithi186 ceph-mon[36597]: from='client.14728 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T20:57:52.030 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:57:51 smithi186 ceph-mon[36597]: pgmap v22: 65 pgs: 65 active+clean; 279 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 43 KiB/s rd, 1.5 MiB/s wr, 431 op/s 2024-04-12T20:57:52.030 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:57:51 smithi186 ceph-mon[36597]: from='client.14732 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T20:57:52.030 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:57:51 smithi186 ceph-mon[36597]: from='client.? 172.21.15.18:0/1948640541' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T20:57:54.031 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:57:53 smithi186 ceph-mon[36597]: from='client.24511 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T20:57:54.031 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:57:53 smithi186 ceph-mon[36597]: pgmap v23: 65 pgs: 65 active+clean; 279 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 30 KiB/s rd, 1008 KiB/s wr, 306 op/s 2024-04-12T20:57:54.031 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:57:53 smithi186 ceph-mon[36597]: from='mgr.14672 172.21.15.18:0/2209228970' entity='mgr.smithi018.xjzwga' 2024-04-12T20:57:54.031 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:57:53 smithi186 ceph-mon[36597]: from='mgr.14672 172.21.15.18:0/2209228970' entity='mgr.smithi018.xjzwga' 2024-04-12T20:57:54.031 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:57:53 smithi186 ceph-mon[36597]: from='mgr.14672 172.21.15.18:0/2209228970' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-04-12T20:57:54.031 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:57:53 smithi186 ceph-mon[36597]: from='mgr.14672 172.21.15.18:0/2209228970' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-12T20:57:54.101 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:57:53 smithi018 ceph-mon[28799]: from='client.24511 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T20:57:54.101 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:57:53 smithi018 ceph-mon[28799]: pgmap v23: 65 pgs: 65 active+clean; 279 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 30 KiB/s rd, 1008 KiB/s wr, 306 op/s 2024-04-12T20:57:54.101 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:57:53 smithi018 ceph-mon[28799]: from='mgr.14672 172.21.15.18:0/2209228970' entity='mgr.smithi018.xjzwga' 2024-04-12T20:57:54.101 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:57:53 smithi018 ceph-mon[28799]: from='mgr.14672 172.21.15.18:0/2209228970' entity='mgr.smithi018.xjzwga' 2024-04-12T20:57:54.102 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:57:53 smithi018 ceph-mon[28799]: from='mgr.14672 172.21.15.18:0/2209228970' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-04-12T20:57:54.102 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:57:53 smithi018 ceph-mon[28799]: from='mgr.14672 172.21.15.18:0/2209228970' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-12T20:57:55.030 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:57:54 smithi186 ceph-mon[36597]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-04-12T20:57:55.101 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:57:54 smithi018 ceph-mon[28799]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-04-12T20:57:56.101 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:57:55 smithi018 ceph-mon[28799]: pgmap v24: 65 pgs: 65 active+clean; 274 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 45 KiB/s rd, 1.5 MiB/s wr, 445 op/s 2024-04-12T20:57:56.102 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:57:55 smithi018 ceph-mon[28799]: from='mgr.14672 172.21.15.18:0/2209228970' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi186.kqcfxk", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-12T20:57:56.102 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:57:55 smithi018 ceph-mon[28799]: from='mgr.14672 172.21.15.18:0/2209228970' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-12T20:57:56.102 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:57:55 smithi018 ceph-mon[28799]: from='mgr.14672 172.21.15.18:0/2209228970' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-12T20:57:56.172 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:57:55 smithi186 ceph-mon[36597]: pgmap v24: 65 pgs: 65 active+clean; 274 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 45 KiB/s rd, 1.5 MiB/s wr, 445 op/s 2024-04-12T20:57:56.172 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:57:55 smithi186 ceph-mon[36597]: from='mgr.14672 172.21.15.18:0/2209228970' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi186.kqcfxk", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-12T20:57:56.172 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:57:55 smithi186 ceph-mon[36597]: from='mgr.14672 172.21.15.18:0/2209228970' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-12T20:57:56.172 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:57:55 smithi186 ceph-mon[36597]: from='mgr.14672 172.21.15.18:0/2209228970' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-12T20:57:57.056 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:57:56 smithi186 ceph-mon[36597]: Upgrade: Updating mgr.smithi186.kqcfxk 2024-04-12T20:57:57.057 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:57:56 smithi186 ceph-mon[36597]: Deploying daemon mgr.smithi186.kqcfxk on smithi186 2024-04-12T20:57:57.102 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:57:56 smithi018 ceph-mon[28799]: Upgrade: Updating mgr.smithi186.kqcfxk 2024-04-12T20:57:57.102 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:57:56 smithi018 ceph-mon[28799]: Deploying daemon mgr.smithi186.kqcfxk on smithi186 2024-04-12T20:57:58.093 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:57:57 smithi018 ceph-mon[28799]: pgmap v25: 65 pgs: 65 active+clean; 274 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 30 KiB/s rd, 1.0 MiB/s wr, 287 op/s 2024-04-12T20:57:58.093 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:57:58 smithi018 ceph-mon[28799]: from='mgr.14672 172.21.15.18:0/2209228970' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T20:57:58.280 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:57:57 smithi186 ceph-mon[36597]: pgmap v25: 65 pgs: 65 active+clean; 274 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 30 KiB/s rd, 1.0 MiB/s wr, 287 op/s 2024-04-12T20:57:58.281 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:57:57 smithi186 ceph-mon[36597]: from='mgr.14672 172.21.15.18:0/2209228970' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T20:58:00.101 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:57:59 smithi018 ceph-mon[28799]: pgmap v26: 65 pgs: 65 active+clean; 274 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 30 KiB/s rd, 1.0 MiB/s wr, 287 op/s 2024-04-12T20:58:00.224 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:57:59 smithi186 ceph-mon[36597]: pgmap v26: 65 pgs: 65 active+clean; 274 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 30 KiB/s rd, 1.0 MiB/s wr, 287 op/s 2024-04-12T20:58:02.102 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:58:01 smithi018 ceph-mon[28799]: pgmap v27: 65 pgs: 65 active+clean; 270 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 45 KiB/s rd, 1.6 MiB/s wr, 447 op/s 2024-04-12T20:58:02.102 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:58:01 smithi018 ceph-mon[28799]: from='mgr.14672 172.21.15.18:0/2209228970' entity='mgr.smithi018.xjzwga' 2024-04-12T20:58:02.102 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:58:01 smithi018 ceph-mon[28799]: from='mgr.14672 172.21.15.18:0/2209228970' entity='mgr.smithi018.xjzwga' 2024-04-12T20:58:02.102 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:58:01 smithi018 ceph-mon[28799]: from='mgr.14672 172.21.15.18:0/2209228970' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-12T20:58:02.116 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:58:01 smithi186 ceph-mon[36597]: pgmap v27: 65 pgs: 65 active+clean; 270 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 45 KiB/s rd, 1.6 MiB/s wr, 447 op/s 2024-04-12T20:58:02.116 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:58:01 smithi186 ceph-mon[36597]: from='mgr.14672 172.21.15.18:0/2209228970' entity='mgr.smithi018.xjzwga' 2024-04-12T20:58:02.116 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:58:01 smithi186 ceph-mon[36597]: from='mgr.14672 172.21.15.18:0/2209228970' entity='mgr.smithi018.xjzwga' 2024-04-12T20:58:02.116 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:58:01 smithi186 ceph-mon[36597]: from='mgr.14672 172.21.15.18:0/2209228970' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-12T20:58:04.030 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:58:03 smithi186 ceph-mon[36597]: pgmap v28: 65 pgs: 65 active+clean; 270 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 31 KiB/s rd, 1.0 MiB/s wr, 299 op/s 2024-04-12T20:58:04.101 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:58:03 smithi018 ceph-mon[28799]: pgmap v28: 65 pgs: 65 active+clean; 270 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 31 KiB/s rd, 1.0 MiB/s wr, 299 op/s 2024-04-12T20:58:06.101 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:58:05 smithi018 ceph-mon[28799]: pgmap v29: 65 pgs: 65 active+clean; 269 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 42 KiB/s rd, 1.5 MiB/s wr, 434 op/s 2024-04-12T20:58:06.280 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:58:05 smithi186 ceph-mon[36597]: pgmap v29: 65 pgs: 65 active+clean; 269 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 42 KiB/s rd, 1.5 MiB/s wr, 434 op/s 2024-04-12T20:58:07.101 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:58:06 smithi018 ceph-mon[28799]: from='mgr.14672 172.21.15.18:0/2209228970' entity='mgr.smithi018.xjzwga' 2024-04-12T20:58:07.101 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:58:06 smithi018 ceph-mon[28799]: from='mgr.14672 172.21.15.18:0/2209228970' entity='mgr.smithi018.xjzwga' 2024-04-12T20:58:07.101 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:58:06 smithi018 ceph-mon[28799]: pgmap v30: 65 pgs: 65 active+clean; 269 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 27 KiB/s rd, 1.0 MiB/s wr, 294 op/s 2024-04-12T20:58:07.182 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:58:06 smithi186 ceph-mon[36597]: from='mgr.14672 172.21.15.18:0/2209228970' entity='mgr.smithi018.xjzwga' 2024-04-12T20:58:07.182 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:58:06 smithi186 ceph-mon[36597]: from='mgr.14672 172.21.15.18:0/2209228970' entity='mgr.smithi018.xjzwga' 2024-04-12T20:58:07.182 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:58:06 smithi186 ceph-mon[36597]: pgmap v30: 65 pgs: 65 active+clean; 269 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 27 KiB/s rd, 1.0 MiB/s wr, 294 op/s 2024-04-12T20:58:09.038 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:58:08 smithi186 ceph-mon[36597]: from='mgr.14672 172.21.15.18:0/2209228970' entity='mgr.smithi018.xjzwga' 2024-04-12T20:58:09.039 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:58:08 smithi186 ceph-mon[36597]: from='mgr.14672 172.21.15.18:0/2209228970' entity='mgr.smithi018.xjzwga' 2024-04-12T20:58:09.039 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:58:08 smithi186 ceph-mon[36597]: pgmap v31: 65 pgs: 65 active+clean; 269 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 27 KiB/s rd, 1.0 MiB/s wr, 294 op/s 2024-04-12T20:58:09.101 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:58:08 smithi018 ceph-mon[28799]: from='mgr.14672 172.21.15.18:0/2209228970' entity='mgr.smithi018.xjzwga' 2024-04-12T20:58:09.101 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:58:08 smithi018 ceph-mon[28799]: from='mgr.14672 172.21.15.18:0/2209228970' entity='mgr.smithi018.xjzwga' 2024-04-12T20:58:09.101 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:58:08 smithi018 ceph-mon[28799]: pgmap v31: 65 pgs: 65 active+clean; 269 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 27 KiB/s rd, 1.0 MiB/s wr, 294 op/s 2024-04-12T20:58:09.458 INFO:tasks.workunit:Stopping ['suites/fsstress.sh'] on client.0... 2024-04-12T20:58:09.459 DEBUG:teuthology.orchestra.run.smithi018:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0 2024-04-12T20:58:10.955 DEBUG:teuthology.parallel:result is None 2024-04-12T20:58:11.601 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:58:11 smithi018 ceph-mon[28799]: pgmap v32: 65 pgs: 65 active+clean; 269 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 40 KiB/s rd, 1.5 MiB/s wr, 426 op/s 2024-04-12T20:58:11.780 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:58:11 smithi186 ceph-mon[36597]: pgmap v32: 65 pgs: 65 active+clean; 269 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 40 KiB/s rd, 1.5 MiB/s wr, 426 op/s 2024-04-12T20:58:13.071 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:58:12 smithi186 ceph-mon[36597]: from='mgr.14672 172.21.15.18:0/2209228970' entity='mgr.smithi018.xjzwga' 2024-04-12T20:58:13.071 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:58:12 smithi186 ceph-mon[36597]: from='mgr.14672 172.21.15.18:0/2209228970' entity='mgr.smithi018.xjzwga' 2024-04-12T20:58:13.071 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:58:12 smithi186 ceph-mon[36597]: Standby manager daemon smithi186.kqcfxk restarted 2024-04-12T20:58:13.071 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:58:12 smithi186 ceph-mon[36597]: Standby manager daemon smithi186.kqcfxk started 2024-04-12T20:58:13.071 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:58:12 smithi186 ceph-mon[36597]: from='mgr.? 172.21.15.186:0/2511046211' entity='mgr.smithi186.kqcfxk' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi186.kqcfxk/crt"}]: dispatch 2024-04-12T20:58:13.071 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:58:12 smithi186 ceph-mon[36597]: from='mgr.? 172.21.15.186:0/2511046211' entity='mgr.smithi186.kqcfxk' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-12T20:58:13.071 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:58:12 smithi186 ceph-mon[36597]: from='mgr.? 172.21.15.186:0/2511046211' entity='mgr.smithi186.kqcfxk' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi186.kqcfxk/key"}]: dispatch 2024-04-12T20:58:13.071 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:58:12 smithi186 ceph-mon[36597]: from='mgr.? 172.21.15.186:0/2511046211' entity='mgr.smithi186.kqcfxk' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-12T20:58:13.072 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:58:12 smithi186 ceph-mon[36597]: from='mgr.14672 172.21.15.18:0/2209228970' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T20:58:13.102 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:58:12 smithi018 ceph-mon[28799]: from='mgr.14672 172.21.15.18:0/2209228970' entity='mgr.smithi018.xjzwga' 2024-04-12T20:58:13.102 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:58:12 smithi018 ceph-mon[28799]: from='mgr.14672 172.21.15.18:0/2209228970' entity='mgr.smithi018.xjzwga' 2024-04-12T20:58:13.102 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:58:12 smithi018 ceph-mon[28799]: Standby manager daemon smithi186.kqcfxk restarted 2024-04-12T20:58:13.102 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:58:12 smithi018 ceph-mon[28799]: Standby manager daemon smithi186.kqcfxk started 2024-04-12T20:58:13.102 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:58:12 smithi018 ceph-mon[28799]: from='mgr.? 172.21.15.186:0/2511046211' entity='mgr.smithi186.kqcfxk' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi186.kqcfxk/crt"}]: dispatch 2024-04-12T20:58:13.102 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:58:12 smithi018 ceph-mon[28799]: from='mgr.? 172.21.15.186:0/2511046211' entity='mgr.smithi186.kqcfxk' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-12T20:58:13.102 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:58:12 smithi018 ceph-mon[28799]: from='mgr.? 172.21.15.186:0/2511046211' entity='mgr.smithi186.kqcfxk' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi186.kqcfxk/key"}]: dispatch 2024-04-12T20:58:13.102 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:58:12 smithi018 ceph-mon[28799]: from='mgr.? 172.21.15.186:0/2511046211' entity='mgr.smithi186.kqcfxk' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-12T20:58:13.102 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:58:12 smithi018 ceph-mon[28799]: from='mgr.14672 172.21.15.18:0/2209228970' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T20:58:14.101 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:58:13 smithi018 ceph-mon[28799]: pgmap v33: 65 pgs: 65 active+clean; 269 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 24 KiB/s rd, 965 KiB/s wr, 266 op/s 2024-04-12T20:58:14.101 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:58:13 smithi018 ceph-mon[28799]: mgrmap e31: smithi018.xjzwga(active, since 60s), standbys: smithi186.kqcfxk 2024-04-12T20:58:14.280 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:58:13 smithi186 ceph-mon[36597]: pgmap v33: 65 pgs: 65 active+clean; 269 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 24 KiB/s rd, 965 KiB/s wr, 266 op/s 2024-04-12T20:58:14.280 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:58:13 smithi186 ceph-mon[36597]: mgrmap e31: smithi018.xjzwga(active, since 60s), standbys: smithi186.kqcfxk 2024-04-12T20:58:15.282 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:58:14 smithi186 ceph-mon[36597]: from='mgr.14672 172.21.15.18:0/2209228970' entity='mgr.smithi018.xjzwga' 2024-04-12T20:58:15.282 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:58:14 smithi186 ceph-mon[36597]: from='mgr.14672 172.21.15.18:0/2209228970' entity='mgr.smithi018.xjzwga' 2024-04-12T20:58:15.282 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:58:14 smithi186 ceph-mon[36597]: from='mgr.14672 172.21.15.18:0/2209228970' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-12T20:58:15.282 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:58:14 smithi186 ceph-mon[36597]: from='mgr.14672 172.21.15.18:0/2209228970' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-12T20:58:15.282 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:58:14 smithi186 ceph-mon[36597]: from='mgr.14672 172.21.15.18:0/2209228970' entity='mgr.smithi018.xjzwga' 2024-04-12T20:58:15.283 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:58:14 smithi186 ceph-mon[36597]: from='mgr.14672 172.21.15.18:0/2209228970' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-12T20:58:15.283 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:58:14 smithi186 ceph-mon[36597]: from='mgr.14672 172.21.15.18:0/2209228970' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T20:58:15.283 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:58:14 smithi186 ceph-mon[36597]: Upgrade: Setting container_image for all mgr 2024-04-12T20:58:15.283 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:58:14 smithi186 ceph-mon[36597]: from='mgr.14672 172.21.15.18:0/2209228970' entity='mgr.smithi018.xjzwga' 2024-04-12T20:58:15.283 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:58:14 smithi186 ceph-mon[36597]: from='mgr.14672 172.21.15.18:0/2209228970' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr.smithi018.xjzwga"}]: dispatch 2024-04-12T20:58:15.283 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:58:14 smithi186 ceph-mon[36597]: from='mgr.14672 172.21.15.18:0/2209228970' entity='mgr.smithi018.xjzwga' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mgr.smithi018.xjzwga"}]': finished 2024-04-12T20:58:15.283 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:58:14 smithi186 ceph-mon[36597]: from='mgr.14672 172.21.15.18:0/2209228970' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr.smithi186.kqcfxk"}]: dispatch 2024-04-12T20:58:15.283 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:58:14 smithi186 ceph-mon[36597]: from='mgr.14672 172.21.15.18:0/2209228970' entity='mgr.smithi018.xjzwga' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mgr.smithi186.kqcfxk"}]': finished 2024-04-12T20:58:15.283 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:58:14 smithi186 ceph-mon[36597]: from='mgr.14672 172.21.15.18:0/2209228970' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T20:58:15.283 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:58:14 smithi186 ceph-mon[36597]: Upgrade: Setting container_image for all rgw 2024-04-12T20:58:15.283 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:58:14 smithi186 ceph-mon[36597]: from='mgr.14672 172.21.15.18:0/2209228970' entity='mgr.smithi018.xjzwga' 2024-04-12T20:58:15.283 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:58:14 smithi186 ceph-mon[36597]: from='mgr.14672 172.21.15.18:0/2209228970' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T20:58:15.283 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:58:14 smithi186 ceph-mon[36597]: Upgrade: Setting container_image for all rbd-mirror 2024-04-12T20:58:15.284 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:58:14 smithi186 ceph-mon[36597]: from='mgr.14672 172.21.15.18:0/2209228970' entity='mgr.smithi018.xjzwga' 2024-04-12T20:58:15.284 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:58:14 smithi186 ceph-mon[36597]: from='mgr.14672 172.21.15.18:0/2209228970' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T20:58:15.284 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:58:14 smithi186 ceph-mon[36597]: Upgrade: Setting container_image for all cephfs-mirror 2024-04-12T20:58:15.284 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:58:14 smithi186 ceph-mon[36597]: from='mgr.14672 172.21.15.18:0/2209228970' entity='mgr.smithi018.xjzwga' 2024-04-12T20:58:15.284 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:58:14 smithi186 ceph-mon[36597]: from='mgr.14672 172.21.15.18:0/2209228970' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T20:58:15.284 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:58:14 smithi186 ceph-mon[36597]: Upgrade: Setting container_image for all ceph-exporter 2024-04-12T20:58:15.284 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:58:14 smithi186 ceph-mon[36597]: from='mgr.14672 172.21.15.18:0/2209228970' entity='mgr.smithi018.xjzwga' 2024-04-12T20:58:15.284 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:58:14 smithi186 ceph-mon[36597]: from='mgr.14672 172.21.15.18:0/2209228970' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T20:58:15.284 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:58:14 smithi186 ceph-mon[36597]: Upgrade: Setting container_image for all iscsi 2024-04-12T20:58:15.284 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:58:14 smithi186 ceph-mon[36597]: from='mgr.14672 172.21.15.18:0/2209228970' entity='mgr.smithi018.xjzwga' 2024-04-12T20:58:15.284 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:58:14 smithi186 ceph-mon[36597]: from='mgr.14672 172.21.15.18:0/2209228970' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T20:58:15.284 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:58:14 smithi186 ceph-mon[36597]: Upgrade: Setting container_image for all nfs 2024-04-12T20:58:15.284 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:58:14 smithi186 ceph-mon[36597]: from='mgr.14672 172.21.15.18:0/2209228970' entity='mgr.smithi018.xjzwga' 2024-04-12T20:58:15.284 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:58:14 smithi186 ceph-mon[36597]: from='mgr.14672 172.21.15.18:0/2209228970' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T20:58:15.284 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:58:14 smithi186 ceph-mon[36597]: Upgrade: Setting container_image for all nvmeof 2024-04-12T20:58:15.284 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:58:14 smithi186 ceph-mon[36597]: from='mgr.14672 172.21.15.18:0/2209228970' entity='mgr.smithi018.xjzwga' 2024-04-12T20:58:15.285 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:58:14 smithi186 ceph-mon[36597]: pgmap v34: 65 pgs: 65 active+clean; 265 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 36 KiB/s rd, 1.2 MiB/s wr, 374 op/s 2024-04-12T20:58:15.352 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:58:14 smithi018 ceph-mon[28799]: from='mgr.14672 172.21.15.18:0/2209228970' entity='mgr.smithi018.xjzwga' 2024-04-12T20:58:15.352 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:58:14 smithi018 ceph-mon[28799]: from='mgr.14672 172.21.15.18:0/2209228970' entity='mgr.smithi018.xjzwga' 2024-04-12T20:58:15.352 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:58:14 smithi018 ceph-mon[28799]: from='mgr.14672 172.21.15.18:0/2209228970' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-12T20:58:15.352 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:58:14 smithi018 ceph-mon[28799]: from='mgr.14672 172.21.15.18:0/2209228970' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-12T20:58:15.352 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:58:14 smithi018 ceph-mon[28799]: from='mgr.14672 172.21.15.18:0/2209228970' entity='mgr.smithi018.xjzwga' 2024-04-12T20:58:15.352 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:58:14 smithi018 ceph-mon[28799]: from='mgr.14672 172.21.15.18:0/2209228970' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-12T20:58:15.352 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:58:14 smithi018 ceph-mon[28799]: from='mgr.14672 172.21.15.18:0/2209228970' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T20:58:15.352 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:58:14 smithi018 ceph-mon[28799]: Upgrade: Setting container_image for all mgr 2024-04-12T20:58:15.352 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:58:14 smithi018 ceph-mon[28799]: from='mgr.14672 172.21.15.18:0/2209228970' entity='mgr.smithi018.xjzwga' 2024-04-12T20:58:15.352 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:58:14 smithi018 ceph-mon[28799]: from='mgr.14672 172.21.15.18:0/2209228970' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr.smithi018.xjzwga"}]: dispatch 2024-04-12T20:58:15.353 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:58:14 smithi018 ceph-mon[28799]: from='mgr.14672 172.21.15.18:0/2209228970' entity='mgr.smithi018.xjzwga' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mgr.smithi018.xjzwga"}]': finished 2024-04-12T20:58:15.353 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:58:14 smithi018 ceph-mon[28799]: from='mgr.14672 172.21.15.18:0/2209228970' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr.smithi186.kqcfxk"}]: dispatch 2024-04-12T20:58:15.353 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:58:14 smithi018 ceph-mon[28799]: from='mgr.14672 172.21.15.18:0/2209228970' entity='mgr.smithi018.xjzwga' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mgr.smithi186.kqcfxk"}]': finished 2024-04-12T20:58:15.353 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:58:14 smithi018 ceph-mon[28799]: from='mgr.14672 172.21.15.18:0/2209228970' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T20:58:15.353 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:58:14 smithi018 ceph-mon[28799]: Upgrade: Setting container_image for all rgw 2024-04-12T20:58:15.353 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:58:14 smithi018 ceph-mon[28799]: from='mgr.14672 172.21.15.18:0/2209228970' entity='mgr.smithi018.xjzwga' 2024-04-12T20:58:15.353 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:58:14 smithi018 ceph-mon[28799]: from='mgr.14672 172.21.15.18:0/2209228970' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T20:58:15.353 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:58:14 smithi018 ceph-mon[28799]: Upgrade: Setting container_image for all rbd-mirror 2024-04-12T20:58:15.353 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:58:14 smithi018 ceph-mon[28799]: from='mgr.14672 172.21.15.18:0/2209228970' entity='mgr.smithi018.xjzwga' 2024-04-12T20:58:15.353 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:58:14 smithi018 ceph-mon[28799]: from='mgr.14672 172.21.15.18:0/2209228970' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T20:58:15.353 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:58:14 smithi018 ceph-mon[28799]: Upgrade: Setting container_image for all cephfs-mirror 2024-04-12T20:58:15.353 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:58:14 smithi018 ceph-mon[28799]: from='mgr.14672 172.21.15.18:0/2209228970' entity='mgr.smithi018.xjzwga' 2024-04-12T20:58:15.353 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:58:14 smithi018 ceph-mon[28799]: from='mgr.14672 172.21.15.18:0/2209228970' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T20:58:15.353 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:58:14 smithi018 ceph-mon[28799]: Upgrade: Setting container_image for all ceph-exporter 2024-04-12T20:58:15.353 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:58:14 smithi018 ceph-mon[28799]: from='mgr.14672 172.21.15.18:0/2209228970' entity='mgr.smithi018.xjzwga' 2024-04-12T20:58:15.354 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:58:14 smithi018 ceph-mon[28799]: from='mgr.14672 172.21.15.18:0/2209228970' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T20:58:15.354 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:58:14 smithi018 ceph-mon[28799]: Upgrade: Setting container_image for all iscsi 2024-04-12T20:58:15.354 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:58:14 smithi018 ceph-mon[28799]: from='mgr.14672 172.21.15.18:0/2209228970' entity='mgr.smithi018.xjzwga' 2024-04-12T20:58:15.354 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:58:14 smithi018 ceph-mon[28799]: from='mgr.14672 172.21.15.18:0/2209228970' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T20:58:15.354 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:58:14 smithi018 ceph-mon[28799]: Upgrade: Setting container_image for all nfs 2024-04-12T20:58:15.354 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:58:14 smithi018 ceph-mon[28799]: from='mgr.14672 172.21.15.18:0/2209228970' entity='mgr.smithi018.xjzwga' 2024-04-12T20:58:15.354 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:58:14 smithi018 ceph-mon[28799]: from='mgr.14672 172.21.15.18:0/2209228970' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T20:58:15.354 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:58:14 smithi018 ceph-mon[28799]: Upgrade: Setting container_image for all nvmeof 2024-04-12T20:58:15.354 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:58:14 smithi018 ceph-mon[28799]: from='mgr.14672 172.21.15.18:0/2209228970' entity='mgr.smithi018.xjzwga' 2024-04-12T20:58:15.354 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:58:14 smithi018 ceph-mon[28799]: pgmap v34: 65 pgs: 65 active+clean; 265 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 36 KiB/s rd, 1.2 MiB/s wr, 374 op/s 2024-04-12T20:58:17.851 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:58:17 smithi018 ceph-mon[28799]: pgmap v35: 65 pgs: 65 active+clean; 265 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 25 KiB/s rd, 760 KiB/s wr, 239 op/s 2024-04-12T20:58:18.030 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:58:17 smithi186 ceph-mon[36597]: pgmap v35: 65 pgs: 65 active+clean; 265 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 25 KiB/s rd, 760 KiB/s wr, 239 op/s 2024-04-12T20:58:18.658 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:58:18 smithi018 ceph-mon[28799]: Upgrade: Updating node-exporter.smithi018 (1/2) 2024-04-12T20:58:18.658 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:58:18 smithi018 ceph-mon[28799]: Deploying daemon node-exporter.smithi018 on smithi018 2024-04-12T20:58:19.030 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:58:18 smithi186 ceph-mon[36597]: Upgrade: Updating node-exporter.smithi018 (1/2) 2024-04-12T20:58:19.030 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:58:18 smithi186 ceph-mon[36597]: Deploying daemon node-exporter.smithi018 on smithi018 2024-04-12T20:58:19.851 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:58:19 smithi018 ceph-mon[28799]: pgmap v36: 65 pgs: 65 active+clean; 265 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 25 KiB/s rd, 760 KiB/s wr, 239 op/s 2024-04-12T20:58:20.030 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:58:19 smithi186 ceph-mon[36597]: pgmap v36: 65 pgs: 65 active+clean; 265 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 25 KiB/s rd, 760 KiB/s wr, 239 op/s 2024-04-12T20:58:21.851 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:58:21 smithi018 ceph-mon[28799]: pgmap v37: 65 pgs: 65 active+clean; 265 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 30 KiB/s rd, 947 KiB/s wr, 295 op/s 2024-04-12T20:58:22.030 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:58:21 smithi186 ceph-mon[36597]: pgmap v37: 65 pgs: 65 active+clean; 265 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 30 KiB/s rd, 947 KiB/s wr, 295 op/s 2024-04-12T20:58:22.157 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:58:22.159+0000 7fb573c00700 1 -- 172.21.15.18:0/1632757096 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb56c1002a0 msgr2=0x7fb56c100710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:58:22.157 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:58:22.159+0000 7fb573c00700 1 --2- 172.21.15.18:0/1632757096 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb56c1002a0 0x7fb56c100710 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fb558004940 tx=0x7fb558009e30 comp rx=0 tx=0).stop 2024-04-12T20:58:22.157 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:58:22.160+0000 7fb573c00700 1 -- 172.21.15.18:0/1632757096 shutdown_connections 2024-04-12T20:58:22.157 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:58:22.160+0000 7fb573c00700 1 --2- 172.21.15.18:0/1632757096 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb56c1002a0 0x7fb56c100710 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:58:22.158 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:58:22.160+0000 7fb573c00700 1 --2- 172.21.15.18:0/1632757096 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb56c0feff0 0x7fb56c0ff400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:58:22.158 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:58:22.160+0000 7fb573c00700 1 -- 172.21.15.18:0/1632757096 >> 172.21.15.18:0/1632757096 conn(0x7fb56c0fa950 msgr2=0x7fb56c0fcd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T20:58:22.158 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:58:22.160+0000 7fb573c00700 1 -- 172.21.15.18:0/1632757096 shutdown_connections 2024-04-12T20:58:22.158 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:58:22.160+0000 7fb573c00700 1 -- 172.21.15.18:0/1632757096 wait complete. 2024-04-12T20:58:22.160 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:58:22.161+0000 7fb573c00700 1 Processor -- start 2024-04-12T20:58:22.160 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:58:22.161+0000 7fb573c00700 1 -- start start 2024-04-12T20:58:22.160 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:58:22.162+0000 7fb573c00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb56c0feff0 0x7fb56c10fb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T20:58:22.160 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:58:22.162+0000 7fb573c00700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb56c1002a0 0x7fb56c1100d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T20:58:22.160 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:58:22.162+0000 7fb573c00700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb56c1106d0 con 0x7fb56c0feff0 2024-04-12T20:58:22.160 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:58:22.162+0000 7fb573c00700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb56c110f20 con 0x7fb56c1002a0 2024-04-12T20:58:22.160 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:58:22.162+0000 7fb571200700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb56c0feff0 0x7fb56c10fb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T20:58:22.160 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:58:22.162+0000 7fb571200700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb56c0feff0 0x7fb56c10fb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:40992/0 (socket says 172.21.15.18:40992) 2024-04-12T20:58:22.160 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:58:22.162+0000 7fb571200700 1 -- 172.21.15.18:0/3871342354 learned_addr learned my addr 172.21.15.18:0/3871342354 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T20:58:22.160 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:58:22.162+0000 7fb56be00700 1 --2- 172.21.15.18:0/3871342354 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb56c1002a0 0x7fb56c1100d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T20:58:22.160 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:58:22.163+0000 7fb571200700 1 -- 172.21.15.18:0/3871342354 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb56c1002a0 msgr2=0x7fb56c1100d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:58:22.160 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:58:22.163+0000 7fb571200700 1 --2- 172.21.15.18:0/3871342354 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb56c1002a0 0x7fb56c1100d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:58:22.161 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:58:22.163+0000 7fb571200700 1 -- 172.21.15.18:0/3871342354 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb558005040 con 0x7fb56c0feff0 2024-04-12T20:58:22.161 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:58:22.163+0000 7fb56be00700 1 --2- 172.21.15.18:0/3871342354 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb56c1002a0 0x7fb56c1100d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-12T20:58:22.161 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:58:22.163+0000 7fb571200700 1 --2- 172.21.15.18:0/3871342354 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb56c0feff0 0x7fb56c10fb90 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fb56000eb40 tx=0x7fb56000b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T20:58:22.161 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:58:22.163+0000 7fb568c00700 1 -- 172.21.15.18:0/3871342354 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb560007cc0 con 0x7fb56c0feff0 2024-04-12T20:58:22.161 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:58:22.163+0000 7fb568c00700 1 -- 172.21.15.18:0/3871342354 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb56000f040 con 0x7fb56c0feff0 2024-04-12T20:58:22.162 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:58:22.163+0000 7fb573c00700 1 -- 172.21.15.18:0/3871342354 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb56c1111a0 con 0x7fb56c0feff0 2024-04-12T20:58:22.162 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:58:22.164+0000 7fb573c00700 1 -- 172.21.15.18:0/3871342354 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb56c1116e0 con 0x7fb56c0feff0 2024-04-12T20:58:22.162 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:58:22.164+0000 7fb568c00700 1 -- 172.21.15.18:0/3871342354 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb5600092b0 con 0x7fb56c0feff0 2024-04-12T20:58:22.162 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:58:22.165+0000 7fb573c00700 1 -- 172.21.15.18:0/3871342354 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb56c05f5f0 con 0x7fb56c0feff0 2024-04-12T20:58:22.163 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:58:22.166+0000 7fb568c00700 1 -- 172.21.15.18:0/3871342354 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 31) v1 ==== 92561+0+0 (secure 0 0 0) 0x7fb560009520 con 0x7fb56c0feff0 2024-04-12T20:58:22.165 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:58:22.168+0000 7fb568c00700 1 --2- 172.21.15.18:0/3871342354 >> [v2:172.21.15.18:6800/1605950634,v1:172.21.15.18:6801/1605950634] conn(0x7fb55406e620 0x7fb554070ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T20:58:22.165 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:58:22.168+0000 7fb568c00700 1 -- 172.21.15.18:0/3871342354 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6066+0+0 (secure 0 0 0) 0x7fb56008e4d0 con 0x7fb56c0feff0 2024-04-12T20:58:22.165 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:58:22.169+0000 7fb56be00700 1 --2- 172.21.15.18:0/3871342354 >> [v2:172.21.15.18:6800/1605950634,v1:172.21.15.18:6801/1605950634] conn(0x7fb55406e620 0x7fb554070ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T20:58:22.166 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:58:22.169+0000 7fb56be00700 1 --2- 172.21.15.18:0/3871342354 >> [v2:172.21.15.18:6800/1605950634,v1:172.21.15.18:6801/1605950634] conn(0x7fb55406e620 0x7fb554070ad0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fb558009fd0 tx=0x7fb558009e30 comp rx=0 tx=0).ready entity=mgr.14672 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T20:58:22.170 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:58:22.173+0000 7fb568c00700 1 -- 172.21.15.18:0/3871342354 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186454 (secure 0 0 0) 0x7fb56005b410 con 0x7fb56c0feff0 2024-04-12T20:58:22.442 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:58:22.446+0000 7fb573c00700 1 -- 172.21.15.18:0/3871342354 --> [v2:172.21.15.18:6800/1605950634,v1:172.21.15.18:6801/1605950634] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fb56c111ad0 con 0x7fb55406e620 2024-04-12T20:58:22.446 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:58:22.448+0000 7fb568c00700 1 -- 172.21.15.18:0/3871342354 <== mgr.14672 v2:172.21.15.18:6800/1605950634 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+370 (secure 0 0 0) 0x7fb56c111ad0 con 0x7fb55406e620 2024-04-12T20:58:22.449 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:58:22.452+0000 7fb573c00700 1 -- 172.21.15.18:0/3871342354 >> [v2:172.21.15.18:6800/1605950634,v1:172.21.15.18:6801/1605950634] conn(0x7fb55406e620 msgr2=0x7fb554070ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:58:22.449 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:58:22.452+0000 7fb573c00700 1 --2- 172.21.15.18:0/3871342354 >> [v2:172.21.15.18:6800/1605950634,v1:172.21.15.18:6801/1605950634] conn(0x7fb55406e620 0x7fb554070ad0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fb558009fd0 tx=0x7fb558009e30 comp rx=0 tx=0).stop 2024-04-12T20:58:22.449 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:58:22.452+0000 7fb573c00700 1 -- 172.21.15.18:0/3871342354 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb56c0feff0 msgr2=0x7fb56c10fb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:58:22.449 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:58:22.452+0000 7fb573c00700 1 --2- 172.21.15.18:0/3871342354 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb56c0feff0 0x7fb56c10fb90 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fb56000eb40 tx=0x7fb56000b040 comp rx=0 tx=0).stop 2024-04-12T20:58:22.449 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:58:22.452+0000 7fb573c00700 1 -- 172.21.15.18:0/3871342354 shutdown_connections 2024-04-12T20:58:22.450 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:58:22.452+0000 7fb573c00700 1 --2- 172.21.15.18:0/3871342354 >> [v2:172.21.15.18:6800/1605950634,v1:172.21.15.18:6801/1605950634] conn(0x7fb55406e620 0x7fb554070ad0 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:58:22.450 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:58:22.452+0000 7fb573c00700 1 --2- 172.21.15.18:0/3871342354 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb56c0feff0 0x7fb56c10fb90 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:58:22.450 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:58:22.452+0000 7fb573c00700 1 --2- 172.21.15.18:0/3871342354 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb56c1002a0 0x7fb56c1100d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:58:22.450 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:58:22.452+0000 7fb573c00700 1 -- 172.21.15.18:0/3871342354 >> 172.21.15.18:0/3871342354 conn(0x7fb56c0fa950 msgr2=0x7fb56c0fcf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T20:58:22.451 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:58:22.453+0000 7fb573c00700 1 -- 172.21.15.18:0/3871342354 shutdown_connections 2024-04-12T20:58:22.451 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:58:22.453+0000 7fb573c00700 1 -- 172.21.15.18:0/3871342354 wait complete. 2024-04-12T20:58:22.470 INFO:teuthology.orchestra.run.smithi018.stdout:true 2024-04-12T20:58:22.626 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:58:22.629+0000 7f5433200700 1 -- 172.21.15.18:0/3059947243 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f542c071f50 msgr2=0x7f542c070460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:58:22.627 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:58:22.629+0000 7f5433200700 1 --2- 172.21.15.18:0/3059947243 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f542c071f50 0x7f542c070460 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f541c004940 tx=0x7f541c009e30 comp rx=0 tx=0).stop 2024-04-12T20:58:22.627 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:58:22.629+0000 7f5433200700 1 -- 172.21.15.18:0/3059947243 shutdown_connections 2024-04-12T20:58:22.627 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:58:22.629+0000 7f5433200700 1 --2- 172.21.15.18:0/3059947243 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f542c0709a0 0x7f542c070e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:58:22.627 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:58:22.629+0000 7f5433200700 1 --2- 172.21.15.18:0/3059947243 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f542c071f50 0x7f542c070460 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:58:22.627 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:58:22.629+0000 7f5433200700 1 -- 172.21.15.18:0/3059947243 >> 172.21.15.18:0/3059947243 conn(0x7f542c0f8560 msgr2=0x7f542c0fa970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T20:58:22.627 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:58:22.629+0000 7f5433200700 1 -- 172.21.15.18:0/3059947243 shutdown_connections 2024-04-12T20:58:22.627 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:58:22.630+0000 7f5433200700 1 -- 172.21.15.18:0/3059947243 wait complete. 2024-04-12T20:58:22.628 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:58:22.630+0000 7f5433200700 1 Processor -- start 2024-04-12T20:58:22.628 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:58:22.631+0000 7f5433200700 1 -- start start 2024-04-12T20:58:22.628 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:58:22.631+0000 7f5433200700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f542c0709a0 0x7f542c10fc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T20:58:22.628 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:58:22.631+0000 7f5433200700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f542c071f50 0x7f542c110170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T20:58:22.628 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:58:22.631+0000 7f5433200700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f542c110770 con 0x7f542c0709a0 2024-04-12T20:58:22.628 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:58:22.631+0000 7f5433200700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f542c110f70 con 0x7f542c071f50 2024-04-12T20:58:22.629 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:58:22.631+0000 7f542be00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f542c0709a0 0x7f542c10fc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T20:58:22.629 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:58:22.631+0000 7f542b400700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f542c071f50 0x7f542c110170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T20:58:22.629 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:58:22.632+0000 7f542be00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f542c0709a0 0x7f542c10fc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:41006/0 (socket says 172.21.15.18:41006) 2024-04-12T20:58:22.629 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:58:22.632+0000 7f542b400700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f542c071f50 0x7f542c110170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.18:48788/0 (socket says 172.21.15.18:48788) 2024-04-12T20:58:22.629 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:58:22.632+0000 7f542be00700 1 -- 172.21.15.18:0/338403443 learned_addr learned my addr 172.21.15.18:0/338403443 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T20:58:22.630 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:58:22.632+0000 7f542be00700 1 -- 172.21.15.18:0/338403443 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f542c071f50 msgr2=0x7f542c110170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:58:22.630 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:58:22.632+0000 7f542be00700 1 --2- 172.21.15.18:0/338403443 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f542c071f50 0x7f542c110170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:58:22.630 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:58:22.632+0000 7f542be00700 1 -- 172.21.15.18:0/338403443 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f541c005040 con 0x7f542c0709a0 2024-04-12T20:58:22.630 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:58:22.632+0000 7f542be00700 1 --2- 172.21.15.18:0/338403443 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f542c0709a0 0x7f542c10fc30 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f541c00b0d0 tx=0x7f541c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T20:58:22.630 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:58:22.633+0000 7f5423e00700 1 -- 172.21.15.18:0/338403443 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f541c00e070 con 0x7f542c0709a0 2024-04-12T20:58:22.631 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:58:22.633+0000 7f5423e00700 1 -- 172.21.15.18:0/338403443 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f541c013de0 con 0x7f542c0709a0 2024-04-12T20:58:22.631 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:58:22.633+0000 7f5423e00700 1 -- 172.21.15.18:0/338403443 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f541c00be30 con 0x7f542c0709a0 2024-04-12T20:58:22.631 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:58:22.633+0000 7f5433200700 1 -- 172.21.15.18:0/338403443 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f542c1111f0 con 0x7f542c0709a0 2024-04-12T20:58:22.631 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:58:22.633+0000 7f5433200700 1 -- 172.21.15.18:0/338403443 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f542c111730 con 0x7f542c0709a0 2024-04-12T20:58:22.632 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:58:22.634+0000 7f5433200700 1 -- 172.21.15.18:0/338403443 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f542c05f5f0 con 0x7f542c0709a0 2024-04-12T20:58:22.632 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:58:22.636+0000 7f5423e00700 1 -- 172.21.15.18:0/338403443 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 31) v1 ==== 92561+0+0 (secure 0 0 0) 0x7f541c00b680 con 0x7f542c0709a0 2024-04-12T20:58:22.633 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:58:22.637+0000 7f5423e00700 1 --2- 172.21.15.18:0/338403443 >> [v2:172.21.15.18:6800/1605950634,v1:172.21.15.18:6801/1605950634] conn(0x7f541006e620 0x7f5410070ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T20:58:22.634 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:58:22.637+0000 7f5423e00700 1 -- 172.21.15.18:0/338403443 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6066+0+0 (secure 0 0 0) 0x7f541c08ece0 con 0x7f542c0709a0 2024-04-12T20:58:22.634 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:58:22.637+0000 7f542b400700 1 --2- 172.21.15.18:0/338403443 >> [v2:172.21.15.18:6800/1605950634,v1:172.21.15.18:6801/1605950634] conn(0x7f541006e620 0x7f5410070ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T20:58:22.634 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:58:22.638+0000 7f542b400700 1 --2- 172.21.15.18:0/338403443 >> [v2:172.21.15.18:6800/1605950634,v1:172.21.15.18:6801/1605950634] conn(0x7f541006e620 0x7f5410070ad0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f5418009b80 tx=0x7f541800b040 comp rx=0 tx=0).ready entity=mgr.14672 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T20:58:22.638 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:58:22.642+0000 7f5423e00700 1 -- 172.21.15.18:0/338403443 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186454 (secure 0 0 0) 0x7f541c05bd60 con 0x7f542c0709a0 2024-04-12T20:58:22.902 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:58:22.905+0000 7f5433200700 1 -- 172.21.15.18:0/338403443 --> [v2:172.21.15.18:6800/1605950634,v1:172.21.15.18:6801/1605950634] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f542c111b20 con 0x7f541006e620 2024-04-12T20:58:22.904 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:58:22.908+0000 7f5423e00700 1 -- 172.21.15.18:0/338403443 <== mgr.14672 v2:172.21.15.18:6800/1605950634 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+370 (secure 0 0 0) 0x7f542c111b20 con 0x7f541006e620 2024-04-12T20:58:22.909 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:58:22.912+0000 7f5433200700 1 -- 172.21.15.18:0/338403443 >> [v2:172.21.15.18:6800/1605950634,v1:172.21.15.18:6801/1605950634] conn(0x7f541006e620 msgr2=0x7f5410070ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:58:22.909 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:58:22.912+0000 7f5433200700 1 --2- 172.21.15.18:0/338403443 >> [v2:172.21.15.18:6800/1605950634,v1:172.21.15.18:6801/1605950634] conn(0x7f541006e620 0x7f5410070ad0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f5418009b80 tx=0x7f541800b040 comp rx=0 tx=0).stop 2024-04-12T20:58:22.909 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:58:22.912+0000 7f5433200700 1 -- 172.21.15.18:0/338403443 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f542c0709a0 msgr2=0x7f542c10fc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:58:22.909 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:58:22.912+0000 7f5433200700 1 --2- 172.21.15.18:0/338403443 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f542c0709a0 0x7f542c10fc30 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f541c00b0d0 tx=0x7f541c009e30 comp rx=0 tx=0).stop 2024-04-12T20:58:22.909 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:58:22.912+0000 7f5433200700 1 -- 172.21.15.18:0/338403443 shutdown_connections 2024-04-12T20:58:22.909 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:58:22.912+0000 7f5433200700 1 --2- 172.21.15.18:0/338403443 >> [v2:172.21.15.18:6800/1605950634,v1:172.21.15.18:6801/1605950634] conn(0x7f541006e620 0x7f5410070ad0 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:58:22.909 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:58:22.912+0000 7f5433200700 1 --2- 172.21.15.18:0/338403443 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f542c0709a0 0x7f542c10fc30 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:58:22.910 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:58:22.912+0000 7f5433200700 1 --2- 172.21.15.18:0/338403443 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f542c071f50 0x7f542c110170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:58:22.910 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:58:22.912+0000 7f5433200700 1 -- 172.21.15.18:0/338403443 >> 172.21.15.18:0/338403443 conn(0x7f542c0f8560 msgr2=0x7f542c0f9bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T20:58:22.910 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:58:22.913+0000 7f5433200700 1 -- 172.21.15.18:0/338403443 shutdown_connections 2024-04-12T20:58:22.910 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:58:22.913+0000 7f5433200700 1 -- 172.21.15.18:0/338403443 wait complete. 2024-04-12T20:58:23.062 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:58:22 smithi186 ceph-mon[36597]: from='mgr.14672 172.21.15.18:0/2209228970' entity='mgr.smithi018.xjzwga' 2024-04-12T20:58:23.062 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:58:22 smithi186 ceph-mon[36597]: from='mgr.14672 172.21.15.18:0/2209228970' entity='mgr.smithi018.xjzwga' 2024-04-12T20:58:23.101 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:58:22 smithi018 ceph-mon[28799]: from='mgr.14672 172.21.15.18:0/2209228970' entity='mgr.smithi018.xjzwga' 2024-04-12T20:58:23.101 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:58:22 smithi018 ceph-mon[28799]: from='mgr.14672 172.21.15.18:0/2209228970' entity='mgr.smithi018.xjzwga' 2024-04-12T20:58:23.112 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:58:23.115+0000 7f0db8c00700 1 -- 172.21.15.18:0/2918756932 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0db40709a0 msgr2=0x7f0db4070e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:58:23.112 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:58:23.115+0000 7f0db8c00700 1 --2- 172.21.15.18:0/2918756932 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0db40709a0 0x7f0db4070e10 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f0da0004940 tx=0x7f0da0009e30 comp rx=0 tx=0).stop 2024-04-12T20:58:23.112 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:58:23.115+0000 7f0db8c00700 1 -- 172.21.15.18:0/2918756932 shutdown_connections 2024-04-12T20:58:23.112 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:58:23.115+0000 7f0db8c00700 1 --2- 172.21.15.18:0/2918756932 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0db40709a0 0x7f0db4070e10 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:58:23.112 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:58:23.115+0000 7f0db8c00700 1 --2- 172.21.15.18:0/2918756932 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0db4071f50 0x7f0db4070460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:58:23.112 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:58:23.115+0000 7f0db8c00700 1 -- 172.21.15.18:0/2918756932 >> 172.21.15.18:0/2918756932 conn(0x7f0db40f8560 msgr2=0x7f0db40fa970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T20:58:23.112 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:58:23.115+0000 7f0db8c00700 1 -- 172.21.15.18:0/2918756932 shutdown_connections 2024-04-12T20:58:23.113 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:58:23.115+0000 7f0db8c00700 1 -- 172.21.15.18:0/2918756932 wait complete. 2024-04-12T20:58:23.113 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:58:23.116+0000 7f0db8c00700 1 Processor -- start 2024-04-12T20:58:23.113 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:58:23.116+0000 7f0db8c00700 1 -- start start 2024-04-12T20:58:23.114 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:58:23.117+0000 7f0db8c00700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0db40709a0 0x7f0db406e2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T20:58:23.114 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:58:23.117+0000 7f0db8c00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0db4071f50 0x7f0db406e7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T20:58:23.114 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:58:23.117+0000 7f0db8c00700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0db406fee0 con 0x7f0db4071f50 2024-04-12T20:58:23.114 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:58:23.117+0000 7f0db8c00700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0db4070020 con 0x7f0db40709a0 2024-04-12T20:58:23.114 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:58:23.117+0000 7f0db1400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0db4071f50 0x7f0db406e7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T20:58:23.114 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:58:23.117+0000 7f0db1400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0db4071f50 0x7f0db406e7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:41020/0 (socket says 172.21.15.18:41020) 2024-04-12T20:58:23.114 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:58:23.117+0000 7f0db1400700 1 -- 172.21.15.18:0/746787638 learned_addr learned my addr 172.21.15.18:0/746787638 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T20:58:23.114 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:58:23.117+0000 7f0db1e00700 1 --2- 172.21.15.18:0/746787638 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0db40709a0 0x7f0db406e2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T20:58:23.115 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:58:23.117+0000 7f0db1400700 1 -- 172.21.15.18:0/746787638 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0db40709a0 msgr2=0x7f0db406e2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:58:23.115 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:58:23.118+0000 7f0db1400700 1 --2- 172.21.15.18:0/746787638 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0db40709a0 0x7f0db406e2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:58:23.115 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:58:23.118+0000 7f0db1400700 1 -- 172.21.15.18:0/746787638 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0da0005040 con 0x7f0db4071f50 2024-04-12T20:58:23.115 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:58:23.118+0000 7f0db1400700 1 --2- 172.21.15.18:0/746787638 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0db4071f50 0x7f0db406e7e0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f0da0013870 tx=0x7f0da0009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T20:58:23.115 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:58:23.118+0000 7f0daa000700 1 -- 172.21.15.18:0/746787638 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0da000e070 con 0x7f0db4071f50 2024-04-12T20:58:23.116 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:58:23.118+0000 7f0daa000700 1 -- 172.21.15.18:0/746787638 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f0da000b410 con 0x7f0db4071f50 2024-04-12T20:58:23.116 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:58:23.118+0000 7f0db8c00700 1 -- 172.21.15.18:0/746787638 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0db406ede0 con 0x7f0db4071f50 2024-04-12T20:58:23.116 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:58:23.118+0000 7f0daa000700 1 -- 172.21.15.18:0/746787638 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0da00125e0 con 0x7f0db4071f50 2024-04-12T20:58:23.116 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:58:23.119+0000 7f0db8c00700 1 -- 172.21.15.18:0/746787638 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0db411e1b0 con 0x7f0db4071f50 2024-04-12T20:58:23.117 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:58:23.120+0000 7f0db8c00700 1 -- 172.21.15.18:0/746787638 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0db405f5f0 con 0x7f0db4071f50 2024-04-12T20:58:23.120 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:58:23.123+0000 7f0daa000700 1 -- 172.21.15.18:0/746787638 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 31) v1 ==== 92561+0+0 (secure 0 0 0) 0x7f0da000b580 con 0x7f0db4071f50 2024-04-12T20:58:23.121 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:58:23.124+0000 7f0daa000700 1 --2- 172.21.15.18:0/746787638 >> [v2:172.21.15.18:6800/1605950634,v1:172.21.15.18:6801/1605950634] conn(0x7f0d9806e6f0 0x7f0d98070ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T20:58:23.121 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:58:23.124+0000 7f0daa000700 1 -- 172.21.15.18:0/746787638 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6066+0+0 (secure 0 0 0) 0x7f0da008f350 con 0x7f0db4071f50 2024-04-12T20:58:23.122 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:58:23.125+0000 7f0db1e00700 1 --2- 172.21.15.18:0/746787638 >> [v2:172.21.15.18:6800/1605950634,v1:172.21.15.18:6801/1605950634] conn(0x7f0d9806e6f0 0x7f0d98070ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T20:58:23.122 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:58:23.126+0000 7f0db1e00700 1 --2- 172.21.15.18:0/746787638 >> [v2:172.21.15.18:6800/1605950634,v1:172.21.15.18:6801/1605950634] conn(0x7f0d9806e6f0 0x7f0d98070ba0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f0db406fb60 tx=0x7f0da400b040 comp rx=0 tx=0).ready entity=mgr.14672 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T20:58:23.123 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:58:23.127+0000 7f0daa000700 1 -- 172.21.15.18:0/746787638 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186454 (secure 0 0 0) 0x7f0da00587e0 con 0x7f0db4071f50 2024-04-12T20:58:23.402 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:58:23.405+0000 7f0db8c00700 1 -- 172.21.15.18:0/746787638 --> [v2:172.21.15.18:6800/1605950634,v1:172.21.15.18:6801/1605950634] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f0db4103d80 con 0x7f0d9806e6f0 2024-04-12T20:58:23.433 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:58:23.436+0000 7f0daa000700 1 -- 172.21.15.18:0/746787638 <== mgr.14672 v2:172.21.15.18:6800/1605950634 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3520 (secure 0 0 0) 0x7f0db4103d80 con 0x7f0d9806e6f0 2024-04-12T20:58:23.434 INFO:teuthology.orchestra.run.smithi018.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-12T20:58:23.434 INFO:teuthology.orchestra.run.smithi018.stdout:alertmanager.smithi018 smithi018 *:9093,9094 running (2m) 11s ago 13m 15.3M - 0.25.0 c8568f914cd2 b4ed63d191d4 2024-04-12T20:58:23.434 INFO:teuthology.orchestra.run.smithi018.stdout:crash.smithi018 smithi018 running (13m) 11s ago 13m 6660k - 17.2.7-827-gd3b5060a 21de9b7369ad 1ac7603bddfd 2024-04-12T20:58:23.434 INFO:teuthology.orchestra.run.smithi018.stdout:crash.smithi186 smithi186 running (12m) 17s ago 12m 6669k - 17.2.7-827-gd3b5060a 21de9b7369ad 14bf8a1672eb 2024-04-12T20:58:23.434 INFO:teuthology.orchestra.run.smithi018.stdout:grafana.smithi018 smithi018 *:3000 running (2m) 11s ago 13m 75.7M - 9.4.7 954c08fa6188 d6251dcdaec0 2024-04-12T20:58:23.434 INFO:teuthology.orchestra.run.smithi018.stdout:mds.cephfs.smithi018.acqbsc smithi018 running (6m) 11s ago 6m 126M - 17.2.7-827-gd3b5060a 21de9b7369ad 5b5fce65f18a 2024-04-12T20:58:23.434 INFO:teuthology.orchestra.run.smithi018.stdout:mds.cephfs.smithi018.szjluo smithi018 running (6m) 11s ago 6m 134M - 17.2.7-827-gd3b5060a 21de9b7369ad cf9bf0c5031f 2024-04-12T20:58:23.434 INFO:teuthology.orchestra.run.smithi018.stdout:mds.cephfs.smithi186.jdllyn smithi186 running (6m) 17s ago 6m 12.9M - 17.2.7-827-gd3b5060a 21de9b7369ad 2b2bc46bd29a 2024-04-12T20:58:23.434 INFO:teuthology.orchestra.run.smithi018.stdout:mds.cephfs.smithi186.wjnafs smithi186 running (6m) 17s ago 6m 12.9M - 17.2.7-827-gd3b5060a 21de9b7369ad 74b115d6d6fd 2024-04-12T20:58:23.435 INFO:teuthology.orchestra.run.smithi018.stdout:mgr.smithi018.xjzwga smithi018 *:8443,9283,8765 running (100s) 11s ago 15m 494M - 19.0.0-2879-g324baf8d 8717720dc5e5 56088305eab7 2024-04-12T20:58:23.435 INFO:teuthology.orchestra.run.smithi018.stdout:mgr.smithi186.kqcfxk smithi186 *:8443,9283,8765 running (22s) 17s ago 12m 63.4M - 19.0.0-2879-g324baf8d 8717720dc5e5 09cd766e5e0d 2024-04-12T20:58:23.435 INFO:teuthology.orchestra.run.smithi018.stdout:mon.smithi018 smithi018 running (15m) 11s ago 15m 52.7M 2048M 17.2.7-827-gd3b5060a 21de9b7369ad cd84caafa056 2024-04-12T20:58:23.435 INFO:teuthology.orchestra.run.smithi018.stdout:mon.smithi186 smithi186 running (12m) 17s ago 12m 41.2M 2048M 17.2.7-827-gd3b5060a 21de9b7369ad d7263c772b73 2024-04-12T20:58:23.435 INFO:teuthology.orchestra.run.smithi018.stdout:node-exporter.smithi018 smithi018 *:9100 starting - - - - 2024-04-12T20:58:23.435 INFO:teuthology.orchestra.run.smithi018.stdout:node-exporter.smithi186 smithi186 *:9100 running (111s) 17s ago 12m 7583k - 1.5.0 0da6a335fe13 4ac3111e3ae2 2024-04-12T20:58:23.435 INFO:teuthology.orchestra.run.smithi018.stdout:osd.0 smithi018 running (11m) 11s ago 11m 211M 1559M 17.2.7-827-gd3b5060a 21de9b7369ad 0d83e027bf0a 2024-04-12T20:58:23.435 INFO:teuthology.orchestra.run.smithi018.stdout:osd.1 smithi018 running (10m) 11s ago 10m 230M 1559M 17.2.7-827-gd3b5060a 21de9b7369ad 6517c9cc7f87 2024-04-12T20:58:23.435 INFO:teuthology.orchestra.run.smithi018.stdout:osd.2 smithi018 running (9m) 11s ago 9m 179M 1559M 17.2.7-827-gd3b5060a 21de9b7369ad 8a83692acaa5 2024-04-12T20:58:23.435 INFO:teuthology.orchestra.run.smithi018.stdout:osd.3 smithi186 running (9m) 17s ago 9m 250M 2581M 17.2.7-827-gd3b5060a 21de9b7369ad c2def7a90d76 2024-04-12T20:58:23.435 INFO:teuthology.orchestra.run.smithi018.stdout:osd.4 smithi186 running (8m) 17s ago 8m 224M 2581M 17.2.7-827-gd3b5060a 21de9b7369ad 757e7c71b7b9 2024-04-12T20:58:23.435 INFO:teuthology.orchestra.run.smithi018.stdout:osd.5 smithi186 running (8m) 17s ago 8m 217M 2581M 17.2.7-827-gd3b5060a 21de9b7369ad 2ee072a436d0 2024-04-12T20:58:23.435 INFO:teuthology.orchestra.run.smithi018.stdout:prometheus.smithi018 smithi018 *:9095 running (31s) 11s ago 12m 42.8M - 2.43.0 a07b618ecd1d 463fc59e8c0e 2024-04-12T20:58:23.438 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:58:23.441+0000 7f0db8c00700 1 -- 172.21.15.18:0/746787638 >> [v2:172.21.15.18:6800/1605950634,v1:172.21.15.18:6801/1605950634] conn(0x7f0d9806e6f0 msgr2=0x7f0d98070ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:58:23.438 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:58:23.441+0000 7f0db8c00700 1 --2- 172.21.15.18:0/746787638 >> [v2:172.21.15.18:6800/1605950634,v1:172.21.15.18:6801/1605950634] conn(0x7f0d9806e6f0 0x7f0d98070ba0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f0db406fb60 tx=0x7f0da400b040 comp rx=0 tx=0).stop 2024-04-12T20:58:23.438 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:58:23.441+0000 7f0db8c00700 1 -- 172.21.15.18:0/746787638 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0db4071f50 msgr2=0x7f0db406e7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:58:23.438 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:58:23.441+0000 7f0db8c00700 1 --2- 172.21.15.18:0/746787638 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0db4071f50 0x7f0db406e7e0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f0da0013870 tx=0x7f0da0009e30 comp rx=0 tx=0).stop 2024-04-12T20:58:23.439 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:58:23.442+0000 7f0db8c00700 1 -- 172.21.15.18:0/746787638 shutdown_connections 2024-04-12T20:58:23.439 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:58:23.442+0000 7f0db8c00700 1 --2- 172.21.15.18:0/746787638 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0db40709a0 0x7f0db406e2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:58:23.439 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:58:23.442+0000 7f0db8c00700 1 --2- 172.21.15.18:0/746787638 >> [v2:172.21.15.18:6800/1605950634,v1:172.21.15.18:6801/1605950634] conn(0x7f0d9806e6f0 0x7f0d98070ba0 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:58:23.439 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:58:23.442+0000 7f0db8c00700 1 --2- 172.21.15.18:0/746787638 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0db4071f50 0x7f0db406e7e0 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:58:23.439 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:58:23.442+0000 7f0db8c00700 1 -- 172.21.15.18:0/746787638 >> 172.21.15.18:0/746787638 conn(0x7f0db40f8560 msgr2=0x7f0db40f9bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T20:58:23.439 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:58:23.442+0000 7f0db8c00700 1 -- 172.21.15.18:0/746787638 shutdown_connections 2024-04-12T20:58:23.439 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:58:23.442+0000 7f0db8c00700 1 -- 172.21.15.18:0/746787638 wait complete. 2024-04-12T20:58:23.606 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:58:23.609+0000 7fe6e0800700 1 -- 172.21.15.18:0/2993742830 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe6d80662e0 msgr2=0x7fe6d8101dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:58:23.606 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:58:23.609+0000 7fe6e0800700 1 --2- 172.21.15.18:0/2993742830 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe6d80662e0 0x7fe6d8101dc0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7fe6cc004940 tx=0x7fe6cc009e30 comp rx=0 tx=0).stop 2024-04-12T20:58:23.606 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:58:23.609+0000 7fe6e0800700 1 -- 172.21.15.18:0/2993742830 shutdown_connections 2024-04-12T20:58:23.606 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:58:23.609+0000 7fe6e0800700 1 --2- 172.21.15.18:0/2993742830 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe6d8102300 0x7fe6d81046e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:58:23.606 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:58:23.609+0000 7fe6e0800700 1 --2- 172.21.15.18:0/2993742830 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe6d80662e0 0x7fe6d8101dc0 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:58:23.606 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:58:23.609+0000 7fe6e0800700 1 -- 172.21.15.18:0/2993742830 >> 172.21.15.18:0/2993742830 conn(0x7fe6d80f7430 msgr2=0x7fe6d80f9860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T20:58:23.607 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:58:23.609+0000 7fe6e0800700 1 -- 172.21.15.18:0/2993742830 shutdown_connections 2024-04-12T20:58:23.607 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:58:23.609+0000 7fe6e0800700 1 -- 172.21.15.18:0/2993742830 wait complete. 2024-04-12T20:58:23.608 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:58:23.610+0000 7fe6e0800700 1 Processor -- start 2024-04-12T20:58:23.608 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:58:23.610+0000 7fe6e0800700 1 -- start start 2024-04-12T20:58:23.608 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:58:23.610+0000 7fe6e0800700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe6d80662e0 0x7fe6d810fc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T20:58:23.608 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:58:23.610+0000 7fe6e0800700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe6d8102300 0x7fe6d8110160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T20:58:23.608 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:58:23.610+0000 7fe6e0800700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6d8110760 con 0x7fe6d80662e0 2024-04-12T20:58:23.608 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:58:23.610+0000 7fe6e0800700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6d8110fb0 con 0x7fe6d8102300 2024-04-12T20:58:23.608 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:58:23.611+0000 7fe6dde00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe6d80662e0 0x7fe6d810fc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T20:58:23.608 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:58:23.611+0000 7fe6dde00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe6d80662e0 0x7fe6d810fc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:41044/0 (socket says 172.21.15.18:41044) 2024-04-12T20:58:23.608 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:58:23.611+0000 7fe6dde00700 1 -- 172.21.15.18:0/3406720837 learned_addr learned my addr 172.21.15.18:0/3406720837 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T20:58:23.609 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:58:23.611+0000 7fe6dd400700 1 --2- 172.21.15.18:0/3406720837 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe6d8102300 0x7fe6d8110160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T20:58:23.609 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:58:23.611+0000 7fe6dde00700 1 -- 172.21.15.18:0/3406720837 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe6d8102300 msgr2=0x7fe6d8110160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:58:23.609 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:58:23.611+0000 7fe6dde00700 1 --2- 172.21.15.18:0/3406720837 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe6d8102300 0x7fe6d8110160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:58:23.609 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:58:23.611+0000 7fe6dde00700 1 -- 172.21.15.18:0/3406720837 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6cc005040 con 0x7fe6d80662e0 2024-04-12T20:58:23.609 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:58:23.612+0000 7fe6dde00700 1 --2- 172.21.15.18:0/3406720837 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe6d80662e0 0x7fe6d810fc20 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fe6cc00b970 tx=0x7fe6cc009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T20:58:23.610 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:58:23.612+0000 7fe6d2000700 1 -- 172.21.15.18:0/3406720837 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe6cc00e040 con 0x7fe6d80662e0 2024-04-12T20:58:23.610 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:58:23.612+0000 7fe6d2000700 1 -- 172.21.15.18:0/3406720837 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe6cc00bbf0 con 0x7fe6d80662e0 2024-04-12T20:58:23.610 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:58:23.612+0000 7fe6d2000700 1 -- 172.21.15.18:0/3406720837 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe6cc01c630 con 0x7fe6d80662e0 2024-04-12T20:58:23.610 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:58:23.612+0000 7fe6e0800700 1 -- 172.21.15.18:0/3406720837 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe6d8111230 con 0x7fe6d80662e0 2024-04-12T20:58:23.610 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:58:23.612+0000 7fe6e0800700 1 -- 172.21.15.18:0/3406720837 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe6d8111770 con 0x7fe6d80662e0 2024-04-12T20:58:23.610 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:58:23.613+0000 7fe6e0800700 1 -- 172.21.15.18:0/3406720837 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe6d805f5f0 con 0x7fe6d80662e0 2024-04-12T20:58:23.611 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:58:23.614+0000 7fe6d2000700 1 -- 172.21.15.18:0/3406720837 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 31) v1 ==== 92561+0+0 (secure 0 0 0) 0x7fe6cc00bd60 con 0x7fe6d80662e0 2024-04-12T20:58:23.612 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:58:23.615+0000 7fe6d2000700 1 --2- 172.21.15.18:0/3406720837 >> [v2:172.21.15.18:6800/1605950634,v1:172.21.15.18:6801/1605950634] conn(0x7fe6c006e620 0x7fe6c0070ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T20:58:23.612 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:58:23.616+0000 7fe6d2000700 1 -- 172.21.15.18:0/3406720837 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6066+0+0 (secure 0 0 0) 0x7fe6cc08f4f0 con 0x7fe6d80662e0 2024-04-12T20:58:23.613 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:58:23.616+0000 7fe6dd400700 1 --2- 172.21.15.18:0/3406720837 >> [v2:172.21.15.18:6800/1605950634,v1:172.21.15.18:6801/1605950634] conn(0x7fe6c006e620 0x7fe6c0070ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T20:58:23.613 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:58:23.617+0000 7fe6dd400700 1 --2- 172.21.15.18:0/3406720837 >> [v2:172.21.15.18:6800/1605950634,v1:172.21.15.18:6801/1605950634] conn(0x7fe6c006e620 0x7fe6c0070ad0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fe6c8007fd0 tx=0x7fe6c8009040 comp rx=0 tx=0).ready entity=mgr.14672 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T20:58:23.618 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:58:23.621+0000 7fe6d2000700 1 -- 172.21.15.18:0/3406720837 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186454 (secure 0 0 0) 0x7fe6cc05c4b0 con 0x7fe6d80662e0 2024-04-12T20:58:23.990 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:58:23.993+0000 7fe6e0800700 1 -- 172.21.15.18:0/3406720837 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fe6d8107ab0 con 0x7fe6d80662e0 2024-04-12T20:58:23.992 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:58:23.995+0000 7fe6d2000700 1 -- 172.21.15.18:0/3406720837 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+739 (secure 0 0 0) 0x7fe6cc05c040 con 0x7fe6d80662e0 2024-04-12T20:58:23.993 INFO:teuthology.orchestra.run.smithi018.stdout:{ 2024-04-12T20:58:23.993 INFO:teuthology.orchestra.run.smithi018.stdout: "mon": { 2024-04-12T20:58:23.993 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 17.2.7-827-gd3b5060a (d3b5060a17449edfae9385cdf792afec109fca5d) quincy (stable)": 2 2024-04-12T20:58:23.993 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-12T20:58:23.993 INFO:teuthology.orchestra.run.smithi018.stdout: "mgr": { 2024-04-12T20:58:23.993 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 19.0.0-2879-g324baf8d (324baf8ddb91dea46cac50cb535c306115791355) squid (dev)": 2 2024-04-12T20:58:23.993 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-12T20:58:23.993 INFO:teuthology.orchestra.run.smithi018.stdout: "osd": { 2024-04-12T20:58:23.994 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 17.2.7-827-gd3b5060a (d3b5060a17449edfae9385cdf792afec109fca5d) quincy (stable)": 6 2024-04-12T20:58:23.994 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-12T20:58:23.994 INFO:teuthology.orchestra.run.smithi018.stdout: "mds": { 2024-04-12T20:58:23.994 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 17.2.7-827-gd3b5060a (d3b5060a17449edfae9385cdf792afec109fca5d) quincy (stable)": 4 2024-04-12T20:58:23.994 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-12T20:58:23.994 INFO:teuthology.orchestra.run.smithi018.stdout: "overall": { 2024-04-12T20:58:23.994 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 17.2.7-827-gd3b5060a (d3b5060a17449edfae9385cdf792afec109fca5d) quincy (stable)": 12, 2024-04-12T20:58:23.994 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 19.0.0-2879-g324baf8d (324baf8ddb91dea46cac50cb535c306115791355) squid (dev)": 2 2024-04-12T20:58:23.994 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-04-12T20:58:23.994 INFO:teuthology.orchestra.run.smithi018.stdout:} 2024-04-12T20:58:23.995 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:58:23.998+0000 7fe6e0800700 1 -- 172.21.15.18:0/3406720837 >> [v2:172.21.15.18:6800/1605950634,v1:172.21.15.18:6801/1605950634] conn(0x7fe6c006e620 msgr2=0x7fe6c0070ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:58:23.996 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:58:23.998+0000 7fe6e0800700 1 --2- 172.21.15.18:0/3406720837 >> [v2:172.21.15.18:6800/1605950634,v1:172.21.15.18:6801/1605950634] conn(0x7fe6c006e620 0x7fe6c0070ad0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fe6c8007fd0 tx=0x7fe6c8009040 comp rx=0 tx=0).stop 2024-04-12T20:58:23.996 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:58:23.998+0000 7fe6e0800700 1 -- 172.21.15.18:0/3406720837 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe6d80662e0 msgr2=0x7fe6d810fc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:58:23.996 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:58:23.998+0000 7fe6e0800700 1 --2- 172.21.15.18:0/3406720837 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe6d80662e0 0x7fe6d810fc20 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fe6cc00b970 tx=0x7fe6cc009e30 comp rx=0 tx=0).stop 2024-04-12T20:58:23.996 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:58:23.999+0000 7fe6e0800700 1 -- 172.21.15.18:0/3406720837 shutdown_connections 2024-04-12T20:58:23.996 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:58:23.999+0000 7fe6e0800700 1 --2- 172.21.15.18:0/3406720837 >> [v2:172.21.15.18:6800/1605950634,v1:172.21.15.18:6801/1605950634] conn(0x7fe6c006e620 0x7fe6c0070ad0 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:58:23.996 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:58:23.999+0000 7fe6e0800700 1 --2- 172.21.15.18:0/3406720837 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe6d80662e0 0x7fe6d810fc20 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:58:23.996 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:58:23.999+0000 7fe6e0800700 1 --2- 172.21.15.18:0/3406720837 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe6d8102300 0x7fe6d8110160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:58:23.996 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:58:23.999+0000 7fe6e0800700 1 -- 172.21.15.18:0/3406720837 >> 172.21.15.18:0/3406720837 conn(0x7fe6d80f7430 msgr2=0x7fe6d80f7850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T20:58:23.997 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:58:23.999+0000 7fe6e0800700 1 -- 172.21.15.18:0/3406720837 shutdown_connections 2024-04-12T20:58:23.997 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:58:23.999+0000 7fe6e0800700 1 -- 172.21.15.18:0/3406720837 wait complete. 2024-04-12T20:58:24.030 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:58:23 smithi186 ceph-mon[36597]: from='client.14746 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T20:58:24.030 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:58:23 smithi186 ceph-mon[36597]: pgmap v38: 65 pgs: 65 active+clean; 265 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 17 KiB/s rd, 477 KiB/s wr, 163 op/s 2024-04-12T20:58:24.101 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:58:23 smithi018 ceph-mon[28799]: from='client.14746 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T20:58:24.101 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:58:23 smithi018 ceph-mon[28799]: pgmap v38: 65 pgs: 65 active+clean; 265 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 17 KiB/s rd, 477 KiB/s wr, 163 op/s 2024-04-12T20:58:24.178 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:58:24.181+0000 7f3e88600700 1 -- 172.21.15.18:0/4169231259 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3e800fcee0 msgr2=0x7f3e800fd310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:58:24.178 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:58:24.181+0000 7f3e88600700 1 --2- 172.21.15.18:0/4169231259 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3e800fcee0 0x7f3e800fd310 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f3e74004940 tx=0x7f3e74009e30 comp rx=0 tx=0).stop 2024-04-12T20:58:24.178 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:58:24.181+0000 7f3e88600700 1 -- 172.21.15.18:0/4169231259 shutdown_connections 2024-04-12T20:58:24.179 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:58:24.181+0000 7f3e88600700 1 --2- 172.21.15.18:0/4169231259 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3e800fe0d0 0x7f3e800fe540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:58:24.179 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:58:24.181+0000 7f3e88600700 1 --2- 172.21.15.18:0/4169231259 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3e800fcee0 0x7f3e800fd310 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:58:24.179 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:58:24.181+0000 7f3e88600700 1 -- 172.21.15.18:0/4169231259 >> 172.21.15.18:0/4169231259 conn(0x7f3e800f8560 msgr2=0x7f3e800fa970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T20:58:24.179 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:58:24.181+0000 7f3e88600700 1 -- 172.21.15.18:0/4169231259 shutdown_connections 2024-04-12T20:58:24.179 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:58:24.181+0000 7f3e88600700 1 -- 172.21.15.18:0/4169231259 wait complete. 2024-04-12T20:58:24.180 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:58:24.182+0000 7f3e88600700 1 Processor -- start 2024-04-12T20:58:24.180 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:58:24.182+0000 7f3e88600700 1 -- start start 2024-04-12T20:58:24.180 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:58:24.183+0000 7f3e88600700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3e800fcee0 0x7f3e8010fd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T20:58:24.180 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:58:24.183+0000 7f3e88600700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3e800fe0d0 0x7f3e80110260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T20:58:24.180 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:58:24.183+0000 7f3e88600700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e80110860 con 0x7f3e800fe0d0 2024-04-12T20:58:24.181 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:58:24.183+0000 7f3e88600700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e80110fa0 con 0x7f3e800fcee0 2024-04-12T20:58:24.181 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:58:24.183+0000 7f3e85200700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3e800fe0d0 0x7f3e80110260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T20:58:24.181 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:58:24.183+0000 7f3e85200700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3e800fe0d0 0x7f3e80110260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:41052/0 (socket says 172.21.15.18:41052) 2024-04-12T20:58:24.181 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:58:24.183+0000 7f3e85200700 1 -- 172.21.15.18:0/1844304562 learned_addr learned my addr 172.21.15.18:0/1844304562 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T20:58:24.181 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:58:24.183+0000 7f3e85c00700 1 --2- 172.21.15.18:0/1844304562 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3e800fcee0 0x7f3e8010fd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T20:58:24.181 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:58:24.184+0000 7f3e85200700 1 -- 172.21.15.18:0/1844304562 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3e800fcee0 msgr2=0x7f3e8010fd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:58:24.181 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:58:24.184+0000 7f3e85200700 1 --2- 172.21.15.18:0/1844304562 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3e800fcee0 0x7f3e8010fd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:58:24.181 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:58:24.184+0000 7f3e85200700 1 -- 172.21.15.18:0/1844304562 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e74005040 con 0x7f3e800fe0d0 2024-04-12T20:58:24.182 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:58:24.184+0000 7f3e85200700 1 --2- 172.21.15.18:0/1844304562 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3e800fe0d0 0x7f3e80110260 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f3e6c002a50 tx=0x7f3e6c0046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T20:58:24.182 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:58:24.184+0000 7f3e7d600700 1 -- 172.21.15.18:0/1844304562 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3e6c007cc0 con 0x7f3e800fe0d0 2024-04-12T20:58:24.182 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:58:24.184+0000 7f3e7d600700 1 -- 172.21.15.18:0/1844304562 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f3e6c00f040 con 0x7f3e800fe0d0 2024-04-12T20:58:24.182 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:58:24.184+0000 7f3e88600700 1 -- 172.21.15.18:0/1844304562 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3e80111280 con 0x7f3e800fe0d0 2024-04-12T20:58:24.182 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:58:24.184+0000 7f3e7d600700 1 -- 172.21.15.18:0/1844304562 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3e6c009640 con 0x7f3e800fe0d0 2024-04-12T20:58:24.182 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:58:24.184+0000 7f3e88600700 1 -- 172.21.15.18:0/1844304562 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3e80111840 con 0x7f3e800fe0d0 2024-04-12T20:58:24.183 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:58:24.186+0000 7f3e88600700 1 -- 172.21.15.18:0/1844304562 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3e80101a60 con 0x7f3e800fe0d0 2024-04-12T20:58:24.184 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:58:24.187+0000 7f3e7d600700 1 -- 172.21.15.18:0/1844304562 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 31) v1 ==== 92561+0+0 (secure 0 0 0) 0x7f3e6c0097a0 con 0x7f3e800fe0d0 2024-04-12T20:58:24.185 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:58:24.189+0000 7f3e7d600700 1 --2- 172.21.15.18:0/1844304562 >> [v2:172.21.15.18:6800/1605950634,v1:172.21.15.18:6801/1605950634] conn(0x7f3e6806e620 0x7f3e68070ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T20:58:24.185 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:58:24.189+0000 7f3e7d600700 1 -- 172.21.15.18:0/1844304562 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6066+0+0 (secure 0 0 0) 0x7f3e6c08de80 con 0x7f3e800fe0d0 2024-04-12T20:58:24.186 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:58:24.189+0000 7f3e85c00700 1 --2- 172.21.15.18:0/1844304562 >> [v2:172.21.15.18:6800/1605950634,v1:172.21.15.18:6801/1605950634] conn(0x7f3e6806e620 0x7f3e68070ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T20:58:24.186 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:58:24.190+0000 7f3e85c00700 1 --2- 172.21.15.18:0/1844304562 >> [v2:172.21.15.18:6800/1605950634,v1:172.21.15.18:6801/1605950634] conn(0x7f3e6806e620 0x7f3e68070ad0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f3e7400b3b0 tx=0x7f3e74009e30 comp rx=0 tx=0).ready entity=mgr.14672 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T20:58:24.190 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:58:24.193+0000 7f3e7d600700 1 -- 172.21.15.18:0/1844304562 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186454 (secure 0 0 0) 0x7f3e6c05ae60 con 0x7f3e800fe0d0 2024-04-12T20:58:24.462 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:58:24.466+0000 7f3e88600700 1 -- 172.21.15.18:0/1844304562 --> [v2:172.21.15.18:6800/1605950634,v1:172.21.15.18:6801/1605950634] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f3e80111ab0 con 0x7f3e6806e620 2024-04-12T20:58:24.465 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:58:24.468+0000 7f3e7d600700 1 -- 172.21.15.18:0/1844304562 <== mgr.14672 v2:172.21.15.18:6800/1605950634 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+370 (secure 0 0 0) 0x7f3e80111ab0 con 0x7f3e6806e620 2024-04-12T20:58:24.465 INFO:teuthology.orchestra.run.smithi018.stdout:{ 2024-04-12T20:58:24.465 INFO:teuthology.orchestra.run.smithi018.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:f4452def3be4d350f4a8e5feaf7ebb1511be9298fe68478ba7a2d6fbd7d69a63", 2024-04-12T20:58:24.465 INFO:teuthology.orchestra.run.smithi018.stdout: "in_progress": true, 2024-04-12T20:58:24.465 INFO:teuthology.orchestra.run.smithi018.stdout: "which": "Upgrading daemons of type(s) mgr", 2024-04-12T20:58:24.465 INFO:teuthology.orchestra.run.smithi018.stdout: "services_complete": [ 2024-04-12T20:58:24.466 INFO:teuthology.orchestra.run.smithi018.stdout: "mgr" 2024-04-12T20:58:24.466 INFO:teuthology.orchestra.run.smithi018.stdout: ], 2024-04-12T20:58:24.466 INFO:teuthology.orchestra.run.smithi018.stdout: "progress": "2/2 daemons upgraded", 2024-04-12T20:58:24.466 INFO:teuthology.orchestra.run.smithi018.stdout: "message": "Currently upgrading node-exporter daemons", 2024-04-12T20:58:24.466 INFO:teuthology.orchestra.run.smithi018.stdout: "is_paused": false 2024-04-12T20:58:24.466 INFO:teuthology.orchestra.run.smithi018.stdout:} 2024-04-12T20:58:24.468 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:58:24.471+0000 7f3e88600700 1 -- 172.21.15.18:0/1844304562 >> [v2:172.21.15.18:6800/1605950634,v1:172.21.15.18:6801/1605950634] conn(0x7f3e6806e620 msgr2=0x7f3e68070ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:58:24.468 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:58:24.471+0000 7f3e88600700 1 --2- 172.21.15.18:0/1844304562 >> [v2:172.21.15.18:6800/1605950634,v1:172.21.15.18:6801/1605950634] conn(0x7f3e6806e620 0x7f3e68070ad0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f3e7400b3b0 tx=0x7f3e74009e30 comp rx=0 tx=0).stop 2024-04-12T20:58:24.468 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:58:24.471+0000 7f3e88600700 1 -- 172.21.15.18:0/1844304562 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3e800fe0d0 msgr2=0x7f3e80110260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:58:24.468 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:58:24.471+0000 7f3e88600700 1 --2- 172.21.15.18:0/1844304562 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3e800fe0d0 0x7f3e80110260 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f3e6c002a50 tx=0x7f3e6c0046b0 comp rx=0 tx=0).stop 2024-04-12T20:58:24.469 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:58:24.471+0000 7f3e88600700 1 -- 172.21.15.18:0/1844304562 shutdown_connections 2024-04-12T20:58:24.469 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:58:24.471+0000 7f3e88600700 1 --2- 172.21.15.18:0/1844304562 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3e800fcee0 0x7f3e8010fd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:58:24.469 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:58:24.471+0000 7f3e88600700 1 --2- 172.21.15.18:0/1844304562 >> [v2:172.21.15.18:6800/1605950634,v1:172.21.15.18:6801/1605950634] conn(0x7f3e6806e620 0x7f3e68070ad0 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:58:24.469 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:58:24.471+0000 7f3e88600700 1 --2- 172.21.15.18:0/1844304562 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3e800fe0d0 0x7f3e80110260 secure :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f3e6c002a50 tx=0x7f3e6c0046b0 comp rx=0 tx=0).stop 2024-04-12T20:58:24.469 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:58:24.471+0000 7f3e88600700 1 -- 172.21.15.18:0/1844304562 >> 172.21.15.18:0/1844304562 conn(0x7f3e800f8560 msgr2=0x7f3e800f9c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T20:58:24.469 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:58:24.472+0000 7f3e88600700 1 -- 172.21.15.18:0/1844304562 shutdown_connections 2024-04-12T20:58:24.470 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:58:24.473+0000 7f3e88600700 1 -- 172.21.15.18:0/1844304562 wait complete. 2024-04-12T20:58:25.101 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:58:24 smithi018 ceph-mon[28799]: from='client.14750 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T20:58:25.101 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:58:24 smithi018 ceph-mon[28799]: from='client.14754 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T20:58:25.101 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:58:24 smithi018 ceph-mon[28799]: from='client.? 172.21.15.18:0/3406720837' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T20:58:25.280 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:58:24 smithi186 ceph-mon[36597]: from='client.14750 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T20:58:25.281 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:58:24 smithi186 ceph-mon[36597]: from='client.14754 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T20:58:25.281 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:58:24 smithi186 ceph-mon[36597]: from='client.? 172.21.15.18:0/3406720837' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T20:58:26.030 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:58:25 smithi186 ceph-mon[36597]: Upgrade: Updating node-exporter.smithi186 (2/2) 2024-04-12T20:58:26.030 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:58:25 smithi186 ceph-mon[36597]: Deploying daemon node-exporter.smithi186 on smithi186 2024-04-12T20:58:26.030 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:58:25 smithi186 ceph-mon[36597]: from='client.14762 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T20:58:26.030 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:58:25 smithi186 ceph-mon[36597]: pgmap v39: 65 pgs: 65 active+clean; 266 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 22 KiB/s rd, 651 KiB/s wr, 213 op/s 2024-04-12T20:58:26.101 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:58:25 smithi018 ceph-mon[28799]: Upgrade: Updating node-exporter.smithi186 (2/2) 2024-04-12T20:58:26.102 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:58:25 smithi018 ceph-mon[28799]: Deploying daemon node-exporter.smithi186 on smithi186 2024-04-12T20:58:26.102 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:58:25 smithi018 ceph-mon[28799]: from='client.14762 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T20:58:26.102 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:58:25 smithi018 ceph-mon[28799]: pgmap v39: 65 pgs: 65 active+clean; 266 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 22 KiB/s rd, 651 KiB/s wr, 213 op/s 2024-04-12T20:58:27.947 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:58:27 smithi186 ceph-mon[36597]: pgmap v40: 65 pgs: 65 active+clean; 266 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 10 KiB/s rd, 360 KiB/s wr, 105 op/s 2024-04-12T20:58:27.947 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:58:27 smithi186 ceph-mon[36597]: from='mgr.14672 172.21.15.18:0/2209228970' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T20:58:28.101 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:58:27 smithi018 ceph-mon[28799]: pgmap v40: 65 pgs: 65 active+clean; 266 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 10 KiB/s rd, 360 KiB/s wr, 105 op/s 2024-04-12T20:58:28.101 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:58:27 smithi018 ceph-mon[28799]: from='mgr.14672 172.21.15.18:0/2209228970' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T20:58:29.531 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:58:29 smithi186 ceph-mon[36597]: from='mgr.14672 172.21.15.18:0/2209228970' entity='mgr.smithi018.xjzwga' 2024-04-12T20:58:29.531 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:58:29 smithi186 ceph-mon[36597]: from='mgr.14672 172.21.15.18:0/2209228970' entity='mgr.smithi018.xjzwga' 2024-04-12T20:58:29.531 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:58:29 smithi186 ceph-mon[36597]: from='mgr.14672 172.21.15.18:0/2209228970' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-12T20:58:29.531 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:58:29 smithi186 ceph-mon[36597]: pgmap v41: 65 pgs: 65 active+clean; 265 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 10 KiB/s rd, 360 KiB/s wr, 110 op/s 2024-04-12T20:58:29.601 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:58:29 smithi018 ceph-mon[28799]: from='mgr.14672 172.21.15.18:0/2209228970' entity='mgr.smithi018.xjzwga' 2024-04-12T20:58:29.602 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:58:29 smithi018 ceph-mon[28799]: from='mgr.14672 172.21.15.18:0/2209228970' entity='mgr.smithi018.xjzwga' 2024-04-12T20:58:29.602 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:58:29 smithi018 ceph-mon[28799]: from='mgr.14672 172.21.15.18:0/2209228970' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-12T20:58:29.602 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:58:29 smithi018 ceph-mon[28799]: pgmap v41: 65 pgs: 65 active+clean; 265 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 10 KiB/s rd, 360 KiB/s wr, 110 op/s 2024-04-12T20:58:31.780 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:58:31 smithi186 ceph-mon[36597]: pgmap v42: 65 pgs: 65 active+clean; 265 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 15 KiB/s rd, 546 KiB/s wr, 158 op/s 2024-04-12T20:58:31.851 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:58:31 smithi018 ceph-mon[28799]: pgmap v42: 65 pgs: 65 active+clean; 265 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 15 KiB/s rd, 546 KiB/s wr, 158 op/s 2024-04-12T20:58:33.851 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:58:33 smithi018 ceph-mon[28799]: pgmap v43: 65 pgs: 65 active+clean; 265 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 10 KiB/s rd, 360 KiB/s wr, 103 op/s 2024-04-12T20:58:33.851 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:58:33 smithi018 ceph-mon[28799]: from='mgr.14672 172.21.15.18:0/2209228970' entity='mgr.smithi018.xjzwga' 2024-04-12T20:58:33.851 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:58:33 smithi018 ceph-mon[28799]: from='mgr.14672 172.21.15.18:0/2209228970' entity='mgr.smithi018.xjzwga' 2024-04-12T20:58:34.030 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:58:33 smithi186 ceph-mon[36597]: pgmap v43: 65 pgs: 65 active+clean; 265 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 10 KiB/s rd, 360 KiB/s wr, 103 op/s 2024-04-12T20:58:34.031 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:58:33 smithi186 ceph-mon[36597]: from='mgr.14672 172.21.15.18:0/2209228970' entity='mgr.smithi018.xjzwga' 2024-04-12T20:58:34.031 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:58:33 smithi186 ceph-mon[36597]: from='mgr.14672 172.21.15.18:0/2209228970' entity='mgr.smithi018.xjzwga' 2024-04-12T20:58:35.851 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:58:35 smithi018 ceph-mon[28799]: pgmap v44: 65 pgs: 65 active+clean; 262 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 15 KiB/s rd, 539 KiB/s wr, 158 op/s 2024-04-12T20:58:35.851 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:58:35 smithi018 ceph-mon[28799]: from='mgr.14672 172.21.15.18:0/2209228970' entity='mgr.smithi018.xjzwga' 2024-04-12T20:58:35.851 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:58:35 smithi018 ceph-mon[28799]: from='mgr.14672 172.21.15.18:0/2209228970' entity='mgr.smithi018.xjzwga' 2024-04-12T20:58:36.075 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:58:35 smithi186 ceph-mon[36597]: pgmap v44: 65 pgs: 65 active+clean; 262 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 15 KiB/s rd, 539 KiB/s wr, 158 op/s 2024-04-12T20:58:36.075 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:58:35 smithi186 ceph-mon[36597]: from='mgr.14672 172.21.15.18:0/2209228970' entity='mgr.smithi018.xjzwga' 2024-04-12T20:58:36.075 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:58:35 smithi186 ceph-mon[36597]: from='mgr.14672 172.21.15.18:0/2209228970' entity='mgr.smithi018.xjzwga' 2024-04-12T20:58:37.733 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:58:37 smithi186 ceph-mon[36597]: from='mgr.14672 172.21.15.18:0/2209228970' entity='mgr.smithi018.xjzwga' 2024-04-12T20:58:37.733 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:58:37 smithi186 ceph-mon[36597]: from='mgr.14672 172.21.15.18:0/2209228970' entity='mgr.smithi018.xjzwga' 2024-04-12T20:58:37.734 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:58:37 smithi186 ceph-mon[36597]: pgmap v45: 65 pgs: 65 active+clean; 262 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 10 KiB/s rd, 365 KiB/s wr, 108 op/s 2024-04-12T20:58:37.851 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:58:37 smithi018 ceph-mon[28799]: from='mgr.14672 172.21.15.18:0/2209228970' entity='mgr.smithi018.xjzwga' 2024-04-12T20:58:37.851 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:58:37 smithi018 ceph-mon[28799]: from='mgr.14672 172.21.15.18:0/2209228970' entity='mgr.smithi018.xjzwga' 2024-04-12T20:58:37.851 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:58:37 smithi018 ceph-mon[28799]: pgmap v45: 65 pgs: 65 active+clean; 262 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 10 KiB/s rd, 365 KiB/s wr, 108 op/s 2024-04-12T20:58:39.884 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:58:39 smithi018 ceph-mon[28799]: pgmap v46: 65 pgs: 65 active+clean; 263 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 10 KiB/s rd, 538 KiB/s wr, 129 op/s 2024-04-12T20:58:39.884 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:58:39 smithi018 ceph-mon[28799]: from='mgr.14672 172.21.15.18:0/2209228970' entity='mgr.smithi018.xjzwga' 2024-04-12T20:58:39.884 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:58:39 smithi018 ceph-mon[28799]: from='mgr.14672 172.21.15.18:0/2209228970' entity='mgr.smithi018.xjzwga' 2024-04-12T20:58:40.030 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:58:39 smithi186 ceph-mon[36597]: pgmap v46: 65 pgs: 65 active+clean; 263 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 10 KiB/s rd, 538 KiB/s wr, 129 op/s 2024-04-12T20:58:40.030 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:58:39 smithi186 ceph-mon[36597]: from='mgr.14672 172.21.15.18:0/2209228970' entity='mgr.smithi018.xjzwga' 2024-04-12T20:58:40.030 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:58:39 smithi186 ceph-mon[36597]: from='mgr.14672 172.21.15.18:0/2209228970' entity='mgr.smithi018.xjzwga' 2024-04-12T20:58:41.506 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:58:41 smithi018 ceph-mon[28799]: from='mgr.14672 172.21.15.18:0/2209228970' entity='mgr.smithi018.xjzwga' 2024-04-12T20:58:41.506 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:58:41 smithi018 ceph-mon[28799]: from='mgr.14672 172.21.15.18:0/2209228970' entity='mgr.smithi018.xjzwga' 2024-04-12T20:58:41.506 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:58:41 smithi018 ceph-mon[28799]: pgmap v47: 65 pgs: 65 active+clean; 262 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 15 KiB/s rd, 544 KiB/s wr, 158 op/s 2024-04-12T20:58:41.530 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:58:41 smithi186 ceph-mon[36597]: from='mgr.14672 172.21.15.18:0/2209228970' entity='mgr.smithi018.xjzwga' 2024-04-12T20:58:41.530 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:58:41 smithi186 ceph-mon[36597]: from='mgr.14672 172.21.15.18:0/2209228970' entity='mgr.smithi018.xjzwga' 2024-04-12T20:58:41.530 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:58:41 smithi186 ceph-mon[36597]: pgmap v47: 65 pgs: 65 active+clean; 262 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 15 KiB/s rd, 544 KiB/s wr, 158 op/s 2024-04-12T20:58:43.852 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:58:43 smithi018 ceph-mon[28799]: pgmap v48: 65 pgs: 65 active+clean; 262 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 10 KiB/s rd, 358 KiB/s wr, 110 op/s 2024-04-12T20:58:43.852 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:58:43 smithi018 ceph-mon[28799]: from='mgr.14672 172.21.15.18:0/2209228970' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T20:58:44.030 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:58:43 smithi186 ceph-mon[36597]: pgmap v48: 65 pgs: 65 active+clean; 262 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 10 KiB/s rd, 358 KiB/s wr, 110 op/s 2024-04-12T20:58:44.030 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:58:43 smithi186 ceph-mon[36597]: from='mgr.14672 172.21.15.18:0/2209228970' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T20:58:44.851 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:58:44 smithi018 ceph-mon[28799]: mgrmap e32: smithi018.xjzwga(active, since 91s), standbys: smithi186.kqcfxk 2024-04-12T20:58:45.030 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:58:44 smithi186 ceph-mon[36597]: mgrmap e32: smithi018.xjzwga(active, since 91s), standbys: smithi186.kqcfxk 2024-04-12T20:58:46.031 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:58:45 smithi186 ceph-mon[36597]: pgmap v49: 65 pgs: 65 active+clean; 261 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 15 KiB/s rd, 543 KiB/s wr, 166 op/s 2024-04-12T20:58:46.031 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:58:45 smithi186 ceph-mon[36597]: from='mgr.14672 172.21.15.18:0/2209228970' entity='mgr.smithi018.xjzwga' 2024-04-12T20:58:46.031 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:58:45 smithi186 ceph-mon[36597]: from='mgr.14672 172.21.15.18:0/2209228970' entity='mgr.smithi018.xjzwga' 2024-04-12T20:58:46.031 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:58:45 smithi186 ceph-mon[36597]: from='mgr.14672 172.21.15.18:0/2209228970' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-12T20:58:46.031 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:58:45 smithi186 ceph-mon[36597]: from='mgr.14672 172.21.15.18:0/2209228970' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-12T20:58:46.031 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:58:45 smithi186 ceph-mon[36597]: from='mgr.14672 172.21.15.18:0/2209228970' entity='mgr.smithi018.xjzwga' 2024-04-12T20:58:46.031 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:58:45 smithi186 ceph-mon[36597]: from='mgr.14672 172.21.15.18:0/2209228970' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-12T20:58:46.031 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:58:45 smithi186 ceph-mon[36597]: from='mgr.14672 172.21.15.18:0/2209228970' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T20:58:46.032 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:58:45 smithi186 ceph-mon[36597]: from='mgr.14672 172.21.15.18:0/2209228970' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T20:58:46.032 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:58:45 smithi186 ceph-mon[36597]: from='mgr.14672 172.21.15.18:0/2209228970' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T20:58:46.032 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:58:45 smithi186 ceph-mon[36597]: from='mgr.14672 172.21.15.18:0/2209228970' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T20:58:46.032 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:58:45 smithi186 ceph-mon[36597]: from='mgr.14672 172.21.15.18:0/2209228970' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T20:58:46.032 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:58:45 smithi186 ceph-mon[36597]: from='mgr.14672 172.21.15.18:0/2209228970' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T20:58:46.032 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:58:45 smithi186 ceph-mon[36597]: from='mgr.14672 172.21.15.18:0/2209228970' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T20:58:46.032 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:58:45 smithi186 ceph-mon[36597]: from='mgr.14672 172.21.15.18:0/2209228970' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T20:58:46.032 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:58:45 smithi186 ceph-mon[36597]: from='mgr.14672 172.21.15.18:0/2209228970' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T20:58:46.074 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:58:45 smithi018 ceph-mon[28799]: pgmap v49: 65 pgs: 65 active+clean; 261 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 15 KiB/s rd, 543 KiB/s wr, 166 op/s 2024-04-12T20:58:46.074 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:58:45 smithi018 ceph-mon[28799]: from='mgr.14672 172.21.15.18:0/2209228970' entity='mgr.smithi018.xjzwga' 2024-04-12T20:58:46.074 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:58:45 smithi018 ceph-mon[28799]: from='mgr.14672 172.21.15.18:0/2209228970' entity='mgr.smithi018.xjzwga' 2024-04-12T20:58:46.074 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:58:45 smithi018 ceph-mon[28799]: from='mgr.14672 172.21.15.18:0/2209228970' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-12T20:58:46.074 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:58:45 smithi018 ceph-mon[28799]: from='mgr.14672 172.21.15.18:0/2209228970' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-12T20:58:46.074 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:58:45 smithi018 ceph-mon[28799]: from='mgr.14672 172.21.15.18:0/2209228970' entity='mgr.smithi018.xjzwga' 2024-04-12T20:58:46.074 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:58:45 smithi018 ceph-mon[28799]: from='mgr.14672 172.21.15.18:0/2209228970' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-12T20:58:46.074 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:58:45 smithi018 ceph-mon[28799]: from='mgr.14672 172.21.15.18:0/2209228970' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T20:58:46.075 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:58:45 smithi018 ceph-mon[28799]: from='mgr.14672 172.21.15.18:0/2209228970' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T20:58:46.075 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:58:45 smithi018 ceph-mon[28799]: from='mgr.14672 172.21.15.18:0/2209228970' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T20:58:46.075 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:58:45 smithi018 ceph-mon[28799]: from='mgr.14672 172.21.15.18:0/2209228970' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T20:58:46.075 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:58:45 smithi018 ceph-mon[28799]: from='mgr.14672 172.21.15.18:0/2209228970' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T20:58:46.075 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:58:45 smithi018 ceph-mon[28799]: from='mgr.14672 172.21.15.18:0/2209228970' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T20:58:46.075 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:58:45 smithi018 ceph-mon[28799]: from='mgr.14672 172.21.15.18:0/2209228970' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T20:58:46.075 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:58:45 smithi018 ceph-mon[28799]: from='mgr.14672 172.21.15.18:0/2209228970' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T20:58:46.075 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:58:45 smithi018 ceph-mon[28799]: from='mgr.14672 172.21.15.18:0/2209228970' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T20:58:48.021 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:58:47 smithi018 ceph-mon[28799]: pgmap v50: 65 pgs: 65 active+clean; 261 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 10 KiB/s rd, 364 KiB/s wr, 110 op/s 2024-04-12T20:58:48.030 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:58:47 smithi186 ceph-mon[36597]: pgmap v50: 65 pgs: 65 active+clean; 261 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 10 KiB/s rd, 364 KiB/s wr, 110 op/s 2024-04-12T20:58:48.935 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:58:48 smithi018 ceph-mon[28799]: Upgrade: Updating prometheus.smithi018 2024-04-12T20:58:49.030 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:58:48 smithi186 ceph-mon[36597]: Upgrade: Updating prometheus.smithi018 2024-04-12T20:58:49.851 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:58:49 smithi018 ceph-mon[28799]: Deploying daemon prometheus.smithi018 on smithi018 2024-04-12T20:58:49.851 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:58:49 smithi018 ceph-mon[28799]: pgmap v51: 65 pgs: 65 active+clean; 263 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 10 KiB/s rd, 499 KiB/s wr, 121 op/s 2024-04-12T20:58:50.030 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:58:49 smithi186 ceph-mon[36597]: Deploying daemon prometheus.smithi018 on smithi018 2024-04-12T20:58:50.030 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:58:49 smithi186 ceph-mon[36597]: pgmap v51: 65 pgs: 65 active+clean; 263 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 10 KiB/s rd, 499 KiB/s wr, 121 op/s 2024-04-12T20:58:51.856 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:58:51 smithi018 ceph-mon[28799]: pgmap v52: 65 pgs: 65 active+clean; 259 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 15 KiB/s rd, 331 KiB/s wr, 133 op/s 2024-04-12T20:58:52.030 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:58:51 smithi186 ceph-mon[36597]: pgmap v52: 65 pgs: 65 active+clean; 259 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 15 KiB/s rd, 331 KiB/s wr, 133 op/s 2024-04-12T20:58:52.440 INFO:tasks.workunit:Stopping ['suites/fsstress.sh'] on client.1... 2024-04-12T20:58:52.440 DEBUG:teuthology.orchestra.run.smithi186:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.1 /home/ubuntu/cephtest/clone.client.1 2024-04-12T20:58:53.540 DEBUG:teuthology.parallel:result is None 2024-04-12T20:58:53.541 DEBUG:teuthology.orchestra.run.smithi018:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0 2024-04-12T20:58:53.648 INFO:tasks.workunit:Deleted dir /home/ubuntu/cephtest/mnt.0/client.0 2024-04-12T20:58:53.648 DEBUG:teuthology.orchestra.run.smithi186:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.1/client.1 2024-04-12T20:58:53.804 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:58:53 smithi186 ceph-mon[36597]: pgmap v53: 65 pgs: 65 active+clean; 259 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 9.7 KiB/s rd, 325 KiB/s wr, 99 op/s 2024-04-12T20:58:53.808 INFO:tasks.workunit:Deleted dir /home/ubuntu/cephtest/mnt.1/client.1 2024-04-12T20:58:53.808 DEBUG:teuthology.parallel:result is None 2024-04-12T20:58:53.851 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:58:53 smithi018 ceph-mon[28799]: pgmap v53: 65 pgs: 65 active+clean; 259 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 9.7 KiB/s rd, 325 KiB/s wr, 99 op/s 2024-04-12T20:58:54.635 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:58:54.635+0000 7f7051400700 1 -- 172.21.15.18:0/2893057681 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f704c0fce70 msgr2=0x7f704c0fd280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:58:54.635 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:58:54.635+0000 7f7051400700 1 --2- 172.21.15.18:0/2893057681 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f704c0fce70 0x7f704c0fd280 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f703c004940 tx=0x7f703c009e30 comp rx=0 tx=0).stop 2024-04-12T20:58:54.636 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:58:54.636+0000 7f7051400700 1 -- 172.21.15.18:0/2893057681 shutdown_connections 2024-04-12T20:58:54.636 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:58:54.636+0000 7f7051400700 1 --2- 172.21.15.18:0/2893057681 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f704c0fdfe0 0x7f704c0fe450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:58:54.636 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:58:54.636+0000 7f7051400700 1 --2- 172.21.15.18:0/2893057681 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f704c0fce70 0x7f704c0fd280 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:58:54.636 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:58:54.636+0000 7f7051400700 1 -- 172.21.15.18:0/2893057681 >> 172.21.15.18:0/2893057681 conn(0x7f704c0f8550 msgr2=0x7f704c0fa960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T20:58:54.636 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:58:54.636+0000 7f7051400700 1 -- 172.21.15.18:0/2893057681 shutdown_connections 2024-04-12T20:58:54.636 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:58:54.636+0000 7f7051400700 1 -- 172.21.15.18:0/2893057681 wait complete. 2024-04-12T20:58:54.636 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:58:54.637+0000 7f7051400700 1 Processor -- start 2024-04-12T20:58:54.636 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:58:54.637+0000 7f7051400700 1 -- start start 2024-04-12T20:58:54.637 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:58:54.637+0000 7f7051400700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f704c0fce70 0x7f704c10fb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T20:58:54.637 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:58:54.637+0000 7f7051400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f704c0fdfe0 0x7f704c110070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T20:58:54.637 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:58:54.637+0000 7f7051400700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f704c110700 con 0x7f704c0fdfe0 2024-04-12T20:58:54.637 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:58:54.637+0000 7f7051400700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f704c1150a0 con 0x7f704c0fce70 2024-04-12T20:58:54.638 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:58:54.638+0000 7f7049e00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f704c0fdfe0 0x7f704c110070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T20:58:54.639 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:58:54.638+0000 7f7049e00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f704c0fdfe0 0x7f704c110070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:56108/0 (socket says 172.21.15.18:56108) 2024-04-12T20:58:54.639 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:58:54.638+0000 7f7049e00700 1 -- 172.21.15.18:0/1914660784 learned_addr learned my addr 172.21.15.18:0/1914660784 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T20:58:54.639 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:58:54.638+0000 7f704a800700 1 --2- 172.21.15.18:0/1914660784 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f704c0fce70 0x7f704c10fb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T20:58:54.639 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:58:54.638+0000 7f7049e00700 1 -- 172.21.15.18:0/1914660784 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f704c0fce70 msgr2=0x7f704c10fb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:58:54.639 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:58:54.638+0000 7f7049e00700 1 --2- 172.21.15.18:0/1914660784 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f704c0fce70 0x7f704c10fb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:58:54.639 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:58:54.638+0000 7f7049e00700 1 -- 172.21.15.18:0/1914660784 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f703c005040 con 0x7f704c0fdfe0 2024-04-12T20:58:54.639 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:58:54.639+0000 7f7049e00700 1 --2- 172.21.15.18:0/1914660784 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f704c0fdfe0 0x7f704c110070 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f703800b790 tx=0x7f7038009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T20:58:54.639 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:58:54.639+0000 7f7042a00700 1 -- 172.21.15.18:0/1914660784 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f703800f770 con 0x7f704c0fdfe0 2024-04-12T20:58:54.639 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:58:54.639+0000 7f7042a00700 1 -- 172.21.15.18:0/1914660784 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f703800fda0 con 0x7f704c0fdfe0 2024-04-12T20:58:54.639 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:58:54.639+0000 7f7042a00700 1 -- 172.21.15.18:0/1914660784 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f703800d310 con 0x7f704c0fdfe0 2024-04-12T20:58:54.639 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:58:54.639+0000 7f7051400700 1 -- 172.21.15.18:0/1914660784 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f704c115300 con 0x7f704c0fdfe0 2024-04-12T20:58:54.639 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:58:54.639+0000 7f7051400700 1 -- 172.21.15.18:0/1914660784 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f704c115870 con 0x7f704c0fdfe0 2024-04-12T20:58:54.639 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:58:54.640+0000 7f7051400700 1 -- 172.21.15.18:0/1914660784 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f704c101970 con 0x7f704c0fdfe0 2024-04-12T20:58:54.640 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:58:54.642+0000 7f7042a00700 1 -- 172.21.15.18:0/1914660784 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 32) v1 ==== 92459+0+0 (secure 0 0 0) 0x7f7038010420 con 0x7f704c0fdfe0 2024-04-12T20:58:54.641 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:58:54.643+0000 7f7042a00700 1 --2- 172.21.15.18:0/1914660784 >> [v2:172.21.15.18:6800/1605950634,v1:172.21.15.18:6801/1605950634] conn(0x7f703006e500 0x7f70300709b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T20:58:54.641 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:58:54.643+0000 7f7042a00700 1 -- 172.21.15.18:0/1914660784 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6066+0+0 (secure 0 0 0) 0x7f703808dfb0 con 0x7f704c0fdfe0 2024-04-12T20:58:54.641 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:58:54.643+0000 7f704a800700 1 --2- 172.21.15.18:0/1914660784 >> [v2:172.21.15.18:6800/1605950634,v1:172.21.15.18:6801/1605950634] conn(0x7f703006e500 0x7f70300709b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T20:58:54.641 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:58:54.644+0000 7f704a800700 1 --2- 172.21.15.18:0/1914660784 >> [v2:172.21.15.18:6800/1605950634,v1:172.21.15.18:6801/1605950634] conn(0x7f703006e500 0x7f70300709b0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f703c00b3b0 tx=0x7f703c009e30 comp rx=0 tx=0).ready entity=mgr.14672 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T20:58:54.646 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:58:54.648+0000 7f7042a00700 1 -- 172.21.15.18:0/1914660784 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186454 (secure 0 0 0) 0x7f70380c2870 con 0x7f704c0fdfe0 2024-04-12T20:58:54.907 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:58:54.909+0000 7f7051400700 1 -- 172.21.15.18:0/1914660784 --> [v2:172.21.15.18:6800/1605950634,v1:172.21.15.18:6801/1605950634] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f704c02b420 con 0x7f703006e500 2024-04-12T20:58:54.910 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:58:54.912+0000 7f7042a00700 1 -- 172.21.15.18:0/1914660784 <== mgr.14672 v2:172.21.15.18:6800/1605950634 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+367 (secure 0 0 0) 0x7f704c02b420 con 0x7f703006e500 2024-04-12T20:58:54.913 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:58:54.915+0000 7f7051400700 1 -- 172.21.15.18:0/1914660784 >> [v2:172.21.15.18:6800/1605950634,v1:172.21.15.18:6801/1605950634] conn(0x7f703006e500 msgr2=0x7f70300709b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:58:54.913 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:58:54.915+0000 7f7051400700 1 --2- 172.21.15.18:0/1914660784 >> [v2:172.21.15.18:6800/1605950634,v1:172.21.15.18:6801/1605950634] conn(0x7f703006e500 0x7f70300709b0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f703c00b3b0 tx=0x7f703c009e30 comp rx=0 tx=0).stop 2024-04-12T20:58:54.913 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:58:54.915+0000 7f7051400700 1 -- 172.21.15.18:0/1914660784 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f704c0fdfe0 msgr2=0x7f704c110070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:58:54.913 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:58:54.915+0000 7f7051400700 1 --2- 172.21.15.18:0/1914660784 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f704c0fdfe0 0x7f704c110070 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f703800b790 tx=0x7f7038009250 comp rx=0 tx=0).stop 2024-04-12T20:58:54.913 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:58:54.915+0000 7f7051400700 1 -- 172.21.15.18:0/1914660784 shutdown_connections 2024-04-12T20:58:54.913 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:58:54.915+0000 7f7051400700 1 --2- 172.21.15.18:0/1914660784 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f704c0fce70 0x7f704c10fb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:58:54.913 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:58:54.915+0000 7f7051400700 1 --2- 172.21.15.18:0/1914660784 >> [v2:172.21.15.18:6800/1605950634,v1:172.21.15.18:6801/1605950634] conn(0x7f703006e500 0x7f70300709b0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:58:54.914 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:58:54.915+0000 7f7051400700 1 --2- 172.21.15.18:0/1914660784 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f704c0fdfe0 0x7f704c110070 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:58:54.914 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:58:54.915+0000 7f7051400700 1 -- 172.21.15.18:0/1914660784 >> 172.21.15.18:0/1914660784 conn(0x7f704c0f8550 msgr2=0x7f704c0f9a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T20:58:54.914 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:58:54.915+0000 7f7051400700 1 -- 172.21.15.18:0/1914660784 shutdown_connections 2024-04-12T20:58:54.914 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:58:54.915+0000 7f7051400700 1 -- 172.21.15.18:0/1914660784 wait complete. 2024-04-12T20:58:54.929 INFO:teuthology.orchestra.run.smithi018.stdout:true 2024-04-12T20:58:55.069 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:58:55.071+0000 7f8f29200700 1 -- 172.21.15.18:0/638528331 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8f240fce70 msgr2=0x7f8f240fd280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:58:55.069 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:58:55.071+0000 7f8f29200700 1 --2- 172.21.15.18:0/638528331 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8f240fce70 0x7f8f240fd280 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f8f14004940 tx=0x7f8f14009e30 comp rx=0 tx=0).stop 2024-04-12T20:58:55.069 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:58:55.071+0000 7f8f29200700 1 -- 172.21.15.18:0/638528331 shutdown_connections 2024-04-12T20:58:55.069 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:58:55.071+0000 7f8f29200700 1 --2- 172.21.15.18:0/638528331 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8f240fdfe0 0x7f8f240fe450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:58:55.069 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:58:55.071+0000 7f8f29200700 1 --2- 172.21.15.18:0/638528331 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8f240fce70 0x7f8f240fd280 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:58:55.069 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:58:55.071+0000 7f8f29200700 1 -- 172.21.15.18:0/638528331 >> 172.21.15.18:0/638528331 conn(0x7f8f240f8550 msgr2=0x7f8f240fa960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T20:58:55.070 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:58:55.071+0000 7f8f29200700 1 -- 172.21.15.18:0/638528331 shutdown_connections 2024-04-12T20:58:55.070 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:58:55.072+0000 7f8f29200700 1 -- 172.21.15.18:0/638528331 wait complete. 2024-04-12T20:58:55.070 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:58:55.072+0000 7f8f29200700 1 Processor -- start 2024-04-12T20:58:55.070 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:58:55.073+0000 7f8f29200700 1 -- start start 2024-04-12T20:58:55.071 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:58:55.073+0000 7f8f29200700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8f240fce70 0x7f8f2410d950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T20:58:55.071 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:58:55.073+0000 7f8f29200700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8f240fdfe0 0x7f8f2410de90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T20:58:55.071 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:58:55.073+0000 7f8f29200700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f2410e490 con 0x7f8f240fce70 2024-04-12T20:58:55.071 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:58:55.073+0000 7f8f29200700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f2410e600 con 0x7f8f240fdfe0 2024-04-12T20:58:55.071 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:58:55.073+0000 7f8f21e00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8f240fce70 0x7f8f2410d950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T20:58:55.071 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:58:55.073+0000 7f8f21e00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8f240fce70 0x7f8f2410d950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:56134/0 (socket says 172.21.15.18:56134) 2024-04-12T20:58:55.071 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:58:55.073+0000 7f8f21e00700 1 -- 172.21.15.18:0/3789666926 learned_addr learned my addr 172.21.15.18:0/3789666926 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T20:58:55.071 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:58:55.074+0000 7f8f21400700 1 --2- 172.21.15.18:0/3789666926 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8f240fdfe0 0x7f8f2410de90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T20:58:55.072 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:58:55.074+0000 7f8f21e00700 1 -- 172.21.15.18:0/3789666926 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8f240fdfe0 msgr2=0x7f8f2410de90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:58:55.072 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:58:55.074+0000 7f8f21e00700 1 --2- 172.21.15.18:0/3789666926 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8f240fdfe0 0x7f8f2410de90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:58:55.073 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:58:55.074+0000 7f8f21e00700 1 -- 172.21.15.18:0/3789666926 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f14005040 con 0x7f8f240fce70 2024-04-12T20:58:55.073 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:58:55.074+0000 7f8f21400700 1 --2- 172.21.15.18:0/3789666926 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8f240fdfe0 0x7f8f2410de90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-12T20:58:55.073 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:58:55.074+0000 7f8f21e00700 1 --2- 172.21.15.18:0/3789666926 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8f240fce70 0x7f8f2410d950 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f8f1400b3b0 tx=0x7f8f14009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T20:58:55.073 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:58:55.075+0000 7f8f1a000700 1 -- 172.21.15.18:0/3789666926 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8f1400e040 con 0x7f8f240fce70 2024-04-12T20:58:55.073 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:58:55.075+0000 7f8f1a000700 1 -- 172.21.15.18:0/3789666926 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f8f14004290 con 0x7f8f240fce70 2024-04-12T20:58:55.073 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:58:55.075+0000 7f8f1a000700 1 -- 172.21.15.18:0/3789666926 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8f1401c630 con 0x7f8f240fce70 2024-04-12T20:58:55.073 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:58:55.075+0000 7f8f29200700 1 -- 172.21.15.18:0/3789666926 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8f24113000 con 0x7f8f240fce70 2024-04-12T20:58:55.073 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:58:55.075+0000 7f8f29200700 1 -- 172.21.15.18:0/3789666926 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8f241135c0 con 0x7f8f240fce70 2024-04-12T20:58:55.074 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:58:55.076+0000 7f8f29200700 1 -- 172.21.15.18:0/3789666926 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8f2404c710 con 0x7f8f240fce70 2024-04-12T20:58:55.074 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:58:55.077+0000 7f8f1a000700 1 -- 172.21.15.18:0/3789666926 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 32) v1 ==== 92459+0+0 (secure 0 0 0) 0x7f8f1400bbf0 con 0x7f8f240fce70 2024-04-12T20:58:55.076 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:58:55.078+0000 7f8f1a000700 1 --2- 172.21.15.18:0/3789666926 >> [v2:172.21.15.18:6800/1605950634,v1:172.21.15.18:6801/1605950634] conn(0x7f8f0806e500 0x7f8f080709b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T20:58:55.076 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:58:55.078+0000 7f8f1a000700 1 -- 172.21.15.18:0/3789666926 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6066+0+0 (secure 0 0 0) 0x7f8f1408efb0 con 0x7f8f240fce70 2024-04-12T20:58:55.076 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:58:55.078+0000 7f8f21400700 1 --2- 172.21.15.18:0/3789666926 >> [v2:172.21.15.18:6800/1605950634,v1:172.21.15.18:6801/1605950634] conn(0x7f8f0806e500 0x7f8f080709b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T20:58:55.077 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:58:55.079+0000 7f8f21400700 1 --2- 172.21.15.18:0/3789666926 >> [v2:172.21.15.18:6800/1605950634,v1:172.21.15.18:6801/1605950634] conn(0x7f8f0806e500 0x7f8f080709b0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f8f10009fd0 tx=0x7f8f10009ee0 comp rx=0 tx=0).ready entity=mgr.14672 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T20:58:55.081 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:58:55.084+0000 7f8f1a000700 1 -- 172.21.15.18:0/3789666926 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186454 (secure 0 0 0) 0x7f8f1405c250 con 0x7f8f240fce70 2024-04-12T20:58:55.365 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:58:55.367+0000 7f8f29200700 1 -- 172.21.15.18:0/3789666926 --> [v2:172.21.15.18:6800/1605950634,v1:172.21.15.18:6801/1605950634] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f8f24102340 con 0x7f8f0806e500 2024-04-12T20:58:55.368 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:58:55.371+0000 7f8f1a000700 1 -- 172.21.15.18:0/3789666926 <== mgr.14672 v2:172.21.15.18:6800/1605950634 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+367 (secure 0 0 0) 0x7f8f24102340 con 0x7f8f0806e500 2024-04-12T20:58:55.371 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:58:55.374+0000 7f8f29200700 1 -- 172.21.15.18:0/3789666926 >> [v2:172.21.15.18:6800/1605950634,v1:172.21.15.18:6801/1605950634] conn(0x7f8f0806e500 msgr2=0x7f8f080709b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:58:55.371 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:58:55.374+0000 7f8f29200700 1 --2- 172.21.15.18:0/3789666926 >> [v2:172.21.15.18:6800/1605950634,v1:172.21.15.18:6801/1605950634] conn(0x7f8f0806e500 0x7f8f080709b0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f8f10009fd0 tx=0x7f8f10009ee0 comp rx=0 tx=0).stop 2024-04-12T20:58:55.372 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:58:55.374+0000 7f8f29200700 1 -- 172.21.15.18:0/3789666926 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8f240fce70 msgr2=0x7f8f2410d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:58:55.372 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:58:55.374+0000 7f8f29200700 1 --2- 172.21.15.18:0/3789666926 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8f240fce70 0x7f8f2410d950 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f8f1400b3b0 tx=0x7f8f14009e30 comp rx=0 tx=0).stop 2024-04-12T20:58:55.372 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:58:55.374+0000 7f8f29200700 1 -- 172.21.15.18:0/3789666926 shutdown_connections 2024-04-12T20:58:55.372 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:58:55.374+0000 7f8f29200700 1 --2- 172.21.15.18:0/3789666926 >> [v2:172.21.15.18:6800/1605950634,v1:172.21.15.18:6801/1605950634] conn(0x7f8f0806e500 0x7f8f080709b0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:58:55.372 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:58:55.374+0000 7f8f29200700 1 --2- 172.21.15.18:0/3789666926 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8f240fce70 0x7f8f2410d950 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:58:55.373 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:58:55.374+0000 7f8f29200700 1 --2- 172.21.15.18:0/3789666926 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8f240fdfe0 0x7f8f2410de90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:58:55.373 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:58:55.374+0000 7f8f29200700 1 -- 172.21.15.18:0/3789666926 >> 172.21.15.18:0/3789666926 conn(0x7f8f240f8550 msgr2=0x7f8f240f9c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T20:58:55.373 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:58:55.374+0000 7f8f29200700 1 -- 172.21.15.18:0/3789666926 shutdown_connections 2024-04-12T20:58:55.373 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:58:55.374+0000 7f8f29200700 1 -- 172.21.15.18:0/3789666926 wait complete. 2024-04-12T20:58:55.557 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:58:55.554+0000 7f2572200700 1 -- 172.21.15.18:0/1912511096 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f256c0fdec0 msgr2=0x7f256c0fe330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:58:55.557 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:58:55.554+0000 7f2572200700 1 --2- 172.21.15.18:0/1912511096 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f256c0fdec0 0x7f256c0fe330 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f2558004940 tx=0x7f2558009e30 comp rx=0 tx=0).stop 2024-04-12T20:58:55.557 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:58:55.554+0000 7f2572200700 1 -- 172.21.15.18:0/1912511096 shutdown_connections 2024-04-12T20:58:55.557 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:58:55.554+0000 7f2572200700 1 --2- 172.21.15.18:0/1912511096 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f256c0fdec0 0x7f256c0fe330 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:58:55.558 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:58:55.554+0000 7f2572200700 1 --2- 172.21.15.18:0/1912511096 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f256c0fcc10 0x7f256c0fd020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:58:55.558 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:58:55.554+0000 7f2572200700 1 -- 172.21.15.18:0/1912511096 >> 172.21.15.18:0/1912511096 conn(0x7f256c0f83d0 msgr2=0x7f256c0fa7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T20:58:55.558 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:58:55.554+0000 7f2572200700 1 -- 172.21.15.18:0/1912511096 shutdown_connections 2024-04-12T20:58:55.558 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:58:55.554+0000 7f2572200700 1 -- 172.21.15.18:0/1912511096 wait complete. 2024-04-12T20:58:55.558 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:58:55.555+0000 7f2572200700 1 Processor -- start 2024-04-12T20:58:55.558 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:58:55.555+0000 7f2572200700 1 -- start start 2024-04-12T20:58:55.558 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:58:55.555+0000 7f2572200700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f256c0fcc10 0x7f256c10fa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T20:58:55.558 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:58:55.555+0000 7f2572200700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f256c0fdec0 0x7f256c10ffb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T20:58:55.559 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:58:55.555+0000 7f2572200700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f256c1105b0 con 0x7f256c0fdec0 2024-04-12T20:58:55.559 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:58:55.555+0000 7f2572200700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f256c110e00 con 0x7f256c0fcc10 2024-04-12T20:58:55.559 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:58:55.556+0000 7f256be00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f256c0fdec0 0x7f256c10ffb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T20:58:55.559 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:58:55.556+0000 7f256be00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f256c0fdec0 0x7f256c10ffb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:56150/0 (socket says 172.21.15.18:56150) 2024-04-12T20:58:55.559 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:58:55.556+0000 7f256be00700 1 -- 172.21.15.18:0/3890374919 learned_addr learned my addr 172.21.15.18:0/3890374919 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T20:58:55.559 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:58:55.556+0000 7f2570e00700 1 --2- 172.21.15.18:0/3890374919 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f256c0fcc10 0x7f256c10fa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T20:58:55.559 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:58:55.556+0000 7f256be00700 1 -- 172.21.15.18:0/3890374919 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f256c0fcc10 msgr2=0x7f256c10fa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:58:55.559 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:58:55.556+0000 7f256be00700 1 --2- 172.21.15.18:0/3890374919 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f256c0fcc10 0x7f256c10fa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:58:55.559 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:58:55.556+0000 7f256be00700 1 -- 172.21.15.18:0/3890374919 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2558005040 con 0x7f256c0fdec0 2024-04-12T20:58:55.560 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:58:55.556+0000 7f2570e00700 1 --2- 172.21.15.18:0/3890374919 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f256c0fcc10 0x7f256c10fa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-12T20:58:55.560 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:58:55.557+0000 7f256be00700 1 --2- 172.21.15.18:0/3890374919 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f256c0fdec0 0x7f256c10ffb0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f255800b3b0 tx=0x7f2558009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T20:58:55.560 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:58:55.557+0000 7f2568c00700 1 -- 172.21.15.18:0/3890374919 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f255800e040 con 0x7f256c0fdec0 2024-04-12T20:58:55.560 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:58:55.557+0000 7f2572200700 1 -- 172.21.15.18:0/3890374919 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f256c111080 con 0x7f256c0fdec0 2024-04-12T20:58:55.560 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:58:55.557+0000 7f2568c00700 1 -- 172.21.15.18:0/3890374919 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f255800bcc0 con 0x7f256c0fdec0 2024-04-12T20:58:55.560 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:58:55.557+0000 7f2572200700 1 -- 172.21.15.18:0/3890374919 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f256c1115c0 con 0x7f256c0fdec0 2024-04-12T20:58:55.560 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:58:55.557+0000 7f2568c00700 1 -- 172.21.15.18:0/3890374919 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2558026620 con 0x7f256c0fdec0 2024-04-12T20:58:55.560 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:58:55.558+0000 7f2572200700 1 -- 172.21.15.18:0/3890374919 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f256c101850 con 0x7f256c0fdec0 2024-04-12T20:58:55.561 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:58:55.559+0000 7f2568c00700 1 -- 172.21.15.18:0/3890374919 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 32) v1 ==== 92459+0+0 (secure 0 0 0) 0x7f2558013980 con 0x7f256c0fdec0 2024-04-12T20:58:55.561 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:58:55.560+0000 7f2568c00700 1 --2- 172.21.15.18:0/3890374919 >> [v2:172.21.15.18:6800/1605950634,v1:172.21.15.18:6801/1605950634] conn(0x7f255406e500 0x7f25540709b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T20:58:55.561 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:58:55.560+0000 7f2568c00700 1 -- 172.21.15.18:0/3890374919 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6066+0+0 (secure 0 0 0) 0x7f255808eaf0 con 0x7f256c0fdec0 2024-04-12T20:58:55.561 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:58:55.561+0000 7f2570e00700 1 --2- 172.21.15.18:0/3890374919 >> [v2:172.21.15.18:6800/1605950634,v1:172.21.15.18:6801/1605950634] conn(0x7f255406e500 0x7f25540709b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T20:58:55.561 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:58:55.562+0000 7f2570e00700 1 --2- 172.21.15.18:0/3890374919 >> [v2:172.21.15.18:6800/1605950634,v1:172.21.15.18:6801/1605950634] conn(0x7f255406e500 0x7f25540709b0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f2560009fd0 tx=0x7f256000b040 comp rx=0 tx=0).ready entity=mgr.14672 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T20:58:55.567 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:58:55.569+0000 7f2568c00700 1 -- 172.21.15.18:0/3890374919 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186454 (secure 0 0 0) 0x7f255805bd20 con 0x7f256c0fdec0 2024-04-12T20:58:55.821 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:58:55.823+0000 7f2572200700 1 -- 172.21.15.18:0/3890374919 --> [v2:172.21.15.18:6800/1605950634,v1:172.21.15.18:6801/1605950634] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f256c1058d0 con 0x7f255406e500 2024-04-12T20:58:55.834 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:58:55.837+0000 7f2568c00700 1 -- 172.21.15.18:0/3890374919 <== mgr.14672 v2:172.21.15.18:6800/1605950634 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7f256000f040 con 0x7f255406e500 2024-04-12T20:58:55.835 INFO:teuthology.orchestra.run.smithi018.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-12T20:58:55.835 INFO:teuthology.orchestra.run.smithi018.stdout:alertmanager.smithi018 smithi018 *:9093,9094 running (3m) 19s ago 13m 15.4M - 0.25.0 c8568f914cd2 b4ed63d191d4 2024-04-12T20:58:55.835 INFO:teuthology.orchestra.run.smithi018.stdout:crash.smithi018 smithi018 running (14m) 19s ago 14m 6660k - 17.2.7-827-gd3b5060a 21de9b7369ad 1ac7603bddfd 2024-04-12T20:58:55.835 INFO:teuthology.orchestra.run.smithi018.stdout:crash.smithi186 smithi186 running (12m) 23s ago 12m 6669k - 17.2.7-827-gd3b5060a 21de9b7369ad 14bf8a1672eb 2024-04-12T20:58:55.835 INFO:teuthology.orchestra.run.smithi018.stdout:grafana.smithi018 smithi018 *:3000 running (3m) 19s ago 13m 76.2M - 9.4.7 954c08fa6188 d6251dcdaec0 2024-04-12T20:58:55.835 INFO:teuthology.orchestra.run.smithi018.stdout:mds.cephfs.smithi018.acqbsc smithi018 running (6m) 19s ago 6m 113M - 17.2.7-827-gd3b5060a 21de9b7369ad 5b5fce65f18a 2024-04-12T20:58:55.835 INFO:teuthology.orchestra.run.smithi018.stdout:mds.cephfs.smithi018.szjluo smithi018 running (6m) 19s ago 6m 125M - 17.2.7-827-gd3b5060a 21de9b7369ad cf9bf0c5031f 2024-04-12T20:58:55.835 INFO:teuthology.orchestra.run.smithi018.stdout:mds.cephfs.smithi186.jdllyn smithi186 running (6m) 23s ago 6m 13.0M - 17.2.7-827-gd3b5060a 21de9b7369ad 2b2bc46bd29a 2024-04-12T20:58:55.835 INFO:teuthology.orchestra.run.smithi018.stdout:mds.cephfs.smithi186.wjnafs smithi186 running (6m) 23s ago 6m 13.0M - 17.2.7-827-gd3b5060a 21de9b7369ad 74b115d6d6fd 2024-04-12T20:58:55.835 INFO:teuthology.orchestra.run.smithi018.stdout:mgr.smithi018.xjzwga smithi018 *:8443,9283,8765 running (2m) 19s ago 15m 496M - 19.0.0-2879-g324baf8d 8717720dc5e5 56088305eab7 2024-04-12T20:58:55.836 INFO:teuthology.orchestra.run.smithi018.stdout:mgr.smithi186.kqcfxk smithi186 *:8443,9283,8765 running (55s) 23s ago 12m 447M - 19.0.0-2879-g324baf8d 8717720dc5e5 09cd766e5e0d 2024-04-12T20:58:55.836 INFO:teuthology.orchestra.run.smithi018.stdout:mon.smithi018 smithi018 running (15m) 19s ago 15m 53.6M 2048M 17.2.7-827-gd3b5060a 21de9b7369ad cd84caafa056 2024-04-12T20:58:55.836 INFO:teuthology.orchestra.run.smithi018.stdout:mon.smithi186 smithi186 running (12m) 23s ago 12m 41.6M 2048M 17.2.7-827-gd3b5060a 21de9b7369ad d7263c772b73 2024-04-12T20:58:55.836 INFO:teuthology.orchestra.run.smithi018.stdout:node-exporter.smithi018 smithi018 *:9100 running (35s) 19s ago 14m 6773k - 1.5.0 0da6a335fe13 326c3d9c6847 2024-04-12T20:58:55.836 INFO:teuthology.orchestra.run.smithi018.stdout:node-exporter.smithi186 smithi186 *:9100 running (28s) 23s ago 12m 2705k - 1.5.0 0da6a335fe13 ec2112baeecf 2024-04-12T20:58:55.836 INFO:teuthology.orchestra.run.smithi018.stdout:osd.0 smithi018 running (11m) 19s ago 11m 218M 1559M 17.2.7-827-gd3b5060a 21de9b7369ad 0d83e027bf0a 2024-04-12T20:58:55.836 INFO:teuthology.orchestra.run.smithi018.stdout:osd.1 smithi018 running (11m) 19s ago 11m 237M 1559M 17.2.7-827-gd3b5060a 21de9b7369ad 6517c9cc7f87 2024-04-12T20:58:55.836 INFO:teuthology.orchestra.run.smithi018.stdout:osd.2 smithi018 running (10m) 19s ago 10m 186M 1559M 17.2.7-827-gd3b5060a 21de9b7369ad 8a83692acaa5 2024-04-12T20:58:55.836 INFO:teuthology.orchestra.run.smithi018.stdout:osd.3 smithi186 running (9m) 23s ago 9m 258M 2581M 17.2.7-827-gd3b5060a 21de9b7369ad c2def7a90d76 2024-04-12T20:58:55.836 INFO:teuthology.orchestra.run.smithi018.stdout:osd.4 smithi186 running (9m) 23s ago 9m 228M 2581M 17.2.7-827-gd3b5060a 21de9b7369ad 757e7c71b7b9 2024-04-12T20:58:55.836 INFO:teuthology.orchestra.run.smithi018.stdout:osd.5 smithi186 running (8m) 23s ago 8m 222M 2581M 17.2.7-827-gd3b5060a 21de9b7369ad 2ee072a436d0 2024-04-12T20:58:55.836 INFO:teuthology.orchestra.run.smithi018.stdout:prometheus.smithi018 smithi018 *:9095 running (64s) 19s ago 13m 46.6M - 2.43.0 a07b618ecd1d 463fc59e8c0e 2024-04-12T20:58:55.838 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:58:55.840+0000 7f2572200700 1 -- 172.21.15.18:0/3890374919 >> [v2:172.21.15.18:6800/1605950634,v1:172.21.15.18:6801/1605950634] conn(0x7f255406e500 msgr2=0x7f25540709b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:58:55.838 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:58:55.840+0000 7f2572200700 1 --2- 172.21.15.18:0/3890374919 >> [v2:172.21.15.18:6800/1605950634,v1:172.21.15.18:6801/1605950634] conn(0x7f255406e500 0x7f25540709b0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f2560009fd0 tx=0x7f256000b040 comp rx=0 tx=0).stop 2024-04-12T20:58:55.838 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:58:55.840+0000 7f2572200700 1 -- 172.21.15.18:0/3890374919 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f256c0fdec0 msgr2=0x7f256c10ffb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:58:55.838 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:58:55.840+0000 7f2572200700 1 --2- 172.21.15.18:0/3890374919 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f256c0fdec0 0x7f256c10ffb0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f255800b3b0 tx=0x7f2558009e30 comp rx=0 tx=0).stop 2024-04-12T20:58:55.838 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:58:55.840+0000 7f2572200700 1 -- 172.21.15.18:0/3890374919 shutdown_connections 2024-04-12T20:58:55.839 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:58:55.840+0000 7f2572200700 1 --2- 172.21.15.18:0/3890374919 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f256c0fcc10 0x7f256c10fa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:58:55.839 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:58:55.840+0000 7f2572200700 1 --2- 172.21.15.18:0/3890374919 >> [v2:172.21.15.18:6800/1605950634,v1:172.21.15.18:6801/1605950634] conn(0x7f255406e500 0x7f25540709b0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:58:55.839 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:58:55.840+0000 7f2572200700 1 --2- 172.21.15.18:0/3890374919 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f256c0fdec0 0x7f256c10ffb0 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:58:55.839 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:58:55.840+0000 7f2572200700 1 -- 172.21.15.18:0/3890374919 >> 172.21.15.18:0/3890374919 conn(0x7f256c0f83d0 msgr2=0x7f256c0f9ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T20:58:55.839 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:58:55.840+0000 7f2572200700 1 -- 172.21.15.18:0/3890374919 shutdown_connections 2024-04-12T20:58:55.839 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:58:55.841+0000 7f2572200700 1 -- 172.21.15.18:0/3890374919 wait complete. 2024-04-12T20:58:55.998 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:58:56.000+0000 7f8245600700 1 -- 172.21.15.18:0/1701050035 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8240106880 msgr2=0x7f8240106cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:58:55.999 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:58:56.000+0000 7f8245600700 1 --2- 172.21.15.18:0/1701050035 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8240106880 0x7f8240106cf0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f822c004940 tx=0x7f822c009e30 comp rx=0 tx=0).stop 2024-04-12T20:58:55.999 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:58:55 smithi018 ceph-mon[28799]: pgmap v54: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 13 KiB/s rd, 403 KiB/s wr, 128 op/s 2024-04-12T20:58:55.999 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:58:56.001+0000 7f8245600700 1 -- 172.21.15.18:0/1701050035 shutdown_connections 2024-04-12T20:58:55.999 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:58:56.001+0000 7f8245600700 1 --2- 172.21.15.18:0/1701050035 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8240106880 0x7f8240106cf0 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:58:55.999 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:58:56.001+0000 7f8245600700 1 --2- 172.21.15.18:0/1701050035 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f82401055d0 0x7f82401059e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:58:56.000 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:58:56.001+0000 7f8245600700 1 -- 172.21.15.18:0/1701050035 >> 172.21.15.18:0/1701050035 conn(0x7f824006bf60 msgr2=0x7f824006e390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T20:58:56.000 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:58:56.001+0000 7f8245600700 1 -- 172.21.15.18:0/1701050035 shutdown_connections 2024-04-12T20:58:56.000 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:58:56.001+0000 7f8245600700 1 -- 172.21.15.18:0/1701050035 wait complete. 2024-04-12T20:58:56.000 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:58:56.002+0000 7f8245600700 1 Processor -- start 2024-04-12T20:58:56.000 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:58:56.003+0000 7f8245600700 1 -- start start 2024-04-12T20:58:56.001 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:58:56.003+0000 7f8245600700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f82401055d0 0x7f82401160c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T20:58:56.001 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:58:56.003+0000 7f8245600700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8240106880 0x7f8240116600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T20:58:56.001 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:58:56.003+0000 7f8245600700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8240116c00 con 0x7f8240106880 2024-04-12T20:58:56.001 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:58:56.003+0000 7f8245600700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8240116d70 con 0x7f82401055d0 2024-04-12T20:58:56.002 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:58:56.003+0000 7f823e000700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8240106880 0x7f8240116600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T20:58:56.002 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:58:56.003+0000 7f823e000700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8240106880 0x7f8240116600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:56170/0 (socket says 172.21.15.18:56170) 2024-04-12T20:58:56.002 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:58:56.003+0000 7f823e000700 1 -- 172.21.15.18:0/2646592389 learned_addr learned my addr 172.21.15.18:0/2646592389 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T20:58:56.002 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:58:56.004+0000 7f823ea00700 1 --2- 172.21.15.18:0/2646592389 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f82401055d0 0x7f82401160c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T20:58:56.002 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:58:56.004+0000 7f823e000700 1 -- 172.21.15.18:0/2646592389 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f82401055d0 msgr2=0x7f82401160c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:58:56.002 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:58:56.004+0000 7f823e000700 1 --2- 172.21.15.18:0/2646592389 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f82401055d0 0x7f82401160c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:58:56.002 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:58:56.004+0000 7f823e000700 1 -- 172.21.15.18:0/2646592389 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f822c005040 con 0x7f8240106880 2024-04-12T20:58:56.003 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:58:56.004+0000 7f823e000700 1 --2- 172.21.15.18:0/2646592389 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8240106880 0x7f8240116600 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f822c013870 tx=0x7f822c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T20:58:56.003 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:58:56.005+0000 7f8236a00700 1 -- 172.21.15.18:0/2646592389 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f822c00e070 con 0x7f8240106880 2024-04-12T20:58:56.003 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:58:56.005+0000 7f8236a00700 1 -- 172.21.15.18:0/2646592389 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f822c01c400 con 0x7f8240106880 2024-04-12T20:58:56.003 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:58:56.005+0000 7f8245600700 1 -- 172.21.15.18:0/2646592389 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f824011b770 con 0x7f8240106880 2024-04-12T20:58:56.003 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:58:56.005+0000 7f8245600700 1 -- 172.21.15.18:0/2646592389 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f824011bc50 con 0x7f8240106880 2024-04-12T20:58:56.003 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:58:56.005+0000 7f8236a00700 1 -- 172.21.15.18:0/2646592389 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f822c00b630 con 0x7f8240106880 2024-04-12T20:58:56.004 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:58:56.007+0000 7f8245600700 1 -- 172.21.15.18:0/2646592389 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f82200052e0 con 0x7f8240106880 2024-04-12T20:58:56.007 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:58:56.009+0000 7f8236a00700 1 -- 172.21.15.18:0/2646592389 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 32) v1 ==== 92459+0+0 (secure 0 0 0) 0x7f822c013af0 con 0x7f8240106880 2024-04-12T20:58:56.008 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:58:56.010+0000 7f8236a00700 1 --2- 172.21.15.18:0/2646592389 >> [v2:172.21.15.18:6800/1605950634,v1:172.21.15.18:6801/1605950634] conn(0x7f8224072910 0x7f8224074dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T20:58:56.008 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:58:56.010+0000 7f8236a00700 1 -- 172.21.15.18:0/2646592389 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6066+0+0 (secure 0 0 0) 0x7f822c090110 con 0x7f8240106880 2024-04-12T20:58:56.009 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:58:56.011+0000 7f823ea00700 1 --2- 172.21.15.18:0/2646592389 >> [v2:172.21.15.18:6800/1605950634,v1:172.21.15.18:6801/1605950634] conn(0x7f8224072910 0x7f8224074dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T20:58:56.009 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:58:56.012+0000 7f823ea00700 1 --2- 172.21.15.18:0/2646592389 >> [v2:172.21.15.18:6800/1605950634,v1:172.21.15.18:6801/1605950634] conn(0x7f8224072910 0x7f8224074dc0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f8230004600 tx=0x7f823000b040 comp rx=0 tx=0).ready entity=mgr.14672 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T20:58:56.012 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:58:56.014+0000 7f8236a00700 1 -- 172.21.15.18:0/2646592389 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186454 (secure 0 0 0) 0x7f822c05d2f0 con 0x7f8240106880 2024-04-12T20:58:56.030 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:58:55 smithi186 ceph-mon[36597]: pgmap v54: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 13 KiB/s rd, 403 KiB/s wr, 128 op/s 2024-04-12T20:58:56.351 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:58:56.353+0000 7f8245600700 1 -- 172.21.15.18:0/2646592389 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f8220005860 con 0x7f8240106880 2024-04-12T20:58:56.352 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:58:56.354+0000 7f8236a00700 1 -- 172.21.15.18:0/2646592389 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+739 (secure 0 0 0) 0x7f822c05ce80 con 0x7f8240106880 2024-04-12T20:58:56.352 INFO:teuthology.orchestra.run.smithi018.stdout:{ 2024-04-12T20:58:56.352 INFO:teuthology.orchestra.run.smithi018.stdout: "mon": { 2024-04-12T20:58:56.352 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 17.2.7-827-gd3b5060a (d3b5060a17449edfae9385cdf792afec109fca5d) quincy (stable)": 2 2024-04-12T20:58:56.352 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-12T20:58:56.352 INFO:teuthology.orchestra.run.smithi018.stdout: "mgr": { 2024-04-12T20:58:56.352 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 19.0.0-2879-g324baf8d (324baf8ddb91dea46cac50cb535c306115791355) squid (dev)": 2 2024-04-12T20:58:56.353 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-12T20:58:56.353 INFO:teuthology.orchestra.run.smithi018.stdout: "osd": { 2024-04-12T20:58:56.353 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 17.2.7-827-gd3b5060a (d3b5060a17449edfae9385cdf792afec109fca5d) quincy (stable)": 6 2024-04-12T20:58:56.353 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-12T20:58:56.353 INFO:teuthology.orchestra.run.smithi018.stdout: "mds": { 2024-04-12T20:58:56.353 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 17.2.7-827-gd3b5060a (d3b5060a17449edfae9385cdf792afec109fca5d) quincy (stable)": 4 2024-04-12T20:58:56.353 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-12T20:58:56.353 INFO:teuthology.orchestra.run.smithi018.stdout: "overall": { 2024-04-12T20:58:56.353 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 17.2.7-827-gd3b5060a (d3b5060a17449edfae9385cdf792afec109fca5d) quincy (stable)": 12, 2024-04-12T20:58:56.353 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 19.0.0-2879-g324baf8d (324baf8ddb91dea46cac50cb535c306115791355) squid (dev)": 2 2024-04-12T20:58:56.353 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-04-12T20:58:56.353 INFO:teuthology.orchestra.run.smithi018.stdout:} 2024-04-12T20:58:56.355 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:58:56.357+0000 7f8245600700 1 -- 172.21.15.18:0/2646592389 >> [v2:172.21.15.18:6800/1605950634,v1:172.21.15.18:6801/1605950634] conn(0x7f8224072910 msgr2=0x7f8224074dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:58:56.355 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:58:56.357+0000 7f8245600700 1 --2- 172.21.15.18:0/2646592389 >> [v2:172.21.15.18:6800/1605950634,v1:172.21.15.18:6801/1605950634] conn(0x7f8224072910 0x7f8224074dc0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f8230004600 tx=0x7f823000b040 comp rx=0 tx=0).stop 2024-04-12T20:58:56.355 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:58:56.357+0000 7f8245600700 1 -- 172.21.15.18:0/2646592389 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8240106880 msgr2=0x7f8240116600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:58:56.355 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:58:56.357+0000 7f8245600700 1 --2- 172.21.15.18:0/2646592389 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8240106880 0x7f8240116600 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f822c013870 tx=0x7f822c009e30 comp rx=0 tx=0).stop 2024-04-12T20:58:56.355 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:58:56.357+0000 7f8245600700 1 -- 172.21.15.18:0/2646592389 shutdown_connections 2024-04-12T20:58:56.355 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:58:56.357+0000 7f8245600700 1 --2- 172.21.15.18:0/2646592389 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f82401055d0 0x7f82401160c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:58:56.356 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:58:56.357+0000 7f8245600700 1 --2- 172.21.15.18:0/2646592389 >> [v2:172.21.15.18:6800/1605950634,v1:172.21.15.18:6801/1605950634] conn(0x7f8224072910 0x7f8224074dc0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:58:56.356 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:58:56.357+0000 7f8245600700 1 --2- 172.21.15.18:0/2646592389 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8240106880 0x7f8240116600 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:58:56.356 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:58:56.357+0000 7f8245600700 1 -- 172.21.15.18:0/2646592389 >> 172.21.15.18:0/2646592389 conn(0x7f824006bf60 msgr2=0x7f824006d500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T20:58:56.356 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:58:56.357+0000 7f8245600700 1 -- 172.21.15.18:0/2646592389 shutdown_connections 2024-04-12T20:58:56.356 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:58:56.358+0000 7f8245600700 1 -- 172.21.15.18:0/2646592389 wait complete. 2024-04-12T20:58:56.545 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:58:56.547+0000 7f8507a00700 1 -- 172.21.15.18:0/1320291687 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f85000662e0 msgr2=0x7f85000ff9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:58:56.545 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:58:56.547+0000 7f8507a00700 1 --2- 172.21.15.18:0/1320291687 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f85000662e0 0x7f85000ff9c0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f84f4004940 tx=0x7f84f4009e30 comp rx=0 tx=0).stop 2024-04-12T20:58:56.545 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:58:56.547+0000 7f8507a00700 1 -- 172.21.15.18:0/1320291687 shutdown_connections 2024-04-12T20:58:56.545 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:58:56.547+0000 7f8507a00700 1 --2- 172.21.15.18:0/1320291687 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f85000fff00 0x7f8500102320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:58:56.545 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:58:56.547+0000 7f8507a00700 1 --2- 172.21.15.18:0/1320291687 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f85000662e0 0x7f85000ff9c0 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:58:56.545 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:58:56.547+0000 7f8507a00700 1 -- 172.21.15.18:0/1320291687 >> 172.21.15.18:0/1320291687 conn(0x7f85000f7430 msgr2=0x7f85000f9860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T20:58:56.545 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:58:56.547+0000 7f8507a00700 1 -- 172.21.15.18:0/1320291687 shutdown_connections 2024-04-12T20:58:56.546 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:58:56.547+0000 7f8507a00700 1 -- 172.21.15.18:0/1320291687 wait complete. 2024-04-12T20:58:56.546 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:58:56.548+0000 7f8507a00700 1 Processor -- start 2024-04-12T20:58:56.546 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:58:56.548+0000 7f8507a00700 1 -- start start 2024-04-12T20:58:56.547 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:58:56.549+0000 7f8507a00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f85000662e0 0x7f850006e480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T20:58:56.547 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:58:56.549+0000 7f8507a00700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f85000fff00 0x7f850006e9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T20:58:56.547 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:58:56.549+0000 7f8507a00700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8500070070 con 0x7f85000662e0 2024-04-12T20:58:56.547 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:58:56.549+0000 7f8505000700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f85000662e0 0x7f850006e480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T20:58:56.547 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:58:56.549+0000 7f8507a00700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f850006ef00 con 0x7f85000fff00 2024-04-12T20:58:56.548 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:58:56.549+0000 7f8505000700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f85000662e0 0x7f850006e480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:56182/0 (socket says 172.21.15.18:56182) 2024-04-12T20:58:56.548 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:58:56.549+0000 7f8505000700 1 -- 172.21.15.18:0/2604852847 learned_addr learned my addr 172.21.15.18:0/2604852847 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T20:58:56.548 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:58:56.550+0000 7f84ffe00700 1 --2- 172.21.15.18:0/2604852847 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f85000fff00 0x7f850006e9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T20:58:56.548 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:58:56.550+0000 7f8505000700 1 -- 172.21.15.18:0/2604852847 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f85000fff00 msgr2=0x7f850006e9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:58:56.548 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:58:56.550+0000 7f8505000700 1 --2- 172.21.15.18:0/2604852847 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f85000fff00 0x7f850006e9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:58:56.549 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:58:56.550+0000 7f8505000700 1 -- 172.21.15.18:0/2604852847 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84f4005040 con 0x7f85000662e0 2024-04-12T20:58:56.549 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:58:56.550+0000 7f84ffe00700 1 --2- 172.21.15.18:0/2604852847 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f85000fff00 0x7f850006e9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-12T20:58:56.549 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:58:56.551+0000 7f8505000700 1 --2- 172.21.15.18:0/2604852847 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f85000662e0 0x7f850006e480 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f84f400b970 tx=0x7f84f4009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T20:58:56.549 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:58:56.551+0000 7f84fcc00700 1 -- 172.21.15.18:0/2604852847 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f84f400e040 con 0x7f85000662e0 2024-04-12T20:58:56.549 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:58:56.551+0000 7f8507a00700 1 -- 172.21.15.18:0/2604852847 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f850006f180 con 0x7f85000662e0 2024-04-12T20:58:56.550 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:58:56.551+0000 7f84fcc00700 1 -- 172.21.15.18:0/2604852847 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f84f4004290 con 0x7f85000662e0 2024-04-12T20:58:56.550 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:58:56.551+0000 7f84fcc00700 1 -- 172.21.15.18:0/2604852847 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f84f4013640 con 0x7f85000662e0 2024-04-12T20:58:56.550 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:58:56.552+0000 7f8507a00700 1 -- 172.21.15.18:0/2604852847 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f85000fba90 con 0x7f85000662e0 2024-04-12T20:58:56.551 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:58:56.553+0000 7f8507a00700 1 -- 172.21.15.18:0/2604852847 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f850005f5f0 con 0x7f85000662e0 2024-04-12T20:58:56.552 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:58:56.554+0000 7f84fcc00700 1 -- 172.21.15.18:0/2604852847 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 32) v1 ==== 92459+0+0 (secure 0 0 0) 0x7f84f400bc20 con 0x7f85000662e0 2024-04-12T20:58:56.554 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:58:56.556+0000 7f84fcc00700 1 --2- 172.21.15.18:0/2604852847 >> [v2:172.21.15.18:6800/1605950634,v1:172.21.15.18:6801/1605950634] conn(0x7f84e806e510 0x7f84e80709c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T20:58:56.554 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:58:56.557+0000 7f84fcc00700 1 -- 172.21.15.18:0/2604852847 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6066+0+0 (secure 0 0 0) 0x7f84f4092d30 con 0x7f85000662e0 2024-04-12T20:58:56.555 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:58:56.557+0000 7f84ffe00700 1 --2- 172.21.15.18:0/2604852847 >> [v2:172.21.15.18:6800/1605950634,v1:172.21.15.18:6801/1605950634] conn(0x7f84e806e510 0x7f84e80709c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T20:58:56.556 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:58:56.558+0000 7f84ffe00700 1 --2- 172.21.15.18:0/2604852847 >> [v2:172.21.15.18:6800/1605950634,v1:172.21.15.18:6801/1605950634] conn(0x7f84e806e510 0x7f84e80709c0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f84ec009fd0 tx=0x7f84ec00b040 comp rx=0 tx=0).ready entity=mgr.14672 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T20:58:56.560 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:58:56.562+0000 7f84fcc00700 1 -- 172.21.15.18:0/2604852847 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186454 (secure 0 0 0) 0x7f84f405c3e0 con 0x7f85000662e0 2024-04-12T20:58:56.696 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:58:56 smithi018 ceph-mon[28799]: from='client.14766 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T20:58:56.696 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:58:56 smithi018 ceph-mon[28799]: from='client.14770 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T20:58:56.696 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:58:56 smithi018 ceph-mon[28799]: from='client.14774 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T20:58:56.696 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:58:56 smithi018 ceph-mon[28799]: from='client.? 172.21.15.18:0/2646592389' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T20:58:56.696 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:58:56 smithi018 ceph-mon[28799]: from='mgr.14672 172.21.15.18:0/2209228970' entity='mgr.smithi018.xjzwga' 2024-04-12T20:58:56.696 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:58:56 smithi018 ceph-mon[28799]: from='mgr.14672 172.21.15.18:0/2209228970' entity='mgr.smithi018.xjzwga' 2024-04-12T20:58:56.696 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:58:56 smithi018 ceph-mon[28799]: from='mgr.14672 172.21.15.18:0/2209228970' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-12T20:58:56.825 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:58:56.827+0000 7f8507a00700 1 -- 172.21.15.18:0/2604852847 --> [v2:172.21.15.18:6800/1605950634,v1:172.21.15.18:6801/1605950634] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f85000f8f70 con 0x7f84e806e510 2024-04-12T20:58:56.826 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:58:56.828+0000 7f84fcc00700 1 -- 172.21.15.18:0/2604852847 <== mgr.14672 v2:172.21.15.18:6800/1605950634 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+367 (secure 0 0 0) 0x7f85000f8f70 con 0x7f84e806e510 2024-04-12T20:58:56.826 INFO:teuthology.orchestra.run.smithi018.stdout:{ 2024-04-12T20:58:56.826 INFO:teuthology.orchestra.run.smithi018.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:f4452def3be4d350f4a8e5feaf7ebb1511be9298fe68478ba7a2d6fbd7d69a63", 2024-04-12T20:58:56.827 INFO:teuthology.orchestra.run.smithi018.stdout: "in_progress": true, 2024-04-12T20:58:56.827 INFO:teuthology.orchestra.run.smithi018.stdout: "which": "Upgrading daemons of type(s) mgr", 2024-04-12T20:58:56.827 INFO:teuthology.orchestra.run.smithi018.stdout: "services_complete": [ 2024-04-12T20:58:56.827 INFO:teuthology.orchestra.run.smithi018.stdout: "mgr" 2024-04-12T20:58:56.827 INFO:teuthology.orchestra.run.smithi018.stdout: ], 2024-04-12T20:58:56.827 INFO:teuthology.orchestra.run.smithi018.stdout: "progress": "2/2 daemons upgraded", 2024-04-12T20:58:56.827 INFO:teuthology.orchestra.run.smithi018.stdout: "message": "Currently upgrading prometheus daemons", 2024-04-12T20:58:56.827 INFO:teuthology.orchestra.run.smithi018.stdout: "is_paused": false 2024-04-12T20:58:56.827 INFO:teuthology.orchestra.run.smithi018.stdout:} 2024-04-12T20:58:56.830 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:58:56.831+0000 7f8507a00700 1 -- 172.21.15.18:0/2604852847 >> [v2:172.21.15.18:6800/1605950634,v1:172.21.15.18:6801/1605950634] conn(0x7f84e806e510 msgr2=0x7f84e80709c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:58:56.830 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:58:56.831+0000 7f8507a00700 1 --2- 172.21.15.18:0/2604852847 >> [v2:172.21.15.18:6800/1605950634,v1:172.21.15.18:6801/1605950634] conn(0x7f84e806e510 0x7f84e80709c0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f84ec009fd0 tx=0x7f84ec00b040 comp rx=0 tx=0).stop 2024-04-12T20:58:56.830 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:58:56.832+0000 7f8507a00700 1 -- 172.21.15.18:0/2604852847 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f85000662e0 msgr2=0x7f850006e480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:58:56.830 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:58:56.832+0000 7f8507a00700 1 --2- 172.21.15.18:0/2604852847 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f85000662e0 0x7f850006e480 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f84f400b970 tx=0x7f84f4009e30 comp rx=0 tx=0).stop 2024-04-12T20:58:56.830 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:58:56.832+0000 7f8507a00700 1 -- 172.21.15.18:0/2604852847 shutdown_connections 2024-04-12T20:58:56.830 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:58:56.832+0000 7f8507a00700 1 --2- 172.21.15.18:0/2604852847 >> [v2:172.21.15.18:6800/1605950634,v1:172.21.15.18:6801/1605950634] conn(0x7f84e806e510 0x7f84e80709c0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:58:56.830 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:58:56.832+0000 7f8507a00700 1 --2- 172.21.15.18:0/2604852847 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f85000662e0 0x7f850006e480 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:58:56.830 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:58:56.832+0000 7f8507a00700 1 --2- 172.21.15.18:0/2604852847 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f85000fff00 0x7f850006e9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:58:56.830 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:58:56.832+0000 7f8507a00700 1 -- 172.21.15.18:0/2604852847 >> 172.21.15.18:0/2604852847 conn(0x7f85000f7430 msgr2=0x7f85000f7850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T20:58:56.830 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:58:56.832+0000 7f8507a00700 1 -- 172.21.15.18:0/2604852847 shutdown_connections 2024-04-12T20:58:56.831 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:58:56.832+0000 7f8507a00700 1 -- 172.21.15.18:0/2604852847 wait complete. 2024-04-12T20:58:57.030 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:58:56 smithi186 ceph-mon[36597]: from='client.14766 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T20:58:57.031 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:58:56 smithi186 ceph-mon[36597]: from='client.14770 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T20:58:57.031 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:58:56 smithi186 ceph-mon[36597]: from='client.14774 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T20:58:57.031 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:58:56 smithi186 ceph-mon[36597]: from='client.? 172.21.15.18:0/2646592389' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T20:58:57.031 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:58:56 smithi186 ceph-mon[36597]: from='mgr.14672 172.21.15.18:0/2209228970' entity='mgr.smithi018.xjzwga' 2024-04-12T20:58:57.031 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:58:56 smithi186 ceph-mon[36597]: from='mgr.14672 172.21.15.18:0/2209228970' entity='mgr.smithi018.xjzwga' 2024-04-12T20:58:57.031 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:58:56 smithi186 ceph-mon[36597]: from='mgr.14672 172.21.15.18:0/2209228970' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-12T20:58:58.030 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:58:57 smithi186 ceph-mon[36597]: pgmap v55: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 7.8 KiB/s rd, 218 KiB/s wr, 72 op/s 2024-04-12T20:58:58.030 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:58:57 smithi186 ceph-mon[36597]: from='client.14782 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T20:58:58.030 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:58:57 smithi186 ceph-mon[36597]: from='mgr.14672 172.21.15.18:0/2209228970' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T20:58:58.068 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:58:57 smithi018 ceph-mon[28799]: pgmap v55: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 7.8 KiB/s rd, 218 KiB/s wr, 72 op/s 2024-04-12T20:58:58.068 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:58:57 smithi018 ceph-mon[28799]: from='client.14782 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T20:58:58.068 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:58:57 smithi018 ceph-mon[28799]: from='mgr.14672 172.21.15.18:0/2209228970' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T20:59:00.030 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:58:59 smithi186 ceph-mon[36597]: pgmap v56: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 8.0 KiB/s rd, 224 KiB/s wr, 74 op/s 2024-04-12T20:59:00.101 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:58:59 smithi018 ceph-mon[28799]: pgmap v56: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 8.0 KiB/s rd, 224 KiB/s wr, 74 op/s 2024-04-12T20:59:02.030 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:59:01 smithi186 ceph-mon[36597]: pgmap v57: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 9.1 KiB/s rd, 98 KiB/s wr, 72 op/s 2024-04-12T20:59:02.084 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:59:01 smithi018 ceph-mon[28799]: pgmap v57: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 9.1 KiB/s rd, 98 KiB/s wr, 72 op/s 2024-04-12T20:59:03.851 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:59:03 smithi018 ceph-mon[28799]: pgmap v58: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 4.4 KiB/s rd, 92 KiB/s wr, 40 op/s 2024-04-12T20:59:04.030 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:59:03 smithi186 ceph-mon[36597]: pgmap v58: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 4.4 KiB/s rd, 92 KiB/s wr, 40 op/s 2024-04-12T20:59:06.030 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:59:05 smithi186 ceph-mon[36597]: pgmap v59: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 4.4 KiB/s rd, 92 KiB/s wr, 40 op/s 2024-04-12T20:59:06.101 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:59:05 smithi018 ceph-mon[28799]: pgmap v59: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 4.4 KiB/s rd, 92 KiB/s wr, 40 op/s 2024-04-12T20:59:07.101 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:59:06 smithi018 ceph-mon[28799]: from='mgr.14672 172.21.15.18:0/2209228970' entity='mgr.smithi018.xjzwga' 2024-04-12T20:59:07.101 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:59:06 smithi018 ceph-mon[28799]: from='mgr.14672 172.21.15.18:0/2209228970' entity='mgr.smithi018.xjzwga' 2024-04-12T20:59:07.280 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:59:06 smithi186 ceph-mon[36597]: from='mgr.14672 172.21.15.18:0/2209228970' entity='mgr.smithi018.xjzwga' 2024-04-12T20:59:07.280 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:59:06 smithi186 ceph-mon[36597]: from='mgr.14672 172.21.15.18:0/2209228970' entity='mgr.smithi018.xjzwga' 2024-04-12T20:59:08.101 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:59:07 smithi018 ceph-mon[28799]: pgmap v60: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.2 KiB/s rd, 15 KiB/s wr, 10 op/s 2024-04-12T20:59:08.280 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:59:07 smithi186 ceph-mon[36597]: pgmap v60: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.2 KiB/s rd, 15 KiB/s wr, 10 op/s 2024-04-12T20:59:09.280 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:59:09 smithi186 ceph-mon[36597]: from='mgr.14672 172.21.15.18:0/2209228970' entity='mgr.smithi018.xjzwga' 2024-04-12T20:59:09.280 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:59:09 smithi186 ceph-mon[36597]: from='mgr.14672 172.21.15.18:0/2209228970' entity='mgr.smithi018.xjzwga' 2024-04-12T20:59:09.280 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:59:09 smithi186 ceph-mon[36597]: pgmap v61: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.2 KiB/s rd, 15 KiB/s wr, 10 op/s 2024-04-12T20:59:09.351 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:59:09 smithi018 ceph-mon[28799]: from='mgr.14672 172.21.15.18:0/2209228970' entity='mgr.smithi018.xjzwga' 2024-04-12T20:59:09.352 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:59:09 smithi018 ceph-mon[28799]: from='mgr.14672 172.21.15.18:0/2209228970' entity='mgr.smithi018.xjzwga' 2024-04-12T20:59:09.352 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:59:09 smithi018 ceph-mon[28799]: pgmap v61: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.2 KiB/s rd, 15 KiB/s wr, 10 op/s 2024-04-12T20:59:12.030 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:59:11 smithi186 ceph-mon[36597]: pgmap v62: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.1 KiB/s rd, 9.0 KiB/s wr, 8 op/s 2024-04-12T20:59:12.101 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:59:11 smithi018 ceph-mon[28799]: pgmap v62: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.1 KiB/s rd, 9.0 KiB/s wr, 8 op/s 2024-04-12T20:59:13.851 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:59:13 smithi018 ceph-mon[28799]: pgmap v63: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-12T20:59:13.852 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:59:13 smithi018 ceph-mon[28799]: from='mgr.14672 172.21.15.18:0/2209228970' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T20:59:14.030 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:59:13 smithi186 ceph-mon[36597]: pgmap v63: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-12T20:59:14.030 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:59:13 smithi186 ceph-mon[36597]: from='mgr.14672 172.21.15.18:0/2209228970' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T20:59:15.029 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:59:14 smithi018 ceph-mon[28799]: from='mgr.14672 172.21.15.18:0/2209228970' entity='mgr.smithi018.xjzwga' 2024-04-12T20:59:15.029 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:59:14 smithi018 ceph-mon[28799]: from='mgr.14672 172.21.15.18:0/2209228970' entity='mgr.smithi018.xjzwga' 2024-04-12T20:59:15.029 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:59:14 smithi018 ceph-mon[28799]: from='mgr.14672 172.21.15.18:0/2209228970' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-12T20:59:15.029 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:59:14 smithi018 ceph-mon[28799]: from='mgr.14672 172.21.15.18:0/2209228970' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-12T20:59:15.029 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:59:14 smithi018 ceph-mon[28799]: from='mgr.14672 172.21.15.18:0/2209228970' entity='mgr.smithi018.xjzwga' 2024-04-12T20:59:15.029 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:59:14 smithi018 ceph-mon[28799]: from='mgr.14672 172.21.15.18:0/2209228970' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-04-12T20:59:15.029 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:59:14 smithi018 ceph-mon[28799]: from='mgr.14672 172.21.15.18:0/2209228970' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-12T20:59:15.029 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:59:14 smithi018 ceph-mon[28799]: from='mgr.14672 172.21.15.18:0/2209228970' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T20:59:15.029 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:59:14 smithi018 ceph-mon[28799]: from='mgr.14672 172.21.15.18:0/2209228970' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T20:59:15.029 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:59:14 smithi018 ceph-mon[28799]: from='mgr.14672 172.21.15.18:0/2209228970' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T20:59:15.030 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:59:14 smithi018 ceph-mon[28799]: from='mgr.14672 172.21.15.18:0/2209228970' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T20:59:15.030 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:59:14 smithi018 ceph-mon[28799]: from='mgr.14672 172.21.15.18:0/2209228970' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T20:59:15.030 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:59:14 smithi018 ceph-mon[28799]: from='mgr.14672 172.21.15.18:0/2209228970' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T20:59:15.030 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:59:14 smithi018 ceph-mon[28799]: from='mgr.14672 172.21.15.18:0/2209228970' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T20:59:15.030 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:59:14 smithi018 ceph-mon[28799]: from='mgr.14672 172.21.15.18:0/2209228970' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T20:59:15.030 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:59:14 smithi018 ceph-mon[28799]: from='mgr.14672 172.21.15.18:0/2209228970' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T20:59:15.030 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:59:14 smithi018 ceph-mon[28799]: from='mgr.14672 172.21.15.18:0/2209228970' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T20:59:15.030 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:59:14 smithi186 ceph-mon[36597]: from='mgr.14672 172.21.15.18:0/2209228970' entity='mgr.smithi018.xjzwga' 2024-04-12T20:59:15.031 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:59:14 smithi186 ceph-mon[36597]: from='mgr.14672 172.21.15.18:0/2209228970' entity='mgr.smithi018.xjzwga' 2024-04-12T20:59:15.031 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:59:14 smithi186 ceph-mon[36597]: from='mgr.14672 172.21.15.18:0/2209228970' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-12T20:59:15.031 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:59:14 smithi186 ceph-mon[36597]: from='mgr.14672 172.21.15.18:0/2209228970' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-12T20:59:15.031 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:59:14 smithi186 ceph-mon[36597]: from='mgr.14672 172.21.15.18:0/2209228970' entity='mgr.smithi018.xjzwga' 2024-04-12T20:59:15.031 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:59:14 smithi186 ceph-mon[36597]: from='mgr.14672 172.21.15.18:0/2209228970' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-04-12T20:59:15.031 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:59:14 smithi186 ceph-mon[36597]: from='mgr.14672 172.21.15.18:0/2209228970' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-12T20:59:15.031 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:59:14 smithi186 ceph-mon[36597]: from='mgr.14672 172.21.15.18:0/2209228970' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T20:59:15.031 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:59:14 smithi186 ceph-mon[36597]: from='mgr.14672 172.21.15.18:0/2209228970' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T20:59:15.031 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:59:14 smithi186 ceph-mon[36597]: from='mgr.14672 172.21.15.18:0/2209228970' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T20:59:15.031 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:59:14 smithi186 ceph-mon[36597]: from='mgr.14672 172.21.15.18:0/2209228970' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T20:59:15.031 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:59:14 smithi186 ceph-mon[36597]: from='mgr.14672 172.21.15.18:0/2209228970' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T20:59:15.031 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:59:14 smithi186 ceph-mon[36597]: from='mgr.14672 172.21.15.18:0/2209228970' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T20:59:15.031 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:59:14 smithi186 ceph-mon[36597]: from='mgr.14672 172.21.15.18:0/2209228970' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T20:59:15.032 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:59:14 smithi186 ceph-mon[36597]: from='mgr.14672 172.21.15.18:0/2209228970' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T20:59:15.032 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:59:14 smithi186 ceph-mon[36597]: from='mgr.14672 172.21.15.18:0/2209228970' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T20:59:15.032 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:59:14 smithi186 ceph-mon[36597]: from='mgr.14672 172.21.15.18:0/2209228970' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T20:59:16.030 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:59:15 smithi186 ceph-mon[36597]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-04-12T20:59:16.030 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:59:15 smithi186 ceph-mon[36597]: pgmap v64: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-12T20:59:16.101 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:59:15 smithi018 ceph-mon[28799]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-04-12T20:59:16.101 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:59:15 smithi018 ceph-mon[28799]: pgmap v64: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-12T20:59:17.886 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:59:17 smithi018 ceph-mon[28799]: Upgrade: Updating alertmanager.smithi018 2024-04-12T20:59:17.886 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:59:17 smithi018 ceph-mon[28799]: Deploying daemon alertmanager.smithi018 on smithi018 2024-04-12T20:59:17.887 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:59:17 smithi018 ceph-mon[28799]: pgmap v65: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-12T20:59:18.030 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:59:17 smithi186 ceph-mon[36597]: Upgrade: Updating alertmanager.smithi018 2024-04-12T20:59:18.030 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:59:17 smithi186 ceph-mon[36597]: Deploying daemon alertmanager.smithi018 on smithi018 2024-04-12T20:59:18.030 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:59:17 smithi186 ceph-mon[36597]: pgmap v65: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-12T20:59:19.851 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:59:19 smithi018 ceph-mon[28799]: pgmap v66: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-12T20:59:20.030 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:59:19 smithi186 ceph-mon[36597]: pgmap v66: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-12T20:59:22.030 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:59:21 smithi186 ceph-mon[36597]: pgmap v67: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-12T20:59:22.062 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:59:21 smithi018 ceph-mon[28799]: pgmap v67: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-12T20:59:23.851 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:59:23 smithi018 ceph-mon[28799]: pgmap v68: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-12T20:59:24.030 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:59:23 smithi186 ceph-mon[36597]: pgmap v68: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-12T20:59:26.030 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:59:25 smithi186 ceph-mon[36597]: pgmap v69: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-12T20:59:26.030 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:59:25 smithi186 ceph-mon[36597]: from='mgr.14672 172.21.15.18:0/2209228970' entity='mgr.smithi018.xjzwga' 2024-04-12T20:59:26.030 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:59:25 smithi186 ceph-mon[36597]: from='mgr.14672 172.21.15.18:0/2209228970' entity='mgr.smithi018.xjzwga' 2024-04-12T20:59:26.030 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:59:25 smithi186 ceph-mon[36597]: from='mgr.14672 172.21.15.18:0/2209228970' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-12T20:59:26.101 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:59:25 smithi018 ceph-mon[28799]: pgmap v69: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-12T20:59:26.102 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:59:25 smithi018 ceph-mon[28799]: from='mgr.14672 172.21.15.18:0/2209228970' entity='mgr.smithi018.xjzwga' 2024-04-12T20:59:26.102 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:59:25 smithi018 ceph-mon[28799]: from='mgr.14672 172.21.15.18:0/2209228970' entity='mgr.smithi018.xjzwga' 2024-04-12T20:59:26.102 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:59:25 smithi018 ceph-mon[28799]: from='mgr.14672 172.21.15.18:0/2209228970' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-12T20:59:26.994 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:59:26.995+0000 7fd2cd200700 1 -- 172.21.15.18:0/3169076730 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd2c8071ea0 msgr2=0x7fd2c80703b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:59:26.994 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:59:26.995+0000 7fd2cd200700 1 --2- 172.21.15.18:0/3169076730 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd2c8071ea0 0x7fd2c80703b0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7fd2b8004940 tx=0x7fd2b8009e30 comp rx=0 tx=0).stop 2024-04-12T20:59:26.994 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:59:26.995+0000 7fd2cd200700 1 -- 172.21.15.18:0/3169076730 shutdown_connections 2024-04-12T20:59:26.995 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:59:26.995+0000 7fd2cd200700 1 --2- 172.21.15.18:0/3169076730 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd2c80708f0 0x7fd2c8070d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:59:26.995 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:59:26.995+0000 7fd2cd200700 1 --2- 172.21.15.18:0/3169076730 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd2c8071ea0 0x7fd2c80703b0 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:59:26.995 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:59:26.995+0000 7fd2cd200700 1 -- 172.21.15.18:0/3169076730 >> 172.21.15.18:0/3169076730 conn(0x7fd2c80f8550 msgr2=0x7fd2c80fa960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T20:59:26.995 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:59:26.995+0000 7fd2cd200700 1 -- 172.21.15.18:0/3169076730 shutdown_connections 2024-04-12T20:59:26.995 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:59:26.996+0000 7fd2cd200700 1 -- 172.21.15.18:0/3169076730 wait complete. 2024-04-12T20:59:26.995 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:59:26.996+0000 7fd2cd200700 1 Processor -- start 2024-04-12T20:59:26.995 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:59:26.997+0000 7fd2cd200700 1 -- start start 2024-04-12T20:59:26.997 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:59:26.997+0000 7fd2cd200700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd2c80708f0 0x7fd2c8114120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T20:59:26.997 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:59:26.997+0000 7fd2cd200700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd2c8114660 0x7fd2c8119690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T20:59:26.997 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:59:26.997+0000 7fd2cd200700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2c8114b20 con 0x7fd2c80708f0 2024-04-12T20:59:26.997 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:59:26.997+0000 7fd2cd200700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2c8114c60 con 0x7fd2c8114660 2024-04-12T20:59:26.997 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:59:26.997+0000 7fd2c5e00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd2c80708f0 0x7fd2c8114120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T20:59:26.997 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:59:26.998+0000 7fd2c5e00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd2c80708f0 0x7fd2c8114120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:50834/0 (socket says 172.21.15.18:50834) 2024-04-12T20:59:26.997 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:59:26.998+0000 7fd2c5e00700 1 -- 172.21.15.18:0/785155153 learned_addr learned my addr 172.21.15.18:0/785155153 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T20:59:26.997 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:59:26.998+0000 7fd2bfe00700 1 --2- 172.21.15.18:0/785155153 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd2c8114660 0x7fd2c8119690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T20:59:26.997 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:59:26.998+0000 7fd2c5e00700 1 -- 172.21.15.18:0/785155153 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd2c8114660 msgr2=0x7fd2c8119690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:59:26.997 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:59:26.998+0000 7fd2c5e00700 1 --2- 172.21.15.18:0/785155153 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd2c8114660 0x7fd2c8119690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:59:26.997 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:59:26.998+0000 7fd2c5e00700 1 -- 172.21.15.18:0/785155153 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2b8005040 con 0x7fd2c80708f0 2024-04-12T20:59:26.997 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:59:26.998+0000 7fd2c5e00700 1 --2- 172.21.15.18:0/785155153 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd2c80708f0 0x7fd2c8114120 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fd2b800b3b0 tx=0x7fd2b8009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T20:59:26.998 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:59:26.999+0000 7fd2be000700 1 -- 172.21.15.18:0/785155153 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd2b800e040 con 0x7fd2c80708f0 2024-04-12T20:59:26.998 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:59:26.999+0000 7fd2be000700 1 -- 172.21.15.18:0/785155153 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd2b800bbf0 con 0x7fd2c80708f0 2024-04-12T20:59:26.998 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:59:26.999+0000 7fd2cd200700 1 -- 172.21.15.18:0/785155153 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd2c8119bd0 con 0x7fd2c80708f0 2024-04-12T20:59:26.998 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:59:26.999+0000 7fd2be000700 1 -- 172.21.15.18:0/785155153 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd2b801c630 con 0x7fd2c80708f0 2024-04-12T20:59:26.998 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:59:26.999+0000 7fd2cd200700 1 -- 172.21.15.18:0/785155153 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd2c811a0c0 con 0x7fd2c80708f0 2024-04-12T20:59:26.998 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:59:27.000+0000 7fd2cd200700 1 -- 172.21.15.18:0/785155153 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd2c804c710 con 0x7fd2c80708f0 2024-04-12T20:59:26.999 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:59:27.001+0000 7fd2be000700 1 -- 172.21.15.18:0/785155153 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 32) v1 ==== 92459+0+0 (secure 0 0 0) 0x7fd2b8013420 con 0x7fd2c80708f0 2024-04-12T20:59:27.000 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:59:27.002+0000 7fd2be000700 1 --2- 172.21.15.18:0/785155153 >> [v2:172.21.15.18:6800/1605950634,v1:172.21.15.18:6801/1605950634] conn(0x7fd2ac06e500 0x7fd2ac0709b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T20:59:27.000 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:59:27.002+0000 7fd2be000700 1 -- 172.21.15.18:0/785155153 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6066+0+0 (secure 0 0 0) 0x7fd2b808f1d0 con 0x7fd2c80708f0 2024-04-12T20:59:27.000 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:59:27.003+0000 7fd2bfe00700 1 --2- 172.21.15.18:0/785155153 >> [v2:172.21.15.18:6800/1605950634,v1:172.21.15.18:6801/1605950634] conn(0x7fd2ac06e500 0x7fd2ac0709b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T20:59:27.001 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:59:27.003+0000 7fd2bfe00700 1 --2- 172.21.15.18:0/785155153 >> [v2:172.21.15.18:6800/1605950634,v1:172.21.15.18:6801/1605950634] conn(0x7fd2ac06e500 0x7fd2ac0709b0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fd2b0004990 tx=0x7fd2b0009ee0 comp rx=0 tx=0).ready entity=mgr.14672 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T20:59:27.005 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:59:27.008+0000 7fd2be000700 1 -- 172.21.15.18:0/785155153 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186454 (secure 0 0 0) 0x7fd2b805c3b0 con 0x7fd2c80708f0 2024-04-12T20:59:27.262 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:59:27.264+0000 7fd2cd200700 1 -- 172.21.15.18:0/785155153 --> [v2:172.21.15.18:6800/1605950634,v1:172.21.15.18:6801/1605950634] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fd2c80ffbb0 con 0x7fd2ac06e500 2024-04-12T20:59:27.263 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:59:27.266+0000 7fd2be000700 1 -- 172.21.15.18:0/785155153 <== mgr.14672 v2:172.21.15.18:6800/1605950634 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+369 (secure 0 0 0) 0x7fd2c80ffbb0 con 0x7fd2ac06e500 2024-04-12T20:59:27.266 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:59:27.269+0000 7fd2cd200700 1 -- 172.21.15.18:0/785155153 >> [v2:172.21.15.18:6800/1605950634,v1:172.21.15.18:6801/1605950634] conn(0x7fd2ac06e500 msgr2=0x7fd2ac0709b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:59:27.266 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:59:27.269+0000 7fd2cd200700 1 --2- 172.21.15.18:0/785155153 >> [v2:172.21.15.18:6800/1605950634,v1:172.21.15.18:6801/1605950634] conn(0x7fd2ac06e500 0x7fd2ac0709b0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fd2b0004990 tx=0x7fd2b0009ee0 comp rx=0 tx=0).stop 2024-04-12T20:59:27.267 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:59:27.269+0000 7fd2cd200700 1 -- 172.21.15.18:0/785155153 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd2c80708f0 msgr2=0x7fd2c8114120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:59:27.267 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:59:27.269+0000 7fd2cd200700 1 --2- 172.21.15.18:0/785155153 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd2c80708f0 0x7fd2c8114120 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fd2b800b3b0 tx=0x7fd2b8009e30 comp rx=0 tx=0).stop 2024-04-12T20:59:27.267 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:59:27.269+0000 7fd2cd200700 1 -- 172.21.15.18:0/785155153 shutdown_connections 2024-04-12T20:59:27.267 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:59:27.269+0000 7fd2cd200700 1 --2- 172.21.15.18:0/785155153 >> [v2:172.21.15.18:6800/1605950634,v1:172.21.15.18:6801/1605950634] conn(0x7fd2ac06e500 0x7fd2ac0709b0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:59:27.267 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:59:27.269+0000 7fd2cd200700 1 --2- 172.21.15.18:0/785155153 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd2c80708f0 0x7fd2c8114120 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:59:27.267 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:59:27.269+0000 7fd2cd200700 1 --2- 172.21.15.18:0/785155153 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd2c8114660 0x7fd2c8119690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:59:27.267 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:59:27.269+0000 7fd2cd200700 1 -- 172.21.15.18:0/785155153 >> 172.21.15.18:0/785155153 conn(0x7fd2c80f8550 msgr2=0x7fd2c80f9db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T20:59:27.268 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:59:27.269+0000 7fd2cd200700 1 -- 172.21.15.18:0/785155153 shutdown_connections 2024-04-12T20:59:27.268 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:59:27.269+0000 7fd2cd200700 1 -- 172.21.15.18:0/785155153 wait complete. 2024-04-12T20:59:27.282 INFO:teuthology.orchestra.run.smithi018.stdout:true 2024-04-12T20:59:27.422 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:59:27.424+0000 7fc0c1000700 1 -- 172.21.15.18:0/3837446730 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc0bc0fcc10 msgr2=0x7fc0bc0fd020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:59:27.422 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:59:27.424+0000 7fc0c1000700 1 --2- 172.21.15.18:0/3837446730 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc0bc0fcc10 0x7fc0bc0fd020 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fc0ac009ce0 tx=0x7fc0ac004d10 comp rx=0 tx=0).stop 2024-04-12T20:59:27.422 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:59:27.424+0000 7fc0c1000700 1 -- 172.21.15.18:0/3837446730 shutdown_connections 2024-04-12T20:59:27.422 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:59:27.424+0000 7fc0c1000700 1 --2- 172.21.15.18:0/3837446730 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc0bc0fdec0 0x7fc0bc0fe330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:59:27.422 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:59:27.424+0000 7fc0c1000700 1 --2- 172.21.15.18:0/3837446730 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc0bc0fcc10 0x7fc0bc0fd020 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:59:27.422 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:59:27.424+0000 7fc0c1000700 1 -- 172.21.15.18:0/3837446730 >> 172.21.15.18:0/3837446730 conn(0x7fc0bc0f83d0 msgr2=0x7fc0bc0fa7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T20:59:27.422 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:59:27.424+0000 7fc0c1000700 1 -- 172.21.15.18:0/3837446730 shutdown_connections 2024-04-12T20:59:27.422 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:59:27.424+0000 7fc0c1000700 1 -- 172.21.15.18:0/3837446730 wait complete. 2024-04-12T20:59:27.424 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:59:27.425+0000 7fc0c1000700 1 Processor -- start 2024-04-12T20:59:27.424 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:59:27.425+0000 7fc0c1000700 1 -- start start 2024-04-12T20:59:27.424 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:59:27.425+0000 7fc0c1000700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc0bc0fcc10 0x7fc0bc10fb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T20:59:27.424 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:59:27.425+0000 7fc0c1000700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc0bc0fdec0 0x7fc0bc1100d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T20:59:27.424 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:59:27.425+0000 7fc0c1000700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0bc110760 con 0x7fc0bc0fcc10 2024-04-12T20:59:27.424 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:59:27.425+0000 7fc0c1000700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0bc110f50 con 0x7fc0bc0fdec0 2024-04-12T20:59:27.424 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:59:27.425+0000 7fc0bb400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc0bc0fcc10 0x7fc0bc10fb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T20:59:27.424 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:59:27.425+0000 7fc0bb400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc0bc0fcc10 0x7fc0bc10fb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:50842/0 (socket says 172.21.15.18:50842) 2024-04-12T20:59:27.424 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:59:27.426+0000 7fc0bb400700 1 -- 172.21.15.18:0/3877293309 learned_addr learned my addr 172.21.15.18:0/3877293309 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T20:59:27.424 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:59:27.426+0000 7fc0bb400700 1 -- 172.21.15.18:0/3877293309 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc0bc0fdec0 msgr2=0x7fc0bc1100d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:59:27.424 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:59:27.426+0000 7fc0bb400700 1 --2- 172.21.15.18:0/3877293309 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc0bc0fdec0 0x7fc0bc1100d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:59:27.424 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:59:27.426+0000 7fc0bb400700 1 -- 172.21.15.18:0/3877293309 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0ac009940 con 0x7fc0bc0fcc10 2024-04-12T20:59:27.424 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:59:27.426+0000 7fc0bb400700 1 --2- 172.21.15.18:0/3877293309 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc0bc0fcc10 0x7fc0bc10fb90 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fc0ac004740 tx=0x7fc0ac004d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T20:59:27.425 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:59:27.427+0000 7fc0b3e00700 1 -- 172.21.15.18:0/3877293309 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc0ac00e040 con 0x7fc0bc0fcc10 2024-04-12T20:59:27.425 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:59:27.427+0000 7fc0c1000700 1 -- 172.21.15.18:0/3877293309 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc0bc1111d0 con 0x7fc0bc0fcc10 2024-04-12T20:59:27.425 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:59:27.427+0000 7fc0b3e00700 1 -- 172.21.15.18:0/3877293309 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc0ac0128e0 con 0x7fc0bc0fcc10 2024-04-12T20:59:27.426 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:59:27.427+0000 7fc0b3e00700 1 -- 172.21.15.18:0/3877293309 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc0ac01ca80 con 0x7fc0bc0fcc10 2024-04-12T20:59:27.426 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:59:27.427+0000 7fc0c1000700 1 -- 172.21.15.18:0/3877293309 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc0bc111790 con 0x7fc0bc0fcc10 2024-04-12T20:59:27.426 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:59:27.428+0000 7fc0c1000700 1 -- 172.21.15.18:0/3877293309 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc0bc101850 con 0x7fc0bc0fcc10 2024-04-12T20:59:27.426 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:59:27.429+0000 7fc0b3e00700 1 -- 172.21.15.18:0/3877293309 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 32) v1 ==== 92459+0+0 (secure 0 0 0) 0x7fc0ac012a50 con 0x7fc0bc0fcc10 2024-04-12T20:59:27.427 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:59:27.430+0000 7fc0b3e00700 1 --2- 172.21.15.18:0/3877293309 >> [v2:172.21.15.18:6800/1605950634,v1:172.21.15.18:6801/1605950634] conn(0x7fc09c06e500 0x7fc09c0709b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T20:59:27.428 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:59:27.430+0000 7fc0b3e00700 1 -- 172.21.15.18:0/3877293309 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6066+0+0 (secure 0 0 0) 0x7fc0ac08fb70 con 0x7fc0bc0fcc10 2024-04-12T20:59:27.428 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:59:27.430+0000 7fc0abe00700 1 --2- 172.21.15.18:0/3877293309 >> [v2:172.21.15.18:6800/1605950634,v1:172.21.15.18:6801/1605950634] conn(0x7fc09c06e500 0x7fc09c0709b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T20:59:27.428 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:59:27.431+0000 7fc0abe00700 1 --2- 172.21.15.18:0/3877293309 >> [v2:172.21.15.18:6800/1605950634,v1:172.21.15.18:6801/1605950634] conn(0x7fc09c06e500 0x7fc09c0709b0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fc0a4006fd0 tx=0x7fc0a4006db0 comp rx=0 tx=0).ready entity=mgr.14672 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T20:59:27.433 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:59:27.435+0000 7fc0b3e00700 1 -- 172.21.15.18:0/3877293309 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186454 (secure 0 0 0) 0x7fc0ac05ccd0 con 0x7fc0bc0fcc10 2024-04-12T20:59:27.699 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:59:27.701+0000 7fc0c1000700 1 -- 172.21.15.18:0/3877293309 --> [v2:172.21.15.18:6800/1605950634,v1:172.21.15.18:6801/1605950634] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fc0bc111a00 con 0x7fc09c06e500 2024-04-12T20:59:27.701 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:59:27.704+0000 7fc0b3e00700 1 -- 172.21.15.18:0/3877293309 <== mgr.14672 v2:172.21.15.18:6800/1605950634 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+369 (secure 0 0 0) 0x7fc0bc111a00 con 0x7fc09c06e500 2024-04-12T20:59:27.704 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:59:27.706+0000 7fc0c1000700 1 -- 172.21.15.18:0/3877293309 >> [v2:172.21.15.18:6800/1605950634,v1:172.21.15.18:6801/1605950634] conn(0x7fc09c06e500 msgr2=0x7fc09c0709b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:59:27.704 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:59:27.706+0000 7fc0c1000700 1 --2- 172.21.15.18:0/3877293309 >> [v2:172.21.15.18:6800/1605950634,v1:172.21.15.18:6801/1605950634] conn(0x7fc09c06e500 0x7fc09c0709b0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fc0a4006fd0 tx=0x7fc0a4006db0 comp rx=0 tx=0).stop 2024-04-12T20:59:27.705 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:59:27.706+0000 7fc0c1000700 1 -- 172.21.15.18:0/3877293309 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc0bc0fcc10 msgr2=0x7fc0bc10fb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:59:27.705 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:59:27.706+0000 7fc0c1000700 1 --2- 172.21.15.18:0/3877293309 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc0bc0fcc10 0x7fc0bc10fb90 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fc0ac004740 tx=0x7fc0ac004d10 comp rx=0 tx=0).stop 2024-04-12T20:59:27.705 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:59:27.707+0000 7fc0c1000700 1 -- 172.21.15.18:0/3877293309 shutdown_connections 2024-04-12T20:59:27.705 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:59:27.707+0000 7fc0c1000700 1 --2- 172.21.15.18:0/3877293309 >> [v2:172.21.15.18:6800/1605950634,v1:172.21.15.18:6801/1605950634] conn(0x7fc09c06e500 0x7fc09c0709b0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:59:27.705 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:59:27.707+0000 7fc0c1000700 1 --2- 172.21.15.18:0/3877293309 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc0bc0fcc10 0x7fc0bc10fb90 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:59:27.705 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:59:27.707+0000 7fc0c1000700 1 --2- 172.21.15.18:0/3877293309 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc0bc0fdec0 0x7fc0bc1100d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:59:27.705 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:59:27.707+0000 7fc0c1000700 1 -- 172.21.15.18:0/3877293309 >> 172.21.15.18:0/3877293309 conn(0x7fc0bc0f83d0 msgr2=0x7fc0bc0f9a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T20:59:27.705 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:59:27.708+0000 7fc0c1000700 1 -- 172.21.15.18:0/3877293309 shutdown_connections 2024-04-12T20:59:27.705 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:59:27.708+0000 7fc0c1000700 1 -- 172.21.15.18:0/3877293309 wait complete. 2024-04-12T20:59:27.856 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:59:27.858+0000 7f8ea9a00700 1 -- 172.21.15.18:0/3633555126 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8ea40708f0 msgr2=0x7f8ea4070d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:59:27.856 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:59:27.858+0000 7f8ea9a00700 1 --2- 172.21.15.18:0/3633555126 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8ea40708f0 0x7f8ea4070d60 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f8e90004940 tx=0x7f8e90009e30 comp rx=0 tx=0).stop 2024-04-12T20:59:27.856 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:59:27.858+0000 7f8ea9a00700 1 -- 172.21.15.18:0/3633555126 shutdown_connections 2024-04-12T20:59:27.856 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:59:27.858+0000 7f8ea9a00700 1 --2- 172.21.15.18:0/3633555126 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8ea40708f0 0x7f8ea4070d60 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:59:27.856 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:59:27.858+0000 7f8ea9a00700 1 --2- 172.21.15.18:0/3633555126 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8ea4071ea0 0x7f8ea40703b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:59:27.857 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:59:27.858+0000 7f8ea9a00700 1 -- 172.21.15.18:0/3633555126 >> 172.21.15.18:0/3633555126 conn(0x7f8ea40f8560 msgr2=0x7f8ea40fa970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T20:59:27.857 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:59:27.858+0000 7f8ea9a00700 1 -- 172.21.15.18:0/3633555126 shutdown_connections 2024-04-12T20:59:27.857 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:59:27.858+0000 7f8ea9a00700 1 -- 172.21.15.18:0/3633555126 wait complete. 2024-04-12T20:59:27.858 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:59:27.859+0000 7f8ea9a00700 1 Processor -- start 2024-04-12T20:59:27.858 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:59:27.859+0000 7f8ea9a00700 1 -- start start 2024-04-12T20:59:27.858 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:59:27.860+0000 7f8ea9a00700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8ea40708f0 0x7f8ea410fc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T20:59:27.858 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:59:27.860+0000 7f8ea9a00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8ea4071ea0 0x7f8ea4110180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T20:59:27.858 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:59:27.860+0000 7f8ea9a00700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ea4110780 con 0x7f8ea4071ea0 2024-04-12T20:59:27.858 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:59:27.860+0000 7f8ea9a00700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ea4110fd0 con 0x7f8ea40708f0 2024-04-12T20:59:27.858 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:59:27.860+0000 7f8ea2000700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8ea4071ea0 0x7f8ea4110180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T20:59:27.858 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:59:27.860+0000 7f8ea2000700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8ea4071ea0 0x7f8ea4110180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:50858/0 (socket says 172.21.15.18:50858) 2024-04-12T20:59:27.858 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:59:27.860+0000 7f8ea2000700 1 -- 172.21.15.18:0/2969939856 learned_addr learned my addr 172.21.15.18:0/2969939856 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T20:59:27.859 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:59:27.860+0000 7f8ea2a00700 1 --2- 172.21.15.18:0/2969939856 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8ea40708f0 0x7f8ea410fc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T20:59:27.859 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:59:27.861+0000 7f8ea2000700 1 -- 172.21.15.18:0/2969939856 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8ea40708f0 msgr2=0x7f8ea410fc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:59:27.859 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:59:27.861+0000 7f8ea2000700 1 --2- 172.21.15.18:0/2969939856 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8ea40708f0 0x7f8ea410fc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:59:27.859 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:59:27.861+0000 7f8ea2000700 1 -- 172.21.15.18:0/2969939856 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e90005040 con 0x7f8ea4071ea0 2024-04-12T20:59:27.859 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:59:27.861+0000 7f8ea2000700 1 --2- 172.21.15.18:0/2969939856 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8ea4071ea0 0x7f8ea4110180 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f8e90009150 tx=0x7f8e90009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T20:59:27.860 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:59:27.861+0000 7f8e9aa00700 1 -- 172.21.15.18:0/2969939856 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8e9000e040 con 0x7f8ea4071ea0 2024-04-12T20:59:27.860 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:59:27.861+0000 7f8e9aa00700 1 -- 172.21.15.18:0/2969939856 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f8e9000bcc0 con 0x7f8ea4071ea0 2024-04-12T20:59:27.860 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:59:27.861+0000 7f8ea9a00700 1 -- 172.21.15.18:0/2969939856 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8ea4111250 con 0x7f8ea4071ea0 2024-04-12T20:59:27.860 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:59:27.861+0000 7f8e9aa00700 1 -- 172.21.15.18:0/2969939856 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8e9001c630 con 0x7f8ea4071ea0 2024-04-12T20:59:27.860 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:59:27.861+0000 7f8ea9a00700 1 -- 172.21.15.18:0/2969939856 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8ea4111790 con 0x7f8ea4071ea0 2024-04-12T20:59:27.860 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:59:27.863+0000 7f8ea9a00700 1 -- 172.21.15.18:0/2969939856 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8ea405f5f0 con 0x7f8ea4071ea0 2024-04-12T20:59:27.861 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:59:27.863+0000 7f8e9aa00700 1 -- 172.21.15.18:0/2969939856 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 32) v1 ==== 92459+0+0 (secure 0 0 0) 0x7f8e90013420 con 0x7f8ea4071ea0 2024-04-12T20:59:27.862 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:59:27.865+0000 7f8e9aa00700 1 --2- 172.21.15.18:0/2969939856 >> [v2:172.21.15.18:6800/1605950634,v1:172.21.15.18:6801/1605950634] conn(0x7f8e8806e500 0x7f8e880709b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T20:59:27.863 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:59:27.865+0000 7f8e9aa00700 1 -- 172.21.15.18:0/2969939856 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6066+0+0 (secure 0 0 0) 0x7f8e9008eaa0 con 0x7f8ea4071ea0 2024-04-12T20:59:27.863 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:59:27.865+0000 7f8ea2a00700 1 --2- 172.21.15.18:0/2969939856 >> [v2:172.21.15.18:6800/1605950634,v1:172.21.15.18:6801/1605950634] conn(0x7f8e8806e500 0x7f8e880709b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T20:59:27.863 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:59:27.866+0000 7f8ea2a00700 1 --2- 172.21.15.18:0/2969939856 >> [v2:172.21.15.18:6800/1605950634,v1:172.21.15.18:6801/1605950634] conn(0x7f8e8806e500 0x7f8e880709b0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f8e94009fd0 tx=0x7f8e94009f00 comp rx=0 tx=0).ready entity=mgr.14672 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T20:59:27.867 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:59:27.870+0000 7f8e9aa00700 1 -- 172.21.15.18:0/2969939856 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186454 (secure 0 0 0) 0x7f8e9005bd40 con 0x7f8ea4071ea0 2024-04-12T20:59:28.030 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:59:27 smithi186 ceph-mon[36597]: pgmap v70: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-12T20:59:28.030 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:59:27 smithi186 ceph-mon[36597]: from='mgr.14672 172.21.15.18:0/2209228970' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T20:59:28.101 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:59:27 smithi018 ceph-mon[28799]: pgmap v70: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-12T20:59:28.101 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:59:27 smithi018 ceph-mon[28799]: from='mgr.14672 172.21.15.18:0/2209228970' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T20:59:28.111 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:59:28.114+0000 7f8ea9a00700 1 -- 172.21.15.18:0/2969939856 --> [v2:172.21.15.18:6800/1605950634,v1:172.21.15.18:6801/1605950634] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f8ea4111ac0 con 0x7f8e8806e500 2024-04-12T20:59:28.122 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:59:28.125+0000 7f8e9aa00700 1 -- 172.21.15.18:0/2969939856 <== mgr.14672 v2:172.21.15.18:6800/1605950634 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7f8ea4111ac0 con 0x7f8e8806e500 2024-04-12T20:59:28.122 INFO:teuthology.orchestra.run.smithi018.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-12T20:59:28.123 INFO:teuthology.orchestra.run.smithi018.stdout:alertmanager.smithi018 smithi018 *:9093,9094 starting - - - - 2024-04-12T20:59:28.123 INFO:teuthology.orchestra.run.smithi018.stdout:crash.smithi018 smithi018 running (14m) 22s ago 14m 6660k - 17.2.7-827-gd3b5060a 21de9b7369ad 1ac7603bddfd 2024-04-12T20:59:28.123 INFO:teuthology.orchestra.run.smithi018.stdout:crash.smithi186 smithi186 running (13m) 55s ago 13m 6669k - 17.2.7-827-gd3b5060a 21de9b7369ad 14bf8a1672eb 2024-04-12T20:59:28.123 INFO:teuthology.orchestra.run.smithi018.stdout:grafana.smithi018 smithi018 *:3000 running (3m) 22s ago 14m 76.2M - 9.4.7 954c08fa6188 d6251dcdaec0 2024-04-12T20:59:28.123 INFO:teuthology.orchestra.run.smithi018.stdout:mds.cephfs.smithi018.acqbsc smithi018 running (7m) 22s ago 7m 113M - 17.2.7-827-gd3b5060a 21de9b7369ad 5b5fce65f18a 2024-04-12T20:59:28.123 INFO:teuthology.orchestra.run.smithi018.stdout:mds.cephfs.smithi018.szjluo smithi018 running (7m) 22s ago 7m 115M - 17.2.7-827-gd3b5060a 21de9b7369ad cf9bf0c5031f 2024-04-12T20:59:28.123 INFO:teuthology.orchestra.run.smithi018.stdout:mds.cephfs.smithi186.jdllyn smithi186 running (7m) 55s ago 7m 13.0M - 17.2.7-827-gd3b5060a 21de9b7369ad 2b2bc46bd29a 2024-04-12T20:59:28.123 INFO:teuthology.orchestra.run.smithi018.stdout:mds.cephfs.smithi186.wjnafs smithi186 running (7m) 55s ago 7m 13.0M - 17.2.7-827-gd3b5060a 21de9b7369ad 74b115d6d6fd 2024-04-12T20:59:28.123 INFO:teuthology.orchestra.run.smithi018.stdout:mgr.smithi018.xjzwga smithi018 *:8443,9283,8765 running (2m) 22s ago 16m 498M - 19.0.0-2879-g324baf8d 8717720dc5e5 56088305eab7 2024-04-12T20:59:28.123 INFO:teuthology.orchestra.run.smithi018.stdout:mgr.smithi186.kqcfxk smithi186 *:8443,9283,8765 running (87s) 55s ago 13m 447M - 19.0.0-2879-g324baf8d 8717720dc5e5 09cd766e5e0d 2024-04-12T20:59:28.123 INFO:teuthology.orchestra.run.smithi018.stdout:mon.smithi018 smithi018 running (16m) 22s ago 16m 55.7M 2048M 17.2.7-827-gd3b5060a 21de9b7369ad cd84caafa056 2024-04-12T20:59:28.123 INFO:teuthology.orchestra.run.smithi018.stdout:mon.smithi186 smithi186 running (13m) 55s ago 13m 41.6M 2048M 17.2.7-827-gd3b5060a 21de9b7369ad d7263c772b73 2024-04-12T20:59:28.123 INFO:teuthology.orchestra.run.smithi018.stdout:node-exporter.smithi018 smithi018 *:9100 running (67s) 22s ago 14m 7570k - 1.5.0 0da6a335fe13 326c3d9c6847 2024-04-12T20:59:28.123 INFO:teuthology.orchestra.run.smithi018.stdout:node-exporter.smithi186 smithi186 *:9100 running (60s) 55s ago 13m 2705k - 1.5.0 0da6a335fe13 ec2112baeecf 2024-04-12T20:59:28.124 INFO:teuthology.orchestra.run.smithi018.stdout:osd.0 smithi018 running (12m) 22s ago 12m 224M 1559M 17.2.7-827-gd3b5060a 21de9b7369ad 0d83e027bf0a 2024-04-12T20:59:28.124 INFO:teuthology.orchestra.run.smithi018.stdout:osd.1 smithi018 running (11m) 22s ago 11m 239M 1559M 17.2.7-827-gd3b5060a 21de9b7369ad 6517c9cc7f87 2024-04-12T20:59:28.124 INFO:teuthology.orchestra.run.smithi018.stdout:osd.2 smithi018 running (10m) 22s ago 10m 192M 1559M 17.2.7-827-gd3b5060a 21de9b7369ad 8a83692acaa5 2024-04-12T20:59:28.124 INFO:teuthology.orchestra.run.smithi018.stdout:osd.3 smithi186 running (10m) 55s ago 10m 258M 2581M 17.2.7-827-gd3b5060a 21de9b7369ad c2def7a90d76 2024-04-12T20:59:28.124 INFO:teuthology.orchestra.run.smithi018.stdout:osd.4 smithi186 running (9m) 55s ago 9m 228M 2581M 17.2.7-827-gd3b5060a 21de9b7369ad 757e7c71b7b9 2024-04-12T20:59:28.124 INFO:teuthology.orchestra.run.smithi018.stdout:osd.5 smithi186 running (9m) 55s ago 9m 222M 2581M 17.2.7-827-gd3b5060a 21de9b7369ad 2ee072a436d0 2024-04-12T20:59:28.124 INFO:teuthology.orchestra.run.smithi018.stdout:prometheus.smithi018 smithi018 *:9095 running (32s) 22s ago 13m 37.4M - 2.43.0 a07b618ecd1d f58841127e21 2024-04-12T20:59:28.126 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:59:28.128+0000 7f8ea9a00700 1 -- 172.21.15.18:0/2969939856 >> [v2:172.21.15.18:6800/1605950634,v1:172.21.15.18:6801/1605950634] conn(0x7f8e8806e500 msgr2=0x7f8e880709b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:59:28.126 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:59:28.128+0000 7f8ea9a00700 1 --2- 172.21.15.18:0/2969939856 >> [v2:172.21.15.18:6800/1605950634,v1:172.21.15.18:6801/1605950634] conn(0x7f8e8806e500 0x7f8e880709b0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f8e94009fd0 tx=0x7f8e94009f00 comp rx=0 tx=0).stop 2024-04-12T20:59:28.126 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:59:28.128+0000 7f8ea9a00700 1 -- 172.21.15.18:0/2969939856 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8ea4071ea0 msgr2=0x7f8ea4110180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:59:28.126 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:59:28.128+0000 7f8ea9a00700 1 --2- 172.21.15.18:0/2969939856 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8ea4071ea0 0x7f8ea4110180 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f8e90009150 tx=0x7f8e90009e30 comp rx=0 tx=0).stop 2024-04-12T20:59:28.126 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:59:28.128+0000 7f8ea9a00700 1 -- 172.21.15.18:0/2969939856 shutdown_connections 2024-04-12T20:59:28.126 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:59:28.128+0000 7f8ea9a00700 1 --2- 172.21.15.18:0/2969939856 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8ea40708f0 0x7f8ea410fc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:59:28.126 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:59:28.128+0000 7f8ea9a00700 1 --2- 172.21.15.18:0/2969939856 >> [v2:172.21.15.18:6800/1605950634,v1:172.21.15.18:6801/1605950634] conn(0x7f8e8806e500 0x7f8e880709b0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:59:28.127 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:59:28.128+0000 7f8ea9a00700 1 --2- 172.21.15.18:0/2969939856 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8ea4071ea0 0x7f8ea4110180 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:59:28.127 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:59:28.128+0000 7f8ea9a00700 1 -- 172.21.15.18:0/2969939856 >> 172.21.15.18:0/2969939856 conn(0x7f8ea40f8560 msgr2=0x7f8ea40f9c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T20:59:28.127 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:59:28.128+0000 7f8ea9a00700 1 -- 172.21.15.18:0/2969939856 shutdown_connections 2024-04-12T20:59:28.127 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:59:28.128+0000 7f8ea9a00700 1 -- 172.21.15.18:0/2969939856 wait complete. 2024-04-12T20:59:28.278 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:59:28.280+0000 7f1fea600700 1 -- 172.21.15.18:0/3348695998 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1fe41046d0 msgr2=0x7f1fe4104b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:59:28.278 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:59:28.280+0000 7f1fea600700 1 --2- 172.21.15.18:0/3348695998 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1fe41046d0 0x7f1fe4104b40 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f1fd0004940 tx=0x7f1fd0009e30 comp rx=0 tx=0).stop 2024-04-12T20:59:28.278 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:59:28.280+0000 7f1fea600700 1 -- 172.21.15.18:0/3348695998 shutdown_connections 2024-04-12T20:59:28.278 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:59:28.280+0000 7f1fea600700 1 --2- 172.21.15.18:0/3348695998 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1fe41046d0 0x7f1fe4104b40 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:59:28.278 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:59:28.280+0000 7f1fea600700 1 --2- 172.21.15.18:0/3348695998 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1fe4103420 0x7f1fe4103830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:59:28.278 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:59:28.280+0000 7f1fea600700 1 -- 172.21.15.18:0/3348695998 >> 172.21.15.18:0/3348695998 conn(0x7f1fe40725a0 msgr2=0x7f1fe40749d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T20:59:28.278 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:59:28.280+0000 7f1fea600700 1 -- 172.21.15.18:0/3348695998 shutdown_connections 2024-04-12T20:59:28.278 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:59:28.280+0000 7f1fea600700 1 -- 172.21.15.18:0/3348695998 wait complete. 2024-04-12T20:59:28.279 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:59:28.281+0000 7f1fea600700 1 Processor -- start 2024-04-12T20:59:28.279 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:59:28.281+0000 7f1fea600700 1 -- start start 2024-04-12T20:59:28.279 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:59:28.282+0000 7f1fea600700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1fe4103420 0x7f1fe4113ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T20:59:28.280 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:59:28.282+0000 7f1fea600700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1fe41046d0 0x7f1fe4114530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T20:59:28.280 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:59:28.282+0000 7f1fea600700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1fe4114bc0 con 0x7f1fe4103420 2024-04-12T20:59:28.280 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:59:28.282+0000 7f1fea600700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1fe4119560 con 0x7f1fe41046d0 2024-04-12T20:59:28.280 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:59:28.282+0000 7f1fe3400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1fe4103420 0x7f1fe4113ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T20:59:28.280 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:59:28.282+0000 7f1fe3400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1fe4103420 0x7f1fe4113ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:50870/0 (socket says 172.21.15.18:50870) 2024-04-12T20:59:28.280 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:59:28.282+0000 7f1fe3400700 1 -- 172.21.15.18:0/4138646245 learned_addr learned my addr 172.21.15.18:0/4138646245 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T20:59:28.281 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:59:28.282+0000 7f1fe3400700 1 -- 172.21.15.18:0/4138646245 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1fe41046d0 msgr2=0x7f1fe4114530 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-12T20:59:28.281 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:59:28.282+0000 7f1fe3400700 1 --2- 172.21.15.18:0/4138646245 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1fe41046d0 0x7f1fe4114530 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:59:28.281 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:59:28.282+0000 7f1fe3400700 1 -- 172.21.15.18:0/4138646245 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1fd0005040 con 0x7f1fe4103420 2024-04-12T20:59:28.281 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:59:28.283+0000 7f1fe3400700 1 --2- 172.21.15.18:0/4138646245 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1fe4103420 0x7f1fe4113ff0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f1fd400b580 tx=0x7f1fd40091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T20:59:28.281 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:59:28.283+0000 7f1fdb400700 1 -- 172.21.15.18:0/4138646245 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1fd400f6f0 con 0x7f1fe4103420 2024-04-12T20:59:28.281 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:59:28.283+0000 7f1fdb400700 1 -- 172.21.15.18:0/4138646245 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f1fd400fd20 con 0x7f1fe4103420 2024-04-12T20:59:28.281 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:59:28.283+0000 7f1fea600700 1 -- 172.21.15.18:0/4138646245 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1fe41197c0 con 0x7f1fe4103420 2024-04-12T20:59:28.281 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:59:28.283+0000 7f1fea600700 1 -- 172.21.15.18:0/4138646245 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1fe4119d30 con 0x7f1fe4103420 2024-04-12T20:59:28.281 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:59:28.283+0000 7f1fdb400700 1 -- 172.21.15.18:0/4138646245 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1fd4011ea0 con 0x7f1fe4103420 2024-04-12T20:59:28.282 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:59:28.284+0000 7f1fea600700 1 -- 172.21.15.18:0/4138646245 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1fe404c710 con 0x7f1fe4103420 2024-04-12T20:59:28.283 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:59:28.286+0000 7f1fdb400700 1 -- 172.21.15.18:0/4138646245 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 32) v1 ==== 92459+0+0 (secure 0 0 0) 0x7f1fd400f850 con 0x7f1fe4103420 2024-04-12T20:59:28.284 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:59:28.286+0000 7f1fdb400700 1 --2- 172.21.15.18:0/4138646245 >> [v2:172.21.15.18:6800/1605950634,v1:172.21.15.18:6801/1605950634] conn(0x7f1fc806e500 0x7f1fc80709b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T20:59:28.284 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:59:28.287+0000 7f1fdb400700 1 -- 172.21.15.18:0/4138646245 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6066+0+0 (secure 0 0 0) 0x7f1fd408dc40 con 0x7f1fe4103420 2024-04-12T20:59:28.284 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:59:28.287+0000 7f1fe2a00700 1 --2- 172.21.15.18:0/4138646245 >> [v2:172.21.15.18:6800/1605950634,v1:172.21.15.18:6801/1605950634] conn(0x7f1fc806e500 0x7f1fc80709b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T20:59:28.285 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:59:28.288+0000 7f1fe2a00700 1 --2- 172.21.15.18:0/4138646245 >> [v2:172.21.15.18:6800/1605950634,v1:172.21.15.18:6801/1605950634] conn(0x7f1fc806e500 0x7f1fc80709b0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f1fd0000c00 tx=0x7f1fd0009e30 comp rx=0 tx=0).ready entity=mgr.14672 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T20:59:28.290 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:59:28.292+0000 7f1fdb400700 1 -- 172.21.15.18:0/4138646245 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186454 (secure 0 0 0) 0x7f1fd405ada0 con 0x7f1fe4103420 2024-04-12T20:59:28.621 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:59:28.623+0000 7f1fea600700 1 -- 172.21.15.18:0/4138646245 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f1fe411a2d0 con 0x7f1fe4103420 2024-04-12T20:59:28.621 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:59:28.624+0000 7f1fdb400700 1 -- 172.21.15.18:0/4138646245 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+739 (secure 0 0 0) 0x7f1fd405a930 con 0x7f1fe4103420 2024-04-12T20:59:28.622 INFO:teuthology.orchestra.run.smithi018.stdout:{ 2024-04-12T20:59:28.622 INFO:teuthology.orchestra.run.smithi018.stdout: "mon": { 2024-04-12T20:59:28.622 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 17.2.7-827-gd3b5060a (d3b5060a17449edfae9385cdf792afec109fca5d) quincy (stable)": 2 2024-04-12T20:59:28.622 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-12T20:59:28.622 INFO:teuthology.orchestra.run.smithi018.stdout: "mgr": { 2024-04-12T20:59:28.622 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 19.0.0-2879-g324baf8d (324baf8ddb91dea46cac50cb535c306115791355) squid (dev)": 2 2024-04-12T20:59:28.622 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-12T20:59:28.622 INFO:teuthology.orchestra.run.smithi018.stdout: "osd": { 2024-04-12T20:59:28.622 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 17.2.7-827-gd3b5060a (d3b5060a17449edfae9385cdf792afec109fca5d) quincy (stable)": 6 2024-04-12T20:59:28.623 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-12T20:59:28.623 INFO:teuthology.orchestra.run.smithi018.stdout: "mds": { 2024-04-12T20:59:28.623 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 17.2.7-827-gd3b5060a (d3b5060a17449edfae9385cdf792afec109fca5d) quincy (stable)": 4 2024-04-12T20:59:28.623 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-12T20:59:28.623 INFO:teuthology.orchestra.run.smithi018.stdout: "overall": { 2024-04-12T20:59:28.623 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 17.2.7-827-gd3b5060a (d3b5060a17449edfae9385cdf792afec109fca5d) quincy (stable)": 12, 2024-04-12T20:59:28.623 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 19.0.0-2879-g324baf8d (324baf8ddb91dea46cac50cb535c306115791355) squid (dev)": 2 2024-04-12T20:59:28.623 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-04-12T20:59:28.623 INFO:teuthology.orchestra.run.smithi018.stdout:} 2024-04-12T20:59:28.624 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:59:28.627+0000 7f1fea600700 1 -- 172.21.15.18:0/4138646245 >> [v2:172.21.15.18:6800/1605950634,v1:172.21.15.18:6801/1605950634] conn(0x7f1fc806e500 msgr2=0x7f1fc80709b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:59:28.624 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:59:28.627+0000 7f1fea600700 1 --2- 172.21.15.18:0/4138646245 >> [v2:172.21.15.18:6800/1605950634,v1:172.21.15.18:6801/1605950634] conn(0x7f1fc806e500 0x7f1fc80709b0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f1fd0000c00 tx=0x7f1fd0009e30 comp rx=0 tx=0).stop 2024-04-12T20:59:28.624 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:59:28.627+0000 7f1fea600700 1 -- 172.21.15.18:0/4138646245 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1fe4103420 msgr2=0x7f1fe4113ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:59:28.625 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:59:28.627+0000 7f1fea600700 1 --2- 172.21.15.18:0/4138646245 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1fe4103420 0x7f1fe4113ff0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f1fd400b580 tx=0x7f1fd40091a0 comp rx=0 tx=0).stop 2024-04-12T20:59:28.625 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:59:28.627+0000 7f1fea600700 1 -- 172.21.15.18:0/4138646245 shutdown_connections 2024-04-12T20:59:28.625 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:59:28.627+0000 7f1fea600700 1 --2- 172.21.15.18:0/4138646245 >> [v2:172.21.15.18:6800/1605950634,v1:172.21.15.18:6801/1605950634] conn(0x7f1fc806e500 0x7f1fc80709b0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:59:28.625 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:59:28.627+0000 7f1fea600700 1 --2- 172.21.15.18:0/4138646245 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1fe4103420 0x7f1fe4113ff0 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:59:28.625 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:59:28.627+0000 7f1fea600700 1 --2- 172.21.15.18:0/4138646245 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1fe41046d0 0x7f1fe4114530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:59:28.625 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:59:28.627+0000 7f1fea600700 1 -- 172.21.15.18:0/4138646245 >> 172.21.15.18:0/4138646245 conn(0x7f1fe40725a0 msgr2=0x7f1fe4073c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T20:59:28.626 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:59:28.627+0000 7f1fea600700 1 -- 172.21.15.18:0/4138646245 shutdown_connections 2024-04-12T20:59:28.626 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:59:28.627+0000 7f1fea600700 1 -- 172.21.15.18:0/4138646245 wait complete. 2024-04-12T20:59:28.788 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:59:28.789+0000 7fe59d200700 1 -- 172.21.15.18:0/3551241031 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe5980fe0d0 msgr2=0x7fe5980fe540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:59:28.788 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:59:28.789+0000 7fe59d200700 1 --2- 172.21.15.18:0/3551241031 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe5980fe0d0 0x7fe5980fe540 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7fe584004940 tx=0x7fe584009e30 comp rx=0 tx=0).stop 2024-04-12T20:59:28.788 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:59:28.790+0000 7fe59d200700 1 -- 172.21.15.18:0/3551241031 shutdown_connections 2024-04-12T20:59:28.788 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:59:28.790+0000 7fe59d200700 1 --2- 172.21.15.18:0/3551241031 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe5980fe0d0 0x7fe5980fe540 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:59:28.788 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:59:28.790+0000 7fe59d200700 1 --2- 172.21.15.18:0/3551241031 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe5980fcee0 0x7fe5980fd310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:59:28.788 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:59:28.790+0000 7fe59d200700 1 -- 172.21.15.18:0/3551241031 >> 172.21.15.18:0/3551241031 conn(0x7fe5980f8560 msgr2=0x7fe5980fa970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T20:59:28.788 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:59:28.790+0000 7fe59d200700 1 -- 172.21.15.18:0/3551241031 shutdown_connections 2024-04-12T20:59:28.788 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:59:28.790+0000 7fe59d200700 1 -- 172.21.15.18:0/3551241031 wait complete. 2024-04-12T20:59:28.789 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:59:28.791+0000 7fe59d200700 1 Processor -- start 2024-04-12T20:59:28.789 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:59:28.791+0000 7fe59d200700 1 -- start start 2024-04-12T20:59:28.790 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:59:28.791+0000 7fe59d200700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe5980fcee0 0x7fe59810fc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T20:59:28.790 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:59:28.791+0000 7fe59d200700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe5980fe0d0 0x7fe5981101a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T20:59:28.790 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:59:28.791+0000 7fe59d200700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe598110830 con 0x7fe5980fe0d0 2024-04-12T20:59:28.790 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:59:28.791+0000 7fe59d200700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe598111020 con 0x7fe5980fcee0 2024-04-12T20:59:28.790 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:59:28.792+0000 7fe595400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe5980fe0d0 0x7fe5981101a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T20:59:28.790 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:59:28.792+0000 7fe595400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe5980fe0d0 0x7fe5981101a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:50888/0 (socket says 172.21.15.18:50888) 2024-04-12T20:59:28.790 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:59:28.792+0000 7fe595400700 1 -- 172.21.15.18:0/3739290735 learned_addr learned my addr 172.21.15.18:0/3739290735 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T20:59:28.791 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:59:28.792+0000 7fe595e00700 1 --2- 172.21.15.18:0/3739290735 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe5980fcee0 0x7fe59810fc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T20:59:28.791 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:59:28.792+0000 7fe595400700 1 -- 172.21.15.18:0/3739290735 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe5980fcee0 msgr2=0x7fe59810fc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:59:28.791 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:59:28.792+0000 7fe595400700 1 --2- 172.21.15.18:0/3739290735 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe5980fcee0 0x7fe59810fc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:59:28.791 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:59:28.792+0000 7fe595400700 1 -- 172.21.15.18:0/3739290735 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe584005040 con 0x7fe5980fe0d0 2024-04-12T20:59:28.791 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:59:28.792+0000 7fe595e00700 1 --2- 172.21.15.18:0/3739290735 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe5980fcee0 0x7fe59810fc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-12T20:59:28.791 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:59:28.793+0000 7fe595400700 1 --2- 172.21.15.18:0/3739290735 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe5980fe0d0 0x7fe5981101a0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7fe58400b3b0 tx=0x7fe584009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T20:59:28.791 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:59:28.793+0000 7fe58e000700 1 -- 172.21.15.18:0/3739290735 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe58400e040 con 0x7fe5980fe0d0 2024-04-12T20:59:28.791 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:59:28.793+0000 7fe58e000700 1 -- 172.21.15.18:0/3739290735 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe584004290 con 0x7fe5980fe0d0 2024-04-12T20:59:28.791 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:59:28.793+0000 7fe59d200700 1 -- 172.21.15.18:0/3739290735 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe5981112a0 con 0x7fe5980fe0d0 2024-04-12T20:59:28.791 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:59:28.793+0000 7fe58e000700 1 -- 172.21.15.18:0/3739290735 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe58401c630 con 0x7fe5980fe0d0 2024-04-12T20:59:28.792 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:59:28.793+0000 7fe59d200700 1 -- 172.21.15.18:0/3739290735 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe598111860 con 0x7fe5980fe0d0 2024-04-12T20:59:28.792 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:59:28.795+0000 7fe59d200700 1 -- 172.21.15.18:0/3739290735 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe598101a60 con 0x7fe5980fe0d0 2024-04-12T20:59:28.793 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:59:28.796+0000 7fe58e000700 1 -- 172.21.15.18:0/3739290735 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 32) v1 ==== 92459+0+0 (secure 0 0 0) 0x7fe58400bbf0 con 0x7fe5980fe0d0 2024-04-12T20:59:28.795 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:59:28.797+0000 7fe58e000700 1 --2- 172.21.15.18:0/3739290735 >> [v2:172.21.15.18:6800/1605950634,v1:172.21.15.18:6801/1605950634] conn(0x7fe57c06e510 0x7fe57c0709c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T20:59:28.795 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:59:28.797+0000 7fe58e000700 1 -- 172.21.15.18:0/3739290735 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6066+0+0 (secure 0 0 0) 0x7fe58408f2d0 con 0x7fe5980fe0d0 2024-04-12T20:59:28.795 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:59:28.798+0000 7fe595e00700 1 --2- 172.21.15.18:0/3739290735 >> [v2:172.21.15.18:6800/1605950634,v1:172.21.15.18:6801/1605950634] conn(0x7fe57c06e510 0x7fe57c0709c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T20:59:28.796 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:59:28.798+0000 7fe595e00700 1 --2- 172.21.15.18:0/3739290735 >> [v2:172.21.15.18:6800/1605950634,v1:172.21.15.18:6801/1605950634] conn(0x7fe57c06e510 0x7fe57c0709c0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fe588006fd0 tx=0x7fe588008040 comp rx=0 tx=0).ready entity=mgr.14672 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T20:59:28.800 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:59:28.802+0000 7fe58e000700 1 -- 172.21.15.18:0/3739290735 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186454 (secure 0 0 0) 0x7fe58405c4b0 con 0x7fe5980fe0d0 2024-04-12T20:59:29.030 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:59:28 smithi186 ceph-mon[36597]: from='client.14786 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T20:59:29.030 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:59:28 smithi186 ceph-mon[36597]: from='client.14790 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T20:59:29.030 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:59:28 smithi186 ceph-mon[36597]: from='client.? 172.21.15.18:0/4138646245' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T20:59:29.057 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:59:29.059+0000 7fe59d200700 1 -- 172.21.15.18:0/3739290735 --> [v2:172.21.15.18:6800/1605950634,v1:172.21.15.18:6801/1605950634] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fe598105b40 con 0x7fe57c06e510 2024-04-12T20:59:29.058 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:59:28 smithi018 ceph-mon[28799]: from='client.14786 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T20:59:29.058 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:59:28 smithi018 ceph-mon[28799]: from='client.14790 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T20:59:29.058 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:59:28 smithi018 ceph-mon[28799]: from='client.? 172.21.15.18:0/4138646245' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T20:59:29.060 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:59:29.062+0000 7fe58e000700 1 -- 172.21.15.18:0/3739290735 <== mgr.14672 v2:172.21.15.18:6800/1605950634 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+369 (secure 0 0 0) 0x7fe598105b40 con 0x7fe57c06e510 2024-04-12T20:59:29.060 INFO:teuthology.orchestra.run.smithi018.stdout:{ 2024-04-12T20:59:29.060 INFO:teuthology.orchestra.run.smithi018.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:f4452def3be4d350f4a8e5feaf7ebb1511be9298fe68478ba7a2d6fbd7d69a63", 2024-04-12T20:59:29.060 INFO:teuthology.orchestra.run.smithi018.stdout: "in_progress": true, 2024-04-12T20:59:29.060 INFO:teuthology.orchestra.run.smithi018.stdout: "which": "Upgrading daemons of type(s) mgr", 2024-04-12T20:59:29.060 INFO:teuthology.orchestra.run.smithi018.stdout: "services_complete": [ 2024-04-12T20:59:29.060 INFO:teuthology.orchestra.run.smithi018.stdout: "mgr" 2024-04-12T20:59:29.060 INFO:teuthology.orchestra.run.smithi018.stdout: ], 2024-04-12T20:59:29.060 INFO:teuthology.orchestra.run.smithi018.stdout: "progress": "2/2 daemons upgraded", 2024-04-12T20:59:29.061 INFO:teuthology.orchestra.run.smithi018.stdout: "message": "Currently upgrading alertmanager daemons", 2024-04-12T20:59:29.061 INFO:teuthology.orchestra.run.smithi018.stdout: "is_paused": false 2024-04-12T20:59:29.061 INFO:teuthology.orchestra.run.smithi018.stdout:} 2024-04-12T20:59:29.063 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:59:29.065+0000 7fe59d200700 1 -- 172.21.15.18:0/3739290735 >> [v2:172.21.15.18:6800/1605950634,v1:172.21.15.18:6801/1605950634] conn(0x7fe57c06e510 msgr2=0x7fe57c0709c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:59:29.063 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:59:29.065+0000 7fe59d200700 1 --2- 172.21.15.18:0/3739290735 >> [v2:172.21.15.18:6800/1605950634,v1:172.21.15.18:6801/1605950634] conn(0x7fe57c06e510 0x7fe57c0709c0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fe588006fd0 tx=0x7fe588008040 comp rx=0 tx=0).stop 2024-04-12T20:59:29.063 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:59:29.065+0000 7fe59d200700 1 -- 172.21.15.18:0/3739290735 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe5980fe0d0 msgr2=0x7fe5981101a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:59:29.063 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:59:29.065+0000 7fe59d200700 1 --2- 172.21.15.18:0/3739290735 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe5980fe0d0 0x7fe5981101a0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7fe58400b3b0 tx=0x7fe584009e30 comp rx=0 tx=0).stop 2024-04-12T20:59:29.063 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:59:29.065+0000 7fe59d200700 1 -- 172.21.15.18:0/3739290735 shutdown_connections 2024-04-12T20:59:29.064 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:59:29.065+0000 7fe59d200700 1 --2- 172.21.15.18:0/3739290735 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe5980fcee0 0x7fe59810fc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:59:29.064 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:59:29.065+0000 7fe59d200700 1 --2- 172.21.15.18:0/3739290735 >> [v2:172.21.15.18:6800/1605950634,v1:172.21.15.18:6801/1605950634] conn(0x7fe57c06e510 0x7fe57c0709c0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:59:29.064 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:59:29.065+0000 7fe59d200700 1 --2- 172.21.15.18:0/3739290735 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe5980fe0d0 0x7fe5981101a0 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:59:29.064 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:59:29.065+0000 7fe59d200700 1 -- 172.21.15.18:0/3739290735 >> 172.21.15.18:0/3739290735 conn(0x7fe5980f8560 msgr2=0x7fe5980f9b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T20:59:29.064 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:59:29.065+0000 7fe59d200700 1 -- 172.21.15.18:0/3739290735 shutdown_connections 2024-04-12T20:59:29.064 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:59:29.065+0000 7fe59d200700 1 -- 172.21.15.18:0/3739290735 wait complete. 2024-04-12T20:59:30.030 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:59:29 smithi186 ceph-mon[36597]: from='client.14794 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T20:59:30.030 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:59:29 smithi186 ceph-mon[36597]: pgmap v71: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-12T20:59:30.045 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:59:29 smithi018 ceph-mon[28799]: from='client.14794 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T20:59:30.045 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:59:29 smithi018 ceph-mon[28799]: pgmap v71: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-12T20:59:31.030 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:59:30 smithi186 ceph-mon[36597]: from='client.14802 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T20:59:31.033 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:59:30 smithi018 ceph-mon[28799]: from='client.14802 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T20:59:32.030 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:59:31 smithi186 ceph-mon[36597]: pgmap v72: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-12T20:59:32.101 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:59:31 smithi018 ceph-mon[28799]: pgmap v72: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-12T20:59:33.601 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:59:33 smithi018 ceph-mon[28799]: from='mgr.14672 172.21.15.18:0/2209228970' entity='mgr.smithi018.xjzwga' 2024-04-12T20:59:33.601 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:59:33 smithi018 ceph-mon[28799]: from='mgr.14672 172.21.15.18:0/2209228970' entity='mgr.smithi018.xjzwga' 2024-04-12T20:59:33.601 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:59:33 smithi018 ceph-mon[28799]: pgmap v73: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-12T20:59:33.780 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:59:33 smithi186 ceph-mon[36597]: from='mgr.14672 172.21.15.18:0/2209228970' entity='mgr.smithi018.xjzwga' 2024-04-12T20:59:33.780 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:59:33 smithi186 ceph-mon[36597]: from='mgr.14672 172.21.15.18:0/2209228970' entity='mgr.smithi018.xjzwga' 2024-04-12T20:59:33.780 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:59:33 smithi186 ceph-mon[36597]: pgmap v73: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-12T20:59:35.660 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:59:35 smithi018 ceph-mon[28799]: from='mgr.14672 172.21.15.18:0/2209228970' entity='mgr.smithi018.xjzwga' 2024-04-12T20:59:35.660 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:59:35 smithi018 ceph-mon[28799]: from='mgr.14672 172.21.15.18:0/2209228970' entity='mgr.smithi018.xjzwga' 2024-04-12T20:59:35.660 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:59:35 smithi018 ceph-mon[28799]: pgmap v74: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-12T20:59:35.780 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:59:35 smithi186 ceph-mon[36597]: from='mgr.14672 172.21.15.18:0/2209228970' entity='mgr.smithi018.xjzwga' 2024-04-12T20:59:35.780 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:59:35 smithi186 ceph-mon[36597]: from='mgr.14672 172.21.15.18:0/2209228970' entity='mgr.smithi018.xjzwga' 2024-04-12T20:59:35.780 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:59:35 smithi186 ceph-mon[36597]: pgmap v74: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-12T20:59:37.851 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:59:37 smithi018 ceph-mon[28799]: pgmap v75: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-12T20:59:38.030 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:59:37 smithi186 ceph-mon[36597]: pgmap v75: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-12T20:59:40.018 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:59:39 smithi018 ceph-mon[28799]: pgmap v76: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-12T20:59:40.030 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:59:39 smithi186 ceph-mon[36597]: pgmap v76: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-12T20:59:41.531 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:59:41 smithi186 ceph-mon[36597]: from='mgr.14672 172.21.15.18:0/2209228970' entity='mgr.smithi018.xjzwga' 2024-04-12T20:59:41.531 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:59:41 smithi186 ceph-mon[36597]: from='mgr.14672 172.21.15.18:0/2209228970' entity='mgr.smithi018.xjzwga' 2024-04-12T20:59:41.531 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:59:41 smithi186 ceph-mon[36597]: from='mgr.14672 172.21.15.18:0/2209228970' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-12T20:59:41.531 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:59:41 smithi186 ceph-mon[36597]: from='mgr.14672 172.21.15.18:0/2209228970' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-12T20:59:41.531 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:59:41 smithi186 ceph-mon[36597]: from='mgr.14672 172.21.15.18:0/2209228970' entity='mgr.smithi018.xjzwga' 2024-04-12T20:59:41.531 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:59:41 smithi186 ceph-mon[36597]: pgmap v77: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-12T20:59:41.531 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:59:41 smithi186 ceph-mon[36597]: from='mgr.14672 172.21.15.18:0/2209228970' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-04-12T20:59:41.531 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:59:41 smithi186 ceph-mon[36597]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-04-12T20:59:41.531 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:59:41 smithi186 ceph-mon[36597]: from='mgr.14672 172.21.15.18:0/2209228970' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-12T20:59:41.531 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:59:41 smithi186 ceph-mon[36597]: from='mgr.14672 172.21.15.18:0/2209228970' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T20:59:41.531 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:59:41 smithi186 ceph-mon[36597]: from='mgr.14672 172.21.15.18:0/2209228970' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T20:59:41.532 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:59:41 smithi186 ceph-mon[36597]: from='mgr.14672 172.21.15.18:0/2209228970' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T20:59:41.532 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:59:41 smithi186 ceph-mon[36597]: from='mgr.14672 172.21.15.18:0/2209228970' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T20:59:41.532 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:59:41 smithi186 ceph-mon[36597]: from='mgr.14672 172.21.15.18:0/2209228970' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T20:59:41.532 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:59:41 smithi186 ceph-mon[36597]: from='mgr.14672 172.21.15.18:0/2209228970' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T20:59:41.532 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:59:41 smithi186 ceph-mon[36597]: from='mgr.14672 172.21.15.18:0/2209228970' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T20:59:41.532 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:59:41 smithi186 ceph-mon[36597]: from='mgr.14672 172.21.15.18:0/2209228970' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T20:59:41.532 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:59:41 smithi186 ceph-mon[36597]: from='mgr.14672 172.21.15.18:0/2209228970' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T20:59:41.532 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:59:41 smithi186 ceph-mon[36597]: from='mgr.14672 172.21.15.18:0/2209228970' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T20:59:41.532 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:59:41 smithi186 ceph-mon[36597]: from='mgr.14672 172.21.15.18:0/2209228970' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T20:59:41.566 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:59:41 smithi018 ceph-mon[28799]: from='mgr.14672 172.21.15.18:0/2209228970' entity='mgr.smithi018.xjzwga' 2024-04-12T20:59:41.566 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:59:41 smithi018 ceph-mon[28799]: from='mgr.14672 172.21.15.18:0/2209228970' entity='mgr.smithi018.xjzwga' 2024-04-12T20:59:41.566 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:59:41 smithi018 ceph-mon[28799]: from='mgr.14672 172.21.15.18:0/2209228970' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-12T20:59:41.567 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:59:41 smithi018 ceph-mon[28799]: from='mgr.14672 172.21.15.18:0/2209228970' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-12T20:59:41.567 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:59:41 smithi018 ceph-mon[28799]: from='mgr.14672 172.21.15.18:0/2209228970' entity='mgr.smithi018.xjzwga' 2024-04-12T20:59:41.567 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:59:41 smithi018 ceph-mon[28799]: pgmap v77: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-12T20:59:41.567 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:59:41 smithi018 ceph-mon[28799]: from='mgr.14672 172.21.15.18:0/2209228970' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-04-12T20:59:41.567 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:59:41 smithi018 ceph-mon[28799]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-04-12T20:59:41.567 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:59:41 smithi018 ceph-mon[28799]: from='mgr.14672 172.21.15.18:0/2209228970' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-12T20:59:41.567 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:59:41 smithi018 ceph-mon[28799]: from='mgr.14672 172.21.15.18:0/2209228970' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T20:59:41.567 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:59:41 smithi018 ceph-mon[28799]: from='mgr.14672 172.21.15.18:0/2209228970' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T20:59:41.567 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:59:41 smithi018 ceph-mon[28799]: from='mgr.14672 172.21.15.18:0/2209228970' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T20:59:41.567 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:59:41 smithi018 ceph-mon[28799]: from='mgr.14672 172.21.15.18:0/2209228970' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T20:59:41.567 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:59:41 smithi018 ceph-mon[28799]: from='mgr.14672 172.21.15.18:0/2209228970' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T20:59:41.567 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:59:41 smithi018 ceph-mon[28799]: from='mgr.14672 172.21.15.18:0/2209228970' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T20:59:41.567 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:59:41 smithi018 ceph-mon[28799]: from='mgr.14672 172.21.15.18:0/2209228970' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T20:59:41.567 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:59:41 smithi018 ceph-mon[28799]: from='mgr.14672 172.21.15.18:0/2209228970' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T20:59:41.568 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:59:41 smithi018 ceph-mon[28799]: from='mgr.14672 172.21.15.18:0/2209228970' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T20:59:41.568 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:59:41 smithi018 ceph-mon[28799]: from='mgr.14672 172.21.15.18:0/2209228970' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T20:59:41.568 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:59:41 smithi018 ceph-mon[28799]: from='mgr.14672 172.21.15.18:0/2209228970' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T20:59:43.851 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:59:43 smithi018 ceph-mon[28799]: pgmap v78: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-12T20:59:43.851 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:59:43 smithi018 ceph-mon[28799]: from='mgr.14672 172.21.15.18:0/2209228970' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T20:59:44.030 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:59:43 smithi186 ceph-mon[36597]: pgmap v78: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-12T20:59:44.030 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:59:43 smithi186 ceph-mon[36597]: from='mgr.14672 172.21.15.18:0/2209228970' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T20:59:45.030 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:59:44 smithi186 ceph-mon[36597]: Upgrade: Updating grafana.smithi018 2024-04-12T20:59:45.030 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:59:44 smithi186 ceph-mon[36597]: Deploying daemon grafana.smithi018 on smithi018 2024-04-12T20:59:45.101 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:59:44 smithi018 ceph-mon[28799]: Upgrade: Updating grafana.smithi018 2024-04-12T20:59:45.101 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:59:44 smithi018 ceph-mon[28799]: Deploying daemon grafana.smithi018 on smithi018 2024-04-12T20:59:46.030 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:59:45 smithi186 ceph-mon[36597]: pgmap v79: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-12T20:59:46.101 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:59:45 smithi018 ceph-mon[28799]: pgmap v79: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-12T20:59:48.030 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:59:47 smithi186 ceph-mon[36597]: pgmap v80: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-12T20:59:48.101 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:59:47 smithi018 ceph-mon[28799]: pgmap v80: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-12T20:59:50.030 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:59:49 smithi186 ceph-mon[36597]: pgmap v81: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-12T20:59:50.101 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:59:49 smithi018 ceph-mon[28799]: pgmap v81: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-12T20:59:52.030 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:59:51 smithi186 ceph-mon[36597]: pgmap v82: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-12T20:59:52.101 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:59:51 smithi018 ceph-mon[28799]: pgmap v82: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-12T20:59:53.862 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:59:53 smithi018 ceph-mon[28799]: pgmap v83: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-12T20:59:54.030 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:59:53 smithi186 ceph-mon[36597]: pgmap v83: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-12T20:59:56.030 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:59:55 smithi186 ceph-mon[36597]: pgmap v84: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-12T20:59:56.032 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:59:55 smithi018 ceph-mon[28799]: pgmap v84: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-12T20:59:58.030 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:59:57 smithi186 ceph-mon[36597]: pgmap v85: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-12T20:59:58.030 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:59:57 smithi186 ceph-mon[36597]: from='mgr.14672 172.21.15.18:0/2209228970' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T20:59:58.101 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:59:57 smithi018 ceph-mon[28799]: pgmap v85: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-12T20:59:58.101 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:59:57 smithi018 ceph-mon[28799]: from='mgr.14672 172.21.15.18:0/2209228970' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T20:59:59.251 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:59:59.251+0000 7fee16e00700 1 -- 172.21.15.18:0/3484435154 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fee100ff010 msgr2=0x7fee100ff420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:59:59.251 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:59:59.251+0000 7fee16e00700 1 --2- 172.21.15.18:0/3484435154 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fee100ff010 0x7fee100ff420 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7fee00004940 tx=0x7fee00009e30 comp rx=0 tx=0).stop 2024-04-12T20:59:59.251 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:59:59.252+0000 7fee16e00700 1 -- 172.21.15.18:0/3484435154 shutdown_connections 2024-04-12T20:59:59.251 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:59:59.252+0000 7fee16e00700 1 --2- 172.21.15.18:0/3484435154 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fee101002c0 0x7fee10100730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:59:59.251 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:59:59.252+0000 7fee16e00700 1 --2- 172.21.15.18:0/3484435154 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fee100ff010 0x7fee100ff420 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:59:59.252 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:59:59.252+0000 7fee16e00700 1 -- 172.21.15.18:0/3484435154 >> 172.21.15.18:0/3484435154 conn(0x7fee100fa790 msgr2=0x7fee100fcba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T20:59:59.252 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:59:59.252+0000 7fee16e00700 1 -- 172.21.15.18:0/3484435154 shutdown_connections 2024-04-12T20:59:59.252 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:59:59.252+0000 7fee16e00700 1 -- 172.21.15.18:0/3484435154 wait complete. 2024-04-12T20:59:59.252 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:59:59.253+0000 7fee16e00700 1 Processor -- start 2024-04-12T20:59:59.252 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:59:59.253+0000 7fee16e00700 1 -- start start 2024-04-12T20:59:59.252 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:59:59.253+0000 7fee16e00700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fee100ff010 0x7fee1010fb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T20:59:59.252 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:59:59.253+0000 7fee16e00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fee101002c0 0x7fee10110090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T20:59:59.252 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:59:59.253+0000 7fee16e00700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee10110690 con 0x7fee101002c0 2024-04-12T20:59:59.252 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:59:59.253+0000 7fee16e00700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee10110800 con 0x7fee100ff010 2024-04-12T20:59:59.252 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:59:59.254+0000 7fee0f400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fee101002c0 0x7fee10110090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T20:59:59.252 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:59:59.254+0000 7fee0f400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fee101002c0 0x7fee10110090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:35762/0 (socket says 172.21.15.18:35762) 2024-04-12T20:59:59.252 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:59:59.254+0000 7fee0f400700 1 -- 172.21.15.18:0/1968454311 learned_addr learned my addr 172.21.15.18:0/1968454311 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T20:59:59.252 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:59:59.254+0000 7fee0fe00700 1 --2- 172.21.15.18:0/1968454311 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fee100ff010 0x7fee1010fb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T20:59:59.253 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:59:59.254+0000 7fee0f400700 1 -- 172.21.15.18:0/1968454311 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fee100ff010 msgr2=0x7fee1010fb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:59:59.253 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:59:59.254+0000 7fee0f400700 1 --2- 172.21.15.18:0/1968454311 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fee100ff010 0x7fee1010fb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:59:59.253 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:59:59.254+0000 7fee0f400700 1 -- 172.21.15.18:0/1968454311 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee00005040 con 0x7fee101002c0 2024-04-12T20:59:59.253 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:59:59.255+0000 7fee0f400700 1 --2- 172.21.15.18:0/1968454311 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fee101002c0 0x7fee10110090 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7fedfc002a50 tx=0x7fedfc0046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T20:59:59.254 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:59:59.255+0000 7fee07e00700 1 -- 172.21.15.18:0/1968454311 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fedfc007cc0 con 0x7fee101002c0 2024-04-12T20:59:59.254 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:59:59.255+0000 7fee07e00700 1 -- 172.21.15.18:0/1968454311 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fedfc00f040 con 0x7fee101002c0 2024-04-12T20:59:59.254 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:59:59.255+0000 7fee07e00700 1 -- 172.21.15.18:0/1968454311 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fedfc00da30 con 0x7fee101002c0 2024-04-12T20:59:59.254 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:59:59.255+0000 7fee16e00700 1 -- 172.21.15.18:0/1968454311 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fee10115260 con 0x7fee101002c0 2024-04-12T20:59:59.254 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:59:59.255+0000 7fee16e00700 1 -- 172.21.15.18:0/1968454311 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fee10115820 con 0x7fee101002c0 2024-04-12T20:59:59.254 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:59:59.256+0000 7fee16e00700 1 -- 172.21.15.18:0/1968454311 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fee1004c710 con 0x7fee101002c0 2024-04-12T20:59:59.254 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:59:59.258+0000 7fee07e00700 1 -- 172.21.15.18:0/1968454311 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 32) v1 ==== 92459+0+0 (secure 0 0 0) 0x7fedfc00db90 con 0x7fee101002c0 2024-04-12T20:59:59.255 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:59:59.259+0000 7fee07e00700 1 --2- 172.21.15.18:0/1968454311 >> [v2:172.21.15.18:6800/1605950634,v1:172.21.15.18:6801/1605950634] conn(0x7fedf406e500 0x7fedf40709b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T20:59:59.256 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:59:59.259+0000 7fee07e00700 1 -- 172.21.15.18:0/1968454311 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6066+0+0 (secure 0 0 0) 0x7fedfc08db90 con 0x7fee101002c0 2024-04-12T20:59:59.256 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:59:59.259+0000 7fee0fe00700 1 --2- 172.21.15.18:0/1968454311 >> [v2:172.21.15.18:6800/1605950634,v1:172.21.15.18:6801/1605950634] conn(0x7fedf406e500 0x7fedf40709b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T20:59:59.256 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:59:59.260+0000 7fee0fe00700 1 --2- 172.21.15.18:0/1968454311 >> [v2:172.21.15.18:6800/1605950634,v1:172.21.15.18:6801/1605950634] conn(0x7fedf406e500 0x7fedf40709b0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fee0000b3b0 tx=0x7fee00009e30 comp rx=0 tx=0).ready entity=mgr.14672 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T20:59:59.261 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:59:59.264+0000 7fee07e00700 1 -- 172.21.15.18:0/1968454311 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186454 (secure 0 0 0) 0x7fedfc05ad90 con 0x7fee101002c0 2024-04-12T20:59:59.515 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:59:59.518+0000 7fee16e00700 1 -- 172.21.15.18:0/1968454311 --> [v2:172.21.15.18:6800/1605950634,v1:172.21.15.18:6801/1605950634] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fee10104680 con 0x7fedf406e500 2024-04-12T20:59:59.518 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:59:59.521+0000 7fee07e00700 1 -- 172.21.15.18:0/1968454311 <== mgr.14672 v2:172.21.15.18:6800/1605950634 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+364 (secure 0 0 0) 0x7fee10104680 con 0x7fedf406e500 2024-04-12T20:59:59.522 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:59:59.525+0000 7fee16e00700 1 -- 172.21.15.18:0/1968454311 >> [v2:172.21.15.18:6800/1605950634,v1:172.21.15.18:6801/1605950634] conn(0x7fedf406e500 msgr2=0x7fedf40709b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:59:59.522 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:59:59.525+0000 7fee16e00700 1 --2- 172.21.15.18:0/1968454311 >> [v2:172.21.15.18:6800/1605950634,v1:172.21.15.18:6801/1605950634] conn(0x7fedf406e500 0x7fedf40709b0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fee0000b3b0 tx=0x7fee00009e30 comp rx=0 tx=0).stop 2024-04-12T20:59:59.522 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:59:59.525+0000 7fee16e00700 1 -- 172.21.15.18:0/1968454311 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fee101002c0 msgr2=0x7fee10110090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:59:59.522 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:59:59.525+0000 7fee16e00700 1 --2- 172.21.15.18:0/1968454311 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fee101002c0 0x7fee10110090 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7fedfc002a50 tx=0x7fedfc0046b0 comp rx=0 tx=0).stop 2024-04-12T20:59:59.522 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:59:59.525+0000 7fee16e00700 1 -- 172.21.15.18:0/1968454311 shutdown_connections 2024-04-12T20:59:59.523 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:59:59.525+0000 7fee16e00700 1 --2- 172.21.15.18:0/1968454311 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fee100ff010 0x7fee1010fb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:59:59.523 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:59:59.525+0000 7fee16e00700 1 --2- 172.21.15.18:0/1968454311 >> [v2:172.21.15.18:6800/1605950634,v1:172.21.15.18:6801/1605950634] conn(0x7fedf406e500 0x7fedf40709b0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:59:59.523 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:59:59.525+0000 7fee16e00700 1 --2- 172.21.15.18:0/1968454311 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fee101002c0 0x7fee10110090 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:59:59.523 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:59:59.525+0000 7fee16e00700 1 -- 172.21.15.18:0/1968454311 >> 172.21.15.18:0/1968454311 conn(0x7fee100fa790 msgr2=0x7fee100fbe00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T20:59:59.523 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:59:59.526+0000 7fee16e00700 1 -- 172.21.15.18:0/1968454311 shutdown_connections 2024-04-12T20:59:59.523 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:59:59.526+0000 7fee16e00700 1 -- 172.21.15.18:0/1968454311 wait complete. 2024-04-12T20:59:59.540 INFO:teuthology.orchestra.run.smithi018.stdout:true 2024-04-12T20:59:59.690 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:59:59.692+0000 7f97d7200700 1 -- 172.21.15.18:0/3911377753 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f97d80fbb10 msgr2=0x7f97d80fbf20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:59:59.690 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:59:59.692+0000 7f97d7200700 1 --2- 172.21.15.18:0/3911377753 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f97d80fbb10 0x7f97d80fbf20 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f97c8004940 tx=0x7f97c8009e30 comp rx=0 tx=0).stop 2024-04-12T20:59:59.690 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:59:59.693+0000 7f97d7200700 1 -- 172.21.15.18:0/3911377753 shutdown_connections 2024-04-12T20:59:59.690 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:59:59.693+0000 7f97d7200700 1 --2- 172.21.15.18:0/3911377753 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f97d80fcdc0 0x7f97d80fd230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:59:59.690 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:59:59.693+0000 7f97d7200700 1 --2- 172.21.15.18:0/3911377753 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f97d80fbb10 0x7f97d80fbf20 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:59:59.690 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:59:59.693+0000 7f97d7200700 1 -- 172.21.15.18:0/3911377753 >> 172.21.15.18:0/3911377753 conn(0x7f97d80f7270 msgr2=0x7f97d80f96a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T20:59:59.690 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:59:59.693+0000 7f97d7200700 1 -- 172.21.15.18:0/3911377753 shutdown_connections 2024-04-12T20:59:59.690 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:59:59.693+0000 7f97d7200700 1 -- 172.21.15.18:0/3911377753 wait complete. 2024-04-12T20:59:59.691 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:59:59.694+0000 7f97d7200700 1 Processor -- start 2024-04-12T20:59:59.691 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:59:59.694+0000 7f97d7200700 1 -- start start 2024-04-12T20:59:59.692 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:59:59.694+0000 7f97d7200700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f97d80fbb10 0x7f97d810faf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T20:59:59.692 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:59:59.694+0000 7f97d7200700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f97d80fcdc0 0x7f97d8110030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T20:59:59.692 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:59:59.694+0000 7f97d7200700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97d8110630 con 0x7f97d80fcdc0 2024-04-12T20:59:59.692 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:59:59.694+0000 7f97d7200700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97d8110e30 con 0x7f97d80fbb10 2024-04-12T20:59:59.692 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:59:59.695+0000 7f97d5400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f97d80fcdc0 0x7f97d8110030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T20:59:59.692 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:59:59.695+0000 7f97d5400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f97d80fcdc0 0x7f97d8110030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:35794/0 (socket says 172.21.15.18:35794) 2024-04-12T20:59:59.692 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:59:59.695+0000 7f97d5400700 1 -- 172.21.15.18:0/557548614 learned_addr learned my addr 172.21.15.18:0/557548614 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T20:59:59.692 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:59:59.695+0000 7f97d5e00700 1 --2- 172.21.15.18:0/557548614 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f97d80fbb10 0x7f97d810faf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T20:59:59.693 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:59:59.696+0000 7f97d5400700 1 -- 172.21.15.18:0/557548614 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f97d80fbb10 msgr2=0x7f97d810faf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:59:59.693 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:59:59.696+0000 7f97d5400700 1 --2- 172.21.15.18:0/557548614 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f97d80fbb10 0x7f97d810faf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:59:59.693 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:59:59.696+0000 7f97d5400700 1 -- 172.21.15.18:0/557548614 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97c8005040 con 0x7f97d80fcdc0 2024-04-12T20:59:59.693 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:59:59.696+0000 7f97d5400700 1 --2- 172.21.15.18:0/557548614 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f97d80fcdc0 0x7f97d8110030 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f97c4002a50 tx=0x7f97c40046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T20:59:59.693 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:59:59.696+0000 7f97ce000700 1 -- 172.21.15.18:0/557548614 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f97c4007cc0 con 0x7f97d80fcdc0 2024-04-12T20:59:59.694 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:59:59.696+0000 7f97ce000700 1 -- 172.21.15.18:0/557548614 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f97c400f040 con 0x7f97d80fcdc0 2024-04-12T20:59:59.694 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:59:59.696+0000 7f97ce000700 1 -- 172.21.15.18:0/557548614 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f97c400da70 con 0x7f97d80fcdc0 2024-04-12T20:59:59.694 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:59:59.696+0000 7f97d7200700 1 -- 172.21.15.18:0/557548614 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f97d8111110 con 0x7f97d80fcdc0 2024-04-12T20:59:59.694 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:59:59.697+0000 7f97d7200700 1 -- 172.21.15.18:0/557548614 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f97d8111650 con 0x7f97d80fcdc0 2024-04-12T20:59:59.694 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:59:59.698+0000 7f97d7200700 1 -- 172.21.15.18:0/557548614 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f97d8101750 con 0x7f97d80fcdc0 2024-04-12T20:59:59.696 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:59:59.699+0000 7f97ce000700 1 -- 172.21.15.18:0/557548614 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 32) v1 ==== 92459+0+0 (secure 0 0 0) 0x7f97c400dc10 con 0x7f97d80fcdc0 2024-04-12T20:59:59.697 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:59:59.700+0000 7f97ce000700 1 --2- 172.21.15.18:0/557548614 >> [v2:172.21.15.18:6800/1605950634,v1:172.21.15.18:6801/1605950634] conn(0x7f97bc06e510 0x7f97bc0709c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T20:59:59.697 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:59:59.700+0000 7f97ce000700 1 -- 172.21.15.18:0/557548614 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6066+0+0 (secure 0 0 0) 0x7f97c408d470 con 0x7f97d80fcdc0 2024-04-12T20:59:59.697 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:59:59.701+0000 7f97d5e00700 1 --2- 172.21.15.18:0/557548614 >> [v2:172.21.15.18:6800/1605950634,v1:172.21.15.18:6801/1605950634] conn(0x7f97bc06e510 0x7f97bc0709c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T20:59:59.698 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:59:59.701+0000 7f97d5e00700 1 --2- 172.21.15.18:0/557548614 >> [v2:172.21.15.18:6800/1605950634,v1:172.21.15.18:6801/1605950634] conn(0x7f97bc06e510 0x7f97bc0709c0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f97c800b3b0 tx=0x7f97c8009e30 comp rx=0 tx=0).ready entity=mgr.14672 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T20:59:59.705 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:59:59.709+0000 7f97ce000700 1 -- 172.21.15.18:0/557548614 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186454 (secure 0 0 0) 0x7f97c405a670 con 0x7f97d80fcdc0 2024-04-12T20:59:59.833 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 20:59:59 smithi018 ceph-mon[28799]: pgmap v86: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-12T20:59:59.967 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:59:59.970+0000 7f97d7200700 1 -- 172.21.15.18:0/557548614 --> [v2:172.21.15.18:6800/1605950634,v1:172.21.15.18:6801/1605950634] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f97d8111930 con 0x7f97bc06e510 2024-04-12T20:59:59.969 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:59:59.973+0000 7f97ce000700 1 -- 172.21.15.18:0/557548614 <== mgr.14672 v2:172.21.15.18:6800/1605950634 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+364 (secure 0 0 0) 0x7f97d8111930 con 0x7f97bc06e510 2024-04-12T20:59:59.974 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:59:59.976+0000 7f97d7200700 1 -- 172.21.15.18:0/557548614 >> [v2:172.21.15.18:6800/1605950634,v1:172.21.15.18:6801/1605950634] conn(0x7f97bc06e510 msgr2=0x7f97bc0709c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:59:59.974 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:59:59.976+0000 7f97d7200700 1 --2- 172.21.15.18:0/557548614 >> [v2:172.21.15.18:6800/1605950634,v1:172.21.15.18:6801/1605950634] conn(0x7f97bc06e510 0x7f97bc0709c0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f97c800b3b0 tx=0x7f97c8009e30 comp rx=0 tx=0).stop 2024-04-12T20:59:59.974 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:59:59.976+0000 7f97d7200700 1 -- 172.21.15.18:0/557548614 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f97d80fcdc0 msgr2=0x7f97d8110030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T20:59:59.974 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:59:59.976+0000 7f97d7200700 1 --2- 172.21.15.18:0/557548614 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f97d80fcdc0 0x7f97d8110030 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f97c4002a50 tx=0x7f97c40046b0 comp rx=0 tx=0).stop 2024-04-12T20:59:59.974 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:59:59.976+0000 7f97d7200700 1 -- 172.21.15.18:0/557548614 shutdown_connections 2024-04-12T20:59:59.974 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:59:59.976+0000 7f97d7200700 1 --2- 172.21.15.18:0/557548614 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f97d80fbb10 0x7f97d810faf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:59:59.974 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:59:59.976+0000 7f97d7200700 1 --2- 172.21.15.18:0/557548614 >> [v2:172.21.15.18:6800/1605950634,v1:172.21.15.18:6801/1605950634] conn(0x7f97bc06e510 0x7f97bc0709c0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:59:59.974 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:59:59.976+0000 7f97d7200700 1 --2- 172.21.15.18:0/557548614 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f97d80fcdc0 0x7f97d8110030 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T20:59:59.974 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:59:59.976+0000 7f97d7200700 1 -- 172.21.15.18:0/557548614 >> 172.21.15.18:0/557548614 conn(0x7f97d80f7270 msgr2=0x7f97d8065bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T20:59:59.974 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:59:59.977+0000 7f97d7200700 1 -- 172.21.15.18:0/557548614 shutdown_connections 2024-04-12T20:59:59.974 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T20:59:59.977+0000 7f97d7200700 1 -- 172.21.15.18:0/557548614 wait complete. 2024-04-12T21:00:00.030 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 20:59:59 smithi186 ceph-mon[36597]: pgmap v86: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-12T21:00:00.150 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:00:00.153+0000 7f9679600700 1 -- 172.21.15.18:0/3649802533 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f96740fffa0 msgr2=0x7f9674102380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:00:00.151 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:00:00.153+0000 7f9679600700 1 --2- 172.21.15.18:0/3649802533 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f96740fffa0 0x7f9674102380 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f9660004940 tx=0x7f9660009e30 comp rx=0 tx=0).stop 2024-04-12T21:00:00.151 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:00:00.153+0000 7f9679600700 1 -- 172.21.15.18:0/3649802533 shutdown_connections 2024-04-12T21:00:00.151 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:00:00.153+0000 7f9679600700 1 --2- 172.21.15.18:0/3649802533 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f96740fffa0 0x7f9674102380 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:00:00.151 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:00:00.153+0000 7f9679600700 1 --2- 172.21.15.18:0/3649802533 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9674066240 0x7f96740ffa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:00:00.151 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:00:00.153+0000 7f9679600700 1 -- 172.21.15.18:0/3649802533 >> 172.21.15.18:0/3649802533 conn(0x7f96740f7430 msgr2=0x7f96740f9860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:00:00.151 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:00:00.153+0000 7f9679600700 1 -- 172.21.15.18:0/3649802533 shutdown_connections 2024-04-12T21:00:00.151 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:00:00.154+0000 7f9679600700 1 -- 172.21.15.18:0/3649802533 wait complete. 2024-04-12T21:00:00.151 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:00:00.155+0000 7f9679600700 1 Processor -- start 2024-04-12T21:00:00.152 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:00:00.155+0000 7f9679600700 1 -- start start 2024-04-12T21:00:00.152 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:00:00.155+0000 7f9679600700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9674066240 0x7f9674114070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:00:00.152 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:00:00.155+0000 7f9679600700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f96740fffa0 0x7f96741145b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:00:00.152 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:00:00.155+0000 7f9679600700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9674114bb0 con 0x7f9674066240 2024-04-12T21:00:00.152 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:00:00.155+0000 7f9679600700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96741153f0 con 0x7f96740fffa0 2024-04-12T21:00:00.153 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:00:00.156+0000 7f9672a00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9674066240 0x7f9674114070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:00:00.153 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:00:00.156+0000 7f9672a00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9674066240 0x7f9674114070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:35808/0 (socket says 172.21.15.18:35808) 2024-04-12T21:00:00.153 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:00:00.156+0000 7f9672a00700 1 -- 172.21.15.18:0/3353582543 learned_addr learned my addr 172.21.15.18:0/3353582543 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:00:00.153 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:00:00.156+0000 7f9672000700 1 --2- 172.21.15.18:0/3353582543 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f96740fffa0 0x7f96741145b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:00:00.153 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:00:00.157+0000 7f9672a00700 1 -- 172.21.15.18:0/3353582543 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f96740fffa0 msgr2=0x7f96741145b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:00:00.153 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:00:00.157+0000 7f9672a00700 1 --2- 172.21.15.18:0/3353582543 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f96740fffa0 0x7f96741145b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:00:00.153 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:00:00.157+0000 7f9672a00700 1 -- 172.21.15.18:0/3353582543 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9660005040 con 0x7f9674066240 2024-04-12T21:00:00.154 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:00:00.157+0000 7f9672a00700 1 --2- 172.21.15.18:0/3353582543 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9674066240 0x7f9674114070 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f9664002a80 tx=0x7f9664004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:00:00.154 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:00:00.158+0000 7f966aa00700 1 -- 172.21.15.18:0/3353582543 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9664007cc0 con 0x7f9674066240 2024-04-12T21:00:00.154 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:00:00.158+0000 7f966aa00700 1 -- 172.21.15.18:0/3353582543 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f966400f040 con 0x7f9674066240 2024-04-12T21:00:00.155 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:00:00.158+0000 7f966aa00700 1 -- 172.21.15.18:0/3353582543 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f966400da30 con 0x7f9674066240 2024-04-12T21:00:00.155 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:00:00.158+0000 7f9679600700 1 -- 172.21.15.18:0/3353582543 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f96741156d0 con 0x7f9674066240 2024-04-12T21:00:00.155 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:00:00.158+0000 7f9679600700 1 -- 172.21.15.18:0/3353582543 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9674115c10 con 0x7f9674066240 2024-04-12T21:00:00.156 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:00:00.159+0000 7f9679600700 1 -- 172.21.15.18:0/3353582543 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f967405f5f0 con 0x7f9674066240 2024-04-12T21:00:00.157 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:00:00.161+0000 7f966aa00700 1 -- 172.21.15.18:0/3353582543 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 32) v1 ==== 92459+0+0 (secure 0 0 0) 0x7f966400db90 con 0x7f9674066240 2024-04-12T21:00:00.158 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:00:00.162+0000 7f966aa00700 1 --2- 172.21.15.18:0/3353582543 >> [v2:172.21.15.18:6800/1605950634,v1:172.21.15.18:6801/1605950634] conn(0x7f965806e510 0x7f96580709c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:00:00.158 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:00:00.162+0000 7f966aa00700 1 -- 172.21.15.18:0/3353582543 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6066+0+0 (secure 0 0 0) 0x7f966408db60 con 0x7f9674066240 2024-04-12T21:00:00.159 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:00:00.162+0000 7f9672000700 1 --2- 172.21.15.18:0/3353582543 >> [v2:172.21.15.18:6800/1605950634,v1:172.21.15.18:6801/1605950634] conn(0x7f965806e510 0x7f96580709c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:00:00.159 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:00:00.163+0000 7f9672000700 1 --2- 172.21.15.18:0/3353582543 >> [v2:172.21.15.18:6800/1605950634,v1:172.21.15.18:6801/1605950634] conn(0x7f965806e510 0x7f96580709c0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f966000b7a0 tx=0x7f9660009e30 comp rx=0 tx=0).ready entity=mgr.14672 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:00:00.164 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:00:00.167+0000 7f966aa00700 1 -- 172.21.15.18:0/3353582543 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186454 (secure 0 0 0) 0x7f966405ad60 con 0x7f9674066240 2024-04-12T21:00:00.429 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:00:00.432+0000 7f9679600700 1 -- 172.21.15.18:0/3353582543 --> [v2:172.21.15.18:6800/1605950634,v1:172.21.15.18:6801/1605950634] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f9674116050 con 0x7f965806e510 2024-04-12T21:00:00.440 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:00:00.443+0000 7f966aa00700 1 -- 172.21.15.18:0/3353582543 <== mgr.14672 v2:172.21.15.18:6800/1605950634 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3520 (secure 0 0 0) 0x7f9674116050 con 0x7f965806e510 2024-04-12T21:00:00.440 INFO:teuthology.orchestra.run.smithi018.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-12T21:00:00.441 INFO:teuthology.orchestra.run.smithi018.stdout:alertmanager.smithi018 smithi018 *:9093,9094 running (36s) 28s ago 15m 14.0M - 0.25.0 c8568f914cd2 c332090068e8 2024-04-12T21:00:00.441 INFO:teuthology.orchestra.run.smithi018.stdout:crash.smithi018 smithi018 running (15m) 28s ago 15m 6660k - 17.2.7-827-gd3b5060a 21de9b7369ad 1ac7603bddfd 2024-04-12T21:00:00.441 INFO:teuthology.orchestra.run.smithi018.stdout:crash.smithi186 smithi186 running (13m) 87s ago 13m 6669k - 17.2.7-827-gd3b5060a 21de9b7369ad 14bf8a1672eb 2024-04-12T21:00:00.441 INFO:teuthology.orchestra.run.smithi018.stdout:grafana.smithi018 smithi018 *:3000 starting - - - - 2024-04-12T21:00:00.441 INFO:teuthology.orchestra.run.smithi018.stdout:mds.cephfs.smithi018.acqbsc smithi018 running (7m) 28s ago 7m 112M - 17.2.7-827-gd3b5060a 21de9b7369ad 5b5fce65f18a 2024-04-12T21:00:00.441 INFO:teuthology.orchestra.run.smithi018.stdout:mds.cephfs.smithi018.szjluo smithi018 running (7m) 28s ago 7m 113M - 17.2.7-827-gd3b5060a 21de9b7369ad cf9bf0c5031f 2024-04-12T21:00:00.441 INFO:teuthology.orchestra.run.smithi018.stdout:mds.cephfs.smithi186.jdllyn smithi186 running (7m) 87s ago 7m 13.0M - 17.2.7-827-gd3b5060a 21de9b7369ad 2b2bc46bd29a 2024-04-12T21:00:00.441 INFO:teuthology.orchestra.run.smithi018.stdout:mds.cephfs.smithi186.wjnafs smithi186 running (7m) 87s ago 7m 13.0M - 17.2.7-827-gd3b5060a 21de9b7369ad 74b115d6d6fd 2024-04-12T21:00:00.441 INFO:teuthology.orchestra.run.smithi018.stdout:mgr.smithi018.xjzwga smithi018 *:8443,9283,8765 running (3m) 28s ago 16m 499M - 19.0.0-2879-g324baf8d 8717720dc5e5 56088305eab7 2024-04-12T21:00:00.441 INFO:teuthology.orchestra.run.smithi018.stdout:mgr.smithi186.kqcfxk smithi186 *:8443,9283,8765 running (119s) 87s ago 13m 447M - 19.0.0-2879-g324baf8d 8717720dc5e5 09cd766e5e0d 2024-04-12T21:00:00.441 INFO:teuthology.orchestra.run.smithi018.stdout:mon.smithi018 smithi018 running (16m) 28s ago 16m 55.7M 2048M 17.2.7-827-gd3b5060a 21de9b7369ad cd84caafa056 2024-04-12T21:00:00.441 INFO:teuthology.orchestra.run.smithi018.stdout:mon.smithi186 smithi186 running (13m) 87s ago 13m 41.6M 2048M 17.2.7-827-gd3b5060a 21de9b7369ad d7263c772b73 2024-04-12T21:00:00.441 INFO:teuthology.orchestra.run.smithi018.stdout:node-exporter.smithi018 smithi018 *:9100 running (99s) 28s ago 15m 7222k - 1.5.0 0da6a335fe13 326c3d9c6847 2024-04-12T21:00:00.441 INFO:teuthology.orchestra.run.smithi018.stdout:node-exporter.smithi186 smithi186 *:9100 running (92s) 87s ago 13m 2705k - 1.5.0 0da6a335fe13 ec2112baeecf 2024-04-12T21:00:00.442 INFO:teuthology.orchestra.run.smithi018.stdout:osd.0 smithi018 running (12m) 28s ago 12m 225M 1559M 17.2.7-827-gd3b5060a 21de9b7369ad 0d83e027bf0a 2024-04-12T21:00:00.442 INFO:teuthology.orchestra.run.smithi018.stdout:osd.1 smithi018 running (12m) 28s ago 12m 239M 1559M 17.2.7-827-gd3b5060a 21de9b7369ad 6517c9cc7f87 2024-04-12T21:00:00.442 INFO:teuthology.orchestra.run.smithi018.stdout:osd.2 smithi018 running (11m) 28s ago 11m 193M 1559M 17.2.7-827-gd3b5060a 21de9b7369ad 8a83692acaa5 2024-04-12T21:00:00.442 INFO:teuthology.orchestra.run.smithi018.stdout:osd.3 smithi186 running (10m) 87s ago 10m 258M 2581M 17.2.7-827-gd3b5060a 21de9b7369ad c2def7a90d76 2024-04-12T21:00:00.442 INFO:teuthology.orchestra.run.smithi018.stdout:osd.4 smithi186 running (10m) 87s ago 10m 228M 2581M 17.2.7-827-gd3b5060a 21de9b7369ad 757e7c71b7b9 2024-04-12T21:00:00.442 INFO:teuthology.orchestra.run.smithi018.stdout:osd.5 smithi186 running (9m) 87s ago 9m 222M 2581M 17.2.7-827-gd3b5060a 21de9b7369ad 2ee072a436d0 2024-04-12T21:00:00.442 INFO:teuthology.orchestra.run.smithi018.stdout:prometheus.smithi018 smithi018 *:9095 running (64s) 28s ago 14m 46.0M - 2.43.0 a07b618ecd1d f58841127e21 2024-04-12T21:00:00.444 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:00:00.447+0000 7f9679600700 1 -- 172.21.15.18:0/3353582543 >> [v2:172.21.15.18:6800/1605950634,v1:172.21.15.18:6801/1605950634] conn(0x7f965806e510 msgr2=0x7f96580709c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:00:00.444 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:00:00.447+0000 7f9679600700 1 --2- 172.21.15.18:0/3353582543 >> [v2:172.21.15.18:6800/1605950634,v1:172.21.15.18:6801/1605950634] conn(0x7f965806e510 0x7f96580709c0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f966000b7a0 tx=0x7f9660009e30 comp rx=0 tx=0).stop 2024-04-12T21:00:00.445 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:00:00.447+0000 7f9679600700 1 -- 172.21.15.18:0/3353582543 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9674066240 msgr2=0x7f9674114070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:00:00.445 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:00:00.447+0000 7f9679600700 1 --2- 172.21.15.18:0/3353582543 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9674066240 0x7f9674114070 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f9664002a80 tx=0x7f9664004600 comp rx=0 tx=0).stop 2024-04-12T21:00:00.445 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:00:00.447+0000 7f9679600700 1 -- 172.21.15.18:0/3353582543 shutdown_connections 2024-04-12T21:00:00.445 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:00:00.447+0000 7f9679600700 1 --2- 172.21.15.18:0/3353582543 >> [v2:172.21.15.18:6800/1605950634,v1:172.21.15.18:6801/1605950634] conn(0x7f965806e510 0x7f96580709c0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:00:00.445 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:00:00.447+0000 7f9679600700 1 --2- 172.21.15.18:0/3353582543 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9674066240 0x7f9674114070 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:00:00.445 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:00:00.447+0000 7f9679600700 1 --2- 172.21.15.18:0/3353582543 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f96740fffa0 0x7f96741145b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:00:00.445 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:00:00.447+0000 7f9679600700 1 -- 172.21.15.18:0/3353582543 >> 172.21.15.18:0/3353582543 conn(0x7f96740f7430 msgr2=0x7f96740fbb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:00:00.445 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:00:00.448+0000 7f9679600700 1 -- 172.21.15.18:0/3353582543 shutdown_connections 2024-04-12T21:00:00.445 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:00:00.448+0000 7f9679600700 1 -- 172.21.15.18:0/3353582543 wait complete. 2024-04-12T21:00:00.603 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:00:00.605+0000 7f0da9000700 1 -- 172.21.15.18:0/3278691515 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0da40fffa0 msgr2=0x7f0da4102380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:00:00.603 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:00:00.605+0000 7f0da9000700 1 --2- 172.21.15.18:0/3278691515 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0da40fffa0 0x7f0da4102380 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f0d90004940 tx=0x7f0d90009e30 comp rx=0 tx=0).stop 2024-04-12T21:00:00.603 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:00:00.605+0000 7f0da9000700 1 -- 172.21.15.18:0/3278691515 shutdown_connections 2024-04-12T21:00:00.603 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:00:00.606+0000 7f0da9000700 1 --2- 172.21.15.18:0/3278691515 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0da40fffa0 0x7f0da4102380 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:00:00.603 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:00:00.606+0000 7f0da9000700 1 --2- 172.21.15.18:0/3278691515 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0da4066eb0 0x7f0da40ffa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:00:00.603 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:00:00.606+0000 7f0da9000700 1 -- 172.21.15.18:0/3278691515 >> 172.21.15.18:0/3278691515 conn(0x7f0da40f7450 msgr2=0x7f0da40f9860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:00:00.603 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:00:00.606+0000 7f0da9000700 1 -- 172.21.15.18:0/3278691515 shutdown_connections 2024-04-12T21:00:00.603 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:00:00.606+0000 7f0da9000700 1 -- 172.21.15.18:0/3278691515 wait complete. 2024-04-12T21:00:00.604 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:00:00.607+0000 7f0da9000700 1 Processor -- start 2024-04-12T21:00:00.604 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:00:00.607+0000 7f0da9000700 1 -- start start 2024-04-12T21:00:00.604 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:00:00.608+0000 7f0da9000700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0da4066eb0 0x7f0da410d970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:00:00.604 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:00:00.608+0000 7f0da9000700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0da40fffa0 0x7f0da410deb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:00:00.604 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:00:00.608+0000 7f0da9000700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0da410e4b0 con 0x7f0da40fffa0 2024-04-12T21:00:00.604 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:00:00.608+0000 7f0da9000700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0da410e620 con 0x7f0da4066eb0 2024-04-12T21:00:00.605 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:00:00.608+0000 7f0da1400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0da40fffa0 0x7f0da410deb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:00:00.605 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:00:00.608+0000 7f0da1400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0da40fffa0 0x7f0da410deb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:35816/0 (socket says 172.21.15.18:35816) 2024-04-12T21:00:00.605 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:00:00.608+0000 7f0da1400700 1 -- 172.21.15.18:0/1379115793 learned_addr learned my addr 172.21.15.18:0/1379115793 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:00:00.605 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:00:00.608+0000 7f0da1e00700 1 --2- 172.21.15.18:0/1379115793 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0da4066eb0 0x7f0da410d970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:00:00.605 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:00:00.608+0000 7f0da1400700 1 -- 172.21.15.18:0/1379115793 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0da4066eb0 msgr2=0x7f0da410d970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:00:00.606 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:00:00.608+0000 7f0da1400700 1 --2- 172.21.15.18:0/1379115793 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0da4066eb0 0x7f0da410d970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:00:00.606 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:00:00.608+0000 7f0da1400700 1 -- 172.21.15.18:0/1379115793 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d90005040 con 0x7f0da40fffa0 2024-04-12T21:00:00.606 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:00:00.608+0000 7f0da1e00700 1 --2- 172.21.15.18:0/1379115793 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0da4066eb0 0x7f0da410d970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-12T21:00:00.606 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:00:00.609+0000 7f0da1400700 1 --2- 172.21.15.18:0/1379115793 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0da40fffa0 0x7f0da410deb0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f0d9000b3b0 tx=0x7f0d90009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:00:00.606 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:00:00.609+0000 7f0d9a000700 1 -- 172.21.15.18:0/1379115793 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0d9000e040 con 0x7f0da40fffa0 2024-04-12T21:00:00.607 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:00:00.609+0000 7f0d9a000700 1 -- 172.21.15.18:0/1379115793 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f0d9000bcc0 con 0x7f0da40fffa0 2024-04-12T21:00:00.607 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:00:00.609+0000 7f0da9000700 1 -- 172.21.15.18:0/1379115793 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0da4112fd0 con 0x7f0da40fffa0 2024-04-12T21:00:00.607 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:00:00.609+0000 7f0d9a000700 1 -- 172.21.15.18:0/1379115793 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0d9001c630 con 0x7f0da40fffa0 2024-04-12T21:00:00.607 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:00:00.609+0000 7f0da9000700 1 -- 172.21.15.18:0/1379115793 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0da4113560 con 0x7f0da40fffa0 2024-04-12T21:00:00.608 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:00:00.611+0000 7f0da9000700 1 -- 172.21.15.18:0/1379115793 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0da404c710 con 0x7f0da40fffa0 2024-04-12T21:00:00.608 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:00:00.612+0000 7f0d9a000700 1 -- 172.21.15.18:0/1379115793 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 32) v1 ==== 92459+0+0 (secure 0 0 0) 0x7f0d900139f0 con 0x7f0da40fffa0 2024-04-12T21:00:00.610 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:00:00.613+0000 7f0d9a000700 1 --2- 172.21.15.18:0/1379115793 >> [v2:172.21.15.18:6800/1605950634,v1:172.21.15.18:6801/1605950634] conn(0x7f0d8806e500 0x7f0d880709b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:00:00.610 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:00:00.614+0000 7f0d9a000700 1 -- 172.21.15.18:0/1379115793 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6066+0+0 (secure 0 0 0) 0x7f0d9008ec10 con 0x7f0da40fffa0 2024-04-12T21:00:00.610 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:00:00.614+0000 7f0da1e00700 1 --2- 172.21.15.18:0/1379115793 >> [v2:172.21.15.18:6800/1605950634,v1:172.21.15.18:6801/1605950634] conn(0x7f0d8806e500 0x7f0d880709b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:00:00.611 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:00:00.615+0000 7f0da1e00700 1 --2- 172.21.15.18:0/1379115793 >> [v2:172.21.15.18:6800/1605950634,v1:172.21.15.18:6801/1605950634] conn(0x7f0d8806e500 0x7f0d880709b0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f0d94009fd0 tx=0x7f0d94009f00 comp rx=0 tx=0).ready entity=mgr.14672 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:00:00.614 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:00:00.618+0000 7f0d9a000700 1 -- 172.21.15.18:0/1379115793 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186454 (secure 0 0 0) 0x7f0d9005bd70 con 0x7f0da40fffa0 2024-04-12T21:00:00.851 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:00:00 smithi018 ceph-mon[28799]: from='client.14806 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:00:00.851 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:00:00 smithi018 ceph-mon[28799]: overall HEALTH_OK 2024-04-12T21:00:00.851 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:00:00 smithi018 ceph-mon[28799]: from='mgr.14672 172.21.15.18:0/2209228970' entity='mgr.smithi018.xjzwga' 2024-04-12T21:00:00.851 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:00:00 smithi018 ceph-mon[28799]: from='mgr.14672 172.21.15.18:0/2209228970' entity='mgr.smithi018.xjzwga' 2024-04-12T21:00:00.851 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:00:00 smithi018 ceph-mon[28799]: from='mgr.14672 172.21.15.18:0/2209228970' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-12T21:00:01.030 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:00:00 smithi186 ceph-mon[36597]: from='client.14806 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:00:01.030 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:00:00 smithi186 ceph-mon[36597]: overall HEALTH_OK 2024-04-12T21:00:01.030 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:00:00 smithi186 ceph-mon[36597]: from='mgr.14672 172.21.15.18:0/2209228970' entity='mgr.smithi018.xjzwga' 2024-04-12T21:00:01.030 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:00:00 smithi186 ceph-mon[36597]: from='mgr.14672 172.21.15.18:0/2209228970' entity='mgr.smithi018.xjzwga' 2024-04-12T21:00:01.030 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:00:00 smithi186 ceph-mon[36597]: from='mgr.14672 172.21.15.18:0/2209228970' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-12T21:00:01.035 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:00:01.038+0000 7f0da9000700 1 -- 172.21.15.18:0/1379115793 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f0da4113a30 con 0x7f0da40fffa0 2024-04-12T21:00:01.035 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:00:01.039+0000 7f0d9a000700 1 -- 172.21.15.18:0/1379115793 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+739 (secure 0 0 0) 0x7f0d9005b900 con 0x7f0da40fffa0 2024-04-12T21:00:01.036 INFO:teuthology.orchestra.run.smithi018.stdout:{ 2024-04-12T21:00:01.036 INFO:teuthology.orchestra.run.smithi018.stdout: "mon": { 2024-04-12T21:00:01.036 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 17.2.7-827-gd3b5060a (d3b5060a17449edfae9385cdf792afec109fca5d) quincy (stable)": 2 2024-04-12T21:00:01.036 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-12T21:00:01.036 INFO:teuthology.orchestra.run.smithi018.stdout: "mgr": { 2024-04-12T21:00:01.036 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 19.0.0-2879-g324baf8d (324baf8ddb91dea46cac50cb535c306115791355) squid (dev)": 2 2024-04-12T21:00:01.036 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-12T21:00:01.036 INFO:teuthology.orchestra.run.smithi018.stdout: "osd": { 2024-04-12T21:00:01.037 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 17.2.7-827-gd3b5060a (d3b5060a17449edfae9385cdf792afec109fca5d) quincy (stable)": 6 2024-04-12T21:00:01.037 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-12T21:00:01.037 INFO:teuthology.orchestra.run.smithi018.stdout: "mds": { 2024-04-12T21:00:01.037 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 17.2.7-827-gd3b5060a (d3b5060a17449edfae9385cdf792afec109fca5d) quincy (stable)": 4 2024-04-12T21:00:01.037 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-12T21:00:01.037 INFO:teuthology.orchestra.run.smithi018.stdout: "overall": { 2024-04-12T21:00:01.037 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 17.2.7-827-gd3b5060a (d3b5060a17449edfae9385cdf792afec109fca5d) quincy (stable)": 12, 2024-04-12T21:00:01.037 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 19.0.0-2879-g324baf8d (324baf8ddb91dea46cac50cb535c306115791355) squid (dev)": 2 2024-04-12T21:00:01.037 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-04-12T21:00:01.037 INFO:teuthology.orchestra.run.smithi018.stdout:} 2024-04-12T21:00:01.038 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:00:01.042+0000 7f0da9000700 1 -- 172.21.15.18:0/1379115793 >> [v2:172.21.15.18:6800/1605950634,v1:172.21.15.18:6801/1605950634] conn(0x7f0d8806e500 msgr2=0x7f0d880709b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:00:01.038 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:00:01.042+0000 7f0da9000700 1 --2- 172.21.15.18:0/1379115793 >> [v2:172.21.15.18:6800/1605950634,v1:172.21.15.18:6801/1605950634] conn(0x7f0d8806e500 0x7f0d880709b0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f0d94009fd0 tx=0x7f0d94009f00 comp rx=0 tx=0).stop 2024-04-12T21:00:01.038 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:00:01.042+0000 7f0da9000700 1 -- 172.21.15.18:0/1379115793 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0da40fffa0 msgr2=0x7f0da410deb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:00:01.038 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:00:01.042+0000 7f0da9000700 1 --2- 172.21.15.18:0/1379115793 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0da40fffa0 0x7f0da410deb0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f0d9000b3b0 tx=0x7f0d90009e30 comp rx=0 tx=0).stop 2024-04-12T21:00:01.039 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:00:01.042+0000 7f0da9000700 1 -- 172.21.15.18:0/1379115793 shutdown_connections 2024-04-12T21:00:01.039 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:00:01.042+0000 7f0da9000700 1 --2- 172.21.15.18:0/1379115793 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0da4066eb0 0x7f0da410d970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:00:01.039 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:00:01.042+0000 7f0da9000700 1 --2- 172.21.15.18:0/1379115793 >> [v2:172.21.15.18:6800/1605950634,v1:172.21.15.18:6801/1605950634] conn(0x7f0d8806e500 0x7f0d880709b0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:00:01.039 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:00:01.042+0000 7f0da9000700 1 --2- 172.21.15.18:0/1379115793 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0da40fffa0 0x7f0da410deb0 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:00:01.039 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:00:01.042+0000 7f0da9000700 1 -- 172.21.15.18:0/1379115793 >> 172.21.15.18:0/1379115793 conn(0x7f0da40f7450 msgr2=0x7f0da40f7820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:00:01.039 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:00:01.042+0000 7f0da9000700 1 -- 172.21.15.18:0/1379115793 shutdown_connections 2024-04-12T21:00:01.039 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:00:01.042+0000 7f0da9000700 1 -- 172.21.15.18:0/1379115793 wait complete. 2024-04-12T21:00:01.200 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:00:01.202+0000 7fa93c400700 1 -- 172.21.15.18:0/631055844 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa934070890 msgr2=0x7fa934070d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:00:01.200 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:00:01.202+0000 7fa93c400700 1 --2- 172.21.15.18:0/631055844 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa934070890 0x7fa934070d00 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7fa920004940 tx=0x7fa920009e30 comp rx=0 tx=0).stop 2024-04-12T21:00:01.200 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:00:01.202+0000 7fa93c400700 1 -- 172.21.15.18:0/631055844 shutdown_connections 2024-04-12T21:00:01.200 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:00:01.202+0000 7fa93c400700 1 --2- 172.21.15.18:0/631055844 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa934070890 0x7fa934070d00 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:00:01.200 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:00:01.202+0000 7fa93c400700 1 --2- 172.21.15.18:0/631055844 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa934071e40 0x7fa934070350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:00:01.200 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:00:01.202+0000 7fa93c400700 1 -- 172.21.15.18:0/631055844 >> 172.21.15.18:0/631055844 conn(0x7fa9340f8540 msgr2=0x7fa9340fa970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:00:01.200 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:00:01.203+0000 7fa93c400700 1 -- 172.21.15.18:0/631055844 shutdown_connections 2024-04-12T21:00:01.200 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:00:01.203+0000 7fa93c400700 1 -- 172.21.15.18:0/631055844 wait complete. 2024-04-12T21:00:01.201 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:00:01.203+0000 7fa93c400700 1 Processor -- start 2024-04-12T21:00:01.201 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:00:01.203+0000 7fa93c400700 1 -- start start 2024-04-12T21:00:01.202 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:00:01.204+0000 7fa93c400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa934070890 0x7fa93410fc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:00:01.202 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:00:01.204+0000 7fa93c400700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa934071e40 0x7fa934110160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:00:01.202 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:00:01.204+0000 7fa93c400700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa934110760 con 0x7fa934070890 2024-04-12T21:00:01.202 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:00:01.204+0000 7fa93c400700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa934110fb0 con 0x7fa934071e40 2024-04-12T21:00:01.202 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:00:01.204+0000 7fa939a00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa934070890 0x7fa93410fc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:00:01.202 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:00:01.204+0000 7fa939a00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa934070890 0x7fa93410fc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:36180/0 (socket says 172.21.15.18:36180) 2024-04-12T21:00:01.203 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:00:01.204+0000 7fa939a00700 1 -- 172.21.15.18:0/1230665549 learned_addr learned my addr 172.21.15.18:0/1230665549 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:00:01.203 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:00:01.204+0000 7fa939000700 1 --2- 172.21.15.18:0/1230665549 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa934071e40 0x7fa934110160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:00:01.203 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:00:01.205+0000 7fa939a00700 1 -- 172.21.15.18:0/1230665549 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa934071e40 msgr2=0x7fa934110160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:00:01.203 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:00:01.205+0000 7fa939a00700 1 --2- 172.21.15.18:0/1230665549 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa934071e40 0x7fa934110160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:00:01.203 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:00:01.205+0000 7fa939a00700 1 -- 172.21.15.18:0/1230665549 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa920005040 con 0x7fa934070890 2024-04-12T21:00:01.203 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:00:01.205+0000 7fa939000700 1 --2- 172.21.15.18:0/1230665549 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa934071e40 0x7fa934110160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-12T21:00:01.203 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:00:01.205+0000 7fa939a00700 1 --2- 172.21.15.18:0/1230665549 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa934070890 0x7fa93410fc20 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fa928002a80 tx=0x7fa928004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:00:01.203 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:00:01.205+0000 7fa931600700 1 -- 172.21.15.18:0/1230665549 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa928007cc0 con 0x7fa934070890 2024-04-12T21:00:01.203 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:00:01.205+0000 7fa931600700 1 -- 172.21.15.18:0/1230665549 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa92800f040 con 0x7fa934070890 2024-04-12T21:00:01.203 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:00:01.205+0000 7fa93c400700 1 -- 172.21.15.18:0/1230665549 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa934111290 con 0x7fa934070890 2024-04-12T21:00:01.203 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:00:01.205+0000 7fa931600700 1 -- 172.21.15.18:0/1230665549 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa92800dad0 con 0x7fa934070890 2024-04-12T21:00:01.203 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:00:01.205+0000 7fa93c400700 1 -- 172.21.15.18:0/1230665549 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa9341117d0 con 0x7fa934070890 2024-04-12T21:00:01.204 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:00:01.207+0000 7fa93c400700 1 -- 172.21.15.18:0/1230665549 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa93405f5f0 con 0x7fa934070890 2024-04-12T21:00:01.204 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:00:01.208+0000 7fa931600700 1 -- 172.21.15.18:0/1230665549 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 32) v1 ==== 92459+0+0 (secure 0 0 0) 0x7fa92800dc30 con 0x7fa934070890 2024-04-12T21:00:01.205 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:00:01.208+0000 7fa931600700 1 --2- 172.21.15.18:0/1230665549 >> [v2:172.21.15.18:6800/1605950634,v1:172.21.15.18:6801/1605950634] conn(0x7fa91c06e510 0x7fa91c0709c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:00:01.205 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:00:01.209+0000 7fa931600700 1 -- 172.21.15.18:0/1230665549 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6066+0+0 (secure 0 0 0) 0x7fa92808dc70 con 0x7fa934070890 2024-04-12T21:00:01.205 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:00:01.209+0000 7fa939000700 1 --2- 172.21.15.18:0/1230665549 >> [v2:172.21.15.18:6800/1605950634,v1:172.21.15.18:6801/1605950634] conn(0x7fa91c06e510 0x7fa91c0709c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:00:01.206 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:00:01.210+0000 7fa939000700 1 --2- 172.21.15.18:0/1230665549 >> [v2:172.21.15.18:6800/1605950634,v1:172.21.15.18:6801/1605950634] conn(0x7fa91c06e510 0x7fa91c0709c0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fa92000b020 tx=0x7fa920009e30 comp rx=0 tx=0).ready entity=mgr.14672 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:00:01.210 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:00:01.213+0000 7fa931600700 1 -- 172.21.15.18:0/1230665549 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186454 (secure 0 0 0) 0x7fa92805adf0 con 0x7fa934070890 2024-04-12T21:00:01.466 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:00:01.469+0000 7fa93c400700 1 -- 172.21.15.18:0/1230665549 --> [v2:172.21.15.18:6800/1605950634,v1:172.21.15.18:6801/1605950634] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fa934111b60 con 0x7fa91c06e510 2024-04-12T21:00:01.467 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:00:01.471+0000 7fa931600700 1 -- 172.21.15.18:0/1230665549 <== mgr.14672 v2:172.21.15.18:6800/1605950634 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+364 (secure 0 0 0) 0x7fa934111b60 con 0x7fa91c06e510 2024-04-12T21:00:01.468 INFO:teuthology.orchestra.run.smithi018.stdout:{ 2024-04-12T21:00:01.468 INFO:teuthology.orchestra.run.smithi018.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:f4452def3be4d350f4a8e5feaf7ebb1511be9298fe68478ba7a2d6fbd7d69a63", 2024-04-12T21:00:01.468 INFO:teuthology.orchestra.run.smithi018.stdout: "in_progress": true, 2024-04-12T21:00:01.468 INFO:teuthology.orchestra.run.smithi018.stdout: "which": "Upgrading daemons of type(s) mgr", 2024-04-12T21:00:01.468 INFO:teuthology.orchestra.run.smithi018.stdout: "services_complete": [ 2024-04-12T21:00:01.468 INFO:teuthology.orchestra.run.smithi018.stdout: "mgr" 2024-04-12T21:00:01.468 INFO:teuthology.orchestra.run.smithi018.stdout: ], 2024-04-12T21:00:01.468 INFO:teuthology.orchestra.run.smithi018.stdout: "progress": "2/2 daemons upgraded", 2024-04-12T21:00:01.468 INFO:teuthology.orchestra.run.smithi018.stdout: "message": "Currently upgrading grafana daemons", 2024-04-12T21:00:01.468 INFO:teuthology.orchestra.run.smithi018.stdout: "is_paused": false 2024-04-12T21:00:01.468 INFO:teuthology.orchestra.run.smithi018.stdout:} 2024-04-12T21:00:01.470 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:00:01.473+0000 7fa93c400700 1 -- 172.21.15.18:0/1230665549 >> [v2:172.21.15.18:6800/1605950634,v1:172.21.15.18:6801/1605950634] conn(0x7fa91c06e510 msgr2=0x7fa91c0709c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:00:01.470 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:00:01.473+0000 7fa93c400700 1 --2- 172.21.15.18:0/1230665549 >> [v2:172.21.15.18:6800/1605950634,v1:172.21.15.18:6801/1605950634] conn(0x7fa91c06e510 0x7fa91c0709c0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fa92000b020 tx=0x7fa920009e30 comp rx=0 tx=0).stop 2024-04-12T21:00:01.470 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:00:01.473+0000 7fa93c400700 1 -- 172.21.15.18:0/1230665549 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa934070890 msgr2=0x7fa93410fc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:00:01.470 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:00:01.474+0000 7fa93c400700 1 --2- 172.21.15.18:0/1230665549 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa934070890 0x7fa93410fc20 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fa928002a80 tx=0x7fa928004600 comp rx=0 tx=0).stop 2024-04-12T21:00:01.471 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:00:01.474+0000 7fa93c400700 1 -- 172.21.15.18:0/1230665549 shutdown_connections 2024-04-12T21:00:01.471 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:00:01.474+0000 7fa93c400700 1 --2- 172.21.15.18:0/1230665549 >> [v2:172.21.15.18:6800/1605950634,v1:172.21.15.18:6801/1605950634] conn(0x7fa91c06e510 0x7fa91c0709c0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:00:01.471 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:00:01.474+0000 7fa93c400700 1 --2- 172.21.15.18:0/1230665549 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa934070890 0x7fa93410fc20 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:00:01.471 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:00:01.474+0000 7fa93c400700 1 --2- 172.21.15.18:0/1230665549 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa934071e40 0x7fa934110160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:00:01.471 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:00:01.474+0000 7fa93c400700 1 -- 172.21.15.18:0/1230665549 >> 172.21.15.18:0/1230665549 conn(0x7fa9340f8540 msgr2=0x7fa9340f9c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:00:01.471 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:00:01.474+0000 7fa93c400700 1 -- 172.21.15.18:0/1230665549 shutdown_connections 2024-04-12T21:00:01.471 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:00:01.474+0000 7fa93c400700 1 -- 172.21.15.18:0/1230665549 wait complete. 2024-04-12T21:00:02.031 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:00:01 smithi186 ceph-mon[36597]: from='client.14810 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:00:02.031 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:00:01 smithi186 ceph-mon[36597]: from='client.14814 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:00:02.031 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:00:01 smithi186 ceph-mon[36597]: pgmap v87: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-12T21:00:02.031 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:00:01 smithi186 ceph-mon[36597]: from='client.? 172.21.15.18:0/1379115793' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T21:00:02.101 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:00:01 smithi018 ceph-mon[28799]: from='client.14810 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:00:02.101 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:00:01 smithi018 ceph-mon[28799]: from='client.14814 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:00:02.101 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:00:01 smithi018 ceph-mon[28799]: pgmap v87: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-12T21:00:02.101 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:00:01 smithi018 ceph-mon[28799]: from='client.? 172.21.15.18:0/1379115793' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T21:00:03.030 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:00:02 smithi186 ceph-mon[36597]: from='client.14822 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:00:03.101 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:00:02 smithi018 ceph-mon[28799]: from='client.14822 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:00:04.030 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:00:03 smithi186 ceph-mon[36597]: pgmap v88: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-12T21:00:04.057 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:00:03 smithi018 ceph-mon[28799]: pgmap v88: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-12T21:00:06.030 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:00:05 smithi186 ceph-mon[36597]: pgmap v89: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-12T21:00:06.101 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:00:05 smithi018 ceph-mon[28799]: pgmap v89: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-12T21:00:08.002 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:00:07 smithi018 ceph-mon[28799]: pgmap v90: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-12T21:00:08.002 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:00:07 smithi018 ceph-mon[28799]: from='mgr.14672 172.21.15.18:0/2209228970' entity='mgr.smithi018.xjzwga' 2024-04-12T21:00:08.002 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:00:07 smithi018 ceph-mon[28799]: from='mgr.14672 172.21.15.18:0/2209228970' entity='mgr.smithi018.xjzwga' 2024-04-12T21:00:08.030 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:00:07 smithi186 ceph-mon[36597]: pgmap v90: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-12T21:00:08.030 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:00:07 smithi186 ceph-mon[36597]: from='mgr.14672 172.21.15.18:0/2209228970' entity='mgr.smithi018.xjzwga' 2024-04-12T21:00:08.030 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:00:07 smithi186 ceph-mon[36597]: from='mgr.14672 172.21.15.18:0/2209228970' entity='mgr.smithi018.xjzwga' 2024-04-12T21:00:10.030 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:00:09 smithi186 ceph-mon[36597]: pgmap v91: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-12T21:00:10.030 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:00:09 smithi186 ceph-mon[36597]: from='mgr.14672 172.21.15.18:0/2209228970' entity='mgr.smithi018.xjzwga' 2024-04-12T21:00:10.030 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:00:09 smithi186 ceph-mon[36597]: from='mgr.14672 172.21.15.18:0/2209228970' entity='mgr.smithi018.xjzwga' 2024-04-12T21:00:10.101 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:00:09 smithi018 ceph-mon[28799]: pgmap v91: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-12T21:00:10.101 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:00:09 smithi018 ceph-mon[28799]: from='mgr.14672 172.21.15.18:0/2209228970' entity='mgr.smithi018.xjzwga' 2024-04-12T21:00:10.101 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:00:09 smithi018 ceph-mon[28799]: from='mgr.14672 172.21.15.18:0/2209228970' entity='mgr.smithi018.xjzwga' 2024-04-12T21:00:12.030 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:00:11 smithi186 ceph-mon[36597]: pgmap v92: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-12T21:00:12.082 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:00:11 smithi018 ceph-mon[28799]: pgmap v92: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-12T21:00:13.030 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:00:12 smithi186 ceph-mon[36597]: from='mgr.14672 172.21.15.18:0/2209228970' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T21:00:13.101 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:00:12 smithi018 ceph-mon[28799]: from='mgr.14672 172.21.15.18:0/2209228970' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T21:00:13.851 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:00:13 smithi018 ceph-mon[28799]: pgmap v93: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-12T21:00:14.030 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:00:13 smithi186 ceph-mon[36597]: pgmap v93: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-12T21:00:16.030 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:00:15 smithi186 ceph-mon[36597]: pgmap v94: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-12T21:00:16.031 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:00:15 smithi186 ceph-mon[36597]: from='mgr.14672 172.21.15.18:0/2209228970' entity='mgr.smithi018.xjzwga' 2024-04-12T21:00:16.031 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:00:15 smithi186 ceph-mon[36597]: from='mgr.14672 172.21.15.18:0/2209228970' entity='mgr.smithi018.xjzwga' 2024-04-12T21:00:16.031 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:00:15 smithi186 ceph-mon[36597]: from='mgr.14672 172.21.15.18:0/2209228970' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-12T21:00:16.031 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:00:15 smithi186 ceph-mon[36597]: from='mgr.14672 172.21.15.18:0/2209228970' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-12T21:00:16.031 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:00:15 smithi186 ceph-mon[36597]: from='mgr.14672 172.21.15.18:0/2209228970' entity='mgr.smithi018.xjzwga' 2024-04-12T21:00:16.031 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:00:15 smithi186 ceph-mon[36597]: from='mgr.14672 172.21.15.18:0/2209228970' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-04-12T21:00:16.031 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:00:15 smithi186 ceph-mon[36597]: from='mgr.14672 172.21.15.18:0/2209228970' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-12T21:00:16.031 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:00:15 smithi186 ceph-mon[36597]: from='mgr.14672 172.21.15.18:0/2209228970' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T21:00:16.031 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:00:15 smithi186 ceph-mon[36597]: from='mgr.14672 172.21.15.18:0/2209228970' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T21:00:16.032 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:00:15 smithi186 ceph-mon[36597]: from='mgr.14672 172.21.15.18:0/2209228970' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T21:00:16.032 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:00:15 smithi186 ceph-mon[36597]: from='mgr.14672 172.21.15.18:0/2209228970' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T21:00:16.032 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:00:15 smithi186 ceph-mon[36597]: from='mgr.14672 172.21.15.18:0/2209228970' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T21:00:16.032 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:00:15 smithi186 ceph-mon[36597]: from='mgr.14672 172.21.15.18:0/2209228970' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T21:00:16.032 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:00:15 smithi186 ceph-mon[36597]: from='mgr.14672 172.21.15.18:0/2209228970' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T21:00:16.032 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:00:15 smithi186 ceph-mon[36597]: from='mgr.14672 172.21.15.18:0/2209228970' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T21:00:16.032 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:00:15 smithi186 ceph-mon[36597]: from='mgr.14672 172.21.15.18:0/2209228970' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T21:00:16.032 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:00:15 smithi186 ceph-mon[36597]: from='mgr.14672 172.21.15.18:0/2209228970' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T21:00:16.032 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:00:15 smithi186 ceph-mon[36597]: from='mgr.14672 172.21.15.18:0/2209228970' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T21:00:16.101 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:00:15 smithi018 ceph-mon[28799]: pgmap v94: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-12T21:00:16.102 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:00:15 smithi018 ceph-mon[28799]: from='mgr.14672 172.21.15.18:0/2209228970' entity='mgr.smithi018.xjzwga' 2024-04-12T21:00:16.102 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:00:15 smithi018 ceph-mon[28799]: from='mgr.14672 172.21.15.18:0/2209228970' entity='mgr.smithi018.xjzwga' 2024-04-12T21:00:16.102 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:00:15 smithi018 ceph-mon[28799]: from='mgr.14672 172.21.15.18:0/2209228970' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-12T21:00:16.102 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:00:15 smithi018 ceph-mon[28799]: from='mgr.14672 172.21.15.18:0/2209228970' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-12T21:00:16.102 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:00:15 smithi018 ceph-mon[28799]: from='mgr.14672 172.21.15.18:0/2209228970' entity='mgr.smithi018.xjzwga' 2024-04-12T21:00:16.102 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:00:15 smithi018 ceph-mon[28799]: from='mgr.14672 172.21.15.18:0/2209228970' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-04-12T21:00:16.102 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:00:15 smithi018 ceph-mon[28799]: from='mgr.14672 172.21.15.18:0/2209228970' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-12T21:00:16.102 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:00:15 smithi018 ceph-mon[28799]: from='mgr.14672 172.21.15.18:0/2209228970' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T21:00:16.102 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:00:15 smithi018 ceph-mon[28799]: from='mgr.14672 172.21.15.18:0/2209228970' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T21:00:16.102 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:00:15 smithi018 ceph-mon[28799]: from='mgr.14672 172.21.15.18:0/2209228970' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T21:00:16.102 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:00:15 smithi018 ceph-mon[28799]: from='mgr.14672 172.21.15.18:0/2209228970' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T21:00:16.102 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:00:15 smithi018 ceph-mon[28799]: from='mgr.14672 172.21.15.18:0/2209228970' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T21:00:16.103 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:00:15 smithi018 ceph-mon[28799]: from='mgr.14672 172.21.15.18:0/2209228970' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T21:00:16.103 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:00:15 smithi018 ceph-mon[28799]: from='mgr.14672 172.21.15.18:0/2209228970' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T21:00:16.103 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:00:15 smithi018 ceph-mon[28799]: from='mgr.14672 172.21.15.18:0/2209228970' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T21:00:16.103 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:00:15 smithi018 ceph-mon[28799]: from='mgr.14672 172.21.15.18:0/2209228970' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T21:00:16.103 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:00:15 smithi018 ceph-mon[28799]: from='mgr.14672 172.21.15.18:0/2209228970' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T21:00:16.103 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:00:15 smithi018 ceph-mon[28799]: from='mgr.14672 172.21.15.18:0/2209228970' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T21:00:17.031 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:00:16 smithi186 ceph-mon[36597]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-04-12T21:00:17.031 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:00:16 smithi186 ceph-mon[36597]: from='mgr.14672 172.21.15.18:0/2209228970' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T21:00:17.031 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:00:16 smithi186 ceph-mon[36597]: from='mgr.14672 172.21.15.18:0/2209228970' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T21:00:17.031 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:00:16 smithi186 ceph-mon[36597]: from='mgr.14672 172.21.15.18:0/2209228970' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T21:00:17.031 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:00:16 smithi186 ceph-mon[36597]: Upgrade: Finalizing container_image settings 2024-04-12T21:00:17.031 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:00:16 smithi186 ceph-mon[36597]: from='mgr.14672 172.21.15.18:0/2209228970' entity='mgr.smithi018.xjzwga' 2024-04-12T21:00:17.031 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:00:16 smithi186 ceph-mon[36597]: from='mgr.14672 172.21.15.18:0/2209228970' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr"}]: dispatch 2024-04-12T21:00:17.031 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:00:16 smithi186 ceph-mon[36597]: from='mgr.14672 172.21.15.18:0/2209228970' entity='mgr.smithi018.xjzwga' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mgr"}]': finished 2024-04-12T21:00:17.032 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:00:16 smithi186 ceph-mon[36597]: from='mgr.14672 172.21.15.18:0/2209228970' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-04-12T21:00:17.032 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:00:16 smithi186 ceph-mon[36597]: from='mgr.14672 172.21.15.18:0/2209228970' entity='mgr.smithi018.xjzwga' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mon"}]': finished 2024-04-12T21:00:17.032 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:00:16 smithi186 ceph-mon[36597]: from='mgr.14672 172.21.15.18:0/2209228970' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.crash"}]: dispatch 2024-04-12T21:00:17.032 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:00:16 smithi186 ceph-mon[36597]: from='mgr.14672 172.21.15.18:0/2209228970' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd"}]: dispatch 2024-04-12T21:00:17.032 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:00:16 smithi186 ceph-mon[36597]: from='mgr.14672 172.21.15.18:0/2209228970' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds"}]: dispatch 2024-04-12T21:00:17.032 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:00:16 smithi186 ceph-mon[36597]: from='mgr.14672 172.21.15.18:0/2209228970' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.rgw"}]: dispatch 2024-04-12T21:00:17.032 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:00:16 smithi186 ceph-mon[36597]: from='mgr.14672 172.21.15.18:0/2209228970' entity='mgr.smithi018.xjzwga' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.rgw"}]': finished 2024-04-12T21:00:17.032 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:00:16 smithi186 ceph-mon[36597]: from='mgr.14672 172.21.15.18:0/2209228970' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.rbd-mirror"}]: dispatch 2024-04-12T21:00:17.032 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:00:16 smithi186 ceph-mon[36597]: from='mgr.14672 172.21.15.18:0/2209228970' entity='mgr.smithi018.xjzwga' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.rbd-mirror"}]': finished 2024-04-12T21:00:17.032 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:00:16 smithi186 ceph-mon[36597]: from='mgr.14672 172.21.15.18:0/2209228970' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-04-12T21:00:17.032 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:00:16 smithi186 ceph-mon[36597]: from='mgr.14672 172.21.15.18:0/2209228970' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter"}]: dispatch 2024-04-12T21:00:17.032 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:00:16 smithi186 ceph-mon[36597]: from='mgr.14672 172.21.15.18:0/2209228970' entity='mgr.smithi018.xjzwga' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter"}]': finished 2024-04-12T21:00:17.032 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:00:16 smithi186 ceph-mon[36597]: from='mgr.14672 172.21.15.18:0/2209228970' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.iscsi"}]: dispatch 2024-04-12T21:00:17.032 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:00:16 smithi186 ceph-mon[36597]: from='mgr.14672 172.21.15.18:0/2209228970' entity='mgr.smithi018.xjzwga' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.iscsi"}]': finished 2024-04-12T21:00:17.033 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:00:16 smithi186 ceph-mon[36597]: from='mgr.14672 172.21.15.18:0/2209228970' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.nfs"}]: dispatch 2024-04-12T21:00:17.033 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:00:16 smithi186 ceph-mon[36597]: from='mgr.14672 172.21.15.18:0/2209228970' entity='mgr.smithi018.xjzwga' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.nfs"}]': finished 2024-04-12T21:00:17.033 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:00:16 smithi186 ceph-mon[36597]: from='mgr.14672 172.21.15.18:0/2209228970' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.nvmeof"}]: dispatch 2024-04-12T21:00:17.033 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:00:16 smithi186 ceph-mon[36597]: from='mgr.14672 172.21.15.18:0/2209228970' entity='mgr.smithi018.xjzwga' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.nvmeof"}]': finished 2024-04-12T21:00:17.033 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:00:16 smithi186 ceph-mon[36597]: from='mgr.14672 172.21.15.18:0/2209228970' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-04-12T21:00:17.033 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:00:16 smithi186 ceph-mon[36597]: from='mgr.14672 172.21.15.18:0/2209228970' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-04-12T21:00:17.033 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:00:16 smithi186 ceph-mon[36597]: from='mgr.14672 172.21.15.18:0/2209228970' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-04-12T21:00:17.033 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:00:16 smithi186 ceph-mon[36597]: from='mgr.14672 172.21.15.18:0/2209228970' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-04-12T21:00:17.033 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:00:16 smithi186 ceph-mon[36597]: from='mgr.14672 172.21.15.18:0/2209228970' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-04-12T21:00:17.034 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:00:16 smithi186 ceph-mon[36597]: from='mgr.14672 172.21.15.18:0/2209228970' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-04-12T21:00:17.034 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:00:16 smithi186 ceph-mon[36597]: from='mgr.14672 172.21.15.18:0/2209228970' entity='mgr.smithi018.xjzwga' cmd=[{"prefix":"config-key del","key":"mgr/cephadm/upgrade_state"}]: dispatch 2024-04-12T21:00:17.034 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:00:16 smithi186 ceph-mon[36597]: from='mgr.14672 172.21.15.18:0/2209228970' entity='mgr.smithi018.xjzwga' cmd='[{"prefix":"config-key del","key":"mgr/cephadm/upgrade_state"}]': finished 2024-04-12T21:00:17.034 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:00:16 smithi186 ceph-mon[36597]: from='mgr.14672 172.21.15.18:0/2209228970' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-12T21:00:17.034 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:00:16 smithi186 ceph-mon[36597]: from='mgr.14672 172.21.15.18:0/2209228970' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-12T21:00:17.034 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:00:16 smithi186 ceph-mon[36597]: from='mgr.14672 172.21.15.18:0/2209228970' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-12T21:00:17.034 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:00:16 smithi186 ceph-mon[36597]: from='mgr.14672 172.21.15.18:0/2209228970' entity='mgr.smithi018.xjzwga' 2024-04-12T21:00:17.034 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:00:16 smithi186 ceph-mon[36597]: from='mgr.14672 172.21.15.18:0/2209228970' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-12T21:00:17.034 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:00:16 smithi186 ceph-mon[36597]: from='mgr.14672 172.21.15.18:0/2209228970' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-12T21:00:17.034 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:00:16 smithi186 ceph-mon[36597]: from='mgr.14672 172.21.15.18:0/2209228970' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-12T21:00:17.034 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:00:16 smithi186 ceph-mon[36597]: from='mgr.14672 172.21.15.18:0/2209228970' entity='mgr.smithi018.xjzwga' 2024-04-12T21:00:17.102 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:00:16 smithi018 ceph-mon[28799]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-04-12T21:00:17.102 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:00:16 smithi018 ceph-mon[28799]: from='mgr.14672 172.21.15.18:0/2209228970' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T21:00:17.102 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:00:16 smithi018 ceph-mon[28799]: from='mgr.14672 172.21.15.18:0/2209228970' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T21:00:17.102 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:00:16 smithi018 ceph-mon[28799]: from='mgr.14672 172.21.15.18:0/2209228970' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T21:00:17.102 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:00:16 smithi018 ceph-mon[28799]: Upgrade: Finalizing container_image settings 2024-04-12T21:00:17.102 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:00:16 smithi018 ceph-mon[28799]: from='mgr.14672 172.21.15.18:0/2209228970' entity='mgr.smithi018.xjzwga' 2024-04-12T21:00:17.102 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:00:16 smithi018 ceph-mon[28799]: from='mgr.14672 172.21.15.18:0/2209228970' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr"}]: dispatch 2024-04-12T21:00:17.102 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:00:16 smithi018 ceph-mon[28799]: from='mgr.14672 172.21.15.18:0/2209228970' entity='mgr.smithi018.xjzwga' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mgr"}]': finished 2024-04-12T21:00:17.102 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:00:16 smithi018 ceph-mon[28799]: from='mgr.14672 172.21.15.18:0/2209228970' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-04-12T21:00:17.102 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:00:16 smithi018 ceph-mon[28799]: from='mgr.14672 172.21.15.18:0/2209228970' entity='mgr.smithi018.xjzwga' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mon"}]': finished 2024-04-12T21:00:17.103 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:00:16 smithi018 ceph-mon[28799]: from='mgr.14672 172.21.15.18:0/2209228970' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.crash"}]: dispatch 2024-04-12T21:00:17.103 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:00:16 smithi018 ceph-mon[28799]: from='mgr.14672 172.21.15.18:0/2209228970' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd"}]: dispatch 2024-04-12T21:00:17.103 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:00:16 smithi018 ceph-mon[28799]: from='mgr.14672 172.21.15.18:0/2209228970' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds"}]: dispatch 2024-04-12T21:00:17.103 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:00:16 smithi018 ceph-mon[28799]: from='mgr.14672 172.21.15.18:0/2209228970' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.rgw"}]: dispatch 2024-04-12T21:00:17.103 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:00:16 smithi018 ceph-mon[28799]: from='mgr.14672 172.21.15.18:0/2209228970' entity='mgr.smithi018.xjzwga' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.rgw"}]': finished 2024-04-12T21:00:17.103 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:00:16 smithi018 ceph-mon[28799]: from='mgr.14672 172.21.15.18:0/2209228970' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.rbd-mirror"}]: dispatch 2024-04-12T21:00:17.103 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:00:16 smithi018 ceph-mon[28799]: from='mgr.14672 172.21.15.18:0/2209228970' entity='mgr.smithi018.xjzwga' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.rbd-mirror"}]': finished 2024-04-12T21:00:17.103 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:00:16 smithi018 ceph-mon[28799]: from='mgr.14672 172.21.15.18:0/2209228970' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-04-12T21:00:17.103 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:00:16 smithi018 ceph-mon[28799]: from='mgr.14672 172.21.15.18:0/2209228970' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter"}]: dispatch 2024-04-12T21:00:17.103 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:00:16 smithi018 ceph-mon[28799]: from='mgr.14672 172.21.15.18:0/2209228970' entity='mgr.smithi018.xjzwga' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter"}]': finished 2024-04-12T21:00:17.103 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:00:16 smithi018 ceph-mon[28799]: from='mgr.14672 172.21.15.18:0/2209228970' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.iscsi"}]: dispatch 2024-04-12T21:00:17.103 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:00:16 smithi018 ceph-mon[28799]: from='mgr.14672 172.21.15.18:0/2209228970' entity='mgr.smithi018.xjzwga' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.iscsi"}]': finished 2024-04-12T21:00:17.104 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:00:16 smithi018 ceph-mon[28799]: from='mgr.14672 172.21.15.18:0/2209228970' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.nfs"}]: dispatch 2024-04-12T21:00:17.104 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:00:16 smithi018 ceph-mon[28799]: from='mgr.14672 172.21.15.18:0/2209228970' entity='mgr.smithi018.xjzwga' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.nfs"}]': finished 2024-04-12T21:00:17.104 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:00:16 smithi018 ceph-mon[28799]: from='mgr.14672 172.21.15.18:0/2209228970' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.nvmeof"}]: dispatch 2024-04-12T21:00:17.104 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:00:16 smithi018 ceph-mon[28799]: from='mgr.14672 172.21.15.18:0/2209228970' entity='mgr.smithi018.xjzwga' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.nvmeof"}]': finished 2024-04-12T21:00:17.104 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:00:16 smithi018 ceph-mon[28799]: from='mgr.14672 172.21.15.18:0/2209228970' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-04-12T21:00:17.104 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:00:16 smithi018 ceph-mon[28799]: from='mgr.14672 172.21.15.18:0/2209228970' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-04-12T21:00:17.104 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:00:16 smithi018 ceph-mon[28799]: from='mgr.14672 172.21.15.18:0/2209228970' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-04-12T21:00:17.104 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:00:16 smithi018 ceph-mon[28799]: from='mgr.14672 172.21.15.18:0/2209228970' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-04-12T21:00:17.104 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:00:16 smithi018 ceph-mon[28799]: from='mgr.14672 172.21.15.18:0/2209228970' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-04-12T21:00:17.104 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:00:16 smithi018 ceph-mon[28799]: from='mgr.14672 172.21.15.18:0/2209228970' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-04-12T21:00:17.104 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:00:16 smithi018 ceph-mon[28799]: from='mgr.14672 172.21.15.18:0/2209228970' entity='mgr.smithi018.xjzwga' cmd=[{"prefix":"config-key del","key":"mgr/cephadm/upgrade_state"}]: dispatch 2024-04-12T21:00:17.104 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:00:16 smithi018 ceph-mon[28799]: from='mgr.14672 172.21.15.18:0/2209228970' entity='mgr.smithi018.xjzwga' cmd='[{"prefix":"config-key del","key":"mgr/cephadm/upgrade_state"}]': finished 2024-04-12T21:00:17.105 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:00:16 smithi018 ceph-mon[28799]: from='mgr.14672 172.21.15.18:0/2209228970' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-12T21:00:17.105 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:00:16 smithi018 ceph-mon[28799]: from='mgr.14672 172.21.15.18:0/2209228970' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-12T21:00:17.105 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:00:16 smithi018 ceph-mon[28799]: from='mgr.14672 172.21.15.18:0/2209228970' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-12T21:00:17.105 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:00:16 smithi018 ceph-mon[28799]: from='mgr.14672 172.21.15.18:0/2209228970' entity='mgr.smithi018.xjzwga' 2024-04-12T21:00:17.105 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:00:16 smithi018 ceph-mon[28799]: from='mgr.14672 172.21.15.18:0/2209228970' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-12T21:00:17.105 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:00:16 smithi018 ceph-mon[28799]: from='mgr.14672 172.21.15.18:0/2209228970' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-12T21:00:17.105 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:00:16 smithi018 ceph-mon[28799]: from='mgr.14672 172.21.15.18:0/2209228970' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-12T21:00:17.105 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:00:16 smithi018 ceph-mon[28799]: from='mgr.14672 172.21.15.18:0/2209228970' entity='mgr.smithi018.xjzwga' 2024-04-12T21:00:18.030 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:00:17 smithi186 ceph-mon[36597]: Upgrade: Complete! 2024-04-12T21:00:18.030 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:00:17 smithi186 ceph-mon[36597]: pgmap v95: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-12T21:00:18.030 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:00:17 smithi186 ceph-mon[36597]: from='mgr.14672 172.21.15.18:0/2209228970' entity='mgr.smithi018.xjzwga' 2024-04-12T21:00:18.101 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:00:17 smithi018 ceph-mon[28799]: Upgrade: Complete! 2024-04-12T21:00:18.101 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:00:17 smithi018 ceph-mon[28799]: pgmap v95: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-12T21:00:18.101 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:00:17 smithi018 ceph-mon[28799]: from='mgr.14672 172.21.15.18:0/2209228970' entity='mgr.smithi018.xjzwga' 2024-04-12T21:00:20.030 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:00:19 smithi186 ceph-mon[36597]: pgmap v96: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-12T21:00:20.101 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:00:19 smithi018 ceph-mon[28799]: pgmap v96: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-12T21:00:22.030 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:00:21 smithi186 ceph-mon[36597]: pgmap v97: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-12T21:00:22.101 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:00:21 smithi018 ceph-mon[28799]: pgmap v97: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-12T21:00:24.030 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:00:23 smithi186 ceph-mon[36597]: pgmap v98: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-12T21:00:24.101 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:00:23 smithi018 ceph-mon[28799]: pgmap v98: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-12T21:00:26.030 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:00:25 smithi186 ceph-mon[36597]: pgmap v99: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-12T21:00:26.101 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:00:25 smithi018 ceph-mon[28799]: pgmap v99: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-12T21:00:28.030 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:00:27 smithi186 ceph-mon[36597]: pgmap v100: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-12T21:00:28.030 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:00:27 smithi186 ceph-mon[36597]: from='mgr.14672 172.21.15.18:0/2209228970' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T21:00:28.101 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:00:27 smithi018 ceph-mon[28799]: pgmap v100: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-12T21:00:28.101 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:00:27 smithi018 ceph-mon[28799]: from='mgr.14672 172.21.15.18:0/2209228970' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T21:00:30.101 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:00:29 smithi018 ceph-mon[28799]: pgmap v101: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-12T21:00:30.280 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:00:29 smithi186 ceph-mon[36597]: pgmap v101: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-12T21:00:31.647 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:00:31.649+0000 7f010c800700 1 -- 172.21.15.18:0/2291251900 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f01040708f0 msgr2=0x7f0104070d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:00:31.648 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:00:31.649+0000 7f010c800700 1 --2- 172.21.15.18:0/2291251900 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f01040708f0 0x7f0104070d60 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f00f4004940 tx=0x7f00f4009e30 comp rx=0 tx=0).stop 2024-04-12T21:00:31.648 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:00:31.649+0000 7f010c800700 1 -- 172.21.15.18:0/2291251900 shutdown_connections 2024-04-12T21:00:31.648 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:00:31.649+0000 7f010c800700 1 --2- 172.21.15.18:0/2291251900 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f01040708f0 0x7f0104070d60 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:00:31.648 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:00:31.649+0000 7f010c800700 1 --2- 172.21.15.18:0/2291251900 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0104071ea0 0x7f01040703b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:00:31.648 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:00:31.649+0000 7f010c800700 1 -- 172.21.15.18:0/2291251900 >> 172.21.15.18:0/2291251900 conn(0x7f01040f8550 msgr2=0x7f01040fa960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:00:31.648 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:00:31.649+0000 7f010c800700 1 -- 172.21.15.18:0/2291251900 shutdown_connections 2024-04-12T21:00:31.648 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:00:31.649+0000 7f010c800700 1 -- 172.21.15.18:0/2291251900 wait complete. 2024-04-12T21:00:31.648 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:00:31.650+0000 7f010c800700 1 Processor -- start 2024-04-12T21:00:31.649 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:00:31.650+0000 7f010c800700 1 -- start start 2024-04-12T21:00:31.651 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:00:31.650+0000 7f010c800700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f01040708f0 0x7f0104113f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:00:31.651 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:00:31.650+0000 7f010c800700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0104071ea0 0x7f0104114480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:00:31.651 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:00:31.650+0000 7f010c800700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0104114a80 con 0x7f0104071ea0 2024-04-12T21:00:31.651 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:00:31.650+0000 7f010c800700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0104114bf0 con 0x7f01040708f0 2024-04-12T21:00:31.651 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:00:31.651+0000 7f0109400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0104071ea0 0x7f0104114480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:00:31.651 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:00:31.651+0000 7f0109400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0104071ea0 0x7f0104114480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:48092/0 (socket says 172.21.15.18:48092) 2024-04-12T21:00:31.651 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:00:31.651+0000 7f0109400700 1 -- 172.21.15.18:0/1708113261 learned_addr learned my addr 172.21.15.18:0/1708113261 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:00:31.651 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:00:31.651+0000 7f0109e00700 1 --2- 172.21.15.18:0/1708113261 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f01040708f0 0x7f0104113f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:00:31.651 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:00:31.651+0000 7f0109400700 1 -- 172.21.15.18:0/1708113261 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f01040708f0 msgr2=0x7f0104113f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:00:31.652 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:00:31.651+0000 7f0109400700 1 --2- 172.21.15.18:0/1708113261 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f01040708f0 0x7f0104113f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:00:31.652 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:00:31.651+0000 7f0109400700 1 -- 172.21.15.18:0/1708113261 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00f4005040 con 0x7f0104071ea0 2024-04-12T21:00:31.652 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:00:31.652+0000 7f0109400700 1 --2- 172.21.15.18:0/1708113261 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0104071ea0 0x7f0104114480 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f00f400b3b0 tx=0x7f00f4009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:00:31.652 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:00:31.652+0000 7f00fe000700 1 -- 172.21.15.18:0/1708113261 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f00f400e040 con 0x7f0104071ea0 2024-04-12T21:00:31.652 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:00:31.652+0000 7f00fe000700 1 -- 172.21.15.18:0/1708113261 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f00f4004290 con 0x7f0104071ea0 2024-04-12T21:00:31.652 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:00:31.652+0000 7f00fe000700 1 -- 172.21.15.18:0/1708113261 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f00f401c630 con 0x7f0104071ea0 2024-04-12T21:00:31.652 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:00:31.652+0000 7f010c800700 1 -- 172.21.15.18:0/1708113261 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f01041195f0 con 0x7f0104071ea0 2024-04-12T21:00:31.652 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:00:31.652+0000 7f010c800700 1 -- 172.21.15.18:0/1708113261 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0104119bb0 con 0x7f0104071ea0 2024-04-12T21:00:31.652 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:00:31.653+0000 7f010c800700 1 -- 172.21.15.18:0/1708113261 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f010404c710 con 0x7f0104071ea0 2024-04-12T21:00:31.653 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:00:31.654+0000 7f00fe000700 1 -- 172.21.15.18:0/1708113261 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 32) v1 ==== 92459+0+0 (secure 0 0 0) 0x7f00f400bbf0 con 0x7f0104071ea0 2024-04-12T21:00:31.653 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:00:31.655+0000 7f00fe000700 1 --2- 172.21.15.18:0/1708113261 >> [v2:172.21.15.18:6800/1605950634,v1:172.21.15.18:6801/1605950634] conn(0x7f00ec06e500 0x7f00ec0709b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:00:31.653 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:00:31.655+0000 7f00fe000700 1 -- 172.21.15.18:0/1708113261 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6066+0+0 (secure 0 0 0) 0x7f00f408f2d0 con 0x7f0104071ea0 2024-04-12T21:00:31.653 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:00:31.656+0000 7f0109e00700 1 --2- 172.21.15.18:0/1708113261 >> [v2:172.21.15.18:6800/1605950634,v1:172.21.15.18:6801/1605950634] conn(0x7f00ec06e500 0x7f00ec0709b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:00:31.654 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:00:31.656+0000 7f0109e00700 1 --2- 172.21.15.18:0/1708113261 >> [v2:172.21.15.18:6800/1605950634,v1:172.21.15.18:6801/1605950634] conn(0x7f00ec06e500 0x7f00ec0709b0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f00f8007fd0 tx=0x7f00f8009040 comp rx=0 tx=0).ready entity=mgr.14672 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:00:31.658 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:00:31.661+0000 7f00fe000700 1 -- 172.21.15.18:0/1708113261 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186454 (secure 0 0 0) 0x7f00f405c4b0 con 0x7f0104071ea0 2024-04-12T21:00:31.912 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:00:31.913+0000 7f010c800700 1 -- 172.21.15.18:0/1708113261 --> [v2:172.21.15.18:6800/1605950634,v1:172.21.15.18:6801/1605950634] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f01040ff7b0 con 0x7f00ec06e500 2024-04-12T21:00:31.912 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:00:31 smithi018 ceph-mon[28799]: pgmap v102: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-12T21:00:31.913 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:00:31.915+0000 7f00fe000700 1 -- 172.21.15.18:0/1708113261 <== mgr.14672 v2:172.21.15.18:6800/1605950634 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+175 (secure 0 0 0) 0x7f01040ff7b0 con 0x7f00ec06e500 2024-04-12T21:00:31.916 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:00:31.918+0000 7f010c800700 1 -- 172.21.15.18:0/1708113261 >> [v2:172.21.15.18:6800/1605950634,v1:172.21.15.18:6801/1605950634] conn(0x7f00ec06e500 msgr2=0x7f00ec0709b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:00:31.916 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:00:31.918+0000 7f010c800700 1 --2- 172.21.15.18:0/1708113261 >> [v2:172.21.15.18:6800/1605950634,v1:172.21.15.18:6801/1605950634] conn(0x7f00ec06e500 0x7f00ec0709b0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f00f8007fd0 tx=0x7f00f8009040 comp rx=0 tx=0).stop 2024-04-12T21:00:31.916 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:00:31.918+0000 7f010c800700 1 -- 172.21.15.18:0/1708113261 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0104071ea0 msgr2=0x7f0104114480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:00:31.917 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:00:31.918+0000 7f010c800700 1 --2- 172.21.15.18:0/1708113261 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0104071ea0 0x7f0104114480 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f00f400b3b0 tx=0x7f00f4009e30 comp rx=0 tx=0).stop 2024-04-12T21:00:31.918 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:00:31.919+0000 7f010c800700 1 -- 172.21.15.18:0/1708113261 shutdown_connections 2024-04-12T21:00:31.918 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:00:31.919+0000 7f010c800700 1 --2- 172.21.15.18:0/1708113261 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f01040708f0 0x7f0104113f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:00:31.918 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:00:31.919+0000 7f010c800700 1 --2- 172.21.15.18:0/1708113261 >> [v2:172.21.15.18:6800/1605950634,v1:172.21.15.18:6801/1605950634] conn(0x7f00ec06e500 0x7f00ec0709b0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:00:31.918 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:00:31.919+0000 7f010c800700 1 --2- 172.21.15.18:0/1708113261 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0104071ea0 0x7f0104114480 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:00:31.919 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:00:31.919+0000 7f010c800700 1 -- 172.21.15.18:0/1708113261 >> 172.21.15.18:0/1708113261 conn(0x7f01040f8550 msgr2=0x7f01040fa090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:00:31.919 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:00:31.919+0000 7f010c800700 1 -- 172.21.15.18:0/1708113261 shutdown_connections 2024-04-12T21:00:31.919 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:00:31.919+0000 7f010c800700 1 -- 172.21.15.18:0/1708113261 wait complete. 2024-04-12T21:00:32.280 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:00:31 smithi186 ceph-mon[36597]: pgmap v102: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-12T21:00:32.753 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1cbe407a-f90d-11ee-bc8d-c7b262605968 -e sha1=324baf8ddb91dea46cac50cb535c306115791355 -- bash -c 'ceph versions | jq -e '"'"'.mgr | length == 1'"'"'' 2024-04-12T21:00:33.281 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/mon.smithi018/config 2024-04-12T21:00:33.966 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:00:33 smithi018 ceph-mon[28799]: from='client.14826 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:00:33.966 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:00:33 smithi018 ceph-mon[28799]: pgmap v103: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-12T21:00:34.280 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:00:33 smithi186 ceph-mon[36597]: from='client.14826 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:00:34.280 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:00:33 smithi186 ceph-mon[36597]: pgmap v103: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-12T21:00:35.568 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:00:35.569+0000 7f7006400700 1 -- 172.21.15.18:0/1938920993 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f70000708a0 msgr2=0x7f7000070d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:00:35.568 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:00:35.569+0000 7f7006400700 1 --2- 172.21.15.18:0/1938920993 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f70000708a0 0x7f7000070d10 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f6fec004970 tx=0x7f6fec009e30 comp rx=0 tx=0).stop 2024-04-12T21:00:35.568 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:00:35.569+0000 7f7006400700 1 -- 172.21.15.18:0/1938920993 shutdown_connections 2024-04-12T21:00:35.568 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:00:35.569+0000 7f7006400700 1 --2- 172.21.15.18:0/1938920993 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f70000708a0 0x7f7000070d10 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:00:35.568 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:00:35.569+0000 7f7006400700 1 --2- 172.21.15.18:0/1938920993 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7000071f90 0x7f7000070360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:00:35.568 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:00:35.569+0000 7f7006400700 1 -- 172.21.15.18:0/1938920993 >> 172.21.15.18:0/1938920993 conn(0x7f70000f8950 msgr2=0x7f70000fad60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:00:35.568 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:00:35.569+0000 7f7006400700 1 -- 172.21.15.18:0/1938920993 shutdown_connections 2024-04-12T21:00:35.568 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:00:35.569+0000 7f7006400700 1 -- 172.21.15.18:0/1938920993 wait complete. 2024-04-12T21:00:35.568 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:00:35.570+0000 7f7006400700 1 Processor -- start 2024-04-12T21:00:35.568 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:00:35.570+0000 7f7006400700 1 -- start start 2024-04-12T21:00:35.568 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:00:35.570+0000 7f7006400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f70000708a0 0x7f7000119f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:00:35.569 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:00:35.570+0000 7f7006400700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7000071f90 0x7f700011a490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:00:35.569 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:00:35.570+0000 7f7006400700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f700011aae0 con 0x7f70000708a0 2024-04-12T21:00:35.569 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:00:35.570+0000 7f7006400700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70001140c0 con 0x7f7000071f90 2024-04-12T21:00:35.570 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:00:35.571+0000 7f6fff400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f70000708a0 0x7f7000119f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:00:35.570 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:00:35.571+0000 7f6fff400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f70000708a0 0x7f7000119f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:48124/0 (socket says 172.21.15.18:48124) 2024-04-12T21:00:35.570 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:00:35.571+0000 7f6fff400700 1 -- 172.21.15.18:0/247536413 learned_addr learned my addr 172.21.15.18:0/247536413 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:00:35.570 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:00:35.571+0000 7f6fff400700 1 -- 172.21.15.18:0/247536413 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7000071f90 msgr2=0x7f700011a490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:00:35.570 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:00:35.571+0000 7f6fff400700 1 --2- 172.21.15.18:0/247536413 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7000071f90 0x7f700011a490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:00:35.570 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:00:35.571+0000 7f6fff400700 1 -- 172.21.15.18:0/247536413 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6fec005040 con 0x7f70000708a0 2024-04-12T21:00:35.570 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:00:35.571+0000 7f6fff400700 1 --2- 172.21.15.18:0/247536413 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f70000708a0 0x7f7000119f50 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f6ff0002a50 tx=0x7f6ff00046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:00:35.570 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:00:35.572+0000 7f6ff7400700 1 -- 172.21.15.18:0/247536413 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6ff0007cc0 con 0x7f70000708a0 2024-04-12T21:00:35.570 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:00:35.572+0000 7f6ff7400700 1 -- 172.21.15.18:0/247536413 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f6ff000f040 con 0x7f70000708a0 2024-04-12T21:00:35.570 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:00:35.572+0000 7f7006400700 1 -- 172.21.15.18:0/247536413 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f70001143a0 con 0x7f70000708a0 2024-04-12T21:00:35.571 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:00:35.572+0000 7f6ff7400700 1 -- 172.21.15.18:0/247536413 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6ff0009640 con 0x7f70000708a0 2024-04-12T21:00:35.571 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:00:35.572+0000 7f7006400700 1 -- 172.21.15.18:0/247536413 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f70001148e0 con 0x7f70000708a0 2024-04-12T21:00:35.571 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:00:35.573+0000 7f7006400700 1 -- 172.21.15.18:0/247536413 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7000071aa0 con 0x7f70000708a0 2024-04-12T21:00:35.571 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:00:35.574+0000 7f6ff7400700 1 -- 172.21.15.18:0/247536413 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 32) v1 ==== 92459+0+0 (secure 0 0 0) 0x7f6ff00097a0 con 0x7f70000708a0 2024-04-12T21:00:35.573 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:00:35.575+0000 7f6ff7400700 1 --2- 172.21.15.18:0/247536413 >> [v2:172.21.15.18:6800/1605950634,v1:172.21.15.18:6801/1605950634] conn(0x7f6fe406e500 0x7f6fe40709b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:00:35.573 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:00:35.575+0000 7f6ff7400700 1 -- 172.21.15.18:0/247536413 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6066+0+0 (secure 0 0 0) 0x7f6ff008dbf0 con 0x7f70000708a0 2024-04-12T21:00:35.573 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:00:35.575+0000 7f6ffea00700 1 --2- 172.21.15.18:0/247536413 >> [v2:172.21.15.18:6800/1605950634,v1:172.21.15.18:6801/1605950634] conn(0x7f6fe406e500 0x7f6fe40709b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:00:35.573 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:00:35.576+0000 7f6ffea00700 1 --2- 172.21.15.18:0/247536413 >> [v2:172.21.15.18:6800/1605950634,v1:172.21.15.18:6801/1605950634] conn(0x7f6fe406e500 0x7f6fe40709b0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f6fec00b9c0 tx=0x7f6fec009e30 comp rx=0 tx=0).ready entity=mgr.14672 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:00:35.578 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:00:35.580+0000 7f6ff7400700 1 -- 172.21.15.18:0/247536413 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186454 (secure 0 0 0) 0x7f6ff005adf0 con 0x7f70000708a0 2024-04-12T21:00:35.799 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:00:35 smithi018 ceph-mon[28799]: pgmap v104: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-12T21:00:35.898 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:00:35.900+0000 7f7006400700 1 -- 172.21.15.18:0/247536413 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f7000110500 con 0x7f70000708a0 2024-04-12T21:00:35.899 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:00:35.901+0000 7f6ff7400700 1 -- 172.21.15.18:0/247536413 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+739 (secure 0 0 0) 0x7f6ff005a980 con 0x7f70000708a0 2024-04-12T21:00:35.901 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:00:35.904+0000 7f7006400700 1 -- 172.21.15.18:0/247536413 >> [v2:172.21.15.18:6800/1605950634,v1:172.21.15.18:6801/1605950634] conn(0x7f6fe406e500 msgr2=0x7f6fe40709b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:00:35.901 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:00:35.904+0000 7f7006400700 1 --2- 172.21.15.18:0/247536413 >> [v2:172.21.15.18:6800/1605950634,v1:172.21.15.18:6801/1605950634] conn(0x7f6fe406e500 0x7f6fe40709b0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f6fec00b9c0 tx=0x7f6fec009e30 comp rx=0 tx=0).stop 2024-04-12T21:00:35.901 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:00:35.904+0000 7f7006400700 1 -- 172.21.15.18:0/247536413 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f70000708a0 msgr2=0x7f7000119f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:00:35.901 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:00:35.904+0000 7f7006400700 1 --2- 172.21.15.18:0/247536413 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f70000708a0 0x7f7000119f50 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f6ff0002a50 tx=0x7f6ff00046b0 comp rx=0 tx=0).stop 2024-04-12T21:00:35.902 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:00:35.904+0000 7f7006400700 1 -- 172.21.15.18:0/247536413 shutdown_connections 2024-04-12T21:00:35.902 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:00:35.904+0000 7f7006400700 1 --2- 172.21.15.18:0/247536413 >> [v2:172.21.15.18:6800/1605950634,v1:172.21.15.18:6801/1605950634] conn(0x7f6fe406e500 0x7f6fe40709b0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:00:35.902 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:00:35.904+0000 7f7006400700 1 --2- 172.21.15.18:0/247536413 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f70000708a0 0x7f7000119f50 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:00:35.902 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:00:35.904+0000 7f7006400700 1 --2- 172.21.15.18:0/247536413 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7000071f90 0x7f700011a490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:00:35.902 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:00:35.904+0000 7f7006400700 1 -- 172.21.15.18:0/247536413 >> 172.21.15.18:0/247536413 conn(0x7f70000f8950 msgr2=0x7f70000f9a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:00:35.903 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:00:35.904+0000 7f7006400700 1 -- 172.21.15.18:0/247536413 shutdown_connections 2024-04-12T21:00:35.903 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:00:35.904+0000 7f7006400700 1 -- 172.21.15.18:0/247536413 wait complete. 2024-04-12T21:00:35.917 INFO:teuthology.orchestra.run.smithi018.stdout:true 2024-04-12T21:00:36.083 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:00:35 smithi186 ceph-mon[36597]: pgmap v104: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-12T21:00:36.820 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1cbe407a-f90d-11ee-bc8d-c7b262605968 -e sha1=324baf8ddb91dea46cac50cb535c306115791355 -- bash -c 'ceph versions | jq -e '"'"'.mgr | keys'"'"' | grep $sha1' 2024-04-12T21:00:37.101 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:00:36 smithi018 ceph-mon[28799]: from='client.? 172.21.15.18:0/247536413' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T21:00:37.280 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:00:36 smithi186 ceph-mon[36597]: from='client.? 172.21.15.18:0/247536413' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T21:00:37.343 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/mon.smithi018/config 2024-04-12T21:00:38.059 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:00:37 smithi018 ceph-mon[28799]: pgmap v105: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-12T21:00:38.280 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:00:37 smithi186 ceph-mon[36597]: pgmap v105: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-12T21:00:39.866 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:00:39.868+0000 7f8de6800700 1 -- 172.21.15.18:0/1451412831 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8de0102e00 msgr2=0x7f8de01031d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:00:39.866 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:00:39.868+0000 7f8de6800700 1 --2- 172.21.15.18:0/1451412831 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8de0102e00 0x7f8de01031d0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f8dd0004940 tx=0x7f8dd0009e30 comp rx=0 tx=0).stop 2024-04-12T21:00:39.866 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:00:39.868+0000 7f8de6800700 1 -- 172.21.15.18:0/1451412831 shutdown_connections 2024-04-12T21:00:39.866 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:00:39.868+0000 7f8de6800700 1 --2- 172.21.15.18:0/1451412831 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8de01037a0 0x7f8de00fce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:00:39.866 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:00:39.868+0000 7f8de6800700 1 --2- 172.21.15.18:0/1451412831 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8de0102e00 0x7f8de01031d0 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:00:39.866 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:00:39.868+0000 7f8de6800700 1 -- 172.21.15.18:0/1451412831 >> 172.21.15.18:0/1451412831 conn(0x7f8de00f89b0 msgr2=0x7f8de00fadc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:00:39.866 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:00:39.868+0000 7f8de6800700 1 -- 172.21.15.18:0/1451412831 shutdown_connections 2024-04-12T21:00:39.867 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:00:39.868+0000 7f8de6800700 1 -- 172.21.15.18:0/1451412831 wait complete. 2024-04-12T21:00:39.867 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:00:39.869+0000 7f8de6800700 1 Processor -- start 2024-04-12T21:00:39.868 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:00:39.869+0000 7f8de6800700 1 -- start start 2024-04-12T21:00:39.868 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:00:39.870+0000 7f8de6800700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8de0102e00 0x7f8de0113ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:00:39.868 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:00:39.870+0000 7f8de6800700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8de01037a0 0x7f8de01140e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:00:39.868 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:00:39.870+0000 7f8de6800700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8de0114730 con 0x7f8de01037a0 2024-04-12T21:00:39.868 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:00:39.870+0000 7f8de6800700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8de010db70 con 0x7f8de0102e00 2024-04-12T21:00:39.868 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:00:39.870+0000 7f8dd7e00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8de01037a0 0x7f8de01140e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:00:39.868 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:00:39.870+0000 7f8dd7e00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8de01037a0 0x7f8de01140e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:48134/0 (socket says 172.21.15.18:48134) 2024-04-12T21:00:39.868 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:00:39.870+0000 7f8dd7e00700 1 -- 172.21.15.18:0/1109224819 learned_addr learned my addr 172.21.15.18:0/1109224819 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:00:39.869 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:00:39.870+0000 7f8ddf400700 1 --2- 172.21.15.18:0/1109224819 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8de0102e00 0x7f8de0113ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:00:39.869 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:00:39.871+0000 7f8dd7e00700 1 -- 172.21.15.18:0/1109224819 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8de0102e00 msgr2=0x7f8de0113ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:00:39.869 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:00:39.871+0000 7f8dd7e00700 1 --2- 172.21.15.18:0/1109224819 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8de0102e00 0x7f8de0113ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:00:39.869 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:00:39.871+0000 7f8dd7e00700 1 -- 172.21.15.18:0/1109224819 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8dd0005040 con 0x7f8de01037a0 2024-04-12T21:00:39.869 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:00:39.871+0000 7f8dd7e00700 1 --2- 172.21.15.18:0/1109224819 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8de01037a0 0x7f8de01140e0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f8dc800c890 tx=0x7f8dc80044c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:00:39.869 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:00:39.871+0000 7f8dd7400700 1 -- 172.21.15.18:0/1109224819 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8dc800a7d0 con 0x7f8de01037a0 2024-04-12T21:00:39.869 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:00:39.871+0000 7f8de6800700 1 -- 172.21.15.18:0/1109224819 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8de010de50 con 0x7f8de01037a0 2024-04-12T21:00:39.869 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:00:39.872+0000 7f8dd7400700 1 -- 172.21.15.18:0/1109224819 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f8dc800ae00 con 0x7f8de01037a0 2024-04-12T21:00:39.870 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:00:39.872+0000 7f8dd7400700 1 -- 172.21.15.18:0/1109224819 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8dc80113a0 con 0x7f8de01037a0 2024-04-12T21:00:39.870 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:00:39.872+0000 7f8de6800700 1 -- 172.21.15.18:0/1109224819 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8de010e410 con 0x7f8de01037a0 2024-04-12T21:00:39.870 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:00:39.873+0000 7f8de6800700 1 -- 172.21.15.18:0/1109224819 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8de0100110 con 0x7f8de01037a0 2024-04-12T21:00:39.871 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:00:39.874+0000 7f8dd7400700 1 -- 172.21.15.18:0/1109224819 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 32) v1 ==== 92459+0+0 (secure 0 0 0) 0x7f8dc8012400 con 0x7f8de01037a0 2024-04-12T21:00:39.872 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:00:39.875+0000 7f8dd7400700 1 --2- 172.21.15.18:0/1109224819 >> [v2:172.21.15.18:6800/1605950634,v1:172.21.15.18:6801/1605950634] conn(0x7f8dc406e500 0x7f8dc40709b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:00:39.873 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:00:39.875+0000 7f8dd7400700 1 -- 172.21.15.18:0/1109224819 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6066+0+0 (secure 0 0 0) 0x7f8dc808ea30 con 0x7f8de01037a0 2024-04-12T21:00:39.873 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:00:39.875+0000 7f8ddf400700 1 --2- 172.21.15.18:0/1109224819 >> [v2:172.21.15.18:6800/1605950634,v1:172.21.15.18:6801/1605950634] conn(0x7f8dc406e500 0x7f8dc40709b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:00:39.873 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:00:39.876+0000 7f8ddf400700 1 --2- 172.21.15.18:0/1109224819 >> [v2:172.21.15.18:6800/1605950634,v1:172.21.15.18:6801/1605950634] conn(0x7f8dc406e500 0x7f8dc40709b0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f8dd0004f90 tx=0x7f8dd0009e30 comp rx=0 tx=0).ready entity=mgr.14672 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:00:39.877 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:00:39.879+0000 7f8dd7400700 1 -- 172.21.15.18:0/1109224819 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186454 (secure 0 0 0) 0x7f8dc805bcd0 con 0x7f8de01037a0 2024-04-12T21:00:40.101 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:00:39 smithi018 ceph-mon[28799]: pgmap v106: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-12T21:00:40.201 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:00:40.203+0000 7f8de6800700 1 -- 172.21.15.18:0/1109224819 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f8de00fe550 con 0x7f8de01037a0 2024-04-12T21:00:40.202 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:00:40.204+0000 7f8dd7400700 1 -- 172.21.15.18:0/1109224819 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+739 (secure 0 0 0) 0x7f8dc805b860 con 0x7f8de01037a0 2024-04-12T21:00:40.205 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:00:40.207+0000 7f8de6800700 1 -- 172.21.15.18:0/1109224819 >> [v2:172.21.15.18:6800/1605950634,v1:172.21.15.18:6801/1605950634] conn(0x7f8dc406e500 msgr2=0x7f8dc40709b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:00:40.206 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:00:40.207+0000 7f8de6800700 1 --2- 172.21.15.18:0/1109224819 >> [v2:172.21.15.18:6800/1605950634,v1:172.21.15.18:6801/1605950634] conn(0x7f8dc406e500 0x7f8dc40709b0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f8dd0004f90 tx=0x7f8dd0009e30 comp rx=0 tx=0).stop 2024-04-12T21:00:40.206 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:00:40.207+0000 7f8de6800700 1 -- 172.21.15.18:0/1109224819 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8de01037a0 msgr2=0x7f8de01140e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:00:40.206 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:00:40.207+0000 7f8de6800700 1 --2- 172.21.15.18:0/1109224819 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8de01037a0 0x7f8de01140e0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f8dc800c890 tx=0x7f8dc80044c0 comp rx=0 tx=0).stop 2024-04-12T21:00:40.206 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:00:40.208+0000 7f8de6800700 1 -- 172.21.15.18:0/1109224819 shutdown_connections 2024-04-12T21:00:40.206 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:00:40.208+0000 7f8de6800700 1 --2- 172.21.15.18:0/1109224819 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8de0102e00 0x7f8de0113ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:00:40.206 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:00:40.208+0000 7f8de6800700 1 --2- 172.21.15.18:0/1109224819 >> [v2:172.21.15.18:6800/1605950634,v1:172.21.15.18:6801/1605950634] conn(0x7f8dc406e500 0x7f8dc40709b0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:00:40.206 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:00:40.208+0000 7f8de6800700 1 --2- 172.21.15.18:0/1109224819 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8de01037a0 0x7f8de01140e0 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:00:40.206 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:00:40.208+0000 7f8de6800700 1 -- 172.21.15.18:0/1109224819 >> 172.21.15.18:0/1109224819 conn(0x7f8de00f89b0 msgr2=0x7f8de01073a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:00:40.206 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:00:40.208+0000 7f8de6800700 1 -- 172.21.15.18:0/1109224819 shutdown_connections 2024-04-12T21:00:40.206 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:00:40.208+0000 7f8de6800700 1 -- 172.21.15.18:0/1109224819 wait complete. 2024-04-12T21:00:40.221 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 19.0.0-2879-g324baf8d (324baf8ddb91dea46cac50cb535c306115791355) squid (dev)" 2024-04-12T21:00:40.280 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:00:39 smithi186 ceph-mon[36597]: pgmap v106: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-12T21:00:41.080 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:00:40 smithi018 ceph-mon[28799]: from='client.? 172.21.15.18:0/1109224819' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T21:00:41.162 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1cbe407a-f90d-11ee-bc8d-c7b262605968 -e sha1=324baf8ddb91dea46cac50cb535c306115791355 -- bash -c 'ceph versions | jq -e '"'"'.overall | length == 2'"'"'' 2024-04-12T21:00:41.280 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:00:40 smithi186 ceph-mon[36597]: from='client.? 172.21.15.18:0/1109224819' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T21:00:41.686 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/mon.smithi018/config 2024-04-12T21:00:42.090 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:00:41 smithi018 ceph-mon[28799]: pgmap v107: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-12T21:00:42.280 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:00:41 smithi186 ceph-mon[36597]: pgmap v107: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-12T21:00:43.094 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:00:42 smithi018 ceph-mon[28799]: from='mgr.14672 172.21.15.18:0/2209228970' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T21:00:43.280 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:00:42 smithi186 ceph-mon[36597]: from='mgr.14672 172.21.15.18:0/2209228970' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T21:00:44.007 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:00:44.009+0000 7f337ae00700 1 -- 172.21.15.18:0/3246555882 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f33741059c0 msgr2=0x7f33740ff080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:00:44.007 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:00:44.009+0000 7f337ae00700 1 --2- 172.21.15.18:0/3246555882 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f33741059c0 0x7f33740ff080 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f3360004940 tx=0x7f3360009e30 comp rx=0 tx=0).stop 2024-04-12T21:00:44.007 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:00:44.009+0000 7f337ae00700 1 -- 172.21.15.18:0/3246555882 shutdown_connections 2024-04-12T21:00:44.007 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:00:44.009+0000 7f337ae00700 1 --2- 172.21.15.18:0/3246555882 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f33741059c0 0x7f33740ff080 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:00:44.007 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:00:44.009+0000 7f337ae00700 1 --2- 172.21.15.18:0/3246555882 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3374105020 0x7f33741053f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:00:44.007 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:00:44.009+0000 7f337ae00700 1 -- 172.21.15.18:0/3246555882 >> 172.21.15.18:0/3246555882 conn(0x7f33740faba0 msgr2=0x7f33740fcfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:00:44.008 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:00:44.009+0000 7f337ae00700 1 -- 172.21.15.18:0/3246555882 shutdown_connections 2024-04-12T21:00:44.008 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:00:44.009+0000 7f337ae00700 1 -- 172.21.15.18:0/3246555882 wait complete. 2024-04-12T21:00:44.009 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:00:44.010+0000 7f337ae00700 1 Processor -- start 2024-04-12T21:00:44.009 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:00:44.010+0000 7f337ae00700 1 -- start start 2024-04-12T21:00:44.009 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:00:44.010+0000 7f337ae00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3374105020 0x7f3374115ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:00:44.009 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:00:44.010+0000 7f337ae00700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f33741059c0 0x7f33741160e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:00:44.009 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:00:44.010+0000 7f337ae00700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3374116730 con 0x7f3374105020 2024-04-12T21:00:44.009 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:00:44.011+0000 7f337ae00700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f337410fd10 con 0x7f33741059c0 2024-04-12T21:00:44.009 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:00:44.011+0000 7f3373e00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3374105020 0x7f3374115ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:00:44.009 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:00:44.011+0000 7f3373e00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3374105020 0x7f3374115ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:56040/0 (socket says 172.21.15.18:56040) 2024-04-12T21:00:44.009 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:00:44.011+0000 7f3373e00700 1 -- 172.21.15.18:0/3826608120 learned_addr learned my addr 172.21.15.18:0/3826608120 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:00:44.009 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:00:44.011+0000 7f3373e00700 1 -- 172.21.15.18:0/3826608120 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f33741059c0 msgr2=0x7f33741160e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:00:44.010 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:00:44.011+0000 7f3373e00700 1 --2- 172.21.15.18:0/3826608120 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f33741059c0 0x7f33741160e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:00:44.010 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:00:44.011+0000 7f3373e00700 1 -- 172.21.15.18:0/3826608120 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3360005040 con 0x7f3374105020 2024-04-12T21:00:44.010 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:00:44.012+0000 7f3373e00700 1 --2- 172.21.15.18:0/3826608120 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3374105020 0x7f3374115ba0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f3364002a50 tx=0x7f33640046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:00:44.010 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:00:44.012+0000 7f336be00700 1 -- 172.21.15.18:0/3826608120 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3364007cc0 con 0x7f3374105020 2024-04-12T21:00:44.010 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:00:44.012+0000 7f336be00700 1 -- 172.21.15.18:0/3826608120 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f336400f040 con 0x7f3374105020 2024-04-12T21:00:44.011 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:00:44.012+0000 7f337ae00700 1 -- 172.21.15.18:0/3826608120 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f337410fff0 con 0x7f3374105020 2024-04-12T21:00:44.011 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:00:44.012+0000 7f336be00700 1 -- 172.21.15.18:0/3826608120 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3364009640 con 0x7f3374105020 2024-04-12T21:00:44.011 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:00:44.012+0000 7f337ae00700 1 -- 172.21.15.18:0/3826608120 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f33741105b0 con 0x7f3374105020 2024-04-12T21:00:44.011 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:00:44.013+0000 7f337ae00700 1 -- 172.21.15.18:0/3826608120 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3374100770 con 0x7f3374105020 2024-04-12T21:00:44.012 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:00:44.014+0000 7f336be00700 1 -- 172.21.15.18:0/3826608120 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 32) v1 ==== 92459+0+0 (secure 0 0 0) 0x7f33640097a0 con 0x7f3374105020 2024-04-12T21:00:44.013 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:00:44.015+0000 7f336be00700 1 --2- 172.21.15.18:0/3826608120 >> [v2:172.21.15.18:6800/1605950634,v1:172.21.15.18:6801/1605950634] conn(0x7f335806e500 0x7f33580709b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:00:44.013 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:00:44.015+0000 7f336be00700 1 -- 172.21.15.18:0/3826608120 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6066+0+0 (secure 0 0 0) 0x7f336408dd60 con 0x7f3374105020 2024-04-12T21:00:44.013 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:00:44.016+0000 7f3373400700 1 --2- 172.21.15.18:0/3826608120 >> [v2:172.21.15.18:6800/1605950634,v1:172.21.15.18:6801/1605950634] conn(0x7f335806e500 0x7f33580709b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:00:44.014 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:00:44.016+0000 7f3373400700 1 --2- 172.21.15.18:0/3826608120 >> [v2:172.21.15.18:6800/1605950634,v1:172.21.15.18:6801/1605950634] conn(0x7f335806e500 0x7f33580709b0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f3360004f90 tx=0x7f3360009e30 comp rx=0 tx=0).ready entity=mgr.14672 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:00:44.018 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:00:44.021+0000 7f336be00700 1 -- 172.21.15.18:0/3826608120 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186454 (secure 0 0 0) 0x7f3364092070 con 0x7f3374105020 2024-04-12T21:00:44.101 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:00:43 smithi018 ceph-mon[28799]: pgmap v108: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-12T21:00:44.280 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:00:43 smithi186 ceph-mon[36597]: pgmap v108: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-12T21:00:44.346 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:00:44.349+0000 7f337ae00700 1 -- 172.21.15.18:0/3826608120 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f337410c100 con 0x7f3374105020 2024-04-12T21:00:44.347 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:00:44.350+0000 7f336be00700 1 -- 172.21.15.18:0/3826608120 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+739 (secure 0 0 0) 0x7f336405ae40 con 0x7f3374105020 2024-04-12T21:00:44.350 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:00:44.353+0000 7f337ae00700 1 -- 172.21.15.18:0/3826608120 >> [v2:172.21.15.18:6800/1605950634,v1:172.21.15.18:6801/1605950634] conn(0x7f335806e500 msgr2=0x7f33580709b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:00:44.351 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:00:44.353+0000 7f337ae00700 1 --2- 172.21.15.18:0/3826608120 >> [v2:172.21.15.18:6800/1605950634,v1:172.21.15.18:6801/1605950634] conn(0x7f335806e500 0x7f33580709b0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f3360004f90 tx=0x7f3360009e30 comp rx=0 tx=0).stop 2024-04-12T21:00:44.351 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:00:44.353+0000 7f337ae00700 1 -- 172.21.15.18:0/3826608120 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3374105020 msgr2=0x7f3374115ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:00:44.351 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:00:44.353+0000 7f337ae00700 1 --2- 172.21.15.18:0/3826608120 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3374105020 0x7f3374115ba0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f3364002a50 tx=0x7f33640046b0 comp rx=0 tx=0).stop 2024-04-12T21:00:44.352 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:00:44.353+0000 7f337ae00700 1 -- 172.21.15.18:0/3826608120 shutdown_connections 2024-04-12T21:00:44.352 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:00:44.353+0000 7f337ae00700 1 --2- 172.21.15.18:0/3826608120 >> [v2:172.21.15.18:6800/1605950634,v1:172.21.15.18:6801/1605950634] conn(0x7f335806e500 0x7f33580709b0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:00:44.352 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:00:44.353+0000 7f337ae00700 1 --2- 172.21.15.18:0/3826608120 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3374105020 0x7f3374115ba0 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:00:44.352 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:00:44.353+0000 7f337ae00700 1 --2- 172.21.15.18:0/3826608120 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f33741059c0 0x7f33741160e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:00:44.352 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:00:44.353+0000 7f337ae00700 1 -- 172.21.15.18:0/3826608120 >> 172.21.15.18:0/3826608120 conn(0x7f33740faba0 msgr2=0x7f33740fbb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:00:44.352 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:00:44.354+0000 7f337ae00700 1 -- 172.21.15.18:0/3826608120 shutdown_connections 2024-04-12T21:00:44.353 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:00:44.354+0000 7f337ae00700 1 -- 172.21.15.18:0/3826608120 wait complete. 2024-04-12T21:00:44.368 INFO:teuthology.orchestra.run.smithi018.stdout:true 2024-04-12T21:00:45.101 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:00:44 smithi018 ceph-mon[28799]: from='client.? 172.21.15.18:0/3826608120' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T21:00:45.283 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:00:44 smithi186 ceph-mon[36597]: from='client.? 172.21.15.18:0/3826608120' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T21:00:45.303 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1cbe407a-f90d-11ee-bc8d-c7b262605968 -e sha1=324baf8ddb91dea46cac50cb535c306115791355 -- bash -c 'ceph orch upgrade check quay.ceph.io/ceph-ci/ceph:$sha1 | jq -e '"'"'.up_to_date | length == 2'"'"'' 2024-04-12T21:00:45.841 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/mon.smithi018/config 2024-04-12T21:00:46.206 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:00:45 smithi018 ceph-mon[28799]: pgmap v109: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-12T21:00:46.280 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:00:45 smithi186 ceph-mon[36597]: pgmap v109: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-12T21:00:47.280 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:00:46 smithi186 ceph-mon[36597]: pgmap v110: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-12T21:00:47.351 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:00:46 smithi018 ceph-mon[28799]: pgmap v110: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-12T21:00:48.339 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:00:48.340+0000 7efe36c00700 1 -- 172.21.15.18:0/709480489 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7efe30109430 msgr2=0x7efe30109800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:00:48.339 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:00:48.340+0000 7efe36c00700 1 --2- 172.21.15.18:0/709480489 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7efe30109430 0x7efe30109800 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7efe20004940 tx=0x7efe20009e30 comp rx=0 tx=0).stop 2024-04-12T21:00:48.339 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:00:48.340+0000 7efe36c00700 1 -- 172.21.15.18:0/709480489 shutdown_connections 2024-04-12T21:00:48.339 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:00:48.340+0000 7efe36c00700 1 --2- 172.21.15.18:0/709480489 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7efe30109dd0 0x7efe30103440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:00:48.339 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:00:48.340+0000 7efe36c00700 1 --2- 172.21.15.18:0/709480489 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7efe30109430 0x7efe30109800 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:00:48.339 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:00:48.340+0000 7efe36c00700 1 -- 172.21.15.18:0/709480489 >> 172.21.15.18:0/709480489 conn(0x7efe30072c00 msgr2=0x7efe30075010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:00:48.339 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:00:48.340+0000 7efe36c00700 1 -- 172.21.15.18:0/709480489 shutdown_connections 2024-04-12T21:00:48.339 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:00:48.340+0000 7efe36c00700 1 -- 172.21.15.18:0/709480489 wait complete. 2024-04-12T21:00:48.339 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:00:48.341+0000 7efe36c00700 1 Processor -- start 2024-04-12T21:00:48.340 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:00:48.341+0000 7efe36c00700 1 -- start start 2024-04-12T21:00:48.341 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:00:48.342+0000 7efe36c00700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7efe30109430 0x7efe30119ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:00:48.341 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:00:48.342+0000 7efe36c00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7efe30109dd0 0x7efe3011a530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:00:48.341 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:00:48.342+0000 7efe36c00700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe3011ac10 con 0x7efe30109dd0 2024-04-12T21:00:48.341 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:00:48.342+0000 7efe36c00700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe30114160 con 0x7efe30109430 2024-04-12T21:00:48.341 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:00:48.342+0000 7efe2f400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7efe30109dd0 0x7efe3011a530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:00:48.341 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:00:48.342+0000 7efe2f400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7efe30109dd0 0x7efe3011a530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:56058/0 (socket says 172.21.15.18:56058) 2024-04-12T21:00:48.341 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:00:48.342+0000 7efe2f400700 1 -- 172.21.15.18:0/2036464758 learned_addr learned my addr 172.21.15.18:0/2036464758 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:00:48.341 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:00:48.343+0000 7efe2f400700 1 -- 172.21.15.18:0/2036464758 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7efe30109430 msgr2=0x7efe30119ff0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-12T21:00:48.341 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:00:48.343+0000 7efe2f400700 1 --2- 172.21.15.18:0/2036464758 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7efe30109430 0x7efe30119ff0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:00:48.341 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:00:48.343+0000 7efe2f400700 1 -- 172.21.15.18:0/2036464758 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe20005040 con 0x7efe30109dd0 2024-04-12T21:00:48.342 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:00:48.343+0000 7efe2f400700 1 --2- 172.21.15.18:0/2036464758 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7efe30109dd0 0x7efe3011a530 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7efe1c00b760 tx=0x7efe1c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:00:48.342 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:00:48.343+0000 7efe27e00700 1 -- 172.21.15.18:0/2036464758 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7efe1c00f6f0 con 0x7efe30109dd0 2024-04-12T21:00:48.342 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:00:48.343+0000 7efe27e00700 1 -- 172.21.15.18:0/2036464758 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7efe1c00fd20 con 0x7efe30109dd0 2024-04-12T21:00:48.342 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:00:48.343+0000 7efe36c00700 1 -- 172.21.15.18:0/2036464758 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efe301143e0 con 0x7efe30109dd0 2024-04-12T21:00:48.342 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:00:48.343+0000 7efe27e00700 1 -- 172.21.15.18:0/2036464758 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7efe1c011e40 con 0x7efe30109dd0 2024-04-12T21:00:48.342 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:00:48.344+0000 7efe36c00700 1 -- 172.21.15.18:0/2036464758 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efe301149a0 con 0x7efe30109dd0 2024-04-12T21:00:48.342 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:00:48.345+0000 7efe36c00700 1 -- 172.21.15.18:0/2036464758 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efe30104b80 con 0x7efe30109dd0 2024-04-12T21:00:48.344 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:00:48.346+0000 7efe27e00700 1 -- 172.21.15.18:0/2036464758 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 32) v1 ==== 92459+0+0 (secure 0 0 0) 0x7efe1c011670 con 0x7efe30109dd0 2024-04-12T21:00:48.345 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:00:48.347+0000 7efe27e00700 1 --2- 172.21.15.18:0/2036464758 >> [v2:172.21.15.18:6800/1605950634,v1:172.21.15.18:6801/1605950634] conn(0x7efe1406e500 0x7efe140709b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:00:48.345 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:00:48.347+0000 7efe27e00700 1 -- 172.21.15.18:0/2036464758 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6066+0+0 (secure 0 0 0) 0x7efe1c08dba0 con 0x7efe30109dd0 2024-04-12T21:00:48.345 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:00:48.348+0000 7efe2fe00700 1 --2- 172.21.15.18:0/2036464758 >> [v2:172.21.15.18:6800/1605950634,v1:172.21.15.18:6801/1605950634] conn(0x7efe1406e500 0x7efe140709b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:00:48.346 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:00:48.348+0000 7efe2fe00700 1 --2- 172.21.15.18:0/2036464758 >> [v2:172.21.15.18:6800/1605950634,v1:172.21.15.18:6801/1605950634] conn(0x7efe1406e500 0x7efe140709b0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7efe20004f90 tx=0x7efe20009e30 comp rx=0 tx=0).ready entity=mgr.14672 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:00:48.350 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:00:48.353+0000 7efe27e00700 1 -- 172.21.15.18:0/2036464758 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186454 (secure 0 0 0) 0x7efe1c05ae30 con 0x7efe30109dd0 2024-04-12T21:00:48.600 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:00:48.602+0000 7efe36c00700 1 -- 172.21.15.18:0/2036464758 --> [v2:172.21.15.18:6800/1605950634,v1:172.21.15.18:6801/1605950634] -- mgr_command(tid 0: {"prefix": "orch upgrade check", "image": "quay.ceph.io/ceph-ci/ceph:324baf8ddb91dea46cac50cb535c306115791355", "target": ["mon-mgr", ""]}) v1 -- 0x7efe30114c20 con 0x7efe1406e500 2024-04-12T21:00:50.020 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:00:49 smithi018 ceph-mon[28799]: pgmap v111: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-12T21:00:50.020 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:00:49 smithi018 ceph-mon[28799]: from='client.14842 -' entity='client.admin' cmd=[{"prefix": "orch upgrade check", "image": "quay.ceph.io/ceph-ci/ceph:324baf8ddb91dea46cac50cb535c306115791355", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:00:50.029 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:00:49 smithi186 ceph-mon[36597]: pgmap v111: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-12T21:00:50.030 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:00:49 smithi186 ceph-mon[36597]: from='client.14842 -' entity='client.admin' cmd=[{"prefix": "orch upgrade check", "image": "quay.ceph.io/ceph-ci/ceph:324baf8ddb91dea46cac50cb535c306115791355", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:00:51.851 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:00:51 smithi018 ceph-mon[28799]: pgmap v112: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-12T21:00:51.979 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:00:51.981+0000 7efe27e00700 1 -- 172.21.15.18:0/2036464758 <== mgr.14672 v2:172.21.15.18:6800/1605950634 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+5092 (secure 0 0 0) 0x7efe30114c20 con 0x7efe1406e500 2024-04-12T21:00:51.982 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:00:51.984+0000 7efe36c00700 1 -- 172.21.15.18:0/2036464758 >> [v2:172.21.15.18:6800/1605950634,v1:172.21.15.18:6801/1605950634] conn(0x7efe1406e500 msgr2=0x7efe140709b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:00:51.982 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:00:51.984+0000 7efe36c00700 1 --2- 172.21.15.18:0/2036464758 >> [v2:172.21.15.18:6800/1605950634,v1:172.21.15.18:6801/1605950634] conn(0x7efe1406e500 0x7efe140709b0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7efe20004f90 tx=0x7efe20009e30 comp rx=0 tx=0).stop 2024-04-12T21:00:51.982 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:00:51.984+0000 7efe36c00700 1 -- 172.21.15.18:0/2036464758 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7efe30109dd0 msgr2=0x7efe3011a530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:00:51.982 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:00:51.984+0000 7efe36c00700 1 --2- 172.21.15.18:0/2036464758 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7efe30109dd0 0x7efe3011a530 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7efe1c00b760 tx=0x7efe1c0091a0 comp rx=0 tx=0).stop 2024-04-12T21:00:51.983 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:00:51.985+0000 7efe36c00700 1 -- 172.21.15.18:0/2036464758 shutdown_connections 2024-04-12T21:00:51.983 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:00:51.985+0000 7efe36c00700 1 --2- 172.21.15.18:0/2036464758 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7efe30109430 0x7efe30119ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:00:51.983 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:00:51.985+0000 7efe36c00700 1 --2- 172.21.15.18:0/2036464758 >> [v2:172.21.15.18:6800/1605950634,v1:172.21.15.18:6801/1605950634] conn(0x7efe1406e500 0x7efe140709b0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:00:51.983 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:00:51.985+0000 7efe36c00700 1 --2- 172.21.15.18:0/2036464758 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7efe30109dd0 0x7efe3011a530 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:00:51.984 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:00:51.985+0000 7efe36c00700 1 -- 172.21.15.18:0/2036464758 >> 172.21.15.18:0/2036464758 conn(0x7efe30072c00 msgr2=0x7efe30074c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:00:51.984 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:00:51.985+0000 7efe36c00700 1 -- 172.21.15.18:0/2036464758 shutdown_connections 2024-04-12T21:00:51.984 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:00:51.985+0000 7efe36c00700 1 -- 172.21.15.18:0/2036464758 wait complete. 2024-04-12T21:00:52.001 INFO:teuthology.orchestra.run.smithi018.stdout:true 2024-04-12T21:00:52.030 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:00:51 smithi186 ceph-mon[36597]: pgmap v112: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-12T21:00:52.867 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1cbe407a-f90d-11ee-bc8d-c7b262605968 -e sha1=324baf8ddb91dea46cac50cb535c306115791355 -- bash -c 'ceph orch ps' 2024-04-12T21:00:53.368 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/mon.smithi018/config 2024-04-12T21:00:53.776 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:00:53 smithi018 ceph-mon[28799]: pgmap v113: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-12T21:00:54.030 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:00:53 smithi186 ceph-mon[36597]: pgmap v113: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-12T21:00:55.530 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:00:55.531+0000 7f8bfe000700 1 -- 172.21.15.18:0/2954160458 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8bf80fdb40 msgr2=0x7f8bf80fdf10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:00:55.530 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:00:55.531+0000 7f8bfe000700 1 --2- 172.21.15.18:0/2954160458 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8bf80fdb40 0x7f8bf80fdf10 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f8bec004940 tx=0x7f8bec009e30 comp rx=0 tx=0).stop 2024-04-12T21:00:55.530 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:00:55.531+0000 7f8bfe000700 1 -- 172.21.15.18:0/2954160458 shutdown_connections 2024-04-12T21:00:55.530 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:00:55.531+0000 7f8bfe000700 1 --2- 172.21.15.18:0/2954160458 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8bf80fb260 0x7f8bf80fb6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:00:55.530 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:00:55.531+0000 7f8bfe000700 1 --2- 172.21.15.18:0/2954160458 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8bf80fdb40 0x7f8bf80fdf10 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:00:55.531 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:00:55.531+0000 7f8bfe000700 1 -- 172.21.15.18:0/2954160458 >> 172.21.15.18:0/2954160458 conn(0x7f8bf8072150 msgr2=0x7f8bf8072550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:00:55.531 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:00:55.531+0000 7f8bfe000700 1 -- 172.21.15.18:0/2954160458 shutdown_connections 2024-04-12T21:00:55.531 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:00:55.531+0000 7f8bfe000700 1 -- 172.21.15.18:0/2954160458 wait complete. 2024-04-12T21:00:55.531 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:00:55.532+0000 7f8bfe000700 1 Processor -- start 2024-04-12T21:00:55.531 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:00:55.532+0000 7f8bfe000700 1 -- start start 2024-04-12T21:00:55.531 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:00:55.533+0000 7f8bfe000700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8bf80fb260 0x7f8bf8115b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:00:55.531 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:00:55.533+0000 7f8bfe000700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8bf80fdb40 0x7f8bf81160c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:00:55.531 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:00:55.533+0000 7f8bfe000700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8bf8116710 con 0x7f8bf80fb260 2024-04-12T21:00:55.531 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:00:55.533+0000 7f8bfe000700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8bf810fb00 con 0x7f8bf80fdb40 2024-04-12T21:00:55.532 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:00:55.533+0000 7f8bfcc00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8bf80fb260 0x7f8bf8115b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:00:55.533 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:00:55.533+0000 7f8bfcc00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8bf80fb260 0x7f8bf8115b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:47998/0 (socket says 172.21.15.18:47998) 2024-04-12T21:00:55.533 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:00:55.533+0000 7f8bfcc00700 1 -- 172.21.15.18:0/2902448490 learned_addr learned my addr 172.21.15.18:0/2902448490 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:00:55.533 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:00:55.533+0000 7f8bf7e00700 1 --2- 172.21.15.18:0/2902448490 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8bf80fdb40 0x7f8bf81160c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:00:55.533 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:00:55.533+0000 7f8bfcc00700 1 -- 172.21.15.18:0/2902448490 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8bf80fdb40 msgr2=0x7f8bf81160c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:00:55.533 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:00:55.534+0000 7f8bfcc00700 1 --2- 172.21.15.18:0/2902448490 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8bf80fdb40 0x7f8bf81160c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:00:55.533 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:00:55.534+0000 7f8bfcc00700 1 -- 172.21.15.18:0/2902448490 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8bec005040 con 0x7f8bf80fb260 2024-04-12T21:00:55.533 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:00:55.534+0000 7f8bf7e00700 1 --2- 172.21.15.18:0/2902448490 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8bf80fdb40 0x7f8bf81160c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-12T21:00:55.533 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:00:55.534+0000 7f8bfcc00700 1 --2- 172.21.15.18:0/2902448490 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8bf80fb260 0x7f8bf8115b80 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f8bec009180 tx=0x7f8bec009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:00:55.533 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:00:55.534+0000 7f8bf4c00700 1 -- 172.21.15.18:0/2902448490 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8bec00e040 con 0x7f8bf80fb260 2024-04-12T21:00:55.533 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:00:55.534+0000 7f8bf4c00700 1 -- 172.21.15.18:0/2902448490 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f8bec00bbc0 con 0x7f8bf80fb260 2024-04-12T21:00:55.533 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:00:55.534+0000 7f8bf4c00700 1 -- 172.21.15.18:0/2902448490 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8bec01c630 con 0x7f8bf80fb260 2024-04-12T21:00:55.533 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:00:55.534+0000 7f8bfe000700 1 -- 172.21.15.18:0/2902448490 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8bf810fd80 con 0x7f8bf80fb260 2024-04-12T21:00:55.533 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:00:55.534+0000 7f8bfe000700 1 -- 172.21.15.18:0/2902448490 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8bf8110340 con 0x7f8bf80fb260 2024-04-12T21:00:55.534 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:00:55.536+0000 7f8bfe000700 1 -- 172.21.15.18:0/2902448490 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8bf8107110 con 0x7f8bf80fb260 2024-04-12T21:00:55.534 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:00:55.537+0000 7f8bf4c00700 1 -- 172.21.15.18:0/2902448490 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 32) v1 ==== 92459+0+0 (secure 0 0 0) 0x7f8bec00bd30 con 0x7f8bf80fb260 2024-04-12T21:00:55.535 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:00:55.538+0000 7f8bf4c00700 1 --2- 172.21.15.18:0/2902448490 >> [v2:172.21.15.18:6800/1605950634,v1:172.21.15.18:6801/1605950634] conn(0x7f8be006e500 0x7f8be00709b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:00:55.535 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:00:55.538+0000 7f8bf4c00700 1 -- 172.21.15.18:0/2902448490 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6066+0+0 (secure 0 0 0) 0x7f8bec08f2c0 con 0x7f8bf80fb260 2024-04-12T21:00:55.535 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:00:55.538+0000 7f8bf7e00700 1 --2- 172.21.15.18:0/2902448490 >> [v2:172.21.15.18:6800/1605950634,v1:172.21.15.18:6801/1605950634] conn(0x7f8be006e500 0x7f8be00709b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:00:55.536 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:00:55.539+0000 7f8bf7e00700 1 --2- 172.21.15.18:0/2902448490 >> [v2:172.21.15.18:6800/1605950634,v1:172.21.15.18:6801/1605950634] conn(0x7f8be006e500 0x7f8be00709b0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f8be4009fd0 tx=0x7f8be400b040 comp rx=0 tx=0).ready entity=mgr.14672 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:00:55.540 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:00:55.543+0000 7f8bf4c00700 1 -- 172.21.15.18:0/2902448490 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186454 (secure 0 0 0) 0x7f8bec05c4a0 con 0x7f8bf80fb260 2024-04-12T21:00:55.779 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:00:55.781+0000 7f8bfe000700 1 -- 172.21.15.18:0/2902448490 --> [v2:172.21.15.18:6800/1605950634,v1:172.21.15.18:6801/1605950634] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f8bf8111240 con 0x7f8be006e500 2024-04-12T21:00:55.805 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:00:55.808+0000 7f8bf4c00700 1 -- 172.21.15.18:0/2902448490 <== mgr.14672 v2:172.21.15.18:6800/1605950634 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3520 (secure 0 0 0) 0x7f8bf8111240 con 0x7f8be006e500 2024-04-12T21:00:55.806 INFO:teuthology.orchestra.run.smithi018.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-12T21:00:55.806 INFO:teuthology.orchestra.run.smithi018.stdout:alertmanager.smithi018 smithi018 *:9093,9094 running (92s) 48s ago 15m 14.2M - 0.25.0 c8568f914cd2 c332090068e8 2024-04-12T21:00:55.806 INFO:teuthology.orchestra.run.smithi018.stdout:crash.smithi018 smithi018 running (16m) 48s ago 16m 6660k - 17.2.7-827-gd3b5060a 21de9b7369ad 1ac7603bddfd 2024-04-12T21:00:55.806 INFO:teuthology.orchestra.run.smithi018.stdout:crash.smithi186 smithi186 running (14m) 2m ago 14m 6669k - 17.2.7-827-gd3b5060a 21de9b7369ad 14bf8a1672eb 2024-04-12T21:00:55.806 INFO:teuthology.orchestra.run.smithi018.stdout:grafana.smithi018 smithi018 *:3000 running (56s) 48s ago 15m 51.4M - 9.4.12 2bacad6d85d8 b08e3bf176dd 2024-04-12T21:00:55.806 INFO:teuthology.orchestra.run.smithi018.stdout:mds.cephfs.smithi018.acqbsc smithi018 running (8m) 48s ago 8m 112M - 17.2.7-827-gd3b5060a 21de9b7369ad 5b5fce65f18a 2024-04-12T21:00:55.806 INFO:teuthology.orchestra.run.smithi018.stdout:mds.cephfs.smithi018.szjluo smithi018 running (8m) 48s ago 8m 113M - 17.2.7-827-gd3b5060a 21de9b7369ad cf9bf0c5031f 2024-04-12T21:00:55.806 INFO:teuthology.orchestra.run.smithi018.stdout:mds.cephfs.smithi186.jdllyn smithi186 running (8m) 2m ago 8m 13.0M - 17.2.7-827-gd3b5060a 21de9b7369ad 2b2bc46bd29a 2024-04-12T21:00:55.806 INFO:teuthology.orchestra.run.smithi018.stdout:mds.cephfs.smithi186.wjnafs smithi186 running (8m) 2m ago 8m 13.0M - 17.2.7-827-gd3b5060a 21de9b7369ad 74b115d6d6fd 2024-04-12T21:00:55.806 INFO:teuthology.orchestra.run.smithi018.stdout:mgr.smithi018.xjzwga smithi018 *:8443,9283,8765 running (4m) 48s ago 17m 507M - 19.0.0-2879-g324baf8d 8717720dc5e5 56088305eab7 2024-04-12T21:00:55.806 INFO:teuthology.orchestra.run.smithi018.stdout:mgr.smithi186.kqcfxk smithi186 *:8443,9283,8765 running (2m) 2m ago 14m 447M - 19.0.0-2879-g324baf8d 8717720dc5e5 09cd766e5e0d 2024-04-12T21:00:55.806 INFO:teuthology.orchestra.run.smithi018.stdout:mon.smithi018 smithi018 running (17m) 48s ago 17m 57.1M 2048M 17.2.7-827-gd3b5060a 21de9b7369ad cd84caafa056 2024-04-12T21:00:55.807 INFO:teuthology.orchestra.run.smithi018.stdout:mon.smithi186 smithi186 running (14m) 2m ago 14m 41.6M 2048M 17.2.7-827-gd3b5060a 21de9b7369ad d7263c772b73 2024-04-12T21:00:55.807 INFO:teuthology.orchestra.run.smithi018.stdout:node-exporter.smithi018 smithi018 *:9100 running (2m) 48s ago 16m 7675k - 1.5.0 0da6a335fe13 326c3d9c6847 2024-04-12T21:00:55.807 INFO:teuthology.orchestra.run.smithi018.stdout:node-exporter.smithi186 smithi186 *:9100 running (2m) 2m ago 14m 2705k - 1.5.0 0da6a335fe13 ec2112baeecf 2024-04-12T21:00:55.807 INFO:teuthology.orchestra.run.smithi018.stdout:osd.0 smithi018 running (13m) 48s ago 13m 225M 1559M 17.2.7-827-gd3b5060a 21de9b7369ad 0d83e027bf0a 2024-04-12T21:00:55.807 INFO:teuthology.orchestra.run.smithi018.stdout:osd.1 smithi018 running (13m) 48s ago 13m 239M 1559M 17.2.7-827-gd3b5060a 21de9b7369ad 6517c9cc7f87 2024-04-12T21:00:55.807 INFO:teuthology.orchestra.run.smithi018.stdout:osd.2 smithi018 running (12m) 48s ago 12m 193M 1559M 17.2.7-827-gd3b5060a 21de9b7369ad 8a83692acaa5 2024-04-12T21:00:55.807 INFO:teuthology.orchestra.run.smithi018.stdout:osd.3 smithi186 running (11m) 2m ago 11m 258M 2581M 17.2.7-827-gd3b5060a 21de9b7369ad c2def7a90d76 2024-04-12T21:00:55.807 INFO:teuthology.orchestra.run.smithi018.stdout:osd.4 smithi186 running (11m) 2m ago 11m 228M 2581M 17.2.7-827-gd3b5060a 21de9b7369ad 757e7c71b7b9 2024-04-12T21:00:55.807 INFO:teuthology.orchestra.run.smithi018.stdout:osd.5 smithi186 running (10m) 2m ago 10m 222M 2581M 17.2.7-827-gd3b5060a 21de9b7369ad 2ee072a436d0 2024-04-12T21:00:55.807 INFO:teuthology.orchestra.run.smithi018.stdout:prometheus.smithi018 smithi018 *:9095 running (119s) 48s ago 15m 49.5M - 2.43.0 a07b618ecd1d f58841127e21 2024-04-12T21:00:55.808 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:00:55.811+0000 7f8bfe000700 1 -- 172.21.15.18:0/2902448490 >> [v2:172.21.15.18:6800/1605950634,v1:172.21.15.18:6801/1605950634] conn(0x7f8be006e500 msgr2=0x7f8be00709b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:00:55.808 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:00:55.811+0000 7f8bfe000700 1 --2- 172.21.15.18:0/2902448490 >> [v2:172.21.15.18:6800/1605950634,v1:172.21.15.18:6801/1605950634] conn(0x7f8be006e500 0x7f8be00709b0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f8be4009fd0 tx=0x7f8be400b040 comp rx=0 tx=0).stop 2024-04-12T21:00:55.808 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:00:55.811+0000 7f8bfe000700 1 -- 172.21.15.18:0/2902448490 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8bf80fb260 msgr2=0x7f8bf8115b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:00:55.808 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:00:55.811+0000 7f8bfe000700 1 --2- 172.21.15.18:0/2902448490 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8bf80fb260 0x7f8bf8115b80 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f8bec009180 tx=0x7f8bec009e30 comp rx=0 tx=0).stop 2024-04-12T21:00:55.809 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:00:55.811+0000 7f8bfe000700 1 -- 172.21.15.18:0/2902448490 shutdown_connections 2024-04-12T21:00:55.809 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:00:55.811+0000 7f8bfe000700 1 --2- 172.21.15.18:0/2902448490 >> [v2:172.21.15.18:6800/1605950634,v1:172.21.15.18:6801/1605950634] conn(0x7f8be006e500 0x7f8be00709b0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:00:55.809 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:00:55.811+0000 7f8bfe000700 1 --2- 172.21.15.18:0/2902448490 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8bf80fb260 0x7f8bf8115b80 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:00:55.809 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:00:55.811+0000 7f8bfe000700 1 --2- 172.21.15.18:0/2902448490 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8bf80fdb40 0x7f8bf81160c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:00:55.809 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:00:55.811+0000 7f8bfe000700 1 -- 172.21.15.18:0/2902448490 >> 172.21.15.18:0/2902448490 conn(0x7f8bf8072150 msgr2=0x7f8bf80f9510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:00:55.809 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:00:55.811+0000 7f8bfe000700 1 -- 172.21.15.18:0/2902448490 shutdown_connections 2024-04-12T21:00:55.809 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:00:55.811+0000 7f8bfe000700 1 -- 172.21.15.18:0/2902448490 wait complete. 2024-04-12T21:00:56.030 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:00:55 smithi186 ceph-mon[36597]: pgmap v114: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-12T21:00:56.101 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:00:55 smithi018 ceph-mon[28799]: pgmap v114: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-12T21:00:56.740 INFO:teuthology.task.sequential:In sequential, running task cephadm.shell... 2024-04-12T21:00:56.741 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi018.front.sepia.ceph.com 2024-04-12T21:00:56.741 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1cbe407a-f90d-11ee-bc8d-c7b262605968 -e sha1=324baf8ddb91dea46cac50cb535c306115791355 -- bash -c 'ceph config set mgr mgr/orchestrator/fail_fs true' 2024-04-12T21:00:57.030 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:00:56 smithi186 ceph-mon[36597]: from='client.14846 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:00:57.101 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:00:56 smithi018 ceph-mon[28799]: from='client.14846 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:00:57.254 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/mon.smithi018/config 2024-04-12T21:00:58.030 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:00:57 smithi186 ceph-mon[36597]: pgmap v115: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-12T21:00:58.030 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:00:57 smithi186 ceph-mon[36597]: from='mgr.14672 172.21.15.18:0/2209228970' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T21:00:58.101 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:00:57 smithi018 ceph-mon[28799]: pgmap v115: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-12T21:00:58.101 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:00:57 smithi018 ceph-mon[28799]: from='mgr.14672 172.21.15.18:0/2209228970' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T21:00:59.030 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:00:58 smithi186 ceph-mon[36597]: fsmap cephfs:2 {0=cephfs.smithi018.acqbsc=up:active,1=cephfs.smithi018.szjluo=up:active} 2 up:standby 2024-04-12T21:00:59.101 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:00:58 smithi018 ceph-mon[28799]: fsmap cephfs:2 {0=cephfs.smithi018.acqbsc=up:active,1=cephfs.smithi018.szjluo=up:active} 2 up:standby 2024-04-12T21:00:59.706 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:00:59.708+0000 7f749e400700 1 -- 172.21.15.18:0/1281947504 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f74980708a0 msgr2=0x7f7498070d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:00:59.706 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:00:59.708+0000 7f749e400700 1 --2- 172.21.15.18:0/1281947504 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f74980708a0 0x7f7498070d10 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f7484004970 tx=0x7f7484009e30 comp rx=0 tx=0).stop 2024-04-12T21:00:59.706 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:00:59.708+0000 7f749e400700 1 -- 172.21.15.18:0/1281947504 shutdown_connections 2024-04-12T21:00:59.706 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:00:59.708+0000 7f749e400700 1 --2- 172.21.15.18:0/1281947504 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f74980708a0 0x7f7498070d10 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:00:59.706 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:00:59.708+0000 7f749e400700 1 --2- 172.21.15.18:0/1281947504 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7498071f90 0x7f7498070360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:00:59.706 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:00:59.708+0000 7f749e400700 1 -- 172.21.15.18:0/1281947504 >> 172.21.15.18:0/1281947504 conn(0x7f74980f89b0 msgr2=0x7f74980fadc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:00:59.706 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:00:59.708+0000 7f749e400700 1 -- 172.21.15.18:0/1281947504 shutdown_connections 2024-04-12T21:00:59.706 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:00:59.708+0000 7f749e400700 1 -- 172.21.15.18:0/1281947504 wait complete. 2024-04-12T21:00:59.707 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:00:59.709+0000 7f749e400700 1 Processor -- start 2024-04-12T21:00:59.707 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:00:59.710+0000 7f749e400700 1 -- start start 2024-04-12T21:00:59.708 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:00:59.710+0000 7f749e400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f74980708a0 0x7f749811a170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:00:59.708 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:00:59.710+0000 7f749e400700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7498071f90 0x7f749811a6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:00:59.708 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:00:59.710+0000 7f749e400700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f749811ad00 con 0x7f74980708a0 2024-04-12T21:00:59.708 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:00:59.710+0000 7f749e400700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74981140d0 con 0x7f7498071f90 2024-04-12T21:00:59.708 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:00:59.710+0000 7f7497400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f74980708a0 0x7f749811a170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:00:59.708 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:00:59.710+0000 7f7497400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f74980708a0 0x7f749811a170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:48008/0 (socket says 172.21.15.18:48008) 2024-04-12T21:00:59.708 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:00:59.710+0000 7f7497400700 1 -- 172.21.15.18:0/2664995908 learned_addr learned my addr 172.21.15.18:0/2664995908 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:00:59.709 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:00:59.711+0000 7f7496a00700 1 --2- 172.21.15.18:0/2664995908 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7498071f90 0x7f749811a6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:00:59.709 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:00:59.711+0000 7f7497400700 1 -- 172.21.15.18:0/2664995908 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7498071f90 msgr2=0x7f749811a6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:00:59.709 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:00:59.711+0000 7f7497400700 1 --2- 172.21.15.18:0/2664995908 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7498071f90 0x7f749811a6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:00:59.709 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:00:59.711+0000 7f7497400700 1 -- 172.21.15.18:0/2664995908 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7484005040 con 0x7f74980708a0 2024-04-12T21:00:59.709 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:00:59.711+0000 7f7496a00700 1 --2- 172.21.15.18:0/2664995908 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7498071f90 0x7f749811a6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-12T21:00:59.710 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:00:59.711+0000 7f7497400700 1 --2- 172.21.15.18:0/2664995908 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f74980708a0 0x7f749811a170 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f7488002a50 tx=0x7f74880046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:00:59.710 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:00:59.712+0000 7f748f400700 1 -- 172.21.15.18:0/2664995908 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7488007cc0 con 0x7f74980708a0 2024-04-12T21:00:59.710 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:00:59.712+0000 7f748f400700 1 -- 172.21.15.18:0/2664995908 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f748800f040 con 0x7f74980708a0 2024-04-12T21:00:59.710 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:00:59.712+0000 7f748f400700 1 -- 172.21.15.18:0/2664995908 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7488009640 con 0x7f74980708a0 2024-04-12T21:00:59.710 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:00:59.712+0000 7f749e400700 1 -- 172.21.15.18:0/2664995908 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f74981143b0 con 0x7f74980708a0 2024-04-12T21:00:59.710 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:00:59.712+0000 7f749e400700 1 -- 172.21.15.18:0/2664995908 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f74981148f0 con 0x7f74980708a0 2024-04-12T21:00:59.710 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:00:59.713+0000 7f749e400700 1 -- 172.21.15.18:0/2664995908 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7498071aa0 con 0x7f74980708a0 2024-04-12T21:00:59.713 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:00:59.714+0000 7f748f400700 1 -- 172.21.15.18:0/2664995908 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 32) v1 ==== 92459+0+0 (secure 0 0 0) 0x7f74880097a0 con 0x7f74980708a0 2024-04-12T21:00:59.714 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:00:59.715+0000 7f748f400700 1 --2- 172.21.15.18:0/2664995908 >> [v2:172.21.15.18:6800/1605950634,v1:172.21.15.18:6801/1605950634] conn(0x7f747c06e510 0x7f747c0709c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:00:59.714 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:00:59.715+0000 7f748f400700 1 -- 172.21.15.18:0/2664995908 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6066+0+0 (secure 0 0 0) 0x7f748808dc90 con 0x7f74980708a0 2024-04-12T21:00:59.714 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:00:59.716+0000 7f7496a00700 1 --2- 172.21.15.18:0/2664995908 >> [v2:172.21.15.18:6800/1605950634,v1:172.21.15.18:6801/1605950634] conn(0x7f747c06e510 0x7f747c0709c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:00:59.714 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:00:59.716+0000 7f7496a00700 1 --2- 172.21.15.18:0/2664995908 >> [v2:172.21.15.18:6800/1605950634,v1:172.21.15.18:6801/1605950634] conn(0x7f747c06e510 0x7f747c0709c0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f748400b9c0 tx=0x7f7484009e30 comp rx=0 tx=0).ready entity=mgr.14672 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:00:59.718 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:00:59.720+0000 7f748f400700 1 -- 172.21.15.18:0/2664995908 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186454 (secure 0 0 0) 0x7f748805adf0 con 0x7f74980708a0 2024-04-12T21:00:59.944 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:00:59.947+0000 7f749e400700 1 -- 172.21.15.18:0/2664995908 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command([{prefix=config set, name=mgr/orchestrator/fail_fs}] v 0) v1 -- 0x7f749805f5f0 con 0x7f74980708a0 2024-04-12T21:00:59.955 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:00:59.957+0000 7f748f400700 1 -- 172.21.15.18:0/2664995908 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/orchestrator/fail_fs}]=0 v46)=0 v46) v1 ==== 125+0+0 (secure 0 0 0) 0x7f748805a980 con 0x7f74980708a0 2024-04-12T21:00:59.955 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:00:59 smithi018 ceph-mon[28799]: pgmap v116: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-12T21:00:59.960 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:00:59.962+0000 7f749e400700 1 -- 172.21.15.18:0/2664995908 >> [v2:172.21.15.18:6800/1605950634,v1:172.21.15.18:6801/1605950634] conn(0x7f747c06e510 msgr2=0x7f747c0709c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:00:59.960 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:00:59.962+0000 7f749e400700 1 --2- 172.21.15.18:0/2664995908 >> [v2:172.21.15.18:6800/1605950634,v1:172.21.15.18:6801/1605950634] conn(0x7f747c06e510 0x7f747c0709c0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f748400b9c0 tx=0x7f7484009e30 comp rx=0 tx=0).stop 2024-04-12T21:00:59.960 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:00:59.962+0000 7f749e400700 1 -- 172.21.15.18:0/2664995908 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f74980708a0 msgr2=0x7f749811a170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:00:59.961 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:00:59.962+0000 7f749e400700 1 --2- 172.21.15.18:0/2664995908 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f74980708a0 0x7f749811a170 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f7488002a50 tx=0x7f74880046b0 comp rx=0 tx=0).stop 2024-04-12T21:00:59.961 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:00:59.962+0000 7f749e400700 1 -- 172.21.15.18:0/2664995908 shutdown_connections 2024-04-12T21:00:59.961 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:00:59.962+0000 7f749e400700 1 --2- 172.21.15.18:0/2664995908 >> [v2:172.21.15.18:6800/1605950634,v1:172.21.15.18:6801/1605950634] conn(0x7f747c06e510 0x7f747c0709c0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:00:59.961 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:00:59.962+0000 7f749e400700 1 --2- 172.21.15.18:0/2664995908 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f74980708a0 0x7f749811a170 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:00:59.961 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:00:59.962+0000 7f749e400700 1 --2- 172.21.15.18:0/2664995908 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7498071f90 0x7f749811a6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:00:59.961 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:00:59.962+0000 7f749e400700 1 -- 172.21.15.18:0/2664995908 >> 172.21.15.18:0/2664995908 conn(0x7f74980f89b0 msgr2=0x7f74980f9a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:00:59.961 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:00:59.963+0000 7f749e400700 1 -- 172.21.15.18:0/2664995908 shutdown_connections 2024-04-12T21:00:59.961 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:00:59.963+0000 7f749e400700 1 -- 172.21.15.18:0/2664995908 wait complete. 2024-04-12T21:01:00.030 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:00:59 smithi186 ceph-mon[36597]: pgmap v116: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-12T21:01:00.831 INFO:teuthology.task.sequential:In sequential, running task cephadm.shell... 2024-04-12T21:01:00.832 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi018.front.sepia.ceph.com 2024-04-12T21:01:00.832 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1cbe407a-f90d-11ee-bc8d-c7b262605968 -e sha1=324baf8ddb91dea46cac50cb535c306115791355 -- bash -c 'ceph config set mon mon_warn_on_insecure_global_id_reclaim false --force' 2024-04-12T21:01:01.102 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:01:00 smithi018 ceph-mon[28799]: from='client.? 172.21.15.18:0/2664995908' entity='client.admin' 2024-04-12T21:01:01.102 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:01:00 smithi018 ceph-mon[28799]: from='mgr.14672 172.21.15.18:0/2209228970' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-12T21:01:01.102 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:01:00 smithi018 ceph-mon[28799]: from='mgr.14672 172.21.15.18:0/2209228970' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-12T21:01:01.102 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:01:00 smithi018 ceph-mon[28799]: from='mgr.14672 172.21.15.18:0/2209228970' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-12T21:01:01.102 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:01:00 smithi018 ceph-mon[28799]: from='mgr.14672 172.21.15.18:0/2209228970' entity='mgr.smithi018.xjzwga' 2024-04-12T21:01:01.102 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:01:00 smithi018 ceph-mon[28799]: pgmap v117: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-12T21:01:01.280 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:01:00 smithi186 ceph-mon[36597]: from='client.? 172.21.15.18:0/2664995908' entity='client.admin' 2024-04-12T21:01:01.280 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:01:00 smithi186 ceph-mon[36597]: from='mgr.14672 172.21.15.18:0/2209228970' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-12T21:01:01.280 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:01:00 smithi186 ceph-mon[36597]: from='mgr.14672 172.21.15.18:0/2209228970' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-12T21:01:01.280 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:01:00 smithi186 ceph-mon[36597]: from='mgr.14672 172.21.15.18:0/2209228970' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-12T21:01:01.280 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:01:00 smithi186 ceph-mon[36597]: from='mgr.14672 172.21.15.18:0/2209228970' entity='mgr.smithi018.xjzwga' 2024-04-12T21:01:01.281 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:01:00 smithi186 ceph-mon[36597]: pgmap v117: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-12T21:01:01.358 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/mon.smithi018/config 2024-04-12T21:01:03.777 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:03.778+0000 7fef1dc00700 1 -- 172.21.15.18:0/160403507 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fef18070300 msgr2=0x7fef180706d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:01:03.777 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:03.778+0000 7fef1dc00700 1 --2- 172.21.15.18:0/160403507 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fef18070300 0x7fef180706d0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7fef08004970 tx=0x7fef08009e30 comp rx=0 tx=0).stop 2024-04-12T21:01:03.777 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:01:03 smithi018 ceph-mon[28799]: pgmap v118: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-12T21:01:03.779 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:03.778+0000 7fef1dc00700 1 -- 172.21.15.18:0/160403507 shutdown_connections 2024-04-12T21:01:03.779 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:03.778+0000 7fef1dc00700 1 --2- 172.21.15.18:0/160403507 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fef18070c10 0x7fef18109170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:01:03.779 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:03.778+0000 7fef1dc00700 1 --2- 172.21.15.18:0/160403507 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fef18070300 0x7fef180706d0 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:01:03.779 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:03.778+0000 7fef1dc00700 1 -- 172.21.15.18:0/160403507 >> 172.21.15.18:0/160403507 conn(0x7fef180f89b0 msgr2=0x7fef180fadc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:01:03.779 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:03.779+0000 7fef1dc00700 1 -- 172.21.15.18:0/160403507 shutdown_connections 2024-04-12T21:01:03.779 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:03.779+0000 7fef1dc00700 1 -- 172.21.15.18:0/160403507 wait complete. 2024-04-12T21:01:03.779 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:03.779+0000 7fef1dc00700 1 Processor -- start 2024-04-12T21:01:03.780 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:03.780+0000 7fef1dc00700 1 -- start start 2024-04-12T21:01:03.780 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:03.780+0000 7fef1dc00700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fef18070300 0x7fef18115b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:01:03.780 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:03.780+0000 7fef1dc00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fef18070c10 0x7fef18116070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:01:03.780 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:03.780+0000 7fef1dc00700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef181166c0 con 0x7fef18070c10 2024-04-12T21:01:03.780 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:03.780+0000 7fef1dc00700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef1810fca0 con 0x7fef18070300 2024-04-12T21:01:03.780 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:03.781+0000 7fef16000700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fef18070c10 0x7fef18116070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:01:03.780 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:03.781+0000 7fef16000700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fef18070c10 0x7fef18116070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:44446/0 (socket says 172.21.15.18:44446) 2024-04-12T21:01:03.780 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:03.781+0000 7fef16000700 1 -- 172.21.15.18:0/3790478461 learned_addr learned my addr 172.21.15.18:0/3790478461 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:01:03.780 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:03.781+0000 7fef16a00700 1 --2- 172.21.15.18:0/3790478461 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fef18070300 0x7fef18115b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:01:03.780 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:03.781+0000 7fef16000700 1 -- 172.21.15.18:0/3790478461 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fef18070300 msgr2=0x7fef18115b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:01:03.780 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:03.781+0000 7fef16000700 1 --2- 172.21.15.18:0/3790478461 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fef18070300 0x7fef18115b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:01:03.780 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:03.781+0000 7fef16000700 1 -- 172.21.15.18:0/3790478461 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef08005040 con 0x7fef18070c10 2024-04-12T21:01:03.780 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:03.782+0000 7fef16000700 1 --2- 172.21.15.18:0/3790478461 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fef18070c10 0x7fef18116070 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7fef04002a50 tx=0x7fef040046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:01:03.781 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:03.782+0000 7fef0ea00700 1 -- 172.21.15.18:0/3790478461 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fef04007cc0 con 0x7fef18070c10 2024-04-12T21:01:03.781 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:03.782+0000 7fef0ea00700 1 -- 172.21.15.18:0/3790478461 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fef0400f040 con 0x7fef18070c10 2024-04-12T21:01:03.781 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:03.782+0000 7fef0ea00700 1 -- 172.21.15.18:0/3790478461 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fef04009640 con 0x7fef18070c10 2024-04-12T21:01:03.781 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:03.782+0000 7fef1dc00700 1 -- 172.21.15.18:0/3790478461 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fef1810ff80 con 0x7fef18070c10 2024-04-12T21:01:03.781 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:03.782+0000 7fef1dc00700 1 -- 172.21.15.18:0/3790478461 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fef181104c0 con 0x7fef18070c10 2024-04-12T21:01:03.781 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:03.783+0000 7fef1dc00700 1 -- 172.21.15.18:0/3790478461 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fef18071a40 con 0x7fef18070c10 2024-04-12T21:01:03.782 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:03.785+0000 7fef0ea00700 1 -- 172.21.15.18:0/3790478461 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 32) v1 ==== 92459+0+0 (secure 0 0 0) 0x7fef040097c0 con 0x7fef18070c10 2024-04-12T21:01:03.783 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:03.786+0000 7fef0ea00700 1 --2- 172.21.15.18:0/3790478461 >> [v2:172.21.15.18:6800/1605950634,v1:172.21.15.18:6801/1605950634] conn(0x7feefc06e500 0x7feefc0709b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:01:03.783 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:03.786+0000 7fef0ea00700 1 -- 172.21.15.18:0/3790478461 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6066+0+0 (secure 0 0 0) 0x7fef0408dbe0 con 0x7fef18070c10 2024-04-12T21:01:03.784 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:03.786+0000 7fef16a00700 1 --2- 172.21.15.18:0/3790478461 >> [v2:172.21.15.18:6800/1605950634,v1:172.21.15.18:6801/1605950634] conn(0x7feefc06e500 0x7feefc0709b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:01:03.784 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:03.787+0000 7fef16a00700 1 --2- 172.21.15.18:0/3790478461 >> [v2:172.21.15.18:6800/1605950634,v1:172.21.15.18:6801/1605950634] conn(0x7feefc06e500 0x7feefc0709b0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fef08004ee0 tx=0x7fef08009e30 comp rx=0 tx=0).ready entity=mgr.14672 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:01:03.789 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:03.791+0000 7fef0ea00700 1 -- 172.21.15.18:0/3790478461 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186454 (secure 0 0 0) 0x7fef0405ade0 con 0x7fef18070c10 2024-04-12T21:01:04.019 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:04.021+0000 7fef1dc00700 1 -- 172.21.15.18:0/3790478461 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command([{prefix=config set, name=mon_warn_on_insecure_global_id_reclaim}] v 0) v1 -- 0x7fef1804c710 con 0x7fef18070c10 2024-04-12T21:01:04.020 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:04.022+0000 7fef0ea00700 1 -- 172.21.15.18:0/3790478461 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_warn_on_insecure_global_id_reclaim}]=0 v46)=0 v46) v1 ==== 155+0+0 (secure 0 0 0) 0x7fef0405a970 con 0x7fef18070c10 2024-04-12T21:01:04.023 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:04.025+0000 7fef1dc00700 1 -- 172.21.15.18:0/3790478461 >> [v2:172.21.15.18:6800/1605950634,v1:172.21.15.18:6801/1605950634] conn(0x7feefc06e500 msgr2=0x7feefc0709b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:01:04.023 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:04.025+0000 7fef1dc00700 1 --2- 172.21.15.18:0/3790478461 >> [v2:172.21.15.18:6800/1605950634,v1:172.21.15.18:6801/1605950634] conn(0x7feefc06e500 0x7feefc0709b0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fef08004ee0 tx=0x7fef08009e30 comp rx=0 tx=0).stop 2024-04-12T21:01:04.023 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:04.025+0000 7fef1dc00700 1 -- 172.21.15.18:0/3790478461 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fef18070c10 msgr2=0x7fef18116070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:01:04.023 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:04.025+0000 7fef1dc00700 1 --2- 172.21.15.18:0/3790478461 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fef18070c10 0x7fef18116070 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7fef04002a50 tx=0x7fef040046b0 comp rx=0 tx=0).stop 2024-04-12T21:01:04.023 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:04.025+0000 7fef1dc00700 1 -- 172.21.15.18:0/3790478461 shutdown_connections 2024-04-12T21:01:04.023 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:04.025+0000 7fef1dc00700 1 --2- 172.21.15.18:0/3790478461 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fef18070300 0x7fef18115b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:01:04.023 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:04.025+0000 7fef1dc00700 1 --2- 172.21.15.18:0/3790478461 >> [v2:172.21.15.18:6800/1605950634,v1:172.21.15.18:6801/1605950634] conn(0x7feefc06e500 0x7feefc0709b0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:01:04.024 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:04.025+0000 7fef1dc00700 1 --2- 172.21.15.18:0/3790478461 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fef18070c10 0x7fef18116070 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:01:04.024 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:04.025+0000 7fef1dc00700 1 -- 172.21.15.18:0/3790478461 >> 172.21.15.18:0/3790478461 conn(0x7fef180f89b0 msgr2=0x7fef180f9e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:01:04.024 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:04.025+0000 7fef1dc00700 1 -- 172.21.15.18:0/3790478461 shutdown_connections 2024-04-12T21:01:04.024 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:04.025+0000 7fef1dc00700 1 -- 172.21.15.18:0/3790478461 wait complete. 2024-04-12T21:01:04.030 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:01:03 smithi186 ceph-mon[36597]: pgmap v118: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-12T21:01:04.884 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1cbe407a-f90d-11ee-bc8d-c7b262605968 -e sha1=324baf8ddb91dea46cac50cb535c306115791355 -- bash -c 'ceph config set mon mon_warn_on_insecure_global_id_reclaim_allowed false --force' 2024-04-12T21:01:05.394 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/mon.smithi018/config 2024-04-12T21:01:05.910 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:01:05 smithi018 ceph-mon[28799]: pgmap v119: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-12T21:01:06.030 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:01:05 smithi186 ceph-mon[36597]: pgmap v119: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-12T21:01:07.864 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:07.866+0000 7f12ac400700 1 -- 172.21.15.18:0/1238334213 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f12a40708a0 msgr2=0x7f12a4070d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:01:07.864 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:07.866+0000 7f12ac400700 1 --2- 172.21.15.18:0/1238334213 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f12a40708a0 0x7f12a4070d10 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f1290004970 tx=0x7f1290009e30 comp rx=0 tx=0).stop 2024-04-12T21:01:07.864 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:07.866+0000 7f12ac400700 1 -- 172.21.15.18:0/1238334213 shutdown_connections 2024-04-12T21:01:07.864 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:07.866+0000 7f12ac400700 1 --2- 172.21.15.18:0/1238334213 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f12a40708a0 0x7f12a4070d10 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:01:07.865 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:07.866+0000 7f12ac400700 1 --2- 172.21.15.18:0/1238334213 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f12a4071f90 0x7f12a4070360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:01:07.865 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:07.866+0000 7f12ac400700 1 -- 172.21.15.18:0/1238334213 >> 172.21.15.18:0/1238334213 conn(0x7f12a40f89b0 msgr2=0x7f12a40fadc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:01:07.865 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:07.866+0000 7f12ac400700 1 -- 172.21.15.18:0/1238334213 shutdown_connections 2024-04-12T21:01:07.865 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:07.866+0000 7f12ac400700 1 -- 172.21.15.18:0/1238334213 wait complete. 2024-04-12T21:01:07.866 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:07.867+0000 7f12ac400700 1 Processor -- start 2024-04-12T21:01:07.866 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:07.867+0000 7f12ac400700 1 -- start start 2024-04-12T21:01:07.866 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:07.868+0000 7f12ac400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f12a40708a0 0x7f12a4119f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:01:07.866 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:07.868+0000 7f12ac400700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f12a4071f90 0x7f12a411a490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:01:07.866 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:07.868+0000 7f12ac400700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12a411aae0 con 0x7f12a40708a0 2024-04-12T21:01:07.866 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:07.868+0000 7f12ac400700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12a41140c0 con 0x7f12a4071f90 2024-04-12T21:01:07.867 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:07.868+0000 7f12a9a00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f12a40708a0 0x7f12a4119f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:01:07.867 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:07.868+0000 7f12a9a00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f12a40708a0 0x7f12a4119f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:44470/0 (socket says 172.21.15.18:44470) 2024-04-12T21:01:07.867 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:07.868+0000 7f12a9a00700 1 -- 172.21.15.18:0/3146849465 learned_addr learned my addr 172.21.15.18:0/3146849465 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:01:07.867 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:07.869+0000 7f12a9000700 1 --2- 172.21.15.18:0/3146849465 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f12a4071f90 0x7f12a411a490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:01:07.867 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:07.869+0000 7f12a9a00700 1 -- 172.21.15.18:0/3146849465 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f12a4071f90 msgr2=0x7f12a411a490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:01:07.867 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:07.869+0000 7f12a9a00700 1 --2- 172.21.15.18:0/3146849465 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f12a4071f90 0x7f12a411a490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:01:07.867 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:07.869+0000 7f12a9a00700 1 -- 172.21.15.18:0/3146849465 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1290005040 con 0x7f12a40708a0 2024-04-12T21:01:07.867 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:07.869+0000 7f12a9000700 1 --2- 172.21.15.18:0/3146849465 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f12a4071f90 0x7f12a411a490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-12T21:01:07.868 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:07.869+0000 7f12a9a00700 1 --2- 172.21.15.18:0/3146849465 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f12a40708a0 0x7f12a4119f50 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f129800cac0 tx=0x7f1298007ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:01:07.868 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:07.869+0000 7f12a1600700 1 -- 172.21.15.18:0/3146849465 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f129800aa50 con 0x7f12a40708a0 2024-04-12T21:01:07.868 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:07.869+0000 7f12a1600700 1 -- 172.21.15.18:0/3146849465 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f129800abb0 con 0x7f12a40708a0 2024-04-12T21:01:07.868 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:07.869+0000 7f12a1600700 1 -- 172.21.15.18:0/3146849465 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f12980115e0 con 0x7f12a40708a0 2024-04-12T21:01:07.868 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:07.870+0000 7f12ac400700 1 -- 172.21.15.18:0/3146849465 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f12a41143a0 con 0x7f12a40708a0 2024-04-12T21:01:07.868 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:07.870+0000 7f12ac400700 1 -- 172.21.15.18:0/3146849465 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f12a41148e0 con 0x7f12a40708a0 2024-04-12T21:01:07.868 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:07.871+0000 7f12ac400700 1 -- 172.21.15.18:0/3146849465 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f12a4071aa0 con 0x7f12a40708a0 2024-04-12T21:01:07.869 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:07.872+0000 7f12a1600700 1 -- 172.21.15.18:0/3146849465 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 32) v1 ==== 92459+0+0 (secure 0 0 0) 0x7f129800ad20 con 0x7f12a40708a0 2024-04-12T21:01:07.870 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:07.873+0000 7f12a1600700 1 --2- 172.21.15.18:0/3146849465 >> [v2:172.21.15.18:6800/1605950634,v1:172.21.15.18:6801/1605950634] conn(0x7f128c06e500 0x7f128c0709b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:01:07.870 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:07.873+0000 7f12a1600700 1 -- 172.21.15.18:0/3146849465 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6066+0+0 (secure 0 0 0) 0x7f129808e690 con 0x7f12a40708a0 2024-04-12T21:01:07.871 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:07.873+0000 7f12a9000700 1 --2- 172.21.15.18:0/3146849465 >> [v2:172.21.15.18:6800/1605950634,v1:172.21.15.18:6801/1605950634] conn(0x7f128c06e500 0x7f128c0709b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:01:07.871 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:07.874+0000 7f12a9000700 1 --2- 172.21.15.18:0/3146849465 >> [v2:172.21.15.18:6800/1605950634,v1:172.21.15.18:6801/1605950634] conn(0x7f128c06e500 0x7f128c0709b0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f129000b9c0 tx=0x7f1290009e30 comp rx=0 tx=0).ready entity=mgr.14672 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:01:07.875 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:07.878+0000 7f12a1600700 1 -- 172.21.15.18:0/3146849465 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186454 (secure 0 0 0) 0x7f129805b770 con 0x7f12a40708a0 2024-04-12T21:01:08.030 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:01:07 smithi186 ceph-mon[36597]: pgmap v120: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-12T21:01:08.101 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:01:07 smithi018 ceph-mon[28799]: pgmap v120: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-12T21:01:08.110 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:08.113+0000 7f12ac400700 1 -- 172.21.15.18:0/3146849465 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command([{prefix=config set, name=mon_warn_on_insecure_global_id_reclaim_allowed}] v 0) v1 -- 0x7f12a4115660 con 0x7f12a40708a0 2024-04-12T21:01:08.111 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:08.113+0000 7f12a1600700 1 -- 172.21.15.18:0/3146849465 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_warn_on_insecure_global_id_reclaim_allowed}]=0 v46)=0 v46) v1 ==== 163+0+0 (secure 0 0 0) 0x7f129805b300 con 0x7f12a40708a0 2024-04-12T21:01:08.113 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:08.116+0000 7f12ac400700 1 -- 172.21.15.18:0/3146849465 >> [v2:172.21.15.18:6800/1605950634,v1:172.21.15.18:6801/1605950634] conn(0x7f128c06e500 msgr2=0x7f128c0709b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:01:08.114 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:08.116+0000 7f12ac400700 1 --2- 172.21.15.18:0/3146849465 >> [v2:172.21.15.18:6800/1605950634,v1:172.21.15.18:6801/1605950634] conn(0x7f128c06e500 0x7f128c0709b0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f129000b9c0 tx=0x7f1290009e30 comp rx=0 tx=0).stop 2024-04-12T21:01:08.114 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:08.116+0000 7f12ac400700 1 -- 172.21.15.18:0/3146849465 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f12a40708a0 msgr2=0x7f12a4119f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:01:08.114 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:08.116+0000 7f12ac400700 1 --2- 172.21.15.18:0/3146849465 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f12a40708a0 0x7f12a4119f50 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f129800cac0 tx=0x7f1298007ee0 comp rx=0 tx=0).stop 2024-04-12T21:01:08.114 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:08.116+0000 7f12ac400700 1 -- 172.21.15.18:0/3146849465 shutdown_connections 2024-04-12T21:01:08.114 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:08.116+0000 7f12ac400700 1 --2- 172.21.15.18:0/3146849465 >> [v2:172.21.15.18:6800/1605950634,v1:172.21.15.18:6801/1605950634] conn(0x7f128c06e500 0x7f128c0709b0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:01:08.115 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:08.116+0000 7f12ac400700 1 --2- 172.21.15.18:0/3146849465 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f12a40708a0 0x7f12a4119f50 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:01:08.115 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:08.116+0000 7f12ac400700 1 --2- 172.21.15.18:0/3146849465 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f12a4071f90 0x7f12a411a490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:01:08.115 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:08.116+0000 7f12ac400700 1 -- 172.21.15.18:0/3146849465 >> 172.21.15.18:0/3146849465 conn(0x7f12a40f89b0 msgr2=0x7f12a40f9a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:01:08.115 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:08.117+0000 7f12ac400700 1 -- 172.21.15.18:0/3146849465 shutdown_connections 2024-04-12T21:01:08.115 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:08.117+0000 7f12ac400700 1 -- 172.21.15.18:0/3146849465 wait complete. 2024-04-12T21:01:08.994 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1cbe407a-f90d-11ee-bc8d-c7b262605968 -e sha1=324baf8ddb91dea46cac50cb535c306115791355 -- bash -c 'ceph config set global log_to_journald false --force' 2024-04-12T21:01:09.516 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/mon.smithi018/config 2024-04-12T21:01:09.931 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:01:09 smithi018 ceph-mon[28799]: pgmap v121: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-12T21:01:10.030 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:01:09 smithi186 ceph-mon[36597]: pgmap v121: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-12T21:01:11.960 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:01:11 smithi018 ceph-mon[28799]: pgmap v122: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-12T21:01:11.960 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:01:11 smithi018 ceph-mon[28799]: fsmap cephfs:2 {0=cephfs.smithi018.acqbsc=up:active,1=cephfs.smithi018.szjluo=up:active} 2 up:standby 2024-04-12T21:01:11.984 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:11.986+0000 7f6456000700 1 -- 172.21.15.18:0/225414694 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6450109db0 msgr2=0x7f6450103420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:01:11.984 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:11.986+0000 7f6456000700 1 --2- 172.21.15.18:0/225414694 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6450109db0 0x7f6450103420 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f6438004940 tx=0x7f6438009e30 comp rx=0 tx=0).stop 2024-04-12T21:01:11.984 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:11.986+0000 7f6456000700 1 -- 172.21.15.18:0/225414694 shutdown_connections 2024-04-12T21:01:11.984 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:11.986+0000 7f6456000700 1 --2- 172.21.15.18:0/225414694 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6450109db0 0x7f6450103420 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:01:11.984 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:11.986+0000 7f6456000700 1 --2- 172.21.15.18:0/225414694 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6450109410 0x7f64501097e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:01:11.984 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:11.986+0000 7f6456000700 1 -- 172.21.15.18:0/225414694 >> 172.21.15.18:0/225414694 conn(0x7f6450072ba0 msgr2=0x7f6450074fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:01:11.984 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:11.986+0000 7f6456000700 1 -- 172.21.15.18:0/225414694 shutdown_connections 2024-04-12T21:01:11.984 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:11.986+0000 7f6456000700 1 -- 172.21.15.18:0/225414694 wait complete. 2024-04-12T21:01:11.985 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:11.987+0000 7f6456000700 1 Processor -- start 2024-04-12T21:01:11.985 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:11.987+0000 7f6456000700 1 -- start start 2024-04-12T21:01:11.985 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:11.987+0000 7f6456000700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6450109410 0x7f6450119f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:01:11.985 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:11.987+0000 7f6456000700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6450109db0 0x7f645011a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:01:11.986 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:11.988+0000 7f6456000700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f645011ab00 con 0x7f6450109410 2024-04-12T21:01:11.986 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:11.988+0000 7f6456000700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64501140e0 con 0x7f6450109db0 2024-04-12T21:01:11.986 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:11.988+0000 7f644f400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6450109410 0x7f6450119f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:01:11.986 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:11.988+0000 7f644f400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6450109410 0x7f6450119f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:43574/0 (socket says 172.21.15.18:43574) 2024-04-12T21:01:11.986 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:11.988+0000 7f644f400700 1 -- 172.21.15.18:0/2048125795 learned_addr learned my addr 172.21.15.18:0/2048125795 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:01:11.987 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:11.988+0000 7f6446a00700 1 --2- 172.21.15.18:0/2048125795 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6450109db0 0x7f645011a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:01:11.987 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:11.988+0000 7f644f400700 1 -- 172.21.15.18:0/2048125795 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6450109db0 msgr2=0x7f645011a4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:01:11.987 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:11.988+0000 7f644f400700 1 --2- 172.21.15.18:0/2048125795 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6450109db0 0x7f645011a4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:01:11.987 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:11.988+0000 7f644f400700 1 -- 172.21.15.18:0/2048125795 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6438005040 con 0x7f6450109410 2024-04-12T21:01:11.987 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:11.988+0000 7f6446a00700 1 --2- 172.21.15.18:0/2048125795 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6450109db0 0x7f645011a4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-12T21:01:11.987 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:11.989+0000 7f644f400700 1 --2- 172.21.15.18:0/2048125795 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6450109410 0x7f6450119f70 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f6440002a50 tx=0x7f64400046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:01:11.987 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:11.989+0000 7f6447e00700 1 -- 172.21.15.18:0/2048125795 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6440007cc0 con 0x7f6450109410 2024-04-12T21:01:11.987 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:11.989+0000 7f6447e00700 1 -- 172.21.15.18:0/2048125795 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f644000f040 con 0x7f6450109410 2024-04-12T21:01:11.987 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:11.989+0000 7f6447e00700 1 -- 172.21.15.18:0/2048125795 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6440009640 con 0x7f6450109410 2024-04-12T21:01:11.988 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:11.989+0000 7f6456000700 1 -- 172.21.15.18:0/2048125795 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f64501143c0 con 0x7f6450109410 2024-04-12T21:01:11.988 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:11.989+0000 7f6456000700 1 -- 172.21.15.18:0/2048125795 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6450114980 con 0x7f6450109410 2024-04-12T21:01:11.988 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:11.991+0000 7f6456000700 1 -- 172.21.15.18:0/2048125795 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6450104b60 con 0x7f6450109410 2024-04-12T21:01:11.989 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:11.992+0000 7f6447e00700 1 -- 172.21.15.18:0/2048125795 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 32) v1 ==== 92459+0+0 (secure 0 0 0) 0x7f64400097a0 con 0x7f6450109410 2024-04-12T21:01:11.990 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:11.993+0000 7f6447e00700 1 --2- 172.21.15.18:0/2048125795 >> [v2:172.21.15.18:6800/1605950634,v1:172.21.15.18:6801/1605950634] conn(0x7f643406e500 0x7f64340709b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:01:11.990 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:11.993+0000 7f6447e00700 1 -- 172.21.15.18:0/2048125795 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6066+0+0 (secure 0 0 0) 0x7f644008dbf0 con 0x7f6450109410 2024-04-12T21:01:11.990 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:11.993+0000 7f6446a00700 1 --2- 172.21.15.18:0/2048125795 >> [v2:172.21.15.18:6800/1605950634,v1:172.21.15.18:6801/1605950634] conn(0x7f643406e500 0x7f64340709b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:01:11.991 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:11.994+0000 7f6446a00700 1 --2- 172.21.15.18:0/2048125795 >> [v2:172.21.15.18:6800/1605950634,v1:172.21.15.18:6801/1605950634] conn(0x7f643406e500 0x7f64340709b0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f6438009fd0 tx=0x7f6438009e30 comp rx=0 tx=0).ready entity=mgr.14672 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:01:11.995 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:11.998+0000 7f6447e00700 1 -- 172.21.15.18:0/2048125795 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186454 (secure 0 0 0) 0x7f644005adf0 con 0x7f6450109410 2024-04-12T21:01:12.030 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:01:11 smithi186 ceph-mon[36597]: pgmap v122: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-12T21:01:12.030 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:01:11 smithi186 ceph-mon[36597]: fsmap cephfs:2 {0=cephfs.smithi018.acqbsc=up:active,1=cephfs.smithi018.szjluo=up:active} 2 up:standby 2024-04-12T21:01:12.227 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:12.229+0000 7f6456000700 1 -- 172.21.15.18:0/2048125795 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command([{prefix=config set, name=log_to_journald}] v 0) v1 -- 0x7f6450115660 con 0x7f6450109410 2024-04-12T21:01:12.227 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:12.230+0000 7f6447e00700 1 -- 172.21.15.18:0/2048125795 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{prefix=config set, name=log_to_journald}]=0 v46)=0 v46) v1 ==== 135+0+0 (secure 0 0 0) 0x7f644005a980 con 0x7f6450109410 2024-04-12T21:01:12.230 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:12.233+0000 7f6456000700 1 -- 172.21.15.18:0/2048125795 >> [v2:172.21.15.18:6800/1605950634,v1:172.21.15.18:6801/1605950634] conn(0x7f643406e500 msgr2=0x7f64340709b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:01:12.231 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:12.233+0000 7f6456000700 1 --2- 172.21.15.18:0/2048125795 >> [v2:172.21.15.18:6800/1605950634,v1:172.21.15.18:6801/1605950634] conn(0x7f643406e500 0x7f64340709b0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f6438009fd0 tx=0x7f6438009e30 comp rx=0 tx=0).stop 2024-04-12T21:01:12.231 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:12.233+0000 7f6456000700 1 -- 172.21.15.18:0/2048125795 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6450109410 msgr2=0x7f6450119f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:01:12.231 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:12.233+0000 7f6456000700 1 --2- 172.21.15.18:0/2048125795 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6450109410 0x7f6450119f70 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f6440002a50 tx=0x7f64400046b0 comp rx=0 tx=0).stop 2024-04-12T21:01:12.231 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:12.233+0000 7f6456000700 1 -- 172.21.15.18:0/2048125795 shutdown_connections 2024-04-12T21:01:12.231 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:12.233+0000 7f6456000700 1 --2- 172.21.15.18:0/2048125795 >> [v2:172.21.15.18:6800/1605950634,v1:172.21.15.18:6801/1605950634] conn(0x7f643406e500 0x7f64340709b0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:01:12.231 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:12.233+0000 7f6456000700 1 --2- 172.21.15.18:0/2048125795 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6450109410 0x7f6450119f70 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:01:12.231 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:12.233+0000 7f6456000700 1 --2- 172.21.15.18:0/2048125795 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6450109db0 0x7f645011a4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:01:12.231 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:12.233+0000 7f6456000700 1 -- 172.21.15.18:0/2048125795 >> 172.21.15.18:0/2048125795 conn(0x7f6450072ba0 msgr2=0x7f6450074ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:01:12.231 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:12.233+0000 7f6456000700 1 -- 172.21.15.18:0/2048125795 shutdown_connections 2024-04-12T21:01:12.232 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:12.233+0000 7f6456000700 1 -- 172.21.15.18:0/2048125795 wait complete. 2024-04-12T21:01:12.990 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:01:12 smithi018 ceph-mon[28799]: from='mgr.14672 172.21.15.18:0/2209228970' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T21:01:13.030 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:01:12 smithi186 ceph-mon[36597]: from='mgr.14672 172.21.15.18:0/2209228970' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T21:01:13.078 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1cbe407a-f90d-11ee-bc8d-c7b262605968 -e sha1=324baf8ddb91dea46cac50cb535c306115791355 -- bash -c 'ceph orch upgrade start --image quay.ceph.io/ceph-ci/ceph:$sha1' 2024-04-12T21:01:13.613 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/mon.smithi018/config 2024-04-12T21:01:13.769 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:01:13 smithi018 ceph-mon[28799]: pgmap v123: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-12T21:01:14.030 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:01:13 smithi186 ceph-mon[36597]: pgmap v123: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-12T21:01:15.941 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:01:15 smithi018 ceph-mon[28799]: pgmap v124: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-12T21:01:15.969 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:15.972+0000 7fccb9800700 1 -- 172.21.15.18:0/1662246514 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fccb4105800 msgr2=0x7fccb40fee70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:01:15.969 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:15.972+0000 7fccb9800700 1 --2- 172.21.15.18:0/1662246514 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fccb4105800 0x7fccb40fee70 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7fcca0004940 tx=0x7fcca0009e30 comp rx=0 tx=0).stop 2024-04-12T21:01:15.969 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:15.972+0000 7fccb9800700 1 -- 172.21.15.18:0/1662246514 shutdown_connections 2024-04-12T21:01:15.969 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:15.972+0000 7fccb9800700 1 --2- 172.21.15.18:0/1662246514 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fccb4105800 0x7fccb40fee70 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:01:15.969 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:15.972+0000 7fccb9800700 1 --2- 172.21.15.18:0/1662246514 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fccb4104e60 0x7fccb4105230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:01:15.969 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:15.972+0000 7fccb9800700 1 -- 172.21.15.18:0/1662246514 >> 172.21.15.18:0/1662246514 conn(0x7fccb40fab10 msgr2=0x7fccb40fcef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:01:15.969 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:15.972+0000 7fccb9800700 1 -- 172.21.15.18:0/1662246514 shutdown_connections 2024-04-12T21:01:15.969 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:15.972+0000 7fccb9800700 1 -- 172.21.15.18:0/1662246514 wait complete. 2024-04-12T21:01:15.970 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:15.973+0000 7fccb9800700 1 Processor -- start 2024-04-12T21:01:15.971 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:15.973+0000 7fccb9800700 1 -- start start 2024-04-12T21:01:15.971 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:15.974+0000 7fccb9800700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fccb4104e60 0x7fccb4115c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:01:15.971 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:15.974+0000 7fccb9800700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fccb4105800 0x7fccb41161a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:01:15.971 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:15.974+0000 7fccb9800700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fccb4116880 con 0x7fccb4105800 2024-04-12T21:01:15.971 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:15.974+0000 7fccb9800700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fccb410fbc0 con 0x7fccb4104e60 2024-04-12T21:01:15.971 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:15.974+0000 7fccb3400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fccb4105800 0x7fccb41161a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:01:15.971 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:15.974+0000 7fccb3400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fccb4105800 0x7fccb41161a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:43604/0 (socket says 172.21.15.18:43604) 2024-04-12T21:01:15.971 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:15.974+0000 7fccb3400700 1 -- 172.21.15.18:0/525937819 learned_addr learned my addr 172.21.15.18:0/525937819 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:01:15.972 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:15.974+0000 7fccb3e00700 1 --2- 172.21.15.18:0/525937819 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fccb4104e60 0x7fccb4115c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:01:15.972 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:15.975+0000 7fccb3400700 1 -- 172.21.15.18:0/525937819 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fccb4104e60 msgr2=0x7fccb4115c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:01:15.972 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:15.975+0000 7fccb3400700 1 --2- 172.21.15.18:0/525937819 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fccb4104e60 0x7fccb4115c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:01:15.972 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:15.975+0000 7fccb3400700 1 -- 172.21.15.18:0/525937819 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcca0005040 con 0x7fccb4105800 2024-04-12T21:01:15.972 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:15.975+0000 7fccb3400700 1 --2- 172.21.15.18:0/525937819 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fccb4105800 0x7fccb41161a0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7fcca0004f80 tx=0x7fcca0009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:01:15.972 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:15.975+0000 7fccabe00700 1 -- 172.21.15.18:0/525937819 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcca000e030 con 0x7fccb4105800 2024-04-12T21:01:15.972 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:15.975+0000 7fccabe00700 1 -- 172.21.15.18:0/525937819 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fcca000bde0 con 0x7fccb4105800 2024-04-12T21:01:15.973 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:15.975+0000 7fccabe00700 1 -- 172.21.15.18:0/525937819 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcca0013d10 con 0x7fccb4105800 2024-04-12T21:01:15.973 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:15.975+0000 7fccb9800700 1 -- 172.21.15.18:0/525937819 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fccb410fde0 con 0x7fccb4105800 2024-04-12T21:01:15.973 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:15.976+0000 7fccb9800700 1 -- 172.21.15.18:0/525937819 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fccb41103a0 con 0x7fccb4105800 2024-04-12T21:01:15.973 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:15.977+0000 7fccb9800700 1 -- 172.21.15.18:0/525937819 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fccb41005b0 con 0x7fccb4105800 2024-04-12T21:01:15.974 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:15.978+0000 7fccabe00700 1 -- 172.21.15.18:0/525937819 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 32) v1 ==== 92459+0+0 (secure 0 0 0) 0x7fcca0013520 con 0x7fccb4105800 2024-04-12T21:01:15.975 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:15.979+0000 7fccabe00700 1 --2- 172.21.15.18:0/525937819 >> [v2:172.21.15.18:6800/1605950634,v1:172.21.15.18:6801/1605950634] conn(0x7fcc9806e500 0x7fcc980709b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:01:15.975 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:15.979+0000 7fccabe00700 1 -- 172.21.15.18:0/525937819 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6066+0+0 (secure 0 0 0) 0x7fcca008f5a0 con 0x7fccb4105800 2024-04-12T21:01:15.976 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:15.979+0000 7fccb3e00700 1 --2- 172.21.15.18:0/525937819 >> [v2:172.21.15.18:6800/1605950634,v1:172.21.15.18:6801/1605950634] conn(0x7fcc9806e500 0x7fcc980709b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:01:15.976 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:15.980+0000 7fccb3e00700 1 --2- 172.21.15.18:0/525937819 >> [v2:172.21.15.18:6800/1605950634,v1:172.21.15.18:6801/1605950634] conn(0x7fcc9806e500 0x7fcc980709b0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fcca4006fd0 tx=0x7fcca4008040 comp rx=0 tx=0).ready entity=mgr.14672 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:01:15.980 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:15.984+0000 7fccabe00700 1 -- 172.21.15.18:0/525937819 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186454 (secure 0 0 0) 0x7fcca005c7a0 con 0x7fccb4105800 2024-04-12T21:01:16.030 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:01:15 smithi186 ceph-mon[36597]: pgmap v124: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-12T21:01:16.230 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:16.234+0000 7fccb9800700 1 -- 172.21.15.18:0/525937819 --> [v2:172.21.15.18:6800/1605950634,v1:172.21.15.18:6801/1605950634] -- mgr_command(tid 0: {"prefix": "orch upgrade start", "image": "quay.ceph.io/ceph-ci/ceph:324baf8ddb91dea46cac50cb535c306115791355", "target": ["mon-mgr", ""]}) v1 -- 0x7fccb404bf70 con 0x7fcc9806e500 2024-04-12T21:01:16.243 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:16.246+0000 7fccabe00700 1 -- 172.21.15.18:0/525937819 <== mgr.14672 v2:172.21.15.18:6800/1605950634 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+89 (secure 0 0 0) 0x7fcca400d660 con 0x7fcc9806e500 2024-04-12T21:01:16.243 INFO:teuthology.orchestra.run.smithi018.stdout:Initiating upgrade to quay.ceph.io/ceph-ci/ceph:324baf8ddb91dea46cac50cb535c306115791355 2024-04-12T21:01:16.246 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:16.249+0000 7fccb9800700 1 -- 172.21.15.18:0/525937819 >> [v2:172.21.15.18:6800/1605950634,v1:172.21.15.18:6801/1605950634] conn(0x7fcc9806e500 msgr2=0x7fcc980709b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:01:16.246 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:16.249+0000 7fccb9800700 1 --2- 172.21.15.18:0/525937819 >> [v2:172.21.15.18:6800/1605950634,v1:172.21.15.18:6801/1605950634] conn(0x7fcc9806e500 0x7fcc980709b0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fcca4006fd0 tx=0x7fcca4008040 comp rx=0 tx=0).stop 2024-04-12T21:01:16.246 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:16.249+0000 7fccb9800700 1 -- 172.21.15.18:0/525937819 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fccb4105800 msgr2=0x7fccb41161a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:01:16.246 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:16.249+0000 7fccb9800700 1 --2- 172.21.15.18:0/525937819 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fccb4105800 0x7fccb41161a0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7fcca0004f80 tx=0x7fcca0009e30 comp rx=0 tx=0).stop 2024-04-12T21:01:16.246 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:16.249+0000 7fccb9800700 1 -- 172.21.15.18:0/525937819 shutdown_connections 2024-04-12T21:01:16.246 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:16.249+0000 7fccb9800700 1 --2- 172.21.15.18:0/525937819 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fccb4104e60 0x7fccb4115c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:01:16.246 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:16.249+0000 7fccb9800700 1 --2- 172.21.15.18:0/525937819 >> [v2:172.21.15.18:6800/1605950634,v1:172.21.15.18:6801/1605950634] conn(0x7fcc9806e500 0x7fcc980709b0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:01:16.246 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:16.249+0000 7fccb9800700 1 --2- 172.21.15.18:0/525937819 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fccb4105800 0x7fccb41161a0 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:01:16.246 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:16.249+0000 7fccb9800700 1 -- 172.21.15.18:0/525937819 >> 172.21.15.18:0/525937819 conn(0x7fccb40fab10 msgr2=0x7fccb40fcab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:01:16.247 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:16.250+0000 7fccb9800700 1 -- 172.21.15.18:0/525937819 shutdown_connections 2024-04-12T21:01:16.247 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:16.250+0000 7fccb9800700 1 -- 172.21.15.18:0/525937819 wait complete. 2024-04-12T21:01:17.041 INFO:teuthology.task.sequential:In sequential, running task cephadm.shell... 2024-04-12T21:01:17.042 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi018.front.sepia.ceph.com 2024-04-12T21:01:17.042 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1cbe407a-f90d-11ee-bc8d-c7b262605968 -e sha1=324baf8ddb91dea46cac50cb535c306115791355 -- bash -c 'while ceph orch upgrade status | jq '"'"'.in_progress'"'"' | grep true && ! ceph orch upgrade status | jq '"'"'.message'"'"' | grep Error ; do ceph orch ps ; ceph versions ; ceph fs dump; ceph orch upgrade status ; ceph health detail ; sleep 30 ; done' 2024-04-12T21:01:17.530 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:01:17 smithi186 ceph-mon[36597]: from='client.14866 -' entity='client.admin' cmd=[{"prefix": "orch upgrade start", "image": "quay.ceph.io/ceph-ci/ceph:324baf8ddb91dea46cac50cb535c306115791355", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:01:17.530 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:01:17 smithi186 ceph-mon[36597]: Upgrade: Started with target quay.ceph.io/ceph-ci/ceph:324baf8ddb91dea46cac50cb535c306115791355 2024-04-12T21:01:17.530 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:01:17 smithi186 ceph-mon[36597]: from='mgr.14672 172.21.15.18:0/2209228970' entity='mgr.smithi018.xjzwga' 2024-04-12T21:01:17.530 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:01:17 smithi186 ceph-mon[36597]: from='mgr.14672 172.21.15.18:0/2209228970' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-12T21:01:17.530 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:01:17 smithi186 ceph-mon[36597]: pgmap v125: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-12T21:01:17.550 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/mon.smithi018/config 2024-04-12T21:01:17.601 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:01:17 smithi018 ceph-mon[28799]: from='client.14866 -' entity='client.admin' cmd=[{"prefix": "orch upgrade start", "image": "quay.ceph.io/ceph-ci/ceph:324baf8ddb91dea46cac50cb535c306115791355", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:01:17.601 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:01:17 smithi018 ceph-mon[28799]: Upgrade: Started with target quay.ceph.io/ceph-ci/ceph:324baf8ddb91dea46cac50cb535c306115791355 2024-04-12T21:01:17.601 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:01:17 smithi018 ceph-mon[28799]: from='mgr.14672 172.21.15.18:0/2209228970' entity='mgr.smithi018.xjzwga' 2024-04-12T21:01:17.601 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:01:17 smithi018 ceph-mon[28799]: from='mgr.14672 172.21.15.18:0/2209228970' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-12T21:01:17.601 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:01:17 smithi018 ceph-mon[28799]: pgmap v125: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-12T21:01:18.295 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:01:18 smithi018 ceph-mon[28799]: from='mgr.14672 172.21.15.18:0/2209228970' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-12T21:01:18.295 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:01:18 smithi018 ceph-mon[28799]: from='mgr.14672 172.21.15.18:0/2209228970' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-12T21:01:18.296 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:01:18 smithi018 ceph-mon[28799]: from='mgr.14672 172.21.15.18:0/2209228970' entity='mgr.smithi018.xjzwga' 2024-04-12T21:01:18.296 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:01:18 smithi018 ceph-mon[28799]: Upgrade: First pull of quay.ceph.io/ceph-ci/ceph:324baf8ddb91dea46cac50cb535c306115791355 2024-04-12T21:01:18.530 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:01:18 smithi186 ceph-mon[36597]: from='mgr.14672 172.21.15.18:0/2209228970' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-12T21:01:18.530 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:01:18 smithi186 ceph-mon[36597]: from='mgr.14672 172.21.15.18:0/2209228970' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-12T21:01:18.530 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:01:18 smithi186 ceph-mon[36597]: from='mgr.14672 172.21.15.18:0/2209228970' entity='mgr.smithi018.xjzwga' 2024-04-12T21:01:18.530 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:01:18 smithi186 ceph-mon[36597]: Upgrade: First pull of quay.ceph.io/ceph-ci/ceph:324baf8ddb91dea46cac50cb535c306115791355 2024-04-12T21:01:19.491 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:01:19 smithi018 ceph-mon[28799]: pgmap v126: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-12T21:01:19.530 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:01:19 smithi186 ceph-mon[36597]: pgmap v126: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-12T21:01:19.840 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:19.843+0000 7f4af3800700 1 -- 172.21.15.18:0/2015457970 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4aec1037a0 msgr2=0x7f4aec0fce10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:01:19.840 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:19.843+0000 7f4af3800700 1 --2- 172.21.15.18:0/2015457970 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4aec1037a0 0x7f4aec0fce10 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f4ad8004940 tx=0x7f4ad8009e30 comp rx=0 tx=0).stop 2024-04-12T21:01:19.840 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:19.843+0000 7f4af3800700 1 -- 172.21.15.18:0/2015457970 shutdown_connections 2024-04-12T21:01:19.840 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:19.843+0000 7f4af3800700 1 --2- 172.21.15.18:0/2015457970 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4aec1037a0 0x7f4aec0fce10 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:01:19.840 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:19.843+0000 7f4af3800700 1 --2- 172.21.15.18:0/2015457970 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4aec102e00 0x7f4aec1031d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:01:19.840 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:19.843+0000 7f4af3800700 1 -- 172.21.15.18:0/2015457970 >> 172.21.15.18:0/2015457970 conn(0x7f4aec0f89b0 msgr2=0x7f4aec0fadc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:01:19.840 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:19.843+0000 7f4af3800700 1 -- 172.21.15.18:0/2015457970 shutdown_connections 2024-04-12T21:01:19.840 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:19.843+0000 7f4af3800700 1 -- 172.21.15.18:0/2015457970 wait complete. 2024-04-12T21:01:19.841 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:19.844+0000 7f4af3800700 1 Processor -- start 2024-04-12T21:01:19.841 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:19.844+0000 7f4af3800700 1 -- start start 2024-04-12T21:01:19.842 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:19.845+0000 7f4af3800700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4aec102e00 0x7f4aec0702c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:01:19.842 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:19.845+0000 7f4af3800700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4aec1037a0 0x7f4aec070800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:01:19.842 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:19.845+0000 7f4af3800700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4aec072210 con 0x7f4aec1037a0 2024-04-12T21:01:19.842 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:19.845+0000 7f4af3800700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4aec070d40 con 0x7f4aec102e00 2024-04-12T21:01:19.842 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:19.845+0000 7f4aebe00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4aec1037a0 0x7f4aec070800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:01:19.842 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:19.845+0000 7f4aebe00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4aec1037a0 0x7f4aec070800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:43628/0 (socket says 172.21.15.18:43628) 2024-04-12T21:01:19.842 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:19.845+0000 7f4aebe00700 1 -- 172.21.15.18:0/984229938 learned_addr learned my addr 172.21.15.18:0/984229938 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:01:19.842 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:19.846+0000 7f4af0e00700 1 --2- 172.21.15.18:0/984229938 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4aec102e00 0x7f4aec0702c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:01:19.843 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:19.846+0000 7f4aebe00700 1 -- 172.21.15.18:0/984229938 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4aec102e00 msgr2=0x7f4aec0702c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:01:19.843 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:19.846+0000 7f4aebe00700 1 --2- 172.21.15.18:0/984229938 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4aec102e00 0x7f4aec0702c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:01:19.843 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:19.846+0000 7f4aebe00700 1 -- 172.21.15.18:0/984229938 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ad8005040 con 0x7f4aec1037a0 2024-04-12T21:01:19.843 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:19.846+0000 7f4aebe00700 1 --2- 172.21.15.18:0/984229938 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4aec1037a0 0x7f4aec070800 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f4ad8009180 tx=0x7f4ad8009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:01:19.844 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:19.847+0000 7f4ae8c00700 1 -- 172.21.15.18:0/984229938 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4ad800e040 con 0x7f4aec1037a0 2024-04-12T21:01:19.844 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:19.847+0000 7f4ae8c00700 1 -- 172.21.15.18:0/984229938 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f4ad800bbc0 con 0x7f4aec1037a0 2024-04-12T21:01:19.844 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:19.847+0000 7f4ae8c00700 1 -- 172.21.15.18:0/984229938 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4ad801c3f0 con 0x7f4aec1037a0 2024-04-12T21:01:19.844 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:19.847+0000 7f4af3800700 1 -- 172.21.15.18:0/984229938 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4aec070fc0 con 0x7f4aec1037a0 2024-04-12T21:01:19.844 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:19.847+0000 7f4af3800700 1 -- 172.21.15.18:0/984229938 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4aec11a110 con 0x7f4aec1037a0 2024-04-12T21:01:19.845 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:19.848+0000 7f4af3800700 1 -- 172.21.15.18:0/984229938 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4aec1052d0 con 0x7f4aec1037a0 2024-04-12T21:01:19.845 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:19.849+0000 7f4ae8c00700 1 -- 172.21.15.18:0/984229938 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 32) v1 ==== 92459+0+0 (secure 0 0 0) 0x7f4ad800bd30 con 0x7f4aec1037a0 2024-04-12T21:01:19.846 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:19.850+0000 7f4ae8c00700 1 --2- 172.21.15.18:0/984229938 >> [v2:172.21.15.18:6800/1605950634,v1:172.21.15.18:6801/1605950634] conn(0x7f4ad406e500 0x7f4ad40709b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:01:19.847 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:19.850+0000 7f4ae8c00700 1 -- 172.21.15.18:0/984229938 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6066+0+0 (secure 0 0 0) 0x7f4ad808f1f0 con 0x7f4aec1037a0 2024-04-12T21:01:19.847 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:19.851+0000 7f4af0e00700 1 --2- 172.21.15.18:0/984229938 >> [v2:172.21.15.18:6800/1605950634,v1:172.21.15.18:6801/1605950634] conn(0x7f4ad406e500 0x7f4ad40709b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:01:19.848 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:19.851+0000 7f4af0e00700 1 --2- 172.21.15.18:0/984229938 >> [v2:172.21.15.18:6800/1605950634,v1:172.21.15.18:6801/1605950634] conn(0x7f4ad406e500 0x7f4ad40709b0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f4ae0007fd0 tx=0x7f4ae0009040 comp rx=0 tx=0).ready entity=mgr.14672 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:01:19.852 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:19.856+0000 7f4ae8c00700 1 -- 172.21.15.18:0/984229938 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186454 (secure 0 0 0) 0x7f4ad80588a0 con 0x7f4aec1037a0 2024-04-12T21:01:20.117 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:20.119+0000 7f4af3800700 1 -- 172.21.15.18:0/984229938 --> [v2:172.21.15.18:6800/1605950634,v1:172.21.15.18:6801/1605950634] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f4aec109ee0 con 0x7f4ad406e500 2024-04-12T21:01:20.118 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:20.121+0000 7f4ae8c00700 1 -- 172.21.15.18:0/984229938 <== mgr.14672 v2:172.21.15.18:6800/1605950634 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+358 (secure 0 0 0) 0x7f4aec109ee0 con 0x7f4ad406e500 2024-04-12T21:01:20.121 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:20.124+0000 7f4af3800700 1 -- 172.21.15.18:0/984229938 >> [v2:172.21.15.18:6800/1605950634,v1:172.21.15.18:6801/1605950634] conn(0x7f4ad406e500 msgr2=0x7f4ad40709b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:01:20.121 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:20.124+0000 7f4af3800700 1 --2- 172.21.15.18:0/984229938 >> [v2:172.21.15.18:6800/1605950634,v1:172.21.15.18:6801/1605950634] conn(0x7f4ad406e500 0x7f4ad40709b0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f4ae0007fd0 tx=0x7f4ae0009040 comp rx=0 tx=0).stop 2024-04-12T21:01:20.121 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:20.124+0000 7f4af3800700 1 -- 172.21.15.18:0/984229938 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4aec1037a0 msgr2=0x7f4aec070800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:01:20.121 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:20.124+0000 7f4af3800700 1 --2- 172.21.15.18:0/984229938 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4aec1037a0 0x7f4aec070800 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f4ad8009180 tx=0x7f4ad8009e30 comp rx=0 tx=0).stop 2024-04-12T21:01:20.121 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:20.124+0000 7f4af3800700 1 -- 172.21.15.18:0/984229938 shutdown_connections 2024-04-12T21:01:20.121 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:20.124+0000 7f4af3800700 1 --2- 172.21.15.18:0/984229938 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4aec102e00 0x7f4aec0702c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:01:20.121 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:20.124+0000 7f4af3800700 1 --2- 172.21.15.18:0/984229938 >> [v2:172.21.15.18:6800/1605950634,v1:172.21.15.18:6801/1605950634] conn(0x7f4ad406e500 0x7f4ad40709b0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:01:20.122 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:20.124+0000 7f4af3800700 1 --2- 172.21.15.18:0/984229938 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4aec1037a0 0x7f4aec070800 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:01:20.122 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:20.125+0000 7f4af3800700 1 -- 172.21.15.18:0/984229938 >> 172.21.15.18:0/984229938 conn(0x7f4aec0f89b0 msgr2=0x7f4aec1021c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:01:20.122 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:20.125+0000 7f4af3800700 1 -- 172.21.15.18:0/984229938 shutdown_connections 2024-04-12T21:01:20.122 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:20.125+0000 7f4af3800700 1 -- 172.21.15.18:0/984229938 wait complete. 2024-04-12T21:01:20.138 INFO:teuthology.orchestra.run.smithi018.stdout:true 2024-04-12T21:01:20.284 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:20.287+0000 7f302f400700 1 -- 172.21.15.18:0/2154234276 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f30280fde90 msgr2=0x7f30280fe260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:01:20.284 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:20.287+0000 7f302f400700 1 --2- 172.21.15.18:0/2154234276 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f30280fde90 0x7f30280fe260 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f301c004940 tx=0x7f301c009e30 comp rx=0 tx=0).stop 2024-04-12T21:01:20.284 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:20.288+0000 7f302f400700 1 -- 172.21.15.18:0/2154234276 shutdown_connections 2024-04-12T21:01:20.285 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:20.288+0000 7f302f400700 1 --2- 172.21.15.18:0/2154234276 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f30280fe830 0x7f3028106f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:01:20.285 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:20.288+0000 7f302f400700 1 --2- 172.21.15.18:0/2154234276 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f30280fde90 0x7f30280fe260 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:01:20.285 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:20.288+0000 7f302f400700 1 -- 172.21.15.18:0/2154234276 >> 172.21.15.18:0/2154234276 conn(0x7f3028072bc0 msgr2=0x7f3028072fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:01:20.285 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:20.288+0000 7f302f400700 1 -- 172.21.15.18:0/2154234276 shutdown_connections 2024-04-12T21:01:20.285 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:20.288+0000 7f302f400700 1 -- 172.21.15.18:0/2154234276 wait complete. 2024-04-12T21:01:20.287 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:20.289+0000 7f302f400700 1 Processor -- start 2024-04-12T21:01:20.287 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:20.289+0000 7f302f400700 1 -- start start 2024-04-12T21:01:20.287 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:20.289+0000 7f302f400700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f30280fde90 0x7f3028115d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:01:20.287 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:20.289+0000 7f302f400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f30280fe830 0x7f30281162c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:01:20.287 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:20.289+0000 7f302f400700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3028116910 con 0x7f30280fe830 2024-04-12T21:01:20.287 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:20.289+0000 7f302f400700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f302810fd20 con 0x7f30280fde90 2024-04-12T21:01:20.287 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:20.290+0000 7f3027e00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f30280fe830 0x7f30281162c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:01:20.287 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:20.290+0000 7f3027e00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f30280fe830 0x7f30281162c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:43652/0 (socket says 172.21.15.18:43652) 2024-04-12T21:01:20.287 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:20.290+0000 7f3027e00700 1 -- 172.21.15.18:0/1908910230 learned_addr learned my addr 172.21.15.18:0/1908910230 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:01:20.288 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:20.290+0000 7f302ca00700 1 --2- 172.21.15.18:0/1908910230 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f30280fde90 0x7f3028115d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:01:20.288 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:20.290+0000 7f3027e00700 1 -- 172.21.15.18:0/1908910230 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f30280fde90 msgr2=0x7f3028115d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:01:20.288 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:20.290+0000 7f3027e00700 1 --2- 172.21.15.18:0/1908910230 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f30280fde90 0x7f3028115d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:01:20.288 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:20.290+0000 7f3027e00700 1 -- 172.21.15.18:0/1908910230 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f301c005040 con 0x7f30280fe830 2024-04-12T21:01:20.288 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:20.291+0000 7f3027e00700 1 --2- 172.21.15.18:0/1908910230 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f30280fe830 0x7f30281162c0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f3014002a50 tx=0x7f30140046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:01:20.289 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:20.291+0000 7f3024c00700 1 -- 172.21.15.18:0/1908910230 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3014007cc0 con 0x7f30280fe830 2024-04-12T21:01:20.289 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:20.291+0000 7f3024c00700 1 -- 172.21.15.18:0/1908910230 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f301400f040 con 0x7f30280fe830 2024-04-12T21:01:20.289 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:20.291+0000 7f302f400700 1 -- 172.21.15.18:0/1908910230 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3028110000 con 0x7f30280fe830 2024-04-12T21:01:20.289 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:20.291+0000 7f3024c00700 1 -- 172.21.15.18:0/1908910230 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3014009640 con 0x7f30280fe830 2024-04-12T21:01:20.289 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:20.291+0000 7f302f400700 1 -- 172.21.15.18:0/1908910230 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3028110540 con 0x7f30280fe830 2024-04-12T21:01:20.289 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:20.292+0000 7f302f400700 1 -- 172.21.15.18:0/1908910230 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3028106a20 con 0x7f30280fe830 2024-04-12T21:01:20.290 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:20.293+0000 7f3024c00700 1 -- 172.21.15.18:0/1908910230 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 32) v1 ==== 92459+0+0 (secure 0 0 0) 0x7f30140097a0 con 0x7f30280fe830 2024-04-12T21:01:20.291 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:20.295+0000 7f3024c00700 1 --2- 172.21.15.18:0/1908910230 >> [v2:172.21.15.18:6800/1605950634,v1:172.21.15.18:6801/1605950634] conn(0x7f301006e510 0x7f30100709c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:01:20.291 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:20.295+0000 7f3024c00700 1 -- 172.21.15.18:0/1908910230 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6066+0+0 (secure 0 0 0) 0x7f301408dd60 con 0x7f30280fe830 2024-04-12T21:01:20.291 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:20.295+0000 7f302ca00700 1 --2- 172.21.15.18:0/1908910230 >> [v2:172.21.15.18:6800/1605950634,v1:172.21.15.18:6801/1605950634] conn(0x7f301006e510 0x7f30100709c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:01:20.292 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:20.296+0000 7f302ca00700 1 --2- 172.21.15.18:0/1908910230 >> [v2:172.21.15.18:6800/1605950634,v1:172.21.15.18:6801/1605950634] conn(0x7f301006e510 0x7f30100709c0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f301c00b3b0 tx=0x7f301c009e30 comp rx=0 tx=0).ready entity=mgr.14672 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:01:20.296 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:20.299+0000 7f3024c00700 1 -- 172.21.15.18:0/1908910230 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186454 (secure 0 0 0) 0x7f3014092070 con 0x7f30280fe830 2024-04-12T21:01:20.551 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:20.554+0000 7f302f400700 1 -- 172.21.15.18:0/1908910230 --> [v2:172.21.15.18:6800/1605950634,v1:172.21.15.18:6801/1605950634] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f3028111410 con 0x7f301006e510 2024-04-12T21:01:20.553 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:20.557+0000 7f3024c00700 1 -- 172.21.15.18:0/1908910230 <== mgr.14672 v2:172.21.15.18:6800/1605950634 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+358 (secure 0 0 0) 0x7f3028111410 con 0x7f301006e510 2024-04-12T21:01:20.556 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:20.560+0000 7f302f400700 1 -- 172.21.15.18:0/1908910230 >> [v2:172.21.15.18:6800/1605950634,v1:172.21.15.18:6801/1605950634] conn(0x7f301006e510 msgr2=0x7f30100709c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:01:20.556 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:20.560+0000 7f302f400700 1 --2- 172.21.15.18:0/1908910230 >> [v2:172.21.15.18:6800/1605950634,v1:172.21.15.18:6801/1605950634] conn(0x7f301006e510 0x7f30100709c0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f301c00b3b0 tx=0x7f301c009e30 comp rx=0 tx=0).stop 2024-04-12T21:01:20.556 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:20.560+0000 7f302f400700 1 -- 172.21.15.18:0/1908910230 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f30280fe830 msgr2=0x7f30281162c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:01:20.557 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:20.560+0000 7f302f400700 1 --2- 172.21.15.18:0/1908910230 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f30280fe830 0x7f30281162c0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f3014002a50 tx=0x7f30140046b0 comp rx=0 tx=0).stop 2024-04-12T21:01:20.557 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:20.560+0000 7f302f400700 1 -- 172.21.15.18:0/1908910230 shutdown_connections 2024-04-12T21:01:20.557 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:20.560+0000 7f302f400700 1 --2- 172.21.15.18:0/1908910230 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f30280fde90 0x7f3028115d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:01:20.557 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:20.560+0000 7f302f400700 1 --2- 172.21.15.18:0/1908910230 >> [v2:172.21.15.18:6800/1605950634,v1:172.21.15.18:6801/1605950634] conn(0x7f301006e510 0x7f30100709c0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:01:20.557 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:20.560+0000 7f302f400700 1 --2- 172.21.15.18:0/1908910230 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f30280fe830 0x7f30281162c0 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:01:20.557 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:20.560+0000 7f302f400700 1 -- 172.21.15.18:0/1908910230 >> 172.21.15.18:0/1908910230 conn(0x7f3028072bc0 msgr2=0x7f30280fb840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:01:20.557 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:20.560+0000 7f302f400700 1 -- 172.21.15.18:0/1908910230 shutdown_connections 2024-04-12T21:01:20.557 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:20.560+0000 7f302f400700 1 -- 172.21.15.18:0/1908910230 wait complete. 2024-04-12T21:01:20.720 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:20.723+0000 7fda74800700 1 -- 172.21.15.18:0/494760353 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fda6c105010 msgr2=0x7fda6c1053e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:01:20.720 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:20.723+0000 7fda74800700 1 --2- 172.21.15.18:0/494760353 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fda6c105010 0x7fda6c1053e0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7fda60004940 tx=0x7fda60009e30 comp rx=0 tx=0).stop 2024-04-12T21:01:20.720 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:20.723+0000 7fda74800700 1 -- 172.21.15.18:0/494760353 shutdown_connections 2024-04-12T21:01:20.720 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:20.723+0000 7fda74800700 1 --2- 172.21.15.18:0/494760353 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fda6c1059b0 0x7fda6c0ff020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:01:20.720 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:20.723+0000 7fda74800700 1 --2- 172.21.15.18:0/494760353 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fda6c105010 0x7fda6c1053e0 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:01:20.720 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:20.723+0000 7fda74800700 1 -- 172.21.15.18:0/494760353 >> 172.21.15.18:0/494760353 conn(0x7fda6c0faba0 msgr2=0x7fda6c0fcfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:01:20.720 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:20.723+0000 7fda74800700 1 -- 172.21.15.18:0/494760353 shutdown_connections 2024-04-12T21:01:20.721 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:20.724+0000 7fda74800700 1 -- 172.21.15.18:0/494760353 wait complete. 2024-04-12T21:01:20.721 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:20.725+0000 7fda74800700 1 Processor -- start 2024-04-12T21:01:20.721 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:20.725+0000 7fda74800700 1 -- start start 2024-04-12T21:01:20.722 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:20.725+0000 7fda74800700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fda6c105010 0x7fda6c115da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:01:20.722 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:20.725+0000 7fda74800700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fda6c1059b0 0x7fda6c1162e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:01:20.722 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:20.725+0000 7fda74800700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda6c116930 con 0x7fda6c105010 2024-04-12T21:01:20.722 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:20.725+0000 7fda74800700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda6c10fd00 con 0x7fda6c1059b0 2024-04-12T21:01:20.722 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:20.726+0000 7fda71e00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fda6c105010 0x7fda6c115da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:01:20.722 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:20.726+0000 7fda71e00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fda6c105010 0x7fda6c115da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:43670/0 (socket says 172.21.15.18:43670) 2024-04-12T21:01:20.722 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:20.726+0000 7fda71e00700 1 -- 172.21.15.18:0/819995492 learned_addr learned my addr 172.21.15.18:0/819995492 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:01:20.723 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:20.726+0000 7fda71400700 1 --2- 172.21.15.18:0/819995492 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fda6c1059b0 0x7fda6c1162e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:01:20.723 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:20.726+0000 7fda71e00700 1 -- 172.21.15.18:0/819995492 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fda6c1059b0 msgr2=0x7fda6c1162e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:01:20.723 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:20.726+0000 7fda71e00700 1 --2- 172.21.15.18:0/819995492 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fda6c1059b0 0x7fda6c1162e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:01:20.724 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:20.726+0000 7fda71e00700 1 -- 172.21.15.18:0/819995492 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda60005040 con 0x7fda6c105010 2024-04-12T21:01:20.724 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:20.726+0000 7fda71400700 1 --2- 172.21.15.18:0/819995492 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fda6c1059b0 0x7fda6c1162e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-12T21:01:20.724 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:20.727+0000 7fda71e00700 1 --2- 172.21.15.18:0/819995492 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fda6c105010 0x7fda6c115da0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7fda60009180 tx=0x7fda60009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:01:20.724 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:20.727+0000 7fda66000700 1 -- 172.21.15.18:0/819995492 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fda6000e040 con 0x7fda6c105010 2024-04-12T21:01:20.724 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:20.727+0000 7fda66000700 1 -- 172.21.15.18:0/819995492 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fda6000bbc0 con 0x7fda6c105010 2024-04-12T21:01:20.724 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:20.727+0000 7fda74800700 1 -- 172.21.15.18:0/819995492 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fda6c10ff80 con 0x7fda6c105010 2024-04-12T21:01:20.724 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:20.727+0000 7fda66000700 1 -- 172.21.15.18:0/819995492 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fda6001c630 con 0x7fda6c105010 2024-04-12T21:01:20.724 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:20.727+0000 7fda74800700 1 -- 172.21.15.18:0/819995492 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fda6c110540 con 0x7fda6c105010 2024-04-12T21:01:20.724 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:20.728+0000 7fda74800700 1 -- 172.21.15.18:0/819995492 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fda6c100760 con 0x7fda6c105010 2024-04-12T21:01:20.725 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:20.729+0000 7fda66000700 1 -- 172.21.15.18:0/819995492 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 32) v1 ==== 92459+0+0 (secure 0 0 0) 0x7fda6000bd30 con 0x7fda6c105010 2024-04-12T21:01:20.727 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:20.730+0000 7fda66000700 1 --2- 172.21.15.18:0/819995492 >> [v2:172.21.15.18:6800/1605950634,v1:172.21.15.18:6801/1605950634] conn(0x7fda5406e510 0x7fda540709c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:01:20.727 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:20.730+0000 7fda66000700 1 -- 172.21.15.18:0/819995492 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6066+0+0 (secure 0 0 0) 0x7fda6008f2a0 con 0x7fda6c105010 2024-04-12T21:01:20.727 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:20.731+0000 7fda71400700 1 --2- 172.21.15.18:0/819995492 >> [v2:172.21.15.18:6800/1605950634,v1:172.21.15.18:6801/1605950634] conn(0x7fda5406e510 0x7fda540709c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:01:20.728 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:20.732+0000 7fda71400700 1 --2- 172.21.15.18:0/819995492 >> [v2:172.21.15.18:6800/1605950634,v1:172.21.15.18:6801/1605950634] conn(0x7fda5406e510 0x7fda540709c0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fda5c007fd0 tx=0x7fda5c009040 comp rx=0 tx=0).ready entity=mgr.14672 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:01:20.733 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:20.737+0000 7fda66000700 1 -- 172.21.15.18:0/819995492 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186454 (secure 0 0 0) 0x7fda6005c4a0 con 0x7fda6c105010 2024-04-12T21:01:20.983 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:20.986+0000 7fda74800700 1 -- 172.21.15.18:0/819995492 --> [v2:172.21.15.18:6800/1605950634,v1:172.21.15.18:6801/1605950634] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fda6c1113f0 con 0x7fda5406e510 2024-04-12T21:01:21.003 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:21.007+0000 7fda66000700 1 -- 172.21.15.18:0/819995492 <== mgr.14672 v2:172.21.15.18:6800/1605950634 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3520 (secure 0 0 0) 0x7fda6c1113f0 con 0x7fda5406e510 2024-04-12T21:01:21.003 INFO:teuthology.orchestra.run.smithi018.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-12T21:01:21.004 INFO:teuthology.orchestra.run.smithi018.stdout:alertmanager.smithi018 smithi018 *:9093,9094 running (117s) 73s ago 16m 14.2M - 0.25.0 c8568f914cd2 c332090068e8 2024-04-12T21:01:21.004 INFO:teuthology.orchestra.run.smithi018.stdout:crash.smithi018 smithi018 running (16m) 73s ago 16m 6660k - 17.2.7-827-gd3b5060a 21de9b7369ad 1ac7603bddfd 2024-04-12T21:01:21.004 INFO:teuthology.orchestra.run.smithi018.stdout:crash.smithi186 smithi186 running (15m) 2m ago 15m 6669k - 17.2.7-827-gd3b5060a 21de9b7369ad 14bf8a1672eb 2024-04-12T21:01:21.004 INFO:teuthology.orchestra.run.smithi018.stdout:grafana.smithi018 smithi018 *:3000 running (81s) 73s ago 16m 51.4M - 9.4.12 2bacad6d85d8 b08e3bf176dd 2024-04-12T21:01:21.004 INFO:teuthology.orchestra.run.smithi018.stdout:mds.cephfs.smithi018.acqbsc smithi018 running (9m) 73s ago 9m 112M - 17.2.7-827-gd3b5060a 21de9b7369ad 5b5fce65f18a 2024-04-12T21:01:21.004 INFO:teuthology.orchestra.run.smithi018.stdout:mds.cephfs.smithi018.szjluo smithi018 running (9m) 73s ago 9m 113M - 17.2.7-827-gd3b5060a 21de9b7369ad cf9bf0c5031f 2024-04-12T21:01:21.004 INFO:teuthology.orchestra.run.smithi018.stdout:mds.cephfs.smithi186.jdllyn smithi186 running (9m) 2m ago 9m 13.0M - 17.2.7-827-gd3b5060a 21de9b7369ad 2b2bc46bd29a 2024-04-12T21:01:21.004 INFO:teuthology.orchestra.run.smithi018.stdout:mds.cephfs.smithi186.wjnafs smithi186 running (9m) 2m ago 9m 13.0M - 17.2.7-827-gd3b5060a 21de9b7369ad 74b115d6d6fd 2024-04-12T21:01:21.004 INFO:teuthology.orchestra.run.smithi018.stdout:mgr.smithi018.xjzwga smithi018 *:8443,9283,8765 running (4m) 73s ago 18m 507M - 19.0.0-2879-g324baf8d 8717720dc5e5 56088305eab7 2024-04-12T21:01:21.004 INFO:teuthology.orchestra.run.smithi018.stdout:mgr.smithi186.kqcfxk smithi186 *:8443,9283,8765 running (3m) 2m ago 15m 447M - 19.0.0-2879-g324baf8d 8717720dc5e5 09cd766e5e0d 2024-04-12T21:01:21.004 INFO:teuthology.orchestra.run.smithi018.stdout:mon.smithi018 smithi018 running (18m) 73s ago 18m 57.1M 2048M 17.2.7-827-gd3b5060a 21de9b7369ad cd84caafa056 2024-04-12T21:01:21.004 INFO:teuthology.orchestra.run.smithi018.stdout:mon.smithi186 smithi186 running (15m) 2m ago 15m 41.6M 2048M 17.2.7-827-gd3b5060a 21de9b7369ad d7263c772b73 2024-04-12T21:01:21.004 INFO:teuthology.orchestra.run.smithi018.stdout:node-exporter.smithi018 smithi018 *:9100 running (3m) 73s ago 16m 7675k - 1.5.0 0da6a335fe13 326c3d9c6847 2024-04-12T21:01:21.004 INFO:teuthology.orchestra.run.smithi018.stdout:node-exporter.smithi186 smithi186 *:9100 running (2m) 2m ago 15m 2705k - 1.5.0 0da6a335fe13 ec2112baeecf 2024-04-12T21:01:21.005 INFO:teuthology.orchestra.run.smithi018.stdout:osd.0 smithi018 running (14m) 73s ago 14m 225M 1559M 17.2.7-827-gd3b5060a 21de9b7369ad 0d83e027bf0a 2024-04-12T21:01:21.005 INFO:teuthology.orchestra.run.smithi018.stdout:osd.1 smithi018 running (13m) 73s ago 13m 239M 1559M 17.2.7-827-gd3b5060a 21de9b7369ad 6517c9cc7f87 2024-04-12T21:01:21.005 INFO:teuthology.orchestra.run.smithi018.stdout:osd.2 smithi018 running (12m) 73s ago 12m 193M 1559M 17.2.7-827-gd3b5060a 21de9b7369ad 8a83692acaa5 2024-04-12T21:01:21.005 INFO:teuthology.orchestra.run.smithi018.stdout:osd.3 smithi186 running (12m) 2m ago 12m 258M 2581M 17.2.7-827-gd3b5060a 21de9b7369ad c2def7a90d76 2024-04-12T21:01:21.005 INFO:teuthology.orchestra.run.smithi018.stdout:osd.4 smithi186 running (11m) 2m ago 11m 228M 2581M 17.2.7-827-gd3b5060a 21de9b7369ad 757e7c71b7b9 2024-04-12T21:01:21.005 INFO:teuthology.orchestra.run.smithi018.stdout:osd.5 smithi186 running (11m) 2m ago 11m 222M 2581M 17.2.7-827-gd3b5060a 21de9b7369ad 2ee072a436d0 2024-04-12T21:01:21.005 INFO:teuthology.orchestra.run.smithi018.stdout:prometheus.smithi018 smithi018 *:9095 running (2m) 73s ago 15m 49.5M - 2.43.0 a07b618ecd1d f58841127e21 2024-04-12T21:01:21.006 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:21.009+0000 7fda74800700 1 -- 172.21.15.18:0/819995492 >> [v2:172.21.15.18:6800/1605950634,v1:172.21.15.18:6801/1605950634] conn(0x7fda5406e510 msgr2=0x7fda540709c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:01:21.006 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:21.009+0000 7fda74800700 1 --2- 172.21.15.18:0/819995492 >> [v2:172.21.15.18:6800/1605950634,v1:172.21.15.18:6801/1605950634] conn(0x7fda5406e510 0x7fda540709c0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fda5c007fd0 tx=0x7fda5c009040 comp rx=0 tx=0).stop 2024-04-12T21:01:21.006 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:21.009+0000 7fda74800700 1 -- 172.21.15.18:0/819995492 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fda6c105010 msgr2=0x7fda6c115da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:01:21.007 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:21.009+0000 7fda74800700 1 --2- 172.21.15.18:0/819995492 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fda6c105010 0x7fda6c115da0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7fda60009180 tx=0x7fda60009e30 comp rx=0 tx=0).stop 2024-04-12T21:01:21.007 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:21.010+0000 7fda74800700 1 -- 172.21.15.18:0/819995492 shutdown_connections 2024-04-12T21:01:21.007 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:21.010+0000 7fda74800700 1 --2- 172.21.15.18:0/819995492 >> [v2:172.21.15.18:6800/1605950634,v1:172.21.15.18:6801/1605950634] conn(0x7fda5406e510 0x7fda540709c0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:01:21.007 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:21.010+0000 7fda74800700 1 --2- 172.21.15.18:0/819995492 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fda6c105010 0x7fda6c115da0 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:01:21.007 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:21.010+0000 7fda74800700 1 --2- 172.21.15.18:0/819995492 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fda6c1059b0 0x7fda6c1162e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:01:21.007 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:21.010+0000 7fda74800700 1 -- 172.21.15.18:0/819995492 >> 172.21.15.18:0/819995492 conn(0x7fda6c0faba0 msgr2=0x7fda6c0fbc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:01:21.007 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:21.010+0000 7fda74800700 1 -- 172.21.15.18:0/819995492 shutdown_connections 2024-04-12T21:01:21.007 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:21.010+0000 7fda74800700 1 -- 172.21.15.18:0/819995492 wait complete. 2024-04-12T21:01:21.159 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:21.162+0000 7f77ed600700 1 -- 172.21.15.18:0/516369893 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f77e81092a0 msgr2=0x7f77e8109670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:01:21.159 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:21.162+0000 7f77ed600700 1 --2- 172.21.15.18:0/516369893 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f77e81092a0 0x7f77e8109670 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f77d8004940 tx=0x7f77d8009e30 comp rx=0 tx=0).stop 2024-04-12T21:01:21.159 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:21.162+0000 7f77ed600700 1 -- 172.21.15.18:0/516369893 shutdown_connections 2024-04-12T21:01:21.159 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:21.162+0000 7f77ed600700 1 --2- 172.21.15.18:0/516369893 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f77e8109c40 0x7f77e81032b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:01:21.159 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:21.162+0000 7f77ed600700 1 --2- 172.21.15.18:0/516369893 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f77e81092a0 0x7f77e8109670 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:01:21.159 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:21.162+0000 7f77ed600700 1 -- 172.21.15.18:0/516369893 >> 172.21.15.18:0/516369893 conn(0x7f77e8072a30 msgr2=0x7f77e8074e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:01:21.159 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:21.162+0000 7f77ed600700 1 -- 172.21.15.18:0/516369893 shutdown_connections 2024-04-12T21:01:21.159 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:21.162+0000 7f77ed600700 1 -- 172.21.15.18:0/516369893 wait complete. 2024-04-12T21:01:21.160 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:21.163+0000 7f77ed600700 1 Processor -- start 2024-04-12T21:01:21.160 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:21.163+0000 7f77ed600700 1 -- start start 2024-04-12T21:01:21.160 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:21.163+0000 7f77ed600700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f77e81092a0 0x7f77e8119ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:01:21.160 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:21.163+0000 7f77ed600700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f77e8109c40 0x7f77e811a530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:01:21.161 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:21.163+0000 7f77ed600700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77e811ab80 con 0x7f77e8109c40 2024-04-12T21:01:21.161 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:21.163+0000 7f77ed600700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77e8113f50 con 0x7f77e81092a0 2024-04-12T21:01:21.161 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:21.164+0000 7f77e7400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f77e8109c40 0x7f77e811a530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:01:21.161 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:21.164+0000 7f77e7400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f77e8109c40 0x7f77e811a530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:33694/0 (socket says 172.21.15.18:33694) 2024-04-12T21:01:21.161 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:21.164+0000 7f77e7400700 1 -- 172.21.15.18:0/3948036812 learned_addr learned my addr 172.21.15.18:0/3948036812 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:01:21.161 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:21.164+0000 7f77e7e00700 1 --2- 172.21.15.18:0/3948036812 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f77e81092a0 0x7f77e8119ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:01:21.162 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:21.164+0000 7f77e7400700 1 -- 172.21.15.18:0/3948036812 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f77e81092a0 msgr2=0x7f77e8119ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:01:21.162 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:21.164+0000 7f77e7400700 1 --2- 172.21.15.18:0/3948036812 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f77e81092a0 0x7f77e8119ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:01:21.162 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:21.164+0000 7f77e7400700 1 -- 172.21.15.18:0/3948036812 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77d8005040 con 0x7f77e8109c40 2024-04-12T21:01:21.162 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:21.165+0000 7f77e7400700 1 --2- 172.21.15.18:0/3948036812 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f77e8109c40 0x7f77e811a530 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f77d400c890 tx=0x7f77d40044c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:01:21.162 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:21.165+0000 7f77dfe00700 1 -- 172.21.15.18:0/3948036812 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f77d400a7d0 con 0x7f77e8109c40 2024-04-12T21:01:21.162 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:21.165+0000 7f77dfe00700 1 -- 172.21.15.18:0/3948036812 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f77d400ae00 con 0x7f77e8109c40 2024-04-12T21:01:21.162 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:21.165+0000 7f77dfe00700 1 -- 172.21.15.18:0/3948036812 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f77d40113a0 con 0x7f77e8109c40 2024-04-12T21:01:21.162 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:21.165+0000 7f77ed600700 1 -- 172.21.15.18:0/3948036812 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f77e8114230 con 0x7f77e8109c40 2024-04-12T21:01:21.162 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:21.165+0000 7f77ed600700 1 -- 172.21.15.18:0/3948036812 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f77e81147f0 con 0x7f77e8109c40 2024-04-12T21:01:21.163 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:21.166+0000 7f77ed600700 1 -- 172.21.15.18:0/3948036812 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f77e81049f0 con 0x7f77e8109c40 2024-04-12T21:01:21.164 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:21.168+0000 7f77dfe00700 1 -- 172.21.15.18:0/3948036812 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 32) v1 ==== 92459+0+0 (secure 0 0 0) 0x7f77d4012400 con 0x7f77e8109c40 2024-04-12T21:01:21.165 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:21.169+0000 7f77dfe00700 1 --2- 172.21.15.18:0/3948036812 >> [v2:172.21.15.18:6800/1605950634,v1:172.21.15.18:6801/1605950634] conn(0x7f77cc06e500 0x7f77cc0709b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:01:21.165 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:21.169+0000 7f77dfe00700 1 -- 172.21.15.18:0/3948036812 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6066+0+0 (secure 0 0 0) 0x7f77d408df80 con 0x7f77e8109c40 2024-04-12T21:01:21.166 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:21.169+0000 7f77e7e00700 1 --2- 172.21.15.18:0/3948036812 >> [v2:172.21.15.18:6800/1605950634,v1:172.21.15.18:6801/1605950634] conn(0x7f77cc06e500 0x7f77cc0709b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:01:21.166 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:21.170+0000 7f77e7e00700 1 --2- 172.21.15.18:0/3948036812 >> [v2:172.21.15.18:6800/1605950634,v1:172.21.15.18:6801/1605950634] conn(0x7f77cc06e500 0x7f77cc0709b0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f77d8004f90 tx=0x7f77d8009e30 comp rx=0 tx=0).ready entity=mgr.14672 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:01:21.170 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:21.174+0000 7f77dfe00700 1 -- 172.21.15.18:0/3948036812 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186454 (secure 0 0 0) 0x7f77d405b160 con 0x7f77e8109c40 2024-04-12T21:01:21.500 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:21.504+0000 7f77ed600700 1 -- 172.21.15.18:0/3948036812 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f77e8076560 con 0x7f77e8109c40 2024-04-12T21:01:21.501 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:21.505+0000 7f77dfe00700 1 -- 172.21.15.18:0/3948036812 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+739 (secure 0 0 0) 0x7f77d405acf0 con 0x7f77e8109c40 2024-04-12T21:01:21.501 INFO:teuthology.orchestra.run.smithi018.stdout:{ 2024-04-12T21:01:21.501 INFO:teuthology.orchestra.run.smithi018.stdout: "mon": { 2024-04-12T21:01:21.502 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 17.2.7-827-gd3b5060a (d3b5060a17449edfae9385cdf792afec109fca5d) quincy (stable)": 2 2024-04-12T21:01:21.502 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-12T21:01:21.502 INFO:teuthology.orchestra.run.smithi018.stdout: "mgr": { 2024-04-12T21:01:21.502 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 19.0.0-2879-g324baf8d (324baf8ddb91dea46cac50cb535c306115791355) squid (dev)": 2 2024-04-12T21:01:21.502 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-12T21:01:21.502 INFO:teuthology.orchestra.run.smithi018.stdout: "osd": { 2024-04-12T21:01:21.502 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 17.2.7-827-gd3b5060a (d3b5060a17449edfae9385cdf792afec109fca5d) quincy (stable)": 6 2024-04-12T21:01:21.502 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-12T21:01:21.502 INFO:teuthology.orchestra.run.smithi018.stdout: "mds": { 2024-04-12T21:01:21.502 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 17.2.7-827-gd3b5060a (d3b5060a17449edfae9385cdf792afec109fca5d) quincy (stable)": 4 2024-04-12T21:01:21.502 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-12T21:01:21.502 INFO:teuthology.orchestra.run.smithi018.stdout: "overall": { 2024-04-12T21:01:21.502 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 17.2.7-827-gd3b5060a (d3b5060a17449edfae9385cdf792afec109fca5d) quincy (stable)": 12, 2024-04-12T21:01:21.503 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 19.0.0-2879-g324baf8d (324baf8ddb91dea46cac50cb535c306115791355) squid (dev)": 2 2024-04-12T21:01:21.503 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-04-12T21:01:21.503 INFO:teuthology.orchestra.run.smithi018.stdout:} 2024-04-12T21:01:21.505 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:21.508+0000 7f77ed600700 1 -- 172.21.15.18:0/3948036812 >> [v2:172.21.15.18:6800/1605950634,v1:172.21.15.18:6801/1605950634] conn(0x7f77cc06e500 msgr2=0x7f77cc0709b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:01:21.505 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:21.508+0000 7f77ed600700 1 --2- 172.21.15.18:0/3948036812 >> [v2:172.21.15.18:6800/1605950634,v1:172.21.15.18:6801/1605950634] conn(0x7f77cc06e500 0x7f77cc0709b0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f77d8004f90 tx=0x7f77d8009e30 comp rx=0 tx=0).stop 2024-04-12T21:01:21.505 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:21.508+0000 7f77ed600700 1 -- 172.21.15.18:0/3948036812 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f77e8109c40 msgr2=0x7f77e811a530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:01:21.505 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:21.508+0000 7f77ed600700 1 --2- 172.21.15.18:0/3948036812 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f77e8109c40 0x7f77e811a530 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f77d400c890 tx=0x7f77d40044c0 comp rx=0 tx=0).stop 2024-04-12T21:01:21.505 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:21.508+0000 7f77ed600700 1 -- 172.21.15.18:0/3948036812 shutdown_connections 2024-04-12T21:01:21.505 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:21.508+0000 7f77ed600700 1 --2- 172.21.15.18:0/3948036812 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f77e81092a0 0x7f77e8119ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:01:21.505 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:21.508+0000 7f77ed600700 1 --2- 172.21.15.18:0/3948036812 >> [v2:172.21.15.18:6800/1605950634,v1:172.21.15.18:6801/1605950634] conn(0x7f77cc06e500 0x7f77cc0709b0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:01:21.505 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:21.508+0000 7f77ed600700 1 --2- 172.21.15.18:0/3948036812 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f77e8109c40 0x7f77e811a530 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:01:21.505 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:21.508+0000 7f77ed600700 1 -- 172.21.15.18:0/3948036812 >> 172.21.15.18:0/3948036812 conn(0x7f77e8072a30 msgr2=0x7f77e80748d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:01:21.505 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:21.508+0000 7f77ed600700 1 -- 172.21.15.18:0/3948036812 shutdown_connections 2024-04-12T21:01:21.505 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:21.508+0000 7f77ed600700 1 -- 172.21.15.18:0/3948036812 wait complete. 2024-04-12T21:01:21.663 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:21.666+0000 7f967d800700 1 -- 172.21.15.18:0/1384873368 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f96781059c0 msgr2=0x7f96780ff080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:01:21.663 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:21.666+0000 7f967d800700 1 --2- 172.21.15.18:0/1384873368 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f96781059c0 0x7f96780ff080 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f9664004940 tx=0x7f9664009e30 comp rx=0 tx=0).stop 2024-04-12T21:01:21.663 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:21.666+0000 7f967d800700 1 -- 172.21.15.18:0/1384873368 shutdown_connections 2024-04-12T21:01:21.663 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:21.666+0000 7f967d800700 1 --2- 172.21.15.18:0/1384873368 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f96781059c0 0x7f96780ff080 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:01:21.663 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:21.666+0000 7f967d800700 1 --2- 172.21.15.18:0/1384873368 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9678105020 0x7f96781053f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:01:21.663 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:21.666+0000 7f967d800700 1 -- 172.21.15.18:0/1384873368 >> 172.21.15.18:0/1384873368 conn(0x7f96780faba0 msgr2=0x7f96780fcfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:01:21.664 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:21.666+0000 7f967d800700 1 -- 172.21.15.18:0/1384873368 shutdown_connections 2024-04-12T21:01:21.664 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:21.666+0000 7f967d800700 1 -- 172.21.15.18:0/1384873368 wait complete. 2024-04-12T21:01:21.664 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:21.667+0000 7f967d800700 1 Processor -- start 2024-04-12T21:01:21.664 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:21.667+0000 7f967d800700 1 -- start start 2024-04-12T21:01:21.664 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:21.668+0000 7f967d800700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9678105020 0x7f9678115b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:01:21.664 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:21.668+0000 7f967d800700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f96781059c0 0x7f9678116050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:01:21.664 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:21.668+0000 7f967d800700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9678116730 con 0x7f96781059c0 2024-04-12T21:01:21.665 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:21.668+0000 7f967d800700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f967810fcc0 con 0x7f9678105020 2024-04-12T21:01:21.665 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:21.668+0000 7f9676000700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f96781059c0 0x7f9678116050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:01:21.665 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:21.668+0000 7f9676000700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f96781059c0 0x7f9678116050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:33706/0 (socket says 172.21.15.18:33706) 2024-04-12T21:01:21.666 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:21.668+0000 7f9676000700 1 -- 172.21.15.18:0/1297276897 learned_addr learned my addr 172.21.15.18:0/1297276897 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:01:21.666 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:21.669+0000 7f9676a00700 1 --2- 172.21.15.18:0/1297276897 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9678105020 0x7f9678115b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:01:21.666 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:21.669+0000 7f9676000700 1 -- 172.21.15.18:0/1297276897 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9678105020 msgr2=0x7f9678115b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:01:21.666 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:21.669+0000 7f9676000700 1 --2- 172.21.15.18:0/1297276897 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9678105020 0x7f9678115b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:01:21.666 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:21.669+0000 7f9676000700 1 -- 172.21.15.18:0/1297276897 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9664005040 con 0x7f96781059c0 2024-04-12T21:01:21.666 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:21.669+0000 7f9676000700 1 --2- 172.21.15.18:0/1297276897 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f96781059c0 0x7f9678116050 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f9664009100 tx=0x7f9664009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:01:21.667 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:21.669+0000 7f966ea00700 1 -- 172.21.15.18:0/1297276897 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f966400e040 con 0x7f96781059c0 2024-04-12T21:01:21.667 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:21.670+0000 7f966ea00700 1 -- 172.21.15.18:0/1297276897 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f9664004290 con 0x7f96781059c0 2024-04-12T21:01:21.667 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:21.670+0000 7f967d800700 1 -- 172.21.15.18:0/1297276897 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f967810ff40 con 0x7f96781059c0 2024-04-12T21:01:21.667 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:21.670+0000 7f966ea00700 1 -- 172.21.15.18:0/1297276897 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f966401c3f0 con 0x7f96781059c0 2024-04-12T21:01:21.667 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:21.670+0000 7f967d800700 1 -- 172.21.15.18:0/1297276897 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9678110500 con 0x7f96781059c0 2024-04-12T21:01:21.667 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:21.671+0000 7f967d800700 1 -- 172.21.15.18:0/1297276897 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9678100770 con 0x7f96781059c0 2024-04-12T21:01:21.668 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:21.672+0000 7f966ea00700 1 -- 172.21.15.18:0/1297276897 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 32) v1 ==== 92459+0+0 (secure 0 0 0) 0x7f9664013790 con 0x7f96781059c0 2024-04-12T21:01:21.670 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:21.673+0000 7f966ea00700 1 --2- 172.21.15.18:0/1297276897 >> [v2:172.21.15.18:6800/1605950634,v1:172.21.15.18:6801/1605950634] conn(0x7f965c06e500 0x7f965c0709b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:01:21.670 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:21.673+0000 7f966ea00700 1 -- 172.21.15.18:0/1297276897 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6066+0+0 (secure 0 0 0) 0x7f966408fb60 con 0x7f96781059c0 2024-04-12T21:01:21.670 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:21.674+0000 7f9676a00700 1 --2- 172.21.15.18:0/1297276897 >> [v2:172.21.15.18:6800/1605950634,v1:172.21.15.18:6801/1605950634] conn(0x7f965c06e500 0x7f965c0709b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:01:21.670 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:21.674+0000 7f9676a00700 1 --2- 172.21.15.18:0/1297276897 >> [v2:172.21.15.18:6800/1605950634,v1:172.21.15.18:6801/1605950634] conn(0x7f965c06e500 0x7f965c0709b0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f9668001250 tx=0x7f9668009620 comp rx=0 tx=0).ready entity=mgr.14672 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:01:21.674 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:21.677+0000 7f966ea00700 1 -- 172.21.15.18:0/1297276897 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186454 (secure 0 0 0) 0x7f96640592d0 con 0x7f96781059c0 2024-04-12T21:01:21.851 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:01:21 smithi018 ceph-mon[28799]: from='client.14870 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:01:21.851 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:01:21 smithi018 ceph-mon[28799]: from='client.14874 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:01:21.851 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:01:21 smithi018 ceph-mon[28799]: pgmap v127: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-12T21:01:21.851 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:01:21 smithi018 ceph-mon[28799]: from='client.? 172.21.15.18:0/3948036812' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T21:01:21.851 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:01:21 smithi018 ceph-mon[28799]: from='mgr.14672 172.21.15.18:0/2209228970' entity='mgr.smithi018.xjzwga' 2024-04-12T21:01:21.852 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:01:21 smithi018 ceph-mon[28799]: from='mgr.14672 172.21.15.18:0/2209228970' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-12T21:01:21.852 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:01:21 smithi018 ceph-mon[28799]: from='mgr.14672 172.21.15.18:0/2209228970' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T21:01:21.852 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:01:21 smithi018 ceph-mon[28799]: from='mgr.14672 172.21.15.18:0/2209228970' entity='mgr.smithi018.xjzwga' 2024-04-12T21:01:21.852 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:01:21 smithi018 ceph-mon[28799]: from='mgr.14672 172.21.15.18:0/2209228970' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "quorum_status"}]: dispatch 2024-04-12T21:01:21.951 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:21.954+0000 7f967d800700 1 -- 172.21.15.18:0/1297276897 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f967804c710 con 0x7f96781059c0 2024-04-12T21:01:21.951 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:21.955+0000 7f966ea00700 1 -- 172.21.15.18:0/1297276897 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 23 v23) v1 ==== 76+0+1816 (secure 0 0 0) 0x7f9664013420 con 0x7f96781059c0 2024-04-12T21:01:21.952 INFO:teuthology.orchestra.run.smithi018.stdout:e23 2024-04-12T21:01:21.952 INFO:teuthology.orchestra.run.smithi018.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-12T21:01:21.952 INFO:teuthology.orchestra.run.smithi018.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-12T21:01:21.952 INFO:teuthology.orchestra.run.smithi018.stdout:legacy client fscid: 1 2024-04-12T21:01:21.952 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-12T21:01:21.952 INFO:teuthology.orchestra.run.smithi018.stdout:Filesystem 'cephfs' (1) 2024-04-12T21:01:21.952 INFO:teuthology.orchestra.run.smithi018.stdout:fs_name cephfs 2024-04-12T21:01:21.952 INFO:teuthology.orchestra.run.smithi018.stdout:epoch 23 2024-04-12T21:01:21.952 INFO:teuthology.orchestra.run.smithi018.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-04-12T21:01:21.952 INFO:teuthology.orchestra.run.smithi018.stdout:created 2024-04-12T20:51:57.362850+0000 2024-04-12T21:01:21.953 INFO:teuthology.orchestra.run.smithi018.stdout:modified 2024-04-12T21:01:10.670287+0000 2024-04-12T21:01:21.953 INFO:teuthology.orchestra.run.smithi018.stdout:tableserver 0 2024-04-12T21:01:21.953 INFO:teuthology.orchestra.run.smithi018.stdout:root 0 2024-04-12T21:01:21.953 INFO:teuthology.orchestra.run.smithi018.stdout:session_timeout 60 2024-04-12T21:01:21.953 INFO:teuthology.orchestra.run.smithi018.stdout:session_autoclose 300 2024-04-12T21:01:21.953 INFO:teuthology.orchestra.run.smithi018.stdout:max_file_size 1099511627776 2024-04-12T21:01:21.953 INFO:teuthology.orchestra.run.smithi018.stdout:required_client_features {} 2024-04-12T21:01:21.953 INFO:teuthology.orchestra.run.smithi018.stdout:last_failure 0 2024-04-12T21:01:21.953 INFO:teuthology.orchestra.run.smithi018.stdout:last_failure_osd_epoch 46 2024-04-12T21:01:21.953 INFO:teuthology.orchestra.run.smithi018.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-12T21:01:21.953 INFO:teuthology.orchestra.run.smithi018.stdout:max_mds 2 2024-04-12T21:01:21.953 INFO:teuthology.orchestra.run.smithi018.stdout:in 0,1 2024-04-12T21:01:21.953 INFO:teuthology.orchestra.run.smithi018.stdout:up {0=14452,1=14464} 2024-04-12T21:01:21.953 INFO:teuthology.orchestra.run.smithi018.stdout:failed 2024-04-12T21:01:21.953 INFO:teuthology.orchestra.run.smithi018.stdout:damaged 2024-04-12T21:01:21.953 INFO:teuthology.orchestra.run.smithi018.stdout:stopped 2024-04-12T21:01:21.954 INFO:teuthology.orchestra.run.smithi018.stdout:data_pools [3] 2024-04-12T21:01:21.954 INFO:teuthology.orchestra.run.smithi018.stdout:metadata_pool 2 2024-04-12T21:01:21.954 INFO:teuthology.orchestra.run.smithi018.stdout:inline_data disabled 2024-04-12T21:01:21.954 INFO:teuthology.orchestra.run.smithi018.stdout:balancer 2024-04-12T21:01:21.954 INFO:teuthology.orchestra.run.smithi018.stdout:standby_count_wanted 1 2024-04-12T21:01:21.954 INFO:teuthology.orchestra.run.smithi018.stdout:[mds.cephfs.smithi018.acqbsc{0:14452} state up:active seq 5 join_fscid=1 addr [v2:172.21.15.18:6826/1564065434,v1:172.21.15.18:6827/1564065434] compat {c=[1],r=[1],i=[7ff]}] 2024-04-12T21:01:21.954 INFO:teuthology.orchestra.run.smithi018.stdout:[mds.cephfs.smithi018.szjluo{1:14464} state up:active seq 6 join_fscid=1 addr [v2:172.21.15.18:6828/722443353,v1:172.21.15.18:6829/722443353] compat {c=[1],r=[1],i=[7ff]}] 2024-04-12T21:01:21.954 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-12T21:01:21.954 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-12T21:01:21.954 INFO:teuthology.orchestra.run.smithi018.stdout:Standby daemons: 2024-04-12T21:01:21.954 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-12T21:01:21.954 INFO:teuthology.orchestra.run.smithi018.stdout:[mds.cephfs.smithi186.wjnafs{-1:24277} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.186:6826/438347207,v1:172.21.15.186:6827/438347207] compat {c=[1],r=[1],i=[7ff]}] 2024-04-12T21:01:21.955 INFO:teuthology.orchestra.run.smithi018.stdout:[mds.cephfs.smithi186.jdllyn{-1:24281} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.186:6824/1464359952,v1:172.21.15.186:6825/1464359952] compat {c=[1],r=[1],i=[7ff]}] 2024-04-12T21:01:21.955 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:21.958+0000 7f967d800700 1 -- 172.21.15.18:0/1297276897 >> [v2:172.21.15.18:6800/1605950634,v1:172.21.15.18:6801/1605950634] conn(0x7f965c06e500 msgr2=0x7f965c0709b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:01:21.955 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:21.958+0000 7f967d800700 1 --2- 172.21.15.18:0/1297276897 >> [v2:172.21.15.18:6800/1605950634,v1:172.21.15.18:6801/1605950634] conn(0x7f965c06e500 0x7f965c0709b0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f9668001250 tx=0x7f9668009620 comp rx=0 tx=0).stop 2024-04-12T21:01:21.956 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:21.958+0000 7f967d800700 1 -- 172.21.15.18:0/1297276897 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f96781059c0 msgr2=0x7f9678116050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:01:21.956 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:21.958+0000 7f967d800700 1 --2- 172.21.15.18:0/1297276897 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f96781059c0 0x7f9678116050 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f9664009100 tx=0x7f9664009e30 comp rx=0 tx=0).stop 2024-04-12T21:01:21.956 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:21.958+0000 7f967d800700 1 -- 172.21.15.18:0/1297276897 shutdown_connections 2024-04-12T21:01:21.956 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:21.958+0000 7f967d800700 1 --2- 172.21.15.18:0/1297276897 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9678105020 0x7f9678115b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:01:21.956 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:21.958+0000 7f967d800700 1 --2- 172.21.15.18:0/1297276897 >> [v2:172.21.15.18:6800/1605950634,v1:172.21.15.18:6801/1605950634] conn(0x7f965c06e500 0x7f965c0709b0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:01:21.957 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:21.958+0000 7f967d800700 1 --2- 172.21.15.18:0/1297276897 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f96781059c0 0x7f9678116050 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:01:21.957 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:21.958+0000 7f967d800700 1 -- 172.21.15.18:0/1297276897 >> 172.21.15.18:0/1297276897 conn(0x7f96780faba0 msgr2=0x7f96780fbaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:01:21.957 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:21.958+0000 7f967d800700 1 -- 172.21.15.18:0/1297276897 shutdown_connections 2024-04-12T21:01:21.957 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:21.958+0000 7f967d800700 1 -- 172.21.15.18:0/1297276897 wait complete. 2024-04-12T21:01:21.957 INFO:teuthology.orchestra.run.smithi018.stderr:dumped fsmap epoch 23 2024-04-12T21:01:22.030 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:01:21 smithi186 ceph-mon[36597]: from='client.14870 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:01:22.030 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:01:21 smithi186 ceph-mon[36597]: from='client.14874 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:01:22.030 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:01:21 smithi186 ceph-mon[36597]: pgmap v127: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-12T21:01:22.030 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:01:21 smithi186 ceph-mon[36597]: from='client.? 172.21.15.18:0/3948036812' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T21:01:22.030 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:01:21 smithi186 ceph-mon[36597]: from='mgr.14672 172.21.15.18:0/2209228970' entity='mgr.smithi018.xjzwga' 2024-04-12T21:01:22.030 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:01:21 smithi186 ceph-mon[36597]: from='mgr.14672 172.21.15.18:0/2209228970' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-12T21:01:22.030 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:01:21 smithi186 ceph-mon[36597]: from='mgr.14672 172.21.15.18:0/2209228970' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T21:01:22.030 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:01:21 smithi186 ceph-mon[36597]: from='mgr.14672 172.21.15.18:0/2209228970' entity='mgr.smithi018.xjzwga' 2024-04-12T21:01:22.030 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:01:21 smithi186 ceph-mon[36597]: from='mgr.14672 172.21.15.18:0/2209228970' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "quorum_status"}]: dispatch 2024-04-12T21:01:22.106 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:22.109+0000 7f93acc00700 1 -- 172.21.15.18:0/2015166275 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f93a8102e00 msgr2=0x7f93a81031d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:01:22.106 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:22.109+0000 7f93acc00700 1 --2- 172.21.15.18:0/2015166275 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f93a8102e00 0x7f93a81031d0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f9398004940 tx=0x7f9398009e30 comp rx=0 tx=0).stop 2024-04-12T21:01:22.106 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:22.109+0000 7f93acc00700 1 -- 172.21.15.18:0/2015166275 shutdown_connections 2024-04-12T21:01:22.107 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:22.109+0000 7f93acc00700 1 --2- 172.21.15.18:0/2015166275 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f93a81037a0 0x7f93a80fce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:01:22.107 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:22.109+0000 7f93acc00700 1 --2- 172.21.15.18:0/2015166275 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f93a8102e00 0x7f93a81031d0 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:01:22.107 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:22.109+0000 7f93acc00700 1 -- 172.21.15.18:0/2015166275 >> 172.21.15.18:0/2015166275 conn(0x7f93a80f89b0 msgr2=0x7f93a80fadc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:01:22.107 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:22.109+0000 7f93acc00700 1 -- 172.21.15.18:0/2015166275 shutdown_connections 2024-04-12T21:01:22.107 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:22.109+0000 7f93acc00700 1 -- 172.21.15.18:0/2015166275 wait complete. 2024-04-12T21:01:22.107 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:22.110+0000 7f93acc00700 1 Processor -- start 2024-04-12T21:01:22.107 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:22.110+0000 7f93acc00700 1 -- start start 2024-04-12T21:01:22.108 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:22.110+0000 7f93acc00700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f93a8102e00 0x7f93a8115dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:01:22.108 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:22.110+0000 7f93acc00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f93a81037a0 0x7f93a8116300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:01:22.108 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:22.110+0000 7f93acc00700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93a8116950 con 0x7f93a81037a0 2024-04-12T21:01:22.109 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:22.110+0000 7f93acc00700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93a810fd20 con 0x7f93a8102e00 2024-04-12T21:01:22.109 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:22.111+0000 7f93a5400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f93a81037a0 0x7f93a8116300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:01:22.109 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:22.111+0000 7f93a5400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f93a81037a0 0x7f93a8116300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:33724/0 (socket says 172.21.15.18:33724) 2024-04-12T21:01:22.109 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:22.111+0000 7f93a5400700 1 -- 172.21.15.18:0/2547519884 learned_addr learned my addr 172.21.15.18:0/2547519884 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:01:22.109 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:22.111+0000 7f93a5e00700 1 --2- 172.21.15.18:0/2547519884 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f93a8102e00 0x7f93a8115dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:01:22.109 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:22.111+0000 7f93a5400700 1 -- 172.21.15.18:0/2547519884 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f93a8102e00 msgr2=0x7f93a8115dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:01:22.109 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:22.111+0000 7f93a5400700 1 --2- 172.21.15.18:0/2547519884 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f93a8102e00 0x7f93a8115dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:01:22.109 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:22.111+0000 7f93a5400700 1 -- 172.21.15.18:0/2547519884 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9398005040 con 0x7f93a81037a0 2024-04-12T21:01:22.109 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:22.111+0000 7f93a5400700 1 --2- 172.21.15.18:0/2547519884 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f93a81037a0 0x7f93a8116300 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f939400c800 tx=0x7f9394004470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:01:22.109 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:22.112+0000 7f939e000700 1 -- 172.21.15.18:0/2547519884 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f939400a760 con 0x7f93a81037a0 2024-04-12T21:01:22.109 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:22.112+0000 7f93acc00700 1 -- 172.21.15.18:0/2547519884 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f93a8110000 con 0x7f93a81037a0 2024-04-12T21:01:22.109 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:22.112+0000 7f939e000700 1 -- 172.21.15.18:0/2547519884 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f939400ad90 con 0x7f93a81037a0 2024-04-12T21:01:22.109 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:22.112+0000 7f939e000700 1 -- 172.21.15.18:0/2547519884 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f93940113a0 con 0x7f93a81037a0 2024-04-12T21:01:22.109 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:22.112+0000 7f93acc00700 1 -- 172.21.15.18:0/2547519884 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f93a81105c0 con 0x7f93a81037a0 2024-04-12T21:01:22.110 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:22.113+0000 7f93acc00700 1 -- 172.21.15.18:0/2547519884 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f93a8106a50 con 0x7f93a81037a0 2024-04-12T21:01:22.110 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:22.114+0000 7f939e000700 1 -- 172.21.15.18:0/2547519884 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 32) v1 ==== 92459+0+0 (secure 0 0 0) 0x7f9394012400 con 0x7f93a81037a0 2024-04-12T21:01:22.111 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:22.115+0000 7f939e000700 1 --2- 172.21.15.18:0/2547519884 >> [v2:172.21.15.18:6800/1605950634,v1:172.21.15.18:6801/1605950634] conn(0x7f938c06e510 0x7f938c0709c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:01:22.111 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:22.115+0000 7f939e000700 1 -- 172.21.15.18:0/2547519884 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6066+0+0 (secure 0 0 0) 0x7f939408de90 con 0x7f93a81037a0 2024-04-12T21:01:22.112 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:22.115+0000 7f93a5e00700 1 --2- 172.21.15.18:0/2547519884 >> [v2:172.21.15.18:6800/1605950634,v1:172.21.15.18:6801/1605950634] conn(0x7f938c06e510 0x7f938c0709c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:01:22.112 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:22.116+0000 7f93a5e00700 1 --2- 172.21.15.18:0/2547519884 >> [v2:172.21.15.18:6800/1605950634,v1:172.21.15.18:6801/1605950634] conn(0x7f938c06e510 0x7f938c0709c0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f939800b020 tx=0x7f9398009e30 comp rx=0 tx=0).ready entity=mgr.14672 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:01:22.116 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:22.120+0000 7f939e000700 1 -- 172.21.15.18:0/2547519884 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186454 (secure 0 0 0) 0x7f939405b070 con 0x7f93a81037a0 2024-04-12T21:01:22.372 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:22.375+0000 7f93acc00700 1 -- 172.21.15.18:0/2547519884 --> [v2:172.21.15.18:6800/1605950634,v1:172.21.15.18:6801/1605950634] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f93a8111480 con 0x7f938c06e510 2024-04-12T21:01:22.374 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:22.377+0000 7f939e000700 1 -- 172.21.15.18:0/2547519884 <== mgr.14672 v2:172.21.15.18:6800/1605950634 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+368 (secure 0 0 0) 0x7f93a8111480 con 0x7f938c06e510 2024-04-12T21:01:22.374 INFO:teuthology.orchestra.run.smithi018.stdout:{ 2024-04-12T21:01:22.374 INFO:teuthology.orchestra.run.smithi018.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:f4452def3be4d350f4a8e5feaf7ebb1511be9298fe68478ba7a2d6fbd7d69a63", 2024-04-12T21:01:22.374 INFO:teuthology.orchestra.run.smithi018.stdout: "in_progress": true, 2024-04-12T21:01:22.374 INFO:teuthology.orchestra.run.smithi018.stdout: "which": "Upgrading all daemon types on all hosts", 2024-04-12T21:01:22.374 INFO:teuthology.orchestra.run.smithi018.stdout: "services_complete": [ 2024-04-12T21:01:22.374 INFO:teuthology.orchestra.run.smithi018.stdout: "mgr" 2024-04-12T21:01:22.374 INFO:teuthology.orchestra.run.smithi018.stdout: ], 2024-04-12T21:01:22.374 INFO:teuthology.orchestra.run.smithi018.stdout: "progress": "2/21 daemons upgraded", 2024-04-12T21:01:22.374 INFO:teuthology.orchestra.run.smithi018.stdout: "message": "Currently upgrading mon daemons", 2024-04-12T21:01:22.375 INFO:teuthology.orchestra.run.smithi018.stdout: "is_paused": false 2024-04-12T21:01:22.375 INFO:teuthology.orchestra.run.smithi018.stdout:} 2024-04-12T21:01:22.376 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:22.380+0000 7f93acc00700 1 -- 172.21.15.18:0/2547519884 >> [v2:172.21.15.18:6800/1605950634,v1:172.21.15.18:6801/1605950634] conn(0x7f938c06e510 msgr2=0x7f938c0709c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:01:22.377 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:22.380+0000 7f93acc00700 1 --2- 172.21.15.18:0/2547519884 >> [v2:172.21.15.18:6800/1605950634,v1:172.21.15.18:6801/1605950634] conn(0x7f938c06e510 0x7f938c0709c0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f939800b020 tx=0x7f9398009e30 comp rx=0 tx=0).stop 2024-04-12T21:01:22.377 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:22.380+0000 7f93acc00700 1 -- 172.21.15.18:0/2547519884 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f93a81037a0 msgr2=0x7f93a8116300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:01:22.377 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:22.380+0000 7f93acc00700 1 --2- 172.21.15.18:0/2547519884 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f93a81037a0 0x7f93a8116300 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f939400c800 tx=0x7f9394004470 comp rx=0 tx=0).stop 2024-04-12T21:01:22.377 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:22.380+0000 7f93acc00700 1 -- 172.21.15.18:0/2547519884 shutdown_connections 2024-04-12T21:01:22.377 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:22.380+0000 7f93acc00700 1 --2- 172.21.15.18:0/2547519884 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f93a8102e00 0x7f93a8115dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:01:22.377 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:22.380+0000 7f93acc00700 1 --2- 172.21.15.18:0/2547519884 >> [v2:172.21.15.18:6800/1605950634,v1:172.21.15.18:6801/1605950634] conn(0x7f938c06e510 0x7f938c0709c0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:01:22.377 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:22.380+0000 7f93acc00700 1 --2- 172.21.15.18:0/2547519884 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f93a81037a0 0x7f93a8116300 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:01:22.377 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:22.380+0000 7f93acc00700 1 -- 172.21.15.18:0/2547519884 >> 172.21.15.18:0/2547519884 conn(0x7f93a80f89b0 msgr2=0x7f93a81073a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:01:22.378 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:22.381+0000 7f93acc00700 1 -- 172.21.15.18:0/2547519884 shutdown_connections 2024-04-12T21:01:22.378 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:22.381+0000 7f93acc00700 1 -- 172.21.15.18:0/2547519884 wait complete. 2024-04-12T21:01:22.521 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:22.524+0000 7ff0a2200700 1 -- 172.21.15.18:0/747024648 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff09c1035d0 msgr2=0x7ff09c0fcc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:01:22.521 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:22.524+0000 7ff0a2200700 1 --2- 172.21.15.18:0/747024648 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff09c1035d0 0x7ff09c0fcc40 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7ff088004940 tx=0x7ff088009e30 comp rx=0 tx=0).stop 2024-04-12T21:01:22.521 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:22.524+0000 7ff0a2200700 1 -- 172.21.15.18:0/747024648 shutdown_connections 2024-04-12T21:01:22.521 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:22.524+0000 7ff0a2200700 1 --2- 172.21.15.18:0/747024648 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff09c1035d0 0x7ff09c0fcc40 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:01:22.521 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:22.524+0000 7ff0a2200700 1 --2- 172.21.15.18:0/747024648 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff09c102c30 0x7ff09c103000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:01:22.521 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:22.524+0000 7ff0a2200700 1 -- 172.21.15.18:0/747024648 >> 172.21.15.18:0/747024648 conn(0x7ff09c0f87a0 msgr2=0x7ff09c0fabb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:01:22.521 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:22.524+0000 7ff0a2200700 1 -- 172.21.15.18:0/747024648 shutdown_connections 2024-04-12T21:01:22.521 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:22.524+0000 7ff0a2200700 1 -- 172.21.15.18:0/747024648 wait complete. 2024-04-12T21:01:22.523 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:22.525+0000 7ff0a2200700 1 Processor -- start 2024-04-12T21:01:22.523 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:22.525+0000 7ff0a2200700 1 -- start start 2024-04-12T21:01:22.523 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:22.526+0000 7ff0a2200700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff09c102c30 0x7ff09c1139c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:01:22.523 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:22.526+0000 7ff0a2200700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff09c1035d0 0x7ff09c113f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:01:22.523 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:22.526+0000 7ff0a2200700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff09c114550 con 0x7ff09c102c30 2024-04-12T21:01:22.523 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:22.526+0000 7ff0a2200700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff09c10d940 con 0x7ff09c1035d0 2024-04-12T21:01:22.523 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:22.526+0000 7ff0a0e00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff09c102c30 0x7ff09c1139c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:01:22.523 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:22.526+0000 7ff0a0e00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff09c102c30 0x7ff09c1139c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:33750/0 (socket says 172.21.15.18:33750) 2024-04-12T21:01:22.523 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:22.526+0000 7ff0a0e00700 1 -- 172.21.15.18:0/657214515 learned_addr learned my addr 172.21.15.18:0/657214515 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:01:22.523 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:22.526+0000 7ff0a0e00700 1 -- 172.21.15.18:0/657214515 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff09c1035d0 msgr2=0x7ff09c113f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:01:22.524 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:22.526+0000 7ff09be00700 1 --2- 172.21.15.18:0/657214515 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff09c1035d0 0x7ff09c113f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:01:22.524 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:22.526+0000 7ff0a0e00700 1 --2- 172.21.15.18:0/657214515 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff09c1035d0 0x7ff09c113f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:01:22.524 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:22.526+0000 7ff0a0e00700 1 -- 172.21.15.18:0/657214515 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff088005040 con 0x7ff09c102c30 2024-04-12T21:01:22.524 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:22.526+0000 7ff09be00700 1 --2- 172.21.15.18:0/657214515 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff09c1035d0 0x7ff09c113f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-12T21:01:22.524 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:22.527+0000 7ff0a0e00700 1 --2- 172.21.15.18:0/657214515 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff09c102c30 0x7ff09c1139c0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7ff090002a50 tx=0x7ff0900046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:01:22.525 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:22.527+0000 7ff098c00700 1 -- 172.21.15.18:0/657214515 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff090007cc0 con 0x7ff09c102c30 2024-04-12T21:01:22.525 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:22.527+0000 7ff098c00700 1 -- 172.21.15.18:0/657214515 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff09000f040 con 0x7ff09c102c30 2024-04-12T21:01:22.525 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:22.527+0000 7ff0a2200700 1 -- 172.21.15.18:0/657214515 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff09c10dc20 con 0x7ff09c102c30 2024-04-12T21:01:22.525 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:22.527+0000 7ff098c00700 1 -- 172.21.15.18:0/657214515 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff09000db70 con 0x7ff09c102c30 2024-04-12T21:01:22.525 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:22.527+0000 7ff0a2200700 1 -- 172.21.15.18:0/657214515 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff09c10e1e0 con 0x7ff09c102c30 2024-04-12T21:01:22.525 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:22.528+0000 7ff0a2200700 1 -- 172.21.15.18:0/657214515 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff09c0fe380 con 0x7ff09c102c30 2024-04-12T21:01:22.525 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:22.529+0000 7ff098c00700 1 -- 172.21.15.18:0/657214515 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 32) v1 ==== 92459+0+0 (secure 0 0 0) 0x7ff09000dd10 con 0x7ff09c102c30 2024-04-12T21:01:22.526 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:22.530+0000 7ff098c00700 1 --2- 172.21.15.18:0/657214515 >> [v2:172.21.15.18:6800/1605950634,v1:172.21.15.18:6801/1605950634] conn(0x7ff08406e500 0x7ff0840709b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:01:22.527 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:22.530+0000 7ff098c00700 1 -- 172.21.15.18:0/657214515 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6066+0+0 (secure 0 0 0) 0x7ff09008d580 con 0x7ff09c102c30 2024-04-12T21:01:22.527 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:22.530+0000 7ff09be00700 1 --2- 172.21.15.18:0/657214515 >> [v2:172.21.15.18:6800/1605950634,v1:172.21.15.18:6801/1605950634] conn(0x7ff08406e500 0x7ff0840709b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:01:22.527 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:22.531+0000 7ff09be00700 1 --2- 172.21.15.18:0/657214515 >> [v2:172.21.15.18:6800/1605950634,v1:172.21.15.18:6801/1605950634] conn(0x7ff08406e500 0x7ff0840709b0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7ff08800b3b0 tx=0x7ff088009e30 comp rx=0 tx=0).ready entity=mgr.14672 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:01:22.531 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:22.535+0000 7ff098c00700 1 -- 172.21.15.18:0/657214515 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186454 (secure 0 0 0) 0x7ff09005a6e0 con 0x7ff09c102c30 2024-04-12T21:01:22.849 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:22.852+0000 7ff0a2200700 1 -- 172.21.15.18:0/657214515 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7ff09c109d20 con 0x7ff09c102c30 2024-04-12T21:01:22.849 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:22.853+0000 7ff098c00700 1 -- 172.21.15.18:0/657214515 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+10 (secure 0 0 0) 0x7ff09005a270 con 0x7ff09c102c30 2024-04-12T21:01:22.850 INFO:teuthology.orchestra.run.smithi018.stdout:HEALTH_OK 2024-04-12T21:01:22.854 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:22.857+0000 7ff0a2200700 1 -- 172.21.15.18:0/657214515 >> [v2:172.21.15.18:6800/1605950634,v1:172.21.15.18:6801/1605950634] conn(0x7ff08406e500 msgr2=0x7ff0840709b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:01:22.854 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:22.857+0000 7ff0a2200700 1 --2- 172.21.15.18:0/657214515 >> [v2:172.21.15.18:6800/1605950634,v1:172.21.15.18:6801/1605950634] conn(0x7ff08406e500 0x7ff0840709b0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7ff08800b3b0 tx=0x7ff088009e30 comp rx=0 tx=0).stop 2024-04-12T21:01:22.854 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:22.857+0000 7ff0a2200700 1 -- 172.21.15.18:0/657214515 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff09c102c30 msgr2=0x7ff09c1139c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:01:22.854 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:22.857+0000 7ff0a2200700 1 --2- 172.21.15.18:0/657214515 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff09c102c30 0x7ff09c1139c0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7ff090002a50 tx=0x7ff0900046b0 comp rx=0 tx=0).stop 2024-04-12T21:01:22.854 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:22.857+0000 7ff0a2200700 1 -- 172.21.15.18:0/657214515 shutdown_connections 2024-04-12T21:01:22.854 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:22.857+0000 7ff0a2200700 1 --2- 172.21.15.18:0/657214515 >> [v2:172.21.15.18:6800/1605950634,v1:172.21.15.18:6801/1605950634] conn(0x7ff08406e500 0x7ff0840709b0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:01:22.854 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:22.857+0000 7ff0a2200700 1 --2- 172.21.15.18:0/657214515 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff09c102c30 0x7ff09c1139c0 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:01:22.855 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:22.857+0000 7ff0a2200700 1 --2- 172.21.15.18:0/657214515 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff09c1035d0 0x7ff09c113f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:01:22.855 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:22.857+0000 7ff0a2200700 1 -- 172.21.15.18:0/657214515 >> 172.21.15.18:0/657214515 conn(0x7ff09c0f87a0 msgr2=0x7ff09c0f9760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:01:22.855 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:22.857+0000 7ff0a2200700 1 -- 172.21.15.18:0/657214515 shutdown_connections 2024-04-12T21:01:22.855 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:22.857+0000 7ff0a2200700 1 -- 172.21.15.18:0/657214515 wait complete. 2024-04-12T21:01:22.951 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:01:22 smithi018 ceph-mon[28799]: from='client.14878 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:01:22.952 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:01:22 smithi018 ceph-mon[28799]: Upgrade: Target is version 19.0.0-2879-g324baf8d (squid) 2024-04-12T21:01:22.952 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:01:22 smithi018 ceph-mon[28799]: Upgrade: Target container is quay.ceph.io/ceph-ci/ceph@sha256:f4452def3be4d350f4a8e5feaf7ebb1511be9298fe68478ba7a2d6fbd7d69a63, digests ['quay.ceph.io/ceph-ci/ceph@sha256:f4452def3be4d350f4a8e5feaf7ebb1511be9298fe68478ba7a2d6fbd7d69a63'] 2024-04-12T21:01:22.952 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:01:22 smithi018 ceph-mon[28799]: Upgrade: Setting container_image for all mgr 2024-04-12T21:01:22.952 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:01:22 smithi018 ceph-mon[28799]: from='client.? 172.21.15.18:0/1297276897' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-12T21:01:23.030 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:01:22 smithi186 ceph-mon[36597]: from='client.14878 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:01:23.030 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:01:22 smithi186 ceph-mon[36597]: Upgrade: Target is version 19.0.0-2879-g324baf8d (squid) 2024-04-12T21:01:23.030 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:01:22 smithi186 ceph-mon[36597]: Upgrade: Target container is quay.ceph.io/ceph-ci/ceph@sha256:f4452def3be4d350f4a8e5feaf7ebb1511be9298fe68478ba7a2d6fbd7d69a63, digests ['quay.ceph.io/ceph-ci/ceph@sha256:f4452def3be4d350f4a8e5feaf7ebb1511be9298fe68478ba7a2d6fbd7d69a63'] 2024-04-12T21:01:23.030 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:01:22 smithi186 ceph-mon[36597]: Upgrade: Setting container_image for all mgr 2024-04-12T21:01:23.030 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:01:22 smithi186 ceph-mon[36597]: from='client.? 172.21.15.18:0/1297276897' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-12T21:01:24.030 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:01:23 smithi186 ceph-mon[36597]: from='client.14890 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:01:24.030 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:01:23 smithi186 ceph-mon[36597]: pgmap v128: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-12T21:01:24.030 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:01:23 smithi186 ceph-mon[36597]: from='client.? 172.21.15.18:0/657214515' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-12T21:01:24.100 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:01:23 smithi018 ceph-mon[28799]: from='client.14890 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:01:24.100 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:01:23 smithi018 ceph-mon[28799]: pgmap v128: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-12T21:01:24.100 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:01:23 smithi018 ceph-mon[28799]: from='client.? 172.21.15.18:0/657214515' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-12T21:01:25.026 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:01:24 smithi018 ceph-mon[28799]: Upgrade: Updating mon.smithi018 2024-04-12T21:01:25.026 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:01:24 smithi018 ceph-mon[28799]: from='mgr.14672 172.21.15.18:0/2209228970' entity='mgr.smithi018.xjzwga' 2024-04-12T21:01:25.026 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:01:24 smithi018 ceph-mon[28799]: from='mgr.14672 172.21.15.18:0/2209228970' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-12T21:01:25.026 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:01:24 smithi018 ceph-mon[28799]: from='mgr.14672 172.21.15.18:0/2209228970' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-12T21:01:25.026 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:01:24 smithi018 ceph-mon[28799]: from='mgr.14672 172.21.15.18:0/2209228970' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-12T21:01:25.026 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:01:24 smithi018 ceph-mon[28799]: Deploying daemon mon.smithi018 on smithi018 2024-04-12T21:01:25.280 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:01:24 smithi186 ceph-mon[36597]: Upgrade: Updating mon.smithi018 2024-04-12T21:01:25.280 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:01:24 smithi186 ceph-mon[36597]: from='mgr.14672 172.21.15.18:0/2209228970' entity='mgr.smithi018.xjzwga' 2024-04-12T21:01:25.281 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:01:24 smithi186 ceph-mon[36597]: from='mgr.14672 172.21.15.18:0/2209228970' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-12T21:01:25.281 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:01:24 smithi186 ceph-mon[36597]: from='mgr.14672 172.21.15.18:0/2209228970' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-12T21:01:25.281 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:01:24 smithi186 ceph-mon[36597]: from='mgr.14672 172.21.15.18:0/2209228970' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-12T21:01:25.281 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:01:24 smithi186 ceph-mon[36597]: Deploying daemon mon.smithi018 on smithi018 2024-04-12T21:01:26.101 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:01:25 smithi018 ceph-mon[28799]: pgmap v129: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-12T21:01:26.280 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:01:25 smithi186 ceph-mon[36597]: pgmap v129: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-12T21:01:27.101 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:01:26 smithi018 systemd[1]: Stopping Ceph mon.smithi018 for 1cbe407a-f90d-11ee-bc8d-c7b262605968... 2024-04-12T21:01:27.506 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:01:27 smithi018 ceph-1cbe407a-f90d-11ee-bc8d-c7b262605968-mon-smithi018[28794]: 2024-04-12T21:01:27.118+0000 7fa264800700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.smithi018 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-04-12T21:01:27.507 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:01:27 smithi018 ceph-1cbe407a-f90d-11ee-bc8d-c7b262605968-mon-smithi018[28794]: 2024-04-12T21:01:27.118+0000 7fa264800700 -1 mon.smithi018@0(leader) e2 *** Got Signal Terminated *** 2024-04-12T21:01:27.507 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:01:27 smithi018 podman[112421]: 2024-04-12 21:01:27.190226474 +0000 UTC m=+0.206977513 container died cd84caafa056950f73264f0fa479839383ed2eb657507a99926d9f5eea267f9b (image=quay.ceph.io/ceph-ci/ceph:quincy, name=ceph-1cbe407a-f90d-11ee-bc8d-c7b262605968-mon-smithi018, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_CLEAN=True, CEPH_POINT_RELEASE=, RELEASE=quincy-d3b5060, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240409, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-12T21:01:27.851 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:01:27 smithi018 podman[112421]: 2024-04-12 21:01:27.619462868 +0000 UTC m=+0.636213955 container cleanup cd84caafa056950f73264f0fa479839383ed2eb657507a99926d9f5eea267f9b (image=quay.ceph.io/ceph-ci/ceph:quincy, name=ceph-1cbe407a-f90d-11ee-bc8d-c7b262605968-mon-smithi018, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, RELEASE=quincy-d3b5060, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, org.label-schema.build-date=20240409, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS) 2024-04-12T21:01:27.851 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:01:27 smithi018 bash[112421]: ceph-1cbe407a-f90d-11ee-bc8d-c7b262605968-mon-smithi018 2024-04-12T21:01:28.351 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:01:27 smithi018 podman[112446]: 2024-04-12 21:01:27.937638022 +0000 UTC m=+0.740802362 container remove cd84caafa056950f73264f0fa479839383ed2eb657507a99926d9f5eea267f9b (image=quay.ceph.io/ceph-ci/ceph:quincy, name=ceph-1cbe407a-f90d-11ee-bc8d-c7b262605968-mon-smithi018, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_CLEAN=True, RELEASE=quincy-d3b5060, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240409, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-12T21:01:28.851 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:01:28 smithi018 systemd[1]: ceph-1cbe407a-f90d-11ee-bc8d-c7b262605968@mon.smithi018.service: Deactivated successfully. 2024-04-12T21:01:28.851 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:01:28 smithi018 systemd[1]: Stopped Ceph mon.smithi018 for 1cbe407a-f90d-11ee-bc8d-c7b262605968. 2024-04-12T21:01:28.851 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:01:28 smithi018 systemd[1]: ceph-1cbe407a-f90d-11ee-bc8d-c7b262605968@mon.smithi018.service: Consumed 30.657s CPU time. 2024-04-12T21:01:29.223 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:01:28 smithi018 systemd[1]: Starting Ceph mon.smithi018 for 1cbe407a-f90d-11ee-bc8d-c7b262605968... 2024-04-12T21:01:29.479 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:01:29 smithi018 podman[112572]: 2024-04-12 21:01:29.133356445 +0000 UTC m=+0.038402718 image pull 8717720dc5e5c56303818ea4d4f0a11f6e2c03dee521fb668f411f1304e0afca quay.ceph.io/ceph-ci/ceph@sha256:f4452def3be4d350f4a8e5feaf7ebb1511be9298fe68478ba7a2d6fbd7d69a63 2024-04-12T21:01:29.479 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:01:29 smithi018 podman[112572]: 2024-04-12 21:01:29.296905629 +0000 UTC m=+0.201951928 container create df3e201dfed6be276236a5d4fccee37b4841b35ee7306e2eb01c136fde328dce (image=quay.ceph.io/ceph-ci/ceph@sha256:f4452def3be4d350f4a8e5feaf7ebb1511be9298fe68478ba7a2d6fbd7d69a63, name=ceph-1cbe407a-f90d-11ee-bc8d-c7b262605968-mon-smithi018, org.label-schema.vendor=CentOS, RELEASE=wip-vshankar-testing-20240411.061452-324baf8, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , ceph=True, io.buildah.version=1.33.5, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240409, org.label-schema.license=GPLv2) 2024-04-12T21:01:29.852 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:01:29 smithi018 podman[112572]: 2024-04-12 21:01:29.575749602 +0000 UTC m=+0.480795904 container init df3e201dfed6be276236a5d4fccee37b4841b35ee7306e2eb01c136fde328dce (image=quay.ceph.io/ceph-ci/ceph@sha256:f4452def3be4d350f4a8e5feaf7ebb1511be9298fe68478ba7a2d6fbd7d69a63, name=ceph-1cbe407a-f90d-11ee-bc8d-c7b262605968-mon-smithi018, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, RELEASE=wip-vshankar-testing-20240411.061452-324baf8, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.build-date=20240409, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, CEPH_POINT_RELEASE=) 2024-04-12T21:01:29.852 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:01:29 smithi018 podman[112572]: 2024-04-12 21:01:29.588242151 +0000 UTC m=+0.493288416 container start df3e201dfed6be276236a5d4fccee37b4841b35ee7306e2eb01c136fde328dce (image=quay.ceph.io/ceph-ci/ceph@sha256:f4452def3be4d350f4a8e5feaf7ebb1511be9298fe68478ba7a2d6fbd7d69a63, name=ceph-1cbe407a-f90d-11ee-bc8d-c7b262605968-mon-smithi018, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240409, RELEASE=wip-vshankar-testing-20240411.061452-324baf8, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, io.buildah.version=1.33.5) 2024-04-12T21:01:29.852 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:01:29 smithi018 ceph-mon[112590]: set uid:gid to 167:167 (ceph:ceph) 2024-04-12T21:01:29.852 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:01:29 smithi018 ceph-mon[112590]: ceph version 19.0.0-2879-g324baf8d (324baf8ddb91dea46cac50cb535c306115791355) squid (dev), process ceph-mon, pid 2 2024-04-12T21:01:29.852 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:01:29 smithi018 ceph-mon[112590]: pidfile_write: ignore empty --pid-file 2024-04-12T21:01:29.852 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:01:29 smithi018 ceph-mon[112590]: load: jerasure load: lrc 2024-04-12T21:01:29.852 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:01:29 smithi018 ceph-mon[112590]: rocksdb: RocksDB version: 7.9.2 2024-04-12T21:01:29.852 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:01:29 smithi018 ceph-mon[112590]: rocksdb: Git sha 0 2024-04-12T21:01:29.852 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:01:29 smithi018 ceph-mon[112590]: rocksdb: Compile date 2024-04-11 06:46:50 2024-04-12T21:01:29.853 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:01:29 smithi018 ceph-mon[112590]: rocksdb: DB SUMMARY 2024-04-12T21:01:29.853 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:01:29 smithi018 ceph-mon[112590]: rocksdb: DB Session ID: PC1J5ZCE39CWQJZZVRT7 2024-04-12T21:01:29.853 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:01:29 smithi018 ceph-mon[112590]: rocksdb: CURRENT file: CURRENT 2024-04-12T21:01:29.853 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:01:29 smithi018 ceph-mon[112590]: rocksdb: IDENTITY file: IDENTITY 2024-04-12T21:01:29.853 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:01:29 smithi018 ceph-mon[112590]: rocksdb: MANIFEST file: MANIFEST-000015 size: 1158 Bytes 2024-04-12T21:01:29.853 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:01:29 smithi018 ceph-mon[112590]: rocksdb: SST files in /var/lib/ceph/mon/ceph-smithi018/store.db dir, Total Num: 1, files: 000033.sst 2024-04-12T21:01:29.853 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:01:29 smithi018 ceph-mon[112590]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-smithi018/store.db: 000031.log size: 5294765 ; 2024-04-12T21:01:29.853 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:01:29 smithi018 ceph-mon[112590]: rocksdb: Options.error_if_exists: 0 2024-04-12T21:01:29.853 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:01:29 smithi018 ceph-mon[112590]: rocksdb: Options.create_if_missing: 0 2024-04-12T21:01:29.853 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:01:29 smithi018 ceph-mon[112590]: rocksdb: Options.paranoid_checks: 1 2024-04-12T21:01:29.853 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:01:29 smithi018 ceph-mon[112590]: rocksdb: Options.flush_verify_memtable_count: 1 2024-04-12T21:01:29.853 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:01:29 smithi018 ceph-mon[112590]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-04-12T21:01:29.853 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:01:29 smithi018 ceph-mon[112590]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-04-12T21:01:29.853 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:01:29 smithi018 ceph-mon[112590]: rocksdb: Options.env: 0x561cec594cc0 2024-04-12T21:01:29.853 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:01:29 smithi018 ceph-mon[112590]: rocksdb: Options.fs: PosixFileSystem 2024-04-12T21:01:29.854 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:01:29 smithi018 ceph-mon[112590]: rocksdb: Options.info_log: 0x561d2bd89ae0 2024-04-12T21:01:29.854 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:01:29 smithi018 ceph-mon[112590]: rocksdb: Options.max_file_opening_threads: 16 2024-04-12T21:01:29.854 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:01:29 smithi018 ceph-mon[112590]: rocksdb: Options.statistics: (nil) 2024-04-12T21:01:29.854 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:01:29 smithi018 ceph-mon[112590]: rocksdb: Options.use_fsync: 0 2024-04-12T21:01:29.854 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:01:29 smithi018 ceph-mon[112590]: rocksdb: Options.max_log_file_size: 0 2024-04-12T21:01:29.854 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:01:29 smithi018 ceph-mon[112590]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-04-12T21:01:29.855 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:01:29 smithi018 ceph-mon[112590]: rocksdb: Options.log_file_time_to_roll: 0 2024-04-12T21:01:29.855 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:01:29 smithi018 ceph-mon[112590]: rocksdb: Options.keep_log_file_num: 1000 2024-04-12T21:01:29.855 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:01:29 smithi018 ceph-mon[112590]: rocksdb: Options.recycle_log_file_num: 0 2024-04-12T21:01:29.855 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:01:29 smithi018 ceph-mon[112590]: rocksdb: Options.allow_fallocate: 1 2024-04-12T21:01:29.855 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:01:29 smithi018 ceph-mon[112590]: rocksdb: Options.allow_mmap_reads: 0 2024-04-12T21:01:29.855 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:01:29 smithi018 ceph-mon[112590]: rocksdb: Options.allow_mmap_writes: 0 2024-04-12T21:01:29.855 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:01:29 smithi018 ceph-mon[112590]: rocksdb: Options.use_direct_reads: 0 2024-04-12T21:01:29.855 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:01:29 smithi018 ceph-mon[112590]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-04-12T21:01:29.855 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:01:29 smithi018 ceph-mon[112590]: rocksdb: Options.create_missing_column_families: 0 2024-04-12T21:01:29.855 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:01:29 smithi018 ceph-mon[112590]: rocksdb: Options.db_log_dir: 2024-04-12T21:01:29.855 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:01:29 smithi018 ceph-mon[112590]: rocksdb: Options.wal_dir: 2024-04-12T21:01:29.855 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:01:29 smithi018 ceph-mon[112590]: rocksdb: Options.table_cache_numshardbits: 6 2024-04-12T21:01:29.855 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:01:29 smithi018 ceph-mon[112590]: rocksdb: Options.WAL_ttl_seconds: 0 2024-04-12T21:01:29.856 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:01:29 smithi018 ceph-mon[112590]: rocksdb: Options.WAL_size_limit_MB: 0 2024-04-12T21:01:29.856 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:01:29 smithi018 ceph-mon[112590]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-04-12T21:01:29.856 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:01:29 smithi018 ceph-mon[112590]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-04-12T21:01:29.856 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:01:29 smithi018 ceph-mon[112590]: rocksdb: Options.is_fd_close_on_exec: 1 2024-04-12T21:01:29.856 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:01:29 smithi018 ceph-mon[112590]: rocksdb: Options.advise_random_on_open: 1 2024-04-12T21:01:29.856 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:01:29 smithi018 ceph-mon[112590]: rocksdb: Options.db_write_buffer_size: 0 2024-04-12T21:01:29.856 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:01:29 smithi018 ceph-mon[112590]: rocksdb: Options.write_buffer_manager: 0x561d2bd5a1e0 2024-04-12T21:01:29.856 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:01:29 smithi018 ceph-mon[112590]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-04-12T21:01:29.856 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:01:29 smithi018 ceph-mon[112590]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-04-12T21:01:29.856 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:01:29 smithi018 ceph-mon[112590]: rocksdb: Options.use_adaptive_mutex: 0 2024-04-12T21:01:29.856 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:01:29 smithi018 ceph-mon[112590]: rocksdb: Options.rate_limiter: (nil) 2024-04-12T21:01:29.856 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:01:29 smithi018 ceph-mon[112590]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-04-12T21:01:29.856 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:01:29 smithi018 ceph-mon[112590]: rocksdb: Options.wal_recovery_mode: 2 2024-04-12T21:01:29.856 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:01:29 smithi018 ceph-mon[112590]: rocksdb: Options.enable_thread_tracking: 0 2024-04-12T21:01:29.856 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:01:29 smithi018 ceph-mon[112590]: rocksdb: Options.enable_pipelined_write: 0 2024-04-12T21:01:29.857 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:01:29 smithi018 ceph-mon[112590]: rocksdb: Options.unordered_write: 0 2024-04-12T21:01:29.857 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:01:29 smithi018 ceph-mon[112590]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-04-12T21:01:29.857 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:01:29 smithi018 ceph-mon[112590]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-04-12T21:01:29.857 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:01:29 smithi018 ceph-mon[112590]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-04-12T21:01:29.857 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:01:29 smithi018 ceph-mon[112590]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-04-12T21:01:29.857 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:01:29 smithi018 ceph-mon[112590]: rocksdb: Options.row_cache: None 2024-04-12T21:01:29.857 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:01:29 smithi018 ceph-mon[112590]: rocksdb: Options.wal_filter: None 2024-04-12T21:01:29.857 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:01:29 smithi018 ceph-mon[112590]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-04-12T21:01:29.857 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:01:29 smithi018 ceph-mon[112590]: rocksdb: Options.allow_ingest_behind: 0 2024-04-12T21:01:29.857 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:01:29 smithi018 ceph-mon[112590]: rocksdb: Options.two_write_queues: 0 2024-04-12T21:01:29.857 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:01:29 smithi018 ceph-mon[112590]: rocksdb: Options.manual_wal_flush: 0 2024-04-12T21:01:29.857 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:01:29 smithi018 ceph-mon[112590]: rocksdb: Options.wal_compression: 0 2024-04-12T21:01:29.857 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:01:29 smithi018 ceph-mon[112590]: rocksdb: Options.atomic_flush: 0 2024-04-12T21:01:29.857 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:01:29 smithi018 ceph-mon[112590]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-04-12T21:01:29.857 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:01:29 smithi018 ceph-mon[112590]: rocksdb: Options.persist_stats_to_disk: 0 2024-04-12T21:01:29.858 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:01:29 smithi018 ceph-mon[112590]: rocksdb: Options.write_dbid_to_manifest: 0 2024-04-12T21:01:29.858 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:01:29 smithi018 ceph-mon[112590]: rocksdb: Options.log_readahead_size: 0 2024-04-12T21:01:29.858 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:01:29 smithi018 ceph-mon[112590]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-04-12T21:01:29.858 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:01:29 smithi018 ceph-mon[112590]: rocksdb: Options.best_efforts_recovery: 0 2024-04-12T21:01:29.858 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:01:29 smithi018 ceph-mon[112590]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-04-12T21:01:29.858 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:01:29 smithi018 ceph-mon[112590]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-04-12T21:01:29.858 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:01:29 smithi018 ceph-mon[112590]: rocksdb: Options.allow_data_in_errors: 0 2024-04-12T21:01:29.858 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:01:29 smithi018 ceph-mon[112590]: rocksdb: Options.db_host_id: __hostname__ 2024-04-12T21:01:29.858 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:01:29 smithi018 ceph-mon[112590]: rocksdb: Options.enforce_single_del_contracts: true 2024-04-12T21:01:29.858 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:01:29 smithi018 ceph-mon[112590]: rocksdb: Options.max_background_jobs: 2 2024-04-12T21:01:29.858 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:01:29 smithi018 ceph-mon[112590]: rocksdb: Options.max_background_compactions: -1 2024-04-12T21:01:29.858 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:01:29 smithi018 ceph-mon[112590]: rocksdb: Options.max_subcompactions: 1 2024-04-12T21:01:29.858 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:01:29 smithi018 ceph-mon[112590]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-04-12T21:01:29.858 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:01:29 smithi018 ceph-mon[112590]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-04-12T21:01:29.858 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:01:29 smithi018 ceph-mon[112590]: rocksdb: Options.delayed_write_rate : 16777216 2024-04-12T21:01:29.859 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:01:29 smithi018 ceph-mon[112590]: rocksdb: Options.max_total_wal_size: 0 2024-04-12T21:01:29.859 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:01:29 smithi018 ceph-mon[112590]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-04-12T21:01:29.859 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:01:29 smithi018 ceph-mon[112590]: rocksdb: Options.stats_dump_period_sec: 600 2024-04-12T21:01:29.859 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:01:29 smithi018 ceph-mon[112590]: rocksdb: Options.stats_persist_period_sec: 600 2024-04-12T21:01:29.859 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:01:29 smithi018 ceph-mon[112590]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-04-12T21:01:29.859 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:01:29 smithi018 ceph-mon[112590]: rocksdb: Options.max_open_files: -1 2024-04-12T21:01:29.859 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:01:29 smithi018 ceph-mon[112590]: rocksdb: Options.bytes_per_sync: 0 2024-04-12T21:01:29.859 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:01:29 smithi018 ceph-mon[112590]: rocksdb: Options.wal_bytes_per_sync: 0 2024-04-12T21:01:29.859 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:01:29 smithi018 ceph-mon[112590]: rocksdb: Options.strict_bytes_per_sync: 0 2024-04-12T21:01:29.859 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:01:29 smithi018 ceph-mon[112590]: rocksdb: Options.compaction_readahead_size: 0 2024-04-12T21:01:29.859 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:01:29 smithi018 ceph-mon[112590]: rocksdb: Options.max_background_flushes: -1 2024-04-12T21:01:29.859 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:01:29 smithi018 ceph-mon[112590]: rocksdb: Compression algorithms supported: 2024-04-12T21:01:29.859 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:01:29 smithi018 ceph-mon[112590]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-04-12T21:01:29.859 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:01:29 smithi018 ceph-mon[112590]: rocksdb: kZSTD supported: 0 2024-04-12T21:01:29.859 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:01:29 smithi018 ceph-mon[112590]: rocksdb: kXpressCompression supported: 0 2024-04-12T21:01:29.860 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:01:29 smithi018 ceph-mon[112590]: rocksdb: kLZ4HCCompression supported: 1 2024-04-12T21:01:29.860 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:01:29 smithi018 ceph-mon[112590]: rocksdb: kZlibCompression supported: 1 2024-04-12T21:01:29.861 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:01:29 smithi018 ceph-mon[112590]: rocksdb: kSnappyCompression supported: 1 2024-04-12T21:01:29.861 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:01:29 smithi018 ceph-mon[112590]: rocksdb: kLZ4Compression supported: 1 2024-04-12T21:01:29.861 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:01:29 smithi018 ceph-mon[112590]: rocksdb: kBZip2Compression supported: 0 2024-04-12T21:01:29.861 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:01:29 smithi018 ceph-mon[112590]: rocksdb: Fast CRC32 supported: Supported on x86 2024-04-12T21:01:29.861 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:01:29 smithi018 ceph-mon[112590]: rocksdb: DMutex implementation: pthread_mutex_t 2024-04-12T21:01:29.861 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:01:29 smithi018 ceph-mon[112590]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-smithi018/store.db/MANIFEST-000015 2024-04-12T21:01:29.861 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:01:29 smithi018 ceph-mon[112590]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-04-12T21:01:29.861 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:01:29 smithi018 ceph-mon[112590]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-04-12T21:01:29.861 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:01:29 smithi018 ceph-mon[112590]: rocksdb: Options.merge_operator: 2024-04-12T21:01:29.862 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:01:29 smithi018 ceph-mon[112590]: rocksdb: Options.compaction_filter: None 2024-04-12T21:01:29.862 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:01:29 smithi018 ceph-mon[112590]: rocksdb: Options.compaction_filter_factory: None 2024-04-12T21:01:29.862 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:01:29 smithi018 ceph-mon[112590]: rocksdb: Options.sst_partitioner_factory: None 2024-04-12T21:01:29.862 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:01:29 smithi018 ceph-mon[112590]: rocksdb: Options.memtable_factory: SkipListFactory 2024-04-12T21:01:29.862 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:01:29 smithi018 ceph-mon[112590]: rocksdb: Options.table_factory: BlockBasedTable 2024-04-12T21:01:29.862 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:01:29 smithi018 ceph-mon[112590]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x561d2bd89cc0) 2024-04-12T21:01:29.862 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout: cache_index_and_filter_blocks: 1 2024-04-12T21:01:29.862 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-04-12T21:01:29.862 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-04-12T21:01:29.862 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout: pin_top_level_index_and_filter: 1 2024-04-12T21:01:29.862 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout: index_type: 0 2024-04-12T21:01:29.862 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout: data_block_index_type: 0 2024-04-12T21:01:29.862 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout: index_shortening: 1 2024-04-12T21:01:29.862 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout: data_block_hash_table_util_ratio: 0.750000 2024-04-12T21:01:29.863 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout: checksum: 4 2024-04-12T21:01:29.863 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout: no_block_cache: 0 2024-04-12T21:01:29.863 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout: block_cache: 0x561d2bd8d090 2024-04-12T21:01:29.863 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout: block_cache_name: BinnedLRUCache 2024-04-12T21:01:29.863 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout: block_cache_options: 2024-04-12T21:01:29.863 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout: capacity : 536870912 2024-04-12T21:01:29.863 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout: num_shard_bits : 4 2024-04-12T21:01:29.863 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout: strict_capacity_limit : 0 2024-04-12T21:01:29.863 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout: high_pri_pool_ratio: 0.000 2024-04-12T21:01:29.863 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout: block_cache_compressed: (nil) 2024-04-12T21:01:29.863 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout: persistent_cache: (nil) 2024-04-12T21:01:29.863 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout: block_size: 4096 2024-04-12T21:01:29.863 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout: block_size_deviation: 10 2024-04-12T21:01:29.863 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout: block_restart_interval: 16 2024-04-12T21:01:29.863 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout: index_block_restart_interval: 1 2024-04-12T21:01:29.864 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout: metadata_block_size: 4096 2024-04-12T21:01:29.864 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout: partition_filters: 0 2024-04-12T21:01:29.864 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout: use_delta_encoding: 1 2024-04-12T21:01:29.864 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout: filter_policy: bloomfilter 2024-04-12T21:01:29.864 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout: whole_key_filtering: 1 2024-04-12T21:01:29.864 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout: verify_compression: 0 2024-04-12T21:01:29.864 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout: read_amp_bytes_per_bit: 0 2024-04-12T21:01:29.864 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout: format_version: 5 2024-04-12T21:01:29.864 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout: enable_index_compression: 1 2024-04-12T21:01:29.864 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout: block_align: 0 2024-04-12T21:01:29.864 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout: max_auto_readahead_size: 262144 2024-04-12T21:01:29.864 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout: prepopulate_block_cache: 0 2024-04-12T21:01:29.864 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout: initial_auto_readahead_size: 8192 2024-04-12T21:01:29.864 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout: num_file_reads_for_auto_readahead: 2 2024-04-12T21:01:29.864 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:01:29 smithi018 ceph-mon[112590]: rocksdb: Options.write_buffer_size: 33554432 2024-04-12T21:01:29.864 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:01:29 smithi018 ceph-mon[112590]: rocksdb: Options.max_write_buffer_number: 2 2024-04-12T21:01:29.865 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:01:29 smithi018 ceph-mon[112590]: rocksdb: Options.compression: NoCompression 2024-04-12T21:01:29.865 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:01:29 smithi018 ceph-mon[112590]: rocksdb: Options.bottommost_compression: Disabled 2024-04-12T21:01:29.865 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:01:29 smithi018 ceph-mon[112590]: rocksdb: Options.prefix_extractor: nullptr 2024-04-12T21:01:29.865 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:01:29 smithi018 ceph-mon[112590]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-04-12T21:01:29.865 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:01:29 smithi018 ceph-mon[112590]: rocksdb: Options.num_levels: 7 2024-04-12T21:01:29.865 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:01:29 smithi018 ceph-mon[112590]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-04-12T21:01:29.865 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:01:29 smithi018 ceph-mon[112590]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-04-12T21:01:29.865 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:01:29 smithi018 ceph-mon[112590]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-04-12T21:01:29.865 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:01:29 smithi018 ceph-mon[112590]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-04-12T21:01:29.865 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:01:29 smithi018 ceph-mon[112590]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-04-12T21:01:29.865 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:01:29 smithi018 ceph-mon[112590]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-04-12T21:01:29.865 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:01:29 smithi018 ceph-mon[112590]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-04-12T21:01:29.865 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:01:29 smithi018 ceph-mon[112590]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-04-12T21:01:29.865 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:01:29 smithi018 ceph-mon[112590]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-04-12T21:01:29.865 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:01:29 smithi018 ceph-mon[112590]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-04-12T21:01:29.866 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:01:29 smithi018 ceph-mon[112590]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-04-12T21:01:29.866 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:01:29 smithi018 ceph-mon[112590]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-04-12T21:01:29.866 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:01:29 smithi018 ceph-mon[112590]: rocksdb: Options.compression_opts.window_bits: -14 2024-04-12T21:01:29.866 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:01:29 smithi018 ceph-mon[112590]: rocksdb: Options.compression_opts.level: 32767 2024-04-12T21:01:29.866 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:01:29 smithi018 ceph-mon[112590]: rocksdb: Options.compression_opts.strategy: 0 2024-04-12T21:01:29.866 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:01:29 smithi018 ceph-mon[112590]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-04-12T21:01:29.866 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:01:29 smithi018 ceph-mon[112590]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-04-12T21:01:29.866 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:01:29 smithi018 ceph-mon[112590]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-04-12T21:01:29.866 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:01:29 smithi018 ceph-mon[112590]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-04-12T21:01:29.866 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:01:29 smithi018 ceph-mon[112590]: rocksdb: Options.compression_opts.enabled: false 2024-04-12T21:01:29.866 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:01:29 smithi018 ceph-mon[112590]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-04-12T21:01:29.866 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:01:29 smithi018 ceph-mon[112590]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-04-12T21:01:29.866 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:01:29 smithi018 ceph-mon[112590]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-04-12T21:01:29.866 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:01:29 smithi018 ceph-mon[112590]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-04-12T21:01:29.866 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:01:29 smithi018 ceph-mon[112590]: rocksdb: Options.target_file_size_base: 67108864 2024-04-12T21:01:29.867 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:01:29 smithi018 ceph-mon[112590]: rocksdb: Options.target_file_size_multiplier: 1 2024-04-12T21:01:29.867 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:01:29 smithi018 ceph-mon[112590]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-04-12T21:01:29.867 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:01:29 smithi018 ceph-mon[112590]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-04-12T21:01:29.867 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:01:29 smithi018 ceph-mon[112590]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-04-12T21:01:29.867 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:01:29 smithi018 ceph-mon[112590]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-04-12T21:01:29.867 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:01:29 smithi018 ceph-mon[112590]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-04-12T21:01:29.867 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:01:29 smithi018 ceph-mon[112590]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-04-12T21:01:29.867 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:01:29 smithi018 ceph-mon[112590]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-04-12T21:01:29.867 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:01:29 smithi018 ceph-mon[112590]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-04-12T21:01:29.867 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:01:29 smithi018 ceph-mon[112590]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-04-12T21:01:29.867 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:01:29 smithi018 ceph-mon[112590]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-04-12T21:01:29.867 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:01:29 smithi018 ceph-mon[112590]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-04-12T21:01:29.867 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:01:29 smithi018 ceph-mon[112590]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-04-12T21:01:29.867 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:01:29 smithi018 ceph-mon[112590]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-04-12T21:01:29.868 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:01:29 smithi018 ceph-mon[112590]: rocksdb: Options.arena_block_size: 1048576 2024-04-12T21:01:29.868 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:01:29 smithi018 ceph-mon[112590]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-04-12T21:01:29.868 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:01:29 smithi018 ceph-mon[112590]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-04-12T21:01:29.868 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:01:29 smithi018 ceph-mon[112590]: rocksdb: Options.disable_auto_compactions: 0 2024-04-12T21:01:29.868 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:01:29 smithi018 ceph-mon[112590]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-04-12T21:01:29.868 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:01:29 smithi018 ceph-mon[112590]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-04-12T21:01:29.868 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:01:29 smithi018 ceph-mon[112590]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-04-12T21:01:29.868 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:01:29 smithi018 ceph-mon[112590]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-04-12T21:01:29.868 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:01:29 smithi018 ceph-mon[112590]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-04-12T21:01:29.868 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:01:29 smithi018 ceph-mon[112590]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-04-12T21:01:29.868 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:01:29 smithi018 ceph-mon[112590]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-04-12T21:01:29.868 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:01:29 smithi018 ceph-mon[112590]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-04-12T21:01:29.868 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:01:29 smithi018 ceph-mon[112590]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-04-12T21:01:29.868 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:01:29 smithi018 ceph-mon[112590]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-04-12T21:01:29.868 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:01:29 smithi018 ceph-mon[112590]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-04-12T21:01:29.869 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:01:29 smithi018 ceph-mon[112590]: rocksdb: Options.inplace_update_support: 0 2024-04-12T21:01:29.869 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:01:29 smithi018 ceph-mon[112590]: rocksdb: Options.inplace_update_num_locks: 10000 2024-04-12T21:01:29.869 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:01:29 smithi018 ceph-mon[112590]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-04-12T21:01:29.869 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:01:29 smithi018 ceph-mon[112590]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-04-12T21:01:29.869 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:01:29 smithi018 ceph-mon[112590]: rocksdb: Options.memtable_huge_page_size: 0 2024-04-12T21:01:29.869 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:01:29 smithi018 ceph-mon[112590]: rocksdb: Options.bloom_locality: 0 2024-04-12T21:01:29.869 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:01:29 smithi018 ceph-mon[112590]: rocksdb: Options.max_successive_merges: 0 2024-04-12T21:01:29.869 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:01:29 smithi018 ceph-mon[112590]: rocksdb: Options.optimize_filters_for_hits: 0 2024-04-12T21:01:29.869 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:01:29 smithi018 ceph-mon[112590]: rocksdb: Options.paranoid_file_checks: 0 2024-04-12T21:01:29.869 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:01:29 smithi018 ceph-mon[112590]: rocksdb: Options.force_consistency_checks: 1 2024-04-12T21:01:29.869 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:01:29 smithi018 ceph-mon[112590]: rocksdb: Options.report_bg_io_stats: 0 2024-04-12T21:01:29.869 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:01:29 smithi018 ceph-mon[112590]: rocksdb: Options.ttl: 2592000 2024-04-12T21:01:29.869 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:01:29 smithi018 ceph-mon[112590]: rocksdb: Options.periodic_compaction_seconds: 0 2024-04-12T21:01:29.869 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:01:29 smithi018 ceph-mon[112590]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-04-12T21:01:29.869 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:01:29 smithi018 ceph-mon[112590]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-04-12T21:01:29.869 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:01:29 smithi018 ceph-mon[112590]: rocksdb: Options.enable_blob_files: false 2024-04-12T21:01:29.870 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:01:29 smithi018 ceph-mon[112590]: rocksdb: Options.min_blob_size: 0 2024-04-12T21:01:29.870 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:01:29 smithi018 ceph-mon[112590]: rocksdb: Options.blob_file_size: 268435456 2024-04-12T21:01:29.870 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:01:29 smithi018 ceph-mon[112590]: rocksdb: Options.blob_compression_type: NoCompression 2024-04-12T21:01:29.870 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:01:29 smithi018 ceph-mon[112590]: rocksdb: Options.enable_blob_garbage_collection: false 2024-04-12T21:01:29.870 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:01:29 smithi018 ceph-mon[112590]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-04-12T21:01:29.870 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:01:29 smithi018 ceph-mon[112590]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-04-12T21:01:29.870 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:01:29 smithi018 ceph-mon[112590]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-04-12T21:01:29.870 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:01:29 smithi018 ceph-mon[112590]: rocksdb: Options.blob_file_starting_level: 0 2024-04-12T21:01:29.870 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:01:29 smithi018 ceph-mon[112590]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-04-12T21:01:29.870 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:01:29 smithi018 ceph-mon[112590]: rocksdb: [table/block_based/block_based_table_reader.cc:721] At least one SST file opened without unique ID to verify: 33.sst 2024-04-12T21:01:29.870 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:01:29 smithi018 ceph-mon[112590]: rocksdb: [db/version_set.cc:4390] More existing levels in DB than needed. max_bytes_for_level_multiplier may not be guaranteed. 2024-04-12T21:01:29.870 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:01:29 smithi018 ceph-mon[112590]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-smithi018/store.db/MANIFEST-000015 succeeded,manifest_file_number is 15, next_file_number is 35, last_sequence is 12022, log_number is 31,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-04-12T21:01:29.870 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:01:29 smithi018 ceph-mon[112590]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 31 2024-04-12T21:01:29.870 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:01:29 smithi018 ceph-mon[112590]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: eaf74de4-f45b-484d-b35e-7954c73b0bf5 2024-04-12T21:01:29.871 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:01:29 smithi018 ceph-mon[112590]: rocksdb: EVENT_LOG_v1 {"time_micros": 1712955689651094, "job": 1, "event": "recovery_started", "wal_files": [31]} 2024-04-12T21:01:29.871 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:01:29 smithi018 ceph-mon[112590]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #31 mode 2 2024-04-12T21:01:29.871 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:01:29 smithi018 ceph-mon[112590]: rocksdb: EVENT_LOG_v1 {"time_micros": 1712955689674453, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 36, "file_size": 4766875, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 12023, "largest_seqno": 15826, "table_properties": {"data_size": 4750676, "index_size": 10736, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 4421, "raw_key_size": 38705, "raw_average_key_size": 21, "raw_value_size": 4715008, "raw_average_value_size": 2675, "num_data_blocks": 486, "num_entries": 1762, "num_filter_entries": 1762, "num_deletions": 260, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1712955689, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "eaf74de4-f45b-484d-b35e-7954c73b0bf5", "db_session_id": "PC1J5ZCE39CWQJZZVRT7", "orig_file_number": 36, "seqno_to_time_mapping": "N/A"}} 2024-04-12T21:01:29.871 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:01:29 smithi018 ceph-mon[112590]: rocksdb: EVENT_LOG_v1 {"time_micros": 1712955689674652, "job": 1, "event": "recovery_finished"} 2024-04-12T21:01:29.871 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:01:29 smithi018 ceph-mon[112590]: rocksdb: [db/version_set.cc:5047] Creating manifest 38 2024-04-12T21:01:29.871 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:01:29 smithi018 ceph-mon[112590]: rocksdb: [db/version_set.cc:4390] More existing levels in DB than needed. max_bytes_for_level_multiplier may not be guaranteed. 2024-04-12T21:01:29.871 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:01:29 smithi018 ceph-mon[112590]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-smithi018/store.db/000031.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-04-12T21:01:29.871 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:01:29 smithi018 ceph-mon[112590]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x561d2be7c000 2024-04-12T21:01:29.871 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:01:29 smithi018 ceph-mon[112590]: rocksdb: DB pointer 0x561d2be66000 2024-04-12T21:01:29.871 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:01:29 smithi018 ceph-mon[112590]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-04-12T21:01:29.871 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:01:29 smithi018 ceph-mon[112590]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-04-12T21:01:29.871 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout: ** DB Stats ** 2024-04-12T21:01:29.871 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-04-12T21:01:29.871 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-04-12T21:01:29.872 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-04-12T21:01:29.872 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-04-12T21:01:29.872 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-04-12T21:01:29.872 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-04-12T21:01:29.872 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-04-12T21:01:29.872 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout: 2024-04-12T21:01:29.872 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout: ** Compaction Stats [default] ** 2024-04-12T21:01:29.872 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-04-12T21:01:29.872 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-04-12T21:01:29.872 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout: L0 1/0 4.55 MB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 511.5 0.01 0.00 1 0.009 0 0 0.0 0.0 2024-04-12T21:01:29.872 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout: L6 1/0 5.46 MB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 2024-04-12T21:01:29.872 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout: Sum 2/0 10.01 MB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 511.5 0.01 0.00 1 0.009 0 0 0.0 0.0 2024-04-12T21:01:29.872 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 511.5 0.01 0.00 1 0.009 0 0 0.0 0.0 2024-04-12T21:01:29.872 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout: 2024-04-12T21:01:29.872 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout: ** Compaction Stats [default] ** 2024-04-12T21:01:29.873 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-04-12T21:01:29.873 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-04-12T21:01:29.873 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 511.5 0.01 0.00 1 0.009 0 0 0.0 0.0 2024-04-12T21:01:29.873 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout: 2024-04-12T21:01:29.873 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-04-12T21:01:29.873 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout: 2024-04-12T21:01:29.873 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-04-12T21:01:29.873 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout: Flush(GB): cumulative 0.004, interval 0.004 2024-04-12T21:01:29.873 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-04-12T21:01:29.873 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-04-12T21:01:29.873 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-04-12T21:01:29.873 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout: AddFile(Keys): cumulative 0, interval 0 2024-04-12T21:01:29.873 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout: Cumulative compaction: 0.00 GB write, 149.89 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-04-12T21:01:29.873 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout: Interval compaction: 0.00 GB write, 149.89 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-04-12T21:01:29.873 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-04-12T21:01:29.874 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout: Block cache BinnedLRUCache@0x561d2bd8d090#2 capacity: 512.00 MB usage: 16.61 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 2.2e-05 secs_since: 0 2024-04-12T21:01:29.874 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,4.53 KB,0.000864267%) IndexBlock(1,12.08 KB,0.00230372%) Misc(1,0.00 KB,0%) 2024-04-12T21:01:29.874 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout: 2024-04-12T21:01:29.874 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout: ** File Read Latency Histogram By Level [default] ** 2024-04-12T21:01:29.874 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:01:29 smithi018 ceph-mon[112590]: starting mon.smithi018 rank 0 at public addrs [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] at bind addrs [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] mon_data /var/lib/ceph/mon/ceph-smithi018 fsid 1cbe407a-f90d-11ee-bc8d-c7b262605968 2024-04-12T21:01:29.874 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:01:29 smithi018 ceph-mon[112590]: mon.smithi018@-1(???) e2 preinit fsid 1cbe407a-f90d-11ee-bc8d-c7b262605968 2024-04-12T21:01:29.874 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:01:29 smithi018 ceph-mon[112590]: mon.smithi018@-1(???).mds e23 new map 2024-04-12T21:01:29.874 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:01:29 smithi018 ceph-mon[112590]: mon.smithi018@-1(???).mds e23 print_map 2024-04-12T21:01:29.874 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout: e23 2024-04-12T21:01:29.874 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-04-12T21:01:29.874 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-12T21:01:29.874 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout: legacy client fscid: 1 2024-04-12T21:01:29.874 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout: 2024-04-12T21:01:29.874 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout: Filesystem 'cephfs' (1) 2024-04-12T21:01:29.874 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout: fs_name cephfs 2024-04-12T21:01:29.875 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout: epoch 23 2024-04-12T21:01:29.875 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout: flags 12 joinable allow_snaps allow_multimds_snaps 2024-04-12T21:01:29.875 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout: created 2024-04-12T20:51:57.362850+0000 2024-04-12T21:01:29.875 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout: modified 2024-04-12T21:01:10.670287+0000 2024-04-12T21:01:29.875 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout: tableserver 0 2024-04-12T21:01:29.875 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout: root 0 2024-04-12T21:01:29.875 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout: session_timeout 60 2024-04-12T21:01:29.875 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout: session_autoclose 300 2024-04-12T21:01:29.875 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout: max_file_size 1099511627776 2024-04-12T21:01:29.875 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout: max_xattr_size 65536 2024-04-12T21:01:29.875 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout: required_client_features {} 2024-04-12T21:01:29.875 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout: last_failure 0 2024-04-12T21:01:29.875 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout: last_failure_osd_epoch 46 2024-04-12T21:01:29.875 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout: compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-12T21:01:29.875 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout: max_mds 2 2024-04-12T21:01:29.876 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout: in 0,1 2024-04-12T21:01:29.876 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout: up {0=14452,1=14464} 2024-04-12T21:01:29.876 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout: failed 2024-04-12T21:01:29.876 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout: damaged 2024-04-12T21:01:29.876 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout: stopped 2024-04-12T21:01:29.876 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout: data_pools [3] 2024-04-12T21:01:29.876 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout: metadata_pool 2 2024-04-12T21:01:29.876 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout: inline_data disabled 2024-04-12T21:01:29.876 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout: balancer 2024-04-12T21:01:29.876 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout: bal_rank_mask -1 2024-04-12T21:01:29.876 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout: standby_count_wanted 1 2024-04-12T21:01:29.876 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout: qdb_cluster leader: 0 members: 2024-04-12T21:01:29.876 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout: [mds.cephfs.smithi018.acqbsc{0:14452} state up:active seq 5 join_fscid=1 addr [v2:172.21.15.18:6826/1564065434,v1:172.21.15.18:6827/1564065434] compat {c=[1],r=[1],i=[7ff]}] 2024-04-12T21:01:29.876 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout: [mds.cephfs.smithi018.szjluo{1:14464} state up:active seq 6 join_fscid=1 addr [v2:172.21.15.18:6828/722443353,v1:172.21.15.18:6829/722443353] compat {c=[1],r=[1],i=[7ff]}] 2024-04-12T21:01:29.876 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout: 2024-04-12T21:01:29.877 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout: 2024-04-12T21:01:29.877 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout: Standby daemons: 2024-04-12T21:01:29.877 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout: 2024-04-12T21:01:29.877 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout: [mds.cephfs.smithi186.wjnafs{-1:24277} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.186:6826/438347207,v1:172.21.15.186:6827/438347207] compat {c=[1],r=[1],i=[7ff]}] 2024-04-12T21:01:29.877 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout: [mds.cephfs.smithi186.jdllyn{-1:24281} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.186:6824/1464359952,v1:172.21.15.186:6825/1464359952] compat {c=[1],r=[1],i=[7ff]}] 2024-04-12T21:01:29.877 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:01:29 smithi018 ceph-mon[112590]: mon.smithi018@-1(???).osd e48 crush map has features 3314933000852226048, adjusting msgr requires 2024-04-12T21:01:29.877 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:01:29 smithi018 ceph-mon[112590]: mon.smithi018@-1(???).osd e48 crush map has features 288514051259236352, adjusting msgr requires 2024-04-12T21:01:29.877 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:01:29 smithi018 ceph-mon[112590]: mon.smithi018@-1(???).osd e48 crush map has features 288514051259236352, adjusting msgr requires 2024-04-12T21:01:29.877 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:01:29 smithi018 ceph-mon[112590]: mon.smithi018@-1(???).osd e48 crush map has features 288514051259236352, adjusting msgr requires 2024-04-12T21:01:29.877 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:01:29 smithi018 ceph-mon[112590]: mon.smithi018@-1(???).paxosservice(auth 1..22) refresh upgraded, format 0 -> 3 2024-04-12T21:01:29.877 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:01:29 smithi018 bash[112572]: df3e201dfed6be276236a5d4fccee37b4841b35ee7306e2eb01c136fde328dce 2024-04-12T21:01:29.877 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:01:29 smithi018 systemd[1]: Started Ceph mon.smithi018 for 1cbe407a-f90d-11ee-bc8d-c7b262605968. 2024-04-12T21:01:30.780 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:01:30 smithi186 ceph-mon[36597]: from='mgr.14672 172.21.15.18:0/2209228970' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "mon metadata", "id": "smithi018"}]: dispatch 2024-04-12T21:01:30.781 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:01:30 smithi186 ceph-mon[36597]: from='mgr.14672 172.21.15.18:0/2209228970' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "mon metadata", "id": "smithi186"}]: dispatch 2024-04-12T21:01:30.781 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:01:30 smithi186 ceph-mon[36597]: from='mgr.14672 172.21.15.18:0/2209228970' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T21:01:30.781 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:01:30 smithi186 ceph-mon[36597]: pgmap v131: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-12T21:01:30.781 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:01:30 smithi186 ceph-mon[36597]: mon.smithi018 calling monitor election 2024-04-12T21:01:30.781 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:01:30 smithi186 ceph-mon[36597]: mon.smithi018 is new leader, mons smithi018,smithi186 in quorum (ranks 0,1) 2024-04-12T21:01:30.781 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:01:30 smithi186 ceph-mon[36597]: monmap epoch 2 2024-04-12T21:01:30.781 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:01:30 smithi186 ceph-mon[36597]: fsid 1cbe407a-f90d-11ee-bc8d-c7b262605968 2024-04-12T21:01:30.781 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:01:30 smithi186 ceph-mon[36597]: last_changed 2024-04-12T20:46:19.383203+0000 2024-04-12T21:01:30.781 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:01:30 smithi186 ceph-mon[36597]: created 2024-04-12T20:42:56.111400+0000 2024-04-12T21:01:30.781 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:01:30 smithi186 ceph-mon[36597]: min_mon_release 17 (quincy) 2024-04-12T21:01:30.781 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:01:30 smithi186 ceph-mon[36597]: election_strategy: 1 2024-04-12T21:01:30.781 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:01:30 smithi186 ceph-mon[36597]: 0: [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] mon.smithi018 2024-04-12T21:01:30.782 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:01:30 smithi186 ceph-mon[36597]: 1: [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] mon.smithi186 2024-04-12T21:01:30.782 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:01:30 smithi186 ceph-mon[36597]: fsmap cephfs:2 {0=cephfs.smithi018.acqbsc=up:active,1=cephfs.smithi018.szjluo=up:active} 2 up:standby 2024-04-12T21:01:30.782 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:01:30 smithi186 ceph-mon[36597]: osdmap e48: 6 total, 6 up, 6 in 2024-04-12T21:01:30.782 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:01:30 smithi186 ceph-mon[36597]: mgrmap e32: smithi018.xjzwga(active, since 4m), standbys: smithi186.kqcfxk 2024-04-12T21:01:30.782 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:01:30 smithi186 ceph-mon[36597]: overall HEALTH_OK 2024-04-12T21:01:30.782 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:01:30 smithi186 ceph-mon[36597]: from='mgr.14672 ' entity='' 2024-04-12T21:01:30.782 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:01:30 smithi186 ceph-mon[36597]: mgrmap e33: smithi018.xjzwga(active, since 4m), standbys: smithi186.kqcfxk 2024-04-12T21:01:30.851 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:01:30 smithi018 ceph-mon[112590]: from='mgr.14672 172.21.15.18:0/2209228970' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "mon metadata", "id": "smithi018"}]: dispatch 2024-04-12T21:01:30.852 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:01:30 smithi018 ceph-mon[112590]: from='mgr.14672 172.21.15.18:0/2209228970' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "mon metadata", "id": "smithi186"}]: dispatch 2024-04-12T21:01:30.852 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:01:30 smithi018 ceph-mon[112590]: from='mgr.14672 172.21.15.18:0/2209228970' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T21:01:30.852 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:01:30 smithi018 ceph-mon[112590]: pgmap v131: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-12T21:01:30.852 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:01:30 smithi018 ceph-mon[112590]: mon.smithi018 calling monitor election 2024-04-12T21:01:30.852 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:01:30 smithi018 ceph-mon[112590]: mon.smithi018 is new leader, mons smithi018,smithi186 in quorum (ranks 0,1) 2024-04-12T21:01:30.852 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:01:30 smithi018 ceph-mon[112590]: monmap epoch 2 2024-04-12T21:01:30.852 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:01:30 smithi018 ceph-mon[112590]: fsid 1cbe407a-f90d-11ee-bc8d-c7b262605968 2024-04-12T21:01:30.852 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:01:30 smithi018 ceph-mon[112590]: last_changed 2024-04-12T20:46:19.383203+0000 2024-04-12T21:01:30.852 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:01:30 smithi018 ceph-mon[112590]: created 2024-04-12T20:42:56.111400+0000 2024-04-12T21:01:30.852 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:01:30 smithi018 ceph-mon[112590]: min_mon_release 17 (quincy) 2024-04-12T21:01:30.852 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:01:30 smithi018 ceph-mon[112590]: election_strategy: 1 2024-04-12T21:01:30.852 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:01:30 smithi018 ceph-mon[112590]: 0: [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] mon.smithi018 2024-04-12T21:01:30.852 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:01:30 smithi018 ceph-mon[112590]: 1: [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] mon.smithi186 2024-04-12T21:01:30.853 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:01:30 smithi018 ceph-mon[112590]: fsmap cephfs:2 {0=cephfs.smithi018.acqbsc=up:active,1=cephfs.smithi018.szjluo=up:active} 2 up:standby 2024-04-12T21:01:30.853 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:01:30 smithi018 ceph-mon[112590]: osdmap e48: 6 total, 6 up, 6 in 2024-04-12T21:01:30.853 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:01:30 smithi018 ceph-mon[112590]: mgrmap e32: smithi018.xjzwga(active, since 4m), standbys: smithi186.kqcfxk 2024-04-12T21:01:30.853 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:01:30 smithi018 ceph-mon[112590]: overall HEALTH_OK 2024-04-12T21:01:30.853 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:01:30 smithi018 ceph-mon[112590]: from='mgr.14672 ' entity='' 2024-04-12T21:01:30.853 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:01:30 smithi018 ceph-mon[112590]: mgrmap e33: smithi018.xjzwga(active, since 4m), standbys: smithi186.kqcfxk 2024-04-12T21:01:39.718 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:01:39 smithi018 ceph-mon[112590]: Standby manager daemon smithi186.kqcfxk restarted 2024-04-12T21:01:39.718 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:01:39 smithi018 ceph-mon[112590]: Standby manager daemon smithi186.kqcfxk started 2024-04-12T21:01:39.718 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:01:39 smithi018 ceph-mon[112590]: from='mgr.? 172.21.15.186:0/2193791934' entity='mgr.smithi186.kqcfxk' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi186.kqcfxk/crt"}]: dispatch 2024-04-12T21:01:39.718 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:01:39 smithi018 ceph-mon[112590]: from='mgr.? 172.21.15.186:0/2193791934' entity='mgr.smithi186.kqcfxk' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-12T21:01:39.718 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:01:39 smithi018 ceph-mon[112590]: from='mgr.? 172.21.15.186:0/2193791934' entity='mgr.smithi186.kqcfxk' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi186.kqcfxk/key"}]: dispatch 2024-04-12T21:01:39.718 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:01:39 smithi018 ceph-mon[112590]: from='mgr.? 172.21.15.186:0/2193791934' entity='mgr.smithi186.kqcfxk' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-12T21:01:39.780 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:01:39 smithi186 ceph-mon[36597]: Standby manager daemon smithi186.kqcfxk restarted 2024-04-12T21:01:39.780 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:01:39 smithi186 ceph-mon[36597]: Standby manager daemon smithi186.kqcfxk started 2024-04-12T21:01:39.780 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:01:39 smithi186 ceph-mon[36597]: from='mgr.? 172.21.15.186:0/2193791934' entity='mgr.smithi186.kqcfxk' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi186.kqcfxk/crt"}]: dispatch 2024-04-12T21:01:39.780 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:01:39 smithi186 ceph-mon[36597]: from='mgr.? 172.21.15.186:0/2193791934' entity='mgr.smithi186.kqcfxk' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-12T21:01:39.780 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:01:39 smithi186 ceph-mon[36597]: from='mgr.? 172.21.15.186:0/2193791934' entity='mgr.smithi186.kqcfxk' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi186.kqcfxk/key"}]: dispatch 2024-04-12T21:01:39.780 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:01:39 smithi186 ceph-mon[36597]: from='mgr.? 172.21.15.186:0/2193791934' entity='mgr.smithi186.kqcfxk' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-12T21:01:40.750 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:01:40 smithi018 ceph-mon[112590]: mgrmap e34: smithi018.xjzwga(active, since 4m), standbys: smithi186.kqcfxk 2024-04-12T21:01:40.751 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:01:40 smithi018 ceph-mon[112590]: Active manager daemon smithi018.xjzwga restarted 2024-04-12T21:01:40.751 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:01:40 smithi018 ceph-mon[112590]: Activating manager daemon smithi018.xjzwga 2024-04-12T21:01:40.751 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:01:40 smithi018 ceph-mon[112590]: osdmap e49: 6 total, 6 up, 6 in 2024-04-12T21:01:40.751 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:01:40 smithi018 ceph-mon[112590]: mgrmap e35: smithi018.xjzwga(active, starting, since 0.0108326s), standbys: smithi186.kqcfxk 2024-04-12T21:01:40.751 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:01:40 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "mon metadata", "id": "smithi018"}]: dispatch 2024-04-12T21:01:40.751 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:01:40 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "mon metadata", "id": "smithi186"}]: dispatch 2024-04-12T21:01:40.751 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:01:40 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi018.acqbsc"}]: dispatch 2024-04-12T21:01:40.751 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:01:40 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi018.szjluo"}]: dispatch 2024-04-12T21:01:40.751 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:01:40 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi186.wjnafs"}]: dispatch 2024-04-12T21:01:40.751 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:01:40 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi186.jdllyn"}]: dispatch 2024-04-12T21:01:40.751 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:01:40 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "mgr metadata", "who": "smithi018.xjzwga", "id": "smithi018.xjzwga"}]: dispatch 2024-04-12T21:01:40.752 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:01:40 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "mgr metadata", "who": "smithi186.kqcfxk", "id": "smithi186.kqcfxk"}]: dispatch 2024-04-12T21:01:40.752 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:01:40 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-12T21:01:40.752 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:01:40 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-12T21:01:40.752 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:01:40 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-12T21:01:40.752 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:01:40 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-12T21:01:40.752 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:01:40 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-12T21:01:40.752 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:01:40 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-12T21:01:40.752 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:01:40 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-12T21:01:40.752 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:01:40 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-12T21:01:40.752 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:01:40 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-12T21:01:40.752 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:01:40 smithi018 ceph-mon[112590]: Manager daemon smithi018.xjzwga is now available 2024-04-12T21:01:40.752 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:01:40 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-12T21:01:40.752 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:01:40 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T21:01:40.752 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:01:40 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi018.xjzwga/mirror_snapshot_schedule"}]: dispatch 2024-04-12T21:01:40.752 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:01:40 smithi018 ceph-mon[112590]: from='mgr.34102 ' entity='mgr.smithi018.xjzwga' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi018.xjzwga/mirror_snapshot_schedule"}]: dispatch 2024-04-12T21:01:40.753 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:01:40 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi018.xjzwga/trash_purge_schedule"}]: dispatch 2024-04-12T21:01:40.753 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:01:40 smithi018 ceph-mon[112590]: from='mgr.34102 ' entity='mgr.smithi018.xjzwga' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi018.xjzwga/trash_purge_schedule"}]: dispatch 2024-04-12T21:01:40.781 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:01:40 smithi186 ceph-mon[36597]: mgrmap e34: smithi018.xjzwga(active, since 4m), standbys: smithi186.kqcfxk 2024-04-12T21:01:40.781 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:01:40 smithi186 ceph-mon[36597]: Active manager daemon smithi018.xjzwga restarted 2024-04-12T21:01:40.781 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:01:40 smithi186 ceph-mon[36597]: Activating manager daemon smithi018.xjzwga 2024-04-12T21:01:40.781 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:01:40 smithi186 ceph-mon[36597]: osdmap e49: 6 total, 6 up, 6 in 2024-04-12T21:01:40.781 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:01:40 smithi186 ceph-mon[36597]: mgrmap e35: smithi018.xjzwga(active, starting, since 0.0108326s), standbys: smithi186.kqcfxk 2024-04-12T21:01:40.781 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:01:40 smithi186 ceph-mon[36597]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "mon metadata", "id": "smithi018"}]: dispatch 2024-04-12T21:01:40.781 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:01:40 smithi186 ceph-mon[36597]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "mon metadata", "id": "smithi186"}]: dispatch 2024-04-12T21:01:40.781 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:01:40 smithi186 ceph-mon[36597]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi018.acqbsc"}]: dispatch 2024-04-12T21:01:40.781 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:01:40 smithi186 ceph-mon[36597]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi018.szjluo"}]: dispatch 2024-04-12T21:01:40.781 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:01:40 smithi186 ceph-mon[36597]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi186.wjnafs"}]: dispatch 2024-04-12T21:01:40.782 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:01:40 smithi186 ceph-mon[36597]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi186.jdllyn"}]: dispatch 2024-04-12T21:01:40.782 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:01:40 smithi186 ceph-mon[36597]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "mgr metadata", "who": "smithi018.xjzwga", "id": "smithi018.xjzwga"}]: dispatch 2024-04-12T21:01:40.782 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:01:40 smithi186 ceph-mon[36597]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "mgr metadata", "who": "smithi186.kqcfxk", "id": "smithi186.kqcfxk"}]: dispatch 2024-04-12T21:01:40.782 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:01:40 smithi186 ceph-mon[36597]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-12T21:01:40.782 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:01:40 smithi186 ceph-mon[36597]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-12T21:01:40.782 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:01:40 smithi186 ceph-mon[36597]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-12T21:01:40.782 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:01:40 smithi186 ceph-mon[36597]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-12T21:01:40.782 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:01:40 smithi186 ceph-mon[36597]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-12T21:01:40.782 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:01:40 smithi186 ceph-mon[36597]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-12T21:01:40.782 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:01:40 smithi186 ceph-mon[36597]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-12T21:01:40.782 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:01:40 smithi186 ceph-mon[36597]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-12T21:01:40.782 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:01:40 smithi186 ceph-mon[36597]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-12T21:01:40.782 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:01:40 smithi186 ceph-mon[36597]: Manager daemon smithi018.xjzwga is now available 2024-04-12T21:01:40.782 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:01:40 smithi186 ceph-mon[36597]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-12T21:01:40.783 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:01:40 smithi186 ceph-mon[36597]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T21:01:40.783 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:01:40 smithi186 ceph-mon[36597]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi018.xjzwga/mirror_snapshot_schedule"}]: dispatch 2024-04-12T21:01:40.783 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:01:40 smithi186 ceph-mon[36597]: from='mgr.34102 ' entity='mgr.smithi018.xjzwga' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi018.xjzwga/mirror_snapshot_schedule"}]: dispatch 2024-04-12T21:01:40.783 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:01:40 smithi186 ceph-mon[36597]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi018.xjzwga/trash_purge_schedule"}]: dispatch 2024-04-12T21:01:40.783 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:01:40 smithi186 ceph-mon[36597]: from='mgr.34102 ' entity='mgr.smithi018.xjzwga' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi018.xjzwga/trash_purge_schedule"}]: dispatch 2024-04-12T21:01:42.030 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:01:41 smithi186 ceph-mon[36597]: mgrmap e36: smithi018.xjzwga(active, since 1.02529s), standbys: smithi186.kqcfxk 2024-04-12T21:01:42.030 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:01:41 smithi186 ceph-mon[36597]: [12/Apr/2024:21:01:40] ENGINE Bus STARTING 2024-04-12T21:01:42.030 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:01:41 smithi186 ceph-mon[36597]: [12/Apr/2024:21:01:40] ENGINE Serving on https://172.21.15.18:7150 2024-04-12T21:01:42.030 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:01:41 smithi186 ceph-mon[36597]: [12/Apr/2024:21:01:41] ENGINE Serving on http://172.21.15.18:8765 2024-04-12T21:01:42.030 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:01:41 smithi186 ceph-mon[36597]: [12/Apr/2024:21:01:41] ENGINE Bus STARTED 2024-04-12T21:01:42.101 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:01:41 smithi018 ceph-mon[112590]: mgrmap e36: smithi018.xjzwga(active, since 1.02529s), standbys: smithi186.kqcfxk 2024-04-12T21:01:42.101 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:01:41 smithi018 ceph-mon[112590]: [12/Apr/2024:21:01:40] ENGINE Bus STARTING 2024-04-12T21:01:42.101 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:01:41 smithi018 ceph-mon[112590]: [12/Apr/2024:21:01:40] ENGINE Serving on https://172.21.15.18:7150 2024-04-12T21:01:42.101 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:01:41 smithi018 ceph-mon[112590]: [12/Apr/2024:21:01:41] ENGINE Serving on http://172.21.15.18:8765 2024-04-12T21:01:42.102 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:01:41 smithi018 ceph-mon[112590]: [12/Apr/2024:21:01:41] ENGINE Bus STARTED 2024-04-12T21:01:43.030 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:01:42 smithi186 ceph-mon[36597]: pgmap v4: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-12T21:01:43.030 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:01:42 smithi186 ceph-mon[36597]: mgrmap e37: smithi018.xjzwga(active, since 2s), standbys: smithi186.kqcfxk 2024-04-12T21:01:43.101 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:01:42 smithi018 ceph-mon[112590]: pgmap v4: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-12T21:01:43.101 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:01:42 smithi018 ceph-mon[112590]: mgrmap e37: smithi018.xjzwga(active, since 2s), standbys: smithi186.kqcfxk 2024-04-12T21:01:45.100 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:01:44 smithi018 ceph-mon[112590]: pgmap v5: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-12T21:01:45.101 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:01:44 smithi018 ceph-mon[112590]: mgrmap e38: smithi018.xjzwga(active, since 4s), standbys: smithi186.kqcfxk 2024-04-12T21:01:45.101 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:01:44 smithi018 ceph-mon[112590]: from='mgr.34102 ' entity='mgr.smithi018.xjzwga' 2024-04-12T21:01:45.101 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:01:44 smithi018 ceph-mon[112590]: from='mgr.34102 ' entity='mgr.smithi018.xjzwga' 2024-04-12T21:01:45.279 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:01:44 smithi186 ceph-mon[36597]: pgmap v5: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-12T21:01:45.280 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:01:44 smithi186 ceph-mon[36597]: mgrmap e38: smithi018.xjzwga(active, since 4s), standbys: smithi186.kqcfxk 2024-04-12T21:01:45.280 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:01:44 smithi186 ceph-mon[36597]: from='mgr.34102 ' entity='mgr.smithi018.xjzwga' 2024-04-12T21:01:45.280 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:01:44 smithi186 ceph-mon[36597]: from='mgr.34102 ' entity='mgr.smithi018.xjzwga' 2024-04-12T21:01:47.739 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:01:47 smithi186 ceph-mon[36597]: pgmap v6: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-12T21:01:47.739 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:01:47 smithi186 ceph-mon[36597]: from='mgr.34102 ' entity='mgr.smithi018.xjzwga' 2024-04-12T21:01:47.739 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:01:47 smithi186 ceph-mon[36597]: from='mgr.34102 ' entity='mgr.smithi018.xjzwga' 2024-04-12T21:01:47.739 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:01:47 smithi186 ceph-mon[36597]: from='mgr.34102 ' entity='mgr.smithi018.xjzwga' 2024-04-12T21:01:47.739 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:01:47 smithi186 ceph-mon[36597]: from='mgr.34102 ' entity='mgr.smithi018.xjzwga' 2024-04-12T21:01:47.851 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:01:47 smithi018 ceph-mon[112590]: pgmap v6: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-12T21:01:47.851 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:01:47 smithi018 ceph-mon[112590]: from='mgr.34102 ' entity='mgr.smithi018.xjzwga' 2024-04-12T21:01:47.851 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:01:47 smithi018 ceph-mon[112590]: from='mgr.34102 ' entity='mgr.smithi018.xjzwga' 2024-04-12T21:01:47.851 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:01:47 smithi018 ceph-mon[112590]: from='mgr.34102 ' entity='mgr.smithi018.xjzwga' 2024-04-12T21:01:47.851 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:01:47 smithi018 ceph-mon[112590]: from='mgr.34102 ' entity='mgr.smithi018.xjzwga' 2024-04-12T21:01:49.780 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:01:49 smithi186 ceph-mon[36597]: pgmap v7: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 29 KiB/s rd, 0 B/s wr, 16 op/s 2024-04-12T21:01:49.807 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:01:49 smithi018 ceph-mon[112590]: pgmap v7: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 29 KiB/s rd, 0 B/s wr, 16 op/s 2024-04-12T21:01:50.800 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:01:50 smithi018 ceph-mon[112590]: from='mgr.34102 ' entity='mgr.smithi018.xjzwga' 2024-04-12T21:01:50.800 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:01:50 smithi018 ceph-mon[112590]: from='mgr.34102 ' entity='mgr.smithi018.xjzwga' 2024-04-12T21:01:50.800 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:01:50 smithi018 ceph-mon[112590]: pgmap v8: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 23 KiB/s rd, 0 B/s wr, 12 op/s 2024-04-12T21:01:51.030 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:01:50 smithi186 ceph-mon[36597]: from='mgr.34102 ' entity='mgr.smithi018.xjzwga' 2024-04-12T21:01:51.030 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:01:50 smithi186 ceph-mon[36597]: from='mgr.34102 ' entity='mgr.smithi018.xjzwga' 2024-04-12T21:01:51.030 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:01:50 smithi186 ceph-mon[36597]: pgmap v8: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 23 KiB/s rd, 0 B/s wr, 12 op/s 2024-04-12T21:01:52.851 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:01:52 smithi018 ceph-mon[112590]: Detected new or changed devices on smithi186 2024-04-12T21:01:52.851 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:01:52 smithi018 ceph-mon[112590]: from='mgr.34102 ' entity='mgr.smithi018.xjzwga' 2024-04-12T21:01:52.851 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:01:52 smithi018 ceph-mon[112590]: from='mgr.34102 ' entity='mgr.smithi018.xjzwga' 2024-04-12T21:01:52.851 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:01:52 smithi018 ceph-mon[112590]: pgmap v9: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 19 KiB/s rd, 0 B/s wr, 10 op/s 2024-04-12T21:01:53.023 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:53.026+0000 7fd6e8c00700 1 -- 172.21.15.18:0/481810734 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd6e40702e0 msgr2=0x7fd6e40706b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:01:53.023 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:53.026+0000 7fd6e8c00700 1 --2- 172.21.15.18:0/481810734 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd6e40702e0 0x7fd6e40706b0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fd6d4004940 tx=0x7fd6d4009e30 comp rx=0 tx=0).stop 2024-04-12T21:01:53.023 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:53.026+0000 7fd6e8c00700 1 -- 172.21.15.18:0/481810734 shutdown_connections 2024-04-12T21:01:53.023 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:53.026+0000 7fd6e8c00700 1 --2- 172.21.15.18:0/481810734 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd6e4070bf0 0x7fd6e41091b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:01:53.023 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:53.026+0000 7fd6e8c00700 1 --2- 172.21.15.18:0/481810734 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd6e40702e0 0x7fd6e40706b0 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:01:53.023 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:53.026+0000 7fd6e8c00700 1 -- 172.21.15.18:0/481810734 >> 172.21.15.18:0/481810734 conn(0x7fd6e40f89d0 msgr2=0x7fd6e40fade0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:01:53.023 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:53.026+0000 7fd6e8c00700 1 -- 172.21.15.18:0/481810734 shutdown_connections 2024-04-12T21:01:53.023 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:53.026+0000 7fd6e8c00700 1 -- 172.21.15.18:0/481810734 wait complete. 2024-04-12T21:01:53.024 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:53.027+0000 7fd6e8c00700 1 Processor -- start 2024-04-12T21:01:53.024 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:53.027+0000 7fd6e8c00700 1 -- start start 2024-04-12T21:01:53.025 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:53.027+0000 7fd6e8c00700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd6e40702e0 0x7fd6e4115e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:01:53.025 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:53.028+0000 7fd6e8c00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd6e4070bf0 0x7fd6e4116340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:01:53.025 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:53.028+0000 7fd6e8c00700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6e4116a20 con 0x7fd6e4070bf0 2024-04-12T21:01:53.025 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:53.028+0000 7fd6e8c00700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6e410fd60 con 0x7fd6e40702e0 2024-04-12T21:01:53.025 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:53.028+0000 7fd6e1400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd6e4070bf0 0x7fd6e4116340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:01:53.025 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:53.028+0000 7fd6e1400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd6e4070bf0 0x7fd6e4116340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:54092/0 (socket says 172.21.15.18:54092) 2024-04-12T21:01:53.025 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:53.028+0000 7fd6e1400700 1 -- 172.21.15.18:0/3247223112 learned_addr learned my addr 172.21.15.18:0/3247223112 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:01:53.025 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:53.028+0000 7fd6e1e00700 1 --2- 172.21.15.18:0/3247223112 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd6e40702e0 0x7fd6e4115e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:01:53.026 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:53.028+0000 7fd6e1400700 1 -- 172.21.15.18:0/3247223112 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd6e40702e0 msgr2=0x7fd6e4115e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:01:53.026 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:53.028+0000 7fd6e1400700 1 --2- 172.21.15.18:0/3247223112 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd6e40702e0 0x7fd6e4115e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:01:53.026 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:53.029+0000 7fd6e1400700 1 -- 172.21.15.18:0/3247223112 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6d4005040 con 0x7fd6e4070bf0 2024-04-12T21:01:53.026 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:53.029+0000 7fd6e1400700 1 --2- 172.21.15.18:0/3247223112 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd6e4070bf0 0x7fd6e4116340 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fd6d000b790 tx=0x7fd6d0009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:01:53.026 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:53.029+0000 7fd6da000700 1 -- 172.21.15.18:0/3247223112 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd6d000f770 con 0x7fd6e4070bf0 2024-04-12T21:01:53.026 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:53.029+0000 7fd6e8c00700 1 -- 172.21.15.18:0/3247223112 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd6e410ffe0 con 0x7fd6e4070bf0 2024-04-12T21:01:53.026 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:53.029+0000 7fd6da000700 1 -- 172.21.15.18:0/3247223112 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd6d000fda0 con 0x7fd6e4070bf0 2024-04-12T21:01:53.027 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:53.029+0000 7fd6da000700 1 -- 172.21.15.18:0/3247223112 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd6d000d310 con 0x7fd6e4070bf0 2024-04-12T21:01:53.027 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:53.029+0000 7fd6e8c00700 1 -- 172.21.15.18:0/3247223112 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd6e4110520 con 0x7fd6e4070bf0 2024-04-12T21:01:53.027 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:53.031+0000 7fd6e8c00700 1 -- 172.21.15.18:0/3247223112 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd6e4071a20 con 0x7fd6e4070bf0 2024-04-12T21:01:53.029 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:53.033+0000 7fd6da000700 1 -- 172.21.15.18:0/3247223112 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 38) v1 ==== 92847+0+0 (secure 0 0 0) 0x7fd6d0010420 con 0x7fd6e4070bf0 2024-04-12T21:01:53.030 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:01:52 smithi186 ceph-mon[36597]: Detected new or changed devices on smithi186 2024-04-12T21:01:53.030 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:01:52 smithi186 ceph-mon[36597]: from='mgr.34102 ' entity='mgr.smithi018.xjzwga' 2024-04-12T21:01:53.030 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:01:52 smithi186 ceph-mon[36597]: from='mgr.34102 ' entity='mgr.smithi018.xjzwga' 2024-04-12T21:01:53.030 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:01:52 smithi186 ceph-mon[36597]: pgmap v9: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 19 KiB/s rd, 0 B/s wr, 10 op/s 2024-04-12T21:01:53.031 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:53.034+0000 7fd6da000700 1 --2- 172.21.15.18:0/3247223112 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fd6c806ed20 0x7fd6c80711d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:01:53.031 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:53.034+0000 7fd6da000700 1 -- 172.21.15.18:0/3247223112 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6324+0+0 (secure 0 0 0) 0x7fd6d008e9c0 con 0x7fd6e4070bf0 2024-04-12T21:01:53.031 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:53.034+0000 7fd6e1e00700 1 --2- 172.21.15.18:0/3247223112 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fd6c806ed20 0x7fd6c80711d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:01:53.031 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:53.035+0000 7fd6e1e00700 1 --2- 172.21.15.18:0/3247223112 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fd6c806ed20 0x7fd6c80711d0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fd6d400b020 tx=0x7fd6d4009e30 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:01:53.040 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:53.044+0000 7fd6da000700 1 -- 172.21.15.18:0/3247223112 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7fd6d005b290 con 0x7fd6e4070bf0 2024-04-12T21:01:53.297 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:53.301+0000 7fd6e8c00700 1 -- 172.21.15.18:0/3247223112 --> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fd6e410c1f0 con 0x7fd6c806ed20 2024-04-12T21:01:53.302 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:53.304+0000 7fd6da000700 1 -- 172.21.15.18:0/3247223112 <== mgr.34102 v2:172.21.15.18:6800/3812891919 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+337 (secure 0 0 0) 0x7fd6e410c1f0 con 0x7fd6c806ed20 2024-04-12T21:01:53.304 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:53.307+0000 7fd6e8c00700 1 -- 172.21.15.18:0/3247223112 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fd6c806ed20 msgr2=0x7fd6c80711d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:01:53.304 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:53.307+0000 7fd6e8c00700 1 --2- 172.21.15.18:0/3247223112 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fd6c806ed20 0x7fd6c80711d0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fd6d400b020 tx=0x7fd6d4009e30 comp rx=0 tx=0).stop 2024-04-12T21:01:53.304 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:53.307+0000 7fd6e8c00700 1 -- 172.21.15.18:0/3247223112 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd6e4070bf0 msgr2=0x7fd6e4116340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:01:53.305 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:53.307+0000 7fd6e8c00700 1 --2- 172.21.15.18:0/3247223112 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd6e4070bf0 0x7fd6e4116340 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fd6d000b790 tx=0x7fd6d0009250 comp rx=0 tx=0).stop 2024-04-12T21:01:53.305 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:53.308+0000 7fd6e8c00700 1 -- 172.21.15.18:0/3247223112 shutdown_connections 2024-04-12T21:01:53.305 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:53.308+0000 7fd6e8c00700 1 --2- 172.21.15.18:0/3247223112 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fd6c806ed20 0x7fd6c80711d0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:01:53.305 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:53.308+0000 7fd6e8c00700 1 --2- 172.21.15.18:0/3247223112 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd6e40702e0 0x7fd6e4115e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:01:53.305 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:53.308+0000 7fd6e8c00700 1 --2- 172.21.15.18:0/3247223112 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd6e4070bf0 0x7fd6e4116340 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:01:53.305 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:53.308+0000 7fd6e8c00700 1 -- 172.21.15.18:0/3247223112 >> 172.21.15.18:0/3247223112 conn(0x7fd6e40f89d0 msgr2=0x7fd6e40f9e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:01:53.305 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:53.308+0000 7fd6e8c00700 1 -- 172.21.15.18:0/3247223112 shutdown_connections 2024-04-12T21:01:53.305 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:53.308+0000 7fd6e8c00700 1 -- 172.21.15.18:0/3247223112 wait complete. 2024-04-12T21:01:53.320 INFO:teuthology.orchestra.run.smithi018.stdout:true 2024-04-12T21:01:53.460 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:53.463+0000 7f6cdf600700 1 -- 172.21.15.18:0/869083633 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6cd80fddf0 msgr2=0x7f6cd80fe1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:01:53.460 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:53.463+0000 7f6cdf600700 1 --2- 172.21.15.18:0/869083633 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6cd80fddf0 0x7f6cd80fe1c0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f6ccc004940 tx=0x7f6ccc009e30 comp rx=0 tx=0).stop 2024-04-12T21:01:53.460 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:53.463+0000 7f6cdf600700 1 -- 172.21.15.18:0/869083633 shutdown_connections 2024-04-12T21:01:53.460 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:53.463+0000 7f6cdf600700 1 --2- 172.21.15.18:0/869083633 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6cd80fe790 0x7f6cd8106dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:01:53.460 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:53.463+0000 7f6cdf600700 1 --2- 172.21.15.18:0/869083633 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6cd80fddf0 0x7f6cd80fe1c0 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:01:53.460 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:53.463+0000 7f6cdf600700 1 -- 172.21.15.18:0/869083633 >> 172.21.15.18:0/869083633 conn(0x7f6cd80f78e0 msgr2=0x7f6cd80f9cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:01:53.460 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:53.463+0000 7f6cdf600700 1 -- 172.21.15.18:0/869083633 shutdown_connections 2024-04-12T21:01:53.461 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:53.463+0000 7f6cdf600700 1 -- 172.21.15.18:0/869083633 wait complete. 2024-04-12T21:01:53.461 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:53.464+0000 7f6cdf600700 1 Processor -- start 2024-04-12T21:01:53.461 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:53.465+0000 7f6cdf600700 1 -- start start 2024-04-12T21:01:53.462 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:53.465+0000 7f6cdf600700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6cd80fddf0 0x7f6cd80fbc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:01:53.462 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:53.465+0000 7f6cdf600700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6cd80fe790 0x7f6cd80fc160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:01:53.462 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:53.465+0000 7f6cdf600700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6cd80fdb70 con 0x7f6cd80fe790 2024-04-12T21:01:53.462 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:53.465+0000 7f6cdf600700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6cd80fc6a0 con 0x7f6cd80fddf0 2024-04-12T21:01:53.462 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:53.466+0000 7f6cd7e00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6cd80fe790 0x7f6cd80fc160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:01:53.463 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:53.466+0000 7f6cd7e00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6cd80fe790 0x7f6cd80fc160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:54122/0 (socket says 172.21.15.18:54122) 2024-04-12T21:01:53.463 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:53.466+0000 7f6cd7e00700 1 -- 172.21.15.18:0/3756740048 learned_addr learned my addr 172.21.15.18:0/3756740048 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:01:53.463 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:53.466+0000 7f6cdcc00700 1 --2- 172.21.15.18:0/3756740048 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6cd80fddf0 0x7f6cd80fbc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:01:53.463 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:53.466+0000 7f6cd7e00700 1 -- 172.21.15.18:0/3756740048 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6cd80fddf0 msgr2=0x7f6cd80fbc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:01:53.463 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:53.466+0000 7f6cd7e00700 1 --2- 172.21.15.18:0/3756740048 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6cd80fddf0 0x7f6cd80fbc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:01:53.463 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:53.466+0000 7f6cd7e00700 1 -- 172.21.15.18:0/3756740048 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ccc005040 con 0x7f6cd80fe790 2024-04-12T21:01:53.463 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:53.467+0000 7f6cd7e00700 1 --2- 172.21.15.18:0/3756740048 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6cd80fe790 0x7f6cd80fc160 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f6cc4002a50 tx=0x7f6cc40046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:01:53.464 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:53.467+0000 7f6cd4c00700 1 -- 172.21.15.18:0/3756740048 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6cc4007cc0 con 0x7f6cd80fe790 2024-04-12T21:01:53.464 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:53.467+0000 7f6cd4c00700 1 -- 172.21.15.18:0/3756740048 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f6cc400f040 con 0x7f6cd80fe790 2024-04-12T21:01:53.464 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:53.467+0000 7f6cd4c00700 1 -- 172.21.15.18:0/3756740048 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6cc4009640 con 0x7f6cd80fe790 2024-04-12T21:01:53.464 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:53.467+0000 7f6cdf600700 1 -- 172.21.15.18:0/3756740048 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6cd80fc980 con 0x7f6cd80fe790 2024-04-12T21:01:53.464 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:53.467+0000 7f6cdf600700 1 -- 172.21.15.18:0/3756740048 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6cd811a090 con 0x7f6cd80fe790 2024-04-12T21:01:53.465 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:53.469+0000 7f6cdf600700 1 -- 172.21.15.18:0/3756740048 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6cd8066770 con 0x7f6cd80fe790 2024-04-12T21:01:53.466 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:53.470+0000 7f6cd4c00700 1 -- 172.21.15.18:0/3756740048 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 38) v1 ==== 92847+0+0 (secure 0 0 0) 0x7f6cc40097a0 con 0x7f6cd80fe790 2024-04-12T21:01:53.467 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:53.471+0000 7f6cd4c00700 1 --2- 172.21.15.18:0/3756740048 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f6cc006ed30 0x7f6cc00711e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:01:53.468 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:53.471+0000 7f6cd4c00700 1 -- 172.21.15.18:0/3756740048 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6324+0+0 (secure 0 0 0) 0x7f6cc408e810 con 0x7f6cd80fe790 2024-04-12T21:01:53.468 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:53.472+0000 7f6cdcc00700 1 --2- 172.21.15.18:0/3756740048 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f6cc006ed30 0x7f6cc00711e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:01:53.469 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:53.473+0000 7f6cdcc00700 1 --2- 172.21.15.18:0/3756740048 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f6cc006ed30 0x7f6cc00711e0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f6ccc009fd0 tx=0x7f6ccc009e30 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:01:53.473 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:53.476+0000 7f6cd4c00700 1 -- 172.21.15.18:0/3756740048 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f6cc4057420 con 0x7f6cd80fe790 2024-04-12T21:01:53.751 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:53.755+0000 7f6cdf600700 1 -- 172.21.15.18:0/3756740048 --> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f6cd80f9650 con 0x7f6cc006ed30 2024-04-12T21:01:53.753 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:53.756+0000 7f6cd4c00700 1 -- 172.21.15.18:0/3756740048 <== mgr.34102 v2:172.21.15.18:6800/3812891919 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+337 (secure 0 0 0) 0x7f6cd80f9650 con 0x7f6cc006ed30 2024-04-12T21:01:53.756 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:53.759+0000 7f6cdf600700 1 -- 172.21.15.18:0/3756740048 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f6cc006ed30 msgr2=0x7f6cc00711e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:01:53.756 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:53.759+0000 7f6cdf600700 1 --2- 172.21.15.18:0/3756740048 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f6cc006ed30 0x7f6cc00711e0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f6ccc009fd0 tx=0x7f6ccc009e30 comp rx=0 tx=0).stop 2024-04-12T21:01:53.756 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:53.759+0000 7f6cdf600700 1 -- 172.21.15.18:0/3756740048 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6cd80fe790 msgr2=0x7f6cd80fc160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:01:53.756 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:53.759+0000 7f6cdf600700 1 --2- 172.21.15.18:0/3756740048 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6cd80fe790 0x7f6cd80fc160 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f6cc4002a50 tx=0x7f6cc40046b0 comp rx=0 tx=0).stop 2024-04-12T21:01:53.756 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:53.759+0000 7f6cdf600700 1 -- 172.21.15.18:0/3756740048 shutdown_connections 2024-04-12T21:01:53.756 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:53.759+0000 7f6cdf600700 1 --2- 172.21.15.18:0/3756740048 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f6cc006ed30 0x7f6cc00711e0 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:01:53.756 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:53.759+0000 7f6cdf600700 1 --2- 172.21.15.18:0/3756740048 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6cd80fddf0 0x7f6cd80fbc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:01:53.756 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:53.759+0000 7f6cdf600700 1 --2- 172.21.15.18:0/3756740048 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6cd80fe790 0x7f6cd80fc160 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:01:53.757 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:53.759+0000 7f6cdf600700 1 -- 172.21.15.18:0/3756740048 >> 172.21.15.18:0/3756740048 conn(0x7f6cd80f78e0 msgr2=0x7f6cd80f8f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:01:53.757 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:53.761+0000 7f6cdf600700 1 -- 172.21.15.18:0/3756740048 shutdown_connections 2024-04-12T21:01:53.757 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:53.761+0000 7f6cdf600700 1 -- 172.21.15.18:0/3756740048 wait complete. 2024-04-12T21:01:53.929 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:53.933+0000 7fe411000700 1 -- 172.21.15.18:0/3982063962 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe40c071f90 msgr2=0x7fe40c070360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:01:53.929 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:53.933+0000 7fe411000700 1 --2- 172.21.15.18:0/3982063962 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe40c071f90 0x7fe40c070360 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fe3fc004970 tx=0x7fe3fc009e30 comp rx=0 tx=0).stop 2024-04-12T21:01:53.930 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:01:53 smithi018 ceph-mon[112590]: from='client.34132 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:01:53.930 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:53.933+0000 7fe411000700 1 -- 172.21.15.18:0/3982063962 shutdown_connections 2024-04-12T21:01:53.931 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:53.933+0000 7fe411000700 1 --2- 172.21.15.18:0/3982063962 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe40c0708a0 0x7fe40c070d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:01:53.931 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:53.933+0000 7fe411000700 1 --2- 172.21.15.18:0/3982063962 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe40c071f90 0x7fe40c070360 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:01:53.931 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:53.933+0000 7fe411000700 1 -- 172.21.15.18:0/3982063962 >> 172.21.15.18:0/3982063962 conn(0x7fe40c0f89b0 msgr2=0x7fe40c0fadc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:01:53.931 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:53.933+0000 7fe411000700 1 -- 172.21.15.18:0/3982063962 shutdown_connections 2024-04-12T21:01:53.931 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:53.933+0000 7fe411000700 1 -- 172.21.15.18:0/3982063962 wait complete. 2024-04-12T21:01:53.931 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:53.934+0000 7fe411000700 1 Processor -- start 2024-04-12T21:01:53.931 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:53.934+0000 7fe411000700 1 -- start start 2024-04-12T21:01:53.932 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:53.935+0000 7fe411000700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe40c0708a0 0x7fe40c11a0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:01:53.932 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:53.935+0000 7fe411000700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe40c071f90 0x7fe40c11a630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:01:53.932 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:53.935+0000 7fe411000700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe40c11ac80 con 0x7fe40c0708a0 2024-04-12T21:01:53.932 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:53.935+0000 7fe411000700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe40c114100 con 0x7fe40c071f90 2024-04-12T21:01:53.932 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:53.935+0000 7fe409e00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe40c0708a0 0x7fe40c11a0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:01:53.932 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:53.935+0000 7fe409e00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe40c0708a0 0x7fe40c11a0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:54146/0 (socket says 172.21.15.18:54146) 2024-04-12T21:01:53.932 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:53.935+0000 7fe409e00700 1 -- 172.21.15.18:0/4092298140 learned_addr learned my addr 172.21.15.18:0/4092298140 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:01:53.933 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:53.936+0000 7fe409400700 1 --2- 172.21.15.18:0/4092298140 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe40c071f90 0x7fe40c11a630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:01:53.933 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:53.936+0000 7fe409e00700 1 -- 172.21.15.18:0/4092298140 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe40c071f90 msgr2=0x7fe40c11a630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:01:53.933 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:53.936+0000 7fe409e00700 1 --2- 172.21.15.18:0/4092298140 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe40c071f90 0x7fe40c11a630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:01:53.933 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:53.936+0000 7fe409e00700 1 -- 172.21.15.18:0/4092298140 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3fc005040 con 0x7fe40c0708a0 2024-04-12T21:01:53.933 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:53.936+0000 7fe409e00700 1 --2- 172.21.15.18:0/4092298140 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe40c0708a0 0x7fe40c11a0f0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fe3fc004f50 tx=0x7fe3fc009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:01:53.933 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:53.936+0000 7fe402000700 1 -- 172.21.15.18:0/4092298140 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe3fc00e040 con 0x7fe40c0708a0 2024-04-12T21:01:53.933 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:53.937+0000 7fe402000700 1 -- 172.21.15.18:0/4092298140 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe3fc004290 con 0x7fe40c0708a0 2024-04-12T21:01:53.934 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:53.937+0000 7fe402000700 1 -- 172.21.15.18:0/4092298140 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe3fc01c630 con 0x7fe40c0708a0 2024-04-12T21:01:53.934 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:53.937+0000 7fe411000700 1 -- 172.21.15.18:0/4092298140 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe40c114380 con 0x7fe40c0708a0 2024-04-12T21:01:53.934 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:53.937+0000 7fe411000700 1 -- 172.21.15.18:0/4092298140 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe40c1148c0 con 0x7fe40c0708a0 2024-04-12T21:01:53.935 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:53.938+0000 7fe411000700 1 -- 172.21.15.18:0/4092298140 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe40c071aa0 con 0x7fe40c0708a0 2024-04-12T21:01:53.936 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:53.940+0000 7fe402000700 1 -- 172.21.15.18:0/4092298140 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 38) v1 ==== 92847+0+0 (secure 0 0 0) 0x7fe3fc00bc10 con 0x7fe40c0708a0 2024-04-12T21:01:53.937 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:53.941+0000 7fe402000700 1 --2- 172.21.15.18:0/4092298140 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fe3f006ed20 0x7fe3f00711d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:01:53.937 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:53.941+0000 7fe402000700 1 -- 172.21.15.18:0/4092298140 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6324+0+0 (secure 0 0 0) 0x7fe3fc08f670 con 0x7fe40c0708a0 2024-04-12T21:01:53.938 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:53.941+0000 7fe409400700 1 --2- 172.21.15.18:0/4092298140 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fe3f006ed20 0x7fe3f00711d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:01:53.938 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:53.942+0000 7fe409400700 1 --2- 172.21.15.18:0/4092298140 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fe3f006ed20 0x7fe3f00711d0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fe40c10cf00 tx=0x7fe3f8009ee0 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:01:53.946 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:53.949+0000 7fe402000700 1 -- 172.21.15.18:0/4092298140 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7fe3fc058300 con 0x7fe40c0708a0 2024-04-12T21:01:54.029 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:01:53 smithi186 ceph-mon[36597]: from='client.34132 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:01:54.193 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:54.196+0000 7fe411000700 1 -- 172.21.15.18:0/4092298140 --> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fe40c110520 con 0x7fe3f006ed20 2024-04-12T21:01:54.207 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:54.211+0000 7fe402000700 1 -- 172.21.15.18:0/4092298140 <== mgr.34102 v2:172.21.15.18:6800/3812891919 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3520 (secure 0 0 0) 0x7fe40c110520 con 0x7fe3f006ed20 2024-04-12T21:01:54.207 INFO:teuthology.orchestra.run.smithi018.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-12T21:01:54.207 INFO:teuthology.orchestra.run.smithi018.stdout:alertmanager.smithi018 smithi018 *:9093,9094 running (2m) 6s ago 16m 14.9M - 0.25.0 c8568f914cd2 c332090068e8 2024-04-12T21:01:54.208 INFO:teuthology.orchestra.run.smithi018.stdout:crash.smithi018 smithi018 running (17m) 6s ago 17m 6660k - 17.2.7-827-gd3b5060a 21de9b7369ad 1ac7603bddfd 2024-04-12T21:01:54.208 INFO:teuthology.orchestra.run.smithi018.stdout:crash.smithi186 smithi186 running (15m) 9s ago 15m 6669k - 17.2.7-827-gd3b5060a 21de9b7369ad 14bf8a1672eb 2024-04-12T21:01:54.208 INFO:teuthology.orchestra.run.smithi018.stdout:grafana.smithi018 smithi018 *:3000 running (115s) 6s ago 16m 74.4M - 9.4.12 2bacad6d85d8 b08e3bf176dd 2024-04-12T21:01:54.208 INFO:teuthology.orchestra.run.smithi018.stdout:mds.cephfs.smithi018.acqbsc smithi018 running (9m) 6s ago 9m 112M - 17.2.7-827-gd3b5060a 21de9b7369ad 5b5fce65f18a 2024-04-12T21:01:54.208 INFO:teuthology.orchestra.run.smithi018.stdout:mds.cephfs.smithi018.szjluo smithi018 running (9m) 6s ago 9m 113M - 17.2.7-827-gd3b5060a 21de9b7369ad cf9bf0c5031f 2024-04-12T21:01:54.208 INFO:teuthology.orchestra.run.smithi018.stdout:mds.cephfs.smithi186.jdllyn smithi186 running (9m) 9s ago 9m 13.9M - 17.2.7-827-gd3b5060a 21de9b7369ad 2b2bc46bd29a 2024-04-12T21:01:54.208 INFO:teuthology.orchestra.run.smithi018.stdout:mds.cephfs.smithi186.wjnafs smithi186 running (9m) 9s ago 9m 14.0M - 17.2.7-827-gd3b5060a 21de9b7369ad 74b115d6d6fd 2024-04-12T21:01:54.208 INFO:teuthology.orchestra.run.smithi018.stdout:mgr.smithi018.xjzwga smithi018 *:8443,9283,8765 running (5m) 6s ago 18m 478M - 19.0.0-2879-g324baf8d 8717720dc5e5 56088305eab7 2024-04-12T21:01:54.208 INFO:teuthology.orchestra.run.smithi018.stdout:mgr.smithi186.kqcfxk smithi186 *:8443,9283,8765 running (3m) 9s ago 15m 448M - 19.0.0-2879-g324baf8d 8717720dc5e5 09cd766e5e0d 2024-04-12T21:01:54.208 INFO:teuthology.orchestra.run.smithi018.stdout:mon.smithi018 smithi018 running (25s) 6s ago 18m 29.8M 2048M 19.0.0-2879-g324baf8d 8717720dc5e5 df3e201dfed6 2024-04-12T21:01:54.208 INFO:teuthology.orchestra.run.smithi018.stdout:mon.smithi186 smithi186 running (15m) 9s ago 15m 51.2M 2048M 17.2.7-827-gd3b5060a 21de9b7369ad d7263c772b73 2024-04-12T21:01:54.208 INFO:teuthology.orchestra.run.smithi018.stdout:node-exporter.smithi018 smithi018 *:9100 running (3m) 6s ago 17m 7898k - 1.5.0 0da6a335fe13 326c3d9c6847 2024-04-12T21:01:54.208 INFO:teuthology.orchestra.run.smithi018.stdout:node-exporter.smithi186 smithi186 *:9100 running (3m) 9s ago 15m 7348k - 1.5.0 0da6a335fe13 ec2112baeecf 2024-04-12T21:01:54.208 INFO:teuthology.orchestra.run.smithi018.stdout:osd.0 smithi018 running (14m) 6s ago 14m 226M 1559M 17.2.7-827-gd3b5060a 21de9b7369ad 0d83e027bf0a 2024-04-12T21:01:54.209 INFO:teuthology.orchestra.run.smithi018.stdout:osd.1 smithi018 running (14m) 6s ago 14m 241M 1559M 17.2.7-827-gd3b5060a 21de9b7369ad 6517c9cc7f87 2024-04-12T21:01:54.209 INFO:teuthology.orchestra.run.smithi018.stdout:osd.2 smithi018 running (13m) 6s ago 13m 194M 1559M 17.2.7-827-gd3b5060a 21de9b7369ad 8a83692acaa5 2024-04-12T21:01:54.209 INFO:teuthology.orchestra.run.smithi018.stdout:osd.3 smithi186 running (12m) 9s ago 12m 265M 2581M 17.2.7-827-gd3b5060a 21de9b7369ad c2def7a90d76 2024-04-12T21:01:54.209 INFO:teuthology.orchestra.run.smithi018.stdout:osd.4 smithi186 running (12m) 9s ago 12m 233M 2581M 17.2.7-827-gd3b5060a 21de9b7369ad 757e7c71b7b9 2024-04-12T21:01:54.209 INFO:teuthology.orchestra.run.smithi018.stdout:osd.5 smithi186 running (11m) 9s ago 11m 228M 2581M 17.2.7-827-gd3b5060a 21de9b7369ad 2ee072a436d0 2024-04-12T21:01:54.209 INFO:teuthology.orchestra.run.smithi018.stdout:prometheus.smithi018 smithi018 *:9095 running (2m) 6s ago 16m 56.0M - 2.43.0 a07b618ecd1d f58841127e21 2024-04-12T21:01:54.211 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:54.214+0000 7fe411000700 1 -- 172.21.15.18:0/4092298140 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fe3f006ed20 msgr2=0x7fe3f00711d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:01:54.211 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:54.214+0000 7fe411000700 1 --2- 172.21.15.18:0/4092298140 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fe3f006ed20 0x7fe3f00711d0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fe40c10cf00 tx=0x7fe3f8009ee0 comp rx=0 tx=0).stop 2024-04-12T21:01:54.211 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:54.214+0000 7fe411000700 1 -- 172.21.15.18:0/4092298140 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe40c0708a0 msgr2=0x7fe40c11a0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:01:54.211 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:54.214+0000 7fe411000700 1 --2- 172.21.15.18:0/4092298140 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe40c0708a0 0x7fe40c11a0f0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fe3fc004f50 tx=0x7fe3fc009e30 comp rx=0 tx=0).stop 2024-04-12T21:01:54.211 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:54.214+0000 7fe411000700 1 -- 172.21.15.18:0/4092298140 shutdown_connections 2024-04-12T21:01:54.211 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:54.214+0000 7fe411000700 1 --2- 172.21.15.18:0/4092298140 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fe3f006ed20 0x7fe3f00711d0 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:01:54.211 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:54.214+0000 7fe411000700 1 --2- 172.21.15.18:0/4092298140 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe40c0708a0 0x7fe40c11a0f0 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:01:54.211 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:54.214+0000 7fe411000700 1 --2- 172.21.15.18:0/4092298140 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe40c071f90 0x7fe40c11a630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:01:54.211 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:54.214+0000 7fe411000700 1 -- 172.21.15.18:0/4092298140 >> 172.21.15.18:0/4092298140 conn(0x7fe40c0f89b0 msgr2=0x7fe40c0f9aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:01:54.212 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:54.215+0000 7fe411000700 1 -- 172.21.15.18:0/4092298140 shutdown_connections 2024-04-12T21:01:54.212 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:54.215+0000 7fe411000700 1 -- 172.21.15.18:0/4092298140 wait complete. 2024-04-12T21:01:54.360 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:54.364+0000 7f2ae2400700 1 -- 172.21.15.18:0/1688686243 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2adc1037a0 msgr2=0x7f2adc0fce10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:01:54.361 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:54.364+0000 7f2ae2400700 1 --2- 172.21.15.18:0/1688686243 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2adc1037a0 0x7f2adc0fce10 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f2ac8004940 tx=0x7f2ac8009e30 comp rx=0 tx=0).stop 2024-04-12T21:01:54.361 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:54.364+0000 7f2ae2400700 1 -- 172.21.15.18:0/1688686243 shutdown_connections 2024-04-12T21:01:54.361 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:54.364+0000 7f2ae2400700 1 --2- 172.21.15.18:0/1688686243 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2adc1037a0 0x7f2adc0fce10 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:01:54.361 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:54.364+0000 7f2ae2400700 1 --2- 172.21.15.18:0/1688686243 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2adc102e00 0x7f2adc1031d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:01:54.361 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:54.364+0000 7f2ae2400700 1 -- 172.21.15.18:0/1688686243 >> 172.21.15.18:0/1688686243 conn(0x7f2adc0f89b0 msgr2=0x7f2adc0fadc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:01:54.361 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:54.364+0000 7f2ae2400700 1 -- 172.21.15.18:0/1688686243 shutdown_connections 2024-04-12T21:01:54.361 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:54.364+0000 7f2ae2400700 1 -- 172.21.15.18:0/1688686243 wait complete. 2024-04-12T21:01:54.362 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:54.365+0000 7f2ae2400700 1 Processor -- start 2024-04-12T21:01:54.362 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:54.365+0000 7f2ae2400700 1 -- start start 2024-04-12T21:01:54.362 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:54.366+0000 7f2ae2400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2adc102e00 0x7f2adc115db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:01:54.362 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:54.366+0000 7f2ae2400700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2adc1037a0 0x7f2adc1162f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:01:54.362 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:54.366+0000 7f2ae2400700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2adc1169d0 con 0x7f2adc102e00 2024-04-12T21:01:54.362 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:54.366+0000 7f2ae2400700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2adc10fd10 con 0x7f2adc1037a0 2024-04-12T21:01:54.363 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:54.366+0000 7f2adb400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2adc102e00 0x7f2adc115db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:01:54.363 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:54.366+0000 7f2adb400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2adc102e00 0x7f2adc115db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:54152/0 (socket says 172.21.15.18:54152) 2024-04-12T21:01:54.363 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:54.366+0000 7f2adb400700 1 -- 172.21.15.18:0/991104961 learned_addr learned my addr 172.21.15.18:0/991104961 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:01:54.363 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:54.367+0000 7f2adaa00700 1 --2- 172.21.15.18:0/991104961 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2adc1037a0 0x7f2adc1162f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:01:54.363 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:54.367+0000 7f2adb400700 1 -- 172.21.15.18:0/991104961 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2adc1037a0 msgr2=0x7f2adc1162f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:01:54.364 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:54.367+0000 7f2adb400700 1 --2- 172.21.15.18:0/991104961 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2adc1037a0 0x7f2adc1162f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:01:54.364 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:54.367+0000 7f2adb400700 1 -- 172.21.15.18:0/991104961 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ac8005040 con 0x7f2adc102e00 2024-04-12T21:01:54.364 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:54.367+0000 7f2adb400700 1 --2- 172.21.15.18:0/991104961 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2adc102e00 0x7f2adc115db0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f2acc00b790 tx=0x7f2acc009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:01:54.364 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:54.368+0000 7f2ad3400700 1 -- 172.21.15.18:0/991104961 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2acc00f770 con 0x7f2adc102e00 2024-04-12T21:01:54.364 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:54.368+0000 7f2ad3400700 1 -- 172.21.15.18:0/991104961 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f2acc00fda0 con 0x7f2adc102e00 2024-04-12T21:01:54.365 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:54.368+0000 7f2ad3400700 1 -- 172.21.15.18:0/991104961 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2acc00d310 con 0x7f2adc102e00 2024-04-12T21:01:54.365 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:54.368+0000 7f2ae2400700 1 -- 172.21.15.18:0/991104961 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2adc10ff90 con 0x7f2adc102e00 2024-04-12T21:01:54.365 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:54.368+0000 7f2ae2400700 1 -- 172.21.15.18:0/991104961 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2adc110550 con 0x7f2adc102e00 2024-04-12T21:01:54.366 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:54.369+0000 7f2ae2400700 1 -- 172.21.15.18:0/991104961 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2adc107300 con 0x7f2adc102e00 2024-04-12T21:01:54.367 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:54.370+0000 7f2ad3400700 1 -- 172.21.15.18:0/991104961 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 38) v1 ==== 92847+0+0 (secure 0 0 0) 0x7f2acc010420 con 0x7f2adc102e00 2024-04-12T21:01:54.368 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:54.371+0000 7f2ad3400700 1 --2- 172.21.15.18:0/991104961 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f2ac006ede0 0x7f2ac0071290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:01:54.368 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:54.371+0000 7f2ad3400700 1 -- 172.21.15.18:0/991104961 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6324+0+0 (secure 0 0 0) 0x7f2acc08e8c0 con 0x7f2adc102e00 2024-04-12T21:01:54.368 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:54.372+0000 7f2adaa00700 1 --2- 172.21.15.18:0/991104961 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f2ac006ede0 0x7f2ac0071290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:01:54.369 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:54.373+0000 7f2adaa00700 1 --2- 172.21.15.18:0/991104961 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f2ac006ede0 0x7f2ac0071290 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f2ac800b3b0 tx=0x7f2ac8009e30 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:01:54.373 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:54.377+0000 7f2ad3400700 1 -- 172.21.15.18:0/991104961 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f2acc05b190 con 0x7f2adc102e00 2024-04-12T21:01:54.704 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:54.706+0000 7f2ae2400700 1 -- 172.21.15.18:0/991104961 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f2adc109fb0 con 0x7f2adc102e00 2024-04-12T21:01:54.704 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:54.708+0000 7f2ad3400700 1 -- 172.21.15.18:0/991104961 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+843 (secure 0 0 0) 0x7f2acc05ad20 con 0x7f2adc102e00 2024-04-12T21:01:54.705 INFO:teuthology.orchestra.run.smithi018.stdout:{ 2024-04-12T21:01:54.705 INFO:teuthology.orchestra.run.smithi018.stdout: "mon": { 2024-04-12T21:01:54.705 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 17.2.7-827-gd3b5060a (d3b5060a17449edfae9385cdf792afec109fca5d) quincy (stable)": 1, 2024-04-12T21:01:54.706 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 19.0.0-2879-g324baf8d (324baf8ddb91dea46cac50cb535c306115791355) squid (dev)": 1 2024-04-12T21:01:54.706 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-12T21:01:54.706 INFO:teuthology.orchestra.run.smithi018.stdout: "mgr": { 2024-04-12T21:01:54.706 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 19.0.0-2879-g324baf8d (324baf8ddb91dea46cac50cb535c306115791355) squid (dev)": 2 2024-04-12T21:01:54.706 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-12T21:01:54.706 INFO:teuthology.orchestra.run.smithi018.stdout: "osd": { 2024-04-12T21:01:54.706 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 17.2.7-827-gd3b5060a (d3b5060a17449edfae9385cdf792afec109fca5d) quincy (stable)": 6 2024-04-12T21:01:54.706 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-12T21:01:54.706 INFO:teuthology.orchestra.run.smithi018.stdout: "mds": { 2024-04-12T21:01:54.706 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 17.2.7-827-gd3b5060a (d3b5060a17449edfae9385cdf792afec109fca5d) quincy (stable)": 4 2024-04-12T21:01:54.706 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-12T21:01:54.706 INFO:teuthology.orchestra.run.smithi018.stdout: "overall": { 2024-04-12T21:01:54.706 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 17.2.7-827-gd3b5060a (d3b5060a17449edfae9385cdf792afec109fca5d) quincy (stable)": 11, 2024-04-12T21:01:54.707 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 19.0.0-2879-g324baf8d (324baf8ddb91dea46cac50cb535c306115791355) squid (dev)": 3 2024-04-12T21:01:54.707 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-04-12T21:01:54.707 INFO:teuthology.orchestra.run.smithi018.stdout:} 2024-04-12T21:01:54.708 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:54.711+0000 7f2ae2400700 1 -- 172.21.15.18:0/991104961 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f2ac006ede0 msgr2=0x7f2ac0071290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:01:54.708 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:54.711+0000 7f2ae2400700 1 --2- 172.21.15.18:0/991104961 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f2ac006ede0 0x7f2ac0071290 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f2ac800b3b0 tx=0x7f2ac8009e30 comp rx=0 tx=0).stop 2024-04-12T21:01:54.709 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:54.711+0000 7f2ae2400700 1 -- 172.21.15.18:0/991104961 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2adc102e00 msgr2=0x7f2adc115db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:01:54.709 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:54.711+0000 7f2ae2400700 1 --2- 172.21.15.18:0/991104961 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2adc102e00 0x7f2adc115db0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f2acc00b790 tx=0x7f2acc009250 comp rx=0 tx=0).stop 2024-04-12T21:01:54.709 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:54.712+0000 7f2ae2400700 1 -- 172.21.15.18:0/991104961 shutdown_connections 2024-04-12T21:01:54.709 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:54.712+0000 7f2ae2400700 1 --2- 172.21.15.18:0/991104961 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f2ac006ede0 0x7f2ac0071290 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:01:54.709 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:54.712+0000 7f2ae2400700 1 --2- 172.21.15.18:0/991104961 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2adc102e00 0x7f2adc115db0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:01:54.709 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:54.712+0000 7f2ae2400700 1 --2- 172.21.15.18:0/991104961 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2adc1037a0 0x7f2adc1162f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:01:54.709 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:54.712+0000 7f2ae2400700 1 -- 172.21.15.18:0/991104961 >> 172.21.15.18:0/991104961 conn(0x7f2adc0f89b0 msgr2=0x7f2adc0fa4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:01:54.709 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:54.712+0000 7f2ae2400700 1 -- 172.21.15.18:0/991104961 shutdown_connections 2024-04-12T21:01:54.709 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:54.712+0000 7f2ae2400700 1 -- 172.21.15.18:0/991104961 wait complete. 2024-04-12T21:01:54.861 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:54.863+0000 7f52a6600700 1 -- 172.21.15.18:0/2195339865 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f52a0070c10 msgr2=0x7f52a0108fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:01:54.861 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:54.863+0000 7f52a6600700 1 --2- 172.21.15.18:0/2195339865 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f52a0070c10 0x7f52a0108fc0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f528c004970 tx=0x7f528c009e30 comp rx=0 tx=0).stop 2024-04-12T21:01:54.861 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:54.864+0000 7f52a6600700 1 -- 172.21.15.18:0/2195339865 shutdown_connections 2024-04-12T21:01:54.861 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:54.864+0000 7f52a6600700 1 --2- 172.21.15.18:0/2195339865 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f52a0070c10 0x7f52a0108fc0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:01:54.861 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:54.864+0000 7f52a6600700 1 --2- 172.21.15.18:0/2195339865 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f52a0070300 0x7f52a00706d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:01:54.861 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:54.864+0000 7f52a6600700 1 -- 172.21.15.18:0/2195339865 >> 172.21.15.18:0/2195339865 conn(0x7f52a00f87e0 msgr2=0x7f52a00fabf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:01:54.861 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:54.864+0000 7f52a6600700 1 -- 172.21.15.18:0/2195339865 shutdown_connections 2024-04-12T21:01:54.861 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:54.864+0000 7f52a6600700 1 -- 172.21.15.18:0/2195339865 wait complete. 2024-04-12T21:01:54.863 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:54.865+0000 7f52a6600700 1 Processor -- start 2024-04-12T21:01:54.863 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:54.866+0000 7f52a6600700 1 -- start start 2024-04-12T21:01:54.863 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:54.866+0000 7f52a6600700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f52a0070300 0x7f52a0115b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:01:54.863 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:54.866+0000 7f52a6600700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f52a0070c10 0x7f52a01160c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:01:54.863 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:54.866+0000 7f52a6600700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52a0116710 con 0x7f52a0070c10 2024-04-12T21:01:54.863 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:54.866+0000 7f52a6600700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52a010fb00 con 0x7f52a0070300 2024-04-12T21:01:54.863 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:54.866+0000 7f529fe00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f52a0070c10 0x7f52a01160c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:01:54.863 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:54.866+0000 7f529fe00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f52a0070c10 0x7f52a01160c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:54162/0 (socket says 172.21.15.18:54162) 2024-04-12T21:01:54.864 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:54.867+0000 7f529fe00700 1 -- 172.21.15.18:0/1571289084 learned_addr learned my addr 172.21.15.18:0/1571289084 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:01:54.864 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:54.867+0000 7f52a5200700 1 --2- 172.21.15.18:0/1571289084 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f52a0070300 0x7f52a0115b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:01:54.865 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:54.867+0000 7f529fe00700 1 -- 172.21.15.18:0/1571289084 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f52a0070300 msgr2=0x7f52a0115b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:01:54.865 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:54.867+0000 7f529fe00700 1 --2- 172.21.15.18:0/1571289084 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f52a0070300 0x7f52a0115b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:01:54.865 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:54.867+0000 7f529fe00700 1 -- 172.21.15.18:0/1571289084 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f528c005040 con 0x7f52a0070c10 2024-04-12T21:01:54.865 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:54.867+0000 7f52a5200700 1 --2- 172.21.15.18:0/1571289084 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f52a0070300 0x7f52a0115b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-12T21:01:54.865 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:54.868+0000 7f529fe00700 1 --2- 172.21.15.18:0/1571289084 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f52a0070c10 0x7f52a01160c0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f528c004f50 tx=0x7f528c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:01:54.865 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:54.868+0000 7f529cc00700 1 -- 172.21.15.18:0/1571289084 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f528c00e040 con 0x7f52a0070c10 2024-04-12T21:01:54.865 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:54.868+0000 7f529cc00700 1 -- 172.21.15.18:0/1571289084 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f528c004290 con 0x7f52a0070c10 2024-04-12T21:01:54.865 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:54.868+0000 7f529cc00700 1 -- 172.21.15.18:0/1571289084 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f528c01c3f0 con 0x7f52a0070c10 2024-04-12T21:01:54.865 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:54.868+0000 7f52a6600700 1 -- 172.21.15.18:0/1571289084 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f52a010fd80 con 0x7f52a0070c10 2024-04-12T21:01:54.866 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:54.869+0000 7f52a6600700 1 -- 172.21.15.18:0/1571289084 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f52a0110270 con 0x7f52a0070c10 2024-04-12T21:01:54.866 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:54.870+0000 7f52a6600700 1 -- 172.21.15.18:0/1571289084 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f52a01039d0 con 0x7f52a0070c10 2024-04-12T21:01:54.868 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:54.872+0000 7f529cc00700 1 -- 172.21.15.18:0/1571289084 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 38) v1 ==== 92847+0+0 (secure 0 0 0) 0x7f528c00bc10 con 0x7f52a0070c10 2024-04-12T21:01:54.870 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:54.874+0000 7f529cc00700 1 --2- 172.21.15.18:0/1571289084 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f528806ed20 0x7f52880711d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:01:54.870 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:54.874+0000 7f529cc00700 1 -- 172.21.15.18:0/1571289084 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6324+0+0 (secure 0 0 0) 0x7f528c090830 con 0x7f52a0070c10 2024-04-12T21:01:54.870 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:54.874+0000 7f52a5200700 1 --2- 172.21.15.18:0/1571289084 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f528806ed20 0x7f52880711d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:01:54.871 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:54.875+0000 7f52a5200700 1 --2- 172.21.15.18:0/1571289084 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f528806ed20 0x7f52880711d0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f5294009fd0 tx=0x7f5294009ee0 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:01:54.874 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:54.878+0000 7f529cc00700 1 -- 172.21.15.18:0/1571289084 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f528c05cff0 con 0x7f52a0070c10 2024-04-12T21:01:55.030 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:01:54 smithi186 ceph-mon[36597]: from='client.34136 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:01:55.030 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:01:54 smithi186 ceph-mon[36597]: pgmap v10: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 17 KiB/s rd, 0 B/s wr, 9 op/s 2024-04-12T21:01:55.030 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:01:54 smithi186 ceph-mon[36597]: from='client.34140 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:01:55.100 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:01:54 smithi018 ceph-mon[112590]: from='client.34136 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:01:55.100 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:01:54 smithi018 ceph-mon[112590]: pgmap v10: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 17 KiB/s rd, 0 B/s wr, 9 op/s 2024-04-12T21:01:55.101 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:01:54 smithi018 ceph-mon[112590]: from='client.34140 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:01:55.147 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:55.150+0000 7f52a6600700 1 -- 172.21.15.18:0/1571289084 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f52a01111d0 con 0x7f52a0070c10 2024-04-12T21:01:55.148 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:55.151+0000 7f529cc00700 1 -- 172.21.15.18:0/1571289084 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 23 v23) v1 ==== 76+0+1886 (secure 0 0 0) 0x7f528c05cb80 con 0x7f52a0070c10 2024-04-12T21:01:55.148 INFO:teuthology.orchestra.run.smithi018.stdout:e23 2024-04-12T21:01:55.148 INFO:teuthology.orchestra.run.smithi018.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-12T21:01:55.148 INFO:teuthology.orchestra.run.smithi018.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-12T21:01:55.148 INFO:teuthology.orchestra.run.smithi018.stdout:legacy client fscid: 1 2024-04-12T21:01:55.148 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-12T21:01:55.149 INFO:teuthology.orchestra.run.smithi018.stdout:Filesystem 'cephfs' (1) 2024-04-12T21:01:55.149 INFO:teuthology.orchestra.run.smithi018.stdout:fs_name cephfs 2024-04-12T21:01:55.149 INFO:teuthology.orchestra.run.smithi018.stdout:epoch 23 2024-04-12T21:01:55.149 INFO:teuthology.orchestra.run.smithi018.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-04-12T21:01:55.149 INFO:teuthology.orchestra.run.smithi018.stdout:created 2024-04-12T20:51:57.362850+0000 2024-04-12T21:01:55.149 INFO:teuthology.orchestra.run.smithi018.stdout:modified 2024-04-12T21:01:10.670287+0000 2024-04-12T21:01:55.149 INFO:teuthology.orchestra.run.smithi018.stdout:tableserver 0 2024-04-12T21:01:55.149 INFO:teuthology.orchestra.run.smithi018.stdout:root 0 2024-04-12T21:01:55.149 INFO:teuthology.orchestra.run.smithi018.stdout:session_timeout 60 2024-04-12T21:01:55.149 INFO:teuthology.orchestra.run.smithi018.stdout:session_autoclose 300 2024-04-12T21:01:55.149 INFO:teuthology.orchestra.run.smithi018.stdout:max_file_size 1099511627776 2024-04-12T21:01:55.149 INFO:teuthology.orchestra.run.smithi018.stdout:max_xattr_size 65536 2024-04-12T21:01:55.149 INFO:teuthology.orchestra.run.smithi018.stdout:required_client_features {} 2024-04-12T21:01:55.150 INFO:teuthology.orchestra.run.smithi018.stdout:last_failure 0 2024-04-12T21:01:55.150 INFO:teuthology.orchestra.run.smithi018.stdout:last_failure_osd_epoch 46 2024-04-12T21:01:55.150 INFO:teuthology.orchestra.run.smithi018.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-12T21:01:55.150 INFO:teuthology.orchestra.run.smithi018.stdout:max_mds 2 2024-04-12T21:01:55.150 INFO:teuthology.orchestra.run.smithi018.stdout:in 0,1 2024-04-12T21:01:55.150 INFO:teuthology.orchestra.run.smithi018.stdout:up {0=14452,1=14464} 2024-04-12T21:01:55.150 INFO:teuthology.orchestra.run.smithi018.stdout:failed 2024-04-12T21:01:55.150 INFO:teuthology.orchestra.run.smithi018.stdout:damaged 2024-04-12T21:01:55.150 INFO:teuthology.orchestra.run.smithi018.stdout:stopped 2024-04-12T21:01:55.150 INFO:teuthology.orchestra.run.smithi018.stdout:data_pools [3] 2024-04-12T21:01:55.150 INFO:teuthology.orchestra.run.smithi018.stdout:metadata_pool 2 2024-04-12T21:01:55.150 INFO:teuthology.orchestra.run.smithi018.stdout:inline_data disabled 2024-04-12T21:01:55.150 INFO:teuthology.orchestra.run.smithi018.stdout:balancer 2024-04-12T21:01:55.150 INFO:teuthology.orchestra.run.smithi018.stdout:bal_rank_mask -1 2024-04-12T21:01:55.151 INFO:teuthology.orchestra.run.smithi018.stdout:standby_count_wanted 1 2024-04-12T21:01:55.151 INFO:teuthology.orchestra.run.smithi018.stdout:qdb_cluster leader: 0 members: 2024-04-12T21:01:55.151 INFO:teuthology.orchestra.run.smithi018.stdout:[mds.cephfs.smithi018.acqbsc{0:14452} state up:active seq 5 join_fscid=1 addr [v2:172.21.15.18:6826/1564065434,v1:172.21.15.18:6827/1564065434] compat {c=[1],r=[1],i=[7ff]}] 2024-04-12T21:01:55.151 INFO:teuthology.orchestra.run.smithi018.stdout:[mds.cephfs.smithi018.szjluo{1:14464} state up:active seq 6 join_fscid=1 addr [v2:172.21.15.18:6828/722443353,v1:172.21.15.18:6829/722443353] compat {c=[1],r=[1],i=[7ff]}] 2024-04-12T21:01:55.151 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-12T21:01:55.151 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-12T21:01:55.151 INFO:teuthology.orchestra.run.smithi018.stdout:Standby daemons: 2024-04-12T21:01:55.151 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-12T21:01:55.151 INFO:teuthology.orchestra.run.smithi018.stdout:[mds.cephfs.smithi186.wjnafs{-1:24277} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.186:6826/438347207,v1:172.21.15.186:6827/438347207] compat {c=[1],r=[1],i=[7ff]}] 2024-04-12T21:01:55.151 INFO:teuthology.orchestra.run.smithi018.stdout:[mds.cephfs.smithi186.jdllyn{-1:24281} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.186:6824/1464359952,v1:172.21.15.186:6825/1464359952] compat {c=[1],r=[1],i=[7ff]}] 2024-04-12T21:01:55.152 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:55.155+0000 7f52a6600700 1 -- 172.21.15.18:0/1571289084 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f528806ed20 msgr2=0x7f52880711d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:01:55.152 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:55.155+0000 7f52a6600700 1 --2- 172.21.15.18:0/1571289084 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f528806ed20 0x7f52880711d0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f5294009fd0 tx=0x7f5294009ee0 comp rx=0 tx=0).stop 2024-04-12T21:01:55.153 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:55.155+0000 7f52a6600700 1 -- 172.21.15.18:0/1571289084 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f52a0070c10 msgr2=0x7f52a01160c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:01:55.153 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:55.155+0000 7f52a6600700 1 --2- 172.21.15.18:0/1571289084 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f52a0070c10 0x7f52a01160c0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f528c004f50 tx=0x7f528c009e30 comp rx=0 tx=0).stop 2024-04-12T21:01:55.153 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:55.156+0000 7f52a6600700 1 -- 172.21.15.18:0/1571289084 shutdown_connections 2024-04-12T21:01:55.153 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:55.156+0000 7f52a6600700 1 --2- 172.21.15.18:0/1571289084 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f528806ed20 0x7f52880711d0 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:01:55.153 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:55.156+0000 7f52a6600700 1 --2- 172.21.15.18:0/1571289084 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f52a0070300 0x7f52a0115b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:01:55.153 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:55.156+0000 7f52a6600700 1 --2- 172.21.15.18:0/1571289084 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f52a0070c10 0x7f52a01160c0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:01:55.153 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:55.156+0000 7f52a6600700 1 -- 172.21.15.18:0/1571289084 >> 172.21.15.18:0/1571289084 conn(0x7f52a00f87e0 msgr2=0x7f52a01014a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:01:55.153 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:55.156+0000 7f52a6600700 1 -- 172.21.15.18:0/1571289084 shutdown_connections 2024-04-12T21:01:55.153 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:55.156+0000 7f52a6600700 1 -- 172.21.15.18:0/1571289084 wait complete. 2024-04-12T21:01:55.154 INFO:teuthology.orchestra.run.smithi018.stderr:dumped fsmap epoch 23 2024-04-12T21:01:55.307 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:55.310+0000 7f662ac00700 1 -- 172.21.15.18:0/3301045756 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f66241037a0 msgr2=0x7f66240fce10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:01:55.307 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:55.310+0000 7f662ac00700 1 --2- 172.21.15.18:0/3301045756 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f66241037a0 0x7f66240fce10 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f6610004940 tx=0x7f6610009e30 comp rx=0 tx=0).stop 2024-04-12T21:01:55.307 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:55.310+0000 7f662ac00700 1 -- 172.21.15.18:0/3301045756 shutdown_connections 2024-04-12T21:01:55.307 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:55.310+0000 7f662ac00700 1 --2- 172.21.15.18:0/3301045756 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f66241037a0 0x7f66240fce10 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:01:55.307 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:55.310+0000 7f662ac00700 1 --2- 172.21.15.18:0/3301045756 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6624102e00 0x7f66241031d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:01:55.307 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:55.310+0000 7f662ac00700 1 -- 172.21.15.18:0/3301045756 >> 172.21.15.18:0/3301045756 conn(0x7f66240f89b0 msgr2=0x7f66240fadc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:01:55.307 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:55.310+0000 7f662ac00700 1 -- 172.21.15.18:0/3301045756 shutdown_connections 2024-04-12T21:01:55.307 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:55.310+0000 7f662ac00700 1 -- 172.21.15.18:0/3301045756 wait complete. 2024-04-12T21:01:55.308 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:55.311+0000 7f662ac00700 1 Processor -- start 2024-04-12T21:01:55.308 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:55.311+0000 7f662ac00700 1 -- start start 2024-04-12T21:01:55.308 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:55.312+0000 7f662ac00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6624102e00 0x7f66240702c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:01:55.308 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:55.312+0000 7f662ac00700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f66241037a0 0x7f6624070800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:01:55.308 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:55.312+0000 7f662ac00700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6624070d40 con 0x7f6624102e00 2024-04-12T21:01:55.308 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:55.312+0000 7f662ac00700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6624070e80 con 0x7f66241037a0 2024-04-12T21:01:55.309 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:55.312+0000 7f6623e00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6624102e00 0x7f66240702c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:01:55.309 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:55.312+0000 7f6623e00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6624102e00 0x7f66240702c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:54192/0 (socket says 172.21.15.18:54192) 2024-04-12T21:01:55.309 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:55.312+0000 7f6623e00700 1 -- 172.21.15.18:0/3766020148 learned_addr learned my addr 172.21.15.18:0/3766020148 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:01:55.309 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:55.312+0000 7f6623400700 1 --2- 172.21.15.18:0/3766020148 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f66241037a0 0x7f6624070800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:01:55.309 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:55.312+0000 7f6623e00700 1 -- 172.21.15.18:0/3766020148 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f66241037a0 msgr2=0x7f6624070800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:01:55.309 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:55.313+0000 7f6623e00700 1 --2- 172.21.15.18:0/3766020148 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f66241037a0 0x7f6624070800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:01:55.309 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:55.313+0000 7f6623e00700 1 -- 172.21.15.18:0/3766020148 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6610005040 con 0x7f6624102e00 2024-04-12T21:01:55.310 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:55.313+0000 7f6623e00700 1 --2- 172.21.15.18:0/3766020148 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6624102e00 0x7f66240702c0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f6614002a50 tx=0x7f66140046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:01:55.310 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:55.313+0000 7f661be00700 1 -- 172.21.15.18:0/3766020148 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6614007cc0 con 0x7f6624102e00 2024-04-12T21:01:55.310 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:55.313+0000 7f661be00700 1 -- 172.21.15.18:0/3766020148 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f661400f040 con 0x7f6624102e00 2024-04-12T21:01:55.310 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:55.313+0000 7f661be00700 1 -- 172.21.15.18:0/3766020148 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f661400da30 con 0x7f6624102e00 2024-04-12T21:01:55.310 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:55.313+0000 7f662ac00700 1 -- 172.21.15.18:0/3766020148 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6624071130 con 0x7f6624102e00 2024-04-12T21:01:55.310 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:55.313+0000 7f662ac00700 1 -- 172.21.15.18:0/3766020148 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f662411a2f0 con 0x7f6624102e00 2024-04-12T21:01:55.311 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:55.315+0000 7f662ac00700 1 -- 172.21.15.18:0/3766020148 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f66241052d0 con 0x7f6624102e00 2024-04-12T21:01:55.312 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:55.316+0000 7f661be00700 1 -- 172.21.15.18:0/3766020148 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 38) v1 ==== 92847+0+0 (secure 0 0 0) 0x7f661400dbb0 con 0x7f6624102e00 2024-04-12T21:01:55.313 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:55.317+0000 7f661be00700 1 --2- 172.21.15.18:0/3766020148 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f660806ed20 0x7f66080711d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:01:55.313 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:55.317+0000 7f661be00700 1 -- 172.21.15.18:0/3766020148 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6324+0+0 (secure 0 0 0) 0x7f661408e700 con 0x7f6624102e00 2024-04-12T21:01:55.314 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:55.317+0000 7f6623400700 1 --2- 172.21.15.18:0/3766020148 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f660806ed20 0x7f66080711d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:01:55.314 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:55.318+0000 7f6623400700 1 --2- 172.21.15.18:0/3766020148 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f660806ed20 0x7f66080711d0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f6624071bb0 tx=0x7f6610009e30 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:01:55.318 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:55.322+0000 7f661be00700 1 -- 172.21.15.18:0/3766020148 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f6614057290 con 0x7f6624102e00 2024-04-12T21:01:55.574 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:55.577+0000 7f662ac00700 1 -- 172.21.15.18:0/3766020148 --> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f6624071130 con 0x7f660806ed20 2024-04-12T21:01:55.576 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:55.580+0000 7f661be00700 1 -- 172.21.15.18:0/3766020148 <== mgr.34102 v2:172.21.15.18:6800/3812891919 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+337 (secure 0 0 0) 0x7f6624071130 con 0x7f660806ed20 2024-04-12T21:01:55.577 INFO:teuthology.orchestra.run.smithi018.stdout:{ 2024-04-12T21:01:55.577 INFO:teuthology.orchestra.run.smithi018.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:f4452def3be4d350f4a8e5feaf7ebb1511be9298fe68478ba7a2d6fbd7d69a63", 2024-04-12T21:01:55.577 INFO:teuthology.orchestra.run.smithi018.stdout: "in_progress": true, 2024-04-12T21:01:55.577 INFO:teuthology.orchestra.run.smithi018.stdout: "which": "Upgrading all daemon types on all hosts", 2024-04-12T21:01:55.577 INFO:teuthology.orchestra.run.smithi018.stdout: "services_complete": [ 2024-04-12T21:01:55.577 INFO:teuthology.orchestra.run.smithi018.stdout: "mgr" 2024-04-12T21:01:55.577 INFO:teuthology.orchestra.run.smithi018.stdout: ], 2024-04-12T21:01:55.577 INFO:teuthology.orchestra.run.smithi018.stdout: "progress": "3/21 daemons upgraded", 2024-04-12T21:01:55.577 INFO:teuthology.orchestra.run.smithi018.stdout: "message": "", 2024-04-12T21:01:55.577 INFO:teuthology.orchestra.run.smithi018.stdout: "is_paused": false 2024-04-12T21:01:55.577 INFO:teuthology.orchestra.run.smithi018.stdout:} 2024-04-12T21:01:55.579 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:55.582+0000 7f662ac00700 1 -- 172.21.15.18:0/3766020148 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f660806ed20 msgr2=0x7f66080711d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:01:55.579 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:55.582+0000 7f662ac00700 1 --2- 172.21.15.18:0/3766020148 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f660806ed20 0x7f66080711d0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f6624071bb0 tx=0x7f6610009e30 comp rx=0 tx=0).stop 2024-04-12T21:01:55.579 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:55.582+0000 7f662ac00700 1 -- 172.21.15.18:0/3766020148 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6624102e00 msgr2=0x7f66240702c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:01:55.579 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:55.582+0000 7f662ac00700 1 --2- 172.21.15.18:0/3766020148 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6624102e00 0x7f66240702c0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f6614002a50 tx=0x7f66140046b0 comp rx=0 tx=0).stop 2024-04-12T21:01:55.579 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:55.583+0000 7f662ac00700 1 -- 172.21.15.18:0/3766020148 shutdown_connections 2024-04-12T21:01:55.579 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:55.583+0000 7f662ac00700 1 --2- 172.21.15.18:0/3766020148 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f660806ed20 0x7f66080711d0 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:01:55.579 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:55.583+0000 7f662ac00700 1 --2- 172.21.15.18:0/3766020148 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6624102e00 0x7f66240702c0 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:01:55.579 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:55.583+0000 7f662ac00700 1 --2- 172.21.15.18:0/3766020148 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f66241037a0 0x7f6624070800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:01:55.580 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:55.583+0000 7f662ac00700 1 -- 172.21.15.18:0/3766020148 >> 172.21.15.18:0/3766020148 conn(0x7f66240f89b0 msgr2=0x7f6624102210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:01:55.580 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:55.583+0000 7f662ac00700 1 -- 172.21.15.18:0/3766020148 shutdown_connections 2024-04-12T21:01:55.580 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:55.583+0000 7f662ac00700 1 -- 172.21.15.18:0/3766020148 wait complete. 2024-04-12T21:01:55.731 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:55.734+0000 7f8846c00700 1 -- 172.21.15.18:0/1132491132 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f88401037a0 msgr2=0x7f88400fce10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:01:55.731 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:55.734+0000 7f8846c00700 1 --2- 172.21.15.18:0/1132491132 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f88401037a0 0x7f88400fce10 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f882c004940 tx=0x7f882c009e30 comp rx=0 tx=0).stop 2024-04-12T21:01:55.731 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:55.734+0000 7f8846c00700 1 -- 172.21.15.18:0/1132491132 shutdown_connections 2024-04-12T21:01:55.731 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:55.734+0000 7f8846c00700 1 --2- 172.21.15.18:0/1132491132 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f88401037a0 0x7f88400fce10 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:01:55.731 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:55.734+0000 7f8846c00700 1 --2- 172.21.15.18:0/1132491132 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8840102e00 0x7f88401031d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:01:55.731 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:55.734+0000 7f8846c00700 1 -- 172.21.15.18:0/1132491132 >> 172.21.15.18:0/1132491132 conn(0x7f88400f89b0 msgr2=0x7f88400fadc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:01:55.731 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:55.734+0000 7f8846c00700 1 -- 172.21.15.18:0/1132491132 shutdown_connections 2024-04-12T21:01:55.731 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:55.734+0000 7f8846c00700 1 -- 172.21.15.18:0/1132491132 wait complete. 2024-04-12T21:01:55.732 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:55.735+0000 7f8846c00700 1 Processor -- start 2024-04-12T21:01:55.732 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:55.735+0000 7f8846c00700 1 -- start start 2024-04-12T21:01:55.732 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:55.735+0000 7f8846c00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8840102e00 0x7f88400702c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:01:55.733 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:55.736+0000 7f8846c00700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f88401037a0 0x7f8840070800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:01:55.733 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:55.736+0000 7f8846c00700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8840070d40 con 0x7f8840102e00 2024-04-12T21:01:55.733 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:55.736+0000 7f8846c00700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8840070e80 con 0x7f88401037a0 2024-04-12T21:01:55.733 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:55.736+0000 7f883fe00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8840102e00 0x7f88400702c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:01:55.733 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:55.736+0000 7f883fe00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8840102e00 0x7f88400702c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:54212/0 (socket says 172.21.15.18:54212) 2024-04-12T21:01:55.733 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:55.736+0000 7f883fe00700 1 -- 172.21.15.18:0/4079899591 learned_addr learned my addr 172.21.15.18:0/4079899591 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:01:55.733 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:55.736+0000 7f883f400700 1 --2- 172.21.15.18:0/4079899591 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f88401037a0 0x7f8840070800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:01:55.734 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:55.736+0000 7f883fe00700 1 -- 172.21.15.18:0/4079899591 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f88401037a0 msgr2=0x7f8840070800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:01:55.734 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:55.736+0000 7f883fe00700 1 --2- 172.21.15.18:0/4079899591 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f88401037a0 0x7f8840070800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:01:55.734 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:55.736+0000 7f883fe00700 1 -- 172.21.15.18:0/4079899591 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f882c005040 con 0x7f8840102e00 2024-04-12T21:01:55.734 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:55.737+0000 7f883fe00700 1 --2- 172.21.15.18:0/4079899591 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8840102e00 0x7f88400702c0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f8830002a50 tx=0x7f88300046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:01:55.734 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:55.737+0000 7f8837e00700 1 -- 172.21.15.18:0/4079899591 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8830007cc0 con 0x7f8840102e00 2024-04-12T21:01:55.734 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:55.737+0000 7f8837e00700 1 -- 172.21.15.18:0/4079899591 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f883000f040 con 0x7f8840102e00 2024-04-12T21:01:55.734 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:55.737+0000 7f8846c00700 1 -- 172.21.15.18:0/4079899591 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8840071130 con 0x7f8840102e00 2024-04-12T21:01:55.734 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:55.737+0000 7f8837e00700 1 -- 172.21.15.18:0/4079899591 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f883000da30 con 0x7f8840102e00 2024-04-12T21:01:55.735 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:55.737+0000 7f8846c00700 1 -- 172.21.15.18:0/4079899591 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f884011a340 con 0x7f8840102e00 2024-04-12T21:01:55.735 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:55.739+0000 7f8846c00700 1 -- 172.21.15.18:0/4079899591 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f88401052d0 con 0x7f8840102e00 2024-04-12T21:01:55.736 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:55.739+0000 7f8837e00700 1 -- 172.21.15.18:0/4079899591 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 38) v1 ==== 92847+0+0 (secure 0 0 0) 0x7f883000db90 con 0x7f8840102e00 2024-04-12T21:01:55.737 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:55.740+0000 7f8837e00700 1 --2- 172.21.15.18:0/4079899591 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f882406ed20 0x7f88240711d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:01:55.737 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:55.740+0000 7f8837e00700 1 -- 172.21.15.18:0/4079899591 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6324+0+0 (secure 0 0 0) 0x7f883008e620 con 0x7f8840102e00 2024-04-12T21:01:55.737 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:55.741+0000 7f883f400700 1 --2- 172.21.15.18:0/4079899591 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f882406ed20 0x7f88240711d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:01:55.738 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:55.742+0000 7f883f400700 1 --2- 172.21.15.18:0/4079899591 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f882406ed20 0x7f88240711d0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f8840071bb0 tx=0x7f882c009e30 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:01:55.742 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:55.745+0000 7f8837e00700 1 -- 172.21.15.18:0/4079899591 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f8830057230 con 0x7f8840102e00 2024-04-12T21:01:56.030 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:01:55 smithi186 ceph-mon[36597]: from='client.? 172.21.15.18:0/991104961' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T21:01:56.030 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:01:55 smithi186 ceph-mon[36597]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T21:01:56.030 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:01:55 smithi186 ceph-mon[36597]: from='client.? 172.21.15.18:0/1571289084' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-12T21:01:56.031 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:01:55 smithi186 ceph-mon[36597]: Detected new or changed devices on smithi018 2024-04-12T21:01:56.031 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:01:55 smithi186 ceph-mon[36597]: from='mgr.34102 ' entity='mgr.smithi018.xjzwga' 2024-04-12T21:01:56.031 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:01:55 smithi186 ceph-mon[36597]: from='mgr.34102 ' entity='mgr.smithi018.xjzwga' 2024-04-12T21:01:56.031 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:01:55 smithi186 ceph-mon[36597]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-12T21:01:56.031 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:01:55 smithi186 ceph-mon[36597]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-12T21:01:56.031 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:01:55 smithi186 ceph-mon[36597]: Updating smithi018:/etc/ceph/ceph.conf 2024-04-12T21:01:56.031 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:01:55 smithi186 ceph-mon[36597]: Updating smithi186:/etc/ceph/ceph.conf 2024-04-12T21:01:56.075 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:56.078+0000 7f8846c00700 1 -- 172.21.15.18:0/4079899591 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f88400fe550 con 0x7f8840102e00 2024-04-12T21:01:56.075 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:01:55 smithi018 ceph-mon[112590]: from='client.? 172.21.15.18:0/991104961' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T21:01:56.075 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:01:55 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T21:01:56.075 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:01:55 smithi018 ceph-mon[112590]: from='client.? 172.21.15.18:0/1571289084' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-12T21:01:56.075 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:01:55 smithi018 ceph-mon[112590]: Detected new or changed devices on smithi018 2024-04-12T21:01:56.075 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:01:55 smithi018 ceph-mon[112590]: from='mgr.34102 ' entity='mgr.smithi018.xjzwga' 2024-04-12T21:01:56.076 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:01:55 smithi018 ceph-mon[112590]: from='mgr.34102 ' entity='mgr.smithi018.xjzwga' 2024-04-12T21:01:56.076 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:01:55 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-12T21:01:56.076 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:01:55 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-12T21:01:56.076 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:01:55 smithi018 ceph-mon[112590]: Updating smithi018:/etc/ceph/ceph.conf 2024-04-12T21:01:56.076 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:01:55 smithi018 ceph-mon[112590]: Updating smithi186:/etc/ceph/ceph.conf 2024-04-12T21:01:56.076 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:56.079+0000 7f8837e00700 1 -- 172.21.15.18:0/4079899591 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+10 (secure 0 0 0) 0x7f883005a8f0 con 0x7f8840102e00 2024-04-12T21:01:56.076 INFO:teuthology.orchestra.run.smithi018.stdout:HEALTH_OK 2024-04-12T21:01:56.080 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:56.083+0000 7f8846c00700 1 -- 172.21.15.18:0/4079899591 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f882406ed20 msgr2=0x7f88240711d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:01:56.080 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:56.083+0000 7f8846c00700 1 --2- 172.21.15.18:0/4079899591 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f882406ed20 0x7f88240711d0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f8840071bb0 tx=0x7f882c009e30 comp rx=0 tx=0).stop 2024-04-12T21:01:56.080 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:56.083+0000 7f8846c00700 1 -- 172.21.15.18:0/4079899591 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8840102e00 msgr2=0x7f88400702c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:01:56.080 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:56.083+0000 7f8846c00700 1 --2- 172.21.15.18:0/4079899591 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8840102e00 0x7f88400702c0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f8830002a50 tx=0x7f88300046b0 comp rx=0 tx=0).stop 2024-04-12T21:01:56.080 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:56.083+0000 7f8846c00700 1 -- 172.21.15.18:0/4079899591 shutdown_connections 2024-04-12T21:01:56.080 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:56.083+0000 7f8846c00700 1 --2- 172.21.15.18:0/4079899591 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f882406ed20 0x7f88240711d0 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:01:56.080 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:56.083+0000 7f8846c00700 1 --2- 172.21.15.18:0/4079899591 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8840102e00 0x7f88400702c0 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:01:56.080 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:56.084+0000 7f8846c00700 1 --2- 172.21.15.18:0/4079899591 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f88401037a0 0x7f8840070800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:01:56.080 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:56.084+0000 7f8846c00700 1 -- 172.21.15.18:0/4079899591 >> 172.21.15.18:0/4079899591 conn(0x7f88400f89b0 msgr2=0x7f8840102210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:01:56.081 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:56.084+0000 7f8846c00700 1 -- 172.21.15.18:0/4079899591 shutdown_connections 2024-04-12T21:01:56.081 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:01:56.084+0000 7f8846c00700 1 -- 172.21.15.18:0/4079899591 wait complete. 2024-04-12T21:01:57.030 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:01:56 smithi186 ceph-mon[36597]: from='client.34152 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:01:57.030 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:01:56 smithi186 ceph-mon[36597]: pgmap v11: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 17 KiB/s rd, 0 B/s wr, 9 op/s 2024-04-12T21:01:57.030 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:01:56 smithi186 ceph-mon[36597]: Updating smithi018:/var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/config/ceph.conf 2024-04-12T21:01:57.030 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:01:56 smithi186 ceph-mon[36597]: Updating smithi186:/var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/config/ceph.conf 2024-04-12T21:01:57.030 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:01:56 smithi186 ceph-mon[36597]: from='client.? 172.21.15.18:0/4079899591' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-12T21:01:57.101 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:01:56 smithi018 ceph-mon[112590]: from='client.34152 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:01:57.101 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:01:56 smithi018 ceph-mon[112590]: pgmap v11: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 17 KiB/s rd, 0 B/s wr, 9 op/s 2024-04-12T21:01:57.101 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:01:56 smithi018 ceph-mon[112590]: Updating smithi018:/var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/config/ceph.conf 2024-04-12T21:01:57.101 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:01:56 smithi018 ceph-mon[112590]: Updating smithi186:/var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/config/ceph.conf 2024-04-12T21:01:57.101 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:01:56 smithi018 ceph-mon[112590]: from='client.? 172.21.15.18:0/4079899591' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-12T21:01:58.030 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:01:57 smithi186 ceph-mon[36597]: Updating smithi018:/etc/ceph/ceph.client.admin.keyring 2024-04-12T21:01:58.030 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:01:57 smithi186 ceph-mon[36597]: Updating smithi186:/etc/ceph/ceph.client.admin.keyring 2024-04-12T21:01:58.030 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:01:57 smithi186 ceph-mon[36597]: Updating smithi018:/var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/config/ceph.client.admin.keyring 2024-04-12T21:01:58.100 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:01:57 smithi018 ceph-mon[112590]: Updating smithi018:/etc/ceph/ceph.client.admin.keyring 2024-04-12T21:01:58.101 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:01:57 smithi018 ceph-mon[112590]: Updating smithi186:/etc/ceph/ceph.client.admin.keyring 2024-04-12T21:01:58.101 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:01:57 smithi018 ceph-mon[112590]: Updating smithi018:/var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/config/ceph.client.admin.keyring 2024-04-12T21:01:59.529 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:01:59 smithi186 ceph-mon[36597]: Updating smithi186:/var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/config/ceph.client.admin.keyring 2024-04-12T21:01:59.530 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:01:59 smithi186 ceph-mon[36597]: pgmap v12: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 17 KiB/s rd, 0 B/s wr, 9 op/s 2024-04-12T21:01:59.530 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:01:59 smithi186 ceph-mon[36597]: from='mgr.34102 ' entity='mgr.smithi018.xjzwga' 2024-04-12T21:01:59.530 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:01:59 smithi186 ceph-mon[36597]: from='mgr.34102 ' entity='mgr.smithi018.xjzwga' 2024-04-12T21:01:59.530 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:01:59 smithi186 ceph-mon[36597]: from='mgr.34102 ' entity='mgr.smithi018.xjzwga' 2024-04-12T21:01:59.530 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:01:59 smithi186 ceph-mon[36597]: from='mgr.34102 ' entity='mgr.smithi018.xjzwga' 2024-04-12T21:01:59.530 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:01:59 smithi186 ceph-mon[36597]: from='mgr.34102 ' entity='mgr.smithi018.xjzwga' 2024-04-12T21:01:59.530 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:01:59 smithi186 ceph-mon[36597]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-12T21:01:59.530 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:01:59 smithi186 ceph-mon[36597]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T21:01:59.530 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:01:59 smithi186 ceph-mon[36597]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "quorum_status"}]: dispatch 2024-04-12T21:01:59.601 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:01:59 smithi018 ceph-mon[112590]: Updating smithi186:/var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/config/ceph.client.admin.keyring 2024-04-12T21:01:59.601 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:01:59 smithi018 ceph-mon[112590]: pgmap v12: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 17 KiB/s rd, 0 B/s wr, 9 op/s 2024-04-12T21:01:59.601 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:01:59 smithi018 ceph-mon[112590]: from='mgr.34102 ' entity='mgr.smithi018.xjzwga' 2024-04-12T21:01:59.601 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:01:59 smithi018 ceph-mon[112590]: from='mgr.34102 ' entity='mgr.smithi018.xjzwga' 2024-04-12T21:01:59.601 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:01:59 smithi018 ceph-mon[112590]: from='mgr.34102 ' entity='mgr.smithi018.xjzwga' 2024-04-12T21:01:59.601 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:01:59 smithi018 ceph-mon[112590]: from='mgr.34102 ' entity='mgr.smithi018.xjzwga' 2024-04-12T21:01:59.601 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:01:59 smithi018 ceph-mon[112590]: from='mgr.34102 ' entity='mgr.smithi018.xjzwga' 2024-04-12T21:01:59.601 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:01:59 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-12T21:01:59.601 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:01:59 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T21:01:59.602 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:01:59 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "quorum_status"}]: dispatch 2024-04-12T21:02:01.530 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:02:01 smithi186 ceph-mon[36597]: pgmap v13: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-12T21:02:01.530 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:02:01 smithi186 ceph-mon[36597]: from='mgr.34102 ' entity='mgr.smithi018.xjzwga' 2024-04-12T21:02:01.530 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:02:01 smithi186 ceph-mon[36597]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-12T21:02:01.530 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:02:01 smithi186 ceph-mon[36597]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-12T21:02:01.530 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:02:01 smithi186 ceph-mon[36597]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-12T21:02:01.601 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:02:01 smithi018 ceph-mon[112590]: pgmap v13: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-12T21:02:01.601 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:02:01 smithi018 ceph-mon[112590]: from='mgr.34102 ' entity='mgr.smithi018.xjzwga' 2024-04-12T21:02:01.601 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:02:01 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-12T21:02:01.601 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:02:01 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-12T21:02:01.601 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:02:01 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-12T21:02:02.529 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:02:02 smithi186 ceph-mon[36597]: Upgrade: Updating mon.smithi186 2024-04-12T21:02:02.530 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:02:02 smithi186 ceph-mon[36597]: Deploying daemon mon.smithi186 on smithi186 2024-04-12T21:02:02.601 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:02:02 smithi018 ceph-mon[112590]: Upgrade: Updating mon.smithi186 2024-04-12T21:02:02.601 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:02:02 smithi018 ceph-mon[112590]: Deploying daemon mon.smithi186 on smithi186 2024-04-12T21:02:03.529 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:02:03 smithi186 ceph-mon[36597]: pgmap v14: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-12T21:02:03.530 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:02:03 smithi186 systemd[1]: Stopping Ceph mon.smithi186 for 1cbe407a-f90d-11ee-bc8d-c7b262605968... 2024-04-12T21:02:03.601 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:02:03 smithi018 ceph-mon[112590]: pgmap v14: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-12T21:02:04.143 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:02:03 smithi186 ceph-1cbe407a-f90d-11ee-bc8d-c7b262605968-mon-smithi186[36586]: 2024-04-12T21:02:03.877+0000 7fd3d8600700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.smithi186 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-04-12T21:02:04.143 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:02:03 smithi186 ceph-1cbe407a-f90d-11ee-bc8d-c7b262605968-mon-smithi186[36586]: 2024-04-12T21:02:03.877+0000 7fd3d8600700 -1 mon.smithi186@1(peon) e2 *** Got Signal Terminated *** 2024-04-12T21:02:04.421 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:02:04 smithi186 podman[86219]: 2024-04-12 21:02:04.145437854 +0000 UTC m=+0.384297104 container stop d7263c772b737048a211b0d8729e54a447612bd7a9019ad582ef84c8def87867 (image=quay.ceph.io/ceph-ci/ceph@sha256:5e334df698f432180b78bc9c46936bfdfbc92bc9a8983463e523f780d23765fe, name=ceph-1cbe407a-f90d-11ee-bc8d-c7b262605968-mon-smithi186, io.buildah.version=1.33.5, org.label-schema.build-date=20240409, RELEASE=quincy-d3b5060, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_CLEAN=True) 2024-04-12T21:02:04.422 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:02:04 smithi186 podman[86219]: 2024-04-12 21:02:04.173432808 +0000 UTC m=+0.412292011 container died d7263c772b737048a211b0d8729e54a447612bd7a9019ad582ef84c8def87867 (image=quay.ceph.io/ceph-ci/ceph@sha256:5e334df698f432180b78bc9c46936bfdfbc92bc9a8983463e523f780d23765fe, name=ceph-1cbe407a-f90d-11ee-bc8d-c7b262605968-mon-smithi186, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240409, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, RELEASE=quincy-d3b5060, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_CLEAN=True) 2024-04-12T21:02:04.780 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:02:04 smithi186 podman[86219]: 2024-04-12 21:02:04.537337971 +0000 UTC m=+0.776197234 container cleanup d7263c772b737048a211b0d8729e54a447612bd7a9019ad582ef84c8def87867 (image=quay.ceph.io/ceph-ci/ceph@sha256:5e334df698f432180b78bc9c46936bfdfbc92bc9a8983463e523f780d23765fe, name=ceph-1cbe407a-f90d-11ee-bc8d-c7b262605968-mon-smithi186, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, ceph=True, maintainer=Guillaume Abrioux , GIT_CLEAN=True, RELEASE=quincy-d3b5060, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240409, org.label-schema.license=GPLv2, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5) 2024-04-12T21:02:04.780 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:02:04 smithi186 bash[86219]: ceph-1cbe407a-f90d-11ee-bc8d-c7b262605968-mon-smithi186 2024-04-12T21:02:05.280 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:02:04 smithi186 podman[86233]: 2024-04-12 21:02:04.78867359 +0000 UTC m=+0.623353443 container remove d7263c772b737048a211b0d8729e54a447612bd7a9019ad582ef84c8def87867 (image=quay.ceph.io/ceph-ci/ceph@sha256:5e334df698f432180b78bc9c46936bfdfbc92bc9a8983463e523f780d23765fe, name=ceph-1cbe407a-f90d-11ee-bc8d-c7b262605968-mon-smithi186, org.label-schema.license=GPLv2, RELEASE=quincy-d3b5060, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.build-date=20240409, ceph=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD) 2024-04-12T21:02:05.746 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:02:05 smithi186 systemd[1]: ceph-1cbe407a-f90d-11ee-bc8d-c7b262605968@mon.smithi186.service: Deactivated successfully. 2024-04-12T21:02:05.746 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:02:05 smithi186 systemd[1]: Stopped Ceph mon.smithi186 for 1cbe407a-f90d-11ee-bc8d-c7b262605968. 2024-04-12T21:02:05.746 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:02:05 smithi186 systemd[1]: ceph-1cbe407a-f90d-11ee-bc8d-c7b262605968@mon.smithi186.service: Consumed 18.595s CPU time. 2024-04-12T21:02:06.029 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:02:05 smithi186 systemd[1]: Starting Ceph mon.smithi186 for 1cbe407a-f90d-11ee-bc8d-c7b262605968... 2024-04-12T21:02:06.416 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:02:06 smithi186 podman[86361]: 2024-04-12 21:02:06.004471525 +0000 UTC m=+0.037333319 image pull 8717720dc5e5c56303818ea4d4f0a11f6e2c03dee521fb668f411f1304e0afca quay.ceph.io/ceph-ci/ceph@sha256:f4452def3be4d350f4a8e5feaf7ebb1511be9298fe68478ba7a2d6fbd7d69a63 2024-04-12T21:02:06.416 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:02:06 smithi186 podman[86361]: 2024-04-12 21:02:06.131603007 +0000 UTC m=+0.164464831 container create c77498da9b2bf5febb1767de4918612d9c9398a028542311285e3b9dcdc4e7a6 (image=quay.ceph.io/ceph-ci/ceph@sha256:f4452def3be4d350f4a8e5feaf7ebb1511be9298fe68478ba7a2d6fbd7d69a63, name=ceph-1cbe407a-f90d-11ee-bc8d-c7b262605968-mon-smithi186, RELEASE=wip-vshankar-testing-20240411.061452-324baf8, maintainer=Guillaume Abrioux , ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240409, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_CLEAN=True) 2024-04-12T21:02:06.416 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:02:06 smithi186 podman[86361]: 2024-04-12 21:02:06.418900225 +0000 UTC m=+0.451762049 container init c77498da9b2bf5febb1767de4918612d9c9398a028542311285e3b9dcdc4e7a6 (image=quay.ceph.io/ceph-ci/ceph@sha256:f4452def3be4d350f4a8e5feaf7ebb1511be9298fe68478ba7a2d6fbd7d69a63, name=ceph-1cbe407a-f90d-11ee-bc8d-c7b262605968-mon-smithi186, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-vshankar-testing-20240411.061452-324baf8, org.label-schema.build-date=20240409, org.label-schema.vendor=CentOS) 2024-04-12T21:02:06.782 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:02:06 smithi186 podman[86361]: 2024-04-12 21:02:06.43109646 +0000 UTC m=+0.463958251 container start c77498da9b2bf5febb1767de4918612d9c9398a028542311285e3b9dcdc4e7a6 (image=quay.ceph.io/ceph-ci/ceph@sha256:f4452def3be4d350f4a8e5feaf7ebb1511be9298fe68478ba7a2d6fbd7d69a63, name=ceph-1cbe407a-f90d-11ee-bc8d-c7b262605968-mon-smithi186, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240409, RELEASE=wip-vshankar-testing-20240411.061452-324baf8, ceph=True) 2024-04-12T21:02:06.783 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:02:06 smithi186 ceph-mon[86379]: set uid:gid to 167:167 (ceph:ceph) 2024-04-12T21:02:06.783 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:02:06 smithi186 ceph-mon[86379]: ceph version 19.0.0-2879-g324baf8d (324baf8ddb91dea46cac50cb535c306115791355) squid (dev), process ceph-mon, pid 2 2024-04-12T21:02:06.783 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:02:06 smithi186 ceph-mon[86379]: pidfile_write: ignore empty --pid-file 2024-04-12T21:02:06.783 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:02:06 smithi186 ceph-mon[86379]: load: jerasure load: lrc 2024-04-12T21:02:06.783 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:02:06 smithi186 ceph-mon[86379]: rocksdb: RocksDB version: 7.9.2 2024-04-12T21:02:06.783 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:02:06 smithi186 ceph-mon[86379]: rocksdb: Git sha 0 2024-04-12T21:02:06.783 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:02:06 smithi186 ceph-mon[86379]: rocksdb: Compile date 2024-04-11 06:46:50 2024-04-12T21:02:06.783 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:02:06 smithi186 ceph-mon[86379]: rocksdb: DB SUMMARY 2024-04-12T21:02:06.783 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:02:06 smithi186 ceph-mon[86379]: rocksdb: DB Session ID: DBWXCQLZC0X5VZ7LVT1Z 2024-04-12T21:02:06.783 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:02:06 smithi186 ceph-mon[86379]: rocksdb: CURRENT file: CURRENT 2024-04-12T21:02:06.783 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:02:06 smithi186 ceph-mon[86379]: rocksdb: IDENTITY file: IDENTITY 2024-04-12T21:02:06.783 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:02:06 smithi186 ceph-mon[86379]: rocksdb: MANIFEST file: MANIFEST-000009 size: 1463 Bytes 2024-04-12T21:02:06.784 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:02:06 smithi186 ceph-mon[86379]: rocksdb: SST files in /var/lib/ceph/mon/ceph-smithi186/store.db dir, Total Num: 1, files: 000033.sst 2024-04-12T21:02:06.784 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:02:06 smithi186 ceph-mon[86379]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-smithi186/store.db: 000031.log size: 482799 ; 2024-04-12T21:02:06.784 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:02:06 smithi186 ceph-mon[86379]: rocksdb: Options.error_if_exists: 0 2024-04-12T21:02:06.784 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:02:06 smithi186 ceph-mon[86379]: rocksdb: Options.create_if_missing: 0 2024-04-12T21:02:06.784 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:02:06 smithi186 ceph-mon[86379]: rocksdb: Options.paranoid_checks: 1 2024-04-12T21:02:06.784 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:02:06 smithi186 ceph-mon[86379]: rocksdb: Options.flush_verify_memtable_count: 1 2024-04-12T21:02:06.784 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:02:06 smithi186 ceph-mon[86379]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-04-12T21:02:06.784 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:02:06 smithi186 ceph-mon[86379]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-04-12T21:02:06.784 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:02:06 smithi186 ceph-mon[86379]: rocksdb: Options.env: 0x5582bbe29cc0 2024-04-12T21:02:06.784 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:02:06 smithi186 ceph-mon[86379]: rocksdb: Options.fs: PosixFileSystem 2024-04-12T21:02:06.784 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:02:06 smithi186 ceph-mon[86379]: rocksdb: Options.info_log: 0x5582f51a5ae0 2024-04-12T21:02:06.784 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:02:06 smithi186 ceph-mon[86379]: rocksdb: Options.max_file_opening_threads: 16 2024-04-12T21:02:06.784 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:02:06 smithi186 ceph-mon[86379]: rocksdb: Options.statistics: (nil) 2024-04-12T21:02:06.784 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:02:06 smithi186 ceph-mon[86379]: rocksdb: Options.use_fsync: 0 2024-04-12T21:02:06.785 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:02:06 smithi186 ceph-mon[86379]: rocksdb: Options.max_log_file_size: 0 2024-04-12T21:02:06.785 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:02:06 smithi186 ceph-mon[86379]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-04-12T21:02:06.785 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:02:06 smithi186 ceph-mon[86379]: rocksdb: Options.log_file_time_to_roll: 0 2024-04-12T21:02:06.785 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:02:06 smithi186 ceph-mon[86379]: rocksdb: Options.keep_log_file_num: 1000 2024-04-12T21:02:06.785 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:02:06 smithi186 ceph-mon[86379]: rocksdb: Options.recycle_log_file_num: 0 2024-04-12T21:02:06.785 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:02:06 smithi186 ceph-mon[86379]: rocksdb: Options.allow_fallocate: 1 2024-04-12T21:02:06.785 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:02:06 smithi186 ceph-mon[86379]: rocksdb: Options.allow_mmap_reads: 0 2024-04-12T21:02:06.785 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:02:06 smithi186 ceph-mon[86379]: rocksdb: Options.allow_mmap_writes: 0 2024-04-12T21:02:06.785 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:02:06 smithi186 ceph-mon[86379]: rocksdb: Options.use_direct_reads: 0 2024-04-12T21:02:06.786 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:02:06 smithi186 ceph-mon[86379]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-04-12T21:02:06.786 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:02:06 smithi186 ceph-mon[86379]: rocksdb: Options.create_missing_column_families: 0 2024-04-12T21:02:06.786 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:02:06 smithi186 ceph-mon[86379]: rocksdb: Options.db_log_dir: 2024-04-12T21:02:06.786 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:02:06 smithi186 ceph-mon[86379]: rocksdb: Options.wal_dir: 2024-04-12T21:02:06.786 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:02:06 smithi186 ceph-mon[86379]: rocksdb: Options.table_cache_numshardbits: 6 2024-04-12T21:02:06.786 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:02:06 smithi186 ceph-mon[86379]: rocksdb: Options.WAL_ttl_seconds: 0 2024-04-12T21:02:06.786 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:02:06 smithi186 ceph-mon[86379]: rocksdb: Options.WAL_size_limit_MB: 0 2024-04-12T21:02:06.786 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:02:06 smithi186 ceph-mon[86379]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-04-12T21:02:06.786 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:02:06 smithi186 ceph-mon[86379]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-04-12T21:02:06.786 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:02:06 smithi186 ceph-mon[86379]: rocksdb: Options.is_fd_close_on_exec: 1 2024-04-12T21:02:06.786 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:02:06 smithi186 ceph-mon[86379]: rocksdb: Options.advise_random_on_open: 1 2024-04-12T21:02:06.786 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:02:06 smithi186 ceph-mon[86379]: rocksdb: Options.db_write_buffer_size: 0 2024-04-12T21:02:06.786 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:02:06 smithi186 ceph-mon[86379]: rocksdb: Options.write_buffer_manager: 0x5582f51761e0 2024-04-12T21:02:06.787 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:02:06 smithi186 ceph-mon[86379]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-04-12T21:02:06.787 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:02:06 smithi186 ceph-mon[86379]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-04-12T21:02:06.787 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:02:06 smithi186 ceph-mon[86379]: rocksdb: Options.use_adaptive_mutex: 0 2024-04-12T21:02:06.787 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:02:06 smithi186 ceph-mon[86379]: rocksdb: Options.rate_limiter: (nil) 2024-04-12T21:02:06.787 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:02:06 smithi186 ceph-mon[86379]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-04-12T21:02:06.787 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:02:06 smithi186 ceph-mon[86379]: rocksdb: Options.wal_recovery_mode: 2 2024-04-12T21:02:06.787 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:02:06 smithi186 ceph-mon[86379]: rocksdb: Options.enable_thread_tracking: 0 2024-04-12T21:02:06.787 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:02:06 smithi186 ceph-mon[86379]: rocksdb: Options.enable_pipelined_write: 0 2024-04-12T21:02:06.787 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:02:06 smithi186 ceph-mon[86379]: rocksdb: Options.unordered_write: 0 2024-04-12T21:02:06.787 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:02:06 smithi186 ceph-mon[86379]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-04-12T21:02:06.787 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:02:06 smithi186 ceph-mon[86379]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-04-12T21:02:06.787 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:02:06 smithi186 ceph-mon[86379]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-04-12T21:02:06.787 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:02:06 smithi186 ceph-mon[86379]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-04-12T21:02:06.787 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:02:06 smithi186 ceph-mon[86379]: rocksdb: Options.row_cache: None 2024-04-12T21:02:06.787 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:02:06 smithi186 ceph-mon[86379]: rocksdb: Options.wal_filter: None 2024-04-12T21:02:06.788 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:02:06 smithi186 ceph-mon[86379]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-04-12T21:02:06.788 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:02:06 smithi186 ceph-mon[86379]: rocksdb: Options.allow_ingest_behind: 0 2024-04-12T21:02:06.788 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:02:06 smithi186 ceph-mon[86379]: rocksdb: Options.two_write_queues: 0 2024-04-12T21:02:06.788 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:02:06 smithi186 ceph-mon[86379]: rocksdb: Options.manual_wal_flush: 0 2024-04-12T21:02:06.788 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:02:06 smithi186 ceph-mon[86379]: rocksdb: Options.wal_compression: 0 2024-04-12T21:02:06.788 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:02:06 smithi186 ceph-mon[86379]: rocksdb: Options.atomic_flush: 0 2024-04-12T21:02:06.788 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:02:06 smithi186 ceph-mon[86379]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-04-12T21:02:06.788 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:02:06 smithi186 ceph-mon[86379]: rocksdb: Options.persist_stats_to_disk: 0 2024-04-12T21:02:06.788 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:02:06 smithi186 ceph-mon[86379]: rocksdb: Options.write_dbid_to_manifest: 0 2024-04-12T21:02:06.789 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:02:06 smithi186 ceph-mon[86379]: rocksdb: Options.log_readahead_size: 0 2024-04-12T21:02:06.789 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:02:06 smithi186 ceph-mon[86379]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-04-12T21:02:06.789 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:02:06 smithi186 ceph-mon[86379]: rocksdb: Options.best_efforts_recovery: 0 2024-04-12T21:02:06.789 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:02:06 smithi186 ceph-mon[86379]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-04-12T21:02:06.789 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:02:06 smithi186 ceph-mon[86379]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-04-12T21:02:06.789 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:02:06 smithi186 ceph-mon[86379]: rocksdb: Options.allow_data_in_errors: 0 2024-04-12T21:02:06.789 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:02:06 smithi186 ceph-mon[86379]: rocksdb: Options.db_host_id: __hostname__ 2024-04-12T21:02:06.789 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:02:06 smithi186 ceph-mon[86379]: rocksdb: Options.enforce_single_del_contracts: true 2024-04-12T21:02:06.789 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:02:06 smithi186 ceph-mon[86379]: rocksdb: Options.max_background_jobs: 2 2024-04-12T21:02:06.789 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:02:06 smithi186 ceph-mon[86379]: rocksdb: Options.max_background_compactions: -1 2024-04-12T21:02:06.789 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:02:06 smithi186 ceph-mon[86379]: rocksdb: Options.max_subcompactions: 1 2024-04-12T21:02:06.789 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:02:06 smithi186 ceph-mon[86379]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-04-12T21:02:06.789 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:02:06 smithi186 ceph-mon[86379]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-04-12T21:02:06.789 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:02:06 smithi186 ceph-mon[86379]: rocksdb: Options.delayed_write_rate : 16777216 2024-04-12T21:02:06.790 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:02:06 smithi186 ceph-mon[86379]: rocksdb: Options.max_total_wal_size: 0 2024-04-12T21:02:06.790 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:02:06 smithi186 ceph-mon[86379]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-04-12T21:02:06.790 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:02:06 smithi186 ceph-mon[86379]: rocksdb: Options.stats_dump_period_sec: 600 2024-04-12T21:02:06.790 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:02:06 smithi186 ceph-mon[86379]: rocksdb: Options.stats_persist_period_sec: 600 2024-04-12T21:02:06.790 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:02:06 smithi186 ceph-mon[86379]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-04-12T21:02:06.790 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:02:06 smithi186 ceph-mon[86379]: rocksdb: Options.max_open_files: -1 2024-04-12T21:02:06.790 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:02:06 smithi186 ceph-mon[86379]: rocksdb: Options.bytes_per_sync: 0 2024-04-12T21:02:06.790 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:02:06 smithi186 ceph-mon[86379]: rocksdb: Options.wal_bytes_per_sync: 0 2024-04-12T21:02:06.790 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:02:06 smithi186 ceph-mon[86379]: rocksdb: Options.strict_bytes_per_sync: 0 2024-04-12T21:02:06.790 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:02:06 smithi186 ceph-mon[86379]: rocksdb: Options.compaction_readahead_size: 0 2024-04-12T21:02:06.790 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:02:06 smithi186 ceph-mon[86379]: rocksdb: Options.max_background_flushes: -1 2024-04-12T21:02:06.790 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:02:06 smithi186 ceph-mon[86379]: rocksdb: Compression algorithms supported: 2024-04-12T21:02:06.790 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:02:06 smithi186 ceph-mon[86379]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-04-12T21:02:06.790 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:02:06 smithi186 ceph-mon[86379]: rocksdb: kZSTD supported: 0 2024-04-12T21:02:06.790 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:02:06 smithi186 ceph-mon[86379]: rocksdb: kXpressCompression supported: 0 2024-04-12T21:02:06.790 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:02:06 smithi186 ceph-mon[86379]: rocksdb: kLZ4HCCompression supported: 1 2024-04-12T21:02:06.791 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:02:06 smithi186 ceph-mon[86379]: rocksdb: kZlibCompression supported: 1 2024-04-12T21:02:06.791 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:02:06 smithi186 ceph-mon[86379]: rocksdb: kSnappyCompression supported: 1 2024-04-12T21:02:06.791 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:02:06 smithi186 ceph-mon[86379]: rocksdb: kLZ4Compression supported: 1 2024-04-12T21:02:06.791 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:02:06 smithi186 ceph-mon[86379]: rocksdb: kBZip2Compression supported: 0 2024-04-12T21:02:06.791 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:02:06 smithi186 ceph-mon[86379]: rocksdb: Fast CRC32 supported: Supported on x86 2024-04-12T21:02:06.791 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:02:06 smithi186 ceph-mon[86379]: rocksdb: DMutex implementation: pthread_mutex_t 2024-04-12T21:02:06.791 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:02:06 smithi186 ceph-mon[86379]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-smithi186/store.db/MANIFEST-000009 2024-04-12T21:02:06.791 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:02:06 smithi186 ceph-mon[86379]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-04-12T21:02:06.791 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:02:06 smithi186 ceph-mon[86379]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-04-12T21:02:06.791 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:02:06 smithi186 ceph-mon[86379]: rocksdb: Options.merge_operator: 2024-04-12T21:02:06.792 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:02:06 smithi186 ceph-mon[86379]: rocksdb: Options.compaction_filter: None 2024-04-12T21:02:06.792 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:02:06 smithi186 ceph-mon[86379]: rocksdb: Options.compaction_filter_factory: None 2024-04-12T21:02:06.792 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:02:06 smithi186 ceph-mon[86379]: rocksdb: Options.sst_partitioner_factory: None 2024-04-12T21:02:06.792 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:02:06 smithi186 ceph-mon[86379]: rocksdb: Options.memtable_factory: SkipListFactory 2024-04-12T21:02:06.792 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:02:06 smithi186 ceph-mon[86379]: rocksdb: Options.table_factory: BlockBasedTable 2024-04-12T21:02:06.792 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:02:06 smithi186 ceph-mon[86379]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x5582f51a5cc0) 2024-04-12T21:02:06.792 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout: cache_index_and_filter_blocks: 1 2024-04-12T21:02:06.792 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-04-12T21:02:06.792 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-04-12T21:02:06.793 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout: pin_top_level_index_and_filter: 1 2024-04-12T21:02:06.793 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout: index_type: 0 2024-04-12T21:02:06.793 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout: data_block_index_type: 0 2024-04-12T21:02:06.793 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout: index_shortening: 1 2024-04-12T21:02:06.793 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout: data_block_hash_table_util_ratio: 0.750000 2024-04-12T21:02:06.793 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout: checksum: 4 2024-04-12T21:02:06.793 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout: no_block_cache: 0 2024-04-12T21:02:06.793 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout: block_cache: 0x5582f51a9090 2024-04-12T21:02:06.793 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout: block_cache_name: BinnedLRUCache 2024-04-12T21:02:06.793 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout: block_cache_options: 2024-04-12T21:02:06.793 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout: capacity : 536870912 2024-04-12T21:02:06.793 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout: num_shard_bits : 4 2024-04-12T21:02:06.793 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout: strict_capacity_limit : 0 2024-04-12T21:02:06.794 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout: high_pri_pool_ratio: 0.000 2024-04-12T21:02:06.794 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout: block_cache_compressed: (nil) 2024-04-12T21:02:06.794 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout: persistent_cache: (nil) 2024-04-12T21:02:06.794 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout: block_size: 4096 2024-04-12T21:02:06.794 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout: block_size_deviation: 10 2024-04-12T21:02:06.794 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout: block_restart_interval: 16 2024-04-12T21:02:06.794 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout: index_block_restart_interval: 1 2024-04-12T21:02:06.794 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout: metadata_block_size: 4096 2024-04-12T21:02:06.794 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout: partition_filters: 0 2024-04-12T21:02:06.794 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout: use_delta_encoding: 1 2024-04-12T21:02:06.794 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout: filter_policy: bloomfilter 2024-04-12T21:02:06.794 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout: whole_key_filtering: 1 2024-04-12T21:02:06.794 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout: verify_compression: 0 2024-04-12T21:02:06.794 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout: read_amp_bytes_per_bit: 0 2024-04-12T21:02:06.795 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout: format_version: 5 2024-04-12T21:02:06.795 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout: enable_index_compression: 1 2024-04-12T21:02:06.795 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout: block_align: 0 2024-04-12T21:02:06.795 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout: max_auto_readahead_size: 262144 2024-04-12T21:02:06.795 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout: prepopulate_block_cache: 0 2024-04-12T21:02:06.795 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout: initial_auto_readahead_size: 8192 2024-04-12T21:02:06.795 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout: num_file_reads_for_auto_readahead: 2 2024-04-12T21:02:06.795 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:02:06 smithi186 ceph-mon[86379]: rocksdb: Options.write_buffer_size: 33554432 2024-04-12T21:02:06.795 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:02:06 smithi186 ceph-mon[86379]: rocksdb: Options.max_write_buffer_number: 2 2024-04-12T21:02:06.795 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:02:06 smithi186 ceph-mon[86379]: rocksdb: Options.compression: NoCompression 2024-04-12T21:02:06.795 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:02:06 smithi186 ceph-mon[86379]: rocksdb: Options.bottommost_compression: Disabled 2024-04-12T21:02:06.795 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:02:06 smithi186 ceph-mon[86379]: rocksdb: Options.prefix_extractor: nullptr 2024-04-12T21:02:06.795 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:02:06 smithi186 ceph-mon[86379]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-04-12T21:02:06.795 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:02:06 smithi186 ceph-mon[86379]: rocksdb: Options.num_levels: 7 2024-04-12T21:02:06.796 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:02:06 smithi186 ceph-mon[86379]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-04-12T21:02:06.796 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:02:06 smithi186 ceph-mon[86379]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-04-12T21:02:06.796 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:02:06 smithi186 ceph-mon[86379]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-04-12T21:02:06.796 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:02:06 smithi186 ceph-mon[86379]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-04-12T21:02:06.796 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:02:06 smithi186 ceph-mon[86379]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-04-12T21:02:06.796 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:02:06 smithi186 ceph-mon[86379]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-04-12T21:02:06.796 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:02:06 smithi186 ceph-mon[86379]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-04-12T21:02:06.796 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:02:06 smithi186 ceph-mon[86379]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-04-12T21:02:06.796 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:02:06 smithi186 ceph-mon[86379]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-04-12T21:02:06.796 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:02:06 smithi186 ceph-mon[86379]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-04-12T21:02:06.796 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:02:06 smithi186 ceph-mon[86379]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-04-12T21:02:06.796 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:02:06 smithi186 ceph-mon[86379]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-04-12T21:02:06.797 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:02:06 smithi186 ceph-mon[86379]: rocksdb: Options.compression_opts.window_bits: -14 2024-04-12T21:02:06.797 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:02:06 smithi186 ceph-mon[86379]: rocksdb: Options.compression_opts.level: 32767 2024-04-12T21:02:06.797 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:02:06 smithi186 ceph-mon[86379]: rocksdb: Options.compression_opts.strategy: 0 2024-04-12T21:02:06.797 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:02:06 smithi186 ceph-mon[86379]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-04-12T21:02:06.797 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:02:06 smithi186 ceph-mon[86379]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-04-12T21:02:06.797 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:02:06 smithi186 ceph-mon[86379]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-04-12T21:02:06.797 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:02:06 smithi186 ceph-mon[86379]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-04-12T21:02:06.797 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:02:06 smithi186 ceph-mon[86379]: rocksdb: Options.compression_opts.enabled: false 2024-04-12T21:02:06.797 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:02:06 smithi186 ceph-mon[86379]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-04-12T21:02:06.797 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:02:06 smithi186 ceph-mon[86379]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-04-12T21:02:06.797 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:02:06 smithi186 ceph-mon[86379]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-04-12T21:02:06.797 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:02:06 smithi186 ceph-mon[86379]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-04-12T21:02:06.797 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:02:06 smithi186 ceph-mon[86379]: rocksdb: Options.target_file_size_base: 67108864 2024-04-12T21:02:06.798 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:02:06 smithi186 ceph-mon[86379]: rocksdb: Options.target_file_size_multiplier: 1 2024-04-12T21:02:06.798 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:02:06 smithi186 ceph-mon[86379]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-04-12T21:02:06.798 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:02:06 smithi186 ceph-mon[86379]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-04-12T21:02:06.798 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:02:06 smithi186 ceph-mon[86379]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-04-12T21:02:06.798 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:02:06 smithi186 ceph-mon[86379]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-04-12T21:02:06.798 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:02:06 smithi186 ceph-mon[86379]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-04-12T21:02:06.798 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:02:06 smithi186 ceph-mon[86379]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-04-12T21:02:06.798 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:02:06 smithi186 ceph-mon[86379]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-04-12T21:02:06.798 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:02:06 smithi186 ceph-mon[86379]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-04-12T21:02:06.798 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:02:06 smithi186 ceph-mon[86379]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-04-12T21:02:06.798 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:02:06 smithi186 ceph-mon[86379]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-04-12T21:02:06.798 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:02:06 smithi186 ceph-mon[86379]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-04-12T21:02:06.798 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:02:06 smithi186 ceph-mon[86379]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-04-12T21:02:06.799 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:02:06 smithi186 ceph-mon[86379]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-04-12T21:02:06.799 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:02:06 smithi186 ceph-mon[86379]: rocksdb: Options.arena_block_size: 1048576 2024-04-12T21:02:06.800 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:02:06 smithi186 ceph-mon[86379]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-04-12T21:02:06.800 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:02:06 smithi186 ceph-mon[86379]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-04-12T21:02:06.800 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:02:06 smithi186 ceph-mon[86379]: rocksdb: Options.disable_auto_compactions: 0 2024-04-12T21:02:06.800 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:02:06 smithi186 ceph-mon[86379]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-04-12T21:02:06.800 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:02:06 smithi186 ceph-mon[86379]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-04-12T21:02:06.800 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:02:06 smithi186 ceph-mon[86379]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-04-12T21:02:06.800 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:02:06 smithi186 ceph-mon[86379]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-04-12T21:02:06.800 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:02:06 smithi186 ceph-mon[86379]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-04-12T21:02:06.800 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:02:06 smithi186 ceph-mon[86379]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-04-12T21:02:06.800 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:02:06 smithi186 ceph-mon[86379]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-04-12T21:02:06.800 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:02:06 smithi186 ceph-mon[86379]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-04-12T21:02:06.800 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:02:06 smithi186 ceph-mon[86379]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-04-12T21:02:06.800 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:02:06 smithi186 ceph-mon[86379]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-04-12T21:02:06.801 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:02:06 smithi186 ceph-mon[86379]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-04-12T21:02:06.801 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:02:06 smithi186 ceph-mon[86379]: rocksdb: Options.inplace_update_support: 0 2024-04-12T21:02:06.801 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:02:06 smithi186 ceph-mon[86379]: rocksdb: Options.inplace_update_num_locks: 10000 2024-04-12T21:02:06.801 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:02:06 smithi186 ceph-mon[86379]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-04-12T21:02:06.801 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:02:06 smithi186 ceph-mon[86379]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-04-12T21:02:06.801 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:02:06 smithi186 ceph-mon[86379]: rocksdb: Options.memtable_huge_page_size: 0 2024-04-12T21:02:06.801 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:02:06 smithi186 ceph-mon[86379]: rocksdb: Options.bloom_locality: 0 2024-04-12T21:02:06.801 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:02:06 smithi186 ceph-mon[86379]: rocksdb: Options.max_successive_merges: 0 2024-04-12T21:02:06.801 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:02:06 smithi186 ceph-mon[86379]: rocksdb: Options.optimize_filters_for_hits: 0 2024-04-12T21:02:06.801 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:02:06 smithi186 ceph-mon[86379]: rocksdb: Options.paranoid_file_checks: 0 2024-04-12T21:02:06.801 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:02:06 smithi186 ceph-mon[86379]: rocksdb: Options.force_consistency_checks: 1 2024-04-12T21:02:06.801 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:02:06 smithi186 ceph-mon[86379]: rocksdb: Options.report_bg_io_stats: 0 2024-04-12T21:02:06.801 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:02:06 smithi186 ceph-mon[86379]: rocksdb: Options.ttl: 2592000 2024-04-12T21:02:06.801 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:02:06 smithi186 ceph-mon[86379]: rocksdb: Options.periodic_compaction_seconds: 0 2024-04-12T21:02:06.801 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:02:06 smithi186 ceph-mon[86379]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-04-12T21:02:06.802 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:02:06 smithi186 ceph-mon[86379]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-04-12T21:02:06.802 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:02:06 smithi186 ceph-mon[86379]: rocksdb: Options.enable_blob_files: false 2024-04-12T21:02:06.802 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:02:06 smithi186 ceph-mon[86379]: rocksdb: Options.min_blob_size: 0 2024-04-12T21:02:06.802 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:02:06 smithi186 ceph-mon[86379]: rocksdb: Options.blob_file_size: 268435456 2024-04-12T21:02:06.802 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:02:06 smithi186 ceph-mon[86379]: rocksdb: Options.blob_compression_type: NoCompression 2024-04-12T21:02:06.802 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:02:06 smithi186 ceph-mon[86379]: rocksdb: Options.enable_blob_garbage_collection: false 2024-04-12T21:02:06.802 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:02:06 smithi186 ceph-mon[86379]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-04-12T21:02:06.802 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:02:06 smithi186 ceph-mon[86379]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-04-12T21:02:06.802 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:02:06 smithi186 ceph-mon[86379]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-04-12T21:02:06.802 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:02:06 smithi186 ceph-mon[86379]: rocksdb: Options.blob_file_starting_level: 0 2024-04-12T21:02:06.802 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:02:06 smithi186 ceph-mon[86379]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-04-12T21:02:06.802 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:02:06 smithi186 ceph-mon[86379]: rocksdb: [table/block_based/block_based_table_reader.cc:721] At least one SST file opened without unique ID to verify: 33.sst 2024-04-12T21:02:06.802 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:02:06 smithi186 ceph-mon[86379]: rocksdb: [db/version_set.cc:4390] More existing levels in DB than needed. max_bytes_for_level_multiplier may not be guaranteed. 2024-04-12T21:02:06.803 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:02:06 smithi186 ceph-mon[86379]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-smithi186/store.db/MANIFEST-000009 succeeded,manifest_file_number is 9, next_file_number is 35, last_sequence is 17234, log_number is 31,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-04-12T21:02:06.803 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:02:06 smithi186 ceph-mon[86379]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 31 2024-04-12T21:02:06.803 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:02:06 smithi186 ceph-mon[86379]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 34cdea1e-5e97-499c-bcd6-6ed0ef5f8b73 2024-04-12T21:02:06.803 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:02:06 smithi186 ceph-mon[86379]: rocksdb: EVENT_LOG_v1 {"time_micros": 1712955726491984, "job": 1, "event": "recovery_started", "wal_files": [31]} 2024-04-12T21:02:06.803 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:02:06 smithi186 ceph-mon[86379]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #31 mode 2 2024-04-12T21:02:06.803 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:02:06 smithi186 ceph-mon[86379]: rocksdb: EVENT_LOG_v1 {"time_micros": 1712955726495262, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 36, "file_size": 294456, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 17239, "largest_seqno": 17488, "table_properties": {"data_size": 292233, "index_size": 865, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 325, "raw_key_size": 2824, "raw_average_key_size": 24, "raw_value_size": 289495, "raw_average_value_size": 2517, "num_data_blocks": 38, "num_entries": 115, "num_filter_entries": 115, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1712955726, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "34cdea1e-5e97-499c-bcd6-6ed0ef5f8b73", "db_session_id": "DBWXCQLZC0X5VZ7LVT1Z", "orig_file_number": 36, "seqno_to_time_mapping": "N/A"}} 2024-04-12T21:02:06.803 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:02:06 smithi186 ceph-mon[86379]: rocksdb: EVENT_LOG_v1 {"time_micros": 1712955726495416, "job": 1, "event": "recovery_finished"} 2024-04-12T21:02:06.803 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:02:06 smithi186 ceph-mon[86379]: rocksdb: [db/version_set.cc:5047] Creating manifest 38 2024-04-12T21:02:06.803 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:02:06 smithi186 ceph-mon[86379]: rocksdb: [db/version_set.cc:4390] More existing levels in DB than needed. max_bytes_for_level_multiplier may not be guaranteed. 2024-04-12T21:02:06.803 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:02:06 smithi186 ceph-mon[86379]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-smithi186/store.db/000031.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-04-12T21:02:06.803 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:02:06 smithi186 ceph-mon[86379]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x5582f5298000 2024-04-12T21:02:06.803 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:02:06 smithi186 ceph-mon[86379]: rocksdb: DB pointer 0x5582f5282000 2024-04-12T21:02:06.803 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:02:06 smithi186 ceph-mon[86379]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-04-12T21:02:06.804 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:02:06 smithi186 ceph-mon[86379]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-04-12T21:02:06.804 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout: ** DB Stats ** 2024-04-12T21:02:06.804 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-04-12T21:02:06.804 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-04-12T21:02:06.804 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-04-12T21:02:06.804 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-04-12T21:02:06.804 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-04-12T21:02:06.804 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-04-12T21:02:06.804 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-04-12T21:02:06.804 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout: 2024-04-12T21:02:06.804 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout: ** Compaction Stats [default] ** 2024-04-12T21:02:06.804 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-04-12T21:02:06.805 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-04-12T21:02:06.805 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout: L0 1/0 287.55 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 152.0 0.00 0.00 1 0.002 0 0 0.0 0.0 2024-04-12T21:02:06.805 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout: L6 1/0 8.06 MB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 2024-04-12T21:02:06.805 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout: Sum 2/0 8.34 MB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 152.0 0.00 0.00 1 0.002 0 0 0.0 0.0 2024-04-12T21:02:06.805 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 152.0 0.00 0.00 1 0.002 0 0 0.0 0.0 2024-04-12T21:02:06.805 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout: 2024-04-12T21:02:06.805 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout: ** Compaction Stats [default] ** 2024-04-12T21:02:06.805 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-04-12T21:02:06.805 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-04-12T21:02:06.805 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 152.0 0.00 0.00 1 0.002 0 0 0.0 0.0 2024-04-12T21:02:06.805 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout: 2024-04-12T21:02:06.805 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-04-12T21:02:06.805 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout: 2024-04-12T21:02:06.805 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-04-12T21:02:06.806 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-04-12T21:02:06.806 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-04-12T21:02:06.806 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-04-12T21:02:06.806 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-04-12T21:02:06.806 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout: AddFile(Keys): cumulative 0, interval 0 2024-04-12T21:02:06.806 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout: Cumulative compaction: 0.00 GB write, 32.60 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-04-12T21:02:06.806 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout: Interval compaction: 0.00 GB write, 32.60 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-04-12T21:02:06.806 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-04-12T21:02:06.806 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout: Block cache BinnedLRUCache@0x5582f51a9090#2 capacity: 512.00 MB usage: 1.33 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 1.2e-05 secs_since: 0 2024-04-12T21:02:06.806 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,0.38 KB,7.15256e-05%) IndexBlock(1,0.95 KB,0.000181794%) Misc(1,0.00 KB,0%) 2024-04-12T21:02:06.806 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout: 2024-04-12T21:02:06.806 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout: ** File Read Latency Histogram By Level [default] ** 2024-04-12T21:02:06.806 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:02:06 smithi186 ceph-mon[86379]: starting mon.smithi186 rank 1 at public addrs [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] at bind addrs [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] mon_data /var/lib/ceph/mon/ceph-smithi186 fsid 1cbe407a-f90d-11ee-bc8d-c7b262605968 2024-04-12T21:02:06.806 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:02:06 smithi186 ceph-mon[86379]: mon.smithi186@-1(???) e2 preinit fsid 1cbe407a-f90d-11ee-bc8d-c7b262605968 2024-04-12T21:02:06.806 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:02:06 smithi186 ceph-mon[86379]: mon.smithi186@-1(???).mds e23 new map 2024-04-12T21:02:06.807 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:02:06 smithi186 ceph-mon[86379]: mon.smithi186@-1(???).mds e23 print_map 2024-04-12T21:02:06.807 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout: e23 2024-04-12T21:02:06.807 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-04-12T21:02:06.807 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-12T21:02:06.807 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout: legacy client fscid: 1 2024-04-12T21:02:06.807 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout: 2024-04-12T21:02:06.807 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout: Filesystem 'cephfs' (1) 2024-04-12T21:02:06.807 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout: fs_name cephfs 2024-04-12T21:02:06.807 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout: epoch 23 2024-04-12T21:02:06.807 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout: flags 12 joinable allow_snaps allow_multimds_snaps 2024-04-12T21:02:06.807 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout: created 2024-04-12T20:51:57.362850+0000 2024-04-12T21:02:06.807 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout: modified 2024-04-12T21:01:10.670287+0000 2024-04-12T21:02:06.807 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout: tableserver 0 2024-04-12T21:02:06.807 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout: root 0 2024-04-12T21:02:06.807 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout: session_timeout 60 2024-04-12T21:02:06.807 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout: session_autoclose 300 2024-04-12T21:02:06.808 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout: max_file_size 1099511627776 2024-04-12T21:02:06.808 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout: max_xattr_size 65536 2024-04-12T21:02:06.808 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout: required_client_features {} 2024-04-12T21:02:06.808 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout: last_failure 0 2024-04-12T21:02:06.808 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout: last_failure_osd_epoch 46 2024-04-12T21:02:06.808 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout: compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-12T21:02:06.808 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout: max_mds 2 2024-04-12T21:02:06.808 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout: in 0,1 2024-04-12T21:02:06.808 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout: up {0=14452,1=14464} 2024-04-12T21:02:06.808 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout: failed 2024-04-12T21:02:06.808 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout: damaged 2024-04-12T21:02:06.808 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout: stopped 2024-04-12T21:02:06.808 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout: data_pools [3] 2024-04-12T21:02:06.808 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout: metadata_pool 2 2024-04-12T21:02:06.809 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout: inline_data disabled 2024-04-12T21:02:06.809 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout: balancer 2024-04-12T21:02:06.809 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout: bal_rank_mask -1 2024-04-12T21:02:06.809 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout: standby_count_wanted 1 2024-04-12T21:02:06.809 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout: qdb_cluster leader: 0 members: 2024-04-12T21:02:06.809 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout: [mds.cephfs.smithi018.acqbsc{0:14452} state up:active seq 5 join_fscid=1 addr [v2:172.21.15.18:6826/1564065434,v1:172.21.15.18:6827/1564065434] compat {c=[1],r=[1],i=[7ff]}] 2024-04-12T21:02:06.809 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout: [mds.cephfs.smithi018.szjluo{1:14464} state up:active seq 6 join_fscid=1 addr [v2:172.21.15.18:6828/722443353,v1:172.21.15.18:6829/722443353] compat {c=[1],r=[1],i=[7ff]}] 2024-04-12T21:02:06.809 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout: 2024-04-12T21:02:06.809 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout: 2024-04-12T21:02:06.809 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout: Standby daemons: 2024-04-12T21:02:06.809 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout: 2024-04-12T21:02:06.809 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout: [mds.cephfs.smithi186.wjnafs{-1:24277} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.186:6826/438347207,v1:172.21.15.186:6827/438347207] compat {c=[1],r=[1],i=[7ff]}] 2024-04-12T21:02:06.809 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout: [mds.cephfs.smithi186.jdllyn{-1:24281} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.186:6824/1464359952,v1:172.21.15.186:6825/1464359952] compat {c=[1],r=[1],i=[7ff]}] 2024-04-12T21:02:06.809 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:02:06 smithi186 ceph-mon[86379]: mon.smithi186@-1(???).osd e49 crush map has features 3314933000852226048, adjusting msgr requires 2024-04-12T21:02:06.809 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:02:06 smithi186 ceph-mon[86379]: mon.smithi186@-1(???).osd e49 crush map has features 288514051259236352, adjusting msgr requires 2024-04-12T21:02:06.810 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:02:06 smithi186 ceph-mon[86379]: mon.smithi186@-1(???).osd e49 crush map has features 288514051259236352, adjusting msgr requires 2024-04-12T21:02:06.810 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:02:06 smithi186 ceph-mon[86379]: mon.smithi186@-1(???).osd e49 crush map has features 288514051259236352, adjusting msgr requires 2024-04-12T21:02:06.810 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:02:06 smithi186 ceph-mon[86379]: mon.smithi186@-1(???).paxosservice(auth 1..23) refresh upgraded, format 0 -> 3 2024-04-12T21:02:06.810 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:02:06 smithi186 bash[86361]: c77498da9b2bf5febb1767de4918612d9c9398a028542311285e3b9dcdc4e7a6 2024-04-12T21:02:06.810 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:02:06 smithi186 systemd[1]: Started Ceph mon.smithi186 for 1cbe407a-f90d-11ee-bc8d-c7b262605968. 2024-04-12T21:02:08.601 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:02:08 smithi018 ceph-mon[112590]: fsmap cephfs:2 {0=cephfs.smithi018.acqbsc=up:active,1=cephfs.smithi018.szjluo=up:active} 2 up:standby 2024-04-12T21:02:08.602 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:02:08 smithi018 ceph-mon[112590]: osdmap e49: 6 total, 6 up, 6 in 2024-04-12T21:02:08.602 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:02:08 smithi018 ceph-mon[112590]: mgrmap e38: smithi018.xjzwga(active, since 27s), standbys: smithi186.kqcfxk 2024-04-12T21:02:08.602 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:02:08 smithi018 ceph-mon[112590]: overall HEALTH_OK 2024-04-12T21:02:08.602 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:02:08 smithi018 ceph-mon[112590]: mon.smithi018 calling monitor election 2024-04-12T21:02:08.602 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:02:08 smithi018 ceph-mon[112590]: mon.smithi186 calling monitor election 2024-04-12T21:02:08.602 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:02:08 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "mon metadata", "id": "smithi018"}]: dispatch 2024-04-12T21:02:08.602 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:02:08 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "mon metadata", "id": "smithi186"}]: dispatch 2024-04-12T21:02:08.602 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:02:08 smithi018 ceph-mon[112590]: mon.smithi018 is new leader, mons smithi018,smithi186 in quorum (ranks 0,1) 2024-04-12T21:02:08.602 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:02:08 smithi018 ceph-mon[112590]: monmap epoch 3 2024-04-12T21:02:08.602 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:02:08 smithi018 ceph-mon[112590]: fsid 1cbe407a-f90d-11ee-bc8d-c7b262605968 2024-04-12T21:02:08.602 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:02:08 smithi018 ceph-mon[112590]: last_changed 2024-04-12T21:02:07.280834+0000 2024-04-12T21:02:08.603 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:02:08 smithi018 ceph-mon[112590]: created 2024-04-12T20:42:56.111400+0000 2024-04-12T21:02:08.603 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:02:08 smithi018 ceph-mon[112590]: min_mon_release 19 (squid) 2024-04-12T21:02:08.603 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:02:08 smithi018 ceph-mon[112590]: election_strategy: 1 2024-04-12T21:02:08.603 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:02:08 smithi018 ceph-mon[112590]: 0: [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] mon.smithi018 2024-04-12T21:02:08.603 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:02:08 smithi018 ceph-mon[112590]: 1: [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] mon.smithi186 2024-04-12T21:02:08.603 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:02:08 smithi018 ceph-mon[112590]: fsmap cephfs:2 {0=cephfs.smithi018.acqbsc=up:active,1=cephfs.smithi018.szjluo=up:active} 2 up:standby 2024-04-12T21:02:08.603 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:02:08 smithi018 ceph-mon[112590]: osdmap e49: 6 total, 6 up, 6 in 2024-04-12T21:02:08.603 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:02:08 smithi018 ceph-mon[112590]: mgrmap e38: smithi018.xjzwga(active, since 27s), standbys: smithi186.kqcfxk 2024-04-12T21:02:08.603 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:02:08 smithi018 ceph-mon[112590]: overall HEALTH_OK 2024-04-12T21:02:08.603 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:02:08 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' 2024-04-12T21:02:08.603 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:02:08 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' 2024-04-12T21:02:08.603 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:02:08 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-12T21:02:08.637 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:02:08 smithi186 ceph-mon[86379]: fsmap cephfs:2 {0=cephfs.smithi018.acqbsc=up:active,1=cephfs.smithi018.szjluo=up:active} 2 up:standby 2024-04-12T21:02:08.637 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:02:08 smithi186 ceph-mon[86379]: osdmap e49: 6 total, 6 up, 6 in 2024-04-12T21:02:08.637 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:02:08 smithi186 ceph-mon[86379]: mgrmap e38: smithi018.xjzwga(active, since 27s), standbys: smithi186.kqcfxk 2024-04-12T21:02:08.637 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:02:08 smithi186 ceph-mon[86379]: overall HEALTH_OK 2024-04-12T21:02:08.637 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:02:08 smithi186 ceph-mon[86379]: mon.smithi018 calling monitor election 2024-04-12T21:02:08.637 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:02:08 smithi186 ceph-mon[86379]: mon.smithi186 calling monitor election 2024-04-12T21:02:08.637 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:02:08 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "mon metadata", "id": "smithi018"}]: dispatch 2024-04-12T21:02:08.637 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:02:08 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "mon metadata", "id": "smithi186"}]: dispatch 2024-04-12T21:02:08.637 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:02:08 smithi186 ceph-mon[86379]: mon.smithi018 is new leader, mons smithi018,smithi186 in quorum (ranks 0,1) 2024-04-12T21:02:08.638 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:02:08 smithi186 ceph-mon[86379]: monmap epoch 3 2024-04-12T21:02:08.638 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:02:08 smithi186 ceph-mon[86379]: fsid 1cbe407a-f90d-11ee-bc8d-c7b262605968 2024-04-12T21:02:08.638 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:02:08 smithi186 ceph-mon[86379]: last_changed 2024-04-12T21:02:07.280834+0000 2024-04-12T21:02:08.638 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:02:08 smithi186 ceph-mon[86379]: created 2024-04-12T20:42:56.111400+0000 2024-04-12T21:02:08.638 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:02:08 smithi186 ceph-mon[86379]: min_mon_release 19 (squid) 2024-04-12T21:02:08.638 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:02:08 smithi186 ceph-mon[86379]: election_strategy: 1 2024-04-12T21:02:08.638 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:02:08 smithi186 ceph-mon[86379]: 0: [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] mon.smithi018 2024-04-12T21:02:08.638 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:02:08 smithi186 ceph-mon[86379]: 1: [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] mon.smithi186 2024-04-12T21:02:08.638 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:02:08 smithi186 ceph-mon[86379]: fsmap cephfs:2 {0=cephfs.smithi018.acqbsc=up:active,1=cephfs.smithi018.szjluo=up:active} 2 up:standby 2024-04-12T21:02:08.638 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:02:08 smithi186 ceph-mon[86379]: osdmap e49: 6 total, 6 up, 6 in 2024-04-12T21:02:08.638 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:02:08 smithi186 ceph-mon[86379]: mgrmap e38: smithi018.xjzwga(active, since 27s), standbys: smithi186.kqcfxk 2024-04-12T21:02:08.638 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:02:08 smithi186 ceph-mon[86379]: overall HEALTH_OK 2024-04-12T21:02:08.638 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:02:08 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' 2024-04-12T21:02:08.638 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:02:08 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' 2024-04-12T21:02:08.639 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:02:08 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-12T21:02:09.600 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:02:09 smithi186 ceph-mon[86379]: pgmap v17: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-12T21:02:09.601 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:02:09 smithi018 ceph-mon[112590]: pgmap v17: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-12T21:02:10.601 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:02:10 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T21:02:10.653 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:02:10 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T21:02:11.529 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:02:11 smithi186 ceph-mon[86379]: pgmap v18: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-12T21:02:11.601 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:02:11 smithi018 ceph-mon[112590]: pgmap v18: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-12T21:02:13.029 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:02:12 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' 2024-04-12T21:02:13.029 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:02:12 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' 2024-04-12T21:02:13.030 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:02:12 smithi186 ceph-mon[86379]: pgmap v19: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-12T21:02:13.101 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:02:12 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' 2024-04-12T21:02:13.101 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:02:12 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' 2024-04-12T21:02:13.101 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:02:12 smithi018 ceph-mon[112590]: pgmap v19: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-12T21:02:15.030 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:02:14 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' 2024-04-12T21:02:15.030 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:02:14 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' 2024-04-12T21:02:15.030 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:02:14 smithi186 ceph-mon[86379]: pgmap v20: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-12T21:02:15.101 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:02:14 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' 2024-04-12T21:02:15.101 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:02:14 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' 2024-04-12T21:02:15.101 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:02:14 smithi018 ceph-mon[112590]: pgmap v20: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-12T21:02:17.101 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:02:16 smithi018 ceph-mon[112590]: pgmap v21: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-12T21:02:17.280 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:02:16 smithi186 ceph-mon[86379]: pgmap v21: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-12T21:02:19.101 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:02:18 smithi018 ceph-mon[112590]: pgmap v22: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-12T21:02:19.280 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:02:18 smithi186 ceph-mon[86379]: pgmap v22: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-12T21:02:20.256 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:02:19 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' 2024-04-12T21:02:20.256 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:02:19 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' 2024-04-12T21:02:20.256 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:02:19 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-12T21:02:20.257 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:02:19 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-12T21:02:20.257 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:02:19 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' 2024-04-12T21:02:20.257 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:02:19 smithi186 ceph-mon[86379]: Reconfiguring mon.smithi018 (monmap changed)... 2024-04-12T21:02:20.257 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:02:19 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-12T21:02:20.257 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:02:19 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-12T21:02:20.257 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:02:19 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-12T21:02:20.257 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:02:19 smithi186 ceph-mon[86379]: Reconfiguring daemon mon.smithi018 on smithi018 2024-04-12T21:02:20.258 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:02:19 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' 2024-04-12T21:02:20.258 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:02:19 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' 2024-04-12T21:02:20.258 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:02:19 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-12T21:02:20.258 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:02:19 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-12T21:02:20.258 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:02:19 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' 2024-04-12T21:02:20.258 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:02:19 smithi018 ceph-mon[112590]: Reconfiguring mon.smithi018 (monmap changed)... 2024-04-12T21:02:20.258 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:02:19 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-12T21:02:20.258 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:02:19 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-12T21:02:20.258 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:02:19 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-12T21:02:20.258 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:02:19 smithi018 ceph-mon[112590]: Reconfiguring daemon mon.smithi018 on smithi018 2024-04-12T21:02:21.278 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:02:20 smithi018 ceph-mon[112590]: pgmap v23: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-12T21:02:21.279 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:02:20 smithi186 ceph-mon[86379]: pgmap v23: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-12T21:02:22.817 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:02:22 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' 2024-04-12T21:02:22.817 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:02:22 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' 2024-04-12T21:02:22.817 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:02:22 smithi018 ceph-mon[112590]: Reconfiguring mgr.smithi018.xjzwga (monmap changed)... 2024-04-12T21:02:22.817 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:02:22 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi018.xjzwga", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-12T21:02:22.817 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:02:22 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-12T21:02:22.818 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:02:22 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-12T21:02:22.818 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:02:22 smithi018 ceph-mon[112590]: Reconfiguring daemon mgr.smithi018.xjzwga on smithi018 2024-04-12T21:02:22.818 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:02:22 smithi018 ceph-mon[112590]: pgmap v24: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-12T21:02:23.030 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:02:22 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' 2024-04-12T21:02:23.030 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:02:22 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' 2024-04-12T21:02:23.030 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:02:22 smithi186 ceph-mon[86379]: Reconfiguring mgr.smithi018.xjzwga (monmap changed)... 2024-04-12T21:02:23.030 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:02:22 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi018.xjzwga", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-12T21:02:23.030 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:02:22 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-12T21:02:23.030 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:02:22 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-12T21:02:23.030 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:02:22 smithi186 ceph-mon[86379]: Reconfiguring daemon mgr.smithi018.xjzwga on smithi018 2024-04-12T21:02:23.030 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:02:22 smithi186 ceph-mon[86379]: pgmap v24: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-12T21:02:25.196 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:02:24 smithi018 ceph-mon[112590]: pgmap v25: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-12T21:02:25.196 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:02:24 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' 2024-04-12T21:02:25.196 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:02:24 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' 2024-04-12T21:02:25.196 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:02:24 smithi018 ceph-mon[112590]: Reconfiguring crash.smithi018 (monmap changed)... 2024-04-12T21:02:25.196 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:02:24 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi018", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-04-12T21:02:25.196 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:02:24 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-12T21:02:25.196 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:02:24 smithi018 ceph-mon[112590]: Reconfiguring daemon crash.smithi018 on smithi018 2024-04-12T21:02:25.197 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:02:24 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T21:02:25.280 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:02:24 smithi186 ceph-mon[86379]: pgmap v25: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-12T21:02:25.280 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:02:24 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' 2024-04-12T21:02:25.280 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:02:24 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' 2024-04-12T21:02:25.280 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:02:24 smithi186 ceph-mon[86379]: Reconfiguring crash.smithi018 (monmap changed)... 2024-04-12T21:02:25.280 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:02:24 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi018", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-04-12T21:02:25.280 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:02:24 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-12T21:02:25.280 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:02:24 smithi186 ceph-mon[86379]: Reconfiguring daemon crash.smithi018 on smithi018 2024-04-12T21:02:25.280 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:02:24 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T21:02:26.257 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:02:26.259+0000 7f038cc00700 1 -- 172.21.15.18:0/4129733217 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0388101d70 msgr2=0x7f0388102140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:02:26.257 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:02:26.259+0000 7f038cc00700 1 --2- 172.21.15.18:0/4129733217 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0388101d70 0x7f0388102140 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f0378004940 tx=0x7f0378009e30 comp rx=0 tx=0).stop 2024-04-12T21:02:26.257 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:02:26.259+0000 7f038cc00700 1 -- 172.21.15.18:0/4129733217 shutdown_connections 2024-04-12T21:02:26.257 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:02:26.259+0000 7f038cc00700 1 --2- 172.21.15.18:0/4129733217 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0388102710 0x7f03880fbd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:02:26.257 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:02:26.259+0000 7f038cc00700 1 --2- 172.21.15.18:0/4129733217 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0388101d70 0x7f0388102140 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:02:26.257 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:02:26.259+0000 7f038cc00700 1 -- 172.21.15.18:0/4129733217 >> 172.21.15.18:0/4129733217 conn(0x7f0388072bc0 msgr2=0x7f0388072fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:02:26.257 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:02:26.259+0000 7f038cc00700 1 -- 172.21.15.18:0/4129733217 shutdown_connections 2024-04-12T21:02:26.257 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:02:26.259+0000 7f038cc00700 1 -- 172.21.15.18:0/4129733217 wait complete. 2024-04-12T21:02:26.258 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:02:26.260+0000 7f038cc00700 1 Processor -- start 2024-04-12T21:02:26.258 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:02:26.261+0000 7f038cc00700 1 -- start start 2024-04-12T21:02:26.259 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:02:26.261+0000 7f038cc00700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0388101d70 0x7f0388115c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:02:26.259 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:02:26.261+0000 7f038cc00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0388102710 0x7f03881161d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:02:26.259 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:02:26.261+0000 7f038cc00700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0388116820 con 0x7f0388102710 2024-04-12T21:02:26.259 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:02:26.261+0000 7f038cc00700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f038810fca0 con 0x7f0388101d70 2024-04-12T21:02:26.259 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:02:26.261+0000 7f0385400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0388102710 0x7f03881161d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:02:26.259 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:02:26.261+0000 7f0385400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0388102710 0x7f03881161d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:36546/0 (socket says 172.21.15.18:36546) 2024-04-12T21:02:26.259 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:02:26.261+0000 7f0385400700 1 -- 172.21.15.18:0/4173746587 learned_addr learned my addr 172.21.15.18:0/4173746587 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:02:26.259 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:02:26.262+0000 7f0385e00700 1 --2- 172.21.15.18:0/4173746587 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0388101d70 0x7f0388115c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:02:26.260 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:02:26.262+0000 7f0385400700 1 -- 172.21.15.18:0/4173746587 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0388101d70 msgr2=0x7f0388115c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:02:26.260 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:02:26.262+0000 7f0385400700 1 --2- 172.21.15.18:0/4173746587 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0388101d70 0x7f0388115c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:02:26.260 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:02:26.262+0000 7f0385400700 1 -- 172.21.15.18:0/4173746587 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0378005040 con 0x7f0388102710 2024-04-12T21:02:26.260 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:02:26.262+0000 7f0385400700 1 --2- 172.21.15.18:0/4173746587 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0388102710 0x7f03881161d0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f0374002a80 tx=0x7f0374004630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:02:26.260 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:02:26.263+0000 7f037e000700 1 -- 172.21.15.18:0/4173746587 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0374007cc0 con 0x7f0388102710 2024-04-12T21:02:26.260 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:02:26.263+0000 7f038cc00700 1 -- 172.21.15.18:0/4173746587 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f038810ff80 con 0x7f0388102710 2024-04-12T21:02:26.260 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:02:26.263+0000 7f038cc00700 1 -- 172.21.15.18:0/4173746587 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0388110540 con 0x7f0388102710 2024-04-12T21:02:26.261 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:02:26.263+0000 7f037e000700 1 -- 172.21.15.18:0/4173746587 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f037400f040 con 0x7f0388102710 2024-04-12T21:02:26.261 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:02:26.263+0000 7f037e000700 1 -- 172.21.15.18:0/4173746587 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f037400da20 con 0x7f0388102710 2024-04-12T21:02:26.261 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:02:26.264+0000 7f038cc00700 1 -- 172.21.15.18:0/4173746587 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f03680052e0 con 0x7f0388102710 2024-04-12T21:02:26.262 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:02:26.265+0000 7f037e000700 1 -- 172.21.15.18:0/4173746587 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 38) v1 ==== 92847+0+0 (secure 0 0 0) 0x7f037400db80 con 0x7f0388102710 2024-04-12T21:02:26.264 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:02:26.267+0000 7f037e000700 1 --2- 172.21.15.18:0/4173746587 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f036c06ed20 0x7f036c0711d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:02:26.264 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:02:26.267+0000 7f037e000700 1 -- 172.21.15.18:0/4173746587 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6324+0+0 (secure 0 0 0) 0x7f037408e0e0 con 0x7f0388102710 2024-04-12T21:02:26.264 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:02:26.267+0000 7f0385e00700 1 --2- 172.21.15.18:0/4173746587 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f036c06ed20 0x7f036c0711d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:02:26.265 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:02:26.268+0000 7f0385e00700 1 --2- 172.21.15.18:0/4173746587 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f036c06ed20 0x7f036c0711d0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f037800b020 tx=0x7f0378009e30 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:02:26.270 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:02:26.273+0000 7f037e000700 1 -- 172.21.15.18:0/4173746587 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f0374056cf0 con 0x7f0388102710 2024-04-12T21:02:26.540 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:02:26.542+0000 7f038cc00700 1 -- 172.21.15.18:0/4173746587 --> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f0368000bf0 con 0x7f036c06ed20 2024-04-12T21:02:26.545 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:02:26.547+0000 7f037e000700 1 -- 172.21.15.18:0/4173746587 <== mgr.34102 v2:172.21.15.18:6800/3812891919 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+383 (secure 0 0 0) 0x7f0368000bf0 con 0x7f036c06ed20 2024-04-12T21:02:26.548 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:02:26.550+0000 7f038cc00700 1 -- 172.21.15.18:0/4173746587 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f036c06ed20 msgr2=0x7f036c0711d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:02:26.548 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:02:26.550+0000 7f038cc00700 1 --2- 172.21.15.18:0/4173746587 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f036c06ed20 0x7f036c0711d0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f037800b020 tx=0x7f0378009e30 comp rx=0 tx=0).stop 2024-04-12T21:02:26.548 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:02:26.550+0000 7f038cc00700 1 -- 172.21.15.18:0/4173746587 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0388102710 msgr2=0x7f03881161d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:02:26.548 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:02:26.550+0000 7f038cc00700 1 --2- 172.21.15.18:0/4173746587 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0388102710 0x7f03881161d0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f0374002a80 tx=0x7f0374004630 comp rx=0 tx=0).stop 2024-04-12T21:02:26.548 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:02:26.550+0000 7f038cc00700 1 -- 172.21.15.18:0/4173746587 shutdown_connections 2024-04-12T21:02:26.548 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:02:26.551+0000 7f038cc00700 1 --2- 172.21.15.18:0/4173746587 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f036c06ed20 0x7f036c0711d0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:02:26.548 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:02:26.551+0000 7f038cc00700 1 --2- 172.21.15.18:0/4173746587 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0388101d70 0x7f0388115c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:02:26.548 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:02:26.551+0000 7f038cc00700 1 --2- 172.21.15.18:0/4173746587 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0388102710 0x7f03881161d0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:02:26.548 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:02:26.551+0000 7f038cc00700 1 -- 172.21.15.18:0/4173746587 >> 172.21.15.18:0/4173746587 conn(0x7f0388072bc0 msgr2=0x7f0388065940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:02:26.548 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:02:26.551+0000 7f038cc00700 1 -- 172.21.15.18:0/4173746587 shutdown_connections 2024-04-12T21:02:26.548 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:02:26.551+0000 7f038cc00700 1 -- 172.21.15.18:0/4173746587 wait complete. 2024-04-12T21:02:26.566 INFO:teuthology.orchestra.run.smithi018.stdout:true 2024-04-12T21:02:26.714 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:02:26.715+0000 7f3577200700 1 -- 172.21.15.18:0/2442231892 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3570102de0 msgr2=0x7f35701031b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:02:26.714 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:02:26.715+0000 7f3577200700 1 --2- 172.21.15.18:0/2442231892 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3570102de0 0x7f35701031b0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f3560004940 tx=0x7f3560009e30 comp rx=0 tx=0).stop 2024-04-12T21:02:26.714 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:02:26.716+0000 7f3577200700 1 -- 172.21.15.18:0/2442231892 shutdown_connections 2024-04-12T21:02:26.714 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:02:26.716+0000 7f3577200700 1 --2- 172.21.15.18:0/2442231892 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3570103780 0x7f35700fcdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:02:26.714 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:02:26.716+0000 7f3577200700 1 --2- 172.21.15.18:0/2442231892 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3570102de0 0x7f35701031b0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:02:26.714 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:02:26.716+0000 7f3577200700 1 -- 172.21.15.18:0/2442231892 >> 172.21.15.18:0/2442231892 conn(0x7f35700f8970 msgr2=0x7f35700fad80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:02:26.714 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:02:26.716+0000 7f3577200700 1 -- 172.21.15.18:0/2442231892 shutdown_connections 2024-04-12T21:02:26.714 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:02:26.716+0000 7f3577200700 1 -- 172.21.15.18:0/2442231892 wait complete. 2024-04-12T21:02:26.715 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:02:26.717+0000 7f3577200700 1 Processor -- start 2024-04-12T21:02:26.715 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:02:26.717+0000 7f3577200700 1 -- start start 2024-04-12T21:02:26.715 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:02:26.717+0000 7f3577200700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3570102de0 0x7f3570115d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:02:26.715 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:02:26.717+0000 7f3577200700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3570103780 0x7f35701162b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:02:26.715 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:02:26.717+0000 7f3577200700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3570116900 con 0x7f3570102de0 2024-04-12T21:02:26.716 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:02:26.717+0000 7f3577200700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f357010fd20 con 0x7f3570103780 2024-04-12T21:02:26.716 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:02:26.718+0000 7f356fe00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3570102de0 0x7f3570115d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:02:26.716 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:02:26.718+0000 7f356fe00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3570102de0 0x7f3570115d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:36566/0 (socket says 172.21.15.18:36566) 2024-04-12T21:02:26.716 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:02:26.718+0000 7f356fe00700 1 -- 172.21.15.18:0/2782206282 learned_addr learned my addr 172.21.15.18:0/2782206282 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:02:26.716 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:02:26.718+0000 7f356f400700 1 --2- 172.21.15.18:0/2782206282 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3570103780 0x7f35701162b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:02:26.716 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:02:26.718+0000 7f356fe00700 1 -- 172.21.15.18:0/2782206282 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3570103780 msgr2=0x7f35701162b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:02:26.716 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:02:26.718+0000 7f356fe00700 1 --2- 172.21.15.18:0/2782206282 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3570103780 0x7f35701162b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:02:26.716 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:02:26.718+0000 7f356fe00700 1 -- 172.21.15.18:0/2782206282 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3560005040 con 0x7f3570102de0 2024-04-12T21:02:26.717 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:02:26.719+0000 7f356fe00700 1 --2- 172.21.15.18:0/2782206282 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3570102de0 0x7f3570115d70 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f3560009180 tx=0x7f3560009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:02:26.717 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:02:26.719+0000 7f3567e00700 1 -- 172.21.15.18:0/2782206282 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f356000e040 con 0x7f3570102de0 2024-04-12T21:02:26.717 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:02:26.719+0000 7f3567e00700 1 -- 172.21.15.18:0/2782206282 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f356000bbc0 con 0x7f3570102de0 2024-04-12T21:02:26.717 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:02:26.719+0000 7f3567e00700 1 -- 172.21.15.18:0/2782206282 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f356001c630 con 0x7f3570102de0 2024-04-12T21:02:26.717 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:02:26.719+0000 7f3577200700 1 -- 172.21.15.18:0/2782206282 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f357010ffa0 con 0x7f3570102de0 2024-04-12T21:02:26.717 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:02:26.719+0000 7f3577200700 1 -- 172.21.15.18:0/2782206282 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3570110560 con 0x7f3570102de0 2024-04-12T21:02:26.717 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:02:26.720+0000 7f3577200700 1 -- 172.21.15.18:0/2782206282 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3570106a80 con 0x7f3570102de0 2024-04-12T21:02:26.720 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:02:26.722+0000 7f3567e00700 1 -- 172.21.15.18:0/2782206282 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 38) v1 ==== 92847+0+0 (secure 0 0 0) 0x7f356000bd30 con 0x7f3570102de0 2024-04-12T21:02:26.721 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:02:26.723+0000 7f3567e00700 1 --2- 172.21.15.18:0/2782206282 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f355406ed20 0x7f35540711d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:02:26.721 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:02:26.724+0000 7f3567e00700 1 -- 172.21.15.18:0/2782206282 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6324+0+0 (secure 0 0 0) 0x7f356008fe60 con 0x7f3570102de0 2024-04-12T21:02:26.721 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:02:26.724+0000 7f356f400700 1 --2- 172.21.15.18:0/2782206282 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f355406ed20 0x7f35540711d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:02:26.722 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:02:26.725+0000 7f356f400700 1 --2- 172.21.15.18:0/2782206282 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f355406ed20 0x7f35540711d0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f3570111610 tx=0x7f355c009ee0 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:02:26.728 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:02:26.730+0000 7f3567e00700 1 -- 172.21.15.18:0/2782206282 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f356005c620 con 0x7f3570102de0 2024-04-12T21:02:26.980 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:02:26.982+0000 7f3577200700 1 -- 172.21.15.18:0/2782206282 --> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f3570109f50 con 0x7f355406ed20 2024-04-12T21:02:26.982 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:02:26.985+0000 7f3567e00700 1 -- 172.21.15.18:0/2782206282 <== mgr.34102 v2:172.21.15.18:6800/3812891919 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+383 (secure 0 0 0) 0x7f3570109f50 con 0x7f355406ed20 2024-04-12T21:02:26.985 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:02:26.988+0000 7f3577200700 1 -- 172.21.15.18:0/2782206282 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f355406ed20 msgr2=0x7f35540711d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:02:26.985 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:02:26.988+0000 7f3577200700 1 --2- 172.21.15.18:0/2782206282 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f355406ed20 0x7f35540711d0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f3570111610 tx=0x7f355c009ee0 comp rx=0 tx=0).stop 2024-04-12T21:02:26.985 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:02:26.988+0000 7f3577200700 1 -- 172.21.15.18:0/2782206282 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3570102de0 msgr2=0x7f3570115d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:02:26.985 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:02:26.988+0000 7f3577200700 1 --2- 172.21.15.18:0/2782206282 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3570102de0 0x7f3570115d70 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f3560009180 tx=0x7f3560009e30 comp rx=0 tx=0).stop 2024-04-12T21:02:26.986 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:02:26.988+0000 7f3577200700 1 -- 172.21.15.18:0/2782206282 shutdown_connections 2024-04-12T21:02:26.986 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:02:26.988+0000 7f3577200700 1 --2- 172.21.15.18:0/2782206282 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f355406ed20 0x7f35540711d0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:02:26.986 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:02:26.988+0000 7f3577200700 1 --2- 172.21.15.18:0/2782206282 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3570102de0 0x7f3570115d70 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:02:26.986 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:02:26.988+0000 7f3577200700 1 --2- 172.21.15.18:0/2782206282 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3570103780 0x7f35701162b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:02:26.986 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:02:26.988+0000 7f3577200700 1 -- 172.21.15.18:0/2782206282 >> 172.21.15.18:0/2782206282 conn(0x7f35700f8970 msgr2=0x7f35701073d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:02:26.986 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:02:26.989+0000 7f3577200700 1 -- 172.21.15.18:0/2782206282 shutdown_connections 2024-04-12T21:02:26.986 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:02:26.989+0000 7f3577200700 1 -- 172.21.15.18:0/2782206282 wait complete. 2024-04-12T21:02:27.130 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:02:27.132+0000 7efd7ca00700 1 -- 172.21.15.18:0/2102730225 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7efd780fde90 msgr2=0x7efd780fe260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:02:27.130 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:02:27.132+0000 7efd7ca00700 1 --2- 172.21.15.18:0/2102730225 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7efd780fde90 0x7efd780fe260 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7efd68004940 tx=0x7efd68009e30 comp rx=0 tx=0).stop 2024-04-12T21:02:27.130 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:02:27.132+0000 7efd7ca00700 1 -- 172.21.15.18:0/2102730225 shutdown_connections 2024-04-12T21:02:27.130 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:02:27.132+0000 7efd7ca00700 1 --2- 172.21.15.18:0/2102730225 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7efd780fe830 0x7efd78106f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:02:27.130 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:02:27.132+0000 7efd7ca00700 1 --2- 172.21.15.18:0/2102730225 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7efd780fde90 0x7efd780fe260 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:02:27.130 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:02:27.132+0000 7efd7ca00700 1 -- 172.21.15.18:0/2102730225 >> 172.21.15.18:0/2102730225 conn(0x7efd78072bc0 msgr2=0x7efd78072fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:02:27.130 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:02:27.132+0000 7efd7ca00700 1 -- 172.21.15.18:0/2102730225 shutdown_connections 2024-04-12T21:02:27.131 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:02:27.132+0000 7efd7ca00700 1 -- 172.21.15.18:0/2102730225 wait complete. 2024-04-12T21:02:27.132 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:02:27.133+0000 7efd7ca00700 1 Processor -- start 2024-04-12T21:02:27.132 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:02:27.133+0000 7efd7ca00700 1 -- start start 2024-04-12T21:02:27.132 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:02:27.134+0000 7efd7ca00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7efd780fde90 0x7efd78115d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:02:27.132 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:02:27.134+0000 7efd7ca00700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7efd780fe830 0x7efd78116280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:02:27.132 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:02:27.134+0000 7efd7ca00700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd78116960 con 0x7efd780fde90 2024-04-12T21:02:27.132 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:02:27.134+0000 7efd7ca00700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd7810fce0 con 0x7efd780fe830 2024-04-12T21:02:27.132 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:02:27.134+0000 7efd75e00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7efd780fde90 0x7efd78115d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:02:27.132 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:02:27.134+0000 7efd75e00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7efd780fde90 0x7efd78115d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:36576/0 (socket says 172.21.15.18:36576) 2024-04-12T21:02:27.132 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:02:27.134+0000 7efd75e00700 1 -- 172.21.15.18:0/1717376142 learned_addr learned my addr 172.21.15.18:0/1717376142 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:02:27.132 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:02:27.134+0000 7efd75400700 1 --2- 172.21.15.18:0/1717376142 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7efd780fe830 0x7efd78116280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:02:27.132 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:02:27.135+0000 7efd75e00700 1 -- 172.21.15.18:0/1717376142 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7efd780fe830 msgr2=0x7efd78116280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:02:27.132 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:02:27.135+0000 7efd75e00700 1 --2- 172.21.15.18:0/1717376142 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7efd780fe830 0x7efd78116280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:02:27.133 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:02:27.135+0000 7efd75e00700 1 -- 172.21.15.18:0/1717376142 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd68005040 con 0x7efd780fde90 2024-04-12T21:02:27.133 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:02:27.135+0000 7efd75e00700 1 --2- 172.21.15.18:0/1717376142 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7efd780fde90 0x7efd78115d40 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7efd68000c00 tx=0x7efd68009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:02:27.133 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:02:27.135+0000 7efd6e000700 1 -- 172.21.15.18:0/1717376142 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7efd6800e040 con 0x7efd780fde90 2024-04-12T21:02:27.133 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:02:27.135+0000 7efd6e000700 1 -- 172.21.15.18:0/1717376142 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7efd68013420 con 0x7efd780fde90 2024-04-12T21:02:27.133 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:02:27.135+0000 7efd6e000700 1 -- 172.21.15.18:0/1717376142 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7efd6801c610 con 0x7efd780fde90 2024-04-12T21:02:27.134 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:02:27.135+0000 7efd7ca00700 1 -- 172.21.15.18:0/1717376142 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efd7810ff60 con 0x7efd780fde90 2024-04-12T21:02:27.134 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:02:27.136+0000 7efd7ca00700 1 -- 172.21.15.18:0/1717376142 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efd781104a0 con 0x7efd780fde90 2024-04-12T21:02:27.134 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:02:27.137+0000 7efd7ca00700 1 -- 172.21.15.18:0/1717376142 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efd78106a20 con 0x7efd780fde90 2024-04-12T21:02:27.135 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:02:27.138+0000 7efd6e000700 1 -- 172.21.15.18:0/1717376142 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 38) v1 ==== 92847+0+0 (secure 0 0 0) 0x7efd68013590 con 0x7efd780fde90 2024-04-12T21:02:27.136 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:02:27.139+0000 7efd6e000700 1 --2- 172.21.15.18:0/1717376142 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7efd5c06ed20 0x7efd5c0711d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:02:27.137 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:02:27.139+0000 7efd6e000700 1 -- 172.21.15.18:0/1717376142 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6324+0+0 (secure 0 0 0) 0x7efd6808fcd0 con 0x7efd780fde90 2024-04-12T21:02:27.137 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:02:27.140+0000 7efd75400700 1 --2- 172.21.15.18:0/1717376142 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7efd5c06ed20 0x7efd5c0711d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:02:27.137 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:02:27.140+0000 7efd75400700 1 --2- 172.21.15.18:0/1717376142 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7efd5c06ed20 0x7efd5c0711d0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7efd64006fd0 tx=0x7efd64008040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:02:27.145 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:02:27.148+0000 7efd6e000700 1 -- 172.21.15.18:0/1717376142 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7efd680588e0 con 0x7efd780fde90 2024-04-12T21:02:27.397 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:02:27.400+0000 7efd7ca00700 1 -- 172.21.15.18:0/1717376142 --> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7efd78109ff0 con 0x7efd5c06ed20 2024-04-12T21:02:27.414 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:02:27.417+0000 7efd6e000700 1 -- 172.21.15.18:0/1717376142 <== mgr.34102 v2:172.21.15.18:6800/3812891919 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7efd78109ff0 con 0x7efd5c06ed20 2024-04-12T21:02:27.414 INFO:teuthology.orchestra.run.smithi018.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-12T21:02:27.414 INFO:teuthology.orchestra.run.smithi018.stdout:alertmanager.smithi018 smithi018 *:9093,9094 running (3m) 40s ago 17m 14.9M - 0.25.0 c8568f914cd2 c332090068e8 2024-04-12T21:02:27.414 INFO:teuthology.orchestra.run.smithi018.stdout:crash.smithi018 smithi018 running (17m) 40s ago 17m 6660k - 17.2.7-827-gd3b5060a 21de9b7369ad 1ac7603bddfd 2024-04-12T21:02:27.415 INFO:teuthology.orchestra.run.smithi018.stdout:crash.smithi186 smithi186 running (16m) 15s ago 16m 6669k - 17.2.7-827-gd3b5060a 21de9b7369ad 14bf8a1672eb 2024-04-12T21:02:27.415 INFO:teuthology.orchestra.run.smithi018.stdout:grafana.smithi018 smithi018 *:3000 running (2m) 40s ago 17m 74.4M - 9.4.12 2bacad6d85d8 b08e3bf176dd 2024-04-12T21:02:27.415 INFO:teuthology.orchestra.run.smithi018.stdout:mds.cephfs.smithi018.acqbsc smithi018 running (10m) 40s ago 10m 112M - 17.2.7-827-gd3b5060a 21de9b7369ad 5b5fce65f18a 2024-04-12T21:02:27.415 INFO:teuthology.orchestra.run.smithi018.stdout:mds.cephfs.smithi018.szjluo smithi018 running (10m) 40s ago 10m 113M - 17.2.7-827-gd3b5060a 21de9b7369ad cf9bf0c5031f 2024-04-12T21:02:27.415 INFO:teuthology.orchestra.run.smithi018.stdout:mds.cephfs.smithi186.jdllyn smithi186 running (10m) 15s ago 10m 14.4M - 17.2.7-827-gd3b5060a 21de9b7369ad 2b2bc46bd29a 2024-04-12T21:02:27.415 INFO:teuthology.orchestra.run.smithi018.stdout:mds.cephfs.smithi186.wjnafs smithi186 running (10m) 15s ago 10m 14.2M - 17.2.7-827-gd3b5060a 21de9b7369ad 74b115d6d6fd 2024-04-12T21:02:27.415 INFO:teuthology.orchestra.run.smithi018.stdout:mgr.smithi018.xjzwga smithi018 *:8443,9283,8765 running (5m) 40s ago 19m 478M - 19.0.0-2879-g324baf8d 8717720dc5e5 56088305eab7 2024-04-12T21:02:27.415 INFO:teuthology.orchestra.run.smithi018.stdout:mgr.smithi186.kqcfxk smithi186 *:8443,9283,8765 running (4m) 15s ago 16m 450M - 19.0.0-2879-g324baf8d 8717720dc5e5 09cd766e5e0d 2024-04-12T21:02:27.415 INFO:teuthology.orchestra.run.smithi018.stdout:mon.smithi018 smithi018 running (58s) 40s ago 19m 29.8M 2048M 19.0.0-2879-g324baf8d 8717720dc5e5 df3e201dfed6 2024-04-12T21:02:27.415 INFO:teuthology.orchestra.run.smithi018.stdout:mon.smithi186 smithi186 running (21s) 15s ago 16m 16.8M 2048M 19.0.0-2879-g324baf8d 8717720dc5e5 c77498da9b2b 2024-04-12T21:02:27.415 INFO:teuthology.orchestra.run.smithi018.stdout:node-exporter.smithi018 smithi018 *:9100 running (4m) 40s ago 17m 7898k - 1.5.0 0da6a335fe13 326c3d9c6847 2024-04-12T21:02:27.415 INFO:teuthology.orchestra.run.smithi018.stdout:node-exporter.smithi186 smithi186 *:9100 running (3m) 15s ago 16m 8044k - 1.5.0 0da6a335fe13 ec2112baeecf 2024-04-12T21:02:27.415 INFO:teuthology.orchestra.run.smithi018.stdout:osd.0 smithi018 running (15m) 40s ago 15m 226M 1559M 17.2.7-827-gd3b5060a 21de9b7369ad 0d83e027bf0a 2024-04-12T21:02:27.415 INFO:teuthology.orchestra.run.smithi018.stdout:osd.1 smithi018 running (14m) 40s ago 14m 241M 1559M 17.2.7-827-gd3b5060a 21de9b7369ad 6517c9cc7f87 2024-04-12T21:02:27.416 INFO:teuthology.orchestra.run.smithi018.stdout:osd.2 smithi018 running (13m) 40s ago 13m 194M 1559M 17.2.7-827-gd3b5060a 21de9b7369ad 8a83692acaa5 2024-04-12T21:02:27.416 INFO:teuthology.orchestra.run.smithi018.stdout:osd.3 smithi186 running (13m) 15s ago 13m 263M 2581M 17.2.7-827-gd3b5060a 21de9b7369ad c2def7a90d76 2024-04-12T21:02:27.416 INFO:teuthology.orchestra.run.smithi018.stdout:osd.4 smithi186 running (12m) 15s ago 12m 233M 2581M 17.2.7-827-gd3b5060a 21de9b7369ad 757e7c71b7b9 2024-04-12T21:02:27.416 INFO:teuthology.orchestra.run.smithi018.stdout:osd.5 smithi186 running (12m) 15s ago 12m 228M 2581M 17.2.7-827-gd3b5060a 21de9b7369ad 2ee072a436d0 2024-04-12T21:02:27.416 INFO:teuthology.orchestra.run.smithi018.stdout:prometheus.smithi018 smithi018 *:9095 running (3m) 40s ago 16m 56.0M - 2.43.0 a07b618ecd1d f58841127e21 2024-04-12T21:02:27.417 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:02:27.419+0000 7efd7ca00700 1 -- 172.21.15.18:0/1717376142 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7efd5c06ed20 msgr2=0x7efd5c0711d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:02:27.417 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:02:27.419+0000 7efd7ca00700 1 --2- 172.21.15.18:0/1717376142 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7efd5c06ed20 0x7efd5c0711d0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7efd64006fd0 tx=0x7efd64008040 comp rx=0 tx=0).stop 2024-04-12T21:02:27.417 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:02:27.419+0000 7efd7ca00700 1 -- 172.21.15.18:0/1717376142 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7efd780fde90 msgr2=0x7efd78115d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:02:27.417 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:02:27.419+0000 7efd7ca00700 1 --2- 172.21.15.18:0/1717376142 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7efd780fde90 0x7efd78115d40 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7efd68000c00 tx=0x7efd68009e30 comp rx=0 tx=0).stop 2024-04-12T21:02:27.417 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:02:27.420+0000 7efd7ca00700 1 -- 172.21.15.18:0/1717376142 shutdown_connections 2024-04-12T21:02:27.417 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:02:27.420+0000 7efd7ca00700 1 --2- 172.21.15.18:0/1717376142 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7efd5c06ed20 0x7efd5c0711d0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:02:27.418 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:02:27.420+0000 7efd7ca00700 1 --2- 172.21.15.18:0/1717376142 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7efd780fde90 0x7efd78115d40 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:02:27.418 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:02:27.420+0000 7efd7ca00700 1 --2- 172.21.15.18:0/1717376142 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7efd780fe830 0x7efd78116280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:02:27.418 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:02:27.420+0000 7efd7ca00700 1 -- 172.21.15.18:0/1717376142 >> 172.21.15.18:0/1717376142 conn(0x7efd78072bc0 msgr2=0x7efd780fb700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:02:27.418 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:02:27.420+0000 7efd7ca00700 1 -- 172.21.15.18:0/1717376142 shutdown_connections 2024-04-12T21:02:27.418 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:02:27.420+0000 7efd7ca00700 1 -- 172.21.15.18:0/1717376142 wait complete. 2024-04-12T21:02:27.570 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:02:27.573+0000 7f9b33c00700 1 -- 172.21.15.18:0/2740959795 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9b2c071f90 msgr2=0x7f9b2c070360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:02:27.571 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:02:27.573+0000 7f9b33c00700 1 --2- 172.21.15.18:0/2740959795 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9b2c071f90 0x7f9b2c070360 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f9b1c004970 tx=0x7f9b1c009e30 comp rx=0 tx=0).stop 2024-04-12T21:02:27.571 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:02:27.573+0000 7f9b33c00700 1 -- 172.21.15.18:0/2740959795 shutdown_connections 2024-04-12T21:02:27.571 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:02:27.573+0000 7f9b33c00700 1 --2- 172.21.15.18:0/2740959795 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9b2c0708a0 0x7f9b2c070d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:02:27.571 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:02:27.573+0000 7f9b33c00700 1 --2- 172.21.15.18:0/2740959795 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9b2c071f90 0x7f9b2c070360 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:02:27.571 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:02:27.573+0000 7f9b33c00700 1 -- 172.21.15.18:0/2740959795 >> 172.21.15.18:0/2740959795 conn(0x7f9b2c0f89b0 msgr2=0x7f9b2c0fadc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:02:27.571 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:02:27 smithi018 ceph-mon[112590]: pgmap v26: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-12T21:02:27.571 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:02:27 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' 2024-04-12T21:02:27.571 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:02:27 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' 2024-04-12T21:02:27.571 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:02:27 smithi018 ceph-mon[112590]: Reconfiguring osd.0 (monmap changed)... 2024-04-12T21:02:27.572 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:02:27 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-04-12T21:02:27.572 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:02:27 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-12T21:02:27.572 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:02:27 smithi018 ceph-mon[112590]: Reconfiguring daemon osd.0 on smithi018 2024-04-12T21:02:27.573 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:02:27.573+0000 7f9b33c00700 1 -- 172.21.15.18:0/2740959795 shutdown_connections 2024-04-12T21:02:27.573 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:02:27.573+0000 7f9b33c00700 1 -- 172.21.15.18:0/2740959795 wait complete. 2024-04-12T21:02:27.574 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:02:27.574+0000 7f9b33c00700 1 Processor -- start 2024-04-12T21:02:27.574 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:02:27.574+0000 7f9b33c00700 1 -- start start 2024-04-12T21:02:27.574 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:02:27.574+0000 7f9b33c00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9b2c0708a0 0x7f9b2c11a1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:02:27.574 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:02:27.574+0000 7f9b33c00700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9b2c071f90 0x7f9b2c11a6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:02:27.574 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:02:27.574+0000 7f9b33c00700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b2c11ad30 con 0x7f9b2c0708a0 2024-04-12T21:02:27.574 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:02:27.574+0000 7f9b33c00700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b2c114100 con 0x7f9b2c071f90 2024-04-12T21:02:27.574 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:02:27.575+0000 7f9b31200700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9b2c0708a0 0x7f9b2c11a1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:02:27.574 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:02:27.575+0000 7f9b31200700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9b2c0708a0 0x7f9b2c11a1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:36594/0 (socket says 172.21.15.18:36594) 2024-04-12T21:02:27.574 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:02:27.575+0000 7f9b31200700 1 -- 172.21.15.18:0/1102510770 learned_addr learned my addr 172.21.15.18:0/1102510770 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:02:27.574 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:02:27.575+0000 7f9b23e00700 1 --2- 172.21.15.18:0/1102510770 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9b2c071f90 0x7f9b2c11a6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:02:27.574 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:02:27.575+0000 7f9b31200700 1 -- 172.21.15.18:0/1102510770 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9b2c071f90 msgr2=0x7f9b2c11a6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:02:27.574 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:02:27.575+0000 7f9b31200700 1 --2- 172.21.15.18:0/1102510770 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9b2c071f90 0x7f9b2c11a6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:02:27.574 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:02:27.575+0000 7f9b31200700 1 -- 172.21.15.18:0/1102510770 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b1c005040 con 0x7f9b2c0708a0 2024-04-12T21:02:27.575 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:02:27.575+0000 7f9b23e00700 1 --2- 172.21.15.18:0/1102510770 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9b2c071f90 0x7f9b2c11a6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-12T21:02:27.575 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:02:27.576+0000 7f9b31200700 1 --2- 172.21.15.18:0/1102510770 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9b2c0708a0 0x7f9b2c11a1a0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f9b1c004f50 tx=0x7f9b1c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:02:27.575 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:02:27.576+0000 7f9b29600700 1 -- 172.21.15.18:0/1102510770 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9b1c00e040 con 0x7f9b2c0708a0 2024-04-12T21:02:27.575 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:02:27.576+0000 7f9b29600700 1 -- 172.21.15.18:0/1102510770 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f9b1c00bc10 con 0x7f9b2c0708a0 2024-04-12T21:02:27.575 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:02:27.576+0000 7f9b33c00700 1 -- 172.21.15.18:0/1102510770 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9b2c114380 con 0x7f9b2c0708a0 2024-04-12T21:02:27.575 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:02:27.576+0000 7f9b29600700 1 -- 172.21.15.18:0/1102510770 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9b1c01c630 con 0x7f9b2c0708a0 2024-04-12T21:02:27.575 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:02:27.576+0000 7f9b33c00700 1 -- 172.21.15.18:0/1102510770 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9b2c1148c0 con 0x7f9b2c0708a0 2024-04-12T21:02:27.575 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:02:27.577+0000 7f9b33c00700 1 -- 172.21.15.18:0/1102510770 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9b2c071aa0 con 0x7f9b2c0708a0 2024-04-12T21:02:27.576 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:02:27.579+0000 7f9b29600700 1 -- 172.21.15.18:0/1102510770 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 38) v1 ==== 92847+0+0 (secure 0 0 0) 0x7f9b1c013420 con 0x7f9b2c0708a0 2024-04-12T21:02:27.577 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:02:27.580+0000 7f9b29600700 1 --2- 172.21.15.18:0/1102510770 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f9b1006ed20 0x7f9b100711d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:02:27.577 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:02:27.580+0000 7f9b29600700 1 -- 172.21.15.18:0/1102510770 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6324+0+0 (secure 0 0 0) 0x7f9b1c08fcf0 con 0x7f9b2c0708a0 2024-04-12T21:02:27.577 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:02:27.580+0000 7f9b23e00700 1 --2- 172.21.15.18:0/1102510770 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f9b1006ed20 0x7f9b100711d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:02:27.578 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:02:27.581+0000 7f9b23e00700 1 --2- 172.21.15.18:0/1102510770 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f9b1006ed20 0x7f9b100711d0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f9b14007fd0 tx=0x7f9b14009040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:02:27.582 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:02:27.585+0000 7f9b29600700 1 -- 172.21.15.18:0/1102510770 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f9b1c05c570 con 0x7f9b2c0708a0 2024-04-12T21:02:27.780 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:02:27 smithi186 ceph-mon[86379]: pgmap v26: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-12T21:02:27.780 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:02:27 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' 2024-04-12T21:02:27.780 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:02:27 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' 2024-04-12T21:02:27.780 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:02:27 smithi186 ceph-mon[86379]: Reconfiguring osd.0 (monmap changed)... 2024-04-12T21:02:27.780 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:02:27 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-04-12T21:02:27.780 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:02:27 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-12T21:02:27.780 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:02:27 smithi186 ceph-mon[86379]: Reconfiguring daemon osd.0 on smithi018 2024-04-12T21:02:27.913 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:02:27.915+0000 7f9b33c00700 1 -- 172.21.15.18:0/1102510770 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f9b2c071aa0 con 0x7f9b2c0708a0 2024-04-12T21:02:27.914 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:02:27.917+0000 7f9b29600700 1 -- 172.21.15.18:0/1102510770 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+736 (secure 0 0 0) 0x7f9b1c05c100 con 0x7f9b2c0708a0 2024-04-12T21:02:27.915 INFO:teuthology.orchestra.run.smithi018.stdout:{ 2024-04-12T21:02:27.915 INFO:teuthology.orchestra.run.smithi018.stdout: "mon": { 2024-04-12T21:02:27.915 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 19.0.0-2879-g324baf8d (324baf8ddb91dea46cac50cb535c306115791355) squid (dev)": 2 2024-04-12T21:02:27.915 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-12T21:02:27.915 INFO:teuthology.orchestra.run.smithi018.stdout: "mgr": { 2024-04-12T21:02:27.915 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 19.0.0-2879-g324baf8d (324baf8ddb91dea46cac50cb535c306115791355) squid (dev)": 2 2024-04-12T21:02:27.915 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-12T21:02:27.915 INFO:teuthology.orchestra.run.smithi018.stdout: "osd": { 2024-04-12T21:02:27.916 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 17.2.7-827-gd3b5060a (d3b5060a17449edfae9385cdf792afec109fca5d) quincy (stable)": 6 2024-04-12T21:02:27.916 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-12T21:02:27.916 INFO:teuthology.orchestra.run.smithi018.stdout: "mds": { 2024-04-12T21:02:27.916 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 17.2.7-827-gd3b5060a (d3b5060a17449edfae9385cdf792afec109fca5d) quincy (stable)": 4 2024-04-12T21:02:27.916 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-12T21:02:27.916 INFO:teuthology.orchestra.run.smithi018.stdout: "overall": { 2024-04-12T21:02:27.916 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 17.2.7-827-gd3b5060a (d3b5060a17449edfae9385cdf792afec109fca5d) quincy (stable)": 10, 2024-04-12T21:02:27.916 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 19.0.0-2879-g324baf8d (324baf8ddb91dea46cac50cb535c306115791355) squid (dev)": 4 2024-04-12T21:02:27.916 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-04-12T21:02:27.916 INFO:teuthology.orchestra.run.smithi018.stdout:} 2024-04-12T21:02:27.918 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:02:27.920+0000 7f9b33c00700 1 -- 172.21.15.18:0/1102510770 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f9b1006ed20 msgr2=0x7f9b100711d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:02:27.918 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:02:27.920+0000 7f9b33c00700 1 --2- 172.21.15.18:0/1102510770 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f9b1006ed20 0x7f9b100711d0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f9b14007fd0 tx=0x7f9b14009040 comp rx=0 tx=0).stop 2024-04-12T21:02:27.918 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:02:27.920+0000 7f9b33c00700 1 -- 172.21.15.18:0/1102510770 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9b2c0708a0 msgr2=0x7f9b2c11a1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:02:27.918 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:02:27.921+0000 7f9b33c00700 1 --2- 172.21.15.18:0/1102510770 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9b2c0708a0 0x7f9b2c11a1a0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f9b1c004f50 tx=0x7f9b1c009e30 comp rx=0 tx=0).stop 2024-04-12T21:02:27.919 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:02:27.921+0000 7f9b33c00700 1 -- 172.21.15.18:0/1102510770 shutdown_connections 2024-04-12T21:02:27.919 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:02:27.921+0000 7f9b33c00700 1 --2- 172.21.15.18:0/1102510770 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f9b1006ed20 0x7f9b100711d0 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:02:27.919 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:02:27.921+0000 7f9b33c00700 1 --2- 172.21.15.18:0/1102510770 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9b2c0708a0 0x7f9b2c11a1a0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:02:27.919 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:02:27.921+0000 7f9b33c00700 1 --2- 172.21.15.18:0/1102510770 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9b2c071f90 0x7f9b2c11a6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:02:27.919 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:02:27.921+0000 7f9b33c00700 1 -- 172.21.15.18:0/1102510770 >> 172.21.15.18:0/1102510770 conn(0x7f9b2c0f89b0 msgr2=0x7f9b2c0f9aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:02:27.919 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:02:27.921+0000 7f9b33c00700 1 -- 172.21.15.18:0/1102510770 shutdown_connections 2024-04-12T21:02:27.919 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:02:27.921+0000 7f9b33c00700 1 -- 172.21.15.18:0/1102510770 wait complete. 2024-04-12T21:02:28.067 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:02:28.070+0000 7f6ac6600700 1 -- 172.21.15.18:0/2025713458 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6ac0102e00 msgr2=0x7f6ac01031d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:02:28.067 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:02:28.070+0000 7f6ac6600700 1 --2- 172.21.15.18:0/2025713458 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6ac0102e00 0x7f6ac01031d0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f6ab0004940 tx=0x7f6ab0009e30 comp rx=0 tx=0).stop 2024-04-12T21:02:28.068 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:02:28.070+0000 7f6ac6600700 1 -- 172.21.15.18:0/2025713458 shutdown_connections 2024-04-12T21:02:28.068 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:02:28.070+0000 7f6ac6600700 1 --2- 172.21.15.18:0/2025713458 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6ac01037a0 0x7f6ac00fce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:02:28.068 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:02:28.070+0000 7f6ac6600700 1 --2- 172.21.15.18:0/2025713458 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6ac0102e00 0x7f6ac01031d0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:02:28.068 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:02:28.070+0000 7f6ac6600700 1 -- 172.21.15.18:0/2025713458 >> 172.21.15.18:0/2025713458 conn(0x7f6ac00f89b0 msgr2=0x7f6ac00fadc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:02:28.068 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:02:28.070+0000 7f6ac6600700 1 -- 172.21.15.18:0/2025713458 shutdown_connections 2024-04-12T21:02:28.068 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:02:28.070+0000 7f6ac6600700 1 -- 172.21.15.18:0/2025713458 wait complete. 2024-04-12T21:02:28.069 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:02:28.071+0000 7f6ac6600700 1 Processor -- start 2024-04-12T21:02:28.069 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:02:28.071+0000 7f6ac6600700 1 -- start start 2024-04-12T21:02:28.069 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:02:28.071+0000 7f6ac6600700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6ac0102e00 0x7f6ac0115b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:02:28.069 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:02:28.071+0000 7f6ac6600700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6ac01037a0 0x7f6ac01160a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:02:28.069 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:02:28.071+0000 7f6ac6600700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ac0116780 con 0x7f6ac0102e00 2024-04-12T21:02:28.070 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:02:28.071+0000 7f6ac6600700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ac010fcd0 con 0x7f6ac01037a0 2024-04-12T21:02:28.070 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:02:28.071+0000 7f6abf400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6ac0102e00 0x7f6ac0115b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:02:28.070 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:02:28.072+0000 7f6abf400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6ac0102e00 0x7f6ac0115b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:36608/0 (socket says 172.21.15.18:36608) 2024-04-12T21:02:28.070 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:02:28.072+0000 7f6abf400700 1 -- 172.21.15.18:0/1231094355 learned_addr learned my addr 172.21.15.18:0/1231094355 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:02:28.070 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:02:28.072+0000 7f6abf400700 1 -- 172.21.15.18:0/1231094355 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6ac01037a0 msgr2=0x7f6ac01160a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:02:28.070 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:02:28.072+0000 7f6abf400700 1 --2- 172.21.15.18:0/1231094355 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6ac01037a0 0x7f6ac01160a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:02:28.070 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:02:28.072+0000 7f6abf400700 1 -- 172.21.15.18:0/1231094355 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ab0005040 con 0x7f6ac0102e00 2024-04-12T21:02:28.070 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:02:28.072+0000 7f6abf400700 1 --2- 172.21.15.18:0/1231094355 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6ac0102e00 0x7f6ac0115b60 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f6ab0009150 tx=0x7f6ab0009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:02:28.071 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:02:28.072+0000 7f6ab7400700 1 -- 172.21.15.18:0/1231094355 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6ab000e040 con 0x7f6ac0102e00 2024-04-12T21:02:28.071 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:02:28.073+0000 7f6ab7400700 1 -- 172.21.15.18:0/1231094355 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f6ab0013420 con 0x7f6ac0102e00 2024-04-12T21:02:28.071 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:02:28.073+0000 7f6ab7400700 1 -- 172.21.15.18:0/1231094355 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6ab001c630 con 0x7f6ac0102e00 2024-04-12T21:02:28.071 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:02:28.073+0000 7f6ac6600700 1 -- 172.21.15.18:0/1231094355 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6ac010fef0 con 0x7f6ac0102e00 2024-04-12T21:02:28.071 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:02:28.073+0000 7f6ac6600700 1 -- 172.21.15.18:0/1231094355 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6ac01104b0 con 0x7f6ac0102e00 2024-04-12T21:02:28.071 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:02:28.074+0000 7f6ac6600700 1 -- 172.21.15.18:0/1231094355 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6ac00fe550 con 0x7f6ac0102e00 2024-04-12T21:02:28.072 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:02:28.075+0000 7f6ab7400700 1 -- 172.21.15.18:0/1231094355 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 38) v1 ==== 92847+0+0 (secure 0 0 0) 0x7f6ab0013590 con 0x7f6ac0102e00 2024-04-12T21:02:28.073 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:02:28.076+0000 7f6ab7400700 1 --2- 172.21.15.18:0/1231094355 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f6aa406ed30 0x7f6aa40711e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:02:28.073 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:02:28.076+0000 7f6ab7400700 1 -- 172.21.15.18:0/1231094355 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6324+0+0 (secure 0 0 0) 0x7f6ab008fc10 con 0x7f6ac0102e00 2024-04-12T21:02:28.073 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:02:28.076+0000 7f6abea00700 1 --2- 172.21.15.18:0/1231094355 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f6aa406ed30 0x7f6aa40711e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:02:28.074 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:02:28.077+0000 7f6abea00700 1 --2- 172.21.15.18:0/1231094355 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f6aa406ed30 0x7f6aa40711e0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f6aac006fd0 tx=0x7f6aac006db0 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:02:28.078 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:02:28.081+0000 7f6ab7400700 1 -- 172.21.15.18:0/1231094355 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f6ab005c490 con 0x7f6ac0102e00 2024-04-12T21:02:28.360 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:02:28.362+0000 7f6ac6600700 1 -- 172.21.15.18:0/1231094355 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f6ac010a000 con 0x7f6ac0102e00 2024-04-12T21:02:28.360 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:02:28.363+0000 7f6ab7400700 1 -- 172.21.15.18:0/1231094355 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 23 v23) v1 ==== 76+0+1886 (secure 0 0 0) 0x7f6ab005c020 con 0x7f6ac0102e00 2024-04-12T21:02:28.362 INFO:teuthology.orchestra.run.smithi018.stdout:e23 2024-04-12T21:02:28.362 INFO:teuthology.orchestra.run.smithi018.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-12T21:02:28.362 INFO:teuthology.orchestra.run.smithi018.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-12T21:02:28.362 INFO:teuthology.orchestra.run.smithi018.stdout:legacy client fscid: 1 2024-04-12T21:02:28.363 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-12T21:02:28.363 INFO:teuthology.orchestra.run.smithi018.stdout:Filesystem 'cephfs' (1) 2024-04-12T21:02:28.363 INFO:teuthology.orchestra.run.smithi018.stdout:fs_name cephfs 2024-04-12T21:02:28.363 INFO:teuthology.orchestra.run.smithi018.stdout:epoch 23 2024-04-12T21:02:28.363 INFO:teuthology.orchestra.run.smithi018.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-04-12T21:02:28.363 INFO:teuthology.orchestra.run.smithi018.stdout:created 2024-04-12T20:51:57.362850+0000 2024-04-12T21:02:28.363 INFO:teuthology.orchestra.run.smithi018.stdout:modified 2024-04-12T21:01:10.670287+0000 2024-04-12T21:02:28.363 INFO:teuthology.orchestra.run.smithi018.stdout:tableserver 0 2024-04-12T21:02:28.363 INFO:teuthology.orchestra.run.smithi018.stdout:root 0 2024-04-12T21:02:28.363 INFO:teuthology.orchestra.run.smithi018.stdout:session_timeout 60 2024-04-12T21:02:28.363 INFO:teuthology.orchestra.run.smithi018.stdout:session_autoclose 300 2024-04-12T21:02:28.363 INFO:teuthology.orchestra.run.smithi018.stdout:max_file_size 1099511627776 2024-04-12T21:02:28.364 INFO:teuthology.orchestra.run.smithi018.stdout:max_xattr_size 65536 2024-04-12T21:02:28.364 INFO:teuthology.orchestra.run.smithi018.stdout:required_client_features {} 2024-04-12T21:02:28.364 INFO:teuthology.orchestra.run.smithi018.stdout:last_failure 0 2024-04-12T21:02:28.364 INFO:teuthology.orchestra.run.smithi018.stdout:last_failure_osd_epoch 46 2024-04-12T21:02:28.364 INFO:teuthology.orchestra.run.smithi018.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-12T21:02:28.364 INFO:teuthology.orchestra.run.smithi018.stdout:max_mds 2 2024-04-12T21:02:28.364 INFO:teuthology.orchestra.run.smithi018.stdout:in 0,1 2024-04-12T21:02:28.364 INFO:teuthology.orchestra.run.smithi018.stdout:up {0=14452,1=14464} 2024-04-12T21:02:28.364 INFO:teuthology.orchestra.run.smithi018.stdout:failed 2024-04-12T21:02:28.364 INFO:teuthology.orchestra.run.smithi018.stdout:damaged 2024-04-12T21:02:28.364 INFO:teuthology.orchestra.run.smithi018.stdout:stopped 2024-04-12T21:02:28.365 INFO:teuthology.orchestra.run.smithi018.stdout:data_pools [3] 2024-04-12T21:02:28.365 INFO:teuthology.orchestra.run.smithi018.stdout:metadata_pool 2 2024-04-12T21:02:28.365 INFO:teuthology.orchestra.run.smithi018.stdout:inline_data disabled 2024-04-12T21:02:28.365 INFO:teuthology.orchestra.run.smithi018.stdout:balancer 2024-04-12T21:02:28.365 INFO:teuthology.orchestra.run.smithi018.stdout:bal_rank_mask -1 2024-04-12T21:02:28.365 INFO:teuthology.orchestra.run.smithi018.stdout:standby_count_wanted 1 2024-04-12T21:02:28.365 INFO:teuthology.orchestra.run.smithi018.stdout:qdb_cluster leader: 0 members: 2024-04-12T21:02:28.365 INFO:teuthology.orchestra.run.smithi018.stdout:[mds.cephfs.smithi018.acqbsc{0:14452} state up:active seq 5 join_fscid=1 addr [v2:172.21.15.18:6826/1564065434,v1:172.21.15.18:6827/1564065434] compat {c=[1],r=[1],i=[7ff]}] 2024-04-12T21:02:28.365 INFO:teuthology.orchestra.run.smithi018.stdout:[mds.cephfs.smithi018.szjluo{1:14464} state up:active seq 6 join_fscid=1 addr [v2:172.21.15.18:6828/722443353,v1:172.21.15.18:6829/722443353] compat {c=[1],r=[1],i=[7ff]}] 2024-04-12T21:02:28.365 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-12T21:02:28.365 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-12T21:02:28.365 INFO:teuthology.orchestra.run.smithi018.stdout:Standby daemons: 2024-04-12T21:02:28.365 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-12T21:02:28.365 INFO:teuthology.orchestra.run.smithi018.stdout:[mds.cephfs.smithi186.wjnafs{-1:24277} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.186:6826/438347207,v1:172.21.15.186:6827/438347207] compat {c=[1],r=[1],i=[7ff]}] 2024-04-12T21:02:28.366 INFO:teuthology.orchestra.run.smithi018.stdout:[mds.cephfs.smithi186.jdllyn{-1:24281} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.186:6824/1464359952,v1:172.21.15.186:6825/1464359952] compat {c=[1],r=[1],i=[7ff]}] 2024-04-12T21:02:28.367 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:02:28.366+0000 7f6ac6600700 1 -- 172.21.15.18:0/1231094355 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f6aa406ed30 msgr2=0x7f6aa40711e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:02:28.367 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:02:28.366+0000 7f6ac6600700 1 --2- 172.21.15.18:0/1231094355 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f6aa406ed30 0x7f6aa40711e0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f6aac006fd0 tx=0x7f6aac006db0 comp rx=0 tx=0).stop 2024-04-12T21:02:28.367 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:02:28.366+0000 7f6ac6600700 1 -- 172.21.15.18:0/1231094355 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6ac0102e00 msgr2=0x7f6ac0115b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:02:28.367 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:02:28.366+0000 7f6ac6600700 1 --2- 172.21.15.18:0/1231094355 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6ac0102e00 0x7f6ac0115b60 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f6ab0009150 tx=0x7f6ab0009e30 comp rx=0 tx=0).stop 2024-04-12T21:02:28.367 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:02:28.366+0000 7f6ac6600700 1 -- 172.21.15.18:0/1231094355 shutdown_connections 2024-04-12T21:02:28.367 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:02:28.366+0000 7f6ac6600700 1 --2- 172.21.15.18:0/1231094355 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f6aa406ed30 0x7f6aa40711e0 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:02:28.367 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:02:28.366+0000 7f6ac6600700 1 --2- 172.21.15.18:0/1231094355 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6ac0102e00 0x7f6ac0115b60 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:02:28.367 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:02:28.366+0000 7f6ac6600700 1 --2- 172.21.15.18:0/1231094355 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6ac01037a0 0x7f6ac01160a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:02:28.367 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:02:28.366+0000 7f6ac6600700 1 -- 172.21.15.18:0/1231094355 >> 172.21.15.18:0/1231094355 conn(0x7f6ac00f89b0 msgr2=0x7f6ac0071c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:02:28.367 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:02:28.366+0000 7f6ac6600700 1 -- 172.21.15.18:0/1231094355 shutdown_connections 2024-04-12T21:02:28.367 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:02:28.366+0000 7f6ac6600700 1 -- 172.21.15.18:0/1231094355 wait complete. 2024-04-12T21:02:28.368 INFO:teuthology.orchestra.run.smithi018.stderr:dumped fsmap epoch 23 2024-04-12T21:02:28.527 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:02:28.529+0000 7f485f800700 1 -- 172.21.15.18:0/602533416 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4858109dd0 msgr2=0x7f4858103440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:02:28.527 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:02:28.529+0000 7f485f800700 1 --2- 172.21.15.18:0/602533416 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4858109dd0 0x7f4858103440 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f4844004940 tx=0x7f4844009e30 comp rx=0 tx=0).stop 2024-04-12T21:02:28.527 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:02:28.529+0000 7f485f800700 1 -- 172.21.15.18:0/602533416 shutdown_connections 2024-04-12T21:02:28.527 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:02:28.529+0000 7f485f800700 1 --2- 172.21.15.18:0/602533416 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4858109dd0 0x7f4858103440 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:02:28.527 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:02:28.529+0000 7f485f800700 1 --2- 172.21.15.18:0/602533416 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4858109430 0x7f4858109800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:02:28.527 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:02:28.529+0000 7f485f800700 1 -- 172.21.15.18:0/602533416 >> 172.21.15.18:0/602533416 conn(0x7f4858072c00 msgr2=0x7f4858075010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:02:28.527 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:02:28.529+0000 7f485f800700 1 -- 172.21.15.18:0/602533416 shutdown_connections 2024-04-12T21:02:28.527 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:02:28.529+0000 7f485f800700 1 -- 172.21.15.18:0/602533416 wait complete. 2024-04-12T21:02:28.529 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:02:28.530+0000 7f485f800700 1 Processor -- start 2024-04-12T21:02:28.529 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:02:28.530+0000 7f485f800700 1 -- start start 2024-04-12T21:02:28.529 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:02:28.531+0000 7f485f800700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4858109430 0x7f4858119f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:02:28.529 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:02:28.531+0000 7f485f800700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4858109dd0 0x7f485811a4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:02:28.529 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:02:28.531+0000 7f485f800700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f485811ab10 con 0x7f4858109430 2024-04-12T21:02:28.529 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:02:28.531+0000 7f485f800700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48581140f0 con 0x7f4858109dd0 2024-04-12T21:02:28.529 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:02:28.531+0000 7f485ce00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4858109430 0x7f4858119f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:02:28.529 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:02:28.531+0000 7f485ce00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4858109430 0x7f4858119f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:36622/0 (socket says 172.21.15.18:36622) 2024-04-12T21:02:28.529 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:02:28.531+0000 7f485ce00700 1 -- 172.21.15.18:0/1213034572 learned_addr learned my addr 172.21.15.18:0/1213034572 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:02:28.529 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:02:28.531+0000 7f4857e00700 1 --2- 172.21.15.18:0/1213034572 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4858109dd0 0x7f485811a4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:02:28.530 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:02:28 smithi018 ceph-mon[112590]: from='client.34160 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:02:28.530 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:02:28 smithi018 ceph-mon[112590]: from='client.34164 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:02:28.530 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:02:28 smithi018 ceph-mon[112590]: from='client.34168 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:02:28.530 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:02:28 smithi018 ceph-mon[112590]: from='client.? 172.21.15.18:0/1102510770' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T21:02:28.530 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:02:28.531+0000 7f485ce00700 1 -- 172.21.15.18:0/1213034572 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4858109dd0 msgr2=0x7f485811a4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:02:28.531 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:02:28.531+0000 7f485ce00700 1 --2- 172.21.15.18:0/1213034572 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4858109dd0 0x7f485811a4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:02:28.531 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:02:28.531+0000 7f485ce00700 1 -- 172.21.15.18:0/1213034572 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4844005040 con 0x7f4858109430 2024-04-12T21:02:28.531 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:02:28.532+0000 7f485ce00700 1 --2- 172.21.15.18:0/1213034572 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4858109430 0x7f4858119f80 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f484c00c8c0 tx=0x7f484c0044c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:02:28.531 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:02:28.532+0000 7f4854c00700 1 -- 172.21.15.18:0/1213034572 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f484c00a980 con 0x7f4858109430 2024-04-12T21:02:28.531 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:02:28.532+0000 7f4854c00700 1 -- 172.21.15.18:0/1213034572 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f484c00ce60 con 0x7f4858109430 2024-04-12T21:02:28.531 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:02:28.532+0000 7f485f800700 1 -- 172.21.15.18:0/1213034572 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f48581143d0 con 0x7f4858109430 2024-04-12T21:02:28.531 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:02:28.532+0000 7f4854c00700 1 -- 172.21.15.18:0/1213034572 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f484c0115e0 con 0x7f4858109430 2024-04-12T21:02:28.531 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:02:28.532+0000 7f485f800700 1 -- 172.21.15.18:0/1213034572 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4858114990 con 0x7f4858109430 2024-04-12T21:02:28.532 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:02:28.534+0000 7f485f800700 1 -- 172.21.15.18:0/1213034572 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4858104b80 con 0x7f4858109430 2024-04-12T21:02:28.532 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:02:28.534+0000 7f4854c00700 1 -- 172.21.15.18:0/1213034572 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 38) v1 ==== 92847+0+0 (secure 0 0 0) 0x7f484c012400 con 0x7f4858109430 2024-04-12T21:02:28.533 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:02:28.536+0000 7f4854c00700 1 --2- 172.21.15.18:0/1213034572 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f484006ed30 0x7f48400711e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:02:28.533 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:02:28.536+0000 7f4854c00700 1 -- 172.21.15.18:0/1213034572 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6324+0+0 (secure 0 0 0) 0x7f484c08ea00 con 0x7f4858109430 2024-04-12T21:02:28.534 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:02:28.536+0000 7f4857e00700 1 --2- 172.21.15.18:0/1213034572 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f484006ed30 0x7f48400711e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:02:28.534 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:02:28.537+0000 7f4857e00700 1 --2- 172.21.15.18:0/1213034572 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f484006ed30 0x7f48400711e0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f4844000c00 tx=0x7f4844009e30 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:02:28.538 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:02:28.541+0000 7f4854c00700 1 -- 172.21.15.18:0/1213034572 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f484c05b2d0 con 0x7f4858109430 2024-04-12T21:02:28.780 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:02:28 smithi186 ceph-mon[86379]: from='client.34160 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:02:28.780 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:02:28 smithi186 ceph-mon[86379]: from='client.34164 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:02:28.780 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:02:28 smithi186 ceph-mon[86379]: from='client.34168 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:02:28.780 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:02:28 smithi186 ceph-mon[86379]: from='client.? 172.21.15.18:0/1102510770' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T21:02:28.841 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:02:28.843+0000 7f485f800700 1 -- 172.21.15.18:0/1213034572 --> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f48581155d0 con 0x7f484006ed30 2024-04-12T21:02:28.843 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:02:28.846+0000 7f4854c00700 1 -- 172.21.15.18:0/1213034572 <== mgr.34102 v2:172.21.15.18:6800/3812891919 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+383 (secure 0 0 0) 0x7f48581155d0 con 0x7f484006ed30 2024-04-12T21:02:28.844 INFO:teuthology.orchestra.run.smithi018.stdout:{ 2024-04-12T21:02:28.844 INFO:teuthology.orchestra.run.smithi018.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:f4452def3be4d350f4a8e5feaf7ebb1511be9298fe68478ba7a2d6fbd7d69a63", 2024-04-12T21:02:28.844 INFO:teuthology.orchestra.run.smithi018.stdout: "in_progress": true, 2024-04-12T21:02:28.844 INFO:teuthology.orchestra.run.smithi018.stdout: "which": "Upgrading all daemon types on all hosts", 2024-04-12T21:02:28.844 INFO:teuthology.orchestra.run.smithi018.stdout: "services_complete": [ 2024-04-12T21:02:28.844 INFO:teuthology.orchestra.run.smithi018.stdout: "mon", 2024-04-12T21:02:28.844 INFO:teuthology.orchestra.run.smithi018.stdout: "mgr" 2024-04-12T21:02:28.844 INFO:teuthology.orchestra.run.smithi018.stdout: ], 2024-04-12T21:02:28.844 INFO:teuthology.orchestra.run.smithi018.stdout: "progress": "4/21 daemons upgraded", 2024-04-12T21:02:28.844 INFO:teuthology.orchestra.run.smithi018.stdout: "message": "Currently upgrading mon daemons", 2024-04-12T21:02:28.844 INFO:teuthology.orchestra.run.smithi018.stdout: "is_paused": false 2024-04-12T21:02:28.844 INFO:teuthology.orchestra.run.smithi018.stdout:} 2024-04-12T21:02:28.846 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:02:28.849+0000 7f485f800700 1 -- 172.21.15.18:0/1213034572 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f484006ed30 msgr2=0x7f48400711e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:02:28.847 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:02:28.849+0000 7f485f800700 1 --2- 172.21.15.18:0/1213034572 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f484006ed30 0x7f48400711e0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f4844000c00 tx=0x7f4844009e30 comp rx=0 tx=0).stop 2024-04-12T21:02:28.847 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:02:28.849+0000 7f485f800700 1 -- 172.21.15.18:0/1213034572 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4858109430 msgr2=0x7f4858119f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:02:28.847 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:02:28.849+0000 7f485f800700 1 --2- 172.21.15.18:0/1213034572 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4858109430 0x7f4858119f80 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f484c00c8c0 tx=0x7f484c0044c0 comp rx=0 tx=0).stop 2024-04-12T21:02:28.847 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:02:28.849+0000 7f485f800700 1 -- 172.21.15.18:0/1213034572 shutdown_connections 2024-04-12T21:02:28.847 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:02:28.849+0000 7f485f800700 1 --2- 172.21.15.18:0/1213034572 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f484006ed30 0x7f48400711e0 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:02:28.847 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:02:28.849+0000 7f485f800700 1 --2- 172.21.15.18:0/1213034572 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4858109430 0x7f4858119f80 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:02:28.847 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:02:28.849+0000 7f485f800700 1 --2- 172.21.15.18:0/1213034572 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4858109dd0 0x7f485811a4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:02:28.847 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:02:28.849+0000 7f485f800700 1 -- 172.21.15.18:0/1213034572 >> 172.21.15.18:0/1213034572 conn(0x7f4858072c00 msgr2=0x7f4858074c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:02:28.847 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:02:28.850+0000 7f485f800700 1 -- 172.21.15.18:0/1213034572 shutdown_connections 2024-04-12T21:02:28.848 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:02:28.850+0000 7f485f800700 1 -- 172.21.15.18:0/1213034572 wait complete. 2024-04-12T21:02:28.997 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:02:28.999+0000 7f81ab600700 1 -- 172.21.15.18:0/3562543329 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f81a4105010 msgr2=0x7f81a41053e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:02:28.997 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:02:28.999+0000 7f81ab600700 1 --2- 172.21.15.18:0/3562543329 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f81a4105010 0x7f81a41053e0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f8198004940 tx=0x7f8198009e30 comp rx=0 tx=0).stop 2024-04-12T21:02:28.997 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:02:28.999+0000 7f81ab600700 1 -- 172.21.15.18:0/3562543329 shutdown_connections 2024-04-12T21:02:28.997 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:02:28.999+0000 7f81ab600700 1 --2- 172.21.15.18:0/3562543329 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f81a41059b0 0x7f81a40ff020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:02:28.997 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:02:28.999+0000 7f81ab600700 1 --2- 172.21.15.18:0/3562543329 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f81a4105010 0x7f81a41053e0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:02:28.997 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:02:28.999+0000 7f81ab600700 1 -- 172.21.15.18:0/3562543329 >> 172.21.15.18:0/3562543329 conn(0x7f81a40faba0 msgr2=0x7f81a40fcfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:02:28.997 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:02:29.000+0000 7f81ab600700 1 -- 172.21.15.18:0/3562543329 shutdown_connections 2024-04-12T21:02:28.998 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:02:29.000+0000 7f81ab600700 1 -- 172.21.15.18:0/3562543329 wait complete. 2024-04-12T21:02:28.998 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:02:29.001+0000 7f81ab600700 1 Processor -- start 2024-04-12T21:02:28.998 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:02:29.001+0000 7f81ab600700 1 -- start start 2024-04-12T21:02:28.998 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:02:29.001+0000 7f81ab600700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f81a4105010 0x7f81a4115da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:02:28.998 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:02:29.001+0000 7f81ab600700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f81a41059b0 0x7f81a41162e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:02:28.999 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:02:29.001+0000 7f81ab600700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81a4116930 con 0x7f81a4105010 2024-04-12T21:02:28.999 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:02:29.001+0000 7f81ab600700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81a410fd00 con 0x7f81a41059b0 2024-04-12T21:02:28.999 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:02:29.001+0000 7f81a8c00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f81a4105010 0x7f81a4115da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:02:28.999 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:02:29.002+0000 7f81a8c00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f81a4105010 0x7f81a4115da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:36642/0 (socket says 172.21.15.18:36642) 2024-04-12T21:02:28.999 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:02:29.002+0000 7f81a8c00700 1 -- 172.21.15.18:0/3979558014 learned_addr learned my addr 172.21.15.18:0/3979558014 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:02:29.000 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:02:29.002+0000 7f81a3e00700 1 --2- 172.21.15.18:0/3979558014 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f81a41059b0 0x7f81a41162e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:02:29.000 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:02:29.002+0000 7f81a8c00700 1 -- 172.21.15.18:0/3979558014 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f81a41059b0 msgr2=0x7f81a41162e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:02:29.000 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:02:29.002+0000 7f81a8c00700 1 --2- 172.21.15.18:0/3979558014 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f81a41059b0 0x7f81a41162e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:02:29.000 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:02:29.002+0000 7f81a8c00700 1 -- 172.21.15.18:0/3979558014 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8198005040 con 0x7f81a4105010 2024-04-12T21:02:29.000 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:02:29.002+0000 7f81a3e00700 1 --2- 172.21.15.18:0/3979558014 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f81a41059b0 0x7f81a41162e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-12T21:02:29.000 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:02:29.002+0000 7f81a8c00700 1 --2- 172.21.15.18:0/3979558014 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f81a4105010 0x7f81a4115da0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f8198009180 tx=0x7f8198009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:02:29.001 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:02:29.003+0000 7f81a0c00700 1 -- 172.21.15.18:0/3979558014 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f819800e040 con 0x7f81a4105010 2024-04-12T21:02:29.001 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:02:29.003+0000 7f81a0c00700 1 -- 172.21.15.18:0/3979558014 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f819800bbc0 con 0x7f81a4105010 2024-04-12T21:02:29.001 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:02:29.003+0000 7f81a0c00700 1 -- 172.21.15.18:0/3979558014 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f819801c630 con 0x7f81a4105010 2024-04-12T21:02:29.001 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:02:29.003+0000 7f81ab600700 1 -- 172.21.15.18:0/3979558014 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f81a410ff80 con 0x7f81a4105010 2024-04-12T21:02:29.001 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:02:29.003+0000 7f81ab600700 1 -- 172.21.15.18:0/3979558014 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f81a41104f0 con 0x7f81a4105010 2024-04-12T21:02:29.002 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:02:29.004+0000 7f81ab600700 1 -- 172.21.15.18:0/3979558014 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f81a4100760 con 0x7f81a4105010 2024-04-12T21:02:29.002 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:02:29.005+0000 7f81a0c00700 1 -- 172.21.15.18:0/3979558014 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 38) v1 ==== 92847+0+0 (secure 0 0 0) 0x7f819800bd30 con 0x7f81a4105010 2024-04-12T21:02:29.004 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:02:29.007+0000 7f81a0c00700 1 --2- 172.21.15.18:0/3979558014 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f818c06ed20 0x7f818c0711d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:02:29.004 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:02:29.007+0000 7f81a0c00700 1 -- 172.21.15.18:0/3979558014 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6324+0+0 (secure 0 0 0) 0x7f819808fee0 con 0x7f81a4105010 2024-04-12T21:02:29.004 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:02:29.007+0000 7f81a3e00700 1 --2- 172.21.15.18:0/3979558014 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f818c06ed20 0x7f818c0711d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:02:29.005 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:02:29.008+0000 7f81a3e00700 1 --2- 172.21.15.18:0/3979558014 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f818c06ed20 0x7f818c0711d0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f8190007fd0 tx=0x7f8190009040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:02:29.009 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:02:29.012+0000 7f81a0c00700 1 -- 172.21.15.18:0/3979558014 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f819805c620 con 0x7f81a4105010 2024-04-12T21:02:29.337 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:02:29.339+0000 7f81ab600700 1 -- 172.21.15.18:0/3979558014 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f81a404bed0 con 0x7f81a4105010 2024-04-12T21:02:29.337 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:02:29.340+0000 7f81a0c00700 1 -- 172.21.15.18:0/3979558014 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+10 (secure 0 0 0) 0x7f819805c1b0 con 0x7f81a4105010 2024-04-12T21:02:29.337 INFO:teuthology.orchestra.run.smithi018.stdout:HEALTH_OK 2024-04-12T21:02:29.341 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:02:29.343+0000 7f81ab600700 1 -- 172.21.15.18:0/3979558014 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f818c06ed20 msgr2=0x7f818c0711d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:02:29.341 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:02:29.343+0000 7f81ab600700 1 --2- 172.21.15.18:0/3979558014 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f818c06ed20 0x7f818c0711d0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f8190007fd0 tx=0x7f8190009040 comp rx=0 tx=0).stop 2024-04-12T21:02:29.341 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:02:29.343+0000 7f81ab600700 1 -- 172.21.15.18:0/3979558014 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f81a4105010 msgr2=0x7f81a4115da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:02:29.341 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:02:29.343+0000 7f81ab600700 1 --2- 172.21.15.18:0/3979558014 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f81a4105010 0x7f81a4115da0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f8198009180 tx=0x7f8198009e30 comp rx=0 tx=0).stop 2024-04-12T21:02:29.341 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:02:29.343+0000 7f81ab600700 1 -- 172.21.15.18:0/3979558014 shutdown_connections 2024-04-12T21:02:29.341 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:02:29.343+0000 7f81ab600700 1 --2- 172.21.15.18:0/3979558014 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f818c06ed20 0x7f818c0711d0 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:02:29.341 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:02:29.343+0000 7f81ab600700 1 --2- 172.21.15.18:0/3979558014 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f81a4105010 0x7f81a4115da0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:02:29.341 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:02:29.343+0000 7f81ab600700 1 --2- 172.21.15.18:0/3979558014 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f81a41059b0 0x7f81a41162e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:02:29.341 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:02:29.343+0000 7f81ab600700 1 -- 172.21.15.18:0/3979558014 >> 172.21.15.18:0/3979558014 conn(0x7f81a40faba0 msgr2=0x7f81a40fbc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:02:29.341 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:02:29.343+0000 7f81ab600700 1 -- 172.21.15.18:0/3979558014 shutdown_connections 2024-04-12T21:02:29.342 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:02:29.343+0000 7f81ab600700 1 -- 172.21.15.18:0/3979558014 wait complete. 2024-04-12T21:02:29.600 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:02:29 smithi018 ceph-mon[112590]: pgmap v27: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-12T21:02:29.601 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:02:29 smithi018 ceph-mon[112590]: from='client.? 172.21.15.18:0/1231094355' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-12T21:02:29.601 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:02:29 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' 2024-04-12T21:02:29.601 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:02:29 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' 2024-04-12T21:02:29.601 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:02:29 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-04-12T21:02:29.601 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:02:29 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-12T21:02:29.780 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:02:29 smithi186 ceph-mon[86379]: pgmap v27: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-12T21:02:29.780 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:02:29 smithi186 ceph-mon[86379]: from='client.? 172.21.15.18:0/1231094355' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-12T21:02:29.780 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:02:29 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' 2024-04-12T21:02:29.780 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:02:29 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' 2024-04-12T21:02:29.781 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:02:29 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-04-12T21:02:29.781 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:02:29 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-12T21:02:30.600 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:02:30 smithi018 ceph-mon[112590]: from='client.34180 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:02:30.601 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:02:30 smithi018 ceph-mon[112590]: Reconfiguring osd.1 (monmap changed)... 2024-04-12T21:02:30.601 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:02:30 smithi018 ceph-mon[112590]: Reconfiguring daemon osd.1 on smithi018 2024-04-12T21:02:30.601 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:02:30 smithi018 ceph-mon[112590]: from='client.? 172.21.15.18:0/3979558014' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-12T21:02:30.780 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:02:30 smithi186 ceph-mon[86379]: from='client.34180 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:02:30.780 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:02:30 smithi186 ceph-mon[86379]: Reconfiguring osd.1 (monmap changed)... 2024-04-12T21:02:30.780 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:02:30 smithi186 ceph-mon[86379]: Reconfiguring daemon osd.1 on smithi018 2024-04-12T21:02:30.780 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:02:30 smithi186 ceph-mon[86379]: from='client.? 172.21.15.18:0/3979558014' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-12T21:02:31.600 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:02:31 smithi018 ceph-mon[112590]: pgmap v28: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-12T21:02:31.779 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:02:31 smithi186 ceph-mon[86379]: pgmap v28: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-12T21:02:32.560 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:02:32 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' 2024-04-12T21:02:32.560 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:02:32 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' 2024-04-12T21:02:32.560 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:02:32 smithi018 ceph-mon[112590]: Reconfiguring osd.2 (monmap changed)... 2024-04-12T21:02:32.560 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:02:32 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-04-12T21:02:32.560 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:02:32 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-12T21:02:32.560 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:02:32 smithi018 ceph-mon[112590]: Reconfiguring daemon osd.2 on smithi018 2024-04-12T21:02:32.780 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:02:32 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' 2024-04-12T21:02:32.780 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:02:32 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' 2024-04-12T21:02:32.780 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:02:32 smithi186 ceph-mon[86379]: Reconfiguring osd.2 (monmap changed)... 2024-04-12T21:02:32.780 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:02:32 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-04-12T21:02:32.780 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:02:32 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-12T21:02:32.780 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:02:32 smithi186 ceph-mon[86379]: Reconfiguring daemon osd.2 on smithi018 2024-04-12T21:02:33.779 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:02:33 smithi186 ceph-mon[86379]: pgmap v29: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-12T21:02:33.850 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:02:33 smithi018 ceph-mon[112590]: pgmap v29: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-12T21:02:35.259 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:02:34 smithi018 ceph-mon[112590]: pgmap v30: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-12T21:02:35.260 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:02:34 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' 2024-04-12T21:02:35.260 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:02:34 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' 2024-04-12T21:02:35.260 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:02:34 smithi018 ceph-mon[112590]: Reconfiguring mds.cephfs.smithi018.acqbsc (monmap changed)... 2024-04-12T21:02:35.260 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:02:34 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi018.acqbsc", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-12T21:02:35.260 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:02:34 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-12T21:02:35.260 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:02:34 smithi018 ceph-mon[112590]: Reconfiguring daemon mds.cephfs.smithi018.acqbsc on smithi018 2024-04-12T21:02:35.280 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:02:34 smithi186 ceph-mon[86379]: pgmap v30: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-12T21:02:35.280 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:02:34 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' 2024-04-12T21:02:35.280 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:02:34 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' 2024-04-12T21:02:35.280 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:02:34 smithi186 ceph-mon[86379]: Reconfiguring mds.cephfs.smithi018.acqbsc (monmap changed)... 2024-04-12T21:02:35.280 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:02:34 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi018.acqbsc", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-12T21:02:35.281 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:02:34 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-12T21:02:35.281 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:02:34 smithi186 ceph-mon[86379]: Reconfiguring daemon mds.cephfs.smithi018.acqbsc on smithi018 2024-04-12T21:02:37.577 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:02:37 smithi018 ceph-mon[112590]: pgmap v31: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-12T21:02:37.577 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:02:37 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' 2024-04-12T21:02:37.577 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:02:37 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' 2024-04-12T21:02:37.577 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:02:37 smithi018 ceph-mon[112590]: Reconfiguring mds.cephfs.smithi018.szjluo (monmap changed)... 2024-04-12T21:02:37.577 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:02:37 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi018.szjluo", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-12T21:02:37.578 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:02:37 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-12T21:02:37.578 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:02:37 smithi018 ceph-mon[112590]: Reconfiguring daemon mds.cephfs.smithi018.szjluo on smithi018 2024-04-12T21:02:37.780 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:02:37 smithi186 ceph-mon[86379]: pgmap v31: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-12T21:02:37.780 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:02:37 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' 2024-04-12T21:02:37.780 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:02:37 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' 2024-04-12T21:02:37.780 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:02:37 smithi186 ceph-mon[86379]: Reconfiguring mds.cephfs.smithi018.szjluo (monmap changed)... 2024-04-12T21:02:37.780 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:02:37 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi018.szjluo", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-12T21:02:37.780 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:02:37 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-12T21:02:37.780 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:02:37 smithi186 ceph-mon[86379]: Reconfiguring daemon mds.cephfs.smithi018.szjluo on smithi018 2024-04-12T21:02:39.583 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:02:39 smithi186 ceph-mon[86379]: pgmap v32: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-12T21:02:39.583 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:02:39 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' 2024-04-12T21:02:39.583 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:02:39 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' 2024-04-12T21:02:39.583 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:02:39 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi186", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-04-12T21:02:39.583 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:02:39 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-12T21:02:39.601 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:02:39 smithi018 ceph-mon[112590]: pgmap v32: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-12T21:02:39.601 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:02:39 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' 2024-04-12T21:02:39.601 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:02:39 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' 2024-04-12T21:02:39.601 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:02:39 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi186", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-04-12T21:02:39.601 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:02:39 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-12T21:02:40.493 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:02:40 smithi186 ceph-mon[86379]: Reconfiguring crash.smithi186 (monmap changed)... 2024-04-12T21:02:40.493 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:02:40 smithi186 ceph-mon[86379]: Reconfiguring daemon crash.smithi186 on smithi186 2024-04-12T21:02:40.494 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:02:40 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T21:02:40.506 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:02:40 smithi018 ceph-mon[112590]: Reconfiguring crash.smithi186 (monmap changed)... 2024-04-12T21:02:40.506 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:02:40 smithi018 ceph-mon[112590]: Reconfiguring daemon crash.smithi186 on smithi186 2024-04-12T21:02:40.506 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:02:40 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T21:02:41.601 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:02:41 smithi018 ceph-mon[112590]: pgmap v33: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-12T21:02:41.601 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:02:41 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' 2024-04-12T21:02:41.601 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:02:41 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' 2024-04-12T21:02:41.601 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:02:41 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi186.kqcfxk", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-12T21:02:41.601 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:02:41 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-12T21:02:41.601 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:02:41 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-12T21:02:41.780 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:02:41 smithi186 ceph-mon[86379]: pgmap v33: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-12T21:02:41.780 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:02:41 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' 2024-04-12T21:02:41.780 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:02:41 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' 2024-04-12T21:02:41.780 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:02:41 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi186.kqcfxk", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-12T21:02:41.780 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:02:41 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-12T21:02:41.780 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:02:41 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-12T21:02:42.530 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:02:42 smithi186 ceph-mon[86379]: Reconfiguring mgr.smithi186.kqcfxk (monmap changed)... 2024-04-12T21:02:42.530 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:02:42 smithi186 ceph-mon[86379]: Reconfiguring daemon mgr.smithi186.kqcfxk on smithi186 2024-04-12T21:02:42.601 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:02:42 smithi018 ceph-mon[112590]: Reconfiguring mgr.smithi186.kqcfxk (monmap changed)... 2024-04-12T21:02:42.601 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:02:42 smithi018 ceph-mon[112590]: Reconfiguring daemon mgr.smithi186.kqcfxk on smithi186 2024-04-12T21:02:43.601 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:02:43 smithi018 ceph-mon[112590]: pgmap v34: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-12T21:02:43.601 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:02:43 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' 2024-04-12T21:02:43.601 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:02:43 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' 2024-04-12T21:02:43.602 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:02:43 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-12T21:02:43.602 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:02:43 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-12T21:02:43.602 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:02:43 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-12T21:02:43.780 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:02:43 smithi186 ceph-mon[86379]: pgmap v34: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-12T21:02:43.780 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:02:43 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' 2024-04-12T21:02:43.780 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:02:43 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' 2024-04-12T21:02:43.780 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:02:43 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-12T21:02:43.780 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:02:43 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-12T21:02:43.780 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:02:43 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-12T21:02:44.779 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:02:44 smithi186 ceph-mon[86379]: Reconfiguring mon.smithi186 (monmap changed)... 2024-04-12T21:02:44.780 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:02:44 smithi186 ceph-mon[86379]: Reconfiguring daemon mon.smithi186 on smithi186 2024-04-12T21:02:44.851 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:02:44 smithi018 ceph-mon[112590]: Reconfiguring mon.smithi186 (monmap changed)... 2024-04-12T21:02:44.851 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:02:44 smithi018 ceph-mon[112590]: Reconfiguring daemon mon.smithi186 on smithi186 2024-04-12T21:02:45.529 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:02:45 smithi186 ceph-mon[86379]: pgmap v35: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-12T21:02:45.529 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:02:45 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' 2024-04-12T21:02:45.850 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:02:45 smithi018 ceph-mon[112590]: pgmap v35: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-12T21:02:45.851 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:02:45 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' 2024-04-12T21:02:46.606 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:02:46 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' 2024-04-12T21:02:46.606 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:02:46 smithi186 ceph-mon[86379]: Reconfiguring osd.3 (monmap changed)... 2024-04-12T21:02:46.606 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:02:46 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-04-12T21:02:46.606 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:02:46 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-12T21:02:46.607 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:02:46 smithi186 ceph-mon[86379]: Reconfiguring daemon osd.3 on smithi186 2024-04-12T21:02:46.851 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:02:46 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' 2024-04-12T21:02:46.851 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:02:46 smithi018 ceph-mon[112590]: Reconfiguring osd.3 (monmap changed)... 2024-04-12T21:02:46.851 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:02:46 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-04-12T21:02:46.851 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:02:46 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-12T21:02:46.851 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:02:46 smithi018 ceph-mon[112590]: Reconfiguring daemon osd.3 on smithi186 2024-04-12T21:02:47.475 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:02:47 smithi186 ceph-mon[86379]: pgmap v36: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-12T21:02:47.850 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:02:47 smithi018 ceph-mon[112590]: pgmap v36: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-12T21:02:49.030 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:02:48 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' 2024-04-12T21:02:49.030 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:02:48 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' 2024-04-12T21:02:49.030 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:02:48 smithi186 ceph-mon[86379]: Reconfiguring osd.4 (monmap changed)... 2024-04-12T21:02:49.030 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:02:48 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-04-12T21:02:49.030 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:02:48 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-12T21:02:49.030 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:02:48 smithi186 ceph-mon[86379]: pgmap v37: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-12T21:02:49.030 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:02:48 smithi186 ceph-mon[86379]: Reconfiguring daemon osd.4 on smithi186 2024-04-12T21:02:49.101 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:02:48 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' 2024-04-12T21:02:49.101 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:02:48 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' 2024-04-12T21:02:49.101 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:02:48 smithi018 ceph-mon[112590]: Reconfiguring osd.4 (monmap changed)... 2024-04-12T21:02:49.101 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:02:48 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-04-12T21:02:49.101 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:02:48 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-12T21:02:49.101 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:02:48 smithi018 ceph-mon[112590]: pgmap v37: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-12T21:02:49.101 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:02:48 smithi018 ceph-mon[112590]: Reconfiguring daemon osd.4 on smithi186 2024-04-12T21:02:51.406 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:02:51 smithi186 ceph-mon[86379]: pgmap v38: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-12T21:02:51.406 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:02:51 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' 2024-04-12T21:02:51.406 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:02:51 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' 2024-04-12T21:02:51.406 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:02:51 smithi186 ceph-mon[86379]: Reconfiguring osd.5 (monmap changed)... 2024-04-12T21:02:51.406 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:02:51 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-04-12T21:02:51.406 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:02:51 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-12T21:02:51.406 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:02:51 smithi186 ceph-mon[86379]: Reconfiguring daemon osd.5 on smithi186 2024-04-12T21:02:51.601 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:02:51 smithi018 ceph-mon[112590]: pgmap v38: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-12T21:02:51.601 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:02:51 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' 2024-04-12T21:02:51.601 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:02:51 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' 2024-04-12T21:02:51.601 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:02:51 smithi018 ceph-mon[112590]: Reconfiguring osd.5 (monmap changed)... 2024-04-12T21:02:51.601 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:02:51 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-04-12T21:02:51.601 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:02:51 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-12T21:02:51.601 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:02:51 smithi018 ceph-mon[112590]: Reconfiguring daemon osd.5 on smithi186 2024-04-12T21:02:53.610 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:02:53 smithi186 ceph-mon[86379]: pgmap v39: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-12T21:02:53.610 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:02:53 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' 2024-04-12T21:02:53.610 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:02:53 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' 2024-04-12T21:02:53.610 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:02:53 smithi186 ceph-mon[86379]: Reconfiguring mds.cephfs.smithi186.jdllyn (monmap changed)... 2024-04-12T21:02:53.610 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:02:53 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi186.jdllyn", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-12T21:02:53.610 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:02:53 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-12T21:02:53.610 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:02:53 smithi186 ceph-mon[86379]: Reconfiguring daemon mds.cephfs.smithi186.jdllyn on smithi186 2024-04-12T21:02:53.851 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:02:53 smithi018 ceph-mon[112590]: pgmap v39: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-12T21:02:53.851 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:02:53 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' 2024-04-12T21:02:53.851 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:02:53 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' 2024-04-12T21:02:53.851 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:02:53 smithi018 ceph-mon[112590]: Reconfiguring mds.cephfs.smithi186.jdllyn (monmap changed)... 2024-04-12T21:02:53.851 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:02:53 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi186.jdllyn", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-12T21:02:53.851 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:02:53 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-12T21:02:53.852 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:02:53 smithi018 ceph-mon[112590]: Reconfiguring daemon mds.cephfs.smithi186.jdllyn on smithi186 2024-04-12T21:02:55.780 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:02:55 smithi186 ceph-mon[86379]: pgmap v40: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-12T21:02:55.780 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:02:55 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' 2024-04-12T21:02:55.780 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:02:55 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' 2024-04-12T21:02:55.780 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:02:55 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi186.wjnafs", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-12T21:02:55.780 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:02:55 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-12T21:02:55.780 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:02:55 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T21:02:55.851 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:02:55 smithi018 ceph-mon[112590]: pgmap v40: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-12T21:02:55.851 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:02:55 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' 2024-04-12T21:02:55.851 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:02:55 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' 2024-04-12T21:02:55.851 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:02:55 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi186.wjnafs", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-12T21:02:55.851 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:02:55 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-12T21:02:55.851 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:02:55 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T21:02:56.692 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:02:56 smithi186 ceph-mon[86379]: Reconfiguring mds.cephfs.smithi186.wjnafs (monmap changed)... 2024-04-12T21:02:56.692 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:02:56 smithi186 ceph-mon[86379]: Reconfiguring daemon mds.cephfs.smithi186.wjnafs on smithi186 2024-04-12T21:02:56.851 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:02:56 smithi018 ceph-mon[112590]: Reconfiguring mds.cephfs.smithi186.wjnafs (monmap changed)... 2024-04-12T21:02:56.851 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:02:56 smithi018 ceph-mon[112590]: Reconfiguring daemon mds.cephfs.smithi186.wjnafs on smithi186 2024-04-12T21:02:57.780 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:02:57 smithi186 ceph-mon[86379]: pgmap v41: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-12T21:02:57.780 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:02:57 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' 2024-04-12T21:02:57.780 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:02:57 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' 2024-04-12T21:02:57.780 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:02:57 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-12T21:02:57.781 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:02:57 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T21:02:57.781 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:02:57 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T21:02:57.781 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:02:57 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' 2024-04-12T21:02:57.781 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:02:57 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon.smithi018"}]: dispatch 2024-04-12T21:02:57.781 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:02:57 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mon.smithi018"}]': finished 2024-04-12T21:02:57.781 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:02:57 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon.smithi186"}]: dispatch 2024-04-12T21:02:57.781 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:02:57 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mon.smithi186"}]': finished 2024-04-12T21:02:57.850 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:02:57 smithi018 ceph-mon[112590]: pgmap v41: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-12T21:02:57.851 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:02:57 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' 2024-04-12T21:02:57.851 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:02:57 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' 2024-04-12T21:02:57.851 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:02:57 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-12T21:02:57.851 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:02:57 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T21:02:57.851 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:02:57 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T21:02:57.851 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:02:57 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' 2024-04-12T21:02:57.851 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:02:57 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon.smithi018"}]: dispatch 2024-04-12T21:02:57.851 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:02:57 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mon.smithi018"}]': finished 2024-04-12T21:02:57.851 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:02:57 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon.smithi186"}]: dispatch 2024-04-12T21:02:57.851 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:02:57 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mon.smithi186"}]': finished 2024-04-12T21:02:58.779 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:02:58 smithi186 ceph-mon[86379]: Upgrade: Setting container_image for all mon 2024-04-12T21:02:58.851 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:02:58 smithi018 ceph-mon[112590]: Upgrade: Setting container_image for all mon 2024-04-12T21:02:59.518 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:02:59.522+0000 7f7dbea00700 1 -- 172.21.15.18:0/3869158078 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7db8070340 msgr2=0x7f7db8070710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:02:59.519 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:02:59.522+0000 7f7dbea00700 1 --2- 172.21.15.18:0/3869158078 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7db8070340 0x7f7db8070710 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f7da8004940 tx=0x7f7da8009e30 comp rx=0 tx=0).stop 2024-04-12T21:02:59.519 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:02:59.522+0000 7f7dbea00700 1 -- 172.21.15.18:0/3869158078 shutdown_connections 2024-04-12T21:02:59.519 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:02:59.522+0000 7f7dbea00700 1 --2- 172.21.15.18:0/3869158078 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7db8070c50 0x7f7db8109130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:02:59.519 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:02:59.522+0000 7f7dbea00700 1 --2- 172.21.15.18:0/3869158078 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7db8070340 0x7f7db8070710 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:02:59.519 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:02:59.522+0000 7f7dbea00700 1 -- 172.21.15.18:0/3869158078 >> 172.21.15.18:0/3869158078 conn(0x7f7db80f89b0 msgr2=0x7f7db80fadc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:02:59.519 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:02:59.522+0000 7f7dbea00700 1 -- 172.21.15.18:0/3869158078 shutdown_connections 2024-04-12T21:02:59.519 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:02:59.522+0000 7f7dbea00700 1 -- 172.21.15.18:0/3869158078 wait complete. 2024-04-12T21:02:59.519 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:02:59.523+0000 7f7dbea00700 1 Processor -- start 2024-04-12T21:02:59.520 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:02:59.523+0000 7f7dbea00700 1 -- start start 2024-04-12T21:02:59.520 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:02:59.524+0000 7f7dbea00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7db8070340 0x7f7db8115bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:02:59.520 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:02:59.524+0000 7f7dbea00700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7db8070c50 0x7f7db8116110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:02:59.520 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:02:59.524+0000 7f7dbea00700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7db8116760 con 0x7f7db8070340 2024-04-12T21:02:59.520 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:02:59.524+0000 7f7dbea00700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7db810fd40 con 0x7f7db8070c50 2024-04-12T21:02:59.521 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:02:59.524+0000 7f7db7e00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7db8070340 0x7f7db8115bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:02:59.521 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:02:59.524+0000 7f7db7e00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7db8070340 0x7f7db8115bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:47090/0 (socket says 172.21.15.18:47090) 2024-04-12T21:02:59.521 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:02:59.524+0000 7f7db7e00700 1 -- 172.21.15.18:0/3949323276 learned_addr learned my addr 172.21.15.18:0/3949323276 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:02:59.522 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:02:59.525+0000 7f7db7400700 1 --2- 172.21.15.18:0/3949323276 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7db8070c50 0x7f7db8116110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:02:59.522 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:02:59.525+0000 7f7db7e00700 1 -- 172.21.15.18:0/3949323276 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7db8070c50 msgr2=0x7f7db8116110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:02:59.522 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:02:59.525+0000 7f7db7e00700 1 --2- 172.21.15.18:0/3949323276 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7db8070c50 0x7f7db8116110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:02:59.522 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:02:59.525+0000 7f7db7e00700 1 -- 172.21.15.18:0/3949323276 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7da8005040 con 0x7f7db8070340 2024-04-12T21:02:59.522 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:02:59.525+0000 7f7db7e00700 1 --2- 172.21.15.18:0/3949323276 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7db8070340 0x7f7db8115bd0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f7da8009180 tx=0x7f7da8009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:02:59.522 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:02:59.525+0000 7f7dafe00700 1 -- 172.21.15.18:0/3949323276 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7da800e040 con 0x7f7db8070340 2024-04-12T21:02:59.522 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:02:59.525+0000 7f7dafe00700 1 -- 172.21.15.18:0/3949323276 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f7da800bbc0 con 0x7f7db8070340 2024-04-12T21:02:59.522 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:02:59.525+0000 7f7dafe00700 1 -- 172.21.15.18:0/3949323276 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7da801c630 con 0x7f7db8070340 2024-04-12T21:02:59.522 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:02:59.525+0000 7f7dbea00700 1 -- 172.21.15.18:0/3949323276 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7db810ffc0 con 0x7f7db8070340 2024-04-12T21:02:59.522 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:02:59.526+0000 7f7dbea00700 1 -- 172.21.15.18:0/3949323276 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7db8110500 con 0x7f7db8070340 2024-04-12T21:02:59.523 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:02:59.527+0000 7f7dbea00700 1 -- 172.21.15.18:0/3949323276 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7db8071a80 con 0x7f7db8070340 2024-04-12T21:02:59.524 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:02:59.528+0000 7f7dafe00700 1 -- 172.21.15.18:0/3949323276 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 38) v1 ==== 92847+0+0 (secure 0 0 0) 0x7f7da800bd30 con 0x7f7db8070340 2024-04-12T21:02:59.525 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:02:59.529+0000 7f7dafe00700 1 --2- 172.21.15.18:0/3949323276 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f7d9c06ed30 0x7f7d9c0711e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:02:59.525 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:02:59.529+0000 7f7dafe00700 1 -- 172.21.15.18:0/3949323276 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6324+0+0 (secure 0 0 0) 0x7f7da808fd80 con 0x7f7db8070340 2024-04-12T21:02:59.526 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:02:59.529+0000 7f7db7400700 1 --2- 172.21.15.18:0/3949323276 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f7d9c06ed30 0x7f7d9c0711e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:02:59.526 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:02:59.530+0000 7f7db7400700 1 --2- 172.21.15.18:0/3949323276 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f7d9c06ed30 0x7f7d9c0711e0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f7da4007fd0 tx=0x7f7da4009040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:02:59.531 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:02:59.535+0000 7f7dafe00700 1 -- 172.21.15.18:0/3949323276 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f7da805c600 con 0x7f7db8070340 2024-04-12T21:02:59.779 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:02:59 smithi186 ceph-mon[86379]: pgmap v42: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-12T21:02:59.780 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:02:59 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' 2024-04-12T21:02:59.780 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:02:59 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi018", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-04-12T21:02:59.780 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:02:59 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-12T21:02:59.792 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:02:59.796+0000 7f7dbea00700 1 -- 172.21.15.18:0/3949323276 --> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f7db8111220 con 0x7f7d9c06ed30 2024-04-12T21:02:59.792 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:02:59 smithi018 ceph-mon[112590]: pgmap v42: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-12T21:02:59.793 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:02:59 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' 2024-04-12T21:02:59.793 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:02:59 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi018", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-04-12T21:02:59.793 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:02:59 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-12T21:02:59.795 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:02:59.799+0000 7f7dafe00700 1 -- 172.21.15.18:0/3949323276 <== mgr.34102 v2:172.21.15.18:6800/3812891919 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+385 (secure 0 0 0) 0x7f7db8111220 con 0x7f7d9c06ed30 2024-04-12T21:02:59.799 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:02:59.802+0000 7f7dbea00700 1 -- 172.21.15.18:0/3949323276 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f7d9c06ed30 msgr2=0x7f7d9c0711e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:02:59.799 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:02:59.802+0000 7f7dbea00700 1 --2- 172.21.15.18:0/3949323276 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f7d9c06ed30 0x7f7d9c0711e0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f7da4007fd0 tx=0x7f7da4009040 comp rx=0 tx=0).stop 2024-04-12T21:02:59.799 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:02:59.802+0000 7f7dbea00700 1 -- 172.21.15.18:0/3949323276 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7db8070340 msgr2=0x7f7db8115bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:02:59.799 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:02:59.802+0000 7f7dbea00700 1 --2- 172.21.15.18:0/3949323276 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7db8070340 0x7f7db8115bd0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f7da8009180 tx=0x7f7da8009e30 comp rx=0 tx=0).stop 2024-04-12T21:02:59.800 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:02:59.802+0000 7f7dbea00700 1 -- 172.21.15.18:0/3949323276 shutdown_connections 2024-04-12T21:02:59.800 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:02:59.802+0000 7f7dbea00700 1 --2- 172.21.15.18:0/3949323276 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f7d9c06ed30 0x7f7d9c0711e0 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:02:59.800 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:02:59.802+0000 7f7dbea00700 1 --2- 172.21.15.18:0/3949323276 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7db8070340 0x7f7db8115bd0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:02:59.800 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:02:59.802+0000 7f7dbea00700 1 --2- 172.21.15.18:0/3949323276 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7db8070c50 0x7f7db8116110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:02:59.800 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:02:59.802+0000 7f7dbea00700 1 -- 172.21.15.18:0/3949323276 >> 172.21.15.18:0/3949323276 conn(0x7f7db80f89b0 msgr2=0x7f7db80f9e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:02:59.800 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:02:59.803+0000 7f7dbea00700 1 -- 172.21.15.18:0/3949323276 shutdown_connections 2024-04-12T21:02:59.800 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:02:59.803+0000 7f7dbea00700 1 -- 172.21.15.18:0/3949323276 wait complete. 2024-04-12T21:02:59.817 INFO:teuthology.orchestra.run.smithi018.stdout:true 2024-04-12T21:02:59.959 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:02:59.962+0000 7f549d200700 1 -- 172.21.15.18:0/773056380 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f54981037b0 msgr2=0x7f54980fce20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:02:59.959 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:02:59.962+0000 7f549d200700 1 --2- 172.21.15.18:0/773056380 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f54981037b0 0x7f54980fce20 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f5484004940 tx=0x7f5484009e30 comp rx=0 tx=0).stop 2024-04-12T21:02:59.959 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:02:59.962+0000 7f549d200700 1 -- 172.21.15.18:0/773056380 shutdown_connections 2024-04-12T21:02:59.959 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:02:59.962+0000 7f549d200700 1 --2- 172.21.15.18:0/773056380 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f54981037b0 0x7f54980fce20 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:02:59.959 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:02:59.962+0000 7f549d200700 1 --2- 172.21.15.18:0/773056380 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5498102e10 0x7f54981031e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:02:59.959 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:02:59.962+0000 7f549d200700 1 -- 172.21.15.18:0/773056380 >> 172.21.15.18:0/773056380 conn(0x7f54980f89a0 msgr2=0x7f54980fadb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:02:59.959 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:02:59.963+0000 7f549d200700 1 -- 172.21.15.18:0/773056380 shutdown_connections 2024-04-12T21:02:59.959 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:02:59.963+0000 7f549d200700 1 -- 172.21.15.18:0/773056380 wait complete. 2024-04-12T21:02:59.960 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:02:59.963+0000 7f549d200700 1 Processor -- start 2024-04-12T21:02:59.960 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:02:59.964+0000 7f549d200700 1 -- start start 2024-04-12T21:02:59.960 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:02:59.964+0000 7f549d200700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5498102e10 0x7f5498115d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:02:59.960 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:02:59.964+0000 7f549d200700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f54981037b0 0x7f5498116290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:02:59.961 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:02:59.964+0000 7f549d200700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54981168e0 con 0x7f54981037b0 2024-04-12T21:02:59.961 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:02:59.964+0000 7f549d200700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f549810fcb0 con 0x7f5498102e10 2024-04-12T21:02:59.961 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:02:59.964+0000 7f5495400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f54981037b0 0x7f5498116290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:02:59.961 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:02:59.964+0000 7f5495400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f54981037b0 0x7f5498116290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:47094/0 (socket says 172.21.15.18:47094) 2024-04-12T21:02:59.961 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:02:59.964+0000 7f5495400700 1 -- 172.21.15.18:0/2758181069 learned_addr learned my addr 172.21.15.18:0/2758181069 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:02:59.962 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:02:59.965+0000 7f5495e00700 1 --2- 172.21.15.18:0/2758181069 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5498102e10 0x7f5498115d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:02:59.962 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:02:59.965+0000 7f5495400700 1 -- 172.21.15.18:0/2758181069 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5498102e10 msgr2=0x7f5498115d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:02:59.962 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:02:59.965+0000 7f5495400700 1 --2- 172.21.15.18:0/2758181069 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5498102e10 0x7f5498115d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:02:59.962 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:02:59.965+0000 7f5495400700 1 -- 172.21.15.18:0/2758181069 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5484005040 con 0x7f54981037b0 2024-04-12T21:02:59.962 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:02:59.965+0000 7f5495400700 1 --2- 172.21.15.18:0/2758181069 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f54981037b0 0x7f5498116290 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f5484009100 tx=0x7f5484009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:02:59.962 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:02:59.966+0000 7f548e000700 1 -- 172.21.15.18:0/2758181069 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f548400e040 con 0x7f54981037b0 2024-04-12T21:02:59.962 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:02:59.966+0000 7f548e000700 1 -- 172.21.15.18:0/2758181069 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f5484004290 con 0x7f54981037b0 2024-04-12T21:02:59.962 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:02:59.966+0000 7f548e000700 1 -- 172.21.15.18:0/2758181069 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f548401c3f0 con 0x7f54981037b0 2024-04-12T21:02:59.962 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:02:59.966+0000 7f549d200700 1 -- 172.21.15.18:0/2758181069 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f549810ff30 con 0x7f54981037b0 2024-04-12T21:02:59.963 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:02:59.966+0000 7f549d200700 1 -- 172.21.15.18:0/2758181069 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f54981104f0 con 0x7f54981037b0 2024-04-12T21:02:59.963 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:02:59.967+0000 7f549d200700 1 -- 172.21.15.18:0/2758181069 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5498102130 con 0x7f54981037b0 2024-04-12T21:02:59.964 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:02:59.968+0000 7f548e000700 1 -- 172.21.15.18:0/2758181069 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 38) v1 ==== 92847+0+0 (secure 0 0 0) 0x7f5484013790 con 0x7f54981037b0 2024-04-12T21:02:59.966 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:02:59.969+0000 7f548e000700 1 --2- 172.21.15.18:0/2758181069 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f547c06ed20 0x7f547c0711d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:02:59.966 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:02:59.969+0000 7f548e000700 1 -- 172.21.15.18:0/2758181069 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6324+0+0 (secure 0 0 0) 0x7f548408fd40 con 0x7f54981037b0 2024-04-12T21:02:59.966 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:02:59.970+0000 7f5495e00700 1 --2- 172.21.15.18:0/2758181069 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f547c06ed20 0x7f547c0711d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:02:59.967 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:02:59.971+0000 7f5495e00700 1 --2- 172.21.15.18:0/2758181069 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f547c06ed20 0x7f547c0711d0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f5488004990 tx=0x7f5488009ee0 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:02:59.971 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:02:59.975+0000 7f548e000700 1 -- 172.21.15.18:0/2758181069 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f548405c520 con 0x7f54981037b0 2024-04-12T21:03:00.255 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:03:00.258+0000 7f549d200700 1 -- 172.21.15.18:0/2758181069 --> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f5498111410 con 0x7f547c06ed20 2024-04-12T21:03:00.256 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:03:00.260+0000 7f548e000700 1 -- 172.21.15.18:0/2758181069 <== mgr.34102 v2:172.21.15.18:6800/3812891919 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+385 (secure 0 0 0) 0x7f5498111410 con 0x7f547c06ed20 2024-04-12T21:03:00.259 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:03:00.262+0000 7f549d200700 1 -- 172.21.15.18:0/2758181069 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f547c06ed20 msgr2=0x7f547c0711d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:03:00.259 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:03:00.262+0000 7f549d200700 1 --2- 172.21.15.18:0/2758181069 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f547c06ed20 0x7f547c0711d0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f5488004990 tx=0x7f5488009ee0 comp rx=0 tx=0).stop 2024-04-12T21:03:00.259 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:03:00.263+0000 7f549d200700 1 -- 172.21.15.18:0/2758181069 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f54981037b0 msgr2=0x7f5498116290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:03:00.259 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:03:00.263+0000 7f549d200700 1 --2- 172.21.15.18:0/2758181069 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f54981037b0 0x7f5498116290 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f5484009100 tx=0x7f5484009e30 comp rx=0 tx=0).stop 2024-04-12T21:03:00.259 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:03:00.263+0000 7f549d200700 1 -- 172.21.15.18:0/2758181069 shutdown_connections 2024-04-12T21:03:00.260 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:03:00.263+0000 7f549d200700 1 --2- 172.21.15.18:0/2758181069 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f547c06ed20 0x7f547c0711d0 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:03:00.260 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:03:00.263+0000 7f549d200700 1 --2- 172.21.15.18:0/2758181069 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5498102e10 0x7f5498115d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:03:00.260 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:03:00.263+0000 7f549d200700 1 --2- 172.21.15.18:0/2758181069 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f54981037b0 0x7f5498116290 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:03:00.260 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:03:00.263+0000 7f549d200700 1 -- 172.21.15.18:0/2758181069 >> 172.21.15.18:0/2758181069 conn(0x7f54980f89a0 msgr2=0x7f54980fa6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:03:00.260 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:03:00.263+0000 7f549d200700 1 -- 172.21.15.18:0/2758181069 shutdown_connections 2024-04-12T21:03:00.260 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:03:00.263+0000 7f549d200700 1 -- 172.21.15.18:0/2758181069 wait complete. 2024-04-12T21:03:00.409 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:03:00.412+0000 7fab47600700 1 -- 172.21.15.18:0/4242160916 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fab400fe7c0 msgr2=0x7fab40106e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:03:00.409 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:03:00.412+0000 7fab47600700 1 --2- 172.21.15.18:0/4242160916 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fab400fe7c0 0x7fab40106e00 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fab2c004940 tx=0x7fab2c009e30 comp rx=0 tx=0).stop 2024-04-12T21:03:00.409 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:03:00.412+0000 7fab47600700 1 -- 172.21.15.18:0/4242160916 shutdown_connections 2024-04-12T21:03:00.409 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:03:00.412+0000 7fab47600700 1 --2- 172.21.15.18:0/4242160916 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fab400fe7c0 0x7fab40106e00 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:03:00.409 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:03:00.412+0000 7fab47600700 1 --2- 172.21.15.18:0/4242160916 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fab400fde20 0x7fab400fe1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:03:00.409 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:03:00.412+0000 7fab47600700 1 -- 172.21.15.18:0/4242160916 >> 172.21.15.18:0/4242160916 conn(0x7fab400f7870 msgr2=0x7fab400f9c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:03:00.410 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:03:00.412+0000 7fab47600700 1 -- 172.21.15.18:0/4242160916 shutdown_connections 2024-04-12T21:03:00.410 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:03:00.412+0000 7fab47600700 1 -- 172.21.15.18:0/4242160916 wait complete. 2024-04-12T21:03:00.412 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:03:00.413+0000 7fab47600700 1 Processor -- start 2024-04-12T21:03:00.412 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:03:00.414+0000 7fab47600700 1 -- start start 2024-04-12T21:03:00.412 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:03:00.414+0000 7fab47600700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fab400fde20 0x7fab400fbc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:03:00.412 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:03:00.414+0000 7fab47600700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fab400fe7c0 0x7fab400fc190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:03:00.412 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:03:00.414+0000 7fab47600700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab400fdba0 con 0x7fab400fe7c0 2024-04-12T21:03:00.412 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:03:00.414+0000 7fab47600700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab400fc6d0 con 0x7fab400fde20 2024-04-12T21:03:00.412 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:03:00.414+0000 7fab3fe00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fab400fe7c0 0x7fab400fc190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:03:00.412 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:03:00.414+0000 7fab3fe00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fab400fe7c0 0x7fab400fc190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:47108/0 (socket says 172.21.15.18:47108) 2024-04-12T21:03:00.413 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:03:00.414+0000 7fab3fe00700 1 -- 172.21.15.18:0/1010479790 learned_addr learned my addr 172.21.15.18:0/1010479790 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:03:00.413 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:03:00.415+0000 7fab44c00700 1 --2- 172.21.15.18:0/1010479790 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fab400fde20 0x7fab400fbc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:03:00.413 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:03:00.415+0000 7fab3fe00700 1 -- 172.21.15.18:0/1010479790 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fab400fde20 msgr2=0x7fab400fbc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:03:00.413 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:03:00.415+0000 7fab3fe00700 1 --2- 172.21.15.18:0/1010479790 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fab400fde20 0x7fab400fbc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:03:00.413 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:03:00.415+0000 7fab3fe00700 1 -- 172.21.15.18:0/1010479790 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab2c005040 con 0x7fab400fe7c0 2024-04-12T21:03:00.413 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:03:00.415+0000 7fab3fe00700 1 --2- 172.21.15.18:0/1010479790 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fab400fe7c0 0x7fab400fc190 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fab2c009380 tx=0x7fab2c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:03:00.413 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:03:00.415+0000 7fab3cc00700 1 -- 172.21.15.18:0/1010479790 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fab2c00e040 con 0x7fab400fe7c0 2024-04-12T21:03:00.413 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:03:00.415+0000 7fab3cc00700 1 -- 172.21.15.18:0/1010479790 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fab2c009af0 con 0x7fab400fe7c0 2024-04-12T21:03:00.413 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:03:00.415+0000 7fab47600700 1 -- 172.21.15.18:0/1010479790 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fab400fc950 con 0x7fab400fe7c0 2024-04-12T21:03:00.413 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:03:00.415+0000 7fab3cc00700 1 -- 172.21.15.18:0/1010479790 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fab2c01c630 con 0x7fab400fe7c0 2024-04-12T21:03:00.413 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:03:00.416+0000 7fab47600700 1 -- 172.21.15.18:0/1010479790 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fab4011a060 con 0x7fab400fe7c0 2024-04-12T21:03:00.414 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:03:00.417+0000 7fab47600700 1 -- 172.21.15.18:0/1010479790 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fab40066b50 con 0x7fab400fe7c0 2024-04-12T21:03:00.414 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:03:00.418+0000 7fab3cc00700 1 -- 172.21.15.18:0/1010479790 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 38) v1 ==== 92847+0+0 (secure 0 0 0) 0x7fab2c00bbc0 con 0x7fab400fe7c0 2024-04-12T21:03:00.415 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:03:00.419+0000 7fab3cc00700 1 --2- 172.21.15.18:0/1010479790 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fab2806ed30 0x7fab280711e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:03:00.415 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:03:00.419+0000 7fab3cc00700 1 -- 172.21.15.18:0/1010479790 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6324+0+0 (secure 0 0 0) 0x7fab2c08fe10 con 0x7fab400fe7c0 2024-04-12T21:03:00.415 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:03:00.419+0000 7fab44c00700 1 --2- 172.21.15.18:0/1010479790 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fab2806ed30 0x7fab280711e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:03:00.416 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:03:00.420+0000 7fab44c00700 1 --2- 172.21.15.18:0/1010479790 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fab2806ed30 0x7fab280711e0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fab34007fd0 tx=0x7fab34009040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:03:00.419 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:03:00.423+0000 7fab3cc00700 1 -- 172.21.15.18:0/1010479790 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7fab2c058a20 con 0x7fab400fe7c0 2024-04-12T21:03:00.520 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:03:00 smithi018 ceph-mon[112590]: Upgrade: Updating crash.smithi018 (1/2) 2024-04-12T21:03:00.520 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:03:00 smithi018 ceph-mon[112590]: Deploying daemon crash.smithi018 on smithi018 2024-04-12T21:03:00.671 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:03:00.674+0000 7fab47600700 1 -- 172.21.15.18:0/1010479790 --> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fab40109f50 con 0x7fab2806ed30 2024-04-12T21:03:00.687 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:03:00.690+0000 7fab3cc00700 1 -- 172.21.15.18:0/1010479790 <== mgr.34102 v2:172.21.15.18:6800/3812891919 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7fab34004db0 con 0x7fab2806ed30 2024-04-12T21:03:00.687 INFO:teuthology.orchestra.run.smithi018.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-12T21:03:00.687 INFO:teuthology.orchestra.run.smithi018.stdout:alertmanager.smithi018 smithi018 *:9093,9094 running (3m) 73s ago 18m 14.9M - 0.25.0 c8568f914cd2 c332090068e8 2024-04-12T21:03:00.687 INFO:teuthology.orchestra.run.smithi018.stdout:crash.smithi018 smithi018 running (18m) 73s ago 18m 6660k - 17.2.7-827-gd3b5060a 21de9b7369ad 1ac7603bddfd 2024-04-12T21:03:00.687 INFO:teuthology.orchestra.run.smithi018.stdout:crash.smithi186 smithi186 running (16m) 49s ago 16m 6669k - 17.2.7-827-gd3b5060a 21de9b7369ad 14bf8a1672eb 2024-04-12T21:03:00.687 INFO:teuthology.orchestra.run.smithi018.stdout:grafana.smithi018 smithi018 *:3000 running (3m) 73s ago 17m 74.4M - 9.4.12 2bacad6d85d8 b08e3bf176dd 2024-04-12T21:03:00.688 INFO:teuthology.orchestra.run.smithi018.stdout:mds.cephfs.smithi018.acqbsc smithi018 running (10m) 73s ago 10m 112M - 17.2.7-827-gd3b5060a 21de9b7369ad 5b5fce65f18a 2024-04-12T21:03:00.688 INFO:teuthology.orchestra.run.smithi018.stdout:mds.cephfs.smithi018.szjluo smithi018 running (10m) 73s ago 10m 113M - 17.2.7-827-gd3b5060a 21de9b7369ad cf9bf0c5031f 2024-04-12T21:03:00.688 INFO:teuthology.orchestra.run.smithi018.stdout:mds.cephfs.smithi186.jdllyn smithi186 running (10m) 49s ago 10m 14.4M - 17.2.7-827-gd3b5060a 21de9b7369ad 2b2bc46bd29a 2024-04-12T21:03:00.688 INFO:teuthology.orchestra.run.smithi018.stdout:mds.cephfs.smithi186.wjnafs smithi186 running (10m) 49s ago 10m 14.2M - 17.2.7-827-gd3b5060a 21de9b7369ad 74b115d6d6fd 2024-04-12T21:03:00.688 INFO:teuthology.orchestra.run.smithi018.stdout:mgr.smithi018.xjzwga smithi018 *:8443,9283,8765 running (6m) 73s ago 19m 478M - 19.0.0-2879-g324baf8d 8717720dc5e5 56088305eab7 2024-04-12T21:03:00.688 INFO:teuthology.orchestra.run.smithi018.stdout:mgr.smithi186.kqcfxk smithi186 *:8443,9283,8765 running (5m) 49s ago 16m 450M - 19.0.0-2879-g324baf8d 8717720dc5e5 09cd766e5e0d 2024-04-12T21:03:00.688 INFO:teuthology.orchestra.run.smithi018.stdout:mon.smithi018 smithi018 running (91s) 73s ago 19m 29.8M 2048M 19.0.0-2879-g324baf8d 8717720dc5e5 df3e201dfed6 2024-04-12T21:03:00.688 INFO:teuthology.orchestra.run.smithi018.stdout:mon.smithi186 smithi186 running (54s) 49s ago 16m 16.8M 2048M 19.0.0-2879-g324baf8d 8717720dc5e5 c77498da9b2b 2024-04-12T21:03:00.688 INFO:teuthology.orchestra.run.smithi018.stdout:node-exporter.smithi018 smithi018 *:9100 running (4m) 73s ago 18m 7898k - 1.5.0 0da6a335fe13 326c3d9c6847 2024-04-12T21:03:00.688 INFO:teuthology.orchestra.run.smithi018.stdout:node-exporter.smithi186 smithi186 *:9100 running (4m) 49s ago 16m 8044k - 1.5.0 0da6a335fe13 ec2112baeecf 2024-04-12T21:03:00.688 INFO:teuthology.orchestra.run.smithi018.stdout:osd.0 smithi018 running (15m) 73s ago 15m 226M 1559M 17.2.7-827-gd3b5060a 21de9b7369ad 0d83e027bf0a 2024-04-12T21:03:00.688 INFO:teuthology.orchestra.run.smithi018.stdout:osd.1 smithi018 running (15m) 73s ago 15m 241M 1559M 17.2.7-827-gd3b5060a 21de9b7369ad 6517c9cc7f87 2024-04-12T21:03:00.688 INFO:teuthology.orchestra.run.smithi018.stdout:osd.2 smithi018 running (14m) 73s ago 14m 194M 1559M 17.2.7-827-gd3b5060a 21de9b7369ad 8a83692acaa5 2024-04-12T21:03:00.688 INFO:teuthology.orchestra.run.smithi018.stdout:osd.3 smithi186 running (13m) 49s ago 13m 263M 2581M 17.2.7-827-gd3b5060a 21de9b7369ad c2def7a90d76 2024-04-12T21:03:00.689 INFO:teuthology.orchestra.run.smithi018.stdout:osd.4 smithi186 running (13m) 49s ago 13m 233M 2581M 17.2.7-827-gd3b5060a 21de9b7369ad 757e7c71b7b9 2024-04-12T21:03:00.689 INFO:teuthology.orchestra.run.smithi018.stdout:osd.5 smithi186 running (12m) 49s ago 12m 228M 2581M 17.2.7-827-gd3b5060a 21de9b7369ad 2ee072a436d0 2024-04-12T21:03:00.689 INFO:teuthology.orchestra.run.smithi018.stdout:prometheus.smithi018 smithi018 *:9095 running (4m) 73s ago 17m 56.0M - 2.43.0 a07b618ecd1d f58841127e21 2024-04-12T21:03:00.690 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:03:00.693+0000 7fab47600700 1 -- 172.21.15.18:0/1010479790 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fab2806ed30 msgr2=0x7fab280711e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:03:00.690 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:03:00.693+0000 7fab47600700 1 --2- 172.21.15.18:0/1010479790 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fab2806ed30 0x7fab280711e0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fab34007fd0 tx=0x7fab34009040 comp rx=0 tx=0).stop 2024-04-12T21:03:00.690 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:03:00.693+0000 7fab47600700 1 -- 172.21.15.18:0/1010479790 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fab400fe7c0 msgr2=0x7fab400fc190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:03:00.690 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:03:00.693+0000 7fab47600700 1 --2- 172.21.15.18:0/1010479790 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fab400fe7c0 0x7fab400fc190 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fab2c009380 tx=0x7fab2c009e30 comp rx=0 tx=0).stop 2024-04-12T21:03:00.690 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:03:00.694+0000 7fab47600700 1 -- 172.21.15.18:0/1010479790 shutdown_connections 2024-04-12T21:03:00.690 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:03:00.694+0000 7fab47600700 1 --2- 172.21.15.18:0/1010479790 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fab2806ed30 0x7fab280711e0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:03:00.691 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:03:00.694+0000 7fab47600700 1 --2- 172.21.15.18:0/1010479790 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fab400fde20 0x7fab400fbc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:03:00.691 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:03:00.694+0000 7fab47600700 1 --2- 172.21.15.18:0/1010479790 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fab400fe7c0 0x7fab400fc190 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:03:00.691 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:03:00.694+0000 7fab47600700 1 -- 172.21.15.18:0/1010479790 >> 172.21.15.18:0/1010479790 conn(0x7fab400f7870 msgr2=0x7fab400f8da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:03:00.691 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:03:00.694+0000 7fab47600700 1 -- 172.21.15.18:0/1010479790 shutdown_connections 2024-04-12T21:03:00.691 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:03:00.694+0000 7fab47600700 1 -- 172.21.15.18:0/1010479790 wait complete. 2024-04-12T21:03:00.779 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:03:00 smithi186 ceph-mon[86379]: Upgrade: Updating crash.smithi018 (1/2) 2024-04-12T21:03:00.780 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:03:00 smithi186 ceph-mon[86379]: Deploying daemon crash.smithi018 on smithi018 2024-04-12T21:03:00.855 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:03:00.857+0000 7f6906c00700 1 -- 172.21.15.18:0/2174440546 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f69001037a0 msgr2=0x7f69000fce10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:03:00.856 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:03:00.857+0000 7f6906c00700 1 --2- 172.21.15.18:0/2174440546 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f69001037a0 0x7f69000fce10 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f68ec004940 tx=0x7f68ec009e30 comp rx=0 tx=0).stop 2024-04-12T21:03:00.856 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:03:00.858+0000 7f6906c00700 1 -- 172.21.15.18:0/2174440546 shutdown_connections 2024-04-12T21:03:00.856 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:03:00.858+0000 7f6906c00700 1 --2- 172.21.15.18:0/2174440546 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f69001037a0 0x7f69000fce10 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:03:00.856 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:03:00.858+0000 7f6906c00700 1 --2- 172.21.15.18:0/2174440546 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6900102e00 0x7f69001031d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:03:00.856 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:03:00.858+0000 7f6906c00700 1 -- 172.21.15.18:0/2174440546 >> 172.21.15.18:0/2174440546 conn(0x7f69000f89b0 msgr2=0x7f69000fadc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:03:00.856 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:03:00.858+0000 7f6906c00700 1 -- 172.21.15.18:0/2174440546 shutdown_connections 2024-04-12T21:03:00.856 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:03:00.858+0000 7f6906c00700 1 -- 172.21.15.18:0/2174440546 wait complete. 2024-04-12T21:03:00.856 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:03:00.859+0000 7f6906c00700 1 Processor -- start 2024-04-12T21:03:00.856 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:03:00.859+0000 7f6906c00700 1 -- start start 2024-04-12T21:03:00.856 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:03:00.859+0000 7f6906c00700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6900102e00 0x7f6900113d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:03:00.856 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:03:00.859+0000 7f6906c00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6900114290 0x7f690010dcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:03:00.856 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:03:00.859+0000 7f6906c00700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69001148a0 con 0x7f6900114290 2024-04-12T21:03:00.857 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:03:00.859+0000 7f6906c00700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f690010e1f0 con 0x7f6900102e00 2024-04-12T21:03:00.857 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:03:00.859+0000 7f68ff400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6900114290 0x7f690010dcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:03:00.857 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:03:00.860+0000 7f68ff400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6900114290 0x7f690010dcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:51872/0 (socket says 172.21.15.18:51872) 2024-04-12T21:03:00.857 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:03:00.860+0000 7f68ff400700 1 -- 172.21.15.18:0/3215516399 learned_addr learned my addr 172.21.15.18:0/3215516399 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:03:00.858 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:03:00.860+0000 7f68ffe00700 1 --2- 172.21.15.18:0/3215516399 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6900102e00 0x7f6900113d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:03:00.858 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:03:00.860+0000 7f68ff400700 1 -- 172.21.15.18:0/3215516399 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6900102e00 msgr2=0x7f6900113d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:03:00.858 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:03:00.860+0000 7f68ff400700 1 --2- 172.21.15.18:0/3215516399 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6900102e00 0x7f6900113d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:03:00.858 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:03:00.860+0000 7f68ff400700 1 -- 172.21.15.18:0/3215516399 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68ec005040 con 0x7f6900114290 2024-04-12T21:03:00.858 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:03:00.860+0000 7f68ffe00700 1 --2- 172.21.15.18:0/3215516399 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6900102e00 0x7f6900113d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-12T21:03:00.858 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:03:00.860+0000 7f68ff400700 1 --2- 172.21.15.18:0/3215516399 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6900114290 0x7f690010dcb0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f68ec000c00 tx=0x7f68ec009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:03:00.858 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:03:00.861+0000 7f68f7e00700 1 -- 172.21.15.18:0/3215516399 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f68ec00e040 con 0x7f6900114290 2024-04-12T21:03:00.859 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:03:00.861+0000 7f6906c00700 1 -- 172.21.15.18:0/3215516399 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f690010e470 con 0x7f6900114290 2024-04-12T21:03:00.859 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:03:00.861+0000 7f68f7e00700 1 -- 172.21.15.18:0/3215516399 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f68ec009430 con 0x7f6900114290 2024-04-12T21:03:00.859 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:03:00.861+0000 7f68f7e00700 1 -- 172.21.15.18:0/3215516399 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f68ec01c3f0 con 0x7f6900114290 2024-04-12T21:03:00.859 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:03:00.861+0000 7f6906c00700 1 -- 172.21.15.18:0/3215516399 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f690010ea30 con 0x7f6900114290 2024-04-12T21:03:00.859 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:03:00.862+0000 7f6906c00700 1 -- 172.21.15.18:0/3215516399 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6900100110 con 0x7f6900114290 2024-04-12T21:03:00.859 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:03:00.863+0000 7f68f7e00700 1 -- 172.21.15.18:0/3215516399 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 38) v1 ==== 92847+0+0 (secure 0 0 0) 0x7f68ec013840 con 0x7f6900114290 2024-04-12T21:03:00.860 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:03:00.864+0000 7f68f7e00700 1 --2- 172.21.15.18:0/3215516399 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f68e406ed20 0x7f68e40711d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:03:00.860 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:03:00.864+0000 7f68f7e00700 1 -- 172.21.15.18:0/3215516399 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6324+0+0 (secure 0 0 0) 0x7f68ec08fe70 con 0x7f6900114290 2024-04-12T21:03:00.860 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:03:00.864+0000 7f68ffe00700 1 --2- 172.21.15.18:0/3215516399 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f68e406ed20 0x7f68e40711d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:03:00.861 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:03:00.865+0000 7f68ffe00700 1 --2- 172.21.15.18:0/3215516399 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f68e406ed20 0x7f68e40711d0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f68f0006fd0 tx=0x7f68f0008040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:03:00.865 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:03:00.869+0000 7f68f7e00700 1 -- 172.21.15.18:0/3215516399 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f68ec05c630 con 0x7f6900114290 2024-04-12T21:03:01.196 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:03:01.199+0000 7f6906c00700 1 -- 172.21.15.18:0/3215516399 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f690010e600 con 0x7f6900114290 2024-04-12T21:03:01.197 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:03:01.201+0000 7f68f7e00700 1 -- 172.21.15.18:0/3215516399 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+736 (secure 0 0 0) 0x7f690010e600 con 0x7f6900114290 2024-04-12T21:03:01.197 INFO:teuthology.orchestra.run.smithi018.stdout:{ 2024-04-12T21:03:01.197 INFO:teuthology.orchestra.run.smithi018.stdout: "mon": { 2024-04-12T21:03:01.198 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 19.0.0-2879-g324baf8d (324baf8ddb91dea46cac50cb535c306115791355) squid (dev)": 2 2024-04-12T21:03:01.198 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-12T21:03:01.198 INFO:teuthology.orchestra.run.smithi018.stdout: "mgr": { 2024-04-12T21:03:01.198 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 19.0.0-2879-g324baf8d (324baf8ddb91dea46cac50cb535c306115791355) squid (dev)": 2 2024-04-12T21:03:01.198 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-12T21:03:01.198 INFO:teuthology.orchestra.run.smithi018.stdout: "osd": { 2024-04-12T21:03:01.198 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 17.2.7-827-gd3b5060a (d3b5060a17449edfae9385cdf792afec109fca5d) quincy (stable)": 6 2024-04-12T21:03:01.198 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-12T21:03:01.198 INFO:teuthology.orchestra.run.smithi018.stdout: "mds": { 2024-04-12T21:03:01.198 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 17.2.7-827-gd3b5060a (d3b5060a17449edfae9385cdf792afec109fca5d) quincy (stable)": 4 2024-04-12T21:03:01.198 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-12T21:03:01.198 INFO:teuthology.orchestra.run.smithi018.stdout: "overall": { 2024-04-12T21:03:01.198 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 17.2.7-827-gd3b5060a (d3b5060a17449edfae9385cdf792afec109fca5d) quincy (stable)": 10, 2024-04-12T21:03:01.199 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 19.0.0-2879-g324baf8d (324baf8ddb91dea46cac50cb535c306115791355) squid (dev)": 4 2024-04-12T21:03:01.199 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-04-12T21:03:01.199 INFO:teuthology.orchestra.run.smithi018.stdout:} 2024-04-12T21:03:01.201 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:03:01.204+0000 7f6906c00700 1 -- 172.21.15.18:0/3215516399 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f68e406ed20 msgr2=0x7f68e40711d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:03:01.201 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:03:01.204+0000 7f6906c00700 1 --2- 172.21.15.18:0/3215516399 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f68e406ed20 0x7f68e40711d0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f68f0006fd0 tx=0x7f68f0008040 comp rx=0 tx=0).stop 2024-04-12T21:03:01.201 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:03:01.204+0000 7f6906c00700 1 -- 172.21.15.18:0/3215516399 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6900114290 msgr2=0x7f690010dcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:03:01.201 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:03:01.204+0000 7f6906c00700 1 --2- 172.21.15.18:0/3215516399 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6900114290 0x7f690010dcb0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f68ec000c00 tx=0x7f68ec009e30 comp rx=0 tx=0).stop 2024-04-12T21:03:01.201 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:03:01.204+0000 7f6906c00700 1 -- 172.21.15.18:0/3215516399 shutdown_connections 2024-04-12T21:03:01.201 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:03:01.204+0000 7f6906c00700 1 --2- 172.21.15.18:0/3215516399 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f68e406ed20 0x7f68e40711d0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:03:01.201 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:03:01.204+0000 7f6906c00700 1 --2- 172.21.15.18:0/3215516399 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6900102e00 0x7f6900113d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:03:01.201 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:03:01.204+0000 7f6906c00700 1 --2- 172.21.15.18:0/3215516399 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6900114290 0x7f690010dcb0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:03:01.201 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:03:01.204+0000 7f6906c00700 1 -- 172.21.15.18:0/3215516399 >> 172.21.15.18:0/3215516399 conn(0x7f69000f89b0 msgr2=0x7f69000faca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:03:01.202 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:03:01.205+0000 7f6906c00700 1 -- 172.21.15.18:0/3215516399 shutdown_connections 2024-04-12T21:03:01.202 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:03:01.205+0000 7f6906c00700 1 -- 172.21.15.18:0/3215516399 wait complete. 2024-04-12T21:03:01.352 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:03:01.355+0000 7f32bf200700 1 -- 172.21.15.18:0/1389514064 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f32b80fe820 msgr2=0x7f32b8106e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:03:01.352 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:03:01.355+0000 7f32bf200700 1 --2- 172.21.15.18:0/1389514064 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f32b80fe820 0x7f32b8106e60 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f32a0004940 tx=0x7f32a0009e30 comp rx=0 tx=0).stop 2024-04-12T21:03:01.352 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:03:01.355+0000 7f32bf200700 1 -- 172.21.15.18:0/1389514064 shutdown_connections 2024-04-12T21:03:01.352 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:03:01.355+0000 7f32bf200700 1 --2- 172.21.15.18:0/1389514064 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f32b80fe820 0x7f32b8106e60 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:03:01.352 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:03:01.355+0000 7f32bf200700 1 --2- 172.21.15.18:0/1389514064 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f32b80fde80 0x7f32b80fe250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:03:01.352 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:03:01.355+0000 7f32bf200700 1 -- 172.21.15.18:0/1389514064 >> 172.21.15.18:0/1389514064 conn(0x7f32b80f78b0 msgr2=0x7f32b80f9cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:03:01.352 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:03:01.355+0000 7f32bf200700 1 -- 172.21.15.18:0/1389514064 shutdown_connections 2024-04-12T21:03:01.352 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:03:01.355+0000 7f32bf200700 1 -- 172.21.15.18:0/1389514064 wait complete. 2024-04-12T21:03:01.354 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:03:01.356+0000 7f32bf200700 1 Processor -- start 2024-04-12T21:03:01.354 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:03:01.356+0000 7f32bf200700 1 -- start start 2024-04-12T21:03:01.354 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:03:01.357+0000 7f32bf200700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f32b80fde80 0x7f32b80fbcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:03:01.354 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:03:01.357+0000 7f32bf200700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f32b80fe820 0x7f32b80fc1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:03:01.354 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:03:01.357+0000 7f32bf200700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32b80fdc00 con 0x7f32b80fe820 2024-04-12T21:03:01.354 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:03:01.357+0000 7f32bf200700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32b80fc730 con 0x7f32b80fde80 2024-04-12T21:03:01.354 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:03:01.357+0000 7f32afe00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f32b80fe820 0x7f32b80fc1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:03:01.354 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:03:01.357+0000 7f32afe00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f32b80fe820 0x7f32b80fc1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:51884/0 (socket says 172.21.15.18:51884) 2024-04-12T21:03:01.354 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:03:01.357+0000 7f32afe00700 1 -- 172.21.15.18:0/3723764067 learned_addr learned my addr 172.21.15.18:0/3723764067 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:03:01.354 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:03:01.357+0000 7f32afe00700 1 -- 172.21.15.18:0/3723764067 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f32b80fde80 msgr2=0x7f32b80fbcb0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-12T21:03:01.355 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:03:01.357+0000 7f32afe00700 1 --2- 172.21.15.18:0/3723764067 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f32b80fde80 0x7f32b80fbcb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:03:01.355 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:03:01.357+0000 7f32afe00700 1 -- 172.21.15.18:0/3723764067 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32a0005040 con 0x7f32b80fe820 2024-04-12T21:03:01.355 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:03:01.358+0000 7f32afe00700 1 --2- 172.21.15.18:0/3723764067 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f32b80fe820 0x7f32b80fc1f0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f32a0004890 tx=0x7f32a0009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:03:01.355 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:03:01.358+0000 7f32b4c00700 1 -- 172.21.15.18:0/3723764067 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f32a000e040 con 0x7f32b80fe820 2024-04-12T21:03:01.355 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:03:01.358+0000 7f32b4c00700 1 -- 172.21.15.18:0/3723764067 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f32a0009af0 con 0x7f32b80fe820 2024-04-12T21:03:01.356 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:03:01.358+0000 7f32b4c00700 1 -- 172.21.15.18:0/3723764067 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f32a001c630 con 0x7f32b80fe820 2024-04-12T21:03:01.356 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:03:01.358+0000 7f32bf200700 1 -- 172.21.15.18:0/3723764067 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f32b80fca10 con 0x7f32b80fe820 2024-04-12T21:03:01.356 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:03:01.358+0000 7f32bf200700 1 -- 172.21.15.18:0/3723764067 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f32b811a040 con 0x7f32b80fe820 2024-04-12T21:03:01.356 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:03:01.359+0000 7f32bf200700 1 -- 172.21.15.18:0/3723764067 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f32b8066af0 con 0x7f32b80fe820 2024-04-12T21:03:01.356 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:03:01.360+0000 7f32b4c00700 1 -- 172.21.15.18:0/3723764067 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 38) v1 ==== 92847+0+0 (secure 0 0 0) 0x7f32a000bbc0 con 0x7f32b80fe820 2024-04-12T21:03:01.357 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:03:01.361+0000 7f32b4c00700 1 --2- 172.21.15.18:0/3723764067 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f329c06ed30 0x7f329c0711e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:03:01.358 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:03:01.361+0000 7f32b4c00700 1 -- 172.21.15.18:0/3723764067 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6324+0+0 (secure 0 0 0) 0x7f32a008fdb0 con 0x7f32b80fe820 2024-04-12T21:03:01.358 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:03:01.362+0000 7f32b7e00700 1 --2- 172.21.15.18:0/3723764067 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f329c06ed30 0x7f329c0711e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:03:01.358 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:03:01.362+0000 7f32b7e00700 1 --2- 172.21.15.18:0/3723764067 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f329c06ed30 0x7f329c0711e0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f32a8004ad0 tx=0x7f32a8007d40 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:03:01.362 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:03:01.366+0000 7f32b4c00700 1 -- 172.21.15.18:0/3723764067 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f32a00589c0 con 0x7f32b80fe820 2024-04-12T21:03:01.648 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:03:01.652+0000 7f32bf200700 1 -- 172.21.15.18:0/3723764067 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f32b8066af0 con 0x7f32b80fe820 2024-04-12T21:03:01.648 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:03:01.652+0000 7f32b4c00700 1 -- 172.21.15.18:0/3723764067 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 23 v23) v1 ==== 76+0+1886 (secure 0 0 0) 0x7f32a005c080 con 0x7f32b80fe820 2024-04-12T21:03:01.650 INFO:teuthology.orchestra.run.smithi018.stdout:e23 2024-04-12T21:03:01.650 INFO:teuthology.orchestra.run.smithi018.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-12T21:03:01.653 INFO:teuthology.orchestra.run.smithi018.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-12T21:03:01.653 INFO:teuthology.orchestra.run.smithi018.stdout:legacy client fscid: 1 2024-04-12T21:03:01.653 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-12T21:03:01.653 INFO:teuthology.orchestra.run.smithi018.stdout:Filesystem 'cephfs' (1) 2024-04-12T21:03:01.653 INFO:teuthology.orchestra.run.smithi018.stdout:fs_name cephfs 2024-04-12T21:03:01.653 INFO:teuthology.orchestra.run.smithi018.stdout:epoch 23 2024-04-12T21:03:01.654 INFO:teuthology.orchestra.run.smithi018.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-04-12T21:03:01.654 INFO:teuthology.orchestra.run.smithi018.stdout:created 2024-04-12T20:51:57.362850+0000 2024-04-12T21:03:01.654 INFO:teuthology.orchestra.run.smithi018.stdout:modified 2024-04-12T21:01:10.670287+0000 2024-04-12T21:03:01.654 INFO:teuthology.orchestra.run.smithi018.stdout:tableserver 0 2024-04-12T21:03:01.654 INFO:teuthology.orchestra.run.smithi018.stdout:root 0 2024-04-12T21:03:01.654 INFO:teuthology.orchestra.run.smithi018.stdout:session_timeout 60 2024-04-12T21:03:01.654 INFO:teuthology.orchestra.run.smithi018.stdout:session_autoclose 300 2024-04-12T21:03:01.654 INFO:teuthology.orchestra.run.smithi018.stdout:max_file_size 1099511627776 2024-04-12T21:03:01.654 INFO:teuthology.orchestra.run.smithi018.stdout:max_xattr_size 65536 2024-04-12T21:03:01.654 INFO:teuthology.orchestra.run.smithi018.stdout:required_client_features {} 2024-04-12T21:03:01.654 INFO:teuthology.orchestra.run.smithi018.stdout:last_failure 0 2024-04-12T21:03:01.654 INFO:teuthology.orchestra.run.smithi018.stdout:last_failure_osd_epoch 46 2024-04-12T21:03:01.654 INFO:teuthology.orchestra.run.smithi018.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-12T21:03:01.654 INFO:teuthology.orchestra.run.smithi018.stdout:max_mds 2 2024-04-12T21:03:01.655 INFO:teuthology.orchestra.run.smithi018.stdout:in 0,1 2024-04-12T21:03:01.655 INFO:teuthology.orchestra.run.smithi018.stdout:up {0=14452,1=14464} 2024-04-12T21:03:01.655 INFO:teuthology.orchestra.run.smithi018.stdout:failed 2024-04-12T21:03:01.655 INFO:teuthology.orchestra.run.smithi018.stdout:damaged 2024-04-12T21:03:01.655 INFO:teuthology.orchestra.run.smithi018.stdout:stopped 2024-04-12T21:03:01.655 INFO:teuthology.orchestra.run.smithi018.stdout:data_pools [3] 2024-04-12T21:03:01.655 INFO:teuthology.orchestra.run.smithi018.stdout:metadata_pool 2 2024-04-12T21:03:01.655 INFO:teuthology.orchestra.run.smithi018.stdout:inline_data disabled 2024-04-12T21:03:01.655 INFO:teuthology.orchestra.run.smithi018.stdout:balancer 2024-04-12T21:03:01.655 INFO:teuthology.orchestra.run.smithi018.stdout:bal_rank_mask -1 2024-04-12T21:03:01.655 INFO:teuthology.orchestra.run.smithi018.stdout:standby_count_wanted 1 2024-04-12T21:03:01.655 INFO:teuthology.orchestra.run.smithi018.stdout:qdb_cluster leader: 0 members: 2024-04-12T21:03:01.655 INFO:teuthology.orchestra.run.smithi018.stdout:[mds.cephfs.smithi018.acqbsc{0:14452} state up:active seq 5 join_fscid=1 addr [v2:172.21.15.18:6826/1564065434,v1:172.21.15.18:6827/1564065434] compat {c=[1],r=[1],i=[7ff]}] 2024-04-12T21:03:01.655 INFO:teuthology.orchestra.run.smithi018.stdout:[mds.cephfs.smithi018.szjluo{1:14464} state up:active seq 6 join_fscid=1 addr [v2:172.21.15.18:6828/722443353,v1:172.21.15.18:6829/722443353] compat {c=[1],r=[1],i=[7ff]}] 2024-04-12T21:03:01.656 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-12T21:03:01.656 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-12T21:03:01.656 INFO:teuthology.orchestra.run.smithi018.stdout:Standby daemons: 2024-04-12T21:03:01.656 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-12T21:03:01.656 INFO:teuthology.orchestra.run.smithi018.stdout:[mds.cephfs.smithi186.wjnafs{-1:24277} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.186:6826/438347207,v1:172.21.15.186:6827/438347207] compat {c=[1],r=[1],i=[7ff]}] 2024-04-12T21:03:01.656 INFO:teuthology.orchestra.run.smithi018.stdout:[mds.cephfs.smithi186.jdllyn{-1:24281} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.186:6824/1464359952,v1:172.21.15.186:6825/1464359952] compat {c=[1],r=[1],i=[7ff]}] 2024-04-12T21:03:01.657 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:03:01.655+0000 7f32bf200700 1 -- 172.21.15.18:0/3723764067 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f329c06ed30 msgr2=0x7f329c0711e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:03:01.657 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:03:01.655+0000 7f32bf200700 1 --2- 172.21.15.18:0/3723764067 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f329c06ed30 0x7f329c0711e0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f32a8004ad0 tx=0x7f32a8007d40 comp rx=0 tx=0).stop 2024-04-12T21:03:01.657 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:03:01.655+0000 7f32bf200700 1 -- 172.21.15.18:0/3723764067 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f32b80fe820 msgr2=0x7f32b80fc1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:03:01.657 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:03:01.655+0000 7f32bf200700 1 --2- 172.21.15.18:0/3723764067 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f32b80fe820 0x7f32b80fc1f0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f32a0004890 tx=0x7f32a0009e30 comp rx=0 tx=0).stop 2024-04-12T21:03:01.657 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:03:01.655+0000 7f32bf200700 1 -- 172.21.15.18:0/3723764067 shutdown_connections 2024-04-12T21:03:01.657 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:03:01.655+0000 7f32bf200700 1 --2- 172.21.15.18:0/3723764067 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f329c06ed30 0x7f329c0711e0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:03:01.657 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:03:01.655+0000 7f32bf200700 1 --2- 172.21.15.18:0/3723764067 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f32b80fde80 0x7f32b80fbcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:03:01.657 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:03:01.655+0000 7f32bf200700 1 --2- 172.21.15.18:0/3723764067 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f32b80fe820 0x7f32b80fc1f0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:03:01.657 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:03:01.655+0000 7f32bf200700 1 -- 172.21.15.18:0/3723764067 >> 172.21.15.18:0/3723764067 conn(0x7f32b80f78b0 msgr2=0x7f32b80f8d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:03:01.657 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:03:01.656+0000 7f32bf200700 1 -- 172.21.15.18:0/3723764067 shutdown_connections 2024-04-12T21:03:01.657 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:03:01.656+0000 7f32bf200700 1 -- 172.21.15.18:0/3723764067 wait complete. 2024-04-12T21:03:01.658 INFO:teuthology.orchestra.run.smithi018.stderr:dumped fsmap epoch 23 2024-04-12T21:03:01.780 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:03:01 smithi186 ceph-mon[86379]: pgmap v43: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-12T21:03:01.780 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:03:01 smithi186 ceph-mon[86379]: from='client.34188 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:03:01.780 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:03:01 smithi186 ceph-mon[86379]: from='client.34192 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:03:01.780 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:03:01 smithi186 ceph-mon[86379]: from='client.? 172.21.15.18:0/3215516399' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T21:03:01.782 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:03:01 smithi018 ceph-mon[112590]: pgmap v43: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-12T21:03:01.782 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:03:01 smithi018 ceph-mon[112590]: from='client.34188 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:03:01.782 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:03:01 smithi018 ceph-mon[112590]: from='client.34192 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:03:01.782 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:03:01 smithi018 ceph-mon[112590]: from='client.? 172.21.15.18:0/3215516399' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T21:03:01.810 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:03:01.813+0000 7f8767e00700 1 -- 172.21.15.18:0/4029409581 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8768104e60 msgr2=0x7f8768105230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:03:01.810 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:03:01.813+0000 7f8767e00700 1 --2- 172.21.15.18:0/4029409581 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8768104e60 0x7f8768105230 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f8758009ce0 tx=0x7f8758004d10 comp rx=0 tx=0).stop 2024-04-12T21:03:01.810 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:03:01.813+0000 7f8767e00700 1 -- 172.21.15.18:0/4029409581 shutdown_connections 2024-04-12T21:03:01.810 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:03:01.813+0000 7f8767e00700 1 --2- 172.21.15.18:0/4029409581 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8768105800 0x7f87680fee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:03:01.810 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:03:01.813+0000 7f8767e00700 1 --2- 172.21.15.18:0/4029409581 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8768104e60 0x7f8768105230 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:03:01.810 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:03:01.813+0000 7f8767e00700 1 -- 172.21.15.18:0/4029409581 >> 172.21.15.18:0/4029409581 conn(0x7f87680fab10 msgr2=0x7f87680fcef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:03:01.810 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:03:01.813+0000 7f8767e00700 1 -- 172.21.15.18:0/4029409581 shutdown_connections 2024-04-12T21:03:01.810 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:03:01.813+0000 7f8767e00700 1 -- 172.21.15.18:0/4029409581 wait complete. 2024-04-12T21:03:01.811 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:03:01.814+0000 7f8767e00700 1 Processor -- start 2024-04-12T21:03:01.811 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:03:01.815+0000 7f8767e00700 1 -- start start 2024-04-12T21:03:01.811 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:03:01.815+0000 7f8767e00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8768104e60 0x7f8768115df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:03:01.811 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:03:01.815+0000 7f8767e00700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8768105800 0x7f8768116330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:03:01.811 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:03:01.815+0000 7f8767e00700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8768116980 con 0x7f8768104e60 2024-04-12T21:03:01.811 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:03:01.815+0000 7f8767e00700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f876810fd50 con 0x7f8768105800 2024-04-12T21:03:01.812 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:03:01.815+0000 7f8766a00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8768104e60 0x7f8768115df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:03:01.812 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:03:01.815+0000 7f8766a00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8768104e60 0x7f8768115df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:51906/0 (socket says 172.21.15.18:51906) 2024-04-12T21:03:01.812 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:03:01.815+0000 7f8766a00700 1 -- 172.21.15.18:0/849545163 learned_addr learned my addr 172.21.15.18:0/849545163 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:03:01.812 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:03:01.816+0000 7f8766a00700 1 -- 172.21.15.18:0/849545163 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8768105800 msgr2=0x7f8768116330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:03:01.812 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:03:01.816+0000 7f8766000700 1 --2- 172.21.15.18:0/849545163 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8768105800 0x7f8768116330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:03:01.812 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:03:01.816+0000 7f8766a00700 1 --2- 172.21.15.18:0/849545163 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8768105800 0x7f8768116330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:03:01.812 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:03:01.816+0000 7f8766a00700 1 -- 172.21.15.18:0/849545163 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8758009940 con 0x7f8768104e60 2024-04-12T21:03:01.813 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:03:01.816+0000 7f8766a00700 1 --2- 172.21.15.18:0/849545163 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8768104e60 0x7f8768115df0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f8758005010 tx=0x7f8758004d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:03:01.813 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:03:01.817+0000 7f875ea00700 1 -- 172.21.15.18:0/849545163 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f875800e040 con 0x7f8768104e60 2024-04-12T21:03:01.813 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:03:01.817+0000 7f875ea00700 1 -- 172.21.15.18:0/849545163 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f8758008da0 con 0x7f8768104e60 2024-04-12T21:03:01.813 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:03:01.817+0000 7f8767e00700 1 -- 172.21.15.18:0/849545163 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f876810ffd0 con 0x7f8768104e60 2024-04-12T21:03:01.813 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:03:01.817+0000 7f875ea00700 1 -- 172.21.15.18:0/849545163 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f875801c640 con 0x7f8768104e60 2024-04-12T21:03:01.814 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:03:01.817+0000 7f8767e00700 1 -- 172.21.15.18:0/849545163 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8768110590 con 0x7f8768104e60 2024-04-12T21:03:01.815 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:03:01.819+0000 7f8767e00700 1 -- 172.21.15.18:0/849545163 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f87681005b0 con 0x7f8768104e60 2024-04-12T21:03:01.816 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:03:01.820+0000 7f875ea00700 1 -- 172.21.15.18:0/849545163 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 38) v1 ==== 92847+0+0 (secure 0 0 0) 0x7f8758012410 con 0x7f8768104e60 2024-04-12T21:03:01.818 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:03:01.822+0000 7f875ea00700 1 --2- 172.21.15.18:0/849545163 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f874c06ed30 0x7f874c0711e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:03:01.818 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:03:01.822+0000 7f875ea00700 1 -- 172.21.15.18:0/849545163 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6324+0+0 (secure 0 0 0) 0x7f8758090500 con 0x7f8768104e60 2024-04-12T21:03:01.819 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:03:01.822+0000 7f8766000700 1 --2- 172.21.15.18:0/849545163 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f874c06ed30 0x7f874c0711e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:03:01.819 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:03:01.823+0000 7f8766000700 1 --2- 172.21.15.18:0/849545163 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f874c06ed30 0x7f874c0711e0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f8768111690 tx=0x7f875400b040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:03:01.823 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:03:01.827+0000 7f875ea00700 1 -- 172.21.15.18:0/849545163 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f875805cce0 con 0x7f8768104e60 2024-04-12T21:03:02.100 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:03:02.103+0000 7f8767e00700 1 -- 172.21.15.18:0/849545163 --> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f876804bed0 con 0x7f874c06ed30 2024-04-12T21:03:02.102 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:03:02.105+0000 7f875ea00700 1 -- 172.21.15.18:0/849545163 <== mgr.34102 v2:172.21.15.18:6800/3812891919 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+385 (secure 0 0 0) 0x7f876804bed0 con 0x7f874c06ed30 2024-04-12T21:03:02.102 INFO:teuthology.orchestra.run.smithi018.stdout:{ 2024-04-12T21:03:02.102 INFO:teuthology.orchestra.run.smithi018.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:f4452def3be4d350f4a8e5feaf7ebb1511be9298fe68478ba7a2d6fbd7d69a63", 2024-04-12T21:03:02.102 INFO:teuthology.orchestra.run.smithi018.stdout: "in_progress": true, 2024-04-12T21:03:02.102 INFO:teuthology.orchestra.run.smithi018.stdout: "which": "Upgrading all daemon types on all hosts", 2024-04-12T21:03:02.102 INFO:teuthology.orchestra.run.smithi018.stdout: "services_complete": [ 2024-04-12T21:03:02.102 INFO:teuthology.orchestra.run.smithi018.stdout: "mon", 2024-04-12T21:03:02.102 INFO:teuthology.orchestra.run.smithi018.stdout: "mgr" 2024-04-12T21:03:02.102 INFO:teuthology.orchestra.run.smithi018.stdout: ], 2024-04-12T21:03:02.103 INFO:teuthology.orchestra.run.smithi018.stdout: "progress": "4/21 daemons upgraded", 2024-04-12T21:03:02.103 INFO:teuthology.orchestra.run.smithi018.stdout: "message": "Currently upgrading crash daemons", 2024-04-12T21:03:02.103 INFO:teuthology.orchestra.run.smithi018.stdout: "is_paused": false 2024-04-12T21:03:02.103 INFO:teuthology.orchestra.run.smithi018.stdout:} 2024-04-12T21:03:02.106 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:03:02.109+0000 7f8767e00700 1 -- 172.21.15.18:0/849545163 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f874c06ed30 msgr2=0x7f874c0711e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:03:02.106 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:03:02.109+0000 7f8767e00700 1 --2- 172.21.15.18:0/849545163 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f874c06ed30 0x7f874c0711e0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f8768111690 tx=0x7f875400b040 comp rx=0 tx=0).stop 2024-04-12T21:03:02.106 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:03:02.109+0000 7f8767e00700 1 -- 172.21.15.18:0/849545163 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8768104e60 msgr2=0x7f8768115df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:03:02.106 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:03:02.109+0000 7f8767e00700 1 --2- 172.21.15.18:0/849545163 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8768104e60 0x7f8768115df0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f8758005010 tx=0x7f8758004d10 comp rx=0 tx=0).stop 2024-04-12T21:03:02.106 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:03:02.109+0000 7f8767e00700 1 -- 172.21.15.18:0/849545163 shutdown_connections 2024-04-12T21:03:02.106 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:03:02.109+0000 7f8767e00700 1 --2- 172.21.15.18:0/849545163 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f874c06ed30 0x7f874c0711e0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:03:02.106 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:03:02.109+0000 7f8767e00700 1 --2- 172.21.15.18:0/849545163 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8768104e60 0x7f8768115df0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:03:02.106 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:03:02.109+0000 7f8767e00700 1 --2- 172.21.15.18:0/849545163 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8768105800 0x7f8768116330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:03:02.106 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:03:02.109+0000 7f8767e00700 1 -- 172.21.15.18:0/849545163 >> 172.21.15.18:0/849545163 conn(0x7f87680fab10 msgr2=0x7f87680fca30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:03:02.106 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:03:02.109+0000 7f8767e00700 1 -- 172.21.15.18:0/849545163 shutdown_connections 2024-04-12T21:03:02.106 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:03:02.109+0000 7f8767e00700 1 -- 172.21.15.18:0/849545163 wait complete. 2024-04-12T21:03:02.261 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:03:02.265+0000 7f927c000700 1 -- 172.21.15.18:0/212115346 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9274105990 msgr2=0x7f92740ff000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:03:02.262 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:03:02.265+0000 7f927c000700 1 --2- 172.21.15.18:0/212115346 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9274105990 0x7f92740ff000 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f9260004940 tx=0x7f9260009e30 comp rx=0 tx=0).stop 2024-04-12T21:03:02.262 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:03:02.265+0000 7f927c000700 1 -- 172.21.15.18:0/212115346 shutdown_connections 2024-04-12T21:03:02.262 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:03:02.265+0000 7f927c000700 1 --2- 172.21.15.18:0/212115346 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9274105990 0x7f92740ff000 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:03:02.262 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:03:02.265+0000 7f927c000700 1 --2- 172.21.15.18:0/212115346 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9274104ff0 0x7f92741053c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:03:02.262 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:03:02.265+0000 7f927c000700 1 -- 172.21.15.18:0/212115346 >> 172.21.15.18:0/212115346 conn(0x7f92740faba0 msgr2=0x7f92740fcfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:03:02.262 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:03:02.265+0000 7f927c000700 1 -- 172.21.15.18:0/212115346 shutdown_connections 2024-04-12T21:03:02.262 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:03:02.265+0000 7f927c000700 1 -- 172.21.15.18:0/212115346 wait complete. 2024-04-12T21:03:02.263 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:03:02.266+0000 7f927c000700 1 Processor -- start 2024-04-12T21:03:02.263 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:03:02.266+0000 7f927c000700 1 -- start start 2024-04-12T21:03:02.263 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:03:02.267+0000 7f927c000700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9274104ff0 0x7f9274115b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:03:02.263 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:03:02.267+0000 7f927c000700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9274105990 0x7f92741160c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:03:02.263 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:03:02.267+0000 7f927c000700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9274116710 con 0x7f9274104ff0 2024-04-12T21:03:02.263 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:03:02.267+0000 7f927c000700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f927410fcf0 con 0x7f9274105990 2024-04-12T21:03:02.263 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:03:02.267+0000 7f9279600700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9274104ff0 0x7f9274115b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:03:02.264 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:03:02.267+0000 7f9279600700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9274104ff0 0x7f9274115b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:51932/0 (socket says 172.21.15.18:51932) 2024-04-12T21:03:02.264 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:03:02.267+0000 7f9279600700 1 -- 172.21.15.18:0/4241650300 learned_addr learned my addr 172.21.15.18:0/4241650300 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:03:02.265 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:03:02.267+0000 7f9278c00700 1 --2- 172.21.15.18:0/4241650300 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9274105990 0x7f92741160c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:03:02.265 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:03:02.267+0000 7f9279600700 1 -- 172.21.15.18:0/4241650300 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9274105990 msgr2=0x7f92741160c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:03:02.265 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:03:02.267+0000 7f9279600700 1 --2- 172.21.15.18:0/4241650300 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9274105990 0x7f92741160c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:03:02.265 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:03:02.267+0000 7f9279600700 1 -- 172.21.15.18:0/4241650300 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9260005040 con 0x7f9274104ff0 2024-04-12T21:03:02.265 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:03:02.267+0000 7f9278c00700 1 --2- 172.21.15.18:0/4241650300 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9274105990 0x7f92741160c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-12T21:03:02.265 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:03:02.268+0000 7f9279600700 1 --2- 172.21.15.18:0/4241650300 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9274104ff0 0x7f9274115b80 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f926800cb10 tx=0x7f9268007f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:03:02.265 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:03:02.268+0000 7f9271600700 1 -- 172.21.15.18:0/4241650300 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9268004200 con 0x7f9274104ff0 2024-04-12T21:03:02.265 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:03:02.268+0000 7f9271600700 1 -- 172.21.15.18:0/4241650300 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f926800acc0 con 0x7f9274104ff0 2024-04-12T21:03:02.265 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:03:02.268+0000 7f927c000700 1 -- 172.21.15.18:0/4241650300 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f927410ffd0 con 0x7f9274104ff0 2024-04-12T21:03:02.265 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:03:02.268+0000 7f9271600700 1 -- 172.21.15.18:0/4241650300 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9268003d90 con 0x7f9274104ff0 2024-04-12T21:03:02.265 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:03:02.268+0000 7f927c000700 1 -- 172.21.15.18:0/4241650300 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9274110590 con 0x7f9274104ff0 2024-04-12T21:03:02.266 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:03:02.270+0000 7f927c000700 1 -- 172.21.15.18:0/4241650300 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9274100740 con 0x7f9274104ff0 2024-04-12T21:03:02.267 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:03:02.271+0000 7f9271600700 1 -- 172.21.15.18:0/4241650300 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 38) v1 ==== 92847+0+0 (secure 0 0 0) 0x7f9268003430 con 0x7f9274104ff0 2024-04-12T21:03:02.269 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:03:02.273+0000 7f9271600700 1 --2- 172.21.15.18:0/4241650300 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f925c06ed30 0x7f925c0711e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:03:02.269 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:03:02.273+0000 7f9271600700 1 -- 172.21.15.18:0/4241650300 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6324+0+0 (secure 0 0 0) 0x7f926808ec70 con 0x7f9274104ff0 2024-04-12T21:03:02.270 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:03:02.273+0000 7f9278c00700 1 --2- 172.21.15.18:0/4241650300 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f925c06ed30 0x7f925c0711e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:03:02.270 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:03:02.274+0000 7f9278c00700 1 --2- 172.21.15.18:0/4241650300 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f925c06ed30 0x7f925c0711e0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f9260009fd0 tx=0x7f9260009e30 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:03:02.272 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:03:02.276+0000 7f9271600700 1 -- 172.21.15.18:0/4241650300 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f9268095080 con 0x7f9274104ff0 2024-04-12T21:03:02.617 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:03:02.620+0000 7f927c000700 1 -- 172.21.15.18:0/4241650300 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f927404c710 con 0x7f9274104ff0 2024-04-12T21:03:02.617 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:03:02.621+0000 7f9271600700 1 -- 172.21.15.18:0/4241650300 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+10 (secure 0 0 0) 0x7f927404c710 con 0x7f9274104ff0 2024-04-12T21:03:02.617 INFO:teuthology.orchestra.run.smithi018.stdout:HEALTH_OK 2024-04-12T21:03:02.620 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:03:02.624+0000 7f927c000700 1 -- 172.21.15.18:0/4241650300 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f925c06ed30 msgr2=0x7f925c0711e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:03:02.621 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:03:02.624+0000 7f927c000700 1 --2- 172.21.15.18:0/4241650300 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f925c06ed30 0x7f925c0711e0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f9260009fd0 tx=0x7f9260009e30 comp rx=0 tx=0).stop 2024-04-12T21:03:02.621 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:03:02.624+0000 7f927c000700 1 -- 172.21.15.18:0/4241650300 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9274104ff0 msgr2=0x7f9274115b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:03:02.621 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:03:02.624+0000 7f927c000700 1 --2- 172.21.15.18:0/4241650300 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9274104ff0 0x7f9274115b80 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f926800cb10 tx=0x7f9268007f90 comp rx=0 tx=0).stop 2024-04-12T21:03:02.621 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:03:02.624+0000 7f927c000700 1 -- 172.21.15.18:0/4241650300 shutdown_connections 2024-04-12T21:03:02.621 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:03:02.624+0000 7f927c000700 1 --2- 172.21.15.18:0/4241650300 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f925c06ed30 0x7f925c0711e0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:03:02.621 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:03:02.624+0000 7f927c000700 1 --2- 172.21.15.18:0/4241650300 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9274104ff0 0x7f9274115b80 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:03:02.621 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:03:02.624+0000 7f927c000700 1 --2- 172.21.15.18:0/4241650300 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9274105990 0x7f92741160c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:03:02.621 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:03:02.624+0000 7f927c000700 1 -- 172.21.15.18:0/4241650300 >> 172.21.15.18:0/4241650300 conn(0x7f92740faba0 msgr2=0x7f92740fbae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:03:02.621 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:03:02.624+0000 7f927c000700 1 -- 172.21.15.18:0/4241650300 shutdown_connections 2024-04-12T21:03:02.621 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:03:02.625+0000 7f927c000700 1 -- 172.21.15.18:0/4241650300 wait complete. 2024-04-12T21:03:02.780 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:03:02 smithi186 ceph-mon[86379]: from='client.34196 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:03:02.780 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:03:02 smithi186 ceph-mon[86379]: from='client.? 172.21.15.18:0/3723764067' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-12T21:03:02.850 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:03:02 smithi018 ceph-mon[112590]: from='client.34196 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:03:02.851 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:03:02 smithi018 ceph-mon[112590]: from='client.? 172.21.15.18:0/3723764067' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-12T21:03:03.750 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:03:03 smithi018 ceph-mon[112590]: pgmap v44: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-12T21:03:03.750 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:03:03 smithi018 ceph-mon[112590]: from='client.34208 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:03:03.750 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:03:03 smithi018 ceph-mon[112590]: from='client.? 172.21.15.18:0/4241650300' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-12T21:03:03.780 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:03:03 smithi186 ceph-mon[86379]: pgmap v44: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-12T21:03:03.780 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:03:03 smithi186 ceph-mon[86379]: from='client.34208 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:03:03.780 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:03:03 smithi186 ceph-mon[86379]: from='client.? 172.21.15.18:0/4241650300' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-12T21:03:05.779 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:03:05 smithi186 ceph-mon[86379]: pgmap v45: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-12T21:03:05.779 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:03:05 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' 2024-04-12T21:03:05.779 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:03:05 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' 2024-04-12T21:03:05.851 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:03:05 smithi018 ceph-mon[112590]: pgmap v45: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-12T21:03:05.851 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:03:05 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' 2024-04-12T21:03:05.851 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:03:05 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' 2024-04-12T21:03:07.780 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:03:07 smithi186 ceph-mon[86379]: pgmap v46: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-12T21:03:07.780 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:03:07 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' 2024-04-12T21:03:07.780 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:03:07 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi186", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-04-12T21:03:07.780 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:03:07 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-12T21:03:07.851 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:03:07 smithi018 ceph-mon[112590]: pgmap v46: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-12T21:03:07.851 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:03:07 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' 2024-04-12T21:03:07.851 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:03:07 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi186", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-04-12T21:03:07.851 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:03:07 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-12T21:03:08.780 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:03:08 smithi186 ceph-mon[86379]: Upgrade: Updating crash.smithi186 (2/2) 2024-04-12T21:03:08.780 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:03:08 smithi186 ceph-mon[86379]: Deploying daemon crash.smithi186 on smithi186 2024-04-12T21:03:08.784 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:03:08 smithi018 ceph-mon[112590]: Upgrade: Updating crash.smithi186 (2/2) 2024-04-12T21:03:08.784 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:03:08 smithi018 ceph-mon[112590]: Deploying daemon crash.smithi186 on smithi186 2024-04-12T21:03:09.580 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:03:09 smithi186 ceph-mon[86379]: pgmap v47: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-12T21:03:09.851 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:03:09 smithi018 ceph-mon[112590]: pgmap v47: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-12T21:03:10.597 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:03:10 smithi186 ceph-mon[86379]: pgmap v48: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-12T21:03:10.597 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:03:10 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T21:03:10.851 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:03:10 smithi018 ceph-mon[112590]: pgmap v48: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-12T21:03:10.851 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:03:10 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T21:03:13.101 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:03:12 smithi018 ceph-mon[112590]: pgmap v49: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 511 B/s wr, 0 op/s 2024-04-12T21:03:13.101 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:03:12 smithi018 ceph-mon[112590]: mgrmap e39: smithi018.xjzwga(active, since 92s), standbys: smithi186.kqcfxk 2024-04-12T21:03:13.101 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:03:12 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' 2024-04-12T21:03:13.101 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:03:12 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' 2024-04-12T21:03:13.101 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:03:12 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-12T21:03:13.280 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:03:12 smithi186 ceph-mon[86379]: pgmap v49: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 511 B/s wr, 0 op/s 2024-04-12T21:03:13.280 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:03:12 smithi186 ceph-mon[86379]: mgrmap e39: smithi018.xjzwga(active, since 92s), standbys: smithi186.kqcfxk 2024-04-12T21:03:13.280 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:03:12 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' 2024-04-12T21:03:13.280 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:03:12 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' 2024-04-12T21:03:13.280 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:03:12 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-12T21:03:15.101 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:03:14 smithi018 ceph-mon[112590]: pgmap v50: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 511 B/s wr, 0 op/s 2024-04-12T21:03:15.279 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:03:14 smithi186 ceph-mon[86379]: pgmap v50: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 511 B/s wr, 0 op/s 2024-04-12T21:03:17.101 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:03:16 smithi018 ceph-mon[112590]: pgmap v51: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 511 B/s wr, 0 op/s 2024-04-12T21:03:17.101 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:03:16 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' 2024-04-12T21:03:17.279 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:03:16 smithi186 ceph-mon[86379]: pgmap v51: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 511 B/s wr, 0 op/s 2024-04-12T21:03:17.279 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:03:16 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' 2024-04-12T21:03:18.246 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:03:17 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' 2024-04-12T21:03:18.279 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:03:17 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' 2024-04-12T21:03:19.183 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:03:18 smithi018 ceph-mon[112590]: pgmap v52: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 511 B/s wr, 0 op/s 2024-04-12T21:03:19.279 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:03:18 smithi186 ceph-mon[86379]: pgmap v52: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 511 B/s wr, 0 op/s 2024-04-12T21:03:20.157 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:03:19 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' 2024-04-12T21:03:20.157 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:03:19 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' 2024-04-12T21:03:20.350 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:03:19 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' 2024-04-12T21:03:20.350 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:03:19 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' 2024-04-12T21:03:21.279 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:03:21 smithi186 ceph-mon[86379]: pgmap v53: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 511 B/s wr, 0 op/s 2024-04-12T21:03:21.280 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:03:21 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' 2024-04-12T21:03:21.280 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:03:21 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' 2024-04-12T21:03:21.600 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:03:21 smithi018 ceph-mon[112590]: pgmap v53: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 511 B/s wr, 0 op/s 2024-04-12T21:03:21.601 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:03:21 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' 2024-04-12T21:03:21.601 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:03:21 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' 2024-04-12T21:03:23.645 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:03:23 smithi018 ceph-mon[112590]: pgmap v54: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 511 B/s wr, 0 op/s 2024-04-12T21:03:23.645 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:03:23 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' 2024-04-12T21:03:23.645 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:03:23 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' 2024-04-12T21:03:23.780 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:03:23 smithi186 ceph-mon[86379]: pgmap v54: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 511 B/s wr, 0 op/s 2024-04-12T21:03:23.780 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:03:23 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' 2024-04-12T21:03:23.780 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:03:23 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' 2024-04-12T21:03:25.291 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:03:25 smithi018 ceph-mon[112590]: pgmap v55: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-12T21:03:25.291 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:03:25 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' 2024-04-12T21:03:25.291 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:03:25 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' 2024-04-12T21:03:25.291 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:03:25 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T21:03:25.529 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:03:25 smithi186 ceph-mon[86379]: pgmap v55: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-12T21:03:25.530 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:03:25 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' 2024-04-12T21:03:25.530 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:03:25 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' 2024-04-12T21:03:25.530 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:03:25 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T21:03:27.377 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:03:27 smithi018 ceph-mon[112590]: pgmap v56: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-12T21:03:27.529 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:03:27 smithi186 ceph-mon[86379]: pgmap v56: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-12T21:03:29.740 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:03:29 smithi018 ceph-mon[112590]: pgmap v57: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-12T21:03:29.741 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:03:29 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' 2024-04-12T21:03:29.741 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:03:29 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' 2024-04-12T21:03:29.741 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:03:29 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-12T21:03:29.741 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:03:29 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-12T21:03:29.741 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:03:29 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' 2024-04-12T21:03:29.741 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:03:29 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-12T21:03:29.742 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:03:29 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T21:03:29.742 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:03:29 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T21:03:29.742 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:03:29 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T21:03:29.742 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:03:29 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' 2024-04-12T21:03:29.742 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:03:29 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.crash.smithi018"}]: dispatch 2024-04-12T21:03:29.742 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:03:29 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.crash.smithi018"}]': finished 2024-04-12T21:03:29.743 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:03:29 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.crash.smithi186"}]: dispatch 2024-04-12T21:03:29.743 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:03:29 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.crash.smithi186"}]': finished 2024-04-12T21:03:29.743 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:03:29 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd ok-to-stop", "ids": ["0"], "max": 16}]: dispatch 2024-04-12T21:03:29.780 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:03:29 smithi186 ceph-mon[86379]: pgmap v57: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-12T21:03:29.780 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:03:29 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' 2024-04-12T21:03:29.780 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:03:29 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' 2024-04-12T21:03:29.780 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:03:29 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-12T21:03:29.781 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:03:29 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-12T21:03:29.781 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:03:29 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' 2024-04-12T21:03:29.781 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:03:29 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-12T21:03:29.781 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:03:29 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T21:03:29.781 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:03:29 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T21:03:29.781 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:03:29 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T21:03:29.781 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:03:29 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' 2024-04-12T21:03:29.781 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:03:29 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.crash.smithi018"}]: dispatch 2024-04-12T21:03:29.781 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:03:29 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.crash.smithi018"}]': finished 2024-04-12T21:03:29.781 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:03:29 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.crash.smithi186"}]: dispatch 2024-04-12T21:03:29.781 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:03:29 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.crash.smithi186"}]': finished 2024-04-12T21:03:29.781 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:03:29 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd ok-to-stop", "ids": ["0"], "max": 16}]: dispatch 2024-04-12T21:03:30.627 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:03:30 smithi018 ceph-mon[112590]: Upgrade: Setting container_image for all crash 2024-04-12T21:03:30.627 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:03:30 smithi018 ceph-mon[112590]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["0"], "max": 16}]: dispatch 2024-04-12T21:03:30.627 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:03:30 smithi018 ceph-mon[112590]: Upgrade: osd.0 is safe to restart 2024-04-12T21:03:30.780 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:03:30 smithi186 ceph-mon[86379]: Upgrade: Setting container_image for all crash 2024-04-12T21:03:30.780 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:03:30 smithi186 ceph-mon[86379]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["0"], "max": 16}]: dispatch 2024-04-12T21:03:30.780 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:03:30 smithi186 ceph-mon[86379]: Upgrade: osd.0 is safe to restart 2024-04-12T21:03:31.780 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:03:31 smithi186 ceph-mon[86379]: pgmap v58: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-12T21:03:31.780 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:03:31 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' 2024-04-12T21:03:31.780 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:03:31 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-04-12T21:03:31.780 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:03:31 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-12T21:03:31.850 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:03:31 smithi018 ceph-mon[112590]: pgmap v58: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-12T21:03:31.850 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:03:31 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' 2024-04-12T21:03:31.851 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:03:31 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-04-12T21:03:31.851 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:03:31 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-12T21:03:32.779 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:03:32 smithi186 ceph-mon[86379]: Upgrade: Updating osd.0 2024-04-12T21:03:32.780 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:03:32 smithi186 ceph-mon[86379]: Deploying daemon osd.0 on smithi018 2024-04-12T21:03:32.788 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:03:32.790+0000 7fc639200700 1 -- 172.21.15.18:0/2475863426 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc6341037a0 msgr2=0x7fc6340fce10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:03:32.788 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:03:32.790+0000 7fc639200700 1 --2- 172.21.15.18:0/2475863426 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc6341037a0 0x7fc6340fce10 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fc620004940 tx=0x7fc620009e30 comp rx=0 tx=0).stop 2024-04-12T21:03:32.788 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:03:32.790+0000 7fc639200700 1 -- 172.21.15.18:0/2475863426 shutdown_connections 2024-04-12T21:03:32.788 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:03:32.790+0000 7fc639200700 1 --2- 172.21.15.18:0/2475863426 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc6341037a0 0x7fc6340fce10 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:03:32.788 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:03:32.790+0000 7fc639200700 1 --2- 172.21.15.18:0/2475863426 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc634102e00 0x7fc6341031d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:03:32.788 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:03:32.790+0000 7fc639200700 1 -- 172.21.15.18:0/2475863426 >> 172.21.15.18:0/2475863426 conn(0x7fc6340f89b0 msgr2=0x7fc6340fadc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:03:32.788 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:03:32 smithi018 ceph-mon[112590]: Upgrade: Updating osd.0 2024-04-12T21:03:32.788 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:03:32 smithi018 ceph-mon[112590]: Deploying daemon osd.0 on smithi018 2024-04-12T21:03:32.789 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:03:32.791+0000 7fc639200700 1 -- 172.21.15.18:0/2475863426 shutdown_connections 2024-04-12T21:03:32.789 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:03:32.791+0000 7fc639200700 1 -- 172.21.15.18:0/2475863426 wait complete. 2024-04-12T21:03:32.789 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:03:32.792+0000 7fc639200700 1 Processor -- start 2024-04-12T21:03:32.789 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:03:32.792+0000 7fc639200700 1 -- start start 2024-04-12T21:03:32.790 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:03:32.792+0000 7fc639200700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc634102e00 0x7fc634115d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:03:32.790 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:03:32.792+0000 7fc639200700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc6341037a0 0x7fc6341162d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:03:32.790 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:03:32.792+0000 7fc639200700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc634116920 con 0x7fc634102e00 2024-04-12T21:03:32.790 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:03:32.792+0000 7fc639200700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc63410fd40 con 0x7fc6341037a0 2024-04-12T21:03:32.790 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:03:32.793+0000 7fc631e00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc634102e00 0x7fc634115d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:03:32.790 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:03:32.793+0000 7fc631e00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc634102e00 0x7fc634115d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:33746/0 (socket says 172.21.15.18:33746) 2024-04-12T21:03:32.790 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:03:32.793+0000 7fc631e00700 1 -- 172.21.15.18:0/2962558872 learned_addr learned my addr 172.21.15.18:0/2962558872 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:03:32.791 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:03:32.793+0000 7fc631400700 1 --2- 172.21.15.18:0/2962558872 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc6341037a0 0x7fc6341162d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:03:32.791 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:03:32.793+0000 7fc631e00700 1 -- 172.21.15.18:0/2962558872 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc6341037a0 msgr2=0x7fc6341162d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:03:32.791 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:03:32.793+0000 7fc631e00700 1 --2- 172.21.15.18:0/2962558872 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc6341037a0 0x7fc6341162d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:03:32.791 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:03:32.794+0000 7fc631e00700 1 -- 172.21.15.18:0/2962558872 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc620005040 con 0x7fc634102e00 2024-04-12T21:03:32.791 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:03:32.794+0000 7fc631e00700 1 --2- 172.21.15.18:0/2962558872 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc634102e00 0x7fc634115d90 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fc624002a50 tx=0x7fc6240046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:03:32.792 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:03:32.794+0000 7fc62a000700 1 -- 172.21.15.18:0/2962558872 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc624007cc0 con 0x7fc634102e00 2024-04-12T21:03:32.792 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:03:32.794+0000 7fc62a000700 1 -- 172.21.15.18:0/2962558872 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc62400f040 con 0x7fc634102e00 2024-04-12T21:03:32.792 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:03:32.794+0000 7fc639200700 1 -- 172.21.15.18:0/2962558872 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc634110020 con 0x7fc634102e00 2024-04-12T21:03:32.792 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:03:32.795+0000 7fc62a000700 1 -- 172.21.15.18:0/2962558872 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc624009640 con 0x7fc634102e00 2024-04-12T21:03:32.792 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:03:32.795+0000 7fc639200700 1 -- 172.21.15.18:0/2962558872 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc6341105e0 con 0x7fc634102e00 2024-04-12T21:03:32.793 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:03:32.796+0000 7fc639200700 1 -- 172.21.15.18:0/2962558872 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc634106aa0 con 0x7fc634102e00 2024-04-12T21:03:32.795 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:03:32.798+0000 7fc62a000700 1 -- 172.21.15.18:0/2962558872 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7fc6240097a0 con 0x7fc634102e00 2024-04-12T21:03:32.797 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:03:32.799+0000 7fc62a000700 1 --2- 172.21.15.18:0/2962558872 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fc61806ec10 0x7fc6180710c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:03:32.797 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:03:32.800+0000 7fc62a000700 1 -- 172.21.15.18:0/2962558872 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6324+0+0 (secure 0 0 0) 0x7fc62408e510 con 0x7fc634102e00 2024-04-12T21:03:32.797 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:03:32.800+0000 7fc631400700 1 --2- 172.21.15.18:0/2962558872 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fc61806ec10 0x7fc6180710c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:03:32.798 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:03:32.801+0000 7fc631400700 1 --2- 172.21.15.18:0/2962558872 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fc61806ec10 0x7fc6180710c0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fc62000b020 tx=0x7fc620009e30 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:03:32.803 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:03:32.806+0000 7fc62a000700 1 -- 172.21.15.18:0/2962558872 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7fc62405af10 con 0x7fc634102e00 2024-04-12T21:03:33.066 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:03:33.068+0000 7fc639200700 1 -- 172.21.15.18:0/2962558872 --> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fc634109f50 con 0x7fc61806ec10 2024-04-12T21:03:33.067 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:03:33.070+0000 7fc62a000700 1 -- 172.21.15.18:0/2962558872 <== mgr.34102 v2:172.21.15.18:6800/3812891919 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7fc634109f50 con 0x7fc61806ec10 2024-04-12T21:03:33.071 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:03:33.073+0000 7fc639200700 1 -- 172.21.15.18:0/2962558872 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fc61806ec10 msgr2=0x7fc6180710c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:03:33.071 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:03:33.073+0000 7fc639200700 1 --2- 172.21.15.18:0/2962558872 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fc61806ec10 0x7fc6180710c0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fc62000b020 tx=0x7fc620009e30 comp rx=0 tx=0).stop 2024-04-12T21:03:33.071 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:03:33.073+0000 7fc639200700 1 -- 172.21.15.18:0/2962558872 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc634102e00 msgr2=0x7fc634115d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:03:33.071 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:03:33.073+0000 7fc639200700 1 --2- 172.21.15.18:0/2962558872 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc634102e00 0x7fc634115d90 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fc624002a50 tx=0x7fc6240046b0 comp rx=0 tx=0).stop 2024-04-12T21:03:33.071 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:03:33.073+0000 7fc639200700 1 -- 172.21.15.18:0/2962558872 shutdown_connections 2024-04-12T21:03:33.071 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:03:33.073+0000 7fc639200700 1 --2- 172.21.15.18:0/2962558872 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fc61806ec10 0x7fc6180710c0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:03:33.071 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:03:33.073+0000 7fc639200700 1 --2- 172.21.15.18:0/2962558872 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc634102e00 0x7fc634115d90 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:03:33.071 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:03:33.073+0000 7fc639200700 1 --2- 172.21.15.18:0/2962558872 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc6341037a0 0x7fc6341162d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:03:33.072 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:03:33.074+0000 7fc639200700 1 -- 172.21.15.18:0/2962558872 >> 172.21.15.18:0/2962558872 conn(0x7fc6340f89b0 msgr2=0x7fc6341073f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:03:33.072 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:03:33.074+0000 7fc639200700 1 -- 172.21.15.18:0/2962558872 shutdown_connections 2024-04-12T21:03:33.072 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:03:33.074+0000 7fc639200700 1 -- 172.21.15.18:0/2962558872 wait complete. 2024-04-12T21:03:33.086 INFO:teuthology.orchestra.run.smithi018.stdout:true 2024-04-12T21:03:33.221 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:03:33.223+0000 7f01b5400700 1 -- 172.21.15.18:0/1181947508 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f01b01059b0 msgr2=0x7f01b00ff020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:03:33.221 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:03:33.223+0000 7f01b5400700 1 --2- 172.21.15.18:0/1181947508 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f01b01059b0 0x7f01b00ff020 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f019c004940 tx=0x7f019c009e30 comp rx=0 tx=0).stop 2024-04-12T21:03:33.221 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:03:33.224+0000 7f01b5400700 1 -- 172.21.15.18:0/1181947508 shutdown_connections 2024-04-12T21:03:33.221 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:03:33.224+0000 7f01b5400700 1 --2- 172.21.15.18:0/1181947508 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f01b01059b0 0x7f01b00ff020 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:03:33.222 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:03:33.224+0000 7f01b5400700 1 --2- 172.21.15.18:0/1181947508 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f01b0105010 0x7f01b01053e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:03:33.222 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:03:33.224+0000 7f01b5400700 1 -- 172.21.15.18:0/1181947508 >> 172.21.15.18:0/1181947508 conn(0x7f01b00faba0 msgr2=0x7f01b00fcfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:03:33.222 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:03:33.224+0000 7f01b5400700 1 -- 172.21.15.18:0/1181947508 shutdown_connections 2024-04-12T21:03:33.222 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:03:33.224+0000 7f01b5400700 1 -- 172.21.15.18:0/1181947508 wait complete. 2024-04-12T21:03:33.223 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:03:33.225+0000 7f01b5400700 1 Processor -- start 2024-04-12T21:03:33.223 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:03:33.225+0000 7f01b5400700 1 -- start start 2024-04-12T21:03:33.223 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:03:33.225+0000 7f01b5400700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f01b0105010 0x7f01b0115d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:03:33.223 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:03:33.225+0000 7f01b5400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f01b01059b0 0x7f01b01162b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:03:33.224 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:03:33.225+0000 7f01b5400700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01b0116900 con 0x7f01b01059b0 2024-04-12T21:03:33.224 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:03:33.225+0000 7f01b5400700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01b010fcd0 con 0x7f01b0105010 2024-04-12T21:03:33.224 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:03:33.225+0000 7f01ade00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f01b01059b0 0x7f01b01162b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:03:33.224 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:03:33.225+0000 7f01ade00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f01b01059b0 0x7f01b01162b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:33768/0 (socket says 172.21.15.18:33768) 2024-04-12T21:03:33.224 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:03:33.225+0000 7f01ade00700 1 -- 172.21.15.18:0/2664820297 learned_addr learned my addr 172.21.15.18:0/2664820297 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:03:33.224 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:03:33.226+0000 7f01ade00700 1 -- 172.21.15.18:0/2664820297 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f01b0105010 msgr2=0x7f01b0115d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:03:33.224 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:03:33.226+0000 7f01ae800700 1 --2- 172.21.15.18:0/2664820297 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f01b0105010 0x7f01b0115d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:03:33.224 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:03:33.226+0000 7f01ade00700 1 --2- 172.21.15.18:0/2664820297 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f01b0105010 0x7f01b0115d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:03:33.224 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:03:33.226+0000 7f01ade00700 1 -- 172.21.15.18:0/2664820297 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f019c005040 con 0x7f01b01059b0 2024-04-12T21:03:33.224 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:03:33.226+0000 7f01ae800700 1 --2- 172.21.15.18:0/2664820297 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f01b0105010 0x7f01b0115d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-12T21:03:33.224 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:03:33.226+0000 7f01ade00700 1 --2- 172.21.15.18:0/2664820297 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f01b01059b0 0x7f01b01162b0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f019c009180 tx=0x7f019c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:03:33.225 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:03:33.227+0000 7f01a6a00700 1 -- 172.21.15.18:0/2664820297 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f019c00e040 con 0x7f01b01059b0 2024-04-12T21:03:33.225 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:03:33.227+0000 7f01a6a00700 1 -- 172.21.15.18:0/2664820297 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f019c00bbc0 con 0x7f01b01059b0 2024-04-12T21:03:33.225 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:03:33.227+0000 7f01a6a00700 1 -- 172.21.15.18:0/2664820297 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f019c01c3f0 con 0x7f01b01059b0 2024-04-12T21:03:33.225 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:03:33.227+0000 7f01b5400700 1 -- 172.21.15.18:0/2664820297 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f01b010ff50 con 0x7f01b01059b0 2024-04-12T21:03:33.225 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:03:33.227+0000 7f01b5400700 1 -- 172.21.15.18:0/2664820297 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f01b0110510 con 0x7f01b01059b0 2024-04-12T21:03:33.225 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:03:33.228+0000 7f01b5400700 1 -- 172.21.15.18:0/2664820297 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f01b0100760 con 0x7f01b01059b0 2024-04-12T21:03:33.226 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:03:33.229+0000 7f01a6a00700 1 -- 172.21.15.18:0/2664820297 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7f019c00bd30 con 0x7f01b01059b0 2024-04-12T21:03:33.227 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:03:33.230+0000 7f01a6a00700 1 --2- 172.21.15.18:0/2664820297 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f019406ec00 0x7f01940710b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:03:33.227 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:03:33.230+0000 7f01a6a00700 1 -- 172.21.15.18:0/2664820297 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6324+0+0 (secure 0 0 0) 0x7f019c08fa90 con 0x7f01b01059b0 2024-04-12T21:03:33.227 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:03:33.230+0000 7f01ae800700 1 --2- 172.21.15.18:0/2664820297 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f019406ec00 0x7f01940710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:03:33.228 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:03:33.231+0000 7f01ae800700 1 --2- 172.21.15.18:0/2664820297 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f019406ec00 0x7f01940710b0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f01a0007fd0 tx=0x7f01a0009040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:03:33.232 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:03:33.235+0000 7f01a6a00700 1 -- 172.21.15.18:0/2664820297 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f019c05c530 con 0x7f01b01059b0 2024-04-12T21:03:33.484 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:03:33.486+0000 7f01b5400700 1 -- 172.21.15.18:0/2664820297 --> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f01b010c160 con 0x7f019406ec00 2024-04-12T21:03:33.485 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:03:33.488+0000 7f01a6a00700 1 -- 172.21.15.18:0/2664820297 <== mgr.34102 v2:172.21.15.18:6800/3812891919 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7f01b010c160 con 0x7f019406ec00 2024-04-12T21:03:33.488 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:03:33.491+0000 7f01b5400700 1 -- 172.21.15.18:0/2664820297 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f019406ec00 msgr2=0x7f01940710b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:03:33.488 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:03:33.491+0000 7f01b5400700 1 --2- 172.21.15.18:0/2664820297 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f019406ec00 0x7f01940710b0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f01a0007fd0 tx=0x7f01a0009040 comp rx=0 tx=0).stop 2024-04-12T21:03:33.488 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:03:33.491+0000 7f01b5400700 1 -- 172.21.15.18:0/2664820297 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f01b01059b0 msgr2=0x7f01b01162b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:03:33.488 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:03:33.491+0000 7f01b5400700 1 --2- 172.21.15.18:0/2664820297 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f01b01059b0 0x7f01b01162b0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f019c009180 tx=0x7f019c009e30 comp rx=0 tx=0).stop 2024-04-12T21:03:33.488 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:03:33.491+0000 7f01b5400700 1 -- 172.21.15.18:0/2664820297 shutdown_connections 2024-04-12T21:03:33.488 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:03:33.491+0000 7f01b5400700 1 --2- 172.21.15.18:0/2664820297 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f019406ec00 0x7f01940710b0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:03:33.488 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:03:33.491+0000 7f01b5400700 1 --2- 172.21.15.18:0/2664820297 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f01b0105010 0x7f01b0115d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:03:33.489 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:03:33.491+0000 7f01b5400700 1 --2- 172.21.15.18:0/2664820297 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f01b01059b0 0x7f01b01162b0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:03:33.489 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:03:33.491+0000 7f01b5400700 1 -- 172.21.15.18:0/2664820297 >> 172.21.15.18:0/2664820297 conn(0x7f01b00faba0 msgr2=0x7f01b00fbcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:03:33.489 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:03:33.491+0000 7f01b5400700 1 -- 172.21.15.18:0/2664820297 shutdown_connections 2024-04-12T21:03:33.489 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:03:33.491+0000 7f01b5400700 1 -- 172.21.15.18:0/2664820297 wait complete. 2024-04-12T21:03:33.642 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:03:33.644+0000 7ffa03400700 1 -- 172.21.15.18:0/3710332499 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff9fc0fe830 msgr2=0x7ff9fc106f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:03:33.642 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:03:33.644+0000 7ffa03400700 1 --2- 172.21.15.18:0/3710332499 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff9fc0fe830 0x7ff9fc106f00 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7ff9e8004940 tx=0x7ff9e8009e30 comp rx=0 tx=0).stop 2024-04-12T21:03:33.642 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:03:33 smithi018 ceph-mon[112590]: pgmap v59: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-12T21:03:33.643 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:03:33.645+0000 7ffa03400700 1 -- 172.21.15.18:0/3710332499 shutdown_connections 2024-04-12T21:03:33.643 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:03:33.645+0000 7ffa03400700 1 --2- 172.21.15.18:0/3710332499 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff9fc0fe830 0x7ff9fc106f00 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:03:33.643 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:03:33.645+0000 7ffa03400700 1 --2- 172.21.15.18:0/3710332499 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff9fc0fde90 0x7ff9fc0fe260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:03:33.643 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:03:33.645+0000 7ffa03400700 1 -- 172.21.15.18:0/3710332499 >> 172.21.15.18:0/3710332499 conn(0x7ff9fc072bc0 msgr2=0x7ff9fc072fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:03:33.643 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:03:33.645+0000 7ffa03400700 1 -- 172.21.15.18:0/3710332499 shutdown_connections 2024-04-12T21:03:33.643 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:03:33.645+0000 7ffa03400700 1 -- 172.21.15.18:0/3710332499 wait complete. 2024-04-12T21:03:33.643 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:03:33.646+0000 7ffa03400700 1 Processor -- start 2024-04-12T21:03:33.643 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:03:33.646+0000 7ffa03400700 1 -- start start 2024-04-12T21:03:33.644 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:03:33.647+0000 7ffa03400700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff9fc0fde90 0x7ff9fc113ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:03:33.645 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:03:33.647+0000 7ffa03400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff9fc0fe830 0x7ff9fc1140e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:03:33.645 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:03:33.647+0000 7ffa03400700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9fc114730 con 0x7ff9fc0fe830 2024-04-12T21:03:33.645 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:03:33.647+0000 7ffa03400700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9fc10db20 con 0x7ff9fc0fde90 2024-04-12T21:03:33.645 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:03:33.647+0000 7ff9fbe00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff9fc0fe830 0x7ff9fc1140e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:03:33.645 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:03:33.647+0000 7ff9fbe00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff9fc0fe830 0x7ff9fc1140e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:33782/0 (socket says 172.21.15.18:33782) 2024-04-12T21:03:33.645 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:03:33.647+0000 7ff9fbe00700 1 -- 172.21.15.18:0/623487376 learned_addr learned my addr 172.21.15.18:0/623487376 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:03:33.645 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:03:33.647+0000 7ffa00a00700 1 --2- 172.21.15.18:0/623487376 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff9fc0fde90 0x7ff9fc113ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:03:33.645 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:03:33.647+0000 7ff9fbe00700 1 -- 172.21.15.18:0/623487376 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff9fc0fde90 msgr2=0x7ff9fc113ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:03:33.645 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:03:33.648+0000 7ff9fbe00700 1 --2- 172.21.15.18:0/623487376 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff9fc0fde90 0x7ff9fc113ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:03:33.645 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:03:33.648+0000 7ff9fbe00700 1 -- 172.21.15.18:0/623487376 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9e8005040 con 0x7ff9fc0fe830 2024-04-12T21:03:33.646 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:03:33.648+0000 7ff9fbe00700 1 --2- 172.21.15.18:0/623487376 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff9fc0fe830 0x7ff9fc1140e0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7ff9e8000c00 tx=0x7ff9e8009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:03:33.646 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:03:33.648+0000 7ff9f8c00700 1 -- 172.21.15.18:0/623487376 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff9e800e040 con 0x7ff9fc0fe830 2024-04-12T21:03:33.646 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:03:33.648+0000 7ff9f8c00700 1 -- 172.21.15.18:0/623487376 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff9e8013420 con 0x7ff9fc0fe830 2024-04-12T21:03:33.646 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:03:33.648+0000 7ffa03400700 1 -- 172.21.15.18:0/623487376 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff9fc10dda0 con 0x7ff9fc0fe830 2024-04-12T21:03:33.647 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:03:33.648+0000 7ff9f8c00700 1 -- 172.21.15.18:0/623487376 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff9e801c610 con 0x7ff9fc0fe830 2024-04-12T21:03:33.647 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:03:33.648+0000 7ffa03400700 1 -- 172.21.15.18:0/623487376 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff9fc10e2e0 con 0x7ff9fc0fe830 2024-04-12T21:03:33.647 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:03:33.650+0000 7ffa03400700 1 -- 172.21.15.18:0/623487376 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff9fc065e30 con 0x7ff9fc0fe830 2024-04-12T21:03:33.648 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:03:33.651+0000 7ff9f8c00700 1 -- 172.21.15.18:0/623487376 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7ff9e8013590 con 0x7ff9fc0fe830 2024-04-12T21:03:33.650 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:03:33.653+0000 7ff9f8c00700 1 --2- 172.21.15.18:0/623487376 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7ff9e406ec10 0x7ff9e40710c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:03:33.650 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:03:33.653+0000 7ff9f8c00700 1 -- 172.21.15.18:0/623487376 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6324+0+0 (secure 0 0 0) 0x7ff9e808fab0 con 0x7ff9fc0fe830 2024-04-12T21:03:33.650 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:03:33.653+0000 7ffa00a00700 1 --2- 172.21.15.18:0/623487376 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7ff9e406ec10 0x7ff9e40710c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:03:33.651 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:03:33.654+0000 7ffa00a00700 1 --2- 172.21.15.18:0/623487376 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7ff9e406ec10 0x7ff9e40710c0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7ff9f0007fd0 tx=0x7ff9f0009040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:03:33.655 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:03:33.658+0000 7ff9f8c00700 1 -- 172.21.15.18:0/623487376 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7ff9e805c430 con 0x7ff9fc0fe830 2024-04-12T21:03:33.779 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:03:33 smithi186 ceph-mon[86379]: pgmap v59: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-12T21:03:33.913 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:03:33.916+0000 7ffa03400700 1 -- 172.21.15.18:0/623487376 --> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7ff9fc109f10 con 0x7ff9e406ec10 2024-04-12T21:03:33.938 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:03:33.941+0000 7ff9f8c00700 1 -- 172.21.15.18:0/623487376 <== mgr.34102 v2:172.21.15.18:6800/3812891919 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7ff9fc109f10 con 0x7ff9e406ec10 2024-04-12T21:03:33.938 INFO:teuthology.orchestra.run.smithi018.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-12T21:03:33.938 INFO:teuthology.orchestra.run.smithi018.stdout:alertmanager.smithi018 smithi018 *:9093,9094 running (4m) 13s ago 18m 15.2M - 0.25.0 c8568f914cd2 c332090068e8 2024-04-12T21:03:33.938 INFO:teuthology.orchestra.run.smithi018.stdout:crash.smithi018 smithi018 running (29s) 13s ago 18m 6665k - 19.0.0-2879-g324baf8d 8717720dc5e5 b7f80d1f7982 2024-04-12T21:03:33.939 INFO:teuthology.orchestra.run.smithi018.stdout:crash.smithi186 smithi186 running (21s) 17s ago 17m 6660k - 19.0.0-2879-g324baf8d 8717720dc5e5 a4f55c81b623 2024-04-12T21:03:33.939 INFO:teuthology.orchestra.run.smithi018.stdout:grafana.smithi018 smithi018 *:3000 running (3m) 13s ago 18m 73.5M - 9.4.12 2bacad6d85d8 b08e3bf176dd 2024-04-12T21:03:33.939 INFO:teuthology.orchestra.run.smithi018.stdout:mds.cephfs.smithi018.acqbsc smithi018 running (11m) 13s ago 11m 112M - 17.2.7-827-gd3b5060a 21de9b7369ad 5b5fce65f18a 2024-04-12T21:03:33.939 INFO:teuthology.orchestra.run.smithi018.stdout:mds.cephfs.smithi018.szjluo smithi018 running (11m) 13s ago 11m 113M - 17.2.7-827-gd3b5060a 21de9b7369ad cf9bf0c5031f 2024-04-12T21:03:33.939 INFO:teuthology.orchestra.run.smithi018.stdout:mds.cephfs.smithi186.jdllyn smithi186 running (11m) 17s ago 11m 14.7M - 17.2.7-827-gd3b5060a 21de9b7369ad 2b2bc46bd29a 2024-04-12T21:03:33.939 INFO:teuthology.orchestra.run.smithi018.stdout:mds.cephfs.smithi186.wjnafs smithi186 running (11m) 17s ago 11m 14.4M - 17.2.7-827-gd3b5060a 21de9b7369ad 74b115d6d6fd 2024-04-12T21:03:33.939 INFO:teuthology.orchestra.run.smithi018.stdout:mgr.smithi018.xjzwga smithi018 *:8443,9283,8765 running (6m) 13s ago 20m 500M - 19.0.0-2879-g324baf8d 8717720dc5e5 56088305eab7 2024-04-12T21:03:33.939 INFO:teuthology.orchestra.run.smithi018.stdout:mgr.smithi186.kqcfxk smithi186 *:8443,9283,8765 running (5m) 17s ago 17m 450M - 19.0.0-2879-g324baf8d 8717720dc5e5 09cd766e5e0d 2024-04-12T21:03:33.939 INFO:teuthology.orchestra.run.smithi018.stdout:mon.smithi018 smithi018 running (2m) 13s ago 20m 45.3M 2048M 19.0.0-2879-g324baf8d 8717720dc5e5 df3e201dfed6 2024-04-12T21:03:33.939 INFO:teuthology.orchestra.run.smithi018.stdout:mon.smithi186 smithi186 running (87s) 17s ago 17m 31.6M 2048M 19.0.0-2879-g324baf8d 8717720dc5e5 c77498da9b2b 2024-04-12T21:03:33.939 INFO:teuthology.orchestra.run.smithi018.stdout:node-exporter.smithi018 smithi018 *:9100 running (5m) 13s ago 18m 7398k - 1.5.0 0da6a335fe13 326c3d9c6847 2024-04-12T21:03:33.939 INFO:teuthology.orchestra.run.smithi018.stdout:node-exporter.smithi186 smithi186 *:9100 running (5m) 17s ago 17m 7818k - 1.5.0 0da6a335fe13 ec2112baeecf 2024-04-12T21:03:33.939 INFO:teuthology.orchestra.run.smithi018.stdout:osd.0 smithi018 running (16m) 13s ago 16m 226M 1559M 17.2.7-827-gd3b5060a 21de9b7369ad 0d83e027bf0a 2024-04-12T21:03:33.939 INFO:teuthology.orchestra.run.smithi018.stdout:osd.1 smithi018 running (15m) 13s ago 15m 240M 1559M 17.2.7-827-gd3b5060a 21de9b7369ad 6517c9cc7f87 2024-04-12T21:03:33.939 INFO:teuthology.orchestra.run.smithi018.stdout:osd.2 smithi018 running (15m) 13s ago 15m 194M 1559M 17.2.7-827-gd3b5060a 21de9b7369ad 8a83692acaa5 2024-04-12T21:03:33.939 INFO:teuthology.orchestra.run.smithi018.stdout:osd.3 smithi186 running (14m) 17s ago 14m 264M 2581M 17.2.7-827-gd3b5060a 21de9b7369ad c2def7a90d76 2024-04-12T21:03:33.940 INFO:teuthology.orchestra.run.smithi018.stdout:osd.4 smithi186 running (13m) 17s ago 13m 233M 2581M 17.2.7-827-gd3b5060a 21de9b7369ad 757e7c71b7b9 2024-04-12T21:03:33.940 INFO:teuthology.orchestra.run.smithi018.stdout:osd.5 smithi186 running (13m) 17s ago 13m 228M 2581M 17.2.7-827-gd3b5060a 21de9b7369ad 2ee072a436d0 2024-04-12T21:03:33.940 INFO:teuthology.orchestra.run.smithi018.stdout:prometheus.smithi018 smithi018 *:9095 running (4m) 13s ago 17m 58.0M - 2.43.0 a07b618ecd1d f58841127e21 2024-04-12T21:03:33.941 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:03:33.943+0000 7ffa03400700 1 -- 172.21.15.18:0/623487376 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7ff9e406ec10 msgr2=0x7ff9e40710c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:03:33.941 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:03:33.943+0000 7ffa03400700 1 --2- 172.21.15.18:0/623487376 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7ff9e406ec10 0x7ff9e40710c0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7ff9f0007fd0 tx=0x7ff9f0009040 comp rx=0 tx=0).stop 2024-04-12T21:03:33.941 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:03:33.943+0000 7ffa03400700 1 -- 172.21.15.18:0/623487376 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff9fc0fe830 msgr2=0x7ff9fc1140e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:03:33.941 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:03:33.943+0000 7ffa03400700 1 --2- 172.21.15.18:0/623487376 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff9fc0fe830 0x7ff9fc1140e0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7ff9e8000c00 tx=0x7ff9e8009e30 comp rx=0 tx=0).stop 2024-04-12T21:03:33.941 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:03:33.944+0000 7ffa03400700 1 -- 172.21.15.18:0/623487376 shutdown_connections 2024-04-12T21:03:33.941 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:03:33.944+0000 7ffa03400700 1 --2- 172.21.15.18:0/623487376 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7ff9e406ec10 0x7ff9e40710c0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:03:33.941 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:03:33.944+0000 7ffa03400700 1 --2- 172.21.15.18:0/623487376 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff9fc0fde90 0x7ff9fc113ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:03:33.941 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:03:33.944+0000 7ffa03400700 1 --2- 172.21.15.18:0/623487376 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff9fc0fe830 0x7ff9fc1140e0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:03:33.941 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:03:33.944+0000 7ffa03400700 1 -- 172.21.15.18:0/623487376 >> 172.21.15.18:0/623487376 conn(0x7ff9fc072bc0 msgr2=0x7ff9fc1037b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:03:33.941 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:03:33.944+0000 7ffa03400700 1 -- 172.21.15.18:0/623487376 shutdown_connections 2024-04-12T21:03:33.942 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:03:33.944+0000 7ffa03400700 1 -- 172.21.15.18:0/623487376 wait complete. 2024-04-12T21:03:34.098 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:03:34.100+0000 7f1132e00700 1 -- 172.21.15.18:0/1345896160 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f112c0fe830 msgr2=0x7f112c106f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:03:34.098 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:03:34.100+0000 7f1132e00700 1 --2- 172.21.15.18:0/1345896160 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f112c0fe830 0x7f112c106f00 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f1118004940 tx=0x7f1118009e30 comp rx=0 tx=0).stop 2024-04-12T21:03:34.098 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:03:34.101+0000 7f1132e00700 1 -- 172.21.15.18:0/1345896160 shutdown_connections 2024-04-12T21:03:34.098 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:03:34.101+0000 7f1132e00700 1 --2- 172.21.15.18:0/1345896160 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f112c0fe830 0x7f112c106f00 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:03:34.098 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:03:34.101+0000 7f1132e00700 1 --2- 172.21.15.18:0/1345896160 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f112c0fde90 0x7f112c0fe260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:03:34.098 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:03:34.101+0000 7f1132e00700 1 -- 172.21.15.18:0/1345896160 >> 172.21.15.18:0/1345896160 conn(0x7f112c072bc0 msgr2=0x7f112c072fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:03:34.099 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:03:34.101+0000 7f1132e00700 1 -- 172.21.15.18:0/1345896160 shutdown_connections 2024-04-12T21:03:34.099 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:03:34.101+0000 7f1132e00700 1 -- 172.21.15.18:0/1345896160 wait complete. 2024-04-12T21:03:34.100 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:03:34.102+0000 7f1132e00700 1 Processor -- start 2024-04-12T21:03:34.100 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:03:34.102+0000 7f1132e00700 1 -- start start 2024-04-12T21:03:34.100 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:03:34.102+0000 7f1132e00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f112c0fde90 0x7f112c115db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:03:34.100 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:03:34.102+0000 7f1132e00700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f112c0fe830 0x7f112c1162f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:03:34.100 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:03:34.102+0000 7f1132e00700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f112c116940 con 0x7f112c0fde90 2024-04-12T21:03:34.100 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:03:34.102+0000 7f1132e00700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f112c10fd10 con 0x7f112c0fe830 2024-04-12T21:03:34.100 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:03:34.103+0000 7f112be00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f112c0fde90 0x7f112c115db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:03:34.100 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:03:34.103+0000 7f112be00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f112c0fde90 0x7f112c115db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:33794/0 (socket says 172.21.15.18:33794) 2024-04-12T21:03:34.100 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:03:34.103+0000 7f112be00700 1 -- 172.21.15.18:0/1803524158 learned_addr learned my addr 172.21.15.18:0/1803524158 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:03:34.101 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:03:34.103+0000 7f112b400700 1 --2- 172.21.15.18:0/1803524158 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f112c0fe830 0x7f112c1162f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:03:34.101 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:03:34.103+0000 7f112be00700 1 -- 172.21.15.18:0/1803524158 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f112c0fe830 msgr2=0x7f112c1162f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:03:34.101 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:03:34.103+0000 7f112be00700 1 --2- 172.21.15.18:0/1803524158 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f112c0fe830 0x7f112c1162f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:03:34.102 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:03:34.103+0000 7f112be00700 1 -- 172.21.15.18:0/1803524158 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1118005040 con 0x7f112c0fde90 2024-04-12T21:03:34.102 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:03:34.103+0000 7f112be00700 1 --2- 172.21.15.18:0/1803524158 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f112c0fde90 0x7f112c115db0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f111c002a50 tx=0x7f111c0046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:03:34.102 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:03:34.104+0000 7f1123e00700 1 -- 172.21.15.18:0/1803524158 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f111c007cc0 con 0x7f112c0fde90 2024-04-12T21:03:34.102 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:03:34.104+0000 7f1123e00700 1 -- 172.21.15.18:0/1803524158 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f111c00f040 con 0x7f112c0fde90 2024-04-12T21:03:34.102 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:03:34.104+0000 7f1123e00700 1 -- 172.21.15.18:0/1803524158 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f111c00db70 con 0x7f112c0fde90 2024-04-12T21:03:34.102 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:03:34.104+0000 7f1132e00700 1 -- 172.21.15.18:0/1803524158 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f112c10fff0 con 0x7f112c0fde90 2024-04-12T21:03:34.102 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:03:34.104+0000 7f1132e00700 1 -- 172.21.15.18:0/1803524158 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f112c110530 con 0x7f112c0fde90 2024-04-12T21:03:34.102 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:03:34.105+0000 7f1132e00700 1 -- 172.21.15.18:0/1803524158 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f112c106a20 con 0x7f112c0fde90 2024-04-12T21:03:34.103 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:03:34.106+0000 7f1123e00700 1 -- 172.21.15.18:0/1803524158 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7f111c00dd10 con 0x7f112c0fde90 2024-04-12T21:03:34.104 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:03:34.107+0000 7f1123e00700 1 --2- 172.21.15.18:0/1803524158 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f111006ec00 0x7f11100710b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:03:34.104 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:03:34.107+0000 7f1123e00700 1 -- 172.21.15.18:0/1803524158 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6324+0+0 (secure 0 0 0) 0x7f111c08dfa0 con 0x7f112c0fde90 2024-04-12T21:03:34.105 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:03:34.108+0000 7f112b400700 1 --2- 172.21.15.18:0/1803524158 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f111006ec00 0x7f11100710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:03:34.105 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:03:34.108+0000 7f112b400700 1 --2- 172.21.15.18:0/1803524158 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f111006ec00 0x7f11100710b0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f1118000c00 tx=0x7f1118009e30 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:03:34.109 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:03:34.112+0000 7f1123e00700 1 -- 172.21.15.18:0/1803524158 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f111c05a880 con 0x7f112c0fde90 2024-04-12T21:03:34.430 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:03:34.432+0000 7f1132e00700 1 -- 172.21.15.18:0/1803524158 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f112c109f30 con 0x7f112c0fde90 2024-04-12T21:03:34.430 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:03:34.433+0000 7f1123e00700 1 -- 172.21.15.18:0/1803524158 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+736 (secure 0 0 0) 0x7f111c05a410 con 0x7f112c0fde90 2024-04-12T21:03:34.431 INFO:teuthology.orchestra.run.smithi018.stdout:{ 2024-04-12T21:03:34.431 INFO:teuthology.orchestra.run.smithi018.stdout: "mon": { 2024-04-12T21:03:34.431 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 19.0.0-2879-g324baf8d (324baf8ddb91dea46cac50cb535c306115791355) squid (dev)": 2 2024-04-12T21:03:34.431 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-12T21:03:34.431 INFO:teuthology.orchestra.run.smithi018.stdout: "mgr": { 2024-04-12T21:03:34.431 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 19.0.0-2879-g324baf8d (324baf8ddb91dea46cac50cb535c306115791355) squid (dev)": 2 2024-04-12T21:03:34.431 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-12T21:03:34.431 INFO:teuthology.orchestra.run.smithi018.stdout: "osd": { 2024-04-12T21:03:34.431 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 17.2.7-827-gd3b5060a (d3b5060a17449edfae9385cdf792afec109fca5d) quincy (stable)": 6 2024-04-12T21:03:34.431 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-12T21:03:34.431 INFO:teuthology.orchestra.run.smithi018.stdout: "mds": { 2024-04-12T21:03:34.431 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 17.2.7-827-gd3b5060a (d3b5060a17449edfae9385cdf792afec109fca5d) quincy (stable)": 4 2024-04-12T21:03:34.431 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-12T21:03:34.431 INFO:teuthology.orchestra.run.smithi018.stdout: "overall": { 2024-04-12T21:03:34.432 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 17.2.7-827-gd3b5060a (d3b5060a17449edfae9385cdf792afec109fca5d) quincy (stable)": 10, 2024-04-12T21:03:34.432 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 19.0.0-2879-g324baf8d (324baf8ddb91dea46cac50cb535c306115791355) squid (dev)": 4 2024-04-12T21:03:34.432 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-04-12T21:03:34.432 INFO:teuthology.orchestra.run.smithi018.stdout:} 2024-04-12T21:03:34.433 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:03:34.436+0000 7f1132e00700 1 -- 172.21.15.18:0/1803524158 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f111006ec00 msgr2=0x7f11100710b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:03:34.433 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:03:34.436+0000 7f1132e00700 1 --2- 172.21.15.18:0/1803524158 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f111006ec00 0x7f11100710b0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f1118000c00 tx=0x7f1118009e30 comp rx=0 tx=0).stop 2024-04-12T21:03:34.434 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:03:34.436+0000 7f1132e00700 1 -- 172.21.15.18:0/1803524158 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f112c0fde90 msgr2=0x7f112c115db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:03:34.434 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:03:34.436+0000 7f1132e00700 1 --2- 172.21.15.18:0/1803524158 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f112c0fde90 0x7f112c115db0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f111c002a50 tx=0x7f111c0046b0 comp rx=0 tx=0).stop 2024-04-12T21:03:34.434 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:03:34.436+0000 7f1132e00700 1 -- 172.21.15.18:0/1803524158 shutdown_connections 2024-04-12T21:03:34.434 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:03:34.436+0000 7f1132e00700 1 --2- 172.21.15.18:0/1803524158 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f111006ec00 0x7f11100710b0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:03:34.435 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:03:34.436+0000 7f1132e00700 1 --2- 172.21.15.18:0/1803524158 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f112c0fde90 0x7f112c115db0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:03:34.435 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:03:34.436+0000 7f1132e00700 1 --2- 172.21.15.18:0/1803524158 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f112c0fe830 0x7f112c1162f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:03:34.435 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:03:34.436+0000 7f1132e00700 1 -- 172.21.15.18:0/1803524158 >> 172.21.15.18:0/1803524158 conn(0x7f112c072bc0 msgr2=0x7f112c0fb730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:03:34.435 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:03:34.437+0000 7f1132e00700 1 -- 172.21.15.18:0/1803524158 shutdown_connections 2024-04-12T21:03:34.435 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:03:34.437+0000 7f1132e00700 1 -- 172.21.15.18:0/1803524158 wait complete. 2024-04-12T21:03:34.583 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:03:34.585+0000 7f1fa7200700 1 -- 172.21.15.18:0/689195287 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1fa81092a0 msgr2=0x7f1fa8109670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:03:34.583 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:03:34.585+0000 7f1fa7200700 1 --2- 172.21.15.18:0/689195287 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1fa81092a0 0x7f1fa8109670 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f1f98004940 tx=0x7f1f98009e30 comp rx=0 tx=0).stop 2024-04-12T21:03:34.583 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:03:34.585+0000 7f1fa7200700 1 -- 172.21.15.18:0/689195287 shutdown_connections 2024-04-12T21:03:34.583 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:03:34.585+0000 7f1fa7200700 1 --2- 172.21.15.18:0/689195287 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1fa8109c40 0x7f1fa81032b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:03:34.583 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:03:34.585+0000 7f1fa7200700 1 --2- 172.21.15.18:0/689195287 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1fa81092a0 0x7f1fa8109670 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:03:34.583 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:03:34.585+0000 7f1fa7200700 1 -- 172.21.15.18:0/689195287 >> 172.21.15.18:0/689195287 conn(0x7f1fa8072a30 msgr2=0x7f1fa8074e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:03:34.583 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:03:34.586+0000 7f1fa7200700 1 -- 172.21.15.18:0/689195287 shutdown_connections 2024-04-12T21:03:34.583 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:03:34.586+0000 7f1fa7200700 1 -- 172.21.15.18:0/689195287 wait complete. 2024-04-12T21:03:34.584 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:03:34.586+0000 7f1fa7200700 1 Processor -- start 2024-04-12T21:03:34.584 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:03:34.587+0000 7f1fa7200700 1 -- start start 2024-04-12T21:03:34.584 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:03:34.587+0000 7f1fa7200700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1fa81092a0 0x7f1fa8119ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:03:34.584 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:03:34.587+0000 7f1fa7200700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1fa8109c40 0x7f1fa811a530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:03:34.584 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:03:34.587+0000 7f1fa7200700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1fa811ab80 con 0x7f1fa8109c40 2024-04-12T21:03:34.584 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:03:34.587+0000 7f1fa7200700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1fa8113f50 con 0x7f1fa81092a0 2024-04-12T21:03:34.585 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:03:34.587+0000 7f1fa5400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1fa8109c40 0x7f1fa811a530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:03:34.585 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:03:34.587+0000 7f1fa5400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1fa8109c40 0x7f1fa811a530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:33808/0 (socket says 172.21.15.18:33808) 2024-04-12T21:03:34.586 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:03:34.587+0000 7f1fa5400700 1 -- 172.21.15.18:0/1239688311 learned_addr learned my addr 172.21.15.18:0/1239688311 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:03:34.586 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:03:34.588+0000 7f1fa5e00700 1 --2- 172.21.15.18:0/1239688311 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1fa81092a0 0x7f1fa8119ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:03:34.586 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:03:34.588+0000 7f1fa5400700 1 -- 172.21.15.18:0/1239688311 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1fa81092a0 msgr2=0x7f1fa8119ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:03:34.586 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:03:34.588+0000 7f1fa5400700 1 --2- 172.21.15.18:0/1239688311 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1fa81092a0 0x7f1fa8119ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:03:34.586 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:03:34.588+0000 7f1fa5400700 1 -- 172.21.15.18:0/1239688311 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f98005040 con 0x7f1fa8109c40 2024-04-12T21:03:34.586 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:03:34.588+0000 7f1fa5400700 1 --2- 172.21.15.18:0/1239688311 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1fa8109c40 0x7f1fa811a530 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f1f9400c800 tx=0x7f1f94004470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:03:34.586 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:03:34.588+0000 7f1f9e000700 1 -- 172.21.15.18:0/1239688311 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1f9400a760 con 0x7f1fa8109c40 2024-04-12T21:03:34.586 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:03:34.588+0000 7f1f9e000700 1 -- 172.21.15.18:0/1239688311 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f1f9400ad90 con 0x7f1fa8109c40 2024-04-12T21:03:34.586 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:03:34.588+0000 7f1fa7200700 1 -- 172.21.15.18:0/1239688311 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1fa8114230 con 0x7f1fa8109c40 2024-04-12T21:03:34.586 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:03:34.588+0000 7f1f9e000700 1 -- 172.21.15.18:0/1239688311 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1f940113a0 con 0x7f1fa8109c40 2024-04-12T21:03:34.586 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:03:34.588+0000 7f1fa7200700 1 -- 172.21.15.18:0/1239688311 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1fa81147f0 con 0x7f1fa8109c40 2024-04-12T21:03:34.587 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:03:34.590+0000 7f1fa7200700 1 -- 172.21.15.18:0/1239688311 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1fa81049f0 con 0x7f1fa8109c40 2024-04-12T21:03:34.588 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:03:34.591+0000 7f1f9e000700 1 -- 172.21.15.18:0/1239688311 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7f1f94012400 con 0x7f1fa8109c40 2024-04-12T21:03:34.589 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:03:34.592+0000 7f1f9e000700 1 --2- 172.21.15.18:0/1239688311 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f1f8c06ec00 0x7f1f8c0710b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:03:34.589 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:03:34.592+0000 7f1f9e000700 1 -- 172.21.15.18:0/1239688311 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6324+0+0 (secure 0 0 0) 0x7f1f9408e7e0 con 0x7f1fa8109c40 2024-04-12T21:03:34.589 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:03:34.592+0000 7f1fa5e00700 1 --2- 172.21.15.18:0/1239688311 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f1f8c06ec00 0x7f1f8c0710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:03:34.590 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:03:34.593+0000 7f1fa5e00700 1 --2- 172.21.15.18:0/1239688311 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f1f8c06ec00 0x7f1f8c0710b0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f1f9800b3b0 tx=0x7f1f98009e30 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:03:34.594 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:03:34.597+0000 7f1f9e000700 1 -- 172.21.15.18:0/1239688311 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f1f9405b1c0 con 0x7f1fa8109c40 2024-04-12T21:03:34.751 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:03:34 smithi018 ceph-mon[112590]: from='client.34216 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:03:34.751 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:03:34 smithi018 ceph-mon[112590]: from='client.34220 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:03:34.779 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:03:34 smithi186 ceph-mon[86379]: from='client.34216 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:03:34.779 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:03:34 smithi186 ceph-mon[86379]: from='client.34220 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:03:34.871 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:03:34.874+0000 7f1fa7200700 1 -- 172.21.15.18:0/1239688311 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f1fa8110390 con 0x7f1fa8109c40 2024-04-12T21:03:34.872 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:03:34.875+0000 7f1f9e000700 1 -- 172.21.15.18:0/1239688311 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 23 v23) v1 ==== 76+0+1886 (secure 0 0 0) 0x7f1f9405ad50 con 0x7f1fa8109c40 2024-04-12T21:03:34.873 INFO:teuthology.orchestra.run.smithi018.stdout:e23 2024-04-12T21:03:34.873 INFO:teuthology.orchestra.run.smithi018.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-12T21:03:34.873 INFO:teuthology.orchestra.run.smithi018.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-12T21:03:34.873 INFO:teuthology.orchestra.run.smithi018.stdout:legacy client fscid: 1 2024-04-12T21:03:34.873 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-12T21:03:34.873 INFO:teuthology.orchestra.run.smithi018.stdout:Filesystem 'cephfs' (1) 2024-04-12T21:03:34.873 INFO:teuthology.orchestra.run.smithi018.stdout:fs_name cephfs 2024-04-12T21:03:34.873 INFO:teuthology.orchestra.run.smithi018.stdout:epoch 23 2024-04-12T21:03:34.873 INFO:teuthology.orchestra.run.smithi018.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-04-12T21:03:34.873 INFO:teuthology.orchestra.run.smithi018.stdout:created 2024-04-12T20:51:57.362850+0000 2024-04-12T21:03:34.873 INFO:teuthology.orchestra.run.smithi018.stdout:modified 2024-04-12T21:01:10.670287+0000 2024-04-12T21:03:34.873 INFO:teuthology.orchestra.run.smithi018.stdout:tableserver 0 2024-04-12T21:03:34.873 INFO:teuthology.orchestra.run.smithi018.stdout:root 0 2024-04-12T21:03:34.873 INFO:teuthology.orchestra.run.smithi018.stdout:session_timeout 60 2024-04-12T21:03:34.874 INFO:teuthology.orchestra.run.smithi018.stdout:session_autoclose 300 2024-04-12T21:03:34.874 INFO:teuthology.orchestra.run.smithi018.stdout:max_file_size 1099511627776 2024-04-12T21:03:34.874 INFO:teuthology.orchestra.run.smithi018.stdout:max_xattr_size 65536 2024-04-12T21:03:34.874 INFO:teuthology.orchestra.run.smithi018.stdout:required_client_features {} 2024-04-12T21:03:34.874 INFO:teuthology.orchestra.run.smithi018.stdout:last_failure 0 2024-04-12T21:03:34.874 INFO:teuthology.orchestra.run.smithi018.stdout:last_failure_osd_epoch 46 2024-04-12T21:03:34.874 INFO:teuthology.orchestra.run.smithi018.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-12T21:03:34.874 INFO:teuthology.orchestra.run.smithi018.stdout:max_mds 2 2024-04-12T21:03:34.874 INFO:teuthology.orchestra.run.smithi018.stdout:in 0,1 2024-04-12T21:03:34.874 INFO:teuthology.orchestra.run.smithi018.stdout:up {0=14452,1=14464} 2024-04-12T21:03:34.874 INFO:teuthology.orchestra.run.smithi018.stdout:failed 2024-04-12T21:03:34.874 INFO:teuthology.orchestra.run.smithi018.stdout:damaged 2024-04-12T21:03:34.874 INFO:teuthology.orchestra.run.smithi018.stdout:stopped 2024-04-12T21:03:34.874 INFO:teuthology.orchestra.run.smithi018.stdout:data_pools [3] 2024-04-12T21:03:34.875 INFO:teuthology.orchestra.run.smithi018.stdout:metadata_pool 2 2024-04-12T21:03:34.875 INFO:teuthology.orchestra.run.smithi018.stdout:inline_data disabled 2024-04-12T21:03:34.875 INFO:teuthology.orchestra.run.smithi018.stdout:balancer 2024-04-12T21:03:34.875 INFO:teuthology.orchestra.run.smithi018.stdout:bal_rank_mask -1 2024-04-12T21:03:34.875 INFO:teuthology.orchestra.run.smithi018.stdout:standby_count_wanted 1 2024-04-12T21:03:34.875 INFO:teuthology.orchestra.run.smithi018.stdout:qdb_cluster leader: 0 members: 2024-04-12T21:03:34.875 INFO:teuthology.orchestra.run.smithi018.stdout:[mds.cephfs.smithi018.acqbsc{0:14452} state up:active seq 5 join_fscid=1 addr [v2:172.21.15.18:6826/1564065434,v1:172.21.15.18:6827/1564065434] compat {c=[1],r=[1],i=[7ff]}] 2024-04-12T21:03:34.875 INFO:teuthology.orchestra.run.smithi018.stdout:[mds.cephfs.smithi018.szjluo{1:14464} state up:active seq 6 join_fscid=1 addr [v2:172.21.15.18:6828/722443353,v1:172.21.15.18:6829/722443353] compat {c=[1],r=[1],i=[7ff]}] 2024-04-12T21:03:34.875 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-12T21:03:34.875 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-12T21:03:34.875 INFO:teuthology.orchestra.run.smithi018.stdout:Standby daemons: 2024-04-12T21:03:34.875 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-12T21:03:34.875 INFO:teuthology.orchestra.run.smithi018.stdout:[mds.cephfs.smithi186.wjnafs{-1:24277} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.186:6826/438347207,v1:172.21.15.186:6827/438347207] compat {c=[1],r=[1],i=[7ff]}] 2024-04-12T21:03:34.875 INFO:teuthology.orchestra.run.smithi018.stdout:[mds.cephfs.smithi186.jdllyn{-1:24281} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.186:6824/1464359952,v1:172.21.15.186:6825/1464359952] compat {c=[1],r=[1],i=[7ff]}] 2024-04-12T21:03:34.876 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:03:34.878+0000 7f1fa7200700 1 -- 172.21.15.18:0/1239688311 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f1f8c06ec00 msgr2=0x7f1f8c0710b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:03:34.876 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:03:34.878+0000 7f1fa7200700 1 --2- 172.21.15.18:0/1239688311 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f1f8c06ec00 0x7f1f8c0710b0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f1f9800b3b0 tx=0x7f1f98009e30 comp rx=0 tx=0).stop 2024-04-12T21:03:34.876 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:03:34.878+0000 7f1fa7200700 1 -- 172.21.15.18:0/1239688311 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1fa8109c40 msgr2=0x7f1fa811a530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:03:34.876 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:03:34.878+0000 7f1fa7200700 1 --2- 172.21.15.18:0/1239688311 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1fa8109c40 0x7f1fa811a530 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f1f9400c800 tx=0x7f1f94004470 comp rx=0 tx=0).stop 2024-04-12T21:03:34.877 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:03:34.878+0000 7f1fa7200700 1 -- 172.21.15.18:0/1239688311 shutdown_connections 2024-04-12T21:03:34.877 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:03:34.878+0000 7f1fa7200700 1 --2- 172.21.15.18:0/1239688311 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f1f8c06ec00 0x7f1f8c0710b0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:03:34.877 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:03:34.878+0000 7f1fa7200700 1 --2- 172.21.15.18:0/1239688311 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1fa81092a0 0x7f1fa8119ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:03:34.877 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:03:34.878+0000 7f1fa7200700 1 --2- 172.21.15.18:0/1239688311 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1fa8109c40 0x7f1fa811a530 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:03:34.877 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:03:34.878+0000 7f1fa7200700 1 -- 172.21.15.18:0/1239688311 >> 172.21.15.18:0/1239688311 conn(0x7f1fa8072a30 msgr2=0x7f1fa80748d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:03:34.877 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:03:34.879+0000 7f1fa7200700 1 -- 172.21.15.18:0/1239688311 shutdown_connections 2024-04-12T21:03:34.877 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:03:34.879+0000 7f1fa7200700 1 -- 172.21.15.18:0/1239688311 wait complete. 2024-04-12T21:03:34.877 INFO:teuthology.orchestra.run.smithi018.stderr:dumped fsmap epoch 23 2024-04-12T21:03:35.028 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:03:35.030+0000 7fced7000700 1 -- 172.21.15.18:0/3019209153 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fced00fe830 msgr2=0x7fced0106f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:03:35.028 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:03:35.030+0000 7fced7000700 1 --2- 172.21.15.18:0/3019209153 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fced00fe830 0x7fced0106f00 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fceb8004940 tx=0x7fceb8009e30 comp rx=0 tx=0).stop 2024-04-12T21:03:35.028 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:03:35.030+0000 7fced7000700 1 -- 172.21.15.18:0/3019209153 shutdown_connections 2024-04-12T21:03:35.028 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:03:35.030+0000 7fced7000700 1 --2- 172.21.15.18:0/3019209153 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fced00fe830 0x7fced0106f00 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:03:35.028 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:03:35.030+0000 7fced7000700 1 --2- 172.21.15.18:0/3019209153 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fced00fde90 0x7fced00fe260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:03:35.028 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:03:35.030+0000 7fced7000700 1 -- 172.21.15.18:0/3019209153 >> 172.21.15.18:0/3019209153 conn(0x7fced0072bc0 msgr2=0x7fced0072fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:03:35.028 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:03:35.030+0000 7fced7000700 1 -- 172.21.15.18:0/3019209153 shutdown_connections 2024-04-12T21:03:35.028 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:03:35.030+0000 7fced7000700 1 -- 172.21.15.18:0/3019209153 wait complete. 2024-04-12T21:03:35.029 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:03:35.031+0000 7fced7000700 1 Processor -- start 2024-04-12T21:03:35.029 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:03:35.031+0000 7fced7000700 1 -- start start 2024-04-12T21:03:35.029 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:03:35.032+0000 7fced7000700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fced00fde90 0x7fced0115d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:03:35.029 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:03:35.032+0000 7fced7000700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fced00fe830 0x7fced01162c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:03:35.029 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:03:35.032+0000 7fced7000700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fced01169a0 con 0x7fced00fe830 2024-04-12T21:03:35.030 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:03:35.032+0000 7fced7000700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fced010fce0 con 0x7fced00fde90 2024-04-12T21:03:35.030 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:03:35.032+0000 7fcec7e00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fced00fe830 0x7fced01162c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:03:35.030 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:03:35.032+0000 7fcec7e00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fced00fe830 0x7fced01162c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:33828/0 (socket says 172.21.15.18:33828) 2024-04-12T21:03:35.030 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:03:35.032+0000 7fcec7e00700 1 -- 172.21.15.18:0/1631002031 learned_addr learned my addr 172.21.15.18:0/1631002031 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:03:35.031 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:03:35.033+0000 7fcecfe00700 1 --2- 172.21.15.18:0/1631002031 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fced00fde90 0x7fced0115d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:03:35.031 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:03:35.033+0000 7fcec7e00700 1 -- 172.21.15.18:0/1631002031 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fced00fde90 msgr2=0x7fced0115d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:03:35.031 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:03:35.033+0000 7fcec7e00700 1 --2- 172.21.15.18:0/1631002031 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fced00fde90 0x7fced0115d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:03:35.031 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:03:35.033+0000 7fcec7e00700 1 -- 172.21.15.18:0/1631002031 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fceb8005040 con 0x7fced00fe830 2024-04-12T21:03:35.031 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:03:35.033+0000 7fcec7e00700 1 --2- 172.21.15.18:0/1631002031 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fced00fe830 0x7fced01162c0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fceb8000c00 tx=0x7fceb8009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:03:35.031 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:03:35.033+0000 7fceccc00700 1 -- 172.21.15.18:0/1631002031 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fceb800e040 con 0x7fced00fe830 2024-04-12T21:03:35.031 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:03:35.033+0000 7fceccc00700 1 -- 172.21.15.18:0/1631002031 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fceb8013420 con 0x7fced00fe830 2024-04-12T21:03:35.031 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:03:35.033+0000 7fceccc00700 1 -- 172.21.15.18:0/1631002031 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fceb801c610 con 0x7fced00fe830 2024-04-12T21:03:35.031 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:03:35.033+0000 7fced7000700 1 -- 172.21.15.18:0/1631002031 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fced010ff00 con 0x7fced00fe830 2024-04-12T21:03:35.032 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:03:35.034+0000 7fced7000700 1 -- 172.21.15.18:0/1631002031 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fced0110440 con 0x7fced00fe830 2024-04-12T21:03:35.032 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:03:35.035+0000 7fced7000700 1 -- 172.21.15.18:0/1631002031 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fced0106a20 con 0x7fced00fe830 2024-04-12T21:03:35.033 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:03:35.036+0000 7fceccc00700 1 -- 172.21.15.18:0/1631002031 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7fceb8013590 con 0x7fced00fe830 2024-04-12T21:03:35.034 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:03:35.037+0000 7fceccc00700 1 --2- 172.21.15.18:0/1631002031 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fceb406ec10 0x7fceb40710c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:03:35.035 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:03:35.037+0000 7fceccc00700 1 -- 172.21.15.18:0/1631002031 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6324+0+0 (secure 0 0 0) 0x7fceb808fa40 con 0x7fced00fe830 2024-04-12T21:03:35.035 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:03:35.038+0000 7fcecfe00700 1 --2- 172.21.15.18:0/1631002031 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fceb406ec10 0x7fceb40710c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:03:35.035 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:03:35.038+0000 7fcecfe00700 1 --2- 172.21.15.18:0/1631002031 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fceb406ec10 0x7fceb40710c0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fcec0006fd0 tx=0x7fcec0008040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:03:35.039 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:03:35.043+0000 7fceccc00700 1 -- 172.21.15.18:0/1631002031 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7fceb805c3a0 con 0x7fced00fe830 2024-04-12T21:03:35.327 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:03:35.330+0000 7fced7000700 1 -- 172.21.15.18:0/1631002031 --> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fced0109f80 con 0x7fceb406ec10 2024-04-12T21:03:35.329 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:03:35.332+0000 7fceccc00700 1 -- 172.21.15.18:0/1631002031 <== mgr.34102 v2:172.21.15.18:6800/3812891919 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7fced0109f80 con 0x7fceb406ec10 2024-04-12T21:03:35.329 INFO:teuthology.orchestra.run.smithi018.stdout:{ 2024-04-12T21:03:35.329 INFO:teuthology.orchestra.run.smithi018.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:f4452def3be4d350f4a8e5feaf7ebb1511be9298fe68478ba7a2d6fbd7d69a63", 2024-04-12T21:03:35.329 INFO:teuthology.orchestra.run.smithi018.stdout: "in_progress": true, 2024-04-12T21:03:35.329 INFO:teuthology.orchestra.run.smithi018.stdout: "which": "Upgrading all daemon types on all hosts", 2024-04-12T21:03:35.329 INFO:teuthology.orchestra.run.smithi018.stdout: "services_complete": [ 2024-04-12T21:03:35.329 INFO:teuthology.orchestra.run.smithi018.stdout: "crash", 2024-04-12T21:03:35.329 INFO:teuthology.orchestra.run.smithi018.stdout: "mon", 2024-04-12T21:03:35.330 INFO:teuthology.orchestra.run.smithi018.stdout: "mgr" 2024-04-12T21:03:35.330 INFO:teuthology.orchestra.run.smithi018.stdout: ], 2024-04-12T21:03:35.330 INFO:teuthology.orchestra.run.smithi018.stdout: "progress": "6/21 daemons upgraded", 2024-04-12T21:03:35.330 INFO:teuthology.orchestra.run.smithi018.stdout: "message": "Currently upgrading osd daemons", 2024-04-12T21:03:35.330 INFO:teuthology.orchestra.run.smithi018.stdout: "is_paused": false 2024-04-12T21:03:35.330 INFO:teuthology.orchestra.run.smithi018.stdout:} 2024-04-12T21:03:35.332 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:03:35.334+0000 7fced7000700 1 -- 172.21.15.18:0/1631002031 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fceb406ec10 msgr2=0x7fceb40710c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:03:35.332 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:03:35.334+0000 7fced7000700 1 --2- 172.21.15.18:0/1631002031 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fceb406ec10 0x7fceb40710c0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fcec0006fd0 tx=0x7fcec0008040 comp rx=0 tx=0).stop 2024-04-12T21:03:35.332 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:03:35.334+0000 7fced7000700 1 -- 172.21.15.18:0/1631002031 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fced00fe830 msgr2=0x7fced01162c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:03:35.332 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:03:35.334+0000 7fced7000700 1 --2- 172.21.15.18:0/1631002031 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fced00fe830 0x7fced01162c0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fceb8000c00 tx=0x7fceb8009e30 comp rx=0 tx=0).stop 2024-04-12T21:03:35.332 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:03:35.334+0000 7fced7000700 1 -- 172.21.15.18:0/1631002031 shutdown_connections 2024-04-12T21:03:35.332 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:03:35.334+0000 7fced7000700 1 --2- 172.21.15.18:0/1631002031 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fceb406ec10 0x7fceb40710c0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:03:35.332 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:03:35.334+0000 7fced7000700 1 --2- 172.21.15.18:0/1631002031 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fced00fde90 0x7fced0115d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:03:35.332 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:03:35.334+0000 7fced7000700 1 --2- 172.21.15.18:0/1631002031 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fced00fe830 0x7fced01162c0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:03:35.332 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:03:35.334+0000 7fced7000700 1 -- 172.21.15.18:0/1631002031 >> 172.21.15.18:0/1631002031 conn(0x7fced0072bc0 msgr2=0x7fced00fb5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:03:35.332 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:03:35.335+0000 7fced7000700 1 -- 172.21.15.18:0/1631002031 shutdown_connections 2024-04-12T21:03:35.333 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:03:35.335+0000 7fced7000700 1 -- 172.21.15.18:0/1631002031 wait complete. 2024-04-12T21:03:35.481 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:03:35.483+0000 7fcb93e00700 1 -- 172.21.15.18:0/3347183441 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fcb94102ca0 msgr2=0x7fcb94103070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:03:35.481 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:03:35.483+0000 7fcb93e00700 1 --2- 172.21.15.18:0/3347183441 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fcb94102ca0 0x7fcb94103070 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fcb84004940 tx=0x7fcb84009e30 comp rx=0 tx=0).stop 2024-04-12T21:03:35.481 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:03:35.484+0000 7fcb93e00700 1 -- 172.21.15.18:0/3347183441 shutdown_connections 2024-04-12T21:03:35.481 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:03:35.484+0000 7fcb93e00700 1 --2- 172.21.15.18:0/3347183441 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcb94065630 0x7fcb94065a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:03:35.481 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:03:35.484+0000 7fcb93e00700 1 --2- 172.21.15.18:0/3347183441 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fcb94102ca0 0x7fcb94103070 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:03:35.482 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:03:35.484+0000 7fcb93e00700 1 -- 172.21.15.18:0/3347183441 >> 172.21.15.18:0/3347183441 conn(0x7fcb94071b70 msgr2=0x7fcb94071f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:03:35.482 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:03:35.484+0000 7fcb93e00700 1 -- 172.21.15.18:0/3347183441 shutdown_connections 2024-04-12T21:03:35.482 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:03:35.484+0000 7fcb93e00700 1 -- 172.21.15.18:0/3347183441 wait complete. 2024-04-12T21:03:35.482 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:03:35.485+0000 7fcb93e00700 1 Processor -- start 2024-04-12T21:03:35.483 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:03:35.485+0000 7fcb93e00700 1 -- start start 2024-04-12T21:03:35.483 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:03:35.485+0000 7fcb93e00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fcb94065630 0x7fcb940fbc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:03:35.483 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:03:35.486+0000 7fcb93e00700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcb94102ca0 0x7fcb940fc160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:03:35.483 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:03:35.486+0000 7fcb93e00700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb940fc6f0 con 0x7fcb94065630 2024-04-12T21:03:35.483 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:03:35.486+0000 7fcb93e00700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb940fc830 con 0x7fcb94102ca0 2024-04-12T21:03:35.484 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:03:35.486+0000 7fcb92a00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fcb94065630 0x7fcb940fbc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:03:35.484 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:03:35.486+0000 7fcb92a00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fcb94065630 0x7fcb940fbc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:33844/0 (socket says 172.21.15.18:33844) 2024-04-12T21:03:35.484 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:03:35.486+0000 7fcb92a00700 1 -- 172.21.15.18:0/2251862430 learned_addr learned my addr 172.21.15.18:0/2251862430 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:03:35.484 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:03:35.486+0000 7fcb92a00700 1 -- 172.21.15.18:0/2251862430 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcb94102ca0 msgr2=0x7fcb940fc160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:03:35.484 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:03:35.486+0000 7fcb92a00700 1 --2- 172.21.15.18:0/2251862430 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcb94102ca0 0x7fcb940fc160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:03:35.484 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:03:35.486+0000 7fcb92a00700 1 -- 172.21.15.18:0/2251862430 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb84005040 con 0x7fcb94065630 2024-04-12T21:03:35.484 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:03:35.487+0000 7fcb92a00700 1 --2- 172.21.15.18:0/2251862430 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fcb94065630 0x7fcb940fbc20 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fcb84009180 tx=0x7fcb84009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:03:35.485 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:03:35.487+0000 7fcb8b400700 1 -- 172.21.15.18:0/2251862430 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcb8400e040 con 0x7fcb94065630 2024-04-12T21:03:35.485 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:03:35.487+0000 7fcb8b400700 1 -- 172.21.15.18:0/2251862430 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fcb8400bbc0 con 0x7fcb94065630 2024-04-12T21:03:35.485 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:03:35.487+0000 7fcb8b400700 1 -- 172.21.15.18:0/2251862430 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcb8401c630 con 0x7fcb94065630 2024-04-12T21:03:35.485 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:03:35.487+0000 7fcb93e00700 1 -- 172.21.15.18:0/2251862430 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcb940fca80 con 0x7fcb94065630 2024-04-12T21:03:35.485 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:03:35.487+0000 7fcb93e00700 1 -- 172.21.15.18:0/2251862430 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcb9411a0f0 con 0x7fcb94065630 2024-04-12T21:03:35.485 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:03:35.488+0000 7fcb93e00700 1 -- 172.21.15.18:0/2251862430 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcb940fe380 con 0x7fcb94065630 2024-04-12T21:03:35.486 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:03:35.489+0000 7fcb8b400700 1 -- 172.21.15.18:0/2251862430 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7fcb8400bd30 con 0x7fcb94065630 2024-04-12T21:03:35.487 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:03:35.490+0000 7fcb8b400700 1 --2- 172.21.15.18:0/2251862430 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fcb7806ec10 0x7fcb780710c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:03:35.487 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:03:35.490+0000 7fcb8b400700 1 -- 172.21.15.18:0/2251862430 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6324+0+0 (secure 0 0 0) 0x7fcb8408fb00 con 0x7fcb94065630 2024-04-12T21:03:35.487 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:03:35.490+0000 7fcb8a000700 1 --2- 172.21.15.18:0/2251862430 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fcb7806ec10 0x7fcb780710c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:03:35.488 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:03:35.491+0000 7fcb8a000700 1 --2- 172.21.15.18:0/2251862430 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fcb7806ec10 0x7fcb780710c0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fcb940fd560 tx=0x7fcb7c007d40 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:03:35.492 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:03:35.495+0000 7fcb8b400700 1 -- 172.21.15.18:0/2251862430 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7fcb84058a70 con 0x7fcb94065630 2024-04-12T21:03:35.601 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:03:35 smithi018 ceph-mon[112590]: pgmap v60: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-12T21:03:35.601 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:03:35 smithi018 ceph-mon[112590]: from='client.34224 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:03:35.601 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:03:35 smithi018 ceph-mon[112590]: from='client.? 172.21.15.18:0/1803524158' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T21:03:35.601 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:03:35 smithi018 ceph-mon[112590]: from='client.? 172.21.15.18:0/1239688311' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-12T21:03:35.779 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:03:35 smithi186 ceph-mon[86379]: pgmap v60: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-12T21:03:35.780 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:03:35 smithi186 ceph-mon[86379]: from='client.34224 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:03:35.780 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:03:35 smithi186 ceph-mon[86379]: from='client.? 172.21.15.18:0/1803524158' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T21:03:35.780 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:03:35 smithi186 ceph-mon[86379]: from='client.? 172.21.15.18:0/1239688311' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-12T21:03:35.837 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:03:35.839+0000 7fcb93e00700 1 -- 172.21.15.18:0/2251862430 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7fcb9404bed0 con 0x7fcb94065630 2024-04-12T21:03:35.838 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:03:35.840+0000 7fcb8b400700 1 -- 172.21.15.18:0/2251862430 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+10 (secure 0 0 0) 0x7fcb8405c130 con 0x7fcb94065630 2024-04-12T21:03:35.838 INFO:teuthology.orchestra.run.smithi018.stdout:HEALTH_OK 2024-04-12T21:03:35.840 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:03:35.843+0000 7fcb93e00700 1 -- 172.21.15.18:0/2251862430 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fcb7806ec10 msgr2=0x7fcb780710c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:03:35.840 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:03:35.843+0000 7fcb93e00700 1 --2- 172.21.15.18:0/2251862430 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fcb7806ec10 0x7fcb780710c0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fcb940fd560 tx=0x7fcb7c007d40 comp rx=0 tx=0).stop 2024-04-12T21:03:35.840 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:03:35.843+0000 7fcb93e00700 1 -- 172.21.15.18:0/2251862430 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fcb94065630 msgr2=0x7fcb940fbc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:03:35.840 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:03:35.843+0000 7fcb93e00700 1 --2- 172.21.15.18:0/2251862430 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fcb94065630 0x7fcb940fbc20 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fcb84009180 tx=0x7fcb84009e30 comp rx=0 tx=0).stop 2024-04-12T21:03:35.841 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:03:35.843+0000 7fcb93e00700 1 -- 172.21.15.18:0/2251862430 shutdown_connections 2024-04-12T21:03:35.841 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:03:35.843+0000 7fcb93e00700 1 --2- 172.21.15.18:0/2251862430 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fcb7806ec10 0x7fcb780710c0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:03:35.841 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:03:35.843+0000 7fcb93e00700 1 --2- 172.21.15.18:0/2251862430 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fcb94065630 0x7fcb940fbc20 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:03:35.841 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:03:35.843+0000 7fcb93e00700 1 --2- 172.21.15.18:0/2251862430 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcb94102ca0 0x7fcb940fc160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:03:35.841 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:03:35.843+0000 7fcb93e00700 1 -- 172.21.15.18:0/2251862430 >> 172.21.15.18:0/2251862430 conn(0x7fcb94071b70 msgr2=0x7fcb940f9fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:03:35.841 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:03:35.843+0000 7fcb93e00700 1 -- 172.21.15.18:0/2251862430 shutdown_connections 2024-04-12T21:03:35.841 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:03:35.844+0000 7fcb93e00700 1 -- 172.21.15.18:0/2251862430 wait complete. 2024-04-12T21:03:36.101 INFO:journalctl@ceph.osd.0.smithi018.stdout:Apr 12 21:03:35 smithi018 systemd[1]: Stopping Ceph osd.0 for 1cbe407a-f90d-11ee-bc8d-c7b262605968... 2024-04-12T21:03:36.779 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:03:36 smithi186 ceph-mon[86379]: from='client.34236 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:03:36.780 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:03:36 smithi186 ceph-mon[86379]: from='client.? 172.21.15.18:0/2251862430' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-12T21:03:36.780 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:03:36 smithi186 ceph-mon[86379]: osd.0 marked itself down and dead 2024-04-12T21:03:36.851 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:03:36 smithi018 ceph-mon[112590]: from='client.34236 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:03:36.851 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:03:36 smithi018 ceph-mon[112590]: from='client.? 172.21.15.18:0/2251862430' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-12T21:03:36.851 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:03:36 smithi018 ceph-mon[112590]: osd.0 marked itself down and dead 2024-04-12T21:03:36.851 INFO:journalctl@ceph.osd.0.smithi018.stdout:Apr 12 21:03:36 smithi018 ceph-1cbe407a-f90d-11ee-bc8d-c7b262605968-osd-0[47341]: 2024-04-12T21:03:36.387+0000 7f7373200700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-12T21:03:36.851 INFO:journalctl@ceph.osd.0.smithi018.stdout:Apr 12 21:03:36 smithi018 ceph-1cbe407a-f90d-11ee-bc8d-c7b262605968-osd-0[47341]: 2024-04-12T21:03:36.387+0000 7f7373200700 -1 osd.0 49 *** Got signal Terminated *** 2024-04-12T21:03:36.851 INFO:journalctl@ceph.osd.0.smithi018.stdout:Apr 12 21:03:36 smithi018 ceph-1cbe407a-f90d-11ee-bc8d-c7b262605968-osd-0[47341]: 2024-04-12T21:03:36.387+0000 7f7373200700 -1 osd.0 49 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-12T21:03:37.342 INFO:journalctl@ceph.osd.0.smithi018.stdout:Apr 12 21:03:37 smithi018 podman[123826]: 2024-04-12 21:03:37.079189363 +0000 UTC m=+0.825568647 container died 0d83e027bf0aa35b55a244a8d810a4efe9ab29c6e640f3d7c764e0b4c68d37e9 (image=quay.ceph.io/ceph-ci/ceph@sha256:5e334df698f432180b78bc9c46936bfdfbc92bc9a8983463e523f780d23765fe, name=ceph-1cbe407a-f90d-11ee-bc8d-c7b262605968-osd-0, io.buildah.version=1.33.5, GIT_CLEAN=True, ceph=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, org.label-schema.build-date=20240409, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, RELEASE=quincy-d3b5060) 2024-04-12T21:03:37.600 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:03:37 smithi018 ceph-mon[112590]: pgmap v61: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-12T21:03:37.600 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:03:37 smithi018 ceph-mon[112590]: Health check failed: 1 osds down (OSD_DOWN) 2024-04-12T21:03:37.601 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:03:37 smithi018 ceph-mon[112590]: osdmap e50: 6 total, 5 up, 6 in 2024-04-12T21:03:37.601 INFO:journalctl@ceph.osd.0.smithi018.stdout:Apr 12 21:03:37 smithi018 podman[123826]: 2024-04-12 21:03:37.514760523 +0000 UTC m=+1.261139873 container cleanup 0d83e027bf0aa35b55a244a8d810a4efe9ab29c6e640f3d7c764e0b4c68d37e9 (image=quay.ceph.io/ceph-ci/ceph@sha256:5e334df698f432180b78bc9c46936bfdfbc92bc9a8983463e523f780d23765fe, name=ceph-1cbe407a-f90d-11ee-bc8d-c7b262605968-osd-0, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.build-date=20240409, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, RELEASE=quincy-d3b5060, ceph=True) 2024-04-12T21:03:37.601 INFO:journalctl@ceph.osd.0.smithi018.stdout:Apr 12 21:03:37 smithi018 bash[123826]: ceph-1cbe407a-f90d-11ee-bc8d-c7b262605968-osd-0 2024-04-12T21:03:37.780 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:03:37 smithi186 ceph-mon[86379]: pgmap v61: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-12T21:03:37.780 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:03:37 smithi186 ceph-mon[86379]: Health check failed: 1 osds down (OSD_DOWN) 2024-04-12T21:03:37.780 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:03:37 smithi186 ceph-mon[86379]: osdmap e50: 6 total, 5 up, 6 in 2024-04-12T21:03:38.100 INFO:journalctl@ceph.osd.0.smithi018.stdout:Apr 12 21:03:37 smithi018 podman[123845]: 2024-04-12 21:03:37.715642354 +0000 UTC m=+0.620949996 container remove 0d83e027bf0aa35b55a244a8d810a4efe9ab29c6e640f3d7c764e0b4c68d37e9 (image=quay.ceph.io/ceph-ci/ceph@sha256:5e334df698f432180b78bc9c46936bfdfbc92bc9a8983463e523f780d23765fe, name=ceph-1cbe407a-f90d-11ee-bc8d-c7b262605968-osd-0, org.label-schema.build-date=20240409, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, RELEASE=quincy-d3b5060, ceph=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD) 2024-04-12T21:03:38.779 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:03:38 smithi186 ceph-mon[86379]: osdmap e51: 6 total, 5 up, 6 in 2024-04-12T21:03:38.846 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:03:38 smithi018 ceph-mon[112590]: osdmap e51: 6 total, 5 up, 6 in 2024-04-12T21:03:38.846 INFO:journalctl@ceph.osd.0.smithi018.stdout:Apr 12 21:03:38 smithi018 podman[123934]: 2024-04-12 21:03:38.519893564 +0000 UTC m=+0.036434324 image pull 8717720dc5e5c56303818ea4d4f0a11f6e2c03dee521fb668f411f1304e0afca quay.ceph.io/ceph-ci/ceph@sha256:f4452def3be4d350f4a8e5feaf7ebb1511be9298fe68478ba7a2d6fbd7d69a63 2024-04-12T21:03:38.846 INFO:journalctl@ceph.osd.0.smithi018.stdout:Apr 12 21:03:38 smithi018 podman[123934]: 2024-04-12 21:03:38.642401758 +0000 UTC m=+0.158942548 container create 3ae6373b00bdabb3ecd133f473d579089593e4f0f032b7eb93fe1a6474abd4e8 (image=quay.ceph.io/ceph-ci/ceph@sha256:f4452def3be4d350f4a8e5feaf7ebb1511be9298fe68478ba7a2d6fbd7d69a63, name=ceph-1cbe407a-f90d-11ee-bc8d-c7b262605968-osd-0-deactivate, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240409, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, RELEASE=wip-vshankar-testing-20240411.061452-324baf8, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-12T21:03:39.101 INFO:journalctl@ceph.osd.0.smithi018.stdout:Apr 12 21:03:39 smithi018 podman[123934]: 2024-04-12 21:03:39.005072727 +0000 UTC m=+0.521613517 container init 3ae6373b00bdabb3ecd133f473d579089593e4f0f032b7eb93fe1a6474abd4e8 (image=quay.ceph.io/ceph-ci/ceph@sha256:f4452def3be4d350f4a8e5feaf7ebb1511be9298fe68478ba7a2d6fbd7d69a63, name=ceph-1cbe407a-f90d-11ee-bc8d-c7b262605968-osd-0-deactivate, io.buildah.version=1.33.5, org.label-schema.build-date=20240409, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, ceph=True, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, RELEASE=wip-vshankar-testing-20240411.061452-324baf8, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-12T21:03:39.102 INFO:journalctl@ceph.osd.0.smithi018.stdout:Apr 12 21:03:39 smithi018 podman[123934]: 2024-04-12 21:03:39.017324654 +0000 UTC m=+0.533865480 container start 3ae6373b00bdabb3ecd133f473d579089593e4f0f032b7eb93fe1a6474abd4e8 (image=quay.ceph.io/ceph-ci/ceph@sha256:f4452def3be4d350f4a8e5feaf7ebb1511be9298fe68478ba7a2d6fbd7d69a63, name=ceph-1cbe407a-f90d-11ee-bc8d-c7b262605968-osd-0-deactivate, io.buildah.version=1.33.5, org.label-schema.build-date=20240409, RELEASE=wip-vshankar-testing-20240411.061452-324baf8, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-12T21:03:39.102 INFO:journalctl@ceph.osd.0.smithi018.stdout:Apr 12 21:03:39 smithi018 podman[123934]: 2024-04-12 21:03:39.095134827 +0000 UTC m=+0.611675617 container attach 3ae6373b00bdabb3ecd133f473d579089593e4f0f032b7eb93fe1a6474abd4e8 (image=quay.ceph.io/ceph-ci/ceph@sha256:f4452def3be4d350f4a8e5feaf7ebb1511be9298fe68478ba7a2d6fbd7d69a63, name=ceph-1cbe407a-f90d-11ee-bc8d-c7b262605968-osd-0-deactivate, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, ceph=True, org.label-schema.vendor=CentOS, GIT_CLEAN=True, maintainer=Guillaume Abrioux , RELEASE=wip-vshankar-testing-20240411.061452-324baf8, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240409, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-12T21:03:39.683 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:03:39 smithi186 ceph-mon[86379]: pgmap v64: 65 pgs: 9 stale+active+clean, 56 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-12T21:03:39.825 INFO:journalctl@ceph.osd.0.smithi018.stdout:Apr 12 21:03:39 smithi018 podman[123934]: 2024-04-12 21:03:39.482721133 +0000 UTC m=+0.999261956 container died 3ae6373b00bdabb3ecd133f473d579089593e4f0f032b7eb93fe1a6474abd4e8 (image=quay.ceph.io/ceph-ci/ceph@sha256:f4452def3be4d350f4a8e5feaf7ebb1511be9298fe68478ba7a2d6fbd7d69a63, name=ceph-1cbe407a-f90d-11ee-bc8d-c7b262605968-osd-0-deactivate, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-vshankar-testing-20240411.061452-324baf8, GIT_CLEAN=True, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.build-date=20240409) 2024-04-12T21:03:39.826 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:03:39 smithi018 ceph-mon[112590]: pgmap v64: 65 pgs: 9 stale+active+clean, 56 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-12T21:03:40.601 INFO:journalctl@ceph.osd.0.smithi018.stdout:Apr 12 21:03:40 smithi018 podman[123960]: 2024-04-12 21:03:40.257093293 +0000 UTC m=+0.755985488 container remove 3ae6373b00bdabb3ecd133f473d579089593e4f0f032b7eb93fe1a6474abd4e8 (image=quay.ceph.io/ceph-ci/ceph@sha256:f4452def3be4d350f4a8e5feaf7ebb1511be9298fe68478ba7a2d6fbd7d69a63, name=ceph-1cbe407a-f90d-11ee-bc8d-c7b262605968-osd-0-deactivate, ceph=True, org.label-schema.build-date=20240409, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, RELEASE=wip-vshankar-testing-20240411.061452-324baf8, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux ) 2024-04-12T21:03:40.601 INFO:journalctl@ceph.osd.0.smithi018.stdout:Apr 12 21:03:40 smithi018 systemd[1]: ceph-1cbe407a-f90d-11ee-bc8d-c7b262605968@osd.0.service: Deactivated successfully. 2024-04-12T21:03:40.601 INFO:journalctl@ceph.osd.0.smithi018.stdout:Apr 12 21:03:40 smithi018 systemd[1]: Stopped Ceph osd.0 for 1cbe407a-f90d-11ee-bc8d-c7b262605968. 2024-04-12T21:03:40.601 INFO:journalctl@ceph.osd.0.smithi018.stdout:Apr 12 21:03:40 smithi018 systemd[1]: ceph-1cbe407a-f90d-11ee-bc8d-c7b262605968@osd.0.service: Consumed 1min 46.907s CPU time. 2024-04-12T21:03:40.602 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:03:40 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.8", "id": [3, 2]}]: dispatch 2024-04-12T21:03:40.602 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:03:40 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [3, 0]}]: dispatch 2024-04-12T21:03:40.602 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:03:40 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [3, 2]}]: dispatch 2024-04-12T21:03:40.602 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:03:40 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.6", "id": [1, 5]}]: dispatch 2024-04-12T21:03:40.602 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:03:40 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.a", "id": [4, 0]}]: dispatch 2024-04-12T21:03:40.602 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:03:40 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.e", "id": [1, 2]}]: dispatch 2024-04-12T21:03:40.602 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:03:40 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.11", "id": [3, 2]}]: dispatch 2024-04-12T21:03:40.602 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:03:40 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.14", "id": [3, 0]}]: dispatch 2024-04-12T21:03:40.602 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:03:40 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1a", "id": [4, 0]}]: dispatch 2024-04-12T21:03:40.602 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:03:40 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' 2024-04-12T21:03:40.602 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:03:40 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T21:03:40.718 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:03:40 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.8", "id": [3, 2]}]: dispatch 2024-04-12T21:03:40.718 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:03:40 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [3, 0]}]: dispatch 2024-04-12T21:03:40.718 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:03:40 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [3, 2]}]: dispatch 2024-04-12T21:03:40.718 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:03:40 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.6", "id": [1, 5]}]: dispatch 2024-04-12T21:03:40.718 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:03:40 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.a", "id": [4, 0]}]: dispatch 2024-04-12T21:03:40.718 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:03:40 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.e", "id": [1, 2]}]: dispatch 2024-04-12T21:03:40.718 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:03:40 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.11", "id": [3, 2]}]: dispatch 2024-04-12T21:03:40.718 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:03:40 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.14", "id": [3, 0]}]: dispatch 2024-04-12T21:03:40.718 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:03:40 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1a", "id": [4, 0]}]: dispatch 2024-04-12T21:03:40.718 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:03:40 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' 2024-04-12T21:03:40.719 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:03:40 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T21:03:41.054 INFO:journalctl@ceph.osd.0.smithi018.stdout:Apr 12 21:03:40 smithi018 systemd[1]: Starting Ceph osd.0 for 1cbe407a-f90d-11ee-bc8d-c7b262605968... 2024-04-12T21:03:41.351 INFO:journalctl@ceph.osd.0.smithi018.stdout:Apr 12 21:03:41 smithi018 podman[124062]: 2024-04-12 21:03:40.962881851 +0000 UTC m=+0.034308471 image pull 8717720dc5e5c56303818ea4d4f0a11f6e2c03dee521fb668f411f1304e0afca quay.ceph.io/ceph-ci/ceph@sha256:f4452def3be4d350f4a8e5feaf7ebb1511be9298fe68478ba7a2d6fbd7d69a63 2024-04-12T21:03:41.351 INFO:journalctl@ceph.osd.0.smithi018.stdout:Apr 12 21:03:41 smithi018 podman[124062]: 2024-04-12 21:03:41.100088979 +0000 UTC m=+0.171515633 container create 15ab23b1c8a387e2589d81c7c42def5a68dec8c73327a08ace4f0e6646918a42 (image=quay.ceph.io/ceph-ci/ceph@sha256:f4452def3be4d350f4a8e5feaf7ebb1511be9298fe68478ba7a2d6fbd7d69a63, name=ceph-1cbe407a-f90d-11ee-bc8d-c7b262605968-osd-0-activate, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240409, io.buildah.version=1.33.5, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-vshankar-testing-20240411.061452-324baf8, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-12T21:03:41.780 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:03:41 smithi186 ceph-mon[86379]: pgmap v65: 65 pgs: 9 stale+active+clean, 56 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-12T21:03:41.780 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:03:41 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.8", "id": [3, 2]}]': finished 2024-04-12T21:03:41.780 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:03:41 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [3, 0]}]': finished 2024-04-12T21:03:41.780 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:03:41 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [3, 2]}]': finished 2024-04-12T21:03:41.780 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:03:41 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.6", "id": [1, 5]}]': finished 2024-04-12T21:03:41.780 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:03:41 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.a", "id": [4, 0]}]': finished 2024-04-12T21:03:41.780 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:03:41 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.e", "id": [1, 2]}]': finished 2024-04-12T21:03:41.781 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:03:41 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.11", "id": [3, 2]}]': finished 2024-04-12T21:03:41.781 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:03:41 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.14", "id": [3, 0]}]': finished 2024-04-12T21:03:41.781 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:03:41 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1a", "id": [4, 0]}]': finished 2024-04-12T21:03:41.781 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:03:41 smithi186 ceph-mon[86379]: osdmap e52: 6 total, 5 up, 6 in 2024-04-12T21:03:41.851 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:03:41 smithi018 ceph-mon[112590]: pgmap v65: 65 pgs: 9 stale+active+clean, 56 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-12T21:03:41.851 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:03:41 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.8", "id": [3, 2]}]': finished 2024-04-12T21:03:41.851 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:03:41 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [3, 0]}]': finished 2024-04-12T21:03:41.851 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:03:41 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [3, 2]}]': finished 2024-04-12T21:03:41.851 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:03:41 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.6", "id": [1, 5]}]': finished 2024-04-12T21:03:41.851 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:03:41 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.a", "id": [4, 0]}]': finished 2024-04-12T21:03:41.851 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:03:41 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.e", "id": [1, 2]}]': finished 2024-04-12T21:03:41.851 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:03:41 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.11", "id": [3, 2]}]': finished 2024-04-12T21:03:41.851 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:03:41 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.14", "id": [3, 0]}]': finished 2024-04-12T21:03:41.851 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:03:41 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1a", "id": [4, 0]}]': finished 2024-04-12T21:03:41.851 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:03:41 smithi018 ceph-mon[112590]: osdmap e52: 6 total, 5 up, 6 in 2024-04-12T21:03:41.852 INFO:journalctl@ceph.osd.0.smithi018.stdout:Apr 12 21:03:41 smithi018 podman[124062]: 2024-04-12 21:03:41.364193258 +0000 UTC m=+0.435619915 container init 15ab23b1c8a387e2589d81c7c42def5a68dec8c73327a08ace4f0e6646918a42 (image=quay.ceph.io/ceph-ci/ceph@sha256:f4452def3be4d350f4a8e5feaf7ebb1511be9298fe68478ba7a2d6fbd7d69a63, name=ceph-1cbe407a-f90d-11ee-bc8d-c7b262605968-osd-0-activate, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_CLEAN=True, ceph=True, org.label-schema.build-date=20240409, RELEASE=wip-vshankar-testing-20240411.061452-324baf8, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS) 2024-04-12T21:03:41.852 INFO:journalctl@ceph.osd.0.smithi018.stdout:Apr 12 21:03:41 smithi018 podman[124062]: 2024-04-12 21:03:41.376844401 +0000 UTC m=+0.448271082 container start 15ab23b1c8a387e2589d81c7c42def5a68dec8c73327a08ace4f0e6646918a42 (image=quay.ceph.io/ceph-ci/ceph@sha256:f4452def3be4d350f4a8e5feaf7ebb1511be9298fe68478ba7a2d6fbd7d69a63, name=ceph-1cbe407a-f90d-11ee-bc8d-c7b262605968-osd-0-activate, RELEASE=wip-vshankar-testing-20240411.061452-324baf8, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240409, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux ) 2024-04-12T21:03:41.852 INFO:journalctl@ceph.osd.0.smithi018.stdout:Apr 12 21:03:41 smithi018 podman[124062]: 2024-04-12 21:03:41.511311351 +0000 UTC m=+0.582738048 container attach 15ab23b1c8a387e2589d81c7c42def5a68dec8c73327a08ace4f0e6646918a42 (image=quay.ceph.io/ceph-ci/ceph@sha256:f4452def3be4d350f4a8e5feaf7ebb1511be9298fe68478ba7a2d6fbd7d69a63, name=ceph-1cbe407a-f90d-11ee-bc8d-c7b262605968-osd-0-activate, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.build-date=20240409, org.label-schema.schema-version=1.0, RELEASE=wip-vshankar-testing-20240411.061452-324baf8, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-12T21:03:42.529 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:03:42 smithi186 ceph-mon[86379]: osdmap e53: 6 total, 5 up, 6 in 2024-04-12T21:03:42.529 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:03:42 smithi186 ceph-mon[86379]: osdmap e54: 6 total, 5 up, 6 in 2024-04-12T21:03:42.601 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:03:42 smithi018 ceph-mon[112590]: osdmap e53: 6 total, 5 up, 6 in 2024-04-12T21:03:42.601 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:03:42 smithi018 ceph-mon[112590]: osdmap e54: 6 total, 5 up, 6 in 2024-04-12T21:03:42.601 INFO:journalctl@ceph.osd.0.smithi018.stdout:Apr 12 21:03:42 smithi018 ceph-1cbe407a-f90d-11ee-bc8d-c7b262605968-osd-0-activate[124077]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-04-12T21:03:42.601 INFO:journalctl@ceph.osd.0.smithi018.stdout:Apr 12 21:03:42 smithi018 ceph-1cbe407a-f90d-11ee-bc8d-c7b262605968-osd-0-activate[124077]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-0 --no-mon-config --dev /dev/mapper/vg_nvme-lv_4 2024-04-12T21:03:42.601 INFO:journalctl@ceph.osd.0.smithi018.stdout:Apr 12 21:03:42 smithi018 bash[124062]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-04-12T21:03:42.601 INFO:journalctl@ceph.osd.0.smithi018.stdout:Apr 12 21:03:42 smithi018 bash[124062]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-0 --no-mon-config --dev /dev/mapper/vg_nvme-lv_4 2024-04-12T21:03:42.601 INFO:journalctl@ceph.osd.0.smithi018.stdout:Apr 12 21:03:42 smithi018 bash[124062]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_4 2024-04-12T21:03:42.602 INFO:journalctl@ceph.osd.0.smithi018.stdout:Apr 12 21:03:42 smithi018 bash[124062]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-3 2024-04-12T21:03:42.602 INFO:journalctl@ceph.osd.0.smithi018.stdout:Apr 12 21:03:42 smithi018 bash[124062]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-0/block 2024-04-12T21:03:42.602 INFO:journalctl@ceph.osd.0.smithi018.stdout:Apr 12 21:03:42 smithi018 bash[124062]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-04-12T21:03:42.602 INFO:journalctl@ceph.osd.0.smithi018.stdout:Apr 12 21:03:42 smithi018 bash[124062]: --> ceph-volume raw activate successful for osd ID: 0 2024-04-12T21:03:42.602 INFO:journalctl@ceph.osd.0.smithi018.stdout:Apr 12 21:03:42 smithi018 ceph-1cbe407a-f90d-11ee-bc8d-c7b262605968-osd-0-activate[124077]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_4 2024-04-12T21:03:42.602 INFO:journalctl@ceph.osd.0.smithi018.stdout:Apr 12 21:03:42 smithi018 ceph-1cbe407a-f90d-11ee-bc8d-c7b262605968-osd-0-activate[124077]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-3 2024-04-12T21:03:42.602 INFO:journalctl@ceph.osd.0.smithi018.stdout:Apr 12 21:03:42 smithi018 ceph-1cbe407a-f90d-11ee-bc8d-c7b262605968-osd-0-activate[124077]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-0/block 2024-04-12T21:03:42.602 INFO:journalctl@ceph.osd.0.smithi018.stdout:Apr 12 21:03:42 smithi018 ceph-1cbe407a-f90d-11ee-bc8d-c7b262605968-osd-0-activate[124077]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-04-12T21:03:42.602 INFO:journalctl@ceph.osd.0.smithi018.stdout:Apr 12 21:03:42 smithi018 ceph-1cbe407a-f90d-11ee-bc8d-c7b262605968-osd-0-activate[124077]: --> ceph-volume raw activate successful for osd ID: 0 2024-04-12T21:03:42.602 INFO:journalctl@ceph.osd.0.smithi018.stdout:Apr 12 21:03:42 smithi018 podman[124062]: 2024-04-12 21:03:42.338201694 +0000 UTC m=+1.409628352 container died 15ab23b1c8a387e2589d81c7c42def5a68dec8c73327a08ace4f0e6646918a42 (image=quay.ceph.io/ceph-ci/ceph@sha256:f4452def3be4d350f4a8e5feaf7ebb1511be9298fe68478ba7a2d6fbd7d69a63, name=ceph-1cbe407a-f90d-11ee-bc8d-c7b262605968-osd-0-activate, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, org.label-schema.build-date=20240409, RELEASE=wip-vshankar-testing-20240411.061452-324baf8, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5) 2024-04-12T21:03:43.483 INFO:journalctl@ceph.osd.0.smithi018.stdout:Apr 12 21:03:43 smithi018 podman[124206]: 2024-04-12 21:03:43.174742892 +0000 UTC m=+0.816031871 container remove 15ab23b1c8a387e2589d81c7c42def5a68dec8c73327a08ace4f0e6646918a42 (image=quay.ceph.io/ceph-ci/ceph@sha256:f4452def3be4d350f4a8e5feaf7ebb1511be9298fe68478ba7a2d6fbd7d69a63, name=ceph-1cbe407a-f90d-11ee-bc8d-c7b262605968-osd-0-activate, ceph=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240409, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-vshankar-testing-20240411.061452-324baf8, org.label-schema.license=GPLv2) 2024-04-12T21:03:43.779 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:03:43 smithi186 ceph-mon[86379]: pgmap v68: 65 pgs: 5 peering, 17 active+undersized+degraded, 13 active+undersized, 29 active+clean, 1 stale+active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 54/324 objects degraded (16.667%) 2024-04-12T21:03:43.780 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:03:43 smithi186 ceph-mon[86379]: Health check failed: Reduced data availability: 2 pgs peering (PG_AVAILABILITY) 2024-04-12T21:03:43.780 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:03:43 smithi186 ceph-mon[86379]: Health check failed: Degraded data redundancy: 54/324 objects degraded (16.667%), 17 pgs degraded (PG_DEGRADED) 2024-04-12T21:03:43.851 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:03:43 smithi018 ceph-mon[112590]: pgmap v68: 65 pgs: 5 peering, 17 active+undersized+degraded, 13 active+undersized, 29 active+clean, 1 stale+active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 54/324 objects degraded (16.667%) 2024-04-12T21:03:43.851 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:03:43 smithi018 ceph-mon[112590]: Health check failed: Reduced data availability: 2 pgs peering (PG_AVAILABILITY) 2024-04-12T21:03:43.851 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:03:43 smithi018 ceph-mon[112590]: Health check failed: Degraded data redundancy: 54/324 objects degraded (16.667%), 17 pgs degraded (PG_DEGRADED) 2024-04-12T21:03:43.851 INFO:journalctl@ceph.osd.0.smithi018.stdout:Apr 12 21:03:43 smithi018 podman[124262]: 2024-04-12 21:03:43.443916479 +0000 UTC m=+0.033279615 image pull 8717720dc5e5c56303818ea4d4f0a11f6e2c03dee521fb668f411f1304e0afca quay.ceph.io/ceph-ci/ceph@sha256:f4452def3be4d350f4a8e5feaf7ebb1511be9298fe68478ba7a2d6fbd7d69a63 2024-04-12T21:03:43.851 INFO:journalctl@ceph.osd.0.smithi018.stdout:Apr 12 21:03:43 smithi018 podman[124262]: 2024-04-12 21:03:43.725786758 +0000 UTC m=+0.315149974 container create 60b1273ca0475debf2761f0f1c017a030f5b7d0923f1f790cdb5b2ac8dfe8e13 (image=quay.ceph.io/ceph-ci/ceph@sha256:f4452def3be4d350f4a8e5feaf7ebb1511be9298fe68478ba7a2d6fbd7d69a63, name=ceph-1cbe407a-f90d-11ee-bc8d-c7b262605968-osd-0, GIT_CLEAN=True, org.label-schema.license=GPLv2, RELEASE=wip-vshankar-testing-20240411.061452-324baf8, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.build-date=20240409) 2024-04-12T21:03:44.351 INFO:journalctl@ceph.osd.0.smithi018.stdout:Apr 12 21:03:44 smithi018 podman[124262]: 2024-04-12 21:03:44.016342759 +0000 UTC m=+0.605705929 container init 60b1273ca0475debf2761f0f1c017a030f5b7d0923f1f790cdb5b2ac8dfe8e13 (image=quay.ceph.io/ceph-ci/ceph@sha256:f4452def3be4d350f4a8e5feaf7ebb1511be9298fe68478ba7a2d6fbd7d69a63, name=ceph-1cbe407a-f90d-11ee-bc8d-c7b262605968-osd-0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.build-date=20240409, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, RELEASE=wip-vshankar-testing-20240411.061452-324baf8, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-12T21:03:44.351 INFO:journalctl@ceph.osd.0.smithi018.stdout:Apr 12 21:03:44 smithi018 podman[124262]: 2024-04-12 21:03:44.031841063 +0000 UTC m=+0.621204216 container start 60b1273ca0475debf2761f0f1c017a030f5b7d0923f1f790cdb5b2ac8dfe8e13 (image=quay.ceph.io/ceph-ci/ceph@sha256:f4452def3be4d350f4a8e5feaf7ebb1511be9298fe68478ba7a2d6fbd7d69a63, name=ceph-1cbe407a-f90d-11ee-bc8d-c7b262605968-osd-0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240409, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-vshankar-testing-20240411.061452-324baf8, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, ceph=True) 2024-04-12T21:03:44.352 INFO:journalctl@ceph.osd.0.smithi018.stdout:Apr 12 21:03:44 smithi018 ceph-osd[124280]: -- 172.21.15.18:0/1574638832 <== mon.0 v2:172.21.15.18:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55da89187680 con 0x55da89150c00 2024-04-12T21:03:44.352 INFO:journalctl@ceph.osd.0.smithi018.stdout:Apr 12 21:03:44 smithi018 bash[124262]: 60b1273ca0475debf2761f0f1c017a030f5b7d0923f1f790cdb5b2ac8dfe8e13 2024-04-12T21:03:44.352 INFO:journalctl@ceph.osd.0.smithi018.stdout:Apr 12 21:03:44 smithi018 systemd[1]: Started Ceph osd.0 for 1cbe407a-f90d-11ee-bc8d-c7b262605968. 2024-04-12T21:03:44.780 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:03:44 smithi186 ceph-mon[86379]: osdmap e55: 6 total, 5 up, 6 in 2024-04-12T21:03:44.780 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:03:44 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' 2024-04-12T21:03:44.780 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:03:44 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' 2024-04-12T21:03:44.780 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:03:44 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-12T21:03:44.851 INFO:journalctl@ceph.osd.0.smithi018.stdout:Apr 12 21:03:44 smithi018 ceph-1cbe407a-f90d-11ee-bc8d-c7b262605968-osd-0[124276]: 2024-04-12T21:03:44.607+0000 7f2f4d71e7c0 -1 Falling back to public interface 2024-04-12T21:03:44.851 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:03:44 smithi018 ceph-mon[112590]: osdmap e55: 6 total, 5 up, 6 in 2024-04-12T21:03:44.851 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:03:44 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' 2024-04-12T21:03:44.851 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:03:44 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' 2024-04-12T21:03:44.851 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:03:44 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-12T21:03:45.731 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:03:45 smithi018 ceph-mon[112590]: pgmap v71: 65 pgs: 5 peering, 17 active+undersized+degraded, 13 active+undersized, 29 active+clean, 1 stale+active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 54/324 objects degraded (16.667%) 2024-04-12T21:03:45.731 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:03:45 smithi018 ceph-mon[112590]: osdmap e56: 6 total, 5 up, 6 in 2024-04-12T21:03:45.731 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:03:45 smithi018 ceph-mon[112590]: osdmap e57: 6 total, 5 up, 6 in 2024-04-12T21:03:45.779 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:03:45 smithi186 ceph-mon[86379]: pgmap v71: 65 pgs: 5 peering, 17 active+undersized+degraded, 13 active+undersized, 29 active+clean, 1 stale+active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 54/324 objects degraded (16.667%) 2024-04-12T21:03:45.780 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:03:45 smithi186 ceph-mon[86379]: osdmap e56: 6 total, 5 up, 6 in 2024-04-12T21:03:45.780 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:03:45 smithi186 ceph-mon[86379]: osdmap e57: 6 total, 5 up, 6 in 2024-04-12T21:03:47.030 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:03:46 smithi186 ceph-mon[86379]: osdmap e58: 6 total, 5 up, 6 in 2024-04-12T21:03:47.030 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:03:46 smithi186 ceph-mon[86379]: pgmap v75: 65 pgs: 27 active+clean, 17 active+undersized+degraded, 6 active+clean+remapped, 1 active+recovery_wait+undersized+remapped, 13 active+undersized, 1 active+recovering+undersized+remapped; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 54/324 objects degraded (16.667%); 768/324 objects misplaced (237.037%); 8.4 MiB/s, 341 objects/s recovering 2024-04-12T21:03:47.066 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:03:46 smithi018 ceph-mon[112590]: osdmap e58: 6 total, 5 up, 6 in 2024-04-12T21:03:47.066 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:03:46 smithi018 ceph-mon[112590]: pgmap v75: 65 pgs: 27 active+clean, 17 active+undersized+degraded, 6 active+clean+remapped, 1 active+recovery_wait+undersized+remapped, 13 active+undersized, 1 active+recovering+undersized+remapped; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 54/324 objects degraded (16.667%); 768/324 objects misplaced (237.037%); 8.4 MiB/s, 341 objects/s recovering 2024-04-12T21:03:48.029 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:03:47 smithi186 ceph-mon[86379]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 2 pgs peering) 2024-04-12T21:03:48.030 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:03:47 smithi186 ceph-mon[86379]: osdmap e59: 6 total, 5 up, 6 in 2024-04-12T21:03:48.101 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:03:47 smithi018 ceph-mon[112590]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 2 pgs peering) 2024-04-12T21:03:48.101 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:03:47 smithi018 ceph-mon[112590]: osdmap e59: 6 total, 5 up, 6 in 2024-04-12T21:03:49.101 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:03:48 smithi018 ceph-mon[112590]: osdmap e60: 6 total, 5 up, 6 in 2024-04-12T21:03:49.101 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:03:48 smithi018 ceph-mon[112590]: pgmap v78: 65 pgs: 27 active+clean, 17 active+undersized+degraded, 6 active+clean+remapped, 1 active+recovery_wait+undersized+remapped, 13 active+undersized, 1 active+recovering+undersized+remapped; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 54/324 objects degraded (16.667%); 768/324 objects misplaced (237.037%); 8.5 MiB/s, 346 objects/s recovering 2024-04-12T21:03:49.279 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:03:48 smithi186 ceph-mon[86379]: osdmap e60: 6 total, 5 up, 6 in 2024-04-12T21:03:49.280 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:03:48 smithi186 ceph-mon[86379]: pgmap v78: 65 pgs: 27 active+clean, 17 active+undersized+degraded, 6 active+clean+remapped, 1 active+recovery_wait+undersized+remapped, 13 active+undersized, 1 active+recovering+undersized+remapped; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 54/324 objects degraded (16.667%); 768/324 objects misplaced (237.037%); 8.5 MiB/s, 346 objects/s recovering 2024-04-12T21:03:49.850 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:03:49 smithi018 ceph-mon[112590]: osdmap e61: 6 total, 5 up, 6 in 2024-04-12T21:03:50.280 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:03:49 smithi186 ceph-mon[86379]: osdmap e61: 6 total, 5 up, 6 in 2024-04-12T21:03:50.350 INFO:journalctl@ceph.osd.0.smithi018.stdout:Apr 12 21:03:50 smithi018 ceph-1cbe407a-f90d-11ee-bc8d-c7b262605968-osd-0[124276]: 2024-04-12T21:03:50.326+0000 7f2f4d71e7c0 -1 osd.0 49 log_to_monitors true 2024-04-12T21:03:51.089 INFO:journalctl@ceph.osd.0.smithi018.stdout:Apr 12 21:03:50 smithi018 ceph-1cbe407a-f90d-11ee-bc8d-c7b262605968-osd-0[124276]: 2024-04-12T21:03:50.840+0000 7f2f3ca00700 -1 osd.0 49 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-12T21:03:51.089 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:03:50 smithi018 ceph-mon[112590]: osdmap e62: 6 total, 5 up, 6 in 2024-04-12T21:03:51.089 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:03:50 smithi018 ceph-mon[112590]: pgmap v81: 65 pgs: 27 active+clean, 17 active+undersized+degraded, 6 active+clean+remapped, 1 active+recovery_wait+undersized+remapped, 13 active+undersized, 1 active+recovering+undersized+remapped; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 54/324 objects degraded (16.667%); 768/324 objects misplaced (237.037%) 2024-04-12T21:03:51.089 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:03:50 smithi018 ceph-mon[112590]: from='osd.0 [v2:172.21.15.18:6802/2573875940,v1:172.21.15.18:6803/2573875940]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-04-12T21:03:51.280 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:03:50 smithi186 ceph-mon[86379]: osdmap e62: 6 total, 5 up, 6 in 2024-04-12T21:03:51.280 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:03:50 smithi186 ceph-mon[86379]: pgmap v81: 65 pgs: 27 active+clean, 17 active+undersized+degraded, 6 active+clean+remapped, 1 active+recovery_wait+undersized+remapped, 13 active+undersized, 1 active+recovering+undersized+remapped; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 54/324 objects degraded (16.667%); 768/324 objects misplaced (237.037%) 2024-04-12T21:03:51.280 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:03:50 smithi186 ceph-mon[86379]: from='osd.0 [v2:172.21.15.18:6802/2573875940,v1:172.21.15.18:6803/2573875940]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-04-12T21:03:52.030 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:03:51 smithi186 ceph-mon[86379]: from='osd.0 [v2:172.21.15.18:6802/2573875940,v1:172.21.15.18:6803/2573875940]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-04-12T21:03:52.030 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:03:51 smithi186 ceph-mon[86379]: osdmap e63: 6 total, 5 up, 6 in 2024-04-12T21:03:52.030 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:03:51 smithi186 ceph-mon[86379]: from='osd.0 [v2:172.21.15.18:6802/2573875940,v1:172.21.15.18:6803/2573875940]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi018", "root=default"]}]: dispatch 2024-04-12T21:03:52.100 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:03:51 smithi018 ceph-mon[112590]: from='osd.0 [v2:172.21.15.18:6802/2573875940,v1:172.21.15.18:6803/2573875940]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-04-12T21:03:52.101 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:03:51 smithi018 ceph-mon[112590]: osdmap e63: 6 total, 5 up, 6 in 2024-04-12T21:03:52.101 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:03:51 smithi018 ceph-mon[112590]: from='osd.0 [v2:172.21.15.18:6802/2573875940,v1:172.21.15.18:6803/2573875940]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi018", "root=default"]}]: dispatch 2024-04-12T21:03:53.101 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:03:52 smithi018 ceph-mon[112590]: pgmap v83: 65 pgs: 27 active+clean, 17 active+undersized+degraded, 8 active+clean+remapped, 13 active+undersized; 257 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 54/324 objects degraded (16.667%); 12/324 objects misplaced (3.704%); 0 B/s, 266 objects/s recovering 2024-04-12T21:03:53.101 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:03:52 smithi018 ceph-mon[112590]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-04-12T21:03:53.101 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:03:52 smithi018 ceph-mon[112590]: osd.0 [v2:172.21.15.18:6802/2573875940,v1:172.21.15.18:6803/2573875940] boot 2024-04-12T21:03:53.101 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:03:52 smithi018 ceph-mon[112590]: osdmap e64: 6 total, 6 up, 6 in 2024-04-12T21:03:53.102 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:03:52 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-12T21:03:53.102 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:03:52 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' 2024-04-12T21:03:53.102 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:03:52 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' 2024-04-12T21:03:53.102 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:03:52 smithi018 ceph-mon[112590]: osdmap e65: 6 total, 6 up, 6 in 2024-04-12T21:03:53.280 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:03:52 smithi186 ceph-mon[86379]: pgmap v83: 65 pgs: 27 active+clean, 17 active+undersized+degraded, 8 active+clean+remapped, 13 active+undersized; 257 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 54/324 objects degraded (16.667%); 12/324 objects misplaced (3.704%); 0 B/s, 266 objects/s recovering 2024-04-12T21:03:53.280 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:03:52 smithi186 ceph-mon[86379]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-04-12T21:03:53.280 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:03:52 smithi186 ceph-mon[86379]: osd.0 [v2:172.21.15.18:6802/2573875940,v1:172.21.15.18:6803/2573875940] boot 2024-04-12T21:03:53.280 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:03:52 smithi186 ceph-mon[86379]: osdmap e64: 6 total, 6 up, 6 in 2024-04-12T21:03:53.280 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:03:52 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-12T21:03:53.280 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:03:52 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' 2024-04-12T21:03:53.280 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:03:52 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' 2024-04-12T21:03:53.280 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:03:52 smithi186 ceph-mon[86379]: osdmap e65: 6 total, 6 up, 6 in 2024-04-12T21:03:55.100 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:03:54 smithi018 ceph-mon[112590]: pgmap v86: 65 pgs: 27 active+clean, 17 active+undersized+degraded, 8 active+clean+remapped, 13 active+undersized; 257 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 54/324 objects degraded (16.667%); 12/324 objects misplaced (3.704%); 0 B/s, 268 objects/s recovering 2024-04-12T21:03:55.100 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:03:54 smithi018 ceph-mon[112590]: osdmap e66: 6 total, 6 up, 6 in 2024-04-12T21:03:55.100 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:03:54 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' 2024-04-12T21:03:55.101 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:03:54 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' 2024-04-12T21:03:55.101 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:03:54 smithi018 ceph-mon[112590]: osdmap e67: 6 total, 6 up, 6 in 2024-04-12T21:03:55.279 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:03:54 smithi186 ceph-mon[86379]: pgmap v86: 65 pgs: 27 active+clean, 17 active+undersized+degraded, 8 active+clean+remapped, 13 active+undersized; 257 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 54/324 objects degraded (16.667%); 12/324 objects misplaced (3.704%); 0 B/s, 268 objects/s recovering 2024-04-12T21:03:55.280 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:03:54 smithi186 ceph-mon[86379]: osdmap e66: 6 total, 6 up, 6 in 2024-04-12T21:03:55.280 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:03:54 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' 2024-04-12T21:03:55.280 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:03:54 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' 2024-04-12T21:03:55.280 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:03:54 smithi186 ceph-mon[86379]: osdmap e67: 6 total, 6 up, 6 in 2024-04-12T21:03:56.100 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:03:55 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' 2024-04-12T21:03:56.101 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:03:55 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T21:03:56.101 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:03:55 smithi018 ceph-mon[112590]: osdmap e68: 6 total, 6 up, 6 in 2024-04-12T21:03:56.280 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:03:55 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' 2024-04-12T21:03:56.280 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:03:55 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T21:03:56.280 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:03:55 smithi186 ceph-mon[86379]: osdmap e68: 6 total, 6 up, 6 in 2024-04-12T21:03:57.083 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:03:56 smithi018 ceph-mon[112590]: pgmap v90: 65 pgs: 53 active+clean, 2 active+recovery_wait+undersized+degraded+remapped, 3 active+undersized+degraded, 1 active+clean+remapped, 1 active+recovering+undersized+remapped, 5 active+undersized; 257 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 1462/324 objects degraded (451.235%); 2.0 MiB/s, 145 objects/s recovering 2024-04-12T21:03:57.083 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:03:56 smithi018 ceph-mon[112590]: Health check update: Degraded data redundancy: 1462/324 objects degraded (451.235%), 5 pgs degraded (PG_DEGRADED) 2024-04-12T21:03:57.280 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:03:56 smithi186 ceph-mon[86379]: pgmap v90: 65 pgs: 53 active+clean, 2 active+recovery_wait+undersized+degraded+remapped, 3 active+undersized+degraded, 1 active+clean+remapped, 1 active+recovering+undersized+remapped, 5 active+undersized; 257 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 1462/324 objects degraded (451.235%); 2.0 MiB/s, 145 objects/s recovering 2024-04-12T21:03:57.280 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:03:56 smithi186 ceph-mon[86379]: Health check update: Degraded data redundancy: 1462/324 objects degraded (451.235%), 5 pgs degraded (PG_DEGRADED) 2024-04-12T21:03:58.279 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:03:57 smithi186 ceph-mon[86379]: osdmap e69: 6 total, 6 up, 6 in 2024-04-12T21:03:58.351 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:03:57 smithi018 ceph-mon[112590]: osdmap e69: 6 total, 6 up, 6 in 2024-04-12T21:03:59.279 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:03:58 smithi186 ceph-mon[86379]: pgmap v92: 65 pgs: 62 active+clean, 2 active+recovery_wait+undersized+degraded+remapped, 1 active+recovering+undersized+remapped; 257 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 1451/324 objects degraded (447.840%); 2.0 MiB/s, 145 objects/s recovering 2024-04-12T21:03:59.280 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:03:58 smithi186 ceph-mon[86379]: osdmap e70: 6 total, 6 up, 6 in 2024-04-12T21:03:59.350 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:03:58 smithi018 ceph-mon[112590]: pgmap v92: 65 pgs: 62 active+clean, 2 active+recovery_wait+undersized+degraded+remapped, 1 active+recovering+undersized+remapped; 257 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 1451/324 objects degraded (447.840%); 2.0 MiB/s, 145 objects/s recovering 2024-04-12T21:03:59.350 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:03:58 smithi018 ceph-mon[112590]: osdmap e70: 6 total, 6 up, 6 in 2024-04-12T21:04:00.279 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:03:59 smithi186 ceph-mon[86379]: osdmap e71: 6 total, 6 up, 6 in 2024-04-12T21:04:00.280 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:03:59 smithi186 ceph-mon[86379]: osdmap e72: 6 total, 6 up, 6 in 2024-04-12T21:04:00.298 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:03:59 smithi018 ceph-mon[112590]: osdmap e71: 6 total, 6 up, 6 in 2024-04-12T21:04:00.299 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:03:59 smithi018 ceph-mon[112590]: osdmap e72: 6 total, 6 up, 6 in 2024-04-12T21:04:01.745 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:04:01 smithi018 ceph-mon[112590]: pgmap v96: 65 pgs: 62 active+clean, 2 active+recovery_wait+undersized+degraded+remapped, 1 active+recovering+undersized+remapped; 257 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 1451/324 objects degraded (447.840%) 2024-04-12T21:04:01.745 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:04:01 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' 2024-04-12T21:04:01.745 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:04:01 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' 2024-04-12T21:04:01.745 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:04:01 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-12T21:04:01.746 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:04:01 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-12T21:04:01.746 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:04:01 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' 2024-04-12T21:04:01.746 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:04:01 smithi018 ceph-mon[112590]: osdmap e73: 6 total, 6 up, 6 in 2024-04-12T21:04:01.746 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:04:01 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-12T21:04:01.746 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:04:01 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T21:04:01.746 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:04:01 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T21:04:01.746 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:04:01 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T21:04:01.746 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:04:01 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd ok-to-stop", "ids": ["1"], "max": 16}]: dispatch 2024-04-12T21:04:01.780 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:04:01 smithi186 ceph-mon[86379]: pgmap v96: 65 pgs: 62 active+clean, 2 active+recovery_wait+undersized+degraded+remapped, 1 active+recovering+undersized+remapped; 257 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 1451/324 objects degraded (447.840%) 2024-04-12T21:04:01.780 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:04:01 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' 2024-04-12T21:04:01.780 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:04:01 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' 2024-04-12T21:04:01.780 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:04:01 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-12T21:04:01.780 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:04:01 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-12T21:04:01.780 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:04:01 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' 2024-04-12T21:04:01.781 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:04:01 smithi186 ceph-mon[86379]: osdmap e73: 6 total, 6 up, 6 in 2024-04-12T21:04:01.781 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:04:01 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-12T21:04:01.781 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:04:01 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T21:04:01.781 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:04:01 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T21:04:01.781 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:04:01 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T21:04:01.781 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:04:01 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd ok-to-stop", "ids": ["1"], "max": 16}]: dispatch 2024-04-12T21:04:02.601 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:04:02 smithi018 ceph-mon[112590]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["1"], "max": 16}]: dispatch 2024-04-12T21:04:02.601 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:04:02 smithi018 ceph-mon[112590]: Upgrade: osd.1 is safe to restart 2024-04-12T21:04:02.779 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:04:02 smithi186 ceph-mon[86379]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["1"], "max": 16}]: dispatch 2024-04-12T21:04:02.780 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:04:02 smithi186 ceph-mon[86379]: Upgrade: osd.1 is safe to restart 2024-04-12T21:04:03.705 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:04:03 smithi018 ceph-mon[112590]: pgmap v98: 65 pgs: 65 active+clean; 257 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 0 B/s, 386 objects/s recovering 2024-04-12T21:04:03.705 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:04:03 smithi018 ceph-mon[112590]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 1451/324 objects degraded (447.840%), 2 pgs degraded) 2024-04-12T21:04:03.706 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:04:03 smithi018 ceph-mon[112590]: Cluster is now healthy 2024-04-12T21:04:03.706 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:04:03 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' 2024-04-12T21:04:03.706 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:04:03 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-04-12T21:04:03.706 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:04:03 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-12T21:04:03.780 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:04:03 smithi186 ceph-mon[86379]: pgmap v98: 65 pgs: 65 active+clean; 257 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 0 B/s, 386 objects/s recovering 2024-04-12T21:04:03.780 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:04:03 smithi186 ceph-mon[86379]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 1451/324 objects degraded (447.840%), 2 pgs degraded) 2024-04-12T21:04:03.780 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:04:03 smithi186 ceph-mon[86379]: Cluster is now healthy 2024-04-12T21:04:03.780 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:04:03 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' 2024-04-12T21:04:03.780 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:04:03 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-04-12T21:04:03.780 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:04:03 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-12T21:04:04.600 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:04:04 smithi018 ceph-mon[112590]: Upgrade: Updating osd.1 2024-04-12T21:04:04.601 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:04:04 smithi018 ceph-mon[112590]: Deploying daemon osd.1 on smithi018 2024-04-12T21:04:04.779 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:04:04 smithi186 ceph-mon[86379]: Upgrade: Updating osd.1 2024-04-12T21:04:04.780 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:04:04 smithi186 ceph-mon[86379]: Deploying daemon osd.1 on smithi018 2024-04-12T21:04:05.600 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:04:05 smithi018 ceph-mon[112590]: pgmap v99: 65 pgs: 65 active+clean; 257 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 0 B/s, 260 objects/s recovering 2024-04-12T21:04:05.779 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:04:05 smithi186 ceph-mon[86379]: pgmap v99: 65 pgs: 65 active+clean; 257 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 0 B/s, 260 objects/s recovering 2024-04-12T21:04:06.017 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:06.020+0000 7feb30c00700 1 -- 172.21.15.18:0/2093148488 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7feb2c072e00 msgr2=0x7feb2c10d800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:04:06.018 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:06.020+0000 7feb30c00700 1 --2- 172.21.15.18:0/2093148488 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7feb2c072e00 0x7feb2c10d800 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7feb18004940 tx=0x7feb18009e30 comp rx=0 tx=0).stop 2024-04-12T21:04:06.018 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:06.020+0000 7feb30c00700 1 -- 172.21.15.18:0/2093148488 shutdown_connections 2024-04-12T21:04:06.018 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:06.020+0000 7feb30c00700 1 --2- 172.21.15.18:0/2093148488 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7feb2c072e00 0x7feb2c10d800 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:04:06.018 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:06.020+0000 7feb30c00700 1 --2- 172.21.15.18:0/2093148488 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7feb2c0724f0 0x7feb2c0728c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:04:06.018 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:06.020+0000 7feb30c00700 1 -- 172.21.15.18:0/2093148488 >> 172.21.15.18:0/2093148488 conn(0x7feb2c0fac00 msgr2=0x7feb2c0fd010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:04:06.018 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:06.020+0000 7feb30c00700 1 -- 172.21.15.18:0/2093148488 shutdown_connections 2024-04-12T21:04:06.018 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:06.020+0000 7feb30c00700 1 -- 172.21.15.18:0/2093148488 wait complete. 2024-04-12T21:04:06.019 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:06.021+0000 7feb30c00700 1 Processor -- start 2024-04-12T21:04:06.019 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:06.021+0000 7feb30c00700 1 -- start start 2024-04-12T21:04:06.019 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:06.022+0000 7feb30c00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7feb2c0724f0 0x7feb2c119f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:04:06.019 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:06.022+0000 7feb30c00700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7feb2c072e00 0x7feb2c11a440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:04:06.019 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:06.022+0000 7feb30c00700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb2c11aa90 con 0x7feb2c0724f0 2024-04-12T21:04:06.020 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:06.022+0000 7feb30c00700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb2c114070 con 0x7feb2c072e00 2024-04-12T21:04:06.020 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:06.022+0000 7feb29e00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7feb2c0724f0 0x7feb2c119f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:04:06.021 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:06.022+0000 7feb29e00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7feb2c0724f0 0x7feb2c119f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:39604/0 (socket says 172.21.15.18:39604) 2024-04-12T21:04:06.021 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:06.022+0000 7feb29e00700 1 -- 172.21.15.18:0/2932664020 learned_addr learned my addr 172.21.15.18:0/2932664020 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:04:06.021 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:06.023+0000 7feb29400700 1 --2- 172.21.15.18:0/2932664020 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7feb2c072e00 0x7feb2c11a440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:04:06.021 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:06.023+0000 7feb29e00700 1 -- 172.21.15.18:0/2932664020 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7feb2c072e00 msgr2=0x7feb2c11a440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:04:06.021 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:06.023+0000 7feb29e00700 1 --2- 172.21.15.18:0/2932664020 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7feb2c072e00 0x7feb2c11a440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:04:06.021 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:06.023+0000 7feb29e00700 1 -- 172.21.15.18:0/2932664020 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb18005040 con 0x7feb2c0724f0 2024-04-12T21:04:06.021 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:06.023+0000 7feb29e00700 1 --2- 172.21.15.18:0/2932664020 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7feb2c0724f0 0x7feb2c119f00 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7feb1c002a50 tx=0x7feb1c0046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:04:06.021 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:06.024+0000 7feb22000700 1 -- 172.21.15.18:0/2932664020 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7feb1c007cc0 con 0x7feb2c0724f0 2024-04-12T21:04:06.021 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:06.024+0000 7feb30c00700 1 -- 172.21.15.18:0/2932664020 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feb2c114350 con 0x7feb2c0724f0 2024-04-12T21:04:06.021 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:06.024+0000 7feb22000700 1 -- 172.21.15.18:0/2932664020 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7feb1c00f040 con 0x7feb2c0724f0 2024-04-12T21:04:06.021 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:06.024+0000 7feb22000700 1 -- 172.21.15.18:0/2932664020 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7feb1c009640 con 0x7feb2c0724f0 2024-04-12T21:04:06.022 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:06.024+0000 7feb30c00700 1 -- 172.21.15.18:0/2932664020 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feb2c114910 con 0x7feb2c0724f0 2024-04-12T21:04:06.022 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:06.025+0000 7feb30c00700 1 -- 172.21.15.18:0/2932664020 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feb2c073c30 con 0x7feb2c0724f0 2024-04-12T21:04:06.023 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:06.026+0000 7feb22000700 1 -- 172.21.15.18:0/2932664020 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7feb1c0097a0 con 0x7feb2c0724f0 2024-04-12T21:04:06.025 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:06.028+0000 7feb22000700 1 --2- 172.21.15.18:0/2932664020 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7feb1006ec00 0x7feb100710b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:04:06.025 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:06.028+0000 7feb22000700 1 -- 172.21.15.18:0/2932664020 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 6585+0+0 (secure 0 0 0) 0x7feb1c08e610 con 0x7feb2c0724f0 2024-04-12T21:04:06.025 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:06.028+0000 7feb29400700 1 --2- 172.21.15.18:0/2932664020 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7feb1006ec00 0x7feb100710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:04:06.026 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:06.029+0000 7feb29400700 1 --2- 172.21.15.18:0/2932664020 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7feb1006ec00 0x7feb100710b0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7feb2c1157a0 tx=0x7feb18009e30 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:04:06.030 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:06.033+0000 7feb22000700 1 -- 172.21.15.18:0/2932664020 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7feb1c05af10 con 0x7feb2c0724f0 2024-04-12T21:04:06.286 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:06.288+0000 7feb30c00700 1 -- 172.21.15.18:0/2932664020 --> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7feb2c1155c0 con 0x7feb1006ec00 2024-04-12T21:04:06.287 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:06.290+0000 7feb22000700 1 -- 172.21.15.18:0/2932664020 <== mgr.34102 v2:172.21.15.18:6800/3812891919 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7feb2c1155c0 con 0x7feb1006ec00 2024-04-12T21:04:06.292 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:06.294+0000 7feb30c00700 1 -- 172.21.15.18:0/2932664020 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7feb1006ec00 msgr2=0x7feb100710b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:04:06.292 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:06.294+0000 7feb30c00700 1 --2- 172.21.15.18:0/2932664020 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7feb1006ec00 0x7feb100710b0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7feb2c1157a0 tx=0x7feb18009e30 comp rx=0 tx=0).stop 2024-04-12T21:04:06.292 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:06.294+0000 7feb30c00700 1 -- 172.21.15.18:0/2932664020 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7feb2c0724f0 msgr2=0x7feb2c119f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:04:06.292 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:06.294+0000 7feb30c00700 1 --2- 172.21.15.18:0/2932664020 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7feb2c0724f0 0x7feb2c119f00 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7feb1c002a50 tx=0x7feb1c0046b0 comp rx=0 tx=0).stop 2024-04-12T21:04:06.292 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:06.295+0000 7feb30c00700 1 -- 172.21.15.18:0/2932664020 shutdown_connections 2024-04-12T21:04:06.292 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:06.295+0000 7feb30c00700 1 --2- 172.21.15.18:0/2932664020 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7feb1006ec00 0x7feb100710b0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:04:06.292 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:06.295+0000 7feb30c00700 1 --2- 172.21.15.18:0/2932664020 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7feb2c0724f0 0x7feb2c119f00 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:04:06.293 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:06.295+0000 7feb30c00700 1 --2- 172.21.15.18:0/2932664020 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7feb2c072e00 0x7feb2c11a440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:04:06.293 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:06.295+0000 7feb30c00700 1 -- 172.21.15.18:0/2932664020 >> 172.21.15.18:0/2932664020 conn(0x7feb2c0fac00 msgr2=0x7feb2c0fc270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:04:06.293 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:06.295+0000 7feb30c00700 1 -- 172.21.15.18:0/2932664020 shutdown_connections 2024-04-12T21:04:06.293 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:06.295+0000 7feb30c00700 1 -- 172.21.15.18:0/2932664020 wait complete. 2024-04-12T21:04:06.308 INFO:teuthology.orchestra.run.smithi018.stdout:true 2024-04-12T21:04:06.452 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:06.454+0000 7fb51b200700 1 -- 172.21.15.18:0/1502385440 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb514105800 msgr2=0x7fb5140fee70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:04:06.452 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:06.454+0000 7fb51b200700 1 --2- 172.21.15.18:0/1502385440 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb514105800 0x7fb5140fee70 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fb504004940 tx=0x7fb504009e30 comp rx=0 tx=0).stop 2024-04-12T21:04:06.452 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:06.454+0000 7fb51b200700 1 -- 172.21.15.18:0/1502385440 shutdown_connections 2024-04-12T21:04:06.452 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:06.455+0000 7fb51b200700 1 --2- 172.21.15.18:0/1502385440 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb514105800 0x7fb5140fee70 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:04:06.452 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:06.455+0000 7fb51b200700 1 --2- 172.21.15.18:0/1502385440 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb514104e60 0x7fb514105230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:04:06.452 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:06.455+0000 7fb51b200700 1 -- 172.21.15.18:0/1502385440 >> 172.21.15.18:0/1502385440 conn(0x7fb5140fab10 msgr2=0x7fb5140fcef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:04:06.452 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:06.455+0000 7fb51b200700 1 -- 172.21.15.18:0/1502385440 shutdown_connections 2024-04-12T21:04:06.453 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:06.455+0000 7fb51b200700 1 -- 172.21.15.18:0/1502385440 wait complete. 2024-04-12T21:04:06.453 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:06.456+0000 7fb51b200700 1 Processor -- start 2024-04-12T21:04:06.453 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:06.456+0000 7fb51b200700 1 -- start start 2024-04-12T21:04:06.454 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:06.456+0000 7fb51b200700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb514104e60 0x7fb514115c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:04:06.454 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:06.456+0000 7fb51b200700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb514105800 0x7fb514116150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:04:06.454 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:06.456+0000 7fb51b200700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5141167a0 con 0x7fb514105800 2024-04-12T21:04:06.455 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:06.456+0000 7fb51b200700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb51410fb90 con 0x7fb514104e60 2024-04-12T21:04:06.455 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:06.457+0000 7fb519400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb514105800 0x7fb514116150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:04:06.455 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:06.457+0000 7fb519400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb514105800 0x7fb514116150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:39620/0 (socket says 172.21.15.18:39620) 2024-04-12T21:04:06.455 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:06.457+0000 7fb519400700 1 -- 172.21.15.18:0/2015037582 learned_addr learned my addr 172.21.15.18:0/2015037582 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:04:06.455 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:06.457+0000 7fb519e00700 1 --2- 172.21.15.18:0/2015037582 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb514104e60 0x7fb514115c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:04:06.455 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:06.457+0000 7fb519400700 1 -- 172.21.15.18:0/2015037582 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb514104e60 msgr2=0x7fb514115c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:04:06.455 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:06.457+0000 7fb519400700 1 --2- 172.21.15.18:0/2015037582 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb514104e60 0x7fb514115c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:04:06.455 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:06.457+0000 7fb519400700 1 -- 172.21.15.18:0/2015037582 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb504005040 con 0x7fb514105800 2024-04-12T21:04:06.456 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:06.458+0000 7fb519400700 1 --2- 172.21.15.18:0/2015037582 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb514105800 0x7fb514116150 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fb504009100 tx=0x7fb504009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:04:06.456 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:06.458+0000 7fb50e000700 1 -- 172.21.15.18:0/2015037582 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb50400e040 con 0x7fb514105800 2024-04-12T21:04:06.456 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:06.458+0000 7fb50e000700 1 -- 172.21.15.18:0/2015037582 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb504004290 con 0x7fb514105800 2024-04-12T21:04:06.456 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:06.458+0000 7fb51b200700 1 -- 172.21.15.18:0/2015037582 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb51410fe10 con 0x7fb514105800 2024-04-12T21:04:06.456 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:06.458+0000 7fb50e000700 1 -- 172.21.15.18:0/2015037582 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb50401c3f0 con 0x7fb514105800 2024-04-12T21:04:06.456 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:06.458+0000 7fb51b200700 1 -- 172.21.15.18:0/2015037582 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb5141103d0 con 0x7fb514105800 2024-04-12T21:04:06.457 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:06.459+0000 7fb51b200700 1 -- 172.21.15.18:0/2015037582 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb514107330 con 0x7fb514105800 2024-04-12T21:04:06.458 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:06.461+0000 7fb50e000700 1 -- 172.21.15.18:0/2015037582 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7fb504013790 con 0x7fb514105800 2024-04-12T21:04:06.459 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:06.462+0000 7fb50e000700 1 --2- 172.21.15.18:0/2015037582 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fb4fc06ec10 0x7fb4fc0710c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:04:06.460 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:06.463+0000 7fb50e000700 1 -- 172.21.15.18:0/2015037582 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 6585+0+0 (secure 0 0 0) 0x7fb50408fb80 con 0x7fb514105800 2024-04-12T21:04:06.460 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:06.463+0000 7fb519e00700 1 --2- 172.21.15.18:0/2015037582 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fb4fc06ec10 0x7fb4fc0710c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:04:06.461 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:06.464+0000 7fb519e00700 1 --2- 172.21.15.18:0/2015037582 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fb4fc06ec10 0x7fb4fc0710c0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fb508004990 tx=0x7fb508009ee0 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:04:06.465 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:06.468+0000 7fb50e000700 1 -- 172.21.15.18:0/2015037582 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7fb50405c510 con 0x7fb514105800 2024-04-12T21:04:06.720 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:06.723+0000 7fb51b200700 1 -- 172.21.15.18:0/2015037582 --> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fb5141111f0 con 0x7fb4fc06ec10 2024-04-12T21:04:06.722 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:06.725+0000 7fb50e000700 1 -- 172.21.15.18:0/2015037582 <== mgr.34102 v2:172.21.15.18:6800/3812891919 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7fb5141111f0 con 0x7fb4fc06ec10 2024-04-12T21:04:06.725 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:06.728+0000 7fb51b200700 1 -- 172.21.15.18:0/2015037582 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fb4fc06ec10 msgr2=0x7fb4fc0710c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:04:06.725 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:06.728+0000 7fb51b200700 1 --2- 172.21.15.18:0/2015037582 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fb4fc06ec10 0x7fb4fc0710c0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fb508004990 tx=0x7fb508009ee0 comp rx=0 tx=0).stop 2024-04-12T21:04:06.726 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:06.728+0000 7fb51b200700 1 -- 172.21.15.18:0/2015037582 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb514105800 msgr2=0x7fb514116150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:04:06.726 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:06.728+0000 7fb51b200700 1 --2- 172.21.15.18:0/2015037582 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb514105800 0x7fb514116150 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fb504009100 tx=0x7fb504009e30 comp rx=0 tx=0).stop 2024-04-12T21:04:06.726 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:06.728+0000 7fb51b200700 1 -- 172.21.15.18:0/2015037582 shutdown_connections 2024-04-12T21:04:06.726 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:06.728+0000 7fb51b200700 1 --2- 172.21.15.18:0/2015037582 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fb4fc06ec10 0x7fb4fc0710c0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:04:06.726 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:06.728+0000 7fb51b200700 1 --2- 172.21.15.18:0/2015037582 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb514104e60 0x7fb514115c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:04:06.726 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:06.728+0000 7fb51b200700 1 --2- 172.21.15.18:0/2015037582 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb514105800 0x7fb514116150 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:04:06.726 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:06.728+0000 7fb51b200700 1 -- 172.21.15.18:0/2015037582 >> 172.21.15.18:0/2015037582 conn(0x7fb5140fab10 msgr2=0x7fb5140fda00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:04:06.726 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:06.728+0000 7fb51b200700 1 -- 172.21.15.18:0/2015037582 shutdown_connections 2024-04-12T21:04:06.726 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:06.729+0000 7fb51b200700 1 -- 172.21.15.18:0/2015037582 wait complete. 2024-04-12T21:04:06.881 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:06.884+0000 7f0ec7200700 1 -- 172.21.15.18:0/862178035 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0ec8109c40 msgr2=0x7f0ec81032b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:04:06.882 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:06.884+0000 7f0ec7200700 1 --2- 172.21.15.18:0/862178035 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0ec8109c40 0x7f0ec81032b0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f0eb4004940 tx=0x7f0eb4009e30 comp rx=0 tx=0).stop 2024-04-12T21:04:06.882 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:06.884+0000 7f0ec7200700 1 -- 172.21.15.18:0/862178035 shutdown_connections 2024-04-12T21:04:06.882 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:06.884+0000 7f0ec7200700 1 --2- 172.21.15.18:0/862178035 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0ec8109c40 0x7f0ec81032b0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:04:06.882 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:06.884+0000 7f0ec7200700 1 --2- 172.21.15.18:0/862178035 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0ec81092a0 0x7f0ec8109670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:04:06.882 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:06.884+0000 7f0ec7200700 1 -- 172.21.15.18:0/862178035 >> 172.21.15.18:0/862178035 conn(0x7f0ec8072a30 msgr2=0x7f0ec8074e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:04:06.882 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:06.884+0000 7f0ec7200700 1 -- 172.21.15.18:0/862178035 shutdown_connections 2024-04-12T21:04:06.882 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:06.884+0000 7f0ec7200700 1 -- 172.21.15.18:0/862178035 wait complete. 2024-04-12T21:04:06.884 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:06.885+0000 7f0ec7200700 1 Processor -- start 2024-04-12T21:04:06.884 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:06.885+0000 7f0ec7200700 1 -- start start 2024-04-12T21:04:06.884 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:06.885+0000 7f0ec7200700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0ec81092a0 0x7f0ec8119fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:04:06.884 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:06.886+0000 7f0ec7200700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0ec8109c40 0x7f0ec811a500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:04:06.885 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:06.886+0000 7f0ec7200700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ec811ab50 con 0x7f0ec8109c40 2024-04-12T21:04:06.885 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:06.886+0000 7f0ec7200700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ec8113f20 con 0x7f0ec81092a0 2024-04-12T21:04:06.885 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:06.886+0000 7f0ec5400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0ec8109c40 0x7f0ec811a500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:04:06.885 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:06.886+0000 7f0ec5400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0ec8109c40 0x7f0ec811a500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:39652/0 (socket says 172.21.15.18:39652) 2024-04-12T21:04:06.885 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:06.886+0000 7f0ec5400700 1 -- 172.21.15.18:0/268881565 learned_addr learned my addr 172.21.15.18:0/268881565 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:04:06.885 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:06.886+0000 7f0ec5e00700 1 --2- 172.21.15.18:0/268881565 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0ec81092a0 0x7f0ec8119fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:04:06.885 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:06.886+0000 7f0ec5400700 1 -- 172.21.15.18:0/268881565 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0ec81092a0 msgr2=0x7f0ec8119fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:04:06.885 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:06.886+0000 7f0ec5400700 1 --2- 172.21.15.18:0/268881565 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0ec81092a0 0x7f0ec8119fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:04:06.885 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:06.886+0000 7f0ec5400700 1 -- 172.21.15.18:0/268881565 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0eb4005040 con 0x7f0ec8109c40 2024-04-12T21:04:06.885 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:06.887+0000 7f0ec5400700 1 --2- 172.21.15.18:0/268881565 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0ec8109c40 0x7f0ec811a500 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f0eb4009100 tx=0x7f0eb4009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:04:06.885 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:06.887+0000 7f0ebe000700 1 -- 172.21.15.18:0/268881565 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0eb400e040 con 0x7f0ec8109c40 2024-04-12T21:04:06.885 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:06.887+0000 7f0ebe000700 1 -- 172.21.15.18:0/268881565 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f0eb4013420 con 0x7f0ec8109c40 2024-04-12T21:04:06.885 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:06.887+0000 7f0ec7200700 1 -- 172.21.15.18:0/268881565 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0ec81141a0 con 0x7f0ec8109c40 2024-04-12T21:04:06.886 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:06.887+0000 7f0ebe000700 1 -- 172.21.15.18:0/268881565 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0eb401c630 con 0x7f0ec8109c40 2024-04-12T21:04:06.886 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:06.887+0000 7f0ec7200700 1 -- 172.21.15.18:0/268881565 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0ec8114760 con 0x7f0ec8109c40 2024-04-12T21:04:06.886 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:06.889+0000 7f0ec7200700 1 -- 172.21.15.18:0/268881565 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0ec81049f0 con 0x7f0ec8109c40 2024-04-12T21:04:06.887 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:06.890+0000 7f0ebe000700 1 -- 172.21.15.18:0/268881565 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7f0eb4013590 con 0x7f0ec8109c40 2024-04-12T21:04:06.888 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:06.891+0000 7f0ebe000700 1 --2- 172.21.15.18:0/268881565 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f0eac06ec00 0x7f0eac0710b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:04:06.888 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:06.891+0000 7f0ebe000700 1 -- 172.21.15.18:0/268881565 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 6585+0+0 (secure 0 0 0) 0x7f0eb408faf0 con 0x7f0ec8109c40 2024-04-12T21:04:06.888 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:06.891+0000 7f0ec5e00700 1 --2- 172.21.15.18:0/268881565 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f0eac06ec00 0x7f0eac0710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:04:06.889 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:06.892+0000 7f0ec5e00700 1 --2- 172.21.15.18:0/268881565 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f0eac06ec00 0x7f0eac0710b0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f0eb8007fd0 tx=0x7f0eb8009040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:04:06.893 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:06.896+0000 7f0ebe000700 1 -- 172.21.15.18:0/268881565 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f0eb405c3d0 con 0x7f0ec8109c40 2024-04-12T21:04:07.137 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:07.140+0000 7f0ec7200700 1 -- 172.21.15.18:0/268881565 --> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f0ec81103e0 con 0x7f0eac06ec00 2024-04-12T21:04:07.153 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:07.156+0000 7f0ebe000700 1 -- 172.21.15.18:0/268881565 <== mgr.34102 v2:172.21.15.18:6800/3812891919 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7f0ec81103e0 con 0x7f0eac06ec00 2024-04-12T21:04:07.153 INFO:teuthology.orchestra.run.smithi018.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-12T21:04:07.154 INFO:teuthology.orchestra.run.smithi018.stdout:alertmanager.smithi018 smithi018 *:9093,9094 running (4m) 15s ago 19m 14.5M - 0.25.0 c8568f914cd2 c332090068e8 2024-04-12T21:04:07.154 INFO:teuthology.orchestra.run.smithi018.stdout:crash.smithi018 smithi018 running (62s) 15s ago 19m 6665k - 19.0.0-2879-g324baf8d 8717720dc5e5 b7f80d1f7982 2024-04-12T21:04:07.154 INFO:teuthology.orchestra.run.smithi018.stdout:crash.smithi186 smithi186 running (55s) 50s ago 18m 6660k - 19.0.0-2879-g324baf8d 8717720dc5e5 a4f55c81b623 2024-04-12T21:04:07.154 INFO:teuthology.orchestra.run.smithi018.stdout:grafana.smithi018 smithi018 *:3000 running (4m) 15s ago 18m 74.2M - 9.4.12 2bacad6d85d8 b08e3bf176dd 2024-04-12T21:04:07.154 INFO:teuthology.orchestra.run.smithi018.stdout:mds.cephfs.smithi018.acqbsc smithi018 running (12m) 15s ago 12m 112M - 17.2.7-827-gd3b5060a 21de9b7369ad 5b5fce65f18a 2024-04-12T21:04:07.154 INFO:teuthology.orchestra.run.smithi018.stdout:mds.cephfs.smithi018.szjluo smithi018 running (11m) 15s ago 11m 113M - 17.2.7-827-gd3b5060a 21de9b7369ad cf9bf0c5031f 2024-04-12T21:04:07.154 INFO:teuthology.orchestra.run.smithi018.stdout:mds.cephfs.smithi186.jdllyn smithi186 running (12m) 50s ago 12m 14.7M - 17.2.7-827-gd3b5060a 21de9b7369ad 2b2bc46bd29a 2024-04-12T21:04:07.154 INFO:teuthology.orchestra.run.smithi018.stdout:mds.cephfs.smithi186.wjnafs smithi186 running (11m) 50s ago 11m 14.4M - 17.2.7-827-gd3b5060a 21de9b7369ad 74b115d6d6fd 2024-04-12T21:04:07.154 INFO:teuthology.orchestra.run.smithi018.stdout:mgr.smithi018.xjzwga smithi018 *:8443,9283,8765 running (7m) 15s ago 20m 502M - 19.0.0-2879-g324baf8d 8717720dc5e5 56088305eab7 2024-04-12T21:04:07.154 INFO:teuthology.orchestra.run.smithi018.stdout:mgr.smithi186.kqcfxk smithi186 *:8443,9283,8765 running (6m) 50s ago 17m 450M - 19.0.0-2879-g324baf8d 8717720dc5e5 09cd766e5e0d 2024-04-12T21:04:07.154 INFO:teuthology.orchestra.run.smithi018.stdout:mon.smithi018 smithi018 running (2m) 15s ago 21m 46.0M 2048M 19.0.0-2879-g324baf8d 8717720dc5e5 df3e201dfed6 2024-04-12T21:04:07.154 INFO:teuthology.orchestra.run.smithi018.stdout:mon.smithi186 smithi186 running (2m) 50s ago 17m 31.6M 2048M 19.0.0-2879-g324baf8d 8717720dc5e5 c77498da9b2b 2024-04-12T21:04:07.154 INFO:teuthology.orchestra.run.smithi018.stdout:node-exporter.smithi018 smithi018 *:9100 running (5m) 15s ago 19m 8032k - 1.5.0 0da6a335fe13 326c3d9c6847 2024-04-12T21:04:07.155 INFO:teuthology.orchestra.run.smithi018.stdout:node-exporter.smithi186 smithi186 *:9100 running (5m) 50s ago 17m 7818k - 1.5.0 0da6a335fe13 ec2112baeecf 2024-04-12T21:04:07.155 INFO:teuthology.orchestra.run.smithi018.stdout:osd.0 smithi018 running (23s) 15s ago 16m 11.2M 1559M 19.0.0-2879-g324baf8d 8717720dc5e5 60b1273ca047 2024-04-12T21:04:07.155 INFO:teuthology.orchestra.run.smithi018.stdout:osd.1 smithi018 running (16m) 15s ago 16m 252M 1559M 17.2.7-827-gd3b5060a 21de9b7369ad 6517c9cc7f87 2024-04-12T21:04:07.155 INFO:teuthology.orchestra.run.smithi018.stdout:osd.2 smithi018 running (15m) 15s ago 15m 210M 1559M 17.2.7-827-gd3b5060a 21de9b7369ad 8a83692acaa5 2024-04-12T21:04:07.155 INFO:teuthology.orchestra.run.smithi018.stdout:osd.3 smithi186 running (15m) 50s ago 15m 264M 2581M 17.2.7-827-gd3b5060a 21de9b7369ad c2def7a90d76 2024-04-12T21:04:07.155 INFO:teuthology.orchestra.run.smithi018.stdout:osd.4 smithi186 running (14m) 50s ago 14m 233M 2581M 17.2.7-827-gd3b5060a 21de9b7369ad 757e7c71b7b9 2024-04-12T21:04:07.155 INFO:teuthology.orchestra.run.smithi018.stdout:osd.5 smithi186 running (13m) 50s ago 13m 228M 2581M 17.2.7-827-gd3b5060a 21de9b7369ad 2ee072a436d0 2024-04-12T21:04:07.155 INFO:teuthology.orchestra.run.smithi018.stdout:prometheus.smithi018 smithi018 *:9095 running (5m) 15s ago 18m 58.5M - 2.43.0 a07b618ecd1d f58841127e21 2024-04-12T21:04:07.156 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:07.159+0000 7f0ec7200700 1 -- 172.21.15.18:0/268881565 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f0eac06ec00 msgr2=0x7f0eac0710b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:04:07.156 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:07.159+0000 7f0ec7200700 1 --2- 172.21.15.18:0/268881565 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f0eac06ec00 0x7f0eac0710b0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f0eb8007fd0 tx=0x7f0eb8009040 comp rx=0 tx=0).stop 2024-04-12T21:04:07.156 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:07.159+0000 7f0ec7200700 1 -- 172.21.15.18:0/268881565 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0ec8109c40 msgr2=0x7f0ec811a500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:04:07.156 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:07.159+0000 7f0ec7200700 1 --2- 172.21.15.18:0/268881565 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0ec8109c40 0x7f0ec811a500 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f0eb4009100 tx=0x7f0eb4009e30 comp rx=0 tx=0).stop 2024-04-12T21:04:07.156 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:07.159+0000 7f0ec7200700 1 -- 172.21.15.18:0/268881565 shutdown_connections 2024-04-12T21:04:07.156 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:07.159+0000 7f0ec7200700 1 --2- 172.21.15.18:0/268881565 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f0eac06ec00 0x7f0eac0710b0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:04:07.157 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:07.159+0000 7f0ec7200700 1 --2- 172.21.15.18:0/268881565 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0ec81092a0 0x7f0ec8119fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:04:07.157 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:07.159+0000 7f0ec7200700 1 --2- 172.21.15.18:0/268881565 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0ec8109c40 0x7f0ec811a500 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:04:07.157 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:07.159+0000 7f0ec7200700 1 -- 172.21.15.18:0/268881565 >> 172.21.15.18:0/268881565 conn(0x7f0ec8072a30 msgr2=0x7f0ec8074910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:04:07.157 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:07.159+0000 7f0ec7200700 1 -- 172.21.15.18:0/268881565 shutdown_connections 2024-04-12T21:04:07.157 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:07.159+0000 7f0ec7200700 1 -- 172.21.15.18:0/268881565 wait complete. 2024-04-12T21:04:07.299 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:07.301+0000 7fa756600700 1 -- 172.21.15.18:0/2648924920 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa7500fc580 msgr2=0x7fa750100ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:04:07.299 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:07.301+0000 7fa756600700 1 --2- 172.21.15.18:0/2648924920 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa7500fc580 0x7fa750100ba0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fa73c004940 tx=0x7fa73c009e30 comp rx=0 tx=0).stop 2024-04-12T21:04:07.299 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:07.302+0000 7fa756600700 1 -- 172.21.15.18:0/2648924920 shutdown_connections 2024-04-12T21:04:07.299 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:07.302+0000 7fa756600700 1 --2- 172.21.15.18:0/2648924920 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa7500fc580 0x7fa750100ba0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:04:07.299 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:07.302+0000 7fa756600700 1 --2- 172.21.15.18:0/2648924920 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa7500fbbe0 0x7fa7500fbfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:04:07.300 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:07.302+0000 7fa756600700 1 -- 172.21.15.18:0/2648924920 >> 172.21.15.18:0/2648924920 conn(0x7fa7500f78e0 msgr2=0x7fa7500f9cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:04:07.300 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:07.302+0000 7fa756600700 1 -- 172.21.15.18:0/2648924920 shutdown_connections 2024-04-12T21:04:07.300 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:07.302+0000 7fa756600700 1 -- 172.21.15.18:0/2648924920 wait complete. 2024-04-12T21:04:07.300 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:07.302+0000 7fa756600700 1 Processor -- start 2024-04-12T21:04:07.301 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:07.303+0000 7fa756600700 1 -- start start 2024-04-12T21:04:07.301 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:07.303+0000 7fa756600700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa7500fbbe0 0x7fa75011a020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:04:07.301 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:07.303+0000 7fa756600700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa7500fc580 0x7fa75011a560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:04:07.301 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:07.303+0000 7fa756600700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa75011abb0 con 0x7fa7500fbbe0 2024-04-12T21:04:07.301 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:07.303+0000 7fa756600700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa750114190 con 0x7fa7500fc580 2024-04-12T21:04:07.301 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:07.303+0000 7fa74f400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa7500fbbe0 0x7fa75011a020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:04:07.301 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:07.303+0000 7fa74f400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa7500fbbe0 0x7fa75011a020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:39684/0 (socket says 172.21.15.18:39684) 2024-04-12T21:04:07.301 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:07.303+0000 7fa74f400700 1 -- 172.21.15.18:0/243569063 learned_addr learned my addr 172.21.15.18:0/243569063 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:04:07.301 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:07.304+0000 7fa74f400700 1 -- 172.21.15.18:0/243569063 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa7500fc580 msgr2=0x7fa75011a560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:04:07.301 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:07.304+0000 7fa74f400700 1 --2- 172.21.15.18:0/243569063 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa7500fc580 0x7fa75011a560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:04:07.302 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:07.304+0000 7fa74f400700 1 -- 172.21.15.18:0/243569063 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa73c005040 con 0x7fa7500fbbe0 2024-04-12T21:04:07.302 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:07.304+0000 7fa74f400700 1 --2- 172.21.15.18:0/243569063 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa7500fbbe0 0x7fa75011a020 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fa74000c8c0 tx=0x7fa7400044c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:04:07.303 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:07.304+0000 7fa747400700 1 -- 172.21.15.18:0/243569063 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa74000a980 con 0x7fa7500fbbe0 2024-04-12T21:04:07.303 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:07.304+0000 7fa747400700 1 -- 172.21.15.18:0/243569063 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa74000ce60 con 0x7fa7500fbbe0 2024-04-12T21:04:07.303 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:07.304+0000 7fa756600700 1 -- 172.21.15.18:0/243569063 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa750114470 con 0x7fa7500fbbe0 2024-04-12T21:04:07.303 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:07.304+0000 7fa747400700 1 -- 172.21.15.18:0/243569063 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa7400115e0 con 0x7fa7500fbbe0 2024-04-12T21:04:07.303 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:07.304+0000 7fa756600700 1 -- 172.21.15.18:0/243569063 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa7501149b0 con 0x7fa7500fbbe0 2024-04-12T21:04:07.303 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:07.305+0000 7fa756600700 1 -- 172.21.15.18:0/243569063 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa750066770 con 0x7fa7500fbbe0 2024-04-12T21:04:07.303 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:07.307+0000 7fa747400700 1 -- 172.21.15.18:0/243569063 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7fa740012400 con 0x7fa7500fbbe0 2024-04-12T21:04:07.305 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:07.308+0000 7fa747400700 1 --2- 172.21.15.18:0/243569063 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fa73406ec00 0x7fa7340710b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:04:07.305 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:07.308+0000 7fa747400700 1 -- 172.21.15.18:0/243569063 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 6585+0+0 (secure 0 0 0) 0x7fa74008e9e0 con 0x7fa7500fbbe0 2024-04-12T21:04:07.306 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:07.309+0000 7fa74ea00700 1 --2- 172.21.15.18:0/243569063 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fa73406ec00 0x7fa7340710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:04:07.306 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:07.309+0000 7fa74ea00700 1 --2- 172.21.15.18:0/243569063 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fa73406ec00 0x7fa7340710b0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fa73c004f90 tx=0x7fa73c009e30 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:04:07.310 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:07.314+0000 7fa747400700 1 -- 172.21.15.18:0/243569063 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7fa74005b2e0 con 0x7fa7500fbbe0 2024-04-12T21:04:07.601 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:04:07 smithi018 ceph-mon[112590]: pgmap v100: 65 pgs: 65 active+clean; 257 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 0 B/s, 223 objects/s recovering 2024-04-12T21:04:07.601 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:04:07 smithi018 ceph-mon[112590]: from='client.34248 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:04:07.625 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:07.628+0000 7fa756600700 1 -- 172.21.15.18:0/243569063 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fa750114c30 con 0x7fa7500fbbe0 2024-04-12T21:04:07.626 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:07.629+0000 7fa747400700 1 -- 172.21.15.18:0/243569063 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+839 (secure 0 0 0) 0x7fa7400126b0 con 0x7fa7500fbbe0 2024-04-12T21:04:07.626 INFO:teuthology.orchestra.run.smithi018.stdout:{ 2024-04-12T21:04:07.626 INFO:teuthology.orchestra.run.smithi018.stdout: "mon": { 2024-04-12T21:04:07.626 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 19.0.0-2879-g324baf8d (324baf8ddb91dea46cac50cb535c306115791355) squid (dev)": 2 2024-04-12T21:04:07.626 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-12T21:04:07.627 INFO:teuthology.orchestra.run.smithi018.stdout: "mgr": { 2024-04-12T21:04:07.627 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 19.0.0-2879-g324baf8d (324baf8ddb91dea46cac50cb535c306115791355) squid (dev)": 2 2024-04-12T21:04:07.627 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-12T21:04:07.627 INFO:teuthology.orchestra.run.smithi018.stdout: "osd": { 2024-04-12T21:04:07.627 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 17.2.7-827-gd3b5060a (d3b5060a17449edfae9385cdf792afec109fca5d) quincy (stable)": 5, 2024-04-12T21:04:07.627 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 19.0.0-2879-g324baf8d (324baf8ddb91dea46cac50cb535c306115791355) squid (dev)": 1 2024-04-12T21:04:07.627 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-12T21:04:07.627 INFO:teuthology.orchestra.run.smithi018.stdout: "mds": { 2024-04-12T21:04:07.627 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 17.2.7-827-gd3b5060a (d3b5060a17449edfae9385cdf792afec109fca5d) quincy (stable)": 4 2024-04-12T21:04:07.627 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-12T21:04:07.627 INFO:teuthology.orchestra.run.smithi018.stdout: "overall": { 2024-04-12T21:04:07.627 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 17.2.7-827-gd3b5060a (d3b5060a17449edfae9385cdf792afec109fca5d) quincy (stable)": 9, 2024-04-12T21:04:07.627 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 19.0.0-2879-g324baf8d (324baf8ddb91dea46cac50cb535c306115791355) squid (dev)": 5 2024-04-12T21:04:07.627 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-04-12T21:04:07.627 INFO:teuthology.orchestra.run.smithi018.stdout:} 2024-04-12T21:04:07.628 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:07.631+0000 7fa756600700 1 -- 172.21.15.18:0/243569063 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fa73406ec00 msgr2=0x7fa7340710b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:04:07.628 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:07.631+0000 7fa756600700 1 --2- 172.21.15.18:0/243569063 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fa73406ec00 0x7fa7340710b0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fa73c004f90 tx=0x7fa73c009e30 comp rx=0 tx=0).stop 2024-04-12T21:04:07.629 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:07.631+0000 7fa756600700 1 -- 172.21.15.18:0/243569063 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa7500fbbe0 msgr2=0x7fa75011a020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:04:07.629 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:07.631+0000 7fa756600700 1 --2- 172.21.15.18:0/243569063 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa7500fbbe0 0x7fa75011a020 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fa74000c8c0 tx=0x7fa7400044c0 comp rx=0 tx=0).stop 2024-04-12T21:04:07.629 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:07.631+0000 7fa756600700 1 -- 172.21.15.18:0/243569063 shutdown_connections 2024-04-12T21:04:07.629 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:07.631+0000 7fa756600700 1 --2- 172.21.15.18:0/243569063 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fa73406ec00 0x7fa7340710b0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:04:07.629 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:07.631+0000 7fa756600700 1 --2- 172.21.15.18:0/243569063 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa7500fbbe0 0x7fa75011a020 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:04:07.629 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:07.632+0000 7fa756600700 1 --2- 172.21.15.18:0/243569063 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa7500fc580 0x7fa75011a560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:04:07.629 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:07.632+0000 7fa756600700 1 -- 172.21.15.18:0/243569063 >> 172.21.15.18:0/243569063 conn(0x7fa7500f78e0 msgr2=0x7fa7500f8f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:04:07.629 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:07.632+0000 7fa756600700 1 -- 172.21.15.18:0/243569063 shutdown_connections 2024-04-12T21:04:07.630 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:07.632+0000 7fa756600700 1 -- 172.21.15.18:0/243569063 wait complete. 2024-04-12T21:04:07.778 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:07.780+0000 7f2405000700 1 -- 172.21.15.18:0/877720195 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f24001037b0 msgr2=0x7f24000fb3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:04:07.778 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:07.780+0000 7f2405000700 1 --2- 172.21.15.18:0/877720195 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f24001037b0 0x7f24000fb3b0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f23ec004940 tx=0x7f23ec009e30 comp rx=0 tx=0).stop 2024-04-12T21:04:07.778 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:07.781+0000 7f2405000700 1 -- 172.21.15.18:0/877720195 shutdown_connections 2024-04-12T21:04:07.778 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:07.781+0000 7f2405000700 1 --2- 172.21.15.18:0/877720195 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f24001037b0 0x7f24000fb3b0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:04:07.778 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:07.781+0000 7f2405000700 1 --2- 172.21.15.18:0/877720195 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2400102e10 0x7f24001031e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:04:07.779 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:07.781+0000 7f2405000700 1 -- 172.21.15.18:0/877720195 >> 172.21.15.18:0/877720195 conn(0x7f2400072310 msgr2=0x7f2400072710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:04:07.779 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:07.781+0000 7f2405000700 1 -- 172.21.15.18:0/877720195 shutdown_connections 2024-04-12T21:04:07.779 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:07.781+0000 7f2405000700 1 -- 172.21.15.18:0/877720195 wait complete. 2024-04-12T21:04:07.779 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:04:07 smithi186 ceph-mon[86379]: pgmap v100: 65 pgs: 65 active+clean; 257 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 0 B/s, 223 objects/s recovering 2024-04-12T21:04:07.779 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:04:07 smithi186 ceph-mon[86379]: from='client.34248 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:04:07.781 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:07.782+0000 7f2405000700 1 Processor -- start 2024-04-12T21:04:07.781 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:07.782+0000 7f2405000700 1 -- start start 2024-04-12T21:04:07.781 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:07.782+0000 7f2405000700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2400102e10 0x7f2400115bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:04:07.781 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:07.782+0000 7f2405000700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f24001037b0 0x7f24001160f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:04:07.781 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:07.782+0000 7f2405000700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2400116740 con 0x7f24001037b0 2024-04-12T21:04:07.781 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:07.782+0000 7f2405000700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f240010fd70 con 0x7f2400102e10 2024-04-12T21:04:07.782 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:07.783+0000 7f23fd400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f24001037b0 0x7f24001160f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:04:07.782 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:07.783+0000 7f23fd400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f24001037b0 0x7f24001160f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:39708/0 (socket says 172.21.15.18:39708) 2024-04-12T21:04:07.782 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:07.783+0000 7f23fd400700 1 -- 172.21.15.18:0/347007342 learned_addr learned my addr 172.21.15.18:0/347007342 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:04:07.782 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:07.783+0000 7f23fde00700 1 --2- 172.21.15.18:0/347007342 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2400102e10 0x7f2400115bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:04:07.782 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:07.783+0000 7f23fd400700 1 -- 172.21.15.18:0/347007342 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2400102e10 msgr2=0x7f2400115bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:04:07.782 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:07.783+0000 7f23fd400700 1 --2- 172.21.15.18:0/347007342 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2400102e10 0x7f2400115bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:04:07.782 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:07.783+0000 7f23fd400700 1 -- 172.21.15.18:0/347007342 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23ec005040 con 0x7f24001037b0 2024-04-12T21:04:07.782 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:07.783+0000 7f23fd400700 1 --2- 172.21.15.18:0/347007342 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f24001037b0 0x7f24001160f0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f23ec009180 tx=0x7f23ec009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:04:07.782 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:07.784+0000 7f23f6000700 1 -- 172.21.15.18:0/347007342 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f23ec00e040 con 0x7f24001037b0 2024-04-12T21:04:07.782 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:07.784+0000 7f2405000700 1 -- 172.21.15.18:0/347007342 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f240010fff0 con 0x7f24001037b0 2024-04-12T21:04:07.782 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:07.784+0000 7f23f6000700 1 -- 172.21.15.18:0/347007342 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f23ec00bbc0 con 0x7f24001037b0 2024-04-12T21:04:07.782 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:07.784+0000 7f2405000700 1 -- 172.21.15.18:0/347007342 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2400110530 con 0x7f24001037b0 2024-04-12T21:04:07.782 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:07.784+0000 7f23f6000700 1 -- 172.21.15.18:0/347007342 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f23ec0255e0 con 0x7f24001037b0 2024-04-12T21:04:07.783 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:07.785+0000 7f2405000700 1 -- 172.21.15.18:0/347007342 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2400106a90 con 0x7f24001037b0 2024-04-12T21:04:07.783 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:07.786+0000 7f23f6000700 1 -- 172.21.15.18:0/347007342 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7f23ec00bd30 con 0x7f24001037b0 2024-04-12T21:04:07.784 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:07.787+0000 7f23f6000700 1 --2- 172.21.15.18:0/347007342 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f23e406ec00 0x7f23e40710b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:04:07.784 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:07.787+0000 7f23f6000700 1 -- 172.21.15.18:0/347007342 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 6585+0+0 (secure 0 0 0) 0x7f23ec08fc40 con 0x7f24001037b0 2024-04-12T21:04:07.784 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:07.787+0000 7f23fde00700 1 --2- 172.21.15.18:0/347007342 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f23e406ec00 0x7f23e40710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:04:07.785 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:07.788+0000 7f23fde00700 1 --2- 172.21.15.18:0/347007342 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f23e406ec00 0x7f23e40710b0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f23f0009fd0 tx=0x7f23f0009f00 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:04:07.789 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:07.792+0000 7f23f6000700 1 -- 172.21.15.18:0/347007342 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f23ec05c4a0 con 0x7f24001037b0 2024-04-12T21:04:08.099 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:08.101+0000 7f2405000700 1 -- 172.21.15.18:0/347007342 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f2400111200 con 0x7f24001037b0 2024-04-12T21:04:08.100 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:08.102+0000 7f23f6000700 1 -- 172.21.15.18:0/347007342 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 23 v23) v1 ==== 76+0+1886 (secure 0 0 0) 0x7f23ec08f770 con 0x7f24001037b0 2024-04-12T21:04:08.100 INFO:teuthology.orchestra.run.smithi018.stdout:e23 2024-04-12T21:04:08.100 INFO:teuthology.orchestra.run.smithi018.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-12T21:04:08.100 INFO:teuthology.orchestra.run.smithi018.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-12T21:04:08.101 INFO:teuthology.orchestra.run.smithi018.stdout:legacy client fscid: 1 2024-04-12T21:04:08.101 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-12T21:04:08.101 INFO:teuthology.orchestra.run.smithi018.stdout:Filesystem 'cephfs' (1) 2024-04-12T21:04:08.101 INFO:teuthology.orchestra.run.smithi018.stdout:fs_name cephfs 2024-04-12T21:04:08.101 INFO:teuthology.orchestra.run.smithi018.stdout:epoch 23 2024-04-12T21:04:08.101 INFO:teuthology.orchestra.run.smithi018.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-04-12T21:04:08.101 INFO:teuthology.orchestra.run.smithi018.stdout:created 2024-04-12T20:51:57.362850+0000 2024-04-12T21:04:08.101 INFO:teuthology.orchestra.run.smithi018.stdout:modified 2024-04-12T21:01:10.670287+0000 2024-04-12T21:04:08.101 INFO:teuthology.orchestra.run.smithi018.stdout:tableserver 0 2024-04-12T21:04:08.101 INFO:teuthology.orchestra.run.smithi018.stdout:root 0 2024-04-12T21:04:08.101 INFO:teuthology.orchestra.run.smithi018.stdout:session_timeout 60 2024-04-12T21:04:08.101 INFO:teuthology.orchestra.run.smithi018.stdout:session_autoclose 300 2024-04-12T21:04:08.101 INFO:teuthology.orchestra.run.smithi018.stdout:max_file_size 1099511627776 2024-04-12T21:04:08.102 INFO:teuthology.orchestra.run.smithi018.stdout:max_xattr_size 65536 2024-04-12T21:04:08.102 INFO:teuthology.orchestra.run.smithi018.stdout:required_client_features {} 2024-04-12T21:04:08.102 INFO:teuthology.orchestra.run.smithi018.stdout:last_failure 0 2024-04-12T21:04:08.102 INFO:teuthology.orchestra.run.smithi018.stdout:last_failure_osd_epoch 46 2024-04-12T21:04:08.102 INFO:teuthology.orchestra.run.smithi018.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-12T21:04:08.102 INFO:teuthology.orchestra.run.smithi018.stdout:max_mds 2 2024-04-12T21:04:08.102 INFO:teuthology.orchestra.run.smithi018.stdout:in 0,1 2024-04-12T21:04:08.102 INFO:teuthology.orchestra.run.smithi018.stdout:up {0=14452,1=14464} 2024-04-12T21:04:08.102 INFO:teuthology.orchestra.run.smithi018.stdout:failed 2024-04-12T21:04:08.102 INFO:teuthology.orchestra.run.smithi018.stdout:damaged 2024-04-12T21:04:08.102 INFO:teuthology.orchestra.run.smithi018.stdout:stopped 2024-04-12T21:04:08.102 INFO:teuthology.orchestra.run.smithi018.stdout:data_pools [3] 2024-04-12T21:04:08.102 INFO:teuthology.orchestra.run.smithi018.stdout:metadata_pool 2 2024-04-12T21:04:08.102 INFO:teuthology.orchestra.run.smithi018.stdout:inline_data disabled 2024-04-12T21:04:08.103 INFO:teuthology.orchestra.run.smithi018.stdout:balancer 2024-04-12T21:04:08.103 INFO:teuthology.orchestra.run.smithi018.stdout:bal_rank_mask -1 2024-04-12T21:04:08.103 INFO:teuthology.orchestra.run.smithi018.stdout:standby_count_wanted 1 2024-04-12T21:04:08.103 INFO:teuthology.orchestra.run.smithi018.stdout:qdb_cluster leader: 0 members: 2024-04-12T21:04:08.103 INFO:teuthology.orchestra.run.smithi018.stdout:[mds.cephfs.smithi018.acqbsc{0:14452} state up:active seq 5 join_fscid=1 addr [v2:172.21.15.18:6826/1564065434,v1:172.21.15.18:6827/1564065434] compat {c=[1],r=[1],i=[7ff]}] 2024-04-12T21:04:08.103 INFO:teuthology.orchestra.run.smithi018.stdout:[mds.cephfs.smithi018.szjluo{1:14464} state up:active seq 6 join_fscid=1 addr [v2:172.21.15.18:6828/722443353,v1:172.21.15.18:6829/722443353] compat {c=[1],r=[1],i=[7ff]}] 2024-04-12T21:04:08.103 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-12T21:04:08.103 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-12T21:04:08.103 INFO:teuthology.orchestra.run.smithi018.stdout:Standby daemons: 2024-04-12T21:04:08.103 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-12T21:04:08.103 INFO:teuthology.orchestra.run.smithi018.stdout:[mds.cephfs.smithi186.wjnafs{-1:24277} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.186:6826/438347207,v1:172.21.15.186:6827/438347207] compat {c=[1],r=[1],i=[7ff]}] 2024-04-12T21:04:08.103 INFO:teuthology.orchestra.run.smithi018.stdout:[mds.cephfs.smithi186.jdllyn{-1:24281} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.186:6824/1464359952,v1:172.21.15.186:6825/1464359952] compat {c=[1],r=[1],i=[7ff]}] 2024-04-12T21:04:08.104 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:08.105+0000 7f2405000700 1 -- 172.21.15.18:0/347007342 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f23e406ec00 msgr2=0x7f23e40710b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:04:08.104 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:08.105+0000 7f2405000700 1 --2- 172.21.15.18:0/347007342 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f23e406ec00 0x7f23e40710b0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f23f0009fd0 tx=0x7f23f0009f00 comp rx=0 tx=0).stop 2024-04-12T21:04:08.104 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:08.105+0000 7f2405000700 1 -- 172.21.15.18:0/347007342 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f24001037b0 msgr2=0x7f24001160f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:04:08.104 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:08.105+0000 7f2405000700 1 --2- 172.21.15.18:0/347007342 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f24001037b0 0x7f24001160f0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f23ec009180 tx=0x7f23ec009e30 comp rx=0 tx=0).stop 2024-04-12T21:04:08.104 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:08.106+0000 7f2405000700 1 -- 172.21.15.18:0/347007342 shutdown_connections 2024-04-12T21:04:08.104 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:08.106+0000 7f2405000700 1 --2- 172.21.15.18:0/347007342 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f23e406ec00 0x7f23e40710b0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:04:08.104 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:08.106+0000 7f2405000700 1 --2- 172.21.15.18:0/347007342 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2400102e10 0x7f2400115bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:04:08.105 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:08.106+0000 7f2405000700 1 --2- 172.21.15.18:0/347007342 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f24001037b0 0x7f24001160f0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:04:08.105 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:08.106+0000 7f2405000700 1 -- 172.21.15.18:0/347007342 >> 172.21.15.18:0/347007342 conn(0x7f2400072310 msgr2=0x7f24000fac40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:04:08.105 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:08.106+0000 7f2405000700 1 -- 172.21.15.18:0/347007342 shutdown_connections 2024-04-12T21:04:08.105 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:08.106+0000 7f2405000700 1 -- 172.21.15.18:0/347007342 wait complete. 2024-04-12T21:04:08.105 INFO:teuthology.orchestra.run.smithi018.stderr:dumped fsmap epoch 23 2024-04-12T21:04:08.254 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:08.257+0000 7f7950c00700 1 -- 172.21.15.18:0/832898844 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f794c1037a0 msgr2=0x7f794c0fce10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:04:08.254 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:08.257+0000 7f7950c00700 1 --2- 172.21.15.18:0/832898844 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f794c1037a0 0x7f794c0fce10 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f7938004940 tx=0x7f7938009e30 comp rx=0 tx=0).stop 2024-04-12T21:04:08.255 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:08.257+0000 7f7950c00700 1 -- 172.21.15.18:0/832898844 shutdown_connections 2024-04-12T21:04:08.255 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:08.257+0000 7f7950c00700 1 --2- 172.21.15.18:0/832898844 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f794c1037a0 0x7f794c0fce10 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:04:08.255 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:08.257+0000 7f7950c00700 1 --2- 172.21.15.18:0/832898844 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f794c102e00 0x7f794c1031d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:04:08.255 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:08.257+0000 7f7950c00700 1 -- 172.21.15.18:0/832898844 >> 172.21.15.18:0/832898844 conn(0x7f794c0f89b0 msgr2=0x7f794c0fadc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:04:08.255 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:08.257+0000 7f7950c00700 1 -- 172.21.15.18:0/832898844 shutdown_connections 2024-04-12T21:04:08.255 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:08.257+0000 7f7950c00700 1 -- 172.21.15.18:0/832898844 wait complete. 2024-04-12T21:04:08.256 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:08.258+0000 7f7950c00700 1 Processor -- start 2024-04-12T21:04:08.256 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:08.258+0000 7f7950c00700 1 -- start start 2024-04-12T21:04:08.256 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:08.259+0000 7f7950c00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f794c102e00 0x7f794c115bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:04:08.256 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:08.259+0000 7f7950c00700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f794c1037a0 0x7f794c116110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:04:08.256 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:08.259+0000 7f7950c00700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f794c116760 con 0x7f794c102e00 2024-04-12T21:04:08.256 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:08.259+0000 7f7950c00700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f794c10fd40 con 0x7f794c1037a0 2024-04-12T21:04:08.257 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:08.259+0000 7f7949e00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f794c102e00 0x7f794c115bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:04:08.257 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:08.259+0000 7f7949e00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f794c102e00 0x7f794c115bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:39738/0 (socket says 172.21.15.18:39738) 2024-04-12T21:04:08.257 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:08.259+0000 7f7949e00700 1 -- 172.21.15.18:0/1810025467 learned_addr learned my addr 172.21.15.18:0/1810025467 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:04:08.257 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:08.259+0000 7f7949400700 1 --2- 172.21.15.18:0/1810025467 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f794c1037a0 0x7f794c116110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:04:08.257 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:08.259+0000 7f7949e00700 1 -- 172.21.15.18:0/1810025467 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f794c1037a0 msgr2=0x7f794c116110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:04:08.258 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:08.259+0000 7f7949e00700 1 --2- 172.21.15.18:0/1810025467 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f794c1037a0 0x7f794c116110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:04:08.258 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:08.259+0000 7f7949e00700 1 -- 172.21.15.18:0/1810025467 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7938005040 con 0x7f794c102e00 2024-04-12T21:04:08.258 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:08.260+0000 7f7949e00700 1 --2- 172.21.15.18:0/1810025467 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f794c102e00 0x7f794c115bd0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f793c00c830 tx=0x7f793c004470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:04:08.258 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:08.260+0000 7f7942000700 1 -- 172.21.15.18:0/1810025467 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f793c00a760 con 0x7f794c102e00 2024-04-12T21:04:08.258 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:08.260+0000 7f7942000700 1 -- 172.21.15.18:0/1810025467 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f793c00ad90 con 0x7f794c102e00 2024-04-12T21:04:08.258 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:08.260+0000 7f7942000700 1 -- 172.21.15.18:0/1810025467 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f793c0113a0 con 0x7f794c102e00 2024-04-12T21:04:08.258 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:08.260+0000 7f7950c00700 1 -- 172.21.15.18:0/1810025467 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f794c110020 con 0x7f794c102e00 2024-04-12T21:04:08.258 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:08.260+0000 7f7950c00700 1 -- 172.21.15.18:0/1810025467 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f794c110560 con 0x7f794c102e00 2024-04-12T21:04:08.258 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:08.261+0000 7f7950c00700 1 -- 172.21.15.18:0/1810025467 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f794c102120 con 0x7f794c102e00 2024-04-12T21:04:08.260 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:08.263+0000 7f7942000700 1 -- 172.21.15.18:0/1810025467 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7f793c012400 con 0x7f794c102e00 2024-04-12T21:04:08.261 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:08.264+0000 7f7942000700 1 --2- 172.21.15.18:0/1810025467 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f793006ec00 0x7f79300710b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:04:08.261 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:08.264+0000 7f7942000700 1 -- 172.21.15.18:0/1810025467 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 6585+0+0 (secure 0 0 0) 0x7f793c08e910 con 0x7f794c102e00 2024-04-12T21:04:08.261 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:08.264+0000 7f7949400700 1 --2- 172.21.15.18:0/1810025467 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f793006ec00 0x7f79300710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:04:08.261 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:08.265+0000 7f7949400700 1 --2- 172.21.15.18:0/1810025467 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f793006ec00 0x7f79300710b0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f793800b020 tx=0x7f7938009e30 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:04:08.265 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:08.268+0000 7f7942000700 1 -- 172.21.15.18:0/1810025467 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f793c05b1f0 con 0x7f794c102e00 2024-04-12T21:04:08.350 INFO:journalctl@ceph.osd.1.smithi018.stdout:Apr 12 21:04:08 smithi018 systemd[1]: Stopping Ceph osd.1 for 1cbe407a-f90d-11ee-bc8d-c7b262605968... 2024-04-12T21:04:08.573 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:08.576+0000 7f7950c00700 1 -- 172.21.15.18:0/1810025467 --> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f794c109f80 con 0x7f793006ec00 2024-04-12T21:04:08.576 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:08.579+0000 7f7942000700 1 -- 172.21.15.18:0/1810025467 <== mgr.34102 v2:172.21.15.18:6800/3812891919 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7f7938008e30 con 0x7f793006ec00 2024-04-12T21:04:08.576 INFO:teuthology.orchestra.run.smithi018.stdout:{ 2024-04-12T21:04:08.576 INFO:teuthology.orchestra.run.smithi018.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:f4452def3be4d350f4a8e5feaf7ebb1511be9298fe68478ba7a2d6fbd7d69a63", 2024-04-12T21:04:08.576 INFO:teuthology.orchestra.run.smithi018.stdout: "in_progress": true, 2024-04-12T21:04:08.576 INFO:teuthology.orchestra.run.smithi018.stdout: "which": "Upgrading all daemon types on all hosts", 2024-04-12T21:04:08.576 INFO:teuthology.orchestra.run.smithi018.stdout: "services_complete": [ 2024-04-12T21:04:08.576 INFO:teuthology.orchestra.run.smithi018.stdout: "crash", 2024-04-12T21:04:08.576 INFO:teuthology.orchestra.run.smithi018.stdout: "mon", 2024-04-12T21:04:08.576 INFO:teuthology.orchestra.run.smithi018.stdout: "mgr" 2024-04-12T21:04:08.576 INFO:teuthology.orchestra.run.smithi018.stdout: ], 2024-04-12T21:04:08.576 INFO:teuthology.orchestra.run.smithi018.stdout: "progress": "7/21 daemons upgraded", 2024-04-12T21:04:08.577 INFO:teuthology.orchestra.run.smithi018.stdout: "message": "Currently upgrading osd daemons", 2024-04-12T21:04:08.577 INFO:teuthology.orchestra.run.smithi018.stdout: "is_paused": false 2024-04-12T21:04:08.577 INFO:teuthology.orchestra.run.smithi018.stdout:} 2024-04-12T21:04:08.579 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:08.581+0000 7f7950c00700 1 -- 172.21.15.18:0/1810025467 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f793006ec00 msgr2=0x7f79300710b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:04:08.579 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:08.582+0000 7f7950c00700 1 --2- 172.21.15.18:0/1810025467 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f793006ec00 0x7f79300710b0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f793800b020 tx=0x7f7938009e30 comp rx=0 tx=0).stop 2024-04-12T21:04:08.579 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:08.582+0000 7f7950c00700 1 -- 172.21.15.18:0/1810025467 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f794c102e00 msgr2=0x7f794c115bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:04:08.579 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:08.582+0000 7f7950c00700 1 --2- 172.21.15.18:0/1810025467 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f794c102e00 0x7f794c115bd0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f793c00c830 tx=0x7f793c004470 comp rx=0 tx=0).stop 2024-04-12T21:04:08.579 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:08.582+0000 7f7950c00700 1 -- 172.21.15.18:0/1810025467 shutdown_connections 2024-04-12T21:04:08.579 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:08.582+0000 7f7950c00700 1 --2- 172.21.15.18:0/1810025467 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f793006ec00 0x7f79300710b0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:04:08.579 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:08.582+0000 7f7950c00700 1 --2- 172.21.15.18:0/1810025467 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f794c102e00 0x7f794c115bd0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:04:08.580 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:08.582+0000 7f7950c00700 1 --2- 172.21.15.18:0/1810025467 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f794c1037a0 0x7f794c116110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:04:08.580 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:08.582+0000 7f7950c00700 1 -- 172.21.15.18:0/1810025467 >> 172.21.15.18:0/1810025467 conn(0x7f794c0f89b0 msgr2=0x7f794c070c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:04:08.580 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:08.582+0000 7f7950c00700 1 -- 172.21.15.18:0/1810025467 shutdown_connections 2024-04-12T21:04:08.580 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:08.582+0000 7f7950c00700 1 -- 172.21.15.18:0/1810025467 wait complete. 2024-04-12T21:04:08.728 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:08.731+0000 7f3bc3c00700 1 -- 172.21.15.18:0/4093543903 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3bbc109dd0 msgr2=0x7f3bbc103440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:04:08.729 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:08.731+0000 7f3bc3c00700 1 --2- 172.21.15.18:0/4093543903 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3bbc109dd0 0x7f3bbc103440 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f3ba8004940 tx=0x7f3ba8009e30 comp rx=0 tx=0).stop 2024-04-12T21:04:08.729 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:08.731+0000 7f3bc3c00700 1 -- 172.21.15.18:0/4093543903 shutdown_connections 2024-04-12T21:04:08.729 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:08.731+0000 7f3bc3c00700 1 --2- 172.21.15.18:0/4093543903 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3bbc109dd0 0x7f3bbc103440 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:04:08.729 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:08.731+0000 7f3bc3c00700 1 --2- 172.21.15.18:0/4093543903 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3bbc109430 0x7f3bbc109800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:04:08.729 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:08.731+0000 7f3bc3c00700 1 -- 172.21.15.18:0/4093543903 >> 172.21.15.18:0/4093543903 conn(0x7f3bbc072c00 msgr2=0x7f3bbc075010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:04:08.729 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:04:08 smithi018 ceph-mon[112590]: from='client.34252 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:04:08.729 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:04:08 smithi018 ceph-mon[112590]: from='client.34256 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:04:08.729 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:04:08 smithi018 ceph-mon[112590]: from='client.? 172.21.15.18:0/243569063' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T21:04:08.729 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:04:08 smithi018 ceph-mon[112590]: from='client.? 172.21.15.18:0/347007342' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-12T21:04:08.730 INFO:journalctl@ceph.osd.1.smithi018.stdout:Apr 12 21:04:08 smithi018 ceph-1cbe407a-f90d-11ee-bc8d-c7b262605968-osd-1[54584]: 2024-04-12T21:04:08.662+0000 7f0bbb000700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-12T21:04:08.730 INFO:journalctl@ceph.osd.1.smithi018.stdout:Apr 12 21:04:08 smithi018 ceph-1cbe407a-f90d-11ee-bc8d-c7b262605968-osd-1[54584]: 2024-04-12T21:04:08.662+0000 7f0bbb000700 -1 osd.1 73 *** Got signal Terminated *** 2024-04-12T21:04:08.730 INFO:journalctl@ceph.osd.1.smithi018.stdout:Apr 12 21:04:08 smithi018 ceph-1cbe407a-f90d-11ee-bc8d-c7b262605968-osd-1[54584]: 2024-04-12T21:04:08.662+0000 7f0bbb000700 -1 osd.1 73 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-12T21:04:08.731 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:08.732+0000 7f3bc3c00700 1 -- 172.21.15.18:0/4093543903 shutdown_connections 2024-04-12T21:04:08.731 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:08.732+0000 7f3bc3c00700 1 -- 172.21.15.18:0/4093543903 wait complete. 2024-04-12T21:04:08.731 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:08.733+0000 7f3bc3c00700 1 Processor -- start 2024-04-12T21:04:08.731 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:08.733+0000 7f3bc3c00700 1 -- start start 2024-04-12T21:04:08.731 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:08.733+0000 7f3bc3c00700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3bbc109430 0x7f3bbc119fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:04:08.732 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:08.733+0000 7f3bc3c00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3bbc109dd0 0x7f3bbc11a520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:04:08.732 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:08.733+0000 7f3bc3c00700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3bbc11ab70 con 0x7f3bbc109dd0 2024-04-12T21:04:08.732 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:08.733+0000 7f3bc3c00700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3bbc114150 con 0x7f3bbc109430 2024-04-12T21:04:08.732 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:08.734+0000 7f3bbbe00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3bbc109dd0 0x7f3bbc11a520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:04:08.732 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:08.734+0000 7f3bbbe00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3bbc109dd0 0x7f3bbc11a520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:39766/0 (socket says 172.21.15.18:39766) 2024-04-12T21:04:08.732 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:08.734+0000 7f3bbbe00700 1 -- 172.21.15.18:0/3128006623 learned_addr learned my addr 172.21.15.18:0/3128006623 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:04:08.732 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:08.734+0000 7f3bc1200700 1 --2- 172.21.15.18:0/3128006623 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3bbc109430 0x7f3bbc119fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:04:08.732 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:08.734+0000 7f3bbbe00700 1 -- 172.21.15.18:0/3128006623 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3bbc109430 msgr2=0x7f3bbc119fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:04:08.732 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:08.734+0000 7f3bbbe00700 1 --2- 172.21.15.18:0/3128006623 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3bbc109430 0x7f3bbc119fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:04:08.732 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:08.734+0000 7f3bbbe00700 1 -- 172.21.15.18:0/3128006623 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ba8005040 con 0x7f3bbc109dd0 2024-04-12T21:04:08.733 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:08.735+0000 7f3bbbe00700 1 --2- 172.21.15.18:0/3128006623 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3bbc109dd0 0x7f3bbc11a520 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f3ba8009100 tx=0x7f3ba8009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:04:08.733 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:08.735+0000 7f3bb8c00700 1 -- 172.21.15.18:0/3128006623 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3ba800e040 con 0x7f3bbc109dd0 2024-04-12T21:04:08.733 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:08.735+0000 7f3bb8c00700 1 -- 172.21.15.18:0/3128006623 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f3ba8013420 con 0x7f3bbc109dd0 2024-04-12T21:04:08.733 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:08.735+0000 7f3bb8c00700 1 -- 172.21.15.18:0/3128006623 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3ba801c630 con 0x7f3bbc109dd0 2024-04-12T21:04:08.733 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:08.735+0000 7f3bc3c00700 1 -- 172.21.15.18:0/3128006623 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3bbc1143d0 con 0x7f3bbc109dd0 2024-04-12T21:04:08.733 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:08.735+0000 7f3bc3c00700 1 -- 172.21.15.18:0/3128006623 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3bbc114990 con 0x7f3bbc109dd0 2024-04-12T21:04:08.734 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:08.736+0000 7f3bc3c00700 1 -- 172.21.15.18:0/3128006623 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3bbc104b80 con 0x7f3bbc109dd0 2024-04-12T21:04:08.735 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:08.738+0000 7f3bb8c00700 1 -- 172.21.15.18:0/3128006623 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7f3ba8013590 con 0x7f3bbc109dd0 2024-04-12T21:04:08.737 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:08.740+0000 7f3bb8c00700 1 --2- 172.21.15.18:0/3128006623 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f3ba406ec00 0x7f3ba40710b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:04:08.737 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:08.740+0000 7f3bb8c00700 1 -- 172.21.15.18:0/3128006623 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 6585+0+0 (secure 0 0 0) 0x7f3ba808fa30 con 0x7f3bbc109dd0 2024-04-12T21:04:08.737 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:08.740+0000 7f3bc1200700 1 --2- 172.21.15.18:0/3128006623 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f3ba406ec00 0x7f3ba40710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:04:08.738 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:08.741+0000 7f3bc1200700 1 --2- 172.21.15.18:0/3128006623 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f3ba406ec00 0x7f3ba40710b0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f3bb0009fd0 tx=0x7f3bb0009f20 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:04:08.742 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:08.745+0000 7f3bb8c00700 1 -- 172.21.15.18:0/3128006623 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f3ba805c3d0 con 0x7f3bbc109dd0 2024-04-12T21:04:08.779 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:04:08 smithi186 ceph-mon[86379]: from='client.34252 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:04:08.779 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:04:08 smithi186 ceph-mon[86379]: from='client.34256 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:04:08.780 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:04:08 smithi186 ceph-mon[86379]: from='client.? 172.21.15.18:0/243569063' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T21:04:08.780 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:04:08 smithi186 ceph-mon[86379]: from='client.? 172.21.15.18:0/347007342' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-12T21:04:09.071 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:09.074+0000 7f3bc3c00700 1 -- 172.21.15.18:0/3128006623 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f3bbc04c710 con 0x7f3bbc109dd0 2024-04-12T21:04:09.072 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:09.075+0000 7f3bb8c00700 1 -- 172.21.15.18:0/3128006623 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+10 (secure 0 0 0) 0x7f3ba8058030 con 0x7f3bbc109dd0 2024-04-12T21:04:09.072 INFO:teuthology.orchestra.run.smithi018.stdout:HEALTH_OK 2024-04-12T21:04:09.076 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:09.078+0000 7f3bc3c00700 1 -- 172.21.15.18:0/3128006623 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f3ba406ec00 msgr2=0x7f3ba40710b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:04:09.076 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:09.078+0000 7f3bc3c00700 1 --2- 172.21.15.18:0/3128006623 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f3ba406ec00 0x7f3ba40710b0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f3bb0009fd0 tx=0x7f3bb0009f20 comp rx=0 tx=0).stop 2024-04-12T21:04:09.076 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:09.078+0000 7f3bc3c00700 1 -- 172.21.15.18:0/3128006623 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3bbc109dd0 msgr2=0x7f3bbc11a520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:04:09.076 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:09.078+0000 7f3bc3c00700 1 --2- 172.21.15.18:0/3128006623 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3bbc109dd0 0x7f3bbc11a520 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f3ba8009100 tx=0x7f3ba8009e30 comp rx=0 tx=0).stop 2024-04-12T21:04:09.076 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:09.078+0000 7f3bc3c00700 1 -- 172.21.15.18:0/3128006623 shutdown_connections 2024-04-12T21:04:09.076 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:09.078+0000 7f3bc3c00700 1 --2- 172.21.15.18:0/3128006623 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f3ba406ec00 0x7f3ba40710b0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:04:09.076 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:09.078+0000 7f3bc3c00700 1 --2- 172.21.15.18:0/3128006623 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3bbc109430 0x7f3bbc119fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:04:09.076 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:09.078+0000 7f3bc3c00700 1 --2- 172.21.15.18:0/3128006623 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3bbc109dd0 0x7f3bbc11a520 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:04:09.076 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:09.078+0000 7f3bc3c00700 1 -- 172.21.15.18:0/3128006623 >> 172.21.15.18:0/3128006623 conn(0x7f3bbc072c00 msgr2=0x7f3bbc074be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:04:09.076 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:09.079+0000 7f3bc3c00700 1 -- 172.21.15.18:0/3128006623 shutdown_connections 2024-04-12T21:04:09.077 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:09.079+0000 7f3bc3c00700 1 -- 172.21.15.18:0/3128006623 wait complete. 2024-04-12T21:04:09.779 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:04:09 smithi186 ceph-mon[86379]: pgmap v101: 65 pgs: 65 active+clean; 257 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 0 B/s, 191 objects/s recovering 2024-04-12T21:04:09.780 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:04:09 smithi186 ceph-mon[86379]: osd.1 marked itself down and dead 2024-04-12T21:04:09.780 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:04:09 smithi186 ceph-mon[86379]: from='client.? 172.21.15.18:0/3128006623' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-12T21:04:09.851 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:04:09 smithi018 ceph-mon[112590]: pgmap v101: 65 pgs: 65 active+clean; 257 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 0 B/s, 191 objects/s recovering 2024-04-12T21:04:09.851 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:04:09 smithi018 ceph-mon[112590]: osd.1 marked itself down and dead 2024-04-12T21:04:09.851 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:04:09 smithi018 ceph-mon[112590]: from='client.? 172.21.15.18:0/3128006623' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-12T21:04:10.349 INFO:journalctl@ceph.osd.1.smithi018.stdout:Apr 12 21:04:10 smithi018 podman[129552]: 2024-04-12 21:04:10.011969264 +0000 UTC m=+1.498421233 container died 6517c9cc7f8799f4348b05bf2e8b239a14f747f088ae356095dc1882b0e7901d (image=quay.ceph.io/ceph-ci/ceph@sha256:5e334df698f432180b78bc9c46936bfdfbc92bc9a8983463e523f780d23765fe, name=ceph-1cbe407a-f90d-11ee-bc8d-c7b262605968-osd-1, org.label-schema.schema-version=1.0, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240409, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=quincy-d3b5060, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=) 2024-04-12T21:04:10.600 INFO:journalctl@ceph.osd.1.smithi018.stdout:Apr 12 21:04:10 smithi018 podman[129552]: 2024-04-12 21:04:10.596019802 +0000 UTC m=+2.082471810 container cleanup 6517c9cc7f8799f4348b05bf2e8b239a14f747f088ae356095dc1882b0e7901d (image=quay.ceph.io/ceph-ci/ceph@sha256:5e334df698f432180b78bc9c46936bfdfbc92bc9a8983463e523f780d23765fe, name=ceph-1cbe407a-f90d-11ee-bc8d-c7b262605968-osd-1, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, RELEASE=quincy-d3b5060, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240409) 2024-04-12T21:04:10.600 INFO:journalctl@ceph.osd.1.smithi018.stdout:Apr 12 21:04:10 smithi018 bash[129552]: ceph-1cbe407a-f90d-11ee-bc8d-c7b262605968-osd-1 2024-04-12T21:04:10.601 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:04:10 smithi018 ceph-mon[112590]: from='client.34268 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:04:10.601 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:04:10 smithi018 ceph-mon[112590]: Health check failed: 1 osds down (OSD_DOWN) 2024-04-12T21:04:10.601 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:04:10 smithi018 ceph-mon[112590]: osdmap e74: 6 total, 5 up, 6 in 2024-04-12T21:04:10.601 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:04:10 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' 2024-04-12T21:04:10.601 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:04:10 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T21:04:10.601 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:04:10 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' 2024-04-12T21:04:10.601 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:04:10 smithi018 ceph-mon[112590]: osdmap e75: 6 total, 5 up, 6 in 2024-04-12T21:04:10.780 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:04:10 smithi186 ceph-mon[86379]: from='client.34268 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:04:10.780 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:04:10 smithi186 ceph-mon[86379]: Health check failed: 1 osds down (OSD_DOWN) 2024-04-12T21:04:10.780 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:04:10 smithi186 ceph-mon[86379]: osdmap e74: 6 total, 5 up, 6 in 2024-04-12T21:04:10.780 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:04:10 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' 2024-04-12T21:04:10.780 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:04:10 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T21:04:10.780 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:04:10 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' 2024-04-12T21:04:10.780 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:04:10 smithi186 ceph-mon[86379]: osdmap e75: 6 total, 5 up, 6 in 2024-04-12T21:04:11.101 INFO:journalctl@ceph.osd.1.smithi018.stdout:Apr 12 21:04:10 smithi018 podman[129598]: 2024-04-12 21:04:10.839651793 +0000 UTC m=+0.814013529 container remove 6517c9cc7f8799f4348b05bf2e8b239a14f747f088ae356095dc1882b0e7901d (image=quay.ceph.io/ceph-ci/ceph@sha256:5e334df698f432180b78bc9c46936bfdfbc92bc9a8983463e523f780d23765fe, name=ceph-1cbe407a-f90d-11ee-bc8d-c7b262605968-osd-1, ceph=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, RELEASE=quincy-d3b5060, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240409, org.label-schema.license=GPLv2) 2024-04-12T21:04:11.754 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:04:11 smithi018 ceph-mon[112590]: pgmap v103: 65 pgs: 12 stale+active+clean, 53 active+clean; 257 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 0 B/s, 170 objects/s recovering 2024-04-12T21:04:11.755 INFO:journalctl@ceph.osd.1.smithi018.stdout:Apr 12 21:04:11 smithi018 podman[129686]: 2024-04-12 21:04:11.642519144 +0000 UTC m=+0.033334936 image pull 8717720dc5e5c56303818ea4d4f0a11f6e2c03dee521fb668f411f1304e0afca quay.ceph.io/ceph-ci/ceph@sha256:f4452def3be4d350f4a8e5feaf7ebb1511be9298fe68478ba7a2d6fbd7d69a63 2024-04-12T21:04:11.779 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:04:11 smithi186 ceph-mon[86379]: pgmap v103: 65 pgs: 12 stale+active+clean, 53 active+clean; 257 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 0 B/s, 170 objects/s recovering 2024-04-12T21:04:12.035 INFO:journalctl@ceph.osd.1.smithi018.stdout:Apr 12 21:04:11 smithi018 podman[129686]: 2024-04-12 21:04:11.757259371 +0000 UTC m=+0.148075198 container create 57ee5d89bc9dc1fa9d47cea1781994996ce7907eacb2556455f3bdf4d19eba69 (image=quay.ceph.io/ceph-ci/ceph@sha256:f4452def3be4d350f4a8e5feaf7ebb1511be9298fe68478ba7a2d6fbd7d69a63, name=ceph-1cbe407a-f90d-11ee-bc8d-c7b262605968-osd-1-deactivate, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240409, RELEASE=wip-vshankar-testing-20240411.061452-324baf8, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.license=GPLv2) 2024-04-12T21:04:12.035 INFO:journalctl@ceph.osd.1.smithi018.stdout:Apr 12 21:04:12 smithi018 podman[129686]: 2024-04-12 21:04:12.037893378 +0000 UTC m=+0.428709249 container init 57ee5d89bc9dc1fa9d47cea1781994996ce7907eacb2556455f3bdf4d19eba69 (image=quay.ceph.io/ceph-ci/ceph@sha256:f4452def3be4d350f4a8e5feaf7ebb1511be9298fe68478ba7a2d6fbd7d69a63, name=ceph-1cbe407a-f90d-11ee-bc8d-c7b262605968-osd-1-deactivate, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, RELEASE=wip-vshankar-testing-20240411.061452-324baf8, ceph=True, org.label-schema.build-date=20240409, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_CLEAN=True, maintainer=Guillaume Abrioux ) 2024-04-12T21:04:12.351 INFO:journalctl@ceph.osd.1.smithi018.stdout:Apr 12 21:04:12 smithi018 podman[129686]: 2024-04-12 21:04:12.051834498 +0000 UTC m=+0.442650288 container start 57ee5d89bc9dc1fa9d47cea1781994996ce7907eacb2556455f3bdf4d19eba69 (image=quay.ceph.io/ceph-ci/ceph@sha256:f4452def3be4d350f4a8e5feaf7ebb1511be9298fe68478ba7a2d6fbd7d69a63, name=ceph-1cbe407a-f90d-11ee-bc8d-c7b262605968-osd-1-deactivate, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, ceph=True, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240409, RELEASE=wip-vshankar-testing-20240411.061452-324baf8, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-12T21:04:12.351 INFO:journalctl@ceph.osd.1.smithi018.stdout:Apr 12 21:04:12 smithi018 podman[129686]: 2024-04-12 21:04:12.15996543 +0000 UTC m=+0.550781313 container attach 57ee5d89bc9dc1fa9d47cea1781994996ce7907eacb2556455f3bdf4d19eba69 (image=quay.ceph.io/ceph-ci/ceph@sha256:f4452def3be4d350f4a8e5feaf7ebb1511be9298fe68478ba7a2d6fbd7d69a63, name=ceph-1cbe407a-f90d-11ee-bc8d-c7b262605968-osd-1-deactivate, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, RELEASE=wip-vshankar-testing-20240411.061452-324baf8, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240409) 2024-04-12T21:04:12.851 INFO:journalctl@ceph.osd.1.smithi018.stdout:Apr 12 21:04:12 smithi018 podman[129686]: 2024-04-12 21:04:12.526048436 +0000 UTC m=+0.916864256 container died 57ee5d89bc9dc1fa9d47cea1781994996ce7907eacb2556455f3bdf4d19eba69 (image=quay.ceph.io/ceph-ci/ceph@sha256:f4452def3be4d350f4a8e5feaf7ebb1511be9298fe68478ba7a2d6fbd7d69a63, name=ceph-1cbe407a-f90d-11ee-bc8d-c7b262605968-osd-1-deactivate, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240409, ceph=True, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-vshankar-testing-20240411.061452-324baf8, maintainer=Guillaume Abrioux ) 2024-04-12T21:04:12.943 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:04:12 smithi018 ceph-mon[112590]: pgmap v105: 65 pgs: 16 active+undersized, 16 active+undersized+degraded, 33 active+clean; 257 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 50/324 objects degraded (15.432%) 2024-04-12T21:04:12.943 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:04:12 smithi018 ceph-mon[112590]: Health check failed: Degraded data redundancy: 50/324 objects degraded (15.432%), 16 pgs degraded (PG_DEGRADED) 2024-04-12T21:04:13.029 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:04:12 smithi186 ceph-mon[86379]: pgmap v105: 65 pgs: 16 active+undersized, 16 active+undersized+degraded, 33 active+clean; 257 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 50/324 objects degraded (15.432%) 2024-04-12T21:04:13.029 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:04:12 smithi186 ceph-mon[86379]: Health check failed: Degraded data redundancy: 50/324 objects degraded (15.432%), 16 pgs degraded (PG_DEGRADED) 2024-04-12T21:04:13.600 INFO:journalctl@ceph.osd.1.smithi018.stdout:Apr 12 21:04:13 smithi018 podman[129712]: 2024-04-12 21:04:13.313845899 +0000 UTC m=+0.768523668 container remove 57ee5d89bc9dc1fa9d47cea1781994996ce7907eacb2556455f3bdf4d19eba69 (image=quay.ceph.io/ceph-ci/ceph@sha256:f4452def3be4d350f4a8e5feaf7ebb1511be9298fe68478ba7a2d6fbd7d69a63, name=ceph-1cbe407a-f90d-11ee-bc8d-c7b262605968-osd-1-deactivate, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-vshankar-testing-20240411.061452-324baf8, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.build-date=20240409, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, GIT_CLEAN=True, io.buildah.version=1.33.5) 2024-04-12T21:04:13.601 INFO:journalctl@ceph.osd.1.smithi018.stdout:Apr 12 21:04:13 smithi018 systemd[1]: ceph-1cbe407a-f90d-11ee-bc8d-c7b262605968@osd.1.service: Deactivated successfully. 2024-04-12T21:04:13.601 INFO:journalctl@ceph.osd.1.smithi018.stdout:Apr 12 21:04:13 smithi018 systemd[1]: Stopped Ceph osd.1 for 1cbe407a-f90d-11ee-bc8d-c7b262605968. 2024-04-12T21:04:13.601 INFO:journalctl@ceph.osd.1.smithi018.stdout:Apr 12 21:04:13 smithi018 systemd[1]: ceph-1cbe407a-f90d-11ee-bc8d-c7b262605968@osd.1.service: Consumed 1min 59.273s CPU time. 2024-04-12T21:04:14.100 INFO:journalctl@ceph.osd.1.smithi018.stdout:Apr 12 21:04:13 smithi018 systemd[1]: Starting Ceph osd.1 for 1cbe407a-f90d-11ee-bc8d-c7b262605968... 2024-04-12T21:04:14.424 INFO:journalctl@ceph.osd.1.smithi018.stdout:Apr 12 21:04:14 smithi018 podman[129815]: 2024-04-12 21:04:14.013887442 +0000 UTC m=+0.036639758 image pull 8717720dc5e5c56303818ea4d4f0a11f6e2c03dee521fb668f411f1304e0afca quay.ceph.io/ceph-ci/ceph@sha256:f4452def3be4d350f4a8e5feaf7ebb1511be9298fe68478ba7a2d6fbd7d69a63 2024-04-12T21:04:14.424 INFO:journalctl@ceph.osd.1.smithi018.stdout:Apr 12 21:04:14 smithi018 podman[129815]: 2024-04-12 21:04:14.131636128 +0000 UTC m=+0.154388464 container create 675081bf11f10faf18029729dbfbd19bd7fbff1dba71be4a73cf64e1da9fe470 (image=quay.ceph.io/ceph-ci/ceph@sha256:f4452def3be4d350f4a8e5feaf7ebb1511be9298fe68478ba7a2d6fbd7d69a63, name=ceph-1cbe407a-f90d-11ee-bc8d-c7b262605968-osd-1-activate, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_CLEAN=True, CEPH_POINT_RELEASE=, RELEASE=wip-vshankar-testing-20240411.061452-324baf8, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.build-date=20240409, ceph=True, org.label-schema.schema-version=1.0) 2024-04-12T21:04:14.424 INFO:journalctl@ceph.osd.1.smithi018.stdout:Apr 12 21:04:14 smithi018 podman[129815]: 2024-04-12 21:04:14.427256834 +0000 UTC m=+0.450009214 container init 675081bf11f10faf18029729dbfbd19bd7fbff1dba71be4a73cf64e1da9fe470 (image=quay.ceph.io/ceph-ci/ceph@sha256:f4452def3be4d350f4a8e5feaf7ebb1511be9298fe68478ba7a2d6fbd7d69a63, name=ceph-1cbe407a-f90d-11ee-bc8d-c7b262605968-osd-1-activate, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.build-date=20240409, ceph=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-vshankar-testing-20240411.061452-324baf8) 2024-04-12T21:04:14.850 INFO:journalctl@ceph.osd.1.smithi018.stdout:Apr 12 21:04:14 smithi018 podman[129815]: 2024-04-12 21:04:14.439312639 +0000 UTC m=+0.462064995 container start 675081bf11f10faf18029729dbfbd19bd7fbff1dba71be4a73cf64e1da9fe470 (image=quay.ceph.io/ceph-ci/ceph@sha256:f4452def3be4d350f4a8e5feaf7ebb1511be9298fe68478ba7a2d6fbd7d69a63, name=ceph-1cbe407a-f90d-11ee-bc8d-c7b262605968-osd-1-activate, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_CLEAN=True, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-vshankar-testing-20240411.061452-324baf8, org.label-schema.license=GPLv2, org.label-schema.build-date=20240409, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0) 2024-04-12T21:04:14.850 INFO:journalctl@ceph.osd.1.smithi018.stdout:Apr 12 21:04:14 smithi018 podman[129815]: 2024-04-12 21:04:14.542695161 +0000 UTC m=+0.565447553 container attach 675081bf11f10faf18029729dbfbd19bd7fbff1dba71be4a73cf64e1da9fe470 (image=quay.ceph.io/ceph-ci/ceph@sha256:f4452def3be4d350f4a8e5feaf7ebb1511be9298fe68478ba7a2d6fbd7d69a63, name=ceph-1cbe407a-f90d-11ee-bc8d-c7b262605968-osd-1-activate, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-vshankar-testing-20240411.061452-324baf8, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, ceph=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.build-date=20240409, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-12T21:04:15.279 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:04:14 smithi186 ceph-mon[86379]: pgmap v106: 65 pgs: 16 active+undersized, 16 active+undersized+degraded, 33 active+clean; 257 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 50/324 objects degraded (15.432%) 2024-04-12T21:04:15.348 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:04:14 smithi018 ceph-mon[112590]: pgmap v106: 65 pgs: 16 active+undersized, 16 active+undersized+degraded, 33 active+clean; 257 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 50/324 objects degraded (15.432%) 2024-04-12T21:04:15.602 INFO:journalctl@ceph.osd.1.smithi018.stdout:Apr 12 21:04:15 smithi018 ceph-1cbe407a-f90d-11ee-bc8d-c7b262605968-osd-1-activate[129830]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-04-12T21:04:15.602 INFO:journalctl@ceph.osd.1.smithi018.stdout:Apr 12 21:04:15 smithi018 ceph-1cbe407a-f90d-11ee-bc8d-c7b262605968-osd-1-activate[129830]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-1 --no-mon-config --dev /dev/mapper/vg_nvme-lv_3 2024-04-12T21:04:15.602 INFO:journalctl@ceph.osd.1.smithi018.stdout:Apr 12 21:04:15 smithi018 bash[129815]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-04-12T21:04:15.602 INFO:journalctl@ceph.osd.1.smithi018.stdout:Apr 12 21:04:15 smithi018 bash[129815]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-1 --no-mon-config --dev /dev/mapper/vg_nvme-lv_3 2024-04-12T21:04:15.602 INFO:journalctl@ceph.osd.1.smithi018.stdout:Apr 12 21:04:15 smithi018 bash[129815]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_3 2024-04-12T21:04:15.602 INFO:journalctl@ceph.osd.1.smithi018.stdout:Apr 12 21:04:15 smithi018 bash[129815]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-2 2024-04-12T21:04:15.602 INFO:journalctl@ceph.osd.1.smithi018.stdout:Apr 12 21:04:15 smithi018 bash[129815]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-1/block 2024-04-12T21:04:15.602 INFO:journalctl@ceph.osd.1.smithi018.stdout:Apr 12 21:04:15 smithi018 bash[129815]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-04-12T21:04:15.602 INFO:journalctl@ceph.osd.1.smithi018.stdout:Apr 12 21:04:15 smithi018 bash[129815]: --> ceph-volume raw activate successful for osd ID: 1 2024-04-12T21:04:15.602 INFO:journalctl@ceph.osd.1.smithi018.stdout:Apr 12 21:04:15 smithi018 ceph-1cbe407a-f90d-11ee-bc8d-c7b262605968-osd-1-activate[129830]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_3 2024-04-12T21:04:15.603 INFO:journalctl@ceph.osd.1.smithi018.stdout:Apr 12 21:04:15 smithi018 ceph-1cbe407a-f90d-11ee-bc8d-c7b262605968-osd-1-activate[129830]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-2 2024-04-12T21:04:15.603 INFO:journalctl@ceph.osd.1.smithi018.stdout:Apr 12 21:04:15 smithi018 ceph-1cbe407a-f90d-11ee-bc8d-c7b262605968-osd-1-activate[129830]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-1/block 2024-04-12T21:04:15.603 INFO:journalctl@ceph.osd.1.smithi018.stdout:Apr 12 21:04:15 smithi018 ceph-1cbe407a-f90d-11ee-bc8d-c7b262605968-osd-1-activate[129830]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-04-12T21:04:15.603 INFO:journalctl@ceph.osd.1.smithi018.stdout:Apr 12 21:04:15 smithi018 ceph-1cbe407a-f90d-11ee-bc8d-c7b262605968-osd-1-activate[129830]: --> ceph-volume raw activate successful for osd ID: 1 2024-04-12T21:04:15.603 INFO:journalctl@ceph.osd.1.smithi018.stdout:Apr 12 21:04:15 smithi018 podman[129815]: 2024-04-12 21:04:15.416673053 +0000 UTC m=+1.439425396 container died 675081bf11f10faf18029729dbfbd19bd7fbff1dba71be4a73cf64e1da9fe470 (image=quay.ceph.io/ceph-ci/ceph@sha256:f4452def3be4d350f4a8e5feaf7ebb1511be9298fe68478ba7a2d6fbd7d69a63, name=ceph-1cbe407a-f90d-11ee-bc8d-c7b262605968-osd-1-activate, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.build-date=20240409, ceph=True, RELEASE=wip-vshankar-testing-20240411.061452-324baf8, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-12T21:04:16.601 INFO:journalctl@ceph.osd.1.smithi018.stdout:Apr 12 21:04:16 smithi018 podman[129969]: 2024-04-12 21:04:16.246591809 +0000 UTC m=+0.809146105 container remove 675081bf11f10faf18029729dbfbd19bd7fbff1dba71be4a73cf64e1da9fe470 (image=quay.ceph.io/ceph-ci/ceph@sha256:f4452def3be4d350f4a8e5feaf7ebb1511be9298fe68478ba7a2d6fbd7d69a63, name=ceph-1cbe407a-f90d-11ee-bc8d-c7b262605968-osd-1-activate, org.label-schema.build-date=20240409, GIT_BRANCH=HEAD, GIT_CLEAN=True, ceph=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, RELEASE=wip-vshankar-testing-20240411.061452-324baf8) 2024-04-12T21:04:16.874 INFO:journalctl@ceph.osd.1.smithi018.stdout:Apr 12 21:04:16 smithi018 podman[130030]: 2024-04-12 21:04:16.514367932 +0000 UTC m=+0.034561443 image pull 8717720dc5e5c56303818ea4d4f0a11f6e2c03dee521fb668f411f1304e0afca quay.ceph.io/ceph-ci/ceph@sha256:f4452def3be4d350f4a8e5feaf7ebb1511be9298fe68478ba7a2d6fbd7d69a63 2024-04-12T21:04:16.874 INFO:journalctl@ceph.osd.1.smithi018.stdout:Apr 12 21:04:16 smithi018 podman[130030]: 2024-04-12 21:04:16.647621527 +0000 UTC m=+0.167815059 container create 170ce1e5fb8dedd565d7c8fc77f3e8eb8db353188932b3fc1f307ddd4d64fac1 (image=quay.ceph.io/ceph-ci/ceph@sha256:f4452def3be4d350f4a8e5feaf7ebb1511be9298fe68478ba7a2d6fbd7d69a63, name=ceph-1cbe407a-f90d-11ee-bc8d-c7b262605968-osd-1, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, ceph=True, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-vshankar-testing-20240411.061452-324baf8, org.label-schema.build-date=20240409, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux ) 2024-04-12T21:04:17.279 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:04:16 smithi186 ceph-mon[86379]: pgmap v107: 65 pgs: 16 active+undersized, 16 active+undersized+degraded, 33 active+clean; 257 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 50/324 objects degraded (15.432%) 2024-04-12T21:04:17.350 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:04:16 smithi018 ceph-mon[112590]: pgmap v107: 65 pgs: 16 active+undersized, 16 active+undersized+degraded, 33 active+clean; 257 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 50/324 objects degraded (15.432%) 2024-04-12T21:04:17.351 INFO:journalctl@ceph.osd.1.smithi018.stdout:Apr 12 21:04:16 smithi018 podman[130030]: 2024-04-12 21:04:16.90938521 +0000 UTC m=+0.429578748 container init 170ce1e5fb8dedd565d7c8fc77f3e8eb8db353188932b3fc1f307ddd4d64fac1 (image=quay.ceph.io/ceph-ci/ceph@sha256:f4452def3be4d350f4a8e5feaf7ebb1511be9298fe68478ba7a2d6fbd7d69a63, name=ceph-1cbe407a-f90d-11ee-bc8d-c7b262605968-osd-1, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-vshankar-testing-20240411.061452-324baf8, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.build-date=20240409) 2024-04-12T21:04:17.351 INFO:journalctl@ceph.osd.1.smithi018.stdout:Apr 12 21:04:16 smithi018 podman[130030]: 2024-04-12 21:04:16.921526114 +0000 UTC m=+0.441719676 container start 170ce1e5fb8dedd565d7c8fc77f3e8eb8db353188932b3fc1f307ddd4d64fac1 (image=quay.ceph.io/ceph-ci/ceph@sha256:f4452def3be4d350f4a8e5feaf7ebb1511be9298fe68478ba7a2d6fbd7d69a63, name=ceph-1cbe407a-f90d-11ee-bc8d-c7b262605968-osd-1, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, RELEASE=wip-vshankar-testing-20240411.061452-324baf8, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240409, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0) 2024-04-12T21:04:17.351 INFO:journalctl@ceph.osd.1.smithi018.stdout:Apr 12 21:04:16 smithi018 ceph-osd[130048]: -- 172.21.15.18:0/2483889285 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x55cad17d3ba0 con 0x55cad17a5000 2024-04-12T21:04:17.351 INFO:journalctl@ceph.osd.1.smithi018.stdout:Apr 12 21:04:16 smithi018 ceph-osd[130048]: -- 172.21.15.18:0/2483889285 <== mon.0 v2:172.21.15.18:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55cad17db680 con 0x55cad17a5000 2024-04-12T21:04:17.351 INFO:journalctl@ceph.osd.1.smithi018.stdout:Apr 12 21:04:17 smithi018 bash[130030]: 170ce1e5fb8dedd565d7c8fc77f3e8eb8db353188932b3fc1f307ddd4d64fac1 2024-04-12T21:04:17.351 INFO:journalctl@ceph.osd.1.smithi018.stdout:Apr 12 21:04:17 smithi018 systemd[1]: Started Ceph osd.1 for 1cbe407a-f90d-11ee-bc8d-c7b262605968. 2024-04-12T21:04:17.850 INFO:journalctl@ceph.osd.1.smithi018.stdout:Apr 12 21:04:17 smithi018 ceph-1cbe407a-f90d-11ee-bc8d-c7b262605968-osd-1[130044]: 2024-04-12T21:04:17.495+0000 7f5ebd5457c0 -1 Falling back to public interface 2024-04-12T21:04:18.529 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:04:18 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' 2024-04-12T21:04:18.530 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:04:18 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' 2024-04-12T21:04:18.530 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:04:18 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-12T21:04:18.547 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:04:18 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' 2024-04-12T21:04:18.547 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:04:18 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' 2024-04-12T21:04:18.547 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:04:18 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-12T21:04:19.529 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:04:19 smithi186 ceph-mon[86379]: pgmap v108: 65 pgs: 16 active+undersized, 16 active+undersized+degraded, 33 active+clean; 257 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 50/324 objects degraded (15.432%) 2024-04-12T21:04:19.601 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:04:19 smithi018 ceph-mon[112590]: pgmap v108: 65 pgs: 16 active+undersized, 16 active+undersized+degraded, 33 active+clean; 257 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 50/324 objects degraded (15.432%) 2024-04-12T21:04:21.350 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:04:21 smithi018 ceph-mon[112590]: pgmap v109: 65 pgs: 16 active+undersized, 16 active+undersized+degraded, 33 active+clean; 257 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 50/324 objects degraded (15.432%) 2024-04-12T21:04:21.530 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:04:21 smithi186 ceph-mon[86379]: pgmap v109: 65 pgs: 16 active+undersized, 16 active+undersized+degraded, 33 active+clean; 257 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 50/324 objects degraded (15.432%) 2024-04-12T21:04:23.529 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:04:23 smithi186 ceph-mon[86379]: pgmap v110: 65 pgs: 16 active+undersized, 16 active+undersized+degraded, 33 active+clean; 257 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 50/324 objects degraded (15.432%) 2024-04-12T21:04:23.600 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:04:23 smithi018 ceph-mon[112590]: pgmap v110: 65 pgs: 16 active+undersized, 16 active+undersized+degraded, 33 active+clean; 257 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 50/324 objects degraded (15.432%) 2024-04-12T21:04:23.601 INFO:journalctl@ceph.osd.1.smithi018.stdout:Apr 12 21:04:23 smithi018 ceph-1cbe407a-f90d-11ee-bc8d-c7b262605968-osd-1[130044]: 2024-04-12T21:04:23.318+0000 7f5ebd5457c0 -1 osd.1 73 log_to_monitors true 2024-04-12T21:04:24.529 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:04:24 smithi186 ceph-mon[86379]: from='osd.1 [v2:172.21.15.18:6810/1955801845,v1:172.21.15.18:6811/1955801845]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-12T21:04:24.600 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:04:24 smithi018 ceph-mon[112590]: from='osd.1 [v2:172.21.15.18:6810/1955801845,v1:172.21.15.18:6811/1955801845]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-12T21:04:24.601 INFO:journalctl@ceph.osd.1.smithi018.stdout:Apr 12 21:04:24 smithi018 ceph-1cbe407a-f90d-11ee-bc8d-c7b262605968-osd-1[130044]: 2024-04-12T21:04:24.275+0000 7f5eac800700 -1 osd.1 73 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-12T21:04:25.530 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:04:25 smithi186 ceph-mon[86379]: pgmap v111: 65 pgs: 16 active+undersized, 16 active+undersized+degraded, 33 active+clean; 257 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 50/324 objects degraded (15.432%) 2024-04-12T21:04:25.530 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:04:25 smithi186 ceph-mon[86379]: from='osd.1 [v2:172.21.15.18:6810/1955801845,v1:172.21.15.18:6811/1955801845]' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-04-12T21:04:25.530 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:04:25 smithi186 ceph-mon[86379]: osdmap e76: 6 total, 5 up, 6 in 2024-04-12T21:04:25.530 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:04:25 smithi186 ceph-mon[86379]: from='osd.1 [v2:172.21.15.18:6810/1955801845,v1:172.21.15.18:6811/1955801845]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi018", "root=default"]}]: dispatch 2024-04-12T21:04:25.530 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:04:25 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' 2024-04-12T21:04:25.530 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:04:25 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' 2024-04-12T21:04:25.530 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:04:25 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' 2024-04-12T21:04:25.530 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:04:25 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T21:04:25.601 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:04:25 smithi018 ceph-mon[112590]: pgmap v111: 65 pgs: 16 active+undersized, 16 active+undersized+degraded, 33 active+clean; 257 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 50/324 objects degraded (15.432%) 2024-04-12T21:04:25.601 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:04:25 smithi018 ceph-mon[112590]: from='osd.1 [v2:172.21.15.18:6810/1955801845,v1:172.21.15.18:6811/1955801845]' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-04-12T21:04:25.601 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:04:25 smithi018 ceph-mon[112590]: osdmap e76: 6 total, 5 up, 6 in 2024-04-12T21:04:25.601 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:04:25 smithi018 ceph-mon[112590]: from='osd.1 [v2:172.21.15.18:6810/1955801845,v1:172.21.15.18:6811/1955801845]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi018", "root=default"]}]: dispatch 2024-04-12T21:04:25.601 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:04:25 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' 2024-04-12T21:04:25.601 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:04:25 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' 2024-04-12T21:04:25.601 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:04:25 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' 2024-04-12T21:04:25.601 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:04:25 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T21:04:26.600 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:04:26 smithi018 ceph-mon[112590]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-04-12T21:04:26.600 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:04:26 smithi018 ceph-mon[112590]: osd.1 [v2:172.21.15.18:6810/1955801845,v1:172.21.15.18:6811/1955801845] boot 2024-04-12T21:04:26.600 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:04:26 smithi018 ceph-mon[112590]: osdmap e77: 6 total, 6 up, 6 in 2024-04-12T21:04:26.600 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:04:26 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-12T21:04:26.601 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:04:26 smithi018 ceph-mon[112590]: osdmap e78: 6 total, 6 up, 6 in 2024-04-12T21:04:26.780 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:04:26 smithi186 ceph-mon[86379]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-04-12T21:04:26.780 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:04:26 smithi186 ceph-mon[86379]: osd.1 [v2:172.21.15.18:6810/1955801845,v1:172.21.15.18:6811/1955801845] boot 2024-04-12T21:04:26.780 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:04:26 smithi186 ceph-mon[86379]: osdmap e77: 6 total, 6 up, 6 in 2024-04-12T21:04:26.780 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:04:26 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-12T21:04:26.780 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:04:26 smithi186 ceph-mon[86379]: osdmap e78: 6 total, 6 up, 6 in 2024-04-12T21:04:27.600 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:04:27 smithi018 ceph-mon[112590]: pgmap v114: 65 pgs: 18 peering, 7 active+undersized, 7 active+undersized+degraded, 33 active+clean; 257 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 21/324 objects degraded (6.481%) 2024-04-12T21:04:27.600 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:04:27 smithi018 ceph-mon[112590]: Health check update: Degraded data redundancy: 21/324 objects degraded (6.481%), 7 pgs degraded (PG_DEGRADED) 2024-04-12T21:04:27.600 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:04:27 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' 2024-04-12T21:04:27.601 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:04:27 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' 2024-04-12T21:04:27.779 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:04:27 smithi186 ceph-mon[86379]: pgmap v114: 65 pgs: 18 peering, 7 active+undersized, 7 active+undersized+degraded, 33 active+clean; 257 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 21/324 objects degraded (6.481%) 2024-04-12T21:04:27.780 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:04:27 smithi186 ceph-mon[86379]: Health check update: Degraded data redundancy: 21/324 objects degraded (6.481%), 7 pgs degraded (PG_DEGRADED) 2024-04-12T21:04:27.780 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:04:27 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' 2024-04-12T21:04:27.780 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:04:27 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' 2024-04-12T21:04:29.601 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:04:29 smithi018 ceph-mon[112590]: pgmap v116: 65 pgs: 20 peering, 5 active+undersized, 7 active+undersized+degraded, 33 active+clean; 257 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 21/324 objects degraded (6.481%) 2024-04-12T21:04:29.779 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:04:29 smithi186 ceph-mon[86379]: pgmap v116: 65 pgs: 20 peering, 5 active+undersized, 7 active+undersized+degraded, 33 active+clean; 257 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 21/324 objects degraded (6.481%) 2024-04-12T21:04:30.600 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:04:30 smithi018 ceph-mon[112590]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 21/324 objects degraded (6.481%), 7 pgs degraded) 2024-04-12T21:04:30.601 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:04:30 smithi018 ceph-mon[112590]: Cluster is now healthy 2024-04-12T21:04:30.779 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:04:30 smithi186 ceph-mon[86379]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 21/324 objects degraded (6.481%), 7 pgs degraded) 2024-04-12T21:04:30.779 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:04:30 smithi186 ceph-mon[86379]: Cluster is now healthy 2024-04-12T21:04:31.600 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:04:31 smithi018 ceph-mon[112590]: pgmap v117: 65 pgs: 20 peering, 45 active+clean; 257 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail 2024-04-12T21:04:31.779 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:04:31 smithi186 ceph-mon[86379]: pgmap v117: 65 pgs: 20 peering, 45 active+clean; 257 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail 2024-04-12T21:04:33.695 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:04:33 smithi018 ceph-mon[112590]: pgmap v118: 65 pgs: 65 active+clean; 257 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail 2024-04-12T21:04:33.695 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:04:33 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' 2024-04-12T21:04:33.695 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:04:33 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' 2024-04-12T21:04:33.695 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:04:33 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-12T21:04:33.695 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:04:33 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-12T21:04:33.695 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:04:33 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' 2024-04-12T21:04:33.695 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:04:33 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-12T21:04:33.695 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:04:33 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T21:04:33.695 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:04:33 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T21:04:33.696 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:04:33 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T21:04:33.696 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:04:33 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd ok-to-stop", "ids": ["2"], "max": 16}]: dispatch 2024-04-12T21:04:33.780 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:04:33 smithi186 ceph-mon[86379]: pgmap v118: 65 pgs: 65 active+clean; 257 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail 2024-04-12T21:04:33.780 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:04:33 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' 2024-04-12T21:04:33.780 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:04:33 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' 2024-04-12T21:04:33.780 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:04:33 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-12T21:04:33.780 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:04:33 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-12T21:04:33.780 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:04:33 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' 2024-04-12T21:04:33.780 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:04:33 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-12T21:04:33.780 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:04:33 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T21:04:33.781 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:04:33 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T21:04:33.781 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:04:33 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T21:04:33.781 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:04:33 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd ok-to-stop", "ids": ["2"], "max": 16}]: dispatch 2024-04-12T21:04:34.600 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:04:34 smithi018 ceph-mon[112590]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["2"], "max": 16}]: dispatch 2024-04-12T21:04:34.600 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:04:34 smithi018 ceph-mon[112590]: Upgrade: osd.2 is safe to restart 2024-04-12T21:04:34.779 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:04:34 smithi186 ceph-mon[86379]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["2"], "max": 16}]: dispatch 2024-04-12T21:04:34.780 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:04:34 smithi186 ceph-mon[86379]: Upgrade: osd.2 is safe to restart 2024-04-12T21:04:35.779 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:04:35 smithi186 ceph-mon[86379]: pgmap v119: 65 pgs: 65 active+clean; 257 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail 2024-04-12T21:04:35.780 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:04:35 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' 2024-04-12T21:04:35.780 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:04:35 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-04-12T21:04:35.780 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:04:35 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-12T21:04:35.850 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:04:35 smithi018 ceph-mon[112590]: pgmap v119: 65 pgs: 65 active+clean; 257 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail 2024-04-12T21:04:35.851 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:04:35 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' 2024-04-12T21:04:35.851 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:04:35 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-04-12T21:04:35.851 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:04:35 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-12T21:04:36.779 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:04:36 smithi186 ceph-mon[86379]: Upgrade: Updating osd.2 2024-04-12T21:04:36.780 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:04:36 smithi186 ceph-mon[86379]: Deploying daemon osd.2 on smithi018 2024-04-12T21:04:36.850 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:04:36 smithi018 ceph-mon[112590]: Upgrade: Updating osd.2 2024-04-12T21:04:36.851 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:04:36 smithi018 ceph-mon[112590]: Deploying daemon osd.2 on smithi018 2024-04-12T21:04:37.600 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:04:37 smithi018 ceph-mon[112590]: pgmap v120: 65 pgs: 65 active+clean; 257 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail 2024-04-12T21:04:37.779 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:04:37 smithi186 ceph-mon[86379]: pgmap v120: 65 pgs: 65 active+clean; 257 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail 2024-04-12T21:04:39.260 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:39.263+0000 7f6ef4800700 1 -- 172.21.15.18:0/1344120406 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6eec0fde90 msgr2=0x7f6eec0fe260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:04:39.261 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:39.263+0000 7f6ef4800700 1 --2- 172.21.15.18:0/1344120406 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6eec0fde90 0x7f6eec0fe260 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f6ee0004940 tx=0x7f6ee0009e30 comp rx=0 tx=0).stop 2024-04-12T21:04:39.261 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:39.263+0000 7f6ef4800700 1 -- 172.21.15.18:0/1344120406 shutdown_connections 2024-04-12T21:04:39.261 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:39.263+0000 7f6ef4800700 1 --2- 172.21.15.18:0/1344120406 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6eec0fe830 0x7f6eec106f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:04:39.261 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:39.263+0000 7f6ef4800700 1 --2- 172.21.15.18:0/1344120406 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6eec0fde90 0x7f6eec0fe260 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:04:39.261 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:39.263+0000 7f6ef4800700 1 -- 172.21.15.18:0/1344120406 >> 172.21.15.18:0/1344120406 conn(0x7f6eec072bc0 msgr2=0x7f6eec072fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:04:39.261 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:39.264+0000 7f6ef4800700 1 -- 172.21.15.18:0/1344120406 shutdown_connections 2024-04-12T21:04:39.261 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:39.264+0000 7f6ef4800700 1 -- 172.21.15.18:0/1344120406 wait complete. 2024-04-12T21:04:39.263 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:39.264+0000 7f6ef4800700 1 Processor -- start 2024-04-12T21:04:39.263 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:39.265+0000 7f6ef4800700 1 -- start start 2024-04-12T21:04:39.263 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:39.265+0000 7f6ef4800700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6eec0fde90 0x7f6eec115d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:04:39.263 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:39.265+0000 7f6ef4800700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6eec0fe830 0x7f6eec116280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:04:39.263 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:39.265+0000 7f6ef4800700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6eec116960 con 0x7f6eec0fe830 2024-04-12T21:04:39.263 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:39.265+0000 7f6ef4800700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6eec10fce0 con 0x7f6eec0fde90 2024-04-12T21:04:39.263 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:39.265+0000 7f6ef1400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6eec0fe830 0x7f6eec116280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:04:39.263 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:39.265+0000 7f6ef1400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6eec0fe830 0x7f6eec116280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:40428/0 (socket says 172.21.15.18:40428) 2024-04-12T21:04:39.263 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:39.265+0000 7f6ef1400700 1 -- 172.21.15.18:0/1696742076 learned_addr learned my addr 172.21.15.18:0/1696742076 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:04:39.264 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:39.266+0000 7f6ef1e00700 1 --2- 172.21.15.18:0/1696742076 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6eec0fde90 0x7f6eec115d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:04:39.264 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:39.266+0000 7f6ef1400700 1 -- 172.21.15.18:0/1696742076 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6eec0fde90 msgr2=0x7f6eec115d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:04:39.264 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:39.266+0000 7f6ef1400700 1 --2- 172.21.15.18:0/1696742076 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6eec0fde90 0x7f6eec115d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:04:39.264 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:39.266+0000 7f6ef1400700 1 -- 172.21.15.18:0/1696742076 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ee0005040 con 0x7f6eec0fe830 2024-04-12T21:04:39.264 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:39.266+0000 7f6ef1400700 1 --2- 172.21.15.18:0/1696742076 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6eec0fe830 0x7f6eec116280 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f6edc00b790 tx=0x7f6edc009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:04:39.264 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:39.266+0000 7f6ee6000700 1 -- 172.21.15.18:0/1696742076 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6edc00f770 con 0x7f6eec0fe830 2024-04-12T21:04:39.264 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:39.267+0000 7f6ee6000700 1 -- 172.21.15.18:0/1696742076 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f6edc00fda0 con 0x7f6eec0fe830 2024-04-12T21:04:39.264 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:39.267+0000 7f6ef4800700 1 -- 172.21.15.18:0/1696742076 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6eec10ffc0 con 0x7f6eec0fe830 2024-04-12T21:04:39.264 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:39.267+0000 7f6ee6000700 1 -- 172.21.15.18:0/1696742076 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6edc00d310 con 0x7f6eec0fe830 2024-04-12T21:04:39.264 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:39.267+0000 7f6ef4800700 1 -- 172.21.15.18:0/1696742076 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6eec110500 con 0x7f6eec0fe830 2024-04-12T21:04:39.265 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:39.268+0000 7f6ef4800700 1 -- 172.21.15.18:0/1696742076 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6eec106a20 con 0x7f6eec0fe830 2024-04-12T21:04:39.265 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:39.269+0000 7f6ee6000700 1 -- 172.21.15.18:0/1696742076 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7f6edc010420 con 0x7f6eec0fe830 2024-04-12T21:04:39.266 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:39.270+0000 7f6ee6000700 1 --2- 172.21.15.18:0/1696742076 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f6ed406ec10 0x7f6ed40710c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:04:39.266 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:39.270+0000 7f6ee6000700 1 -- 172.21.15.18:0/1696742076 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 6585+0+0 (secure 0 0 0) 0x7f6edc08e8e0 con 0x7f6eec0fe830 2024-04-12T21:04:39.266 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:39.270+0000 7f6ef1e00700 1 --2- 172.21.15.18:0/1696742076 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f6ed406ec10 0x7f6ed40710c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:04:39.267 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:39.271+0000 7f6ef1e00700 1 --2- 172.21.15.18:0/1696742076 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f6ed406ec10 0x7f6ed40710c0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f6ee0004f90 tx=0x7f6ee0009e30 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:04:39.271 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:39.275+0000 7f6ee6000700 1 -- 172.21.15.18:0/1696742076 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f6edc057690 con 0x7f6eec0fe830 2024-04-12T21:04:39.533 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:39.537+0000 7f6ef4800700 1 -- 172.21.15.18:0/1696742076 --> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f6eec1113d0 con 0x7f6ed406ec10 2024-04-12T21:04:39.535 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:39.539+0000 7f6ee6000700 1 -- 172.21.15.18:0/1696742076 <== mgr.34102 v2:172.21.15.18:6800/3812891919 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7f6eec1113d0 con 0x7f6ed406ec10 2024-04-12T21:04:39.538 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:39.542+0000 7f6ef4800700 1 -- 172.21.15.18:0/1696742076 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f6ed406ec10 msgr2=0x7f6ed40710c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:04:39.538 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:39.542+0000 7f6ef4800700 1 --2- 172.21.15.18:0/1696742076 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f6ed406ec10 0x7f6ed40710c0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f6ee0004f90 tx=0x7f6ee0009e30 comp rx=0 tx=0).stop 2024-04-12T21:04:39.538 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:39.542+0000 7f6ef4800700 1 -- 172.21.15.18:0/1696742076 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6eec0fe830 msgr2=0x7f6eec116280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:04:39.538 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:39.542+0000 7f6ef4800700 1 --2- 172.21.15.18:0/1696742076 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6eec0fe830 0x7f6eec116280 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f6edc00b790 tx=0x7f6edc009250 comp rx=0 tx=0).stop 2024-04-12T21:04:39.539 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:39.542+0000 7f6ef4800700 1 -- 172.21.15.18:0/1696742076 shutdown_connections 2024-04-12T21:04:39.539 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:39.542+0000 7f6ef4800700 1 --2- 172.21.15.18:0/1696742076 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f6ed406ec10 0x7f6ed40710c0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:04:39.539 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:39.542+0000 7f6ef4800700 1 --2- 172.21.15.18:0/1696742076 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6eec0fde90 0x7f6eec115d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:04:39.539 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:39.542+0000 7f6ef4800700 1 --2- 172.21.15.18:0/1696742076 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6eec0fe830 0x7f6eec116280 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:04:39.539 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:39.542+0000 7f6ef4800700 1 -- 172.21.15.18:0/1696742076 >> 172.21.15.18:0/1696742076 conn(0x7f6eec072bc0 msgr2=0x7f6eec0fb700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:04:39.539 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:39.543+0000 7f6ef4800700 1 -- 172.21.15.18:0/1696742076 shutdown_connections 2024-04-12T21:04:39.539 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:39.543+0000 7f6ef4800700 1 -- 172.21.15.18:0/1696742076 wait complete. 2024-04-12T21:04:39.555 INFO:teuthology.orchestra.run.smithi018.stdout:true 2024-04-12T21:04:39.702 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:04:39 smithi018 ceph-mon[112590]: pgmap v121: 65 pgs: 65 active+clean; 257 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail 2024-04-12T21:04:39.703 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:39.705+0000 7f013c400700 1 -- 172.21.15.18:0/1085790891 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0134102e20 msgr2=0x7f01341031f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:04:39.703 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:39.705+0000 7f013c400700 1 --2- 172.21.15.18:0/1085790891 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0134102e20 0x7f01341031f0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f0128004940 tx=0x7f0128009e30 comp rx=0 tx=0).stop 2024-04-12T21:04:39.703 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:39.705+0000 7f013c400700 1 -- 172.21.15.18:0/1085790891 shutdown_connections 2024-04-12T21:04:39.703 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:39.705+0000 7f013c400700 1 --2- 172.21.15.18:0/1085790891 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f01341037c0 0x7f01340fce30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:04:39.703 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:39.705+0000 7f013c400700 1 --2- 172.21.15.18:0/1085790891 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0134102e20 0x7f01341031f0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:04:39.703 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:39.705+0000 7f013c400700 1 -- 172.21.15.18:0/1085790891 >> 172.21.15.18:0/1085790891 conn(0x7f01340f8970 msgr2=0x7f01340fad80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:04:39.703 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:39.705+0000 7f013c400700 1 -- 172.21.15.18:0/1085790891 shutdown_connections 2024-04-12T21:04:39.703 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:39.705+0000 7f013c400700 1 -- 172.21.15.18:0/1085790891 wait complete. 2024-04-12T21:04:39.703 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:39.707+0000 7f013c400700 1 Processor -- start 2024-04-12T21:04:39.703 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:39.707+0000 7f013c400700 1 -- start start 2024-04-12T21:04:39.704 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:39.707+0000 7f013c400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0134102e20 0x7f0134115d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:04:39.704 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:39.707+0000 7f013c400700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f01341037c0 0x7f01341162a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:04:39.704 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:39.707+0000 7f013c400700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01341168f0 con 0x7f0134102e20 2024-04-12T21:04:39.704 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:39.707+0000 7f013c400700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f013410fcc0 con 0x7f01341037c0 2024-04-12T21:04:39.705 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:39.708+0000 7f0139a00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0134102e20 0x7f0134115d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:04:39.705 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:39.708+0000 7f0139a00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0134102e20 0x7f0134115d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:40440/0 (socket says 172.21.15.18:40440) 2024-04-12T21:04:39.705 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:39.708+0000 7f0139a00700 1 -- 172.21.15.18:0/3166322879 learned_addr learned my addr 172.21.15.18:0/3166322879 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:04:39.705 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:39.708+0000 7f0139a00700 1 -- 172.21.15.18:0/3166322879 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f01341037c0 msgr2=0x7f01341162a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:04:39.705 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:39.708+0000 7f0139000700 1 --2- 172.21.15.18:0/3166322879 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f01341037c0 0x7f01341162a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:04:39.705 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:39.708+0000 7f0139a00700 1 --2- 172.21.15.18:0/3166322879 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f01341037c0 0x7f01341162a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:04:39.705 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:39.708+0000 7f0139a00700 1 -- 172.21.15.18:0/3166322879 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0128005040 con 0x7f0134102e20 2024-04-12T21:04:39.705 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:39.708+0000 7f0139000700 1 --2- 172.21.15.18:0/3166322879 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f01341037c0 0x7f01341162a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-12T21:04:39.705 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:39.709+0000 7f0139a00700 1 --2- 172.21.15.18:0/3166322879 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0134102e20 0x7f0134115d60 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f012800b3b0 tx=0x7f0128009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:04:39.706 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:39.709+0000 7f0131600700 1 -- 172.21.15.18:0/3166322879 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f012800e040 con 0x7f0134102e20 2024-04-12T21:04:39.706 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:39.709+0000 7f013c400700 1 -- 172.21.15.18:0/3166322879 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f013410ff40 con 0x7f0134102e20 2024-04-12T21:04:39.706 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:39.709+0000 7f0131600700 1 -- 172.21.15.18:0/3166322879 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f012800bbc0 con 0x7f0134102e20 2024-04-12T21:04:39.706 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:39.709+0000 7f0131600700 1 -- 172.21.15.18:0/3166322879 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f012801c3f0 con 0x7f0134102e20 2024-04-12T21:04:39.706 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:39.709+0000 7f013c400700 1 -- 172.21.15.18:0/3166322879 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0134110500 con 0x7f0134102e20 2024-04-12T21:04:39.707 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:39.711+0000 7f0126a00700 1 -- 172.21.15.18:0/3166322879 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f01100052b0 con 0x7f0134102e20 2024-04-12T21:04:39.709 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:39.714+0000 7f0131600700 1 -- 172.21.15.18:0/3166322879 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7f012800bd30 con 0x7f0134102e20 2024-04-12T21:04:39.710 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:39.715+0000 7f0131600700 1 --2- 172.21.15.18:0/3166322879 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f011c06ec00 0x7f011c0710b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:04:39.711 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:39.715+0000 7f0131600700 1 -- 172.21.15.18:0/3166322879 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 6585+0+0 (secure 0 0 0) 0x7f012808f9e0 con 0x7f0134102e20 2024-04-12T21:04:39.711 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:39.715+0000 7f0139000700 1 --2- 172.21.15.18:0/3166322879 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f011c06ec00 0x7f011c0710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:04:39.712 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:39.716+0000 7f0139000700 1 --2- 172.21.15.18:0/3166322879 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f011c06ec00 0x7f011c0710b0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f0120009fd0 tx=0x7f012000b040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:04:39.713 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:39.717+0000 7f0131600700 1 -- 172.21.15.18:0/3166322879 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f012805c240 con 0x7f0134102e20 2024-04-12T21:04:39.779 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:04:39 smithi186 ceph-mon[86379]: pgmap v121: 65 pgs: 65 active+clean; 257 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail 2024-04-12T21:04:39.977 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:39.980+0000 7f0126a00700 1 -- 172.21.15.18:0/3166322879 --> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f0110000bc0 con 0x7f011c06ec00 2024-04-12T21:04:39.978 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:39.982+0000 7f0131600700 1 -- 172.21.15.18:0/3166322879 <== mgr.34102 v2:172.21.15.18:6800/3812891919 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7f0110000bc0 con 0x7f011c06ec00 2024-04-12T21:04:39.981 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:39.985+0000 7f0126a00700 1 -- 172.21.15.18:0/3166322879 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f011c06ec00 msgr2=0x7f011c0710b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:04:39.981 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:39.985+0000 7f0126a00700 1 --2- 172.21.15.18:0/3166322879 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f011c06ec00 0x7f011c0710b0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f0120009fd0 tx=0x7f012000b040 comp rx=0 tx=0).stop 2024-04-12T21:04:39.981 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:39.985+0000 7f0126a00700 1 -- 172.21.15.18:0/3166322879 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0134102e20 msgr2=0x7f0134115d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:04:39.982 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:39.985+0000 7f0126a00700 1 --2- 172.21.15.18:0/3166322879 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0134102e20 0x7f0134115d60 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f012800b3b0 tx=0x7f0128009e30 comp rx=0 tx=0).stop 2024-04-12T21:04:39.982 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:39.985+0000 7f0126a00700 1 -- 172.21.15.18:0/3166322879 shutdown_connections 2024-04-12T21:04:39.982 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:39.985+0000 7f0126a00700 1 --2- 172.21.15.18:0/3166322879 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f011c06ec00 0x7f011c0710b0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:04:39.982 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:39.985+0000 7f0126a00700 1 --2- 172.21.15.18:0/3166322879 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0134102e20 0x7f0134115d60 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:04:39.982 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:39.985+0000 7f0126a00700 1 --2- 172.21.15.18:0/3166322879 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f01341037c0 0x7f01341162a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:04:39.982 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:39.985+0000 7f0126a00700 1 -- 172.21.15.18:0/3166322879 >> 172.21.15.18:0/3166322879 conn(0x7f01340f8970 msgr2=0x7f0134104120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:04:39.982 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:39.985+0000 7f0126a00700 1 -- 172.21.15.18:0/3166322879 shutdown_connections 2024-04-12T21:04:39.982 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:39.986+0000 7f0126a00700 1 -- 172.21.15.18:0/3166322879 wait complete. 2024-04-12T21:04:40.148 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:40.151+0000 7fa1e8800700 1 -- 172.21.15.18:0/4249478584 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa1e0105010 msgr2=0x7fa1e01053e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:04:40.148 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:40.151+0000 7fa1e8800700 1 --2- 172.21.15.18:0/4249478584 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa1e0105010 0x7fa1e01053e0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fa1d4004940 tx=0x7fa1d4009e30 comp rx=0 tx=0).stop 2024-04-12T21:04:40.148 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:40.152+0000 7fa1e8800700 1 -- 172.21.15.18:0/4249478584 shutdown_connections 2024-04-12T21:04:40.148 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:40.152+0000 7fa1e8800700 1 --2- 172.21.15.18:0/4249478584 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa1e01059b0 0x7fa1e00ff020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:04:40.148 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:40.152+0000 7fa1e8800700 1 --2- 172.21.15.18:0/4249478584 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa1e0105010 0x7fa1e01053e0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:04:40.149 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:40.152+0000 7fa1e8800700 1 -- 172.21.15.18:0/4249478584 >> 172.21.15.18:0/4249478584 conn(0x7fa1e00faba0 msgr2=0x7fa1e00fcfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:04:40.149 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:40.152+0000 7fa1e8800700 1 -- 172.21.15.18:0/4249478584 shutdown_connections 2024-04-12T21:04:40.149 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:40.152+0000 7fa1e8800700 1 -- 172.21.15.18:0/4249478584 wait complete. 2024-04-12T21:04:40.149 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:40.153+0000 7fa1e8800700 1 Processor -- start 2024-04-12T21:04:40.150 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:40.153+0000 7fa1e8800700 1 -- start start 2024-04-12T21:04:40.150 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:40.153+0000 7fa1e8800700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa1e0105010 0x7fa1e0115da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:04:40.150 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:40.153+0000 7fa1e8800700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa1e01059b0 0x7fa1e01162e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:04:40.150 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:40.153+0000 7fa1e8800700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1e0116930 con 0x7fa1e01059b0 2024-04-12T21:04:40.150 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:40.153+0000 7fa1e8800700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1e010fd00 con 0x7fa1e0105010 2024-04-12T21:04:40.150 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:40.154+0000 7fa1e5400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa1e01059b0 0x7fa1e01162e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:04:40.151 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:40.154+0000 7fa1e5400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa1e01059b0 0x7fa1e01162e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:40456/0 (socket says 172.21.15.18:40456) 2024-04-12T21:04:40.151 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:40.154+0000 7fa1e5400700 1 -- 172.21.15.18:0/3343524036 learned_addr learned my addr 172.21.15.18:0/3343524036 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:04:40.151 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:40.154+0000 7fa1e5e00700 1 --2- 172.21.15.18:0/3343524036 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa1e0105010 0x7fa1e0115da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:04:40.151 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:40.154+0000 7fa1e5400700 1 -- 172.21.15.18:0/3343524036 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa1e0105010 msgr2=0x7fa1e0115da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:04:40.151 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:40.154+0000 7fa1e5400700 1 --2- 172.21.15.18:0/3343524036 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa1e0105010 0x7fa1e0115da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:04:40.152 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:40.154+0000 7fa1e5400700 1 -- 172.21.15.18:0/3343524036 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1d4005040 con 0x7fa1e01059b0 2024-04-12T21:04:40.152 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:40.155+0000 7fa1e5400700 1 --2- 172.21.15.18:0/3343524036 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa1e01059b0 0x7fa1e01162e0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fa1d0002a50 tx=0x7fa1d0004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:04:40.152 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:40.155+0000 7fa1da000700 1 -- 172.21.15.18:0/3343524036 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa1d0007cc0 con 0x7fa1e01059b0 2024-04-12T21:04:40.152 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:40.155+0000 7fa1da000700 1 -- 172.21.15.18:0/3343524036 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa1d000f040 con 0x7fa1e01059b0 2024-04-12T21:04:40.152 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:40.155+0000 7fa1da000700 1 -- 172.21.15.18:0/3343524036 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa1d000da40 con 0x7fa1e01059b0 2024-04-12T21:04:40.152 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:40.155+0000 7fa1e8800700 1 -- 172.21.15.18:0/3343524036 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa1e010ffe0 con 0x7fa1e01059b0 2024-04-12T21:04:40.152 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:40.155+0000 7fa1e8800700 1 -- 172.21.15.18:0/3343524036 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa1e01105a0 con 0x7fa1e01059b0 2024-04-12T21:04:40.152 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:40.156+0000 7fa1e8800700 1 -- 172.21.15.18:0/3343524036 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa1e0100760 con 0x7fa1e01059b0 2024-04-12T21:04:40.154 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:40.158+0000 7fa1da000700 1 -- 172.21.15.18:0/3343524036 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7fa1d00097c0 con 0x7fa1e01059b0 2024-04-12T21:04:40.156 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:40.159+0000 7fa1da000700 1 --2- 172.21.15.18:0/3343524036 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fa1c806ec10 0x7fa1c80710c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:04:40.156 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:40.159+0000 7fa1da000700 1 -- 172.21.15.18:0/3343524036 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 6585+0+0 (secure 0 0 0) 0x7fa1d008e7b0 con 0x7fa1e01059b0 2024-04-12T21:04:40.156 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:40.160+0000 7fa1e5e00700 1 --2- 172.21.15.18:0/3343524036 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fa1c806ec10 0x7fa1c80710c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:04:40.156 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:40.160+0000 7fa1e5e00700 1 --2- 172.21.15.18:0/3343524036 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fa1c806ec10 0x7fa1c80710c0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fa1d400b020 tx=0x7fa1d4009e30 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:04:40.160 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:40.165+0000 7fa1da000700 1 -- 172.21.15.18:0/3343524036 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7fa1d005ad00 con 0x7fa1e01059b0 2024-04-12T21:04:40.407 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:40.410+0000 7fa1e8800700 1 -- 172.21.15.18:0/3343524036 --> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fa1e010c180 con 0x7fa1c806ec10 2024-04-12T21:04:40.417 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:40.421+0000 7fa1da000700 1 -- 172.21.15.18:0/3343524036 <== mgr.34102 v2:172.21.15.18:6800/3812891919 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7fa1e010c180 con 0x7fa1c806ec10 2024-04-12T21:04:40.418 INFO:teuthology.orchestra.run.smithi018.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-12T21:04:40.418 INFO:teuthology.orchestra.run.smithi018.stdout:alertmanager.smithi018 smithi018 *:9093,9094 running (5m) 15s ago 19m 14.5M - 0.25.0 c8568f914cd2 c332090068e8 2024-04-12T21:04:40.418 INFO:teuthology.orchestra.run.smithi018.stdout:crash.smithi018 smithi018 running (95s) 15s ago 20m 6665k - 19.0.0-2879-g324baf8d 8717720dc5e5 b7f80d1f7982 2024-04-12T21:04:40.418 INFO:teuthology.orchestra.run.smithi018.stdout:crash.smithi186 smithi186 running (88s) 83s ago 18m 6660k - 19.0.0-2879-g324baf8d 8717720dc5e5 a4f55c81b623 2024-04-12T21:04:40.418 INFO:teuthology.orchestra.run.smithi018.stdout:grafana.smithi018 smithi018 *:3000 running (4m) 15s ago 19m 74.7M - 9.4.12 2bacad6d85d8 b08e3bf176dd 2024-04-12T21:04:40.418 INFO:teuthology.orchestra.run.smithi018.stdout:mds.cephfs.smithi018.acqbsc smithi018 running (12m) 15s ago 12m 112M - 17.2.7-827-gd3b5060a 21de9b7369ad 5b5fce65f18a 2024-04-12T21:04:40.418 INFO:teuthology.orchestra.run.smithi018.stdout:mds.cephfs.smithi018.szjluo smithi018 running (12m) 15s ago 12m 113M - 17.2.7-827-gd3b5060a 21de9b7369ad cf9bf0c5031f 2024-04-12T21:04:40.418 INFO:teuthology.orchestra.run.smithi018.stdout:mds.cephfs.smithi186.jdllyn smithi186 running (12m) 83s ago 12m 14.7M - 17.2.7-827-gd3b5060a 21de9b7369ad 2b2bc46bd29a 2024-04-12T21:04:40.418 INFO:teuthology.orchestra.run.smithi018.stdout:mds.cephfs.smithi186.wjnafs smithi186 running (12m) 83s ago 12m 14.4M - 17.2.7-827-gd3b5060a 21de9b7369ad 74b115d6d6fd 2024-04-12T21:04:40.418 INFO:teuthology.orchestra.run.smithi018.stdout:mgr.smithi018.xjzwga smithi018 *:8443,9283,8765 running (7m) 15s ago 21m 503M - 19.0.0-2879-g324baf8d 8717720dc5e5 56088305eab7 2024-04-12T21:04:40.419 INFO:teuthology.orchestra.run.smithi018.stdout:mgr.smithi186.kqcfxk smithi186 *:8443,9283,8765 running (6m) 83s ago 18m 450M - 19.0.0-2879-g324baf8d 8717720dc5e5 09cd766e5e0d 2024-04-12T21:04:40.419 INFO:teuthology.orchestra.run.smithi018.stdout:mon.smithi018 smithi018 running (3m) 15s ago 21m 49.6M 2048M 19.0.0-2879-g324baf8d 8717720dc5e5 df3e201dfed6 2024-04-12T21:04:40.419 INFO:teuthology.orchestra.run.smithi018.stdout:mon.smithi186 smithi186 running (2m) 83s ago 18m 31.6M 2048M 19.0.0-2879-g324baf8d 8717720dc5e5 c77498da9b2b 2024-04-12T21:04:40.419 INFO:teuthology.orchestra.run.smithi018.stdout:node-exporter.smithi018 smithi018 *:9100 running (6m) 15s ago 19m 7721k - 1.5.0 0da6a335fe13 326c3d9c6847 2024-04-12T21:04:40.419 INFO:teuthology.orchestra.run.smithi018.stdout:node-exporter.smithi186 smithi186 *:9100 running (6m) 83s ago 18m 7818k - 1.5.0 0da6a335fe13 ec2112baeecf 2024-04-12T21:04:40.419 INFO:teuthology.orchestra.run.smithi018.stdout:osd.0 smithi018 running (56s) 15s ago 17m 117M 1559M 19.0.0-2879-g324baf8d 8717720dc5e5 60b1273ca047 2024-04-12T21:04:40.419 INFO:teuthology.orchestra.run.smithi018.stdout:osd.1 smithi018 running (23s) 15s ago 16m 11.0M 1559M 19.0.0-2879-g324baf8d 8717720dc5e5 170ce1e5fb8d 2024-04-12T21:04:40.419 INFO:teuthology.orchestra.run.smithi018.stdout:osd.2 smithi018 running (16m) 15s ago 16m 209M 1559M 17.2.7-827-gd3b5060a 21de9b7369ad 8a83692acaa5 2024-04-12T21:04:40.419 INFO:teuthology.orchestra.run.smithi018.stdout:osd.3 smithi186 running (15m) 83s ago 15m 264M 2581M 17.2.7-827-gd3b5060a 21de9b7369ad c2def7a90d76 2024-04-12T21:04:40.419 INFO:teuthology.orchestra.run.smithi018.stdout:osd.4 smithi186 running (15m) 83s ago 15m 233M 2581M 17.2.7-827-gd3b5060a 21de9b7369ad 757e7c71b7b9 2024-04-12T21:04:40.419 INFO:teuthology.orchestra.run.smithi018.stdout:osd.5 smithi186 running (14m) 83s ago 14m 228M 2581M 17.2.7-827-gd3b5060a 21de9b7369ad 2ee072a436d0 2024-04-12T21:04:40.419 INFO:teuthology.orchestra.run.smithi018.stdout:prometheus.smithi018 smithi018 *:9095 running (5m) 15s ago 19m 59.0M - 2.43.0 a07b618ecd1d f58841127e21 2024-04-12T21:04:40.421 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:40.424+0000 7fa1e8800700 1 -- 172.21.15.18:0/3343524036 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fa1c806ec10 msgr2=0x7fa1c80710c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:04:40.421 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:40.424+0000 7fa1e8800700 1 --2- 172.21.15.18:0/3343524036 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fa1c806ec10 0x7fa1c80710c0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fa1d400b020 tx=0x7fa1d4009e30 comp rx=0 tx=0).stop 2024-04-12T21:04:40.421 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:40.424+0000 7fa1e8800700 1 -- 172.21.15.18:0/3343524036 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa1e01059b0 msgr2=0x7fa1e01162e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:04:40.421 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:40.424+0000 7fa1e8800700 1 --2- 172.21.15.18:0/3343524036 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa1e01059b0 0x7fa1e01162e0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fa1d0002a50 tx=0x7fa1d0004600 comp rx=0 tx=0).stop 2024-04-12T21:04:40.421 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:40.424+0000 7fa1e8800700 1 -- 172.21.15.18:0/3343524036 shutdown_connections 2024-04-12T21:04:40.421 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:40.424+0000 7fa1e8800700 1 --2- 172.21.15.18:0/3343524036 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fa1c806ec10 0x7fa1c80710c0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:04:40.421 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:40.424+0000 7fa1e8800700 1 --2- 172.21.15.18:0/3343524036 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa1e0105010 0x7fa1e0115da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:04:40.421 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:40.424+0000 7fa1e8800700 1 --2- 172.21.15.18:0/3343524036 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa1e01059b0 0x7fa1e01162e0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:04:40.421 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:40.424+0000 7fa1e8800700 1 -- 172.21.15.18:0/3343524036 >> 172.21.15.18:0/3343524036 conn(0x7fa1e00faba0 msgr2=0x7fa1e00fbc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:04:40.421 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:40.425+0000 7fa1e8800700 1 -- 172.21.15.18:0/3343524036 shutdown_connections 2024-04-12T21:04:40.421 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:40.425+0000 7fa1e8800700 1 -- 172.21.15.18:0/3343524036 wait complete. 2024-04-12T21:04:40.574 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:40.576+0000 7feb19a00700 1 -- 172.21.15.18:0/2966554800 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7feb14109db0 msgr2=0x7feb14103420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:04:40.574 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:40.576+0000 7feb19a00700 1 --2- 172.21.15.18:0/2966554800 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7feb14109db0 0x7feb14103420 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7feb00004940 tx=0x7feb00009e30 comp rx=0 tx=0).stop 2024-04-12T21:04:40.575 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:40.576+0000 7feb19a00700 1 -- 172.21.15.18:0/2966554800 shutdown_connections 2024-04-12T21:04:40.575 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:40.576+0000 7feb19a00700 1 --2- 172.21.15.18:0/2966554800 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7feb14109db0 0x7feb14103420 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:04:40.575 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:40.576+0000 7feb19a00700 1 --2- 172.21.15.18:0/2966554800 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7feb14109410 0x7feb141097e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:04:40.575 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:40.576+0000 7feb19a00700 1 -- 172.21.15.18:0/2966554800 >> 172.21.15.18:0/2966554800 conn(0x7feb14072ba0 msgr2=0x7feb14074fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:04:40.575 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:40.577+0000 7feb19a00700 1 -- 172.21.15.18:0/2966554800 shutdown_connections 2024-04-12T21:04:40.575 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:40.577+0000 7feb19a00700 1 -- 172.21.15.18:0/2966554800 wait complete. 2024-04-12T21:04:40.575 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:40.577+0000 7feb19a00700 1 Processor -- start 2024-04-12T21:04:40.575 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:40.578+0000 7feb19a00700 1 -- start start 2024-04-12T21:04:40.576 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:40.578+0000 7feb19a00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7feb14109410 0x7feb14119fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:04:40.576 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:40.578+0000 7feb19a00700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7feb14109db0 0x7feb1411a510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:04:40.576 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:40.578+0000 7feb19a00700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb1411ab60 con 0x7feb14109410 2024-04-12T21:04:40.577 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:40.578+0000 7feb19a00700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb14114140 con 0x7feb14109db0 2024-04-12T21:04:40.577 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:40.578+0000 7feb12a00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7feb14109410 0x7feb14119fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:04:40.577 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:40.578+0000 7feb12a00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7feb14109410 0x7feb14119fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:40472/0 (socket says 172.21.15.18:40472) 2024-04-12T21:04:40.577 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:40.578+0000 7feb12a00700 1 -- 172.21.15.18:0/2521113816 learned_addr learned my addr 172.21.15.18:0/2521113816 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:04:40.577 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:40.579+0000 7feb12000700 1 --2- 172.21.15.18:0/2521113816 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7feb14109db0 0x7feb1411a510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:04:40.577 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:40.579+0000 7feb12a00700 1 -- 172.21.15.18:0/2521113816 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7feb14109db0 msgr2=0x7feb1411a510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:04:40.577 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:40.579+0000 7feb12a00700 1 --2- 172.21.15.18:0/2521113816 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7feb14109db0 0x7feb1411a510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:04:40.577 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:40.579+0000 7feb12a00700 1 -- 172.21.15.18:0/2521113816 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb00005040 con 0x7feb14109410 2024-04-12T21:04:40.577 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:40.579+0000 7feb12a00700 1 --2- 172.21.15.18:0/2521113816 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7feb14109410 0x7feb14119fd0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7feb0400c830 tx=0x7feb04004470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:04:40.577 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:40.580+0000 7feb0aa00700 1 -- 172.21.15.18:0/2521113816 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7feb0400a760 con 0x7feb14109410 2024-04-12T21:04:40.577 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:40.580+0000 7feb19a00700 1 -- 172.21.15.18:0/2521113816 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feb14114420 con 0x7feb14109410 2024-04-12T21:04:40.577 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:40.580+0000 7feb0aa00700 1 -- 172.21.15.18:0/2521113816 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7feb0400ad90 con 0x7feb14109410 2024-04-12T21:04:40.577 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:40.580+0000 7feb0aa00700 1 -- 172.21.15.18:0/2521113816 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7feb040113a0 con 0x7feb14109410 2024-04-12T21:04:40.577 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:40.580+0000 7feb19a00700 1 -- 172.21.15.18:0/2521113816 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feb141149e0 con 0x7feb14109410 2024-04-12T21:04:40.578 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:40.581+0000 7feb19a00700 1 -- 172.21.15.18:0/2521113816 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feb14104b60 con 0x7feb14109410 2024-04-12T21:04:40.579 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:40.583+0000 7feb0aa00700 1 -- 172.21.15.18:0/2521113816 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7feb04012400 con 0x7feb14109410 2024-04-12T21:04:40.580 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:40.584+0000 7feb0aa00700 1 --2- 172.21.15.18:0/2521113816 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7feaf806ec00 0x7feaf80710b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:04:40.580 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:40.584+0000 7feb0aa00700 1 -- 172.21.15.18:0/2521113816 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 6585+0+0 (secure 0 0 0) 0x7feb0408e8c0 con 0x7feb14109410 2024-04-12T21:04:40.581 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:40.585+0000 7feb12000700 1 --2- 172.21.15.18:0/2521113816 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7feaf806ec00 0x7feaf80710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:04:40.581 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:40.586+0000 7feb12000700 1 --2- 172.21.15.18:0/2521113816 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7feaf806ec00 0x7feaf80710b0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7feb0000b3b0 tx=0x7feb00009e30 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:04:40.585 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:40.589+0000 7feb0aa00700 1 -- 172.21.15.18:0/2521113816 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7feb0405b1a0 con 0x7feb14109410 2024-04-12T21:04:40.792 INFO:journalctl@ceph.osd.2.smithi018.stdout:Apr 12 21:04:40 smithi018 systemd[1]: Stopping Ceph osd.2 for 1cbe407a-f90d-11ee-bc8d-c7b262605968... 2024-04-12T21:04:40.937 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:40.941+0000 7feb19a00700 1 -- 172.21.15.18:0/2521113816 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7feb14115650 con 0x7feb14109410 2024-04-12T21:04:40.938 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:40.942+0000 7feb0aa00700 1 -- 172.21.15.18:0/2521113816 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+839 (secure 0 0 0) 0x7feb0405ad30 con 0x7feb14109410 2024-04-12T21:04:40.939 INFO:teuthology.orchestra.run.smithi018.stdout:{ 2024-04-12T21:04:40.939 INFO:teuthology.orchestra.run.smithi018.stdout: "mon": { 2024-04-12T21:04:40.939 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 19.0.0-2879-g324baf8d (324baf8ddb91dea46cac50cb535c306115791355) squid (dev)": 2 2024-04-12T21:04:40.939 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-12T21:04:40.939 INFO:teuthology.orchestra.run.smithi018.stdout: "mgr": { 2024-04-12T21:04:40.939 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 19.0.0-2879-g324baf8d (324baf8ddb91dea46cac50cb535c306115791355) squid (dev)": 2 2024-04-12T21:04:40.939 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-12T21:04:40.939 INFO:teuthology.orchestra.run.smithi018.stdout: "osd": { 2024-04-12T21:04:40.939 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 17.2.7-827-gd3b5060a (d3b5060a17449edfae9385cdf792afec109fca5d) quincy (stable)": 4, 2024-04-12T21:04:40.939 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 19.0.0-2879-g324baf8d (324baf8ddb91dea46cac50cb535c306115791355) squid (dev)": 2 2024-04-12T21:04:40.940 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-12T21:04:40.940 INFO:teuthology.orchestra.run.smithi018.stdout: "mds": { 2024-04-12T21:04:40.940 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 17.2.7-827-gd3b5060a (d3b5060a17449edfae9385cdf792afec109fca5d) quincy (stable)": 4 2024-04-12T21:04:40.940 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-12T21:04:40.940 INFO:teuthology.orchestra.run.smithi018.stdout: "overall": { 2024-04-12T21:04:40.940 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 17.2.7-827-gd3b5060a (d3b5060a17449edfae9385cdf792afec109fca5d) quincy (stable)": 8, 2024-04-12T21:04:40.940 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 19.0.0-2879-g324baf8d (324baf8ddb91dea46cac50cb535c306115791355) squid (dev)": 6 2024-04-12T21:04:40.940 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-04-12T21:04:40.940 INFO:teuthology.orchestra.run.smithi018.stdout:} 2024-04-12T21:04:40.942 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:40.945+0000 7feb19a00700 1 -- 172.21.15.18:0/2521113816 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7feaf806ec00 msgr2=0x7feaf80710b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:04:40.942 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:40.945+0000 7feb19a00700 1 --2- 172.21.15.18:0/2521113816 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7feaf806ec00 0x7feaf80710b0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7feb0000b3b0 tx=0x7feb00009e30 comp rx=0 tx=0).stop 2024-04-12T21:04:40.942 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:40.945+0000 7feb19a00700 1 -- 172.21.15.18:0/2521113816 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7feb14109410 msgr2=0x7feb14119fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:04:40.942 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:40.945+0000 7feb19a00700 1 --2- 172.21.15.18:0/2521113816 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7feb14109410 0x7feb14119fd0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7feb0400c830 tx=0x7feb04004470 comp rx=0 tx=0).stop 2024-04-12T21:04:40.942 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:40.946+0000 7feb19a00700 1 -- 172.21.15.18:0/2521113816 shutdown_connections 2024-04-12T21:04:40.942 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:40.946+0000 7feb19a00700 1 --2- 172.21.15.18:0/2521113816 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7feaf806ec00 0x7feaf80710b0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:04:40.942 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:40.946+0000 7feb19a00700 1 --2- 172.21.15.18:0/2521113816 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7feb14109410 0x7feb14119fd0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:04:40.942 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:40.946+0000 7feb19a00700 1 --2- 172.21.15.18:0/2521113816 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7feb14109db0 0x7feb1411a510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:04:40.942 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:40.946+0000 7feb19a00700 1 -- 172.21.15.18:0/2521113816 >> 172.21.15.18:0/2521113816 conn(0x7feb14072ba0 msgr2=0x7feb14074cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:04:40.943 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:40.946+0000 7feb19a00700 1 -- 172.21.15.18:0/2521113816 shutdown_connections 2024-04-12T21:04:40.943 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:40.946+0000 7feb19a00700 1 -- 172.21.15.18:0/2521113816 wait complete. 2024-04-12T21:04:41.100 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:04:40 smithi018 ceph-mon[112590]: from='client.34280 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:04:41.100 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:04:40 smithi018 ceph-mon[112590]: pgmap v122: 65 pgs: 65 active+clean; 257 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail 2024-04-12T21:04:41.100 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:04:40 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' 2024-04-12T21:04:41.100 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:04:40 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T21:04:41.101 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:04:40 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' 2024-04-12T21:04:41.101 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:04:40 smithi018 ceph-mon[112590]: from='client.34284 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:04:41.101 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:04:40 smithi018 ceph-mon[112590]: from='client.34288 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:04:41.101 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:04:40 smithi018 ceph-mon[112590]: osd.2 marked itself down and dead 2024-04-12T21:04:41.101 INFO:journalctl@ceph.osd.2.smithi018.stdout:Apr 12 21:04:40 smithi018 ceph-1cbe407a-f90d-11ee-bc8d-c7b262605968-osd-2[62168]: 2024-04-12T21:04:40.794+0000 7fd7e2600700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-12T21:04:41.101 INFO:journalctl@ceph.osd.2.smithi018.stdout:Apr 12 21:04:40 smithi018 ceph-1cbe407a-f90d-11ee-bc8d-c7b262605968-osd-2[62168]: 2024-04-12T21:04:40.794+0000 7fd7e2600700 -1 osd.2 78 *** Got signal Terminated *** 2024-04-12T21:04:41.101 INFO:journalctl@ceph.osd.2.smithi018.stdout:Apr 12 21:04:40 smithi018 ceph-1cbe407a-f90d-11ee-bc8d-c7b262605968-osd-2[62168]: 2024-04-12T21:04:40.794+0000 7fd7e2600700 -1 osd.2 78 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-12T21:04:41.103 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:41.107+0000 7f9245e00700 1 -- 172.21.15.18:0/2846631960 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9240109430 msgr2=0x7f9240109800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:04:41.103 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:41.107+0000 7f9245e00700 1 --2- 172.21.15.18:0/2846631960 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9240109430 0x7f9240109800 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f9230004940 tx=0x7f9230009e30 comp rx=0 tx=0).stop 2024-04-12T21:04:41.103 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:41.107+0000 7f9245e00700 1 -- 172.21.15.18:0/2846631960 shutdown_connections 2024-04-12T21:04:41.103 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:41.107+0000 7f9245e00700 1 --2- 172.21.15.18:0/2846631960 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9240109dd0 0x7f9240103440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:04:41.104 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:41.107+0000 7f9245e00700 1 --2- 172.21.15.18:0/2846631960 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9240109430 0x7f9240109800 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:04:41.104 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:41.107+0000 7f9245e00700 1 -- 172.21.15.18:0/2846631960 >> 172.21.15.18:0/2846631960 conn(0x7f9240072c00 msgr2=0x7f9240075010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:04:41.104 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:41.107+0000 7f9245e00700 1 -- 172.21.15.18:0/2846631960 shutdown_connections 2024-04-12T21:04:41.104 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:41.107+0000 7f9245e00700 1 -- 172.21.15.18:0/2846631960 wait complete. 2024-04-12T21:04:41.104 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:41.108+0000 7f9245e00700 1 Processor -- start 2024-04-12T21:04:41.104 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:41.109+0000 7f9245e00700 1 -- start start 2024-04-12T21:04:41.106 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:41.109+0000 7f9245e00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9240109430 0x7f9240119f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:04:41.106 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:41.109+0000 7f9245e00700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9240109dd0 0x7f924011a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:04:41.106 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:41.109+0000 7f9245e00700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f924011ab60 con 0x7f9240109430 2024-04-12T21:04:41.106 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:41.109+0000 7f9245e00700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92401140b0 con 0x7f9240109dd0 2024-04-12T21:04:41.106 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:41.109+0000 7f923ea00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9240109430 0x7f9240119f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:04:41.106 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:41.109+0000 7f923ea00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9240109430 0x7f9240119f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:49244/0 (socket says 172.21.15.18:49244) 2024-04-12T21:04:41.106 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:41.109+0000 7f923ea00700 1 -- 172.21.15.18:0/3855309318 learned_addr learned my addr 172.21.15.18:0/3855309318 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:04:41.106 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:41.110+0000 7f9237e00700 1 --2- 172.21.15.18:0/3855309318 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9240109dd0 0x7f924011a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:04:41.107 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:41.110+0000 7f923ea00700 1 -- 172.21.15.18:0/3855309318 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9240109dd0 msgr2=0x7f924011a480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:04:41.107 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:41.110+0000 7f923ea00700 1 --2- 172.21.15.18:0/3855309318 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9240109dd0 0x7f924011a480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:04:41.107 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:41.110+0000 7f923ea00700 1 -- 172.21.15.18:0/3855309318 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9230005040 con 0x7f9240109430 2024-04-12T21:04:41.107 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:41.110+0000 7f923ea00700 1 --2- 172.21.15.18:0/3855309318 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9240109430 0x7f9240119f40 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f9230009180 tx=0x7f9230009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:04:41.107 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:41.111+0000 7f9236a00700 1 -- 172.21.15.18:0/3855309318 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f923000e040 con 0x7f9240109430 2024-04-12T21:04:41.107 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:41.111+0000 7f9236a00700 1 -- 172.21.15.18:0/3855309318 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f923000bbc0 con 0x7f9240109430 2024-04-12T21:04:41.108 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:41.111+0000 7f9236a00700 1 -- 172.21.15.18:0/3855309318 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f923001c630 con 0x7f9240109430 2024-04-12T21:04:41.108 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:41.111+0000 7f9245e00700 1 -- 172.21.15.18:0/3855309318 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f92401142d0 con 0x7f9240109430 2024-04-12T21:04:41.108 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:41.111+0000 7f9245e00700 1 -- 172.21.15.18:0/3855309318 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9240114890 con 0x7f9240109430 2024-04-12T21:04:41.108 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:41.112+0000 7f9245e00700 1 -- 172.21.15.18:0/3855309318 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9240104b80 con 0x7f9240109430 2024-04-12T21:04:41.109 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:41.113+0000 7f9236a00700 1 -- 172.21.15.18:0/3855309318 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7f923000bd30 con 0x7f9240109430 2024-04-12T21:04:41.110 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:41.114+0000 7f9236a00700 1 --2- 172.21.15.18:0/3855309318 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f922406ec00 0x7f92240710b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:04:41.110 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:41.114+0000 7f9236a00700 1 -- 172.21.15.18:0/3855309318 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 6585+0+0 (secure 0 0 0) 0x7f923008fbd0 con 0x7f9240109430 2024-04-12T21:04:41.110 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:41.114+0000 7f9237e00700 1 --2- 172.21.15.18:0/3855309318 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f922406ec00 0x7f92240710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:04:41.111 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:41.115+0000 7f9237e00700 1 --2- 172.21.15.18:0/3855309318 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f922406ec00 0x7f92240710b0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f9228006fd0 tx=0x7f9228008040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:04:41.115 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:41.119+0000 7f9236a00700 1 -- 172.21.15.18:0/3855309318 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f923005c570 con 0x7f9240109430 2024-04-12T21:04:41.280 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:04:40 smithi186 ceph-mon[86379]: from='client.34280 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:04:41.280 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:04:40 smithi186 ceph-mon[86379]: pgmap v122: 65 pgs: 65 active+clean; 257 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail 2024-04-12T21:04:41.280 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:04:40 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' 2024-04-12T21:04:41.280 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:04:40 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T21:04:41.280 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:04:40 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' 2024-04-12T21:04:41.280 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:04:40 smithi186 ceph-mon[86379]: from='client.34284 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:04:41.280 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:04:40 smithi186 ceph-mon[86379]: from='client.34288 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:04:41.280 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:04:40 smithi186 ceph-mon[86379]: osd.2 marked itself down and dead 2024-04-12T21:04:41.395 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:41.399+0000 7f9245e00700 1 -- 172.21.15.18:0/3855309318 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f92401155c0 con 0x7f9240109430 2024-04-12T21:04:41.396 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:41.400+0000 7f9236a00700 1 -- 172.21.15.18:0/3855309318 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 23 v23) v1 ==== 76+0+1886 (secure 0 0 0) 0x7f923005c100 con 0x7f9240109430 2024-04-12T21:04:41.396 INFO:teuthology.orchestra.run.smithi018.stdout:e23 2024-04-12T21:04:41.396 INFO:teuthology.orchestra.run.smithi018.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-12T21:04:41.396 INFO:teuthology.orchestra.run.smithi018.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-12T21:04:41.397 INFO:teuthology.orchestra.run.smithi018.stdout:legacy client fscid: 1 2024-04-12T21:04:41.397 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-12T21:04:41.397 INFO:teuthology.orchestra.run.smithi018.stdout:Filesystem 'cephfs' (1) 2024-04-12T21:04:41.397 INFO:teuthology.orchestra.run.smithi018.stdout:fs_name cephfs 2024-04-12T21:04:41.397 INFO:teuthology.orchestra.run.smithi018.stdout:epoch 23 2024-04-12T21:04:41.397 INFO:teuthology.orchestra.run.smithi018.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-04-12T21:04:41.397 INFO:teuthology.orchestra.run.smithi018.stdout:created 2024-04-12T20:51:57.362850+0000 2024-04-12T21:04:41.398 INFO:teuthology.orchestra.run.smithi018.stdout:modified 2024-04-12T21:01:10.670287+0000 2024-04-12T21:04:41.398 INFO:teuthology.orchestra.run.smithi018.stdout:tableserver 0 2024-04-12T21:04:41.398 INFO:teuthology.orchestra.run.smithi018.stdout:root 0 2024-04-12T21:04:41.398 INFO:teuthology.orchestra.run.smithi018.stdout:session_timeout 60 2024-04-12T21:04:41.398 INFO:teuthology.orchestra.run.smithi018.stdout:session_autoclose 300 2024-04-12T21:04:41.398 INFO:teuthology.orchestra.run.smithi018.stdout:max_file_size 1099511627776 2024-04-12T21:04:41.398 INFO:teuthology.orchestra.run.smithi018.stdout:max_xattr_size 65536 2024-04-12T21:04:41.398 INFO:teuthology.orchestra.run.smithi018.stdout:required_client_features {} 2024-04-12T21:04:41.398 INFO:teuthology.orchestra.run.smithi018.stdout:last_failure 0 2024-04-12T21:04:41.399 INFO:teuthology.orchestra.run.smithi018.stdout:last_failure_osd_epoch 46 2024-04-12T21:04:41.399 INFO:teuthology.orchestra.run.smithi018.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-12T21:04:41.399 INFO:teuthology.orchestra.run.smithi018.stdout:max_mds 2 2024-04-12T21:04:41.399 INFO:teuthology.orchestra.run.smithi018.stdout:in 0,1 2024-04-12T21:04:41.399 INFO:teuthology.orchestra.run.smithi018.stdout:up {0=14452,1=14464} 2024-04-12T21:04:41.399 INFO:teuthology.orchestra.run.smithi018.stdout:failed 2024-04-12T21:04:41.400 INFO:teuthology.orchestra.run.smithi018.stdout:damaged 2024-04-12T21:04:41.400 INFO:teuthology.orchestra.run.smithi018.stdout:stopped 2024-04-12T21:04:41.400 INFO:teuthology.orchestra.run.smithi018.stdout:data_pools [3] 2024-04-12T21:04:41.401 INFO:teuthology.orchestra.run.smithi018.stdout:metadata_pool 2 2024-04-12T21:04:41.401 INFO:teuthology.orchestra.run.smithi018.stdout:inline_data disabled 2024-04-12T21:04:41.401 INFO:teuthology.orchestra.run.smithi018.stdout:balancer 2024-04-12T21:04:41.401 INFO:teuthology.orchestra.run.smithi018.stdout:bal_rank_mask -1 2024-04-12T21:04:41.401 INFO:teuthology.orchestra.run.smithi018.stdout:standby_count_wanted 1 2024-04-12T21:04:41.401 INFO:teuthology.orchestra.run.smithi018.stdout:qdb_cluster leader: 0 members: 2024-04-12T21:04:41.401 INFO:teuthology.orchestra.run.smithi018.stdout:[mds.cephfs.smithi018.acqbsc{0:14452} state up:active seq 5 join_fscid=1 addr [v2:172.21.15.18:6826/1564065434,v1:172.21.15.18:6827/1564065434] compat {c=[1],r=[1],i=[7ff]}] 2024-04-12T21:04:41.401 INFO:teuthology.orchestra.run.smithi018.stdout:[mds.cephfs.smithi018.szjluo{1:14464} state up:active seq 6 join_fscid=1 addr [v2:172.21.15.18:6828/722443353,v1:172.21.15.18:6829/722443353] compat {c=[1],r=[1],i=[7ff]}] 2024-04-12T21:04:41.401 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-12T21:04:41.401 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-12T21:04:41.401 INFO:teuthology.orchestra.run.smithi018.stdout:Standby daemons: 2024-04-12T21:04:41.401 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-12T21:04:41.401 INFO:teuthology.orchestra.run.smithi018.stdout:[mds.cephfs.smithi186.wjnafs{-1:24277} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.186:6826/438347207,v1:172.21.15.186:6827/438347207] compat {c=[1],r=[1],i=[7ff]}] 2024-04-12T21:04:41.402 INFO:teuthology.orchestra.run.smithi018.stdout:[mds.cephfs.smithi186.jdllyn{-1:24281} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.186:6824/1464359952,v1:172.21.15.186:6825/1464359952] compat {c=[1],r=[1],i=[7ff]}] 2024-04-12T21:04:41.402 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:41.403+0000 7f9245e00700 1 -- 172.21.15.18:0/3855309318 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f922406ec00 msgr2=0x7f92240710b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:04:41.403 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:41.403+0000 7f9245e00700 1 --2- 172.21.15.18:0/3855309318 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f922406ec00 0x7f92240710b0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f9228006fd0 tx=0x7f9228008040 comp rx=0 tx=0).stop 2024-04-12T21:04:41.403 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:41.403+0000 7f9245e00700 1 -- 172.21.15.18:0/3855309318 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9240109430 msgr2=0x7f9240119f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:04:41.403 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:41.403+0000 7f9245e00700 1 --2- 172.21.15.18:0/3855309318 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9240109430 0x7f9240119f40 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f9230009180 tx=0x7f9230009e30 comp rx=0 tx=0).stop 2024-04-12T21:04:41.403 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:41.403+0000 7f9245e00700 1 -- 172.21.15.18:0/3855309318 shutdown_connections 2024-04-12T21:04:41.403 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:41.403+0000 7f9245e00700 1 --2- 172.21.15.18:0/3855309318 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f922406ec00 0x7f92240710b0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:04:41.403 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:41.403+0000 7f9245e00700 1 --2- 172.21.15.18:0/3855309318 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9240109430 0x7f9240119f40 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:04:41.403 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:41.403+0000 7f9245e00700 1 --2- 172.21.15.18:0/3855309318 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9240109dd0 0x7f924011a480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:04:41.403 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:41.403+0000 7f9245e00700 1 -- 172.21.15.18:0/3855309318 >> 172.21.15.18:0/3855309318 conn(0x7f9240072c00 msgr2=0x7f9240074c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:04:41.403 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:41.403+0000 7f9245e00700 1 -- 172.21.15.18:0/3855309318 shutdown_connections 2024-04-12T21:04:41.403 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:41.403+0000 7f9245e00700 1 -- 172.21.15.18:0/3855309318 wait complete. 2024-04-12T21:04:41.403 INFO:teuthology.orchestra.run.smithi018.stderr:dumped fsmap epoch 23 2024-04-12T21:04:41.569 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:41.573+0000 7f620b400700 1 -- 172.21.15.18:0/1152192055 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6204105020 msgr2=0x7f62041053f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:04:41.569 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:41.573+0000 7f620b400700 1 --2- 172.21.15.18:0/1152192055 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6204105020 0x7f62041053f0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f61f4004940 tx=0x7f61f4009e30 comp rx=0 tx=0).stop 2024-04-12T21:04:41.569 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:41.573+0000 7f620b400700 1 -- 172.21.15.18:0/1152192055 shutdown_connections 2024-04-12T21:04:41.569 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:41.573+0000 7f620b400700 1 --2- 172.21.15.18:0/1152192055 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f62041059c0 0x7f62040ff080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:04:41.570 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:41.573+0000 7f620b400700 1 --2- 172.21.15.18:0/1152192055 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6204105020 0x7f62041053f0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:04:41.570 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:41.573+0000 7f620b400700 1 -- 172.21.15.18:0/1152192055 >> 172.21.15.18:0/1152192055 conn(0x7f62040faba0 msgr2=0x7f62040fcfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:04:41.570 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:41.573+0000 7f620b400700 1 -- 172.21.15.18:0/1152192055 shutdown_connections 2024-04-12T21:04:41.570 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:41.573+0000 7f620b400700 1 -- 172.21.15.18:0/1152192055 wait complete. 2024-04-12T21:04:41.571 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:41.574+0000 7f620b400700 1 Processor -- start 2024-04-12T21:04:41.571 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:41.574+0000 7f620b400700 1 -- start start 2024-04-12T21:04:41.571 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:41.575+0000 7f620b400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6204105020 0x7f6204115bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:04:41.571 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:41.575+0000 7f620b400700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f62041059c0 0x7f6204116130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:04:41.571 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:41.575+0000 7f620b400700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6204116810 con 0x7f6204105020 2024-04-12T21:04:41.571 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:41.575+0000 7f620b400700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f620410fd60 con 0x7f62041059c0 2024-04-12T21:04:41.571 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:41.575+0000 7f6208a00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6204105020 0x7f6204115bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:04:41.571 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:41.575+0000 7f6208a00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6204105020 0x7f6204115bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:49264/0 (socket says 172.21.15.18:49264) 2024-04-12T21:04:41.572 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:41.575+0000 7f6208a00700 1 -- 172.21.15.18:0/795337617 learned_addr learned my addr 172.21.15.18:0/795337617 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:04:41.573 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:41.575+0000 7f61fbe00700 1 --2- 172.21.15.18:0/795337617 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f62041059c0 0x7f6204116130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:04:41.573 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:41.575+0000 7f6208a00700 1 -- 172.21.15.18:0/795337617 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f62041059c0 msgr2=0x7f6204116130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:04:41.573 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:41.575+0000 7f6208a00700 1 --2- 172.21.15.18:0/795337617 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f62041059c0 0x7f6204116130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:04:41.573 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:41.575+0000 7f6208a00700 1 -- 172.21.15.18:0/795337617 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61f4005040 con 0x7f6204105020 2024-04-12T21:04:41.573 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:41.575+0000 7f61fbe00700 1 --2- 172.21.15.18:0/795337617 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f62041059c0 0x7f6204116130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-12T21:04:41.573 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:41.576+0000 7f6208a00700 1 --2- 172.21.15.18:0/795337617 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6204105020 0x7f6204115bf0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f61f4009150 tx=0x7f61f4009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:04:41.573 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:41.576+0000 7f6201600700 1 -- 172.21.15.18:0/795337617 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f61f400e040 con 0x7f6204105020 2024-04-12T21:04:41.573 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:41.576+0000 7f620b400700 1 -- 172.21.15.18:0/795337617 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f620410ff80 con 0x7f6204105020 2024-04-12T21:04:41.573 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:41.576+0000 7f6201600700 1 -- 172.21.15.18:0/795337617 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f61f4013420 con 0x7f6204105020 2024-04-12T21:04:41.573 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:41.576+0000 7f620b400700 1 -- 172.21.15.18:0/795337617 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f62041104c0 con 0x7f6204105020 2024-04-12T21:04:41.574 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:41.577+0000 7f6201600700 1 -- 172.21.15.18:0/795337617 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f61f40255e0 con 0x7f6204105020 2024-04-12T21:04:41.574 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:41.577+0000 7f620b400700 1 -- 172.21.15.18:0/795337617 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6204100770 con 0x7f6204105020 2024-04-12T21:04:41.575 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:41.579+0000 7f6201600700 1 -- 172.21.15.18:0/795337617 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7f61f4013590 con 0x7f6204105020 2024-04-12T21:04:41.576 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:41.580+0000 7f6201600700 1 --2- 172.21.15.18:0/795337617 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f61e806ec00 0x7f61e80710b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:04:41.576 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:41.580+0000 7f6201600700 1 -- 172.21.15.18:0/795337617 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 6585+0+0 (secure 0 0 0) 0x7f61f408fbc0 con 0x7f6204105020 2024-04-12T21:04:41.576 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:41.580+0000 7f61fbe00700 1 --2- 172.21.15.18:0/795337617 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f61e806ec00 0x7f61e80710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:04:41.576 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:41.581+0000 7f61fbe00700 1 --2- 172.21.15.18:0/795337617 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f61e806ec00 0x7f61e80710b0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f61ec006fd0 tx=0x7f61ec008040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:04:41.580 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:41.585+0000 7f6201600700 1 -- 172.21.15.18:0/795337617 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f61f405c420 con 0x7f6204105020 2024-04-12T21:04:41.839 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:41.842+0000 7f620b400700 1 -- 172.21.15.18:0/795337617 --> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f6204111310 con 0x7f61e806ec00 2024-04-12T21:04:41.840 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:41.844+0000 7f6201600700 1 -- 172.21.15.18:0/795337617 <== mgr.34102 v2:172.21.15.18:6800/3812891919 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7f61ec00c660 con 0x7f61e806ec00 2024-04-12T21:04:41.840 INFO:teuthology.orchestra.run.smithi018.stdout:{ 2024-04-12T21:04:41.840 INFO:teuthology.orchestra.run.smithi018.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:f4452def3be4d350f4a8e5feaf7ebb1511be9298fe68478ba7a2d6fbd7d69a63", 2024-04-12T21:04:41.840 INFO:teuthology.orchestra.run.smithi018.stdout: "in_progress": true, 2024-04-12T21:04:41.840 INFO:teuthology.orchestra.run.smithi018.stdout: "which": "Upgrading all daemon types on all hosts", 2024-04-12T21:04:41.840 INFO:teuthology.orchestra.run.smithi018.stdout: "services_complete": [ 2024-04-12T21:04:41.841 INFO:teuthology.orchestra.run.smithi018.stdout: "crash", 2024-04-12T21:04:41.841 INFO:teuthology.orchestra.run.smithi018.stdout: "mon", 2024-04-12T21:04:41.841 INFO:teuthology.orchestra.run.smithi018.stdout: "mgr" 2024-04-12T21:04:41.841 INFO:teuthology.orchestra.run.smithi018.stdout: ], 2024-04-12T21:04:41.841 INFO:teuthology.orchestra.run.smithi018.stdout: "progress": "8/21 daemons upgraded", 2024-04-12T21:04:41.841 INFO:teuthology.orchestra.run.smithi018.stdout: "message": "Currently upgrading osd daemons", 2024-04-12T21:04:41.841 INFO:teuthology.orchestra.run.smithi018.stdout: "is_paused": false 2024-04-12T21:04:41.841 INFO:teuthology.orchestra.run.smithi018.stdout:} 2024-04-12T21:04:41.842 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:41.846+0000 7f620b400700 1 -- 172.21.15.18:0/795337617 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f61e806ec00 msgr2=0x7f61e80710b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:04:41.842 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:41.846+0000 7f620b400700 1 --2- 172.21.15.18:0/795337617 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f61e806ec00 0x7f61e80710b0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f61ec006fd0 tx=0x7f61ec008040 comp rx=0 tx=0).stop 2024-04-12T21:04:41.843 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:41.846+0000 7f620b400700 1 -- 172.21.15.18:0/795337617 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6204105020 msgr2=0x7f6204115bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:04:41.843 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:41.846+0000 7f620b400700 1 --2- 172.21.15.18:0/795337617 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6204105020 0x7f6204115bf0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f61f4009150 tx=0x7f61f4009e30 comp rx=0 tx=0).stop 2024-04-12T21:04:41.843 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:41.847+0000 7f620b400700 1 -- 172.21.15.18:0/795337617 shutdown_connections 2024-04-12T21:04:41.843 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:41.847+0000 7f620b400700 1 --2- 172.21.15.18:0/795337617 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f61e806ec00 0x7f61e80710b0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:04:41.843 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:41.847+0000 7f620b400700 1 --2- 172.21.15.18:0/795337617 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6204105020 0x7f6204115bf0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:04:41.843 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:41.847+0000 7f620b400700 1 --2- 172.21.15.18:0/795337617 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f62041059c0 0x7f6204116130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:04:41.843 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:41.847+0000 7f620b400700 1 -- 172.21.15.18:0/795337617 >> 172.21.15.18:0/795337617 conn(0x7f62040faba0 msgr2=0x7f62040fbb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:04:41.844 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:41.847+0000 7f620b400700 1 -- 172.21.15.18:0/795337617 shutdown_connections 2024-04-12T21:04:41.844 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:41.847+0000 7f620b400700 1 -- 172.21.15.18:0/795337617 wait complete. 2024-04-12T21:04:42.006 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:42.010+0000 7fdf53e00700 1 -- 172.21.15.18:0/108573407 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fdf541092a0 msgr2=0x7fdf54109670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:04:42.006 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:42.010+0000 7fdf53e00700 1 --2- 172.21.15.18:0/108573407 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fdf541092a0 0x7fdf54109670 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fdf44004940 tx=0x7fdf44009e30 comp rx=0 tx=0).stop 2024-04-12T21:04:42.007 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:42.010+0000 7fdf53e00700 1 -- 172.21.15.18:0/108573407 shutdown_connections 2024-04-12T21:04:42.007 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:42.010+0000 7fdf53e00700 1 --2- 172.21.15.18:0/108573407 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdf54109c40 0x7fdf541032b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:04:42.007 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:42.010+0000 7fdf53e00700 1 --2- 172.21.15.18:0/108573407 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fdf541092a0 0x7fdf54109670 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:04:42.007 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:42.010+0000 7fdf53e00700 1 -- 172.21.15.18:0/108573407 >> 172.21.15.18:0/108573407 conn(0x7fdf54072a30 msgr2=0x7fdf54074e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:04:42.007 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:42.010+0000 7fdf53e00700 1 -- 172.21.15.18:0/108573407 shutdown_connections 2024-04-12T21:04:42.007 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:42.010+0000 7fdf53e00700 1 -- 172.21.15.18:0/108573407 wait complete. 2024-04-12T21:04:42.008 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:42.011+0000 7fdf53e00700 1 Processor -- start 2024-04-12T21:04:42.008 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:42.011+0000 7fdf53e00700 1 -- start start 2024-04-12T21:04:42.008 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:42.011+0000 7fdf53e00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fdf541092a0 0x7fdf54119f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:04:42.008 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:42.012+0000 7fdf53e00700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdf54109c40 0x7fdf5411a4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:04:42.008 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:42.012+0000 7fdf53e00700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf5411aba0 con 0x7fdf541092a0 2024-04-12T21:04:42.008 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:42.012+0000 7fdf53e00700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf54113ee0 con 0x7fdf54109c40 2024-04-12T21:04:42.008 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:42.012+0000 7fdf52a00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fdf541092a0 0x7fdf54119f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:04:42.008 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:42.012+0000 7fdf52a00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fdf541092a0 0x7fdf54119f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:49284/0 (socket says 172.21.15.18:49284) 2024-04-12T21:04:42.008 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:42.012+0000 7fdf52a00700 1 -- 172.21.15.18:0/3483206152 learned_addr learned my addr 172.21.15.18:0/3483206152 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:04:42.009 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:42.012+0000 7fdf52000700 1 --2- 172.21.15.18:0/3483206152 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdf54109c40 0x7fdf5411a4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:04:42.009 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:42.012+0000 7fdf52a00700 1 -- 172.21.15.18:0/3483206152 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdf54109c40 msgr2=0x7fdf5411a4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:04:42.009 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:42.013+0000 7fdf52a00700 1 --2- 172.21.15.18:0/3483206152 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdf54109c40 0x7fdf5411a4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:04:42.009 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:42.013+0000 7fdf52a00700 1 -- 172.21.15.18:0/3483206152 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf44005040 con 0x7fdf541092a0 2024-04-12T21:04:42.010 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:42.013+0000 7fdf52000700 1 --2- 172.21.15.18:0/3483206152 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdf54109c40 0x7fdf5411a4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-12T21:04:42.010 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:42.013+0000 7fdf52a00700 1 --2- 172.21.15.18:0/3483206152 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fdf541092a0 0x7fdf54119f80 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fdf44009180 tx=0x7fdf44009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:04:42.010 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:42.013+0000 7fdf4aa00700 1 -- 172.21.15.18:0/3483206152 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdf4400e040 con 0x7fdf541092a0 2024-04-12T21:04:42.010 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:42.013+0000 7fdf4aa00700 1 -- 172.21.15.18:0/3483206152 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fdf4400bbc0 con 0x7fdf541092a0 2024-04-12T21:04:42.010 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:42.013+0000 7fdf53e00700 1 -- 172.21.15.18:0/3483206152 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdf54114100 con 0x7fdf541092a0 2024-04-12T21:04:42.010 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:42.013+0000 7fdf4aa00700 1 -- 172.21.15.18:0/3483206152 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdf4401c630 con 0x7fdf541092a0 2024-04-12T21:04:42.010 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:42.014+0000 7fdf53e00700 1 -- 172.21.15.18:0/3483206152 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdf541146c0 con 0x7fdf541092a0 2024-04-12T21:04:42.011 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:42.015+0000 7fdf53e00700 1 -- 172.21.15.18:0/3483206152 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdf541049f0 con 0x7fdf541092a0 2024-04-12T21:04:42.012 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:42.016+0000 7fdf4aa00700 1 -- 172.21.15.18:0/3483206152 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7fdf4400bd30 con 0x7fdf541092a0 2024-04-12T21:04:42.013 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:42.017+0000 7fdf4aa00700 1 --2- 172.21.15.18:0/3483206152 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fdf3806ec00 0x7fdf380710b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:04:42.013 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:42.017+0000 7fdf4aa00700 1 -- 172.21.15.18:0/3483206152 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 6585+0+0 (secure 0 0 0) 0x7fdf4408fcc0 con 0x7fdf541092a0 2024-04-12T21:04:42.013 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:42.018+0000 7fdf52000700 1 --2- 172.21.15.18:0/3483206152 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fdf3806ec00 0x7fdf380710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:04:42.014 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:42.018+0000 7fdf52000700 1 --2- 172.21.15.18:0/3483206152 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fdf3806ec00 0x7fdf380710b0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fdf40006fd0 tx=0x7fdf40008040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:04:42.021 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:42.026+0000 7fdf4aa00700 1 -- 172.21.15.18:0/3483206152 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7fdf4405c5c0 con 0x7fdf541092a0 2024-04-12T21:04:42.100 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:04:41 smithi018 ceph-mon[112590]: from='client.? 172.21.15.18:0/2521113816' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T21:04:42.100 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:04:41 smithi018 ceph-mon[112590]: from='client.? 172.21.15.18:0/3855309318' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-12T21:04:42.100 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:04:41 smithi018 ceph-mon[112590]: Health check failed: 1 osds down (OSD_DOWN) 2024-04-12T21:04:42.100 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:04:41 smithi018 ceph-mon[112590]: osdmap e79: 6 total, 5 up, 6 in 2024-04-12T21:04:42.279 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:04:41 smithi186 ceph-mon[86379]: from='client.? 172.21.15.18:0/2521113816' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T21:04:42.280 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:04:41 smithi186 ceph-mon[86379]: from='client.? 172.21.15.18:0/3855309318' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-12T21:04:42.280 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:04:41 smithi186 ceph-mon[86379]: Health check failed: 1 osds down (OSD_DOWN) 2024-04-12T21:04:42.280 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:04:41 smithi186 ceph-mon[86379]: osdmap e79: 6 total, 5 up, 6 in 2024-04-12T21:04:42.365 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:42.369+0000 7fdf53e00700 1 -- 172.21.15.18:0/3483206152 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7fdf54076560 con 0x7fdf541092a0 2024-04-12T21:04:42.366 INFO:journalctl@ceph.osd.2.smithi018.stdout:Apr 12 21:04:42 smithi018 podman[134866]: 2024-04-12 21:04:42.107333426 +0000 UTC m=+1.446882317 container died 8a83692acaa52eb40e54a1cb2ceed1055a6ca6320dc4cf0cc9f7848a316b116d (image=quay.ceph.io/ceph-ci/ceph@sha256:5e334df698f432180b78bc9c46936bfdfbc92bc9a8983463e523f780d23765fe, name=ceph-1cbe407a-f90d-11ee-bc8d-c7b262605968-osd-2, GIT_CLEAN=True, org.label-schema.build-date=20240409, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, RELEASE=quincy-d3b5060, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-12T21:04:42.366 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:42.369+0000 7fdf4aa00700 1 -- 172.21.15.18:0/3483206152 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+100 (secure 0 0 0) 0x7fdf54076560 con 0x7fdf541092a0 2024-04-12T21:04:42.366 INFO:teuthology.orchestra.run.smithi018.stdout:HEALTH_WARN 1 osds down 2024-04-12T21:04:42.366 INFO:teuthology.orchestra.run.smithi018.stdout:[WRN] OSD_DOWN: 1 osds down 2024-04-12T21:04:42.367 INFO:teuthology.orchestra.run.smithi018.stdout: osd.2 (root=default,host=smithi018) is down 2024-04-12T21:04:42.370 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:42.373+0000 7fdf53e00700 1 -- 172.21.15.18:0/3483206152 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fdf3806ec00 msgr2=0x7fdf380710b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:04:42.370 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:42.373+0000 7fdf53e00700 1 --2- 172.21.15.18:0/3483206152 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fdf3806ec00 0x7fdf380710b0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fdf40006fd0 tx=0x7fdf40008040 comp rx=0 tx=0).stop 2024-04-12T21:04:42.370 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:42.373+0000 7fdf53e00700 1 -- 172.21.15.18:0/3483206152 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fdf541092a0 msgr2=0x7fdf54119f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:04:42.370 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:42.373+0000 7fdf53e00700 1 --2- 172.21.15.18:0/3483206152 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fdf541092a0 0x7fdf54119f80 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fdf44009180 tx=0x7fdf44009e30 comp rx=0 tx=0).stop 2024-04-12T21:04:42.370 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:42.374+0000 7fdf53e00700 1 -- 172.21.15.18:0/3483206152 shutdown_connections 2024-04-12T21:04:42.370 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:42.374+0000 7fdf53e00700 1 --2- 172.21.15.18:0/3483206152 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fdf3806ec00 0x7fdf380710b0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:04:42.370 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:42.374+0000 7fdf53e00700 1 --2- 172.21.15.18:0/3483206152 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fdf541092a0 0x7fdf54119f80 secure :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fdf44009180 tx=0x7fdf44009e30 comp rx=0 tx=0).stop 2024-04-12T21:04:42.370 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:42.374+0000 7fdf53e00700 1 --2- 172.21.15.18:0/3483206152 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdf54109c40 0x7fdf5411a4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:04:42.370 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:42.374+0000 7fdf53e00700 1 -- 172.21.15.18:0/3483206152 >> 172.21.15.18:0/3483206152 conn(0x7fdf54072a30 msgr2=0x7fdf540748f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:04:42.371 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:42.374+0000 7fdf53e00700 1 -- 172.21.15.18:0/3483206152 shutdown_connections 2024-04-12T21:04:42.371 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:04:42.375+0000 7fdf53e00700 1 -- 172.21.15.18:0/3483206152 wait complete. 2024-04-12T21:04:43.101 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:04:42 smithi018 ceph-mon[112590]: pgmap v124: 65 pgs: 12 stale+active+clean, 53 active+clean; 257 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail 2024-04-12T21:04:43.101 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:04:42 smithi018 ceph-mon[112590]: from='client.34300 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:04:43.101 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:04:42 smithi018 ceph-mon[112590]: from='client.? 172.21.15.18:0/3483206152' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-12T21:04:43.101 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:04:42 smithi018 ceph-mon[112590]: osdmap e80: 6 total, 5 up, 6 in 2024-04-12T21:04:43.101 INFO:journalctl@ceph.osd.2.smithi018.stdout:Apr 12 21:04:42 smithi018 podman[134866]: 2024-04-12 21:04:42.704930609 +0000 UTC m=+2.044479543 container cleanup 8a83692acaa52eb40e54a1cb2ceed1055a6ca6320dc4cf0cc9f7848a316b116d (image=quay.ceph.io/ceph-ci/ceph@sha256:5e334df698f432180b78bc9c46936bfdfbc92bc9a8983463e523f780d23765fe, name=ceph-1cbe407a-f90d-11ee-bc8d-c7b262605968-osd-2, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240409, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, ceph=True, RELEASE=quincy-d3b5060, io.buildah.version=1.33.5) 2024-04-12T21:04:43.102 INFO:journalctl@ceph.osd.2.smithi018.stdout:Apr 12 21:04:42 smithi018 bash[134866]: ceph-1cbe407a-f90d-11ee-bc8d-c7b262605968-osd-2 2024-04-12T21:04:43.102 INFO:journalctl@ceph.osd.2.smithi018.stdout:Apr 12 21:04:42 smithi018 podman[134955]: 2024-04-12 21:04:42.914948154 +0000 UTC m=+0.795428089 container remove 8a83692acaa52eb40e54a1cb2ceed1055a6ca6320dc4cf0cc9f7848a316b116d (image=quay.ceph.io/ceph-ci/ceph@sha256:5e334df698f432180b78bc9c46936bfdfbc92bc9a8983463e523f780d23765fe, name=ceph-1cbe407a-f90d-11ee-bc8d-c7b262605968-osd-2, org.label-schema.build-date=20240409, org.label-schema.vendor=CentOS, RELEASE=quincy-d3b5060, ceph=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, org.label-schema.schema-version=1.0) 2024-04-12T21:04:43.279 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:04:42 smithi186 ceph-mon[86379]: pgmap v124: 65 pgs: 12 stale+active+clean, 53 active+clean; 257 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail 2024-04-12T21:04:43.280 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:04:42 smithi186 ceph-mon[86379]: from='client.34300 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:04:43.280 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:04:42 smithi186 ceph-mon[86379]: from='client.? 172.21.15.18:0/3483206152' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-12T21:04:43.280 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:04:42 smithi186 ceph-mon[86379]: osdmap e80: 6 total, 5 up, 6 in 2024-04-12T21:04:44.069 INFO:journalctl@ceph.osd.2.smithi018.stdout:Apr 12 21:04:43 smithi018 podman[135046]: 2024-04-12 21:04:43.693831614 +0000 UTC m=+0.035651795 image pull 8717720dc5e5c56303818ea4d4f0a11f6e2c03dee521fb668f411f1304e0afca quay.ceph.io/ceph-ci/ceph@sha256:f4452def3be4d350f4a8e5feaf7ebb1511be9298fe68478ba7a2d6fbd7d69a63 2024-04-12T21:04:44.069 INFO:journalctl@ceph.osd.2.smithi018.stdout:Apr 12 21:04:43 smithi018 podman[135046]: 2024-04-12 21:04:43.840906481 +0000 UTC m=+0.182726730 container create 4ce0bf85e65d4fa4294f926ae4c8c0b268fa60b4f103c53d1e06053e3f9ed777 (image=quay.ceph.io/ceph-ci/ceph@sha256:f4452def3be4d350f4a8e5feaf7ebb1511be9298fe68478ba7a2d6fbd7d69a63, name=ceph-1cbe407a-f90d-11ee-bc8d-c7b262605968-osd-2-deactivate, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, ceph=True, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, RELEASE=wip-vshankar-testing-20240411.061452-324baf8, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240409, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-12T21:04:44.351 INFO:journalctl@ceph.osd.2.smithi018.stdout:Apr 12 21:04:44 smithi018 podman[135046]: 2024-04-12 21:04:44.186203157 +0000 UTC m=+0.528023383 container init 4ce0bf85e65d4fa4294f926ae4c8c0b268fa60b4f103c53d1e06053e3f9ed777 (image=quay.ceph.io/ceph-ci/ceph@sha256:f4452def3be4d350f4a8e5feaf7ebb1511be9298fe68478ba7a2d6fbd7d69a63, name=ceph-1cbe407a-f90d-11ee-bc8d-c7b262605968-osd-2-deactivate, RELEASE=wip-vshankar-testing-20240411.061452-324baf8, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240409, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.license=GPLv2, ceph=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-12T21:04:44.351 INFO:journalctl@ceph.osd.2.smithi018.stdout:Apr 12 21:04:44 smithi018 podman[135046]: 2024-04-12 21:04:44.198110237 +0000 UTC m=+0.539930459 container start 4ce0bf85e65d4fa4294f926ae4c8c0b268fa60b4f103c53d1e06053e3f9ed777 (image=quay.ceph.io/ceph-ci/ceph@sha256:f4452def3be4d350f4a8e5feaf7ebb1511be9298fe68478ba7a2d6fbd7d69a63, name=ceph-1cbe407a-f90d-11ee-bc8d-c7b262605968-osd-2-deactivate, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, RELEASE=wip-vshankar-testing-20240411.061452-324baf8, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.build-date=20240409) 2024-04-12T21:04:44.351 INFO:journalctl@ceph.osd.2.smithi018.stdout:Apr 12 21:04:44 smithi018 podman[135046]: 2024-04-12 21:04:44.276605174 +0000 UTC m=+0.618425436 container attach 4ce0bf85e65d4fa4294f926ae4c8c0b268fa60b4f103c53d1e06053e3f9ed777 (image=quay.ceph.io/ceph-ci/ceph@sha256:f4452def3be4d350f4a8e5feaf7ebb1511be9298fe68478ba7a2d6fbd7d69a63, name=ceph-1cbe407a-f90d-11ee-bc8d-c7b262605968-osd-2-deactivate, io.buildah.version=1.33.5, RELEASE=wip-vshankar-testing-20240411.061452-324baf8, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240409, CEPH_POINT_RELEASE=, GIT_CLEAN=True) 2024-04-12T21:04:45.061 INFO:journalctl@ceph.osd.2.smithi018.stdout:Apr 12 21:04:44 smithi018 podman[135046]: 2024-04-12 21:04:44.666588044 +0000 UTC m=+1.008408300 container died 4ce0bf85e65d4fa4294f926ae4c8c0b268fa60b4f103c53d1e06053e3f9ed777 (image=quay.ceph.io/ceph-ci/ceph@sha256:f4452def3be4d350f4a8e5feaf7ebb1511be9298fe68478ba7a2d6fbd7d69a63, name=ceph-1cbe407a-f90d-11ee-bc8d-c7b262605968-osd-2-deactivate, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.build-date=20240409, CEPH_POINT_RELEASE=, RELEASE=wip-vshankar-testing-20240411.061452-324baf8, ceph=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-12T21:04:45.061 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:04:44 smithi018 ceph-mon[112590]: pgmap v126: 65 pgs: 12 stale+active+clean, 53 active+clean; 257 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail 2024-04-12T21:04:45.280 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:04:44 smithi186 ceph-mon[86379]: pgmap v126: 65 pgs: 12 stale+active+clean, 53 active+clean; 257 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail 2024-04-12T21:04:45.851 INFO:journalctl@ceph.osd.2.smithi018.stdout:Apr 12 21:04:45 smithi018 podman[135072]: 2024-04-12 21:04:45.44982655 +0000 UTC m=+0.763270186 container remove 4ce0bf85e65d4fa4294f926ae4c8c0b268fa60b4f103c53d1e06053e3f9ed777 (image=quay.ceph.io/ceph-ci/ceph@sha256:f4452def3be4d350f4a8e5feaf7ebb1511be9298fe68478ba7a2d6fbd7d69a63, name=ceph-1cbe407a-f90d-11ee-bc8d-c7b262605968-osd-2-deactivate, ceph=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240409, RELEASE=wip-vshankar-testing-20240411.061452-324baf8, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-12T21:04:45.851 INFO:journalctl@ceph.osd.2.smithi018.stdout:Apr 12 21:04:45 smithi018 systemd[1]: ceph-1cbe407a-f90d-11ee-bc8d-c7b262605968@osd.2.service: Deactivated successfully. 2024-04-12T21:04:45.851 INFO:journalctl@ceph.osd.2.smithi018.stdout:Apr 12 21:04:45 smithi018 systemd[1]: Stopped Ceph osd.2 for 1cbe407a-f90d-11ee-bc8d-c7b262605968. 2024-04-12T21:04:45.851 INFO:journalctl@ceph.osd.2.smithi018.stdout:Apr 12 21:04:45 smithi018 systemd[1]: ceph-1cbe407a-f90d-11ee-bc8d-c7b262605968@osd.2.service: Consumed 1min 41.569s CPU time. 2024-04-12T21:04:46.277 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:04:45 smithi018 ceph-mon[112590]: Health check failed: Degraded data redundancy: 35/324 objects degraded (10.802%), 10 pgs degraded (PG_DEGRADED) 2024-04-12T21:04:46.277 INFO:journalctl@ceph.osd.2.smithi018.stdout:Apr 12 21:04:45 smithi018 systemd[1]: Starting Ceph osd.2 for 1cbe407a-f90d-11ee-bc8d-c7b262605968... 2024-04-12T21:04:46.279 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:04:45 smithi186 ceph-mon[86379]: Health check failed: Degraded data redundancy: 35/324 objects degraded (10.802%), 10 pgs degraded (PG_DEGRADED) 2024-04-12T21:04:46.601 INFO:journalctl@ceph.osd.2.smithi018.stdout:Apr 12 21:04:46 smithi018 podman[135170]: 2024-04-12 21:04:46.189442377 +0000 UTC m=+0.043573455 image pull 8717720dc5e5c56303818ea4d4f0a11f6e2c03dee521fb668f411f1304e0afca quay.ceph.io/ceph-ci/ceph@sha256:f4452def3be4d350f4a8e5feaf7ebb1511be9298fe68478ba7a2d6fbd7d69a63 2024-04-12T21:04:46.601 INFO:journalctl@ceph.osd.2.smithi018.stdout:Apr 12 21:04:46 smithi018 podman[135170]: 2024-04-12 21:04:46.317191164 +0000 UTC m=+0.171322280 container create fc1abe2e6bce3c231d11cb4e5b1e1ae474cc8a8833b3eee289f8b41456628dc7 (image=quay.ceph.io/ceph-ci/ceph@sha256:f4452def3be4d350f4a8e5feaf7ebb1511be9298fe68478ba7a2d6fbd7d69a63, name=ceph-1cbe407a-f90d-11ee-bc8d-c7b262605968-osd-2-activate, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-vshankar-testing-20240411.061452-324baf8, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, io.buildah.version=1.33.5, ceph=True, org.label-schema.build-date=20240409, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-12T21:04:46.924 INFO:journalctl@ceph.osd.2.smithi018.stdout:Apr 12 21:04:46 smithi018 podman[135170]: 2024-04-12 21:04:46.641960879 +0000 UTC m=+0.496091992 container init fc1abe2e6bce3c231d11cb4e5b1e1ae474cc8a8833b3eee289f8b41456628dc7 (image=quay.ceph.io/ceph-ci/ceph@sha256:f4452def3be4d350f4a8e5feaf7ebb1511be9298fe68478ba7a2d6fbd7d69a63, name=ceph-1cbe407a-f90d-11ee-bc8d-c7b262605968-osd-2-activate, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, RELEASE=wip-vshankar-testing-20240411.061452-324baf8, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240409, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0) 2024-04-12T21:04:46.925 INFO:journalctl@ceph.osd.2.smithi018.stdout:Apr 12 21:04:46 smithi018 podman[135170]: 2024-04-12 21:04:46.654738179 +0000 UTC m=+0.508869269 container start fc1abe2e6bce3c231d11cb4e5b1e1ae474cc8a8833b3eee289f8b41456628dc7 (image=quay.ceph.io/ceph-ci/ceph@sha256:f4452def3be4d350f4a8e5feaf7ebb1511be9298fe68478ba7a2d6fbd7d69a63, name=ceph-1cbe407a-f90d-11ee-bc8d-c7b262605968-osd-2-activate, RELEASE=wip-vshankar-testing-20240411.061452-324baf8, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240409) 2024-04-12T21:04:46.925 INFO:journalctl@ceph.osd.2.smithi018.stdout:Apr 12 21:04:46 smithi018 podman[135170]: 2024-04-12 21:04:46.753164124 +0000 UTC m=+0.607295250 container attach fc1abe2e6bce3c231d11cb4e5b1e1ae474cc8a8833b3eee289f8b41456628dc7 (image=quay.ceph.io/ceph-ci/ceph@sha256:f4452def3be4d350f4a8e5feaf7ebb1511be9298fe68478ba7a2d6fbd7d69a63, name=ceph-1cbe407a-f90d-11ee-bc8d-c7b262605968-osd-2-activate, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240409, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, RELEASE=wip-vshankar-testing-20240411.061452-324baf8, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-12T21:04:47.279 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:04:46 smithi186 ceph-mon[86379]: pgmap v127: 65 pgs: 12 active+undersized, 10 active+undersized+degraded, 3 stale+active+clean, 40 active+clean; 257 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 35/324 objects degraded (10.802%) 2024-04-12T21:04:47.350 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:04:46 smithi018 ceph-mon[112590]: pgmap v127: 65 pgs: 12 active+undersized, 10 active+undersized+degraded, 3 stale+active+clean, 40 active+clean; 257 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 35/324 objects degraded (10.802%) 2024-04-12T21:04:47.852 INFO:journalctl@ceph.osd.2.smithi018.stdout:Apr 12 21:04:47 smithi018 ceph-1cbe407a-f90d-11ee-bc8d-c7b262605968-osd-2-activate[135185]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-04-12T21:04:47.852 INFO:journalctl@ceph.osd.2.smithi018.stdout:Apr 12 21:04:47 smithi018 ceph-1cbe407a-f90d-11ee-bc8d-c7b262605968-osd-2-activate[135185]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-2 --no-mon-config --dev /dev/mapper/vg_nvme-lv_2 2024-04-12T21:04:47.852 INFO:journalctl@ceph.osd.2.smithi018.stdout:Apr 12 21:04:47 smithi018 bash[135170]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-04-12T21:04:47.852 INFO:journalctl@ceph.osd.2.smithi018.stdout:Apr 12 21:04:47 smithi018 bash[135170]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-2 --no-mon-config --dev /dev/mapper/vg_nvme-lv_2 2024-04-12T21:04:47.852 INFO:journalctl@ceph.osd.2.smithi018.stdout:Apr 12 21:04:47 smithi018 bash[135170]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_2 2024-04-12T21:04:47.852 INFO:journalctl@ceph.osd.2.smithi018.stdout:Apr 12 21:04:47 smithi018 bash[135170]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-1 2024-04-12T21:04:47.852 INFO:journalctl@ceph.osd.2.smithi018.stdout:Apr 12 21:04:47 smithi018 bash[135170]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-2/block 2024-04-12T21:04:47.852 INFO:journalctl@ceph.osd.2.smithi018.stdout:Apr 12 21:04:47 smithi018 bash[135170]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-04-12T21:04:47.852 INFO:journalctl@ceph.osd.2.smithi018.stdout:Apr 12 21:04:47 smithi018 bash[135170]: --> ceph-volume raw activate successful for osd ID: 2 2024-04-12T21:04:47.852 INFO:journalctl@ceph.osd.2.smithi018.stdout:Apr 12 21:04:47 smithi018 ceph-1cbe407a-f90d-11ee-bc8d-c7b262605968-osd-2-activate[135185]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_2 2024-04-12T21:04:47.852 INFO:journalctl@ceph.osd.2.smithi018.stdout:Apr 12 21:04:47 smithi018 ceph-1cbe407a-f90d-11ee-bc8d-c7b262605968-osd-2-activate[135185]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-1 2024-04-12T21:04:47.852 INFO:journalctl@ceph.osd.2.smithi018.stdout:Apr 12 21:04:47 smithi018 ceph-1cbe407a-f90d-11ee-bc8d-c7b262605968-osd-2-activate[135185]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-2/block 2024-04-12T21:04:47.853 INFO:journalctl@ceph.osd.2.smithi018.stdout:Apr 12 21:04:47 smithi018 ceph-1cbe407a-f90d-11ee-bc8d-c7b262605968-osd-2-activate[135185]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-04-12T21:04:47.853 INFO:journalctl@ceph.osd.2.smithi018.stdout:Apr 12 21:04:47 smithi018 ceph-1cbe407a-f90d-11ee-bc8d-c7b262605968-osd-2-activate[135185]: --> ceph-volume raw activate successful for osd ID: 2 2024-04-12T21:04:47.853 INFO:journalctl@ceph.osd.2.smithi018.stdout:Apr 12 21:04:47 smithi018 podman[135170]: 2024-04-12 21:04:47.643050766 +0000 UTC m=+1.497181912 container died fc1abe2e6bce3c231d11cb4e5b1e1ae474cc8a8833b3eee289f8b41456628dc7 (image=quay.ceph.io/ceph-ci/ceph@sha256:f4452def3be4d350f4a8e5feaf7ebb1511be9298fe68478ba7a2d6fbd7d69a63, name=ceph-1cbe407a-f90d-11ee-bc8d-c7b262605968-osd-2-activate, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240409, RELEASE=wip-vshankar-testing-20240411.061452-324baf8, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=) 2024-04-12T21:04:48.800 INFO:journalctl@ceph.osd.2.smithi018.stdout:Apr 12 21:04:48 smithi018 podman[135314]: 2024-04-12 21:04:48.44063562 +0000 UTC m=+0.777402449 container remove fc1abe2e6bce3c231d11cb4e5b1e1ae474cc8a8833b3eee289f8b41456628dc7 (image=quay.ceph.io/ceph-ci/ceph@sha256:f4452def3be4d350f4a8e5feaf7ebb1511be9298fe68478ba7a2d6fbd7d69a63, name=ceph-1cbe407a-f90d-11ee-bc8d-c7b262605968-osd-2-activate, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240409, org.label-schema.vendor=CentOS, RELEASE=wip-vshankar-testing-20240411.061452-324baf8, ceph=True, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-12T21:04:49.100 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:04:48 smithi018 ceph-mon[112590]: pgmap v128: 65 pgs: 17 active+undersized, 14 active+undersized+degraded, 34 active+clean; 257 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 48/324 objects degraded (14.815%) 2024-04-12T21:04:49.101 INFO:journalctl@ceph.osd.2.smithi018.stdout:Apr 12 21:04:48 smithi018 podman[135377]: 2024-04-12 21:04:48.710178386 +0000 UTC m=+0.039052150 image pull 8717720dc5e5c56303818ea4d4f0a11f6e2c03dee521fb668f411f1304e0afca quay.ceph.io/ceph-ci/ceph@sha256:f4452def3be4d350f4a8e5feaf7ebb1511be9298fe68478ba7a2d6fbd7d69a63 2024-04-12T21:04:49.101 INFO:journalctl@ceph.osd.2.smithi018.stdout:Apr 12 21:04:48 smithi018 podman[135377]: 2024-04-12 21:04:48.891760778 +0000 UTC m=+0.220634576 container create c2cbc6aa2839b571d7972062a0f7b290aac7616bef836d4755b6844b541f65ad (image=quay.ceph.io/ceph-ci/ceph@sha256:f4452def3be4d350f4a8e5feaf7ebb1511be9298fe68478ba7a2d6fbd7d69a63, name=ceph-1cbe407a-f90d-11ee-bc8d-c7b262605968-osd-2, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, ceph=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240409, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-vshankar-testing-20240411.061452-324baf8, maintainer=Guillaume Abrioux ) 2024-04-12T21:04:49.279 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:04:48 smithi186 ceph-mon[86379]: pgmap v128: 65 pgs: 17 active+undersized, 14 active+undersized+degraded, 34 active+clean; 257 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 48/324 objects degraded (14.815%) 2024-04-12T21:04:49.601 INFO:journalctl@ceph.osd.2.smithi018.stdout:Apr 12 21:04:49 smithi018 podman[135377]: 2024-04-12 21:04:49.161829469 +0000 UTC m=+0.490703270 container init c2cbc6aa2839b571d7972062a0f7b290aac7616bef836d4755b6844b541f65ad (image=quay.ceph.io/ceph-ci/ceph@sha256:f4452def3be4d350f4a8e5feaf7ebb1511be9298fe68478ba7a2d6fbd7d69a63, name=ceph-1cbe407a-f90d-11ee-bc8d-c7b262605968-osd-2, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-vshankar-testing-20240411.061452-324baf8, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240409, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, ceph=True, GIT_BRANCH=HEAD) 2024-04-12T21:04:49.601 INFO:journalctl@ceph.osd.2.smithi018.stdout:Apr 12 21:04:49 smithi018 podman[135377]: 2024-04-12 21:04:49.173162017 +0000 UTC m=+0.502035838 container start c2cbc6aa2839b571d7972062a0f7b290aac7616bef836d4755b6844b541f65ad (image=quay.ceph.io/ceph-ci/ceph@sha256:f4452def3be4d350f4a8e5feaf7ebb1511be9298fe68478ba7a2d6fbd7d69a63, name=ceph-1cbe407a-f90d-11ee-bc8d-c7b262605968-osd-2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240409, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, ceph=True, RELEASE=wip-vshankar-testing-20240411.061452-324baf8, CEPH_POINT_RELEASE=) 2024-04-12T21:04:49.601 INFO:journalctl@ceph.osd.2.smithi018.stdout:Apr 12 21:04:49 smithi018 ceph-osd[135396]: -- 172.21.15.18:0/3034452002 <== mon.0 v2:172.21.15.18:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55f8e2e5d680 con 0x55f8e2e27000 2024-04-12T21:04:49.601 INFO:journalctl@ceph.osd.2.smithi018.stdout:Apr 12 21:04:49 smithi018 bash[135377]: c2cbc6aa2839b571d7972062a0f7b290aac7616bef836d4755b6844b541f65ad 2024-04-12T21:04:49.602 INFO:journalctl@ceph.osd.2.smithi018.stdout:Apr 12 21:04:49 smithi018 systemd[1]: Started Ceph osd.2 for 1cbe407a-f90d-11ee-bc8d-c7b262605968. 2024-04-12T21:04:50.100 INFO:journalctl@ceph.osd.2.smithi018.stdout:Apr 12 21:04:49 smithi018 ceph-1cbe407a-f90d-11ee-bc8d-c7b262605968-osd-2[135391]: 2024-04-12T21:04:49.754+0000 7fb087f4f7c0 -1 Falling back to public interface 2024-04-12T21:04:50.779 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:04:50 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' 2024-04-12T21:04:50.780 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:04:50 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' 2024-04-12T21:04:50.780 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:04:50 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-12T21:04:50.780 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:04:50 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' 2024-04-12T21:04:50.780 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:04:50 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' 2024-04-12T21:04:50.850 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:04:50 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' 2024-04-12T21:04:50.851 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:04:50 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' 2024-04-12T21:04:50.851 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:04:50 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-12T21:04:50.851 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:04:50 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' 2024-04-12T21:04:50.851 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:04:50 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' 2024-04-12T21:04:51.726 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:04:51 smithi018 ceph-mon[112590]: pgmap v129: 65 pgs: 17 active+undersized, 14 active+undersized+degraded, 34 active+clean; 257 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 48/324 objects degraded (14.815%) 2024-04-12T21:04:51.779 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:04:51 smithi186 ceph-mon[86379]: pgmap v129: 65 pgs: 17 active+undersized, 14 active+undersized+degraded, 34 active+clean; 257 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 48/324 objects degraded (14.815%) 2024-04-12T21:04:53.707 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:04:53 smithi018 ceph-mon[112590]: pgmap v130: 65 pgs: 17 active+undersized, 14 active+undersized+degraded, 34 active+clean; 257 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 48/324 objects degraded (14.815%) 2024-04-12T21:04:53.779 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:04:53 smithi186 ceph-mon[86379]: pgmap v130: 65 pgs: 17 active+undersized, 14 active+undersized+degraded, 34 active+clean; 257 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 48/324 objects degraded (14.815%) 2024-04-12T21:04:55.350 INFO:journalctl@ceph.osd.2.smithi018.stdout:Apr 12 21:04:55 smithi018 ceph-1cbe407a-f90d-11ee-bc8d-c7b262605968-osd-2[135391]: 2024-04-12T21:04:55.010+0000 7fb087f4f7c0 -1 osd.2 78 log_to_monitors true 2024-04-12T21:04:55.722 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:04:55 smithi018 ceph-mon[112590]: pgmap v131: 65 pgs: 17 active+undersized, 14 active+undersized+degraded, 34 active+clean; 257 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 48/324 objects degraded (14.815%) 2024-04-12T21:04:55.723 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:04:55 smithi018 ceph-mon[112590]: Health check update: Degraded data redundancy: 48/324 objects degraded (14.815%), 14 pgs degraded (PG_DEGRADED) 2024-04-12T21:04:55.723 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:04:55 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' 2024-04-12T21:04:55.723 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:04:55 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T21:04:55.723 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:04:55 smithi018 ceph-mon[112590]: from='osd.2 [v2:172.21.15.18:6818/1361831750,v1:172.21.15.18:6819/1361831750]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-12T21:04:55.779 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:04:55 smithi186 ceph-mon[86379]: pgmap v131: 65 pgs: 17 active+undersized, 14 active+undersized+degraded, 34 active+clean; 257 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 48/324 objects degraded (14.815%) 2024-04-12T21:04:55.780 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:04:55 smithi186 ceph-mon[86379]: Health check update: Degraded data redundancy: 48/324 objects degraded (14.815%), 14 pgs degraded (PG_DEGRADED) 2024-04-12T21:04:55.780 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:04:55 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' 2024-04-12T21:04:55.780 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:04:55 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T21:04:55.780 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:04:55 smithi186 ceph-mon[86379]: from='osd.2 [v2:172.21.15.18:6818/1361831750,v1:172.21.15.18:6819/1361831750]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-12T21:04:56.025 INFO:journalctl@ceph.osd.2.smithi018.stdout:Apr 12 21:04:55 smithi018 ceph-1cbe407a-f90d-11ee-bc8d-c7b262605968-osd-2[135391]: 2024-04-12T21:04:55.892+0000 7fb077200700 -1 osd.2 78 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-12T21:04:57.196 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:04:56 smithi018 ceph-mon[112590]: pgmap v132: 65 pgs: 17 active+undersized, 14 active+undersized+degraded, 34 active+clean; 257 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 48/324 objects degraded (14.815%) 2024-04-12T21:04:57.196 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:04:56 smithi018 ceph-mon[112590]: from='osd.2 [v2:172.21.15.18:6818/1361831750,v1:172.21.15.18:6819/1361831750]' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-04-12T21:04:57.197 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:04:56 smithi018 ceph-mon[112590]: osdmap e81: 6 total, 5 up, 6 in 2024-04-12T21:04:57.197 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:04:56 smithi018 ceph-mon[112590]: from='osd.2 [v2:172.21.15.18:6818/1361831750,v1:172.21.15.18:6819/1361831750]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi018", "root=default"]}]: dispatch 2024-04-12T21:04:57.280 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:04:56 smithi186 ceph-mon[86379]: pgmap v132: 65 pgs: 17 active+undersized, 14 active+undersized+degraded, 34 active+clean; 257 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 48/324 objects degraded (14.815%) 2024-04-12T21:04:57.280 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:04:56 smithi186 ceph-mon[86379]: from='osd.2 [v2:172.21.15.18:6818/1361831750,v1:172.21.15.18:6819/1361831750]' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-04-12T21:04:57.280 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:04:56 smithi186 ceph-mon[86379]: osdmap e81: 6 total, 5 up, 6 in 2024-04-12T21:04:57.280 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:04:56 smithi186 ceph-mon[86379]: from='osd.2 [v2:172.21.15.18:6818/1361831750,v1:172.21.15.18:6819/1361831750]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi018", "root=default"]}]: dispatch 2024-04-12T21:04:58.279 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:04:57 smithi186 ceph-mon[86379]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-04-12T21:04:58.280 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:04:57 smithi186 ceph-mon[86379]: osd.2 [v2:172.21.15.18:6818/1361831750,v1:172.21.15.18:6819/1361831750] boot 2024-04-12T21:04:58.280 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:04:57 smithi186 ceph-mon[86379]: osdmap e82: 6 total, 6 up, 6 in 2024-04-12T21:04:58.280 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:04:57 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-12T21:04:58.280 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:04:57 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' 2024-04-12T21:04:58.280 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:04:57 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' 2024-04-12T21:04:58.280 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:04:57 smithi186 ceph-mon[86379]: osdmap e83: 6 total, 6 up, 6 in 2024-04-12T21:04:58.350 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:04:57 smithi018 ceph-mon[112590]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-04-12T21:04:58.351 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:04:57 smithi018 ceph-mon[112590]: osd.2 [v2:172.21.15.18:6818/1361831750,v1:172.21.15.18:6819/1361831750] boot 2024-04-12T21:04:58.351 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:04:57 smithi018 ceph-mon[112590]: osdmap e82: 6 total, 6 up, 6 in 2024-04-12T21:04:58.351 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:04:57 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-12T21:04:58.351 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:04:57 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' 2024-04-12T21:04:58.351 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:04:57 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' 2024-04-12T21:04:58.351 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:04:57 smithi018 ceph-mon[112590]: osdmap e83: 6 total, 6 up, 6 in 2024-04-12T21:04:59.279 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:04:58 smithi186 ceph-mon[86379]: pgmap v135: 65 pgs: 17 active+undersized, 14 active+undersized+degraded, 34 active+clean; 257 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 48/324 objects degraded (14.815%) 2024-04-12T21:04:59.350 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:04:58 smithi018 ceph-mon[112590]: pgmap v135: 65 pgs: 17 active+undersized, 14 active+undersized+degraded, 34 active+clean; 257 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 48/324 objects degraded (14.815%) 2024-04-12T21:05:00.279 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:04:59 smithi186 ceph-mon[86379]: Health check update: Degraded data redundancy: 45/324 objects degraded (13.889%), 13 pgs degraded (PG_DEGRADED) 2024-04-12T21:05:00.351 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:04:59 smithi018 ceph-mon[112590]: Health check update: Degraded data redundancy: 45/324 objects degraded (13.889%), 13 pgs degraded (PG_DEGRADED) 2024-04-12T21:05:01.279 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:05:00 smithi186 ceph-mon[86379]: pgmap v137: 65 pgs: 15 active+undersized, 13 active+undersized+degraded, 37 active+clean; 257 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 45/324 objects degraded (13.889%) 2024-04-12T21:05:01.279 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:05:00 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' 2024-04-12T21:05:01.280 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:05:00 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' 2024-04-12T21:05:01.316 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:05:00 smithi018 ceph-mon[112590]: pgmap v137: 65 pgs: 15 active+undersized, 13 active+undersized+degraded, 37 active+clean; 257 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 45/324 objects degraded (13.889%) 2024-04-12T21:05:01.316 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:05:00 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' 2024-04-12T21:05:01.316 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:05:00 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' 2024-04-12T21:05:02.100 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:05:02 smithi018 ceph-mon[112590]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 45/324 objects degraded (13.889%), 13 pgs degraded) 2024-04-12T21:05:02.100 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:05:02 smithi018 ceph-mon[112590]: Cluster is now healthy 2024-04-12T21:05:02.279 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:05:02 smithi186 ceph-mon[86379]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 45/324 objects degraded (13.889%), 13 pgs degraded) 2024-04-12T21:05:02.279 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:05:02 smithi186 ceph-mon[86379]: Cluster is now healthy 2024-04-12T21:05:03.279 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:05:03 smithi186 ceph-mon[86379]: pgmap v138: 65 pgs: 65 active+clean; 257 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-12T21:05:03.351 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:05:03 smithi018 ceph-mon[112590]: pgmap v138: 65 pgs: 65 active+clean; 257 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-12T21:05:05.279 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:05:05 smithi186 ceph-mon[86379]: pgmap v139: 65 pgs: 65 active+clean; 257 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-12T21:05:05.350 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:05:05 smithi018 ceph-mon[112590]: pgmap v139: 65 pgs: 65 active+clean; 257 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-12T21:05:06.973 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:05:06 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' 2024-04-12T21:05:06.973 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:05:06 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' 2024-04-12T21:05:06.973 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:05:06 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-12T21:05:06.973 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:05:06 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-12T21:05:06.973 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:05:06 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' 2024-04-12T21:05:06.974 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:05:06 smithi186 ceph-mon[86379]: pgmap v140: 65 pgs: 65 active+clean; 257 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-12T21:05:06.974 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:05:06 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-12T21:05:06.974 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:05:06 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T21:05:06.974 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:05:06 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T21:05:06.974 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:05:06 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T21:05:06.974 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:05:06 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd ok-to-stop", "ids": ["3"], "max": 16}]: dispatch 2024-04-12T21:05:06.974 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:05:06 smithi186 ceph-mon[86379]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["3"], "max": 16}]: dispatch 2024-04-12T21:05:06.974 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:05:06 smithi186 ceph-mon[86379]: Upgrade: osd.3 is safe to restart 2024-04-12T21:05:07.101 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:05:06 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' 2024-04-12T21:05:07.101 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:05:06 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' 2024-04-12T21:05:07.101 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:05:06 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-12T21:05:07.101 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:05:06 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-12T21:05:07.101 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:05:06 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' 2024-04-12T21:05:07.101 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:05:06 smithi018 ceph-mon[112590]: pgmap v140: 65 pgs: 65 active+clean; 257 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-12T21:05:07.101 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:05:06 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-12T21:05:07.101 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:05:06 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T21:05:07.101 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:05:06 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T21:05:07.102 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:05:06 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T21:05:07.102 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:05:06 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd ok-to-stop", "ids": ["3"], "max": 16}]: dispatch 2024-04-12T21:05:07.102 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:05:06 smithi018 ceph-mon[112590]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["3"], "max": 16}]: dispatch 2024-04-12T21:05:07.102 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:05:06 smithi018 ceph-mon[112590]: Upgrade: osd.3 is safe to restart 2024-04-12T21:05:09.529 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:05:09 smithi186 ceph-mon[86379]: pgmap v141: 65 pgs: 65 active+clean; 257 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-12T21:05:09.530 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:05:09 smithi186 ceph-mon[86379]: Upgrade: Updating osd.3 2024-04-12T21:05:09.530 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:05:09 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' 2024-04-12T21:05:09.530 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:05:09 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-04-12T21:05:09.530 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:05:09 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-12T21:05:09.530 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:05:09 smithi186 ceph-mon[86379]: Deploying daemon osd.3 on smithi186 2024-04-12T21:05:09.600 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:05:09 smithi018 ceph-mon[112590]: pgmap v141: 65 pgs: 65 active+clean; 257 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-12T21:05:09.601 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:05:09 smithi018 ceph-mon[112590]: Upgrade: Updating osd.3 2024-04-12T21:05:09.601 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:05:09 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' 2024-04-12T21:05:09.601 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:05:09 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-04-12T21:05:09.601 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:05:09 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-12T21:05:09.601 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:05:09 smithi018 ceph-mon[112590]: Deploying daemon osd.3 on smithi186 2024-04-12T21:05:11.075 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:05:10 smithi186 ceph-mon[86379]: pgmap v142: 65 pgs: 65 active+clean; 257 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-12T21:05:11.075 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:05:10 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' 2024-04-12T21:05:11.076 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:05:10 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T21:05:11.076 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:05:10 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' 2024-04-12T21:05:11.350 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:05:10 smithi018 ceph-mon[112590]: pgmap v142: 65 pgs: 65 active+clean; 257 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-12T21:05:11.351 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:05:10 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' 2024-04-12T21:05:11.351 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:05:10 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T21:05:11.351 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:05:10 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' 2024-04-12T21:05:12.547 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:12.548+0000 7f129c400700 1 -- 172.21.15.18:0/1326920690 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f12941037a0 msgr2=0x7f12940fce10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:05:12.547 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:12.548+0000 7f129c400700 1 --2- 172.21.15.18:0/1326920690 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f12941037a0 0x7f12940fce10 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f1280004940 tx=0x7f1280009e30 comp rx=0 tx=0).stop 2024-04-12T21:05:12.547 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:12.548+0000 7f129c400700 1 -- 172.21.15.18:0/1326920690 shutdown_connections 2024-04-12T21:05:12.547 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:12.548+0000 7f129c400700 1 --2- 172.21.15.18:0/1326920690 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f12941037a0 0x7f12940fce10 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:05:12.547 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:12.548+0000 7f129c400700 1 --2- 172.21.15.18:0/1326920690 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1294102e00 0x7f12941031d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:05:12.547 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:12.548+0000 7f129c400700 1 -- 172.21.15.18:0/1326920690 >> 172.21.15.18:0/1326920690 conn(0x7f12940f89b0 msgr2=0x7f12940fadc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:05:12.547 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:12.548+0000 7f129c400700 1 -- 172.21.15.18:0/1326920690 shutdown_connections 2024-04-12T21:05:12.547 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:12.548+0000 7f129c400700 1 -- 172.21.15.18:0/1326920690 wait complete. 2024-04-12T21:05:12.547 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:12.549+0000 7f129c400700 1 Processor -- start 2024-04-12T21:05:12.547 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:12.549+0000 7f129c400700 1 -- start start 2024-04-12T21:05:12.548 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:12.549+0000 7f129c400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1294102e00 0x7f1294115bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:05:12.548 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:12.550+0000 7f129c400700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f12941037a0 0x7f1294116110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:05:12.548 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:12.550+0000 7f129c400700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1294116760 con 0x7f1294102e00 2024-04-12T21:05:12.548 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:12.550+0000 7f129c400700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f129410fd40 con 0x7f12941037a0 2024-04-12T21:05:12.548 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:12.550+0000 7f1299a00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1294102e00 0x7f1294115bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:05:12.548 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:12.550+0000 7f1299a00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1294102e00 0x7f1294115bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:46748/0 (socket says 172.21.15.18:46748) 2024-04-12T21:05:12.548 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:12.550+0000 7f1299a00700 1 -- 172.21.15.18:0/3590241805 learned_addr learned my addr 172.21.15.18:0/3590241805 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:05:12.549 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:12.550+0000 7f1299000700 1 --2- 172.21.15.18:0/3590241805 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f12941037a0 0x7f1294116110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:05:12.549 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:12.550+0000 7f1299a00700 1 -- 172.21.15.18:0/3590241805 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f12941037a0 msgr2=0x7f1294116110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:05:12.549 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:12.550+0000 7f1299a00700 1 --2- 172.21.15.18:0/3590241805 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f12941037a0 0x7f1294116110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:05:12.549 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:12.550+0000 7f1299a00700 1 -- 172.21.15.18:0/3590241805 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1280005040 con 0x7f1294102e00 2024-04-12T21:05:12.549 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:12.551+0000 7f1299a00700 1 --2- 172.21.15.18:0/3590241805 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1294102e00 0x7f1294115bd0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f128800c830 tx=0x7f1288004470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:05:12.549 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:12.551+0000 7f1291600700 1 -- 172.21.15.18:0/3590241805 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f128800a760 con 0x7f1294102e00 2024-04-12T21:05:12.549 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:12.551+0000 7f1291600700 1 -- 172.21.15.18:0/3590241805 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f128800ad90 con 0x7f1294102e00 2024-04-12T21:05:12.549 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:12.551+0000 7f1291600700 1 -- 172.21.15.18:0/3590241805 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f12880113a0 con 0x7f1294102e00 2024-04-12T21:05:12.549 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:12.551+0000 7f129c400700 1 -- 172.21.15.18:0/3590241805 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1294110020 con 0x7f1294102e00 2024-04-12T21:05:12.550 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:12.551+0000 7f129c400700 1 -- 172.21.15.18:0/3590241805 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1294110560 con 0x7f1294102e00 2024-04-12T21:05:12.550 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:12.552+0000 7f129c400700 1 -- 172.21.15.18:0/3590241805 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1294102120 con 0x7f1294102e00 2024-04-12T21:05:12.550 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:12.553+0000 7f1291600700 1 -- 172.21.15.18:0/3590241805 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7f1288012400 con 0x7f1294102e00 2024-04-12T21:05:12.551 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:12.554+0000 7f1291600700 1 --2- 172.21.15.18:0/3590241805 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f127c06ec00 0x7f127c0710b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:05:12.551 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:12.555+0000 7f1291600700 1 -- 172.21.15.18:0/3590241805 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6585+0+0 (secure 0 0 0) 0x7f128808e9f0 con 0x7f1294102e00 2024-04-12T21:05:12.552 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:12.555+0000 7f1299000700 1 --2- 172.21.15.18:0/3590241805 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f127c06ec00 0x7f127c0710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:05:12.552 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:12.555+0000 7f1299000700 1 --2- 172.21.15.18:0/3590241805 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f127c06ec00 0x7f127c0710b0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f128000b020 tx=0x7f1280009e30 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:05:12.556 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:12.560+0000 7f1291600700 1 -- 172.21.15.18:0/3590241805 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f128805b250 con 0x7f1294102e00 2024-04-12T21:05:12.808 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:12.811+0000 7f129c400700 1 -- 172.21.15.18:0/3590241805 --> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f1294109ee0 con 0x7f127c06ec00 2024-04-12T21:05:12.810 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:12.813+0000 7f1291600700 1 -- 172.21.15.18:0/3590241805 <== mgr.34102 v2:172.21.15.18:6800/3812891919 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7f1294109ee0 con 0x7f127c06ec00 2024-04-12T21:05:12.813 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:12.816+0000 7f129c400700 1 -- 172.21.15.18:0/3590241805 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f127c06ec00 msgr2=0x7f127c0710b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:05:12.813 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:12.816+0000 7f129c400700 1 --2- 172.21.15.18:0/3590241805 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f127c06ec00 0x7f127c0710b0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f128000b020 tx=0x7f1280009e30 comp rx=0 tx=0).stop 2024-04-12T21:05:12.813 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:12.816+0000 7f129c400700 1 -- 172.21.15.18:0/3590241805 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1294102e00 msgr2=0x7f1294115bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:05:12.813 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:12.816+0000 7f129c400700 1 --2- 172.21.15.18:0/3590241805 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1294102e00 0x7f1294115bd0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f128800c830 tx=0x7f1288004470 comp rx=0 tx=0).stop 2024-04-12T21:05:12.813 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:12.816+0000 7f129c400700 1 -- 172.21.15.18:0/3590241805 shutdown_connections 2024-04-12T21:05:12.813 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:12.816+0000 7f129c400700 1 --2- 172.21.15.18:0/3590241805 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f127c06ec00 0x7f127c0710b0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:05:12.813 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:12.816+0000 7f129c400700 1 --2- 172.21.15.18:0/3590241805 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1294102e00 0x7f1294115bd0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:05:12.813 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:12.816+0000 7f129c400700 1 --2- 172.21.15.18:0/3590241805 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f12941037a0 0x7f1294116110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:05:12.813 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:12.816+0000 7f129c400700 1 -- 172.21.15.18:0/3590241805 >> 172.21.15.18:0/3590241805 conn(0x7f12940f89b0 msgr2=0x7f1294070c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:05:12.814 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:12.816+0000 7f129c400700 1 -- 172.21.15.18:0/3590241805 shutdown_connections 2024-04-12T21:05:12.814 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:12.817+0000 7f129c400700 1 -- 172.21.15.18:0/3590241805 wait complete. 2024-04-12T21:05:12.829 INFO:teuthology.orchestra.run.smithi018.stdout:true 2024-04-12T21:05:12.975 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:12.977+0000 7f22aa800700 1 -- 172.21.15.18:0/4014561102 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f22a4109dd0 msgr2=0x7f22a4103440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:05:12.975 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:12.977+0000 7f22aa800700 1 --2- 172.21.15.18:0/4014561102 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f22a4109dd0 0x7f22a4103440 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f2290004940 tx=0x7f2290009e30 comp rx=0 tx=0).stop 2024-04-12T21:05:12.975 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:12.978+0000 7f22aa800700 1 -- 172.21.15.18:0/4014561102 shutdown_connections 2024-04-12T21:05:12.975 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:12.978+0000 7f22aa800700 1 --2- 172.21.15.18:0/4014561102 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f22a4109dd0 0x7f22a4103440 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:05:12.975 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:12.978+0000 7f22aa800700 1 --2- 172.21.15.18:0/4014561102 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f22a4109430 0x7f22a4109800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:05:12.975 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:12.978+0000 7f22aa800700 1 -- 172.21.15.18:0/4014561102 >> 172.21.15.18:0/4014561102 conn(0x7f22a4072c00 msgr2=0x7f22a4075010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:05:12.975 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:12.978+0000 7f22aa800700 1 -- 172.21.15.18:0/4014561102 shutdown_connections 2024-04-12T21:05:12.976 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:12.978+0000 7f22aa800700 1 -- 172.21.15.18:0/4014561102 wait complete. 2024-04-12T21:05:12.976 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:12.979+0000 7f22aa800700 1 Processor -- start 2024-04-12T21:05:12.977 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:12.979+0000 7f22aa800700 1 -- start start 2024-04-12T21:05:12.977 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:12.979+0000 7f22aa800700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f22a4109430 0x7f22a411a220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:05:12.977 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:12.979+0000 7f22aa800700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f22a411a760 0x7f22a4114390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:05:12.977 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:12.979+0000 7f22aa800700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22a411ace0 con 0x7f22a4109430 2024-04-12T21:05:12.977 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:12.979+0000 7f22aa800700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22a411ae50 con 0x7f22a411a760 2024-04-12T21:05:12.977 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:12.980+0000 7f22a3400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f22a4109430 0x7f22a411a220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:05:12.977 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:12.980+0000 7f22a3400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f22a4109430 0x7f22a411a220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:46764/0 (socket says 172.21.15.18:46764) 2024-04-12T21:05:12.977 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:12.980+0000 7f22a3400700 1 -- 172.21.15.18:0/1367300931 learned_addr learned my addr 172.21.15.18:0/1367300931 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:05:12.978 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:12.980+0000 7f22a2a00700 1 --2- 172.21.15.18:0/1367300931 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f22a411a760 0x7f22a4114390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:05:12.978 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:12.980+0000 7f22a3400700 1 -- 172.21.15.18:0/1367300931 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f22a411a760 msgr2=0x7f22a4114390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:05:12.978 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:12.980+0000 7f22a3400700 1 --2- 172.21.15.18:0/1367300931 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f22a411a760 0x7f22a4114390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:05:12.978 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:12.980+0000 7f22a3400700 1 -- 172.21.15.18:0/1367300931 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2290005040 con 0x7f22a4109430 2024-04-12T21:05:12.978 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:12.980+0000 7f22a3400700 1 --2- 172.21.15.18:0/1367300931 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f22a4109430 0x7f22a411a220 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f229400c8c0 tx=0x7f22940044c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:05:12.978 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:12.981+0000 7f229b400700 1 -- 172.21.15.18:0/1367300931 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f229400a980 con 0x7f22a4109430 2024-04-12T21:05:12.978 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:12.981+0000 7f229b400700 1 -- 172.21.15.18:0/1367300931 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f229400ce60 con 0x7f22a4109430 2024-04-12T21:05:12.978 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:12.981+0000 7f22aa800700 1 -- 172.21.15.18:0/1367300931 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f22a4114990 con 0x7f22a4109430 2024-04-12T21:05:12.979 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:12.981+0000 7f229b400700 1 -- 172.21.15.18:0/1367300931 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f22940115e0 con 0x7f22a4109430 2024-04-12T21:05:12.979 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:12.981+0000 7f22aa800700 1 -- 172.21.15.18:0/1367300931 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f22a4114f50 con 0x7f22a4109430 2024-04-12T21:05:12.979 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:12.982+0000 7f22aa800700 1 -- 172.21.15.18:0/1367300931 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f22a4104b80 con 0x7f22a4109430 2024-04-12T21:05:12.980 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:12.983+0000 7f229b400700 1 -- 172.21.15.18:0/1367300931 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7f2294012400 con 0x7f22a4109430 2024-04-12T21:05:12.981 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:12.984+0000 7f229b400700 1 --2- 172.21.15.18:0/1367300931 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f228806ec00 0x7f22880710b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:05:12.981 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:12.984+0000 7f229b400700 1 -- 172.21.15.18:0/1367300931 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6585+0+0 (secure 0 0 0) 0x7f229408e9e0 con 0x7f22a4109430 2024-04-12T21:05:12.981 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:12.984+0000 7f22a2a00700 1 --2- 172.21.15.18:0/1367300931 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f228806ec00 0x7f22880710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:05:12.982 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:12.985+0000 7f22a2a00700 1 --2- 172.21.15.18:0/1367300931 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f228806ec00 0x7f22880710b0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f2290009040 tx=0x7f2290009e30 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:05:12.986 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:12.989+0000 7f229b400700 1 -- 172.21.15.18:0/1367300931 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f229405b2c0 con 0x7f22a4109430 2024-04-12T21:05:13.029 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:05:12 smithi186 ceph-mon[86379]: pgmap v143: 65 pgs: 65 active+clean; 257 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-12T21:05:13.100 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:05:12 smithi018 ceph-mon[112590]: pgmap v143: 65 pgs: 65 active+clean; 257 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-12T21:05:13.239 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:13.242+0000 7f22aa800700 1 -- 172.21.15.18:0/1367300931 --> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f22a4115870 con 0x7f228806ec00 2024-04-12T21:05:13.242 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:13.245+0000 7f229b400700 1 -- 172.21.15.18:0/1367300931 <== mgr.34102 v2:172.21.15.18:6800/3812891919 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7f22a4115870 con 0x7f228806ec00 2024-04-12T21:05:13.245 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:13.247+0000 7f22aa800700 1 -- 172.21.15.18:0/1367300931 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f228806ec00 msgr2=0x7f22880710b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:05:13.245 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:13.247+0000 7f22aa800700 1 --2- 172.21.15.18:0/1367300931 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f228806ec00 0x7f22880710b0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f2290009040 tx=0x7f2290009e30 comp rx=0 tx=0).stop 2024-04-12T21:05:13.245 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:13.247+0000 7f22aa800700 1 -- 172.21.15.18:0/1367300931 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f22a4109430 msgr2=0x7f22a411a220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:05:13.245 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:13.248+0000 7f22aa800700 1 --2- 172.21.15.18:0/1367300931 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f22a4109430 0x7f22a411a220 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f229400c8c0 tx=0x7f22940044c0 comp rx=0 tx=0).stop 2024-04-12T21:05:13.245 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:13.248+0000 7f22aa800700 1 -- 172.21.15.18:0/1367300931 shutdown_connections 2024-04-12T21:05:13.245 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:13.248+0000 7f22aa800700 1 --2- 172.21.15.18:0/1367300931 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f228806ec00 0x7f22880710b0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:05:13.245 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:13.248+0000 7f22aa800700 1 --2- 172.21.15.18:0/1367300931 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f22a4109430 0x7f22a411a220 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:05:13.245 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:13.248+0000 7f22aa800700 1 --2- 172.21.15.18:0/1367300931 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f22a411a760 0x7f22a4114390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:05:13.245 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:13.248+0000 7f22aa800700 1 -- 172.21.15.18:0/1367300931 >> 172.21.15.18:0/1367300931 conn(0x7f22a4072c00 msgr2=0x7f22a4074da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:05:13.246 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:13.248+0000 7f22aa800700 1 -- 172.21.15.18:0/1367300931 shutdown_connections 2024-04-12T21:05:13.246 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:13.248+0000 7f22aa800700 1 -- 172.21.15.18:0/1367300931 wait complete. 2024-04-12T21:05:13.407 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:13.409+0000 7fd277200700 1 -- 172.21.15.18:0/557108129 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd278104e60 msgr2=0x7fd278105230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:05:13.407 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:13.409+0000 7fd277200700 1 --2- 172.21.15.18:0/557108129 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd278104e60 0x7fd278105230 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fd268004940 tx=0x7fd268009e30 comp rx=0 tx=0).stop 2024-04-12T21:05:13.407 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:13.409+0000 7fd277200700 1 -- 172.21.15.18:0/557108129 shutdown_connections 2024-04-12T21:05:13.407 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:13.409+0000 7fd277200700 1 --2- 172.21.15.18:0/557108129 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd278105800 0x7fd2780fee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:05:13.407 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:13.409+0000 7fd277200700 1 --2- 172.21.15.18:0/557108129 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd278104e60 0x7fd278105230 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:05:13.407 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:13.409+0000 7fd277200700 1 -- 172.21.15.18:0/557108129 >> 172.21.15.18:0/557108129 conn(0x7fd2780fab10 msgr2=0x7fd2780fcef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:05:13.407 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:13.410+0000 7fd277200700 1 -- 172.21.15.18:0/557108129 shutdown_connections 2024-04-12T21:05:13.407 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:13.410+0000 7fd277200700 1 -- 172.21.15.18:0/557108129 wait complete. 2024-04-12T21:05:13.408 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:13.411+0000 7fd277200700 1 Processor -- start 2024-04-12T21:05:13.408 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:13.411+0000 7fd277200700 1 -- start start 2024-04-12T21:05:13.408 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:13.411+0000 7fd277200700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd278104e60 0x7fd278115b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:05:13.408 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:13.411+0000 7fd277200700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd278105800 0x7fd2781160d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:05:13.409 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:13.411+0000 7fd277200700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd278116720 con 0x7fd278104e60 2024-04-12T21:05:13.409 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:13.411+0000 7fd277200700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd27810fb10 con 0x7fd278105800 2024-04-12T21:05:13.410 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:13.412+0000 7fd275e00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd278104e60 0x7fd278115b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:05:13.410 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:13.412+0000 7fd275e00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd278104e60 0x7fd278115b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:46796/0 (socket says 172.21.15.18:46796) 2024-04-12T21:05:13.410 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:13.412+0000 7fd275e00700 1 -- 172.21.15.18:0/1143514706 learned_addr learned my addr 172.21.15.18:0/1143514706 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:05:13.410 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:13.412+0000 7fd275400700 1 --2- 172.21.15.18:0/1143514706 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd278105800 0x7fd2781160d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:05:13.410 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:13.412+0000 7fd275e00700 1 -- 172.21.15.18:0/1143514706 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd278105800 msgr2=0x7fd2781160d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:05:13.410 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:13.412+0000 7fd275e00700 1 --2- 172.21.15.18:0/1143514706 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd278105800 0x7fd2781160d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:05:13.410 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:13.412+0000 7fd275e00700 1 -- 172.21.15.18:0/1143514706 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd268005040 con 0x7fd278104e60 2024-04-12T21:05:13.410 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:13.413+0000 7fd275e00700 1 --2- 172.21.15.18:0/1143514706 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd278104e60 0x7fd278115b90 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fd2680090d0 tx=0x7fd268009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:05:13.411 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:13.413+0000 7fd26e000700 1 -- 172.21.15.18:0/1143514706 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd268021070 con 0x7fd278104e60 2024-04-12T21:05:13.411 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:13.413+0000 7fd26e000700 1 -- 172.21.15.18:0/1143514706 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd268013c00 con 0x7fd278104e60 2024-04-12T21:05:13.411 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:13.413+0000 7fd277200700 1 -- 172.21.15.18:0/1143514706 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd27810fd90 con 0x7fd278104e60 2024-04-12T21:05:13.411 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:13.413+0000 7fd26e000700 1 -- 172.21.15.18:0/1143514706 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd26800e070 con 0x7fd278104e60 2024-04-12T21:05:13.411 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:13.413+0000 7fd277200700 1 -- 172.21.15.18:0/1143514706 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd2781102f0 con 0x7fd278104e60 2024-04-12T21:05:13.412 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:13.415+0000 7fd277200700 1 -- 172.21.15.18:0/1143514706 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd2781005b0 con 0x7fd278104e60 2024-04-12T21:05:13.414 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:13.417+0000 7fd26e000700 1 -- 172.21.15.18:0/1143514706 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7fd26801c400 con 0x7fd278104e60 2024-04-12T21:05:13.415 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:13.418+0000 7fd26e000700 1 --2- 172.21.15.18:0/1143514706 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fd25c06ec10 0x7fd25c0710c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:05:13.415 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:13.418+0000 7fd26e000700 1 -- 172.21.15.18:0/1143514706 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6585+0+0 (secure 0 0 0) 0x7fd268090490 con 0x7fd278104e60 2024-04-12T21:05:13.415 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:13.418+0000 7fd275400700 1 --2- 172.21.15.18:0/1143514706 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fd25c06ec10 0x7fd25c0710c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:05:13.416 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:13.419+0000 7fd275400700 1 --2- 172.21.15.18:0/1143514706 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fd25c06ec10 0x7fd25c0710c0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fd278110cd0 tx=0x7fd26400b040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:05:13.419 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:13.422+0000 7fd26e000700 1 -- 172.21.15.18:0/1143514706 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7fd26805ce30 con 0x7fd278104e60 2024-04-12T21:05:13.488 INFO:journalctl@ceph.osd.3.smithi186.stdout:Apr 12 21:05:13 smithi186 systemd[1]: Stopping Ceph osd.3 for 1cbe407a-f90d-11ee-bc8d-c7b262605968... 2024-04-12T21:05:13.661 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:13.663+0000 7fd277200700 1 -- 172.21.15.18:0/1143514706 --> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fd27804bed0 con 0x7fd25c06ec10 2024-04-12T21:05:13.676 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:13.679+0000 7fd26e000700 1 -- 172.21.15.18:0/1143514706 <== mgr.34102 v2:172.21.15.18:6800/3812891919 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7fd27804bed0 con 0x7fd25c06ec10 2024-04-12T21:05:13.677 INFO:teuthology.orchestra.run.smithi018.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-12T21:05:13.677 INFO:teuthology.orchestra.run.smithi018.stdout:alertmanager.smithi018 smithi018 *:9093,9094 running (5m) 15s ago 20m 14.9M - 0.25.0 c8568f914cd2 c332090068e8 2024-04-12T21:05:13.678 INFO:teuthology.orchestra.run.smithi018.stdout:crash.smithi018 smithi018 running (2m) 15s ago 20m 6665k - 19.0.0-2879-g324baf8d 8717720dc5e5 b7f80d1f7982 2024-04-12T21:05:13.678 INFO:teuthology.orchestra.run.smithi018.stdout:crash.smithi186 smithi186 running (2m) 116s ago 19m 6660k - 19.0.0-2879-g324baf8d 8717720dc5e5 a4f55c81b623 2024-04-12T21:05:13.678 INFO:teuthology.orchestra.run.smithi018.stdout:grafana.smithi018 smithi018 *:3000 running (5m) 15s ago 19m 74.5M - 9.4.12 2bacad6d85d8 b08e3bf176dd 2024-04-12T21:05:13.678 INFO:teuthology.orchestra.run.smithi018.stdout:mds.cephfs.smithi018.acqbsc smithi018 running (13m) 15s ago 13m 112M - 17.2.7-827-gd3b5060a 21de9b7369ad 5b5fce65f18a 2024-04-12T21:05:13.678 INFO:teuthology.orchestra.run.smithi018.stdout:mds.cephfs.smithi018.szjluo smithi018 running (13m) 15s ago 13m 113M - 17.2.7-827-gd3b5060a 21de9b7369ad cf9bf0c5031f 2024-04-12T21:05:13.678 INFO:teuthology.orchestra.run.smithi018.stdout:mds.cephfs.smithi186.jdllyn smithi186 running (13m) 116s ago 13m 14.7M - 17.2.7-827-gd3b5060a 21de9b7369ad 2b2bc46bd29a 2024-04-12T21:05:13.678 INFO:teuthology.orchestra.run.smithi018.stdout:mds.cephfs.smithi186.wjnafs smithi186 running (13m) 116s ago 12m 14.4M - 17.2.7-827-gd3b5060a 21de9b7369ad 74b115d6d6fd 2024-04-12T21:05:13.678 INFO:teuthology.orchestra.run.smithi018.stdout:mgr.smithi018.xjzwga smithi018 *:8443,9283,8765 running (8m) 15s ago 22m 505M - 19.0.0-2879-g324baf8d 8717720dc5e5 56088305eab7 2024-04-12T21:05:13.678 INFO:teuthology.orchestra.run.smithi018.stdout:mgr.smithi186.kqcfxk smithi186 *:8443,9283,8765 running (7m) 116s ago 19m 450M - 19.0.0-2879-g324baf8d 8717720dc5e5 09cd766e5e0d 2024-04-12T21:05:13.678 INFO:teuthology.orchestra.run.smithi018.stdout:mon.smithi018 smithi018 running (3m) 15s ago 22m 50.8M 2048M 19.0.0-2879-g324baf8d 8717720dc5e5 df3e201dfed6 2024-04-12T21:05:13.678 INFO:teuthology.orchestra.run.smithi018.stdout:mon.smithi186 smithi186 running (3m) 116s ago 18m 31.6M 2048M 19.0.0-2879-g324baf8d 8717720dc5e5 c77498da9b2b 2024-04-12T21:05:13.678 INFO:teuthology.orchestra.run.smithi018.stdout:node-exporter.smithi018 smithi018 *:9100 running (6m) 15s ago 20m 7818k - 1.5.0 0da6a335fe13 326c3d9c6847 2024-04-12T21:05:13.678 INFO:teuthology.orchestra.run.smithi018.stdout:node-exporter.smithi186 smithi186 *:9100 running (6m) 116s ago 19m 7818k - 1.5.0 0da6a335fe13 ec2112baeecf 2024-04-12T21:05:13.679 INFO:teuthology.orchestra.run.smithi018.stdout:osd.0 smithi018 running (90s) 15s ago 17m 117M 1559M 19.0.0-2879-g324baf8d 8717720dc5e5 60b1273ca047 2024-04-12T21:05:13.679 INFO:teuthology.orchestra.run.smithi018.stdout:osd.1 smithi018 running (57s) 15s ago 17m 88.0M 1559M 19.0.0-2879-g324baf8d 8717720dc5e5 170ce1e5fb8d 2024-04-12T21:05:13.679 INFO:teuthology.orchestra.run.smithi018.stdout:osd.2 smithi018 running (24s) 15s ago 16m 11.2M 1559M 19.0.0-2879-g324baf8d 8717720dc5e5 c2cbc6aa2839 2024-04-12T21:05:13.679 INFO:teuthology.orchestra.run.smithi018.stdout:osd.3 smithi186 running (16m) 116s ago 16m 264M 2581M 17.2.7-827-gd3b5060a 21de9b7369ad c2def7a90d76 2024-04-12T21:05:13.679 INFO:teuthology.orchestra.run.smithi018.stdout:osd.4 smithi186 running (15m) 116s ago 15m 233M 2581M 17.2.7-827-gd3b5060a 21de9b7369ad 757e7c71b7b9 2024-04-12T21:05:13.679 INFO:teuthology.orchestra.run.smithi018.stdout:osd.5 smithi186 running (15m) 116s ago 15m 228M 2581M 17.2.7-827-gd3b5060a 21de9b7369ad 2ee072a436d0 2024-04-12T21:05:13.679 INFO:teuthology.orchestra.run.smithi018.stdout:prometheus.smithi018 smithi018 *:9095 running (6m) 15s ago 19m 60.0M - 2.43.0 a07b618ecd1d f58841127e21 2024-04-12T21:05:13.680 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:13.682+0000 7fd277200700 1 -- 172.21.15.18:0/1143514706 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fd25c06ec10 msgr2=0x7fd25c0710c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:05:13.680 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:13.683+0000 7fd277200700 1 --2- 172.21.15.18:0/1143514706 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fd25c06ec10 0x7fd25c0710c0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fd278110cd0 tx=0x7fd26400b040 comp rx=0 tx=0).stop 2024-04-12T21:05:13.680 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:13.683+0000 7fd277200700 1 -- 172.21.15.18:0/1143514706 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd278104e60 msgr2=0x7fd278115b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:05:13.680 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:13.683+0000 7fd277200700 1 --2- 172.21.15.18:0/1143514706 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd278104e60 0x7fd278115b90 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fd2680090d0 tx=0x7fd268009e30 comp rx=0 tx=0).stop 2024-04-12T21:05:13.680 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:13.683+0000 7fd277200700 1 -- 172.21.15.18:0/1143514706 shutdown_connections 2024-04-12T21:05:13.680 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:13.683+0000 7fd277200700 1 --2- 172.21.15.18:0/1143514706 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fd25c06ec10 0x7fd25c0710c0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:05:13.680 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:13.683+0000 7fd277200700 1 --2- 172.21.15.18:0/1143514706 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd278104e60 0x7fd278115b90 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:05:13.680 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:13.683+0000 7fd277200700 1 --2- 172.21.15.18:0/1143514706 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd278105800 0x7fd2781160d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:05:13.681 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:13.683+0000 7fd277200700 1 -- 172.21.15.18:0/1143514706 >> 172.21.15.18:0/1143514706 conn(0x7fd2780fab10 msgr2=0x7fd2780fca50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:05:13.681 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:13.683+0000 7fd277200700 1 -- 172.21.15.18:0/1143514706 shutdown_connections 2024-04-12T21:05:13.681 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:13.683+0000 7fd277200700 1 -- 172.21.15.18:0/1143514706 wait complete. 2024-04-12T21:05:13.779 INFO:journalctl@ceph.osd.3.smithi186.stdout:Apr 12 21:05:13 smithi186 ceph-1cbe407a-f90d-11ee-bc8d-c7b262605968-osd-3[42045]: 2024-04-12T21:05:13.489+0000 7f541ba00700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.3 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-12T21:05:13.780 INFO:journalctl@ceph.osd.3.smithi186.stdout:Apr 12 21:05:13 smithi186 ceph-1cbe407a-f90d-11ee-bc8d-c7b262605968-osd-3[42045]: 2024-04-12T21:05:13.490+0000 7f541ba00700 -1 osd.3 83 *** Got signal Terminated *** 2024-04-12T21:05:13.780 INFO:journalctl@ceph.osd.3.smithi186.stdout:Apr 12 21:05:13 smithi186 ceph-1cbe407a-f90d-11ee-bc8d-c7b262605968-osd-3[42045]: 2024-04-12T21:05:13.490+0000 7f541ba00700 -1 osd.3 83 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-12T21:05:13.838 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:13.841+0000 7f6e6e800700 1 -- 172.21.15.18:0/4185502889 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6e681059a0 msgr2=0x7f6e680ff010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:05:13.839 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:13.841+0000 7f6e6e800700 1 --2- 172.21.15.18:0/4185502889 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6e681059a0 0x7f6e680ff010 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f6e54004940 tx=0x7f6e54009e30 comp rx=0 tx=0).stop 2024-04-12T21:05:13.839 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:13.841+0000 7f6e6e800700 1 -- 172.21.15.18:0/4185502889 shutdown_connections 2024-04-12T21:05:13.839 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:13.841+0000 7f6e6e800700 1 --2- 172.21.15.18:0/4185502889 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6e681059a0 0x7f6e680ff010 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:05:13.839 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:13.841+0000 7f6e6e800700 1 --2- 172.21.15.18:0/4185502889 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6e68105000 0x7f6e681053d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:05:13.839 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:13.841+0000 7f6e6e800700 1 -- 172.21.15.18:0/4185502889 >> 172.21.15.18:0/4185502889 conn(0x7f6e680fab70 msgr2=0x7f6e680fcf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:05:13.839 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:13.841+0000 7f6e6e800700 1 -- 172.21.15.18:0/4185502889 shutdown_connections 2024-04-12T21:05:13.839 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:13.841+0000 7f6e6e800700 1 -- 172.21.15.18:0/4185502889 wait complete. 2024-04-12T21:05:13.840 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:13.842+0000 7f6e6e800700 1 Processor -- start 2024-04-12T21:05:13.840 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:13.842+0000 7f6e6e800700 1 -- start start 2024-04-12T21:05:13.841 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:13.842+0000 7f6e6e800700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6e68105000 0x7f6e68115b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:05:13.841 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:13.842+0000 7f6e6e800700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6e681059a0 0x7f6e681160b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:05:13.841 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:13.842+0000 7f6e6e800700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e68116790 con 0x7f6e68105000 2024-04-12T21:05:13.841 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:13.842+0000 7f6e6e800700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e6810fce0 con 0x7f6e681059a0 2024-04-12T21:05:13.841 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:13.843+0000 7f6e67400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6e68105000 0x7f6e68115b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:05:13.841 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:13.843+0000 7f6e67400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6e68105000 0x7f6e68115b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:46820/0 (socket says 172.21.15.18:46820) 2024-04-12T21:05:13.841 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:13.843+0000 7f6e67400700 1 -- 172.21.15.18:0/1410912995 learned_addr learned my addr 172.21.15.18:0/1410912995 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:05:13.841 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:13.843+0000 7f6e66a00700 1 --2- 172.21.15.18:0/1410912995 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6e681059a0 0x7f6e681160b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:05:13.841 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:13.843+0000 7f6e67400700 1 -- 172.21.15.18:0/1410912995 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6e681059a0 msgr2=0x7f6e681160b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:05:13.841 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:13.843+0000 7f6e67400700 1 --2- 172.21.15.18:0/1410912995 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6e681059a0 0x7f6e681160b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:05:13.841 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:13.843+0000 7f6e67400700 1 -- 172.21.15.18:0/1410912995 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e54005040 con 0x7f6e68105000 2024-04-12T21:05:13.841 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:13.844+0000 7f6e67400700 1 --2- 172.21.15.18:0/1410912995 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6e68105000 0x7f6e68115b70 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f6e5800b790 tx=0x7f6e58009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:05:13.842 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:13.844+0000 7f6e5f400700 1 -- 172.21.15.18:0/1410912995 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6e5800f770 con 0x7f6e68105000 2024-04-12T21:05:13.842 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:13.844+0000 7f6e5f400700 1 -- 172.21.15.18:0/1410912995 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f6e5800fda0 con 0x7f6e68105000 2024-04-12T21:05:13.842 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:13.844+0000 7f6e5f400700 1 -- 172.21.15.18:0/1410912995 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6e5800d310 con 0x7f6e68105000 2024-04-12T21:05:13.842 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:13.844+0000 7f6e6e800700 1 -- 172.21.15.18:0/1410912995 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6e6810ff60 con 0x7f6e68105000 2024-04-12T21:05:13.843 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:13.844+0000 7f6e6e800700 1 -- 172.21.15.18:0/1410912995 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6e68110520 con 0x7f6e68105000 2024-04-12T21:05:13.843 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:13.845+0000 7f6e6e800700 1 -- 172.21.15.18:0/1410912995 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6e68100750 con 0x7f6e68105000 2024-04-12T21:05:13.843 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:13.846+0000 7f6e5f400700 1 -- 172.21.15.18:0/1410912995 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7f6e58010420 con 0x7f6e68105000 2024-04-12T21:05:13.844 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:13.847+0000 7f6e5f400700 1 --2- 172.21.15.18:0/1410912995 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f6e4c06ec00 0x7f6e4c0710b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:05:13.844 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:13.847+0000 7f6e5f400700 1 -- 172.21.15.18:0/1410912995 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6585+0+0 (secure 0 0 0) 0x7f6e5808e830 con 0x7f6e68105000 2024-04-12T21:05:13.845 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:13.848+0000 7f6e66a00700 1 --2- 172.21.15.18:0/1410912995 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f6e4c06ec00 0x7f6e4c0710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:05:13.845 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:13.849+0000 7f6e66a00700 1 --2- 172.21.15.18:0/1410912995 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f6e4c06ec00 0x7f6e4c0710b0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f6e68111410 tx=0x7f6e54009e30 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:05:13.849 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:13.853+0000 7f6e5f400700 1 -- 172.21.15.18:0/1410912995 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f6e5805b1d0 con 0x7f6e68105000 2024-04-12T21:05:14.100 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:05:13 smithi018 ceph-mon[112590]: from='client.34312 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:05:14.100 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:05:13 smithi018 ceph-mon[112590]: from='client.34316 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:05:14.101 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:05:13 smithi018 ceph-mon[112590]: osd.3 marked itself down and dead 2024-04-12T21:05:14.182 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:14.185+0000 7f6e6e800700 1 -- 172.21.15.18:0/1410912995 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f6e681111f0 con 0x7f6e68105000 2024-04-12T21:05:14.183 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:14.186+0000 7f6e5f400700 1 -- 172.21.15.18:0/1410912995 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+839 (secure 0 0 0) 0x7f6e5805ad60 con 0x7f6e68105000 2024-04-12T21:05:14.183 INFO:teuthology.orchestra.run.smithi018.stdout:{ 2024-04-12T21:05:14.183 INFO:teuthology.orchestra.run.smithi018.stdout: "mon": { 2024-04-12T21:05:14.183 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 19.0.0-2879-g324baf8d (324baf8ddb91dea46cac50cb535c306115791355) squid (dev)": 2 2024-04-12T21:05:14.183 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-12T21:05:14.183 INFO:teuthology.orchestra.run.smithi018.stdout: "mgr": { 2024-04-12T21:05:14.183 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 19.0.0-2879-g324baf8d (324baf8ddb91dea46cac50cb535c306115791355) squid (dev)": 2 2024-04-12T21:05:14.183 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-12T21:05:14.184 INFO:teuthology.orchestra.run.smithi018.stdout: "osd": { 2024-04-12T21:05:14.184 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 17.2.7-827-gd3b5060a (d3b5060a17449edfae9385cdf792afec109fca5d) quincy (stable)": 2, 2024-04-12T21:05:14.184 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 19.0.0-2879-g324baf8d (324baf8ddb91dea46cac50cb535c306115791355) squid (dev)": 3 2024-04-12T21:05:14.184 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-12T21:05:14.184 INFO:teuthology.orchestra.run.smithi018.stdout: "mds": { 2024-04-12T21:05:14.184 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 17.2.7-827-gd3b5060a (d3b5060a17449edfae9385cdf792afec109fca5d) quincy (stable)": 4 2024-04-12T21:05:14.184 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-12T21:05:14.184 INFO:teuthology.orchestra.run.smithi018.stdout: "overall": { 2024-04-12T21:05:14.184 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 17.2.7-827-gd3b5060a (d3b5060a17449edfae9385cdf792afec109fca5d) quincy (stable)": 6, 2024-04-12T21:05:14.184 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 19.0.0-2879-g324baf8d (324baf8ddb91dea46cac50cb535c306115791355) squid (dev)": 7 2024-04-12T21:05:14.184 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-04-12T21:05:14.184 INFO:teuthology.orchestra.run.smithi018.stdout:} 2024-04-12T21:05:14.186 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:14.189+0000 7f6e6e800700 1 -- 172.21.15.18:0/1410912995 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f6e4c06ec00 msgr2=0x7f6e4c0710b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:05:14.186 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:14.189+0000 7f6e6e800700 1 --2- 172.21.15.18:0/1410912995 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f6e4c06ec00 0x7f6e4c0710b0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f6e68111410 tx=0x7f6e54009e30 comp rx=0 tx=0).stop 2024-04-12T21:05:14.186 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:14.189+0000 7f6e6e800700 1 -- 172.21.15.18:0/1410912995 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6e68105000 msgr2=0x7f6e68115b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:05:14.186 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:14.189+0000 7f6e6e800700 1 --2- 172.21.15.18:0/1410912995 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6e68105000 0x7f6e68115b70 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f6e5800b790 tx=0x7f6e58009250 comp rx=0 tx=0).stop 2024-04-12T21:05:14.187 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:14.189+0000 7f6e6e800700 1 -- 172.21.15.18:0/1410912995 shutdown_connections 2024-04-12T21:05:14.187 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:14.189+0000 7f6e6e800700 1 --2- 172.21.15.18:0/1410912995 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f6e4c06ec00 0x7f6e4c0710b0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:05:14.187 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:14.189+0000 7f6e6e800700 1 --2- 172.21.15.18:0/1410912995 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6e68105000 0x7f6e68115b70 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:05:14.187 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:14.189+0000 7f6e6e800700 1 --2- 172.21.15.18:0/1410912995 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6e681059a0 0x7f6e681160b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:05:14.187 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:14.189+0000 7f6e6e800700 1 -- 172.21.15.18:0/1410912995 >> 172.21.15.18:0/1410912995 conn(0x7f6e680fab70 msgr2=0x7f6e680fbb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:05:14.187 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:14.189+0000 7f6e6e800700 1 -- 172.21.15.18:0/1410912995 shutdown_connections 2024-04-12T21:05:14.187 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:14.190+0000 7f6e6e800700 1 -- 172.21.15.18:0/1410912995 wait complete. 2024-04-12T21:05:14.280 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:05:13 smithi186 ceph-mon[86379]: from='client.34312 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:05:14.280 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:05:13 smithi186 ceph-mon[86379]: from='client.34316 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:05:14.280 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:05:13 smithi186 ceph-mon[86379]: osd.3 marked itself down and dead 2024-04-12T21:05:14.339 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:14.342+0000 7f0f2f600700 1 -- 172.21.15.18:0/687795447 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0f28109dd0 msgr2=0x7f0f28103440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:05:14.339 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:14.342+0000 7f0f2f600700 1 --2- 172.21.15.18:0/687795447 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0f28109dd0 0x7f0f28103440 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f0f14004940 tx=0x7f0f14009e30 comp rx=0 tx=0).stop 2024-04-12T21:05:14.339 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:14.342+0000 7f0f2f600700 1 -- 172.21.15.18:0/687795447 shutdown_connections 2024-04-12T21:05:14.339 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:14.342+0000 7f0f2f600700 1 --2- 172.21.15.18:0/687795447 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0f28109dd0 0x7f0f28103440 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:05:14.339 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:14.342+0000 7f0f2f600700 1 --2- 172.21.15.18:0/687795447 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0f28109430 0x7f0f28109800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:05:14.340 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:14.342+0000 7f0f2f600700 1 -- 172.21.15.18:0/687795447 >> 172.21.15.18:0/687795447 conn(0x7f0f28072c00 msgr2=0x7f0f28075010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:05:14.340 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:14.342+0000 7f0f2f600700 1 -- 172.21.15.18:0/687795447 shutdown_connections 2024-04-12T21:05:14.340 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:14.342+0000 7f0f2f600700 1 -- 172.21.15.18:0/687795447 wait complete. 2024-04-12T21:05:14.341 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:14.343+0000 7f0f2f600700 1 Processor -- start 2024-04-12T21:05:14.341 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:14.343+0000 7f0f2f600700 1 -- start start 2024-04-12T21:05:14.341 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:14.343+0000 7f0f2f600700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0f28109430 0x7f0f28119f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:05:14.341 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:14.343+0000 7f0f2f600700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0f28109dd0 0x7f0f2811a4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:05:14.341 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:14.343+0000 7f0f2f600700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f2811ab10 con 0x7f0f28109dd0 2024-04-12T21:05:14.341 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:14.343+0000 7f0f2f600700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f281140f0 con 0x7f0f28109430 2024-04-12T21:05:14.342 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:14.344+0000 7f0f27e00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0f28109dd0 0x7f0f2811a4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:05:14.342 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:14.344+0000 7f0f27e00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0f28109dd0 0x7f0f2811a4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:46838/0 (socket says 172.21.15.18:46838) 2024-04-12T21:05:14.342 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:14.344+0000 7f0f27e00700 1 -- 172.21.15.18:0/1672156783 learned_addr learned my addr 172.21.15.18:0/1672156783 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:05:14.342 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:14.344+0000 7f0f2cc00700 1 --2- 172.21.15.18:0/1672156783 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0f28109430 0x7f0f28119f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:05:14.342 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:14.344+0000 7f0f27e00700 1 -- 172.21.15.18:0/1672156783 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0f28109430 msgr2=0x7f0f28119f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:05:14.342 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:14.344+0000 7f0f27e00700 1 --2- 172.21.15.18:0/1672156783 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0f28109430 0x7f0f28119f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:05:14.342 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:14.344+0000 7f0f27e00700 1 -- 172.21.15.18:0/1672156783 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f14005040 con 0x7f0f28109dd0 2024-04-12T21:05:14.342 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:14.344+0000 7f0f2cc00700 1 --2- 172.21.15.18:0/1672156783 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0f28109430 0x7f0f28119f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-12T21:05:14.342 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:14.344+0000 7f0f27e00700 1 --2- 172.21.15.18:0/1672156783 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0f28109dd0 0x7f0f2811a4c0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f0f14009100 tx=0x7f0f14009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:05:14.343 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:14.345+0000 7f0f24c00700 1 -- 172.21.15.18:0/1672156783 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0f1400e040 con 0x7f0f28109dd0 2024-04-12T21:05:14.343 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:14.345+0000 7f0f24c00700 1 -- 172.21.15.18:0/1672156783 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f0f14013420 con 0x7f0f28109dd0 2024-04-12T21:05:14.343 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:14.345+0000 7f0f24c00700 1 -- 172.21.15.18:0/1672156783 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0f1401c610 con 0x7f0f28109dd0 2024-04-12T21:05:14.343 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:14.345+0000 7f0f2f600700 1 -- 172.21.15.18:0/1672156783 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0f28114370 con 0x7f0f28109dd0 2024-04-12T21:05:14.343 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:14.345+0000 7f0f2f600700 1 -- 172.21.15.18:0/1672156783 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0f28114930 con 0x7f0f28109dd0 2024-04-12T21:05:14.343 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:14.346+0000 7f0f2f600700 1 -- 172.21.15.18:0/1672156783 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0f28104b80 con 0x7f0f28109dd0 2024-04-12T21:05:14.344 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:14.347+0000 7f0f24c00700 1 -- 172.21.15.18:0/1672156783 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7f0f14013590 con 0x7f0f28109dd0 2024-04-12T21:05:14.345 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:14.348+0000 7f0f24c00700 1 --2- 172.21.15.18:0/1672156783 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f0f1006ec00 0x7f0f100710b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:05:14.346 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:14.348+0000 7f0f24c00700 1 -- 172.21.15.18:0/1672156783 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6585+0+0 (secure 0 0 0) 0x7f0f1408fb40 con 0x7f0f28109dd0 2024-04-12T21:05:14.346 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:14.348+0000 7f0f2cc00700 1 --2- 172.21.15.18:0/1672156783 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f0f1006ec00 0x7f0f100710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:05:14.346 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:14.349+0000 7f0f2cc00700 1 --2- 172.21.15.18:0/1672156783 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f0f1006ec00 0x7f0f100710b0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f0f1c0041d0 tx=0x7f0f1c009040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:05:14.349 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:14.352+0000 7f0f24c00700 1 -- 172.21.15.18:0/1672156783 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f0f1405c420 con 0x7f0f28109dd0 2024-04-12T21:05:14.627 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:14.629+0000 7f0f2f600700 1 -- 172.21.15.18:0/1672156783 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f0f28110500 con 0x7f0f28109dd0 2024-04-12T21:05:14.627 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:14.631+0000 7f0f24c00700 1 -- 172.21.15.18:0/1672156783 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 23 v23) v1 ==== 76+0+1886 (secure 0 0 0) 0x7f0f1405bfb0 con 0x7f0f28109dd0 2024-04-12T21:05:14.629 INFO:teuthology.orchestra.run.smithi018.stdout:e23 2024-04-12T21:05:14.629 INFO:teuthology.orchestra.run.smithi018.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-12T21:05:14.629 INFO:teuthology.orchestra.run.smithi018.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-12T21:05:14.629 INFO:teuthology.orchestra.run.smithi018.stdout:legacy client fscid: 1 2024-04-12T21:05:14.629 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-12T21:05:14.629 INFO:teuthology.orchestra.run.smithi018.stdout:Filesystem 'cephfs' (1) 2024-04-12T21:05:14.629 INFO:teuthology.orchestra.run.smithi018.stdout:fs_name cephfs 2024-04-12T21:05:14.630 INFO:teuthology.orchestra.run.smithi018.stdout:epoch 23 2024-04-12T21:05:14.630 INFO:teuthology.orchestra.run.smithi018.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-04-12T21:05:14.630 INFO:teuthology.orchestra.run.smithi018.stdout:created 2024-04-12T20:51:57.362850+0000 2024-04-12T21:05:14.630 INFO:teuthology.orchestra.run.smithi018.stdout:modified 2024-04-12T21:01:10.670287+0000 2024-04-12T21:05:14.630 INFO:teuthology.orchestra.run.smithi018.stdout:tableserver 0 2024-04-12T21:05:14.630 INFO:teuthology.orchestra.run.smithi018.stdout:root 0 2024-04-12T21:05:14.630 INFO:teuthology.orchestra.run.smithi018.stdout:session_timeout 60 2024-04-12T21:05:14.630 INFO:teuthology.orchestra.run.smithi018.stdout:session_autoclose 300 2024-04-12T21:05:14.630 INFO:teuthology.orchestra.run.smithi018.stdout:max_file_size 1099511627776 2024-04-12T21:05:14.630 INFO:teuthology.orchestra.run.smithi018.stdout:max_xattr_size 65536 2024-04-12T21:05:14.630 INFO:teuthology.orchestra.run.smithi018.stdout:required_client_features {} 2024-04-12T21:05:14.630 INFO:teuthology.orchestra.run.smithi018.stdout:last_failure 0 2024-04-12T21:05:14.630 INFO:teuthology.orchestra.run.smithi018.stdout:last_failure_osd_epoch 46 2024-04-12T21:05:14.630 INFO:teuthology.orchestra.run.smithi018.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-12T21:05:14.630 INFO:teuthology.orchestra.run.smithi018.stdout:max_mds 2 2024-04-12T21:05:14.631 INFO:teuthology.orchestra.run.smithi018.stdout:in 0,1 2024-04-12T21:05:14.631 INFO:teuthology.orchestra.run.smithi018.stdout:up {0=14452,1=14464} 2024-04-12T21:05:14.631 INFO:teuthology.orchestra.run.smithi018.stdout:failed 2024-04-12T21:05:14.631 INFO:teuthology.orchestra.run.smithi018.stdout:damaged 2024-04-12T21:05:14.631 INFO:teuthology.orchestra.run.smithi018.stdout:stopped 2024-04-12T21:05:14.631 INFO:teuthology.orchestra.run.smithi018.stdout:data_pools [3] 2024-04-12T21:05:14.631 INFO:teuthology.orchestra.run.smithi018.stdout:metadata_pool 2 2024-04-12T21:05:14.631 INFO:teuthology.orchestra.run.smithi018.stdout:inline_data disabled 2024-04-12T21:05:14.631 INFO:teuthology.orchestra.run.smithi018.stdout:balancer 2024-04-12T21:05:14.631 INFO:teuthology.orchestra.run.smithi018.stdout:bal_rank_mask -1 2024-04-12T21:05:14.631 INFO:teuthology.orchestra.run.smithi018.stdout:standby_count_wanted 1 2024-04-12T21:05:14.631 INFO:teuthology.orchestra.run.smithi018.stdout:qdb_cluster leader: 0 members: 2024-04-12T21:05:14.631 INFO:teuthology.orchestra.run.smithi018.stdout:[mds.cephfs.smithi018.acqbsc{0:14452} state up:active seq 5 join_fscid=1 addr [v2:172.21.15.18:6826/1564065434,v1:172.21.15.18:6827/1564065434] compat {c=[1],r=[1],i=[7ff]}] 2024-04-12T21:05:14.631 INFO:teuthology.orchestra.run.smithi018.stdout:[mds.cephfs.smithi018.szjluo{1:14464} state up:active seq 6 join_fscid=1 addr [v2:172.21.15.18:6828/722443353,v1:172.21.15.18:6829/722443353] compat {c=[1],r=[1],i=[7ff]}] 2024-04-12T21:05:14.632 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-12T21:05:14.632 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-12T21:05:14.632 INFO:teuthology.orchestra.run.smithi018.stdout:Standby daemons: 2024-04-12T21:05:14.632 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-12T21:05:14.632 INFO:teuthology.orchestra.run.smithi018.stdout:[mds.cephfs.smithi186.wjnafs{-1:24277} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.186:6826/438347207,v1:172.21.15.186:6827/438347207] compat {c=[1],r=[1],i=[7ff]}] 2024-04-12T21:05:14.632 INFO:teuthology.orchestra.run.smithi018.stdout:[mds.cephfs.smithi186.jdllyn{-1:24281} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.186:6824/1464359952,v1:172.21.15.186:6825/1464359952] compat {c=[1],r=[1],i=[7ff]}] 2024-04-12T21:05:14.633 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:14.633+0000 7f0f2f600700 1 -- 172.21.15.18:0/1672156783 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f0f1006ec00 msgr2=0x7f0f100710b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:05:14.633 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:14.633+0000 7f0f2f600700 1 --2- 172.21.15.18:0/1672156783 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f0f1006ec00 0x7f0f100710b0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f0f1c0041d0 tx=0x7f0f1c009040 comp rx=0 tx=0).stop 2024-04-12T21:05:14.633 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:14.633+0000 7f0f2f600700 1 -- 172.21.15.18:0/1672156783 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0f28109dd0 msgr2=0x7f0f2811a4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:05:14.633 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:14.633+0000 7f0f2f600700 1 --2- 172.21.15.18:0/1672156783 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0f28109dd0 0x7f0f2811a4c0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f0f14009100 tx=0x7f0f14009e30 comp rx=0 tx=0).stop 2024-04-12T21:05:14.633 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:14.633+0000 7f0f2f600700 1 -- 172.21.15.18:0/1672156783 shutdown_connections 2024-04-12T21:05:14.633 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:14.633+0000 7f0f2f600700 1 --2- 172.21.15.18:0/1672156783 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f0f1006ec00 0x7f0f100710b0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:05:14.633 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:14.633+0000 7f0f2f600700 1 --2- 172.21.15.18:0/1672156783 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0f28109430 0x7f0f28119f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:05:14.633 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:14.633+0000 7f0f2f600700 1 --2- 172.21.15.18:0/1672156783 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0f28109dd0 0x7f0f2811a4c0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:05:14.633 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:14.633+0000 7f0f2f600700 1 -- 172.21.15.18:0/1672156783 >> 172.21.15.18:0/1672156783 conn(0x7f0f28072c00 msgr2=0x7f0f28074c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:05:14.633 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:14.634+0000 7f0f2f600700 1 -- 172.21.15.18:0/1672156783 shutdown_connections 2024-04-12T21:05:14.633 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:14.634+0000 7f0f2f600700 1 -- 172.21.15.18:0/1672156783 wait complete. 2024-04-12T21:05:14.633 INFO:teuthology.orchestra.run.smithi018.stderr:dumped fsmap epoch 23 2024-04-12T21:05:14.731 INFO:journalctl@ceph.osd.3.smithi186.stdout:Apr 12 21:05:14 smithi186 podman[94971]: 2024-04-12 21:05:14.468035105 +0000 UTC m=+1.123819754 container died c2def7a90d76f453e5f6a21ed79ab158bba2e35a620cd678e684de95bdbd2a31 (image=quay.ceph.io/ceph-ci/ceph@sha256:5e334df698f432180b78bc9c46936bfdfbc92bc9a8983463e523f780d23765fe, name=ceph-1cbe407a-f90d-11ee-bc8d-c7b262605968-osd-3, org.label-schema.build-date=20240409, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, RELEASE=quincy-d3b5060, io.buildah.version=1.33.5, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0) 2024-04-12T21:05:14.782 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:14.784+0000 7f1338600700 1 -- 172.21.15.18:0/2427841118 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f13301037a0 msgr2=0x7f13300fce10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:05:14.782 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:14.784+0000 7f1338600700 1 --2- 172.21.15.18:0/2427841118 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f13301037a0 0x7f13300fce10 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f131c004940 tx=0x7f131c009e30 comp rx=0 tx=0).stop 2024-04-12T21:05:14.782 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:14.785+0000 7f1338600700 1 -- 172.21.15.18:0/2427841118 shutdown_connections 2024-04-12T21:05:14.782 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:14.785+0000 7f1338600700 1 --2- 172.21.15.18:0/2427841118 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f13301037a0 0x7f13300fce10 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:05:14.783 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:14.785+0000 7f1338600700 1 --2- 172.21.15.18:0/2427841118 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1330102e00 0x7f13301031d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:05:14.783 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:14.785+0000 7f1338600700 1 -- 172.21.15.18:0/2427841118 >> 172.21.15.18:0/2427841118 conn(0x7f13300f8970 msgr2=0x7f13300fad80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:05:14.783 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:14.785+0000 7f1338600700 1 -- 172.21.15.18:0/2427841118 shutdown_connections 2024-04-12T21:05:14.783 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:14.785+0000 7f1338600700 1 -- 172.21.15.18:0/2427841118 wait complete. 2024-04-12T21:05:14.784 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:14.786+0000 7f1338600700 1 Processor -- start 2024-04-12T21:05:14.784 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:14.786+0000 7f1338600700 1 -- start start 2024-04-12T21:05:14.785 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:14.786+0000 7f1338600700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1330102e00 0x7f1330115da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:05:14.785 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:14.786+0000 7f1338600700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f13301037a0 0x7f13301162e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:05:14.785 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:14.786+0000 7f1338600700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1330116930 con 0x7f13301037a0 2024-04-12T21:05:14.785 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:14.786+0000 7f1338600700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f133010fd00 con 0x7f1330102e00 2024-04-12T21:05:14.785 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:14.786+0000 7f1335200700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f13301037a0 0x7f13301162e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:05:14.785 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:14.786+0000 7f1335200700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f13301037a0 0x7f13301162e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:46856/0 (socket says 172.21.15.18:46856) 2024-04-12T21:05:14.785 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:14.786+0000 7f1335200700 1 -- 172.21.15.18:0/1344894220 learned_addr learned my addr 172.21.15.18:0/1344894220 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:05:14.785 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:14.787+0000 7f1335c00700 1 --2- 172.21.15.18:0/1344894220 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1330102e00 0x7f1330115da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:05:14.785 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:14.787+0000 7f1335200700 1 -- 172.21.15.18:0/1344894220 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1330102e00 msgr2=0x7f1330115da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:05:14.785 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:14.787+0000 7f1335200700 1 --2- 172.21.15.18:0/1344894220 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1330102e00 0x7f1330115da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:05:14.785 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:14.787+0000 7f1335200700 1 -- 172.21.15.18:0/1344894220 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f131c005040 con 0x7f13301037a0 2024-04-12T21:05:14.785 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:14.787+0000 7f1335200700 1 --2- 172.21.15.18:0/1344894220 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f13301037a0 0x7f13301162e0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f131c009100 tx=0x7f131c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:05:14.785 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:14.787+0000 7f132d600700 1 -- 172.21.15.18:0/1344894220 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f131c00e040 con 0x7f13301037a0 2024-04-12T21:05:14.785 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:14.787+0000 7f132d600700 1 -- 172.21.15.18:0/1344894220 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f131c013420 con 0x7f13301037a0 2024-04-12T21:05:14.786 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:14.788+0000 7f132d600700 1 -- 172.21.15.18:0/1344894220 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f131c01c630 con 0x7f13301037a0 2024-04-12T21:05:14.786 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:14.788+0000 7f1338600700 1 -- 172.21.15.18:0/1344894220 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f133010ff80 con 0x7f13301037a0 2024-04-12T21:05:14.786 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:14.788+0000 7f1338600700 1 -- 172.21.15.18:0/1344894220 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1330110540 con 0x7f13301037a0 2024-04-12T21:05:14.786 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:14.789+0000 7f1338600700 1 -- 172.21.15.18:0/1344894220 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1330102120 con 0x7f13301037a0 2024-04-12T21:05:14.787 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:14.790+0000 7f132d600700 1 -- 172.21.15.18:0/1344894220 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7f131c013590 con 0x7f13301037a0 2024-04-12T21:05:14.788 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:14.791+0000 7f132d600700 1 --2- 172.21.15.18:0/1344894220 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f131806ec10 0x7f13180710c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:05:14.788 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:14.791+0000 7f132d600700 1 -- 172.21.15.18:0/1344894220 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6585+0+0 (secure 0 0 0) 0x7f131c08faf0 con 0x7f13301037a0 2024-04-12T21:05:14.788 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:14.791+0000 7f1335c00700 1 --2- 172.21.15.18:0/1344894220 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f131806ec10 0x7f13180710c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:05:14.789 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:14.792+0000 7f1335c00700 1 --2- 172.21.15.18:0/1344894220 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f131806ec10 0x7f13180710c0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f1324004d60 tx=0x7f132400b040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:05:14.793 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:14.796+0000 7f132d600700 1 -- 172.21.15.18:0/1344894220 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f131c05c3d0 con 0x7f13301037a0 2024-04-12T21:05:15.029 INFO:journalctl@ceph.osd.3.smithi186.stdout:Apr 12 21:05:14 smithi186 podman[94971]: 2024-04-12 21:05:14.860077573 +0000 UTC m=+1.515862219 container cleanup c2def7a90d76f453e5f6a21ed79ab158bba2e35a620cd678e684de95bdbd2a31 (image=quay.ceph.io/ceph-ci/ceph@sha256:5e334df698f432180b78bc9c46936bfdfbc92bc9a8983463e523f780d23765fe, name=ceph-1cbe407a-f90d-11ee-bc8d-c7b262605968-osd-3, RELEASE=quincy-d3b5060, ceph=True, org.label-schema.build-date=20240409, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-12T21:05:15.030 INFO:journalctl@ceph.osd.3.smithi186.stdout:Apr 12 21:05:14 smithi186 bash[94971]: ceph-1cbe407a-f90d-11ee-bc8d-c7b262605968-osd-3 2024-04-12T21:05:15.030 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:05:14 smithi186 ceph-mon[86379]: from='client.34320 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:05:15.030 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:05:14 smithi186 ceph-mon[86379]: pgmap v144: 65 pgs: 65 active+clean; 257 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-12T21:05:15.030 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:05:14 smithi186 ceph-mon[86379]: Health check failed: 1 osds down (OSD_DOWN) 2024-04-12T21:05:15.030 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:05:14 smithi186 ceph-mon[86379]: osdmap e84: 6 total, 5 up, 6 in 2024-04-12T21:05:15.030 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:05:14 smithi186 ceph-mon[86379]: from='client.? 172.21.15.18:0/1410912995' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T21:05:15.030 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:05:14 smithi186 ceph-mon[86379]: from='client.? 172.21.15.18:0/1672156783' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-12T21:05:15.070 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:15.073+0000 7f1338600700 1 -- 172.21.15.18:0/1344894220 --> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f13301113f0 con 0x7f131806ec10 2024-04-12T21:05:15.073 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:15.076+0000 7f132d600700 1 -- 172.21.15.18:0/1344894220 <== mgr.34102 v2:172.21.15.18:6800/3812891919 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7f13301113f0 con 0x7f131806ec10 2024-04-12T21:05:15.073 INFO:teuthology.orchestra.run.smithi018.stdout:{ 2024-04-12T21:05:15.074 INFO:teuthology.orchestra.run.smithi018.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:f4452def3be4d350f4a8e5feaf7ebb1511be9298fe68478ba7a2d6fbd7d69a63", 2024-04-12T21:05:15.074 INFO:teuthology.orchestra.run.smithi018.stdout: "in_progress": true, 2024-04-12T21:05:15.074 INFO:teuthology.orchestra.run.smithi018.stdout: "which": "Upgrading all daemon types on all hosts", 2024-04-12T21:05:15.074 INFO:teuthology.orchestra.run.smithi018.stdout: "services_complete": [ 2024-04-12T21:05:15.074 INFO:teuthology.orchestra.run.smithi018.stdout: "crash", 2024-04-12T21:05:15.074 INFO:teuthology.orchestra.run.smithi018.stdout: "mon", 2024-04-12T21:05:15.074 INFO:teuthology.orchestra.run.smithi018.stdout: "mgr" 2024-04-12T21:05:15.074 INFO:teuthology.orchestra.run.smithi018.stdout: ], 2024-04-12T21:05:15.074 INFO:teuthology.orchestra.run.smithi018.stdout: "progress": "9/21 daemons upgraded", 2024-04-12T21:05:15.074 INFO:teuthology.orchestra.run.smithi018.stdout: "message": "Currently upgrading osd daemons", 2024-04-12T21:05:15.074 INFO:teuthology.orchestra.run.smithi018.stdout: "is_paused": false 2024-04-12T21:05:15.074 INFO:teuthology.orchestra.run.smithi018.stdout:} 2024-04-12T21:05:15.076 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:15.078+0000 7f1338600700 1 -- 172.21.15.18:0/1344894220 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f131806ec10 msgr2=0x7f13180710c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:05:15.076 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:15.078+0000 7f1338600700 1 --2- 172.21.15.18:0/1344894220 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f131806ec10 0x7f13180710c0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f1324004d60 tx=0x7f132400b040 comp rx=0 tx=0).stop 2024-04-12T21:05:15.076 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:15.078+0000 7f1338600700 1 -- 172.21.15.18:0/1344894220 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f13301037a0 msgr2=0x7f13301162e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:05:15.076 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:15.078+0000 7f1338600700 1 --2- 172.21.15.18:0/1344894220 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f13301037a0 0x7f13301162e0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f131c009100 tx=0x7f131c009e30 comp rx=0 tx=0).stop 2024-04-12T21:05:15.076 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:15.079+0000 7f1338600700 1 -- 172.21.15.18:0/1344894220 shutdown_connections 2024-04-12T21:05:15.076 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:15.079+0000 7f1338600700 1 --2- 172.21.15.18:0/1344894220 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f131806ec10 0x7f13180710c0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:05:15.076 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:15.079+0000 7f1338600700 1 --2- 172.21.15.18:0/1344894220 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1330102e00 0x7f1330115da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:05:15.076 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:15.079+0000 7f1338600700 1 --2- 172.21.15.18:0/1344894220 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f13301037a0 0x7f13301162e0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:05:15.077 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:15.079+0000 7f1338600700 1 -- 172.21.15.18:0/1344894220 >> 172.21.15.18:0/1344894220 conn(0x7f13300f8970 msgr2=0x7f13300fa590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:05:15.077 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:15.079+0000 7f1338600700 1 -- 172.21.15.18:0/1344894220 shutdown_connections 2024-04-12T21:05:15.077 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:15.079+0000 7f1338600700 1 -- 172.21.15.18:0/1344894220 wait complete. 2024-04-12T21:05:15.228 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:15.231+0000 7fa277e00700 1 -- 172.21.15.18:0/3839895765 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa278065b70 msgr2=0x7fa278065fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:05:15.229 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:15.231+0000 7fa277e00700 1 --2- 172.21.15.18:0/3839895765 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa278065b70 0x7fa278065fe0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fa264004940 tx=0x7fa264009e30 comp rx=0 tx=0).stop 2024-04-12T21:05:15.229 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:15.231+0000 7fa277e00700 1 -- 172.21.15.18:0/3839895765 shutdown_connections 2024-04-12T21:05:15.229 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:15.231+0000 7fa277e00700 1 --2- 172.21.15.18:0/3839895765 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa278065b70 0x7fa278065fe0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:05:15.229 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:15.231+0000 7fa277e00700 1 --2- 172.21.15.18:0/3839895765 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa278101ae0 0x7fa278065630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:05:15.229 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:15.231+0000 7fa277e00700 1 -- 172.21.15.18:0/3839895765 >> 172.21.15.18:0/3839895765 conn(0x7fa278071b70 msgr2=0x7fa278071f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:05:15.229 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:15.231+0000 7fa277e00700 1 -- 172.21.15.18:0/3839895765 shutdown_connections 2024-04-12T21:05:15.229 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:15.231+0000 7fa277e00700 1 -- 172.21.15.18:0/3839895765 wait complete. 2024-04-12T21:05:15.229 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:05:14 smithi018 ceph-mon[112590]: from='client.34320 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:05:15.229 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:05:14 smithi018 ceph-mon[112590]: pgmap v144: 65 pgs: 65 active+clean; 257 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-12T21:05:15.230 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:05:14 smithi018 ceph-mon[112590]: Health check failed: 1 osds down (OSD_DOWN) 2024-04-12T21:05:15.230 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:05:14 smithi018 ceph-mon[112590]: osdmap e84: 6 total, 5 up, 6 in 2024-04-12T21:05:15.230 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:05:14 smithi018 ceph-mon[112590]: from='client.? 172.21.15.18:0/1410912995' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T21:05:15.230 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:05:14 smithi018 ceph-mon[112590]: from='client.? 172.21.15.18:0/1672156783' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-12T21:05:15.231 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:15.232+0000 7fa277e00700 1 Processor -- start 2024-04-12T21:05:15.231 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:15.232+0000 7fa277e00700 1 -- start start 2024-04-12T21:05:15.231 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:15.232+0000 7fa277e00700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa278065b70 0x7fa278119ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:05:15.232 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:15.232+0000 7fa277e00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa278101ae0 0x7fa27811a530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:05:15.232 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:15.232+0000 7fa277e00700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa27811ab80 con 0x7fa278101ae0 2024-04-12T21:05:15.232 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:15.232+0000 7fa277e00700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa278113f50 con 0x7fa278065b70 2024-04-12T21:05:15.232 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:15.233+0000 7fa276000700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa278101ae0 0x7fa27811a530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:05:15.232 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:15.233+0000 7fa276000700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa278101ae0 0x7fa27811a530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:46872/0 (socket says 172.21.15.18:46872) 2024-04-12T21:05:15.232 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:15.233+0000 7fa276000700 1 -- 172.21.15.18:0/3047131858 learned_addr learned my addr 172.21.15.18:0/3047131858 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:05:15.232 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:15.233+0000 7fa276a00700 1 --2- 172.21.15.18:0/3047131858 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa278065b70 0x7fa278119ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:05:15.232 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:15.233+0000 7fa276000700 1 -- 172.21.15.18:0/3047131858 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa278065b70 msgr2=0x7fa278119ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:05:15.232 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:15.233+0000 7fa276000700 1 --2- 172.21.15.18:0/3047131858 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa278065b70 0x7fa278119ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:05:15.232 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:15.233+0000 7fa276000700 1 -- 172.21.15.18:0/3047131858 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa264005040 con 0x7fa278101ae0 2024-04-12T21:05:15.232 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:15.234+0000 7fa276000700 1 --2- 172.21.15.18:0/3047131858 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa278101ae0 0x7fa27811a530 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fa264009150 tx=0x7fa264009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:05:15.232 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:15.234+0000 7fa26ea00700 1 -- 172.21.15.18:0/3047131858 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa26400e040 con 0x7fa278101ae0 2024-04-12T21:05:15.232 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:15.234+0000 7fa26ea00700 1 -- 172.21.15.18:0/3047131858 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa264013420 con 0x7fa278101ae0 2024-04-12T21:05:15.232 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:15.234+0000 7fa26ea00700 1 -- 172.21.15.18:0/3047131858 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa26401c630 con 0x7fa278101ae0 2024-04-12T21:05:15.232 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:15.234+0000 7fa277e00700 1 -- 172.21.15.18:0/3047131858 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa2781141d0 con 0x7fa278101ae0 2024-04-12T21:05:15.233 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:15.234+0000 7fa277e00700 1 -- 172.21.15.18:0/3047131858 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa278114710 con 0x7fa278101ae0 2024-04-12T21:05:15.233 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:15.235+0000 7fa277e00700 1 -- 172.21.15.18:0/3047131858 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa2780fc170 con 0x7fa278101ae0 2024-04-12T21:05:15.233 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:15.237+0000 7fa26ea00700 1 -- 172.21.15.18:0/3047131858 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7fa264013590 con 0x7fa278101ae0 2024-04-12T21:05:15.234 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:15.237+0000 7fa26ea00700 1 --2- 172.21.15.18:0/3047131858 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fa25c06ec10 0x7fa25c0710c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:05:15.235 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:15.238+0000 7fa26ea00700 1 -- 172.21.15.18:0/3047131858 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6585+0+0 (secure 0 0 0) 0x7fa26408fb60 con 0x7fa278101ae0 2024-04-12T21:05:15.235 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:15.238+0000 7fa276a00700 1 --2- 172.21.15.18:0/3047131858 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fa25c06ec10 0x7fa25c0710c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:05:15.235 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:15.238+0000 7fa276a00700 1 --2- 172.21.15.18:0/3047131858 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fa25c06ec10 0x7fa25c0710c0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fa268004990 tx=0x7fa268009ee0 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:05:15.239 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:15.242+0000 7fa26ea00700 1 -- 172.21.15.18:0/3047131858 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7fa26405c440 con 0x7fa278101ae0 2024-04-12T21:05:15.529 INFO:journalctl@ceph.osd.3.smithi186.stdout:Apr 12 21:05:15 smithi186 podman[94990]: 2024-04-12 21:05:15.069620002 +0000 UTC m=+0.583272485 container remove c2def7a90d76f453e5f6a21ed79ab158bba2e35a620cd678e684de95bdbd2a31 (image=quay.ceph.io/ceph-ci/ceph@sha256:5e334df698f432180b78bc9c46936bfdfbc92bc9a8983463e523f780d23765fe, name=ceph-1cbe407a-f90d-11ee-bc8d-c7b262605968-osd-3, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240409, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, GIT_BRANCH=HEAD, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=quincy-d3b5060) 2024-04-12T21:05:15.547 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:15.549+0000 7fa277e00700 1 -- 172.21.15.18:0/3047131858 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7fa27805f5f0 con 0x7fa278101ae0 2024-04-12T21:05:15.547 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:15.550+0000 7fa26ea00700 1 -- 172.21.15.18:0/3047131858 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+100 (secure 0 0 0) 0x7fa26405bfd0 con 0x7fa278101ae0 2024-04-12T21:05:15.547 INFO:teuthology.orchestra.run.smithi018.stdout:HEALTH_WARN 1 osds down 2024-04-12T21:05:15.547 INFO:teuthology.orchestra.run.smithi018.stdout:[WRN] OSD_DOWN: 1 osds down 2024-04-12T21:05:15.548 INFO:teuthology.orchestra.run.smithi018.stdout: osd.3 (root=default,host=smithi186) is down 2024-04-12T21:05:15.550 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:15.553+0000 7fa277e00700 1 -- 172.21.15.18:0/3047131858 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fa25c06ec10 msgr2=0x7fa25c0710c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:05:15.550 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:15.553+0000 7fa277e00700 1 --2- 172.21.15.18:0/3047131858 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fa25c06ec10 0x7fa25c0710c0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fa268004990 tx=0x7fa268009ee0 comp rx=0 tx=0).stop 2024-04-12T21:05:15.550 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:15.553+0000 7fa277e00700 1 -- 172.21.15.18:0/3047131858 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa278101ae0 msgr2=0x7fa27811a530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:05:15.551 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:15.553+0000 7fa277e00700 1 --2- 172.21.15.18:0/3047131858 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa278101ae0 0x7fa27811a530 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fa264009150 tx=0x7fa264009e30 comp rx=0 tx=0).stop 2024-04-12T21:05:15.551 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:15.553+0000 7fa277e00700 1 -- 172.21.15.18:0/3047131858 shutdown_connections 2024-04-12T21:05:15.551 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:15.553+0000 7fa277e00700 1 --2- 172.21.15.18:0/3047131858 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fa25c06ec10 0x7fa25c0710c0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:05:15.551 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:15.553+0000 7fa277e00700 1 --2- 172.21.15.18:0/3047131858 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa278065b70 0x7fa278119ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:05:15.551 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:15.553+0000 7fa277e00700 1 --2- 172.21.15.18:0/3047131858 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa278101ae0 0x7fa27811a530 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:05:15.551 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:15.553+0000 7fa277e00700 1 -- 172.21.15.18:0/3047131858 >> 172.21.15.18:0/3047131858 conn(0x7fa278071b70 msgr2=0x7fa2780f9b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:05:15.551 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:15.553+0000 7fa277e00700 1 -- 172.21.15.18:0/3047131858 shutdown_connections 2024-04-12T21:05:15.551 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:15.554+0000 7fa277e00700 1 -- 172.21.15.18:0/3047131858 wait complete. 2024-04-12T21:05:16.229 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:05:15 smithi186 ceph-mon[86379]: osdmap e85: 6 total, 5 up, 6 in 2024-04-12T21:05:16.229 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:05:15 smithi186 ceph-mon[86379]: from='client.34332 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:05:16.230 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:05:15 smithi186 ceph-mon[86379]: from='client.? 172.21.15.18:0/3047131858' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-12T21:05:16.230 INFO:journalctl@ceph.osd.3.smithi186.stdout:Apr 12 21:05:15 smithi186 podman[95076]: 2024-04-12 21:05:15.805695957 +0000 UTC m=+0.032097091 image pull 8717720dc5e5c56303818ea4d4f0a11f6e2c03dee521fb668f411f1304e0afca quay.ceph.io/ceph-ci/ceph@sha256:f4452def3be4d350f4a8e5feaf7ebb1511be9298fe68478ba7a2d6fbd7d69a63 2024-04-12T21:05:16.230 INFO:journalctl@ceph.osd.3.smithi186.stdout:Apr 12 21:05:15 smithi186 podman[95076]: 2024-04-12 21:05:15.912196185 +0000 UTC m=+0.138597344 container create 6be919bd7b4518c112e9d8f001495cd76dd6f1ce442f205442a376473c2bc54c (image=quay.ceph.io/ceph-ci/ceph@sha256:f4452def3be4d350f4a8e5feaf7ebb1511be9298fe68478ba7a2d6fbd7d69a63, name=ceph-1cbe407a-f90d-11ee-bc8d-c7b262605968-osd-3-deactivate, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , ceph=True, RELEASE=wip-vshankar-testing-20240411.061452-324baf8, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240409) 2024-04-12T21:05:16.230 INFO:journalctl@ceph.osd.3.smithi186.stdout:Apr 12 21:05:16 smithi186 podman[95076]: 2024-04-12 21:05:16.233215144 +0000 UTC m=+0.459616309 container init 6be919bd7b4518c112e9d8f001495cd76dd6f1ce442f205442a376473c2bc54c (image=quay.ceph.io/ceph-ci/ceph@sha256:f4452def3be4d350f4a8e5feaf7ebb1511be9298fe68478ba7a2d6fbd7d69a63, name=ceph-1cbe407a-f90d-11ee-bc8d-c7b262605968-osd-3-deactivate, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_CLEAN=True, RELEASE=wip-vshankar-testing-20240411.061452-324baf8, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240409, org.label-schema.license=GPLv2, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-12T21:05:16.350 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:05:15 smithi018 ceph-mon[112590]: osdmap e85: 6 total, 5 up, 6 in 2024-04-12T21:05:16.351 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:05:15 smithi018 ceph-mon[112590]: from='client.34332 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:05:16.351 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:05:15 smithi018 ceph-mon[112590]: from='client.? 172.21.15.18:0/3047131858' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-12T21:05:16.530 INFO:journalctl@ceph.osd.3.smithi186.stdout:Apr 12 21:05:16 smithi186 podman[95076]: 2024-04-12 21:05:16.244662887 +0000 UTC m=+0.471064052 container start 6be919bd7b4518c112e9d8f001495cd76dd6f1ce442f205442a376473c2bc54c (image=quay.ceph.io/ceph-ci/ceph@sha256:f4452def3be4d350f4a8e5feaf7ebb1511be9298fe68478ba7a2d6fbd7d69a63, name=ceph-1cbe407a-f90d-11ee-bc8d-c7b262605968-osd-3-deactivate, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240409, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, RELEASE=wip-vshankar-testing-20240411.061452-324baf8, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, ceph=True) 2024-04-12T21:05:16.530 INFO:journalctl@ceph.osd.3.smithi186.stdout:Apr 12 21:05:16 smithi186 podman[95076]: 2024-04-12 21:05:16.331351089 +0000 UTC m=+0.557752284 container attach 6be919bd7b4518c112e9d8f001495cd76dd6f1ce442f205442a376473c2bc54c (image=quay.ceph.io/ceph-ci/ceph@sha256:f4452def3be4d350f4a8e5feaf7ebb1511be9298fe68478ba7a2d6fbd7d69a63, name=ceph-1cbe407a-f90d-11ee-bc8d-c7b262605968-osd-3-deactivate, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, RELEASE=wip-vshankar-testing-20240411.061452-324baf8, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.build-date=20240409, ceph=True) 2024-04-12T21:05:16.976 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:05:16 smithi186 ceph-mon[86379]: pgmap v147: 65 pgs: 6 active+undersized, 7 active+undersized+degraded, 8 peering, 5 stale+active+clean, 39 active+clean; 257 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 30/324 objects degraded (9.259%) 2024-04-12T21:05:16.977 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:05:16 smithi186 ceph-mon[86379]: Health check failed: Reduced data availability: 2 pgs peering (PG_AVAILABILITY) 2024-04-12T21:05:16.977 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:05:16 smithi186 ceph-mon[86379]: Health check failed: Degraded data redundancy: 30/324 objects degraded (9.259%), 7 pgs degraded (PG_DEGRADED) 2024-04-12T21:05:16.977 INFO:journalctl@ceph.osd.3.smithi186.stdout:Apr 12 21:05:16 smithi186 podman[95076]: 2024-04-12 21:05:16.700835823 +0000 UTC m=+0.927236958 container died 6be919bd7b4518c112e9d8f001495cd76dd6f1ce442f205442a376473c2bc54c (image=quay.ceph.io/ceph-ci/ceph@sha256:f4452def3be4d350f4a8e5feaf7ebb1511be9298fe68478ba7a2d6fbd7d69a63, name=ceph-1cbe407a-f90d-11ee-bc8d-c7b262605968-osd-3-deactivate, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, RELEASE=wip-vshankar-testing-20240411.061452-324baf8, org.label-schema.build-date=20240409, ceph=True) 2024-04-12T21:05:17.350 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:05:16 smithi018 ceph-mon[112590]: pgmap v147: 65 pgs: 6 active+undersized, 7 active+undersized+degraded, 8 peering, 5 stale+active+clean, 39 active+clean; 257 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 30/324 objects degraded (9.259%) 2024-04-12T21:05:17.351 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:05:16 smithi018 ceph-mon[112590]: Health check failed: Reduced data availability: 2 pgs peering (PG_AVAILABILITY) 2024-04-12T21:05:17.351 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:05:16 smithi018 ceph-mon[112590]: Health check failed: Degraded data redundancy: 30/324 objects degraded (9.259%), 7 pgs degraded (PG_DEGRADED) 2024-04-12T21:05:17.780 INFO:journalctl@ceph.osd.3.smithi186.stdout:Apr 12 21:05:17 smithi186 podman[95102]: 2024-04-12 21:05:17.402005144 +0000 UTC m=+0.680752188 container remove 6be919bd7b4518c112e9d8f001495cd76dd6f1ce442f205442a376473c2bc54c (image=quay.ceph.io/ceph-ci/ceph@sha256:f4452def3be4d350f4a8e5feaf7ebb1511be9298fe68478ba7a2d6fbd7d69a63, name=ceph-1cbe407a-f90d-11ee-bc8d-c7b262605968-osd-3-deactivate, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240409, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, RELEASE=wip-vshankar-testing-20240411.061452-324baf8, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2) 2024-04-12T21:05:17.780 INFO:journalctl@ceph.osd.3.smithi186.stdout:Apr 12 21:05:17 smithi186 systemd[1]: ceph-1cbe407a-f90d-11ee-bc8d-c7b262605968@osd.3.service: Deactivated successfully. 2024-04-12T21:05:17.780 INFO:journalctl@ceph.osd.3.smithi186.stdout:Apr 12 21:05:17 smithi186 systemd[1]: Stopped Ceph osd.3 for 1cbe407a-f90d-11ee-bc8d-c7b262605968. 2024-04-12T21:05:17.780 INFO:journalctl@ceph.osd.3.smithi186.stdout:Apr 12 21:05:17 smithi186 systemd[1]: ceph-1cbe407a-f90d-11ee-bc8d-c7b262605968@osd.3.service: Consumed 2min 41.969s CPU time. 2024-04-12T21:05:18.204 INFO:journalctl@ceph.osd.3.smithi186.stdout:Apr 12 21:05:17 smithi186 systemd[1]: Starting Ceph osd.3 for 1cbe407a-f90d-11ee-bc8d-c7b262605968... 2024-04-12T21:05:18.530 INFO:journalctl@ceph.osd.3.smithi186.stdout:Apr 12 21:05:18 smithi186 podman[95203]: 2024-04-12 21:05:18.113884881 +0000 UTC m=+0.033784396 image pull 8717720dc5e5c56303818ea4d4f0a11f6e2c03dee521fb668f411f1304e0afca quay.ceph.io/ceph-ci/ceph@sha256:f4452def3be4d350f4a8e5feaf7ebb1511be9298fe68478ba7a2d6fbd7d69a63 2024-04-12T21:05:18.530 INFO:journalctl@ceph.osd.3.smithi186.stdout:Apr 12 21:05:18 smithi186 podman[95203]: 2024-04-12 21:05:18.278191114 +0000 UTC m=+0.198090667 container create dc952646a8b54a3a04cc3ef5b15a06d28a7a983ae22b5d7d82a7098e83354447 (image=quay.ceph.io/ceph-ci/ceph@sha256:f4452def3be4d350f4a8e5feaf7ebb1511be9298fe68478ba7a2d6fbd7d69a63, name=ceph-1cbe407a-f90d-11ee-bc8d-c7b262605968-osd-3-activate, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.build-date=20240409, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-vshankar-testing-20240411.061452-324baf8, GIT_CLEAN=True, ceph=True, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-12T21:05:18.899 INFO:journalctl@ceph.osd.3.smithi186.stdout:Apr 12 21:05:18 smithi186 podman[95203]: 2024-04-12 21:05:18.540050131 +0000 UTC m=+0.459949690 container init dc952646a8b54a3a04cc3ef5b15a06d28a7a983ae22b5d7d82a7098e83354447 (image=quay.ceph.io/ceph-ci/ceph@sha256:f4452def3be4d350f4a8e5feaf7ebb1511be9298fe68478ba7a2d6fbd7d69a63, name=ceph-1cbe407a-f90d-11ee-bc8d-c7b262605968-osd-3-activate, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.schema-version=1.0, RELEASE=wip-vshankar-testing-20240411.061452-324baf8, ceph=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240409, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-12T21:05:18.899 INFO:journalctl@ceph.osd.3.smithi186.stdout:Apr 12 21:05:18 smithi186 podman[95203]: 2024-04-12 21:05:18.552098115 +0000 UTC m=+0.471997654 container start dc952646a8b54a3a04cc3ef5b15a06d28a7a983ae22b5d7d82a7098e83354447 (image=quay.ceph.io/ceph-ci/ceph@sha256:f4452def3be4d350f4a8e5feaf7ebb1511be9298fe68478ba7a2d6fbd7d69a63, name=ceph-1cbe407a-f90d-11ee-bc8d-c7b262605968-osd-3-activate, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240409, org.label-schema.vendor=CentOS, ceph=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , RELEASE=wip-vshankar-testing-20240411.061452-324baf8, GIT_CLEAN=True, org.label-schema.license=GPLv2) 2024-04-12T21:05:18.899 INFO:journalctl@ceph.osd.3.smithi186.stdout:Apr 12 21:05:18 smithi186 podman[95203]: 2024-04-12 21:05:18.63038622 +0000 UTC m=+0.550285829 container attach dc952646a8b54a3a04cc3ef5b15a06d28a7a983ae22b5d7d82a7098e83354447 (image=quay.ceph.io/ceph-ci/ceph@sha256:f4452def3be4d350f4a8e5feaf7ebb1511be9298fe68478ba7a2d6fbd7d69a63, name=ceph-1cbe407a-f90d-11ee-bc8d-c7b262605968-osd-3-activate, CEPH_POINT_RELEASE=, RELEASE=wip-vshankar-testing-20240411.061452-324baf8, GIT_CLEAN=True, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.build-date=20240409, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-12T21:05:19.279 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:05:18 smithi186 ceph-mon[86379]: pgmap v148: 65 pgs: 14 active+undersized, 11 active+undersized+degraded, 8 peering, 32 active+clean; 257 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 40/324 objects degraded (12.346%) 2024-04-12T21:05:19.350 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:05:18 smithi018 ceph-mon[112590]: pgmap v148: 65 pgs: 14 active+undersized, 11 active+undersized+degraded, 8 peering, 32 active+clean; 257 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 40/324 objects degraded (12.346%) 2024-04-12T21:05:19.780 INFO:journalctl@ceph.osd.3.smithi186.stdout:Apr 12 21:05:19 smithi186 ceph-1cbe407a-f90d-11ee-bc8d-c7b262605968-osd-3-activate[95218]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-04-12T21:05:19.781 INFO:journalctl@ceph.osd.3.smithi186.stdout:Apr 12 21:05:19 smithi186 ceph-1cbe407a-f90d-11ee-bc8d-c7b262605968-osd-3-activate[95218]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-3 --no-mon-config --dev /dev/mapper/vg_nvme-lv_4 2024-04-12T21:05:19.781 INFO:journalctl@ceph.osd.3.smithi186.stdout:Apr 12 21:05:19 smithi186 bash[95203]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-04-12T21:05:19.781 INFO:journalctl@ceph.osd.3.smithi186.stdout:Apr 12 21:05:19 smithi186 bash[95203]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-3 --no-mon-config --dev /dev/mapper/vg_nvme-lv_4 2024-04-12T21:05:19.781 INFO:journalctl@ceph.osd.3.smithi186.stdout:Apr 12 21:05:19 smithi186 bash[95203]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_4 2024-04-12T21:05:19.781 INFO:journalctl@ceph.osd.3.smithi186.stdout:Apr 12 21:05:19 smithi186 bash[95203]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-3 2024-04-12T21:05:19.781 INFO:journalctl@ceph.osd.3.smithi186.stdout:Apr 12 21:05:19 smithi186 bash[95203]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3/block 2024-04-12T21:05:19.781 INFO:journalctl@ceph.osd.3.smithi186.stdout:Apr 12 21:05:19 smithi186 bash[95203]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-04-12T21:05:19.781 INFO:journalctl@ceph.osd.3.smithi186.stdout:Apr 12 21:05:19 smithi186 bash[95203]: --> ceph-volume raw activate successful for osd ID: 3 2024-04-12T21:05:19.782 INFO:journalctl@ceph.osd.3.smithi186.stdout:Apr 12 21:05:19 smithi186 ceph-1cbe407a-f90d-11ee-bc8d-c7b262605968-osd-3-activate[95218]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_4 2024-04-12T21:05:19.782 INFO:journalctl@ceph.osd.3.smithi186.stdout:Apr 12 21:05:19 smithi186 ceph-1cbe407a-f90d-11ee-bc8d-c7b262605968-osd-3-activate[95218]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-3 2024-04-12T21:05:19.782 INFO:journalctl@ceph.osd.3.smithi186.stdout:Apr 12 21:05:19 smithi186 ceph-1cbe407a-f90d-11ee-bc8d-c7b262605968-osd-3-activate[95218]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3/block 2024-04-12T21:05:19.782 INFO:journalctl@ceph.osd.3.smithi186.stdout:Apr 12 21:05:19 smithi186 ceph-1cbe407a-f90d-11ee-bc8d-c7b262605968-osd-3-activate[95218]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-04-12T21:05:19.782 INFO:journalctl@ceph.osd.3.smithi186.stdout:Apr 12 21:05:19 smithi186 ceph-1cbe407a-f90d-11ee-bc8d-c7b262605968-osd-3-activate[95218]: --> ceph-volume raw activate successful for osd ID: 3 2024-04-12T21:05:19.782 INFO:journalctl@ceph.osd.3.smithi186.stdout:Apr 12 21:05:19 smithi186 podman[95203]: 2024-04-12 21:05:19.492891511 +0000 UTC m=+1.412791057 container died dc952646a8b54a3a04cc3ef5b15a06d28a7a983ae22b5d7d82a7098e83354447 (image=quay.ceph.io/ceph-ci/ceph@sha256:f4452def3be4d350f4a8e5feaf7ebb1511be9298fe68478ba7a2d6fbd7d69a63, name=ceph-1cbe407a-f90d-11ee-bc8d-c7b262605968-osd-3-activate, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240409, org.label-schema.vendor=CentOS, RELEASE=wip-vshankar-testing-20240411.061452-324baf8) 2024-04-12T21:05:20.214 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:05:19 smithi186 ceph-mon[86379]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 2 pgs peering) 2024-04-12T21:05:20.350 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:05:19 smithi018 ceph-mon[112590]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 2 pgs peering) 2024-04-12T21:05:20.529 INFO:journalctl@ceph.osd.3.smithi186.stdout:Apr 12 21:05:20 smithi186 podman[95343]: 2024-04-12 21:05:20.217382766 +0000 UTC m=+0.705099386 container remove dc952646a8b54a3a04cc3ef5b15a06d28a7a983ae22b5d7d82a7098e83354447 (image=quay.ceph.io/ceph-ci/ceph@sha256:f4452def3be4d350f4a8e5feaf7ebb1511be9298fe68478ba7a2d6fbd7d69a63, name=ceph-1cbe407a-f90d-11ee-bc8d-c7b262605968-osd-3-activate, ceph=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-vshankar-testing-20240411.061452-324baf8, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240409) 2024-04-12T21:05:20.851 INFO:journalctl@ceph.osd.3.smithi186.stdout:Apr 12 21:05:20 smithi186 podman[95407]: 2024-04-12 21:05:20.482144733 +0000 UTC m=+0.035611888 image pull 8717720dc5e5c56303818ea4d4f0a11f6e2c03dee521fb668f411f1304e0afca quay.ceph.io/ceph-ci/ceph@sha256:f4452def3be4d350f4a8e5feaf7ebb1511be9298fe68478ba7a2d6fbd7d69a63 2024-04-12T21:05:20.851 INFO:journalctl@ceph.osd.3.smithi186.stdout:Apr 12 21:05:20 smithi186 podman[95407]: 2024-04-12 21:05:20.593344578 +0000 UTC m=+0.146811740 container create 0138e8262e4bab328e00c422fb40fa78a35c6daf7fe25c2b12b216036536a403 (image=quay.ceph.io/ceph-ci/ceph@sha256:f4452def3be4d350f4a8e5feaf7ebb1511be9298fe68478ba7a2d6fbd7d69a63, name=ceph-1cbe407a-f90d-11ee-bc8d-c7b262605968-osd-3, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240409, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.vendor=CentOS, ceph=True, RELEASE=wip-vshankar-testing-20240411.061452-324baf8, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=) 2024-04-12T21:05:20.852 INFO:journalctl@ceph.osd.3.smithi186.stdout:Apr 12 21:05:20 smithi186 podman[95407]: 2024-04-12 21:05:20.855162426 +0000 UTC m=+0.408629595 container init 0138e8262e4bab328e00c422fb40fa78a35c6daf7fe25c2b12b216036536a403 (image=quay.ceph.io/ceph-ci/ceph@sha256:f4452def3be4d350f4a8e5feaf7ebb1511be9298fe68478ba7a2d6fbd7d69a63, name=ceph-1cbe407a-f90d-11ee-bc8d-c7b262605968-osd-3, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240409, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, RELEASE=wip-vshankar-testing-20240411.061452-324baf8, GIT_CLEAN=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux ) 2024-04-12T21:05:21.176 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:05:20 smithi186 ceph-mon[86379]: pgmap v149: 65 pgs: 17 active+undersized, 16 active+undersized+degraded, 32 active+clean; 257 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 56/324 objects degraded (17.284%) 2024-04-12T21:05:21.176 INFO:journalctl@ceph.osd.3.smithi186.stdout:Apr 12 21:05:20 smithi186 podman[95407]: 2024-04-12 21:05:20.866188679 +0000 UTC m=+0.419655823 container start 0138e8262e4bab328e00c422fb40fa78a35c6daf7fe25c2b12b216036536a403 (image=quay.ceph.io/ceph-ci/ceph@sha256:f4452def3be4d350f4a8e5feaf7ebb1511be9298fe68478ba7a2d6fbd7d69a63, name=ceph-1cbe407a-f90d-11ee-bc8d-c7b262605968-osd-3, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, RELEASE=wip-vshankar-testing-20240411.061452-324baf8, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240409, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True) 2024-04-12T21:05:21.176 INFO:journalctl@ceph.osd.3.smithi186.stdout:Apr 12 21:05:20 smithi186 ceph-osd[95426]: -- 172.21.15.186:0/2069484467 <== mon.1 v2:172.21.15.186:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x561334757680 con 0x561334720c00 2024-04-12T21:05:21.176 INFO:journalctl@ceph.osd.3.smithi186.stdout:Apr 12 21:05:20 smithi186 bash[95407]: 0138e8262e4bab328e00c422fb40fa78a35c6daf7fe25c2b12b216036536a403 2024-04-12T21:05:21.177 INFO:journalctl@ceph.osd.3.smithi186.stdout:Apr 12 21:05:20 smithi186 systemd[1]: Started Ceph osd.3 for 1cbe407a-f90d-11ee-bc8d-c7b262605968. 2024-04-12T21:05:21.350 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:05:20 smithi018 ceph-mon[112590]: pgmap v149: 65 pgs: 17 active+undersized, 16 active+undersized+degraded, 32 active+clean; 257 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 56/324 objects degraded (17.284%) 2024-04-12T21:05:21.779 INFO:journalctl@ceph.osd.3.smithi186.stdout:Apr 12 21:05:21 smithi186 ceph-1cbe407a-f90d-11ee-bc8d-c7b262605968-osd-3[95422]: 2024-04-12T21:05:21.447+0000 7f172ddc17c0 -1 Falling back to public interface 2024-04-12T21:05:22.350 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:05:22 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' 2024-04-12T21:05:22.351 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:05:22 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' 2024-04-12T21:05:22.351 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:05:22 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-12T21:05:22.397 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:05:22 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' 2024-04-12T21:05:22.397 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:05:22 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' 2024-04-12T21:05:22.397 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:05:22 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-12T21:05:23.350 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:05:23 smithi018 ceph-mon[112590]: pgmap v150: 65 pgs: 17 active+undersized, 16 active+undersized+degraded, 32 active+clean; 257 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 56/324 objects degraded (17.284%) 2024-04-12T21:05:23.424 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:05:23 smithi186 ceph-mon[86379]: pgmap v150: 65 pgs: 17 active+undersized, 16 active+undersized+degraded, 32 active+clean; 257 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 56/324 objects degraded (17.284%) 2024-04-12T21:05:25.350 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:05:25 smithi018 ceph-mon[112590]: pgmap v151: 65 pgs: 17 active+undersized, 16 active+undersized+degraded, 32 active+clean; 257 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 56/324 objects degraded (17.284%) 2024-04-12T21:05:25.351 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:05:25 smithi018 ceph-mon[112590]: Health check update: Degraded data redundancy: 56/324 objects degraded (17.284%), 16 pgs degraded (PG_DEGRADED) 2024-04-12T21:05:25.351 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:05:25 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' 2024-04-12T21:05:25.351 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:05:25 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T21:05:25.529 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:05:25 smithi186 ceph-mon[86379]: pgmap v151: 65 pgs: 17 active+undersized, 16 active+undersized+degraded, 32 active+clean; 257 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 56/324 objects degraded (17.284%) 2024-04-12T21:05:25.529 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:05:25 smithi186 ceph-mon[86379]: Health check update: Degraded data redundancy: 56/324 objects degraded (17.284%), 16 pgs degraded (PG_DEGRADED) 2024-04-12T21:05:25.529 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:05:25 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' 2024-04-12T21:05:25.529 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:05:25 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T21:05:26.529 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:05:26 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' 2024-04-12T21:05:26.529 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:05:26 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' 2024-04-12T21:05:26.600 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:05:26 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' 2024-04-12T21:05:26.600 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:05:26 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' 2024-04-12T21:05:27.529 INFO:journalctl@ceph.osd.3.smithi186.stdout:Apr 12 21:05:27 smithi186 ceph-1cbe407a-f90d-11ee-bc8d-c7b262605968-osd-3[95422]: 2024-04-12T21:05:27.271+0000 7f172ddc17c0 -1 osd.3 83 log_to_monitors true 2024-04-12T21:05:27.529 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:05:27 smithi186 ceph-mon[86379]: pgmap v152: 65 pgs: 17 active+undersized, 16 active+undersized+degraded, 32 active+clean; 257 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 56/324 objects degraded (17.284%) 2024-04-12T21:05:27.529 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:05:27 smithi186 ceph-mon[86379]: from='osd.3 [v2:172.21.15.186:6800/1952273925,v1:172.21.15.186:6801/1952273925]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-04-12T21:05:27.530 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:05:27 smithi186 ceph-mon[86379]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-04-12T21:05:27.600 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:05:27 smithi018 ceph-mon[112590]: pgmap v152: 65 pgs: 17 active+undersized, 16 active+undersized+degraded, 32 active+clean; 257 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 56/324 objects degraded (17.284%) 2024-04-12T21:05:27.601 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:05:27 smithi018 ceph-mon[112590]: from='osd.3 [v2:172.21.15.186:6800/1952273925,v1:172.21.15.186:6801/1952273925]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-04-12T21:05:27.601 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:05:27 smithi018 ceph-mon[112590]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-04-12T21:05:28.601 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:05:28 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' 2024-04-12T21:05:28.601 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:05:28 smithi018 ceph-mon[112590]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-04-12T21:05:28.601 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:05:28 smithi018 ceph-mon[112590]: from='osd.3 [v2:172.21.15.186:6800/1952273925,v1:172.21.15.186:6801/1952273925]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi186", "root=default"]}]: dispatch 2024-04-12T21:05:28.601 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:05:28 smithi018 ceph-mon[112590]: osdmap e86: 6 total, 5 up, 6 in 2024-04-12T21:05:28.601 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:05:28 smithi018 ceph-mon[112590]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi186", "root=default"]}]: dispatch 2024-04-12T21:05:28.601 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:05:28 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' 2024-04-12T21:05:28.615 INFO:journalctl@ceph.osd.3.smithi186.stdout:Apr 12 21:05:28 smithi186 ceph-1cbe407a-f90d-11ee-bc8d-c7b262605968-osd-3[95422]: 2024-04-12T21:05:28.318+0000 7f1718200700 -1 osd.3 83 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-12T21:05:28.615 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:05:28 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' 2024-04-12T21:05:28.615 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:05:28 smithi186 ceph-mon[86379]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-04-12T21:05:28.615 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:05:28 smithi186 ceph-mon[86379]: from='osd.3 [v2:172.21.15.186:6800/1952273925,v1:172.21.15.186:6801/1952273925]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi186", "root=default"]}]: dispatch 2024-04-12T21:05:28.615 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:05:28 smithi186 ceph-mon[86379]: osdmap e86: 6 total, 5 up, 6 in 2024-04-12T21:05:28.615 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:05:28 smithi186 ceph-mon[86379]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi186", "root=default"]}]: dispatch 2024-04-12T21:05:28.615 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:05:28 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' 2024-04-12T21:05:29.600 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:05:29 smithi018 ceph-mon[112590]: pgmap v154: 65 pgs: 17 active+undersized, 16 active+undersized+degraded, 32 active+clean; 257 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 56/324 objects degraded (17.284%) 2024-04-12T21:05:29.600 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:05:29 smithi018 ceph-mon[112590]: from='osd.3 ' entity='osd.3' 2024-04-12T21:05:29.637 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:05:29 smithi186 ceph-mon[86379]: pgmap v154: 65 pgs: 17 active+undersized, 16 active+undersized+degraded, 32 active+clean; 257 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 56/324 objects degraded (17.284%) 2024-04-12T21:05:29.637 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:05:29 smithi186 ceph-mon[86379]: from='osd.3 ' entity='osd.3' 2024-04-12T21:05:30.600 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:05:30 smithi018 ceph-mon[112590]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-04-12T21:05:30.601 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:05:30 smithi018 ceph-mon[112590]: osd.3 [v2:172.21.15.186:6800/1952273925,v1:172.21.15.186:6801/1952273925] boot 2024-04-12T21:05:30.601 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:05:30 smithi018 ceph-mon[112590]: osdmap e87: 6 total, 6 up, 6 in 2024-04-12T21:05:30.601 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:05:30 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-12T21:05:30.601 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:05:30 smithi018 ceph-mon[112590]: osdmap e88: 6 total, 6 up, 6 in 2024-04-12T21:05:30.779 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:05:30 smithi186 ceph-mon[86379]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-04-12T21:05:30.779 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:05:30 smithi186 ceph-mon[86379]: osd.3 [v2:172.21.15.186:6800/1952273925,v1:172.21.15.186:6801/1952273925] boot 2024-04-12T21:05:30.780 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:05:30 smithi186 ceph-mon[86379]: osdmap e87: 6 total, 6 up, 6 in 2024-04-12T21:05:30.780 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:05:30 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-12T21:05:30.780 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:05:30 smithi186 ceph-mon[86379]: osdmap e88: 6 total, 6 up, 6 in 2024-04-12T21:05:31.781 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:05:31 smithi186 ceph-mon[86379]: pgmap v156: 65 pgs: 17 active+undersized, 16 active+undersized+degraded, 32 active+clean; 257 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 56/324 objects degraded (17.284%) 2024-04-12T21:05:31.850 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:05:31 smithi018 ceph-mon[112590]: pgmap v156: 65 pgs: 17 active+undersized, 16 active+undersized+degraded, 32 active+clean; 257 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 56/324 objects degraded (17.284%) 2024-04-12T21:05:32.530 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:05:32 smithi186 ceph-mon[86379]: Health check update: Degraded data redundancy: 40/324 objects degraded (12.346%), 11 pgs degraded (PG_DEGRADED) 2024-04-12T21:05:32.850 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:05:32 smithi018 ceph-mon[112590]: Health check update: Degraded data redundancy: 40/324 objects degraded (12.346%), 11 pgs degraded (PG_DEGRADED) 2024-04-12T21:05:33.780 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:05:33 smithi186 ceph-mon[86379]: pgmap v158: 65 pgs: 7 active+undersized, 11 active+undersized+degraded, 47 active+clean; 257 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 40/324 objects degraded (12.346%) 2024-04-12T21:05:33.780 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:05:33 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' 2024-04-12T21:05:33.780 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:05:33 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' 2024-04-12T21:05:33.780 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:05:33 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-12T21:05:33.780 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:05:33 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-12T21:05:33.780 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:05:33 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' 2024-04-12T21:05:33.780 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:05:33 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-12T21:05:33.780 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:05:33 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T21:05:33.780 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:05:33 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T21:05:33.780 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:05:33 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T21:05:33.781 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:05:33 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd ok-to-stop", "ids": ["4"], "max": 16}]: dispatch 2024-04-12T21:05:33.851 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:05:33 smithi018 ceph-mon[112590]: pgmap v158: 65 pgs: 7 active+undersized, 11 active+undersized+degraded, 47 active+clean; 257 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 40/324 objects degraded (12.346%) 2024-04-12T21:05:33.851 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:05:33 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' 2024-04-12T21:05:33.851 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:05:33 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' 2024-04-12T21:05:33.851 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:05:33 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-12T21:05:33.852 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:05:33 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-12T21:05:33.852 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:05:33 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' 2024-04-12T21:05:33.852 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:05:33 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-12T21:05:33.852 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:05:33 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T21:05:33.852 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:05:33 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T21:05:33.852 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:05:33 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T21:05:33.852 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:05:33 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd ok-to-stop", "ids": ["4"], "max": 16}]: dispatch 2024-04-12T21:05:34.779 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:05:34 smithi186 ceph-mon[86379]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["4"], "max": 16}]: dispatch 2024-04-12T21:05:34.780 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:05:34 smithi186 ceph-mon[86379]: Upgrade: unsafe to stop osd(s) at this time (9 PGs are or would become offline) 2024-04-12T21:05:34.850 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:05:34 smithi018 ceph-mon[112590]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["4"], "max": 16}]: dispatch 2024-04-12T21:05:34.851 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:05:34 smithi018 ceph-mon[112590]: Upgrade: unsafe to stop osd(s) at this time (9 PGs are or would become offline) 2024-04-12T21:05:35.779 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:05:35 smithi186 ceph-mon[86379]: pgmap v159: 65 pgs: 3 active+undersized, 3 active+undersized+degraded, 59 active+clean; 257 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 9/324 objects degraded (2.778%) 2024-04-12T21:05:35.850 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:05:35 smithi018 ceph-mon[112590]: pgmap v159: 65 pgs: 3 active+undersized, 3 active+undersized+degraded, 59 active+clean; 257 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 9/324 objects degraded (2.778%) 2024-04-12T21:05:36.779 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:05:36 smithi186 ceph-mon[86379]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 9/324 objects degraded (2.778%), 3 pgs degraded) 2024-04-12T21:05:36.779 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:05:36 smithi186 ceph-mon[86379]: Cluster is now healthy 2024-04-12T21:05:36.850 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:05:36 smithi018 ceph-mon[112590]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 9/324 objects degraded (2.778%), 3 pgs degraded) 2024-04-12T21:05:36.851 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:05:36 smithi018 ceph-mon[112590]: Cluster is now healthy 2024-04-12T21:05:37.779 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:05:37 smithi186 ceph-mon[86379]: pgmap v160: 65 pgs: 65 active+clean; 257 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-04-12T21:05:37.850 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:05:37 smithi018 ceph-mon[112590]: pgmap v160: 65 pgs: 65 active+clean; 257 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-04-12T21:05:39.780 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:05:39 smithi186 ceph-mon[86379]: pgmap v161: 65 pgs: 65 active+clean; 257 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-04-12T21:05:39.850 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:05:39 smithi018 ceph-mon[112590]: pgmap v161: 65 pgs: 65 active+clean; 257 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-04-12T21:05:41.182 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:05:40 smithi186 ceph-mon[86379]: pgmap v162: 65 pgs: 65 active+clean; 257 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-04-12T21:05:41.182 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:05:40 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' 2024-04-12T21:05:41.182 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:05:40 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T21:05:41.350 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:05:40 smithi018 ceph-mon[112590]: pgmap v162: 65 pgs: 65 active+clean; 257 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-04-12T21:05:41.351 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:05:40 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' 2024-04-12T21:05:41.351 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:05:40 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T21:05:43.279 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:05:42 smithi186 ceph-mon[86379]: pgmap v163: 65 pgs: 65 active+clean; 257 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-04-12T21:05:43.350 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:05:42 smithi018 ceph-mon[112590]: pgmap v163: 65 pgs: 65 active+clean; 257 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-04-12T21:05:45.279 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:05:44 smithi186 ceph-mon[86379]: pgmap v164: 65 pgs: 65 active+clean; 257 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-04-12T21:05:45.350 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:05:44 smithi018 ceph-mon[112590]: pgmap v164: 65 pgs: 65 active+clean; 257 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-04-12T21:05:45.721 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:45.724+0000 7f474ca00700 1 -- 172.21.15.18:0/2652507001 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4748109dd0 msgr2=0x7f4748103440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:05:45.722 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:45.724+0000 7f474ca00700 1 --2- 172.21.15.18:0/2652507001 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4748109dd0 0x7f4748103440 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f4734004940 tx=0x7f4734009e30 comp rx=0 tx=0).stop 2024-04-12T21:05:45.722 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:45.724+0000 7f474ca00700 1 -- 172.21.15.18:0/2652507001 shutdown_connections 2024-04-12T21:05:45.722 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:45.724+0000 7f474ca00700 1 --2- 172.21.15.18:0/2652507001 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4748109dd0 0x7f4748103440 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:05:45.722 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:45.724+0000 7f474ca00700 1 --2- 172.21.15.18:0/2652507001 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4748109430 0x7f4748109800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:05:45.722 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:45.724+0000 7f474ca00700 1 -- 172.21.15.18:0/2652507001 >> 172.21.15.18:0/2652507001 conn(0x7f4748072c00 msgr2=0x7f4748075010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:05:45.722 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:45.724+0000 7f474ca00700 1 -- 172.21.15.18:0/2652507001 shutdown_connections 2024-04-12T21:05:45.722 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:45.724+0000 7f474ca00700 1 -- 172.21.15.18:0/2652507001 wait complete. 2024-04-12T21:05:45.723 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:45.725+0000 7f474ca00700 1 Processor -- start 2024-04-12T21:05:45.723 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:45.726+0000 7f474ca00700 1 -- start start 2024-04-12T21:05:45.723 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:45.726+0000 7f474ca00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4748109430 0x7f4748119f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:05:45.723 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:45.726+0000 7f474ca00700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4748109dd0 0x7f474811a4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:05:45.723 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:45.726+0000 7f474ca00700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f474811ab20 con 0x7f4748109430 2024-04-12T21:05:45.724 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:45.726+0000 7f474ca00700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4748114100 con 0x7f4748109dd0 2024-04-12T21:05:45.724 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:45.726+0000 7f4745e00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4748109430 0x7f4748119f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:05:45.724 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:45.726+0000 7f4745e00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4748109430 0x7f4748119f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:55790/0 (socket says 172.21.15.18:55790) 2024-04-12T21:05:45.724 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:45.726+0000 7f4745e00700 1 -- 172.21.15.18:0/3300063489 learned_addr learned my addr 172.21.15.18:0/3300063489 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:05:45.725 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:45.727+0000 7f4745e00700 1 -- 172.21.15.18:0/3300063489 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4748109dd0 msgr2=0x7f474811a4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:05:45.725 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:45.727+0000 7f4745e00700 1 --2- 172.21.15.18:0/3300063489 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4748109dd0 0x7f474811a4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:05:45.725 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:45.727+0000 7f4745e00700 1 -- 172.21.15.18:0/3300063489 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4734005040 con 0x7f4748109430 2024-04-12T21:05:45.725 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:45.727+0000 7f4745e00700 1 --2- 172.21.15.18:0/3300063489 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4748109430 0x7f4748119f90 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f473800c8c0 tx=0x7f47380044c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:05:45.725 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:45.727+0000 7f473e000700 1 -- 172.21.15.18:0/3300063489 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f473800a980 con 0x7f4748109430 2024-04-12T21:05:45.725 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:45.727+0000 7f473e000700 1 -- 172.21.15.18:0/3300063489 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f473800ce60 con 0x7f4748109430 2024-04-12T21:05:45.725 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:45.727+0000 7f473e000700 1 -- 172.21.15.18:0/3300063489 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f47380115e0 con 0x7f4748109430 2024-04-12T21:05:45.725 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:45.727+0000 7f474ca00700 1 -- 172.21.15.18:0/3300063489 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f47481143e0 con 0x7f4748109430 2024-04-12T21:05:45.725 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:45.728+0000 7f474ca00700 1 -- 172.21.15.18:0/3300063489 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f47481149a0 con 0x7f4748109430 2024-04-12T21:05:45.726 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:45.729+0000 7f474ca00700 1 -- 172.21.15.18:0/3300063489 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4748104b80 con 0x7f4748109430 2024-04-12T21:05:45.728 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:45.731+0000 7f473e000700 1 -- 172.21.15.18:0/3300063489 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7f4738012400 con 0x7f4748109430 2024-04-12T21:05:45.729 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:45.732+0000 7f473e000700 1 --2- 172.21.15.18:0/3300063489 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f472c06ec00 0x7f472c0710b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:05:45.729 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:45.732+0000 7f473e000700 1 -- 172.21.15.18:0/3300063489 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6585+0+0 (secure 0 0 0) 0x7f473808f5b0 con 0x7f4748109430 2024-04-12T21:05:45.729 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:45.732+0000 7f4745400700 1 --2- 172.21.15.18:0/3300063489 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f472c06ec00 0x7f472c0710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:05:45.730 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:45.733+0000 7f4745400700 1 --2- 172.21.15.18:0/3300063489 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f472c06ec00 0x7f472c0710b0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f4734004f90 tx=0x7f4734009e30 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:05:45.734 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:45.737+0000 7f473e000700 1 -- 172.21.15.18:0/3300063489 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f473805bf50 con 0x7f4748109430 2024-04-12T21:05:45.994 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:45.997+0000 7f474ca00700 1 -- 172.21.15.18:0/3300063489 --> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f4748110570 con 0x7f472c06ec00 2024-04-12T21:05:45.995 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:05:45 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' 2024-04-12T21:05:45.996 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:45.999+0000 7f473e000700 1 -- 172.21.15.18:0/3300063489 <== mgr.34102 v2:172.21.15.18:6800/3812891919 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+401 (secure 0 0 0) 0x7f4748110570 con 0x7f472c06ec00 2024-04-12T21:05:45.999 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:46.002+0000 7f474ca00700 1 -- 172.21.15.18:0/3300063489 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f472c06ec00 msgr2=0x7f472c0710b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:05:45.999 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:46.002+0000 7f474ca00700 1 --2- 172.21.15.18:0/3300063489 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f472c06ec00 0x7f472c0710b0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f4734004f90 tx=0x7f4734009e30 comp rx=0 tx=0).stop 2024-04-12T21:05:45.999 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:46.002+0000 7f474ca00700 1 -- 172.21.15.18:0/3300063489 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4748109430 msgr2=0x7f4748119f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:05:45.999 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:46.002+0000 7f474ca00700 1 --2- 172.21.15.18:0/3300063489 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4748109430 0x7f4748119f90 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f473800c8c0 tx=0x7f47380044c0 comp rx=0 tx=0).stop 2024-04-12T21:05:46.000 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:46.002+0000 7f474ca00700 1 -- 172.21.15.18:0/3300063489 shutdown_connections 2024-04-12T21:05:46.000 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:46.002+0000 7f474ca00700 1 --2- 172.21.15.18:0/3300063489 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f472c06ec00 0x7f472c0710b0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:05:46.000 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:46.002+0000 7f474ca00700 1 --2- 172.21.15.18:0/3300063489 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4748109430 0x7f4748119f90 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:05:46.000 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:46.002+0000 7f474ca00700 1 --2- 172.21.15.18:0/3300063489 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4748109dd0 0x7f474811a4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:05:46.000 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:46.002+0000 7f474ca00700 1 -- 172.21.15.18:0/3300063489 >> 172.21.15.18:0/3300063489 conn(0x7f4748072c00 msgr2=0x7f4748074c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:05:46.000 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:46.003+0000 7f474ca00700 1 -- 172.21.15.18:0/3300063489 shutdown_connections 2024-04-12T21:05:46.000 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:46.003+0000 7f474ca00700 1 -- 172.21.15.18:0/3300063489 wait complete. 2024-04-12T21:05:46.017 INFO:teuthology.orchestra.run.smithi018.stdout:true 2024-04-12T21:05:46.156 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:46.158+0000 7efc66e00700 1 -- 172.21.15.18:0/2553577935 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7efc60103780 msgr2=0x7efc600fcdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:05:46.156 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:46.158+0000 7efc66e00700 1 --2- 172.21.15.18:0/2553577935 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7efc60103780 0x7efc600fcdf0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7efc4c004940 tx=0x7efc4c009e30 comp rx=0 tx=0).stop 2024-04-12T21:05:46.157 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:46.159+0000 7efc66e00700 1 -- 172.21.15.18:0/2553577935 shutdown_connections 2024-04-12T21:05:46.157 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:46.159+0000 7efc66e00700 1 --2- 172.21.15.18:0/2553577935 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7efc60103780 0x7efc600fcdf0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:05:46.157 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:46.159+0000 7efc66e00700 1 --2- 172.21.15.18:0/2553577935 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7efc60102de0 0x7efc601031b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:05:46.157 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:46.159+0000 7efc66e00700 1 -- 172.21.15.18:0/2553577935 >> 172.21.15.18:0/2553577935 conn(0x7efc600f8950 msgr2=0x7efc600fad60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:05:46.157 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:46.159+0000 7efc66e00700 1 -- 172.21.15.18:0/2553577935 shutdown_connections 2024-04-12T21:05:46.157 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:46.159+0000 7efc66e00700 1 -- 172.21.15.18:0/2553577935 wait complete. 2024-04-12T21:05:46.157 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:46.160+0000 7efc66e00700 1 Processor -- start 2024-04-12T21:05:46.159 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:46.160+0000 7efc66e00700 1 -- start start 2024-04-12T21:05:46.159 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:46.160+0000 7efc66e00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7efc60102de0 0x7efc60115c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:05:46.159 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:46.160+0000 7efc66e00700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7efc60103780 0x7efc60116190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:05:46.159 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:46.160+0000 7efc66e00700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc601167e0 con 0x7efc60102de0 2024-04-12T21:05:46.159 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:46.160+0000 7efc66e00700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc6010fdc0 con 0x7efc60103780 2024-04-12T21:05:46.159 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:46.161+0000 7efc5fe00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7efc60102de0 0x7efc60115c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:05:46.159 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:46.161+0000 7efc5fe00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7efc60102de0 0x7efc60115c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:55796/0 (socket says 172.21.15.18:55796) 2024-04-12T21:05:46.159 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:46.161+0000 7efc5fe00700 1 -- 172.21.15.18:0/2226007175 learned_addr learned my addr 172.21.15.18:0/2226007175 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:05:46.159 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:46.161+0000 7efc5f400700 1 --2- 172.21.15.18:0/2226007175 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7efc60103780 0x7efc60116190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:05:46.159 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:46.161+0000 7efc5fe00700 1 -- 172.21.15.18:0/2226007175 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7efc60103780 msgr2=0x7efc60116190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:05:46.159 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:46.161+0000 7efc5fe00700 1 --2- 172.21.15.18:0/2226007175 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7efc60103780 0x7efc60116190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:05:46.159 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:46.161+0000 7efc5fe00700 1 -- 172.21.15.18:0/2226007175 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc4c005040 con 0x7efc60102de0 2024-04-12T21:05:46.160 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:46.161+0000 7efc5f400700 1 --2- 172.21.15.18:0/2226007175 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7efc60103780 0x7efc60116190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-12T21:05:46.160 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:46.162+0000 7efc5fe00700 1 --2- 172.21.15.18:0/2226007175 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7efc60102de0 0x7efc60115c50 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7efc50002a50 tx=0x7efc500046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:05:46.160 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:46.162+0000 7efc57e00700 1 -- 172.21.15.18:0/2226007175 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7efc50007cc0 con 0x7efc60102de0 2024-04-12T21:05:46.160 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:46.162+0000 7efc57e00700 1 -- 172.21.15.18:0/2226007175 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7efc5000f040 con 0x7efc60102de0 2024-04-12T21:05:46.160 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:46.162+0000 7efc66e00700 1 -- 172.21.15.18:0/2226007175 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efc601100a0 con 0x7efc60102de0 2024-04-12T21:05:46.160 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:46.162+0000 7efc57e00700 1 -- 172.21.15.18:0/2226007175 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7efc5000db70 con 0x7efc60102de0 2024-04-12T21:05:46.160 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:46.162+0000 7efc66e00700 1 -- 172.21.15.18:0/2226007175 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efc601105e0 con 0x7efc60102de0 2024-04-12T21:05:46.160 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:46.163+0000 7efc66e00700 1 -- 172.21.15.18:0/2226007175 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efc60102100 con 0x7efc60102de0 2024-04-12T21:05:46.161 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:46.165+0000 7efc57e00700 1 -- 172.21.15.18:0/2226007175 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7efc5000dcd0 con 0x7efc60102de0 2024-04-12T21:05:46.162 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:46.166+0000 7efc57e00700 1 --2- 172.21.15.18:0/2226007175 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7efc4406ec00 0x7efc440710b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:05:46.163 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:46.166+0000 7efc57e00700 1 -- 172.21.15.18:0/2226007175 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6585+0+0 (secure 0 0 0) 0x7efc5008e2c0 con 0x7efc60102de0 2024-04-12T21:05:46.163 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:46.166+0000 7efc5f400700 1 --2- 172.21.15.18:0/2226007175 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7efc4406ec00 0x7efc440710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:05:46.163 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:46.167+0000 7efc5f400700 1 --2- 172.21.15.18:0/2226007175 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7efc4406ec00 0x7efc440710b0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7efc4c009fd0 tx=0x7efc4c009e30 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:05:46.166 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:46.169+0000 7efc57e00700 1 -- 172.21.15.18:0/2226007175 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7efc5005ab20 con 0x7efc60102de0 2024-04-12T21:05:46.279 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:05:45 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' 2024-04-12T21:05:46.429 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:46.432+0000 7efc66e00700 1 -- 172.21.15.18:0/2226007175 --> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7efc60109f80 con 0x7efc4406ec00 2024-04-12T21:05:46.431 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:46.435+0000 7efc57e00700 1 -- 172.21.15.18:0/2226007175 <== mgr.34102 v2:172.21.15.18:6800/3812891919 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+401 (secure 0 0 0) 0x7efc4c008e30 con 0x7efc4406ec00 2024-04-12T21:05:46.434 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:46.437+0000 7efc66e00700 1 -- 172.21.15.18:0/2226007175 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7efc4406ec00 msgr2=0x7efc440710b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:05:46.435 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:46.437+0000 7efc66e00700 1 --2- 172.21.15.18:0/2226007175 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7efc4406ec00 0x7efc440710b0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7efc4c009fd0 tx=0x7efc4c009e30 comp rx=0 tx=0).stop 2024-04-12T21:05:46.435 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:46.437+0000 7efc66e00700 1 -- 172.21.15.18:0/2226007175 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7efc60102de0 msgr2=0x7efc60115c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:05:46.435 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:46.437+0000 7efc66e00700 1 --2- 172.21.15.18:0/2226007175 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7efc60102de0 0x7efc60115c50 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7efc50002a50 tx=0x7efc500046b0 comp rx=0 tx=0).stop 2024-04-12T21:05:46.435 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:46.437+0000 7efc66e00700 1 -- 172.21.15.18:0/2226007175 shutdown_connections 2024-04-12T21:05:46.435 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:46.437+0000 7efc66e00700 1 --2- 172.21.15.18:0/2226007175 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7efc4406ec00 0x7efc440710b0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:05:46.435 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:46.437+0000 7efc66e00700 1 --2- 172.21.15.18:0/2226007175 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7efc60102de0 0x7efc60115c50 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:05:46.435 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:46.437+0000 7efc66e00700 1 --2- 172.21.15.18:0/2226007175 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7efc60103780 0x7efc60116190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:05:46.435 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:46.437+0000 7efc66e00700 1 -- 172.21.15.18:0/2226007175 >> 172.21.15.18:0/2226007175 conn(0x7efc600f8950 msgr2=0x7efc600fa680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:05:46.435 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:46.438+0000 7efc66e00700 1 -- 172.21.15.18:0/2226007175 shutdown_connections 2024-04-12T21:05:46.436 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:46.438+0000 7efc66e00700 1 -- 172.21.15.18:0/2226007175 wait complete. 2024-04-12T21:05:46.591 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:46.594+0000 7f61e6000700 1 -- 172.21.15.18:0/664609805 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f61e00fe620 msgr2=0x7f61e0106c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:05:46.592 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:46.594+0000 7f61e6000700 1 --2- 172.21.15.18:0/664609805 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f61e00fe620 0x7f61e0106c60 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f61cc004940 tx=0x7f61cc009e30 comp rx=0 tx=0).stop 2024-04-12T21:05:46.592 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:46.594+0000 7f61e6000700 1 -- 172.21.15.18:0/664609805 shutdown_connections 2024-04-12T21:05:46.592 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:46.594+0000 7f61e6000700 1 --2- 172.21.15.18:0/664609805 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f61e00fe620 0x7f61e0106c60 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:05:46.592 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:46.594+0000 7f61e6000700 1 --2- 172.21.15.18:0/664609805 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f61e00fdc80 0x7f61e00fe050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:05:46.592 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:46.594+0000 7f61e6000700 1 -- 172.21.15.18:0/664609805 >> 172.21.15.18:0/664609805 conn(0x7f61e00f76b0 msgr2=0x7f61e00f9ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:05:46.592 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:46.594+0000 7f61e6000700 1 -- 172.21.15.18:0/664609805 shutdown_connections 2024-04-12T21:05:46.592 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:46.595+0000 7f61e6000700 1 -- 172.21.15.18:0/664609805 wait complete. 2024-04-12T21:05:46.593 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:46.595+0000 7f61e6000700 1 Processor -- start 2024-04-12T21:05:46.593 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:46.595+0000 7f61e6000700 1 -- start start 2024-04-12T21:05:46.593 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:46.596+0000 7f61e6000700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f61e00fdc80 0x7f61e0115ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:05:46.593 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:46.596+0000 7f61e6000700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f61e00fe620 0x7f61e01160e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:05:46.593 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:46.596+0000 7f61e6000700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61e0116730 con 0x7f61e00fdc80 2024-04-12T21:05:46.593 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:46.596+0000 7f61e6000700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61e010fb20 con 0x7f61e00fe620 2024-04-12T21:05:46.593 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:46.596+0000 7f61e4c00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f61e00fdc80 0x7f61e0115ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:05:46.593 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:46.596+0000 7f61e4c00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f61e00fdc80 0x7f61e0115ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:55816/0 (socket says 172.21.15.18:55816) 2024-04-12T21:05:46.594 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:46.596+0000 7f61e4c00700 1 -- 172.21.15.18:0/2282724860 learned_addr learned my addr 172.21.15.18:0/2282724860 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:05:46.595 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:46.597+0000 7f61dfe00700 1 --2- 172.21.15.18:0/2282724860 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f61e00fe620 0x7f61e01160e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:05:46.595 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:46.597+0000 7f61e4c00700 1 -- 172.21.15.18:0/2282724860 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f61e00fe620 msgr2=0x7f61e01160e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:05:46.595 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:46.597+0000 7f61e4c00700 1 --2- 172.21.15.18:0/2282724860 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f61e00fe620 0x7f61e01160e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:05:46.595 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:46.597+0000 7f61e4c00700 1 -- 172.21.15.18:0/2282724860 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61cc005040 con 0x7f61e00fdc80 2024-04-12T21:05:46.595 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:46.597+0000 7f61e4c00700 1 --2- 172.21.15.18:0/2282724860 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f61e00fdc80 0x7f61e0115ba0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f61d4001d80 tx=0x7f61d4009ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:05:46.595 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:46.597+0000 7f61dcc00700 1 -- 172.21.15.18:0/2282724860 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f61d4007d10 con 0x7f61e00fdc80 2024-04-12T21:05:46.595 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:46.597+0000 7f61dcc00700 1 -- 172.21.15.18:0/2282724860 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f61d4007e70 con 0x7f61e00fdc80 2024-04-12T21:05:46.595 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:46.597+0000 7f61dcc00700 1 -- 172.21.15.18:0/2282724860 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f61d4014640 con 0x7f61e00fdc80 2024-04-12T21:05:46.595 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:46.597+0000 7f61e6000700 1 -- 172.21.15.18:0/2282724860 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f61e010fe00 con 0x7f61e00fdc80 2024-04-12T21:05:46.595 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:46.598+0000 7f61e6000700 1 -- 172.21.15.18:0/2282724860 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f61e0110340 con 0x7f61e00fdc80 2024-04-12T21:05:46.596 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:46.599+0000 7f61e6000700 1 -- 172.21.15.18:0/2282724860 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f61e00668b0 con 0x7f61e00fdc80 2024-04-12T21:05:46.596 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:46.600+0000 7f61dcc00700 1 -- 172.21.15.18:0/2282724860 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7f61d400e960 con 0x7f61e00fdc80 2024-04-12T21:05:46.598 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:46.601+0000 7f61dcc00700 1 --2- 172.21.15.18:0/2282724860 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f61c806ec00 0x7f61c80710b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:05:46.598 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:46.601+0000 7f61dcc00700 1 -- 172.21.15.18:0/2282724860 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6585+0+0 (secure 0 0 0) 0x7f61d408e6d0 con 0x7f61e00fdc80 2024-04-12T21:05:46.598 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:46.601+0000 7f61dfe00700 1 --2- 172.21.15.18:0/2282724860 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f61c806ec00 0x7f61c80710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:05:46.599 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:46.602+0000 7f61dfe00700 1 --2- 172.21.15.18:0/2282724860 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f61c806ec00 0x7f61c80710b0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f61e0110ce0 tx=0x7f61cc009e30 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:05:46.603 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:46.606+0000 7f61dcc00700 1 -- 172.21.15.18:0/2282724860 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f61d405afe0 con 0x7f61e00fdc80 2024-04-12T21:05:46.842 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:46.844+0000 7f61e6000700 1 -- 172.21.15.18:0/2282724860 --> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f61e0109d10 con 0x7f61c806ec00 2024-04-12T21:05:46.853 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:46.856+0000 7f61dcc00700 1 -- 172.21.15.18:0/2282724860 <== mgr.34102 v2:172.21.15.18:6800/3812891919 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7f61e0109d10 con 0x7f61c806ec00 2024-04-12T21:05:46.854 INFO:teuthology.orchestra.run.smithi018.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-12T21:05:46.854 INFO:teuthology.orchestra.run.smithi018.stdout:alertmanager.smithi018 smithi018 *:9093,9094 running (6m) 49s ago 20m 14.9M - 0.25.0 c8568f914cd2 c332090068e8 2024-04-12T21:05:46.854 INFO:teuthology.orchestra.run.smithi018.stdout:crash.smithi018 smithi018 running (2m) 49s ago 21m 6665k - 19.0.0-2879-g324baf8d 8717720dc5e5 b7f80d1f7982 2024-04-12T21:05:46.854 INFO:teuthology.orchestra.run.smithi018.stdout:crash.smithi186 smithi186 running (2m) 21s ago 19m 6660k - 19.0.0-2879-g324baf8d 8717720dc5e5 a4f55c81b623 2024-04-12T21:05:46.854 INFO:teuthology.orchestra.run.smithi018.stdout:grafana.smithi018 smithi018 *:3000 running (5m) 49s ago 20m 74.5M - 9.4.12 2bacad6d85d8 b08e3bf176dd 2024-04-12T21:05:46.854 INFO:teuthology.orchestra.run.smithi018.stdout:mds.cephfs.smithi018.acqbsc smithi018 running (13m) 49s ago 13m 112M - 17.2.7-827-gd3b5060a 21de9b7369ad 5b5fce65f18a 2024-04-12T21:05:46.854 INFO:teuthology.orchestra.run.smithi018.stdout:mds.cephfs.smithi018.szjluo smithi018 running (13m) 49s ago 13m 113M - 17.2.7-827-gd3b5060a 21de9b7369ad cf9bf0c5031f 2024-04-12T21:05:46.854 INFO:teuthology.orchestra.run.smithi018.stdout:mds.cephfs.smithi186.jdllyn smithi186 running (13m) 21s ago 13m 15.2M - 17.2.7-827-gd3b5060a 21de9b7369ad 2b2bc46bd29a 2024-04-12T21:05:46.854 INFO:teuthology.orchestra.run.smithi018.stdout:mds.cephfs.smithi186.wjnafs smithi186 running (13m) 21s ago 13m 15.0M - 17.2.7-827-gd3b5060a 21de9b7369ad 74b115d6d6fd 2024-04-12T21:05:46.854 INFO:teuthology.orchestra.run.smithi018.stdout:mgr.smithi018.xjzwga smithi018 *:8443,9283,8765 running (9m) 49s ago 22m 505M - 19.0.0-2879-g324baf8d 8717720dc5e5 56088305eab7 2024-04-12T21:05:46.854 INFO:teuthology.orchestra.run.smithi018.stdout:mgr.smithi186.kqcfxk smithi186 *:8443,9283,8765 running (7m) 21s ago 19m 452M - 19.0.0-2879-g324baf8d 8717720dc5e5 09cd766e5e0d 2024-04-12T21:05:46.855 INFO:teuthology.orchestra.run.smithi018.stdout:mon.smithi018 smithi018 running (4m) 49s ago 22m 50.8M 2048M 19.0.0-2879-g324baf8d 8717720dc5e5 df3e201dfed6 2024-04-12T21:05:46.855 INFO:teuthology.orchestra.run.smithi018.stdout:mon.smithi186 smithi186 running (3m) 21s ago 19m 36.0M 2048M 19.0.0-2879-g324baf8d 8717720dc5e5 c77498da9b2b 2024-04-12T21:05:46.855 INFO:teuthology.orchestra.run.smithi018.stdout:node-exporter.smithi018 smithi018 *:9100 running (7m) 49s ago 21m 7818k - 1.5.0 0da6a335fe13 326c3d9c6847 2024-04-12T21:05:46.855 INFO:teuthology.orchestra.run.smithi018.stdout:node-exporter.smithi186 smithi186 *:9100 running (7m) 21s ago 19m 8078k - 1.5.0 0da6a335fe13 ec2112baeecf 2024-04-12T21:05:46.855 INFO:teuthology.orchestra.run.smithi018.stdout:osd.0 smithi018 running (2m) 49s ago 18m 117M 1559M 19.0.0-2879-g324baf8d 8717720dc5e5 60b1273ca047 2024-04-12T21:05:46.855 INFO:teuthology.orchestra.run.smithi018.stdout:osd.1 smithi018 running (90s) 49s ago 17m 88.0M 1559M 19.0.0-2879-g324baf8d 8717720dc5e5 170ce1e5fb8d 2024-04-12T21:05:46.855 INFO:teuthology.orchestra.run.smithi018.stdout:osd.2 smithi018 running (58s) 49s ago 17m 11.2M 1559M 19.0.0-2879-g324baf8d 8717720dc5e5 c2cbc6aa2839 2024-04-12T21:05:46.855 INFO:teuthology.orchestra.run.smithi018.stdout:osd.3 smithi186 running (26s) 21s ago 16m 11.0M 2581M 19.0.0-2879-g324baf8d 8717720dc5e5 0138e8262e4b 2024-04-12T21:05:46.855 INFO:teuthology.orchestra.run.smithi018.stdout:osd.4 smithi186 running (16m) 21s ago 16m 257M 2581M 17.2.7-827-gd3b5060a 21de9b7369ad 757e7c71b7b9 2024-04-12T21:05:46.855 INFO:teuthology.orchestra.run.smithi018.stdout:osd.5 smithi186 running (15m) 21s ago 15m 260M 2581M 17.2.7-827-gd3b5060a 21de9b7369ad 2ee072a436d0 2024-04-12T21:05:46.855 INFO:teuthology.orchestra.run.smithi018.stdout:prometheus.smithi018 smithi018 *:9095 running (6m) 49s ago 20m 60.0M - 2.43.0 a07b618ecd1d f58841127e21 2024-04-12T21:05:46.856 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:46.859+0000 7f61e6000700 1 -- 172.21.15.18:0/2282724860 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f61c806ec00 msgr2=0x7f61c80710b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:05:46.856 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:46.859+0000 7f61e6000700 1 --2- 172.21.15.18:0/2282724860 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f61c806ec00 0x7f61c80710b0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f61e0110ce0 tx=0x7f61cc009e30 comp rx=0 tx=0).stop 2024-04-12T21:05:46.856 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:46.859+0000 7f61e6000700 1 -- 172.21.15.18:0/2282724860 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f61e00fdc80 msgr2=0x7f61e0115ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:05:46.857 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:46.859+0000 7f61e6000700 1 --2- 172.21.15.18:0/2282724860 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f61e00fdc80 0x7f61e0115ba0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f61d4001d80 tx=0x7f61d4009ec0 comp rx=0 tx=0).stop 2024-04-12T21:05:46.857 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:46.859+0000 7f61e6000700 1 -- 172.21.15.18:0/2282724860 shutdown_connections 2024-04-12T21:05:46.857 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:46.859+0000 7f61e6000700 1 --2- 172.21.15.18:0/2282724860 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f61c806ec00 0x7f61c80710b0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:05:46.857 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:46.859+0000 7f61e6000700 1 --2- 172.21.15.18:0/2282724860 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f61e00fdc80 0x7f61e0115ba0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:05:46.857 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:46.859+0000 7f61e6000700 1 --2- 172.21.15.18:0/2282724860 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f61e00fe620 0x7f61e01160e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:05:46.857 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:46.859+0000 7f61e6000700 1 -- 172.21.15.18:0/2282724860 >> 172.21.15.18:0/2282724860 conn(0x7f61e00f76b0 msgr2=0x7f61e00f8b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:05:46.857 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:46.859+0000 7f61e6000700 1 -- 172.21.15.18:0/2282724860 shutdown_connections 2024-04-12T21:05:46.857 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:46.860+0000 7f61e6000700 1 -- 172.21.15.18:0/2282724860 wait complete. 2024-04-12T21:05:47.010 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:47.013+0000 7fd387000700 1 -- 172.21.15.18:0/296803592 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd3801059c0 msgr2=0x7fd3800ff080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:05:47.011 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:47.013+0000 7fd387000700 1 --2- 172.21.15.18:0/296803592 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd3801059c0 0x7fd3800ff080 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fd368004940 tx=0x7fd368009e30 comp rx=0 tx=0).stop 2024-04-12T21:05:47.011 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:47.013+0000 7fd387000700 1 -- 172.21.15.18:0/296803592 shutdown_connections 2024-04-12T21:05:47.011 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:47.013+0000 7fd387000700 1 --2- 172.21.15.18:0/296803592 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd3801059c0 0x7fd3800ff080 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:05:47.011 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:47.013+0000 7fd387000700 1 --2- 172.21.15.18:0/296803592 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd380105020 0x7fd3801053f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:05:47.011 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:47.013+0000 7fd387000700 1 -- 172.21.15.18:0/296803592 >> 172.21.15.18:0/296803592 conn(0x7fd3800faba0 msgr2=0x7fd3800fcfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:05:47.011 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:47.013+0000 7fd387000700 1 -- 172.21.15.18:0/296803592 shutdown_connections 2024-04-12T21:05:47.011 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:47.013+0000 7fd387000700 1 -- 172.21.15.18:0/296803592 wait complete. 2024-04-12T21:05:47.012 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:47.014+0000 7fd387000700 1 Processor -- start 2024-04-12T21:05:47.013 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:47.014+0000 7fd387000700 1 -- start start 2024-04-12T21:05:47.013 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:47.015+0000 7fd387000700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd380105020 0x7fd380115ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:05:47.013 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:47.015+0000 7fd387000700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd3801059c0 0x7fd3801160e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:05:47.013 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:47.015+0000 7fd387000700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd380116730 con 0x7fd380105020 2024-04-12T21:05:47.013 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:47.015+0000 7fd387000700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd38010fd10 con 0x7fd3801059c0 2024-04-12T21:05:47.013 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:47.015+0000 7fd37fe00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd380105020 0x7fd380115ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:05:47.013 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:47.015+0000 7fd37fe00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd380105020 0x7fd380115ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:55842/0 (socket says 172.21.15.18:55842) 2024-04-12T21:05:47.013 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:47.015+0000 7fd37fe00700 1 -- 172.21.15.18:0/2685627036 learned_addr learned my addr 172.21.15.18:0/2685627036 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:05:47.013 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:47.015+0000 7fd37f400700 1 --2- 172.21.15.18:0/2685627036 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd3801059c0 0x7fd3801160e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:05:47.013 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:47.015+0000 7fd37fe00700 1 -- 172.21.15.18:0/2685627036 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd3801059c0 msgr2=0x7fd3801160e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:05:47.013 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:47.016+0000 7fd37fe00700 1 --2- 172.21.15.18:0/2685627036 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd3801059c0 0x7fd3801160e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:05:47.013 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:47.016+0000 7fd37fe00700 1 -- 172.21.15.18:0/2685627036 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd368005040 con 0x7fd380105020 2024-04-12T21:05:47.013 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:47.016+0000 7fd37fe00700 1 --2- 172.21.15.18:0/2685627036 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd380105020 0x7fd380115ba0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fd370002a50 tx=0x7fd3700046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:05:47.014 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:47.016+0000 7fd377e00700 1 -- 172.21.15.18:0/2685627036 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd370007cc0 con 0x7fd380105020 2024-04-12T21:05:47.014 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:47.016+0000 7fd377e00700 1 -- 172.21.15.18:0/2685627036 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd37000f040 con 0x7fd380105020 2024-04-12T21:05:47.014 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:47.016+0000 7fd377e00700 1 -- 172.21.15.18:0/2685627036 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd370009640 con 0x7fd380105020 2024-04-12T21:05:47.014 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:47.016+0000 7fd387000700 1 -- 172.21.15.18:0/2685627036 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd38010fff0 con 0x7fd380105020 2024-04-12T21:05:47.014 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:47.016+0000 7fd387000700 1 -- 172.21.15.18:0/2685627036 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd3801105b0 con 0x7fd380105020 2024-04-12T21:05:47.015 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:47.018+0000 7fd387000700 1 -- 172.21.15.18:0/2685627036 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd380100770 con 0x7fd380105020 2024-04-12T21:05:47.015 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:47.018+0000 7fd377e00700 1 -- 172.21.15.18:0/2685627036 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7fd3700097a0 con 0x7fd380105020 2024-04-12T21:05:47.016 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:47.019+0000 7fd377e00700 1 --2- 172.21.15.18:0/2685627036 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fd36406ec00 0x7fd3640710b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:05:47.016 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:47.019+0000 7fd377e00700 1 -- 172.21.15.18:0/2685627036 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6585+0+0 (secure 0 0 0) 0x7fd37008e6b0 con 0x7fd380105020 2024-04-12T21:05:47.016 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:47.020+0000 7fd37f400700 1 --2- 172.21.15.18:0/2685627036 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fd36406ec00 0x7fd3640710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:05:47.017 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:47.020+0000 7fd37f400700 1 --2- 172.21.15.18:0/2685627036 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fd36406ec00 0x7fd3640710b0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fd36800b020 tx=0x7fd368009e30 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:05:47.021 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:47.024+0000 7fd377e00700 1 -- 172.21.15.18:0/2685627036 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7fd37005af10 con 0x7fd380105020 2024-04-12T21:05:47.100 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:05:46 smithi018 ceph-mon[112590]: pgmap v165: 65 pgs: 65 active+clean; 257 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-04-12T21:05:47.100 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:05:46 smithi018 ceph-mon[112590]: from='client.34342 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:05:47.100 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:05:46 smithi018 ceph-mon[112590]: from='client.34346 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:05:47.279 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:05:46 smithi186 ceph-mon[86379]: pgmap v165: 65 pgs: 65 active+clean; 257 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-04-12T21:05:47.279 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:05:46 smithi186 ceph-mon[86379]: from='client.34342 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:05:47.279 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:05:46 smithi186 ceph-mon[86379]: from='client.34346 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:05:47.336 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:47.339+0000 7fd387000700 1 -- 172.21.15.18:0/2685627036 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fd3801111f0 con 0x7fd380105020 2024-04-12T21:05:47.337 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:47.340+0000 7fd377e00700 1 -- 172.21.15.18:0/2685627036 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+839 (secure 0 0 0) 0x7fd37005aaa0 con 0x7fd380105020 2024-04-12T21:05:47.338 INFO:teuthology.orchestra.run.smithi018.stdout:{ 2024-04-12T21:05:47.338 INFO:teuthology.orchestra.run.smithi018.stdout: "mon": { 2024-04-12T21:05:47.338 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 19.0.0-2879-g324baf8d (324baf8ddb91dea46cac50cb535c306115791355) squid (dev)": 2 2024-04-12T21:05:47.338 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-12T21:05:47.338 INFO:teuthology.orchestra.run.smithi018.stdout: "mgr": { 2024-04-12T21:05:47.338 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 19.0.0-2879-g324baf8d (324baf8ddb91dea46cac50cb535c306115791355) squid (dev)": 2 2024-04-12T21:05:47.338 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-12T21:05:47.338 INFO:teuthology.orchestra.run.smithi018.stdout: "osd": { 2024-04-12T21:05:47.338 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 17.2.7-827-gd3b5060a (d3b5060a17449edfae9385cdf792afec109fca5d) quincy (stable)": 2, 2024-04-12T21:05:47.338 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 19.0.0-2879-g324baf8d (324baf8ddb91dea46cac50cb535c306115791355) squid (dev)": 4 2024-04-12T21:05:47.339 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-12T21:05:47.339 INFO:teuthology.orchestra.run.smithi018.stdout: "mds": { 2024-04-12T21:05:47.339 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 17.2.7-827-gd3b5060a (d3b5060a17449edfae9385cdf792afec109fca5d) quincy (stable)": 4 2024-04-12T21:05:47.339 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-12T21:05:47.339 INFO:teuthology.orchestra.run.smithi018.stdout: "overall": { 2024-04-12T21:05:47.339 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 17.2.7-827-gd3b5060a (d3b5060a17449edfae9385cdf792afec109fca5d) quincy (stable)": 6, 2024-04-12T21:05:47.339 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 19.0.0-2879-g324baf8d (324baf8ddb91dea46cac50cb535c306115791355) squid (dev)": 8 2024-04-12T21:05:47.339 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-04-12T21:05:47.339 INFO:teuthology.orchestra.run.smithi018.stdout:} 2024-04-12T21:05:47.341 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:47.344+0000 7fd387000700 1 -- 172.21.15.18:0/2685627036 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fd36406ec00 msgr2=0x7fd3640710b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:05:47.342 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:47.344+0000 7fd387000700 1 --2- 172.21.15.18:0/2685627036 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fd36406ec00 0x7fd3640710b0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fd36800b020 tx=0x7fd368009e30 comp rx=0 tx=0).stop 2024-04-12T21:05:47.342 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:47.344+0000 7fd387000700 1 -- 172.21.15.18:0/2685627036 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd380105020 msgr2=0x7fd380115ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:05:47.342 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:47.344+0000 7fd387000700 1 --2- 172.21.15.18:0/2685627036 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd380105020 0x7fd380115ba0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fd370002a50 tx=0x7fd3700046b0 comp rx=0 tx=0).stop 2024-04-12T21:05:47.342 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:47.344+0000 7fd387000700 1 -- 172.21.15.18:0/2685627036 shutdown_connections 2024-04-12T21:05:47.342 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:47.344+0000 7fd387000700 1 --2- 172.21.15.18:0/2685627036 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fd36406ec00 0x7fd3640710b0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:05:47.342 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:47.344+0000 7fd387000700 1 --2- 172.21.15.18:0/2685627036 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd380105020 0x7fd380115ba0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:05:47.342 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:47.344+0000 7fd387000700 1 --2- 172.21.15.18:0/2685627036 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd3801059c0 0x7fd3801160e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:05:47.342 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:47.344+0000 7fd387000700 1 -- 172.21.15.18:0/2685627036 >> 172.21.15.18:0/2685627036 conn(0x7fd3800faba0 msgr2=0x7fd3800fbb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:05:47.342 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:47.345+0000 7fd387000700 1 -- 172.21.15.18:0/2685627036 shutdown_connections 2024-04-12T21:05:47.342 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:47.345+0000 7fd387000700 1 -- 172.21.15.18:0/2685627036 wait complete. 2024-04-12T21:05:47.496 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:47.499+0000 7fd3be400700 1 -- 172.21.15.18:0/2994711086 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd3b80724f0 msgr2=0x7fd3b80728c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:05:47.497 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:47.499+0000 7fd3be400700 1 --2- 172.21.15.18:0/2994711086 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd3b80724f0 0x7fd3b80728c0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fd3a8004940 tx=0x7fd3a8009e30 comp rx=0 tx=0).stop 2024-04-12T21:05:47.497 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:47.499+0000 7fd3be400700 1 -- 172.21.15.18:0/2994711086 shutdown_connections 2024-04-12T21:05:47.497 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:47.499+0000 7fd3be400700 1 --2- 172.21.15.18:0/2994711086 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd3b8072e00 0x7fd3b810d770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:05:47.497 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:47.499+0000 7fd3be400700 1 --2- 172.21.15.18:0/2994711086 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd3b80724f0 0x7fd3b80728c0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:05:47.497 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:47.499+0000 7fd3be400700 1 -- 172.21.15.18:0/2994711086 >> 172.21.15.18:0/2994711086 conn(0x7fd3b80fabd0 msgr2=0x7fd3b80fcfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:05:47.497 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:47.499+0000 7fd3be400700 1 -- 172.21.15.18:0/2994711086 shutdown_connections 2024-04-12T21:05:47.497 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:47.499+0000 7fd3be400700 1 -- 172.21.15.18:0/2994711086 wait complete. 2024-04-12T21:05:47.499 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:47.500+0000 7fd3be400700 1 Processor -- start 2024-04-12T21:05:47.499 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:47.500+0000 7fd3be400700 1 -- start start 2024-04-12T21:05:47.499 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:47.501+0000 7fd3be400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd3b80724f0 0x7fd3b811a120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:05:47.499 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:47.501+0000 7fd3be400700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd3b8072e00 0x7fd3b811a660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:05:47.499 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:47.501+0000 7fd3be400700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3b811acb0 con 0x7fd3b80724f0 2024-04-12T21:05:47.499 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:47.501+0000 7fd3be400700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3b8114080 con 0x7fd3b8072e00 2024-04-12T21:05:47.499 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:47.501+0000 7fd3b7400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd3b80724f0 0x7fd3b811a120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:05:47.499 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:47.501+0000 7fd3b7400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd3b80724f0 0x7fd3b811a120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:55862/0 (socket says 172.21.15.18:55862) 2024-04-12T21:05:47.500 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:47.501+0000 7fd3b7400700 1 -- 172.21.15.18:0/1468853105 learned_addr learned my addr 172.21.15.18:0/1468853105 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:05:47.500 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:47.501+0000 7fd3b6a00700 1 --2- 172.21.15.18:0/1468853105 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd3b8072e00 0x7fd3b811a660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:05:47.500 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:47.501+0000 7fd3b7400700 1 -- 172.21.15.18:0/1468853105 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd3b8072e00 msgr2=0x7fd3b811a660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:05:47.500 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:47.501+0000 7fd3b7400700 1 --2- 172.21.15.18:0/1468853105 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd3b8072e00 0x7fd3b811a660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:05:47.500 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:47.501+0000 7fd3b7400700 1 -- 172.21.15.18:0/1468853105 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3a8005040 con 0x7fd3b80724f0 2024-04-12T21:05:47.500 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:47.501+0000 7fd3b6a00700 1 --2- 172.21.15.18:0/1468853105 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd3b8072e00 0x7fd3b811a660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-12T21:05:47.500 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:47.502+0000 7fd3b7400700 1 --2- 172.21.15.18:0/1468853105 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd3b80724f0 0x7fd3b811a120 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fd3a8009180 tx=0x7fd3a8009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:05:47.500 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:47.502+0000 7fd3af400700 1 -- 172.21.15.18:0/1468853105 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd3a800e040 con 0x7fd3b80724f0 2024-04-12T21:05:47.500 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:47.502+0000 7fd3af400700 1 -- 172.21.15.18:0/1468853105 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd3a8013420 con 0x7fd3b80724f0 2024-04-12T21:05:47.500 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:47.502+0000 7fd3be400700 1 -- 172.21.15.18:0/1468853105 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd3b8114300 con 0x7fd3b80724f0 2024-04-12T21:05:47.500 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:47.502+0000 7fd3af400700 1 -- 172.21.15.18:0/1468853105 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd3a801c630 con 0x7fd3b80724f0 2024-04-12T21:05:47.500 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:47.502+0000 7fd3be400700 1 -- 172.21.15.18:0/1468853105 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd3b81148c0 con 0x7fd3b80724f0 2024-04-12T21:05:47.501 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:47.503+0000 7fd3be400700 1 -- 172.21.15.18:0/1468853105 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd3b810a5b0 con 0x7fd3b80724f0 2024-04-12T21:05:47.501 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:47.504+0000 7fd3af400700 1 -- 172.21.15.18:0/1468853105 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7fd3a8013590 con 0x7fd3b80724f0 2024-04-12T21:05:47.502 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:47.505+0000 7fd3af400700 1 --2- 172.21.15.18:0/1468853105 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fd39c06ec10 0x7fd39c0710c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:05:47.502 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:47.505+0000 7fd3af400700 1 -- 172.21.15.18:0/1468853105 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6585+0+0 (secure 0 0 0) 0x7fd3a808fca0 con 0x7fd3b80724f0 2024-04-12T21:05:47.502 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:47.506+0000 7fd3b6a00700 1 --2- 172.21.15.18:0/1468853105 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fd39c06ec10 0x7fd39c0710c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:05:47.503 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:47.506+0000 7fd3b6a00700 1 --2- 172.21.15.18:0/1468853105 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fd39c06ec10 0x7fd39c0710c0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fd3a4009fd0 tx=0x7fd3a400b040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:05:47.507 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:47.510+0000 7fd3af400700 1 -- 172.21.15.18:0/1468853105 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7fd3a805c580 con 0x7fd3b80724f0 2024-04-12T21:05:47.781 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:47.783+0000 7fd3be400700 1 -- 172.21.15.18:0/1468853105 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fd3b81104a0 con 0x7fd3b80724f0 2024-04-12T21:05:47.781 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:47.784+0000 7fd3af400700 1 -- 172.21.15.18:0/1468853105 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 23 v23) v1 ==== 76+0+1886 (secure 0 0 0) 0x7fd3a805c110 con 0x7fd3b80724f0 2024-04-12T21:05:47.782 INFO:teuthology.orchestra.run.smithi018.stdout:e23 2024-04-12T21:05:47.782 INFO:teuthology.orchestra.run.smithi018.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-12T21:05:47.782 INFO:teuthology.orchestra.run.smithi018.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-12T21:05:47.782 INFO:teuthology.orchestra.run.smithi018.stdout:legacy client fscid: 1 2024-04-12T21:05:47.782 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-12T21:05:47.782 INFO:teuthology.orchestra.run.smithi018.stdout:Filesystem 'cephfs' (1) 2024-04-12T21:05:47.782 INFO:teuthology.orchestra.run.smithi018.stdout:fs_name cephfs 2024-04-12T21:05:47.782 INFO:teuthology.orchestra.run.smithi018.stdout:epoch 23 2024-04-12T21:05:47.782 INFO:teuthology.orchestra.run.smithi018.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-04-12T21:05:47.782 INFO:teuthology.orchestra.run.smithi018.stdout:created 2024-04-12T20:51:57.362850+0000 2024-04-12T21:05:47.782 INFO:teuthology.orchestra.run.smithi018.stdout:modified 2024-04-12T21:01:10.670287+0000 2024-04-12T21:05:47.782 INFO:teuthology.orchestra.run.smithi018.stdout:tableserver 0 2024-04-12T21:05:47.783 INFO:teuthology.orchestra.run.smithi018.stdout:root 0 2024-04-12T21:05:47.783 INFO:teuthology.orchestra.run.smithi018.stdout:session_timeout 60 2024-04-12T21:05:47.783 INFO:teuthology.orchestra.run.smithi018.stdout:session_autoclose 300 2024-04-12T21:05:47.783 INFO:teuthology.orchestra.run.smithi018.stdout:max_file_size 1099511627776 2024-04-12T21:05:47.783 INFO:teuthology.orchestra.run.smithi018.stdout:max_xattr_size 65536 2024-04-12T21:05:47.783 INFO:teuthology.orchestra.run.smithi018.stdout:required_client_features {} 2024-04-12T21:05:47.783 INFO:teuthology.orchestra.run.smithi018.stdout:last_failure 0 2024-04-12T21:05:47.783 INFO:teuthology.orchestra.run.smithi018.stdout:last_failure_osd_epoch 46 2024-04-12T21:05:47.783 INFO:teuthology.orchestra.run.smithi018.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-12T21:05:47.783 INFO:teuthology.orchestra.run.smithi018.stdout:max_mds 2 2024-04-12T21:05:47.783 INFO:teuthology.orchestra.run.smithi018.stdout:in 0,1 2024-04-12T21:05:47.783 INFO:teuthology.orchestra.run.smithi018.stdout:up {0=14452,1=14464} 2024-04-12T21:05:47.783 INFO:teuthology.orchestra.run.smithi018.stdout:failed 2024-04-12T21:05:47.783 INFO:teuthology.orchestra.run.smithi018.stdout:damaged 2024-04-12T21:05:47.784 INFO:teuthology.orchestra.run.smithi018.stdout:stopped 2024-04-12T21:05:47.784 INFO:teuthology.orchestra.run.smithi018.stdout:data_pools [3] 2024-04-12T21:05:47.784 INFO:teuthology.orchestra.run.smithi018.stdout:metadata_pool 2 2024-04-12T21:05:47.784 INFO:teuthology.orchestra.run.smithi018.stdout:inline_data disabled 2024-04-12T21:05:47.784 INFO:teuthology.orchestra.run.smithi018.stdout:balancer 2024-04-12T21:05:47.784 INFO:teuthology.orchestra.run.smithi018.stdout:bal_rank_mask -1 2024-04-12T21:05:47.784 INFO:teuthology.orchestra.run.smithi018.stdout:standby_count_wanted 1 2024-04-12T21:05:47.784 INFO:teuthology.orchestra.run.smithi018.stdout:qdb_cluster leader: 0 members: 2024-04-12T21:05:47.784 INFO:teuthology.orchestra.run.smithi018.stdout:[mds.cephfs.smithi018.acqbsc{0:14452} state up:active seq 5 join_fscid=1 addr [v2:172.21.15.18:6826/1564065434,v1:172.21.15.18:6827/1564065434] compat {c=[1],r=[1],i=[7ff]}] 2024-04-12T21:05:47.784 INFO:teuthology.orchestra.run.smithi018.stdout:[mds.cephfs.smithi018.szjluo{1:14464} state up:active seq 6 join_fscid=1 addr [v2:172.21.15.18:6828/722443353,v1:172.21.15.18:6829/722443353] compat {c=[1],r=[1],i=[7ff]}] 2024-04-12T21:05:47.784 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-12T21:05:47.784 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-12T21:05:47.784 INFO:teuthology.orchestra.run.smithi018.stdout:Standby daemons: 2024-04-12T21:05:47.784 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-12T21:05:47.784 INFO:teuthology.orchestra.run.smithi018.stdout:[mds.cephfs.smithi186.wjnafs{-1:24277} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.186:6826/438347207,v1:172.21.15.186:6827/438347207] compat {c=[1],r=[1],i=[7ff]}] 2024-04-12T21:05:47.785 INFO:teuthology.orchestra.run.smithi018.stdout:[mds.cephfs.smithi186.jdllyn{-1:24281} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.186:6824/1464359952,v1:172.21.15.186:6825/1464359952] compat {c=[1],r=[1],i=[7ff]}] 2024-04-12T21:05:47.786 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:47.787+0000 7fd3be400700 1 -- 172.21.15.18:0/1468853105 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fd39c06ec10 msgr2=0x7fd39c0710c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:05:47.786 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:47.787+0000 7fd3be400700 1 --2- 172.21.15.18:0/1468853105 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fd39c06ec10 0x7fd39c0710c0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fd3a4009fd0 tx=0x7fd3a400b040 comp rx=0 tx=0).stop 2024-04-12T21:05:47.786 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:47.787+0000 7fd3be400700 1 -- 172.21.15.18:0/1468853105 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd3b80724f0 msgr2=0x7fd3b811a120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:05:47.786 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:47.787+0000 7fd3be400700 1 --2- 172.21.15.18:0/1468853105 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd3b80724f0 0x7fd3b811a120 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fd3a8009180 tx=0x7fd3a8009e30 comp rx=0 tx=0).stop 2024-04-12T21:05:47.786 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:47.787+0000 7fd3be400700 1 -- 172.21.15.18:0/1468853105 shutdown_connections 2024-04-12T21:05:47.786 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:47.787+0000 7fd3be400700 1 --2- 172.21.15.18:0/1468853105 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fd39c06ec10 0x7fd39c0710c0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:05:47.786 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:47.787+0000 7fd3be400700 1 --2- 172.21.15.18:0/1468853105 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd3b80724f0 0x7fd3b811a120 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:05:47.786 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:47.787+0000 7fd3be400700 1 --2- 172.21.15.18:0/1468853105 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd3b8072e00 0x7fd3b811a660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:05:47.786 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:47.787+0000 7fd3be400700 1 -- 172.21.15.18:0/1468853105 >> 172.21.15.18:0/1468853105 conn(0x7fd3b80fabd0 msgr2=0x7fd3b8107730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:05:47.786 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:47.787+0000 7fd3be400700 1 -- 172.21.15.18:0/1468853105 shutdown_connections 2024-04-12T21:05:47.786 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:47.787+0000 7fd3be400700 1 -- 172.21.15.18:0/1468853105 wait complete. 2024-04-12T21:05:47.786 INFO:teuthology.orchestra.run.smithi018.stderr:dumped fsmap epoch 23 2024-04-12T21:05:47.932 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:47.934+0000 7fb59d400700 1 -- 172.21.15.18:0/48736358 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb5981092a0 msgr2=0x7fb598109670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:05:47.932 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:47.934+0000 7fb59d400700 1 --2- 172.21.15.18:0/48736358 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb5981092a0 0x7fb598109670 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fb588004940 tx=0x7fb588009e30 comp rx=0 tx=0).stop 2024-04-12T21:05:47.932 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:47.934+0000 7fb59d400700 1 -- 172.21.15.18:0/48736358 shutdown_connections 2024-04-12T21:05:47.932 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:47.934+0000 7fb59d400700 1 --2- 172.21.15.18:0/48736358 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb598109c40 0x7fb5981032b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:05:47.932 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:47.934+0000 7fb59d400700 1 --2- 172.21.15.18:0/48736358 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb5981092a0 0x7fb598109670 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:05:47.932 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:47.934+0000 7fb59d400700 1 -- 172.21.15.18:0/48736358 >> 172.21.15.18:0/48736358 conn(0x7fb598072a30 msgr2=0x7fb598074e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:05:47.932 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:47.934+0000 7fb59d400700 1 -- 172.21.15.18:0/48736358 shutdown_connections 2024-04-12T21:05:47.932 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:47.934+0000 7fb59d400700 1 -- 172.21.15.18:0/48736358 wait complete. 2024-04-12T21:05:47.933 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:47.935+0000 7fb59d400700 1 Processor -- start 2024-04-12T21:05:47.933 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:47.935+0000 7fb59d400700 1 -- start start 2024-04-12T21:05:47.934 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:47.936+0000 7fb59d400700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb5981092a0 0x7fb598119f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:05:47.934 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:47.936+0000 7fb59d400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb598109c40 0x7fb59811a4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:05:47.934 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:47.936+0000 7fb59d400700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb59811aba0 con 0x7fb598109c40 2024-04-12T21:05:47.934 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:47.936+0000 7fb59d400700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb598113ee0 con 0x7fb5981092a0 2024-04-12T21:05:47.934 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:47.936+0000 7fb58fe00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb598109c40 0x7fb59811a4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:05:47.934 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:47.936+0000 7fb58fe00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb598109c40 0x7fb59811a4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:55884/0 (socket says 172.21.15.18:55884) 2024-04-12T21:05:47.934 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:47.936+0000 7fb58fe00700 1 -- 172.21.15.18:0/1527238567 learned_addr learned my addr 172.21.15.18:0/1527238567 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:05:47.934 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:47.937+0000 7fb597e00700 1 --2- 172.21.15.18:0/1527238567 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb5981092a0 0x7fb598119f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:05:47.934 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:47.937+0000 7fb58fe00700 1 -- 172.21.15.18:0/1527238567 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb5981092a0 msgr2=0x7fb598119f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:05:47.934 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:47.937+0000 7fb58fe00700 1 --2- 172.21.15.18:0/1527238567 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb5981092a0 0x7fb598119f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:05:47.934 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:47.937+0000 7fb58fe00700 1 -- 172.21.15.18:0/1527238567 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb588005040 con 0x7fb598109c40 2024-04-12T21:05:47.935 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:47.937+0000 7fb58fe00700 1 --2- 172.21.15.18:0/1527238567 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb598109c40 0x7fb59811a4c0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fb58000b790 tx=0x7fb580009220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:05:47.935 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:47.937+0000 7fb594c00700 1 -- 172.21.15.18:0/1527238567 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb58000f6f0 con 0x7fb598109c40 2024-04-12T21:05:47.935 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:47.937+0000 7fb594c00700 1 -- 172.21.15.18:0/1527238567 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb58000fd20 con 0x7fb598109c40 2024-04-12T21:05:47.935 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:47.937+0000 7fb59d400700 1 -- 172.21.15.18:0/1527238567 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb598114160 con 0x7fb598109c40 2024-04-12T21:05:47.935 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:47.937+0000 7fb594c00700 1 -- 172.21.15.18:0/1527238567 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb5800103a0 con 0x7fb598109c40 2024-04-12T21:05:47.935 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:47.937+0000 7fb59d400700 1 -- 172.21.15.18:0/1527238567 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb598114720 con 0x7fb598109c40 2024-04-12T21:05:47.936 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:47.939+0000 7fb59d400700 1 -- 172.21.15.18:0/1527238567 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb5981049f0 con 0x7fb598109c40 2024-04-12T21:05:47.936 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:47.940+0000 7fb594c00700 1 -- 172.21.15.18:0/1527238567 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7fb580011400 con 0x7fb598109c40 2024-04-12T21:05:47.937 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:47.941+0000 7fb594c00700 1 --2- 172.21.15.18:0/1527238567 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fb57c06ec00 0x7fb57c0710b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:05:47.938 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:47.941+0000 7fb594c00700 1 -- 172.21.15.18:0/1527238567 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6585+0+0 (secure 0 0 0) 0x7fb58008e7a0 con 0x7fb598109c40 2024-04-12T21:05:47.938 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:47.941+0000 7fb597e00700 1 --2- 172.21.15.18:0/1527238567 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fb57c06ec00 0x7fb57c0710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:05:47.938 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:47.942+0000 7fb597e00700 1 --2- 172.21.15.18:0/1527238567 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fb57c06ec00 0x7fb57c0710b0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fb588004f90 tx=0x7fb588009e30 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:05:47.942 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:47.945+0000 7fb594c00700 1 -- 172.21.15.18:0/1527238567 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7fb58005b020 con 0x7fb598109c40 2024-04-12T21:05:48.100 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:05:47 smithi018 ceph-mon[112590]: from='client.34350 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:05:48.101 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:05:47 smithi018 ceph-mon[112590]: from='client.? 172.21.15.18:0/2685627036' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T21:05:48.101 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:05:47 smithi018 ceph-mon[112590]: from='client.? 172.21.15.18:0/1468853105' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-12T21:05:48.185 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:48.188+0000 7fb59d400700 1 -- 172.21.15.18:0/1527238567 --> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fb598110480 con 0x7fb57c06ec00 2024-04-12T21:05:48.187 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:48.190+0000 7fb594c00700 1 -- 172.21.15.18:0/1527238567 <== mgr.34102 v2:172.21.15.18:6800/3812891919 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+401 (secure 0 0 0) 0x7fb598110480 con 0x7fb57c06ec00 2024-04-12T21:05:48.187 INFO:teuthology.orchestra.run.smithi018.stdout:{ 2024-04-12T21:05:48.187 INFO:teuthology.orchestra.run.smithi018.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:f4452def3be4d350f4a8e5feaf7ebb1511be9298fe68478ba7a2d6fbd7d69a63", 2024-04-12T21:05:48.187 INFO:teuthology.orchestra.run.smithi018.stdout: "in_progress": true, 2024-04-12T21:05:48.187 INFO:teuthology.orchestra.run.smithi018.stdout: "which": "Upgrading all daemon types on all hosts", 2024-04-12T21:05:48.188 INFO:teuthology.orchestra.run.smithi018.stdout: "services_complete": [ 2024-04-12T21:05:48.188 INFO:teuthology.orchestra.run.smithi018.stdout: "crash", 2024-04-12T21:05:48.188 INFO:teuthology.orchestra.run.smithi018.stdout: "mon", 2024-04-12T21:05:48.188 INFO:teuthology.orchestra.run.smithi018.stdout: "mgr" 2024-04-12T21:05:48.188 INFO:teuthology.orchestra.run.smithi018.stdout: ], 2024-04-12T21:05:48.188 INFO:teuthology.orchestra.run.smithi018.stdout: "progress": "10/21 daemons upgraded", 2024-04-12T21:05:48.188 INFO:teuthology.orchestra.run.smithi018.stdout: "message": "Currently upgrading osd daemons", 2024-04-12T21:05:48.188 INFO:teuthology.orchestra.run.smithi018.stdout: "is_paused": false 2024-04-12T21:05:48.188 INFO:teuthology.orchestra.run.smithi018.stdout:} 2024-04-12T21:05:48.190 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:48.193+0000 7fb59d400700 1 -- 172.21.15.18:0/1527238567 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fb57c06ec00 msgr2=0x7fb57c0710b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:05:48.190 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:48.193+0000 7fb59d400700 1 --2- 172.21.15.18:0/1527238567 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fb57c06ec00 0x7fb57c0710b0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fb588004f90 tx=0x7fb588009e30 comp rx=0 tx=0).stop 2024-04-12T21:05:48.190 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:48.193+0000 7fb59d400700 1 -- 172.21.15.18:0/1527238567 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb598109c40 msgr2=0x7fb59811a4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:05:48.190 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:48.193+0000 7fb59d400700 1 --2- 172.21.15.18:0/1527238567 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb598109c40 0x7fb59811a4c0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fb58000b790 tx=0x7fb580009220 comp rx=0 tx=0).stop 2024-04-12T21:05:48.190 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:48.193+0000 7fb59d400700 1 -- 172.21.15.18:0/1527238567 shutdown_connections 2024-04-12T21:05:48.191 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:48.193+0000 7fb59d400700 1 --2- 172.21.15.18:0/1527238567 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fb57c06ec00 0x7fb57c0710b0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:05:48.191 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:48.193+0000 7fb59d400700 1 --2- 172.21.15.18:0/1527238567 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb5981092a0 0x7fb598119f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:05:48.191 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:48.193+0000 7fb59d400700 1 --2- 172.21.15.18:0/1527238567 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb598109c40 0x7fb59811a4c0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:05:48.191 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:48.193+0000 7fb59d400700 1 -- 172.21.15.18:0/1527238567 >> 172.21.15.18:0/1527238567 conn(0x7fb598072a30 msgr2=0x7fb5980748f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:05:48.191 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:48.193+0000 7fb59d400700 1 -- 172.21.15.18:0/1527238567 shutdown_connections 2024-04-12T21:05:48.191 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:48.193+0000 7fb59d400700 1 -- 172.21.15.18:0/1527238567 wait complete. 2024-04-12T21:05:48.279 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:05:47 smithi186 ceph-mon[86379]: from='client.34350 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:05:48.279 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:05:47 smithi186 ceph-mon[86379]: from='client.? 172.21.15.18:0/2685627036' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T21:05:48.279 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:05:47 smithi186 ceph-mon[86379]: from='client.? 172.21.15.18:0/1468853105' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-12T21:05:48.348 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:48.351+0000 7efc58200700 1 -- 172.21.15.18:0/4157066461 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7efc500fe870 msgr2=0x7efc50106f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:05:48.349 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:48.351+0000 7efc58200700 1 --2- 172.21.15.18:0/4157066461 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7efc500fe870 0x7efc50106f40 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7efc3c004940 tx=0x7efc3c009e30 comp rx=0 tx=0).stop 2024-04-12T21:05:48.349 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:48.351+0000 7efc58200700 1 -- 172.21.15.18:0/4157066461 shutdown_connections 2024-04-12T21:05:48.349 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:48.351+0000 7efc58200700 1 --2- 172.21.15.18:0/4157066461 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7efc500fe870 0x7efc50106f40 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:05:48.349 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:48.351+0000 7efc58200700 1 --2- 172.21.15.18:0/4157066461 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7efc500fded0 0x7efc500fe2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:05:48.349 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:48.351+0000 7efc58200700 1 -- 172.21.15.18:0/4157066461 >> 172.21.15.18:0/4157066461 conn(0x7efc50072bc0 msgr2=0x7efc50072fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:05:48.349 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:48.352+0000 7efc58200700 1 -- 172.21.15.18:0/4157066461 shutdown_connections 2024-04-12T21:05:48.349 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:48.352+0000 7efc58200700 1 -- 172.21.15.18:0/4157066461 wait complete. 2024-04-12T21:05:48.351 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:48.352+0000 7efc58200700 1 Processor -- start 2024-04-12T21:05:48.351 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:48.353+0000 7efc58200700 1 -- start start 2024-04-12T21:05:48.351 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:48.353+0000 7efc58200700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7efc500fded0 0x7efc50113b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:05:48.351 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:48.353+0000 7efc58200700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7efc500fe870 0x7efc50114040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:05:48.351 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:48.353+0000 7efc58200700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc50114690 con 0x7efc500fe870 2024-04-12T21:05:48.351 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:48.353+0000 7efc58200700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc5010da80 con 0x7efc500fded0 2024-04-12T21:05:48.351 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:48.353+0000 7efc54e00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7efc500fe870 0x7efc50114040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:05:48.351 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:48.353+0000 7efc54e00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7efc500fe870 0x7efc50114040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:55916/0 (socket says 172.21.15.18:55916) 2024-04-12T21:05:48.351 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:48.353+0000 7efc54e00700 1 -- 172.21.15.18:0/1659030451 learned_addr learned my addr 172.21.15.18:0/1659030451 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:05:48.351 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:48.354+0000 7efc55800700 1 --2- 172.21.15.18:0/1659030451 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7efc500fded0 0x7efc50113b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:05:48.351 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:48.354+0000 7efc54e00700 1 -- 172.21.15.18:0/1659030451 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7efc500fded0 msgr2=0x7efc50113b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:05:48.351 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:48.354+0000 7efc54e00700 1 --2- 172.21.15.18:0/1659030451 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7efc500fded0 0x7efc50113b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:05:48.351 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:48.354+0000 7efc54e00700 1 -- 172.21.15.18:0/1659030451 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc3c005040 con 0x7efc500fe870 2024-04-12T21:05:48.352 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:48.354+0000 7efc54e00700 1 --2- 172.21.15.18:0/1659030451 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7efc500fe870 0x7efc50114040 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7efc3c000c00 tx=0x7efc3c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:05:48.352 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:48.355+0000 7efc4d600700 1 -- 172.21.15.18:0/1659030451 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7efc3c021070 con 0x7efc500fe870 2024-04-12T21:05:48.352 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:48.355+0000 7efc4d600700 1 -- 172.21.15.18:0/1659030451 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7efc3c013c00 con 0x7efc500fe870 2024-04-12T21:05:48.352 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:48.355+0000 7efc58200700 1 -- 172.21.15.18:0/1659030451 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efc5010dd00 con 0x7efc500fe870 2024-04-12T21:05:48.352 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:48.355+0000 7efc4d600700 1 -- 172.21.15.18:0/1659030451 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7efc3c00e070 con 0x7efc500fe870 2024-04-12T21:05:48.352 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:48.355+0000 7efc58200700 1 -- 172.21.15.18:0/1659030451 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efc5010e240 con 0x7efc500fe870 2024-04-12T21:05:48.353 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:48.356+0000 7efc58200700 1 -- 172.21.15.18:0/1659030451 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efc50065df0 con 0x7efc500fe870 2024-04-12T21:05:48.354 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:48.357+0000 7efc4d600700 1 -- 172.21.15.18:0/1659030451 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7efc3c01c9d0 con 0x7efc500fe870 2024-04-12T21:05:48.355 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:48.358+0000 7efc4d600700 1 --2- 172.21.15.18:0/1659030451 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7efc3806ec10 0x7efc380710c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:05:48.355 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:48.358+0000 7efc4d600700 1 -- 172.21.15.18:0/1659030451 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6585+0+0 (secure 0 0 0) 0x7efc3c08fda0 con 0x7efc500fe870 2024-04-12T21:05:48.355 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:48.358+0000 7efc55800700 1 --2- 172.21.15.18:0/1659030451 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7efc3806ec10 0x7efc380710c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:05:48.356 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:48.359+0000 7efc55800700 1 --2- 172.21.15.18:0/1659030451 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7efc3806ec10 0x7efc380710c0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7efc44009b80 tx=0x7efc4400b040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:05:48.360 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:48.363+0000 7efc4d600700 1 -- 172.21.15.18:0/1659030451 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7efc3c05c600 con 0x7efc500fe870 2024-04-12T21:05:48.689 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:48.692+0000 7efc58200700 1 -- 172.21.15.18:0/1659030451 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7efc50109e90 con 0x7efc500fe870 2024-04-12T21:05:48.690 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:48.693+0000 7efc4d600700 1 -- 172.21.15.18:0/1659030451 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+10 (secure 0 0 0) 0x7efc3c05c190 con 0x7efc500fe870 2024-04-12T21:05:48.690 INFO:teuthology.orchestra.run.smithi018.stdout:HEALTH_OK 2024-04-12T21:05:48.693 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:48.696+0000 7efc58200700 1 -- 172.21.15.18:0/1659030451 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7efc3806ec10 msgr2=0x7efc380710c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:05:48.694 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:48.696+0000 7efc58200700 1 --2- 172.21.15.18:0/1659030451 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7efc3806ec10 0x7efc380710c0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7efc44009b80 tx=0x7efc4400b040 comp rx=0 tx=0).stop 2024-04-12T21:05:48.694 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:48.696+0000 7efc58200700 1 -- 172.21.15.18:0/1659030451 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7efc500fe870 msgr2=0x7efc50114040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:05:48.694 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:48.696+0000 7efc58200700 1 --2- 172.21.15.18:0/1659030451 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7efc500fe870 0x7efc50114040 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7efc3c000c00 tx=0x7efc3c009e30 comp rx=0 tx=0).stop 2024-04-12T21:05:48.694 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:48.696+0000 7efc58200700 1 -- 172.21.15.18:0/1659030451 shutdown_connections 2024-04-12T21:05:48.694 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:48.696+0000 7efc58200700 1 --2- 172.21.15.18:0/1659030451 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7efc3806ec10 0x7efc380710c0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:05:48.694 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:48.696+0000 7efc58200700 1 --2- 172.21.15.18:0/1659030451 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7efc500fded0 0x7efc50113b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:05:48.694 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:48.696+0000 7efc58200700 1 --2- 172.21.15.18:0/1659030451 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7efc500fe870 0x7efc50114040 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:05:48.694 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:48.696+0000 7efc58200700 1 -- 172.21.15.18:0/1659030451 >> 172.21.15.18:0/1659030451 conn(0x7efc50072bc0 msgr2=0x7efc50103790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:05:48.694 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:48.696+0000 7efc58200700 1 -- 172.21.15.18:0/1659030451 shutdown_connections 2024-04-12T21:05:48.694 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:05:48.697+0000 7efc58200700 1 -- 172.21.15.18:0/1659030451 wait complete. 2024-04-12T21:05:49.230 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:05:48 smithi186 ceph-mon[86379]: pgmap v166: 65 pgs: 65 active+clean; 257 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-04-12T21:05:49.230 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:05:48 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd ok-to-stop", "ids": ["4"], "max": 16}]: dispatch 2024-04-12T21:05:49.230 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:05:48 smithi186 ceph-mon[86379]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["4"], "max": 16}]: dispatch 2024-04-12T21:05:49.230 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:05:48 smithi186 ceph-mon[86379]: Upgrade: osd.4 is safe to restart 2024-04-12T21:05:49.230 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:05:48 smithi186 ceph-mon[86379]: from='client.34362 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:05:49.230 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:05:48 smithi186 ceph-mon[86379]: from='client.? 172.21.15.18:0/1659030451' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-12T21:05:49.350 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:05:48 smithi018 ceph-mon[112590]: pgmap v166: 65 pgs: 65 active+clean; 257 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-04-12T21:05:49.351 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:05:48 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd ok-to-stop", "ids": ["4"], "max": 16}]: dispatch 2024-04-12T21:05:49.351 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:05:48 smithi018 ceph-mon[112590]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["4"], "max": 16}]: dispatch 2024-04-12T21:05:49.351 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:05:48 smithi018 ceph-mon[112590]: Upgrade: osd.4 is safe to restart 2024-04-12T21:05:49.351 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:05:48 smithi018 ceph-mon[112590]: from='client.34362 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:05:49.351 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:05:48 smithi018 ceph-mon[112590]: from='client.? 172.21.15.18:0/1659030451' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-12T21:05:51.529 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:05:51 smithi186 ceph-mon[86379]: pgmap v167: 65 pgs: 65 active+clean; 257 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-04-12T21:05:51.529 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:05:51 smithi186 ceph-mon[86379]: Upgrade: Updating osd.4 2024-04-12T21:05:51.530 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:05:51 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' 2024-04-12T21:05:51.530 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:05:51 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-04-12T21:05:51.530 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:05:51 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-12T21:05:51.530 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:05:51 smithi186 ceph-mon[86379]: Deploying daemon osd.4 on smithi186 2024-04-12T21:05:51.850 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:05:51 smithi018 ceph-mon[112590]: pgmap v167: 65 pgs: 65 active+clean; 257 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-04-12T21:05:51.851 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:05:51 smithi018 ceph-mon[112590]: Upgrade: Updating osd.4 2024-04-12T21:05:51.851 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:05:51 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' 2024-04-12T21:05:51.851 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:05:51 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-04-12T21:05:51.851 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:05:51 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-12T21:05:51.851 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:05:51 smithi018 ceph-mon[112590]: Deploying daemon osd.4 on smithi186 2024-04-12T21:05:53.530 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:05:53 smithi186 ceph-mon[86379]: pgmap v168: 65 pgs: 65 active+clean; 257 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-04-12T21:05:53.850 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:05:53 smithi018 ceph-mon[112590]: pgmap v168: 65 pgs: 65 active+clean; 257 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-04-12T21:05:55.715 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:05:55 smithi186 ceph-mon[86379]: pgmap v169: 65 pgs: 65 active+clean; 257 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-04-12T21:05:55.715 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:05:55 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T21:05:55.715 INFO:journalctl@ceph.osd.4.smithi186.stdout:Apr 12 21:05:55 smithi186 systemd[1]: Stopping Ceph osd.4 for 1cbe407a-f90d-11ee-bc8d-c7b262605968... 2024-04-12T21:05:55.850 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:05:55 smithi018 ceph-mon[112590]: pgmap v169: 65 pgs: 65 active+clean; 257 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-04-12T21:05:55.850 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:05:55 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T21:05:56.030 INFO:journalctl@ceph.osd.4.smithi186.stdout:Apr 12 21:05:55 smithi186 ceph-1cbe407a-f90d-11ee-bc8d-c7b262605968-osd-4[48457]: 2024-04-12T21:05:55.717+0000 7f5d5ae00700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.4 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-12T21:05:56.030 INFO:journalctl@ceph.osd.4.smithi186.stdout:Apr 12 21:05:55 smithi186 ceph-1cbe407a-f90d-11ee-bc8d-c7b262605968-osd-4[48457]: 2024-04-12T21:05:55.717+0000 7f5d5ae00700 -1 osd.4 88 *** Got signal Terminated *** 2024-04-12T21:05:56.030 INFO:journalctl@ceph.osd.4.smithi186.stdout:Apr 12 21:05:55 smithi186 ceph-1cbe407a-f90d-11ee-bc8d-c7b262605968-osd-4[48457]: 2024-04-12T21:05:55.717+0000 7f5d5ae00700 -1 osd.4 88 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-12T21:05:56.779 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:05:56 smithi186 ceph-mon[86379]: osd.4 marked itself down and dead 2024-04-12T21:05:56.850 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:05:56 smithi018 ceph-mon[112590]: osd.4 marked itself down and dead 2024-04-12T21:05:57.107 INFO:journalctl@ceph.osd.4.smithi186.stdout:Apr 12 21:05:56 smithi186 podman[100253]: 2024-04-12 21:05:56.838455252 +0000 UTC m=+1.235021524 container died 757e7c71b7b92ec0278e26d2cf953b4392deca638f05ce7d7b1d279f7bc08f5a (image=quay.ceph.io/ceph-ci/ceph@sha256:5e334df698f432180b78bc9c46936bfdfbc92bc9a8983463e523f780d23765fe, name=ceph-1cbe407a-f90d-11ee-bc8d-c7b262605968-osd-4, org.label-schema.schema-version=1.0, RELEASE=quincy-d3b5060, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240409, GIT_CLEAN=True, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS) 2024-04-12T21:05:57.383 INFO:journalctl@ceph.osd.4.smithi186.stdout:Apr 12 21:05:57 smithi186 podman[100253]: 2024-04-12 21:05:57.27825451 +0000 UTC m=+1.674820776 container cleanup 757e7c71b7b92ec0278e26d2cf953b4392deca638f05ce7d7b1d279f7bc08f5a (image=quay.ceph.io/ceph-ci/ceph@sha256:5e334df698f432180b78bc9c46936bfdfbc92bc9a8983463e523f780d23765fe, name=ceph-1cbe407a-f90d-11ee-bc8d-c7b262605968-osd-4, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.build-date=20240409, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, RELEASE=quincy-d3b5060, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0) 2024-04-12T21:05:57.383 INFO:journalctl@ceph.osd.4.smithi186.stdout:Apr 12 21:05:57 smithi186 bash[100253]: ceph-1cbe407a-f90d-11ee-bc8d-c7b262605968-osd-4 2024-04-12T21:05:57.779 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:05:57 smithi186 ceph-mon[86379]: pgmap v170: 65 pgs: 65 active+clean; 257 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-04-12T21:05:57.779 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:05:57 smithi186 ceph-mon[86379]: Health check failed: 1 osds down (OSD_DOWN) 2024-04-12T21:05:57.779 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:05:57 smithi186 ceph-mon[86379]: osdmap e89: 6 total, 5 up, 6 in 2024-04-12T21:05:57.780 INFO:journalctl@ceph.osd.4.smithi186.stdout:Apr 12 21:05:57 smithi186 podman[100270]: 2024-04-12 21:05:57.512871528 +0000 UTC m=+0.664959346 container remove 757e7c71b7b92ec0278e26d2cf953b4392deca638f05ce7d7b1d279f7bc08f5a (image=quay.ceph.io/ceph-ci/ceph@sha256:5e334df698f432180b78bc9c46936bfdfbc92bc9a8983463e523f780d23765fe, name=ceph-1cbe407a-f90d-11ee-bc8d-c7b262605968-osd-4, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, ceph=True, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, RELEASE=quincy-d3b5060, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240409, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5) 2024-04-12T21:05:57.850 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:05:57 smithi018 ceph-mon[112590]: pgmap v170: 65 pgs: 65 active+clean; 257 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-04-12T21:05:57.851 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:05:57 smithi018 ceph-mon[112590]: Health check failed: 1 osds down (OSD_DOWN) 2024-04-12T21:05:57.851 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:05:57 smithi018 ceph-mon[112590]: osdmap e89: 6 total, 5 up, 6 in 2024-04-12T21:05:58.714 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:05:58 smithi186 ceph-mon[86379]: osdmap e90: 6 total, 5 up, 6 in 2024-04-12T21:05:58.714 INFO:journalctl@ceph.osd.4.smithi186.stdout:Apr 12 21:05:58 smithi186 podman[100356]: 2024-04-12 21:05:58.248186425 +0000 UTC m=+0.032644185 image pull 8717720dc5e5c56303818ea4d4f0a11f6e2c03dee521fb668f411f1304e0afca quay.ceph.io/ceph-ci/ceph@sha256:f4452def3be4d350f4a8e5feaf7ebb1511be9298fe68478ba7a2d6fbd7d69a63 2024-04-12T21:05:58.714 INFO:journalctl@ceph.osd.4.smithi186.stdout:Apr 12 21:05:58 smithi186 podman[100356]: 2024-04-12 21:05:58.389082972 +0000 UTC m=+0.173540763 container create 50c0fa643eaf4fd5a9aefd54516c431e30cffc2bca8964e3e0fbd62d6350b681 (image=quay.ceph.io/ceph-ci/ceph@sha256:f4452def3be4d350f4a8e5feaf7ebb1511be9298fe68478ba7a2d6fbd7d69a63, name=ceph-1cbe407a-f90d-11ee-bc8d-c7b262605968-osd-4-deactivate, RELEASE=wip-vshankar-testing-20240411.061452-324baf8, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.build-date=20240409, ceph=True) 2024-04-12T21:05:58.714 INFO:journalctl@ceph.osd.4.smithi186.stdout:Apr 12 21:05:58 smithi186 podman[100356]: 2024-04-12 21:05:58.717766777 +0000 UTC m=+0.502224575 container init 50c0fa643eaf4fd5a9aefd54516c431e30cffc2bca8964e3e0fbd62d6350b681 (image=quay.ceph.io/ceph-ci/ceph@sha256:f4452def3be4d350f4a8e5feaf7ebb1511be9298fe68478ba7a2d6fbd7d69a63, name=ceph-1cbe407a-f90d-11ee-bc8d-c7b262605968-osd-4-deactivate, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, RELEASE=wip-vshankar-testing-20240411.061452-324baf8, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.build-date=20240409, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-12T21:05:58.850 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:05:58 smithi018 ceph-mon[112590]: osdmap e90: 6 total, 5 up, 6 in 2024-04-12T21:05:59.029 INFO:journalctl@ceph.osd.4.smithi186.stdout:Apr 12 21:05:58 smithi186 podman[100356]: 2024-04-12 21:05:58.730926159 +0000 UTC m=+0.515383960 container start 50c0fa643eaf4fd5a9aefd54516c431e30cffc2bca8964e3e0fbd62d6350b681 (image=quay.ceph.io/ceph-ci/ceph@sha256:f4452def3be4d350f4a8e5feaf7ebb1511be9298fe68478ba7a2d6fbd7d69a63, name=ceph-1cbe407a-f90d-11ee-bc8d-c7b262605968-osd-4-deactivate, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, RELEASE=wip-vshankar-testing-20240411.061452-324baf8, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.build-date=20240409) 2024-04-12T21:05:59.029 INFO:journalctl@ceph.osd.4.smithi186.stdout:Apr 12 21:05:58 smithi186 podman[100356]: 2024-04-12 21:05:58.808184667 +0000 UTC m=+0.592642430 container attach 50c0fa643eaf4fd5a9aefd54516c431e30cffc2bca8964e3e0fbd62d6350b681 (image=quay.ceph.io/ceph-ci/ceph@sha256:f4452def3be4d350f4a8e5feaf7ebb1511be9298fe68478ba7a2d6fbd7d69a63, name=ceph-1cbe407a-f90d-11ee-bc8d-c7b262605968-osd-4-deactivate, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240409, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, ceph=True, RELEASE=wip-vshankar-testing-20240411.061452-324baf8, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-12T21:05:59.530 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:05:59 smithi186 ceph-mon[86379]: pgmap v173: 65 pgs: 8 peering, 4 stale+active+clean, 53 active+clean; 257 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-04-12T21:05:59.530 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:05:59 smithi186 ceph-mon[86379]: Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY) 2024-04-12T21:05:59.530 INFO:journalctl@ceph.osd.4.smithi186.stdout:Apr 12 21:05:59 smithi186 podman[100356]: 2024-04-12 21:05:59.177424898 +0000 UTC m=+0.961882696 container died 50c0fa643eaf4fd5a9aefd54516c431e30cffc2bca8964e3e0fbd62d6350b681 (image=quay.ceph.io/ceph-ci/ceph@sha256:f4452def3be4d350f4a8e5feaf7ebb1511be9298fe68478ba7a2d6fbd7d69a63, name=ceph-1cbe407a-f90d-11ee-bc8d-c7b262605968-osd-4-deactivate, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240409, CEPH_POINT_RELEASE=, ceph=True, RELEASE=wip-vshankar-testing-20240411.061452-324baf8, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS) 2024-04-12T21:05:59.850 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:05:59 smithi018 ceph-mon[112590]: pgmap v173: 65 pgs: 8 peering, 4 stale+active+clean, 53 active+clean; 257 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-04-12T21:05:59.850 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:05:59 smithi018 ceph-mon[112590]: Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY) 2024-04-12T21:06:00.279 INFO:journalctl@ceph.osd.4.smithi186.stdout:Apr 12 21:05:59 smithi186 podman[100381]: 2024-04-12 21:05:59.869991505 +0000 UTC m=+0.668592620 container remove 50c0fa643eaf4fd5a9aefd54516c431e30cffc2bca8964e3e0fbd62d6350b681 (image=quay.ceph.io/ceph-ci/ceph@sha256:f4452def3be4d350f4a8e5feaf7ebb1511be9298fe68478ba7a2d6fbd7d69a63, name=ceph-1cbe407a-f90d-11ee-bc8d-c7b262605968-osd-4-deactivate, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240409, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-vshankar-testing-20240411.061452-324baf8, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, ceph=True, org.label-schema.schema-version=1.0) 2024-04-12T21:06:00.279 INFO:journalctl@ceph.osd.4.smithi186.stdout:Apr 12 21:05:59 smithi186 systemd[1]: ceph-1cbe407a-f90d-11ee-bc8d-c7b262605968@osd.4.service: Deactivated successfully. 2024-04-12T21:06:00.279 INFO:journalctl@ceph.osd.4.smithi186.stdout:Apr 12 21:05:59 smithi186 systemd[1]: Stopped Ceph osd.4 for 1cbe407a-f90d-11ee-bc8d-c7b262605968. 2024-04-12T21:06:00.280 INFO:journalctl@ceph.osd.4.smithi186.stdout:Apr 12 21:05:59 smithi186 systemd[1]: ceph-1cbe407a-f90d-11ee-bc8d-c7b262605968@osd.4.service: Consumed 2min 12.184s CPU time. 2024-04-12T21:06:00.674 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:06:00 smithi186 ceph-mon[86379]: Health check failed: Degraded data redundancy: 10/324 objects degraded (3.086%), 4 pgs degraded (PG_DEGRADED) 2024-04-12T21:06:00.674 INFO:journalctl@ceph.osd.4.smithi186.stdout:Apr 12 21:06:00 smithi186 systemd[1]: Starting Ceph osd.4 for 1cbe407a-f90d-11ee-bc8d-c7b262605968... 2024-04-12T21:06:00.850 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:06:00 smithi018 ceph-mon[112590]: Health check failed: Degraded data redundancy: 10/324 objects degraded (3.086%), 4 pgs degraded (PG_DEGRADED) 2024-04-12T21:06:00.996 INFO:journalctl@ceph.osd.4.smithi186.stdout:Apr 12 21:06:00 smithi186 podman[100481]: 2024-04-12 21:06:00.583480049 +0000 UTC m=+0.036391106 image pull 8717720dc5e5c56303818ea4d4f0a11f6e2c03dee521fb668f411f1304e0afca quay.ceph.io/ceph-ci/ceph@sha256:f4452def3be4d350f4a8e5feaf7ebb1511be9298fe68478ba7a2d6fbd7d69a63 2024-04-12T21:06:00.997 INFO:journalctl@ceph.osd.4.smithi186.stdout:Apr 12 21:06:00 smithi186 podman[100481]: 2024-04-12 21:06:00.729634979 +0000 UTC m=+0.182546072 container create c3e6904e51eef6a8ce92a17ae4a7160e8b4effc023a7723f251c77ab67f8dd03 (image=quay.ceph.io/ceph-ci/ceph@sha256:f4452def3be4d350f4a8e5feaf7ebb1511be9298fe68478ba7a2d6fbd7d69a63, name=ceph-1cbe407a-f90d-11ee-bc8d-c7b262605968-osd-4-activate, org.label-schema.build-date=20240409, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, RELEASE=wip-vshankar-testing-20240411.061452-324baf8, org.label-schema.license=GPLv2, ceph=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-12T21:06:00.997 INFO:journalctl@ceph.osd.4.smithi186.stdout:Apr 12 21:06:01 smithi186 podman[100481]: 2024-04-12 21:06:00.99978118 +0000 UTC m=+0.452692279 container init c3e6904e51eef6a8ce92a17ae4a7160e8b4effc023a7723f251c77ab67f8dd03 (image=quay.ceph.io/ceph-ci/ceph@sha256:f4452def3be4d350f4a8e5feaf7ebb1511be9298fe68478ba7a2d6fbd7d69a63, name=ceph-1cbe407a-f90d-11ee-bc8d-c7b262605968-osd-4-activate, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, org.label-schema.schema-version=1.0, RELEASE=wip-vshankar-testing-20240411.061452-324baf8, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240409) 2024-04-12T21:06:01.280 INFO:journalctl@ceph.osd.4.smithi186.stdout:Apr 12 21:06:01 smithi186 podman[100481]: 2024-04-12 21:06:01.014377081 +0000 UTC m=+0.467288181 container start c3e6904e51eef6a8ce92a17ae4a7160e8b4effc023a7723f251c77ab67f8dd03 (image=quay.ceph.io/ceph-ci/ceph@sha256:f4452def3be4d350f4a8e5feaf7ebb1511be9298fe68478ba7a2d6fbd7d69a63, name=ceph-1cbe407a-f90d-11ee-bc8d-c7b262605968-osd-4-activate, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, ceph=True, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.build-date=20240409, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-vshankar-testing-20240411.061452-324baf8, org.label-schema.vendor=CentOS) 2024-04-12T21:06:01.280 INFO:journalctl@ceph.osd.4.smithi186.stdout:Apr 12 21:06:01 smithi186 podman[100481]: 2024-04-12 21:06:01.098595297 +0000 UTC m=+0.551506443 container attach c3e6904e51eef6a8ce92a17ae4a7160e8b4effc023a7723f251c77ab67f8dd03 (image=quay.ceph.io/ceph-ci/ceph@sha256:f4452def3be4d350f4a8e5feaf7ebb1511be9298fe68478ba7a2d6fbd7d69a63, name=ceph-1cbe407a-f90d-11ee-bc8d-c7b262605968-osd-4-activate, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, RELEASE=wip-vshankar-testing-20240411.061452-324baf8, GIT_BRANCH=HEAD, org.label-schema.build-date=20240409, GIT_CLEAN=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-12T21:06:01.779 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:06:01 smithi186 ceph-mon[86379]: pgmap v174: 65 pgs: 3 active+undersized, 4 active+undersized+degraded, 8 peering, 4 stale+active+clean, 46 active+clean; 257 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 10/324 objects degraded (3.086%) 2024-04-12T21:06:01.850 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:06:01 smithi018 ceph-mon[112590]: pgmap v174: 65 pgs: 3 active+undersized, 4 active+undersized+degraded, 8 peering, 4 stale+active+clean, 46 active+clean; 257 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 10/324 objects degraded (3.086%) 2024-04-12T21:06:02.237 INFO:journalctl@ceph.osd.4.smithi186.stdout:Apr 12 21:06:01 smithi186 ceph-1cbe407a-f90d-11ee-bc8d-c7b262605968-osd-4-activate[100496]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-04-12T21:06:02.237 INFO:journalctl@ceph.osd.4.smithi186.stdout:Apr 12 21:06:01 smithi186 ceph-1cbe407a-f90d-11ee-bc8d-c7b262605968-osd-4-activate[100496]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-4 --no-mon-config --dev /dev/mapper/vg_nvme-lv_3 2024-04-12T21:06:02.237 INFO:journalctl@ceph.osd.4.smithi186.stdout:Apr 12 21:06:01 smithi186 bash[100481]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-04-12T21:06:02.237 INFO:journalctl@ceph.osd.4.smithi186.stdout:Apr 12 21:06:01 smithi186 bash[100481]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-4 --no-mon-config --dev /dev/mapper/vg_nvme-lv_3 2024-04-12T21:06:02.237 INFO:journalctl@ceph.osd.4.smithi186.stdout:Apr 12 21:06:01 smithi186 bash[100481]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_3 2024-04-12T21:06:02.237 INFO:journalctl@ceph.osd.4.smithi186.stdout:Apr 12 21:06:01 smithi186 bash[100481]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-2 2024-04-12T21:06:02.237 INFO:journalctl@ceph.osd.4.smithi186.stdout:Apr 12 21:06:01 smithi186 bash[100481]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-4/block 2024-04-12T21:06:02.238 INFO:journalctl@ceph.osd.4.smithi186.stdout:Apr 12 21:06:01 smithi186 bash[100481]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-04-12T21:06:02.238 INFO:journalctl@ceph.osd.4.smithi186.stdout:Apr 12 21:06:01 smithi186 bash[100481]: --> ceph-volume raw activate successful for osd ID: 4 2024-04-12T21:06:02.238 INFO:journalctl@ceph.osd.4.smithi186.stdout:Apr 12 21:06:01 smithi186 ceph-1cbe407a-f90d-11ee-bc8d-c7b262605968-osd-4-activate[100496]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_3 2024-04-12T21:06:02.238 INFO:journalctl@ceph.osd.4.smithi186.stdout:Apr 12 21:06:01 smithi186 ceph-1cbe407a-f90d-11ee-bc8d-c7b262605968-osd-4-activate[100496]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-2 2024-04-12T21:06:02.238 INFO:journalctl@ceph.osd.4.smithi186.stdout:Apr 12 21:06:01 smithi186 ceph-1cbe407a-f90d-11ee-bc8d-c7b262605968-osd-4-activate[100496]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-4/block 2024-04-12T21:06:02.238 INFO:journalctl@ceph.osd.4.smithi186.stdout:Apr 12 21:06:01 smithi186 ceph-1cbe407a-f90d-11ee-bc8d-c7b262605968-osd-4-activate[100496]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-04-12T21:06:02.238 INFO:journalctl@ceph.osd.4.smithi186.stdout:Apr 12 21:06:01 smithi186 ceph-1cbe407a-f90d-11ee-bc8d-c7b262605968-osd-4-activate[100496]: --> ceph-volume raw activate successful for osd ID: 4 2024-04-12T21:06:02.238 INFO:journalctl@ceph.osd.4.smithi186.stdout:Apr 12 21:06:01 smithi186 podman[100481]: 2024-04-12 21:06:01.938463027 +0000 UTC m=+1.391374090 container died c3e6904e51eef6a8ce92a17ae4a7160e8b4effc023a7723f251c77ab67f8dd03 (image=quay.ceph.io/ceph-ci/ceph@sha256:f4452def3be4d350f4a8e5feaf7ebb1511be9298fe68478ba7a2d6fbd7d69a63, name=ceph-1cbe407a-f90d-11ee-bc8d-c7b262605968-osd-4-activate, RELEASE=wip-vshankar-testing-20240411.061452-324baf8, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.build-date=20240409, GIT_CLEAN=True, GIT_BRANCH=HEAD, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-12T21:06:02.993 INFO:journalctl@ceph.osd.4.smithi186.stdout:Apr 12 21:06:02 smithi186 podman[100631]: 2024-04-12 21:06:02.635661589 +0000 UTC m=+0.678029523 container remove c3e6904e51eef6a8ce92a17ae4a7160e8b4effc023a7723f251c77ab67f8dd03 (image=quay.ceph.io/ceph-ci/ceph@sha256:f4452def3be4d350f4a8e5feaf7ebb1511be9298fe68478ba7a2d6fbd7d69a63, name=ceph-1cbe407a-f90d-11ee-bc8d-c7b262605968-osd-4-activate, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, RELEASE=wip-vshankar-testing-20240411.061452-324baf8, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240409) 2024-04-12T21:06:03.264 INFO:journalctl@ceph.osd.4.smithi186.stdout:Apr 12 21:06:02 smithi186 podman[100693]: 2024-04-12 21:06:02.901726995 +0000 UTC m=+0.034744090 image pull 8717720dc5e5c56303818ea4d4f0a11f6e2c03dee521fb668f411f1304e0afca quay.ceph.io/ceph-ci/ceph@sha256:f4452def3be4d350f4a8e5feaf7ebb1511be9298fe68478ba7a2d6fbd7d69a63 2024-04-12T21:06:03.264 INFO:journalctl@ceph.osd.4.smithi186.stdout:Apr 12 21:06:03 smithi186 podman[100693]: 2024-04-12 21:06:03.0115698 +0000 UTC m=+0.144586932 container create b20cb63b25c9f6653442c9f5146dd3b2af3ffe45af103d11ab22c9991f5e799d (image=quay.ceph.io/ceph-ci/ceph@sha256:f4452def3be4d350f4a8e5feaf7ebb1511be9298fe68478ba7a2d6fbd7d69a63, name=ceph-1cbe407a-f90d-11ee-bc8d-c7b262605968-osd-4, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240409, GIT_BRANCH=HEAD, RELEASE=wip-vshankar-testing-20240411.061452-324baf8, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2) 2024-04-12T21:06:03.529 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:06:03 smithi186 ceph-mon[86379]: pgmap v175: 65 pgs: 14 active+undersized, 10 active+undersized+degraded, 8 peering, 33 active+clean; 257 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 28/324 objects degraded (8.642%) 2024-04-12T21:06:03.530 INFO:journalctl@ceph.osd.4.smithi186.stdout:Apr 12 21:06:03 smithi186 podman[100693]: 2024-04-12 21:06:03.268000211 +0000 UTC m=+0.401017350 container init b20cb63b25c9f6653442c9f5146dd3b2af3ffe45af103d11ab22c9991f5e799d (image=quay.ceph.io/ceph-ci/ceph@sha256:f4452def3be4d350f4a8e5feaf7ebb1511be9298fe68478ba7a2d6fbd7d69a63, name=ceph-1cbe407a-f90d-11ee-bc8d-c7b262605968-osd-4, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, RELEASE=wip-vshankar-testing-20240411.061452-324baf8, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240409, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS) 2024-04-12T21:06:03.530 INFO:journalctl@ceph.osd.4.smithi186.stdout:Apr 12 21:06:03 smithi186 podman[100693]: 2024-04-12 21:06:03.277915356 +0000 UTC m=+0.410932524 container start b20cb63b25c9f6653442c9f5146dd3b2af3ffe45af103d11ab22c9991f5e799d (image=quay.ceph.io/ceph-ci/ceph@sha256:f4452def3be4d350f4a8e5feaf7ebb1511be9298fe68478ba7a2d6fbd7d69a63, name=ceph-1cbe407a-f90d-11ee-bc8d-c7b262605968-osd-4, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, RELEASE=wip-vshankar-testing-20240411.061452-324baf8, ceph=True, org.label-schema.build-date=20240409) 2024-04-12T21:06:03.530 INFO:journalctl@ceph.osd.4.smithi186.stdout:Apr 12 21:06:03 smithi186 ceph-osd[100712]: -- 172.21.15.186:0/1323937553 <== mon.1 v2:172.21.15.186:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55b64dd2b680 con 0x55b64dcf4c00 2024-04-12T21:06:03.530 INFO:journalctl@ceph.osd.4.smithi186.stdout:Apr 12 21:06:03 smithi186 bash[100693]: b20cb63b25c9f6653442c9f5146dd3b2af3ffe45af103d11ab22c9991f5e799d 2024-04-12T21:06:03.530 INFO:journalctl@ceph.osd.4.smithi186.stdout:Apr 12 21:06:03 smithi186 systemd[1]: Started Ceph osd.4 for 1cbe407a-f90d-11ee-bc8d-c7b262605968. 2024-04-12T21:06:03.850 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:06:03 smithi018 ceph-mon[112590]: pgmap v175: 65 pgs: 14 active+undersized, 10 active+undersized+degraded, 8 peering, 33 active+clean; 257 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 28/324 objects degraded (8.642%) 2024-04-12T21:06:04.279 INFO:journalctl@ceph.osd.4.smithi186.stdout:Apr 12 21:06:03 smithi186 ceph-1cbe407a-f90d-11ee-bc8d-c7b262605968-osd-4[100708]: 2024-04-12T21:06:03.847+0000 7fbce56b07c0 -1 Falling back to public interface 2024-04-12T21:06:04.841 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:06:04 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' 2024-04-12T21:06:04.841 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:06:04 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' 2024-04-12T21:06:04.841 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:06:04 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-12T21:06:04.841 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:06:04 smithi186 ceph-mon[86379]: pgmap v176: 65 pgs: 16 active+undersized, 16 active+undersized+degraded, 33 active+clean; 257 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 53/324 objects degraded (16.358%) 2024-04-12T21:06:04.850 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:06:04 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' 2024-04-12T21:06:04.850 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:06:04 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' 2024-04-12T21:06:04.850 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:06:04 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-12T21:06:04.850 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:06:04 smithi018 ceph-mon[112590]: pgmap v176: 65 pgs: 16 active+undersized, 16 active+undersized+degraded, 33 active+clean; 257 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 53/324 objects degraded (16.358%) 2024-04-12T21:06:05.829 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:06:05 smithi186 ceph-mon[86379]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg peering) 2024-04-12T21:06:05.850 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:06:05 smithi018 ceph-mon[112590]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg peering) 2024-04-12T21:06:06.850 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:06:06 smithi018 ceph-mon[112590]: pgmap v177: 65 pgs: 16 active+undersized, 16 active+undersized+degraded, 33 active+clean; 257 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 53/324 objects degraded (16.358%) 2024-04-12T21:06:06.942 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:06:06 smithi186 ceph-mon[86379]: pgmap v177: 65 pgs: 16 active+undersized, 16 active+undersized+degraded, 33 active+clean; 257 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 53/324 objects degraded (16.358%) 2024-04-12T21:06:09.029 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:06:08 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' 2024-04-12T21:06:09.029 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:06:08 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' 2024-04-12T21:06:09.029 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:06:08 smithi186 ceph-mon[86379]: pgmap v178: 65 pgs: 16 active+undersized, 16 active+undersized+degraded, 33 active+clean; 257 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 53/324 objects degraded (16.358%) 2024-04-12T21:06:09.100 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:06:08 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' 2024-04-12T21:06:09.101 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:06:08 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' 2024-04-12T21:06:09.101 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:06:08 smithi018 ceph-mon[112590]: pgmap v178: 65 pgs: 16 active+undersized, 16 active+undersized+degraded, 33 active+clean; 257 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 53/324 objects degraded (16.358%) 2024-04-12T21:06:10.029 INFO:journalctl@ceph.osd.4.smithi186.stdout:Apr 12 21:06:09 smithi186 ceph-1cbe407a-f90d-11ee-bc8d-c7b262605968-osd-4[100708]: 2024-04-12T21:06:09.672+0000 7fbce56b07c0 -1 osd.4 88 log_to_monitors true 2024-04-12T21:06:10.874 INFO:journalctl@ceph.osd.4.smithi186.stdout:Apr 12 21:06:10 smithi186 ceph-1cbe407a-f90d-11ee-bc8d-c7b262605968-osd-4[100708]: 2024-04-12T21:06:10.719+0000 7fbcd4a00700 -1 osd.4 88 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-12T21:06:10.874 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:06:10 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' 2024-04-12T21:06:10.875 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:06:10 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' 2024-04-12T21:06:10.875 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:06:10 smithi186 ceph-mon[86379]: from='osd.4 [v2:172.21.15.186:6808/2123883537,v1:172.21.15.186:6809/2123883537]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-12T21:06:10.875 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:06:10 smithi186 ceph-mon[86379]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-12T21:06:10.875 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:06:10 smithi186 ceph-mon[86379]: Health check update: Degraded data redundancy: 53/324 objects degraded (16.358%), 16 pgs degraded (PG_DEGRADED) 2024-04-12T21:06:10.875 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:06:10 smithi186 ceph-mon[86379]: pgmap v179: 65 pgs: 16 active+undersized, 16 active+undersized+degraded, 33 active+clean; 257 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 53/324 objects degraded (16.358%) 2024-04-12T21:06:10.875 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:06:10 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' 2024-04-12T21:06:10.875 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:06:10 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T21:06:10.904 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:06:10 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' 2024-04-12T21:06:10.904 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:06:10 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' 2024-04-12T21:06:10.904 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:06:10 smithi018 ceph-mon[112590]: from='osd.4 [v2:172.21.15.186:6808/2123883537,v1:172.21.15.186:6809/2123883537]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-12T21:06:10.904 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:06:10 smithi018 ceph-mon[112590]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-12T21:06:10.904 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:06:10 smithi018 ceph-mon[112590]: Health check update: Degraded data redundancy: 53/324 objects degraded (16.358%), 16 pgs degraded (PG_DEGRADED) 2024-04-12T21:06:10.904 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:06:10 smithi018 ceph-mon[112590]: pgmap v179: 65 pgs: 16 active+undersized, 16 active+undersized+degraded, 33 active+clean; 257 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 53/324 objects degraded (16.358%) 2024-04-12T21:06:10.904 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:06:10 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' 2024-04-12T21:06:10.904 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:06:10 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T21:06:11.964 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:06:11 smithi186 ceph-mon[86379]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-04-12T21:06:11.964 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:06:11 smithi186 ceph-mon[86379]: osdmap e91: 6 total, 5 up, 6 in 2024-04-12T21:06:11.964 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:06:11 smithi186 ceph-mon[86379]: from='osd.4 [v2:172.21.15.186:6808/2123883537,v1:172.21.15.186:6809/2123883537]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi186", "root=default"]}]: dispatch 2024-04-12T21:06:11.964 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:06:11 smithi186 ceph-mon[86379]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi186", "root=default"]}]: dispatch 2024-04-12T21:06:12.100 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:06:11 smithi018 ceph-mon[112590]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-04-12T21:06:12.100 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:06:11 smithi018 ceph-mon[112590]: osdmap e91: 6 total, 5 up, 6 in 2024-04-12T21:06:12.100 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:06:11 smithi018 ceph-mon[112590]: from='osd.4 [v2:172.21.15.186:6808/2123883537,v1:172.21.15.186:6809/2123883537]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi186", "root=default"]}]: dispatch 2024-04-12T21:06:12.101 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:06:11 smithi018 ceph-mon[112590]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi186", "root=default"]}]: dispatch 2024-04-12T21:06:13.030 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:06:12 smithi186 ceph-mon[86379]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-04-12T21:06:13.030 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:06:12 smithi186 ceph-mon[86379]: osd.4 [v2:172.21.15.186:6808/2123883537,v1:172.21.15.186:6809/2123883537] boot 2024-04-12T21:06:13.030 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:06:12 smithi186 ceph-mon[86379]: osdmap e92: 6 total, 6 up, 6 in 2024-04-12T21:06:13.030 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:06:12 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-12T21:06:13.030 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:06:12 smithi186 ceph-mon[86379]: pgmap v182: 65 pgs: 16 active+undersized, 16 active+undersized+degraded, 33 active+clean; 257 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 53/324 objects degraded (16.358%) 2024-04-12T21:06:13.100 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:06:12 smithi018 ceph-mon[112590]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-04-12T21:06:13.100 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:06:12 smithi018 ceph-mon[112590]: osd.4 [v2:172.21.15.186:6808/2123883537,v1:172.21.15.186:6809/2123883537] boot 2024-04-12T21:06:13.100 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:06:12 smithi018 ceph-mon[112590]: osdmap e92: 6 total, 6 up, 6 in 2024-04-12T21:06:13.101 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:06:12 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-12T21:06:13.101 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:06:12 smithi018 ceph-mon[112590]: pgmap v182: 65 pgs: 16 active+undersized, 16 active+undersized+degraded, 33 active+clean; 257 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 53/324 objects degraded (16.358%) 2024-04-12T21:06:14.029 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:06:13 smithi186 ceph-mon[86379]: osdmap e93: 6 total, 6 up, 6 in 2024-04-12T21:06:14.100 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:06:13 smithi018 ceph-mon[112590]: osdmap e93: 6 total, 6 up, 6 in 2024-04-12T21:06:15.029 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:06:14 smithi186 ceph-mon[86379]: pgmap v184: 65 pgs: 7 peering, 14 active+undersized, 11 active+undersized+degraded, 33 active+clean; 257 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 31/324 objects degraded (9.568%) 2024-04-12T21:06:15.100 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:06:14 smithi018 ceph-mon[112590]: pgmap v184: 65 pgs: 7 peering, 14 active+undersized, 11 active+undersized+degraded, 33 active+clean; 257 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 31/324 objects degraded (9.568%) 2024-04-12T21:06:16.030 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:06:15 smithi186 ceph-mon[86379]: Health check update: Degraded data redundancy: 31/324 objects degraded (9.568%), 11 pgs degraded (PG_DEGRADED) 2024-04-12T21:06:16.030 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:06:15 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' 2024-04-12T21:06:16.030 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:06:15 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' 2024-04-12T21:06:16.030 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:06:15 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-12T21:06:16.030 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:06:15 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-12T21:06:16.030 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:06:15 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' 2024-04-12T21:06:16.030 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:06:15 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-12T21:06:16.030 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:06:15 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T21:06:16.030 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:06:15 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T21:06:16.031 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:06:15 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T21:06:16.031 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:06:15 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd ok-to-stop", "ids": ["5"], "max": 16}]: dispatch 2024-04-12T21:06:16.031 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:06:15 smithi186 ceph-mon[86379]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["5"], "max": 16}]: dispatch 2024-04-12T21:06:16.031 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:06:15 smithi186 ceph-mon[86379]: Upgrade: unsafe to stop osd(s) at this time (12 PGs are or would become offline) 2024-04-12T21:06:16.101 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:06:15 smithi018 ceph-mon[112590]: Health check update: Degraded data redundancy: 31/324 objects degraded (9.568%), 11 pgs degraded (PG_DEGRADED) 2024-04-12T21:06:16.101 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:06:15 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' 2024-04-12T21:06:16.101 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:06:15 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' 2024-04-12T21:06:16.101 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:06:15 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-12T21:06:16.101 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:06:15 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-12T21:06:16.101 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:06:15 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' 2024-04-12T21:06:16.101 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:06:15 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-12T21:06:16.101 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:06:15 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T21:06:16.101 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:06:15 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T21:06:16.101 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:06:15 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T21:06:16.102 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:06:15 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd ok-to-stop", "ids": ["5"], "max": 16}]: dispatch 2024-04-12T21:06:16.102 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:06:15 smithi018 ceph-mon[112590]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["5"], "max": 16}]: dispatch 2024-04-12T21:06:16.102 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:06:15 smithi018 ceph-mon[112590]: Upgrade: unsafe to stop osd(s) at this time (12 PGs are or would become offline) 2024-04-12T21:06:17.029 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:06:16 smithi186 ceph-mon[86379]: pgmap v185: 65 pgs: 7 peering, 2 active+undersized, 56 active+clean; 257 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-12T21:06:17.100 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:06:16 smithi018 ceph-mon[112590]: pgmap v185: 65 pgs: 7 peering, 2 active+undersized, 56 active+clean; 257 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-12T21:06:18.029 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:06:17 smithi186 ceph-mon[86379]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 31/324 objects degraded (9.568%), 11 pgs degraded) 2024-04-12T21:06:18.029 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:06:17 smithi186 ceph-mon[86379]: Cluster is now healthy 2024-04-12T21:06:18.100 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:06:17 smithi018 ceph-mon[112590]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 31/324 objects degraded (9.568%), 11 pgs degraded) 2024-04-12T21:06:18.101 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:06:17 smithi018 ceph-mon[112590]: Cluster is now healthy 2024-04-12T21:06:18.870 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:18.874+0000 7f920ea00700 1 -- 172.21.15.18:0/471876203 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9208104e60 msgr2=0x7f9208105230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:06:18.870 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:18.874+0000 7f920ea00700 1 --2- 172.21.15.18:0/471876203 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9208104e60 0x7f9208105230 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f91fc004940 tx=0x7f91fc009e30 comp rx=0 tx=0).stop 2024-04-12T21:06:18.871 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:18.874+0000 7f920ea00700 1 -- 172.21.15.18:0/471876203 shutdown_connections 2024-04-12T21:06:18.871 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:18.874+0000 7f920ea00700 1 --2- 172.21.15.18:0/471876203 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9208105800 0x7f92080fee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:06:18.871 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:18.874+0000 7f920ea00700 1 --2- 172.21.15.18:0/471876203 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9208104e60 0x7f9208105230 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:06:18.871 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:18.874+0000 7f920ea00700 1 -- 172.21.15.18:0/471876203 >> 172.21.15.18:0/471876203 conn(0x7f92080fab10 msgr2=0x7f92080fcef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:06:18.871 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:18.874+0000 7f920ea00700 1 -- 172.21.15.18:0/471876203 shutdown_connections 2024-04-12T21:06:18.871 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:18.874+0000 7f920ea00700 1 -- 172.21.15.18:0/471876203 wait complete. 2024-04-12T21:06:18.871 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:18.875+0000 7f920ea00700 1 Processor -- start 2024-04-12T21:06:18.872 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:18.876+0000 7f920ea00700 1 -- start start 2024-04-12T21:06:18.872 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:18.876+0000 7f920ea00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9208104e60 0x7f9208115bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:06:18.872 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:18.876+0000 7f920ea00700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9208105800 0x7f92081160f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:06:18.872 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:18.876+0000 7f920ea00700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9208116740 con 0x7f9208104e60 2024-04-12T21:06:18.873 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:18.876+0000 7f920ea00700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f920810fb50 con 0x7f9208105800 2024-04-12T21:06:18.873 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:18.877+0000 7f920d600700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9208104e60 0x7f9208115bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:06:18.873 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:18.877+0000 7f920d600700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9208104e60 0x7f9208115bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:49438/0 (socket says 172.21.15.18:49438) 2024-04-12T21:06:18.873 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:18.877+0000 7f920d600700 1 -- 172.21.15.18:0/1917215978 learned_addr learned my addr 172.21.15.18:0/1917215978 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:06:18.874 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:18.877+0000 7f920cc00700 1 --2- 172.21.15.18:0/1917215978 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9208105800 0x7f92081160f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:06:18.874 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:18.877+0000 7f920d600700 1 -- 172.21.15.18:0/1917215978 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9208105800 msgr2=0x7f92081160f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:06:18.874 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:18.877+0000 7f920d600700 1 --2- 172.21.15.18:0/1917215978 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9208105800 0x7f92081160f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:06:18.874 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:18.877+0000 7f920d600700 1 -- 172.21.15.18:0/1917215978 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91fc005040 con 0x7f9208104e60 2024-04-12T21:06:18.874 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:18.878+0000 7f920d600700 1 --2- 172.21.15.18:0/1917215978 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9208104e60 0x7f9208115bb0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f91fc009150 tx=0x7f91fc009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:06:18.874 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:18.878+0000 7f9205600700 1 -- 172.21.15.18:0/1917215978 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f91fc00e040 con 0x7f9208104e60 2024-04-12T21:06:18.874 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:18.878+0000 7f9205600700 1 -- 172.21.15.18:0/1917215978 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f91fc013420 con 0x7f9208104e60 2024-04-12T21:06:18.874 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:18.878+0000 7f920ea00700 1 -- 172.21.15.18:0/1917215978 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f920810fdd0 con 0x7f9208104e60 2024-04-12T21:06:18.874 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:18.878+0000 7f9205600700 1 -- 172.21.15.18:0/1917215978 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f91fc01c630 con 0x7f9208104e60 2024-04-12T21:06:18.875 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:18.878+0000 7f920ea00700 1 -- 172.21.15.18:0/1917215978 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9208110390 con 0x7f9208104e60 2024-04-12T21:06:18.875 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:18.879+0000 7f920ea00700 1 -- 172.21.15.18:0/1917215978 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f92081005b0 con 0x7f9208104e60 2024-04-12T21:06:18.876 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:18.881+0000 7f9205600700 1 -- 172.21.15.18:0/1917215978 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7f91fc013590 con 0x7f9208104e60 2024-04-12T21:06:18.877 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:18.881+0000 7f9205600700 1 --2- 172.21.15.18:0/1917215978 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f91f006ec00 0x7f91f00710b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:06:18.877 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:18.882+0000 7f9205600700 1 -- 172.21.15.18:0/1917215978 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 6585+0+0 (secure 0 0 0) 0x7f91fc0906e0 con 0x7f9208104e60 2024-04-12T21:06:18.878 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:18.882+0000 7f920cc00700 1 --2- 172.21.15.18:0/1917215978 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f91f006ec00 0x7f91f00710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:06:18.878 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:18.883+0000 7f920cc00700 1 --2- 172.21.15.18:0/1917215978 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f91f006ec00 0x7f91f00710b0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f9208111450 tx=0x7f91f4009040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:06:18.881 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:18.885+0000 7f9205600700 1 -- 172.21.15.18:0/1917215978 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f91fc059490 con 0x7f9208104e60 2024-04-12T21:06:19.029 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:06:18 smithi186 ceph-mon[86379]: pgmap v186: 65 pgs: 65 active+clean; 257 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-12T21:06:19.100 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:06:18 smithi018 ceph-mon[112590]: pgmap v186: 65 pgs: 65 active+clean; 257 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-12T21:06:19.135 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:19.139+0000 7f920ea00700 1 -- 172.21.15.18:0/1917215978 --> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f920804bf40 con 0x7f91f006ec00 2024-04-12T21:06:19.137 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:19.141+0000 7f9205600700 1 -- 172.21.15.18:0/1917215978 <== mgr.34102 v2:172.21.15.18:6800/3812891919 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+401 (secure 0 0 0) 0x7f920804bf40 con 0x7f91f006ec00 2024-04-12T21:06:19.141 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:19.144+0000 7f920ea00700 1 -- 172.21.15.18:0/1917215978 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f91f006ec00 msgr2=0x7f91f00710b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:06:19.141 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:19.144+0000 7f920ea00700 1 --2- 172.21.15.18:0/1917215978 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f91f006ec00 0x7f91f00710b0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f9208111450 tx=0x7f91f4009040 comp rx=0 tx=0).stop 2024-04-12T21:06:19.141 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:19.144+0000 7f920ea00700 1 -- 172.21.15.18:0/1917215978 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9208104e60 msgr2=0x7f9208115bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:06:19.142 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:19.144+0000 7f920ea00700 1 --2- 172.21.15.18:0/1917215978 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9208104e60 0x7f9208115bb0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f91fc009150 tx=0x7f91fc009e30 comp rx=0 tx=0).stop 2024-04-12T21:06:19.142 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:19.144+0000 7f920ea00700 1 -- 172.21.15.18:0/1917215978 shutdown_connections 2024-04-12T21:06:19.142 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:19.144+0000 7f920ea00700 1 --2- 172.21.15.18:0/1917215978 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f91f006ec00 0x7f91f00710b0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:06:19.142 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:19.144+0000 7f920ea00700 1 --2- 172.21.15.18:0/1917215978 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9208104e60 0x7f9208115bb0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:06:19.142 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:19.144+0000 7f920ea00700 1 --2- 172.21.15.18:0/1917215978 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9208105800 0x7f92081160f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:06:19.142 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:19.144+0000 7f920ea00700 1 -- 172.21.15.18:0/1917215978 >> 172.21.15.18:0/1917215978 conn(0x7f92080fab10 msgr2=0x7f92080fca40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:06:19.143 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:19.145+0000 7f920ea00700 1 -- 172.21.15.18:0/1917215978 shutdown_connections 2024-04-12T21:06:19.143 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:19.145+0000 7f920ea00700 1 -- 172.21.15.18:0/1917215978 wait complete. 2024-04-12T21:06:19.157 INFO:teuthology.orchestra.run.smithi018.stdout:true 2024-04-12T21:06:19.303 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:19.306+0000 7fca48400700 1 -- 172.21.15.18:0/1984129560 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fca40104ff0 msgr2=0x7fca401053c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:06:19.303 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:19.306+0000 7fca48400700 1 --2- 172.21.15.18:0/1984129560 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fca40104ff0 0x7fca401053c0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fca34004940 tx=0x7fca34009e30 comp rx=0 tx=0).stop 2024-04-12T21:06:19.303 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:19.307+0000 7fca48400700 1 -- 172.21.15.18:0/1984129560 shutdown_connections 2024-04-12T21:06:19.303 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:19.307+0000 7fca48400700 1 --2- 172.21.15.18:0/1984129560 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fca40105990 0x7fca400ff000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:06:19.303 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:19.307+0000 7fca48400700 1 --2- 172.21.15.18:0/1984129560 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fca40104ff0 0x7fca401053c0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:06:19.303 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:19.307+0000 7fca48400700 1 -- 172.21.15.18:0/1984129560 >> 172.21.15.18:0/1984129560 conn(0x7fca400faba0 msgr2=0x7fca400fcfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:06:19.304 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:19.307+0000 7fca48400700 1 -- 172.21.15.18:0/1984129560 shutdown_connections 2024-04-12T21:06:19.304 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:19.307+0000 7fca48400700 1 -- 172.21.15.18:0/1984129560 wait complete. 2024-04-12T21:06:19.305 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:19.308+0000 7fca48400700 1 Processor -- start 2024-04-12T21:06:19.305 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:19.308+0000 7fca48400700 1 -- start start 2024-04-12T21:06:19.305 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:19.308+0000 7fca48400700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fca40104ff0 0x7fca40115b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:06:19.305 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:19.308+0000 7fca48400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fca40105990 0x7fca401160a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:06:19.305 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:19.308+0000 7fca48400700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca401166f0 con 0x7fca40105990 2024-04-12T21:06:19.306 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:19.308+0000 7fca48400700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca4010fcd0 con 0x7fca40104ff0 2024-04-12T21:06:19.306 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:19.309+0000 7fca45000700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fca40105990 0x7fca401160a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:06:19.306 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:19.309+0000 7fca45000700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fca40105990 0x7fca401160a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:49452/0 (socket says 172.21.15.18:49452) 2024-04-12T21:06:19.306 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:19.309+0000 7fca45000700 1 -- 172.21.15.18:0/4041596449 learned_addr learned my addr 172.21.15.18:0/4041596449 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:06:19.306 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:19.309+0000 7fca45a00700 1 --2- 172.21.15.18:0/4041596449 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fca40104ff0 0x7fca40115b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:06:19.306 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:19.309+0000 7fca45000700 1 -- 172.21.15.18:0/4041596449 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fca40104ff0 msgr2=0x7fca40115b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:06:19.306 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:19.309+0000 7fca45000700 1 --2- 172.21.15.18:0/4041596449 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fca40104ff0 0x7fca40115b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:06:19.306 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:19.309+0000 7fca45000700 1 -- 172.21.15.18:0/4041596449 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca34005040 con 0x7fca40105990 2024-04-12T21:06:19.306 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:19.309+0000 7fca45a00700 1 --2- 172.21.15.18:0/4041596449 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fca40104ff0 0x7fca40115b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-12T21:06:19.306 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:19.309+0000 7fca45000700 1 --2- 172.21.15.18:0/4041596449 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fca40105990 0x7fca401160a0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fca2c002a50 tx=0x7fca2c004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:06:19.307 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:19.310+0000 7fca3d600700 1 -- 172.21.15.18:0/4041596449 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fca2c007cc0 con 0x7fca40105990 2024-04-12T21:06:19.307 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:19.310+0000 7fca3d600700 1 -- 172.21.15.18:0/4041596449 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fca2c00f040 con 0x7fca40105990 2024-04-12T21:06:19.307 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:19.310+0000 7fca3d600700 1 -- 172.21.15.18:0/4041596449 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fca2c00daf0 con 0x7fca40105990 2024-04-12T21:06:19.307 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:19.310+0000 7fca48400700 1 -- 172.21.15.18:0/4041596449 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fca4010ffb0 con 0x7fca40105990 2024-04-12T21:06:19.308 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:19.310+0000 7fca48400700 1 -- 172.21.15.18:0/4041596449 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fca40110570 con 0x7fca40105990 2024-04-12T21:06:19.308 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:19.311+0000 7fca48400700 1 -- 172.21.15.18:0/4041596449 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fca40100740 con 0x7fca40105990 2024-04-12T21:06:19.308 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:19.312+0000 7fca3d600700 1 -- 172.21.15.18:0/4041596449 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7fca2c009810 con 0x7fca40105990 2024-04-12T21:06:19.310 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:19.314+0000 7fca3d600700 1 --2- 172.21.15.18:0/4041596449 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fca2806ec00 0x7fca280710b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:06:19.310 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:19.314+0000 7fca3d600700 1 -- 172.21.15.18:0/4041596449 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 6585+0+0 (secure 0 0 0) 0x7fca2c08e650 con 0x7fca40105990 2024-04-12T21:06:19.310 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:19.314+0000 7fca45a00700 1 --2- 172.21.15.18:0/4041596449 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fca2806ec00 0x7fca280710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:06:19.311 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:19.315+0000 7fca45a00700 1 --2- 172.21.15.18:0/4041596449 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fca2806ec00 0x7fca280710b0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fca3400b3b0 tx=0x7fca34009e30 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:06:19.314 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:19.318+0000 7fca3d600700 1 -- 172.21.15.18:0/4041596449 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7fca2c05aeb0 con 0x7fca40105990 2024-04-12T21:06:19.565 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:19.568+0000 7fca48400700 1 -- 172.21.15.18:0/4041596449 --> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fca4010c160 con 0x7fca2806ec00 2024-04-12T21:06:19.566 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:19.571+0000 7fca3d600700 1 -- 172.21.15.18:0/4041596449 <== mgr.34102 v2:172.21.15.18:6800/3812891919 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+401 (secure 0 0 0) 0x7fca4010c160 con 0x7fca2806ec00 2024-04-12T21:06:19.569 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:19.573+0000 7fca48400700 1 -- 172.21.15.18:0/4041596449 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fca2806ec00 msgr2=0x7fca280710b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:06:19.569 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:19.573+0000 7fca48400700 1 --2- 172.21.15.18:0/4041596449 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fca2806ec00 0x7fca280710b0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fca3400b3b0 tx=0x7fca34009e30 comp rx=0 tx=0).stop 2024-04-12T21:06:19.569 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:19.573+0000 7fca48400700 1 -- 172.21.15.18:0/4041596449 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fca40105990 msgr2=0x7fca401160a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:06:19.569 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:19.573+0000 7fca48400700 1 --2- 172.21.15.18:0/4041596449 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fca40105990 0x7fca401160a0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fca2c002a50 tx=0x7fca2c004600 comp rx=0 tx=0).stop 2024-04-12T21:06:19.569 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:19.573+0000 7fca48400700 1 -- 172.21.15.18:0/4041596449 shutdown_connections 2024-04-12T21:06:19.569 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:19.573+0000 7fca48400700 1 --2- 172.21.15.18:0/4041596449 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fca2806ec00 0x7fca280710b0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:06:19.570 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:19.573+0000 7fca48400700 1 --2- 172.21.15.18:0/4041596449 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fca40104ff0 0x7fca40115b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:06:19.570 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:19.573+0000 7fca48400700 1 --2- 172.21.15.18:0/4041596449 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fca40105990 0x7fca401160a0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:06:19.570 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:19.573+0000 7fca48400700 1 -- 172.21.15.18:0/4041596449 >> 172.21.15.18:0/4041596449 conn(0x7fca400faba0 msgr2=0x7fca400fbb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:06:19.570 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:19.574+0000 7fca48400700 1 -- 172.21.15.18:0/4041596449 shutdown_connections 2024-04-12T21:06:19.570 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:19.574+0000 7fca48400700 1 -- 172.21.15.18:0/4041596449 wait complete. 2024-04-12T21:06:19.726 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:19.730+0000 7f2327400700 1 -- 172.21.15.18:0/1441764170 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2320101d10 msgr2=0x7f23201020e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:06:19.727 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:19.730+0000 7f2327400700 1 --2- 172.21.15.18:0/1441764170 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2320101d10 0x7f23201020e0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f2314004940 tx=0x7f2314009e30 comp rx=0 tx=0).stop 2024-04-12T21:06:19.727 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:19.730+0000 7f2327400700 1 -- 172.21.15.18:0/1441764170 shutdown_connections 2024-04-12T21:06:19.727 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:19.730+0000 7f2327400700 1 --2- 172.21.15.18:0/1441764170 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f23201026b0 0x7f23200fbd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:06:19.727 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:19.730+0000 7f2327400700 1 --2- 172.21.15.18:0/1441764170 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2320101d10 0x7f23201020e0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:06:19.727 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:19.730+0000 7f2327400700 1 -- 172.21.15.18:0/1441764170 >> 172.21.15.18:0/1441764170 conn(0x7f23200f7860 msgr2=0x7f23200f9c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:06:19.727 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:19.730+0000 7f2327400700 1 -- 172.21.15.18:0/1441764170 shutdown_connections 2024-04-12T21:06:19.727 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:19.730+0000 7f2327400700 1 -- 172.21.15.18:0/1441764170 wait complete. 2024-04-12T21:06:19.729 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:19.731+0000 7f2327400700 1 Processor -- start 2024-04-12T21:06:19.729 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:19.732+0000 7f2327400700 1 -- start start 2024-04-12T21:06:19.729 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:19.732+0000 7f2327400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2320101d10 0x7f2320102b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:06:19.729 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:19.732+0000 7f2327400700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f23201026b0 0x7f23201030b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:06:19.729 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:19.732+0000 7f2327400700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23201036d0 con 0x7f2320101d10 2024-04-12T21:06:19.729 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:19.732+0000 7f2327400700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2320103810 con 0x7f23201026b0 2024-04-12T21:06:19.729 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:19.732+0000 7f2324a00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2320101d10 0x7f2320102b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:06:19.729 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:19.732+0000 7f2324a00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2320101d10 0x7f2320102b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:49464/0 (socket says 172.21.15.18:49464) 2024-04-12T21:06:19.730 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:19.732+0000 7f2324a00700 1 -- 172.21.15.18:0/1387384901 learned_addr learned my addr 172.21.15.18:0/1387384901 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:06:19.730 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:19.733+0000 7f2324a00700 1 -- 172.21.15.18:0/1387384901 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f23201026b0 msgr2=0x7f23201030b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:06:19.730 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:19.733+0000 7f2324a00700 1 --2- 172.21.15.18:0/1387384901 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f23201026b0 0x7f23201030b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:06:19.730 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:19.733+0000 7f2324a00700 1 -- 172.21.15.18:0/1387384901 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2314005040 con 0x7f2320101d10 2024-04-12T21:06:19.730 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:19.733+0000 7f2324a00700 1 --2- 172.21.15.18:0/1387384901 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2320101d10 0x7f2320102b70 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f231400b8b0 tx=0x7f2314009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:06:19.730 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:19.733+0000 7f231cc00700 1 -- 172.21.15.18:0/1387384901 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f231400e040 con 0x7f2320101d10 2024-04-12T21:06:19.730 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:19.733+0000 7f231cc00700 1 -- 172.21.15.18:0/1387384901 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f2314004290 con 0x7f2320101d10 2024-04-12T21:06:19.730 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:19.733+0000 7f231cc00700 1 -- 172.21.15.18:0/1387384901 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f231401c3f0 con 0x7f2320101d10 2024-04-12T21:06:19.731 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:19.733+0000 7f2327400700 1 -- 172.21.15.18:0/1387384901 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2320119ed0 con 0x7f2320101d10 2024-04-12T21:06:19.731 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:19.733+0000 7f2327400700 1 -- 172.21.15.18:0/1387384901 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f232011a3b0 con 0x7f2320101d10 2024-04-12T21:06:19.731 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:19.734+0000 7f2327400700 1 -- 172.21.15.18:0/1387384901 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f23200fd460 con 0x7f2320101d10 2024-04-12T21:06:19.732 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:19.736+0000 7f231cc00700 1 -- 172.21.15.18:0/1387384901 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7f23140137b0 con 0x7f2320101d10 2024-04-12T21:06:19.733 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:19.736+0000 7f231cc00700 1 --2- 172.21.15.18:0/1387384901 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f230806ec00 0x7f23080710b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:06:19.733 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:19.737+0000 7f231cc00700 1 -- 172.21.15.18:0/1387384901 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 6585+0+0 (secure 0 0 0) 0x7f231408fbf0 con 0x7f2320101d10 2024-04-12T21:06:19.733 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:19.737+0000 7f231fe00700 1 --2- 172.21.15.18:0/1387384901 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f230806ec00 0x7f23080710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:06:19.733 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:19.738+0000 7f231fe00700 1 --2- 172.21.15.18:0/1387384901 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f230806ec00 0x7f23080710b0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f230c006fd0 tx=0x7f230c006db0 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:06:19.737 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:19.741+0000 7f231cc00700 1 -- 172.21.15.18:0/1387384901 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f23140589a0 con 0x7f2320101d10 2024-04-12T21:06:19.850 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:06:19 smithi018 ceph-mon[112590]: from='client.34372 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:06:19.985 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:19.989+0000 7f2327400700 1 -- 172.21.15.18:0/1387384901 --> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f2320108ef0 con 0x7f230806ec00 2024-04-12T21:06:20.002 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:20.006+0000 7f231cc00700 1 -- 172.21.15.18:0/1387384901 <== mgr.34102 v2:172.21.15.18:6800/3812891919 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7f2320108ef0 con 0x7f230806ec00 2024-04-12T21:06:20.003 INFO:teuthology.orchestra.run.smithi018.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-12T21:06:20.003 INFO:teuthology.orchestra.run.smithi018.stdout:alertmanager.smithi018 smithi018 *:9093,9094 running (6m) 82s ago 21m 14.9M - 0.25.0 c8568f914cd2 c332090068e8 2024-04-12T21:06:20.003 INFO:teuthology.orchestra.run.smithi018.stdout:crash.smithi018 smithi018 running (3m) 82s ago 21m 6665k - 19.0.0-2879-g324baf8d 8717720dc5e5 b7f80d1f7982 2024-04-12T21:06:20.003 INFO:teuthology.orchestra.run.smithi018.stdout:crash.smithi186 smithi186 running (3m) 12s ago 20m 6660k - 19.0.0-2879-g324baf8d 8717720dc5e5 a4f55c81b623 2024-04-12T21:06:20.003 INFO:teuthology.orchestra.run.smithi018.stdout:grafana.smithi018 smithi018 *:3000 running (6m) 82s ago 21m 74.5M - 9.4.12 2bacad6d85d8 b08e3bf176dd 2024-04-12T21:06:20.003 INFO:teuthology.orchestra.run.smithi018.stdout:mds.cephfs.smithi018.acqbsc smithi018 running (14m) 82s ago 14m 112M - 17.2.7-827-gd3b5060a 21de9b7369ad 5b5fce65f18a 2024-04-12T21:06:20.003 INFO:teuthology.orchestra.run.smithi018.stdout:mds.cephfs.smithi018.szjluo smithi018 running (14m) 82s ago 14m 113M - 17.2.7-827-gd3b5060a 21de9b7369ad cf9bf0c5031f 2024-04-12T21:06:20.003 INFO:teuthology.orchestra.run.smithi018.stdout:mds.cephfs.smithi186.jdllyn smithi186 running (14m) 12s ago 14m 15.4M - 17.2.7-827-gd3b5060a 21de9b7369ad 2b2bc46bd29a 2024-04-12T21:06:20.003 INFO:teuthology.orchestra.run.smithi018.stdout:mds.cephfs.smithi186.wjnafs smithi186 running (14m) 12s ago 14m 15.2M - 17.2.7-827-gd3b5060a 21de9b7369ad 74b115d6d6fd 2024-04-12T21:06:20.003 INFO:teuthology.orchestra.run.smithi018.stdout:mgr.smithi018.xjzwga smithi018 *:8443,9283,8765 running (9m) 82s ago 23m 505M - 19.0.0-2879-g324baf8d 8717720dc5e5 56088305eab7 2024-04-12T21:06:20.004 INFO:teuthology.orchestra.run.smithi018.stdout:mgr.smithi186.kqcfxk smithi186 *:8443,9283,8765 running (8m) 12s ago 20m 452M - 19.0.0-2879-g324baf8d 8717720dc5e5 09cd766e5e0d 2024-04-12T21:06:20.004 INFO:teuthology.orchestra.run.smithi018.stdout:mon.smithi018 smithi018 running (4m) 82s ago 23m 50.8M 2048M 19.0.0-2879-g324baf8d 8717720dc5e5 df3e201dfed6 2024-04-12T21:06:20.004 INFO:teuthology.orchestra.run.smithi018.stdout:mon.smithi186 smithi186 running (4m) 12s ago 20m 35.4M 2048M 19.0.0-2879-g324baf8d 8717720dc5e5 c77498da9b2b 2024-04-12T21:06:20.004 INFO:teuthology.orchestra.run.smithi018.stdout:node-exporter.smithi018 smithi018 *:9100 running (7m) 82s ago 21m 7818k - 1.5.0 0da6a335fe13 326c3d9c6847 2024-04-12T21:06:20.004 INFO:teuthology.orchestra.run.smithi018.stdout:node-exporter.smithi186 smithi186 *:9100 running (7m) 12s ago 20m 7860k - 1.5.0 0da6a335fe13 ec2112baeecf 2024-04-12T21:06:20.004 INFO:teuthology.orchestra.run.smithi018.stdout:osd.0 smithi018 running (2m) 82s ago 19m 117M 1559M 19.0.0-2879-g324baf8d 8717720dc5e5 60b1273ca047 2024-04-12T21:06:20.004 INFO:teuthology.orchestra.run.smithi018.stdout:osd.1 smithi018 running (2m) 82s ago 18m 88.0M 1559M 19.0.0-2879-g324baf8d 8717720dc5e5 170ce1e5fb8d 2024-04-12T21:06:20.004 INFO:teuthology.orchestra.run.smithi018.stdout:osd.2 smithi018 running (91s) 82s ago 17m 11.2M 1559M 19.0.0-2879-g324baf8d 8717720dc5e5 c2cbc6aa2839 2024-04-12T21:06:20.004 INFO:teuthology.orchestra.run.smithi018.stdout:osd.3 smithi186 running (59s) 12s ago 17m 96.2M 2581M 19.0.0-2879-g324baf8d 8717720dc5e5 0138e8262e4b 2024-04-12T21:06:20.004 INFO:teuthology.orchestra.run.smithi018.stdout:osd.4 smithi186 running (17s) 12s ago 16m 11.1M 2581M 19.0.0-2879-g324baf8d 8717720dc5e5 b20cb63b25c9 2024-04-12T21:06:20.004 INFO:teuthology.orchestra.run.smithi018.stdout:osd.5 smithi186 running (16m) 12s ago 16m 261M 2581M 17.2.7-827-gd3b5060a 21de9b7369ad 2ee072a436d0 2024-04-12T21:06:20.004 INFO:teuthology.orchestra.run.smithi018.stdout:prometheus.smithi018 smithi018 *:9095 running (7m) 82s ago 20m 60.0M - 2.43.0 a07b618ecd1d f58841127e21 2024-04-12T21:06:20.005 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:20.009+0000 7f2327400700 1 -- 172.21.15.18:0/1387384901 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f230806ec00 msgr2=0x7f23080710b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:06:20.006 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:20.009+0000 7f2327400700 1 --2- 172.21.15.18:0/1387384901 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f230806ec00 0x7f23080710b0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f230c006fd0 tx=0x7f230c006db0 comp rx=0 tx=0).stop 2024-04-12T21:06:20.006 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:20.010+0000 7f2327400700 1 -- 172.21.15.18:0/1387384901 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2320101d10 msgr2=0x7f2320102b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:06:20.006 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:20.010+0000 7f2327400700 1 --2- 172.21.15.18:0/1387384901 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2320101d10 0x7f2320102b70 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f231400b8b0 tx=0x7f2314009e30 comp rx=0 tx=0).stop 2024-04-12T21:06:20.006 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:20.010+0000 7f2327400700 1 -- 172.21.15.18:0/1387384901 shutdown_connections 2024-04-12T21:06:20.006 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:20.010+0000 7f2327400700 1 --2- 172.21.15.18:0/1387384901 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f230806ec00 0x7f23080710b0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:06:20.006 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:20.010+0000 7f2327400700 1 --2- 172.21.15.18:0/1387384901 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2320101d10 0x7f2320102b70 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:06:20.007 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:20.010+0000 7f2327400700 1 --2- 172.21.15.18:0/1387384901 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f23201026b0 0x7f23201030b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:06:20.007 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:20.010+0000 7f2327400700 1 -- 172.21.15.18:0/1387384901 >> 172.21.15.18:0/1387384901 conn(0x7f23200f7860 msgr2=0x7f23200f8ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:06:20.007 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:20.010+0000 7f2327400700 1 -- 172.21.15.18:0/1387384901 shutdown_connections 2024-04-12T21:06:20.007 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:20.010+0000 7f2327400700 1 -- 172.21.15.18:0/1387384901 wait complete. 2024-04-12T21:06:20.029 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:06:19 smithi186 ceph-mon[86379]: from='client.34372 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:06:20.150 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:20.154+0000 7feefca00700 1 -- 172.21.15.18:0/1210707374 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7feef8105020 msgr2=0x7feef81053f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:06:20.150 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:20.154+0000 7feefca00700 1 --2- 172.21.15.18:0/1210707374 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7feef8105020 0x7feef81053f0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7feee8004940 tx=0x7feee8009e30 comp rx=0 tx=0).stop 2024-04-12T21:06:20.150 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:20.154+0000 7feefca00700 1 -- 172.21.15.18:0/1210707374 shutdown_connections 2024-04-12T21:06:20.151 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:20.154+0000 7feefca00700 1 --2- 172.21.15.18:0/1210707374 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7feef81059c0 0x7feef80ff080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:06:20.151 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:20.154+0000 7feefca00700 1 --2- 172.21.15.18:0/1210707374 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7feef8105020 0x7feef81053f0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:06:20.151 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:20.154+0000 7feefca00700 1 -- 172.21.15.18:0/1210707374 >> 172.21.15.18:0/1210707374 conn(0x7feef80faba0 msgr2=0x7feef80fcfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:06:20.151 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:20.154+0000 7feefca00700 1 -- 172.21.15.18:0/1210707374 shutdown_connections 2024-04-12T21:06:20.151 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:20.154+0000 7feefca00700 1 -- 172.21.15.18:0/1210707374 wait complete. 2024-04-12T21:06:20.152 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:20.155+0000 7feefca00700 1 Processor -- start 2024-04-12T21:06:20.153 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:20.155+0000 7feefca00700 1 -- start start 2024-04-12T21:06:20.153 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:20.156+0000 7feefca00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7feef8105020 0x7feef8115b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:06:20.153 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:20.156+0000 7feefca00700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7feef81059c0 0x7feef81160d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:06:20.153 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:20.156+0000 7feefca00700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feef8116720 con 0x7feef8105020 2024-04-12T21:06:20.153 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:20.156+0000 7feefca00700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feef810fd00 con 0x7feef81059c0 2024-04-12T21:06:20.153 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:20.156+0000 7feef5e00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7feef8105020 0x7feef8115b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:06:20.153 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:20.156+0000 7feef5e00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7feef8105020 0x7feef8115b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:49478/0 (socket says 172.21.15.18:49478) 2024-04-12T21:06:20.153 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:20.156+0000 7feef5e00700 1 -- 172.21.15.18:0/4040577817 learned_addr learned my addr 172.21.15.18:0/4040577817 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:06:20.153 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:20.156+0000 7feef5e00700 1 -- 172.21.15.18:0/4040577817 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7feef81059c0 msgr2=0x7feef81160d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:06:20.153 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:20.156+0000 7feef5400700 1 --2- 172.21.15.18:0/4040577817 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7feef81059c0 0x7feef81160d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:06:20.153 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:20.156+0000 7feef5e00700 1 --2- 172.21.15.18:0/4040577817 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7feef81059c0 0x7feef81160d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:06:20.153 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:20.156+0000 7feef5e00700 1 -- 172.21.15.18:0/4040577817 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feee8005040 con 0x7feef8105020 2024-04-12T21:06:20.153 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:20.157+0000 7feef5e00700 1 --2- 172.21.15.18:0/4040577817 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7feef8105020 0x7feef8115b90 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7feee8009180 tx=0x7feee8009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:06:20.153 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:20.157+0000 7feeee000700 1 -- 172.21.15.18:0/4040577817 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7feee800e040 con 0x7feef8105020 2024-04-12T21:06:20.154 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:20.157+0000 7feeee000700 1 -- 172.21.15.18:0/4040577817 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7feee800bbc0 con 0x7feef8105020 2024-04-12T21:06:20.154 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:20.157+0000 7feefca00700 1 -- 172.21.15.18:0/4040577817 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feef810ff80 con 0x7feef8105020 2024-04-12T21:06:20.154 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:20.157+0000 7feeee000700 1 -- 172.21.15.18:0/4040577817 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7feee801c630 con 0x7feef8105020 2024-04-12T21:06:20.154 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:20.157+0000 7feefca00700 1 -- 172.21.15.18:0/4040577817 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feef8110540 con 0x7feef8105020 2024-04-12T21:06:20.154 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:20.158+0000 7feefca00700 1 -- 172.21.15.18:0/4040577817 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feef8100770 con 0x7feef8105020 2024-04-12T21:06:20.155 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:20.159+0000 7feeee000700 1 -- 172.21.15.18:0/4040577817 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7feee800bd30 con 0x7feef8105020 2024-04-12T21:06:20.156 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:20.160+0000 7feeee000700 1 --2- 172.21.15.18:0/4040577817 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7feedc06ec00 0x7feedc0710b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:06:20.156 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:20.160+0000 7feeee000700 1 -- 172.21.15.18:0/4040577817 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 6585+0+0 (secure 0 0 0) 0x7feee808fd60 con 0x7feef8105020 2024-04-12T21:06:20.156 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:20.160+0000 7feef5400700 1 --2- 172.21.15.18:0/4040577817 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7feedc06ec00 0x7feedc0710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:06:20.156 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:20.161+0000 7feef5400700 1 --2- 172.21.15.18:0/4040577817 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7feedc06ec00 0x7feedc0710b0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7feef8111430 tx=0x7feee4009040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:06:20.161 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:20.165+0000 7feeee000700 1 -- 172.21.15.18:0/4040577817 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7feee8094070 con 0x7feef8105020 2024-04-12T21:06:20.476 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:20.480+0000 7feefca00700 1 -- 172.21.15.18:0/4040577817 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7feef81111e0 con 0x7feef8105020 2024-04-12T21:06:20.477 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:20.481+0000 7feeee000700 1 -- 172.21.15.18:0/4040577817 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+839 (secure 0 0 0) 0x7feee805c5c0 con 0x7feef8105020 2024-04-12T21:06:20.477 INFO:teuthology.orchestra.run.smithi018.stdout:{ 2024-04-12T21:06:20.477 INFO:teuthology.orchestra.run.smithi018.stdout: "mon": { 2024-04-12T21:06:20.477 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 19.0.0-2879-g324baf8d (324baf8ddb91dea46cac50cb535c306115791355) squid (dev)": 2 2024-04-12T21:06:20.477 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-12T21:06:20.477 INFO:teuthology.orchestra.run.smithi018.stdout: "mgr": { 2024-04-12T21:06:20.477 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 19.0.0-2879-g324baf8d (324baf8ddb91dea46cac50cb535c306115791355) squid (dev)": 2 2024-04-12T21:06:20.478 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-12T21:06:20.478 INFO:teuthology.orchestra.run.smithi018.stdout: "osd": { 2024-04-12T21:06:20.478 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 17.2.7-827-gd3b5060a (d3b5060a17449edfae9385cdf792afec109fca5d) quincy (stable)": 1, 2024-04-12T21:06:20.478 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 19.0.0-2879-g324baf8d (324baf8ddb91dea46cac50cb535c306115791355) squid (dev)": 5 2024-04-12T21:06:20.478 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-12T21:06:20.478 INFO:teuthology.orchestra.run.smithi018.stdout: "mds": { 2024-04-12T21:06:20.478 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 17.2.7-827-gd3b5060a (d3b5060a17449edfae9385cdf792afec109fca5d) quincy (stable)": 4 2024-04-12T21:06:20.478 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-12T21:06:20.478 INFO:teuthology.orchestra.run.smithi018.stdout: "overall": { 2024-04-12T21:06:20.478 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 17.2.7-827-gd3b5060a (d3b5060a17449edfae9385cdf792afec109fca5d) quincy (stable)": 5, 2024-04-12T21:06:20.478 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 19.0.0-2879-g324baf8d (324baf8ddb91dea46cac50cb535c306115791355) squid (dev)": 9 2024-04-12T21:06:20.478 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-04-12T21:06:20.478 INFO:teuthology.orchestra.run.smithi018.stdout:} 2024-04-12T21:06:20.480 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:20.483+0000 7feefca00700 1 -- 172.21.15.18:0/4040577817 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7feedc06ec00 msgr2=0x7feedc0710b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:06:20.480 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:20.483+0000 7feefca00700 1 --2- 172.21.15.18:0/4040577817 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7feedc06ec00 0x7feedc0710b0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7feef8111430 tx=0x7feee4009040 comp rx=0 tx=0).stop 2024-04-12T21:06:20.480 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:20.483+0000 7feefca00700 1 -- 172.21.15.18:0/4040577817 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7feef8105020 msgr2=0x7feef8115b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:06:20.480 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:20.483+0000 7feefca00700 1 --2- 172.21.15.18:0/4040577817 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7feef8105020 0x7feef8115b90 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7feee8009180 tx=0x7feee8009e30 comp rx=0 tx=0).stop 2024-04-12T21:06:20.480 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:20.484+0000 7feefca00700 1 -- 172.21.15.18:0/4040577817 shutdown_connections 2024-04-12T21:06:20.480 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:20.484+0000 7feefca00700 1 --2- 172.21.15.18:0/4040577817 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7feedc06ec00 0x7feedc0710b0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:06:20.480 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:20.484+0000 7feefca00700 1 --2- 172.21.15.18:0/4040577817 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7feef8105020 0x7feef8115b90 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:06:20.480 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:20.484+0000 7feefca00700 1 --2- 172.21.15.18:0/4040577817 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7feef81059c0 0x7feef81160d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:06:20.480 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:20.484+0000 7feefca00700 1 -- 172.21.15.18:0/4040577817 >> 172.21.15.18:0/4040577817 conn(0x7feef80faba0 msgr2=0x7feef80fbb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:06:20.480 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:20.484+0000 7feefca00700 1 -- 172.21.15.18:0/4040577817 shutdown_connections 2024-04-12T21:06:20.480 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:20.484+0000 7feefca00700 1 -- 172.21.15.18:0/4040577817 wait complete. 2024-04-12T21:06:20.632 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:20.636+0000 7f75fdc00700 1 -- 172.21.15.18:0/2159465282 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f75f8105ae0 msgr2=0x7f75f80ff090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:06:20.632 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:20.636+0000 7f75fdc00700 1 --2- 172.21.15.18:0/2159465282 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f75f8105ae0 0x7f75f80ff090 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f75e4004940 tx=0x7f75e4009e30 comp rx=0 tx=0).stop 2024-04-12T21:06:20.632 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:20.636+0000 7f75fdc00700 1 -- 172.21.15.18:0/2159465282 shutdown_connections 2024-04-12T21:06:20.632 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:20.636+0000 7f75fdc00700 1 --2- 172.21.15.18:0/2159465282 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f75f8105ae0 0x7f75f80ff090 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:06:20.632 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:20.636+0000 7f75fdc00700 1 --2- 172.21.15.18:0/2159465282 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f75f8105140 0x7f75f8105510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:06:20.632 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:20.636+0000 7f75fdc00700 1 -- 172.21.15.18:0/2159465282 >> 172.21.15.18:0/2159465282 conn(0x7f75f80fabd0 msgr2=0x7f75f80fcfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:06:20.633 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:20.636+0000 7f75fdc00700 1 -- 172.21.15.18:0/2159465282 shutdown_connections 2024-04-12T21:06:20.633 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:20.636+0000 7f75fdc00700 1 -- 172.21.15.18:0/2159465282 wait complete. 2024-04-12T21:06:20.634 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:20.637+0000 7f75fdc00700 1 Processor -- start 2024-04-12T21:06:20.634 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:20.637+0000 7f75fdc00700 1 -- start start 2024-04-12T21:06:20.634 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:20.638+0000 7f75fdc00700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f75f8105140 0x7f75f8115cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:06:20.634 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:20.638+0000 7f75fdc00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f75f8105ae0 0x7f75f8116200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:06:20.634 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:20.638+0000 7f75fdc00700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75f8116850 con 0x7f75f8105ae0 2024-04-12T21:06:20.634 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:20.638+0000 7f75fdc00700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75f810fe30 con 0x7f75f8105140 2024-04-12T21:06:20.634 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:20.638+0000 7f75f6000700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f75f8105ae0 0x7f75f8116200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:06:20.635 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:20.638+0000 7f75f6000700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f75f8105ae0 0x7f75f8116200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:49496/0 (socket says 172.21.15.18:49496) 2024-04-12T21:06:20.635 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:20.638+0000 7f75f6000700 1 -- 172.21.15.18:0/4048781998 learned_addr learned my addr 172.21.15.18:0/4048781998 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:06:20.635 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:20.638+0000 7f75f6000700 1 -- 172.21.15.18:0/4048781998 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f75f8105140 msgr2=0x7f75f8115cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:06:20.635 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:20.638+0000 7f75f6000700 1 --2- 172.21.15.18:0/4048781998 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f75f8105140 0x7f75f8115cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:06:20.635 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:20.638+0000 7f75f6000700 1 -- 172.21.15.18:0/4048781998 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75e4005040 con 0x7f75f8105ae0 2024-04-12T21:06:20.635 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:20.639+0000 7f75f6000700 1 --2- 172.21.15.18:0/4048781998 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f75f8105ae0 0x7f75f8116200 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f75e4009100 tx=0x7f75e4009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:06:20.636 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:20.639+0000 7f75eea00700 1 -- 172.21.15.18:0/4048781998 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f75e400e040 con 0x7f75f8105ae0 2024-04-12T21:06:20.636 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:20.639+0000 7f75eea00700 1 -- 172.21.15.18:0/4048781998 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f75e4013420 con 0x7f75f8105ae0 2024-04-12T21:06:20.636 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:20.639+0000 7f75fdc00700 1 -- 172.21.15.18:0/4048781998 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f75f81100b0 con 0x7f75f8105ae0 2024-04-12T21:06:20.636 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:20.639+0000 7f75eea00700 1 -- 172.21.15.18:0/4048781998 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f75e401c630 con 0x7f75f8105ae0 2024-04-12T21:06:20.636 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:20.639+0000 7f75fdc00700 1 -- 172.21.15.18:0/4048781998 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f75f8110670 con 0x7f75f8105ae0 2024-04-12T21:06:20.636 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:20.640+0000 7f75fdc00700 1 -- 172.21.15.18:0/4048781998 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f75f804c710 con 0x7f75f8105ae0 2024-04-12T21:06:20.637 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:20.641+0000 7f75eea00700 1 -- 172.21.15.18:0/4048781998 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7f75e4013590 con 0x7f75f8105ae0 2024-04-12T21:06:20.638 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:20.642+0000 7f75eea00700 1 --2- 172.21.15.18:0/4048781998 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f75dc06ec00 0x7f75dc0710b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:06:20.638 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:20.642+0000 7f75eea00700 1 -- 172.21.15.18:0/4048781998 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 6585+0+0 (secure 0 0 0) 0x7f75e408faf0 con 0x7f75f8105ae0 2024-04-12T21:06:20.638 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:20.643+0000 7f75f6a00700 1 --2- 172.21.15.18:0/4048781998 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f75dc06ec00 0x7f75dc0710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:06:20.639 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:20.643+0000 7f75f6a00700 1 --2- 172.21.15.18:0/4048781998 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f75dc06ec00 0x7f75dc0710b0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f75e8004d00 tx=0x7f75e8009ce0 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:06:20.643 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:20.647+0000 7f75eea00700 1 -- 172.21.15.18:0/4048781998 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f75e405c3d0 con 0x7f75f8105ae0 2024-04-12T21:06:20.917 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:20.921+0000 7f75fdc00700 1 -- 172.21.15.18:0/4048781998 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f75f8111310 con 0x7f75f8105ae0 2024-04-12T21:06:20.917 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:20.921+0000 7f75eea00700 1 -- 172.21.15.18:0/4048781998 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 23 v23) v1 ==== 76+0+1886 (secure 0 0 0) 0x7f75e4058030 con 0x7f75f8105ae0 2024-04-12T21:06:20.919 INFO:teuthology.orchestra.run.smithi018.stdout:e23 2024-04-12T21:06:20.919 INFO:teuthology.orchestra.run.smithi018.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-12T21:06:20.919 INFO:teuthology.orchestra.run.smithi018.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-12T21:06:20.919 INFO:teuthology.orchestra.run.smithi018.stdout:legacy client fscid: 1 2024-04-12T21:06:20.919 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-12T21:06:20.920 INFO:teuthology.orchestra.run.smithi018.stdout:Filesystem 'cephfs' (1) 2024-04-12T21:06:20.920 INFO:teuthology.orchestra.run.smithi018.stdout:fs_name cephfs 2024-04-12T21:06:20.920 INFO:teuthology.orchestra.run.smithi018.stdout:epoch 23 2024-04-12T21:06:20.920 INFO:teuthology.orchestra.run.smithi018.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-04-12T21:06:20.920 INFO:teuthology.orchestra.run.smithi018.stdout:created 2024-04-12T20:51:57.362850+0000 2024-04-12T21:06:20.920 INFO:teuthology.orchestra.run.smithi018.stdout:modified 2024-04-12T21:01:10.670287+0000 2024-04-12T21:06:20.920 INFO:teuthology.orchestra.run.smithi018.stdout:tableserver 0 2024-04-12T21:06:20.920 INFO:teuthology.orchestra.run.smithi018.stdout:root 0 2024-04-12T21:06:20.920 INFO:teuthology.orchestra.run.smithi018.stdout:session_timeout 60 2024-04-12T21:06:20.920 INFO:teuthology.orchestra.run.smithi018.stdout:session_autoclose 300 2024-04-12T21:06:20.920 INFO:teuthology.orchestra.run.smithi018.stdout:max_file_size 1099511627776 2024-04-12T21:06:20.920 INFO:teuthology.orchestra.run.smithi018.stdout:max_xattr_size 65536 2024-04-12T21:06:20.920 INFO:teuthology.orchestra.run.smithi018.stdout:required_client_features {} 2024-04-12T21:06:20.920 INFO:teuthology.orchestra.run.smithi018.stdout:last_failure 0 2024-04-12T21:06:20.921 INFO:teuthology.orchestra.run.smithi018.stdout:last_failure_osd_epoch 46 2024-04-12T21:06:20.921 INFO:teuthology.orchestra.run.smithi018.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-12T21:06:20.921 INFO:teuthology.orchestra.run.smithi018.stdout:max_mds 2 2024-04-12T21:06:20.921 INFO:teuthology.orchestra.run.smithi018.stdout:in 0,1 2024-04-12T21:06:20.921 INFO:teuthology.orchestra.run.smithi018.stdout:up {0=14452,1=14464} 2024-04-12T21:06:20.921 INFO:teuthology.orchestra.run.smithi018.stdout:failed 2024-04-12T21:06:20.921 INFO:teuthology.orchestra.run.smithi018.stdout:damaged 2024-04-12T21:06:20.921 INFO:teuthology.orchestra.run.smithi018.stdout:stopped 2024-04-12T21:06:20.921 INFO:teuthology.orchestra.run.smithi018.stdout:data_pools [3] 2024-04-12T21:06:20.921 INFO:teuthology.orchestra.run.smithi018.stdout:metadata_pool 2 2024-04-12T21:06:20.921 INFO:teuthology.orchestra.run.smithi018.stdout:inline_data disabled 2024-04-12T21:06:20.921 INFO:teuthology.orchestra.run.smithi018.stdout:balancer 2024-04-12T21:06:20.921 INFO:teuthology.orchestra.run.smithi018.stdout:bal_rank_mask -1 2024-04-12T21:06:20.921 INFO:teuthology.orchestra.run.smithi018.stdout:standby_count_wanted 1 2024-04-12T21:06:20.922 INFO:teuthology.orchestra.run.smithi018.stdout:qdb_cluster leader: 0 members: 2024-04-12T21:06:20.922 INFO:teuthology.orchestra.run.smithi018.stdout:[mds.cephfs.smithi018.acqbsc{0:14452} state up:active seq 5 join_fscid=1 addr [v2:172.21.15.18:6826/1564065434,v1:172.21.15.18:6827/1564065434] compat {c=[1],r=[1],i=[7ff]}] 2024-04-12T21:06:20.922 INFO:teuthology.orchestra.run.smithi018.stdout:[mds.cephfs.smithi018.szjluo{1:14464} state up:active seq 6 join_fscid=1 addr [v2:172.21.15.18:6828/722443353,v1:172.21.15.18:6829/722443353] compat {c=[1],r=[1],i=[7ff]}] 2024-04-12T21:06:20.922 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-12T21:06:20.922 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-12T21:06:20.922 INFO:teuthology.orchestra.run.smithi018.stdout:Standby daemons: 2024-04-12T21:06:20.922 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-12T21:06:20.922 INFO:teuthology.orchestra.run.smithi018.stdout:[mds.cephfs.smithi186.wjnafs{-1:24277} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.186:6826/438347207,v1:172.21.15.186:6827/438347207] compat {c=[1],r=[1],i=[7ff]}] 2024-04-12T21:06:20.922 INFO:teuthology.orchestra.run.smithi018.stdout:[mds.cephfs.smithi186.jdllyn{-1:24281} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.186:6824/1464359952,v1:172.21.15.186:6825/1464359952] compat {c=[1],r=[1],i=[7ff]}] 2024-04-12T21:06:20.923 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:20.924+0000 7f75fdc00700 1 -- 172.21.15.18:0/4048781998 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f75dc06ec00 msgr2=0x7f75dc0710b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:06:20.923 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:20.924+0000 7f75fdc00700 1 --2- 172.21.15.18:0/4048781998 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f75dc06ec00 0x7f75dc0710b0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f75e8004d00 tx=0x7f75e8009ce0 comp rx=0 tx=0).stop 2024-04-12T21:06:20.923 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:20.924+0000 7f75fdc00700 1 -- 172.21.15.18:0/4048781998 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f75f8105ae0 msgr2=0x7f75f8116200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:06:20.923 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:20.924+0000 7f75fdc00700 1 --2- 172.21.15.18:0/4048781998 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f75f8105ae0 0x7f75f8116200 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f75e4009100 tx=0x7f75e4009e30 comp rx=0 tx=0).stop 2024-04-12T21:06:20.924 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:20.924+0000 7f75fdc00700 1 -- 172.21.15.18:0/4048781998 shutdown_connections 2024-04-12T21:06:20.924 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:20.924+0000 7f75fdc00700 1 --2- 172.21.15.18:0/4048781998 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f75dc06ec00 0x7f75dc0710b0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:06:20.924 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:20.924+0000 7f75fdc00700 1 --2- 172.21.15.18:0/4048781998 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f75f8105140 0x7f75f8115cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:06:20.924 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:20.924+0000 7f75fdc00700 1 --2- 172.21.15.18:0/4048781998 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f75f8105ae0 0x7f75f8116200 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:06:20.924 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:20.924+0000 7f75fdc00700 1 -- 172.21.15.18:0/4048781998 >> 172.21.15.18:0/4048781998 conn(0x7f75f80fabd0 msgr2=0x7f75f80fbbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:06:20.924 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:20.924+0000 7f75fdc00700 1 -- 172.21.15.18:0/4048781998 shutdown_connections 2024-04-12T21:06:20.924 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:20.924+0000 7f75fdc00700 1 -- 172.21.15.18:0/4048781998 wait complete. 2024-04-12T21:06:20.924 INFO:teuthology.orchestra.run.smithi018.stderr:dumped fsmap epoch 23 2024-04-12T21:06:21.029 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:06:20 smithi186 ceph-mon[86379]: from='client.34376 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:06:21.029 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:06:20 smithi186 ceph-mon[86379]: pgmap v187: 65 pgs: 65 active+clean; 257 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-12T21:06:21.029 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:06:20 smithi186 ceph-mon[86379]: from='client.34380 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:06:21.030 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:06:20 smithi186 ceph-mon[86379]: from='client.? 172.21.15.18:0/4040577817' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T21:06:21.076 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:21.079+0000 7fd2b1600700 1 -- 172.21.15.18:0/3625520004 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd2ac102ca0 msgr2=0x7fd2ac103070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:06:21.076 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:21.079+0000 7fd2b1600700 1 --2- 172.21.15.18:0/3625520004 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd2ac102ca0 0x7fd2ac103070 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fd29c009ce0 tx=0x7fd29c004d10 comp rx=0 tx=0).stop 2024-04-12T21:06:21.076 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:21.080+0000 7fd2b1600700 1 -- 172.21.15.18:0/3625520004 shutdown_connections 2024-04-12T21:06:21.076 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:21.080+0000 7fd2b1600700 1 --2- 172.21.15.18:0/3625520004 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd2ac065630 0x7fd2ac065a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:06:21.076 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:21.080+0000 7fd2b1600700 1 --2- 172.21.15.18:0/3625520004 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd2ac102ca0 0x7fd2ac103070 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:06:21.076 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:21.080+0000 7fd2b1600700 1 -- 172.21.15.18:0/3625520004 >> 172.21.15.18:0/3625520004 conn(0x7fd2ac071b70 msgr2=0x7fd2ac071f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:06:21.076 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:06:20 smithi018 ceph-mon[112590]: from='client.34376 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:06:21.077 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:06:20 smithi018 ceph-mon[112590]: pgmap v187: 65 pgs: 65 active+clean; 257 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-12T21:06:21.077 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:06:20 smithi018 ceph-mon[112590]: from='client.34380 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:06:21.077 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:06:20 smithi018 ceph-mon[112590]: from='client.? 172.21.15.18:0/4040577817' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T21:06:21.078 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:21.080+0000 7fd2b1600700 1 -- 172.21.15.18:0/3625520004 shutdown_connections 2024-04-12T21:06:21.078 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:21.080+0000 7fd2b1600700 1 -- 172.21.15.18:0/3625520004 wait complete. 2024-04-12T21:06:21.078 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:21.081+0000 7fd2b1600700 1 Processor -- start 2024-04-12T21:06:21.078 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:21.081+0000 7fd2b1600700 1 -- start start 2024-04-12T21:06:21.078 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:21.081+0000 7fd2b1600700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd2ac065630 0x7fd2ac115d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:06:21.078 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:21.081+0000 7fd2b1600700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd2ac102ca0 0x7fd2ac116280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:06:21.078 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:21.081+0000 7fd2b1600700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2ac1168d0 con 0x7fd2ac065630 2024-04-12T21:06:21.078 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:21.082+0000 7fd2b1600700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2ac10fca0 con 0x7fd2ac102ca0 2024-04-12T21:06:21.079 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:21.082+0000 7fd2abe00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd2ac065630 0x7fd2ac115d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:06:21.079 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:21.082+0000 7fd2abe00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd2ac065630 0x7fd2ac115d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:53380/0 (socket says 172.21.15.18:53380) 2024-04-12T21:06:21.079 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:21.082+0000 7fd2abe00700 1 -- 172.21.15.18:0/506263929 learned_addr learned my addr 172.21.15.18:0/506263929 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:06:21.079 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:21.082+0000 7fd2abe00700 1 -- 172.21.15.18:0/506263929 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd2ac102ca0 msgr2=0x7fd2ac116280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:06:21.079 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:21.082+0000 7fd2abe00700 1 --2- 172.21.15.18:0/506263929 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd2ac102ca0 0x7fd2ac116280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:06:21.079 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:21.082+0000 7fd2abe00700 1 -- 172.21.15.18:0/506263929 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd29c009940 con 0x7fd2ac065630 2024-04-12T21:06:21.080 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:21.083+0000 7fd2abe00700 1 --2- 172.21.15.18:0/506263929 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd2ac065630 0x7fd2ac115d40 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fd29c00be90 tx=0x7fd29c004d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:06:21.080 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:21.083+0000 7fd2a8c00700 1 -- 172.21.15.18:0/506263929 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd29c00e040 con 0x7fd2ac065630 2024-04-12T21:06:21.080 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:21.083+0000 7fd2a8c00700 1 -- 172.21.15.18:0/506263929 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd29c008ea0 con 0x7fd2ac065630 2024-04-12T21:06:21.080 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:21.083+0000 7fd2b1600700 1 -- 172.21.15.18:0/506263929 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd2ac10ff20 con 0x7fd2ac065630 2024-04-12T21:06:21.080 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:21.083+0000 7fd2a8c00700 1 -- 172.21.15.18:0/506263929 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd29c01c810 con 0x7fd2ac065630 2024-04-12T21:06:21.080 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:21.083+0000 7fd2b1600700 1 -- 172.21.15.18:0/506263929 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd2ac1104e0 con 0x7fd2ac065630 2024-04-12T21:06:21.080 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:21.085+0000 7fd2b1600700 1 -- 172.21.15.18:0/506263929 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd2ac0fe380 con 0x7fd2ac065630 2024-04-12T21:06:21.081 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:21.085+0000 7fd2a8c00700 1 -- 172.21.15.18:0/506263929 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7fd29c012c00 con 0x7fd2ac065630 2024-04-12T21:06:21.083 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:21.087+0000 7fd2a8c00700 1 --2- 172.21.15.18:0/506263929 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fd29006ec00 0x7fd2900710b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:06:21.083 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:21.087+0000 7fd2a8c00700 1 -- 172.21.15.18:0/506263929 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 6585+0+0 (secure 0 0 0) 0x7fd29c0904f0 con 0x7fd2ac065630 2024-04-12T21:06:21.083 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:21.087+0000 7fd2a3400700 1 --2- 172.21.15.18:0/506263929 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fd29006ec00 0x7fd2900710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:06:21.084 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:21.088+0000 7fd2a3400700 1 --2- 172.21.15.18:0/506263929 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fd29006ec00 0x7fd2900710b0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fd2ac1115e0 tx=0x7fd294007d40 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:06:21.087 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:21.091+0000 7fd2a8c00700 1 -- 172.21.15.18:0/506263929 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7fd29c05cd50 con 0x7fd2ac065630 2024-04-12T21:06:21.339 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:21.342+0000 7fd2b1600700 1 -- 172.21.15.18:0/506263929 --> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fd2ac109ce0 con 0x7fd29006ec00 2024-04-12T21:06:21.341 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:21.345+0000 7fd2a8c00700 1 -- 172.21.15.18:0/506263929 <== mgr.34102 v2:172.21.15.18:6800/3812891919 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+401 (secure 0 0 0) 0x7fd2ac109ce0 con 0x7fd29006ec00 2024-04-12T21:06:21.341 INFO:teuthology.orchestra.run.smithi018.stdout:{ 2024-04-12T21:06:21.342 INFO:teuthology.orchestra.run.smithi018.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:f4452def3be4d350f4a8e5feaf7ebb1511be9298fe68478ba7a2d6fbd7d69a63", 2024-04-12T21:06:21.342 INFO:teuthology.orchestra.run.smithi018.stdout: "in_progress": true, 2024-04-12T21:06:21.342 INFO:teuthology.orchestra.run.smithi018.stdout: "which": "Upgrading all daemon types on all hosts", 2024-04-12T21:06:21.342 INFO:teuthology.orchestra.run.smithi018.stdout: "services_complete": [ 2024-04-12T21:06:21.342 INFO:teuthology.orchestra.run.smithi018.stdout: "crash", 2024-04-12T21:06:21.342 INFO:teuthology.orchestra.run.smithi018.stdout: "mon", 2024-04-12T21:06:21.343 INFO:teuthology.orchestra.run.smithi018.stdout: "mgr" 2024-04-12T21:06:21.343 INFO:teuthology.orchestra.run.smithi018.stdout: ], 2024-04-12T21:06:21.343 INFO:teuthology.orchestra.run.smithi018.stdout: "progress": "11/21 daemons upgraded", 2024-04-12T21:06:21.343 INFO:teuthology.orchestra.run.smithi018.stdout: "message": "Currently upgrading osd daemons", 2024-04-12T21:06:21.343 INFO:teuthology.orchestra.run.smithi018.stdout: "is_paused": false 2024-04-12T21:06:21.343 INFO:teuthology.orchestra.run.smithi018.stdout:} 2024-04-12T21:06:21.344 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:21.348+0000 7fd2b1600700 1 -- 172.21.15.18:0/506263929 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fd29006ec00 msgr2=0x7fd2900710b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:06:21.344 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:21.348+0000 7fd2b1600700 1 --2- 172.21.15.18:0/506263929 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fd29006ec00 0x7fd2900710b0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fd2ac1115e0 tx=0x7fd294007d40 comp rx=0 tx=0).stop 2024-04-12T21:06:21.344 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:21.348+0000 7fd2b1600700 1 -- 172.21.15.18:0/506263929 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd2ac065630 msgr2=0x7fd2ac115d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:06:21.344 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:21.348+0000 7fd2b1600700 1 --2- 172.21.15.18:0/506263929 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd2ac065630 0x7fd2ac115d40 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fd29c00be90 tx=0x7fd29c004d10 comp rx=0 tx=0).stop 2024-04-12T21:06:21.345 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:21.348+0000 7fd2b1600700 1 -- 172.21.15.18:0/506263929 shutdown_connections 2024-04-12T21:06:21.345 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:21.348+0000 7fd2b1600700 1 --2- 172.21.15.18:0/506263929 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fd29006ec00 0x7fd2900710b0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:06:21.345 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:21.348+0000 7fd2b1600700 1 --2- 172.21.15.18:0/506263929 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd2ac065630 0x7fd2ac115d40 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:06:21.345 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:21.348+0000 7fd2b1600700 1 --2- 172.21.15.18:0/506263929 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd2ac102ca0 0x7fd2ac116280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:06:21.345 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:21.348+0000 7fd2b1600700 1 -- 172.21.15.18:0/506263929 >> 172.21.15.18:0/506263929 conn(0x7fd2ac071b70 msgr2=0x7fd2ac0f9fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:06:21.345 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:21.349+0000 7fd2b1600700 1 -- 172.21.15.18:0/506263929 shutdown_connections 2024-04-12T21:06:21.345 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:21.349+0000 7fd2b1600700 1 -- 172.21.15.18:0/506263929 wait complete. 2024-04-12T21:06:21.502 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:21.505+0000 7fcdeba00700 1 -- 172.21.15.18:0/545776422 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fcde4070c70 msgr2=0x7fcde41090f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:06:21.502 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:21.505+0000 7fcdeba00700 1 --2- 172.21.15.18:0/545776422 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fcde4070c70 0x7fcde41090f0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fcdd0004970 tx=0x7fcdd0009e30 comp rx=0 tx=0).stop 2024-04-12T21:06:21.502 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:21.505+0000 7fcdeba00700 1 -- 172.21.15.18:0/545776422 shutdown_connections 2024-04-12T21:06:21.502 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:21.505+0000 7fcdeba00700 1 --2- 172.21.15.18:0/545776422 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fcde4070c70 0x7fcde41090f0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:06:21.502 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:21.505+0000 7fcdeba00700 1 --2- 172.21.15.18:0/545776422 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcde4070360 0x7fcde4070730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:06:21.502 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:21.505+0000 7fcdeba00700 1 -- 172.21.15.18:0/545776422 >> 172.21.15.18:0/545776422 conn(0x7fcde40f8970 msgr2=0x7fcde40fad80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:06:21.502 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:21.506+0000 7fcdeba00700 1 -- 172.21.15.18:0/545776422 shutdown_connections 2024-04-12T21:06:21.502 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:21.506+0000 7fcdeba00700 1 -- 172.21.15.18:0/545776422 wait complete. 2024-04-12T21:06:21.504 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:21.506+0000 7fcdeba00700 1 Processor -- start 2024-04-12T21:06:21.504 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:21.507+0000 7fcdeba00700 1 -- start start 2024-04-12T21:06:21.504 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:21.507+0000 7fcdeba00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fcde4070360 0x7fcde4115d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:06:21.504 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:21.507+0000 7fcdeba00700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcde4070c70 0x7fcde4116290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:06:21.504 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:21.507+0000 7fcdeba00700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcde41168e0 con 0x7fcde4070360 2024-04-12T21:06:21.504 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:21.507+0000 7fcdeba00700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcde410fcb0 con 0x7fcde4070c70 2024-04-12T21:06:21.504 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:21.507+0000 7fcde9000700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fcde4070360 0x7fcde4115d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:06:21.504 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:21.507+0000 7fcde9000700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fcde4070360 0x7fcde4115d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:53402/0 (socket says 172.21.15.18:53402) 2024-04-12T21:06:21.504 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:21.507+0000 7fcde9000700 1 -- 172.21.15.18:0/888490892 learned_addr learned my addr 172.21.15.18:0/888490892 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:06:21.504 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:21.508+0000 7fcde3e00700 1 --2- 172.21.15.18:0/888490892 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcde4070c70 0x7fcde4116290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:06:21.504 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:21.508+0000 7fcde9000700 1 -- 172.21.15.18:0/888490892 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcde4070c70 msgr2=0x7fcde4116290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:06:21.504 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:21.508+0000 7fcde9000700 1 --2- 172.21.15.18:0/888490892 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcde4070c70 0x7fcde4116290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:06:21.505 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:21.508+0000 7fcde9000700 1 -- 172.21.15.18:0/888490892 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcdd0005040 con 0x7fcde4070360 2024-04-12T21:06:21.505 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:21.508+0000 7fcde9000700 1 --2- 172.21.15.18:0/888490892 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fcde4070360 0x7fcde4115d50 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fcdd800c8c0 tx=0x7fcdd80044c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:06:21.505 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:21.509+0000 7fcde0c00700 1 -- 172.21.15.18:0/888490892 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcdd800a980 con 0x7fcde4070360 2024-04-12T21:06:21.506 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:21.509+0000 7fcde0c00700 1 -- 172.21.15.18:0/888490892 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fcdd800ce60 con 0x7fcde4070360 2024-04-12T21:06:21.506 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:21.509+0000 7fcdeba00700 1 -- 172.21.15.18:0/888490892 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcde410ff90 con 0x7fcde4070360 2024-04-12T21:06:21.506 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:21.509+0000 7fcde0c00700 1 -- 172.21.15.18:0/888490892 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcdd80115e0 con 0x7fcde4070360 2024-04-12T21:06:21.506 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:21.509+0000 7fcdeba00700 1 -- 172.21.15.18:0/888490892 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcde41104d0 con 0x7fcde4070360 2024-04-12T21:06:21.506 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:21.510+0000 7fcdeba00700 1 -- 172.21.15.18:0/888490892 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcde4071aa0 con 0x7fcde4070360 2024-04-12T21:06:21.507 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:21.511+0000 7fcde0c00700 1 -- 172.21.15.18:0/888490892 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7fcdd8012400 con 0x7fcde4070360 2024-04-12T21:06:21.508 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:21.512+0000 7fcde0c00700 1 --2- 172.21.15.18:0/888490892 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fcdcc06ec10 0x7fcdcc0710c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:06:21.508 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:21.512+0000 7fcde0c00700 1 -- 172.21.15.18:0/888490892 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 6585+0+0 (secure 0 0 0) 0x7fcdd808e8d0 con 0x7fcde4070360 2024-04-12T21:06:21.508 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:21.512+0000 7fcde3e00700 1 --2- 172.21.15.18:0/888490892 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fcdcc06ec10 0x7fcdcc0710c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:06:21.509 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:21.513+0000 7fcde3e00700 1 --2- 172.21.15.18:0/888490892 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fcdcc06ec10 0x7fcdcc0710c0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fcde41115f0 tx=0x7fcdd0009e30 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:06:21.513 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:21.517+0000 7fcde0c00700 1 -- 172.21.15.18:0/888490892 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7fcdd805b270 con 0x7fcde4070360 2024-04-12T21:06:21.827 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:21.831+0000 7fcdeba00700 1 -- 172.21.15.18:0/888490892 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7fcde405f5f0 con 0x7fcde4070360 2024-04-12T21:06:21.827 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:21.831+0000 7fcde0c00700 1 -- 172.21.15.18:0/888490892 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+10 (secure 0 0 0) 0x7fcdd80126b0 con 0x7fcde4070360 2024-04-12T21:06:21.828 INFO:teuthology.orchestra.run.smithi018.stdout:HEALTH_OK 2024-04-12T21:06:21.831 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:21.834+0000 7fcdeba00700 1 -- 172.21.15.18:0/888490892 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fcdcc06ec10 msgr2=0x7fcdcc0710c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:06:21.831 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:21.834+0000 7fcdeba00700 1 --2- 172.21.15.18:0/888490892 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fcdcc06ec10 0x7fcdcc0710c0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fcde41115f0 tx=0x7fcdd0009e30 comp rx=0 tx=0).stop 2024-04-12T21:06:21.831 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:21.834+0000 7fcdeba00700 1 -- 172.21.15.18:0/888490892 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fcde4070360 msgr2=0x7fcde4115d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:06:21.831 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:21.834+0000 7fcdeba00700 1 --2- 172.21.15.18:0/888490892 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fcde4070360 0x7fcde4115d50 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fcdd800c8c0 tx=0x7fcdd80044c0 comp rx=0 tx=0).stop 2024-04-12T21:06:21.831 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:21.835+0000 7fcdeba00700 1 -- 172.21.15.18:0/888490892 shutdown_connections 2024-04-12T21:06:21.831 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:21.835+0000 7fcdeba00700 1 --2- 172.21.15.18:0/888490892 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fcdcc06ec10 0x7fcdcc0710c0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:06:21.831 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:21.835+0000 7fcdeba00700 1 --2- 172.21.15.18:0/888490892 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fcde4070360 0x7fcde4115d50 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:06:21.831 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:21.835+0000 7fcdeba00700 1 --2- 172.21.15.18:0/888490892 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcde4070c70 0x7fcde4116290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:06:21.831 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:21.835+0000 7fcdeba00700 1 -- 172.21.15.18:0/888490892 >> 172.21.15.18:0/888490892 conn(0x7fcde40f8970 msgr2=0x7fcde40f9d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:06:21.831 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:21.835+0000 7fcdeba00700 1 -- 172.21.15.18:0/888490892 shutdown_connections 2024-04-12T21:06:21.831 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:21.835+0000 7fcdeba00700 1 -- 172.21.15.18:0/888490892 wait complete. 2024-04-12T21:06:22.029 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:06:21 smithi186 ceph-mon[86379]: from='client.? 172.21.15.18:0/4048781998' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-12T21:06:22.029 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:06:21 smithi186 ceph-mon[86379]: from='client.34392 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:06:22.101 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:06:21 smithi018 ceph-mon[112590]: from='client.? 172.21.15.18:0/4048781998' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-12T21:06:22.101 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:06:21 smithi018 ceph-mon[112590]: from='client.34392 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:06:23.029 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:06:22 smithi186 ceph-mon[86379]: from='client.? 172.21.15.18:0/888490892' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-12T21:06:23.029 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:06:22 smithi186 ceph-mon[86379]: pgmap v188: 65 pgs: 65 active+clean; 257 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-12T21:06:23.100 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:06:22 smithi018 ceph-mon[112590]: from='client.? 172.21.15.18:0/888490892' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-12T21:06:23.100 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:06:22 smithi018 ceph-mon[112590]: pgmap v188: 65 pgs: 65 active+clean; 257 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-12T21:06:25.279 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:06:24 smithi186 ceph-mon[86379]: pgmap v189: 65 pgs: 65 active+clean; 257 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-12T21:06:25.279 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:06:24 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' 2024-04-12T21:06:25.279 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:06:24 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T21:06:25.350 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:06:24 smithi018 ceph-mon[112590]: pgmap v189: 65 pgs: 65 active+clean; 257 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-12T21:06:25.350 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:06:24 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' 2024-04-12T21:06:25.350 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:06:24 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T21:06:26.279 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:06:25 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' 2024-04-12T21:06:26.350 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:06:25 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' 2024-04-12T21:06:27.279 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:06:26 smithi186 ceph-mon[86379]: pgmap v190: 65 pgs: 65 active+clean; 257 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-12T21:06:27.350 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:06:26 smithi018 ceph-mon[112590]: pgmap v190: 65 pgs: 65 active+clean; 257 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-12T21:06:29.279 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:06:28 smithi186 ceph-mon[86379]: pgmap v191: 65 pgs: 65 active+clean; 257 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-12T21:06:29.350 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:06:28 smithi018 ceph-mon[112590]: pgmap v191: 65 pgs: 65 active+clean; 257 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-12T21:06:31.279 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:06:30 smithi186 ceph-mon[86379]: pgmap v192: 65 pgs: 65 active+clean; 257 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-12T21:06:31.279 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:06:30 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd ok-to-stop", "ids": ["5"], "max": 16}]: dispatch 2024-04-12T21:06:31.279 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:06:30 smithi186 ceph-mon[86379]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["5"], "max": 16}]: dispatch 2024-04-12T21:06:31.280 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:06:30 smithi186 ceph-mon[86379]: Upgrade: osd.5 is safe to restart 2024-04-12T21:06:31.350 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:06:30 smithi018 ceph-mon[112590]: pgmap v192: 65 pgs: 65 active+clean; 257 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-12T21:06:31.350 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:06:30 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd ok-to-stop", "ids": ["5"], "max": 16}]: dispatch 2024-04-12T21:06:31.350 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:06:30 smithi018 ceph-mon[112590]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["5"], "max": 16}]: dispatch 2024-04-12T21:06:31.350 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:06:30 smithi018 ceph-mon[112590]: Upgrade: osd.5 is safe to restart 2024-04-12T21:06:33.587 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:06:33 smithi186 ceph-mon[86379]: pgmap v193: 65 pgs: 65 active+clean; 257 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-12T21:06:33.587 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:06:33 smithi186 ceph-mon[86379]: Upgrade: Updating osd.5 2024-04-12T21:06:33.587 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:06:33 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' 2024-04-12T21:06:33.587 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:06:33 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-04-12T21:06:33.587 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:06:33 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-12T21:06:33.587 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:06:33 smithi186 ceph-mon[86379]: Deploying daemon osd.5 on smithi186 2024-04-12T21:06:33.851 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:06:33 smithi018 ceph-mon[112590]: pgmap v193: 65 pgs: 65 active+clean; 257 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-12T21:06:33.851 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:06:33 smithi018 ceph-mon[112590]: Upgrade: Updating osd.5 2024-04-12T21:06:33.851 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:06:33 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' 2024-04-12T21:06:33.851 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:06:33 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-04-12T21:06:33.851 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:06:33 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-12T21:06:33.851 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:06:33 smithi018 ceph-mon[112590]: Deploying daemon osd.5 on smithi186 2024-04-12T21:06:35.542 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:06:35 smithi186 ceph-mon[86379]: pgmap v194: 65 pgs: 65 active+clean; 257 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-12T21:06:35.850 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:06:35 smithi018 ceph-mon[112590]: pgmap v194: 65 pgs: 65 active+clean; 257 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-12T21:06:37.529 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:06:37 smithi186 ceph-mon[86379]: pgmap v195: 65 pgs: 65 active+clean; 257 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-12T21:06:37.530 INFO:journalctl@ceph.osd.5.smithi186.stdout:Apr 12 21:06:37 smithi186 systemd[1]: Stopping Ceph osd.5 for 1cbe407a-f90d-11ee-bc8d-c7b262605968... 2024-04-12T21:06:37.850 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:06:37 smithi018 ceph-mon[112590]: pgmap v195: 65 pgs: 65 active+clean; 257 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-12T21:06:38.030 INFO:journalctl@ceph.osd.5.smithi186.stdout:Apr 12 21:06:37 smithi186 ceph-1cbe407a-f90d-11ee-bc8d-c7b262605968-osd-5[55456]: 2024-04-12T21:06:37.695+0000 7fbd38c00700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.5 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-12T21:06:38.031 INFO:journalctl@ceph.osd.5.smithi186.stdout:Apr 12 21:06:37 smithi186 ceph-1cbe407a-f90d-11ee-bc8d-c7b262605968-osd-5[55456]: 2024-04-12T21:06:37.696+0000 7fbd38c00700 -1 osd.5 93 *** Got signal Terminated *** 2024-04-12T21:06:38.031 INFO:journalctl@ceph.osd.5.smithi186.stdout:Apr 12 21:06:37 smithi186 ceph-1cbe407a-f90d-11ee-bc8d-c7b262605968-osd-5[55456]: 2024-04-12T21:06:37.696+0000 7fbd38c00700 -1 osd.5 93 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-12T21:06:38.779 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:06:38 smithi186 ceph-mon[86379]: osd.5 marked itself down and dead 2024-04-12T21:06:38.850 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:06:38 smithi018 ceph-mon[112590]: osd.5 marked itself down and dead 2024-04-12T21:06:39.410 INFO:journalctl@ceph.osd.5.smithi186.stdout:Apr 12 21:06:39 smithi186 podman[105528]: 2024-04-12 21:06:39.07411501 +0000 UTC m=+1.524074486 container died 2ee072a436d0c54ee7b01683c902d2f1b6b316d6fd5d9c49b798fc7bcef9aa4a (image=quay.ceph.io/ceph-ci/ceph@sha256:5e334df698f432180b78bc9c46936bfdfbc92bc9a8983463e523f780d23765fe, name=ceph-1cbe407a-f90d-11ee-bc8d-c7b262605968-osd-5, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , ceph=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240409, GIT_BRANCH=HEAD, RELEASE=quincy-d3b5060, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0) 2024-04-12T21:06:39.744 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:06:39 smithi186 ceph-mon[86379]: pgmap v196: 65 pgs: 65 active+clean; 257 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-12T21:06:39.745 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:06:39 smithi186 ceph-mon[86379]: Health check failed: 1 osds down (OSD_DOWN) 2024-04-12T21:06:39.745 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:06:39 smithi186 ceph-mon[86379]: Health check failed: all OSDs are running squid or later but require_osd_release < squid (OSD_UPGRADE_FINISHED) 2024-04-12T21:06:39.745 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:06:39 smithi186 ceph-mon[86379]: osdmap e94: 6 total, 5 up, 6 in 2024-04-12T21:06:39.745 INFO:journalctl@ceph.osd.5.smithi186.stdout:Apr 12 21:06:39 smithi186 podman[105528]: 2024-04-12 21:06:39.547172113 +0000 UTC m=+1.997131591 container cleanup 2ee072a436d0c54ee7b01683c902d2f1b6b316d6fd5d9c49b798fc7bcef9aa4a (image=quay.ceph.io/ceph-ci/ceph@sha256:5e334df698f432180b78bc9c46936bfdfbc92bc9a8983463e523f780d23765fe, name=ceph-1cbe407a-f90d-11ee-bc8d-c7b262605968-osd-5, org.label-schema.vendor=CentOS, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, org.label-schema.build-date=20240409, RELEASE=quincy-d3b5060, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, ceph=True, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-12T21:06:39.745 INFO:journalctl@ceph.osd.5.smithi186.stdout:Apr 12 21:06:39 smithi186 bash[105528]: ceph-1cbe407a-f90d-11ee-bc8d-c7b262605968-osd-5 2024-04-12T21:06:39.850 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:06:39 smithi018 ceph-mon[112590]: pgmap v196: 65 pgs: 65 active+clean; 257 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-12T21:06:39.851 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:06:39 smithi018 ceph-mon[112590]: Health check failed: 1 osds down (OSD_DOWN) 2024-04-12T21:06:39.851 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:06:39 smithi018 ceph-mon[112590]: Health check failed: all OSDs are running squid or later but require_osd_release < squid (OSD_UPGRADE_FINISHED) 2024-04-12T21:06:39.851 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:06:39 smithi018 ceph-mon[112590]: osdmap e94: 6 total, 5 up, 6 in 2024-04-12T21:06:40.029 INFO:journalctl@ceph.osd.5.smithi186.stdout:Apr 12 21:06:39 smithi186 podman[105550]: 2024-04-12 21:06:39.748121053 +0000 UTC m=+0.664755802 container remove 2ee072a436d0c54ee7b01683c902d2f1b6b316d6fd5d9c49b798fc7bcef9aa4a (image=quay.ceph.io/ceph-ci/ceph@sha256:5e334df698f432180b78bc9c46936bfdfbc92bc9a8983463e523f780d23765fe, name=ceph-1cbe407a-f90d-11ee-bc8d-c7b262605968-osd-5, org.label-schema.vendor=CentOS, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240409, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=quincy-d3b5060, org.label-schema.license=GPLv2, io.buildah.version=1.33.5) 2024-04-12T21:06:40.748 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:06:40 smithi186 ceph-mon[86379]: osdmap e95: 6 total, 5 up, 6 in 2024-04-12T21:06:40.748 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:06:40 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' 2024-04-12T21:06:40.748 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:06:40 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T21:06:40.748 INFO:journalctl@ceph.osd.5.smithi186.stdout:Apr 12 21:06:40 smithi186 podman[105638]: 2024-04-12 21:06:40.468881159 +0000 UTC m=+0.037034802 image pull 8717720dc5e5c56303818ea4d4f0a11f6e2c03dee521fb668f411f1304e0afca quay.ceph.io/ceph-ci/ceph@sha256:f4452def3be4d350f4a8e5feaf7ebb1511be9298fe68478ba7a2d6fbd7d69a63 2024-04-12T21:06:40.748 INFO:journalctl@ceph.osd.5.smithi186.stdout:Apr 12 21:06:40 smithi186 podman[105638]: 2024-04-12 21:06:40.573996435 +0000 UTC m=+0.142150103 container create 96c299146f64b03817c90102f5c24692724b81ba96e5f35d8d7aa39e92e6f4f7 (image=quay.ceph.io/ceph-ci/ceph@sha256:f4452def3be4d350f4a8e5feaf7ebb1511be9298fe68478ba7a2d6fbd7d69a63, name=ceph-1cbe407a-f90d-11ee-bc8d-c7b262605968-osd-5-deactivate, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, RELEASE=wip-vshankar-testing-20240411.061452-324baf8, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, org.label-schema.build-date=20240409, CEPH_POINT_RELEASE=) 2024-04-12T21:06:40.752 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:06:40 smithi018 ceph-mon[112590]: osdmap e95: 6 total, 5 up, 6 in 2024-04-12T21:06:40.752 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:06:40 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' 2024-04-12T21:06:40.752 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:06:40 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T21:06:41.029 INFO:journalctl@ceph.osd.5.smithi186.stdout:Apr 12 21:06:40 smithi186 podman[105638]: 2024-04-12 21:06:40.878489272 +0000 UTC m=+0.446642946 container init 96c299146f64b03817c90102f5c24692724b81ba96e5f35d8d7aa39e92e6f4f7 (image=quay.ceph.io/ceph-ci/ceph@sha256:f4452def3be4d350f4a8e5feaf7ebb1511be9298fe68478ba7a2d6fbd7d69a63, name=ceph-1cbe407a-f90d-11ee-bc8d-c7b262605968-osd-5-deactivate, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, GIT_CLEAN=True, RELEASE=wip-vshankar-testing-20240411.061452-324baf8, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, ceph=True, org.label-schema.build-date=20240409) 2024-04-12T21:06:41.029 INFO:journalctl@ceph.osd.5.smithi186.stdout:Apr 12 21:06:40 smithi186 podman[105638]: 2024-04-12 21:06:40.890022126 +0000 UTC m=+0.458175779 container start 96c299146f64b03817c90102f5c24692724b81ba96e5f35d8d7aa39e92e6f4f7 (image=quay.ceph.io/ceph-ci/ceph@sha256:f4452def3be4d350f4a8e5feaf7ebb1511be9298fe68478ba7a2d6fbd7d69a63, name=ceph-1cbe407a-f90d-11ee-bc8d-c7b262605968-osd-5-deactivate, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , GIT_CLEAN=True, RELEASE=wip-vshankar-testing-20240411.061452-324baf8, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240409, org.label-schema.license=GPLv2, ceph=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0) 2024-04-12T21:06:41.030 INFO:journalctl@ceph.osd.5.smithi186.stdout:Apr 12 21:06:40 smithi186 podman[105638]: 2024-04-12 21:06:40.968044751 +0000 UTC m=+0.536198462 container attach 96c299146f64b03817c90102f5c24692724b81ba96e5f35d8d7aa39e92e6f4f7 (image=quay.ceph.io/ceph-ci/ceph@sha256:f4452def3be4d350f4a8e5feaf7ebb1511be9298fe68478ba7a2d6fbd7d69a63, name=ceph-1cbe407a-f90d-11ee-bc8d-c7b262605968-osd-5-deactivate, ceph=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_CLEAN=True, RELEASE=wip-vshankar-testing-20240411.061452-324baf8, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240409, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux ) 2024-04-12T21:06:41.691 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:06:41 smithi186 ceph-mon[86379]: pgmap v199: 65 pgs: 3 peering, 12 stale+active+clean, 50 active+clean; 257 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-12T21:06:41.691 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:06:41 smithi186 ceph-mon[86379]: Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY) 2024-04-12T21:06:41.692 INFO:journalctl@ceph.osd.5.smithi186.stdout:Apr 12 21:06:41 smithi186 podman[105638]: 2024-04-12 21:06:41.350350503 +0000 UTC m=+0.918504194 container died 96c299146f64b03817c90102f5c24692724b81ba96e5f35d8d7aa39e92e6f4f7 (image=quay.ceph.io/ceph-ci/ceph@sha256:f4452def3be4d350f4a8e5feaf7ebb1511be9298fe68478ba7a2d6fbd7d69a63, name=ceph-1cbe407a-f90d-11ee-bc8d-c7b262605968-osd-5-deactivate, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240409, RELEASE=wip-vshankar-testing-20240411.061452-324baf8, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-12T21:06:41.850 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:06:41 smithi018 ceph-mon[112590]: pgmap v199: 65 pgs: 3 peering, 12 stale+active+clean, 50 active+clean; 257 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-12T21:06:41.850 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:06:41 smithi018 ceph-mon[112590]: Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY) 2024-04-12T21:06:42.280 INFO:journalctl@ceph.osd.5.smithi186.stdout:Apr 12 21:06:41 smithi186 podman[105664]: 2024-04-12 21:06:41.963008359 +0000 UTC m=+0.598898923 container remove 96c299146f64b03817c90102f5c24692724b81ba96e5f35d8d7aa39e92e6f4f7 (image=quay.ceph.io/ceph-ci/ceph@sha256:f4452def3be4d350f4a8e5feaf7ebb1511be9298fe68478ba7a2d6fbd7d69a63, name=ceph-1cbe407a-f90d-11ee-bc8d-c7b262605968-osd-5-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, RELEASE=wip-vshankar-testing-20240411.061452-324baf8, GIT_BRANCH=HEAD, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240409) 2024-04-12T21:06:42.280 INFO:journalctl@ceph.osd.5.smithi186.stdout:Apr 12 21:06:41 smithi186 systemd[1]: ceph-1cbe407a-f90d-11ee-bc8d-c7b262605968@osd.5.service: Deactivated successfully. 2024-04-12T21:06:42.280 INFO:journalctl@ceph.osd.5.smithi186.stdout:Apr 12 21:06:41 smithi186 systemd[1]: Stopped Ceph osd.5 for 1cbe407a-f90d-11ee-bc8d-c7b262605968. 2024-04-12T21:06:42.280 INFO:journalctl@ceph.osd.5.smithi186.stdout:Apr 12 21:06:41 smithi186 systemd[1]: ceph-1cbe407a-f90d-11ee-bc8d-c7b262605968@osd.5.service: Consumed 2min 19.891s CPU time. 2024-04-12T21:06:42.700 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:06:42 smithi186 ceph-mon[86379]: Health check failed: Degraded data redundancy: 23/324 objects degraded (7.099%), 8 pgs degraded (PG_DEGRADED) 2024-04-12T21:06:42.700 INFO:journalctl@ceph.osd.5.smithi186.stdout:Apr 12 21:06:42 smithi186 systemd[1]: Starting Ceph osd.5 for 1cbe407a-f90d-11ee-bc8d-c7b262605968... 2024-04-12T21:06:42.850 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:06:42 smithi018 ceph-mon[112590]: Health check failed: Degraded data redundancy: 23/324 objects degraded (7.099%), 8 pgs degraded (PG_DEGRADED) 2024-04-12T21:06:43.005 INFO:journalctl@ceph.osd.5.smithi186.stdout:Apr 12 21:06:42 smithi186 podman[105765]: 2024-04-12 21:06:42.611016207 +0000 UTC m=+0.032798673 image pull 8717720dc5e5c56303818ea4d4f0a11f6e2c03dee521fb668f411f1304e0afca quay.ceph.io/ceph-ci/ceph@sha256:f4452def3be4d350f4a8e5feaf7ebb1511be9298fe68478ba7a2d6fbd7d69a63 2024-04-12T21:06:43.005 INFO:journalctl@ceph.osd.5.smithi186.stdout:Apr 12 21:06:42 smithi186 podman[105765]: 2024-04-12 21:06:42.738994233 +0000 UTC m=+0.160776729 container create 87900d20838459155fd096d2634a7388b54c041be478d6d182dd0d6fbe1e11c8 (image=quay.ceph.io/ceph-ci/ceph@sha256:f4452def3be4d350f4a8e5feaf7ebb1511be9298fe68478ba7a2d6fbd7d69a63, name=ceph-1cbe407a-f90d-11ee-bc8d-c7b262605968-osd-5-activate, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, ceph=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240409, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-vshankar-testing-20240411.061452-324baf8, org.label-schema.schema-version=1.0) 2024-04-12T21:06:43.005 INFO:journalctl@ceph.osd.5.smithi186.stdout:Apr 12 21:06:43 smithi186 podman[105765]: 2024-04-12 21:06:43.00893346 +0000 UTC m=+0.430715963 container init 87900d20838459155fd096d2634a7388b54c041be478d6d182dd0d6fbe1e11c8 (image=quay.ceph.io/ceph-ci/ceph@sha256:f4452def3be4d350f4a8e5feaf7ebb1511be9298fe68478ba7a2d6fbd7d69a63, name=ceph-1cbe407a-f90d-11ee-bc8d-c7b262605968-osd-5-activate, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, RELEASE=wip-vshankar-testing-20240411.061452-324baf8, org.label-schema.vendor=CentOS, ceph=True, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.build-date=20240409, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2) 2024-04-12T21:06:43.279 INFO:journalctl@ceph.osd.5.smithi186.stdout:Apr 12 21:06:43 smithi186 podman[105765]: 2024-04-12 21:06:43.020308463 +0000 UTC m=+0.442090953 container start 87900d20838459155fd096d2634a7388b54c041be478d6d182dd0d6fbe1e11c8 (image=quay.ceph.io/ceph-ci/ceph@sha256:f4452def3be4d350f4a8e5feaf7ebb1511be9298fe68478ba7a2d6fbd7d69a63, name=ceph-1cbe407a-f90d-11ee-bc8d-c7b262605968-osd-5-activate, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, RELEASE=wip-vshankar-testing-20240411.061452-324baf8, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.build-date=20240409) 2024-04-12T21:06:43.280 INFO:journalctl@ceph.osd.5.smithi186.stdout:Apr 12 21:06:43 smithi186 podman[105765]: 2024-04-12 21:06:43.107803393 +0000 UTC m=+0.529585867 container attach 87900d20838459155fd096d2634a7388b54c041be478d6d182dd0d6fbe1e11c8 (image=quay.ceph.io/ceph-ci/ceph@sha256:f4452def3be4d350f4a8e5feaf7ebb1511be9298fe68478ba7a2d6fbd7d69a63, name=ceph-1cbe407a-f90d-11ee-bc8d-c7b262605968-osd-5-activate, ceph=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.build-date=20240409, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-vshankar-testing-20240411.061452-324baf8, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=) 2024-04-12T21:06:43.779 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:06:43 smithi186 ceph-mon[86379]: pgmap v200: 65 pgs: 11 active+undersized, 8 active+undersized+degraded, 3 peering, 3 stale+active+clean, 40 active+clean; 257 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 23/324 objects degraded (7.099%) 2024-04-12T21:06:43.850 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:06:43 smithi018 ceph-mon[112590]: pgmap v200: 65 pgs: 11 active+undersized, 8 active+undersized+degraded, 3 peering, 3 stale+active+clean, 40 active+clean; 257 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 23/324 objects degraded (7.099%) 2024-04-12T21:06:44.190 INFO:journalctl@ceph.osd.5.smithi186.stdout:Apr 12 21:06:43 smithi186 ceph-1cbe407a-f90d-11ee-bc8d-c7b262605968-osd-5-activate[105780]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2024-04-12T21:06:44.190 INFO:journalctl@ceph.osd.5.smithi186.stdout:Apr 12 21:06:43 smithi186 ceph-1cbe407a-f90d-11ee-bc8d-c7b262605968-osd-5-activate[105780]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-5 --no-mon-config --dev /dev/mapper/vg_nvme-lv_2 2024-04-12T21:06:44.191 INFO:journalctl@ceph.osd.5.smithi186.stdout:Apr 12 21:06:43 smithi186 bash[105765]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2024-04-12T21:06:44.191 INFO:journalctl@ceph.osd.5.smithi186.stdout:Apr 12 21:06:43 smithi186 bash[105765]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-5 --no-mon-config --dev /dev/mapper/vg_nvme-lv_2 2024-04-12T21:06:44.191 INFO:journalctl@ceph.osd.5.smithi186.stdout:Apr 12 21:06:43 smithi186 bash[105765]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_2 2024-04-12T21:06:44.191 INFO:journalctl@ceph.osd.5.smithi186.stdout:Apr 12 21:06:43 smithi186 bash[105765]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-1 2024-04-12T21:06:44.191 INFO:journalctl@ceph.osd.5.smithi186.stdout:Apr 12 21:06:43 smithi186 bash[105765]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5/block 2024-04-12T21:06:44.191 INFO:journalctl@ceph.osd.5.smithi186.stdout:Apr 12 21:06:43 smithi186 bash[105765]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2024-04-12T21:06:44.191 INFO:journalctl@ceph.osd.5.smithi186.stdout:Apr 12 21:06:43 smithi186 bash[105765]: --> ceph-volume raw activate successful for osd ID: 5 2024-04-12T21:06:44.192 INFO:journalctl@ceph.osd.5.smithi186.stdout:Apr 12 21:06:43 smithi186 ceph-1cbe407a-f90d-11ee-bc8d-c7b262605968-osd-5-activate[105780]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_2 2024-04-12T21:06:44.192 INFO:journalctl@ceph.osd.5.smithi186.stdout:Apr 12 21:06:43 smithi186 ceph-1cbe407a-f90d-11ee-bc8d-c7b262605968-osd-5-activate[105780]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-1 2024-04-12T21:06:44.192 INFO:journalctl@ceph.osd.5.smithi186.stdout:Apr 12 21:06:43 smithi186 ceph-1cbe407a-f90d-11ee-bc8d-c7b262605968-osd-5-activate[105780]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5/block 2024-04-12T21:06:44.192 INFO:journalctl@ceph.osd.5.smithi186.stdout:Apr 12 21:06:43 smithi186 ceph-1cbe407a-f90d-11ee-bc8d-c7b262605968-osd-5-activate[105780]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2024-04-12T21:06:44.192 INFO:journalctl@ceph.osd.5.smithi186.stdout:Apr 12 21:06:43 smithi186 ceph-1cbe407a-f90d-11ee-bc8d-c7b262605968-osd-5-activate[105780]: --> ceph-volume raw activate successful for osd ID: 5 2024-04-12T21:06:44.192 INFO:journalctl@ceph.osd.5.smithi186.stdout:Apr 12 21:06:43 smithi186 podman[105765]: 2024-04-12 21:06:43.90120003 +0000 UTC m=+1.322982539 container died 87900d20838459155fd096d2634a7388b54c041be478d6d182dd0d6fbe1e11c8 (image=quay.ceph.io/ceph-ci/ceph@sha256:f4452def3be4d350f4a8e5feaf7ebb1511be9298fe68478ba7a2d6fbd7d69a63, name=ceph-1cbe407a-f90d-11ee-bc8d-c7b262605968-osd-5-activate, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-vshankar-testing-20240411.061452-324baf8, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.build-date=20240409, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5) 2024-04-12T21:06:44.897 INFO:journalctl@ceph.osd.5.smithi186.stdout:Apr 12 21:06:44 smithi186 podman[105899]: 2024-04-12 21:06:44.544551724 +0000 UTC m=+0.622634474 container remove 87900d20838459155fd096d2634a7388b54c041be478d6d182dd0d6fbe1e11c8 (image=quay.ceph.io/ceph-ci/ceph@sha256:f4452def3be4d350f4a8e5feaf7ebb1511be9298fe68478ba7a2d6fbd7d69a63, name=ceph-1cbe407a-f90d-11ee-bc8d-c7b262605968-osd-5-activate, GIT_CLEAN=True, io.buildah.version=1.33.5, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, RELEASE=wip-vshankar-testing-20240411.061452-324baf8, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240409) 2024-04-12T21:06:45.151 INFO:journalctl@ceph.osd.5.smithi186.stdout:Apr 12 21:06:44 smithi186 podman[105961]: 2024-04-12 21:06:44.805951692 +0000 UTC m=+0.034816788 image pull 8717720dc5e5c56303818ea4d4f0a11f6e2c03dee521fb668f411f1304e0afca quay.ceph.io/ceph-ci/ceph@sha256:f4452def3be4d350f4a8e5feaf7ebb1511be9298fe68478ba7a2d6fbd7d69a63 2024-04-12T21:06:45.151 INFO:journalctl@ceph.osd.5.smithi186.stdout:Apr 12 21:06:44 smithi186 podman[105961]: 2024-04-12 21:06:44.90366998 +0000 UTC m=+0.132535104 container create 2513bea2610e64f5e1b138f0de68b834e0764f2534134d402ed08f2ada5b32b6 (image=quay.ceph.io/ceph-ci/ceph@sha256:f4452def3be4d350f4a8e5feaf7ebb1511be9298fe68478ba7a2d6fbd7d69a63, name=ceph-1cbe407a-f90d-11ee-bc8d-c7b262605968-osd-5, CEPH_POINT_RELEASE=, RELEASE=wip-vshankar-testing-20240411.061452-324baf8, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240409, maintainer=Guillaume Abrioux ) 2024-04-12T21:06:45.151 INFO:journalctl@ceph.osd.5.smithi186.stdout:Apr 12 21:06:45 smithi186 podman[105961]: 2024-04-12 21:06:45.142595825 +0000 UTC m=+0.371460956 container init 2513bea2610e64f5e1b138f0de68b834e0764f2534134d402ed08f2ada5b32b6 (image=quay.ceph.io/ceph-ci/ceph@sha256:f4452def3be4d350f4a8e5feaf7ebb1511be9298fe68478ba7a2d6fbd7d69a63, name=ceph-1cbe407a-f90d-11ee-bc8d-c7b262605968-osd-5, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, ceph=True, org.label-schema.build-date=20240409, RELEASE=wip-vshankar-testing-20240411.061452-324baf8, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS) 2024-04-12T21:06:45.151 INFO:journalctl@ceph.osd.5.smithi186.stdout:Apr 12 21:06:45 smithi186 podman[105961]: 2024-04-12 21:06:45.155206366 +0000 UTC m=+0.384071460 container start 2513bea2610e64f5e1b138f0de68b834e0764f2534134d402ed08f2ada5b32b6 (image=quay.ceph.io/ceph-ci/ceph@sha256:f4452def3be4d350f4a8e5feaf7ebb1511be9298fe68478ba7a2d6fbd7d69a63, name=ceph-1cbe407a-f90d-11ee-bc8d-c7b262605968-osd-5, ceph=True, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240409, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, RELEASE=wip-vshankar-testing-20240411.061452-324baf8) 2024-04-12T21:06:45.529 INFO:journalctl@ceph.osd.5.smithi186.stdout:Apr 12 21:06:45 smithi186 ceph-osd[105980]: -- 172.21.15.186:0/2448543747 <== mon.1 v2:172.21.15.186:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55ed80ead680 con 0x55ed80e76c00 2024-04-12T21:06:45.529 INFO:journalctl@ceph.osd.5.smithi186.stdout:Apr 12 21:06:45 smithi186 bash[105961]: 2513bea2610e64f5e1b138f0de68b834e0764f2534134d402ed08f2ada5b32b6 2024-04-12T21:06:45.529 INFO:journalctl@ceph.osd.5.smithi186.stdout:Apr 12 21:06:45 smithi186 systemd[1]: Started Ceph osd.5 for 1cbe407a-f90d-11ee-bc8d-c7b262605968. 2024-04-12T21:06:45.850 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:06:45 smithi018 ceph-mon[112590]: pgmap v201: 65 pgs: 14 active+undersized, 12 active+undersized+degraded, 3 peering, 36 active+clean; 257 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 44/324 objects degraded (13.580%) 2024-04-12T21:06:45.850 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:06:45 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' 2024-04-12T21:06:45.851 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:06:45 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' 2024-04-12T21:06:45.851 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:06:45 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-12T21:06:46.029 INFO:journalctl@ceph.osd.5.smithi186.stdout:Apr 12 21:06:45 smithi186 ceph-1cbe407a-f90d-11ee-bc8d-c7b262605968-osd-5[105976]: 2024-04-12T21:06:45.732+0000 7f82204537c0 -1 Falling back to public interface 2024-04-12T21:06:46.029 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:06:45 smithi186 ceph-mon[86379]: pgmap v201: 65 pgs: 14 active+undersized, 12 active+undersized+degraded, 3 peering, 36 active+clean; 257 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 44/324 objects degraded (13.580%) 2024-04-12T21:06:46.029 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:06:45 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' 2024-04-12T21:06:46.029 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:06:45 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' 2024-04-12T21:06:46.029 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:06:45 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-12T21:06:47.529 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:06:47 smithi186 ceph-mon[86379]: pgmap v202: 65 pgs: 15 active+undersized, 14 active+undersized+degraded, 36 active+clean; 257 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 51/324 objects degraded (15.741%) 2024-04-12T21:06:47.529 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:06:47 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' 2024-04-12T21:06:47.530 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:06:47 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' 2024-04-12T21:06:47.530 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:06:47 smithi186 ceph-mon[86379]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg peering) 2024-04-12T21:06:47.600 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:06:47 smithi018 ceph-mon[112590]: pgmap v202: 65 pgs: 15 active+undersized, 14 active+undersized+degraded, 36 active+clean; 257 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 51/324 objects degraded (15.741%) 2024-04-12T21:06:47.600 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:06:47 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' 2024-04-12T21:06:47.601 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:06:47 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' 2024-04-12T21:06:47.601 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:06:47 smithi018 ceph-mon[112590]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg peering) 2024-04-12T21:06:49.529 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:06:49 smithi186 ceph-mon[86379]: pgmap v203: 65 pgs: 15 active+undersized, 14 active+undersized+degraded, 36 active+clean; 257 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 51/324 objects degraded (15.741%) 2024-04-12T21:06:49.600 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:06:49 smithi018 ceph-mon[112590]: pgmap v203: 65 pgs: 15 active+undersized, 14 active+undersized+degraded, 36 active+clean; 257 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 51/324 objects degraded (15.741%) 2024-04-12T21:06:50.600 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:06:50 smithi018 ceph-mon[112590]: Health check update: Degraded data redundancy: 51/324 objects degraded (15.741%), 14 pgs degraded (PG_DEGRADED) 2024-04-12T21:06:50.779 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:06:50 smithi186 ceph-mon[86379]: Health check update: Degraded data redundancy: 51/324 objects degraded (15.741%), 14 pgs degraded (PG_DEGRADED) 2024-04-12T21:06:51.529 INFO:journalctl@ceph.osd.5.smithi186.stdout:Apr 12 21:06:51 smithi186 ceph-1cbe407a-f90d-11ee-bc8d-c7b262605968-osd-5[105976]: 2024-04-12T21:06:51.254+0000 7f82204537c0 -1 osd.5 93 log_to_monitors true 2024-04-12T21:06:51.529 INFO:journalctl@ceph.osd.5.smithi186.stdout:Apr 12 21:06:51 smithi186 ceph-1cbe407a-f90d-11ee-bc8d-c7b262605968-osd-5[105976]: 2024-04-12T21:06:51.480+0000 7f820f800700 -1 osd.5 93 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-12T21:06:51.529 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:06:51 smithi186 ceph-mon[86379]: pgmap v204: 65 pgs: 15 active+undersized, 14 active+undersized+degraded, 36 active+clean; 257 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 51/324 objects degraded (15.741%) 2024-04-12T21:06:51.530 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:06:51 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' 2024-04-12T21:06:51.530 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:06:51 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' 2024-04-12T21:06:51.530 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:06:51 smithi186 ceph-mon[86379]: from='osd.5 [v2:172.21.15.186:6816/2849575493,v1:172.21.15.186:6817/2849575493]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-12T21:06:51.530 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:06:51 smithi186 ceph-mon[86379]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-12T21:06:51.850 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:06:51 smithi018 ceph-mon[112590]: pgmap v204: 65 pgs: 15 active+undersized, 14 active+undersized+degraded, 36 active+clean; 257 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 51/324 objects degraded (15.741%) 2024-04-12T21:06:51.850 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:06:51 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' 2024-04-12T21:06:51.851 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:06:51 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' 2024-04-12T21:06:51.851 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:06:51 smithi018 ceph-mon[112590]: from='osd.5 [v2:172.21.15.186:6816/2849575493,v1:172.21.15.186:6817/2849575493]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-12T21:06:51.851 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:06:51 smithi018 ceph-mon[112590]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-12T21:06:52.004 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:52.007+0000 7f8ca2000700 1 -- 172.21.15.18:0/1105822501 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8c9c105020 msgr2=0x7f8c9c1053f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:06:52.004 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:52.007+0000 7f8ca2000700 1 --2- 172.21.15.18:0/1105822501 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8c9c105020 0x7f8c9c1053f0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f8c8c004940 tx=0x7f8c8c009e30 comp rx=0 tx=0).stop 2024-04-12T21:06:52.004 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:52.007+0000 7f8ca2000700 1 -- 172.21.15.18:0/1105822501 shutdown_connections 2024-04-12T21:06:52.004 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:52.007+0000 7f8ca2000700 1 --2- 172.21.15.18:0/1105822501 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8c9c1059c0 0x7f8c9c0ff080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:06:52.005 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:52.007+0000 7f8ca2000700 1 --2- 172.21.15.18:0/1105822501 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8c9c105020 0x7f8c9c1053f0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:06:52.005 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:52.007+0000 7f8ca2000700 1 -- 172.21.15.18:0/1105822501 >> 172.21.15.18:0/1105822501 conn(0x7f8c9c0faba0 msgr2=0x7f8c9c0fcfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:06:52.005 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:52.007+0000 7f8ca2000700 1 -- 172.21.15.18:0/1105822501 shutdown_connections 2024-04-12T21:06:52.005 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:52.007+0000 7f8ca2000700 1 -- 172.21.15.18:0/1105822501 wait complete. 2024-04-12T21:06:52.005 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:52.008+0000 7f8ca2000700 1 Processor -- start 2024-04-12T21:06:52.005 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:52.008+0000 7f8ca2000700 1 -- start start 2024-04-12T21:06:52.005 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:52.009+0000 7f8ca2000700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8c9c105020 0x7f8c9c115ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:06:52.006 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:52.009+0000 7f8ca2000700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8c9c1059c0 0x7f8c9c116020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:06:52.006 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:52.009+0000 7f8ca2000700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c9c116670 con 0x7f8c9c1059c0 2024-04-12T21:06:52.006 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:52.009+0000 7f8ca2000700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c9c10fc50 con 0x7f8c9c105020 2024-04-12T21:06:52.007 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:52.009+0000 7f8c93e00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8c9c1059c0 0x7f8c9c116020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:06:52.007 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:52.009+0000 7f8c93e00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8c9c1059c0 0x7f8c9c116020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:58996/0 (socket says 172.21.15.18:58996) 2024-04-12T21:06:52.007 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:52.009+0000 7f8c93e00700 1 -- 172.21.15.18:0/3966324802 learned_addr learned my addr 172.21.15.18:0/3966324802 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:06:52.007 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:52.009+0000 7f8c9b400700 1 --2- 172.21.15.18:0/3966324802 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8c9c105020 0x7f8c9c115ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:06:52.007 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:52.010+0000 7f8c93e00700 1 -- 172.21.15.18:0/3966324802 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8c9c105020 msgr2=0x7f8c9c115ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:06:52.007 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:52.010+0000 7f8c93e00700 1 --2- 172.21.15.18:0/3966324802 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8c9c105020 0x7f8c9c115ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:06:52.007 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:52.010+0000 7f8c93e00700 1 -- 172.21.15.18:0/3966324802 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c8c005040 con 0x7f8c9c1059c0 2024-04-12T21:06:52.007 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:52.010+0000 7f8c9b400700 1 --2- 172.21.15.18:0/3966324802 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8c9c105020 0x7f8c9c115ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-12T21:06:52.007 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:52.010+0000 7f8c93e00700 1 --2- 172.21.15.18:0/3966324802 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8c9c1059c0 0x7f8c9c116020 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f8c8400eb40 tx=0x7f8c8400b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:06:52.007 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:52.010+0000 7f8c93400700 1 -- 172.21.15.18:0/3966324802 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8c84007cc0 con 0x7f8c9c1059c0 2024-04-12T21:06:52.008 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:52.010+0000 7f8c93400700 1 -- 172.21.15.18:0/3966324802 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f8c8400f040 con 0x7f8c9c1059c0 2024-04-12T21:06:52.008 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:52.010+0000 7f8ca2000700 1 -- 172.21.15.18:0/3966324802 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8c9c10fed0 con 0x7f8c9c1059c0 2024-04-12T21:06:52.008 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:52.010+0000 7f8c93400700 1 -- 172.21.15.18:0/3966324802 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8c840095e0 con 0x7f8c9c1059c0 2024-04-12T21:06:52.008 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:52.010+0000 7f8ca2000700 1 -- 172.21.15.18:0/3966324802 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8c9c110410 con 0x7f8c9c1059c0 2024-04-12T21:06:52.008 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:52.012+0000 7f8ca2000700 1 -- 172.21.15.18:0/3966324802 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8c9c100770 con 0x7f8c9c1059c0 2024-04-12T21:06:52.009 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:52.013+0000 7f8c93400700 1 -- 172.21.15.18:0/3966324802 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7f8c84009740 con 0x7f8c9c1059c0 2024-04-12T21:06:52.011 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:52.014+0000 7f8c93400700 1 --2- 172.21.15.18:0/3966324802 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f8c8006ec00 0x7f8c800710b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:06:52.011 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:52.014+0000 7f8c93400700 1 -- 172.21.15.18:0/3966324802 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 6585+0+0 (secure 0 0 0) 0x7f8c8408e480 con 0x7f8c9c1059c0 2024-04-12T21:06:52.011 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:52.015+0000 7f8c9b400700 1 --2- 172.21.15.18:0/3966324802 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f8c8006ec00 0x7f8c800710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:06:52.012 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:52.015+0000 7f8c9b400700 1 --2- 172.21.15.18:0/3966324802 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f8c8006ec00 0x7f8c800710b0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f8c8c00b0d0 tx=0x7f8c8c009e30 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:06:52.016 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:52.020+0000 7f8c93400700 1 -- 172.21.15.18:0/3966324802 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f8c8405ad10 con 0x7f8c9c1059c0 2024-04-12T21:06:52.265 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:52.268+0000 7f8ca2000700 1 -- 172.21.15.18:0/3966324802 --> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f8c9c10c190 con 0x7f8c8006ec00 2024-04-12T21:06:52.269 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:52.271+0000 7f8c93400700 1 -- 172.21.15.18:0/3966324802 <== mgr.34102 v2:172.21.15.18:6800/3812891919 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f8c9c10c190 con 0x7f8c8006ec00 2024-04-12T21:06:52.272 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:52.274+0000 7f8ca2000700 1 -- 172.21.15.18:0/3966324802 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f8c8006ec00 msgr2=0x7f8c800710b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:06:52.272 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:52.274+0000 7f8ca2000700 1 --2- 172.21.15.18:0/3966324802 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f8c8006ec00 0x7f8c800710b0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f8c8c00b0d0 tx=0x7f8c8c009e30 comp rx=0 tx=0).stop 2024-04-12T21:06:52.272 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:52.274+0000 7f8ca2000700 1 -- 172.21.15.18:0/3966324802 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8c9c1059c0 msgr2=0x7f8c9c116020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:06:52.272 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:52.274+0000 7f8ca2000700 1 --2- 172.21.15.18:0/3966324802 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8c9c1059c0 0x7f8c9c116020 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f8c8400eb40 tx=0x7f8c8400b040 comp rx=0 tx=0).stop 2024-04-12T21:06:52.272 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:52.274+0000 7f8ca2000700 1 -- 172.21.15.18:0/3966324802 shutdown_connections 2024-04-12T21:06:52.272 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:52.274+0000 7f8ca2000700 1 --2- 172.21.15.18:0/3966324802 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f8c8006ec00 0x7f8c800710b0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:06:52.272 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:52.274+0000 7f8ca2000700 1 --2- 172.21.15.18:0/3966324802 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8c9c105020 0x7f8c9c115ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:06:52.273 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:52.275+0000 7f8ca2000700 1 --2- 172.21.15.18:0/3966324802 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8c9c1059c0 0x7f8c9c116020 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:06:52.273 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:52.275+0000 7f8ca2000700 1 -- 172.21.15.18:0/3966324802 >> 172.21.15.18:0/3966324802 conn(0x7f8c9c0faba0 msgr2=0x7f8c9c0fba90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:06:52.273 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:52.275+0000 7f8ca2000700 1 -- 172.21.15.18:0/3966324802 shutdown_connections 2024-04-12T21:06:52.273 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:52.275+0000 7f8ca2000700 1 -- 172.21.15.18:0/3966324802 wait complete. 2024-04-12T21:06:52.288 INFO:teuthology.orchestra.run.smithi018.stdout:true 2024-04-12T21:06:52.435 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:52.437+0000 7f36d3200700 1 -- 172.21.15.18:0/2195265815 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f36d4102ca0 msgr2=0x7f36d4103070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:06:52.435 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:52.437+0000 7f36d3200700 1 --2- 172.21.15.18:0/2195265815 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f36d4102ca0 0x7f36d4103070 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f36c4004940 tx=0x7f36c4009e30 comp rx=0 tx=0).stop 2024-04-12T21:06:52.435 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:52.437+0000 7f36d3200700 1 -- 172.21.15.18:0/2195265815 shutdown_connections 2024-04-12T21:06:52.435 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:52.437+0000 7f36d3200700 1 --2- 172.21.15.18:0/2195265815 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f36d4065630 0x7f36d4065a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:06:52.435 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:52.437+0000 7f36d3200700 1 --2- 172.21.15.18:0/2195265815 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f36d4102ca0 0x7f36d4103070 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:06:52.435 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:52.437+0000 7f36d3200700 1 -- 172.21.15.18:0/2195265815 >> 172.21.15.18:0/2195265815 conn(0x7f36d4071b70 msgr2=0x7f36d4071f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:06:52.435 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:52.437+0000 7f36d3200700 1 -- 172.21.15.18:0/2195265815 shutdown_connections 2024-04-12T21:06:52.435 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:52.437+0000 7f36d3200700 1 -- 172.21.15.18:0/2195265815 wait complete. 2024-04-12T21:06:52.436 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:52.438+0000 7f36d3200700 1 Processor -- start 2024-04-12T21:06:52.436 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:52.439+0000 7f36d3200700 1 -- start start 2024-04-12T21:06:52.437 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:52.439+0000 7f36d3200700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f36d4065630 0x7f36d4115b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:06:52.437 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:52.439+0000 7f36d3200700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f36d4102ca0 0x7f36d4116060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:06:52.437 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:52.439+0000 7f36d3200700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36d41166b0 con 0x7f36d4102ca0 2024-04-12T21:06:52.437 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:52.439+0000 7f36d3200700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36d410faa0 con 0x7f36d4065630 2024-04-12T21:06:52.437 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:52.439+0000 7f36c9400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f36d4102ca0 0x7f36d4116060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:06:52.437 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:52.439+0000 7f36c9400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f36d4102ca0 0x7f36d4116060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:59014/0 (socket says 172.21.15.18:59014) 2024-04-12T21:06:52.437 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:52.439+0000 7f36c9400700 1 -- 172.21.15.18:0/1864751753 learned_addr learned my addr 172.21.15.18:0/1864751753 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:06:52.437 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:52.439+0000 7f36d1e00700 1 --2- 172.21.15.18:0/1864751753 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f36d4065630 0x7f36d4115b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:06:52.437 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:52.440+0000 7f36c9400700 1 -- 172.21.15.18:0/1864751753 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f36d4065630 msgr2=0x7f36d4115b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:06:52.437 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:52.440+0000 7f36c9400700 1 --2- 172.21.15.18:0/1864751753 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f36d4065630 0x7f36d4115b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:06:52.437 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:52.440+0000 7f36c9400700 1 -- 172.21.15.18:0/1864751753 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36c4005040 con 0x7f36d4102ca0 2024-04-12T21:06:52.438 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:52.440+0000 7f36c9400700 1 --2- 172.21.15.18:0/1864751753 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f36d4102ca0 0x7f36d4116060 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f36bc00ec90 tx=0x7f36bc00b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:06:52.438 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:52.440+0000 7f36caa00700 1 -- 172.21.15.18:0/1864751753 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f36bc007b30 con 0x7f36d4102ca0 2024-04-12T21:06:52.438 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:52.441+0000 7f36caa00700 1 -- 172.21.15.18:0/1864751753 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f36bc00f040 con 0x7f36d4102ca0 2024-04-12T21:06:52.439 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:52.441+0000 7f36caa00700 1 -- 172.21.15.18:0/1864751753 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f36bc00cc80 con 0x7f36d4102ca0 2024-04-12T21:06:52.439 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:52.441+0000 7f36d3200700 1 -- 172.21.15.18:0/1864751753 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f36d410fd80 con 0x7f36d4102ca0 2024-04-12T21:06:52.439 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:52.441+0000 7f36d3200700 1 -- 172.21.15.18:0/1864751753 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f36d4110340 con 0x7f36d4102ca0 2024-04-12T21:06:52.439 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:52.442+0000 7f36d3200700 1 -- 172.21.15.18:0/1864751753 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f36d40fe380 con 0x7f36d4102ca0 2024-04-12T21:06:52.440 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:52.443+0000 7f36caa00700 1 -- 172.21.15.18:0/1864751753 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7f36bc007c90 con 0x7f36d4102ca0 2024-04-12T21:06:52.441 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:52.444+0000 7f36caa00700 1 --2- 172.21.15.18:0/1864751753 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f36b806ec10 0x7f36b80710c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:06:52.441 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:52.444+0000 7f36caa00700 1 -- 172.21.15.18:0/1864751753 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 6585+0+0 (secure 0 0 0) 0x7f36bc08ed70 con 0x7f36d4102ca0 2024-04-12T21:06:52.441 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:52.444+0000 7f36d1e00700 1 --2- 172.21.15.18:0/1864751753 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f36b806ec10 0x7f36b80710c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:06:52.441 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:52.445+0000 7f36d1e00700 1 --2- 172.21.15.18:0/1864751753 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f36b806ec10 0x7f36b80710c0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f36c40045a0 tx=0x7f36c4009e30 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:06:52.446 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:52.449+0000 7f36caa00700 1 -- 172.21.15.18:0/1864751753 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f36bc05b5d0 con 0x7f36d4102ca0 2024-04-12T21:06:52.600 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:06:52 smithi018 ceph-mon[112590]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-04-12T21:06:52.601 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:06:52 smithi018 ceph-mon[112590]: osdmap e96: 6 total, 5 up, 6 in 2024-04-12T21:06:52.601 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:06:52 smithi018 ceph-mon[112590]: from='osd.5 [v2:172.21.15.186:6816/2849575493,v1:172.21.15.186:6817/2849575493]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi186", "root=default"]}]: dispatch 2024-04-12T21:06:52.601 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:06:52 smithi018 ceph-mon[112590]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi186", "root=default"]}]: dispatch 2024-04-12T21:06:52.712 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:52.715+0000 7f36d3200700 1 -- 172.21.15.18:0/1864751753 --> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f36d41157f0 con 0x7f36b806ec10 2024-04-12T21:06:52.714 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:52.717+0000 7f36caa00700 1 -- 172.21.15.18:0/1864751753 <== mgr.34102 v2:172.21.15.18:6800/3812891919 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f36d41157f0 con 0x7f36b806ec10 2024-04-12T21:06:52.717 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:52.720+0000 7f36d3200700 1 -- 172.21.15.18:0/1864751753 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f36b806ec10 msgr2=0x7f36b80710c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:06:52.717 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:52.720+0000 7f36d3200700 1 --2- 172.21.15.18:0/1864751753 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f36b806ec10 0x7f36b80710c0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f36c40045a0 tx=0x7f36c4009e30 comp rx=0 tx=0).stop 2024-04-12T21:06:52.717 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:52.720+0000 7f36d3200700 1 -- 172.21.15.18:0/1864751753 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f36d4102ca0 msgr2=0x7f36d4116060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:06:52.717 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:52.720+0000 7f36d3200700 1 --2- 172.21.15.18:0/1864751753 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f36d4102ca0 0x7f36d4116060 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f36bc00ec90 tx=0x7f36bc00b040 comp rx=0 tx=0).stop 2024-04-12T21:06:52.718 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:52.720+0000 7f36d3200700 1 -- 172.21.15.18:0/1864751753 shutdown_connections 2024-04-12T21:06:52.718 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:52.720+0000 7f36d3200700 1 --2- 172.21.15.18:0/1864751753 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f36b806ec10 0x7f36b80710c0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:06:52.718 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:52.720+0000 7f36d3200700 1 --2- 172.21.15.18:0/1864751753 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f36d4065630 0x7f36d4115b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:06:52.718 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:52.720+0000 7f36d3200700 1 --2- 172.21.15.18:0/1864751753 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f36d4102ca0 0x7f36d4116060 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:06:52.718 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:52.720+0000 7f36d3200700 1 -- 172.21.15.18:0/1864751753 >> 172.21.15.18:0/1864751753 conn(0x7f36d4071b70 msgr2=0x7f36d40fd100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:06:52.718 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:52.721+0000 7f36d3200700 1 -- 172.21.15.18:0/1864751753 shutdown_connections 2024-04-12T21:06:52.718 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:52.721+0000 7f36d3200700 1 -- 172.21.15.18:0/1864751753 wait complete. 2024-04-12T21:06:52.779 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:06:52 smithi186 ceph-mon[86379]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-04-12T21:06:52.779 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:06:52 smithi186 ceph-mon[86379]: osdmap e96: 6 total, 5 up, 6 in 2024-04-12T21:06:52.779 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:06:52 smithi186 ceph-mon[86379]: from='osd.5 [v2:172.21.15.186:6816/2849575493,v1:172.21.15.186:6817/2849575493]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi186", "root=default"]}]: dispatch 2024-04-12T21:06:52.779 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:06:52 smithi186 ceph-mon[86379]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi186", "root=default"]}]: dispatch 2024-04-12T21:06:52.882 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:52.885+0000 7f90bfe00700 1 -- 172.21.15.18:0/2102155020 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f90c0109c40 msgr2=0x7f90c01032b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:06:52.882 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:52.885+0000 7f90bfe00700 1 --2- 172.21.15.18:0/2102155020 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f90c0109c40 0x7f90c01032b0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f90ac004940 tx=0x7f90ac009e30 comp rx=0 tx=0).stop 2024-04-12T21:06:52.882 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:52.885+0000 7f90bfe00700 1 -- 172.21.15.18:0/2102155020 shutdown_connections 2024-04-12T21:06:52.883 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:52.885+0000 7f90bfe00700 1 --2- 172.21.15.18:0/2102155020 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f90c0109c40 0x7f90c01032b0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:06:52.883 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:52.885+0000 7f90bfe00700 1 --2- 172.21.15.18:0/2102155020 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f90c01092a0 0x7f90c0109670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:06:52.883 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:52.885+0000 7f90bfe00700 1 -- 172.21.15.18:0/2102155020 >> 172.21.15.18:0/2102155020 conn(0x7f90c0072a30 msgr2=0x7f90c0074e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:06:52.883 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:52.885+0000 7f90bfe00700 1 -- 172.21.15.18:0/2102155020 shutdown_connections 2024-04-12T21:06:52.883 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:52.885+0000 7f90bfe00700 1 -- 172.21.15.18:0/2102155020 wait complete. 2024-04-12T21:06:52.884 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:52.886+0000 7f90bfe00700 1 Processor -- start 2024-04-12T21:06:52.884 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:52.886+0000 7f90bfe00700 1 -- start start 2024-04-12T21:06:52.884 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:52.887+0000 7f90bfe00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f90c01092a0 0x7f90c0119fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:06:52.884 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:52.887+0000 7f90bfe00700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f90c0109c40 0x7f90c011a500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:06:52.884 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:52.887+0000 7f90bfe00700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90c011ab50 con 0x7f90c01092a0 2024-04-12T21:06:52.884 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:52.887+0000 7f90bfe00700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90c0113f20 con 0x7f90c0109c40 2024-04-12T21:06:52.884 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:52.887+0000 7f90bea00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f90c01092a0 0x7f90c0119fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:06:52.884 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:52.887+0000 7f90bea00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f90c01092a0 0x7f90c0119fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:59042/0 (socket says 172.21.15.18:59042) 2024-04-12T21:06:52.884 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:52.887+0000 7f90bea00700 1 -- 172.21.15.18:0/4035869043 learned_addr learned my addr 172.21.15.18:0/4035869043 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:06:52.885 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:52.888+0000 7f90be000700 1 --2- 172.21.15.18:0/4035869043 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f90c0109c40 0x7f90c011a500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:06:52.886 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:52.888+0000 7f90bea00700 1 -- 172.21.15.18:0/4035869043 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f90c0109c40 msgr2=0x7f90c011a500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:06:52.886 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:52.888+0000 7f90bea00700 1 --2- 172.21.15.18:0/4035869043 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f90c0109c40 0x7f90c011a500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:06:52.886 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:52.888+0000 7f90bea00700 1 -- 172.21.15.18:0/4035869043 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90ac005040 con 0x7f90c01092a0 2024-04-12T21:06:52.886 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:52.888+0000 7f90bea00700 1 --2- 172.21.15.18:0/4035869043 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f90c01092a0 0x7f90c0119fc0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f90b0002a50 tx=0x7f90b00046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:06:52.886 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:52.889+0000 7f90b6a00700 1 -- 172.21.15.18:0/4035869043 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f90b0007cc0 con 0x7f90c01092a0 2024-04-12T21:06:52.886 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:52.889+0000 7f90b6a00700 1 -- 172.21.15.18:0/4035869043 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f90b000f040 con 0x7f90c01092a0 2024-04-12T21:06:52.886 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:52.889+0000 7f90bfe00700 1 -- 172.21.15.18:0/4035869043 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f90c0114200 con 0x7f90c01092a0 2024-04-12T21:06:52.886 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:52.889+0000 7f90b6a00700 1 -- 172.21.15.18:0/4035869043 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f90b0009640 con 0x7f90c01092a0 2024-04-12T21:06:52.886 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:52.889+0000 7f90bfe00700 1 -- 172.21.15.18:0/4035869043 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f90c01147c0 con 0x7f90c01092a0 2024-04-12T21:06:52.887 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:52.890+0000 7f90bfe00700 1 -- 172.21.15.18:0/4035869043 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f90c01049f0 con 0x7f90c01092a0 2024-04-12T21:06:52.888 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:52.891+0000 7f90b6a00700 1 -- 172.21.15.18:0/4035869043 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7f90b00097a0 con 0x7f90c01092a0 2024-04-12T21:06:52.889 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:52.892+0000 7f90b6a00700 1 --2- 172.21.15.18:0/4035869043 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f90a406ec00 0x7f90a40710b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:06:52.889 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:52.892+0000 7f90b6a00700 1 -- 172.21.15.18:0/4035869043 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 6585+0+0 (secure 0 0 0) 0x7f90b008e780 con 0x7f90c01092a0 2024-04-12T21:06:52.889 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:52.892+0000 7f90be000700 1 --2- 172.21.15.18:0/4035869043 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f90a406ec00 0x7f90a40710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:06:52.890 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:52.893+0000 7f90be000700 1 --2- 172.21.15.18:0/4035869043 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f90a406ec00 0x7f90a40710b0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f90ac00b020 tx=0x7f90ac009e30 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:06:52.894 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:52.897+0000 7f90b6a00700 1 -- 172.21.15.18:0/4035869043 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f90b005af60 con 0x7f90c01092a0 2024-04-12T21:06:53.141 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:53.144+0000 7f90bfe00700 1 -- 172.21.15.18:0/4035869043 --> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f90c004bf40 con 0x7f90a406ec00 2024-04-12T21:06:53.170 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:53.173+0000 7f90b6a00700 1 -- 172.21.15.18:0/4035869043 <== mgr.34102 v2:172.21.15.18:6800/3812891919 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7f90c004bf40 con 0x7f90a406ec00 2024-04-12T21:06:53.171 INFO:teuthology.orchestra.run.smithi018.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-12T21:06:53.171 INFO:teuthology.orchestra.run.smithi018.stdout:alertmanager.smithi018 smithi018 *:9093,9094 running (7m) 115s ago 21m 14.9M - 0.25.0 c8568f914cd2 c332090068e8 2024-04-12T21:06:53.171 INFO:teuthology.orchestra.run.smithi018.stdout:crash.smithi018 smithi018 running (3m) 115s ago 22m 6665k - 19.0.0-2879-g324baf8d 8717720dc5e5 b7f80d1f7982 2024-04-12T21:06:53.171 INFO:teuthology.orchestra.run.smithi018.stdout:crash.smithi186 smithi186 running (3m) 2s ago 20m 6660k - 19.0.0-2879-g324baf8d 8717720dc5e5 a4f55c81b623 2024-04-12T21:06:53.171 INFO:teuthology.orchestra.run.smithi018.stdout:grafana.smithi018 smithi018 *:3000 running (6m) 115s ago 21m 74.5M - 9.4.12 2bacad6d85d8 b08e3bf176dd 2024-04-12T21:06:53.171 INFO:teuthology.orchestra.run.smithi018.stdout:mds.cephfs.smithi018.acqbsc smithi018 running (14m) 115s ago 14m 112M - 17.2.7-827-gd3b5060a 21de9b7369ad 5b5fce65f18a 2024-04-12T21:06:53.171 INFO:teuthology.orchestra.run.smithi018.stdout:mds.cephfs.smithi018.szjluo smithi018 running (14m) 115s ago 14m 113M - 17.2.7-827-gd3b5060a 21de9b7369ad cf9bf0c5031f 2024-04-12T21:06:53.171 INFO:teuthology.orchestra.run.smithi018.stdout:mds.cephfs.smithi186.jdllyn smithi186 running (14m) 2s ago 14m 15.6M - 17.2.7-827-gd3b5060a 21de9b7369ad 2b2bc46bd29a 2024-04-12T21:06:53.171 INFO:teuthology.orchestra.run.smithi018.stdout:mds.cephfs.smithi186.wjnafs smithi186 running (14m) 2s ago 14m 15.4M - 17.2.7-827-gd3b5060a 21de9b7369ad 74b115d6d6fd 2024-04-12T21:06:53.171 INFO:teuthology.orchestra.run.smithi018.stdout:mgr.smithi018.xjzwga smithi018 *:8443,9283,8765 running (10m) 115s ago 23m 505M - 19.0.0-2879-g324baf8d 8717720dc5e5 56088305eab7 2024-04-12T21:06:53.171 INFO:teuthology.orchestra.run.smithi018.stdout:mgr.smithi186.kqcfxk smithi186 *:8443,9283,8765 running (8m) 2s ago 20m 452M - 19.0.0-2879-g324baf8d 8717720dc5e5 09cd766e5e0d 2024-04-12T21:06:53.171 INFO:teuthology.orchestra.run.smithi018.stdout:mon.smithi018 smithi018 running (5m) 115s ago 23m 50.8M 2048M 19.0.0-2879-g324baf8d 8717720dc5e5 df3e201dfed6 2024-04-12T21:06:53.171 INFO:teuthology.orchestra.run.smithi018.stdout:mon.smithi186 smithi186 running (4m) 2s ago 20m 37.7M 2048M 19.0.0-2879-g324baf8d 8717720dc5e5 c77498da9b2b 2024-04-12T21:06:53.172 INFO:teuthology.orchestra.run.smithi018.stdout:node-exporter.smithi018 smithi018 *:9100 running (8m) 115s ago 22m 7818k - 1.5.0 0da6a335fe13 326c3d9c6847 2024-04-12T21:06:53.172 INFO:teuthology.orchestra.run.smithi018.stdout:node-exporter.smithi186 smithi186 *:9100 running (8m) 2s ago 20m 7826k - 1.5.0 0da6a335fe13 ec2112baeecf 2024-04-12T21:06:53.172 INFO:teuthology.orchestra.run.smithi018.stdout:osd.0 smithi018 running (3m) 115s ago 19m 117M 1559M 19.0.0-2879-g324baf8d 8717720dc5e5 60b1273ca047 2024-04-12T21:06:53.172 INFO:teuthology.orchestra.run.smithi018.stdout:osd.1 smithi018 running (2m) 115s ago 19m 88.0M 1559M 19.0.0-2879-g324baf8d 8717720dc5e5 170ce1e5fb8d 2024-04-12T21:06:53.172 INFO:teuthology.orchestra.run.smithi018.stdout:osd.2 smithi018 running (2m) 115s ago 18m 11.2M 1559M 19.0.0-2879-g324baf8d 8717720dc5e5 c2cbc6aa2839 2024-04-12T21:06:53.172 INFO:teuthology.orchestra.run.smithi018.stdout:osd.3 smithi186 running (92s) 2s ago 17m 96.6M 2581M 19.0.0-2879-g324baf8d 8717720dc5e5 0138e8262e4b 2024-04-12T21:06:53.172 INFO:teuthology.orchestra.run.smithi018.stdout:osd.4 smithi186 running (50s) 2s ago 17m 87.2M 2581M 19.0.0-2879-g324baf8d 8717720dc5e5 b20cb63b25c9 2024-04-12T21:06:53.172 INFO:teuthology.orchestra.run.smithi018.stdout:osd.5 smithi186 running (8s) 2s ago 16m 11.1M 2581M 19.0.0-2879-g324baf8d 8717720dc5e5 2513bea2610e 2024-04-12T21:06:53.172 INFO:teuthology.orchestra.run.smithi018.stdout:prometheus.smithi018 smithi018 *:9095 running (7m) 115s ago 21m 60.0M - 2.43.0 a07b618ecd1d f58841127e21 2024-04-12T21:06:53.173 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:53.176+0000 7f90bfe00700 1 -- 172.21.15.18:0/4035869043 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f90a406ec00 msgr2=0x7f90a40710b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:06:53.173 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:53.176+0000 7f90bfe00700 1 --2- 172.21.15.18:0/4035869043 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f90a406ec00 0x7f90a40710b0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f90ac00b020 tx=0x7f90ac009e30 comp rx=0 tx=0).stop 2024-04-12T21:06:53.174 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:53.176+0000 7f90bfe00700 1 -- 172.21.15.18:0/4035869043 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f90c01092a0 msgr2=0x7f90c0119fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:06:53.174 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:53.176+0000 7f90bfe00700 1 --2- 172.21.15.18:0/4035869043 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f90c01092a0 0x7f90c0119fc0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f90b0002a50 tx=0x7f90b00046b0 comp rx=0 tx=0).stop 2024-04-12T21:06:53.174 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:53.176+0000 7f90bfe00700 1 -- 172.21.15.18:0/4035869043 shutdown_connections 2024-04-12T21:06:53.174 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:53.176+0000 7f90bfe00700 1 --2- 172.21.15.18:0/4035869043 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f90a406ec00 0x7f90a40710b0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:06:53.174 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:53.176+0000 7f90bfe00700 1 --2- 172.21.15.18:0/4035869043 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f90c01092a0 0x7f90c0119fc0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:06:53.174 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:53.176+0000 7f90bfe00700 1 --2- 172.21.15.18:0/4035869043 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f90c0109c40 0x7f90c011a500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:06:53.174 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:53.176+0000 7f90bfe00700 1 -- 172.21.15.18:0/4035869043 >> 172.21.15.18:0/4035869043 conn(0x7f90c0072a30 msgr2=0x7f90c0074910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:06:53.174 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:53.176+0000 7f90bfe00700 1 -- 172.21.15.18:0/4035869043 shutdown_connections 2024-04-12T21:06:53.174 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:53.177+0000 7f90bfe00700 1 -- 172.21.15.18:0/4035869043 wait complete. 2024-04-12T21:06:53.336 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:53.338+0000 7fd5e0c00700 1 -- 172.21.15.18:0/1466975949 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd5dc109dd0 msgr2=0x7fd5dc103440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:06:53.336 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:53.338+0000 7fd5e0c00700 1 --2- 172.21.15.18:0/1466975949 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd5dc109dd0 0x7fd5dc103440 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fd5c4004940 tx=0x7fd5c4009e30 comp rx=0 tx=0).stop 2024-04-12T21:06:53.336 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:53.339+0000 7fd5e0c00700 1 -- 172.21.15.18:0/1466975949 shutdown_connections 2024-04-12T21:06:53.336 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:53.339+0000 7fd5e0c00700 1 --2- 172.21.15.18:0/1466975949 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd5dc109dd0 0x7fd5dc103440 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:06:53.336 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:53.339+0000 7fd5e0c00700 1 --2- 172.21.15.18:0/1466975949 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd5dc109430 0x7fd5dc109800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:06:53.336 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:53.339+0000 7fd5e0c00700 1 -- 172.21.15.18:0/1466975949 >> 172.21.15.18:0/1466975949 conn(0x7fd5dc072c00 msgr2=0x7fd5dc075010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:06:53.336 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:53.339+0000 7fd5e0c00700 1 -- 172.21.15.18:0/1466975949 shutdown_connections 2024-04-12T21:06:53.336 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:53.339+0000 7fd5e0c00700 1 -- 172.21.15.18:0/1466975949 wait complete. 2024-04-12T21:06:53.338 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:53.340+0000 7fd5e0c00700 1 Processor -- start 2024-04-12T21:06:53.338 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:53.340+0000 7fd5e0c00700 1 -- start start 2024-04-12T21:06:53.338 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:53.340+0000 7fd5e0c00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd5dc109430 0x7fd5dc119f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:06:53.338 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:53.340+0000 7fd5e0c00700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd5dc109dd0 0x7fd5dc11a450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:06:53.338 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:53.340+0000 7fd5e0c00700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5dc11aaa0 con 0x7fd5dc109430 2024-04-12T21:06:53.338 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:53.340+0000 7fd5e0c00700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5dc114080 con 0x7fd5dc109dd0 2024-04-12T21:06:53.338 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:53.341+0000 7fd5d9e00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd5dc109430 0x7fd5dc119f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:06:53.338 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:53.341+0000 7fd5d9e00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd5dc109430 0x7fd5dc119f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:59066/0 (socket says 172.21.15.18:59066) 2024-04-12T21:06:53.338 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:53.341+0000 7fd5d9e00700 1 -- 172.21.15.18:0/38730129 learned_addr learned my addr 172.21.15.18:0/38730129 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:06:53.338 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:53.341+0000 7fd5d3e00700 1 --2- 172.21.15.18:0/38730129 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd5dc109dd0 0x7fd5dc11a450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:06:53.338 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:53.341+0000 7fd5d9e00700 1 -- 172.21.15.18:0/38730129 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd5dc109dd0 msgr2=0x7fd5dc11a450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:06:53.339 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:53.341+0000 7fd5d9e00700 1 --2- 172.21.15.18:0/38730129 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd5dc109dd0 0x7fd5dc11a450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:06:53.339 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:53.341+0000 7fd5d9e00700 1 -- 172.21.15.18:0/38730129 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5c4005040 con 0x7fd5dc109430 2024-04-12T21:06:53.339 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:53.341+0000 7fd5d9e00700 1 --2- 172.21.15.18:0/38730129 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd5dc109430 0x7fd5dc119f10 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fd5cc00ec90 tx=0x7fd5cc00b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:06:53.339 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:53.342+0000 7fd5d2000700 1 -- 172.21.15.18:0/38730129 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd5cc007b30 con 0x7fd5dc109430 2024-04-12T21:06:53.339 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:53.342+0000 7fd5d2000700 1 -- 172.21.15.18:0/38730129 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd5cc00f040 con 0x7fd5dc109430 2024-04-12T21:06:53.340 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:53.342+0000 7fd5d2000700 1 -- 172.21.15.18:0/38730129 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd5cc00cc80 con 0x7fd5dc109430 2024-04-12T21:06:53.340 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:53.342+0000 7fd5e0c00700 1 -- 172.21.15.18:0/38730129 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd5dc114360 con 0x7fd5dc109430 2024-04-12T21:06:53.340 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:53.342+0000 7fd5e0c00700 1 -- 172.21.15.18:0/38730129 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd5dc114920 con 0x7fd5dc109430 2024-04-12T21:06:53.340 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:53.343+0000 7fd5e0c00700 1 -- 172.21.15.18:0/38730129 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd5dc104b80 con 0x7fd5dc109430 2024-04-12T21:06:53.341 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:53.344+0000 7fd5d2000700 1 -- 172.21.15.18:0/38730129 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7fd5cc007c90 con 0x7fd5dc109430 2024-04-12T21:06:53.342 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:53.345+0000 7fd5d2000700 1 --2- 172.21.15.18:0/38730129 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fd5c006ec00 0x7fd5c00710b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:06:53.343 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:53.345+0000 7fd5d2000700 1 -- 172.21.15.18:0/38730129 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 6585+0+0 (secure 0 0 0) 0x7fd5cc08ecd0 con 0x7fd5dc109430 2024-04-12T21:06:53.343 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:53.346+0000 7fd5d3e00700 1 --2- 172.21.15.18:0/38730129 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fd5c006ec00 0x7fd5c00710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:06:53.343 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:53.346+0000 7fd5d3e00700 1 --2- 172.21.15.18:0/38730129 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fd5c006ec00 0x7fd5c00710b0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fd5dc1157b0 tx=0x7fd5c4009e30 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:06:53.347 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:53.350+0000 7fd5d2000700 1 -- 172.21.15.18:0/38730129 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7fd5cc05b530 con 0x7fd5dc109430 2024-04-12T21:06:53.696 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:53.699+0000 7fd5e0c00700 1 -- 172.21.15.18:0/38730129 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fd5dc115560 con 0x7fd5dc109430 2024-04-12T21:06:53.697 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:53.700+0000 7fd5d2000700 1 -- 172.21.15.18:0/38730129 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+733 (secure 0 0 0) 0x7fd5cc05b0c0 con 0x7fd5dc109430 2024-04-12T21:06:53.697 INFO:teuthology.orchestra.run.smithi018.stdout:{ 2024-04-12T21:06:53.698 INFO:teuthology.orchestra.run.smithi018.stdout: "mon": { 2024-04-12T21:06:53.698 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 19.0.0-2879-g324baf8d (324baf8ddb91dea46cac50cb535c306115791355) squid (dev)": 2 2024-04-12T21:06:53.698 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-12T21:06:53.698 INFO:teuthology.orchestra.run.smithi018.stdout: "mgr": { 2024-04-12T21:06:53.698 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 19.0.0-2879-g324baf8d (324baf8ddb91dea46cac50cb535c306115791355) squid (dev)": 2 2024-04-12T21:06:53.698 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-12T21:06:53.698 INFO:teuthology.orchestra.run.smithi018.stdout: "osd": { 2024-04-12T21:06:53.698 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 19.0.0-2879-g324baf8d (324baf8ddb91dea46cac50cb535c306115791355) squid (dev)": 6 2024-04-12T21:06:53.698 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-12T21:06:53.698 INFO:teuthology.orchestra.run.smithi018.stdout: "mds": { 2024-04-12T21:06:53.698 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 17.2.7-827-gd3b5060a (d3b5060a17449edfae9385cdf792afec109fca5d) quincy (stable)": 4 2024-04-12T21:06:53.698 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-12T21:06:53.698 INFO:teuthology.orchestra.run.smithi018.stdout: "overall": { 2024-04-12T21:06:53.698 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 17.2.7-827-gd3b5060a (d3b5060a17449edfae9385cdf792afec109fca5d) quincy (stable)": 4, 2024-04-12T21:06:53.699 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 19.0.0-2879-g324baf8d (324baf8ddb91dea46cac50cb535c306115791355) squid (dev)": 10 2024-04-12T21:06:53.699 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-04-12T21:06:53.699 INFO:teuthology.orchestra.run.smithi018.stdout:} 2024-04-12T21:06:53.700 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:53.703+0000 7fd5e0c00700 1 -- 172.21.15.18:0/38730129 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fd5c006ec00 msgr2=0x7fd5c00710b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:06:53.700 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:53.703+0000 7fd5e0c00700 1 --2- 172.21.15.18:0/38730129 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fd5c006ec00 0x7fd5c00710b0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fd5dc1157b0 tx=0x7fd5c4009e30 comp rx=0 tx=0).stop 2024-04-12T21:06:53.700 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:53.703+0000 7fd5e0c00700 1 -- 172.21.15.18:0/38730129 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd5dc109430 msgr2=0x7fd5dc119f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:06:53.700 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:53.703+0000 7fd5e0c00700 1 --2- 172.21.15.18:0/38730129 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd5dc109430 0x7fd5dc119f10 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fd5cc00ec90 tx=0x7fd5cc00b040 comp rx=0 tx=0).stop 2024-04-12T21:06:53.700 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:53.703+0000 7fd5e0c00700 1 -- 172.21.15.18:0/38730129 shutdown_connections 2024-04-12T21:06:53.700 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:53.703+0000 7fd5e0c00700 1 --2- 172.21.15.18:0/38730129 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fd5c006ec00 0x7fd5c00710b0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:06:53.700 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:53.703+0000 7fd5e0c00700 1 --2- 172.21.15.18:0/38730129 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd5dc109430 0x7fd5dc119f10 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:06:53.700 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:53.703+0000 7fd5e0c00700 1 --2- 172.21.15.18:0/38730129 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd5dc109dd0 0x7fd5dc11a450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:06:53.700 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:53.703+0000 7fd5e0c00700 1 -- 172.21.15.18:0/38730129 >> 172.21.15.18:0/38730129 conn(0x7fd5dc072c00 msgr2=0x7fd5dc1073a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:06:53.700 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:53.703+0000 7fd5e0c00700 1 -- 172.21.15.18:0/38730129 shutdown_connections 2024-04-12T21:06:53.700 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:53.703+0000 7fd5e0c00700 1 -- 172.21.15.18:0/38730129 wait complete. 2024-04-12T21:06:53.729 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:06:53 smithi186 ceph-mon[86379]: pgmap v206: 65 pgs: 15 active+undersized, 14 active+undersized+degraded, 36 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail; 51/324 objects degraded (15.741%) 2024-04-12T21:06:53.729 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:06:53 smithi186 ceph-mon[86379]: from='client.34400 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:06:53.729 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:06:53 smithi186 ceph-mon[86379]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-04-12T21:06:53.729 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:06:53 smithi186 ceph-mon[86379]: osd.5 [v2:172.21.15.186:6816/2849575493,v1:172.21.15.186:6817/2849575493] boot 2024-04-12T21:06:53.729 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:06:53 smithi186 ceph-mon[86379]: osdmap e97: 6 total, 6 up, 6 in 2024-04-12T21:06:53.730 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:06:53 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-12T21:06:53.730 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:06:53 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' 2024-04-12T21:06:53.730 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:06:53 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' 2024-04-12T21:06:53.850 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:06:53 smithi018 ceph-mon[112590]: pgmap v206: 65 pgs: 15 active+undersized, 14 active+undersized+degraded, 36 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail; 51/324 objects degraded (15.741%) 2024-04-12T21:06:53.850 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:06:53 smithi018 ceph-mon[112590]: from='client.34400 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:06:53.850 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:06:53 smithi018 ceph-mon[112590]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-04-12T21:06:53.850 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:06:53 smithi018 ceph-mon[112590]: osd.5 [v2:172.21.15.186:6816/2849575493,v1:172.21.15.186:6817/2849575493] boot 2024-04-12T21:06:53.850 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:06:53 smithi018 ceph-mon[112590]: osdmap e97: 6 total, 6 up, 6 in 2024-04-12T21:06:53.850 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:06:53 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-12T21:06:53.850 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:06:53 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' 2024-04-12T21:06:53.850 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:06:53 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' 2024-04-12T21:06:53.856 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:53.859+0000 7f5ef0e00700 1 -- 172.21.15.18:0/399963018 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5eec109dd0 msgr2=0x7f5eec103440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:06:53.856 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:53.859+0000 7f5ef0e00700 1 --2- 172.21.15.18:0/399963018 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5eec109dd0 0x7f5eec103440 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f5ed8004940 tx=0x7f5ed8009e30 comp rx=0 tx=0).stop 2024-04-12T21:06:53.856 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:53.859+0000 7f5ef0e00700 1 -- 172.21.15.18:0/399963018 shutdown_connections 2024-04-12T21:06:53.856 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:53.859+0000 7f5ef0e00700 1 --2- 172.21.15.18:0/399963018 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5eec109dd0 0x7f5eec103440 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:06:53.856 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:53.859+0000 7f5ef0e00700 1 --2- 172.21.15.18:0/399963018 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5eec109430 0x7f5eec109800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:06:53.856 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:53.859+0000 7f5ef0e00700 1 -- 172.21.15.18:0/399963018 >> 172.21.15.18:0/399963018 conn(0x7f5eec072c00 msgr2=0x7f5eec075010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:06:53.856 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:53.859+0000 7f5ef0e00700 1 -- 172.21.15.18:0/399963018 shutdown_connections 2024-04-12T21:06:53.856 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:53.859+0000 7f5ef0e00700 1 -- 172.21.15.18:0/399963018 wait complete. 2024-04-12T21:06:53.858 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:53.860+0000 7f5ef0e00700 1 Processor -- start 2024-04-12T21:06:53.858 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:53.860+0000 7f5ef0e00700 1 -- start start 2024-04-12T21:06:53.858 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:53.860+0000 7f5ef0e00700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5eec109430 0x7f5eec119f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:06:53.858 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:53.860+0000 7f5ef0e00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5eec109dd0 0x7f5eec11a4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:06:53.858 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:53.860+0000 7f5ef0e00700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5eec11ab10 con 0x7f5eec109dd0 2024-04-12T21:06:53.858 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:53.860+0000 7f5ef0e00700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5eec1140f0 con 0x7f5eec109430 2024-04-12T21:06:53.858 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:53.861+0000 7f5ee9400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5eec109dd0 0x7f5eec11a4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:06:53.858 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:53.861+0000 7f5ee9400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5eec109dd0 0x7f5eec11a4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:59092/0 (socket says 172.21.15.18:59092) 2024-04-12T21:06:53.858 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:53.861+0000 7f5ee9400700 1 -- 172.21.15.18:0/4231166179 learned_addr learned my addr 172.21.15.18:0/4231166179 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:06:53.858 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:53.861+0000 7f5ee9e00700 1 --2- 172.21.15.18:0/4231166179 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5eec109430 0x7f5eec119f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:06:53.858 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:53.861+0000 7f5ee9400700 1 -- 172.21.15.18:0/4231166179 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5eec109430 msgr2=0x7f5eec119f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:06:53.859 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:53.861+0000 7f5ee9400700 1 --2- 172.21.15.18:0/4231166179 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5eec109430 0x7f5eec119f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:06:53.859 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:53.861+0000 7f5ee9400700 1 -- 172.21.15.18:0/4231166179 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ed8005040 con 0x7f5eec109dd0 2024-04-12T21:06:53.859 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:53.862+0000 7f5ee9400700 1 --2- 172.21.15.18:0/4231166179 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5eec109dd0 0x7f5eec11a4c0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f5ed8009100 tx=0x7f5ed8009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:06:53.859 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:53.862+0000 7f5ee2000700 1 -- 172.21.15.18:0/4231166179 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5ed800e040 con 0x7f5eec109dd0 2024-04-12T21:06:53.860 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:53.862+0000 7f5ef0e00700 1 -- 172.21.15.18:0/4231166179 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5eec114370 con 0x7f5eec109dd0 2024-04-12T21:06:53.860 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:53.862+0000 7f5ee2000700 1 -- 172.21.15.18:0/4231166179 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f5ed8013420 con 0x7f5eec109dd0 2024-04-12T21:06:53.860 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:53.862+0000 7f5ee2000700 1 -- 172.21.15.18:0/4231166179 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5ed801c630 con 0x7f5eec109dd0 2024-04-12T21:06:53.860 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:53.862+0000 7f5ef0e00700 1 -- 172.21.15.18:0/4231166179 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5eec114930 con 0x7f5eec109dd0 2024-04-12T21:06:53.860 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:53.863+0000 7f5ef0e00700 1 -- 172.21.15.18:0/4231166179 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5eec104b80 con 0x7f5eec109dd0 2024-04-12T21:06:53.861 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:53.864+0000 7f5ee2000700 1 -- 172.21.15.18:0/4231166179 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7f5ed8013590 con 0x7f5eec109dd0 2024-04-12T21:06:53.862 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:53.865+0000 7f5ee2000700 1 --2- 172.21.15.18:0/4231166179 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f5ed006ec00 0x7f5ed00710b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:06:53.862 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:53.865+0000 7f5ee2000700 1 -- 172.21.15.18:0/4231166179 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 6585+0+0 (secure 0 0 0) 0x7f5ed808faf0 con 0x7f5eec109dd0 2024-04-12T21:06:53.862 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:53.866+0000 7f5ee9e00700 1 --2- 172.21.15.18:0/4231166179 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f5ed006ec00 0x7f5ed00710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:06:53.863 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:53.866+0000 7f5ee9e00700 1 --2- 172.21.15.18:0/4231166179 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f5ed006ec00 0x7f5ed00710b0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f5edc007fd0 tx=0x7f5edc009040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:06:53.867 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:53.870+0000 7f5ee2000700 1 -- 172.21.15.18:0/4231166179 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f5ed805c3d0 con 0x7f5eec109dd0 2024-04-12T21:06:54.146 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:54.148+0000 7f5ef0e00700 1 -- 172.21.15.18:0/4231166179 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f5eec04c710 con 0x7f5eec109dd0 2024-04-12T21:06:54.146 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:54.149+0000 7f5ee2000700 1 -- 172.21.15.18:0/4231166179 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 23 v23) v1 ==== 76+0+1886 (secure 0 0 0) 0x7f5ed8058030 con 0x7f5eec109dd0 2024-04-12T21:06:54.147 INFO:teuthology.orchestra.run.smithi018.stdout:e23 2024-04-12T21:06:54.147 INFO:teuthology.orchestra.run.smithi018.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-12T21:06:54.147 INFO:teuthology.orchestra.run.smithi018.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-12T21:06:54.147 INFO:teuthology.orchestra.run.smithi018.stdout:legacy client fscid: 1 2024-04-12T21:06:54.147 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-12T21:06:54.147 INFO:teuthology.orchestra.run.smithi018.stdout:Filesystem 'cephfs' (1) 2024-04-12T21:06:54.147 INFO:teuthology.orchestra.run.smithi018.stdout:fs_name cephfs 2024-04-12T21:06:54.147 INFO:teuthology.orchestra.run.smithi018.stdout:epoch 23 2024-04-12T21:06:54.147 INFO:teuthology.orchestra.run.smithi018.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-04-12T21:06:54.148 INFO:teuthology.orchestra.run.smithi018.stdout:created 2024-04-12T20:51:57.362850+0000 2024-04-12T21:06:54.148 INFO:teuthology.orchestra.run.smithi018.stdout:modified 2024-04-12T21:01:10.670287+0000 2024-04-12T21:06:54.148 INFO:teuthology.orchestra.run.smithi018.stdout:tableserver 0 2024-04-12T21:06:54.148 INFO:teuthology.orchestra.run.smithi018.stdout:root 0 2024-04-12T21:06:54.148 INFO:teuthology.orchestra.run.smithi018.stdout:session_timeout 60 2024-04-12T21:06:54.148 INFO:teuthology.orchestra.run.smithi018.stdout:session_autoclose 300 2024-04-12T21:06:54.148 INFO:teuthology.orchestra.run.smithi018.stdout:max_file_size 1099511627776 2024-04-12T21:06:54.148 INFO:teuthology.orchestra.run.smithi018.stdout:max_xattr_size 65536 2024-04-12T21:06:54.148 INFO:teuthology.orchestra.run.smithi018.stdout:required_client_features {} 2024-04-12T21:06:54.148 INFO:teuthology.orchestra.run.smithi018.stdout:last_failure 0 2024-04-12T21:06:54.148 INFO:teuthology.orchestra.run.smithi018.stdout:last_failure_osd_epoch 46 2024-04-12T21:06:54.148 INFO:teuthology.orchestra.run.smithi018.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-12T21:06:54.148 INFO:teuthology.orchestra.run.smithi018.stdout:max_mds 2 2024-04-12T21:06:54.148 INFO:teuthology.orchestra.run.smithi018.stdout:in 0,1 2024-04-12T21:06:54.149 INFO:teuthology.orchestra.run.smithi018.stdout:up {0=14452,1=14464} 2024-04-12T21:06:54.149 INFO:teuthology.orchestra.run.smithi018.stdout:failed 2024-04-12T21:06:54.149 INFO:teuthology.orchestra.run.smithi018.stdout:damaged 2024-04-12T21:06:54.149 INFO:teuthology.orchestra.run.smithi018.stdout:stopped 2024-04-12T21:06:54.149 INFO:teuthology.orchestra.run.smithi018.stdout:data_pools [3] 2024-04-12T21:06:54.149 INFO:teuthology.orchestra.run.smithi018.stdout:metadata_pool 2 2024-04-12T21:06:54.149 INFO:teuthology.orchestra.run.smithi018.stdout:inline_data disabled 2024-04-12T21:06:54.149 INFO:teuthology.orchestra.run.smithi018.stdout:balancer 2024-04-12T21:06:54.149 INFO:teuthology.orchestra.run.smithi018.stdout:bal_rank_mask -1 2024-04-12T21:06:54.149 INFO:teuthology.orchestra.run.smithi018.stdout:standby_count_wanted 1 2024-04-12T21:06:54.149 INFO:teuthology.orchestra.run.smithi018.stdout:qdb_cluster leader: 0 members: 2024-04-12T21:06:54.150 INFO:teuthology.orchestra.run.smithi018.stdout:[mds.cephfs.smithi018.acqbsc{0:14452} state up:active seq 5 join_fscid=1 addr [v2:172.21.15.18:6826/1564065434,v1:172.21.15.18:6827/1564065434] compat {c=[1],r=[1],i=[7ff]}] 2024-04-12T21:06:54.150 INFO:teuthology.orchestra.run.smithi018.stdout:[mds.cephfs.smithi018.szjluo{1:14464} state up:active seq 6 join_fscid=1 addr [v2:172.21.15.18:6828/722443353,v1:172.21.15.18:6829/722443353] compat {c=[1],r=[1],i=[7ff]}] 2024-04-12T21:06:54.150 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-12T21:06:54.150 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-12T21:06:54.150 INFO:teuthology.orchestra.run.smithi018.stdout:Standby daemons: 2024-04-12T21:06:54.150 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-12T21:06:54.150 INFO:teuthology.orchestra.run.smithi018.stdout:[mds.cephfs.smithi186.wjnafs{-1:24277} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.186:6826/438347207,v1:172.21.15.186:6827/438347207] compat {c=[1],r=[1],i=[7ff]}] 2024-04-12T21:06:54.150 INFO:teuthology.orchestra.run.smithi018.stdout:[mds.cephfs.smithi186.jdllyn{-1:24281} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.186:6824/1464359952,v1:172.21.15.186:6825/1464359952] compat {c=[1],r=[1],i=[7ff]}] 2024-04-12T21:06:54.152 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:54.154+0000 7f5ef0e00700 1 -- 172.21.15.18:0/4231166179 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f5ed006ec00 msgr2=0x7f5ed00710b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:06:54.152 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:54.154+0000 7f5ef0e00700 1 --2- 172.21.15.18:0/4231166179 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f5ed006ec00 0x7f5ed00710b0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f5edc007fd0 tx=0x7f5edc009040 comp rx=0 tx=0).stop 2024-04-12T21:06:54.152 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:54.154+0000 7f5ef0e00700 1 -- 172.21.15.18:0/4231166179 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5eec109dd0 msgr2=0x7f5eec11a4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:06:54.152 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:54.154+0000 7f5ef0e00700 1 --2- 172.21.15.18:0/4231166179 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5eec109dd0 0x7f5eec11a4c0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f5ed8009100 tx=0x7f5ed8009e30 comp rx=0 tx=0).stop 2024-04-12T21:06:54.152 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:54.154+0000 7f5ef0e00700 1 -- 172.21.15.18:0/4231166179 shutdown_connections 2024-04-12T21:06:54.152 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:54.154+0000 7f5ef0e00700 1 --2- 172.21.15.18:0/4231166179 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f5ed006ec00 0x7f5ed00710b0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:06:54.152 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:54.154+0000 7f5ef0e00700 1 --2- 172.21.15.18:0/4231166179 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5eec109430 0x7f5eec119f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:06:54.152 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:54.154+0000 7f5ef0e00700 1 --2- 172.21.15.18:0/4231166179 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5eec109dd0 0x7f5eec11a4c0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:06:54.152 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:54.154+0000 7f5ef0e00700 1 -- 172.21.15.18:0/4231166179 >> 172.21.15.18:0/4231166179 conn(0x7f5eec072c00 msgr2=0x7f5eec074c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:06:54.152 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:54.155+0000 7f5ef0e00700 1 -- 172.21.15.18:0/4231166179 shutdown_connections 2024-04-12T21:06:54.152 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:54.155+0000 7f5ef0e00700 1 -- 172.21.15.18:0/4231166179 wait complete. 2024-04-12T21:06:54.153 INFO:teuthology.orchestra.run.smithi018.stderr:dumped fsmap epoch 23 2024-04-12T21:06:54.313 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:54.316+0000 7f55e7200700 1 -- 172.21.15.18:0/3825864366 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f55e8101ae0 msgr2=0x7f55e8065630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:06:54.313 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:54.316+0000 7f55e7200700 1 --2- 172.21.15.18:0/3825864366 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f55e8101ae0 0x7f55e8065630 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f55d8004940 tx=0x7f55d8009e30 comp rx=0 tx=0).stop 2024-04-12T21:06:54.313 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:54.316+0000 7f55e7200700 1 -- 172.21.15.18:0/3825864366 shutdown_connections 2024-04-12T21:06:54.313 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:54.316+0000 7f55e7200700 1 --2- 172.21.15.18:0/3825864366 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f55e8065b70 0x7f55e8065fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:06:54.313 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:54.316+0000 7f55e7200700 1 --2- 172.21.15.18:0/3825864366 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f55e8101ae0 0x7f55e8065630 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:06:54.314 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:54.316+0000 7f55e7200700 1 -- 172.21.15.18:0/3825864366 >> 172.21.15.18:0/3825864366 conn(0x7f55e8071b70 msgr2=0x7f55e8071f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:06:54.314 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:54.316+0000 7f55e7200700 1 -- 172.21.15.18:0/3825864366 shutdown_connections 2024-04-12T21:06:54.314 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:54.316+0000 7f55e7200700 1 -- 172.21.15.18:0/3825864366 wait complete. 2024-04-12T21:06:54.315 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:54.317+0000 7f55e7200700 1 Processor -- start 2024-04-12T21:06:54.315 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:54.317+0000 7f55e7200700 1 -- start start 2024-04-12T21:06:54.315 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:54.318+0000 7f55e7200700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f55e8065b70 0x7f55e811a020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:06:54.315 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:54.318+0000 7f55e7200700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f55e8101ae0 0x7f55e811a560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:06:54.315 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:54.318+0000 7f55e7200700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55e811abb0 con 0x7f55e8101ae0 2024-04-12T21:06:54.315 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:54.318+0000 7f55e7200700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55e8113f80 con 0x7f55e8065b70 2024-04-12T21:06:54.315 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:54.318+0000 7f55e5400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f55e8101ae0 0x7f55e811a560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:06:54.315 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:54.318+0000 7f55e5400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f55e8101ae0 0x7f55e811a560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:59104/0 (socket says 172.21.15.18:59104) 2024-04-12T21:06:54.315 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:54.318+0000 7f55e5400700 1 -- 172.21.15.18:0/1362080121 learned_addr learned my addr 172.21.15.18:0/1362080121 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:06:54.316 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:54.319+0000 7f55e5400700 1 -- 172.21.15.18:0/1362080121 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f55e8065b70 msgr2=0x7f55e811a020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:06:54.316 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:54.319+0000 7f55e5400700 1 --2- 172.21.15.18:0/1362080121 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f55e8065b70 0x7f55e811a020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:06:54.316 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:54.319+0000 7f55e5400700 1 -- 172.21.15.18:0/1362080121 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55d8005040 con 0x7f55e8101ae0 2024-04-12T21:06:54.316 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:54.319+0000 7f55e5400700 1 --2- 172.21.15.18:0/1362080121 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f55e8101ae0 0x7f55e811a560 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f55d4002a50 tx=0x7f55d40046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:06:54.317 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:54.319+0000 7f55de000700 1 -- 172.21.15.18:0/1362080121 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f55d4007cc0 con 0x7f55e8101ae0 2024-04-12T21:06:54.317 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:54.319+0000 7f55de000700 1 -- 172.21.15.18:0/1362080121 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f55d400f040 con 0x7f55e8101ae0 2024-04-12T21:06:54.317 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:54.319+0000 7f55de000700 1 -- 172.21.15.18:0/1362080121 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f55d4009640 con 0x7f55e8101ae0 2024-04-12T21:06:54.317 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:54.319+0000 7f55e7200700 1 -- 172.21.15.18:0/1362080121 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f55e8114260 con 0x7f55e8101ae0 2024-04-12T21:06:54.317 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:54.319+0000 7f55e7200700 1 -- 172.21.15.18:0/1362080121 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f55e81147a0 con 0x7f55e8101ae0 2024-04-12T21:06:54.317 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:54.321+0000 7f55e7200700 1 -- 172.21.15.18:0/1362080121 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f55e80fc170 con 0x7f55e8101ae0 2024-04-12T21:06:54.318 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:54.322+0000 7f55de000700 1 -- 172.21.15.18:0/1362080121 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7f55d40097a0 con 0x7f55e8101ae0 2024-04-12T21:06:54.319 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:54.322+0000 7f55de000700 1 --2- 172.21.15.18:0/1362080121 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f55cc06ec00 0x7f55cc0710b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:06:54.319 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:54.323+0000 7f55de000700 1 -- 172.21.15.18:0/1362080121 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 6585+0+0 (secure 0 0 0) 0x7f55d408e6e0 con 0x7f55e8101ae0 2024-04-12T21:06:54.320 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:54.323+0000 7f55e5e00700 1 --2- 172.21.15.18:0/1362080121 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f55cc06ec00 0x7f55cc0710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:06:54.320 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:54.324+0000 7f55e5e00700 1 --2- 172.21.15.18:0/1362080121 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f55cc06ec00 0x7f55cc0710b0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f55d8004f90 tx=0x7f55d8009e30 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:06:54.326 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:54.328+0000 7f55de000700 1 -- 172.21.15.18:0/1362080121 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f55d405af60 con 0x7f55e8101ae0 2024-04-12T21:06:54.530 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:06:54 smithi186 ceph-mon[86379]: from='client.34404 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:06:54.530 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:06:54 smithi186 ceph-mon[86379]: from='client.34408 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:06:54.531 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:06:54 smithi186 ceph-mon[86379]: osdmap e98: 6 total, 6 up, 6 in 2024-04-12T21:06:54.531 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:06:54 smithi186 ceph-mon[86379]: from='client.? 172.21.15.18:0/38730129' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T21:06:54.531 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:06:54 smithi186 ceph-mon[86379]: from='client.? 172.21.15.18:0/4231166179' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-12T21:06:54.571 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:54.574+0000 7f55e7200700 1 -- 172.21.15.18:0/1362080121 --> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f55e8110380 con 0x7f55cc06ec00 2024-04-12T21:06:54.572 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:54.576+0000 7f55de000700 1 -- 172.21.15.18:0/1362080121 <== mgr.34102 v2:172.21.15.18:6800/3812891919 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f55e8110380 con 0x7f55cc06ec00 2024-04-12T21:06:54.573 INFO:teuthology.orchestra.run.smithi018.stdout:{ 2024-04-12T21:06:54.573 INFO:teuthology.orchestra.run.smithi018.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:f4452def3be4d350f4a8e5feaf7ebb1511be9298fe68478ba7a2d6fbd7d69a63", 2024-04-12T21:06:54.573 INFO:teuthology.orchestra.run.smithi018.stdout: "in_progress": true, 2024-04-12T21:06:54.573 INFO:teuthology.orchestra.run.smithi018.stdout: "which": "Upgrading all daemon types on all hosts", 2024-04-12T21:06:54.573 INFO:teuthology.orchestra.run.smithi018.stdout: "services_complete": [ 2024-04-12T21:06:54.573 INFO:teuthology.orchestra.run.smithi018.stdout: "crash", 2024-04-12T21:06:54.573 INFO:teuthology.orchestra.run.smithi018.stdout: "osd", 2024-04-12T21:06:54.573 INFO:teuthology.orchestra.run.smithi018.stdout: "mon", 2024-04-12T21:06:54.573 INFO:teuthology.orchestra.run.smithi018.stdout: "mgr" 2024-04-12T21:06:54.573 INFO:teuthology.orchestra.run.smithi018.stdout: ], 2024-04-12T21:06:54.573 INFO:teuthology.orchestra.run.smithi018.stdout: "progress": "12/21 daemons upgraded", 2024-04-12T21:06:54.574 INFO:teuthology.orchestra.run.smithi018.stdout: "message": "Currently upgrading osd daemons", 2024-04-12T21:06:54.574 INFO:teuthology.orchestra.run.smithi018.stdout: "is_paused": false 2024-04-12T21:06:54.574 INFO:teuthology.orchestra.run.smithi018.stdout:} 2024-04-12T21:06:54.575 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:54.578+0000 7f55e7200700 1 -- 172.21.15.18:0/1362080121 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f55cc06ec00 msgr2=0x7f55cc0710b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:06:54.575 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:54.578+0000 7f55e7200700 1 --2- 172.21.15.18:0/1362080121 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f55cc06ec00 0x7f55cc0710b0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f55d8004f90 tx=0x7f55d8009e30 comp rx=0 tx=0).stop 2024-04-12T21:06:54.576 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:54.578+0000 7f55e7200700 1 -- 172.21.15.18:0/1362080121 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f55e8101ae0 msgr2=0x7f55e811a560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:06:54.576 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:54.578+0000 7f55e7200700 1 --2- 172.21.15.18:0/1362080121 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f55e8101ae0 0x7f55e811a560 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f55d4002a50 tx=0x7f55d40046b0 comp rx=0 tx=0).stop 2024-04-12T21:06:54.576 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:54.578+0000 7f55e7200700 1 -- 172.21.15.18:0/1362080121 shutdown_connections 2024-04-12T21:06:54.576 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:54.578+0000 7f55e7200700 1 --2- 172.21.15.18:0/1362080121 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f55cc06ec00 0x7f55cc0710b0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:06:54.576 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:54.578+0000 7f55e7200700 1 --2- 172.21.15.18:0/1362080121 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f55e8065b70 0x7f55e811a020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:06:54.576 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:54.578+0000 7f55e7200700 1 --2- 172.21.15.18:0/1362080121 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f55e8101ae0 0x7f55e811a560 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:06:54.576 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:54.578+0000 7f55e7200700 1 -- 172.21.15.18:0/1362080121 >> 172.21.15.18:0/1362080121 conn(0x7f55e8071b70 msgr2=0x7f55e80f9b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:06:54.576 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:54.579+0000 7f55e7200700 1 -- 172.21.15.18:0/1362080121 shutdown_connections 2024-04-12T21:06:54.576 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:54.579+0000 7f55e7200700 1 -- 172.21.15.18:0/1362080121 wait complete. 2024-04-12T21:06:54.733 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:54.736+0000 7f3022600700 1 -- 172.21.15.18:0/2903295061 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f301c0fe830 msgr2=0x7f301c106f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:06:54.733 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:54.736+0000 7f3022600700 1 --2- 172.21.15.18:0/2903295061 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f301c0fe830 0x7f301c106f00 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f3008004940 tx=0x7f3008009e30 comp rx=0 tx=0).stop 2024-04-12T21:06:54.734 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:54.736+0000 7f3022600700 1 -- 172.21.15.18:0/2903295061 shutdown_connections 2024-04-12T21:06:54.734 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:54.736+0000 7f3022600700 1 --2- 172.21.15.18:0/2903295061 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f301c0fe830 0x7f301c106f00 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:06:54.734 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:54.736+0000 7f3022600700 1 --2- 172.21.15.18:0/2903295061 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f301c0fde90 0x7f301c0fe260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:06:54.734 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:54.736+0000 7f3022600700 1 -- 172.21.15.18:0/2903295061 >> 172.21.15.18:0/2903295061 conn(0x7f301c072bc0 msgr2=0x7f301c072fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:06:54.734 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:06:54 smithi018 ceph-mon[112590]: from='client.34404 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:06:54.734 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:06:54 smithi018 ceph-mon[112590]: from='client.34408 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:06:54.734 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:06:54 smithi018 ceph-mon[112590]: osdmap e98: 6 total, 6 up, 6 in 2024-04-12T21:06:54.734 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:06:54 smithi018 ceph-mon[112590]: from='client.? 172.21.15.18:0/38730129' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T21:06:54.734 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:06:54 smithi018 ceph-mon[112590]: from='client.? 172.21.15.18:0/4231166179' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-12T21:06:54.736 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:54.737+0000 7f3022600700 1 -- 172.21.15.18:0/2903295061 shutdown_connections 2024-04-12T21:06:54.736 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:54.737+0000 7f3022600700 1 -- 172.21.15.18:0/2903295061 wait complete. 2024-04-12T21:06:54.736 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:54.738+0000 7f3022600700 1 Processor -- start 2024-04-12T21:06:54.736 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:54.738+0000 7f3022600700 1 -- start start 2024-04-12T21:06:54.736 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:54.738+0000 7f3022600700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f301c0fde90 0x7f301c0fbd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:06:54.736 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:54.738+0000 7f3022600700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f301c0fe830 0x7f301c0fc290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:06:54.736 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:54.738+0000 7f3022600700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f301c0fc820 con 0x7f301c0fde90 2024-04-12T21:06:54.737 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:54.738+0000 7f3022600700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f301c0fc960 con 0x7f301c0fe830 2024-04-12T21:06:54.737 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:54.739+0000 7f301b400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f301c0fde90 0x7f301c0fbd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:06:54.737 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:54.739+0000 7f301b400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f301c0fde90 0x7f301c0fbd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:59120/0 (socket says 172.21.15.18:59120) 2024-04-12T21:06:54.737 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:54.739+0000 7f301b400700 1 -- 172.21.15.18:0/951471861 learned_addr learned my addr 172.21.15.18:0/951471861 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:06:54.737 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:54.739+0000 7f301aa00700 1 --2- 172.21.15.18:0/951471861 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f301c0fe830 0x7f301c0fc290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:06:54.737 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:54.739+0000 7f301b400700 1 -- 172.21.15.18:0/951471861 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f301c0fe830 msgr2=0x7f301c0fc290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:06:54.737 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:54.739+0000 7f301b400700 1 --2- 172.21.15.18:0/951471861 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f301c0fe830 0x7f301c0fc290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:06:54.737 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:54.739+0000 7f301b400700 1 -- 172.21.15.18:0/951471861 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3008005040 con 0x7f301c0fde90 2024-04-12T21:06:54.738 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:54.740+0000 7f301b400700 1 --2- 172.21.15.18:0/951471861 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f301c0fde90 0x7f301c0fbd50 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f300c002a50 tx=0x7f300c0046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:06:54.738 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:54.740+0000 7f3013400700 1 -- 172.21.15.18:0/951471861 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f300c007cc0 con 0x7f301c0fde90 2024-04-12T21:06:54.738 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:54.740+0000 7f3013400700 1 -- 172.21.15.18:0/951471861 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f300c00f040 con 0x7f301c0fde90 2024-04-12T21:06:54.738 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:54.740+0000 7f3022600700 1 -- 172.21.15.18:0/951471861 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f301c119ea0 con 0x7f301c0fde90 2024-04-12T21:06:54.738 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:54.740+0000 7f3013400700 1 -- 172.21.15.18:0/951471861 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f300c009640 con 0x7f301c0fde90 2024-04-12T21:06:54.738 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:54.740+0000 7f3022600700 1 -- 172.21.15.18:0/951471861 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f301c11a2a0 con 0x7f301c0fde90 2024-04-12T21:06:54.739 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:54.742+0000 7f3022600700 1 -- 172.21.15.18:0/951471861 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f301c065e30 con 0x7f301c0fde90 2024-04-12T21:06:54.739 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:54.743+0000 7f3013400700 1 -- 172.21.15.18:0/951471861 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7f300c0097a0 con 0x7f301c0fde90 2024-04-12T21:06:54.740 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:54.744+0000 7f3013400700 1 --2- 172.21.15.18:0/951471861 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f300006ec10 0x7f30000710c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:06:54.740 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:54.744+0000 7f3013400700 1 -- 172.21.15.18:0/951471861 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 6585+0+0 (secure 0 0 0) 0x7f300c08e3b0 con 0x7f301c0fde90 2024-04-12T21:06:54.741 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:54.744+0000 7f301aa00700 1 --2- 172.21.15.18:0/951471861 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f300006ec10 0x7f30000710c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:06:54.741 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:54.745+0000 7f301aa00700 1 --2- 172.21.15.18:0/951471861 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f300006ec10 0x7f30000710c0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f301c0fd690 tx=0x7f3008009e30 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:06:54.748 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:54.752+0000 7f3013400700 1 -- 172.21.15.18:0/951471861 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f300c057180 con 0x7f301c0fde90 2024-04-12T21:06:55.071 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:55.073+0000 7f3022600700 1 -- 172.21.15.18:0/951471861 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f301c05f5f0 con 0x7f301c0fde90 2024-04-12T21:06:55.071 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:55.074+0000 7f3013400700 1 -- 172.21.15.18:0/951471861 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+1201 (secure 0 0 0) 0x7f300c05a840 con 0x7f301c0fde90 2024-04-12T21:06:55.072 INFO:teuthology.orchestra.run.smithi018.stdout:HEALTH_WARN all OSDs are running squid or later but require_osd_release < squid; Degraded data redundancy: 51/324 objects degraded (15.741%), 14 pgs degraded 2024-04-12T21:06:55.072 INFO:teuthology.orchestra.run.smithi018.stdout:[WRN] OSD_UPGRADE_FINISHED: all OSDs are running squid or later but require_osd_release < squid 2024-04-12T21:06:55.072 INFO:teuthology.orchestra.run.smithi018.stdout: all OSDs are running squid or later but require_osd_release < squid 2024-04-12T21:06:55.072 INFO:teuthology.orchestra.run.smithi018.stdout:[WRN] PG_DEGRADED: Degraded data redundancy: 51/324 objects degraded (15.741%), 14 pgs degraded 2024-04-12T21:06:55.072 INFO:teuthology.orchestra.run.smithi018.stdout: pg 2.2 is active+undersized+degraded, acting [1,0] 2024-04-12T21:06:55.072 INFO:teuthology.orchestra.run.smithi018.stdout: pg 2.3 is active+undersized+degraded, acting [2,1] 2024-04-12T21:06:55.072 INFO:teuthology.orchestra.run.smithi018.stdout: pg 2.8 is active+undersized+degraded, acting [2,0] 2024-04-12T21:06:55.072 INFO:teuthology.orchestra.run.smithi018.stdout: pg 2.b is active+undersized+degraded, acting [3,4] 2024-04-12T21:06:55.072 INFO:teuthology.orchestra.run.smithi018.stdout: pg 2.f is active+undersized+degraded, acting [4,0] 2024-04-12T21:06:55.072 INFO:teuthology.orchestra.run.smithi018.stdout: pg 2.14 is active+undersized+degraded, acting [3,4] 2024-04-12T21:06:55.072 INFO:teuthology.orchestra.run.smithi018.stdout: pg 2.16 is active+undersized+degraded, acting [3,2] 2024-04-12T21:06:55.072 INFO:teuthology.orchestra.run.smithi018.stdout: pg 2.17 is active+undersized+degraded, acting [2,1] 2024-04-12T21:06:55.073 INFO:teuthology.orchestra.run.smithi018.stdout: pg 2.18 is active+undersized+degraded, acting [4,3] 2024-04-12T21:06:55.073 INFO:teuthology.orchestra.run.smithi018.stdout: pg 2.1a is active+undersized+degraded, acting [3,4] 2024-04-12T21:06:55.073 INFO:teuthology.orchestra.run.smithi018.stdout: pg 2.1b is active+undersized+degraded, acting [1,0] 2024-04-12T21:06:55.073 INFO:teuthology.orchestra.run.smithi018.stdout: pg 2.1c is active+undersized+degraded, acting [4,2] 2024-04-12T21:06:55.073 INFO:teuthology.orchestra.run.smithi018.stdout: pg 2.1d is active+undersized+degraded, acting [2,0] 2024-04-12T21:06:55.073 INFO:teuthology.orchestra.run.smithi018.stdout: pg 2.1e is active+undersized+degraded, acting [2,0] 2024-04-12T21:06:55.074 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:55.077+0000 7f3022600700 1 -- 172.21.15.18:0/951471861 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f300006ec10 msgr2=0x7f30000710c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:06:55.074 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:55.077+0000 7f3022600700 1 --2- 172.21.15.18:0/951471861 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f300006ec10 0x7f30000710c0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f301c0fd690 tx=0x7f3008009e30 comp rx=0 tx=0).stop 2024-04-12T21:06:55.075 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:55.077+0000 7f3022600700 1 -- 172.21.15.18:0/951471861 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f301c0fde90 msgr2=0x7f301c0fbd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:06:55.075 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:55.077+0000 7f3022600700 1 --2- 172.21.15.18:0/951471861 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f301c0fde90 0x7f301c0fbd50 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f300c002a50 tx=0x7f300c0046b0 comp rx=0 tx=0).stop 2024-04-12T21:06:55.075 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:55.077+0000 7f3022600700 1 -- 172.21.15.18:0/951471861 shutdown_connections 2024-04-12T21:06:55.075 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:55.077+0000 7f3022600700 1 --2- 172.21.15.18:0/951471861 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f300006ec10 0x7f30000710c0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:06:55.075 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:55.077+0000 7f3022600700 1 --2- 172.21.15.18:0/951471861 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f301c0fde90 0x7f301c0fbd50 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:06:55.075 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:55.077+0000 7f3022600700 1 --2- 172.21.15.18:0/951471861 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f301c0fe830 0x7f301c0fc290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:06:55.075 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:55.077+0000 7f3022600700 1 -- 172.21.15.18:0/951471861 >> 172.21.15.18:0/951471861 conn(0x7f301c072bc0 msgr2=0x7f301c1037b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:06:55.075 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:55.077+0000 7f3022600700 1 -- 172.21.15.18:0/951471861 shutdown_connections 2024-04-12T21:06:55.075 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:06:55.078+0000 7f3022600700 1 -- 172.21.15.18:0/951471861 wait complete. 2024-04-12T21:06:55.780 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:06:55 smithi186 ceph-mon[86379]: pgmap v209: 65 pgs: 15 active+undersized, 14 active+undersized+degraded, 36 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail; 51/324 objects degraded (15.741%) 2024-04-12T21:06:55.780 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:06:55 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' 2024-04-12T21:06:55.780 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:06:55 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T21:06:55.780 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:06:55 smithi186 ceph-mon[86379]: from='client.? 172.21.15.18:0/951471861' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-12T21:06:55.850 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:06:55 smithi018 ceph-mon[112590]: pgmap v209: 65 pgs: 15 active+undersized, 14 active+undersized+degraded, 36 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail; 51/324 objects degraded (15.741%) 2024-04-12T21:06:55.850 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:06:55 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' 2024-04-12T21:06:55.851 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:06:55 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T21:06:55.851 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:06:55 smithi018 ceph-mon[112590]: from='client.? 172.21.15.18:0/951471861' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-12T21:06:56.783 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:06:56 smithi186 ceph-mon[86379]: from='client.34420 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:06:56.850 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:06:56 smithi018 ceph-mon[112590]: from='client.34420 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:06:57.773 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:06:57 smithi186 ceph-mon[86379]: pgmap v210: 65 pgs: 10 active+undersized, 11 active+undersized+degraded, 44 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail; 43/324 objects degraded (13.272%) 2024-04-12T21:06:57.773 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:06:57 smithi186 ceph-mon[86379]: Health check update: Degraded data redundancy: 43/324 objects degraded (13.272%), 11 pgs degraded (PG_DEGRADED) 2024-04-12T21:06:57.850 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:06:57 smithi018 ceph-mon[112590]: pgmap v210: 65 pgs: 10 active+undersized, 11 active+undersized+degraded, 44 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail; 43/324 objects degraded (13.272%) 2024-04-12T21:06:57.851 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:06:57 smithi018 ceph-mon[112590]: Health check update: Degraded data redundancy: 43/324 objects degraded (13.272%), 11 pgs degraded (PG_DEGRADED) 2024-04-12T21:06:59.530 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:06:59 smithi186 ceph-mon[86379]: pgmap v211: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:06:59.530 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:06:59 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' 2024-04-12T21:06:59.530 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:06:59 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' 2024-04-12T21:06:59.530 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:06:59 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-12T21:06:59.531 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:06:59 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-12T21:06:59.531 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:06:59 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' 2024-04-12T21:06:59.531 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:06:59 smithi186 ceph-mon[86379]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 43/324 objects degraded (13.272%), 11 pgs degraded) 2024-04-12T21:06:59.531 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:06:59 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-12T21:06:59.531 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:06:59 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T21:06:59.531 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:06:59 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T21:06:59.531 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:06:59 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T21:06:59.531 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:06:59 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T21:06:59.531 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:06:59 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' 2024-04-12T21:06:59.531 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:06:59 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.0"}]: dispatch 2024-04-12T21:06:59.532 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:06:59 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.0"}]': finished 2024-04-12T21:06:59.532 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:06:59 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.1"}]: dispatch 2024-04-12T21:06:59.532 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:06:59 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.1"}]': finished 2024-04-12T21:06:59.532 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:06:59 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.2"}]: dispatch 2024-04-12T21:06:59.532 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:06:59 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.2"}]': finished 2024-04-12T21:06:59.532 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:06:59 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.3"}]: dispatch 2024-04-12T21:06:59.532 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:06:59 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.3"}]': finished 2024-04-12T21:06:59.532 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:06:59 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.4"}]: dispatch 2024-04-12T21:06:59.532 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:06:59 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.4"}]': finished 2024-04-12T21:06:59.532 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:06:59 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.5"}]: dispatch 2024-04-12T21:06:59.532 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:06:59 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.5"}]': finished 2024-04-12T21:06:59.532 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:06:59 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd require-osd-release", "release": "squid"}]: dispatch 2024-04-12T21:06:59.601 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:06:59 smithi018 ceph-mon[112590]: pgmap v211: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:06:59.602 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:06:59 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' 2024-04-12T21:06:59.602 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:06:59 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' 2024-04-12T21:06:59.602 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:06:59 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-12T21:06:59.602 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:06:59 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-12T21:06:59.602 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:06:59 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' 2024-04-12T21:06:59.602 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:06:59 smithi018 ceph-mon[112590]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 43/324 objects degraded (13.272%), 11 pgs degraded) 2024-04-12T21:06:59.602 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:06:59 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-12T21:06:59.602 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:06:59 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T21:06:59.602 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:06:59 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T21:06:59.602 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:06:59 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T21:06:59.602 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:06:59 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T21:06:59.602 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:06:59 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' 2024-04-12T21:06:59.602 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:06:59 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.0"}]: dispatch 2024-04-12T21:06:59.602 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:06:59 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.0"}]': finished 2024-04-12T21:06:59.603 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:06:59 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.1"}]: dispatch 2024-04-12T21:06:59.603 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:06:59 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.1"}]': finished 2024-04-12T21:06:59.603 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:06:59 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.2"}]: dispatch 2024-04-12T21:06:59.603 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:06:59 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.2"}]': finished 2024-04-12T21:06:59.603 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:06:59 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.3"}]: dispatch 2024-04-12T21:06:59.603 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:06:59 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.3"}]': finished 2024-04-12T21:06:59.603 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:06:59 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.4"}]: dispatch 2024-04-12T21:06:59.603 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:06:59 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.4"}]': finished 2024-04-12T21:06:59.603 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:06:59 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.5"}]: dispatch 2024-04-12T21:06:59.603 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:06:59 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.5"}]': finished 2024-04-12T21:06:59.603 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:06:59 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd require-osd-release", "release": "squid"}]: dispatch 2024-04-12T21:07:00.100 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:06:59 smithi018 ceph-1cbe407a-f90d-11ee-bc8d-c7b262605968-mon-smithi018[112586]: 2024-04-12T21:06:59.716+0000 7fd457400700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-12T21:07:00.530 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:07:00 smithi186 ceph-mon[86379]: Upgrade: Setting container_image for all osd 2024-04-12T21:07:00.530 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:07:00 smithi186 ceph-mon[86379]: Upgrade: Setting require_osd_release to 19 squid 2024-04-12T21:07:00.530 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:07:00 smithi186 ceph-mon[86379]: Health check cleared: OSD_UPGRADE_FINISHED (was: all OSDs are running squid or later but require_osd_release < squid) 2024-04-12T21:07:00.530 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:07:00 smithi186 ceph-mon[86379]: Cluster is now healthy 2024-04-12T21:07:00.530 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:07:00 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd='[{"prefix": "osd require-osd-release", "release": "squid"}]': finished 2024-04-12T21:07:00.530 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:07:00 smithi186 ceph-mon[86379]: osdmap e99: 6 total, 6 up, 6 in 2024-04-12T21:07:00.530 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:07:00 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-04-12T21:07:00.530 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:07:00 smithi186 ceph-mon[86379]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-04-12T21:07:00.530 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:07:00 smithi186 ceph-mon[86379]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-12T21:07:00.530 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:07:00 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-04-12T21:07:00.531 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:07:00 smithi186 ceph-mon[86379]: osdmap e100: 6 total, 6 up, 6 in 2024-04-12T21:07:00.531 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:07:00 smithi186 ceph-mon[86379]: fsmap cephfs:0/2 2 up:standby, 2 failed 2024-04-12T21:07:00.585 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:07:00 smithi018 ceph-mon[112590]: Upgrade: Setting container_image for all osd 2024-04-12T21:07:00.585 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:07:00 smithi018 ceph-mon[112590]: Upgrade: Setting require_osd_release to 19 squid 2024-04-12T21:07:00.586 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:07:00 smithi018 ceph-mon[112590]: Health check cleared: OSD_UPGRADE_FINISHED (was: all OSDs are running squid or later but require_osd_release < squid) 2024-04-12T21:07:00.586 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:07:00 smithi018 ceph-mon[112590]: Cluster is now healthy 2024-04-12T21:07:00.586 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:07:00 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd='[{"prefix": "osd require-osd-release", "release": "squid"}]': finished 2024-04-12T21:07:00.586 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:07:00 smithi018 ceph-mon[112590]: osdmap e99: 6 total, 6 up, 6 in 2024-04-12T21:07:00.586 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:07:00 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-04-12T21:07:00.586 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:07:00 smithi018 ceph-mon[112590]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-04-12T21:07:00.586 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:07:00 smithi018 ceph-mon[112590]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-12T21:07:00.586 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:07:00 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-04-12T21:07:00.586 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:07:00 smithi018 ceph-mon[112590]: osdmap e100: 6 total, 6 up, 6 in 2024-04-12T21:07:00.586 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:07:00 smithi018 ceph-mon[112590]: fsmap cephfs:0/2 2 up:standby, 2 failed 2024-04-12T21:07:01.471 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:07:01 smithi018 ceph-mon[112590]: Upgrade: failing fs cephfs for rapid multi-rank mds upgrade 2024-04-12T21:07:01.472 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:07:01 smithi018 ceph-mon[112590]: pgmap v214: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:07:01.472 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:07:01 smithi018 ceph-mon[112590]: mds.? [v2:172.21.15.18:6826/785330430,v1:172.21.15.18:6827/785330430] up:boot 2024-04-12T21:07:01.472 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:07:01 smithi018 ceph-mon[112590]: mds.? [v2:172.21.15.18:6828/2928314745,v1:172.21.15.18:6829/2928314745] up:boot 2024-04-12T21:07:01.472 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:07:01 smithi018 ceph-mon[112590]: fsmap cephfs:0/2 4 up:standby, 2 failed 2024-04-12T21:07:01.472 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:07:01 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi018.szjluo"}]: dispatch 2024-04-12T21:07:01.472 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:07:01 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi018.acqbsc"}]: dispatch 2024-04-12T21:07:01.529 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:07:01 smithi186 ceph-mon[86379]: Upgrade: failing fs cephfs for rapid multi-rank mds upgrade 2024-04-12T21:07:01.529 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:07:01 smithi186 ceph-mon[86379]: pgmap v214: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:07:01.529 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:07:01 smithi186 ceph-mon[86379]: mds.? [v2:172.21.15.18:6826/785330430,v1:172.21.15.18:6827/785330430] up:boot 2024-04-12T21:07:01.529 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:07:01 smithi186 ceph-mon[86379]: mds.? [v2:172.21.15.18:6828/2928314745,v1:172.21.15.18:6829/2928314745] up:boot 2024-04-12T21:07:01.530 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:07:01 smithi186 ceph-mon[86379]: fsmap cephfs:0/2 4 up:standby, 2 failed 2024-04-12T21:07:01.530 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:07:01 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi018.szjluo"}]: dispatch 2024-04-12T21:07:01.530 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:07:01 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi018.acqbsc"}]: dispatch 2024-04-12T21:07:03.251 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:07:03 smithi018 ceph-mon[112590]: pgmap v215: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:07:03.251 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:07:03 smithi018 ceph-mon[112590]: Upgrade: Updating mds.cephfs.smithi018.acqbsc 2024-04-12T21:07:03.251 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:07:03 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' 2024-04-12T21:07:03.251 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:07:03 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi018.acqbsc", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-12T21:07:03.251 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:07:03 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-12T21:07:03.251 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:07:03 smithi018 ceph-mon[112590]: Deploying daemon mds.cephfs.smithi018.acqbsc on smithi018 2024-04-12T21:07:03.279 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:07:03 smithi186 ceph-mon[86379]: pgmap v215: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:07:03.279 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:07:03 smithi186 ceph-mon[86379]: Upgrade: Updating mds.cephfs.smithi018.acqbsc 2024-04-12T21:07:03.279 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:07:03 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' 2024-04-12T21:07:03.279 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:07:03 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi018.acqbsc", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-12T21:07:03.279 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:07:03 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-12T21:07:03.279 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:07:03 smithi186 ceph-mon[86379]: Deploying daemon mds.cephfs.smithi018.acqbsc on smithi018 2024-04-12T21:07:05.279 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:07:05 smithi186 ceph-mon[86379]: pgmap v216: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:07:05.279 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:07:05 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' 2024-04-12T21:07:05.351 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:07:05 smithi018 ceph-mon[112590]: pgmap v216: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:07:05.351 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:07:05 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' 2024-04-12T21:07:07.279 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:07:07 smithi186 ceph-mon[86379]: pgmap v217: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:07:07.279 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:07:07 smithi186 ceph-mon[86379]: osdmap e101: 6 total, 6 up, 6 in 2024-04-12T21:07:07.279 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:07:07 smithi186 ceph-mon[86379]: fsmap cephfs:0/2 3 up:standby, 2 failed 2024-04-12T21:07:07.350 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:07:07 smithi018 ceph-mon[112590]: pgmap v217: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:07:07.350 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:07:07 smithi018 ceph-mon[112590]: osdmap e101: 6 total, 6 up, 6 in 2024-04-12T21:07:07.350 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:07:07 smithi018 ceph-mon[112590]: fsmap cephfs:0/2 3 up:standby, 2 failed 2024-04-12T21:07:09.351 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:07:09 smithi018 ceph-mon[112590]: pgmap v219: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:07:09.460 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:07:09 smithi186 ceph-mon[86379]: pgmap v219: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:07:11.029 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:07:10 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' 2024-04-12T21:07:11.029 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:07:10 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T21:07:11.029 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:07:10 smithi186 ceph-mon[86379]: pgmap v220: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:07:11.030 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:07:10 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' 2024-04-12T21:07:11.030 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:07:10 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' 2024-04-12T21:07:11.350 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:07:10 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' 2024-04-12T21:07:11.350 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:07:10 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T21:07:11.350 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:07:10 smithi018 ceph-mon[112590]: pgmap v220: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:07:11.350 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:07:10 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' 2024-04-12T21:07:11.351 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:07:10 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' 2024-04-12T21:07:12.163 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:07:11 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-12T21:07:12.163 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:07:11 smithi018 ceph-mon[112590]: mds.? [v2:172.21.15.18:6828/755179764,v1:172.21.15.18:6829/755179764] up:boot 2024-04-12T21:07:12.163 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:07:11 smithi018 ceph-mon[112590]: fsmap cephfs:0/2 4 up:standby, 2 failed 2024-04-12T21:07:12.163 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:07:11 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi018.acqbsc"}]: dispatch 2024-04-12T21:07:12.279 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:07:11 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-12T21:07:12.280 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:07:11 smithi186 ceph-mon[86379]: mds.? [v2:172.21.15.18:6828/755179764,v1:172.21.15.18:6829/755179764] up:boot 2024-04-12T21:07:12.280 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:07:11 smithi186 ceph-mon[86379]: fsmap cephfs:0/2 4 up:standby, 2 failed 2024-04-12T21:07:12.280 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:07:11 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi018.acqbsc"}]: dispatch 2024-04-12T21:07:13.279 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:07:12 smithi186 ceph-mon[86379]: pgmap v221: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:07:13.312 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:07:12 smithi018 ceph-mon[112590]: pgmap v221: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:07:15.233 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:07:14 smithi018 ceph-mon[112590]: pgmap v222: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:07:15.279 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:07:14 smithi186 ceph-mon[86379]: pgmap v222: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:07:17.279 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:07:16 smithi186 ceph-mon[86379]: pgmap v223: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:07:17.350 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:07:16 smithi018 ceph-mon[112590]: pgmap v223: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:07:19.529 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:07:19 smithi186 ceph-mon[86379]: pgmap v224: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:07:19.529 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:07:19 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' 2024-04-12T21:07:19.529 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:07:19 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' 2024-04-12T21:07:19.600 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:07:19 smithi018 ceph-mon[112590]: pgmap v224: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:07:19.600 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:07:19 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' 2024-04-12T21:07:19.600 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:07:19 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' 2024-04-12T21:07:21.636 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:07:21 smithi018 ceph-mon[112590]: pgmap v225: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:07:21.637 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:07:21 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' 2024-04-12T21:07:21.637 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:07:21 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' 2024-04-12T21:07:21.779 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:07:21 smithi186 ceph-mon[86379]: pgmap v225: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:07:21.779 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:07:21 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' 2024-04-12T21:07:21.779 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:07:21 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' 2024-04-12T21:07:23.600 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:07:23 smithi018 ceph-mon[112590]: pgmap v226: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:07:23.779 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:07:23 smithi186 ceph-mon[86379]: pgmap v226: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:07:25.261 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:07:25.265+0000 7f47b2e00700 1 -- 172.21.15.18:0/4256351892 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f47ac0fc510 msgr2=0x7f47ac109270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:07:25.262 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:07:25.265+0000 7f47b2e00700 1 --2- 172.21.15.18:0/4256351892 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f47ac0fc510 0x7f47ac109270 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f4798004940 tx=0x7f4798009e30 comp rx=0 tx=0).stop 2024-04-12T21:07:25.262 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:07:25.265+0000 7f47b2e00700 1 -- 172.21.15.18:0/4256351892 shutdown_connections 2024-04-12T21:07:25.262 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:07:25.265+0000 7f47b2e00700 1 --2- 172.21.15.18:0/4256351892 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f47ac0fc510 0x7f47ac109270 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:07:25.262 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:07:25.265+0000 7f47b2e00700 1 --2- 172.21.15.18:0/4256351892 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f47ac0fbc00 0x7f47ac0fbfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:07:25.262 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:07:25.265+0000 7f47b2e00700 1 -- 172.21.15.18:0/4256351892 >> 172.21.15.18:0/4256351892 conn(0x7f47ac0f7870 msgr2=0x7f47ac0f9c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:07:25.262 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:07:25.265+0000 7f47b2e00700 1 -- 172.21.15.18:0/4256351892 shutdown_connections 2024-04-12T21:07:25.262 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:07:25.265+0000 7f47b2e00700 1 -- 172.21.15.18:0/4256351892 wait complete. 2024-04-12T21:07:25.263 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:07:25.266+0000 7f47b2e00700 1 Processor -- start 2024-04-12T21:07:25.263 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:07:25.266+0000 7f47b2e00700 1 -- start start 2024-04-12T21:07:25.263 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:07:25.267+0000 7f47b2e00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f47ac0fbc00 0x7f47ac115b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:07:25.263 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:07:25.267+0000 7f47b2e00700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f47ac0fc510 0x7f47ac1160d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:07:25.263 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:07:25.267+0000 7f47b2e00700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47ac116720 con 0x7f47ac0fbc00 2024-04-12T21:07:25.263 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:07:25.267+0000 7f47b2e00700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47ac10fd00 con 0x7f47ac0fc510 2024-04-12T21:07:25.263 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:07:25.267+0000 7f47abe00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f47ac0fbc00 0x7f47ac115b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:07:25.264 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:07:25.267+0000 7f47abe00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f47ac0fbc00 0x7f47ac115b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:55186/0 (socket says 172.21.15.18:55186) 2024-04-12T21:07:25.264 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:07:25.267+0000 7f47abe00700 1 -- 172.21.15.18:0/898594629 learned_addr learned my addr 172.21.15.18:0/898594629 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:07:25.264 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:07:25.268+0000 7f47ab400700 1 --2- 172.21.15.18:0/898594629 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f47ac0fc510 0x7f47ac1160d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:07:25.264 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:07:25.268+0000 7f47abe00700 1 -- 172.21.15.18:0/898594629 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f47ac0fc510 msgr2=0x7f47ac1160d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:07:25.265 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:07:25.268+0000 7f47abe00700 1 --2- 172.21.15.18:0/898594629 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f47ac0fc510 0x7f47ac1160d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:07:25.265 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:07:25.268+0000 7f47abe00700 1 -- 172.21.15.18:0/898594629 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4798005040 con 0x7f47ac0fbc00 2024-04-12T21:07:25.265 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:07:25.268+0000 7f47ab400700 1 --2- 172.21.15.18:0/898594629 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f47ac0fc510 0x7f47ac1160d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-12T21:07:25.265 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:07:25.268+0000 7f47abe00700 1 --2- 172.21.15.18:0/898594629 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f47ac0fbc00 0x7f47ac115b90 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f479c00c830 tx=0x7f479c004470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:07:25.265 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:07:25.269+0000 7f47a3e00700 1 -- 172.21.15.18:0/898594629 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f479c00a760 con 0x7f47ac0fbc00 2024-04-12T21:07:25.265 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:07:25.269+0000 7f47a3e00700 1 -- 172.21.15.18:0/898594629 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f479c00ad90 con 0x7f47ac0fbc00 2024-04-12T21:07:25.265 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:07:25.269+0000 7f47a3e00700 1 -- 172.21.15.18:0/898594629 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f479c0113a0 con 0x7f47ac0fbc00 2024-04-12T21:07:25.265 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:07:25.269+0000 7f47b2e00700 1 -- 172.21.15.18:0/898594629 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f47ac10ffe0 con 0x7f47ac0fbc00 2024-04-12T21:07:25.265 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:07:25.269+0000 7f47b2e00700 1 -- 172.21.15.18:0/898594629 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f47ac1105a0 con 0x7f47ac0fbc00 2024-04-12T21:07:25.266 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:07:25.270+0000 7f47b2e00700 1 -- 172.21.15.18:0/898594629 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f47ac066ad0 con 0x7f47ac0fbc00 2024-04-12T21:07:25.267 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:07:25.271+0000 7f47a3e00700 1 -- 172.21.15.18:0/898594629 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7f479c012400 con 0x7f47ac0fbc00 2024-04-12T21:07:25.268 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:07:25.272+0000 7f47a3e00700 1 --2- 172.21.15.18:0/898594629 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f479006ec00 0x7f47900710b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:07:25.268 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:07:25.272+0000 7f47a3e00700 1 -- 172.21.15.18:0/898594629 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 6757+0+0 (secure 0 0 0) 0x7f479c08e900 con 0x7f47ac0fbc00 2024-04-12T21:07:25.268 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:07:25.273+0000 7f47ab400700 1 --2- 172.21.15.18:0/898594629 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f479006ec00 0x7f47900710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:07:25.269 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:07:25.274+0000 7f47ab400700 1 --2- 172.21.15.18:0/898594629 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f479006ec00 0x7f47900710b0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f479800b3b0 tx=0x7f4798009e30 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:07:25.274 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:07:25.279+0000 7f47a3e00700 1 -- 172.21.15.18:0/898594629 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f479c05b1f0 con 0x7f47ac0fbc00 2024-04-12T21:07:25.546 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:07:25.550+0000 7f47b2e00700 1 -- 172.21.15.18:0/898594629 --> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f47ac111280 con 0x7f479006ec00 2024-04-12T21:07:25.546 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:07:25 smithi018 ceph-mon[112590]: pgmap v227: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:07:25.546 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:07:25 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T21:07:25.548 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:07:25.552+0000 7f47a3e00700 1 -- 172.21.15.18:0/898594629 <== mgr.34102 v2:172.21.15.18:6800/3812891919 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f47ac111280 con 0x7f479006ec00 2024-04-12T21:07:25.551 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:07:25.555+0000 7f47b2e00700 1 -- 172.21.15.18:0/898594629 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f479006ec00 msgr2=0x7f47900710b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:07:25.552 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:07:25.555+0000 7f47b2e00700 1 --2- 172.21.15.18:0/898594629 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f479006ec00 0x7f47900710b0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f479800b3b0 tx=0x7f4798009e30 comp rx=0 tx=0).stop 2024-04-12T21:07:25.552 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:07:25.556+0000 7f47b2e00700 1 -- 172.21.15.18:0/898594629 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f47ac0fbc00 msgr2=0x7f47ac115b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:07:25.552 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:07:25.556+0000 7f47b2e00700 1 --2- 172.21.15.18:0/898594629 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f47ac0fbc00 0x7f47ac115b90 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f479c00c830 tx=0x7f479c004470 comp rx=0 tx=0).stop 2024-04-12T21:07:25.552 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:07:25.556+0000 7f47b2e00700 1 -- 172.21.15.18:0/898594629 shutdown_connections 2024-04-12T21:07:25.552 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:07:25.556+0000 7f47b2e00700 1 --2- 172.21.15.18:0/898594629 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f479006ec00 0x7f47900710b0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:07:25.552 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:07:25.556+0000 7f47b2e00700 1 --2- 172.21.15.18:0/898594629 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f47ac0fbc00 0x7f47ac115b90 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:07:25.552 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:07:25.556+0000 7f47b2e00700 1 --2- 172.21.15.18:0/898594629 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f47ac0fc510 0x7f47ac1160d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:07:25.552 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:07:25.556+0000 7f47b2e00700 1 -- 172.21.15.18:0/898594629 >> 172.21.15.18:0/898594629 conn(0x7f47ac0f7870 msgr2=0x7f47ac0f7d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:07:25.552 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:07:25.556+0000 7f47b2e00700 1 -- 172.21.15.18:0/898594629 shutdown_connections 2024-04-12T21:07:25.553 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:07:25.556+0000 7f47b2e00700 1 -- 172.21.15.18:0/898594629 wait complete. 2024-04-12T21:07:25.568 INFO:teuthology.orchestra.run.smithi018.stdout:true 2024-04-12T21:07:25.713 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:07:25.716+0000 7f921d200700 1 -- 172.21.15.18:0/3791185 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f92181037c0 msgr2=0x7f92180fce30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:07:25.713 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:07:25.716+0000 7f921d200700 1 --2- 172.21.15.18:0/3791185 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f92181037c0 0x7f92180fce30 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f9204004940 tx=0x7f9204009e30 comp rx=0 tx=0).stop 2024-04-12T21:07:25.713 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:07:25.716+0000 7f921d200700 1 -- 172.21.15.18:0/3791185 shutdown_connections 2024-04-12T21:07:25.713 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:07:25.716+0000 7f921d200700 1 --2- 172.21.15.18:0/3791185 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f92181037c0 0x7f92180fce30 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:07:25.713 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:07:25.716+0000 7f921d200700 1 --2- 172.21.15.18:0/3791185 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9218102e20 0x7f92181031f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:07:25.713 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:07:25.716+0000 7f921d200700 1 -- 172.21.15.18:0/3791185 >> 172.21.15.18:0/3791185 conn(0x7f92180f89d0 msgr2=0x7f92180fade0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:07:25.713 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:07:25.716+0000 7f921d200700 1 -- 172.21.15.18:0/3791185 shutdown_connections 2024-04-12T21:07:25.713 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:07:25.716+0000 7f921d200700 1 -- 172.21.15.18:0/3791185 wait complete. 2024-04-12T21:07:25.714 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:07:25.717+0000 7f921d200700 1 Processor -- start 2024-04-12T21:07:25.714 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:07:25.717+0000 7f921d200700 1 -- start start 2024-04-12T21:07:25.715 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:07:25.718+0000 7f921d200700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9218102e20 0x7f92180702c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:07:25.715 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:07:25.718+0000 7f921d200700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f92181037c0 0x7f9218070800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:07:25.715 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:07:25.718+0000 7f921d200700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9218070d90 con 0x7f92181037c0 2024-04-12T21:07:25.715 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:07:25.718+0000 7f921d200700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9218070ed0 con 0x7f9218102e20 2024-04-12T21:07:25.715 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:07:25.718+0000 7f9215400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f92181037c0 0x7f9218070800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:07:25.715 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:07:25.718+0000 7f9215400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f92181037c0 0x7f9218070800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:55212/0 (socket says 172.21.15.18:55212) 2024-04-12T21:07:25.715 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:07:25.718+0000 7f9215400700 1 -- 172.21.15.18:0/3834115958 learned_addr learned my addr 172.21.15.18:0/3834115958 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:07:25.716 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:07:25.718+0000 7f9215400700 1 -- 172.21.15.18:0/3834115958 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9218102e20 msgr2=0x7f92180702c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:07:25.716 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:07:25.718+0000 7f9215e00700 1 --2- 172.21.15.18:0/3834115958 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9218102e20 0x7f92180702c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:07:25.716 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:07:25.718+0000 7f9215400700 1 --2- 172.21.15.18:0/3834115958 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9218102e20 0x7f92180702c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:07:25.716 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:07:25.718+0000 7f9215400700 1 -- 172.21.15.18:0/3834115958 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9204005040 con 0x7f92181037c0 2024-04-12T21:07:25.716 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:07:25.719+0000 7f9215400700 1 --2- 172.21.15.18:0/3834115958 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f92181037c0 0x7f9218070800 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f9204009100 tx=0x7f9204009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:07:25.716 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:07:25.719+0000 7f920e000700 1 -- 172.21.15.18:0/3834115958 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f920400e040 con 0x7f92181037c0 2024-04-12T21:07:25.716 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:07:25.719+0000 7f920e000700 1 -- 172.21.15.18:0/3834115958 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f9204013420 con 0x7f92181037c0 2024-04-12T21:07:25.716 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:07:25.719+0000 7f920e000700 1 -- 172.21.15.18:0/3834115958 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f920401c630 con 0x7f92181037c0 2024-04-12T21:07:25.716 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:07:25.719+0000 7f921d200700 1 -- 172.21.15.18:0/3834115958 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9218071120 con 0x7f92181037c0 2024-04-12T21:07:25.717 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:07:25.719+0000 7f921d200700 1 -- 172.21.15.18:0/3834115958 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f921811a2e0 con 0x7f92181037c0 2024-04-12T21:07:25.717 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:07:25.720+0000 7f921d200700 1 -- 172.21.15.18:0/3834115958 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f92181052f0 con 0x7f92181037c0 2024-04-12T21:07:25.718 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:07:25.722+0000 7f920e000700 1 -- 172.21.15.18:0/3834115958 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7f9204013590 con 0x7f92181037c0 2024-04-12T21:07:25.719 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:07:25.723+0000 7f920e000700 1 --2- 172.21.15.18:0/3834115958 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f91fc06ec10 0x7f91fc0710c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:07:25.719 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:07:25.723+0000 7f920e000700 1 -- 172.21.15.18:0/3834115958 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 6757+0+0 (secure 0 0 0) 0x7f92040905b0 con 0x7f92181037c0 2024-04-12T21:07:25.719 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:07:25.723+0000 7f9215e00700 1 --2- 172.21.15.18:0/3834115958 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f91fc06ec10 0x7f91fc0710c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:07:25.720 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:07:25.724+0000 7f9215e00700 1 --2- 172.21.15.18:0/3834115958 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f91fc06ec10 0x7f91fc0710c0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f9208004d60 tx=0x7f920800b040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:07:25.723 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:07:25.727+0000 7f920e000700 1 -- 172.21.15.18:0/3834115958 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f9204059370 con 0x7f92181037c0 2024-04-12T21:07:25.779 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:07:25 smithi186 ceph-mon[86379]: pgmap v227: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:07:25.779 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:07:25 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T21:07:25.977 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:07:25.981+0000 7f921d200700 1 -- 172.21.15.18:0/3834115958 --> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f9218109f00 con 0x7f91fc06ec10 2024-04-12T21:07:25.979 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:07:25.983+0000 7f920e000700 1 -- 172.21.15.18:0/3834115958 <== mgr.34102 v2:172.21.15.18:6800/3812891919 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f9218109f00 con 0x7f91fc06ec10 2024-04-12T21:07:25.982 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:07:25.986+0000 7f921d200700 1 -- 172.21.15.18:0/3834115958 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f91fc06ec10 msgr2=0x7f91fc0710c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:07:25.982 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:07:25.986+0000 7f921d200700 1 --2- 172.21.15.18:0/3834115958 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f91fc06ec10 0x7f91fc0710c0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f9208004d60 tx=0x7f920800b040 comp rx=0 tx=0).stop 2024-04-12T21:07:25.982 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:07:25.986+0000 7f921d200700 1 -- 172.21.15.18:0/3834115958 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f92181037c0 msgr2=0x7f9218070800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:07:25.982 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:07:25.986+0000 7f921d200700 1 --2- 172.21.15.18:0/3834115958 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f92181037c0 0x7f9218070800 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f9204009100 tx=0x7f9204009e30 comp rx=0 tx=0).stop 2024-04-12T21:07:25.982 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:07:25.986+0000 7f921d200700 1 -- 172.21.15.18:0/3834115958 shutdown_connections 2024-04-12T21:07:25.982 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:07:25.986+0000 7f921d200700 1 --2- 172.21.15.18:0/3834115958 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f91fc06ec10 0x7f91fc0710c0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:07:25.982 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:07:25.986+0000 7f921d200700 1 --2- 172.21.15.18:0/3834115958 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9218102e20 0x7f92180702c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:07:25.982 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:07:25.986+0000 7f921d200700 1 --2- 172.21.15.18:0/3834115958 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f92181037c0 0x7f9218070800 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:07:25.982 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:07:25.986+0000 7f921d200700 1 -- 172.21.15.18:0/3834115958 >> 172.21.15.18:0/3834115958 conn(0x7f92180f89d0 msgr2=0x7f92180fac80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:07:25.983 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:07:25.986+0000 7f921d200700 1 -- 172.21.15.18:0/3834115958 shutdown_connections 2024-04-12T21:07:25.983 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:07:25.986+0000 7f921d200700 1 -- 172.21.15.18:0/3834115958 wait complete. 2024-04-12T21:07:26.133 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:07:26.137+0000 7f6ba5600700 1 -- 172.21.15.18:0/1573228945 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6ba00fba90 msgr2=0x7f6ba00fbe60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:07:26.134 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:07:26.137+0000 7f6ba5600700 1 --2- 172.21.15.18:0/1573228945 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6ba00fba90 0x7f6ba00fbe60 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f6b90004940 tx=0x7f6b90009e30 comp rx=0 tx=0).stop 2024-04-12T21:07:26.134 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:07:26.137+0000 7f6ba5600700 1 -- 172.21.15.18:0/1573228945 shutdown_connections 2024-04-12T21:07:26.134 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:07:26.137+0000 7f6ba5600700 1 --2- 172.21.15.18:0/1573228945 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6ba00fc430 0x7f6ba0100a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:07:26.134 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:07:26.137+0000 7f6ba5600700 1 --2- 172.21.15.18:0/1573228945 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6ba00fba90 0x7f6ba00fbe60 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:07:26.134 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:07:26.137+0000 7f6ba5600700 1 -- 172.21.15.18:0/1573228945 >> 172.21.15.18:0/1573228945 conn(0x7f6ba00f7710 msgr2=0x7f6ba00f9b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:07:26.134 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:07:26.137+0000 7f6ba5600700 1 -- 172.21.15.18:0/1573228945 shutdown_connections 2024-04-12T21:07:26.134 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:07:26.137+0000 7f6ba5600700 1 -- 172.21.15.18:0/1573228945 wait complete. 2024-04-12T21:07:26.135 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:07:26.138+0000 7f6ba5600700 1 Processor -- start 2024-04-12T21:07:26.135 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:07:26.139+0000 7f6ba5600700 1 -- start start 2024-04-12T21:07:26.136 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:07:26.139+0000 7f6ba5600700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6ba00fba90 0x7f6ba011a010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:07:26.136 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:07:26.139+0000 7f6ba5600700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6ba00fc430 0x7f6ba011a550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:07:26.136 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:07:26.139+0000 7f6ba5600700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ba011ac30 con 0x7f6ba00fc430 2024-04-12T21:07:26.136 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:07:26.139+0000 7f6ba5600700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ba0113f70 con 0x7f6ba00fba90 2024-04-12T21:07:26.136 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:07:26.139+0000 7f6b9f400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6ba00fc430 0x7f6ba011a550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:07:26.136 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:07:26.139+0000 7f6b9f400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6ba00fc430 0x7f6ba011a550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:55228/0 (socket says 172.21.15.18:55228) 2024-04-12T21:07:26.136 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:07:26.139+0000 7f6b9f400700 1 -- 172.21.15.18:0/2975918827 learned_addr learned my addr 172.21.15.18:0/2975918827 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:07:26.136 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:07:26.140+0000 7f6b9fe00700 1 --2- 172.21.15.18:0/2975918827 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6ba00fba90 0x7f6ba011a010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:07:26.136 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:07:26.140+0000 7f6b9f400700 1 -- 172.21.15.18:0/2975918827 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6ba00fba90 msgr2=0x7f6ba011a010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:07:26.137 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:07:26.140+0000 7f6b9f400700 1 --2- 172.21.15.18:0/2975918827 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6ba00fba90 0x7f6ba011a010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:07:26.137 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:07:26.140+0000 7f6b9f400700 1 -- 172.21.15.18:0/2975918827 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b90005040 con 0x7f6ba00fc430 2024-04-12T21:07:26.138 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:07:26.140+0000 7f6b9f400700 1 --2- 172.21.15.18:0/2975918827 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6ba00fc430 0x7f6ba011a550 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f6b8c00b790 tx=0x7f6b8c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:07:26.138 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:07:26.140+0000 7f6b97e00700 1 -- 172.21.15.18:0/2975918827 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6b8c00f770 con 0x7f6ba00fc430 2024-04-12T21:07:26.138 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:07:26.140+0000 7f6b97e00700 1 -- 172.21.15.18:0/2975918827 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f6b8c00fda0 con 0x7f6ba00fc430 2024-04-12T21:07:26.138 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:07:26.140+0000 7f6ba5600700 1 -- 172.21.15.18:0/2975918827 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6ba01141f0 con 0x7f6ba00fc430 2024-04-12T21:07:26.138 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:07:26.141+0000 7f6b97e00700 1 -- 172.21.15.18:0/2975918827 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6b8c00d310 con 0x7f6ba00fc430 2024-04-12T21:07:26.138 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:07:26.141+0000 7f6ba5600700 1 -- 172.21.15.18:0/2975918827 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6ba0114730 con 0x7f6ba00fc430 2024-04-12T21:07:26.138 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:07:26.142+0000 7f6ba5600700 1 -- 172.21.15.18:0/2975918827 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6ba00668d0 con 0x7f6ba00fc430 2024-04-12T21:07:26.139 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:07:26.143+0000 7f6b97e00700 1 -- 172.21.15.18:0/2975918827 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7f6b8c010420 con 0x7f6ba00fc430 2024-04-12T21:07:26.140 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:07:26.144+0000 7f6b97e00700 1 --2- 172.21.15.18:0/2975918827 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f6b8406ec00 0x7f6b840710b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:07:26.140 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:07:26.144+0000 7f6b97e00700 1 -- 172.21.15.18:0/2975918827 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 6757+0+0 (secure 0 0 0) 0x7f6b8c08e9e0 con 0x7f6ba00fc430 2024-04-12T21:07:26.140 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:07:26.145+0000 7f6b9fe00700 1 --2- 172.21.15.18:0/2975918827 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f6b8406ec00 0x7f6b840710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:07:26.141 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:07:26.145+0000 7f6b9fe00700 1 --2- 172.21.15.18:0/2975918827 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f6b8406ec00 0x7f6b840710b0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f6b90004f90 tx=0x7f6b90009e30 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:07:26.145 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:07:26.149+0000 7f6b97e00700 1 -- 172.21.15.18:0/2975918827 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f6b8c05b230 con 0x7f6ba00fc430 2024-04-12T21:07:26.391 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:07:26.395+0000 7f6ba5600700 1 -- 172.21.15.18:0/2975918827 --> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f6ba0110490 con 0x7f6b8406ec00 2024-04-12T21:07:26.409 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:07:26.413+0000 7f6b97e00700 1 -- 172.21.15.18:0/2975918827 <== mgr.34102 v2:172.21.15.18:6800/3812891919 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7f6ba0110490 con 0x7f6b8406ec00 2024-04-12T21:07:26.409 INFO:teuthology.orchestra.run.smithi018.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-12T21:07:26.409 INFO:teuthology.orchestra.run.smithi018.stdout:alertmanager.smithi018 smithi018 *:9093,9094 running (8m) 8s ago 22m 15.0M - 0.25.0 c8568f914cd2 c332090068e8 2024-04-12T21:07:26.409 INFO:teuthology.orchestra.run.smithi018.stdout:crash.smithi018 smithi018 running (4m) 8s ago 22m 6665k - 19.0.0-2879-g324baf8d 8717720dc5e5 b7f80d1f7982 2024-04-12T21:07:26.409 INFO:teuthology.orchestra.run.smithi018.stdout:crash.smithi186 smithi186 running (4m) 35s ago 21m 6660k - 19.0.0-2879-g324baf8d 8717720dc5e5 a4f55c81b623 2024-04-12T21:07:26.409 INFO:teuthology.orchestra.run.smithi018.stdout:grafana.smithi018 smithi018 *:3000 running (7m) 8s ago 22m 73.7M - 9.4.12 2bacad6d85d8 b08e3bf176dd 2024-04-12T21:07:26.409 INFO:teuthology.orchestra.run.smithi018.stdout:mds.cephfs.smithi018.acqbsc smithi018 running (16s) 8s ago 15m 10.9M - 19.0.0-2879-g324baf8d 8717720dc5e5 021f99ab24ae 2024-04-12T21:07:26.409 INFO:teuthology.orchestra.run.smithi018.stdout:mds.cephfs.smithi018.szjluo smithi018 running (15m) 8s ago 15m 10.4M - 17.2.7-827-gd3b5060a 21de9b7369ad cf9bf0c5031f 2024-04-12T21:07:26.409 INFO:teuthology.orchestra.run.smithi018.stdout:mds.cephfs.smithi186.jdllyn smithi186 running (15m) 35s ago 15m 15.6M - 17.2.7-827-gd3b5060a 21de9b7369ad 2b2bc46bd29a 2024-04-12T21:07:26.409 INFO:teuthology.orchestra.run.smithi018.stdout:mds.cephfs.smithi186.wjnafs smithi186 running (15m) 35s ago 15m 15.4M - 17.2.7-827-gd3b5060a 21de9b7369ad 74b115d6d6fd 2024-04-12T21:07:26.410 INFO:teuthology.orchestra.run.smithi018.stdout:mgr.smithi018.xjzwga smithi018 *:8443,9283,8765 running (10m) 8s ago 24m 508M - 19.0.0-2879-g324baf8d 8717720dc5e5 56088305eab7 2024-04-12T21:07:26.410 INFO:teuthology.orchestra.run.smithi018.stdout:mgr.smithi186.kqcfxk smithi186 *:8443,9283,8765 running (9m) 35s ago 21m 452M - 19.0.0-2879-g324baf8d 8717720dc5e5 09cd766e5e0d 2024-04-12T21:07:26.410 INFO:teuthology.orchestra.run.smithi018.stdout:mon.smithi018 smithi018 running (5m) 8s ago 24m 55.8M 2048M 19.0.0-2879-g324baf8d 8717720dc5e5 df3e201dfed6 2024-04-12T21:07:26.410 INFO:teuthology.orchestra.run.smithi018.stdout:mon.smithi186 smithi186 running (5m) 35s ago 21m 37.7M 2048M 19.0.0-2879-g324baf8d 8717720dc5e5 c77498da9b2b 2024-04-12T21:07:26.410 INFO:teuthology.orchestra.run.smithi018.stdout:node-exporter.smithi018 smithi018 *:9100 running (9m) 8s ago 22m 8066k - 1.5.0 0da6a335fe13 326c3d9c6847 2024-04-12T21:07:26.410 INFO:teuthology.orchestra.run.smithi018.stdout:node-exporter.smithi186 smithi186 *:9100 running (8m) 35s ago 21m 7826k - 1.5.0 0da6a335fe13 ec2112baeecf 2024-04-12T21:07:26.410 INFO:teuthology.orchestra.run.smithi018.stdout:osd.0 smithi018 running (3m) 8s ago 20m 120M 1559M 19.0.0-2879-g324baf8d 8717720dc5e5 60b1273ca047 2024-04-12T21:07:26.410 INFO:teuthology.orchestra.run.smithi018.stdout:osd.1 smithi018 running (3m) 8s ago 19m 91.1M 1559M 19.0.0-2879-g324baf8d 8717720dc5e5 170ce1e5fb8d 2024-04-12T21:07:26.410 INFO:teuthology.orchestra.run.smithi018.stdout:osd.2 smithi018 running (2m) 8s ago 18m 90.1M 1559M 19.0.0-2879-g324baf8d 8717720dc5e5 c2cbc6aa2839 2024-04-12T21:07:26.410 INFO:teuthology.orchestra.run.smithi018.stdout:osd.3 smithi186 running (2m) 35s ago 18m 96.6M 2581M 19.0.0-2879-g324baf8d 8717720dc5e5 0138e8262e4b 2024-04-12T21:07:26.410 INFO:teuthology.orchestra.run.smithi018.stdout:osd.4 smithi186 running (83s) 35s ago 17m 87.2M 2581M 19.0.0-2879-g324baf8d 8717720dc5e5 b20cb63b25c9 2024-04-12T21:07:26.410 INFO:teuthology.orchestra.run.smithi018.stdout:osd.5 smithi186 running (41s) 35s ago 17m 11.1M 2581M 19.0.0-2879-g324baf8d 8717720dc5e5 2513bea2610e 2024-04-12T21:07:26.410 INFO:teuthology.orchestra.run.smithi018.stdout:prometheus.smithi018 smithi018 *:9095 running (8m) 8s ago 21m 60.5M - 2.43.0 a07b618ecd1d f58841127e21 2024-04-12T21:07:26.411 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:07:26.415+0000 7f6ba5600700 1 -- 172.21.15.18:0/2975918827 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f6b8406ec00 msgr2=0x7f6b840710b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:07:26.411 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:07:26.415+0000 7f6ba5600700 1 --2- 172.21.15.18:0/2975918827 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f6b8406ec00 0x7f6b840710b0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f6b90004f90 tx=0x7f6b90009e30 comp rx=0 tx=0).stop 2024-04-12T21:07:26.411 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:07:26.415+0000 7f6ba5600700 1 -- 172.21.15.18:0/2975918827 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6ba00fc430 msgr2=0x7f6ba011a550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:07:26.411 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:07:26.415+0000 7f6ba5600700 1 --2- 172.21.15.18:0/2975918827 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6ba00fc430 0x7f6ba011a550 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f6b8c00b790 tx=0x7f6b8c009250 comp rx=0 tx=0).stop 2024-04-12T21:07:26.412 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:07:26.416+0000 7f6ba5600700 1 -- 172.21.15.18:0/2975918827 shutdown_connections 2024-04-12T21:07:26.412 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:07:26.416+0000 7f6ba5600700 1 --2- 172.21.15.18:0/2975918827 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f6b8406ec00 0x7f6b840710b0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:07:26.412 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:07:26.416+0000 7f6ba5600700 1 --2- 172.21.15.18:0/2975918827 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6ba00fba90 0x7f6ba011a010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:07:26.412 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:07:26.416+0000 7f6ba5600700 1 --2- 172.21.15.18:0/2975918827 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6ba00fc430 0x7f6ba011a550 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:07:26.412 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:07:26.416+0000 7f6ba5600700 1 -- 172.21.15.18:0/2975918827 >> 172.21.15.18:0/2975918827 conn(0x7f6ba00f7710 msgr2=0x7f6ba010a830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:07:26.412 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:07:26.416+0000 7f6ba5600700 1 -- 172.21.15.18:0/2975918827 shutdown_connections 2024-04-12T21:07:26.412 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:07:26.416+0000 7f6ba5600700 1 -- 172.21.15.18:0/2975918827 wait complete. 2024-04-12T21:07:26.564 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:07:26.568+0000 7f259cc00700 1 -- 172.21.15.18:0/3334096122 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f25981059b0 msgr2=0x7f25980ff020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:07:26.564 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:07:26.568+0000 7f259cc00700 1 --2- 172.21.15.18:0/3334096122 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f25981059b0 0x7f25980ff020 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f2584004940 tx=0x7f2584009e30 comp rx=0 tx=0).stop 2024-04-12T21:07:26.564 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:07:26.568+0000 7f259cc00700 1 -- 172.21.15.18:0/3334096122 shutdown_connections 2024-04-12T21:07:26.564 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:07:26.568+0000 7f259cc00700 1 --2- 172.21.15.18:0/3334096122 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f25981059b0 0x7f25980ff020 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:07:26.564 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:07:26.568+0000 7f259cc00700 1 --2- 172.21.15.18:0/3334096122 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2598105010 0x7f25981053e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:07:26.564 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:07:26.568+0000 7f259cc00700 1 -- 172.21.15.18:0/3334096122 >> 172.21.15.18:0/3334096122 conn(0x7f25980faba0 msgr2=0x7f25980fcfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:07:26.564 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:07:26.568+0000 7f259cc00700 1 -- 172.21.15.18:0/3334096122 shutdown_connections 2024-04-12T21:07:26.564 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:07:26.568+0000 7f259cc00700 1 -- 172.21.15.18:0/3334096122 wait complete. 2024-04-12T21:07:26.565 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:07:26 smithi018 ceph-mon[112590]: from='client.34440 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:07:26.565 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:07:26 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' 2024-04-12T21:07:26.565 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:07:26 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' 2024-04-12T21:07:26.565 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:07:26 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-12T21:07:26.565 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:07:26 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-12T21:07:26.565 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:07:26 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' 2024-04-12T21:07:26.566 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:07:26.569+0000 7f259cc00700 1 Processor -- start 2024-04-12T21:07:26.566 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:07:26.569+0000 7f259cc00700 1 -- start start 2024-04-12T21:07:26.566 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:07:26.570+0000 7f259cc00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2598105010 0x7f2598115d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:07:26.567 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:07:26.570+0000 7f259cc00700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f25981059b0 0x7f25981162b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:07:26.567 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:07:26.570+0000 7f259cc00700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2598116900 con 0x7f2598105010 2024-04-12T21:07:26.567 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:07:26.570+0000 7f259cc00700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f259810fcd0 con 0x7f25981059b0 2024-04-12T21:07:26.567 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:07:26.570+0000 7f2595e00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2598105010 0x7f2598115d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:07:26.567 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:07:26.570+0000 7f2595e00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2598105010 0x7f2598115d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:55242/0 (socket says 172.21.15.18:55242) 2024-04-12T21:07:26.567 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:07:26.570+0000 7f2595e00700 1 -- 172.21.15.18:0/485771209 learned_addr learned my addr 172.21.15.18:0/485771209 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:07:26.567 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:07:26.570+0000 7f2595400700 1 --2- 172.21.15.18:0/485771209 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f25981059b0 0x7f25981162b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:07:26.567 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:07:26.571+0000 7f2595e00700 1 -- 172.21.15.18:0/485771209 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f25981059b0 msgr2=0x7f25981162b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:07:26.567 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:07:26.571+0000 7f2595e00700 1 --2- 172.21.15.18:0/485771209 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f25981059b0 0x7f25981162b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:07:26.567 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:07:26.571+0000 7f2595e00700 1 -- 172.21.15.18:0/485771209 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2584005040 con 0x7f2598105010 2024-04-12T21:07:26.567 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:07:26.571+0000 7f2595e00700 1 --2- 172.21.15.18:0/485771209 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2598105010 0x7f2598115d70 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f2588002a50 tx=0x7f25880046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:07:26.568 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:07:26.571+0000 7f258e000700 1 -- 172.21.15.18:0/485771209 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2588007cc0 con 0x7f2598105010 2024-04-12T21:07:26.568 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:07:26.571+0000 7f258e000700 1 -- 172.21.15.18:0/485771209 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f258800f040 con 0x7f2598105010 2024-04-12T21:07:26.568 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:07:26.571+0000 7f259cc00700 1 -- 172.21.15.18:0/485771209 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f259810ffb0 con 0x7f2598105010 2024-04-12T21:07:26.568 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:07:26.571+0000 7f258e000700 1 -- 172.21.15.18:0/485771209 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f258800da30 con 0x7f2598105010 2024-04-12T21:07:26.568 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:07:26.571+0000 7f259cc00700 1 -- 172.21.15.18:0/485771209 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2598110570 con 0x7f2598105010 2024-04-12T21:07:26.568 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:07:26.573+0000 7f259cc00700 1 -- 172.21.15.18:0/485771209 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2598100760 con 0x7f2598105010 2024-04-12T21:07:26.569 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:07:26.574+0000 7f258e000700 1 -- 172.21.15.18:0/485771209 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7f258800db90 con 0x7f2598105010 2024-04-12T21:07:26.570 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:07:26.575+0000 7f258e000700 1 --2- 172.21.15.18:0/485771209 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f257c06ec00 0x7f257c0710b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:07:26.570 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:07:26.575+0000 7f258e000700 1 -- 172.21.15.18:0/485771209 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 6757+0+0 (secure 0 0 0) 0x7f258808e660 con 0x7f2598105010 2024-04-12T21:07:26.571 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:07:26.575+0000 7f2595400700 1 --2- 172.21.15.18:0/485771209 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f257c06ec00 0x7f257c0710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:07:26.571 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:07:26.576+0000 7f2595400700 1 --2- 172.21.15.18:0/485771209 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f257c06ec00 0x7f257c0710b0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f2598111610 tx=0x7f2584009e30 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:07:26.575 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:07:26.580+0000 7f258e000700 1 -- 172.21.15.18:0/485771209 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f258805aeb0 con 0x7f2598105010 2024-04-12T21:07:26.779 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:07:26 smithi186 ceph-mon[86379]: from='client.34440 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:07:26.779 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:07:26 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' 2024-04-12T21:07:26.779 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:07:26 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' 2024-04-12T21:07:26.780 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:07:26 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-12T21:07:26.780 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:07:26 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-12T21:07:26.780 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:07:26 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' 2024-04-12T21:07:26.911 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:07:26.916+0000 7f259cc00700 1 -- 172.21.15.18:0/485771209 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f259804bed0 con 0x7f2598105010 2024-04-12T21:07:26.912 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:07:26.916+0000 7f258e000700 1 -- 172.21.15.18:0/485771209 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+837 (secure 0 0 0) 0x7f258805aa40 con 0x7f2598105010 2024-04-12T21:07:26.912 INFO:teuthology.orchestra.run.smithi018.stdout:{ 2024-04-12T21:07:26.912 INFO:teuthology.orchestra.run.smithi018.stdout: "mon": { 2024-04-12T21:07:26.912 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 19.0.0-2879-g324baf8d (324baf8ddb91dea46cac50cb535c306115791355) squid (dev)": 2 2024-04-12T21:07:26.913 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-12T21:07:26.913 INFO:teuthology.orchestra.run.smithi018.stdout: "mgr": { 2024-04-12T21:07:26.913 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 19.0.0-2879-g324baf8d (324baf8ddb91dea46cac50cb535c306115791355) squid (dev)": 2 2024-04-12T21:07:26.913 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-12T21:07:26.913 INFO:teuthology.orchestra.run.smithi018.stdout: "osd": { 2024-04-12T21:07:26.913 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 19.0.0-2879-g324baf8d (324baf8ddb91dea46cac50cb535c306115791355) squid (dev)": 6 2024-04-12T21:07:26.913 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-12T21:07:26.913 INFO:teuthology.orchestra.run.smithi018.stdout: "mds": { 2024-04-12T21:07:26.913 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 17.2.7-827-gd3b5060a (d3b5060a17449edfae9385cdf792afec109fca5d) quincy (stable)": 3, 2024-04-12T21:07:26.913 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 19.0.0-2879-g324baf8d (324baf8ddb91dea46cac50cb535c306115791355) squid (dev)": 1 2024-04-12T21:07:26.913 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-12T21:07:26.913 INFO:teuthology.orchestra.run.smithi018.stdout: "overall": { 2024-04-12T21:07:26.913 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 17.2.7-827-gd3b5060a (d3b5060a17449edfae9385cdf792afec109fca5d) quincy (stable)": 3, 2024-04-12T21:07:26.913 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 19.0.0-2879-g324baf8d (324baf8ddb91dea46cac50cb535c306115791355) squid (dev)": 11 2024-04-12T21:07:26.914 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-04-12T21:07:26.914 INFO:teuthology.orchestra.run.smithi018.stdout:} 2024-04-12T21:07:26.914 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:07:26.919+0000 7f259cc00700 1 -- 172.21.15.18:0/485771209 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f257c06ec00 msgr2=0x7f257c0710b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:07:26.915 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:07:26.919+0000 7f259cc00700 1 --2- 172.21.15.18:0/485771209 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f257c06ec00 0x7f257c0710b0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f2598111610 tx=0x7f2584009e30 comp rx=0 tx=0).stop 2024-04-12T21:07:26.915 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:07:26.919+0000 7f259cc00700 1 -- 172.21.15.18:0/485771209 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2598105010 msgr2=0x7f2598115d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:07:26.915 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:07:26.919+0000 7f259cc00700 1 --2- 172.21.15.18:0/485771209 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2598105010 0x7f2598115d70 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f2588002a50 tx=0x7f25880046b0 comp rx=0 tx=0).stop 2024-04-12T21:07:26.915 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:07:26.919+0000 7f259cc00700 1 -- 172.21.15.18:0/485771209 shutdown_connections 2024-04-12T21:07:26.916 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:07:26.919+0000 7f259cc00700 1 --2- 172.21.15.18:0/485771209 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f257c06ec00 0x7f257c0710b0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:07:26.916 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:07:26.919+0000 7f259cc00700 1 --2- 172.21.15.18:0/485771209 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2598105010 0x7f2598115d70 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:07:26.916 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:07:26.919+0000 7f259cc00700 1 --2- 172.21.15.18:0/485771209 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f25981059b0 0x7f25981162b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:07:26.916 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:07:26.919+0000 7f259cc00700 1 -- 172.21.15.18:0/485771209 >> 172.21.15.18:0/485771209 conn(0x7f25980faba0 msgr2=0x7f25980fbcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:07:26.916 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:07:26.919+0000 7f259cc00700 1 -- 172.21.15.18:0/485771209 shutdown_connections 2024-04-12T21:07:26.916 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:07:26.920+0000 7f259cc00700 1 -- 172.21.15.18:0/485771209 wait complete. 2024-04-12T21:07:27.071 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:07:27.075+0000 7f8b80e00700 1 -- 172.21.15.18:0/2222293229 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8b7c105990 msgr2=0x7f8b7c0ff000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:07:27.071 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:07:27.075+0000 7f8b80e00700 1 --2- 172.21.15.18:0/2222293229 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8b7c105990 0x7f8b7c0ff000 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f8b68004940 tx=0x7f8b68009e30 comp rx=0 tx=0).stop 2024-04-12T21:07:27.071 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:07:27.075+0000 7f8b80e00700 1 -- 172.21.15.18:0/2222293229 shutdown_connections 2024-04-12T21:07:27.071 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:07:27.075+0000 7f8b80e00700 1 --2- 172.21.15.18:0/2222293229 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8b7c105990 0x7f8b7c0ff000 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:07:27.071 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:07:27.075+0000 7f8b80e00700 1 --2- 172.21.15.18:0/2222293229 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8b7c104ff0 0x7f8b7c1053c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:07:27.072 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:07:27.075+0000 7f8b80e00700 1 -- 172.21.15.18:0/2222293229 >> 172.21.15.18:0/2222293229 conn(0x7f8b7c0faba0 msgr2=0x7f8b7c0fcfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:07:27.072 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:07:27.075+0000 7f8b80e00700 1 -- 172.21.15.18:0/2222293229 shutdown_connections 2024-04-12T21:07:27.072 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:07:27.075+0000 7f8b80e00700 1 -- 172.21.15.18:0/2222293229 wait complete. 2024-04-12T21:07:27.072 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:07:27.076+0000 7f8b80e00700 1 Processor -- start 2024-04-12T21:07:27.072 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:07:27.076+0000 7f8b80e00700 1 -- start start 2024-04-12T21:07:27.074 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:07:27.077+0000 7f8b80e00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8b7c104ff0 0x7f8b7c115b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:07:27.074 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:07:27.077+0000 7f8b80e00700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8b7c105990 0x7f8b7c1160c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:07:27.074 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:07:27.077+0000 7f8b80e00700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b7c116710 con 0x7f8b7c104ff0 2024-04-12T21:07:27.074 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:07:27.077+0000 7f8b80e00700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b7c10fcf0 con 0x7f8b7c105990 2024-04-12T21:07:27.074 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:07:27.077+0000 7f8b79e00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8b7c104ff0 0x7f8b7c115b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:07:27.074 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:07:27.077+0000 7f8b79e00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8b7c104ff0 0x7f8b7c115b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:55270/0 (socket says 172.21.15.18:55270) 2024-04-12T21:07:27.074 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:07:27.077+0000 7f8b79e00700 1 -- 172.21.15.18:0/3719251089 learned_addr learned my addr 172.21.15.18:0/3719251089 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:07:27.074 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:07:27.077+0000 7f8b79400700 1 --2- 172.21.15.18:0/3719251089 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8b7c105990 0x7f8b7c1160c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:07:27.074 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:07:27.078+0000 7f8b79e00700 1 -- 172.21.15.18:0/3719251089 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8b7c105990 msgr2=0x7f8b7c1160c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:07:27.074 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:07:27.078+0000 7f8b79e00700 1 --2- 172.21.15.18:0/3719251089 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8b7c105990 0x7f8b7c1160c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:07:27.075 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:07:27.078+0000 7f8b79e00700 1 -- 172.21.15.18:0/3719251089 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b68005040 con 0x7f8b7c104ff0 2024-04-12T21:07:27.075 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:07:27.078+0000 7f8b79e00700 1 --2- 172.21.15.18:0/3719251089 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8b7c104ff0 0x7f8b7c115b80 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f8b6c00c8c0 tx=0x7f8b6c0044c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:07:27.076 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:07:27.078+0000 7f8b72000700 1 -- 172.21.15.18:0/3719251089 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8b6c00a980 con 0x7f8b7c104ff0 2024-04-12T21:07:27.076 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:07:27.078+0000 7f8b72000700 1 -- 172.21.15.18:0/3719251089 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f8b6c00ce60 con 0x7f8b7c104ff0 2024-04-12T21:07:27.076 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:07:27.078+0000 7f8b72000700 1 -- 172.21.15.18:0/3719251089 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8b6c0115e0 con 0x7f8b7c104ff0 2024-04-12T21:07:27.076 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:07:27.078+0000 7f8b80e00700 1 -- 172.21.15.18:0/3719251089 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8b7c10ffd0 con 0x7f8b7c104ff0 2024-04-12T21:07:27.076 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:07:27.079+0000 7f8b80e00700 1 -- 172.21.15.18:0/3719251089 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8b7c110590 con 0x7f8b7c104ff0 2024-04-12T21:07:27.076 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:07:27.080+0000 7f8b80e00700 1 -- 172.21.15.18:0/3719251089 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8b7c100740 con 0x7f8b7c104ff0 2024-04-12T21:07:27.077 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:07:27.081+0000 7f8b72000700 1 -- 172.21.15.18:0/3719251089 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7f8b6c012400 con 0x7f8b7c104ff0 2024-04-12T21:07:27.078 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:07:27.082+0000 7f8b72000700 1 --2- 172.21.15.18:0/3719251089 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f8b6006ec00 0x7f8b600710b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:07:27.078 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:07:27.082+0000 7f8b72000700 1 -- 172.21.15.18:0/3719251089 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 6757+0+0 (secure 0 0 0) 0x7f8b6c08eb30 con 0x7f8b7c104ff0 2024-04-12T21:07:27.078 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:07:27.082+0000 7f8b79400700 1 --2- 172.21.15.18:0/3719251089 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f8b6006ec00 0x7f8b600710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:07:27.079 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:07:27.083+0000 7f8b79400700 1 --2- 172.21.15.18:0/3719251089 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f8b6006ec00 0x7f8b600710b0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f8b7c111420 tx=0x7f8b68009e30 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:07:27.083 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:07:27.087+0000 7f8b72000700 1 -- 172.21.15.18:0/3719251089 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f8b6c05b2e0 con 0x7f8b7c104ff0 2024-04-12T21:07:27.366 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:07:27.370+0000 7f8b80e00700 1 -- 172.21.15.18:0/3719251089 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f8b7c04c710 con 0x7f8b7c104ff0 2024-04-12T21:07:27.366 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:07:27.371+0000 7f8b72000700 1 -- 172.21.15.18:0/3719251089 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 27 v27) v1 ==== 76+0+1869 (secure 0 0 0) 0x7f8b6c0126b0 con 0x7f8b7c104ff0 2024-04-12T21:07:27.367 INFO:teuthology.orchestra.run.smithi018.stdout:e27 2024-04-12T21:07:27.367 INFO:teuthology.orchestra.run.smithi018.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-12T21:07:27.367 INFO:teuthology.orchestra.run.smithi018.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-12T21:07:27.367 INFO:teuthology.orchestra.run.smithi018.stdout:legacy client fscid: 1 2024-04-12T21:07:27.367 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-12T21:07:27.367 INFO:teuthology.orchestra.run.smithi018.stdout:Filesystem 'cephfs' (1) 2024-04-12T21:07:27.367 INFO:teuthology.orchestra.run.smithi018.stdout:fs_name cephfs 2024-04-12T21:07:27.367 INFO:teuthology.orchestra.run.smithi018.stdout:epoch 24 2024-04-12T21:07:27.367 INFO:teuthology.orchestra.run.smithi018.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-04-12T21:07:27.367 INFO:teuthology.orchestra.run.smithi018.stdout:created 2024-04-12T20:51:57.362850+0000 2024-04-12T21:07:27.367 INFO:teuthology.orchestra.run.smithi018.stdout:modified 2024-04-12T21:06:59.717190+0000 2024-04-12T21:07:27.368 INFO:teuthology.orchestra.run.smithi018.stdout:tableserver 0 2024-04-12T21:07:27.368 INFO:teuthology.orchestra.run.smithi018.stdout:root 0 2024-04-12T21:07:27.368 INFO:teuthology.orchestra.run.smithi018.stdout:session_timeout 60 2024-04-12T21:07:27.368 INFO:teuthology.orchestra.run.smithi018.stdout:session_autoclose 300 2024-04-12T21:07:27.368 INFO:teuthology.orchestra.run.smithi018.stdout:max_file_size 1099511627776 2024-04-12T21:07:27.368 INFO:teuthology.orchestra.run.smithi018.stdout:max_xattr_size 65536 2024-04-12T21:07:27.368 INFO:teuthology.orchestra.run.smithi018.stdout:required_client_features {} 2024-04-12T21:07:27.368 INFO:teuthology.orchestra.run.smithi018.stdout:last_failure 0 2024-04-12T21:07:27.368 INFO:teuthology.orchestra.run.smithi018.stdout:last_failure_osd_epoch 100 2024-04-12T21:07:27.368 INFO:teuthology.orchestra.run.smithi018.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-12T21:07:27.368 INFO:teuthology.orchestra.run.smithi018.stdout:max_mds 2 2024-04-12T21:07:27.368 INFO:teuthology.orchestra.run.smithi018.stdout:in 0,1 2024-04-12T21:07:27.368 INFO:teuthology.orchestra.run.smithi018.stdout:up {} 2024-04-12T21:07:27.368 INFO:teuthology.orchestra.run.smithi018.stdout:failed 0,1 2024-04-12T21:07:27.369 INFO:teuthology.orchestra.run.smithi018.stdout:damaged 2024-04-12T21:07:27.369 INFO:teuthology.orchestra.run.smithi018.stdout:stopped 2024-04-12T21:07:27.369 INFO:teuthology.orchestra.run.smithi018.stdout:data_pools [3] 2024-04-12T21:07:27.369 INFO:teuthology.orchestra.run.smithi018.stdout:metadata_pool 2 2024-04-12T21:07:27.369 INFO:teuthology.orchestra.run.smithi018.stdout:inline_data disabled 2024-04-12T21:07:27.369 INFO:teuthology.orchestra.run.smithi018.stdout:balancer 2024-04-12T21:07:27.369 INFO:teuthology.orchestra.run.smithi018.stdout:bal_rank_mask -1 2024-04-12T21:07:27.369 INFO:teuthology.orchestra.run.smithi018.stdout:standby_count_wanted 1 2024-04-12T21:07:27.369 INFO:teuthology.orchestra.run.smithi018.stdout:qdb_cluster leader: 0 members: 2024-04-12T21:07:27.369 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-12T21:07:27.369 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-12T21:07:27.369 INFO:teuthology.orchestra.run.smithi018.stdout:Standby daemons: 2024-04-12T21:07:27.369 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-12T21:07:27.369 INFO:teuthology.orchestra.run.smithi018.stdout:[mds.cephfs.smithi186.wjnafs{-1:24277} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.186:6826/438347207,v1:172.21.15.186:6827/438347207] compat {c=[1],r=[1],i=[7ff]}] 2024-04-12T21:07:27.369 INFO:teuthology.orchestra.run.smithi018.stdout:[mds.cephfs.smithi186.jdllyn{-1:24281} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.186:6824/1464359952,v1:172.21.15.186:6825/1464359952] compat {c=[1],r=[1],i=[7ff]}] 2024-04-12T21:07:27.370 INFO:teuthology.orchestra.run.smithi018.stdout:[mds.cephfs.smithi018.szjluo{-1:34430} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.18:6826/785330430,v1:172.21.15.18:6827/785330430] compat {c=[1],r=[1],i=[7ff]}] 2024-04-12T21:07:27.370 INFO:teuthology.orchestra.run.smithi018.stdout:[mds.cephfs.smithi018.acqbsc{-1:34436} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.18:6828/755179764,v1:172.21.15.18:6829/755179764] compat {c=[1],r=[1],i=[1fff]}] 2024-04-12T21:07:27.370 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:07:27.373+0000 7f8b80e00700 1 -- 172.21.15.18:0/3719251089 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f8b6006ec00 msgr2=0x7f8b600710b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:07:27.370 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:07:27.373+0000 7f8b80e00700 1 --2- 172.21.15.18:0/3719251089 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f8b6006ec00 0x7f8b600710b0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f8b7c111420 tx=0x7f8b68009e30 comp rx=0 tx=0).stop 2024-04-12T21:07:27.370 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:07:27.373+0000 7f8b80e00700 1 -- 172.21.15.18:0/3719251089 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8b7c104ff0 msgr2=0x7f8b7c115b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:07:27.371 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:07:27.373+0000 7f8b80e00700 1 --2- 172.21.15.18:0/3719251089 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8b7c104ff0 0x7f8b7c115b80 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f8b6c00c8c0 tx=0x7f8b6c0044c0 comp rx=0 tx=0).stop 2024-04-12T21:07:27.371 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:07:27.373+0000 7f8b80e00700 1 -- 172.21.15.18:0/3719251089 shutdown_connections 2024-04-12T21:07:27.371 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:07:27.373+0000 7f8b80e00700 1 --2- 172.21.15.18:0/3719251089 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f8b6006ec00 0x7f8b600710b0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:07:27.371 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:07:27.373+0000 7f8b80e00700 1 --2- 172.21.15.18:0/3719251089 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8b7c104ff0 0x7f8b7c115b80 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:07:27.371 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:07:27.373+0000 7f8b80e00700 1 --2- 172.21.15.18:0/3719251089 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8b7c105990 0x7f8b7c1160c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:07:27.371 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:07:27.373+0000 7f8b80e00700 1 -- 172.21.15.18:0/3719251089 >> 172.21.15.18:0/3719251089 conn(0x7f8b7c0faba0 msgr2=0x7f8b7c0fbae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:07:27.371 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:07:27.374+0000 7f8b80e00700 1 -- 172.21.15.18:0/3719251089 shutdown_connections 2024-04-12T21:07:27.371 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:07:27.374+0000 7f8b80e00700 1 -- 172.21.15.18:0/3719251089 wait complete. 2024-04-12T21:07:27.372 INFO:teuthology.orchestra.run.smithi018.stderr:dumped fsmap epoch 27 2024-04-12T21:07:27.528 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:07:27.531+0000 7f7f8e200700 1 -- 172.21.15.18:0/2516673892 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7f88101cf0 msgr2=0x7f7f881020c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:07:27.528 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:07:27.531+0000 7f7f8e200700 1 --2- 172.21.15.18:0/2516673892 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7f88101cf0 0x7f7f881020c0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f7f78004940 tx=0x7f7f78009e30 comp rx=0 tx=0).stop 2024-04-12T21:07:27.528 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:07:27.531+0000 7f7f8e200700 1 -- 172.21.15.18:0/2516673892 shutdown_connections 2024-04-12T21:07:27.528 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:07:27.531+0000 7f7f8e200700 1 --2- 172.21.15.18:0/2516673892 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7f88102690 0x7f7f880fbd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:07:27.528 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:07:27.531+0000 7f7f8e200700 1 --2- 172.21.15.18:0/2516673892 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7f88101cf0 0x7f7f881020c0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:07:27.528 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:07:27.531+0000 7f7f8e200700 1 -- 172.21.15.18:0/2516673892 >> 172.21.15.18:0/2516673892 conn(0x7f7f880f7860 msgr2=0x7f7f880f9c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:07:27.528 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:07:27.532+0000 7f7f8e200700 1 -- 172.21.15.18:0/2516673892 shutdown_connections 2024-04-12T21:07:27.529 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:07:27.532+0000 7f7f8e200700 1 -- 172.21.15.18:0/2516673892 wait complete. 2024-04-12T21:07:27.529 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:07:27.532+0000 7f7f8e200700 1 Processor -- start 2024-04-12T21:07:27.529 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:07:27.533+0000 7f7f8e200700 1 -- start start 2024-04-12T21:07:27.530 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:07:27.533+0000 7f7f8e200700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7f88101cf0 0x7f7f88115d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:07:27.530 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:07:27.533+0000 7f7f8e200700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7f88102690 0x7f7f881162c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:07:27.530 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:07:27.533+0000 7f7f8e200700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f88116910 con 0x7f7f88102690 2024-04-12T21:07:27.530 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:07:27.533+0000 7f7f8e200700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f8810fce0 con 0x7f7f88101cf0 2024-04-12T21:07:27.530 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:07:27.533+0000 7f7f86a00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7f88102690 0x7f7f881162c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:07:27.530 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:07:27.533+0000 7f7f86a00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7f88102690 0x7f7f881162c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:55284/0 (socket says 172.21.15.18:55284) 2024-04-12T21:07:27.530 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:07:27.533+0000 7f7f86a00700 1 -- 172.21.15.18:0/1482640811 learned_addr learned my addr 172.21.15.18:0/1482640811 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:07:27.530 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:07:27.534+0000 7f7f87400700 1 --2- 172.21.15.18:0/1482640811 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7f88101cf0 0x7f7f88115d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:07:27.530 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:07:27.534+0000 7f7f86a00700 1 -- 172.21.15.18:0/1482640811 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7f88101cf0 msgr2=0x7f7f88115d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:07:27.530 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:07:27.534+0000 7f7f86a00700 1 --2- 172.21.15.18:0/1482640811 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7f88101cf0 0x7f7f88115d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:07:27.531 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:07:27.534+0000 7f7f86a00700 1 -- 172.21.15.18:0/1482640811 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f78005040 con 0x7f7f88102690 2024-04-12T21:07:27.531 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:07:27.534+0000 7f7f86a00700 1 --2- 172.21.15.18:0/1482640811 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7f88102690 0x7f7f881162c0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f7f7400c800 tx=0x7f7f74004470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:07:27.531 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:07:27.534+0000 7f7f7f400700 1 -- 172.21.15.18:0/1482640811 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7f7400a760 con 0x7f7f88102690 2024-04-12T21:07:27.531 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:07:27.534+0000 7f7f7f400700 1 -- 172.21.15.18:0/1482640811 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f7f7400ad90 con 0x7f7f88102690 2024-04-12T21:07:27.531 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:07:27.534+0000 7f7f7f400700 1 -- 172.21.15.18:0/1482640811 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7f740113a0 con 0x7f7f88102690 2024-04-12T21:07:27.532 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:07:27.534+0000 7f7f8e200700 1 -- 172.21.15.18:0/1482640811 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7f8810ffc0 con 0x7f7f88102690 2024-04-12T21:07:27.532 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:07:27.535+0000 7f7f8e200700 1 -- 172.21.15.18:0/1482640811 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7f88110580 con 0x7f7f88102690 2024-04-12T21:07:27.532 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:07:27.536+0000 7f7f8e200700 1 -- 172.21.15.18:0/1482640811 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7f880fd440 con 0x7f7f88102690 2024-04-12T21:07:27.532 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:07:27.537+0000 7f7f7f400700 1 -- 172.21.15.18:0/1482640811 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7f7f74012400 con 0x7f7f88102690 2024-04-12T21:07:27.533 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:07:27.538+0000 7f7f7f400700 1 --2- 172.21.15.18:0/1482640811 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f7f6c06ec10 0x7f7f6c0710c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:07:27.534 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:07:27.538+0000 7f7f7f400700 1 -- 172.21.15.18:0/1482640811 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 6757+0+0 (secure 0 0 0) 0x7f7f7408e8d0 con 0x7f7f88102690 2024-04-12T21:07:27.534 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:07:27.538+0000 7f7f87400700 1 --2- 172.21.15.18:0/1482640811 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f7f6c06ec10 0x7f7f6c0710c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:07:27.535 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:07:27.539+0000 7f7f87400700 1 --2- 172.21.15.18:0/1482640811 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f7f6c06ec10 0x7f7f6c0710c0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f7f7800b3b0 tx=0x7f7f78009e30 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:07:27.538 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:07:27.542+0000 7f7f7f400700 1 -- 172.21.15.18:0/1482640811 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f7f7405b1c0 con 0x7f7f88102690 2024-04-12T21:07:27.650 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:07:27 smithi018 ceph-mon[112590]: pgmap v228: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:07:27.650 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:07:27 smithi018 ceph-mon[112590]: from='client.34444 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:07:27.650 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:07:27 smithi018 ceph-mon[112590]: from='client.34448 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:07:27.650 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:07:27 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-12T21:07:27.650 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:07:27 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T21:07:27.650 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:07:27 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T21:07:27.650 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:07:27 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T21:07:27.650 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:07:27 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T21:07:27.650 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:07:27 smithi018 ceph-mon[112590]: from='client.? 172.21.15.18:0/485771209' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T21:07:27.650 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:07:27 smithi018 ceph-mon[112590]: from='client.? 172.21.15.18:0/3719251089' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-12T21:07:27.779 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:07:27 smithi186 ceph-mon[86379]: pgmap v228: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:07:27.780 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:07:27 smithi186 ceph-mon[86379]: from='client.34444 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:07:27.780 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:07:27 smithi186 ceph-mon[86379]: from='client.34448 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:07:27.780 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:07:27 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-12T21:07:27.780 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:07:27 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T21:07:27.780 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:07:27 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T21:07:27.780 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:07:27 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T21:07:27.780 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:07:27 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T21:07:27.780 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:07:27 smithi186 ceph-mon[86379]: from='client.? 172.21.15.18:0/485771209' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T21:07:27.780 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:07:27 smithi186 ceph-mon[86379]: from='client.? 172.21.15.18:0/3719251089' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-12T21:07:27.795 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:07:27.799+0000 7f7f8e200700 1 -- 172.21.15.18:0/1482640811 --> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f7f88108e60 con 0x7f7f6c06ec10 2024-04-12T21:07:27.797 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:07:27.801+0000 7f7f7f400700 1 -- 172.21.15.18:0/1482640811 <== mgr.34102 v2:172.21.15.18:6800/3812891919 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f7f88108e60 con 0x7f7f6c06ec10 2024-04-12T21:07:27.797 INFO:teuthology.orchestra.run.smithi018.stdout:{ 2024-04-12T21:07:27.797 INFO:teuthology.orchestra.run.smithi018.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:f4452def3be4d350f4a8e5feaf7ebb1511be9298fe68478ba7a2d6fbd7d69a63", 2024-04-12T21:07:27.797 INFO:teuthology.orchestra.run.smithi018.stdout: "in_progress": true, 2024-04-12T21:07:27.798 INFO:teuthology.orchestra.run.smithi018.stdout: "which": "Upgrading all daemon types on all hosts", 2024-04-12T21:07:27.798 INFO:teuthology.orchestra.run.smithi018.stdout: "services_complete": [ 2024-04-12T21:07:27.798 INFO:teuthology.orchestra.run.smithi018.stdout: "crash", 2024-04-12T21:07:27.798 INFO:teuthology.orchestra.run.smithi018.stdout: "osd", 2024-04-12T21:07:27.798 INFO:teuthology.orchestra.run.smithi018.stdout: "mon", 2024-04-12T21:07:27.798 INFO:teuthology.orchestra.run.smithi018.stdout: "mgr" 2024-04-12T21:07:27.798 INFO:teuthology.orchestra.run.smithi018.stdout: ], 2024-04-12T21:07:27.798 INFO:teuthology.orchestra.run.smithi018.stdout: "progress": "13/21 daemons upgraded", 2024-04-12T21:07:27.798 INFO:teuthology.orchestra.run.smithi018.stdout: "message": "Currently upgrading mds daemons", 2024-04-12T21:07:27.798 INFO:teuthology.orchestra.run.smithi018.stdout: "is_paused": false 2024-04-12T21:07:27.798 INFO:teuthology.orchestra.run.smithi018.stdout:} 2024-04-12T21:07:27.800 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:07:27.804+0000 7f7f8e200700 1 -- 172.21.15.18:0/1482640811 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f7f6c06ec10 msgr2=0x7f7f6c0710c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:07:27.800 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:07:27.804+0000 7f7f8e200700 1 --2- 172.21.15.18:0/1482640811 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f7f6c06ec10 0x7f7f6c0710c0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f7f7800b3b0 tx=0x7f7f78009e30 comp rx=0 tx=0).stop 2024-04-12T21:07:27.800 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:07:27.804+0000 7f7f8e200700 1 -- 172.21.15.18:0/1482640811 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7f88102690 msgr2=0x7f7f881162c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:07:27.800 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:07:27.804+0000 7f7f8e200700 1 --2- 172.21.15.18:0/1482640811 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7f88102690 0x7f7f881162c0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f7f7400c800 tx=0x7f7f74004470 comp rx=0 tx=0).stop 2024-04-12T21:07:27.800 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:07:27.804+0000 7f7f8e200700 1 -- 172.21.15.18:0/1482640811 shutdown_connections 2024-04-12T21:07:27.800 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:07:27.804+0000 7f7f8e200700 1 --2- 172.21.15.18:0/1482640811 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f7f6c06ec10 0x7f7f6c0710c0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:07:27.800 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:07:27.804+0000 7f7f8e200700 1 --2- 172.21.15.18:0/1482640811 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7f88101cf0 0x7f7f88115d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:07:27.800 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:07:27.804+0000 7f7f8e200700 1 --2- 172.21.15.18:0/1482640811 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7f88102690 0x7f7f881162c0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:07:27.801 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:07:27.804+0000 7f7f8e200700 1 -- 172.21.15.18:0/1482640811 >> 172.21.15.18:0/1482640811 conn(0x7f7f880f7860 msgr2=0x7f7f880f8f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:07:27.801 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:07:27.805+0000 7f7f8e200700 1 -- 172.21.15.18:0/1482640811 shutdown_connections 2024-04-12T21:07:27.801 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:07:27.805+0000 7f7f8e200700 1 -- 172.21.15.18:0/1482640811 wait complete. 2024-04-12T21:07:27.959 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:07:27.959+0000 7f8414200700 1 -- 172.21.15.18:0/1225468710 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f840c109430 msgr2=0x7f840c109800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:07:27.959 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:07:27.959+0000 7f8414200700 1 --2- 172.21.15.18:0/1225468710 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f840c109430 0x7f840c109800 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f8400004940 tx=0x7f8400009e30 comp rx=0 tx=0).stop 2024-04-12T21:07:27.959 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:07:27.959+0000 7f8414200700 1 -- 172.21.15.18:0/1225468710 shutdown_connections 2024-04-12T21:07:27.959 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:07:27.959+0000 7f8414200700 1 --2- 172.21.15.18:0/1225468710 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f840c109dd0 0x7f840c103440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:07:27.959 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:07:27.959+0000 7f8414200700 1 --2- 172.21.15.18:0/1225468710 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f840c109430 0x7f840c109800 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:07:27.959 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:07:27.959+0000 7f8414200700 1 -- 172.21.15.18:0/1225468710 >> 172.21.15.18:0/1225468710 conn(0x7f840c072c00 msgr2=0x7f840c075010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:07:27.959 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:07:27.959+0000 7f8414200700 1 -- 172.21.15.18:0/1225468710 shutdown_connections 2024-04-12T21:07:27.959 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:07:27.959+0000 7f8414200700 1 -- 172.21.15.18:0/1225468710 wait complete. 2024-04-12T21:07:27.959 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:07:27.960+0000 7f8414200700 1 Processor -- start 2024-04-12T21:07:27.959 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:07:27.960+0000 7f8414200700 1 -- start start 2024-04-12T21:07:27.959 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:07:27.961+0000 7f8414200700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f840c109430 0x7f840c119f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:07:27.959 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:07:27.961+0000 7f8414200700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f840c109dd0 0x7f840c11a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:07:27.960 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:07:27.961+0000 7f8414200700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f840c11ab60 con 0x7f840c109430 2024-04-12T21:07:27.960 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:07:27.961+0000 7f8414200700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f840c1140b0 con 0x7f840c109dd0 2024-04-12T21:07:27.960 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:07:27.961+0000 7f8411800700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f840c109430 0x7f840c119f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:07:27.960 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:07:27.961+0000 7f8411800700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f840c109430 0x7f840c119f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:55300/0 (socket says 172.21.15.18:55300) 2024-04-12T21:07:27.960 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:07:27.961+0000 7f8411800700 1 -- 172.21.15.18:0/2464931544 learned_addr learned my addr 172.21.15.18:0/2464931544 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:07:27.960 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:07:27.961+0000 7f8410e00700 1 --2- 172.21.15.18:0/2464931544 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f840c109dd0 0x7f840c11a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:07:27.960 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:07:27.961+0000 7f8411800700 1 -- 172.21.15.18:0/2464931544 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f840c109dd0 msgr2=0x7f840c11a480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:07:27.961 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:07:27.961+0000 7f8411800700 1 --2- 172.21.15.18:0/2464931544 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f840c109dd0 0x7f840c11a480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:07:27.961 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:07:27.961+0000 7f8411800700 1 -- 172.21.15.18:0/2464931544 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8400005040 con 0x7f840c109430 2024-04-12T21:07:27.961 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:07:27.961+0000 7f8410e00700 1 --2- 172.21.15.18:0/2464931544 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f840c109dd0 0x7f840c11a480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-12T21:07:27.961 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:07:27.962+0000 7f8411800700 1 --2- 172.21.15.18:0/2464931544 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f840c109430 0x7f840c119f40 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f8400009180 tx=0x7f8400009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:07:27.961 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:07:27.962+0000 7f8409600700 1 -- 172.21.15.18:0/2464931544 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f840000e040 con 0x7f840c109430 2024-04-12T21:07:27.961 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:07:27.962+0000 7f8409600700 1 -- 172.21.15.18:0/2464931544 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f840000bbc0 con 0x7f840c109430 2024-04-12T21:07:27.961 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:07:27.962+0000 7f8414200700 1 -- 172.21.15.18:0/2464931544 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f840c1142d0 con 0x7f840c109430 2024-04-12T21:07:27.961 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:07:27.962+0000 7f8414200700 1 -- 172.21.15.18:0/2464931544 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f840c114890 con 0x7f840c109430 2024-04-12T21:07:27.961 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:07:27.963+0000 7f8409600700 1 -- 172.21.15.18:0/2464931544 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f840001c630 con 0x7f840c109430 2024-04-12T21:07:27.962 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:07:27.964+0000 7f8414200700 1 -- 172.21.15.18:0/2464931544 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f840c104b80 con 0x7f840c109430 2024-04-12T21:07:27.962 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:07:27.965+0000 7f8409600700 1 -- 172.21.15.18:0/2464931544 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7f840000bd30 con 0x7f840c109430 2024-04-12T21:07:27.962 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:07:27.966+0000 7f8409600700 1 --2- 172.21.15.18:0/2464931544 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f83f406ec00 0x7f83f40710b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:07:27.962 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:07:27.966+0000 7f8409600700 1 -- 172.21.15.18:0/2464931544 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 6757+0+0 (secure 0 0 0) 0x7f840008fd70 con 0x7f840c109430 2024-04-12T21:07:27.962 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:07:27.966+0000 7f8410e00700 1 --2- 172.21.15.18:0/2464931544 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f83f406ec00 0x7f83f40710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:07:27.963 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:07:27.967+0000 7f8410e00700 1 --2- 172.21.15.18:0/2464931544 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f83f406ec00 0x7f83f40710b0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f83fc006fd0 tx=0x7f83fc008040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:07:27.966 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:07:27.971+0000 7f8409600700 1 -- 172.21.15.18:0/2464931544 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f840005c5c0 con 0x7f840c109430 2024-04-12T21:07:28.283 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:07:28.288+0000 7f8414200700 1 -- 172.21.15.18:0/2464931544 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f840c115590 con 0x7f840c109430 2024-04-12T21:07:28.284 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:07:28.289+0000 7f8409600700 1 -- 172.21.15.18:0/2464931544 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+233 (secure 0 0 0) 0x7f840005c150 con 0x7f840c109430 2024-04-12T21:07:28.284 INFO:teuthology.orchestra.run.smithi018.stdout:HEALTH_ERR 1 filesystem is degraded; 1 filesystem is offline 2024-04-12T21:07:28.284 INFO:teuthology.orchestra.run.smithi018.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-04-12T21:07:28.284 INFO:teuthology.orchestra.run.smithi018.stdout: fs cephfs is degraded 2024-04-12T21:07:28.285 INFO:teuthology.orchestra.run.smithi018.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-04-12T21:07:28.285 INFO:teuthology.orchestra.run.smithi018.stdout: fs cephfs is offline because no MDS is active for it. 2024-04-12T21:07:28.287 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:07:28.291+0000 7f8414200700 1 -- 172.21.15.18:0/2464931544 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f83f406ec00 msgr2=0x7f83f40710b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:07:28.287 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:07:28.291+0000 7f8414200700 1 --2- 172.21.15.18:0/2464931544 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f83f406ec00 0x7f83f40710b0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f83fc006fd0 tx=0x7f83fc008040 comp rx=0 tx=0).stop 2024-04-12T21:07:28.287 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:07:28.291+0000 7f8414200700 1 -- 172.21.15.18:0/2464931544 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f840c109430 msgr2=0x7f840c119f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:07:28.287 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:07:28.291+0000 7f8414200700 1 --2- 172.21.15.18:0/2464931544 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f840c109430 0x7f840c119f40 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f8400009180 tx=0x7f8400009e30 comp rx=0 tx=0).stop 2024-04-12T21:07:28.288 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:07:28.291+0000 7f8414200700 1 -- 172.21.15.18:0/2464931544 shutdown_connections 2024-04-12T21:07:28.288 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:07:28.291+0000 7f8414200700 1 --2- 172.21.15.18:0/2464931544 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f83f406ec00 0x7f83f40710b0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:07:28.288 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:07:28.291+0000 7f8414200700 1 --2- 172.21.15.18:0/2464931544 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f840c109430 0x7f840c119f40 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:07:28.288 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:07:28.291+0000 7f8414200700 1 --2- 172.21.15.18:0/2464931544 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f840c109dd0 0x7f840c11a480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:07:28.288 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:07:28.291+0000 7f8414200700 1 -- 172.21.15.18:0/2464931544 >> 172.21.15.18:0/2464931544 conn(0x7f840c072c00 msgr2=0x7f840c074c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:07:28.288 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:07:28.292+0000 7f8414200700 1 -- 172.21.15.18:0/2464931544 shutdown_connections 2024-04-12T21:07:28.288 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:07:28.292+0000 7f8414200700 1 -- 172.21.15.18:0/2464931544 wait complete. 2024-04-12T21:07:28.600 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:07:28 smithi018 ceph-mon[112590]: from='client.? 172.21.15.18:0/2464931544' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-12T21:07:28.779 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:07:28 smithi186 ceph-mon[86379]: from='client.? 172.21.15.18:0/2464931544' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-12T21:07:29.779 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:07:29 smithi186 ceph-mon[86379]: from='client.34460 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:07:29.779 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:07:29 smithi186 ceph-mon[86379]: pgmap v229: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:07:29.780 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:07:29 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' 2024-04-12T21:07:29.780 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:07:29 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi018.szjluo", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-12T21:07:29.780 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:07:29 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-12T21:07:29.850 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:07:29 smithi018 ceph-mon[112590]: from='client.34460 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:07:29.850 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:07:29 smithi018 ceph-mon[112590]: pgmap v229: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:07:29.851 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:07:29 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' 2024-04-12T21:07:29.851 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:07:29 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi018.szjluo", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-12T21:07:29.851 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:07:29 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-12T21:07:30.779 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:07:30 smithi186 ceph-mon[86379]: Upgrade: Updating mds.cephfs.smithi018.szjluo 2024-04-12T21:07:30.779 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:07:30 smithi186 ceph-mon[86379]: Deploying daemon mds.cephfs.smithi018.szjluo on smithi018 2024-04-12T21:07:30.850 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:07:30 smithi018 ceph-mon[112590]: Upgrade: Updating mds.cephfs.smithi018.szjluo 2024-04-12T21:07:30.850 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:07:30 smithi018 ceph-mon[112590]: Deploying daemon mds.cephfs.smithi018.szjluo on smithi018 2024-04-12T21:07:31.610 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:07:31 smithi018 ceph-mon[112590]: pgmap v230: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:07:31.779 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:07:31 smithi186 ceph-mon[86379]: pgmap v230: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:07:33.601 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:07:33 smithi018 ceph-mon[112590]: pgmap v231: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:07:33.779 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:07:33 smithi186 ceph-mon[86379]: pgmap v231: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:07:34.779 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:07:34 smithi186 ceph-mon[86379]: osdmap e102: 6 total, 6 up, 6 in 2024-04-12T21:07:34.779 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:07:34 smithi186 ceph-mon[86379]: fsmap cephfs:0/2 3 up:standby, 2 failed 2024-04-12T21:07:34.850 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:07:34 smithi018 ceph-mon[112590]: osdmap e102: 6 total, 6 up, 6 in 2024-04-12T21:07:34.850 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:07:34 smithi018 ceph-mon[112590]: fsmap cephfs:0/2 3 up:standby, 2 failed 2024-04-12T21:07:35.779 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:07:35 smithi186 ceph-mon[86379]: pgmap v233: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:07:35.850 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:07:35 smithi018 ceph-mon[112590]: pgmap v233: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:07:36.779 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:07:36 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi018.szjluo"}]: dispatch 2024-04-12T21:07:36.850 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:07:36 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi018.szjluo"}]: dispatch 2024-04-12T21:07:37.779 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:07:37 smithi186 ceph-mon[86379]: pgmap v234: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:07:37.850 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:07:37 smithi018 ceph-mon[112590]: pgmap v234: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:07:39.780 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:07:39 smithi186 ceph-mon[86379]: pgmap v235: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:07:39.780 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:07:39 smithi186 ceph-mon[86379]: mds.? [v2:172.21.15.18:6826/2407992653,v1:172.21.15.18:6827/2407992653] up:boot 2024-04-12T21:07:39.780 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:07:39 smithi186 ceph-mon[86379]: fsmap cephfs:0/2 4 up:standby, 2 failed 2024-04-12T21:07:39.780 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:07:39 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi018.szjluo"}]: dispatch 2024-04-12T21:07:39.780 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:07:39 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' 2024-04-12T21:07:39.780 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:07:39 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' 2024-04-12T21:07:39.780 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:07:39 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-12T21:07:39.850 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:07:39 smithi018 ceph-mon[112590]: pgmap v235: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:07:39.851 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:07:39 smithi018 ceph-mon[112590]: mds.? [v2:172.21.15.18:6826/2407992653,v1:172.21.15.18:6827/2407992653] up:boot 2024-04-12T21:07:39.851 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:07:39 smithi018 ceph-mon[112590]: fsmap cephfs:0/2 4 up:standby, 2 failed 2024-04-12T21:07:39.851 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:07:39 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi018.szjluo"}]: dispatch 2024-04-12T21:07:39.851 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:07:39 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' 2024-04-12T21:07:39.851 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:07:39 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' 2024-04-12T21:07:39.851 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:07:39 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-12T21:07:40.779 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:07:40 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T21:07:40.850 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:07:40 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T21:07:41.779 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:07:41 smithi186 ceph-mon[86379]: pgmap v236: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:07:41.850 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:07:41 smithi018 ceph-mon[112590]: pgmap v236: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:07:43.779 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:07:43 smithi186 ceph-mon[86379]: pgmap v237: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:07:43.850 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:07:43 smithi018 ceph-mon[112590]: pgmap v237: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:07:45.779 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:07:45 smithi186 ceph-mon[86379]: pgmap v238: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:07:45.779 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:07:45 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' 2024-04-12T21:07:45.779 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:07:45 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' 2024-04-12T21:07:45.850 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:07:45 smithi018 ceph-mon[112590]: pgmap v238: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:07:45.850 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:07:45 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' 2024-04-12T21:07:45.850 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:07:45 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' 2024-04-12T21:07:47.779 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:07:47 smithi186 ceph-mon[86379]: pgmap v239: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:07:47.779 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:07:47 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' 2024-04-12T21:07:47.779 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:07:47 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' 2024-04-12T21:07:47.850 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:07:47 smithi018 ceph-mon[112590]: pgmap v239: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:07:47.850 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:07:47 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' 2024-04-12T21:07:47.850 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:07:47 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' 2024-04-12T21:07:49.600 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:07:49 smithi018 ceph-mon[112590]: pgmap v240: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:07:50.029 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:07:49 smithi186 ceph-mon[86379]: pgmap v240: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:07:51.850 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:07:51 smithi018 ceph-mon[112590]: pgmap v241: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:07:52.029 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:07:51 smithi186 ceph-mon[86379]: pgmap v241: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:07:53.850 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:07:53 smithi018 ceph-mon[112590]: pgmap v242: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:07:54.029 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:07:53 smithi186 ceph-mon[86379]: pgmap v242: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:07:55.850 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:07:55 smithi018 ceph-mon[112590]: pgmap v243: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:07:55.850 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:07:55 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T21:07:56.029 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:07:55 smithi186 ceph-mon[86379]: pgmap v243: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:07:56.030 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:07:55 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T21:07:57.850 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:07:57 smithi018 ceph-mon[112590]: pgmap v244: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:07:58.029 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:07:57 smithi186 ceph-mon[86379]: pgmap v244: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:07:58.464 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:07:58.467+0000 7f6664400700 1 -- 172.21.15.18:0/574391535 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f665c109430 msgr2=0x7f665c109800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:07:58.464 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:07:58.467+0000 7f6664400700 1 --2- 172.21.15.18:0/574391535 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f665c109430 0x7f665c109800 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f6650004940 tx=0x7f6650009e30 comp rx=0 tx=0).stop 2024-04-12T21:07:58.464 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:07:58.467+0000 7f6664400700 1 -- 172.21.15.18:0/574391535 shutdown_connections 2024-04-12T21:07:58.464 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:07:58.467+0000 7f6664400700 1 --2- 172.21.15.18:0/574391535 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f665c109dd0 0x7f665c103440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:07:58.464 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:07:58.467+0000 7f6664400700 1 --2- 172.21.15.18:0/574391535 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f665c109430 0x7f665c109800 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:07:58.464 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:07:58.467+0000 7f6664400700 1 -- 172.21.15.18:0/574391535 >> 172.21.15.18:0/574391535 conn(0x7f665c072c00 msgr2=0x7f665c075010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:07:58.464 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:07:58.467+0000 7f6664400700 1 -- 172.21.15.18:0/574391535 shutdown_connections 2024-04-12T21:07:58.464 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:07:58.467+0000 7f6664400700 1 -- 172.21.15.18:0/574391535 wait complete. 2024-04-12T21:07:58.464 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:07:58.468+0000 7f6664400700 1 Processor -- start 2024-04-12T21:07:58.466 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:07:58.468+0000 7f6664400700 1 -- start start 2024-04-12T21:07:58.466 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:07:58.469+0000 7f6664400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f665c109430 0x7f665c119f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:07:58.466 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:07:58.469+0000 7f6664400700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f665c109dd0 0x7f665c11a4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:07:58.466 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:07:58.469+0000 7f6664400700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f665c11ab20 con 0x7f665c109430 2024-04-12T21:07:58.466 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:07:58.469+0000 7f6664400700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f665c114100 con 0x7f665c109dd0 2024-04-12T21:07:58.466 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:07:58.469+0000 7f6661a00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f665c109430 0x7f665c119f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:07:58.466 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:07:58.469+0000 7f6661a00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f665c109430 0x7f665c119f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:37096/0 (socket says 172.21.15.18:37096) 2024-04-12T21:07:58.466 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:07:58.469+0000 7f6661a00700 1 -- 172.21.15.18:0/3297879639 learned_addr learned my addr 172.21.15.18:0/3297879639 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:07:58.466 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:07:58.470+0000 7f6661000700 1 --2- 172.21.15.18:0/3297879639 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f665c109dd0 0x7f665c11a4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:07:58.466 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:07:58.470+0000 7f6661a00700 1 -- 172.21.15.18:0/3297879639 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f665c109dd0 msgr2=0x7f665c11a4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:07:58.467 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:07:58.470+0000 7f6661a00700 1 --2- 172.21.15.18:0/3297879639 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f665c109dd0 0x7f665c11a4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:07:58.467 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:07:58.470+0000 7f6661a00700 1 -- 172.21.15.18:0/3297879639 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6650005040 con 0x7f665c109430 2024-04-12T21:07:58.467 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:07:58.470+0000 7f6661a00700 1 --2- 172.21.15.18:0/3297879639 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f665c109430 0x7f665c119f90 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f6650009180 tx=0x7f6650009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:07:58.467 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:07:58.470+0000 7f6659600700 1 -- 172.21.15.18:0/3297879639 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f665000e040 con 0x7f665c109430 2024-04-12T21:07:58.467 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:07:58.470+0000 7f6659600700 1 -- 172.21.15.18:0/3297879639 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f665000bbc0 con 0x7f665c109430 2024-04-12T21:07:58.467 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:07:58.470+0000 7f6664400700 1 -- 172.21.15.18:0/3297879639 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f665c114380 con 0x7f665c109430 2024-04-12T21:07:58.467 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:07:58.470+0000 7f6659600700 1 -- 172.21.15.18:0/3297879639 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f665001c630 con 0x7f665c109430 2024-04-12T21:07:58.467 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:07:58.471+0000 7f6664400700 1 -- 172.21.15.18:0/3297879639 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f665c114940 con 0x7f665c109430 2024-04-12T21:07:58.468 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:07:58.472+0000 7f6664400700 1 -- 172.21.15.18:0/3297879639 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f665c104b80 con 0x7f665c109430 2024-04-12T21:07:58.468 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:07:58.473+0000 7f6659600700 1 -- 172.21.15.18:0/3297879639 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7f665000bd30 con 0x7f665c109430 2024-04-12T21:07:58.469 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:07:58.474+0000 7f6659600700 1 --2- 172.21.15.18:0/3297879639 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f664406ec00 0x7f66440710b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:07:58.470 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:07:58.474+0000 7f6659600700 1 -- 172.21.15.18:0/3297879639 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 6757+0+0 (secure 0 0 0) 0x7f665008fd40 con 0x7f665c109430 2024-04-12T21:07:58.470 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:07:58.474+0000 7f6661000700 1 --2- 172.21.15.18:0/3297879639 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f664406ec00 0x7f66440710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:07:58.470 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:07:58.475+0000 7f6661000700 1 --2- 172.21.15.18:0/3297879639 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f664406ec00 0x7f66440710b0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f665c115830 tx=0x7f6648009040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:07:58.476 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:07:58.480+0000 7f6659600700 1 -- 172.21.15.18:0/3297879639 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f665005c570 con 0x7f665c109430 2024-04-12T21:07:58.732 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:07:58.736+0000 7f6664400700 1 -- 172.21.15.18:0/3297879639 --> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f665c115650 con 0x7f664406ec00 2024-04-12T21:07:58.734 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:07:58.739+0000 7f6659600700 1 -- 172.21.15.18:0/3297879639 <== mgr.34102 v2:172.21.15.18:6800/3812891919 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f665c115650 con 0x7f664406ec00 2024-04-12T21:07:58.738 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:07:58.742+0000 7f6664400700 1 -- 172.21.15.18:0/3297879639 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f664406ec00 msgr2=0x7f66440710b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:07:58.738 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:07:58.742+0000 7f6664400700 1 --2- 172.21.15.18:0/3297879639 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f664406ec00 0x7f66440710b0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f665c115830 tx=0x7f6648009040 comp rx=0 tx=0).stop 2024-04-12T21:07:58.738 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:07:58.742+0000 7f6664400700 1 -- 172.21.15.18:0/3297879639 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f665c109430 msgr2=0x7f665c119f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:07:58.738 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:07:58.742+0000 7f6664400700 1 --2- 172.21.15.18:0/3297879639 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f665c109430 0x7f665c119f90 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f6650009180 tx=0x7f6650009e30 comp rx=0 tx=0).stop 2024-04-12T21:07:58.738 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:07:58.742+0000 7f6664400700 1 -- 172.21.15.18:0/3297879639 shutdown_connections 2024-04-12T21:07:58.738 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:07:58.742+0000 7f6664400700 1 --2- 172.21.15.18:0/3297879639 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f664406ec00 0x7f66440710b0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:07:58.738 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:07:58.742+0000 7f6664400700 1 --2- 172.21.15.18:0/3297879639 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f665c109430 0x7f665c119f90 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:07:58.738 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:07:58.742+0000 7f6664400700 1 --2- 172.21.15.18:0/3297879639 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f665c109dd0 0x7f665c11a4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:07:58.739 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:07:58.742+0000 7f6664400700 1 -- 172.21.15.18:0/3297879639 >> 172.21.15.18:0/3297879639 conn(0x7f665c072c00 msgr2=0x7f665c074c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:07:58.739 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:07:58.742+0000 7f6664400700 1 -- 172.21.15.18:0/3297879639 shutdown_connections 2024-04-12T21:07:58.739 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:07:58.742+0000 7f6664400700 1 -- 172.21.15.18:0/3297879639 wait complete. 2024-04-12T21:07:58.755 INFO:teuthology.orchestra.run.smithi018.stdout:true 2024-04-12T21:07:58.902 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:07:58.906+0000 7f06c6e00700 1 -- 172.21.15.18:0/2935547620 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f06c0102e20 msgr2=0x7f06c01031f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:07:58.902 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:07:58.906+0000 7f06c6e00700 1 --2- 172.21.15.18:0/2935547620 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f06c0102e20 0x7f06c01031f0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f06b0004940 tx=0x7f06b0009e30 comp rx=0 tx=0).stop 2024-04-12T21:07:58.902 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:07:58.906+0000 7f06c6e00700 1 -- 172.21.15.18:0/2935547620 shutdown_connections 2024-04-12T21:07:58.902 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:07:58.906+0000 7f06c6e00700 1 --2- 172.21.15.18:0/2935547620 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f06c01037c0 0x7f06c00fce30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:07:58.902 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:07:58.906+0000 7f06c6e00700 1 --2- 172.21.15.18:0/2935547620 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f06c0102e20 0x7f06c01031f0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:07:58.902 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:07:58.906+0000 7f06c6e00700 1 -- 172.21.15.18:0/2935547620 >> 172.21.15.18:0/2935547620 conn(0x7f06c00f8970 msgr2=0x7f06c00fad80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:07:58.903 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:07:58.906+0000 7f06c6e00700 1 -- 172.21.15.18:0/2935547620 shutdown_connections 2024-04-12T21:07:58.903 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:07:58.906+0000 7f06c6e00700 1 -- 172.21.15.18:0/2935547620 wait complete. 2024-04-12T21:07:58.903 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:07:58.907+0000 7f06c6e00700 1 Processor -- start 2024-04-12T21:07:58.903 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:07:58.907+0000 7f06c6e00700 1 -- start start 2024-04-12T21:07:58.903 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:07:58.908+0000 7f06c6e00700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f06c0102e20 0x7f06c0115d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:07:58.904 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:07:58.908+0000 7f06c6e00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f06c01037c0 0x7f06c0116290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:07:58.904 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:07:58.908+0000 7f06c6e00700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06c01168e0 con 0x7f06c01037c0 2024-04-12T21:07:58.904 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:07:58.908+0000 7f06c6e00700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06c010fd00 con 0x7f06c0102e20 2024-04-12T21:07:58.904 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:07:58.908+0000 7f06bf400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f06c01037c0 0x7f06c0116290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:07:58.905 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:07:58.908+0000 7f06bf400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f06c01037c0 0x7f06c0116290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:37110/0 (socket says 172.21.15.18:37110) 2024-04-12T21:07:58.905 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:07:58.908+0000 7f06bf400700 1 -- 172.21.15.18:0/1445062681 learned_addr learned my addr 172.21.15.18:0/1445062681 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:07:58.905 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:07:58.909+0000 7f06bfe00700 1 --2- 172.21.15.18:0/1445062681 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f06c0102e20 0x7f06c0115d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:07:58.905 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:07:58.909+0000 7f06bf400700 1 -- 172.21.15.18:0/1445062681 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f06c0102e20 msgr2=0x7f06c0115d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:07:58.905 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:07:58.909+0000 7f06bf400700 1 --2- 172.21.15.18:0/1445062681 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f06c0102e20 0x7f06c0115d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:07:58.905 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:07:58.909+0000 7f06bf400700 1 -- 172.21.15.18:0/1445062681 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06b0005040 con 0x7f06c01037c0 2024-04-12T21:07:58.905 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:07:58.909+0000 7f06bf400700 1 --2- 172.21.15.18:0/1445062681 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f06c01037c0 0x7f06c0116290 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f06ac002a50 tx=0x7f06ac004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:07:58.906 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:07:58.909+0000 7f06b7e00700 1 -- 172.21.15.18:0/1445062681 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f06ac007cc0 con 0x7f06c01037c0 2024-04-12T21:07:58.906 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:07:58.909+0000 7f06b7e00700 1 -- 172.21.15.18:0/1445062681 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f06ac00f040 con 0x7f06c01037c0 2024-04-12T21:07:58.906 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:07:58.909+0000 7f06b7e00700 1 -- 172.21.15.18:0/1445062681 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f06ac00daf0 con 0x7f06c01037c0 2024-04-12T21:07:58.906 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:07:58.909+0000 7f06c6e00700 1 -- 172.21.15.18:0/1445062681 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f06c010ffe0 con 0x7f06c01037c0 2024-04-12T21:07:58.906 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:07:58.910+0000 7f06c6e00700 1 -- 172.21.15.18:0/1445062681 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f06c01105a0 con 0x7f06c01037c0 2024-04-12T21:07:58.906 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:07:58.911+0000 7f06c6e00700 1 -- 172.21.15.18:0/1445062681 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f06c0106a70 con 0x7f06c01037c0 2024-04-12T21:07:58.908 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:07:58.912+0000 7f06b7e00700 1 -- 172.21.15.18:0/1445062681 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7f06ac009810 con 0x7f06c01037c0 2024-04-12T21:07:58.909 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:07:58.913+0000 7f06b7e00700 1 --2- 172.21.15.18:0/1445062681 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f06a406ec10 0x7f06a40710c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:07:58.909 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:07:58.913+0000 7f06b7e00700 1 -- 172.21.15.18:0/1445062681 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 6757+0+0 (secure 0 0 0) 0x7f06ac08e640 con 0x7f06c01037c0 2024-04-12T21:07:58.909 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:07:58.914+0000 7f06bfe00700 1 --2- 172.21.15.18:0/1445062681 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f06a406ec10 0x7f06a40710c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:07:58.910 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:07:58.914+0000 7f06bfe00700 1 --2- 172.21.15.18:0/1445062681 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f06a406ec10 0x7f06a40710c0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f06b000b3b0 tx=0x7f06b0009e30 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:07:58.914 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:07:58.918+0000 7f06b7e00700 1 -- 172.21.15.18:0/1445062681 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f06ac05ae90 con 0x7f06c01037c0 2024-04-12T21:07:59.164 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:07:59.168+0000 7f06c6e00700 1 -- 172.21.15.18:0/1445062681 --> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f06c0111410 con 0x7f06a406ec10 2024-04-12T21:07:59.166 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:07:59.171+0000 7f06b7e00700 1 -- 172.21.15.18:0/1445062681 <== mgr.34102 v2:172.21.15.18:6800/3812891919 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f06c0111410 con 0x7f06a406ec10 2024-04-12T21:07:59.169 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:07:59.173+0000 7f06c6e00700 1 -- 172.21.15.18:0/1445062681 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f06a406ec10 msgr2=0x7f06a40710c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:07:59.170 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:07:59.173+0000 7f06c6e00700 1 --2- 172.21.15.18:0/1445062681 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f06a406ec10 0x7f06a40710c0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f06b000b3b0 tx=0x7f06b0009e30 comp rx=0 tx=0).stop 2024-04-12T21:07:59.170 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:07:59.173+0000 7f06c6e00700 1 -- 172.21.15.18:0/1445062681 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f06c01037c0 msgr2=0x7f06c0116290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:07:59.170 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:07:59.173+0000 7f06c6e00700 1 --2- 172.21.15.18:0/1445062681 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f06c01037c0 0x7f06c0116290 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f06ac002a50 tx=0x7f06ac004600 comp rx=0 tx=0).stop 2024-04-12T21:07:59.170 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:07:59.174+0000 7f06c6e00700 1 -- 172.21.15.18:0/1445062681 shutdown_connections 2024-04-12T21:07:59.170 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:07:59.174+0000 7f06c6e00700 1 --2- 172.21.15.18:0/1445062681 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f06a406ec10 0x7f06a40710c0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:07:59.170 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:07:59.174+0000 7f06c6e00700 1 --2- 172.21.15.18:0/1445062681 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f06c0102e20 0x7f06c0115d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:07:59.170 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:07:59.174+0000 7f06c6e00700 1 --2- 172.21.15.18:0/1445062681 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f06c01037c0 0x7f06c0116290 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:07:59.170 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:07:59.174+0000 7f06c6e00700 1 -- 172.21.15.18:0/1445062681 >> 172.21.15.18:0/1445062681 conn(0x7f06c00f8970 msgr2=0x7f06c01073c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:07:59.170 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:07:59.174+0000 7f06c6e00700 1 -- 172.21.15.18:0/1445062681 shutdown_connections 2024-04-12T21:07:59.170 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:07:59.174+0000 7f06c6e00700 1 -- 172.21.15.18:0/1445062681 wait complete. 2024-04-12T21:07:59.316 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:07:59.320+0000 7f6e8c600700 1 -- 172.21.15.18:0/2217216810 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6e840fddf0 msgr2=0x7f6e840fe1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:07:59.316 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:07:59.320+0000 7f6e8c600700 1 --2- 172.21.15.18:0/2217216810 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6e840fddf0 0x7f6e840fe1c0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f6e78004940 tx=0x7f6e78009e30 comp rx=0 tx=0).stop 2024-04-12T21:07:59.316 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:07:59.320+0000 7f6e8c600700 1 -- 172.21.15.18:0/2217216810 shutdown_connections 2024-04-12T21:07:59.316 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:07:59.320+0000 7f6e8c600700 1 --2- 172.21.15.18:0/2217216810 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6e840fe790 0x7f6e84106dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:07:59.316 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:07:59.320+0000 7f6e8c600700 1 --2- 172.21.15.18:0/2217216810 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6e840fddf0 0x7f6e840fe1c0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:07:59.316 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:07:59.320+0000 7f6e8c600700 1 -- 172.21.15.18:0/2217216810 >> 172.21.15.18:0/2217216810 conn(0x7f6e840f78e0 msgr2=0x7f6e840f9cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:07:59.316 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:07:59.320+0000 7f6e8c600700 1 -- 172.21.15.18:0/2217216810 shutdown_connections 2024-04-12T21:07:59.316 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:07:59.320+0000 7f6e8c600700 1 -- 172.21.15.18:0/2217216810 wait complete. 2024-04-12T21:07:59.318 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:07:59.321+0000 7f6e8c600700 1 Processor -- start 2024-04-12T21:07:59.318 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:07:59.321+0000 7f6e8c600700 1 -- start start 2024-04-12T21:07:59.318 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:07:59.321+0000 7f6e8c600700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6e840fddf0 0x7f6e84115b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:07:59.318 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:07:59.321+0000 7f6e8c600700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6e840fe790 0x7f6e84116070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:07:59.318 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:07:59.321+0000 7f6e8c600700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e841166c0 con 0x7f6e840fe790 2024-04-12T21:07:59.318 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:07:59.321+0000 7f6e8c600700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e8410fca0 con 0x7f6e840fddf0 2024-04-12T21:07:59.318 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:07:59.322+0000 7f6e89200700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6e840fe790 0x7f6e84116070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:07:59.318 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:07:59.322+0000 7f6e89200700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6e840fe790 0x7f6e84116070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:37128/0 (socket says 172.21.15.18:37128) 2024-04-12T21:07:59.318 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:07:59.322+0000 7f6e89200700 1 -- 172.21.15.18:0/3307641395 learned_addr learned my addr 172.21.15.18:0/3307641395 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:07:59.318 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:07:59.322+0000 7f6e89c00700 1 --2- 172.21.15.18:0/3307641395 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6e840fddf0 0x7f6e84115b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:07:59.318 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:07:59.322+0000 7f6e89200700 1 -- 172.21.15.18:0/3307641395 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6e840fddf0 msgr2=0x7f6e84115b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:07:59.319 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:07:59.322+0000 7f6e89200700 1 --2- 172.21.15.18:0/3307641395 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6e840fddf0 0x7f6e84115b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:07:59.319 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:07:59.322+0000 7f6e89200700 1 -- 172.21.15.18:0/3307641395 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e78005040 con 0x7f6e840fe790 2024-04-12T21:07:59.319 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:07:59.323+0000 7f6e89200700 1 --2- 172.21.15.18:0/3307641395 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6e840fe790 0x7f6e84116070 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f6e70002a50 tx=0x7f6e700046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:07:59.320 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:07:59.323+0000 7f6e81600700 1 -- 172.21.15.18:0/3307641395 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6e70007cc0 con 0x7f6e840fe790 2024-04-12T21:07:59.320 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:07:59.323+0000 7f6e81600700 1 -- 172.21.15.18:0/3307641395 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f6e7000f040 con 0x7f6e840fe790 2024-04-12T21:07:59.320 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:07:59.323+0000 7f6e8c600700 1 -- 172.21.15.18:0/3307641395 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6e8410ff80 con 0x7f6e840fe790 2024-04-12T21:07:59.320 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:07:59.323+0000 7f6e81600700 1 -- 172.21.15.18:0/3307641395 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6e70009640 con 0x7f6e840fe790 2024-04-12T21:07:59.320 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:07:59.323+0000 7f6e8c600700 1 -- 172.21.15.18:0/3307641395 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6e841104c0 con 0x7f6e840fe790 2024-04-12T21:07:59.320 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:07:59.324+0000 7f6e8c600700 1 -- 172.21.15.18:0/3307641395 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6e84066770 con 0x7f6e840fe790 2024-04-12T21:07:59.320 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:07:59.325+0000 7f6e81600700 1 -- 172.21.15.18:0/3307641395 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7f6e700097a0 con 0x7f6e840fe790 2024-04-12T21:07:59.322 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:07:59.326+0000 7f6e81600700 1 --2- 172.21.15.18:0/3307641395 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f6e6c06ec00 0x7f6e6c0710b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:07:59.322 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:07:59.326+0000 7f6e81600700 1 -- 172.21.15.18:0/3307641395 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 6757+0+0 (secure 0 0 0) 0x7f6e7008e760 con 0x7f6e840fe790 2024-04-12T21:07:59.322 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:07:59.327+0000 7f6e89c00700 1 --2- 172.21.15.18:0/3307641395 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f6e6c06ec00 0x7f6e6c0710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:07:59.323 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:07:59.327+0000 7f6e89c00700 1 --2- 172.21.15.18:0/3307641395 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f6e6c06ec00 0x7f6e6c0710b0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f6e78009fd0 tx=0x7f6e78009e30 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:07:59.326 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:07:59.331+0000 7f6e81600700 1 -- 172.21.15.18:0/3307641395 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f6e7005af10 con 0x7f6e840fe790 2024-04-12T21:07:59.557 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:07:59.561+0000 7f6e8c600700 1 -- 172.21.15.18:0/3307641395 --> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f6e84109ef0 con 0x7f6e6c06ec00 2024-04-12T21:07:59.586 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:07:59.590+0000 7f6e81600700 1 -- 172.21.15.18:0/3307641395 <== mgr.34102 v2:172.21.15.18:6800/3812891919 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3520 (secure 0 0 0) 0x7f6e84109ef0 con 0x7f6e6c06ec00 2024-04-12T21:07:59.586 INFO:teuthology.orchestra.run.smithi018.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-12T21:07:59.587 INFO:teuthology.orchestra.run.smithi018.stdout:alertmanager.smithi018 smithi018 *:9093,9094 running (8m) 14s ago 23m 14.7M - 0.25.0 c8568f914cd2 c332090068e8 2024-04-12T21:07:59.587 INFO:teuthology.orchestra.run.smithi018.stdout:crash.smithi018 smithi018 running (4m) 14s ago 23m 6665k - 19.0.0-2879-g324baf8d 8717720dc5e5 b7f80d1f7982 2024-04-12T21:07:59.587 INFO:teuthology.orchestra.run.smithi018.stdout:crash.smithi186 smithi186 running (4m) 69s ago 21m 6660k - 19.0.0-2879-g324baf8d 8717720dc5e5 a4f55c81b623 2024-04-12T21:07:59.587 INFO:teuthology.orchestra.run.smithi018.stdout:grafana.smithi018 smithi018 *:3000 running (8m) 14s ago 22m 74.8M - 9.4.12 2bacad6d85d8 b08e3bf176dd 2024-04-12T21:07:59.587 INFO:teuthology.orchestra.run.smithi018.stdout:mds.cephfs.smithi018.acqbsc smithi018 running (49s) 14s ago 15m 11.1M - 19.0.0-2879-g324baf8d 8717720dc5e5 021f99ab24ae 2024-04-12T21:07:59.587 INFO:teuthology.orchestra.run.smithi018.stdout:mds.cephfs.smithi018.szjluo smithi018 running (21s) 14s ago 15m 10.9M - 19.0.0-2879-g324baf8d 8717720dc5e5 0003117479d4 2024-04-12T21:07:59.587 INFO:teuthology.orchestra.run.smithi018.stdout:mds.cephfs.smithi186.jdllyn smithi186 running (15m) 69s ago 15m 15.6M - 17.2.7-827-gd3b5060a 21de9b7369ad 2b2bc46bd29a 2024-04-12T21:07:59.587 INFO:teuthology.orchestra.run.smithi018.stdout:mds.cephfs.smithi186.wjnafs smithi186 running (15m) 69s ago 15m 15.4M - 17.2.7-827-gd3b5060a 21de9b7369ad 74b115d6d6fd 2024-04-12T21:07:59.587 INFO:teuthology.orchestra.run.smithi018.stdout:mgr.smithi018.xjzwga smithi018 *:8443,9283,8765 running (11m) 14s ago 24m 509M - 19.0.0-2879-g324baf8d 8717720dc5e5 56088305eab7 2024-04-12T21:07:59.587 INFO:teuthology.orchestra.run.smithi018.stdout:mgr.smithi186.kqcfxk smithi186 *:8443,9283,8765 running (9m) 69s ago 21m 452M - 19.0.0-2879-g324baf8d 8717720dc5e5 09cd766e5e0d 2024-04-12T21:07:59.587 INFO:teuthology.orchestra.run.smithi018.stdout:mon.smithi018 smithi018 running (6m) 14s ago 24m 57.0M 2048M 19.0.0-2879-g324baf8d 8717720dc5e5 df3e201dfed6 2024-04-12T21:07:59.587 INFO:teuthology.orchestra.run.smithi018.stdout:mon.smithi186 smithi186 running (5m) 69s ago 21m 37.7M 2048M 19.0.0-2879-g324baf8d 8717720dc5e5 c77498da9b2b 2024-04-12T21:07:59.587 INFO:teuthology.orchestra.run.smithi018.stdout:node-exporter.smithi018 smithi018 *:9100 running (9m) 14s ago 23m 7780k - 1.5.0 0da6a335fe13 326c3d9c6847 2024-04-12T21:07:59.588 INFO:teuthology.orchestra.run.smithi018.stdout:node-exporter.smithi186 smithi186 *:9100 running (9m) 69s ago 21m 7826k - 1.5.0 0da6a335fe13 ec2112baeecf 2024-04-12T21:07:59.588 INFO:teuthology.orchestra.run.smithi018.stdout:osd.0 smithi018 running (4m) 14s ago 20m 121M 1559M 19.0.0-2879-g324baf8d 8717720dc5e5 60b1273ca047 2024-04-12T21:07:59.588 INFO:teuthology.orchestra.run.smithi018.stdout:osd.1 smithi018 running (3m) 14s ago 20m 91.3M 1559M 19.0.0-2879-g324baf8d 8717720dc5e5 170ce1e5fb8d 2024-04-12T21:07:59.588 INFO:teuthology.orchestra.run.smithi018.stdout:osd.2 smithi018 running (3m) 14s ago 19m 90.3M 1559M 19.0.0-2879-g324baf8d 8717720dc5e5 c2cbc6aa2839 2024-04-12T21:07:59.588 INFO:teuthology.orchestra.run.smithi018.stdout:osd.3 smithi186 running (2m) 69s ago 18m 96.6M 2581M 19.0.0-2879-g324baf8d 8717720dc5e5 0138e8262e4b 2024-04-12T21:07:59.588 INFO:teuthology.orchestra.run.smithi018.stdout:osd.4 smithi186 running (116s) 69s ago 18m 87.2M 2581M 19.0.0-2879-g324baf8d 8717720dc5e5 b20cb63b25c9 2024-04-12T21:07:59.588 INFO:teuthology.orchestra.run.smithi018.stdout:osd.5 smithi186 running (74s) 69s ago 17m 11.1M 2581M 19.0.0-2879-g324baf8d 8717720dc5e5 2513bea2610e 2024-04-12T21:07:59.588 INFO:teuthology.orchestra.run.smithi018.stdout:prometheus.smithi018 smithi018 *:9095 running (9m) 14s ago 22m 60.6M - 2.43.0 a07b618ecd1d f58841127e21 2024-04-12T21:07:59.589 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:07:59.593+0000 7f6e8c600700 1 -- 172.21.15.18:0/3307641395 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f6e6c06ec00 msgr2=0x7f6e6c0710b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:07:59.589 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:07:59.593+0000 7f6e8c600700 1 --2- 172.21.15.18:0/3307641395 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f6e6c06ec00 0x7f6e6c0710b0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f6e78009fd0 tx=0x7f6e78009e30 comp rx=0 tx=0).stop 2024-04-12T21:07:59.589 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:07:59.593+0000 7f6e8c600700 1 -- 172.21.15.18:0/3307641395 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6e840fe790 msgr2=0x7f6e84116070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:07:59.589 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:07:59.593+0000 7f6e8c600700 1 --2- 172.21.15.18:0/3307641395 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6e840fe790 0x7f6e84116070 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f6e70002a50 tx=0x7f6e700046b0 comp rx=0 tx=0).stop 2024-04-12T21:07:59.589 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:07:59.593+0000 7f6e8c600700 1 -- 172.21.15.18:0/3307641395 shutdown_connections 2024-04-12T21:07:59.590 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:07:59.593+0000 7f6e8c600700 1 --2- 172.21.15.18:0/3307641395 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f6e6c06ec00 0x7f6e6c0710b0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:07:59.590 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:07:59.593+0000 7f6e8c600700 1 --2- 172.21.15.18:0/3307641395 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6e840fddf0 0x7f6e84115b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:07:59.590 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:07:59.593+0000 7f6e8c600700 1 --2- 172.21.15.18:0/3307641395 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6e840fe790 0x7f6e84116070 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:07:59.590 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:07:59.593+0000 7f6e8c600700 1 -- 172.21.15.18:0/3307641395 >> 172.21.15.18:0/3307641395 conn(0x7f6e840f78e0 msgr2=0x7f6e840f8e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:07:59.590 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:07:59.593+0000 7f6e8c600700 1 -- 172.21.15.18:0/3307641395 shutdown_connections 2024-04-12T21:07:59.590 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:07:59.593+0000 7f6e8c600700 1 -- 172.21.15.18:0/3307641395 wait complete. 2024-04-12T21:07:59.743 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:07:59.747+0000 7f2e5ee00700 1 -- 172.21.15.18:0/2635410695 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2e581037a0 msgr2=0x7f2e580fce10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:07:59.743 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:07:59.747+0000 7f2e5ee00700 1 --2- 172.21.15.18:0/2635410695 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2e581037a0 0x7f2e580fce10 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f2e44004940 tx=0x7f2e44009e30 comp rx=0 tx=0).stop 2024-04-12T21:07:59.743 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:07:59.747+0000 7f2e5ee00700 1 -- 172.21.15.18:0/2635410695 shutdown_connections 2024-04-12T21:07:59.743 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:07:59.747+0000 7f2e5ee00700 1 --2- 172.21.15.18:0/2635410695 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2e581037a0 0x7f2e580fce10 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:07:59.743 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:07:59.747+0000 7f2e5ee00700 1 --2- 172.21.15.18:0/2635410695 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2e58102e00 0x7f2e581031d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:07:59.743 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:07:59.747+0000 7f2e5ee00700 1 -- 172.21.15.18:0/2635410695 >> 172.21.15.18:0/2635410695 conn(0x7f2e580f89b0 msgr2=0x7f2e580fadc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:07:59.743 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:07:59.747+0000 7f2e5ee00700 1 -- 172.21.15.18:0/2635410695 shutdown_connections 2024-04-12T21:07:59.743 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:07:59.747+0000 7f2e5ee00700 1 -- 172.21.15.18:0/2635410695 wait complete. 2024-04-12T21:07:59.744 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:07:59.748+0000 7f2e5ee00700 1 Processor -- start 2024-04-12T21:07:59.744 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:07:59.748+0000 7f2e5ee00700 1 -- start start 2024-04-12T21:07:59.744 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:07:59.748+0000 7f2e5ee00700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2e58102e00 0x7f2e58115c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:07:59.744 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:07:59.748+0000 7f2e5ee00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2e581037a0 0x7f2e58116150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:07:59.744 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:07:59.748+0000 7f2e5ee00700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e581167a0 con 0x7f2e581037a0 2024-04-12T21:07:59.744 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:07:59.748+0000 7f2e5ee00700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e5810fd80 con 0x7f2e58102e00 2024-04-12T21:07:59.745 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:07:59.749+0000 7f2e57400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2e581037a0 0x7f2e58116150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:07:59.745 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:07:59.749+0000 7f2e57400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2e581037a0 0x7f2e58116150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:37140/0 (socket says 172.21.15.18:37140) 2024-04-12T21:07:59.745 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:07:59.749+0000 7f2e57400700 1 -- 172.21.15.18:0/1622763686 learned_addr learned my addr 172.21.15.18:0/1622763686 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:07:59.746 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:07:59.749+0000 7f2e57e00700 1 --2- 172.21.15.18:0/1622763686 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2e58102e00 0x7f2e58115c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:07:59.746 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:07:59.749+0000 7f2e57400700 1 -- 172.21.15.18:0/1622763686 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2e58102e00 msgr2=0x7f2e58115c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:07:59.746 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:07:59.749+0000 7f2e57400700 1 --2- 172.21.15.18:0/1622763686 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2e58102e00 0x7f2e58115c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:07:59.746 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:07:59.749+0000 7f2e57400700 1 -- 172.21.15.18:0/1622763686 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e44005040 con 0x7f2e581037a0 2024-04-12T21:07:59.746 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:07:59.749+0000 7f2e57e00700 1 --2- 172.21.15.18:0/1622763686 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2e58102e00 0x7f2e58115c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-12T21:07:59.746 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:07:59.750+0000 7f2e57400700 1 --2- 172.21.15.18:0/1622763686 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2e581037a0 0x7f2e58116150 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f2e44009100 tx=0x7f2e44009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:07:59.746 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:07:59.750+0000 7f2e4fe00700 1 -- 172.21.15.18:0/1622763686 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2e4400e040 con 0x7f2e581037a0 2024-04-12T21:07:59.746 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:07:59.750+0000 7f2e4fe00700 1 -- 172.21.15.18:0/1622763686 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f2e44013420 con 0x7f2e581037a0 2024-04-12T21:07:59.747 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:07:59.750+0000 7f2e4fe00700 1 -- 172.21.15.18:0/1622763686 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2e4401c630 con 0x7f2e581037a0 2024-04-12T21:07:59.747 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:07:59.750+0000 7f2e5ee00700 1 -- 172.21.15.18:0/1622763686 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2e58110000 con 0x7f2e581037a0 2024-04-12T21:07:59.747 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:07:59.750+0000 7f2e5ee00700 1 -- 172.21.15.18:0/1622763686 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2e581104f0 con 0x7f2e581037a0 2024-04-12T21:07:59.747 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:07:59.751+0000 7f2e5ee00700 1 -- 172.21.15.18:0/1622763686 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2e58102120 con 0x7f2e581037a0 2024-04-12T21:07:59.748 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:07:59.752+0000 7f2e4fe00700 1 -- 172.21.15.18:0/1622763686 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7f2e44013590 con 0x7f2e581037a0 2024-04-12T21:07:59.749 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:07:59.753+0000 7f2e4fe00700 1 --2- 172.21.15.18:0/1622763686 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f2e3c06ec00 0x7f2e3c0710b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:07:59.749 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:07:59.753+0000 7f2e4fe00700 1 -- 172.21.15.18:0/1622763686 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 6757+0+0 (secure 0 0 0) 0x7f2e4408fbd0 con 0x7f2e581037a0 2024-04-12T21:07:59.749 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:07:59.754+0000 7f2e57e00700 1 --2- 172.21.15.18:0/1622763686 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f2e3c06ec00 0x7f2e3c0710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:07:59.750 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:07:59.754+0000 7f2e57e00700 1 --2- 172.21.15.18:0/1622763686 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f2e3c06ec00 0x7f2e3c0710b0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f2e48009fd0 tx=0x7f2e4800b040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:07:59.754 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:07:59.758+0000 7f2e4fe00700 1 -- 172.21.15.18:0/1622763686 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f2e4405c420 con 0x7f2e581037a0 2024-04-12T21:07:59.850 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:07:59 smithi018 ceph-mon[112590]: pgmap v245: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:08:00.029 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:07:59 smithi186 ceph-mon[86379]: pgmap v245: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:08:00.070 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:08:00.074+0000 7f2e5ee00700 1 -- 172.21.15.18:0/1622763686 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f2e58111260 con 0x7f2e581037a0 2024-04-12T21:08:00.071 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:08:00.075+0000 7f2e4fe00700 1 -- 172.21.15.18:0/1622763686 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+837 (secure 0 0 0) 0x7f2e4405bfb0 con 0x7f2e581037a0 2024-04-12T21:08:00.071 INFO:teuthology.orchestra.run.smithi018.stdout:{ 2024-04-12T21:08:00.071 INFO:teuthology.orchestra.run.smithi018.stdout: "mon": { 2024-04-12T21:08:00.071 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 19.0.0-2879-g324baf8d (324baf8ddb91dea46cac50cb535c306115791355) squid (dev)": 2 2024-04-12T21:08:00.071 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-12T21:08:00.072 INFO:teuthology.orchestra.run.smithi018.stdout: "mgr": { 2024-04-12T21:08:00.072 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 19.0.0-2879-g324baf8d (324baf8ddb91dea46cac50cb535c306115791355) squid (dev)": 2 2024-04-12T21:08:00.072 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-12T21:08:00.072 INFO:teuthology.orchestra.run.smithi018.stdout: "osd": { 2024-04-12T21:08:00.072 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 19.0.0-2879-g324baf8d (324baf8ddb91dea46cac50cb535c306115791355) squid (dev)": 6 2024-04-12T21:08:00.072 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-12T21:08:00.072 INFO:teuthology.orchestra.run.smithi018.stdout: "mds": { 2024-04-12T21:08:00.072 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 17.2.7-827-gd3b5060a (d3b5060a17449edfae9385cdf792afec109fca5d) quincy (stable)": 2, 2024-04-12T21:08:00.072 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 19.0.0-2879-g324baf8d (324baf8ddb91dea46cac50cb535c306115791355) squid (dev)": 2 2024-04-12T21:08:00.072 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-12T21:08:00.072 INFO:teuthology.orchestra.run.smithi018.stdout: "overall": { 2024-04-12T21:08:00.072 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 17.2.7-827-gd3b5060a (d3b5060a17449edfae9385cdf792afec109fca5d) quincy (stable)": 2, 2024-04-12T21:08:00.072 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 19.0.0-2879-g324baf8d (324baf8ddb91dea46cac50cb535c306115791355) squid (dev)": 12 2024-04-12T21:08:00.072 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-04-12T21:08:00.072 INFO:teuthology.orchestra.run.smithi018.stdout:} 2024-04-12T21:08:00.073 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:08:00.078+0000 7f2e5ee00700 1 -- 172.21.15.18:0/1622763686 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f2e3c06ec00 msgr2=0x7f2e3c0710b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:08:00.074 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:08:00.078+0000 7f2e5ee00700 1 --2- 172.21.15.18:0/1622763686 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f2e3c06ec00 0x7f2e3c0710b0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f2e48009fd0 tx=0x7f2e4800b040 comp rx=0 tx=0).stop 2024-04-12T21:08:00.074 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:08:00.078+0000 7f2e5ee00700 1 -- 172.21.15.18:0/1622763686 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2e581037a0 msgr2=0x7f2e58116150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:08:00.074 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:08:00.078+0000 7f2e5ee00700 1 --2- 172.21.15.18:0/1622763686 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2e581037a0 0x7f2e58116150 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f2e44009100 tx=0x7f2e44009e30 comp rx=0 tx=0).stop 2024-04-12T21:08:00.074 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:08:00.078+0000 7f2e5ee00700 1 -- 172.21.15.18:0/1622763686 shutdown_connections 2024-04-12T21:08:00.074 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:08:00.078+0000 7f2e5ee00700 1 --2- 172.21.15.18:0/1622763686 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f2e3c06ec00 0x7f2e3c0710b0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:08:00.075 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:08:00.078+0000 7f2e5ee00700 1 --2- 172.21.15.18:0/1622763686 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2e58102e00 0x7f2e58115c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:08:00.075 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:08:00.078+0000 7f2e5ee00700 1 --2- 172.21.15.18:0/1622763686 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2e581037a0 0x7f2e58116150 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:08:00.075 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:08:00.078+0000 7f2e5ee00700 1 -- 172.21.15.18:0/1622763686 >> 172.21.15.18:0/1622763686 conn(0x7f2e580f89b0 msgr2=0x7f2e58070ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:08:00.075 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:08:00.078+0000 7f2e5ee00700 1 -- 172.21.15.18:0/1622763686 shutdown_connections 2024-04-12T21:08:00.075 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:08:00.078+0000 7f2e5ee00700 1 -- 172.21.15.18:0/1622763686 wait complete. 2024-04-12T21:08:00.232 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:08:00.236+0000 7f0651e00700 1 -- 172.21.15.18:0/2194750968 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f064c1092a0 msgr2=0x7f064c109670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:08:00.232 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:08:00.236+0000 7f0651e00700 1 --2- 172.21.15.18:0/2194750968 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f064c1092a0 0x7f064c109670 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f063c004940 tx=0x7f063c009e30 comp rx=0 tx=0).stop 2024-04-12T21:08:00.232 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:08:00.236+0000 7f0651e00700 1 -- 172.21.15.18:0/2194750968 shutdown_connections 2024-04-12T21:08:00.232 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:08:00.236+0000 7f0651e00700 1 --2- 172.21.15.18:0/2194750968 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f064c109c40 0x7f064c1032b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:08:00.232 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:08:00.236+0000 7f0651e00700 1 --2- 172.21.15.18:0/2194750968 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f064c1092a0 0x7f064c109670 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:08:00.232 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:08:00.236+0000 7f0651e00700 1 -- 172.21.15.18:0/2194750968 >> 172.21.15.18:0/2194750968 conn(0x7f064c072a30 msgr2=0x7f064c074e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:08:00.233 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:08:00.236+0000 7f0651e00700 1 -- 172.21.15.18:0/2194750968 shutdown_connections 2024-04-12T21:08:00.233 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:08:00.236+0000 7f0651e00700 1 -- 172.21.15.18:0/2194750968 wait complete. 2024-04-12T21:08:00.234 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:08:00.237+0000 7f0651e00700 1 Processor -- start 2024-04-12T21:08:00.234 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:08:00.237+0000 7f0651e00700 1 -- start start 2024-04-12T21:08:00.234 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:08:00.238+0000 7f0651e00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f064c1092a0 0x7f064c119ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:08:00.234 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:08:00.238+0000 7f0651e00700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f064c109c40 0x7f064c11a530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:08:00.234 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:08:00.238+0000 7f0651e00700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f064c11ab80 con 0x7f064c1092a0 2024-04-12T21:08:00.234 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:08:00.238+0000 7f0651e00700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f064c113f50 con 0x7f064c109c40 2024-04-12T21:08:00.234 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:08:00.238+0000 7f0650a00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f064c1092a0 0x7f064c119ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:08:00.234 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:08:00.238+0000 7f0650a00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f064c1092a0 0x7f064c119ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:37150/0 (socket says 172.21.15.18:37150) 2024-04-12T21:08:00.235 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:08:00.238+0000 7f0650a00700 1 -- 172.21.15.18:0/3822239847 learned_addr learned my addr 172.21.15.18:0/3822239847 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:08:00.235 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:08:00.238+0000 7f0650a00700 1 -- 172.21.15.18:0/3822239847 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f064c109c40 msgr2=0x7f064c11a530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:08:00.235 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:08:00.238+0000 7f0650a00700 1 --2- 172.21.15.18:0/3822239847 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f064c109c40 0x7f064c11a530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:08:00.235 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:08:00.238+0000 7f0650a00700 1 -- 172.21.15.18:0/3822239847 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f063c005040 con 0x7f064c1092a0 2024-04-12T21:08:00.235 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:08:00.239+0000 7f0650a00700 1 --2- 172.21.15.18:0/3822239847 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f064c1092a0 0x7f064c119ff0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f063c009150 tx=0x7f063c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:08:00.235 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:08:00.239+0000 7f0649600700 1 -- 172.21.15.18:0/3822239847 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f063c00e040 con 0x7f064c1092a0 2024-04-12T21:08:00.235 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:08:00.239+0000 7f0649600700 1 -- 172.21.15.18:0/3822239847 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f063c013420 con 0x7f064c1092a0 2024-04-12T21:08:00.235 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:08:00.239+0000 7f0651e00700 1 -- 172.21.15.18:0/3822239847 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f064c1141d0 con 0x7f064c1092a0 2024-04-12T21:08:00.236 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:08:00.239+0000 7f0649600700 1 -- 172.21.15.18:0/3822239847 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f063c01c630 con 0x7f064c1092a0 2024-04-12T21:08:00.236 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:08:00.239+0000 7f0651e00700 1 -- 172.21.15.18:0/3822239847 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f064c114790 con 0x7f064c1092a0 2024-04-12T21:08:00.236 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:08:00.240+0000 7f0651e00700 1 -- 172.21.15.18:0/3822239847 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f064c1049f0 con 0x7f064c1092a0 2024-04-12T21:08:00.236 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:08:00.241+0000 7f0649600700 1 -- 172.21.15.18:0/3822239847 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7f063c013590 con 0x7f064c1092a0 2024-04-12T21:08:00.238 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:08:00.242+0000 7f0649600700 1 --2- 172.21.15.18:0/3822239847 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f063006ec00 0x7f06300710b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:08:00.238 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:08:00.242+0000 7f0649600700 1 -- 172.21.15.18:0/3822239847 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 6757+0+0 (secure 0 0 0) 0x7f063c08fc00 con 0x7f064c1092a0 2024-04-12T21:08:00.238 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:08:00.242+0000 7f0643e00700 1 --2- 172.21.15.18:0/3822239847 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f063006ec00 0x7f06300710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:08:00.238 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:08:00.243+0000 7f0643e00700 1 --2- 172.21.15.18:0/3822239847 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f063006ec00 0x7f06300710b0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f064c115890 tx=0x7f0634009ce0 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:08:00.243 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:08:00.247+0000 7f0649600700 1 -- 172.21.15.18:0/3822239847 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f063c05c430 con 0x7f064c1092a0 2024-04-12T21:08:00.522 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:08:00.526+0000 7f0651e00700 1 -- 172.21.15.18:0/3822239847 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f064c1156d0 con 0x7f064c1092a0 2024-04-12T21:08:00.523 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:08:00.527+0000 7f0649600700 1 -- 172.21.15.18:0/3822239847 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 29 v29) v1 ==== 76+0+1872 (secure 0 0 0) 0x7f064c1156d0 con 0x7f064c1092a0 2024-04-12T21:08:00.524 INFO:teuthology.orchestra.run.smithi018.stdout:e29 2024-04-12T21:08:00.524 INFO:teuthology.orchestra.run.smithi018.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-12T21:08:00.524 INFO:teuthology.orchestra.run.smithi018.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-12T21:08:00.524 INFO:teuthology.orchestra.run.smithi018.stdout:legacy client fscid: 1 2024-04-12T21:08:00.524 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-12T21:08:00.524 INFO:teuthology.orchestra.run.smithi018.stdout:Filesystem 'cephfs' (1) 2024-04-12T21:08:00.524 INFO:teuthology.orchestra.run.smithi018.stdout:fs_name cephfs 2024-04-12T21:08:00.524 INFO:teuthology.orchestra.run.smithi018.stdout:epoch 24 2024-04-12T21:08:00.524 INFO:teuthology.orchestra.run.smithi018.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-04-12T21:08:00.524 INFO:teuthology.orchestra.run.smithi018.stdout:created 2024-04-12T20:51:57.362850+0000 2024-04-12T21:08:00.525 INFO:teuthology.orchestra.run.smithi018.stdout:modified 2024-04-12T21:06:59.717190+0000 2024-04-12T21:08:00.525 INFO:teuthology.orchestra.run.smithi018.stdout:tableserver 0 2024-04-12T21:08:00.525 INFO:teuthology.orchestra.run.smithi018.stdout:root 0 2024-04-12T21:08:00.525 INFO:teuthology.orchestra.run.smithi018.stdout:session_timeout 60 2024-04-12T21:08:00.525 INFO:teuthology.orchestra.run.smithi018.stdout:session_autoclose 300 2024-04-12T21:08:00.525 INFO:teuthology.orchestra.run.smithi018.stdout:max_file_size 1099511627776 2024-04-12T21:08:00.525 INFO:teuthology.orchestra.run.smithi018.stdout:max_xattr_size 65536 2024-04-12T21:08:00.525 INFO:teuthology.orchestra.run.smithi018.stdout:required_client_features {} 2024-04-12T21:08:00.525 INFO:teuthology.orchestra.run.smithi018.stdout:last_failure 0 2024-04-12T21:08:00.525 INFO:teuthology.orchestra.run.smithi018.stdout:last_failure_osd_epoch 100 2024-04-12T21:08:00.525 INFO:teuthology.orchestra.run.smithi018.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-12T21:08:00.525 INFO:teuthology.orchestra.run.smithi018.stdout:max_mds 2 2024-04-12T21:08:00.525 INFO:teuthology.orchestra.run.smithi018.stdout:in 0,1 2024-04-12T21:08:00.525 INFO:teuthology.orchestra.run.smithi018.stdout:up {} 2024-04-12T21:08:00.525 INFO:teuthology.orchestra.run.smithi018.stdout:failed 0,1 2024-04-12T21:08:00.525 INFO:teuthology.orchestra.run.smithi018.stdout:damaged 2024-04-12T21:08:00.526 INFO:teuthology.orchestra.run.smithi018.stdout:stopped 2024-04-12T21:08:00.526 INFO:teuthology.orchestra.run.smithi018.stdout:data_pools [3] 2024-04-12T21:08:00.526 INFO:teuthology.orchestra.run.smithi018.stdout:metadata_pool 2 2024-04-12T21:08:00.526 INFO:teuthology.orchestra.run.smithi018.stdout:inline_data disabled 2024-04-12T21:08:00.526 INFO:teuthology.orchestra.run.smithi018.stdout:balancer 2024-04-12T21:08:00.526 INFO:teuthology.orchestra.run.smithi018.stdout:bal_rank_mask -1 2024-04-12T21:08:00.526 INFO:teuthology.orchestra.run.smithi018.stdout:standby_count_wanted 1 2024-04-12T21:08:00.526 INFO:teuthology.orchestra.run.smithi018.stdout:qdb_cluster leader: 0 members: 2024-04-12T21:08:00.526 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-12T21:08:00.526 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-12T21:08:00.526 INFO:teuthology.orchestra.run.smithi018.stdout:Standby daemons: 2024-04-12T21:08:00.526 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-12T21:08:00.526 INFO:teuthology.orchestra.run.smithi018.stdout:[mds.cephfs.smithi186.wjnafs{-1:24277} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.186:6826/438347207,v1:172.21.15.186:6827/438347207] compat {c=[1],r=[1],i=[7ff]}] 2024-04-12T21:08:00.526 INFO:teuthology.orchestra.run.smithi018.stdout:[mds.cephfs.smithi186.jdllyn{-1:24281} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.186:6824/1464359952,v1:172.21.15.186:6825/1464359952] compat {c=[1],r=[1],i=[7ff]}] 2024-04-12T21:08:00.526 INFO:teuthology.orchestra.run.smithi018.stdout:[mds.cephfs.smithi018.acqbsc{-1:34436} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.18:6828/755179764,v1:172.21.15.18:6829/755179764] compat {c=[1],r=[1],i=[1fff]}] 2024-04-12T21:08:00.527 INFO:teuthology.orchestra.run.smithi018.stdout:[mds.cephfs.smithi018.szjluo{-1:34468} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.18:6826/2407992653,v1:172.21.15.18:6827/2407992653] compat {c=[1],r=[1],i=[1fff]}] 2024-04-12T21:08:00.527 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:08:00.530+0000 7f0651e00700 1 -- 172.21.15.18:0/3822239847 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f063006ec00 msgr2=0x7f06300710b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:08:00.527 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:08:00.530+0000 7f0651e00700 1 --2- 172.21.15.18:0/3822239847 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f063006ec00 0x7f06300710b0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f064c115890 tx=0x7f0634009ce0 comp rx=0 tx=0).stop 2024-04-12T21:08:00.527 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:08:00.530+0000 7f0651e00700 1 -- 172.21.15.18:0/3822239847 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f064c1092a0 msgr2=0x7f064c119ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:08:00.527 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:08:00.530+0000 7f0651e00700 1 --2- 172.21.15.18:0/3822239847 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f064c1092a0 0x7f064c119ff0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f063c009150 tx=0x7f063c009e30 comp rx=0 tx=0).stop 2024-04-12T21:08:00.528 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:08:00.530+0000 7f0651e00700 1 -- 172.21.15.18:0/3822239847 shutdown_connections 2024-04-12T21:08:00.528 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:08:00.530+0000 7f0651e00700 1 --2- 172.21.15.18:0/3822239847 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f063006ec00 0x7f06300710b0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:08:00.528 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:08:00.530+0000 7f0651e00700 1 --2- 172.21.15.18:0/3822239847 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f064c1092a0 0x7f064c119ff0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:08:00.528 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:08:00.530+0000 7f0651e00700 1 --2- 172.21.15.18:0/3822239847 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f064c109c40 0x7f064c11a530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:08:00.528 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:08:00.530+0000 7f0651e00700 1 -- 172.21.15.18:0/3822239847 >> 172.21.15.18:0/3822239847 conn(0x7f064c072a30 msgr2=0x7f064c0748d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:08:00.528 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:08:00.530+0000 7f0651e00700 1 -- 172.21.15.18:0/3822239847 shutdown_connections 2024-04-12T21:08:00.528 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:08:00.530+0000 7f0651e00700 1 -- 172.21.15.18:0/3822239847 wait complete. 2024-04-12T21:08:00.528 INFO:teuthology.orchestra.run.smithi018.stderr:dumped fsmap epoch 29 2024-04-12T21:08:00.679 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:08:00.682+0000 7fceae800700 1 -- 172.21.15.18:0/3378757556 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fcea81059c0 msgr2=0x7fcea80ff080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:08:00.679 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:08:00.682+0000 7fceae800700 1 --2- 172.21.15.18:0/3378757556 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fcea81059c0 0x7fcea80ff080 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fce94004940 tx=0x7fce94009e30 comp rx=0 tx=0).stop 2024-04-12T21:08:00.679 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:08:00.683+0000 7fceae800700 1 -- 172.21.15.18:0/3378757556 shutdown_connections 2024-04-12T21:08:00.679 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:08:00.683+0000 7fceae800700 1 --2- 172.21.15.18:0/3378757556 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fcea81059c0 0x7fcea80ff080 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:08:00.679 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:08:00.683+0000 7fceae800700 1 --2- 172.21.15.18:0/3378757556 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcea8105020 0x7fcea81053f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:08:00.679 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:08:00.683+0000 7fceae800700 1 -- 172.21.15.18:0/3378757556 >> 172.21.15.18:0/3378757556 conn(0x7fcea80faba0 msgr2=0x7fcea80fcfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:08:00.679 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:08:00.683+0000 7fceae800700 1 -- 172.21.15.18:0/3378757556 shutdown_connections 2024-04-12T21:08:00.679 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:08:00.683+0000 7fceae800700 1 -- 172.21.15.18:0/3378757556 wait complete. 2024-04-12T21:08:00.682 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:08:00.684+0000 7fceae800700 1 Processor -- start 2024-04-12T21:08:00.682 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:08:00.684+0000 7fceae800700 1 -- start start 2024-04-12T21:08:00.682 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:08:00.684+0000 7fceae800700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcea8105020 0x7fcea8115ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:08:00.682 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:08:00.684+0000 7fceae800700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fcea81059c0 0x7fcea81160e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:08:00.682 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:08:00.684+0000 7fceae800700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcea8116730 con 0x7fcea81059c0 2024-04-12T21:08:00.682 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:08:00.684+0000 7fceae800700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcea810fd10 con 0x7fcea8105020 2024-04-12T21:08:00.682 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:08:00.684+0000 7fcea6a00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fcea81059c0 0x7fcea81160e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:08:00.682 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:08:00.684+0000 7fcea6a00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fcea81059c0 0x7fcea81160e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:37176/0 (socket says 172.21.15.18:37176) 2024-04-12T21:08:00.682 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:08:00.684+0000 7fcea6a00700 1 -- 172.21.15.18:0/877514391 learned_addr learned my addr 172.21.15.18:0/877514391 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:08:00.683 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:08:00.685+0000 7fcea7400700 1 --2- 172.21.15.18:0/877514391 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcea8105020 0x7fcea8115ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:08:00.683 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:08:00.685+0000 7fcea6a00700 1 -- 172.21.15.18:0/877514391 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcea8105020 msgr2=0x7fcea8115ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:08:00.683 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:08:00.685+0000 7fcea6a00700 1 --2- 172.21.15.18:0/877514391 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcea8105020 0x7fcea8115ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:08:00.683 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:08:00.685+0000 7fcea6a00700 1 -- 172.21.15.18:0/877514391 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce94005040 con 0x7fcea81059c0 2024-04-12T21:08:00.683 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:08:00.685+0000 7fcea6a00700 1 --2- 172.21.15.18:0/877514391 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fcea81059c0 0x7fcea81160e0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fce94009100 tx=0x7fce94009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:08:00.683 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:08:00.685+0000 7fce9f400700 1 -- 172.21.15.18:0/877514391 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fce9400e040 con 0x7fcea81059c0 2024-04-12T21:08:00.683 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:08:00.685+0000 7fce9f400700 1 -- 172.21.15.18:0/877514391 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fce94013420 con 0x7fcea81059c0 2024-04-12T21:08:00.683 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:08:00.685+0000 7fce9f400700 1 -- 172.21.15.18:0/877514391 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fce9401c630 con 0x7fcea81059c0 2024-04-12T21:08:00.683 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:08:00.685+0000 7fceae800700 1 -- 172.21.15.18:0/877514391 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcea810ff90 con 0x7fcea81059c0 2024-04-12T21:08:00.683 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:08:00.686+0000 7fceae800700 1 -- 172.21.15.18:0/877514391 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcea8110550 con 0x7fcea81059c0 2024-04-12T21:08:00.684 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:08:00.687+0000 7fceae800700 1 -- 172.21.15.18:0/877514391 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcea8100770 con 0x7fcea81059c0 2024-04-12T21:08:00.684 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:08:00.688+0000 7fce9f400700 1 -- 172.21.15.18:0/877514391 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7fce94013590 con 0x7fcea81059c0 2024-04-12T21:08:00.684 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:08:00.689+0000 7fce9f400700 1 --2- 172.21.15.18:0/877514391 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fce8c06ec00 0x7fce8c0710b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:08:00.684 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:08:00.689+0000 7fce9f400700 1 -- 172.21.15.18:0/877514391 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 6757+0+0 (secure 0 0 0) 0x7fce9408fba0 con 0x7fcea81059c0 2024-04-12T21:08:00.685 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:08:00.689+0000 7fcea7400700 1 --2- 172.21.15.18:0/877514391 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fce8c06ec00 0x7fce8c0710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:08:00.685 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:08:00.690+0000 7fcea7400700 1 --2- 172.21.15.18:0/877514391 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fce8c06ec00 0x7fce8c0710b0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fce98004d60 tx=0x7fce9800b040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:08:00.689 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:08:00.694+0000 7fce9f400700 1 -- 172.21.15.18:0/877514391 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7fce9405c3d0 con 0x7fcea81059c0 2024-04-12T21:08:00.850 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:08:00 smithi018 ceph-mon[112590]: from='client.34472 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:08:00.850 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:08:00 smithi018 ceph-mon[112590]: from='client.34476 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:08:00.850 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:08:00 smithi018 ceph-mon[112590]: from='client.34480 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:08:00.850 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:08:00 smithi018 ceph-mon[112590]: from='client.? 172.21.15.18:0/1622763686' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T21:08:00.850 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:08:00 smithi018 ceph-mon[112590]: from='client.? 172.21.15.18:0/3822239847' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-12T21:08:00.934 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:08:00.938+0000 7fceae800700 1 -- 172.21.15.18:0/877514391 --> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fcea810c170 con 0x7fce8c06ec00 2024-04-12T21:08:00.937 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:08:00.941+0000 7fce9f400700 1 -- 172.21.15.18:0/877514391 <== mgr.34102 v2:172.21.15.18:6800/3812891919 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7fcea810c170 con 0x7fce8c06ec00 2024-04-12T21:08:00.937 INFO:teuthology.orchestra.run.smithi018.stdout:{ 2024-04-12T21:08:00.937 INFO:teuthology.orchestra.run.smithi018.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:f4452def3be4d350f4a8e5feaf7ebb1511be9298fe68478ba7a2d6fbd7d69a63", 2024-04-12T21:08:00.937 INFO:teuthology.orchestra.run.smithi018.stdout: "in_progress": true, 2024-04-12T21:08:00.937 INFO:teuthology.orchestra.run.smithi018.stdout: "which": "Upgrading all daemon types on all hosts", 2024-04-12T21:08:00.937 INFO:teuthology.orchestra.run.smithi018.stdout: "services_complete": [ 2024-04-12T21:08:00.938 INFO:teuthology.orchestra.run.smithi018.stdout: "crash", 2024-04-12T21:08:00.938 INFO:teuthology.orchestra.run.smithi018.stdout: "osd", 2024-04-12T21:08:00.938 INFO:teuthology.orchestra.run.smithi018.stdout: "mon", 2024-04-12T21:08:00.938 INFO:teuthology.orchestra.run.smithi018.stdout: "mgr" 2024-04-12T21:08:00.938 INFO:teuthology.orchestra.run.smithi018.stdout: ], 2024-04-12T21:08:00.938 INFO:teuthology.orchestra.run.smithi018.stdout: "progress": "14/21 daemons upgraded", 2024-04-12T21:08:00.938 INFO:teuthology.orchestra.run.smithi018.stdout: "message": "Currently upgrading mds daemons", 2024-04-12T21:08:00.938 INFO:teuthology.orchestra.run.smithi018.stdout: "is_paused": false 2024-04-12T21:08:00.938 INFO:teuthology.orchestra.run.smithi018.stdout:} 2024-04-12T21:08:00.940 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:08:00.944+0000 7fceae800700 1 -- 172.21.15.18:0/877514391 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fce8c06ec00 msgr2=0x7fce8c0710b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:08:00.940 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:08:00.944+0000 7fceae800700 1 --2- 172.21.15.18:0/877514391 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fce8c06ec00 0x7fce8c0710b0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fce98004d60 tx=0x7fce9800b040 comp rx=0 tx=0).stop 2024-04-12T21:08:00.940 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:08:00.944+0000 7fceae800700 1 -- 172.21.15.18:0/877514391 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fcea81059c0 msgr2=0x7fcea81160e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:08:00.940 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:08:00.944+0000 7fceae800700 1 --2- 172.21.15.18:0/877514391 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fcea81059c0 0x7fcea81160e0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fce94009100 tx=0x7fce94009e30 comp rx=0 tx=0).stop 2024-04-12T21:08:00.940 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:08:00.944+0000 7fceae800700 1 -- 172.21.15.18:0/877514391 shutdown_connections 2024-04-12T21:08:00.940 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:08:00.944+0000 7fceae800700 1 --2- 172.21.15.18:0/877514391 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fce8c06ec00 0x7fce8c0710b0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:08:00.940 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:08:00.944+0000 7fceae800700 1 --2- 172.21.15.18:0/877514391 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcea8105020 0x7fcea8115ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:08:00.940 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:08:00.944+0000 7fceae800700 1 --2- 172.21.15.18:0/877514391 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fcea81059c0 0x7fcea81160e0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:08:00.940 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:08:00.944+0000 7fceae800700 1 -- 172.21.15.18:0/877514391 >> 172.21.15.18:0/877514391 conn(0x7fcea80faba0 msgr2=0x7fcea80fbb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:08:00.940 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:08:00.944+0000 7fceae800700 1 -- 172.21.15.18:0/877514391 shutdown_connections 2024-04-12T21:08:00.941 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:08:00.944+0000 7fceae800700 1 -- 172.21.15.18:0/877514391 wait complete. 2024-04-12T21:08:01.029 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:08:00 smithi186 ceph-mon[86379]: from='client.34472 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:08:01.029 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:08:00 smithi186 ceph-mon[86379]: from='client.34476 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:08:01.029 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:08:00 smithi186 ceph-mon[86379]: from='client.34480 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:08:01.030 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:08:00 smithi186 ceph-mon[86379]: from='client.? 172.21.15.18:0/1622763686' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T21:08:01.030 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:08:00 smithi186 ceph-mon[86379]: from='client.? 172.21.15.18:0/3822239847' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-12T21:08:01.099 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:08:01.101+0000 7f5355000700 1 -- 172.21.15.18:0/2911336552 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f53500656a0 msgr2=0x7f5350065af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:08:01.099 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:08:01.101+0000 7f5355000700 1 --2- 172.21.15.18:0/2911336552 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f53500656a0 0x7f5350065af0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f533c004940 tx=0x7f533c009e30 comp rx=0 tx=0).stop 2024-04-12T21:08:01.099 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:08:01.101+0000 7f5355000700 1 -- 172.21.15.18:0/2911336552 shutdown_connections 2024-04-12T21:08:01.099 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:08:01.101+0000 7f5355000700 1 --2- 172.21.15.18:0/2911336552 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f53500656a0 0x7f5350065af0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:08:01.099 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:08:01.101+0000 7f5355000700 1 --2- 172.21.15.18:0/2911336552 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5350102de0 0x7f53501031b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:08:01.099 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:08:01.101+0000 7f5355000700 1 -- 172.21.15.18:0/2911336552 >> 172.21.15.18:0/2911336552 conn(0x7f5350072bc0 msgr2=0x7f5350072fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:08:01.099 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:08:01.102+0000 7f5355000700 1 -- 172.21.15.18:0/2911336552 shutdown_connections 2024-04-12T21:08:01.099 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:08:01.102+0000 7f5355000700 1 -- 172.21.15.18:0/2911336552 wait complete. 2024-04-12T21:08:01.099 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:08:01.102+0000 7f5355000700 1 Processor -- start 2024-04-12T21:08:01.101 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:08:01.103+0000 7f5355000700 1 -- start start 2024-04-12T21:08:01.101 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:08:01.103+0000 7f5355000700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f53500656a0 0x7f5350115d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:08:01.101 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:08:01.103+0000 7f5355000700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5350102de0 0x7f5350116280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:08:01.101 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:08:01.103+0000 7f5355000700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53501168d0 con 0x7f5350102de0 2024-04-12T21:08:01.101 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:08:01.103+0000 7f5355000700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f535010fca0 con 0x7f53500656a0 2024-04-12T21:08:01.101 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:08:01.103+0000 7f534d400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5350102de0 0x7f5350116280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:08:01.101 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:08:01.103+0000 7f534d400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5350102de0 0x7f5350116280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:60052/0 (socket says 172.21.15.18:60052) 2024-04-12T21:08:01.101 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:08:01.103+0000 7f534d400700 1 -- 172.21.15.18:0/1132515926 learned_addr learned my addr 172.21.15.18:0/1132515926 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:08:01.101 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:08:01.104+0000 7f534de00700 1 --2- 172.21.15.18:0/1132515926 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f53500656a0 0x7f5350115d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:08:01.101 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:08:01.104+0000 7f534d400700 1 -- 172.21.15.18:0/1132515926 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f53500656a0 msgr2=0x7f5350115d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:08:01.101 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:08:01.104+0000 7f534d400700 1 --2- 172.21.15.18:0/1132515926 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f53500656a0 0x7f5350115d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:08:01.101 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:08:01.104+0000 7f534d400700 1 -- 172.21.15.18:0/1132515926 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f533c005040 con 0x7f5350102de0 2024-04-12T21:08:01.101 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:08:01.104+0000 7f534d400700 1 --2- 172.21.15.18:0/1132515926 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5350102de0 0x7f5350116280 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f533c009180 tx=0x7f533c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:08:01.102 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:08:01.105+0000 7f5346000700 1 -- 172.21.15.18:0/1132515926 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f533c00e040 con 0x7f5350102de0 2024-04-12T21:08:01.102 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:08:01.105+0000 7f5346000700 1 -- 172.21.15.18:0/1132515926 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f533c00bbc0 con 0x7f5350102de0 2024-04-12T21:08:01.102 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:08:01.105+0000 7f5355000700 1 -- 172.21.15.18:0/1132515926 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f535010ff20 con 0x7f5350102de0 2024-04-12T21:08:01.103 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:08:01.105+0000 7f5346000700 1 -- 172.21.15.18:0/1132515926 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f533c01c630 con 0x7f5350102de0 2024-04-12T21:08:01.103 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:08:01.105+0000 7f5355000700 1 -- 172.21.15.18:0/1132515926 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f53501104e0 con 0x7f5350102de0 2024-04-12T21:08:01.103 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:08:01.106+0000 7f5355000700 1 -- 172.21.15.18:0/1132515926 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f53501017a0 con 0x7f5350102de0 2024-04-12T21:08:01.103 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:08:01.107+0000 7f5346000700 1 -- 172.21.15.18:0/1132515926 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7f533c00bd30 con 0x7f5350102de0 2024-04-12T21:08:01.104 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:08:01.108+0000 7f5346000700 1 --2- 172.21.15.18:0/1132515926 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f533406ec10 0x7f53340710c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:08:01.105 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:08:01.108+0000 7f5346000700 1 -- 172.21.15.18:0/1132515926 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 6757+0+0 (secure 0 0 0) 0x7f533c08fd40 con 0x7f5350102de0 2024-04-12T21:08:01.105 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:08:01.108+0000 7f534de00700 1 --2- 172.21.15.18:0/1132515926 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f533406ec10 0x7f53340710c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:08:01.105 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:08:01.109+0000 7f534de00700 1 --2- 172.21.15.18:0/1132515926 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f533406ec10 0x7f53340710c0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f5340007fd0 tx=0x7f5340009040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:08:01.109 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:08:01.112+0000 7f5346000700 1 -- 172.21.15.18:0/1132515926 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f533c05c570 con 0x7f5350102de0 2024-04-12T21:08:01.420 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:08:01.423+0000 7f5355000700 1 -- 172.21.15.18:0/1132515926 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f5350111390 con 0x7f5350102de0 2024-04-12T21:08:01.420 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:08:01.424+0000 7f5346000700 1 -- 172.21.15.18:0/1132515926 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+233 (secure 0 0 0) 0x7f533c05c100 con 0x7f5350102de0 2024-04-12T21:08:01.421 INFO:teuthology.orchestra.run.smithi018.stdout:HEALTH_ERR 1 filesystem is degraded; 1 filesystem is offline 2024-04-12T21:08:01.421 INFO:teuthology.orchestra.run.smithi018.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-04-12T21:08:01.421 INFO:teuthology.orchestra.run.smithi018.stdout: fs cephfs is degraded 2024-04-12T21:08:01.421 INFO:teuthology.orchestra.run.smithi018.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-04-12T21:08:01.421 INFO:teuthology.orchestra.run.smithi018.stdout: fs cephfs is offline because no MDS is active for it. 2024-04-12T21:08:01.423 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:08:01.426+0000 7f5355000700 1 -- 172.21.15.18:0/1132515926 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f533406ec10 msgr2=0x7f53340710c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:08:01.424 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:08:01.427+0000 7f5355000700 1 --2- 172.21.15.18:0/1132515926 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f533406ec10 0x7f53340710c0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f5340007fd0 tx=0x7f5340009040 comp rx=0 tx=0).stop 2024-04-12T21:08:01.424 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:08:01.427+0000 7f5355000700 1 -- 172.21.15.18:0/1132515926 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5350102de0 msgr2=0x7f5350116280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:08:01.424 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:08:01.427+0000 7f5355000700 1 --2- 172.21.15.18:0/1132515926 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5350102de0 0x7f5350116280 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f533c009180 tx=0x7f533c009e30 comp rx=0 tx=0).stop 2024-04-12T21:08:01.424 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:08:01.427+0000 7f5355000700 1 -- 172.21.15.18:0/1132515926 shutdown_connections 2024-04-12T21:08:01.424 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:08:01.427+0000 7f5355000700 1 --2- 172.21.15.18:0/1132515926 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f533406ec10 0x7f53340710c0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:08:01.424 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:08:01.427+0000 7f5355000700 1 --2- 172.21.15.18:0/1132515926 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f53500656a0 0x7f5350115d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:08:01.424 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:08:01.427+0000 7f5355000700 1 --2- 172.21.15.18:0/1132515926 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5350102de0 0x7f5350116280 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:08:01.424 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:08:01.427+0000 7f5355000700 1 -- 172.21.15.18:0/1132515926 >> 172.21.15.18:0/1132515926 conn(0x7f5350072bc0 msgr2=0x7f53500fe920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:08:01.424 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:08:01.427+0000 7f5355000700 1 -- 172.21.15.18:0/1132515926 shutdown_connections 2024-04-12T21:08:01.425 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:08:01.427+0000 7f5355000700 1 -- 172.21.15.18:0/1132515926 wait complete. 2024-04-12T21:08:01.850 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:08:01 smithi018 ceph-mon[112590]: pgmap v246: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:08:01.850 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:08:01 smithi018 ceph-mon[112590]: from='client.34492 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:08:01.850 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:08:01 smithi018 ceph-mon[112590]: from='client.? 172.21.15.18:0/1132515926' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-12T21:08:02.029 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:08:01 smithi186 ceph-mon[86379]: pgmap v246: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:08:02.029 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:08:01 smithi186 ceph-mon[86379]: from='client.34492 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:08:02.029 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:08:01 smithi186 ceph-mon[86379]: from='client.? 172.21.15.18:0/1132515926' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-12T21:08:02.850 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:08:02 smithi018 ceph-mon[112590]: pgmap v247: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:08:03.029 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:08:02 smithi186 ceph-mon[86379]: pgmap v247: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:08:05.279 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:08:04 smithi186 ceph-mon[86379]: pgmap v248: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:08:05.350 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:08:04 smithi018 ceph-mon[112590]: pgmap v248: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:08:07.279 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:08:06 smithi186 ceph-mon[86379]: pgmap v249: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:08:07.350 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:08:06 smithi018 ceph-mon[112590]: pgmap v249: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:08:09.279 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:08:08 smithi186 ceph-mon[86379]: pgmap v250: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:08:09.350 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:08:08 smithi018 ceph-mon[112590]: pgmap v250: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:08:10.252 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:08:09 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T21:08:10.279 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:08:09 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T21:08:11.279 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:08:10 smithi186 ceph-mon[86379]: pgmap v251: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:08:11.286 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:08:10 smithi018 ceph-mon[112590]: pgmap v251: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:08:13.279 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:08:13 smithi186 ceph-mon[86379]: pgmap v252: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:08:13.350 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:08:12 smithi018 ceph-mon[112590]: pgmap v252: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:08:15.279 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:08:15 smithi186 ceph-mon[86379]: pgmap v253: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:08:15.350 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:08:15 smithi018 ceph-mon[112590]: pgmap v253: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:08:17.279 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:08:17 smithi186 ceph-mon[86379]: pgmap v254: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:08:17.350 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:08:17 smithi018 ceph-mon[112590]: pgmap v254: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:08:19.279 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:08:19 smithi186 ceph-mon[86379]: pgmap v255: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:08:19.350 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:08:19 smithi018 ceph-mon[112590]: pgmap v255: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:08:21.279 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:08:20 smithi186 ceph-mon[86379]: pgmap v256: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:08:21.350 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:08:20 smithi018 ceph-mon[112590]: pgmap v256: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:08:23.279 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:08:22 smithi186 ceph-mon[86379]: pgmap v257: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:08:23.350 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:08:22 smithi018 ceph-mon[112590]: pgmap v257: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:08:25.279 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:08:24 smithi186 ceph-mon[86379]: pgmap v258: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:08:25.279 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:08:24 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T21:08:25.350 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:08:24 smithi018 ceph-mon[112590]: pgmap v258: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:08:25.350 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:08:24 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T21:08:27.279 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:08:26 smithi186 ceph-mon[86379]: pgmap v259: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:08:27.350 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:08:26 smithi018 ceph-mon[112590]: pgmap v259: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:08:29.279 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:08:28 smithi186 ceph-mon[86379]: pgmap v260: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:08:29.350 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:08:28 smithi018 ceph-mon[112590]: pgmap v260: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:08:31.279 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:08:31 smithi186 ceph-mon[86379]: pgmap v261: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:08:31.350 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:08:31 smithi018 ceph-mon[112590]: pgmap v261: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:08:31.600 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:08:31.602+0000 7f6a7f200700 1 -- 172.21.15.18:0/4089735623 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6a80070bf0 msgr2=0x7f6a80108fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:08:31.600 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:08:31.602+0000 7f6a7f200700 1 --2- 172.21.15.18:0/4089735623 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6a80070bf0 0x7f6a80108fc0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f6a6c004940 tx=0x7f6a6c009e30 comp rx=0 tx=0).stop 2024-04-12T21:08:31.600 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:08:31.602+0000 7f6a7f200700 1 -- 172.21.15.18:0/4089735623 shutdown_connections 2024-04-12T21:08:31.600 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:08:31.602+0000 7f6a7f200700 1 --2- 172.21.15.18:0/4089735623 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6a80070bf0 0x7f6a80108fc0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:08:31.600 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:08:31.602+0000 7f6a7f200700 1 --2- 172.21.15.18:0/4089735623 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6a800702e0 0x7f6a800706b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:08:31.600 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:08:31.602+0000 7f6a7f200700 1 -- 172.21.15.18:0/4089735623 >> 172.21.15.18:0/4089735623 conn(0x7f6a800f8760 msgr2=0x7f6a800fab70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:08:31.601 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:08:31.602+0000 7f6a7f200700 1 -- 172.21.15.18:0/4089735623 shutdown_connections 2024-04-12T21:08:31.601 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:08:31.602+0000 7f6a7f200700 1 -- 172.21.15.18:0/4089735623 wait complete. 2024-04-12T21:08:31.601 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:08:31.603+0000 7f6a7f200700 1 Processor -- start 2024-04-12T21:08:31.601 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:08:31.603+0000 7f6a7f200700 1 -- start start 2024-04-12T21:08:31.603 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:08:31.604+0000 7f6a7f200700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6a800702e0 0x7f6a80115c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:08:31.604 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:08:31.604+0000 7f6a7f200700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6a80070bf0 0x7f6a80116160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:08:31.604 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:08:31.604+0000 7f6a7f200700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a801167b0 con 0x7f6a800702e0 2024-04-12T21:08:31.604 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:08:31.604+0000 7f6a7f200700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a8010fba0 con 0x7f6a80070bf0 2024-04-12T21:08:31.604 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:08:31.604+0000 7f6a7de00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6a800702e0 0x7f6a80115c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:08:31.604 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:08:31.604+0000 7f6a7de00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6a800702e0 0x7f6a80115c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:59168/0 (socket says 172.21.15.18:59168) 2024-04-12T21:08:31.604 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:08:31.604+0000 7f6a7de00700 1 -- 172.21.15.18:0/3908812566 learned_addr learned my addr 172.21.15.18:0/3908812566 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:08:31.604 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:08:31.605+0000 7f6a7d400700 1 --2- 172.21.15.18:0/3908812566 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6a80070bf0 0x7f6a80116160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:08:31.604 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:08:31.605+0000 7f6a7de00700 1 -- 172.21.15.18:0/3908812566 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6a80070bf0 msgr2=0x7f6a80116160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:08:31.604 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:08:31.605+0000 7f6a7de00700 1 --2- 172.21.15.18:0/3908812566 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6a80070bf0 0x7f6a80116160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:08:31.604 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:08:31.605+0000 7f6a7de00700 1 -- 172.21.15.18:0/3908812566 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a6c005040 con 0x7f6a800702e0 2024-04-12T21:08:31.604 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:08:31.605+0000 7f6a7de00700 1 --2- 172.21.15.18:0/3908812566 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6a800702e0 0x7f6a80115c20 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f6a7000c830 tx=0x7f6a70004470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:08:31.604 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:08:31.605+0000 7f6a76000700 1 -- 172.21.15.18:0/3908812566 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6a7000a760 con 0x7f6a800702e0 2024-04-12T21:08:31.604 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:08:31.605+0000 7f6a76000700 1 -- 172.21.15.18:0/3908812566 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f6a7000ad90 con 0x7f6a800702e0 2024-04-12T21:08:31.604 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:08:31.605+0000 7f6a76000700 1 -- 172.21.15.18:0/3908812566 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6a700113a0 con 0x7f6a800702e0 2024-04-12T21:08:31.605 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:08:31.605+0000 7f6a7f200700 1 -- 172.21.15.18:0/3908812566 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6a8010fe80 con 0x7f6a800702e0 2024-04-12T21:08:31.605 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:08:31.606+0000 7f6a7f200700 1 -- 172.21.15.18:0/3908812566 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6a801103c0 con 0x7f6a800702e0 2024-04-12T21:08:31.605 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:08:31.607+0000 7f6a7f200700 1 -- 172.21.15.18:0/3908812566 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6a80071a20 con 0x7f6a800702e0 2024-04-12T21:08:31.605 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:08:31.608+0000 7f6a76000700 1 -- 172.21.15.18:0/3908812566 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7f6a70012400 con 0x7f6a800702e0 2024-04-12T21:08:31.606 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:08:31.609+0000 7f6a76000700 1 --2- 172.21.15.18:0/3908812566 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f6a6406ec00 0x7f6a640710b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:08:31.606 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:08:31.609+0000 7f6a76000700 1 -- 172.21.15.18:0/3908812566 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 6757+0+0 (secure 0 0 0) 0x7f6a7008e9c0 con 0x7f6a800702e0 2024-04-12T21:08:31.606 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:08:31.609+0000 7f6a7d400700 1 --2- 172.21.15.18:0/3908812566 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f6a6406ec00 0x7f6a640710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:08:31.606 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:08:31.610+0000 7f6a7d400700 1 --2- 172.21.15.18:0/3908812566 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f6a6406ec00 0x7f6a640710b0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f6a6c00b020 tx=0x7f6a6c009e30 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:08:31.611 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:08:31.614+0000 7f6a76000700 1 -- 172.21.15.18:0/3908812566 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f6a7005b1f0 con 0x7f6a800702e0 2024-04-12T21:08:31.863 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:08:31.866+0000 7f6a7f200700 1 -- 172.21.15.18:0/3908812566 --> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f6a8004bf40 con 0x7f6a6406ec00 2024-04-12T21:08:31.864 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:08:31.868+0000 7f6a76000700 1 -- 172.21.15.18:0/3908812566 <== mgr.34102 v2:172.21.15.18:6800/3812891919 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f6a8004bf40 con 0x7f6a6406ec00 2024-04-12T21:08:31.868 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:08:31.870+0000 7f6a7f200700 1 -- 172.21.15.18:0/3908812566 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f6a6406ec00 msgr2=0x7f6a640710b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:08:31.868 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:08:31.870+0000 7f6a7f200700 1 --2- 172.21.15.18:0/3908812566 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f6a6406ec00 0x7f6a640710b0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f6a6c00b020 tx=0x7f6a6c009e30 comp rx=0 tx=0).stop 2024-04-12T21:08:31.868 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:08:31.870+0000 7f6a7f200700 1 -- 172.21.15.18:0/3908812566 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6a800702e0 msgr2=0x7f6a80115c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:08:31.868 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:08:31.870+0000 7f6a7f200700 1 --2- 172.21.15.18:0/3908812566 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6a800702e0 0x7f6a80115c20 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f6a7000c830 tx=0x7f6a70004470 comp rx=0 tx=0).stop 2024-04-12T21:08:31.868 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:08:31.870+0000 7f6a7f200700 1 -- 172.21.15.18:0/3908812566 shutdown_connections 2024-04-12T21:08:31.868 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:08:31.870+0000 7f6a7f200700 1 --2- 172.21.15.18:0/3908812566 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f6a6406ec00 0x7f6a640710b0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:08:31.868 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:08:31.870+0000 7f6a7f200700 1 --2- 172.21.15.18:0/3908812566 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6a800702e0 0x7f6a80115c20 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:08:31.868 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:08:31.870+0000 7f6a7f200700 1 --2- 172.21.15.18:0/3908812566 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6a80070bf0 0x7f6a80116160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:08:31.869 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:08:31.870+0000 7f6a7f200700 1 -- 172.21.15.18:0/3908812566 >> 172.21.15.18:0/3908812566 conn(0x7f6a800f8760 msgr2=0x7f6a801058e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:08:31.869 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:08:31.871+0000 7f6a7f200700 1 -- 172.21.15.18:0/3908812566 shutdown_connections 2024-04-12T21:08:31.869 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:08:31.871+0000 7f6a7f200700 1 -- 172.21.15.18:0/3908812566 wait complete. 2024-04-12T21:08:31.884 INFO:teuthology.orchestra.run.smithi018.stdout:true 2024-04-12T21:08:32.029 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:08:32.031+0000 7f43c8800700 1 -- 172.21.15.18:0/3741988609 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f43c00724f0 msgr2=0x7f43c00728c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:08:32.029 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:08:32.031+0000 7f43c8800700 1 --2- 172.21.15.18:0/3741988609 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f43c00724f0 0x7f43c00728c0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f43b4004940 tx=0x7f43b4009e30 comp rx=0 tx=0).stop 2024-04-12T21:08:32.029 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:08:32.031+0000 7f43c8800700 1 -- 172.21.15.18:0/3741988609 shutdown_connections 2024-04-12T21:08:32.029 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:08:32.031+0000 7f43c8800700 1 --2- 172.21.15.18:0/3741988609 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f43c0072e00 0x7f43c010d770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:08:32.029 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:08:32.031+0000 7f43c8800700 1 --2- 172.21.15.18:0/3741988609 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f43c00724f0 0x7f43c00728c0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:08:32.029 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:08:32.031+0000 7f43c8800700 1 -- 172.21.15.18:0/3741988609 >> 172.21.15.18:0/3741988609 conn(0x7f43c00fab70 msgr2=0x7f43c00fcf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:08:32.029 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:08:32.032+0000 7f43c8800700 1 -- 172.21.15.18:0/3741988609 shutdown_connections 2024-04-12T21:08:32.029 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:08:32.032+0000 7f43c8800700 1 -- 172.21.15.18:0/3741988609 wait complete. 2024-04-12T21:08:32.031 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:08:32.033+0000 7f43c8800700 1 Processor -- start 2024-04-12T21:08:32.031 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:08:32.033+0000 7f43c8800700 1 -- start start 2024-04-12T21:08:32.031 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:08:32.033+0000 7f43c8800700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f43c00724f0 0x7f43c0119ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:08:32.031 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:08:32.033+0000 7f43c8800700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f43c0072e00 0x7f43c011a420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:08:32.031 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:08:32.033+0000 7f43c8800700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43c011aa70 con 0x7f43c00724f0 2024-04-12T21:08:32.031 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:08:32.033+0000 7f43c8800700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43c0114050 con 0x7f43c0072e00 2024-04-12T21:08:32.031 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:08:32.034+0000 7f43c5e00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f43c00724f0 0x7f43c0119ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:08:32.031 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:08:32.034+0000 7f43c5e00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f43c00724f0 0x7f43c0119ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:59200/0 (socket says 172.21.15.18:59200) 2024-04-12T21:08:32.031 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:08:32.034+0000 7f43c5e00700 1 -- 172.21.15.18:0/1044375730 learned_addr learned my addr 172.21.15.18:0/1044375730 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:08:32.031 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:08:32.034+0000 7f43c5e00700 1 -- 172.21.15.18:0/1044375730 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f43c0072e00 msgr2=0x7f43c011a420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:08:32.031 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:08:32.034+0000 7f43c5400700 1 --2- 172.21.15.18:0/1044375730 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f43c0072e00 0x7f43c011a420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:08:32.032 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:08:32.034+0000 7f43c5e00700 1 --2- 172.21.15.18:0/1044375730 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f43c0072e00 0x7f43c011a420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:08:32.032 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:08:32.034+0000 7f43c5e00700 1 -- 172.21.15.18:0/1044375730 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43b4005040 con 0x7f43c00724f0 2024-04-12T21:08:32.032 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:08:32.034+0000 7f43c5400700 1 --2- 172.21.15.18:0/1044375730 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f43c0072e00 0x7f43c011a420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-12T21:08:32.032 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:08:32.035+0000 7f43c5e00700 1 --2- 172.21.15.18:0/1044375730 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f43c00724f0 0x7f43c0119ee0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f43b4004890 tx=0x7f43b4009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:08:32.033 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:08:32.035+0000 7f43ba000700 1 -- 172.21.15.18:0/1044375730 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f43b400e040 con 0x7f43c00724f0 2024-04-12T21:08:32.033 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:08:32.035+0000 7f43ba000700 1 -- 172.21.15.18:0/1044375730 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f43b4009af0 con 0x7f43c00724f0 2024-04-12T21:08:32.033 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:08:32.035+0000 7f43ba000700 1 -- 172.21.15.18:0/1044375730 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f43b401c630 con 0x7f43c00724f0 2024-04-12T21:08:32.033 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:08:32.035+0000 7f43c8800700 1 -- 172.21.15.18:0/1044375730 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f43c01142d0 con 0x7f43c00724f0 2024-04-12T21:08:32.033 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:08:32.035+0000 7f43c8800700 1 -- 172.21.15.18:0/1044375730 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f43c0114890 con 0x7f43c00724f0 2024-04-12T21:08:32.033 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:08:32.036+0000 7f43c8800700 1 -- 172.21.15.18:0/1044375730 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f43c010a5b0 con 0x7f43c00724f0 2024-04-12T21:08:32.034 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:08:32.037+0000 7f43ba000700 1 -- 172.21.15.18:0/1044375730 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7f43b400bbc0 con 0x7f43c00724f0 2024-04-12T21:08:32.034 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:08:32.038+0000 7f43ba000700 1 --2- 172.21.15.18:0/1044375730 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f43a806ec00 0x7f43a80710b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:08:32.035 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:08:32.038+0000 7f43ba000700 1 -- 172.21.15.18:0/1044375730 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 6757+0+0 (secure 0 0 0) 0x7f43b408fc10 con 0x7f43c00724f0 2024-04-12T21:08:32.035 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:08:32.039+0000 7f43c5400700 1 --2- 172.21.15.18:0/1044375730 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f43a806ec00 0x7f43a80710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:08:32.036 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:08:32.039+0000 7f43c5400700 1 --2- 172.21.15.18:0/1044375730 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f43a806ec00 0x7f43a80710b0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f43b0007fd0 tx=0x7f43b0009040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:08:32.040 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:08:32.043+0000 7f43ba000700 1 -- 172.21.15.18:0/1044375730 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f43b405c500 con 0x7f43c00724f0 2024-04-12T21:08:32.281 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:08:32.284+0000 7f43c8800700 1 -- 172.21.15.18:0/1044375730 --> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f43c01104e0 con 0x7f43a806ec00 2024-04-12T21:08:32.283 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:08:32.286+0000 7f43ba000700 1 -- 172.21.15.18:0/1044375730 <== mgr.34102 v2:172.21.15.18:6800/3812891919 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f43c01104e0 con 0x7f43a806ec00 2024-04-12T21:08:32.286 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:08:32.289+0000 7f43c8800700 1 -- 172.21.15.18:0/1044375730 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f43a806ec00 msgr2=0x7f43a80710b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:08:32.286 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:08:32.289+0000 7f43c8800700 1 --2- 172.21.15.18:0/1044375730 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f43a806ec00 0x7f43a80710b0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f43b0007fd0 tx=0x7f43b0009040 comp rx=0 tx=0).stop 2024-04-12T21:08:32.286 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:08:32.289+0000 7f43c8800700 1 -- 172.21.15.18:0/1044375730 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f43c00724f0 msgr2=0x7f43c0119ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:08:32.287 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:08:32.289+0000 7f43c8800700 1 --2- 172.21.15.18:0/1044375730 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f43c00724f0 0x7f43c0119ee0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f43b4004890 tx=0x7f43b4009e30 comp rx=0 tx=0).stop 2024-04-12T21:08:32.287 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:08:32.289+0000 7f43c8800700 1 -- 172.21.15.18:0/1044375730 shutdown_connections 2024-04-12T21:08:32.287 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:08:32.289+0000 7f43c8800700 1 --2- 172.21.15.18:0/1044375730 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f43a806ec00 0x7f43a80710b0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:08:32.287 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:08:32.289+0000 7f43c8800700 1 --2- 172.21.15.18:0/1044375730 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f43c00724f0 0x7f43c0119ee0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:08:32.287 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:08:32.289+0000 7f43c8800700 1 --2- 172.21.15.18:0/1044375730 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f43c0072e00 0x7f43c011a420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:08:32.287 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:08:32.289+0000 7f43c8800700 1 -- 172.21.15.18:0/1044375730 >> 172.21.15.18:0/1044375730 conn(0x7f43c00fab70 msgr2=0x7f43c0107730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:08:32.287 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:08:32.290+0000 7f43c8800700 1 -- 172.21.15.18:0/1044375730 shutdown_connections 2024-04-12T21:08:32.287 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:08:32.290+0000 7f43c8800700 1 -- 172.21.15.18:0/1044375730 wait complete. 2024-04-12T21:08:32.439 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:08:32.442+0000 7effc9a00700 1 -- 172.21.15.18:0/1593721036 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7effc4109dd0 msgr2=0x7effc4103440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:08:32.439 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:08:32.442+0000 7effc9a00700 1 --2- 172.21.15.18:0/1593721036 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7effc4109dd0 0x7effc4103440 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7effb0004940 tx=0x7effb0009e30 comp rx=0 tx=0).stop 2024-04-12T21:08:32.439 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:08:32.442+0000 7effc9a00700 1 -- 172.21.15.18:0/1593721036 shutdown_connections 2024-04-12T21:08:32.439 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:08:32.442+0000 7effc9a00700 1 --2- 172.21.15.18:0/1593721036 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7effc4109dd0 0x7effc4103440 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:08:32.440 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:08:32.442+0000 7effc9a00700 1 --2- 172.21.15.18:0/1593721036 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7effc4109430 0x7effc4109800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:08:32.440 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:08:32.442+0000 7effc9a00700 1 -- 172.21.15.18:0/1593721036 >> 172.21.15.18:0/1593721036 conn(0x7effc4072c00 msgr2=0x7effc4075010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:08:32.440 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:08:32.442+0000 7effc9a00700 1 -- 172.21.15.18:0/1593721036 shutdown_connections 2024-04-12T21:08:32.440 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:08:32.442+0000 7effc9a00700 1 -- 172.21.15.18:0/1593721036 wait complete. 2024-04-12T21:08:32.441 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:08:32.443+0000 7effc9a00700 1 Processor -- start 2024-04-12T21:08:32.441 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:08:32.443+0000 7effc9a00700 1 -- start start 2024-04-12T21:08:32.441 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:08:32.444+0000 7effc9a00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7effc4109430 0x7effc4119fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:08:32.441 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:08:32.444+0000 7effc9a00700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7effc4109dd0 0x7effc411a520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:08:32.441 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:08:32.444+0000 7effc9a00700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7effc411ab70 con 0x7effc4109430 2024-04-12T21:08:32.441 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:08:32.444+0000 7effc9a00700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7effc4114150 con 0x7effc4109dd0 2024-04-12T21:08:32.441 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:08:32.444+0000 7effc2a00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7effc4109430 0x7effc4119fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:08:32.441 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:08:32.444+0000 7effc2a00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7effc4109430 0x7effc4119fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:59210/0 (socket says 172.21.15.18:59210) 2024-04-12T21:08:32.441 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:08:32.444+0000 7effc2a00700 1 -- 172.21.15.18:0/3498178242 learned_addr learned my addr 172.21.15.18:0/3498178242 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:08:32.442 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:08:32.445+0000 7effc2000700 1 --2- 172.21.15.18:0/3498178242 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7effc4109dd0 0x7effc411a520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:08:32.442 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:08:32.445+0000 7effc2a00700 1 -- 172.21.15.18:0/3498178242 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7effc4109dd0 msgr2=0x7effc411a520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:08:32.443 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:08:32.445+0000 7effc2a00700 1 --2- 172.21.15.18:0/3498178242 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7effc4109dd0 0x7effc411a520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:08:32.443 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:08:32.445+0000 7effc2a00700 1 -- 172.21.15.18:0/3498178242 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7effb0005040 con 0x7effc4109430 2024-04-12T21:08:32.443 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:08:32.445+0000 7effc2a00700 1 --2- 172.21.15.18:0/3498178242 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7effc4109430 0x7effc4119fe0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7effb400c810 tx=0x7effb4004470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:08:32.443 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:08:32.445+0000 7effbaa00700 1 -- 172.21.15.18:0/3498178242 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7effb4012810 con 0x7effc4109430 2024-04-12T21:08:32.443 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:08:32.445+0000 7effbaa00700 1 -- 172.21.15.18:0/3498178242 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7effb4012e40 con 0x7effc4109430 2024-04-12T21:08:32.443 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:08:32.445+0000 7effc9a00700 1 -- 172.21.15.18:0/3498178242 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7effc4114430 con 0x7effc4109430 2024-04-12T21:08:32.443 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:08:32.445+0000 7effbaa00700 1 -- 172.21.15.18:0/3498178242 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7effb400a370 con 0x7effc4109430 2024-04-12T21:08:32.443 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:08:32.445+0000 7effc9a00700 1 -- 172.21.15.18:0/3498178242 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7effc41149f0 con 0x7effc4109430 2024-04-12T21:08:32.443 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:08:32.447+0000 7effc9a00700 1 -- 172.21.15.18:0/3498178242 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7effc4104b80 con 0x7effc4109430 2024-04-12T21:08:32.444 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:08:32.448+0000 7effbaa00700 1 -- 172.21.15.18:0/3498178242 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7effb4011810 con 0x7effc4109430 2024-04-12T21:08:32.445 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:08:32.449+0000 7effbaa00700 1 --2- 172.21.15.18:0/3498178242 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7effa806ec00 0x7effa80710b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:08:32.445 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:08:32.449+0000 7effbaa00700 1 -- 172.21.15.18:0/3498178242 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 6757+0+0 (secure 0 0 0) 0x7effb408edd0 con 0x7effc4109430 2024-04-12T21:08:32.445 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:08:32.449+0000 7effc2000700 1 --2- 172.21.15.18:0/3498178242 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7effa806ec00 0x7effa80710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:08:32.446 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:08:32.450+0000 7effc2000700 1 --2- 172.21.15.18:0/3498178242 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7effa806ec00 0x7effa80710b0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7effb000b3b0 tx=0x7effb0009e30 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:08:32.450 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:08:32.453+0000 7effbaa00700 1 -- 172.21.15.18:0/3498178242 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7effb405b580 con 0x7effc4109430 2024-04-12T21:08:32.696 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:08:32.699+0000 7effc9a00700 1 -- 172.21.15.18:0/3498178242 --> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7effc41156a0 con 0x7effa806ec00 2024-04-12T21:08:32.707 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:08:32.710+0000 7effbaa00700 1 -- 172.21.15.18:0/3498178242 <== mgr.34102 v2:172.21.15.18:6800/3812891919 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3520 (secure 0 0 0) 0x7effc41156a0 con 0x7effa806ec00 2024-04-12T21:08:32.708 INFO:teuthology.orchestra.run.smithi018.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-12T21:08:32.708 INFO:teuthology.orchestra.run.smithi018.stdout:alertmanager.smithi018 smithi018 *:9093,9094 running (9m) 47s ago 23m 14.7M - 0.25.0 c8568f914cd2 c332090068e8 2024-04-12T21:08:32.708 INFO:teuthology.orchestra.run.smithi018.stdout:crash.smithi018 smithi018 running (5m) 47s ago 23m 6665k - 19.0.0-2879-g324baf8d 8717720dc5e5 b7f80d1f7982 2024-04-12T21:08:32.708 INFO:teuthology.orchestra.run.smithi018.stdout:crash.smithi186 smithi186 running (5m) 102s ago 22m 6660k - 19.0.0-2879-g324baf8d 8717720dc5e5 a4f55c81b623 2024-04-12T21:08:32.708 INFO:teuthology.orchestra.run.smithi018.stdout:grafana.smithi018 smithi018 *:3000 running (8m) 47s ago 23m 74.8M - 9.4.12 2bacad6d85d8 b08e3bf176dd 2024-04-12T21:08:32.708 INFO:teuthology.orchestra.run.smithi018.stdout:mds.cephfs.smithi018.acqbsc smithi018 running (82s) 47s ago 16m 11.1M - 19.0.0-2879-g324baf8d 8717720dc5e5 021f99ab24ae 2024-04-12T21:08:32.708 INFO:teuthology.orchestra.run.smithi018.stdout:mds.cephfs.smithi018.szjluo smithi018 running (54s) 47s ago 16m 10.9M - 19.0.0-2879-g324baf8d 8717720dc5e5 0003117479d4 2024-04-12T21:08:32.708 INFO:teuthology.orchestra.run.smithi018.stdout:mds.cephfs.smithi186.jdllyn smithi186 running (16m) 102s ago 16m 15.6M - 17.2.7-827-gd3b5060a 21de9b7369ad 2b2bc46bd29a 2024-04-12T21:08:32.708 INFO:teuthology.orchestra.run.smithi018.stdout:mds.cephfs.smithi186.wjnafs smithi186 running (16m) 102s ago 16m 15.4M - 17.2.7-827-gd3b5060a 21de9b7369ad 74b115d6d6fd 2024-04-12T21:08:32.708 INFO:teuthology.orchestra.run.smithi018.stdout:mgr.smithi018.xjzwga smithi018 *:8443,9283,8765 running (11m) 47s ago 25m 509M - 19.0.0-2879-g324baf8d 8717720dc5e5 56088305eab7 2024-04-12T21:08:32.708 INFO:teuthology.orchestra.run.smithi018.stdout:mgr.smithi186.kqcfxk smithi186 *:8443,9283,8765 running (10m) 102s ago 22m 452M - 19.0.0-2879-g324baf8d 8717720dc5e5 09cd766e5e0d 2024-04-12T21:08:32.708 INFO:teuthology.orchestra.run.smithi018.stdout:mon.smithi018 smithi018 running (7m) 47s ago 25m 57.0M 2048M 19.0.0-2879-g324baf8d 8717720dc5e5 df3e201dfed6 2024-04-12T21:08:32.708 INFO:teuthology.orchestra.run.smithi018.stdout:mon.smithi186 smithi186 running (6m) 102s ago 22m 37.7M 2048M 19.0.0-2879-g324baf8d 8717720dc5e5 c77498da9b2b 2024-04-12T21:08:32.709 INFO:teuthology.orchestra.run.smithi018.stdout:node-exporter.smithi018 smithi018 *:9100 running (10m) 47s ago 23m 7780k - 1.5.0 0da6a335fe13 326c3d9c6847 2024-04-12T21:08:32.709 INFO:teuthology.orchestra.run.smithi018.stdout:node-exporter.smithi186 smithi186 *:9100 running (10m) 102s ago 22m 7826k - 1.5.0 0da6a335fe13 ec2112baeecf 2024-04-12T21:08:32.709 INFO:teuthology.orchestra.run.smithi018.stdout:osd.0 smithi018 running (4m) 47s ago 21m 121M 1559M 19.0.0-2879-g324baf8d 8717720dc5e5 60b1273ca047 2024-04-12T21:08:32.709 INFO:teuthology.orchestra.run.smithi018.stdout:osd.1 smithi018 running (4m) 47s ago 20m 91.3M 1559M 19.0.0-2879-g324baf8d 8717720dc5e5 170ce1e5fb8d 2024-04-12T21:08:32.709 INFO:teuthology.orchestra.run.smithi018.stdout:osd.2 smithi018 running (3m) 47s ago 20m 90.3M 1559M 19.0.0-2879-g324baf8d 8717720dc5e5 c2cbc6aa2839 2024-04-12T21:08:32.709 INFO:teuthology.orchestra.run.smithi018.stdout:osd.3 smithi186 running (3m) 102s ago 19m 96.6M 2581M 19.0.0-2879-g324baf8d 8717720dc5e5 0138e8262e4b 2024-04-12T21:08:32.709 INFO:teuthology.orchestra.run.smithi018.stdout:osd.4 smithi186 running (2m) 102s ago 18m 87.2M 2581M 19.0.0-2879-g324baf8d 8717720dc5e5 b20cb63b25c9 2024-04-12T21:08:32.709 INFO:teuthology.orchestra.run.smithi018.stdout:osd.5 smithi186 running (107s) 102s ago 18m 11.1M 2581M 19.0.0-2879-g324baf8d 8717720dc5e5 2513bea2610e 2024-04-12T21:08:32.709 INFO:teuthology.orchestra.run.smithi018.stdout:prometheus.smithi018 smithi018 *:9095 running (9m) 47s ago 22m 60.6M - 2.43.0 a07b618ecd1d f58841127e21 2024-04-12T21:08:32.710 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:08:32.713+0000 7effc9a00700 1 -- 172.21.15.18:0/3498178242 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7effa806ec00 msgr2=0x7effa80710b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:08:32.710 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:08:32.713+0000 7effc9a00700 1 --2- 172.21.15.18:0/3498178242 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7effa806ec00 0x7effa80710b0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7effb000b3b0 tx=0x7effb0009e30 comp rx=0 tx=0).stop 2024-04-12T21:08:32.710 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:08:32.713+0000 7effc9a00700 1 -- 172.21.15.18:0/3498178242 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7effc4109430 msgr2=0x7effc4119fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:08:32.710 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:08:32.713+0000 7effc9a00700 1 --2- 172.21.15.18:0/3498178242 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7effc4109430 0x7effc4119fe0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7effb400c810 tx=0x7effb4004470 comp rx=0 tx=0).stop 2024-04-12T21:08:32.711 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:08:32.714+0000 7effc9a00700 1 -- 172.21.15.18:0/3498178242 shutdown_connections 2024-04-12T21:08:32.711 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:08:32.714+0000 7effc9a00700 1 --2- 172.21.15.18:0/3498178242 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7effa806ec00 0x7effa80710b0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:08:32.711 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:08:32.714+0000 7effc9a00700 1 --2- 172.21.15.18:0/3498178242 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7effc4109430 0x7effc4119fe0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:08:32.711 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:08:32.714+0000 7effc9a00700 1 --2- 172.21.15.18:0/3498178242 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7effc4109dd0 0x7effc411a520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:08:32.711 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:08:32.714+0000 7effc9a00700 1 -- 172.21.15.18:0/3498178242 >> 172.21.15.18:0/3498178242 conn(0x7effc4072c00 msgr2=0x7effc4074be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:08:32.711 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:08:32.714+0000 7effc9a00700 1 -- 172.21.15.18:0/3498178242 shutdown_connections 2024-04-12T21:08:32.711 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:08:32.714+0000 7effc9a00700 1 -- 172.21.15.18:0/3498178242 wait complete. 2024-04-12T21:08:32.868 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:08:32.871+0000 7fef3be00700 1 -- 172.21.15.18:0/1640365549 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fef340fde20 msgr2=0x7fef340fe1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:08:32.868 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:08:32.871+0000 7fef3be00700 1 --2- 172.21.15.18:0/1640365549 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fef340fde20 0x7fef340fe1f0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fef28004940 tx=0x7fef28009e30 comp rx=0 tx=0).stop 2024-04-12T21:08:32.868 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:08:32.871+0000 7fef3be00700 1 -- 172.21.15.18:0/1640365549 shutdown_connections 2024-04-12T21:08:32.868 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:08:32.871+0000 7fef3be00700 1 --2- 172.21.15.18:0/1640365549 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fef340fe7c0 0x7fef34106e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:08:32.868 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:08:32.871+0000 7fef3be00700 1 --2- 172.21.15.18:0/1640365549 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fef340fde20 0x7fef340fe1f0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:08:32.868 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:08:32.871+0000 7fef3be00700 1 -- 172.21.15.18:0/1640365549 >> 172.21.15.18:0/1640365549 conn(0x7fef340f7870 msgr2=0x7fef340f9c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:08:32.868 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:08:32.871+0000 7fef3be00700 1 -- 172.21.15.18:0/1640365549 shutdown_connections 2024-04-12T21:08:32.868 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:08:32.871+0000 7fef3be00700 1 -- 172.21.15.18:0/1640365549 wait complete. 2024-04-12T21:08:32.870 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:08:32.872+0000 7fef3be00700 1 Processor -- start 2024-04-12T21:08:32.870 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:08:32.872+0000 7fef3be00700 1 -- start start 2024-04-12T21:08:32.870 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:08:32.872+0000 7fef3be00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fef340fde20 0x7fef34115bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:08:32.870 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:08:32.872+0000 7fef3be00700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fef340fe7c0 0x7fef341160f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:08:32.870 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:08:32.872+0000 7fef3be00700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef34116740 con 0x7fef340fde20 2024-04-12T21:08:32.870 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:08:32.872+0000 7fef3be00700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef3410fd20 con 0x7fef340fe7c0 2024-04-12T21:08:32.870 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:08:32.873+0000 7fef39400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fef340fde20 0x7fef34115bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:08:32.871 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:08:32.873+0000 7fef39400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fef340fde20 0x7fef34115bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:59232/0 (socket says 172.21.15.18:59232) 2024-04-12T21:08:32.871 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:08:32.873+0000 7fef39400700 1 -- 172.21.15.18:0/3513596003 learned_addr learned my addr 172.21.15.18:0/3513596003 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:08:32.871 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:08:32.873+0000 7fef38a00700 1 --2- 172.21.15.18:0/3513596003 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fef340fe7c0 0x7fef341160f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:08:32.871 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:08:32.873+0000 7fef39400700 1 -- 172.21.15.18:0/3513596003 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fef340fe7c0 msgr2=0x7fef341160f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:08:32.871 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:08:32.873+0000 7fef39400700 1 --2- 172.21.15.18:0/3513596003 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fef340fe7c0 0x7fef341160f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:08:32.871 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:08:32.873+0000 7fef39400700 1 -- 172.21.15.18:0/3513596003 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef28005040 con 0x7fef340fde20 2024-04-12T21:08:32.871 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:08:32.873+0000 7fef39400700 1 --2- 172.21.15.18:0/3513596003 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fef340fde20 0x7fef34115bb0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fef28004890 tx=0x7fef28009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:08:32.871 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:08:32.874+0000 7fef31600700 1 -- 172.21.15.18:0/3513596003 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fef2800e040 con 0x7fef340fde20 2024-04-12T21:08:32.871 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:08:32.874+0000 7fef31600700 1 -- 172.21.15.18:0/3513596003 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fef28009af0 con 0x7fef340fde20 2024-04-12T21:08:32.871 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:08:32.874+0000 7fef3be00700 1 -- 172.21.15.18:0/3513596003 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fef3410ffa0 con 0x7fef340fde20 2024-04-12T21:08:32.871 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:08:32.874+0000 7fef31600700 1 -- 172.21.15.18:0/3513596003 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fef2801c630 con 0x7fef340fde20 2024-04-12T21:08:32.871 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:08:32.874+0000 7fef3be00700 1 -- 172.21.15.18:0/3513596003 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fef341104e0 con 0x7fef340fde20 2024-04-12T21:08:32.872 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:08:32.875+0000 7fef3be00700 1 -- 172.21.15.18:0/3513596003 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fef34066ad0 con 0x7fef340fde20 2024-04-12T21:08:32.872 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:08:32.876+0000 7fef31600700 1 -- 172.21.15.18:0/3513596003 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7fef2800bbc0 con 0x7fef340fde20 2024-04-12T21:08:32.873 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:08:32.877+0000 7fef31600700 1 --2- 172.21.15.18:0/3513596003 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fef1c06ec00 0x7fef1c0710b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:08:32.874 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:08:32.877+0000 7fef31600700 1 -- 172.21.15.18:0/3513596003 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 6757+0+0 (secure 0 0 0) 0x7fef2808fd50 con 0x7fef340fde20 2024-04-12T21:08:32.874 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:08:32.877+0000 7fef38a00700 1 --2- 172.21.15.18:0/3513596003 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fef1c06ec00 0x7fef1c0710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:08:32.874 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:08:32.878+0000 7fef38a00700 1 --2- 172.21.15.18:0/3513596003 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fef1c06ec00 0x7fef1c0710b0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fef34111450 tx=0x7fef2000b040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:08:32.878 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:08:32.881+0000 7fef31600700 1 -- 172.21.15.18:0/3513596003 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7fef2805c500 con 0x7fef340fde20 2024-04-12T21:08:33.204 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:08:33.207+0000 7fef3be00700 1 -- 172.21.15.18:0/3513596003 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fef34066ad0 con 0x7fef340fde20 2024-04-12T21:08:33.204 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:08:33.208+0000 7fef31600700 1 -- 172.21.15.18:0/3513596003 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+837 (secure 0 0 0) 0x7fef2805c090 con 0x7fef340fde20 2024-04-12T21:08:33.205 INFO:teuthology.orchestra.run.smithi018.stdout:{ 2024-04-12T21:08:33.205 INFO:teuthology.orchestra.run.smithi018.stdout: "mon": { 2024-04-12T21:08:33.205 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 19.0.0-2879-g324baf8d (324baf8ddb91dea46cac50cb535c306115791355) squid (dev)": 2 2024-04-12T21:08:33.205 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-12T21:08:33.205 INFO:teuthology.orchestra.run.smithi018.stdout: "mgr": { 2024-04-12T21:08:33.206 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 19.0.0-2879-g324baf8d (324baf8ddb91dea46cac50cb535c306115791355) squid (dev)": 2 2024-04-12T21:08:33.206 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-12T21:08:33.206 INFO:teuthology.orchestra.run.smithi018.stdout: "osd": { 2024-04-12T21:08:33.206 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 19.0.0-2879-g324baf8d (324baf8ddb91dea46cac50cb535c306115791355) squid (dev)": 6 2024-04-12T21:08:33.206 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-12T21:08:33.206 INFO:teuthology.orchestra.run.smithi018.stdout: "mds": { 2024-04-12T21:08:33.206 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 17.2.7-827-gd3b5060a (d3b5060a17449edfae9385cdf792afec109fca5d) quincy (stable)": 2, 2024-04-12T21:08:33.206 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 19.0.0-2879-g324baf8d (324baf8ddb91dea46cac50cb535c306115791355) squid (dev)": 2 2024-04-12T21:08:33.206 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-12T21:08:33.206 INFO:teuthology.orchestra.run.smithi018.stdout: "overall": { 2024-04-12T21:08:33.206 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 17.2.7-827-gd3b5060a (d3b5060a17449edfae9385cdf792afec109fca5d) quincy (stable)": 2, 2024-04-12T21:08:33.206 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 19.0.0-2879-g324baf8d (324baf8ddb91dea46cac50cb535c306115791355) squid (dev)": 12 2024-04-12T21:08:33.207 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-04-12T21:08:33.207 INFO:teuthology.orchestra.run.smithi018.stdout:} 2024-04-12T21:08:33.207 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:08:33.210+0000 7fef3be00700 1 -- 172.21.15.18:0/3513596003 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fef1c06ec00 msgr2=0x7fef1c0710b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:08:33.207 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:08:33.210+0000 7fef3be00700 1 --2- 172.21.15.18:0/3513596003 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fef1c06ec00 0x7fef1c0710b0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fef34111450 tx=0x7fef2000b040 comp rx=0 tx=0).stop 2024-04-12T21:08:33.207 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:08:33.211+0000 7fef3be00700 1 -- 172.21.15.18:0/3513596003 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fef340fde20 msgr2=0x7fef34115bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:08:33.208 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:08:33.211+0000 7fef3be00700 1 --2- 172.21.15.18:0/3513596003 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fef340fde20 0x7fef34115bb0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fef28004890 tx=0x7fef28009e30 comp rx=0 tx=0).stop 2024-04-12T21:08:33.208 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:08:33.211+0000 7fef3be00700 1 -- 172.21.15.18:0/3513596003 shutdown_connections 2024-04-12T21:08:33.208 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:08:33.211+0000 7fef3be00700 1 --2- 172.21.15.18:0/3513596003 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fef1c06ec00 0x7fef1c0710b0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:08:33.208 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:08:33.211+0000 7fef3be00700 1 --2- 172.21.15.18:0/3513596003 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fef340fde20 0x7fef34115bb0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:08:33.209 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:08:33.211+0000 7fef3be00700 1 --2- 172.21.15.18:0/3513596003 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fef340fe7c0 0x7fef341160f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:08:33.209 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:08:33.211+0000 7fef3be00700 1 -- 172.21.15.18:0/3513596003 >> 172.21.15.18:0/3513596003 conn(0x7fef340f7870 msgr2=0x7fef340fbe10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:08:33.209 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:08:33.211+0000 7fef3be00700 1 -- 172.21.15.18:0/3513596003 shutdown_connections 2024-04-12T21:08:33.209 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:08:33.211+0000 7fef3be00700 1 -- 172.21.15.18:0/3513596003 wait complete. 2024-04-12T21:08:33.279 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:08:33 smithi186 ceph-mon[86379]: from='client.34500 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:08:33.279 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:08:33 smithi186 ceph-mon[86379]: pgmap v262: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:08:33.279 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:08:33 smithi186 ceph-mon[86379]: from='client.34504 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:08:33.350 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:08:33 smithi018 ceph-mon[112590]: from='client.34500 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:08:33.350 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:08:33 smithi018 ceph-mon[112590]: pgmap v262: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:08:33.350 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:08:33 smithi018 ceph-mon[112590]: from='client.34504 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:08:33.366 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:08:33.369+0000 7f6c09600700 1 -- 172.21.15.18:0/862768686 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6c04109c40 msgr2=0x7f6c041032b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:08:33.366 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:08:33.369+0000 7f6c09600700 1 --2- 172.21.15.18:0/862768686 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6c04109c40 0x7f6c041032b0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f6bf0004940 tx=0x7f6bf0009e30 comp rx=0 tx=0).stop 2024-04-12T21:08:33.366 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:08:33.369+0000 7f6c09600700 1 -- 172.21.15.18:0/862768686 shutdown_connections 2024-04-12T21:08:33.366 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:08:33.369+0000 7f6c09600700 1 --2- 172.21.15.18:0/862768686 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6c04109c40 0x7f6c041032b0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:08:33.366 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:08:33.369+0000 7f6c09600700 1 --2- 172.21.15.18:0/862768686 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6c041092a0 0x7f6c04109670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:08:33.366 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:08:33.369+0000 7f6c09600700 1 -- 172.21.15.18:0/862768686 >> 172.21.15.18:0/862768686 conn(0x7f6c04072a30 msgr2=0x7f6c04074e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:08:33.366 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:08:33.369+0000 7f6c09600700 1 -- 172.21.15.18:0/862768686 shutdown_connections 2024-04-12T21:08:33.367 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:08:33.369+0000 7f6c09600700 1 -- 172.21.15.18:0/862768686 wait complete. 2024-04-12T21:08:33.368 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:08:33.370+0000 7f6c09600700 1 Processor -- start 2024-04-12T21:08:33.368 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:08:33.370+0000 7f6c09600700 1 -- start start 2024-04-12T21:08:33.368 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:08:33.370+0000 7f6c09600700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6c041092a0 0x7f6c04119fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:08:33.368 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:08:33.370+0000 7f6c09600700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6c04109c40 0x7f6c0411a500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:08:33.368 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:08:33.370+0000 7f6c09600700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c0411ab50 con 0x7f6c04109c40 2024-04-12T21:08:33.368 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:08:33.370+0000 7f6c09600700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c04113f20 con 0x7f6c041092a0 2024-04-12T21:08:33.368 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:08:33.371+0000 7f6c03400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6c04109c40 0x7f6c0411a500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:08:33.369 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:08:33.371+0000 7f6c03400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6c04109c40 0x7f6c0411a500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:59240/0 (socket says 172.21.15.18:59240) 2024-04-12T21:08:33.369 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:08:33.371+0000 7f6c03400700 1 -- 172.21.15.18:0/2500460291 learned_addr learned my addr 172.21.15.18:0/2500460291 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:08:33.369 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:08:33.371+0000 7f6c03e00700 1 --2- 172.21.15.18:0/2500460291 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6c041092a0 0x7f6c04119fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:08:33.369 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:08:33.371+0000 7f6c03400700 1 -- 172.21.15.18:0/2500460291 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6c041092a0 msgr2=0x7f6c04119fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:08:33.369 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:08:33.371+0000 7f6c03400700 1 --2- 172.21.15.18:0/2500460291 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6c041092a0 0x7f6c04119fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:08:33.369 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:08:33.371+0000 7f6c03400700 1 -- 172.21.15.18:0/2500460291 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6bf0005040 con 0x7f6c04109c40 2024-04-12T21:08:33.369 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:08:33.371+0000 7f6c03400700 1 --2- 172.21.15.18:0/2500460291 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6c04109c40 0x7f6c0411a500 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f6bf0009100 tx=0x7f6bf0009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:08:33.370 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:08:33.372+0000 7f6bfbe00700 1 -- 172.21.15.18:0/2500460291 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6bf000e040 con 0x7f6c04109c40 2024-04-12T21:08:33.370 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:08:33.372+0000 7f6bfbe00700 1 -- 172.21.15.18:0/2500460291 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f6bf0013420 con 0x7f6c04109c40 2024-04-12T21:08:33.370 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:08:33.372+0000 7f6bfbe00700 1 -- 172.21.15.18:0/2500460291 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6bf001c630 con 0x7f6c04109c40 2024-04-12T21:08:33.370 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:08:33.372+0000 7f6c09600700 1 -- 172.21.15.18:0/2500460291 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6c041141a0 con 0x7f6c04109c40 2024-04-12T21:08:33.370 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:08:33.372+0000 7f6c09600700 1 -- 172.21.15.18:0/2500460291 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6c04114760 con 0x7f6c04109c40 2024-04-12T21:08:33.370 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:08:33.373+0000 7f6c09600700 1 -- 172.21.15.18:0/2500460291 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6c041049f0 con 0x7f6c04109c40 2024-04-12T21:08:33.371 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:08:33.375+0000 7f6bfbe00700 1 -- 172.21.15.18:0/2500460291 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7f6bf0013590 con 0x7f6c04109c40 2024-04-12T21:08:33.372 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:08:33.375+0000 7f6bfbe00700 1 --2- 172.21.15.18:0/2500460291 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f6be806ec00 0x7f6be80710b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:08:33.372 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:08:33.376+0000 7f6bfbe00700 1 -- 172.21.15.18:0/2500460291 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 6757+0+0 (secure 0 0 0) 0x7f6bf008fb60 con 0x7f6c04109c40 2024-04-12T21:08:33.372 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:08:33.376+0000 7f6c03e00700 1 --2- 172.21.15.18:0/2500460291 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f6be806ec00 0x7f6be80710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:08:33.373 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:08:33.377+0000 7f6c03e00700 1 --2- 172.21.15.18:0/2500460291 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f6be806ec00 0x7f6be80710b0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f6bf4004d60 tx=0x7f6bf400b040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:08:33.377 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:08:33.380+0000 7f6bfbe00700 1 -- 172.21.15.18:0/2500460291 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f6bf005c3d0 con 0x7f6c04109c40 2024-04-12T21:08:33.656 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:08:33.659+0000 7f6c09600700 1 -- 172.21.15.18:0/2500460291 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f6c0404bed0 con 0x7f6c04109c40 2024-04-12T21:08:33.657 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:08:33.660+0000 7f6bfbe00700 1 -- 172.21.15.18:0/2500460291 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 29 v29) v1 ==== 76+0+1872 (secure 0 0 0) 0x7f6bf0058030 con 0x7f6c04109c40 2024-04-12T21:08:33.658 INFO:teuthology.orchestra.run.smithi018.stdout:e29 2024-04-12T21:08:33.658 INFO:teuthology.orchestra.run.smithi018.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-12T21:08:33.658 INFO:teuthology.orchestra.run.smithi018.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-12T21:08:33.659 INFO:teuthology.orchestra.run.smithi018.stdout:legacy client fscid: 1 2024-04-12T21:08:33.659 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-12T21:08:33.659 INFO:teuthology.orchestra.run.smithi018.stdout:Filesystem 'cephfs' (1) 2024-04-12T21:08:33.659 INFO:teuthology.orchestra.run.smithi018.stdout:fs_name cephfs 2024-04-12T21:08:33.659 INFO:teuthology.orchestra.run.smithi018.stdout:epoch 24 2024-04-12T21:08:33.659 INFO:teuthology.orchestra.run.smithi018.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-04-12T21:08:33.659 INFO:teuthology.orchestra.run.smithi018.stdout:created 2024-04-12T20:51:57.362850+0000 2024-04-12T21:08:33.659 INFO:teuthology.orchestra.run.smithi018.stdout:modified 2024-04-12T21:06:59.717190+0000 2024-04-12T21:08:33.659 INFO:teuthology.orchestra.run.smithi018.stdout:tableserver 0 2024-04-12T21:08:33.659 INFO:teuthology.orchestra.run.smithi018.stdout:root 0 2024-04-12T21:08:33.659 INFO:teuthology.orchestra.run.smithi018.stdout:session_timeout 60 2024-04-12T21:08:33.659 INFO:teuthology.orchestra.run.smithi018.stdout:session_autoclose 300 2024-04-12T21:08:33.659 INFO:teuthology.orchestra.run.smithi018.stdout:max_file_size 1099511627776 2024-04-12T21:08:33.659 INFO:teuthology.orchestra.run.smithi018.stdout:max_xattr_size 65536 2024-04-12T21:08:33.659 INFO:teuthology.orchestra.run.smithi018.stdout:required_client_features {} 2024-04-12T21:08:33.660 INFO:teuthology.orchestra.run.smithi018.stdout:last_failure 0 2024-04-12T21:08:33.660 INFO:teuthology.orchestra.run.smithi018.stdout:last_failure_osd_epoch 100 2024-04-12T21:08:33.660 INFO:teuthology.orchestra.run.smithi018.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-12T21:08:33.660 INFO:teuthology.orchestra.run.smithi018.stdout:max_mds 2 2024-04-12T21:08:33.660 INFO:teuthology.orchestra.run.smithi018.stdout:in 0,1 2024-04-12T21:08:33.660 INFO:teuthology.orchestra.run.smithi018.stdout:up {} 2024-04-12T21:08:33.660 INFO:teuthology.orchestra.run.smithi018.stdout:failed 0,1 2024-04-12T21:08:33.660 INFO:teuthology.orchestra.run.smithi018.stdout:damaged 2024-04-12T21:08:33.660 INFO:teuthology.orchestra.run.smithi018.stdout:stopped 2024-04-12T21:08:33.660 INFO:teuthology.orchestra.run.smithi018.stdout:data_pools [3] 2024-04-12T21:08:33.660 INFO:teuthology.orchestra.run.smithi018.stdout:metadata_pool 2 2024-04-12T21:08:33.660 INFO:teuthology.orchestra.run.smithi018.stdout:inline_data disabled 2024-04-12T21:08:33.660 INFO:teuthology.orchestra.run.smithi018.stdout:balancer 2024-04-12T21:08:33.660 INFO:teuthology.orchestra.run.smithi018.stdout:bal_rank_mask -1 2024-04-12T21:08:33.660 INFO:teuthology.orchestra.run.smithi018.stdout:standby_count_wanted 1 2024-04-12T21:08:33.661 INFO:teuthology.orchestra.run.smithi018.stdout:qdb_cluster leader: 0 members: 2024-04-12T21:08:33.661 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-12T21:08:33.661 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-12T21:08:33.661 INFO:teuthology.orchestra.run.smithi018.stdout:Standby daemons: 2024-04-12T21:08:33.661 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-12T21:08:33.661 INFO:teuthology.orchestra.run.smithi018.stdout:[mds.cephfs.smithi186.wjnafs{-1:24277} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.186:6826/438347207,v1:172.21.15.186:6827/438347207] compat {c=[1],r=[1],i=[7ff]}] 2024-04-12T21:08:33.661 INFO:teuthology.orchestra.run.smithi018.stdout:[mds.cephfs.smithi186.jdllyn{-1:24281} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.186:6824/1464359952,v1:172.21.15.186:6825/1464359952] compat {c=[1],r=[1],i=[7ff]}] 2024-04-12T21:08:33.661 INFO:teuthology.orchestra.run.smithi018.stdout:[mds.cephfs.smithi018.acqbsc{-1:34436} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.18:6828/755179764,v1:172.21.15.18:6829/755179764] compat {c=[1],r=[1],i=[1fff]}] 2024-04-12T21:08:33.661 INFO:teuthology.orchestra.run.smithi018.stdout:[mds.cephfs.smithi018.szjluo{-1:34468} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.18:6826/2407992653,v1:172.21.15.18:6827/2407992653] compat {c=[1],r=[1],i=[1fff]}] 2024-04-12T21:08:33.662 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:08:33.663+0000 7f6c09600700 1 -- 172.21.15.18:0/2500460291 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f6be806ec00 msgr2=0x7f6be80710b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:08:33.662 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:08:33.663+0000 7f6c09600700 1 --2- 172.21.15.18:0/2500460291 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f6be806ec00 0x7f6be80710b0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f6bf4004d60 tx=0x7f6bf400b040 comp rx=0 tx=0).stop 2024-04-12T21:08:33.662 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:08:33.663+0000 7f6c09600700 1 -- 172.21.15.18:0/2500460291 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6c04109c40 msgr2=0x7f6c0411a500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:08:33.662 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:08:33.663+0000 7f6c09600700 1 --2- 172.21.15.18:0/2500460291 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6c04109c40 0x7f6c0411a500 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f6bf0009100 tx=0x7f6bf0009e30 comp rx=0 tx=0).stop 2024-04-12T21:08:33.662 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:08:33.663+0000 7f6c09600700 1 -- 172.21.15.18:0/2500460291 shutdown_connections 2024-04-12T21:08:33.662 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:08:33.663+0000 7f6c09600700 1 --2- 172.21.15.18:0/2500460291 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f6be806ec00 0x7f6be80710b0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:08:33.663 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:08:33.663+0000 7f6c09600700 1 --2- 172.21.15.18:0/2500460291 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6c041092a0 0x7f6c04119fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:08:33.663 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:08:33.663+0000 7f6c09600700 1 --2- 172.21.15.18:0/2500460291 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6c04109c40 0x7f6c0411a500 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:08:33.663 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:08:33.663+0000 7f6c09600700 1 -- 172.21.15.18:0/2500460291 >> 172.21.15.18:0/2500460291 conn(0x7f6c04072a30 msgr2=0x7f6c04074910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:08:33.663 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:08:33.663+0000 7f6c09600700 1 -- 172.21.15.18:0/2500460291 shutdown_connections 2024-04-12T21:08:33.663 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:08:33.663+0000 7f6c09600700 1 -- 172.21.15.18:0/2500460291 wait complete. 2024-04-12T21:08:33.663 INFO:teuthology.orchestra.run.smithi018.stderr:dumped fsmap epoch 29 2024-04-12T21:08:33.823 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:08:33.826+0000 7fc8bca00700 1 -- 172.21.15.18:0/2248550253 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc8b8101d80 msgr2=0x7fc8b8102150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:08:33.823 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:08:33.826+0000 7fc8bca00700 1 --2- 172.21.15.18:0/2248550253 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc8b8101d80 0x7fc8b8102150 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fc8a8004940 tx=0x7fc8a8009e30 comp rx=0 tx=0).stop 2024-04-12T21:08:33.823 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:08:33.826+0000 7fc8bca00700 1 -- 172.21.15.18:0/2248550253 shutdown_connections 2024-04-12T21:08:33.823 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:08:33.826+0000 7fc8bca00700 1 --2- 172.21.15.18:0/2248550253 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc8b8102720 0x7fc8b80fbd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:08:33.823 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:08:33.826+0000 7fc8bca00700 1 --2- 172.21.15.18:0/2248550253 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc8b8101d80 0x7fc8b8102150 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:08:33.823 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:08:33.826+0000 7fc8bca00700 1 -- 172.21.15.18:0/2248550253 >> 172.21.15.18:0/2248550253 conn(0x7fc8b80f78f0 msgr2=0x7fc8b80f9d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:08:33.823 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:08:33.826+0000 7fc8bca00700 1 -- 172.21.15.18:0/2248550253 shutdown_connections 2024-04-12T21:08:33.824 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:08:33.826+0000 7fc8bca00700 1 -- 172.21.15.18:0/2248550253 wait complete. 2024-04-12T21:08:33.824 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:08:33.827+0000 7fc8bca00700 1 Processor -- start 2024-04-12T21:08:33.824 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:08:33.828+0000 7fc8bca00700 1 -- start start 2024-04-12T21:08:33.827 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:08:33.828+0000 7fc8bca00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc8b8101d80 0x7fc8b8115b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:08:33.827 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:08:33.828+0000 7fc8bca00700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc8b8102720 0x7fc8b8116050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:08:33.827 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:08:33.828+0000 7fc8bca00700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8b8116730 con 0x7fc8b8101d80 2024-04-12T21:08:33.827 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:08:33.828+0000 7fc8bca00700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8b810fc80 con 0x7fc8b8102720 2024-04-12T21:08:33.827 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:08:33.829+0000 7fc8b5e00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc8b8101d80 0x7fc8b8115b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:08:33.827 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:08:33.829+0000 7fc8b5e00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc8b8101d80 0x7fc8b8115b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:59262/0 (socket says 172.21.15.18:59262) 2024-04-12T21:08:33.827 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:08:33.829+0000 7fc8b5e00700 1 -- 172.21.15.18:0/264081460 learned_addr learned my addr 172.21.15.18:0/264081460 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:08:33.828 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:08:33.829+0000 7fc8b5400700 1 --2- 172.21.15.18:0/264081460 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc8b8102720 0x7fc8b8116050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:08:33.828 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:08:33.829+0000 7fc8b5e00700 1 -- 172.21.15.18:0/264081460 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc8b8102720 msgr2=0x7fc8b8116050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:08:33.828 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:08:33.829+0000 7fc8b5e00700 1 --2- 172.21.15.18:0/264081460 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc8b8102720 0x7fc8b8116050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:08:33.828 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:08:33.829+0000 7fc8b5e00700 1 -- 172.21.15.18:0/264081460 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8a8005040 con 0x7fc8b8101d80 2024-04-12T21:08:33.828 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:08:33.829+0000 7fc8b5400700 1 --2- 172.21.15.18:0/264081460 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc8b8102720 0x7fc8b8116050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-12T21:08:33.828 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:08:33.829+0000 7fc8b5e00700 1 --2- 172.21.15.18:0/264081460 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc8b8101d80 0x7fc8b8115b10 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fc8a8009180 tx=0x7fc8a8009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:08:33.828 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:08:33.830+0000 7fc8ae000700 1 -- 172.21.15.18:0/264081460 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc8a800e040 con 0x7fc8b8101d80 2024-04-12T21:08:33.828 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:08:33.830+0000 7fc8ae000700 1 -- 172.21.15.18:0/264081460 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc8a800bbc0 con 0x7fc8b8101d80 2024-04-12T21:08:33.828 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:08:33.830+0000 7fc8bca00700 1 -- 172.21.15.18:0/264081460 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc8b810fea0 con 0x7fc8b8101d80 2024-04-12T21:08:33.828 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:08:33.830+0000 7fc8ae000700 1 -- 172.21.15.18:0/264081460 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc8a801c630 con 0x7fc8b8101d80 2024-04-12T21:08:33.828 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:08:33.830+0000 7fc8bca00700 1 -- 172.21.15.18:0/264081460 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc8b8110460 con 0x7fc8b8101d80 2024-04-12T21:08:33.829 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:08:33.831+0000 7fc8bca00700 1 -- 172.21.15.18:0/264081460 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc8b80fd4d0 con 0x7fc8b8101d80 2024-04-12T21:08:33.829 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:08:33.832+0000 7fc8ae000700 1 -- 172.21.15.18:0/264081460 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7fc8a800bd30 con 0x7fc8b8101d80 2024-04-12T21:08:33.829 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:08:33.833+0000 7fc8ae000700 1 --2- 172.21.15.18:0/264081460 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fc89c06ec00 0x7fc89c0710b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:08:33.830 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:08:33.833+0000 7fc8ae000700 1 -- 172.21.15.18:0/264081460 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 6757+0+0 (secure 0 0 0) 0x7fc8a808fe10 con 0x7fc8b8101d80 2024-04-12T21:08:33.830 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:08:33.833+0000 7fc8b5400700 1 --2- 172.21.15.18:0/264081460 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fc89c06ec00 0x7fc89c0710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:08:33.831 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:08:33.834+0000 7fc8b5400700 1 --2- 172.21.15.18:0/264081460 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fc89c06ec00 0x7fc89c0710b0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fc8a4006fd0 tx=0x7fc8a4008040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:08:33.835 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:08:33.838+0000 7fc8ae000700 1 -- 172.21.15.18:0/264081460 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7fc8a805c5c0 con 0x7fc8b8101d80 2024-04-12T21:08:34.094 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:08:34.097+0000 7fc8bca00700 1 -- 172.21.15.18:0/264081460 --> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fc8b8111200 con 0x7fc89c06ec00 2024-04-12T21:08:34.096 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:08:34.100+0000 7fc8ae000700 1 -- 172.21.15.18:0/264081460 <== mgr.34102 v2:172.21.15.18:6800/3812891919 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7fc8b8111200 con 0x7fc89c06ec00 2024-04-12T21:08:34.097 INFO:teuthology.orchestra.run.smithi018.stdout:{ 2024-04-12T21:08:34.097 INFO:teuthology.orchestra.run.smithi018.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:f4452def3be4d350f4a8e5feaf7ebb1511be9298fe68478ba7a2d6fbd7d69a63", 2024-04-12T21:08:34.097 INFO:teuthology.orchestra.run.smithi018.stdout: "in_progress": true, 2024-04-12T21:08:34.097 INFO:teuthology.orchestra.run.smithi018.stdout: "which": "Upgrading all daemon types on all hosts", 2024-04-12T21:08:34.097 INFO:teuthology.orchestra.run.smithi018.stdout: "services_complete": [ 2024-04-12T21:08:34.097 INFO:teuthology.orchestra.run.smithi018.stdout: "crash", 2024-04-12T21:08:34.097 INFO:teuthology.orchestra.run.smithi018.stdout: "osd", 2024-04-12T21:08:34.097 INFO:teuthology.orchestra.run.smithi018.stdout: "mon", 2024-04-12T21:08:34.097 INFO:teuthology.orchestra.run.smithi018.stdout: "mgr" 2024-04-12T21:08:34.097 INFO:teuthology.orchestra.run.smithi018.stdout: ], 2024-04-12T21:08:34.097 INFO:teuthology.orchestra.run.smithi018.stdout: "progress": "14/21 daemons upgraded", 2024-04-12T21:08:34.097 INFO:teuthology.orchestra.run.smithi018.stdout: "message": "Currently upgrading mds daemons", 2024-04-12T21:08:34.097 INFO:teuthology.orchestra.run.smithi018.stdout: "is_paused": false 2024-04-12T21:08:34.098 INFO:teuthology.orchestra.run.smithi018.stdout:} 2024-04-12T21:08:34.100 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:08:34.103+0000 7fc8bca00700 1 -- 172.21.15.18:0/264081460 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fc89c06ec00 msgr2=0x7fc89c0710b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:08:34.100 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:08:34.103+0000 7fc8bca00700 1 --2- 172.21.15.18:0/264081460 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fc89c06ec00 0x7fc89c0710b0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fc8a4006fd0 tx=0x7fc8a4008040 comp rx=0 tx=0).stop 2024-04-12T21:08:34.100 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:08:34.103+0000 7fc8bca00700 1 -- 172.21.15.18:0/264081460 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc8b8101d80 msgr2=0x7fc8b8115b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:08:34.100 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:08:34.103+0000 7fc8bca00700 1 --2- 172.21.15.18:0/264081460 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc8b8101d80 0x7fc8b8115b10 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fc8a8009180 tx=0x7fc8a8009e30 comp rx=0 tx=0).stop 2024-04-12T21:08:34.101 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:08:34.103+0000 7fc8bca00700 1 -- 172.21.15.18:0/264081460 shutdown_connections 2024-04-12T21:08:34.101 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:08:34.103+0000 7fc8bca00700 1 --2- 172.21.15.18:0/264081460 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fc89c06ec00 0x7fc89c0710b0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:08:34.101 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:08:34.103+0000 7fc8bca00700 1 --2- 172.21.15.18:0/264081460 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc8b8101d80 0x7fc8b8115b10 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:08:34.101 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:08:34.103+0000 7fc8bca00700 1 --2- 172.21.15.18:0/264081460 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc8b8102720 0x7fc8b8116050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:08:34.101 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:08:34.103+0000 7fc8bca00700 1 -- 172.21.15.18:0/264081460 >> 172.21.15.18:0/264081460 conn(0x7fc8b80f78f0 msgr2=0x7fc8b8066450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:08:34.101 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:08:34.104+0000 7fc8bca00700 1 -- 172.21.15.18:0/264081460 shutdown_connections 2024-04-12T21:08:34.101 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:08:34.104+0000 7fc8bca00700 1 -- 172.21.15.18:0/264081460 wait complete. 2024-04-12T21:08:34.259 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:08:34.262+0000 7efd5f200700 1 -- 172.21.15.18:0/372762909 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7efd58105020 msgr2=0x7efd581053f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:08:34.259 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:08:34.262+0000 7efd5f200700 1 --2- 172.21.15.18:0/372762909 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7efd58105020 0x7efd581053f0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7efd48004940 tx=0x7efd48009e30 comp rx=0 tx=0).stop 2024-04-12T21:08:34.259 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:08:34.262+0000 7efd5f200700 1 -- 172.21.15.18:0/372762909 shutdown_connections 2024-04-12T21:08:34.259 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:08:34.262+0000 7efd5f200700 1 --2- 172.21.15.18:0/372762909 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7efd581059c0 0x7efd580ff080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:08:34.259 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:08:34.262+0000 7efd5f200700 1 --2- 172.21.15.18:0/372762909 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7efd58105020 0x7efd581053f0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:08:34.259 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:08:34.262+0000 7efd5f200700 1 -- 172.21.15.18:0/372762909 >> 172.21.15.18:0/372762909 conn(0x7efd580faba0 msgr2=0x7efd580fcfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:08:34.259 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:08:34.262+0000 7efd5f200700 1 -- 172.21.15.18:0/372762909 shutdown_connections 2024-04-12T21:08:34.259 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:08:34.262+0000 7efd5f200700 1 -- 172.21.15.18:0/372762909 wait complete. 2024-04-12T21:08:34.260 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:08:34.263+0000 7efd5f200700 1 Processor -- start 2024-04-12T21:08:34.260 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:08:34.263+0000 7efd5f200700 1 -- start start 2024-04-12T21:08:34.260 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:08:34.264+0000 7efd5f200700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7efd58105020 0x7efd58115bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:08:34.260 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:08:34.264+0000 7efd5f200700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7efd581059c0 0x7efd58116100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:08:34.260 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:08:34.264+0000 7efd5f200700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd58116750 con 0x7efd581059c0 2024-04-12T21:08:34.261 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:08:34.264+0000 7efd5f200700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd5810fd30 con 0x7efd58105020 2024-04-12T21:08:34.262 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:08:34.264+0000 7efd57400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7efd581059c0 0x7efd58116100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:08:34.262 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:08:34.264+0000 7efd57400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7efd581059c0 0x7efd58116100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:59280/0 (socket says 172.21.15.18:59280) 2024-04-12T21:08:34.262 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:08:34.264+0000 7efd57400700 1 -- 172.21.15.18:0/384992807 learned_addr learned my addr 172.21.15.18:0/384992807 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:08:34.262 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:08:34.264+0000 7efd57400700 1 -- 172.21.15.18:0/384992807 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7efd58105020 msgr2=0x7efd58115bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:08:34.262 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:08:34.264+0000 7efd57e00700 1 --2- 172.21.15.18:0/384992807 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7efd58105020 0x7efd58115bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:08:34.262 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:08:34.264+0000 7efd57400700 1 --2- 172.21.15.18:0/384992807 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7efd58105020 0x7efd58115bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:08:34.262 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:08:34.264+0000 7efd57400700 1 -- 172.21.15.18:0/384992807 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd48005040 con 0x7efd581059c0 2024-04-12T21:08:34.262 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:08:34.265+0000 7efd57400700 1 --2- 172.21.15.18:0/384992807 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7efd581059c0 0x7efd58116100 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7efd44002a80 tx=0x7efd44004630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:08:34.262 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:08:34.265+0000 7efd4fe00700 1 -- 172.21.15.18:0/384992807 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7efd44007cc0 con 0x7efd581059c0 2024-04-12T21:08:34.263 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:08:34.265+0000 7efd4fe00700 1 -- 172.21.15.18:0/384992807 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7efd4400f040 con 0x7efd581059c0 2024-04-12T21:08:34.263 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:08:34.265+0000 7efd4fe00700 1 -- 172.21.15.18:0/384992807 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7efd4400daf0 con 0x7efd581059c0 2024-04-12T21:08:34.263 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:08:34.265+0000 7efd5f200700 1 -- 172.21.15.18:0/384992807 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efd58110010 con 0x7efd581059c0 2024-04-12T21:08:34.263 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:08:34.265+0000 7efd5f200700 1 -- 172.21.15.18:0/384992807 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efd58110580 con 0x7efd581059c0 2024-04-12T21:08:34.263 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:08:34.266+0000 7efd5f200700 1 -- 172.21.15.18:0/384992807 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efd58100770 con 0x7efd581059c0 2024-04-12T21:08:34.263 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:08:34 smithi018 ceph-mon[112590]: from='client.34508 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:08:34.263 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:08:34 smithi018 ceph-mon[112590]: from='client.? 172.21.15.18:0/3513596003' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T21:08:34.263 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:08:34 smithi018 ceph-mon[112590]: from='client.? 172.21.15.18:0/2500460291' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-12T21:08:34.265 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:08:34.269+0000 7efd4fe00700 1 -- 172.21.15.18:0/384992807 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7efd4400dc90 con 0x7efd581059c0 2024-04-12T21:08:34.266 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:08:34.270+0000 7efd4fe00700 1 --2- 172.21.15.18:0/384992807 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7efd3c06ec00 0x7efd3c0710b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:08:34.267 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:08:34.270+0000 7efd4fe00700 1 -- 172.21.15.18:0/384992807 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 6757+0+0 (secure 0 0 0) 0x7efd4408e140 con 0x7efd581059c0 2024-04-12T21:08:34.267 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:08:34.270+0000 7efd57e00700 1 --2- 172.21.15.18:0/384992807 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7efd3c06ec00 0x7efd3c0710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:08:34.267 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:08:34.271+0000 7efd57e00700 1 --2- 172.21.15.18:0/384992807 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7efd3c06ec00 0x7efd3c0710b0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7efd48004f90 tx=0x7efd48009e30 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:08:34.271 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:08:34.275+0000 7efd4fe00700 1 -- 172.21.15.18:0/384992807 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7efd4405a870 con 0x7efd581059c0 2024-04-12T21:08:34.279 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:08:34 smithi186 ceph-mon[86379]: from='client.34508 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:08:34.279 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:08:34 smithi186 ceph-mon[86379]: from='client.? 172.21.15.18:0/3513596003' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T21:08:34.279 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:08:34 smithi186 ceph-mon[86379]: from='client.? 172.21.15.18:0/2500460291' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-12T21:08:34.594 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:08:34.597+0000 7efd5f200700 1 -- 172.21.15.18:0/384992807 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7efd5804c710 con 0x7efd581059c0 2024-04-12T21:08:34.594 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:08:34.597+0000 7efd4fe00700 1 -- 172.21.15.18:0/384992807 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+233 (secure 0 0 0) 0x7efd4405a400 con 0x7efd581059c0 2024-04-12T21:08:34.594 INFO:teuthology.orchestra.run.smithi018.stdout:HEALTH_ERR 1 filesystem is degraded; 1 filesystem is offline 2024-04-12T21:08:34.595 INFO:teuthology.orchestra.run.smithi018.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-04-12T21:08:34.595 INFO:teuthology.orchestra.run.smithi018.stdout: fs cephfs is degraded 2024-04-12T21:08:34.595 INFO:teuthology.orchestra.run.smithi018.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-04-12T21:08:34.595 INFO:teuthology.orchestra.run.smithi018.stdout: fs cephfs is offline because no MDS is active for it. 2024-04-12T21:08:34.598 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:08:34.600+0000 7efd5f200700 1 -- 172.21.15.18:0/384992807 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7efd3c06ec00 msgr2=0x7efd3c0710b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:08:34.598 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:08:34.600+0000 7efd5f200700 1 --2- 172.21.15.18:0/384992807 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7efd3c06ec00 0x7efd3c0710b0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7efd48004f90 tx=0x7efd48009e30 comp rx=0 tx=0).stop 2024-04-12T21:08:34.598 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:08:34.600+0000 7efd5f200700 1 -- 172.21.15.18:0/384992807 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7efd581059c0 msgr2=0x7efd58116100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:08:34.598 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:08:34.600+0000 7efd5f200700 1 --2- 172.21.15.18:0/384992807 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7efd581059c0 0x7efd58116100 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7efd44002a80 tx=0x7efd44004630 comp rx=0 tx=0).stop 2024-04-12T21:08:34.598 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:08:34.601+0000 7efd5f200700 1 -- 172.21.15.18:0/384992807 shutdown_connections 2024-04-12T21:08:34.598 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:08:34.601+0000 7efd5f200700 1 --2- 172.21.15.18:0/384992807 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7efd3c06ec00 0x7efd3c0710b0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:08:34.598 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:08:34.601+0000 7efd5f200700 1 --2- 172.21.15.18:0/384992807 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7efd58105020 0x7efd58115bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:08:34.598 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:08:34.601+0000 7efd5f200700 1 --2- 172.21.15.18:0/384992807 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7efd581059c0 0x7efd58116100 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:08:34.598 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:08:34.601+0000 7efd5f200700 1 -- 172.21.15.18:0/384992807 >> 172.21.15.18:0/384992807 conn(0x7efd580faba0 msgr2=0x7efd580fbba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:08:34.598 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:08:34.601+0000 7efd5f200700 1 -- 172.21.15.18:0/384992807 shutdown_connections 2024-04-12T21:08:34.598 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:08:34.601+0000 7efd5f200700 1 -- 172.21.15.18:0/384992807 wait complete. 2024-04-12T21:08:35.279 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:08:35 smithi186 ceph-mon[86379]: pgmap v263: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:08:35.279 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:08:35 smithi186 ceph-mon[86379]: from='client.34520 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:08:35.279 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:08:35 smithi186 ceph-mon[86379]: from='client.? 172.21.15.18:0/384992807' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-12T21:08:35.350 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:08:35 smithi018 ceph-mon[112590]: pgmap v263: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:08:35.351 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:08:35 smithi018 ceph-mon[112590]: from='client.34520 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:08:35.351 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:08:35 smithi018 ceph-mon[112590]: from='client.? 172.21.15.18:0/384992807' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-12T21:08:37.279 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:08:37 smithi186 ceph-mon[86379]: pgmap v264: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:08:37.350 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:08:37 smithi018 ceph-mon[112590]: pgmap v264: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:08:39.350 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:08:39 smithi018 ceph-mon[112590]: pgmap v265: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:08:39.529 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:08:39 smithi186 ceph-mon[86379]: pgmap v265: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:08:40.350 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:08:40 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T21:08:40.529 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:08:40 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T21:08:41.350 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:08:41 smithi018 ceph-mon[112590]: pgmap v266: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:08:41.529 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:08:41 smithi186 ceph-mon[86379]: pgmap v266: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:08:43.350 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:08:43 smithi018 ceph-mon[112590]: pgmap v267: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:08:43.529 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:08:43 smithi186 ceph-mon[86379]: pgmap v267: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:08:45.350 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:08:45 smithi018 ceph-mon[112590]: pgmap v268: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:08:45.529 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:08:45 smithi186 ceph-mon[86379]: pgmap v268: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:08:47.350 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:08:47 smithi018 ceph-mon[112590]: pgmap v269: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:08:47.529 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:08:47 smithi186 ceph-mon[86379]: pgmap v269: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:08:49.350 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:08:49 smithi018 ceph-mon[112590]: pgmap v270: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:08:49.529 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:08:49 smithi186 ceph-mon[86379]: pgmap v270: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:08:51.350 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:08:51 smithi018 ceph-mon[112590]: pgmap v271: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:08:51.529 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:08:51 smithi186 ceph-mon[86379]: pgmap v271: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:08:53.350 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:08:53 smithi018 ceph-mon[112590]: pgmap v272: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:08:53.529 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:08:53 smithi186 ceph-mon[86379]: pgmap v272: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:08:55.350 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:08:55 smithi018 ceph-mon[112590]: pgmap v273: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:08:55.350 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:08:55 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T21:08:55.529 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:08:55 smithi186 ceph-mon[86379]: pgmap v273: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:08:55.529 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:08:55 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T21:08:57.529 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:08:57 smithi186 ceph-mon[86379]: pgmap v274: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:08:57.600 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:08:57 smithi018 ceph-mon[112590]: pgmap v274: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:08:59.529 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:08:59 smithi186 ceph-mon[86379]: pgmap v275: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:08:59.600 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:08:59 smithi018 ceph-mon[112590]: pgmap v275: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:09:01.529 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:09:01 smithi186 ceph-mon[86379]: pgmap v276: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:09:01.600 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:09:01 smithi018 ceph-mon[112590]: pgmap v276: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:09:03.529 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:09:03 smithi186 ceph-mon[86379]: pgmap v277: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:09:03.600 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:09:03 smithi018 ceph-mon[112590]: pgmap v277: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:09:04.773 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:04.776+0000 7f2447200700 1 -- 172.21.15.18:0/3753018273 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2440102de0 msgr2=0x7f24401031b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:09:04.773 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:04.776+0000 7f2447200700 1 --2- 172.21.15.18:0/3753018273 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2440102de0 0x7f24401031b0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f2430004940 tx=0x7f2430009e30 comp rx=0 tx=0).stop 2024-04-12T21:09:04.773 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:04.777+0000 7f2447200700 1 -- 172.21.15.18:0/3753018273 shutdown_connections 2024-04-12T21:09:04.773 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:04.777+0000 7f2447200700 1 --2- 172.21.15.18:0/3753018273 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2440103780 0x7f24400fcdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:09:04.773 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:04.777+0000 7f2447200700 1 --2- 172.21.15.18:0/3753018273 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2440102de0 0x7f24401031b0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:09:04.773 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:04.777+0000 7f2447200700 1 -- 172.21.15.18:0/3753018273 >> 172.21.15.18:0/3753018273 conn(0x7f24400f8970 msgr2=0x7f24400fad80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:09:04.773 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:04.777+0000 7f2447200700 1 -- 172.21.15.18:0/3753018273 shutdown_connections 2024-04-12T21:09:04.773 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:04.777+0000 7f2447200700 1 -- 172.21.15.18:0/3753018273 wait complete. 2024-04-12T21:09:04.774 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:04.778+0000 7f2447200700 1 Processor -- start 2024-04-12T21:09:04.774 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:04.778+0000 7f2447200700 1 -- start start 2024-04-12T21:09:04.774 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:04.778+0000 7f2447200700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2440102de0 0x7f2440115d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:09:04.774 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:04.778+0000 7f2447200700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2440103780 0x7f24401162c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:09:04.774 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:04.778+0000 7f2447200700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2440116910 con 0x7f2440103780 2024-04-12T21:09:04.775 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:04.778+0000 7f2447200700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f244010fce0 con 0x7f2440102de0 2024-04-12T21:09:04.775 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:04.779+0000 7f243f400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2440103780 0x7f24401162c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:09:04.775 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:04.779+0000 7f243f400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2440103780 0x7f24401162c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:37418/0 (socket says 172.21.15.18:37418) 2024-04-12T21:09:04.775 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:04.779+0000 7f243f400700 1 -- 172.21.15.18:0/85754053 learned_addr learned my addr 172.21.15.18:0/85754053 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:09:04.775 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:04.779+0000 7f243fe00700 1 --2- 172.21.15.18:0/85754053 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2440102de0 0x7f2440115d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:09:04.776 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:04.779+0000 7f243f400700 1 -- 172.21.15.18:0/85754053 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2440102de0 msgr2=0x7f2440115d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:09:04.776 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:04.779+0000 7f243f400700 1 --2- 172.21.15.18:0/85754053 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2440102de0 0x7f2440115d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:09:04.776 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:04.779+0000 7f243f400700 1 -- 172.21.15.18:0/85754053 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2430005040 con 0x7f2440103780 2024-04-12T21:09:04.776 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:04.780+0000 7f243f400700 1 --2- 172.21.15.18:0/85754053 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2440103780 0x7f24401162c0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f242c002a50 tx=0x7f242c004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:09:04.776 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:04.780+0000 7f2437e00700 1 -- 172.21.15.18:0/85754053 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f242c007cc0 con 0x7f2440103780 2024-04-12T21:09:04.776 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:04.780+0000 7f2447200700 1 -- 172.21.15.18:0/85754053 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f244010ffc0 con 0x7f2440103780 2024-04-12T21:09:04.776 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:04.780+0000 7f2437e00700 1 -- 172.21.15.18:0/85754053 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f242c00f040 con 0x7f2440103780 2024-04-12T21:09:04.776 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:04.780+0000 7f2437e00700 1 -- 172.21.15.18:0/85754053 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f242c00da40 con 0x7f2440103780 2024-04-12T21:09:04.776 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:04.780+0000 7f2447200700 1 -- 172.21.15.18:0/85754053 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2440110580 con 0x7f2440103780 2024-04-12T21:09:04.777 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:04.781+0000 7f2447200700 1 -- 172.21.15.18:0/85754053 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f24401072e0 con 0x7f2440103780 2024-04-12T21:09:04.778 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:04.783+0000 7f2437e00700 1 -- 172.21.15.18:0/85754053 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7f242c0097c0 con 0x7f2440103780 2024-04-12T21:09:04.780 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:04.784+0000 7f2437e00700 1 --2- 172.21.15.18:0/85754053 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f242406ec00 0x7f24240710b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:09:04.780 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:04.784+0000 7f2437e00700 1 -- 172.21.15.18:0/85754053 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 6757+0+0 (secure 0 0 0) 0x7f242c08e7b0 con 0x7f2440103780 2024-04-12T21:09:04.780 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:04.784+0000 7f243fe00700 1 --2- 172.21.15.18:0/85754053 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f242406ec00 0x7f24240710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:09:04.781 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:04.785+0000 7f243fe00700 1 --2- 172.21.15.18:0/85754053 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f242406ec00 0x7f24240710b0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f2430004f90 tx=0x7f2430009e30 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:09:04.785 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:04.789+0000 7f2437e00700 1 -- 172.21.15.18:0/85754053 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f242c05acd0 con 0x7f2440103780 2024-04-12T21:09:05.042 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:05.045+0000 7f2447200700 1 -- 172.21.15.18:0/85754053 --> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f2440111440 con 0x7f242406ec00 2024-04-12T21:09:05.044 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:05.048+0000 7f2437e00700 1 -- 172.21.15.18:0/85754053 <== mgr.34102 v2:172.21.15.18:6800/3812891919 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f2440111440 con 0x7f242406ec00 2024-04-12T21:09:05.047 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:05.051+0000 7f2447200700 1 -- 172.21.15.18:0/85754053 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f242406ec00 msgr2=0x7f24240710b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:09:05.047 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:05.051+0000 7f2447200700 1 --2- 172.21.15.18:0/85754053 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f242406ec00 0x7f24240710b0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f2430004f90 tx=0x7f2430009e30 comp rx=0 tx=0).stop 2024-04-12T21:09:05.047 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:05.051+0000 7f2447200700 1 -- 172.21.15.18:0/85754053 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2440103780 msgr2=0x7f24401162c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:09:05.047 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:05.051+0000 7f2447200700 1 --2- 172.21.15.18:0/85754053 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2440103780 0x7f24401162c0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f242c002a50 tx=0x7f242c004600 comp rx=0 tx=0).stop 2024-04-12T21:09:05.048 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:05.052+0000 7f2447200700 1 -- 172.21.15.18:0/85754053 shutdown_connections 2024-04-12T21:09:05.048 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:05.052+0000 7f2447200700 1 --2- 172.21.15.18:0/85754053 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f242406ec00 0x7f24240710b0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:09:05.048 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:05.052+0000 7f2447200700 1 --2- 172.21.15.18:0/85754053 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2440102de0 0x7f2440115d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:09:05.048 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:05.052+0000 7f2447200700 1 --2- 172.21.15.18:0/85754053 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2440103780 0x7f24401162c0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:09:05.048 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:05.052+0000 7f2447200700 1 -- 172.21.15.18:0/85754053 >> 172.21.15.18:0/85754053 conn(0x7f24400f8970 msgr2=0x7f2440070c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:09:05.048 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:05.052+0000 7f2447200700 1 -- 172.21.15.18:0/85754053 shutdown_connections 2024-04-12T21:09:05.048 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:05.052+0000 7f2447200700 1 -- 172.21.15.18:0/85754053 wait complete. 2024-04-12T21:09:05.064 INFO:teuthology.orchestra.run.smithi018.stdout:true 2024-04-12T21:09:05.222 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:05.226+0000 7fec03600700 1 -- 172.21.15.18:0/2689426850 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7febfc109dd0 msgr2=0x7febfc103440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:09:05.223 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:05.226+0000 7fec03600700 1 --2- 172.21.15.18:0/2689426850 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7febfc109dd0 0x7febfc103440 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7febe8004940 tx=0x7febe8009e30 comp rx=0 tx=0).stop 2024-04-12T21:09:05.223 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:05.226+0000 7fec03600700 1 -- 172.21.15.18:0/2689426850 shutdown_connections 2024-04-12T21:09:05.223 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:05.226+0000 7fec03600700 1 --2- 172.21.15.18:0/2689426850 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7febfc109dd0 0x7febfc103440 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:09:05.223 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:05.226+0000 7fec03600700 1 --2- 172.21.15.18:0/2689426850 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7febfc109430 0x7febfc109800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:09:05.223 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:05.226+0000 7fec03600700 1 -- 172.21.15.18:0/2689426850 >> 172.21.15.18:0/2689426850 conn(0x7febfc072c00 msgr2=0x7febfc075010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:09:05.223 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:05.227+0000 7fec03600700 1 -- 172.21.15.18:0/2689426850 shutdown_connections 2024-04-12T21:09:05.223 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:05.227+0000 7fec03600700 1 -- 172.21.15.18:0/2689426850 wait complete. 2024-04-12T21:09:05.224 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:05.227+0000 7fec03600700 1 Processor -- start 2024-04-12T21:09:05.224 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:05.228+0000 7fec03600700 1 -- start start 2024-04-12T21:09:05.224 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:05.228+0000 7fec03600700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7febfc109430 0x7febfc119f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:09:05.224 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:05.228+0000 7fec03600700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7febfc109dd0 0x7febfc11a4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:09:05.224 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:05.228+0000 7fec03600700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7febfc11ab10 con 0x7febfc109430 2024-04-12T21:09:05.224 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:05.228+0000 7fec03600700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7febfc1140f0 con 0x7febfc109dd0 2024-04-12T21:09:05.225 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:05.228+0000 7fec00c00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7febfc109430 0x7febfc119f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:09:05.225 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:05.228+0000 7fec00c00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7febfc109430 0x7febfc119f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:37430/0 (socket says 172.21.15.18:37430) 2024-04-12T21:09:05.225 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:05.228+0000 7fec00c00700 1 -- 172.21.15.18:0/386855154 learned_addr learned my addr 172.21.15.18:0/386855154 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:09:05.225 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:05.229+0000 7febfbe00700 1 --2- 172.21.15.18:0/386855154 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7febfc109dd0 0x7febfc11a4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:09:05.225 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:05.229+0000 7fec00c00700 1 -- 172.21.15.18:0/386855154 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7febfc109dd0 msgr2=0x7febfc11a4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:09:05.225 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:05.229+0000 7fec00c00700 1 --2- 172.21.15.18:0/386855154 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7febfc109dd0 0x7febfc11a4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:09:05.225 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:05.229+0000 7fec00c00700 1 -- 172.21.15.18:0/386855154 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7febe8005040 con 0x7febfc109430 2024-04-12T21:09:05.226 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:05.229+0000 7fec00c00700 1 --2- 172.21.15.18:0/386855154 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7febfc109430 0x7febfc119f80 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7febf0002a50 tx=0x7febf00046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:09:05.226 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:05.229+0000 7febf8c00700 1 -- 172.21.15.18:0/386855154 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7febf0007cc0 con 0x7febfc109430 2024-04-12T21:09:05.226 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:05.230+0000 7fec03600700 1 -- 172.21.15.18:0/386855154 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7febfc1143d0 con 0x7febfc109430 2024-04-12T21:09:05.226 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:05.230+0000 7febf8c00700 1 -- 172.21.15.18:0/386855154 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7febf000f040 con 0x7febfc109430 2024-04-12T21:09:05.226 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:05.230+0000 7febf8c00700 1 -- 172.21.15.18:0/386855154 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7febf0009640 con 0x7febfc109430 2024-04-12T21:09:05.226 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:05.230+0000 7fec03600700 1 -- 172.21.15.18:0/386855154 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7febfc114990 con 0x7febfc109430 2024-04-12T21:09:05.227 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:05.231+0000 7fec03600700 1 -- 172.21.15.18:0/386855154 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7febfc104b80 con 0x7febfc109430 2024-04-12T21:09:05.227 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:05.232+0000 7febf8c00700 1 -- 172.21.15.18:0/386855154 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7febf00097a0 con 0x7febfc109430 2024-04-12T21:09:05.229 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:05.233+0000 7febf8c00700 1 --2- 172.21.15.18:0/386855154 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7febe406ec00 0x7febe40710b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:09:05.229 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:05.234+0000 7febf8c00700 1 -- 172.21.15.18:0/386855154 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 6757+0+0 (secure 0 0 0) 0x7febf008e760 con 0x7febfc109430 2024-04-12T21:09:05.229 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:05.234+0000 7febfbe00700 1 --2- 172.21.15.18:0/386855154 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7febe406ec00 0x7febe40710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:09:05.230 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:05.234+0000 7febfbe00700 1 --2- 172.21.15.18:0/386855154 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7febe406ec00 0x7febe40710b0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7febe800b3b0 tx=0x7febe8009e30 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:09:05.234 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:05.239+0000 7febf8c00700 1 -- 172.21.15.18:0/386855154 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7febf005af10 con 0x7febfc109430 2024-04-12T21:09:05.350 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:09:05 smithi018 ceph-mon[112590]: pgmap v278: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:09:05.486 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:05.491+0000 7fec03600700 1 -- 172.21.15.18:0/386855154 --> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7febfc115640 con 0x7febe406ec00 2024-04-12T21:09:05.489 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:05.493+0000 7febf8c00700 1 -- 172.21.15.18:0/386855154 <== mgr.34102 v2:172.21.15.18:6800/3812891919 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7febfc115640 con 0x7febe406ec00 2024-04-12T21:09:05.491 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:05.496+0000 7fec03600700 1 -- 172.21.15.18:0/386855154 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7febe406ec00 msgr2=0x7febe40710b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:09:05.491 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:05.496+0000 7fec03600700 1 --2- 172.21.15.18:0/386855154 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7febe406ec00 0x7febe40710b0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7febe800b3b0 tx=0x7febe8009e30 comp rx=0 tx=0).stop 2024-04-12T21:09:05.492 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:05.496+0000 7fec03600700 1 -- 172.21.15.18:0/386855154 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7febfc109430 msgr2=0x7febfc119f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:09:05.492 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:05.496+0000 7fec03600700 1 --2- 172.21.15.18:0/386855154 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7febfc109430 0x7febfc119f80 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7febf0002a50 tx=0x7febf00046b0 comp rx=0 tx=0).stop 2024-04-12T21:09:05.492 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:05.496+0000 7fec03600700 1 -- 172.21.15.18:0/386855154 shutdown_connections 2024-04-12T21:09:05.493 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:05.496+0000 7fec03600700 1 --2- 172.21.15.18:0/386855154 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7febe406ec00 0x7febe40710b0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:09:05.493 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:05.496+0000 7fec03600700 1 --2- 172.21.15.18:0/386855154 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7febfc109430 0x7febfc119f80 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:09:05.493 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:05.496+0000 7fec03600700 1 --2- 172.21.15.18:0/386855154 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7febfc109dd0 0x7febfc11a4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:09:05.493 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:05.496+0000 7fec03600700 1 -- 172.21.15.18:0/386855154 >> 172.21.15.18:0/386855154 conn(0x7febfc072c00 msgr2=0x7febfc074c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:09:05.493 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:05.497+0000 7fec03600700 1 -- 172.21.15.18:0/386855154 shutdown_connections 2024-04-12T21:09:05.493 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:05.497+0000 7fec03600700 1 -- 172.21.15.18:0/386855154 wait complete. 2024-04-12T21:09:05.529 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:09:05 smithi186 ceph-mon[86379]: pgmap v278: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:09:05.653 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:05.656+0000 7f4925e00700 1 -- 172.21.15.18:0/3852504207 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4920109430 msgr2=0x7f4920109800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:09:05.653 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:05.656+0000 7f4925e00700 1 --2- 172.21.15.18:0/3852504207 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4920109430 0x7f4920109800 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f4910004940 tx=0x7f4910009e30 comp rx=0 tx=0).stop 2024-04-12T21:09:05.653 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:05.657+0000 7f4925e00700 1 -- 172.21.15.18:0/3852504207 shutdown_connections 2024-04-12T21:09:05.653 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:05.657+0000 7f4925e00700 1 --2- 172.21.15.18:0/3852504207 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4920109dd0 0x7f4920103440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:09:05.653 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:05.657+0000 7f4925e00700 1 --2- 172.21.15.18:0/3852504207 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4920109430 0x7f4920109800 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:09:05.653 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:05.657+0000 7f4925e00700 1 -- 172.21.15.18:0/3852504207 >> 172.21.15.18:0/3852504207 conn(0x7f4920072c00 msgr2=0x7f4920075010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:09:05.653 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:05.657+0000 7f4925e00700 1 -- 172.21.15.18:0/3852504207 shutdown_connections 2024-04-12T21:09:05.653 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:05.657+0000 7f4925e00700 1 -- 172.21.15.18:0/3852504207 wait complete. 2024-04-12T21:09:05.654 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:05.658+0000 7f4925e00700 1 Processor -- start 2024-04-12T21:09:05.654 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:05.658+0000 7f4925e00700 1 -- start start 2024-04-12T21:09:05.655 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:05.659+0000 7f4925e00700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4920109430 0x7f4920119ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:09:05.655 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:05.659+0000 7f4925e00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4920109dd0 0x7f492011a400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:09:05.655 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:05.659+0000 7f4925e00700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f492011aa50 con 0x7f4920109dd0 2024-04-12T21:09:05.655 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:05.659+0000 7f4925e00700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4920114030 con 0x7f4920109430 2024-04-12T21:09:05.655 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:05.659+0000 7f491e000700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4920109dd0 0x7f492011a400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:09:05.655 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:05.659+0000 7f491e000700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4920109dd0 0x7f492011a400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:37452/0 (socket says 172.21.15.18:37452) 2024-04-12T21:09:05.656 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:05.659+0000 7f491e000700 1 -- 172.21.15.18:0/1118177810 learned_addr learned my addr 172.21.15.18:0/1118177810 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:09:05.656 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:05.659+0000 7f491ea00700 1 --2- 172.21.15.18:0/1118177810 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4920109430 0x7f4920119ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:09:05.656 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:05.660+0000 7f491e000700 1 -- 172.21.15.18:0/1118177810 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4920109430 msgr2=0x7f4920119ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:09:05.656 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:05.660+0000 7f491e000700 1 --2- 172.21.15.18:0/1118177810 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4920109430 0x7f4920119ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:09:05.656 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:05.660+0000 7f491e000700 1 -- 172.21.15.18:0/1118177810 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4910005040 con 0x7f4920109dd0 2024-04-12T21:09:05.656 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:05.660+0000 7f491ea00700 1 --2- 172.21.15.18:0/1118177810 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4920109430 0x7f4920119ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-12T21:09:05.656 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:05.660+0000 7f491e000700 1 --2- 172.21.15.18:0/1118177810 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4920109dd0 0x7f492011a400 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f490c00eb40 tx=0x7f490c00b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:09:05.657 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:05.660+0000 7f4916a00700 1 -- 172.21.15.18:0/1118177810 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f490c007cc0 con 0x7f4920109dd0 2024-04-12T21:09:05.657 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:05.660+0000 7f4916a00700 1 -- 172.21.15.18:0/1118177810 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f490c00f040 con 0x7f4920109dd0 2024-04-12T21:09:05.657 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:05.660+0000 7f4916a00700 1 -- 172.21.15.18:0/1118177810 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f490c0095e0 con 0x7f4920109dd0 2024-04-12T21:09:05.657 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:05.660+0000 7f4925e00700 1 -- 172.21.15.18:0/1118177810 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f49201142b0 con 0x7f4920109dd0 2024-04-12T21:09:05.657 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:05.661+0000 7f4925e00700 1 -- 172.21.15.18:0/1118177810 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f49201147f0 con 0x7f4920109dd0 2024-04-12T21:09:05.657 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:05.662+0000 7f4925e00700 1 -- 172.21.15.18:0/1118177810 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4920104b80 con 0x7f4920109dd0 2024-04-12T21:09:05.658 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:05.663+0000 7f4916a00700 1 -- 172.21.15.18:0/1118177810 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7f490c009740 con 0x7f4920109dd0 2024-04-12T21:09:05.659 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:05.664+0000 7f4916a00700 1 --2- 172.21.15.18:0/1118177810 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f490406ec00 0x7f49040710b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:09:05.659 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:05.664+0000 7f4916a00700 1 -- 172.21.15.18:0/1118177810 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 6757+0+0 (secure 0 0 0) 0x7f490c08e430 con 0x7f4920109dd0 2024-04-12T21:09:05.660 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:05.664+0000 7f491ea00700 1 --2- 172.21.15.18:0/1118177810 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f490406ec00 0x7f49040710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:09:05.660 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:05.665+0000 7f491ea00700 1 --2- 172.21.15.18:0/1118177810 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f490406ec00 0x7f49040710b0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f4910007f50 tx=0x7f4910009e30 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:09:05.664 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:05.669+0000 7f4916a00700 1 -- 172.21.15.18:0/1118177810 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f490c05ab60 con 0x7f4920109dd0 2024-04-12T21:09:05.911 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:05.915+0000 7f4925e00700 1 -- 172.21.15.18:0/1118177810 --> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f4920110590 con 0x7f490406ec00 2024-04-12T21:09:05.943 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:05.947+0000 7f4916a00700 1 -- 172.21.15.18:0/1118177810 <== mgr.34102 v2:172.21.15.18:6800/3812891919 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3520 (secure 0 0 0) 0x7f4920110590 con 0x7f490406ec00 2024-04-12T21:09:05.944 INFO:teuthology.orchestra.run.smithi018.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-12T21:09:05.944 INFO:teuthology.orchestra.run.smithi018.stdout:alertmanager.smithi018 smithi018 *:9093,9094 running (9m) 80s ago 24m 14.7M - 0.25.0 c8568f914cd2 c332090068e8 2024-04-12T21:09:05.944 INFO:teuthology.orchestra.run.smithi018.stdout:crash.smithi018 smithi018 running (6m) 80s ago 24m 6665k - 19.0.0-2879-g324baf8d 8717720dc5e5 b7f80d1f7982 2024-04-12T21:09:05.944 INFO:teuthology.orchestra.run.smithi018.stdout:crash.smithi186 smithi186 running (5m) 2m ago 23m 6660k - 19.0.0-2879-g324baf8d 8717720dc5e5 a4f55c81b623 2024-04-12T21:09:05.944 INFO:teuthology.orchestra.run.smithi018.stdout:grafana.smithi018 smithi018 *:3000 running (9m) 80s ago 23m 74.8M - 9.4.12 2bacad6d85d8 b08e3bf176dd 2024-04-12T21:09:05.944 INFO:teuthology.orchestra.run.smithi018.stdout:mds.cephfs.smithi018.acqbsc smithi018 running (116s) 80s ago 17m 11.1M - 19.0.0-2879-g324baf8d 8717720dc5e5 021f99ab24ae 2024-04-12T21:09:05.944 INFO:teuthology.orchestra.run.smithi018.stdout:mds.cephfs.smithi018.szjluo smithi018 running (88s) 80s ago 16m 10.9M - 19.0.0-2879-g324baf8d 8717720dc5e5 0003117479d4 2024-04-12T21:09:05.944 INFO:teuthology.orchestra.run.smithi018.stdout:mds.cephfs.smithi186.jdllyn smithi186 running (17m) 2m ago 16m 15.6M - 17.2.7-827-gd3b5060a 21de9b7369ad 2b2bc46bd29a 2024-04-12T21:09:05.944 INFO:teuthology.orchestra.run.smithi018.stdout:mds.cephfs.smithi186.wjnafs smithi186 running (16m) 2m ago 16m 15.4M - 17.2.7-827-gd3b5060a 21de9b7369ad 74b115d6d6fd 2024-04-12T21:09:05.944 INFO:teuthology.orchestra.run.smithi018.stdout:mgr.smithi018.xjzwga smithi018 *:8443,9283,8765 running (12m) 80s ago 25m 509M - 19.0.0-2879-g324baf8d 8717720dc5e5 56088305eab7 2024-04-12T21:09:05.944 INFO:teuthology.orchestra.run.smithi018.stdout:mgr.smithi186.kqcfxk smithi186 *:8443,9283,8765 running (11m) 2m ago 22m 452M - 19.0.0-2879-g324baf8d 8717720dc5e5 09cd766e5e0d 2024-04-12T21:09:05.945 INFO:teuthology.orchestra.run.smithi018.stdout:mon.smithi018 smithi018 running (7m) 80s ago 26m 57.0M 2048M 19.0.0-2879-g324baf8d 8717720dc5e5 df3e201dfed6 2024-04-12T21:09:05.945 INFO:teuthology.orchestra.run.smithi018.stdout:mon.smithi186 smithi186 running (6m) 2m ago 22m 37.7M 2048M 19.0.0-2879-g324baf8d 8717720dc5e5 c77498da9b2b 2024-04-12T21:09:05.945 INFO:teuthology.orchestra.run.smithi018.stdout:node-exporter.smithi018 smithi018 *:9100 running (10m) 80s ago 24m 7780k - 1.5.0 0da6a335fe13 326c3d9c6847 2024-04-12T21:09:05.945 INFO:teuthology.orchestra.run.smithi018.stdout:node-exporter.smithi186 smithi186 *:9100 running (10m) 2m ago 22m 7826k - 1.5.0 0da6a335fe13 ec2112baeecf 2024-04-12T21:09:05.945 INFO:teuthology.orchestra.run.smithi018.stdout:osd.0 smithi018 running (5m) 80s ago 21m 121M 1559M 19.0.0-2879-g324baf8d 8717720dc5e5 60b1273ca047 2024-04-12T21:09:05.945 INFO:teuthology.orchestra.run.smithi018.stdout:osd.1 smithi018 running (4m) 80s ago 21m 91.3M 1559M 19.0.0-2879-g324baf8d 8717720dc5e5 170ce1e5fb8d 2024-04-12T21:09:05.945 INFO:teuthology.orchestra.run.smithi018.stdout:osd.2 smithi018 running (4m) 80s ago 20m 90.3M 1559M 19.0.0-2879-g324baf8d 8717720dc5e5 c2cbc6aa2839 2024-04-12T21:09:05.945 INFO:teuthology.orchestra.run.smithi018.stdout:osd.3 smithi186 running (3m) 2m ago 20m 96.6M 2581M 19.0.0-2879-g324baf8d 8717720dc5e5 0138e8262e4b 2024-04-12T21:09:05.945 INFO:teuthology.orchestra.run.smithi018.stdout:osd.4 smithi186 running (3m) 2m ago 19m 87.2M 2581M 19.0.0-2879-g324baf8d 8717720dc5e5 b20cb63b25c9 2024-04-12T21:09:05.945 INFO:teuthology.orchestra.run.smithi018.stdout:osd.5 smithi186 running (2m) 2m ago 18m 11.1M 2581M 19.0.0-2879-g324baf8d 8717720dc5e5 2513bea2610e 2024-04-12T21:09:05.945 INFO:teuthology.orchestra.run.smithi018.stdout:prometheus.smithi018 smithi018 *:9095 running (10m) 80s ago 23m 60.6M - 2.43.0 a07b618ecd1d f58841127e21 2024-04-12T21:09:05.946 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:05.950+0000 7f4925e00700 1 -- 172.21.15.18:0/1118177810 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f490406ec00 msgr2=0x7f49040710b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:09:05.946 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:05.950+0000 7f4925e00700 1 --2- 172.21.15.18:0/1118177810 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f490406ec00 0x7f49040710b0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f4910007f50 tx=0x7f4910009e30 comp rx=0 tx=0).stop 2024-04-12T21:09:05.946 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:05.950+0000 7f4925e00700 1 -- 172.21.15.18:0/1118177810 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4920109dd0 msgr2=0x7f492011a400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:09:05.946 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:05.950+0000 7f4925e00700 1 --2- 172.21.15.18:0/1118177810 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4920109dd0 0x7f492011a400 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f490c00eb40 tx=0x7f490c00b040 comp rx=0 tx=0).stop 2024-04-12T21:09:05.946 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:05.951+0000 7f4925e00700 1 -- 172.21.15.18:0/1118177810 shutdown_connections 2024-04-12T21:09:05.947 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:05.951+0000 7f4925e00700 1 --2- 172.21.15.18:0/1118177810 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f490406ec00 0x7f49040710b0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:09:05.947 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:05.951+0000 7f4925e00700 1 --2- 172.21.15.18:0/1118177810 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4920109430 0x7f4920119ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:09:05.947 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:05.951+0000 7f4925e00700 1 --2- 172.21.15.18:0/1118177810 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4920109dd0 0x7f492011a400 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:09:05.947 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:05.951+0000 7f4925e00700 1 -- 172.21.15.18:0/1118177810 >> 172.21.15.18:0/1118177810 conn(0x7f4920072c00 msgr2=0x7f4920074bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:09:05.947 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:05.951+0000 7f4925e00700 1 -- 172.21.15.18:0/1118177810 shutdown_connections 2024-04-12T21:09:05.947 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:05.951+0000 7f4925e00700 1 -- 172.21.15.18:0/1118177810 wait complete. 2024-04-12T21:09:06.106 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:06.110+0000 7f01ed000700 1 -- 172.21.15.18:0/1437254621 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f01e81059c0 msgr2=0x7f01e80ff080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:09:06.106 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:06.110+0000 7f01ed000700 1 --2- 172.21.15.18:0/1437254621 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f01e81059c0 0x7f01e80ff080 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f01d0004940 tx=0x7f01d0009e30 comp rx=0 tx=0).stop 2024-04-12T21:09:06.106 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:06.110+0000 7f01ed000700 1 -- 172.21.15.18:0/1437254621 shutdown_connections 2024-04-12T21:09:06.107 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:06.110+0000 7f01ed000700 1 --2- 172.21.15.18:0/1437254621 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f01e81059c0 0x7f01e80ff080 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:09:06.107 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:06.110+0000 7f01ed000700 1 --2- 172.21.15.18:0/1437254621 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f01e8105020 0x7f01e81053f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:09:06.107 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:06.110+0000 7f01ed000700 1 -- 172.21.15.18:0/1437254621 >> 172.21.15.18:0/1437254621 conn(0x7f01e80faba0 msgr2=0x7f01e80fcfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:09:06.107 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:06.110+0000 7f01ed000700 1 -- 172.21.15.18:0/1437254621 shutdown_connections 2024-04-12T21:09:06.107 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:06.111+0000 7f01ed000700 1 -- 172.21.15.18:0/1437254621 wait complete. 2024-04-12T21:09:06.108 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:06.111+0000 7f01ed000700 1 Processor -- start 2024-04-12T21:09:06.108 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:06.111+0000 7f01ed000700 1 -- start start 2024-04-12T21:09:06.108 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:06.112+0000 7f01ed000700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f01e8105020 0x7f01e8115ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:09:06.108 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:06.112+0000 7f01ed000700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f01e81059c0 0x7f01e81160e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:09:06.108 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:06.112+0000 7f01ed000700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01e8116730 con 0x7f01e8105020 2024-04-12T21:09:06.108 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:06.112+0000 7f01ed000700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01e810fd10 con 0x7f01e81059c0 2024-04-12T21:09:06.108 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:06.112+0000 7f01e5e00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f01e8105020 0x7f01e8115ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:09:06.109 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:06.112+0000 7f01e5e00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f01e8105020 0x7f01e8115ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:37476/0 (socket says 172.21.15.18:37476) 2024-04-12T21:09:06.109 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:06.112+0000 7f01e5e00700 1 -- 172.21.15.18:0/3147770208 learned_addr learned my addr 172.21.15.18:0/3147770208 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:09:06.109 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:06.113+0000 7f01dd400700 1 --2- 172.21.15.18:0/3147770208 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f01e81059c0 0x7f01e81160e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:09:06.109 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:06.113+0000 7f01e5e00700 1 -- 172.21.15.18:0/3147770208 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f01e81059c0 msgr2=0x7f01e81160e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:09:06.109 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:06.113+0000 7f01e5e00700 1 --2- 172.21.15.18:0/3147770208 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f01e81059c0 0x7f01e81160e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:09:06.109 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:06.113+0000 7f01e5e00700 1 -- 172.21.15.18:0/3147770208 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01d0005040 con 0x7f01e8105020 2024-04-12T21:09:06.110 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:06.113+0000 7f01e5e00700 1 --2- 172.21.15.18:0/3147770208 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f01e8105020 0x7f01e8115ba0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f01d800c830 tx=0x7f01d8004470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:09:06.110 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:06.113+0000 7f01dea00700 1 -- 172.21.15.18:0/3147770208 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f01d800a760 con 0x7f01e8105020 2024-04-12T21:09:06.110 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:06.113+0000 7f01dea00700 1 -- 172.21.15.18:0/3147770208 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f01d800ad90 con 0x7f01e8105020 2024-04-12T21:09:06.110 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:06.113+0000 7f01dea00700 1 -- 172.21.15.18:0/3147770208 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f01d80113a0 con 0x7f01e8105020 2024-04-12T21:09:06.110 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:06.113+0000 7f01ed000700 1 -- 172.21.15.18:0/3147770208 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f01e810fff0 con 0x7f01e8105020 2024-04-12T21:09:06.110 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:06.114+0000 7f01ed000700 1 -- 172.21.15.18:0/3147770208 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f01e81105b0 con 0x7f01e8105020 2024-04-12T21:09:06.110 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:06.115+0000 7f01ed000700 1 -- 172.21.15.18:0/3147770208 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f01e8100770 con 0x7f01e8105020 2024-04-12T21:09:06.111 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:06.116+0000 7f01dea00700 1 -- 172.21.15.18:0/3147770208 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7f01d8012400 con 0x7f01e8105020 2024-04-12T21:09:06.112 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:06.117+0000 7f01dea00700 1 --2- 172.21.15.18:0/3147770208 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f01cc06ec00 0x7f01cc0710b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:09:06.112 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:06.117+0000 7f01dea00700 1 -- 172.21.15.18:0/3147770208 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 6757+0+0 (secure 0 0 0) 0x7f01d808ea00 con 0x7f01e8105020 2024-04-12T21:09:06.112 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:06.117+0000 7f01dd400700 1 --2- 172.21.15.18:0/3147770208 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f01cc06ec00 0x7f01cc0710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:09:06.113 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:06.118+0000 7f01dd400700 1 --2- 172.21.15.18:0/3147770208 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f01cc06ec00 0x7f01cc0710b0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f01d000b020 tx=0x7f01d0009e30 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:09:06.117 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:06.122+0000 7f01dea00700 1 -- 172.21.15.18:0/3147770208 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f01d805b250 con 0x7f01e8105020 2024-04-12T21:09:06.350 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:09:06 smithi018 ceph-mon[112590]: from='client.34528 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:09:06.350 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:09:06 smithi018 ceph-mon[112590]: from='client.34532 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:09:06.438 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:06.442+0000 7f01ed000700 1 -- 172.21.15.18:0/3147770208 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f01e81111f0 con 0x7f01e8105020 2024-04-12T21:09:06.439 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:06.443+0000 7f01dea00700 1 -- 172.21.15.18:0/3147770208 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+837 (secure 0 0 0) 0x7f01d805ade0 con 0x7f01e8105020 2024-04-12T21:09:06.439 INFO:teuthology.orchestra.run.smithi018.stdout:{ 2024-04-12T21:09:06.439 INFO:teuthology.orchestra.run.smithi018.stdout: "mon": { 2024-04-12T21:09:06.439 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 19.0.0-2879-g324baf8d (324baf8ddb91dea46cac50cb535c306115791355) squid (dev)": 2 2024-04-12T21:09:06.439 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-12T21:09:06.439 INFO:teuthology.orchestra.run.smithi018.stdout: "mgr": { 2024-04-12T21:09:06.439 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 19.0.0-2879-g324baf8d (324baf8ddb91dea46cac50cb535c306115791355) squid (dev)": 2 2024-04-12T21:09:06.439 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-12T21:09:06.440 INFO:teuthology.orchestra.run.smithi018.stdout: "osd": { 2024-04-12T21:09:06.440 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 19.0.0-2879-g324baf8d (324baf8ddb91dea46cac50cb535c306115791355) squid (dev)": 6 2024-04-12T21:09:06.440 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-12T21:09:06.440 INFO:teuthology.orchestra.run.smithi018.stdout: "mds": { 2024-04-12T21:09:06.440 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 17.2.7-827-gd3b5060a (d3b5060a17449edfae9385cdf792afec109fca5d) quincy (stable)": 2, 2024-04-12T21:09:06.440 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 19.0.0-2879-g324baf8d (324baf8ddb91dea46cac50cb535c306115791355) squid (dev)": 2 2024-04-12T21:09:06.440 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-12T21:09:06.440 INFO:teuthology.orchestra.run.smithi018.stdout: "overall": { 2024-04-12T21:09:06.440 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 17.2.7-827-gd3b5060a (d3b5060a17449edfae9385cdf792afec109fca5d) quincy (stable)": 2, 2024-04-12T21:09:06.440 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 19.0.0-2879-g324baf8d (324baf8ddb91dea46cac50cb535c306115791355) squid (dev)": 12 2024-04-12T21:09:06.440 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-04-12T21:09:06.440 INFO:teuthology.orchestra.run.smithi018.stdout:} 2024-04-12T21:09:06.441 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:06.446+0000 7f01ed000700 1 -- 172.21.15.18:0/3147770208 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f01cc06ec00 msgr2=0x7f01cc0710b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:09:06.442 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:06.446+0000 7f01ed000700 1 --2- 172.21.15.18:0/3147770208 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f01cc06ec00 0x7f01cc0710b0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f01d000b020 tx=0x7f01d0009e30 comp rx=0 tx=0).stop 2024-04-12T21:09:06.442 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:06.446+0000 7f01ed000700 1 -- 172.21.15.18:0/3147770208 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f01e8105020 msgr2=0x7f01e8115ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:09:06.442 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:06.446+0000 7f01ed000700 1 --2- 172.21.15.18:0/3147770208 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f01e8105020 0x7f01e8115ba0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f01d800c830 tx=0x7f01d8004470 comp rx=0 tx=0).stop 2024-04-12T21:09:06.442 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:06.446+0000 7f01ed000700 1 -- 172.21.15.18:0/3147770208 shutdown_connections 2024-04-12T21:09:06.442 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:06.446+0000 7f01ed000700 1 --2- 172.21.15.18:0/3147770208 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f01cc06ec00 0x7f01cc0710b0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:09:06.442 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:06.446+0000 7f01ed000700 1 --2- 172.21.15.18:0/3147770208 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f01e8105020 0x7f01e8115ba0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:09:06.442 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:06.446+0000 7f01ed000700 1 --2- 172.21.15.18:0/3147770208 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f01e81059c0 0x7f01e81160e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:09:06.442 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:06.446+0000 7f01ed000700 1 -- 172.21.15.18:0/3147770208 >> 172.21.15.18:0/3147770208 conn(0x7f01e80faba0 msgr2=0x7f01e80fbb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:09:06.443 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:06.446+0000 7f01ed000700 1 -- 172.21.15.18:0/3147770208 shutdown_connections 2024-04-12T21:09:06.443 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:06.447+0000 7f01ed000700 1 -- 172.21.15.18:0/3147770208 wait complete. 2024-04-12T21:09:06.529 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:09:06 smithi186 ceph-mon[86379]: from='client.34528 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:09:06.529 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:09:06 smithi186 ceph-mon[86379]: from='client.34532 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:09:06.590 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:06.594+0000 7f9cdcc00700 1 -- 172.21.15.18:0/2170825697 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9cd81092a0 msgr2=0x7f9cd8109670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:09:06.590 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:06.594+0000 7f9cdcc00700 1 --2- 172.21.15.18:0/2170825697 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9cd81092a0 0x7f9cd8109670 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f9cc8004940 tx=0x7f9cc8009e30 comp rx=0 tx=0).stop 2024-04-12T21:09:06.590 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:06.594+0000 7f9cdcc00700 1 -- 172.21.15.18:0/2170825697 shutdown_connections 2024-04-12T21:09:06.590 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:06.594+0000 7f9cdcc00700 1 --2- 172.21.15.18:0/2170825697 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9cd8109c40 0x7f9cd81032b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:09:06.590 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:06.594+0000 7f9cdcc00700 1 --2- 172.21.15.18:0/2170825697 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9cd81092a0 0x7f9cd8109670 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:09:06.590 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:06.594+0000 7f9cdcc00700 1 -- 172.21.15.18:0/2170825697 >> 172.21.15.18:0/2170825697 conn(0x7f9cd8072a30 msgr2=0x7f9cd8074e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:09:06.590 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:06.594+0000 7f9cdcc00700 1 -- 172.21.15.18:0/2170825697 shutdown_connections 2024-04-12T21:09:06.590 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:06.594+0000 7f9cdcc00700 1 -- 172.21.15.18:0/2170825697 wait complete. 2024-04-12T21:09:06.592 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:06.595+0000 7f9cdcc00700 1 Processor -- start 2024-04-12T21:09:06.592 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:06.595+0000 7f9cdcc00700 1 -- start start 2024-04-12T21:09:06.592 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:06.595+0000 7f9cdcc00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9cd8109c40 0x7f9cd811a180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:09:06.592 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:06.595+0000 7f9cdcc00700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9cd811a6c0 0x7f9cd81140e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:09:06.592 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:06.595+0000 7f9cdcc00700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9cd811acd0 con 0x7f9cd8109c40 2024-04-12T21:09:06.592 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:06.595+0000 7f9cdcc00700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9cd8114620 con 0x7f9cd811a6c0 2024-04-12T21:09:06.592 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:06.596+0000 7f9cd7400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9cd8109c40 0x7f9cd811a180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:09:06.592 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:06.596+0000 7f9cd7400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9cd8109c40 0x7f9cd811a180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:37494/0 (socket says 172.21.15.18:37494) 2024-04-12T21:09:06.592 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:06.596+0000 7f9cd7400700 1 -- 172.21.15.18:0/1360639453 learned_addr learned my addr 172.21.15.18:0/1360639453 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:09:06.592 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:06.596+0000 7f9cd6a00700 1 --2- 172.21.15.18:0/1360639453 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9cd811a6c0 0x7f9cd81140e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:09:06.592 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:06.596+0000 7f9cd7400700 1 -- 172.21.15.18:0/1360639453 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9cd811a6c0 msgr2=0x7f9cd81140e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:09:06.593 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:06.596+0000 7f9cd7400700 1 --2- 172.21.15.18:0/1360639453 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9cd811a6c0 0x7f9cd81140e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:09:06.593 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:06.596+0000 7f9cd7400700 1 -- 172.21.15.18:0/1360639453 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9cc8005040 con 0x7f9cd8109c40 2024-04-12T21:09:06.593 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:06.596+0000 7f9cd6a00700 1 --2- 172.21.15.18:0/1360639453 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9cd811a6c0 0x7f9cd81140e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-12T21:09:06.593 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:06.596+0000 7f9cd7400700 1 --2- 172.21.15.18:0/1360639453 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9cd8109c40 0x7f9cd811a180 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f9cc800b3b0 tx=0x7f9cc8009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:09:06.593 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:06.597+0000 7f9ccf400700 1 -- 172.21.15.18:0/1360639453 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9cc800e040 con 0x7f9cd8109c40 2024-04-12T21:09:06.594 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:06.597+0000 7f9ccf400700 1 -- 172.21.15.18:0/1360639453 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f9cc8009430 con 0x7f9cd8109c40 2024-04-12T21:09:06.594 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:06.597+0000 7f9cdcc00700 1 -- 172.21.15.18:0/1360639453 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9cd81148a0 con 0x7f9cd8109c40 2024-04-12T21:09:06.594 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:06.597+0000 7f9ccf400700 1 -- 172.21.15.18:0/1360639453 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9cc8013e90 con 0x7f9cd8109c40 2024-04-12T21:09:06.594 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:06.597+0000 7f9cdcc00700 1 -- 172.21.15.18:0/1360639453 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9cd8114e60 con 0x7f9cd8109c40 2024-04-12T21:09:06.594 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:06.598+0000 7f9cdcc00700 1 -- 172.21.15.18:0/1360639453 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9cd81049f0 con 0x7f9cd8109c40 2024-04-12T21:09:06.594 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:06.599+0000 7f9ccf400700 1 -- 172.21.15.18:0/1360639453 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7f9cc8013640 con 0x7f9cd8109c40 2024-04-12T21:09:06.596 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:06.600+0000 7f9ccf400700 1 --2- 172.21.15.18:0/1360639453 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f9cbc06ec00 0x7f9cbc0710b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:09:06.596 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:06.600+0000 7f9ccf400700 1 -- 172.21.15.18:0/1360639453 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 6757+0+0 (secure 0 0 0) 0x7f9cc808fd00 con 0x7f9cd8109c40 2024-04-12T21:09:06.596 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:06.600+0000 7f9cd6a00700 1 --2- 172.21.15.18:0/1360639453 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f9cbc06ec00 0x7f9cbc0710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:09:06.596 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:06.601+0000 7f9cd6a00700 1 --2- 172.21.15.18:0/1360639453 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f9cbc06ec00 0x7f9cbc0710b0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f9cc0006fd0 tx=0x7f9cc0008040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:09:06.600 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:06.605+0000 7f9ccf400700 1 -- 172.21.15.18:0/1360639453 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f9cc8093030 con 0x7f9cd8109c40 2024-04-12T21:09:06.873 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:06.877+0000 7f9cdcc00700 1 -- 172.21.15.18:0/1360639453 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f9cd804bed0 con 0x7f9cd8109c40 2024-04-12T21:09:06.873 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:06.878+0000 7f9ccf400700 1 -- 172.21.15.18:0/1360639453 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 29 v29) v1 ==== 76+0+1872 (secure 0 0 0) 0x7f9cc805c550 con 0x7f9cd8109c40 2024-04-12T21:09:06.874 INFO:teuthology.orchestra.run.smithi018.stdout:e29 2024-04-12T21:09:06.874 INFO:teuthology.orchestra.run.smithi018.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-12T21:09:06.874 INFO:teuthology.orchestra.run.smithi018.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-12T21:09:06.874 INFO:teuthology.orchestra.run.smithi018.stdout:legacy client fscid: 1 2024-04-12T21:09:06.874 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-12T21:09:06.874 INFO:teuthology.orchestra.run.smithi018.stdout:Filesystem 'cephfs' (1) 2024-04-12T21:09:06.875 INFO:teuthology.orchestra.run.smithi018.stdout:fs_name cephfs 2024-04-12T21:09:06.875 INFO:teuthology.orchestra.run.smithi018.stdout:epoch 24 2024-04-12T21:09:06.875 INFO:teuthology.orchestra.run.smithi018.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-04-12T21:09:06.875 INFO:teuthology.orchestra.run.smithi018.stdout:created 2024-04-12T20:51:57.362850+0000 2024-04-12T21:09:06.875 INFO:teuthology.orchestra.run.smithi018.stdout:modified 2024-04-12T21:06:59.717190+0000 2024-04-12T21:09:06.875 INFO:teuthology.orchestra.run.smithi018.stdout:tableserver 0 2024-04-12T21:09:06.875 INFO:teuthology.orchestra.run.smithi018.stdout:root 0 2024-04-12T21:09:06.875 INFO:teuthology.orchestra.run.smithi018.stdout:session_timeout 60 2024-04-12T21:09:06.875 INFO:teuthology.orchestra.run.smithi018.stdout:session_autoclose 300 2024-04-12T21:09:06.875 INFO:teuthology.orchestra.run.smithi018.stdout:max_file_size 1099511627776 2024-04-12T21:09:06.875 INFO:teuthology.orchestra.run.smithi018.stdout:max_xattr_size 65536 2024-04-12T21:09:06.875 INFO:teuthology.orchestra.run.smithi018.stdout:required_client_features {} 2024-04-12T21:09:06.875 INFO:teuthology.orchestra.run.smithi018.stdout:last_failure 0 2024-04-12T21:09:06.875 INFO:teuthology.orchestra.run.smithi018.stdout:last_failure_osd_epoch 100 2024-04-12T21:09:06.876 INFO:teuthology.orchestra.run.smithi018.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-12T21:09:06.876 INFO:teuthology.orchestra.run.smithi018.stdout:max_mds 2 2024-04-12T21:09:06.876 INFO:teuthology.orchestra.run.smithi018.stdout:in 0,1 2024-04-12T21:09:06.876 INFO:teuthology.orchestra.run.smithi018.stdout:up {} 2024-04-12T21:09:06.876 INFO:teuthology.orchestra.run.smithi018.stdout:failed 0,1 2024-04-12T21:09:06.876 INFO:teuthology.orchestra.run.smithi018.stdout:damaged 2024-04-12T21:09:06.876 INFO:teuthology.orchestra.run.smithi018.stdout:stopped 2024-04-12T21:09:06.876 INFO:teuthology.orchestra.run.smithi018.stdout:data_pools [3] 2024-04-12T21:09:06.876 INFO:teuthology.orchestra.run.smithi018.stdout:metadata_pool 2 2024-04-12T21:09:06.876 INFO:teuthology.orchestra.run.smithi018.stdout:inline_data disabled 2024-04-12T21:09:06.876 INFO:teuthology.orchestra.run.smithi018.stdout:balancer 2024-04-12T21:09:06.876 INFO:teuthology.orchestra.run.smithi018.stdout:bal_rank_mask -1 2024-04-12T21:09:06.876 INFO:teuthology.orchestra.run.smithi018.stdout:standby_count_wanted 1 2024-04-12T21:09:06.876 INFO:teuthology.orchestra.run.smithi018.stdout:qdb_cluster leader: 0 members: 2024-04-12T21:09:06.876 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-12T21:09:06.877 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-12T21:09:06.877 INFO:teuthology.orchestra.run.smithi018.stdout:Standby daemons: 2024-04-12T21:09:06.877 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-12T21:09:06.877 INFO:teuthology.orchestra.run.smithi018.stdout:[mds.cephfs.smithi186.wjnafs{-1:24277} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.186:6826/438347207,v1:172.21.15.186:6827/438347207] compat {c=[1],r=[1],i=[7ff]}] 2024-04-12T21:09:06.877 INFO:teuthology.orchestra.run.smithi018.stdout:[mds.cephfs.smithi186.jdllyn{-1:24281} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.186:6824/1464359952,v1:172.21.15.186:6825/1464359952] compat {c=[1],r=[1],i=[7ff]}] 2024-04-12T21:09:06.877 INFO:teuthology.orchestra.run.smithi018.stdout:[mds.cephfs.smithi018.acqbsc{-1:34436} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.18:6828/755179764,v1:172.21.15.18:6829/755179764] compat {c=[1],r=[1],i=[1fff]}] 2024-04-12T21:09:06.877 INFO:teuthology.orchestra.run.smithi018.stdout:[mds.cephfs.smithi018.szjluo{-1:34468} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.18:6826/2407992653,v1:172.21.15.18:6827/2407992653] compat {c=[1],r=[1],i=[1fff]}] 2024-04-12T21:09:06.878 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:06.881+0000 7f9cdcc00700 1 -- 172.21.15.18:0/1360639453 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f9cbc06ec00 msgr2=0x7f9cbc0710b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:09:06.878 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:06.881+0000 7f9cdcc00700 1 --2- 172.21.15.18:0/1360639453 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f9cbc06ec00 0x7f9cbc0710b0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f9cc0006fd0 tx=0x7f9cc0008040 comp rx=0 tx=0).stop 2024-04-12T21:09:06.878 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:06.881+0000 7f9cdcc00700 1 -- 172.21.15.18:0/1360639453 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9cd8109c40 msgr2=0x7f9cd811a180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:09:06.878 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:06.881+0000 7f9cdcc00700 1 --2- 172.21.15.18:0/1360639453 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9cd8109c40 0x7f9cd811a180 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f9cc800b3b0 tx=0x7f9cc8009e30 comp rx=0 tx=0).stop 2024-04-12T21:09:06.878 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:06.881+0000 7f9cdcc00700 1 -- 172.21.15.18:0/1360639453 shutdown_connections 2024-04-12T21:09:06.878 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:06.881+0000 7f9cdcc00700 1 --2- 172.21.15.18:0/1360639453 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f9cbc06ec00 0x7f9cbc0710b0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:09:06.878 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:06.881+0000 7f9cdcc00700 1 --2- 172.21.15.18:0/1360639453 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9cd8109c40 0x7f9cd811a180 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:09:06.878 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:06.881+0000 7f9cdcc00700 1 --2- 172.21.15.18:0/1360639453 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9cd811a6c0 0x7f9cd81140e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:09:06.878 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:06.881+0000 7f9cdcc00700 1 -- 172.21.15.18:0/1360639453 >> 172.21.15.18:0/1360639453 conn(0x7f9cd8072a30 msgr2=0x7f9cd80749e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:09:06.878 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:06.881+0000 7f9cdcc00700 1 -- 172.21.15.18:0/1360639453 shutdown_connections 2024-04-12T21:09:06.878 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:06.881+0000 7f9cdcc00700 1 -- 172.21.15.18:0/1360639453 wait complete. 2024-04-12T21:09:06.879 INFO:teuthology.orchestra.run.smithi018.stderr:dumped fsmap epoch 29 2024-04-12T21:09:07.038 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:07.042+0000 7feb58e00700 1 -- 172.21.15.18:0/103440226 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7feb540fda60 msgr2=0x7feb540fde30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:09:07.038 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:07.042+0000 7feb58e00700 1 --2- 172.21.15.18:0/103440226 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7feb540fda60 0x7feb540fde30 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7feb44004940 tx=0x7feb44009e30 comp rx=0 tx=0).stop 2024-04-12T21:09:07.038 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:07.042+0000 7feb58e00700 1 -- 172.21.15.18:0/103440226 shutdown_connections 2024-04-12T21:09:07.038 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:07.042+0000 7feb58e00700 1 --2- 172.21.15.18:0/103440226 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7feb540fb250 0x7feb540fb6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:09:07.038 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:07.042+0000 7feb58e00700 1 --2- 172.21.15.18:0/103440226 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7feb540fda60 0x7feb540fde30 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:09:07.038 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:07.042+0000 7feb58e00700 1 -- 172.21.15.18:0/103440226 >> 172.21.15.18:0/103440226 conn(0x7feb540721b0 msgr2=0x7feb540725b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:09:07.038 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:07.042+0000 7feb58e00700 1 -- 172.21.15.18:0/103440226 shutdown_connections 2024-04-12T21:09:07.038 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:07.042+0000 7feb58e00700 1 -- 172.21.15.18:0/103440226 wait complete. 2024-04-12T21:09:07.039 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:07.043+0000 7feb58e00700 1 Processor -- start 2024-04-12T21:09:07.039 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:07.043+0000 7feb58e00700 1 -- start start 2024-04-12T21:09:07.040 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:07.044+0000 7feb58e00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7feb540fb250 0x7feb54115bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:09:07.040 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:07.044+0000 7feb58e00700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7feb540fda60 0x7feb541160f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:09:07.040 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:07.044+0000 7feb58e00700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb54116740 con 0x7feb540fb250 2024-04-12T21:09:07.040 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:07.044+0000 7feb58e00700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb5410fb30 con 0x7feb540fda60 2024-04-12T21:09:07.040 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:07.044+0000 7feb53400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7feb540fb250 0x7feb54115bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:09:07.040 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:07.044+0000 7feb53400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7feb540fb250 0x7feb54115bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:37512/0 (socket says 172.21.15.18:37512) 2024-04-12T21:09:07.040 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:07.044+0000 7feb53400700 1 -- 172.21.15.18:0/1537990309 learned_addr learned my addr 172.21.15.18:0/1537990309 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:09:07.041 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:07.044+0000 7feb53400700 1 -- 172.21.15.18:0/1537990309 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7feb540fda60 msgr2=0x7feb541160f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:09:07.041 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:07.044+0000 7feb53400700 1 --2- 172.21.15.18:0/1537990309 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7feb540fda60 0x7feb541160f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:09:07.041 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:07.045+0000 7feb53400700 1 -- 172.21.15.18:0/1537990309 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb44005040 con 0x7feb540fb250 2024-04-12T21:09:07.041 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:07.045+0000 7feb53400700 1 --2- 172.21.15.18:0/1537990309 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7feb540fb250 0x7feb54115bb0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7feb44009180 tx=0x7feb44009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:09:07.041 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:07.045+0000 7feb4b400700 1 -- 172.21.15.18:0/1537990309 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7feb4400e040 con 0x7feb540fb250 2024-04-12T21:09:07.041 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:07.045+0000 7feb4b400700 1 -- 172.21.15.18:0/1537990309 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7feb4400bbc0 con 0x7feb540fb250 2024-04-12T21:09:07.041 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:07.045+0000 7feb58e00700 1 -- 172.21.15.18:0/1537990309 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feb5410fdb0 con 0x7feb540fb250 2024-04-12T21:09:07.041 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:07.045+0000 7feb4b400700 1 -- 172.21.15.18:0/1537990309 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7feb4401c630 con 0x7feb540fb250 2024-04-12T21:09:07.041 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:07.045+0000 7feb58e00700 1 -- 172.21.15.18:0/1537990309 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feb541102f0 con 0x7feb540fb250 2024-04-12T21:09:07.042 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:07.046+0000 7feb58e00700 1 -- 172.21.15.18:0/1537990309 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feb54107100 con 0x7feb540fb250 2024-04-12T21:09:07.043 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:07.048+0000 7feb4b400700 1 -- 172.21.15.18:0/1537990309 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7feb4400bd30 con 0x7feb540fb250 2024-04-12T21:09:07.044 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:07.049+0000 7feb4b400700 1 --2- 172.21.15.18:0/1537990309 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7feb3806ec00 0x7feb380710b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:09:07.044 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:07.049+0000 7feb4b400700 1 -- 172.21.15.18:0/1537990309 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 6757+0+0 (secure 0 0 0) 0x7feb4408fd90 con 0x7feb540fb250 2024-04-12T21:09:07.044 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:07.049+0000 7feb52a00700 1 --2- 172.21.15.18:0/1537990309 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7feb3806ec00 0x7feb380710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:09:07.045 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:07.050+0000 7feb52a00700 1 --2- 172.21.15.18:0/1537990309 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7feb3806ec00 0x7feb380710b0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7feb54110cf0 tx=0x7feb40009ce0 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:09:07.050 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:07.054+0000 7feb4b400700 1 -- 172.21.15.18:0/1537990309 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7feb4405c5c0 con 0x7feb540fb250 2024-04-12T21:09:07.303 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:07.308+0000 7feb58e00700 1 -- 172.21.15.18:0/1537990309 --> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7feb54111270 con 0x7feb3806ec00 2024-04-12T21:09:07.306 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:07.310+0000 7feb4b400700 1 -- 172.21.15.18:0/1537990309 <== mgr.34102 v2:172.21.15.18:6800/3812891919 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7feb54111270 con 0x7feb3806ec00 2024-04-12T21:09:07.306 INFO:teuthology.orchestra.run.smithi018.stdout:{ 2024-04-12T21:09:07.307 INFO:teuthology.orchestra.run.smithi018.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:f4452def3be4d350f4a8e5feaf7ebb1511be9298fe68478ba7a2d6fbd7d69a63", 2024-04-12T21:09:07.307 INFO:teuthology.orchestra.run.smithi018.stdout: "in_progress": true, 2024-04-12T21:09:07.307 INFO:teuthology.orchestra.run.smithi018.stdout: "which": "Upgrading all daemon types on all hosts", 2024-04-12T21:09:07.307 INFO:teuthology.orchestra.run.smithi018.stdout: "services_complete": [ 2024-04-12T21:09:07.307 INFO:teuthology.orchestra.run.smithi018.stdout: "crash", 2024-04-12T21:09:07.307 INFO:teuthology.orchestra.run.smithi018.stdout: "osd", 2024-04-12T21:09:07.307 INFO:teuthology.orchestra.run.smithi018.stdout: "mon", 2024-04-12T21:09:07.307 INFO:teuthology.orchestra.run.smithi018.stdout: "mgr" 2024-04-12T21:09:07.307 INFO:teuthology.orchestra.run.smithi018.stdout: ], 2024-04-12T21:09:07.307 INFO:teuthology.orchestra.run.smithi018.stdout: "progress": "14/21 daemons upgraded", 2024-04-12T21:09:07.307 INFO:teuthology.orchestra.run.smithi018.stdout: "message": "Currently upgrading mds daemons", 2024-04-12T21:09:07.307 INFO:teuthology.orchestra.run.smithi018.stdout: "is_paused": false 2024-04-12T21:09:07.307 INFO:teuthology.orchestra.run.smithi018.stdout:} 2024-04-12T21:09:07.310 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:07.315+0000 7feb58e00700 1 -- 172.21.15.18:0/1537990309 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7feb3806ec00 msgr2=0x7feb380710b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:09:07.310 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:07.315+0000 7feb58e00700 1 --2- 172.21.15.18:0/1537990309 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7feb3806ec00 0x7feb380710b0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7feb54110cf0 tx=0x7feb40009ce0 comp rx=0 tx=0).stop 2024-04-12T21:09:07.311 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:07.315+0000 7feb58e00700 1 -- 172.21.15.18:0/1537990309 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7feb540fb250 msgr2=0x7feb54115bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:09:07.311 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:07.315+0000 7feb58e00700 1 --2- 172.21.15.18:0/1537990309 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7feb540fb250 0x7feb54115bb0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7feb44009180 tx=0x7feb44009e30 comp rx=0 tx=0).stop 2024-04-12T21:09:07.311 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:07.315+0000 7feb58e00700 1 -- 172.21.15.18:0/1537990309 shutdown_connections 2024-04-12T21:09:07.311 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:07.315+0000 7feb58e00700 1 --2- 172.21.15.18:0/1537990309 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7feb3806ec00 0x7feb380710b0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:09:07.312 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:07.315+0000 7feb58e00700 1 --2- 172.21.15.18:0/1537990309 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7feb540fb250 0x7feb54115bb0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:09:07.312 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:07.315+0000 7feb58e00700 1 --2- 172.21.15.18:0/1537990309 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7feb540fda60 0x7feb541160f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:09:07.312 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:07.315+0000 7feb58e00700 1 -- 172.21.15.18:0/1537990309 >> 172.21.15.18:0/1537990309 conn(0x7feb540721b0 msgr2=0x7feb540fa9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:09:07.312 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:07.316+0000 7feb58e00700 1 -- 172.21.15.18:0/1537990309 shutdown_connections 2024-04-12T21:09:07.312 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:07.316+0000 7feb58e00700 1 -- 172.21.15.18:0/1537990309 wait complete. 2024-04-12T21:09:07.472 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:07.476+0000 7fa5bce00700 1 -- 172.21.15.18:0/2247370460 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa5b8103710 msgr2=0x7fa5b80fcd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:09:07.472 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:07.476+0000 7fa5bce00700 1 --2- 172.21.15.18:0/2247370460 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa5b8103710 0x7fa5b80fcd80 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fa5a0004940 tx=0x7fa5a0009e30 comp rx=0 tx=0).stop 2024-04-12T21:09:07.472 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:09:07 smithi018 ceph-mon[112590]: from='client.34536 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:09:07.472 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:09:07 smithi018 ceph-mon[112590]: pgmap v279: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:09:07.472 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:09:07 smithi018 ceph-mon[112590]: from='client.? 172.21.15.18:0/3147770208' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T21:09:07.472 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:09:07 smithi018 ceph-mon[112590]: from='client.? 172.21.15.18:0/1360639453' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-12T21:09:07.473 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:07.476+0000 7fa5bce00700 1 -- 172.21.15.18:0/2247370460 shutdown_connections 2024-04-12T21:09:07.473 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:07.476+0000 7fa5bce00700 1 --2- 172.21.15.18:0/2247370460 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa5b8103710 0x7fa5b80fcd80 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:09:07.473 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:07.476+0000 7fa5bce00700 1 --2- 172.21.15.18:0/2247370460 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa5b8102d70 0x7fa5b8103140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:09:07.473 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:07.476+0000 7fa5bce00700 1 -- 172.21.15.18:0/2247370460 >> 172.21.15.18:0/2247370460 conn(0x7fa5b80f8920 msgr2=0x7fa5b80fad30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:09:07.473 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:07.476+0000 7fa5bce00700 1 -- 172.21.15.18:0/2247370460 shutdown_connections 2024-04-12T21:09:07.473 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:07.476+0000 7fa5bce00700 1 -- 172.21.15.18:0/2247370460 wait complete. 2024-04-12T21:09:07.473 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:07.477+0000 7fa5bce00700 1 Processor -- start 2024-04-12T21:09:07.473 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:07.477+0000 7fa5bce00700 1 -- start start 2024-04-12T21:09:07.474 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:07.478+0000 7fa5bce00700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa5b8102d70 0x7fa5b80702c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:09:07.474 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:07.478+0000 7fa5bce00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa5b8103710 0x7fa5b8070800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:09:07.475 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:07.478+0000 7fa5bce00700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5b8070d90 con 0x7fa5b8103710 2024-04-12T21:09:07.475 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:07.478+0000 7fa5bce00700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5b8070ed0 con 0x7fa5b8102d70 2024-04-12T21:09:07.475 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:07.478+0000 7fa5afe00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa5b8103710 0x7fa5b8070800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:09:07.475 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:07.478+0000 7fa5afe00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa5b8103710 0x7fa5b8070800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:37532/0 (socket says 172.21.15.18:37532) 2024-04-12T21:09:07.475 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:07.478+0000 7fa5afe00700 1 -- 172.21.15.18:0/2594678847 learned_addr learned my addr 172.21.15.18:0/2594678847 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:09:07.475 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:07.479+0000 7fa5b5e00700 1 --2- 172.21.15.18:0/2594678847 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa5b8102d70 0x7fa5b80702c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:09:07.475 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:07.479+0000 7fa5afe00700 1 -- 172.21.15.18:0/2594678847 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa5b8102d70 msgr2=0x7fa5b80702c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:09:07.475 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:07.479+0000 7fa5afe00700 1 --2- 172.21.15.18:0/2594678847 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa5b8102d70 0x7fa5b80702c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:09:07.475 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:07.479+0000 7fa5afe00700 1 -- 172.21.15.18:0/2594678847 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5a0005040 con 0x7fa5b8103710 2024-04-12T21:09:07.475 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:07.479+0000 7fa5afe00700 1 --2- 172.21.15.18:0/2594678847 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa5b8103710 0x7fa5b8070800 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fa5a0009100 tx=0x7fa5a0009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:09:07.476 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:07.479+0000 7fa5ae000700 1 -- 172.21.15.18:0/2594678847 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa5a000e040 con 0x7fa5b8103710 2024-04-12T21:09:07.476 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:07.479+0000 7fa5ae000700 1 -- 172.21.15.18:0/2594678847 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa5a0004290 con 0x7fa5b8103710 2024-04-12T21:09:07.476 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:07.479+0000 7fa5bce00700 1 -- 172.21.15.18:0/2594678847 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa5b8071120 con 0x7fa5b8103710 2024-04-12T21:09:07.476 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:07.480+0000 7fa5ae000700 1 -- 172.21.15.18:0/2594678847 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa5a001c3f0 con 0x7fa5b8103710 2024-04-12T21:09:07.476 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:07.480+0000 7fa5bce00700 1 -- 172.21.15.18:0/2594678847 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa5b811a260 con 0x7fa5b8103710 2024-04-12T21:09:07.477 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:07.481+0000 7fa5bce00700 1 -- 172.21.15.18:0/2594678847 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa5b80fe4c0 con 0x7fa5b8103710 2024-04-12T21:09:07.478 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:07.483+0000 7fa5ae000700 1 -- 172.21.15.18:0/2594678847 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7fa5a0013790 con 0x7fa5b8103710 2024-04-12T21:09:07.480 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:07.484+0000 7fa5ae000700 1 --2- 172.21.15.18:0/2594678847 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fa59c06ec10 0x7fa59c0710c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:09:07.480 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:07.484+0000 7fa5ae000700 1 -- 172.21.15.18:0/2594678847 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 6757+0+0 (secure 0 0 0) 0x7fa5a008fd10 con 0x7fa5b8103710 2024-04-12T21:09:07.480 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:07.484+0000 7fa5b5e00700 1 --2- 172.21.15.18:0/2594678847 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fa59c06ec10 0x7fa59c0710c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:09:07.480 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:07.485+0000 7fa5b5e00700 1 --2- 172.21.15.18:0/2594678847 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fa59c06ec10 0x7fa59c0710c0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fa5a8004990 tx=0x7fa5a8009ee0 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:09:07.484 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:07.489+0000 7fa5ae000700 1 -- 172.21.15.18:0/2594678847 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7fa5a0058990 con 0x7fa5b8103710 2024-04-12T21:09:07.529 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:09:07 smithi186 ceph-mon[86379]: from='client.34536 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:09:07.529 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:09:07 smithi186 ceph-mon[86379]: pgmap v279: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:09:07.529 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:09:07 smithi186 ceph-mon[86379]: from='client.? 172.21.15.18:0/3147770208' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T21:09:07.529 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:09:07 smithi186 ceph-mon[86379]: from='client.? 172.21.15.18:0/1360639453' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-12T21:09:07.802 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:07.806+0000 7fa5bce00700 1 -- 172.21.15.18:0/2594678847 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7fa5b805f5f0 con 0x7fa5b8103710 2024-04-12T21:09:07.802 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:07.807+0000 7fa5ae000700 1 -- 172.21.15.18:0/2594678847 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+233 (secure 0 0 0) 0x7fa5a0013420 con 0x7fa5b8103710 2024-04-12T21:09:07.803 INFO:teuthology.orchestra.run.smithi018.stdout:HEALTH_ERR 1 filesystem is degraded; 1 filesystem is offline 2024-04-12T21:09:07.803 INFO:teuthology.orchestra.run.smithi018.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-04-12T21:09:07.803 INFO:teuthology.orchestra.run.smithi018.stdout: fs cephfs is degraded 2024-04-12T21:09:07.803 INFO:teuthology.orchestra.run.smithi018.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-04-12T21:09:07.803 INFO:teuthology.orchestra.run.smithi018.stdout: fs cephfs is offline because no MDS is active for it. 2024-04-12T21:09:07.806 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:07.809+0000 7fa5bce00700 1 -- 172.21.15.18:0/2594678847 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fa59c06ec10 msgr2=0x7fa59c0710c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:09:07.806 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:07.810+0000 7fa5bce00700 1 --2- 172.21.15.18:0/2594678847 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fa59c06ec10 0x7fa59c0710c0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fa5a8004990 tx=0x7fa5a8009ee0 comp rx=0 tx=0).stop 2024-04-12T21:09:07.806 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:07.810+0000 7fa5bce00700 1 -- 172.21.15.18:0/2594678847 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa5b8103710 msgr2=0x7fa5b8070800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:09:07.806 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:07.810+0000 7fa5bce00700 1 --2- 172.21.15.18:0/2594678847 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa5b8103710 0x7fa5b8070800 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fa5a0009100 tx=0x7fa5a0009e30 comp rx=0 tx=0).stop 2024-04-12T21:09:07.806 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:07.810+0000 7fa5bce00700 1 -- 172.21.15.18:0/2594678847 shutdown_connections 2024-04-12T21:09:07.806 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:07.810+0000 7fa5bce00700 1 --2- 172.21.15.18:0/2594678847 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fa59c06ec10 0x7fa59c0710c0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:09:07.806 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:07.810+0000 7fa5bce00700 1 --2- 172.21.15.18:0/2594678847 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa5b8102d70 0x7fa5b80702c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:09:07.806 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:07.810+0000 7fa5bce00700 1 --2- 172.21.15.18:0/2594678847 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa5b8103710 0x7fa5b8070800 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:09:07.806 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:07.810+0000 7fa5bce00700 1 -- 172.21.15.18:0/2594678847 >> 172.21.15.18:0/2594678847 conn(0x7fa5b80f8920 msgr2=0x7fa5b80f9940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:09:07.806 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:07.810+0000 7fa5bce00700 1 -- 172.21.15.18:0/2594678847 shutdown_connections 2024-04-12T21:09:07.806 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:07.810+0000 7fa5bce00700 1 -- 172.21.15.18:0/2594678847 wait complete. 2024-04-12T21:09:08.529 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:09:08 smithi186 ceph-mon[86379]: from='client.34548 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:09:08.529 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:09:08 smithi186 ceph-mon[86379]: from='client.? 172.21.15.18:0/2594678847' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-12T21:09:08.600 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:09:08 smithi018 ceph-mon[112590]: from='client.34548 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:09:08.600 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:09:08 smithi018 ceph-mon[112590]: from='client.? 172.21.15.18:0/2594678847' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-12T21:09:09.431 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:09:09 smithi186 ceph-mon[86379]: pgmap v280: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:09:09.600 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:09:09 smithi018 ceph-mon[112590]: pgmap v280: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:09:10.465 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:09:10 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T21:09:10.502 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:09:10 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T21:09:11.529 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:09:11 smithi186 ceph-mon[86379]: pgmap v281: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:09:11.600 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:09:11 smithi018 ceph-mon[112590]: pgmap v281: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:09:13.529 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:09:13 smithi186 ceph-mon[86379]: pgmap v282: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:09:13.600 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:09:13 smithi018 ceph-mon[112590]: pgmap v282: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:09:15.529 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:09:15 smithi186 ceph-mon[86379]: pgmap v283: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:09:15.600 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:09:15 smithi018 ceph-mon[112590]: pgmap v283: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:09:17.529 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:09:17 smithi186 ceph-mon[86379]: pgmap v284: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:09:17.600 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:09:17 smithi018 ceph-mon[112590]: pgmap v284: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:09:19.529 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:09:19 smithi186 ceph-mon[86379]: pgmap v285: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:09:19.600 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:09:19 smithi018 ceph-mon[112590]: pgmap v285: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:09:21.492 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:09:21 smithi186 ceph-mon[86379]: pgmap v286: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:09:21.600 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:09:21 smithi018 ceph-mon[112590]: pgmap v286: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:09:23.529 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:09:23 smithi186 ceph-mon[86379]: pgmap v287: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:09:23.600 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:09:23 smithi018 ceph-mon[112590]: pgmap v287: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:09:25.529 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:09:25 smithi186 ceph-mon[86379]: pgmap v288: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:09:25.529 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:09:25 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T21:09:25.600 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:09:25 smithi018 ceph-mon[112590]: pgmap v288: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:09:25.600 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:09:25 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T21:09:27.529 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:09:27 smithi186 ceph-mon[86379]: pgmap v289: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:09:27.600 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:09:27 smithi018 ceph-mon[112590]: pgmap v289: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:09:29.529 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:09:29 smithi186 ceph-mon[86379]: pgmap v290: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:09:29.600 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:09:29 smithi018 ceph-mon[112590]: pgmap v290: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:09:31.529 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:09:31 smithi186 ceph-mon[86379]: pgmap v291: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:09:31.600 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:09:31 smithi018 ceph-mon[112590]: pgmap v291: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:09:33.529 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:09:33 smithi186 ceph-mon[86379]: pgmap v292: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:09:33.600 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:09:33 smithi018 ceph-mon[112590]: pgmap v292: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:09:35.529 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:09:35 smithi186 ceph-mon[86379]: pgmap v293: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:09:35.600 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:09:35 smithi018 ceph-mon[112590]: pgmap v293: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:09:37.529 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:09:37 smithi186 ceph-mon[86379]: pgmap v294: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:09:37.600 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:09:37 smithi018 ceph-mon[112590]: pgmap v294: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:09:37.979 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:37.982+0000 7f0f69600700 1 -- 172.21.15.18:0/2229129097 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0f64109dd0 msgr2=0x7f0f64103440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:09:37.979 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:37.982+0000 7f0f69600700 1 --2- 172.21.15.18:0/2229129097 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0f64109dd0 0x7f0f64103440 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f0f50004940 tx=0x7f0f50009e30 comp rx=0 tx=0).stop 2024-04-12T21:09:37.979 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:37.982+0000 7f0f69600700 1 -- 172.21.15.18:0/2229129097 shutdown_connections 2024-04-12T21:09:37.979 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:37.982+0000 7f0f69600700 1 --2- 172.21.15.18:0/2229129097 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0f64109dd0 0x7f0f64103440 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:09:37.979 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:37.982+0000 7f0f69600700 1 --2- 172.21.15.18:0/2229129097 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0f64109430 0x7f0f64109800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:09:37.979 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:37.982+0000 7f0f69600700 1 -- 172.21.15.18:0/2229129097 >> 172.21.15.18:0/2229129097 conn(0x7f0f64072c00 msgr2=0x7f0f64075010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:09:37.980 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:37.982+0000 7f0f69600700 1 -- 172.21.15.18:0/2229129097 shutdown_connections 2024-04-12T21:09:37.980 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:37.982+0000 7f0f69600700 1 -- 172.21.15.18:0/2229129097 wait complete. 2024-04-12T21:09:37.980 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:37.983+0000 7f0f69600700 1 Processor -- start 2024-04-12T21:09:37.981 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:37.983+0000 7f0f69600700 1 -- start start 2024-04-12T21:09:37.981 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:37.984+0000 7f0f69600700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0f64109430 0x7f0f64119f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:09:37.981 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:37.984+0000 7f0f69600700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0f64109dd0 0x7f0f6411a4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:09:37.981 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:37.984+0000 7f0f69600700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f6411ab10 con 0x7f0f64109dd0 2024-04-12T21:09:37.981 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:37.984+0000 7f0f69600700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f641140f0 con 0x7f0f64109430 2024-04-12T21:09:37.981 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:37.984+0000 7f0f62000700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0f64109dd0 0x7f0f6411a4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:09:37.981 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:37.984+0000 7f0f62000700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0f64109dd0 0x7f0f6411a4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:40652/0 (socket says 172.21.15.18:40652) 2024-04-12T21:09:37.981 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:37.984+0000 7f0f62000700 1 -- 172.21.15.18:0/2111693222 learned_addr learned my addr 172.21.15.18:0/2111693222 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:09:37.982 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:37.985+0000 7f0f62a00700 1 --2- 172.21.15.18:0/2111693222 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0f64109430 0x7f0f64119f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:09:37.982 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:37.985+0000 7f0f62000700 1 -- 172.21.15.18:0/2111693222 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0f64109430 msgr2=0x7f0f64119f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:09:37.982 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:37.985+0000 7f0f62000700 1 --2- 172.21.15.18:0/2111693222 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0f64109430 0x7f0f64119f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:09:37.982 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:37.985+0000 7f0f62000700 1 -- 172.21.15.18:0/2111693222 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f50005040 con 0x7f0f64109dd0 2024-04-12T21:09:37.982 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:37.985+0000 7f0f62000700 1 --2- 172.21.15.18:0/2111693222 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0f64109dd0 0x7f0f6411a4c0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f0f50009100 tx=0x7f0f50009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:09:37.983 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:37.985+0000 7f0f5aa00700 1 -- 172.21.15.18:0/2111693222 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0f5000e040 con 0x7f0f64109dd0 2024-04-12T21:09:37.983 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:37.986+0000 7f0f5aa00700 1 -- 172.21.15.18:0/2111693222 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f0f50013420 con 0x7f0f64109dd0 2024-04-12T21:09:37.983 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:37.986+0000 7f0f69600700 1 -- 172.21.15.18:0/2111693222 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0f64114370 con 0x7f0f64109dd0 2024-04-12T21:09:37.983 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:37.986+0000 7f0f5aa00700 1 -- 172.21.15.18:0/2111693222 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0f5001c630 con 0x7f0f64109dd0 2024-04-12T21:09:37.983 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:37.986+0000 7f0f69600700 1 -- 172.21.15.18:0/2111693222 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0f64114930 con 0x7f0f64109dd0 2024-04-12T21:09:37.983 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:37.987+0000 7f0f69600700 1 -- 172.21.15.18:0/2111693222 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0f64104b80 con 0x7f0f64109dd0 2024-04-12T21:09:37.984 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:37.988+0000 7f0f5aa00700 1 -- 172.21.15.18:0/2111693222 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7f0f50013590 con 0x7f0f64109dd0 2024-04-12T21:09:37.985 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:37.989+0000 7f0f5aa00700 1 --2- 172.21.15.18:0/2111693222 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f0f4806ec00 0x7f0f480710b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:09:37.986 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:37.989+0000 7f0f5aa00700 1 -- 172.21.15.18:0/2111693222 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 6757+0+0 (secure 0 0 0) 0x7f0f5008fc70 con 0x7f0f64109dd0 2024-04-12T21:09:37.986 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:37.989+0000 7f0f62a00700 1 --2- 172.21.15.18:0/2111693222 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f0f4806ec00 0x7f0f480710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:09:37.986 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:37.990+0000 7f0f62a00700 1 --2- 172.21.15.18:0/2111693222 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f0f4806ec00 0x7f0f480710b0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f0f54007fd0 tx=0x7f0f54009040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:09:37.991 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:37.995+0000 7f0f5aa00700 1 -- 172.21.15.18:0/2111693222 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f0f50096080 con 0x7f0f64109dd0 2024-04-12T21:09:38.247 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:38.250+0000 7f0f69600700 1 -- 172.21.15.18:0/2111693222 --> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f0f64115640 con 0x7f0f4806ec00 2024-04-12T21:09:38.249 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:38.252+0000 7f0f5aa00700 1 -- 172.21.15.18:0/2111693222 <== mgr.34102 v2:172.21.15.18:6800/3812891919 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f0f64115640 con 0x7f0f4806ec00 2024-04-12T21:09:38.252 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:38.255+0000 7f0f69600700 1 -- 172.21.15.18:0/2111693222 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f0f4806ec00 msgr2=0x7f0f480710b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:09:38.252 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:38.255+0000 7f0f69600700 1 --2- 172.21.15.18:0/2111693222 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f0f4806ec00 0x7f0f480710b0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f0f54007fd0 tx=0x7f0f54009040 comp rx=0 tx=0).stop 2024-04-12T21:09:38.252 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:38.255+0000 7f0f69600700 1 -- 172.21.15.18:0/2111693222 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0f64109dd0 msgr2=0x7f0f6411a4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:09:38.252 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:38.255+0000 7f0f69600700 1 --2- 172.21.15.18:0/2111693222 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0f64109dd0 0x7f0f6411a4c0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f0f50009100 tx=0x7f0f50009e30 comp rx=0 tx=0).stop 2024-04-12T21:09:38.253 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:38.256+0000 7f0f69600700 1 -- 172.21.15.18:0/2111693222 shutdown_connections 2024-04-12T21:09:38.253 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:38.256+0000 7f0f69600700 1 --2- 172.21.15.18:0/2111693222 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f0f4806ec00 0x7f0f480710b0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:09:38.253 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:38.256+0000 7f0f69600700 1 --2- 172.21.15.18:0/2111693222 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0f64109430 0x7f0f64119f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:09:38.253 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:38.256+0000 7f0f69600700 1 --2- 172.21.15.18:0/2111693222 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0f64109dd0 0x7f0f6411a4c0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:09:38.253 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:38.256+0000 7f0f69600700 1 -- 172.21.15.18:0/2111693222 >> 172.21.15.18:0/2111693222 conn(0x7f0f64072c00 msgr2=0x7f0f64074c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:09:38.253 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:38.256+0000 7f0f69600700 1 -- 172.21.15.18:0/2111693222 shutdown_connections 2024-04-12T21:09:38.253 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:38.256+0000 7f0f69600700 1 -- 172.21.15.18:0/2111693222 wait complete. 2024-04-12T21:09:38.270 INFO:teuthology.orchestra.run.smithi018.stdout:true 2024-04-12T21:09:38.417 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:38.420+0000 7f5eb0c00700 1 -- 172.21.15.18:0/1430836795 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5eac109dd0 msgr2=0x7f5eac103440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:09:38.418 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:38.420+0000 7f5eb0c00700 1 --2- 172.21.15.18:0/1430836795 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5eac109dd0 0x7f5eac103440 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f5e98004940 tx=0x7f5e98009e30 comp rx=0 tx=0).stop 2024-04-12T21:09:38.418 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:38.421+0000 7f5eb0c00700 1 -- 172.21.15.18:0/1430836795 shutdown_connections 2024-04-12T21:09:38.418 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:38.421+0000 7f5eb0c00700 1 --2- 172.21.15.18:0/1430836795 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5eac109dd0 0x7f5eac103440 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:09:38.418 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:38.421+0000 7f5eb0c00700 1 --2- 172.21.15.18:0/1430836795 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5eac109430 0x7f5eac109800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:09:38.418 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:38.421+0000 7f5eb0c00700 1 -- 172.21.15.18:0/1430836795 >> 172.21.15.18:0/1430836795 conn(0x7f5eac072c00 msgr2=0x7f5eac075010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:09:38.418 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:38.421+0000 7f5eb0c00700 1 -- 172.21.15.18:0/1430836795 shutdown_connections 2024-04-12T21:09:38.418 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:38.421+0000 7f5eb0c00700 1 -- 172.21.15.18:0/1430836795 wait complete. 2024-04-12T21:09:38.419 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:38.422+0000 7f5eb0c00700 1 Processor -- start 2024-04-12T21:09:38.419 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:38.422+0000 7f5eb0c00700 1 -- start start 2024-04-12T21:09:38.419 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:38.422+0000 7f5eb0c00700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5eac109430 0x7f5eac119fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:09:38.419 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:38.422+0000 7f5eb0c00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5eac109dd0 0x7f5eac11a4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:09:38.419 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:38.422+0000 7f5eb0c00700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5eac11abc0 con 0x7f5eac109dd0 2024-04-12T21:09:38.419 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:38.422+0000 7f5eb0c00700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5eac114110 con 0x7f5eac109430 2024-04-12T21:09:38.419 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:38.422+0000 7f5ea9400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5eac109dd0 0x7f5eac11a4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:09:38.419 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:38.422+0000 7f5ea9400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5eac109dd0 0x7f5eac11a4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:40666/0 (socket says 172.21.15.18:40666) 2024-04-12T21:09:38.419 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:38.422+0000 7f5ea9400700 1 -- 172.21.15.18:0/344464712 learned_addr learned my addr 172.21.15.18:0/344464712 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:09:38.420 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:38.423+0000 7f5ea9400700 1 -- 172.21.15.18:0/344464712 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5eac109430 msgr2=0x7f5eac119fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:09:38.421 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:38.423+0000 7f5ea9400700 1 --2- 172.21.15.18:0/344464712 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5eac109430 0x7f5eac119fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:09:38.421 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:38.423+0000 7f5ea9400700 1 -- 172.21.15.18:0/344464712 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e98005040 con 0x7f5eac109dd0 2024-04-12T21:09:38.421 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:38.423+0000 7f5ea9400700 1 --2- 172.21.15.18:0/344464712 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5eac109dd0 0x7f5eac11a4e0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f5e98009180 tx=0x7f5e98009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:09:38.421 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:38.423+0000 7f5ea2000700 1 -- 172.21.15.18:0/344464712 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5e9800e040 con 0x7f5eac109dd0 2024-04-12T21:09:38.421 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:38.424+0000 7f5ea2000700 1 -- 172.21.15.18:0/344464712 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f5e9800bbc0 con 0x7f5eac109dd0 2024-04-12T21:09:38.421 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:38.424+0000 7f5ea2000700 1 -- 172.21.15.18:0/344464712 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5e9801c3f0 con 0x7f5eac109dd0 2024-04-12T21:09:38.421 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:38.424+0000 7f5eb0c00700 1 -- 172.21.15.18:0/344464712 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5eac114330 con 0x7f5eac109dd0 2024-04-12T21:09:38.421 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:38.424+0000 7f5eb0c00700 1 -- 172.21.15.18:0/344464712 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5eac1148f0 con 0x7f5eac109dd0 2024-04-12T21:09:38.421 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:38.425+0000 7f5eb0c00700 1 -- 172.21.15.18:0/344464712 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5eac104b80 con 0x7f5eac109dd0 2024-04-12T21:09:38.422 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:38.426+0000 7f5ea2000700 1 -- 172.21.15.18:0/344464712 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7f5e9800bd30 con 0x7f5eac109dd0 2024-04-12T21:09:38.423 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:38.427+0000 7f5ea2000700 1 --2- 172.21.15.18:0/344464712 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f5e9006ec00 0x7f5e900710b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:09:38.424 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:38.427+0000 7f5ea2000700 1 -- 172.21.15.18:0/344464712 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 6757+0+0 (secure 0 0 0) 0x7f5e9808fd00 con 0x7f5eac109dd0 2024-04-12T21:09:38.424 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:38.427+0000 7f5ea9e00700 1 --2- 172.21.15.18:0/344464712 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f5e9006ec00 0x7f5e900710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:09:38.424 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:38.428+0000 7f5ea9e00700 1 --2- 172.21.15.18:0/344464712 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f5e9006ec00 0x7f5e900710b0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f5e9c006fd0 tx=0x7f5e9c006db0 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:09:38.428 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:38.432+0000 7f5ea2000700 1 -- 172.21.15.18:0/344464712 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f5e9805c530 con 0x7f5eac109dd0 2024-04-12T21:09:38.678 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:38.681+0000 7f5eb0c00700 1 -- 172.21.15.18:0/344464712 --> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f5eac114c20 con 0x7f5e9006ec00 2024-04-12T21:09:38.679 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:38.683+0000 7f5ea2000700 1 -- 172.21.15.18:0/344464712 <== mgr.34102 v2:172.21.15.18:6800/3812891919 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f5eac114c20 con 0x7f5e9006ec00 2024-04-12T21:09:38.682 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:38.685+0000 7f5eb0c00700 1 -- 172.21.15.18:0/344464712 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f5e9006ec00 msgr2=0x7f5e900710b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:09:38.683 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:38.686+0000 7f5eb0c00700 1 --2- 172.21.15.18:0/344464712 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f5e9006ec00 0x7f5e900710b0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f5e9c006fd0 tx=0x7f5e9c006db0 comp rx=0 tx=0).stop 2024-04-12T21:09:38.683 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:38.686+0000 7f5eb0c00700 1 -- 172.21.15.18:0/344464712 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5eac109dd0 msgr2=0x7f5eac11a4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:09:38.683 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:38.686+0000 7f5eb0c00700 1 --2- 172.21.15.18:0/344464712 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5eac109dd0 0x7f5eac11a4e0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f5e98009180 tx=0x7f5e98009e30 comp rx=0 tx=0).stop 2024-04-12T21:09:38.683 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:38.686+0000 7f5eb0c00700 1 -- 172.21.15.18:0/344464712 shutdown_connections 2024-04-12T21:09:38.683 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:38.686+0000 7f5eb0c00700 1 --2- 172.21.15.18:0/344464712 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f5e9006ec00 0x7f5e900710b0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:09:38.683 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:38.686+0000 7f5eb0c00700 1 --2- 172.21.15.18:0/344464712 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5eac109430 0x7f5eac119fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:09:38.683 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:38.686+0000 7f5eb0c00700 1 --2- 172.21.15.18:0/344464712 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5eac109dd0 0x7f5eac11a4e0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:09:38.683 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:38.686+0000 7f5eb0c00700 1 -- 172.21.15.18:0/344464712 >> 172.21.15.18:0/344464712 conn(0x7f5eac072c00 msgr2=0x7f5eac074b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:09:38.683 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:38.686+0000 7f5eb0c00700 1 -- 172.21.15.18:0/344464712 shutdown_connections 2024-04-12T21:09:38.683 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:38.686+0000 7f5eb0c00700 1 -- 172.21.15.18:0/344464712 wait complete. 2024-04-12T21:09:38.839 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:38.842+0000 7f3011800700 1 -- 172.21.15.18:0/4181404079 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f300c105010 msgr2=0x7f300c1053e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:09:38.839 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:38.842+0000 7f3011800700 1 --2- 172.21.15.18:0/4181404079 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f300c105010 0x7f300c1053e0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f2ffc004940 tx=0x7f2ffc009e30 comp rx=0 tx=0).stop 2024-04-12T21:09:38.839 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:38.842+0000 7f3011800700 1 -- 172.21.15.18:0/4181404079 shutdown_connections 2024-04-12T21:09:38.839 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:38.842+0000 7f3011800700 1 --2- 172.21.15.18:0/4181404079 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f300c1059b0 0x7f300c0ff020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:09:38.839 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:38.842+0000 7f3011800700 1 --2- 172.21.15.18:0/4181404079 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f300c105010 0x7f300c1053e0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:09:38.839 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:38.842+0000 7f3011800700 1 -- 172.21.15.18:0/4181404079 >> 172.21.15.18:0/4181404079 conn(0x7f300c0faba0 msgr2=0x7f300c0fcfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:09:38.840 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:38.842+0000 7f3011800700 1 -- 172.21.15.18:0/4181404079 shutdown_connections 2024-04-12T21:09:38.840 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:38.842+0000 7f3011800700 1 -- 172.21.15.18:0/4181404079 wait complete. 2024-04-12T21:09:38.840 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:38.843+0000 7f3011800700 1 Processor -- start 2024-04-12T21:09:38.841 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:38.843+0000 7f3011800700 1 -- start start 2024-04-12T21:09:38.841 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:38.844+0000 7f3011800700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f300c105010 0x7f300c115d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:09:38.841 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:38.844+0000 7f3011800700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f300c1059b0 0x7f300c1162d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:09:38.841 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:38.844+0000 7f3011800700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f300c1169b0 con 0x7f300c105010 2024-04-12T21:09:38.841 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:38.844+0000 7f3011800700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f300c10fcf0 con 0x7f300c1059b0 2024-04-12T21:09:38.841 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:38.844+0000 7f300aa00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f300c105010 0x7f300c115d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:09:38.841 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:38.844+0000 7f300aa00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f300c105010 0x7f300c115d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:40688/0 (socket says 172.21.15.18:40688) 2024-04-12T21:09:38.841 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:38.844+0000 7f300aa00700 1 -- 172.21.15.18:0/158065980 learned_addr learned my addr 172.21.15.18:0/158065980 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:09:38.842 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:38.844+0000 7f3002000700 1 --2- 172.21.15.18:0/158065980 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f300c1059b0 0x7f300c1162d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:09:38.842 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:38.845+0000 7f300aa00700 1 -- 172.21.15.18:0/158065980 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f300c1059b0 msgr2=0x7f300c1162d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:09:38.843 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:38.845+0000 7f300aa00700 1 --2- 172.21.15.18:0/158065980 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f300c1059b0 0x7f300c1162d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:09:38.843 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:38.845+0000 7f300aa00700 1 -- 172.21.15.18:0/158065980 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ffc005040 con 0x7f300c105010 2024-04-12T21:09:38.843 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:38.845+0000 7f3002000700 1 --2- 172.21.15.18:0/158065980 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f300c1059b0 0x7f300c1162d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-12T21:09:38.843 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:38.845+0000 7f300aa00700 1 --2- 172.21.15.18:0/158065980 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f300c105010 0x7f300c115d90 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f2ffc009150 tx=0x7f2ffc009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:09:38.843 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:38.845+0000 7f3003400700 1 -- 172.21.15.18:0/158065980 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2ffc00e040 con 0x7f300c105010 2024-04-12T21:09:38.843 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:38.845+0000 7f3003400700 1 -- 172.21.15.18:0/158065980 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f2ffc013420 con 0x7f300c105010 2024-04-12T21:09:38.843 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:38.845+0000 7f3011800700 1 -- 172.21.15.18:0/158065980 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f300c10ff10 con 0x7f300c105010 2024-04-12T21:09:38.843 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:38.845+0000 7f3003400700 1 -- 172.21.15.18:0/158065980 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2ffc01c630 con 0x7f300c105010 2024-04-12T21:09:38.843 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:38.845+0000 7f3011800700 1 -- 172.21.15.18:0/158065980 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f300c1104d0 con 0x7f300c105010 2024-04-12T21:09:38.843 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:38.846+0000 7f3011800700 1 -- 172.21.15.18:0/158065980 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f300c100760 con 0x7f300c105010 2024-04-12T21:09:38.844 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:38.848+0000 7f3003400700 1 -- 172.21.15.18:0/158065980 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7f2ffc013590 con 0x7f300c105010 2024-04-12T21:09:38.845 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:38.849+0000 7f3003400700 1 --2- 172.21.15.18:0/158065980 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f2fec06ec10 0x7f2fec0710c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:09:38.846 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:38.849+0000 7f3003400700 1 -- 172.21.15.18:0/158065980 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 6757+0+0 (secure 0 0 0) 0x7f2ffc08fc00 con 0x7f300c105010 2024-04-12T21:09:38.846 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:38.849+0000 7f3002000700 1 --2- 172.21.15.18:0/158065980 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f2fec06ec10 0x7f2fec0710c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:09:38.846 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:38.850+0000 7f3002000700 1 --2- 172.21.15.18:0/158065980 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f2fec06ec10 0x7f2fec0710c0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f2ff4006fd0 tx=0x7f2ff4008040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:09:38.850 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:38.854+0000 7f3003400700 1 -- 172.21.15.18:0/158065980 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f2ffc05c430 con 0x7f300c105010 2024-04-12T21:09:39.085 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:39.089+0000 7f3011800700 1 -- 172.21.15.18:0/158065980 --> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f300c1113e0 con 0x7f2fec06ec10 2024-04-12T21:09:39.118 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:39.121+0000 7f3003400700 1 -- 172.21.15.18:0/158065980 <== mgr.34102 v2:172.21.15.18:6800/3812891919 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7f300c1113e0 con 0x7f2fec06ec10 2024-04-12T21:09:39.118 INFO:teuthology.orchestra.run.smithi018.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-12T21:09:39.118 INFO:teuthology.orchestra.run.smithi018.stdout:alertmanager.smithi018 smithi018 *:9093,9094 running (10m) 113s ago 24m 14.7M - 0.25.0 c8568f914cd2 c332090068e8 2024-04-12T21:09:39.118 INFO:teuthology.orchestra.run.smithi018.stdout:crash.smithi018 smithi018 running (6m) 113s ago 25m 6665k - 19.0.0-2879-g324baf8d 8717720dc5e5 b7f80d1f7982 2024-04-12T21:09:39.119 INFO:teuthology.orchestra.run.smithi018.stdout:crash.smithi186 smithi186 running (6m) 2m ago 23m 6660k - 19.0.0-2879-g324baf8d 8717720dc5e5 a4f55c81b623 2024-04-12T21:09:39.119 INFO:teuthology.orchestra.run.smithi018.stdout:grafana.smithi018 smithi018 *:3000 running (9m) 113s ago 24m 74.8M - 9.4.12 2bacad6d85d8 b08e3bf176dd 2024-04-12T21:09:39.119 INFO:teuthology.orchestra.run.smithi018.stdout:mds.cephfs.smithi018.acqbsc smithi018 running (2m) 113s ago 17m 11.1M - 19.0.0-2879-g324baf8d 8717720dc5e5 021f99ab24ae 2024-04-12T21:09:39.119 INFO:teuthology.orchestra.run.smithi018.stdout:mds.cephfs.smithi018.szjluo smithi018 running (2m) 113s ago 17m 10.9M - 19.0.0-2879-g324baf8d 8717720dc5e5 0003117479d4 2024-04-12T21:09:39.119 INFO:teuthology.orchestra.run.smithi018.stdout:mds.cephfs.smithi186.jdllyn smithi186 running (17m) 2m ago 17m 15.6M - 17.2.7-827-gd3b5060a 21de9b7369ad 2b2bc46bd29a 2024-04-12T21:09:39.119 INFO:teuthology.orchestra.run.smithi018.stdout:mds.cephfs.smithi186.wjnafs smithi186 running (17m) 2m ago 17m 15.4M - 17.2.7-827-gd3b5060a 21de9b7369ad 74b115d6d6fd 2024-04-12T21:09:39.119 INFO:teuthology.orchestra.run.smithi018.stdout:mgr.smithi018.xjzwga smithi018 *:8443,9283,8765 running (12m) 113s ago 26m 509M - 19.0.0-2879-g324baf8d 8717720dc5e5 56088305eab7 2024-04-12T21:09:39.119 INFO:teuthology.orchestra.run.smithi018.stdout:mgr.smithi186.kqcfxk smithi186 *:8443,9283,8765 running (11m) 2m ago 23m 452M - 19.0.0-2879-g324baf8d 8717720dc5e5 09cd766e5e0d 2024-04-12T21:09:39.119 INFO:teuthology.orchestra.run.smithi018.stdout:mon.smithi018 smithi018 running (8m) 113s ago 26m 57.0M 2048M 19.0.0-2879-g324baf8d 8717720dc5e5 df3e201dfed6 2024-04-12T21:09:39.119 INFO:teuthology.orchestra.run.smithi018.stdout:mon.smithi186 smithi186 running (7m) 2m ago 23m 37.7M 2048M 19.0.0-2879-g324baf8d 8717720dc5e5 c77498da9b2b 2024-04-12T21:09:39.119 INFO:teuthology.orchestra.run.smithi018.stdout:node-exporter.smithi018 smithi018 *:9100 running (11m) 113s ago 24m 7780k - 1.5.0 0da6a335fe13 326c3d9c6847 2024-04-12T21:09:39.119 INFO:teuthology.orchestra.run.smithi018.stdout:node-exporter.smithi186 smithi186 *:9100 running (11m) 2m ago 23m 7826k - 1.5.0 0da6a335fe13 ec2112baeecf 2024-04-12T21:09:39.119 INFO:teuthology.orchestra.run.smithi018.stdout:osd.0 smithi018 running (5m) 113s ago 22m 121M 1559M 19.0.0-2879-g324baf8d 8717720dc5e5 60b1273ca047 2024-04-12T21:09:39.119 INFO:teuthology.orchestra.run.smithi018.stdout:osd.1 smithi018 running (5m) 113s ago 21m 91.3M 1559M 19.0.0-2879-g324baf8d 8717720dc5e5 170ce1e5fb8d 2024-04-12T21:09:39.119 INFO:teuthology.orchestra.run.smithi018.stdout:osd.2 smithi018 running (4m) 113s ago 21m 90.3M 1559M 19.0.0-2879-g324baf8d 8717720dc5e5 c2cbc6aa2839 2024-04-12T21:09:39.120 INFO:teuthology.orchestra.run.smithi018.stdout:osd.3 smithi186 running (4m) 2m ago 20m 96.6M 2581M 19.0.0-2879-g324baf8d 8717720dc5e5 0138e8262e4b 2024-04-12T21:09:39.120 INFO:teuthology.orchestra.run.smithi018.stdout:osd.4 smithi186 running (3m) 2m ago 20m 87.2M 2581M 19.0.0-2879-g324baf8d 8717720dc5e5 b20cb63b25c9 2024-04-12T21:09:39.120 INFO:teuthology.orchestra.run.smithi018.stdout:osd.5 smithi186 running (2m) 2m ago 19m 11.1M 2581M 19.0.0-2879-g324baf8d 8717720dc5e5 2513bea2610e 2024-04-12T21:09:39.120 INFO:teuthology.orchestra.run.smithi018.stdout:prometheus.smithi018 smithi018 *:9095 running (10m) 113s ago 24m 60.6M - 2.43.0 a07b618ecd1d f58841127e21 2024-04-12T21:09:39.121 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:39.124+0000 7f3011800700 1 -- 172.21.15.18:0/158065980 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f2fec06ec10 msgr2=0x7f2fec0710c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:09:39.121 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:39.124+0000 7f3011800700 1 --2- 172.21.15.18:0/158065980 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f2fec06ec10 0x7f2fec0710c0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f2ff4006fd0 tx=0x7f2ff4008040 comp rx=0 tx=0).stop 2024-04-12T21:09:39.121 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:39.124+0000 7f3011800700 1 -- 172.21.15.18:0/158065980 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f300c105010 msgr2=0x7f300c115d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:09:39.121 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:39.124+0000 7f3011800700 1 --2- 172.21.15.18:0/158065980 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f300c105010 0x7f300c115d90 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f2ffc009150 tx=0x7f2ffc009e30 comp rx=0 tx=0).stop 2024-04-12T21:09:39.121 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:39.124+0000 7f3011800700 1 -- 172.21.15.18:0/158065980 shutdown_connections 2024-04-12T21:09:39.121 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:39.124+0000 7f3011800700 1 --2- 172.21.15.18:0/158065980 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f2fec06ec10 0x7f2fec0710c0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:09:39.122 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:39.124+0000 7f3011800700 1 --2- 172.21.15.18:0/158065980 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f300c105010 0x7f300c115d90 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:09:39.122 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:39.124+0000 7f3011800700 1 --2- 172.21.15.18:0/158065980 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f300c1059b0 0x7f300c1162d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:09:39.122 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:39.124+0000 7f3011800700 1 -- 172.21.15.18:0/158065980 >> 172.21.15.18:0/158065980 conn(0x7f300c0faba0 msgr2=0x7f300c0fbb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:09:39.122 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:39.125+0000 7f3011800700 1 -- 172.21.15.18:0/158065980 shutdown_connections 2024-04-12T21:09:39.122 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:39.125+0000 7f3011800700 1 -- 172.21.15.18:0/158065980 wait complete. 2024-04-12T21:09:39.264 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:39.267+0000 7f95a4a00700 1 -- 172.21.15.18:0/3660538091 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f95a0065630 msgr2=0x7f95a0065a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:09:39.265 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:39.267+0000 7f95a4a00700 1 --2- 172.21.15.18:0/3660538091 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f95a0065630 0x7f95a0065a80 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f958c004940 tx=0x7f958c009e30 comp rx=0 tx=0).stop 2024-04-12T21:09:39.265 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:39.267+0000 7f95a4a00700 1 -- 172.21.15.18:0/3660538091 shutdown_connections 2024-04-12T21:09:39.265 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:39.267+0000 7f95a4a00700 1 --2- 172.21.15.18:0/3660538091 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f95a0065630 0x7f95a0065a80 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:09:39.265 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:39.267+0000 7f95a4a00700 1 --2- 172.21.15.18:0/3660538091 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f95a0102ca0 0x7f95a0103070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:09:39.265 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:39.267+0000 7f95a4a00700 1 -- 172.21.15.18:0/3660538091 >> 172.21.15.18:0/3660538091 conn(0x7f95a0071b70 msgr2=0x7f95a0071f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:09:39.265 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:39.268+0000 7f95a4a00700 1 -- 172.21.15.18:0/3660538091 shutdown_connections 2024-04-12T21:09:39.265 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:39.268+0000 7f95a4a00700 1 -- 172.21.15.18:0/3660538091 wait complete. 2024-04-12T21:09:39.266 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:39.268+0000 7f95a4a00700 1 Processor -- start 2024-04-12T21:09:39.266 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:39.269+0000 7f95a4a00700 1 -- start start 2024-04-12T21:09:39.267 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:39.269+0000 7f95a4a00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f95a0065630 0x7f95a0115ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:09:39.267 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:39.269+0000 7f95a4a00700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f95a0102ca0 0x7f95a01160e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:09:39.267 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:39.269+0000 7f95a4a00700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95a0116730 con 0x7f95a0065630 2024-04-12T21:09:39.267 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:39.269+0000 7f95a4a00700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95a010fb20 con 0x7f95a0102ca0 2024-04-12T21:09:39.267 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:39.269+0000 7f959f400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f95a0065630 0x7f95a0115ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:09:39.267 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:39.269+0000 7f959f400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f95a0065630 0x7f95a0115ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:40708/0 (socket says 172.21.15.18:40708) 2024-04-12T21:09:39.267 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:39.269+0000 7f959f400700 1 -- 172.21.15.18:0/2985927620 learned_addr learned my addr 172.21.15.18:0/2985927620 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:09:39.267 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:39.270+0000 7f959ea00700 1 --2- 172.21.15.18:0/2985927620 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f95a0102ca0 0x7f95a01160e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:09:39.267 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:39.270+0000 7f959f400700 1 -- 172.21.15.18:0/2985927620 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f95a0102ca0 msgr2=0x7f95a01160e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:09:39.267 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:39.270+0000 7f959f400700 1 --2- 172.21.15.18:0/2985927620 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f95a0102ca0 0x7f95a01160e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:09:39.267 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:39.270+0000 7f959f400700 1 -- 172.21.15.18:0/2985927620 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f958c005040 con 0x7f95a0065630 2024-04-12T21:09:39.267 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:39.270+0000 7f959f400700 1 --2- 172.21.15.18:0/2985927620 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f95a0065630 0x7f95a0115ba0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f959000c830 tx=0x7f9590004470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:09:39.268 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:39.270+0000 7f9597400700 1 -- 172.21.15.18:0/2985927620 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f959000a760 con 0x7f95a0065630 2024-04-12T21:09:39.268 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:39.270+0000 7f9597400700 1 -- 172.21.15.18:0/2985927620 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f959000ad90 con 0x7f95a0065630 2024-04-12T21:09:39.268 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:39.270+0000 7f9597400700 1 -- 172.21.15.18:0/2985927620 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f95900113a0 con 0x7f95a0065630 2024-04-12T21:09:39.268 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:39.271+0000 7f95a4a00700 1 -- 172.21.15.18:0/2985927620 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f95a010fe00 con 0x7f95a0065630 2024-04-12T21:09:39.268 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:39.271+0000 7f95a4a00700 1 -- 172.21.15.18:0/2985927620 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f95a01103c0 con 0x7f95a0065630 2024-04-12T21:09:39.268 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:39.272+0000 7f95a4a00700 1 -- 172.21.15.18:0/2985927620 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f95a0101660 con 0x7f95a0065630 2024-04-12T21:09:39.269 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:39.273+0000 7f9597400700 1 -- 172.21.15.18:0/2985927620 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7f9590012400 con 0x7f95a0065630 2024-04-12T21:09:39.270 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:39.274+0000 7f9597400700 1 --2- 172.21.15.18:0/2985927620 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f958406ec10 0x7f95840710c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:09:39.270 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:39.274+0000 7f9597400700 1 -- 172.21.15.18:0/2985927620 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 6757+0+0 (secure 0 0 0) 0x7f959008e9c0 con 0x7f95a0065630 2024-04-12T21:09:39.270 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:39.274+0000 7f959ea00700 1 --2- 172.21.15.18:0/2985927620 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f958406ec10 0x7f95840710c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:09:39.271 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:39.275+0000 7f959ea00700 1 --2- 172.21.15.18:0/2985927620 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f958406ec10 0x7f95840710c0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f958c00b020 tx=0x7f958c009e30 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:09:39.275 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:39.279+0000 7f9597400700 1 -- 172.21.15.18:0/2985927620 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f959005b1f0 con 0x7f95a0065630 2024-04-12T21:09:39.350 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:09:39 smithi018 ceph-mon[112590]: pgmap v295: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:09:39.350 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:09:39 smithi018 ceph-mon[112590]: from='client.34556 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:09:39.529 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:09:39 smithi186 ceph-mon[86379]: pgmap v295: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:09:39.529 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:09:39 smithi186 ceph-mon[86379]: from='client.34556 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:09:39.601 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:39.604+0000 7f95a4a00700 1 -- 172.21.15.18:0/2985927620 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f95a0111250 con 0x7f95a0065630 2024-04-12T21:09:39.601 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:39.605+0000 7f9597400700 1 -- 172.21.15.18:0/2985927620 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+837 (secure 0 0 0) 0x7f959005ad80 con 0x7f95a0065630 2024-04-12T21:09:39.602 INFO:teuthology.orchestra.run.smithi018.stdout:{ 2024-04-12T21:09:39.602 INFO:teuthology.orchestra.run.smithi018.stdout: "mon": { 2024-04-12T21:09:39.602 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 19.0.0-2879-g324baf8d (324baf8ddb91dea46cac50cb535c306115791355) squid (dev)": 2 2024-04-12T21:09:39.602 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-12T21:09:39.602 INFO:teuthology.orchestra.run.smithi018.stdout: "mgr": { 2024-04-12T21:09:39.602 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 19.0.0-2879-g324baf8d (324baf8ddb91dea46cac50cb535c306115791355) squid (dev)": 2 2024-04-12T21:09:39.602 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-12T21:09:39.602 INFO:teuthology.orchestra.run.smithi018.stdout: "osd": { 2024-04-12T21:09:39.603 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 19.0.0-2879-g324baf8d (324baf8ddb91dea46cac50cb535c306115791355) squid (dev)": 6 2024-04-12T21:09:39.603 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-12T21:09:39.603 INFO:teuthology.orchestra.run.smithi018.stdout: "mds": { 2024-04-12T21:09:39.603 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 17.2.7-827-gd3b5060a (d3b5060a17449edfae9385cdf792afec109fca5d) quincy (stable)": 2, 2024-04-12T21:09:39.603 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 19.0.0-2879-g324baf8d (324baf8ddb91dea46cac50cb535c306115791355) squid (dev)": 2 2024-04-12T21:09:39.603 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-12T21:09:39.603 INFO:teuthology.orchestra.run.smithi018.stdout: "overall": { 2024-04-12T21:09:39.603 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 17.2.7-827-gd3b5060a (d3b5060a17449edfae9385cdf792afec109fca5d) quincy (stable)": 2, 2024-04-12T21:09:39.603 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 19.0.0-2879-g324baf8d (324baf8ddb91dea46cac50cb535c306115791355) squid (dev)": 12 2024-04-12T21:09:39.603 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-04-12T21:09:39.603 INFO:teuthology.orchestra.run.smithi018.stdout:} 2024-04-12T21:09:39.604 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:39.607+0000 7f95a4a00700 1 -- 172.21.15.18:0/2985927620 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f958406ec10 msgr2=0x7f95840710c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:09:39.604 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:39.607+0000 7f95a4a00700 1 --2- 172.21.15.18:0/2985927620 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f958406ec10 0x7f95840710c0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f958c00b020 tx=0x7f958c009e30 comp rx=0 tx=0).stop 2024-04-12T21:09:39.604 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:39.607+0000 7f95a4a00700 1 -- 172.21.15.18:0/2985927620 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f95a0065630 msgr2=0x7f95a0115ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:09:39.604 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:39.607+0000 7f95a4a00700 1 --2- 172.21.15.18:0/2985927620 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f95a0065630 0x7f95a0115ba0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f959000c830 tx=0x7f9590004470 comp rx=0 tx=0).stop 2024-04-12T21:09:39.605 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:39.607+0000 7f95a4a00700 1 -- 172.21.15.18:0/2985927620 shutdown_connections 2024-04-12T21:09:39.605 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:39.607+0000 7f95a4a00700 1 --2- 172.21.15.18:0/2985927620 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f958406ec10 0x7f95840710c0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:09:39.605 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:39.607+0000 7f95a4a00700 1 --2- 172.21.15.18:0/2985927620 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f95a0065630 0x7f95a0115ba0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:09:39.605 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:39.608+0000 7f95a4a00700 1 --2- 172.21.15.18:0/2985927620 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f95a0102ca0 0x7f95a01160e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:09:39.605 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:39.608+0000 7f95a4a00700 1 -- 172.21.15.18:0/2985927620 >> 172.21.15.18:0/2985927620 conn(0x7f95a0071b70 msgr2=0x7f95a00fe7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:09:39.605 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:39.608+0000 7f95a4a00700 1 -- 172.21.15.18:0/2985927620 shutdown_connections 2024-04-12T21:09:39.605 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:39.608+0000 7f95a4a00700 1 -- 172.21.15.18:0/2985927620 wait complete. 2024-04-12T21:09:39.749 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:39.752+0000 7f2159600700 1 -- 172.21.15.18:0/65359260 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2154071f90 msgr2=0x7f2154070360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:09:39.749 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:39.752+0000 7f2159600700 1 --2- 172.21.15.18:0/65359260 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2154071f90 0x7f2154070360 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f2144004970 tx=0x7f2144009e30 comp rx=0 tx=0).stop 2024-04-12T21:09:39.749 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:39.752+0000 7f2159600700 1 -- 172.21.15.18:0/65359260 shutdown_connections 2024-04-12T21:09:39.749 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:39.752+0000 7f2159600700 1 --2- 172.21.15.18:0/65359260 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f21540708a0 0x7f2154070d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:09:39.749 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:39.752+0000 7f2159600700 1 --2- 172.21.15.18:0/65359260 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2154071f90 0x7f2154070360 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:09:39.749 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:39.752+0000 7f2159600700 1 -- 172.21.15.18:0/65359260 >> 172.21.15.18:0/65359260 conn(0x7f21540f89b0 msgr2=0x7f21540fadc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:09:39.749 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:39.752+0000 7f2159600700 1 -- 172.21.15.18:0/65359260 shutdown_connections 2024-04-12T21:09:39.750 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:39.753+0000 7f2159600700 1 -- 172.21.15.18:0/65359260 wait complete. 2024-04-12T21:09:39.750 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:39.753+0000 7f2159600700 1 Processor -- start 2024-04-12T21:09:39.750 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:39.754+0000 7f2159600700 1 -- start start 2024-04-12T21:09:39.751 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:39.754+0000 7f2159600700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f21540708a0 0x7f215411a1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:09:39.751 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:39.754+0000 7f2159600700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2154071f90 0x7f215411a6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:09:39.751 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:39.754+0000 7f2159600700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f215411ad30 con 0x7f2154071f90 2024-04-12T21:09:39.751 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:39.754+0000 7f2159600700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2154114100 con 0x7f21540708a0 2024-04-12T21:09:39.752 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:39.755+0000 7f214be00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2154071f90 0x7f215411a6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:09:39.752 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:39.755+0000 7f214be00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2154071f90 0x7f215411a6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:40720/0 (socket says 172.21.15.18:40720) 2024-04-12T21:09:39.752 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:39.755+0000 7f214be00700 1 -- 172.21.15.18:0/2002963029 learned_addr learned my addr 172.21.15.18:0/2002963029 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:09:39.752 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:39.755+0000 7f2152a00700 1 --2- 172.21.15.18:0/2002963029 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f21540708a0 0x7f215411a1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:09:39.752 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:39.755+0000 7f214be00700 1 -- 172.21.15.18:0/2002963029 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f21540708a0 msgr2=0x7f215411a1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:09:39.752 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:39.755+0000 7f214be00700 1 --2- 172.21.15.18:0/2002963029 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f21540708a0 0x7f215411a1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:09:39.752 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:39.755+0000 7f214be00700 1 -- 172.21.15.18:0/2002963029 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2144005040 con 0x7f2154071f90 2024-04-12T21:09:39.754 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:39.756+0000 7f214be00700 1 --2- 172.21.15.18:0/2002963029 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2154071f90 0x7f215411a6e0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f213c002a50 tx=0x7f213c0046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:09:39.754 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:39.756+0000 7f214aa00700 1 -- 172.21.15.18:0/2002963029 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f213c007cc0 con 0x7f2154071f90 2024-04-12T21:09:39.754 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:39.756+0000 7f214aa00700 1 -- 172.21.15.18:0/2002963029 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f213c00f040 con 0x7f2154071f90 2024-04-12T21:09:39.754 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:39.756+0000 7f214aa00700 1 -- 172.21.15.18:0/2002963029 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f213c009640 con 0x7f2154071f90 2024-04-12T21:09:39.754 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:39.756+0000 7f2159600700 1 -- 172.21.15.18:0/2002963029 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f21541143e0 con 0x7f2154071f90 2024-04-12T21:09:39.754 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:39.756+0000 7f2159600700 1 -- 172.21.15.18:0/2002963029 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2154114920 con 0x7f2154071f90 2024-04-12T21:09:39.754 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:39.757+0000 7f2159600700 1 -- 172.21.15.18:0/2002963029 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2154071aa0 con 0x7f2154071f90 2024-04-12T21:09:39.755 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:39.758+0000 7f214aa00700 1 -- 172.21.15.18:0/2002963029 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7f213c0097a0 con 0x7f2154071f90 2024-04-12T21:09:39.756 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:39.759+0000 7f214aa00700 1 --2- 172.21.15.18:0/2002963029 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f213806ec00 0x7f21380710b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:09:39.756 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:39.759+0000 7f214aa00700 1 -- 172.21.15.18:0/2002963029 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 6757+0+0 (secure 0 0 0) 0x7f213c08e510 con 0x7f2154071f90 2024-04-12T21:09:39.756 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:39.760+0000 7f2152a00700 1 --2- 172.21.15.18:0/2002963029 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f213806ec00 0x7f21380710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:09:39.757 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:39.760+0000 7f2152a00700 1 --2- 172.21.15.18:0/2002963029 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f213806ec00 0x7f21380710b0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f2144004f50 tx=0x7f2144009e30 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:09:39.760 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:39.764+0000 7f214aa00700 1 -- 172.21.15.18:0/2002963029 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f213c05acc0 con 0x7f2154071f90 2024-04-12T21:09:40.031 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:40.034+0000 7f2159600700 1 -- 172.21.15.18:0/2002963029 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f215405f5f0 con 0x7f2154071f90 2024-04-12T21:09:40.032 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:40.036+0000 7f214aa00700 1 -- 172.21.15.18:0/2002963029 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 29 v29) v1 ==== 76+0+1872 (secure 0 0 0) 0x7f213c05a850 con 0x7f2154071f90 2024-04-12T21:09:40.033 INFO:teuthology.orchestra.run.smithi018.stdout:e29 2024-04-12T21:09:40.033 INFO:teuthology.orchestra.run.smithi018.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-12T21:09:40.033 INFO:teuthology.orchestra.run.smithi018.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-12T21:09:40.033 INFO:teuthology.orchestra.run.smithi018.stdout:legacy client fscid: 1 2024-04-12T21:09:40.033 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-12T21:09:40.033 INFO:teuthology.orchestra.run.smithi018.stdout:Filesystem 'cephfs' (1) 2024-04-12T21:09:40.033 INFO:teuthology.orchestra.run.smithi018.stdout:fs_name cephfs 2024-04-12T21:09:40.033 INFO:teuthology.orchestra.run.smithi018.stdout:epoch 24 2024-04-12T21:09:40.033 INFO:teuthology.orchestra.run.smithi018.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-04-12T21:09:40.033 INFO:teuthology.orchestra.run.smithi018.stdout:created 2024-04-12T20:51:57.362850+0000 2024-04-12T21:09:40.034 INFO:teuthology.orchestra.run.smithi018.stdout:modified 2024-04-12T21:06:59.717190+0000 2024-04-12T21:09:40.034 INFO:teuthology.orchestra.run.smithi018.stdout:tableserver 0 2024-04-12T21:09:40.034 INFO:teuthology.orchestra.run.smithi018.stdout:root 0 2024-04-12T21:09:40.034 INFO:teuthology.orchestra.run.smithi018.stdout:session_timeout 60 2024-04-12T21:09:40.034 INFO:teuthology.orchestra.run.smithi018.stdout:session_autoclose 300 2024-04-12T21:09:40.034 INFO:teuthology.orchestra.run.smithi018.stdout:max_file_size 1099511627776 2024-04-12T21:09:40.034 INFO:teuthology.orchestra.run.smithi018.stdout:max_xattr_size 65536 2024-04-12T21:09:40.034 INFO:teuthology.orchestra.run.smithi018.stdout:required_client_features {} 2024-04-12T21:09:40.034 INFO:teuthology.orchestra.run.smithi018.stdout:last_failure 0 2024-04-12T21:09:40.034 INFO:teuthology.orchestra.run.smithi018.stdout:last_failure_osd_epoch 100 2024-04-12T21:09:40.034 INFO:teuthology.orchestra.run.smithi018.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-12T21:09:40.034 INFO:teuthology.orchestra.run.smithi018.stdout:max_mds 2 2024-04-12T21:09:40.034 INFO:teuthology.orchestra.run.smithi018.stdout:in 0,1 2024-04-12T21:09:40.034 INFO:teuthology.orchestra.run.smithi018.stdout:up {} 2024-04-12T21:09:40.035 INFO:teuthology.orchestra.run.smithi018.stdout:failed 0,1 2024-04-12T21:09:40.035 INFO:teuthology.orchestra.run.smithi018.stdout:damaged 2024-04-12T21:09:40.035 INFO:teuthology.orchestra.run.smithi018.stdout:stopped 2024-04-12T21:09:40.035 INFO:teuthology.orchestra.run.smithi018.stdout:data_pools [3] 2024-04-12T21:09:40.035 INFO:teuthology.orchestra.run.smithi018.stdout:metadata_pool 2 2024-04-12T21:09:40.035 INFO:teuthology.orchestra.run.smithi018.stdout:inline_data disabled 2024-04-12T21:09:40.035 INFO:teuthology.orchestra.run.smithi018.stdout:balancer 2024-04-12T21:09:40.035 INFO:teuthology.orchestra.run.smithi018.stdout:bal_rank_mask -1 2024-04-12T21:09:40.035 INFO:teuthology.orchestra.run.smithi018.stdout:standby_count_wanted 1 2024-04-12T21:09:40.035 INFO:teuthology.orchestra.run.smithi018.stdout:qdb_cluster leader: 0 members: 2024-04-12T21:09:40.036 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-12T21:09:40.036 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-12T21:09:40.036 INFO:teuthology.orchestra.run.smithi018.stdout:Standby daemons: 2024-04-12T21:09:40.036 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-12T21:09:40.036 INFO:teuthology.orchestra.run.smithi018.stdout:[mds.cephfs.smithi186.wjnafs{-1:24277} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.186:6826/438347207,v1:172.21.15.186:6827/438347207] compat {c=[1],r=[1],i=[7ff]}] 2024-04-12T21:09:40.036 INFO:teuthology.orchestra.run.smithi018.stdout:[mds.cephfs.smithi186.jdllyn{-1:24281} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.186:6824/1464359952,v1:172.21.15.186:6825/1464359952] compat {c=[1],r=[1],i=[7ff]}] 2024-04-12T21:09:40.036 INFO:teuthology.orchestra.run.smithi018.stdout:[mds.cephfs.smithi018.acqbsc{-1:34436} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.18:6828/755179764,v1:172.21.15.18:6829/755179764] compat {c=[1],r=[1],i=[1fff]}] 2024-04-12T21:09:40.036 INFO:teuthology.orchestra.run.smithi018.stdout:[mds.cephfs.smithi018.szjluo{-1:34468} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.18:6826/2407992653,v1:172.21.15.18:6827/2407992653] compat {c=[1],r=[1],i=[1fff]}] 2024-04-12T21:09:40.037 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:40.039+0000 7f2159600700 1 -- 172.21.15.18:0/2002963029 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f213806ec00 msgr2=0x7f21380710b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:09:40.037 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:40.039+0000 7f2159600700 1 --2- 172.21.15.18:0/2002963029 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f213806ec00 0x7f21380710b0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f2144004f50 tx=0x7f2144009e30 comp rx=0 tx=0).stop 2024-04-12T21:09:40.037 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:40.039+0000 7f2159600700 1 -- 172.21.15.18:0/2002963029 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2154071f90 msgr2=0x7f215411a6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:09:40.037 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:40.039+0000 7f2159600700 1 --2- 172.21.15.18:0/2002963029 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2154071f90 0x7f215411a6e0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f213c002a50 tx=0x7f213c0046b0 comp rx=0 tx=0).stop 2024-04-12T21:09:40.037 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:40.039+0000 7f2159600700 1 -- 172.21.15.18:0/2002963029 shutdown_connections 2024-04-12T21:09:40.037 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:40.039+0000 7f2159600700 1 --2- 172.21.15.18:0/2002963029 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f213806ec00 0x7f21380710b0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:09:40.037 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:40.039+0000 7f2159600700 1 --2- 172.21.15.18:0/2002963029 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f21540708a0 0x7f215411a1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:09:40.037 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:40.039+0000 7f2159600700 1 --2- 172.21.15.18:0/2002963029 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2154071f90 0x7f215411a6e0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:09:40.038 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:40.039+0000 7f2159600700 1 -- 172.21.15.18:0/2002963029 >> 172.21.15.18:0/2002963029 conn(0x7f21540f89b0 msgr2=0x7f21540f9aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:09:40.038 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:40.039+0000 7f2159600700 1 -- 172.21.15.18:0/2002963029 shutdown_connections 2024-04-12T21:09:40.038 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:40.039+0000 7f2159600700 1 -- 172.21.15.18:0/2002963029 wait complete. 2024-04-12T21:09:40.038 INFO:teuthology.orchestra.run.smithi018.stderr:dumped fsmap epoch 29 2024-04-12T21:09:40.194 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:40.197+0000 7fa496600700 1 -- 172.21.15.18:0/3343119664 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa4900724f0 msgr2=0x7fa4900728c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:09:40.194 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:40.197+0000 7fa496600700 1 --2- 172.21.15.18:0/3343119664 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa4900724f0 0x7fa4900728c0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fa480004940 tx=0x7fa480009e30 comp rx=0 tx=0).stop 2024-04-12T21:09:40.194 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:40.197+0000 7fa496600700 1 -- 172.21.15.18:0/3343119664 shutdown_connections 2024-04-12T21:09:40.194 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:40.197+0000 7fa496600700 1 --2- 172.21.15.18:0/3343119664 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa490072e00 0x7fa49010d760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:09:40.195 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:40.197+0000 7fa496600700 1 --2- 172.21.15.18:0/3343119664 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa4900724f0 0x7fa4900728c0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:09:40.195 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:40.197+0000 7fa496600700 1 -- 172.21.15.18:0/3343119664 >> 172.21.15.18:0/3343119664 conn(0x7fa4900faba0 msgr2=0x7fa4900fcfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:09:40.195 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:40.197+0000 7fa496600700 1 -- 172.21.15.18:0/3343119664 shutdown_connections 2024-04-12T21:09:40.195 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:40.198+0000 7fa496600700 1 -- 172.21.15.18:0/3343119664 wait complete. 2024-04-12T21:09:40.196 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:40.198+0000 7fa496600700 1 Processor -- start 2024-04-12T21:09:40.197 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:40.198+0000 7fa496600700 1 -- start start 2024-04-12T21:09:40.197 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:40.199+0000 7fa496600700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa4900724f0 0x7fa490119e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:09:40.197 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:40.199+0000 7fa496600700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa490072e00 0x7fa49011a3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:09:40.197 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:40.199+0000 7fa496600700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa49011aa10 con 0x7fa490072e00 2024-04-12T21:09:40.197 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:40.199+0000 7fa496600700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa490114080 con 0x7fa4900724f0 2024-04-12T21:09:40.197 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:40.199+0000 7fa48ea00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa490072e00 0x7fa49011a3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:09:40.197 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:40.199+0000 7fa48ea00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa490072e00 0x7fa49011a3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:40732/0 (socket says 172.21.15.18:40732) 2024-04-12T21:09:40.197 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:40.199+0000 7fa48ea00700 1 -- 172.21.15.18:0/3406886738 learned_addr learned my addr 172.21.15.18:0/3406886738 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:09:40.197 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:40.199+0000 7fa48f400700 1 --2- 172.21.15.18:0/3406886738 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa4900724f0 0x7fa490119e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:09:40.197 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:40.199+0000 7fa48ea00700 1 -- 172.21.15.18:0/3406886738 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa4900724f0 msgr2=0x7fa490119e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:09:40.197 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:40.200+0000 7fa48ea00700 1 --2- 172.21.15.18:0/3406886738 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa4900724f0 0x7fa490119e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:09:40.197 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:40.200+0000 7fa48ea00700 1 -- 172.21.15.18:0/3406886738 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa480005040 con 0x7fa490072e00 2024-04-12T21:09:40.197 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:40.200+0000 7fa48ea00700 1 --2- 172.21.15.18:0/3406886738 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa490072e00 0x7fa49011a3c0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fa47c00c830 tx=0x7fa47c004470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:09:40.198 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:40.200+0000 7fa487400700 1 -- 172.21.15.18:0/3406886738 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa47c00a760 con 0x7fa490072e00 2024-04-12T21:09:40.198 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:40.200+0000 7fa487400700 1 -- 172.21.15.18:0/3406886738 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa47c00ad90 con 0x7fa490072e00 2024-04-12T21:09:40.198 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:40.200+0000 7fa487400700 1 -- 172.21.15.18:0/3406886738 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa47c0113a0 con 0x7fa490072e00 2024-04-12T21:09:40.198 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:40.200+0000 7fa496600700 1 -- 172.21.15.18:0/3406886738 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa490114360 con 0x7fa490072e00 2024-04-12T21:09:40.198 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:40.200+0000 7fa496600700 1 -- 172.21.15.18:0/3406886738 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa490114920 con 0x7fa490072e00 2024-04-12T21:09:40.198 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:40.202+0000 7fa496600700 1 -- 172.21.15.18:0/3406886738 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa49010a5a0 con 0x7fa490072e00 2024-04-12T21:09:40.199 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:40.203+0000 7fa487400700 1 -- 172.21.15.18:0/3406886738 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7fa47c012400 con 0x7fa490072e00 2024-04-12T21:09:40.200 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:40.204+0000 7fa487400700 1 --2- 172.21.15.18:0/3406886738 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fa47406ec00 0x7fa4740710b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:09:40.200 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:40.204+0000 7fa487400700 1 -- 172.21.15.18:0/3406886738 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 6757+0+0 (secure 0 0 0) 0x7fa47c08e9c0 con 0x7fa490072e00 2024-04-12T21:09:40.200 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:40.204+0000 7fa48f400700 1 --2- 172.21.15.18:0/3406886738 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fa47406ec00 0x7fa4740710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:09:40.201 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:40.205+0000 7fa48f400700 1 --2- 172.21.15.18:0/3406886738 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fa47406ec00 0x7fa4740710b0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fa480000c00 tx=0x7fa480009e30 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:09:40.205 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:40.209+0000 7fa487400700 1 -- 172.21.15.18:0/3406886738 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7fa47c0576c0 con 0x7fa490072e00 2024-04-12T21:09:40.350 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:09:40 smithi018 ceph-mon[112590]: from='client.34560 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:09:40.350 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:09:40 smithi018 ceph-mon[112590]: from='client.34564 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:09:40.350 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:09:40 smithi018 ceph-mon[112590]: from='client.? 172.21.15.18:0/2985927620' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T21:09:40.350 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:09:40 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T21:09:40.350 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:09:40 smithi018 ceph-mon[112590]: from='client.? 172.21.15.18:0/2002963029' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-12T21:09:40.457 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:40.461+0000 7fa496600700 1 -- 172.21.15.18:0/3406886738 --> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fa490115510 con 0x7fa47406ec00 2024-04-12T21:09:40.460 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:40.463+0000 7fa487400700 1 -- 172.21.15.18:0/3406886738 <== mgr.34102 v2:172.21.15.18:6800/3812891919 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7fa490115510 con 0x7fa47406ec00 2024-04-12T21:09:40.460 INFO:teuthology.orchestra.run.smithi018.stdout:{ 2024-04-12T21:09:40.460 INFO:teuthology.orchestra.run.smithi018.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:f4452def3be4d350f4a8e5feaf7ebb1511be9298fe68478ba7a2d6fbd7d69a63", 2024-04-12T21:09:40.460 INFO:teuthology.orchestra.run.smithi018.stdout: "in_progress": true, 2024-04-12T21:09:40.461 INFO:teuthology.orchestra.run.smithi018.stdout: "which": "Upgrading all daemon types on all hosts", 2024-04-12T21:09:40.461 INFO:teuthology.orchestra.run.smithi018.stdout: "services_complete": [ 2024-04-12T21:09:40.461 INFO:teuthology.orchestra.run.smithi018.stdout: "crash", 2024-04-12T21:09:40.461 INFO:teuthology.orchestra.run.smithi018.stdout: "osd", 2024-04-12T21:09:40.461 INFO:teuthology.orchestra.run.smithi018.stdout: "mon", 2024-04-12T21:09:40.461 INFO:teuthology.orchestra.run.smithi018.stdout: "mgr" 2024-04-12T21:09:40.461 INFO:teuthology.orchestra.run.smithi018.stdout: ], 2024-04-12T21:09:40.461 INFO:teuthology.orchestra.run.smithi018.stdout: "progress": "14/21 daemons upgraded", 2024-04-12T21:09:40.462 INFO:teuthology.orchestra.run.smithi018.stdout: "message": "Currently upgrading mds daemons", 2024-04-12T21:09:40.462 INFO:teuthology.orchestra.run.smithi018.stdout: "is_paused": false 2024-04-12T21:09:40.462 INFO:teuthology.orchestra.run.smithi018.stdout:} 2024-04-12T21:09:40.463 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:40.466+0000 7fa496600700 1 -- 172.21.15.18:0/3406886738 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fa47406ec00 msgr2=0x7fa4740710b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:09:40.463 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:40.466+0000 7fa496600700 1 --2- 172.21.15.18:0/3406886738 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fa47406ec00 0x7fa4740710b0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fa480000c00 tx=0x7fa480009e30 comp rx=0 tx=0).stop 2024-04-12T21:09:40.463 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:40.466+0000 7fa496600700 1 -- 172.21.15.18:0/3406886738 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa490072e00 msgr2=0x7fa49011a3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:09:40.463 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:40.466+0000 7fa496600700 1 --2- 172.21.15.18:0/3406886738 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa490072e00 0x7fa49011a3c0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fa47c00c830 tx=0x7fa47c004470 comp rx=0 tx=0).stop 2024-04-12T21:09:40.463 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:40.467+0000 7fa496600700 1 -- 172.21.15.18:0/3406886738 shutdown_connections 2024-04-12T21:09:40.464 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:40.467+0000 7fa496600700 1 --2- 172.21.15.18:0/3406886738 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fa47406ec00 0x7fa4740710b0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:09:40.464 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:40.467+0000 7fa496600700 1 --2- 172.21.15.18:0/3406886738 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa4900724f0 0x7fa490119e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:09:40.464 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:40.467+0000 7fa496600700 1 --2- 172.21.15.18:0/3406886738 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa490072e00 0x7fa49011a3c0 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:09:40.464 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:40.467+0000 7fa496600700 1 -- 172.21.15.18:0/3406886738 >> 172.21.15.18:0/3406886738 conn(0x7fa4900faba0 msgr2=0x7fa490107720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:09:40.464 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:40.467+0000 7fa496600700 1 -- 172.21.15.18:0/3406886738 shutdown_connections 2024-04-12T21:09:40.464 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:40.467+0000 7fa496600700 1 -- 172.21.15.18:0/3406886738 wait complete. 2024-04-12T21:09:40.529 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:09:40 smithi186 ceph-mon[86379]: from='client.34560 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:09:40.529 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:09:40 smithi186 ceph-mon[86379]: from='client.34564 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:09:40.529 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:09:40 smithi186 ceph-mon[86379]: from='client.? 172.21.15.18:0/2985927620' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T21:09:40.529 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:09:40 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T21:09:40.530 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:09:40 smithi186 ceph-mon[86379]: from='client.? 172.21.15.18:0/2002963029' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-12T21:09:40.625 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:40.628+0000 7f19eb200700 1 -- 172.21.15.18:0/4268469286 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f19ec071f90 msgr2=0x7f19ec070360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:09:40.625 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:40.628+0000 7f19eb200700 1 --2- 172.21.15.18:0/4268469286 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f19ec071f90 0x7f19ec070360 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f19dc004970 tx=0x7f19dc009e30 comp rx=0 tx=0).stop 2024-04-12T21:09:40.625 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:40.628+0000 7f19eb200700 1 -- 172.21.15.18:0/4268469286 shutdown_connections 2024-04-12T21:09:40.626 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:40.628+0000 7f19eb200700 1 --2- 172.21.15.18:0/4268469286 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f19ec0708a0 0x7f19ec070d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:09:40.626 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:40.629+0000 7f19eb200700 1 --2- 172.21.15.18:0/4268469286 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f19ec071f90 0x7f19ec070360 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:09:40.626 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:40.629+0000 7f19eb200700 1 -- 172.21.15.18:0/4268469286 >> 172.21.15.18:0/4268469286 conn(0x7f19ec0f8760 msgr2=0x7f19ec0fab70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:09:40.626 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:40.629+0000 7f19eb200700 1 -- 172.21.15.18:0/4268469286 shutdown_connections 2024-04-12T21:09:40.626 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:40.629+0000 7f19eb200700 1 -- 172.21.15.18:0/4268469286 wait complete. 2024-04-12T21:09:40.627 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:40.630+0000 7f19eb200700 1 Processor -- start 2024-04-12T21:09:40.627 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:40.630+0000 7f19eb200700 1 -- start start 2024-04-12T21:09:40.627 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:40.630+0000 7f19eb200700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f19ec0708a0 0x7f19ec119fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:09:40.627 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:40.630+0000 7f19eb200700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f19ec071f90 0x7f19ec11a4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:09:40.627 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:40.630+0000 7f19eb200700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19ec11ab30 con 0x7f19ec071f90 2024-04-12T21:09:40.627 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:40.630+0000 7f19eb200700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19ec113f00 con 0x7f19ec0708a0 2024-04-12T21:09:40.627 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:40.630+0000 7f19e9400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f19ec071f90 0x7f19ec11a4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:09:40.628 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:40.631+0000 7f19e9400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f19ec071f90 0x7f19ec11a4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:40752/0 (socket says 172.21.15.18:40752) 2024-04-12T21:09:40.628 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:40.631+0000 7f19e9400700 1 -- 172.21.15.18:0/2277126312 learned_addr learned my addr 172.21.15.18:0/2277126312 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:09:40.629 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:40.631+0000 7f19e9400700 1 -- 172.21.15.18:0/2277126312 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f19ec0708a0 msgr2=0x7f19ec119fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:09:40.629 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:40.631+0000 7f19e9400700 1 --2- 172.21.15.18:0/2277126312 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f19ec0708a0 0x7f19ec119fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:09:40.629 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:40.631+0000 7f19e9400700 1 -- 172.21.15.18:0/2277126312 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19dc005040 con 0x7f19ec071f90 2024-04-12T21:09:40.629 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:40.631+0000 7f19e9400700 1 --2- 172.21.15.18:0/2277126312 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f19ec071f90 0x7f19ec11a4e0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f19d8002a50 tx=0x7f19d80046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:09:40.629 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:40.632+0000 7f19e2000700 1 -- 172.21.15.18:0/2277126312 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f19d8007cc0 con 0x7f19ec071f90 2024-04-12T21:09:40.629 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:40.632+0000 7f19e2000700 1 -- 172.21.15.18:0/2277126312 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f19d800f040 con 0x7f19ec071f90 2024-04-12T21:09:40.629 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:40.632+0000 7f19e2000700 1 -- 172.21.15.18:0/2277126312 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f19d8009640 con 0x7f19ec071f90 2024-04-12T21:09:40.629 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:40.632+0000 7f19eb200700 1 -- 172.21.15.18:0/2277126312 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f19ec1141e0 con 0x7f19ec071f90 2024-04-12T21:09:40.629 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:40.632+0000 7f19eb200700 1 -- 172.21.15.18:0/2277126312 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f19ec114720 con 0x7f19ec071f90 2024-04-12T21:09:40.630 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:40.633+0000 7f19eb200700 1 -- 172.21.15.18:0/2277126312 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f19ec071aa0 con 0x7f19ec071f90 2024-04-12T21:09:40.630 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:40.634+0000 7f19e2000700 1 -- 172.21.15.18:0/2277126312 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7f19d80097a0 con 0x7f19ec071f90 2024-04-12T21:09:40.632 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:40.635+0000 7f19e2000700 1 --2- 172.21.15.18:0/2277126312 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f19d006ec00 0x7f19d00710b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:09:40.632 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:40.635+0000 7f19e2000700 1 -- 172.21.15.18:0/2277126312 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 6757+0+0 (secure 0 0 0) 0x7f19d808e760 con 0x7f19ec071f90 2024-04-12T21:09:40.632 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:40.635+0000 7f19e9e00700 1 --2- 172.21.15.18:0/2277126312 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f19d006ec00 0x7f19d00710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:09:40.632 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:40.636+0000 7f19e9e00700 1 --2- 172.21.15.18:0/2277126312 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f19d006ec00 0x7f19d00710b0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f19dc004540 tx=0x7f19dc009e30 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:09:40.636 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:40.640+0000 7f19e2000700 1 -- 172.21.15.18:0/2277126312 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f19d805af10 con 0x7f19ec071f90 2024-04-12T21:09:40.969 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:40.972+0000 7f19eb200700 1 -- 172.21.15.18:0/2277126312 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f19ec0fc3c0 con 0x7f19ec071f90 2024-04-12T21:09:40.970 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:40.973+0000 7f19e2000700 1 -- 172.21.15.18:0/2277126312 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+233 (secure 0 0 0) 0x7f19d805aaa0 con 0x7f19ec071f90 2024-04-12T21:09:40.970 INFO:teuthology.orchestra.run.smithi018.stdout:HEALTH_ERR 1 filesystem is degraded; 1 filesystem is offline 2024-04-12T21:09:40.970 INFO:teuthology.orchestra.run.smithi018.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-04-12T21:09:40.970 INFO:teuthology.orchestra.run.smithi018.stdout: fs cephfs is degraded 2024-04-12T21:09:40.970 INFO:teuthology.orchestra.run.smithi018.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-04-12T21:09:40.971 INFO:teuthology.orchestra.run.smithi018.stdout: fs cephfs is offline because no MDS is active for it. 2024-04-12T21:09:40.973 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:40.976+0000 7f19eb200700 1 -- 172.21.15.18:0/2277126312 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f19d006ec00 msgr2=0x7f19d00710b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:09:40.973 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:40.976+0000 7f19eb200700 1 --2- 172.21.15.18:0/2277126312 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f19d006ec00 0x7f19d00710b0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f19dc004540 tx=0x7f19dc009e30 comp rx=0 tx=0).stop 2024-04-12T21:09:40.973 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:40.976+0000 7f19eb200700 1 -- 172.21.15.18:0/2277126312 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f19ec071f90 msgr2=0x7f19ec11a4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:09:40.973 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:40.976+0000 7f19eb200700 1 --2- 172.21.15.18:0/2277126312 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f19ec071f90 0x7f19ec11a4e0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f19d8002a50 tx=0x7f19d80046b0 comp rx=0 tx=0).stop 2024-04-12T21:09:40.973 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:40.976+0000 7f19eb200700 1 -- 172.21.15.18:0/2277126312 shutdown_connections 2024-04-12T21:09:40.973 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:40.976+0000 7f19eb200700 1 --2- 172.21.15.18:0/2277126312 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f19d006ec00 0x7f19d00710b0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:09:40.974 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:40.976+0000 7f19eb200700 1 --2- 172.21.15.18:0/2277126312 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f19ec0708a0 0x7f19ec119fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:09:40.974 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:40.976+0000 7f19eb200700 1 --2- 172.21.15.18:0/2277126312 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f19ec071f90 0x7f19ec11a4e0 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:09:40.974 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:40.976+0000 7f19eb200700 1 -- 172.21.15.18:0/2277126312 >> 172.21.15.18:0/2277126312 conn(0x7f19ec0f8760 msgr2=0x7f19ec0f9810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:09:40.974 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:40.976+0000 7f19eb200700 1 -- 172.21.15.18:0/2277126312 shutdown_connections 2024-04-12T21:09:40.974 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:09:40.977+0000 7f19eb200700 1 -- 172.21.15.18:0/2277126312 wait complete. 2024-04-12T21:09:41.529 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:09:41 smithi186 ceph-mon[86379]: pgmap v296: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:09:41.529 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:09:41 smithi186 ceph-mon[86379]: from='client.34576 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:09:41.529 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:09:41 smithi186 ceph-mon[86379]: from='client.? 172.21.15.18:0/2277126312' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-12T21:09:41.600 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:09:41 smithi018 ceph-mon[112590]: pgmap v296: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:09:41.600 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:09:41 smithi018 ceph-mon[112590]: from='client.34576 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:09:41.600 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:09:41 smithi018 ceph-mon[112590]: from='client.? 172.21.15.18:0/2277126312' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-12T21:09:43.529 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:09:43 smithi186 ceph-mon[86379]: pgmap v297: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:09:43.600 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:09:43 smithi018 ceph-mon[112590]: pgmap v297: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:09:45.600 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:09:45 smithi018 ceph-mon[112590]: pgmap v298: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:09:45.781 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:09:45 smithi186 ceph-mon[86379]: pgmap v298: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:09:47.600 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:09:47 smithi018 ceph-mon[112590]: pgmap v299: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:09:47.779 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:09:47 smithi186 ceph-mon[86379]: pgmap v299: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:09:49.600 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:09:49 smithi018 ceph-mon[112590]: pgmap v300: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:09:49.779 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:09:49 smithi186 ceph-mon[86379]: pgmap v300: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:09:51.600 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:09:51 smithi018 ceph-mon[112590]: pgmap v301: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:09:51.779 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:09:51 smithi186 ceph-mon[86379]: pgmap v301: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:09:53.600 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:09:53 smithi018 ceph-mon[112590]: pgmap v302: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:09:53.779 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:09:53 smithi186 ceph-mon[86379]: pgmap v302: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:09:55.600 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:09:55 smithi018 ceph-mon[112590]: pgmap v303: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:09:55.600 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:09:55 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T21:09:55.779 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:09:55 smithi186 ceph-mon[86379]: pgmap v303: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:09:55.779 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:09:55 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T21:09:57.600 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:09:57 smithi018 ceph-mon[112590]: pgmap v304: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:09:57.779 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:09:57 smithi186 ceph-mon[86379]: pgmap v304: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:09:59.600 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:09:59 smithi018 ceph-mon[112590]: pgmap v305: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:09:59.779 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:09:59 smithi186 ceph-mon[86379]: pgmap v305: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:10:00.339 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:10:00 smithi018 ceph-1cbe407a-f90d-11ee-bc8d-c7b262605968-mon-smithi018[112586]: 2024-04-12T21:09:59.998+0000 7fd45a600700 -1 log_channel(cluster) log [ERR] : Health detail: HEALTH_ERR 1 filesystem is degraded; 1 filesystem is offline 2024-04-12T21:10:00.339 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:10:00 smithi018 ceph-1cbe407a-f90d-11ee-bc8d-c7b262605968-mon-smithi018[112586]: 2024-04-12T21:09:59.998+0000 7fd45a600700 -1 log_channel(cluster) log [ERR] : [WRN] FS_DEGRADED: 1 filesystem is degraded 2024-04-12T21:10:00.339 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:10:00 smithi018 ceph-1cbe407a-f90d-11ee-bc8d-c7b262605968-mon-smithi018[112586]: 2024-04-12T21:09:59.998+0000 7fd45a600700 -1 log_channel(cluster) log [ERR] : fs cephfs is degraded 2024-04-12T21:10:00.339 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:10:00 smithi018 ceph-1cbe407a-f90d-11ee-bc8d-c7b262605968-mon-smithi018[112586]: 2024-04-12T21:09:59.998+0000 7fd45a600700 -1 log_channel(cluster) log [ERR] : [ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-04-12T21:10:00.339 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:10:00 smithi018 ceph-1cbe407a-f90d-11ee-bc8d-c7b262605968-mon-smithi018[112586]: 2024-04-12T21:09:59.998+0000 7fd45a600700 -1 log_channel(cluster) log [ERR] : fs cephfs is offline because no MDS is active for it. 2024-04-12T21:10:00.600 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:10:00 smithi018 ceph-mon[112590]: Health detail: HEALTH_ERR 1 filesystem is degraded; 1 filesystem is offline 2024-04-12T21:10:00.600 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:10:00 smithi018 ceph-mon[112590]: [WRN] FS_DEGRADED: 1 filesystem is degraded 2024-04-12T21:10:00.600 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:10:00 smithi018 ceph-mon[112590]: fs cephfs is degraded 2024-04-12T21:10:00.600 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:10:00 smithi018 ceph-mon[112590]: [ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-04-12T21:10:00.600 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:10:00 smithi018 ceph-mon[112590]: fs cephfs is offline because no MDS is active for it. 2024-04-12T21:10:00.779 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:10:00 smithi186 ceph-mon[86379]: Health detail: HEALTH_ERR 1 filesystem is degraded; 1 filesystem is offline 2024-04-12T21:10:00.779 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:10:00 smithi186 ceph-mon[86379]: [WRN] FS_DEGRADED: 1 filesystem is degraded 2024-04-12T21:10:00.779 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:10:00 smithi186 ceph-mon[86379]: fs cephfs is degraded 2024-04-12T21:10:00.779 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:10:00 smithi186 ceph-mon[86379]: [ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-04-12T21:10:00.779 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:10:00 smithi186 ceph-mon[86379]: fs cephfs is offline because no MDS is active for it. 2024-04-12T21:10:01.600 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:10:01 smithi018 ceph-mon[112590]: pgmap v306: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:10:01.779 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:10:01 smithi186 ceph-mon[86379]: pgmap v306: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:10:03.600 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:10:03 smithi018 ceph-mon[112590]: pgmap v307: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:10:03.779 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:10:03 smithi186 ceph-mon[86379]: pgmap v307: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:10:05.779 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:10:05 smithi186 ceph-mon[86379]: pgmap v308: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:10:05.850 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:10:05 smithi018 ceph-mon[112590]: pgmap v308: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:10:07.779 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:10:07 smithi186 ceph-mon[86379]: pgmap v309: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:10:07.850 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:10:07 smithi018 ceph-mon[112590]: pgmap v309: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:10:09.779 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:10:09 smithi186 ceph-mon[86379]: pgmap v310: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:10:09.850 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:10:09 smithi018 ceph-mon[112590]: pgmap v310: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:10:10.752 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:10:10 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T21:10:10.779 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:10:10 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T21:10:11.145 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:11.148+0000 7f543cc00700 1 -- 172.21.15.18:0/2606960662 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5438105810 msgr2=0x7f54380feed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:10:11.145 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:11.148+0000 7f543cc00700 1 --2- 172.21.15.18:0/2606960662 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5438105810 0x7f54380feed0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f5424004940 tx=0x7f5424009e30 comp rx=0 tx=0).stop 2024-04-12T21:10:11.146 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:11.149+0000 7f543cc00700 1 -- 172.21.15.18:0/2606960662 shutdown_connections 2024-04-12T21:10:11.146 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:11.149+0000 7f543cc00700 1 --2- 172.21.15.18:0/2606960662 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5438105810 0x7f54380feed0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:10:11.146 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:11.149+0000 7f543cc00700 1 --2- 172.21.15.18:0/2606960662 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5438104e70 0x7f5438105240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:10:11.146 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:11.149+0000 7f543cc00700 1 -- 172.21.15.18:0/2606960662 >> 172.21.15.18:0/2606960662 conn(0x7f54380fab10 msgr2=0x7f54380fcef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:10:11.146 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:11.149+0000 7f543cc00700 1 -- 172.21.15.18:0/2606960662 shutdown_connections 2024-04-12T21:10:11.146 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:11.149+0000 7f543cc00700 1 -- 172.21.15.18:0/2606960662 wait complete. 2024-04-12T21:10:11.147 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:11.150+0000 7f543cc00700 1 Processor -- start 2024-04-12T21:10:11.147 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:11.150+0000 7f543cc00700 1 -- start start 2024-04-12T21:10:11.147 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:11.150+0000 7f543cc00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5438104e70 0x7f5438115b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:10:11.147 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:11.150+0000 7f543cc00700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5438105810 0x7f54381160c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:10:11.147 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:11.150+0000 7f543cc00700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5438116710 con 0x7f5438104e70 2024-04-12T21:10:11.147 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:11.150+0000 7f543cc00700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f543810fb00 con 0x7f5438105810 2024-04-12T21:10:11.148 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:11.151+0000 7f5437400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5438104e70 0x7f5438115b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:10:11.148 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:11.151+0000 7f5437400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5438104e70 0x7f5438115b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:43242/0 (socket says 172.21.15.18:43242) 2024-04-12T21:10:11.148 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:11.151+0000 7f5437400700 1 -- 172.21.15.18:0/1917752829 learned_addr learned my addr 172.21.15.18:0/1917752829 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:10:11.148 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:11.151+0000 7f5436a00700 1 --2- 172.21.15.18:0/1917752829 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5438105810 0x7f54381160c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:10:11.148 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:11.151+0000 7f5437400700 1 -- 172.21.15.18:0/1917752829 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5438105810 msgr2=0x7f54381160c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:10:11.148 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:11.151+0000 7f5437400700 1 --2- 172.21.15.18:0/1917752829 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5438105810 0x7f54381160c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:10:11.148 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:11.151+0000 7f5437400700 1 -- 172.21.15.18:0/1917752829 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5424005040 con 0x7f5438104e70 2024-04-12T21:10:11.148 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:11.152+0000 7f5437400700 1 --2- 172.21.15.18:0/1917752829 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5438104e70 0x7f5438115b80 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f542800c830 tx=0x7f5428004470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:10:11.149 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:11.152+0000 7f542f400700 1 -- 172.21.15.18:0/1917752829 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f542800a760 con 0x7f5438104e70 2024-04-12T21:10:11.149 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:11.152+0000 7f542f400700 1 -- 172.21.15.18:0/1917752829 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f542800ad90 con 0x7f5438104e70 2024-04-12T21:10:11.149 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:11.152+0000 7f543cc00700 1 -- 172.21.15.18:0/1917752829 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f543810fde0 con 0x7f5438104e70 2024-04-12T21:10:11.149 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:11.152+0000 7f542f400700 1 -- 172.21.15.18:0/1917752829 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f54280113a0 con 0x7f5438104e70 2024-04-12T21:10:11.149 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:11.152+0000 7f543cc00700 1 -- 172.21.15.18:0/1917752829 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f54381103a0 con 0x7f5438104e70 2024-04-12T21:10:11.150 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:11.154+0000 7f543cc00700 1 -- 172.21.15.18:0/1917752829 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5438107340 con 0x7f5438104e70 2024-04-12T21:10:11.151 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:11.155+0000 7f542f400700 1 -- 172.21.15.18:0/1917752829 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7f5428012400 con 0x7f5438104e70 2024-04-12T21:10:11.152 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:11.156+0000 7f542f400700 1 --2- 172.21.15.18:0/1917752829 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f541c06ec00 0x7f541c0710b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:10:11.152 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:11.156+0000 7f542f400700 1 -- 172.21.15.18:0/1917752829 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 6757+0+0 (secure 0 0 0) 0x7f542808ea00 con 0x7f5438104e70 2024-04-12T21:10:11.153 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:11.156+0000 7f5436a00700 1 --2- 172.21.15.18:0/1917752829 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f541c06ec00 0x7f541c0710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:10:11.153 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:11.157+0000 7f5436a00700 1 --2- 172.21.15.18:0/1917752829 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f541c06ec00 0x7f541c0710b0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f5438110cc0 tx=0x7f5424009e30 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:10:11.158 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:11.161+0000 7f542f400700 1 -- 172.21.15.18:0/1917752829 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f542805b250 con 0x7f5438104e70 2024-04-12T21:10:11.411 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:11.414+0000 7f543cc00700 1 -- 172.21.15.18:0/1917752829 --> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f543810bf50 con 0x7f541c06ec00 2024-04-12T21:10:11.411 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:10:11 smithi018 ceph-mon[112590]: pgmap v311: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:10:11.412 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:11.416+0000 7f542f400700 1 -- 172.21.15.18:0/1917752829 <== mgr.34102 v2:172.21.15.18:6800/3812891919 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f543810bf50 con 0x7f541c06ec00 2024-04-12T21:10:11.416 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:11.419+0000 7f543cc00700 1 -- 172.21.15.18:0/1917752829 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f541c06ec00 msgr2=0x7f541c0710b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:10:11.416 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:11.419+0000 7f543cc00700 1 --2- 172.21.15.18:0/1917752829 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f541c06ec00 0x7f541c0710b0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f5438110cc0 tx=0x7f5424009e30 comp rx=0 tx=0).stop 2024-04-12T21:10:11.417 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:11.419+0000 7f543cc00700 1 -- 172.21.15.18:0/1917752829 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5438104e70 msgr2=0x7f5438115b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:10:11.417 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:11.419+0000 7f543cc00700 1 --2- 172.21.15.18:0/1917752829 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5438104e70 0x7f5438115b80 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f542800c830 tx=0x7f5428004470 comp rx=0 tx=0).stop 2024-04-12T21:10:11.417 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:11.420+0000 7f543cc00700 1 -- 172.21.15.18:0/1917752829 shutdown_connections 2024-04-12T21:10:11.417 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:11.420+0000 7f543cc00700 1 --2- 172.21.15.18:0/1917752829 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f541c06ec00 0x7f541c0710b0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:10:11.417 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:11.420+0000 7f543cc00700 1 --2- 172.21.15.18:0/1917752829 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5438104e70 0x7f5438115b80 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:10:11.417 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:11.420+0000 7f543cc00700 1 --2- 172.21.15.18:0/1917752829 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5438105810 0x7f54381160c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:10:11.417 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:11.420+0000 7f543cc00700 1 -- 172.21.15.18:0/1917752829 >> 172.21.15.18:0/1917752829 conn(0x7f54380fab10 msgr2=0x7f54380fd8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:10:11.417 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:11.420+0000 7f543cc00700 1 -- 172.21.15.18:0/1917752829 shutdown_connections 2024-04-12T21:10:11.417 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:11.420+0000 7f543cc00700 1 -- 172.21.15.18:0/1917752829 wait complete. 2024-04-12T21:10:11.434 INFO:teuthology.orchestra.run.smithi018.stdout:true 2024-04-12T21:10:11.576 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:11.579+0000 7fadbf200700 1 -- 172.21.15.18:0/2285766178 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fadb80fc620 msgr2=0x7fadb8100cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:10:11.576 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:11.579+0000 7fadbf200700 1 --2- 172.21.15.18:0/2285766178 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fadb80fc620 0x7fadb8100cd0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fada4004940 tx=0x7fada4009e30 comp rx=0 tx=0).stop 2024-04-12T21:10:11.576 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:11.579+0000 7fadbf200700 1 -- 172.21.15.18:0/2285766178 shutdown_connections 2024-04-12T21:10:11.576 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:11.579+0000 7fadbf200700 1 --2- 172.21.15.18:0/2285766178 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fadb80fc620 0x7fadb8100cd0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:10:11.576 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:11.579+0000 7fadbf200700 1 --2- 172.21.15.18:0/2285766178 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fadb80fbc80 0x7fadb80fc050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:10:11.577 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:11.579+0000 7fadbf200700 1 -- 172.21.15.18:0/2285766178 >> 172.21.15.18:0/2285766178 conn(0x7fadb8072bc0 msgr2=0x7fadb8072fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:10:11.577 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:11.579+0000 7fadbf200700 1 -- 172.21.15.18:0/2285766178 shutdown_connections 2024-04-12T21:10:11.577 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:11.579+0000 7fadbf200700 1 -- 172.21.15.18:0/2285766178 wait complete. 2024-04-12T21:10:11.578 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:11.580+0000 7fadbf200700 1 Processor -- start 2024-04-12T21:10:11.578 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:11.580+0000 7fadbf200700 1 -- start start 2024-04-12T21:10:11.578 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:11.581+0000 7fadbf200700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fadb80fbc80 0x7fadb811a1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:10:11.578 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:11.581+0000 7fadbf200700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fadb80fc620 0x7fadb811a6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:10:11.578 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:11.581+0000 7fadbf200700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fadb811add0 con 0x7fadb80fbc80 2024-04-12T21:10:11.578 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:11.581+0000 7fadbf200700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fadb8114110 con 0x7fadb80fc620 2024-04-12T21:10:11.578 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:11.581+0000 7fadb7e00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fadb80fbc80 0x7fadb811a1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:10:11.579 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:11.581+0000 7fadb7e00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fadb80fbc80 0x7fadb811a1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:43254/0 (socket says 172.21.15.18:43254) 2024-04-12T21:10:11.579 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:11.581+0000 7fadb7e00700 1 -- 172.21.15.18:0/631063820 learned_addr learned my addr 172.21.15.18:0/631063820 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:10:11.579 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:11.582+0000 7fadb7400700 1 --2- 172.21.15.18:0/631063820 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fadb80fc620 0x7fadb811a6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:10:11.579 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:11.582+0000 7fadb7e00700 1 -- 172.21.15.18:0/631063820 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fadb80fc620 msgr2=0x7fadb811a6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:10:11.579 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:11.582+0000 7fadb7e00700 1 --2- 172.21.15.18:0/631063820 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fadb80fc620 0x7fadb811a6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:10:11.579 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:11.582+0000 7fadb7e00700 1 -- 172.21.15.18:0/631063820 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fada4005040 con 0x7fadb80fbc80 2024-04-12T21:10:11.580 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:11.582+0000 7fadb7e00700 1 --2- 172.21.15.18:0/631063820 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fadb80fbc80 0x7fadb811a1b0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fada800b790 tx=0x7fada8009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:10:11.580 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:11.582+0000 7fadafe00700 1 -- 172.21.15.18:0/631063820 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fada800f770 con 0x7fadb80fbc80 2024-04-12T21:10:11.580 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:11.582+0000 7fadafe00700 1 -- 172.21.15.18:0/631063820 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fada800fda0 con 0x7fadb80fbc80 2024-04-12T21:10:11.580 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:11.582+0000 7fadafe00700 1 -- 172.21.15.18:0/631063820 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fada800d310 con 0x7fadb80fbc80 2024-04-12T21:10:11.580 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:11.582+0000 7fadbf200700 1 -- 172.21.15.18:0/631063820 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fadb8114390 con 0x7fadb80fbc80 2024-04-12T21:10:11.580 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:11.583+0000 7fadbf200700 1 -- 172.21.15.18:0/631063820 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fadb81148d0 con 0x7fadb80fbc80 2024-04-12T21:10:11.580 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:11.584+0000 7fadbf200700 1 -- 172.21.15.18:0/631063820 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fadb8065e30 con 0x7fadb80fbc80 2024-04-12T21:10:11.581 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:11.585+0000 7fadafe00700 1 -- 172.21.15.18:0/631063820 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7fada8010420 con 0x7fadb80fbc80 2024-04-12T21:10:11.582 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:11.586+0000 7fadafe00700 1 --2- 172.21.15.18:0/631063820 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fad9c06ec10 0x7fad9c0710c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:10:11.583 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:11.586+0000 7fadafe00700 1 -- 172.21.15.18:0/631063820 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 6757+0+0 (secure 0 0 0) 0x7fada808ea80 con 0x7fadb80fbc80 2024-04-12T21:10:11.583 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:11.586+0000 7fadb7400700 1 --2- 172.21.15.18:0/631063820 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fad9c06ec10 0x7fad9c0710c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:10:11.584 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:11.587+0000 7fadb7400700 1 --2- 172.21.15.18:0/631063820 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fad9c06ec10 0x7fad9c0710c0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fada400b020 tx=0x7fada4009e30 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:10:11.588 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:11.591+0000 7fadafe00700 1 -- 172.21.15.18:0/631063820 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7fada805b230 con 0x7fadb80fbc80 2024-04-12T21:10:11.779 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:10:11 smithi186 ceph-mon[86379]: pgmap v311: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:10:11.834 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:11.837+0000 7fadbf200700 1 -- 172.21.15.18:0/631063820 --> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fadb8115800 con 0x7fad9c06ec10 2024-04-12T21:10:11.838 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:11.841+0000 7fadafe00700 1 -- 172.21.15.18:0/631063820 <== mgr.34102 v2:172.21.15.18:6800/3812891919 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7fadb8115800 con 0x7fad9c06ec10 2024-04-12T21:10:11.841 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:11.844+0000 7fadbf200700 1 -- 172.21.15.18:0/631063820 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fad9c06ec10 msgr2=0x7fad9c0710c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:10:11.841 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:11.844+0000 7fadbf200700 1 --2- 172.21.15.18:0/631063820 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fad9c06ec10 0x7fad9c0710c0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fada400b020 tx=0x7fada4009e30 comp rx=0 tx=0).stop 2024-04-12T21:10:11.841 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:11.844+0000 7fadbf200700 1 -- 172.21.15.18:0/631063820 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fadb80fbc80 msgr2=0x7fadb811a1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:10:11.841 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:11.844+0000 7fadbf200700 1 --2- 172.21.15.18:0/631063820 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fadb80fbc80 0x7fadb811a1b0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fada800b790 tx=0x7fada8009250 comp rx=0 tx=0).stop 2024-04-12T21:10:11.841 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:11.844+0000 7fadbf200700 1 -- 172.21.15.18:0/631063820 shutdown_connections 2024-04-12T21:10:11.841 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:11.844+0000 7fadbf200700 1 --2- 172.21.15.18:0/631063820 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fad9c06ec10 0x7fad9c0710c0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:10:11.841 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:11.844+0000 7fadbf200700 1 --2- 172.21.15.18:0/631063820 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fadb80fbc80 0x7fadb811a1b0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:10:11.841 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:11.844+0000 7fadbf200700 1 --2- 172.21.15.18:0/631063820 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fadb80fc620 0x7fadb811a6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:10:11.841 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:11.844+0000 7fadbf200700 1 -- 172.21.15.18:0/631063820 >> 172.21.15.18:0/631063820 conn(0x7fadb8072bc0 msgr2=0x7fadb810aab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:10:11.842 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:11.845+0000 7fadbf200700 1 -- 172.21.15.18:0/631063820 shutdown_connections 2024-04-12T21:10:11.842 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:11.845+0000 7fadbf200700 1 -- 172.21.15.18:0/631063820 wait complete. 2024-04-12T21:10:11.999 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:12.002+0000 7fd809800700 1 -- 172.21.15.18:0/2644282700 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd8040724f0 msgr2=0x7fd8040728c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:10:11.999 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:12.002+0000 7fd809800700 1 --2- 172.21.15.18:0/2644282700 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd8040724f0 0x7fd8040728c0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fd7f4009ce0 tx=0x7fd7f4004d10 comp rx=0 tx=0).stop 2024-04-12T21:10:12.000 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:12.002+0000 7fd809800700 1 -- 172.21.15.18:0/2644282700 shutdown_connections 2024-04-12T21:10:12.000 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:12.002+0000 7fd809800700 1 --2- 172.21.15.18:0/2644282700 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd804072e00 0x7fd80410d770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:10:12.000 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:12.002+0000 7fd809800700 1 --2- 172.21.15.18:0/2644282700 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd8040724f0 0x7fd8040728c0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:10:12.000 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:12.002+0000 7fd809800700 1 -- 172.21.15.18:0/2644282700 >> 172.21.15.18:0/2644282700 conn(0x7fd8040fabd0 msgr2=0x7fd8040fcfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:10:12.000 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:12.002+0000 7fd809800700 1 -- 172.21.15.18:0/2644282700 shutdown_connections 2024-04-12T21:10:12.000 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:12.002+0000 7fd809800700 1 -- 172.21.15.18:0/2644282700 wait complete. 2024-04-12T21:10:12.001 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:12.003+0000 7fd809800700 1 Processor -- start 2024-04-12T21:10:12.001 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:12.004+0000 7fd809800700 1 -- start start 2024-04-12T21:10:12.001 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:12.004+0000 7fd809800700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd8040724f0 0x7fd80411a200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:10:12.001 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:12.004+0000 7fd809800700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd804072e00 0x7fd80411a740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:10:12.001 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:12.004+0000 7fd809800700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd80411ad90 con 0x7fd804072e00 2024-04-12T21:10:12.002 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:12.004+0000 7fd809800700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd804114160 con 0x7fd8040724f0 2024-04-12T21:10:12.002 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:12.005+0000 7fd7fa000700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd804072e00 0x7fd80411a740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:10:12.002 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:12.005+0000 7fd7fa000700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd804072e00 0x7fd80411a740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:43268/0 (socket says 172.21.15.18:43268) 2024-04-12T21:10:12.002 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:12.005+0000 7fd7fa000700 1 -- 172.21.15.18:0/4288614504 learned_addr learned my addr 172.21.15.18:0/4288614504 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:10:12.002 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:12.005+0000 7fd802a00700 1 --2- 172.21.15.18:0/4288614504 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd8040724f0 0x7fd80411a200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:10:12.003 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:12.005+0000 7fd7fa000700 1 -- 172.21.15.18:0/4288614504 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd8040724f0 msgr2=0x7fd80411a200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:10:12.003 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:12.005+0000 7fd7fa000700 1 --2- 172.21.15.18:0/4288614504 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd8040724f0 0x7fd80411a200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:10:12.003 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:12.005+0000 7fd7fa000700 1 -- 172.21.15.18:0/4288614504 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7f4009940 con 0x7fd804072e00 2024-04-12T21:10:12.003 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:12.006+0000 7fd7fa000700 1 --2- 172.21.15.18:0/4288614504 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd804072e00 0x7fd80411a740 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fd7ec002a50 tx=0x7fd7ec0046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:10:12.003 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:12.006+0000 7fd7fb400700 1 -- 172.21.15.18:0/4288614504 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd7ec007cc0 con 0x7fd804072e00 2024-04-12T21:10:12.003 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:12.006+0000 7fd7fb400700 1 -- 172.21.15.18:0/4288614504 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd7ec00f040 con 0x7fd804072e00 2024-04-12T21:10:12.004 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:12.006+0000 7fd809800700 1 -- 172.21.15.18:0/4288614504 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd804114440 con 0x7fd804072e00 2024-04-12T21:10:12.004 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:12.006+0000 7fd7fb400700 1 -- 172.21.15.18:0/4288614504 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd7ec009640 con 0x7fd804072e00 2024-04-12T21:10:12.004 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:12.006+0000 7fd809800700 1 -- 172.21.15.18:0/4288614504 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd804114980 con 0x7fd804072e00 2024-04-12T21:10:12.004 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:12.008+0000 7fd809800700 1 -- 172.21.15.18:0/4288614504 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd804073c30 con 0x7fd804072e00 2024-04-12T21:10:12.005 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:12.009+0000 7fd7fb400700 1 -- 172.21.15.18:0/4288614504 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7fd7ec0097a0 con 0x7fd804072e00 2024-04-12T21:10:12.006 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:12.010+0000 7fd7fb400700 1 --2- 172.21.15.18:0/4288614504 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fd7e806ec10 0x7fd7e80710c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:10:12.007 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:12.010+0000 7fd7fb400700 1 -- 172.21.15.18:0/4288614504 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 6757+0+0 (secure 0 0 0) 0x7fd7ec08e760 con 0x7fd804072e00 2024-04-12T21:10:12.007 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:12.011+0000 7fd802a00700 1 --2- 172.21.15.18:0/4288614504 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fd7e806ec10 0x7fd7e80710c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:10:12.008 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:12.011+0000 7fd802a00700 1 --2- 172.21.15.18:0/4288614504 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fd7e806ec10 0x7fd7e80710c0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fd7f4005010 tx=0x7fd7f4004d10 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:10:12.012 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:12.016+0000 7fd7fb400700 1 -- 172.21.15.18:0/4288614504 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7fd7ec05af10 con 0x7fd804072e00 2024-04-12T21:10:12.253 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:12.256+0000 7fd809800700 1 -- 172.21.15.18:0/4288614504 --> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fd8041104e0 con 0x7fd7e806ec10 2024-04-12T21:10:12.283 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:12.287+0000 7fd7fb400700 1 -- 172.21.15.18:0/4288614504 <== mgr.34102 v2:172.21.15.18:6800/3812891919 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7fd8041104e0 con 0x7fd7e806ec10 2024-04-12T21:10:12.284 INFO:teuthology.orchestra.run.smithi018.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-12T21:10:12.284 INFO:teuthology.orchestra.run.smithi018.stdout:alertmanager.smithi018 smithi018 *:9093,9094 running (10m) 2m ago 25m 14.7M - 0.25.0 c8568f914cd2 c332090068e8 2024-04-12T21:10:12.284 INFO:teuthology.orchestra.run.smithi018.stdout:crash.smithi018 smithi018 running (7m) 2m ago 25m 6665k - 19.0.0-2879-g324baf8d 8717720dc5e5 b7f80d1f7982 2024-04-12T21:10:12.284 INFO:teuthology.orchestra.run.smithi018.stdout:crash.smithi186 smithi186 running (7m) 3m ago 24m 6660k - 19.0.0-2879-g324baf8d 8717720dc5e5 a4f55c81b623 2024-04-12T21:10:12.284 INFO:teuthology.orchestra.run.smithi018.stdout:grafana.smithi018 smithi018 *:3000 running (10m) 2m ago 24m 74.8M - 9.4.12 2bacad6d85d8 b08e3bf176dd 2024-04-12T21:10:12.284 INFO:teuthology.orchestra.run.smithi018.stdout:mds.cephfs.smithi018.acqbsc smithi018 running (3m) 2m ago 18m 11.1M - 19.0.0-2879-g324baf8d 8717720dc5e5 021f99ab24ae 2024-04-12T21:10:12.284 INFO:teuthology.orchestra.run.smithi018.stdout:mds.cephfs.smithi018.szjluo smithi018 running (2m) 2m ago 18m 10.9M - 19.0.0-2879-g324baf8d 8717720dc5e5 0003117479d4 2024-04-12T21:10:12.284 INFO:teuthology.orchestra.run.smithi018.stdout:mds.cephfs.smithi186.jdllyn smithi186 running (18m) 3m ago 18m 15.6M - 17.2.7-827-gd3b5060a 21de9b7369ad 2b2bc46bd29a 2024-04-12T21:10:12.284 INFO:teuthology.orchestra.run.smithi018.stdout:mds.cephfs.smithi186.wjnafs smithi186 running (17m) 3m ago 17m 15.4M - 17.2.7-827-gd3b5060a 21de9b7369ad 74b115d6d6fd 2024-04-12T21:10:12.284 INFO:teuthology.orchestra.run.smithi018.stdout:mgr.smithi018.xjzwga smithi018 *:8443,9283,8765 running (13m) 2m ago 27m 509M - 19.0.0-2879-g324baf8d 8717720dc5e5 56088305eab7 2024-04-12T21:10:12.285 INFO:teuthology.orchestra.run.smithi018.stdout:mgr.smithi186.kqcfxk smithi186 *:8443,9283,8765 running (12m) 3m ago 24m 452M - 19.0.0-2879-g324baf8d 8717720dc5e5 09cd766e5e0d 2024-04-12T21:10:12.285 INFO:teuthology.orchestra.run.smithi018.stdout:mon.smithi018 smithi018 running (8m) 2m ago 27m 57.0M 2048M 19.0.0-2879-g324baf8d 8717720dc5e5 df3e201dfed6 2024-04-12T21:10:12.285 INFO:teuthology.orchestra.run.smithi018.stdout:mon.smithi186 smithi186 running (8m) 3m ago 23m 37.7M 2048M 19.0.0-2879-g324baf8d 8717720dc5e5 c77498da9b2b 2024-04-12T21:10:12.285 INFO:teuthology.orchestra.run.smithi018.stdout:node-exporter.smithi018 smithi018 *:9100 running (11m) 2m ago 25m 7780k - 1.5.0 0da6a335fe13 326c3d9c6847 2024-04-12T21:10:12.285 INFO:teuthology.orchestra.run.smithi018.stdout:node-exporter.smithi186 smithi186 *:9100 running (11m) 3m ago 24m 7826k - 1.5.0 0da6a335fe13 ec2112baeecf 2024-04-12T21:10:12.285 INFO:teuthology.orchestra.run.smithi018.stdout:osd.0 smithi018 running (6m) 2m ago 22m 121M 1559M 19.0.0-2879-g324baf8d 8717720dc5e5 60b1273ca047 2024-04-12T21:10:12.285 INFO:teuthology.orchestra.run.smithi018.stdout:osd.1 smithi018 running (5m) 2m ago 22m 91.3M 1559M 19.0.0-2879-g324baf8d 8717720dc5e5 170ce1e5fb8d 2024-04-12T21:10:12.285 INFO:teuthology.orchestra.run.smithi018.stdout:osd.2 smithi018 running (5m) 2m ago 21m 90.3M 1559M 19.0.0-2879-g324baf8d 8717720dc5e5 c2cbc6aa2839 2024-04-12T21:10:12.285 INFO:teuthology.orchestra.run.smithi018.stdout:osd.3 smithi186 running (4m) 3m ago 21m 96.6M 2581M 19.0.0-2879-g324baf8d 8717720dc5e5 0138e8262e4b 2024-04-12T21:10:12.285 INFO:teuthology.orchestra.run.smithi018.stdout:osd.4 smithi186 running (4m) 3m ago 20m 87.2M 2581M 19.0.0-2879-g324baf8d 8717720dc5e5 b20cb63b25c9 2024-04-12T21:10:12.285 INFO:teuthology.orchestra.run.smithi018.stdout:osd.5 smithi186 running (3m) 3m ago 20m 11.1M 2581M 19.0.0-2879-g324baf8d 8717720dc5e5 2513bea2610e 2024-04-12T21:10:12.286 INFO:teuthology.orchestra.run.smithi018.stdout:prometheus.smithi018 smithi018 *:9095 running (11m) 2m ago 24m 60.6M - 2.43.0 a07b618ecd1d f58841127e21 2024-04-12T21:10:12.286 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:12.290+0000 7fd809800700 1 -- 172.21.15.18:0/4288614504 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fd7e806ec10 msgr2=0x7fd7e80710c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:10:12.286 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:12.290+0000 7fd809800700 1 --2- 172.21.15.18:0/4288614504 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fd7e806ec10 0x7fd7e80710c0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fd7f4005010 tx=0x7fd7f4004d10 comp rx=0 tx=0).stop 2024-04-12T21:10:12.286 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:12.290+0000 7fd809800700 1 -- 172.21.15.18:0/4288614504 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd804072e00 msgr2=0x7fd80411a740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:10:12.286 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:12.290+0000 7fd809800700 1 --2- 172.21.15.18:0/4288614504 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd804072e00 0x7fd80411a740 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fd7ec002a50 tx=0x7fd7ec0046b0 comp rx=0 tx=0).stop 2024-04-12T21:10:12.287 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:12.290+0000 7fd809800700 1 -- 172.21.15.18:0/4288614504 shutdown_connections 2024-04-12T21:10:12.287 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:12.290+0000 7fd809800700 1 --2- 172.21.15.18:0/4288614504 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fd7e806ec10 0x7fd7e80710c0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:10:12.288 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:12.290+0000 7fd809800700 1 --2- 172.21.15.18:0/4288614504 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd8040724f0 0x7fd80411a200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:10:12.288 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:12.290+0000 7fd809800700 1 --2- 172.21.15.18:0/4288614504 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd804072e00 0x7fd80411a740 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:10:12.288 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:12.290+0000 7fd809800700 1 -- 172.21.15.18:0/4288614504 >> 172.21.15.18:0/4288614504 conn(0x7fd8040fabd0 msgr2=0x7fd8040fc0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:10:12.288 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:12.290+0000 7fd809800700 1 -- 172.21.15.18:0/4288614504 shutdown_connections 2024-04-12T21:10:12.288 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:12.290+0000 7fd809800700 1 -- 172.21.15.18:0/4288614504 wait complete. 2024-04-12T21:10:12.431 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:12.434+0000 7fc0fe400700 1 -- 172.21.15.18:0/1801608163 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc0f8065b70 msgr2=0x7fc0f8065fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:10:12.431 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:12.434+0000 7fc0fe400700 1 --2- 172.21.15.18:0/1801608163 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc0f8065b70 0x7fc0f8065fe0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fc0e4004940 tx=0x7fc0e4009e30 comp rx=0 tx=0).stop 2024-04-12T21:10:12.431 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:12.434+0000 7fc0fe400700 1 -- 172.21.15.18:0/1801608163 shutdown_connections 2024-04-12T21:10:12.431 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:12.434+0000 7fc0fe400700 1 --2- 172.21.15.18:0/1801608163 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc0f8065b70 0x7fc0f8065fe0 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:10:12.431 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:12.434+0000 7fc0fe400700 1 --2- 172.21.15.18:0/1801608163 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc0f8101ae0 0x7fc0f8065630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:10:12.431 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:12.434+0000 7fc0fe400700 1 -- 172.21.15.18:0/1801608163 >> 172.21.15.18:0/1801608163 conn(0x7fc0f8071b70 msgr2=0x7fc0f8071f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:10:12.432 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:12.434+0000 7fc0fe400700 1 -- 172.21.15.18:0/1801608163 shutdown_connections 2024-04-12T21:10:12.432 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:12.434+0000 7fc0fe400700 1 -- 172.21.15.18:0/1801608163 wait complete. 2024-04-12T21:10:12.433 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:12.435+0000 7fc0fe400700 1 Processor -- start 2024-04-12T21:10:12.433 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:12.435+0000 7fc0fe400700 1 -- start start 2024-04-12T21:10:12.433 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:12.436+0000 7fc0fe400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc0f8065b70 0x7fc0f8119ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:10:12.433 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:12.436+0000 7fc0fe400700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc0f8101ae0 0x7fc0f811a530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:10:12.433 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:12.436+0000 7fc0fe400700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0f811ab80 con 0x7fc0f8065b70 2024-04-12T21:10:12.433 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:12.436+0000 7fc0fe400700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0f8113f50 con 0x7fc0f8101ae0 2024-04-12T21:10:12.434 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:12.436+0000 7fc0fd000700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc0f8065b70 0x7fc0f8119ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:10:12.434 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:12.436+0000 7fc0fd000700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc0f8065b70 0x7fc0f8119ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:43286/0 (socket says 172.21.15.18:43286) 2024-04-12T21:10:12.434 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:12.436+0000 7fc0fd000700 1 -- 172.21.15.18:0/2589867384 learned_addr learned my addr 172.21.15.18:0/2589867384 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:10:12.434 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:12.436+0000 7fc0f7e00700 1 --2- 172.21.15.18:0/2589867384 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc0f8101ae0 0x7fc0f811a530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:10:12.434 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:12.436+0000 7fc0fd000700 1 -- 172.21.15.18:0/2589867384 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc0f8101ae0 msgr2=0x7fc0f811a530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:10:12.434 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:12.436+0000 7fc0fd000700 1 --2- 172.21.15.18:0/2589867384 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc0f8101ae0 0x7fc0f811a530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:10:12.434 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:12.436+0000 7fc0fd000700 1 -- 172.21.15.18:0/2589867384 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0e4005040 con 0x7fc0f8065b70 2024-04-12T21:10:12.434 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:12.437+0000 7fc0fd000700 1 --2- 172.21.15.18:0/2589867384 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc0f8065b70 0x7fc0f8119ff0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fc0ec00c830 tx=0x7fc0ec004470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:10:12.435 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:12.437+0000 7fc0f4c00700 1 -- 172.21.15.18:0/2589867384 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc0ec00a760 con 0x7fc0f8065b70 2024-04-12T21:10:12.435 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:12.437+0000 7fc0f4c00700 1 -- 172.21.15.18:0/2589867384 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc0ec00ad90 con 0x7fc0f8065b70 2024-04-12T21:10:12.435 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:12.437+0000 7fc0f4c00700 1 -- 172.21.15.18:0/2589867384 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc0ec0113a0 con 0x7fc0f8065b70 2024-04-12T21:10:12.435 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:12.437+0000 7fc0fe400700 1 -- 172.21.15.18:0/2589867384 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc0f8114230 con 0x7fc0f8065b70 2024-04-12T21:10:12.435 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:12.437+0000 7fc0fe400700 1 -- 172.21.15.18:0/2589867384 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc0f8114770 con 0x7fc0f8065b70 2024-04-12T21:10:12.435 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:12.438+0000 7fc0fe400700 1 -- 172.21.15.18:0/2589867384 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc0f80fc170 con 0x7fc0f8065b70 2024-04-12T21:10:12.436 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:12.439+0000 7fc0f4c00700 1 -- 172.21.15.18:0/2589867384 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7fc0ec012400 con 0x7fc0f8065b70 2024-04-12T21:10:12.437 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:12.440+0000 7fc0f4c00700 1 --2- 172.21.15.18:0/2589867384 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fc0e006ec00 0x7fc0e00710b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:10:12.437 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:12.441+0000 7fc0f4c00700 1 -- 172.21.15.18:0/2589867384 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 6757+0+0 (secure 0 0 0) 0x7fc0ec08e900 con 0x7fc0f8065b70 2024-04-12T21:10:12.437 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:12.441+0000 7fc0f7e00700 1 --2- 172.21.15.18:0/2589867384 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fc0e006ec00 0x7fc0e00710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:10:12.438 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:12.441+0000 7fc0f7e00700 1 --2- 172.21.15.18:0/2589867384 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fc0e006ec00 0x7fc0e00710b0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fc0e4004f90 tx=0x7fc0e4009e30 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:10:12.441 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:12.445+0000 7fc0f4c00700 1 -- 172.21.15.18:0/2589867384 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7fc0ec05b1f0 con 0x7fc0f8065b70 2024-04-12T21:10:12.600 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:10:12 smithi018 ceph-mon[112590]: from='client.34584 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:10:12.773 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:12.776+0000 7fc0fe400700 1 -- 172.21.15.18:0/2589867384 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fc0f804bf00 con 0x7fc0f8065b70 2024-04-12T21:10:12.774 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:12.777+0000 7fc0f4c00700 1 -- 172.21.15.18:0/2589867384 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+837 (secure 0 0 0) 0x7fc0ec05ad80 con 0x7fc0f8065b70 2024-04-12T21:10:12.774 INFO:teuthology.orchestra.run.smithi018.stdout:{ 2024-04-12T21:10:12.774 INFO:teuthology.orchestra.run.smithi018.stdout: "mon": { 2024-04-12T21:10:12.774 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 19.0.0-2879-g324baf8d (324baf8ddb91dea46cac50cb535c306115791355) squid (dev)": 2 2024-04-12T21:10:12.774 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-12T21:10:12.775 INFO:teuthology.orchestra.run.smithi018.stdout: "mgr": { 2024-04-12T21:10:12.775 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 19.0.0-2879-g324baf8d (324baf8ddb91dea46cac50cb535c306115791355) squid (dev)": 2 2024-04-12T21:10:12.775 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-12T21:10:12.775 INFO:teuthology.orchestra.run.smithi018.stdout: "osd": { 2024-04-12T21:10:12.775 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 19.0.0-2879-g324baf8d (324baf8ddb91dea46cac50cb535c306115791355) squid (dev)": 6 2024-04-12T21:10:12.775 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-12T21:10:12.775 INFO:teuthology.orchestra.run.smithi018.stdout: "mds": { 2024-04-12T21:10:12.775 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 17.2.7-827-gd3b5060a (d3b5060a17449edfae9385cdf792afec109fca5d) quincy (stable)": 2, 2024-04-12T21:10:12.775 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 19.0.0-2879-g324baf8d (324baf8ddb91dea46cac50cb535c306115791355) squid (dev)": 2 2024-04-12T21:10:12.776 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-12T21:10:12.776 INFO:teuthology.orchestra.run.smithi018.stdout: "overall": { 2024-04-12T21:10:12.776 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 17.2.7-827-gd3b5060a (d3b5060a17449edfae9385cdf792afec109fca5d) quincy (stable)": 2, 2024-04-12T21:10:12.776 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 19.0.0-2879-g324baf8d (324baf8ddb91dea46cac50cb535c306115791355) squid (dev)": 12 2024-04-12T21:10:12.776 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-04-12T21:10:12.776 INFO:teuthology.orchestra.run.smithi018.stdout:} 2024-04-12T21:10:12.776 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:12.780+0000 7fc0fe400700 1 -- 172.21.15.18:0/2589867384 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fc0e006ec00 msgr2=0x7fc0e00710b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:10:12.777 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:12.780+0000 7fc0fe400700 1 --2- 172.21.15.18:0/2589867384 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fc0e006ec00 0x7fc0e00710b0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fc0e4004f90 tx=0x7fc0e4009e30 comp rx=0 tx=0).stop 2024-04-12T21:10:12.777 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:12.780+0000 7fc0fe400700 1 -- 172.21.15.18:0/2589867384 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc0f8065b70 msgr2=0x7fc0f8119ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:10:12.777 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:12.780+0000 7fc0fe400700 1 --2- 172.21.15.18:0/2589867384 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc0f8065b70 0x7fc0f8119ff0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fc0ec00c830 tx=0x7fc0ec004470 comp rx=0 tx=0).stop 2024-04-12T21:10:12.778 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:12.780+0000 7fc0fe400700 1 -- 172.21.15.18:0/2589867384 shutdown_connections 2024-04-12T21:10:12.778 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:12.780+0000 7fc0fe400700 1 --2- 172.21.15.18:0/2589867384 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fc0e006ec00 0x7fc0e00710b0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:10:12.778 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:12.780+0000 7fc0fe400700 1 --2- 172.21.15.18:0/2589867384 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc0f8065b70 0x7fc0f8119ff0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:10:12.778 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:12.780+0000 7fc0fe400700 1 --2- 172.21.15.18:0/2589867384 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc0f8101ae0 0x7fc0f811a530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:10:12.778 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:12.780+0000 7fc0fe400700 1 -- 172.21.15.18:0/2589867384 >> 172.21.15.18:0/2589867384 conn(0x7fc0f8071b70 msgr2=0x7fc0f80f9b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:10:12.778 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:12.781+0000 7fc0fe400700 1 -- 172.21.15.18:0/2589867384 shutdown_connections 2024-04-12T21:10:12.778 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:12.781+0000 7fc0fe400700 1 -- 172.21.15.18:0/2589867384 wait complete. 2024-04-12T21:10:12.779 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:10:12 smithi186 ceph-mon[86379]: from='client.34584 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:10:12.925 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:12.928+0000 7fbeaaa00700 1 -- 172.21.15.18:0/2703806156 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fbea4104ff0 msgr2=0x7fbea41053c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:10:12.925 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:12.928+0000 7fbeaaa00700 1 --2- 172.21.15.18:0/2703806156 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fbea4104ff0 0x7fbea41053c0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fbe94004940 tx=0x7fbe94009e30 comp rx=0 tx=0).stop 2024-04-12T21:10:12.925 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:12.928+0000 7fbeaaa00700 1 -- 172.21.15.18:0/2703806156 shutdown_connections 2024-04-12T21:10:12.925 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:12.928+0000 7fbeaaa00700 1 --2- 172.21.15.18:0/2703806156 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbea4105990 0x7fbea40ff000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:10:12.925 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:12.928+0000 7fbeaaa00700 1 --2- 172.21.15.18:0/2703806156 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fbea4104ff0 0x7fbea41053c0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:10:12.925 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:12.928+0000 7fbeaaa00700 1 -- 172.21.15.18:0/2703806156 >> 172.21.15.18:0/2703806156 conn(0x7fbea40faba0 msgr2=0x7fbea40fcfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:10:12.925 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:12.928+0000 7fbeaaa00700 1 -- 172.21.15.18:0/2703806156 shutdown_connections 2024-04-12T21:10:12.925 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:12.928+0000 7fbeaaa00700 1 -- 172.21.15.18:0/2703806156 wait complete. 2024-04-12T21:10:12.927 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:12.929+0000 7fbeaaa00700 1 Processor -- start 2024-04-12T21:10:12.927 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:12.929+0000 7fbeaaa00700 1 -- start start 2024-04-12T21:10:12.927 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:12.929+0000 7fbeaaa00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fbea4104ff0 0x7fbea4115b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:10:12.927 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:12.929+0000 7fbeaaa00700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbea4105990 0x7fbea4116080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:10:12.927 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:12.929+0000 7fbeaaa00700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbea4116760 con 0x7fbea4104ff0 2024-04-12T21:10:12.927 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:12.929+0000 7fbeaaa00700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbea410fcb0 con 0x7fbea4105990 2024-04-12T21:10:12.927 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:12.930+0000 7fbea3e00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fbea4104ff0 0x7fbea4115b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:10:12.927 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:12.930+0000 7fbea3e00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fbea4104ff0 0x7fbea4115b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:43308/0 (socket says 172.21.15.18:43308) 2024-04-12T21:10:12.927 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:12.930+0000 7fbea3e00700 1 -- 172.21.15.18:0/1356855206 learned_addr learned my addr 172.21.15.18:0/1356855206 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:10:12.927 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:12.930+0000 7fbea3e00700 1 -- 172.21.15.18:0/1356855206 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbea4105990 msgr2=0x7fbea4116080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:10:12.927 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:12.930+0000 7fbea3e00700 1 --2- 172.21.15.18:0/1356855206 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbea4105990 0x7fbea4116080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:10:12.927 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:12.930+0000 7fbea3e00700 1 -- 172.21.15.18:0/1356855206 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe94005040 con 0x7fbea4104ff0 2024-04-12T21:10:12.927 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:12.930+0000 7fbea3e00700 1 --2- 172.21.15.18:0/1356855206 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fbea4104ff0 0x7fbea4115b40 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fbe94009150 tx=0x7fbe94009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:10:12.928 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:12.931+0000 7fbe9be00700 1 -- 172.21.15.18:0/1356855206 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbe9400e040 con 0x7fbea4104ff0 2024-04-12T21:10:12.928 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:12.931+0000 7fbe9be00700 1 -- 172.21.15.18:0/1356855206 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fbe94013420 con 0x7fbea4104ff0 2024-04-12T21:10:12.928 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:12.931+0000 7fbeaaa00700 1 -- 172.21.15.18:0/1356855206 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbea410fed0 con 0x7fbea4104ff0 2024-04-12T21:10:12.928 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:12.931+0000 7fbe9be00700 1 -- 172.21.15.18:0/1356855206 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbe9401c630 con 0x7fbea4104ff0 2024-04-12T21:10:12.928 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:12.931+0000 7fbeaaa00700 1 -- 172.21.15.18:0/1356855206 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbea4110490 con 0x7fbea4104ff0 2024-04-12T21:10:12.929 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:12.932+0000 7fbeaaa00700 1 -- 172.21.15.18:0/1356855206 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbea4100740 con 0x7fbea4104ff0 2024-04-12T21:10:12.929 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:12.933+0000 7fbe9be00700 1 -- 172.21.15.18:0/1356855206 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7fbe94013590 con 0x7fbea4104ff0 2024-04-12T21:10:12.930 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:12.934+0000 7fbe9be00700 1 --2- 172.21.15.18:0/1356855206 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fbe8806ec00 0x7fbe880710b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:10:12.930 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:12.934+0000 7fbe9be00700 1 -- 172.21.15.18:0/1356855206 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 6757+0+0 (secure 0 0 0) 0x7fbe9408fc00 con 0x7fbea4104ff0 2024-04-12T21:10:12.931 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:12.934+0000 7fbea3400700 1 --2- 172.21.15.18:0/1356855206 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fbe8806ec00 0x7fbe880710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:10:12.931 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:12.935+0000 7fbea3400700 1 --2- 172.21.15.18:0/1356855206 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fbe8806ec00 0x7fbe880710b0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fbe90006fd0 tx=0x7fbe90006db0 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:10:12.935 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:12.939+0000 7fbe9be00700 1 -- 172.21.15.18:0/1356855206 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7fbe9405c430 con 0x7fbea4104ff0 2024-04-12T21:10:13.205 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:13.208+0000 7fbeaaa00700 1 -- 172.21.15.18:0/1356855206 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fbea4110700 con 0x7fbea4104ff0 2024-04-12T21:10:13.206 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:13.209+0000 7fbe9be00700 1 -- 172.21.15.18:0/1356855206 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 29 v29) v1 ==== 76+0+1872 (secure 0 0 0) 0x7fbe9405bfc0 con 0x7fbea4104ff0 2024-04-12T21:10:13.207 INFO:teuthology.orchestra.run.smithi018.stdout:e29 2024-04-12T21:10:13.207 INFO:teuthology.orchestra.run.smithi018.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-12T21:10:13.207 INFO:teuthology.orchestra.run.smithi018.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-12T21:10:13.207 INFO:teuthology.orchestra.run.smithi018.stdout:legacy client fscid: 1 2024-04-12T21:10:13.207 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-12T21:10:13.207 INFO:teuthology.orchestra.run.smithi018.stdout:Filesystem 'cephfs' (1) 2024-04-12T21:10:13.208 INFO:teuthology.orchestra.run.smithi018.stdout:fs_name cephfs 2024-04-12T21:10:13.208 INFO:teuthology.orchestra.run.smithi018.stdout:epoch 24 2024-04-12T21:10:13.208 INFO:teuthology.orchestra.run.smithi018.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-04-12T21:10:13.208 INFO:teuthology.orchestra.run.smithi018.stdout:created 2024-04-12T20:51:57.362850+0000 2024-04-12T21:10:13.208 INFO:teuthology.orchestra.run.smithi018.stdout:modified 2024-04-12T21:06:59.717190+0000 2024-04-12T21:10:13.208 INFO:teuthology.orchestra.run.smithi018.stdout:tableserver 0 2024-04-12T21:10:13.208 INFO:teuthology.orchestra.run.smithi018.stdout:root 0 2024-04-12T21:10:13.208 INFO:teuthology.orchestra.run.smithi018.stdout:session_timeout 60 2024-04-12T21:10:13.208 INFO:teuthology.orchestra.run.smithi018.stdout:session_autoclose 300 2024-04-12T21:10:13.208 INFO:teuthology.orchestra.run.smithi018.stdout:max_file_size 1099511627776 2024-04-12T21:10:13.208 INFO:teuthology.orchestra.run.smithi018.stdout:max_xattr_size 65536 2024-04-12T21:10:13.208 INFO:teuthology.orchestra.run.smithi018.stdout:required_client_features {} 2024-04-12T21:10:13.209 INFO:teuthology.orchestra.run.smithi018.stdout:last_failure 0 2024-04-12T21:10:13.209 INFO:teuthology.orchestra.run.smithi018.stdout:last_failure_osd_epoch 100 2024-04-12T21:10:13.209 INFO:teuthology.orchestra.run.smithi018.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-12T21:10:13.209 INFO:teuthology.orchestra.run.smithi018.stdout:max_mds 2 2024-04-12T21:10:13.209 INFO:teuthology.orchestra.run.smithi018.stdout:in 0,1 2024-04-12T21:10:13.209 INFO:teuthology.orchestra.run.smithi018.stdout:up {} 2024-04-12T21:10:13.209 INFO:teuthology.orchestra.run.smithi018.stdout:failed 0,1 2024-04-12T21:10:13.209 INFO:teuthology.orchestra.run.smithi018.stdout:damaged 2024-04-12T21:10:13.209 INFO:teuthology.orchestra.run.smithi018.stdout:stopped 2024-04-12T21:10:13.209 INFO:teuthology.orchestra.run.smithi018.stdout:data_pools [3] 2024-04-12T21:10:13.209 INFO:teuthology.orchestra.run.smithi018.stdout:metadata_pool 2 2024-04-12T21:10:13.210 INFO:teuthology.orchestra.run.smithi018.stdout:inline_data disabled 2024-04-12T21:10:13.210 INFO:teuthology.orchestra.run.smithi018.stdout:balancer 2024-04-12T21:10:13.210 INFO:teuthology.orchestra.run.smithi018.stdout:bal_rank_mask -1 2024-04-12T21:10:13.210 INFO:teuthology.orchestra.run.smithi018.stdout:standby_count_wanted 1 2024-04-12T21:10:13.210 INFO:teuthology.orchestra.run.smithi018.stdout:qdb_cluster leader: 0 members: 2024-04-12T21:10:13.210 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-12T21:10:13.210 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-12T21:10:13.210 INFO:teuthology.orchestra.run.smithi018.stdout:Standby daemons: 2024-04-12T21:10:13.210 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-12T21:10:13.210 INFO:teuthology.orchestra.run.smithi018.stdout:[mds.cephfs.smithi186.wjnafs{-1:24277} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.186:6826/438347207,v1:172.21.15.186:6827/438347207] compat {c=[1],r=[1],i=[7ff]}] 2024-04-12T21:10:13.210 INFO:teuthology.orchestra.run.smithi018.stdout:[mds.cephfs.smithi186.jdllyn{-1:24281} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.186:6824/1464359952,v1:172.21.15.186:6825/1464359952] compat {c=[1],r=[1],i=[7ff]}] 2024-04-12T21:10:13.210 INFO:teuthology.orchestra.run.smithi018.stdout:[mds.cephfs.smithi018.acqbsc{-1:34436} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.18:6828/755179764,v1:172.21.15.18:6829/755179764] compat {c=[1],r=[1],i=[1fff]}] 2024-04-12T21:10:13.211 INFO:teuthology.orchestra.run.smithi018.stdout:[mds.cephfs.smithi018.szjluo{-1:34468} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.18:6826/2407992653,v1:172.21.15.18:6827/2407992653] compat {c=[1],r=[1],i=[1fff]}] 2024-04-12T21:10:13.211 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:13.212+0000 7fbeaaa00700 1 -- 172.21.15.18:0/1356855206 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fbe8806ec00 msgr2=0x7fbe880710b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:10:13.212 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:13.212+0000 7fbeaaa00700 1 --2- 172.21.15.18:0/1356855206 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fbe8806ec00 0x7fbe880710b0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fbe90006fd0 tx=0x7fbe90006db0 comp rx=0 tx=0).stop 2024-04-12T21:10:13.212 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:13.212+0000 7fbeaaa00700 1 -- 172.21.15.18:0/1356855206 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fbea4104ff0 msgr2=0x7fbea4115b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:10:13.212 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:13.212+0000 7fbeaaa00700 1 --2- 172.21.15.18:0/1356855206 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fbea4104ff0 0x7fbea4115b40 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fbe94009150 tx=0x7fbe94009e30 comp rx=0 tx=0).stop 2024-04-12T21:10:13.212 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:13.212+0000 7fbeaaa00700 1 -- 172.21.15.18:0/1356855206 shutdown_connections 2024-04-12T21:10:13.212 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:13.212+0000 7fbeaaa00700 1 --2- 172.21.15.18:0/1356855206 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fbe8806ec00 0x7fbe880710b0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:10:13.212 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:13.212+0000 7fbeaaa00700 1 --2- 172.21.15.18:0/1356855206 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fbea4104ff0 0x7fbea4115b40 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:10:13.212 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:13.212+0000 7fbeaaa00700 1 --2- 172.21.15.18:0/1356855206 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbea4105990 0x7fbea4116080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:10:13.212 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:13.212+0000 7fbeaaa00700 1 -- 172.21.15.18:0/1356855206 >> 172.21.15.18:0/1356855206 conn(0x7fbea40faba0 msgr2=0x7fbea40fbae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:10:13.212 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:13.213+0000 7fbeaaa00700 1 -- 172.21.15.18:0/1356855206 shutdown_connections 2024-04-12T21:10:13.212 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:13.213+0000 7fbeaaa00700 1 -- 172.21.15.18:0/1356855206 wait complete. 2024-04-12T21:10:13.213 INFO:teuthology.orchestra.run.smithi018.stderr:dumped fsmap epoch 29 2024-04-12T21:10:13.366 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:13.371+0000 7f1af3200700 1 -- 172.21.15.18:0/714732779 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1af4104e60 msgr2=0x7f1af4105230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:10:13.367 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:13.371+0000 7f1af3200700 1 --2- 172.21.15.18:0/714732779 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1af4104e60 0x7f1af4105230 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f1ae4004940 tx=0x7f1ae4009e30 comp rx=0 tx=0).stop 2024-04-12T21:10:13.367 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:13.371+0000 7f1af3200700 1 -- 172.21.15.18:0/714732779 shutdown_connections 2024-04-12T21:10:13.367 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:13.371+0000 7f1af3200700 1 --2- 172.21.15.18:0/714732779 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1af4105800 0x7f1af40fee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:10:13.367 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:13.371+0000 7f1af3200700 1 --2- 172.21.15.18:0/714732779 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1af4104e60 0x7f1af4105230 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:10:13.367 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:13.371+0000 7f1af3200700 1 -- 172.21.15.18:0/714732779 >> 172.21.15.18:0/714732779 conn(0x7f1af40fab10 msgr2=0x7f1af40fcef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:10:13.367 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:13.371+0000 7f1af3200700 1 -- 172.21.15.18:0/714732779 shutdown_connections 2024-04-12T21:10:13.367 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:13.371+0000 7f1af3200700 1 -- 172.21.15.18:0/714732779 wait complete. 2024-04-12T21:10:13.368 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:13.372+0000 7f1af3200700 1 Processor -- start 2024-04-12T21:10:13.369 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:13.372+0000 7f1af3200700 1 -- start start 2024-04-12T21:10:13.369 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:13.372+0000 7f1af3200700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1af4104e60 0x7f1af40724d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:10:13.369 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:13.372+0000 7f1af3200700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1af4105800 0x7f1af4072a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:10:13.369 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:13.372+0000 7f1af3200700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1af4076660 con 0x7f1af4105800 2024-04-12T21:10:13.369 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:13.372+0000 7f1af3200700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1af4072f50 con 0x7f1af4104e60 2024-04-12T21:10:13.369 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:13.373+0000 7f1af1400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1af4105800 0x7f1af4072a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:10:13.369 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:13.373+0000 7f1af1400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1af4105800 0x7f1af4072a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:43316/0 (socket says 172.21.15.18:43316) 2024-04-12T21:10:13.369 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:13.373+0000 7f1af1400700 1 -- 172.21.15.18:0/8678322 learned_addr learned my addr 172.21.15.18:0/8678322 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:10:13.369 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:13.373+0000 7f1af1e00700 1 --2- 172.21.15.18:0/8678322 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1af4104e60 0x7f1af40724d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:10:13.369 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:13.373+0000 7f1af1400700 1 -- 172.21.15.18:0/8678322 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1af4104e60 msgr2=0x7f1af40724d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:10:13.369 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:13.373+0000 7f1af1400700 1 --2- 172.21.15.18:0/8678322 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1af4104e60 0x7f1af40724d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:10:13.370 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:13.373+0000 7f1af1400700 1 -- 172.21.15.18:0/8678322 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ae4005040 con 0x7f1af4105800 2024-04-12T21:10:13.370 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:13.373+0000 7f1af1e00700 1 --2- 172.21.15.18:0/8678322 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1af4104e60 0x7f1af40724d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-12T21:10:13.371 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:13.374+0000 7f1af1400700 1 --2- 172.21.15.18:0/8678322 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1af4105800 0x7f1af4072a10 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f1ae0002a50 tx=0x7f1ae00046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:10:13.371 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:13.374+0000 7f1aea000700 1 -- 172.21.15.18:0/8678322 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1ae0007cc0 con 0x7f1af4105800 2024-04-12T21:10:13.371 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:13.374+0000 7f1aea000700 1 -- 172.21.15.18:0/8678322 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f1ae000f040 con 0x7f1af4105800 2024-04-12T21:10:13.371 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:13.374+0000 7f1af3200700 1 -- 172.21.15.18:0/8678322 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1af4073230 con 0x7f1af4105800 2024-04-12T21:10:13.371 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:13.374+0000 7f1aea000700 1 -- 172.21.15.18:0/8678322 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1ae0009630 con 0x7f1af4105800 2024-04-12T21:10:13.371 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:13.374+0000 7f1af3200700 1 -- 172.21.15.18:0/8678322 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1af411e500 con 0x7f1af4105800 2024-04-12T21:10:13.371 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:13.375+0000 7f1af3200700 1 -- 172.21.15.18:0/8678322 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1af41005b0 con 0x7f1af4105800 2024-04-12T21:10:13.371 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:13.376+0000 7f1aea000700 1 -- 172.21.15.18:0/8678322 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7f1ae0009790 con 0x7f1af4105800 2024-04-12T21:10:13.373 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:13.376+0000 7f1aea000700 1 --2- 172.21.15.18:0/8678322 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f1ad806ec00 0x7f1ad80710b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:10:13.373 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:13.376+0000 7f1aea000700 1 -- 172.21.15.18:0/8678322 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 6757+0+0 (secure 0 0 0) 0x7f1ae008e6f0 con 0x7f1af4105800 2024-04-12T21:10:13.373 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:13.378+0000 7f1af1e00700 1 --2- 172.21.15.18:0/8678322 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f1ad806ec00 0x7f1ad80710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:10:13.373 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:13.378+0000 7f1af1e00700 1 --2- 172.21.15.18:0/8678322 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f1ad806ec00 0x7f1ad80710b0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f1ae400b3b0 tx=0x7f1ae4009e30 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:10:13.379 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:13.384+0000 7f1aea000700 1 -- 172.21.15.18:0/8678322 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f1ae00573a0 con 0x7f1af4105800 2024-04-12T21:10:13.600 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:10:13 smithi018 ceph-mon[112590]: from='client.34588 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:10:13.600 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:10:13 smithi018 ceph-mon[112590]: pgmap v312: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:10:13.600 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:10:13 smithi018 ceph-mon[112590]: from='client.34592 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:10:13.600 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:10:13 smithi018 ceph-mon[112590]: from='client.? 172.21.15.18:0/2589867384' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T21:10:13.600 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:10:13 smithi018 ceph-mon[112590]: from='client.? 172.21.15.18:0/1356855206' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-12T21:10:13.635 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:13.640+0000 7f1af3200700 1 -- 172.21.15.18:0/8678322 --> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f1af404bed0 con 0x7f1ad806ec00 2024-04-12T21:10:13.637 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:13.642+0000 7f1aea000700 1 -- 172.21.15.18:0/8678322 <== mgr.34102 v2:172.21.15.18:6800/3812891919 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f1af404bed0 con 0x7f1ad806ec00 2024-04-12T21:10:13.638 INFO:teuthology.orchestra.run.smithi018.stdout:{ 2024-04-12T21:10:13.638 INFO:teuthology.orchestra.run.smithi018.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:f4452def3be4d350f4a8e5feaf7ebb1511be9298fe68478ba7a2d6fbd7d69a63", 2024-04-12T21:10:13.638 INFO:teuthology.orchestra.run.smithi018.stdout: "in_progress": true, 2024-04-12T21:10:13.638 INFO:teuthology.orchestra.run.smithi018.stdout: "which": "Upgrading all daemon types on all hosts", 2024-04-12T21:10:13.638 INFO:teuthology.orchestra.run.smithi018.stdout: "services_complete": [ 2024-04-12T21:10:13.638 INFO:teuthology.orchestra.run.smithi018.stdout: "crash", 2024-04-12T21:10:13.638 INFO:teuthology.orchestra.run.smithi018.stdout: "osd", 2024-04-12T21:10:13.638 INFO:teuthology.orchestra.run.smithi018.stdout: "mon", 2024-04-12T21:10:13.638 INFO:teuthology.orchestra.run.smithi018.stdout: "mgr" 2024-04-12T21:10:13.638 INFO:teuthology.orchestra.run.smithi018.stdout: ], 2024-04-12T21:10:13.639 INFO:teuthology.orchestra.run.smithi018.stdout: "progress": "14/21 daemons upgraded", 2024-04-12T21:10:13.639 INFO:teuthology.orchestra.run.smithi018.stdout: "message": "Currently upgrading mds daemons", 2024-04-12T21:10:13.639 INFO:teuthology.orchestra.run.smithi018.stdout: "is_paused": false 2024-04-12T21:10:13.639 INFO:teuthology.orchestra.run.smithi018.stdout:} 2024-04-12T21:10:13.641 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:13.645+0000 7f1af3200700 1 -- 172.21.15.18:0/8678322 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f1ad806ec00 msgr2=0x7f1ad80710b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:10:13.641 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:13.645+0000 7f1af3200700 1 --2- 172.21.15.18:0/8678322 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f1ad806ec00 0x7f1ad80710b0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f1ae400b3b0 tx=0x7f1ae4009e30 comp rx=0 tx=0).stop 2024-04-12T21:10:13.641 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:13.645+0000 7f1af3200700 1 -- 172.21.15.18:0/8678322 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1af4105800 msgr2=0x7f1af4072a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:10:13.641 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:13.645+0000 7f1af3200700 1 --2- 172.21.15.18:0/8678322 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1af4105800 0x7f1af4072a10 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f1ae0002a50 tx=0x7f1ae00046b0 comp rx=0 tx=0).stop 2024-04-12T21:10:13.641 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:13.645+0000 7f1af3200700 1 -- 172.21.15.18:0/8678322 shutdown_connections 2024-04-12T21:10:13.641 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:13.645+0000 7f1af3200700 1 --2- 172.21.15.18:0/8678322 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f1ad806ec00 0x7f1ad80710b0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:10:13.641 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:13.645+0000 7f1af3200700 1 --2- 172.21.15.18:0/8678322 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1af4104e60 0x7f1af40724d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:10:13.641 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:13.645+0000 7f1af3200700 1 --2- 172.21.15.18:0/8678322 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1af4105800 0x7f1af4072a10 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:10:13.642 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:13.645+0000 7f1af3200700 1 -- 172.21.15.18:0/8678322 >> 172.21.15.18:0/8678322 conn(0x7f1af40fab10 msgr2=0x7f1af40fc9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:10:13.642 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:13.646+0000 7f1af3200700 1 -- 172.21.15.18:0/8678322 shutdown_connections 2024-04-12T21:10:13.642 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:13.646+0000 7f1af3200700 1 -- 172.21.15.18:0/8678322 wait complete. 2024-04-12T21:10:13.779 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:10:13 smithi186 ceph-mon[86379]: from='client.34588 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:10:13.779 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:10:13 smithi186 ceph-mon[86379]: pgmap v312: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:10:13.779 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:10:13 smithi186 ceph-mon[86379]: from='client.34592 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:10:13.779 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:10:13 smithi186 ceph-mon[86379]: from='client.? 172.21.15.18:0/2589867384' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T21:10:13.779 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:10:13 smithi186 ceph-mon[86379]: from='client.? 172.21.15.18:0/1356855206' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-12T21:10:13.798 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:13.802+0000 7fce22600700 1 -- 172.21.15.18:0/1527665850 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fce1c101d70 msgr2=0x7fce1c102140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:10:13.798 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:13.802+0000 7fce22600700 1 --2- 172.21.15.18:0/1527665850 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fce1c101d70 0x7fce1c102140 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fce0c004940 tx=0x7fce0c009e30 comp rx=0 tx=0).stop 2024-04-12T21:10:13.798 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:13.802+0000 7fce22600700 1 -- 172.21.15.18:0/1527665850 shutdown_connections 2024-04-12T21:10:13.798 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:13.802+0000 7fce22600700 1 --2- 172.21.15.18:0/1527665850 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fce1c102710 0x7fce1c0fbd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:10:13.798 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:13.802+0000 7fce22600700 1 --2- 172.21.15.18:0/1527665850 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fce1c101d70 0x7fce1c102140 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:10:13.798 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:13.802+0000 7fce22600700 1 -- 172.21.15.18:0/1527665850 >> 172.21.15.18:0/1527665850 conn(0x7fce1c072bc0 msgr2=0x7fce1c072fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:10:13.798 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:13.802+0000 7fce22600700 1 -- 172.21.15.18:0/1527665850 shutdown_connections 2024-04-12T21:10:13.798 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:13.802+0000 7fce22600700 1 -- 172.21.15.18:0/1527665850 wait complete. 2024-04-12T21:10:13.800 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:13.803+0000 7fce22600700 1 Processor -- start 2024-04-12T21:10:13.800 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:13.803+0000 7fce22600700 1 -- start start 2024-04-12T21:10:13.800 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:13.804+0000 7fce22600700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fce1c101d70 0x7fce1c113b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:10:13.800 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:13.804+0000 7fce22600700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fce1c102710 0x7fce1c1140d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:10:13.800 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:13.804+0000 7fce22600700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce1c114720 con 0x7fce1c101d70 2024-04-12T21:10:13.800 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:13.804+0000 7fce22600700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce1c10db10 con 0x7fce1c102710 2024-04-12T21:10:13.800 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:13.804+0000 7fce1b400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fce1c101d70 0x7fce1c113b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:10:13.800 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:13.804+0000 7fce1b400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fce1c101d70 0x7fce1c113b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:43332/0 (socket says 172.21.15.18:43332) 2024-04-12T21:10:13.801 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:13.804+0000 7fce1b400700 1 -- 172.21.15.18:0/1039254863 learned_addr learned my addr 172.21.15.18:0/1039254863 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:10:13.801 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:13.805+0000 7fce1aa00700 1 --2- 172.21.15.18:0/1039254863 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fce1c102710 0x7fce1c1140d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:10:13.801 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:13.805+0000 7fce1b400700 1 -- 172.21.15.18:0/1039254863 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fce1c102710 msgr2=0x7fce1c1140d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:10:13.801 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:13.805+0000 7fce1b400700 1 --2- 172.21.15.18:0/1039254863 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fce1c102710 0x7fce1c1140d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:10:13.802 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:13.805+0000 7fce1b400700 1 -- 172.21.15.18:0/1039254863 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce0c005040 con 0x7fce1c101d70 2024-04-12T21:10:13.802 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:13.805+0000 7fce1b400700 1 --2- 172.21.15.18:0/1039254863 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fce1c101d70 0x7fce1c113b90 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fce0c00b8b0 tx=0x7fce0c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:10:13.802 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:13.806+0000 7fce13400700 1 -- 172.21.15.18:0/1039254863 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fce0c00e040 con 0x7fce1c101d70 2024-04-12T21:10:13.802 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:13.806+0000 7fce22600700 1 -- 172.21.15.18:0/1039254863 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fce1c10dd90 con 0x7fce1c101d70 2024-04-12T21:10:13.802 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:13.806+0000 7fce13400700 1 -- 172.21.15.18:0/1039254863 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fce0c004290 con 0x7fce1c101d70 2024-04-12T21:10:13.802 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:13.806+0000 7fce13400700 1 -- 172.21.15.18:0/1039254863 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fce0c01c3f0 con 0x7fce1c101d70 2024-04-12T21:10:13.802 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:13.806+0000 7fce22600700 1 -- 172.21.15.18:0/1039254863 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fce1c10e350 con 0x7fce1c101d70 2024-04-12T21:10:13.802 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:13.807+0000 7fce22600700 1 -- 172.21.15.18:0/1039254863 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fce1c0ff080 con 0x7fce1c101d70 2024-04-12T21:10:13.803 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:13.808+0000 7fce13400700 1 -- 172.21.15.18:0/1039254863 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7fce0c0137b0 con 0x7fce1c101d70 2024-04-12T21:10:13.804 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:13.809+0000 7fce13400700 1 --2- 172.21.15.18:0/1039254863 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fce0006ec00 0x7fce000710b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:10:13.804 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:13.809+0000 7fce13400700 1 -- 172.21.15.18:0/1039254863 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 6757+0+0 (secure 0 0 0) 0x7fce0c08fce0 con 0x7fce1c101d70 2024-04-12T21:10:13.805 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:13.809+0000 7fce1aa00700 1 --2- 172.21.15.18:0/1039254863 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fce0006ec00 0x7fce000710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:10:13.805 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:13.810+0000 7fce1aa00700 1 --2- 172.21.15.18:0/1039254863 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fce0006ec00 0x7fce000710b0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fce08007fd0 tx=0x7fce08009040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:10:13.809 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:13.814+0000 7fce13400700 1 -- 172.21.15.18:0/1039254863 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7fce0c05c530 con 0x7fce1c101d70 2024-04-12T21:10:14.132 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:14.137+0000 7fce22600700 1 -- 172.21.15.18:0/1039254863 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7fce1c05f5f0 con 0x7fce1c101d70 2024-04-12T21:10:14.133 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:14.137+0000 7fce13400700 1 -- 172.21.15.18:0/1039254863 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+233 (secure 0 0 0) 0x7fce0c05c0c0 con 0x7fce1c101d70 2024-04-12T21:10:14.133 INFO:teuthology.orchestra.run.smithi018.stdout:HEALTH_ERR 1 filesystem is degraded; 1 filesystem is offline 2024-04-12T21:10:14.134 INFO:teuthology.orchestra.run.smithi018.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-04-12T21:10:14.134 INFO:teuthology.orchestra.run.smithi018.stdout: fs cephfs is degraded 2024-04-12T21:10:14.134 INFO:teuthology.orchestra.run.smithi018.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-04-12T21:10:14.134 INFO:teuthology.orchestra.run.smithi018.stdout: fs cephfs is offline because no MDS is active for it. 2024-04-12T21:10:14.136 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:14.140+0000 7fce22600700 1 -- 172.21.15.18:0/1039254863 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fce0006ec00 msgr2=0x7fce000710b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:10:14.136 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:14.140+0000 7fce22600700 1 --2- 172.21.15.18:0/1039254863 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fce0006ec00 0x7fce000710b0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fce08007fd0 tx=0x7fce08009040 comp rx=0 tx=0).stop 2024-04-12T21:10:14.137 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:14.140+0000 7fce22600700 1 -- 172.21.15.18:0/1039254863 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fce1c101d70 msgr2=0x7fce1c113b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:10:14.137 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:14.140+0000 7fce22600700 1 --2- 172.21.15.18:0/1039254863 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fce1c101d70 0x7fce1c113b90 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fce0c00b8b0 tx=0x7fce0c009e30 comp rx=0 tx=0).stop 2024-04-12T21:10:14.137 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:14.141+0000 7fce22600700 1 -- 172.21.15.18:0/1039254863 shutdown_connections 2024-04-12T21:10:14.137 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:14.141+0000 7fce22600700 1 --2- 172.21.15.18:0/1039254863 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fce0006ec00 0x7fce000710b0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:10:14.137 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:14.141+0000 7fce22600700 1 --2- 172.21.15.18:0/1039254863 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fce1c101d70 0x7fce1c113b90 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:10:14.137 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:14.141+0000 7fce22600700 1 --2- 172.21.15.18:0/1039254863 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fce1c102710 0x7fce1c1140d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:10:14.137 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:14.141+0000 7fce22600700 1 -- 172.21.15.18:0/1039254863 >> 172.21.15.18:0/1039254863 conn(0x7fce1c072bc0 msgr2=0x7fce1c0fafa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:10:14.137 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:14.141+0000 7fce22600700 1 -- 172.21.15.18:0/1039254863 shutdown_connections 2024-04-12T21:10:14.137 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:14.141+0000 7fce22600700 1 -- 172.21.15.18:0/1039254863 wait complete. 2024-04-12T21:10:14.779 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:10:14 smithi186 ceph-mon[86379]: from='client.? 172.21.15.18:0/1039254863' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-12T21:10:14.850 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:10:14 smithi018 ceph-mon[112590]: from='client.? 172.21.15.18:0/1039254863' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-12T21:10:15.779 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:10:15 smithi186 ceph-mon[86379]: from='client.34604 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:10:15.779 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:10:15 smithi186 ceph-mon[86379]: pgmap v313: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:10:15.850 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:10:15 smithi018 ceph-mon[112590]: from='client.34604 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:10:15.850 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:10:15 smithi018 ceph-mon[112590]: pgmap v313: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:10:17.779 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:10:17 smithi186 ceph-mon[86379]: pgmap v314: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:10:17.850 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:10:17 smithi018 ceph-mon[112590]: pgmap v314: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:10:19.779 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:10:19 smithi186 ceph-mon[86379]: pgmap v315: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:10:19.850 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:10:19 smithi018 ceph-mon[112590]: pgmap v315: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:10:21.779 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:10:21 smithi186 ceph-mon[86379]: pgmap v316: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:10:21.850 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:10:21 smithi018 ceph-mon[112590]: pgmap v316: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:10:23.779 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:10:23 smithi186 ceph-mon[86379]: pgmap v317: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:10:23.850 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:10:23 smithi018 ceph-mon[112590]: pgmap v317: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:10:25.779 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:10:25 smithi186 ceph-mon[86379]: pgmap v318: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:10:25.779 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:10:25 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T21:10:25.850 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:10:25 smithi018 ceph-mon[112590]: pgmap v318: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:10:25.850 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:10:25 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T21:10:27.779 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:10:27 smithi186 ceph-mon[86379]: pgmap v319: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:10:27.850 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:10:27 smithi018 ceph-mon[112590]: pgmap v319: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:10:29.779 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:10:29 smithi186 ceph-mon[86379]: pgmap v320: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:10:29.850 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:10:29 smithi018 ceph-mon[112590]: pgmap v320: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:10:31.779 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:10:31 smithi186 ceph-mon[86379]: pgmap v321: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:10:31.850 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:10:31 smithi018 ceph-mon[112590]: pgmap v321: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:10:33.779 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:10:33 smithi186 ceph-mon[86379]: pgmap v322: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:10:33.850 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:10:33 smithi018 ceph-mon[112590]: pgmap v322: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:10:35.779 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:10:35 smithi186 ceph-mon[86379]: pgmap v323: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:10:35.850 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:10:35 smithi018 ceph-mon[112590]: pgmap v323: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:10:37.782 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:10:37 smithi186 ceph-mon[86379]: pgmap v324: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:10:37.850 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:10:37 smithi018 ceph-mon[112590]: pgmap v324: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:10:39.779 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:10:39 smithi186 ceph-mon[86379]: pgmap v325: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:10:39.850 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:10:39 smithi018 ceph-mon[112590]: pgmap v325: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:10:40.752 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:10:40 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T21:10:40.779 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:10:40 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T21:10:41.779 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:10:41 smithi186 ceph-mon[86379]: pgmap v326: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:10:41.850 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:10:41 smithi018 ceph-mon[112590]: pgmap v326: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:10:43.779 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:10:43 smithi186 ceph-mon[86379]: pgmap v327: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:10:43.838 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:10:43 smithi018 ceph-mon[112590]: pgmap v327: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:10:44.307 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:44.310+0000 7f2099c00700 1 -- 172.21.15.18:0/2116823169 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f20940724f0 msgr2=0x7f20940728c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:10:44.307 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:44.310+0000 7f2099c00700 1 --2- 172.21.15.18:0/2116823169 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f20940724f0 0x7f20940728c0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f2084004940 tx=0x7f2084009e30 comp rx=0 tx=0).stop 2024-04-12T21:10:44.307 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:44.311+0000 7f2099c00700 1 -- 172.21.15.18:0/2116823169 shutdown_connections 2024-04-12T21:10:44.307 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:44.311+0000 7f2099c00700 1 --2- 172.21.15.18:0/2116823169 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2094072e00 0x7f209410d780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:10:44.307 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:44.311+0000 7f2099c00700 1 --2- 172.21.15.18:0/2116823169 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f20940724f0 0x7f20940728c0 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:10:44.307 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:44.311+0000 7f2099c00700 1 -- 172.21.15.18:0/2116823169 >> 172.21.15.18:0/2116823169 conn(0x7f20940faba0 msgr2=0x7f20940fcfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:10:44.307 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:44.311+0000 7f2099c00700 1 -- 172.21.15.18:0/2116823169 shutdown_connections 2024-04-12T21:10:44.307 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:44.311+0000 7f2099c00700 1 -- 172.21.15.18:0/2116823169 wait complete. 2024-04-12T21:10:44.308 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:44.312+0000 7f2099c00700 1 Processor -- start 2024-04-12T21:10:44.309 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:44.312+0000 7f2099c00700 1 -- start start 2024-04-12T21:10:44.309 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:44.312+0000 7f2099c00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f20940724f0 0x7f209411a0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:10:44.309 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:44.312+0000 7f2099c00700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2094072e00 0x7f209411a5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:10:44.309 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:44.312+0000 7f2099c00700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f209411ac40 con 0x7f20940724f0 2024-04-12T21:10:44.309 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:44.312+0000 7f2099c00700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2094114050 con 0x7f2094072e00 2024-04-12T21:10:44.309 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:44.313+0000 7f2092a00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f20940724f0 0x7f209411a0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:10:44.309 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:44.313+0000 7f2092a00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f20940724f0 0x7f209411a0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:42786/0 (socket says 172.21.15.18:42786) 2024-04-12T21:10:44.309 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:44.313+0000 7f2092a00700 1 -- 172.21.15.18:0/233716549 learned_addr learned my addr 172.21.15.18:0/233716549 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:10:44.309 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:44.313+0000 7f208be00700 1 --2- 172.21.15.18:0/233716549 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2094072e00 0x7f209411a5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:10:44.309 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:44.313+0000 7f2092a00700 1 -- 172.21.15.18:0/233716549 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2094072e00 msgr2=0x7f209411a5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:10:44.309 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:44.313+0000 7f2092a00700 1 --2- 172.21.15.18:0/233716549 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2094072e00 0x7f209411a5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:10:44.309 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:44.313+0000 7f2092a00700 1 -- 172.21.15.18:0/233716549 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2084005040 con 0x7f20940724f0 2024-04-12T21:10:44.309 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:44.313+0000 7f2092a00700 1 --2- 172.21.15.18:0/233716549 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f20940724f0 0x7f209411a0b0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f2084009180 tx=0x7f2084009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:10:44.310 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:44.314+0000 7f208aa00700 1 -- 172.21.15.18:0/233716549 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f208400e040 con 0x7f20940724f0 2024-04-12T21:10:44.310 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:44.314+0000 7f208aa00700 1 -- 172.21.15.18:0/233716549 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f2084013420 con 0x7f20940724f0 2024-04-12T21:10:44.310 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:44.314+0000 7f2099c00700 1 -- 172.21.15.18:0/233716549 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f20941142d0 con 0x7f20940724f0 2024-04-12T21:10:44.310 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:44.314+0000 7f208aa00700 1 -- 172.21.15.18:0/233716549 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f208401c630 con 0x7f20940724f0 2024-04-12T21:10:44.310 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:44.314+0000 7f2099c00700 1 -- 172.21.15.18:0/233716549 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2094114890 con 0x7f20940724f0 2024-04-12T21:10:44.311 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:44.315+0000 7f2099c00700 1 -- 172.21.15.18:0/233716549 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2094073c30 con 0x7f20940724f0 2024-04-12T21:10:44.311 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:44.316+0000 7f208aa00700 1 -- 172.21.15.18:0/233716549 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7f2084013590 con 0x7f20940724f0 2024-04-12T21:10:44.312 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:44.317+0000 7f208aa00700 1 --2- 172.21.15.18:0/233716549 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f207806ec10 0x7f20780710c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:10:44.313 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:44.317+0000 7f208aa00700 1 -- 172.21.15.18:0/233716549 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 6757+0+0 (secure 0 0 0) 0x7f208408fe00 con 0x7f20940724f0 2024-04-12T21:10:44.313 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:44.317+0000 7f208be00700 1 --2- 172.21.15.18:0/233716549 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f207806ec10 0x7f20780710c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:10:44.313 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:44.318+0000 7f208be00700 1 --2- 172.21.15.18:0/233716549 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f207806ec10 0x7f20780710c0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f2094115990 tx=0x7f207c009040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:10:44.318 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:44.322+0000 7f208aa00700 1 -- 172.21.15.18:0/233716549 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f2084058a80 con 0x7f20940724f0 2024-04-12T21:10:44.573 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:44.577+0000 7f2099c00700 1 -- 172.21.15.18:0/233716549 --> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f209404bed0 con 0x7f207806ec10 2024-04-12T21:10:44.574 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:44.579+0000 7f208aa00700 1 -- 172.21.15.18:0/233716549 <== mgr.34102 v2:172.21.15.18:6800/3812891919 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f209404bed0 con 0x7f207806ec10 2024-04-12T21:10:44.578 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:44.582+0000 7f2099c00700 1 -- 172.21.15.18:0/233716549 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f207806ec10 msgr2=0x7f20780710c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:10:44.578 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:44.582+0000 7f2099c00700 1 --2- 172.21.15.18:0/233716549 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f207806ec10 0x7f20780710c0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f2094115990 tx=0x7f207c009040 comp rx=0 tx=0).stop 2024-04-12T21:10:44.578 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:44.582+0000 7f2099c00700 1 -- 172.21.15.18:0/233716549 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f20940724f0 msgr2=0x7f209411a0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:10:44.578 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:44.582+0000 7f2099c00700 1 --2- 172.21.15.18:0/233716549 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f20940724f0 0x7f209411a0b0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f2084009180 tx=0x7f2084009e30 comp rx=0 tx=0).stop 2024-04-12T21:10:44.579 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:44.582+0000 7f2099c00700 1 -- 172.21.15.18:0/233716549 shutdown_connections 2024-04-12T21:10:44.579 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:44.582+0000 7f2099c00700 1 --2- 172.21.15.18:0/233716549 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f207806ec10 0x7f20780710c0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:10:44.579 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:44.582+0000 7f2099c00700 1 --2- 172.21.15.18:0/233716549 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f20940724f0 0x7f209411a0b0 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:10:44.579 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:44.582+0000 7f2099c00700 1 --2- 172.21.15.18:0/233716549 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2094072e00 0x7f209411a5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:10:44.579 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:44.582+0000 7f2099c00700 1 -- 172.21.15.18:0/233716549 >> 172.21.15.18:0/233716549 conn(0x7f20940faba0 msgr2=0x7f20940faf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:10:44.579 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:44.583+0000 7f2099c00700 1 -- 172.21.15.18:0/233716549 shutdown_connections 2024-04-12T21:10:44.579 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:44.583+0000 7f2099c00700 1 -- 172.21.15.18:0/233716549 wait complete. 2024-04-12T21:10:44.595 INFO:teuthology.orchestra.run.smithi018.stdout:true 2024-04-12T21:10:44.743 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:44.747+0000 7f290f200700 1 -- 172.21.15.18:0/1851060753 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2910070c10 msgr2=0x7f2910108fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:10:44.743 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:44.747+0000 7f290f200700 1 --2- 172.21.15.18:0/1851060753 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2910070c10 0x7f2910108fa0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f28fc004970 tx=0x7f28fc009e30 comp rx=0 tx=0).stop 2024-04-12T21:10:44.743 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:44.747+0000 7f290f200700 1 -- 172.21.15.18:0/1851060753 shutdown_connections 2024-04-12T21:10:44.743 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:44.747+0000 7f290f200700 1 --2- 172.21.15.18:0/1851060753 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2910070c10 0x7f2910108fa0 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:10:44.743 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:44.747+0000 7f290f200700 1 --2- 172.21.15.18:0/1851060753 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2910070300 0x7f29100706d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:10:44.743 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:44.747+0000 7f290f200700 1 -- 172.21.15.18:0/1851060753 >> 172.21.15.18:0/1851060753 conn(0x7f29100f87a0 msgr2=0x7f29100fabb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:10:44.743 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:44.747+0000 7f290f200700 1 -- 172.21.15.18:0/1851060753 shutdown_connections 2024-04-12T21:10:44.743 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:44.747+0000 7f290f200700 1 -- 172.21.15.18:0/1851060753 wait complete. 2024-04-12T21:10:44.744 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:44.748+0000 7f290f200700 1 Processor -- start 2024-04-12T21:10:44.744 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:44.748+0000 7f290f200700 1 -- start start 2024-04-12T21:10:44.744 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:44.748+0000 7f290f200700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2910070300 0x7f2910115b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:10:44.744 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:44.748+0000 7f290f200700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2910070c10 0x7f2910116080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:10:44.744 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:44.748+0000 7f290f200700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29101166d0 con 0x7f2910070300 2024-04-12T21:10:44.744 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:44.748+0000 7f290f200700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f291010fac0 con 0x7f2910070c10 2024-04-12T21:10:44.745 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:44.749+0000 7f290de00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2910070300 0x7f2910115b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:10:44.745 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:44.749+0000 7f290de00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2910070300 0x7f2910115b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:42812/0 (socket says 172.21.15.18:42812) 2024-04-12T21:10:44.745 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:44.749+0000 7f290de00700 1 -- 172.21.15.18:0/2350788743 learned_addr learned my addr 172.21.15.18:0/2350788743 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:10:44.746 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:44.749+0000 7f290de00700 1 -- 172.21.15.18:0/2350788743 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2910070c10 msgr2=0x7f2910116080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:10:44.746 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:44.749+0000 7f290d400700 1 --2- 172.21.15.18:0/2350788743 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2910070c10 0x7f2910116080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:10:44.746 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:44.749+0000 7f290de00700 1 --2- 172.21.15.18:0/2350788743 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2910070c10 0x7f2910116080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:10:44.746 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:44.749+0000 7f290de00700 1 -- 172.21.15.18:0/2350788743 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28fc005040 con 0x7f2910070300 2024-04-12T21:10:44.746 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:44.750+0000 7f290de00700 1 --2- 172.21.15.18:0/2350788743 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2910070300 0x7f2910115b40 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f290000eb40 tx=0x7f290000b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:10:44.746 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:44.750+0000 7f2906000700 1 -- 172.21.15.18:0/2350788743 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2900007cc0 con 0x7f2910070300 2024-04-12T21:10:44.746 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:44.750+0000 7f2906000700 1 -- 172.21.15.18:0/2350788743 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f290000f040 con 0x7f2910070300 2024-04-12T21:10:44.746 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:44.750+0000 7f2906000700 1 -- 172.21.15.18:0/2350788743 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2900009620 con 0x7f2910070300 2024-04-12T21:10:44.746 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:44.750+0000 7f290f200700 1 -- 172.21.15.18:0/2350788743 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f291010fd40 con 0x7f2910070300 2024-04-12T21:10:44.746 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:44.750+0000 7f290f200700 1 -- 172.21.15.18:0/2350788743 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2910110280 con 0x7f2910070300 2024-04-12T21:10:44.747 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:44.752+0000 7f290f200700 1 -- 172.21.15.18:0/2350788743 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2910071a40 con 0x7f2910070300 2024-04-12T21:10:44.748 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:44.753+0000 7f2906000700 1 -- 172.21.15.18:0/2350788743 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7f2900009780 con 0x7f2910070300 2024-04-12T21:10:44.750 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:44.754+0000 7f2906000700 1 --2- 172.21.15.18:0/2350788743 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f28f406ec10 0x7f28f40710c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:10:44.750 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:44.754+0000 7f2906000700 1 -- 172.21.15.18:0/2350788743 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 6757+0+0 (secure 0 0 0) 0x7f290008e650 con 0x7f2910070300 2024-04-12T21:10:44.750 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:44.755+0000 7f290d400700 1 --2- 172.21.15.18:0/2350788743 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f28f406ec10 0x7f28f40710c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:10:44.751 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:44.756+0000 7f290d400700 1 --2- 172.21.15.18:0/2350788743 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f28f406ec10 0x7f28f40710c0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f28fc005010 tx=0x7f28fc009e30 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:10:44.755 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:44.760+0000 7f2906000700 1 -- 172.21.15.18:0/2350788743 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f290005aed0 con 0x7f2910070300 2024-04-12T21:10:45.010 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:45.015+0000 7f290f200700 1 -- 172.21.15.18:0/2350788743 --> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f291010bfa0 con 0x7f28f406ec10 2024-04-12T21:10:45.012 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:45.017+0000 7f2906000700 1 -- 172.21.15.18:0/2350788743 <== mgr.34102 v2:172.21.15.18:6800/3812891919 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f291010bfa0 con 0x7f28f406ec10 2024-04-12T21:10:45.015 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:45.020+0000 7f290f200700 1 -- 172.21.15.18:0/2350788743 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f28f406ec10 msgr2=0x7f28f40710c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:10:45.016 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:45.020+0000 7f290f200700 1 --2- 172.21.15.18:0/2350788743 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f28f406ec10 0x7f28f40710c0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f28fc005010 tx=0x7f28fc009e30 comp rx=0 tx=0).stop 2024-04-12T21:10:45.016 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:45.020+0000 7f290f200700 1 -- 172.21.15.18:0/2350788743 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2910070300 msgr2=0x7f2910115b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:10:45.016 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:45.020+0000 7f290f200700 1 --2- 172.21.15.18:0/2350788743 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2910070300 0x7f2910115b40 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f290000eb40 tx=0x7f290000b040 comp rx=0 tx=0).stop 2024-04-12T21:10:45.016 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:45.020+0000 7f290f200700 1 -- 172.21.15.18:0/2350788743 shutdown_connections 2024-04-12T21:10:45.016 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:45.020+0000 7f290f200700 1 --2- 172.21.15.18:0/2350788743 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f28f406ec10 0x7f28f40710c0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:10:45.016 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:45.020+0000 7f290f200700 1 --2- 172.21.15.18:0/2350788743 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2910070300 0x7f2910115b40 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:10:45.017 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:45.020+0000 7f290f200700 1 --2- 172.21.15.18:0/2350788743 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2910070c10 0x7f2910116080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:10:45.017 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:45.020+0000 7f290f200700 1 -- 172.21.15.18:0/2350788743 >> 172.21.15.18:0/2350788743 conn(0x7f29100f87a0 msgr2=0x7f29101057f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:10:45.017 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:45.021+0000 7f290f200700 1 -- 172.21.15.18:0/2350788743 shutdown_connections 2024-04-12T21:10:45.017 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:45.021+0000 7f290f200700 1 -- 172.21.15.18:0/2350788743 wait complete. 2024-04-12T21:10:45.180 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:45.184+0000 7f14ca000700 1 -- 172.21.15.18:0/4145380624 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f14c4071f90 msgr2=0x7f14c4070360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:10:45.180 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:45.184+0000 7f14ca000700 1 --2- 172.21.15.18:0/4145380624 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f14c4071f90 0x7f14c4070360 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f14b4004970 tx=0x7f14b4009e30 comp rx=0 tx=0).stop 2024-04-12T21:10:45.180 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:45.184+0000 7f14ca000700 1 -- 172.21.15.18:0/4145380624 shutdown_connections 2024-04-12T21:10:45.180 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:45.184+0000 7f14ca000700 1 --2- 172.21.15.18:0/4145380624 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f14c40708a0 0x7f14c4070d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:10:45.180 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:45.184+0000 7f14ca000700 1 --2- 172.21.15.18:0/4145380624 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f14c4071f90 0x7f14c4070360 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:10:45.180 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:45.184+0000 7f14ca000700 1 -- 172.21.15.18:0/4145380624 >> 172.21.15.18:0/4145380624 conn(0x7f14c40f89b0 msgr2=0x7f14c40fadc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:10:45.180 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:45.184+0000 7f14ca000700 1 -- 172.21.15.18:0/4145380624 shutdown_connections 2024-04-12T21:10:45.180 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:45.184+0000 7f14ca000700 1 -- 172.21.15.18:0/4145380624 wait complete. 2024-04-12T21:10:45.181 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:45.185+0000 7f14ca000700 1 Processor -- start 2024-04-12T21:10:45.181 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:45.185+0000 7f14ca000700 1 -- start start 2024-04-12T21:10:45.181 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:45.186+0000 7f14ca000700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f14c40708a0 0x7f14c411a1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:10:45.181 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:45.186+0000 7f14ca000700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f14c4071f90 0x7f14c411a6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:10:45.181 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:45.186+0000 7f14ca000700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14c411ad30 con 0x7f14c40708a0 2024-04-12T21:10:45.181 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:45.186+0000 7f14ca000700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14c4114100 con 0x7f14c4071f90 2024-04-12T21:10:45.182 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:45.186+0000 7f14c3400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f14c40708a0 0x7f14c411a1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:10:45.182 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:45.186+0000 7f14bbe00700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f14c4071f90 0x7f14c411a6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:10:45.182 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:45.186+0000 7f14c3400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f14c40708a0 0x7f14c411a1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:42836/0 (socket says 172.21.15.18:42836) 2024-04-12T21:10:45.182 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:45.187+0000 7f14bbe00700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f14c4071f90 0x7f14c411a6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.18:54120/0 (socket says 172.21.15.18:54120) 2024-04-12T21:10:45.182 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:45.187+0000 7f14c3400700 1 -- 172.21.15.18:0/3601775339 learned_addr learned my addr 172.21.15.18:0/3601775339 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:10:45.183 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:45.187+0000 7f14c3400700 1 -- 172.21.15.18:0/3601775339 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f14c4071f90 msgr2=0x7f14c411a6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:10:45.183 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:45.187+0000 7f14c3400700 1 --2- 172.21.15.18:0/3601775339 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f14c4071f90 0x7f14c411a6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:10:45.183 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:45.187+0000 7f14c3400700 1 -- 172.21.15.18:0/3601775339 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14b4005040 con 0x7f14c40708a0 2024-04-12T21:10:45.183 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:45.188+0000 7f14c3400700 1 --2- 172.21.15.18:0/3601775339 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f14c40708a0 0x7f14c411a1a0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f14b4004f50 tx=0x7f14b4009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:10:45.184 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:45.189+0000 7f14bb400700 1 -- 172.21.15.18:0/3601775339 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f14b400e040 con 0x7f14c40708a0 2024-04-12T21:10:45.184 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:45.189+0000 7f14bb400700 1 -- 172.21.15.18:0/3601775339 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f14b400bc10 con 0x7f14c40708a0 2024-04-12T21:10:45.184 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:45.189+0000 7f14bb400700 1 -- 172.21.15.18:0/3601775339 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f14b401c3f0 con 0x7f14c40708a0 2024-04-12T21:10:45.184 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:45.189+0000 7f14ca000700 1 -- 172.21.15.18:0/3601775339 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f14c4114380 con 0x7f14c40708a0 2024-04-12T21:10:45.185 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:45.189+0000 7f14ca000700 1 -- 172.21.15.18:0/3601775339 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f14c41148c0 con 0x7f14c40708a0 2024-04-12T21:10:45.186 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:45.191+0000 7f14ca000700 1 -- 172.21.15.18:0/3601775339 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f14c4071aa0 con 0x7f14c40708a0 2024-04-12T21:10:45.187 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:45.192+0000 7f14bb400700 1 -- 172.21.15.18:0/3601775339 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7f14b4013420 con 0x7f14c40708a0 2024-04-12T21:10:45.188 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:45.193+0000 7f14bb400700 1 --2- 172.21.15.18:0/3601775339 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f14a806ec10 0x7f14a80710c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:10:45.188 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:45.193+0000 7f14bb400700 1 -- 172.21.15.18:0/3601775339 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 6757+0+0 (secure 0 0 0) 0x7f14b408f900 con 0x7f14c40708a0 2024-04-12T21:10:45.188 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:45.193+0000 7f14bbe00700 1 --2- 172.21.15.18:0/3601775339 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f14a806ec10 0x7f14a80710c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:10:45.189 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:45.194+0000 7f14bbe00700 1 --2- 172.21.15.18:0/3601775339 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f14a806ec10 0x7f14a80710c0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f14c4115a40 tx=0x7f14ac00b040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:10:45.193 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:45.198+0000 7f14bb400700 1 -- 172.21.15.18:0/3601775339 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f14b405c1f0 con 0x7f14c40708a0 2024-04-12T21:10:45.432 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:45.436+0000 7f14ca000700 1 -- 172.21.15.18:0/3601775339 --> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f14c404bf40 con 0x7f14a806ec10 2024-04-12T21:10:45.443 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:45.448+0000 7f14bb400700 1 -- 172.21.15.18:0/3601775339 <== mgr.34102 v2:172.21.15.18:6800/3812891919 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7f14c404bf40 con 0x7f14a806ec10 2024-04-12T21:10:45.444 INFO:teuthology.orchestra.run.smithi018.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-12T21:10:45.444 INFO:teuthology.orchestra.run.smithi018.stdout:alertmanager.smithi018 smithi018 *:9093,9094 running (11m) 3m ago 25m 14.7M - 0.25.0 c8568f914cd2 c332090068e8 2024-04-12T21:10:45.444 INFO:teuthology.orchestra.run.smithi018.stdout:crash.smithi018 smithi018 running (7m) 3m ago 26m 6665k - 19.0.0-2879-g324baf8d 8717720dc5e5 b7f80d1f7982 2024-04-12T21:10:45.444 INFO:teuthology.orchestra.run.smithi018.stdout:crash.smithi186 smithi186 running (7m) 3m ago 24m 6660k - 19.0.0-2879-g324baf8d 8717720dc5e5 a4f55c81b623 2024-04-12T21:10:45.444 INFO:teuthology.orchestra.run.smithi018.stdout:grafana.smithi018 smithi018 *:3000 running (10m) 3m ago 25m 74.8M - 9.4.12 2bacad6d85d8 b08e3bf176dd 2024-04-12T21:10:45.444 INFO:teuthology.orchestra.run.smithi018.stdout:mds.cephfs.smithi018.acqbsc smithi018 running (3m) 3m ago 18m 11.1M - 19.0.0-2879-g324baf8d 8717720dc5e5 021f99ab24ae 2024-04-12T21:10:45.445 INFO:teuthology.orchestra.run.smithi018.stdout:mds.cephfs.smithi018.szjluo smithi018 running (3m) 3m ago 18m 10.9M - 19.0.0-2879-g324baf8d 8717720dc5e5 0003117479d4 2024-04-12T21:10:45.445 INFO:teuthology.orchestra.run.smithi018.stdout:mds.cephfs.smithi186.jdllyn smithi186 running (18m) 3m ago 18m 15.6M - 17.2.7-827-gd3b5060a 21de9b7369ad 2b2bc46bd29a 2024-04-12T21:10:45.445 INFO:teuthology.orchestra.run.smithi018.stdout:mds.cephfs.smithi186.wjnafs smithi186 running (18m) 3m ago 18m 15.4M - 17.2.7-827-gd3b5060a 21de9b7369ad 74b115d6d6fd 2024-04-12T21:10:45.445 INFO:teuthology.orchestra.run.smithi018.stdout:mgr.smithi018.xjzwga smithi018 *:8443,9283,8765 running (14m) 3m ago 27m 509M - 19.0.0-2879-g324baf8d 8717720dc5e5 56088305eab7 2024-04-12T21:10:45.445 INFO:teuthology.orchestra.run.smithi018.stdout:mgr.smithi186.kqcfxk smithi186 *:8443,9283,8765 running (12m) 3m ago 24m 452M - 19.0.0-2879-g324baf8d 8717720dc5e5 09cd766e5e0d 2024-04-12T21:10:45.445 INFO:teuthology.orchestra.run.smithi018.stdout:mon.smithi018 smithi018 running (9m) 3m ago 27m 57.0M 2048M 19.0.0-2879-g324baf8d 8717720dc5e5 df3e201dfed6 2024-04-12T21:10:45.445 INFO:teuthology.orchestra.run.smithi018.stdout:mon.smithi186 smithi186 running (8m) 3m ago 24m 37.7M 2048M 19.0.0-2879-g324baf8d 8717720dc5e5 c77498da9b2b 2024-04-12T21:10:45.445 INFO:teuthology.orchestra.run.smithi018.stdout:node-exporter.smithi018 smithi018 *:9100 running (12m) 3m ago 26m 7780k - 1.5.0 0da6a335fe13 326c3d9c6847 2024-04-12T21:10:45.445 INFO:teuthology.orchestra.run.smithi018.stdout:node-exporter.smithi186 smithi186 *:9100 running (12m) 3m ago 24m 7826k - 1.5.0 0da6a335fe13 ec2112baeecf 2024-04-12T21:10:45.445 INFO:teuthology.orchestra.run.smithi018.stdout:osd.0 smithi018 running (7m) 3m ago 23m 121M 1559M 19.0.0-2879-g324baf8d 8717720dc5e5 60b1273ca047 2024-04-12T21:10:45.445 INFO:teuthology.orchestra.run.smithi018.stdout:osd.1 smithi018 running (6m) 3m ago 22m 91.3M 1559M 19.0.0-2879-g324baf8d 8717720dc5e5 170ce1e5fb8d 2024-04-12T21:10:45.445 INFO:teuthology.orchestra.run.smithi018.stdout:osd.2 smithi018 running (5m) 3m ago 22m 90.3M 1559M 19.0.0-2879-g324baf8d 8717720dc5e5 c2cbc6aa2839 2024-04-12T21:10:45.446 INFO:teuthology.orchestra.run.smithi018.stdout:osd.3 smithi186 running (5m) 3m ago 21m 96.6M 2581M 19.0.0-2879-g324baf8d 8717720dc5e5 0138e8262e4b 2024-04-12T21:10:45.446 INFO:teuthology.orchestra.run.smithi018.stdout:osd.4 smithi186 running (4m) 3m ago 21m 87.2M 2581M 19.0.0-2879-g324baf8d 8717720dc5e5 b20cb63b25c9 2024-04-12T21:10:45.446 INFO:teuthology.orchestra.run.smithi018.stdout:osd.5 smithi186 running (4m) 3m ago 20m 11.1M 2581M 19.0.0-2879-g324baf8d 8717720dc5e5 2513bea2610e 2024-04-12T21:10:45.446 INFO:teuthology.orchestra.run.smithi018.stdout:prometheus.smithi018 smithi018 *:9095 running (11m) 3m ago 25m 60.6M - 2.43.0 a07b618ecd1d f58841127e21 2024-04-12T21:10:45.447 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:45.451+0000 7f14ca000700 1 -- 172.21.15.18:0/3601775339 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f14a806ec10 msgr2=0x7f14a80710c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:10:45.447 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:45.451+0000 7f14ca000700 1 --2- 172.21.15.18:0/3601775339 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f14a806ec10 0x7f14a80710c0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f14c4115a40 tx=0x7f14ac00b040 comp rx=0 tx=0).stop 2024-04-12T21:10:45.447 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:45.451+0000 7f14ca000700 1 -- 172.21.15.18:0/3601775339 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f14c40708a0 msgr2=0x7f14c411a1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:10:45.447 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:45.451+0000 7f14ca000700 1 --2- 172.21.15.18:0/3601775339 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f14c40708a0 0x7f14c411a1a0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f14b4004f50 tx=0x7f14b4009e30 comp rx=0 tx=0).stop 2024-04-12T21:10:45.447 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:45.451+0000 7f14ca000700 1 -- 172.21.15.18:0/3601775339 shutdown_connections 2024-04-12T21:10:45.447 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:45.451+0000 7f14ca000700 1 --2- 172.21.15.18:0/3601775339 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f14a806ec10 0x7f14a80710c0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:10:45.447 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:45.451+0000 7f14ca000700 1 --2- 172.21.15.18:0/3601775339 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f14c40708a0 0x7f14c411a1a0 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:10:45.447 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:45.451+0000 7f14ca000700 1 --2- 172.21.15.18:0/3601775339 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f14c4071f90 0x7f14c411a6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:10:45.447 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:45.451+0000 7f14ca000700 1 -- 172.21.15.18:0/3601775339 >> 172.21.15.18:0/3601775339 conn(0x7f14c40f89b0 msgr2=0x7f14c40f9aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:10:45.448 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:45.451+0000 7f14ca000700 1 -- 172.21.15.18:0/3601775339 shutdown_connections 2024-04-12T21:10:45.448 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:45.451+0000 7f14ca000700 1 -- 172.21.15.18:0/3601775339 wait complete. 2024-04-12T21:10:45.610 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:45.614+0000 7fd603200700 1 -- 172.21.15.18:0/206140459 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd604104e60 msgr2=0x7fd604105230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:10:45.610 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:45.614+0000 7fd603200700 1 --2- 172.21.15.18:0/206140459 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd604104e60 0x7fd604105230 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fd5f4004940 tx=0x7fd5f4009e30 comp rx=0 tx=0).stop 2024-04-12T21:10:45.610 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:45.614+0000 7fd603200700 1 -- 172.21.15.18:0/206140459 shutdown_connections 2024-04-12T21:10:45.610 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:45.614+0000 7fd603200700 1 --2- 172.21.15.18:0/206140459 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd604105800 0x7fd6040fee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:10:45.610 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:45.614+0000 7fd603200700 1 --2- 172.21.15.18:0/206140459 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd604104e60 0x7fd604105230 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:10:45.610 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:45.614+0000 7fd603200700 1 -- 172.21.15.18:0/206140459 >> 172.21.15.18:0/206140459 conn(0x7fd6040fab10 msgr2=0x7fd6040fcef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:10:45.611 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:45.614+0000 7fd603200700 1 -- 172.21.15.18:0/206140459 shutdown_connections 2024-04-12T21:10:45.611 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:45.614+0000 7fd603200700 1 -- 172.21.15.18:0/206140459 wait complete. 2024-04-12T21:10:45.612 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:45.615+0000 7fd603200700 1 Processor -- start 2024-04-12T21:10:45.612 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:45.615+0000 7fd603200700 1 -- start start 2024-04-12T21:10:45.612 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:45.616+0000 7fd603200700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd604104e60 0x7fd604115c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:10:45.612 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:45.616+0000 7fd603200700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd604105800 0x7fd604116160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:10:45.612 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:45.616+0000 7fd603200700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6041167b0 con 0x7fd604105800 2024-04-12T21:10:45.612 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:45.616+0000 7fd603200700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd60410fba0 con 0x7fd604104e60 2024-04-12T21:10:45.612 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:45.616+0000 7fd601400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd604105800 0x7fd604116160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:10:45.612 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:45.616+0000 7fd601400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd604105800 0x7fd604116160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:42844/0 (socket says 172.21.15.18:42844) 2024-04-12T21:10:45.613 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:45.616+0000 7fd601400700 1 -- 172.21.15.18:0/2495591664 learned_addr learned my addr 172.21.15.18:0/2495591664 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:10:45.613 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:45.616+0000 7fd601e00700 1 --2- 172.21.15.18:0/2495591664 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd604104e60 0x7fd604115c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:10:45.613 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:45.617+0000 7fd601400700 1 -- 172.21.15.18:0/2495591664 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd604104e60 msgr2=0x7fd604115c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:10:45.613 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:45.617+0000 7fd601400700 1 --2- 172.21.15.18:0/2495591664 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd604104e60 0x7fd604115c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:10:45.614 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:45.617+0000 7fd601400700 1 -- 172.21.15.18:0/2495591664 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5f4005040 con 0x7fd604105800 2024-04-12T21:10:45.614 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:45.617+0000 7fd601400700 1 --2- 172.21.15.18:0/2495591664 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd604105800 0x7fd604116160 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fd5f0002a50 tx=0x7fd5f0004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:10:45.614 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:45.617+0000 7fd5fa000700 1 -- 172.21.15.18:0/2495591664 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd5f0007cc0 con 0x7fd604105800 2024-04-12T21:10:45.614 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:45.617+0000 7fd5fa000700 1 -- 172.21.15.18:0/2495591664 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd5f000f040 con 0x7fd604105800 2024-04-12T21:10:45.614 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:45.617+0000 7fd603200700 1 -- 172.21.15.18:0/2495591664 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd60410fe80 con 0x7fd604105800 2024-04-12T21:10:45.614 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:45.618+0000 7fd5fa000700 1 -- 172.21.15.18:0/2495591664 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd5f000daf0 con 0x7fd604105800 2024-04-12T21:10:45.614 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:45.618+0000 7fd603200700 1 -- 172.21.15.18:0/2495591664 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd604110440 con 0x7fd604105800 2024-04-12T21:10:45.614 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:45.619+0000 7fd603200700 1 -- 172.21.15.18:0/2495591664 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd6041005b0 con 0x7fd604105800 2024-04-12T21:10:45.615 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:45.620+0000 7fd5fa000700 1 -- 172.21.15.18:0/2495591664 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7fd5f0009810 con 0x7fd604105800 2024-04-12T21:10:45.616 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:45.621+0000 7fd5fa000700 1 --2- 172.21.15.18:0/2495591664 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fd5e806ec10 0x7fd5e80710c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:10:45.616 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:45.621+0000 7fd5fa000700 1 -- 172.21.15.18:0/2495591664 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 6757+0+0 (secure 0 0 0) 0x7fd5f008e660 con 0x7fd604105800 2024-04-12T21:10:45.617 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:45.621+0000 7fd601e00700 1 --2- 172.21.15.18:0/2495591664 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fd5e806ec10 0x7fd5e80710c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:10:45.617 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:45.622+0000 7fd601e00700 1 --2- 172.21.15.18:0/2495591664 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fd5e806ec10 0x7fd5e80710c0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fd5f400b3b0 tx=0x7fd5f4009e30 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:10:45.621 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:45.625+0000 7fd5fa000700 1 -- 172.21.15.18:0/2495591664 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7fd5f005aeb0 con 0x7fd604105800 2024-04-12T21:10:45.779 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:10:45 smithi186 ceph-mon[86379]: pgmap v328: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:10:45.779 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:10:45 smithi186 ceph-mon[86379]: from='client.34612 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:10:45.850 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:10:45 smithi018 ceph-mon[112590]: pgmap v328: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:10:45.850 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:10:45 smithi018 ceph-mon[112590]: from='client.34612 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:10:45.952 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:45.956+0000 7fd603200700 1 -- 172.21.15.18:0/2495591664 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fd60410bf90 con 0x7fd604105800 2024-04-12T21:10:45.952 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:45.957+0000 7fd5fa000700 1 -- 172.21.15.18:0/2495591664 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+837 (secure 0 0 0) 0x7fd5f005aa40 con 0x7fd604105800 2024-04-12T21:10:45.953 INFO:teuthology.orchestra.run.smithi018.stdout:{ 2024-04-12T21:10:45.953 INFO:teuthology.orchestra.run.smithi018.stdout: "mon": { 2024-04-12T21:10:45.953 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 19.0.0-2879-g324baf8d (324baf8ddb91dea46cac50cb535c306115791355) squid (dev)": 2 2024-04-12T21:10:45.953 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-12T21:10:45.953 INFO:teuthology.orchestra.run.smithi018.stdout: "mgr": { 2024-04-12T21:10:45.953 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 19.0.0-2879-g324baf8d (324baf8ddb91dea46cac50cb535c306115791355) squid (dev)": 2 2024-04-12T21:10:45.954 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-12T21:10:45.954 INFO:teuthology.orchestra.run.smithi018.stdout: "osd": { 2024-04-12T21:10:45.954 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 19.0.0-2879-g324baf8d (324baf8ddb91dea46cac50cb535c306115791355) squid (dev)": 6 2024-04-12T21:10:45.954 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-12T21:10:45.954 INFO:teuthology.orchestra.run.smithi018.stdout: "mds": { 2024-04-12T21:10:45.954 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 17.2.7-827-gd3b5060a (d3b5060a17449edfae9385cdf792afec109fca5d) quincy (stable)": 2, 2024-04-12T21:10:45.954 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 19.0.0-2879-g324baf8d (324baf8ddb91dea46cac50cb535c306115791355) squid (dev)": 2 2024-04-12T21:10:45.954 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-12T21:10:45.955 INFO:teuthology.orchestra.run.smithi018.stdout: "overall": { 2024-04-12T21:10:45.955 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 17.2.7-827-gd3b5060a (d3b5060a17449edfae9385cdf792afec109fca5d) quincy (stable)": 2, 2024-04-12T21:10:45.955 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 19.0.0-2879-g324baf8d (324baf8ddb91dea46cac50cb535c306115791355) squid (dev)": 12 2024-04-12T21:10:45.955 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-04-12T21:10:45.955 INFO:teuthology.orchestra.run.smithi018.stdout:} 2024-04-12T21:10:45.957 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:45.960+0000 7fd603200700 1 -- 172.21.15.18:0/2495591664 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fd5e806ec10 msgr2=0x7fd5e80710c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:10:45.957 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:45.960+0000 7fd603200700 1 --2- 172.21.15.18:0/2495591664 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fd5e806ec10 0x7fd5e80710c0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fd5f400b3b0 tx=0x7fd5f4009e30 comp rx=0 tx=0).stop 2024-04-12T21:10:45.957 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:45.960+0000 7fd603200700 1 -- 172.21.15.18:0/2495591664 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd604105800 msgr2=0x7fd604116160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:10:45.957 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:45.960+0000 7fd603200700 1 --2- 172.21.15.18:0/2495591664 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd604105800 0x7fd604116160 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fd5f0002a50 tx=0x7fd5f0004600 comp rx=0 tx=0).stop 2024-04-12T21:10:45.957 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:45.961+0000 7fd603200700 1 -- 172.21.15.18:0/2495591664 shutdown_connections 2024-04-12T21:10:45.957 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:45.961+0000 7fd603200700 1 --2- 172.21.15.18:0/2495591664 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fd5e806ec10 0x7fd5e80710c0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:10:45.957 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:45.961+0000 7fd603200700 1 --2- 172.21.15.18:0/2495591664 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd604104e60 0x7fd604115c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:10:45.958 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:45.961+0000 7fd603200700 1 --2- 172.21.15.18:0/2495591664 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd604105800 0x7fd604116160 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:10:45.958 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:45.961+0000 7fd603200700 1 -- 172.21.15.18:0/2495591664 >> 172.21.15.18:0/2495591664 conn(0x7fd6040fab10 msgr2=0x7fd6040fcb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:10:45.958 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:45.961+0000 7fd603200700 1 -- 172.21.15.18:0/2495591664 shutdown_connections 2024-04-12T21:10:45.958 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:45.961+0000 7fd603200700 1 -- 172.21.15.18:0/2495591664 wait complete. 2024-04-12T21:10:46.116 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:46.120+0000 7f9c0e200700 1 -- 172.21.15.18:0/3589869692 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9c080724f0 msgr2=0x7f9c080728c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:10:46.117 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:46.120+0000 7f9c0e200700 1 --2- 172.21.15.18:0/3589869692 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9c080724f0 0x7f9c080728c0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f9bfc004940 tx=0x7f9bfc009e30 comp rx=0 tx=0).stop 2024-04-12T21:10:46.117 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:46.120+0000 7f9c0e200700 1 -- 172.21.15.18:0/3589869692 shutdown_connections 2024-04-12T21:10:46.117 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:46.120+0000 7f9c0e200700 1 --2- 172.21.15.18:0/3589869692 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9c08072e00 0x7f9c0810d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:10:46.117 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:46.120+0000 7f9c0e200700 1 --2- 172.21.15.18:0/3589869692 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9c080724f0 0x7f9c080728c0 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:10:46.117 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:46.120+0000 7f9c0e200700 1 -- 172.21.15.18:0/3589869692 >> 172.21.15.18:0/3589869692 conn(0x7f9c080fab10 msgr2=0x7f9c080fcef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:10:46.117 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:46.121+0000 7f9c0e200700 1 -- 172.21.15.18:0/3589869692 shutdown_connections 2024-04-12T21:10:46.117 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:46.121+0000 7f9c0e200700 1 -- 172.21.15.18:0/3589869692 wait complete. 2024-04-12T21:10:46.119 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:46.121+0000 7f9c0e200700 1 Processor -- start 2024-04-12T21:10:46.119 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:46.122+0000 7f9c0e200700 1 -- start start 2024-04-12T21:10:46.119 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:46.122+0000 7f9c0e200700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9c080724f0 0x7f9c08119f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:10:46.119 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:46.122+0000 7f9c0e200700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9c08072e00 0x7f9c0811a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:10:46.119 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:46.122+0000 7f9c0e200700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c0811aad0 con 0x7f9c08072e00 2024-04-12T21:10:46.119 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:46.122+0000 7f9c0e200700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c08113ea0 con 0x7f9c080724f0 2024-04-12T21:10:46.119 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:46.122+0000 7f9c07e00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9c08072e00 0x7f9c0811a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:10:46.119 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:46.122+0000 7f9c07e00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9c08072e00 0x7f9c0811a480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:42874/0 (socket says 172.21.15.18:42874) 2024-04-12T21:10:46.119 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:46.122+0000 7f9c07e00700 1 -- 172.21.15.18:0/1515675650 learned_addr learned my addr 172.21.15.18:0/1515675650 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:10:46.120 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:46.123+0000 7f9c0ce00700 1 --2- 172.21.15.18:0/1515675650 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9c080724f0 0x7f9c08119f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:10:46.120 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:46.123+0000 7f9c07e00700 1 -- 172.21.15.18:0/1515675650 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9c080724f0 msgr2=0x7f9c08119f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:10:46.120 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:46.123+0000 7f9c07e00700 1 --2- 172.21.15.18:0/1515675650 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9c080724f0 0x7f9c08119f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:10:46.120 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:46.123+0000 7f9c07e00700 1 -- 172.21.15.18:0/1515675650 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9bfc005040 con 0x7f9c08072e00 2024-04-12T21:10:46.120 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:46.123+0000 7f9c07e00700 1 --2- 172.21.15.18:0/1515675650 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9c08072e00 0x7f9c0811a480 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f9bf4002a50 tx=0x7f9bf40046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:10:46.120 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:46.123+0000 7f9c04c00700 1 -- 172.21.15.18:0/1515675650 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9bf4007cc0 con 0x7f9c08072e00 2024-04-12T21:10:46.120 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:46.123+0000 7f9c0e200700 1 -- 172.21.15.18:0/1515675650 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9c08114180 con 0x7f9c08072e00 2024-04-12T21:10:46.120 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:46.123+0000 7f9c04c00700 1 -- 172.21.15.18:0/1515675650 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f9bf400f040 con 0x7f9c08072e00 2024-04-12T21:10:46.120 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:46.123+0000 7f9c04c00700 1 -- 172.21.15.18:0/1515675650 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9bf4009640 con 0x7f9c08072e00 2024-04-12T21:10:46.120 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:46.124+0000 7f9c0e200700 1 -- 172.21.15.18:0/1515675650 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9c08114740 con 0x7f9c08072e00 2024-04-12T21:10:46.121 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:46.125+0000 7f9c0e200700 1 -- 172.21.15.18:0/1515675650 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9c08073c30 con 0x7f9c08072e00 2024-04-12T21:10:46.121 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:46.126+0000 7f9c04c00700 1 -- 172.21.15.18:0/1515675650 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7f9bf40097a0 con 0x7f9c08072e00 2024-04-12T21:10:46.122 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:46.127+0000 7f9c04c00700 1 --2- 172.21.15.18:0/1515675650 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f9bf006ec00 0x7f9bf00710b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:10:46.123 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:46.127+0000 7f9c04c00700 1 -- 172.21.15.18:0/1515675650 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 6757+0+0 (secure 0 0 0) 0x7f9bf408e4c0 con 0x7f9c08072e00 2024-04-12T21:10:46.123 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:46.127+0000 7f9c0ce00700 1 --2- 172.21.15.18:0/1515675650 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f9bf006ec00 0x7f9bf00710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:10:46.123 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:46.128+0000 7f9c0ce00700 1 --2- 172.21.15.18:0/1515675650 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f9bf006ec00 0x7f9bf00710b0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f9bfc009fd0 tx=0x7f9bfc009e30 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:10:46.127 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:46.131+0000 7f9c04c00700 1 -- 172.21.15.18:0/1515675650 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f9bf405ac90 con 0x7f9c08072e00 2024-04-12T21:10:46.397 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:46.401+0000 7f9c0e200700 1 -- 172.21.15.18:0/1515675650 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f9c0804bed0 con 0x7f9c08072e00 2024-04-12T21:10:46.397 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:46.402+0000 7f9c04c00700 1 -- 172.21.15.18:0/1515675650 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 29 v29) v1 ==== 76+0+1872 (secure 0 0 0) 0x7f9bf405a820 con 0x7f9c08072e00 2024-04-12T21:10:46.399 INFO:teuthology.orchestra.run.smithi018.stdout:e29 2024-04-12T21:10:46.400 INFO:teuthology.orchestra.run.smithi018.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-12T21:10:46.400 INFO:teuthology.orchestra.run.smithi018.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-12T21:10:46.400 INFO:teuthology.orchestra.run.smithi018.stdout:legacy client fscid: 1 2024-04-12T21:10:46.400 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-12T21:10:46.400 INFO:teuthology.orchestra.run.smithi018.stdout:Filesystem 'cephfs' (1) 2024-04-12T21:10:46.400 INFO:teuthology.orchestra.run.smithi018.stdout:fs_name cephfs 2024-04-12T21:10:46.400 INFO:teuthology.orchestra.run.smithi018.stdout:epoch 24 2024-04-12T21:10:46.400 INFO:teuthology.orchestra.run.smithi018.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-04-12T21:10:46.400 INFO:teuthology.orchestra.run.smithi018.stdout:created 2024-04-12T20:51:57.362850+0000 2024-04-12T21:10:46.400 INFO:teuthology.orchestra.run.smithi018.stdout:modified 2024-04-12T21:06:59.717190+0000 2024-04-12T21:10:46.400 INFO:teuthology.orchestra.run.smithi018.stdout:tableserver 0 2024-04-12T21:10:46.400 INFO:teuthology.orchestra.run.smithi018.stdout:root 0 2024-04-12T21:10:46.400 INFO:teuthology.orchestra.run.smithi018.stdout:session_timeout 60 2024-04-12T21:10:46.400 INFO:teuthology.orchestra.run.smithi018.stdout:session_autoclose 300 2024-04-12T21:10:46.401 INFO:teuthology.orchestra.run.smithi018.stdout:max_file_size 1099511627776 2024-04-12T21:10:46.401 INFO:teuthology.orchestra.run.smithi018.stdout:max_xattr_size 65536 2024-04-12T21:10:46.401 INFO:teuthology.orchestra.run.smithi018.stdout:required_client_features {} 2024-04-12T21:10:46.401 INFO:teuthology.orchestra.run.smithi018.stdout:last_failure 0 2024-04-12T21:10:46.401 INFO:teuthology.orchestra.run.smithi018.stdout:last_failure_osd_epoch 100 2024-04-12T21:10:46.401 INFO:teuthology.orchestra.run.smithi018.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-12T21:10:46.401 INFO:teuthology.orchestra.run.smithi018.stdout:max_mds 2 2024-04-12T21:10:46.401 INFO:teuthology.orchestra.run.smithi018.stdout:in 0,1 2024-04-12T21:10:46.401 INFO:teuthology.orchestra.run.smithi018.stdout:up {} 2024-04-12T21:10:46.401 INFO:teuthology.orchestra.run.smithi018.stdout:failed 0,1 2024-04-12T21:10:46.401 INFO:teuthology.orchestra.run.smithi018.stdout:damaged 2024-04-12T21:10:46.401 INFO:teuthology.orchestra.run.smithi018.stdout:stopped 2024-04-12T21:10:46.401 INFO:teuthology.orchestra.run.smithi018.stdout:data_pools [3] 2024-04-12T21:10:46.401 INFO:teuthology.orchestra.run.smithi018.stdout:metadata_pool 2 2024-04-12T21:10:46.401 INFO:teuthology.orchestra.run.smithi018.stdout:inline_data disabled 2024-04-12T21:10:46.402 INFO:teuthology.orchestra.run.smithi018.stdout:balancer 2024-04-12T21:10:46.402 INFO:teuthology.orchestra.run.smithi018.stdout:bal_rank_mask -1 2024-04-12T21:10:46.402 INFO:teuthology.orchestra.run.smithi018.stdout:standby_count_wanted 1 2024-04-12T21:10:46.402 INFO:teuthology.orchestra.run.smithi018.stdout:qdb_cluster leader: 0 members: 2024-04-12T21:10:46.402 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-12T21:10:46.402 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-12T21:10:46.402 INFO:teuthology.orchestra.run.smithi018.stdout:Standby daemons: 2024-04-12T21:10:46.402 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-12T21:10:46.402 INFO:teuthology.orchestra.run.smithi018.stdout:[mds.cephfs.smithi186.wjnafs{-1:24277} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.186:6826/438347207,v1:172.21.15.186:6827/438347207] compat {c=[1],r=[1],i=[7ff]}] 2024-04-12T21:10:46.402 INFO:teuthology.orchestra.run.smithi018.stdout:[mds.cephfs.smithi186.jdllyn{-1:24281} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.186:6824/1464359952,v1:172.21.15.186:6825/1464359952] compat {c=[1],r=[1],i=[7ff]}] 2024-04-12T21:10:46.402 INFO:teuthology.orchestra.run.smithi018.stdout:[mds.cephfs.smithi018.acqbsc{-1:34436} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.18:6828/755179764,v1:172.21.15.18:6829/755179764] compat {c=[1],r=[1],i=[1fff]}] 2024-04-12T21:10:46.402 INFO:teuthology.orchestra.run.smithi018.stdout:[mds.cephfs.smithi018.szjluo{-1:34468} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.18:6826/2407992653,v1:172.21.15.18:6827/2407992653] compat {c=[1],r=[1],i=[1fff]}] 2024-04-12T21:10:46.403 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:46.405+0000 7f9c0e200700 1 -- 172.21.15.18:0/1515675650 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f9bf006ec00 msgr2=0x7f9bf00710b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:10:46.403 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:46.405+0000 7f9c0e200700 1 --2- 172.21.15.18:0/1515675650 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f9bf006ec00 0x7f9bf00710b0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f9bfc009fd0 tx=0x7f9bfc009e30 comp rx=0 tx=0).stop 2024-04-12T21:10:46.403 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:46.405+0000 7f9c0e200700 1 -- 172.21.15.18:0/1515675650 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9c08072e00 msgr2=0x7f9c0811a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:10:46.403 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:46.405+0000 7f9c0e200700 1 --2- 172.21.15.18:0/1515675650 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9c08072e00 0x7f9c0811a480 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f9bf4002a50 tx=0x7f9bf40046b0 comp rx=0 tx=0).stop 2024-04-12T21:10:46.403 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:46.405+0000 7f9c0e200700 1 -- 172.21.15.18:0/1515675650 shutdown_connections 2024-04-12T21:10:46.403 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:46.405+0000 7f9c0e200700 1 --2- 172.21.15.18:0/1515675650 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f9bf006ec00 0x7f9bf00710b0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:10:46.403 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:46.405+0000 7f9c0e200700 1 --2- 172.21.15.18:0/1515675650 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9c080724f0 0x7f9c08119f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:10:46.404 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:46.405+0000 7f9c0e200700 1 --2- 172.21.15.18:0/1515675650 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9c08072e00 0x7f9c0811a480 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:10:46.404 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:46.405+0000 7f9c0e200700 1 -- 172.21.15.18:0/1515675650 >> 172.21.15.18:0/1515675650 conn(0x7f9c080fab10 msgr2=0x7f9c080faee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:10:46.404 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:46.405+0000 7f9c0e200700 1 -- 172.21.15.18:0/1515675650 shutdown_connections 2024-04-12T21:10:46.404 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:46.405+0000 7f9c0e200700 1 -- 172.21.15.18:0/1515675650 wait complete. 2024-04-12T21:10:46.404 INFO:teuthology.orchestra.run.smithi018.stderr:dumped fsmap epoch 29 2024-04-12T21:10:46.550 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:46.554+0000 7fdd51c00700 1 -- 172.21.15.18:0/3310179648 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fdd4c109c40 msgr2=0x7fdd4c1032b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:10:46.550 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:46.554+0000 7fdd51c00700 1 --2- 172.21.15.18:0/3310179648 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fdd4c109c40 0x7fdd4c1032b0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fdd38004940 tx=0x7fdd38009e30 comp rx=0 tx=0).stop 2024-04-12T21:10:46.550 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:46.554+0000 7fdd51c00700 1 -- 172.21.15.18:0/3310179648 shutdown_connections 2024-04-12T21:10:46.550 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:46.554+0000 7fdd51c00700 1 --2- 172.21.15.18:0/3310179648 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fdd4c109c40 0x7fdd4c1032b0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:10:46.550 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:46.554+0000 7fdd51c00700 1 --2- 172.21.15.18:0/3310179648 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdd4c1092a0 0x7fdd4c109670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:10:46.550 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:46.554+0000 7fdd51c00700 1 -- 172.21.15.18:0/3310179648 >> 172.21.15.18:0/3310179648 conn(0x7fdd4c072a30 msgr2=0x7fdd4c074e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:10:46.550 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:46.554+0000 7fdd51c00700 1 -- 172.21.15.18:0/3310179648 shutdown_connections 2024-04-12T21:10:46.550 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:46.554+0000 7fdd51c00700 1 -- 172.21.15.18:0/3310179648 wait complete. 2024-04-12T21:10:46.551 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:46.555+0000 7fdd51c00700 1 Processor -- start 2024-04-12T21:10:46.551 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:46.556+0000 7fdd51c00700 1 -- start start 2024-04-12T21:10:46.552 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:46.556+0000 7fdd51c00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fdd4c1092a0 0x7fdd4c119fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:10:46.552 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:46.556+0000 7fdd51c00700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdd4c109c40 0x7fdd4c11a500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:10:46.552 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:46.556+0000 7fdd51c00700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd4c11ab50 con 0x7fdd4c1092a0 2024-04-12T21:10:46.552 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:46.556+0000 7fdd51c00700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd4c113f20 con 0x7fdd4c109c40 2024-04-12T21:10:46.552 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:46.556+0000 7fdd4be00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fdd4c1092a0 0x7fdd4c119fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:10:46.552 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:46.556+0000 7fdd4be00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fdd4c1092a0 0x7fdd4c119fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:42902/0 (socket says 172.21.15.18:42902) 2024-04-12T21:10:46.552 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:46.556+0000 7fdd4be00700 1 -- 172.21.15.18:0/2397454711 learned_addr learned my addr 172.21.15.18:0/2397454711 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:10:46.553 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:46.557+0000 7fdd4b400700 1 --2- 172.21.15.18:0/2397454711 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdd4c109c40 0x7fdd4c11a500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:10:46.553 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:46.557+0000 7fdd4be00700 1 -- 172.21.15.18:0/2397454711 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdd4c109c40 msgr2=0x7fdd4c11a500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:10:46.553 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:46.557+0000 7fdd4be00700 1 --2- 172.21.15.18:0/2397454711 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdd4c109c40 0x7fdd4c11a500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:10:46.553 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:46.557+0000 7fdd4be00700 1 -- 172.21.15.18:0/2397454711 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd38005040 con 0x7fdd4c1092a0 2024-04-12T21:10:46.553 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:46.557+0000 7fdd4be00700 1 --2- 172.21.15.18:0/2397454711 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fdd4c1092a0 0x7fdd4c119fc0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fdd3c00c8c0 tx=0x7fdd3c0044c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:10:46.554 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:46.558+0000 7fdd43e00700 1 -- 172.21.15.18:0/2397454711 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdd3c00a980 con 0x7fdd4c1092a0 2024-04-12T21:10:46.554 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:46.558+0000 7fdd43e00700 1 -- 172.21.15.18:0/2397454711 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fdd3c00ce60 con 0x7fdd4c1092a0 2024-04-12T21:10:46.554 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:46.558+0000 7fdd43e00700 1 -- 172.21.15.18:0/2397454711 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdd3c0115e0 con 0x7fdd4c1092a0 2024-04-12T21:10:46.554 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:46.558+0000 7fdd51c00700 1 -- 172.21.15.18:0/2397454711 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdd4c114200 con 0x7fdd4c1092a0 2024-04-12T21:10:46.554 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:46.558+0000 7fdd51c00700 1 -- 172.21.15.18:0/2397454711 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdd4c1147c0 con 0x7fdd4c1092a0 2024-04-12T21:10:46.554 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:46.559+0000 7fdd51c00700 1 -- 172.21.15.18:0/2397454711 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdd4c1049f0 con 0x7fdd4c1092a0 2024-04-12T21:10:46.555 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:46.560+0000 7fdd43e00700 1 -- 172.21.15.18:0/2397454711 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7fdd3c012400 con 0x7fdd4c1092a0 2024-04-12T21:10:46.556 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:46.561+0000 7fdd43e00700 1 --2- 172.21.15.18:0/2397454711 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fdd3006ec00 0x7fdd300710b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:10:46.556 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:46.561+0000 7fdd43e00700 1 -- 172.21.15.18:0/2397454711 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 6757+0+0 (secure 0 0 0) 0x7fdd3c08ea90 con 0x7fdd4c1092a0 2024-04-12T21:10:46.556 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:46.561+0000 7fdd4b400700 1 --2- 172.21.15.18:0/2397454711 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fdd3006ec00 0x7fdd300710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:10:46.557 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:46.562+0000 7fdd4b400700 1 --2- 172.21.15.18:0/2397454711 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fdd3006ec00 0x7fdd300710b0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fdd3800b3b0 tx=0x7fdd38009e30 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:10:46.561 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:46.565+0000 7fdd43e00700 1 -- 172.21.15.18:0/2397454711 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7fdd3c05b2e0 con 0x7fdd4c1092a0 2024-04-12T21:10:46.779 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:10:46 smithi186 ceph-mon[86379]: from='client.34616 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:10:46.779 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:10:46 smithi186 ceph-mon[86379]: from='client.34620 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:10:46.779 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:10:46 smithi186 ceph-mon[86379]: from='client.? 172.21.15.18:0/2495591664' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T21:10:46.779 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:10:46 smithi186 ceph-mon[86379]: from='client.? 172.21.15.18:0/1515675650' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-12T21:10:46.805 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:46.809+0000 7fdd51c00700 1 -- 172.21.15.18:0/2397454711 --> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fdd4c04bed0 con 0x7fdd3006ec00 2024-04-12T21:10:46.805 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:10:46 smithi018 ceph-mon[112590]: from='client.34616 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:10:46.805 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:10:46 smithi018 ceph-mon[112590]: from='client.34620 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:10:46.805 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:10:46 smithi018 ceph-mon[112590]: from='client.? 172.21.15.18:0/2495591664' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T21:10:46.805 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:10:46 smithi018 ceph-mon[112590]: from='client.? 172.21.15.18:0/1515675650' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-12T21:10:46.806 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:46.811+0000 7fdd43e00700 1 -- 172.21.15.18:0/2397454711 <== mgr.34102 v2:172.21.15.18:6800/3812891919 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7fdd4c04bed0 con 0x7fdd3006ec00 2024-04-12T21:10:46.807 INFO:teuthology.orchestra.run.smithi018.stdout:{ 2024-04-12T21:10:46.807 INFO:teuthology.orchestra.run.smithi018.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:f4452def3be4d350f4a8e5feaf7ebb1511be9298fe68478ba7a2d6fbd7d69a63", 2024-04-12T21:10:46.807 INFO:teuthology.orchestra.run.smithi018.stdout: "in_progress": true, 2024-04-12T21:10:46.807 INFO:teuthology.orchestra.run.smithi018.stdout: "which": "Upgrading all daemon types on all hosts", 2024-04-12T21:10:46.807 INFO:teuthology.orchestra.run.smithi018.stdout: "services_complete": [ 2024-04-12T21:10:46.807 INFO:teuthology.orchestra.run.smithi018.stdout: "crash", 2024-04-12T21:10:46.808 INFO:teuthology.orchestra.run.smithi018.stdout: "osd", 2024-04-12T21:10:46.808 INFO:teuthology.orchestra.run.smithi018.stdout: "mon", 2024-04-12T21:10:46.808 INFO:teuthology.orchestra.run.smithi018.stdout: "mgr" 2024-04-12T21:10:46.808 INFO:teuthology.orchestra.run.smithi018.stdout: ], 2024-04-12T21:10:46.808 INFO:teuthology.orchestra.run.smithi018.stdout: "progress": "14/21 daemons upgraded", 2024-04-12T21:10:46.808 INFO:teuthology.orchestra.run.smithi018.stdout: "message": "Currently upgrading mds daemons", 2024-04-12T21:10:46.808 INFO:teuthology.orchestra.run.smithi018.stdout: "is_paused": false 2024-04-12T21:10:46.808 INFO:teuthology.orchestra.run.smithi018.stdout:} 2024-04-12T21:10:46.811 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:46.815+0000 7fdd51c00700 1 -- 172.21.15.18:0/2397454711 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fdd3006ec00 msgr2=0x7fdd300710b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:10:46.811 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:46.815+0000 7fdd51c00700 1 --2- 172.21.15.18:0/2397454711 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fdd3006ec00 0x7fdd300710b0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fdd3800b3b0 tx=0x7fdd38009e30 comp rx=0 tx=0).stop 2024-04-12T21:10:46.811 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:46.815+0000 7fdd51c00700 1 -- 172.21.15.18:0/2397454711 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fdd4c1092a0 msgr2=0x7fdd4c119fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:10:46.811 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:46.815+0000 7fdd51c00700 1 --2- 172.21.15.18:0/2397454711 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fdd4c1092a0 0x7fdd4c119fc0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fdd3c00c8c0 tx=0x7fdd3c0044c0 comp rx=0 tx=0).stop 2024-04-12T21:10:46.811 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:46.815+0000 7fdd51c00700 1 -- 172.21.15.18:0/2397454711 shutdown_connections 2024-04-12T21:10:46.811 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:46.815+0000 7fdd51c00700 1 --2- 172.21.15.18:0/2397454711 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fdd3006ec00 0x7fdd300710b0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:10:46.811 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:46.815+0000 7fdd51c00700 1 --2- 172.21.15.18:0/2397454711 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fdd4c1092a0 0x7fdd4c119fc0 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:10:46.812 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:46.815+0000 7fdd51c00700 1 --2- 172.21.15.18:0/2397454711 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdd4c109c40 0x7fdd4c11a500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:10:46.812 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:46.815+0000 7fdd51c00700 1 -- 172.21.15.18:0/2397454711 >> 172.21.15.18:0/2397454711 conn(0x7fdd4c072a30 msgr2=0x7fdd4c074910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:10:46.812 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:46.816+0000 7fdd51c00700 1 -- 172.21.15.18:0/2397454711 shutdown_connections 2024-04-12T21:10:46.812 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:46.816+0000 7fdd51c00700 1 -- 172.21.15.18:0/2397454711 wait complete. 2024-04-12T21:10:46.968 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:46.973+0000 7f3666c00700 1 -- 172.21.15.18:0/3588216284 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f36600724f0 msgr2=0x7f36600728c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:10:46.969 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:46.973+0000 7f3666c00700 1 --2- 172.21.15.18:0/3588216284 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f36600724f0 0x7f36600728c0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f3650004940 tx=0x7f3650009e30 comp rx=0 tx=0).stop 2024-04-12T21:10:46.969 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:46.973+0000 7f3666c00700 1 -- 172.21.15.18:0/3588216284 shutdown_connections 2024-04-12T21:10:46.969 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:46.973+0000 7f3666c00700 1 --2- 172.21.15.18:0/3588216284 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3660072e00 0x7f366010d770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:10:46.969 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:46.973+0000 7f3666c00700 1 --2- 172.21.15.18:0/3588216284 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f36600724f0 0x7f36600728c0 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:10:46.969 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:46.973+0000 7f3666c00700 1 -- 172.21.15.18:0/3588216284 >> 172.21.15.18:0/3588216284 conn(0x7f36600fabd0 msgr2=0x7f36600fcfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:10:46.969 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:46.973+0000 7f3666c00700 1 -- 172.21.15.18:0/3588216284 shutdown_connections 2024-04-12T21:10:46.969 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:46.973+0000 7f3666c00700 1 -- 172.21.15.18:0/3588216284 wait complete. 2024-04-12T21:10:46.970 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:46.974+0000 7f3666c00700 1 Processor -- start 2024-04-12T21:10:46.970 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:46.974+0000 7f3666c00700 1 -- start start 2024-04-12T21:10:46.970 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:46.975+0000 7f3666c00700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f36600724f0 0x7f366011a120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:10:46.970 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:46.975+0000 7f3666c00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3660072e00 0x7f366011a660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:10:46.970 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:46.975+0000 7f3666c00700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f366011acb0 con 0x7f3660072e00 2024-04-12T21:10:46.970 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:46.975+0000 7f3666c00700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36601140c0 con 0x7f36600724f0 2024-04-12T21:10:46.972 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:46.975+0000 7f365f400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3660072e00 0x7f366011a660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:10:46.972 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:46.975+0000 7f365f400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3660072e00 0x7f366011a660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:42920/0 (socket says 172.21.15.18:42920) 2024-04-12T21:10:46.972 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:46.975+0000 7f365f400700 1 -- 172.21.15.18:0/1859880207 learned_addr learned my addr 172.21.15.18:0/1859880207 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:10:46.972 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:46.975+0000 7f365fe00700 1 --2- 172.21.15.18:0/1859880207 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f36600724f0 0x7f366011a120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:10:46.972 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:46.976+0000 7f365f400700 1 -- 172.21.15.18:0/1859880207 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f36600724f0 msgr2=0x7f366011a120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:10:46.972 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:46.976+0000 7f365f400700 1 --2- 172.21.15.18:0/1859880207 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f36600724f0 0x7f366011a120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:10:46.972 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:46.976+0000 7f365f400700 1 -- 172.21.15.18:0/1859880207 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3650005040 con 0x7f3660072e00 2024-04-12T21:10:46.972 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:46.976+0000 7f365f400700 1 --2- 172.21.15.18:0/1859880207 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3660072e00 0x7f366011a660 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f364c00c810 tx=0x7f364c004470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:10:46.972 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:46.976+0000 7f3657e00700 1 -- 172.21.15.18:0/1859880207 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f364c012810 con 0x7f3660072e00 2024-04-12T21:10:46.972 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:46.976+0000 7f3657e00700 1 -- 172.21.15.18:0/1859880207 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f364c012e40 con 0x7f3660072e00 2024-04-12T21:10:46.973 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:46.976+0000 7f3666c00700 1 -- 172.21.15.18:0/1859880207 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f36601143a0 con 0x7f3660072e00 2024-04-12T21:10:46.973 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:46.976+0000 7f3666c00700 1 -- 172.21.15.18:0/1859880207 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3660114960 con 0x7f3660072e00 2024-04-12T21:10:46.973 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:46.976+0000 7f3657e00700 1 -- 172.21.15.18:0/1859880207 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f364c00a370 con 0x7f3660072e00 2024-04-12T21:10:46.973 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:46.978+0000 7f3666c00700 1 -- 172.21.15.18:0/1859880207 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f36400052b0 con 0x7f3660072e00 2024-04-12T21:10:46.974 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:46.979+0000 7f3657e00700 1 -- 172.21.15.18:0/1859880207 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7f364c012970 con 0x7f3660072e00 2024-04-12T21:10:46.975 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:46.980+0000 7f3657e00700 1 --2- 172.21.15.18:0/1859880207 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f364406ec10 0x7f36440710c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:10:46.976 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:46.980+0000 7f3657e00700 1 -- 172.21.15.18:0/1859880207 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 6757+0+0 (secure 0 0 0) 0x7f364c08ebc0 con 0x7f3660072e00 2024-04-12T21:10:46.976 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:46.980+0000 7f365fe00700 1 --2- 172.21.15.18:0/1859880207 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f364406ec10 0x7f36440710c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:10:46.976 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:46.981+0000 7f365fe00700 1 --2- 172.21.15.18:0/1859880207 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f364406ec10 0x7f36440710c0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f3650009fd0 tx=0x7f3650009e30 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:10:46.981 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:46.985+0000 7f3657e00700 1 -- 172.21.15.18:0/1859880207 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f364c0578e0 con 0x7f3660072e00 2024-04-12T21:10:47.310 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:47.314+0000 7f3666c00700 1 -- 172.21.15.18:0/1859880207 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f3640005560 con 0x7f3660072e00 2024-04-12T21:10:47.310 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:47.315+0000 7f3657e00700 1 -- 172.21.15.18:0/1859880207 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+233 (secure 0 0 0) 0x7f364c016100 con 0x7f3660072e00 2024-04-12T21:10:47.310 INFO:teuthology.orchestra.run.smithi018.stdout:HEALTH_ERR 1 filesystem is degraded; 1 filesystem is offline 2024-04-12T21:10:47.310 INFO:teuthology.orchestra.run.smithi018.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-04-12T21:10:47.311 INFO:teuthology.orchestra.run.smithi018.stdout: fs cephfs is degraded 2024-04-12T21:10:47.311 INFO:teuthology.orchestra.run.smithi018.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-04-12T21:10:47.311 INFO:teuthology.orchestra.run.smithi018.stdout: fs cephfs is offline because no MDS is active for it. 2024-04-12T21:10:47.313 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:47.317+0000 7f3666c00700 1 -- 172.21.15.18:0/1859880207 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f364406ec10 msgr2=0x7f36440710c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:10:47.313 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:47.317+0000 7f3666c00700 1 --2- 172.21.15.18:0/1859880207 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f364406ec10 0x7f36440710c0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f3650009fd0 tx=0x7f3650009e30 comp rx=0 tx=0).stop 2024-04-12T21:10:47.313 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:47.318+0000 7f3666c00700 1 -- 172.21.15.18:0/1859880207 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3660072e00 msgr2=0x7f366011a660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:10:47.313 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:47.318+0000 7f3666c00700 1 --2- 172.21.15.18:0/1859880207 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3660072e00 0x7f366011a660 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f364c00c810 tx=0x7f364c004470 comp rx=0 tx=0).stop 2024-04-12T21:10:47.314 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:47.318+0000 7f3666c00700 1 -- 172.21.15.18:0/1859880207 shutdown_connections 2024-04-12T21:10:47.314 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:47.318+0000 7f3666c00700 1 --2- 172.21.15.18:0/1859880207 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f364406ec10 0x7f36440710c0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:10:47.314 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:47.318+0000 7f3666c00700 1 --2- 172.21.15.18:0/1859880207 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f36600724f0 0x7f366011a120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:10:47.314 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:47.318+0000 7f3666c00700 1 --2- 172.21.15.18:0/1859880207 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3660072e00 0x7f366011a660 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:10:47.314 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:47.318+0000 7f3666c00700 1 -- 172.21.15.18:0/1859880207 >> 172.21.15.18:0/1859880207 conn(0x7f36600fabd0 msgr2=0x7f36600fafa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:10:47.314 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:47.318+0000 7f3666c00700 1 -- 172.21.15.18:0/1859880207 shutdown_connections 2024-04-12T21:10:47.314 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:10:47.318+0000 7f3666c00700 1 -- 172.21.15.18:0/1859880207 wait complete. 2024-04-12T21:10:47.601 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:10:47 smithi018 ceph-mon[112590]: pgmap v329: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:10:47.601 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:10:47 smithi018 ceph-mon[112590]: from='client.? 172.21.15.18:0/1859880207' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-12T21:10:47.779 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:10:47 smithi186 ceph-mon[86379]: pgmap v329: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:10:47.779 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:10:47 smithi186 ceph-mon[86379]: from='client.? 172.21.15.18:0/1859880207' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-12T21:10:48.850 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:10:48 smithi018 ceph-mon[112590]: from='client.34632 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:10:49.029 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:10:48 smithi186 ceph-mon[86379]: from='client.34632 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:10:49.850 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:10:49 smithi018 ceph-mon[112590]: pgmap v330: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:10:50.029 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:10:49 smithi186 ceph-mon[86379]: pgmap v330: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:10:51.850 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:10:51 smithi018 ceph-mon[112590]: pgmap v331: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:10:52.029 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:10:51 smithi186 ceph-mon[86379]: pgmap v331: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:10:53.850 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:10:53 smithi018 ceph-mon[112590]: pgmap v332: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:10:54.029 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:10:53 smithi186 ceph-mon[86379]: pgmap v332: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:10:55.850 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:10:55 smithi018 ceph-mon[112590]: pgmap v333: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:10:55.850 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:10:55 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T21:10:56.029 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:10:55 smithi186 ceph-mon[86379]: pgmap v333: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:10:56.029 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:10:55 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T21:10:57.850 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:10:57 smithi018 ceph-mon[112590]: pgmap v334: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:10:58.029 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:10:57 smithi186 ceph-mon[86379]: pgmap v334: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:10:59.850 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:10:59 smithi018 ceph-mon[112590]: pgmap v335: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:11:00.029 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:10:59 smithi186 ceph-mon[86379]: pgmap v335: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:11:01.850 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:11:01 smithi018 ceph-mon[112590]: pgmap v336: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:11:02.029 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:11:01 smithi186 ceph-mon[86379]: pgmap v336: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:11:03.850 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:11:03 smithi018 ceph-mon[112590]: pgmap v337: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:11:04.029 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:11:03 smithi186 ceph-mon[86379]: pgmap v337: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:11:05.850 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:11:05 smithi018 ceph-mon[112590]: pgmap v338: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:11:06.029 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:11:05 smithi186 ceph-mon[86379]: pgmap v338: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:11:07.850 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:11:07 smithi018 ceph-mon[112590]: pgmap v339: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:11:08.029 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:11:07 smithi186 ceph-mon[86379]: pgmap v339: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:11:09.850 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:11:09 smithi018 ceph-mon[112590]: pgmap v340: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:11:10.029 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:11:09 smithi186 ceph-mon[86379]: pgmap v340: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:11:11.029 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:11:10 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T21:11:11.100 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:11:10 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T21:11:12.029 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:11:11 smithi186 ceph-mon[86379]: pgmap v341: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:11:12.100 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:11:11 smithi018 ceph-mon[112590]: pgmap v341: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:11:13.029 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:11:12 smithi186 ceph-mon[86379]: pgmap v342: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:11:13.100 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:11:12 smithi018 ceph-mon[112590]: pgmap v342: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:11:15.350 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:11:15 smithi018 ceph-mon[112590]: pgmap v343: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:11:15.529 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:11:15 smithi186 ceph-mon[86379]: pgmap v343: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:11:17.350 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:11:17 smithi018 ceph-mon[112590]: pgmap v344: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:11:17.485 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:17.488+0000 7feb0ac00700 1 -- 172.21.15.18:0/76748723 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7feb041059c0 msgr2=0x7feb040ff080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:11:17.485 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:17.488+0000 7feb0ac00700 1 --2- 172.21.15.18:0/76748723 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7feb041059c0 0x7feb040ff080 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7feaf0004940 tx=0x7feaf0009e30 comp rx=0 tx=0).stop 2024-04-12T21:11:17.485 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:17.488+0000 7feb0ac00700 1 -- 172.21.15.18:0/76748723 shutdown_connections 2024-04-12T21:11:17.485 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:17.488+0000 7feb0ac00700 1 --2- 172.21.15.18:0/76748723 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7feb041059c0 0x7feb040ff080 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:11:17.486 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:17.488+0000 7feb0ac00700 1 --2- 172.21.15.18:0/76748723 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7feb04105020 0x7feb041053f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:11:17.486 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:17.488+0000 7feb0ac00700 1 -- 172.21.15.18:0/76748723 >> 172.21.15.18:0/76748723 conn(0x7feb040faba0 msgr2=0x7feb040fcfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:11:17.486 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:17.489+0000 7feb0ac00700 1 -- 172.21.15.18:0/76748723 shutdown_connections 2024-04-12T21:11:17.486 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:17.489+0000 7feb0ac00700 1 -- 172.21.15.18:0/76748723 wait complete. 2024-04-12T21:11:17.486 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:17.490+0000 7feb0ac00700 1 Processor -- start 2024-04-12T21:11:17.487 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:17.490+0000 7feb0ac00700 1 -- start start 2024-04-12T21:11:17.487 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:17.490+0000 7feb0ac00700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7feb04105020 0x7feb04115b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:11:17.487 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:17.490+0000 7feb0ac00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7feb041059c0 0x7feb04116050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:11:17.487 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:17.490+0000 7feb0ac00700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb04116730 con 0x7feb041059c0 2024-04-12T21:11:17.487 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:17.490+0000 7feb0ac00700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb0410fcc0 con 0x7feb04105020 2024-04-12T21:11:17.488 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:17.491+0000 7feb03400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7feb041059c0 0x7feb04116050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:11:17.488 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:17.491+0000 7feb03400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7feb041059c0 0x7feb04116050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:55472/0 (socket says 172.21.15.18:55472) 2024-04-12T21:11:17.488 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:17.491+0000 7feb03400700 1 -- 172.21.15.18:0/2178448514 learned_addr learned my addr 172.21.15.18:0/2178448514 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:11:17.488 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:17.491+0000 7feb03e00700 1 --2- 172.21.15.18:0/2178448514 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7feb04105020 0x7feb04115b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:11:17.488 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:17.491+0000 7feb03400700 1 -- 172.21.15.18:0/2178448514 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7feb04105020 msgr2=0x7feb04115b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:11:17.488 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:17.491+0000 7feb03400700 1 --2- 172.21.15.18:0/2178448514 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7feb04105020 0x7feb04115b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:11:17.488 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:17.491+0000 7feb03400700 1 -- 172.21.15.18:0/2178448514 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feaf0005040 con 0x7feb041059c0 2024-04-12T21:11:17.488 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:17.492+0000 7feb03400700 1 --2- 172.21.15.18:0/2178448514 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7feb041059c0 0x7feb04116050 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7feaf0009180 tx=0x7feaf0009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:11:17.489 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:17.492+0000 7feafbe00700 1 -- 172.21.15.18:0/2178448514 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7feaf000e040 con 0x7feb041059c0 2024-04-12T21:11:17.489 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:17.492+0000 7feafbe00700 1 -- 172.21.15.18:0/2178448514 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7feaf000bbc0 con 0x7feb041059c0 2024-04-12T21:11:17.489 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:17.492+0000 7feafbe00700 1 -- 172.21.15.18:0/2178448514 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7feaf001c3f0 con 0x7feb041059c0 2024-04-12T21:11:17.489 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:17.492+0000 7feb0ac00700 1 -- 172.21.15.18:0/2178448514 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feb0410ff40 con 0x7feb041059c0 2024-04-12T21:11:17.489 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:17.492+0000 7feb0ac00700 1 -- 172.21.15.18:0/2178448514 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feb04110500 con 0x7feb041059c0 2024-04-12T21:11:17.490 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:17.493+0000 7feb0ac00700 1 -- 172.21.15.18:0/2178448514 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feb04100770 con 0x7feb041059c0 2024-04-12T21:11:17.491 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:17.494+0000 7feafbe00700 1 -- 172.21.15.18:0/2178448514 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7feaf000bd30 con 0x7feb041059c0 2024-04-12T21:11:17.492 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:17.495+0000 7feafbe00700 1 --2- 172.21.15.18:0/2178448514 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7feae806ec00 0x7feae80710b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:11:17.492 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:17.495+0000 7feafbe00700 1 -- 172.21.15.18:0/2178448514 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 6757+0+0 (secure 0 0 0) 0x7feaf008fcc0 con 0x7feb041059c0 2024-04-12T21:11:17.492 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:17.496+0000 7feb03e00700 1 --2- 172.21.15.18:0/2178448514 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7feae806ec00 0x7feae80710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:11:17.492 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:17.496+0000 7feb03e00700 1 --2- 172.21.15.18:0/2178448514 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7feae806ec00 0x7feae80710b0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7feaf4006fd0 tx=0x7feaf4008040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:11:17.497 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:17.501+0000 7feafbe00700 1 -- 172.21.15.18:0/2178448514 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7feaf0095020 con 0x7feb041059c0 2024-04-12T21:11:17.528 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:11:17 smithi186 ceph-mon[86379]: pgmap v344: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:11:17.752 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:17.755+0000 7feb0ac00700 1 -- 172.21.15.18:0/2178448514 --> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7feb0410c1e0 con 0x7feae806ec00 2024-04-12T21:11:17.754 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:17.758+0000 7feafbe00700 1 -- 172.21.15.18:0/2178448514 <== mgr.34102 v2:172.21.15.18:6800/3812891919 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7feb0410c1e0 con 0x7feae806ec00 2024-04-12T21:11:17.757 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:17.760+0000 7feb0ac00700 1 -- 172.21.15.18:0/2178448514 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7feae806ec00 msgr2=0x7feae80710b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:11:17.757 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:17.760+0000 7feb0ac00700 1 --2- 172.21.15.18:0/2178448514 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7feae806ec00 0x7feae80710b0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7feaf4006fd0 tx=0x7feaf4008040 comp rx=0 tx=0).stop 2024-04-12T21:11:17.757 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:17.761+0000 7feb0ac00700 1 -- 172.21.15.18:0/2178448514 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7feb041059c0 msgr2=0x7feb04116050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:11:17.757 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:17.761+0000 7feb0ac00700 1 --2- 172.21.15.18:0/2178448514 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7feb041059c0 0x7feb04116050 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7feaf0009180 tx=0x7feaf0009e30 comp rx=0 tx=0).stop 2024-04-12T21:11:17.758 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:17.761+0000 7feb0ac00700 1 -- 172.21.15.18:0/2178448514 shutdown_connections 2024-04-12T21:11:17.758 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:17.761+0000 7feb0ac00700 1 --2- 172.21.15.18:0/2178448514 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7feae806ec00 0x7feae80710b0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:11:17.758 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:17.761+0000 7feb0ac00700 1 --2- 172.21.15.18:0/2178448514 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7feb04105020 0x7feb04115b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:11:17.758 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:17.761+0000 7feb0ac00700 1 --2- 172.21.15.18:0/2178448514 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7feb041059c0 0x7feb04116050 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:11:17.758 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:17.761+0000 7feb0ac00700 1 -- 172.21.15.18:0/2178448514 >> 172.21.15.18:0/2178448514 conn(0x7feb040faba0 msgr2=0x7feb040fbaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:11:17.758 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:17.761+0000 7feb0ac00700 1 -- 172.21.15.18:0/2178448514 shutdown_connections 2024-04-12T21:11:17.758 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:17.761+0000 7feb0ac00700 1 -- 172.21.15.18:0/2178448514 wait complete. 2024-04-12T21:11:17.774 INFO:teuthology.orchestra.run.smithi018.stdout:true 2024-04-12T21:11:17.921 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:17.924+0000 7f55aca00700 1 -- 172.21.15.18:0/1862173071 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f55a8109430 msgr2=0x7f55a8109800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:11:17.921 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:17.924+0000 7f55aca00700 1 --2- 172.21.15.18:0/1862173071 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f55a8109430 0x7f55a8109800 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f5598004940 tx=0x7f5598009e30 comp rx=0 tx=0).stop 2024-04-12T21:11:17.921 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:17.924+0000 7f55aca00700 1 -- 172.21.15.18:0/1862173071 shutdown_connections 2024-04-12T21:11:17.921 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:17.924+0000 7f55aca00700 1 --2- 172.21.15.18:0/1862173071 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f55a8109dd0 0x7f55a8103440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:11:17.921 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:17.924+0000 7f55aca00700 1 --2- 172.21.15.18:0/1862173071 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f55a8109430 0x7f55a8109800 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:11:17.921 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:17.924+0000 7f55aca00700 1 -- 172.21.15.18:0/1862173071 >> 172.21.15.18:0/1862173071 conn(0x7f55a8072c00 msgr2=0x7f55a8075010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:11:17.921 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:17.924+0000 7f55aca00700 1 -- 172.21.15.18:0/1862173071 shutdown_connections 2024-04-12T21:11:17.921 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:17.924+0000 7f55aca00700 1 -- 172.21.15.18:0/1862173071 wait complete. 2024-04-12T21:11:17.922 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:17.925+0000 7f55aca00700 1 Processor -- start 2024-04-12T21:11:17.923 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:17.925+0000 7f55aca00700 1 -- start start 2024-04-12T21:11:17.923 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:17.926+0000 7f55aca00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f55a8109430 0x7f55a8119f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:11:17.923 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:17.926+0000 7f55aca00700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f55a8109dd0 0x7f55a811a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:11:17.923 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:17.926+0000 7f55aca00700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55a811aad0 con 0x7f55a8109430 2024-04-12T21:11:17.923 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:17.926+0000 7f55aca00700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55a8114100 con 0x7f55a8109dd0 2024-04-12T21:11:17.923 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:17.926+0000 7f55a5e00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f55a8109430 0x7f55a8119f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:11:17.923 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:17.926+0000 7f55a5e00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f55a8109430 0x7f55a8119f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:55492/0 (socket says 172.21.15.18:55492) 2024-04-12T21:11:17.923 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:17.926+0000 7f55a5e00700 1 -- 172.21.15.18:0/2065427565 learned_addr learned my addr 172.21.15.18:0/2065427565 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:11:17.924 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:17.926+0000 7f55a5e00700 1 -- 172.21.15.18:0/2065427565 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f55a8109dd0 msgr2=0x7f55a811a480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:11:17.924 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:17.926+0000 7f55a5e00700 1 --2- 172.21.15.18:0/2065427565 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f55a8109dd0 0x7f55a811a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:11:17.924 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:17.927+0000 7f55a5e00700 1 -- 172.21.15.18:0/2065427565 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5598005040 con 0x7f55a8109430 2024-04-12T21:11:17.924 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:17.927+0000 7f55a5e00700 1 --2- 172.21.15.18:0/2065427565 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f55a8109430 0x7f55a8119f40 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f5598009150 tx=0x7f5598009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:11:17.924 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:17.927+0000 7f559e000700 1 -- 172.21.15.18:0/2065427565 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f559800e040 con 0x7f55a8109430 2024-04-12T21:11:17.924 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:17.927+0000 7f55aca00700 1 -- 172.21.15.18:0/2065427565 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f55a8114380 con 0x7f55a8109430 2024-04-12T21:11:17.924 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:17.927+0000 7f559e000700 1 -- 172.21.15.18:0/2065427565 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f5598013420 con 0x7f55a8109430 2024-04-12T21:11:17.924 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:17.927+0000 7f559e000700 1 -- 172.21.15.18:0/2065427565 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f559801c630 con 0x7f55a8109430 2024-04-12T21:11:17.925 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:17.927+0000 7f55aca00700 1 -- 172.21.15.18:0/2065427565 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f55a8114940 con 0x7f55a8109430 2024-04-12T21:11:17.925 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:17.928+0000 7f55aca00700 1 -- 172.21.15.18:0/2065427565 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f55a8104b80 con 0x7f55a8109430 2024-04-12T21:11:17.926 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:17.930+0000 7f559e000700 1 -- 172.21.15.18:0/2065427565 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7f5598013590 con 0x7f55a8109430 2024-04-12T21:11:17.927 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:17.930+0000 7f559e000700 1 --2- 172.21.15.18:0/2065427565 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f558c06ec00 0x7f558c0710b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:11:17.927 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:17.931+0000 7f559e000700 1 -- 172.21.15.18:0/2065427565 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 6757+0+0 (secure 0 0 0) 0x7f559808fc40 con 0x7f55a8109430 2024-04-12T21:11:17.927 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:17.931+0000 7f55a5400700 1 --2- 172.21.15.18:0/2065427565 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f558c06ec00 0x7f558c0710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:11:17.928 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:17.932+0000 7f55a5400700 1 --2- 172.21.15.18:0/2065427565 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f558c06ec00 0x7f558c0710b0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f55a81157e0 tx=0x7f5594009ce0 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:11:17.932 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:17.935+0000 7f559e000700 1 -- 172.21.15.18:0/2065427565 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f559805c490 con 0x7f55a8109430 2024-04-12T21:11:18.187 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:18.190+0000 7f55aca00700 1 -- 172.21.15.18:0/2065427565 --> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f55a8110590 con 0x7f558c06ec00 2024-04-12T21:11:18.189 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:18.193+0000 7f559e000700 1 -- 172.21.15.18:0/2065427565 <== mgr.34102 v2:172.21.15.18:6800/3812891919 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f55a8110590 con 0x7f558c06ec00 2024-04-12T21:11:18.193 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:18.196+0000 7f55aca00700 1 -- 172.21.15.18:0/2065427565 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f558c06ec00 msgr2=0x7f558c0710b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:11:18.193 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:18.196+0000 7f55aca00700 1 --2- 172.21.15.18:0/2065427565 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f558c06ec00 0x7f558c0710b0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f55a81157e0 tx=0x7f5594009ce0 comp rx=0 tx=0).stop 2024-04-12T21:11:18.193 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:18.196+0000 7f55aca00700 1 -- 172.21.15.18:0/2065427565 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f55a8109430 msgr2=0x7f55a8119f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:11:18.193 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:18.196+0000 7f55aca00700 1 --2- 172.21.15.18:0/2065427565 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f55a8109430 0x7f55a8119f40 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f5598009150 tx=0x7f5598009e30 comp rx=0 tx=0).stop 2024-04-12T21:11:18.193 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:18.196+0000 7f55aca00700 1 -- 172.21.15.18:0/2065427565 shutdown_connections 2024-04-12T21:11:18.193 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:18.196+0000 7f55aca00700 1 --2- 172.21.15.18:0/2065427565 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f558c06ec00 0x7f558c0710b0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:11:18.193 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:18.196+0000 7f55aca00700 1 --2- 172.21.15.18:0/2065427565 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f55a8109430 0x7f55a8119f40 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:11:18.193 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:18.196+0000 7f55aca00700 1 --2- 172.21.15.18:0/2065427565 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f55a8109dd0 0x7f55a811a480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:11:18.194 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:18.196+0000 7f55aca00700 1 -- 172.21.15.18:0/2065427565 >> 172.21.15.18:0/2065427565 conn(0x7f55a8072c00 msgr2=0x7f55a8074c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:11:18.194 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:18.196+0000 7f55aca00700 1 -- 172.21.15.18:0/2065427565 shutdown_connections 2024-04-12T21:11:18.194 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:18.196+0000 7f55aca00700 1 -- 172.21.15.18:0/2065427565 wait complete. 2024-04-12T21:11:18.351 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:18.354+0000 7f995fc00700 1 -- 172.21.15.18:0/765427702 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f99580fde10 msgr2=0x7f99580fe1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:11:18.351 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:18.354+0000 7f995fc00700 1 --2- 172.21.15.18:0/765427702 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f99580fde10 0x7f99580fe1e0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f994c004940 tx=0x7f994c009e30 comp rx=0 tx=0).stop 2024-04-12T21:11:18.351 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:18.354+0000 7f995fc00700 1 -- 172.21.15.18:0/765427702 shutdown_connections 2024-04-12T21:11:18.351 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:18.354+0000 7f995fc00700 1 --2- 172.21.15.18:0/765427702 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f99580fe7b0 0x7f9958106df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:11:18.351 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:18.354+0000 7f995fc00700 1 --2- 172.21.15.18:0/765427702 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f99580fde10 0x7f99580fe1e0 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:11:18.351 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:18.354+0000 7f995fc00700 1 -- 172.21.15.18:0/765427702 >> 172.21.15.18:0/765427702 conn(0x7f99580f7860 msgr2=0x7f99580f9c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:11:18.351 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:18.354+0000 7f995fc00700 1 -- 172.21.15.18:0/765427702 shutdown_connections 2024-04-12T21:11:18.351 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:18.354+0000 7f995fc00700 1 -- 172.21.15.18:0/765427702 wait complete. 2024-04-12T21:11:18.352 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:18.355+0000 7f995fc00700 1 Processor -- start 2024-04-12T21:11:18.352 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:18.355+0000 7f995fc00700 1 -- start start 2024-04-12T21:11:18.352 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:18.356+0000 7f995fc00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f99580fde10 0x7f9958115b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:11:18.353 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:18.356+0000 7f995fc00700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f99580fe7b0 0x7f9958116070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:11:18.353 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:18.356+0000 7f995fc00700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99581166c0 con 0x7f99580fde10 2024-04-12T21:11:18.353 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:18.356+0000 7f995fc00700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f995810fca0 con 0x7f99580fe7b0 2024-04-12T21:11:18.353 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:18.356+0000 7f995d200700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f99580fde10 0x7f9958115b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:11:18.353 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:18.356+0000 7f995d200700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f99580fde10 0x7f9958115b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:55512/0 (socket says 172.21.15.18:55512) 2024-04-12T21:11:18.353 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:18.356+0000 7f995d200700 1 -- 172.21.15.18:0/3810803781 learned_addr learned my addr 172.21.15.18:0/3810803781 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:11:18.354 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:18.357+0000 7f9957e00700 1 --2- 172.21.15.18:0/3810803781 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f99580fe7b0 0x7f9958116070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:11:18.354 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:18.357+0000 7f995d200700 1 -- 172.21.15.18:0/3810803781 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f99580fe7b0 msgr2=0x7f9958116070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:11:18.354 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:18.357+0000 7f995d200700 1 --2- 172.21.15.18:0/3810803781 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f99580fe7b0 0x7f9958116070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:11:18.354 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:18.357+0000 7f995d200700 1 -- 172.21.15.18:0/3810803781 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f994c005040 con 0x7f99580fde10 2024-04-12T21:11:18.354 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:18.357+0000 7f995d200700 1 --2- 172.21.15.18:0/3810803781 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f99580fde10 0x7f9958115b30 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f994c009180 tx=0x7f994c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:11:18.354 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:18.357+0000 7f9954c00700 1 -- 172.21.15.18:0/3810803781 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f994c00e040 con 0x7f99580fde10 2024-04-12T21:11:18.354 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:18.357+0000 7f9954c00700 1 -- 172.21.15.18:0/3810803781 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f994c013420 con 0x7f99580fde10 2024-04-12T21:11:18.354 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:18.357+0000 7f995fc00700 1 -- 172.21.15.18:0/3810803781 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f995810ff20 con 0x7f99580fde10 2024-04-12T21:11:18.355 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:18.357+0000 7f9954c00700 1 -- 172.21.15.18:0/3810803781 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f994c01c630 con 0x7f99580fde10 2024-04-12T21:11:18.355 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:18.358+0000 7f995fc00700 1 -- 172.21.15.18:0/3810803781 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9958110460 con 0x7f99580fde10 2024-04-12T21:11:18.355 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:18.359+0000 7f995fc00700 1 -- 172.21.15.18:0/3810803781 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9958066af0 con 0x7f99580fde10 2024-04-12T21:11:18.356 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:18.360+0000 7f9954c00700 1 -- 172.21.15.18:0/3810803781 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7f994c013590 con 0x7f99580fde10 2024-04-12T21:11:18.357 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:18.361+0000 7f9954c00700 1 --2- 172.21.15.18:0/3810803781 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f994006ec00 0x7f99400710b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:11:18.358 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:18.361+0000 7f9954c00700 1 -- 172.21.15.18:0/3810803781 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 6757+0+0 (secure 0 0 0) 0x7f994c08fde0 con 0x7f99580fde10 2024-04-12T21:11:18.358 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:18.361+0000 7f9957e00700 1 --2- 172.21.15.18:0/3810803781 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f994006ec00 0x7f99400710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:11:18.358 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:18.362+0000 7f9957e00700 1 --2- 172.21.15.18:0/3810803781 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f994006ec00 0x7f99400710b0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f9944007fd0 tx=0x7f9944009040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:11:18.362 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:18.366+0000 7f9954c00700 1 -- 172.21.15.18:0/3810803781 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f994c05c590 con 0x7f99580fde10 2024-04-12T21:11:18.602 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:18.605+0000 7f995fc00700 1 -- 172.21.15.18:0/3810803781 --> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f9958109f40 con 0x7f994006ec00 2024-04-12T21:11:18.638 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:18.641+0000 7f9954c00700 1 -- 172.21.15.18:0/3810803781 <== mgr.34102 v2:172.21.15.18:6800/3812891919 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7f9944004db0 con 0x7f994006ec00 2024-04-12T21:11:18.638 INFO:teuthology.orchestra.run.smithi018.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-12T21:11:18.638 INFO:teuthology.orchestra.run.smithi018.stdout:alertmanager.smithi018 smithi018 *:9093,9094 running (11m) 3m ago 26m 14.7M - 0.25.0 c8568f914cd2 c332090068e8 2024-04-12T21:11:18.638 INFO:teuthology.orchestra.run.smithi018.stdout:crash.smithi018 smithi018 running (8m) 3m ago 26m 6665k - 19.0.0-2879-g324baf8d 8717720dc5e5 b7f80d1f7982 2024-04-12T21:11:18.638 INFO:teuthology.orchestra.run.smithi018.stdout:crash.smithi186 smithi186 running (8m) 4m ago 25m 6660k - 19.0.0-2879-g324baf8d 8717720dc5e5 a4f55c81b623 2024-04-12T21:11:18.639 INFO:teuthology.orchestra.run.smithi018.stdout:grafana.smithi018 smithi018 *:3000 running (11m) 3m ago 26m 74.8M - 9.4.12 2bacad6d85d8 b08e3bf176dd 2024-04-12T21:11:18.639 INFO:teuthology.orchestra.run.smithi018.stdout:mds.cephfs.smithi018.acqbsc smithi018 running (4m) 3m ago 19m 11.1M - 19.0.0-2879-g324baf8d 8717720dc5e5 021f99ab24ae 2024-04-12T21:11:18.639 INFO:teuthology.orchestra.run.smithi018.stdout:mds.cephfs.smithi018.szjluo smithi018 running (3m) 3m ago 19m 10.9M - 19.0.0-2879-g324baf8d 8717720dc5e5 0003117479d4 2024-04-12T21:11:18.639 INFO:teuthology.orchestra.run.smithi018.stdout:mds.cephfs.smithi186.jdllyn smithi186 running (19m) 4m ago 19m 15.6M - 17.2.7-827-gd3b5060a 21de9b7369ad 2b2bc46bd29a 2024-04-12T21:11:18.639 INFO:teuthology.orchestra.run.smithi018.stdout:mds.cephfs.smithi186.wjnafs smithi186 running (19m) 4m ago 19m 15.4M - 17.2.7-827-gd3b5060a 21de9b7369ad 74b115d6d6fd 2024-04-12T21:11:18.639 INFO:teuthology.orchestra.run.smithi018.stdout:mgr.smithi018.xjzwga smithi018 *:8443,9283,8765 running (14m) 3m ago 28m 509M - 19.0.0-2879-g324baf8d 8717720dc5e5 56088305eab7 2024-04-12T21:11:18.639 INFO:teuthology.orchestra.run.smithi018.stdout:mgr.smithi186.kqcfxk smithi186 *:8443,9283,8765 running (13m) 4m ago 25m 452M - 19.0.0-2879-g324baf8d 8717720dc5e5 09cd766e5e0d 2024-04-12T21:11:18.639 INFO:teuthology.orchestra.run.smithi018.stdout:mon.smithi018 smithi018 running (9m) 3m ago 28m 57.0M 2048M 19.0.0-2879-g324baf8d 8717720dc5e5 df3e201dfed6 2024-04-12T21:11:18.639 INFO:teuthology.orchestra.run.smithi018.stdout:mon.smithi186 smithi186 running (9m) 4m ago 24m 37.7M 2048M 19.0.0-2879-g324baf8d 8717720dc5e5 c77498da9b2b 2024-04-12T21:11:18.639 INFO:teuthology.orchestra.run.smithi018.stdout:node-exporter.smithi018 smithi018 *:9100 running (12m) 3m ago 26m 7780k - 1.5.0 0da6a335fe13 326c3d9c6847 2024-04-12T21:11:18.639 INFO:teuthology.orchestra.run.smithi018.stdout:node-exporter.smithi186 smithi186 *:9100 running (12m) 4m ago 25m 7826k - 1.5.0 0da6a335fe13 ec2112baeecf 2024-04-12T21:11:18.639 INFO:teuthology.orchestra.run.smithi018.stdout:osd.0 smithi018 running (7m) 3m ago 24m 121M 1559M 19.0.0-2879-g324baf8d 8717720dc5e5 60b1273ca047 2024-04-12T21:11:18.639 INFO:teuthology.orchestra.run.smithi018.stdout:osd.1 smithi018 running (7m) 3m ago 23m 91.3M 1559M 19.0.0-2879-g324baf8d 8717720dc5e5 170ce1e5fb8d 2024-04-12T21:11:18.640 INFO:teuthology.orchestra.run.smithi018.stdout:osd.2 smithi018 running (6m) 3m ago 22m 90.3M 1559M 19.0.0-2879-g324baf8d 8717720dc5e5 c2cbc6aa2839 2024-04-12T21:11:18.640 INFO:teuthology.orchestra.run.smithi018.stdout:osd.3 smithi186 running (5m) 4m ago 22m 96.6M 2581M 19.0.0-2879-g324baf8d 8717720dc5e5 0138e8262e4b 2024-04-12T21:11:18.640 INFO:teuthology.orchestra.run.smithi018.stdout:osd.4 smithi186 running (5m) 4m ago 21m 87.2M 2581M 19.0.0-2879-g324baf8d 8717720dc5e5 b20cb63b25c9 2024-04-12T21:11:18.640 INFO:teuthology.orchestra.run.smithi018.stdout:osd.5 smithi186 running (4m) 4m ago 21m 11.1M 2581M 19.0.0-2879-g324baf8d 8717720dc5e5 2513bea2610e 2024-04-12T21:11:18.640 INFO:teuthology.orchestra.run.smithi018.stdout:prometheus.smithi018 smithi018 *:9095 running (12m) 3m ago 25m 60.6M - 2.43.0 a07b618ecd1d f58841127e21 2024-04-12T21:11:18.640 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:18.644+0000 7f995fc00700 1 -- 172.21.15.18:0/3810803781 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f994006ec00 msgr2=0x7f99400710b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:11:18.640 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:18.644+0000 7f995fc00700 1 --2- 172.21.15.18:0/3810803781 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f994006ec00 0x7f99400710b0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f9944007fd0 tx=0x7f9944009040 comp rx=0 tx=0).stop 2024-04-12T21:11:18.641 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:18.644+0000 7f995fc00700 1 -- 172.21.15.18:0/3810803781 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f99580fde10 msgr2=0x7f9958115b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:11:18.641 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:18.644+0000 7f995fc00700 1 --2- 172.21.15.18:0/3810803781 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f99580fde10 0x7f9958115b30 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f994c009180 tx=0x7f994c009e30 comp rx=0 tx=0).stop 2024-04-12T21:11:18.641 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:18.644+0000 7f995fc00700 1 -- 172.21.15.18:0/3810803781 shutdown_connections 2024-04-12T21:11:18.641 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:18.644+0000 7f995fc00700 1 --2- 172.21.15.18:0/3810803781 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f994006ec00 0x7f99400710b0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:11:18.641 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:18.644+0000 7f995fc00700 1 --2- 172.21.15.18:0/3810803781 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f99580fde10 0x7f9958115b30 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:11:18.641 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:18.644+0000 7f995fc00700 1 --2- 172.21.15.18:0/3810803781 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f99580fe7b0 0x7f9958116070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:11:18.641 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:18.644+0000 7f995fc00700 1 -- 172.21.15.18:0/3810803781 >> 172.21.15.18:0/3810803781 conn(0x7f99580f7860 msgr2=0x7f99580f8d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:11:18.641 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:18.644+0000 7f995fc00700 1 -- 172.21.15.18:0/3810803781 shutdown_connections 2024-04-12T21:11:18.642 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:18.644+0000 7f995fc00700 1 -- 172.21.15.18:0/3810803781 wait complete. 2024-04-12T21:11:18.796 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:18.799+0000 7fdafb000700 1 -- 172.21.15.18:0/3148554308 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fdaf40fde90 msgr2=0x7fdaf40fe260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:11:18.796 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:18.799+0000 7fdafb000700 1 --2- 172.21.15.18:0/3148554308 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fdaf40fde90 0x7fdaf40fe260 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fdae4004940 tx=0x7fdae4009e30 comp rx=0 tx=0).stop 2024-04-12T21:11:18.796 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:18.799+0000 7fdafb000700 1 -- 172.21.15.18:0/3148554308 shutdown_connections 2024-04-12T21:11:18.797 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:18.799+0000 7fdafb000700 1 --2- 172.21.15.18:0/3148554308 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdaf40fe830 0x7fdaf4106f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:11:18.797 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:18.799+0000 7fdafb000700 1 --2- 172.21.15.18:0/3148554308 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fdaf40fde90 0x7fdaf40fe260 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:11:18.797 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:18.799+0000 7fdafb000700 1 -- 172.21.15.18:0/3148554308 >> 172.21.15.18:0/3148554308 conn(0x7fdaf4072bc0 msgr2=0x7fdaf4072fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:11:18.797 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:18.800+0000 7fdafb000700 1 -- 172.21.15.18:0/3148554308 shutdown_connections 2024-04-12T21:11:18.797 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:18.800+0000 7fdafb000700 1 -- 172.21.15.18:0/3148554308 wait complete. 2024-04-12T21:11:18.798 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:18.800+0000 7fdafb000700 1 Processor -- start 2024-04-12T21:11:18.798 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:18.801+0000 7fdafb000700 1 -- start start 2024-04-12T21:11:18.798 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:18.801+0000 7fdafb000700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fdaf40fde90 0x7fdaf4115e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:11:18.798 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:18.801+0000 7fdafb000700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdaf40fe830 0x7fdaf4116360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:11:18.799 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:18.801+0000 7fdafb000700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdaf4116a40 con 0x7fdaf40fde90 2024-04-12T21:11:18.799 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:18.801+0000 7fdafb000700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdaf410fdc0 con 0x7fdaf40fe830 2024-04-12T21:11:18.799 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:18.801+0000 7fdaf3e00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fdaf40fde90 0x7fdaf4115e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:11:18.799 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:18.801+0000 7fdaf3e00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fdaf40fde90 0x7fdaf4115e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:55516/0 (socket says 172.21.15.18:55516) 2024-04-12T21:11:18.799 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:18.801+0000 7fdaf3e00700 1 -- 172.21.15.18:0/167683941 learned_addr learned my addr 172.21.15.18:0/167683941 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:11:18.799 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:18.802+0000 7fdaf3e00700 1 -- 172.21.15.18:0/167683941 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdaf40fe830 msgr2=0x7fdaf4116360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:11:18.799 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:18.802+0000 7fdaf3400700 1 --2- 172.21.15.18:0/167683941 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdaf40fe830 0x7fdaf4116360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:11:18.799 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:18.802+0000 7fdaf3e00700 1 --2- 172.21.15.18:0/167683941 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdaf40fe830 0x7fdaf4116360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:11:18.799 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:18.802+0000 7fdaf3e00700 1 -- 172.21.15.18:0/167683941 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdae4005040 con 0x7fdaf40fde90 2024-04-12T21:11:18.799 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:18.802+0000 7fdaf3400700 1 --2- 172.21.15.18:0/167683941 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdaf40fe830 0x7fdaf4116360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-12T21:11:18.799 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:18.802+0000 7fdaf3e00700 1 --2- 172.21.15.18:0/167683941 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fdaf40fde90 0x7fdaf4115e20 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fdae400b3b0 tx=0x7fdae4009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:11:18.800 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:18.802+0000 7fdaebe00700 1 -- 172.21.15.18:0/167683941 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdae400e040 con 0x7fdaf40fde90 2024-04-12T21:11:18.800 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:18.802+0000 7fdaebe00700 1 -- 172.21.15.18:0/167683941 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fdae4013420 con 0x7fdaf40fde90 2024-04-12T21:11:18.800 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:18.802+0000 7fdaebe00700 1 -- 172.21.15.18:0/167683941 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdae401c610 con 0x7fdaf40fde90 2024-04-12T21:11:18.800 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:18.802+0000 7fdafb000700 1 -- 172.21.15.18:0/167683941 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdaf4110040 con 0x7fdaf40fde90 2024-04-12T21:11:18.800 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:18.803+0000 7fdafb000700 1 -- 172.21.15.18:0/167683941 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdaf4110580 con 0x7fdaf40fde90 2024-04-12T21:11:18.800 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:18.804+0000 7fdafb000700 1 -- 172.21.15.18:0/167683941 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdaf4103620 con 0x7fdaf40fde90 2024-04-12T21:11:18.801 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:18.805+0000 7fdaebe00700 1 -- 172.21.15.18:0/167683941 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7fdae401c840 con 0x7fdaf40fde90 2024-04-12T21:11:18.802 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:18.805+0000 7fdaebe00700 1 --2- 172.21.15.18:0/167683941 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fdad806ec00 0x7fdad80710b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:11:18.802 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:18.806+0000 7fdaebe00700 1 -- 172.21.15.18:0/167683941 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 6757+0+0 (secure 0 0 0) 0x7fdae408fb40 con 0x7fdaf40fde90 2024-04-12T21:11:18.802 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:18.806+0000 7fdaf3400700 1 --2- 172.21.15.18:0/167683941 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fdad806ec00 0x7fdad80710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:11:18.803 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:18.806+0000 7fdaf3400700 1 --2- 172.21.15.18:0/167683941 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fdad806ec00 0x7fdad80710b0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fdae0006fd0 tx=0x7fdae0008040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:11:18.807 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:18.811+0000 7fdaebe00700 1 -- 172.21.15.18:0/167683941 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7fdae4058970 con 0x7fdaf40fde90 2024-04-12T21:11:19.131 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:19.134+0000 7fdafb000700 1 -- 172.21.15.18:0/167683941 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fdaf405f5f0 con 0x7fdaf40fde90 2024-04-12T21:11:19.132 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:19.135+0000 7fdaebe00700 1 -- 172.21.15.18:0/167683941 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+837 (secure 0 0 0) 0x7fdae405c030 con 0x7fdaf40fde90 2024-04-12T21:11:19.132 INFO:teuthology.orchestra.run.smithi018.stdout:{ 2024-04-12T21:11:19.132 INFO:teuthology.orchestra.run.smithi018.stdout: "mon": { 2024-04-12T21:11:19.132 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 19.0.0-2879-g324baf8d (324baf8ddb91dea46cac50cb535c306115791355) squid (dev)": 2 2024-04-12T21:11:19.132 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-12T21:11:19.132 INFO:teuthology.orchestra.run.smithi018.stdout: "mgr": { 2024-04-12T21:11:19.133 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 19.0.0-2879-g324baf8d (324baf8ddb91dea46cac50cb535c306115791355) squid (dev)": 2 2024-04-12T21:11:19.133 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-12T21:11:19.133 INFO:teuthology.orchestra.run.smithi018.stdout: "osd": { 2024-04-12T21:11:19.133 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 19.0.0-2879-g324baf8d (324baf8ddb91dea46cac50cb535c306115791355) squid (dev)": 6 2024-04-12T21:11:19.133 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-12T21:11:19.133 INFO:teuthology.orchestra.run.smithi018.stdout: "mds": { 2024-04-12T21:11:19.133 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 17.2.7-827-gd3b5060a (d3b5060a17449edfae9385cdf792afec109fca5d) quincy (stable)": 2, 2024-04-12T21:11:19.133 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 19.0.0-2879-g324baf8d (324baf8ddb91dea46cac50cb535c306115791355) squid (dev)": 2 2024-04-12T21:11:19.133 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-12T21:11:19.133 INFO:teuthology.orchestra.run.smithi018.stdout: "overall": { 2024-04-12T21:11:19.133 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 17.2.7-827-gd3b5060a (d3b5060a17449edfae9385cdf792afec109fca5d) quincy (stable)": 2, 2024-04-12T21:11:19.133 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 19.0.0-2879-g324baf8d (324baf8ddb91dea46cac50cb535c306115791355) squid (dev)": 12 2024-04-12T21:11:19.133 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-04-12T21:11:19.133 INFO:teuthology.orchestra.run.smithi018.stdout:} 2024-04-12T21:11:19.135 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:19.138+0000 7fdafb000700 1 -- 172.21.15.18:0/167683941 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fdad806ec00 msgr2=0x7fdad80710b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:11:19.135 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:19.138+0000 7fdafb000700 1 --2- 172.21.15.18:0/167683941 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fdad806ec00 0x7fdad80710b0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fdae0006fd0 tx=0x7fdae0008040 comp rx=0 tx=0).stop 2024-04-12T21:11:19.135 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:19.138+0000 7fdafb000700 1 -- 172.21.15.18:0/167683941 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fdaf40fde90 msgr2=0x7fdaf4115e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:11:19.135 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:19.138+0000 7fdafb000700 1 --2- 172.21.15.18:0/167683941 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fdaf40fde90 0x7fdaf4115e20 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fdae400b3b0 tx=0x7fdae4009e30 comp rx=0 tx=0).stop 2024-04-12T21:11:19.135 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:19.139+0000 7fdafb000700 1 -- 172.21.15.18:0/167683941 shutdown_connections 2024-04-12T21:11:19.135 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:19.139+0000 7fdafb000700 1 --2- 172.21.15.18:0/167683941 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fdad806ec00 0x7fdad80710b0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:11:19.136 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:19.139+0000 7fdafb000700 1 --2- 172.21.15.18:0/167683941 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fdaf40fde90 0x7fdaf4115e20 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:11:19.136 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:19.139+0000 7fdafb000700 1 --2- 172.21.15.18:0/167683941 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdaf40fe830 0x7fdaf4116360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:11:19.136 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:19.139+0000 7fdafb000700 1 -- 172.21.15.18:0/167683941 >> 172.21.15.18:0/167683941 conn(0x7fdaf4072bc0 msgr2=0x7fdaf40fa170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:11:19.136 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:19.139+0000 7fdafb000700 1 -- 172.21.15.18:0/167683941 shutdown_connections 2024-04-12T21:11:19.136 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:19.139+0000 7fdafb000700 1 -- 172.21.15.18:0/167683941 wait complete. 2024-04-12T21:11:19.294 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:19.297+0000 7ff8dbc00700 1 -- 172.21.15.18:0/97937960 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff8d4102e00 msgr2=0x7ff8d41031d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:11:19.295 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:19.297+0000 7ff8dbc00700 1 --2- 172.21.15.18:0/97937960 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff8d4102e00 0x7ff8d41031d0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7ff8c8004940 tx=0x7ff8c8009e30 comp rx=0 tx=0).stop 2024-04-12T21:11:19.295 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:19.298+0000 7ff8dbc00700 1 -- 172.21.15.18:0/97937960 shutdown_connections 2024-04-12T21:11:19.295 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:19.298+0000 7ff8dbc00700 1 --2- 172.21.15.18:0/97937960 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff8d41037a0 0x7ff8d40fce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:11:19.295 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:19.298+0000 7ff8dbc00700 1 --2- 172.21.15.18:0/97937960 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff8d4102e00 0x7ff8d41031d0 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:11:19.296 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:19.298+0000 7ff8dbc00700 1 -- 172.21.15.18:0/97937960 >> 172.21.15.18:0/97937960 conn(0x7ff8d40f89b0 msgr2=0x7ff8d40fadc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:11:19.296 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:19.298+0000 7ff8dbc00700 1 -- 172.21.15.18:0/97937960 shutdown_connections 2024-04-12T21:11:19.296 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:19.298+0000 7ff8dbc00700 1 -- 172.21.15.18:0/97937960 wait complete. 2024-04-12T21:11:19.298 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:19.299+0000 7ff8dbc00700 1 Processor -- start 2024-04-12T21:11:19.298 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:19.299+0000 7ff8dbc00700 1 -- start start 2024-04-12T21:11:19.298 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:19.299+0000 7ff8dbc00700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff8d4102e00 0x7ff8d4113ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:11:19.298 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:19.299+0000 7ff8dbc00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff8d41037a0 0x7ff8d41140e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:11:19.298 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:19.299+0000 7ff8dbc00700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8d4114730 con 0x7ff8d41037a0 2024-04-12T21:11:19.298 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:19.299+0000 7ff8dbc00700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8d410db70 con 0x7ff8d4102e00 2024-04-12T21:11:19.298 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:19.299+0000 7ff8d3e00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff8d41037a0 0x7ff8d41140e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:11:19.299 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:19.299+0000 7ff8d3e00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff8d41037a0 0x7ff8d41140e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:55544/0 (socket says 172.21.15.18:55544) 2024-04-12T21:11:19.299 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:19.299+0000 7ff8d3e00700 1 -- 172.21.15.18:0/1189938544 learned_addr learned my addr 172.21.15.18:0/1189938544 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:11:19.299 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:19.300+0000 7ff8d9200700 1 --2- 172.21.15.18:0/1189938544 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff8d4102e00 0x7ff8d4113ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:11:19.299 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:19.300+0000 7ff8d3e00700 1 -- 172.21.15.18:0/1189938544 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff8d4102e00 msgr2=0x7ff8d4113ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:11:19.299 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:19.300+0000 7ff8d3e00700 1 --2- 172.21.15.18:0/1189938544 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff8d4102e00 0x7ff8d4113ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:11:19.299 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:19.300+0000 7ff8d3e00700 1 -- 172.21.15.18:0/1189938544 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8c8005040 con 0x7ff8d41037a0 2024-04-12T21:11:19.299 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:19.300+0000 7ff8d3e00700 1 --2- 172.21.15.18:0/1189938544 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff8d41037a0 0x7ff8d41140e0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7ff8c000c890 tx=0x7ff8c00044c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:11:19.299 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:19.300+0000 7ff8d0c00700 1 -- 172.21.15.18:0/1189938544 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff8c000a7d0 con 0x7ff8d41037a0 2024-04-12T21:11:19.299 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:19.300+0000 7ff8d0c00700 1 -- 172.21.15.18:0/1189938544 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff8c000ae00 con 0x7ff8d41037a0 2024-04-12T21:11:19.299 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:19.301+0000 7ff8dbc00700 1 -- 172.21.15.18:0/1189938544 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff8d410de50 con 0x7ff8d41037a0 2024-04-12T21:11:19.299 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:19.301+0000 7ff8d0c00700 1 -- 172.21.15.18:0/1189938544 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff8c00113a0 con 0x7ff8d41037a0 2024-04-12T21:11:19.299 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:19.301+0000 7ff8dbc00700 1 -- 172.21.15.18:0/1189938544 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff8d410e410 con 0x7ff8d41037a0 2024-04-12T21:11:19.299 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:19.302+0000 7ff8dbc00700 1 -- 172.21.15.18:0/1189938544 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff8d4100110 con 0x7ff8d41037a0 2024-04-12T21:11:19.300 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:19.303+0000 7ff8d0c00700 1 -- 172.21.15.18:0/1189938544 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7ff8c0012400 con 0x7ff8d41037a0 2024-04-12T21:11:19.300 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:19.304+0000 7ff8d0c00700 1 --2- 172.21.15.18:0/1189938544 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7ff8bc06ec10 0x7ff8bc0710c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:11:19.301 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:19.304+0000 7ff8d0c00700 1 -- 172.21.15.18:0/1189938544 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 6757+0+0 (secure 0 0 0) 0x7ff8c008ea40 con 0x7ff8d41037a0 2024-04-12T21:11:19.301 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:19.304+0000 7ff8d9200700 1 --2- 172.21.15.18:0/1189938544 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7ff8bc06ec10 0x7ff8bc0710c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:11:19.301 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:19.305+0000 7ff8d9200700 1 --2- 172.21.15.18:0/1189938544 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7ff8bc06ec10 0x7ff8bc0710c0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7ff8c800b3b0 tx=0x7ff8c8009e30 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:11:19.305 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:19.309+0000 7ff8d0c00700 1 -- 172.21.15.18:0/1189938544 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7ff8c005b270 con 0x7ff8d41037a0 2024-04-12T21:11:19.306 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:11:19 smithi018 ceph-mon[112590]: from='client.34640 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:11:19.306 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:11:19 smithi018 ceph-mon[112590]: pgmap v345: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:11:19.306 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:11:19 smithi018 ceph-mon[112590]: from='client.34644 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:11:19.306 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:11:19 smithi018 ceph-mon[112590]: from='client.34648 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:11:19.529 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:11:19 smithi186 ceph-mon[86379]: from='client.34640 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:11:19.529 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:11:19 smithi186 ceph-mon[86379]: pgmap v345: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:11:19.529 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:11:19 smithi186 ceph-mon[86379]: from='client.34644 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:11:19.529 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:11:19 smithi186 ceph-mon[86379]: from='client.34648 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:11:19.576 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:19.580+0000 7ff8dbc00700 1 -- 172.21.15.18:0/1189938544 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7ff8d410f1f0 con 0x7ff8d41037a0 2024-04-12T21:11:19.577 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:19.581+0000 7ff8d0c00700 1 -- 172.21.15.18:0/1189938544 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 29 v29) v1 ==== 76+0+1872 (secure 0 0 0) 0x7ff8c005ae00 con 0x7ff8d41037a0 2024-04-12T21:11:19.579 INFO:teuthology.orchestra.run.smithi018.stdout:e29 2024-04-12T21:11:19.579 INFO:teuthology.orchestra.run.smithi018.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-12T21:11:19.579 INFO:teuthology.orchestra.run.smithi018.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-12T21:11:19.579 INFO:teuthology.orchestra.run.smithi018.stdout:legacy client fscid: 1 2024-04-12T21:11:19.579 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-12T21:11:19.579 INFO:teuthology.orchestra.run.smithi018.stdout:Filesystem 'cephfs' (1) 2024-04-12T21:11:19.579 INFO:teuthology.orchestra.run.smithi018.stdout:fs_name cephfs 2024-04-12T21:11:19.580 INFO:teuthology.orchestra.run.smithi018.stdout:epoch 24 2024-04-12T21:11:19.580 INFO:teuthology.orchestra.run.smithi018.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-04-12T21:11:19.580 INFO:teuthology.orchestra.run.smithi018.stdout:created 2024-04-12T20:51:57.362850+0000 2024-04-12T21:11:19.580 INFO:teuthology.orchestra.run.smithi018.stdout:modified 2024-04-12T21:06:59.717190+0000 2024-04-12T21:11:19.580 INFO:teuthology.orchestra.run.smithi018.stdout:tableserver 0 2024-04-12T21:11:19.580 INFO:teuthology.orchestra.run.smithi018.stdout:root 0 2024-04-12T21:11:19.580 INFO:teuthology.orchestra.run.smithi018.stdout:session_timeout 60 2024-04-12T21:11:19.580 INFO:teuthology.orchestra.run.smithi018.stdout:session_autoclose 300 2024-04-12T21:11:19.580 INFO:teuthology.orchestra.run.smithi018.stdout:max_file_size 1099511627776 2024-04-12T21:11:19.580 INFO:teuthology.orchestra.run.smithi018.stdout:max_xattr_size 65536 2024-04-12T21:11:19.580 INFO:teuthology.orchestra.run.smithi018.stdout:required_client_features {} 2024-04-12T21:11:19.580 INFO:teuthology.orchestra.run.smithi018.stdout:last_failure 0 2024-04-12T21:11:19.580 INFO:teuthology.orchestra.run.smithi018.stdout:last_failure_osd_epoch 100 2024-04-12T21:11:19.580 INFO:teuthology.orchestra.run.smithi018.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-12T21:11:19.580 INFO:teuthology.orchestra.run.smithi018.stdout:max_mds 2 2024-04-12T21:11:19.581 INFO:teuthology.orchestra.run.smithi018.stdout:in 0,1 2024-04-12T21:11:19.581 INFO:teuthology.orchestra.run.smithi018.stdout:up {} 2024-04-12T21:11:19.581 INFO:teuthology.orchestra.run.smithi018.stdout:failed 0,1 2024-04-12T21:11:19.581 INFO:teuthology.orchestra.run.smithi018.stdout:damaged 2024-04-12T21:11:19.581 INFO:teuthology.orchestra.run.smithi018.stdout:stopped 2024-04-12T21:11:19.581 INFO:teuthology.orchestra.run.smithi018.stdout:data_pools [3] 2024-04-12T21:11:19.581 INFO:teuthology.orchestra.run.smithi018.stdout:metadata_pool 2 2024-04-12T21:11:19.581 INFO:teuthology.orchestra.run.smithi018.stdout:inline_data disabled 2024-04-12T21:11:19.581 INFO:teuthology.orchestra.run.smithi018.stdout:balancer 2024-04-12T21:11:19.581 INFO:teuthology.orchestra.run.smithi018.stdout:bal_rank_mask -1 2024-04-12T21:11:19.581 INFO:teuthology.orchestra.run.smithi018.stdout:standby_count_wanted 1 2024-04-12T21:11:19.581 INFO:teuthology.orchestra.run.smithi018.stdout:qdb_cluster leader: 0 members: 2024-04-12T21:11:19.581 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-12T21:11:19.581 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-12T21:11:19.581 INFO:teuthology.orchestra.run.smithi018.stdout:Standby daemons: 2024-04-12T21:11:19.582 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-12T21:11:19.582 INFO:teuthology.orchestra.run.smithi018.stdout:[mds.cephfs.smithi186.wjnafs{-1:24277} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.186:6826/438347207,v1:172.21.15.186:6827/438347207] compat {c=[1],r=[1],i=[7ff]}] 2024-04-12T21:11:19.582 INFO:teuthology.orchestra.run.smithi018.stdout:[mds.cephfs.smithi186.jdllyn{-1:24281} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.186:6824/1464359952,v1:172.21.15.186:6825/1464359952] compat {c=[1],r=[1],i=[7ff]}] 2024-04-12T21:11:19.582 INFO:teuthology.orchestra.run.smithi018.stdout:[mds.cephfs.smithi018.acqbsc{-1:34436} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.18:6828/755179764,v1:172.21.15.18:6829/755179764] compat {c=[1],r=[1],i=[1fff]}] 2024-04-12T21:11:19.582 INFO:teuthology.orchestra.run.smithi018.stdout:[mds.cephfs.smithi018.szjluo{-1:34468} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.18:6826/2407992653,v1:172.21.15.18:6827/2407992653] compat {c=[1],r=[1],i=[1fff]}] 2024-04-12T21:11:19.583 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:19.583+0000 7ff8dbc00700 1 -- 172.21.15.18:0/1189938544 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7ff8bc06ec10 msgr2=0x7ff8bc0710c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:11:19.583 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:19.583+0000 7ff8dbc00700 1 --2- 172.21.15.18:0/1189938544 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7ff8bc06ec10 0x7ff8bc0710c0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7ff8c800b3b0 tx=0x7ff8c8009e30 comp rx=0 tx=0).stop 2024-04-12T21:11:19.583 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:19.583+0000 7ff8dbc00700 1 -- 172.21.15.18:0/1189938544 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff8d41037a0 msgr2=0x7ff8d41140e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:11:19.583 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:19.583+0000 7ff8dbc00700 1 --2- 172.21.15.18:0/1189938544 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff8d41037a0 0x7ff8d41140e0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7ff8c000c890 tx=0x7ff8c00044c0 comp rx=0 tx=0).stop 2024-04-12T21:11:19.583 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:19.584+0000 7ff8dbc00700 1 -- 172.21.15.18:0/1189938544 shutdown_connections 2024-04-12T21:11:19.583 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:19.584+0000 7ff8dbc00700 1 --2- 172.21.15.18:0/1189938544 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7ff8bc06ec10 0x7ff8bc0710c0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:11:19.583 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:19.584+0000 7ff8dbc00700 1 --2- 172.21.15.18:0/1189938544 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff8d4102e00 0x7ff8d4113ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:11:19.583 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:19.584+0000 7ff8dbc00700 1 --2- 172.21.15.18:0/1189938544 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff8d41037a0 0x7ff8d41140e0 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:11:19.583 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:19.584+0000 7ff8dbc00700 1 -- 172.21.15.18:0/1189938544 >> 172.21.15.18:0/1189938544 conn(0x7ff8d40f89b0 msgr2=0x7ff8d41073a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:11:19.583 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:19.584+0000 7ff8dbc00700 1 -- 172.21.15.18:0/1189938544 shutdown_connections 2024-04-12T21:11:19.583 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:19.584+0000 7ff8dbc00700 1 -- 172.21.15.18:0/1189938544 wait complete. 2024-04-12T21:11:19.583 INFO:teuthology.orchestra.run.smithi018.stderr:dumped fsmap epoch 29 2024-04-12T21:11:19.726 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:19.729+0000 7fd78da00700 1 -- 172.21.15.18:0/3040190745 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd7880fe830 msgr2=0x7fd788106f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:11:19.726 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:19.729+0000 7fd78da00700 1 --2- 172.21.15.18:0/3040190745 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd7880fe830 0x7fd788106f00 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fd774004940 tx=0x7fd774009e30 comp rx=0 tx=0).stop 2024-04-12T21:11:19.726 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:19.729+0000 7fd78da00700 1 -- 172.21.15.18:0/3040190745 shutdown_connections 2024-04-12T21:11:19.726 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:19.729+0000 7fd78da00700 1 --2- 172.21.15.18:0/3040190745 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd7880fe830 0x7fd788106f00 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:11:19.726 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:19.729+0000 7fd78da00700 1 --2- 172.21.15.18:0/3040190745 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd7880fde90 0x7fd7880fe260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:11:19.726 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:19.729+0000 7fd78da00700 1 -- 172.21.15.18:0/3040190745 >> 172.21.15.18:0/3040190745 conn(0x7fd788072bc0 msgr2=0x7fd788072fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:11:19.726 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:19.729+0000 7fd78da00700 1 -- 172.21.15.18:0/3040190745 shutdown_connections 2024-04-12T21:11:19.726 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:19.729+0000 7fd78da00700 1 -- 172.21.15.18:0/3040190745 wait complete. 2024-04-12T21:11:19.731 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:19.731+0000 7fd78da00700 1 Processor -- start 2024-04-12T21:11:19.731 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:19.731+0000 7fd78da00700 1 -- start start 2024-04-12T21:11:19.732 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:19.731+0000 7fd78da00700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd7880fde90 0x7fd788115db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:11:19.732 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:19.731+0000 7fd78da00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd7880fe830 0x7fd7881162f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:11:19.732 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:19.731+0000 7fd78da00700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd788116940 con 0x7fd7880fe830 2024-04-12T21:11:19.732 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:19.731+0000 7fd78da00700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd78810fd10 con 0x7fd7880fde90 2024-04-12T21:11:19.732 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:19.731+0000 7fd786000700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd7880fe830 0x7fd7881162f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:11:19.732 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:19.731+0000 7fd786000700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd7880fe830 0x7fd7881162f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:55562/0 (socket says 172.21.15.18:55562) 2024-04-12T21:11:19.732 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:19.731+0000 7fd786000700 1 -- 172.21.15.18:0/259514494 learned_addr learned my addr 172.21.15.18:0/259514494 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:11:19.732 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:19.732+0000 7fd786a00700 1 --2- 172.21.15.18:0/259514494 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd7880fde90 0x7fd788115db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:11:19.732 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:19.732+0000 7fd786000700 1 -- 172.21.15.18:0/259514494 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd7880fde90 msgr2=0x7fd788115db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:11:19.732 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:19.732+0000 7fd786000700 1 --2- 172.21.15.18:0/259514494 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd7880fde90 0x7fd788115db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:11:19.732 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:19.732+0000 7fd786000700 1 -- 172.21.15.18:0/259514494 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd774005040 con 0x7fd7880fe830 2024-04-12T21:11:19.732 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:19.732+0000 7fd786000700 1 --2- 172.21.15.18:0/259514494 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd7880fe830 0x7fd7881162f0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fd774000c00 tx=0x7fd774009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:11:19.732 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:19.732+0000 7fd77ea00700 1 -- 172.21.15.18:0/259514494 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd77400e040 con 0x7fd7880fe830 2024-04-12T21:11:19.732 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:19.733+0000 7fd77ea00700 1 -- 172.21.15.18:0/259514494 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd774013420 con 0x7fd7880fe830 2024-04-12T21:11:19.733 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:19.733+0000 7fd77ea00700 1 -- 172.21.15.18:0/259514494 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd77401c610 con 0x7fd7880fe830 2024-04-12T21:11:19.733 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:19.733+0000 7fd78da00700 1 -- 172.21.15.18:0/259514494 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd78810ff90 con 0x7fd7880fe830 2024-04-12T21:11:19.733 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:19.733+0000 7fd78da00700 1 -- 172.21.15.18:0/259514494 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd7881104d0 con 0x7fd7880fe830 2024-04-12T21:11:19.733 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:19.734+0000 7fd78da00700 1 -- 172.21.15.18:0/259514494 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd788106a20 con 0x7fd7880fe830 2024-04-12T21:11:19.733 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:19.735+0000 7fd77ea00700 1 -- 172.21.15.18:0/259514494 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7fd774013590 con 0x7fd7880fe830 2024-04-12T21:11:19.734 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:19.736+0000 7fd77ea00700 1 --2- 172.21.15.18:0/259514494 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fd76c06ec10 0x7fd76c0710c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:11:19.734 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:19.736+0000 7fd77ea00700 1 -- 172.21.15.18:0/259514494 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 6757+0+0 (secure 0 0 0) 0x7fd77408fc60 con 0x7fd7880fe830 2024-04-12T21:11:19.734 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:19.737+0000 7fd786a00700 1 --2- 172.21.15.18:0/259514494 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fd76c06ec10 0x7fd76c0710c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:11:19.734 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:19.737+0000 7fd786a00700 1 --2- 172.21.15.18:0/259514494 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fd76c06ec10 0x7fd76c0710c0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fd778004d60 tx=0x7fd77800b040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:11:19.737 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:19.741+0000 7fd77ea00700 1 -- 172.21.15.18:0/259514494 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7fd77405c430 con 0x7fd7880fe830 2024-04-12T21:11:19.977 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:19.980+0000 7fd78da00700 1 -- 172.21.15.18:0/259514494 --> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fd788111400 con 0x7fd76c06ec10 2024-04-12T21:11:19.980 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:19.983+0000 7fd77ea00700 1 -- 172.21.15.18:0/259514494 <== mgr.34102 v2:172.21.15.18:6800/3812891919 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7fd788111400 con 0x7fd76c06ec10 2024-04-12T21:11:19.980 INFO:teuthology.orchestra.run.smithi018.stdout:{ 2024-04-12T21:11:19.980 INFO:teuthology.orchestra.run.smithi018.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:f4452def3be4d350f4a8e5feaf7ebb1511be9298fe68478ba7a2d6fbd7d69a63", 2024-04-12T21:11:19.980 INFO:teuthology.orchestra.run.smithi018.stdout: "in_progress": true, 2024-04-12T21:11:19.980 INFO:teuthology.orchestra.run.smithi018.stdout: "which": "Upgrading all daemon types on all hosts", 2024-04-12T21:11:19.980 INFO:teuthology.orchestra.run.smithi018.stdout: "services_complete": [ 2024-04-12T21:11:19.980 INFO:teuthology.orchestra.run.smithi018.stdout: "crash", 2024-04-12T21:11:19.980 INFO:teuthology.orchestra.run.smithi018.stdout: "osd", 2024-04-12T21:11:19.981 INFO:teuthology.orchestra.run.smithi018.stdout: "mon", 2024-04-12T21:11:19.981 INFO:teuthology.orchestra.run.smithi018.stdout: "mgr" 2024-04-12T21:11:19.981 INFO:teuthology.orchestra.run.smithi018.stdout: ], 2024-04-12T21:11:19.981 INFO:teuthology.orchestra.run.smithi018.stdout: "progress": "14/21 daemons upgraded", 2024-04-12T21:11:19.981 INFO:teuthology.orchestra.run.smithi018.stdout: "message": "Currently upgrading mds daemons", 2024-04-12T21:11:19.981 INFO:teuthology.orchestra.run.smithi018.stdout: "is_paused": false 2024-04-12T21:11:19.981 INFO:teuthology.orchestra.run.smithi018.stdout:} 2024-04-12T21:11:19.983 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:19.986+0000 7fd78da00700 1 -- 172.21.15.18:0/259514494 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fd76c06ec10 msgr2=0x7fd76c0710c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:11:19.983 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:19.986+0000 7fd78da00700 1 --2- 172.21.15.18:0/259514494 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fd76c06ec10 0x7fd76c0710c0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fd778004d60 tx=0x7fd77800b040 comp rx=0 tx=0).stop 2024-04-12T21:11:19.983 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:19.986+0000 7fd78da00700 1 -- 172.21.15.18:0/259514494 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd7880fe830 msgr2=0x7fd7881162f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:11:19.983 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:19.986+0000 7fd78da00700 1 --2- 172.21.15.18:0/259514494 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd7880fe830 0x7fd7881162f0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fd774000c00 tx=0x7fd774009e30 comp rx=0 tx=0).stop 2024-04-12T21:11:19.984 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:19.987+0000 7fd78da00700 1 -- 172.21.15.18:0/259514494 shutdown_connections 2024-04-12T21:11:19.984 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:19.987+0000 7fd78da00700 1 --2- 172.21.15.18:0/259514494 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fd76c06ec10 0x7fd76c0710c0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:11:19.984 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:19.987+0000 7fd78da00700 1 --2- 172.21.15.18:0/259514494 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd7880fde90 0x7fd788115db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:11:19.984 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:19.987+0000 7fd78da00700 1 --2- 172.21.15.18:0/259514494 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd7880fe830 0x7fd7881162f0 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:11:19.984 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:19.987+0000 7fd78da00700 1 -- 172.21.15.18:0/259514494 >> 172.21.15.18:0/259514494 conn(0x7fd788072bc0 msgr2=0x7fd7880fb730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:11:19.984 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:19.987+0000 7fd78da00700 1 -- 172.21.15.18:0/259514494 shutdown_connections 2024-04-12T21:11:19.984 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:19.987+0000 7fd78da00700 1 -- 172.21.15.18:0/259514494 wait complete. 2024-04-12T21:11:20.130 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:20.133+0000 7f0fd3000700 1 -- 172.21.15.18:0/1527509340 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0fcc105020 msgr2=0x7f0fcc1053f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:11:20.130 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:20.133+0000 7f0fd3000700 1 --2- 172.21.15.18:0/1527509340 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0fcc105020 0x7f0fcc1053f0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f0fbc004940 tx=0x7f0fbc009e30 comp rx=0 tx=0).stop 2024-04-12T21:11:20.130 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:20.133+0000 7f0fd3000700 1 -- 172.21.15.18:0/1527509340 shutdown_connections 2024-04-12T21:11:20.130 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:20.133+0000 7f0fd3000700 1 --2- 172.21.15.18:0/1527509340 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0fcc1059c0 0x7f0fcc0ff080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:11:20.130 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:20.133+0000 7f0fd3000700 1 --2- 172.21.15.18:0/1527509340 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0fcc105020 0x7f0fcc1053f0 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:11:20.130 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:20.133+0000 7f0fd3000700 1 -- 172.21.15.18:0/1527509340 >> 172.21.15.18:0/1527509340 conn(0x7f0fcc0faba0 msgr2=0x7f0fcc0fcfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:11:20.130 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:20.133+0000 7f0fd3000700 1 -- 172.21.15.18:0/1527509340 shutdown_connections 2024-04-12T21:11:20.130 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:20.133+0000 7f0fd3000700 1 -- 172.21.15.18:0/1527509340 wait complete. 2024-04-12T21:11:20.132 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:20.134+0000 7f0fd3000700 1 Processor -- start 2024-04-12T21:11:20.132 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:20.134+0000 7f0fd3000700 1 -- start start 2024-04-12T21:11:20.132 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:20.134+0000 7f0fd3000700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0fcc105020 0x7f0fcc115bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:11:20.132 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:20.134+0000 7f0fd3000700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0fcc1059c0 0x7f0fcc116100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:11:20.132 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:20.134+0000 7f0fd3000700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fcc116750 con 0x7f0fcc1059c0 2024-04-12T21:11:20.132 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:20.134+0000 7f0fd3000700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fcc10fd30 con 0x7f0fcc105020 2024-04-12T21:11:20.133 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:20.135+0000 7f0fcb400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0fcc1059c0 0x7f0fcc116100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:11:20.133 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:20.135+0000 7f0fcb400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0fcc1059c0 0x7f0fcc116100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:55570/0 (socket says 172.21.15.18:55570) 2024-04-12T21:11:20.133 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:20.135+0000 7f0fcb400700 1 -- 172.21.15.18:0/1277289358 learned_addr learned my addr 172.21.15.18:0/1277289358 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:11:20.133 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:20.135+0000 7f0fcbe00700 1 --2- 172.21.15.18:0/1277289358 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0fcc105020 0x7f0fcc115bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:11:20.133 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:20.135+0000 7f0fcb400700 1 -- 172.21.15.18:0/1277289358 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0fcc105020 msgr2=0x7f0fcc115bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:11:20.133 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:20.135+0000 7f0fcb400700 1 --2- 172.21.15.18:0/1277289358 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0fcc105020 0x7f0fcc115bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:11:20.133 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:20.135+0000 7f0fcb400700 1 -- 172.21.15.18:0/1277289358 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0fbc005040 con 0x7f0fcc1059c0 2024-04-12T21:11:20.133 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:20.135+0000 7f0fcb400700 1 --2- 172.21.15.18:0/1277289358 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0fcc1059c0 0x7f0fcc116100 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f0fb8002a80 tx=0x7f0fb8004630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:11:20.133 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:20.136+0000 7f0fc3e00700 1 -- 172.21.15.18:0/1277289358 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0fb8007cc0 con 0x7f0fcc1059c0 2024-04-12T21:11:20.133 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:20.136+0000 7f0fc3e00700 1 -- 172.21.15.18:0/1277289358 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f0fb800f040 con 0x7f0fcc1059c0 2024-04-12T21:11:20.133 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:20.136+0000 7f0fc3e00700 1 -- 172.21.15.18:0/1277289358 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0fb800daf0 con 0x7f0fcc1059c0 2024-04-12T21:11:20.133 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:20.136+0000 7f0fd3000700 1 -- 172.21.15.18:0/1277289358 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0fcc110010 con 0x7f0fcc1059c0 2024-04-12T21:11:20.133 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:20.136+0000 7f0fd3000700 1 -- 172.21.15.18:0/1277289358 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0fcc1105d0 con 0x7f0fcc1059c0 2024-04-12T21:11:20.134 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:20.137+0000 7f0fd3000700 1 -- 172.21.15.18:0/1277289358 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0fcc100770 con 0x7f0fcc1059c0 2024-04-12T21:11:20.134 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:20.138+0000 7f0fc3e00700 1 -- 172.21.15.18:0/1277289358 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7f0fb800dc90 con 0x7f0fcc1059c0 2024-04-12T21:11:20.138 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:20.139+0000 7f0fc3e00700 1 --2- 172.21.15.18:0/1277289358 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f0fb006ec00 0x7f0fb00710b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:11:20.138 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:20.139+0000 7f0fc3e00700 1 -- 172.21.15.18:0/1277289358 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 6757+0+0 (secure 0 0 0) 0x7f0fb808e090 con 0x7f0fcc1059c0 2024-04-12T21:11:20.139 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:20.139+0000 7f0fcbe00700 1 --2- 172.21.15.18:0/1277289358 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f0fb006ec00 0x7f0fb00710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:11:20.139 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:20.140+0000 7f0fcbe00700 1 --2- 172.21.15.18:0/1277289358 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f0fb006ec00 0x7f0fb00710b0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f0fbc004560 tx=0x7f0fbc009e30 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:11:20.141 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:20.145+0000 7f0fc3e00700 1 -- 172.21.15.18:0/1277289358 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f0fb805a840 con 0x7f0fcc1059c0 2024-04-12T21:11:20.350 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:11:20 smithi018 ceph-mon[112590]: from='client.? 172.21.15.18:0/167683941' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T21:11:20.351 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:11:20 smithi018 ceph-mon[112590]: from='client.? 172.21.15.18:0/1189938544' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-12T21:11:20.451 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:20.455+0000 7f0fd3000700 1 -- 172.21.15.18:0/1277289358 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f0fcc10c140 con 0x7f0fcc1059c0 2024-04-12T21:11:20.452 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:20.455+0000 7f0fc3e00700 1 -- 172.21.15.18:0/1277289358 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+233 (secure 0 0 0) 0x7f0fb805a3d0 con 0x7f0fcc1059c0 2024-04-12T21:11:20.452 INFO:teuthology.orchestra.run.smithi018.stdout:HEALTH_ERR 1 filesystem is degraded; 1 filesystem is offline 2024-04-12T21:11:20.452 INFO:teuthology.orchestra.run.smithi018.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-04-12T21:11:20.452 INFO:teuthology.orchestra.run.smithi018.stdout: fs cephfs is degraded 2024-04-12T21:11:20.452 INFO:teuthology.orchestra.run.smithi018.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-04-12T21:11:20.452 INFO:teuthology.orchestra.run.smithi018.stdout: fs cephfs is offline because no MDS is active for it. 2024-04-12T21:11:20.455 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:20.459+0000 7f0fd3000700 1 -- 172.21.15.18:0/1277289358 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f0fb006ec00 msgr2=0x7f0fb00710b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:11:20.456 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:20.459+0000 7f0fd3000700 1 --2- 172.21.15.18:0/1277289358 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f0fb006ec00 0x7f0fb00710b0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f0fbc004560 tx=0x7f0fbc009e30 comp rx=0 tx=0).stop 2024-04-12T21:11:20.456 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:20.459+0000 7f0fd3000700 1 -- 172.21.15.18:0/1277289358 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0fcc1059c0 msgr2=0x7f0fcc116100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:11:20.456 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:20.459+0000 7f0fd3000700 1 --2- 172.21.15.18:0/1277289358 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0fcc1059c0 0x7f0fcc116100 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f0fb8002a80 tx=0x7f0fb8004630 comp rx=0 tx=0).stop 2024-04-12T21:11:20.456 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:20.459+0000 7f0fd3000700 1 -- 172.21.15.18:0/1277289358 shutdown_connections 2024-04-12T21:11:20.456 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:20.459+0000 7f0fd3000700 1 --2- 172.21.15.18:0/1277289358 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f0fb006ec00 0x7f0fb00710b0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:11:20.456 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:20.459+0000 7f0fd3000700 1 --2- 172.21.15.18:0/1277289358 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0fcc105020 0x7f0fcc115bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:11:20.456 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:20.459+0000 7f0fd3000700 1 --2- 172.21.15.18:0/1277289358 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0fcc1059c0 0x7f0fcc116100 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:11:20.456 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:20.459+0000 7f0fd3000700 1 -- 172.21.15.18:0/1277289358 >> 172.21.15.18:0/1277289358 conn(0x7f0fcc0faba0 msgr2=0x7f0fcc0fbba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:11:20.456 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:20.459+0000 7f0fd3000700 1 -- 172.21.15.18:0/1277289358 shutdown_connections 2024-04-12T21:11:20.456 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:20.459+0000 7f0fd3000700 1 -- 172.21.15.18:0/1277289358 wait complete. 2024-04-12T21:11:20.529 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:11:20 smithi186 ceph-mon[86379]: from='client.? 172.21.15.18:0/167683941' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T21:11:20.529 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:11:20 smithi186 ceph-mon[86379]: from='client.? 172.21.15.18:0/1189938544' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-12T21:11:21.350 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:11:21 smithi018 ceph-mon[112590]: from='client.34660 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:11:21.350 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:11:21 smithi018 ceph-mon[112590]: pgmap v346: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:11:21.350 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:11:21 smithi018 ceph-mon[112590]: from='client.? 172.21.15.18:0/1277289358' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-12T21:11:21.529 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:11:21 smithi186 ceph-mon[86379]: from='client.34660 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:11:21.529 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:11:21 smithi186 ceph-mon[86379]: pgmap v346: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:11:21.529 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:11:21 smithi186 ceph-mon[86379]: from='client.? 172.21.15.18:0/1277289358' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-12T21:11:23.350 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:11:23 smithi018 ceph-mon[112590]: pgmap v347: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:11:23.529 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:11:23 smithi186 ceph-mon[86379]: pgmap v347: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:11:25.350 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:11:25 smithi018 ceph-mon[112590]: pgmap v348: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:11:25.350 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:11:25 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T21:11:25.529 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:11:25 smithi186 ceph-mon[86379]: pgmap v348: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:11:25.529 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:11:25 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T21:11:27.350 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:11:27 smithi018 ceph-mon[112590]: pgmap v349: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:11:27.529 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:11:27 smithi186 ceph-mon[86379]: pgmap v349: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:11:29.350 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:11:29 smithi018 ceph-mon[112590]: pgmap v350: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:11:29.529 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:11:29 smithi186 ceph-mon[86379]: pgmap v350: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:11:31.350 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:11:31 smithi018 ceph-mon[112590]: pgmap v351: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:11:31.529 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:11:31 smithi186 ceph-mon[86379]: pgmap v351: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:11:33.529 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:11:33 smithi186 ceph-mon[86379]: pgmap v352: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:11:33.600 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:11:33 smithi018 ceph-mon[112590]: pgmap v352: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:11:35.379 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:11:35 smithi018 ceph-mon[112590]: pgmap v353: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:11:35.529 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:11:35 smithi186 ceph-mon[86379]: pgmap v353: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:11:37.529 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:11:37 smithi186 ceph-mon[86379]: pgmap v354: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:11:37.599 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:11:37 smithi018 ceph-mon[112590]: pgmap v354: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:11:39.529 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:11:39 smithi186 ceph-mon[86379]: pgmap v355: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:11:39.600 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:11:39 smithi018 ceph-mon[112590]: pgmap v355: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:11:40.502 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:11:40 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T21:11:40.529 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:11:40 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T21:11:41.528 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:11:41 smithi186 ceph-mon[86379]: pgmap v356: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:11:41.600 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:11:41 smithi018 ceph-mon[112590]: pgmap v356: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:11:43.529 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:11:43 smithi186 ceph-mon[86379]: pgmap v357: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:11:43.600 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:11:43 smithi018 ceph-mon[112590]: pgmap v357: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:11:45.529 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:11:45 smithi186 ceph-mon[86379]: pgmap v358: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:11:45.600 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:11:45 smithi018 ceph-mon[112590]: pgmap v358: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:11:47.529 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:11:47 smithi186 ceph-mon[86379]: pgmap v359: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:11:47.599 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:11:47 smithi018 ceph-mon[112590]: pgmap v359: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:11:49.529 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:11:49 smithi186 ceph-mon[86379]: pgmap v360: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:11:49.600 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:11:49 smithi018 ceph-mon[112590]: pgmap v360: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:11:50.629 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:50.633+0000 7f9d7e800700 1 -- 172.21.15.18:0/2177017962 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9d781035d0 msgr2=0x7f9d780fcc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:11:50.629 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:50.633+0000 7f9d7e800700 1 --2- 172.21.15.18:0/2177017962 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9d781035d0 0x7f9d780fcc40 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f9d64004940 tx=0x7f9d64009e30 comp rx=0 tx=0).stop 2024-04-12T21:11:50.629 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:50.633+0000 7f9d7e800700 1 -- 172.21.15.18:0/2177017962 shutdown_connections 2024-04-12T21:11:50.629 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:50.633+0000 7f9d7e800700 1 --2- 172.21.15.18:0/2177017962 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9d781035d0 0x7f9d780fcc40 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:11:50.629 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:50.633+0000 7f9d7e800700 1 --2- 172.21.15.18:0/2177017962 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9d78102c30 0x7f9d78103000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:11:50.629 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:50.633+0000 7f9d7e800700 1 -- 172.21.15.18:0/2177017962 >> 172.21.15.18:0/2177017962 conn(0x7f9d780f87a0 msgr2=0x7f9d780fabb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:11:50.629 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:50.634+0000 7f9d7e800700 1 -- 172.21.15.18:0/2177017962 shutdown_connections 2024-04-12T21:11:50.630 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:50.634+0000 7f9d7e800700 1 -- 172.21.15.18:0/2177017962 wait complete. 2024-04-12T21:11:50.630 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:50.634+0000 7f9d7e800700 1 Processor -- start 2024-04-12T21:11:50.631 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:50.635+0000 7f9d7e800700 1 -- start start 2024-04-12T21:11:50.631 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:50.635+0000 7f9d7e800700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9d78102c30 0x7f9d78115ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:11:50.631 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:50.635+0000 7f9d7e800700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9d781035d0 0x7f9d781160e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:11:50.631 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:50.635+0000 7f9d7e800700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d78116730 con 0x7f9d781035d0 2024-04-12T21:11:50.631 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:50.635+0000 7f9d7e800700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d7810fb40 con 0x7f9d78102c30 2024-04-12T21:11:50.631 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:50.635+0000 7f9d7ca00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9d781035d0 0x7f9d781160e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:11:50.631 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:50.635+0000 7f9d7ca00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9d781035d0 0x7f9d781160e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:44520/0 (socket says 172.21.15.18:44520) 2024-04-12T21:11:50.631 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:50.635+0000 7f9d7ca00700 1 -- 172.21.15.18:0/2817814977 learned_addr learned my addr 172.21.15.18:0/2817814977 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:11:50.632 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:50.636+0000 7f9d7d400700 1 --2- 172.21.15.18:0/2817814977 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9d78102c30 0x7f9d78115ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:11:50.632 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:50.636+0000 7f9d7ca00700 1 -- 172.21.15.18:0/2817814977 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9d78102c30 msgr2=0x7f9d78115ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:11:50.632 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:50.636+0000 7f9d7ca00700 1 --2- 172.21.15.18:0/2817814977 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9d78102c30 0x7f9d78115ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:11:50.632 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:50.636+0000 7f9d7ca00700 1 -- 172.21.15.18:0/2817814977 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d64005040 con 0x7f9d781035d0 2024-04-12T21:11:50.632 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:50.636+0000 7f9d7ca00700 1 --2- 172.21.15.18:0/2817814977 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9d781035d0 0x7f9d781160e0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f9d64009100 tx=0x7f9d64009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:11:50.632 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:50.637+0000 7f9d75600700 1 -- 172.21.15.18:0/2817814977 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9d6400e040 con 0x7f9d781035d0 2024-04-12T21:11:50.633 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:50.637+0000 7f9d75600700 1 -- 172.21.15.18:0/2817814977 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f9d64013420 con 0x7f9d781035d0 2024-04-12T21:11:50.633 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:50.637+0000 7f9d7e800700 1 -- 172.21.15.18:0/2817814977 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9d7810fdc0 con 0x7f9d781035d0 2024-04-12T21:11:50.633 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:50.637+0000 7f9d75600700 1 -- 172.21.15.18:0/2817814977 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9d6401c630 con 0x7f9d781035d0 2024-04-12T21:11:50.633 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:50.637+0000 7f9d7e800700 1 -- 172.21.15.18:0/2817814977 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9d78110380 con 0x7f9d781035d0 2024-04-12T21:11:50.633 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:50.638+0000 7f9d7e800700 1 -- 172.21.15.18:0/2817814977 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9d780fe380 con 0x7f9d781035d0 2024-04-12T21:11:50.634 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:50.639+0000 7f9d75600700 1 -- 172.21.15.18:0/2817814977 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7f9d64013590 con 0x7f9d781035d0 2024-04-12T21:11:50.636 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:50.641+0000 7f9d75600700 1 --2- 172.21.15.18:0/2817814977 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f9d6006ec10 0x7f9d600710c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:11:50.636 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:50.641+0000 7f9d75600700 1 -- 172.21.15.18:0/2817814977 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 6757+0+0 (secure 0 0 0) 0x7f9d6408fba0 con 0x7f9d781035d0 2024-04-12T21:11:50.636 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:50.641+0000 7f9d7d400700 1 --2- 172.21.15.18:0/2817814977 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f9d6006ec10 0x7f9d600710c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:11:50.637 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:50.642+0000 7f9d7d400700 1 --2- 172.21.15.18:0/2817814977 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f9d6006ec10 0x7f9d600710c0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f9d6c007fd0 tx=0x7f9d6c009040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:11:50.641 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:50.646+0000 7f9d75600700 1 -- 172.21.15.18:0/2817814977 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f9d640588a0 con 0x7f9d781035d0 2024-04-12T21:11:50.890 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:50.895+0000 7f9d7e800700 1 -- 172.21.15.18:0/2817814977 --> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f9d78109d00 con 0x7f9d6006ec10 2024-04-12T21:11:50.893 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:50.897+0000 7f9d75600700 1 -- 172.21.15.18:0/2817814977 <== mgr.34102 v2:172.21.15.18:6800/3812891919 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f9d78109d00 con 0x7f9d6006ec10 2024-04-12T21:11:50.896 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:50.901+0000 7f9d7e800700 1 -- 172.21.15.18:0/2817814977 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f9d6006ec10 msgr2=0x7f9d600710c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:11:50.896 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:50.901+0000 7f9d7e800700 1 --2- 172.21.15.18:0/2817814977 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f9d6006ec10 0x7f9d600710c0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f9d6c007fd0 tx=0x7f9d6c009040 comp rx=0 tx=0).stop 2024-04-12T21:11:50.897 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:50.901+0000 7f9d7e800700 1 -- 172.21.15.18:0/2817814977 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9d781035d0 msgr2=0x7f9d781160e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:11:50.897 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:50.901+0000 7f9d7e800700 1 --2- 172.21.15.18:0/2817814977 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9d781035d0 0x7f9d781160e0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f9d64009100 tx=0x7f9d64009e30 comp rx=0 tx=0).stop 2024-04-12T21:11:50.897 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:50.901+0000 7f9d7e800700 1 -- 172.21.15.18:0/2817814977 shutdown_connections 2024-04-12T21:11:50.897 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:50.901+0000 7f9d7e800700 1 --2- 172.21.15.18:0/2817814977 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f9d6006ec10 0x7f9d600710c0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:11:50.897 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:50.901+0000 7f9d7e800700 1 --2- 172.21.15.18:0/2817814977 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9d78102c30 0x7f9d78115ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:11:50.897 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:50.901+0000 7f9d7e800700 1 --2- 172.21.15.18:0/2817814977 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9d781035d0 0x7f9d781160e0 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:11:50.897 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:50.901+0000 7f9d7e800700 1 -- 172.21.15.18:0/2817814977 >> 172.21.15.18:0/2817814977 conn(0x7f9d780f87a0 msgr2=0x7f9d780f97c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:11:50.897 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:50.901+0000 7f9d7e800700 1 -- 172.21.15.18:0/2817814977 shutdown_connections 2024-04-12T21:11:50.897 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:50.901+0000 7f9d7e800700 1 -- 172.21.15.18:0/2817814977 wait complete. 2024-04-12T21:11:50.913 INFO:teuthology.orchestra.run.smithi018.stdout:true 2024-04-12T21:11:51.051 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:51.056+0000 7f7a70000700 1 -- 172.21.15.18:0/413778692 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7a680fc620 msgr2=0x7f7a68100cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:11:51.052 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:51.056+0000 7f7a70000700 1 --2- 172.21.15.18:0/413778692 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7a680fc620 0x7f7a68100cd0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f7a54004940 tx=0x7f7a54009e30 comp rx=0 tx=0).stop 2024-04-12T21:11:51.052 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:51.056+0000 7f7a70000700 1 -- 172.21.15.18:0/413778692 shutdown_connections 2024-04-12T21:11:51.052 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:51.056+0000 7f7a70000700 1 --2- 172.21.15.18:0/413778692 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7a680fc620 0x7f7a68100cd0 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:11:51.052 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:51.056+0000 7f7a70000700 1 --2- 172.21.15.18:0/413778692 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7a680fbc80 0x7f7a680fc050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:11:51.052 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:51.056+0000 7f7a70000700 1 -- 172.21.15.18:0/413778692 >> 172.21.15.18:0/413778692 conn(0x7f7a68072bc0 msgr2=0x7f7a68072fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:11:51.052 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:51.056+0000 7f7a70000700 1 -- 172.21.15.18:0/413778692 shutdown_connections 2024-04-12T21:11:51.052 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:51.056+0000 7f7a70000700 1 -- 172.21.15.18:0/413778692 wait complete. 2024-04-12T21:11:51.054 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:51.057+0000 7f7a70000700 1 Processor -- start 2024-04-12T21:11:51.054 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:51.057+0000 7f7a70000700 1 -- start start 2024-04-12T21:11:51.054 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:51.057+0000 7f7a70000700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7a680fbc80 0x7f7a6811a1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:11:51.054 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:51.057+0000 7f7a70000700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7a680fc620 0x7f7a6811a730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:11:51.054 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:51.057+0000 7f7a70000700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a6811ad80 con 0x7f7a680fc620 2024-04-12T21:11:51.054 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:51.057+0000 7f7a70000700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a68114150 con 0x7f7a680fbc80 2024-04-12T21:11:51.054 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:51.057+0000 7f7a6cc00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7a680fc620 0x7f7a6811a730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:11:51.055 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:51.058+0000 7f7a6cc00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7a680fc620 0x7f7a6811a730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:46564/0 (socket says 172.21.15.18:46564) 2024-04-12T21:11:51.055 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:51.058+0000 7f7a6cc00700 1 -- 172.21.15.18:0/280233941 learned_addr learned my addr 172.21.15.18:0/280233941 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:11:51.055 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:51.058+0000 7f7a6d600700 1 --2- 172.21.15.18:0/280233941 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7a680fbc80 0x7f7a6811a1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:11:51.055 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:51.058+0000 7f7a6cc00700 1 -- 172.21.15.18:0/280233941 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7a680fbc80 msgr2=0x7f7a6811a1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:11:51.055 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:51.058+0000 7f7a6cc00700 1 --2- 172.21.15.18:0/280233941 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7a680fbc80 0x7f7a6811a1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:11:51.055 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:51.058+0000 7f7a6cc00700 1 -- 172.21.15.18:0/280233941 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a54005040 con 0x7f7a680fc620 2024-04-12T21:11:51.055 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:51.058+0000 7f7a6cc00700 1 --2- 172.21.15.18:0/280233941 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7a680fc620 0x7f7a6811a730 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f7a54009180 tx=0x7f7a54009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:11:51.055 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:51.059+0000 7f7a65600700 1 -- 172.21.15.18:0/280233941 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7a5400e040 con 0x7f7a680fc620 2024-04-12T21:11:51.055 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:51.059+0000 7f7a65600700 1 -- 172.21.15.18:0/280233941 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f7a5400bbc0 con 0x7f7a680fc620 2024-04-12T21:11:51.055 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:51.059+0000 7f7a65600700 1 -- 172.21.15.18:0/280233941 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7a5401c630 con 0x7f7a680fc620 2024-04-12T21:11:51.055 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:51.059+0000 7f7a70000700 1 -- 172.21.15.18:0/280233941 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7a681143d0 con 0x7f7a680fc620 2024-04-12T21:11:51.055 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:51.059+0000 7f7a70000700 1 -- 172.21.15.18:0/280233941 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7a68114910 con 0x7f7a680fc620 2024-04-12T21:11:51.056 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:51.060+0000 7f7a70000700 1 -- 172.21.15.18:0/280233941 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7a68065e30 con 0x7f7a680fc620 2024-04-12T21:11:51.056 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:51.061+0000 7f7a65600700 1 -- 172.21.15.18:0/280233941 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7f7a5400bd30 con 0x7f7a680fc620 2024-04-12T21:11:51.057 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:51.062+0000 7f7a65600700 1 --2- 172.21.15.18:0/280233941 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f7a5006ec10 0x7f7a500710c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:11:51.057 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:51.062+0000 7f7a65600700 1 -- 172.21.15.18:0/280233941 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 6757+0+0 (secure 0 0 0) 0x7f7a5408fcb0 con 0x7f7a680fc620 2024-04-12T21:11:51.058 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:51.062+0000 7f7a6d600700 1 --2- 172.21.15.18:0/280233941 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f7a5006ec10 0x7f7a500710c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:11:51.058 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:51.063+0000 7f7a6d600700 1 --2- 172.21.15.18:0/280233941 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f7a5006ec10 0x7f7a500710c0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f7a5c007fd0 tx=0x7f7a5c009040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:11:51.062 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:51.067+0000 7f7a65600700 1 -- 172.21.15.18:0/280233941 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f7a5405c5a0 con 0x7f7a680fc620 2024-04-12T21:11:51.315 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:51.319+0000 7f7a70000700 1 -- 172.21.15.18:0/280233941 --> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f7a68110580 con 0x7f7a5006ec10 2024-04-12T21:11:51.316 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:51.321+0000 7f7a65600700 1 -- 172.21.15.18:0/280233941 <== mgr.34102 v2:172.21.15.18:6800/3812891919 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f7a68110580 con 0x7f7a5006ec10 2024-04-12T21:11:51.320 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:51.324+0000 7f7a70000700 1 -- 172.21.15.18:0/280233941 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f7a5006ec10 msgr2=0x7f7a500710c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:11:51.320 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:51.324+0000 7f7a70000700 1 --2- 172.21.15.18:0/280233941 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f7a5006ec10 0x7f7a500710c0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f7a5c007fd0 tx=0x7f7a5c009040 comp rx=0 tx=0).stop 2024-04-12T21:11:51.320 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:51.324+0000 7f7a70000700 1 -- 172.21.15.18:0/280233941 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7a680fc620 msgr2=0x7f7a6811a730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:11:51.320 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:51.324+0000 7f7a70000700 1 --2- 172.21.15.18:0/280233941 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7a680fc620 0x7f7a6811a730 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f7a54009180 tx=0x7f7a54009e30 comp rx=0 tx=0).stop 2024-04-12T21:11:51.320 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:51.324+0000 7f7a70000700 1 -- 172.21.15.18:0/280233941 shutdown_connections 2024-04-12T21:11:51.321 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:51.324+0000 7f7a70000700 1 --2- 172.21.15.18:0/280233941 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f7a5006ec10 0x7f7a500710c0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:11:51.321 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:51.324+0000 7f7a70000700 1 --2- 172.21.15.18:0/280233941 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7a680fbc80 0x7f7a6811a1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:11:51.321 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:51.324+0000 7f7a70000700 1 --2- 172.21.15.18:0/280233941 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7a680fc620 0x7f7a6811a730 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:11:51.321 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:51.324+0000 7f7a70000700 1 -- 172.21.15.18:0/280233941 >> 172.21.15.18:0/280233941 conn(0x7f7a68072bc0 msgr2=0x7f7a6810aab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:11:51.321 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:51.325+0000 7f7a70000700 1 -- 172.21.15.18:0/280233941 shutdown_connections 2024-04-12T21:11:51.321 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:51.325+0000 7f7a70000700 1 -- 172.21.15.18:0/280233941 wait complete. 2024-04-12T21:11:51.480 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:51.485+0000 7f581b800700 1 -- 172.21.15.18:0/2543128362 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f58140fde90 msgr2=0x7f58140fe260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:11:51.480 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:51.485+0000 7f581b800700 1 --2- 172.21.15.18:0/2543128362 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f58140fde90 0x7f58140fe260 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f5808004940 tx=0x7f5808009e30 comp rx=0 tx=0).stop 2024-04-12T21:11:51.481 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:11:51 smithi018 ceph-mon[112590]: pgmap v361: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:11:51.482 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:51.485+0000 7f581b800700 1 -- 172.21.15.18:0/2543128362 shutdown_connections 2024-04-12T21:11:51.482 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:51.485+0000 7f581b800700 1 --2- 172.21.15.18:0/2543128362 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f58140fe830 0x7f5814106f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:11:51.482 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:51.485+0000 7f581b800700 1 --2- 172.21.15.18:0/2543128362 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f58140fde90 0x7f58140fe260 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:11:51.482 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:51.485+0000 7f581b800700 1 -- 172.21.15.18:0/2543128362 >> 172.21.15.18:0/2543128362 conn(0x7f5814072bc0 msgr2=0x7f5814072fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:11:51.482 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:51.485+0000 7f581b800700 1 -- 172.21.15.18:0/2543128362 shutdown_connections 2024-04-12T21:11:51.482 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:51.485+0000 7f581b800700 1 -- 172.21.15.18:0/2543128362 wait complete. 2024-04-12T21:11:51.482 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:51.486+0000 7f581b800700 1 Processor -- start 2024-04-12T21:11:51.482 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:51.486+0000 7f581b800700 1 -- start start 2024-04-12T21:11:51.484 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:51.487+0000 7f581b800700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f58140fde90 0x7f58140fbd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:11:51.484 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:51.487+0000 7f581b800700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f58140fe830 0x7f58140fc290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:11:51.484 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:51.487+0000 7f581b800700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58140fc8b0 con 0x7f58140fde90 2024-04-12T21:11:51.484 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:51.487+0000 7f581b800700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58140fc9f0 con 0x7f58140fe830 2024-04-12T21:11:51.484 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:51.487+0000 7f5818e00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f58140fde90 0x7f58140fbd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:11:51.484 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:51.487+0000 7f5818e00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f58140fde90 0x7f58140fbd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:46586/0 (socket says 172.21.15.18:46586) 2024-04-12T21:11:51.484 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:51.487+0000 7f5818e00700 1 -- 172.21.15.18:0/660339464 learned_addr learned my addr 172.21.15.18:0/660339464 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:11:51.484 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:51.488+0000 7f5813e00700 1 --2- 172.21.15.18:0/660339464 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f58140fe830 0x7f58140fc290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:11:51.484 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:51.488+0000 7f5818e00700 1 -- 172.21.15.18:0/660339464 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f58140fe830 msgr2=0x7f58140fc290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:11:51.484 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:51.488+0000 7f5818e00700 1 --2- 172.21.15.18:0/660339464 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f58140fe830 0x7f58140fc290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:11:51.484 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:51.488+0000 7f5818e00700 1 -- 172.21.15.18:0/660339464 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5808005040 con 0x7f58140fde90 2024-04-12T21:11:51.485 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:51.488+0000 7f5813e00700 1 --2- 172.21.15.18:0/660339464 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f58140fe830 0x7f58140fc290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-12T21:11:51.485 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:51.488+0000 7f5818e00700 1 --2- 172.21.15.18:0/660339464 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f58140fde90 0x7f58140fbd50 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f5808009360 tx=0x7f5808009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:11:51.485 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:51.488+0000 7f5810c00700 1 -- 172.21.15.18:0/660339464 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f580800e040 con 0x7f58140fde90 2024-04-12T21:11:51.485 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:51.488+0000 7f5810c00700 1 -- 172.21.15.18:0/660339464 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f5808013420 con 0x7f58140fde90 2024-04-12T21:11:51.485 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:51.488+0000 7f581b800700 1 -- 172.21.15.18:0/660339464 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5814119ed0 con 0x7f58140fde90 2024-04-12T21:11:51.485 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:51.488+0000 7f5810c00700 1 -- 172.21.15.18:0/660339464 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f580801c630 con 0x7f58140fde90 2024-04-12T21:11:51.485 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:51.489+0000 7f581b800700 1 -- 172.21.15.18:0/660339464 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f581411a330 con 0x7f58140fde90 2024-04-12T21:11:51.485 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:51.490+0000 7f581b800700 1 -- 172.21.15.18:0/660339464 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5814065e30 con 0x7f58140fde90 2024-04-12T21:11:51.486 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:51.491+0000 7f5810c00700 1 -- 172.21.15.18:0/660339464 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7f5808013a90 con 0x7f58140fde90 2024-04-12T21:11:51.487 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:51.492+0000 7f5810c00700 1 --2- 172.21.15.18:0/660339464 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f57fc06ec00 0x7f57fc0710b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:11:51.487 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:51.492+0000 7f5810c00700 1 -- 172.21.15.18:0/660339464 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 6757+0+0 (secure 0 0 0) 0x7f580808fb60 con 0x7f58140fde90 2024-04-12T21:11:51.488 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:51.492+0000 7f5813e00700 1 --2- 172.21.15.18:0/660339464 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f57fc06ec00 0x7f57fc0710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:11:51.488 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:51.493+0000 7f5813e00700 1 --2- 172.21.15.18:0/660339464 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f57fc06ec00 0x7f57fc0710b0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f5800006fd0 tx=0x7f5800008040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:11:51.492 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:51.496+0000 7f5810c00700 1 -- 172.21.15.18:0/660339464 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f5808058860 con 0x7f58140fde90 2024-04-12T21:11:51.529 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:11:51 smithi186 ceph-mon[86379]: pgmap v361: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:11:51.721 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:51.726+0000 7f581b800700 1 -- 172.21.15.18:0/660339464 --> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f581404bed0 con 0x7f57fc06ec00 2024-04-12T21:11:51.734 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:51.738+0000 7f5810c00700 1 -- 172.21.15.18:0/660339464 <== mgr.34102 v2:172.21.15.18:6800/3812891919 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7f581404bed0 con 0x7f57fc06ec00 2024-04-12T21:11:51.734 INFO:teuthology.orchestra.run.smithi018.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-12T21:11:51.734 INFO:teuthology.orchestra.run.smithi018.stdout:alertmanager.smithi018 smithi018 *:9093,9094 running (12m) 4m ago 26m 14.7M - 0.25.0 c8568f914cd2 c332090068e8 2024-04-12T21:11:51.735 INFO:teuthology.orchestra.run.smithi018.stdout:crash.smithi018 smithi018 running (8m) 4m ago 27m 6665k - 19.0.0-2879-g324baf8d 8717720dc5e5 b7f80d1f7982 2024-04-12T21:11:51.735 INFO:teuthology.orchestra.run.smithi018.stdout:crash.smithi186 smithi186 running (8m) 5m ago 25m 6660k - 19.0.0-2879-g324baf8d 8717720dc5e5 a4f55c81b623 2024-04-12T21:11:51.735 INFO:teuthology.orchestra.run.smithi018.stdout:grafana.smithi018 smithi018 *:3000 running (11m) 4m ago 26m 74.8M - 9.4.12 2bacad6d85d8 b08e3bf176dd 2024-04-12T21:11:51.735 INFO:teuthology.orchestra.run.smithi018.stdout:mds.cephfs.smithi018.acqbsc smithi018 running (4m) 4m ago 19m 11.1M - 19.0.0-2879-g324baf8d 8717720dc5e5 021f99ab24ae 2024-04-12T21:11:51.735 INFO:teuthology.orchestra.run.smithi018.stdout:mds.cephfs.smithi018.szjluo smithi018 running (4m) 4m ago 19m 10.9M - 19.0.0-2879-g324baf8d 8717720dc5e5 0003117479d4 2024-04-12T21:11:51.735 INFO:teuthology.orchestra.run.smithi018.stdout:mds.cephfs.smithi186.jdllyn smithi186 running (19m) 5m ago 19m 15.6M - 17.2.7-827-gd3b5060a 21de9b7369ad 2b2bc46bd29a 2024-04-12T21:11:51.735 INFO:teuthology.orchestra.run.smithi018.stdout:mds.cephfs.smithi186.wjnafs smithi186 running (19m) 5m ago 19m 15.4M - 17.2.7-827-gd3b5060a 21de9b7369ad 74b115d6d6fd 2024-04-12T21:11:51.735 INFO:teuthology.orchestra.run.smithi018.stdout:mgr.smithi018.xjzwga smithi018 *:8443,9283,8765 running (15m) 4m ago 28m 509M - 19.0.0-2879-g324baf8d 8717720dc5e5 56088305eab7 2024-04-12T21:11:51.735 INFO:teuthology.orchestra.run.smithi018.stdout:mgr.smithi186.kqcfxk smithi186 *:8443,9283,8765 running (13m) 5m ago 25m 452M - 19.0.0-2879-g324baf8d 8717720dc5e5 09cd766e5e0d 2024-04-12T21:11:51.735 INFO:teuthology.orchestra.run.smithi018.stdout:mon.smithi018 smithi018 running (10m) 4m ago 28m 57.0M 2048M 19.0.0-2879-g324baf8d 8717720dc5e5 df3e201dfed6 2024-04-12T21:11:51.735 INFO:teuthology.orchestra.run.smithi018.stdout:mon.smithi186 smithi186 running (9m) 5m ago 25m 37.7M 2048M 19.0.0-2879-g324baf8d 8717720dc5e5 c77498da9b2b 2024-04-12T21:11:51.735 INFO:teuthology.orchestra.run.smithi018.stdout:node-exporter.smithi018 smithi018 *:9100 running (13m) 4m ago 27m 7780k - 1.5.0 0da6a335fe13 326c3d9c6847 2024-04-12T21:11:51.735 INFO:teuthology.orchestra.run.smithi018.stdout:node-exporter.smithi186 smithi186 *:9100 running (13m) 5m ago 25m 7826k - 1.5.0 0da6a335fe13 ec2112baeecf 2024-04-12T21:11:51.735 INFO:teuthology.orchestra.run.smithi018.stdout:osd.0 smithi018 running (8m) 4m ago 24m 121M 1559M 19.0.0-2879-g324baf8d 8717720dc5e5 60b1273ca047 2024-04-12T21:11:51.735 INFO:teuthology.orchestra.run.smithi018.stdout:osd.1 smithi018 running (7m) 4m ago 23m 91.3M 1559M 19.0.0-2879-g324baf8d 8717720dc5e5 170ce1e5fb8d 2024-04-12T21:11:51.736 INFO:teuthology.orchestra.run.smithi018.stdout:osd.2 smithi018 running (7m) 4m ago 23m 90.3M 1559M 19.0.0-2879-g324baf8d 8717720dc5e5 c2cbc6aa2839 2024-04-12T21:11:51.736 INFO:teuthology.orchestra.run.smithi018.stdout:osd.3 smithi186 running (6m) 5m ago 22m 96.6M 2581M 19.0.0-2879-g324baf8d 8717720dc5e5 0138e8262e4b 2024-04-12T21:11:51.736 INFO:teuthology.orchestra.run.smithi018.stdout:osd.4 smithi186 running (5m) 5m ago 22m 87.2M 2581M 19.0.0-2879-g324baf8d 8717720dc5e5 b20cb63b25c9 2024-04-12T21:11:51.736 INFO:teuthology.orchestra.run.smithi018.stdout:osd.5 smithi186 running (5m) 5m ago 21m 11.1M 2581M 19.0.0-2879-g324baf8d 8717720dc5e5 2513bea2610e 2024-04-12T21:11:51.736 INFO:teuthology.orchestra.run.smithi018.stdout:prometheus.smithi018 smithi018 *:9095 running (12m) 4m ago 26m 60.6M - 2.43.0 a07b618ecd1d f58841127e21 2024-04-12T21:11:51.738 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:51.742+0000 7f581b800700 1 -- 172.21.15.18:0/660339464 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f57fc06ec00 msgr2=0x7f57fc0710b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:11:51.738 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:51.742+0000 7f581b800700 1 --2- 172.21.15.18:0/660339464 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f57fc06ec00 0x7f57fc0710b0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f5800006fd0 tx=0x7f5800008040 comp rx=0 tx=0).stop 2024-04-12T21:11:51.738 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:51.742+0000 7f581b800700 1 -- 172.21.15.18:0/660339464 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f58140fde90 msgr2=0x7f58140fbd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:11:51.738 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:51.742+0000 7f581b800700 1 --2- 172.21.15.18:0/660339464 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f58140fde90 0x7f58140fbd50 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f5808009360 tx=0x7f5808009e30 comp rx=0 tx=0).stop 2024-04-12T21:11:51.738 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:51.742+0000 7f581b800700 1 -- 172.21.15.18:0/660339464 shutdown_connections 2024-04-12T21:11:51.738 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:51.742+0000 7f581b800700 1 --2- 172.21.15.18:0/660339464 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f57fc06ec00 0x7f57fc0710b0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:11:51.738 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:51.742+0000 7f581b800700 1 --2- 172.21.15.18:0/660339464 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f58140fde90 0x7f58140fbd50 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:11:51.738 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:51.742+0000 7f581b800700 1 --2- 172.21.15.18:0/660339464 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f58140fe830 0x7f58140fc290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:11:51.738 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:51.742+0000 7f581b800700 1 -- 172.21.15.18:0/660339464 >> 172.21.15.18:0/660339464 conn(0x7f5814072bc0 msgr2=0x7f58140ff440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:11:51.738 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:51.742+0000 7f581b800700 1 -- 172.21.15.18:0/660339464 shutdown_connections 2024-04-12T21:11:51.739 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:51.742+0000 7f581b800700 1 -- 172.21.15.18:0/660339464 wait complete. 2024-04-12T21:11:51.894 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:51.898+0000 7f7910e00700 1 -- 172.21.15.18:0/4256797816 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f790c0fddf0 msgr2=0x7f790c0fe1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:11:51.894 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:51.898+0000 7f7910e00700 1 --2- 172.21.15.18:0/4256797816 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f790c0fddf0 0x7f790c0fe1c0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f78fc004940 tx=0x7f78fc009e30 comp rx=0 tx=0).stop 2024-04-12T21:11:51.895 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:51.899+0000 7f7910e00700 1 -- 172.21.15.18:0/4256797816 shutdown_connections 2024-04-12T21:11:51.895 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:51.899+0000 7f7910e00700 1 --2- 172.21.15.18:0/4256797816 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f790c0fe790 0x7f790c106dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:11:51.895 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:51.899+0000 7f7910e00700 1 --2- 172.21.15.18:0/4256797816 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f790c0fddf0 0x7f790c0fe1c0 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:11:51.895 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:51.899+0000 7f7910e00700 1 -- 172.21.15.18:0/4256797816 >> 172.21.15.18:0/4256797816 conn(0x7f790c0f7860 msgr2=0x7f790c0f9c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:11:51.895 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:51.899+0000 7f7910e00700 1 -- 172.21.15.18:0/4256797816 shutdown_connections 2024-04-12T21:11:51.895 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:51.899+0000 7f7910e00700 1 -- 172.21.15.18:0/4256797816 wait complete. 2024-04-12T21:11:51.896 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:51.900+0000 7f7910e00700 1 Processor -- start 2024-04-12T21:11:51.896 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:51.900+0000 7f7910e00700 1 -- start start 2024-04-12T21:11:51.896 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:51.900+0000 7f7910e00700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f790c0fddf0 0x7f790c115d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:11:51.896 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:51.900+0000 7f7910e00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f790c0fe790 0x7f790c1162b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:11:51.896 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:51.900+0000 7f7910e00700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f790c116900 con 0x7f790c0fe790 2024-04-12T21:11:51.896 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:51.900+0000 7f7910e00700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f790c10fcd0 con 0x7f790c0fddf0 2024-04-12T21:11:51.896 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:51.901+0000 7f7909400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f790c0fe790 0x7f790c1162b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:11:51.896 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:51.901+0000 7f7909400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f790c0fe790 0x7f790c1162b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:46606/0 (socket says 172.21.15.18:46606) 2024-04-12T21:11:51.897 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:51.901+0000 7f7909400700 1 -- 172.21.15.18:0/548408227 learned_addr learned my addr 172.21.15.18:0/548408227 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:11:51.897 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:51.901+0000 7f7909e00700 1 --2- 172.21.15.18:0/548408227 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f790c0fddf0 0x7f790c115d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:11:51.898 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:51.901+0000 7f7909400700 1 -- 172.21.15.18:0/548408227 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f790c0fddf0 msgr2=0x7f790c115d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:11:51.898 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:51.901+0000 7f7909400700 1 --2- 172.21.15.18:0/548408227 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f790c0fddf0 0x7f790c115d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:11:51.898 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:51.901+0000 7f7909400700 1 -- 172.21.15.18:0/548408227 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78fc005040 con 0x7f790c0fe790 2024-04-12T21:11:51.898 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:51.902+0000 7f7909400700 1 --2- 172.21.15.18:0/548408227 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f790c0fe790 0x7f790c1162b0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f78f8002a50 tx=0x7f78f80046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:11:51.898 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:51.902+0000 7f7902000700 1 -- 172.21.15.18:0/548408227 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f78f8007cc0 con 0x7f790c0fe790 2024-04-12T21:11:51.898 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:51.902+0000 7f7902000700 1 -- 172.21.15.18:0/548408227 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f78f800f040 con 0x7f790c0fe790 2024-04-12T21:11:51.898 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:51.902+0000 7f7902000700 1 -- 172.21.15.18:0/548408227 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f78f8009640 con 0x7f790c0fe790 2024-04-12T21:11:51.898 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:51.902+0000 7f7910e00700 1 -- 172.21.15.18:0/548408227 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f790c10ffb0 con 0x7f790c0fe790 2024-04-12T21:11:51.898 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:51.902+0000 7f7910e00700 1 -- 172.21.15.18:0/548408227 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f790c1104f0 con 0x7f790c0fe790 2024-04-12T21:11:51.899 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:51.903+0000 7f7910e00700 1 -- 172.21.15.18:0/548408227 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f790c066770 con 0x7f790c0fe790 2024-04-12T21:11:51.900 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:51.905+0000 7f7902000700 1 -- 172.21.15.18:0/548408227 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7f78f80097a0 con 0x7f790c0fe790 2024-04-12T21:11:51.901 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:51.905+0000 7f7902000700 1 --2- 172.21.15.18:0/548408227 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f78f006ec10 0x7f78f00710c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:11:51.901 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:51.906+0000 7f7902000700 1 -- 172.21.15.18:0/548408227 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 6757+0+0 (secure 0 0 0) 0x7f78f808e6c0 con 0x7f790c0fe790 2024-04-12T21:11:51.901 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:51.906+0000 7f7909e00700 1 --2- 172.21.15.18:0/548408227 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f78f006ec10 0x7f78f00710c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:11:51.902 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:51.907+0000 7f7909e00700 1 --2- 172.21.15.18:0/548408227 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f78f006ec10 0x7f78f00710c0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f78fc000c00 tx=0x7f78fc009e30 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:11:51.906 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:51.911+0000 7f7902000700 1 -- 172.21.15.18:0/548408227 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f78f805af10 con 0x7f790c0fe790 2024-04-12T21:11:52.232 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:52.237+0000 7f7910e00700 1 -- 172.21.15.18:0/548408227 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f790c04bed0 con 0x7f790c0fe790 2024-04-12T21:11:52.233 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:52.237+0000 7f7902000700 1 -- 172.21.15.18:0/548408227 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+837 (secure 0 0 0) 0x7f78f805aaa0 con 0x7f790c0fe790 2024-04-12T21:11:52.233 INFO:teuthology.orchestra.run.smithi018.stdout:{ 2024-04-12T21:11:52.233 INFO:teuthology.orchestra.run.smithi018.stdout: "mon": { 2024-04-12T21:11:52.233 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 19.0.0-2879-g324baf8d (324baf8ddb91dea46cac50cb535c306115791355) squid (dev)": 2 2024-04-12T21:11:52.234 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-12T21:11:52.234 INFO:teuthology.orchestra.run.smithi018.stdout: "mgr": { 2024-04-12T21:11:52.234 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 19.0.0-2879-g324baf8d (324baf8ddb91dea46cac50cb535c306115791355) squid (dev)": 2 2024-04-12T21:11:52.234 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-12T21:11:52.234 INFO:teuthology.orchestra.run.smithi018.stdout: "osd": { 2024-04-12T21:11:52.234 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 19.0.0-2879-g324baf8d (324baf8ddb91dea46cac50cb535c306115791355) squid (dev)": 6 2024-04-12T21:11:52.234 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-12T21:11:52.234 INFO:teuthology.orchestra.run.smithi018.stdout: "mds": { 2024-04-12T21:11:52.234 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 17.2.7-827-gd3b5060a (d3b5060a17449edfae9385cdf792afec109fca5d) quincy (stable)": 2, 2024-04-12T21:11:52.234 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 19.0.0-2879-g324baf8d (324baf8ddb91dea46cac50cb535c306115791355) squid (dev)": 2 2024-04-12T21:11:52.234 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-12T21:11:52.234 INFO:teuthology.orchestra.run.smithi018.stdout: "overall": { 2024-04-12T21:11:52.234 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 17.2.7-827-gd3b5060a (d3b5060a17449edfae9385cdf792afec109fca5d) quincy (stable)": 2, 2024-04-12T21:11:52.235 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 19.0.0-2879-g324baf8d (324baf8ddb91dea46cac50cb535c306115791355) squid (dev)": 12 2024-04-12T21:11:52.235 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-04-12T21:11:52.235 INFO:teuthology.orchestra.run.smithi018.stdout:} 2024-04-12T21:11:52.236 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:52.240+0000 7f7910e00700 1 -- 172.21.15.18:0/548408227 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f78f006ec10 msgr2=0x7f78f00710c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:11:52.236 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:52.240+0000 7f7910e00700 1 --2- 172.21.15.18:0/548408227 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f78f006ec10 0x7f78f00710c0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f78fc000c00 tx=0x7f78fc009e30 comp rx=0 tx=0).stop 2024-04-12T21:11:52.237 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:52.240+0000 7f7910e00700 1 -- 172.21.15.18:0/548408227 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f790c0fe790 msgr2=0x7f790c1162b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:11:52.237 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:52.240+0000 7f7910e00700 1 --2- 172.21.15.18:0/548408227 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f790c0fe790 0x7f790c1162b0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f78f8002a50 tx=0x7f78f80046b0 comp rx=0 tx=0).stop 2024-04-12T21:11:52.237 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:52.241+0000 7f7910e00700 1 -- 172.21.15.18:0/548408227 shutdown_connections 2024-04-12T21:11:52.237 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:52.241+0000 7f7910e00700 1 --2- 172.21.15.18:0/548408227 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f78f006ec10 0x7f78f00710c0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:11:52.237 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:52.241+0000 7f7910e00700 1 --2- 172.21.15.18:0/548408227 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f790c0fddf0 0x7f790c115d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:11:52.237 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:52.241+0000 7f7910e00700 1 --2- 172.21.15.18:0/548408227 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f790c0fe790 0x7f790c1162b0 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:11:52.237 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:52.241+0000 7f7910e00700 1 -- 172.21.15.18:0/548408227 >> 172.21.15.18:0/548408227 conn(0x7f790c0f7860 msgr2=0x7f790c0f8ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:11:52.237 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:52.241+0000 7f7910e00700 1 -- 172.21.15.18:0/548408227 shutdown_connections 2024-04-12T21:11:52.237 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:52.241+0000 7f7910e00700 1 -- 172.21.15.18:0/548408227 wait complete. 2024-04-12T21:11:52.392 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:52.396+0000 7f9583400700 1 -- 172.21.15.18:0/2481692423 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f957c1037a0 msgr2=0x7f957c0fce10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:11:52.392 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:52.396+0000 7f9583400700 1 --2- 172.21.15.18:0/2481692423 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f957c1037a0 0x7f957c0fce10 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f9564004940 tx=0x7f9564009e30 comp rx=0 tx=0).stop 2024-04-12T21:11:52.392 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:52.396+0000 7f9583400700 1 -- 172.21.15.18:0/2481692423 shutdown_connections 2024-04-12T21:11:52.392 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:52.396+0000 7f9583400700 1 --2- 172.21.15.18:0/2481692423 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f957c1037a0 0x7f957c0fce10 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:11:52.392 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:52.396+0000 7f9583400700 1 --2- 172.21.15.18:0/2481692423 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f957c102e00 0x7f957c1031d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:11:52.392 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:52.396+0000 7f9583400700 1 -- 172.21.15.18:0/2481692423 >> 172.21.15.18:0/2481692423 conn(0x7f957c0f89b0 msgr2=0x7f957c0fadc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:11:52.392 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:52.396+0000 7f9583400700 1 -- 172.21.15.18:0/2481692423 shutdown_connections 2024-04-12T21:11:52.392 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:52.396+0000 7f9583400700 1 -- 172.21.15.18:0/2481692423 wait complete. 2024-04-12T21:11:52.393 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:52.397+0000 7f9583400700 1 Processor -- start 2024-04-12T21:11:52.394 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:52.397+0000 7f9583400700 1 -- start start 2024-04-12T21:11:52.394 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:52.398+0000 7f9583400700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f957c102e00 0x7f957c113ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:11:52.394 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:52.398+0000 7f9583400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f957c1037a0 0x7f957c1140e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:11:52.394 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:52.398+0000 7f9583400700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f957c1147c0 con 0x7f957c1037a0 2024-04-12T21:11:52.394 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:52.398+0000 7f9583400700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f957c10db40 con 0x7f957c102e00 2024-04-12T21:11:52.394 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:52.398+0000 7f9573e00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f957c1037a0 0x7f957c1140e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:11:52.394 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:52.398+0000 7f9573e00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f957c1037a0 0x7f957c1140e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:46614/0 (socket says 172.21.15.18:46614) 2024-04-12T21:11:52.394 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:52.398+0000 7f9573e00700 1 -- 172.21.15.18:0/3138116388 learned_addr learned my addr 172.21.15.18:0/3138116388 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:11:52.394 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:52.398+0000 7f9580a00700 1 --2- 172.21.15.18:0/3138116388 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f957c102e00 0x7f957c113ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:11:52.395 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:52.398+0000 7f9573e00700 1 -- 172.21.15.18:0/3138116388 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f957c102e00 msgr2=0x7f957c113ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:11:52.395 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:52.398+0000 7f9573e00700 1 --2- 172.21.15.18:0/3138116388 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f957c102e00 0x7f957c113ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:11:52.395 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:52.398+0000 7f9573e00700 1 -- 172.21.15.18:0/3138116388 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9564005040 con 0x7f957c1037a0 2024-04-12T21:11:52.395 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:52.398+0000 7f9580a00700 1 --2- 172.21.15.18:0/3138116388 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f957c102e00 0x7f957c113ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-12T21:11:52.395 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:52.399+0000 7f9573e00700 1 --2- 172.21.15.18:0/3138116388 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f957c1037a0 0x7f957c1140e0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f9564009100 tx=0x7f9564009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:11:52.396 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:52.399+0000 7f9579600700 1 -- 172.21.15.18:0/3138116388 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f956400e040 con 0x7f957c1037a0 2024-04-12T21:11:52.396 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:52.399+0000 7f9579600700 1 -- 172.21.15.18:0/3138116388 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f9564013420 con 0x7f957c1037a0 2024-04-12T21:11:52.396 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:52.399+0000 7f9579600700 1 -- 172.21.15.18:0/3138116388 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f956401c630 con 0x7f957c1037a0 2024-04-12T21:11:52.396 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:52.399+0000 7f9583400700 1 -- 172.21.15.18:0/3138116388 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f957c10ddc0 con 0x7f957c1037a0 2024-04-12T21:11:52.396 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:52.399+0000 7f9583400700 1 -- 172.21.15.18:0/3138116388 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f957c10e380 con 0x7f957c1037a0 2024-04-12T21:11:52.396 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:52.400+0000 7f9583400700 1 -- 172.21.15.18:0/3138116388 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f957c100110 con 0x7f957c1037a0 2024-04-12T21:11:52.397 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:52.401+0000 7f9579600700 1 -- 172.21.15.18:0/3138116388 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7f9564013590 con 0x7f957c1037a0 2024-04-12T21:11:52.398 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:52.402+0000 7f9579600700 1 --2- 172.21.15.18:0/3138116388 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f956006ec10 0x7f95600710c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:11:52.398 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:52.403+0000 7f9579600700 1 -- 172.21.15.18:0/3138116388 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 6757+0+0 (secure 0 0 0) 0x7f956408fae0 con 0x7f957c1037a0 2024-04-12T21:11:52.398 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:52.403+0000 7f9580a00700 1 --2- 172.21.15.18:0/3138116388 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f956006ec10 0x7f95600710c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:11:52.399 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:52.404+0000 7f9580a00700 1 --2- 172.21.15.18:0/3138116388 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f956006ec10 0x7f95600710c0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f956c006fd0 tx=0x7f956c008040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:11:52.403 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:52.407+0000 7f9579600700 1 -- 172.21.15.18:0/3138116388 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f95640588a0 con 0x7f957c1037a0 2024-04-12T21:11:52.529 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:11:52 smithi186 ceph-mon[86379]: from='client.34668 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:11:52.529 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:11:52 smithi186 ceph-mon[86379]: from='client.34672 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:11:52.600 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:11:52 smithi018 ceph-mon[112590]: from='client.34668 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:11:52.600 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:11:52 smithi018 ceph-mon[112590]: from='client.34672 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:11:52.669 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:52.673+0000 7f9583400700 1 -- 172.21.15.18:0/3138116388 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f957c10f1f0 con 0x7f957c1037a0 2024-04-12T21:11:52.669 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:52.674+0000 7f9579600700 1 -- 172.21.15.18:0/3138116388 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 29 v29) v1 ==== 76+0+1872 (secure 0 0 0) 0x7f9564058030 con 0x7f957c1037a0 2024-04-12T21:11:52.671 INFO:teuthology.orchestra.run.smithi018.stdout:e29 2024-04-12T21:11:52.671 INFO:teuthology.orchestra.run.smithi018.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-12T21:11:52.671 INFO:teuthology.orchestra.run.smithi018.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-12T21:11:52.671 INFO:teuthology.orchestra.run.smithi018.stdout:legacy client fscid: 1 2024-04-12T21:11:52.672 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-12T21:11:52.672 INFO:teuthology.orchestra.run.smithi018.stdout:Filesystem 'cephfs' (1) 2024-04-12T21:11:52.672 INFO:teuthology.orchestra.run.smithi018.stdout:fs_name cephfs 2024-04-12T21:11:52.672 INFO:teuthology.orchestra.run.smithi018.stdout:epoch 24 2024-04-12T21:11:52.672 INFO:teuthology.orchestra.run.smithi018.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-04-12T21:11:52.672 INFO:teuthology.orchestra.run.smithi018.stdout:created 2024-04-12T20:51:57.362850+0000 2024-04-12T21:11:52.672 INFO:teuthology.orchestra.run.smithi018.stdout:modified 2024-04-12T21:06:59.717190+0000 2024-04-12T21:11:52.672 INFO:teuthology.orchestra.run.smithi018.stdout:tableserver 0 2024-04-12T21:11:52.672 INFO:teuthology.orchestra.run.smithi018.stdout:root 0 2024-04-12T21:11:52.672 INFO:teuthology.orchestra.run.smithi018.stdout:session_timeout 60 2024-04-12T21:11:52.672 INFO:teuthology.orchestra.run.smithi018.stdout:session_autoclose 300 2024-04-12T21:11:52.672 INFO:teuthology.orchestra.run.smithi018.stdout:max_file_size 1099511627776 2024-04-12T21:11:52.672 INFO:teuthology.orchestra.run.smithi018.stdout:max_xattr_size 65536 2024-04-12T21:11:52.672 INFO:teuthology.orchestra.run.smithi018.stdout:required_client_features {} 2024-04-12T21:11:52.672 INFO:teuthology.orchestra.run.smithi018.stdout:last_failure 0 2024-04-12T21:11:52.673 INFO:teuthology.orchestra.run.smithi018.stdout:last_failure_osd_epoch 100 2024-04-12T21:11:52.673 INFO:teuthology.orchestra.run.smithi018.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-12T21:11:52.673 INFO:teuthology.orchestra.run.smithi018.stdout:max_mds 2 2024-04-12T21:11:52.673 INFO:teuthology.orchestra.run.smithi018.stdout:in 0,1 2024-04-12T21:11:52.673 INFO:teuthology.orchestra.run.smithi018.stdout:up {} 2024-04-12T21:11:52.673 INFO:teuthology.orchestra.run.smithi018.stdout:failed 0,1 2024-04-12T21:11:52.673 INFO:teuthology.orchestra.run.smithi018.stdout:damaged 2024-04-12T21:11:52.673 INFO:teuthology.orchestra.run.smithi018.stdout:stopped 2024-04-12T21:11:52.673 INFO:teuthology.orchestra.run.smithi018.stdout:data_pools [3] 2024-04-12T21:11:52.673 INFO:teuthology.orchestra.run.smithi018.stdout:metadata_pool 2 2024-04-12T21:11:52.673 INFO:teuthology.orchestra.run.smithi018.stdout:inline_data disabled 2024-04-12T21:11:52.673 INFO:teuthology.orchestra.run.smithi018.stdout:balancer 2024-04-12T21:11:52.673 INFO:teuthology.orchestra.run.smithi018.stdout:bal_rank_mask -1 2024-04-12T21:11:52.673 INFO:teuthology.orchestra.run.smithi018.stdout:standby_count_wanted 1 2024-04-12T21:11:52.674 INFO:teuthology.orchestra.run.smithi018.stdout:qdb_cluster leader: 0 members: 2024-04-12T21:11:52.674 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-12T21:11:52.674 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-12T21:11:52.674 INFO:teuthology.orchestra.run.smithi018.stdout:Standby daemons: 2024-04-12T21:11:52.674 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-12T21:11:52.674 INFO:teuthology.orchestra.run.smithi018.stdout:[mds.cephfs.smithi186.wjnafs{-1:24277} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.186:6826/438347207,v1:172.21.15.186:6827/438347207] compat {c=[1],r=[1],i=[7ff]}] 2024-04-12T21:11:52.674 INFO:teuthology.orchestra.run.smithi018.stdout:[mds.cephfs.smithi186.jdllyn{-1:24281} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.186:6824/1464359952,v1:172.21.15.186:6825/1464359952] compat {c=[1],r=[1],i=[7ff]}] 2024-04-12T21:11:52.674 INFO:teuthology.orchestra.run.smithi018.stdout:[mds.cephfs.smithi018.acqbsc{-1:34436} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.18:6828/755179764,v1:172.21.15.18:6829/755179764] compat {c=[1],r=[1],i=[1fff]}] 2024-04-12T21:11:52.674 INFO:teuthology.orchestra.run.smithi018.stdout:[mds.cephfs.smithi018.szjluo{-1:34468} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.18:6826/2407992653,v1:172.21.15.18:6827/2407992653] compat {c=[1],r=[1],i=[1fff]}] 2024-04-12T21:11:52.675 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:52.677+0000 7f9583400700 1 -- 172.21.15.18:0/3138116388 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f956006ec10 msgr2=0x7f95600710c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:11:52.675 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:52.677+0000 7f9583400700 1 --2- 172.21.15.18:0/3138116388 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f956006ec10 0x7f95600710c0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f956c006fd0 tx=0x7f956c008040 comp rx=0 tx=0).stop 2024-04-12T21:11:52.675 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:52.677+0000 7f9583400700 1 -- 172.21.15.18:0/3138116388 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f957c1037a0 msgr2=0x7f957c1140e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:11:52.675 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:52.677+0000 7f9583400700 1 --2- 172.21.15.18:0/3138116388 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f957c1037a0 0x7f957c1140e0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f9564009100 tx=0x7f9564009e30 comp rx=0 tx=0).stop 2024-04-12T21:11:52.675 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:52.677+0000 7f9583400700 1 -- 172.21.15.18:0/3138116388 shutdown_connections 2024-04-12T21:11:52.675 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:52.677+0000 7f9583400700 1 --2- 172.21.15.18:0/3138116388 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f956006ec10 0x7f95600710c0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:11:52.675 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:52.677+0000 7f9583400700 1 --2- 172.21.15.18:0/3138116388 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f957c102e00 0x7f957c113ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:11:52.675 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:52.677+0000 7f9583400700 1 --2- 172.21.15.18:0/3138116388 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f957c1037a0 0x7f957c1140e0 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:11:52.676 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:52.677+0000 7f9583400700 1 -- 172.21.15.18:0/3138116388 >> 172.21.15.18:0/3138116388 conn(0x7f957c0f89b0 msgr2=0x7f957c0fa7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:11:52.676 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:52.677+0000 7f9583400700 1 -- 172.21.15.18:0/3138116388 shutdown_connections 2024-04-12T21:11:52.676 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:52.677+0000 7f9583400700 1 -- 172.21.15.18:0/3138116388 wait complete. 2024-04-12T21:11:52.676 INFO:teuthology.orchestra.run.smithi018.stderr:dumped fsmap epoch 29 2024-04-12T21:11:52.830 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:52.834+0000 7f86bc000700 1 -- 172.21.15.18:0/1336452883 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f86b4109430 msgr2=0x7f86b4109800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:11:52.831 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:52.834+0000 7f86bc000700 1 --2- 172.21.15.18:0/1336452883 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f86b4109430 0x7f86b4109800 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f86a8004940 tx=0x7f86a8009e30 comp rx=0 tx=0).stop 2024-04-12T21:11:52.831 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:52.835+0000 7f86bc000700 1 -- 172.21.15.18:0/1336452883 shutdown_connections 2024-04-12T21:11:52.831 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:52.835+0000 7f86bc000700 1 --2- 172.21.15.18:0/1336452883 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f86b4109dd0 0x7f86b4103440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:11:52.831 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:52.835+0000 7f86bc000700 1 --2- 172.21.15.18:0/1336452883 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f86b4109430 0x7f86b4109800 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:11:52.831 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:52.835+0000 7f86bc000700 1 -- 172.21.15.18:0/1336452883 >> 172.21.15.18:0/1336452883 conn(0x7f86b4072c00 msgr2=0x7f86b4075010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:11:52.831 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:52.835+0000 7f86bc000700 1 -- 172.21.15.18:0/1336452883 shutdown_connections 2024-04-12T21:11:52.831 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:52.835+0000 7f86bc000700 1 -- 172.21.15.18:0/1336452883 wait complete. 2024-04-12T21:11:52.833 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:52.836+0000 7f86bc000700 1 Processor -- start 2024-04-12T21:11:52.833 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:52.836+0000 7f86bc000700 1 -- start start 2024-04-12T21:11:52.833 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:52.836+0000 7f86bc000700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f86b4109430 0x7f86b4119f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:11:52.833 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:52.836+0000 7f86bc000700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f86b4109dd0 0x7f86b411a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:11:52.833 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:52.836+0000 7f86bc000700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86b411ab60 con 0x7f86b4109dd0 2024-04-12T21:11:52.833 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:52.836+0000 7f86bc000700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86b41140b0 con 0x7f86b4109430 2024-04-12T21:11:52.834 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:52.836+0000 7f86b8c00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f86b4109dd0 0x7f86b411a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:11:52.834 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:52.836+0000 7f86b8c00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f86b4109dd0 0x7f86b411a480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:46644/0 (socket says 172.21.15.18:46644) 2024-04-12T21:11:52.834 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:52.836+0000 7f86b8c00700 1 -- 172.21.15.18:0/3750548844 learned_addr learned my addr 172.21.15.18:0/3750548844 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:11:52.834 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:52.837+0000 7f86b9600700 1 --2- 172.21.15.18:0/3750548844 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f86b4109430 0x7f86b4119f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:11:52.834 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:52.837+0000 7f86b8c00700 1 -- 172.21.15.18:0/3750548844 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f86b4109430 msgr2=0x7f86b4119f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:11:52.834 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:52.837+0000 7f86b8c00700 1 --2- 172.21.15.18:0/3750548844 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f86b4109430 0x7f86b4119f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:11:52.834 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:52.837+0000 7f86b8c00700 1 -- 172.21.15.18:0/3750548844 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86a8005040 con 0x7f86b4109dd0 2024-04-12T21:11:52.834 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:52.837+0000 7f86b8c00700 1 --2- 172.21.15.18:0/3750548844 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f86b4109dd0 0x7f86b411a480 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f86a000b790 tx=0x7f86a0009220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:11:52.834 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:52.837+0000 7f86b1600700 1 -- 172.21.15.18:0/3750548844 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f86a000f6f0 con 0x7f86b4109dd0 2024-04-12T21:11:52.834 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:52.837+0000 7f86b1600700 1 -- 172.21.15.18:0/3750548844 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f86a000fd20 con 0x7f86b4109dd0 2024-04-12T21:11:52.834 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:52.837+0000 7f86b1600700 1 -- 172.21.15.18:0/3750548844 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f86a00103a0 con 0x7f86b4109dd0 2024-04-12T21:11:52.834 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:52.838+0000 7f86bc000700 1 -- 172.21.15.18:0/3750548844 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f86b4114330 con 0x7f86b4109dd0 2024-04-12T21:11:52.835 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:52.838+0000 7f86bc000700 1 -- 172.21.15.18:0/3750548844 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f86b41148f0 con 0x7f86b4109dd0 2024-04-12T21:11:52.835 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:52.839+0000 7f86bc000700 1 -- 172.21.15.18:0/3750548844 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f86b4104b80 con 0x7f86b4109dd0 2024-04-12T21:11:52.835 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:52.840+0000 7f86b1600700 1 -- 172.21.15.18:0/3750548844 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7f86a0011400 con 0x7f86b4109dd0 2024-04-12T21:11:52.837 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:52.841+0000 7f86b1600700 1 --2- 172.21.15.18:0/3750548844 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f869c06ec00 0x7f869c0710b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:11:52.837 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:52.841+0000 7f86b1600700 1 -- 172.21.15.18:0/3750548844 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 6757+0+0 (secure 0 0 0) 0x7f86a008e780 con 0x7f86b4109dd0 2024-04-12T21:11:52.837 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:52.842+0000 7f86b9600700 1 --2- 172.21.15.18:0/3750548844 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f869c06ec00 0x7f869c0710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:11:52.837 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:52.842+0000 7f86b9600700 1 --2- 172.21.15.18:0/3750548844 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f869c06ec00 0x7f869c0710b0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f86a800b3b0 tx=0x7f86a8009e30 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:11:52.841 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:52.846+0000 7f86b1600700 1 -- 172.21.15.18:0/3750548844 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f86a005afd0 con 0x7f86b4109dd0 2024-04-12T21:11:53.093 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:53.097+0000 7f86bc000700 1 -- 172.21.15.18:0/3750548844 --> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f86b4114c20 con 0x7f869c06ec00 2024-04-12T21:11:53.096 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:53.100+0000 7f86b1600700 1 -- 172.21.15.18:0/3750548844 <== mgr.34102 v2:172.21.15.18:6800/3812891919 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f86b4114c20 con 0x7f869c06ec00 2024-04-12T21:11:53.096 INFO:teuthology.orchestra.run.smithi018.stdout:{ 2024-04-12T21:11:53.096 INFO:teuthology.orchestra.run.smithi018.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:f4452def3be4d350f4a8e5feaf7ebb1511be9298fe68478ba7a2d6fbd7d69a63", 2024-04-12T21:11:53.096 INFO:teuthology.orchestra.run.smithi018.stdout: "in_progress": true, 2024-04-12T21:11:53.096 INFO:teuthology.orchestra.run.smithi018.stdout: "which": "Upgrading all daemon types on all hosts", 2024-04-12T21:11:53.096 INFO:teuthology.orchestra.run.smithi018.stdout: "services_complete": [ 2024-04-12T21:11:53.097 INFO:teuthology.orchestra.run.smithi018.stdout: "crash", 2024-04-12T21:11:53.097 INFO:teuthology.orchestra.run.smithi018.stdout: "osd", 2024-04-12T21:11:53.097 INFO:teuthology.orchestra.run.smithi018.stdout: "mon", 2024-04-12T21:11:53.097 INFO:teuthology.orchestra.run.smithi018.stdout: "mgr" 2024-04-12T21:11:53.097 INFO:teuthology.orchestra.run.smithi018.stdout: ], 2024-04-12T21:11:53.097 INFO:teuthology.orchestra.run.smithi018.stdout: "progress": "14/21 daemons upgraded", 2024-04-12T21:11:53.097 INFO:teuthology.orchestra.run.smithi018.stdout: "message": "Currently upgrading mds daemons", 2024-04-12T21:11:53.097 INFO:teuthology.orchestra.run.smithi018.stdout: "is_paused": false 2024-04-12T21:11:53.097 INFO:teuthology.orchestra.run.smithi018.stdout:} 2024-04-12T21:11:53.100 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:53.104+0000 7f86bc000700 1 -- 172.21.15.18:0/3750548844 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f869c06ec00 msgr2=0x7f869c0710b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:11:53.100 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:53.104+0000 7f86bc000700 1 --2- 172.21.15.18:0/3750548844 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f869c06ec00 0x7f869c0710b0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f86a800b3b0 tx=0x7f86a8009e30 comp rx=0 tx=0).stop 2024-04-12T21:11:53.100 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:53.104+0000 7f86bc000700 1 -- 172.21.15.18:0/3750548844 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f86b4109dd0 msgr2=0x7f86b411a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:11:53.100 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:53.104+0000 7f86bc000700 1 --2- 172.21.15.18:0/3750548844 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f86b4109dd0 0x7f86b411a480 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f86a000b790 tx=0x7f86a0009220 comp rx=0 tx=0).stop 2024-04-12T21:11:53.100 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:53.105+0000 7f86bc000700 1 -- 172.21.15.18:0/3750548844 shutdown_connections 2024-04-12T21:11:53.100 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:53.105+0000 7f86bc000700 1 --2- 172.21.15.18:0/3750548844 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f869c06ec00 0x7f869c0710b0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:11:53.101 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:53.105+0000 7f86bc000700 1 --2- 172.21.15.18:0/3750548844 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f86b4109430 0x7f86b4119f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:11:53.101 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:53.105+0000 7f86bc000700 1 --2- 172.21.15.18:0/3750548844 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f86b4109dd0 0x7f86b411a480 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:11:53.101 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:53.105+0000 7f86bc000700 1 -- 172.21.15.18:0/3750548844 >> 172.21.15.18:0/3750548844 conn(0x7f86b4072c00 msgr2=0x7f86b4074c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:11:53.101 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:53.105+0000 7f86bc000700 1 -- 172.21.15.18:0/3750548844 shutdown_connections 2024-04-12T21:11:53.101 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:53.105+0000 7f86bc000700 1 -- 172.21.15.18:0/3750548844 wait complete. 2024-04-12T21:11:53.265 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:53.269+0000 7f7839c00700 1 -- 172.21.15.18:0/593680704 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7834105000 msgr2=0x7f78341053d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:11:53.265 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:53.269+0000 7f7839c00700 1 --2- 172.21.15.18:0/593680704 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7834105000 0x7f78341053d0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f7824004940 tx=0x7f7824009e30 comp rx=0 tx=0).stop 2024-04-12T21:11:53.265 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:53.269+0000 7f7839c00700 1 -- 172.21.15.18:0/593680704 shutdown_connections 2024-04-12T21:11:53.265 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:53.269+0000 7f7839c00700 1 --2- 172.21.15.18:0/593680704 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f78341059a0 0x7f78340ff010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:11:53.265 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:53.269+0000 7f7839c00700 1 --2- 172.21.15.18:0/593680704 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7834105000 0x7f78341053d0 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:11:53.265 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:53.269+0000 7f7839c00700 1 -- 172.21.15.18:0/593680704 >> 172.21.15.18:0/593680704 conn(0x7f78340fab70 msgr2=0x7f78340fcf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:11:53.265 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:53.269+0000 7f7839c00700 1 -- 172.21.15.18:0/593680704 shutdown_connections 2024-04-12T21:11:53.266 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:53.269+0000 7f7839c00700 1 -- 172.21.15.18:0/593680704 wait complete. 2024-04-12T21:11:53.267 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:53.270+0000 7f7839c00700 1 Processor -- start 2024-04-12T21:11:53.267 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:53.270+0000 7f7839c00700 1 -- start start 2024-04-12T21:11:53.267 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:53.271+0000 7f7839c00700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7834105000 0x7f7834115b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:11:53.267 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:53.271+0000 7f7839c00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f78341059a0 0x7f78341160b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:11:53.267 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:53.271+0000 7f7839c00700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7834116700 con 0x7f78341059a0 2024-04-12T21:11:53.267 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:53.271+0000 7f7839c00700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f783410fce0 con 0x7f7834105000 2024-04-12T21:11:53.267 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:53.271+0000 7f7832000700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f78341059a0 0x7f78341160b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:11:53.267 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:53.271+0000 7f7832000700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f78341059a0 0x7f78341160b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:46658/0 (socket says 172.21.15.18:46658) 2024-04-12T21:11:53.267 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:53.271+0000 7f7832000700 1 -- 172.21.15.18:0/4018562176 learned_addr learned my addr 172.21.15.18:0/4018562176 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:11:53.267 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:53.272+0000 7f7832a00700 1 --2- 172.21.15.18:0/4018562176 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7834105000 0x7f7834115b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:11:53.268 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:53.272+0000 7f7832000700 1 -- 172.21.15.18:0/4018562176 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7834105000 msgr2=0x7f7834115b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:11:53.268 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:53.272+0000 7f7832000700 1 --2- 172.21.15.18:0/4018562176 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7834105000 0x7f7834115b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:11:53.268 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:53.272+0000 7f7832000700 1 -- 172.21.15.18:0/4018562176 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7824005040 con 0x7f78341059a0 2024-04-12T21:11:53.268 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:53.272+0000 7f7832000700 1 --2- 172.21.15.18:0/4018562176 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f78341059a0 0x7f78341160b0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f7820002a50 tx=0x7f7820004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:11:53.269 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:53.272+0000 7f782aa00700 1 -- 172.21.15.18:0/4018562176 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7820007cc0 con 0x7f78341059a0 2024-04-12T21:11:53.269 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:53.272+0000 7f782aa00700 1 -- 172.21.15.18:0/4018562176 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f782000f040 con 0x7f78341059a0 2024-04-12T21:11:53.269 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:53.272+0000 7f7839c00700 1 -- 172.21.15.18:0/4018562176 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f783410ffc0 con 0x7f78341059a0 2024-04-12T21:11:53.269 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:53.273+0000 7f782aa00700 1 -- 172.21.15.18:0/4018562176 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f782000daf0 con 0x7f78341059a0 2024-04-12T21:11:53.269 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:53.273+0000 7f7839c00700 1 -- 172.21.15.18:0/4018562176 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7834110580 con 0x7f78341059a0 2024-04-12T21:11:53.269 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:53.274+0000 7f7839c00700 1 -- 172.21.15.18:0/4018562176 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7834100750 con 0x7f78341059a0 2024-04-12T21:11:53.270 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:53.275+0000 7f782aa00700 1 -- 172.21.15.18:0/4018562176 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7f7820009810 con 0x7f78341059a0 2024-04-12T21:11:53.272 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:53.276+0000 7f782aa00700 1 --2- 172.21.15.18:0/4018562176 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f781806ec00 0x7f78180710b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:11:53.272 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:53.277+0000 7f782aa00700 1 -- 172.21.15.18:0/4018562176 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 6757+0+0 (secure 0 0 0) 0x7f782008e700 con 0x7f78341059a0 2024-04-12T21:11:53.272 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:53.277+0000 7f7832a00700 1 --2- 172.21.15.18:0/4018562176 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f781806ec00 0x7f78180710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:11:53.273 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:53.278+0000 7f7832a00700 1 --2- 172.21.15.18:0/4018562176 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f781806ec00 0x7f78180710b0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f782400b3b0 tx=0x7f7824009e30 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:11:53.277 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:53.281+0000 7f782aa00700 1 -- 172.21.15.18:0/4018562176 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f782005aeb0 con 0x7f78341059a0 2024-04-12T21:11:53.350 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:11:53 smithi018 ceph-mon[112590]: from='client.34676 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:11:53.350 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:11:53 smithi018 ceph-mon[112590]: pgmap v362: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:11:53.350 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:11:53 smithi018 ceph-mon[112590]: from='client.? 172.21.15.18:0/548408227' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T21:11:53.350 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:11:53 smithi018 ceph-mon[112590]: from='client.? 172.21.15.18:0/3138116388' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-12T21:11:53.529 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:11:53 smithi186 ceph-mon[86379]: from='client.34676 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:11:53.529 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:11:53 smithi186 ceph-mon[86379]: pgmap v362: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:11:53.529 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:11:53 smithi186 ceph-mon[86379]: from='client.? 172.21.15.18:0/548408227' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T21:11:53.529 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:11:53 smithi186 ceph-mon[86379]: from='client.? 172.21.15.18:0/3138116388' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-12T21:11:53.604 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:53.607+0000 7f7839c00700 1 -- 172.21.15.18:0/4018562176 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f783410c100 con 0x7f78341059a0 2024-04-12T21:11:53.604 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:53.608+0000 7f782aa00700 1 -- 172.21.15.18:0/4018562176 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+233 (secure 0 0 0) 0x7f782005aa40 con 0x7f78341059a0 2024-04-12T21:11:53.604 INFO:teuthology.orchestra.run.smithi018.stdout:HEALTH_ERR 1 filesystem is degraded; 1 filesystem is offline 2024-04-12T21:11:53.604 INFO:teuthology.orchestra.run.smithi018.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-04-12T21:11:53.604 INFO:teuthology.orchestra.run.smithi018.stdout: fs cephfs is degraded 2024-04-12T21:11:53.604 INFO:teuthology.orchestra.run.smithi018.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-04-12T21:11:53.605 INFO:teuthology.orchestra.run.smithi018.stdout: fs cephfs is offline because no MDS is active for it. 2024-04-12T21:11:53.607 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:53.611+0000 7f7839c00700 1 -- 172.21.15.18:0/4018562176 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f781806ec00 msgr2=0x7f78180710b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:11:53.607 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:53.611+0000 7f7839c00700 1 --2- 172.21.15.18:0/4018562176 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f781806ec00 0x7f78180710b0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f782400b3b0 tx=0x7f7824009e30 comp rx=0 tx=0).stop 2024-04-12T21:11:53.607 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:53.611+0000 7f7839c00700 1 -- 172.21.15.18:0/4018562176 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f78341059a0 msgr2=0x7f78341160b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:11:53.607 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:53.611+0000 7f7839c00700 1 --2- 172.21.15.18:0/4018562176 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f78341059a0 0x7f78341160b0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f7820002a50 tx=0x7f7820004600 comp rx=0 tx=0).stop 2024-04-12T21:11:53.607 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:53.611+0000 7f7839c00700 1 -- 172.21.15.18:0/4018562176 shutdown_connections 2024-04-12T21:11:53.607 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:53.611+0000 7f7839c00700 1 --2- 172.21.15.18:0/4018562176 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f781806ec00 0x7f78180710b0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:11:53.607 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:53.611+0000 7f7839c00700 1 --2- 172.21.15.18:0/4018562176 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7834105000 0x7f7834115b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:11:53.607 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:53.611+0000 7f7839c00700 1 --2- 172.21.15.18:0/4018562176 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f78341059a0 0x7f78341160b0 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:11:53.607 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:53.611+0000 7f7839c00700 1 -- 172.21.15.18:0/4018562176 >> 172.21.15.18:0/4018562176 conn(0x7f78340fab70 msgr2=0x7f78340fbb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:11:53.607 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:53.611+0000 7f7839c00700 1 -- 172.21.15.18:0/4018562176 shutdown_connections 2024-04-12T21:11:53.608 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:11:53.612+0000 7f7839c00700 1 -- 172.21.15.18:0/4018562176 wait complete. 2024-04-12T21:11:54.529 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:11:54 smithi186 ceph-mon[86379]: from='client.34688 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:11:54.529 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:11:54 smithi186 ceph-mon[86379]: from='client.? 172.21.15.18:0/4018562176' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-12T21:11:54.600 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:11:54 smithi018 ceph-mon[112590]: from='client.34688 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:11:54.600 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:11:54 smithi018 ceph-mon[112590]: from='client.? 172.21.15.18:0/4018562176' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-12T21:11:55.529 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:11:55 smithi186 ceph-mon[86379]: pgmap v363: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:11:55.529 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:11:55 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T21:11:55.600 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:11:55 smithi018 ceph-mon[112590]: pgmap v363: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:11:55.600 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:11:55 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T21:11:57.529 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:11:57 smithi186 ceph-mon[86379]: pgmap v364: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:11:57.600 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:11:57 smithi018 ceph-mon[112590]: pgmap v364: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:11:59.529 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:11:59 smithi186 ceph-mon[86379]: pgmap v365: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:11:59.600 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:11:59 smithi018 ceph-mon[112590]: pgmap v365: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:12:01.529 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:12:01 smithi186 ceph-mon[86379]: pgmap v366: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:12:01.600 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:12:01 smithi018 ceph-mon[112590]: pgmap v366: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:12:03.529 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:12:03 smithi186 ceph-mon[86379]: pgmap v367: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:12:03.600 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:12:03 smithi018 ceph-mon[112590]: pgmap v367: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:12:05.529 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:12:05 smithi186 ceph-mon[86379]: pgmap v368: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:12:05.600 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:12:05 smithi018 ceph-mon[112590]: pgmap v368: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:12:07.529 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:12:07 smithi186 ceph-mon[86379]: pgmap v369: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:12:07.599 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:12:07 smithi018 ceph-mon[112590]: pgmap v369: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:12:09.529 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:12:09 smithi186 ceph-mon[86379]: pgmap v370: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:12:09.600 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:12:09 smithi018 ceph-mon[112590]: pgmap v370: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:12:10.502 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:12:10 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T21:12:10.528 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:12:10 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T21:12:11.529 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:12:11 smithi186 ceph-mon[86379]: pgmap v371: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:12:11.600 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:12:11 smithi018 ceph-mon[112590]: pgmap v371: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:12:13.529 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:12:13 smithi186 ceph-mon[86379]: pgmap v372: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:12:13.599 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:12:13 smithi018 ceph-mon[112590]: pgmap v372: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:12:15.529 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:12:15 smithi186 ceph-mon[86379]: pgmap v373: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:12:15.600 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:12:15 smithi018 ceph-mon[112590]: pgmap v373: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:12:17.529 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:12:17 smithi186 ceph-mon[86379]: pgmap v374: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:12:17.600 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:12:17 smithi018 ceph-mon[112590]: pgmap v374: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:12:19.529 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:12:19 smithi186 ceph-mon[86379]: pgmap v375: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:12:19.600 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:12:19 smithi018 ceph-mon[112590]: pgmap v375: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:12:21.600 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:12:21 smithi018 ceph-mon[112590]: pgmap v376: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:12:21.779 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:12:21 smithi186 ceph-mon[86379]: pgmap v376: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:12:23.600 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:12:23 smithi018 ceph-mon[112590]: pgmap v377: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:12:23.779 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:12:23 smithi186 ceph-mon[86379]: pgmap v377: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:12:23.780 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:23.783+0000 7fc689800700 1 -- 172.21.15.18:0/3770773500 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc6840fde30 msgr2=0x7fc6840fe200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:12:23.780 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:23.783+0000 7fc689800700 1 --2- 172.21.15.18:0/3770773500 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc6840fde30 0x7fc6840fe200 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fc674004940 tx=0x7fc674009e30 comp rx=0 tx=0).stop 2024-04-12T21:12:23.780 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:23.783+0000 7fc689800700 1 -- 172.21.15.18:0/3770773500 shutdown_connections 2024-04-12T21:12:23.780 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:23.783+0000 7fc689800700 1 --2- 172.21.15.18:0/3770773500 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc6840fe7d0 0x7fc684106e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:12:23.780 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:23.783+0000 7fc689800700 1 --2- 172.21.15.18:0/3770773500 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc6840fde30 0x7fc6840fe200 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:12:23.780 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:23.783+0000 7fc689800700 1 -- 172.21.15.18:0/3770773500 >> 172.21.15.18:0/3770773500 conn(0x7fc6840f7860 msgr2=0x7fc6840f9c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:12:23.780 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:23.783+0000 7fc689800700 1 -- 172.21.15.18:0/3770773500 shutdown_connections 2024-04-12T21:12:23.780 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:23.783+0000 7fc689800700 1 -- 172.21.15.18:0/3770773500 wait complete. 2024-04-12T21:12:23.781 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:23.784+0000 7fc689800700 1 Processor -- start 2024-04-12T21:12:23.781 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:23.784+0000 7fc689800700 1 -- start start 2024-04-12T21:12:23.781 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:23.785+0000 7fc689800700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc6840fde30 0x7fc684113970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:12:23.781 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:23.785+0000 7fc689800700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc6840fe7d0 0x7fc684113eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:12:23.781 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:23.785+0000 7fc689800700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc684114500 con 0x7fc6840fde30 2024-04-12T21:12:23.781 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:23.785+0000 7fc689800700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc68410db00 con 0x7fc6840fe7d0 2024-04-12T21:12:23.783 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:23.785+0000 7fc682a00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc6840fde30 0x7fc684113970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:12:23.783 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:23.785+0000 7fc682a00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc6840fde30 0x7fc684113970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:54348/0 (socket says 172.21.15.18:54348) 2024-04-12T21:12:23.783 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:23.785+0000 7fc682a00700 1 -- 172.21.15.18:0/4088195903 learned_addr learned my addr 172.21.15.18:0/4088195903 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:12:23.783 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:23.785+0000 7fc682000700 1 --2- 172.21.15.18:0/4088195903 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc6840fe7d0 0x7fc684113eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:12:23.783 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:23.786+0000 7fc682a00700 1 -- 172.21.15.18:0/4088195903 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc6840fe7d0 msgr2=0x7fc684113eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:12:23.783 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:23.786+0000 7fc682a00700 1 --2- 172.21.15.18:0/4088195903 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc6840fe7d0 0x7fc684113eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:12:23.783 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:23.786+0000 7fc682a00700 1 -- 172.21.15.18:0/4088195903 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc674005040 con 0x7fc6840fde30 2024-04-12T21:12:23.783 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:23.786+0000 7fc682000700 1 --2- 172.21.15.18:0/4088195903 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc6840fe7d0 0x7fc684113eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-12T21:12:23.783 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:23.786+0000 7fc682a00700 1 --2- 172.21.15.18:0/4088195903 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc6840fde30 0x7fc684113970 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fc67400b3b0 tx=0x7fc674009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:12:23.783 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:23.786+0000 7fc67aa00700 1 -- 172.21.15.18:0/4088195903 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc67400e040 con 0x7fc6840fde30 2024-04-12T21:12:23.784 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:23.786+0000 7fc67aa00700 1 -- 172.21.15.18:0/4088195903 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc674013420 con 0x7fc6840fde30 2024-04-12T21:12:23.784 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:23.786+0000 7fc67aa00700 1 -- 172.21.15.18:0/4088195903 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc67401c630 con 0x7fc6840fde30 2024-04-12T21:12:23.784 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:23.786+0000 7fc689800700 1 -- 172.21.15.18:0/4088195903 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc68410dd80 con 0x7fc6840fde30 2024-04-12T21:12:23.784 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:23.787+0000 7fc689800700 1 -- 172.21.15.18:0/4088195903 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc68410e270 con 0x7fc6840fde30 2024-04-12T21:12:23.784 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:23.788+0000 7fc689800700 1 -- 172.21.15.18:0/4088195903 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc684066ad0 con 0x7fc6840fde30 2024-04-12T21:12:23.785 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:23.789+0000 7fc67aa00700 1 -- 172.21.15.18:0/4088195903 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7fc674013590 con 0x7fc6840fde30 2024-04-12T21:12:23.786 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:23.790+0000 7fc67aa00700 1 --2- 172.21.15.18:0/4088195903 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fc66806ec00 0x7fc6680710b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:12:23.786 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:23.790+0000 7fc67aa00700 1 -- 172.21.15.18:0/4088195903 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 6757+0+0 (secure 0 0 0) 0x7fc67408fc50 con 0x7fc6840fde30 2024-04-12T21:12:23.786 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:23.790+0000 7fc682000700 1 --2- 172.21.15.18:0/4088195903 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fc66806ec00 0x7fc6680710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:12:23.787 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:23.791+0000 7fc682000700 1 --2- 172.21.15.18:0/4088195903 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fc66806ec00 0x7fc6680710b0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fc670009fd0 tx=0x7fc67000b040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:12:23.791 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:23.795+0000 7fc67aa00700 1 -- 172.21.15.18:0/4088195903 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7fc67405c4a0 con 0x7fc6840fde30 2024-04-12T21:12:24.043 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:24.047+0000 7fc689800700 1 -- 172.21.15.18:0/4088195903 --> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fc684109f10 con 0x7fc66806ec00 2024-04-12T21:12:24.047 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:24.051+0000 7fc67aa00700 1 -- 172.21.15.18:0/4088195903 <== mgr.34102 v2:172.21.15.18:6800/3812891919 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7fc684109f10 con 0x7fc66806ec00 2024-04-12T21:12:24.050 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:24.054+0000 7fc689800700 1 -- 172.21.15.18:0/4088195903 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fc66806ec00 msgr2=0x7fc6680710b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:12:24.050 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:24.054+0000 7fc689800700 1 --2- 172.21.15.18:0/4088195903 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fc66806ec00 0x7fc6680710b0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fc670009fd0 tx=0x7fc67000b040 comp rx=0 tx=0).stop 2024-04-12T21:12:24.051 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:24.054+0000 7fc689800700 1 -- 172.21.15.18:0/4088195903 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc6840fde30 msgr2=0x7fc684113970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:12:24.051 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:24.054+0000 7fc689800700 1 --2- 172.21.15.18:0/4088195903 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc6840fde30 0x7fc684113970 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fc67400b3b0 tx=0x7fc674009e30 comp rx=0 tx=0).stop 2024-04-12T21:12:24.051 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:24.054+0000 7fc689800700 1 -- 172.21.15.18:0/4088195903 shutdown_connections 2024-04-12T21:12:24.051 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:24.054+0000 7fc689800700 1 --2- 172.21.15.18:0/4088195903 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fc66806ec00 0x7fc6680710b0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:12:24.051 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:24.054+0000 7fc689800700 1 --2- 172.21.15.18:0/4088195903 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc6840fde30 0x7fc684113970 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:12:24.051 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:24.054+0000 7fc689800700 1 --2- 172.21.15.18:0/4088195903 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc6840fe7d0 0x7fc684113eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:12:24.051 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:24.054+0000 7fc689800700 1 -- 172.21.15.18:0/4088195903 >> 172.21.15.18:0/4088195903 conn(0x7fc6840f7860 msgr2=0x7fc6840f8cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:12:24.051 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:24.055+0000 7fc689800700 1 -- 172.21.15.18:0/4088195903 shutdown_connections 2024-04-12T21:12:24.051 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:24.055+0000 7fc689800700 1 -- 172.21.15.18:0/4088195903 wait complete. 2024-04-12T21:12:24.066 INFO:teuthology.orchestra.run.smithi018.stdout:true 2024-04-12T21:12:24.198 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:24.202+0000 7fd519e00700 1 -- 172.21.15.18:0/2487522627 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd514105800 msgr2=0x7fd5140fee70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:12:24.199 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:24.202+0000 7fd519e00700 1 --2- 172.21.15.18:0/2487522627 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd514105800 0x7fd5140fee70 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fd500004940 tx=0x7fd500009e30 comp rx=0 tx=0).stop 2024-04-12T21:12:24.199 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:24.202+0000 7fd519e00700 1 -- 172.21.15.18:0/2487522627 shutdown_connections 2024-04-12T21:12:24.199 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:24.202+0000 7fd519e00700 1 --2- 172.21.15.18:0/2487522627 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd514105800 0x7fd5140fee70 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:12:24.199 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:24.202+0000 7fd519e00700 1 --2- 172.21.15.18:0/2487522627 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd514104e60 0x7fd514105230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:12:24.199 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:24.202+0000 7fd519e00700 1 -- 172.21.15.18:0/2487522627 >> 172.21.15.18:0/2487522627 conn(0x7fd5140fab10 msgr2=0x7fd5140fcef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:12:24.199 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:24.202+0000 7fd519e00700 1 -- 172.21.15.18:0/2487522627 shutdown_connections 2024-04-12T21:12:24.199 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:24.202+0000 7fd519e00700 1 -- 172.21.15.18:0/2487522627 wait complete. 2024-04-12T21:12:24.200 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:24.203+0000 7fd519e00700 1 Processor -- start 2024-04-12T21:12:24.200 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:24.203+0000 7fd519e00700 1 -- start start 2024-04-12T21:12:24.200 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:24.204+0000 7fd519e00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd514104e60 0x7fd514115bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:12:24.200 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:24.204+0000 7fd519e00700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd514105800 0x7fd5141160f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:12:24.200 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:24.204+0000 7fd519e00700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd514116740 con 0x7fd514104e60 2024-04-12T21:12:24.201 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:24.204+0000 7fd519e00700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd51410fb30 con 0x7fd514105800 2024-04-12T21:12:24.201 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:24.204+0000 7fd518a00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd514104e60 0x7fd514115bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:12:24.201 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:24.204+0000 7fd518a00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd514104e60 0x7fd514115bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:54356/0 (socket says 172.21.15.18:54356) 2024-04-12T21:12:24.201 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:24.204+0000 7fd518a00700 1 -- 172.21.15.18:0/4285396619 learned_addr learned my addr 172.21.15.18:0/4285396619 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:12:24.202 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:24.204+0000 7fd513e00700 1 --2- 172.21.15.18:0/4285396619 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd514105800 0x7fd5141160f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:12:24.202 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:24.204+0000 7fd518a00700 1 -- 172.21.15.18:0/4285396619 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd514105800 msgr2=0x7fd5141160f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:12:24.202 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:24.204+0000 7fd518a00700 1 --2- 172.21.15.18:0/4285396619 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd514105800 0x7fd5141160f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:12:24.202 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:24.204+0000 7fd518a00700 1 -- 172.21.15.18:0/4285396619 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd500005040 con 0x7fd514104e60 2024-04-12T21:12:24.202 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:24.205+0000 7fd518a00700 1 --2- 172.21.15.18:0/4285396619 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd514104e60 0x7fd514115bb0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fd50800c8c0 tx=0x7fd5080044c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:12:24.202 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:24.205+0000 7fd510c00700 1 -- 172.21.15.18:0/4285396619 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd50800a980 con 0x7fd514104e60 2024-04-12T21:12:24.202 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:24.205+0000 7fd510c00700 1 -- 172.21.15.18:0/4285396619 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd50800ce60 con 0x7fd514104e60 2024-04-12T21:12:24.202 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:24.205+0000 7fd510c00700 1 -- 172.21.15.18:0/4285396619 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd5080115e0 con 0x7fd514104e60 2024-04-12T21:12:24.202 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:24.205+0000 7fd519e00700 1 -- 172.21.15.18:0/4285396619 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd51410fe10 con 0x7fd514104e60 2024-04-12T21:12:24.203 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:24.205+0000 7fd519e00700 1 -- 172.21.15.18:0/4285396619 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd5141103d0 con 0x7fd514104e60 2024-04-12T21:12:24.203 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:24.206+0000 7fd519e00700 1 -- 172.21.15.18:0/4285396619 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd514107330 con 0x7fd514104e60 2024-04-12T21:12:24.203 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:24.207+0000 7fd510c00700 1 -- 172.21.15.18:0/4285396619 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7fd508012400 con 0x7fd514104e60 2024-04-12T21:12:24.205 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:24.208+0000 7fd510c00700 1 --2- 172.21.15.18:0/4285396619 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fd4fc06ec00 0x7fd4fc0710b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:12:24.205 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:24.208+0000 7fd510c00700 1 -- 172.21.15.18:0/4285396619 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 6757+0+0 (secure 0 0 0) 0x7fd50808e980 con 0x7fd514104e60 2024-04-12T21:12:24.205 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:24.209+0000 7fd513e00700 1 --2- 172.21.15.18:0/4285396619 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fd4fc06ec00 0x7fd4fc0710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:12:24.205 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:24.209+0000 7fd513e00700 1 --2- 172.21.15.18:0/4285396619 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fd4fc06ec00 0x7fd4fc0710b0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fd50000b3b0 tx=0x7fd500009e30 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:12:24.209 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:24.213+0000 7fd510c00700 1 -- 172.21.15.18:0/4285396619 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7fd50805b270 con 0x7fd514104e60 2024-04-12T21:12:24.461 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:24.464+0000 7fd519e00700 1 -- 172.21.15.18:0/4285396619 --> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fd514111200 con 0x7fd4fc06ec00 2024-04-12T21:12:24.464 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:24.467+0000 7fd510c00700 1 -- 172.21.15.18:0/4285396619 <== mgr.34102 v2:172.21.15.18:6800/3812891919 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7fd514111200 con 0x7fd4fc06ec00 2024-04-12T21:12:24.467 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:24.470+0000 7fd519e00700 1 -- 172.21.15.18:0/4285396619 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fd4fc06ec00 msgr2=0x7fd4fc0710b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:12:24.467 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:24.470+0000 7fd519e00700 1 --2- 172.21.15.18:0/4285396619 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fd4fc06ec00 0x7fd4fc0710b0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fd50000b3b0 tx=0x7fd500009e30 comp rx=0 tx=0).stop 2024-04-12T21:12:24.468 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:24.470+0000 7fd519e00700 1 -- 172.21.15.18:0/4285396619 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd514104e60 msgr2=0x7fd514115bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:12:24.468 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:24.470+0000 7fd519e00700 1 --2- 172.21.15.18:0/4285396619 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd514104e60 0x7fd514115bb0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fd50800c8c0 tx=0x7fd5080044c0 comp rx=0 tx=0).stop 2024-04-12T21:12:24.468 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:24.470+0000 7fd519e00700 1 -- 172.21.15.18:0/4285396619 shutdown_connections 2024-04-12T21:12:24.468 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:24.470+0000 7fd519e00700 1 --2- 172.21.15.18:0/4285396619 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fd4fc06ec00 0x7fd4fc0710b0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:12:24.468 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:24.470+0000 7fd519e00700 1 --2- 172.21.15.18:0/4285396619 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd514104e60 0x7fd514115bb0 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:12:24.468 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:24.470+0000 7fd519e00700 1 --2- 172.21.15.18:0/4285396619 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd514105800 0x7fd5141160f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:12:24.468 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:24.471+0000 7fd519e00700 1 -- 172.21.15.18:0/4285396619 >> 172.21.15.18:0/4285396619 conn(0x7fd5140fab10 msgr2=0x7fd5140fd940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:12:24.468 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:24.471+0000 7fd519e00700 1 -- 172.21.15.18:0/4285396619 shutdown_connections 2024-04-12T21:12:24.468 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:24.471+0000 7fd519e00700 1 -- 172.21.15.18:0/4285396619 wait complete. 2024-04-12T21:12:24.616 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:24.619+0000 7f80aba00700 1 -- 172.21.15.18:0/2761879252 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f80a40724f0 msgr2=0x7f80a40728c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:12:24.616 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:24.619+0000 7f80aba00700 1 --2- 172.21.15.18:0/2761879252 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f80a40724f0 0x7f80a40728c0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f8098004940 tx=0x7f8098009e30 comp rx=0 tx=0).stop 2024-04-12T21:12:24.616 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:24.619+0000 7f80aba00700 1 -- 172.21.15.18:0/2761879252 shutdown_connections 2024-04-12T21:12:24.616 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:24.619+0000 7f80aba00700 1 --2- 172.21.15.18:0/2761879252 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f80a4072e00 0x7f80a410d770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:12:24.616 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:24.619+0000 7f80aba00700 1 --2- 172.21.15.18:0/2761879252 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f80a40724f0 0x7f80a40728c0 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:12:24.616 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:24.619+0000 7f80aba00700 1 -- 172.21.15.18:0/2761879252 >> 172.21.15.18:0/2761879252 conn(0x7f80a40fabd0 msgr2=0x7f80a40fcfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:12:24.616 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:24.619+0000 7f80aba00700 1 -- 172.21.15.18:0/2761879252 shutdown_connections 2024-04-12T21:12:24.616 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:24.619+0000 7f80aba00700 1 -- 172.21.15.18:0/2761879252 wait complete. 2024-04-12T21:12:24.617 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:24.620+0000 7f80aba00700 1 Processor -- start 2024-04-12T21:12:24.617 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:24.621+0000 7f80aba00700 1 -- start start 2024-04-12T21:12:24.618 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:24.621+0000 7f80aba00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f80a40724f0 0x7f80a4119010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:12:24.618 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:24.621+0000 7f80aba00700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f80a4072e00 0x7f80a4119550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:12:24.618 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:24.621+0000 7f80aba00700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80a4119b50 con 0x7f80a40724f0 2024-04-12T21:12:24.618 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:24.621+0000 7f80aba00700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80a4114120 con 0x7f80a4072e00 2024-04-12T21:12:24.618 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:24.621+0000 7f80a9000700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f80a40724f0 0x7f80a4119010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:12:24.618 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:24.621+0000 7f80a9000700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f80a40724f0 0x7f80a4119010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:54372/0 (socket says 172.21.15.18:54372) 2024-04-12T21:12:24.618 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:24.621+0000 7f80a9000700 1 -- 172.21.15.18:0/1202648236 learned_addr learned my addr 172.21.15.18:0/1202648236 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:12:24.619 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:24.622+0000 7f80a3e00700 1 --2- 172.21.15.18:0/1202648236 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f80a4072e00 0x7f80a4119550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:12:24.619 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:24.622+0000 7f80a9000700 1 -- 172.21.15.18:0/1202648236 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f80a4072e00 msgr2=0x7f80a4119550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:12:24.619 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:24.622+0000 7f80a9000700 1 --2- 172.21.15.18:0/1202648236 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f80a4072e00 0x7f80a4119550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:12:24.619 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:24.622+0000 7f80a9000700 1 -- 172.21.15.18:0/1202648236 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8098005040 con 0x7f80a40724f0 2024-04-12T21:12:24.619 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:24.622+0000 7f80a3e00700 1 --2- 172.21.15.18:0/1202648236 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f80a4072e00 0x7f80a4119550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-12T21:12:24.619 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:24.622+0000 7f80a9000700 1 --2- 172.21.15.18:0/1202648236 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f80a40724f0 0x7f80a4119010 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f8098009180 tx=0x7f8098009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:12:24.619 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:24.622+0000 7f80a0c00700 1 -- 172.21.15.18:0/1202648236 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f809800e040 con 0x7f80a40724f0 2024-04-12T21:12:24.620 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:24.622+0000 7f80a0c00700 1 -- 172.21.15.18:0/1202648236 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f8098013420 con 0x7f80a40724f0 2024-04-12T21:12:24.620 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:24.622+0000 7f80aba00700 1 -- 172.21.15.18:0/1202648236 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f80a41143a0 con 0x7f80a40724f0 2024-04-12T21:12:24.620 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:24.623+0000 7f80a0c00700 1 -- 172.21.15.18:0/1202648236 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f809801c630 con 0x7f80a40724f0 2024-04-12T21:12:24.620 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:24.623+0000 7f80aba00700 1 -- 172.21.15.18:0/1202648236 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f80a4114960 con 0x7f80a40724f0 2024-04-12T21:12:24.620 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:24.623+0000 7f80aba00700 1 -- 172.21.15.18:0/1202648236 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f80a4073c30 con 0x7f80a40724f0 2024-04-12T21:12:24.621 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:24.625+0000 7f80a0c00700 1 -- 172.21.15.18:0/1202648236 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7f8098013590 con 0x7f80a40724f0 2024-04-12T21:12:24.622 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:24.626+0000 7f80a0c00700 1 --2- 172.21.15.18:0/1202648236 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f808c06ec10 0x7f808c0710c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:12:24.622 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:24.626+0000 7f80a0c00700 1 -- 172.21.15.18:0/1202648236 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 6757+0+0 (secure 0 0 0) 0x7f809808fd40 con 0x7f80a40724f0 2024-04-12T21:12:24.622 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:24.626+0000 7f80a3e00700 1 --2- 172.21.15.18:0/1202648236 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f808c06ec10 0x7f808c0710c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:12:24.623 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:24.627+0000 7f80a3e00700 1 --2- 172.21.15.18:0/1202648236 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f808c06ec10 0x7f808c0710c0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f8090007fd0 tx=0x7f8090009040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:12:24.627 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:24.630+0000 7f80a0c00700 1 -- 172.21.15.18:0/1202648236 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f8098058a60 con 0x7f80a40724f0 2024-04-12T21:12:24.863 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:24.867+0000 7f80aba00700 1 -- 172.21.15.18:0/1202648236 --> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f80a404bed0 con 0x7f808c06ec10 2024-04-12T21:12:24.930 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:24.934+0000 7f80a0c00700 1 -- 172.21.15.18:0/1202648236 <== mgr.34102 v2:172.21.15.18:6800/3812891919 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7f80a404bed0 con 0x7f808c06ec10 2024-04-12T21:12:24.931 INFO:teuthology.orchestra.run.smithi018.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-12T21:12:24.931 INFO:teuthology.orchestra.run.smithi018.stdout:alertmanager.smithi018 smithi018 *:9093,9094 running (13m) 4m ago 27m 14.7M - 0.25.0 c8568f914cd2 c332090068e8 2024-04-12T21:12:24.931 INFO:teuthology.orchestra.run.smithi018.stdout:crash.smithi018 smithi018 running (9m) 4m ago 27m 6665k - 19.0.0-2879-g324baf8d 8717720dc5e5 b7f80d1f7982 2024-04-12T21:12:24.931 INFO:teuthology.orchestra.run.smithi018.stdout:crash.smithi186 smithi186 running (9m) 5m ago 26m 6660k - 19.0.0-2879-g324baf8d 8717720dc5e5 a4f55c81b623 2024-04-12T21:12:24.931 INFO:teuthology.orchestra.run.smithi018.stdout:grafana.smithi018 smithi018 *:3000 running (12m) 4m ago 27m 74.8M - 9.4.12 2bacad6d85d8 b08e3bf176dd 2024-04-12T21:12:24.931 INFO:teuthology.orchestra.run.smithi018.stdout:mds.cephfs.smithi018.acqbsc smithi018 running (5m) 4m ago 20m 11.1M - 19.0.0-2879-g324baf8d 8717720dc5e5 021f99ab24ae 2024-04-12T21:12:24.931 INFO:teuthology.orchestra.run.smithi018.stdout:mds.cephfs.smithi018.szjluo smithi018 running (4m) 4m ago 20m 10.9M - 19.0.0-2879-g324baf8d 8717720dc5e5 0003117479d4 2024-04-12T21:12:24.931 INFO:teuthology.orchestra.run.smithi018.stdout:mds.cephfs.smithi186.jdllyn smithi186 running (20m) 5m ago 20m 15.6M - 17.2.7-827-gd3b5060a 21de9b7369ad 2b2bc46bd29a 2024-04-12T21:12:24.932 INFO:teuthology.orchestra.run.smithi018.stdout:mds.cephfs.smithi186.wjnafs smithi186 running (20m) 5m ago 20m 15.4M - 17.2.7-827-gd3b5060a 21de9b7369ad 74b115d6d6fd 2024-04-12T21:12:24.932 INFO:teuthology.orchestra.run.smithi018.stdout:mgr.smithi018.xjzwga smithi018 *:8443,9283,8765 running (15m) 4m ago 29m 509M - 19.0.0-2879-g324baf8d 8717720dc5e5 56088305eab7 2024-04-12T21:12:24.932 INFO:teuthology.orchestra.run.smithi018.stdout:mgr.smithi186.kqcfxk smithi186 *:8443,9283,8765 running (14m) 5m ago 26m 452M - 19.0.0-2879-g324baf8d 8717720dc5e5 09cd766e5e0d 2024-04-12T21:12:24.932 INFO:teuthology.orchestra.run.smithi018.stdout:mon.smithi018 smithi018 running (10m) 4m ago 29m 57.0M 2048M 19.0.0-2879-g324baf8d 8717720dc5e5 df3e201dfed6 2024-04-12T21:12:24.932 INFO:teuthology.orchestra.run.smithi018.stdout:mon.smithi186 smithi186 running (10m) 5m ago 26m 37.7M 2048M 19.0.0-2879-g324baf8d 8717720dc5e5 c77498da9b2b 2024-04-12T21:12:24.932 INFO:teuthology.orchestra.run.smithi018.stdout:node-exporter.smithi018 smithi018 *:9100 running (14m) 4m ago 27m 7780k - 1.5.0 0da6a335fe13 326c3d9c6847 2024-04-12T21:12:24.932 INFO:teuthology.orchestra.run.smithi018.stdout:node-exporter.smithi186 smithi186 *:9100 running (13m) 5m ago 26m 7826k - 1.5.0 0da6a335fe13 ec2112baeecf 2024-04-12T21:12:24.932 INFO:teuthology.orchestra.run.smithi018.stdout:osd.0 smithi018 running (8m) 4m ago 25m 121M 1559M 19.0.0-2879-g324baf8d 8717720dc5e5 60b1273ca047 2024-04-12T21:12:24.932 INFO:teuthology.orchestra.run.smithi018.stdout:osd.1 smithi018 running (8m) 4m ago 24m 91.3M 1559M 19.0.0-2879-g324baf8d 8717720dc5e5 170ce1e5fb8d 2024-04-12T21:12:24.932 INFO:teuthology.orchestra.run.smithi018.stdout:osd.2 smithi018 running (7m) 4m ago 23m 90.3M 1559M 19.0.0-2879-g324baf8d 8717720dc5e5 c2cbc6aa2839 2024-04-12T21:12:24.932 INFO:teuthology.orchestra.run.smithi018.stdout:osd.3 smithi186 running (7m) 5m ago 23m 96.6M 2581M 19.0.0-2879-g324baf8d 8717720dc5e5 0138e8262e4b 2024-04-12T21:12:24.932 INFO:teuthology.orchestra.run.smithi018.stdout:osd.4 smithi186 running (6m) 5m ago 22m 87.2M 2581M 19.0.0-2879-g324baf8d 8717720dc5e5 b20cb63b25c9 2024-04-12T21:12:24.932 INFO:teuthology.orchestra.run.smithi018.stdout:osd.5 smithi186 running (5m) 5m ago 22m 11.1M 2581M 19.0.0-2879-g324baf8d 8717720dc5e5 2513bea2610e 2024-04-12T21:12:24.932 INFO:teuthology.orchestra.run.smithi018.stdout:prometheus.smithi018 smithi018 *:9095 running (13m) 4m ago 26m 60.6M - 2.43.0 a07b618ecd1d f58841127e21 2024-04-12T21:12:24.934 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:24.937+0000 7f80aba00700 1 -- 172.21.15.18:0/1202648236 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f808c06ec10 msgr2=0x7f808c0710c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:12:24.934 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:24.937+0000 7f80aba00700 1 --2- 172.21.15.18:0/1202648236 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f808c06ec10 0x7f808c0710c0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f8090007fd0 tx=0x7f8090009040 comp rx=0 tx=0).stop 2024-04-12T21:12:24.934 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:24.937+0000 7f80aba00700 1 -- 172.21.15.18:0/1202648236 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f80a40724f0 msgr2=0x7f80a4119010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:12:24.934 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:24.937+0000 7f80aba00700 1 --2- 172.21.15.18:0/1202648236 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f80a40724f0 0x7f80a4119010 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f8098009180 tx=0x7f8098009e30 comp rx=0 tx=0).stop 2024-04-12T21:12:24.934 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:24.937+0000 7f80aba00700 1 -- 172.21.15.18:0/1202648236 shutdown_connections 2024-04-12T21:12:24.935 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:24.937+0000 7f80aba00700 1 --2- 172.21.15.18:0/1202648236 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f808c06ec10 0x7f808c0710c0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:12:24.935 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:24.937+0000 7f80aba00700 1 --2- 172.21.15.18:0/1202648236 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f80a40724f0 0x7f80a4119010 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:12:24.935 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:24.937+0000 7f80aba00700 1 --2- 172.21.15.18:0/1202648236 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f80a4072e00 0x7f80a4119550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:12:24.935 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:24.938+0000 7f80aba00700 1 -- 172.21.15.18:0/1202648236 >> 172.21.15.18:0/1202648236 conn(0x7f80a40fabd0 msgr2=0x7f80a40fafa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:12:24.935 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:24.938+0000 7f80aba00700 1 -- 172.21.15.18:0/1202648236 shutdown_connections 2024-04-12T21:12:24.935 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:24.938+0000 7f80aba00700 1 -- 172.21.15.18:0/1202648236 wait complete. 2024-04-12T21:12:25.088 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:25.091+0000 7fb6c9400700 1 -- 172.21.15.18:0/495724022 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb6c4109430 msgr2=0x7fb6c4109800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:12:25.088 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:25.091+0000 7fb6c9400700 1 --2- 172.21.15.18:0/495724022 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb6c4109430 0x7fb6c4109800 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fb6b4004940 tx=0x7fb6b4009e30 comp rx=0 tx=0).stop 2024-04-12T21:12:25.088 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:25.091+0000 7fb6c9400700 1 -- 172.21.15.18:0/495724022 shutdown_connections 2024-04-12T21:12:25.088 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:25.091+0000 7fb6c9400700 1 --2- 172.21.15.18:0/495724022 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb6c4109dd0 0x7fb6c4103440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:12:25.088 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:25.091+0000 7fb6c9400700 1 --2- 172.21.15.18:0/495724022 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb6c4109430 0x7fb6c4109800 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:12:25.088 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:25.091+0000 7fb6c9400700 1 -- 172.21.15.18:0/495724022 >> 172.21.15.18:0/495724022 conn(0x7fb6c4072c00 msgr2=0x7fb6c4075010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:12:25.088 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:25.091+0000 7fb6c9400700 1 -- 172.21.15.18:0/495724022 shutdown_connections 2024-04-12T21:12:25.088 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:25.091+0000 7fb6c9400700 1 -- 172.21.15.18:0/495724022 wait complete. 2024-04-12T21:12:25.090 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:25.092+0000 7fb6c9400700 1 Processor -- start 2024-04-12T21:12:25.090 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:25.092+0000 7fb6c9400700 1 -- start start 2024-04-12T21:12:25.090 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:25.093+0000 7fb6c9400700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb6c4109430 0x7fb6c4119f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:12:25.090 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:25.093+0000 7fb6c9400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb6c4109dd0 0x7fb6c411a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:12:25.090 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:25.093+0000 7fb6c9400700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6c411ab60 con 0x7fb6c4109dd0 2024-04-12T21:12:25.090 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:25.093+0000 7fb6c9400700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6c41140b0 con 0x7fb6c4109430 2024-04-12T21:12:25.090 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:25.093+0000 7fb6c1e00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb6c4109dd0 0x7fb6c411a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:12:25.090 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:25.093+0000 7fb6c1e00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb6c4109dd0 0x7fb6c411a480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:54396/0 (socket says 172.21.15.18:54396) 2024-04-12T21:12:25.090 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:25.093+0000 7fb6c1e00700 1 -- 172.21.15.18:0/3452128119 learned_addr learned my addr 172.21.15.18:0/3452128119 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:12:25.090 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:25.093+0000 7fb6c2800700 1 --2- 172.21.15.18:0/3452128119 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb6c4109430 0x7fb6c4119f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:12:25.090 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:25.093+0000 7fb6c1e00700 1 -- 172.21.15.18:0/3452128119 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb6c4109430 msgr2=0x7fb6c4119f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:12:25.091 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:25.093+0000 7fb6c1e00700 1 --2- 172.21.15.18:0/3452128119 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb6c4109430 0x7fb6c4119f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:12:25.091 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:25.093+0000 7fb6c1e00700 1 -- 172.21.15.18:0/3452128119 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6b4005040 con 0x7fb6c4109dd0 2024-04-12T21:12:25.091 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:25.094+0000 7fb6c1e00700 1 --2- 172.21.15.18:0/3452128119 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb6c4109dd0 0x7fb6c411a480 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fb6b000b790 tx=0x7fb6b0009220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:12:25.091 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:25.094+0000 7fb6baa00700 1 -- 172.21.15.18:0/3452128119 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb6b000f6f0 con 0x7fb6c4109dd0 2024-04-12T21:12:25.091 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:25.094+0000 7fb6baa00700 1 -- 172.21.15.18:0/3452128119 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb6b000fd20 con 0x7fb6c4109dd0 2024-04-12T21:12:25.092 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:25.094+0000 7fb6c9400700 1 -- 172.21.15.18:0/3452128119 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb6c4114330 con 0x7fb6c4109dd0 2024-04-12T21:12:25.092 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:25.094+0000 7fb6baa00700 1 -- 172.21.15.18:0/3452128119 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb6b00103a0 con 0x7fb6c4109dd0 2024-04-12T21:12:25.092 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:25.094+0000 7fb6c9400700 1 -- 172.21.15.18:0/3452128119 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb6c41148f0 con 0x7fb6c4109dd0 2024-04-12T21:12:25.092 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:25.095+0000 7fb6c9400700 1 -- 172.21.15.18:0/3452128119 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb6c4104b80 con 0x7fb6c4109dd0 2024-04-12T21:12:25.092 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:25.096+0000 7fb6baa00700 1 -- 172.21.15.18:0/3452128119 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7fb6b0011400 con 0x7fb6c4109dd0 2024-04-12T21:12:25.093 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:25.097+0000 7fb6baa00700 1 --2- 172.21.15.18:0/3452128119 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fb6a806ec00 0x7fb6a80710b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:12:25.094 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:25.097+0000 7fb6baa00700 1 -- 172.21.15.18:0/3452128119 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 6757+0+0 (secure 0 0 0) 0x7fb6b008e780 con 0x7fb6c4109dd0 2024-04-12T21:12:25.094 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:25.098+0000 7fb6c2800700 1 --2- 172.21.15.18:0/3452128119 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fb6a806ec00 0x7fb6a80710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:12:25.094 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:25.098+0000 7fb6c2800700 1 --2- 172.21.15.18:0/3452128119 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fb6a806ec00 0x7fb6a80710b0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fb6b400b3b0 tx=0x7fb6b4009e30 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:12:25.099 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:25.102+0000 7fb6baa00700 1 -- 172.21.15.18:0/3452128119 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7fb6b005afd0 con 0x7fb6c4109dd0 2024-04-12T21:12:25.434 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:25.437+0000 7fb6c9400700 1 -- 172.21.15.18:0/3452128119 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fb6c404c710 con 0x7fb6c4109dd0 2024-04-12T21:12:25.435 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:25.439+0000 7fb6baa00700 1 -- 172.21.15.18:0/3452128119 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+837 (secure 0 0 0) 0x7fb6b005ab60 con 0x7fb6c4109dd0 2024-04-12T21:12:25.435 INFO:teuthology.orchestra.run.smithi018.stdout:{ 2024-04-12T21:12:25.435 INFO:teuthology.orchestra.run.smithi018.stdout: "mon": { 2024-04-12T21:12:25.436 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 19.0.0-2879-g324baf8d (324baf8ddb91dea46cac50cb535c306115791355) squid (dev)": 2 2024-04-12T21:12:25.436 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-12T21:12:25.436 INFO:teuthology.orchestra.run.smithi018.stdout: "mgr": { 2024-04-12T21:12:25.436 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 19.0.0-2879-g324baf8d (324baf8ddb91dea46cac50cb535c306115791355) squid (dev)": 2 2024-04-12T21:12:25.436 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-12T21:12:25.436 INFO:teuthology.orchestra.run.smithi018.stdout: "osd": { 2024-04-12T21:12:25.436 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 19.0.0-2879-g324baf8d (324baf8ddb91dea46cac50cb535c306115791355) squid (dev)": 6 2024-04-12T21:12:25.436 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-12T21:12:25.436 INFO:teuthology.orchestra.run.smithi018.stdout: "mds": { 2024-04-12T21:12:25.436 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 17.2.7-827-gd3b5060a (d3b5060a17449edfae9385cdf792afec109fca5d) quincy (stable)": 2, 2024-04-12T21:12:25.436 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 19.0.0-2879-g324baf8d (324baf8ddb91dea46cac50cb535c306115791355) squid (dev)": 2 2024-04-12T21:12:25.436 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-12T21:12:25.436 INFO:teuthology.orchestra.run.smithi018.stdout: "overall": { 2024-04-12T21:12:25.436 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 17.2.7-827-gd3b5060a (d3b5060a17449edfae9385cdf792afec109fca5d) quincy (stable)": 2, 2024-04-12T21:12:25.436 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 19.0.0-2879-g324baf8d (324baf8ddb91dea46cac50cb535c306115791355) squid (dev)": 12 2024-04-12T21:12:25.436 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-04-12T21:12:25.437 INFO:teuthology.orchestra.run.smithi018.stdout:} 2024-04-12T21:12:25.438 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:25.441+0000 7fb6c9400700 1 -- 172.21.15.18:0/3452128119 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fb6a806ec00 msgr2=0x7fb6a80710b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:12:25.438 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:25.441+0000 7fb6c9400700 1 --2- 172.21.15.18:0/3452128119 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fb6a806ec00 0x7fb6a80710b0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fb6b400b3b0 tx=0x7fb6b4009e30 comp rx=0 tx=0).stop 2024-04-12T21:12:25.438 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:25.441+0000 7fb6c9400700 1 -- 172.21.15.18:0/3452128119 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb6c4109dd0 msgr2=0x7fb6c411a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:12:25.438 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:25.441+0000 7fb6c9400700 1 --2- 172.21.15.18:0/3452128119 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb6c4109dd0 0x7fb6c411a480 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fb6b000b790 tx=0x7fb6b0009220 comp rx=0 tx=0).stop 2024-04-12T21:12:25.438 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:25.441+0000 7fb6c9400700 1 -- 172.21.15.18:0/3452128119 shutdown_connections 2024-04-12T21:12:25.438 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:25.441+0000 7fb6c9400700 1 --2- 172.21.15.18:0/3452128119 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fb6a806ec00 0x7fb6a80710b0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:12:25.438 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:25.441+0000 7fb6c9400700 1 --2- 172.21.15.18:0/3452128119 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb6c4109430 0x7fb6c4119f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:12:25.438 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:25.441+0000 7fb6c9400700 1 --2- 172.21.15.18:0/3452128119 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb6c4109dd0 0x7fb6c411a480 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:12:25.438 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:25.441+0000 7fb6c9400700 1 -- 172.21.15.18:0/3452128119 >> 172.21.15.18:0/3452128119 conn(0x7fb6c4072c00 msgr2=0x7fb6c4074c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:12:25.439 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:25.442+0000 7fb6c9400700 1 -- 172.21.15.18:0/3452128119 shutdown_connections 2024-04-12T21:12:25.439 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:25.442+0000 7fb6c9400700 1 -- 172.21.15.18:0/3452128119 wait complete. 2024-04-12T21:12:25.593 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:25.596+0000 7fa716800700 1 -- 172.21.15.18:0/1235547232 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa7100708a0 msgr2=0x7fa710070d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:12:25.593 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:25.596+0000 7fa716800700 1 --2- 172.21.15.18:0/1235547232 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa7100708a0 0x7fa710070d10 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fa6fc004970 tx=0x7fa6fc009e30 comp rx=0 tx=0).stop 2024-04-12T21:12:25.594 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:12:25 smithi018 ceph-mon[112590]: pgmap v378: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:12:25.594 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:12:25 smithi018 ceph-mon[112590]: from='client.34696 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:12:25.594 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:12:25 smithi018 ceph-mon[112590]: from='client.34700 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:12:25.594 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:12:25 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T21:12:25.595 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:25.597+0000 7fa716800700 1 -- 172.21.15.18:0/1235547232 shutdown_connections 2024-04-12T21:12:25.595 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:25.597+0000 7fa716800700 1 --2- 172.21.15.18:0/1235547232 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa7100708a0 0x7fa710070d10 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:12:25.596 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:25.597+0000 7fa716800700 1 --2- 172.21.15.18:0/1235547232 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa710071f90 0x7fa710070360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:12:25.596 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:25.597+0000 7fa716800700 1 -- 172.21.15.18:0/1235547232 >> 172.21.15.18:0/1235547232 conn(0x7fa7100f89d0 msgr2=0x7fa7100fade0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:12:25.596 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:25.597+0000 7fa716800700 1 -- 172.21.15.18:0/1235547232 shutdown_connections 2024-04-12T21:12:25.596 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:25.597+0000 7fa716800700 1 -- 172.21.15.18:0/1235547232 wait complete. 2024-04-12T21:12:25.596 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:25.598+0000 7fa716800700 1 Processor -- start 2024-04-12T21:12:25.596 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:25.598+0000 7fa716800700 1 -- start start 2024-04-12T21:12:25.596 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:25.598+0000 7fa716800700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa7100708a0 0x7fa71011a1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:12:25.596 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:25.598+0000 7fa716800700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa710071f90 0x7fa71011a6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:12:25.596 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:25.598+0000 7fa716800700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa71011ad40 con 0x7fa7100708a0 2024-04-12T21:12:25.596 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:25.598+0000 7fa716800700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa710114110 con 0x7fa710071f90 2024-04-12T21:12:25.596 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:25.599+0000 7fa70f400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa7100708a0 0x7fa71011a1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:12:25.596 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:25.599+0000 7fa70f400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa7100708a0 0x7fa71011a1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:54412/0 (socket says 172.21.15.18:54412) 2024-04-12T21:12:25.596 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:25.599+0000 7fa70f400700 1 -- 172.21.15.18:0/1646020985 learned_addr learned my addr 172.21.15.18:0/1646020985 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:12:25.596 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:25.599+0000 7fa70ea00700 1 --2- 172.21.15.18:0/1646020985 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa710071f90 0x7fa71011a6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:12:25.597 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:25.599+0000 7fa70f400700 1 -- 172.21.15.18:0/1646020985 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa710071f90 msgr2=0x7fa71011a6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:12:25.597 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:25.599+0000 7fa70f400700 1 --2- 172.21.15.18:0/1646020985 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa710071f90 0x7fa71011a6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:12:25.597 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:25.599+0000 7fa70f400700 1 -- 172.21.15.18:0/1646020985 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6fc005040 con 0x7fa7100708a0 2024-04-12T21:12:25.597 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:25.600+0000 7fa70f400700 1 --2- 172.21.15.18:0/1646020985 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa7100708a0 0x7fa71011a1b0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fa70000cb10 tx=0x7fa700007f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:12:25.597 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:25.600+0000 7fa707400700 1 -- 172.21.15.18:0/1646020985 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa700004200 con 0x7fa7100708a0 2024-04-12T21:12:25.598 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:25.600+0000 7fa707400700 1 -- 172.21.15.18:0/1646020985 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa70000acc0 con 0x7fa7100708a0 2024-04-12T21:12:25.598 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:25.600+0000 7fa716800700 1 -- 172.21.15.18:0/1646020985 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa7101143f0 con 0x7fa7100708a0 2024-04-12T21:12:25.598 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:25.600+0000 7fa707400700 1 -- 172.21.15.18:0/1646020985 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa700003d90 con 0x7fa7100708a0 2024-04-12T21:12:25.598 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:25.600+0000 7fa716800700 1 -- 172.21.15.18:0/1646020985 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa710114930 con 0x7fa7100708a0 2024-04-12T21:12:25.598 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:25.602+0000 7fa716800700 1 -- 172.21.15.18:0/1646020985 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa710071aa0 con 0x7fa7100708a0 2024-04-12T21:12:25.599 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:25.603+0000 7fa707400700 1 -- 172.21.15.18:0/1646020985 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7fa700003430 con 0x7fa7100708a0 2024-04-12T21:12:25.600 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:25.604+0000 7fa707400700 1 --2- 172.21.15.18:0/1646020985 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fa6f406ec10 0x7fa6f40710c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:12:25.600 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:25.604+0000 7fa707400700 1 -- 172.21.15.18:0/1646020985 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 6757+0+0 (secure 0 0 0) 0x7fa70008efa0 con 0x7fa7100708a0 2024-04-12T21:12:25.600 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:25.604+0000 7fa70ea00700 1 --2- 172.21.15.18:0/1646020985 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fa6f406ec10 0x7fa6f40710c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:12:25.601 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:25.605+0000 7fa70ea00700 1 --2- 172.21.15.18:0/1646020985 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fa6f406ec10 0x7fa6f40710c0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fa710115a50 tx=0x7fa6fc009e30 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:12:25.605 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:25.609+0000 7fa707400700 1 -- 172.21.15.18:0/1646020985 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7fa70005b770 con 0x7fa7100708a0 2024-04-12T21:12:25.779 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:12:25 smithi186 ceph-mon[86379]: pgmap v378: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:12:25.779 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:12:25 smithi186 ceph-mon[86379]: from='client.34696 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:12:25.779 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:12:25 smithi186 ceph-mon[86379]: from='client.34700 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:12:25.779 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:12:25 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T21:12:25.888 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:25.891+0000 7fa716800700 1 -- 172.21.15.18:0/1646020985 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fa71004bf00 con 0x7fa7100708a0 2024-04-12T21:12:25.888 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:25.892+0000 7fa707400700 1 -- 172.21.15.18:0/1646020985 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 29 v29) v1 ==== 76+0+1872 (secure 0 0 0) 0x7fa70005b300 con 0x7fa7100708a0 2024-04-12T21:12:25.889 INFO:teuthology.orchestra.run.smithi018.stdout:e29 2024-04-12T21:12:25.889 INFO:teuthology.orchestra.run.smithi018.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-12T21:12:25.889 INFO:teuthology.orchestra.run.smithi018.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-12T21:12:25.889 INFO:teuthology.orchestra.run.smithi018.stdout:legacy client fscid: 1 2024-04-12T21:12:25.889 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-12T21:12:25.889 INFO:teuthology.orchestra.run.smithi018.stdout:Filesystem 'cephfs' (1) 2024-04-12T21:12:25.889 INFO:teuthology.orchestra.run.smithi018.stdout:fs_name cephfs 2024-04-12T21:12:25.889 INFO:teuthology.orchestra.run.smithi018.stdout:epoch 24 2024-04-12T21:12:25.889 INFO:teuthology.orchestra.run.smithi018.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-04-12T21:12:25.889 INFO:teuthology.orchestra.run.smithi018.stdout:created 2024-04-12T20:51:57.362850+0000 2024-04-12T21:12:25.890 INFO:teuthology.orchestra.run.smithi018.stdout:modified 2024-04-12T21:06:59.717190+0000 2024-04-12T21:12:25.890 INFO:teuthology.orchestra.run.smithi018.stdout:tableserver 0 2024-04-12T21:12:25.890 INFO:teuthology.orchestra.run.smithi018.stdout:root 0 2024-04-12T21:12:25.890 INFO:teuthology.orchestra.run.smithi018.stdout:session_timeout 60 2024-04-12T21:12:25.890 INFO:teuthology.orchestra.run.smithi018.stdout:session_autoclose 300 2024-04-12T21:12:25.890 INFO:teuthology.orchestra.run.smithi018.stdout:max_file_size 1099511627776 2024-04-12T21:12:25.890 INFO:teuthology.orchestra.run.smithi018.stdout:max_xattr_size 65536 2024-04-12T21:12:25.890 INFO:teuthology.orchestra.run.smithi018.stdout:required_client_features {} 2024-04-12T21:12:25.890 INFO:teuthology.orchestra.run.smithi018.stdout:last_failure 0 2024-04-12T21:12:25.890 INFO:teuthology.orchestra.run.smithi018.stdout:last_failure_osd_epoch 100 2024-04-12T21:12:25.890 INFO:teuthology.orchestra.run.smithi018.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-12T21:12:25.890 INFO:teuthology.orchestra.run.smithi018.stdout:max_mds 2 2024-04-12T21:12:25.890 INFO:teuthology.orchestra.run.smithi018.stdout:in 0,1 2024-04-12T21:12:25.890 INFO:teuthology.orchestra.run.smithi018.stdout:up {} 2024-04-12T21:12:25.891 INFO:teuthology.orchestra.run.smithi018.stdout:failed 0,1 2024-04-12T21:12:25.891 INFO:teuthology.orchestra.run.smithi018.stdout:damaged 2024-04-12T21:12:25.891 INFO:teuthology.orchestra.run.smithi018.stdout:stopped 2024-04-12T21:12:25.891 INFO:teuthology.orchestra.run.smithi018.stdout:data_pools [3] 2024-04-12T21:12:25.891 INFO:teuthology.orchestra.run.smithi018.stdout:metadata_pool 2 2024-04-12T21:12:25.891 INFO:teuthology.orchestra.run.smithi018.stdout:inline_data disabled 2024-04-12T21:12:25.891 INFO:teuthology.orchestra.run.smithi018.stdout:balancer 2024-04-12T21:12:25.891 INFO:teuthology.orchestra.run.smithi018.stdout:bal_rank_mask -1 2024-04-12T21:12:25.891 INFO:teuthology.orchestra.run.smithi018.stdout:standby_count_wanted 1 2024-04-12T21:12:25.891 INFO:teuthology.orchestra.run.smithi018.stdout:qdb_cluster leader: 0 members: 2024-04-12T21:12:25.891 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-12T21:12:25.891 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-12T21:12:25.891 INFO:teuthology.orchestra.run.smithi018.stdout:Standby daemons: 2024-04-12T21:12:25.891 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-12T21:12:25.891 INFO:teuthology.orchestra.run.smithi018.stdout:[mds.cephfs.smithi186.wjnafs{-1:24277} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.186:6826/438347207,v1:172.21.15.186:6827/438347207] compat {c=[1],r=[1],i=[7ff]}] 2024-04-12T21:12:25.891 INFO:teuthology.orchestra.run.smithi018.stdout:[mds.cephfs.smithi186.jdllyn{-1:24281} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.186:6824/1464359952,v1:172.21.15.186:6825/1464359952] compat {c=[1],r=[1],i=[7ff]}] 2024-04-12T21:12:25.892 INFO:teuthology.orchestra.run.smithi018.stdout:[mds.cephfs.smithi018.acqbsc{-1:34436} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.18:6828/755179764,v1:172.21.15.18:6829/755179764] compat {c=[1],r=[1],i=[1fff]}] 2024-04-12T21:12:25.892 INFO:teuthology.orchestra.run.smithi018.stdout:[mds.cephfs.smithi018.szjluo{-1:34468} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.18:6826/2407992653,v1:172.21.15.18:6827/2407992653] compat {c=[1],r=[1],i=[1fff]}] 2024-04-12T21:12:25.892 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:25.895+0000 7fa716800700 1 -- 172.21.15.18:0/1646020985 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fa6f406ec10 msgr2=0x7fa6f40710c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:12:25.892 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:25.895+0000 7fa716800700 1 --2- 172.21.15.18:0/1646020985 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fa6f406ec10 0x7fa6f40710c0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fa710115a50 tx=0x7fa6fc009e30 comp rx=0 tx=0).stop 2024-04-12T21:12:25.892 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:25.895+0000 7fa716800700 1 -- 172.21.15.18:0/1646020985 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa7100708a0 msgr2=0x7fa71011a1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:12:25.892 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:25.895+0000 7fa716800700 1 --2- 172.21.15.18:0/1646020985 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa7100708a0 0x7fa71011a1b0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fa70000cb10 tx=0x7fa700007f90 comp rx=0 tx=0).stop 2024-04-12T21:12:25.893 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:25.895+0000 7fa716800700 1 -- 172.21.15.18:0/1646020985 shutdown_connections 2024-04-12T21:12:25.893 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:25.895+0000 7fa716800700 1 --2- 172.21.15.18:0/1646020985 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fa6f406ec10 0x7fa6f40710c0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:12:25.893 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:25.895+0000 7fa716800700 1 --2- 172.21.15.18:0/1646020985 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa7100708a0 0x7fa71011a1b0 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:12:25.893 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:25.895+0000 7fa716800700 1 --2- 172.21.15.18:0/1646020985 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa710071f90 0x7fa71011a6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:12:25.893 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:25.895+0000 7fa716800700 1 -- 172.21.15.18:0/1646020985 >> 172.21.15.18:0/1646020985 conn(0x7fa7100f89d0 msgr2=0x7fa7100f9b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:12:25.893 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:25.895+0000 7fa716800700 1 -- 172.21.15.18:0/1646020985 shutdown_connections 2024-04-12T21:12:25.893 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:25.895+0000 7fa716800700 1 -- 172.21.15.18:0/1646020985 wait complete. 2024-04-12T21:12:25.893 INFO:teuthology.orchestra.run.smithi018.stderr:dumped fsmap epoch 29 2024-04-12T21:12:26.053 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:26.056+0000 7f3b41200700 1 -- 172.21.15.18:0/3112007311 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3b3c0fbc80 msgr2=0x7f3b3c0fc050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:12:26.053 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:26.056+0000 7f3b41200700 1 --2- 172.21.15.18:0/3112007311 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3b3c0fbc80 0x7f3b3c0fc050 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f3b2c004940 tx=0x7f3b2c009e30 comp rx=0 tx=0).stop 2024-04-12T21:12:26.053 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:26.057+0000 7f3b41200700 1 -- 172.21.15.18:0/3112007311 shutdown_connections 2024-04-12T21:12:26.054 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:26.057+0000 7f3b41200700 1 --2- 172.21.15.18:0/3112007311 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3b3c0fc620 0x7f3b3c100cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:12:26.054 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:26.057+0000 7f3b41200700 1 --2- 172.21.15.18:0/3112007311 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3b3c0fbc80 0x7f3b3c0fc050 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:12:26.054 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:26.057+0000 7f3b41200700 1 -- 172.21.15.18:0/3112007311 >> 172.21.15.18:0/3112007311 conn(0x7f3b3c072bc0 msgr2=0x7f3b3c072fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:12:26.054 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:26.057+0000 7f3b41200700 1 -- 172.21.15.18:0/3112007311 shutdown_connections 2024-04-12T21:12:26.054 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:26.057+0000 7f3b41200700 1 -- 172.21.15.18:0/3112007311 wait complete. 2024-04-12T21:12:26.055 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:26.058+0000 7f3b41200700 1 Processor -- start 2024-04-12T21:12:26.055 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:26.058+0000 7f3b41200700 1 -- start start 2024-04-12T21:12:26.055 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:26.058+0000 7f3b41200700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3b3c0fbc80 0x7f3b3c11a1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:12:26.055 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:26.058+0000 7f3b41200700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3b3c0fc620 0x7f3b3c11a710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:12:26.055 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:26.058+0000 7f3b41200700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b3c11ad60 con 0x7f3b3c0fc620 2024-04-12T21:12:26.055 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:26.058+0000 7f3b41200700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b3c114130 con 0x7f3b3c0fbc80 2024-04-12T21:12:26.055 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:26.058+0000 7f3b39400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3b3c0fc620 0x7f3b3c11a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:12:26.055 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:26.059+0000 7f3b39400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3b3c0fc620 0x7f3b3c11a710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:54426/0 (socket says 172.21.15.18:54426) 2024-04-12T21:12:26.055 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:26.059+0000 7f3b39400700 1 -- 172.21.15.18:0/2488015877 learned_addr learned my addr 172.21.15.18:0/2488015877 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:12:26.056 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:26.059+0000 7f3b39e00700 1 --2- 172.21.15.18:0/2488015877 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3b3c0fbc80 0x7f3b3c11a1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:12:26.057 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:26.059+0000 7f3b39400700 1 -- 172.21.15.18:0/2488015877 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3b3c0fbc80 msgr2=0x7f3b3c11a1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:12:26.057 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:26.059+0000 7f3b39400700 1 --2- 172.21.15.18:0/2488015877 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3b3c0fbc80 0x7f3b3c11a1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:12:26.057 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:26.059+0000 7f3b39400700 1 -- 172.21.15.18:0/2488015877 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b2c005040 con 0x7f3b3c0fc620 2024-04-12T21:12:26.057 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:26.059+0000 7f3b39400700 1 --2- 172.21.15.18:0/2488015877 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3b3c0fc620 0x7f3b3c11a710 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f3b2800c830 tx=0x7f3b28004470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:12:26.057 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:26.060+0000 7f3b32000700 1 -- 172.21.15.18:0/2488015877 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3b2800a760 con 0x7f3b3c0fc620 2024-04-12T21:12:26.057 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:26.060+0000 7f3b32000700 1 -- 172.21.15.18:0/2488015877 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f3b2800ad90 con 0x7f3b3c0fc620 2024-04-12T21:12:26.057 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:26.060+0000 7f3b32000700 1 -- 172.21.15.18:0/2488015877 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3b280113a0 con 0x7f3b3c0fc620 2024-04-12T21:12:26.057 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:26.060+0000 7f3b41200700 1 -- 172.21.15.18:0/2488015877 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3b3c114410 con 0x7f3b3c0fc620 2024-04-12T21:12:26.057 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:26.060+0000 7f3b41200700 1 -- 172.21.15.18:0/2488015877 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3b3c114950 con 0x7f3b3c0fc620 2024-04-12T21:12:26.057 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:26.061+0000 7f3b41200700 1 -- 172.21.15.18:0/2488015877 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3b3c065e30 con 0x7f3b3c0fc620 2024-04-12T21:12:26.058 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:26.062+0000 7f3b32000700 1 -- 172.21.15.18:0/2488015877 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7f3b28012400 con 0x7f3b3c0fc620 2024-04-12T21:12:26.059 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:26.063+0000 7f3b32000700 1 --2- 172.21.15.18:0/2488015877 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f3b2006ec10 0x7f3b200710c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:12:26.060 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:26.063+0000 7f3b32000700 1 -- 172.21.15.18:0/2488015877 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 6757+0+0 (secure 0 0 0) 0x7f3b2808e9c0 con 0x7f3b3c0fc620 2024-04-12T21:12:26.060 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:26.063+0000 7f3b39e00700 1 --2- 172.21.15.18:0/2488015877 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f3b2006ec10 0x7f3b200710c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:12:26.060 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:26.064+0000 7f3b39e00700 1 --2- 172.21.15.18:0/2488015877 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f3b2006ec10 0x7f3b200710c0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f3b2c00b3b0 tx=0x7f3b2c009e30 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:12:26.064 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:26.068+0000 7f3b32000700 1 -- 172.21.15.18:0/2488015877 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f3b2805b1f0 con 0x7f3b3c0fc620 2024-04-12T21:12:26.305 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:26.308+0000 7f3b41200700 1 -- 172.21.15.18:0/2488015877 --> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f3b3c110600 con 0x7f3b2006ec10 2024-04-12T21:12:26.309 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:26.313+0000 7f3b32000700 1 -- 172.21.15.18:0/2488015877 <== mgr.34102 v2:172.21.15.18:6800/3812891919 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f3b2c009710 con 0x7f3b2006ec10 2024-04-12T21:12:26.309 INFO:teuthology.orchestra.run.smithi018.stdout:{ 2024-04-12T21:12:26.309 INFO:teuthology.orchestra.run.smithi018.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:f4452def3be4d350f4a8e5feaf7ebb1511be9298fe68478ba7a2d6fbd7d69a63", 2024-04-12T21:12:26.309 INFO:teuthology.orchestra.run.smithi018.stdout: "in_progress": true, 2024-04-12T21:12:26.309 INFO:teuthology.orchestra.run.smithi018.stdout: "which": "Upgrading all daemon types on all hosts", 2024-04-12T21:12:26.309 INFO:teuthology.orchestra.run.smithi018.stdout: "services_complete": [ 2024-04-12T21:12:26.309 INFO:teuthology.orchestra.run.smithi018.stdout: "crash", 2024-04-12T21:12:26.310 INFO:teuthology.orchestra.run.smithi018.stdout: "osd", 2024-04-12T21:12:26.310 INFO:teuthology.orchestra.run.smithi018.stdout: "mon", 2024-04-12T21:12:26.310 INFO:teuthology.orchestra.run.smithi018.stdout: "mgr" 2024-04-12T21:12:26.310 INFO:teuthology.orchestra.run.smithi018.stdout: ], 2024-04-12T21:12:26.310 INFO:teuthology.orchestra.run.smithi018.stdout: "progress": "14/21 daemons upgraded", 2024-04-12T21:12:26.310 INFO:teuthology.orchestra.run.smithi018.stdout: "message": "Currently upgrading mds daemons", 2024-04-12T21:12:26.310 INFO:teuthology.orchestra.run.smithi018.stdout: "is_paused": false 2024-04-12T21:12:26.310 INFO:teuthology.orchestra.run.smithi018.stdout:} 2024-04-12T21:12:26.312 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:26.315+0000 7f3b41200700 1 -- 172.21.15.18:0/2488015877 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f3b2006ec10 msgr2=0x7f3b200710c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:12:26.312 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:26.315+0000 7f3b41200700 1 --2- 172.21.15.18:0/2488015877 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f3b2006ec10 0x7f3b200710c0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f3b2c00b3b0 tx=0x7f3b2c009e30 comp rx=0 tx=0).stop 2024-04-12T21:12:26.312 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:26.315+0000 7f3b41200700 1 -- 172.21.15.18:0/2488015877 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3b3c0fc620 msgr2=0x7f3b3c11a710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:12:26.312 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:26.315+0000 7f3b41200700 1 --2- 172.21.15.18:0/2488015877 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3b3c0fc620 0x7f3b3c11a710 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f3b2800c830 tx=0x7f3b28004470 comp rx=0 tx=0).stop 2024-04-12T21:12:26.313 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:26.316+0000 7f3b41200700 1 -- 172.21.15.18:0/2488015877 shutdown_connections 2024-04-12T21:12:26.313 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:26.316+0000 7f3b41200700 1 --2- 172.21.15.18:0/2488015877 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f3b2006ec10 0x7f3b200710c0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:12:26.313 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:26.316+0000 7f3b41200700 1 --2- 172.21.15.18:0/2488015877 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3b3c0fbc80 0x7f3b3c11a1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:12:26.313 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:26.316+0000 7f3b41200700 1 --2- 172.21.15.18:0/2488015877 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3b3c0fc620 0x7f3b3c11a710 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:12:26.313 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:26.316+0000 7f3b41200700 1 -- 172.21.15.18:0/2488015877 >> 172.21.15.18:0/2488015877 conn(0x7f3b3c072bc0 msgr2=0x7f3b3c10aab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:12:26.313 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:26.316+0000 7f3b41200700 1 -- 172.21.15.18:0/2488015877 shutdown_connections 2024-04-12T21:12:26.313 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:26.316+0000 7f3b41200700 1 -- 172.21.15.18:0/2488015877 wait complete. 2024-04-12T21:12:26.473 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:26.477+0000 7f31eca00700 1 -- 172.21.15.18:0/1248381253 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f31e8070300 msgr2=0x7f31e80706d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:12:26.473 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:26.477+0000 7f31eca00700 1 --2- 172.21.15.18:0/1248381253 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f31e8070300 0x7f31e80706d0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f31d8004970 tx=0x7f31d8009e30 comp rx=0 tx=0).stop 2024-04-12T21:12:26.474 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:26.477+0000 7f31eca00700 1 -- 172.21.15.18:0/1248381253 shutdown_connections 2024-04-12T21:12:26.474 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:26.477+0000 7f31eca00700 1 --2- 172.21.15.18:0/1248381253 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f31e8070c10 0x7f31e8108fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:12:26.474 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:26.477+0000 7f31eca00700 1 --2- 172.21.15.18:0/1248381253 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f31e8070300 0x7f31e80706d0 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:12:26.474 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:26.477+0000 7f31eca00700 1 -- 172.21.15.18:0/1248381253 >> 172.21.15.18:0/1248381253 conn(0x7f31e80f87a0 msgr2=0x7f31e80fabb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:12:26.474 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:26.477+0000 7f31eca00700 1 -- 172.21.15.18:0/1248381253 shutdown_connections 2024-04-12T21:12:26.474 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:26.477+0000 7f31eca00700 1 -- 172.21.15.18:0/1248381253 wait complete. 2024-04-12T21:12:26.475 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:26.478+0000 7f31eca00700 1 Processor -- start 2024-04-12T21:12:26.475 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:26.478+0000 7f31eca00700 1 -- start start 2024-04-12T21:12:26.475 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:26.479+0000 7f31eca00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f31e8070300 0x7f31e8115c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:12:26.475 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:26.479+0000 7f31eca00700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f31e8070c10 0x7f31e8116170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:12:26.475 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:26.479+0000 7f31eca00700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31e81167c0 con 0x7f31e8070300 2024-04-12T21:12:26.475 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:26.479+0000 7f31eca00700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31e810fb90 con 0x7f31e8070c10 2024-04-12T21:12:26.476 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:26.479+0000 7f31e7400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f31e8070300 0x7f31e8115c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:12:26.476 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:26.479+0000 7f31e7400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f31e8070300 0x7f31e8115c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:54446/0 (socket says 172.21.15.18:54446) 2024-04-12T21:12:26.476 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:26.479+0000 7f31e7400700 1 -- 172.21.15.18:0/1809588006 learned_addr learned my addr 172.21.15.18:0/1809588006 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:12:26.476 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:26.479+0000 7f31e7400700 1 -- 172.21.15.18:0/1809588006 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f31e8070c10 msgr2=0x7f31e8116170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:12:26.476 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:26.479+0000 7f31e7400700 1 --2- 172.21.15.18:0/1809588006 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f31e8070c10 0x7f31e8116170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:12:26.477 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:26.479+0000 7f31e7400700 1 -- 172.21.15.18:0/1809588006 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31d8005040 con 0x7f31e8070300 2024-04-12T21:12:26.477 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:26.480+0000 7f31e7400700 1 --2- 172.21.15.18:0/1809588006 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f31e8070300 0x7f31e8115c30 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f31d8004f50 tx=0x7f31d8009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:12:26.477 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:26.480+0000 7f31df400700 1 -- 172.21.15.18:0/1809588006 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f31d800e040 con 0x7f31e8070300 2024-04-12T21:12:26.477 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:26.480+0000 7f31df400700 1 -- 172.21.15.18:0/1809588006 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f31d800bc10 con 0x7f31e8070300 2024-04-12T21:12:26.477 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:26.480+0000 7f31eca00700 1 -- 172.21.15.18:0/1809588006 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f31e810fe10 con 0x7f31e8070300 2024-04-12T21:12:26.477 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:26.480+0000 7f31df400700 1 -- 172.21.15.18:0/1809588006 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f31d801c630 con 0x7f31e8070300 2024-04-12T21:12:26.477 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:26.480+0000 7f31eca00700 1 -- 172.21.15.18:0/1809588006 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f31e8110350 con 0x7f31e8070300 2024-04-12T21:12:26.477 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:26.481+0000 7f31eca00700 1 -- 172.21.15.18:0/1809588006 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f31e8071a40 con 0x7f31e8070300 2024-04-12T21:12:26.478 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:26.482+0000 7f31df400700 1 -- 172.21.15.18:0/1809588006 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7f31d8013420 con 0x7f31e8070300 2024-04-12T21:12:26.480 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:26.483+0000 7f31df400700 1 --2- 172.21.15.18:0/1809588006 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f31cc06ec10 0x7f31cc0710c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:12:26.480 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:26.484+0000 7f31df400700 1 -- 172.21.15.18:0/1809588006 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 6757+0+0 (secure 0 0 0) 0x7f31d808fd30 con 0x7f31e8070300 2024-04-12T21:12:26.480 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:26.484+0000 7f31e6a00700 1 --2- 172.21.15.18:0/1809588006 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f31cc06ec10 0x7f31cc0710c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:12:26.480 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:26.484+0000 7f31e6a00700 1 --2- 172.21.15.18:0/1809588006 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f31cc06ec10 0x7f31cc0710c0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f31e81114d0 tx=0x7f31d4009ce0 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:12:26.484 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:26.488+0000 7f31df400700 1 -- 172.21.15.18:0/1809588006 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f31d805c4e0 con 0x7f31e8070300 2024-04-12T21:12:26.600 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:12:26 smithi018 ceph-mon[112590]: from='client.34704 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:12:26.600 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:12:26 smithi018 ceph-mon[112590]: from='client.? 172.21.15.18:0/3452128119' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T21:12:26.600 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:12:26 smithi018 ceph-mon[112590]: from='client.? 172.21.15.18:0/1646020985' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-12T21:12:26.779 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:12:26 smithi186 ceph-mon[86379]: from='client.34704 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:12:26.779 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:12:26 smithi186 ceph-mon[86379]: from='client.? 172.21.15.18:0/3452128119' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T21:12:26.779 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:12:26 smithi186 ceph-mon[86379]: from='client.? 172.21.15.18:0/1646020985' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-12T21:12:26.811 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:26.814+0000 7f31eca00700 1 -- 172.21.15.18:0/1809588006 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f31e805f5f0 con 0x7f31e8070300 2024-04-12T21:12:26.812 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:26.815+0000 7f31df400700 1 -- 172.21.15.18:0/1809588006 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+233 (secure 0 0 0) 0x7f31d805c070 con 0x7f31e8070300 2024-04-12T21:12:26.812 INFO:teuthology.orchestra.run.smithi018.stdout:HEALTH_ERR 1 filesystem is degraded; 1 filesystem is offline 2024-04-12T21:12:26.812 INFO:teuthology.orchestra.run.smithi018.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-04-12T21:12:26.812 INFO:teuthology.orchestra.run.smithi018.stdout: fs cephfs is degraded 2024-04-12T21:12:26.812 INFO:teuthology.orchestra.run.smithi018.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-04-12T21:12:26.812 INFO:teuthology.orchestra.run.smithi018.stdout: fs cephfs is offline because no MDS is active for it. 2024-04-12T21:12:26.815 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:26.818+0000 7f31eca00700 1 -- 172.21.15.18:0/1809588006 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f31cc06ec10 msgr2=0x7f31cc0710c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:12:26.815 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:26.818+0000 7f31eca00700 1 --2- 172.21.15.18:0/1809588006 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f31cc06ec10 0x7f31cc0710c0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f31e81114d0 tx=0x7f31d4009ce0 comp rx=0 tx=0).stop 2024-04-12T21:12:26.815 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:26.818+0000 7f31eca00700 1 -- 172.21.15.18:0/1809588006 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f31e8070300 msgr2=0x7f31e8115c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:12:26.815 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:26.818+0000 7f31eca00700 1 --2- 172.21.15.18:0/1809588006 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f31e8070300 0x7f31e8115c30 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f31d8004f50 tx=0x7f31d8009e30 comp rx=0 tx=0).stop 2024-04-12T21:12:26.815 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:26.819+0000 7f31eca00700 1 -- 172.21.15.18:0/1809588006 shutdown_connections 2024-04-12T21:12:26.816 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:26.819+0000 7f31eca00700 1 --2- 172.21.15.18:0/1809588006 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f31cc06ec10 0x7f31cc0710c0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:12:26.816 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:26.819+0000 7f31eca00700 1 --2- 172.21.15.18:0/1809588006 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f31e8070300 0x7f31e8115c30 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:12:26.816 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:26.819+0000 7f31eca00700 1 --2- 172.21.15.18:0/1809588006 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f31e8070c10 0x7f31e8116170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:12:26.816 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:26.819+0000 7f31eca00700 1 -- 172.21.15.18:0/1809588006 >> 172.21.15.18:0/1809588006 conn(0x7f31e80f87a0 msgr2=0x7f31e81058c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:12:26.816 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:26.819+0000 7f31eca00700 1 -- 172.21.15.18:0/1809588006 shutdown_connections 2024-04-12T21:12:26.816 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:26.819+0000 7f31eca00700 1 -- 172.21.15.18:0/1809588006 wait complete. 2024-04-12T21:12:27.600 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:12:27 smithi018 ceph-mon[112590]: pgmap v379: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:12:27.600 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:12:27 smithi018 ceph-mon[112590]: from='client.34716 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:12:27.600 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:12:27 smithi018 ceph-mon[112590]: from='client.? 172.21.15.18:0/1809588006' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-12T21:12:27.779 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:12:27 smithi186 ceph-mon[86379]: pgmap v379: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:12:27.779 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:12:27 smithi186 ceph-mon[86379]: from='client.34716 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:12:27.779 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:12:27 smithi186 ceph-mon[86379]: from='client.? 172.21.15.18:0/1809588006' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-12T21:12:29.600 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:12:29 smithi018 ceph-mon[112590]: pgmap v380: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:12:29.779 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:12:29 smithi186 ceph-mon[86379]: pgmap v380: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:12:31.600 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:12:31 smithi018 ceph-mon[112590]: pgmap v381: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:12:31.779 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:12:31 smithi186 ceph-mon[86379]: pgmap v381: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:12:33.600 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:12:33 smithi018 ceph-mon[112590]: pgmap v382: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:12:33.779 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:12:33 smithi186 ceph-mon[86379]: pgmap v382: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:12:35.600 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:12:35 smithi018 ceph-mon[112590]: pgmap v383: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:12:35.779 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:12:35 smithi186 ceph-mon[86379]: pgmap v383: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:12:37.600 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:12:37 smithi018 ceph-mon[112590]: pgmap v384: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:12:37.779 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:12:37 smithi186 ceph-mon[86379]: pgmap v384: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:12:39.600 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:12:39 smithi018 ceph-mon[112590]: pgmap v385: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:12:39.779 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:12:39 smithi186 ceph-mon[86379]: pgmap v385: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:12:40.779 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:12:40 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T21:12:40.850 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:12:40 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T21:12:41.778 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:12:41 smithi186 ceph-mon[86379]: pgmap v386: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:12:41.849 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:12:41 smithi018 ceph-mon[112590]: pgmap v386: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:12:43.778 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:12:43 smithi186 ceph-mon[86379]: pgmap v387: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:12:43.849 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:12:43 smithi018 ceph-mon[112590]: pgmap v387: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:12:45.779 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:12:45 smithi186 ceph-mon[86379]: pgmap v388: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:12:45.850 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:12:45 smithi018 ceph-mon[112590]: pgmap v388: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:12:47.779 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:12:47 smithi186 ceph-mon[86379]: pgmap v389: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:12:47.850 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:12:47 smithi018 ceph-mon[112590]: pgmap v389: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:12:49.779 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:12:49 smithi186 ceph-mon[86379]: pgmap v390: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:12:49.849 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:12:49 smithi018 ceph-mon[112590]: pgmap v390: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:12:51.778 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:12:51 smithi186 ceph-mon[86379]: pgmap v391: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:12:51.849 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:12:51 smithi018 ceph-mon[112590]: pgmap v391: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:12:53.779 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:12:53 smithi186 ceph-mon[86379]: pgmap v392: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:12:53.849 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:12:53 smithi018 ceph-mon[112590]: pgmap v392: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:12:55.779 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:12:55 smithi186 ceph-mon[86379]: pgmap v393: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:12:55.779 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:12:55 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T21:12:55.850 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:12:55 smithi018 ceph-mon[112590]: pgmap v393: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:12:55.850 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:12:55 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T21:12:56.990 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:56.993+0000 7f58a9e00700 1 -- 172.21.15.18:0/423171787 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f58a4102c30 msgr2=0x7f58a4103000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:12:56.991 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:56.993+0000 7f58a9e00700 1 --2- 172.21.15.18:0/423171787 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f58a4102c30 0x7f58a4103000 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f5898004940 tx=0x7f5898009e30 comp rx=0 tx=0).stop 2024-04-12T21:12:56.991 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:56.994+0000 7f58a9e00700 1 -- 172.21.15.18:0/423171787 shutdown_connections 2024-04-12T21:12:56.991 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:56.994+0000 7f58a9e00700 1 --2- 172.21.15.18:0/423171787 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f58a41035d0 0x7f58a40fcc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:12:56.991 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:56.994+0000 7f58a9e00700 1 --2- 172.21.15.18:0/423171787 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f58a4102c30 0x7f58a4103000 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:12:56.991 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:56.994+0000 7f58a9e00700 1 -- 172.21.15.18:0/423171787 >> 172.21.15.18:0/423171787 conn(0x7f58a40f8760 msgr2=0x7f58a40fab70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:12:56.991 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:56.994+0000 7f58a9e00700 1 -- 172.21.15.18:0/423171787 shutdown_connections 2024-04-12T21:12:56.991 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:56.994+0000 7f58a9e00700 1 -- 172.21.15.18:0/423171787 wait complete. 2024-04-12T21:12:56.991 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:56.995+0000 7f58a9e00700 1 Processor -- start 2024-04-12T21:12:56.992 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:56.995+0000 7f58a9e00700 1 -- start start 2024-04-12T21:12:56.992 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:56.996+0000 7f58a9e00700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f58a4102c30 0x7f58a40702c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:12:56.992 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:56.996+0000 7f58a9e00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f58a41035d0 0x7f58a4070800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:12:56.992 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:56.996+0000 7f58a9e00700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58a4070d90 con 0x7f58a41035d0 2024-04-12T21:12:56.992 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:56.996+0000 7f58a9e00700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58a4070ed0 con 0x7f58a4102c30 2024-04-12T21:12:56.993 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:56.996+0000 7f58a3e00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f58a41035d0 0x7f58a4070800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:12:56.993 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:56.996+0000 7f58a3e00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f58a41035d0 0x7f58a4070800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:57988/0 (socket says 172.21.15.18:57988) 2024-04-12T21:12:56.993 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:56.996+0000 7f58a3e00700 1 -- 172.21.15.18:0/2037473064 learned_addr learned my addr 172.21.15.18:0/2037473064 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:12:56.993 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:56.997+0000 7f58a8a00700 1 --2- 172.21.15.18:0/2037473064 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f58a4102c30 0x7f58a40702c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:12:56.993 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:56.997+0000 7f58a3e00700 1 -- 172.21.15.18:0/2037473064 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f58a4102c30 msgr2=0x7f58a40702c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:12:56.994 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:56.997+0000 7f58a3e00700 1 --2- 172.21.15.18:0/2037473064 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f58a4102c30 0x7f58a40702c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:12:56.994 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:56.997+0000 7f58a3e00700 1 -- 172.21.15.18:0/2037473064 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5898005040 con 0x7f58a41035d0 2024-04-12T21:12:56.994 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:56.997+0000 7f58a8a00700 1 --2- 172.21.15.18:0/2037473064 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f58a4102c30 0x7f58a40702c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-12T21:12:56.999 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:56.997+0000 7f58a3e00700 1 --2- 172.21.15.18:0/2037473064 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f58a41035d0 0x7f58a4070800 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f5890002a50 tx=0x7f5890004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:12:56.999 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:56.998+0000 7f58a0c00700 1 -- 172.21.15.18:0/2037473064 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5890007cc0 con 0x7f58a41035d0 2024-04-12T21:12:56.999 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:56.998+0000 7f58a9e00700 1 -- 172.21.15.18:0/2037473064 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f58a4119ce0 con 0x7f58a41035d0 2024-04-12T21:12:56.999 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:56.998+0000 7f58a0c00700 1 -- 172.21.15.18:0/2037473064 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f589000f040 con 0x7f58a41035d0 2024-04-12T21:12:56.999 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:56.998+0000 7f58a9e00700 1 -- 172.21.15.18:0/2037473064 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f58a411a160 con 0x7f58a41035d0 2024-04-12T21:12:56.999 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:56.998+0000 7f58a0c00700 1 -- 172.21.15.18:0/2037473064 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f589000da00 con 0x7f58a41035d0 2024-04-12T21:12:56.999 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:56.999+0000 7f58a9e00700 1 -- 172.21.15.18:0/2037473064 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f58a40fe380 con 0x7f58a41035d0 2024-04-12T21:12:56.999 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:57.000+0000 7f58a0c00700 1 -- 172.21.15.18:0/2037473064 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7f58900097c0 con 0x7f58a41035d0 2024-04-12T21:12:56.999 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:57.001+0000 7f58a0c00700 1 --2- 172.21.15.18:0/2037473064 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f588c06ec00 0x7f588c0710b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:12:56.999 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:57.002+0000 7f58a0c00700 1 -- 172.21.15.18:0/2037473064 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 6757+0+0 (secure 0 0 0) 0x7f589008e850 con 0x7f58a41035d0 2024-04-12T21:12:57.000 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:57.002+0000 7f58a8a00700 1 --2- 172.21.15.18:0/2037473064 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f588c06ec00 0x7f588c0710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:12:57.000 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:57.003+0000 7f58a8a00700 1 --2- 172.21.15.18:0/2037473064 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f588c06ec00 0x7f588c0710b0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f589800b3b0 tx=0x7f5898009e30 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:12:57.002 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:57.006+0000 7f58a0c00700 1 -- 172.21.15.18:0/2037473064 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f5890057450 con 0x7f58a41035d0 2024-04-12T21:12:57.253 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:57.257+0000 7f58a9e00700 1 -- 172.21.15.18:0/2037473064 --> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f58a404bed0 con 0x7f588c06ec00 2024-04-12T21:12:57.256 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:57.260+0000 7f58a0c00700 1 -- 172.21.15.18:0/2037473064 <== mgr.34102 v2:172.21.15.18:6800/3812891919 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f58a404bed0 con 0x7f588c06ec00 2024-04-12T21:12:57.261 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:57.264+0000 7f58a9e00700 1 -- 172.21.15.18:0/2037473064 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f588c06ec00 msgr2=0x7f588c0710b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:12:57.261 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:57.264+0000 7f58a9e00700 1 --2- 172.21.15.18:0/2037473064 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f588c06ec00 0x7f588c0710b0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f589800b3b0 tx=0x7f5898009e30 comp rx=0 tx=0).stop 2024-04-12T21:12:57.262 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:57.264+0000 7f58a9e00700 1 -- 172.21.15.18:0/2037473064 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f58a41035d0 msgr2=0x7f58a4070800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:12:57.262 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:57.264+0000 7f58a9e00700 1 --2- 172.21.15.18:0/2037473064 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f58a41035d0 0x7f58a4070800 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f5890002a50 tx=0x7f5890004600 comp rx=0 tx=0).stop 2024-04-12T21:12:57.262 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:57.265+0000 7f58a9e00700 1 -- 172.21.15.18:0/2037473064 shutdown_connections 2024-04-12T21:12:57.262 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:57.265+0000 7f58a9e00700 1 --2- 172.21.15.18:0/2037473064 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f588c06ec00 0x7f588c0710b0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:12:57.262 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:57.265+0000 7f58a9e00700 1 --2- 172.21.15.18:0/2037473064 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f58a4102c30 0x7f58a40702c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:12:57.262 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:57.265+0000 7f58a9e00700 1 --2- 172.21.15.18:0/2037473064 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f58a41035d0 0x7f58a4070800 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:12:57.262 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:57.265+0000 7f58a9e00700 1 -- 172.21.15.18:0/2037473064 >> 172.21.15.18:0/2037473064 conn(0x7f58a40f8760 msgr2=0x7f58a40f9760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:12:57.262 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:57.265+0000 7f58a9e00700 1 -- 172.21.15.18:0/2037473064 shutdown_connections 2024-04-12T21:12:57.262 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:57.265+0000 7f58a9e00700 1 -- 172.21.15.18:0/2037473064 wait complete. 2024-04-12T21:12:57.278 INFO:teuthology.orchestra.run.smithi018.stdout:true 2024-04-12T21:12:57.425 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:57.427+0000 7fd7c5c00700 1 -- 172.21.15.18:0/2612458274 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd7c0071f90 msgr2=0x7fd7c0070360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:12:57.425 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:57.427+0000 7fd7c5c00700 1 --2- 172.21.15.18:0/2612458274 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd7c0071f90 0x7fd7c0070360 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fd7b0009d10 tx=0x7fd7b0004d10 comp rx=0 tx=0).stop 2024-04-12T21:12:57.425 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:57.427+0000 7fd7c5c00700 1 -- 172.21.15.18:0/2612458274 shutdown_connections 2024-04-12T21:12:57.425 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:57.427+0000 7fd7c5c00700 1 --2- 172.21.15.18:0/2612458274 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd7c00708a0 0x7fd7c0070d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:12:57.425 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:57.427+0000 7fd7c5c00700 1 --2- 172.21.15.18:0/2612458274 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd7c0071f90 0x7fd7c0070360 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:12:57.425 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:57.427+0000 7fd7c5c00700 1 -- 172.21.15.18:0/2612458274 >> 172.21.15.18:0/2612458274 conn(0x7fd7c00f89b0 msgr2=0x7fd7c00fadc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:12:57.426 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:57.428+0000 7fd7c5c00700 1 -- 172.21.15.18:0/2612458274 shutdown_connections 2024-04-12T21:12:57.426 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:57.428+0000 7fd7c5c00700 1 -- 172.21.15.18:0/2612458274 wait complete. 2024-04-12T21:12:57.426 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:57.428+0000 7fd7c5c00700 1 Processor -- start 2024-04-12T21:12:57.426 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:57.429+0000 7fd7c5c00700 1 -- start start 2024-04-12T21:12:57.427 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:57.429+0000 7fd7c5c00700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd7c00708a0 0x7fd7c011a3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:12:57.428 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:57.429+0000 7fd7c5c00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd7c0071f90 0x7fd7c011a910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:12:57.428 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:57.429+0000 7fd7c5c00700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7c011af60 con 0x7fd7c0071f90 2024-04-12T21:12:57.428 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:57.429+0000 7fd7c5c00700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7c0114330 con 0x7fd7c00708a0 2024-04-12T21:12:57.428 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:57.429+0000 7fd7b6000700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd7c0071f90 0x7fd7c011a910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:12:57.428 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:57.429+0000 7fd7b6000700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd7c0071f90 0x7fd7c011a910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:57998/0 (socket says 172.21.15.18:57998) 2024-04-12T21:12:57.428 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:57.430+0000 7fd7b6000700 1 -- 172.21.15.18:0/2723816482 learned_addr learned my addr 172.21.15.18:0/2723816482 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:12:57.428 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:57.430+0000 7fd7b6000700 1 -- 172.21.15.18:0/2723816482 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd7c00708a0 msgr2=0x7fd7c011a3d0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-12T21:12:57.428 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:57.430+0000 7fd7b6000700 1 --2- 172.21.15.18:0/2723816482 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd7c00708a0 0x7fd7c011a3d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:12:57.428 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:57.430+0000 7fd7b6000700 1 -- 172.21.15.18:0/2723816482 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7b0009940 con 0x7fd7c0071f90 2024-04-12T21:12:57.428 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:57.430+0000 7fd7b6000700 1 --2- 172.21.15.18:0/2723816482 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd7c0071f90 0x7fd7c011a910 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fd7a8002a50 tx=0x7fd7a80046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:12:57.428 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:57.431+0000 7fd7b7400700 1 -- 172.21.15.18:0/2723816482 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd7a8007cc0 con 0x7fd7c0071f90 2024-04-12T21:12:57.429 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:57.431+0000 7fd7b7400700 1 -- 172.21.15.18:0/2723816482 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd7a800f040 con 0x7fd7c0071f90 2024-04-12T21:12:57.429 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:57.431+0000 7fd7c5c00700 1 -- 172.21.15.18:0/2723816482 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd7c0114610 con 0x7fd7c0071f90 2024-04-12T21:12:57.429 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:57.431+0000 7fd7b7400700 1 -- 172.21.15.18:0/2723816482 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd7a8009640 con 0x7fd7c0071f90 2024-04-12T21:12:57.429 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:57.431+0000 7fd7c5c00700 1 -- 172.21.15.18:0/2723816482 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd7c0114b50 con 0x7fd7c0071f90 2024-04-12T21:12:57.429 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:57.432+0000 7fd7c5c00700 1 -- 172.21.15.18:0/2723816482 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd7c0071aa0 con 0x7fd7c0071f90 2024-04-12T21:12:57.430 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:57.433+0000 7fd7b7400700 1 -- 172.21.15.18:0/2723816482 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7fd7a80097a0 con 0x7fd7c0071f90 2024-04-12T21:12:57.430 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:57.434+0000 7fd7b7400700 1 --2- 172.21.15.18:0/2723816482 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fd7a406ec10 0x7fd7a40710c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:12:57.431 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:57.434+0000 7fd7b7400700 1 -- 172.21.15.18:0/2723816482 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 6757+0+0 (secure 0 0 0) 0x7fd7a808e6c0 con 0x7fd7c0071f90 2024-04-12T21:12:57.431 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:57.435+0000 7fd7bea00700 1 --2- 172.21.15.18:0/2723816482 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fd7a406ec10 0x7fd7a40710c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:12:57.432 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:57.436+0000 7fd7bea00700 1 --2- 172.21.15.18:0/2723816482 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fd7a406ec10 0x7fd7a40710c0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fd7b000b700 tx=0x7fd7b0004d10 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:12:57.435 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:57.439+0000 7fd7b7400700 1 -- 172.21.15.18:0/2723816482 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7fd7a805af10 con 0x7fd7c0071f90 2024-04-12T21:12:57.599 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:12:57 smithi018 ceph-mon[112590]: pgmap v394: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:12:57.692 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:57.696+0000 7fd7c5c00700 1 -- 172.21.15.18:0/2723816482 --> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fd7c0110590 con 0x7fd7a406ec10 2024-04-12T21:12:57.695 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:57.698+0000 7fd7b7400700 1 -- 172.21.15.18:0/2723816482 <== mgr.34102 v2:172.21.15.18:6800/3812891919 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7fd7c0110590 con 0x7fd7a406ec10 2024-04-12T21:12:57.698 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:57.702+0000 7fd7c5c00700 1 -- 172.21.15.18:0/2723816482 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fd7a406ec10 msgr2=0x7fd7a40710c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:12:57.698 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:57.702+0000 7fd7c5c00700 1 --2- 172.21.15.18:0/2723816482 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fd7a406ec10 0x7fd7a40710c0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fd7b000b700 tx=0x7fd7b0004d10 comp rx=0 tx=0).stop 2024-04-12T21:12:57.699 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:57.702+0000 7fd7c5c00700 1 -- 172.21.15.18:0/2723816482 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd7c0071f90 msgr2=0x7fd7c011a910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:12:57.699 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:57.702+0000 7fd7c5c00700 1 --2- 172.21.15.18:0/2723816482 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd7c0071f90 0x7fd7c011a910 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fd7a8002a50 tx=0x7fd7a80046b0 comp rx=0 tx=0).stop 2024-04-12T21:12:57.699 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:57.702+0000 7fd7c5c00700 1 -- 172.21.15.18:0/2723816482 shutdown_connections 2024-04-12T21:12:57.699 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:57.702+0000 7fd7c5c00700 1 --2- 172.21.15.18:0/2723816482 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fd7a406ec10 0x7fd7a40710c0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:12:57.699 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:57.702+0000 7fd7c5c00700 1 --2- 172.21.15.18:0/2723816482 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd7c00708a0 0x7fd7c011a3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:12:57.699 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:57.702+0000 7fd7c5c00700 1 --2- 172.21.15.18:0/2723816482 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd7c0071f90 0x7fd7c011a910 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:12:57.699 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:57.702+0000 7fd7c5c00700 1 -- 172.21.15.18:0/2723816482 >> 172.21.15.18:0/2723816482 conn(0x7fd7c00f89b0 msgr2=0x7fd7c00f9bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:12:57.699 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:57.702+0000 7fd7c5c00700 1 -- 172.21.15.18:0/2723816482 shutdown_connections 2024-04-12T21:12:57.699 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:57.702+0000 7fd7c5c00700 1 -- 172.21.15.18:0/2723816482 wait complete. 2024-04-12T21:12:57.778 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:12:57 smithi186 ceph-mon[86379]: pgmap v394: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:12:57.857 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:57.860+0000 7fcfc7c00700 1 -- 172.21.15.18:0/2867652772 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fcfc0109dd0 msgr2=0x7fcfc0103440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:12:57.857 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:57.860+0000 7fcfc7c00700 1 --2- 172.21.15.18:0/2867652772 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fcfc0109dd0 0x7fcfc0103440 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fcfac004940 tx=0x7fcfac009e30 comp rx=0 tx=0).stop 2024-04-12T21:12:57.857 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:57.861+0000 7fcfc7c00700 1 -- 172.21.15.18:0/2867652772 shutdown_connections 2024-04-12T21:12:57.857 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:57.861+0000 7fcfc7c00700 1 --2- 172.21.15.18:0/2867652772 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fcfc0109dd0 0x7fcfc0103440 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:12:57.858 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:57.861+0000 7fcfc7c00700 1 --2- 172.21.15.18:0/2867652772 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcfc0109430 0x7fcfc0109800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:12:57.858 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:57.861+0000 7fcfc7c00700 1 -- 172.21.15.18:0/2867652772 >> 172.21.15.18:0/2867652772 conn(0x7fcfc0072c00 msgr2=0x7fcfc0075010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:12:57.858 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:57.861+0000 7fcfc7c00700 1 -- 172.21.15.18:0/2867652772 shutdown_connections 2024-04-12T21:12:57.858 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:57.861+0000 7fcfc7c00700 1 -- 172.21.15.18:0/2867652772 wait complete. 2024-04-12T21:12:57.859 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:57.861+0000 7fcfc7c00700 1 Processor -- start 2024-04-12T21:12:57.860 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:57.862+0000 7fcfc7c00700 1 -- start start 2024-04-12T21:12:57.860 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:57.862+0000 7fcfc7c00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fcfc0109430 0x7fcfc0119f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:12:57.860 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:57.862+0000 7fcfc7c00700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcfc0109dd0 0x7fcfc011a4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:12:57.860 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:57.862+0000 7fcfc7c00700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcfc011ab10 con 0x7fcfc0109430 2024-04-12T21:12:57.860 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:57.862+0000 7fcfc7c00700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcfc01140f0 con 0x7fcfc0109dd0 2024-04-12T21:12:57.860 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:57.862+0000 7fcfc5200700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fcfc0109430 0x7fcfc0119f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:12:57.860 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:57.862+0000 7fcfc5200700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fcfc0109430 0x7fcfc0119f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:58018/0 (socket says 172.21.15.18:58018) 2024-04-12T21:12:57.860 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:57.862+0000 7fcfc5200700 1 -- 172.21.15.18:0/851392069 learned_addr learned my addr 172.21.15.18:0/851392069 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:12:57.860 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:57.863+0000 7fcfbfe00700 1 --2- 172.21.15.18:0/851392069 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcfc0109dd0 0x7fcfc011a4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:12:57.860 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:57.863+0000 7fcfc5200700 1 -- 172.21.15.18:0/851392069 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcfc0109dd0 msgr2=0x7fcfc011a4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:12:57.860 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:57.863+0000 7fcfc5200700 1 --2- 172.21.15.18:0/851392069 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcfc0109dd0 0x7fcfc011a4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:12:57.860 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:57.863+0000 7fcfc5200700 1 -- 172.21.15.18:0/851392069 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcfac005040 con 0x7fcfc0109430 2024-04-12T21:12:57.861 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:57.863+0000 7fcfc5200700 1 --2- 172.21.15.18:0/851392069 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fcfc0109430 0x7fcfc0119f80 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fcfb4002a50 tx=0x7fcfb40046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:12:57.861 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:57.863+0000 7fcfbcc00700 1 -- 172.21.15.18:0/851392069 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcfb4007cc0 con 0x7fcfc0109430 2024-04-12T21:12:57.861 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:57.863+0000 7fcfbcc00700 1 -- 172.21.15.18:0/851392069 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fcfb400f040 con 0x7fcfc0109430 2024-04-12T21:12:57.861 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:57.863+0000 7fcfc7c00700 1 -- 172.21.15.18:0/851392069 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcfc01143d0 con 0x7fcfc0109430 2024-04-12T21:12:57.861 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:57.863+0000 7fcfbcc00700 1 -- 172.21.15.18:0/851392069 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcfb4009640 con 0x7fcfc0109430 2024-04-12T21:12:57.861 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:57.864+0000 7fcfc7c00700 1 -- 172.21.15.18:0/851392069 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcfc0114990 con 0x7fcfc0109430 2024-04-12T21:12:57.861 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:57.864+0000 7fcfc7c00700 1 -- 172.21.15.18:0/851392069 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcfc0104b80 con 0x7fcfc0109430 2024-04-12T21:12:57.862 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:57.866+0000 7fcfbcc00700 1 -- 172.21.15.18:0/851392069 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7fcfb40097a0 con 0x7fcfc0109430 2024-04-12T21:12:57.863 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:57.866+0000 7fcfbcc00700 1 --2- 172.21.15.18:0/851392069 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fcfa806ec00 0x7fcfa80710b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:12:57.863 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:57.867+0000 7fcfbcc00700 1 -- 172.21.15.18:0/851392069 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 6757+0+0 (secure 0 0 0) 0x7fcfb408e760 con 0x7fcfc0109430 2024-04-12T21:12:57.863 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:57.867+0000 7fcfbfe00700 1 --2- 172.21.15.18:0/851392069 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fcfa806ec00 0x7fcfa80710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:12:57.864 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:57.868+0000 7fcfbfe00700 1 --2- 172.21.15.18:0/851392069 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fcfa806ec00 0x7fcfa80710b0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fcfac00b3b0 tx=0x7fcfac009e30 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:12:57.868 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:57.871+0000 7fcfbcc00700 1 -- 172.21.15.18:0/851392069 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7fcfb405af10 con 0x7fcfc0109430 2024-04-12T21:12:58.097 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:58.101+0000 7fcfc7c00700 1 -- 172.21.15.18:0/851392069 --> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fcfc0115670 con 0x7fcfa806ec00 2024-04-12T21:12:58.111 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:58.115+0000 7fcfbcc00700 1 -- 172.21.15.18:0/851392069 <== mgr.34102 v2:172.21.15.18:6800/3812891919 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7fcfc0115670 con 0x7fcfa806ec00 2024-04-12T21:12:58.112 INFO:teuthology.orchestra.run.smithi018.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-12T21:12:58.112 INFO:teuthology.orchestra.run.smithi018.stdout:alertmanager.smithi018 smithi018 *:9093,9094 running (13m) 5m ago 28m 14.7M - 0.25.0 c8568f914cd2 c332090068e8 2024-04-12T21:12:58.112 INFO:teuthology.orchestra.run.smithi018.stdout:crash.smithi018 smithi018 running (9m) 5m ago 28m 6665k - 19.0.0-2879-g324baf8d 8717720dc5e5 b7f80d1f7982 2024-04-12T21:12:58.112 INFO:teuthology.orchestra.run.smithi018.stdout:crash.smithi186 smithi186 running (9m) 6m ago 26m 6660k - 19.0.0-2879-g324baf8d 8717720dc5e5 a4f55c81b623 2024-04-12T21:12:58.112 INFO:teuthology.orchestra.run.smithi018.stdout:grafana.smithi018 smithi018 *:3000 running (12m) 5m ago 27m 74.8M - 9.4.12 2bacad6d85d8 b08e3bf176dd 2024-04-12T21:12:58.112 INFO:teuthology.orchestra.run.smithi018.stdout:mds.cephfs.smithi018.acqbsc smithi018 running (5m) 5m ago 20m 11.1M - 19.0.0-2879-g324baf8d 8717720dc5e5 021f99ab24ae 2024-04-12T21:12:58.112 INFO:teuthology.orchestra.run.smithi018.stdout:mds.cephfs.smithi018.szjluo smithi018 running (5m) 5m ago 20m 10.9M - 19.0.0-2879-g324baf8d 8717720dc5e5 0003117479d4 2024-04-12T21:12:58.112 INFO:teuthology.orchestra.run.smithi018.stdout:mds.cephfs.smithi186.jdllyn smithi186 running (20m) 6m ago 20m 15.6M - 17.2.7-827-gd3b5060a 21de9b7369ad 2b2bc46bd29a 2024-04-12T21:12:58.112 INFO:teuthology.orchestra.run.smithi018.stdout:mds.cephfs.smithi186.wjnafs smithi186 running (20m) 6m ago 20m 15.4M - 17.2.7-827-gd3b5060a 21de9b7369ad 74b115d6d6fd 2024-04-12T21:12:58.112 INFO:teuthology.orchestra.run.smithi018.stdout:mgr.smithi018.xjzwga smithi018 *:8443,9283,8765 running (16m) 5m ago 29m 509M - 19.0.0-2879-g324baf8d 8717720dc5e5 56088305eab7 2024-04-12T21:12:58.112 INFO:teuthology.orchestra.run.smithi018.stdout:mgr.smithi186.kqcfxk smithi186 *:8443,9283,8765 running (14m) 6m ago 26m 452M - 19.0.0-2879-g324baf8d 8717720dc5e5 09cd766e5e0d 2024-04-12T21:12:58.112 INFO:teuthology.orchestra.run.smithi018.stdout:mon.smithi018 smithi018 running (11m) 5m ago 29m 57.0M 2048M 19.0.0-2879-g324baf8d 8717720dc5e5 df3e201dfed6 2024-04-12T21:12:58.113 INFO:teuthology.orchestra.run.smithi018.stdout:mon.smithi186 smithi186 running (10m) 6m ago 26m 37.7M 2048M 19.0.0-2879-g324baf8d 8717720dc5e5 c77498da9b2b 2024-04-12T21:12:58.113 INFO:teuthology.orchestra.run.smithi018.stdout:node-exporter.smithi018 smithi018 *:9100 running (14m) 5m ago 28m 7780k - 1.5.0 0da6a335fe13 326c3d9c6847 2024-04-12T21:12:58.113 INFO:teuthology.orchestra.run.smithi018.stdout:node-exporter.smithi186 smithi186 *:9100 running (14m) 6m ago 26m 7826k - 1.5.0 0da6a335fe13 ec2112baeecf 2024-04-12T21:12:58.113 INFO:teuthology.orchestra.run.smithi018.stdout:osd.0 smithi018 running (9m) 5m ago 25m 121M 1559M 19.0.0-2879-g324baf8d 8717720dc5e5 60b1273ca047 2024-04-12T21:12:58.113 INFO:teuthology.orchestra.run.smithi018.stdout:osd.1 smithi018 running (8m) 5m ago 25m 91.3M 1559M 19.0.0-2879-g324baf8d 8717720dc5e5 170ce1e5fb8d 2024-04-12T21:12:58.113 INFO:teuthology.orchestra.run.smithi018.stdout:osd.2 smithi018 running (8m) 5m ago 24m 90.3M 1559M 19.0.0-2879-g324baf8d 8717720dc5e5 c2cbc6aa2839 2024-04-12T21:12:58.113 INFO:teuthology.orchestra.run.smithi018.stdout:osd.3 smithi186 running (7m) 6m ago 23m 96.6M 2581M 19.0.0-2879-g324baf8d 8717720dc5e5 0138e8262e4b 2024-04-12T21:12:58.113 INFO:teuthology.orchestra.run.smithi018.stdout:osd.4 smithi186 running (6m) 6m ago 23m 87.2M 2581M 19.0.0-2879-g324baf8d 8717720dc5e5 b20cb63b25c9 2024-04-12T21:12:58.113 INFO:teuthology.orchestra.run.smithi018.stdout:osd.5 smithi186 running (6m) 6m ago 22m 11.1M 2581M 19.0.0-2879-g324baf8d 8717720dc5e5 2513bea2610e 2024-04-12T21:12:58.113 INFO:teuthology.orchestra.run.smithi018.stdout:prometheus.smithi018 smithi018 *:9095 running (14m) 5m ago 27m 60.6M - 2.43.0 a07b618ecd1d f58841127e21 2024-04-12T21:12:58.115 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:58.118+0000 7fcfc7c00700 1 -- 172.21.15.18:0/851392069 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fcfa806ec00 msgr2=0x7fcfa80710b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:12:58.115 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:58.118+0000 7fcfc7c00700 1 --2- 172.21.15.18:0/851392069 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fcfa806ec00 0x7fcfa80710b0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fcfac00b3b0 tx=0x7fcfac009e30 comp rx=0 tx=0).stop 2024-04-12T21:12:58.115 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:58.118+0000 7fcfc7c00700 1 -- 172.21.15.18:0/851392069 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fcfc0109430 msgr2=0x7fcfc0119f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:12:58.115 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:58.118+0000 7fcfc7c00700 1 --2- 172.21.15.18:0/851392069 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fcfc0109430 0x7fcfc0119f80 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fcfb4002a50 tx=0x7fcfb40046b0 comp rx=0 tx=0).stop 2024-04-12T21:12:58.115 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:58.118+0000 7fcfc7c00700 1 -- 172.21.15.18:0/851392069 shutdown_connections 2024-04-12T21:12:58.115 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:58.118+0000 7fcfc7c00700 1 --2- 172.21.15.18:0/851392069 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fcfa806ec00 0x7fcfa80710b0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:12:58.115 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:58.118+0000 7fcfc7c00700 1 --2- 172.21.15.18:0/851392069 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fcfc0109430 0x7fcfc0119f80 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:12:58.115 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:58.118+0000 7fcfc7c00700 1 --2- 172.21.15.18:0/851392069 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcfc0109dd0 0x7fcfc011a4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:12:58.115 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:58.118+0000 7fcfc7c00700 1 -- 172.21.15.18:0/851392069 >> 172.21.15.18:0/851392069 conn(0x7fcfc0072c00 msgr2=0x7fcfc0074c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:12:58.115 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:58.118+0000 7fcfc7c00700 1 -- 172.21.15.18:0/851392069 shutdown_connections 2024-04-12T21:12:58.115 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:58.119+0000 7fcfc7c00700 1 -- 172.21.15.18:0/851392069 wait complete. 2024-04-12T21:12:58.269 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:58.272+0000 7fe685000700 1 -- 172.21.15.18:0/772659266 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe680065f40 msgr2=0x7fe6801091c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:12:58.269 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:58.272+0000 7fe685000700 1 --2- 172.21.15.18:0/772659266 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe680065f40 0x7fe6801091c0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fe66c004940 tx=0x7fe66c009e30 comp rx=0 tx=0).stop 2024-04-12T21:12:58.269 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:58.272+0000 7fe685000700 1 -- 172.21.15.18:0/772659266 shutdown_connections 2024-04-12T21:12:58.269 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:58.272+0000 7fe685000700 1 --2- 172.21.15.18:0/772659266 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe680065f40 0x7fe6801091c0 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:12:58.269 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:58.272+0000 7fe685000700 1 --2- 172.21.15.18:0/772659266 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe680065630 0x7fe680065a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:12:58.269 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:58.272+0000 7fe685000700 1 -- 172.21.15.18:0/772659266 >> 172.21.15.18:0/772659266 conn(0x7fe680071b70 msgr2=0x7fe680071f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:12:58.269 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:58.272+0000 7fe685000700 1 -- 172.21.15.18:0/772659266 shutdown_connections 2024-04-12T21:12:58.269 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:58.272+0000 7fe685000700 1 -- 172.21.15.18:0/772659266 wait complete. 2024-04-12T21:12:58.270 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:58.273+0000 7fe685000700 1 Processor -- start 2024-04-12T21:12:58.270 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:58.274+0000 7fe685000700 1 -- start start 2024-04-12T21:12:58.271 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:58.274+0000 7fe685000700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe680065630 0x7fe680115b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:12:58.271 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:58.274+0000 7fe685000700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe680065f40 0x7fe6801160c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:12:58.271 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:58.274+0000 7fe685000700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe680116710 con 0x7fe680065f40 2024-04-12T21:12:58.271 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:58.274+0000 7fe685000700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe68010fb20 con 0x7fe680065630 2024-04-12T21:12:58.272 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:58.275+0000 7fe67ea00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe680065f40 0x7fe6801160c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:12:58.272 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:58.275+0000 7fe67ea00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe680065f40 0x7fe6801160c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:58026/0 (socket says 172.21.15.18:58026) 2024-04-12T21:12:58.272 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:58.275+0000 7fe67ea00700 1 -- 172.21.15.18:0/2408826490 learned_addr learned my addr 172.21.15.18:0/2408826490 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:12:58.272 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:58.275+0000 7fe67f400700 1 --2- 172.21.15.18:0/2408826490 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe680065630 0x7fe680115b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:12:58.272 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:58.275+0000 7fe67ea00700 1 -- 172.21.15.18:0/2408826490 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe680065630 msgr2=0x7fe680115b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:12:58.272 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:58.275+0000 7fe67ea00700 1 --2- 172.21.15.18:0/2408826490 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe680065630 0x7fe680115b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:12:58.273 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:58.275+0000 7fe67ea00700 1 -- 172.21.15.18:0/2408826490 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe66c005040 con 0x7fe680065f40 2024-04-12T21:12:58.273 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:58.275+0000 7fe67ea00700 1 --2- 172.21.15.18:0/2408826490 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe680065f40 0x7fe6801160c0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fe66c00b8b0 tx=0x7fe66c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:12:58.273 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:58.276+0000 7fe677400700 1 -- 172.21.15.18:0/2408826490 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe66c00e040 con 0x7fe680065f40 2024-04-12T21:12:58.273 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:58.276+0000 7fe677400700 1 -- 172.21.15.18:0/2408826490 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe66c004290 con 0x7fe680065f40 2024-04-12T21:12:58.273 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:58.276+0000 7fe677400700 1 -- 172.21.15.18:0/2408826490 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe66c01c3f0 con 0x7fe680065f40 2024-04-12T21:12:58.273 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:58.276+0000 7fe685000700 1 -- 172.21.15.18:0/2408826490 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe68010fda0 con 0x7fe680065f40 2024-04-12T21:12:58.273 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:58.276+0000 7fe685000700 1 -- 172.21.15.18:0/2408826490 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe680110360 con 0x7fe680065f40 2024-04-12T21:12:58.273 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:58.277+0000 7fe685000700 1 -- 172.21.15.18:0/2408826490 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe6800fc170 con 0x7fe680065f40 2024-04-12T21:12:58.274 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:58.278+0000 7fe677400700 1 -- 172.21.15.18:0/2408826490 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7fe66c0137b0 con 0x7fe680065f40 2024-04-12T21:12:58.275 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:58.279+0000 7fe677400700 1 --2- 172.21.15.18:0/2408826490 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fe66406ec00 0x7fe6640710b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:12:58.275 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:58.279+0000 7fe677400700 1 -- 172.21.15.18:0/2408826490 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 6757+0+0 (secure 0 0 0) 0x7fe66c08fce0 con 0x7fe680065f40 2024-04-12T21:12:58.276 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:58.280+0000 7fe67f400700 1 --2- 172.21.15.18:0/2408826490 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fe66406ec00 0x7fe6640710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:12:58.276 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:58.280+0000 7fe67f400700 1 --2- 172.21.15.18:0/2408826490 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fe66406ec00 0x7fe6640710b0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fe670007fd0 tx=0x7fe670009040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:12:58.280 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:58.284+0000 7fe677400700 1 -- 172.21.15.18:0/2408826490 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7fe66c058a00 con 0x7fe680065f40 2024-04-12T21:12:58.609 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:58.612+0000 7fe685000700 1 -- 172.21.15.18:0/2408826490 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fe680111200 con 0x7fe680065f40 2024-04-12T21:12:58.610 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:58.614+0000 7fe677400700 1 -- 172.21.15.18:0/2408826490 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+837 (secure 0 0 0) 0x7fe66c05c0c0 con 0x7fe680065f40 2024-04-12T21:12:58.610 INFO:teuthology.orchestra.run.smithi018.stdout:{ 2024-04-12T21:12:58.611 INFO:teuthology.orchestra.run.smithi018.stdout: "mon": { 2024-04-12T21:12:58.611 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 19.0.0-2879-g324baf8d (324baf8ddb91dea46cac50cb535c306115791355) squid (dev)": 2 2024-04-12T21:12:58.611 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-12T21:12:58.611 INFO:teuthology.orchestra.run.smithi018.stdout: "mgr": { 2024-04-12T21:12:58.611 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 19.0.0-2879-g324baf8d (324baf8ddb91dea46cac50cb535c306115791355) squid (dev)": 2 2024-04-12T21:12:58.611 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-12T21:12:58.611 INFO:teuthology.orchestra.run.smithi018.stdout: "osd": { 2024-04-12T21:12:58.611 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 19.0.0-2879-g324baf8d (324baf8ddb91dea46cac50cb535c306115791355) squid (dev)": 6 2024-04-12T21:12:58.611 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-12T21:12:58.611 INFO:teuthology.orchestra.run.smithi018.stdout: "mds": { 2024-04-12T21:12:58.611 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 17.2.7-827-gd3b5060a (d3b5060a17449edfae9385cdf792afec109fca5d) quincy (stable)": 2, 2024-04-12T21:12:58.611 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 19.0.0-2879-g324baf8d (324baf8ddb91dea46cac50cb535c306115791355) squid (dev)": 2 2024-04-12T21:12:58.611 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-12T21:12:58.611 INFO:teuthology.orchestra.run.smithi018.stdout: "overall": { 2024-04-12T21:12:58.611 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 17.2.7-827-gd3b5060a (d3b5060a17449edfae9385cdf792afec109fca5d) quincy (stable)": 2, 2024-04-12T21:12:58.612 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 19.0.0-2879-g324baf8d (324baf8ddb91dea46cac50cb535c306115791355) squid (dev)": 12 2024-04-12T21:12:58.612 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-04-12T21:12:58.612 INFO:teuthology.orchestra.run.smithi018.stdout:} 2024-04-12T21:12:58.613 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:58.617+0000 7fe685000700 1 -- 172.21.15.18:0/2408826490 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fe66406ec00 msgr2=0x7fe6640710b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:12:58.613 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:58.617+0000 7fe685000700 1 --2- 172.21.15.18:0/2408826490 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fe66406ec00 0x7fe6640710b0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fe670007fd0 tx=0x7fe670009040 comp rx=0 tx=0).stop 2024-04-12T21:12:58.613 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:58.617+0000 7fe685000700 1 -- 172.21.15.18:0/2408826490 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe680065f40 msgr2=0x7fe6801160c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:12:58.613 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:58.617+0000 7fe685000700 1 --2- 172.21.15.18:0/2408826490 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe680065f40 0x7fe6801160c0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fe66c00b8b0 tx=0x7fe66c009e30 comp rx=0 tx=0).stop 2024-04-12T21:12:58.614 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:58.617+0000 7fe685000700 1 -- 172.21.15.18:0/2408826490 shutdown_connections 2024-04-12T21:12:58.614 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:58.617+0000 7fe685000700 1 --2- 172.21.15.18:0/2408826490 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fe66406ec00 0x7fe6640710b0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:12:58.614 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:58.617+0000 7fe685000700 1 --2- 172.21.15.18:0/2408826490 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe680065630 0x7fe680115b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:12:58.614 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:58.617+0000 7fe685000700 1 --2- 172.21.15.18:0/2408826490 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe680065f40 0x7fe6801160c0 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:12:58.614 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:58.617+0000 7fe685000700 1 -- 172.21.15.18:0/2408826490 >> 172.21.15.18:0/2408826490 conn(0x7fe680071b70 msgr2=0x7fe680105a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:12:58.614 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:58.617+0000 7fe685000700 1 -- 172.21.15.18:0/2408826490 shutdown_connections 2024-04-12T21:12:58.614 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:58.617+0000 7fe685000700 1 -- 172.21.15.18:0/2408826490 wait complete. 2024-04-12T21:12:58.779 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:12:58 smithi186 ceph-mon[86379]: from='client.34724 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:12:58.779 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:58.783+0000 7f3079c00700 1 -- 172.21.15.18:0/774142773 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3074102e10 msgr2=0x7f30741031e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:12:58.779 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:58.783+0000 7f3079c00700 1 --2- 172.21.15.18:0/774142773 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3074102e10 0x7f30741031e0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f3064004940 tx=0x7f3064009e30 comp rx=0 tx=0).stop 2024-04-12T21:12:58.779 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:58.783+0000 7f3079c00700 1 -- 172.21.15.18:0/774142773 shutdown_connections 2024-04-12T21:12:58.780 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:58.783+0000 7f3079c00700 1 --2- 172.21.15.18:0/774142773 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f30741037b0 0x7f30740fce20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:12:58.780 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:58.783+0000 7f3079c00700 1 --2- 172.21.15.18:0/774142773 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3074102e10 0x7f30741031e0 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:12:58.780 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:58.783+0000 7f3079c00700 1 -- 172.21.15.18:0/774142773 >> 172.21.15.18:0/774142773 conn(0x7f30740f89a0 msgr2=0x7f30740fadb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:12:58.780 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:12:58 smithi018 ceph-mon[112590]: from='client.34724 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:12:58.781 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:58.783+0000 7f3079c00700 1 -- 172.21.15.18:0/774142773 shutdown_connections 2024-04-12T21:12:58.781 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:58.783+0000 7f3079c00700 1 -- 172.21.15.18:0/774142773 wait complete. 2024-04-12T21:12:58.781 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:58.784+0000 7f3079c00700 1 Processor -- start 2024-04-12T21:12:58.781 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:58.784+0000 7f3079c00700 1 -- start start 2024-04-12T21:12:58.781 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:58.785+0000 7f3079c00700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3074102e10 0x7f3074115da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:12:58.781 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:58.785+0000 7f3079c00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f30741037b0 0x7f30741162e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:12:58.781 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:58.785+0000 7f3079c00700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30741169c0 con 0x7f30741037b0 2024-04-12T21:12:58.781 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:58.785+0000 7f3079c00700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f307410fd00 con 0x7f3074102e10 2024-04-12T21:12:58.783 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:58.785+0000 7f3072000700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f30741037b0 0x7f30741162e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:12:58.783 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:58.785+0000 7f3072000700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f30741037b0 0x7f30741162e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:58044/0 (socket says 172.21.15.18:58044) 2024-04-12T21:12:58.783 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:58.785+0000 7f3072000700 1 -- 172.21.15.18:0/1421668859 learned_addr learned my addr 172.21.15.18:0/1421668859 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:12:58.783 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:58.785+0000 7f3072a00700 1 --2- 172.21.15.18:0/1421668859 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3074102e10 0x7f3074115da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:12:58.783 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:58.786+0000 7f3072000700 1 -- 172.21.15.18:0/1421668859 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3074102e10 msgr2=0x7f3074115da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:12:58.783 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:58.786+0000 7f3072000700 1 --2- 172.21.15.18:0/1421668859 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3074102e10 0x7f3074115da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:12:58.783 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:58.786+0000 7f3072000700 1 -- 172.21.15.18:0/1421668859 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3064005040 con 0x7f30741037b0 2024-04-12T21:12:58.783 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:58.786+0000 7f3072000700 1 --2- 172.21.15.18:0/1421668859 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f30741037b0 0x7f30741162e0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f306000b790 tx=0x7f3060009220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:12:58.783 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:58.786+0000 7f306aa00700 1 -- 172.21.15.18:0/1421668859 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f306000f6f0 con 0x7f30741037b0 2024-04-12T21:12:58.784 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:58.786+0000 7f306aa00700 1 -- 172.21.15.18:0/1421668859 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f306000fd20 con 0x7f30741037b0 2024-04-12T21:12:58.784 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:58.786+0000 7f3079c00700 1 -- 172.21.15.18:0/1421668859 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f307410ff80 con 0x7f30741037b0 2024-04-12T21:12:58.784 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:58.787+0000 7f306aa00700 1 -- 172.21.15.18:0/1421668859 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f30600103a0 con 0x7f30741037b0 2024-04-12T21:12:58.784 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:58.787+0000 7f3079c00700 1 -- 172.21.15.18:0/1421668859 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3074110540 con 0x7f30741037b0 2024-04-12T21:12:58.784 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:58.788+0000 7f3079c00700 1 -- 172.21.15.18:0/1421668859 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f30740fe560 con 0x7f30741037b0 2024-04-12T21:12:58.786 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:58.790+0000 7f306aa00700 1 -- 172.21.15.18:0/1421668859 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7f306000f850 con 0x7f30741037b0 2024-04-12T21:12:58.787 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:58.791+0000 7f306aa00700 1 --2- 172.21.15.18:0/1421668859 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f305806ec10 0x7f30580710c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:12:58.787 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:58.791+0000 7f306aa00700 1 -- 172.21.15.18:0/1421668859 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 6757+0+0 (secure 0 0 0) 0x7f306008e800 con 0x7f30741037b0 2024-04-12T21:12:58.787 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:58.791+0000 7f3072a00700 1 --2- 172.21.15.18:0/1421668859 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f305806ec10 0x7f30580710c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:12:58.788 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:58.792+0000 7f3072a00700 1 --2- 172.21.15.18:0/1421668859 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f305806ec10 0x7f30580710c0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f306400b020 tx=0x7f3064009e30 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:12:58.792 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:58.796+0000 7f306aa00700 1 -- 172.21.15.18:0/1421668859 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f306005afd0 con 0x7f30741037b0 2024-04-12T21:12:59.069 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:59.072+0000 7f3079c00700 1 -- 172.21.15.18:0/1421668859 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f307410a010 con 0x7f30741037b0 2024-04-12T21:12:59.070 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:59.073+0000 7f306aa00700 1 -- 172.21.15.18:0/1421668859 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 29 v29) v1 ==== 76+0+1872 (secure 0 0 0) 0x7f306005ab60 con 0x7f30741037b0 2024-04-12T21:12:59.072 INFO:teuthology.orchestra.run.smithi018.stdout:e29 2024-04-12T21:12:59.072 INFO:teuthology.orchestra.run.smithi018.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-12T21:12:59.072 INFO:teuthology.orchestra.run.smithi018.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-12T21:12:59.072 INFO:teuthology.orchestra.run.smithi018.stdout:legacy client fscid: 1 2024-04-12T21:12:59.072 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-12T21:12:59.072 INFO:teuthology.orchestra.run.smithi018.stdout:Filesystem 'cephfs' (1) 2024-04-12T21:12:59.072 INFO:teuthology.orchestra.run.smithi018.stdout:fs_name cephfs 2024-04-12T21:12:59.072 INFO:teuthology.orchestra.run.smithi018.stdout:epoch 24 2024-04-12T21:12:59.072 INFO:teuthology.orchestra.run.smithi018.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-04-12T21:12:59.072 INFO:teuthology.orchestra.run.smithi018.stdout:created 2024-04-12T20:51:57.362850+0000 2024-04-12T21:12:59.072 INFO:teuthology.orchestra.run.smithi018.stdout:modified 2024-04-12T21:06:59.717190+0000 2024-04-12T21:12:59.072 INFO:teuthology.orchestra.run.smithi018.stdout:tableserver 0 2024-04-12T21:12:59.073 INFO:teuthology.orchestra.run.smithi018.stdout:root 0 2024-04-12T21:12:59.073 INFO:teuthology.orchestra.run.smithi018.stdout:session_timeout 60 2024-04-12T21:12:59.073 INFO:teuthology.orchestra.run.smithi018.stdout:session_autoclose 300 2024-04-12T21:12:59.073 INFO:teuthology.orchestra.run.smithi018.stdout:max_file_size 1099511627776 2024-04-12T21:12:59.073 INFO:teuthology.orchestra.run.smithi018.stdout:max_xattr_size 65536 2024-04-12T21:12:59.073 INFO:teuthology.orchestra.run.smithi018.stdout:required_client_features {} 2024-04-12T21:12:59.073 INFO:teuthology.orchestra.run.smithi018.stdout:last_failure 0 2024-04-12T21:12:59.073 INFO:teuthology.orchestra.run.smithi018.stdout:last_failure_osd_epoch 100 2024-04-12T21:12:59.073 INFO:teuthology.orchestra.run.smithi018.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-12T21:12:59.073 INFO:teuthology.orchestra.run.smithi018.stdout:max_mds 2 2024-04-12T21:12:59.073 INFO:teuthology.orchestra.run.smithi018.stdout:in 0,1 2024-04-12T21:12:59.073 INFO:teuthology.orchestra.run.smithi018.stdout:up {} 2024-04-12T21:12:59.073 INFO:teuthology.orchestra.run.smithi018.stdout:failed 0,1 2024-04-12T21:12:59.073 INFO:teuthology.orchestra.run.smithi018.stdout:damaged 2024-04-12T21:12:59.073 INFO:teuthology.orchestra.run.smithi018.stdout:stopped 2024-04-12T21:12:59.074 INFO:teuthology.orchestra.run.smithi018.stdout:data_pools [3] 2024-04-12T21:12:59.074 INFO:teuthology.orchestra.run.smithi018.stdout:metadata_pool 2 2024-04-12T21:12:59.074 INFO:teuthology.orchestra.run.smithi018.stdout:inline_data disabled 2024-04-12T21:12:59.074 INFO:teuthology.orchestra.run.smithi018.stdout:balancer 2024-04-12T21:12:59.074 INFO:teuthology.orchestra.run.smithi018.stdout:bal_rank_mask -1 2024-04-12T21:12:59.074 INFO:teuthology.orchestra.run.smithi018.stdout:standby_count_wanted 1 2024-04-12T21:12:59.074 INFO:teuthology.orchestra.run.smithi018.stdout:qdb_cluster leader: 0 members: 2024-04-12T21:12:59.074 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-12T21:12:59.074 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-12T21:12:59.074 INFO:teuthology.orchestra.run.smithi018.stdout:Standby daemons: 2024-04-12T21:12:59.074 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-12T21:12:59.074 INFO:teuthology.orchestra.run.smithi018.stdout:[mds.cephfs.smithi186.wjnafs{-1:24277} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.186:6826/438347207,v1:172.21.15.186:6827/438347207] compat {c=[1],r=[1],i=[7ff]}] 2024-04-12T21:12:59.074 INFO:teuthology.orchestra.run.smithi018.stdout:[mds.cephfs.smithi186.jdllyn{-1:24281} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.186:6824/1464359952,v1:172.21.15.186:6825/1464359952] compat {c=[1],r=[1],i=[7ff]}] 2024-04-12T21:12:59.074 INFO:teuthology.orchestra.run.smithi018.stdout:[mds.cephfs.smithi018.acqbsc{-1:34436} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.18:6828/755179764,v1:172.21.15.18:6829/755179764] compat {c=[1],r=[1],i=[1fff]}] 2024-04-12T21:12:59.074 INFO:teuthology.orchestra.run.smithi018.stdout:[mds.cephfs.smithi018.szjluo{-1:34468} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.18:6826/2407992653,v1:172.21.15.18:6827/2407992653] compat {c=[1],r=[1],i=[1fff]}] 2024-04-12T21:12:59.075 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:59.076+0000 7f3079c00700 1 -- 172.21.15.18:0/1421668859 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f305806ec10 msgr2=0x7f30580710c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:12:59.075 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:59.076+0000 7f3079c00700 1 --2- 172.21.15.18:0/1421668859 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f305806ec10 0x7f30580710c0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f306400b020 tx=0x7f3064009e30 comp rx=0 tx=0).stop 2024-04-12T21:12:59.076 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:59.076+0000 7f3079c00700 1 -- 172.21.15.18:0/1421668859 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f30741037b0 msgr2=0x7f30741162e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:12:59.076 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:59.076+0000 7f3079c00700 1 --2- 172.21.15.18:0/1421668859 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f30741037b0 0x7f30741162e0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f306000b790 tx=0x7f3060009220 comp rx=0 tx=0).stop 2024-04-12T21:12:59.076 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:59.076+0000 7f3079c00700 1 -- 172.21.15.18:0/1421668859 shutdown_connections 2024-04-12T21:12:59.076 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:59.076+0000 7f3079c00700 1 --2- 172.21.15.18:0/1421668859 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f305806ec10 0x7f30580710c0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:12:59.076 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:59.076+0000 7f3079c00700 1 --2- 172.21.15.18:0/1421668859 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3074102e10 0x7f3074115da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:12:59.076 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:59.076+0000 7f3079c00700 1 --2- 172.21.15.18:0/1421668859 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f30741037b0 0x7f30741162e0 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:12:59.076 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:59.076+0000 7f3079c00700 1 -- 172.21.15.18:0/1421668859 >> 172.21.15.18:0/1421668859 conn(0x7f30740f89a0 msgr2=0x7f3074071a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:12:59.076 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:59.077+0000 7f3079c00700 1 -- 172.21.15.18:0/1421668859 shutdown_connections 2024-04-12T21:12:59.076 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:59.077+0000 7f3079c00700 1 -- 172.21.15.18:0/1421668859 wait complete. 2024-04-12T21:12:59.076 INFO:teuthology.orchestra.run.smithi018.stderr:dumped fsmap epoch 29 2024-04-12T21:12:59.219 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:59.222+0000 7f57d5600700 1 -- 172.21.15.18:0/1963471399 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f57d01059a0 msgr2=0x7f57d00ff010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:12:59.219 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:59.222+0000 7f57d5600700 1 --2- 172.21.15.18:0/1963471399 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f57d01059a0 0x7f57d00ff010 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f57bc004940 tx=0x7f57bc009e30 comp rx=0 tx=0).stop 2024-04-12T21:12:59.219 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:59.222+0000 7f57d5600700 1 -- 172.21.15.18:0/1963471399 shutdown_connections 2024-04-12T21:12:59.219 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:59.222+0000 7f57d5600700 1 --2- 172.21.15.18:0/1963471399 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f57d01059a0 0x7f57d00ff010 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:12:59.219 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:59.222+0000 7f57d5600700 1 --2- 172.21.15.18:0/1963471399 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f57d0105000 0x7f57d01053d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:12:59.219 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:59.222+0000 7f57d5600700 1 -- 172.21.15.18:0/1963471399 >> 172.21.15.18:0/1963471399 conn(0x7f57d00fabd0 msgr2=0x7f57d00fcfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:12:59.219 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:59.223+0000 7f57d5600700 1 -- 172.21.15.18:0/1963471399 shutdown_connections 2024-04-12T21:12:59.219 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:59.223+0000 7f57d5600700 1 -- 172.21.15.18:0/1963471399 wait complete. 2024-04-12T21:12:59.220 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:59.224+0000 7f57d5600700 1 Processor -- start 2024-04-12T21:12:59.220 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:59.224+0000 7f57d5600700 1 -- start start 2024-04-12T21:12:59.220 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:59.224+0000 7f57d5600700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f57d0105000 0x7f57d0115d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:12:59.221 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:59.224+0000 7f57d5600700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f57d01059a0 0x7f57d01162c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:12:59.221 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:59.224+0000 7f57d5600700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57d0116910 con 0x7f57d01059a0 2024-04-12T21:12:59.221 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:59.224+0000 7f57d5600700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57d010fce0 con 0x7f57d0105000 2024-04-12T21:12:59.221 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:59.224+0000 7f57ce000700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f57d01059a0 0x7f57d01162c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:12:59.221 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:59.225+0000 7f57ce000700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f57d01059a0 0x7f57d01162c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:58066/0 (socket says 172.21.15.18:58066) 2024-04-12T21:12:59.221 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:59.225+0000 7f57ce000700 1 -- 172.21.15.18:0/1134935898 learned_addr learned my addr 172.21.15.18:0/1134935898 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:12:59.222 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:59.225+0000 7f57ce000700 1 -- 172.21.15.18:0/1134935898 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f57d0105000 msgr2=0x7f57d0115d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:12:59.222 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:59.225+0000 7f57ce000700 1 --2- 172.21.15.18:0/1134935898 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f57d0105000 0x7f57d0115d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:12:59.222 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:59.225+0000 7f57ce000700 1 -- 172.21.15.18:0/1134935898 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57bc005040 con 0x7f57d01059a0 2024-04-12T21:12:59.222 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:59.225+0000 7f57ce000700 1 --2- 172.21.15.18:0/1134935898 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f57d01059a0 0x7f57d01162c0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f57bc009100 tx=0x7f57bc009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:12:59.222 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:59.225+0000 7f57c6a00700 1 -- 172.21.15.18:0/1134935898 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f57bc00e040 con 0x7f57d01059a0 2024-04-12T21:12:59.222 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:59.226+0000 7f57c6a00700 1 -- 172.21.15.18:0/1134935898 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f57bc013420 con 0x7f57d01059a0 2024-04-12T21:12:59.222 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:59.226+0000 7f57c6a00700 1 -- 172.21.15.18:0/1134935898 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f57bc01c610 con 0x7f57d01059a0 2024-04-12T21:12:59.222 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:59.226+0000 7f57d5600700 1 -- 172.21.15.18:0/1134935898 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f57d010ff60 con 0x7f57d01059a0 2024-04-12T21:12:59.222 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:59.226+0000 7f57d5600700 1 -- 172.21.15.18:0/1134935898 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f57d0110520 con 0x7f57d01059a0 2024-04-12T21:12:59.223 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:59.227+0000 7f57d5600700 1 -- 172.21.15.18:0/1134935898 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f57d0100750 con 0x7f57d01059a0 2024-04-12T21:12:59.224 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:59.228+0000 7f57c6a00700 1 -- 172.21.15.18:0/1134935898 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7f57bc013590 con 0x7f57d01059a0 2024-04-12T21:12:59.226 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:59.229+0000 7f57c6a00700 1 --2- 172.21.15.18:0/1134935898 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f57b406ec00 0x7f57b40710b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:12:59.226 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:59.229+0000 7f57c6a00700 1 -- 172.21.15.18:0/1134935898 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 6757+0+0 (secure 0 0 0) 0x7f57bc08fbb0 con 0x7f57d01059a0 2024-04-12T21:12:59.226 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:59.230+0000 7f57cea00700 1 --2- 172.21.15.18:0/1134935898 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f57b406ec00 0x7f57b40710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:12:59.226 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:59.230+0000 7f57cea00700 1 --2- 172.21.15.18:0/1134935898 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f57b406ec00 0x7f57b40710b0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f57c0004a50 tx=0x7f57c0007cb0 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:12:59.230 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:59.234+0000 7f57c6a00700 1 -- 172.21.15.18:0/1134935898 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f57bc05c3e0 con 0x7f57d01059a0 2024-04-12T21:12:59.469 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:59.472+0000 7f57d5600700 1 -- 172.21.15.18:0/1134935898 --> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f57d010c150 con 0x7f57b406ec00 2024-04-12T21:12:59.472 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:59.475+0000 7f57c6a00700 1 -- 172.21.15.18:0/1134935898 <== mgr.34102 v2:172.21.15.18:6800/3812891919 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f57d010c150 con 0x7f57b406ec00 2024-04-12T21:12:59.472 INFO:teuthology.orchestra.run.smithi018.stdout:{ 2024-04-12T21:12:59.472 INFO:teuthology.orchestra.run.smithi018.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:f4452def3be4d350f4a8e5feaf7ebb1511be9298fe68478ba7a2d6fbd7d69a63", 2024-04-12T21:12:59.472 INFO:teuthology.orchestra.run.smithi018.stdout: "in_progress": true, 2024-04-12T21:12:59.472 INFO:teuthology.orchestra.run.smithi018.stdout: "which": "Upgrading all daemon types on all hosts", 2024-04-12T21:12:59.472 INFO:teuthology.orchestra.run.smithi018.stdout: "services_complete": [ 2024-04-12T21:12:59.472 INFO:teuthology.orchestra.run.smithi018.stdout: "crash", 2024-04-12T21:12:59.472 INFO:teuthology.orchestra.run.smithi018.stdout: "osd", 2024-04-12T21:12:59.472 INFO:teuthology.orchestra.run.smithi018.stdout: "mon", 2024-04-12T21:12:59.472 INFO:teuthology.orchestra.run.smithi018.stdout: "mgr" 2024-04-12T21:12:59.473 INFO:teuthology.orchestra.run.smithi018.stdout: ], 2024-04-12T21:12:59.473 INFO:teuthology.orchestra.run.smithi018.stdout: "progress": "14/21 daemons upgraded", 2024-04-12T21:12:59.473 INFO:teuthology.orchestra.run.smithi018.stdout: "message": "Currently upgrading mds daemons", 2024-04-12T21:12:59.473 INFO:teuthology.orchestra.run.smithi018.stdout: "is_paused": false 2024-04-12T21:12:59.473 INFO:teuthology.orchestra.run.smithi018.stdout:} 2024-04-12T21:12:59.475 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:59.478+0000 7f57d5600700 1 -- 172.21.15.18:0/1134935898 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f57b406ec00 msgr2=0x7f57b40710b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:12:59.475 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:59.478+0000 7f57d5600700 1 --2- 172.21.15.18:0/1134935898 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f57b406ec00 0x7f57b40710b0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f57c0004a50 tx=0x7f57c0007cb0 comp rx=0 tx=0).stop 2024-04-12T21:12:59.475 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:59.478+0000 7f57d5600700 1 -- 172.21.15.18:0/1134935898 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f57d01059a0 msgr2=0x7f57d01162c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:12:59.475 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:59.478+0000 7f57d5600700 1 --2- 172.21.15.18:0/1134935898 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f57d01059a0 0x7f57d01162c0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f57bc009100 tx=0x7f57bc009e30 comp rx=0 tx=0).stop 2024-04-12T21:12:59.475 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:59.478+0000 7f57d5600700 1 -- 172.21.15.18:0/1134935898 shutdown_connections 2024-04-12T21:12:59.475 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:59.478+0000 7f57d5600700 1 --2- 172.21.15.18:0/1134935898 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f57b406ec00 0x7f57b40710b0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:12:59.475 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:59.478+0000 7f57d5600700 1 --2- 172.21.15.18:0/1134935898 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f57d0105000 0x7f57d0115d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:12:59.475 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:59.478+0000 7f57d5600700 1 --2- 172.21.15.18:0/1134935898 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f57d01059a0 0x7f57d01162c0 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:12:59.475 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:59.478+0000 7f57d5600700 1 -- 172.21.15.18:0/1134935898 >> 172.21.15.18:0/1134935898 conn(0x7f57d00fabd0 msgr2=0x7f57d00fbcc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:12:59.475 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:59.478+0000 7f57d5600700 1 -- 172.21.15.18:0/1134935898 shutdown_connections 2024-04-12T21:12:59.475 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:59.479+0000 7f57d5600700 1 -- 172.21.15.18:0/1134935898 wait complete. 2024-04-12T21:12:59.629 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:59.632+0000 7f1cf2200700 1 -- 172.21.15.18:0/2052505449 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1cec102de0 msgr2=0x7f1cec1031b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:12:59.629 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:59.632+0000 7f1cf2200700 1 --2- 172.21.15.18:0/2052505449 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1cec102de0 0x7f1cec1031b0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f1cdc004940 tx=0x7f1cdc009e30 comp rx=0 tx=0).stop 2024-04-12T21:12:59.629 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:59.632+0000 7f1cf2200700 1 -- 172.21.15.18:0/2052505449 shutdown_connections 2024-04-12T21:12:59.629 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:59.632+0000 7f1cf2200700 1 --2- 172.21.15.18:0/2052505449 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1cec0656a0 0x7f1cec065af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:12:59.629 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:59.632+0000 7f1cf2200700 1 --2- 172.21.15.18:0/2052505449 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1cec102de0 0x7f1cec1031b0 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:12:59.629 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:59.632+0000 7f1cf2200700 1 -- 172.21.15.18:0/2052505449 >> 172.21.15.18:0/2052505449 conn(0x7f1cec072bc0 msgr2=0x7f1cec072fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:12:59.629 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:59.632+0000 7f1cf2200700 1 -- 172.21.15.18:0/2052505449 shutdown_connections 2024-04-12T21:12:59.629 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:59.632+0000 7f1cf2200700 1 -- 172.21.15.18:0/2052505449 wait complete. 2024-04-12T21:12:59.631 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:59.633+0000 7f1cf2200700 1 Processor -- start 2024-04-12T21:12:59.631 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:59.633+0000 7f1cf2200700 1 -- start start 2024-04-12T21:12:59.631 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:59.634+0000 7f1cf2200700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1cec0656a0 0x7f1cec113b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:12:59.631 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:59.634+0000 7f1cf2200700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1cec102de0 0x7f1cec114060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:12:59.631 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:59.634+0000 7f1cf2200700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1cec1146b0 con 0x7f1cec0656a0 2024-04-12T21:12:59.631 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:59.634+0000 7f1cf2200700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1cec10daa0 con 0x7f1cec102de0 2024-04-12T21:12:59.631 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:59.634+0000 7f1ceb400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1cec0656a0 0x7f1cec113b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:12:59.631 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:59.634+0000 7f1ceb400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1cec0656a0 0x7f1cec113b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:58086/0 (socket says 172.21.15.18:58086) 2024-04-12T21:12:59.631 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:59.634+0000 7f1ceb400700 1 -- 172.21.15.18:0/1117189347 learned_addr learned my addr 172.21.15.18:0/1117189347 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:12:59.631 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:59.634+0000 7f1ceb400700 1 -- 172.21.15.18:0/1117189347 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1cec102de0 msgr2=0x7f1cec114060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:12:59.632 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:59.634+0000 7f1ceb400700 1 --2- 172.21.15.18:0/1117189347 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1cec102de0 0x7f1cec114060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:12:59.632 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:59.634+0000 7f1ceb400700 1 -- 172.21.15.18:0/1117189347 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1cdc005040 con 0x7f1cec0656a0 2024-04-12T21:12:59.632 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:59.635+0000 7f1ceb400700 1 --2- 172.21.15.18:0/1117189347 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1cec0656a0 0x7f1cec113b20 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f1cdc009150 tx=0x7f1cdc009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:12:59.633 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:59.635+0000 7f1ce3400700 1 -- 172.21.15.18:0/1117189347 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1cdc00e040 con 0x7f1cec0656a0 2024-04-12T21:12:59.633 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:59.635+0000 7f1ce3400700 1 -- 172.21.15.18:0/1117189347 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f1cdc013420 con 0x7f1cec0656a0 2024-04-12T21:12:59.633 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:59.635+0000 7f1cf2200700 1 -- 172.21.15.18:0/1117189347 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1cec10dd20 con 0x7f1cec0656a0 2024-04-12T21:12:59.633 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:59.635+0000 7f1ce3400700 1 -- 172.21.15.18:0/1117189347 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1cdc01c630 con 0x7f1cec0656a0 2024-04-12T21:12:59.633 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:59.635+0000 7f1cf2200700 1 -- 172.21.15.18:0/1117189347 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1cec10e2e0 con 0x7f1cec0656a0 2024-04-12T21:12:59.633 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:59.636+0000 7f1cf2200700 1 -- 172.21.15.18:0/1117189347 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1cec0fe4c0 con 0x7f1cec0656a0 2024-04-12T21:12:59.634 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:59.637+0000 7f1ce3400700 1 -- 172.21.15.18:0/1117189347 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7f1cdc013590 con 0x7f1cec0656a0 2024-04-12T21:12:59.634 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:59.638+0000 7f1ce3400700 1 --2- 172.21.15.18:0/1117189347 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f1cd006ec10 0x7f1cd00710c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:12:59.635 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:59.638+0000 7f1ce3400700 1 -- 172.21.15.18:0/1117189347 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 6757+0+0 (secure 0 0 0) 0x7f1cdc08fce0 con 0x7f1cec0656a0 2024-04-12T21:12:59.635 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:59.639+0000 7f1ceaa00700 1 --2- 172.21.15.18:0/1117189347 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f1cd006ec10 0x7f1cd00710c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:12:59.635 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:59.639+0000 7f1ceaa00700 1 --2- 172.21.15.18:0/1117189347 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f1cd006ec10 0x7f1cd00710c0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f1cec10ec60 tx=0x7f1cd8009ce0 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:12:59.639 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:59.643+0000 7f1ce3400700 1 -- 172.21.15.18:0/1117189347 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f1cdc05c490 con 0x7f1cec0656a0 2024-04-12T21:12:59.779 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:12:59 smithi186 ceph-mon[86379]: from='client.34728 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:12:59.779 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:12:59 smithi186 ceph-mon[86379]: pgmap v395: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:12:59.779 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:12:59 smithi186 ceph-mon[86379]: from='client.34732 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:12:59.779 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:12:59 smithi186 ceph-mon[86379]: from='client.? 172.21.15.18:0/2408826490' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T21:12:59.779 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:12:59 smithi186 ceph-mon[86379]: from='client.? 172.21.15.18:0/1421668859' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-12T21:12:59.850 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:12:59 smithi018 ceph-mon[112590]: from='client.34728 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:12:59.850 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:12:59 smithi018 ceph-mon[112590]: pgmap v395: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:12:59.850 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:12:59 smithi018 ceph-mon[112590]: from='client.34732 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:12:59.850 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:12:59 smithi018 ceph-mon[112590]: from='client.? 172.21.15.18:0/2408826490' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T21:12:59.850 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:12:59 smithi018 ceph-mon[112590]: from='client.? 172.21.15.18:0/1421668859' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-12T21:12:59.955 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:59.958+0000 7f1cf2200700 1 -- 172.21.15.18:0/1117189347 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f1cec109e90 con 0x7f1cec0656a0 2024-04-12T21:12:59.955 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:59.959+0000 7f1ce3400700 1 -- 172.21.15.18:0/1117189347 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+233 (secure 0 0 0) 0x7f1cdc05c020 con 0x7f1cec0656a0 2024-04-12T21:12:59.956 INFO:teuthology.orchestra.run.smithi018.stdout:HEALTH_ERR 1 filesystem is degraded; 1 filesystem is offline 2024-04-12T21:12:59.956 INFO:teuthology.orchestra.run.smithi018.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-04-12T21:12:59.956 INFO:teuthology.orchestra.run.smithi018.stdout: fs cephfs is degraded 2024-04-12T21:12:59.956 INFO:teuthology.orchestra.run.smithi018.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-04-12T21:12:59.956 INFO:teuthology.orchestra.run.smithi018.stdout: fs cephfs is offline because no MDS is active for it. 2024-04-12T21:12:59.959 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:59.962+0000 7f1cf2200700 1 -- 172.21.15.18:0/1117189347 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f1cd006ec10 msgr2=0x7f1cd00710c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:12:59.959 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:59.962+0000 7f1cf2200700 1 --2- 172.21.15.18:0/1117189347 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f1cd006ec10 0x7f1cd00710c0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f1cec10ec60 tx=0x7f1cd8009ce0 comp rx=0 tx=0).stop 2024-04-12T21:12:59.959 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:59.962+0000 7f1cf2200700 1 -- 172.21.15.18:0/1117189347 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1cec0656a0 msgr2=0x7f1cec113b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:12:59.959 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:59.962+0000 7f1cf2200700 1 --2- 172.21.15.18:0/1117189347 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1cec0656a0 0x7f1cec113b20 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f1cdc009150 tx=0x7f1cdc009e30 comp rx=0 tx=0).stop 2024-04-12T21:12:59.959 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:59.962+0000 7f1cf2200700 1 -- 172.21.15.18:0/1117189347 shutdown_connections 2024-04-12T21:12:59.959 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:59.962+0000 7f1cf2200700 1 --2- 172.21.15.18:0/1117189347 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f1cd006ec10 0x7f1cd00710c0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:12:59.959 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:59.962+0000 7f1cf2200700 1 --2- 172.21.15.18:0/1117189347 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1cec0656a0 0x7f1cec113b20 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:12:59.959 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:59.962+0000 7f1cf2200700 1 --2- 172.21.15.18:0/1117189347 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1cec102de0 0x7f1cec114060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:12:59.959 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:59.962+0000 7f1cf2200700 1 -- 172.21.15.18:0/1117189347 >> 172.21.15.18:0/1117189347 conn(0x7f1cec072bc0 msgr2=0x7f1cec105b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:12:59.959 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:59.962+0000 7f1cf2200700 1 -- 172.21.15.18:0/1117189347 shutdown_connections 2024-04-12T21:12:59.959 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:12:59.962+0000 7f1cf2200700 1 -- 172.21.15.18:0/1117189347 wait complete. 2024-04-12T21:13:00.779 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:13:00 smithi186 ceph-mon[86379]: from='client.34744 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:13:00.779 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:13:00 smithi186 ceph-mon[86379]: from='client.? 172.21.15.18:0/1117189347' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-12T21:13:00.850 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:13:00 smithi018 ceph-mon[112590]: from='client.34744 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:13:00.850 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:13:00 smithi018 ceph-mon[112590]: from='client.? 172.21.15.18:0/1117189347' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-12T21:13:01.779 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:13:01 smithi186 ceph-mon[86379]: pgmap v396: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:13:01.849 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:13:01 smithi018 ceph-mon[112590]: pgmap v396: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:13:03.779 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:13:03 smithi186 ceph-mon[86379]: pgmap v397: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:13:03.849 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:13:03 smithi018 ceph-mon[112590]: pgmap v397: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:13:05.779 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:13:05 smithi186 ceph-mon[86379]: pgmap v398: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:13:05.849 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:13:05 smithi018 ceph-mon[112590]: pgmap v398: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:13:07.779 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:13:07 smithi186 ceph-mon[86379]: pgmap v399: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:13:07.850 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:13:07 smithi018 ceph-mon[112590]: pgmap v399: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:13:09.779 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:13:09 smithi186 ceph-mon[86379]: pgmap v400: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:13:09.849 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:13:09 smithi018 ceph-mon[112590]: pgmap v400: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:13:10.751 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:13:10 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T21:13:10.778 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:13:10 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T21:13:11.778 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:13:11 smithi186 ceph-mon[86379]: pgmap v401: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:13:11.849 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:13:11 smithi018 ceph-mon[112590]: pgmap v401: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:13:13.779 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:13:13 smithi186 ceph-mon[86379]: pgmap v402: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:13:13.850 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:13:13 smithi018 ceph-mon[112590]: pgmap v402: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:13:15.779 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:13:15 smithi186 ceph-mon[86379]: pgmap v403: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:13:15.850 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:13:15 smithi018 ceph-mon[112590]: pgmap v403: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:13:17.778 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:13:17 smithi186 ceph-mon[86379]: pgmap v404: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:13:17.849 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:13:17 smithi018 ceph-mon[112590]: pgmap v404: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:13:19.778 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:13:19 smithi186 ceph-mon[86379]: pgmap v405: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:13:19.850 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:13:19 smithi018 ceph-mon[112590]: pgmap v405: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:13:21.778 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:13:21 smithi186 ceph-mon[86379]: pgmap v406: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:13:21.850 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:13:21 smithi018 ceph-mon[112590]: pgmap v406: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:13:23.850 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:13:23 smithi018 ceph-mon[112590]: pgmap v407: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:13:24.028 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:13:23 smithi186 ceph-mon[86379]: pgmap v407: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:13:25.850 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:13:25 smithi018 ceph-mon[112590]: pgmap v408: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:13:25.850 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:13:25 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T21:13:26.029 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:13:25 smithi186 ceph-mon[86379]: pgmap v408: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:13:26.029 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:13:25 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T21:13:27.850 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:13:27 smithi018 ceph-mon[112590]: pgmap v409: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:13:28.028 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:13:27 smithi186 ceph-mon[86379]: pgmap v409: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:13:29.850 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:13:29 smithi018 ceph-mon[112590]: pgmap v410: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:13:30.029 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:13:29 smithi186 ceph-mon[86379]: pgmap v410: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:13:30.134 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:13:30.138+0000 7efc0c600700 1 -- 172.21.15.18:0/2136155305 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7efc04109430 msgr2=0x7efc04109800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:13:30.134 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:13:30.138+0000 7efc0c600700 1 --2- 172.21.15.18:0/2136155305 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7efc04109430 0x7efc04109800 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7efbf8004940 tx=0x7efbf8009e30 comp rx=0 tx=0).stop 2024-04-12T21:13:30.134 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:13:30.138+0000 7efc0c600700 1 -- 172.21.15.18:0/2136155305 shutdown_connections 2024-04-12T21:13:30.134 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:13:30.138+0000 7efc0c600700 1 --2- 172.21.15.18:0/2136155305 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7efc04109dd0 0x7efc04103440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:13:30.134 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:13:30.138+0000 7efc0c600700 1 --2- 172.21.15.18:0/2136155305 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7efc04109430 0x7efc04109800 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:13:30.134 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:13:30.138+0000 7efc0c600700 1 -- 172.21.15.18:0/2136155305 >> 172.21.15.18:0/2136155305 conn(0x7efc04072c00 msgr2=0x7efc04075010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:13:30.134 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:13:30.138+0000 7efc0c600700 1 -- 172.21.15.18:0/2136155305 shutdown_connections 2024-04-12T21:13:30.134 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:13:30.138+0000 7efc0c600700 1 -- 172.21.15.18:0/2136155305 wait complete. 2024-04-12T21:13:30.135 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:13:30.140+0000 7efc0c600700 1 Processor -- start 2024-04-12T21:13:30.135 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:13:30.140+0000 7efc0c600700 1 -- start start 2024-04-12T21:13:30.136 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:13:30.140+0000 7efc0c600700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7efc04109430 0x7efc04119f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:13:30.136 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:13:30.140+0000 7efc0c600700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7efc04109dd0 0x7efc0411a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:13:30.136 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:13:30.140+0000 7efc0c600700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc0411aad0 con 0x7efc04109dd0 2024-04-12T21:13:30.136 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:13:30.140+0000 7efc0c600700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc04114100 con 0x7efc04109430 2024-04-12T21:13:30.137 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:13:30.141+0000 7efc09200700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7efc04109dd0 0x7efc0411a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:13:30.137 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:13:30.141+0000 7efc09200700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7efc04109dd0 0x7efc0411a480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:38430/0 (socket says 172.21.15.18:38430) 2024-04-12T21:13:30.137 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:13:30.141+0000 7efc09200700 1 -- 172.21.15.18:0/341610197 learned_addr learned my addr 172.21.15.18:0/341610197 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:13:30.137 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:13:30.141+0000 7efc09c00700 1 --2- 172.21.15.18:0/341610197 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7efc04109430 0x7efc04119f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:13:30.137 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:13:30.141+0000 7efc09200700 1 -- 172.21.15.18:0/341610197 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7efc04109430 msgr2=0x7efc04119f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:13:30.137 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:13:30.141+0000 7efc09200700 1 --2- 172.21.15.18:0/341610197 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7efc04109430 0x7efc04119f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:13:30.137 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:13:30.141+0000 7efc09200700 1 -- 172.21.15.18:0/341610197 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efbf8005040 con 0x7efc04109dd0 2024-04-12T21:13:30.138 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:13:30.142+0000 7efc09200700 1 --2- 172.21.15.18:0/341610197 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7efc04109dd0 0x7efc0411a480 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7efbf0002a50 tx=0x7efbf0004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:13:30.138 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:13:30.142+0000 7efc01600700 1 -- 172.21.15.18:0/341610197 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7efbf0007cc0 con 0x7efc04109dd0 2024-04-12T21:13:30.138 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:13:30.142+0000 7efc01600700 1 -- 172.21.15.18:0/341610197 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7efbf000f040 con 0x7efc04109dd0 2024-04-12T21:13:30.138 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:13:30.142+0000 7efc01600700 1 -- 172.21.15.18:0/341610197 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7efbf000daf0 con 0x7efc04109dd0 2024-04-12T21:13:30.138 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:13:30.142+0000 7efc0c600700 1 -- 172.21.15.18:0/341610197 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efc041143e0 con 0x7efc04109dd0 2024-04-12T21:13:30.138 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:13:30.142+0000 7efc0c600700 1 -- 172.21.15.18:0/341610197 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efc04114950 con 0x7efc04109dd0 2024-04-12T21:13:30.139 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:13:30.143+0000 7efc0c600700 1 -- 172.21.15.18:0/341610197 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efc04104b80 con 0x7efc04109dd0 2024-04-12T21:13:30.140 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:13:30.145+0000 7efc01600700 1 -- 172.21.15.18:0/341610197 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7efbf0009810 con 0x7efc04109dd0 2024-04-12T21:13:30.141 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:13:30.146+0000 7efc01600700 1 --2- 172.21.15.18:0/341610197 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7efbec06ec00 0x7efbec0710b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:13:30.141 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:13:30.146+0000 7efc01600700 1 -- 172.21.15.18:0/341610197 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 6757+0+0 (secure 0 0 0) 0x7efbf008e7d0 con 0x7efc04109dd0 2024-04-12T21:13:30.141 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:13:30.146+0000 7efc09c00700 1 --2- 172.21.15.18:0/341610197 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7efbec06ec00 0x7efbec0710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:13:30.142 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:13:30.147+0000 7efc09c00700 1 --2- 172.21.15.18:0/341610197 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7efbec06ec00 0x7efbec0710b0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7efbf8004f90 tx=0x7efbf8009e30 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:13:30.146 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:13:30.151+0000 7efc01600700 1 -- 172.21.15.18:0/341610197 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7efbf005af00 con 0x7efc04109dd0 2024-04-12T21:13:30.401 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:13:30.406+0000 7efc0c600700 1 -- 172.21.15.18:0/341610197 --> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7efc04110590 con 0x7efbec06ec00 2024-04-12T21:13:30.404 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:13:30.409+0000 7efc01600700 1 -- 172.21.15.18:0/341610197 <== mgr.34102 v2:172.21.15.18:6800/3812891919 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7efc04110590 con 0x7efbec06ec00 2024-04-12T21:13:30.408 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:13:30.412+0000 7efc0c600700 1 -- 172.21.15.18:0/341610197 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7efbec06ec00 msgr2=0x7efbec0710b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:13:30.408 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:13:30.412+0000 7efc0c600700 1 --2- 172.21.15.18:0/341610197 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7efbec06ec00 0x7efbec0710b0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7efbf8004f90 tx=0x7efbf8009e30 comp rx=0 tx=0).stop 2024-04-12T21:13:30.408 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:13:30.412+0000 7efc0c600700 1 -- 172.21.15.18:0/341610197 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7efc04109dd0 msgr2=0x7efc0411a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:13:30.408 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:13:30.412+0000 7efc0c600700 1 --2- 172.21.15.18:0/341610197 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7efc04109dd0 0x7efc0411a480 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7efbf0002a50 tx=0x7efbf0004600 comp rx=0 tx=0).stop 2024-04-12T21:13:30.408 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:13:30.412+0000 7efc0c600700 1 -- 172.21.15.18:0/341610197 shutdown_connections 2024-04-12T21:13:30.408 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:13:30.412+0000 7efc0c600700 1 --2- 172.21.15.18:0/341610197 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7efbec06ec00 0x7efbec0710b0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:13:30.408 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:13:30.412+0000 7efc0c600700 1 --2- 172.21.15.18:0/341610197 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7efc04109430 0x7efc04119f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:13:30.408 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:13:30.412+0000 7efc0c600700 1 --2- 172.21.15.18:0/341610197 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7efc04109dd0 0x7efc0411a480 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:13:30.408 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:13:30.412+0000 7efc0c600700 1 -- 172.21.15.18:0/341610197 >> 172.21.15.18:0/341610197 conn(0x7efc04072c00 msgr2=0x7efc04074c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:13:30.408 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:13:30.412+0000 7efc0c600700 1 -- 172.21.15.18:0/341610197 shutdown_connections 2024-04-12T21:13:30.409 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:13:30.412+0000 7efc0c600700 1 -- 172.21.15.18:0/341610197 wait complete. 2024-04-12T21:13:30.424 INFO:teuthology.orchestra.run.smithi018.stdout:true 2024-04-12T21:13:30.570 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:13:30.574+0000 7fc345e00700 1 -- 172.21.15.18:0/1542560239 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc3401059c0 msgr2=0x7fc3400ff080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:13:30.570 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:13:30.574+0000 7fc345e00700 1 --2- 172.21.15.18:0/1542560239 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc3401059c0 0x7fc3400ff080 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fc32c004940 tx=0x7fc32c009e30 comp rx=0 tx=0).stop 2024-04-12T21:13:30.570 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:13:30.575+0000 7fc345e00700 1 -- 172.21.15.18:0/1542560239 shutdown_connections 2024-04-12T21:13:30.570 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:13:30.575+0000 7fc345e00700 1 --2- 172.21.15.18:0/1542560239 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc3401059c0 0x7fc3400ff080 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:13:30.571 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:13:30.575+0000 7fc345e00700 1 --2- 172.21.15.18:0/1542560239 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc340105020 0x7fc3401053f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:13:30.571 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:13:30.575+0000 7fc345e00700 1 -- 172.21.15.18:0/1542560239 >> 172.21.15.18:0/1542560239 conn(0x7fc3400faba0 msgr2=0x7fc3400fcfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:13:30.571 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:13:30.575+0000 7fc345e00700 1 -- 172.21.15.18:0/1542560239 shutdown_connections 2024-04-12T21:13:30.571 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:13:30.575+0000 7fc345e00700 1 -- 172.21.15.18:0/1542560239 wait complete. 2024-04-12T21:13:30.572 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:13:30.576+0000 7fc345e00700 1 Processor -- start 2024-04-12T21:13:30.572 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:13:30.576+0000 7fc345e00700 1 -- start start 2024-04-12T21:13:30.572 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:13:30.576+0000 7fc345e00700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc340105020 0x7fc340115ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:13:30.572 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:13:30.576+0000 7fc345e00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc3401059c0 0x7fc3401160e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:13:30.572 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:13:30.576+0000 7fc345e00700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc340116730 con 0x7fc3401059c0 2024-04-12T21:13:30.573 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:13:30.576+0000 7fc345e00700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc34010fd10 con 0x7fc340105020 2024-04-12T21:13:30.573 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:13:30.577+0000 7fc33e000700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc3401059c0 0x7fc3401160e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:13:30.573 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:13:30.577+0000 7fc33e000700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc3401059c0 0x7fc3401160e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:38448/0 (socket says 172.21.15.18:38448) 2024-04-12T21:13:30.573 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:13:30.577+0000 7fc33e000700 1 -- 172.21.15.18:0/1659696415 learned_addr learned my addr 172.21.15.18:0/1659696415 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:13:30.573 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:13:30.577+0000 7fc33ea00700 1 --2- 172.21.15.18:0/1659696415 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc340105020 0x7fc340115ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:13:30.573 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:13:30.577+0000 7fc33e000700 1 -- 172.21.15.18:0/1659696415 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc340105020 msgr2=0x7fc340115ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:13:30.573 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:13:30.577+0000 7fc33e000700 1 --2- 172.21.15.18:0/1659696415 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc340105020 0x7fc340115ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:13:30.573 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:13:30.577+0000 7fc33e000700 1 -- 172.21.15.18:0/1659696415 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc32c005040 con 0x7fc3401059c0 2024-04-12T21:13:30.574 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:13:30.577+0000 7fc33e000700 1 --2- 172.21.15.18:0/1659696415 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc3401059c0 0x7fc3401160e0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fc32c009100 tx=0x7fc32c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:13:30.574 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:13:30.578+0000 7fc336a00700 1 -- 172.21.15.18:0/1659696415 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc32c00e040 con 0x7fc3401059c0 2024-04-12T21:13:30.574 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:13:30.578+0000 7fc336a00700 1 -- 172.21.15.18:0/1659696415 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc32c013420 con 0x7fc3401059c0 2024-04-12T21:13:30.574 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:13:30.578+0000 7fc336a00700 1 -- 172.21.15.18:0/1659696415 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc32c01c630 con 0x7fc3401059c0 2024-04-12T21:13:30.574 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:13:30.578+0000 7fc345e00700 1 -- 172.21.15.18:0/1659696415 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc34010ff90 con 0x7fc3401059c0 2024-04-12T21:13:30.574 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:13:30.578+0000 7fc345e00700 1 -- 172.21.15.18:0/1659696415 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc340110550 con 0x7fc3401059c0 2024-04-12T21:13:30.575 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:13:30.580+0000 7fc345e00700 1 -- 172.21.15.18:0/1659696415 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc340100770 con 0x7fc3401059c0 2024-04-12T21:13:30.575 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:13:30.580+0000 7fc336a00700 1 -- 172.21.15.18:0/1659696415 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7fc32c013590 con 0x7fc3401059c0 2024-04-12T21:13:30.577 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:13:30.581+0000 7fc336a00700 1 --2- 172.21.15.18:0/1659696415 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fc32406ec00 0x7fc3240710b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:13:30.577 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:13:30.582+0000 7fc336a00700 1 -- 172.21.15.18:0/1659696415 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 6757+0+0 (secure 0 0 0) 0x7fc32c08fae0 con 0x7fc3401059c0 2024-04-12T21:13:30.577 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:13:30.582+0000 7fc33ea00700 1 --2- 172.21.15.18:0/1659696415 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fc32406ec00 0x7fc3240710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:13:30.578 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:13:30.583+0000 7fc33ea00700 1 --2- 172.21.15.18:0/1659696415 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fc32406ec00 0x7fc3240710b0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fc330004990 tx=0x7fc330009ee0 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:13:30.582 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:13:30.587+0000 7fc336a00700 1 -- 172.21.15.18:0/1659696415 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7fc32c05c3d0 con 0x7fc3401059c0 2024-04-12T21:13:30.835 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:13:30.840+0000 7fc345e00700 1 -- 172.21.15.18:0/1659696415 --> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fc34010c170 con 0x7fc32406ec00 2024-04-12T21:13:30.837 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:13:30.842+0000 7fc336a00700 1 -- 172.21.15.18:0/1659696415 <== mgr.34102 v2:172.21.15.18:6800/3812891919 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7fc34010c170 con 0x7fc32406ec00 2024-04-12T21:13:30.842 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:13:30.846+0000 7fc345e00700 1 -- 172.21.15.18:0/1659696415 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fc32406ec00 msgr2=0x7fc3240710b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:13:30.842 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:13:30.846+0000 7fc345e00700 1 --2- 172.21.15.18:0/1659696415 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fc32406ec00 0x7fc3240710b0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fc330004990 tx=0x7fc330009ee0 comp rx=0 tx=0).stop 2024-04-12T21:13:30.842 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:13:30.846+0000 7fc345e00700 1 -- 172.21.15.18:0/1659696415 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc3401059c0 msgr2=0x7fc3401160e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:13:30.842 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:13:30.846+0000 7fc345e00700 1 --2- 172.21.15.18:0/1659696415 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc3401059c0 0x7fc3401160e0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fc32c009100 tx=0x7fc32c009e30 comp rx=0 tx=0).stop 2024-04-12T21:13:30.842 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:13:30.846+0000 7fc345e00700 1 -- 172.21.15.18:0/1659696415 shutdown_connections 2024-04-12T21:13:30.842 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:13:30.846+0000 7fc345e00700 1 --2- 172.21.15.18:0/1659696415 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fc32406ec00 0x7fc3240710b0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:13:30.842 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:13:30.846+0000 7fc345e00700 1 --2- 172.21.15.18:0/1659696415 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc340105020 0x7fc340115ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:13:30.843 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:13:30.846+0000 7fc345e00700 1 --2- 172.21.15.18:0/1659696415 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc3401059c0 0x7fc3401160e0 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:13:30.843 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:13:30.846+0000 7fc345e00700 1 -- 172.21.15.18:0/1659696415 >> 172.21.15.18:0/1659696415 conn(0x7fc3400faba0 msgr2=0x7fc3400fbb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:13:30.843 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:13:30.847+0000 7fc345e00700 1 -- 172.21.15.18:0/1659696415 shutdown_connections 2024-04-12T21:13:30.843 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:13:30.847+0000 7fc345e00700 1 -- 172.21.15.18:0/1659696415 wait complete. 2024-04-12T21:13:31.001 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:13:31.005+0000 7fcaf6200700 1 -- 172.21.15.18:0/3838672538 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fcaf0102e20 msgr2=0x7fcaf01031f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:13:31.001 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:13:31.005+0000 7fcaf6200700 1 --2- 172.21.15.18:0/3838672538 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fcaf0102e20 0x7fcaf01031f0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fcae0004940 tx=0x7fcae0009e30 comp rx=0 tx=0).stop 2024-04-12T21:13:31.001 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:13:31.006+0000 7fcaf6200700 1 -- 172.21.15.18:0/3838672538 shutdown_connections 2024-04-12T21:13:31.002 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:13:31.006+0000 7fcaf6200700 1 --2- 172.21.15.18:0/3838672538 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcaf01037c0 0x7fcaf00fce30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:13:31.002 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:13:31.006+0000 7fcaf6200700 1 --2- 172.21.15.18:0/3838672538 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fcaf0102e20 0x7fcaf01031f0 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:13:31.002 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:13:31.006+0000 7fcaf6200700 1 -- 172.21.15.18:0/3838672538 >> 172.21.15.18:0/3838672538 conn(0x7fcaf00f8970 msgr2=0x7fcaf00fad80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:13:31.002 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:13:31.006+0000 7fcaf6200700 1 -- 172.21.15.18:0/3838672538 shutdown_connections 2024-04-12T21:13:31.002 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:13:31.006+0000 7fcaf6200700 1 -- 172.21.15.18:0/3838672538 wait complete. 2024-04-12T21:13:31.003 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:13:31.007+0000 7fcaf6200700 1 Processor -- start 2024-04-12T21:13:31.003 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:13:31.007+0000 7fcaf6200700 1 -- start start 2024-04-12T21:13:31.003 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:13:31.007+0000 7fcaf6200700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fcaf0102e20 0x7fcaf0113b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:13:31.003 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:13:31.007+0000 7fcaf6200700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcaf01037c0 0x7fcaf01140d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:13:31.003 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:13:31.007+0000 7fcaf6200700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcaf0114720 con 0x7fcaf0102e20 2024-04-12T21:13:31.003 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:13:31.007+0000 7fcaf6200700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcaf010db10 con 0x7fcaf01037c0 2024-04-12T21:13:31.003 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:13:31.008+0000 7fcaef400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fcaf0102e20 0x7fcaf0113b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:13:31.003 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:13:31.008+0000 7fcaef400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fcaf0102e20 0x7fcaf0113b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:58376/0 (socket says 172.21.15.18:58376) 2024-04-12T21:13:31.003 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:13:31.008+0000 7fcaef400700 1 -- 172.21.15.18:0/1834583785 learned_addr learned my addr 172.21.15.18:0/1834583785 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:13:31.004 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:13:31.008+0000 7fcaeea00700 1 --2- 172.21.15.18:0/1834583785 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcaf01037c0 0x7fcaf01140d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:13:31.004 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:13:31.008+0000 7fcaef400700 1 -- 172.21.15.18:0/1834583785 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcaf01037c0 msgr2=0x7fcaf01140d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:13:31.004 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:13:31.008+0000 7fcaef400700 1 --2- 172.21.15.18:0/1834583785 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcaf01037c0 0x7fcaf01140d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:13:31.004 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:13:31.008+0000 7fcaef400700 1 -- 172.21.15.18:0/1834583785 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcae0005040 con 0x7fcaf0102e20 2024-04-12T21:13:31.004 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:13:31.009+0000 7fcaef400700 1 --2- 172.21.15.18:0/1834583785 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fcaf0102e20 0x7fcaf0113b90 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fcae0009150 tx=0x7fcae0009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:13:31.005 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:13:31.009+0000 7fcae7400700 1 -- 172.21.15.18:0/1834583785 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcae0021070 con 0x7fcaf0102e20 2024-04-12T21:13:31.005 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:13:31.009+0000 7fcae7400700 1 -- 172.21.15.18:0/1834583785 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fcae0013c00 con 0x7fcaf0102e20 2024-04-12T21:13:31.005 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:13:31.009+0000 7fcaf6200700 1 -- 172.21.15.18:0/1834583785 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcaf010dd90 con 0x7fcaf0102e20 2024-04-12T21:13:31.005 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:13:31.009+0000 7fcae7400700 1 -- 172.21.15.18:0/1834583785 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcae000e070 con 0x7fcaf0102e20 2024-04-12T21:13:31.005 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:13:31.009+0000 7fcaf6200700 1 -- 172.21.15.18:0/1834583785 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcaf010e270 con 0x7fcaf0102e20 2024-04-12T21:13:31.005 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:13:31.010+0000 7fcaf6200700 1 -- 172.21.15.18:0/1834583785 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcaf0100130 con 0x7fcaf0102e20 2024-04-12T21:13:31.006 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:13:31.011+0000 7fcae7400700 1 -- 172.21.15.18:0/1834583785 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7fcae001c400 con 0x7fcaf0102e20 2024-04-12T21:13:31.008 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:13:31.012+0000 7fcae7400700 1 --2- 172.21.15.18:0/1834583785 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fcad406ec10 0x7fcad40710c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:13:31.008 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:13:31.013+0000 7fcae7400700 1 -- 172.21.15.18:0/1834583785 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 6757+0+0 (secure 0 0 0) 0x7fcae008fe80 con 0x7fcaf0102e20 2024-04-12T21:13:31.008 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:13:31.013+0000 7fcaeea00700 1 --2- 172.21.15.18:0/1834583785 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fcad406ec10 0x7fcad40710c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:13:31.009 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:13:31.014+0000 7fcaeea00700 1 --2- 172.21.15.18:0/1834583785 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fcad406ec10 0x7fcad40710c0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fcadc004600 tx=0x7fcadc00b040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:13:31.013 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:13:31.018+0000 7fcae7400700 1 -- 172.21.15.18:0/1834583785 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7fcae005c6b0 con 0x7fcaf0102e20 2024-04-12T21:13:31.256 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:13:31.260+0000 7fcaf6200700 1 -- 172.21.15.18:0/1834583785 --> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fcaf010f1e0 con 0x7fcad406ec10 2024-04-12T21:13:31.278 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:13:31.282+0000 7fcae7400700 1 -- 172.21.15.18:0/1834583785 <== mgr.34102 v2:172.21.15.18:6800/3812891919 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7fcaf010f1e0 con 0x7fcad406ec10 2024-04-12T21:13:31.278 INFO:teuthology.orchestra.run.smithi018.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-12T21:13:31.278 INFO:teuthology.orchestra.run.smithi018.stdout:alertmanager.smithi018 smithi018 *:9093,9094 running (14m) 5m ago 28m 14.7M - 0.25.0 c8568f914cd2 c332090068e8 2024-04-12T21:13:31.278 INFO:teuthology.orchestra.run.smithi018.stdout:crash.smithi018 smithi018 running (10m) 5m ago 28m 6665k - 19.0.0-2879-g324baf8d 8717720dc5e5 b7f80d1f7982 2024-04-12T21:13:31.278 INFO:teuthology.orchestra.run.smithi018.stdout:crash.smithi186 smithi186 running (10m) 6m ago 27m 6660k - 19.0.0-2879-g324baf8d 8717720dc5e5 a4f55c81b623 2024-04-12T21:13:31.278 INFO:teuthology.orchestra.run.smithi018.stdout:grafana.smithi018 smithi018 *:3000 running (13m) 5m ago 28m 74.8M - 9.4.12 2bacad6d85d8 b08e3bf176dd 2024-04-12T21:13:31.278 INFO:teuthology.orchestra.run.smithi018.stdout:mds.cephfs.smithi018.acqbsc smithi018 running (6m) 5m ago 21m 11.1M - 19.0.0-2879-g324baf8d 8717720dc5e5 021f99ab24ae 2024-04-12T21:13:31.279 INFO:teuthology.orchestra.run.smithi018.stdout:mds.cephfs.smithi018.szjluo smithi018 running (5m) 5m ago 21m 10.9M - 19.0.0-2879-g324baf8d 8717720dc5e5 0003117479d4 2024-04-12T21:13:31.279 INFO:teuthology.orchestra.run.smithi018.stdout:mds.cephfs.smithi186.jdllyn smithi186 running (21m) 6m ago 21m 15.6M - 17.2.7-827-gd3b5060a 21de9b7369ad 2b2bc46bd29a 2024-04-12T21:13:31.279 INFO:teuthology.orchestra.run.smithi018.stdout:mds.cephfs.smithi186.wjnafs smithi186 running (21m) 6m ago 21m 15.4M - 17.2.7-827-gd3b5060a 21de9b7369ad 74b115d6d6fd 2024-04-12T21:13:31.279 INFO:teuthology.orchestra.run.smithi018.stdout:mgr.smithi018.xjzwga smithi018 *:8443,9283,8765 running (16m) 5m ago 30m 509M - 19.0.0-2879-g324baf8d 8717720dc5e5 56088305eab7 2024-04-12T21:13:31.279 INFO:teuthology.orchestra.run.smithi018.stdout:mgr.smithi186.kqcfxk smithi186 *:8443,9283,8765 running (15m) 6m ago 27m 452M - 19.0.0-2879-g324baf8d 8717720dc5e5 09cd766e5e0d 2024-04-12T21:13:31.279 INFO:teuthology.orchestra.run.smithi018.stdout:mon.smithi018 smithi018 running (12m) 5m ago 30m 57.0M 2048M 19.0.0-2879-g324baf8d 8717720dc5e5 df3e201dfed6 2024-04-12T21:13:31.279 INFO:teuthology.orchestra.run.smithi018.stdout:mon.smithi186 smithi186 running (11m) 6m ago 27m 37.7M 2048M 19.0.0-2879-g324baf8d 8717720dc5e5 c77498da9b2b 2024-04-12T21:13:31.279 INFO:teuthology.orchestra.run.smithi018.stdout:node-exporter.smithi018 smithi018 *:9100 running (15m) 5m ago 28m 7780k - 1.5.0 0da6a335fe13 326c3d9c6847 2024-04-12T21:13:31.279 INFO:teuthology.orchestra.run.smithi018.stdout:node-exporter.smithi186 smithi186 *:9100 running (15m) 6m ago 27m 7826k - 1.5.0 0da6a335fe13 ec2112baeecf 2024-04-12T21:13:31.279 INFO:teuthology.orchestra.run.smithi018.stdout:osd.0 smithi018 running (9m) 5m ago 26m 121M 1559M 19.0.0-2879-g324baf8d 8717720dc5e5 60b1273ca047 2024-04-12T21:13:31.279 INFO:teuthology.orchestra.run.smithi018.stdout:osd.1 smithi018 running (9m) 5m ago 25m 91.3M 1559M 19.0.0-2879-g324baf8d 8717720dc5e5 170ce1e5fb8d 2024-04-12T21:13:31.279 INFO:teuthology.orchestra.run.smithi018.stdout:osd.2 smithi018 running (8m) 5m ago 25m 90.3M 1559M 19.0.0-2879-g324baf8d 8717720dc5e5 c2cbc6aa2839 2024-04-12T21:13:31.279 INFO:teuthology.orchestra.run.smithi018.stdout:osd.3 smithi186 running (8m) 6m ago 24m 96.6M 2581M 19.0.0-2879-g324baf8d 8717720dc5e5 0138e8262e4b 2024-04-12T21:13:31.279 INFO:teuthology.orchestra.run.smithi018.stdout:osd.4 smithi186 running (7m) 6m ago 23m 87.2M 2581M 19.0.0-2879-g324baf8d 8717720dc5e5 b20cb63b25c9 2024-04-12T21:13:31.279 INFO:teuthology.orchestra.run.smithi018.stdout:osd.5 smithi186 running (6m) 6m ago 23m 11.1M 2581M 19.0.0-2879-g324baf8d 8717720dc5e5 2513bea2610e 2024-04-12T21:13:31.280 INFO:teuthology.orchestra.run.smithi018.stdout:prometheus.smithi018 smithi018 *:9095 running (14m) 5m ago 27m 60.6M - 2.43.0 a07b618ecd1d f58841127e21 2024-04-12T21:13:31.281 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:13:31.285+0000 7fcaf6200700 1 -- 172.21.15.18:0/1834583785 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fcad406ec10 msgr2=0x7fcad40710c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:13:31.281 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:13:31.285+0000 7fcaf6200700 1 --2- 172.21.15.18:0/1834583785 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fcad406ec10 0x7fcad40710c0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fcadc004600 tx=0x7fcadc00b040 comp rx=0 tx=0).stop 2024-04-12T21:13:31.281 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:13:31.285+0000 7fcaf6200700 1 -- 172.21.15.18:0/1834583785 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fcaf0102e20 msgr2=0x7fcaf0113b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:13:31.281 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:13:31.285+0000 7fcaf6200700 1 --2- 172.21.15.18:0/1834583785 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fcaf0102e20 0x7fcaf0113b90 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fcae0009150 tx=0x7fcae0009e30 comp rx=0 tx=0).stop 2024-04-12T21:13:31.281 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:13:31.286+0000 7fcaf6200700 1 -- 172.21.15.18:0/1834583785 shutdown_connections 2024-04-12T21:13:31.281 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:13:31.286+0000 7fcaf6200700 1 --2- 172.21.15.18:0/1834583785 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fcad406ec10 0x7fcad40710c0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:13:31.282 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:13:31.286+0000 7fcaf6200700 1 --2- 172.21.15.18:0/1834583785 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fcaf0102e20 0x7fcaf0113b90 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:13:31.282 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:13:31.286+0000 7fcaf6200700 1 --2- 172.21.15.18:0/1834583785 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcaf01037c0 0x7fcaf01140d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:13:31.282 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:13:31.286+0000 7fcaf6200700 1 -- 172.21.15.18:0/1834583785 >> 172.21.15.18:0/1834583785 conn(0x7fcaf00f8970 msgr2=0x7fcaf0107410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:13:31.282 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:13:31.286+0000 7fcaf6200700 1 -- 172.21.15.18:0/1834583785 shutdown_connections 2024-04-12T21:13:31.282 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:13:31.286+0000 7fcaf6200700 1 -- 172.21.15.18:0/1834583785 wait complete. 2024-04-12T21:13:31.430 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:13:31.434+0000 7fb7db200700 1 -- 172.21.15.18:0/1198161999 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb7d4102e20 msgr2=0x7fb7d41031f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:13:31.430 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:13:31.434+0000 7fb7db200700 1 --2- 172.21.15.18:0/1198161999 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb7d4102e20 0x7fb7d41031f0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fb7c4004940 tx=0x7fb7c4009e30 comp rx=0 tx=0).stop 2024-04-12T21:13:31.430 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:13:31.434+0000 7fb7db200700 1 -- 172.21.15.18:0/1198161999 shutdown_connections 2024-04-12T21:13:31.430 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:13:31.434+0000 7fb7db200700 1 --2- 172.21.15.18:0/1198161999 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb7d41037c0 0x7fb7d40fce30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:13:31.430 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:13:31.434+0000 7fb7db200700 1 --2- 172.21.15.18:0/1198161999 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb7d4102e20 0x7fb7d41031f0 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:13:31.430 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:13:31.434+0000 7fb7db200700 1 -- 172.21.15.18:0/1198161999 >> 172.21.15.18:0/1198161999 conn(0x7fb7d40f89d0 msgr2=0x7fb7d40fade0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:13:31.430 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:13:31.434+0000 7fb7db200700 1 -- 172.21.15.18:0/1198161999 shutdown_connections 2024-04-12T21:13:31.430 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:13:31.434+0000 7fb7db200700 1 -- 172.21.15.18:0/1198161999 wait complete. 2024-04-12T21:13:31.431 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:13:31.435+0000 7fb7db200700 1 Processor -- start 2024-04-12T21:13:31.431 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:13:31.435+0000 7fb7db200700 1 -- start start 2024-04-12T21:13:31.431 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:13:31.436+0000 7fb7db200700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb7d4102e20 0x7fb7d40702c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:13:31.432 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:13:31.436+0000 7fb7db200700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb7d41037c0 0x7fb7d4070800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:13:31.432 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:13:31.436+0000 7fb7db200700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7d4070d90 con 0x7fb7d41037c0 2024-04-12T21:13:31.432 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:13:31.436+0000 7fb7db200700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7d4070ed0 con 0x7fb7d4102e20 2024-04-12T21:13:31.432 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:13:31.436+0000 7fb7d3400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb7d41037c0 0x7fb7d4070800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:13:31.432 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:13:31.436+0000 7fb7d3400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb7d41037c0 0x7fb7d4070800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:58398/0 (socket says 172.21.15.18:58398) 2024-04-12T21:13:31.432 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:13:31.436+0000 7fb7d3400700 1 -- 172.21.15.18:0/825851610 learned_addr learned my addr 172.21.15.18:0/825851610 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:13:31.433 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:13:31.437+0000 7fb7d3e00700 1 --2- 172.21.15.18:0/825851610 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb7d4102e20 0x7fb7d40702c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:13:31.433 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:13:31.437+0000 7fb7d3400700 1 -- 172.21.15.18:0/825851610 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb7d4102e20 msgr2=0x7fb7d40702c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:13:31.433 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:13:31.437+0000 7fb7d3400700 1 --2- 172.21.15.18:0/825851610 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb7d4102e20 0x7fb7d40702c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:13:31.433 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:13:31.437+0000 7fb7d3400700 1 -- 172.21.15.18:0/825851610 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7c4005040 con 0x7fb7d41037c0 2024-04-12T21:13:31.433 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:13:31.437+0000 7fb7d3400700 1 --2- 172.21.15.18:0/825851610 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb7d41037c0 0x7fb7d4070800 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fb7c000c890 tx=0x7fb7c00044c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:13:31.433 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:13:31.437+0000 7fb7cbe00700 1 -- 172.21.15.18:0/825851610 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb7c000a7d0 con 0x7fb7d41037c0 2024-04-12T21:13:31.433 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:13:31.437+0000 7fb7cbe00700 1 -- 172.21.15.18:0/825851610 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb7c000ae00 con 0x7fb7d41037c0 2024-04-12T21:13:31.433 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:13:31.437+0000 7fb7db200700 1 -- 172.21.15.18:0/825851610 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb7d4119f10 con 0x7fb7d41037c0 2024-04-12T21:13:31.433 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:13:31.438+0000 7fb7cbe00700 1 -- 172.21.15.18:0/825851610 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb7c00113a0 con 0x7fb7d41037c0 2024-04-12T21:13:31.434 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:13:31.438+0000 7fb7db200700 1 -- 172.21.15.18:0/825851610 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb7d411a310 con 0x7fb7d41037c0 2024-04-12T21:13:31.434 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:13:31.439+0000 7fb7db200700 1 -- 172.21.15.18:0/825851610 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb7d4100130 con 0x7fb7d41037c0 2024-04-12T21:13:31.435 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:13:31.440+0000 7fb7cbe00700 1 -- 172.21.15.18:0/825851610 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7fb7c0012400 con 0x7fb7d41037c0 2024-04-12T21:13:31.436 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:13:31.441+0000 7fb7cbe00700 1 --2- 172.21.15.18:0/825851610 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fb7b806ec00 0x7fb7b80710b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:13:31.437 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:13:31.441+0000 7fb7cbe00700 1 -- 172.21.15.18:0/825851610 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 6757+0+0 (secure 0 0 0) 0x7fb7c008ea40 con 0x7fb7d41037c0 2024-04-12T21:13:31.437 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:13:31.441+0000 7fb7d3e00700 1 --2- 172.21.15.18:0/825851610 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fb7b806ec00 0x7fb7b80710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:13:31.437 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:13:31.442+0000 7fb7d3e00700 1 --2- 172.21.15.18:0/825851610 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fb7b806ec00 0x7fb7b80710b0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fb7c4004f90 tx=0x7fb7c4009e30 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:13:31.441 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:13:31.446+0000 7fb7cbe00700 1 -- 172.21.15.18:0/825851610 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7fb7c0057740 con 0x7fb7d41037c0 2024-04-12T21:13:31.755 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:13:31.759+0000 7fb7db200700 1 -- 172.21.15.18:0/825851610 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fb7d4109f20 con 0x7fb7d41037c0 2024-04-12T21:13:31.756 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:13:31.760+0000 7fb7cbe00700 1 -- 172.21.15.18:0/825851610 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+837 (secure 0 0 0) 0x7fb7c005ae00 con 0x7fb7d41037c0 2024-04-12T21:13:31.756 INFO:teuthology.orchestra.run.smithi018.stdout:{ 2024-04-12T21:13:31.756 INFO:teuthology.orchestra.run.smithi018.stdout: "mon": { 2024-04-12T21:13:31.756 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 19.0.0-2879-g324baf8d (324baf8ddb91dea46cac50cb535c306115791355) squid (dev)": 2 2024-04-12T21:13:31.756 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-12T21:13:31.756 INFO:teuthology.orchestra.run.smithi018.stdout: "mgr": { 2024-04-12T21:13:31.756 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 19.0.0-2879-g324baf8d (324baf8ddb91dea46cac50cb535c306115791355) squid (dev)": 2 2024-04-12T21:13:31.757 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-12T21:13:31.757 INFO:teuthology.orchestra.run.smithi018.stdout: "osd": { 2024-04-12T21:13:31.757 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 19.0.0-2879-g324baf8d (324baf8ddb91dea46cac50cb535c306115791355) squid (dev)": 6 2024-04-12T21:13:31.757 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-12T21:13:31.757 INFO:teuthology.orchestra.run.smithi018.stdout: "mds": { 2024-04-12T21:13:31.757 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 17.2.7-827-gd3b5060a (d3b5060a17449edfae9385cdf792afec109fca5d) quincy (stable)": 2, 2024-04-12T21:13:31.757 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 19.0.0-2879-g324baf8d (324baf8ddb91dea46cac50cb535c306115791355) squid (dev)": 2 2024-04-12T21:13:31.757 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-12T21:13:31.757 INFO:teuthology.orchestra.run.smithi018.stdout: "overall": { 2024-04-12T21:13:31.757 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 17.2.7-827-gd3b5060a (d3b5060a17449edfae9385cdf792afec109fca5d) quincy (stable)": 2, 2024-04-12T21:13:31.757 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 19.0.0-2879-g324baf8d (324baf8ddb91dea46cac50cb535c306115791355) squid (dev)": 12 2024-04-12T21:13:31.757 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-04-12T21:13:31.757 INFO:teuthology.orchestra.run.smithi018.stdout:} 2024-04-12T21:13:31.759 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:13:31.764+0000 7fb7db200700 1 -- 172.21.15.18:0/825851610 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fb7b806ec00 msgr2=0x7fb7b80710b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:13:31.759 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:13:31.764+0000 7fb7db200700 1 --2- 172.21.15.18:0/825851610 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fb7b806ec00 0x7fb7b80710b0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fb7c4004f90 tx=0x7fb7c4009e30 comp rx=0 tx=0).stop 2024-04-12T21:13:31.759 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:13:31.764+0000 7fb7db200700 1 -- 172.21.15.18:0/825851610 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb7d41037c0 msgr2=0x7fb7d4070800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:13:31.759 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:13:31.764+0000 7fb7db200700 1 --2- 172.21.15.18:0/825851610 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb7d41037c0 0x7fb7d4070800 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fb7c000c890 tx=0x7fb7c00044c0 comp rx=0 tx=0).stop 2024-04-12T21:13:31.760 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:13:31.764+0000 7fb7db200700 1 -- 172.21.15.18:0/825851610 shutdown_connections 2024-04-12T21:13:31.760 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:13:31.764+0000 7fb7db200700 1 --2- 172.21.15.18:0/825851610 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fb7b806ec00 0x7fb7b80710b0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:13:31.760 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:13:31.764+0000 7fb7db200700 1 --2- 172.21.15.18:0/825851610 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb7d4102e20 0x7fb7d40702c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:13:31.760 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:13:31.764+0000 7fb7db200700 1 --2- 172.21.15.18:0/825851610 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb7d41037c0 0x7fb7d4070800 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:13:31.760 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:13:31.764+0000 7fb7db200700 1 -- 172.21.15.18:0/825851610 >> 172.21.15.18:0/825851610 conn(0x7fb7d40f89d0 msgr2=0x7fb7d41073c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:13:31.760 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:13:31.764+0000 7fb7db200700 1 -- 172.21.15.18:0/825851610 shutdown_connections 2024-04-12T21:13:31.760 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:13:31.764+0000 7fb7db200700 1 -- 172.21.15.18:0/825851610 wait complete. 2024-04-12T21:13:31.850 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:13:31 smithi018 ceph-mon[112590]: pgmap v411: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:13:31.850 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:13:31 smithi018 ceph-mon[112590]: from='client.34752 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:13:31.916 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:13:31.921+0000 7f7f52600700 1 -- 172.21.15.18:0/3295475897 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7f4c072e00 msgr2=0x7f7f4c10d800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:13:31.917 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:13:31.921+0000 7f7f52600700 1 --2- 172.21.15.18:0/3295475897 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7f4c072e00 0x7f7f4c10d800 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f7f38004940 tx=0x7f7f38009e30 comp rx=0 tx=0).stop 2024-04-12T21:13:31.917 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:13:31.921+0000 7f7f52600700 1 -- 172.21.15.18:0/3295475897 shutdown_connections 2024-04-12T21:13:31.917 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:13:31.921+0000 7f7f52600700 1 --2- 172.21.15.18:0/3295475897 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7f4c072e00 0x7f7f4c10d800 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:13:31.917 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:13:31.921+0000 7f7f52600700 1 --2- 172.21.15.18:0/3295475897 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7f4c0724f0 0x7f7f4c0728c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:13:31.917 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:13:31.921+0000 7f7f52600700 1 -- 172.21.15.18:0/3295475897 >> 172.21.15.18:0/3295475897 conn(0x7f7f4c0fac00 msgr2=0x7f7f4c0fd010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:13:31.917 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:13:31.921+0000 7f7f52600700 1 -- 172.21.15.18:0/3295475897 shutdown_connections 2024-04-12T21:13:31.917 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:13:31.921+0000 7f7f52600700 1 -- 172.21.15.18:0/3295475897 wait complete. 2024-04-12T21:13:31.918 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:13:31.922+0000 7f7f52600700 1 Processor -- start 2024-04-12T21:13:31.918 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:13:31.922+0000 7f7f52600700 1 -- start start 2024-04-12T21:13:31.918 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:13:31.923+0000 7f7f52600700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7f4c0724f0 0x7f7f4c119f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:13:31.918 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:13:31.923+0000 7f7f52600700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7f4c072e00 0x7f7f4c11a440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:13:31.918 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:13:31.923+0000 7f7f52600700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f4c11aa90 con 0x7f7f4c072e00 2024-04-12T21:13:31.918 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:13:31.923+0000 7f7f52600700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f4c114070 con 0x7f7f4c0724f0 2024-04-12T21:13:31.918 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:13:31.923+0000 7f7f4aa00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7f4c072e00 0x7f7f4c11a440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:13:31.918 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:13:31.923+0000 7f7f4aa00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7f4c072e00 0x7f7f4c11a440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:58416/0 (socket says 172.21.15.18:58416) 2024-04-12T21:13:31.919 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:13:31.923+0000 7f7f4aa00700 1 -- 172.21.15.18:0/3542089346 learned_addr learned my addr 172.21.15.18:0/3542089346 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:13:31.919 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:13:31.923+0000 7f7f4b400700 1 --2- 172.21.15.18:0/3542089346 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7f4c0724f0 0x7f7f4c119f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:13:31.920 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:13:31.923+0000 7f7f4aa00700 1 -- 172.21.15.18:0/3542089346 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7f4c0724f0 msgr2=0x7f7f4c119f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:13:31.920 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:13:31.923+0000 7f7f4aa00700 1 --2- 172.21.15.18:0/3542089346 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7f4c0724f0 0x7f7f4c119f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:13:31.920 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:13:31.923+0000 7f7f4aa00700 1 -- 172.21.15.18:0/3542089346 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f38005040 con 0x7f7f4c072e00 2024-04-12T21:13:31.920 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:13:31.924+0000 7f7f4aa00700 1 --2- 172.21.15.18:0/3542089346 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7f4c072e00 0x7f7f4c11a440 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f7f38009180 tx=0x7f7f38009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:13:31.920 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:13:31.924+0000 7f7f43400700 1 -- 172.21.15.18:0/3542089346 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7f3800e040 con 0x7f7f4c072e00 2024-04-12T21:13:31.920 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:13:31.924+0000 7f7f43400700 1 -- 172.21.15.18:0/3542089346 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f7f38013420 con 0x7f7f4c072e00 2024-04-12T21:13:31.920 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:13:31.924+0000 7f7f43400700 1 -- 172.21.15.18:0/3542089346 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7f3801c630 con 0x7f7f4c072e00 2024-04-12T21:13:31.920 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:13:31.924+0000 7f7f52600700 1 -- 172.21.15.18:0/3542089346 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7f4c1142f0 con 0x7f7f4c072e00 2024-04-12T21:13:31.920 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:13:31.924+0000 7f7f52600700 1 -- 172.21.15.18:0/3542089346 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7f4c1148b0 con 0x7f7f4c072e00 2024-04-12T21:13:31.921 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:13:31.925+0000 7f7f52600700 1 -- 172.21.15.18:0/3542089346 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7f4c073c30 con 0x7f7f4c072e00 2024-04-12T21:13:31.921 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:13:31.926+0000 7f7f43400700 1 -- 172.21.15.18:0/3542089346 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7f7f38013590 con 0x7f7f4c072e00 2024-04-12T21:13:31.922 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:13:31.927+0000 7f7f43400700 1 --2- 172.21.15.18:0/3542089346 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f7f3006ec00 0x7f7f300710b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:13:31.922 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:13:31.927+0000 7f7f43400700 1 -- 172.21.15.18:0/3542089346 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 6757+0+0 (secure 0 0 0) 0x7f7f3808fd50 con 0x7f7f4c072e00 2024-04-12T21:13:31.923 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:13:31.927+0000 7f7f4b400700 1 --2- 172.21.15.18:0/3542089346 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f7f3006ec00 0x7f7f300710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:13:31.923 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:13:31.928+0000 7f7f4b400700 1 --2- 172.21.15.18:0/3542089346 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f7f3006ec00 0x7f7f300710b0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f7f3c004d60 tx=0x7f7f3c00b040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:13:31.927 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:13:31.932+0000 7f7f43400700 1 -- 172.21.15.18:0/3542089346 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f7f3805c580 con 0x7f7f4c072e00 2024-04-12T21:13:32.029 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:13:31 smithi186 ceph-mon[86379]: pgmap v411: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:13:32.029 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:13:31 smithi186 ceph-mon[86379]: from='client.34752 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:13:32.192 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:13:32.197+0000 7f7f52600700 1 -- 172.21.15.18:0/3542089346 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f7f4c073c30 con 0x7f7f4c072e00 2024-04-12T21:13:32.193 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:13:32.198+0000 7f7f43400700 1 -- 172.21.15.18:0/3542089346 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 29 v29) v1 ==== 76+0+1872 (secure 0 0 0) 0x7f7f3805c110 con 0x7f7f4c072e00 2024-04-12T21:13:32.194 INFO:teuthology.orchestra.run.smithi018.stdout:e29 2024-04-12T21:13:32.194 INFO:teuthology.orchestra.run.smithi018.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-12T21:13:32.194 INFO:teuthology.orchestra.run.smithi018.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-12T21:13:32.194 INFO:teuthology.orchestra.run.smithi018.stdout:legacy client fscid: 1 2024-04-12T21:13:32.195 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-12T21:13:32.195 INFO:teuthology.orchestra.run.smithi018.stdout:Filesystem 'cephfs' (1) 2024-04-12T21:13:32.195 INFO:teuthology.orchestra.run.smithi018.stdout:fs_name cephfs 2024-04-12T21:13:32.195 INFO:teuthology.orchestra.run.smithi018.stdout:epoch 24 2024-04-12T21:13:32.195 INFO:teuthology.orchestra.run.smithi018.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-04-12T21:13:32.195 INFO:teuthology.orchestra.run.smithi018.stdout:created 2024-04-12T20:51:57.362850+0000 2024-04-12T21:13:32.195 INFO:teuthology.orchestra.run.smithi018.stdout:modified 2024-04-12T21:06:59.717190+0000 2024-04-12T21:13:32.195 INFO:teuthology.orchestra.run.smithi018.stdout:tableserver 0 2024-04-12T21:13:32.195 INFO:teuthology.orchestra.run.smithi018.stdout:root 0 2024-04-12T21:13:32.195 INFO:teuthology.orchestra.run.smithi018.stdout:session_timeout 60 2024-04-12T21:13:32.195 INFO:teuthology.orchestra.run.smithi018.stdout:session_autoclose 300 2024-04-12T21:13:32.195 INFO:teuthology.orchestra.run.smithi018.stdout:max_file_size 1099511627776 2024-04-12T21:13:32.195 INFO:teuthology.orchestra.run.smithi018.stdout:max_xattr_size 65536 2024-04-12T21:13:32.195 INFO:teuthology.orchestra.run.smithi018.stdout:required_client_features {} 2024-04-12T21:13:32.195 INFO:teuthology.orchestra.run.smithi018.stdout:last_failure 0 2024-04-12T21:13:32.196 INFO:teuthology.orchestra.run.smithi018.stdout:last_failure_osd_epoch 100 2024-04-12T21:13:32.196 INFO:teuthology.orchestra.run.smithi018.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-12T21:13:32.196 INFO:teuthology.orchestra.run.smithi018.stdout:max_mds 2 2024-04-12T21:13:32.196 INFO:teuthology.orchestra.run.smithi018.stdout:in 0,1 2024-04-12T21:13:32.196 INFO:teuthology.orchestra.run.smithi018.stdout:up {} 2024-04-12T21:13:32.196 INFO:teuthology.orchestra.run.smithi018.stdout:failed 0,1 2024-04-12T21:13:32.196 INFO:teuthology.orchestra.run.smithi018.stdout:damaged 2024-04-12T21:13:32.196 INFO:teuthology.orchestra.run.smithi018.stdout:stopped 2024-04-12T21:13:32.196 INFO:teuthology.orchestra.run.smithi018.stdout:data_pools [3] 2024-04-12T21:13:32.196 INFO:teuthology.orchestra.run.smithi018.stdout:metadata_pool 2 2024-04-12T21:13:32.196 INFO:teuthology.orchestra.run.smithi018.stdout:inline_data disabled 2024-04-12T21:13:32.196 INFO:teuthology.orchestra.run.smithi018.stdout:balancer 2024-04-12T21:13:32.196 INFO:teuthology.orchestra.run.smithi018.stdout:bal_rank_mask -1 2024-04-12T21:13:32.196 INFO:teuthology.orchestra.run.smithi018.stdout:standby_count_wanted 1 2024-04-12T21:13:32.196 INFO:teuthology.orchestra.run.smithi018.stdout:qdb_cluster leader: 0 members: 2024-04-12T21:13:32.197 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-12T21:13:32.197 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-12T21:13:32.197 INFO:teuthology.orchestra.run.smithi018.stdout:Standby daemons: 2024-04-12T21:13:32.197 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-12T21:13:32.197 INFO:teuthology.orchestra.run.smithi018.stdout:[mds.cephfs.smithi186.wjnafs{-1:24277} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.186:6826/438347207,v1:172.21.15.186:6827/438347207] compat {c=[1],r=[1],i=[7ff]}] 2024-04-12T21:13:32.197 INFO:teuthology.orchestra.run.smithi018.stdout:[mds.cephfs.smithi186.jdllyn{-1:24281} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.186:6824/1464359952,v1:172.21.15.186:6825/1464359952] compat {c=[1],r=[1],i=[7ff]}] 2024-04-12T21:13:32.197 INFO:teuthology.orchestra.run.smithi018.stdout:[mds.cephfs.smithi018.acqbsc{-1:34436} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.18:6828/755179764,v1:172.21.15.18:6829/755179764] compat {c=[1],r=[1],i=[1fff]}] 2024-04-12T21:13:32.197 INFO:teuthology.orchestra.run.smithi018.stdout:[mds.cephfs.smithi018.szjluo{-1:34468} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.18:6826/2407992653,v1:172.21.15.18:6827/2407992653] compat {c=[1],r=[1],i=[1fff]}] 2024-04-12T21:13:32.198 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:13:32.202+0000 7f7f52600700 1 -- 172.21.15.18:0/3542089346 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f7f3006ec00 msgr2=0x7f7f300710b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:13:32.198 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:13:32.202+0000 7f7f52600700 1 --2- 172.21.15.18:0/3542089346 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f7f3006ec00 0x7f7f300710b0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f7f3c004d60 tx=0x7f7f3c00b040 comp rx=0 tx=0).stop 2024-04-12T21:13:32.198 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:13:32.202+0000 7f7f52600700 1 -- 172.21.15.18:0/3542089346 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7f4c072e00 msgr2=0x7f7f4c11a440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:13:32.198 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:13:32.202+0000 7f7f52600700 1 --2- 172.21.15.18:0/3542089346 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7f4c072e00 0x7f7f4c11a440 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f7f38009180 tx=0x7f7f38009e30 comp rx=0 tx=0).stop 2024-04-12T21:13:32.198 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:13:32.202+0000 7f7f52600700 1 -- 172.21.15.18:0/3542089346 shutdown_connections 2024-04-12T21:13:32.198 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:13:32.202+0000 7f7f52600700 1 --2- 172.21.15.18:0/3542089346 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f7f3006ec00 0x7f7f300710b0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:13:32.198 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:13:32.202+0000 7f7f52600700 1 --2- 172.21.15.18:0/3542089346 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7f4c0724f0 0x7f7f4c119f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:13:32.198 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:13:32.202+0000 7f7f52600700 1 --2- 172.21.15.18:0/3542089346 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7f4c072e00 0x7f7f4c11a440 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:13:32.198 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:13:32.202+0000 7f7f52600700 1 -- 172.21.15.18:0/3542089346 >> 172.21.15.18:0/3542089346 conn(0x7f7f4c0fac00 msgr2=0x7f7f4c0fc270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:13:32.198 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:13:32.202+0000 7f7f52600700 1 -- 172.21.15.18:0/3542089346 shutdown_connections 2024-04-12T21:13:32.198 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:13:32.203+0000 7f7f52600700 1 -- 172.21.15.18:0/3542089346 wait complete. 2024-04-12T21:13:32.199 INFO:teuthology.orchestra.run.smithi018.stderr:dumped fsmap epoch 29 2024-04-12T21:13:32.358 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:13:32.362+0000 7fc908400700 1 -- 172.21.15.18:0/1502214841 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc900109dd0 msgr2=0x7fc900103440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:13:32.358 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:13:32.362+0000 7fc908400700 1 --2- 172.21.15.18:0/1502214841 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc900109dd0 0x7fc900103440 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fc8f0004940 tx=0x7fc8f0009e30 comp rx=0 tx=0).stop 2024-04-12T21:13:32.359 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:13:32.363+0000 7fc908400700 1 -- 172.21.15.18:0/1502214841 shutdown_connections 2024-04-12T21:13:32.359 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:13:32.363+0000 7fc908400700 1 --2- 172.21.15.18:0/1502214841 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc900109dd0 0x7fc900103440 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:13:32.359 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:13:32.363+0000 7fc908400700 1 --2- 172.21.15.18:0/1502214841 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc900109430 0x7fc900109800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:13:32.359 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:13:32.363+0000 7fc908400700 1 -- 172.21.15.18:0/1502214841 >> 172.21.15.18:0/1502214841 conn(0x7fc900072c00 msgr2=0x7fc900075010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:13:32.359 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:13:32.363+0000 7fc908400700 1 -- 172.21.15.18:0/1502214841 shutdown_connections 2024-04-12T21:13:32.359 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:13:32.363+0000 7fc908400700 1 -- 172.21.15.18:0/1502214841 wait complete. 2024-04-12T21:13:32.359 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:13:32.364+0000 7fc908400700 1 Processor -- start 2024-04-12T21:13:32.360 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:13:32.364+0000 7fc908400700 1 -- start start 2024-04-12T21:13:32.360 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:13:32.364+0000 7fc908400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc900109430 0x7fc900119f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:13:32.361 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:13:32.364+0000 7fc908400700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc900109dd0 0x7fc90011a4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:13:32.361 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:13:32.364+0000 7fc908400700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc90011ab10 con 0x7fc900109430 2024-04-12T21:13:32.361 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:13:32.364+0000 7fc908400700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9001140f0 con 0x7fc900109dd0 2024-04-12T21:13:32.361 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:13:32.365+0000 7fc905a00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc900109430 0x7fc900119f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:13:32.361 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:13:32.365+0000 7fc905a00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc900109430 0x7fc900119f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:58432/0 (socket says 172.21.15.18:58432) 2024-04-12T21:13:32.361 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:13:32.365+0000 7fc905a00700 1 -- 172.21.15.18:0/1301129812 learned_addr learned my addr 172.21.15.18:0/1301129812 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:13:32.361 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:13:32.365+0000 7fc905000700 1 --2- 172.21.15.18:0/1301129812 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc900109dd0 0x7fc90011a4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:13:32.361 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:13:32.365+0000 7fc905a00700 1 -- 172.21.15.18:0/1301129812 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc900109dd0 msgr2=0x7fc90011a4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:13:32.361 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:13:32.365+0000 7fc905a00700 1 --2- 172.21.15.18:0/1301129812 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc900109dd0 0x7fc90011a4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:13:32.361 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:13:32.365+0000 7fc905a00700 1 -- 172.21.15.18:0/1301129812 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8f0005040 con 0x7fc900109430 2024-04-12T21:13:32.362 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:13:32.365+0000 7fc905a00700 1 --2- 172.21.15.18:0/1301129812 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc900109430 0x7fc900119f80 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fc8f400c8c0 tx=0x7fc8f40044c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:13:32.362 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:13:32.366+0000 7fc8fd600700 1 -- 172.21.15.18:0/1301129812 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc8f400a980 con 0x7fc900109430 2024-04-12T21:13:32.362 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:13:32.366+0000 7fc8fd600700 1 -- 172.21.15.18:0/1301129812 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc8f400ce60 con 0x7fc900109430 2024-04-12T21:13:32.362 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:13:32.366+0000 7fc8fd600700 1 -- 172.21.15.18:0/1301129812 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc8f40115e0 con 0x7fc900109430 2024-04-12T21:13:32.363 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:13:32.366+0000 7fc908400700 1 -- 172.21.15.18:0/1301129812 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc9001143d0 con 0x7fc900109430 2024-04-12T21:13:32.363 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:13:32.366+0000 7fc908400700 1 -- 172.21.15.18:0/1301129812 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc900114990 con 0x7fc900109430 2024-04-12T21:13:32.363 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:13:32.367+0000 7fc908400700 1 -- 172.21.15.18:0/1301129812 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc900104b80 con 0x7fc900109430 2024-04-12T21:13:32.363 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:13:32.368+0000 7fc8fd600700 1 -- 172.21.15.18:0/1301129812 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7fc8f4012400 con 0x7fc900109430 2024-04-12T21:13:32.365 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:13:32.369+0000 7fc8fd600700 1 --2- 172.21.15.18:0/1301129812 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fc8e806ec00 0x7fc8e80710b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:13:32.365 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:13:32.369+0000 7fc8fd600700 1 -- 172.21.15.18:0/1301129812 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 6757+0+0 (secure 0 0 0) 0x7fc8f408e980 con 0x7fc900109430 2024-04-12T21:13:32.365 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:13:32.370+0000 7fc905000700 1 --2- 172.21.15.18:0/1301129812 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fc8e806ec00 0x7fc8e80710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:13:32.365 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:13:32.370+0000 7fc905000700 1 --2- 172.21.15.18:0/1301129812 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fc8e806ec00 0x7fc8e80710b0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fc8f000b3b0 tx=0x7fc8f0009e30 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:13:32.369 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:13:32.374+0000 7fc8fd600700 1 -- 172.21.15.18:0/1301129812 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7fc8f405b270 con 0x7fc900109430 2024-04-12T21:13:32.624 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:13:32.629+0000 7fc908400700 1 -- 172.21.15.18:0/1301129812 --> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fc9001155d0 con 0x7fc8e806ec00 2024-04-12T21:13:32.627 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:13:32.632+0000 7fc8fd600700 1 -- 172.21.15.18:0/1301129812 <== mgr.34102 v2:172.21.15.18:6800/3812891919 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7fc9001155d0 con 0x7fc8e806ec00 2024-04-12T21:13:32.627 INFO:teuthology.orchestra.run.smithi018.stdout:{ 2024-04-12T21:13:32.627 INFO:teuthology.orchestra.run.smithi018.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:f4452def3be4d350f4a8e5feaf7ebb1511be9298fe68478ba7a2d6fbd7d69a63", 2024-04-12T21:13:32.627 INFO:teuthology.orchestra.run.smithi018.stdout: "in_progress": true, 2024-04-12T21:13:32.627 INFO:teuthology.orchestra.run.smithi018.stdout: "which": "Upgrading all daemon types on all hosts", 2024-04-12T21:13:32.628 INFO:teuthology.orchestra.run.smithi018.stdout: "services_complete": [ 2024-04-12T21:13:32.628 INFO:teuthology.orchestra.run.smithi018.stdout: "crash", 2024-04-12T21:13:32.628 INFO:teuthology.orchestra.run.smithi018.stdout: "osd", 2024-04-12T21:13:32.628 INFO:teuthology.orchestra.run.smithi018.stdout: "mon", 2024-04-12T21:13:32.628 INFO:teuthology.orchestra.run.smithi018.stdout: "mgr" 2024-04-12T21:13:32.628 INFO:teuthology.orchestra.run.smithi018.stdout: ], 2024-04-12T21:13:32.628 INFO:teuthology.orchestra.run.smithi018.stdout: "progress": "14/21 daemons upgraded", 2024-04-12T21:13:32.628 INFO:teuthology.orchestra.run.smithi018.stdout: "message": "Currently upgrading mds daemons", 2024-04-12T21:13:32.628 INFO:teuthology.orchestra.run.smithi018.stdout: "is_paused": false 2024-04-12T21:13:32.629 INFO:teuthology.orchestra.run.smithi018.stdout:} 2024-04-12T21:13:32.630 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:13:32.634+0000 7fc908400700 1 -- 172.21.15.18:0/1301129812 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fc8e806ec00 msgr2=0x7fc8e80710b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:13:32.631 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:13:32.634+0000 7fc908400700 1 --2- 172.21.15.18:0/1301129812 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fc8e806ec00 0x7fc8e80710b0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fc8f000b3b0 tx=0x7fc8f0009e30 comp rx=0 tx=0).stop 2024-04-12T21:13:32.631 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:13:32.635+0000 7fc908400700 1 -- 172.21.15.18:0/1301129812 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc900109430 msgr2=0x7fc900119f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:13:32.631 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:13:32.635+0000 7fc908400700 1 --2- 172.21.15.18:0/1301129812 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc900109430 0x7fc900119f80 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fc8f400c8c0 tx=0x7fc8f40044c0 comp rx=0 tx=0).stop 2024-04-12T21:13:32.631 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:13:32.635+0000 7fc908400700 1 -- 172.21.15.18:0/1301129812 shutdown_connections 2024-04-12T21:13:32.631 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:13:32.635+0000 7fc908400700 1 --2- 172.21.15.18:0/1301129812 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fc8e806ec00 0x7fc8e80710b0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:13:32.631 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:13:32.635+0000 7fc908400700 1 --2- 172.21.15.18:0/1301129812 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc900109430 0x7fc900119f80 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:13:32.631 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:13:32.635+0000 7fc908400700 1 --2- 172.21.15.18:0/1301129812 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc900109dd0 0x7fc90011a4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:13:32.631 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:13:32.635+0000 7fc908400700 1 -- 172.21.15.18:0/1301129812 >> 172.21.15.18:0/1301129812 conn(0x7fc900072c00 msgr2=0x7fc900074c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:13:32.631 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:13:32.635+0000 7fc908400700 1 -- 172.21.15.18:0/1301129812 shutdown_connections 2024-04-12T21:13:32.631 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:13:32.635+0000 7fc908400700 1 -- 172.21.15.18:0/1301129812 wait complete. 2024-04-12T21:13:32.792 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:13:32.796+0000 7f53e5800700 1 -- 172.21.15.18:0/3771832580 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f53e0105020 msgr2=0x7f53e01053f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:13:32.792 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:13:32.796+0000 7f53e5800700 1 --2- 172.21.15.18:0/3771832580 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f53e0105020 0x7f53e01053f0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f53d0004940 tx=0x7f53d0009e30 comp rx=0 tx=0).stop 2024-04-12T21:13:32.792 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:13:32.796+0000 7f53e5800700 1 -- 172.21.15.18:0/3771832580 shutdown_connections 2024-04-12T21:13:32.792 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:13:32.796+0000 7f53e5800700 1 --2- 172.21.15.18:0/3771832580 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f53e01059c0 0x7f53e00ff080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:13:32.792 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:13:32.796+0000 7f53e5800700 1 --2- 172.21.15.18:0/3771832580 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f53e0105020 0x7f53e01053f0 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:13:32.792 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:13:32.796+0000 7f53e5800700 1 -- 172.21.15.18:0/3771832580 >> 172.21.15.18:0/3771832580 conn(0x7f53e00faba0 msgr2=0x7f53e00fcfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:13:32.793 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:13:32.796+0000 7f53e5800700 1 -- 172.21.15.18:0/3771832580 shutdown_connections 2024-04-12T21:13:32.793 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:13:32.796+0000 7f53e5800700 1 -- 172.21.15.18:0/3771832580 wait complete. 2024-04-12T21:13:32.794 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:13:32.797+0000 7f53e5800700 1 Processor -- start 2024-04-12T21:13:32.794 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:13:32.797+0000 7f53e5800700 1 -- start start 2024-04-12T21:13:32.794 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:13:32.798+0000 7f53e5800700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f53e0105020 0x7f53e0115b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:13:32.794 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:13:32.798+0000 7f53e5800700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f53e01059c0 0x7f53e01160d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:13:32.794 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:13:32.798+0000 7f53e5800700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53e0116720 con 0x7f53e01059c0 2024-04-12T21:13:32.794 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:13:32.798+0000 7f53e5800700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53e010fd00 con 0x7f53e0105020 2024-04-12T21:13:32.794 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:13:32.798+0000 7f53de000700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f53e01059c0 0x7f53e01160d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:13:32.794 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:13:32.798+0000 7f53de000700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f53e01059c0 0x7f53e01160d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:58450/0 (socket says 172.21.15.18:58450) 2024-04-12T21:13:32.794 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:13:32.798+0000 7f53de000700 1 -- 172.21.15.18:0/1309551585 learned_addr learned my addr 172.21.15.18:0/1309551585 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:13:32.794 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:13:32.798+0000 7f53dea00700 1 --2- 172.21.15.18:0/1309551585 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f53e0105020 0x7f53e0115b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:13:32.795 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:13:32.799+0000 7f53de000700 1 -- 172.21.15.18:0/1309551585 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f53e0105020 msgr2=0x7f53e0115b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:13:32.795 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:13:32.799+0000 7f53de000700 1 --2- 172.21.15.18:0/1309551585 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f53e0105020 0x7f53e0115b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:13:32.795 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:13:32.799+0000 7f53de000700 1 -- 172.21.15.18:0/1309551585 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53d0005040 con 0x7f53e01059c0 2024-04-12T21:13:32.795 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:13:32.799+0000 7f53de000700 1 --2- 172.21.15.18:0/1309551585 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f53e01059c0 0x7f53e01160d0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f53cc00c800 tx=0x7f53cc004470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:13:32.795 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:13:32.799+0000 7f53d6a00700 1 -- 172.21.15.18:0/1309551585 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f53cc00a760 con 0x7f53e01059c0 2024-04-12T21:13:32.796 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:13:32.799+0000 7f53d6a00700 1 -- 172.21.15.18:0/1309551585 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f53cc00ad90 con 0x7f53e01059c0 2024-04-12T21:13:32.796 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:13:32.799+0000 7f53e5800700 1 -- 172.21.15.18:0/1309551585 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f53e010ffe0 con 0x7f53e01059c0 2024-04-12T21:13:32.796 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:13:32.799+0000 7f53d6a00700 1 -- 172.21.15.18:0/1309551585 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f53cc0113a0 con 0x7f53e01059c0 2024-04-12T21:13:32.796 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:13:32.799+0000 7f53e5800700 1 -- 172.21.15.18:0/1309551585 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f53e01105a0 con 0x7f53e01059c0 2024-04-12T21:13:32.796 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:13:32.800+0000 7f53e5800700 1 -- 172.21.15.18:0/1309551585 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f53e0100770 con 0x7f53e01059c0 2024-04-12T21:13:32.796 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:13:32.801+0000 7f53d6a00700 1 -- 172.21.15.18:0/1309551585 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7f53cc012400 con 0x7f53e01059c0 2024-04-12T21:13:32.797 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:13:32.802+0000 7f53d6a00700 1 --2- 172.21.15.18:0/1309551585 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f53c406ec00 0x7f53c40710b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:13:32.798 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:13:32.803+0000 7f53d6a00700 1 -- 172.21.15.18:0/1309551585 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 6757+0+0 (secure 0 0 0) 0x7f53cc08e990 con 0x7f53e01059c0 2024-04-12T21:13:32.798 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:13:32.803+0000 7f53dea00700 1 --2- 172.21.15.18:0/1309551585 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f53c406ec00 0x7f53c40710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:13:32.799 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:13:32.804+0000 7f53dea00700 1 --2- 172.21.15.18:0/1309551585 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f53c406ec00 0x7f53c40710b0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f53d000b020 tx=0x7f53d0009e30 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:13:32.803 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:13:32.807+0000 7f53d6a00700 1 -- 172.21.15.18:0/1309551585 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f53cc05b1c0 con 0x7f53e01059c0 2024-04-12T21:13:32.850 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:13:32 smithi018 ceph-mon[112590]: from='client.34756 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:13:32.850 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:13:32 smithi018 ceph-mon[112590]: from='client.34760 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:13:32.850 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:13:32 smithi018 ceph-mon[112590]: from='client.? 172.21.15.18:0/825851610' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T21:13:32.850 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:13:32 smithi018 ceph-mon[112590]: from='client.? 172.21.15.18:0/3542089346' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-12T21:13:33.029 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:13:32 smithi186 ceph-mon[86379]: from='client.34756 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:13:33.029 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:13:32 smithi186 ceph-mon[86379]: from='client.34760 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:13:33.029 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:13:32 smithi186 ceph-mon[86379]: from='client.? 172.21.15.18:0/825851610' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T21:13:33.029 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:13:32 smithi186 ceph-mon[86379]: from='client.? 172.21.15.18:0/3542089346' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-12T21:13:33.115 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:13:33.119+0000 7f53e5800700 1 -- 172.21.15.18:0/1309551585 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f53e010c120 con 0x7f53e01059c0 2024-04-12T21:13:33.116 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:13:33.120+0000 7f53d6a00700 1 -- 172.21.15.18:0/1309551585 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+233 (secure 0 0 0) 0x7f53cc05ad50 con 0x7f53e01059c0 2024-04-12T21:13:33.116 INFO:teuthology.orchestra.run.smithi018.stdout:HEALTH_ERR 1 filesystem is degraded; 1 filesystem is offline 2024-04-12T21:13:33.116 INFO:teuthology.orchestra.run.smithi018.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-04-12T21:13:33.116 INFO:teuthology.orchestra.run.smithi018.stdout: fs cephfs is degraded 2024-04-12T21:13:33.116 INFO:teuthology.orchestra.run.smithi018.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-04-12T21:13:33.116 INFO:teuthology.orchestra.run.smithi018.stdout: fs cephfs is offline because no MDS is active for it. 2024-04-12T21:13:33.118 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:13:33.123+0000 7f53e5800700 1 -- 172.21.15.18:0/1309551585 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f53c406ec00 msgr2=0x7f53c40710b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:13:33.119 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:13:33.123+0000 7f53e5800700 1 --2- 172.21.15.18:0/1309551585 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f53c406ec00 0x7f53c40710b0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f53d000b020 tx=0x7f53d0009e30 comp rx=0 tx=0).stop 2024-04-12T21:13:33.119 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:13:33.123+0000 7f53e5800700 1 -- 172.21.15.18:0/1309551585 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f53e01059c0 msgr2=0x7f53e01160d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:13:33.119 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:13:33.123+0000 7f53e5800700 1 --2- 172.21.15.18:0/1309551585 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f53e01059c0 0x7f53e01160d0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f53cc00c800 tx=0x7f53cc004470 comp rx=0 tx=0).stop 2024-04-12T21:13:33.119 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:13:33.123+0000 7f53e5800700 1 -- 172.21.15.18:0/1309551585 shutdown_connections 2024-04-12T21:13:33.119 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:13:33.123+0000 7f53e5800700 1 --2- 172.21.15.18:0/1309551585 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f53c406ec00 0x7f53c40710b0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:13:33.119 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:13:33.123+0000 7f53e5800700 1 --2- 172.21.15.18:0/1309551585 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f53e0105020 0x7f53e0115b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:13:33.119 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:13:33.123+0000 7f53e5800700 1 --2- 172.21.15.18:0/1309551585 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f53e01059c0 0x7f53e01160d0 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:13:33.119 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:13:33.123+0000 7f53e5800700 1 -- 172.21.15.18:0/1309551585 >> 172.21.15.18:0/1309551585 conn(0x7f53e00faba0 msgr2=0x7f53e00fbb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:13:33.119 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:13:33.123+0000 7f53e5800700 1 -- 172.21.15.18:0/1309551585 shutdown_connections 2024-04-12T21:13:33.119 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:13:33.123+0000 7f53e5800700 1 -- 172.21.15.18:0/1309551585 wait complete. 2024-04-12T21:13:33.850 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:13:33 smithi018 ceph-mon[112590]: pgmap v412: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:13:33.850 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:13:33 smithi018 ceph-mon[112590]: from='client.34772 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:13:33.850 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:13:33 smithi018 ceph-mon[112590]: from='client.? 172.21.15.18:0/1309551585' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-12T21:13:34.029 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:13:33 smithi186 ceph-mon[86379]: pgmap v412: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:13:34.029 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:13:33 smithi186 ceph-mon[86379]: from='client.34772 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:13:34.029 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:13:33 smithi186 ceph-mon[86379]: from='client.? 172.21.15.18:0/1309551585' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-12T21:13:35.850 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:13:35 smithi018 ceph-mon[112590]: pgmap v413: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:13:36.029 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:13:35 smithi186 ceph-mon[86379]: pgmap v413: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:13:37.850 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:13:37 smithi018 ceph-mon[112590]: pgmap v414: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:13:38.028 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:13:37 smithi186 ceph-mon[86379]: pgmap v414: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:13:39.850 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:13:39 smithi018 ceph-mon[112590]: pgmap v415: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:13:40.028 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:13:39 smithi186 ceph-mon[86379]: pgmap v415: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:13:40.849 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:13:40 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T21:13:41.029 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:13:40 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T21:13:41.850 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:13:41 smithi018 ceph-mon[112590]: pgmap v416: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:13:42.028 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:13:41 smithi186 ceph-mon[86379]: pgmap v416: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:13:44.029 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:13:43 smithi186 ceph-mon[86379]: pgmap v417: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:13:44.100 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:13:43 smithi018 ceph-mon[112590]: pgmap v417: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:13:45.849 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:13:45 smithi018 ceph-mon[112590]: pgmap v418: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:13:46.029 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:13:45 smithi186 ceph-mon[86379]: pgmap v418: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:13:48.029 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:13:47 smithi186 ceph-mon[86379]: pgmap v419: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:13:48.100 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:13:47 smithi018 ceph-mon[112590]: pgmap v419: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:13:50.029 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:13:49 smithi186 ceph-mon[86379]: pgmap v420: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:13:50.100 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:13:49 smithi018 ceph-mon[112590]: pgmap v420: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:13:52.028 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:13:51 smithi186 ceph-mon[86379]: pgmap v421: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:13:52.099 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:13:51 smithi018 ceph-mon[112590]: pgmap v421: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:13:54.029 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:13:53 smithi186 ceph-mon[86379]: pgmap v422: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:13:54.099 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:13:53 smithi018 ceph-mon[112590]: pgmap v422: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:13:55.028 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:13:54 smithi186 ceph-mon[86379]: pgmap v423: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:13:55.099 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:13:54 smithi018 ceph-mon[112590]: pgmap v423: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:13:56.028 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:13:55 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T21:13:56.099 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:13:55 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T21:13:57.028 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:13:56 smithi186 ceph-mon[86379]: pgmap v424: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:13:57.099 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:13:56 smithi018 ceph-mon[112590]: pgmap v424: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:13:59.528 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:13:59 smithi186 ceph-mon[86379]: pgmap v425: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:13:59.600 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:13:59 smithi018 ceph-mon[112590]: pgmap v425: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:14:01.529 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:14:01 smithi186 ceph-mon[86379]: pgmap v426: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:14:01.599 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:14:01 smithi018 ceph-mon[112590]: pgmap v426: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:14:03.293 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:03.296+0000 7fc378c00700 1 -- 172.21.15.18:0/1760177723 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc374071f90 msgr2=0x7fc374070360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:14:03.293 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:03.296+0000 7fc378c00700 1 --2- 172.21.15.18:0/1760177723 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc374071f90 0x7fc374070360 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fc364004970 tx=0x7fc364009e30 comp rx=0 tx=0).stop 2024-04-12T21:14:03.293 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:03.296+0000 7fc378c00700 1 -- 172.21.15.18:0/1760177723 shutdown_connections 2024-04-12T21:14:03.293 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:03.296+0000 7fc378c00700 1 --2- 172.21.15.18:0/1760177723 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc3740708a0 0x7fc374070d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:14:03.294 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:03.296+0000 7fc378c00700 1 --2- 172.21.15.18:0/1760177723 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc374071f90 0x7fc374070360 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:14:03.294 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:03.296+0000 7fc378c00700 1 -- 172.21.15.18:0/1760177723 >> 172.21.15.18:0/1760177723 conn(0x7fc3740f8970 msgr2=0x7fc3740fad80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:14:03.294 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:03.297+0000 7fc378c00700 1 -- 172.21.15.18:0/1760177723 shutdown_connections 2024-04-12T21:14:03.294 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:03.297+0000 7fc378c00700 1 -- 172.21.15.18:0/1760177723 wait complete. 2024-04-12T21:14:03.296 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:03.297+0000 7fc378c00700 1 Processor -- start 2024-04-12T21:14:03.296 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:03.298+0000 7fc378c00700 1 -- start start 2024-04-12T21:14:03.296 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:03.298+0000 7fc378c00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc3740708a0 0x7fc37411a160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:14:03.296 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:03.298+0000 7fc378c00700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc374071f90 0x7fc37411a6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:14:03.296 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:03.298+0000 7fc378c00700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc37411acf0 con 0x7fc3740708a0 2024-04-12T21:14:03.296 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:03.298+0000 7fc378c00700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3741140c0 con 0x7fc374071f90 2024-04-12T21:14:03.296 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:03.298+0000 7fc371e00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc3740708a0 0x7fc37411a160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:14:03.296 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:03.298+0000 7fc371e00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc3740708a0 0x7fc37411a160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:55634/0 (socket says 172.21.15.18:55634) 2024-04-12T21:14:03.297 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:03.298+0000 7fc371e00700 1 -- 172.21.15.18:0/1519316880 learned_addr learned my addr 172.21.15.18:0/1519316880 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:14:03.297 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:03.299+0000 7fc371400700 1 --2- 172.21.15.18:0/1519316880 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc374071f90 0x7fc37411a6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:14:03.297 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:03.299+0000 7fc371e00700 1 -- 172.21.15.18:0/1519316880 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc374071f90 msgr2=0x7fc37411a6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:14:03.297 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:03.299+0000 7fc371e00700 1 --2- 172.21.15.18:0/1519316880 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc374071f90 0x7fc37411a6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:14:03.297 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:03.299+0000 7fc371e00700 1 -- 172.21.15.18:0/1519316880 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc364005040 con 0x7fc3740708a0 2024-04-12T21:14:03.297 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:03.299+0000 7fc371e00700 1 --2- 172.21.15.18:0/1519316880 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc3740708a0 0x7fc37411a160 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fc364005010 tx=0x7fc364009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:14:03.297 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:03.300+0000 7fc36a000700 1 -- 172.21.15.18:0/1519316880 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc36400e040 con 0x7fc3740708a0 2024-04-12T21:14:03.297 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:03.300+0000 7fc36a000700 1 -- 172.21.15.18:0/1519316880 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc364009950 con 0x7fc3740708a0 2024-04-12T21:14:03.297 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:03.300+0000 7fc36a000700 1 -- 172.21.15.18:0/1519316880 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc36401c3f0 con 0x7fc3740708a0 2024-04-12T21:14:03.297 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:03.300+0000 7fc378c00700 1 -- 172.21.15.18:0/1519316880 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc374114340 con 0x7fc3740708a0 2024-04-12T21:14:03.297 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:03.300+0000 7fc378c00700 1 -- 172.21.15.18:0/1519316880 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc374114880 con 0x7fc3740708a0 2024-04-12T21:14:03.298 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:03.301+0000 7fc378c00700 1 -- 172.21.15.18:0/1519316880 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc374071aa0 con 0x7fc3740708a0 2024-04-12T21:14:03.298 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:03.302+0000 7fc36a000700 1 -- 172.21.15.18:0/1519316880 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7fc3640137b0 con 0x7fc3740708a0 2024-04-12T21:14:03.300 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:03.304+0000 7fc36a000700 1 --2- 172.21.15.18:0/1519316880 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fc35806ec10 0x7fc3580710c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:14:03.300 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:03.304+0000 7fc36a000700 1 -- 172.21.15.18:0/1519316880 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 6757+0+0 (secure 0 0 0) 0x7fc36408fc60 con 0x7fc3740708a0 2024-04-12T21:14:03.300 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:03.304+0000 7fc371400700 1 --2- 172.21.15.18:0/1519316880 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fc35806ec10 0x7fc3580710c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:14:03.301 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:03.305+0000 7fc371400700 1 --2- 172.21.15.18:0/1519316880 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fc35806ec10 0x7fc3580710c0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fc360004600 tx=0x7fc36000b040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:14:03.305 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:03.309+0000 7fc36a000700 1 -- 172.21.15.18:0/1519316880 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7fc36405c490 con 0x7fc3740708a0 2024-04-12T21:14:03.528 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:14:03 smithi186 ceph-mon[86379]: pgmap v427: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:14:03.562 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:03.565+0000 7fc378c00700 1 -- 172.21.15.18:0/1519316880 --> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fc374071aa0 con 0x7fc35806ec10 2024-04-12T21:14:03.562 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:14:03 smithi018 ceph-mon[112590]: pgmap v427: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:14:03.564 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:03.568+0000 7fc36a000700 1 -- 172.21.15.18:0/1519316880 <== mgr.34102 v2:172.21.15.18:6800/3812891919 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7fc374071aa0 con 0x7fc35806ec10 2024-04-12T21:14:03.568 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:03.571+0000 7fc378c00700 1 -- 172.21.15.18:0/1519316880 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fc35806ec10 msgr2=0x7fc3580710c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:14:03.568 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:03.571+0000 7fc378c00700 1 --2- 172.21.15.18:0/1519316880 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fc35806ec10 0x7fc3580710c0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fc360004600 tx=0x7fc36000b040 comp rx=0 tx=0).stop 2024-04-12T21:14:03.568 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:03.571+0000 7fc378c00700 1 -- 172.21.15.18:0/1519316880 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc3740708a0 msgr2=0x7fc37411a160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:14:03.568 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:03.571+0000 7fc378c00700 1 --2- 172.21.15.18:0/1519316880 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc3740708a0 0x7fc37411a160 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fc364005010 tx=0x7fc364009e30 comp rx=0 tx=0).stop 2024-04-12T21:14:03.568 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:03.571+0000 7fc378c00700 1 -- 172.21.15.18:0/1519316880 shutdown_connections 2024-04-12T21:14:03.568 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:03.571+0000 7fc378c00700 1 --2- 172.21.15.18:0/1519316880 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fc35806ec10 0x7fc3580710c0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:14:03.568 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:03.571+0000 7fc378c00700 1 --2- 172.21.15.18:0/1519316880 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc3740708a0 0x7fc37411a160 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:14:03.568 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:03.571+0000 7fc378c00700 1 --2- 172.21.15.18:0/1519316880 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc374071f90 0x7fc37411a6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:14:03.568 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:03.571+0000 7fc378c00700 1 -- 172.21.15.18:0/1519316880 >> 172.21.15.18:0/1519316880 conn(0x7fc3740f8970 msgr2=0x7fc3740f9a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:14:03.568 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:03.571+0000 7fc378c00700 1 -- 172.21.15.18:0/1519316880 shutdown_connections 2024-04-12T21:14:03.568 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:03.571+0000 7fc378c00700 1 -- 172.21.15.18:0/1519316880 wait complete. 2024-04-12T21:14:03.584 INFO:teuthology.orchestra.run.smithi018.stdout:true 2024-04-12T21:14:03.729 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:03.732+0000 7f97f5000700 1 -- 172.21.15.18:0/725040522 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f97f00fde80 msgr2=0x7f97f00fe250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:14:03.729 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:03.732+0000 7f97f5000700 1 --2- 172.21.15.18:0/725040522 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f97f00fde80 0x7f97f00fe250 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f97e0004940 tx=0x7f97e0009e30 comp rx=0 tx=0).stop 2024-04-12T21:14:03.729 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:03.732+0000 7f97f5000700 1 -- 172.21.15.18:0/725040522 shutdown_connections 2024-04-12T21:14:03.729 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:03.732+0000 7f97f5000700 1 --2- 172.21.15.18:0/725040522 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f97f00fe820 0x7f97f0106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:14:03.729 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:03.732+0000 7f97f5000700 1 --2- 172.21.15.18:0/725040522 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f97f00fde80 0x7f97f00fe250 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:14:03.729 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:03.732+0000 7f97f5000700 1 -- 172.21.15.18:0/725040522 >> 172.21.15.18:0/725040522 conn(0x7f97f00f78f0 msgr2=0x7f97f00f9d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:14:03.729 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:03.732+0000 7f97f5000700 1 -- 172.21.15.18:0/725040522 shutdown_connections 2024-04-12T21:14:03.729 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:03.732+0000 7f97f5000700 1 -- 172.21.15.18:0/725040522 wait complete. 2024-04-12T21:14:03.731 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:03.733+0000 7f97f5000700 1 Processor -- start 2024-04-12T21:14:03.732 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:03.733+0000 7f97f5000700 1 -- start start 2024-04-12T21:14:03.732 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:03.734+0000 7f97f5000700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f97f00fde80 0x7f97f00fbcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:14:03.732 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:03.734+0000 7f97f5000700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f97f00fe820 0x7f97f00fc1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:14:03.732 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:03.734+0000 7f97f5000700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97f00fdc00 con 0x7f97f00fde80 2024-04-12T21:14:03.732 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:03.734+0000 7f97f5000700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97f00fc730 con 0x7f97f00fe820 2024-04-12T21:14:03.732 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:03.734+0000 7f97ede00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f97f00fde80 0x7f97f00fbcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:14:03.732 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:03.734+0000 7f97ede00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f97f00fde80 0x7f97f00fbcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:55654/0 (socket says 172.21.15.18:55654) 2024-04-12T21:14:03.732 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:03.734+0000 7f97ede00700 1 -- 172.21.15.18:0/1399709596 learned_addr learned my addr 172.21.15.18:0/1399709596 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:14:03.732 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:03.734+0000 7f97ed400700 1 --2- 172.21.15.18:0/1399709596 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f97f00fe820 0x7f97f00fc1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:14:03.732 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:03.735+0000 7f97ede00700 1 -- 172.21.15.18:0/1399709596 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f97f00fe820 msgr2=0x7f97f00fc1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:14:03.732 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:03.735+0000 7f97ede00700 1 --2- 172.21.15.18:0/1399709596 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f97f00fe820 0x7f97f00fc1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:14:03.732 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:03.735+0000 7f97ede00700 1 -- 172.21.15.18:0/1399709596 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97e0005040 con 0x7f97f00fde80 2024-04-12T21:14:03.733 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:03.735+0000 7f97ed400700 1 --2- 172.21.15.18:0/1399709596 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f97f00fe820 0x7f97f00fc1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-12T21:14:03.733 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:03.735+0000 7f97ede00700 1 --2- 172.21.15.18:0/1399709596 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f97f00fde80 0x7f97f00fbcb0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f97e0000c00 tx=0x7f97e0009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:14:03.733 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:03.735+0000 7f97e6000700 1 -- 172.21.15.18:0/1399709596 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f97e0021070 con 0x7f97f00fde80 2024-04-12T21:14:03.733 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:03.735+0000 7f97e6000700 1 -- 172.21.15.18:0/1399709596 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f97e0013cd0 con 0x7f97f00fde80 2024-04-12T21:14:03.733 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:03.735+0000 7f97f5000700 1 -- 172.21.15.18:0/1399709596 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f97f00fc9b0 con 0x7f97f00fde80 2024-04-12T21:14:03.733 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:03.735+0000 7f97e6000700 1 -- 172.21.15.18:0/1399709596 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f97e000e070 con 0x7f97f00fde80 2024-04-12T21:14:03.733 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:03.736+0000 7f97f5000700 1 -- 172.21.15.18:0/1399709596 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f97f0119f60 con 0x7f97f00fde80 2024-04-12T21:14:03.734 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:03.737+0000 7f97f5000700 1 -- 172.21.15.18:0/1399709596 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f97f0066af0 con 0x7f97f00fde80 2024-04-12T21:14:03.734 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:03.738+0000 7f97e6000700 1 -- 172.21.15.18:0/1399709596 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7f97e001ca60 con 0x7f97f00fde80 2024-04-12T21:14:03.735 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:03.738+0000 7f97e6000700 1 --2- 172.21.15.18:0/1399709596 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f97d406ec00 0x7f97d40710b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:14:03.735 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:03.739+0000 7f97e6000700 1 -- 172.21.15.18:0/1399709596 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 6757+0+0 (secure 0 0 0) 0x7f97e008fe70 con 0x7f97f00fde80 2024-04-12T21:14:03.735 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:03.739+0000 7f97ed400700 1 --2- 172.21.15.18:0/1399709596 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f97d406ec00 0x7f97d40710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:14:03.736 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:03.740+0000 7f97ed400700 1 --2- 172.21.15.18:0/1399709596 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f97d406ec00 0x7f97d40710b0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f97dc004600 tx=0x7f97dc00b040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:14:03.740 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:03.743+0000 7f97e6000700 1 -- 172.21.15.18:0/1399709596 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f97e0058af0 con 0x7f97f00fde80 2024-04-12T21:14:04.000 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:04.003+0000 7f97f5000700 1 -- 172.21.15.18:0/1399709596 --> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f97f0102070 con 0x7f97d406ec00 2024-04-12T21:14:04.002 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:04.006+0000 7f97e6000700 1 -- 172.21.15.18:0/1399709596 <== mgr.34102 v2:172.21.15.18:6800/3812891919 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f97f0102070 con 0x7f97d406ec00 2024-04-12T21:14:04.006 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:04.009+0000 7f97f5000700 1 -- 172.21.15.18:0/1399709596 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f97d406ec00 msgr2=0x7f97d40710b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:14:04.006 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:04.009+0000 7f97f5000700 1 --2- 172.21.15.18:0/1399709596 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f97d406ec00 0x7f97d40710b0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f97dc004600 tx=0x7f97dc00b040 comp rx=0 tx=0).stop 2024-04-12T21:14:04.006 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:04.009+0000 7f97f5000700 1 -- 172.21.15.18:0/1399709596 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f97f00fde80 msgr2=0x7f97f00fbcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:14:04.006 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:04.009+0000 7f97f5000700 1 --2- 172.21.15.18:0/1399709596 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f97f00fde80 0x7f97f00fbcb0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f97e0000c00 tx=0x7f97e0009e30 comp rx=0 tx=0).stop 2024-04-12T21:14:04.006 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:04.010+0000 7f97f5000700 1 -- 172.21.15.18:0/1399709596 shutdown_connections 2024-04-12T21:14:04.006 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:04.010+0000 7f97f5000700 1 --2- 172.21.15.18:0/1399709596 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f97d406ec00 0x7f97d40710b0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:14:04.006 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:04.010+0000 7f97f5000700 1 --2- 172.21.15.18:0/1399709596 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f97f00fde80 0x7f97f00fbcb0 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:14:04.006 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:04.010+0000 7f97f5000700 1 --2- 172.21.15.18:0/1399709596 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f97f00fe820 0x7f97f00fc1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:14:04.006 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:04.010+0000 7f97f5000700 1 -- 172.21.15.18:0/1399709596 >> 172.21.15.18:0/1399709596 conn(0x7f97f00f78f0 msgr2=0x7f97f0103660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:14:04.007 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:04.010+0000 7f97f5000700 1 -- 172.21.15.18:0/1399709596 shutdown_connections 2024-04-12T21:14:04.007 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:04.010+0000 7f97f5000700 1 -- 172.21.15.18:0/1399709596 wait complete. 2024-04-12T21:14:04.168 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:04.171+0000 7f67ca600700 1 -- 172.21.15.18:0/755977359 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f67c40fe850 msgr2=0x7f67c4106f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:14:04.168 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:04.171+0000 7f67ca600700 1 --2- 172.21.15.18:0/755977359 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f67c40fe850 0x7f67c4106f20 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f67b0004970 tx=0x7f67b0009e30 comp rx=0 tx=0).stop 2024-04-12T21:14:04.168 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:04.171+0000 7f67ca600700 1 -- 172.21.15.18:0/755977359 shutdown_connections 2024-04-12T21:14:04.168 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:04.171+0000 7f67ca600700 1 --2- 172.21.15.18:0/755977359 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f67c40fe850 0x7f67c4106f20 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:14:04.168 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:04.171+0000 7f67ca600700 1 --2- 172.21.15.18:0/755977359 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f67c40fdeb0 0x7f67c40fe280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:14:04.168 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:04.171+0000 7f67ca600700 1 -- 172.21.15.18:0/755977359 >> 172.21.15.18:0/755977359 conn(0x7f67c4072bc0 msgr2=0x7f67c4072fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:14:04.169 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:04.172+0000 7f67ca600700 1 -- 172.21.15.18:0/755977359 shutdown_connections 2024-04-12T21:14:04.169 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:04.172+0000 7f67ca600700 1 -- 172.21.15.18:0/755977359 wait complete. 2024-04-12T21:14:04.170 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:04.172+0000 7f67ca600700 1 Processor -- start 2024-04-12T21:14:04.170 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:04.173+0000 7f67ca600700 1 -- start start 2024-04-12T21:14:04.170 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:04.173+0000 7f67ca600700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f67c40fdeb0 0x7f67c4115ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:14:04.170 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:04.173+0000 7f67ca600700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f67c40fe850 0x7f67c4116220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:14:04.171 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:04.173+0000 7f67ca600700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67c4116870 con 0x7f67c40fe850 2024-04-12T21:14:04.171 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:04.173+0000 7f67ca600700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67c410fc40 con 0x7f67c40fdeb0 2024-04-12T21:14:04.171 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:04.173+0000 7f67c2a00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f67c40fe850 0x7f67c4116220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:14:04.171 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:04.173+0000 7f67c2a00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f67c40fe850 0x7f67c4116220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:55682/0 (socket says 172.21.15.18:55682) 2024-04-12T21:14:04.171 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:04.173+0000 7f67c2a00700 1 -- 172.21.15.18:0/255330232 learned_addr learned my addr 172.21.15.18:0/255330232 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:14:04.171 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:04.174+0000 7f67c3400700 1 --2- 172.21.15.18:0/255330232 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f67c40fdeb0 0x7f67c4115ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:14:04.171 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:04.174+0000 7f67c2a00700 1 -- 172.21.15.18:0/255330232 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f67c40fdeb0 msgr2=0x7f67c4115ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:14:04.171 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:04.174+0000 7f67c2a00700 1 --2- 172.21.15.18:0/255330232 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f67c40fdeb0 0x7f67c4115ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:14:04.171 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:04.174+0000 7f67c2a00700 1 -- 172.21.15.18:0/255330232 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67b0005040 con 0x7f67c40fe850 2024-04-12T21:14:04.171 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:04.174+0000 7f67c2a00700 1 --2- 172.21.15.18:0/255330232 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f67c40fe850 0x7f67c4116220 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f67b0004f50 tx=0x7f67b0009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:14:04.172 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:04.175+0000 7f67bb400700 1 -- 172.21.15.18:0/255330232 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f67b000e040 con 0x7f67c40fe850 2024-04-12T21:14:04.172 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:04.175+0000 7f67ca600700 1 -- 172.21.15.18:0/255330232 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f67c410fec0 con 0x7f67c40fe850 2024-04-12T21:14:04.172 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:04.175+0000 7f67bb400700 1 -- 172.21.15.18:0/255330232 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f67b000bc10 con 0x7f67c40fe850 2024-04-12T21:14:04.172 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:04.175+0000 7f67bb400700 1 -- 172.21.15.18:0/255330232 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f67b001c630 con 0x7f67c40fe850 2024-04-12T21:14:04.172 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:04.175+0000 7f67ca600700 1 -- 172.21.15.18:0/255330232 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f67c4110400 con 0x7f67c40fe850 2024-04-12T21:14:04.172 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:04.176+0000 7f67ca600700 1 -- 172.21.15.18:0/255330232 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f67c41069f0 con 0x7f67c40fe850 2024-04-12T21:14:04.173 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:04.177+0000 7f67bb400700 1 -- 172.21.15.18:0/255330232 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7f67b0013420 con 0x7f67c40fe850 2024-04-12T21:14:04.174 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:04.178+0000 7f67bb400700 1 --2- 172.21.15.18:0/255330232 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f67a806ec10 0x7f67a80710c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:14:04.174 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:04.178+0000 7f67bb400700 1 -- 172.21.15.18:0/255330232 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 6757+0+0 (secure 0 0 0) 0x7f67b008fc60 con 0x7f67c40fe850 2024-04-12T21:14:04.174 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:04.178+0000 7f67c3400700 1 --2- 172.21.15.18:0/255330232 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f67a806ec10 0x7f67a80710c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:14:04.175 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:04.179+0000 7f67c3400700 1 --2- 172.21.15.18:0/255330232 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f67a806ec10 0x7f67a80710c0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f67b4004990 tx=0x7f67b4009ee0 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:14:04.179 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:04.183+0000 7f67bb400700 1 -- 172.21.15.18:0/255330232 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f67b005c490 con 0x7f67c40fe850 2024-04-12T21:14:04.350 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:14:04 smithi018 ceph-mon[112590]: from='client.34780 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:14:04.420 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:04.424+0000 7f67ca600700 1 -- 172.21.15.18:0/255330232 --> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f67c41113a0 con 0x7f67a806ec10 2024-04-12T21:14:04.450 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:04.454+0000 7f67bb400700 1 -- 172.21.15.18:0/255330232 <== mgr.34102 v2:172.21.15.18:6800/3812891919 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7f67c41113a0 con 0x7f67a806ec10 2024-04-12T21:14:04.450 INFO:teuthology.orchestra.run.smithi018.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-12T21:14:04.451 INFO:teuthology.orchestra.run.smithi018.stdout:alertmanager.smithi018 smithi018 *:9093,9094 running (14m) 6m ago 29m 14.7M - 0.25.0 c8568f914cd2 c332090068e8 2024-04-12T21:14:04.451 INFO:teuthology.orchestra.run.smithi018.stdout:crash.smithi018 smithi018 running (11m) 6m ago 29m 6665k - 19.0.0-2879-g324baf8d 8717720dc5e5 b7f80d1f7982 2024-04-12T21:14:04.451 INFO:teuthology.orchestra.run.smithi018.stdout:crash.smithi186 smithi186 running (10m) 7m ago 28m 6660k - 19.0.0-2879-g324baf8d 8717720dc5e5 a4f55c81b623 2024-04-12T21:14:04.451 INFO:teuthology.orchestra.run.smithi018.stdout:grafana.smithi018 smithi018 *:3000 running (14m) 6m ago 28m 74.8M - 9.4.12 2bacad6d85d8 b08e3bf176dd 2024-04-12T21:14:04.451 INFO:teuthology.orchestra.run.smithi018.stdout:mds.cephfs.smithi018.acqbsc smithi018 running (6m) 6m ago 22m 11.1M - 19.0.0-2879-g324baf8d 8717720dc5e5 021f99ab24ae 2024-04-12T21:14:04.451 INFO:teuthology.orchestra.run.smithi018.stdout:mds.cephfs.smithi018.szjluo smithi018 running (6m) 6m ago 21m 10.9M - 19.0.0-2879-g324baf8d 8717720dc5e5 0003117479d4 2024-04-12T21:14:04.451 INFO:teuthology.orchestra.run.smithi018.stdout:mds.cephfs.smithi186.jdllyn smithi186 running (21m) 7m ago 21m 15.6M - 17.2.7-827-gd3b5060a 21de9b7369ad 2b2bc46bd29a 2024-04-12T21:14:04.451 INFO:teuthology.orchestra.run.smithi018.stdout:mds.cephfs.smithi186.wjnafs smithi186 running (21m) 7m ago 21m 15.4M - 17.2.7-827-gd3b5060a 21de9b7369ad 74b115d6d6fd 2024-04-12T21:14:04.451 INFO:teuthology.orchestra.run.smithi018.stdout:mgr.smithi018.xjzwga smithi018 *:8443,9283,8765 running (17m) 6m ago 30m 509M - 19.0.0-2879-g324baf8d 8717720dc5e5 56088305eab7 2024-04-12T21:14:04.451 INFO:teuthology.orchestra.run.smithi018.stdout:mgr.smithi186.kqcfxk smithi186 *:8443,9283,8765 running (16m) 7m ago 27m 452M - 19.0.0-2879-g324baf8d 8717720dc5e5 09cd766e5e0d 2024-04-12T21:14:04.451 INFO:teuthology.orchestra.run.smithi018.stdout:mon.smithi018 smithi018 running (12m) 6m ago 31m 57.0M 2048M 19.0.0-2879-g324baf8d 8717720dc5e5 df3e201dfed6 2024-04-12T21:14:04.451 INFO:teuthology.orchestra.run.smithi018.stdout:mon.smithi186 smithi186 running (11m) 7m ago 27m 37.7M 2048M 19.0.0-2879-g324baf8d 8717720dc5e5 c77498da9b2b 2024-04-12T21:14:04.451 INFO:teuthology.orchestra.run.smithi018.stdout:node-exporter.smithi018 smithi018 *:9100 running (15m) 6m ago 29m 7780k - 1.5.0 0da6a335fe13 326c3d9c6847 2024-04-12T21:14:04.451 INFO:teuthology.orchestra.run.smithi018.stdout:node-exporter.smithi186 smithi186 *:9100 running (15m) 7m ago 27m 7826k - 1.5.0 0da6a335fe13 ec2112baeecf 2024-04-12T21:14:04.451 INFO:teuthology.orchestra.run.smithi018.stdout:osd.0 smithi018 running (10m) 6m ago 26m 121M 1559M 19.0.0-2879-g324baf8d 8717720dc5e5 60b1273ca047 2024-04-12T21:14:04.451 INFO:teuthology.orchestra.run.smithi018.stdout:osd.1 smithi018 running (9m) 6m ago 26m 91.3M 1559M 19.0.0-2879-g324baf8d 8717720dc5e5 170ce1e5fb8d 2024-04-12T21:14:04.452 INFO:teuthology.orchestra.run.smithi018.stdout:osd.2 smithi018 running (9m) 6m ago 25m 90.3M 1559M 19.0.0-2879-g324baf8d 8717720dc5e5 c2cbc6aa2839 2024-04-12T21:14:04.452 INFO:teuthology.orchestra.run.smithi018.stdout:osd.3 smithi186 running (8m) 7m ago 25m 96.6M 2581M 19.0.0-2879-g324baf8d 8717720dc5e5 0138e8262e4b 2024-04-12T21:14:04.452 INFO:teuthology.orchestra.run.smithi018.stdout:osd.4 smithi186 running (8m) 7m ago 24m 87.2M 2581M 19.0.0-2879-g324baf8d 8717720dc5e5 b20cb63b25c9 2024-04-12T21:14:04.452 INFO:teuthology.orchestra.run.smithi018.stdout:osd.5 smithi186 running (7m) 7m ago 23m 11.1M 2581M 19.0.0-2879-g324baf8d 8717720dc5e5 2513bea2610e 2024-04-12T21:14:04.452 INFO:teuthology.orchestra.run.smithi018.stdout:prometheus.smithi018 smithi018 *:9095 running (15m) 6m ago 28m 60.6M - 2.43.0 a07b618ecd1d f58841127e21 2024-04-12T21:14:04.452 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:04.456+0000 7f67ca600700 1 -- 172.21.15.18:0/255330232 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f67a806ec10 msgr2=0x7f67a80710c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:14:04.452 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:04.456+0000 7f67ca600700 1 --2- 172.21.15.18:0/255330232 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f67a806ec10 0x7f67a80710c0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f67b4004990 tx=0x7f67b4009ee0 comp rx=0 tx=0).stop 2024-04-12T21:14:04.453 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:04.456+0000 7f67ca600700 1 -- 172.21.15.18:0/255330232 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f67c40fe850 msgr2=0x7f67c4116220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:14:04.453 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:04.456+0000 7f67ca600700 1 --2- 172.21.15.18:0/255330232 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f67c40fe850 0x7f67c4116220 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f67b0004f50 tx=0x7f67b0009e30 comp rx=0 tx=0).stop 2024-04-12T21:14:04.453 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:04.457+0000 7f67ca600700 1 -- 172.21.15.18:0/255330232 shutdown_connections 2024-04-12T21:14:04.453 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:04.457+0000 7f67ca600700 1 --2- 172.21.15.18:0/255330232 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f67a806ec10 0x7f67a80710c0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:14:04.454 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:04.457+0000 7f67ca600700 1 --2- 172.21.15.18:0/255330232 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f67c40fdeb0 0x7f67c4115ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:14:04.454 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:04.457+0000 7f67ca600700 1 --2- 172.21.15.18:0/255330232 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f67c40fe850 0x7f67c4116220 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:14:04.454 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:04.457+0000 7f67ca600700 1 -- 172.21.15.18:0/255330232 >> 172.21.15.18:0/255330232 conn(0x7f67c4072bc0 msgr2=0x7f67c40fb580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:14:04.454 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:04.457+0000 7f67ca600700 1 -- 172.21.15.18:0/255330232 shutdown_connections 2024-04-12T21:14:04.454 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:04.457+0000 7f67ca600700 1 -- 172.21.15.18:0/255330232 wait complete. 2024-04-12T21:14:04.528 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:14:04 smithi186 ceph-mon[86379]: from='client.34780 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:14:04.605 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:04.608+0000 7f45c1c00700 1 -- 172.21.15.18:0/3762216061 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f45bc109430 msgr2=0x7f45bc109800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:14:04.605 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:04.608+0000 7f45c1c00700 1 --2- 172.21.15.18:0/3762216061 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f45bc109430 0x7f45bc109800 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f45ac004940 tx=0x7f45ac009e30 comp rx=0 tx=0).stop 2024-04-12T21:14:04.605 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:04.609+0000 7f45c1c00700 1 -- 172.21.15.18:0/3762216061 shutdown_connections 2024-04-12T21:14:04.606 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:04.609+0000 7f45c1c00700 1 --2- 172.21.15.18:0/3762216061 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f45bc109dd0 0x7f45bc103440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:14:04.606 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:04.609+0000 7f45c1c00700 1 --2- 172.21.15.18:0/3762216061 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f45bc109430 0x7f45bc109800 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:14:04.606 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:04.609+0000 7f45c1c00700 1 -- 172.21.15.18:0/3762216061 >> 172.21.15.18:0/3762216061 conn(0x7f45bc072c00 msgr2=0x7f45bc075010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:14:04.606 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:04.609+0000 7f45c1c00700 1 -- 172.21.15.18:0/3762216061 shutdown_connections 2024-04-12T21:14:04.606 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:04.609+0000 7f45c1c00700 1 -- 172.21.15.18:0/3762216061 wait complete. 2024-04-12T21:14:04.608 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:04.610+0000 7f45c1c00700 1 Processor -- start 2024-04-12T21:14:04.608 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:04.610+0000 7f45c1c00700 1 -- start start 2024-04-12T21:14:04.608 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:04.610+0000 7f45c1c00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f45bc109430 0x7f45bc119f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:14:04.608 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:04.610+0000 7f45c1c00700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f45bc109dd0 0x7f45bc11a4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:14:04.608 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:04.610+0000 7f45c1c00700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45bc11ab20 con 0x7f45bc109430 2024-04-12T21:14:04.608 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:04.610+0000 7f45c1c00700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45bc114100 con 0x7f45bc109dd0 2024-04-12T21:14:04.608 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:04.610+0000 7f45baa00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f45bc109430 0x7f45bc119f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:14:04.608 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:04.610+0000 7f45baa00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f45bc109430 0x7f45bc119f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:55694/0 (socket says 172.21.15.18:55694) 2024-04-12T21:14:04.608 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:04.610+0000 7f45baa00700 1 -- 172.21.15.18:0/750189005 learned_addr learned my addr 172.21.15.18:0/750189005 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:14:04.608 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:04.611+0000 7f45ba000700 1 --2- 172.21.15.18:0/750189005 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f45bc109dd0 0x7f45bc11a4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:14:04.609 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:04.611+0000 7f45baa00700 1 -- 172.21.15.18:0/750189005 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f45bc109dd0 msgr2=0x7f45bc11a4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:14:04.609 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:04.611+0000 7f45baa00700 1 --2- 172.21.15.18:0/750189005 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f45bc109dd0 0x7f45bc11a4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:14:04.609 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:04.611+0000 7f45baa00700 1 -- 172.21.15.18:0/750189005 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45ac005040 con 0x7f45bc109430 2024-04-12T21:14:04.609 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:04.611+0000 7f45ba000700 1 --2- 172.21.15.18:0/750189005 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f45bc109dd0 0x7f45bc11a4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-12T21:14:04.609 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:04.611+0000 7f45baa00700 1 --2- 172.21.15.18:0/750189005 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f45bc109430 0x7f45bc119f90 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f45ac00b8b0 tx=0x7f45ac009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:14:04.609 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:04.611+0000 7f45b2a00700 1 -- 172.21.15.18:0/750189005 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f45ac00e040 con 0x7f45bc109430 2024-04-12T21:14:04.609 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:04.611+0000 7f45c1c00700 1 -- 172.21.15.18:0/750189005 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f45bc114380 con 0x7f45bc109430 2024-04-12T21:14:04.609 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:04.611+0000 7f45b2a00700 1 -- 172.21.15.18:0/750189005 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f45ac004290 con 0x7f45bc109430 2024-04-12T21:14:04.609 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:04.612+0000 7f45c1c00700 1 -- 172.21.15.18:0/750189005 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f45bc114940 con 0x7f45bc109430 2024-04-12T21:14:04.609 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:04.612+0000 7f45b2a00700 1 -- 172.21.15.18:0/750189005 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f45ac025400 con 0x7f45bc109430 2024-04-12T21:14:04.610 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:04.613+0000 7f45c1c00700 1 -- 172.21.15.18:0/750189005 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f45bc104b80 con 0x7f45bc109430 2024-04-12T21:14:04.610 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:04.614+0000 7f45b2a00700 1 -- 172.21.15.18:0/750189005 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7f45ac0137b0 con 0x7f45bc109430 2024-04-12T21:14:04.611 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:04.615+0000 7f45b2a00700 1 --2- 172.21.15.18:0/750189005 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f45a006ec00 0x7f45a00710b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:14:04.611 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:04.615+0000 7f45b2a00700 1 -- 172.21.15.18:0/750189005 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 6757+0+0 (secure 0 0 0) 0x7f45ac08fcf0 con 0x7f45bc109430 2024-04-12T21:14:04.611 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:04.615+0000 7f45ba000700 1 --2- 172.21.15.18:0/750189005 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f45a006ec00 0x7f45a00710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:14:04.612 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:04.616+0000 7f45ba000700 1 --2- 172.21.15.18:0/750189005 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f45a006ec00 0x7f45a00710b0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f45a8007fd0 tx=0x7f45a8009040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:14:04.616 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:04.620+0000 7f45b2a00700 1 -- 172.21.15.18:0/750189005 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f45ac05c4a0 con 0x7f45bc109430 2024-04-12T21:14:04.944 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:04.948+0000 7f45c1c00700 1 -- 172.21.15.18:0/750189005 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f45bc1155e0 con 0x7f45bc109430 2024-04-12T21:14:04.946 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:04.949+0000 7f45b2a00700 1 -- 172.21.15.18:0/750189005 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+837 (secure 0 0 0) 0x7f45ac08f070 con 0x7f45bc109430 2024-04-12T21:14:04.946 INFO:teuthology.orchestra.run.smithi018.stdout:{ 2024-04-12T21:14:04.946 INFO:teuthology.orchestra.run.smithi018.stdout: "mon": { 2024-04-12T21:14:04.946 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 19.0.0-2879-g324baf8d (324baf8ddb91dea46cac50cb535c306115791355) squid (dev)": 2 2024-04-12T21:14:04.946 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-12T21:14:04.946 INFO:teuthology.orchestra.run.smithi018.stdout: "mgr": { 2024-04-12T21:14:04.946 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 19.0.0-2879-g324baf8d (324baf8ddb91dea46cac50cb535c306115791355) squid (dev)": 2 2024-04-12T21:14:04.946 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-12T21:14:04.947 INFO:teuthology.orchestra.run.smithi018.stdout: "osd": { 2024-04-12T21:14:04.947 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 19.0.0-2879-g324baf8d (324baf8ddb91dea46cac50cb535c306115791355) squid (dev)": 6 2024-04-12T21:14:04.947 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-12T21:14:04.947 INFO:teuthology.orchestra.run.smithi018.stdout: "mds": { 2024-04-12T21:14:04.947 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 17.2.7-827-gd3b5060a (d3b5060a17449edfae9385cdf792afec109fca5d) quincy (stable)": 2, 2024-04-12T21:14:04.947 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 19.0.0-2879-g324baf8d (324baf8ddb91dea46cac50cb535c306115791355) squid (dev)": 2 2024-04-12T21:14:04.947 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-12T21:14:04.947 INFO:teuthology.orchestra.run.smithi018.stdout: "overall": { 2024-04-12T21:14:04.947 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 17.2.7-827-gd3b5060a (d3b5060a17449edfae9385cdf792afec109fca5d) quincy (stable)": 2, 2024-04-12T21:14:04.947 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 19.0.0-2879-g324baf8d (324baf8ddb91dea46cac50cb535c306115791355) squid (dev)": 12 2024-04-12T21:14:04.947 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-04-12T21:14:04.947 INFO:teuthology.orchestra.run.smithi018.stdout:} 2024-04-12T21:14:04.948 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:04.952+0000 7f45c1c00700 1 -- 172.21.15.18:0/750189005 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f45a006ec00 msgr2=0x7f45a00710b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:14:04.949 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:04.952+0000 7f45c1c00700 1 --2- 172.21.15.18:0/750189005 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f45a006ec00 0x7f45a00710b0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f45a8007fd0 tx=0x7f45a8009040 comp rx=0 tx=0).stop 2024-04-12T21:14:04.949 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:04.952+0000 7f45c1c00700 1 -- 172.21.15.18:0/750189005 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f45bc109430 msgr2=0x7f45bc119f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:14:04.949 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:04.952+0000 7f45c1c00700 1 --2- 172.21.15.18:0/750189005 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f45bc109430 0x7f45bc119f90 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f45ac00b8b0 tx=0x7f45ac009e30 comp rx=0 tx=0).stop 2024-04-12T21:14:04.949 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:04.953+0000 7f45c1c00700 1 -- 172.21.15.18:0/750189005 shutdown_connections 2024-04-12T21:14:04.950 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:04.953+0000 7f45c1c00700 1 --2- 172.21.15.18:0/750189005 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f45a006ec00 0x7f45a00710b0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:14:04.950 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:04.953+0000 7f45c1c00700 1 --2- 172.21.15.18:0/750189005 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f45bc109430 0x7f45bc119f90 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:14:04.950 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:04.953+0000 7f45c1c00700 1 --2- 172.21.15.18:0/750189005 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f45bc109dd0 0x7f45bc11a4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:14:04.950 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:04.953+0000 7f45c1c00700 1 -- 172.21.15.18:0/750189005 >> 172.21.15.18:0/750189005 conn(0x7f45bc072c00 msgr2=0x7f45bc074c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:14:04.950 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:04.953+0000 7f45c1c00700 1 -- 172.21.15.18:0/750189005 shutdown_connections 2024-04-12T21:14:04.950 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:04.953+0000 7f45c1c00700 1 -- 172.21.15.18:0/750189005 wait complete. 2024-04-12T21:14:05.112 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:05.115+0000 7f37b0800700 1 -- 172.21.15.18:0/1152468076 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f37a80702e0 msgr2=0x7f37a80706b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:14:05.112 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:05.115+0000 7f37b0800700 1 --2- 172.21.15.18:0/1152468076 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f37a80702e0 0x7f37a80706b0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f379c004940 tx=0x7f379c009e30 comp rx=0 tx=0).stop 2024-04-12T21:14:05.112 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:05.116+0000 7f37b0800700 1 -- 172.21.15.18:0/1152468076 shutdown_connections 2024-04-12T21:14:05.113 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:05.116+0000 7f37b0800700 1 --2- 172.21.15.18:0/1152468076 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f37a8070bf0 0x7f37a8109190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:14:05.113 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:05.116+0000 7f37b0800700 1 --2- 172.21.15.18:0/1152468076 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f37a80702e0 0x7f37a80706b0 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:14:05.113 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:05.116+0000 7f37b0800700 1 -- 172.21.15.18:0/1152468076 >> 172.21.15.18:0/1152468076 conn(0x7f37a80f89b0 msgr2=0x7f37a80fadc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:14:05.113 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:05.116+0000 7f37b0800700 1 -- 172.21.15.18:0/1152468076 shutdown_connections 2024-04-12T21:14:05.113 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:05.116+0000 7f37b0800700 1 -- 172.21.15.18:0/1152468076 wait complete. 2024-04-12T21:14:05.114 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:05.117+0000 7f37b0800700 1 Processor -- start 2024-04-12T21:14:05.114 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:05.117+0000 7f37b0800700 1 -- start start 2024-04-12T21:14:05.114 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:05.117+0000 7f37b0800700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f37a80702e0 0x7f37a8115b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:14:05.114 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:05.117+0000 7f37b0800700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f37a8070bf0 0x7f37a81160c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:14:05.114 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:05.117+0000 7f37b0800700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37a81167a0 con 0x7f37a8070bf0 2024-04-12T21:14:05.114 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:05.117+0000 7f37b0800700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37a810fcf0 con 0x7f37a80702e0 2024-04-12T21:14:05.114 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:05.118+0000 7f37ad400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f37a8070bf0 0x7f37a81160c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:14:05.114 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:05.118+0000 7f37ad400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f37a8070bf0 0x7f37a81160c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:55706/0 (socket says 172.21.15.18:55706) 2024-04-12T21:14:05.115 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:05.118+0000 7f37ad400700 1 -- 172.21.15.18:0/3331585114 learned_addr learned my addr 172.21.15.18:0/3331585114 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:14:05.115 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:05.118+0000 7f37ad400700 1 -- 172.21.15.18:0/3331585114 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f37a80702e0 msgr2=0x7f37a8115b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:14:05.115 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:05.118+0000 7f37ad400700 1 --2- 172.21.15.18:0/3331585114 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f37a80702e0 0x7f37a8115b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:14:05.116 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:05.118+0000 7f37ad400700 1 -- 172.21.15.18:0/3331585114 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f379c005040 con 0x7f37a8070bf0 2024-04-12T21:14:05.116 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:05.119+0000 7f37ad400700 1 --2- 172.21.15.18:0/3331585114 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f37a8070bf0 0x7f37a81160c0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f379800b790 tx=0x7f3798009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:14:05.116 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:05.119+0000 7f37a2000700 1 -- 172.21.15.18:0/3331585114 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f379800f770 con 0x7f37a8070bf0 2024-04-12T21:14:05.116 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:05.119+0000 7f37a2000700 1 -- 172.21.15.18:0/3331585114 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f379800fda0 con 0x7f37a8070bf0 2024-04-12T21:14:05.116 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:05.119+0000 7f37b0800700 1 -- 172.21.15.18:0/3331585114 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f37a810ff70 con 0x7f37a8070bf0 2024-04-12T21:14:05.116 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:05.119+0000 7f37a2000700 1 -- 172.21.15.18:0/3331585114 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f379800d310 con 0x7f37a8070bf0 2024-04-12T21:14:05.116 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:05.119+0000 7f37b0800700 1 -- 172.21.15.18:0/3331585114 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f37a81104b0 con 0x7f37a8070bf0 2024-04-12T21:14:05.117 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:05.121+0000 7f37b0800700 1 -- 172.21.15.18:0/3331585114 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f37a8071a20 con 0x7f37a8070bf0 2024-04-12T21:14:05.118 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:05.122+0000 7f37a2000700 1 -- 172.21.15.18:0/3331585114 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7f3798010420 con 0x7f37a8070bf0 2024-04-12T21:14:05.120 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:05.123+0000 7f37a2000700 1 --2- 172.21.15.18:0/3331585114 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f379006ec00 0x7f37900710b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:14:05.120 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:05.124+0000 7f37a2000700 1 -- 172.21.15.18:0/3331585114 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 6757+0+0 (secure 0 0 0) 0x7f379808ea80 con 0x7f37a8070bf0 2024-04-12T21:14:05.120 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:05.124+0000 7f37ade00700 1 --2- 172.21.15.18:0/3331585114 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f379006ec00 0x7f37900710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:14:05.121 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:05.125+0000 7f37ade00700 1 --2- 172.21.15.18:0/3331585114 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f379006ec00 0x7f37900710b0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f379c004f90 tx=0x7f379c009e30 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:14:05.125 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:05.129+0000 7f37a2000700 1 -- 172.21.15.18:0/3331585114 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f379805b230 con 0x7f37a8070bf0 2024-04-12T21:14:05.350 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:14:05 smithi018 ceph-mon[112590]: from='client.34784 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:14:05.350 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:14:05 smithi018 ceph-mon[112590]: pgmap v428: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:14:05.350 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:14:05 smithi018 ceph-mon[112590]: from='client.34788 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:14:05.350 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:14:05 smithi018 ceph-mon[112590]: from='client.? 172.21.15.18:0/750189005' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T21:14:05.409 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:05.412+0000 7f37b0800700 1 -- 172.21.15.18:0/3331585114 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f37a804c710 con 0x7f37a8070bf0 2024-04-12T21:14:05.409 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:05.413+0000 7f37a2000700 1 -- 172.21.15.18:0/3331585114 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 29 v29) v1 ==== 76+0+1872 (secure 0 0 0) 0x7f379805adc0 con 0x7f37a8070bf0 2024-04-12T21:14:05.410 INFO:teuthology.orchestra.run.smithi018.stdout:e29 2024-04-12T21:14:05.410 INFO:teuthology.orchestra.run.smithi018.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-12T21:14:05.410 INFO:teuthology.orchestra.run.smithi018.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-12T21:14:05.410 INFO:teuthology.orchestra.run.smithi018.stdout:legacy client fscid: 1 2024-04-12T21:14:05.410 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-12T21:14:05.410 INFO:teuthology.orchestra.run.smithi018.stdout:Filesystem 'cephfs' (1) 2024-04-12T21:14:05.410 INFO:teuthology.orchestra.run.smithi018.stdout:fs_name cephfs 2024-04-12T21:14:05.410 INFO:teuthology.orchestra.run.smithi018.stdout:epoch 24 2024-04-12T21:14:05.410 INFO:teuthology.orchestra.run.smithi018.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-04-12T21:14:05.411 INFO:teuthology.orchestra.run.smithi018.stdout:created 2024-04-12T20:51:57.362850+0000 2024-04-12T21:14:05.411 INFO:teuthology.orchestra.run.smithi018.stdout:modified 2024-04-12T21:06:59.717190+0000 2024-04-12T21:14:05.411 INFO:teuthology.orchestra.run.smithi018.stdout:tableserver 0 2024-04-12T21:14:05.411 INFO:teuthology.orchestra.run.smithi018.stdout:root 0 2024-04-12T21:14:05.411 INFO:teuthology.orchestra.run.smithi018.stdout:session_timeout 60 2024-04-12T21:14:05.411 INFO:teuthology.orchestra.run.smithi018.stdout:session_autoclose 300 2024-04-12T21:14:05.411 INFO:teuthology.orchestra.run.smithi018.stdout:max_file_size 1099511627776 2024-04-12T21:14:05.411 INFO:teuthology.orchestra.run.smithi018.stdout:max_xattr_size 65536 2024-04-12T21:14:05.411 INFO:teuthology.orchestra.run.smithi018.stdout:required_client_features {} 2024-04-12T21:14:05.411 INFO:teuthology.orchestra.run.smithi018.stdout:last_failure 0 2024-04-12T21:14:05.411 INFO:teuthology.orchestra.run.smithi018.stdout:last_failure_osd_epoch 100 2024-04-12T21:14:05.411 INFO:teuthology.orchestra.run.smithi018.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-12T21:14:05.411 INFO:teuthology.orchestra.run.smithi018.stdout:max_mds 2 2024-04-12T21:14:05.411 INFO:teuthology.orchestra.run.smithi018.stdout:in 0,1 2024-04-12T21:14:05.411 INFO:teuthology.orchestra.run.smithi018.stdout:up {} 2024-04-12T21:14:05.411 INFO:teuthology.orchestra.run.smithi018.stdout:failed 0,1 2024-04-12T21:14:05.412 INFO:teuthology.orchestra.run.smithi018.stdout:damaged 2024-04-12T21:14:05.412 INFO:teuthology.orchestra.run.smithi018.stdout:stopped 2024-04-12T21:14:05.412 INFO:teuthology.orchestra.run.smithi018.stdout:data_pools [3] 2024-04-12T21:14:05.412 INFO:teuthology.orchestra.run.smithi018.stdout:metadata_pool 2 2024-04-12T21:14:05.412 INFO:teuthology.orchestra.run.smithi018.stdout:inline_data disabled 2024-04-12T21:14:05.412 INFO:teuthology.orchestra.run.smithi018.stdout:balancer 2024-04-12T21:14:05.412 INFO:teuthology.orchestra.run.smithi018.stdout:bal_rank_mask -1 2024-04-12T21:14:05.412 INFO:teuthology.orchestra.run.smithi018.stdout:standby_count_wanted 1 2024-04-12T21:14:05.412 INFO:teuthology.orchestra.run.smithi018.stdout:qdb_cluster leader: 0 members: 2024-04-12T21:14:05.412 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-12T21:14:05.412 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-12T21:14:05.412 INFO:teuthology.orchestra.run.smithi018.stdout:Standby daemons: 2024-04-12T21:14:05.412 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-12T21:14:05.412 INFO:teuthology.orchestra.run.smithi018.stdout:[mds.cephfs.smithi186.wjnafs{-1:24277} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.186:6826/438347207,v1:172.21.15.186:6827/438347207] compat {c=[1],r=[1],i=[7ff]}] 2024-04-12T21:14:05.412 INFO:teuthology.orchestra.run.smithi018.stdout:[mds.cephfs.smithi186.jdllyn{-1:24281} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.186:6824/1464359952,v1:172.21.15.186:6825/1464359952] compat {c=[1],r=[1],i=[7ff]}] 2024-04-12T21:14:05.412 INFO:teuthology.orchestra.run.smithi018.stdout:[mds.cephfs.smithi018.acqbsc{-1:34436} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.18:6828/755179764,v1:172.21.15.18:6829/755179764] compat {c=[1],r=[1],i=[1fff]}] 2024-04-12T21:14:05.413 INFO:teuthology.orchestra.run.smithi018.stdout:[mds.cephfs.smithi018.szjluo{-1:34468} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.18:6826/2407992653,v1:172.21.15.18:6827/2407992653] compat {c=[1],r=[1],i=[1fff]}] 2024-04-12T21:14:05.413 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:05.416+0000 7f37b0800700 1 -- 172.21.15.18:0/3331585114 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f379006ec00 msgr2=0x7f37900710b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:14:05.413 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:05.416+0000 7f37b0800700 1 --2- 172.21.15.18:0/3331585114 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f379006ec00 0x7f37900710b0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f379c004f90 tx=0x7f379c009e30 comp rx=0 tx=0).stop 2024-04-12T21:14:05.413 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:05.416+0000 7f37b0800700 1 -- 172.21.15.18:0/3331585114 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f37a8070bf0 msgr2=0x7f37a81160c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:14:05.413 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:05.416+0000 7f37b0800700 1 --2- 172.21.15.18:0/3331585114 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f37a8070bf0 0x7f37a81160c0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f379800b790 tx=0x7f3798009250 comp rx=0 tx=0).stop 2024-04-12T21:14:05.414 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:05.416+0000 7f37b0800700 1 -- 172.21.15.18:0/3331585114 shutdown_connections 2024-04-12T21:14:05.414 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:05.416+0000 7f37b0800700 1 --2- 172.21.15.18:0/3331585114 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f379006ec00 0x7f37900710b0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:14:05.414 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:05.416+0000 7f37b0800700 1 --2- 172.21.15.18:0/3331585114 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f37a80702e0 0x7f37a8115b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:14:05.414 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:05.416+0000 7f37b0800700 1 --2- 172.21.15.18:0/3331585114 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f37a8070bf0 0x7f37a81160c0 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:14:05.414 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:05.416+0000 7f37b0800700 1 -- 172.21.15.18:0/3331585114 >> 172.21.15.18:0/3331585114 conn(0x7f37a80f89b0 msgr2=0x7f37a80f9d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:14:05.414 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:05.416+0000 7f37b0800700 1 -- 172.21.15.18:0/3331585114 shutdown_connections 2024-04-12T21:14:05.414 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:05.416+0000 7f37b0800700 1 -- 172.21.15.18:0/3331585114 wait complete. 2024-04-12T21:14:05.414 INFO:teuthology.orchestra.run.smithi018.stderr:dumped fsmap epoch 29 2024-04-12T21:14:05.529 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:14:05 smithi186 ceph-mon[86379]: from='client.34784 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:14:05.529 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:14:05 smithi186 ceph-mon[86379]: pgmap v428: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:14:05.529 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:14:05 smithi186 ceph-mon[86379]: from='client.34788 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:14:05.529 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:14:05 smithi186 ceph-mon[86379]: from='client.? 172.21.15.18:0/750189005' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T21:14:05.578 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:05.581+0000 7f31e8600700 1 -- 172.21.15.18:0/910861297 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f31e0109dd0 msgr2=0x7f31e0103440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:14:05.578 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:05.581+0000 7f31e8600700 1 --2- 172.21.15.18:0/910861297 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f31e0109dd0 0x7f31e0103440 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f31cc004940 tx=0x7f31cc009e30 comp rx=0 tx=0).stop 2024-04-12T21:14:05.578 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:05.581+0000 7f31e8600700 1 -- 172.21.15.18:0/910861297 shutdown_connections 2024-04-12T21:14:05.578 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:05.581+0000 7f31e8600700 1 --2- 172.21.15.18:0/910861297 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f31e0109dd0 0x7f31e0103440 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:14:05.578 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:05.581+0000 7f31e8600700 1 --2- 172.21.15.18:0/910861297 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f31e0109430 0x7f31e0109800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:14:05.578 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:05.581+0000 7f31e8600700 1 -- 172.21.15.18:0/910861297 >> 172.21.15.18:0/910861297 conn(0x7f31e0072c00 msgr2=0x7f31e0075010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:14:05.578 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:05.581+0000 7f31e8600700 1 -- 172.21.15.18:0/910861297 shutdown_connections 2024-04-12T21:14:05.578 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:05.582+0000 7f31e8600700 1 -- 172.21.15.18:0/910861297 wait complete. 2024-04-12T21:14:05.579 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:05.582+0000 7f31e8600700 1 Processor -- start 2024-04-12T21:14:05.579 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:05.583+0000 7f31e8600700 1 -- start start 2024-04-12T21:14:05.579 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:05.583+0000 7f31e8600700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f31e0109430 0x7f31e0119f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:14:05.579 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:05.583+0000 7f31e8600700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f31e0109dd0 0x7f31e011a4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:14:05.579 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:05.583+0000 7f31e8600700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31e011ab10 con 0x7f31e0109430 2024-04-12T21:14:05.579 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:05.583+0000 7f31e8600700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31e01140f0 con 0x7f31e0109dd0 2024-04-12T21:14:05.580 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:05.583+0000 7f31e5c00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f31e0109430 0x7f31e0119f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:14:05.580 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:05.583+0000 7f31e5c00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f31e0109430 0x7f31e0119f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:55736/0 (socket says 172.21.15.18:55736) 2024-04-12T21:14:05.580 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:05.583+0000 7f31e5c00700 1 -- 172.21.15.18:0/579935836 learned_addr learned my addr 172.21.15.18:0/579935836 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:14:05.581 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:05.584+0000 7f31e5200700 1 --2- 172.21.15.18:0/579935836 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f31e0109dd0 0x7f31e011a4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:14:05.581 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:05.584+0000 7f31e5c00700 1 -- 172.21.15.18:0/579935836 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f31e0109dd0 msgr2=0x7f31e011a4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:14:05.581 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:05.584+0000 7f31e5c00700 1 --2- 172.21.15.18:0/579935836 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f31e0109dd0 0x7f31e011a4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:14:05.581 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:05.584+0000 7f31e5c00700 1 -- 172.21.15.18:0/579935836 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31cc005040 con 0x7f31e0109430 2024-04-12T21:14:05.581 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:05.584+0000 7f31e5c00700 1 --2- 172.21.15.18:0/579935836 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f31e0109430 0x7f31e0119f80 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f31d4002a50 tx=0x7f31d40046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:14:05.582 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:05.584+0000 7f31dd600700 1 -- 172.21.15.18:0/579935836 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f31d4007cc0 con 0x7f31e0109430 2024-04-12T21:14:05.582 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:05.585+0000 7f31dd600700 1 -- 172.21.15.18:0/579935836 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f31d400f040 con 0x7f31e0109430 2024-04-12T21:14:05.582 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:05.585+0000 7f31e8600700 1 -- 172.21.15.18:0/579935836 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f31e01143d0 con 0x7f31e0109430 2024-04-12T21:14:05.582 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:05.585+0000 7f31dd600700 1 -- 172.21.15.18:0/579935836 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f31d4009640 con 0x7f31e0109430 2024-04-12T21:14:05.582 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:05.585+0000 7f31e8600700 1 -- 172.21.15.18:0/579935836 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f31e0114990 con 0x7f31e0109430 2024-04-12T21:14:05.582 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:05.586+0000 7f31e8600700 1 -- 172.21.15.18:0/579935836 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f31e0104b80 con 0x7f31e0109430 2024-04-12T21:14:05.583 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:05.587+0000 7f31dd600700 1 -- 172.21.15.18:0/579935836 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7f31d40097a0 con 0x7f31e0109430 2024-04-12T21:14:05.584 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:05.588+0000 7f31dd600700 1 --2- 172.21.15.18:0/579935836 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f31c806ec00 0x7f31c80710b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:14:05.584 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:05.588+0000 7f31dd600700 1 -- 172.21.15.18:0/579935836 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 6757+0+0 (secure 0 0 0) 0x7f31d408e830 con 0x7f31e0109430 2024-04-12T21:14:05.584 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:05.588+0000 7f31e5200700 1 --2- 172.21.15.18:0/579935836 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f31c806ec00 0x7f31c80710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:14:05.585 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:05.589+0000 7f31e5200700 1 --2- 172.21.15.18:0/579935836 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f31c806ec00 0x7f31c80710b0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f31cc00b3b0 tx=0x7f31cc009e30 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:14:05.589 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:05.593+0000 7f31dd600700 1 -- 172.21.15.18:0/579935836 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f31d405af60 con 0x7f31e0109430 2024-04-12T21:14:05.844 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:05.848+0000 7f31e8600700 1 -- 172.21.15.18:0/579935836 --> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f31e0115640 con 0x7f31c806ec00 2024-04-12T21:14:05.846 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:05.850+0000 7f31dd600700 1 -- 172.21.15.18:0/579935836 <== mgr.34102 v2:172.21.15.18:6800/3812891919 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f31e0115640 con 0x7f31c806ec00 2024-04-12T21:14:05.847 INFO:teuthology.orchestra.run.smithi018.stdout:{ 2024-04-12T21:14:05.847 INFO:teuthology.orchestra.run.smithi018.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:f4452def3be4d350f4a8e5feaf7ebb1511be9298fe68478ba7a2d6fbd7d69a63", 2024-04-12T21:14:05.847 INFO:teuthology.orchestra.run.smithi018.stdout: "in_progress": true, 2024-04-12T21:14:05.847 INFO:teuthology.orchestra.run.smithi018.stdout: "which": "Upgrading all daemon types on all hosts", 2024-04-12T21:14:05.847 INFO:teuthology.orchestra.run.smithi018.stdout: "services_complete": [ 2024-04-12T21:14:05.847 INFO:teuthology.orchestra.run.smithi018.stdout: "crash", 2024-04-12T21:14:05.847 INFO:teuthology.orchestra.run.smithi018.stdout: "osd", 2024-04-12T21:14:05.847 INFO:teuthology.orchestra.run.smithi018.stdout: "mon", 2024-04-12T21:14:05.847 INFO:teuthology.orchestra.run.smithi018.stdout: "mgr" 2024-04-12T21:14:05.847 INFO:teuthology.orchestra.run.smithi018.stdout: ], 2024-04-12T21:14:05.847 INFO:teuthology.orchestra.run.smithi018.stdout: "progress": "14/21 daemons upgraded", 2024-04-12T21:14:05.847 INFO:teuthology.orchestra.run.smithi018.stdout: "message": "Currently upgrading mds daemons", 2024-04-12T21:14:05.847 INFO:teuthology.orchestra.run.smithi018.stdout: "is_paused": false 2024-04-12T21:14:05.847 INFO:teuthology.orchestra.run.smithi018.stdout:} 2024-04-12T21:14:05.850 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:05.853+0000 7f31e8600700 1 -- 172.21.15.18:0/579935836 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f31c806ec00 msgr2=0x7f31c80710b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:14:05.850 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:05.853+0000 7f31e8600700 1 --2- 172.21.15.18:0/579935836 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f31c806ec00 0x7f31c80710b0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f31cc00b3b0 tx=0x7f31cc009e30 comp rx=0 tx=0).stop 2024-04-12T21:14:05.850 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:05.853+0000 7f31e8600700 1 -- 172.21.15.18:0/579935836 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f31e0109430 msgr2=0x7f31e0119f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:14:05.850 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:05.853+0000 7f31e8600700 1 --2- 172.21.15.18:0/579935836 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f31e0109430 0x7f31e0119f80 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f31d4002a50 tx=0x7f31d40046b0 comp rx=0 tx=0).stop 2024-04-12T21:14:05.851 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:05.854+0000 7f31e8600700 1 -- 172.21.15.18:0/579935836 shutdown_connections 2024-04-12T21:14:05.851 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:05.854+0000 7f31e8600700 1 --2- 172.21.15.18:0/579935836 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f31c806ec00 0x7f31c80710b0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:14:05.851 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:05.854+0000 7f31e8600700 1 --2- 172.21.15.18:0/579935836 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f31e0109430 0x7f31e0119f80 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:14:05.851 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:05.854+0000 7f31e8600700 1 --2- 172.21.15.18:0/579935836 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f31e0109dd0 0x7f31e011a4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:14:05.851 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:05.854+0000 7f31e8600700 1 -- 172.21.15.18:0/579935836 >> 172.21.15.18:0/579935836 conn(0x7f31e0072c00 msgr2=0x7f31e0074c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:14:05.851 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:05.854+0000 7f31e8600700 1 -- 172.21.15.18:0/579935836 shutdown_connections 2024-04-12T21:14:05.851 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:05.854+0000 7f31e8600700 1 -- 172.21.15.18:0/579935836 wait complete. 2024-04-12T21:14:06.013 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:06.016+0000 7fd28e800700 1 -- 172.21.15.18:0/1739588363 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd288065630 msgr2=0x7fd288065a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:14:06.013 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:06.016+0000 7fd28e800700 1 --2- 172.21.15.18:0/1739588363 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd288065630 0x7fd288065a80 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7fd274004940 tx=0x7fd274009e30 comp rx=0 tx=0).stop 2024-04-12T21:14:06.013 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:06.016+0000 7fd28e800700 1 -- 172.21.15.18:0/1739588363 shutdown_connections 2024-04-12T21:14:06.013 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:06.016+0000 7fd28e800700 1 --2- 172.21.15.18:0/1739588363 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd288065630 0x7fd288065a80 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:14:06.013 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:06.016+0000 7fd28e800700 1 --2- 172.21.15.18:0/1739588363 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd288102ca0 0x7fd288103070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:14:06.013 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:06.016+0000 7fd28e800700 1 -- 172.21.15.18:0/1739588363 >> 172.21.15.18:0/1739588363 conn(0x7fd288071b70 msgr2=0x7fd288071f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:14:06.013 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:06.016+0000 7fd28e800700 1 -- 172.21.15.18:0/1739588363 shutdown_connections 2024-04-12T21:14:06.013 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:06.017+0000 7fd28e800700 1 -- 172.21.15.18:0/1739588363 wait complete. 2024-04-12T21:14:06.014 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:06.017+0000 7fd28e800700 1 Processor -- start 2024-04-12T21:14:06.014 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:06.018+0000 7fd28e800700 1 -- start start 2024-04-12T21:14:06.015 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:06.018+0000 7fd28e800700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd288065630 0x7fd2880fbc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:14:06.015 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:06.018+0000 7fd28e800700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd288102ca0 0x7fd2880fc160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:14:06.015 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:06.018+0000 7fd28e800700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2880fc6f0 con 0x7fd288065630 2024-04-12T21:14:06.015 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:06.018+0000 7fd28e800700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2880fc830 con 0x7fd288102ca0 2024-04-12T21:14:06.015 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:06.018+0000 7fd28d400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd288065630 0x7fd2880fbc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:14:06.015 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:06.018+0000 7fd28d400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd288065630 0x7fd2880fbc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:55750/0 (socket says 172.21.15.18:55750) 2024-04-12T21:14:06.015 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:06.018+0000 7fd28d400700 1 -- 172.21.15.18:0/2003325030 learned_addr learned my addr 172.21.15.18:0/2003325030 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:14:06.016 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:06.019+0000 7fd28ca00700 1 --2- 172.21.15.18:0/2003325030 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd288102ca0 0x7fd2880fc160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:14:06.016 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:06.019+0000 7fd28d400700 1 -- 172.21.15.18:0/2003325030 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd288102ca0 msgr2=0x7fd2880fc160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:14:06.016 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:06.019+0000 7fd28d400700 1 --2- 172.21.15.18:0/2003325030 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd288102ca0 0x7fd2880fc160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:14:06.016 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:06.019+0000 7fd28d400700 1 -- 172.21.15.18:0/2003325030 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd274005040 con 0x7fd288065630 2024-04-12T21:14:06.017 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:06.019+0000 7fd28d400700 1 --2- 172.21.15.18:0/2003325030 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd288065630 0x7fd2880fbc20 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7fd27c00c830 tx=0x7fd27c004470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:14:06.017 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:06.019+0000 7fd285600700 1 -- 172.21.15.18:0/2003325030 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd27c00a760 con 0x7fd288065630 2024-04-12T21:14:06.017 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:06.020+0000 7fd285600700 1 -- 172.21.15.18:0/2003325030 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd27c00ad90 con 0x7fd288065630 2024-04-12T21:14:06.017 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:06.020+0000 7fd28e800700 1 -- 172.21.15.18:0/2003325030 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd288119c90 con 0x7fd288065630 2024-04-12T21:14:06.017 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:06.020+0000 7fd285600700 1 -- 172.21.15.18:0/2003325030 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd27c0113a0 con 0x7fd288065630 2024-04-12T21:14:06.017 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:06.020+0000 7fd28e800700 1 -- 172.21.15.18:0/2003325030 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd28811a110 con 0x7fd288065630 2024-04-12T21:14:06.017 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:06.021+0000 7fd28e800700 1 -- 172.21.15.18:0/2003325030 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd2880fe380 con 0x7fd288065630 2024-04-12T21:14:06.018 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:06.022+0000 7fd285600700 1 -- 172.21.15.18:0/2003325030 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7fd27c012400 con 0x7fd288065630 2024-04-12T21:14:06.019 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:06.023+0000 7fd285600700 1 --2- 172.21.15.18:0/2003325030 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fd27006ec10 0x7fd2700710c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:14:06.019 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:06.023+0000 7fd285600700 1 -- 172.21.15.18:0/2003325030 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 6757+0+0 (secure 0 0 0) 0x7fd27c08ea40 con 0x7fd288065630 2024-04-12T21:14:06.020 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:06.023+0000 7fd28ca00700 1 --2- 172.21.15.18:0/2003325030 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fd27006ec10 0x7fd2700710c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:14:06.020 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:06.024+0000 7fd28ca00700 1 --2- 172.21.15.18:0/2003325030 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fd27006ec10 0x7fd2700710c0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fd27400b020 tx=0x7fd274009e30 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:14:06.024 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:06.028+0000 7fd285600700 1 -- 172.21.15.18:0/2003325030 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7fd27c0576c0 con 0x7fd288065630 2024-04-12T21:14:06.350 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:06.353+0000 7fd28e800700 1 -- 172.21.15.18:0/2003325030 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7fd28804bf60 con 0x7fd288065630 2024-04-12T21:14:06.350 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:06.354+0000 7fd285600700 1 -- 172.21.15.18:0/2003325030 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+233 (secure 0 0 0) 0x7fd27c05ad80 con 0x7fd288065630 2024-04-12T21:14:06.351 INFO:teuthology.orchestra.run.smithi018.stdout:HEALTH_ERR 1 filesystem is degraded; 1 filesystem is offline 2024-04-12T21:14:06.351 INFO:teuthology.orchestra.run.smithi018.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-04-12T21:14:06.351 INFO:teuthology.orchestra.run.smithi018.stdout: fs cephfs is degraded 2024-04-12T21:14:06.351 INFO:teuthology.orchestra.run.smithi018.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-04-12T21:14:06.351 INFO:teuthology.orchestra.run.smithi018.stdout: fs cephfs is offline because no MDS is active for it. 2024-04-12T21:14:06.353 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:06.357+0000 7fd28e800700 1 -- 172.21.15.18:0/2003325030 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fd27006ec10 msgr2=0x7fd2700710c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:14:06.353 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:06.357+0000 7fd28e800700 1 --2- 172.21.15.18:0/2003325030 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fd27006ec10 0x7fd2700710c0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fd27400b020 tx=0x7fd274009e30 comp rx=0 tx=0).stop 2024-04-12T21:14:06.353 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:06.357+0000 7fd28e800700 1 -- 172.21.15.18:0/2003325030 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd288065630 msgr2=0x7fd2880fbc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:14:06.354 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:06.357+0000 7fd28e800700 1 --2- 172.21.15.18:0/2003325030 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd288065630 0x7fd2880fbc20 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7fd27c00c830 tx=0x7fd27c004470 comp rx=0 tx=0).stop 2024-04-12T21:14:06.354 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:06.357+0000 7fd28e800700 1 -- 172.21.15.18:0/2003325030 shutdown_connections 2024-04-12T21:14:06.354 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:06.357+0000 7fd28e800700 1 --2- 172.21.15.18:0/2003325030 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fd27006ec10 0x7fd2700710c0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:14:06.354 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:06.357+0000 7fd28e800700 1 --2- 172.21.15.18:0/2003325030 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd288065630 0x7fd2880fbc20 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:14:06.354 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:06.357+0000 7fd28e800700 1 --2- 172.21.15.18:0/2003325030 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd288102ca0 0x7fd2880fc160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:14:06.354 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:06.357+0000 7fd28e800700 1 -- 172.21.15.18:0/2003325030 >> 172.21.15.18:0/2003325030 conn(0x7fd288071b70 msgr2=0x7fd2881007f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:14:06.354 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:06.357+0000 7fd28e800700 1 -- 172.21.15.18:0/2003325030 shutdown_connections 2024-04-12T21:14:06.354 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:06.357+0000 7fd28e800700 1 -- 172.21.15.18:0/2003325030 wait complete. 2024-04-12T21:14:06.528 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:14:06 smithi186 ceph-mon[86379]: from='client.? 172.21.15.18:0/3331585114' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-12T21:14:06.600 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:14:06 smithi018 ceph-mon[112590]: from='client.? 172.21.15.18:0/3331585114' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-12T21:14:07.528 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:14:07 smithi186 ceph-mon[86379]: from='client.34800 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:14:07.529 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:14:07 smithi186 ceph-mon[86379]: pgmap v429: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:14:07.529 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:14:07 smithi186 ceph-mon[86379]: from='client.? 172.21.15.18:0/2003325030' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-12T21:14:07.600 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:14:07 smithi018 ceph-mon[112590]: from='client.34800 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:14:07.600 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:14:07 smithi018 ceph-mon[112590]: pgmap v429: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:14:07.600 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:14:07 smithi018 ceph-mon[112590]: from='client.? 172.21.15.18:0/2003325030' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-12T21:14:09.529 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:14:09 smithi186 ceph-mon[86379]: pgmap v430: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:14:09.600 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:14:09 smithi018 ceph-mon[112590]: pgmap v430: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:14:10.501 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:14:10 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T21:14:10.528 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:14:10 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T21:14:11.528 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:14:11 smithi186 ceph-mon[86379]: pgmap v431: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:14:11.599 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:14:11 smithi018 ceph-mon[112590]: pgmap v431: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:14:13.529 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:14:13 smithi186 ceph-mon[86379]: pgmap v432: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:14:13.599 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:14:13 smithi018 ceph-mon[112590]: pgmap v432: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:14:15.529 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:14:15 smithi186 ceph-mon[86379]: pgmap v433: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:14:15.599 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:14:15 smithi018 ceph-mon[112590]: pgmap v433: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:14:17.528 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:14:17 smithi186 ceph-mon[86379]: pgmap v434: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:14:17.599 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:14:17 smithi018 ceph-mon[112590]: pgmap v434: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:14:19.528 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:14:19 smithi186 ceph-mon[86379]: pgmap v435: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:14:19.600 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:14:19 smithi018 ceph-mon[112590]: pgmap v435: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:14:21.529 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:14:21 smithi186 ceph-mon[86379]: pgmap v436: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:14:21.600 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:14:21 smithi018 ceph-mon[112590]: pgmap v436: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:14:23.529 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:14:23 smithi186 ceph-mon[86379]: pgmap v437: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:14:23.599 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:14:23 smithi018 ceph-mon[112590]: pgmap v437: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:14:25.529 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:14:25 smithi186 ceph-mon[86379]: pgmap v438: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:14:25.529 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:14:25 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T21:14:25.600 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:14:25 smithi018 ceph-mon[112590]: pgmap v438: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:14:25.600 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:14:25 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T21:14:27.528 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:14:27 smithi186 ceph-mon[86379]: pgmap v439: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:14:27.599 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:14:27 smithi018 ceph-mon[112590]: pgmap v439: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:14:29.528 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:14:29 smithi186 ceph-mon[86379]: pgmap v440: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:14:29.599 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:14:29 smithi018 ceph-mon[112590]: pgmap v440: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:14:31.528 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:14:31 smithi186 ceph-mon[86379]: pgmap v441: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:14:31.599 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:14:31 smithi018 ceph-mon[112590]: pgmap v441: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:14:33.529 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:14:33 smithi186 ceph-mon[86379]: pgmap v442: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:14:33.599 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:14:33 smithi018 ceph-mon[112590]: pgmap v442: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:14:35.528 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:14:35 smithi186 ceph-mon[86379]: pgmap v443: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:14:35.600 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:14:35 smithi018 ceph-mon[112590]: pgmap v443: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:14:36.528 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:36.532+0000 7f7150000700 1 -- 172.21.15.18:0/596537205 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f71480fc5b0 msgr2=0x7f7148100bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:14:36.528 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:36.532+0000 7f7150000700 1 --2- 172.21.15.18:0/596537205 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f71480fc5b0 0x7f7148100bd0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f7134004940 tx=0x7f7134009e30 comp rx=0 tx=0).stop 2024-04-12T21:14:36.528 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:36.532+0000 7f7150000700 1 -- 172.21.15.18:0/596537205 shutdown_connections 2024-04-12T21:14:36.528 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:36.532+0000 7f7150000700 1 --2- 172.21.15.18:0/596537205 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f71480fc5b0 0x7f7148100bd0 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:14:36.528 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:36.532+0000 7f7150000700 1 --2- 172.21.15.18:0/596537205 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f71480fbc10 0x7f71480fbfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:14:36.528 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:36.532+0000 7f7150000700 1 -- 172.21.15.18:0/596537205 >> 172.21.15.18:0/596537205 conn(0x7f71480f7870 msgr2=0x7f71480f9c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:14:36.529 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:36.532+0000 7f7150000700 1 -- 172.21.15.18:0/596537205 shutdown_connections 2024-04-12T21:14:36.529 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:36.533+0000 7f7150000700 1 -- 172.21.15.18:0/596537205 wait complete. 2024-04-12T21:14:36.529 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:36.534+0000 7f7150000700 1 Processor -- start 2024-04-12T21:14:36.529 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:36.534+0000 7f7150000700 1 -- start start 2024-04-12T21:14:36.530 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:36.535+0000 7f7150000700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f71480fbc10 0x7f714811a1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:14:36.530 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:36.535+0000 7f7150000700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f714811a6e0 0x7f7148114310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:14:36.531 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:36.535+0000 7f7150000700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f714811acf0 con 0x7f71480fbc10 2024-04-12T21:14:36.531 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:36.535+0000 7f7150000700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7148114850 con 0x7f714811a6e0 2024-04-12T21:14:36.531 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:36.535+0000 7f714d600700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f71480fbc10 0x7f714811a1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:14:36.531 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:36.535+0000 7f714d600700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f71480fbc10 0x7f714811a1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:35762/0 (socket says 172.21.15.18:35762) 2024-04-12T21:14:36.531 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:36.535+0000 7f714d600700 1 -- 172.21.15.18:0/970056288 learned_addr learned my addr 172.21.15.18:0/970056288 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:14:36.532 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:36.536+0000 7f714cc00700 1 --2- 172.21.15.18:0/970056288 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f714811a6e0 0x7f7148114310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:14:36.532 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:36.536+0000 7f714d600700 1 -- 172.21.15.18:0/970056288 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f714811a6e0 msgr2=0x7f7148114310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:14:36.532 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:36.536+0000 7f714d600700 1 --2- 172.21.15.18:0/970056288 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f714811a6e0 0x7f7148114310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:14:36.532 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:36.536+0000 7f714d600700 1 -- 172.21.15.18:0/970056288 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7134005040 con 0x7f71480fbc10 2024-04-12T21:14:36.532 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:36.536+0000 7f714d600700 1 --2- 172.21.15.18:0/970056288 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f71480fbc10 0x7f714811a1a0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f713c00b790 tx=0x7f713c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:14:36.532 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:36.536+0000 7f7145600700 1 -- 172.21.15.18:0/970056288 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f713c00f770 con 0x7f71480fbc10 2024-04-12T21:14:36.532 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:36.536+0000 7f7145600700 1 -- 172.21.15.18:0/970056288 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f713c00fda0 con 0x7f71480fbc10 2024-04-12T21:14:36.532 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:36.536+0000 7f7150000700 1 -- 172.21.15.18:0/970056288 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7148114b30 con 0x7f71480fbc10 2024-04-12T21:14:36.532 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:36.536+0000 7f7145600700 1 -- 172.21.15.18:0/970056288 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f713c00d310 con 0x7f71480fbc10 2024-04-12T21:14:36.533 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:36.537+0000 7f7150000700 1 -- 172.21.15.18:0/970056288 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7148115070 con 0x7f71480fbc10 2024-04-12T21:14:36.533 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:36.538+0000 7f7150000700 1 -- 172.21.15.18:0/970056288 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7148066b50 con 0x7f71480fbc10 2024-04-12T21:14:36.534 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:36.539+0000 7f7145600700 1 -- 172.21.15.18:0/970056288 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7f713c010420 con 0x7f71480fbc10 2024-04-12T21:14:36.535 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:36.540+0000 7f7145600700 1 --2- 172.21.15.18:0/970056288 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f713006ec00 0x7f71300710b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:14:36.535 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:36.540+0000 7f7145600700 1 -- 172.21.15.18:0/970056288 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 6757+0+0 (secure 0 0 0) 0x7f713c08ea80 con 0x7f71480fbc10 2024-04-12T21:14:36.535 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:36.540+0000 7f714cc00700 1 --2- 172.21.15.18:0/970056288 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f713006ec00 0x7f71300710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:14:36.536 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:36.541+0000 7f714cc00700 1 --2- 172.21.15.18:0/970056288 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f713006ec00 0x7f71300710b0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f7134000c00 tx=0x7f7134009e30 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:14:36.540 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:36.545+0000 7f7145600700 1 -- 172.21.15.18:0/970056288 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f713c05b230 con 0x7f71480fbc10 2024-04-12T21:14:36.801 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:36.805+0000 7f7150000700 1 -- 172.21.15.18:0/970056288 --> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f7148110640 con 0x7f713006ec00 2024-04-12T21:14:36.803 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:36.808+0000 7f7145600700 1 -- 172.21.15.18:0/970056288 <== mgr.34102 v2:172.21.15.18:6800/3812891919 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f7134007cc0 con 0x7f713006ec00 2024-04-12T21:14:36.806 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:36.810+0000 7f7150000700 1 -- 172.21.15.18:0/970056288 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f713006ec00 msgr2=0x7f71300710b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:14:36.806 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:36.810+0000 7f7150000700 1 --2- 172.21.15.18:0/970056288 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f713006ec00 0x7f71300710b0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f7134000c00 tx=0x7f7134009e30 comp rx=0 tx=0).stop 2024-04-12T21:14:36.806 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:36.810+0000 7f7150000700 1 -- 172.21.15.18:0/970056288 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f71480fbc10 msgr2=0x7f714811a1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:14:36.806 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:36.811+0000 7f7150000700 1 --2- 172.21.15.18:0/970056288 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f71480fbc10 0x7f714811a1a0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f713c00b790 tx=0x7f713c009250 comp rx=0 tx=0).stop 2024-04-12T21:14:36.806 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:36.811+0000 7f7150000700 1 -- 172.21.15.18:0/970056288 shutdown_connections 2024-04-12T21:14:36.806 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:36.811+0000 7f7150000700 1 --2- 172.21.15.18:0/970056288 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f713006ec00 0x7f71300710b0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:14:36.807 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:36.811+0000 7f7150000700 1 --2- 172.21.15.18:0/970056288 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f71480fbc10 0x7f714811a1a0 secure :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f713c00b790 tx=0x7f713c009250 comp rx=0 tx=0).stop 2024-04-12T21:14:36.807 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:36.811+0000 7f7150000700 1 --2- 172.21.15.18:0/970056288 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f714811a6e0 0x7f7148114310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:14:36.807 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:36.811+0000 7f7150000700 1 -- 172.21.15.18:0/970056288 >> 172.21.15.18:0/970056288 conn(0x7f71480f7870 msgr2=0x7f71480f8870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:14:36.807 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:36.811+0000 7f7150000700 1 -- 172.21.15.18:0/970056288 shutdown_connections 2024-04-12T21:14:36.807 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:36.811+0000 7f7150000700 1 -- 172.21.15.18:0/970056288 wait complete. 2024-04-12T21:14:36.823 INFO:teuthology.orchestra.run.smithi018.stdout:true 2024-04-12T21:14:36.968 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:36.972+0000 7f7ff1e00700 1 -- 172.21.15.18:0/2465347470 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7fec070360 msgr2=0x7f7fec070730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:14:36.968 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:36.972+0000 7f7ff1e00700 1 --2- 172.21.15.18:0/2465347470 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7fec070360 0x7f7fec070730 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f7fdc004970 tx=0x7f7fdc009e30 comp rx=0 tx=0).stop 2024-04-12T21:14:36.968 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:36.972+0000 7f7ff1e00700 1 -- 172.21.15.18:0/2465347470 shutdown_connections 2024-04-12T21:14:36.968 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:36.972+0000 7f7ff1e00700 1 --2- 172.21.15.18:0/2465347470 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7fec070c70 0x7f7fec1090f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:14:36.968 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:36.972+0000 7f7ff1e00700 1 --2- 172.21.15.18:0/2465347470 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7fec070360 0x7f7fec070730 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:14:36.968 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:36.972+0000 7f7ff1e00700 1 -- 172.21.15.18:0/2465347470 >> 172.21.15.18:0/2465347470 conn(0x7f7fec0f8970 msgr2=0x7f7fec0fad80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:14:36.968 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:36.973+0000 7f7ff1e00700 1 -- 172.21.15.18:0/2465347470 shutdown_connections 2024-04-12T21:14:36.969 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:36.973+0000 7f7ff1e00700 1 -- 172.21.15.18:0/2465347470 wait complete. 2024-04-12T21:14:36.969 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:36.974+0000 7f7ff1e00700 1 Processor -- start 2024-04-12T21:14:36.969 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:36.974+0000 7f7ff1e00700 1 -- start start 2024-04-12T21:14:36.970 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:36.974+0000 7f7ff1e00700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7fec070360 0x7f7fec115d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:14:36.970 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:36.974+0000 7f7ff1e00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7fec070c70 0x7f7fec116290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:14:36.970 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:36.974+0000 7f7ff1e00700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7fec1168e0 con 0x7f7fec070c70 2024-04-12T21:14:36.970 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:36.974+0000 7f7ff1e00700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7fec10fcb0 con 0x7f7fec070360 2024-04-12T21:14:36.970 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:36.975+0000 7f7fea000700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7fec070c70 0x7f7fec116290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:14:36.971 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:36.975+0000 7f7fea000700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7fec070c70 0x7f7fec116290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:35778/0 (socket says 172.21.15.18:35778) 2024-04-12T21:14:36.971 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:36.975+0000 7f7fea000700 1 -- 172.21.15.18:0/3697777412 learned_addr learned my addr 172.21.15.18:0/3697777412 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:14:36.971 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:36.975+0000 7f7feaa00700 1 --2- 172.21.15.18:0/3697777412 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7fec070360 0x7f7fec115d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:14:36.971 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:36.975+0000 7f7fea000700 1 -- 172.21.15.18:0/3697777412 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7fec070360 msgr2=0x7f7fec115d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:14:36.971 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:36.975+0000 7f7fea000700 1 --2- 172.21.15.18:0/3697777412 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7fec070360 0x7f7fec115d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:14:36.971 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:36.975+0000 7f7fea000700 1 -- 172.21.15.18:0/3697777412 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7fdc005040 con 0x7f7fec070c70 2024-04-12T21:14:36.972 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:36.976+0000 7f7fea000700 1 --2- 172.21.15.18:0/3697777412 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7fec070c70 0x7f7fec116290 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f7fd8002a50 tx=0x7f7fd80046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:14:36.972 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:36.976+0000 7f7fe2a00700 1 -- 172.21.15.18:0/3697777412 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7fd8007cc0 con 0x7f7fec070c70 2024-04-12T21:14:36.972 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:36.976+0000 7f7fe2a00700 1 -- 172.21.15.18:0/3697777412 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f7fd800f040 con 0x7f7fec070c70 2024-04-12T21:14:36.972 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:36.976+0000 7f7fe2a00700 1 -- 172.21.15.18:0/3697777412 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7fd8009640 con 0x7f7fec070c70 2024-04-12T21:14:36.972 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:36.976+0000 7f7ff1e00700 1 -- 172.21.15.18:0/3697777412 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7fec10ff90 con 0x7f7fec070c70 2024-04-12T21:14:36.972 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:36.976+0000 7f7ff1e00700 1 -- 172.21.15.18:0/3697777412 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7fec1104d0 con 0x7f7fec070c70 2024-04-12T21:14:36.972 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:36.977+0000 7f7ff1e00700 1 -- 172.21.15.18:0/3697777412 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7fec071aa0 con 0x7f7fec070c70 2024-04-12T21:14:36.973 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:36.978+0000 7f7fe2a00700 1 -- 172.21.15.18:0/3697777412 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7f7fd80097a0 con 0x7f7fec070c70 2024-04-12T21:14:36.975 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:36.979+0000 7f7fe2a00700 1 --2- 172.21.15.18:0/3697777412 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f7fd006ec10 0x7f7fd00710c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:14:36.975 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:36.979+0000 7f7fe2a00700 1 -- 172.21.15.18:0/3697777412 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 6757+0+0 (secure 0 0 0) 0x7f7fd808e830 con 0x7f7fec070c70 2024-04-12T21:14:36.975 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:36.980+0000 7f7feaa00700 1 --2- 172.21.15.18:0/3697777412 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f7fd006ec10 0x7f7fd00710c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:14:36.975 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:36.980+0000 7f7feaa00700 1 --2- 172.21.15.18:0/3697777412 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f7fd006ec10 0x7f7fd00710c0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f7fdc009fd0 tx=0x7f7fdc009e30 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:14:36.979 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:36.984+0000 7f7fe2a00700 1 -- 172.21.15.18:0/3697777412 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f7fd805af60 con 0x7f7fec070c70 2024-04-12T21:14:37.227 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:37.232+0000 7f7ff1e00700 1 -- 172.21.15.18:0/3697777412 --> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f7fec10c140 con 0x7f7fd006ec10 2024-04-12T21:14:37.230 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:37.235+0000 7f7fe2a00700 1 -- 172.21.15.18:0/3697777412 <== mgr.34102 v2:172.21.15.18:6800/3812891919 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f7fec10c140 con 0x7f7fd006ec10 2024-04-12T21:14:37.233 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:37.237+0000 7f7ff1e00700 1 -- 172.21.15.18:0/3697777412 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f7fd006ec10 msgr2=0x7f7fd00710c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:14:37.233 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:37.238+0000 7f7ff1e00700 1 --2- 172.21.15.18:0/3697777412 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f7fd006ec10 0x7f7fd00710c0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f7fdc009fd0 tx=0x7f7fdc009e30 comp rx=0 tx=0).stop 2024-04-12T21:14:37.233 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:37.238+0000 7f7ff1e00700 1 -- 172.21.15.18:0/3697777412 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7fec070c70 msgr2=0x7f7fec116290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:14:37.233 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:37.238+0000 7f7ff1e00700 1 --2- 172.21.15.18:0/3697777412 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7fec070c70 0x7f7fec116290 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f7fd8002a50 tx=0x7f7fd80046b0 comp rx=0 tx=0).stop 2024-04-12T21:14:37.233 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:37.238+0000 7f7ff1e00700 1 -- 172.21.15.18:0/3697777412 shutdown_connections 2024-04-12T21:14:37.234 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:37.238+0000 7f7ff1e00700 1 --2- 172.21.15.18:0/3697777412 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f7fd006ec10 0x7f7fd00710c0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:14:37.234 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:37.238+0000 7f7ff1e00700 1 --2- 172.21.15.18:0/3697777412 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7fec070360 0x7f7fec115d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:14:37.234 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:37.238+0000 7f7ff1e00700 1 --2- 172.21.15.18:0/3697777412 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7fec070c70 0x7f7fec116290 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:14:37.234 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:37.238+0000 7f7ff1e00700 1 -- 172.21.15.18:0/3697777412 >> 172.21.15.18:0/3697777412 conn(0x7f7fec0f8970 msgr2=0x7f7fec0f9da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:14:37.234 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:37.238+0000 7f7ff1e00700 1 -- 172.21.15.18:0/3697777412 shutdown_connections 2024-04-12T21:14:37.234 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:37.238+0000 7f7ff1e00700 1 -- 172.21.15.18:0/3697777412 wait complete. 2024-04-12T21:14:37.393 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:37.397+0000 7f7b06e00700 1 -- 172.21.15.18:0/2057294576 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7b001035e0 msgr2=0x7f7b000fcc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:14:37.393 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:37.397+0000 7f7b06e00700 1 --2- 172.21.15.18:0/2057294576 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7b001035e0 0x7f7b000fcc50 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f7aec004940 tx=0x7f7aec009e30 comp rx=0 tx=0).stop 2024-04-12T21:14:37.393 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:37.397+0000 7f7b06e00700 1 -- 172.21.15.18:0/2057294576 shutdown_connections 2024-04-12T21:14:37.393 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:37.397+0000 7f7b06e00700 1 --2- 172.21.15.18:0/2057294576 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7b001035e0 0x7f7b000fcc50 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:14:37.393 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:37.397+0000 7f7b06e00700 1 --2- 172.21.15.18:0/2057294576 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7b00102c40 0x7f7b00103010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:14:37.393 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:37.397+0000 7f7b06e00700 1 -- 172.21.15.18:0/2057294576 >> 172.21.15.18:0/2057294576 conn(0x7f7b000f87b0 msgr2=0x7f7b000fabc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:14:37.393 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:37.397+0000 7f7b06e00700 1 -- 172.21.15.18:0/2057294576 shutdown_connections 2024-04-12T21:14:37.393 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:37.397+0000 7f7b06e00700 1 -- 172.21.15.18:0/2057294576 wait complete. 2024-04-12T21:14:37.394 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:37.398+0000 7f7b06e00700 1 Processor -- start 2024-04-12T21:14:37.394 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:37.398+0000 7f7b06e00700 1 -- start start 2024-04-12T21:14:37.394 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:37.399+0000 7f7b06e00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7b00102c40 0x7f7b00115bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:14:37.394 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:37.399+0000 7f7b06e00700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7b001035e0 0x7f7b001160f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:14:37.394 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:37.399+0000 7f7b06e00700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b00116740 con 0x7f7b00102c40 2024-04-12T21:14:37.395 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:37.399+0000 7f7b06e00700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b0010fb50 con 0x7f7b001035e0 2024-04-12T21:14:37.395 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:37.399+0000 7f7b05a00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7b00102c40 0x7f7b00115bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:14:37.395 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:37.399+0000 7f7b05a00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7b00102c40 0x7f7b00115bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:35786/0 (socket says 172.21.15.18:35786) 2024-04-12T21:14:37.395 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:37.399+0000 7f7b05a00700 1 -- 172.21.15.18:0/1367766124 learned_addr learned my addr 172.21.15.18:0/1367766124 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:14:37.396 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:37.399+0000 7f7b05000700 1 --2- 172.21.15.18:0/1367766124 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7b001035e0 0x7f7b001160f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:14:37.396 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:37.400+0000 7f7b05a00700 1 -- 172.21.15.18:0/1367766124 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7b001035e0 msgr2=0x7f7b001160f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:14:37.396 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:37.400+0000 7f7b05a00700 1 --2- 172.21.15.18:0/1367766124 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7b001035e0 0x7f7b001160f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:14:37.396 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:37.400+0000 7f7b05a00700 1 -- 172.21.15.18:0/1367766124 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7aec005040 con 0x7f7b00102c40 2024-04-12T21:14:37.396 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:37.400+0000 7f7b05000700 1 --2- 172.21.15.18:0/1367766124 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7b001035e0 0x7f7b001160f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-12T21:14:37.396 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:37.400+0000 7f7b05a00700 1 --2- 172.21.15.18:0/1367766124 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7b00102c40 0x7f7b00115bb0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f7af400c830 tx=0x7f7af4004470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:14:37.396 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:37.400+0000 7f7afd600700 1 -- 172.21.15.18:0/1367766124 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7af400a760 con 0x7f7b00102c40 2024-04-12T21:14:37.396 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:37.400+0000 7f7afd600700 1 -- 172.21.15.18:0/1367766124 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f7af400ad90 con 0x7f7b00102c40 2024-04-12T21:14:37.396 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:37.400+0000 7f7afd600700 1 -- 172.21.15.18:0/1367766124 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7af40113a0 con 0x7f7b00102c40 2024-04-12T21:14:37.396 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:37.400+0000 7f7b06e00700 1 -- 172.21.15.18:0/1367766124 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7b0010fe30 con 0x7f7b00102c40 2024-04-12T21:14:37.397 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:37.401+0000 7f7b06e00700 1 -- 172.21.15.18:0/1367766124 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7b001103f0 con 0x7f7b00102c40 2024-04-12T21:14:37.397 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:37.402+0000 7f7b06e00700 1 -- 172.21.15.18:0/1367766124 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7b000fe390 con 0x7f7b00102c40 2024-04-12T21:14:37.398 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:37.403+0000 7f7afd600700 1 -- 172.21.15.18:0/1367766124 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7f7af4012400 con 0x7f7b00102c40 2024-04-12T21:14:37.399 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:37.404+0000 7f7afd600700 1 --2- 172.21.15.18:0/1367766124 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f7ae806ec10 0x7f7ae80710c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:14:37.399 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:37.404+0000 7f7afd600700 1 -- 172.21.15.18:0/1367766124 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 6757+0+0 (secure 0 0 0) 0x7f7af408e900 con 0x7f7b00102c40 2024-04-12T21:14:37.399 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:37.404+0000 7f7b05000700 1 --2- 172.21.15.18:0/1367766124 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f7ae806ec10 0x7f7ae80710c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:14:37.400 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:37.405+0000 7f7b05000700 1 --2- 172.21.15.18:0/1367766124 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f7ae806ec10 0x7f7ae80710c0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f7aec00b3b0 tx=0x7f7aec009e30 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:14:37.404 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:37.409+0000 7f7afd600700 1 -- 172.21.15.18:0/1367766124 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f7af40576c0 con 0x7f7b00102c40 2024-04-12T21:14:37.528 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:14:37 smithi186 ceph-mon[86379]: pgmap v444: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:14:37.599 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:14:37 smithi018 ceph-mon[112590]: pgmap v444: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:14:37.653 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:37.658+0000 7f7b06e00700 1 -- 172.21.15.18:0/1367766124 --> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f7b0004bed0 con 0x7f7ae806ec10 2024-04-12T21:14:37.683 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:37.687+0000 7f7afd600700 1 -- 172.21.15.18:0/1367766124 <== mgr.34102 v2:172.21.15.18:6800/3812891919 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7f7b0004bed0 con 0x7f7ae806ec10 2024-04-12T21:14:37.683 INFO:teuthology.orchestra.run.smithi018.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-12T21:14:37.683 INFO:teuthology.orchestra.run.smithi018.stdout:alertmanager.smithi018 smithi018 *:9093,9094 running (15m) 6m ago 29m 14.7M - 0.25.0 c8568f914cd2 c332090068e8 2024-04-12T21:14:37.684 INFO:teuthology.orchestra.run.smithi018.stdout:crash.smithi018 smithi018 running (11m) 6m ago 30m 6665k - 19.0.0-2879-g324baf8d 8717720dc5e5 b7f80d1f7982 2024-04-12T21:14:37.684 INFO:teuthology.orchestra.run.smithi018.stdout:crash.smithi186 smithi186 running (11m) 7m ago 28m 6660k - 19.0.0-2879-g324baf8d 8717720dc5e5 a4f55c81b623 2024-04-12T21:14:37.684 INFO:teuthology.orchestra.run.smithi018.stdout:grafana.smithi018 smithi018 *:3000 running (14m) 6m ago 29m 74.8M - 9.4.12 2bacad6d85d8 b08e3bf176dd 2024-04-12T21:14:37.684 INFO:teuthology.orchestra.run.smithi018.stdout:mds.cephfs.smithi018.acqbsc smithi018 running (7m) 6m ago 22m 11.1M - 19.0.0-2879-g324baf8d 8717720dc5e5 021f99ab24ae 2024-04-12T21:14:37.684 INFO:teuthology.orchestra.run.smithi018.stdout:mds.cephfs.smithi018.szjluo smithi018 running (6m) 6m ago 22m 10.9M - 19.0.0-2879-g324baf8d 8717720dc5e5 0003117479d4 2024-04-12T21:14:37.684 INFO:teuthology.orchestra.run.smithi018.stdout:mds.cephfs.smithi186.jdllyn smithi186 running (22m) 7m ago 22m 15.6M - 17.2.7-827-gd3b5060a 21de9b7369ad 2b2bc46bd29a 2024-04-12T21:14:37.684 INFO:teuthology.orchestra.run.smithi018.stdout:mds.cephfs.smithi186.wjnafs smithi186 running (22m) 7m ago 22m 15.4M - 17.2.7-827-gd3b5060a 21de9b7369ad 74b115d6d6fd 2024-04-12T21:14:37.684 INFO:teuthology.orchestra.run.smithi018.stdout:mgr.smithi018.xjzwga smithi018 *:8443,9283,8765 running (17m) 6m ago 31m 509M - 19.0.0-2879-g324baf8d 8717720dc5e5 56088305eab7 2024-04-12T21:14:37.684 INFO:teuthology.orchestra.run.smithi018.stdout:mgr.smithi186.kqcfxk smithi186 *:8443,9283,8765 running (16m) 7m ago 28m 452M - 19.0.0-2879-g324baf8d 8717720dc5e5 09cd766e5e0d 2024-04-12T21:14:37.684 INFO:teuthology.orchestra.run.smithi018.stdout:mon.smithi018 smithi018 running (13m) 6m ago 31m 57.0M 2048M 19.0.0-2879-g324baf8d 8717720dc5e5 df3e201dfed6 2024-04-12T21:14:37.684 INFO:teuthology.orchestra.run.smithi018.stdout:mon.smithi186 smithi186 running (12m) 7m ago 28m 37.7M 2048M 19.0.0-2879-g324baf8d 8717720dc5e5 c77498da9b2b 2024-04-12T21:14:37.684 INFO:teuthology.orchestra.run.smithi018.stdout:node-exporter.smithi018 smithi018 *:9100 running (16m) 6m ago 29m 7780k - 1.5.0 0da6a335fe13 326c3d9c6847 2024-04-12T21:14:37.685 INFO:teuthology.orchestra.run.smithi018.stdout:node-exporter.smithi186 smithi186 *:9100 running (16m) 7m ago 28m 7826k - 1.5.0 0da6a335fe13 ec2112baeecf 2024-04-12T21:14:37.685 INFO:teuthology.orchestra.run.smithi018.stdout:osd.0 smithi018 running (10m) 6m ago 27m 121M 1559M 19.0.0-2879-g324baf8d 8717720dc5e5 60b1273ca047 2024-04-12T21:14:37.685 INFO:teuthology.orchestra.run.smithi018.stdout:osd.1 smithi018 running (10m) 6m ago 26m 91.3M 1559M 19.0.0-2879-g324baf8d 8717720dc5e5 170ce1e5fb8d 2024-04-12T21:14:37.685 INFO:teuthology.orchestra.run.smithi018.stdout:osd.2 smithi018 running (9m) 6m ago 26m 90.3M 1559M 19.0.0-2879-g324baf8d 8717720dc5e5 c2cbc6aa2839 2024-04-12T21:14:37.685 INFO:teuthology.orchestra.run.smithi018.stdout:osd.3 smithi186 running (9m) 7m ago 25m 96.6M 2581M 19.0.0-2879-g324baf8d 8717720dc5e5 0138e8262e4b 2024-04-12T21:14:37.685 INFO:teuthology.orchestra.run.smithi018.stdout:osd.4 smithi186 running (8m) 7m ago 25m 87.2M 2581M 19.0.0-2879-g324baf8d 8717720dc5e5 b20cb63b25c9 2024-04-12T21:14:37.685 INFO:teuthology.orchestra.run.smithi018.stdout:osd.5 smithi186 running (7m) 7m ago 24m 11.1M 2581M 19.0.0-2879-g324baf8d 8717720dc5e5 2513bea2610e 2024-04-12T21:14:37.685 INFO:teuthology.orchestra.run.smithi018.stdout:prometheus.smithi018 smithi018 *:9095 running (15m) 6m ago 29m 60.6M - 2.43.0 a07b618ecd1d f58841127e21 2024-04-12T21:14:37.686 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:37.690+0000 7f7b06e00700 1 -- 172.21.15.18:0/1367766124 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f7ae806ec10 msgr2=0x7f7ae80710c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:14:37.686 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:37.690+0000 7f7b06e00700 1 --2- 172.21.15.18:0/1367766124 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f7ae806ec10 0x7f7ae80710c0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f7aec00b3b0 tx=0x7f7aec009e30 comp rx=0 tx=0).stop 2024-04-12T21:14:37.686 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:37.690+0000 7f7b06e00700 1 -- 172.21.15.18:0/1367766124 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7b00102c40 msgr2=0x7f7b00115bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:14:37.686 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:37.690+0000 7f7b06e00700 1 --2- 172.21.15.18:0/1367766124 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7b00102c40 0x7f7b00115bb0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f7af400c830 tx=0x7f7af4004470 comp rx=0 tx=0).stop 2024-04-12T21:14:37.686 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:37.691+0000 7f7b06e00700 1 -- 172.21.15.18:0/1367766124 shutdown_connections 2024-04-12T21:14:37.686 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:37.691+0000 7f7b06e00700 1 --2- 172.21.15.18:0/1367766124 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f7ae806ec10 0x7f7ae80710c0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:14:37.686 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:37.691+0000 7f7b06e00700 1 --2- 172.21.15.18:0/1367766124 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7b00102c40 0x7f7b00115bb0 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:14:37.686 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:37.691+0000 7f7b06e00700 1 --2- 172.21.15.18:0/1367766124 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7b001035e0 0x7f7b001160f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:14:37.687 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:37.691+0000 7f7b06e00700 1 -- 172.21.15.18:0/1367766124 >> 172.21.15.18:0/1367766124 conn(0x7f7b000f87b0 msgr2=0x7f7b000f96b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:14:37.687 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:37.691+0000 7f7b06e00700 1 -- 172.21.15.18:0/1367766124 shutdown_connections 2024-04-12T21:14:37.687 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:37.691+0000 7f7b06e00700 1 -- 172.21.15.18:0/1367766124 wait complete. 2024-04-12T21:14:37.847 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:37.852+0000 7fa41e400700 1 -- 172.21.15.18:0/4186132045 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa4181057f0 msgr2=0x7fa4180fee60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:14:37.847 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:37.852+0000 7fa41e400700 1 --2- 172.21.15.18:0/4186132045 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa4181057f0 0x7fa4180fee60 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7fa404004940 tx=0x7fa404009e30 comp rx=0 tx=0).stop 2024-04-12T21:14:37.848 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:37.852+0000 7fa41e400700 1 -- 172.21.15.18:0/4186132045 shutdown_connections 2024-04-12T21:14:37.848 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:37.852+0000 7fa41e400700 1 --2- 172.21.15.18:0/4186132045 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa4181057f0 0x7fa4180fee60 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:14:37.848 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:37.852+0000 7fa41e400700 1 --2- 172.21.15.18:0/4186132045 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa418104e50 0x7fa418105220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:14:37.848 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:37.852+0000 7fa41e400700 1 -- 172.21.15.18:0/4186132045 >> 172.21.15.18:0/4186132045 conn(0x7fa4180fab10 msgr2=0x7fa4180fcef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:14:37.848 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:37.852+0000 7fa41e400700 1 -- 172.21.15.18:0/4186132045 shutdown_connections 2024-04-12T21:14:37.848 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:37.852+0000 7fa41e400700 1 -- 172.21.15.18:0/4186132045 wait complete. 2024-04-12T21:14:37.849 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:37.853+0000 7fa41e400700 1 Processor -- start 2024-04-12T21:14:37.850 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:37.853+0000 7fa41e400700 1 -- start start 2024-04-12T21:14:37.850 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:37.853+0000 7fa41e400700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa418104e50 0x7fa418115bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:14:37.850 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:37.853+0000 7fa41e400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa4181057f0 0x7fa4181160f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:14:37.850 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:37.853+0000 7fa41e400700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4181167d0 con 0x7fa4181057f0 2024-04-12T21:14:37.850 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:37.853+0000 7fa41e400700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa41810fb50 con 0x7fa418104e50 2024-04-12T21:14:37.850 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:37.854+0000 7fa417e00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa4181057f0 0x7fa4181160f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:14:37.850 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:37.854+0000 7fa417e00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa4181057f0 0x7fa4181160f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:35798/0 (socket says 172.21.15.18:35798) 2024-04-12T21:14:37.850 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:37.854+0000 7fa417e00700 1 -- 172.21.15.18:0/3271676691 learned_addr learned my addr 172.21.15.18:0/3271676691 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:14:37.850 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:37.854+0000 7fa41d000700 1 --2- 172.21.15.18:0/3271676691 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa418104e50 0x7fa418115bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:14:37.850 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:37.854+0000 7fa417e00700 1 -- 172.21.15.18:0/3271676691 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa418104e50 msgr2=0x7fa418115bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:14:37.850 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:37.854+0000 7fa417e00700 1 --2- 172.21.15.18:0/3271676691 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa418104e50 0x7fa418115bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:14:37.850 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:37.854+0000 7fa417e00700 1 -- 172.21.15.18:0/3271676691 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa404005040 con 0x7fa4181057f0 2024-04-12T21:14:37.851 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:37.854+0000 7fa41d000700 1 --2- 172.21.15.18:0/3271676691 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa418104e50 0x7fa418115bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-12T21:14:37.851 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:37.855+0000 7fa417e00700 1 --2- 172.21.15.18:0/3271676691 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa4181057f0 0x7fa4181160f0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7fa404009180 tx=0x7fa404009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:14:37.851 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:37.855+0000 7fa414c00700 1 -- 172.21.15.18:0/3271676691 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa40400e040 con 0x7fa4181057f0 2024-04-12T21:14:37.852 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:37.855+0000 7fa41e400700 1 -- 172.21.15.18:0/3271676691 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa41810fdd0 con 0x7fa4181057f0 2024-04-12T21:14:37.852 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:37.855+0000 7fa414c00700 1 -- 172.21.15.18:0/3271676691 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa40400bbc0 con 0x7fa4181057f0 2024-04-12T21:14:37.852 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:37.855+0000 7fa414c00700 1 -- 172.21.15.18:0/3271676691 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa40401c3f0 con 0x7fa4181057f0 2024-04-12T21:14:37.852 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:37.855+0000 7fa41e400700 1 -- 172.21.15.18:0/3271676691 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa418110390 con 0x7fa4181057f0 2024-04-12T21:14:37.852 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:37.856+0000 7fa41e400700 1 -- 172.21.15.18:0/3271676691 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa4181005a0 con 0x7fa4181057f0 2024-04-12T21:14:37.853 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:37.858+0000 7fa414c00700 1 -- 172.21.15.18:0/3271676691 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7fa40400bd30 con 0x7fa4181057f0 2024-04-12T21:14:37.854 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:37.859+0000 7fa414c00700 1 --2- 172.21.15.18:0/3271676691 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fa40006ecd0 0x7fa400071180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:14:37.854 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:37.859+0000 7fa414c00700 1 -- 172.21.15.18:0/3271676691 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 6757+0+0 (secure 0 0 0) 0x7fa40408fd00 con 0x7fa4181057f0 2024-04-12T21:14:37.854 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:37.859+0000 7fa41d000700 1 --2- 172.21.15.18:0/3271676691 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fa40006ecd0 0x7fa400071180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:14:37.855 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:37.860+0000 7fa41d000700 1 --2- 172.21.15.18:0/3271676691 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fa40006ecd0 0x7fa400071180 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fa40c006fd0 tx=0x7fa40c008040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:14:37.859 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:37.864+0000 7fa414c00700 1 -- 172.21.15.18:0/3271676691 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7fa404058a00 con 0x7fa4181057f0 2024-04-12T21:14:38.178 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:38.183+0000 7fa41e400700 1 -- 172.21.15.18:0/3271676691 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fa41805f5f0 con 0x7fa4181057f0 2024-04-12T21:14:38.179 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:38.183+0000 7fa414c00700 1 -- 172.21.15.18:0/3271676691 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+837 (secure 0 0 0) 0x7fa40405c0c0 con 0x7fa4181057f0 2024-04-12T21:14:38.179 INFO:teuthology.orchestra.run.smithi018.stdout:{ 2024-04-12T21:14:38.179 INFO:teuthology.orchestra.run.smithi018.stdout: "mon": { 2024-04-12T21:14:38.179 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 19.0.0-2879-g324baf8d (324baf8ddb91dea46cac50cb535c306115791355) squid (dev)": 2 2024-04-12T21:14:38.179 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-12T21:14:38.179 INFO:teuthology.orchestra.run.smithi018.stdout: "mgr": { 2024-04-12T21:14:38.180 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 19.0.0-2879-g324baf8d (324baf8ddb91dea46cac50cb535c306115791355) squid (dev)": 2 2024-04-12T21:14:38.180 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-12T21:14:38.180 INFO:teuthology.orchestra.run.smithi018.stdout: "osd": { 2024-04-12T21:14:38.180 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 19.0.0-2879-g324baf8d (324baf8ddb91dea46cac50cb535c306115791355) squid (dev)": 6 2024-04-12T21:14:38.180 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-12T21:14:38.180 INFO:teuthology.orchestra.run.smithi018.stdout: "mds": { 2024-04-12T21:14:38.180 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 17.2.7-827-gd3b5060a (d3b5060a17449edfae9385cdf792afec109fca5d) quincy (stable)": 2, 2024-04-12T21:14:38.180 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 19.0.0-2879-g324baf8d (324baf8ddb91dea46cac50cb535c306115791355) squid (dev)": 2 2024-04-12T21:14:38.180 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-12T21:14:38.180 INFO:teuthology.orchestra.run.smithi018.stdout: "overall": { 2024-04-12T21:14:38.180 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 17.2.7-827-gd3b5060a (d3b5060a17449edfae9385cdf792afec109fca5d) quincy (stable)": 2, 2024-04-12T21:14:38.181 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 19.0.0-2879-g324baf8d (324baf8ddb91dea46cac50cb535c306115791355) squid (dev)": 12 2024-04-12T21:14:38.181 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-04-12T21:14:38.181 INFO:teuthology.orchestra.run.smithi018.stdout:} 2024-04-12T21:14:38.182 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:38.186+0000 7fa41e400700 1 -- 172.21.15.18:0/3271676691 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fa40006ecd0 msgr2=0x7fa400071180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:14:38.182 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:38.186+0000 7fa41e400700 1 --2- 172.21.15.18:0/3271676691 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fa40006ecd0 0x7fa400071180 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fa40c006fd0 tx=0x7fa40c008040 comp rx=0 tx=0).stop 2024-04-12T21:14:38.182 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:38.186+0000 7fa41e400700 1 -- 172.21.15.18:0/3271676691 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa4181057f0 msgr2=0x7fa4181160f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:14:38.182 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:38.186+0000 7fa41e400700 1 --2- 172.21.15.18:0/3271676691 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa4181057f0 0x7fa4181160f0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7fa404009180 tx=0x7fa404009e30 comp rx=0 tx=0).stop 2024-04-12T21:14:38.182 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:38.186+0000 7fa41e400700 1 -- 172.21.15.18:0/3271676691 shutdown_connections 2024-04-12T21:14:38.182 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:38.186+0000 7fa41e400700 1 --2- 172.21.15.18:0/3271676691 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fa40006ecd0 0x7fa400071180 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:14:38.182 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:38.186+0000 7fa41e400700 1 --2- 172.21.15.18:0/3271676691 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa418104e50 0x7fa418115bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:14:38.182 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:38.186+0000 7fa41e400700 1 --2- 172.21.15.18:0/3271676691 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa4181057f0 0x7fa4181160f0 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:14:38.182 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:38.186+0000 7fa41e400700 1 -- 172.21.15.18:0/3271676691 >> 172.21.15.18:0/3271676691 conn(0x7fa4180fab10 msgr2=0x7fa4180fca20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:14:38.182 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:38.187+0000 7fa41e400700 1 -- 172.21.15.18:0/3271676691 shutdown_connections 2024-04-12T21:14:38.182 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:38.187+0000 7fa41e400700 1 -- 172.21.15.18:0/3271676691 wait complete. 2024-04-12T21:14:38.334 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:38.339+0000 7f8942c00700 1 -- 172.21.15.18:0/180569013 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f893c105800 msgr2=0x7f893c0fee70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:14:38.335 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:38.339+0000 7f8942c00700 1 --2- 172.21.15.18:0/180569013 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f893c105800 0x7f893c0fee70 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f8928004940 tx=0x7f8928009e30 comp rx=0 tx=0).stop 2024-04-12T21:14:38.335 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:38.339+0000 7f8942c00700 1 -- 172.21.15.18:0/180569013 shutdown_connections 2024-04-12T21:14:38.335 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:38.339+0000 7f8942c00700 1 --2- 172.21.15.18:0/180569013 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f893c105800 0x7f893c0fee70 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:14:38.335 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:38.339+0000 7f8942c00700 1 --2- 172.21.15.18:0/180569013 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f893c104e60 0x7f893c105230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:14:38.335 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:38.339+0000 7f8942c00700 1 -- 172.21.15.18:0/180569013 >> 172.21.15.18:0/180569013 conn(0x7f893c0fab10 msgr2=0x7f893c0fcef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:14:38.335 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:38.339+0000 7f8942c00700 1 -- 172.21.15.18:0/180569013 shutdown_connections 2024-04-12T21:14:38.335 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:38.339+0000 7f8942c00700 1 -- 172.21.15.18:0/180569013 wait complete. 2024-04-12T21:14:38.336 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:38.340+0000 7f8942c00700 1 Processor -- start 2024-04-12T21:14:38.336 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:38.340+0000 7f8942c00700 1 -- start start 2024-04-12T21:14:38.336 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:38.340+0000 7f8942c00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f893c104e60 0x7f893c115ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:14:38.336 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:38.341+0000 7f8942c00700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f893c105800 0x7f893c1160e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:14:38.336 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:38.341+0000 7f8942c00700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f893c1167c0 con 0x7f893c104e60 2024-04-12T21:14:38.336 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:38.341+0000 7f8942c00700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f893c10fb40 con 0x7f893c105800 2024-04-12T21:14:38.337 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:38.341+0000 7f8941800700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f893c104e60 0x7f893c115ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:14:38.337 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:38.341+0000 7f8941800700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f893c104e60 0x7f893c115ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:35806/0 (socket says 172.21.15.18:35806) 2024-04-12T21:14:38.337 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:38.341+0000 7f8941800700 1 -- 172.21.15.18:0/3201251352 learned_addr learned my addr 172.21.15.18:0/3201251352 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:14:38.337 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:38.341+0000 7f8941800700 1 -- 172.21.15.18:0/3201251352 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f893c105800 msgr2=0x7f893c1160e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:14:38.337 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:38.341+0000 7f8941800700 1 --2- 172.21.15.18:0/3201251352 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f893c105800 0x7f893c1160e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:14:38.337 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:38.341+0000 7f8941800700 1 -- 172.21.15.18:0/3201251352 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8928005040 con 0x7f893c104e60 2024-04-12T21:14:38.338 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:38.341+0000 7f8941800700 1 --2- 172.21.15.18:0/3201251352 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f893c104e60 0x7f893c115ba0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f893000b790 tx=0x7f8930009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:14:38.338 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:38.342+0000 7f8939600700 1 -- 172.21.15.18:0/3201251352 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f893000f770 con 0x7f893c104e60 2024-04-12T21:14:38.338 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:38.342+0000 7f8939600700 1 -- 172.21.15.18:0/3201251352 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f893000fda0 con 0x7f893c104e60 2024-04-12T21:14:38.338 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:38.342+0000 7f8942c00700 1 -- 172.21.15.18:0/3201251352 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f893c10fe20 con 0x7f893c104e60 2024-04-12T21:14:38.338 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:38.342+0000 7f8939600700 1 -- 172.21.15.18:0/3201251352 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f893000d310 con 0x7f893c104e60 2024-04-12T21:14:38.338 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:38.342+0000 7f8942c00700 1 -- 172.21.15.18:0/3201251352 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f893c110360 con 0x7f893c104e60 2024-04-12T21:14:38.339 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:38.343+0000 7f8942c00700 1 -- 172.21.15.18:0/3201251352 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f893c1005b0 con 0x7f893c104e60 2024-04-12T21:14:38.339 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:38.344+0000 7f8939600700 1 -- 172.21.15.18:0/3201251352 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7f8930010420 con 0x7f893c104e60 2024-04-12T21:14:38.340 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:38.345+0000 7f8939600700 1 --2- 172.21.15.18:0/3201251352 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f892406ec00 0x7f89240710b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:14:38.340 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:38.345+0000 7f8939600700 1 -- 172.21.15.18:0/3201251352 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 6757+0+0 (secure 0 0 0) 0x7f893008e9c0 con 0x7f893c104e60 2024-04-12T21:14:38.340 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:38.345+0000 7f8940e00700 1 --2- 172.21.15.18:0/3201251352 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f892406ec00 0x7f89240710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:14:38.341 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:38.346+0000 7f8940e00700 1 --2- 172.21.15.18:0/3201251352 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f892406ec00 0x7f89240710b0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f8928009fd0 tx=0x7f8928009e30 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:14:38.345 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:38.350+0000 7f8939600700 1 -- 172.21.15.18:0/3201251352 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f89300576e0 con 0x7f893c104e60 2024-04-12T21:14:38.529 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:14:38 smithi186 ceph-mon[86379]: from='client.34808 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:14:38.529 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:14:38 smithi186 ceph-mon[86379]: from='client.34812 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:14:38.529 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:14:38 smithi186 ceph-mon[86379]: from='client.? 172.21.15.18:0/3271676691' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T21:14:38.599 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:14:38 smithi018 ceph-mon[112590]: from='client.34808 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:14:38.600 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:14:38 smithi018 ceph-mon[112590]: from='client.34812 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:14:38.600 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:14:38 smithi018 ceph-mon[112590]: from='client.? 172.21.15.18:0/3271676691' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T21:14:38.626 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:38.630+0000 7f8942c00700 1 -- 172.21.15.18:0/3201251352 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f893c0fe610 con 0x7f893c104e60 2024-04-12T21:14:38.626 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:38.631+0000 7f8939600700 1 -- 172.21.15.18:0/3201251352 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 29 v29) v1 ==== 76+0+1872 (secure 0 0 0) 0x7f893005ada0 con 0x7f893c104e60 2024-04-12T21:14:38.628 INFO:teuthology.orchestra.run.smithi018.stdout:e29 2024-04-12T21:14:38.628 INFO:teuthology.orchestra.run.smithi018.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-12T21:14:38.628 INFO:teuthology.orchestra.run.smithi018.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-12T21:14:38.628 INFO:teuthology.orchestra.run.smithi018.stdout:legacy client fscid: 1 2024-04-12T21:14:38.628 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-12T21:14:38.628 INFO:teuthology.orchestra.run.smithi018.stdout:Filesystem 'cephfs' (1) 2024-04-12T21:14:38.629 INFO:teuthology.orchestra.run.smithi018.stdout:fs_name cephfs 2024-04-12T21:14:38.629 INFO:teuthology.orchestra.run.smithi018.stdout:epoch 24 2024-04-12T21:14:38.629 INFO:teuthology.orchestra.run.smithi018.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-04-12T21:14:38.629 INFO:teuthology.orchestra.run.smithi018.stdout:created 2024-04-12T20:51:57.362850+0000 2024-04-12T21:14:38.629 INFO:teuthology.orchestra.run.smithi018.stdout:modified 2024-04-12T21:06:59.717190+0000 2024-04-12T21:14:38.629 INFO:teuthology.orchestra.run.smithi018.stdout:tableserver 0 2024-04-12T21:14:38.629 INFO:teuthology.orchestra.run.smithi018.stdout:root 0 2024-04-12T21:14:38.629 INFO:teuthology.orchestra.run.smithi018.stdout:session_timeout 60 2024-04-12T21:14:38.629 INFO:teuthology.orchestra.run.smithi018.stdout:session_autoclose 300 2024-04-12T21:14:38.629 INFO:teuthology.orchestra.run.smithi018.stdout:max_file_size 1099511627776 2024-04-12T21:14:38.629 INFO:teuthology.orchestra.run.smithi018.stdout:max_xattr_size 65536 2024-04-12T21:14:38.629 INFO:teuthology.orchestra.run.smithi018.stdout:required_client_features {} 2024-04-12T21:14:38.629 INFO:teuthology.orchestra.run.smithi018.stdout:last_failure 0 2024-04-12T21:14:38.629 INFO:teuthology.orchestra.run.smithi018.stdout:last_failure_osd_epoch 100 2024-04-12T21:14:38.630 INFO:teuthology.orchestra.run.smithi018.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-12T21:14:38.630 INFO:teuthology.orchestra.run.smithi018.stdout:max_mds 2 2024-04-12T21:14:38.630 INFO:teuthology.orchestra.run.smithi018.stdout:in 0,1 2024-04-12T21:14:38.630 INFO:teuthology.orchestra.run.smithi018.stdout:up {} 2024-04-12T21:14:38.630 INFO:teuthology.orchestra.run.smithi018.stdout:failed 0,1 2024-04-12T21:14:38.630 INFO:teuthology.orchestra.run.smithi018.stdout:damaged 2024-04-12T21:14:38.630 INFO:teuthology.orchestra.run.smithi018.stdout:stopped 2024-04-12T21:14:38.630 INFO:teuthology.orchestra.run.smithi018.stdout:data_pools [3] 2024-04-12T21:14:38.630 INFO:teuthology.orchestra.run.smithi018.stdout:metadata_pool 2 2024-04-12T21:14:38.630 INFO:teuthology.orchestra.run.smithi018.stdout:inline_data disabled 2024-04-12T21:14:38.630 INFO:teuthology.orchestra.run.smithi018.stdout:balancer 2024-04-12T21:14:38.630 INFO:teuthology.orchestra.run.smithi018.stdout:bal_rank_mask -1 2024-04-12T21:14:38.631 INFO:teuthology.orchestra.run.smithi018.stdout:standby_count_wanted 1 2024-04-12T21:14:38.631 INFO:teuthology.orchestra.run.smithi018.stdout:qdb_cluster leader: 0 members: 2024-04-12T21:14:38.631 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-12T21:14:38.631 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-12T21:14:38.631 INFO:teuthology.orchestra.run.smithi018.stdout:Standby daemons: 2024-04-12T21:14:38.631 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-12T21:14:38.631 INFO:teuthology.orchestra.run.smithi018.stdout:[mds.cephfs.smithi186.wjnafs{-1:24277} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.186:6826/438347207,v1:172.21.15.186:6827/438347207] compat {c=[1],r=[1],i=[7ff]}] 2024-04-12T21:14:38.631 INFO:teuthology.orchestra.run.smithi018.stdout:[mds.cephfs.smithi186.jdllyn{-1:24281} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.186:6824/1464359952,v1:172.21.15.186:6825/1464359952] compat {c=[1],r=[1],i=[7ff]}] 2024-04-12T21:14:38.631 INFO:teuthology.orchestra.run.smithi018.stdout:[mds.cephfs.smithi018.acqbsc{-1:34436} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.18:6828/755179764,v1:172.21.15.18:6829/755179764] compat {c=[1],r=[1],i=[1fff]}] 2024-04-12T21:14:38.631 INFO:teuthology.orchestra.run.smithi018.stdout:[mds.cephfs.smithi018.szjluo{-1:34468} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.18:6826/2407992653,v1:172.21.15.18:6827/2407992653] compat {c=[1],r=[1],i=[1fff]}] 2024-04-12T21:14:38.632 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:38.634+0000 7f8942c00700 1 -- 172.21.15.18:0/3201251352 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f892406ec00 msgr2=0x7f89240710b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:14:38.632 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:38.634+0000 7f8942c00700 1 --2- 172.21.15.18:0/3201251352 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f892406ec00 0x7f89240710b0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f8928009fd0 tx=0x7f8928009e30 comp rx=0 tx=0).stop 2024-04-12T21:14:38.633 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:38.634+0000 7f8942c00700 1 -- 172.21.15.18:0/3201251352 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f893c104e60 msgr2=0x7f893c115ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:14:38.633 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:38.634+0000 7f8942c00700 1 --2- 172.21.15.18:0/3201251352 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f893c104e60 0x7f893c115ba0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f893000b790 tx=0x7f8930009250 comp rx=0 tx=0).stop 2024-04-12T21:14:38.633 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:38.634+0000 7f8942c00700 1 -- 172.21.15.18:0/3201251352 shutdown_connections 2024-04-12T21:14:38.633 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:38.634+0000 7f8942c00700 1 --2- 172.21.15.18:0/3201251352 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f892406ec00 0x7f89240710b0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:14:38.633 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:38.634+0000 7f8942c00700 1 --2- 172.21.15.18:0/3201251352 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f893c104e60 0x7f893c115ba0 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:14:38.633 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:38.634+0000 7f8942c00700 1 --2- 172.21.15.18:0/3201251352 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f893c105800 0x7f893c1160e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:14:38.633 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:38.634+0000 7f8942c00700 1 -- 172.21.15.18:0/3201251352 >> 172.21.15.18:0/3201251352 conn(0x7f893c0fab10 msgr2=0x7f893c0fca30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:14:38.633 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:38.634+0000 7f8942c00700 1 -- 172.21.15.18:0/3201251352 shutdown_connections 2024-04-12T21:14:38.633 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:38.634+0000 7f8942c00700 1 -- 172.21.15.18:0/3201251352 wait complete. 2024-04-12T21:14:38.633 INFO:teuthology.orchestra.run.smithi018.stderr:dumped fsmap epoch 29 2024-04-12T21:14:38.789 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:38.793+0000 7f945c800700 1 -- 172.21.15.18:0/3105608377 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9454070bf0 msgr2=0x7f94541091b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:14:38.789 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:38.793+0000 7f945c800700 1 --2- 172.21.15.18:0/3105608377 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9454070bf0 0x7f94541091b0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f9444004940 tx=0x7f9444009e30 comp rx=0 tx=0).stop 2024-04-12T21:14:38.789 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:38.793+0000 7f945c800700 1 -- 172.21.15.18:0/3105608377 shutdown_connections 2024-04-12T21:14:38.789 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:38.793+0000 7f945c800700 1 --2- 172.21.15.18:0/3105608377 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9454070bf0 0x7f94541091b0 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:14:38.789 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:38.793+0000 7f945c800700 1 --2- 172.21.15.18:0/3105608377 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f94540702e0 0x7f94540706b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:14:38.789 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:38.793+0000 7f945c800700 1 -- 172.21.15.18:0/3105608377 >> 172.21.15.18:0/3105608377 conn(0x7f94540f8970 msgr2=0x7f94540fad80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:14:38.789 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:38.793+0000 7f945c800700 1 -- 172.21.15.18:0/3105608377 shutdown_connections 2024-04-12T21:14:38.789 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:38.794+0000 7f945c800700 1 -- 172.21.15.18:0/3105608377 wait complete. 2024-04-12T21:14:38.791 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:38.794+0000 7f945c800700 1 Processor -- start 2024-04-12T21:14:38.791 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:38.794+0000 7f945c800700 1 -- start start 2024-04-12T21:14:38.791 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:38.795+0000 7f945c800700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f94540702e0 0x7f9454115e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:14:38.791 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:38.795+0000 7f945c800700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9454070bf0 0x7f9454116370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:14:38.791 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:38.795+0000 7f945c800700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94541169c0 con 0x7f94540702e0 2024-04-12T21:14:38.791 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:38.795+0000 7f945c800700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f945410fd90 con 0x7f9454070bf0 2024-04-12T21:14:38.791 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:38.795+0000 7f9459e00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f94540702e0 0x7f9454115e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:14:38.792 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:38.795+0000 7f9459e00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f94540702e0 0x7f9454115e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:35824/0 (socket says 172.21.15.18:35824) 2024-04-12T21:14:38.792 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:38.795+0000 7f9459e00700 1 -- 172.21.15.18:0/2105821155 learned_addr learned my addr 172.21.15.18:0/2105821155 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:14:38.792 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:38.795+0000 7f9459400700 1 --2- 172.21.15.18:0/2105821155 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9454070bf0 0x7f9454116370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:14:38.792 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:38.796+0000 7f9459e00700 1 -- 172.21.15.18:0/2105821155 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9454070bf0 msgr2=0x7f9454116370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:14:38.792 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:38.796+0000 7f9459e00700 1 --2- 172.21.15.18:0/2105821155 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9454070bf0 0x7f9454116370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:14:38.792 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:38.796+0000 7f9459e00700 1 -- 172.21.15.18:0/2105821155 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9444005040 con 0x7f94540702e0 2024-04-12T21:14:38.792 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:38.796+0000 7f9459e00700 1 --2- 172.21.15.18:0/2105821155 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f94540702e0 0x7f9454115e30 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f944800c830 tx=0x7f9448004470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:14:38.793 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:38.796+0000 7f944e000700 1 -- 172.21.15.18:0/2105821155 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f944800a760 con 0x7f94540702e0 2024-04-12T21:14:38.793 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:38.796+0000 7f944e000700 1 -- 172.21.15.18:0/2105821155 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f944800ad90 con 0x7f94540702e0 2024-04-12T21:14:38.793 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:38.796+0000 7f945c800700 1 -- 172.21.15.18:0/2105821155 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9454110070 con 0x7f94540702e0 2024-04-12T21:14:38.793 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:38.796+0000 7f944e000700 1 -- 172.21.15.18:0/2105821155 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f94480113a0 con 0x7f94540702e0 2024-04-12T21:14:38.793 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:38.796+0000 7f945c800700 1 -- 172.21.15.18:0/2105821155 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f94541105b0 con 0x7f94540702e0 2024-04-12T21:14:38.793 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:38.797+0000 7f945c800700 1 -- 172.21.15.18:0/2105821155 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9454071a20 con 0x7f94540702e0 2024-04-12T21:14:38.794 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:38.798+0000 7f944e000700 1 -- 172.21.15.18:0/2105821155 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7f9448012400 con 0x7f94540702e0 2024-04-12T21:14:38.794 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:38.799+0000 7f944e000700 1 --2- 172.21.15.18:0/2105821155 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f943c06ec10 0x7f943c0710c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:14:38.794 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:38.799+0000 7f944e000700 1 -- 172.21.15.18:0/2105821155 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 6757+0+0 (secure 0 0 0) 0x7f944808ea00 con 0x7f94540702e0 2024-04-12T21:14:38.795 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:38.800+0000 7f9459400700 1 --2- 172.21.15.18:0/2105821155 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f943c06ec10 0x7f943c0710c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:14:38.795 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:38.800+0000 7f9459400700 1 --2- 172.21.15.18:0/2105821155 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f943c06ec10 0x7f943c0710c0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f944400b3b0 tx=0x7f9444009e30 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:14:38.799 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:38.804+0000 7f944e000700 1 -- 172.21.15.18:0/2105821155 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f944805b250 con 0x7f94540702e0 2024-04-12T21:14:39.040 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:39.045+0000 7f945c800700 1 -- 172.21.15.18:0/2105821155 --> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f945404bed0 con 0x7f943c06ec10 2024-04-12T21:14:39.043 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:39.048+0000 7f944e000700 1 -- 172.21.15.18:0/2105821155 <== mgr.34102 v2:172.21.15.18:6800/3812891919 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f945404bed0 con 0x7f943c06ec10 2024-04-12T21:14:39.044 INFO:teuthology.orchestra.run.smithi018.stdout:{ 2024-04-12T21:14:39.044 INFO:teuthology.orchestra.run.smithi018.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:f4452def3be4d350f4a8e5feaf7ebb1511be9298fe68478ba7a2d6fbd7d69a63", 2024-04-12T21:14:39.044 INFO:teuthology.orchestra.run.smithi018.stdout: "in_progress": true, 2024-04-12T21:14:39.044 INFO:teuthology.orchestra.run.smithi018.stdout: "which": "Upgrading all daemon types on all hosts", 2024-04-12T21:14:39.044 INFO:teuthology.orchestra.run.smithi018.stdout: "services_complete": [ 2024-04-12T21:14:39.044 INFO:teuthology.orchestra.run.smithi018.stdout: "crash", 2024-04-12T21:14:39.044 INFO:teuthology.orchestra.run.smithi018.stdout: "osd", 2024-04-12T21:14:39.044 INFO:teuthology.orchestra.run.smithi018.stdout: "mon", 2024-04-12T21:14:39.044 INFO:teuthology.orchestra.run.smithi018.stdout: "mgr" 2024-04-12T21:14:39.044 INFO:teuthology.orchestra.run.smithi018.stdout: ], 2024-04-12T21:14:39.044 INFO:teuthology.orchestra.run.smithi018.stdout: "progress": "14/21 daemons upgraded", 2024-04-12T21:14:39.044 INFO:teuthology.orchestra.run.smithi018.stdout: "message": "Currently upgrading mds daemons", 2024-04-12T21:14:39.044 INFO:teuthology.orchestra.run.smithi018.stdout: "is_paused": false 2024-04-12T21:14:39.045 INFO:teuthology.orchestra.run.smithi018.stdout:} 2024-04-12T21:14:39.048 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:39.052+0000 7f945c800700 1 -- 172.21.15.18:0/2105821155 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f943c06ec10 msgr2=0x7f943c0710c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:14:39.048 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:39.052+0000 7f945c800700 1 --2- 172.21.15.18:0/2105821155 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f943c06ec10 0x7f943c0710c0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f944400b3b0 tx=0x7f9444009e30 comp rx=0 tx=0).stop 2024-04-12T21:14:39.048 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:39.053+0000 7f945c800700 1 -- 172.21.15.18:0/2105821155 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f94540702e0 msgr2=0x7f9454115e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:14:39.049 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:39.053+0000 7f945c800700 1 --2- 172.21.15.18:0/2105821155 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f94540702e0 0x7f9454115e30 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f944800c830 tx=0x7f9448004470 comp rx=0 tx=0).stop 2024-04-12T21:14:39.049 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:39.053+0000 7f945c800700 1 -- 172.21.15.18:0/2105821155 shutdown_connections 2024-04-12T21:14:39.049 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:39.053+0000 7f945c800700 1 --2- 172.21.15.18:0/2105821155 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f943c06ec10 0x7f943c0710c0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:14:39.049 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:39.053+0000 7f945c800700 1 --2- 172.21.15.18:0/2105821155 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f94540702e0 0x7f9454115e30 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:14:39.049 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:39.053+0000 7f945c800700 1 --2- 172.21.15.18:0/2105821155 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9454070bf0 0x7f9454116370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:14:39.049 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:39.053+0000 7f945c800700 1 -- 172.21.15.18:0/2105821155 >> 172.21.15.18:0/2105821155 conn(0x7f94540f8970 msgr2=0x7f94540f9ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:14:39.049 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:39.053+0000 7f945c800700 1 -- 172.21.15.18:0/2105821155 shutdown_connections 2024-04-12T21:14:39.049 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:39.053+0000 7f945c800700 1 -- 172.21.15.18:0/2105821155 wait complete. 2024-04-12T21:14:39.197 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:39.201+0000 7f14ae400700 1 -- 172.21.15.18:0/1275687416 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f14a8102c30 msgr2=0x7f14a8103000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:14:39.197 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:39.201+0000 7f14ae400700 1 --2- 172.21.15.18:0/1275687416 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f14a8102c30 0x7f14a8103000 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f149c004940 tx=0x7f149c009e30 comp rx=0 tx=0).stop 2024-04-12T21:14:39.197 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:39.201+0000 7f14ae400700 1 -- 172.21.15.18:0/1275687416 shutdown_connections 2024-04-12T21:14:39.197 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:39.201+0000 7f14ae400700 1 --2- 172.21.15.18:0/1275687416 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f14a81035d0 0x7f14a80fcc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:14:39.197 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:39.201+0000 7f14ae400700 1 --2- 172.21.15.18:0/1275687416 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f14a8102c30 0x7f14a8103000 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:14:39.197 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:39.201+0000 7f14ae400700 1 -- 172.21.15.18:0/1275687416 >> 172.21.15.18:0/1275687416 conn(0x7f14a80f8760 msgr2=0x7f14a80fab70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:14:39.197 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:39.201+0000 7f14ae400700 1 -- 172.21.15.18:0/1275687416 shutdown_connections 2024-04-12T21:14:39.197 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:39.202+0000 7f14ae400700 1 -- 172.21.15.18:0/1275687416 wait complete. 2024-04-12T21:14:39.199 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:39.202+0000 7f14ae400700 1 Processor -- start 2024-04-12T21:14:39.199 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:39.202+0000 7f14ae400700 1 -- start start 2024-04-12T21:14:39.199 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:39.203+0000 7f14ae400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f14a8102c30 0x7f14a8115b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:14:39.199 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:39.203+0000 7f14ae400700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f14a81035d0 0x7f14a81160c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:14:39.199 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:39.203+0000 7f14ae400700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14a8116710 con 0x7f14a8102c30 2024-04-12T21:14:39.199 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:39.203+0000 7f14ae400700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14a810fb20 con 0x7f14a81035d0 2024-04-12T21:14:39.199 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:39.203+0000 7f14ad000700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f14a8102c30 0x7f14a8115b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:14:39.199 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:39.203+0000 7f14ad000700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f14a8102c30 0x7f14a8115b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:35842/0 (socket says 172.21.15.18:35842) 2024-04-12T21:14:39.199 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:39.203+0000 7f14ad000700 1 -- 172.21.15.18:0/3072568416 learned_addr learned my addr 172.21.15.18:0/3072568416 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:14:39.199 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:39.203+0000 7f14a7e00700 1 --2- 172.21.15.18:0/3072568416 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f14a81035d0 0x7f14a81160c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:14:39.200 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:39.203+0000 7f14ad000700 1 -- 172.21.15.18:0/3072568416 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f14a81035d0 msgr2=0x7f14a81160c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:14:39.200 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:39.204+0000 7f14ad000700 1 --2- 172.21.15.18:0/3072568416 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f14a81035d0 0x7f14a81160c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:14:39.200 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:39.204+0000 7f14ad000700 1 -- 172.21.15.18:0/3072568416 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f149c005040 con 0x7f14a8102c30 2024-04-12T21:14:39.200 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:39.204+0000 7f14a7e00700 1 --2- 172.21.15.18:0/3072568416 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f14a81035d0 0x7f14a81160c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-12T21:14:39.200 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:39.204+0000 7f14ad000700 1 --2- 172.21.15.18:0/3072568416 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f14a8102c30 0x7f14a8115b80 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f149c009150 tx=0x7f149c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:14:39.201 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:39.204+0000 7f14a4c00700 1 -- 172.21.15.18:0/3072568416 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f149c00e040 con 0x7f14a8102c30 2024-04-12T21:14:39.201 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:39.204+0000 7f14a4c00700 1 -- 172.21.15.18:0/3072568416 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f149c013420 con 0x7f14a8102c30 2024-04-12T21:14:39.201 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:39.204+0000 7f14ae400700 1 -- 172.21.15.18:0/3072568416 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f14a810fda0 con 0x7f14a8102c30 2024-04-12T21:14:39.201 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:39.204+0000 7f14a4c00700 1 -- 172.21.15.18:0/3072568416 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f149c01c630 con 0x7f14a8102c30 2024-04-12T21:14:39.201 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:39.204+0000 7f14ae400700 1 -- 172.21.15.18:0/3072568416 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f14a8110360 con 0x7f14a8102c30 2024-04-12T21:14:39.201 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:39.205+0000 7f14ae400700 1 -- 172.21.15.18:0/3072568416 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f14a80fe380 con 0x7f14a8102c30 2024-04-12T21:14:39.202 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:39.207+0000 7f14a4c00700 1 -- 172.21.15.18:0/3072568416 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7f149c013590 con 0x7f14a8102c30 2024-04-12T21:14:39.203 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:39.208+0000 7f14a4c00700 1 --2- 172.21.15.18:0/3072568416 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f149006ec00 0x7f14900710b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:14:39.203 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:39.208+0000 7f14a4c00700 1 -- 172.21.15.18:0/3072568416 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 6757+0+0 (secure 0 0 0) 0x7f149c08fc10 con 0x7f14a8102c30 2024-04-12T21:14:39.204 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:39.208+0000 7f14a7e00700 1 --2- 172.21.15.18:0/3072568416 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f149006ec00 0x7f14900710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:14:39.204 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:39.209+0000 7f14a7e00700 1 --2- 172.21.15.18:0/3072568416 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f149006ec00 0x7f14900710b0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f1494009fd0 tx=0x7f149400b040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:14:39.210 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:39.213+0000 7f14a4c00700 1 -- 172.21.15.18:0/3072568416 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f149c058910 con 0x7f14a8102c30 2024-04-12T21:14:39.350 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:14:39 smithi018 ceph-mon[112590]: from='client.34816 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:14:39.350 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:14:39 smithi018 ceph-mon[112590]: pgmap v445: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:14:39.350 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:14:39 smithi018 ceph-mon[112590]: from='client.? 172.21.15.18:0/3201251352' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-12T21:14:39.526 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:39.530+0000 7f14ae400700 1 -- 172.21.15.18:0/3072568416 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f14a804bed0 con 0x7f14a8102c30 2024-04-12T21:14:39.526 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:39.531+0000 7f14a4c00700 1 -- 172.21.15.18:0/3072568416 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+233 (secure 0 0 0) 0x7f149c05bfd0 con 0x7f14a8102c30 2024-04-12T21:14:39.527 INFO:teuthology.orchestra.run.smithi018.stdout:HEALTH_ERR 1 filesystem is degraded; 1 filesystem is offline 2024-04-12T21:14:39.527 INFO:teuthology.orchestra.run.smithi018.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-04-12T21:14:39.527 INFO:teuthology.orchestra.run.smithi018.stdout: fs cephfs is degraded 2024-04-12T21:14:39.527 INFO:teuthology.orchestra.run.smithi018.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-04-12T21:14:39.527 INFO:teuthology.orchestra.run.smithi018.stdout: fs cephfs is offline because no MDS is active for it. 2024-04-12T21:14:39.528 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:14:39 smithi186 ceph-mon[86379]: from='client.34816 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:14:39.528 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:14:39 smithi186 ceph-mon[86379]: pgmap v445: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:14:39.528 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:14:39 smithi186 ceph-mon[86379]: from='client.? 172.21.15.18:0/3201251352' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-12T21:14:39.530 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:39.534+0000 7f14ae400700 1 -- 172.21.15.18:0/3072568416 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f149006ec00 msgr2=0x7f14900710b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:14:39.530 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:39.534+0000 7f14ae400700 1 --2- 172.21.15.18:0/3072568416 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f149006ec00 0x7f14900710b0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f1494009fd0 tx=0x7f149400b040 comp rx=0 tx=0).stop 2024-04-12T21:14:39.530 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:39.534+0000 7f14ae400700 1 -- 172.21.15.18:0/3072568416 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f14a8102c30 msgr2=0x7f14a8115b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:14:39.530 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:39.534+0000 7f14ae400700 1 --2- 172.21.15.18:0/3072568416 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f14a8102c30 0x7f14a8115b80 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f149c009150 tx=0x7f149c009e30 comp rx=0 tx=0).stop 2024-04-12T21:14:39.530 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:39.534+0000 7f14ae400700 1 -- 172.21.15.18:0/3072568416 shutdown_connections 2024-04-12T21:14:39.530 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:39.534+0000 7f14ae400700 1 --2- 172.21.15.18:0/3072568416 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f149006ec00 0x7f14900710b0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:14:39.530 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:39.534+0000 7f14ae400700 1 --2- 172.21.15.18:0/3072568416 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f14a8102c30 0x7f14a8115b80 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:14:39.530 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:39.534+0000 7f14ae400700 1 --2- 172.21.15.18:0/3072568416 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f14a81035d0 0x7f14a81160c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:14:39.530 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:39.535+0000 7f14ae400700 1 -- 172.21.15.18:0/3072568416 >> 172.21.15.18:0/3072568416 conn(0x7f14a80f8760 msgr2=0x7f14a8071b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:14:39.530 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:39.535+0000 7f14ae400700 1 -- 172.21.15.18:0/3072568416 shutdown_connections 2024-04-12T21:14:39.530 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:14:39.535+0000 7f14ae400700 1 -- 172.21.15.18:0/3072568416 wait complete. 2024-04-12T21:14:40.529 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:14:40 smithi186 ceph-mon[86379]: from='client.34828 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:14:40.529 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:14:40 smithi186 ceph-mon[86379]: from='client.? 172.21.15.18:0/3072568416' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-12T21:14:40.529 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:14:40 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T21:14:40.600 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:14:40 smithi018 ceph-mon[112590]: from='client.34828 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:14:40.600 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:14:40 smithi018 ceph-mon[112590]: from='client.? 172.21.15.18:0/3072568416' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-12T21:14:40.600 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:14:40 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T21:14:41.528 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:14:41 smithi186 ceph-mon[86379]: pgmap v446: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:14:41.600 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:14:41 smithi018 ceph-mon[112590]: pgmap v446: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:14:43.600 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:14:43 smithi018 ceph-mon[112590]: pgmap v447: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:14:43.778 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:14:43 smithi186 ceph-mon[86379]: pgmap v447: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:14:45.599 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:14:45 smithi018 ceph-mon[112590]: pgmap v448: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:14:45.778 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:14:45 smithi186 ceph-mon[86379]: pgmap v448: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:14:47.599 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:14:47 smithi018 ceph-mon[112590]: pgmap v449: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:14:47.778 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:14:47 smithi186 ceph-mon[86379]: pgmap v449: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:14:49.597 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:14:49 smithi018 ceph-mon[112590]: pgmap v450: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:14:49.778 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:14:49 smithi186 ceph-mon[86379]: pgmap v450: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:14:51.600 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:14:51 smithi018 ceph-mon[112590]: pgmap v451: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:14:51.779 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:14:51 smithi186 ceph-mon[86379]: pgmap v451: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:14:53.599 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:14:53 smithi018 ceph-mon[112590]: pgmap v452: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:14:53.778 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:14:53 smithi186 ceph-mon[86379]: pgmap v452: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:14:55.600 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:14:55 smithi018 ceph-mon[112590]: pgmap v453: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:14:55.600 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:14:55 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T21:14:55.778 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:14:55 smithi186 ceph-mon[86379]: pgmap v453: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:14:55.779 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:14:55 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T21:14:57.599 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:14:57 smithi018 ceph-mon[112590]: pgmap v454: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:14:57.778 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:14:57 smithi186 ceph-mon[86379]: pgmap v454: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:14:59.600 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:14:59 smithi018 ceph-mon[112590]: pgmap v455: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:14:59.779 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:14:59 smithi186 ceph-mon[86379]: pgmap v455: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:15:01.600 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:15:01 smithi018 ceph-mon[112590]: pgmap v456: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:15:01.778 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:15:01 smithi186 ceph-mon[86379]: pgmap v456: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:15:03.599 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:15:03 smithi018 ceph-mon[112590]: pgmap v457: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:15:03.779 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:15:03 smithi186 ceph-mon[86379]: pgmap v457: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:15:05.778 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:15:05 smithi186 ceph-mon[86379]: pgmap v458: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:15:05.849 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:15:05 smithi018 ceph-mon[112590]: pgmap v458: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:15:07.778 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:15:07 smithi186 ceph-mon[86379]: pgmap v459: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:15:07.849 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:15:07 smithi018 ceph-mon[112590]: pgmap v459: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:15:09.702 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:09.705+0000 7f6113000700 1 -- 172.21.15.18:0/3915968034 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f610c071f90 msgr2=0x7f610c070360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:15:09.702 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:09.705+0000 7f6113000700 1 --2- 172.21.15.18:0/3915968034 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f610c071f90 0x7f610c070360 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f60fc004970 tx=0x7f60fc009e30 comp rx=0 tx=0).stop 2024-04-12T21:15:09.702 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:09.705+0000 7f6113000700 1 -- 172.21.15.18:0/3915968034 shutdown_connections 2024-04-12T21:15:09.702 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:09.705+0000 7f6113000700 1 --2- 172.21.15.18:0/3915968034 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f610c0708a0 0x7f610c070d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:15:09.702 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:09.705+0000 7f6113000700 1 --2- 172.21.15.18:0/3915968034 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f610c071f90 0x7f610c070360 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:15:09.702 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:09.705+0000 7f6113000700 1 -- 172.21.15.18:0/3915968034 >> 172.21.15.18:0/3915968034 conn(0x7f610c0f89b0 msgr2=0x7f610c0fadc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:15:09.702 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:15:09 smithi018 ceph-mon[112590]: pgmap v460: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:15:09.704 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:09.705+0000 7f6113000700 1 -- 172.21.15.18:0/3915968034 shutdown_connections 2024-04-12T21:15:09.704 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:09.706+0000 7f6113000700 1 -- 172.21.15.18:0/3915968034 wait complete. 2024-04-12T21:15:09.704 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:09.706+0000 7f6113000700 1 Processor -- start 2024-04-12T21:15:09.704 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:09.707+0000 7f6113000700 1 -- start start 2024-04-12T21:15:09.704 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:09.707+0000 7f6113000700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f610c0708a0 0x7f610c11a070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:15:09.704 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:09.707+0000 7f6113000700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f610c11a5b0 0x7f610c114290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:15:09.704 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:09.707+0000 7f6113000700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f610c11ab30 con 0x7f610c11a5b0 2024-04-12T21:15:09.704 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:09.707+0000 7f6113000700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f610c11aca0 con 0x7f610c0708a0 2024-04-12T21:15:09.704 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:09.707+0000 7f610b400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f610c11a5b0 0x7f610c114290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:15:09.704 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:09.708+0000 7f610b400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f610c11a5b0 0x7f610c114290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:44178/0 (socket says 172.21.15.18:44178) 2024-04-12T21:15:09.704 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:09.708+0000 7f610b400700 1 -- 172.21.15.18:0/2080389755 learned_addr learned my addr 172.21.15.18:0/2080389755 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:15:09.704 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:09.708+0000 7f610be00700 1 --2- 172.21.15.18:0/2080389755 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f610c0708a0 0x7f610c11a070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:15:09.705 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:09.708+0000 7f610b400700 1 -- 172.21.15.18:0/2080389755 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f610c0708a0 msgr2=0x7f610c11a070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:15:09.706 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:09.708+0000 7f610b400700 1 --2- 172.21.15.18:0/2080389755 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f610c0708a0 0x7f610c11a070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:15:09.706 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:09.708+0000 7f610b400700 1 -- 172.21.15.18:0/2080389755 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60fc005040 con 0x7f610c11a5b0 2024-04-12T21:15:09.706 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:09.708+0000 7f610b400700 1 --2- 172.21.15.18:0/2080389755 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f610c11a5b0 0x7f610c114290 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f60f800c840 tx=0x7f60f8004470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:15:09.706 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:09.709+0000 7f6103e00700 1 -- 172.21.15.18:0/2080389755 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f60f8012810 con 0x7f610c11a5b0 2024-04-12T21:15:09.706 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:09.709+0000 7f6103e00700 1 -- 172.21.15.18:0/2080389755 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f60f8012e40 con 0x7f610c11a5b0 2024-04-12T21:15:09.706 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:09.709+0000 7f6103e00700 1 -- 172.21.15.18:0/2080389755 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f60f800a370 con 0x7f610c11a5b0 2024-04-12T21:15:09.706 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:09.709+0000 7f6113000700 1 -- 172.21.15.18:0/2080389755 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f610c114830 con 0x7f610c11a5b0 2024-04-12T21:15:09.706 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:09.709+0000 7f6113000700 1 -- 172.21.15.18:0/2080389755 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f610c114d70 con 0x7f610c11a5b0 2024-04-12T21:15:09.706 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:09.710+0000 7f6113000700 1 -- 172.21.15.18:0/2080389755 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f610c1150a0 con 0x7f610c11a5b0 2024-04-12T21:15:09.708 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:09.712+0000 7f6103e00700 1 -- 172.21.15.18:0/2080389755 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7f60f8011380 con 0x7f610c11a5b0 2024-04-12T21:15:09.709 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:09.713+0000 7f6103e00700 1 --2- 172.21.15.18:0/2080389755 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f60f006ec00 0x7f60f00710b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:15:09.709 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:09.713+0000 7f6103e00700 1 -- 172.21.15.18:0/2080389755 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 6757+0+0 (secure 0 0 0) 0x7f60f808ed90 con 0x7f610c11a5b0 2024-04-12T21:15:09.709 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:09.713+0000 7f610be00700 1 --2- 172.21.15.18:0/2080389755 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f60f006ec00 0x7f60f00710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:15:09.710 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:09.714+0000 7f610be00700 1 --2- 172.21.15.18:0/2080389755 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f60f006ec00 0x7f60f00710b0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f60fc004f50 tx=0x7f60fc009e30 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:15:09.714 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:09.718+0000 7f6103e00700 1 -- 172.21.15.18:0/2080389755 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f60f8057ab0 con 0x7f610c11a5b0 2024-04-12T21:15:09.779 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:15:09 smithi186 ceph-mon[86379]: pgmap v460: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:15:09.969 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:09.972+0000 7f6113000700 1 -- 172.21.15.18:0/2080389755 --> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f610c110540 con 0x7f60f006ec00 2024-04-12T21:15:09.971 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:09.975+0000 7f6103e00700 1 -- 172.21.15.18:0/2080389755 <== mgr.34102 v2:172.21.15.18:6800/3812891919 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f610c110540 con 0x7f60f006ec00 2024-04-12T21:15:09.974 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:09.977+0000 7f6113000700 1 -- 172.21.15.18:0/2080389755 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f60f006ec00 msgr2=0x7f60f00710b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:15:09.975 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:09.977+0000 7f6113000700 1 --2- 172.21.15.18:0/2080389755 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f60f006ec00 0x7f60f00710b0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f60fc004f50 tx=0x7f60fc009e30 comp rx=0 tx=0).stop 2024-04-12T21:15:09.975 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:09.978+0000 7f6113000700 1 -- 172.21.15.18:0/2080389755 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f610c11a5b0 msgr2=0x7f610c114290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:15:09.975 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:09.978+0000 7f6113000700 1 --2- 172.21.15.18:0/2080389755 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f610c11a5b0 0x7f610c114290 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f60f800c840 tx=0x7f60f8004470 comp rx=0 tx=0).stop 2024-04-12T21:15:09.975 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:09.978+0000 7f6113000700 1 -- 172.21.15.18:0/2080389755 shutdown_connections 2024-04-12T21:15:09.975 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:09.978+0000 7f6113000700 1 --2- 172.21.15.18:0/2080389755 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f60f006ec00 0x7f60f00710b0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:15:09.975 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:09.978+0000 7f6113000700 1 --2- 172.21.15.18:0/2080389755 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f610c0708a0 0x7f610c11a070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:15:09.975 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:09.978+0000 7f6113000700 1 --2- 172.21.15.18:0/2080389755 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f610c11a5b0 0x7f610c114290 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:15:09.975 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:09.978+0000 7f6113000700 1 -- 172.21.15.18:0/2080389755 >> 172.21.15.18:0/2080389755 conn(0x7f610c0f89b0 msgr2=0x7f610c0f9b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:15:09.975 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:09.978+0000 7f6113000700 1 -- 172.21.15.18:0/2080389755 shutdown_connections 2024-04-12T21:15:09.975 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:09.978+0000 7f6113000700 1 -- 172.21.15.18:0/2080389755 wait complete. 2024-04-12T21:15:09.992 INFO:teuthology.orchestra.run.smithi018.stdout:true 2024-04-12T21:15:10.140 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:10.143+0000 7f0499800700 1 -- 172.21.15.18:0/232812815 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f04940702e0 msgr2=0x7f04940706b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:15:10.140 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:10.143+0000 7f0499800700 1 --2- 172.21.15.18:0/232812815 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f04940702e0 0x7f04940706b0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f0484004940 tx=0x7f0484009e30 comp rx=0 tx=0).stop 2024-04-12T21:15:10.140 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:10.143+0000 7f0499800700 1 -- 172.21.15.18:0/232812815 shutdown_connections 2024-04-12T21:15:10.140 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:10.143+0000 7f0499800700 1 --2- 172.21.15.18:0/232812815 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0494070bf0 0x7f0494108fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:15:10.140 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:10.143+0000 7f0499800700 1 --2- 172.21.15.18:0/232812815 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f04940702e0 0x7f04940706b0 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:15:10.140 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:10.143+0000 7f0499800700 1 -- 172.21.15.18:0/232812815 >> 172.21.15.18:0/232812815 conn(0x7f04940f87a0 msgr2=0x7f04940fabb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:15:10.140 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:10.144+0000 7f0499800700 1 -- 172.21.15.18:0/232812815 shutdown_connections 2024-04-12T21:15:10.140 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:10.144+0000 7f0499800700 1 -- 172.21.15.18:0/232812815 wait complete. 2024-04-12T21:15:10.142 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:10.144+0000 7f0499800700 1 Processor -- start 2024-04-12T21:15:10.142 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:10.145+0000 7f0499800700 1 -- start start 2024-04-12T21:15:10.142 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:10.145+0000 7f0499800700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f04940702e0 0x7f0494115ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:15:10.142 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:10.145+0000 7f0499800700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0494070bf0 0x7f04941160e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:15:10.142 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:10.145+0000 7f0499800700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0494116730 con 0x7f0494070bf0 2024-04-12T21:15:10.142 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:10.145+0000 7f0499800700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f049410fb40 con 0x7f04940702e0 2024-04-12T21:15:10.142 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:10.145+0000 7f048b400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0494070bf0 0x7f04941160e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:15:10.142 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:10.145+0000 7f048b400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0494070bf0 0x7f04941160e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:44196/0 (socket says 172.21.15.18:44196) 2024-04-12T21:15:10.142 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:10.145+0000 7f048b400700 1 -- 172.21.15.18:0/2863889236 learned_addr learned my addr 172.21.15.18:0/2863889236 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:15:10.142 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:10.146+0000 7f0493e00700 1 --2- 172.21.15.18:0/2863889236 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f04940702e0 0x7f0494115ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:15:10.143 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:10.146+0000 7f048b400700 1 -- 172.21.15.18:0/2863889236 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f04940702e0 msgr2=0x7f0494115ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:15:10.143 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:10.146+0000 7f048b400700 1 --2- 172.21.15.18:0/2863889236 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f04940702e0 0x7f0494115ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:15:10.143 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:10.146+0000 7f048b400700 1 -- 172.21.15.18:0/2863889236 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0484005040 con 0x7f0494070bf0 2024-04-12T21:15:10.143 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:10.146+0000 7f048b400700 1 --2- 172.21.15.18:0/2863889236 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0494070bf0 0x7f04941160e0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f047c00c830 tx=0x7f047c004470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:15:10.144 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:10.147+0000 7f0490c00700 1 -- 172.21.15.18:0/2863889236 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f047c00a760 con 0x7f0494070bf0 2024-04-12T21:15:10.144 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:10.147+0000 7f0490c00700 1 -- 172.21.15.18:0/2863889236 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f047c00ad90 con 0x7f0494070bf0 2024-04-12T21:15:10.144 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:10.147+0000 7f0499800700 1 -- 172.21.15.18:0/2863889236 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f049410fe20 con 0x7f0494070bf0 2024-04-12T21:15:10.144 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:10.147+0000 7f0490c00700 1 -- 172.21.15.18:0/2863889236 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f047c0113a0 con 0x7f0494070bf0 2024-04-12T21:15:10.144 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:10.147+0000 7f0499800700 1 -- 172.21.15.18:0/2863889236 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0494110360 con 0x7f0494070bf0 2024-04-12T21:15:10.144 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:10.148+0000 7f0499800700 1 -- 172.21.15.18:0/2863889236 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0494071a20 con 0x7f0494070bf0 2024-04-12T21:15:10.145 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:10.149+0000 7f0490c00700 1 -- 172.21.15.18:0/2863889236 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7f047c012400 con 0x7f0494070bf0 2024-04-12T21:15:10.146 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:10.150+0000 7f0490c00700 1 --2- 172.21.15.18:0/2863889236 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f047806ec10 0x7f04780710c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:15:10.146 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:10.150+0000 7f0490c00700 1 -- 172.21.15.18:0/2863889236 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 6757+0+0 (secure 0 0 0) 0x7f047c08e9c0 con 0x7f0494070bf0 2024-04-12T21:15:10.146 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:10.150+0000 7f0493e00700 1 --2- 172.21.15.18:0/2863889236 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f047806ec10 0x7f04780710c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:15:10.147 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:10.151+0000 7f0493e00700 1 --2- 172.21.15.18:0/2863889236 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f047806ec10 0x7f04780710c0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f048400b3b0 tx=0x7f0484009e30 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:15:10.151 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:10.155+0000 7f0490c00700 1 -- 172.21.15.18:0/2863889236 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f047c0576c0 con 0x7f0494070bf0 2024-04-12T21:15:10.417 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:10.420+0000 7f0499800700 1 -- 172.21.15.18:0/2863889236 --> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f049410bf80 con 0x7f047806ec10 2024-04-12T21:15:10.419 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:10.422+0000 7f0490c00700 1 -- 172.21.15.18:0/2863889236 <== mgr.34102 v2:172.21.15.18:6800/3812891919 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f049410bf80 con 0x7f047806ec10 2024-04-12T21:15:10.422 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:10.425+0000 7f0499800700 1 -- 172.21.15.18:0/2863889236 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f047806ec10 msgr2=0x7f04780710c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:15:10.422 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:10.425+0000 7f0499800700 1 --2- 172.21.15.18:0/2863889236 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f047806ec10 0x7f04780710c0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f048400b3b0 tx=0x7f0484009e30 comp rx=0 tx=0).stop 2024-04-12T21:15:10.422 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:10.425+0000 7f0499800700 1 -- 172.21.15.18:0/2863889236 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0494070bf0 msgr2=0x7f04941160e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:15:10.422 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:10.425+0000 7f0499800700 1 --2- 172.21.15.18:0/2863889236 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0494070bf0 0x7f04941160e0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f047c00c830 tx=0x7f047c004470 comp rx=0 tx=0).stop 2024-04-12T21:15:10.422 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:10.425+0000 7f0499800700 1 -- 172.21.15.18:0/2863889236 shutdown_connections 2024-04-12T21:15:10.422 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:10.425+0000 7f0499800700 1 --2- 172.21.15.18:0/2863889236 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f047806ec10 0x7f04780710c0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:15:10.423 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:10.425+0000 7f0499800700 1 --2- 172.21.15.18:0/2863889236 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f04940702e0 0x7f0494115ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:15:10.423 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:10.425+0000 7f0499800700 1 --2- 172.21.15.18:0/2863889236 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0494070bf0 0x7f04941160e0 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:15:10.423 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:10.425+0000 7f0499800700 1 -- 172.21.15.18:0/2863889236 >> 172.21.15.18:0/2863889236 conn(0x7f04940f87a0 msgr2=0x7f04941058e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:15:10.423 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:10.426+0000 7f0499800700 1 -- 172.21.15.18:0/2863889236 shutdown_connections 2024-04-12T21:15:10.423 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:10.426+0000 7f0499800700 1 -- 172.21.15.18:0/2863889236 wait complete. 2024-04-12T21:15:10.582 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:10.585+0000 7fe0dbe00700 1 -- 172.21.15.18:0/891980286 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe0dc072e00 msgr2=0x7fe0dc10d630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:15:10.582 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:10.585+0000 7fe0dbe00700 1 --2- 172.21.15.18:0/891980286 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe0dc072e00 0x7fe0dc10d630 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fe0c8004940 tx=0x7fe0c8009e30 comp rx=0 tx=0).stop 2024-04-12T21:15:10.582 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:10.585+0000 7fe0dbe00700 1 -- 172.21.15.18:0/891980286 shutdown_connections 2024-04-12T21:15:10.582 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:10.585+0000 7fe0dbe00700 1 --2- 172.21.15.18:0/891980286 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe0dc072e00 0x7fe0dc10d630 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:15:10.582 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:10.585+0000 7fe0dbe00700 1 --2- 172.21.15.18:0/891980286 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe0dc0724f0 0x7fe0dc0728c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:15:10.582 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:10.585+0000 7fe0dbe00700 1 -- 172.21.15.18:0/891980286 >> 172.21.15.18:0/891980286 conn(0x7fe0dc0fab10 msgr2=0x7fe0dc0fcef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:15:10.582 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:10.586+0000 7fe0dbe00700 1 -- 172.21.15.18:0/891980286 shutdown_connections 2024-04-12T21:15:10.582 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:10.586+0000 7fe0dbe00700 1 -- 172.21.15.18:0/891980286 wait complete. 2024-04-12T21:15:10.583 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:10.587+0000 7fe0dbe00700 1 Processor -- start 2024-04-12T21:15:10.583 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:10.587+0000 7fe0dbe00700 1 -- start start 2024-04-12T21:15:10.584 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:10.587+0000 7fe0dbe00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe0dc0724f0 0x7fe0dc119f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:15:10.584 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:10.587+0000 7fe0dbe00700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe0dc072e00 0x7fe0dc11a4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:15:10.584 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:10.587+0000 7fe0dbe00700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0dc11ab10 con 0x7fe0dc0724f0 2024-04-12T21:15:10.584 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:10.587+0000 7fe0dbe00700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0dc113ee0 con 0x7fe0dc072e00 2024-04-12T21:15:10.584 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:10.588+0000 7fe0daa00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe0dc0724f0 0x7fe0dc119f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:15:10.584 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:10.588+0000 7fe0daa00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe0dc0724f0 0x7fe0dc119f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:44222/0 (socket says 172.21.15.18:44222) 2024-04-12T21:15:10.584 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:10.588+0000 7fe0daa00700 1 -- 172.21.15.18:0/2062510668 learned_addr learned my addr 172.21.15.18:0/2062510668 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:15:10.585 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:10.588+0000 7fe0da000700 1 --2- 172.21.15.18:0/2062510668 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe0dc072e00 0x7fe0dc11a4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:15:10.585 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:10.588+0000 7fe0daa00700 1 -- 172.21.15.18:0/2062510668 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe0dc072e00 msgr2=0x7fe0dc11a4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:15:10.585 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:10.588+0000 7fe0daa00700 1 --2- 172.21.15.18:0/2062510668 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe0dc072e00 0x7fe0dc11a4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:15:10.585 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:10.588+0000 7fe0daa00700 1 -- 172.21.15.18:0/2062510668 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0c8005040 con 0x7fe0dc0724f0 2024-04-12T21:15:10.586 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:10.589+0000 7fe0daa00700 1 --2- 172.21.15.18:0/2062510668 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe0dc0724f0 0x7fe0dc119f80 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7fe0cc00c830 tx=0x7fe0cc004470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:15:10.586 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:10.589+0000 7fe0d2a00700 1 -- 172.21.15.18:0/2062510668 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe0cc00a760 con 0x7fe0dc0724f0 2024-04-12T21:15:10.586 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:10.589+0000 7fe0d2a00700 1 -- 172.21.15.18:0/2062510668 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe0cc00ad90 con 0x7fe0dc0724f0 2024-04-12T21:15:10.586 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:10.589+0000 7fe0d2a00700 1 -- 172.21.15.18:0/2062510668 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe0cc0113a0 con 0x7fe0dc0724f0 2024-04-12T21:15:10.586 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:10.589+0000 7fe0dbe00700 1 -- 172.21.15.18:0/2062510668 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe0dc1141c0 con 0x7fe0dc0724f0 2024-04-12T21:15:10.586 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:10.589+0000 7fe0dbe00700 1 -- 172.21.15.18:0/2062510668 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe0dc114780 con 0x7fe0dc0724f0 2024-04-12T21:15:10.586 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:10.591+0000 7fe0dbe00700 1 -- 172.21.15.18:0/2062510668 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe0dc073be0 con 0x7fe0dc0724f0 2024-04-12T21:15:10.588 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:10.592+0000 7fe0d2a00700 1 -- 172.21.15.18:0/2062510668 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7fe0cc012400 con 0x7fe0dc0724f0 2024-04-12T21:15:10.589 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:10.593+0000 7fe0d2a00700 1 --2- 172.21.15.18:0/2062510668 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fe0c006ec00 0x7fe0c00710b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:15:10.589 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:10.593+0000 7fe0d2a00700 1 -- 172.21.15.18:0/2062510668 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 6757+0+0 (secure 0 0 0) 0x7fe0cc08ea00 con 0x7fe0dc0724f0 2024-04-12T21:15:10.589 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:10.593+0000 7fe0da000700 1 --2- 172.21.15.18:0/2062510668 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fe0c006ec00 0x7fe0c00710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:15:10.590 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:10.594+0000 7fe0da000700 1 --2- 172.21.15.18:0/2062510668 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fe0c006ec00 0x7fe0c00710b0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fe0c800b3b0 tx=0x7fe0c8009e30 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:15:10.594 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:10.598+0000 7fe0d2a00700 1 -- 172.21.15.18:0/2062510668 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7fe0cc05b250 con 0x7fe0dc0724f0 2024-04-12T21:15:10.778 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:15:10 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T21:15:10.836 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:10.840+0000 7fe0dbe00700 1 -- 172.21.15.18:0/2062510668 --> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fe0dc04bed0 con 0x7fe0c006ec00 2024-04-12T21:15:10.837 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:15:10 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T21:15:10.854 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:10.857+0000 7fe0d2a00700 1 -- 172.21.15.18:0/2062510668 <== mgr.34102 v2:172.21.15.18:6800/3812891919 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7fe0dc04bed0 con 0x7fe0c006ec00 2024-04-12T21:15:10.854 INFO:teuthology.orchestra.run.smithi018.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-12T21:15:10.854 INFO:teuthology.orchestra.run.smithi018.stdout:alertmanager.smithi018 smithi018 *:9093,9094 running (15m) 7m ago 30m 14.7M - 0.25.0 c8568f914cd2 c332090068e8 2024-04-12T21:15:10.854 INFO:teuthology.orchestra.run.smithi018.stdout:crash.smithi018 smithi018 running (12m) 7m ago 30m 6665k - 19.0.0-2879-g324baf8d 8717720dc5e5 b7f80d1f7982 2024-04-12T21:15:10.854 INFO:teuthology.orchestra.run.smithi018.stdout:crash.smithi186 smithi186 running (11m) 8m ago 29m 6660k - 19.0.0-2879-g324baf8d 8717720dc5e5 a4f55c81b623 2024-04-12T21:15:10.854 INFO:teuthology.orchestra.run.smithi018.stdout:grafana.smithi018 smithi018 *:3000 running (15m) 7m ago 29m 74.8M - 9.4.12 2bacad6d85d8 b08e3bf176dd 2024-04-12T21:15:10.855 INFO:teuthology.orchestra.run.smithi018.stdout:mds.cephfs.smithi018.acqbsc smithi018 running (8m) 7m ago 23m 11.1M - 19.0.0-2879-g324baf8d 8717720dc5e5 021f99ab24ae 2024-04-12T21:15:10.855 INFO:teuthology.orchestra.run.smithi018.stdout:mds.cephfs.smithi018.szjluo smithi018 running (7m) 7m ago 23m 10.9M - 19.0.0-2879-g324baf8d 8717720dc5e5 0003117479d4 2024-04-12T21:15:10.855 INFO:teuthology.orchestra.run.smithi018.stdout:mds.cephfs.smithi186.jdllyn smithi186 running (23m) 8m ago 23m 15.6M - 17.2.7-827-gd3b5060a 21de9b7369ad 2b2bc46bd29a 2024-04-12T21:15:10.855 INFO:teuthology.orchestra.run.smithi018.stdout:mds.cephfs.smithi186.wjnafs smithi186 running (22m) 8m ago 22m 15.4M - 17.2.7-827-gd3b5060a 21de9b7369ad 74b115d6d6fd 2024-04-12T21:15:10.855 INFO:teuthology.orchestra.run.smithi018.stdout:mgr.smithi018.xjzwga smithi018 *:8443,9283,8765 running (18m) 7m ago 32m 509M - 19.0.0-2879-g324baf8d 8717720dc5e5 56088305eab7 2024-04-12T21:15:10.855 INFO:teuthology.orchestra.run.smithi018.stdout:mgr.smithi186.kqcfxk smithi186 *:8443,9283,8765 running (17m) 8m ago 28m 452M - 19.0.0-2879-g324baf8d 8717720dc5e5 09cd766e5e0d 2024-04-12T21:15:10.855 INFO:teuthology.orchestra.run.smithi018.stdout:mon.smithi018 smithi018 running (13m) 7m ago 32m 57.0M 2048M 19.0.0-2879-g324baf8d 8717720dc5e5 df3e201dfed6 2024-04-12T21:15:10.855 INFO:teuthology.orchestra.run.smithi018.stdout:mon.smithi186 smithi186 running (13m) 8m ago 28m 37.7M 2048M 19.0.0-2879-g324baf8d 8717720dc5e5 c77498da9b2b 2024-04-12T21:15:10.855 INFO:teuthology.orchestra.run.smithi018.stdout:node-exporter.smithi018 smithi018 *:9100 running (16m) 7m ago 30m 7780k - 1.5.0 0da6a335fe13 326c3d9c6847 2024-04-12T21:15:10.855 INFO:teuthology.orchestra.run.smithi018.stdout:node-exporter.smithi186 smithi186 *:9100 running (16m) 8m ago 29m 7826k - 1.5.0 0da6a335fe13 ec2112baeecf 2024-04-12T21:15:10.855 INFO:teuthology.orchestra.run.smithi018.stdout:osd.0 smithi018 running (11m) 7m ago 27m 121M 1559M 19.0.0-2879-g324baf8d 8717720dc5e5 60b1273ca047 2024-04-12T21:15:10.855 INFO:teuthology.orchestra.run.smithi018.stdout:osd.1 smithi018 running (10m) 7m ago 27m 91.3M 1559M 19.0.0-2879-g324baf8d 8717720dc5e5 170ce1e5fb8d 2024-04-12T21:15:10.855 INFO:teuthology.orchestra.run.smithi018.stdout:osd.2 smithi018 running (10m) 7m ago 26m 90.3M 1559M 19.0.0-2879-g324baf8d 8717720dc5e5 c2cbc6aa2839 2024-04-12T21:15:10.856 INFO:teuthology.orchestra.run.smithi018.stdout:osd.3 smithi186 running (9m) 8m ago 26m 96.6M 2581M 19.0.0-2879-g324baf8d 8717720dc5e5 0138e8262e4b 2024-04-12T21:15:10.856 INFO:teuthology.orchestra.run.smithi018.stdout:osd.4 smithi186 running (9m) 8m ago 25m 87.2M 2581M 19.0.0-2879-g324baf8d 8717720dc5e5 b20cb63b25c9 2024-04-12T21:15:10.856 INFO:teuthology.orchestra.run.smithi018.stdout:osd.5 smithi186 running (8m) 8m ago 24m 11.1M 2581M 19.0.0-2879-g324baf8d 8717720dc5e5 2513bea2610e 2024-04-12T21:15:10.856 INFO:teuthology.orchestra.run.smithi018.stdout:prometheus.smithi018 smithi018 *:9095 running (16m) 7m ago 29m 60.6M - 2.43.0 a07b618ecd1d f58841127e21 2024-04-12T21:15:10.857 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:10.861+0000 7fe0dbe00700 1 -- 172.21.15.18:0/2062510668 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fe0c006ec00 msgr2=0x7fe0c00710b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:15:10.857 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:10.861+0000 7fe0dbe00700 1 --2- 172.21.15.18:0/2062510668 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fe0c006ec00 0x7fe0c00710b0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fe0c800b3b0 tx=0x7fe0c8009e30 comp rx=0 tx=0).stop 2024-04-12T21:15:10.857 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:10.861+0000 7fe0dbe00700 1 -- 172.21.15.18:0/2062510668 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe0dc0724f0 msgr2=0x7fe0dc119f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:15:10.857 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:10.861+0000 7fe0dbe00700 1 --2- 172.21.15.18:0/2062510668 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe0dc0724f0 0x7fe0dc119f80 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7fe0cc00c830 tx=0x7fe0cc004470 comp rx=0 tx=0).stop 2024-04-12T21:15:10.858 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:10.861+0000 7fe0dbe00700 1 -- 172.21.15.18:0/2062510668 shutdown_connections 2024-04-12T21:15:10.858 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:10.861+0000 7fe0dbe00700 1 --2- 172.21.15.18:0/2062510668 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fe0c006ec00 0x7fe0c00710b0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:15:10.858 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:10.861+0000 7fe0dbe00700 1 --2- 172.21.15.18:0/2062510668 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe0dc0724f0 0x7fe0dc119f80 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:15:10.858 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:10.861+0000 7fe0dbe00700 1 --2- 172.21.15.18:0/2062510668 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe0dc072e00 0x7fe0dc11a4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:15:10.858 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:10.861+0000 7fe0dbe00700 1 -- 172.21.15.18:0/2062510668 >> 172.21.15.18:0/2062510668 conn(0x7fe0dc0fab10 msgr2=0x7fe0dc0faee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:15:10.858 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:10.861+0000 7fe0dbe00700 1 -- 172.21.15.18:0/2062510668 shutdown_connections 2024-04-12T21:15:10.858 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:10.861+0000 7fe0dbe00700 1 -- 172.21.15.18:0/2062510668 wait complete. 2024-04-12T21:15:11.013 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:11.017+0000 7f076a400700 1 -- 172.21.15.18:0/19700205 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0764105000 msgr2=0x7f07641053d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:15:11.013 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:11.017+0000 7f076a400700 1 --2- 172.21.15.18:0/19700205 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0764105000 0x7f07641053d0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f0754004940 tx=0x7f0754009e30 comp rx=0 tx=0).stop 2024-04-12T21:15:11.014 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:11.017+0000 7f076a400700 1 -- 172.21.15.18:0/19700205 shutdown_connections 2024-04-12T21:15:11.014 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:11.017+0000 7f076a400700 1 --2- 172.21.15.18:0/19700205 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f07641059a0 0x7f07640ff010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:15:11.014 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:11.017+0000 7f076a400700 1 --2- 172.21.15.18:0/19700205 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0764105000 0x7f07641053d0 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:15:11.014 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:11.017+0000 7f076a400700 1 -- 172.21.15.18:0/19700205 >> 172.21.15.18:0/19700205 conn(0x7f07640fab70 msgr2=0x7f07640fcf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:15:11.014 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:11.017+0000 7f076a400700 1 -- 172.21.15.18:0/19700205 shutdown_connections 2024-04-12T21:15:11.014 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:11.017+0000 7f076a400700 1 -- 172.21.15.18:0/19700205 wait complete. 2024-04-12T21:15:11.014 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:11.018+0000 7f076a400700 1 Processor -- start 2024-04-12T21:15:11.015 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:11.019+0000 7f076a400700 1 -- start start 2024-04-12T21:15:11.015 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:11.019+0000 7f076a400700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0764105000 0x7f0764115b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:15:11.015 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:11.019+0000 7f076a400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f07641059a0 0x7f07641160c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:15:11.016 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:11.019+0000 7f076a400700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0764116710 con 0x7f07641059a0 2024-04-12T21:15:11.016 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:11.019+0000 7f076a400700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f076410fcf0 con 0x7f0764105000 2024-04-12T21:15:11.017 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:11.020+0000 7f075aa00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f07641059a0 0x7f07641160c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:15:11.017 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:11.020+0000 7f075aa00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f07641059a0 0x7f07641160c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:40442/0 (socket says 172.21.15.18:40442) 2024-04-12T21:15:11.017 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:11.020+0000 7f075aa00700 1 -- 172.21.15.18:0/1565019370 learned_addr learned my addr 172.21.15.18:0/1565019370 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:15:11.017 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:11.020+0000 7f0763400700 1 --2- 172.21.15.18:0/1565019370 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0764105000 0x7f0764115b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:15:11.017 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:11.020+0000 7f075aa00700 1 -- 172.21.15.18:0/1565019370 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0764105000 msgr2=0x7f0764115b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:15:11.017 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:11.020+0000 7f075aa00700 1 --2- 172.21.15.18:0/1565019370 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0764105000 0x7f0764115b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:15:11.017 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:11.020+0000 7f075aa00700 1 -- 172.21.15.18:0/1565019370 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0754005040 con 0x7f07641059a0 2024-04-12T21:15:11.018 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:11.021+0000 7f075aa00700 1 --2- 172.21.15.18:0/1565019370 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f07641059a0 0x7f07641160c0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f074c001d00 tx=0x7f074c009e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:15:11.018 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:11.021+0000 7f075be00700 1 -- 172.21.15.18:0/1565019370 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f074c007d10 con 0x7f07641059a0 2024-04-12T21:15:11.018 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:11.021+0000 7f075be00700 1 -- 172.21.15.18:0/1565019370 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f074c007e70 con 0x7f07641059a0 2024-04-12T21:15:11.018 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:11.021+0000 7f076a400700 1 -- 172.21.15.18:0/1565019370 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f076410ffd0 con 0x7f07641059a0 2024-04-12T21:15:11.018 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:11.021+0000 7f075be00700 1 -- 172.21.15.18:0/1565019370 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f074c014400 con 0x7f07641059a0 2024-04-12T21:15:11.018 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:11.021+0000 7f076a400700 1 -- 172.21.15.18:0/1565019370 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0764110590 con 0x7f07641059a0 2024-04-12T21:15:11.018 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:11.022+0000 7f076a400700 1 -- 172.21.15.18:0/1565019370 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0764100750 con 0x7f07641059a0 2024-04-12T21:15:11.019 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:11.023+0000 7f075be00700 1 -- 172.21.15.18:0/1565019370 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7f074c0091d0 con 0x7f07641059a0 2024-04-12T21:15:11.020 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:11.024+0000 7f075be00700 1 --2- 172.21.15.18:0/1565019370 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f074806ec00 0x7f07480710b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:15:11.020 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:11.024+0000 7f075be00700 1 -- 172.21.15.18:0/1565019370 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 6757+0+0 (secure 0 0 0) 0x7f074c08e850 con 0x7f07641059a0 2024-04-12T21:15:11.021 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:11.025+0000 7f0763400700 1 --2- 172.21.15.18:0/1565019370 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f074806ec00 0x7f07480710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:15:11.021 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:11.025+0000 7f0763400700 1 --2- 172.21.15.18:0/1565019370 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f074806ec00 0x7f07480710b0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f075400b3b0 tx=0x7f0754009e30 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:15:11.025 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:11.029+0000 7f075be00700 1 -- 172.21.15.18:0/1565019370 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f074c05b0d0 con 0x7f07641059a0 2024-04-12T21:15:11.341 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:11.344+0000 7f076a400700 1 -- 172.21.15.18:0/1565019370 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f076410c100 con 0x7f07641059a0 2024-04-12T21:15:11.342 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:11.345+0000 7f075be00700 1 -- 172.21.15.18:0/1565019370 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+837 (secure 0 0 0) 0x7f074c05ac60 con 0x7f07641059a0 2024-04-12T21:15:11.342 INFO:teuthology.orchestra.run.smithi018.stdout:{ 2024-04-12T21:15:11.342 INFO:teuthology.orchestra.run.smithi018.stdout: "mon": { 2024-04-12T21:15:11.342 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 19.0.0-2879-g324baf8d (324baf8ddb91dea46cac50cb535c306115791355) squid (dev)": 2 2024-04-12T21:15:11.342 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-12T21:15:11.343 INFO:teuthology.orchestra.run.smithi018.stdout: "mgr": { 2024-04-12T21:15:11.343 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 19.0.0-2879-g324baf8d (324baf8ddb91dea46cac50cb535c306115791355) squid (dev)": 2 2024-04-12T21:15:11.343 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-12T21:15:11.343 INFO:teuthology.orchestra.run.smithi018.stdout: "osd": { 2024-04-12T21:15:11.343 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 19.0.0-2879-g324baf8d (324baf8ddb91dea46cac50cb535c306115791355) squid (dev)": 6 2024-04-12T21:15:11.343 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-12T21:15:11.343 INFO:teuthology.orchestra.run.smithi018.stdout: "mds": { 2024-04-12T21:15:11.343 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 17.2.7-827-gd3b5060a (d3b5060a17449edfae9385cdf792afec109fca5d) quincy (stable)": 2, 2024-04-12T21:15:11.343 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 19.0.0-2879-g324baf8d (324baf8ddb91dea46cac50cb535c306115791355) squid (dev)": 2 2024-04-12T21:15:11.343 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-12T21:15:11.343 INFO:teuthology.orchestra.run.smithi018.stdout: "overall": { 2024-04-12T21:15:11.344 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 17.2.7-827-gd3b5060a (d3b5060a17449edfae9385cdf792afec109fca5d) quincy (stable)": 2, 2024-04-12T21:15:11.344 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 19.0.0-2879-g324baf8d (324baf8ddb91dea46cac50cb535c306115791355) squid (dev)": 12 2024-04-12T21:15:11.344 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-04-12T21:15:11.344 INFO:teuthology.orchestra.run.smithi018.stdout:} 2024-04-12T21:15:11.345 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:11.348+0000 7f076a400700 1 -- 172.21.15.18:0/1565019370 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f074806ec00 msgr2=0x7f07480710b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:15:11.345 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:11.348+0000 7f076a400700 1 --2- 172.21.15.18:0/1565019370 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f074806ec00 0x7f07480710b0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f075400b3b0 tx=0x7f0754009e30 comp rx=0 tx=0).stop 2024-04-12T21:15:11.345 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:11.348+0000 7f076a400700 1 -- 172.21.15.18:0/1565019370 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f07641059a0 msgr2=0x7f07641160c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:15:11.345 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:11.348+0000 7f076a400700 1 --2- 172.21.15.18:0/1565019370 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f07641059a0 0x7f07641160c0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f074c001d00 tx=0x7f074c009e40 comp rx=0 tx=0).stop 2024-04-12T21:15:11.345 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:11.348+0000 7f076a400700 1 -- 172.21.15.18:0/1565019370 shutdown_connections 2024-04-12T21:15:11.345 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:11.348+0000 7f076a400700 1 --2- 172.21.15.18:0/1565019370 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f074806ec00 0x7f07480710b0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:15:11.345 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:11.348+0000 7f076a400700 1 --2- 172.21.15.18:0/1565019370 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0764105000 0x7f0764115b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:15:11.345 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:11.348+0000 7f076a400700 1 --2- 172.21.15.18:0/1565019370 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f07641059a0 0x7f07641160c0 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:15:11.345 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:11.348+0000 7f076a400700 1 -- 172.21.15.18:0/1565019370 >> 172.21.15.18:0/1565019370 conn(0x7f07640fab70 msgr2=0x7f07640fbb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:15:11.345 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:11.348+0000 7f076a400700 1 -- 172.21.15.18:0/1565019370 shutdown_connections 2024-04-12T21:15:11.345 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:11.348+0000 7f076a400700 1 -- 172.21.15.18:0/1565019370 wait complete. 2024-04-12T21:15:11.491 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:11.495+0000 7fc240c00700 1 -- 172.21.15.18:0/2408797083 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc23c104e60 msgr2=0x7fc23c105230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:15:11.492 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:11.495+0000 7fc240c00700 1 --2- 172.21.15.18:0/2408797083 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc23c104e60 0x7fc23c105230 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7fc22c004940 tx=0x7fc22c009e30 comp rx=0 tx=0).stop 2024-04-12T21:15:11.492 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:11.495+0000 7fc240c00700 1 -- 172.21.15.18:0/2408797083 shutdown_connections 2024-04-12T21:15:11.492 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:11.495+0000 7fc240c00700 1 --2- 172.21.15.18:0/2408797083 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc23c105800 0x7fc23c0fee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:15:11.492 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:11.495+0000 7fc240c00700 1 --2- 172.21.15.18:0/2408797083 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc23c104e60 0x7fc23c105230 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:15:11.492 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:11.495+0000 7fc240c00700 1 -- 172.21.15.18:0/2408797083 >> 172.21.15.18:0/2408797083 conn(0x7fc23c0fab10 msgr2=0x7fc23c0fcef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:15:11.492 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:11.495+0000 7fc240c00700 1 -- 172.21.15.18:0/2408797083 shutdown_connections 2024-04-12T21:15:11.492 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:11.495+0000 7fc240c00700 1 -- 172.21.15.18:0/2408797083 wait complete. 2024-04-12T21:15:11.494 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:11.496+0000 7fc240c00700 1 Processor -- start 2024-04-12T21:15:11.494 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:11.496+0000 7fc240c00700 1 -- start start 2024-04-12T21:15:11.494 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:11.496+0000 7fc240c00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc23c104e60 0x7fc23c115b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:15:11.494 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:11.496+0000 7fc240c00700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc23c105800 0x7fc23c1160c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:15:11.494 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:11.496+0000 7fc240c00700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc23c1167a0 con 0x7fc23c104e60 2024-04-12T21:15:11.494 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:11.496+0000 7fc240c00700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc23c10fb20 con 0x7fc23c105800 2024-04-12T21:15:11.495 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:11.497+0000 7fc23b400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc23c104e60 0x7fc23c115b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:15:11.495 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:11.497+0000 7fc23b400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc23c104e60 0x7fc23c115b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:40468/0 (socket says 172.21.15.18:40468) 2024-04-12T21:15:11.495 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:11.497+0000 7fc23b400700 1 -- 172.21.15.18:0/2295078729 learned_addr learned my addr 172.21.15.18:0/2295078729 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:15:11.495 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:11.497+0000 7fc23aa00700 1 --2- 172.21.15.18:0/2295078729 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc23c105800 0x7fc23c1160c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:15:11.495 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:11.497+0000 7fc23b400700 1 -- 172.21.15.18:0/2295078729 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc23c105800 msgr2=0x7fc23c1160c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:15:11.495 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:11.497+0000 7fc23b400700 1 --2- 172.21.15.18:0/2295078729 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc23c105800 0x7fc23c1160c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:15:11.495 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:11.497+0000 7fc23b400700 1 -- 172.21.15.18:0/2295078729 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc22c005040 con 0x7fc23c104e60 2024-04-12T21:15:11.495 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:11.497+0000 7fc23aa00700 1 --2- 172.21.15.18:0/2295078729 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc23c105800 0x7fc23c1160c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-12T21:15:11.495 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:11.498+0000 7fc23b400700 1 --2- 172.21.15.18:0/2295078729 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc23c104e60 0x7fc23c115b80 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7fc22c009180 tx=0x7fc22c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:15:11.495 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:11.498+0000 7fc233400700 1 -- 172.21.15.18:0/2295078729 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc22c00e040 con 0x7fc23c104e60 2024-04-12T21:15:11.495 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:11.498+0000 7fc233400700 1 -- 172.21.15.18:0/2295078729 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc22c00bbc0 con 0x7fc23c104e60 2024-04-12T21:15:11.495 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:11.498+0000 7fc240c00700 1 -- 172.21.15.18:0/2295078729 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc23c10fda0 con 0x7fc23c104e60 2024-04-12T21:15:11.495 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:11.498+0000 7fc233400700 1 -- 172.21.15.18:0/2295078729 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc22c01c630 con 0x7fc23c104e60 2024-04-12T21:15:11.496 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:11.498+0000 7fc240c00700 1 -- 172.21.15.18:0/2295078729 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc23c110360 con 0x7fc23c104e60 2024-04-12T21:15:11.496 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:11.499+0000 7fc240c00700 1 -- 172.21.15.18:0/2295078729 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc23c1005b0 con 0x7fc23c104e60 2024-04-12T21:15:11.496 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:11.500+0000 7fc233400700 1 -- 172.21.15.18:0/2295078729 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7fc22c00bd30 con 0x7fc23c104e60 2024-04-12T21:15:11.497 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:11.501+0000 7fc233400700 1 --2- 172.21.15.18:0/2295078729 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fc22006ec00 0x7fc2200710b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:15:11.497 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:11.501+0000 7fc233400700 1 -- 172.21.15.18:0/2295078729 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 6757+0+0 (secure 0 0 0) 0x7fc22c08fd70 con 0x7fc23c104e60 2024-04-12T21:15:11.498 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:11.502+0000 7fc23aa00700 1 --2- 172.21.15.18:0/2295078729 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fc22006ec00 0x7fc2200710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:15:11.498 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:11.502+0000 7fc23aa00700 1 --2- 172.21.15.18:0/2295078729 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fc22006ec00 0x7fc2200710b0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fc228006fd0 tx=0x7fc228008040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:15:11.502 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:11.506+0000 7fc233400700 1 -- 172.21.15.18:0/2295078729 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7fc22c058a70 con 0x7fc23c104e60 2024-04-12T21:15:11.600 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:15:11 smithi018 ceph-mon[112590]: from='client.34836 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:15:11.600 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:15:11 smithi018 ceph-mon[112590]: pgmap v461: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:15:11.600 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:15:11 smithi018 ceph-mon[112590]: from='client.34840 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:15:11.600 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:15:11 smithi018 ceph-mon[112590]: from='client.? 172.21.15.18:0/1565019370' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T21:15:11.777 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:11.781+0000 7fc240c00700 1 -- 172.21.15.18:0/2295078729 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fc23c04bed0 con 0x7fc23c104e60 2024-04-12T21:15:11.778 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:11.782+0000 7fc233400700 1 -- 172.21.15.18:0/2295078729 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 29 v29) v1 ==== 76+0+1872 (secure 0 0 0) 0x7fc22c05c130 con 0x7fc23c104e60 2024-04-12T21:15:11.778 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:15:11 smithi186 ceph-mon[86379]: from='client.34836 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:15:11.778 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:15:11 smithi186 ceph-mon[86379]: pgmap v461: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:15:11.778 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:15:11 smithi186 ceph-mon[86379]: from='client.34840 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:15:11.778 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:15:11 smithi186 ceph-mon[86379]: from='client.? 172.21.15.18:0/1565019370' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T21:15:11.779 INFO:teuthology.orchestra.run.smithi018.stdout:e29 2024-04-12T21:15:11.779 INFO:teuthology.orchestra.run.smithi018.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-12T21:15:11.779 INFO:teuthology.orchestra.run.smithi018.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-12T21:15:11.779 INFO:teuthology.orchestra.run.smithi018.stdout:legacy client fscid: 1 2024-04-12T21:15:11.779 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-12T21:15:11.779 INFO:teuthology.orchestra.run.smithi018.stdout:Filesystem 'cephfs' (1) 2024-04-12T21:15:11.779 INFO:teuthology.orchestra.run.smithi018.stdout:fs_name cephfs 2024-04-12T21:15:11.780 INFO:teuthology.orchestra.run.smithi018.stdout:epoch 24 2024-04-12T21:15:11.780 INFO:teuthology.orchestra.run.smithi018.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-04-12T21:15:11.780 INFO:teuthology.orchestra.run.smithi018.stdout:created 2024-04-12T20:51:57.362850+0000 2024-04-12T21:15:11.780 INFO:teuthology.orchestra.run.smithi018.stdout:modified 2024-04-12T21:06:59.717190+0000 2024-04-12T21:15:11.780 INFO:teuthology.orchestra.run.smithi018.stdout:tableserver 0 2024-04-12T21:15:11.780 INFO:teuthology.orchestra.run.smithi018.stdout:root 0 2024-04-12T21:15:11.780 INFO:teuthology.orchestra.run.smithi018.stdout:session_timeout 60 2024-04-12T21:15:11.780 INFO:teuthology.orchestra.run.smithi018.stdout:session_autoclose 300 2024-04-12T21:15:11.780 INFO:teuthology.orchestra.run.smithi018.stdout:max_file_size 1099511627776 2024-04-12T21:15:11.780 INFO:teuthology.orchestra.run.smithi018.stdout:max_xattr_size 65536 2024-04-12T21:15:11.780 INFO:teuthology.orchestra.run.smithi018.stdout:required_client_features {} 2024-04-12T21:15:11.780 INFO:teuthology.orchestra.run.smithi018.stdout:last_failure 0 2024-04-12T21:15:11.780 INFO:teuthology.orchestra.run.smithi018.stdout:last_failure_osd_epoch 100 2024-04-12T21:15:11.781 INFO:teuthology.orchestra.run.smithi018.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-12T21:15:11.781 INFO:teuthology.orchestra.run.smithi018.stdout:max_mds 2 2024-04-12T21:15:11.781 INFO:teuthology.orchestra.run.smithi018.stdout:in 0,1 2024-04-12T21:15:11.781 INFO:teuthology.orchestra.run.smithi018.stdout:up {} 2024-04-12T21:15:11.781 INFO:teuthology.orchestra.run.smithi018.stdout:failed 0,1 2024-04-12T21:15:11.781 INFO:teuthology.orchestra.run.smithi018.stdout:damaged 2024-04-12T21:15:11.781 INFO:teuthology.orchestra.run.smithi018.stdout:stopped 2024-04-12T21:15:11.781 INFO:teuthology.orchestra.run.smithi018.stdout:data_pools [3] 2024-04-12T21:15:11.781 INFO:teuthology.orchestra.run.smithi018.stdout:metadata_pool 2 2024-04-12T21:15:11.781 INFO:teuthology.orchestra.run.smithi018.stdout:inline_data disabled 2024-04-12T21:15:11.781 INFO:teuthology.orchestra.run.smithi018.stdout:balancer 2024-04-12T21:15:11.781 INFO:teuthology.orchestra.run.smithi018.stdout:bal_rank_mask -1 2024-04-12T21:15:11.781 INFO:teuthology.orchestra.run.smithi018.stdout:standby_count_wanted 1 2024-04-12T21:15:11.781 INFO:teuthology.orchestra.run.smithi018.stdout:qdb_cluster leader: 0 members: 2024-04-12T21:15:11.781 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-12T21:15:11.782 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-12T21:15:11.782 INFO:teuthology.orchestra.run.smithi018.stdout:Standby daemons: 2024-04-12T21:15:11.782 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-12T21:15:11.782 INFO:teuthology.orchestra.run.smithi018.stdout:[mds.cephfs.smithi186.wjnafs{-1:24277} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.186:6826/438347207,v1:172.21.15.186:6827/438347207] compat {c=[1],r=[1],i=[7ff]}] 2024-04-12T21:15:11.782 INFO:teuthology.orchestra.run.smithi018.stdout:[mds.cephfs.smithi186.jdllyn{-1:24281} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.186:6824/1464359952,v1:172.21.15.186:6825/1464359952] compat {c=[1],r=[1],i=[7ff]}] 2024-04-12T21:15:11.782 INFO:teuthology.orchestra.run.smithi018.stdout:[mds.cephfs.smithi018.acqbsc{-1:34436} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.18:6828/755179764,v1:172.21.15.18:6829/755179764] compat {c=[1],r=[1],i=[1fff]}] 2024-04-12T21:15:11.782 INFO:teuthology.orchestra.run.smithi018.stdout:[mds.cephfs.smithi018.szjluo{-1:34468} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.18:6826/2407992653,v1:172.21.15.18:6827/2407992653] compat {c=[1],r=[1],i=[1fff]}] 2024-04-12T21:15:11.783 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:11.785+0000 7fc240c00700 1 -- 172.21.15.18:0/2295078729 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fc22006ec00 msgr2=0x7fc2200710b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:15:11.783 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:11.785+0000 7fc240c00700 1 --2- 172.21.15.18:0/2295078729 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fc22006ec00 0x7fc2200710b0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fc228006fd0 tx=0x7fc228008040 comp rx=0 tx=0).stop 2024-04-12T21:15:11.783 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:11.785+0000 7fc240c00700 1 -- 172.21.15.18:0/2295078729 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc23c104e60 msgr2=0x7fc23c115b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:15:11.783 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:11.785+0000 7fc240c00700 1 --2- 172.21.15.18:0/2295078729 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc23c104e60 0x7fc23c115b80 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7fc22c009180 tx=0x7fc22c009e30 comp rx=0 tx=0).stop 2024-04-12T21:15:11.783 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:11.785+0000 7fc240c00700 1 -- 172.21.15.18:0/2295078729 shutdown_connections 2024-04-12T21:15:11.783 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:11.785+0000 7fc240c00700 1 --2- 172.21.15.18:0/2295078729 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fc22006ec00 0x7fc2200710b0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:15:11.783 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:11.785+0000 7fc240c00700 1 --2- 172.21.15.18:0/2295078729 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc23c104e60 0x7fc23c115b80 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:15:11.783 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:11.785+0000 7fc240c00700 1 --2- 172.21.15.18:0/2295078729 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc23c105800 0x7fc23c1160c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:15:11.783 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:11.785+0000 7fc240c00700 1 -- 172.21.15.18:0/2295078729 >> 172.21.15.18:0/2295078729 conn(0x7fc23c0fab10 msgr2=0x7fc23c0fca80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:15:11.783 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:11.785+0000 7fc240c00700 1 -- 172.21.15.18:0/2295078729 shutdown_connections 2024-04-12T21:15:11.784 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:11.785+0000 7fc240c00700 1 -- 172.21.15.18:0/2295078729 wait complete. 2024-04-12T21:15:11.784 INFO:teuthology.orchestra.run.smithi018.stderr:dumped fsmap epoch 29 2024-04-12T21:15:11.929 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:11.933+0000 7fac9e800700 1 -- 172.21.15.18:0/1539221893 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fac98102e00 msgr2=0x7fac981031d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:15:11.930 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:11.933+0000 7fac9e800700 1 --2- 172.21.15.18:0/1539221893 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fac98102e00 0x7fac981031d0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7fac88004940 tx=0x7fac88009e30 comp rx=0 tx=0).stop 2024-04-12T21:15:11.930 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:11.933+0000 7fac9e800700 1 -- 172.21.15.18:0/1539221893 shutdown_connections 2024-04-12T21:15:11.930 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:11.933+0000 7fac9e800700 1 --2- 172.21.15.18:0/1539221893 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fac981037a0 0x7fac980fce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:15:11.930 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:11.933+0000 7fac9e800700 1 --2- 172.21.15.18:0/1539221893 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fac98102e00 0x7fac981031d0 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:15:11.930 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:11.933+0000 7fac9e800700 1 -- 172.21.15.18:0/1539221893 >> 172.21.15.18:0/1539221893 conn(0x7fac980f89b0 msgr2=0x7fac980fadc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:15:11.930 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:11.933+0000 7fac9e800700 1 -- 172.21.15.18:0/1539221893 shutdown_connections 2024-04-12T21:15:11.930 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:11.933+0000 7fac9e800700 1 -- 172.21.15.18:0/1539221893 wait complete. 2024-04-12T21:15:11.932 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:11.934+0000 7fac9e800700 1 Processor -- start 2024-04-12T21:15:11.932 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:11.934+0000 7fac9e800700 1 -- start start 2024-04-12T21:15:11.932 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:11.934+0000 7fac9e800700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fac98102e00 0x7fac98115bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:15:11.932 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:11.934+0000 7fac9e800700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fac981037a0 0x7fac981160f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:15:11.932 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:11.934+0000 7fac9e800700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac98116740 con 0x7fac98102e00 2024-04-12T21:15:11.932 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:11.934+0000 7fac9e800700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac9810fd20 con 0x7fac981037a0 2024-04-12T21:15:11.932 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:11.935+0000 7fac97400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fac98102e00 0x7fac98115bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:15:11.932 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:11.935+0000 7fac97400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fac98102e00 0x7fac98115bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:40492/0 (socket says 172.21.15.18:40492) 2024-04-12T21:15:11.932 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:11.935+0000 7fac97400700 1 -- 172.21.15.18:0/3413352378 learned_addr learned my addr 172.21.15.18:0/3413352378 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:15:11.932 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:11.935+0000 7fac96a00700 1 --2- 172.21.15.18:0/3413352378 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fac981037a0 0x7fac981160f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:15:11.933 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:11.935+0000 7fac97400700 1 -- 172.21.15.18:0/3413352378 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fac981037a0 msgr2=0x7fac981160f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:15:11.933 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:11.935+0000 7fac97400700 1 --2- 172.21.15.18:0/3413352378 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fac981037a0 0x7fac981160f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:15:11.933 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:11.935+0000 7fac97400700 1 -- 172.21.15.18:0/3413352378 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac88005040 con 0x7fac98102e00 2024-04-12T21:15:11.933 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:11.935+0000 7fac96a00700 1 --2- 172.21.15.18:0/3413352378 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fac981037a0 0x7fac981160f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-12T21:15:11.933 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:11.936+0000 7fac97400700 1 --2- 172.21.15.18:0/3413352378 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fac98102e00 0x7fac98115bb0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7fac88009180 tx=0x7fac88009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:15:11.933 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:11.936+0000 7fac8f400700 1 -- 172.21.15.18:0/3413352378 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fac8800e040 con 0x7fac98102e00 2024-04-12T21:15:11.933 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:11.936+0000 7fac8f400700 1 -- 172.21.15.18:0/3413352378 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fac8800bbc0 con 0x7fac98102e00 2024-04-12T21:15:11.933 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:11.936+0000 7fac9e800700 1 -- 172.21.15.18:0/3413352378 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fac9810ffa0 con 0x7fac98102e00 2024-04-12T21:15:11.933 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:11.936+0000 7fac8f400700 1 -- 172.21.15.18:0/3413352378 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fac8801c630 con 0x7fac98102e00 2024-04-12T21:15:11.933 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:11.936+0000 7fac9e800700 1 -- 172.21.15.18:0/3413352378 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fac98110560 con 0x7fac98102e00 2024-04-12T21:15:11.934 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:11.937+0000 7fac9e800700 1 -- 172.21.15.18:0/3413352378 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fac98106a50 con 0x7fac98102e00 2024-04-12T21:15:11.934 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:11.938+0000 7fac8f400700 1 -- 172.21.15.18:0/3413352378 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7fac8800bd30 con 0x7fac98102e00 2024-04-12T21:15:11.935 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:11.939+0000 7fac8f400700 1 --2- 172.21.15.18:0/3413352378 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fac7c06ec00 0x7fac7c0710b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:15:11.935 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:11.939+0000 7fac8f400700 1 -- 172.21.15.18:0/3413352378 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 6757+0+0 (secure 0 0 0) 0x7fac8808fe30 con 0x7fac98102e00 2024-04-12T21:15:11.935 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:11.939+0000 7fac96a00700 1 --2- 172.21.15.18:0/3413352378 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fac7c06ec00 0x7fac7c0710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:15:11.936 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:11.940+0000 7fac96a00700 1 --2- 172.21.15.18:0/3413352378 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fac7c06ec00 0x7fac7c0710b0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fac84009fd0 tx=0x7fac8400b040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:15:11.940 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:11.944+0000 7fac8f400700 1 -- 172.21.15.18:0/3413352378 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7fac8805c5e0 con 0x7fac98102e00 2024-04-12T21:15:12.189 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:12.193+0000 7fac9e800700 1 -- 172.21.15.18:0/3413352378 --> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fac98109f70 con 0x7fac7c06ec00 2024-04-12T21:15:12.191 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:12.195+0000 7fac8f400700 1 -- 172.21.15.18:0/3413352378 <== mgr.34102 v2:172.21.15.18:6800/3812891919 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7fac98109f70 con 0x7fac7c06ec00 2024-04-12T21:15:12.191 INFO:teuthology.orchestra.run.smithi018.stdout:{ 2024-04-12T21:15:12.191 INFO:teuthology.orchestra.run.smithi018.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:f4452def3be4d350f4a8e5feaf7ebb1511be9298fe68478ba7a2d6fbd7d69a63", 2024-04-12T21:15:12.191 INFO:teuthology.orchestra.run.smithi018.stdout: "in_progress": true, 2024-04-12T21:15:12.191 INFO:teuthology.orchestra.run.smithi018.stdout: "which": "Upgrading all daemon types on all hosts", 2024-04-12T21:15:12.192 INFO:teuthology.orchestra.run.smithi018.stdout: "services_complete": [ 2024-04-12T21:15:12.192 INFO:teuthology.orchestra.run.smithi018.stdout: "crash", 2024-04-12T21:15:12.192 INFO:teuthology.orchestra.run.smithi018.stdout: "osd", 2024-04-12T21:15:12.192 INFO:teuthology.orchestra.run.smithi018.stdout: "mon", 2024-04-12T21:15:12.192 INFO:teuthology.orchestra.run.smithi018.stdout: "mgr" 2024-04-12T21:15:12.192 INFO:teuthology.orchestra.run.smithi018.stdout: ], 2024-04-12T21:15:12.192 INFO:teuthology.orchestra.run.smithi018.stdout: "progress": "14/21 daemons upgraded", 2024-04-12T21:15:12.192 INFO:teuthology.orchestra.run.smithi018.stdout: "message": "Currently upgrading mds daemons", 2024-04-12T21:15:12.192 INFO:teuthology.orchestra.run.smithi018.stdout: "is_paused": false 2024-04-12T21:15:12.192 INFO:teuthology.orchestra.run.smithi018.stdout:} 2024-04-12T21:15:12.194 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:12.198+0000 7fac9e800700 1 -- 172.21.15.18:0/3413352378 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fac7c06ec00 msgr2=0x7fac7c0710b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:15:12.194 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:12.198+0000 7fac9e800700 1 --2- 172.21.15.18:0/3413352378 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fac7c06ec00 0x7fac7c0710b0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fac84009fd0 tx=0x7fac8400b040 comp rx=0 tx=0).stop 2024-04-12T21:15:12.194 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:12.198+0000 7fac9e800700 1 -- 172.21.15.18:0/3413352378 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fac98102e00 msgr2=0x7fac98115bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:15:12.194 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:12.198+0000 7fac9e800700 1 --2- 172.21.15.18:0/3413352378 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fac98102e00 0x7fac98115bb0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7fac88009180 tx=0x7fac88009e30 comp rx=0 tx=0).stop 2024-04-12T21:15:12.195 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:12.198+0000 7fac9e800700 1 -- 172.21.15.18:0/3413352378 shutdown_connections 2024-04-12T21:15:12.195 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:12.198+0000 7fac9e800700 1 --2- 172.21.15.18:0/3413352378 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fac7c06ec00 0x7fac7c0710b0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:15:12.195 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:12.198+0000 7fac9e800700 1 --2- 172.21.15.18:0/3413352378 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fac98102e00 0x7fac98115bb0 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:15:12.195 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:12.198+0000 7fac9e800700 1 --2- 172.21.15.18:0/3413352378 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fac981037a0 0x7fac981160f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:15:12.195 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:12.198+0000 7fac9e800700 1 -- 172.21.15.18:0/3413352378 >> 172.21.15.18:0/3413352378 conn(0x7fac980f89b0 msgr2=0x7fac981073a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:15:12.195 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:12.198+0000 7fac9e800700 1 -- 172.21.15.18:0/3413352378 shutdown_connections 2024-04-12T21:15:12.195 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:12.198+0000 7fac9e800700 1 -- 172.21.15.18:0/3413352378 wait complete. 2024-04-12T21:15:12.347 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:12.350+0000 7f8bfe400700 1 -- 172.21.15.18:0/4091408052 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8bf80fde90 msgr2=0x7f8bf80fe260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:15:12.347 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:12.350+0000 7f8bfe400700 1 --2- 172.21.15.18:0/4091408052 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8bf80fde90 0x7f8bf80fe260 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f8be8009ce0 tx=0x7f8be8004d10 comp rx=0 tx=0).stop 2024-04-12T21:15:12.347 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:12.350+0000 7f8bfe400700 1 -- 172.21.15.18:0/4091408052 shutdown_connections 2024-04-12T21:15:12.347 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:12.351+0000 7f8bfe400700 1 --2- 172.21.15.18:0/4091408052 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8bf80fe830 0x7f8bf8106f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:15:12.348 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:12.351+0000 7f8bfe400700 1 --2- 172.21.15.18:0/4091408052 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8bf80fde90 0x7f8bf80fe260 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:15:12.348 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:12.351+0000 7f8bfe400700 1 -- 172.21.15.18:0/4091408052 >> 172.21.15.18:0/4091408052 conn(0x7f8bf8072bc0 msgr2=0x7f8bf8072fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:15:12.348 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:12.351+0000 7f8bfe400700 1 -- 172.21.15.18:0/4091408052 shutdown_connections 2024-04-12T21:15:12.348 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:12.351+0000 7f8bfe400700 1 -- 172.21.15.18:0/4091408052 wait complete. 2024-04-12T21:15:12.349 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:12.352+0000 7f8bfe400700 1 Processor -- start 2024-04-12T21:15:12.349 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:12.352+0000 7f8bfe400700 1 -- start start 2024-04-12T21:15:12.349 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:12.352+0000 7f8bfe400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8bf80fde90 0x7f8bf8115d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:15:12.350 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:12.352+0000 7f8bfe400700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8bf80fe830 0x7f8bf81162c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:15:12.350 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:12.352+0000 7f8bfe400700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8bf8116910 con 0x7f8bf80fde90 2024-04-12T21:15:12.350 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:12.352+0000 7f8bfe400700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8bf810fd20 con 0x7f8bf80fe830 2024-04-12T21:15:12.350 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:12.353+0000 7f8bf7400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8bf80fde90 0x7f8bf8115d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:15:12.350 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:12.353+0000 7f8bf7400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8bf80fde90 0x7f8bf8115d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:40512/0 (socket says 172.21.15.18:40512) 2024-04-12T21:15:12.350 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:12.353+0000 7f8bf7400700 1 -- 172.21.15.18:0/2257337573 learned_addr learned my addr 172.21.15.18:0/2257337573 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:15:12.350 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:12.353+0000 7f8bf7400700 1 -- 172.21.15.18:0/2257337573 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8bf80fe830 msgr2=0x7f8bf81162c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:15:12.350 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:12.353+0000 7f8befe00700 1 --2- 172.21.15.18:0/2257337573 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8bf80fe830 0x7f8bf81162c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:15:12.350 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:12.353+0000 7f8bf7400700 1 --2- 172.21.15.18:0/2257337573 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8bf80fe830 0x7f8bf81162c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:15:12.350 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:12.353+0000 7f8bf7400700 1 -- 172.21.15.18:0/2257337573 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8be8009940 con 0x7f8bf80fde90 2024-04-12T21:15:12.351 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:12.354+0000 7f8bf7400700 1 --2- 172.21.15.18:0/2257337573 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8bf80fde90 0x7f8bf8115d80 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f8be800b740 tx=0x7f8be8004d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:15:12.351 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:12.354+0000 7f8bef400700 1 -- 172.21.15.18:0/2257337573 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8be800e040 con 0x7f8bf80fde90 2024-04-12T21:15:12.352 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:12.354+0000 7f8bef400700 1 -- 172.21.15.18:0/2257337573 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f8be80128e0 con 0x7f8bf80fde90 2024-04-12T21:15:12.352 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:12.354+0000 7f8bef400700 1 -- 172.21.15.18:0/2257337573 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8be801c850 con 0x7f8bf80fde90 2024-04-12T21:15:12.352 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:12.354+0000 7f8bfe400700 1 -- 172.21.15.18:0/2257337573 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8bf810ffa0 con 0x7f8bf80fde90 2024-04-12T21:15:12.352 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:12.354+0000 7f8bfe400700 1 -- 172.21.15.18:0/2257337573 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8bf81104e0 con 0x7f8bf80fde90 2024-04-12T21:15:12.352 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:12.355+0000 7f8bfe400700 1 -- 172.21.15.18:0/2257337573 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8bf8106a20 con 0x7f8bf80fde90 2024-04-12T21:15:12.353 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:12.357+0000 7f8bef400700 1 -- 172.21.15.18:0/2257337573 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7f8be8012410 con 0x7f8bf80fde90 2024-04-12T21:15:12.354 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:12.358+0000 7f8bef400700 1 --2- 172.21.15.18:0/2257337573 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f8bdc06ec10 0x7f8bdc0710c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:15:12.354 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:12.358+0000 7f8bef400700 1 -- 172.21.15.18:0/2257337573 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 6757+0+0 (secure 0 0 0) 0x7f8be8090780 con 0x7f8bf80fde90 2024-04-12T21:15:12.354 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:12.358+0000 7f8befe00700 1 --2- 172.21.15.18:0/2257337573 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f8bdc06ec10 0x7f8bdc0710c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:15:12.355 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:12.359+0000 7f8befe00700 1 --2- 172.21.15.18:0/2257337573 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f8bdc06ec10 0x7f8bdc0710c0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f8bf8111660 tx=0x7f8be0009040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:15:12.358 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:12.362+0000 7f8bef400700 1 -- 172.21.15.18:0/2257337573 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f8be8059380 con 0x7f8bf80fde90 2024-04-12T21:15:12.599 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:15:12 smithi018 ceph-mon[112590]: from='client.34844 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:15:12.600 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:15:12 smithi018 ceph-mon[112590]: from='client.? 172.21.15.18:0/2295078729' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-12T21:15:12.673 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:12.677+0000 7f8bfe400700 1 -- 172.21.15.18:0/2257337573 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f8bf8065e30 con 0x7f8bf80fde90 2024-04-12T21:15:12.674 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:12.678+0000 7f8bef400700 1 -- 172.21.15.18:0/2257337573 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+233 (secure 0 0 0) 0x7f8be805ca40 con 0x7f8bf80fde90 2024-04-12T21:15:12.674 INFO:teuthology.orchestra.run.smithi018.stdout:HEALTH_ERR 1 filesystem is degraded; 1 filesystem is offline 2024-04-12T21:15:12.674 INFO:teuthology.orchestra.run.smithi018.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-04-12T21:15:12.675 INFO:teuthology.orchestra.run.smithi018.stdout: fs cephfs is degraded 2024-04-12T21:15:12.675 INFO:teuthology.orchestra.run.smithi018.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-04-12T21:15:12.675 INFO:teuthology.orchestra.run.smithi018.stdout: fs cephfs is offline because no MDS is active for it. 2024-04-12T21:15:12.677 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:12.681+0000 7f8bfe400700 1 -- 172.21.15.18:0/2257337573 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f8bdc06ec10 msgr2=0x7f8bdc0710c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:15:12.677 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:12.681+0000 7f8bfe400700 1 --2- 172.21.15.18:0/2257337573 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f8bdc06ec10 0x7f8bdc0710c0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f8bf8111660 tx=0x7f8be0009040 comp rx=0 tx=0).stop 2024-04-12T21:15:12.677 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:12.681+0000 7f8bfe400700 1 -- 172.21.15.18:0/2257337573 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8bf80fde90 msgr2=0x7f8bf8115d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:15:12.677 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:12.681+0000 7f8bfe400700 1 --2- 172.21.15.18:0/2257337573 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8bf80fde90 0x7f8bf8115d80 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f8be800b740 tx=0x7f8be8004d10 comp rx=0 tx=0).stop 2024-04-12T21:15:12.677 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:12.681+0000 7f8bfe400700 1 -- 172.21.15.18:0/2257337573 shutdown_connections 2024-04-12T21:15:12.678 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:12.681+0000 7f8bfe400700 1 --2- 172.21.15.18:0/2257337573 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f8bdc06ec10 0x7f8bdc0710c0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:15:12.678 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:12.681+0000 7f8bfe400700 1 --2- 172.21.15.18:0/2257337573 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8bf80fde90 0x7f8bf8115d80 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:15:12.678 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:12.681+0000 7f8bfe400700 1 --2- 172.21.15.18:0/2257337573 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8bf80fe830 0x7f8bf81162c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:15:12.678 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:12.681+0000 7f8bfe400700 1 -- 172.21.15.18:0/2257337573 >> 172.21.15.18:0/2257337573 conn(0x7f8bf8072bc0 msgr2=0x7f8bf80fb840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:15:12.678 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:12.681+0000 7f8bfe400700 1 -- 172.21.15.18:0/2257337573 shutdown_connections 2024-04-12T21:15:12.678 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:12.681+0000 7f8bfe400700 1 -- 172.21.15.18:0/2257337573 wait complete. 2024-04-12T21:15:12.779 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:15:12 smithi186 ceph-mon[86379]: from='client.34844 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:15:12.779 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:15:12 smithi186 ceph-mon[86379]: from='client.? 172.21.15.18:0/2295078729' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-12T21:15:13.779 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:15:13 smithi186 ceph-mon[86379]: pgmap v462: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:15:13.779 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:15:13 smithi186 ceph-mon[86379]: from='client.34856 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:15:13.779 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:15:13 smithi186 ceph-mon[86379]: from='client.? 172.21.15.18:0/2257337573' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-12T21:15:13.850 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:15:13 smithi018 ceph-mon[112590]: pgmap v462: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:15:13.850 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:15:13 smithi018 ceph-mon[112590]: from='client.34856 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:15:13.850 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:15:13 smithi018 ceph-mon[112590]: from='client.? 172.21.15.18:0/2257337573' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-12T21:15:15.778 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:15:15 smithi186 ceph-mon[86379]: pgmap v463: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:15:15.850 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:15:15 smithi018 ceph-mon[112590]: pgmap v463: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:15:17.778 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:15:17 smithi186 ceph-mon[86379]: pgmap v464: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:15:17.849 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:15:17 smithi018 ceph-mon[112590]: pgmap v464: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:15:19.779 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:15:19 smithi186 ceph-mon[86379]: pgmap v465: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:15:19.849 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:15:19 smithi018 ceph-mon[112590]: pgmap v465: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:15:21.778 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:15:21 smithi186 ceph-mon[86379]: pgmap v466: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:15:21.849 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:15:21 smithi018 ceph-mon[112590]: pgmap v466: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:15:23.778 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:15:23 smithi186 ceph-mon[86379]: pgmap v467: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:15:23.849 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:15:23 smithi018 ceph-mon[112590]: pgmap v467: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:15:25.778 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:15:25 smithi186 ceph-mon[86379]: pgmap v468: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:15:25.779 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:15:25 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T21:15:25.849 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:15:25 smithi018 ceph-mon[112590]: pgmap v468: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:15:25.850 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:15:25 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T21:15:27.778 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:15:27 smithi186 ceph-mon[86379]: pgmap v469: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:15:27.849 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:15:27 smithi018 ceph-mon[112590]: pgmap v469: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:15:29.778 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:15:29 smithi186 ceph-mon[86379]: pgmap v470: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:15:29.849 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:15:29 smithi018 ceph-mon[112590]: pgmap v470: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:15:31.779 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:15:31 smithi186 ceph-mon[86379]: pgmap v471: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:15:31.849 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:15:31 smithi018 ceph-mon[112590]: pgmap v471: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:15:33.778 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:15:33 smithi186 ceph-mon[86379]: pgmap v472: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:15:33.849 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:15:33 smithi018 ceph-mon[112590]: pgmap v472: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:15:35.778 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:15:35 smithi186 ceph-mon[86379]: pgmap v473: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:15:35.849 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:15:35 smithi018 ceph-mon[112590]: pgmap v473: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:15:37.778 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:15:37 smithi186 ceph-mon[86379]: pgmap v474: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:15:37.849 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:15:37 smithi018 ceph-mon[112590]: pgmap v474: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:15:39.778 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:15:39 smithi186 ceph-mon[86379]: pgmap v475: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:15:39.849 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:15:39 smithi018 ceph-mon[112590]: pgmap v475: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:15:40.751 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:15:40 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T21:15:40.778 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:15:40 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T21:15:41.778 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:15:41 smithi186 ceph-mon[86379]: pgmap v476: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:15:41.849 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:15:41 smithi018 ceph-mon[112590]: pgmap v476: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:15:42.850 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:42.853+0000 7fef59a00700 1 -- 172.21.15.18:0/1880147364 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fef54072e00 msgr2=0x7fef5410d630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:15:42.850 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:42.853+0000 7fef59a00700 1 --2- 172.21.15.18:0/1880147364 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fef54072e00 0x7fef5410d630 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7fef40004940 tx=0x7fef40009e30 comp rx=0 tx=0).stop 2024-04-12T21:15:42.850 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:42.853+0000 7fef59a00700 1 -- 172.21.15.18:0/1880147364 shutdown_connections 2024-04-12T21:15:42.850 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:42.853+0000 7fef59a00700 1 --2- 172.21.15.18:0/1880147364 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fef54072e00 0x7fef5410d630 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:15:42.850 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:42.853+0000 7fef59a00700 1 --2- 172.21.15.18:0/1880147364 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fef540724f0 0x7fef540728c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:15:42.850 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:42.853+0000 7fef59a00700 1 -- 172.21.15.18:0/1880147364 >> 172.21.15.18:0/1880147364 conn(0x7fef540fab10 msgr2=0x7fef540fcef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:15:42.850 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:42.853+0000 7fef59a00700 1 -- 172.21.15.18:0/1880147364 shutdown_connections 2024-04-12T21:15:42.850 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:42.853+0000 7fef59a00700 1 -- 172.21.15.18:0/1880147364 wait complete. 2024-04-12T21:15:42.852 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:42.854+0000 7fef59a00700 1 Processor -- start 2024-04-12T21:15:42.852 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:42.855+0000 7fef59a00700 1 -- start start 2024-04-12T21:15:42.853 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:42.855+0000 7fef59a00700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fef540724f0 0x7fef54119f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:15:42.853 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:42.855+0000 7fef59a00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fef54072e00 0x7fef5411a4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:15:42.853 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:42.855+0000 7fef59a00700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef5411ab10 con 0x7fef54072e00 2024-04-12T21:15:42.853 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:42.855+0000 7fef59a00700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef54113ee0 con 0x7fef540724f0 2024-04-12T21:15:42.853 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:42.855+0000 7fef53400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fef54072e00 0x7fef5411a4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:15:42.853 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:42.856+0000 7fef53400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fef54072e00 0x7fef5411a4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:46826/0 (socket says 172.21.15.18:46826) 2024-04-12T21:15:42.853 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:42.856+0000 7fef53400700 1 -- 172.21.15.18:0/511427049 learned_addr learned my addr 172.21.15.18:0/511427049 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:15:42.853 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:42.856+0000 7fef53e00700 1 --2- 172.21.15.18:0/511427049 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fef540724f0 0x7fef54119f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:15:42.853 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:42.856+0000 7fef53400700 1 -- 172.21.15.18:0/511427049 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fef540724f0 msgr2=0x7fef54119f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:15:42.853 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:42.856+0000 7fef53400700 1 --2- 172.21.15.18:0/511427049 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fef540724f0 0x7fef54119f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:15:42.854 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:42.856+0000 7fef53400700 1 -- 172.21.15.18:0/511427049 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef40005040 con 0x7fef54072e00 2024-04-12T21:15:42.854 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:42.856+0000 7fef53e00700 1 --2- 172.21.15.18:0/511427049 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fef540724f0 0x7fef54119f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-12T21:15:42.854 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:42.856+0000 7fef53400700 1 --2- 172.21.15.18:0/511427049 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fef54072e00 0x7fef5411a4c0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7fef40009180 tx=0x7fef40009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:15:42.855 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:42.857+0000 7fef4be00700 1 -- 172.21.15.18:0/511427049 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fef4000e040 con 0x7fef54072e00 2024-04-12T21:15:42.855 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:42.857+0000 7fef4be00700 1 -- 172.21.15.18:0/511427049 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fef4000bbc0 con 0x7fef54072e00 2024-04-12T21:15:42.855 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:42.857+0000 7fef4be00700 1 -- 172.21.15.18:0/511427049 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fef4001c630 con 0x7fef54072e00 2024-04-12T21:15:42.855 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:42.857+0000 7fef59a00700 1 -- 172.21.15.18:0/511427049 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fef54114160 con 0x7fef54072e00 2024-04-12T21:15:42.855 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:42.857+0000 7fef59a00700 1 -- 172.21.15.18:0/511427049 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fef54114720 con 0x7fef54072e00 2024-04-12T21:15:42.855 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:42.858+0000 7fef59a00700 1 -- 172.21.15.18:0/511427049 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fef54073be0 con 0x7fef54072e00 2024-04-12T21:15:42.855 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:42.859+0000 7fef4be00700 1 -- 172.21.15.18:0/511427049 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7fef4000bd30 con 0x7fef54072e00 2024-04-12T21:15:42.857 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:42.861+0000 7fef4be00700 1 --2- 172.21.15.18:0/511427049 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fef3806ec00 0x7fef380710b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:15:42.857 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:42.861+0000 7fef4be00700 1 -- 172.21.15.18:0/511427049 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 6757+0+0 (secure 0 0 0) 0x7fef4008fcb0 con 0x7fef54072e00 2024-04-12T21:15:42.857 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:42.861+0000 7fef53e00700 1 --2- 172.21.15.18:0/511427049 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fef3806ec00 0x7fef380710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:15:42.858 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:42.862+0000 7fef53e00700 1 --2- 172.21.15.18:0/511427049 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fef3806ec00 0x7fef380710b0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fef44007fd0 tx=0x7fef44009040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:15:42.862 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:42.866+0000 7fef4be00700 1 -- 172.21.15.18:0/511427049 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7fef4005c5a0 con 0x7fef54072e00 2024-04-12T21:15:43.117 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:43.121+0000 7fef59a00700 1 -- 172.21.15.18:0/511427049 --> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fef54110350 con 0x7fef3806ec00 2024-04-12T21:15:43.120 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:43.124+0000 7fef4be00700 1 -- 172.21.15.18:0/511427049 <== mgr.34102 v2:172.21.15.18:6800/3812891919 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7fef54110350 con 0x7fef3806ec00 2024-04-12T21:15:43.123 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:43.126+0000 7fef59a00700 1 -- 172.21.15.18:0/511427049 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fef3806ec00 msgr2=0x7fef380710b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:15:43.123 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:43.126+0000 7fef59a00700 1 --2- 172.21.15.18:0/511427049 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fef3806ec00 0x7fef380710b0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fef44007fd0 tx=0x7fef44009040 comp rx=0 tx=0).stop 2024-04-12T21:15:43.123 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:43.126+0000 7fef59a00700 1 -- 172.21.15.18:0/511427049 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fef54072e00 msgr2=0x7fef5411a4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:15:43.123 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:43.126+0000 7fef59a00700 1 --2- 172.21.15.18:0/511427049 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fef54072e00 0x7fef5411a4c0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7fef40009180 tx=0x7fef40009e30 comp rx=0 tx=0).stop 2024-04-12T21:15:43.124 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:43.127+0000 7fef59a00700 1 -- 172.21.15.18:0/511427049 shutdown_connections 2024-04-12T21:15:43.124 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:43.127+0000 7fef59a00700 1 --2- 172.21.15.18:0/511427049 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fef3806ec00 0x7fef380710b0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:15:43.124 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:43.127+0000 7fef59a00700 1 --2- 172.21.15.18:0/511427049 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fef540724f0 0x7fef54119f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:15:43.124 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:43.127+0000 7fef59a00700 1 --2- 172.21.15.18:0/511427049 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fef54072e00 0x7fef5411a4c0 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:15:43.124 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:43.127+0000 7fef59a00700 1 -- 172.21.15.18:0/511427049 >> 172.21.15.18:0/511427049 conn(0x7fef540fab10 msgr2=0x7fef540faee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:15:43.124 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:43.127+0000 7fef59a00700 1 -- 172.21.15.18:0/511427049 shutdown_connections 2024-04-12T21:15:43.124 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:43.127+0000 7fef59a00700 1 -- 172.21.15.18:0/511427049 wait complete. 2024-04-12T21:15:43.141 INFO:teuthology.orchestra.run.smithi018.stdout:true 2024-04-12T21:15:43.288 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:43.291+0000 7f4ad7000700 1 -- 172.21.15.18:0/3330959391 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4ad0109430 msgr2=0x7f4ad0109800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:15:43.288 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:43.291+0000 7f4ad7000700 1 --2- 172.21.15.18:0/3330959391 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4ad0109430 0x7f4ad0109800 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f4ac0004940 tx=0x7f4ac0009e30 comp rx=0 tx=0).stop 2024-04-12T21:15:43.288 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:43.291+0000 7f4ad7000700 1 -- 172.21.15.18:0/3330959391 shutdown_connections 2024-04-12T21:15:43.288 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:43.291+0000 7f4ad7000700 1 --2- 172.21.15.18:0/3330959391 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4ad0109dd0 0x7f4ad0103440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:15:43.288 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:43.291+0000 7f4ad7000700 1 --2- 172.21.15.18:0/3330959391 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4ad0109430 0x7f4ad0109800 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:15:43.288 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:43.291+0000 7f4ad7000700 1 -- 172.21.15.18:0/3330959391 >> 172.21.15.18:0/3330959391 conn(0x7f4ad0072c00 msgr2=0x7f4ad0075010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:15:43.288 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:43.291+0000 7f4ad7000700 1 -- 172.21.15.18:0/3330959391 shutdown_connections 2024-04-12T21:15:43.289 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:43.291+0000 7f4ad7000700 1 -- 172.21.15.18:0/3330959391 wait complete. 2024-04-12T21:15:43.290 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:43.292+0000 7f4ad7000700 1 Processor -- start 2024-04-12T21:15:43.290 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:43.293+0000 7f4ad7000700 1 -- start start 2024-04-12T21:15:43.291 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:43.293+0000 7f4ad7000700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4ad0109430 0x7f4ad0119f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:15:43.291 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:43.293+0000 7f4ad7000700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4ad0109dd0 0x7f4ad011a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:15:43.291 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:43.293+0000 7f4ad7000700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ad011aad0 con 0x7f4ad0109430 2024-04-12T21:15:43.291 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:43.293+0000 7f4ad7000700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ad0114100 con 0x7f4ad0109dd0 2024-04-12T21:15:43.291 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:43.293+0000 7f4acfe00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4ad0109430 0x7f4ad0119f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:15:43.291 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:43.293+0000 7f4acfe00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4ad0109430 0x7f4ad0119f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:46848/0 (socket says 172.21.15.18:46848) 2024-04-12T21:15:43.291 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:43.293+0000 7f4acfe00700 1 -- 172.21.15.18:0/3314572545 learned_addr learned my addr 172.21.15.18:0/3314572545 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:15:43.291 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:43.294+0000 7f4acfe00700 1 -- 172.21.15.18:0/3314572545 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4ad0109dd0 msgr2=0x7f4ad011a480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:15:43.291 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:43.294+0000 7f4ac7e00700 1 --2- 172.21.15.18:0/3314572545 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4ad0109dd0 0x7f4ad011a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:15:43.291 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:43.294+0000 7f4acfe00700 1 --2- 172.21.15.18:0/3314572545 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4ad0109dd0 0x7f4ad011a480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:15:43.291 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:43.294+0000 7f4acfe00700 1 -- 172.21.15.18:0/3314572545 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ac0005040 con 0x7f4ad0109430 2024-04-12T21:15:43.291 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:43.294+0000 7f4acfe00700 1 --2- 172.21.15.18:0/3314572545 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4ad0109430 0x7f4ad0119f40 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f4ac0009180 tx=0x7f4ac0009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:15:43.291 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:43.294+0000 7f4accc00700 1 -- 172.21.15.18:0/3314572545 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4ac000e040 con 0x7f4ad0109430 2024-04-12T21:15:43.291 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:43.294+0000 7f4accc00700 1 -- 172.21.15.18:0/3314572545 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f4ac000bbc0 con 0x7f4ad0109430 2024-04-12T21:15:43.291 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:43.294+0000 7f4ad7000700 1 -- 172.21.15.18:0/3314572545 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4ad0114380 con 0x7f4ad0109430 2024-04-12T21:15:43.292 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:43.294+0000 7f4accc00700 1 -- 172.21.15.18:0/3314572545 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4ac001c630 con 0x7f4ad0109430 2024-04-12T21:15:43.292 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:43.295+0000 7f4ad7000700 1 -- 172.21.15.18:0/3314572545 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4ad0114940 con 0x7f4ad0109430 2024-04-12T21:15:43.292 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:43.296+0000 7f4ad7000700 1 -- 172.21.15.18:0/3314572545 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4ad0104b80 con 0x7f4ad0109430 2024-04-12T21:15:43.293 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:43.297+0000 7f4accc00700 1 -- 172.21.15.18:0/3314572545 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7f4ac000bd30 con 0x7f4ad0109430 2024-04-12T21:15:43.294 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:43.298+0000 7f4accc00700 1 --2- 172.21.15.18:0/3314572545 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f4ab406ec00 0x7f4ab40710b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:15:43.294 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:43.298+0000 7f4accc00700 1 -- 172.21.15.18:0/3314572545 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 6757+0+0 (secure 0 0 0) 0x7f4ac008fc80 con 0x7f4ad0109430 2024-04-12T21:15:43.295 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:43.299+0000 7f4ac7e00700 1 --2- 172.21.15.18:0/3314572545 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f4ab406ec00 0x7f4ab40710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:15:43.295 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:43.299+0000 7f4ac7e00700 1 --2- 172.21.15.18:0/3314572545 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f4ab406ec00 0x7f4ab40710b0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f4ad01157e0 tx=0x7f4ab800b040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:15:43.299 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:43.303+0000 7f4accc00700 1 -- 172.21.15.18:0/3314572545 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f4ac005c570 con 0x7f4ad0109430 2024-04-12T21:15:43.558 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:43.562+0000 7f4ad7000700 1 -- 172.21.15.18:0/3314572545 --> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f4ad0115630 con 0x7f4ab406ec00 2024-04-12T21:15:43.560 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:43.564+0000 7f4accc00700 1 -- 172.21.15.18:0/3314572545 <== mgr.34102 v2:172.21.15.18:6800/3812891919 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f4ad0115630 con 0x7f4ab406ec00 2024-04-12T21:15:43.563 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:43.566+0000 7f4ad7000700 1 -- 172.21.15.18:0/3314572545 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f4ab406ec00 msgr2=0x7f4ab40710b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:15:43.563 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:43.566+0000 7f4ad7000700 1 --2- 172.21.15.18:0/3314572545 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f4ab406ec00 0x7f4ab40710b0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f4ad01157e0 tx=0x7f4ab800b040 comp rx=0 tx=0).stop 2024-04-12T21:15:43.563 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:43.567+0000 7f4ad7000700 1 -- 172.21.15.18:0/3314572545 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4ad0109430 msgr2=0x7f4ad0119f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:15:43.564 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:43.567+0000 7f4ad7000700 1 --2- 172.21.15.18:0/3314572545 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4ad0109430 0x7f4ad0119f40 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f4ac0009180 tx=0x7f4ac0009e30 comp rx=0 tx=0).stop 2024-04-12T21:15:43.564 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:43.567+0000 7f4ad7000700 1 -- 172.21.15.18:0/3314572545 shutdown_connections 2024-04-12T21:15:43.564 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:43.567+0000 7f4ad7000700 1 --2- 172.21.15.18:0/3314572545 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f4ab406ec00 0x7f4ab40710b0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:15:43.564 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:43.567+0000 7f4ad7000700 1 --2- 172.21.15.18:0/3314572545 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4ad0109430 0x7f4ad0119f40 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:15:43.564 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:43.567+0000 7f4ad7000700 1 --2- 172.21.15.18:0/3314572545 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4ad0109dd0 0x7f4ad011a480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:15:43.564 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:43.567+0000 7f4ad7000700 1 -- 172.21.15.18:0/3314572545 >> 172.21.15.18:0/3314572545 conn(0x7f4ad0072c00 msgr2=0x7f4ad0074c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:15:43.564 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:43.567+0000 7f4ad7000700 1 -- 172.21.15.18:0/3314572545 shutdown_connections 2024-04-12T21:15:43.564 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:43.567+0000 7f4ad7000700 1 -- 172.21.15.18:0/3314572545 wait complete. 2024-04-12T21:15:43.727 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:43.730+0000 7f9025600700 1 -- 172.21.15.18:0/3804333951 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9020109430 msgr2=0x7f9020109800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:15:43.727 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:43.730+0000 7f9025600700 1 --2- 172.21.15.18:0/3804333951 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9020109430 0x7f9020109800 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f9010004940 tx=0x7f9010009e30 comp rx=0 tx=0).stop 2024-04-12T21:15:43.727 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:43.730+0000 7f9025600700 1 -- 172.21.15.18:0/3804333951 shutdown_connections 2024-04-12T21:15:43.727 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:43.730+0000 7f9025600700 1 --2- 172.21.15.18:0/3804333951 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9020109dd0 0x7f9020103440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:15:43.727 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:43.730+0000 7f9025600700 1 --2- 172.21.15.18:0/3804333951 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9020109430 0x7f9020109800 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:15:43.727 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:43.730+0000 7f9025600700 1 -- 172.21.15.18:0/3804333951 >> 172.21.15.18:0/3804333951 conn(0x7f9020072c00 msgr2=0x7f9020075010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:15:43.727 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:43.730+0000 7f9025600700 1 -- 172.21.15.18:0/3804333951 shutdown_connections 2024-04-12T21:15:43.727 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:43.731+0000 7f9025600700 1 -- 172.21.15.18:0/3804333951 wait complete. 2024-04-12T21:15:43.729 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:43.731+0000 7f9025600700 1 Processor -- start 2024-04-12T21:15:43.729 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:43.732+0000 7f9025600700 1 -- start start 2024-04-12T21:15:43.729 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:43.732+0000 7f9025600700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9020109430 0x7f9020119f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:15:43.729 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:43.732+0000 7f9025600700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9020109dd0 0x7f902011a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:15:43.729 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:43.732+0000 7f9025600700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f902011ab60 con 0x7f9020109dd0 2024-04-12T21:15:43.730 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:43.732+0000 7f9025600700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90201140b0 con 0x7f9020109430 2024-04-12T21:15:43.730 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:43.732+0000 7f901e000700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9020109dd0 0x7f902011a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:15:43.730 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:43.733+0000 7f901e000700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9020109dd0 0x7f902011a480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:46874/0 (socket says 172.21.15.18:46874) 2024-04-12T21:15:43.730 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:43.733+0000 7f901e000700 1 -- 172.21.15.18:0/3636062621 learned_addr learned my addr 172.21.15.18:0/3636062621 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:15:43.730 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:43.733+0000 7f901ea00700 1 --2- 172.21.15.18:0/3636062621 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9020109430 0x7f9020119f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:15:43.730 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:43.733+0000 7f901e000700 1 -- 172.21.15.18:0/3636062621 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9020109430 msgr2=0x7f9020119f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:15:43.730 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:43.733+0000 7f901e000700 1 --2- 172.21.15.18:0/3636062621 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9020109430 0x7f9020119f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:15:43.730 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:43.733+0000 7f901e000700 1 -- 172.21.15.18:0/3636062621 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9010005040 con 0x7f9020109dd0 2024-04-12T21:15:43.730 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:43.733+0000 7f901e000700 1 --2- 172.21.15.18:0/3636062621 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9020109dd0 0x7f902011a480 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f900c00b790 tx=0x7f900c009220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:15:43.731 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:43.734+0000 7f9016a00700 1 -- 172.21.15.18:0/3636062621 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f900c00f6f0 con 0x7f9020109dd0 2024-04-12T21:15:43.731 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:43.734+0000 7f9025600700 1 -- 172.21.15.18:0/3636062621 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9020114330 con 0x7f9020109dd0 2024-04-12T21:15:43.731 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:43.734+0000 7f9016a00700 1 -- 172.21.15.18:0/3636062621 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f900c00fd20 con 0x7f9020109dd0 2024-04-12T21:15:43.731 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:43.734+0000 7f9016a00700 1 -- 172.21.15.18:0/3636062621 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f900c0103a0 con 0x7f9020109dd0 2024-04-12T21:15:43.731 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:43.734+0000 7f9025600700 1 -- 172.21.15.18:0/3636062621 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f90201148f0 con 0x7f9020109dd0 2024-04-12T21:15:43.731 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:43.735+0000 7f9025600700 1 -- 172.21.15.18:0/3636062621 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9020104b80 con 0x7f9020109dd0 2024-04-12T21:15:43.732 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:43.736+0000 7f9016a00700 1 -- 172.21.15.18:0/3636062621 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7f900c011400 con 0x7f9020109dd0 2024-04-12T21:15:43.733 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:43.737+0000 7f9016a00700 1 --2- 172.21.15.18:0/3636062621 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f900406ec00 0x7f90040710b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:15:43.733 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:43.737+0000 7f9016a00700 1 -- 172.21.15.18:0/3636062621 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 6757+0+0 (secure 0 0 0) 0x7f900c08e780 con 0x7f9020109dd0 2024-04-12T21:15:43.734 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:43.738+0000 7f901ea00700 1 --2- 172.21.15.18:0/3636062621 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f900406ec00 0x7f90040710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:15:43.734 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:43.738+0000 7f901ea00700 1 --2- 172.21.15.18:0/3636062621 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f900406ec00 0x7f90040710b0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f901000b3b0 tx=0x7f9010009e30 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:15:43.738 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:43.742+0000 7f9016a00700 1 -- 172.21.15.18:0/3636062621 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f900c05afd0 con 0x7f9020109dd0 2024-04-12T21:15:43.778 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:15:43 smithi186 ceph-mon[86379]: pgmap v477: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:15:43.849 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:15:43 smithi018 ceph-mon[112590]: pgmap v477: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:15:43.982 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:43.986+0000 7f9025600700 1 -- 172.21.15.18:0/3636062621 --> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f9020114bc0 con 0x7f900406ec00 2024-04-12T21:15:44.012 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:44.015+0000 7f9016a00700 1 -- 172.21.15.18:0/3636062621 <== mgr.34102 v2:172.21.15.18:6800/3812891919 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7f9020114bc0 con 0x7f900406ec00 2024-04-12T21:15:44.012 INFO:teuthology.orchestra.run.smithi018.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-12T21:15:44.012 INFO:teuthology.orchestra.run.smithi018.stdout:alertmanager.smithi018 smithi018 *:9093,9094 running (16m) 7m ago 30m 14.7M - 0.25.0 c8568f914cd2 c332090068e8 2024-04-12T21:15:44.012 INFO:teuthology.orchestra.run.smithi018.stdout:crash.smithi018 smithi018 running (12m) 7m ago 31m 6665k - 19.0.0-2879-g324baf8d 8717720dc5e5 b7f80d1f7982 2024-04-12T21:15:44.012 INFO:teuthology.orchestra.run.smithi018.stdout:crash.smithi186 smithi186 running (12m) 8m ago 29m 6660k - 19.0.0-2879-g324baf8d 8717720dc5e5 a4f55c81b623 2024-04-12T21:15:44.012 INFO:teuthology.orchestra.run.smithi018.stdout:grafana.smithi018 smithi018 *:3000 running (15m) 7m ago 30m 74.8M - 9.4.12 2bacad6d85d8 b08e3bf176dd 2024-04-12T21:15:44.012 INFO:teuthology.orchestra.run.smithi018.stdout:mds.cephfs.smithi018.acqbsc smithi018 running (8m) 7m ago 23m 11.1M - 19.0.0-2879-g324baf8d 8717720dc5e5 021f99ab24ae 2024-04-12T21:15:44.012 INFO:teuthology.orchestra.run.smithi018.stdout:mds.cephfs.smithi018.szjluo smithi018 running (8m) 7m ago 23m 10.9M - 19.0.0-2879-g324baf8d 8717720dc5e5 0003117479d4 2024-04-12T21:15:44.012 INFO:teuthology.orchestra.run.smithi018.stdout:mds.cephfs.smithi186.jdllyn smithi186 running (23m) 8m ago 23m 15.6M - 17.2.7-827-gd3b5060a 21de9b7369ad 2b2bc46bd29a 2024-04-12T21:15:44.012 INFO:teuthology.orchestra.run.smithi018.stdout:mds.cephfs.smithi186.wjnafs smithi186 running (23m) 8m ago 23m 15.4M - 17.2.7-827-gd3b5060a 21de9b7369ad 74b115d6d6fd 2024-04-12T21:15:44.012 INFO:teuthology.orchestra.run.smithi018.stdout:mgr.smithi018.xjzwga smithi018 *:8443,9283,8765 running (19m) 7m ago 32m 509M - 19.0.0-2879-g324baf8d 8717720dc5e5 56088305eab7 2024-04-12T21:15:44.013 INFO:teuthology.orchestra.run.smithi018.stdout:mgr.smithi186.kqcfxk smithi186 *:8443,9283,8765 running (17m) 8m ago 29m 452M - 19.0.0-2879-g324baf8d 8717720dc5e5 09cd766e5e0d 2024-04-12T21:15:44.013 INFO:teuthology.orchestra.run.smithi018.stdout:mon.smithi018 smithi018 running (14m) 7m ago 32m 57.0M 2048M 19.0.0-2879-g324baf8d 8717720dc5e5 df3e201dfed6 2024-04-12T21:15:44.013 INFO:teuthology.orchestra.run.smithi018.stdout:mon.smithi186 smithi186 running (13m) 8m ago 29m 37.7M 2048M 19.0.0-2879-g324baf8d 8717720dc5e5 c77498da9b2b 2024-04-12T21:15:44.013 INFO:teuthology.orchestra.run.smithi018.stdout:node-exporter.smithi018 smithi018 *:9100 running (17m) 7m ago 31m 7780k - 1.5.0 0da6a335fe13 326c3d9c6847 2024-04-12T21:15:44.013 INFO:teuthology.orchestra.run.smithi018.stdout:node-exporter.smithi186 smithi186 *:9100 running (17m) 8m ago 29m 7826k - 1.5.0 0da6a335fe13 ec2112baeecf 2024-04-12T21:15:44.013 INFO:teuthology.orchestra.run.smithi018.stdout:osd.0 smithi018 running (12m) 7m ago 28m 121M 1559M 19.0.0-2879-g324baf8d 8717720dc5e5 60b1273ca047 2024-04-12T21:15:44.013 INFO:teuthology.orchestra.run.smithi018.stdout:osd.1 smithi018 running (11m) 7m ago 27m 91.3M 1559M 19.0.0-2879-g324baf8d 8717720dc5e5 170ce1e5fb8d 2024-04-12T21:15:44.013 INFO:teuthology.orchestra.run.smithi018.stdout:osd.2 smithi018 running (10m) 7m ago 27m 90.3M 1559M 19.0.0-2879-g324baf8d 8717720dc5e5 c2cbc6aa2839 2024-04-12T21:15:44.013 INFO:teuthology.orchestra.run.smithi018.stdout:osd.3 smithi186 running (10m) 8m ago 26m 96.6M 2581M 19.0.0-2879-g324baf8d 8717720dc5e5 0138e8262e4b 2024-04-12T21:15:44.013 INFO:teuthology.orchestra.run.smithi018.stdout:osd.4 smithi186 running (9m) 8m ago 26m 87.2M 2581M 19.0.0-2879-g324baf8d 8717720dc5e5 b20cb63b25c9 2024-04-12T21:15:44.013 INFO:teuthology.orchestra.run.smithi018.stdout:osd.5 smithi186 running (8m) 8m ago 25m 11.1M 2581M 19.0.0-2879-g324baf8d 8717720dc5e5 2513bea2610e 2024-04-12T21:15:44.013 INFO:teuthology.orchestra.run.smithi018.stdout:prometheus.smithi018 smithi018 *:9095 running (16m) 7m ago 30m 60.6M - 2.43.0 a07b618ecd1d f58841127e21 2024-04-12T21:15:44.014 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:44.018+0000 7f9025600700 1 -- 172.21.15.18:0/3636062621 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f900406ec00 msgr2=0x7f90040710b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:15:44.014 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:44.018+0000 7f9025600700 1 --2- 172.21.15.18:0/3636062621 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f900406ec00 0x7f90040710b0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f901000b3b0 tx=0x7f9010009e30 comp rx=0 tx=0).stop 2024-04-12T21:15:44.015 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:44.018+0000 7f9025600700 1 -- 172.21.15.18:0/3636062621 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9020109dd0 msgr2=0x7f902011a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:15:44.015 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:44.018+0000 7f9025600700 1 --2- 172.21.15.18:0/3636062621 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9020109dd0 0x7f902011a480 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f900c00b790 tx=0x7f900c009220 comp rx=0 tx=0).stop 2024-04-12T21:15:44.015 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:44.019+0000 7f9025600700 1 -- 172.21.15.18:0/3636062621 shutdown_connections 2024-04-12T21:15:44.015 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:44.019+0000 7f9025600700 1 --2- 172.21.15.18:0/3636062621 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f900406ec00 0x7f90040710b0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:15:44.015 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:44.019+0000 7f9025600700 1 --2- 172.21.15.18:0/3636062621 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9020109430 0x7f9020119f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:15:44.016 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:44.019+0000 7f9025600700 1 --2- 172.21.15.18:0/3636062621 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9020109dd0 0x7f902011a480 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:15:44.016 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:44.019+0000 7f9025600700 1 -- 172.21.15.18:0/3636062621 >> 172.21.15.18:0/3636062621 conn(0x7f9020072c00 msgr2=0x7f9020074c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:15:44.016 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:44.019+0000 7f9025600700 1 -- 172.21.15.18:0/3636062621 shutdown_connections 2024-04-12T21:15:44.016 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:44.019+0000 7f9025600700 1 -- 172.21.15.18:0/3636062621 wait complete. 2024-04-12T21:15:44.174 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:44.178+0000 7f12ef600700 1 -- 172.21.15.18:0/990577163 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f12e8070c70 msgr2=0x7f12e81090f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:15:44.175 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:44.178+0000 7f12ef600700 1 --2- 172.21.15.18:0/990577163 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f12e8070c70 0x7f12e81090f0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f12d4004970 tx=0x7f12d4009e30 comp rx=0 tx=0).stop 2024-04-12T21:15:44.175 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:44.178+0000 7f12ef600700 1 -- 172.21.15.18:0/990577163 shutdown_connections 2024-04-12T21:15:44.175 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:44.178+0000 7f12ef600700 1 --2- 172.21.15.18:0/990577163 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f12e8070c70 0x7f12e81090f0 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:15:44.175 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:44.178+0000 7f12ef600700 1 --2- 172.21.15.18:0/990577163 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f12e8070360 0x7f12e8070730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:15:44.175 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:44.178+0000 7f12ef600700 1 -- 172.21.15.18:0/990577163 >> 172.21.15.18:0/990577163 conn(0x7f12e80f8970 msgr2=0x7f12e80fad80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:15:44.175 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:44.178+0000 7f12ef600700 1 -- 172.21.15.18:0/990577163 shutdown_connections 2024-04-12T21:15:44.175 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:44.178+0000 7f12ef600700 1 -- 172.21.15.18:0/990577163 wait complete. 2024-04-12T21:15:44.176 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:44.179+0000 7f12ef600700 1 Processor -- start 2024-04-12T21:15:44.176 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:44.179+0000 7f12ef600700 1 -- start start 2024-04-12T21:15:44.177 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:44.179+0000 7f12ef600700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f12e8070360 0x7f12e8115d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:15:44.177 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:44.179+0000 7f12ef600700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f12e8070c70 0x7f12e8116290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:15:44.177 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:44.179+0000 7f12ef600700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12e81168e0 con 0x7f12e8070c70 2024-04-12T21:15:44.177 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:44.179+0000 7f12ef600700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12e810fcb0 con 0x7f12e8070360 2024-04-12T21:15:44.177 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:44.180+0000 7f12e7e00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f12e8070c70 0x7f12e8116290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:15:44.177 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:44.180+0000 7f12e7e00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f12e8070c70 0x7f12e8116290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:46896/0 (socket says 172.21.15.18:46896) 2024-04-12T21:15:44.177 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:44.180+0000 7f12e7e00700 1 -- 172.21.15.18:0/1255822155 learned_addr learned my addr 172.21.15.18:0/1255822155 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:15:44.177 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:44.180+0000 7f12e7e00700 1 -- 172.21.15.18:0/1255822155 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f12e8070360 msgr2=0x7f12e8115d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:15:44.177 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:44.180+0000 7f12e7e00700 1 --2- 172.21.15.18:0/1255822155 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f12e8070360 0x7f12e8115d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:15:44.177 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:44.180+0000 7f12e7e00700 1 -- 172.21.15.18:0/1255822155 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12d4005040 con 0x7f12e8070c70 2024-04-12T21:15:44.177 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:44.180+0000 7f12e7e00700 1 --2- 172.21.15.18:0/1255822155 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f12e8070c70 0x7f12e8116290 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f12d4004f50 tx=0x7f12d4009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:15:44.178 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:44.181+0000 7f12e4c00700 1 -- 172.21.15.18:0/1255822155 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f12d400e040 con 0x7f12e8070c70 2024-04-12T21:15:44.178 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:44.181+0000 7f12e4c00700 1 -- 172.21.15.18:0/1255822155 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f12d400bc10 con 0x7f12e8070c70 2024-04-12T21:15:44.178 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:44.181+0000 7f12e4c00700 1 -- 172.21.15.18:0/1255822155 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f12d401c630 con 0x7f12e8070c70 2024-04-12T21:15:44.178 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:44.181+0000 7f12ef600700 1 -- 172.21.15.18:0/1255822155 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f12e810ff30 con 0x7f12e8070c70 2024-04-12T21:15:44.178 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:44.181+0000 7f12ef600700 1 -- 172.21.15.18:0/1255822155 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f12e8110470 con 0x7f12e8070c70 2024-04-12T21:15:44.178 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:44.182+0000 7f12ef600700 1 -- 172.21.15.18:0/1255822155 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f12e8071aa0 con 0x7f12e8070c70 2024-04-12T21:15:44.179 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:44.183+0000 7f12e4c00700 1 -- 172.21.15.18:0/1255822155 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7f12d4013420 con 0x7f12e8070c70 2024-04-12T21:15:44.181 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:44.185+0000 7f12e4c00700 1 --2- 172.21.15.18:0/1255822155 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f12d006ec10 0x7f12d00710c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:15:44.181 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:44.185+0000 7f12e4c00700 1 -- 172.21.15.18:0/1255822155 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 6757+0+0 (secure 0 0 0) 0x7f12d408fc70 con 0x7f12e8070c70 2024-04-12T21:15:44.181 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:44.185+0000 7f12ecc00700 1 --2- 172.21.15.18:0/1255822155 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f12d006ec10 0x7f12d00710c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:15:44.182 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:44.186+0000 7f12ecc00700 1 --2- 172.21.15.18:0/1255822155 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f12d006ec10 0x7f12d00710c0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f12dc004ad0 tx=0x7f12dc007d40 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:15:44.186 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:44.190+0000 7f12e4c00700 1 -- 172.21.15.18:0/1255822155 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f12d4096080 con 0x7f12e8070c70 2024-04-12T21:15:44.512 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:44.516+0000 7f12ef600700 1 -- 172.21.15.18:0/1255822155 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f12e804bed0 con 0x7f12e8070c70 2024-04-12T21:15:44.513 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:44.517+0000 7f12e4c00700 1 -- 172.21.15.18:0/1255822155 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+837 (secure 0 0 0) 0x7f12d408de60 con 0x7f12e8070c70 2024-04-12T21:15:44.514 INFO:teuthology.orchestra.run.smithi018.stdout:{ 2024-04-12T21:15:44.514 INFO:teuthology.orchestra.run.smithi018.stdout: "mon": { 2024-04-12T21:15:44.514 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 19.0.0-2879-g324baf8d (324baf8ddb91dea46cac50cb535c306115791355) squid (dev)": 2 2024-04-12T21:15:44.514 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-12T21:15:44.514 INFO:teuthology.orchestra.run.smithi018.stdout: "mgr": { 2024-04-12T21:15:44.514 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 19.0.0-2879-g324baf8d (324baf8ddb91dea46cac50cb535c306115791355) squid (dev)": 2 2024-04-12T21:15:44.514 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-12T21:15:44.514 INFO:teuthology.orchestra.run.smithi018.stdout: "osd": { 2024-04-12T21:15:44.514 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 19.0.0-2879-g324baf8d (324baf8ddb91dea46cac50cb535c306115791355) squid (dev)": 6 2024-04-12T21:15:44.515 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-12T21:15:44.515 INFO:teuthology.orchestra.run.smithi018.stdout: "mds": { 2024-04-12T21:15:44.515 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 17.2.7-827-gd3b5060a (d3b5060a17449edfae9385cdf792afec109fca5d) quincy (stable)": 2, 2024-04-12T21:15:44.515 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 19.0.0-2879-g324baf8d (324baf8ddb91dea46cac50cb535c306115791355) squid (dev)": 2 2024-04-12T21:15:44.515 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-12T21:15:44.515 INFO:teuthology.orchestra.run.smithi018.stdout: "overall": { 2024-04-12T21:15:44.515 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 17.2.7-827-gd3b5060a (d3b5060a17449edfae9385cdf792afec109fca5d) quincy (stable)": 2, 2024-04-12T21:15:44.515 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 19.0.0-2879-g324baf8d (324baf8ddb91dea46cac50cb535c306115791355) squid (dev)": 12 2024-04-12T21:15:44.515 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-04-12T21:15:44.515 INFO:teuthology.orchestra.run.smithi018.stdout:} 2024-04-12T21:15:44.516 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:44.520+0000 7f12ef600700 1 -- 172.21.15.18:0/1255822155 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f12d006ec10 msgr2=0x7f12d00710c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:15:44.516 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:44.520+0000 7f12ef600700 1 --2- 172.21.15.18:0/1255822155 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f12d006ec10 0x7f12d00710c0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f12dc004ad0 tx=0x7f12dc007d40 comp rx=0 tx=0).stop 2024-04-12T21:15:44.516 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:44.520+0000 7f12ef600700 1 -- 172.21.15.18:0/1255822155 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f12e8070c70 msgr2=0x7f12e8116290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:15:44.516 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:44.520+0000 7f12ef600700 1 --2- 172.21.15.18:0/1255822155 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f12e8070c70 0x7f12e8116290 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f12d4004f50 tx=0x7f12d4009e30 comp rx=0 tx=0).stop 2024-04-12T21:15:44.517 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:44.520+0000 7f12ef600700 1 -- 172.21.15.18:0/1255822155 shutdown_connections 2024-04-12T21:15:44.517 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:44.520+0000 7f12ef600700 1 --2- 172.21.15.18:0/1255822155 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f12d006ec10 0x7f12d00710c0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:15:44.517 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:44.520+0000 7f12ef600700 1 --2- 172.21.15.18:0/1255822155 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f12e8070360 0x7f12e8115d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:15:44.517 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:44.520+0000 7f12ef600700 1 --2- 172.21.15.18:0/1255822155 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f12e8070c70 0x7f12e8116290 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:15:44.518 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:44.520+0000 7f12ef600700 1 -- 172.21.15.18:0/1255822155 >> 172.21.15.18:0/1255822155 conn(0x7f12e80f8970 msgr2=0x7f12e80f9d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:15:44.518 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:44.521+0000 7f12ef600700 1 -- 172.21.15.18:0/1255822155 shutdown_connections 2024-04-12T21:15:44.518 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:44.521+0000 7f12ef600700 1 -- 172.21.15.18:0/1255822155 wait complete. 2024-04-12T21:15:44.662 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:44.666+0000 7f595c200700 1 -- 172.21.15.18:0/2343627018 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f59540fde90 msgr2=0x7f59540fe260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:15:44.663 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:44.666+0000 7f595c200700 1 --2- 172.21.15.18:0/2343627018 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f59540fde90 0x7f59540fe260 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f5948009ce0 tx=0x7f5948004d10 comp rx=0 tx=0).stop 2024-04-12T21:15:44.663 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:44.666+0000 7f595c200700 1 -- 172.21.15.18:0/2343627018 shutdown_connections 2024-04-12T21:15:44.663 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:44.666+0000 7f595c200700 1 --2- 172.21.15.18:0/2343627018 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f59540fe830 0x7f5954106f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:15:44.663 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:44.666+0000 7f595c200700 1 --2- 172.21.15.18:0/2343627018 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f59540fde90 0x7f59540fe260 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:15:44.663 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:44.666+0000 7f595c200700 1 -- 172.21.15.18:0/2343627018 >> 172.21.15.18:0/2343627018 conn(0x7f5954072bc0 msgr2=0x7f5954072fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:15:44.663 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:44.666+0000 7f595c200700 1 -- 172.21.15.18:0/2343627018 shutdown_connections 2024-04-12T21:15:44.663 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:44.666+0000 7f595c200700 1 -- 172.21.15.18:0/2343627018 wait complete. 2024-04-12T21:15:44.665 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:44.667+0000 7f595c200700 1 Processor -- start 2024-04-12T21:15:44.665 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:44.667+0000 7f595c200700 1 -- start start 2024-04-12T21:15:44.665 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:44.668+0000 7f595c200700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f59540fde90 0x7f59540fbd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:15:44.665 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:44.668+0000 7f595c200700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f59540fe830 0x7f59540fc290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:15:44.665 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:44.668+0000 7f595c200700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59540fc820 con 0x7f59540fe830 2024-04-12T21:15:44.665 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:44.668+0000 7f595c200700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59540fc960 con 0x7f59540fde90 2024-04-12T21:15:44.666 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:44.668+0000 7f5958e00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f59540fe830 0x7f59540fc290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:15:44.666 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:44.668+0000 7f5958e00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f59540fe830 0x7f59540fc290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:46922/0 (socket says 172.21.15.18:46922) 2024-04-12T21:15:44.666 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:44.668+0000 7f5958e00700 1 -- 172.21.15.18:0/3761613287 learned_addr learned my addr 172.21.15.18:0/3761613287 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:15:44.666 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:44.668+0000 7f5959800700 1 --2- 172.21.15.18:0/3761613287 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f59540fde90 0x7f59540fbd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:15:44.666 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:44.668+0000 7f5958e00700 1 -- 172.21.15.18:0/3761613287 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f59540fde90 msgr2=0x7f59540fbd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:15:44.666 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:44.668+0000 7f5958e00700 1 --2- 172.21.15.18:0/3761613287 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f59540fde90 0x7f59540fbd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:15:44.666 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:44.668+0000 7f5958e00700 1 -- 172.21.15.18:0/3761613287 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5948009940 con 0x7f59540fe830 2024-04-12T21:15:44.666 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:44.669+0000 7f5958e00700 1 --2- 172.21.15.18:0/3761613287 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f59540fe830 0x7f59540fc290 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f5940002a50 tx=0x7f59400046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:15:44.666 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:44.669+0000 7f5951600700 1 -- 172.21.15.18:0/3761613287 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5940007cc0 con 0x7f59540fe830 2024-04-12T21:15:44.666 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:44.669+0000 7f5951600700 1 -- 172.21.15.18:0/3761613287 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f594000f040 con 0x7f59540fe830 2024-04-12T21:15:44.666 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:44.669+0000 7f5951600700 1 -- 172.21.15.18:0/3761613287 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5940009640 con 0x7f59540fe830 2024-04-12T21:15:44.666 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:44.669+0000 7f595c200700 1 -- 172.21.15.18:0/3761613287 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5954119ed0 con 0x7f59540fe830 2024-04-12T21:15:44.666 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:44.669+0000 7f595c200700 1 -- 172.21.15.18:0/3761613287 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f595411a2d0 con 0x7f59540fe830 2024-04-12T21:15:44.667 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:44.670+0000 7f595c200700 1 -- 172.21.15.18:0/3761613287 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5954065e30 con 0x7f59540fe830 2024-04-12T21:15:44.667 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:44.671+0000 7f5951600700 1 -- 172.21.15.18:0/3761613287 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7f59400097a0 con 0x7f59540fe830 2024-04-12T21:15:44.669 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:44.673+0000 7f5951600700 1 --2- 172.21.15.18:0/3761613287 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f593c06ec10 0x7f593c0710c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:15:44.669 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:44.673+0000 7f5951600700 1 -- 172.21.15.18:0/3761613287 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 6757+0+0 (secure 0 0 0) 0x7f594008e740 con 0x7f59540fe830 2024-04-12T21:15:44.669 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:44.673+0000 7f5959800700 1 --2- 172.21.15.18:0/3761613287 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f593c06ec10 0x7f593c0710c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:15:44.669 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:44.673+0000 7f5959800700 1 --2- 172.21.15.18:0/3761613287 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f593c06ec10 0x7f593c0710c0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f594800b740 tx=0x7f5948004d10 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:15:44.673 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:44.677+0000 7f5951600700 1 -- 172.21.15.18:0/3761613287 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f59400573c0 con 0x7f59540fe830 2024-04-12T21:15:44.778 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:15:44 smithi186 ceph-mon[86379]: from='client.34864 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:15:44.779 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:15:44 smithi186 ceph-mon[86379]: from='client.34868 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:15:44.849 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:15:44 smithi018 ceph-mon[112590]: from='client.34864 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:15:44.850 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:15:44 smithi018 ceph-mon[112590]: from='client.34868 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:15:44.950 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:44.954+0000 7f595c200700 1 -- 172.21.15.18:0/3761613287 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f595404bed0 con 0x7f59540fe830 2024-04-12T21:15:44.950 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:44.954+0000 7f5951600700 1 -- 172.21.15.18:0/3761613287 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 29 v29) v1 ==== 76+0+1872 (secure 0 0 0) 0x7f594005aa80 con 0x7f59540fe830 2024-04-12T21:15:44.951 INFO:teuthology.orchestra.run.smithi018.stdout:e29 2024-04-12T21:15:44.951 INFO:teuthology.orchestra.run.smithi018.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-12T21:15:44.951 INFO:teuthology.orchestra.run.smithi018.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-12T21:15:44.951 INFO:teuthology.orchestra.run.smithi018.stdout:legacy client fscid: 1 2024-04-12T21:15:44.951 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-12T21:15:44.951 INFO:teuthology.orchestra.run.smithi018.stdout:Filesystem 'cephfs' (1) 2024-04-12T21:15:44.952 INFO:teuthology.orchestra.run.smithi018.stdout:fs_name cephfs 2024-04-12T21:15:44.952 INFO:teuthology.orchestra.run.smithi018.stdout:epoch 24 2024-04-12T21:15:44.952 INFO:teuthology.orchestra.run.smithi018.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-04-12T21:15:44.952 INFO:teuthology.orchestra.run.smithi018.stdout:created 2024-04-12T20:51:57.362850+0000 2024-04-12T21:15:44.952 INFO:teuthology.orchestra.run.smithi018.stdout:modified 2024-04-12T21:06:59.717190+0000 2024-04-12T21:15:44.952 INFO:teuthology.orchestra.run.smithi018.stdout:tableserver 0 2024-04-12T21:15:44.952 INFO:teuthology.orchestra.run.smithi018.stdout:root 0 2024-04-12T21:15:44.952 INFO:teuthology.orchestra.run.smithi018.stdout:session_timeout 60 2024-04-12T21:15:44.952 INFO:teuthology.orchestra.run.smithi018.stdout:session_autoclose 300 2024-04-12T21:15:44.952 INFO:teuthology.orchestra.run.smithi018.stdout:max_file_size 1099511627776 2024-04-12T21:15:44.952 INFO:teuthology.orchestra.run.smithi018.stdout:max_xattr_size 65536 2024-04-12T21:15:44.952 INFO:teuthology.orchestra.run.smithi018.stdout:required_client_features {} 2024-04-12T21:15:44.952 INFO:teuthology.orchestra.run.smithi018.stdout:last_failure 0 2024-04-12T21:15:44.952 INFO:teuthology.orchestra.run.smithi018.stdout:last_failure_osd_epoch 100 2024-04-12T21:15:44.953 INFO:teuthology.orchestra.run.smithi018.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-12T21:15:44.953 INFO:teuthology.orchestra.run.smithi018.stdout:max_mds 2 2024-04-12T21:15:44.953 INFO:teuthology.orchestra.run.smithi018.stdout:in 0,1 2024-04-12T21:15:44.953 INFO:teuthology.orchestra.run.smithi018.stdout:up {} 2024-04-12T21:15:44.953 INFO:teuthology.orchestra.run.smithi018.stdout:failed 0,1 2024-04-12T21:15:44.953 INFO:teuthology.orchestra.run.smithi018.stdout:damaged 2024-04-12T21:15:44.953 INFO:teuthology.orchestra.run.smithi018.stdout:stopped 2024-04-12T21:15:44.953 INFO:teuthology.orchestra.run.smithi018.stdout:data_pools [3] 2024-04-12T21:15:44.953 INFO:teuthology.orchestra.run.smithi018.stdout:metadata_pool 2 2024-04-12T21:15:44.953 INFO:teuthology.orchestra.run.smithi018.stdout:inline_data disabled 2024-04-12T21:15:44.954 INFO:teuthology.orchestra.run.smithi018.stdout:balancer 2024-04-12T21:15:44.954 INFO:teuthology.orchestra.run.smithi018.stdout:bal_rank_mask -1 2024-04-12T21:15:44.954 INFO:teuthology.orchestra.run.smithi018.stdout:standby_count_wanted 1 2024-04-12T21:15:44.954 INFO:teuthology.orchestra.run.smithi018.stdout:qdb_cluster leader: 0 members: 2024-04-12T21:15:44.954 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-12T21:15:44.954 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-12T21:15:44.954 INFO:teuthology.orchestra.run.smithi018.stdout:Standby daemons: 2024-04-12T21:15:44.954 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-12T21:15:44.954 INFO:teuthology.orchestra.run.smithi018.stdout:[mds.cephfs.smithi186.wjnafs{-1:24277} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.186:6826/438347207,v1:172.21.15.186:6827/438347207] compat {c=[1],r=[1],i=[7ff]}] 2024-04-12T21:15:44.954 INFO:teuthology.orchestra.run.smithi018.stdout:[mds.cephfs.smithi186.jdllyn{-1:24281} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.186:6824/1464359952,v1:172.21.15.186:6825/1464359952] compat {c=[1],r=[1],i=[7ff]}] 2024-04-12T21:15:44.954 INFO:teuthology.orchestra.run.smithi018.stdout:[mds.cephfs.smithi018.acqbsc{-1:34436} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.18:6828/755179764,v1:172.21.15.18:6829/755179764] compat {c=[1],r=[1],i=[1fff]}] 2024-04-12T21:15:44.955 INFO:teuthology.orchestra.run.smithi018.stdout:[mds.cephfs.smithi018.szjluo{-1:34468} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.18:6826/2407992653,v1:172.21.15.18:6827/2407992653] compat {c=[1],r=[1],i=[1fff]}] 2024-04-12T21:15:44.955 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:44.957+0000 7f595c200700 1 -- 172.21.15.18:0/3761613287 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f593c06ec10 msgr2=0x7f593c0710c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:15:44.955 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:44.957+0000 7f595c200700 1 --2- 172.21.15.18:0/3761613287 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f593c06ec10 0x7f593c0710c0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f594800b740 tx=0x7f5948004d10 comp rx=0 tx=0).stop 2024-04-12T21:15:44.956 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:44.957+0000 7f595c200700 1 -- 172.21.15.18:0/3761613287 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f59540fe830 msgr2=0x7f59540fc290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:15:44.956 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:44.957+0000 7f595c200700 1 --2- 172.21.15.18:0/3761613287 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f59540fe830 0x7f59540fc290 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f5940002a50 tx=0x7f59400046b0 comp rx=0 tx=0).stop 2024-04-12T21:15:44.956 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:44.957+0000 7f595c200700 1 -- 172.21.15.18:0/3761613287 shutdown_connections 2024-04-12T21:15:44.956 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:44.957+0000 7f595c200700 1 --2- 172.21.15.18:0/3761613287 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f593c06ec10 0x7f593c0710c0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:15:44.956 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:44.957+0000 7f595c200700 1 --2- 172.21.15.18:0/3761613287 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f59540fde90 0x7f59540fbd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:15:44.956 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:44.957+0000 7f595c200700 1 --2- 172.21.15.18:0/3761613287 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f59540fe830 0x7f59540fc290 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:15:44.956 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:44.957+0000 7f595c200700 1 -- 172.21.15.18:0/3761613287 >> 172.21.15.18:0/3761613287 conn(0x7f5954072bc0 msgr2=0x7f59540ff3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:15:44.956 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:44.957+0000 7f595c200700 1 -- 172.21.15.18:0/3761613287 shutdown_connections 2024-04-12T21:15:44.956 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:44.958+0000 7f595c200700 1 -- 172.21.15.18:0/3761613287 wait complete. 2024-04-12T21:15:44.956 INFO:teuthology.orchestra.run.smithi018.stderr:dumped fsmap epoch 29 2024-04-12T21:15:45.108 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:45.112+0000 7f3421e00700 1 -- 172.21.15.18:0/1332500178 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f341c0fde20 msgr2=0x7f341c0fe1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:15:45.108 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:45.112+0000 7f3421e00700 1 --2- 172.21.15.18:0/1332500178 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f341c0fde20 0x7f341c0fe1f0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f340c004940 tx=0x7f340c009e30 comp rx=0 tx=0).stop 2024-04-12T21:15:45.108 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:45.112+0000 7f3421e00700 1 -- 172.21.15.18:0/1332500178 shutdown_connections 2024-04-12T21:15:45.109 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:45.112+0000 7f3421e00700 1 --2- 172.21.15.18:0/1332500178 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f341c0fe7c0 0x7f341c106e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:15:45.109 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:45.112+0000 7f3421e00700 1 --2- 172.21.15.18:0/1332500178 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f341c0fde20 0x7f341c0fe1f0 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:15:45.109 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:45.112+0000 7f3421e00700 1 -- 172.21.15.18:0/1332500178 >> 172.21.15.18:0/1332500178 conn(0x7f341c0f7870 msgr2=0x7f341c0f9c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:15:45.109 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:45.112+0000 7f3421e00700 1 -- 172.21.15.18:0/1332500178 shutdown_connections 2024-04-12T21:15:45.109 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:45.112+0000 7f3421e00700 1 -- 172.21.15.18:0/1332500178 wait complete. 2024-04-12T21:15:45.111 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:45.113+0000 7f3421e00700 1 Processor -- start 2024-04-12T21:15:45.111 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:45.113+0000 7f3421e00700 1 -- start start 2024-04-12T21:15:45.111 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:45.113+0000 7f3421e00700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f341c0fde20 0x7f341c115c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:15:45.112 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:45.113+0000 7f3421e00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f341c0fe7c0 0x7f341c116160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:15:45.112 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:45.113+0000 7f3421e00700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f341c1167b0 con 0x7f341c0fe7c0 2024-04-12T21:15:45.112 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:45.113+0000 7f3421e00700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f341c10fd90 con 0x7f341c0fde20 2024-04-12T21:15:45.112 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:45.114+0000 7f341a000700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f341c0fe7c0 0x7f341c116160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:15:45.112 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:45.114+0000 7f341a000700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f341c0fe7c0 0x7f341c116160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:46942/0 (socket says 172.21.15.18:46942) 2024-04-12T21:15:45.112 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:45.114+0000 7f341a000700 1 -- 172.21.15.18:0/2122893448 learned_addr learned my addr 172.21.15.18:0/2122893448 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:15:45.112 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:45.114+0000 7f341aa00700 1 --2- 172.21.15.18:0/2122893448 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f341c0fde20 0x7f341c115c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:15:45.112 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:45.114+0000 7f341a000700 1 -- 172.21.15.18:0/2122893448 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f341c0fde20 msgr2=0x7f341c115c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:15:45.112 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:45.114+0000 7f341a000700 1 --2- 172.21.15.18:0/2122893448 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f341c0fde20 0x7f341c115c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:15:45.112 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:45.114+0000 7f341a000700 1 -- 172.21.15.18:0/2122893448 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f340c005040 con 0x7f341c0fe7c0 2024-04-12T21:15:45.112 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:45.114+0000 7f341aa00700 1 --2- 172.21.15.18:0/2122893448 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f341c0fde20 0x7f341c115c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-12T21:15:45.112 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:45.115+0000 7f341a000700 1 --2- 172.21.15.18:0/2122893448 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f341c0fe7c0 0x7f341c116160 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f3408002a50 tx=0x7f34080046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:15:45.113 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:45.115+0000 7f3412a00700 1 -- 172.21.15.18:0/2122893448 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3408007cc0 con 0x7f341c0fe7c0 2024-04-12T21:15:45.113 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:45.115+0000 7f3412a00700 1 -- 172.21.15.18:0/2122893448 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f340800f040 con 0x7f341c0fe7c0 2024-04-12T21:15:45.113 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:45.115+0000 7f3412a00700 1 -- 172.21.15.18:0/2122893448 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3408009640 con 0x7f341c0fe7c0 2024-04-12T21:15:45.113 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:45.115+0000 7f3421e00700 1 -- 172.21.15.18:0/2122893448 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f341c110070 con 0x7f341c0fe7c0 2024-04-12T21:15:45.113 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:45.115+0000 7f3421e00700 1 -- 172.21.15.18:0/2122893448 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f341c110560 con 0x7f341c0fe7c0 2024-04-12T21:15:45.113 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:45.116+0000 7f3421e00700 1 -- 172.21.15.18:0/2122893448 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f341c066b50 con 0x7f341c0fe7c0 2024-04-12T21:15:45.114 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:45.118+0000 7f3412a00700 1 -- 172.21.15.18:0/2122893448 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7f34080097c0 con 0x7f341c0fe7c0 2024-04-12T21:15:45.115 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:45.118+0000 7f3412a00700 1 --2- 172.21.15.18:0/2122893448 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f340006ec00 0x7f34000710b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:15:45.115 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:45.119+0000 7f3412a00700 1 -- 172.21.15.18:0/2122893448 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 6757+0+0 (secure 0 0 0) 0x7f340808e730 con 0x7f341c0fe7c0 2024-04-12T21:15:45.115 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:45.119+0000 7f341aa00700 1 --2- 172.21.15.18:0/2122893448 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f340006ec00 0x7f34000710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:15:45.115 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:45.120+0000 7f341aa00700 1 --2- 172.21.15.18:0/2122893448 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f340006ec00 0x7f34000710b0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f340c009fd0 tx=0x7f340c009e30 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:15:45.120 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:45.124+0000 7f3412a00700 1 -- 172.21.15.18:0/2122893448 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f340805af00 con 0x7f341c0fe7c0 2024-04-12T21:15:45.363 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:45.367+0000 7f3421e00700 1 -- 172.21.15.18:0/2122893448 --> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f341c109eb0 con 0x7f340006ec00 2024-04-12T21:15:45.367 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:45.371+0000 7f3412a00700 1 -- 172.21.15.18:0/2122893448 <== mgr.34102 v2:172.21.15.18:6800/3812891919 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f341c109eb0 con 0x7f340006ec00 2024-04-12T21:15:45.367 INFO:teuthology.orchestra.run.smithi018.stdout:{ 2024-04-12T21:15:45.368 INFO:teuthology.orchestra.run.smithi018.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:f4452def3be4d350f4a8e5feaf7ebb1511be9298fe68478ba7a2d6fbd7d69a63", 2024-04-12T21:15:45.368 INFO:teuthology.orchestra.run.smithi018.stdout: "in_progress": true, 2024-04-12T21:15:45.368 INFO:teuthology.orchestra.run.smithi018.stdout: "which": "Upgrading all daemon types on all hosts", 2024-04-12T21:15:45.368 INFO:teuthology.orchestra.run.smithi018.stdout: "services_complete": [ 2024-04-12T21:15:45.368 INFO:teuthology.orchestra.run.smithi018.stdout: "crash", 2024-04-12T21:15:45.368 INFO:teuthology.orchestra.run.smithi018.stdout: "osd", 2024-04-12T21:15:45.368 INFO:teuthology.orchestra.run.smithi018.stdout: "mon", 2024-04-12T21:15:45.368 INFO:teuthology.orchestra.run.smithi018.stdout: "mgr" 2024-04-12T21:15:45.368 INFO:teuthology.orchestra.run.smithi018.stdout: ], 2024-04-12T21:15:45.368 INFO:teuthology.orchestra.run.smithi018.stdout: "progress": "14/21 daemons upgraded", 2024-04-12T21:15:45.368 INFO:teuthology.orchestra.run.smithi018.stdout: "message": "Currently upgrading mds daemons", 2024-04-12T21:15:45.368 INFO:teuthology.orchestra.run.smithi018.stdout: "is_paused": false 2024-04-12T21:15:45.368 INFO:teuthology.orchestra.run.smithi018.stdout:} 2024-04-12T21:15:45.371 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:45.374+0000 7f3421e00700 1 -- 172.21.15.18:0/2122893448 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f340006ec00 msgr2=0x7f34000710b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:15:45.372 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:45.374+0000 7f3421e00700 1 --2- 172.21.15.18:0/2122893448 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f340006ec00 0x7f34000710b0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f340c009fd0 tx=0x7f340c009e30 comp rx=0 tx=0).stop 2024-04-12T21:15:45.372 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:45.374+0000 7f3421e00700 1 -- 172.21.15.18:0/2122893448 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f341c0fe7c0 msgr2=0x7f341c116160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:15:45.372 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:45.374+0000 7f3421e00700 1 --2- 172.21.15.18:0/2122893448 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f341c0fe7c0 0x7f341c116160 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f3408002a50 tx=0x7f34080046b0 comp rx=0 tx=0).stop 2024-04-12T21:15:45.372 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:45.375+0000 7f3421e00700 1 -- 172.21.15.18:0/2122893448 shutdown_connections 2024-04-12T21:15:45.372 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:45.375+0000 7f3421e00700 1 --2- 172.21.15.18:0/2122893448 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f340006ec00 0x7f34000710b0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:15:45.372 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:45.375+0000 7f3421e00700 1 --2- 172.21.15.18:0/2122893448 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f341c0fde20 0x7f341c115c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:15:45.372 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:45.375+0000 7f3421e00700 1 --2- 172.21.15.18:0/2122893448 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f341c0fe7c0 0x7f341c116160 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:15:45.372 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:45.375+0000 7f3421e00700 1 -- 172.21.15.18:0/2122893448 >> 172.21.15.18:0/2122893448 conn(0x7f341c0f7870 msgr2=0x7f341c0f8db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:15:45.372 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:45.375+0000 7f3421e00700 1 -- 172.21.15.18:0/2122893448 shutdown_connections 2024-04-12T21:15:45.372 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:45.375+0000 7f3421e00700 1 -- 172.21.15.18:0/2122893448 wait complete. 2024-04-12T21:15:45.523 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:45.526+0000 7fba39e00700 1 -- 172.21.15.18:0/227156706 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fba34109430 msgr2=0x7fba34109800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:15:45.523 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:45.526+0000 7fba39e00700 1 --2- 172.21.15.18:0/227156706 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fba34109430 0x7fba34109800 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7fba24004940 tx=0x7fba24009e30 comp rx=0 tx=0).stop 2024-04-12T21:15:45.523 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:45.526+0000 7fba39e00700 1 -- 172.21.15.18:0/227156706 shutdown_connections 2024-04-12T21:15:45.523 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:45.526+0000 7fba39e00700 1 --2- 172.21.15.18:0/227156706 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fba34109dd0 0x7fba34103440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:15:45.524 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:45.526+0000 7fba39e00700 1 --2- 172.21.15.18:0/227156706 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fba34109430 0x7fba34109800 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:15:45.524 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:45.526+0000 7fba39e00700 1 -- 172.21.15.18:0/227156706 >> 172.21.15.18:0/227156706 conn(0x7fba34072c00 msgr2=0x7fba34075010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:15:45.524 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:45.527+0000 7fba39e00700 1 -- 172.21.15.18:0/227156706 shutdown_connections 2024-04-12T21:15:45.524 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:45.527+0000 7fba39e00700 1 -- 172.21.15.18:0/227156706 wait complete. 2024-04-12T21:15:45.524 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:45.527+0000 7fba39e00700 1 Processor -- start 2024-04-12T21:15:45.526 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:45.527+0000 7fba39e00700 1 -- start start 2024-04-12T21:15:45.526 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:45.528+0000 7fba39e00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fba34109430 0x7fba34119f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:15:45.526 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:45.528+0000 7fba39e00700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fba34109dd0 0x7fba3411a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:15:45.526 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:45.528+0000 7fba39e00700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba3411ab60 con 0x7fba34109430 2024-04-12T21:15:45.526 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:45.528+0000 7fba39e00700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba341140b0 con 0x7fba34109dd0 2024-04-12T21:15:45.526 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:45.528+0000 7fba32a00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fba34109430 0x7fba34119f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:15:45.526 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:45.528+0000 7fba32a00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fba34109430 0x7fba34119f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:46960/0 (socket says 172.21.15.18:46960) 2024-04-12T21:15:45.526 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:45.528+0000 7fba32a00700 1 -- 172.21.15.18:0/3300308877 learned_addr learned my addr 172.21.15.18:0/3300308877 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:15:45.526 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:45.529+0000 7fba32000700 1 --2- 172.21.15.18:0/3300308877 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fba34109dd0 0x7fba3411a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:15:45.526 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:45.529+0000 7fba32a00700 1 -- 172.21.15.18:0/3300308877 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fba34109dd0 msgr2=0x7fba3411a480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:15:45.526 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:45.529+0000 7fba32a00700 1 --2- 172.21.15.18:0/3300308877 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fba34109dd0 0x7fba3411a480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:15:45.527 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:45.529+0000 7fba32a00700 1 -- 172.21.15.18:0/3300308877 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba24005040 con 0x7fba34109430 2024-04-12T21:15:45.527 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:45.529+0000 7fba32000700 1 --2- 172.21.15.18:0/3300308877 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fba34109dd0 0x7fba3411a480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-12T21:15:45.527 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:45.529+0000 7fba32a00700 1 --2- 172.21.15.18:0/3300308877 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fba34109430 0x7fba34119f40 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7fba24009180 tx=0x7fba24009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:15:45.527 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:45.529+0000 7fba2aa00700 1 -- 172.21.15.18:0/3300308877 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fba2400e040 con 0x7fba34109430 2024-04-12T21:15:45.527 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:45.529+0000 7fba39e00700 1 -- 172.21.15.18:0/3300308877 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fba341142d0 con 0x7fba34109430 2024-04-12T21:15:45.527 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:45.529+0000 7fba2aa00700 1 -- 172.21.15.18:0/3300308877 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fba2400bbc0 con 0x7fba34109430 2024-04-12T21:15:45.527 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:45.529+0000 7fba2aa00700 1 -- 172.21.15.18:0/3300308877 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fba2401c630 con 0x7fba34109430 2024-04-12T21:15:45.527 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:45.530+0000 7fba39e00700 1 -- 172.21.15.18:0/3300308877 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fba34114890 con 0x7fba34109430 2024-04-12T21:15:45.527 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:45.530+0000 7fba39e00700 1 -- 172.21.15.18:0/3300308877 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fba34104b80 con 0x7fba34109430 2024-04-12T21:15:45.528 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:45.532+0000 7fba2aa00700 1 -- 172.21.15.18:0/3300308877 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7fba2400bd30 con 0x7fba34109430 2024-04-12T21:15:45.529 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:45.532+0000 7fba2aa00700 1 --2- 172.21.15.18:0/3300308877 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fba1806ec00 0x7fba180710b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:15:45.529 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:45.533+0000 7fba2aa00700 1 -- 172.21.15.18:0/3300308877 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 6757+0+0 (secure 0 0 0) 0x7fba2408fd90 con 0x7fba34109430 2024-04-12T21:15:45.529 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:45.533+0000 7fba32000700 1 --2- 172.21.15.18:0/3300308877 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fba1806ec00 0x7fba180710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:15:45.529 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:45.534+0000 7fba32000700 1 --2- 172.21.15.18:0/3300308877 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fba1806ec00 0x7fba180710b0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fba20006fd0 tx=0x7fba20008040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:15:45.533 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:45.537+0000 7fba2aa00700 1 -- 172.21.15.18:0/3300308877 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7fba2405c5c0 con 0x7fba34109430 2024-04-12T21:15:45.778 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:15:45 smithi186 ceph-mon[86379]: from='client.34872 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:15:45.779 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:15:45 smithi186 ceph-mon[86379]: pgmap v478: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:15:45.779 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:15:45 smithi186 ceph-mon[86379]: from='client.? 172.21.15.18:0/1255822155' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T21:15:45.779 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:15:45 smithi186 ceph-mon[86379]: from='client.? 172.21.15.18:0/3761613287' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-12T21:15:45.849 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:15:45 smithi018 ceph-mon[112590]: from='client.34872 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:15:45.850 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:15:45 smithi018 ceph-mon[112590]: pgmap v478: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:15:45.850 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:15:45 smithi018 ceph-mon[112590]: from='client.? 172.21.15.18:0/1255822155' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T21:15:45.850 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:15:45 smithi018 ceph-mon[112590]: from='client.? 172.21.15.18:0/3761613287' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-12T21:15:45.854 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:45.858+0000 7fba39e00700 1 -- 172.21.15.18:0/3300308877 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7fba3404c710 con 0x7fba34109430 2024-04-12T21:15:45.855 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:45.858+0000 7fba2aa00700 1 -- 172.21.15.18:0/3300308877 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+233 (secure 0 0 0) 0x7fba2405c150 con 0x7fba34109430 2024-04-12T21:15:45.855 INFO:teuthology.orchestra.run.smithi018.stdout:HEALTH_ERR 1 filesystem is degraded; 1 filesystem is offline 2024-04-12T21:15:45.855 INFO:teuthology.orchestra.run.smithi018.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-04-12T21:15:45.855 INFO:teuthology.orchestra.run.smithi018.stdout: fs cephfs is degraded 2024-04-12T21:15:45.855 INFO:teuthology.orchestra.run.smithi018.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-04-12T21:15:45.855 INFO:teuthology.orchestra.run.smithi018.stdout: fs cephfs is offline because no MDS is active for it. 2024-04-12T21:15:45.857 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:45.861+0000 7fba39e00700 1 -- 172.21.15.18:0/3300308877 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fba1806ec00 msgr2=0x7fba180710b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:15:45.857 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:45.861+0000 7fba39e00700 1 --2- 172.21.15.18:0/3300308877 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fba1806ec00 0x7fba180710b0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fba20006fd0 tx=0x7fba20008040 comp rx=0 tx=0).stop 2024-04-12T21:15:45.857 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:45.861+0000 7fba39e00700 1 -- 172.21.15.18:0/3300308877 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fba34109430 msgr2=0x7fba34119f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:15:45.857 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:45.861+0000 7fba39e00700 1 --2- 172.21.15.18:0/3300308877 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fba34109430 0x7fba34119f40 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7fba24009180 tx=0x7fba24009e30 comp rx=0 tx=0).stop 2024-04-12T21:15:45.858 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:45.861+0000 7fba39e00700 1 -- 172.21.15.18:0/3300308877 shutdown_connections 2024-04-12T21:15:45.858 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:45.861+0000 7fba39e00700 1 --2- 172.21.15.18:0/3300308877 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fba1806ec00 0x7fba180710b0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:15:45.858 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:45.861+0000 7fba39e00700 1 --2- 172.21.15.18:0/3300308877 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fba34109430 0x7fba34119f40 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:15:45.858 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:45.861+0000 7fba39e00700 1 --2- 172.21.15.18:0/3300308877 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fba34109dd0 0x7fba3411a480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:15:45.859 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:45.861+0000 7fba39e00700 1 -- 172.21.15.18:0/3300308877 >> 172.21.15.18:0/3300308877 conn(0x7fba34072c00 msgr2=0x7fba34074c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:15:45.859 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:45.862+0000 7fba39e00700 1 -- 172.21.15.18:0/3300308877 shutdown_connections 2024-04-12T21:15:45.859 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:15:45.862+0000 7fba39e00700 1 -- 172.21.15.18:0/3300308877 wait complete. 2024-04-12T21:15:46.778 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:15:46 smithi186 ceph-mon[86379]: from='client.34884 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:15:46.779 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:15:46 smithi186 ceph-mon[86379]: from='client.? 172.21.15.18:0/3300308877' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-12T21:15:46.849 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:15:46 smithi018 ceph-mon[112590]: from='client.34884 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:15:46.850 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:15:46 smithi018 ceph-mon[112590]: from='client.? 172.21.15.18:0/3300308877' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-12T21:15:47.778 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:15:47 smithi186 ceph-mon[86379]: pgmap v479: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:15:47.849 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:15:47 smithi018 ceph-mon[112590]: pgmap v479: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:15:49.778 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:15:49 smithi186 ceph-mon[86379]: pgmap v480: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:15:49.849 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:15:49 smithi018 ceph-mon[112590]: pgmap v480: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:15:51.849 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:15:51 smithi018 ceph-mon[112590]: pgmap v481: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:15:52.028 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:15:51 smithi186 ceph-mon[86379]: pgmap v481: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:15:53.849 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:15:53 smithi018 ceph-mon[112590]: pgmap v482: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:15:54.028 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:15:53 smithi186 ceph-mon[86379]: pgmap v482: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:15:55.850 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:15:55 smithi018 ceph-mon[112590]: pgmap v483: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:15:55.850 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:15:55 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T21:15:56.028 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:15:55 smithi186 ceph-mon[86379]: pgmap v483: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:15:56.029 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:15:55 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T21:15:57.850 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:15:57 smithi018 ceph-mon[112590]: pgmap v484: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:15:58.028 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:15:57 smithi186 ceph-mon[86379]: pgmap v484: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:15:59.849 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:15:59 smithi018 ceph-mon[112590]: pgmap v485: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:16:00.028 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:15:59 smithi186 ceph-mon[86379]: pgmap v485: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:16:01.850 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:16:01 smithi018 ceph-mon[112590]: pgmap v486: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:16:02.028 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:16:01 smithi186 ceph-mon[86379]: pgmap v486: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:16:03.849 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:16:03 smithi018 ceph-mon[112590]: pgmap v487: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:16:04.029 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:16:03 smithi186 ceph-mon[86379]: pgmap v487: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:16:05.850 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:16:05 smithi018 ceph-mon[112590]: pgmap v488: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:16:06.028 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:16:05 smithi186 ceph-mon[86379]: pgmap v488: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:16:07.849 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:16:07 smithi018 ceph-mon[112590]: pgmap v489: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:16:08.028 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:16:07 smithi186 ceph-mon[86379]: pgmap v489: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:16:09.849 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:16:09 smithi018 ceph-mon[112590]: pgmap v490: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:16:10.028 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:16:09 smithi186 ceph-mon[86379]: pgmap v490: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:16:10.849 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:16:10 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T21:16:11.028 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:16:10 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T21:16:11.850 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:16:11 smithi018 ceph-mon[112590]: pgmap v491: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:16:12.028 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:16:11 smithi186 ceph-mon[86379]: pgmap v491: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:16:14.028 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:16:13 smithi186 ceph-mon[86379]: pgmap v492: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:16:14.099 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:16:13 smithi018 ceph-mon[112590]: pgmap v492: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:16:15.850 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:16:15 smithi018 ceph-mon[112590]: pgmap v493: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:16:16.028 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:16:15 smithi186 ceph-mon[86379]: pgmap v493: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:16:16.031 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:16.036+0000 7f331d800700 1 -- 172.21.15.18:0/1559571224 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f33180708a0 msgr2=0x7f3318070d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:16:16.031 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:16.036+0000 7f331d800700 1 --2- 172.21.15.18:0/1559571224 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f33180708a0 0x7f3318070d10 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f3300004970 tx=0x7f3300009e30 comp rx=0 tx=0).stop 2024-04-12T21:16:16.031 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:16.036+0000 7f331d800700 1 -- 172.21.15.18:0/1559571224 shutdown_connections 2024-04-12T21:16:16.032 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:16.036+0000 7f331d800700 1 --2- 172.21.15.18:0/1559571224 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f33180708a0 0x7f3318070d10 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:16:16.032 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:16.036+0000 7f331d800700 1 --2- 172.21.15.18:0/1559571224 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3318071f90 0x7f3318070360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:16:16.032 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:16.036+0000 7f331d800700 1 -- 172.21.15.18:0/1559571224 >> 172.21.15.18:0/1559571224 conn(0x7f33180f8a20 msgr2=0x7f33180fae30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:16:16.032 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:16.036+0000 7f331d800700 1 -- 172.21.15.18:0/1559571224 shutdown_connections 2024-04-12T21:16:16.032 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:16.036+0000 7f331d800700 1 -- 172.21.15.18:0/1559571224 wait complete. 2024-04-12T21:16:16.033 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:16.037+0000 7f331d800700 1 Processor -- start 2024-04-12T21:16:16.033 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:16.037+0000 7f331d800700 1 -- start start 2024-04-12T21:16:16.033 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:16.037+0000 7f331d800700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f33180708a0 0x7f3318119f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:16:16.034 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:16.037+0000 7f331d800700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3318071f90 0x7f331811a4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:16:16.034 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:16.037+0000 7f331d800700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f331811ab20 con 0x7f33180708a0 2024-04-12T21:16:16.034 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:16.037+0000 7f331d800700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3318114100 con 0x7f3318071f90 2024-04-12T21:16:16.034 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:16.038+0000 7f3316a00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f33180708a0 0x7f3318119f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:16:16.034 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:16.038+0000 7f3316a00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f33180708a0 0x7f3318119f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:36432/0 (socket says 172.21.15.18:36432) 2024-04-12T21:16:16.034 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:16.038+0000 7f3316a00700 1 -- 172.21.15.18:0/3999779580 learned_addr learned my addr 172.21.15.18:0/3999779580 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:16:16.034 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:16.038+0000 7f330e000700 1 --2- 172.21.15.18:0/3999779580 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3318071f90 0x7f331811a4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:16:16.034 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:16.038+0000 7f3316a00700 1 -- 172.21.15.18:0/3999779580 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3318071f90 msgr2=0x7f331811a4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:16:16.034 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:16.038+0000 7f3316a00700 1 --2- 172.21.15.18:0/3999779580 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3318071f90 0x7f331811a4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:16:16.034 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:16.038+0000 7f3316a00700 1 -- 172.21.15.18:0/3999779580 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3300005040 con 0x7f33180708a0 2024-04-12T21:16:16.035 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:16.039+0000 7f3316a00700 1 --2- 172.21.15.18:0/3999779580 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f33180708a0 0x7f3318119f90 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f3308001d80 tx=0x7f3308009ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:16:16.035 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:16.039+0000 7f330f400700 1 -- 172.21.15.18:0/3999779580 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3308007d10 con 0x7f33180708a0 2024-04-12T21:16:16.035 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:16.039+0000 7f330f400700 1 -- 172.21.15.18:0/3999779580 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f3308007e70 con 0x7f33180708a0 2024-04-12T21:16:16.035 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:16.039+0000 7f330f400700 1 -- 172.21.15.18:0/3999779580 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3308014640 con 0x7f33180708a0 2024-04-12T21:16:16.035 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:16.039+0000 7f331d800700 1 -- 172.21.15.18:0/3999779580 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f33181143e0 con 0x7f33180708a0 2024-04-12T21:16:16.035 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:16.039+0000 7f331d800700 1 -- 172.21.15.18:0/3999779580 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f33181148f0 con 0x7f33180708a0 2024-04-12T21:16:16.036 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:16.041+0000 7f331d800700 1 -- 172.21.15.18:0/3999779580 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3318071aa0 con 0x7f33180708a0 2024-04-12T21:16:16.036 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:16.042+0000 7f330f400700 1 -- 172.21.15.18:0/3999779580 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7f330800e960 con 0x7f33180708a0 2024-04-12T21:16:16.037 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:16.042+0000 7f330f400700 1 --2- 172.21.15.18:0/3999779580 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f32fc06ec00 0x7f32fc0710b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:16:16.038 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:16.043+0000 7f330f400700 1 -- 172.21.15.18:0/3999779580 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 6757+0+0 (secure 0 0 0) 0x7f330808e780 con 0x7f33180708a0 2024-04-12T21:16:16.038 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:16.043+0000 7f330e000700 1 --2- 172.21.15.18:0/3999779580 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f32fc06ec00 0x7f32fc0710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:16:16.038 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:16.044+0000 7f330e000700 1 --2- 172.21.15.18:0/3999779580 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f32fc06ec00 0x7f32fc0710b0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f3300009fd0 tx=0x7f3300009e30 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:16:16.043 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:16.048+0000 7f330f400700 1 -- 172.21.15.18:0/3999779580 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f330805afe0 con 0x7f33180708a0 2024-04-12T21:16:16.294 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:16.298+0000 7f331d800700 1 -- 172.21.15.18:0/3999779580 --> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f33181155e0 con 0x7f32fc06ec00 2024-04-12T21:16:16.295 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:16.300+0000 7f330f400700 1 -- 172.21.15.18:0/3999779580 <== mgr.34102 v2:172.21.15.18:6800/3812891919 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f33181155e0 con 0x7f32fc06ec00 2024-04-12T21:16:16.299 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:16.304+0000 7f331d800700 1 -- 172.21.15.18:0/3999779580 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f32fc06ec00 msgr2=0x7f32fc0710b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:16:16.299 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:16.304+0000 7f331d800700 1 --2- 172.21.15.18:0/3999779580 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f32fc06ec00 0x7f32fc0710b0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f3300009fd0 tx=0x7f3300009e30 comp rx=0 tx=0).stop 2024-04-12T21:16:16.299 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:16.304+0000 7f331d800700 1 -- 172.21.15.18:0/3999779580 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f33180708a0 msgr2=0x7f3318119f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:16:16.300 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:16.304+0000 7f331d800700 1 --2- 172.21.15.18:0/3999779580 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f33180708a0 0x7f3318119f90 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f3308001d80 tx=0x7f3308009ec0 comp rx=0 tx=0).stop 2024-04-12T21:16:16.300 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:16.304+0000 7f331d800700 1 -- 172.21.15.18:0/3999779580 shutdown_connections 2024-04-12T21:16:16.300 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:16.304+0000 7f331d800700 1 --2- 172.21.15.18:0/3999779580 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f32fc06ec00 0x7f32fc0710b0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:16:16.300 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:16.304+0000 7f331d800700 1 --2- 172.21.15.18:0/3999779580 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f33180708a0 0x7f3318119f90 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:16:16.300 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:16.304+0000 7f331d800700 1 --2- 172.21.15.18:0/3999779580 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3318071f90 0x7f331811a4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:16:16.300 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:16.304+0000 7f331d800700 1 -- 172.21.15.18:0/3999779580 >> 172.21.15.18:0/3999779580 conn(0x7f33180f8a20 msgr2=0x7f33180f9a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:16:16.300 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:16.304+0000 7f331d800700 1 -- 172.21.15.18:0/3999779580 shutdown_connections 2024-04-12T21:16:16.300 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:16.304+0000 7f331d800700 1 -- 172.21.15.18:0/3999779580 wait complete. 2024-04-12T21:16:16.316 INFO:teuthology.orchestra.run.smithi018.stdout:true 2024-04-12T21:16:16.464 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:16.468+0000 7fad5fe00700 1 -- 172.21.15.18:0/2043739923 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fad60065630 msgr2=0x7fad60065a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:16:16.464 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:16.468+0000 7fad5fe00700 1 --2- 172.21.15.18:0/2043739923 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fad60065630 0x7fad60065a80 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7fad48004940 tx=0x7fad48009e30 comp rx=0 tx=0).stop 2024-04-12T21:16:16.464 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:16.469+0000 7fad5fe00700 1 -- 172.21.15.18:0/2043739923 shutdown_connections 2024-04-12T21:16:16.464 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:16.469+0000 7fad5fe00700 1 --2- 172.21.15.18:0/2043739923 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fad60065630 0x7fad60065a80 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:16:16.464 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:16.469+0000 7fad5fe00700 1 --2- 172.21.15.18:0/2043739923 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fad60102ca0 0x7fad60103070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:16:16.464 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:16.469+0000 7fad5fe00700 1 -- 172.21.15.18:0/2043739923 >> 172.21.15.18:0/2043739923 conn(0x7fad60071b70 msgr2=0x7fad60071f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:16:16.464 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:16.469+0000 7fad5fe00700 1 -- 172.21.15.18:0/2043739923 shutdown_connections 2024-04-12T21:16:16.464 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:16.469+0000 7fad5fe00700 1 -- 172.21.15.18:0/2043739923 wait complete. 2024-04-12T21:16:16.465 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:16.470+0000 7fad5fe00700 1 Processor -- start 2024-04-12T21:16:16.465 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:16.470+0000 7fad5fe00700 1 -- start start 2024-04-12T21:16:16.465 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:16.470+0000 7fad5fe00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fad60065630 0x7fad60113990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:16:16.466 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:16.471+0000 7fad5fe00700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fad60102ca0 0x7fad60113ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:16:16.466 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:16.471+0000 7fad5fe00700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad60114520 con 0x7fad60065630 2024-04-12T21:16:16.466 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:16.471+0000 7fad5fe00700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad6010d910 con 0x7fad60102ca0 2024-04-12T21:16:16.467 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:16.471+0000 7fad5ea00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fad60065630 0x7fad60113990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:16:16.467 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:16.471+0000 7fad5ea00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fad60065630 0x7fad60113990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:36446/0 (socket says 172.21.15.18:36446) 2024-04-12T21:16:16.467 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:16.471+0000 7fad5ea00700 1 -- 172.21.15.18:0/1529073091 learned_addr learned my addr 172.21.15.18:0/1529073091 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:16:16.467 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:16.471+0000 7fad57e00700 1 --2- 172.21.15.18:0/1529073091 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fad60102ca0 0x7fad60113ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:16:16.467 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:16.471+0000 7fad5ea00700 1 -- 172.21.15.18:0/1529073091 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fad60102ca0 msgr2=0x7fad60113ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:16:16.467 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:16.471+0000 7fad5ea00700 1 --2- 172.21.15.18:0/1529073091 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fad60102ca0 0x7fad60113ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:16:16.467 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:16.471+0000 7fad5ea00700 1 -- 172.21.15.18:0/1529073091 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad48005040 con 0x7fad60065630 2024-04-12T21:16:16.467 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:16.472+0000 7fad5ea00700 1 --2- 172.21.15.18:0/1529073091 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fad60065630 0x7fad60113990 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7fad5000cac0 tx=0x7fad50007ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:16:16.468 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:16.472+0000 7fad56a00700 1 -- 172.21.15.18:0/1529073091 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fad5000aa50 con 0x7fad60065630 2024-04-12T21:16:16.468 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:16.472+0000 7fad56a00700 1 -- 172.21.15.18:0/1529073091 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fad5000abb0 con 0x7fad60065630 2024-04-12T21:16:16.468 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:16.472+0000 7fad56a00700 1 -- 172.21.15.18:0/1529073091 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fad500115e0 con 0x7fad60065630 2024-04-12T21:16:16.468 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:16.472+0000 7fad5fe00700 1 -- 172.21.15.18:0/1529073091 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fad6010dbf0 con 0x7fad60065630 2024-04-12T21:16:16.468 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:16.472+0000 7fad5fe00700 1 -- 172.21.15.18:0/1529073091 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fad6010e1b0 con 0x7fad60065630 2024-04-12T21:16:16.468 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:16.473+0000 7fad5fe00700 1 -- 172.21.15.18:0/1529073091 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fad600fe380 con 0x7fad60065630 2024-04-12T21:16:16.469 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:16.474+0000 7fad56a00700 1 -- 172.21.15.18:0/1529073091 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7fad5000ad20 con 0x7fad60065630 2024-04-12T21:16:16.470 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:16.475+0000 7fad56a00700 1 --2- 172.21.15.18:0/1529073091 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fad4406ec10 0x7fad440710c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:16:16.470 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:16.476+0000 7fad56a00700 1 -- 172.21.15.18:0/1529073091 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 6757+0+0 (secure 0 0 0) 0x7fad5008f0e0 con 0x7fad60065630 2024-04-12T21:16:16.471 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:16.476+0000 7fad57e00700 1 --2- 172.21.15.18:0/1529073091 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fad4406ec10 0x7fad440710c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:16:16.471 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:16.477+0000 7fad57e00700 1 --2- 172.21.15.18:0/1529073091 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fad4406ec10 0x7fad440710c0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fad4800b020 tx=0x7fad48009e30 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:16:16.475 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:16.481+0000 7fad56a00700 1 -- 172.21.15.18:0/1529073091 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7fad5005b890 con 0x7fad60065630 2024-04-12T21:16:16.731 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:16.735+0000 7fad5fe00700 1 -- 172.21.15.18:0/1529073091 --> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fad60109ce0 con 0x7fad4406ec10 2024-04-12T21:16:16.733 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:16.738+0000 7fad56a00700 1 -- 172.21.15.18:0/1529073091 <== mgr.34102 v2:172.21.15.18:6800/3812891919 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7fad60109ce0 con 0x7fad4406ec10 2024-04-12T21:16:16.736 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:16.740+0000 7fad5fe00700 1 -- 172.21.15.18:0/1529073091 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fad4406ec10 msgr2=0x7fad440710c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:16:16.736 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:16.740+0000 7fad5fe00700 1 --2- 172.21.15.18:0/1529073091 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fad4406ec10 0x7fad440710c0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fad4800b020 tx=0x7fad48009e30 comp rx=0 tx=0).stop 2024-04-12T21:16:16.736 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:16.740+0000 7fad5fe00700 1 -- 172.21.15.18:0/1529073091 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fad60065630 msgr2=0x7fad60113990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:16:16.736 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:16.740+0000 7fad5fe00700 1 --2- 172.21.15.18:0/1529073091 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fad60065630 0x7fad60113990 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7fad5000cac0 tx=0x7fad50007ee0 comp rx=0 tx=0).stop 2024-04-12T21:16:16.736 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:16.741+0000 7fad5fe00700 1 -- 172.21.15.18:0/1529073091 shutdown_connections 2024-04-12T21:16:16.736 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:16.741+0000 7fad5fe00700 1 --2- 172.21.15.18:0/1529073091 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fad4406ec10 0x7fad440710c0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:16:16.736 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:16.741+0000 7fad5fe00700 1 --2- 172.21.15.18:0/1529073091 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fad60065630 0x7fad60113990 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:16:16.736 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:16.741+0000 7fad5fe00700 1 --2- 172.21.15.18:0/1529073091 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fad60102ca0 0x7fad60113ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:16:16.736 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:16.741+0000 7fad5fe00700 1 -- 172.21.15.18:0/1529073091 >> 172.21.15.18:0/1529073091 conn(0x7fad60071b70 msgr2=0x7fad601007f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:16:16.737 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:16.741+0000 7fad5fe00700 1 -- 172.21.15.18:0/1529073091 shutdown_connections 2024-04-12T21:16:16.737 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:16.741+0000 7fad5fe00700 1 -- 172.21.15.18:0/1529073091 wait complete. 2024-04-12T21:16:16.898 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:16.902+0000 7fc436a00700 1 -- 172.21.15.18:0/2002427893 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc4300fc4e0 msgr2=0x7fc430109240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:16:16.898 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:16.902+0000 7fc436a00700 1 --2- 172.21.15.18:0/2002427893 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc4300fc4e0 0x7fc430109240 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7fc41c004970 tx=0x7fc41c009e30 comp rx=0 tx=0).stop 2024-04-12T21:16:16.898 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:16.902+0000 7fc436a00700 1 -- 172.21.15.18:0/2002427893 shutdown_connections 2024-04-12T21:16:16.898 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:16.902+0000 7fc436a00700 1 --2- 172.21.15.18:0/2002427893 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc4300fc4e0 0x7fc430109240 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:16:16.898 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:16.902+0000 7fc436a00700 1 --2- 172.21.15.18:0/2002427893 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc4300fbbd0 0x7fc4300fbfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:16:16.898 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:16.902+0000 7fc436a00700 1 -- 172.21.15.18:0/2002427893 >> 172.21.15.18:0/2002427893 conn(0x7fc4300f7860 msgr2=0x7fc4300f9c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:16:16.898 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:16.902+0000 7fc436a00700 1 -- 172.21.15.18:0/2002427893 shutdown_connections 2024-04-12T21:16:16.898 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:16.902+0000 7fc436a00700 1 -- 172.21.15.18:0/2002427893 wait complete. 2024-04-12T21:16:16.900 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:16.903+0000 7fc436a00700 1 Processor -- start 2024-04-12T21:16:16.900 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:16.903+0000 7fc436a00700 1 -- start start 2024-04-12T21:16:16.900 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:16.904+0000 7fc436a00700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc4300fbbd0 0x7fc430115cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:16:16.900 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:16.904+0000 7fc436a00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc4300fc4e0 0x7fc430116210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:16:16.900 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:16.904+0000 7fc436a00700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc430116860 con 0x7fc4300fc4e0 2024-04-12T21:16:16.900 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:16.904+0000 7fc436a00700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc43010fc70 con 0x7fc4300fbbd0 2024-04-12T21:16:16.900 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:16.904+0000 7fc42f400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc4300fc4e0 0x7fc430116210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:16:16.900 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:16.904+0000 7fc42f400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc4300fc4e0 0x7fc430116210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:36466/0 (socket says 172.21.15.18:36466) 2024-04-12T21:16:16.900 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:16.904+0000 7fc42f400700 1 -- 172.21.15.18:0/2602620188 learned_addr learned my addr 172.21.15.18:0/2602620188 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:16:16.901 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:16.904+0000 7fc42fe00700 1 --2- 172.21.15.18:0/2602620188 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc4300fbbd0 0x7fc430115cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:16:16.901 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:16.905+0000 7fc42f400700 1 -- 172.21.15.18:0/2602620188 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc4300fbbd0 msgr2=0x7fc430115cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:16:16.901 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:16.905+0000 7fc42f400700 1 --2- 172.21.15.18:0/2602620188 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc4300fbbd0 0x7fc430115cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:16:16.901 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:16.905+0000 7fc42f400700 1 -- 172.21.15.18:0/2602620188 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc41c005040 con 0x7fc4300fc4e0 2024-04-12T21:16:16.901 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:16.905+0000 7fc42f400700 1 --2- 172.21.15.18:0/2602620188 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc4300fc4e0 0x7fc430116210 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7fc41c000c00 tx=0x7fc41c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:16:16.902 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:16.905+0000 7fc427e00700 1 -- 172.21.15.18:0/2602620188 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc41c00e040 con 0x7fc4300fc4e0 2024-04-12T21:16:16.902 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:16.905+0000 7fc427e00700 1 -- 172.21.15.18:0/2602620188 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc41c00bc10 con 0x7fc4300fc4e0 2024-04-12T21:16:16.902 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:16.905+0000 7fc427e00700 1 -- 172.21.15.18:0/2602620188 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc41c01c3f0 con 0x7fc4300fc4e0 2024-04-12T21:16:16.902 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:16.905+0000 7fc436a00700 1 -- 172.21.15.18:0/2602620188 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc43010fef0 con 0x7fc4300fc4e0 2024-04-12T21:16:16.902 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:16.906+0000 7fc436a00700 1 -- 172.21.15.18:0/2602620188 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc4301104b0 con 0x7fc4300fc4e0 2024-04-12T21:16:16.902 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:16.907+0000 7fc436a00700 1 -- 172.21.15.18:0/2602620188 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc430066710 con 0x7fc4300fc4e0 2024-04-12T21:16:16.903 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:16.908+0000 7fc427e00700 1 -- 172.21.15.18:0/2602620188 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7fc41c00bd80 con 0x7fc4300fc4e0 2024-04-12T21:16:16.904 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:16.909+0000 7fc427e00700 1 --2- 172.21.15.18:0/2602620188 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fc41406ec00 0x7fc4140710b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:16:16.904 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:16.909+0000 7fc427e00700 1 -- 172.21.15.18:0/2602620188 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 6757+0+0 (secure 0 0 0) 0x7fc41c08fc70 con 0x7fc4300fc4e0 2024-04-12T21:16:16.904 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:16.909+0000 7fc42fe00700 1 --2- 172.21.15.18:0/2602620188 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fc41406ec00 0x7fc4140710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:16:16.905 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:16.910+0000 7fc42fe00700 1 --2- 172.21.15.18:0/2602620188 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fc41406ec00 0x7fc4140710b0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fc420004990 tx=0x7fc420009ee0 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:16:16.909 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:16.914+0000 7fc427e00700 1 -- 172.21.15.18:0/2602620188 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7fc41c096080 con 0x7fc4300fc4e0 2024-04-12T21:16:17.149 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:17.153+0000 7fc436a00700 1 -- 172.21.15.18:0/2602620188 --> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fc43010c0f0 con 0x7fc41406ec00 2024-04-12T21:16:17.177 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:17.181+0000 7fc427e00700 1 -- 172.21.15.18:0/2602620188 <== mgr.34102 v2:172.21.15.18:6800/3812891919 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7fc43010c0f0 con 0x7fc41406ec00 2024-04-12T21:16:17.177 INFO:teuthology.orchestra.run.smithi018.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-12T21:16:17.177 INFO:teuthology.orchestra.run.smithi018.stdout:alertmanager.smithi018 smithi018 *:9093,9094 running (16m) 8m ago 31m 14.7M - 0.25.0 c8568f914cd2 c332090068e8 2024-04-12T21:16:17.177 INFO:teuthology.orchestra.run.smithi018.stdout:crash.smithi018 smithi018 running (13m) 8m ago 31m 6665k - 19.0.0-2879-g324baf8d 8717720dc5e5 b7f80d1f7982 2024-04-12T21:16:17.177 INFO:teuthology.orchestra.run.smithi018.stdout:crash.smithi186 smithi186 running (13m) 9m ago 30m 6660k - 19.0.0-2879-g324baf8d 8717720dc5e5 a4f55c81b623 2024-04-12T21:16:17.177 INFO:teuthology.orchestra.run.smithi018.stdout:grafana.smithi018 smithi018 *:3000 running (16m) 8m ago 31m 74.8M - 9.4.12 2bacad6d85d8 b08e3bf176dd 2024-04-12T21:16:17.178 INFO:teuthology.orchestra.run.smithi018.stdout:mds.cephfs.smithi018.acqbsc smithi018 running (9m) 8m ago 24m 11.1M - 19.0.0-2879-g324baf8d 8717720dc5e5 021f99ab24ae 2024-04-12T21:16:17.178 INFO:teuthology.orchestra.run.smithi018.stdout:mds.cephfs.smithi018.szjluo smithi018 running (8m) 8m ago 24m 10.9M - 19.0.0-2879-g324baf8d 8717720dc5e5 0003117479d4 2024-04-12T21:16:17.178 INFO:teuthology.orchestra.run.smithi018.stdout:mds.cephfs.smithi186.jdllyn smithi186 running (24m) 9m ago 24m 15.6M - 17.2.7-827-gd3b5060a 21de9b7369ad 2b2bc46bd29a 2024-04-12T21:16:17.178 INFO:teuthology.orchestra.run.smithi018.stdout:mds.cephfs.smithi186.wjnafs smithi186 running (24m) 9m ago 24m 15.4M - 17.2.7-827-gd3b5060a 21de9b7369ad 74b115d6d6fd 2024-04-12T21:16:17.178 INFO:teuthology.orchestra.run.smithi018.stdout:mgr.smithi018.xjzwga smithi018 *:8443,9283,8765 running (19m) 8m ago 33m 509M - 19.0.0-2879-g324baf8d 8717720dc5e5 56088305eab7 2024-04-12T21:16:17.178 INFO:teuthology.orchestra.run.smithi018.stdout:mgr.smithi186.kqcfxk smithi186 *:8443,9283,8765 running (18m) 9m ago 30m 452M - 19.0.0-2879-g324baf8d 8717720dc5e5 09cd766e5e0d 2024-04-12T21:16:17.178 INFO:teuthology.orchestra.run.smithi018.stdout:mon.smithi018 smithi018 running (14m) 8m ago 33m 57.0M 2048M 19.0.0-2879-g324baf8d 8717720dc5e5 df3e201dfed6 2024-04-12T21:16:17.178 INFO:teuthology.orchestra.run.smithi018.stdout:mon.smithi186 smithi186 running (14m) 9m ago 29m 37.7M 2048M 19.0.0-2879-g324baf8d 8717720dc5e5 c77498da9b2b 2024-04-12T21:16:17.178 INFO:teuthology.orchestra.run.smithi018.stdout:node-exporter.smithi018 smithi018 *:9100 running (17m) 8m ago 31m 7780k - 1.5.0 0da6a335fe13 326c3d9c6847 2024-04-12T21:16:17.178 INFO:teuthology.orchestra.run.smithi018.stdout:node-exporter.smithi186 smithi186 *:9100 running (17m) 9m ago 30m 7826k - 1.5.0 0da6a335fe13 ec2112baeecf 2024-04-12T21:16:17.178 INFO:teuthology.orchestra.run.smithi018.stdout:osd.0 smithi018 running (12m) 8m ago 29m 121M 1559M 19.0.0-2879-g324baf8d 8717720dc5e5 60b1273ca047 2024-04-12T21:16:17.178 INFO:teuthology.orchestra.run.smithi018.stdout:osd.1 smithi018 running (12m) 8m ago 28m 91.3M 1559M 19.0.0-2879-g324baf8d 8717720dc5e5 170ce1e5fb8d 2024-04-12T21:16:17.178 INFO:teuthology.orchestra.run.smithi018.stdout:osd.2 smithi018 running (11m) 8m ago 27m 90.3M 1559M 19.0.0-2879-g324baf8d 8717720dc5e5 c2cbc6aa2839 2024-04-12T21:16:17.178 INFO:teuthology.orchestra.run.smithi018.stdout:osd.3 smithi186 running (10m) 9m ago 27m 96.6M 2581M 19.0.0-2879-g324baf8d 8717720dc5e5 0138e8262e4b 2024-04-12T21:16:17.179 INFO:teuthology.orchestra.run.smithi018.stdout:osd.4 smithi186 running (10m) 9m ago 26m 87.2M 2581M 19.0.0-2879-g324baf8d 8717720dc5e5 b20cb63b25c9 2024-04-12T21:16:17.179 INFO:teuthology.orchestra.run.smithi018.stdout:osd.5 smithi186 running (9m) 9m ago 26m 11.1M 2581M 19.0.0-2879-g324baf8d 8717720dc5e5 2513bea2610e 2024-04-12T21:16:17.179 INFO:teuthology.orchestra.run.smithi018.stdout:prometheus.smithi018 smithi018 *:9095 running (17m) 8m ago 30m 60.6M - 2.43.0 a07b618ecd1d f58841127e21 2024-04-12T21:16:17.181 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:17.185+0000 7fc436a00700 1 -- 172.21.15.18:0/2602620188 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fc41406ec00 msgr2=0x7fc4140710b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:16:17.181 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:17.185+0000 7fc436a00700 1 --2- 172.21.15.18:0/2602620188 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fc41406ec00 0x7fc4140710b0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fc420004990 tx=0x7fc420009ee0 comp rx=0 tx=0).stop 2024-04-12T21:16:17.181 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:17.185+0000 7fc436a00700 1 -- 172.21.15.18:0/2602620188 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc4300fc4e0 msgr2=0x7fc430116210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:16:17.181 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:17.185+0000 7fc436a00700 1 --2- 172.21.15.18:0/2602620188 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc4300fc4e0 0x7fc430116210 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7fc41c000c00 tx=0x7fc41c009e30 comp rx=0 tx=0).stop 2024-04-12T21:16:17.181 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:17.185+0000 7fc436a00700 1 -- 172.21.15.18:0/2602620188 shutdown_connections 2024-04-12T21:16:17.181 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:17.185+0000 7fc436a00700 1 --2- 172.21.15.18:0/2602620188 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fc41406ec00 0x7fc4140710b0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:16:17.181 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:17.185+0000 7fc436a00700 1 --2- 172.21.15.18:0/2602620188 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc4300fbbd0 0x7fc430115cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:16:17.181 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:17.185+0000 7fc436a00700 1 --2- 172.21.15.18:0/2602620188 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc4300fc4e0 0x7fc430116210 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:16:17.181 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:17.185+0000 7fc436a00700 1 -- 172.21.15.18:0/2602620188 >> 172.21.15.18:0/2602620188 conn(0x7fc4300f7860 msgr2=0x7fc4300f7c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:16:17.181 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:17.185+0000 7fc436a00700 1 -- 172.21.15.18:0/2602620188 shutdown_connections 2024-04-12T21:16:17.181 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:17.186+0000 7fc436a00700 1 -- 172.21.15.18:0/2602620188 wait complete. 2024-04-12T21:16:17.326 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:17.331+0000 7f98d1a00700 1 -- 172.21.15.18:0/2567531736 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f98cc0fe870 msgr2=0x7f98cc106f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:16:17.326 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:17.331+0000 7f98d1a00700 1 --2- 172.21.15.18:0/2567531736 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f98cc0fe870 0x7f98cc106f40 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f98b8004940 tx=0x7f98b8009e30 comp rx=0 tx=0).stop 2024-04-12T21:16:17.327 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:17.331+0000 7f98d1a00700 1 -- 172.21.15.18:0/2567531736 shutdown_connections 2024-04-12T21:16:17.327 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:17.331+0000 7f98d1a00700 1 --2- 172.21.15.18:0/2567531736 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f98cc0fe870 0x7f98cc106f40 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:16:17.327 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:17.331+0000 7f98d1a00700 1 --2- 172.21.15.18:0/2567531736 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f98cc0fded0 0x7f98cc0fe2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:16:17.327 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:17.331+0000 7f98d1a00700 1 -- 172.21.15.18:0/2567531736 >> 172.21.15.18:0/2567531736 conn(0x7f98cc072bc0 msgr2=0x7f98cc072fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:16:17.327 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:17.331+0000 7f98d1a00700 1 -- 172.21.15.18:0/2567531736 shutdown_connections 2024-04-12T21:16:17.327 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:17.331+0000 7f98d1a00700 1 -- 172.21.15.18:0/2567531736 wait complete. 2024-04-12T21:16:17.329 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:17.332+0000 7f98d1a00700 1 Processor -- start 2024-04-12T21:16:17.329 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:17.332+0000 7f98d1a00700 1 -- start start 2024-04-12T21:16:17.329 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:17.332+0000 7f98d1a00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f98cc0fded0 0x7f98cc115c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:16:17.329 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:17.332+0000 7f98d1a00700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f98cc0fe870 0x7f98cc1161b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:16:17.329 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:17.332+0000 7f98d1a00700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98cc116800 con 0x7f98cc0fded0 2024-04-12T21:16:17.329 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:17.332+0000 7f98d1a00700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98cc10fbd0 con 0x7f98cc0fe870 2024-04-12T21:16:17.329 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:17.332+0000 7f98caa00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f98cc0fded0 0x7f98cc115c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:16:17.329 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:17.333+0000 7f98caa00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f98cc0fded0 0x7f98cc115c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:36478/0 (socket says 172.21.15.18:36478) 2024-04-12T21:16:17.329 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:17.333+0000 7f98caa00700 1 -- 172.21.15.18:0/3347903669 learned_addr learned my addr 172.21.15.18:0/3347903669 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:16:17.329 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:17.333+0000 7f98caa00700 1 -- 172.21.15.18:0/3347903669 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f98cc0fe870 msgr2=0x7f98cc1161b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:16:17.330 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:17.333+0000 7f98caa00700 1 --2- 172.21.15.18:0/3347903669 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f98cc0fe870 0x7f98cc1161b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:16:17.330 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:17.333+0000 7f98caa00700 1 -- 172.21.15.18:0/3347903669 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98b8005040 con 0x7f98cc0fded0 2024-04-12T21:16:17.330 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:17.333+0000 7f98caa00700 1 --2- 172.21.15.18:0/3347903669 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f98cc0fded0 0x7f98cc115c70 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f98bc002a80 tx=0x7f98bc0046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:16:17.330 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:17.333+0000 7f98c2a00700 1 -- 172.21.15.18:0/3347903669 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f98bc007cc0 con 0x7f98cc0fded0 2024-04-12T21:16:17.330 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:17.334+0000 7f98d1a00700 1 -- 172.21.15.18:0/3347903669 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f98cc10feb0 con 0x7f98cc0fded0 2024-04-12T21:16:17.330 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:17.334+0000 7f98c2a00700 1 -- 172.21.15.18:0/3347903669 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f98bc00f040 con 0x7f98cc0fded0 2024-04-12T21:16:17.330 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:17.334+0000 7f98c2a00700 1 -- 172.21.15.18:0/3347903669 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f98bc0096f0 con 0x7f98cc0fded0 2024-04-12T21:16:17.330 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:17.334+0000 7f98d1a00700 1 -- 172.21.15.18:0/3347903669 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f98cc1103f0 con 0x7f98cc0fded0 2024-04-12T21:16:17.331 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:17.335+0000 7f98d1a00700 1 -- 172.21.15.18:0/3347903669 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f98cc106a00 con 0x7f98cc0fded0 2024-04-12T21:16:17.331 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:17.336+0000 7f98c2a00700 1 -- 172.21.15.18:0/3347903669 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7f98bc00d690 con 0x7f98cc0fded0 2024-04-12T21:16:17.332 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:17.337+0000 7f98c2a00700 1 --2- 172.21.15.18:0/3347903669 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f98b006ec00 0x7f98b00710b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:16:17.332 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:17.337+0000 7f98c2a00700 1 -- 172.21.15.18:0/3347903669 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 6757+0+0 (secure 0 0 0) 0x7f98bc08e070 con 0x7f98cc0fded0 2024-04-12T21:16:17.332 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:17.337+0000 7f98ca000700 1 --2- 172.21.15.18:0/3347903669 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f98b006ec00 0x7f98b00710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:16:17.332 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:17.338+0000 7f98ca000700 1 --2- 172.21.15.18:0/3347903669 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f98b006ec00 0x7f98b00710b0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f98b8004f90 tx=0x7f98b8009e30 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:16:17.336 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:17.341+0000 7f98c2a00700 1 -- 172.21.15.18:0/3347903669 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f98bc05a8c0 con 0x7f98cc0fded0 2024-04-12T21:16:17.662 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:17.667+0000 7f98d1a00700 1 -- 172.21.15.18:0/3347903669 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f98cc109e90 con 0x7f98cc0fded0 2024-04-12T21:16:17.663 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:17.668+0000 7f98c2a00700 1 -- 172.21.15.18:0/3347903669 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+837 (secure 0 0 0) 0x7f98bc05a450 con 0x7f98cc0fded0 2024-04-12T21:16:17.663 INFO:teuthology.orchestra.run.smithi018.stdout:{ 2024-04-12T21:16:17.664 INFO:teuthology.orchestra.run.smithi018.stdout: "mon": { 2024-04-12T21:16:17.664 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 19.0.0-2879-g324baf8d (324baf8ddb91dea46cac50cb535c306115791355) squid (dev)": 2 2024-04-12T21:16:17.664 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-12T21:16:17.664 INFO:teuthology.orchestra.run.smithi018.stdout: "mgr": { 2024-04-12T21:16:17.664 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 19.0.0-2879-g324baf8d (324baf8ddb91dea46cac50cb535c306115791355) squid (dev)": 2 2024-04-12T21:16:17.664 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-12T21:16:17.664 INFO:teuthology.orchestra.run.smithi018.stdout: "osd": { 2024-04-12T21:16:17.664 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 19.0.0-2879-g324baf8d (324baf8ddb91dea46cac50cb535c306115791355) squid (dev)": 6 2024-04-12T21:16:17.664 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-12T21:16:17.664 INFO:teuthology.orchestra.run.smithi018.stdout: "mds": { 2024-04-12T21:16:17.664 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 17.2.7-827-gd3b5060a (d3b5060a17449edfae9385cdf792afec109fca5d) quincy (stable)": 2, 2024-04-12T21:16:17.664 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 19.0.0-2879-g324baf8d (324baf8ddb91dea46cac50cb535c306115791355) squid (dev)": 2 2024-04-12T21:16:17.664 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-12T21:16:17.664 INFO:teuthology.orchestra.run.smithi018.stdout: "overall": { 2024-04-12T21:16:17.664 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 17.2.7-827-gd3b5060a (d3b5060a17449edfae9385cdf792afec109fca5d) quincy (stable)": 2, 2024-04-12T21:16:17.664 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 19.0.0-2879-g324baf8d (324baf8ddb91dea46cac50cb535c306115791355) squid (dev)": 12 2024-04-12T21:16:17.665 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-04-12T21:16:17.665 INFO:teuthology.orchestra.run.smithi018.stdout:} 2024-04-12T21:16:17.666 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:17.671+0000 7f98d1a00700 1 -- 172.21.15.18:0/3347903669 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f98b006ec00 msgr2=0x7f98b00710b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:16:17.666 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:17.671+0000 7f98d1a00700 1 --2- 172.21.15.18:0/3347903669 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f98b006ec00 0x7f98b00710b0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f98b8004f90 tx=0x7f98b8009e30 comp rx=0 tx=0).stop 2024-04-12T21:16:17.666 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:17.671+0000 7f98d1a00700 1 -- 172.21.15.18:0/3347903669 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f98cc0fded0 msgr2=0x7f98cc115c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:16:17.667 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:17.671+0000 7f98d1a00700 1 --2- 172.21.15.18:0/3347903669 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f98cc0fded0 0x7f98cc115c70 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f98bc002a80 tx=0x7f98bc0046b0 comp rx=0 tx=0).stop 2024-04-12T21:16:17.667 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:17.671+0000 7f98d1a00700 1 -- 172.21.15.18:0/3347903669 shutdown_connections 2024-04-12T21:16:17.667 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:17.671+0000 7f98d1a00700 1 --2- 172.21.15.18:0/3347903669 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f98b006ec00 0x7f98b00710b0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:16:17.667 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:17.671+0000 7f98d1a00700 1 --2- 172.21.15.18:0/3347903669 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f98cc0fded0 0x7f98cc115c70 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:16:17.667 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:17.671+0000 7f98d1a00700 1 --2- 172.21.15.18:0/3347903669 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f98cc0fe870 0x7f98cc1161b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:16:17.667 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:17.671+0000 7f98d1a00700 1 -- 172.21.15.18:0/3347903669 >> 172.21.15.18:0/3347903669 conn(0x7f98cc072bc0 msgr2=0x7f98cc0fb650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:16:17.667 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:17.671+0000 7f98d1a00700 1 -- 172.21.15.18:0/3347903669 shutdown_connections 2024-04-12T21:16:17.667 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:17.671+0000 7f98d1a00700 1 -- 172.21.15.18:0/3347903669 wait complete. 2024-04-12T21:16:17.820 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:17.824+0000 7fbfa4800700 1 -- 172.21.15.18:0/3658735006 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fbf9c070300 msgr2=0x7fbf9c0706d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:16:17.820 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:17.824+0000 7fbfa4800700 1 --2- 172.21.15.18:0/3658735006 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fbf9c070300 0x7fbf9c0706d0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7fbf94009d10 tx=0x7fbf94004d10 comp rx=0 tx=0).stop 2024-04-12T21:16:17.820 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:17.824+0000 7fbfa4800700 1 -- 172.21.15.18:0/3658735006 shutdown_connections 2024-04-12T21:16:17.820 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:17.824+0000 7fbfa4800700 1 --2- 172.21.15.18:0/3658735006 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbf9c070c10 0x7fbf9c109180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:16:17.820 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:17.824+0000 7fbfa4800700 1 --2- 172.21.15.18:0/3658735006 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fbf9c070300 0x7fbf9c0706d0 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:16:17.820 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:17.824+0000 7fbfa4800700 1 -- 172.21.15.18:0/3658735006 >> 172.21.15.18:0/3658735006 conn(0x7fbf9c0f89a0 msgr2=0x7fbf9c0fadb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:16:17.820 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:17.824+0000 7fbfa4800700 1 -- 172.21.15.18:0/3658735006 shutdown_connections 2024-04-12T21:16:17.820 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:17.824+0000 7fbfa4800700 1 -- 172.21.15.18:0/3658735006 wait complete. 2024-04-12T21:16:17.822 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:17.825+0000 7fbfa4800700 1 Processor -- start 2024-04-12T21:16:17.823 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:17.825+0000 7fbfa4800700 1 -- start start 2024-04-12T21:16:17.823 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:17.825+0000 7fbfa4800700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbf9c070300 0x7fbf9c116030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:16:17.823 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:17.825+0000 7fbfa4800700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fbf9c070c10 0x7fbf9c116570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:16:17.823 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:17.825+0000 7fbfa4800700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf9c116bc0 con 0x7fbf9c070c10 2024-04-12T21:16:17.823 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:17.825+0000 7fbfa4800700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf9c10ff90 con 0x7fbf9c070300 2024-04-12T21:16:17.823 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:17.826+0000 7fbfa1400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fbf9c070c10 0x7fbf9c116570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:16:17.823 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:17.826+0000 7fbfa1400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fbf9c070c10 0x7fbf9c116570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:36486/0 (socket says 172.21.15.18:36486) 2024-04-12T21:16:17.823 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:17.826+0000 7fbfa1400700 1 -- 172.21.15.18:0/4225416665 learned_addr learned my addr 172.21.15.18:0/4225416665 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:16:17.823 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:17.826+0000 7fbfa1e00700 1 --2- 172.21.15.18:0/4225416665 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbf9c070300 0x7fbf9c116030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:16:17.823 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:17.826+0000 7fbfa1400700 1 -- 172.21.15.18:0/4225416665 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbf9c070300 msgr2=0x7fbf9c116030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:16:17.823 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:17.826+0000 7fbfa1400700 1 --2- 172.21.15.18:0/4225416665 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbf9c070300 0x7fbf9c116030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:16:17.824 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:17.826+0000 7fbfa1400700 1 -- 172.21.15.18:0/4225416665 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf94009940 con 0x7fbf9c070c10 2024-04-12T21:16:17.824 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:17.826+0000 7fbfa1e00700 1 --2- 172.21.15.18:0/4225416665 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbf9c070300 0x7fbf9c116030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-12T21:16:17.824 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:17.827+0000 7fbfa1400700 1 --2- 172.21.15.18:0/4225416665 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fbf9c070c10 0x7fbf9c116570 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7fbf8c00c830 tx=0x7fbf8c004470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:16:17.824 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:17.827+0000 7fbf92000700 1 -- 172.21.15.18:0/4225416665 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbf8c00a760 con 0x7fbf9c070c10 2024-04-12T21:16:17.824 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:17.827+0000 7fbf92000700 1 -- 172.21.15.18:0/4225416665 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fbf8c00ad90 con 0x7fbf9c070c10 2024-04-12T21:16:17.824 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:17.827+0000 7fbf92000700 1 -- 172.21.15.18:0/4225416665 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbf8c0113a0 con 0x7fbf9c070c10 2024-04-12T21:16:17.824 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:17.827+0000 7fbfa4800700 1 -- 172.21.15.18:0/4225416665 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbf9c110270 con 0x7fbf9c070c10 2024-04-12T21:16:17.824 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:17.827+0000 7fbfa4800700 1 -- 172.21.15.18:0/4225416665 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbf9c110760 con 0x7fbf9c070c10 2024-04-12T21:16:17.825 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:17.828+0000 7fbfa4800700 1 -- 172.21.15.18:0/4225416665 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbf9c071a40 con 0x7fbf9c070c10 2024-04-12T21:16:17.825 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:17.829+0000 7fbf92000700 1 -- 172.21.15.18:0/4225416665 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7fbf8c012400 con 0x7fbf9c070c10 2024-04-12T21:16:17.825 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:17.830+0000 7fbf92000700 1 --2- 172.21.15.18:0/4225416665 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fbf8806ec10 0x7fbf880710c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:16:17.826 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:17.830+0000 7fbf92000700 1 -- 172.21.15.18:0/4225416665 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 6757+0+0 (secure 0 0 0) 0x7fbf8c08eaa0 con 0x7fbf9c070c10 2024-04-12T21:16:17.826 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:17.831+0000 7fbfa1e00700 1 --2- 172.21.15.18:0/4225416665 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fbf8806ec10 0x7fbf880710c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:16:17.826 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:17.831+0000 7fbfa1e00700 1 --2- 172.21.15.18:0/4225416665 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fbf8806ec10 0x7fbf880710c0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fbf94005010 tx=0x7fbf94004d10 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:16:17.830 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:17.835+0000 7fbf92000700 1 -- 172.21.15.18:0/4225416665 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7fbf8c05b250 con 0x7fbf9c070c10 2024-04-12T21:16:18.028 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:16:17 smithi186 ceph-mon[86379]: pgmap v494: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:16:18.028 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:16:17 smithi186 ceph-mon[86379]: from='client.34892 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:16:18.097 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:18.101+0000 7fbfa4800700 1 -- 172.21.15.18:0/4225416665 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fbf9c04bed0 con 0x7fbf9c070c10 2024-04-12T21:16:18.097 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:16:17 smithi018 ceph-mon[112590]: pgmap v494: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:16:18.097 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:16:17 smithi018 ceph-mon[112590]: from='client.34892 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:16:18.098 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:18.102+0000 7fbf92000700 1 -- 172.21.15.18:0/4225416665 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 29 v29) v1 ==== 76+0+1872 (secure 0 0 0) 0x7fbf8c05ade0 con 0x7fbf9c070c10 2024-04-12T21:16:18.098 INFO:teuthology.orchestra.run.smithi018.stdout:e29 2024-04-12T21:16:18.098 INFO:teuthology.orchestra.run.smithi018.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-12T21:16:18.098 INFO:teuthology.orchestra.run.smithi018.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-12T21:16:18.098 INFO:teuthology.orchestra.run.smithi018.stdout:legacy client fscid: 1 2024-04-12T21:16:18.098 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-12T21:16:18.099 INFO:teuthology.orchestra.run.smithi018.stdout:Filesystem 'cephfs' (1) 2024-04-12T21:16:18.099 INFO:teuthology.orchestra.run.smithi018.stdout:fs_name cephfs 2024-04-12T21:16:18.099 INFO:teuthology.orchestra.run.smithi018.stdout:epoch 24 2024-04-12T21:16:18.099 INFO:teuthology.orchestra.run.smithi018.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-04-12T21:16:18.099 INFO:teuthology.orchestra.run.smithi018.stdout:created 2024-04-12T20:51:57.362850+0000 2024-04-12T21:16:18.099 INFO:teuthology.orchestra.run.smithi018.stdout:modified 2024-04-12T21:06:59.717190+0000 2024-04-12T21:16:18.099 INFO:teuthology.orchestra.run.smithi018.stdout:tableserver 0 2024-04-12T21:16:18.099 INFO:teuthology.orchestra.run.smithi018.stdout:root 0 2024-04-12T21:16:18.099 INFO:teuthology.orchestra.run.smithi018.stdout:session_timeout 60 2024-04-12T21:16:18.099 INFO:teuthology.orchestra.run.smithi018.stdout:session_autoclose 300 2024-04-12T21:16:18.099 INFO:teuthology.orchestra.run.smithi018.stdout:max_file_size 1099511627776 2024-04-12T21:16:18.099 INFO:teuthology.orchestra.run.smithi018.stdout:max_xattr_size 65536 2024-04-12T21:16:18.099 INFO:teuthology.orchestra.run.smithi018.stdout:required_client_features {} 2024-04-12T21:16:18.099 INFO:teuthology.orchestra.run.smithi018.stdout:last_failure 0 2024-04-12T21:16:18.099 INFO:teuthology.orchestra.run.smithi018.stdout:last_failure_osd_epoch 100 2024-04-12T21:16:18.100 INFO:teuthology.orchestra.run.smithi018.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-12T21:16:18.100 INFO:teuthology.orchestra.run.smithi018.stdout:max_mds 2 2024-04-12T21:16:18.100 INFO:teuthology.orchestra.run.smithi018.stdout:in 0,1 2024-04-12T21:16:18.100 INFO:teuthology.orchestra.run.smithi018.stdout:up {} 2024-04-12T21:16:18.100 INFO:teuthology.orchestra.run.smithi018.stdout:failed 0,1 2024-04-12T21:16:18.100 INFO:teuthology.orchestra.run.smithi018.stdout:damaged 2024-04-12T21:16:18.100 INFO:teuthology.orchestra.run.smithi018.stdout:stopped 2024-04-12T21:16:18.100 INFO:teuthology.orchestra.run.smithi018.stdout:data_pools [3] 2024-04-12T21:16:18.100 INFO:teuthology.orchestra.run.smithi018.stdout:metadata_pool 2 2024-04-12T21:16:18.100 INFO:teuthology.orchestra.run.smithi018.stdout:inline_data disabled 2024-04-12T21:16:18.100 INFO:teuthology.orchestra.run.smithi018.stdout:balancer 2024-04-12T21:16:18.100 INFO:teuthology.orchestra.run.smithi018.stdout:bal_rank_mask -1 2024-04-12T21:16:18.101 INFO:teuthology.orchestra.run.smithi018.stdout:standby_count_wanted 1 2024-04-12T21:16:18.101 INFO:teuthology.orchestra.run.smithi018.stdout:qdb_cluster leader: 0 members: 2024-04-12T21:16:18.101 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-12T21:16:18.101 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-12T21:16:18.101 INFO:teuthology.orchestra.run.smithi018.stdout:Standby daemons: 2024-04-12T21:16:18.101 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-12T21:16:18.101 INFO:teuthology.orchestra.run.smithi018.stdout:[mds.cephfs.smithi186.wjnafs{-1:24277} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.186:6826/438347207,v1:172.21.15.186:6827/438347207] compat {c=[1],r=[1],i=[7ff]}] 2024-04-12T21:16:18.101 INFO:teuthology.orchestra.run.smithi018.stdout:[mds.cephfs.smithi186.jdllyn{-1:24281} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.186:6824/1464359952,v1:172.21.15.186:6825/1464359952] compat {c=[1],r=[1],i=[7ff]}] 2024-04-12T21:16:18.101 INFO:teuthology.orchestra.run.smithi018.stdout:[mds.cephfs.smithi018.acqbsc{-1:34436} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.18:6828/755179764,v1:172.21.15.18:6829/755179764] compat {c=[1],r=[1],i=[1fff]}] 2024-04-12T21:16:18.101 INFO:teuthology.orchestra.run.smithi018.stdout:[mds.cephfs.smithi018.szjluo{-1:34468} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.18:6826/2407992653,v1:172.21.15.18:6827/2407992653] compat {c=[1],r=[1],i=[1fff]}] 2024-04-12T21:16:18.102 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:18.105+0000 7fbfa4800700 1 -- 172.21.15.18:0/4225416665 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fbf8806ec10 msgr2=0x7fbf880710c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:16:18.102 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:18.105+0000 7fbfa4800700 1 --2- 172.21.15.18:0/4225416665 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fbf8806ec10 0x7fbf880710c0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fbf94005010 tx=0x7fbf94004d10 comp rx=0 tx=0).stop 2024-04-12T21:16:18.102 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:18.105+0000 7fbfa4800700 1 -- 172.21.15.18:0/4225416665 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fbf9c070c10 msgr2=0x7fbf9c116570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:16:18.102 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:18.105+0000 7fbfa4800700 1 --2- 172.21.15.18:0/4225416665 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fbf9c070c10 0x7fbf9c116570 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7fbf8c00c830 tx=0x7fbf8c004470 comp rx=0 tx=0).stop 2024-04-12T21:16:18.102 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:18.105+0000 7fbfa4800700 1 -- 172.21.15.18:0/4225416665 shutdown_connections 2024-04-12T21:16:18.102 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:18.105+0000 7fbfa4800700 1 --2- 172.21.15.18:0/4225416665 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fbf8806ec10 0x7fbf880710c0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:16:18.103 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:18.105+0000 7fbfa4800700 1 --2- 172.21.15.18:0/4225416665 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbf9c070300 0x7fbf9c116030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:16:18.103 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:18.105+0000 7fbfa4800700 1 --2- 172.21.15.18:0/4225416665 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fbf9c070c10 0x7fbf9c116570 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:16:18.103 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:18.105+0000 7fbfa4800700 1 -- 172.21.15.18:0/4225416665 >> 172.21.15.18:0/4225416665 conn(0x7fbf9c0f89a0 msgr2=0x7fbf9c0f9f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:16:18.103 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:18.106+0000 7fbfa4800700 1 -- 172.21.15.18:0/4225416665 shutdown_connections 2024-04-12T21:16:18.103 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:18.106+0000 7fbfa4800700 1 -- 172.21.15.18:0/4225416665 wait complete. 2024-04-12T21:16:18.103 INFO:teuthology.orchestra.run.smithi018.stderr:dumped fsmap epoch 29 2024-04-12T21:16:18.261 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:18.266+0000 7fb912600700 1 -- 172.21.15.18:0/1533013044 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb90c0724f0 msgr2=0x7fb90c0728c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:16:18.262 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:18.266+0000 7fb912600700 1 --2- 172.21.15.18:0/1533013044 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb90c0724f0 0x7fb90c0728c0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7fb8fc004940 tx=0x7fb8fc009e30 comp rx=0 tx=0).stop 2024-04-12T21:16:18.262 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:18.266+0000 7fb912600700 1 -- 172.21.15.18:0/1533013044 shutdown_connections 2024-04-12T21:16:18.262 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:18.266+0000 7fb912600700 1 --2- 172.21.15.18:0/1533013044 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb90c072e00 0x7fb90c10d770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:16:18.262 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:18.266+0000 7fb912600700 1 --2- 172.21.15.18:0/1533013044 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb90c0724f0 0x7fb90c0728c0 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:16:18.262 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:18.266+0000 7fb912600700 1 -- 172.21.15.18:0/1533013044 >> 172.21.15.18:0/1533013044 conn(0x7fb90c0fabd0 msgr2=0x7fb90c0fcfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:16:18.262 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:18.266+0000 7fb912600700 1 -- 172.21.15.18:0/1533013044 shutdown_connections 2024-04-12T21:16:18.262 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:18.266+0000 7fb912600700 1 -- 172.21.15.18:0/1533013044 wait complete. 2024-04-12T21:16:18.263 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:18.267+0000 7fb912600700 1 Processor -- start 2024-04-12T21:16:18.263 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:18.267+0000 7fb912600700 1 -- start start 2024-04-12T21:16:18.263 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:18.268+0000 7fb912600700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb90c0724f0 0x7fb90c11a0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:16:18.263 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:18.268+0000 7fb912600700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb90c072e00 0x7fb90c11a610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:16:18.264 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:18.268+0000 7fb912600700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb90c11ac60 con 0x7fb90c0724f0 2024-04-12T21:16:18.264 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:18.268+0000 7fb912600700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb90c1140c0 con 0x7fb90c072e00 2024-04-12T21:16:18.264 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:18.268+0000 7fb90b400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb90c0724f0 0x7fb90c11a0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:16:18.264 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:18.268+0000 7fb90b400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb90c0724f0 0x7fb90c11a0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:36508/0 (socket says 172.21.15.18:36508) 2024-04-12T21:16:18.264 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:18.268+0000 7fb90b400700 1 -- 172.21.15.18:0/1662136632 learned_addr learned my addr 172.21.15.18:0/1662136632 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:16:18.265 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:18.269+0000 7fb90aa00700 1 --2- 172.21.15.18:0/1662136632 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb90c072e00 0x7fb90c11a610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:16:18.265 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:18.269+0000 7fb90b400700 1 -- 172.21.15.18:0/1662136632 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb90c072e00 msgr2=0x7fb90c11a610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:16:18.265 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:18.269+0000 7fb90b400700 1 --2- 172.21.15.18:0/1662136632 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb90c072e00 0x7fb90c11a610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:16:18.265 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:18.269+0000 7fb90b400700 1 -- 172.21.15.18:0/1662136632 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8fc005040 con 0x7fb90c0724f0 2024-04-12T21:16:18.265 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:18.269+0000 7fb90aa00700 1 --2- 172.21.15.18:0/1662136632 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb90c072e00 0x7fb90c11a610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-12T21:16:18.265 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:18.269+0000 7fb90b400700 1 --2- 172.21.15.18:0/1662136632 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb90c0724f0 0x7fb90c11a0d0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7fb8fc009180 tx=0x7fb8fc009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:16:18.265 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:18.269+0000 7fb903400700 1 -- 172.21.15.18:0/1662136632 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb8fc00e040 con 0x7fb90c0724f0 2024-04-12T21:16:18.265 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:18.269+0000 7fb903400700 1 -- 172.21.15.18:0/1662136632 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb8fc013420 con 0x7fb90c0724f0 2024-04-12T21:16:18.266 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:18.269+0000 7fb903400700 1 -- 172.21.15.18:0/1662136632 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb8fc01c630 con 0x7fb90c0724f0 2024-04-12T21:16:18.266 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:18.270+0000 7fb912600700 1 -- 172.21.15.18:0/1662136632 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb90c114340 con 0x7fb90c0724f0 2024-04-12T21:16:18.266 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:18.270+0000 7fb912600700 1 -- 172.21.15.18:0/1662136632 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb90c114900 con 0x7fb90c0724f0 2024-04-12T21:16:18.266 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:18.271+0000 7fb912600700 1 -- 172.21.15.18:0/1662136632 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb90c073c30 con 0x7fb90c0724f0 2024-04-12T21:16:18.267 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:18.272+0000 7fb903400700 1 -- 172.21.15.18:0/1662136632 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7fb8fc013590 con 0x7fb90c0724f0 2024-04-12T21:16:18.268 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:18.273+0000 7fb903400700 1 --2- 172.21.15.18:0/1662136632 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fb8f006ec10 0x7fb8f00710c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:16:18.268 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:18.273+0000 7fb903400700 1 -- 172.21.15.18:0/1662136632 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 6757+0+0 (secure 0 0 0) 0x7fb8fc08fc60 con 0x7fb90c0724f0 2024-04-12T21:16:18.269 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:18.273+0000 7fb90aa00700 1 --2- 172.21.15.18:0/1662136632 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fb8f006ec10 0x7fb8f00710c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:16:18.269 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:18.274+0000 7fb90aa00700 1 --2- 172.21.15.18:0/1662136632 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fb8f006ec10 0x7fb8f00710c0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fb8f8009fd0 tx=0x7fb8f800b040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:16:18.273 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:18.278+0000 7fb903400700 1 -- 172.21.15.18:0/1662136632 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7fb8fc058a20 con 0x7fb90c0724f0 2024-04-12T21:16:18.527 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:18.532+0000 7fb912600700 1 -- 172.21.15.18:0/1662136632 --> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fb90c04bf40 con 0x7fb8f006ec10 2024-04-12T21:16:18.529 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:18.534+0000 7fb903400700 1 -- 172.21.15.18:0/1662136632 <== mgr.34102 v2:172.21.15.18:6800/3812891919 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7fb90c04bf40 con 0x7fb8f006ec10 2024-04-12T21:16:18.529 INFO:teuthology.orchestra.run.smithi018.stdout:{ 2024-04-12T21:16:18.529 INFO:teuthology.orchestra.run.smithi018.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:f4452def3be4d350f4a8e5feaf7ebb1511be9298fe68478ba7a2d6fbd7d69a63", 2024-04-12T21:16:18.529 INFO:teuthology.orchestra.run.smithi018.stdout: "in_progress": true, 2024-04-12T21:16:18.530 INFO:teuthology.orchestra.run.smithi018.stdout: "which": "Upgrading all daemon types on all hosts", 2024-04-12T21:16:18.530 INFO:teuthology.orchestra.run.smithi018.stdout: "services_complete": [ 2024-04-12T21:16:18.530 INFO:teuthology.orchestra.run.smithi018.stdout: "crash", 2024-04-12T21:16:18.530 INFO:teuthology.orchestra.run.smithi018.stdout: "osd", 2024-04-12T21:16:18.530 INFO:teuthology.orchestra.run.smithi018.stdout: "mon", 2024-04-12T21:16:18.530 INFO:teuthology.orchestra.run.smithi018.stdout: "mgr" 2024-04-12T21:16:18.530 INFO:teuthology.orchestra.run.smithi018.stdout: ], 2024-04-12T21:16:18.530 INFO:teuthology.orchestra.run.smithi018.stdout: "progress": "14/21 daemons upgraded", 2024-04-12T21:16:18.530 INFO:teuthology.orchestra.run.smithi018.stdout: "message": "Currently upgrading mds daemons", 2024-04-12T21:16:18.530 INFO:teuthology.orchestra.run.smithi018.stdout: "is_paused": false 2024-04-12T21:16:18.530 INFO:teuthology.orchestra.run.smithi018.stdout:} 2024-04-12T21:16:18.532 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:18.536+0000 7fb912600700 1 -- 172.21.15.18:0/1662136632 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fb8f006ec10 msgr2=0x7fb8f00710c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:16:18.532 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:18.536+0000 7fb912600700 1 --2- 172.21.15.18:0/1662136632 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fb8f006ec10 0x7fb8f00710c0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fb8f8009fd0 tx=0x7fb8f800b040 comp rx=0 tx=0).stop 2024-04-12T21:16:18.533 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:18.536+0000 7fb912600700 1 -- 172.21.15.18:0/1662136632 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb90c0724f0 msgr2=0x7fb90c11a0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:16:18.533 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:18.537+0000 7fb912600700 1 --2- 172.21.15.18:0/1662136632 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb90c0724f0 0x7fb90c11a0d0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7fb8fc009180 tx=0x7fb8fc009e30 comp rx=0 tx=0).stop 2024-04-12T21:16:18.533 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:18.537+0000 7fb912600700 1 -- 172.21.15.18:0/1662136632 shutdown_connections 2024-04-12T21:16:18.533 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:18.537+0000 7fb912600700 1 --2- 172.21.15.18:0/1662136632 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fb8f006ec10 0x7fb8f00710c0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:16:18.533 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:18.537+0000 7fb912600700 1 --2- 172.21.15.18:0/1662136632 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb90c0724f0 0x7fb90c11a0d0 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:16:18.533 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:18.537+0000 7fb912600700 1 --2- 172.21.15.18:0/1662136632 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb90c072e00 0x7fb90c11a610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:16:18.533 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:18.537+0000 7fb912600700 1 -- 172.21.15.18:0/1662136632 >> 172.21.15.18:0/1662136632 conn(0x7fb90c0fabd0 msgr2=0x7fb90c0fc0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:16:18.533 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:18.537+0000 7fb912600700 1 -- 172.21.15.18:0/1662136632 shutdown_connections 2024-04-12T21:16:18.533 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:18.537+0000 7fb912600700 1 -- 172.21.15.18:0/1662136632 wait complete. 2024-04-12T21:16:18.694 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:18.699+0000 7f7f1f200700 1 -- 172.21.15.18:0/1713011124 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7f20101ae0 msgr2=0x7f7f20065630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:16:18.694 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:18.699+0000 7f7f1f200700 1 --2- 172.21.15.18:0/1713011124 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7f20101ae0 0x7f7f20065630 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f7f10004940 tx=0x7f7f10009e30 comp rx=0 tx=0).stop 2024-04-12T21:16:18.695 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:18.699+0000 7f7f1f200700 1 -- 172.21.15.18:0/1713011124 shutdown_connections 2024-04-12T21:16:18.695 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:18.699+0000 7f7f1f200700 1 --2- 172.21.15.18:0/1713011124 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7f20065b70 0x7f7f20065fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:16:18.695 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:18.699+0000 7f7f1f200700 1 --2- 172.21.15.18:0/1713011124 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7f20101ae0 0x7f7f20065630 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:16:18.695 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:18.699+0000 7f7f1f200700 1 -- 172.21.15.18:0/1713011124 >> 172.21.15.18:0/1713011124 conn(0x7f7f20071b70 msgr2=0x7f7f20071f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:16:18.695 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:18.699+0000 7f7f1f200700 1 -- 172.21.15.18:0/1713011124 shutdown_connections 2024-04-12T21:16:18.695 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:18.699+0000 7f7f1f200700 1 -- 172.21.15.18:0/1713011124 wait complete. 2024-04-12T21:16:18.696 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:18.700+0000 7f7f1f200700 1 Processor -- start 2024-04-12T21:16:18.696 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:18.700+0000 7f7f1f200700 1 -- start start 2024-04-12T21:16:18.697 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:18.701+0000 7f7f1f200700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7f20065b70 0x7f7f2011a020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:16:18.697 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:18.701+0000 7f7f1f200700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7f20101ae0 0x7f7f2011a560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:16:18.697 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:18.701+0000 7f7f1f200700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f2011abb0 con 0x7f7f20101ae0 2024-04-12T21:16:18.697 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:18.701+0000 7f7f1f200700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f20113f80 con 0x7f7f20065b70 2024-04-12T21:16:18.697 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:18.701+0000 7f7f17e00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7f20101ae0 0x7f7f2011a560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:16:18.697 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:18.701+0000 7f7f17e00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7f20101ae0 0x7f7f2011a560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:36540/0 (socket says 172.21.15.18:36540) 2024-04-12T21:16:18.697 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:18.701+0000 7f7f17e00700 1 -- 172.21.15.18:0/1492976818 learned_addr learned my addr 172.21.15.18:0/1492976818 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:16:18.697 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:18.701+0000 7f7f1de00700 1 --2- 172.21.15.18:0/1492976818 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7f20065b70 0x7f7f2011a020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:16:18.697 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:18.701+0000 7f7f17e00700 1 -- 172.21.15.18:0/1492976818 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7f20065b70 msgr2=0x7f7f2011a020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:16:18.697 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:18.701+0000 7f7f17e00700 1 --2- 172.21.15.18:0/1492976818 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7f20065b70 0x7f7f2011a020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:16:18.697 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:18.702+0000 7f7f17e00700 1 -- 172.21.15.18:0/1492976818 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f10005040 con 0x7f7f20101ae0 2024-04-12T21:16:18.699 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:18.702+0000 7f7f17e00700 1 --2- 172.21.15.18:0/1492976818 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7f20101ae0 0x7f7f2011a560 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f7f0800c830 tx=0x7f7f08004470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:16:18.699 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:18.702+0000 7f7f16000700 1 -- 172.21.15.18:0/1492976818 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7f0800a760 con 0x7f7f20101ae0 2024-04-12T21:16:18.699 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:18.702+0000 7f7f16000700 1 -- 172.21.15.18:0/1492976818 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f7f0800ad90 con 0x7f7f20101ae0 2024-04-12T21:16:18.699 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:18.702+0000 7f7f16000700 1 -- 172.21.15.18:0/1492976818 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7f080113a0 con 0x7f7f20101ae0 2024-04-12T21:16:18.699 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:18.702+0000 7f7f1f200700 1 -- 172.21.15.18:0/1492976818 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7f20114260 con 0x7f7f20101ae0 2024-04-12T21:16:18.700 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:18.702+0000 7f7f1f200700 1 -- 172.21.15.18:0/1492976818 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7f201147a0 con 0x7f7f20101ae0 2024-04-12T21:16:18.700 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:18.703+0000 7f7f1f200700 1 -- 172.21.15.18:0/1492976818 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7f200fc170 con 0x7f7f20101ae0 2024-04-12T21:16:18.701 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:18.705+0000 7f7f16000700 1 -- 172.21.15.18:0/1492976818 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7f7f08012400 con 0x7f7f20101ae0 2024-04-12T21:16:18.701 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:18.706+0000 7f7f16000700 1 --2- 172.21.15.18:0/1492976818 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f7f0406ec00 0x7f7f040710b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:16:18.701 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:18.706+0000 7f7f16000700 1 -- 172.21.15.18:0/1492976818 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 6757+0+0 (secure 0 0 0) 0x7f7f0808e900 con 0x7f7f20101ae0 2024-04-12T21:16:18.701 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:18.706+0000 7f7f1de00700 1 --2- 172.21.15.18:0/1492976818 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f7f0406ec00 0x7f7f040710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:16:18.702 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:18.707+0000 7f7f1de00700 1 --2- 172.21.15.18:0/1492976818 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f7f0406ec00 0x7f7f040710b0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f7f10004f90 tx=0x7f7f10009e30 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:16:18.706 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:18.711+0000 7f7f16000700 1 -- 172.21.15.18:0/1492976818 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f7f0805b1f0 con 0x7f7f20101ae0 2024-04-12T21:16:18.850 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:16:18 smithi018 ceph-mon[112590]: from='client.34896 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:16:18.850 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:16:18 smithi018 ceph-mon[112590]: from='client.34900 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:16:18.850 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:16:18 smithi018 ceph-mon[112590]: from='client.? 172.21.15.18:0/3347903669' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T21:16:18.850 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:16:18 smithi018 ceph-mon[112590]: from='client.? 172.21.15.18:0/4225416665' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-12T21:16:19.028 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:16:18 smithi186 ceph-mon[86379]: from='client.34896 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:16:19.029 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:16:18 smithi186 ceph-mon[86379]: from='client.34900 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:16:19.029 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:16:18 smithi186 ceph-mon[86379]: from='client.? 172.21.15.18:0/3347903669' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T21:16:19.029 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:16:18 smithi186 ceph-mon[86379]: from='client.? 172.21.15.18:0/4225416665' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-12T21:16:19.032 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:19.036+0000 7f7f1f200700 1 -- 172.21.15.18:0/1492976818 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f7f2005f5f0 con 0x7f7f20101ae0 2024-04-12T21:16:19.032 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:19.037+0000 7f7f16000700 1 -- 172.21.15.18:0/1492976818 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+233 (secure 0 0 0) 0x7f7f0805ad80 con 0x7f7f20101ae0 2024-04-12T21:16:19.033 INFO:teuthology.orchestra.run.smithi018.stdout:HEALTH_ERR 1 filesystem is degraded; 1 filesystem is offline 2024-04-12T21:16:19.033 INFO:teuthology.orchestra.run.smithi018.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-04-12T21:16:19.033 INFO:teuthology.orchestra.run.smithi018.stdout: fs cephfs is degraded 2024-04-12T21:16:19.033 INFO:teuthology.orchestra.run.smithi018.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-04-12T21:16:19.033 INFO:teuthology.orchestra.run.smithi018.stdout: fs cephfs is offline because no MDS is active for it. 2024-04-12T21:16:19.035 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:19.040+0000 7f7f1f200700 1 -- 172.21.15.18:0/1492976818 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f7f0406ec00 msgr2=0x7f7f040710b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:16:19.035 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:19.040+0000 7f7f1f200700 1 --2- 172.21.15.18:0/1492976818 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f7f0406ec00 0x7f7f040710b0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f7f10004f90 tx=0x7f7f10009e30 comp rx=0 tx=0).stop 2024-04-12T21:16:19.035 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:19.040+0000 7f7f1f200700 1 -- 172.21.15.18:0/1492976818 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7f20101ae0 msgr2=0x7f7f2011a560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:16:19.036 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:19.040+0000 7f7f1f200700 1 --2- 172.21.15.18:0/1492976818 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7f20101ae0 0x7f7f2011a560 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f7f0800c830 tx=0x7f7f08004470 comp rx=0 tx=0).stop 2024-04-12T21:16:19.036 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:19.040+0000 7f7f1f200700 1 -- 172.21.15.18:0/1492976818 shutdown_connections 2024-04-12T21:16:19.036 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:19.040+0000 7f7f1f200700 1 --2- 172.21.15.18:0/1492976818 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f7f0406ec00 0x7f7f040710b0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:16:19.036 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:19.040+0000 7f7f1f200700 1 --2- 172.21.15.18:0/1492976818 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7f20065b70 0x7f7f2011a020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:16:19.036 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:19.040+0000 7f7f1f200700 1 --2- 172.21.15.18:0/1492976818 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7f20101ae0 0x7f7f2011a560 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:16:19.037 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:19.040+0000 7f7f1f200700 1 -- 172.21.15.18:0/1492976818 >> 172.21.15.18:0/1492976818 conn(0x7f7f20071b70 msgr2=0x7f7f200f9b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:16:19.037 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:19.040+0000 7f7f1f200700 1 -- 172.21.15.18:0/1492976818 shutdown_connections 2024-04-12T21:16:19.037 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:19.040+0000 7f7f1f200700 1 -- 172.21.15.18:0/1492976818 wait complete. 2024-04-12T21:16:20.028 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:16:19 smithi186 ceph-mon[86379]: pgmap v495: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:16:20.029 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:16:19 smithi186 ceph-mon[86379]: from='client.34912 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:16:20.029 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:16:19 smithi186 ceph-mon[86379]: from='client.? 172.21.15.18:0/1492976818' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-12T21:16:20.099 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:16:19 smithi018 ceph-mon[112590]: pgmap v495: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:16:20.100 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:16:19 smithi018 ceph-mon[112590]: from='client.34912 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:16:20.100 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:16:19 smithi018 ceph-mon[112590]: from='client.? 172.21.15.18:0/1492976818' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-12T21:16:22.028 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:16:21 smithi186 ceph-mon[86379]: pgmap v496: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:16:22.099 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:16:21 smithi018 ceph-mon[112590]: pgmap v496: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:16:24.028 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:16:23 smithi186 ceph-mon[86379]: pgmap v497: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:16:24.099 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:16:23 smithi018 ceph-mon[112590]: pgmap v497: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:16:26.028 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:16:25 smithi186 ceph-mon[86379]: pgmap v498: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:16:26.029 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:16:25 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T21:16:26.099 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:16:25 smithi018 ceph-mon[112590]: pgmap v498: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:16:26.100 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:16:25 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T21:16:28.028 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:16:27 smithi186 ceph-mon[86379]: pgmap v499: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:16:28.099 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:16:27 smithi018 ceph-mon[112590]: pgmap v499: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:16:30.028 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:16:29 smithi186 ceph-mon[86379]: pgmap v500: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:16:30.099 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:16:29 smithi018 ceph-mon[112590]: pgmap v500: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:16:32.028 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:16:31 smithi186 ceph-mon[86379]: pgmap v501: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:16:32.099 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:16:31 smithi018 ceph-mon[112590]: pgmap v501: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:16:34.028 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:16:33 smithi186 ceph-mon[86379]: pgmap v502: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:16:34.099 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:16:33 smithi018 ceph-mon[112590]: pgmap v502: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:16:35.028 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:16:34 smithi186 ceph-mon[86379]: pgmap v503: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:16:35.102 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:16:34 smithi018 ceph-mon[112590]: pgmap v503: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:16:37.528 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:16:37 smithi186 ceph-mon[86379]: pgmap v504: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:16:37.599 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:16:37 smithi018 ceph-mon[112590]: pgmap v504: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:16:39.528 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:16:39 smithi186 ceph-mon[86379]: pgmap v505: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:16:39.599 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:16:39 smithi018 ceph-mon[112590]: pgmap v505: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:16:40.500 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:16:40 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T21:16:40.528 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:16:40 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T21:16:41.528 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:16:41 smithi186 ceph-mon[86379]: pgmap v506: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:16:41.599 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:16:41 smithi018 ceph-mon[112590]: pgmap v506: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:16:43.528 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:16:43 smithi186 ceph-mon[86379]: pgmap v507: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:16:43.599 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:16:43 smithi018 ceph-mon[112590]: pgmap v507: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:16:45.528 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:16:45 smithi186 ceph-mon[86379]: pgmap v508: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:16:45.599 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:16:45 smithi018 ceph-mon[112590]: pgmap v508: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:16:47.528 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:16:47 smithi186 ceph-mon[86379]: pgmap v509: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:16:47.599 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:16:47 smithi018 ceph-mon[112590]: pgmap v509: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:16:49.209 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:49.212+0000 7f23edc00700 1 -- 172.21.15.18:0/2749292773 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f23e8109430 msgr2=0x7f23e8109800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:16:49.209 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:49.212+0000 7f23edc00700 1 --2- 172.21.15.18:0/2749292773 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f23e8109430 0x7f23e8109800 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f23d8004940 tx=0x7f23d8009e30 comp rx=0 tx=0).stop 2024-04-12T21:16:49.209 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:49.213+0000 7f23edc00700 1 -- 172.21.15.18:0/2749292773 shutdown_connections 2024-04-12T21:16:49.209 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:49.213+0000 7f23edc00700 1 --2- 172.21.15.18:0/2749292773 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f23e8109dd0 0x7f23e8103440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:16:49.209 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:49.213+0000 7f23edc00700 1 --2- 172.21.15.18:0/2749292773 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f23e8109430 0x7f23e8109800 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:16:49.210 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:49.213+0000 7f23edc00700 1 -- 172.21.15.18:0/2749292773 >> 172.21.15.18:0/2749292773 conn(0x7f23e8072c00 msgr2=0x7f23e8075010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:16:49.210 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:49.213+0000 7f23edc00700 1 -- 172.21.15.18:0/2749292773 shutdown_connections 2024-04-12T21:16:49.210 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:49.213+0000 7f23edc00700 1 -- 172.21.15.18:0/2749292773 wait complete. 2024-04-12T21:16:49.210 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:49.214+0000 7f23edc00700 1 Processor -- start 2024-04-12T21:16:49.210 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:49.215+0000 7f23edc00700 1 -- start start 2024-04-12T21:16:49.211 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:49.215+0000 7f23edc00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f23e8109430 0x7f23e8119f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:16:49.211 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:49.215+0000 7f23edc00700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f23e8109dd0 0x7f23e811a4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:16:49.211 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:49.215+0000 7f23edc00700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23e811ab20 con 0x7f23e8109430 2024-04-12T21:16:49.211 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:49.215+0000 7f23edc00700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23e8114100 con 0x7f23e8109dd0 2024-04-12T21:16:49.212 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:49.215+0000 7f23e6a00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f23e8109430 0x7f23e8119f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:16:49.212 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:49.216+0000 7f23e6a00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f23e8109430 0x7f23e8119f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:48068/0 (socket says 172.21.15.18:48068) 2024-04-12T21:16:49.212 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:49.216+0000 7f23e6a00700 1 -- 172.21.15.18:0/4284419137 learned_addr learned my addr 172.21.15.18:0/4284419137 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:16:49.213 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:49.216+0000 7f23dfe00700 1 --2- 172.21.15.18:0/4284419137 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f23e8109dd0 0x7f23e811a4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:16:49.213 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:49.216+0000 7f23e6a00700 1 -- 172.21.15.18:0/4284419137 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f23e8109dd0 msgr2=0x7f23e811a4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:16:49.213 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:49.216+0000 7f23e6a00700 1 --2- 172.21.15.18:0/4284419137 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f23e8109dd0 0x7f23e811a4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:16:49.213 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:49.216+0000 7f23e6a00700 1 -- 172.21.15.18:0/4284419137 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23d8005040 con 0x7f23e8109430 2024-04-12T21:16:49.213 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:49.216+0000 7f23dfe00700 1 --2- 172.21.15.18:0/4284419137 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f23e8109dd0 0x7f23e811a4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-12T21:16:49.213 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:49.216+0000 7f23e6a00700 1 --2- 172.21.15.18:0/4284419137 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f23e8109430 0x7f23e8119f90 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f23d8009180 tx=0x7f23d8009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:16:49.214 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:49.217+0000 7f23dea00700 1 -- 172.21.15.18:0/4284419137 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f23d800e040 con 0x7f23e8109430 2024-04-12T21:16:49.214 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:49.217+0000 7f23dea00700 1 -- 172.21.15.18:0/4284419137 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f23d800bbc0 con 0x7f23e8109430 2024-04-12T21:16:49.214 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:49.217+0000 7f23dea00700 1 -- 172.21.15.18:0/4284419137 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f23d801c630 con 0x7f23e8109430 2024-04-12T21:16:49.214 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:49.217+0000 7f23edc00700 1 -- 172.21.15.18:0/4284419137 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f23e8114380 con 0x7f23e8109430 2024-04-12T21:16:49.214 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:49.217+0000 7f23edc00700 1 -- 172.21.15.18:0/4284419137 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f23e8114940 con 0x7f23e8109430 2024-04-12T21:16:49.215 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:49.219+0000 7f23edc00700 1 -- 172.21.15.18:0/4284419137 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f23e8104b80 con 0x7f23e8109430 2024-04-12T21:16:49.216 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:49.220+0000 7f23dea00700 1 -- 172.21.15.18:0/4284419137 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7f23d800bd30 con 0x7f23e8109430 2024-04-12T21:16:49.217 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:49.221+0000 7f23dea00700 1 --2- 172.21.15.18:0/4284419137 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f23cc06ec00 0x7f23cc0710b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:16:49.217 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:49.222+0000 7f23dea00700 1 -- 172.21.15.18:0/4284419137 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 6757+0+0 (secure 0 0 0) 0x7f23d808fe10 con 0x7f23e8109430 2024-04-12T21:16:49.218 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:49.222+0000 7f23dfe00700 1 --2- 172.21.15.18:0/4284419137 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f23cc06ec00 0x7f23cc0710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:16:49.218 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:49.223+0000 7f23dfe00700 1 --2- 172.21.15.18:0/4284419137 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f23cc06ec00 0x7f23cc0710b0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f23d0009fd0 tx=0x7f23d000b040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:16:49.226 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:49.230+0000 7f23dea00700 1 -- 172.21.15.18:0/4284419137 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f23d805c5c0 con 0x7f23e8109430 2024-04-12T21:16:49.481 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:49.485+0000 7f23edc00700 1 -- 172.21.15.18:0/4284419137 --> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f23e8115650 con 0x7f23cc06ec00 2024-04-12T21:16:49.482 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:16:49 smithi018 ceph-mon[112590]: pgmap v510: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:16:49.484 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:49.488+0000 7f23dea00700 1 -- 172.21.15.18:0/4284419137 <== mgr.34102 v2:172.21.15.18:6800/3812891919 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f23e8115650 con 0x7f23cc06ec00 2024-04-12T21:16:49.488 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:49.491+0000 7f23edc00700 1 -- 172.21.15.18:0/4284419137 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f23cc06ec00 msgr2=0x7f23cc0710b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:16:49.488 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:49.491+0000 7f23edc00700 1 --2- 172.21.15.18:0/4284419137 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f23cc06ec00 0x7f23cc0710b0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f23d0009fd0 tx=0x7f23d000b040 comp rx=0 tx=0).stop 2024-04-12T21:16:49.488 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:49.491+0000 7f23edc00700 1 -- 172.21.15.18:0/4284419137 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f23e8109430 msgr2=0x7f23e8119f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:16:49.488 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:49.491+0000 7f23edc00700 1 --2- 172.21.15.18:0/4284419137 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f23e8109430 0x7f23e8119f90 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f23d8009180 tx=0x7f23d8009e30 comp rx=0 tx=0).stop 2024-04-12T21:16:49.488 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:49.491+0000 7f23edc00700 1 -- 172.21.15.18:0/4284419137 shutdown_connections 2024-04-12T21:16:49.488 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:49.491+0000 7f23edc00700 1 --2- 172.21.15.18:0/4284419137 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f23cc06ec00 0x7f23cc0710b0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:16:49.488 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:49.491+0000 7f23edc00700 1 --2- 172.21.15.18:0/4284419137 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f23e8109430 0x7f23e8119f90 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:16:49.488 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:49.491+0000 7f23edc00700 1 --2- 172.21.15.18:0/4284419137 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f23e8109dd0 0x7f23e811a4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:16:49.488 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:49.491+0000 7f23edc00700 1 -- 172.21.15.18:0/4284419137 >> 172.21.15.18:0/4284419137 conn(0x7f23e8072c00 msgr2=0x7f23e8074c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:16:49.488 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:49.492+0000 7f23edc00700 1 -- 172.21.15.18:0/4284419137 shutdown_connections 2024-04-12T21:16:49.488 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:49.492+0000 7f23edc00700 1 -- 172.21.15.18:0/4284419137 wait complete. 2024-04-12T21:16:49.505 INFO:teuthology.orchestra.run.smithi018.stdout:true 2024-04-12T21:16:49.528 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:16:49 smithi186 ceph-mon[86379]: pgmap v510: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:16:49.653 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:49.656+0000 7f066f000700 1 -- 172.21.15.18:0/3025372390 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f06680fdc40 msgr2=0x7f06680fe010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:16:49.653 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:49.656+0000 7f066f000700 1 --2- 172.21.15.18:0/3025372390 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f06680fdc40 0x7f06680fe010 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f065c004940 tx=0x7f065c009e30 comp rx=0 tx=0).stop 2024-04-12T21:16:49.653 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:49.657+0000 7f066f000700 1 -- 172.21.15.18:0/3025372390 shutdown_connections 2024-04-12T21:16:49.653 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:49.657+0000 7f066f000700 1 --2- 172.21.15.18:0/3025372390 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f06680fe5e0 0x7f0668106c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:16:49.654 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:49.657+0000 7f066f000700 1 --2- 172.21.15.18:0/3025372390 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f06680fdc40 0x7f06680fe010 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:16:49.654 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:49.657+0000 7f066f000700 1 -- 172.21.15.18:0/3025372390 >> 172.21.15.18:0/3025372390 conn(0x7f06680f7670 msgr2=0x7f06680f9a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:16:49.654 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:49.657+0000 7f066f000700 1 -- 172.21.15.18:0/3025372390 shutdown_connections 2024-04-12T21:16:49.654 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:49.657+0000 7f066f000700 1 -- 172.21.15.18:0/3025372390 wait complete. 2024-04-12T21:16:49.654 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:49.658+0000 7f066f000700 1 Processor -- start 2024-04-12T21:16:49.654 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:49.658+0000 7f066f000700 1 -- start start 2024-04-12T21:16:49.655 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:49.659+0000 7f066f000700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f06680fdc40 0x7f06680fba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:16:49.655 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:49.659+0000 7f066f000700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f06680fe5e0 0x7f06680fbfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:16:49.655 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:49.659+0000 7f066f000700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06680fc540 con 0x7f06680fdc40 2024-04-12T21:16:49.655 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:49.659+0000 7f066f000700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06680fc680 con 0x7f06680fe5e0 2024-04-12T21:16:49.657 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:49.659+0000 7f066dc00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f06680fdc40 0x7f06680fba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:16:49.657 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:49.659+0000 7f066dc00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f06680fdc40 0x7f06680fba70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:48086/0 (socket says 172.21.15.18:48086) 2024-04-12T21:16:49.657 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:49.659+0000 7f066dc00700 1 -- 172.21.15.18:0/893831168 learned_addr learned my addr 172.21.15.18:0/893831168 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:16:49.657 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:49.660+0000 7f066d200700 1 --2- 172.21.15.18:0/893831168 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f06680fe5e0 0x7f06680fbfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:16:49.657 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:49.660+0000 7f066dc00700 1 -- 172.21.15.18:0/893831168 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f06680fe5e0 msgr2=0x7f06680fbfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:16:49.657 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:49.660+0000 7f066dc00700 1 --2- 172.21.15.18:0/893831168 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f06680fe5e0 0x7f06680fbfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:16:49.658 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:49.660+0000 7f066dc00700 1 -- 172.21.15.18:0/893831168 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f065c005040 con 0x7f06680fdc40 2024-04-12T21:16:49.658 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:49.660+0000 7f066dc00700 1 --2- 172.21.15.18:0/893831168 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f06680fdc40 0x7f06680fba70 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f065c00b3b0 tx=0x7f065c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:16:49.658 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:49.661+0000 7f0665600700 1 -- 172.21.15.18:0/893831168 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f065c00e040 con 0x7f06680fdc40 2024-04-12T21:16:49.658 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:49.661+0000 7f0665600700 1 -- 172.21.15.18:0/893831168 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f065c013420 con 0x7f06680fdc40 2024-04-12T21:16:49.658 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:49.661+0000 7f0665600700 1 -- 172.21.15.18:0/893831168 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f065c01c630 con 0x7f06680fdc40 2024-04-12T21:16:49.658 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:49.661+0000 7f066f000700 1 -- 172.21.15.18:0/893831168 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f06680fc8d0 con 0x7f06680fdc40 2024-04-12T21:16:49.658 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:49.661+0000 7f066f000700 1 -- 172.21.15.18:0/893831168 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f066811a090 con 0x7f06680fdc40 2024-04-12T21:16:49.658 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:49.662+0000 7f066f000700 1 -- 172.21.15.18:0/893831168 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f06680668b0 con 0x7f06680fdc40 2024-04-12T21:16:49.659 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:49.663+0000 7f0665600700 1 -- 172.21.15.18:0/893831168 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7f065c00baf0 con 0x7f06680fdc40 2024-04-12T21:16:49.660 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:49.664+0000 7f0665600700 1 --2- 172.21.15.18:0/893831168 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f065006ec00 0x7f06500710b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:16:49.661 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:49.664+0000 7f0665600700 1 -- 172.21.15.18:0/893831168 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 6757+0+0 (secure 0 0 0) 0x7f065c08fb60 con 0x7f06680fdc40 2024-04-12T21:16:49.661 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:49.665+0000 7f066d200700 1 --2- 172.21.15.18:0/893831168 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f065006ec00 0x7f06500710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:16:49.661 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:49.665+0000 7f066d200700 1 --2- 172.21.15.18:0/893831168 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f065006ec00 0x7f06500710b0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f06680fd3b0 tx=0x7f0654009040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:16:49.665 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:49.669+0000 7f0665600700 1 -- 172.21.15.18:0/893831168 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f065c058920 con 0x7f06680fdc40 2024-04-12T21:16:49.921 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:49.925+0000 7f066f000700 1 -- 172.21.15.18:0/893831168 --> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f06680fd160 con 0x7f065006ec00 2024-04-12T21:16:49.924 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:49.928+0000 7f0665600700 1 -- 172.21.15.18:0/893831168 <== mgr.34102 v2:172.21.15.18:6800/3812891919 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f06680fd160 con 0x7f065006ec00 2024-04-12T21:16:49.927 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:49.931+0000 7f066f000700 1 -- 172.21.15.18:0/893831168 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f065006ec00 msgr2=0x7f06500710b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:16:49.927 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:49.931+0000 7f066f000700 1 --2- 172.21.15.18:0/893831168 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f065006ec00 0x7f06500710b0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f06680fd3b0 tx=0x7f0654009040 comp rx=0 tx=0).stop 2024-04-12T21:16:49.927 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:49.931+0000 7f066f000700 1 -- 172.21.15.18:0/893831168 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f06680fdc40 msgr2=0x7f06680fba70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:16:49.928 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:49.931+0000 7f066f000700 1 --2- 172.21.15.18:0/893831168 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f06680fdc40 0x7f06680fba70 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f065c00b3b0 tx=0x7f065c009e30 comp rx=0 tx=0).stop 2024-04-12T21:16:49.928 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:49.931+0000 7f066f000700 1 -- 172.21.15.18:0/893831168 shutdown_connections 2024-04-12T21:16:49.928 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:49.931+0000 7f066f000700 1 --2- 172.21.15.18:0/893831168 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f065006ec00 0x7f06500710b0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:16:49.928 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:49.931+0000 7f066f000700 1 --2- 172.21.15.18:0/893831168 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f06680fdc40 0x7f06680fba70 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:16:49.928 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:49.931+0000 7f066f000700 1 --2- 172.21.15.18:0/893831168 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f06680fe5e0 0x7f06680fbfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:16:49.928 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:49.931+0000 7f066f000700 1 -- 172.21.15.18:0/893831168 >> 172.21.15.18:0/893831168 conn(0x7f06680f7670 msgr2=0x7f06680f8c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:16:49.928 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:49.931+0000 7f066f000700 1 -- 172.21.15.18:0/893831168 shutdown_connections 2024-04-12T21:16:49.928 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:49.931+0000 7f066f000700 1 -- 172.21.15.18:0/893831168 wait complete. 2024-04-12T21:16:50.090 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:50.094+0000 7fa457600700 1 -- 172.21.15.18:0/3822267707 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa450109430 msgr2=0x7fa450109800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:16:50.091 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:50.094+0000 7fa457600700 1 --2- 172.21.15.18:0/3822267707 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa450109430 0x7fa450109800 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7fa444004940 tx=0x7fa444009e30 comp rx=0 tx=0).stop 2024-04-12T21:16:50.091 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:50.094+0000 7fa457600700 1 -- 172.21.15.18:0/3822267707 shutdown_connections 2024-04-12T21:16:50.091 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:50.094+0000 7fa457600700 1 --2- 172.21.15.18:0/3822267707 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa450109dd0 0x7fa450103440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:16:50.091 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:50.094+0000 7fa457600700 1 --2- 172.21.15.18:0/3822267707 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa450109430 0x7fa450109800 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:16:50.091 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:50.094+0000 7fa457600700 1 -- 172.21.15.18:0/3822267707 >> 172.21.15.18:0/3822267707 conn(0x7fa450072c00 msgr2=0x7fa450075010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:16:50.091 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:50.094+0000 7fa457600700 1 -- 172.21.15.18:0/3822267707 shutdown_connections 2024-04-12T21:16:50.091 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:50.094+0000 7fa457600700 1 -- 172.21.15.18:0/3822267707 wait complete. 2024-04-12T21:16:50.092 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:50.095+0000 7fa457600700 1 Processor -- start 2024-04-12T21:16:50.092 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:50.096+0000 7fa457600700 1 -- start start 2024-04-12T21:16:50.093 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:50.096+0000 7fa457600700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa450109430 0x7fa450119f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:16:50.093 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:50.096+0000 7fa457600700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa450109dd0 0x7fa45011a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:16:50.093 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:50.096+0000 7fa457600700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa45011aad0 con 0x7fa450109430 2024-04-12T21:16:50.093 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:50.096+0000 7fa457600700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa450114100 con 0x7fa450109dd0 2024-04-12T21:16:50.093 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:50.096+0000 7fa454c00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa450109430 0x7fa450119f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:16:50.093 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:50.096+0000 7fa454c00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa450109430 0x7fa450119f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:48096/0 (socket says 172.21.15.18:48096) 2024-04-12T21:16:50.093 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:50.097+0000 7fa454c00700 1 -- 172.21.15.18:0/3714661229 learned_addr learned my addr 172.21.15.18:0/3714661229 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:16:50.094 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:50.097+0000 7fa44fe00700 1 --2- 172.21.15.18:0/3714661229 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa450109dd0 0x7fa45011a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:16:50.094 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:50.097+0000 7fa454c00700 1 -- 172.21.15.18:0/3714661229 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa450109dd0 msgr2=0x7fa45011a480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:16:50.094 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:50.097+0000 7fa454c00700 1 --2- 172.21.15.18:0/3714661229 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa450109dd0 0x7fa45011a480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:16:50.094 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:50.097+0000 7fa454c00700 1 -- 172.21.15.18:0/3714661229 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa444005040 con 0x7fa450109430 2024-04-12T21:16:50.094 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:50.097+0000 7fa44fe00700 1 --2- 172.21.15.18:0/3714661229 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa450109dd0 0x7fa45011a480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-12T21:16:50.094 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:50.098+0000 7fa454c00700 1 --2- 172.21.15.18:0/3714661229 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa450109430 0x7fa450119f40 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7fa444009150 tx=0x7fa444009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:16:50.095 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:50.098+0000 7fa44cc00700 1 -- 172.21.15.18:0/3714661229 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa44400e040 con 0x7fa450109430 2024-04-12T21:16:50.095 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:50.098+0000 7fa44cc00700 1 -- 172.21.15.18:0/3714661229 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa444013420 con 0x7fa450109430 2024-04-12T21:16:50.095 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:50.098+0000 7fa457600700 1 -- 172.21.15.18:0/3714661229 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa450114380 con 0x7fa450109430 2024-04-12T21:16:50.095 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:50.098+0000 7fa44cc00700 1 -- 172.21.15.18:0/3714661229 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa44401c630 con 0x7fa450109430 2024-04-12T21:16:50.095 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:50.098+0000 7fa457600700 1 -- 172.21.15.18:0/3714661229 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa450114940 con 0x7fa450109430 2024-04-12T21:16:50.095 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:50.099+0000 7fa457600700 1 -- 172.21.15.18:0/3714661229 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa450104b80 con 0x7fa450109430 2024-04-12T21:16:50.097 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:50.101+0000 7fa44cc00700 1 -- 172.21.15.18:0/3714661229 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7fa444013590 con 0x7fa450109430 2024-04-12T21:16:50.098 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:50.102+0000 7fa44cc00700 1 --2- 172.21.15.18:0/3714661229 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fa43806ec00 0x7fa4380710b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:16:50.098 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:50.102+0000 7fa44cc00700 1 -- 172.21.15.18:0/3714661229 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 6757+0+0 (secure 0 0 0) 0x7fa44408fce0 con 0x7fa450109430 2024-04-12T21:16:50.098 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:50.102+0000 7fa44fe00700 1 --2- 172.21.15.18:0/3714661229 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fa43806ec00 0x7fa4380710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:16:50.099 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:50.103+0000 7fa44fe00700 1 --2- 172.21.15.18:0/3714661229 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fa43806ec00 0x7fa4380710b0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fa43c009fd0 tx=0x7fa43c00b040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:16:50.103 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:50.107+0000 7fa44cc00700 1 -- 172.21.15.18:0/3714661229 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7fa44405c490 con 0x7fa450109430 2024-04-12T21:16:50.352 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:50.356+0000 7fa457600700 1 -- 172.21.15.18:0/3714661229 --> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fa450115630 con 0x7fa43806ec00 2024-04-12T21:16:50.379 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:50.383+0000 7fa44cc00700 1 -- 172.21.15.18:0/3714661229 <== mgr.34102 v2:172.21.15.18:6800/3812891919 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7fa450115630 con 0x7fa43806ec00 2024-04-12T21:16:50.380 INFO:teuthology.orchestra.run.smithi018.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-12T21:16:50.380 INFO:teuthology.orchestra.run.smithi018.stdout:alertmanager.smithi018 smithi018 *:9093,9094 running (17m) 9m ago 31m 14.7M - 0.25.0 c8568f914cd2 c332090068e8 2024-04-12T21:16:50.380 INFO:teuthology.orchestra.run.smithi018.stdout:crash.smithi018 smithi018 running (13m) 9m ago 32m 6665k - 19.0.0-2879-g324baf8d 8717720dc5e5 b7f80d1f7982 2024-04-12T21:16:50.380 INFO:teuthology.orchestra.run.smithi018.stdout:crash.smithi186 smithi186 running (13m) 9m ago 30m 6660k - 19.0.0-2879-g324baf8d 8717720dc5e5 a4f55c81b623 2024-04-12T21:16:50.380 INFO:teuthology.orchestra.run.smithi018.stdout:grafana.smithi018 smithi018 *:3000 running (16m) 9m ago 31m 74.8M - 9.4.12 2bacad6d85d8 b08e3bf176dd 2024-04-12T21:16:50.380 INFO:teuthology.orchestra.run.smithi018.stdout:mds.cephfs.smithi018.acqbsc smithi018 running (9m) 9m ago 24m 11.1M - 19.0.0-2879-g324baf8d 8717720dc5e5 021f99ab24ae 2024-04-12T21:16:50.380 INFO:teuthology.orchestra.run.smithi018.stdout:mds.cephfs.smithi018.szjluo smithi018 running (9m) 9m ago 24m 10.9M - 19.0.0-2879-g324baf8d 8717720dc5e5 0003117479d4 2024-04-12T21:16:50.380 INFO:teuthology.orchestra.run.smithi018.stdout:mds.cephfs.smithi186.jdllyn smithi186 running (24m) 9m ago 24m 15.6M - 17.2.7-827-gd3b5060a 21de9b7369ad 2b2bc46bd29a 2024-04-12T21:16:50.380 INFO:teuthology.orchestra.run.smithi018.stdout:mds.cephfs.smithi186.wjnafs smithi186 running (24m) 9m ago 24m 15.4M - 17.2.7-827-gd3b5060a 21de9b7369ad 74b115d6d6fd 2024-04-12T21:16:50.380 INFO:teuthology.orchestra.run.smithi018.stdout:mgr.smithi018.xjzwga smithi018 *:8443,9283,8765 running (20m) 9m ago 33m 509M - 19.0.0-2879-g324baf8d 8717720dc5e5 56088305eab7 2024-04-12T21:16:50.381 INFO:teuthology.orchestra.run.smithi018.stdout:mgr.smithi186.kqcfxk smithi186 *:8443,9283,8765 running (18m) 9m ago 30m 452M - 19.0.0-2879-g324baf8d 8717720dc5e5 09cd766e5e0d 2024-04-12T21:16:50.381 INFO:teuthology.orchestra.run.smithi018.stdout:mon.smithi018 smithi018 running (15m) 9m ago 33m 57.0M 2048M 19.0.0-2879-g324baf8d 8717720dc5e5 df3e201dfed6 2024-04-12T21:16:50.381 INFO:teuthology.orchestra.run.smithi018.stdout:mon.smithi186 smithi186 running (14m) 9m ago 30m 37.7M 2048M 19.0.0-2879-g324baf8d 8717720dc5e5 c77498da9b2b 2024-04-12T21:16:50.381 INFO:teuthology.orchestra.run.smithi018.stdout:node-exporter.smithi018 smithi018 *:9100 running (18m) 9m ago 32m 7780k - 1.5.0 0da6a335fe13 326c3d9c6847 2024-04-12T21:16:50.381 INFO:teuthology.orchestra.run.smithi018.stdout:node-exporter.smithi186 smithi186 *:9100 running (18m) 9m ago 30m 7826k - 1.5.0 0da6a335fe13 ec2112baeecf 2024-04-12T21:16:50.381 INFO:teuthology.orchestra.run.smithi018.stdout:osd.0 smithi018 running (13m) 9m ago 29m 121M 1559M 19.0.0-2879-g324baf8d 8717720dc5e5 60b1273ca047 2024-04-12T21:16:50.381 INFO:teuthology.orchestra.run.smithi018.stdout:osd.1 smithi018 running (12m) 9m ago 28m 91.3M 1559M 19.0.0-2879-g324baf8d 8717720dc5e5 170ce1e5fb8d 2024-04-12T21:16:50.381 INFO:teuthology.orchestra.run.smithi018.stdout:osd.2 smithi018 running (12m) 9m ago 28m 90.3M 1559M 19.0.0-2879-g324baf8d 8717720dc5e5 c2cbc6aa2839 2024-04-12T21:16:50.381 INFO:teuthology.orchestra.run.smithi018.stdout:osd.3 smithi186 running (11m) 9m ago 27m 96.6M 2581M 19.0.0-2879-g324baf8d 8717720dc5e5 0138e8262e4b 2024-04-12T21:16:50.381 INFO:teuthology.orchestra.run.smithi018.stdout:osd.4 smithi186 running (10m) 9m ago 27m 87.2M 2581M 19.0.0-2879-g324baf8d 8717720dc5e5 b20cb63b25c9 2024-04-12T21:16:50.381 INFO:teuthology.orchestra.run.smithi018.stdout:osd.5 smithi186 running (10m) 9m ago 26m 11.1M 2581M 19.0.0-2879-g324baf8d 8717720dc5e5 2513bea2610e 2024-04-12T21:16:50.381 INFO:teuthology.orchestra.run.smithi018.stdout:prometheus.smithi018 smithi018 *:9095 running (17m) 9m ago 31m 60.6M - 2.43.0 a07b618ecd1d f58841127e21 2024-04-12T21:16:50.383 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:50.386+0000 7fa457600700 1 -- 172.21.15.18:0/3714661229 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fa43806ec00 msgr2=0x7fa4380710b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:16:50.383 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:50.386+0000 7fa457600700 1 --2- 172.21.15.18:0/3714661229 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fa43806ec00 0x7fa4380710b0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fa43c009fd0 tx=0x7fa43c00b040 comp rx=0 tx=0).stop 2024-04-12T21:16:50.383 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:50.386+0000 7fa457600700 1 -- 172.21.15.18:0/3714661229 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa450109430 msgr2=0x7fa450119f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:16:50.383 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:50.386+0000 7fa457600700 1 --2- 172.21.15.18:0/3714661229 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa450109430 0x7fa450119f40 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7fa444009150 tx=0x7fa444009e30 comp rx=0 tx=0).stop 2024-04-12T21:16:50.383 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:50.386+0000 7fa457600700 1 -- 172.21.15.18:0/3714661229 shutdown_connections 2024-04-12T21:16:50.383 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:50.386+0000 7fa457600700 1 --2- 172.21.15.18:0/3714661229 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fa43806ec00 0x7fa4380710b0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:16:50.383 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:50.386+0000 7fa457600700 1 --2- 172.21.15.18:0/3714661229 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa450109430 0x7fa450119f40 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:16:50.383 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:50.386+0000 7fa457600700 1 --2- 172.21.15.18:0/3714661229 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa450109dd0 0x7fa45011a480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:16:50.383 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:50.386+0000 7fa457600700 1 -- 172.21.15.18:0/3714661229 >> 172.21.15.18:0/3714661229 conn(0x7fa450072c00 msgr2=0x7fa450074c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:16:50.383 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:50.387+0000 7fa457600700 1 -- 172.21.15.18:0/3714661229 shutdown_connections 2024-04-12T21:16:50.383 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:50.387+0000 7fa457600700 1 -- 172.21.15.18:0/3714661229 wait complete. 2024-04-12T21:16:50.528 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:16:50 smithi186 ceph-mon[86379]: from='client.34920 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:16:50.547 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:50.551+0000 7f3aace00700 1 -- 172.21.15.18:0/2851855854 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3aa8101020 msgr2=0x7f3aa81013f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:16:50.547 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:50.551+0000 7f3aace00700 1 --2- 172.21.15.18:0/2851855854 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3aa8101020 0x7f3aa81013f0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f3a98004940 tx=0x7f3a98009e30 comp rx=0 tx=0).stop 2024-04-12T21:16:50.547 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:16:50 smithi018 ceph-mon[112590]: from='client.34920 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:16:50.548 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:50.551+0000 7f3aace00700 1 -- 172.21.15.18:0/2851855854 shutdown_connections 2024-04-12T21:16:50.548 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:50.551+0000 7f3aace00700 1 --2- 172.21.15.18:0/2851855854 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3aa81019c0 0x7f3aa80fb030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:16:50.548 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:50.551+0000 7f3aace00700 1 --2- 172.21.15.18:0/2851855854 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3aa8101020 0x7f3aa81013f0 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:16:50.549 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:50.551+0000 7f3aace00700 1 -- 172.21.15.18:0/2851855854 >> 172.21.15.18:0/2851855854 conn(0x7f3aa80f6bd0 msgr2=0x7f3aa80f8fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:16:50.549 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:50.551+0000 7f3aace00700 1 -- 172.21.15.18:0/2851855854 shutdown_connections 2024-04-12T21:16:50.549 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:50.551+0000 7f3aace00700 1 -- 172.21.15.18:0/2851855854 wait complete. 2024-04-12T21:16:50.549 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:50.552+0000 7f3aace00700 1 Processor -- start 2024-04-12T21:16:50.549 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:50.552+0000 7f3aace00700 1 -- start start 2024-04-12T21:16:50.551 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:50.553+0000 7f3aace00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3aa8101020 0x7f3aa8101e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:16:50.551 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:50.553+0000 7f3aace00700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3aa81019c0 0x7f3aa81023c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:16:50.551 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:50.553+0000 7f3aace00700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3aa8103dd0 con 0x7f3aa8101020 2024-04-12T21:16:50.551 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:50.553+0000 7f3aace00700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3aa8102900 con 0x7f3aa81019c0 2024-04-12T21:16:50.551 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:50.553+0000 7f3aa5e00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3aa8101020 0x7f3aa8101e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:16:50.551 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:50.553+0000 7f3aa5e00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3aa8101020 0x7f3aa8101e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:48106/0 (socket says 172.21.15.18:48106) 2024-04-12T21:16:50.551 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:50.553+0000 7f3aa5e00700 1 -- 172.21.15.18:0/1797416180 learned_addr learned my addr 172.21.15.18:0/1797416180 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:16:50.551 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:50.553+0000 7f3a9fe00700 1 --2- 172.21.15.18:0/1797416180 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3aa81019c0 0x7f3aa81023c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:16:50.551 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:50.554+0000 7f3aa5e00700 1 -- 172.21.15.18:0/1797416180 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3aa81019c0 msgr2=0x7f3aa81023c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:16:50.551 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:50.554+0000 7f3aa5e00700 1 --2- 172.21.15.18:0/1797416180 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3aa81019c0 0x7f3aa81023c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:16:50.552 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:50.554+0000 7f3aa5e00700 1 -- 172.21.15.18:0/1797416180 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a98005040 con 0x7f3aa8101020 2024-04-12T21:16:50.552 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:50.554+0000 7f3aa5e00700 1 --2- 172.21.15.18:0/1797416180 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3aa8101020 0x7f3aa8101e80 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f3a98009150 tx=0x7f3a98009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:16:50.552 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:50.554+0000 7f3a9e000700 1 -- 172.21.15.18:0/1797416180 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3a98021070 con 0x7f3aa8101020 2024-04-12T21:16:50.552 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:50.554+0000 7f3a9e000700 1 -- 172.21.15.18:0/1797416180 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f3a98013c00 con 0x7f3aa8101020 2024-04-12T21:16:50.552 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:50.554+0000 7f3aace00700 1 -- 172.21.15.18:0/1797416180 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3aa8102b80 con 0x7f3aa8101020 2024-04-12T21:16:50.552 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:50.554+0000 7f3a9e000700 1 -- 172.21.15.18:0/1797416180 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3a9800e070 con 0x7f3aa8101020 2024-04-12T21:16:50.552 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:50.555+0000 7f3aace00700 1 -- 172.21.15.18:0/1797416180 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3aa8119fb0 con 0x7f3aa8101020 2024-04-12T21:16:50.552 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:50.556+0000 7f3aace00700 1 -- 172.21.15.18:0/1797416180 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3aa8100340 con 0x7f3aa8101020 2024-04-12T21:16:50.553 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:50.557+0000 7f3a9e000700 1 -- 172.21.15.18:0/1797416180 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7f3a9801c400 con 0x7f3aa8101020 2024-04-12T21:16:50.554 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:50.558+0000 7f3a9e000700 1 --2- 172.21.15.18:0/1797416180 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f3a8c06ec00 0x7f3a8c0710b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:16:50.554 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:50.558+0000 7f3a9e000700 1 -- 172.21.15.18:0/1797416180 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 6757+0+0 (secure 0 0 0) 0x7f3a9808ff20 con 0x7f3aa8101020 2024-04-12T21:16:50.554 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:50.558+0000 7f3a9fe00700 1 --2- 172.21.15.18:0/1797416180 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f3a8c06ec00 0x7f3a8c0710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:16:50.555 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:50.559+0000 7f3a9fe00700 1 --2- 172.21.15.18:0/1797416180 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f3a8c06ec00 0x7f3a8c0710b0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f3a90004710 tx=0x7f3a9000c040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:16:50.559 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:50.563+0000 7f3a9e000700 1 -- 172.21.15.18:0/1797416180 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f3a98058ba0 con 0x7f3aa8101020 2024-04-12T21:16:50.885 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:50.889+0000 7f3aace00700 1 -- 172.21.15.18:0/1797416180 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f3aa8107d60 con 0x7f3aa8101020 2024-04-12T21:16:50.886 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:50.890+0000 7f3a9e000700 1 -- 172.21.15.18:0/1797416180 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+837 (secure 0 0 0) 0x7f3a9805c260 con 0x7f3aa8101020 2024-04-12T21:16:50.887 INFO:teuthology.orchestra.run.smithi018.stdout:{ 2024-04-12T21:16:50.887 INFO:teuthology.orchestra.run.smithi018.stdout: "mon": { 2024-04-12T21:16:50.887 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 19.0.0-2879-g324baf8d (324baf8ddb91dea46cac50cb535c306115791355) squid (dev)": 2 2024-04-12T21:16:50.887 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-12T21:16:50.887 INFO:teuthology.orchestra.run.smithi018.stdout: "mgr": { 2024-04-12T21:16:50.887 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 19.0.0-2879-g324baf8d (324baf8ddb91dea46cac50cb535c306115791355) squid (dev)": 2 2024-04-12T21:16:50.888 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-12T21:16:50.888 INFO:teuthology.orchestra.run.smithi018.stdout: "osd": { 2024-04-12T21:16:50.888 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 19.0.0-2879-g324baf8d (324baf8ddb91dea46cac50cb535c306115791355) squid (dev)": 6 2024-04-12T21:16:50.888 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-12T21:16:50.888 INFO:teuthology.orchestra.run.smithi018.stdout: "mds": { 2024-04-12T21:16:50.888 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 17.2.7-827-gd3b5060a (d3b5060a17449edfae9385cdf792afec109fca5d) quincy (stable)": 2, 2024-04-12T21:16:50.888 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 19.0.0-2879-g324baf8d (324baf8ddb91dea46cac50cb535c306115791355) squid (dev)": 2 2024-04-12T21:16:50.888 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-12T21:16:50.888 INFO:teuthology.orchestra.run.smithi018.stdout: "overall": { 2024-04-12T21:16:50.888 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 17.2.7-827-gd3b5060a (d3b5060a17449edfae9385cdf792afec109fca5d) quincy (stable)": 2, 2024-04-12T21:16:50.888 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 19.0.0-2879-g324baf8d (324baf8ddb91dea46cac50cb535c306115791355) squid (dev)": 12 2024-04-12T21:16:50.888 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-04-12T21:16:50.888 INFO:teuthology.orchestra.run.smithi018.stdout:} 2024-04-12T21:16:50.889 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:50.893+0000 7f3aace00700 1 -- 172.21.15.18:0/1797416180 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f3a8c06ec00 msgr2=0x7f3a8c0710b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:16:50.889 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:50.893+0000 7f3aace00700 1 --2- 172.21.15.18:0/1797416180 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f3a8c06ec00 0x7f3a8c0710b0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f3a90004710 tx=0x7f3a9000c040 comp rx=0 tx=0).stop 2024-04-12T21:16:50.889 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:50.893+0000 7f3aace00700 1 -- 172.21.15.18:0/1797416180 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3aa8101020 msgr2=0x7f3aa8101e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:16:50.889 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:50.893+0000 7f3aace00700 1 --2- 172.21.15.18:0/1797416180 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3aa8101020 0x7f3aa8101e80 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f3a98009150 tx=0x7f3a98009e30 comp rx=0 tx=0).stop 2024-04-12T21:16:50.889 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:50.893+0000 7f3aace00700 1 -- 172.21.15.18:0/1797416180 shutdown_connections 2024-04-12T21:16:50.890 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:50.893+0000 7f3aace00700 1 --2- 172.21.15.18:0/1797416180 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f3a8c06ec00 0x7f3a8c0710b0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:16:50.890 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:50.893+0000 7f3aace00700 1 --2- 172.21.15.18:0/1797416180 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3aa8101020 0x7f3aa8101e80 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:16:50.890 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:50.893+0000 7f3aace00700 1 --2- 172.21.15.18:0/1797416180 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3aa81019c0 0x7f3aa81023c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:16:50.890 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:50.893+0000 7f3aace00700 1 -- 172.21.15.18:0/1797416180 >> 172.21.15.18:0/1797416180 conn(0x7f3aa80f6bd0 msgr2=0x7f3aa8103fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:16:50.890 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:50.893+0000 7f3aace00700 1 -- 172.21.15.18:0/1797416180 shutdown_connections 2024-04-12T21:16:50.890 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:50.893+0000 7f3aace00700 1 -- 172.21.15.18:0/1797416180 wait complete. 2024-04-12T21:16:51.035 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:51.039+0000 7f313be00700 1 -- 172.21.15.18:0/1579625805 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f313c072e00 msgr2=0x7f313c10d630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:16:51.036 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:51.039+0000 7f313be00700 1 --2- 172.21.15.18:0/1579625805 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f313c072e00 0x7f313c10d630 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f3128004940 tx=0x7f3128009e30 comp rx=0 tx=0).stop 2024-04-12T21:16:51.036 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:51.039+0000 7f313be00700 1 -- 172.21.15.18:0/1579625805 shutdown_connections 2024-04-12T21:16:51.036 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:51.039+0000 7f313be00700 1 --2- 172.21.15.18:0/1579625805 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f313c072e00 0x7f313c10d630 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:16:51.036 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:51.039+0000 7f313be00700 1 --2- 172.21.15.18:0/1579625805 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f313c0724f0 0x7f313c0728c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:16:51.036 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:51.039+0000 7f313be00700 1 -- 172.21.15.18:0/1579625805 >> 172.21.15.18:0/1579625805 conn(0x7f313c0fab10 msgr2=0x7f313c0fcef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:16:51.036 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:51.039+0000 7f313be00700 1 -- 172.21.15.18:0/1579625805 shutdown_connections 2024-04-12T21:16:51.036 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:51.039+0000 7f313be00700 1 -- 172.21.15.18:0/1579625805 wait complete. 2024-04-12T21:16:51.037 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:51.040+0000 7f313be00700 1 Processor -- start 2024-04-12T21:16:51.037 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:51.040+0000 7f313be00700 1 -- start start 2024-04-12T21:16:51.037 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:51.041+0000 7f313be00700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f313c0724f0 0x7f313c119f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:16:51.037 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:51.041+0000 7f313be00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f313c072e00 0x7f313c11a4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:16:51.037 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:51.041+0000 7f313be00700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f313c11ab10 con 0x7f313c072e00 2024-04-12T21:16:51.037 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:51.041+0000 7f313be00700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f313c113ee0 con 0x7f313c0724f0 2024-04-12T21:16:51.037 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:51.041+0000 7f313a000700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f313c072e00 0x7f313c11a4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:16:51.037 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:51.041+0000 7f313a000700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f313c072e00 0x7f313c11a4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:42792/0 (socket says 172.21.15.18:42792) 2024-04-12T21:16:51.037 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:51.041+0000 7f313a000700 1 -- 172.21.15.18:0/1146537 learned_addr learned my addr 172.21.15.18:0/1146537 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:16:51.038 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:51.041+0000 7f313a000700 1 -- 172.21.15.18:0/1146537 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f313c0724f0 msgr2=0x7f313c119f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:16:51.038 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:51.041+0000 7f313a000700 1 --2- 172.21.15.18:0/1146537 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f313c0724f0 0x7f313c119f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:16:51.039 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:51.041+0000 7f313a000700 1 -- 172.21.15.18:0/1146537 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3128005040 con 0x7f313c072e00 2024-04-12T21:16:51.039 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:51.042+0000 7f313a000700 1 --2- 172.21.15.18:0/1146537 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f313c072e00 0x7f313c11a4c0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f3128009180 tx=0x7f3128009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:16:51.039 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:51.042+0000 7f3132a00700 1 -- 172.21.15.18:0/1146537 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f312800e040 con 0x7f313c072e00 2024-04-12T21:16:51.039 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:51.042+0000 7f3132a00700 1 -- 172.21.15.18:0/1146537 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f312800bbc0 con 0x7f313c072e00 2024-04-12T21:16:51.039 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:51.042+0000 7f313be00700 1 -- 172.21.15.18:0/1146537 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f313c114160 con 0x7f313c072e00 2024-04-12T21:16:51.039 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:51.042+0000 7f3132a00700 1 -- 172.21.15.18:0/1146537 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f312801c630 con 0x7f313c072e00 2024-04-12T21:16:51.039 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:51.042+0000 7f313be00700 1 -- 172.21.15.18:0/1146537 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f313c114720 con 0x7f313c072e00 2024-04-12T21:16:51.039 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:51.043+0000 7f313be00700 1 -- 172.21.15.18:0/1146537 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f313c073be0 con 0x7f313c072e00 2024-04-12T21:16:51.040 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:51.044+0000 7f3132a00700 1 -- 172.21.15.18:0/1146537 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7f312800bd30 con 0x7f313c072e00 2024-04-12T21:16:51.041 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:51.045+0000 7f3132a00700 1 --2- 172.21.15.18:0/1146537 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f312006ec00 0x7f31200710b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:16:51.041 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:51.045+0000 7f3132a00700 1 -- 172.21.15.18:0/1146537 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 6757+0+0 (secure 0 0 0) 0x7f312808fcd0 con 0x7f313c072e00 2024-04-12T21:16:51.042 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:51.046+0000 7f313aa00700 1 --2- 172.21.15.18:0/1146537 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f312006ec00 0x7f31200710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:16:51.042 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:51.046+0000 7f313aa00700 1 --2- 172.21.15.18:0/1146537 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f312006ec00 0x7f31200710b0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f312c004ad0 tx=0x7f312c007d40 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:16:51.046 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:51.050+0000 7f3132a00700 1 -- 172.21.15.18:0/1146537 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f312805c5c0 con 0x7f313c072e00 2024-04-12T21:16:51.321 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:51.324+0000 7f313be00700 1 -- 172.21.15.18:0/1146537 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f313c04bed0 con 0x7f313c072e00 2024-04-12T21:16:51.321 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:51.325+0000 7f3132a00700 1 -- 172.21.15.18:0/1146537 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 29 v29) v1 ==== 76+0+1872 (secure 0 0 0) 0x7f312805c150 con 0x7f313c072e00 2024-04-12T21:16:51.322 INFO:teuthology.orchestra.run.smithi018.stdout:e29 2024-04-12T21:16:51.322 INFO:teuthology.orchestra.run.smithi018.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-12T21:16:51.322 INFO:teuthology.orchestra.run.smithi018.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-12T21:16:51.322 INFO:teuthology.orchestra.run.smithi018.stdout:legacy client fscid: 1 2024-04-12T21:16:51.322 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-12T21:16:51.322 INFO:teuthology.orchestra.run.smithi018.stdout:Filesystem 'cephfs' (1) 2024-04-12T21:16:51.322 INFO:teuthology.orchestra.run.smithi018.stdout:fs_name cephfs 2024-04-12T21:16:51.322 INFO:teuthology.orchestra.run.smithi018.stdout:epoch 24 2024-04-12T21:16:51.322 INFO:teuthology.orchestra.run.smithi018.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-04-12T21:16:51.322 INFO:teuthology.orchestra.run.smithi018.stdout:created 2024-04-12T20:51:57.362850+0000 2024-04-12T21:16:51.322 INFO:teuthology.orchestra.run.smithi018.stdout:modified 2024-04-12T21:06:59.717190+0000 2024-04-12T21:16:51.323 INFO:teuthology.orchestra.run.smithi018.stdout:tableserver 0 2024-04-12T21:16:51.323 INFO:teuthology.orchestra.run.smithi018.stdout:root 0 2024-04-12T21:16:51.323 INFO:teuthology.orchestra.run.smithi018.stdout:session_timeout 60 2024-04-12T21:16:51.323 INFO:teuthology.orchestra.run.smithi018.stdout:session_autoclose 300 2024-04-12T21:16:51.323 INFO:teuthology.orchestra.run.smithi018.stdout:max_file_size 1099511627776 2024-04-12T21:16:51.323 INFO:teuthology.orchestra.run.smithi018.stdout:max_xattr_size 65536 2024-04-12T21:16:51.323 INFO:teuthology.orchestra.run.smithi018.stdout:required_client_features {} 2024-04-12T21:16:51.323 INFO:teuthology.orchestra.run.smithi018.stdout:last_failure 0 2024-04-12T21:16:51.323 INFO:teuthology.orchestra.run.smithi018.stdout:last_failure_osd_epoch 100 2024-04-12T21:16:51.323 INFO:teuthology.orchestra.run.smithi018.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-12T21:16:51.323 INFO:teuthology.orchestra.run.smithi018.stdout:max_mds 2 2024-04-12T21:16:51.323 INFO:teuthology.orchestra.run.smithi018.stdout:in 0,1 2024-04-12T21:16:51.323 INFO:teuthology.orchestra.run.smithi018.stdout:up {} 2024-04-12T21:16:51.323 INFO:teuthology.orchestra.run.smithi018.stdout:failed 0,1 2024-04-12T21:16:51.323 INFO:teuthology.orchestra.run.smithi018.stdout:damaged 2024-04-12T21:16:51.324 INFO:teuthology.orchestra.run.smithi018.stdout:stopped 2024-04-12T21:16:51.324 INFO:teuthology.orchestra.run.smithi018.stdout:data_pools [3] 2024-04-12T21:16:51.324 INFO:teuthology.orchestra.run.smithi018.stdout:metadata_pool 2 2024-04-12T21:16:51.324 INFO:teuthology.orchestra.run.smithi018.stdout:inline_data disabled 2024-04-12T21:16:51.324 INFO:teuthology.orchestra.run.smithi018.stdout:balancer 2024-04-12T21:16:51.324 INFO:teuthology.orchestra.run.smithi018.stdout:bal_rank_mask -1 2024-04-12T21:16:51.324 INFO:teuthology.orchestra.run.smithi018.stdout:standby_count_wanted 1 2024-04-12T21:16:51.324 INFO:teuthology.orchestra.run.smithi018.stdout:qdb_cluster leader: 0 members: 2024-04-12T21:16:51.324 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-12T21:16:51.324 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-12T21:16:51.324 INFO:teuthology.orchestra.run.smithi018.stdout:Standby daemons: 2024-04-12T21:16:51.324 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-12T21:16:51.324 INFO:teuthology.orchestra.run.smithi018.stdout:[mds.cephfs.smithi186.wjnafs{-1:24277} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.186:6826/438347207,v1:172.21.15.186:6827/438347207] compat {c=[1],r=[1],i=[7ff]}] 2024-04-12T21:16:51.324 INFO:teuthology.orchestra.run.smithi018.stdout:[mds.cephfs.smithi186.jdllyn{-1:24281} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.186:6824/1464359952,v1:172.21.15.186:6825/1464359952] compat {c=[1],r=[1],i=[7ff]}] 2024-04-12T21:16:51.324 INFO:teuthology.orchestra.run.smithi018.stdout:[mds.cephfs.smithi018.acqbsc{-1:34436} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.18:6828/755179764,v1:172.21.15.18:6829/755179764] compat {c=[1],r=[1],i=[1fff]}] 2024-04-12T21:16:51.324 INFO:teuthology.orchestra.run.smithi018.stdout:[mds.cephfs.smithi018.szjluo{-1:34468} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.18:6826/2407992653,v1:172.21.15.18:6827/2407992653] compat {c=[1],r=[1],i=[1fff]}] 2024-04-12T21:16:51.326 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:51.329+0000 7f313be00700 1 -- 172.21.15.18:0/1146537 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f312006ec00 msgr2=0x7f31200710b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:16:51.326 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:51.329+0000 7f313be00700 1 --2- 172.21.15.18:0/1146537 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f312006ec00 0x7f31200710b0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f312c004ad0 tx=0x7f312c007d40 comp rx=0 tx=0).stop 2024-04-12T21:16:51.326 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:51.330+0000 7f313be00700 1 -- 172.21.15.18:0/1146537 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f313c072e00 msgr2=0x7f313c11a4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:16:51.327 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:51.330+0000 7f313be00700 1 --2- 172.21.15.18:0/1146537 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f313c072e00 0x7f313c11a4c0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f3128009180 tx=0x7f3128009e30 comp rx=0 tx=0).stop 2024-04-12T21:16:51.327 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:51.330+0000 7f313be00700 1 -- 172.21.15.18:0/1146537 shutdown_connections 2024-04-12T21:16:51.327 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:51.330+0000 7f313be00700 1 --2- 172.21.15.18:0/1146537 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f312006ec00 0x7f31200710b0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:16:51.327 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:51.330+0000 7f313be00700 1 --2- 172.21.15.18:0/1146537 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f313c0724f0 0x7f313c119f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:16:51.327 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:51.330+0000 7f313be00700 1 --2- 172.21.15.18:0/1146537 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f313c072e00 0x7f313c11a4c0 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:16:51.327 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:51.330+0000 7f313be00700 1 -- 172.21.15.18:0/1146537 >> 172.21.15.18:0/1146537 conn(0x7f313c0fab10 msgr2=0x7f313c0faee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:16:51.327 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:51.330+0000 7f313be00700 1 -- 172.21.15.18:0/1146537 shutdown_connections 2024-04-12T21:16:51.327 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:51.330+0000 7f313be00700 1 -- 172.21.15.18:0/1146537 wait complete. 2024-04-12T21:16:51.327 INFO:teuthology.orchestra.run.smithi018.stderr:dumped fsmap epoch 29 2024-04-12T21:16:51.480 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:51.484+0000 7fda62800700 1 -- 172.21.15.18:0/2231920138 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fda5c0fde90 msgr2=0x7fda5c0fe260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:16:51.480 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:51.484+0000 7fda62800700 1 --2- 172.21.15.18:0/2231920138 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fda5c0fde90 0x7fda5c0fe260 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7fda4c004940 tx=0x7fda4c009e30 comp rx=0 tx=0).stop 2024-04-12T21:16:51.480 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:16:51 smithi018 ceph-mon[112590]: from='client.34924 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:16:51.481 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:16:51 smithi018 ceph-mon[112590]: pgmap v511: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:16:51.481 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:16:51 smithi018 ceph-mon[112590]: from='client.34928 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:16:51.481 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:16:51 smithi018 ceph-mon[112590]: from='client.? 172.21.15.18:0/1797416180' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T21:16:51.481 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:51.484+0000 7fda62800700 1 -- 172.21.15.18:0/2231920138 shutdown_connections 2024-04-12T21:16:51.481 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:51.484+0000 7fda62800700 1 --2- 172.21.15.18:0/2231920138 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fda5c0fe830 0x7fda5c106f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:16:51.481 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:51.484+0000 7fda62800700 1 --2- 172.21.15.18:0/2231920138 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fda5c0fde90 0x7fda5c0fe260 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:16:51.481 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:51.484+0000 7fda62800700 1 -- 172.21.15.18:0/2231920138 >> 172.21.15.18:0/2231920138 conn(0x7fda5c072bc0 msgr2=0x7fda5c072fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:16:51.481 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:51.484+0000 7fda62800700 1 -- 172.21.15.18:0/2231920138 shutdown_connections 2024-04-12T21:16:51.482 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:51.484+0000 7fda62800700 1 -- 172.21.15.18:0/2231920138 wait complete. 2024-04-12T21:16:51.482 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:51.486+0000 7fda62800700 1 Processor -- start 2024-04-12T21:16:51.482 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:51.486+0000 7fda62800700 1 -- start start 2024-04-12T21:16:51.483 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:51.487+0000 7fda62800700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fda5c0fde90 0x7fda5c0fbd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:16:51.483 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:51.487+0000 7fda62800700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fda5c0fe830 0x7fda5c0fc290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:16:51.483 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:51.487+0000 7fda62800700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda5c0fc820 con 0x7fda5c0fde90 2024-04-12T21:16:51.483 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:51.487+0000 7fda62800700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda5c0fc960 con 0x7fda5c0fe830 2024-04-12T21:16:51.483 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:51.487+0000 7fda5b400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fda5c0fde90 0x7fda5c0fbd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:16:51.483 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:51.487+0000 7fda5b400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fda5c0fde90 0x7fda5c0fbd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:42804/0 (socket says 172.21.15.18:42804) 2024-04-12T21:16:51.483 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:51.487+0000 7fda5b400700 1 -- 172.21.15.18:0/3073091626 learned_addr learned my addr 172.21.15.18:0/3073091626 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:16:51.484 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:51.487+0000 7fda5b400700 1 -- 172.21.15.18:0/3073091626 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fda5c0fe830 msgr2=0x7fda5c0fc290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:16:51.484 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:51.487+0000 7fda5b400700 1 --2- 172.21.15.18:0/3073091626 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fda5c0fe830 0x7fda5c0fc290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:16:51.484 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:51.487+0000 7fda5b400700 1 -- 172.21.15.18:0/3073091626 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda4c005040 con 0x7fda5c0fde90 2024-04-12T21:16:51.484 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:51.488+0000 7fda5b400700 1 --2- 172.21.15.18:0/3073091626 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fda5c0fde90 0x7fda5c0fbd50 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7fda4c000c00 tx=0x7fda4c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:16:51.484 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:51.488+0000 7fda53400700 1 -- 172.21.15.18:0/3073091626 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fda4c00e040 con 0x7fda5c0fde90 2024-04-12T21:16:51.484 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:51.488+0000 7fda53400700 1 -- 172.21.15.18:0/3073091626 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fda4c013420 con 0x7fda5c0fde90 2024-04-12T21:16:51.484 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:51.488+0000 7fda53400700 1 -- 172.21.15.18:0/3073091626 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fda4c01c610 con 0x7fda5c0fde90 2024-04-12T21:16:51.485 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:51.488+0000 7fda62800700 1 -- 172.21.15.18:0/3073091626 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fda5c0fcbb0 con 0x7fda5c0fde90 2024-04-12T21:16:51.485 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:51.488+0000 7fda62800700 1 -- 172.21.15.18:0/3073091626 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fda5c11a280 con 0x7fda5c0fde90 2024-04-12T21:16:51.485 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:51.489+0000 7fda62800700 1 -- 172.21.15.18:0/3073091626 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fda5c065e30 con 0x7fda5c0fde90 2024-04-12T21:16:51.486 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:51.491+0000 7fda53400700 1 -- 172.21.15.18:0/3073091626 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7fda4c013590 con 0x7fda5c0fde90 2024-04-12T21:16:51.488 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:51.492+0000 7fda53400700 1 --2- 172.21.15.18:0/3073091626 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fda4006ec10 0x7fda400710c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:16:51.488 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:51.492+0000 7fda53400700 1 -- 172.21.15.18:0/3073091626 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 6757+0+0 (secure 0 0 0) 0x7fda4c08fbd0 con 0x7fda5c0fde90 2024-04-12T21:16:51.488 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:51.492+0000 7fda5aa00700 1 --2- 172.21.15.18:0/3073091626 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fda4006ec10 0x7fda400710c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:16:51.489 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:51.493+0000 7fda5aa00700 1 --2- 172.21.15.18:0/3073091626 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fda4006ec10 0x7fda400710c0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fda5c0fd690 tx=0x7fda48007d40 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:16:51.492 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:51.496+0000 7fda53400700 1 -- 172.21.15.18:0/3073091626 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7fda4c058850 con 0x7fda5c0fde90 2024-04-12T21:16:51.529 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:16:51 smithi186 ceph-mon[86379]: from='client.34924 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:16:51.529 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:16:51 smithi186 ceph-mon[86379]: pgmap v511: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:16:51.529 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:16:51 smithi186 ceph-mon[86379]: from='client.34928 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:16:51.529 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:16:51 smithi186 ceph-mon[86379]: from='client.? 172.21.15.18:0/1797416180' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T21:16:51.743 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:51.746+0000 7fda62800700 1 -- 172.21.15.18:0/3073091626 --> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fda5c04bf40 con 0x7fda4006ec10 2024-04-12T21:16:51.745 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:51.749+0000 7fda53400700 1 -- 172.21.15.18:0/3073091626 <== mgr.34102 v2:172.21.15.18:6800/3812891919 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7fda5c04bf40 con 0x7fda4006ec10 2024-04-12T21:16:51.745 INFO:teuthology.orchestra.run.smithi018.stdout:{ 2024-04-12T21:16:51.746 INFO:teuthology.orchestra.run.smithi018.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:f4452def3be4d350f4a8e5feaf7ebb1511be9298fe68478ba7a2d6fbd7d69a63", 2024-04-12T21:16:51.746 INFO:teuthology.orchestra.run.smithi018.stdout: "in_progress": true, 2024-04-12T21:16:51.746 INFO:teuthology.orchestra.run.smithi018.stdout: "which": "Upgrading all daemon types on all hosts", 2024-04-12T21:16:51.746 INFO:teuthology.orchestra.run.smithi018.stdout: "services_complete": [ 2024-04-12T21:16:51.746 INFO:teuthology.orchestra.run.smithi018.stdout: "crash", 2024-04-12T21:16:51.746 INFO:teuthology.orchestra.run.smithi018.stdout: "osd", 2024-04-12T21:16:51.746 INFO:teuthology.orchestra.run.smithi018.stdout: "mon", 2024-04-12T21:16:51.746 INFO:teuthology.orchestra.run.smithi018.stdout: "mgr" 2024-04-12T21:16:51.746 INFO:teuthology.orchestra.run.smithi018.stdout: ], 2024-04-12T21:16:51.746 INFO:teuthology.orchestra.run.smithi018.stdout: "progress": "14/21 daemons upgraded", 2024-04-12T21:16:51.746 INFO:teuthology.orchestra.run.smithi018.stdout: "message": "Currently upgrading mds daemons", 2024-04-12T21:16:51.746 INFO:teuthology.orchestra.run.smithi018.stdout: "is_paused": false 2024-04-12T21:16:51.746 INFO:teuthology.orchestra.run.smithi018.stdout:} 2024-04-12T21:16:51.748 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:51.752+0000 7fda62800700 1 -- 172.21.15.18:0/3073091626 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fda4006ec10 msgr2=0x7fda400710c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:16:51.748 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:51.752+0000 7fda62800700 1 --2- 172.21.15.18:0/3073091626 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fda4006ec10 0x7fda400710c0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fda5c0fd690 tx=0x7fda48007d40 comp rx=0 tx=0).stop 2024-04-12T21:16:51.749 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:51.752+0000 7fda62800700 1 -- 172.21.15.18:0/3073091626 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fda5c0fde90 msgr2=0x7fda5c0fbd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:16:51.749 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:51.752+0000 7fda62800700 1 --2- 172.21.15.18:0/3073091626 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fda5c0fde90 0x7fda5c0fbd50 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7fda4c000c00 tx=0x7fda4c009e30 comp rx=0 tx=0).stop 2024-04-12T21:16:51.749 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:51.752+0000 7fda62800700 1 -- 172.21.15.18:0/3073091626 shutdown_connections 2024-04-12T21:16:51.749 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:51.752+0000 7fda62800700 1 --2- 172.21.15.18:0/3073091626 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fda4006ec10 0x7fda400710c0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:16:51.749 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:51.752+0000 7fda62800700 1 --2- 172.21.15.18:0/3073091626 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fda5c0fde90 0x7fda5c0fbd50 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:16:51.749 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:51.752+0000 7fda62800700 1 --2- 172.21.15.18:0/3073091626 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fda5c0fe830 0x7fda5c0fc290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:16:51.749 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:51.752+0000 7fda62800700 1 -- 172.21.15.18:0/3073091626 >> 172.21.15.18:0/3073091626 conn(0x7fda5c072bc0 msgr2=0x7fda5c0ff3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:16:51.749 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:51.752+0000 7fda62800700 1 -- 172.21.15.18:0/3073091626 shutdown_connections 2024-04-12T21:16:51.749 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:51.752+0000 7fda62800700 1 -- 172.21.15.18:0/3073091626 wait complete. 2024-04-12T21:16:51.910 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:51.914+0000 7f2183200700 1 -- 172.21.15.18:0/1873476473 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f217c109dd0 msgr2=0x7f217c103440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:16:51.911 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:51.914+0000 7f2183200700 1 --2- 172.21.15.18:0/1873476473 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f217c109dd0 0x7f217c103440 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f2168004940 tx=0x7f2168009e30 comp rx=0 tx=0).stop 2024-04-12T21:16:51.911 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:51.914+0000 7f2183200700 1 -- 172.21.15.18:0/1873476473 shutdown_connections 2024-04-12T21:16:51.911 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:51.914+0000 7f2183200700 1 --2- 172.21.15.18:0/1873476473 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f217c109dd0 0x7f217c103440 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:16:51.911 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:51.914+0000 7f2183200700 1 --2- 172.21.15.18:0/1873476473 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f217c109430 0x7f217c109800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:16:51.911 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:51.914+0000 7f2183200700 1 -- 172.21.15.18:0/1873476473 >> 172.21.15.18:0/1873476473 conn(0x7f217c072c00 msgr2=0x7f217c075010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:16:51.911 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:51.914+0000 7f2183200700 1 -- 172.21.15.18:0/1873476473 shutdown_connections 2024-04-12T21:16:51.911 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:51.914+0000 7f2183200700 1 -- 172.21.15.18:0/1873476473 wait complete. 2024-04-12T21:16:51.911 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:51.915+0000 7f2183200700 1 Processor -- start 2024-04-12T21:16:51.912 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:51.916+0000 7f2183200700 1 -- start start 2024-04-12T21:16:51.912 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:51.916+0000 7f2183200700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f217c109430 0x7f217c119f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:16:51.912 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:51.916+0000 7f2183200700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f217c109dd0 0x7f217c11a4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:16:51.912 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:51.916+0000 7f2183200700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f217c11ab10 con 0x7f217c109dd0 2024-04-12T21:16:51.913 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:51.916+0000 7f2183200700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f217c1140f0 con 0x7f217c109430 2024-04-12T21:16:51.913 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:51.916+0000 7f217b400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f217c109dd0 0x7f217c11a4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:16:51.913 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:51.916+0000 7f217b400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f217c109dd0 0x7f217c11a4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:42828/0 (socket says 172.21.15.18:42828) 2024-04-12T21:16:51.913 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:51.916+0000 7f217b400700 1 -- 172.21.15.18:0/3065543567 learned_addr learned my addr 172.21.15.18:0/3065543567 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:16:51.914 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:51.917+0000 7f217be00700 1 --2- 172.21.15.18:0/3065543567 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f217c109430 0x7f217c119f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:16:51.914 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:51.917+0000 7f217b400700 1 -- 172.21.15.18:0/3065543567 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f217c109430 msgr2=0x7f217c119f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:16:51.914 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:51.917+0000 7f217b400700 1 --2- 172.21.15.18:0/3065543567 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f217c109430 0x7f217c119f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:16:51.914 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:51.917+0000 7f217b400700 1 -- 172.21.15.18:0/3065543567 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2168005040 con 0x7f217c109dd0 2024-04-12T21:16:51.914 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:51.917+0000 7f217be00700 1 --2- 172.21.15.18:0/3065543567 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f217c109430 0x7f217c119f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-12T21:16:51.914 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:51.917+0000 7f217b400700 1 --2- 172.21.15.18:0/3065543567 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f217c109dd0 0x7f217c11a4c0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f2168009180 tx=0x7f2168009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:16:51.914 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:51.918+0000 7f2173e00700 1 -- 172.21.15.18:0/3065543567 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f216800e040 con 0x7f217c109dd0 2024-04-12T21:16:51.914 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:51.918+0000 7f2173e00700 1 -- 172.21.15.18:0/3065543567 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f216800bbc0 con 0x7f217c109dd0 2024-04-12T21:16:51.915 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:51.918+0000 7f2173e00700 1 -- 172.21.15.18:0/3065543567 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f216801c3f0 con 0x7f217c109dd0 2024-04-12T21:16:51.915 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:51.918+0000 7f2183200700 1 -- 172.21.15.18:0/3065543567 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f217c114370 con 0x7f217c109dd0 2024-04-12T21:16:51.915 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:51.918+0000 7f2183200700 1 -- 172.21.15.18:0/3065543567 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f217c114930 con 0x7f217c109dd0 2024-04-12T21:16:51.915 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:51.919+0000 7f2183200700 1 -- 172.21.15.18:0/3065543567 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f217c104b80 con 0x7f217c109dd0 2024-04-12T21:16:51.916 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:51.920+0000 7f2173e00700 1 -- 172.21.15.18:0/3065543567 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7f216800bd30 con 0x7f217c109dd0 2024-04-12T21:16:51.917 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:51.921+0000 7f2173e00700 1 --2- 172.21.15.18:0/3065543567 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f216006ec00 0x7f21600710b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:16:51.917 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:51.921+0000 7f2173e00700 1 -- 172.21.15.18:0/3065543567 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 6757+0+0 (secure 0 0 0) 0x7f216808fc40 con 0x7f217c109dd0 2024-04-12T21:16:51.918 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:51.922+0000 7f217be00700 1 --2- 172.21.15.18:0/3065543567 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f216006ec00 0x7f21600710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:16:51.918 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:51.922+0000 7f217be00700 1 --2- 172.21.15.18:0/3065543567 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f216006ec00 0x7f21600710b0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f216c009fd0 tx=0x7f216c00b040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:16:51.922 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:51.926+0000 7f2173e00700 1 -- 172.21.15.18:0/3065543567 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f216805c530 con 0x7f217c109dd0 2024-04-12T21:16:52.248 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:52.252+0000 7f2183200700 1 -- 172.21.15.18:0/3065543567 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f217c04c710 con 0x7f217c109dd0 2024-04-12T21:16:52.249 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:52.252+0000 7f2173e00700 1 -- 172.21.15.18:0/3065543567 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+233 (secure 0 0 0) 0x7f216805c0c0 con 0x7f217c109dd0 2024-04-12T21:16:52.249 INFO:teuthology.orchestra.run.smithi018.stdout:HEALTH_ERR 1 filesystem is degraded; 1 filesystem is offline 2024-04-12T21:16:52.249 INFO:teuthology.orchestra.run.smithi018.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-04-12T21:16:52.249 INFO:teuthology.orchestra.run.smithi018.stdout: fs cephfs is degraded 2024-04-12T21:16:52.249 INFO:teuthology.orchestra.run.smithi018.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-04-12T21:16:52.249 INFO:teuthology.orchestra.run.smithi018.stdout: fs cephfs is offline because no MDS is active for it. 2024-04-12T21:16:52.251 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:52.255+0000 7f2183200700 1 -- 172.21.15.18:0/3065543567 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f216006ec00 msgr2=0x7f21600710b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:16:52.251 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:52.255+0000 7f2183200700 1 --2- 172.21.15.18:0/3065543567 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f216006ec00 0x7f21600710b0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f216c009fd0 tx=0x7f216c00b040 comp rx=0 tx=0).stop 2024-04-12T21:16:52.251 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:52.255+0000 7f2183200700 1 -- 172.21.15.18:0/3065543567 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f217c109dd0 msgr2=0x7f217c11a4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:16:52.251 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:52.255+0000 7f2183200700 1 --2- 172.21.15.18:0/3065543567 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f217c109dd0 0x7f217c11a4c0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f2168009180 tx=0x7f2168009e30 comp rx=0 tx=0).stop 2024-04-12T21:16:52.252 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:52.255+0000 7f2183200700 1 -- 172.21.15.18:0/3065543567 shutdown_connections 2024-04-12T21:16:52.252 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:52.255+0000 7f2183200700 1 --2- 172.21.15.18:0/3065543567 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f216006ec00 0x7f21600710b0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:16:52.252 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:52.255+0000 7f2183200700 1 --2- 172.21.15.18:0/3065543567 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f217c109430 0x7f217c119f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:16:52.253 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:52.255+0000 7f2183200700 1 --2- 172.21.15.18:0/3065543567 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f217c109dd0 0x7f217c11a4c0 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:16:52.253 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:52.255+0000 7f2183200700 1 -- 172.21.15.18:0/3065543567 >> 172.21.15.18:0/3065543567 conn(0x7f217c072c00 msgr2=0x7f217c074c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:16:52.253 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:52.256+0000 7f2183200700 1 -- 172.21.15.18:0/3065543567 shutdown_connections 2024-04-12T21:16:52.253 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:16:52.256+0000 7f2183200700 1 -- 172.21.15.18:0/3065543567 wait complete. 2024-04-12T21:16:52.494 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:16:52 smithi018 ceph-mon[112590]: from='client.? 172.21.15.18:0/1146537' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-12T21:16:52.528 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:16:52 smithi186 ceph-mon[86379]: from='client.? 172.21.15.18:0/1146537' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-12T21:16:53.528 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:16:53 smithi186 ceph-mon[86379]: from='client.34940 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:16:53.529 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:16:53 smithi186 ceph-mon[86379]: pgmap v512: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:16:53.529 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:16:53 smithi186 ceph-mon[86379]: from='client.? 172.21.15.18:0/3065543567' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-12T21:16:53.599 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:16:53 smithi018 ceph-mon[112590]: from='client.34940 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:16:53.600 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:16:53 smithi018 ceph-mon[112590]: pgmap v512: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:16:53.600 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:16:53 smithi018 ceph-mon[112590]: from='client.? 172.21.15.18:0/3065543567' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-12T21:16:55.528 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:16:55 smithi186 ceph-mon[86379]: pgmap v513: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:16:55.528 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:16:55 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T21:16:55.599 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:16:55 smithi018 ceph-mon[112590]: pgmap v513: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:16:55.599 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:16:55 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T21:16:57.528 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:16:57 smithi186 ceph-mon[86379]: pgmap v514: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:16:57.599 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:16:57 smithi018 ceph-mon[112590]: pgmap v514: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:16:59.528 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:16:59 smithi186 ceph-mon[86379]: pgmap v515: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:16:59.599 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:16:59 smithi018 ceph-mon[112590]: pgmap v515: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:17:01.528 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:17:01 smithi186 ceph-mon[86379]: pgmap v516: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:17:01.599 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:17:01 smithi018 ceph-mon[112590]: pgmap v516: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:17:03.528 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:17:03 smithi186 ceph-mon[86379]: pgmap v517: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:17:03.599 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:17:03 smithi018 ceph-mon[112590]: pgmap v517: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:17:05.599 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:17:05 smithi018 ceph-mon[112590]: pgmap v518: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:17:05.778 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:17:05 smithi186 ceph-mon[86379]: pgmap v518: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:17:07.599 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:17:07 smithi018 ceph-mon[112590]: pgmap v519: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:17:07.778 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:17:07 smithi186 ceph-mon[86379]: pgmap v519: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:17:09.581 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:17:09 smithi186 ceph-mon[86379]: pgmap v520: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:17:09.599 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:17:09 smithi018 ceph-mon[112590]: pgmap v520: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:17:10.599 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:17:10 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T21:17:10.778 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:17:10 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T21:17:11.599 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:17:11 smithi018 ceph-mon[112590]: pgmap v521: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:17:11.778 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:17:11 smithi186 ceph-mon[86379]: pgmap v521: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:17:13.599 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:17:13 smithi018 ceph-mon[112590]: pgmap v522: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:17:13.778 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:17:13 smithi186 ceph-mon[86379]: pgmap v522: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:17:15.599 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:17:15 smithi018 ceph-mon[112590]: pgmap v523: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:17:15.778 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:17:15 smithi186 ceph-mon[86379]: pgmap v523: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:17:17.600 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:17:17 smithi018 ceph-mon[112590]: pgmap v524: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:17:17.778 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:17:17 smithi186 ceph-mon[86379]: pgmap v524: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:17:19.599 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:17:19 smithi018 ceph-mon[112590]: pgmap v525: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:17:19.778 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:17:19 smithi186 ceph-mon[86379]: pgmap v525: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:17:21.599 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:17:21 smithi018 ceph-mon[112590]: pgmap v526: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:17:21.778 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:17:21 smithi186 ceph-mon[86379]: pgmap v526: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:17:22.428 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:22.432+0000 7f3043600700 1 -- 172.21.15.18:0/85426856 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f303c0fde90 msgr2=0x7f303c0fe260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:17:22.428 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:22.432+0000 7f3043600700 1 --2- 172.21.15.18:0/85426856 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f303c0fde90 0x7f303c0fe260 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f302c009ce0 tx=0x7f302c004d10 comp rx=0 tx=0).stop 2024-04-12T21:17:22.428 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:22.432+0000 7f3043600700 1 -- 172.21.15.18:0/85426856 shutdown_connections 2024-04-12T21:17:22.428 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:22.432+0000 7f3043600700 1 --2- 172.21.15.18:0/85426856 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f303c0fe830 0x7f303c106f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:17:22.428 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:22.432+0000 7f3043600700 1 --2- 172.21.15.18:0/85426856 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f303c0fde90 0x7f303c0fe260 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:17:22.428 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:22.432+0000 7f3043600700 1 -- 172.21.15.18:0/85426856 >> 172.21.15.18:0/85426856 conn(0x7f303c072bc0 msgr2=0x7f303c072fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:17:22.429 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:22.433+0000 7f3043600700 1 -- 172.21.15.18:0/85426856 shutdown_connections 2024-04-12T21:17:22.429 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:22.433+0000 7f3043600700 1 -- 172.21.15.18:0/85426856 wait complete. 2024-04-12T21:17:22.430 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:22.434+0000 7f3043600700 1 Processor -- start 2024-04-12T21:17:22.430 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:22.434+0000 7f3043600700 1 -- start start 2024-04-12T21:17:22.430 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:22.434+0000 7f3043600700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f303c0fde90 0x7f303c113b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:17:22.430 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:22.434+0000 7f3043600700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f303c0fe830 0x7f303c1140c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:17:22.430 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:22.434+0000 7f3043600700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f303c1147a0 con 0x7f303c0fde90 2024-04-12T21:17:22.431 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:22.434+0000 7f3043600700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f303c10db50 con 0x7f303c0fe830 2024-04-12T21:17:22.431 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:22.434+0000 7f3040c00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f303c0fde90 0x7f303c113b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:17:22.431 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:22.435+0000 7f3040c00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f303c0fde90 0x7f303c113b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:59584/0 (socket says 172.21.15.18:59584) 2024-04-12T21:17:22.431 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:22.435+0000 7f3040c00700 1 -- 172.21.15.18:0/492030431 learned_addr learned my addr 172.21.15.18:0/492030431 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:17:22.431 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:22.435+0000 7f3040c00700 1 -- 172.21.15.18:0/492030431 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f303c0fe830 msgr2=0x7f303c1140c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:17:22.431 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:22.435+0000 7f3033e00700 1 --2- 172.21.15.18:0/492030431 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f303c0fe830 0x7f303c1140c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:17:22.431 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:22.435+0000 7f3040c00700 1 --2- 172.21.15.18:0/492030431 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f303c0fe830 0x7f303c1140c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:17:22.431 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:22.435+0000 7f3040c00700 1 -- 172.21.15.18:0/492030431 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f302c009940 con 0x7f303c0fde90 2024-04-12T21:17:22.431 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:22.435+0000 7f3033e00700 1 --2- 172.21.15.18:0/492030431 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f303c0fe830 0x7f303c1140c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-12T21:17:22.431 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:22.435+0000 7f3040c00700 1 --2- 172.21.15.18:0/492030431 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f303c0fde90 0x7f303c113b80 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f302c00b740 tx=0x7f302c004d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:17:22.432 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:22.436+0000 7f3039600700 1 -- 172.21.15.18:0/492030431 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f302c00e040 con 0x7f303c0fde90 2024-04-12T21:17:22.432 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:22.436+0000 7f3039600700 1 -- 172.21.15.18:0/492030431 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f302c0128e0 con 0x7f303c0fde90 2024-04-12T21:17:22.432 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:22.436+0000 7f3043600700 1 -- 172.21.15.18:0/492030431 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f303c10ddd0 con 0x7f303c0fde90 2024-04-12T21:17:22.432 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:22.436+0000 7f3039600700 1 -- 172.21.15.18:0/492030431 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f302c01c7c0 con 0x7f303c0fde90 2024-04-12T21:17:22.432 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:22.436+0000 7f3043600700 1 -- 172.21.15.18:0/492030431 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f303c10e310 con 0x7f303c0fde90 2024-04-12T21:17:22.433 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:22.437+0000 7f3043600700 1 -- 172.21.15.18:0/492030431 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f303c065e30 con 0x7f303c0fde90 2024-04-12T21:17:22.433 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:22.438+0000 7f3039600700 1 -- 172.21.15.18:0/492030431 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7f302c012410 con 0x7f303c0fde90 2024-04-12T21:17:22.435 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:22.440+0000 7f3039600700 1 --2- 172.21.15.18:0/492030431 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f302006ec00 0x7f30200710b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:17:22.435 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:22.440+0000 7f3039600700 1 -- 172.21.15.18:0/492030431 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 6757+0+0 (secure 0 0 0) 0x7f302c090780 con 0x7f303c0fde90 2024-04-12T21:17:22.435 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:22.440+0000 7f3033e00700 1 --2- 172.21.15.18:0/492030431 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f302006ec00 0x7f30200710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:17:22.435 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:22.441+0000 7f3033e00700 1 --2- 172.21.15.18:0/492030431 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f302006ec00 0x7f30200710b0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f3024006fd0 tx=0x7f3024008040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:17:22.440 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:22.445+0000 7f3039600700 1 -- 172.21.15.18:0/492030431 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f302c05cf50 con 0x7f303c0fde90 2024-04-12T21:17:22.692 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:22.697+0000 7f3043600700 1 -- 172.21.15.18:0/492030431 --> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f303c109f80 con 0x7f302006ec00 2024-04-12T21:17:22.694 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:22.699+0000 7f3039600700 1 -- 172.21.15.18:0/492030431 <== mgr.34102 v2:172.21.15.18:6800/3812891919 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f303c109f80 con 0x7f302006ec00 2024-04-12T21:17:22.697 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:22.701+0000 7f3043600700 1 -- 172.21.15.18:0/492030431 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f302006ec00 msgr2=0x7f30200710b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:17:22.697 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:22.701+0000 7f3043600700 1 --2- 172.21.15.18:0/492030431 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f302006ec00 0x7f30200710b0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f3024006fd0 tx=0x7f3024008040 comp rx=0 tx=0).stop 2024-04-12T21:17:22.697 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:22.702+0000 7f3043600700 1 -- 172.21.15.18:0/492030431 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f303c0fde90 msgr2=0x7f303c113b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:17:22.697 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:22.702+0000 7f3043600700 1 --2- 172.21.15.18:0/492030431 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f303c0fde90 0x7f303c113b80 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f302c00b740 tx=0x7f302c004d10 comp rx=0 tx=0).stop 2024-04-12T21:17:22.697 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:22.702+0000 7f3043600700 1 -- 172.21.15.18:0/492030431 shutdown_connections 2024-04-12T21:17:22.697 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:22.702+0000 7f3043600700 1 --2- 172.21.15.18:0/492030431 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f302006ec00 0x7f30200710b0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:17:22.698 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:22.702+0000 7f3043600700 1 --2- 172.21.15.18:0/492030431 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f303c0fde90 0x7f303c113b80 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:17:22.698 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:22.702+0000 7f3043600700 1 --2- 172.21.15.18:0/492030431 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f303c0fe830 0x7f303c1140c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:17:22.698 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:22.702+0000 7f3043600700 1 -- 172.21.15.18:0/492030431 >> 172.21.15.18:0/492030431 conn(0x7f303c072bc0 msgr2=0x7f303c0ff440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:17:22.698 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:22.702+0000 7f3043600700 1 -- 172.21.15.18:0/492030431 shutdown_connections 2024-04-12T21:17:22.698 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:22.702+0000 7f3043600700 1 -- 172.21.15.18:0/492030431 wait complete. 2024-04-12T21:17:22.712 INFO:teuthology.orchestra.run.smithi018.stdout:true 2024-04-12T21:17:22.847 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:22.852+0000 7fa0f4a00700 1 -- 172.21.15.18:0/216554431 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa0f00fe620 msgr2=0x7fa0f0106cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:17:22.847 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:22.852+0000 7fa0f4a00700 1 --2- 172.21.15.18:0/216554431 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa0f00fe620 0x7fa0f0106cf0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7fa0dc004940 tx=0x7fa0dc009e30 comp rx=0 tx=0).stop 2024-04-12T21:17:22.848 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:22.852+0000 7fa0f4a00700 1 -- 172.21.15.18:0/216554431 shutdown_connections 2024-04-12T21:17:22.848 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:22.852+0000 7fa0f4a00700 1 --2- 172.21.15.18:0/216554431 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa0f00fe620 0x7fa0f0106cf0 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:17:22.848 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:22.852+0000 7fa0f4a00700 1 --2- 172.21.15.18:0/216554431 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa0f00fdc80 0x7fa0f00fe050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:17:22.848 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:22.852+0000 7fa0f4a00700 1 -- 172.21.15.18:0/216554431 >> 172.21.15.18:0/216554431 conn(0x7fa0f00f76e0 msgr2=0x7fa0f00f9af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:17:22.848 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:22.852+0000 7fa0f4a00700 1 -- 172.21.15.18:0/216554431 shutdown_connections 2024-04-12T21:17:22.848 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:22.852+0000 7fa0f4a00700 1 -- 172.21.15.18:0/216554431 wait complete. 2024-04-12T21:17:22.850 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:22.853+0000 7fa0f4a00700 1 Processor -- start 2024-04-12T21:17:22.850 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:22.853+0000 7fa0f4a00700 1 -- start start 2024-04-12T21:17:22.850 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:22.853+0000 7fa0f4a00700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa0f00fdc80 0x7fa0f0115b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:17:22.850 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:22.853+0000 7fa0f4a00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa0f00fe620 0x7fa0f01160c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:17:22.850 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:22.853+0000 7fa0f4a00700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0f0116710 con 0x7fa0f00fe620 2024-04-12T21:17:22.850 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:22.853+0000 7fa0f4a00700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0f010fb20 con 0x7fa0f00fdc80 2024-04-12T21:17:22.850 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:22.854+0000 7fa0eea00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa0f00fe620 0x7fa0f01160c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:17:22.850 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:22.854+0000 7fa0eea00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa0f00fe620 0x7fa0f01160c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:59600/0 (socket says 172.21.15.18:59600) 2024-04-12T21:17:22.850 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:22.854+0000 7fa0eea00700 1 -- 172.21.15.18:0/4053804345 learned_addr learned my addr 172.21.15.18:0/4053804345 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:17:22.850 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:22.854+0000 7fa0ef400700 1 --2- 172.21.15.18:0/4053804345 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa0f00fdc80 0x7fa0f0115b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:17:22.850 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:22.854+0000 7fa0eea00700 1 -- 172.21.15.18:0/4053804345 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa0f00fdc80 msgr2=0x7fa0f0115b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:17:22.851 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:22.854+0000 7fa0eea00700 1 --2- 172.21.15.18:0/4053804345 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa0f00fdc80 0x7fa0f0115b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:17:22.851 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:22.854+0000 7fa0eea00700 1 -- 172.21.15.18:0/4053804345 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0dc005040 con 0x7fa0f00fe620 2024-04-12T21:17:22.851 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:22.854+0000 7fa0eea00700 1 --2- 172.21.15.18:0/4053804345 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa0f00fe620 0x7fa0f01160c0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7fa0dc009180 tx=0x7fa0dc009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:17:22.851 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:22.855+0000 7fa0e7400700 1 -- 172.21.15.18:0/4053804345 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa0dc00e040 con 0x7fa0f00fe620 2024-04-12T21:17:22.851 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:22.855+0000 7fa0e7400700 1 -- 172.21.15.18:0/4053804345 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa0dc013420 con 0x7fa0f00fe620 2024-04-12T21:17:22.851 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:22.855+0000 7fa0e7400700 1 -- 172.21.15.18:0/4053804345 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa0dc01c630 con 0x7fa0f00fe620 2024-04-12T21:17:22.851 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:22.855+0000 7fa0f4a00700 1 -- 172.21.15.18:0/4053804345 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa0f010fda0 con 0x7fa0f00fe620 2024-04-12T21:17:22.851 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:22.855+0000 7fa0f4a00700 1 -- 172.21.15.18:0/4053804345 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa0f01102e0 con 0x7fa0f00fe620 2024-04-12T21:17:22.851 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:22.856+0000 7fa0f4a00700 1 -- 172.21.15.18:0/4053804345 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa0f01067b0 con 0x7fa0f00fe620 2024-04-12T21:17:22.852 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:22.857+0000 7fa0e7400700 1 -- 172.21.15.18:0/4053804345 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7fa0dc013590 con 0x7fa0f00fe620 2024-04-12T21:17:22.853 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:22.858+0000 7fa0e7400700 1 --2- 172.21.15.18:0/4053804345 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fa0d406ec00 0x7fa0d40710b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:17:22.853 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:22.858+0000 7fa0e7400700 1 -- 172.21.15.18:0/4053804345 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 6757+0+0 (secure 0 0 0) 0x7fa0dc08fc60 con 0x7fa0f00fe620 2024-04-12T21:17:22.853 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:22.858+0000 7fa0ef400700 1 --2- 172.21.15.18:0/4053804345 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fa0d406ec00 0x7fa0d40710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:17:22.854 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:22.859+0000 7fa0ef400700 1 --2- 172.21.15.18:0/4053804345 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fa0d406ec00 0x7fa0d40710b0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fa0e0009b80 tx=0x7fa0e000b040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:17:22.858 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:22.863+0000 7fa0e7400700 1 -- 172.21.15.18:0/4053804345 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7fa0dc058a20 con 0x7fa0f00fe620 2024-04-12T21:17:23.098 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:23.103+0000 7fa0f4a00700 1 -- 172.21.15.18:0/4053804345 --> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fa0f01111d0 con 0x7fa0d406ec00 2024-04-12T21:17:23.099 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:23.104+0000 7fa0e7400700 1 -- 172.21.15.18:0/4053804345 <== mgr.34102 v2:172.21.15.18:6800/3812891919 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7fa0f01111d0 con 0x7fa0d406ec00 2024-04-12T21:17:23.103 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:23.107+0000 7fa0f4a00700 1 -- 172.21.15.18:0/4053804345 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fa0d406ec00 msgr2=0x7fa0d40710b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:17:23.103 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:23.107+0000 7fa0f4a00700 1 --2- 172.21.15.18:0/4053804345 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fa0d406ec00 0x7fa0d40710b0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fa0e0009b80 tx=0x7fa0e000b040 comp rx=0 tx=0).stop 2024-04-12T21:17:23.103 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:23.107+0000 7fa0f4a00700 1 -- 172.21.15.18:0/4053804345 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa0f00fe620 msgr2=0x7fa0f01160c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:17:23.103 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:23.107+0000 7fa0f4a00700 1 --2- 172.21.15.18:0/4053804345 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa0f00fe620 0x7fa0f01160c0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7fa0dc009180 tx=0x7fa0dc009e30 comp rx=0 tx=0).stop 2024-04-12T21:17:23.103 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:23.107+0000 7fa0f4a00700 1 -- 172.21.15.18:0/4053804345 shutdown_connections 2024-04-12T21:17:23.103 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:23.107+0000 7fa0f4a00700 1 --2- 172.21.15.18:0/4053804345 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fa0d406ec00 0x7fa0d40710b0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:17:23.103 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:23.107+0000 7fa0f4a00700 1 --2- 172.21.15.18:0/4053804345 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa0f00fdc80 0x7fa0f0115b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:17:23.103 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:23.107+0000 7fa0f4a00700 1 --2- 172.21.15.18:0/4053804345 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa0f00fe620 0x7fa0f01160c0 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:17:23.104 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:23.107+0000 7fa0f4a00700 1 -- 172.21.15.18:0/4053804345 >> 172.21.15.18:0/4053804345 conn(0x7fa0f00f76e0 msgr2=0x7fa0f00fd2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:17:23.104 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:23.107+0000 7fa0f4a00700 1 -- 172.21.15.18:0/4053804345 shutdown_connections 2024-04-12T21:17:23.104 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:23.107+0000 7fa0f4a00700 1 -- 172.21.15.18:0/4053804345 wait complete. 2024-04-12T21:17:23.256 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:23.261+0000 7f550e200700 1 -- 172.21.15.18:0/2244540000 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f55081092a0 msgr2=0x7f5508109670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:17:23.256 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:23.261+0000 7f550e200700 1 --2- 172.21.15.18:0/2244540000 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f55081092a0 0x7f5508109670 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f54fc004940 tx=0x7f54fc009e30 comp rx=0 tx=0).stop 2024-04-12T21:17:23.256 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:23.261+0000 7f550e200700 1 -- 172.21.15.18:0/2244540000 shutdown_connections 2024-04-12T21:17:23.256 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:23.261+0000 7f550e200700 1 --2- 172.21.15.18:0/2244540000 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5508109c40 0x7f55081032b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:17:23.256 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:23.261+0000 7f550e200700 1 --2- 172.21.15.18:0/2244540000 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f55081092a0 0x7f5508109670 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:17:23.256 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:23.261+0000 7f550e200700 1 -- 172.21.15.18:0/2244540000 >> 172.21.15.18:0/2244540000 conn(0x7f5508072a30 msgr2=0x7f5508074e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:17:23.257 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:23.261+0000 7f550e200700 1 -- 172.21.15.18:0/2244540000 shutdown_connections 2024-04-12T21:17:23.257 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:23.261+0000 7f550e200700 1 -- 172.21.15.18:0/2244540000 wait complete. 2024-04-12T21:17:23.257 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:23.262+0000 7f550e200700 1 Processor -- start 2024-04-12T21:17:23.257 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:23.262+0000 7f550e200700 1 -- start start 2024-04-12T21:17:23.258 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:23.263+0000 7f550e200700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f55081092a0 0x7f5508119f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:17:23.258 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:23.263+0000 7f550e200700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5508109c40 0x7f550811a4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:17:23.258 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:23.263+0000 7f550e200700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f550811aaf0 con 0x7f55081092a0 2024-04-12T21:17:23.258 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:23.263+0000 7f550e200700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5508113ec0 con 0x7f5508109c40 2024-04-12T21:17:23.259 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:23.263+0000 7f550ce00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f55081092a0 0x7f5508119f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:17:23.259 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:23.263+0000 7f550ce00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f55081092a0 0x7f5508119f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:59632/0 (socket says 172.21.15.18:59632) 2024-04-12T21:17:23.259 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:23.263+0000 7f550ce00700 1 -- 172.21.15.18:0/875349836 learned_addr learned my addr 172.21.15.18:0/875349836 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:17:23.259 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:23.263+0000 7f5507e00700 1 --2- 172.21.15.18:0/875349836 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5508109c40 0x7f550811a4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:17:23.259 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:23.264+0000 7f550ce00700 1 -- 172.21.15.18:0/875349836 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5508109c40 msgr2=0x7f550811a4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:17:23.259 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:23.264+0000 7f550ce00700 1 --2- 172.21.15.18:0/875349836 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5508109c40 0x7f550811a4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:17:23.259 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:23.264+0000 7f550ce00700 1 -- 172.21.15.18:0/875349836 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54fc005040 con 0x7f55081092a0 2024-04-12T21:17:23.259 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:23.264+0000 7f550ce00700 1 --2- 172.21.15.18:0/875349836 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f55081092a0 0x7f5508119f60 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f54fc009150 tx=0x7f54fc009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:17:23.260 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:23.264+0000 7f5504c00700 1 -- 172.21.15.18:0/875349836 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f54fc021070 con 0x7f55081092a0 2024-04-12T21:17:23.260 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:23.264+0000 7f5504c00700 1 -- 172.21.15.18:0/875349836 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f54fc013c00 con 0x7f55081092a0 2024-04-12T21:17:23.260 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:23.264+0000 7f5504c00700 1 -- 172.21.15.18:0/875349836 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f54fc00e070 con 0x7f55081092a0 2024-04-12T21:17:23.260 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:23.264+0000 7f550e200700 1 -- 172.21.15.18:0/875349836 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5508114140 con 0x7f55081092a0 2024-04-12T21:17:23.260 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:23.265+0000 7f550e200700 1 -- 172.21.15.18:0/875349836 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5508114620 con 0x7f55081092a0 2024-04-12T21:17:23.261 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:23.266+0000 7f550e200700 1 -- 172.21.15.18:0/875349836 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f55081049f0 con 0x7f55081092a0 2024-04-12T21:17:23.261 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:23.267+0000 7f5504c00700 1 -- 172.21.15.18:0/875349836 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7f54fc01c400 con 0x7f55081092a0 2024-04-12T21:17:23.262 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:23.268+0000 7f5504c00700 1 --2- 172.21.15.18:0/875349836 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f54f006ec00 0x7f54f00710b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:17:23.263 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:23.268+0000 7f5504c00700 1 -- 172.21.15.18:0/875349836 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 6757+0+0 (secure 0 0 0) 0x7f54fc08fe60 con 0x7f55081092a0 2024-04-12T21:17:23.263 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:23.268+0000 7f5507e00700 1 --2- 172.21.15.18:0/875349836 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f54f006ec00 0x7f54f00710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:17:23.263 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:23.269+0000 7f5507e00700 1 --2- 172.21.15.18:0/875349836 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f54f006ec00 0x7f54f00710b0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f54f4004600 tx=0x7f54f400b040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:17:23.267 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:23.272+0000 7f5504c00700 1 -- 172.21.15.18:0/875349836 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f54fc05c6b0 con 0x7f55081092a0 2024-04-12T21:17:23.349 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:17:23 smithi018 ceph-mon[112590]: pgmap v527: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:17:23.514 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:23.519+0000 7f550e200700 1 -- 172.21.15.18:0/875349836 --> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f550804bf40 con 0x7f54f006ec00 2024-04-12T21:17:23.525 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:23.530+0000 7f5504c00700 1 -- 172.21.15.18:0/875349836 <== mgr.34102 v2:172.21.15.18:6800/3812891919 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7f550804bf40 con 0x7f54f006ec00 2024-04-12T21:17:23.526 INFO:teuthology.orchestra.run.smithi018.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-12T21:17:23.526 INFO:teuthology.orchestra.run.smithi018.stdout:alertmanager.smithi018 smithi018 *:9093,9094 running (17m) 9m ago 32m 14.7M - 0.25.0 c8568f914cd2 c332090068e8 2024-04-12T21:17:23.526 INFO:teuthology.orchestra.run.smithi018.stdout:crash.smithi018 smithi018 running (14m) 9m ago 32m 6665k - 19.0.0-2879-g324baf8d 8717720dc5e5 b7f80d1f7982 2024-04-12T21:17:23.526 INFO:teuthology.orchestra.run.smithi018.stdout:crash.smithi186 smithi186 running (14m) 10m ago 31m 6660k - 19.0.0-2879-g324baf8d 8717720dc5e5 a4f55c81b623 2024-04-12T21:17:23.526 INFO:teuthology.orchestra.run.smithi018.stdout:grafana.smithi018 smithi018 *:3000 running (17m) 9m ago 32m 74.8M - 9.4.12 2bacad6d85d8 b08e3bf176dd 2024-04-12T21:17:23.526 INFO:teuthology.orchestra.run.smithi018.stdout:mds.cephfs.smithi018.acqbsc smithi018 running (10m) 9m ago 25m 11.1M - 19.0.0-2879-g324baf8d 8717720dc5e5 021f99ab24ae 2024-04-12T21:17:23.526 INFO:teuthology.orchestra.run.smithi018.stdout:mds.cephfs.smithi018.szjluo smithi018 running (9m) 9m ago 25m 10.9M - 19.0.0-2879-g324baf8d 8717720dc5e5 0003117479d4 2024-04-12T21:17:23.526 INFO:teuthology.orchestra.run.smithi018.stdout:mds.cephfs.smithi186.jdllyn smithi186 running (25m) 10m ago 25m 15.6M - 17.2.7-827-gd3b5060a 21de9b7369ad 2b2bc46bd29a 2024-04-12T21:17:23.526 INFO:teuthology.orchestra.run.smithi018.stdout:mds.cephfs.smithi186.wjnafs smithi186 running (25m) 10m ago 25m 15.4M - 17.2.7-827-gd3b5060a 21de9b7369ad 74b115d6d6fd 2024-04-12T21:17:23.526 INFO:teuthology.orchestra.run.smithi018.stdout:mgr.smithi018.xjzwga smithi018 *:8443,9283,8765 running (20m) 9m ago 34m 509M - 19.0.0-2879-g324baf8d 8717720dc5e5 56088305eab7 2024-04-12T21:17:23.526 INFO:teuthology.orchestra.run.smithi018.stdout:mgr.smithi186.kqcfxk smithi186 *:8443,9283,8765 running (19m) 10m ago 31m 452M - 19.0.0-2879-g324baf8d 8717720dc5e5 09cd766e5e0d 2024-04-12T21:17:23.526 INFO:teuthology.orchestra.run.smithi018.stdout:mon.smithi018 smithi018 running (15m) 9m ago 34m 57.0M 2048M 19.0.0-2879-g324baf8d 8717720dc5e5 df3e201dfed6 2024-04-12T21:17:23.527 INFO:teuthology.orchestra.run.smithi018.stdout:mon.smithi186 smithi186 running (15m) 10m ago 31m 37.7M 2048M 19.0.0-2879-g324baf8d 8717720dc5e5 c77498da9b2b 2024-04-12T21:17:23.527 INFO:teuthology.orchestra.run.smithi018.stdout:node-exporter.smithi018 smithi018 *:9100 running (19m) 9m ago 32m 7780k - 1.5.0 0da6a335fe13 326c3d9c6847 2024-04-12T21:17:23.527 INFO:teuthology.orchestra.run.smithi018.stdout:node-exporter.smithi186 smithi186 *:9100 running (18m) 10m ago 31m 7826k - 1.5.0 0da6a335fe13 ec2112baeecf 2024-04-12T21:17:23.527 INFO:teuthology.orchestra.run.smithi018.stdout:osd.0 smithi018 running (13m) 9m ago 30m 121M 1559M 19.0.0-2879-g324baf8d 8717720dc5e5 60b1273ca047 2024-04-12T21:17:23.527 INFO:teuthology.orchestra.run.smithi018.stdout:osd.1 smithi018 running (13m) 9m ago 29m 91.3M 1559M 19.0.0-2879-g324baf8d 8717720dc5e5 170ce1e5fb8d 2024-04-12T21:17:23.527 INFO:teuthology.orchestra.run.smithi018.stdout:osd.2 smithi018 running (12m) 9m ago 28m 90.3M 1559M 19.0.0-2879-g324baf8d 8717720dc5e5 c2cbc6aa2839 2024-04-12T21:17:23.527 INFO:teuthology.orchestra.run.smithi018.stdout:osd.3 smithi186 running (12m) 10m ago 28m 96.6M 2581M 19.0.0-2879-g324baf8d 8717720dc5e5 0138e8262e4b 2024-04-12T21:17:23.527 INFO:teuthology.orchestra.run.smithi018.stdout:osd.4 smithi186 running (11m) 10m ago 27m 87.2M 2581M 19.0.0-2879-g324baf8d 8717720dc5e5 b20cb63b25c9 2024-04-12T21:17:23.527 INFO:teuthology.orchestra.run.smithi018.stdout:osd.5 smithi186 running (10m) 10m ago 27m 11.1M 2581M 19.0.0-2879-g324baf8d 8717720dc5e5 2513bea2610e 2024-04-12T21:17:23.527 INFO:teuthology.orchestra.run.smithi018.stdout:prometheus.smithi018 smithi018 *:9095 running (18m) 9m ago 31m 60.6M - 2.43.0 a07b618ecd1d f58841127e21 2024-04-12T21:17:23.528 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:23.533+0000 7f550e200700 1 -- 172.21.15.18:0/875349836 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f54f006ec00 msgr2=0x7f54f00710b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:17:23.528 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:23.533+0000 7f550e200700 1 --2- 172.21.15.18:0/875349836 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f54f006ec00 0x7f54f00710b0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f54f4004600 tx=0x7f54f400b040 comp rx=0 tx=0).stop 2024-04-12T21:17:23.529 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:23.533+0000 7f550e200700 1 -- 172.21.15.18:0/875349836 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f55081092a0 msgr2=0x7f5508119f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:17:23.529 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:23.533+0000 7f550e200700 1 --2- 172.21.15.18:0/875349836 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f55081092a0 0x7f5508119f60 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f54fc009150 tx=0x7f54fc009e30 comp rx=0 tx=0).stop 2024-04-12T21:17:23.529 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:23.533+0000 7f550e200700 1 -- 172.21.15.18:0/875349836 shutdown_connections 2024-04-12T21:17:23.529 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:23.533+0000 7f550e200700 1 --2- 172.21.15.18:0/875349836 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f54f006ec00 0x7f54f00710b0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:17:23.529 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:23.533+0000 7f550e200700 1 --2- 172.21.15.18:0/875349836 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f55081092a0 0x7f5508119f60 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:17:23.529 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:23.533+0000 7f550e200700 1 --2- 172.21.15.18:0/875349836 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5508109c40 0x7f550811a4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:17:23.529 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:23.533+0000 7f550e200700 1 -- 172.21.15.18:0/875349836 >> 172.21.15.18:0/875349836 conn(0x7f5508072a30 msgr2=0x7f55080748d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:17:23.529 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:23.533+0000 7f550e200700 1 -- 172.21.15.18:0/875349836 shutdown_connections 2024-04-12T21:17:23.529 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:23.533+0000 7f550e200700 1 -- 172.21.15.18:0/875349836 wait complete. 2024-04-12T21:17:23.680 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:23.684+0000 7f19f5400700 1 -- 172.21.15.18:0/588701644 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f19f0109dd0 msgr2=0x7f19f0103440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:17:23.680 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:23.684+0000 7f19f5400700 1 --2- 172.21.15.18:0/588701644 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f19f0109dd0 0x7f19f0103440 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f19d8004940 tx=0x7f19d8009e30 comp rx=0 tx=0).stop 2024-04-12T21:17:23.680 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:23.685+0000 7f19f5400700 1 -- 172.21.15.18:0/588701644 shutdown_connections 2024-04-12T21:17:23.680 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:23.685+0000 7f19f5400700 1 --2- 172.21.15.18:0/588701644 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f19f0109dd0 0x7f19f0103440 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:17:23.680 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:23.685+0000 7f19f5400700 1 --2- 172.21.15.18:0/588701644 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f19f0109430 0x7f19f0109800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:17:23.680 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:23.685+0000 7f19f5400700 1 -- 172.21.15.18:0/588701644 >> 172.21.15.18:0/588701644 conn(0x7f19f0072c00 msgr2=0x7f19f0075010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:17:23.680 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:23.685+0000 7f19f5400700 1 -- 172.21.15.18:0/588701644 shutdown_connections 2024-04-12T21:17:23.681 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:23.685+0000 7f19f5400700 1 -- 172.21.15.18:0/588701644 wait complete. 2024-04-12T21:17:23.681 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:23.686+0000 7f19f5400700 1 Processor -- start 2024-04-12T21:17:23.681 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:23.686+0000 7f19f5400700 1 -- start start 2024-04-12T21:17:23.682 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:23.687+0000 7f19f5400700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f19f0109430 0x7f19f0119f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:17:23.682 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:23.687+0000 7f19f5400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f19f0109dd0 0x7f19f011a4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:17:23.682 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:23.687+0000 7f19f5400700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19f011ab10 con 0x7f19f0109dd0 2024-04-12T21:17:23.682 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:23.687+0000 7f19f5400700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19f01140f0 con 0x7f19f0109430 2024-04-12T21:17:23.683 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:23.687+0000 7f19e5e00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f19f0109dd0 0x7f19f011a4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:17:23.683 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:23.687+0000 7f19e5e00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f19f0109dd0 0x7f19f011a4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:59638/0 (socket says 172.21.15.18:59638) 2024-04-12T21:17:23.683 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:23.687+0000 7f19e5e00700 1 -- 172.21.15.18:0/2590316161 learned_addr learned my addr 172.21.15.18:0/2590316161 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:17:23.683 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:23.687+0000 7f19ee800700 1 --2- 172.21.15.18:0/2590316161 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f19f0109430 0x7f19f0119f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:17:23.683 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:23.687+0000 7f19e5e00700 1 -- 172.21.15.18:0/2590316161 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f19f0109430 msgr2=0x7f19f0119f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:17:23.683 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:23.687+0000 7f19e5e00700 1 --2- 172.21.15.18:0/2590316161 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f19f0109430 0x7f19f0119f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:17:23.683 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:23.687+0000 7f19e5e00700 1 -- 172.21.15.18:0/2590316161 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19d8005040 con 0x7f19f0109dd0 2024-04-12T21:17:23.683 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:23.687+0000 7f19ee800700 1 --2- 172.21.15.18:0/2590316161 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f19f0109430 0x7f19f0119f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-12T21:17:23.684 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:23.688+0000 7f19e5e00700 1 --2- 172.21.15.18:0/2590316161 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f19f0109dd0 0x7f19f011a4c0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f19d8009100 tx=0x7f19d8009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:17:23.684 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:23.688+0000 7f19e7400700 1 -- 172.21.15.18:0/2590316161 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f19d800e040 con 0x7f19f0109dd0 2024-04-12T21:17:23.684 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:23.688+0000 7f19f5400700 1 -- 172.21.15.18:0/2590316161 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f19f0114370 con 0x7f19f0109dd0 2024-04-12T21:17:23.684 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:23.688+0000 7f19e7400700 1 -- 172.21.15.18:0/2590316161 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f19d8013420 con 0x7f19f0109dd0 2024-04-12T21:17:23.684 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:23.688+0000 7f19e7400700 1 -- 172.21.15.18:0/2590316161 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f19d801c630 con 0x7f19f0109dd0 2024-04-12T21:17:23.684 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:23.688+0000 7f19f5400700 1 -- 172.21.15.18:0/2590316161 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f19f0114930 con 0x7f19f0109dd0 2024-04-12T21:17:23.684 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:23.689+0000 7f19f5400700 1 -- 172.21.15.18:0/2590316161 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f19f0104b80 con 0x7f19f0109dd0 2024-04-12T21:17:23.685 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:23.690+0000 7f19e7400700 1 -- 172.21.15.18:0/2590316161 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7f19d8013590 con 0x7f19f0109dd0 2024-04-12T21:17:23.686 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:23.691+0000 7f19e7400700 1 --2- 172.21.15.18:0/2590316161 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f19d406ec00 0x7f19d40710b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:17:23.686 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:23.691+0000 7f19e7400700 1 -- 172.21.15.18:0/2590316161 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 6757+0+0 (secure 0 0 0) 0x7f19d808fba0 con 0x7f19f0109dd0 2024-04-12T21:17:23.686 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:23.692+0000 7f19ee800700 1 --2- 172.21.15.18:0/2590316161 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f19d406ec00 0x7f19d40710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:17:23.687 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:23.692+0000 7f19ee800700 1 --2- 172.21.15.18:0/2590316161 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f19d406ec00 0x7f19d40710b0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f19e0004550 tx=0x7f19e0009040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:17:23.691 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:23.696+0000 7f19e7400700 1 -- 172.21.15.18:0/2590316161 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f19d805c3d0 con 0x7f19f0109dd0 2024-04-12T21:17:23.778 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:17:23 smithi186 ceph-mon[86379]: pgmap v527: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:17:24.011 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:24.015+0000 7f19f5400700 1 -- 172.21.15.18:0/2590316161 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f19f0110500 con 0x7f19f0109dd0 2024-04-12T21:17:24.011 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:24.016+0000 7f19e7400700 1 -- 172.21.15.18:0/2590316161 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+837 (secure 0 0 0) 0x7f19d8058030 con 0x7f19f0109dd0 2024-04-12T21:17:24.012 INFO:teuthology.orchestra.run.smithi018.stdout:{ 2024-04-12T21:17:24.012 INFO:teuthology.orchestra.run.smithi018.stdout: "mon": { 2024-04-12T21:17:24.012 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 19.0.0-2879-g324baf8d (324baf8ddb91dea46cac50cb535c306115791355) squid (dev)": 2 2024-04-12T21:17:24.012 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-12T21:17:24.012 INFO:teuthology.orchestra.run.smithi018.stdout: "mgr": { 2024-04-12T21:17:24.012 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 19.0.0-2879-g324baf8d (324baf8ddb91dea46cac50cb535c306115791355) squid (dev)": 2 2024-04-12T21:17:24.012 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-12T21:17:24.012 INFO:teuthology.orchestra.run.smithi018.stdout: "osd": { 2024-04-12T21:17:24.012 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 19.0.0-2879-g324baf8d (324baf8ddb91dea46cac50cb535c306115791355) squid (dev)": 6 2024-04-12T21:17:24.012 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-12T21:17:24.012 INFO:teuthology.orchestra.run.smithi018.stdout: "mds": { 2024-04-12T21:17:24.013 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 17.2.7-827-gd3b5060a (d3b5060a17449edfae9385cdf792afec109fca5d) quincy (stable)": 2, 2024-04-12T21:17:24.013 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 19.0.0-2879-g324baf8d (324baf8ddb91dea46cac50cb535c306115791355) squid (dev)": 2 2024-04-12T21:17:24.013 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-12T21:17:24.013 INFO:teuthology.orchestra.run.smithi018.stdout: "overall": { 2024-04-12T21:17:24.013 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 17.2.7-827-gd3b5060a (d3b5060a17449edfae9385cdf792afec109fca5d) quincy (stable)": 2, 2024-04-12T21:17:24.013 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 19.0.0-2879-g324baf8d (324baf8ddb91dea46cac50cb535c306115791355) squid (dev)": 12 2024-04-12T21:17:24.013 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-04-12T21:17:24.013 INFO:teuthology.orchestra.run.smithi018.stdout:} 2024-04-12T21:17:24.015 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:24.019+0000 7f19f5400700 1 -- 172.21.15.18:0/2590316161 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f19d406ec00 msgr2=0x7f19d40710b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:17:24.015 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:24.019+0000 7f19f5400700 1 --2- 172.21.15.18:0/2590316161 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f19d406ec00 0x7f19d40710b0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f19e0004550 tx=0x7f19e0009040 comp rx=0 tx=0).stop 2024-04-12T21:17:24.015 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:24.019+0000 7f19f5400700 1 -- 172.21.15.18:0/2590316161 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f19f0109dd0 msgr2=0x7f19f011a4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:17:24.015 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:24.019+0000 7f19f5400700 1 --2- 172.21.15.18:0/2590316161 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f19f0109dd0 0x7f19f011a4c0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f19d8009100 tx=0x7f19d8009e30 comp rx=0 tx=0).stop 2024-04-12T21:17:24.015 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:24.019+0000 7f19f5400700 1 -- 172.21.15.18:0/2590316161 shutdown_connections 2024-04-12T21:17:24.015 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:24.019+0000 7f19f5400700 1 --2- 172.21.15.18:0/2590316161 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f19d406ec00 0x7f19d40710b0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:17:24.015 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:24.019+0000 7f19f5400700 1 --2- 172.21.15.18:0/2590316161 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f19f0109430 0x7f19f0119f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:17:24.015 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:24.019+0000 7f19f5400700 1 --2- 172.21.15.18:0/2590316161 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f19f0109dd0 0x7f19f011a4c0 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:17:24.015 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:24.019+0000 7f19f5400700 1 -- 172.21.15.18:0/2590316161 >> 172.21.15.18:0/2590316161 conn(0x7f19f0072c00 msgr2=0x7f19f0074c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:17:24.016 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:24.020+0000 7f19f5400700 1 -- 172.21.15.18:0/2590316161 shutdown_connections 2024-04-12T21:17:24.016 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:24.020+0000 7f19f5400700 1 -- 172.21.15.18:0/2590316161 wait complete. 2024-04-12T21:17:24.170 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:24.174+0000 7f47b5400700 1 -- 172.21.15.18:0/4279230419 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f47b0109dd0 msgr2=0x7f47b0103440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:17:24.170 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:24.174+0000 7f47b5400700 1 --2- 172.21.15.18:0/4279230419 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f47b0109dd0 0x7f47b0103440 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f479c004940 tx=0x7f479c009e30 comp rx=0 tx=0).stop 2024-04-12T21:17:24.170 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:24.174+0000 7f47b5400700 1 -- 172.21.15.18:0/4279230419 shutdown_connections 2024-04-12T21:17:24.170 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:24.174+0000 7f47b5400700 1 --2- 172.21.15.18:0/4279230419 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f47b0109dd0 0x7f47b0103440 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:17:24.170 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:24.174+0000 7f47b5400700 1 --2- 172.21.15.18:0/4279230419 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f47b0109430 0x7f47b0109800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:17:24.170 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:24.174+0000 7f47b5400700 1 -- 172.21.15.18:0/4279230419 >> 172.21.15.18:0/4279230419 conn(0x7f47b0072c00 msgr2=0x7f47b0075010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:17:24.170 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:24.174+0000 7f47b5400700 1 -- 172.21.15.18:0/4279230419 shutdown_connections 2024-04-12T21:17:24.170 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:24.175+0000 7f47b5400700 1 -- 172.21.15.18:0/4279230419 wait complete. 2024-04-12T21:17:24.171 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:24.175+0000 7f47b5400700 1 Processor -- start 2024-04-12T21:17:24.172 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:24.175+0000 7f47b5400700 1 -- start start 2024-04-12T21:17:24.172 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:24.176+0000 7f47b5400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f47b0109430 0x7f47b0119f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:17:24.172 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:24.176+0000 7f47b5400700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f47b0109dd0 0x7f47b011a4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:17:24.172 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:24.176+0000 7f47b5400700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47b011ab10 con 0x7f47b0109430 2024-04-12T21:17:24.172 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:24.176+0000 7f47b5400700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47b01140f0 con 0x7f47b0109dd0 2024-04-12T21:17:24.172 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:24.176+0000 7f47ae800700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f47b0109430 0x7f47b0119f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:17:24.172 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:24.176+0000 7f47ae800700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f47b0109430 0x7f47b0119f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:59656/0 (socket says 172.21.15.18:59656) 2024-04-12T21:17:24.172 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:24.176+0000 7f47ae800700 1 -- 172.21.15.18:0/3066645680 learned_addr learned my addr 172.21.15.18:0/3066645680 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:17:24.172 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:24.176+0000 7f47ae800700 1 -- 172.21.15.18:0/3066645680 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f47b0109dd0 msgr2=0x7f47b011a4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:17:24.172 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:24.176+0000 7f47ae800700 1 --2- 172.21.15.18:0/3066645680 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f47b0109dd0 0x7f47b011a4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:17:24.172 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:24.176+0000 7f47ae800700 1 -- 172.21.15.18:0/3066645680 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f479c005040 con 0x7f47b0109430 2024-04-12T21:17:24.172 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:24.177+0000 7f47ae800700 1 --2- 172.21.15.18:0/3066645680 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f47b0109430 0x7f47b0119f80 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f47a0002a50 tx=0x7f47a00046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:17:24.173 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:24.177+0000 7f47a6a00700 1 -- 172.21.15.18:0/3066645680 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f47a0007cc0 con 0x7f47b0109430 2024-04-12T21:17:24.173 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:24.177+0000 7f47a6a00700 1 -- 172.21.15.18:0/3066645680 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f47a000f040 con 0x7f47b0109430 2024-04-12T21:17:24.173 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:24.177+0000 7f47a6a00700 1 -- 172.21.15.18:0/3066645680 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f47a0009640 con 0x7f47b0109430 2024-04-12T21:17:24.173 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:24.177+0000 7f47b5400700 1 -- 172.21.15.18:0/3066645680 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f47b01143d0 con 0x7f47b0109430 2024-04-12T21:17:24.173 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:24.177+0000 7f47b5400700 1 -- 172.21.15.18:0/3066645680 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f47b0114990 con 0x7f47b0109430 2024-04-12T21:17:24.173 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:24.178+0000 7f47b5400700 1 -- 172.21.15.18:0/3066645680 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f47b0104b80 con 0x7f47b0109430 2024-04-12T21:17:24.174 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:24.180+0000 7f47a6a00700 1 -- 172.21.15.18:0/3066645680 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7f47a00097c0 con 0x7f47b0109430 2024-04-12T21:17:24.175 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:24.180+0000 7f47a6a00700 1 --2- 172.21.15.18:0/3066645680 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f479406ec00 0x7f47940710b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:17:24.176 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:24.181+0000 7f47a6a00700 1 -- 172.21.15.18:0/3066645680 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 6757+0+0 (secure 0 0 0) 0x7f47a008e700 con 0x7f47b0109430 2024-04-12T21:17:24.176 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:24.181+0000 7f47ade00700 1 --2- 172.21.15.18:0/3066645680 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f479406ec00 0x7f47940710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:17:24.176 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:24.181+0000 7f47ade00700 1 --2- 172.21.15.18:0/3066645680 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f479406ec00 0x7f47940710b0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f479c004f90 tx=0x7f479c009e30 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:17:24.180 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:24.185+0000 7f47a6a00700 1 -- 172.21.15.18:0/3066645680 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f47a005aeb0 con 0x7f47b0109430 2024-04-12T21:17:24.460 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:24.464+0000 7f47b5400700 1 -- 172.21.15.18:0/3066645680 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f47b0110500 con 0x7f47b0109430 2024-04-12T21:17:24.460 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:24.465+0000 7f47a6a00700 1 -- 172.21.15.18:0/3066645680 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 29 v29) v1 ==== 76+0+1872 (secure 0 0 0) 0x7f47a005aa40 con 0x7f47b0109430 2024-04-12T21:17:24.461 INFO:teuthology.orchestra.run.smithi018.stdout:e29 2024-04-12T21:17:24.461 INFO:teuthology.orchestra.run.smithi018.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-12T21:17:24.461 INFO:teuthology.orchestra.run.smithi018.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-12T21:17:24.462 INFO:teuthology.orchestra.run.smithi018.stdout:legacy client fscid: 1 2024-04-12T21:17:24.462 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-12T21:17:24.462 INFO:teuthology.orchestra.run.smithi018.stdout:Filesystem 'cephfs' (1) 2024-04-12T21:17:24.462 INFO:teuthology.orchestra.run.smithi018.stdout:fs_name cephfs 2024-04-12T21:17:24.462 INFO:teuthology.orchestra.run.smithi018.stdout:epoch 24 2024-04-12T21:17:24.462 INFO:teuthology.orchestra.run.smithi018.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-04-12T21:17:24.462 INFO:teuthology.orchestra.run.smithi018.stdout:created 2024-04-12T20:51:57.362850+0000 2024-04-12T21:17:24.462 INFO:teuthology.orchestra.run.smithi018.stdout:modified 2024-04-12T21:06:59.717190+0000 2024-04-12T21:17:24.462 INFO:teuthology.orchestra.run.smithi018.stdout:tableserver 0 2024-04-12T21:17:24.462 INFO:teuthology.orchestra.run.smithi018.stdout:root 0 2024-04-12T21:17:24.462 INFO:teuthology.orchestra.run.smithi018.stdout:session_timeout 60 2024-04-12T21:17:24.462 INFO:teuthology.orchestra.run.smithi018.stdout:session_autoclose 300 2024-04-12T21:17:24.462 INFO:teuthology.orchestra.run.smithi018.stdout:max_file_size 1099511627776 2024-04-12T21:17:24.462 INFO:teuthology.orchestra.run.smithi018.stdout:max_xattr_size 65536 2024-04-12T21:17:24.463 INFO:teuthology.orchestra.run.smithi018.stdout:required_client_features {} 2024-04-12T21:17:24.463 INFO:teuthology.orchestra.run.smithi018.stdout:last_failure 0 2024-04-12T21:17:24.463 INFO:teuthology.orchestra.run.smithi018.stdout:last_failure_osd_epoch 100 2024-04-12T21:17:24.463 INFO:teuthology.orchestra.run.smithi018.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-12T21:17:24.463 INFO:teuthology.orchestra.run.smithi018.stdout:max_mds 2 2024-04-12T21:17:24.463 INFO:teuthology.orchestra.run.smithi018.stdout:in 0,1 2024-04-12T21:17:24.463 INFO:teuthology.orchestra.run.smithi018.stdout:up {} 2024-04-12T21:17:24.463 INFO:teuthology.orchestra.run.smithi018.stdout:failed 0,1 2024-04-12T21:17:24.463 INFO:teuthology.orchestra.run.smithi018.stdout:damaged 2024-04-12T21:17:24.463 INFO:teuthology.orchestra.run.smithi018.stdout:stopped 2024-04-12T21:17:24.463 INFO:teuthology.orchestra.run.smithi018.stdout:data_pools [3] 2024-04-12T21:17:24.463 INFO:teuthology.orchestra.run.smithi018.stdout:metadata_pool 2 2024-04-12T21:17:24.463 INFO:teuthology.orchestra.run.smithi018.stdout:inline_data disabled 2024-04-12T21:17:24.463 INFO:teuthology.orchestra.run.smithi018.stdout:balancer 2024-04-12T21:17:24.463 INFO:teuthology.orchestra.run.smithi018.stdout:bal_rank_mask -1 2024-04-12T21:17:24.464 INFO:teuthology.orchestra.run.smithi018.stdout:standby_count_wanted 1 2024-04-12T21:17:24.464 INFO:teuthology.orchestra.run.smithi018.stdout:qdb_cluster leader: 0 members: 2024-04-12T21:17:24.464 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-12T21:17:24.464 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-12T21:17:24.464 INFO:teuthology.orchestra.run.smithi018.stdout:Standby daemons: 2024-04-12T21:17:24.464 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-12T21:17:24.464 INFO:teuthology.orchestra.run.smithi018.stdout:[mds.cephfs.smithi186.wjnafs{-1:24277} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.186:6826/438347207,v1:172.21.15.186:6827/438347207] compat {c=[1],r=[1],i=[7ff]}] 2024-04-12T21:17:24.464 INFO:teuthology.orchestra.run.smithi018.stdout:[mds.cephfs.smithi186.jdllyn{-1:24281} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.186:6824/1464359952,v1:172.21.15.186:6825/1464359952] compat {c=[1],r=[1],i=[7ff]}] 2024-04-12T21:17:24.464 INFO:teuthology.orchestra.run.smithi018.stdout:[mds.cephfs.smithi018.acqbsc{-1:34436} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.18:6828/755179764,v1:172.21.15.18:6829/755179764] compat {c=[1],r=[1],i=[1fff]}] 2024-04-12T21:17:24.464 INFO:teuthology.orchestra.run.smithi018.stdout:[mds.cephfs.smithi018.szjluo{-1:34468} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.18:6826/2407992653,v1:172.21.15.18:6827/2407992653] compat {c=[1],r=[1],i=[1fff]}] 2024-04-12T21:17:24.465 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:24.469+0000 7f47b5400700 1 -- 172.21.15.18:0/3066645680 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f479406ec00 msgr2=0x7f47940710b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:17:24.465 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:24.469+0000 7f47b5400700 1 --2- 172.21.15.18:0/3066645680 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f479406ec00 0x7f47940710b0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f479c004f90 tx=0x7f479c009e30 comp rx=0 tx=0).stop 2024-04-12T21:17:24.465 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:24.469+0000 7f47b5400700 1 -- 172.21.15.18:0/3066645680 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f47b0109430 msgr2=0x7f47b0119f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:17:24.465 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:24.469+0000 7f47b5400700 1 --2- 172.21.15.18:0/3066645680 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f47b0109430 0x7f47b0119f80 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f47a0002a50 tx=0x7f47a00046b0 comp rx=0 tx=0).stop 2024-04-12T21:17:24.465 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:24.469+0000 7f47b5400700 1 -- 172.21.15.18:0/3066645680 shutdown_connections 2024-04-12T21:17:24.465 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:24.469+0000 7f47b5400700 1 --2- 172.21.15.18:0/3066645680 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f479406ec00 0x7f47940710b0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:17:24.465 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:24.470+0000 7f47b5400700 1 --2- 172.21.15.18:0/3066645680 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f47b0109430 0x7f47b0119f80 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:17:24.466 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:24.470+0000 7f47b5400700 1 --2- 172.21.15.18:0/3066645680 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f47b0109dd0 0x7f47b011a4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:17:24.466 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:24.470+0000 7f47b5400700 1 -- 172.21.15.18:0/3066645680 >> 172.21.15.18:0/3066645680 conn(0x7f47b0072c00 msgr2=0x7f47b0074c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:17:24.466 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:24.470+0000 7f47b5400700 1 -- 172.21.15.18:0/3066645680 shutdown_connections 2024-04-12T21:17:24.466 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:24.470+0000 7f47b5400700 1 -- 172.21.15.18:0/3066645680 wait complete. 2024-04-12T21:17:24.466 INFO:teuthology.orchestra.run.smithi018.stderr:dumped fsmap epoch 29 2024-04-12T21:17:24.612 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:24.616+0000 7f759a000700 1 -- 172.21.15.18:0/3165717866 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f75941035d0 msgr2=0x7f75940fcc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:17:24.612 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:24.616+0000 7f759a000700 1 --2- 172.21.15.18:0/3165717866 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f75941035d0 0x7f75940fcc40 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f757c004940 tx=0x7f757c009e30 comp rx=0 tx=0).stop 2024-04-12T21:17:24.612 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:24.616+0000 7f759a000700 1 -- 172.21.15.18:0/3165717866 shutdown_connections 2024-04-12T21:17:24.612 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:24.616+0000 7f759a000700 1 --2- 172.21.15.18:0/3165717866 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f75941035d0 0x7f75940fcc40 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:17:24.612 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:24.616+0000 7f759a000700 1 --2- 172.21.15.18:0/3165717866 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7594102c30 0x7f7594103000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:17:24.612 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:24.617+0000 7f759a000700 1 -- 172.21.15.18:0/3165717866 >> 172.21.15.18:0/3165717866 conn(0x7f75940f8760 msgr2=0x7f75940fab70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:17:24.613 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:17:24 smithi018 ceph-mon[112590]: from='client.34948 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:17:24.613 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:17:24 smithi018 ceph-mon[112590]: from='client.34952 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:17:24.613 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:17:24 smithi018 ceph-mon[112590]: from='client.34956 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:17:24.613 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:17:24 smithi018 ceph-mon[112590]: from='client.? 172.21.15.18:0/2590316161' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T21:17:24.614 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:24.617+0000 7f759a000700 1 -- 172.21.15.18:0/3165717866 shutdown_connections 2024-04-12T21:17:24.614 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:24.617+0000 7f759a000700 1 -- 172.21.15.18:0/3165717866 wait complete. 2024-04-12T21:17:24.614 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:24.618+0000 7f759a000700 1 Processor -- start 2024-04-12T21:17:24.614 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:24.618+0000 7f759a000700 1 -- start start 2024-04-12T21:17:24.614 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:24.618+0000 7f759a000700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7594102c30 0x7f75941139d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:17:24.614 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:24.618+0000 7f759a000700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f75941035d0 0x7f7594113f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:17:24.615 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:24.618+0000 7f759a000700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75941145f0 con 0x7f7594102c30 2024-04-12T21:17:24.615 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:24.618+0000 7f759a000700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f759410d950 con 0x7f75941035d0 2024-04-12T21:17:24.615 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:24.619+0000 7f7598c00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7594102c30 0x7f75941139d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:17:24.615 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:24.619+0000 7f7598c00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7594102c30 0x7f75941139d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:59672/0 (socket says 172.21.15.18:59672) 2024-04-12T21:17:24.615 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:24.619+0000 7f7598c00700 1 -- 172.21.15.18:0/3974320195 learned_addr learned my addr 172.21.15.18:0/3974320195 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:17:24.615 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:24.619+0000 7f758be00700 1 --2- 172.21.15.18:0/3974320195 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f75941035d0 0x7f7594113f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:17:24.615 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:24.619+0000 7f7598c00700 1 -- 172.21.15.18:0/3974320195 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f75941035d0 msgr2=0x7f7594113f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:17:24.615 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:24.619+0000 7f7598c00700 1 --2- 172.21.15.18:0/3974320195 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f75941035d0 0x7f7594113f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:17:24.615 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:24.619+0000 7f7598c00700 1 -- 172.21.15.18:0/3974320195 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f757c005040 con 0x7f7594102c30 2024-04-12T21:17:24.616 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:24.620+0000 7f7598c00700 1 --2- 172.21.15.18:0/3974320195 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7594102c30 0x7f75941139d0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f758400b790 tx=0x7f7584009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:17:24.616 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:24.620+0000 7f7591600700 1 -- 172.21.15.18:0/3974320195 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f758400f770 con 0x7f7594102c30 2024-04-12T21:17:24.617 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:24.620+0000 7f7591600700 1 -- 172.21.15.18:0/3974320195 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f758400fda0 con 0x7f7594102c30 2024-04-12T21:17:24.617 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:24.620+0000 7f759a000700 1 -- 172.21.15.18:0/3974320195 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f759410dc30 con 0x7f7594102c30 2024-04-12T21:17:24.617 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:24.620+0000 7f7591600700 1 -- 172.21.15.18:0/3974320195 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f758400d310 con 0x7f7594102c30 2024-04-12T21:17:24.617 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:24.620+0000 7f759a000700 1 -- 172.21.15.18:0/3974320195 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f759410e1f0 con 0x7f7594102c30 2024-04-12T21:17:24.617 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:24.621+0000 7f759a000700 1 -- 172.21.15.18:0/3974320195 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f75940fe380 con 0x7f7594102c30 2024-04-12T21:17:24.618 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:24.623+0000 7f7591600700 1 -- 172.21.15.18:0/3974320195 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7f7584010420 con 0x7f7594102c30 2024-04-12T21:17:24.619 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:24.624+0000 7f7591600700 1 --2- 172.21.15.18:0/3974320195 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f757806ec00 0x7f75780710b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:17:24.619 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:24.624+0000 7f7591600700 1 -- 172.21.15.18:0/3974320195 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 6757+0+0 (secure 0 0 0) 0x7f758408e9b0 con 0x7f7594102c30 2024-04-12T21:17:24.619 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:24.625+0000 7f758be00700 1 --2- 172.21.15.18:0/3974320195 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f757806ec00 0x7f75780710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:17:24.621 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:24.625+0000 7f758be00700 1 --2- 172.21.15.18:0/3974320195 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f757806ec00 0x7f75780710b0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f757c009fd0 tx=0x7f757c009e30 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:17:24.624 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:24.630+0000 7f7591600700 1 -- 172.21.15.18:0/3974320195 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f758405b1e0 con 0x7f7594102c30 2024-04-12T21:17:24.778 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:17:24 smithi186 ceph-mon[86379]: from='client.34948 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:17:24.779 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:17:24 smithi186 ceph-mon[86379]: from='client.34952 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:17:24.779 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:17:24 smithi186 ceph-mon[86379]: from='client.34956 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:17:24.779 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:17:24 smithi186 ceph-mon[86379]: from='client.? 172.21.15.18:0/2590316161' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T21:17:24.868 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:24.873+0000 7f759a000700 1 -- 172.21.15.18:0/3974320195 --> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f759404bed0 con 0x7f757806ec00 2024-04-12T21:17:24.871 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:24.876+0000 7f7591600700 1 -- 172.21.15.18:0/3974320195 <== mgr.34102 v2:172.21.15.18:6800/3812891919 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f759404bed0 con 0x7f757806ec00 2024-04-12T21:17:24.871 INFO:teuthology.orchestra.run.smithi018.stdout:{ 2024-04-12T21:17:24.871 INFO:teuthology.orchestra.run.smithi018.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:f4452def3be4d350f4a8e5feaf7ebb1511be9298fe68478ba7a2d6fbd7d69a63", 2024-04-12T21:17:24.872 INFO:teuthology.orchestra.run.smithi018.stdout: "in_progress": true, 2024-04-12T21:17:24.872 INFO:teuthology.orchestra.run.smithi018.stdout: "which": "Upgrading all daemon types on all hosts", 2024-04-12T21:17:24.872 INFO:teuthology.orchestra.run.smithi018.stdout: "services_complete": [ 2024-04-12T21:17:24.872 INFO:teuthology.orchestra.run.smithi018.stdout: "crash", 2024-04-12T21:17:24.872 INFO:teuthology.orchestra.run.smithi018.stdout: "osd", 2024-04-12T21:17:24.872 INFO:teuthology.orchestra.run.smithi018.stdout: "mon", 2024-04-12T21:17:24.872 INFO:teuthology.orchestra.run.smithi018.stdout: "mgr" 2024-04-12T21:17:24.872 INFO:teuthology.orchestra.run.smithi018.stdout: ], 2024-04-12T21:17:24.872 INFO:teuthology.orchestra.run.smithi018.stdout: "progress": "14/21 daemons upgraded", 2024-04-12T21:17:24.872 INFO:teuthology.orchestra.run.smithi018.stdout: "message": "Currently upgrading mds daemons", 2024-04-12T21:17:24.872 INFO:teuthology.orchestra.run.smithi018.stdout: "is_paused": false 2024-04-12T21:17:24.872 INFO:teuthology.orchestra.run.smithi018.stdout:} 2024-04-12T21:17:24.874 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:24.878+0000 7f759a000700 1 -- 172.21.15.18:0/3974320195 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f757806ec00 msgr2=0x7f75780710b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:17:24.874 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:24.879+0000 7f759a000700 1 --2- 172.21.15.18:0/3974320195 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f757806ec00 0x7f75780710b0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f757c009fd0 tx=0x7f757c009e30 comp rx=0 tx=0).stop 2024-04-12T21:17:24.874 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:24.879+0000 7f759a000700 1 -- 172.21.15.18:0/3974320195 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7594102c30 msgr2=0x7f75941139d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:17:24.874 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:24.879+0000 7f759a000700 1 --2- 172.21.15.18:0/3974320195 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7594102c30 0x7f75941139d0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f758400b790 tx=0x7f7584009250 comp rx=0 tx=0).stop 2024-04-12T21:17:24.874 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:24.879+0000 7f759a000700 1 -- 172.21.15.18:0/3974320195 shutdown_connections 2024-04-12T21:17:24.874 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:24.879+0000 7f759a000700 1 --2- 172.21.15.18:0/3974320195 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f757806ec00 0x7f75780710b0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:17:24.874 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:24.879+0000 7f759a000700 1 --2- 172.21.15.18:0/3974320195 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7594102c30 0x7f75941139d0 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:17:24.874 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:24.879+0000 7f759a000700 1 --2- 172.21.15.18:0/3974320195 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f75941035d0 0x7f7594113f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:17:24.874 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:24.879+0000 7f759a000700 1 -- 172.21.15.18:0/3974320195 >> 172.21.15.18:0/3974320195 conn(0x7f75940f8760 msgr2=0x7f75940f9720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:17:24.875 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:24.879+0000 7f759a000700 1 -- 172.21.15.18:0/3974320195 shutdown_connections 2024-04-12T21:17:24.875 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:24.879+0000 7f759a000700 1 -- 172.21.15.18:0/3974320195 wait complete. 2024-04-12T21:17:25.031 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:25.035+0000 7ffacce00700 1 -- 172.21.15.18:0/3698911937 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ffac8104ff0 msgr2=0x7ffac81053c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:17:25.031 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:25.035+0000 7ffacce00700 1 --2- 172.21.15.18:0/3698911937 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ffac8104ff0 0x7ffac81053c0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7ffab8004940 tx=0x7ffab8009e30 comp rx=0 tx=0).stop 2024-04-12T21:17:25.031 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:25.036+0000 7ffacce00700 1 -- 172.21.15.18:0/3698911937 shutdown_connections 2024-04-12T21:17:25.031 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:25.036+0000 7ffacce00700 1 --2- 172.21.15.18:0/3698911937 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ffac8105990 0x7ffac80ff000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:17:25.031 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:25.036+0000 7ffacce00700 1 --2- 172.21.15.18:0/3698911937 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ffac8104ff0 0x7ffac81053c0 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:17:25.032 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:25.036+0000 7ffacce00700 1 -- 172.21.15.18:0/3698911937 >> 172.21.15.18:0/3698911937 conn(0x7ffac80faba0 msgr2=0x7ffac80fcfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:17:25.032 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:25.036+0000 7ffacce00700 1 -- 172.21.15.18:0/3698911937 shutdown_connections 2024-04-12T21:17:25.032 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:25.036+0000 7ffacce00700 1 -- 172.21.15.18:0/3698911937 wait complete. 2024-04-12T21:17:25.033 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:25.037+0000 7ffacce00700 1 Processor -- start 2024-04-12T21:17:25.033 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:25.037+0000 7ffacce00700 1 -- start start 2024-04-12T21:17:25.033 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:25.037+0000 7ffacce00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ffac8104ff0 0x7ffac8115d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:17:25.033 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:25.037+0000 7ffacce00700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ffac8105990 0x7ffac81162b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:17:25.033 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:25.037+0000 7ffacce00700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffac8116900 con 0x7ffac8104ff0 2024-04-12T21:17:25.033 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:25.037+0000 7ffacce00700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffac810fcd0 con 0x7ffac8105990 2024-04-12T21:17:25.033 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:25.038+0000 7ffac5e00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ffac8104ff0 0x7ffac8115d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:17:25.033 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:25.038+0000 7ffac5e00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ffac8104ff0 0x7ffac8115d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:59692/0 (socket says 172.21.15.18:59692) 2024-04-12T21:17:25.033 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:25.038+0000 7ffac5e00700 1 -- 172.21.15.18:0/3995770388 learned_addr learned my addr 172.21.15.18:0/3995770388 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:17:25.034 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:25.038+0000 7ffac5e00700 1 -- 172.21.15.18:0/3995770388 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ffac8105990 msgr2=0x7ffac81162b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:17:25.035 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:25.038+0000 7ffabfe00700 1 --2- 172.21.15.18:0/3995770388 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ffac8105990 0x7ffac81162b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:17:25.035 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:25.038+0000 7ffac5e00700 1 --2- 172.21.15.18:0/3995770388 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ffac8105990 0x7ffac81162b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:17:25.035 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:25.038+0000 7ffac5e00700 1 -- 172.21.15.18:0/3995770388 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffab8005040 con 0x7ffac8104ff0 2024-04-12T21:17:25.035 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:25.039+0000 7ffac5e00700 1 --2- 172.21.15.18:0/3995770388 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ffac8104ff0 0x7ffac8115d70 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7ffab8009180 tx=0x7ffab8009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:17:25.035 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:25.039+0000 7ffabe000700 1 -- 172.21.15.18:0/3995770388 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ffab800e040 con 0x7ffac8104ff0 2024-04-12T21:17:25.035 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:25.039+0000 7ffabe000700 1 -- 172.21.15.18:0/3995770388 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ffab800bbc0 con 0x7ffac8104ff0 2024-04-12T21:17:25.035 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:25.039+0000 7ffacce00700 1 -- 172.21.15.18:0/3995770388 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffac810ff50 con 0x7ffac8104ff0 2024-04-12T21:17:25.035 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:25.039+0000 7ffabe000700 1 -- 172.21.15.18:0/3995770388 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ffab801c630 con 0x7ffac8104ff0 2024-04-12T21:17:25.035 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:25.039+0000 7ffacce00700 1 -- 172.21.15.18:0/3995770388 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffac8110510 con 0x7ffac8104ff0 2024-04-12T21:17:25.036 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:25.040+0000 7ffacce00700 1 -- 172.21.15.18:0/3995770388 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffac8100740 con 0x7ffac8104ff0 2024-04-12T21:17:25.036 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:25.041+0000 7ffabe000700 1 -- 172.21.15.18:0/3995770388 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7ffab800bd30 con 0x7ffac8104ff0 2024-04-12T21:17:25.037 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:25.042+0000 7ffabe000700 1 --2- 172.21.15.18:0/3995770388 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7ffaac06ec10 0x7ffaac0710c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:17:25.038 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:25.043+0000 7ffabe000700 1 -- 172.21.15.18:0/3995770388 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 6757+0+0 (secure 0 0 0) 0x7ffab808fcb0 con 0x7ffac8104ff0 2024-04-12T21:17:25.038 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:25.043+0000 7ffabfe00700 1 --2- 172.21.15.18:0/3995770388 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7ffaac06ec10 0x7ffaac0710c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:17:25.038 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:25.043+0000 7ffabfe00700 1 --2- 172.21.15.18:0/3995770388 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7ffaac06ec10 0x7ffaac0710c0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7ffac8111610 tx=0x7ffab0009040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:17:25.042 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:25.048+0000 7ffabe000700 1 -- 172.21.15.18:0/3995770388 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7ffab805c5a0 con 0x7ffac8104ff0 2024-04-12T21:17:25.356 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:25.361+0000 7ffacce00700 1 -- 172.21.15.18:0/3995770388 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7ffac804bed0 con 0x7ffac8104ff0 2024-04-12T21:17:25.357 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:25.362+0000 7ffabe000700 1 -- 172.21.15.18:0/3995770388 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+233 (secure 0 0 0) 0x7ffab805c130 con 0x7ffac8104ff0 2024-04-12T21:17:25.357 INFO:teuthology.orchestra.run.smithi018.stdout:HEALTH_ERR 1 filesystem is degraded; 1 filesystem is offline 2024-04-12T21:17:25.357 INFO:teuthology.orchestra.run.smithi018.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-04-12T21:17:25.357 INFO:teuthology.orchestra.run.smithi018.stdout: fs cephfs is degraded 2024-04-12T21:17:25.357 INFO:teuthology.orchestra.run.smithi018.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-04-12T21:17:25.358 INFO:teuthology.orchestra.run.smithi018.stdout: fs cephfs is offline because no MDS is active for it. 2024-04-12T21:17:25.360 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:25.364+0000 7ffacce00700 1 -- 172.21.15.18:0/3995770388 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7ffaac06ec10 msgr2=0x7ffaac0710c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:17:25.360 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:25.364+0000 7ffacce00700 1 --2- 172.21.15.18:0/3995770388 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7ffaac06ec10 0x7ffaac0710c0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7ffac8111610 tx=0x7ffab0009040 comp rx=0 tx=0).stop 2024-04-12T21:17:25.361 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:25.364+0000 7ffacce00700 1 -- 172.21.15.18:0/3995770388 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ffac8104ff0 msgr2=0x7ffac8115d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:17:25.361 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:25.364+0000 7ffacce00700 1 --2- 172.21.15.18:0/3995770388 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ffac8104ff0 0x7ffac8115d70 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7ffab8009180 tx=0x7ffab8009e30 comp rx=0 tx=0).stop 2024-04-12T21:17:25.361 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:25.365+0000 7ffacce00700 1 -- 172.21.15.18:0/3995770388 shutdown_connections 2024-04-12T21:17:25.361 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:25.365+0000 7ffacce00700 1 --2- 172.21.15.18:0/3995770388 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7ffaac06ec10 0x7ffaac0710c0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:17:25.361 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:25.365+0000 7ffacce00700 1 --2- 172.21.15.18:0/3995770388 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ffac8104ff0 0x7ffac8115d70 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:17:25.361 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:25.365+0000 7ffacce00700 1 --2- 172.21.15.18:0/3995770388 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ffac8105990 0x7ffac81162b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:17:25.362 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:25.365+0000 7ffacce00700 1 -- 172.21.15.18:0/3995770388 >> 172.21.15.18:0/3995770388 conn(0x7ffac80faba0 msgr2=0x7ffac80fbbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:17:25.362 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:25.365+0000 7ffacce00700 1 -- 172.21.15.18:0/3995770388 shutdown_connections 2024-04-12T21:17:25.362 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:25.365+0000 7ffacce00700 1 -- 172.21.15.18:0/3995770388 wait complete. 2024-04-12T21:17:25.778 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:17:25 smithi186 ceph-mon[86379]: pgmap v528: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:17:25.779 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:17:25 smithi186 ceph-mon[86379]: from='client.? 172.21.15.18:0/3066645680' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-12T21:17:25.779 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:17:25 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T21:17:25.779 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:17:25 smithi186 ceph-mon[86379]: from='client.? 172.21.15.18:0/3995770388' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-12T21:17:25.850 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:17:25 smithi018 ceph-mon[112590]: pgmap v528: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:17:25.850 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:17:25 smithi018 ceph-mon[112590]: from='client.? 172.21.15.18:0/3066645680' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-12T21:17:25.850 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:17:25 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T21:17:25.850 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:17:25 smithi018 ceph-mon[112590]: from='client.? 172.21.15.18:0/3995770388' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-12T21:17:26.778 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:17:26 smithi186 ceph-mon[86379]: from='client.34968 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:17:26.849 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:17:26 smithi018 ceph-mon[112590]: from='client.34968 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:17:27.778 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:17:27 smithi186 ceph-mon[86379]: pgmap v529: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:17:27.850 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:17:27 smithi018 ceph-mon[112590]: pgmap v529: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:17:29.778 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:17:29 smithi186 ceph-mon[86379]: pgmap v530: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:17:29.849 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:17:29 smithi018 ceph-mon[112590]: pgmap v530: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:17:31.778 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:17:31 smithi186 ceph-mon[86379]: pgmap v531: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:17:31.849 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:17:31 smithi018 ceph-mon[112590]: pgmap v531: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:17:33.778 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:17:33 smithi186 ceph-mon[86379]: pgmap v532: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:17:33.850 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:17:33 smithi018 ceph-mon[112590]: pgmap v532: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:17:35.778 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:17:35 smithi186 ceph-mon[86379]: pgmap v533: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:17:35.849 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:17:35 smithi018 ceph-mon[112590]: pgmap v533: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:17:37.778 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:17:37 smithi186 ceph-mon[86379]: pgmap v534: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:17:37.849 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:17:37 smithi018 ceph-mon[112590]: pgmap v534: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:17:39.778 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:17:39 smithi186 ceph-mon[86379]: pgmap v535: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:17:39.849 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:17:39 smithi018 ceph-mon[112590]: pgmap v535: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:17:40.751 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:17:40 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T21:17:40.778 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:17:40 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T21:17:41.778 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:17:41 smithi186 ceph-mon[86379]: pgmap v536: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:17:41.849 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:17:41 smithi018 ceph-mon[112590]: pgmap v536: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:17:43.778 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:17:43 smithi186 ceph-mon[86379]: pgmap v537: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:17:43.849 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:17:43 smithi018 ceph-mon[112590]: pgmap v537: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:17:45.778 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:17:45 smithi186 ceph-mon[86379]: pgmap v538: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:17:45.849 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:17:45 smithi018 ceph-mon[112590]: pgmap v538: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:17:47.778 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:17:47 smithi186 ceph-mon[86379]: pgmap v539: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:17:47.849 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:17:47 smithi018 ceph-mon[112590]: pgmap v539: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:17:49.778 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:17:49 smithi186 ceph-mon[86379]: pgmap v540: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:17:49.849 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:17:49 smithi018 ceph-mon[112590]: pgmap v540: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:17:51.778 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:17:51 smithi186 ceph-mon[86379]: pgmap v541: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:17:51.849 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:17:51 smithi018 ceph-mon[112590]: pgmap v541: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:17:53.778 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:17:53 smithi186 ceph-mon[86379]: pgmap v542: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:17:53.849 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:17:53 smithi018 ceph-mon[112590]: pgmap v542: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:17:55.543 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:55.547+0000 7fe9d3200700 1 -- 172.21.15.18:0/3526108374 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe9cc0fe600 msgr2=0x7fe9cc106c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:17:55.544 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:55.547+0000 7fe9d3200700 1 --2- 172.21.15.18:0/3526108374 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe9cc0fe600 0x7fe9cc106c40 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7fe9bc004940 tx=0x7fe9bc009e30 comp rx=0 tx=0).stop 2024-04-12T21:17:55.544 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:55.547+0000 7fe9d3200700 1 -- 172.21.15.18:0/3526108374 shutdown_connections 2024-04-12T21:17:55.544 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:55.547+0000 7fe9d3200700 1 --2- 172.21.15.18:0/3526108374 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe9cc0fe600 0x7fe9cc106c40 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:17:55.544 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:55.547+0000 7fe9d3200700 1 --2- 172.21.15.18:0/3526108374 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe9cc0fdc60 0x7fe9cc0fe030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:17:55.544 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:55.547+0000 7fe9d3200700 1 -- 172.21.15.18:0/3526108374 >> 172.21.15.18:0/3526108374 conn(0x7fe9cc0f76b0 msgr2=0x7fe9cc0f9ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:17:55.544 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:55.547+0000 7fe9d3200700 1 -- 172.21.15.18:0/3526108374 shutdown_connections 2024-04-12T21:17:55.544 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:55.547+0000 7fe9d3200700 1 -- 172.21.15.18:0/3526108374 wait complete. 2024-04-12T21:17:55.545 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:55.548+0000 7fe9d3200700 1 Processor -- start 2024-04-12T21:17:55.545 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:55.548+0000 7fe9d3200700 1 -- start start 2024-04-12T21:17:55.545 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:55.549+0000 7fe9d3200700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe9cc0fdc60 0x7fe9cc115b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:17:55.545 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:55.549+0000 7fe9d3200700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe9cc0fe600 0x7fe9cc1160d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:17:55.545 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:55.549+0000 7fe9d3200700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9cc116720 con 0x7fe9cc0fe600 2024-04-12T21:17:55.545 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:55.549+0000 7fe9d3200700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9cc10fb10 con 0x7fe9cc0fdc60 2024-04-12T21:17:55.546 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:55.549+0000 7fe9d1400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe9cc0fe600 0x7fe9cc1160d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:17:55.546 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:55.549+0000 7fe9d1400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe9cc0fe600 0x7fe9cc1160d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:37506/0 (socket says 172.21.15.18:37506) 2024-04-12T21:17:55.546 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:55.549+0000 7fe9d1400700 1 -- 172.21.15.18:0/1519602777 learned_addr learned my addr 172.21.15.18:0/1519602777 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:17:55.546 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:55.550+0000 7fe9d1e00700 1 --2- 172.21.15.18:0/1519602777 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe9cc0fdc60 0x7fe9cc115b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:17:55.546 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:55.550+0000 7fe9d1400700 1 -- 172.21.15.18:0/1519602777 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe9cc0fdc60 msgr2=0x7fe9cc115b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:17:55.547 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:55.550+0000 7fe9d1400700 1 --2- 172.21.15.18:0/1519602777 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe9cc0fdc60 0x7fe9cc115b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:17:55.547 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:55.550+0000 7fe9d1400700 1 -- 172.21.15.18:0/1519602777 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9bc005040 con 0x7fe9cc0fe600 2024-04-12T21:17:55.547 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:55.550+0000 7fe9d1e00700 1 --2- 172.21.15.18:0/1519602777 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe9cc0fdc60 0x7fe9cc115b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-12T21:17:55.547 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:55.550+0000 7fe9d1400700 1 --2- 172.21.15.18:0/1519602777 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe9cc0fe600 0x7fe9cc1160d0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7fe9bc009180 tx=0x7fe9bc009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:17:55.547 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:55.550+0000 7fe9c6000700 1 -- 172.21.15.18:0/1519602777 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe9bc00e040 con 0x7fe9cc0fe600 2024-04-12T21:17:55.547 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:55.550+0000 7fe9d3200700 1 -- 172.21.15.18:0/1519602777 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe9cc10fd90 con 0x7fe9cc0fe600 2024-04-12T21:17:55.547 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:55.550+0000 7fe9c6000700 1 -- 172.21.15.18:0/1519602777 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe9bc013420 con 0x7fe9cc0fe600 2024-04-12T21:17:55.547 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:55.550+0000 7fe9c6000700 1 -- 172.21.15.18:0/1519602777 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe9bc01c630 con 0x7fe9cc0fe600 2024-04-12T21:17:55.547 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:55.551+0000 7fe9d3200700 1 -- 172.21.15.18:0/1519602777 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe9cc1102d0 con 0x7fe9cc0fe600 2024-04-12T21:17:55.548 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:55.552+0000 7fe9d3200700 1 -- 172.21.15.18:0/1519602777 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe9cc0668d0 con 0x7fe9cc0fe600 2024-04-12T21:17:55.548 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:55.553+0000 7fe9c6000700 1 -- 172.21.15.18:0/1519602777 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7fe9bc013590 con 0x7fe9cc0fe600 2024-04-12T21:17:55.550 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:55.554+0000 7fe9c6000700 1 --2- 172.21.15.18:0/1519602777 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fe9b406ec00 0x7fe9b40710b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:17:55.550 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:55.554+0000 7fe9c6000700 1 -- 172.21.15.18:0/1519602777 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 6757+0+0 (secure 0 0 0) 0x7fe9bc08fc90 con 0x7fe9cc0fe600 2024-04-12T21:17:55.550 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:55.554+0000 7fe9d1e00700 1 --2- 172.21.15.18:0/1519602777 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fe9b406ec00 0x7fe9b40710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:17:55.550 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:55.555+0000 7fe9d1e00700 1 --2- 172.21.15.18:0/1519602777 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fe9b406ec00 0x7fe9b40710b0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fe9c0007fd0 tx=0x7fe9c0009040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:17:55.555 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:55.559+0000 7fe9c6000700 1 -- 172.21.15.18:0/1519602777 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7fe9bc05c580 con 0x7fe9cc0fe600 2024-04-12T21:17:55.778 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:17:55 smithi186 ceph-mon[86379]: pgmap v543: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:17:55.778 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:17:55 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T21:17:55.806 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:55.810+0000 7fe9d3200700 1 -- 172.21.15.18:0/1519602777 --> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fe9cc0668d0 con 0x7fe9b406ec00 2024-04-12T21:17:55.806 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:17:55 smithi018 ceph-mon[112590]: pgmap v543: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:17:55.806 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:17:55 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T21:17:55.808 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:55.812+0000 7fe9c6000700 1 -- 172.21.15.18:0/1519602777 <== mgr.34102 v2:172.21.15.18:6800/3812891919 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7fe9cc0668d0 con 0x7fe9b406ec00 2024-04-12T21:17:55.811 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:55.815+0000 7fe9d3200700 1 -- 172.21.15.18:0/1519602777 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fe9b406ec00 msgr2=0x7fe9b40710b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:17:55.811 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:55.815+0000 7fe9d3200700 1 --2- 172.21.15.18:0/1519602777 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fe9b406ec00 0x7fe9b40710b0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fe9c0007fd0 tx=0x7fe9c0009040 comp rx=0 tx=0).stop 2024-04-12T21:17:55.811 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:55.815+0000 7fe9d3200700 1 -- 172.21.15.18:0/1519602777 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe9cc0fe600 msgr2=0x7fe9cc1160d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:17:55.811 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:55.815+0000 7fe9d3200700 1 --2- 172.21.15.18:0/1519602777 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe9cc0fe600 0x7fe9cc1160d0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7fe9bc009180 tx=0x7fe9bc009e30 comp rx=0 tx=0).stop 2024-04-12T21:17:55.811 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:55.815+0000 7fe9d3200700 1 -- 172.21.15.18:0/1519602777 shutdown_connections 2024-04-12T21:17:55.812 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:55.815+0000 7fe9d3200700 1 --2- 172.21.15.18:0/1519602777 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fe9b406ec00 0x7fe9b40710b0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:17:55.812 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:55.815+0000 7fe9d3200700 1 --2- 172.21.15.18:0/1519602777 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe9cc0fdc60 0x7fe9cc115b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:17:55.812 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:55.815+0000 7fe9d3200700 1 --2- 172.21.15.18:0/1519602777 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe9cc0fe600 0x7fe9cc1160d0 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:17:55.812 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:55.815+0000 7fe9d3200700 1 -- 172.21.15.18:0/1519602777 >> 172.21.15.18:0/1519602777 conn(0x7fe9cc0f76b0 msgr2=0x7fe9cc0f8b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:17:55.812 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:55.815+0000 7fe9d3200700 1 -- 172.21.15.18:0/1519602777 shutdown_connections 2024-04-12T21:17:55.812 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:55.815+0000 7fe9d3200700 1 -- 172.21.15.18:0/1519602777 wait complete. 2024-04-12T21:17:55.828 INFO:teuthology.orchestra.run.smithi018.stdout:true 2024-04-12T21:17:55.970 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:55.973+0000 7f272ce00700 1 -- 172.21.15.18:0/1248602291 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f27281059c0 msgr2=0x7f27280ff080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:17:55.970 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:55.973+0000 7f272ce00700 1 --2- 172.21.15.18:0/1248602291 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f27281059c0 0x7f27280ff080 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f2714004940 tx=0x7f2714009e30 comp rx=0 tx=0).stop 2024-04-12T21:17:55.970 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:55.973+0000 7f272ce00700 1 -- 172.21.15.18:0/1248602291 shutdown_connections 2024-04-12T21:17:55.970 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:55.973+0000 7f272ce00700 1 --2- 172.21.15.18:0/1248602291 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f27281059c0 0x7f27280ff080 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:17:55.970 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:55.973+0000 7f272ce00700 1 --2- 172.21.15.18:0/1248602291 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2728105020 0x7f27281053f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:17:55.970 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:55.973+0000 7f272ce00700 1 -- 172.21.15.18:0/1248602291 >> 172.21.15.18:0/1248602291 conn(0x7f27280faba0 msgr2=0x7f27280fcfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:17:55.970 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:55.974+0000 7f272ce00700 1 -- 172.21.15.18:0/1248602291 shutdown_connections 2024-04-12T21:17:55.970 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:55.974+0000 7f272ce00700 1 -- 172.21.15.18:0/1248602291 wait complete. 2024-04-12T21:17:55.971 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:55.975+0000 7f272ce00700 1 Processor -- start 2024-04-12T21:17:55.971 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:55.975+0000 7f272ce00700 1 -- start start 2024-04-12T21:17:55.971 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:55.975+0000 7f272ce00700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2728105020 0x7f2728115ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:17:55.972 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:55.976+0000 7f272ce00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f27281059c0 0x7f27281160e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:17:55.972 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:55.976+0000 7f272ce00700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2728116730 con 0x7f27281059c0 2024-04-12T21:17:55.972 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:55.976+0000 7f272ce00700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f272810fd10 con 0x7f2728105020 2024-04-12T21:17:55.973 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:55.976+0000 7f2725400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f27281059c0 0x7f27281160e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:17:55.973 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:55.976+0000 7f2725400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f27281059c0 0x7f27281160e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:37530/0 (socket says 172.21.15.18:37530) 2024-04-12T21:17:55.973 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:55.976+0000 7f2725400700 1 -- 172.21.15.18:0/3926584547 learned_addr learned my addr 172.21.15.18:0/3926584547 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:17:55.973 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:55.976+0000 7f2725e00700 1 --2- 172.21.15.18:0/3926584547 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2728105020 0x7f2728115ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:17:55.973 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:55.977+0000 7f2725400700 1 -- 172.21.15.18:0/3926584547 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2728105020 msgr2=0x7f2728115ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:17:55.973 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:55.977+0000 7f2725400700 1 --2- 172.21.15.18:0/3926584547 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2728105020 0x7f2728115ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:17:55.973 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:55.977+0000 7f2725400700 1 -- 172.21.15.18:0/3926584547 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2714005040 con 0x7f27281059c0 2024-04-12T21:17:55.973 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:55.977+0000 7f2725400700 1 --2- 172.21.15.18:0/3926584547 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f27281059c0 0x7f27281160e0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f2714009100 tx=0x7f2714009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:17:55.974 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:55.977+0000 7f271e000700 1 -- 172.21.15.18:0/3926584547 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f271400e040 con 0x7f27281059c0 2024-04-12T21:17:55.974 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:55.977+0000 7f271e000700 1 -- 172.21.15.18:0/3926584547 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f2714013420 con 0x7f27281059c0 2024-04-12T21:17:55.974 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:55.977+0000 7f271e000700 1 -- 172.21.15.18:0/3926584547 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f271401c630 con 0x7f27281059c0 2024-04-12T21:17:55.974 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:55.977+0000 7f272ce00700 1 -- 172.21.15.18:0/3926584547 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f272810ff90 con 0x7f27281059c0 2024-04-12T21:17:55.974 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:55.977+0000 7f272ce00700 1 -- 172.21.15.18:0/3926584547 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2728110550 con 0x7f27281059c0 2024-04-12T21:17:55.974 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:55.978+0000 7f272ce00700 1 -- 172.21.15.18:0/3926584547 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2728100770 con 0x7f27281059c0 2024-04-12T21:17:55.975 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:55.980+0000 7f271e000700 1 -- 172.21.15.18:0/3926584547 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7f2714013590 con 0x7f27281059c0 2024-04-12T21:17:55.976 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:55.980+0000 7f271e000700 1 --2- 172.21.15.18:0/3926584547 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f270c06ec00 0x7f270c0710b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:17:55.977 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:55.981+0000 7f271e000700 1 -- 172.21.15.18:0/3926584547 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 6757+0+0 (secure 0 0 0) 0x7f271408fae0 con 0x7f27281059c0 2024-04-12T21:17:55.977 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:55.981+0000 7f2725e00700 1 --2- 172.21.15.18:0/3926584547 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f270c06ec00 0x7f270c0710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:17:55.977 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:55.982+0000 7f2725e00700 1 --2- 172.21.15.18:0/3926584547 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f270c06ec00 0x7f270c0710b0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f2718009fd0 tx=0x7f2718009f20 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:17:55.981 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:55.985+0000 7f271e000700 1 -- 172.21.15.18:0/3926584547 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f271405c3d0 con 0x7f27281059c0 2024-04-12T21:17:56.231 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:56.235+0000 7f272ce00700 1 -- 172.21.15.18:0/3926584547 --> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f272810c190 con 0x7f270c06ec00 2024-04-12T21:17:56.234 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:56.238+0000 7f271e000700 1 -- 172.21.15.18:0/3926584547 <== mgr.34102 v2:172.21.15.18:6800/3812891919 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f272810c190 con 0x7f270c06ec00 2024-04-12T21:17:56.238 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:56.241+0000 7f272ce00700 1 -- 172.21.15.18:0/3926584547 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f270c06ec00 msgr2=0x7f270c0710b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:17:56.238 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:56.241+0000 7f272ce00700 1 --2- 172.21.15.18:0/3926584547 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f270c06ec00 0x7f270c0710b0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f2718009fd0 tx=0x7f2718009f20 comp rx=0 tx=0).stop 2024-04-12T21:17:56.238 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:56.241+0000 7f272ce00700 1 -- 172.21.15.18:0/3926584547 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f27281059c0 msgr2=0x7f27281160e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:17:56.238 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:56.241+0000 7f272ce00700 1 --2- 172.21.15.18:0/3926584547 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f27281059c0 0x7f27281160e0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f2714009100 tx=0x7f2714009e30 comp rx=0 tx=0).stop 2024-04-12T21:17:56.238 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:56.241+0000 7f272ce00700 1 -- 172.21.15.18:0/3926584547 shutdown_connections 2024-04-12T21:17:56.238 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:56.241+0000 7f272ce00700 1 --2- 172.21.15.18:0/3926584547 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f270c06ec00 0x7f270c0710b0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:17:56.238 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:56.241+0000 7f272ce00700 1 --2- 172.21.15.18:0/3926584547 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2728105020 0x7f2728115ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:17:56.238 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:56.241+0000 7f272ce00700 1 --2- 172.21.15.18:0/3926584547 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f27281059c0 0x7f27281160e0 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:17:56.238 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:56.241+0000 7f272ce00700 1 -- 172.21.15.18:0/3926584547 >> 172.21.15.18:0/3926584547 conn(0x7f27280faba0 msgr2=0x7f27280fbac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:17:56.238 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:56.241+0000 7f272ce00700 1 -- 172.21.15.18:0/3926584547 shutdown_connections 2024-04-12T21:17:56.238 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:56.241+0000 7f272ce00700 1 -- 172.21.15.18:0/3926584547 wait complete. 2024-04-12T21:17:56.394 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:56.398+0000 7fd1ad600700 1 -- 172.21.15.18:0/3112394840 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd1a80702e0 msgr2=0x7fd1a80706b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:17:56.395 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:56.398+0000 7fd1ad600700 1 --2- 172.21.15.18:0/3112394840 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd1a80702e0 0x7fd1a80706b0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7fd198004940 tx=0x7fd198009e30 comp rx=0 tx=0).stop 2024-04-12T21:17:56.395 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:56.398+0000 7fd1ad600700 1 -- 172.21.15.18:0/3112394840 shutdown_connections 2024-04-12T21:17:56.395 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:56.398+0000 7fd1ad600700 1 --2- 172.21.15.18:0/3112394840 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd1a8070bf0 0x7fd1a8109190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:17:56.395 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:56.398+0000 7fd1ad600700 1 --2- 172.21.15.18:0/3112394840 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd1a80702e0 0x7fd1a80706b0 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:17:56.395 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:56.398+0000 7fd1ad600700 1 -- 172.21.15.18:0/3112394840 >> 172.21.15.18:0/3112394840 conn(0x7fd1a80f89b0 msgr2=0x7fd1a80fadc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:17:56.395 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:56.398+0000 7fd1ad600700 1 -- 172.21.15.18:0/3112394840 shutdown_connections 2024-04-12T21:17:56.395 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:56.398+0000 7fd1ad600700 1 -- 172.21.15.18:0/3112394840 wait complete. 2024-04-12T21:17:56.396 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:56.399+0000 7fd1ad600700 1 Processor -- start 2024-04-12T21:17:56.396 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:56.400+0000 7fd1ad600700 1 -- start start 2024-04-12T21:17:56.396 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:56.400+0000 7fd1ad600700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd1a80702e0 0x7fd1a8115bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:17:56.396 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:56.400+0000 7fd1ad600700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd1a8070bf0 0x7fd1a81160f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:17:56.396 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:56.400+0000 7fd1ad600700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1a8116740 con 0x7fd1a8070bf0 2024-04-12T21:17:56.396 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:56.400+0000 7fd1ad600700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1a810fd20 con 0x7fd1a80702e0 2024-04-12T21:17:56.397 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:56.400+0000 7fd1a6000700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd1a8070bf0 0x7fd1a81160f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:17:56.397 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:56.400+0000 7fd1a6000700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd1a8070bf0 0x7fd1a81160f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:37552/0 (socket says 172.21.15.18:37552) 2024-04-12T21:17:56.397 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:56.400+0000 7fd1a6000700 1 -- 172.21.15.18:0/2931633544 learned_addr learned my addr 172.21.15.18:0/2931633544 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:17:56.397 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:56.401+0000 7fd1a6a00700 1 --2- 172.21.15.18:0/2931633544 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd1a80702e0 0x7fd1a8115bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:17:56.397 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:56.401+0000 7fd1a6000700 1 -- 172.21.15.18:0/2931633544 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd1a80702e0 msgr2=0x7fd1a8115bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:17:56.398 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:56.401+0000 7fd1a6000700 1 --2- 172.21.15.18:0/2931633544 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd1a80702e0 0x7fd1a8115bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:17:56.398 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:56.401+0000 7fd1a6000700 1 -- 172.21.15.18:0/2931633544 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd198005040 con 0x7fd1a8070bf0 2024-04-12T21:17:56.398 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:56.401+0000 7fd1a6000700 1 --2- 172.21.15.18:0/2931633544 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd1a8070bf0 0x7fd1a81160f0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7fd19400c830 tx=0x7fd194004470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:17:56.398 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:56.402+0000 7fd19ea00700 1 -- 172.21.15.18:0/2931633544 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd19400a760 con 0x7fd1a8070bf0 2024-04-12T21:17:56.398 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:56.402+0000 7fd19ea00700 1 -- 172.21.15.18:0/2931633544 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd19400ad90 con 0x7fd1a8070bf0 2024-04-12T21:17:56.398 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:56.402+0000 7fd1ad600700 1 -- 172.21.15.18:0/2931633544 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd1a8110000 con 0x7fd1a8070bf0 2024-04-12T21:17:56.399 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:56.402+0000 7fd19ea00700 1 -- 172.21.15.18:0/2931633544 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd1940113a0 con 0x7fd1a8070bf0 2024-04-12T21:17:56.399 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:56.402+0000 7fd1ad600700 1 -- 172.21.15.18:0/2931633544 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd1a8110540 con 0x7fd1a8070bf0 2024-04-12T21:17:56.399 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:56.403+0000 7fd1ad600700 1 -- 172.21.15.18:0/2931633544 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd1a8071a20 con 0x7fd1a8070bf0 2024-04-12T21:17:56.400 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:56.404+0000 7fd19ea00700 1 -- 172.21.15.18:0/2931633544 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7fd194012400 con 0x7fd1a8070bf0 2024-04-12T21:17:56.401 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:56.405+0000 7fd19ea00700 1 --2- 172.21.15.18:0/2931633544 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fd18c06ec00 0x7fd18c0710b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:17:56.401 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:56.406+0000 7fd19ea00700 1 -- 172.21.15.18:0/2931633544 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 6757+0+0 (secure 0 0 0) 0x7fd19408eaa0 con 0x7fd1a8070bf0 2024-04-12T21:17:56.402 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:56.406+0000 7fd1a6a00700 1 --2- 172.21.15.18:0/2931633544 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fd18c06ec00 0x7fd18c0710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:17:56.402 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:56.406+0000 7fd1a6a00700 1 --2- 172.21.15.18:0/2931633544 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fd18c06ec00 0x7fd18c0710b0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fd19800b020 tx=0x7fd198009e30 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:17:56.406 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:56.411+0000 7fd19ea00700 1 -- 172.21.15.18:0/2931633544 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7fd19405b250 con 0x7fd1a8070bf0 2024-04-12T21:17:56.642 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:56.646+0000 7fd1ad600700 1 -- 172.21.15.18:0/2931633544 --> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fd1a810c180 con 0x7fd18c06ec00 2024-04-12T21:17:56.660 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:56.664+0000 7fd19ea00700 1 -- 172.21.15.18:0/2931633544 <== mgr.34102 v2:172.21.15.18:6800/3812891919 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7fd19800b5c0 con 0x7fd18c06ec00 2024-04-12T21:17:56.661 INFO:teuthology.orchestra.run.smithi018.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-12T21:17:56.661 INFO:teuthology.orchestra.run.smithi018.stdout:alertmanager.smithi018 smithi018 *:9093,9094 running (18m) 10m ago 33m 14.7M - 0.25.0 c8568f914cd2 c332090068e8 2024-04-12T21:17:56.661 INFO:teuthology.orchestra.run.smithi018.stdout:crash.smithi018 smithi018 running (14m) 10m ago 33m 6665k - 19.0.0-2879-g324baf8d 8717720dc5e5 b7f80d1f7982 2024-04-12T21:17:56.661 INFO:teuthology.orchestra.run.smithi018.stdout:crash.smithi186 smithi186 running (14m) 11m ago 31m 6660k - 19.0.0-2879-g324baf8d 8717720dc5e5 a4f55c81b623 2024-04-12T21:17:56.661 INFO:teuthology.orchestra.run.smithi018.stdout:grafana.smithi018 smithi018 *:3000 running (17m) 10m ago 32m 74.8M - 9.4.12 2bacad6d85d8 b08e3bf176dd 2024-04-12T21:17:56.661 INFO:teuthology.orchestra.run.smithi018.stdout:mds.cephfs.smithi018.acqbsc smithi018 running (10m) 10m ago 25m 11.1M - 19.0.0-2879-g324baf8d 8717720dc5e5 021f99ab24ae 2024-04-12T21:17:56.661 INFO:teuthology.orchestra.run.smithi018.stdout:mds.cephfs.smithi018.szjluo smithi018 running (10m) 10m ago 25m 10.9M - 19.0.0-2879-g324baf8d 8717720dc5e5 0003117479d4 2024-04-12T21:17:56.661 INFO:teuthology.orchestra.run.smithi018.stdout:mds.cephfs.smithi186.jdllyn smithi186 running (25m) 11m ago 25m 15.6M - 17.2.7-827-gd3b5060a 21de9b7369ad 2b2bc46bd29a 2024-04-12T21:17:56.661 INFO:teuthology.orchestra.run.smithi018.stdout:mds.cephfs.smithi186.wjnafs smithi186 running (25m) 11m ago 25m 15.4M - 17.2.7-827-gd3b5060a 21de9b7369ad 74b115d6d6fd 2024-04-12T21:17:56.661 INFO:teuthology.orchestra.run.smithi018.stdout:mgr.smithi018.xjzwga smithi018 *:8443,9283,8765 running (21m) 10m ago 34m 509M - 19.0.0-2879-g324baf8d 8717720dc5e5 56088305eab7 2024-04-12T21:17:56.661 INFO:teuthology.orchestra.run.smithi018.stdout:mgr.smithi186.kqcfxk smithi186 *:8443,9283,8765 running (19m) 11m ago 31m 452M - 19.0.0-2879-g324baf8d 8717720dc5e5 09cd766e5e0d 2024-04-12T21:17:56.661 INFO:teuthology.orchestra.run.smithi018.stdout:mon.smithi018 smithi018 running (16m) 10m ago 34m 57.0M 2048M 19.0.0-2879-g324baf8d 8717720dc5e5 df3e201dfed6 2024-04-12T21:17:56.661 INFO:teuthology.orchestra.run.smithi018.stdout:mon.smithi186 smithi186 running (15m) 11m ago 31m 37.7M 2048M 19.0.0-2879-g324baf8d 8717720dc5e5 c77498da9b2b 2024-04-12T21:17:56.662 INFO:teuthology.orchestra.run.smithi018.stdout:node-exporter.smithi018 smithi018 *:9100 running (19m) 10m ago 33m 7780k - 1.5.0 0da6a335fe13 326c3d9c6847 2024-04-12T21:17:56.662 INFO:teuthology.orchestra.run.smithi018.stdout:node-exporter.smithi186 smithi186 *:9100 running (19m) 11m ago 31m 7826k - 1.5.0 0da6a335fe13 ec2112baeecf 2024-04-12T21:17:56.662 INFO:teuthology.orchestra.run.smithi018.stdout:osd.0 smithi018 running (14m) 10m ago 30m 121M 1559M 19.0.0-2879-g324baf8d 8717720dc5e5 60b1273ca047 2024-04-12T21:17:56.662 INFO:teuthology.orchestra.run.smithi018.stdout:osd.1 smithi018 running (13m) 10m ago 30m 91.3M 1559M 19.0.0-2879-g324baf8d 8717720dc5e5 170ce1e5fb8d 2024-04-12T21:17:56.662 INFO:teuthology.orchestra.run.smithi018.stdout:osd.2 smithi018 running (13m) 10m ago 29m 90.3M 1559M 19.0.0-2879-g324baf8d 8717720dc5e5 c2cbc6aa2839 2024-04-12T21:17:56.662 INFO:teuthology.orchestra.run.smithi018.stdout:osd.3 smithi186 running (12m) 11m ago 28m 96.6M 2581M 19.0.0-2879-g324baf8d 8717720dc5e5 0138e8262e4b 2024-04-12T21:17:56.662 INFO:teuthology.orchestra.run.smithi018.stdout:osd.4 smithi186 running (11m) 11m ago 28m 87.2M 2581M 19.0.0-2879-g324baf8d 8717720dc5e5 b20cb63b25c9 2024-04-12T21:17:56.662 INFO:teuthology.orchestra.run.smithi018.stdout:osd.5 smithi186 running (11m) 11m ago 27m 11.1M 2581M 19.0.0-2879-g324baf8d 8717720dc5e5 2513bea2610e 2024-04-12T21:17:56.662 INFO:teuthology.orchestra.run.smithi018.stdout:prometheus.smithi018 smithi018 *:9095 running (19m) 10m ago 32m 60.6M - 2.43.0 a07b618ecd1d f58841127e21 2024-04-12T21:17:56.664 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:56.667+0000 7fd1ad600700 1 -- 172.21.15.18:0/2931633544 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fd18c06ec00 msgr2=0x7fd18c0710b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:17:56.664 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:56.667+0000 7fd1ad600700 1 --2- 172.21.15.18:0/2931633544 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fd18c06ec00 0x7fd18c0710b0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fd19800b020 tx=0x7fd198009e30 comp rx=0 tx=0).stop 2024-04-12T21:17:56.664 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:56.667+0000 7fd1ad600700 1 -- 172.21.15.18:0/2931633544 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd1a8070bf0 msgr2=0x7fd1a81160f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:17:56.664 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:56.667+0000 7fd1ad600700 1 --2- 172.21.15.18:0/2931633544 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd1a8070bf0 0x7fd1a81160f0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7fd19400c830 tx=0x7fd194004470 comp rx=0 tx=0).stop 2024-04-12T21:17:56.664 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:56.667+0000 7fd1ad600700 1 -- 172.21.15.18:0/2931633544 shutdown_connections 2024-04-12T21:17:56.664 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:56.667+0000 7fd1ad600700 1 --2- 172.21.15.18:0/2931633544 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fd18c06ec00 0x7fd18c0710b0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:17:56.664 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:56.667+0000 7fd1ad600700 1 --2- 172.21.15.18:0/2931633544 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd1a80702e0 0x7fd1a8115bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:17:56.664 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:56.667+0000 7fd1ad600700 1 --2- 172.21.15.18:0/2931633544 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd1a8070bf0 0x7fd1a81160f0 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:17:56.664 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:56.667+0000 7fd1ad600700 1 -- 172.21.15.18:0/2931633544 >> 172.21.15.18:0/2931633544 conn(0x7fd1a80f89b0 msgr2=0x7fd1a80f9e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:17:56.664 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:56.668+0000 7fd1ad600700 1 -- 172.21.15.18:0/2931633544 shutdown_connections 2024-04-12T21:17:56.664 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:56.668+0000 7fd1ad600700 1 -- 172.21.15.18:0/2931633544 wait complete. 2024-04-12T21:17:56.816 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:56.820+0000 7f5d34a00700 1 -- 172.21.15.18:0/558724582 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5d300702e0 msgr2=0x7f5d300706b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:17:56.817 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:56.820+0000 7f5d34a00700 1 --2- 172.21.15.18:0/558724582 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5d300702e0 0x7f5d300706b0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f5d20004940 tx=0x7f5d20009e30 comp rx=0 tx=0).stop 2024-04-12T21:17:56.817 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:56.820+0000 7f5d34a00700 1 -- 172.21.15.18:0/558724582 shutdown_connections 2024-04-12T21:17:56.817 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:56.820+0000 7f5d34a00700 1 --2- 172.21.15.18:0/558724582 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5d30070bf0 0x7f5d30108fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:17:56.817 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:56.820+0000 7f5d34a00700 1 --2- 172.21.15.18:0/558724582 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5d300702e0 0x7f5d300706b0 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:17:56.817 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:56.820+0000 7f5d34a00700 1 -- 172.21.15.18:0/558724582 >> 172.21.15.18:0/558724582 conn(0x7f5d300f87a0 msgr2=0x7f5d300fabb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:17:56.817 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:56.820+0000 7f5d34a00700 1 -- 172.21.15.18:0/558724582 shutdown_connections 2024-04-12T21:17:56.817 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:56.820+0000 7f5d34a00700 1 -- 172.21.15.18:0/558724582 wait complete. 2024-04-12T21:17:56.818 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:56.821+0000 7f5d34a00700 1 Processor -- start 2024-04-12T21:17:56.818 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:56.821+0000 7f5d34a00700 1 -- start start 2024-04-12T21:17:56.819 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:56.822+0000 7f5d34a00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5d300702e0 0x7f5d30115ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:17:56.819 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:56.822+0000 7f5d34a00700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5d30070bf0 0x7f5d301160e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:17:56.819 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:56.822+0000 7f5d34a00700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d30116730 con 0x7f5d300702e0 2024-04-12T21:17:56.819 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:56.822+0000 7f5d34a00700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d3010fb40 con 0x7f5d30070bf0 2024-04-12T21:17:56.819 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:56.822+0000 7f5d2e800700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5d300702e0 0x7f5d30115ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:17:56.819 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:56.823+0000 7f5d2e800700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5d300702e0 0x7f5d30115ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:37572/0 (socket says 172.21.15.18:37572) 2024-04-12T21:17:56.819 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:56.823+0000 7f5d2e800700 1 -- 172.21.15.18:0/96185666 learned_addr learned my addr 172.21.15.18:0/96185666 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:17:56.820 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:56.823+0000 7f5d2de00700 1 --2- 172.21.15.18:0/96185666 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5d30070bf0 0x7f5d301160e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:17:56.820 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:56.823+0000 7f5d2e800700 1 -- 172.21.15.18:0/96185666 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5d30070bf0 msgr2=0x7f5d301160e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:17:56.820 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:56.823+0000 7f5d2e800700 1 --2- 172.21.15.18:0/96185666 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5d30070bf0 0x7f5d301160e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:17:56.820 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:56.823+0000 7f5d2e800700 1 -- 172.21.15.18:0/96185666 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d20005040 con 0x7f5d300702e0 2024-04-12T21:17:56.821 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:56.824+0000 7f5d2e800700 1 --2- 172.21.15.18:0/96185666 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5d300702e0 0x7f5d30115ba0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f5d20009180 tx=0x7f5d20009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:17:56.821 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:56.824+0000 7f5d26a00700 1 -- 172.21.15.18:0/96185666 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5d2000e040 con 0x7f5d300702e0 2024-04-12T21:17:56.821 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:56.824+0000 7f5d34a00700 1 -- 172.21.15.18:0/96185666 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5d3010fdc0 con 0x7f5d300702e0 2024-04-12T21:17:56.821 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:56.824+0000 7f5d26a00700 1 -- 172.21.15.18:0/96185666 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f5d2000bbc0 con 0x7f5d300702e0 2024-04-12T21:17:56.821 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:56.824+0000 7f5d26a00700 1 -- 172.21.15.18:0/96185666 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5d2001c630 con 0x7f5d300702e0 2024-04-12T21:17:56.821 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:56.824+0000 7f5d34a00700 1 -- 172.21.15.18:0/96185666 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5d30110300 con 0x7f5d300702e0 2024-04-12T21:17:56.821 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:56.825+0000 7f5d34a00700 1 -- 172.21.15.18:0/96185666 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5d30071a20 con 0x7f5d300702e0 2024-04-12T21:17:56.823 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:56.827+0000 7f5d26a00700 1 -- 172.21.15.18:0/96185666 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7f5d2000bd30 con 0x7f5d300702e0 2024-04-12T21:17:56.824 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:56.828+0000 7f5d26a00700 1 --2- 172.21.15.18:0/96185666 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f5d1406ec10 0x7f5d140710c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:17:56.824 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:56.828+0000 7f5d26a00700 1 -- 172.21.15.18:0/96185666 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 6757+0+0 (secure 0 0 0) 0x7f5d2008fd90 con 0x7f5d300702e0 2024-04-12T21:17:56.824 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:56.828+0000 7f5d2de00700 1 --2- 172.21.15.18:0/96185666 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f5d1406ec10 0x7f5d140710c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:17:56.825 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:56.829+0000 7f5d2de00700 1 --2- 172.21.15.18:0/96185666 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f5d1406ec10 0x7f5d140710c0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f5d1c007fd0 tx=0x7f5d1c009040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:17:56.829 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:56.834+0000 7f5d26a00700 1 -- 172.21.15.18:0/96185666 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f5d20058a90 con 0x7f5d300702e0 2024-04-12T21:17:57.141 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:57.145+0000 7f5d34a00700 1 -- 172.21.15.18:0/96185666 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f5d3004bed0 con 0x7f5d300702e0 2024-04-12T21:17:57.142 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:57.147+0000 7f5d26a00700 1 -- 172.21.15.18:0/96185666 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+837 (secure 0 0 0) 0x7f5d2005c150 con 0x7f5d300702e0 2024-04-12T21:17:57.143 INFO:teuthology.orchestra.run.smithi018.stdout:{ 2024-04-12T21:17:57.143 INFO:teuthology.orchestra.run.smithi018.stdout: "mon": { 2024-04-12T21:17:57.143 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 19.0.0-2879-g324baf8d (324baf8ddb91dea46cac50cb535c306115791355) squid (dev)": 2 2024-04-12T21:17:57.143 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-12T21:17:57.143 INFO:teuthology.orchestra.run.smithi018.stdout: "mgr": { 2024-04-12T21:17:57.143 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 19.0.0-2879-g324baf8d (324baf8ddb91dea46cac50cb535c306115791355) squid (dev)": 2 2024-04-12T21:17:57.143 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-12T21:17:57.143 INFO:teuthology.orchestra.run.smithi018.stdout: "osd": { 2024-04-12T21:17:57.143 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 19.0.0-2879-g324baf8d (324baf8ddb91dea46cac50cb535c306115791355) squid (dev)": 6 2024-04-12T21:17:57.144 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-12T21:17:57.144 INFO:teuthology.orchestra.run.smithi018.stdout: "mds": { 2024-04-12T21:17:57.144 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 17.2.7-827-gd3b5060a (d3b5060a17449edfae9385cdf792afec109fca5d) quincy (stable)": 2, 2024-04-12T21:17:57.144 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 19.0.0-2879-g324baf8d (324baf8ddb91dea46cac50cb535c306115791355) squid (dev)": 2 2024-04-12T21:17:57.144 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-12T21:17:57.144 INFO:teuthology.orchestra.run.smithi018.stdout: "overall": { 2024-04-12T21:17:57.144 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 17.2.7-827-gd3b5060a (d3b5060a17449edfae9385cdf792afec109fca5d) quincy (stable)": 2, 2024-04-12T21:17:57.144 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 19.0.0-2879-g324baf8d (324baf8ddb91dea46cac50cb535c306115791355) squid (dev)": 12 2024-04-12T21:17:57.144 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-04-12T21:17:57.144 INFO:teuthology.orchestra.run.smithi018.stdout:} 2024-04-12T21:17:57.146 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:57.149+0000 7f5d34a00700 1 -- 172.21.15.18:0/96185666 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f5d1406ec10 msgr2=0x7f5d140710c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:17:57.146 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:57.149+0000 7f5d34a00700 1 --2- 172.21.15.18:0/96185666 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f5d1406ec10 0x7f5d140710c0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f5d1c007fd0 tx=0x7f5d1c009040 comp rx=0 tx=0).stop 2024-04-12T21:17:57.146 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:57.149+0000 7f5d34a00700 1 -- 172.21.15.18:0/96185666 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5d300702e0 msgr2=0x7f5d30115ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:17:57.146 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:57.149+0000 7f5d34a00700 1 --2- 172.21.15.18:0/96185666 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5d300702e0 0x7f5d30115ba0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f5d20009180 tx=0x7f5d20009e30 comp rx=0 tx=0).stop 2024-04-12T21:17:57.146 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:57.150+0000 7f5d34a00700 1 -- 172.21.15.18:0/96185666 shutdown_connections 2024-04-12T21:17:57.146 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:57.150+0000 7f5d34a00700 1 --2- 172.21.15.18:0/96185666 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f5d1406ec10 0x7f5d140710c0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:17:57.146 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:57.150+0000 7f5d34a00700 1 --2- 172.21.15.18:0/96185666 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5d300702e0 0x7f5d30115ba0 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:17:57.146 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:57.150+0000 7f5d34a00700 1 --2- 172.21.15.18:0/96185666 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5d30070bf0 0x7f5d301160e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:17:57.146 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:57.150+0000 7f5d34a00700 1 -- 172.21.15.18:0/96185666 >> 172.21.15.18:0/96185666 conn(0x7f5d300f87a0 msgr2=0x7f5d301058e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:17:57.146 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:57.150+0000 7f5d34a00700 1 -- 172.21.15.18:0/96185666 shutdown_connections 2024-04-12T21:17:57.147 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:57.150+0000 7f5d34a00700 1 -- 172.21.15.18:0/96185666 wait complete. 2024-04-12T21:17:57.294 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:57.297+0000 7fdae1800700 1 -- 172.21.15.18:0/2562754521 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fdadc070360 msgr2=0x7fdadc070730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:17:57.294 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:57.297+0000 7fdae1800700 1 --2- 172.21.15.18:0/2562754521 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fdadc070360 0x7fdadc070730 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7fdacc004970 tx=0x7fdacc009e30 comp rx=0 tx=0).stop 2024-04-12T21:17:57.294 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:57.297+0000 7fdae1800700 1 -- 172.21.15.18:0/2562754521 shutdown_connections 2024-04-12T21:17:57.294 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:57.297+0000 7fdae1800700 1 --2- 172.21.15.18:0/2562754521 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdadc070c70 0x7fdadc109130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:17:57.294 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:57.297+0000 7fdae1800700 1 --2- 172.21.15.18:0/2562754521 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fdadc070360 0x7fdadc070730 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:17:57.294 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:57.298+0000 7fdae1800700 1 -- 172.21.15.18:0/2562754521 >> 172.21.15.18:0/2562754521 conn(0x7fdadc0f8970 msgr2=0x7fdadc0fad80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:17:57.294 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:57.298+0000 7fdae1800700 1 -- 172.21.15.18:0/2562754521 shutdown_connections 2024-04-12T21:17:57.294 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:57.298+0000 7fdae1800700 1 -- 172.21.15.18:0/2562754521 wait complete. 2024-04-12T21:17:57.296 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:57.298+0000 7fdae1800700 1 Processor -- start 2024-04-12T21:17:57.296 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:57.299+0000 7fdae1800700 1 -- start start 2024-04-12T21:17:57.296 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:57.299+0000 7fdae1800700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fdadc070360 0x7fdadc115d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:17:57.296 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:57.299+0000 7fdae1800700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdadc070c70 0x7fdadc1162c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:17:57.296 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:57.299+0000 7fdae1800700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdadc116910 con 0x7fdadc070360 2024-04-12T21:17:57.296 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:57.299+0000 7fdae1800700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdadc10fd20 con 0x7fdadc070c70 2024-04-12T21:17:57.296 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:57.299+0000 7fdadaa00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fdadc070360 0x7fdadc115d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:17:57.296 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:57.299+0000 7fdadaa00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fdadc070360 0x7fdadc115d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:37588/0 (socket says 172.21.15.18:37588) 2024-04-12T21:17:57.296 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:57.299+0000 7fdadaa00700 1 -- 172.21.15.18:0/2604689820 learned_addr learned my addr 172.21.15.18:0/2604689820 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:17:57.296 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:57.300+0000 7fdadaa00700 1 -- 172.21.15.18:0/2604689820 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdadc070c70 msgr2=0x7fdadc1162c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:17:57.296 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:57.300+0000 7fdadaa00700 1 --2- 172.21.15.18:0/2604689820 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdadc070c70 0x7fdadc1162c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:17:57.297 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:57.300+0000 7fdadaa00700 1 -- 172.21.15.18:0/2604689820 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdacc005040 con 0x7fdadc070360 2024-04-12T21:17:57.297 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:57.300+0000 7fdadaa00700 1 --2- 172.21.15.18:0/2604689820 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fdadc070360 0x7fdadc115d80 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7fdacc000c00 tx=0x7fdacc009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:17:57.297 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:57.300+0000 7fdad2a00700 1 -- 172.21.15.18:0/2604689820 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdacc00e040 con 0x7fdadc070360 2024-04-12T21:17:57.297 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:57.300+0000 7fdad2a00700 1 -- 172.21.15.18:0/2604689820 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fdacc00bc10 con 0x7fdadc070360 2024-04-12T21:17:57.298 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:57.300+0000 7fdae1800700 1 -- 172.21.15.18:0/2604689820 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdadc10ffa0 con 0x7fdadc070360 2024-04-12T21:17:57.298 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:57.300+0000 7fdad2a00700 1 -- 172.21.15.18:0/2604689820 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdacc01c3f0 con 0x7fdadc070360 2024-04-12T21:17:57.298 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:57.301+0000 7fdae1800700 1 -- 172.21.15.18:0/2604689820 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdadc1104e0 con 0x7fdadc070360 2024-04-12T21:17:57.298 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:57.302+0000 7fdae1800700 1 -- 172.21.15.18:0/2604689820 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdadc071aa0 con 0x7fdadc070360 2024-04-12T21:17:57.298 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:57.303+0000 7fdad2a00700 1 -- 172.21.15.18:0/2604689820 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7fdacc00bd80 con 0x7fdadc070360 2024-04-12T21:17:57.299 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:57.303+0000 7fdad2a00700 1 --2- 172.21.15.18:0/2604689820 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fdac006ec00 0x7fdac00710b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:17:57.300 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:57.304+0000 7fdad2a00700 1 -- 172.21.15.18:0/2604689820 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 6757+0+0 (secure 0 0 0) 0x7fdacc08fd60 con 0x7fdadc070360 2024-04-12T21:17:57.300 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:57.304+0000 7fdada000700 1 --2- 172.21.15.18:0/2604689820 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fdac006ec00 0x7fdac00710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:17:57.300 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:57.305+0000 7fdada000700 1 --2- 172.21.15.18:0/2604689820 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fdac006ec00 0x7fdac00710b0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fdadc111660 tx=0x7fdac8007d40 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:17:57.304 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:57.309+0000 7fdad2a00700 1 -- 172.21.15.18:0/2604689820 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7fdacc094070 con 0x7fdadc070360 2024-04-12T21:17:57.578 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:57.581+0000 7fdae1800700 1 -- 172.21.15.18:0/2604689820 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fdadc04bed0 con 0x7fdadc070360 2024-04-12T21:17:57.578 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:57.582+0000 7fdad2a00700 1 -- 172.21.15.18:0/2604689820 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 29 v29) v1 ==== 76+0+1872 (secure 0 0 0) 0x7fdacc0589e0 con 0x7fdadc070360 2024-04-12T21:17:57.579 INFO:teuthology.orchestra.run.smithi018.stdout:e29 2024-04-12T21:17:57.579 INFO:teuthology.orchestra.run.smithi018.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-12T21:17:57.579 INFO:teuthology.orchestra.run.smithi018.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-12T21:17:57.579 INFO:teuthology.orchestra.run.smithi018.stdout:legacy client fscid: 1 2024-04-12T21:17:57.579 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-12T21:17:57.579 INFO:teuthology.orchestra.run.smithi018.stdout:Filesystem 'cephfs' (1) 2024-04-12T21:17:57.580 INFO:teuthology.orchestra.run.smithi018.stdout:fs_name cephfs 2024-04-12T21:17:57.580 INFO:teuthology.orchestra.run.smithi018.stdout:epoch 24 2024-04-12T21:17:57.580 INFO:teuthology.orchestra.run.smithi018.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-04-12T21:17:57.580 INFO:teuthology.orchestra.run.smithi018.stdout:created 2024-04-12T20:51:57.362850+0000 2024-04-12T21:17:57.580 INFO:teuthology.orchestra.run.smithi018.stdout:modified 2024-04-12T21:06:59.717190+0000 2024-04-12T21:17:57.580 INFO:teuthology.orchestra.run.smithi018.stdout:tableserver 0 2024-04-12T21:17:57.580 INFO:teuthology.orchestra.run.smithi018.stdout:root 0 2024-04-12T21:17:57.580 INFO:teuthology.orchestra.run.smithi018.stdout:session_timeout 60 2024-04-12T21:17:57.580 INFO:teuthology.orchestra.run.smithi018.stdout:session_autoclose 300 2024-04-12T21:17:57.580 INFO:teuthology.orchestra.run.smithi018.stdout:max_file_size 1099511627776 2024-04-12T21:17:57.580 INFO:teuthology.orchestra.run.smithi018.stdout:max_xattr_size 65536 2024-04-12T21:17:57.580 INFO:teuthology.orchestra.run.smithi018.stdout:required_client_features {} 2024-04-12T21:17:57.580 INFO:teuthology.orchestra.run.smithi018.stdout:last_failure 0 2024-04-12T21:17:57.580 INFO:teuthology.orchestra.run.smithi018.stdout:last_failure_osd_epoch 100 2024-04-12T21:17:57.580 INFO:teuthology.orchestra.run.smithi018.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-12T21:17:57.581 INFO:teuthology.orchestra.run.smithi018.stdout:max_mds 2 2024-04-12T21:17:57.581 INFO:teuthology.orchestra.run.smithi018.stdout:in 0,1 2024-04-12T21:17:57.581 INFO:teuthology.orchestra.run.smithi018.stdout:up {} 2024-04-12T21:17:57.581 INFO:teuthology.orchestra.run.smithi018.stdout:failed 0,1 2024-04-12T21:17:57.581 INFO:teuthology.orchestra.run.smithi018.stdout:damaged 2024-04-12T21:17:57.581 INFO:teuthology.orchestra.run.smithi018.stdout:stopped 2024-04-12T21:17:57.581 INFO:teuthology.orchestra.run.smithi018.stdout:data_pools [3] 2024-04-12T21:17:57.581 INFO:teuthology.orchestra.run.smithi018.stdout:metadata_pool 2 2024-04-12T21:17:57.581 INFO:teuthology.orchestra.run.smithi018.stdout:inline_data disabled 2024-04-12T21:17:57.581 INFO:teuthology.orchestra.run.smithi018.stdout:balancer 2024-04-12T21:17:57.581 INFO:teuthology.orchestra.run.smithi018.stdout:bal_rank_mask -1 2024-04-12T21:17:57.581 INFO:teuthology.orchestra.run.smithi018.stdout:standby_count_wanted 1 2024-04-12T21:17:57.581 INFO:teuthology.orchestra.run.smithi018.stdout:qdb_cluster leader: 0 members: 2024-04-12T21:17:57.581 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-12T21:17:57.581 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-12T21:17:57.582 INFO:teuthology.orchestra.run.smithi018.stdout:Standby daemons: 2024-04-12T21:17:57.582 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-12T21:17:57.582 INFO:teuthology.orchestra.run.smithi018.stdout:[mds.cephfs.smithi186.wjnafs{-1:24277} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.186:6826/438347207,v1:172.21.15.186:6827/438347207] compat {c=[1],r=[1],i=[7ff]}] 2024-04-12T21:17:57.582 INFO:teuthology.orchestra.run.smithi018.stdout:[mds.cephfs.smithi186.jdllyn{-1:24281} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.186:6824/1464359952,v1:172.21.15.186:6825/1464359952] compat {c=[1],r=[1],i=[7ff]}] 2024-04-12T21:17:57.582 INFO:teuthology.orchestra.run.smithi018.stdout:[mds.cephfs.smithi018.acqbsc{-1:34436} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.18:6828/755179764,v1:172.21.15.18:6829/755179764] compat {c=[1],r=[1],i=[1fff]}] 2024-04-12T21:17:57.582 INFO:teuthology.orchestra.run.smithi018.stdout:[mds.cephfs.smithi018.szjluo{-1:34468} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.18:6826/2407992653,v1:172.21.15.18:6827/2407992653] compat {c=[1],r=[1],i=[1fff]}] 2024-04-12T21:17:57.583 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:57.585+0000 7fdae1800700 1 -- 172.21.15.18:0/2604689820 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fdac006ec00 msgr2=0x7fdac00710b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:17:57.583 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:57.585+0000 7fdae1800700 1 --2- 172.21.15.18:0/2604689820 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fdac006ec00 0x7fdac00710b0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fdadc111660 tx=0x7fdac8007d40 comp rx=0 tx=0).stop 2024-04-12T21:17:57.583 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:57.585+0000 7fdae1800700 1 -- 172.21.15.18:0/2604689820 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fdadc070360 msgr2=0x7fdadc115d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:17:57.583 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:57.585+0000 7fdae1800700 1 --2- 172.21.15.18:0/2604689820 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fdadc070360 0x7fdadc115d80 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7fdacc000c00 tx=0x7fdacc009e30 comp rx=0 tx=0).stop 2024-04-12T21:17:57.583 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:57.585+0000 7fdae1800700 1 -- 172.21.15.18:0/2604689820 shutdown_connections 2024-04-12T21:17:57.583 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:57.585+0000 7fdae1800700 1 --2- 172.21.15.18:0/2604689820 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fdac006ec00 0x7fdac00710b0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:17:57.583 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:57.585+0000 7fdae1800700 1 --2- 172.21.15.18:0/2604689820 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fdadc070360 0x7fdadc115d80 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:17:57.583 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:57.585+0000 7fdae1800700 1 --2- 172.21.15.18:0/2604689820 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdadc070c70 0x7fdadc1162c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:17:57.583 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:57.585+0000 7fdae1800700 1 -- 172.21.15.18:0/2604689820 >> 172.21.15.18:0/2604689820 conn(0x7fdadc0f8970 msgr2=0x7fdadc0f9e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:17:57.583 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:57.585+0000 7fdae1800700 1 -- 172.21.15.18:0/2604689820 shutdown_connections 2024-04-12T21:17:57.583 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:57.585+0000 7fdae1800700 1 -- 172.21.15.18:0/2604689820 wait complete. 2024-04-12T21:17:57.584 INFO:teuthology.orchestra.run.smithi018.stderr:dumped fsmap epoch 29 2024-04-12T21:17:57.741 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:57.744+0000 7ff4d2c00700 1 -- 172.21.15.18:0/440891003 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff4cc1037a0 msgr2=0x7ff4cc0fce10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:17:57.741 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:57.744+0000 7ff4d2c00700 1 --2- 172.21.15.18:0/440891003 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff4cc1037a0 0x7ff4cc0fce10 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7ff4b8004940 tx=0x7ff4b8009e30 comp rx=0 tx=0).stop 2024-04-12T21:17:57.741 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:17:57 smithi018 ceph-mon[112590]: from='client.34976 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:17:57.741 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:17:57 smithi018 ceph-mon[112590]: pgmap v544: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:17:57.741 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:17:57 smithi018 ceph-mon[112590]: from='client.34980 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:17:57.742 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:17:57 smithi018 ceph-mon[112590]: from='client.34984 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:17:57.742 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:17:57 smithi018 ceph-mon[112590]: from='client.? 172.21.15.18:0/96185666' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T21:17:57.742 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:57.745+0000 7ff4d2c00700 1 -- 172.21.15.18:0/440891003 shutdown_connections 2024-04-12T21:17:57.742 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:57.745+0000 7ff4d2c00700 1 --2- 172.21.15.18:0/440891003 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff4cc1037a0 0x7ff4cc0fce10 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:17:57.742 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:57.745+0000 7ff4d2c00700 1 --2- 172.21.15.18:0/440891003 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff4cc102e00 0x7ff4cc1031d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:17:57.743 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:57.745+0000 7ff4d2c00700 1 -- 172.21.15.18:0/440891003 >> 172.21.15.18:0/440891003 conn(0x7ff4cc0f89b0 msgr2=0x7ff4cc0fadc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:17:57.743 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:57.745+0000 7ff4d2c00700 1 -- 172.21.15.18:0/440891003 shutdown_connections 2024-04-12T21:17:57.743 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:57.745+0000 7ff4d2c00700 1 -- 172.21.15.18:0/440891003 wait complete. 2024-04-12T21:17:57.743 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:57.746+0000 7ff4d2c00700 1 Processor -- start 2024-04-12T21:17:57.743 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:57.746+0000 7ff4d2c00700 1 -- start start 2024-04-12T21:17:57.744 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:57.747+0000 7ff4d2c00700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff4cc102e00 0x7ff4cc0702c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:17:57.744 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:57.747+0000 7ff4d2c00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff4cc1037a0 0x7ff4cc070800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:17:57.744 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:57.747+0000 7ff4d2c00700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4cc070e20 con 0x7ff4cc1037a0 2024-04-12T21:17:57.745 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:57.747+0000 7ff4d2c00700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4cc070f60 con 0x7ff4cc102e00 2024-04-12T21:17:57.745 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:57.747+0000 7ff4cb400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff4cc1037a0 0x7ff4cc070800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:17:57.745 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:57.747+0000 7ff4cb400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff4cc1037a0 0x7ff4cc070800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:37614/0 (socket says 172.21.15.18:37614) 2024-04-12T21:17:57.745 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:57.747+0000 7ff4cb400700 1 -- 172.21.15.18:0/1012725350 learned_addr learned my addr 172.21.15.18:0/1012725350 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:17:57.745 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:57.748+0000 7ff4cbe00700 1 --2- 172.21.15.18:0/1012725350 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff4cc102e00 0x7ff4cc0702c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:17:57.745 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:57.748+0000 7ff4cb400700 1 -- 172.21.15.18:0/1012725350 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff4cc102e00 msgr2=0x7ff4cc0702c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:17:57.745 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:57.748+0000 7ff4cb400700 1 --2- 172.21.15.18:0/1012725350 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff4cc102e00 0x7ff4cc0702c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:17:57.745 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:57.748+0000 7ff4cb400700 1 -- 172.21.15.18:0/1012725350 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4b8005040 con 0x7ff4cc1037a0 2024-04-12T21:17:57.745 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:57.748+0000 7ff4cb400700 1 --2- 172.21.15.18:0/1012725350 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff4cc1037a0 0x7ff4cc070800 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7ff4b8009100 tx=0x7ff4b8009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:17:57.745 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:57.748+0000 7ff4c3e00700 1 -- 172.21.15.18:0/1012725350 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff4b800e040 con 0x7ff4cc1037a0 2024-04-12T21:17:57.745 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:57.749+0000 7ff4c3e00700 1 -- 172.21.15.18:0/1012725350 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff4b8004290 con 0x7ff4cc1037a0 2024-04-12T21:17:57.745 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:57.749+0000 7ff4d2c00700 1 -- 172.21.15.18:0/1012725350 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff4cc119ee0 con 0x7ff4cc1037a0 2024-04-12T21:17:57.746 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:57.749+0000 7ff4c3e00700 1 -- 172.21.15.18:0/1012725350 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff4b801c3f0 con 0x7ff4cc1037a0 2024-04-12T21:17:57.746 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:57.749+0000 7ff4d2c00700 1 -- 172.21.15.18:0/1012725350 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff4cc11a3c0 con 0x7ff4cc1037a0 2024-04-12T21:17:57.746 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:57.750+0000 7ff4d2c00700 1 -- 172.21.15.18:0/1012725350 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff4cc100110 con 0x7ff4cc1037a0 2024-04-12T21:17:57.747 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:57.751+0000 7ff4c3e00700 1 -- 172.21.15.18:0/1012725350 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7ff4b8013790 con 0x7ff4cc1037a0 2024-04-12T21:17:57.748 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:57.752+0000 7ff4c3e00700 1 --2- 172.21.15.18:0/1012725350 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7ff4b006ec10 0x7ff4b00710c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:17:57.748 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:57.752+0000 7ff4c3e00700 1 -- 172.21.15.18:0/1012725350 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 6757+0+0 (secure 0 0 0) 0x7ff4b808fc70 con 0x7ff4cc1037a0 2024-04-12T21:17:57.748 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:57.753+0000 7ff4cbe00700 1 --2- 172.21.15.18:0/1012725350 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7ff4b006ec10 0x7ff4b00710c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:17:57.749 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:57.753+0000 7ff4cbe00700 1 --2- 172.21.15.18:0/1012725350 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7ff4b006ec10 0x7ff4b00710c0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7ff4bc006fd0 tx=0x7ff4bc008040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:17:57.753 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:57.757+0000 7ff4c3e00700 1 -- 172.21.15.18:0/1012725350 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7ff4b8058990 con 0x7ff4cc1037a0 2024-04-12T21:17:57.778 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:17:57 smithi186 ceph-mon[86379]: from='client.34976 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:17:57.779 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:17:57 smithi186 ceph-mon[86379]: pgmap v544: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:17:57.779 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:17:57 smithi186 ceph-mon[86379]: from='client.34980 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:17:57.779 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:17:57 smithi186 ceph-mon[86379]: from='client.34984 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:17:57.779 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:17:57 smithi186 ceph-mon[86379]: from='client.? 172.21.15.18:0/96185666' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T21:17:58.007 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:58.010+0000 7ff4d2c00700 1 -- 172.21.15.18:0/1012725350 --> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7ff4cc071a20 con 0x7ff4b006ec10 2024-04-12T21:17:58.009 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:58.013+0000 7ff4c3e00700 1 -- 172.21.15.18:0/1012725350 <== mgr.34102 v2:172.21.15.18:6800/3812891919 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7ff4cc071a20 con 0x7ff4b006ec10 2024-04-12T21:17:58.009 INFO:teuthology.orchestra.run.smithi018.stdout:{ 2024-04-12T21:17:58.009 INFO:teuthology.orchestra.run.smithi018.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:f4452def3be4d350f4a8e5feaf7ebb1511be9298fe68478ba7a2d6fbd7d69a63", 2024-04-12T21:17:58.010 INFO:teuthology.orchestra.run.smithi018.stdout: "in_progress": true, 2024-04-12T21:17:58.010 INFO:teuthology.orchestra.run.smithi018.stdout: "which": "Upgrading all daemon types on all hosts", 2024-04-12T21:17:58.010 INFO:teuthology.orchestra.run.smithi018.stdout: "services_complete": [ 2024-04-12T21:17:58.010 INFO:teuthology.orchestra.run.smithi018.stdout: "crash", 2024-04-12T21:17:58.010 INFO:teuthology.orchestra.run.smithi018.stdout: "osd", 2024-04-12T21:17:58.010 INFO:teuthology.orchestra.run.smithi018.stdout: "mon", 2024-04-12T21:17:58.010 INFO:teuthology.orchestra.run.smithi018.stdout: "mgr" 2024-04-12T21:17:58.010 INFO:teuthology.orchestra.run.smithi018.stdout: ], 2024-04-12T21:17:58.010 INFO:teuthology.orchestra.run.smithi018.stdout: "progress": "14/21 daemons upgraded", 2024-04-12T21:17:58.010 INFO:teuthology.orchestra.run.smithi018.stdout: "message": "Currently upgrading mds daemons", 2024-04-12T21:17:58.010 INFO:teuthology.orchestra.run.smithi018.stdout: "is_paused": false 2024-04-12T21:17:58.010 INFO:teuthology.orchestra.run.smithi018.stdout:} 2024-04-12T21:17:58.012 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:58.016+0000 7ff4d2c00700 1 -- 172.21.15.18:0/1012725350 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7ff4b006ec10 msgr2=0x7ff4b00710c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:17:58.012 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:58.016+0000 7ff4d2c00700 1 --2- 172.21.15.18:0/1012725350 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7ff4b006ec10 0x7ff4b00710c0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7ff4bc006fd0 tx=0x7ff4bc008040 comp rx=0 tx=0).stop 2024-04-12T21:17:58.012 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:58.016+0000 7ff4d2c00700 1 -- 172.21.15.18:0/1012725350 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff4cc1037a0 msgr2=0x7ff4cc070800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:17:58.012 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:58.016+0000 7ff4d2c00700 1 --2- 172.21.15.18:0/1012725350 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff4cc1037a0 0x7ff4cc070800 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7ff4b8009100 tx=0x7ff4b8009e30 comp rx=0 tx=0).stop 2024-04-12T21:17:58.013 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:58.016+0000 7ff4d2c00700 1 -- 172.21.15.18:0/1012725350 shutdown_connections 2024-04-12T21:17:58.013 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:58.016+0000 7ff4d2c00700 1 --2- 172.21.15.18:0/1012725350 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7ff4b006ec10 0x7ff4b00710c0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:17:58.013 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:58.016+0000 7ff4d2c00700 1 --2- 172.21.15.18:0/1012725350 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff4cc102e00 0x7ff4cc0702c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:17:58.013 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:58.016+0000 7ff4d2c00700 1 --2- 172.21.15.18:0/1012725350 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff4cc1037a0 0x7ff4cc070800 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:17:58.014 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:58.016+0000 7ff4d2c00700 1 -- 172.21.15.18:0/1012725350 >> 172.21.15.18:0/1012725350 conn(0x7ff4cc0f89b0 msgr2=0x7ff4cc0fa7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:17:58.014 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:58.016+0000 7ff4d2c00700 1 -- 172.21.15.18:0/1012725350 shutdown_connections 2024-04-12T21:17:58.014 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:58.016+0000 7ff4d2c00700 1 -- 172.21.15.18:0/1012725350 wait complete. 2024-04-12T21:17:58.175 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:58.178+0000 7fce2fa00700 1 -- 172.21.15.18:0/1533622713 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fce28105030 msgr2=0x7fce28105400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:17:58.175 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:58.178+0000 7fce2fa00700 1 --2- 172.21.15.18:0/1533622713 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fce28105030 0x7fce28105400 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7fce1c009ce0 tx=0x7fce1c004d10 comp rx=0 tx=0).stop 2024-04-12T21:17:58.175 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:58.178+0000 7fce2fa00700 1 -- 172.21.15.18:0/1533622713 shutdown_connections 2024-04-12T21:17:58.175 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:58.178+0000 7fce2fa00700 1 --2- 172.21.15.18:0/1533622713 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fce281059d0 0x7fce280ff040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:17:58.175 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:58.178+0000 7fce2fa00700 1 --2- 172.21.15.18:0/1533622713 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fce28105030 0x7fce28105400 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:17:58.175 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:58.178+0000 7fce2fa00700 1 -- 172.21.15.18:0/1533622713 >> 172.21.15.18:0/1533622713 conn(0x7fce280faba0 msgr2=0x7fce280fcfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:17:58.175 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:58.178+0000 7fce2fa00700 1 -- 172.21.15.18:0/1533622713 shutdown_connections 2024-04-12T21:17:58.175 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:58.178+0000 7fce2fa00700 1 -- 172.21.15.18:0/1533622713 wait complete. 2024-04-12T21:17:58.176 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:58.179+0000 7fce2fa00700 1 Processor -- start 2024-04-12T21:17:58.177 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:58.179+0000 7fce2fa00700 1 -- start start 2024-04-12T21:17:58.177 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:58.180+0000 7fce2fa00700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fce28105030 0x7fce28115c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:17:58.177 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:58.180+0000 7fce2fa00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fce281059d0 0x7fce28116190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:17:58.177 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:58.180+0000 7fce2fa00700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce28116870 con 0x7fce281059d0 2024-04-12T21:17:58.177 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:58.180+0000 7fce2fa00700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce2810fe70 con 0x7fce28105030 2024-04-12T21:17:58.177 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:58.180+0000 7fce27e00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fce281059d0 0x7fce28116190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:17:58.177 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:58.180+0000 7fce27e00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fce281059d0 0x7fce28116190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:37622/0 (socket says 172.21.15.18:37622) 2024-04-12T21:17:58.177 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:58.180+0000 7fce27e00700 1 -- 172.21.15.18:0/4064646930 learned_addr learned my addr 172.21.15.18:0/4064646930 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:17:58.177 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:58.181+0000 7fce2d000700 1 --2- 172.21.15.18:0/4064646930 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fce28105030 0x7fce28115c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:17:58.177 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:58.181+0000 7fce27e00700 1 -- 172.21.15.18:0/4064646930 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fce28105030 msgr2=0x7fce28115c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:17:58.177 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:58.181+0000 7fce27e00700 1 --2- 172.21.15.18:0/4064646930 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fce28105030 0x7fce28115c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:17:58.177 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:58.181+0000 7fce27e00700 1 -- 172.21.15.18:0/4064646930 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce1c009940 con 0x7fce281059d0 2024-04-12T21:17:58.178 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:58.181+0000 7fce27e00700 1 --2- 172.21.15.18:0/4064646930 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fce281059d0 0x7fce28116190 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7fce1400b790 tx=0x7fce14009220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:17:58.178 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:58.181+0000 7fce24c00700 1 -- 172.21.15.18:0/4064646930 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fce1400f6f0 con 0x7fce281059d0 2024-04-12T21:17:58.178 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:58.181+0000 7fce24c00700 1 -- 172.21.15.18:0/4064646930 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fce1400fd20 con 0x7fce281059d0 2024-04-12T21:17:58.178 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:58.181+0000 7fce2fa00700 1 -- 172.21.15.18:0/4064646930 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fce28110150 con 0x7fce281059d0 2024-04-12T21:17:58.178 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:58.181+0000 7fce24c00700 1 -- 172.21.15.18:0/4064646930 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fce140103a0 con 0x7fce281059d0 2024-04-12T21:17:58.178 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:58.182+0000 7fce2fa00700 1 -- 172.21.15.18:0/4064646930 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fce28110710 con 0x7fce281059d0 2024-04-12T21:17:58.179 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:58.183+0000 7fce2fa00700 1 -- 172.21.15.18:0/4064646930 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fce28100780 con 0x7fce281059d0 2024-04-12T21:17:58.180 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:58.184+0000 7fce24c00700 1 -- 172.21.15.18:0/4064646930 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7fce14011400 con 0x7fce281059d0 2024-04-12T21:17:58.181 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:58.185+0000 7fce24c00700 1 --2- 172.21.15.18:0/4064646930 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fce1006ec00 0x7fce100710b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:17:58.181 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:58.185+0000 7fce24c00700 1 -- 172.21.15.18:0/4064646930 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 6757+0+0 (secure 0 0 0) 0x7fce1408e8f0 con 0x7fce281059d0 2024-04-12T21:17:58.182 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:58.185+0000 7fce2d000700 1 --2- 172.21.15.18:0/4064646930 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fce1006ec00 0x7fce100710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:17:58.182 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:58.186+0000 7fce2d000700 1 --2- 172.21.15.18:0/4064646930 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fce1006ec00 0x7fce100710b0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fce1c00b740 tx=0x7fce1c004d10 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:17:58.186 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:58.190+0000 7fce24c00700 1 -- 172.21.15.18:0/4064646930 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7fce140574f0 con 0x7fce281059d0 2024-04-12T21:17:58.517 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:58.521+0000 7fce2fa00700 1 -- 172.21.15.18:0/4064646930 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7fce2804c710 con 0x7fce281059d0 2024-04-12T21:17:58.518 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:58.522+0000 7fce24c00700 1 -- 172.21.15.18:0/4064646930 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+233 (secure 0 0 0) 0x7fce1405abb0 con 0x7fce281059d0 2024-04-12T21:17:58.518 INFO:teuthology.orchestra.run.smithi018.stdout:HEALTH_ERR 1 filesystem is degraded; 1 filesystem is offline 2024-04-12T21:17:58.518 INFO:teuthology.orchestra.run.smithi018.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-04-12T21:17:58.518 INFO:teuthology.orchestra.run.smithi018.stdout: fs cephfs is degraded 2024-04-12T21:17:58.518 INFO:teuthology.orchestra.run.smithi018.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-04-12T21:17:58.518 INFO:teuthology.orchestra.run.smithi018.stdout: fs cephfs is offline because no MDS is active for it. 2024-04-12T21:17:58.521 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:58.525+0000 7fce2fa00700 1 -- 172.21.15.18:0/4064646930 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fce1006ec00 msgr2=0x7fce100710b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:17:58.521 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:58.525+0000 7fce2fa00700 1 --2- 172.21.15.18:0/4064646930 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fce1006ec00 0x7fce100710b0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fce1c00b740 tx=0x7fce1c004d10 comp rx=0 tx=0).stop 2024-04-12T21:17:58.521 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:58.525+0000 7fce2fa00700 1 -- 172.21.15.18:0/4064646930 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fce281059d0 msgr2=0x7fce28116190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:17:58.521 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:58.525+0000 7fce2fa00700 1 --2- 172.21.15.18:0/4064646930 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fce281059d0 0x7fce28116190 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7fce1400b790 tx=0x7fce14009220 comp rx=0 tx=0).stop 2024-04-12T21:17:58.521 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:58.525+0000 7fce2fa00700 1 -- 172.21.15.18:0/4064646930 shutdown_connections 2024-04-12T21:17:58.522 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:58.525+0000 7fce2fa00700 1 --2- 172.21.15.18:0/4064646930 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fce1006ec00 0x7fce100710b0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:17:58.522 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:58.525+0000 7fce2fa00700 1 --2- 172.21.15.18:0/4064646930 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fce28105030 0x7fce28115c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:17:58.522 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:58.525+0000 7fce2fa00700 1 --2- 172.21.15.18:0/4064646930 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fce281059d0 0x7fce28116190 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:17:58.522 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:58.525+0000 7fce2fa00700 1 -- 172.21.15.18:0/4064646930 >> 172.21.15.18:0/4064646930 conn(0x7fce280faba0 msgr2=0x7fce280fbb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:17:58.522 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:58.525+0000 7fce2fa00700 1 -- 172.21.15.18:0/4064646930 shutdown_connections 2024-04-12T21:17:58.522 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:17:58.525+0000 7fce2fa00700 1 -- 172.21.15.18:0/4064646930 wait complete. 2024-04-12T21:17:58.778 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:17:58 smithi186 ceph-mon[86379]: from='client.? 172.21.15.18:0/2604689820' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-12T21:17:58.849 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:17:58 smithi018 ceph-mon[112590]: from='client.? 172.21.15.18:0/2604689820' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-12T21:17:59.778 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:17:59 smithi186 ceph-mon[86379]: from='client.34996 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:17:59.779 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:17:59 smithi186 ceph-mon[86379]: pgmap v545: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:17:59.779 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:17:59 smithi186 ceph-mon[86379]: from='client.? 172.21.15.18:0/4064646930' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-12T21:17:59.849 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:17:59 smithi018 ceph-mon[112590]: from='client.34996 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:17:59.849 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:17:59 smithi018 ceph-mon[112590]: pgmap v545: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:17:59.850 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:17:59 smithi018 ceph-mon[112590]: from='client.? 172.21.15.18:0/4064646930' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-12T21:18:01.778 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:18:01 smithi186 ceph-mon[86379]: pgmap v546: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:18:01.849 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:18:01 smithi018 ceph-mon[112590]: pgmap v546: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:18:03.778 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:18:03 smithi186 ceph-mon[86379]: pgmap v547: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:18:03.849 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:18:03 smithi018 ceph-mon[112590]: pgmap v547: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:18:05.778 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:18:05 smithi186 ceph-mon[86379]: pgmap v548: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:18:05.849 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:18:05 smithi018 ceph-mon[112590]: pgmap v548: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:18:07.778 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:18:07 smithi186 ceph-mon[86379]: pgmap v549: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:18:07.849 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:18:07 smithi018 ceph-mon[112590]: pgmap v549: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:18:09.849 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:18:09 smithi018 ceph-mon[112590]: pgmap v550: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:18:10.028 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:18:09 smithi186 ceph-mon[86379]: pgmap v550: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:18:10.849 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:18:10 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T21:18:11.028 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:18:10 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T21:18:11.849 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:18:11 smithi018 ceph-mon[112590]: pgmap v551: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:18:12.028 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:18:11 smithi186 ceph-mon[86379]: pgmap v551: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:18:13.849 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:18:13 smithi018 ceph-mon[112590]: pgmap v552: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:18:14.028 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:18:13 smithi186 ceph-mon[86379]: pgmap v552: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:18:15.849 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:18:15 smithi018 ceph-mon[112590]: pgmap v553: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:18:16.028 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:18:15 smithi186 ceph-mon[86379]: pgmap v553: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:18:17.849 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:18:17 smithi018 ceph-mon[112590]: pgmap v554: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:18:18.028 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:18:17 smithi186 ceph-mon[86379]: pgmap v554: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:18:19.849 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:18:19 smithi018 ceph-mon[112590]: pgmap v555: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:18:20.028 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:18:19 smithi186 ceph-mon[86379]: pgmap v555: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:18:21.849 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:18:21 smithi018 ceph-mon[112590]: pgmap v556: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:18:22.028 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:18:21 smithi186 ceph-mon[86379]: pgmap v556: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:18:23.849 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:18:23 smithi018 ceph-mon[112590]: pgmap v557: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:18:24.028 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:18:23 smithi186 ceph-mon[86379]: pgmap v557: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:18:25.849 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:18:25 smithi018 ceph-mon[112590]: pgmap v558: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:18:25.850 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:18:25 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T21:18:26.028 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:18:25 smithi186 ceph-mon[86379]: pgmap v558: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:18:26.028 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:18:25 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T21:18:27.849 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:18:27 smithi018 ceph-mon[112590]: pgmap v559: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:18:28.028 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:18:27 smithi186 ceph-mon[86379]: pgmap v559: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:18:28.697 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:18:28.701+0000 7f72f6c00700 1 -- 172.21.15.18:0/617555752 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f72f01092a0 msgr2=0x7f72f0109670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:18:28.697 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:18:28.701+0000 7f72f6c00700 1 --2- 172.21.15.18:0/617555752 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f72f01092a0 0x7f72f0109670 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f72e4004940 tx=0x7f72e4009e30 comp rx=0 tx=0).stop 2024-04-12T21:18:28.698 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:18:28.701+0000 7f72f6c00700 1 -- 172.21.15.18:0/617555752 shutdown_connections 2024-04-12T21:18:28.698 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:18:28.701+0000 7f72f6c00700 1 --2- 172.21.15.18:0/617555752 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f72f0109c40 0x7f72f01032b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:18:28.698 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:18:28.701+0000 7f72f6c00700 1 --2- 172.21.15.18:0/617555752 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f72f01092a0 0x7f72f0109670 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:18:28.698 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:18:28.701+0000 7f72f6c00700 1 -- 172.21.15.18:0/617555752 >> 172.21.15.18:0/617555752 conn(0x7f72f0072a30 msgr2=0x7f72f0074e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:18:28.698 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:18:28.702+0000 7f72f6c00700 1 -- 172.21.15.18:0/617555752 shutdown_connections 2024-04-12T21:18:28.698 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:18:28.702+0000 7f72f6c00700 1 -- 172.21.15.18:0/617555752 wait complete. 2024-04-12T21:18:28.699 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:18:28.702+0000 7f72f6c00700 1 Processor -- start 2024-04-12T21:18:28.699 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:18:28.703+0000 7f72f6c00700 1 -- start start 2024-04-12T21:18:28.700 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:18:28.703+0000 7f72f6c00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f72f01092a0 0x7f72f0119f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:18:28.700 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:18:28.703+0000 7f72f6c00700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f72f0109c40 0x7f72f011a4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:18:28.700 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:18:28.703+0000 7f72f6c00700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72f011aaf0 con 0x7f72f01092a0 2024-04-12T21:18:28.700 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:18:28.703+0000 7f72f6c00700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72f0113ec0 con 0x7f72f0109c40 2024-04-12T21:18:28.700 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:18:28.703+0000 7f72f5800700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f72f01092a0 0x7f72f0119f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:18:28.700 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:18:28.703+0000 7f72f5800700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f72f01092a0 0x7f72f0119f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:34964/0 (socket says 172.21.15.18:34964) 2024-04-12T21:18:28.700 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:18:28.703+0000 7f72f5800700 1 -- 172.21.15.18:0/1317185573 learned_addr learned my addr 172.21.15.18:0/1317185573 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:18:28.701 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:18:28.704+0000 7f72f5800700 1 -- 172.21.15.18:0/1317185573 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f72f0109c40 msgr2=0x7f72f011a4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:18:28.701 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:18:28.704+0000 7f72f5800700 1 --2- 172.21.15.18:0/1317185573 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f72f0109c40 0x7f72f011a4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:18:28.701 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:18:28.704+0000 7f72f5800700 1 -- 172.21.15.18:0/1317185573 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72e4005040 con 0x7f72f01092a0 2024-04-12T21:18:28.701 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:18:28.704+0000 7f72f5800700 1 --2- 172.21.15.18:0/1317185573 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f72f01092a0 0x7f72f0119f60 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f72e40090d0 tx=0x7f72e4009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:18:28.701 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:18:28.704+0000 7f72ed600700 1 -- 172.21.15.18:0/1317185573 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f72e4021070 con 0x7f72f01092a0 2024-04-12T21:18:28.701 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:18:28.705+0000 7f72ed600700 1 -- 172.21.15.18:0/1317185573 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f72e4013c00 con 0x7f72f01092a0 2024-04-12T21:18:28.701 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:18:28.705+0000 7f72ed600700 1 -- 172.21.15.18:0/1317185573 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f72e400e070 con 0x7f72f01092a0 2024-04-12T21:18:28.701 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:18:28.705+0000 7f72f6c00700 1 -- 172.21.15.18:0/1317185573 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f72f0114140 con 0x7f72f01092a0 2024-04-12T21:18:28.702 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:18:28.705+0000 7f72f6c00700 1 -- 172.21.15.18:0/1317185573 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f72f01146a0 con 0x7f72f01092a0 2024-04-12T21:18:28.702 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:18:28.706+0000 7f72f6c00700 1 -- 172.21.15.18:0/1317185573 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f72f01049f0 con 0x7f72f01092a0 2024-04-12T21:18:28.703 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:18:28.707+0000 7f72ed600700 1 -- 172.21.15.18:0/1317185573 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7f72e401c400 con 0x7f72f01092a0 2024-04-12T21:18:28.705 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:18:28.708+0000 7f72ed600700 1 --2- 172.21.15.18:0/1317185573 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f72d806ec00 0x7f72d80710b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:18:28.705 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:18:28.709+0000 7f72ed600700 1 -- 172.21.15.18:0/1317185573 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 6757+0+0 (secure 0 0 0) 0x7f72e408ff10 con 0x7f72f01092a0 2024-04-12T21:18:28.705 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:18:28.709+0000 7f72f4e00700 1 --2- 172.21.15.18:0/1317185573 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f72d806ec00 0x7f72d80710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:18:28.705 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:18:28.709+0000 7f72f4e00700 1 --2- 172.21.15.18:0/1317185573 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f72d806ec00 0x7f72d80710b0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f72f0115800 tx=0x7f72dc00b040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:18:28.711 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:18:28.715+0000 7f72ed600700 1 -- 172.21.15.18:0/1317185573 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f72e405c740 con 0x7f72f01092a0 2024-04-12T21:18:28.969 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:18:28.973+0000 7f72f6c00700 1 -- 172.21.15.18:0/1317185573 --> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f72f0110390 con 0x7f72d806ec00 2024-04-12T21:18:28.971 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:18:28.975+0000 7f72ed600700 1 -- 172.21.15.18:0/1317185573 <== mgr.34102 v2:172.21.15.18:6800/3812891919 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f72f0110390 con 0x7f72d806ec00 2024-04-12T21:18:28.974 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:18:28.978+0000 7f72f6c00700 1 -- 172.21.15.18:0/1317185573 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f72d806ec00 msgr2=0x7f72d80710b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:18:28.974 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:18:28.978+0000 7f72f6c00700 1 --2- 172.21.15.18:0/1317185573 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f72d806ec00 0x7f72d80710b0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f72f0115800 tx=0x7f72dc00b040 comp rx=0 tx=0).stop 2024-04-12T21:18:28.974 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:18:28.978+0000 7f72f6c00700 1 -- 172.21.15.18:0/1317185573 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f72f01092a0 msgr2=0x7f72f0119f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:18:28.974 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:18:28.978+0000 7f72f6c00700 1 --2- 172.21.15.18:0/1317185573 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f72f01092a0 0x7f72f0119f60 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f72e40090d0 tx=0x7f72e4009e30 comp rx=0 tx=0).stop 2024-04-12T21:18:28.975 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:18:28.978+0000 7f72f6c00700 1 -- 172.21.15.18:0/1317185573 shutdown_connections 2024-04-12T21:18:28.975 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:18:28.978+0000 7f72f6c00700 1 --2- 172.21.15.18:0/1317185573 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f72d806ec00 0x7f72d80710b0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:18:28.975 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:18:28.978+0000 7f72f6c00700 1 --2- 172.21.15.18:0/1317185573 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f72f01092a0 0x7f72f0119f60 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:18:28.975 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:18:28.978+0000 7f72f6c00700 1 --2- 172.21.15.18:0/1317185573 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f72f0109c40 0x7f72f011a4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:18:28.975 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:18:28.978+0000 7f72f6c00700 1 -- 172.21.15.18:0/1317185573 >> 172.21.15.18:0/1317185573 conn(0x7f72f0072a30 msgr2=0x7f72f00748d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:18:28.975 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:18:28.978+0000 7f72f6c00700 1 -- 172.21.15.18:0/1317185573 shutdown_connections 2024-04-12T21:18:28.975 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:18:28.978+0000 7f72f6c00700 1 -- 172.21.15.18:0/1317185573 wait complete. 2024-04-12T21:18:28.991 INFO:teuthology.orchestra.run.smithi018.stdout:true 2024-04-12T21:18:29.140 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:18:29.143+0000 7fb146200700 1 -- 172.21.15.18:0/3733961548 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb140071f90 msgr2=0x7fb140070360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:18:29.140 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:18:29.143+0000 7fb146200700 1 --2- 172.21.15.18:0/3733961548 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb140071f90 0x7fb140070360 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7fb130004970 tx=0x7fb130009e30 comp rx=0 tx=0).stop 2024-04-12T21:18:29.140 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:18:29.144+0000 7fb146200700 1 -- 172.21.15.18:0/3733961548 shutdown_connections 2024-04-12T21:18:29.140 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:18:29.144+0000 7fb146200700 1 --2- 172.21.15.18:0/3733961548 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb1400708a0 0x7fb140070d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:18:29.140 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:18:29.144+0000 7fb146200700 1 --2- 172.21.15.18:0/3733961548 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb140071f90 0x7fb140070360 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:18:29.140 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:18:29.144+0000 7fb146200700 1 -- 172.21.15.18:0/3733961548 >> 172.21.15.18:0/3733961548 conn(0x7fb1400f8970 msgr2=0x7fb1400fad80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:18:29.140 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:18:29.144+0000 7fb146200700 1 -- 172.21.15.18:0/3733961548 shutdown_connections 2024-04-12T21:18:29.141 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:18:29.144+0000 7fb146200700 1 -- 172.21.15.18:0/3733961548 wait complete. 2024-04-12T21:18:29.142 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:18:29.144+0000 7fb146200700 1 Processor -- start 2024-04-12T21:18:29.142 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:18:29.145+0000 7fb146200700 1 -- start start 2024-04-12T21:18:29.142 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:18:29.145+0000 7fb146200700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb1400708a0 0x7fb14011a160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:18:29.142 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:18:29.145+0000 7fb146200700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb140071f90 0x7fb14011a6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:18:29.142 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:18:29.145+0000 7fb146200700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb14011acf0 con 0x7fb140071f90 2024-04-12T21:18:29.142 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:18:29.145+0000 7fb146200700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1401140c0 con 0x7fb1400708a0 2024-04-12T21:18:29.142 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:18:29.145+0000 7fb137e00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb140071f90 0x7fb14011a6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:18:29.143 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:18:29.145+0000 7fb137e00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb140071f90 0x7fb14011a6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:34982/0 (socket says 172.21.15.18:34982) 2024-04-12T21:18:29.143 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:18:29.145+0000 7fb137e00700 1 -- 172.21.15.18:0/1064212262 learned_addr learned my addr 172.21.15.18:0/1064212262 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:18:29.143 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:18:29.146+0000 7fb13f400700 1 --2- 172.21.15.18:0/1064212262 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb1400708a0 0x7fb14011a160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:18:29.143 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:18:29.146+0000 7fb137e00700 1 -- 172.21.15.18:0/1064212262 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb1400708a0 msgr2=0x7fb14011a160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:18:29.143 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:18:29.146+0000 7fb137e00700 1 --2- 172.21.15.18:0/1064212262 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb1400708a0 0x7fb14011a160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:18:29.143 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:18:29.146+0000 7fb137e00700 1 -- 172.21.15.18:0/1064212262 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb130005040 con 0x7fb140071f90 2024-04-12T21:18:29.143 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:18:29.146+0000 7fb137e00700 1 --2- 172.21.15.18:0/1064212262 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb140071f90 0x7fb14011a6a0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7fb12800c830 tx=0x7fb128004470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:18:29.144 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:18:29.146+0000 7fb137400700 1 -- 172.21.15.18:0/1064212262 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb12800a760 con 0x7fb140071f90 2024-04-12T21:18:29.144 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:18:29.147+0000 7fb137400700 1 -- 172.21.15.18:0/1064212262 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb12800ad90 con 0x7fb140071f90 2024-04-12T21:18:29.144 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:18:29.147+0000 7fb137400700 1 -- 172.21.15.18:0/1064212262 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb1280113a0 con 0x7fb140071f90 2024-04-12T21:18:29.144 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:18:29.147+0000 7fb146200700 1 -- 172.21.15.18:0/1064212262 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb1401143a0 con 0x7fb140071f90 2024-04-12T21:18:29.144 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:18:29.147+0000 7fb146200700 1 -- 172.21.15.18:0/1064212262 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb1401148e0 con 0x7fb140071f90 2024-04-12T21:18:29.144 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:18:29.148+0000 7fb146200700 1 -- 172.21.15.18:0/1064212262 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb140071aa0 con 0x7fb140071f90 2024-04-12T21:18:29.145 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:18:29.149+0000 7fb137400700 1 -- 172.21.15.18:0/1064212262 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7fb128012400 con 0x7fb140071f90 2024-04-12T21:18:29.146 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:18:29.150+0000 7fb137400700 1 --2- 172.21.15.18:0/1064212262 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fb12406ec10 0x7fb1240710c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:18:29.146 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:18:29.150+0000 7fb137400700 1 -- 172.21.15.18:0/1064212262 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 6757+0+0 (secure 0 0 0) 0x7fb12808e9c0 con 0x7fb140071f90 2024-04-12T21:18:29.146 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:18:29.150+0000 7fb13f400700 1 --2- 172.21.15.18:0/1064212262 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fb12406ec10 0x7fb1240710c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:18:29.147 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:18:29.151+0000 7fb13f400700 1 --2- 172.21.15.18:0/1064212262 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fb12406ec10 0x7fb1240710c0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fb130004f50 tx=0x7fb130009e30 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:18:29.151 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:18:29.155+0000 7fb137400700 1 -- 172.21.15.18:0/1064212262 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7fb12805b1f0 con 0x7fb140071f90 2024-04-12T21:18:29.406 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:18:29.410+0000 7fb146200700 1 -- 172.21.15.18:0/1064212262 --> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fb1401105a0 con 0x7fb12406ec10 2024-04-12T21:18:29.408 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:18:29.412+0000 7fb137400700 1 -- 172.21.15.18:0/1064212262 <== mgr.34102 v2:172.21.15.18:6800/3812891919 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7fb130008d60 con 0x7fb12406ec10 2024-04-12T21:18:29.411 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:18:29.415+0000 7fb146200700 1 -- 172.21.15.18:0/1064212262 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fb12406ec10 msgr2=0x7fb1240710c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:18:29.412 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:18:29.415+0000 7fb146200700 1 --2- 172.21.15.18:0/1064212262 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fb12406ec10 0x7fb1240710c0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fb130004f50 tx=0x7fb130009e30 comp rx=0 tx=0).stop 2024-04-12T21:18:29.412 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:18:29.415+0000 7fb146200700 1 -- 172.21.15.18:0/1064212262 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb140071f90 msgr2=0x7fb14011a6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:18:29.412 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:18:29.415+0000 7fb146200700 1 --2- 172.21.15.18:0/1064212262 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb140071f90 0x7fb14011a6a0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7fb12800c830 tx=0x7fb128004470 comp rx=0 tx=0).stop 2024-04-12T21:18:29.412 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:18:29.415+0000 7fb146200700 1 -- 172.21.15.18:0/1064212262 shutdown_connections 2024-04-12T21:18:29.412 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:18:29.415+0000 7fb146200700 1 --2- 172.21.15.18:0/1064212262 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fb12406ec10 0x7fb1240710c0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:18:29.412 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:18:29.415+0000 7fb146200700 1 --2- 172.21.15.18:0/1064212262 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb1400708a0 0x7fb14011a160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:18:29.412 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:18:29.415+0000 7fb146200700 1 --2- 172.21.15.18:0/1064212262 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb140071f90 0x7fb14011a6a0 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:18:29.412 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:18:29.415+0000 7fb146200700 1 -- 172.21.15.18:0/1064212262 >> 172.21.15.18:0/1064212262 conn(0x7fb1400f8970 msgr2=0x7fb1400f9a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:18:29.412 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:18:29.415+0000 7fb146200700 1 -- 172.21.15.18:0/1064212262 shutdown_connections 2024-04-12T21:18:29.412 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:18:29.416+0000 7fb146200700 1 -- 172.21.15.18:0/1064212262 wait complete. 2024-04-12T21:18:29.571 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:18:29.574+0000 7f4924a00700 1 -- 172.21.15.18:0/1870284485 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4920105800 msgr2=0x7f49200fee70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:18:29.571 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:18:29.574+0000 7f4924a00700 1 --2- 172.21.15.18:0/1870284485 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4920105800 0x7f49200fee70 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f490c004940 tx=0x7f490c009e30 comp rx=0 tx=0).stop 2024-04-12T21:18:29.571 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:18:29.574+0000 7f4924a00700 1 -- 172.21.15.18:0/1870284485 shutdown_connections 2024-04-12T21:18:29.571 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:18:29.574+0000 7f4924a00700 1 --2- 172.21.15.18:0/1870284485 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4920105800 0x7f49200fee70 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:18:29.571 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:18:29.574+0000 7f4924a00700 1 --2- 172.21.15.18:0/1870284485 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4920104e60 0x7f4920105230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:18:29.571 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:18:29.574+0000 7f4924a00700 1 -- 172.21.15.18:0/1870284485 >> 172.21.15.18:0/1870284485 conn(0x7f49200fab10 msgr2=0x7f49200fcef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:18:29.571 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:18:29.574+0000 7f4924a00700 1 -- 172.21.15.18:0/1870284485 shutdown_connections 2024-04-12T21:18:29.571 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:18:29.574+0000 7f4924a00700 1 -- 172.21.15.18:0/1870284485 wait complete. 2024-04-12T21:18:29.572 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:18:29.575+0000 7f4924a00700 1 Processor -- start 2024-04-12T21:18:29.573 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:18:29.575+0000 7f4924a00700 1 -- start start 2024-04-12T21:18:29.573 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:18:29.576+0000 7f4924a00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4920104e60 0x7f4920115c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:18:29.573 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:18:29.576+0000 7f4924a00700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4920105800 0x7f4920116150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:18:29.573 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:18:29.576+0000 7f4924a00700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49201167a0 con 0x7f4920104e60 2024-04-12T21:18:29.573 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:18:29.576+0000 7f4924a00700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f492010fb90 con 0x7f4920105800 2024-04-12T21:18:29.573 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:18:29.576+0000 7f491e800700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4920104e60 0x7f4920115c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:18:29.573 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:18:29.576+0000 7f491e800700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4920104e60 0x7f4920115c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:34994/0 (socket says 172.21.15.18:34994) 2024-04-12T21:18:29.573 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:18:29.576+0000 7f491e800700 1 -- 172.21.15.18:0/3426438009 learned_addr learned my addr 172.21.15.18:0/3426438009 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:18:29.573 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:18:29.577+0000 7f491de00700 1 --2- 172.21.15.18:0/3426438009 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4920105800 0x7f4920116150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:18:29.573 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:18:29.577+0000 7f491e800700 1 -- 172.21.15.18:0/3426438009 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4920105800 msgr2=0x7f4920116150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:18:29.573 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:18:29.577+0000 7f491e800700 1 --2- 172.21.15.18:0/3426438009 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4920105800 0x7f4920116150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:18:29.573 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:18:29.577+0000 7f491e800700 1 -- 172.21.15.18:0/3426438009 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f490c005040 con 0x7f4920104e60 2024-04-12T21:18:29.574 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:18:29.577+0000 7f491e800700 1 --2- 172.21.15.18:0/3426438009 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4920104e60 0x7f4920115c10 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f4910002a50 tx=0x7f49100046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:18:29.574 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:18:29.577+0000 7f4916a00700 1 -- 172.21.15.18:0/3426438009 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4910007cc0 con 0x7f4920104e60 2024-04-12T21:18:29.574 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:18:29.577+0000 7f4916a00700 1 -- 172.21.15.18:0/3426438009 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f491000f040 con 0x7f4920104e60 2024-04-12T21:18:29.574 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:18:29.577+0000 7f4916a00700 1 -- 172.21.15.18:0/3426438009 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f491000db70 con 0x7f4920104e60 2024-04-12T21:18:29.575 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:18:29.577+0000 7f4924a00700 1 -- 172.21.15.18:0/3426438009 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f492010fe70 con 0x7f4920104e60 2024-04-12T21:18:29.575 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:18:29.578+0000 7f4924a00700 1 -- 172.21.15.18:0/3426438009 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4920110430 con 0x7f4920104e60 2024-04-12T21:18:29.575 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:18:29.579+0000 7f4924a00700 1 -- 172.21.15.18:0/3426438009 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f49201005b0 con 0x7f4920104e60 2024-04-12T21:18:29.576 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:18:29.580+0000 7f4916a00700 1 -- 172.21.15.18:0/3426438009 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7f491000dd10 con 0x7f4920104e60 2024-04-12T21:18:29.577 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:18:29.581+0000 7f4916a00700 1 --2- 172.21.15.18:0/3426438009 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f490406ec00 0x7f49040710b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:18:29.577 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:18:29.581+0000 7f4916a00700 1 -- 172.21.15.18:0/3426438009 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 6757+0+0 (secure 0 0 0) 0x7f491008e0b0 con 0x7f4920104e60 2024-04-12T21:18:29.577 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:18:29.581+0000 7f491de00700 1 --2- 172.21.15.18:0/3426438009 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f490406ec00 0x7f49040710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:18:29.578 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:18:29.582+0000 7f491de00700 1 --2- 172.21.15.18:0/3426438009 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f490406ec00 0x7f49040710b0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f4920111440 tx=0x7f490c009e30 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:18:29.581 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:18:29.586+0000 7f4916a00700 1 -- 172.21.15.18:0/3426438009 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f491005a880 con 0x7f4920104e60 2024-04-12T21:18:29.820 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:18:29.823+0000 7f4924a00700 1 -- 172.21.15.18:0/3426438009 --> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f492004bed0 con 0x7f490406ec00 2024-04-12T21:18:29.820 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:18:29 smithi018 ceph-mon[112590]: pgmap v560: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:18:29.840 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:18:29.844+0000 7f4916a00700 1 -- 172.21.15.18:0/3426438009 <== mgr.34102 v2:172.21.15.18:6800/3812891919 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7f492004bed0 con 0x7f490406ec00 2024-04-12T21:18:29.841 INFO:teuthology.orchestra.run.smithi018.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-12T21:18:29.841 INFO:teuthology.orchestra.run.smithi018.stdout:alertmanager.smithi018 smithi018 *:9093,9094 running (19m) 10m ago 33m 14.7M - 0.25.0 c8568f914cd2 c332090068e8 2024-04-12T21:18:29.841 INFO:teuthology.orchestra.run.smithi018.stdout:crash.smithi018 smithi018 running (15m) 10m ago 33m 6665k - 19.0.0-2879-g324baf8d 8717720dc5e5 b7f80d1f7982 2024-04-12T21:18:29.841 INFO:teuthology.orchestra.run.smithi018.stdout:crash.smithi186 smithi186 running (15m) 11m ago 32m 6660k - 19.0.0-2879-g324baf8d 8717720dc5e5 a4f55c81b623 2024-04-12T21:18:29.841 INFO:teuthology.orchestra.run.smithi018.stdout:grafana.smithi018 smithi018 *:3000 running (18m) 10m ago 33m 74.8M - 9.4.12 2bacad6d85d8 b08e3bf176dd 2024-04-12T21:18:29.841 INFO:teuthology.orchestra.run.smithi018.stdout:mds.cephfs.smithi018.acqbsc smithi018 running (11m) 10m ago 26m 11.1M - 19.0.0-2879-g324baf8d 8717720dc5e5 021f99ab24ae 2024-04-12T21:18:29.841 INFO:teuthology.orchestra.run.smithi018.stdout:mds.cephfs.smithi018.szjluo smithi018 running (10m) 10m ago 26m 10.9M - 19.0.0-2879-g324baf8d 8717720dc5e5 0003117479d4 2024-04-12T21:18:29.841 INFO:teuthology.orchestra.run.smithi018.stdout:mds.cephfs.smithi186.jdllyn smithi186 running (26m) 11m ago 26m 15.6M - 17.2.7-827-gd3b5060a 21de9b7369ad 2b2bc46bd29a 2024-04-12T21:18:29.842 INFO:teuthology.orchestra.run.smithi018.stdout:mds.cephfs.smithi186.wjnafs smithi186 running (26m) 11m ago 26m 15.4M - 17.2.7-827-gd3b5060a 21de9b7369ad 74b115d6d6fd 2024-04-12T21:18:29.842 INFO:teuthology.orchestra.run.smithi018.stdout:mgr.smithi018.xjzwga smithi018 *:8443,9283,8765 running (21m) 10m ago 35m 509M - 19.0.0-2879-g324baf8d 8717720dc5e5 56088305eab7 2024-04-12T21:18:29.842 INFO:teuthology.orchestra.run.smithi018.stdout:mgr.smithi186.kqcfxk smithi186 *:8443,9283,8765 running (20m) 11m ago 32m 452M - 19.0.0-2879-g324baf8d 8717720dc5e5 09cd766e5e0d 2024-04-12T21:18:29.842 INFO:teuthology.orchestra.run.smithi018.stdout:mon.smithi018 smithi018 running (17m) 10m ago 35m 57.0M 2048M 19.0.0-2879-g324baf8d 8717720dc5e5 df3e201dfed6 2024-04-12T21:18:29.842 INFO:teuthology.orchestra.run.smithi018.stdout:mon.smithi186 smithi186 running (16m) 11m ago 32m 37.7M 2048M 19.0.0-2879-g324baf8d 8717720dc5e5 c77498da9b2b 2024-04-12T21:18:29.842 INFO:teuthology.orchestra.run.smithi018.stdout:node-exporter.smithi018 smithi018 *:9100 running (20m) 10m ago 33m 7780k - 1.5.0 0da6a335fe13 326c3d9c6847 2024-04-12T21:18:29.842 INFO:teuthology.orchestra.run.smithi018.stdout:node-exporter.smithi186 smithi186 *:9100 running (20m) 11m ago 32m 7826k - 1.5.0 0da6a335fe13 ec2112baeecf 2024-04-12T21:18:29.842 INFO:teuthology.orchestra.run.smithi018.stdout:osd.0 smithi018 running (14m) 10m ago 31m 121M 1559M 19.0.0-2879-g324baf8d 8717720dc5e5 60b1273ca047 2024-04-12T21:18:29.842 INFO:teuthology.orchestra.run.smithi018.stdout:osd.1 smithi018 running (14m) 10m ago 30m 91.3M 1559M 19.0.0-2879-g324baf8d 8717720dc5e5 170ce1e5fb8d 2024-04-12T21:18:29.842 INFO:teuthology.orchestra.run.smithi018.stdout:osd.2 smithi018 running (13m) 10m ago 29m 90.3M 1559M 19.0.0-2879-g324baf8d 8717720dc5e5 c2cbc6aa2839 2024-04-12T21:18:29.842 INFO:teuthology.orchestra.run.smithi018.stdout:osd.3 smithi186 running (13m) 11m ago 29m 96.6M 2581M 19.0.0-2879-g324baf8d 8717720dc5e5 0138e8262e4b 2024-04-12T21:18:29.842 INFO:teuthology.orchestra.run.smithi018.stdout:osd.4 smithi186 running (12m) 11m ago 28m 87.2M 2581M 19.0.0-2879-g324baf8d 8717720dc5e5 b20cb63b25c9 2024-04-12T21:18:29.842 INFO:teuthology.orchestra.run.smithi018.stdout:osd.5 smithi186 running (11m) 11m ago 28m 11.1M 2581M 19.0.0-2879-g324baf8d 8717720dc5e5 2513bea2610e 2024-04-12T21:18:29.842 INFO:teuthology.orchestra.run.smithi018.stdout:prometheus.smithi018 smithi018 *:9095 running (19m) 10m ago 32m 60.6M - 2.43.0 a07b618ecd1d f58841127e21 2024-04-12T21:18:29.844 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:18:29.848+0000 7f4924a00700 1 -- 172.21.15.18:0/3426438009 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f490406ec00 msgr2=0x7f49040710b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:18:29.844 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:18:29.848+0000 7f4924a00700 1 --2- 172.21.15.18:0/3426438009 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f490406ec00 0x7f49040710b0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f4920111440 tx=0x7f490c009e30 comp rx=0 tx=0).stop 2024-04-12T21:18:29.844 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:18:29.848+0000 7f4924a00700 1 -- 172.21.15.18:0/3426438009 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4920104e60 msgr2=0x7f4920115c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:18:29.845 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:18:29.848+0000 7f4924a00700 1 --2- 172.21.15.18:0/3426438009 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4920104e60 0x7f4920115c10 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f4910002a50 tx=0x7f49100046b0 comp rx=0 tx=0).stop 2024-04-12T21:18:29.845 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:18:29.848+0000 7f4924a00700 1 -- 172.21.15.18:0/3426438009 shutdown_connections 2024-04-12T21:18:29.845 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:18:29.848+0000 7f4924a00700 1 --2- 172.21.15.18:0/3426438009 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f490406ec00 0x7f49040710b0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:18:29.845 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:18:29.848+0000 7f4924a00700 1 --2- 172.21.15.18:0/3426438009 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4920104e60 0x7f4920115c10 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:18:29.845 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:18:29.848+0000 7f4924a00700 1 --2- 172.21.15.18:0/3426438009 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4920105800 0x7f4920116150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:18:29.845 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:18:29.848+0000 7f4924a00700 1 -- 172.21.15.18:0/3426438009 >> 172.21.15.18:0/3426438009 conn(0x7f49200fab10 msgr2=0x7f49200fcab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:18:29.845 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:18:29.848+0000 7f4924a00700 1 -- 172.21.15.18:0/3426438009 shutdown_connections 2024-04-12T21:18:29.845 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:18:29.848+0000 7f4924a00700 1 -- 172.21.15.18:0/3426438009 wait complete. 2024-04-12T21:18:29.988 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:18:29.992+0000 7f7e6cc00700 1 -- 172.21.15.18:0/1970302101 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7e68102710 msgr2=0x7f7e680fc520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:18:29.988 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:18:29.992+0000 7f7e6cc00700 1 --2- 172.21.15.18:0/1970302101 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7e68102710 0x7f7e680fc520 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f7e54004940 tx=0x7f7e54009e30 comp rx=0 tx=0).stop 2024-04-12T21:18:29.988 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:18:29.992+0000 7f7e6cc00700 1 -- 172.21.15.18:0/1970302101 shutdown_connections 2024-04-12T21:18:29.989 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:18:29.992+0000 7f7e6cc00700 1 --2- 172.21.15.18:0/1970302101 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7e68102710 0x7f7e680fc520 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:18:29.989 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:18:29.992+0000 7f7e6cc00700 1 --2- 172.21.15.18:0/1970302101 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7e68101d70 0x7f7e68102140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:18:29.989 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:18:29.992+0000 7f7e6cc00700 1 -- 172.21.15.18:0/1970302101 >> 172.21.15.18:0/1970302101 conn(0x7f7e68072370 msgr2=0x7f7e68072770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:18:29.989 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:18:29.992+0000 7f7e6cc00700 1 -- 172.21.15.18:0/1970302101 shutdown_connections 2024-04-12T21:18:29.989 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:18:29.992+0000 7f7e6cc00700 1 -- 172.21.15.18:0/1970302101 wait complete. 2024-04-12T21:18:29.990 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:18:29.993+0000 7f7e6cc00700 1 Processor -- start 2024-04-12T21:18:29.990 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:18:29.993+0000 7f7e6cc00700 1 -- start start 2024-04-12T21:18:29.990 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:18:29.994+0000 7f7e6cc00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7e68101d70 0x7f7e681139a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:18:29.990 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:18:29.994+0000 7f7e6cc00700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7e68102710 0x7f7e68113ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:18:29.990 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:18:29.994+0000 7f7e6cc00700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e68114530 con 0x7f7e68101d70 2024-04-12T21:18:29.990 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:18:29.994+0000 7f7e6cc00700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e6810db50 con 0x7f7e68102710 2024-04-12T21:18:29.991 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:18:29.994+0000 7f7e65e00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7e68101d70 0x7f7e681139a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:18:29.991 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:18:29.994+0000 7f7e65e00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7e68101d70 0x7f7e681139a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:35010/0 (socket says 172.21.15.18:35010) 2024-04-12T21:18:29.991 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:18:29.994+0000 7f7e65e00700 1 -- 172.21.15.18:0/1247344259 learned_addr learned my addr 172.21.15.18:0/1247344259 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:18:29.991 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:18:29.994+0000 7f7e65400700 1 --2- 172.21.15.18:0/1247344259 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7e68102710 0x7f7e68113ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:18:29.992 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:18:29.994+0000 7f7e65e00700 1 -- 172.21.15.18:0/1247344259 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7e68102710 msgr2=0x7f7e68113ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:18:29.992 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:18:29.994+0000 7f7e65e00700 1 --2- 172.21.15.18:0/1247344259 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7e68102710 0x7f7e68113ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:18:29.992 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:18:29.994+0000 7f7e65e00700 1 -- 172.21.15.18:0/1247344259 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e54005040 con 0x7f7e68101d70 2024-04-12T21:18:29.992 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:18:29.995+0000 7f7e65e00700 1 --2- 172.21.15.18:0/1247344259 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7e68101d70 0x7f7e681139a0 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f7e58002a50 tx=0x7f7e580046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:18:29.992 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:18:29.995+0000 7f7e5e000700 1 -- 172.21.15.18:0/1247344259 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7e58007cc0 con 0x7f7e68101d70 2024-04-12T21:18:29.992 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:18:29.995+0000 7f7e5e000700 1 -- 172.21.15.18:0/1247344259 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f7e5800f040 con 0x7f7e68101d70 2024-04-12T21:18:29.992 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:18:29.995+0000 7f7e6cc00700 1 -- 172.21.15.18:0/1247344259 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7e6810de30 con 0x7f7e68101d70 2024-04-12T21:18:29.992 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:18:29.995+0000 7f7e5e000700 1 -- 172.21.15.18:0/1247344259 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7e58009640 con 0x7f7e68101d70 2024-04-12T21:18:29.992 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:18:29.995+0000 7f7e6cc00700 1 -- 172.21.15.18:0/1247344259 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7e6810e370 con 0x7f7e68101d70 2024-04-12T21:18:29.992 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:18:29.996+0000 7f7e6cc00700 1 -- 172.21.15.18:0/1247344259 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7e680fd530 con 0x7f7e68101d70 2024-04-12T21:18:29.993 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:18:29.997+0000 7f7e5e000700 1 -- 172.21.15.18:0/1247344259 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7f7e580097a0 con 0x7f7e68101d70 2024-04-12T21:18:29.994 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:18:29.998+0000 7f7e5e000700 1 --2- 172.21.15.18:0/1247344259 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f7e4c06ec00 0x7f7e4c0710b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:18:29.995 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:18:29.999+0000 7f7e5e000700 1 -- 172.21.15.18:0/1247344259 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 6757+0+0 (secure 0 0 0) 0x7f7e5808e760 con 0x7f7e68101d70 2024-04-12T21:18:29.995 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:18:29.999+0000 7f7e65400700 1 --2- 172.21.15.18:0/1247344259 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f7e4c06ec00 0x7f7e4c0710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:18:29.995 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:18:30.000+0000 7f7e65400700 1 --2- 172.21.15.18:0/1247344259 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f7e4c06ec00 0x7f7e4c0710b0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f7e6810f220 tx=0x7f7e54009e30 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:18:29.999 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:18:30.003+0000 7f7e5e000700 1 -- 172.21.15.18:0/1247344259 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f7e580573e0 con 0x7f7e68101d70 2024-04-12T21:18:30.028 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:18:29 smithi186 ceph-mon[86379]: pgmap v560: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:18:30.326 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:18:30.330+0000 7f7e6cc00700 1 -- 172.21.15.18:0/1247344259 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f7e6810e5e0 con 0x7f7e68101d70 2024-04-12T21:18:30.326 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:18:30.331+0000 7f7e5e000700 1 -- 172.21.15.18:0/1247344259 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+837 (secure 0 0 0) 0x7f7e5805aaa0 con 0x7f7e68101d70 2024-04-12T21:18:30.327 INFO:teuthology.orchestra.run.smithi018.stdout:{ 2024-04-12T21:18:30.327 INFO:teuthology.orchestra.run.smithi018.stdout: "mon": { 2024-04-12T21:18:30.327 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 19.0.0-2879-g324baf8d (324baf8ddb91dea46cac50cb535c306115791355) squid (dev)": 2 2024-04-12T21:18:30.327 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-12T21:18:30.327 INFO:teuthology.orchestra.run.smithi018.stdout: "mgr": { 2024-04-12T21:18:30.327 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 19.0.0-2879-g324baf8d (324baf8ddb91dea46cac50cb535c306115791355) squid (dev)": 2 2024-04-12T21:18:30.327 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-12T21:18:30.327 INFO:teuthology.orchestra.run.smithi018.stdout: "osd": { 2024-04-12T21:18:30.327 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 19.0.0-2879-g324baf8d (324baf8ddb91dea46cac50cb535c306115791355) squid (dev)": 6 2024-04-12T21:18:30.327 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-12T21:18:30.327 INFO:teuthology.orchestra.run.smithi018.stdout: "mds": { 2024-04-12T21:18:30.328 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 17.2.7-827-gd3b5060a (d3b5060a17449edfae9385cdf792afec109fca5d) quincy (stable)": 2, 2024-04-12T21:18:30.328 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 19.0.0-2879-g324baf8d (324baf8ddb91dea46cac50cb535c306115791355) squid (dev)": 2 2024-04-12T21:18:30.328 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-12T21:18:30.328 INFO:teuthology.orchestra.run.smithi018.stdout: "overall": { 2024-04-12T21:18:30.328 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 17.2.7-827-gd3b5060a (d3b5060a17449edfae9385cdf792afec109fca5d) quincy (stable)": 2, 2024-04-12T21:18:30.328 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 19.0.0-2879-g324baf8d (324baf8ddb91dea46cac50cb535c306115791355) squid (dev)": 12 2024-04-12T21:18:30.328 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-04-12T21:18:30.328 INFO:teuthology.orchestra.run.smithi018.stdout:} 2024-04-12T21:18:30.330 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:18:30.334+0000 7f7e6cc00700 1 -- 172.21.15.18:0/1247344259 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f7e4c06ec00 msgr2=0x7f7e4c0710b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:18:30.330 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:18:30.334+0000 7f7e6cc00700 1 --2- 172.21.15.18:0/1247344259 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f7e4c06ec00 0x7f7e4c0710b0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f7e6810f220 tx=0x7f7e54009e30 comp rx=0 tx=0).stop 2024-04-12T21:18:30.330 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:18:30.334+0000 7f7e6cc00700 1 -- 172.21.15.18:0/1247344259 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7e68101d70 msgr2=0x7f7e681139a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:18:30.331 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:18:30.334+0000 7f7e6cc00700 1 --2- 172.21.15.18:0/1247344259 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7e68101d70 0x7f7e681139a0 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f7e58002a50 tx=0x7f7e580046b0 comp rx=0 tx=0).stop 2024-04-12T21:18:30.331 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:18:30.334+0000 7f7e6cc00700 1 -- 172.21.15.18:0/1247344259 shutdown_connections 2024-04-12T21:18:30.331 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:18:30.334+0000 7f7e6cc00700 1 --2- 172.21.15.18:0/1247344259 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f7e4c06ec00 0x7f7e4c0710b0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:18:30.331 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:18:30.334+0000 7f7e6cc00700 1 --2- 172.21.15.18:0/1247344259 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7e68101d70 0x7f7e681139a0 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:18:30.331 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:18:30.334+0000 7f7e6cc00700 1 --2- 172.21.15.18:0/1247344259 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7e68102710 0x7f7e68113ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:18:30.331 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:18:30.334+0000 7f7e6cc00700 1 -- 172.21.15.18:0/1247344259 >> 172.21.15.18:0/1247344259 conn(0x7f7e68072370 msgr2=0x7f7e68104af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:18:30.331 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:18:30.334+0000 7f7e6cc00700 1 -- 172.21.15.18:0/1247344259 shutdown_connections 2024-04-12T21:18:30.331 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:18:30.334+0000 7f7e6cc00700 1 -- 172.21.15.18:0/1247344259 wait complete. 2024-04-12T21:18:30.484 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:18:30.488+0000 7f6875000700 1 -- 172.21.15.18:0/4094428171 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f68701026b0 msgr2=0x7f68700fbd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:18:30.484 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:18:30.488+0000 7f6875000700 1 --2- 172.21.15.18:0/4094428171 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f68701026b0 0x7f68700fbd20 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f685c004940 tx=0x7f685c009e30 comp rx=0 tx=0).stop 2024-04-12T21:18:30.484 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:18:30.488+0000 7f6875000700 1 -- 172.21.15.18:0/4094428171 shutdown_connections 2024-04-12T21:18:30.485 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:18:30.488+0000 7f6875000700 1 --2- 172.21.15.18:0/4094428171 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f68701026b0 0x7f68700fbd20 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:18:30.485 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:18:30.488+0000 7f6875000700 1 --2- 172.21.15.18:0/4094428171 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6870101d10 0x7f68701020e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:18:30.485 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:18:30.488+0000 7f6875000700 1 -- 172.21.15.18:0/4094428171 >> 172.21.15.18:0/4094428171 conn(0x7f68700f7860 msgr2=0x7f68700f9c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:18:30.485 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:18:30.488+0000 7f6875000700 1 -- 172.21.15.18:0/4094428171 shutdown_connections 2024-04-12T21:18:30.485 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:18:30.488+0000 7f6875000700 1 -- 172.21.15.18:0/4094428171 wait complete. 2024-04-12T21:18:30.486 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:18:30.489+0000 7f6875000700 1 Processor -- start 2024-04-12T21:18:30.486 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:18:30.489+0000 7f6875000700 1 -- start start 2024-04-12T21:18:30.486 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:18:30.490+0000 7f6875000700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6870101d10 0x7f6870115e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:18:30.486 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:18:30.490+0000 7f6875000700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f68701026b0 0x7f6870116340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:18:30.486 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:18:30.490+0000 7f6875000700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6870116990 con 0x7f6870101d10 2024-04-12T21:18:30.486 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:18:30.490+0000 7f6875000700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f687010fd60 con 0x7f68701026b0 2024-04-12T21:18:30.486 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:18:30.490+0000 7f686de00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6870101d10 0x7f6870115e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:18:30.486 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:18:30.490+0000 7f686de00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6870101d10 0x7f6870115e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:35020/0 (socket says 172.21.15.18:35020) 2024-04-12T21:18:30.486 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:18:30.490+0000 7f686de00700 1 -- 172.21.15.18:0/2187847260 learned_addr learned my addr 172.21.15.18:0/2187847260 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:18:30.487 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:18:30.490+0000 7f686d400700 1 --2- 172.21.15.18:0/2187847260 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f68701026b0 0x7f6870116340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:18:30.488 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:18:30.491+0000 7f686de00700 1 -- 172.21.15.18:0/2187847260 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f68701026b0 msgr2=0x7f6870116340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:18:30.488 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:18:30.491+0000 7f686de00700 1 --2- 172.21.15.18:0/2187847260 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f68701026b0 0x7f6870116340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:18:30.488 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:18:30.491+0000 7f686de00700 1 -- 172.21.15.18:0/2187847260 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f685c005040 con 0x7f6870101d10 2024-04-12T21:18:30.488 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:18:30.491+0000 7f686d400700 1 --2- 172.21.15.18:0/2187847260 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f68701026b0 0x7f6870116340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-12T21:18:30.488 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:18:30.491+0000 7f686de00700 1 --2- 172.21.15.18:0/2187847260 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6870101d10 0x7f6870115e00 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f686000c830 tx=0x7f6860004470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:18:30.488 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:18:30.491+0000 7f6866000700 1 -- 172.21.15.18:0/2187847260 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f686000a760 con 0x7f6870101d10 2024-04-12T21:18:30.488 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:18:30.491+0000 7f6866000700 1 -- 172.21.15.18:0/2187847260 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f686000ad90 con 0x7f6870101d10 2024-04-12T21:18:30.488 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:18:30.491+0000 7f6875000700 1 -- 172.21.15.18:0/2187847260 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6870110040 con 0x7f6870101d10 2024-04-12T21:18:30.488 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:18:30.491+0000 7f6866000700 1 -- 172.21.15.18:0/2187847260 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f68600113a0 con 0x7f6870101d10 2024-04-12T21:18:30.488 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:18:30.491+0000 7f6875000700 1 -- 172.21.15.18:0/2187847260 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6870110600 con 0x7f6870101d10 2024-04-12T21:18:30.489 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:18:30.493+0000 7f6875000700 1 -- 172.21.15.18:0/2187847260 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f68500052b0 con 0x7f6870101d10 2024-04-12T21:18:30.489 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:18:30.494+0000 7f6866000700 1 -- 172.21.15.18:0/2187847260 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7f6860012400 con 0x7f6870101d10 2024-04-12T21:18:30.490 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:18:30.494+0000 7f6866000700 1 --2- 172.21.15.18:0/2187847260 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f685406ec10 0x7f68540710c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:18:30.491 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:18:30.495+0000 7f6866000700 1 -- 172.21.15.18:0/2187847260 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 6757+0+0 (secure 0 0 0) 0x7f686008e900 con 0x7f6870101d10 2024-04-12T21:18:30.491 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:18:30.495+0000 7f686d400700 1 --2- 172.21.15.18:0/2187847260 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f685406ec10 0x7f68540710c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:18:30.491 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:18:30.495+0000 7f686d400700 1 --2- 172.21.15.18:0/2187847260 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f685406ec10 0x7f68540710c0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f685c00b3b0 tx=0x7f685c009e30 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:18:30.495 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:18:30.500+0000 7f6866000700 1 -- 172.21.15.18:0/2187847260 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f686005b1f0 con 0x7f6870101d10 2024-04-12T21:18:30.773 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:18:30.777+0000 7f6875000700 1 -- 172.21.15.18:0/2187847260 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f6850005d70 con 0x7f6870101d10 2024-04-12T21:18:30.774 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:18:30.778+0000 7f6866000700 1 -- 172.21.15.18:0/2187847260 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 29 v29) v1 ==== 76+0+1872 (secure 0 0 0) 0x7f686005ad80 con 0x7f6870101d10 2024-04-12T21:18:30.775 INFO:teuthology.orchestra.run.smithi018.stdout:e29 2024-04-12T21:18:30.775 INFO:teuthology.orchestra.run.smithi018.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-12T21:18:30.775 INFO:teuthology.orchestra.run.smithi018.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-12T21:18:30.775 INFO:teuthology.orchestra.run.smithi018.stdout:legacy client fscid: 1 2024-04-12T21:18:30.775 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-12T21:18:30.776 INFO:teuthology.orchestra.run.smithi018.stdout:Filesystem 'cephfs' (1) 2024-04-12T21:18:30.776 INFO:teuthology.orchestra.run.smithi018.stdout:fs_name cephfs 2024-04-12T21:18:30.776 INFO:teuthology.orchestra.run.smithi018.stdout:epoch 24 2024-04-12T21:18:30.776 INFO:teuthology.orchestra.run.smithi018.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-04-12T21:18:30.776 INFO:teuthology.orchestra.run.smithi018.stdout:created 2024-04-12T20:51:57.362850+0000 2024-04-12T21:18:30.776 INFO:teuthology.orchestra.run.smithi018.stdout:modified 2024-04-12T21:06:59.717190+0000 2024-04-12T21:18:30.776 INFO:teuthology.orchestra.run.smithi018.stdout:tableserver 0 2024-04-12T21:18:30.776 INFO:teuthology.orchestra.run.smithi018.stdout:root 0 2024-04-12T21:18:30.776 INFO:teuthology.orchestra.run.smithi018.stdout:session_timeout 60 2024-04-12T21:18:30.776 INFO:teuthology.orchestra.run.smithi018.stdout:session_autoclose 300 2024-04-12T21:18:30.776 INFO:teuthology.orchestra.run.smithi018.stdout:max_file_size 1099511627776 2024-04-12T21:18:30.776 INFO:teuthology.orchestra.run.smithi018.stdout:max_xattr_size 65536 2024-04-12T21:18:30.776 INFO:teuthology.orchestra.run.smithi018.stdout:required_client_features {} 2024-04-12T21:18:30.776 INFO:teuthology.orchestra.run.smithi018.stdout:last_failure 0 2024-04-12T21:18:30.777 INFO:teuthology.orchestra.run.smithi018.stdout:last_failure_osd_epoch 100 2024-04-12T21:18:30.777 INFO:teuthology.orchestra.run.smithi018.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-12T21:18:30.777 INFO:teuthology.orchestra.run.smithi018.stdout:max_mds 2 2024-04-12T21:18:30.777 INFO:teuthology.orchestra.run.smithi018.stdout:in 0,1 2024-04-12T21:18:30.777 INFO:teuthology.orchestra.run.smithi018.stdout:up {} 2024-04-12T21:18:30.777 INFO:teuthology.orchestra.run.smithi018.stdout:failed 0,1 2024-04-12T21:18:30.777 INFO:teuthology.orchestra.run.smithi018.stdout:damaged 2024-04-12T21:18:30.777 INFO:teuthology.orchestra.run.smithi018.stdout:stopped 2024-04-12T21:18:30.777 INFO:teuthology.orchestra.run.smithi018.stdout:data_pools [3] 2024-04-12T21:18:30.777 INFO:teuthology.orchestra.run.smithi018.stdout:metadata_pool 2 2024-04-12T21:18:30.777 INFO:teuthology.orchestra.run.smithi018.stdout:inline_data disabled 2024-04-12T21:18:30.777 INFO:teuthology.orchestra.run.smithi018.stdout:balancer 2024-04-12T21:18:30.777 INFO:teuthology.orchestra.run.smithi018.stdout:bal_rank_mask -1 2024-04-12T21:18:30.778 INFO:teuthology.orchestra.run.smithi018.stdout:standby_count_wanted 1 2024-04-12T21:18:30.778 INFO:teuthology.orchestra.run.smithi018.stdout:qdb_cluster leader: 0 members: 2024-04-12T21:18:30.778 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-12T21:18:30.778 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-12T21:18:30.778 INFO:teuthology.orchestra.run.smithi018.stdout:Standby daemons: 2024-04-12T21:18:30.778 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-12T21:18:30.778 INFO:teuthology.orchestra.run.smithi018.stdout:[mds.cephfs.smithi186.wjnafs{-1:24277} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.186:6826/438347207,v1:172.21.15.186:6827/438347207] compat {c=[1],r=[1],i=[7ff]}] 2024-04-12T21:18:30.778 INFO:teuthology.orchestra.run.smithi018.stdout:[mds.cephfs.smithi186.jdllyn{-1:24281} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.186:6824/1464359952,v1:172.21.15.186:6825/1464359952] compat {c=[1],r=[1],i=[7ff]}] 2024-04-12T21:18:30.778 INFO:teuthology.orchestra.run.smithi018.stdout:[mds.cephfs.smithi018.acqbsc{-1:34436} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.18:6828/755179764,v1:172.21.15.18:6829/755179764] compat {c=[1],r=[1],i=[1fff]}] 2024-04-12T21:18:30.778 INFO:teuthology.orchestra.run.smithi018.stdout:[mds.cephfs.smithi018.szjluo{-1:34468} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.18:6826/2407992653,v1:172.21.15.18:6827/2407992653] compat {c=[1],r=[1],i=[1fff]}] 2024-04-12T21:18:30.779 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:18:30.781+0000 7f6875000700 1 -- 172.21.15.18:0/2187847260 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f685406ec10 msgr2=0x7f68540710c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:18:30.779 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:18:30.781+0000 7f6875000700 1 --2- 172.21.15.18:0/2187847260 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f685406ec10 0x7f68540710c0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f685c00b3b0 tx=0x7f685c009e30 comp rx=0 tx=0).stop 2024-04-12T21:18:30.779 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:18:30.781+0000 7f6875000700 1 -- 172.21.15.18:0/2187847260 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6870101d10 msgr2=0x7f6870115e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:18:30.779 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:18:30.781+0000 7f6875000700 1 --2- 172.21.15.18:0/2187847260 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6870101d10 0x7f6870115e00 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f686000c830 tx=0x7f6860004470 comp rx=0 tx=0).stop 2024-04-12T21:18:30.779 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:18:30.781+0000 7f6875000700 1 -- 172.21.15.18:0/2187847260 shutdown_connections 2024-04-12T21:18:30.779 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:18:30.781+0000 7f6875000700 1 --2- 172.21.15.18:0/2187847260 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f685406ec10 0x7f68540710c0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:18:30.779 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:18:30.781+0000 7f6875000700 1 --2- 172.21.15.18:0/2187847260 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6870101d10 0x7f6870115e00 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:18:30.780 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:18:30.781+0000 7f6875000700 1 --2- 172.21.15.18:0/2187847260 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f68701026b0 0x7f6870116340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:18:30.780 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:18:30.781+0000 7f6875000700 1 -- 172.21.15.18:0/2187847260 >> 172.21.15.18:0/2187847260 conn(0x7f68700f7860 msgr2=0x7f68700f8f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:18:30.780 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:18:30.781+0000 7f6875000700 1 -- 172.21.15.18:0/2187847260 shutdown_connections 2024-04-12T21:18:30.780 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:18:30.781+0000 7f6875000700 1 -- 172.21.15.18:0/2187847260 wait complete. 2024-04-12T21:18:30.780 INFO:teuthology.orchestra.run.smithi018.stderr:dumped fsmap epoch 29 2024-04-12T21:18:30.938 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:18:30.942+0000 7f0cd9200700 1 -- 172.21.15.18:0/585874125 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0cd4071f90 msgr2=0x7f0cd4070360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:18:30.939 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:18:30.942+0000 7f0cd9200700 1 --2- 172.21.15.18:0/585874125 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0cd4071f90 0x7f0cd4070360 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f0cc4004970 tx=0x7f0cc4009e30 comp rx=0 tx=0).stop 2024-04-12T21:18:30.939 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:18:30.942+0000 7f0cd9200700 1 -- 172.21.15.18:0/585874125 shutdown_connections 2024-04-12T21:18:30.939 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:18:30.942+0000 7f0cd9200700 1 --2- 172.21.15.18:0/585874125 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0cd40708a0 0x7f0cd4070d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:18:30.939 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:18:30.942+0000 7f0cd9200700 1 --2- 172.21.15.18:0/585874125 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0cd4071f90 0x7f0cd4070360 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:18:30.939 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:18:30.942+0000 7f0cd9200700 1 -- 172.21.15.18:0/585874125 >> 172.21.15.18:0/585874125 conn(0x7f0cd40f89b0 msgr2=0x7f0cd40fadc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:18:30.940 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:18:30 smithi018 ceph-mon[112590]: from='client.35004 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:18:30.940 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:18:30 smithi018 ceph-mon[112590]: from='client.35008 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:18:30.940 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:18:30 smithi018 ceph-mon[112590]: from='client.? 172.21.15.18:0/1247344259' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T21:18:30.942 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:18:30.942+0000 7f0cd9200700 1 -- 172.21.15.18:0/585874125 shutdown_connections 2024-04-12T21:18:30.942 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:18:30.942+0000 7f0cd9200700 1 -- 172.21.15.18:0/585874125 wait complete. 2024-04-12T21:18:30.942 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:18:30.943+0000 7f0cd9200700 1 Processor -- start 2024-04-12T21:18:30.942 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:18:30.944+0000 7f0cd9200700 1 -- start start 2024-04-12T21:18:30.943 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:18:30.944+0000 7f0cd9200700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0cd40708a0 0x7f0cd4119f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:18:30.943 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:18:30.944+0000 7f0cd9200700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0cd4071f90 0x7f0cd411a4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:18:30.943 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:18:30.944+0000 7f0cd9200700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0cd411ab10 con 0x7f0cd40708a0 2024-04-12T21:18:30.943 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:18:30.944+0000 7f0cd9200700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0cd41140f0 con 0x7f0cd4071f90 2024-04-12T21:18:30.943 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:18:30.944+0000 7f0cd1e00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0cd40708a0 0x7f0cd4119f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:18:30.943 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:18:30.945+0000 7f0cd1e00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0cd40708a0 0x7f0cd4119f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:56998/0 (socket says 172.21.15.18:56998) 2024-04-12T21:18:30.943 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:18:30.945+0000 7f0cd1e00700 1 -- 172.21.15.18:0/972222194 learned_addr learned my addr 172.21.15.18:0/972222194 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:18:30.943 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:18:30.945+0000 7f0cd1400700 1 --2- 172.21.15.18:0/972222194 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0cd4071f90 0x7f0cd411a4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:18:30.943 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:18:30.945+0000 7f0cd1e00700 1 -- 172.21.15.18:0/972222194 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0cd4071f90 msgr2=0x7f0cd411a4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:18:30.943 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:18:30.945+0000 7f0cd1e00700 1 --2- 172.21.15.18:0/972222194 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0cd4071f90 0x7f0cd411a4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:18:30.943 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:18:30.945+0000 7f0cd1e00700 1 -- 172.21.15.18:0/972222194 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0cc4005040 con 0x7f0cd40708a0 2024-04-12T21:18:30.943 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:18:30.945+0000 7f0cd1400700 1 --2- 172.21.15.18:0/972222194 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0cd4071f90 0x7f0cd411a4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-12T21:18:30.943 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:18:30.945+0000 7f0cd1e00700 1 --2- 172.21.15.18:0/972222194 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0cd40708a0 0x7f0cd4119f80 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f0cc4005010 tx=0x7f0cc4009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:18:30.943 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:18:30.946+0000 7f0cca000700 1 -- 172.21.15.18:0/972222194 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0cc400e040 con 0x7f0cd40708a0 2024-04-12T21:18:30.944 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:18:30.946+0000 7f0cca000700 1 -- 172.21.15.18:0/972222194 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f0cc4013420 con 0x7f0cd40708a0 2024-04-12T21:18:30.944 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:18:30.946+0000 7f0cd9200700 1 -- 172.21.15.18:0/972222194 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0cd4114370 con 0x7f0cd40708a0 2024-04-12T21:18:30.944 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:18:30.946+0000 7f0cca000700 1 -- 172.21.15.18:0/972222194 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0cc401c630 con 0x7f0cd40708a0 2024-04-12T21:18:30.944 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:18:30.946+0000 7f0cd9200700 1 -- 172.21.15.18:0/972222194 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0cd41148b0 con 0x7f0cd40708a0 2024-04-12T21:18:30.944 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:18:30.947+0000 7f0cd9200700 1 -- 172.21.15.18:0/972222194 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0cd4071aa0 con 0x7f0cd40708a0 2024-04-12T21:18:30.945 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:18:30.949+0000 7f0cca000700 1 -- 172.21.15.18:0/972222194 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7f0cc4013590 con 0x7f0cd40708a0 2024-04-12T21:18:30.947 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:18:30.951+0000 7f0cca000700 1 --2- 172.21.15.18:0/972222194 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f0cb806ec00 0x7f0cb80710b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:18:30.947 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:18:30.951+0000 7f0cca000700 1 -- 172.21.15.18:0/972222194 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 6757+0+0 (secure 0 0 0) 0x7f0cc408fc30 con 0x7f0cd40708a0 2024-04-12T21:18:30.947 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:18:30.951+0000 7f0cd1400700 1 --2- 172.21.15.18:0/972222194 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f0cb806ec00 0x7f0cb80710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:18:30.948 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:18:30.952+0000 7f0cd1400700 1 --2- 172.21.15.18:0/972222194 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f0cb806ec00 0x7f0cb80710b0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f0cc0004600 tx=0x7f0cc000b040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:18:30.952 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:18:30.956+0000 7f0cca000700 1 -- 172.21.15.18:0/972222194 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f0cc405c510 con 0x7f0cd40708a0 2024-04-12T21:18:31.028 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:18:30 smithi186 ceph-mon[86379]: from='client.35004 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:18:31.028 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:18:30 smithi186 ceph-mon[86379]: from='client.35008 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:18:31.029 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:18:30 smithi186 ceph-mon[86379]: from='client.? 172.21.15.18:0/1247344259' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T21:18:31.207 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:18:31.211+0000 7f0cd9200700 1 -- 172.21.15.18:0/972222194 --> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f0cd4115640 con 0x7f0cb806ec00 2024-04-12T21:18:31.209 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:18:31.213+0000 7f0cca000700 1 -- 172.21.15.18:0/972222194 <== mgr.34102 v2:172.21.15.18:6800/3812891919 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f0cd4115640 con 0x7f0cb806ec00 2024-04-12T21:18:31.209 INFO:teuthology.orchestra.run.smithi018.stdout:{ 2024-04-12T21:18:31.209 INFO:teuthology.orchestra.run.smithi018.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:f4452def3be4d350f4a8e5feaf7ebb1511be9298fe68478ba7a2d6fbd7d69a63", 2024-04-12T21:18:31.209 INFO:teuthology.orchestra.run.smithi018.stdout: "in_progress": true, 2024-04-12T21:18:31.210 INFO:teuthology.orchestra.run.smithi018.stdout: "which": "Upgrading all daemon types on all hosts", 2024-04-12T21:18:31.210 INFO:teuthology.orchestra.run.smithi018.stdout: "services_complete": [ 2024-04-12T21:18:31.210 INFO:teuthology.orchestra.run.smithi018.stdout: "crash", 2024-04-12T21:18:31.210 INFO:teuthology.orchestra.run.smithi018.stdout: "osd", 2024-04-12T21:18:31.210 INFO:teuthology.orchestra.run.smithi018.stdout: "mon", 2024-04-12T21:18:31.210 INFO:teuthology.orchestra.run.smithi018.stdout: "mgr" 2024-04-12T21:18:31.210 INFO:teuthology.orchestra.run.smithi018.stdout: ], 2024-04-12T21:18:31.210 INFO:teuthology.orchestra.run.smithi018.stdout: "progress": "14/21 daemons upgraded", 2024-04-12T21:18:31.210 INFO:teuthology.orchestra.run.smithi018.stdout: "message": "Currently upgrading mds daemons", 2024-04-12T21:18:31.210 INFO:teuthology.orchestra.run.smithi018.stdout: "is_paused": false 2024-04-12T21:18:31.210 INFO:teuthology.orchestra.run.smithi018.stdout:} 2024-04-12T21:18:31.213 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:18:31.216+0000 7f0cd9200700 1 -- 172.21.15.18:0/972222194 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f0cb806ec00 msgr2=0x7f0cb80710b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:18:31.213 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:18:31.216+0000 7f0cd9200700 1 --2- 172.21.15.18:0/972222194 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f0cb806ec00 0x7f0cb80710b0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f0cc0004600 tx=0x7f0cc000b040 comp rx=0 tx=0).stop 2024-04-12T21:18:31.213 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:18:31.216+0000 7f0cd9200700 1 -- 172.21.15.18:0/972222194 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0cd40708a0 msgr2=0x7f0cd4119f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:18:31.213 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:18:31.216+0000 7f0cd9200700 1 --2- 172.21.15.18:0/972222194 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0cd40708a0 0x7f0cd4119f80 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f0cc4005010 tx=0x7f0cc4009e30 comp rx=0 tx=0).stop 2024-04-12T21:18:31.213 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:18:31.217+0000 7f0cd9200700 1 -- 172.21.15.18:0/972222194 shutdown_connections 2024-04-12T21:18:31.214 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:18:31.217+0000 7f0cd9200700 1 --2- 172.21.15.18:0/972222194 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f0cb806ec00 0x7f0cb80710b0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:18:31.214 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:18:31.217+0000 7f0cd9200700 1 --2- 172.21.15.18:0/972222194 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0cd40708a0 0x7f0cd4119f80 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:18:31.214 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:18:31.217+0000 7f0cd9200700 1 --2- 172.21.15.18:0/972222194 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0cd4071f90 0x7f0cd411a4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:18:31.214 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:18:31.217+0000 7f0cd9200700 1 -- 172.21.15.18:0/972222194 >> 172.21.15.18:0/972222194 conn(0x7f0cd40f89b0 msgr2=0x7f0cd40f9a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:18:31.214 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:18:31.217+0000 7f0cd9200700 1 -- 172.21.15.18:0/972222194 shutdown_connections 2024-04-12T21:18:31.214 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:18:31.217+0000 7f0cd9200700 1 -- 172.21.15.18:0/972222194 wait complete. 2024-04-12T21:18:31.367 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:18:31.370+0000 7f528d800700 1 -- 172.21.15.18:0/211627981 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f52881037b0 msgr2=0x7f52880fce20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:18:31.367 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:18:31.370+0000 7f528d800700 1 --2- 172.21.15.18:0/211627981 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f52881037b0 0x7f52880fce20 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f5270004940 tx=0x7f5270009e30 comp rx=0 tx=0).stop 2024-04-12T21:18:31.367 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:18:31.370+0000 7f528d800700 1 -- 172.21.15.18:0/211627981 shutdown_connections 2024-04-12T21:18:31.367 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:18:31.370+0000 7f528d800700 1 --2- 172.21.15.18:0/211627981 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f52881037b0 0x7f52880fce20 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:18:31.367 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:18:31.370+0000 7f528d800700 1 --2- 172.21.15.18:0/211627981 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5288102e10 0x7f52881031e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:18:31.367 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:18:31.370+0000 7f528d800700 1 -- 172.21.15.18:0/211627981 >> 172.21.15.18:0/211627981 conn(0x7f52880f8980 msgr2=0x7f52880fad90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:18:31.367 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:18:31.370+0000 7f528d800700 1 -- 172.21.15.18:0/211627981 shutdown_connections 2024-04-12T21:18:31.367 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:18:31.370+0000 7f528d800700 1 -- 172.21.15.18:0/211627981 wait complete. 2024-04-12T21:18:31.369 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:18:31.371+0000 7f528d800700 1 Processor -- start 2024-04-12T21:18:31.369 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:18:31.371+0000 7f528d800700 1 -- start start 2024-04-12T21:18:31.369 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:18:31.372+0000 7f528d800700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5288102e10 0x7f52880702c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:18:31.369 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:18:31.372+0000 7f528d800700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f52881037b0 0x7f5288070800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:18:31.369 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:18:31.372+0000 7f528d800700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5288070d90 con 0x7f52881037b0 2024-04-12T21:18:31.369 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:18:31.372+0000 7f528d800700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5288070ed0 con 0x7f5288102e10 2024-04-12T21:18:31.369 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:18:31.372+0000 7f527fe00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f52881037b0 0x7f5288070800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:18:31.370 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:18:31.372+0000 7f527fe00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f52881037b0 0x7f5288070800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:57006/0 (socket says 172.21.15.18:57006) 2024-04-12T21:18:31.370 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:18:31.372+0000 7f527fe00700 1 -- 172.21.15.18:0/2079926976 learned_addr learned my addr 172.21.15.18:0/2079926976 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:18:31.370 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:18:31.372+0000 7f527fe00700 1 -- 172.21.15.18:0/2079926976 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5288102e10 msgr2=0x7f52880702c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:18:31.370 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:18:31.372+0000 7f527fe00700 1 --2- 172.21.15.18:0/2079926976 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5288102e10 0x7f52880702c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:18:31.370 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:18:31.372+0000 7f527fe00700 1 -- 172.21.15.18:0/2079926976 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5270005040 con 0x7f52881037b0 2024-04-12T21:18:31.370 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:18:31.373+0000 7f527fe00700 1 --2- 172.21.15.18:0/2079926976 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f52881037b0 0x7f5288070800 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f5270009100 tx=0x7f5270009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:18:31.370 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:18:31.373+0000 7f527ea00700 1 -- 172.21.15.18:0/2079926976 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f527000e040 con 0x7f52881037b0 2024-04-12T21:18:31.370 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:18:31.373+0000 7f527ea00700 1 -- 172.21.15.18:0/2079926976 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f5270013420 con 0x7f52881037b0 2024-04-12T21:18:31.370 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:18:31.373+0000 7f527ea00700 1 -- 172.21.15.18:0/2079926976 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f527001c630 con 0x7f52881037b0 2024-04-12T21:18:31.370 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:18:31.373+0000 7f528d800700 1 -- 172.21.15.18:0/2079926976 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5288071120 con 0x7f52881037b0 2024-04-12T21:18:31.370 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:18:31.373+0000 7f528d800700 1 -- 172.21.15.18:0/2079926976 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f528811a2b0 con 0x7f52881037b0 2024-04-12T21:18:31.371 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:18:31.374+0000 7f528d800700 1 -- 172.21.15.18:0/2079926976 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f52881052e0 con 0x7f52881037b0 2024-04-12T21:18:31.371 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:18:31.375+0000 7f527ea00700 1 -- 172.21.15.18:0/2079926976 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7f5270013590 con 0x7f52881037b0 2024-04-12T21:18:31.372 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:18:31.376+0000 7f527ea00700 1 --2- 172.21.15.18:0/2079926976 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f526c06ec10 0x7f526c0710c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:18:31.372 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:18:31.376+0000 7f527ea00700 1 -- 172.21.15.18:0/2079926976 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 6757+0+0 (secure 0 0 0) 0x7f527008fba0 con 0x7f52881037b0 2024-04-12T21:18:31.372 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:18:31.377+0000 7f5286a00700 1 --2- 172.21.15.18:0/2079926976 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f526c06ec10 0x7f526c0710c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:18:31.373 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:18:31.377+0000 7f5286a00700 1 --2- 172.21.15.18:0/2079926976 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f526c06ec10 0x7f526c0710c0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f5278004ad0 tx=0x7f5278007d40 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:18:31.377 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:18:31.381+0000 7f527ea00700 1 -- 172.21.15.18:0/2079926976 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f52700588a0 con 0x7f52881037b0 2024-04-12T21:18:31.697 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:18:31.701+0000 7f528d800700 1 -- 172.21.15.18:0/2079926976 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f52880fe560 con 0x7f52881037b0 2024-04-12T21:18:31.698 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:18:31.702+0000 7f527ea00700 1 -- 172.21.15.18:0/2079926976 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+233 (secure 0 0 0) 0x7f5270058030 con 0x7f52881037b0 2024-04-12T21:18:31.698 INFO:teuthology.orchestra.run.smithi018.stdout:HEALTH_ERR 1 filesystem is degraded; 1 filesystem is offline 2024-04-12T21:18:31.698 INFO:teuthology.orchestra.run.smithi018.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-04-12T21:18:31.698 INFO:teuthology.orchestra.run.smithi018.stdout: fs cephfs is degraded 2024-04-12T21:18:31.698 INFO:teuthology.orchestra.run.smithi018.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-04-12T21:18:31.699 INFO:teuthology.orchestra.run.smithi018.stdout: fs cephfs is offline because no MDS is active for it. 2024-04-12T21:18:31.700 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:18:31.705+0000 7f528d800700 1 -- 172.21.15.18:0/2079926976 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f526c06ec10 msgr2=0x7f526c0710c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:18:31.701 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:18:31.705+0000 7f528d800700 1 --2- 172.21.15.18:0/2079926976 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f526c06ec10 0x7f526c0710c0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f5278004ad0 tx=0x7f5278007d40 comp rx=0 tx=0).stop 2024-04-12T21:18:31.701 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:18:31.705+0000 7f528d800700 1 -- 172.21.15.18:0/2079926976 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f52881037b0 msgr2=0x7f5288070800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:18:31.701 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:18:31.705+0000 7f528d800700 1 --2- 172.21.15.18:0/2079926976 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f52881037b0 0x7f5288070800 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f5270009100 tx=0x7f5270009e30 comp rx=0 tx=0).stop 2024-04-12T21:18:31.701 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:18:31.705+0000 7f528d800700 1 -- 172.21.15.18:0/2079926976 shutdown_connections 2024-04-12T21:18:31.701 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:18:31.705+0000 7f528d800700 1 --2- 172.21.15.18:0/2079926976 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f526c06ec10 0x7f526c0710c0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:18:31.701 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:18:31.705+0000 7f528d800700 1 --2- 172.21.15.18:0/2079926976 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5288102e10 0x7f52880702c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:18:31.702 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:18:31.705+0000 7f528d800700 1 --2- 172.21.15.18:0/2079926976 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f52881037b0 0x7f5288070800 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:18:31.702 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:18:31.705+0000 7f528d800700 1 -- 172.21.15.18:0/2079926976 >> 172.21.15.18:0/2079926976 conn(0x7f52880f8980 msgr2=0x7f52881021d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:18:31.702 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:18:31.705+0000 7f528d800700 1 -- 172.21.15.18:0/2079926976 shutdown_connections 2024-04-12T21:18:31.702 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:18:31.705+0000 7f528d800700 1 -- 172.21.15.18:0/2079926976 wait complete. 2024-04-12T21:18:32.028 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:18:31 smithi186 ceph-mon[86379]: from='client.35012 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:18:32.029 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:18:31 smithi186 ceph-mon[86379]: pgmap v561: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:18:32.029 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:18:31 smithi186 ceph-mon[86379]: from='client.? 172.21.15.18:0/2187847260' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-12T21:18:32.099 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:18:31 smithi018 ceph-mon[112590]: from='client.35012 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:18:32.100 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:18:31 smithi018 ceph-mon[112590]: pgmap v561: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:18:32.100 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:18:31 smithi018 ceph-mon[112590]: from='client.? 172.21.15.18:0/2187847260' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-12T21:18:33.028 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:18:32 smithi186 ceph-mon[86379]: from='client.35024 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:18:33.028 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:18:32 smithi186 ceph-mon[86379]: from='client.? 172.21.15.18:0/2079926976' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-12T21:18:33.099 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:18:32 smithi018 ceph-mon[112590]: from='client.35024 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:18:33.100 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:18:32 smithi018 ceph-mon[112590]: from='client.? 172.21.15.18:0/2079926976' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-12T21:18:34.028 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:18:33 smithi186 ceph-mon[86379]: pgmap v562: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:18:34.099 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:18:33 smithi018 ceph-mon[112590]: pgmap v562: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:18:36.028 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:18:35 smithi186 ceph-mon[86379]: pgmap v563: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:18:36.099 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:18:35 smithi018 ceph-mon[112590]: pgmap v563: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:18:38.028 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:18:37 smithi186 ceph-mon[86379]: pgmap v564: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:18:38.099 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:18:37 smithi018 ceph-mon[112590]: pgmap v564: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:18:40.028 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:18:39 smithi186 ceph-mon[86379]: pgmap v565: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:18:40.099 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:18:39 smithi018 ceph-mon[112590]: pgmap v565: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:18:41.001 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:18:40 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T21:18:41.028 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:18:40 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T21:18:42.028 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:18:41 smithi186 ceph-mon[86379]: pgmap v566: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:18:42.099 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:18:41 smithi018 ceph-mon[112590]: pgmap v566: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:18:44.028 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:18:43 smithi186 ceph-mon[86379]: pgmap v567: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:18:44.099 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:18:43 smithi018 ceph-mon[112590]: pgmap v567: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:18:46.028 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:18:45 smithi186 ceph-mon[86379]: pgmap v568: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:18:46.099 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:18:45 smithi018 ceph-mon[112590]: pgmap v568: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:18:48.028 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:18:47 smithi186 ceph-mon[86379]: pgmap v569: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:18:48.099 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:18:47 smithi018 ceph-mon[112590]: pgmap v569: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:18:50.028 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:18:49 smithi186 ceph-mon[86379]: pgmap v570: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:18:50.099 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:18:49 smithi018 ceph-mon[112590]: pgmap v570: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:18:52.028 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:18:51 smithi186 ceph-mon[86379]: pgmap v571: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:18:52.099 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:18:51 smithi018 ceph-mon[112590]: pgmap v571: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:18:54.028 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:18:53 smithi186 ceph-mon[86379]: pgmap v572: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:18:54.099 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:18:53 smithi018 ceph-mon[112590]: pgmap v572: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:18:56.028 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:18:55 smithi186 ceph-mon[86379]: pgmap v573: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:18:56.028 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:18:55 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T21:18:56.099 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:18:55 smithi018 ceph-mon[112590]: pgmap v573: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:18:56.100 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:18:55 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T21:18:57.028 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:18:56 smithi186 ceph-mon[86379]: pgmap v574: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:18:57.099 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:18:56 smithi018 ceph-mon[112590]: pgmap v574: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:18:59.528 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:18:59 smithi186 ceph-mon[86379]: pgmap v575: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:18:59.599 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:18:59 smithi018 ceph-mon[112590]: pgmap v575: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:19:01.528 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:19:01 smithi186 ceph-mon[86379]: pgmap v576: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:19:01.599 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:19:01 smithi018 ceph-mon[112590]: pgmap v576: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:19:01.875 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:01.880+0000 7f7c94a00700 1 -- 172.21.15.18:0/3990234715 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7c90102690 msgr2=0x7f7c900fbd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:19:01.876 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:01.880+0000 7f7c94a00700 1 --2- 172.21.15.18:0/3990234715 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7c90102690 0x7f7c900fbd00 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f7c78004940 tx=0x7f7c78009e30 comp rx=0 tx=0).stop 2024-04-12T21:19:01.876 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:01.880+0000 7f7c94a00700 1 -- 172.21.15.18:0/3990234715 shutdown_connections 2024-04-12T21:19:01.876 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:01.880+0000 7f7c94a00700 1 --2- 172.21.15.18:0/3990234715 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7c90102690 0x7f7c900fbd00 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:19:01.876 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:01.880+0000 7f7c94a00700 1 --2- 172.21.15.18:0/3990234715 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7c90101cf0 0x7f7c901020c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:19:01.876 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:01.880+0000 7f7c94a00700 1 -- 172.21.15.18:0/3990234715 >> 172.21.15.18:0/3990234715 conn(0x7f7c900f7860 msgr2=0x7f7c900f9c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:19:01.876 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:01.880+0000 7f7c94a00700 1 -- 172.21.15.18:0/3990234715 shutdown_connections 2024-04-12T21:19:01.876 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:01.880+0000 7f7c94a00700 1 -- 172.21.15.18:0/3990234715 wait complete. 2024-04-12T21:19:01.877 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:01.881+0000 7f7c94a00700 1 Processor -- start 2024-04-12T21:19:01.877 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:01.882+0000 7f7c94a00700 1 -- start start 2024-04-12T21:19:01.877 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:01.882+0000 7f7c94a00700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7c90101cf0 0x7f7c90113b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:19:01.877 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:01.882+0000 7f7c94a00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7c90102690 0x7f7c90114090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:19:01.877 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:01.882+0000 7f7c94a00700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c90114770 con 0x7f7c90102690 2024-04-12T21:19:01.877 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:01.882+0000 7f7c94a00700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c9010dad0 con 0x7f7c90101cf0 2024-04-12T21:19:01.879 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:01.882+0000 7f7c85400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7c90102690 0x7f7c90114090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:19:01.879 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:01.882+0000 7f7c85400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7c90102690 0x7f7c90114090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:58938/0 (socket says 172.21.15.18:58938) 2024-04-12T21:19:01.879 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:01.882+0000 7f7c85400700 1 -- 172.21.15.18:0/1896028504 learned_addr learned my addr 172.21.15.18:0/1896028504 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:19:01.879 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:01.883+0000 7f7c8de00700 1 --2- 172.21.15.18:0/1896028504 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7c90101cf0 0x7f7c90113b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:19:01.879 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:01.883+0000 7f7c85400700 1 -- 172.21.15.18:0/1896028504 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7c90101cf0 msgr2=0x7f7c90113b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:19:01.879 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:01.883+0000 7f7c85400700 1 --2- 172.21.15.18:0/1896028504 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7c90101cf0 0x7f7c90113b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:19:01.879 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:01.883+0000 7f7c85400700 1 -- 172.21.15.18:0/1896028504 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c78005040 con 0x7f7c90102690 2024-04-12T21:19:01.879 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:01.883+0000 7f7c85400700 1 --2- 172.21.15.18:0/1896028504 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7c90102690 0x7f7c90114090 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f7c78009180 tx=0x7f7c78009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:19:01.879 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:01.884+0000 7f7c86a00700 1 -- 172.21.15.18:0/1896028504 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7c7800e040 con 0x7f7c90102690 2024-04-12T21:19:01.879 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:01.884+0000 7f7c86a00700 1 -- 172.21.15.18:0/1896028504 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f7c7800bbc0 con 0x7f7c90102690 2024-04-12T21:19:01.879 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:01.884+0000 7f7c94a00700 1 -- 172.21.15.18:0/1896028504 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7c9010dd50 con 0x7f7c90102690 2024-04-12T21:19:01.879 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:01.884+0000 7f7c86a00700 1 -- 172.21.15.18:0/1896028504 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7c7801c3f0 con 0x7f7c90102690 2024-04-12T21:19:01.879 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:01.884+0000 7f7c94a00700 1 -- 172.21.15.18:0/1896028504 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7c9010e310 con 0x7f7c90102690 2024-04-12T21:19:01.880 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:01.885+0000 7f7c94a00700 1 -- 172.21.15.18:0/1896028504 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7c900fd440 con 0x7f7c90102690 2024-04-12T21:19:01.881 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:01.886+0000 7f7c86a00700 1 -- 172.21.15.18:0/1896028504 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7f7c7800bd30 con 0x7f7c90102690 2024-04-12T21:19:01.882 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:01.887+0000 7f7c86a00700 1 --2- 172.21.15.18:0/1896028504 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f7c7406ec00 0x7f7c740710b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:19:01.882 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:01.887+0000 7f7c86a00700 1 -- 172.21.15.18:0/1896028504 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 6757+0+0 (secure 0 0 0) 0x7f7c7808fcc0 con 0x7f7c90102690 2024-04-12T21:19:01.882 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:01.888+0000 7f7c8de00700 1 --2- 172.21.15.18:0/1896028504 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f7c7406ec00 0x7f7c740710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:19:01.883 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:01.888+0000 7f7c8de00700 1 --2- 172.21.15.18:0/1896028504 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f7c7406ec00 0x7f7c740710b0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f7c80001250 tx=0x7f7c80009620 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:19:01.887 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:01.893+0000 7f7c86a00700 1 -- 172.21.15.18:0/1896028504 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f7c78095020 con 0x7f7c90102690 2024-04-12T21:19:02.145 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:02.150+0000 7f7c94a00700 1 -- 172.21.15.18:0/1896028504 --> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f7c90108e40 con 0x7f7c7406ec00 2024-04-12T21:19:02.147 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:02.152+0000 7f7c86a00700 1 -- 172.21.15.18:0/1896028504 <== mgr.34102 v2:172.21.15.18:6800/3812891919 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f7c90108e40 con 0x7f7c7406ec00 2024-04-12T21:19:02.151 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:02.155+0000 7f7c94a00700 1 -- 172.21.15.18:0/1896028504 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f7c7406ec00 msgr2=0x7f7c740710b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:19:02.151 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:02.155+0000 7f7c94a00700 1 --2- 172.21.15.18:0/1896028504 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f7c7406ec00 0x7f7c740710b0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f7c80001250 tx=0x7f7c80009620 comp rx=0 tx=0).stop 2024-04-12T21:19:02.151 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:02.155+0000 7f7c94a00700 1 -- 172.21.15.18:0/1896028504 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7c90102690 msgr2=0x7f7c90114090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:19:02.151 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:02.155+0000 7f7c94a00700 1 --2- 172.21.15.18:0/1896028504 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7c90102690 0x7f7c90114090 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f7c78009180 tx=0x7f7c78009e30 comp rx=0 tx=0).stop 2024-04-12T21:19:02.151 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:02.155+0000 7f7c94a00700 1 -- 172.21.15.18:0/1896028504 shutdown_connections 2024-04-12T21:19:02.151 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:02.155+0000 7f7c94a00700 1 --2- 172.21.15.18:0/1896028504 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f7c7406ec00 0x7f7c740710b0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:19:02.151 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:02.155+0000 7f7c94a00700 1 --2- 172.21.15.18:0/1896028504 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7c90101cf0 0x7f7c90113b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:19:02.151 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:02.155+0000 7f7c94a00700 1 --2- 172.21.15.18:0/1896028504 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7c90102690 0x7f7c90114090 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:19:02.151 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:02.155+0000 7f7c94a00700 1 -- 172.21.15.18:0/1896028504 >> 172.21.15.18:0/1896028504 conn(0x7f7c900f7860 msgr2=0x7f7c90065e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:19:02.151 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:02.155+0000 7f7c94a00700 1 -- 172.21.15.18:0/1896028504 shutdown_connections 2024-04-12T21:19:02.151 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:02.155+0000 7f7c94a00700 1 -- 172.21.15.18:0/1896028504 wait complete. 2024-04-12T21:19:02.168 INFO:teuthology.orchestra.run.smithi018.stdout:true 2024-04-12T21:19:02.317 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:02.322+0000 7f00eac00700 1 -- 172.21.15.18:0/2178887751 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f00e41037a0 msgr2=0x7f00e40fce10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:19:02.317 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:02.322+0000 7f00eac00700 1 --2- 172.21.15.18:0/2178887751 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f00e41037a0 0x7f00e40fce10 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f00d0004940 tx=0x7f00d0009e30 comp rx=0 tx=0).stop 2024-04-12T21:19:02.318 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:02.322+0000 7f00eac00700 1 -- 172.21.15.18:0/2178887751 shutdown_connections 2024-04-12T21:19:02.318 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:02.322+0000 7f00eac00700 1 --2- 172.21.15.18:0/2178887751 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f00e41037a0 0x7f00e40fce10 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:19:02.318 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:02.322+0000 7f00eac00700 1 --2- 172.21.15.18:0/2178887751 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f00e4102e00 0x7f00e41031d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:19:02.318 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:02.322+0000 7f00eac00700 1 -- 172.21.15.18:0/2178887751 >> 172.21.15.18:0/2178887751 conn(0x7f00e40f89b0 msgr2=0x7f00e40fadc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:19:02.318 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:02.322+0000 7f00eac00700 1 -- 172.21.15.18:0/2178887751 shutdown_connections 2024-04-12T21:19:02.318 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:02.322+0000 7f00eac00700 1 -- 172.21.15.18:0/2178887751 wait complete. 2024-04-12T21:19:02.319 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:02.323+0000 7f00eac00700 1 Processor -- start 2024-04-12T21:19:02.319 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:02.323+0000 7f00eac00700 1 -- start start 2024-04-12T21:19:02.319 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:02.324+0000 7f00eac00700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f00e4102e00 0x7f00e4115de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:19:02.319 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:02.324+0000 7f00eac00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f00e41037a0 0x7f00e4116320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:19:02.319 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:02.324+0000 7f00eac00700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00e4116970 con 0x7f00e41037a0 2024-04-12T21:19:02.319 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:02.324+0000 7f00eac00700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00e410fd40 con 0x7f00e4102e00 2024-04-12T21:19:02.320 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:02.324+0000 7f00e3400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f00e41037a0 0x7f00e4116320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:19:02.321 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:02.324+0000 7f00e3400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f00e41037a0 0x7f00e4116320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:58956/0 (socket says 172.21.15.18:58956) 2024-04-12T21:19:02.321 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:02.324+0000 7f00e3400700 1 -- 172.21.15.18:0/1213571173 learned_addr learned my addr 172.21.15.18:0/1213571173 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:19:02.321 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:02.325+0000 7f00e3e00700 1 --2- 172.21.15.18:0/1213571173 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f00e4102e00 0x7f00e4115de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:19:02.321 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:02.325+0000 7f00e3400700 1 -- 172.21.15.18:0/1213571173 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f00e4102e00 msgr2=0x7f00e4115de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:19:02.321 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:02.325+0000 7f00e3400700 1 --2- 172.21.15.18:0/1213571173 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f00e4102e00 0x7f00e4115de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:19:02.321 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:02.325+0000 7f00e3400700 1 -- 172.21.15.18:0/1213571173 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00d0005040 con 0x7f00e41037a0 2024-04-12T21:19:02.321 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:02.325+0000 7f00e3400700 1 --2- 172.21.15.18:0/1213571173 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f00e41037a0 0x7f00e4116320 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f00d0009100 tx=0x7f00d0009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:19:02.321 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:02.326+0000 7f00dbe00700 1 -- 172.21.15.18:0/1213571173 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f00d000e040 con 0x7f00e41037a0 2024-04-12T21:19:02.321 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:02.326+0000 7f00dbe00700 1 -- 172.21.15.18:0/1213571173 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f00d0013420 con 0x7f00e41037a0 2024-04-12T21:19:02.321 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:02.326+0000 7f00eac00700 1 -- 172.21.15.18:0/1213571173 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f00e410ffc0 con 0x7f00e41037a0 2024-04-12T21:19:02.321 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:02.326+0000 7f00dbe00700 1 -- 172.21.15.18:0/1213571173 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f00d001c630 con 0x7f00e41037a0 2024-04-12T21:19:02.321 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:02.326+0000 7f00eac00700 1 -- 172.21.15.18:0/1213571173 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f00e4110580 con 0x7f00e41037a0 2024-04-12T21:19:02.322 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:02.327+0000 7f00eac00700 1 -- 172.21.15.18:0/1213571173 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f00e41018c0 con 0x7f00e41037a0 2024-04-12T21:19:02.323 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:02.328+0000 7f00dbe00700 1 -- 172.21.15.18:0/1213571173 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7f00d0013590 con 0x7f00e41037a0 2024-04-12T21:19:02.324 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:02.329+0000 7f00dbe00700 1 --2- 172.21.15.18:0/1213571173 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f00c806ec10 0x7f00c80710c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:19:02.324 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:02.329+0000 7f00dbe00700 1 -- 172.21.15.18:0/1213571173 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 6757+0+0 (secure 0 0 0) 0x7f00d008fba0 con 0x7f00e41037a0 2024-04-12T21:19:02.324 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:02.329+0000 7f00e3e00700 1 --2- 172.21.15.18:0/1213571173 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f00c806ec10 0x7f00c80710c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:19:02.325 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:02.330+0000 7f00e3e00700 1 --2- 172.21.15.18:0/1213571173 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f00c806ec10 0x7f00c80710c0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f00d4004550 tx=0x7f00d4009040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:19:02.329 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:02.334+0000 7f00dbe00700 1 -- 172.21.15.18:0/1213571173 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f00d005c3d0 con 0x7f00e41037a0 2024-04-12T21:19:02.586 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:02.590+0000 7f00eac00700 1 -- 172.21.15.18:0/1213571173 --> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f00e41114a0 con 0x7f00c806ec10 2024-04-12T21:19:02.588 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:02.592+0000 7f00dbe00700 1 -- 172.21.15.18:0/1213571173 <== mgr.34102 v2:172.21.15.18:6800/3812891919 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f00e41114a0 con 0x7f00c806ec10 2024-04-12T21:19:02.592 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:02.597+0000 7f00eac00700 1 -- 172.21.15.18:0/1213571173 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f00c806ec10 msgr2=0x7f00c80710c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:19:02.592 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:02.597+0000 7f00eac00700 1 --2- 172.21.15.18:0/1213571173 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f00c806ec10 0x7f00c80710c0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f00d4004550 tx=0x7f00d4009040 comp rx=0 tx=0).stop 2024-04-12T21:19:02.592 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:02.597+0000 7f00eac00700 1 -- 172.21.15.18:0/1213571173 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f00e41037a0 msgr2=0x7f00e4116320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:19:02.592 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:02.597+0000 7f00eac00700 1 --2- 172.21.15.18:0/1213571173 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f00e41037a0 0x7f00e4116320 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f00d0009100 tx=0x7f00d0009e30 comp rx=0 tx=0).stop 2024-04-12T21:19:02.592 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:02.597+0000 7f00eac00700 1 -- 172.21.15.18:0/1213571173 shutdown_connections 2024-04-12T21:19:02.592 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:02.597+0000 7f00eac00700 1 --2- 172.21.15.18:0/1213571173 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f00c806ec10 0x7f00c80710c0 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:19:02.593 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:02.597+0000 7f00eac00700 1 --2- 172.21.15.18:0/1213571173 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f00e4102e00 0x7f00e4115de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:19:02.593 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:02.597+0000 7f00eac00700 1 --2- 172.21.15.18:0/1213571173 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f00e41037a0 0x7f00e4116320 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:19:02.593 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:02.597+0000 7f00eac00700 1 -- 172.21.15.18:0/1213571173 >> 172.21.15.18:0/1213571173 conn(0x7f00e40f89b0 msgr2=0x7f00e4102210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:19:02.593 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:02.597+0000 7f00eac00700 1 -- 172.21.15.18:0/1213571173 shutdown_connections 2024-04-12T21:19:02.593 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:02.597+0000 7f00eac00700 1 -- 172.21.15.18:0/1213571173 wait complete. 2024-04-12T21:19:02.755 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:02.760+0000 7fd7bbe00700 1 -- 172.21.15.18:0/3405803434 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd7bc104e60 msgr2=0x7fd7bc105230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:19:02.755 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:02.760+0000 7fd7bbe00700 1 --2- 172.21.15.18:0/3405803434 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd7bc104e60 0x7fd7bc105230 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7fd7ac004940 tx=0x7fd7ac009e30 comp rx=0 tx=0).stop 2024-04-12T21:19:02.755 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:02.760+0000 7fd7bbe00700 1 -- 172.21.15.18:0/3405803434 shutdown_connections 2024-04-12T21:19:02.756 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:02.760+0000 7fd7bbe00700 1 --2- 172.21.15.18:0/3405803434 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd7bc105800 0x7fd7bc0fee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:19:02.756 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:02.760+0000 7fd7bbe00700 1 --2- 172.21.15.18:0/3405803434 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd7bc104e60 0x7fd7bc105230 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:19:02.756 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:02.760+0000 7fd7bbe00700 1 -- 172.21.15.18:0/3405803434 >> 172.21.15.18:0/3405803434 conn(0x7fd7bc0fab10 msgr2=0x7fd7bc0fcef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:19:02.756 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:02.760+0000 7fd7bbe00700 1 -- 172.21.15.18:0/3405803434 shutdown_connections 2024-04-12T21:19:02.756 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:02.760+0000 7fd7bbe00700 1 -- 172.21.15.18:0/3405803434 wait complete. 2024-04-12T21:19:02.757 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:02.761+0000 7fd7bbe00700 1 Processor -- start 2024-04-12T21:19:02.757 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:02.761+0000 7fd7bbe00700 1 -- start start 2024-04-12T21:19:02.757 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:02.762+0000 7fd7bbe00700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd7bc104e60 0x7fd7bc115bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:19:02.757 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:02.762+0000 7fd7bbe00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd7bc105800 0x7fd7bc1160f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:19:02.757 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:02.762+0000 7fd7bbe00700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7bc116740 con 0x7fd7bc105800 2024-04-12T21:19:02.757 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:02.762+0000 7fd7bbe00700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7bc10fb30 con 0x7fd7bc104e60 2024-04-12T21:19:02.757 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:02.762+0000 7fd7ba000700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd7bc105800 0x7fd7bc1160f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:19:02.757 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:02.762+0000 7fd7ba000700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd7bc105800 0x7fd7bc1160f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:58972/0 (socket says 172.21.15.18:58972) 2024-04-12T21:19:02.758 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:02.762+0000 7fd7ba000700 1 -- 172.21.15.18:0/3798602313 learned_addr learned my addr 172.21.15.18:0/3798602313 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:19:02.759 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:02.763+0000 7fd7baa00700 1 --2- 172.21.15.18:0/3798602313 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd7bc104e60 0x7fd7bc115bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:19:02.759 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:02.763+0000 7fd7ba000700 1 -- 172.21.15.18:0/3798602313 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd7bc104e60 msgr2=0x7fd7bc115bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:19:02.759 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:02.763+0000 7fd7ba000700 1 --2- 172.21.15.18:0/3798602313 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd7bc104e60 0x7fd7bc115bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:19:02.759 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:02.763+0000 7fd7ba000700 1 -- 172.21.15.18:0/3798602313 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7ac005040 con 0x7fd7bc105800 2024-04-12T21:19:02.759 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:02.763+0000 7fd7ba000700 1 --2- 172.21.15.18:0/3798602313 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd7bc105800 0x7fd7bc1160f0 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7fd7a8002a50 tx=0x7fd7a8004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:19:02.759 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:02.763+0000 7fd7b2a00700 1 -- 172.21.15.18:0/3798602313 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd7a8007cc0 con 0x7fd7bc105800 2024-04-12T21:19:02.759 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:02.763+0000 7fd7b2a00700 1 -- 172.21.15.18:0/3798602313 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd7a800f040 con 0x7fd7bc105800 2024-04-12T21:19:02.759 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:02.763+0000 7fd7b2a00700 1 -- 172.21.15.18:0/3798602313 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd7a800daf0 con 0x7fd7bc105800 2024-04-12T21:19:02.759 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:02.763+0000 7fd7bbe00700 1 -- 172.21.15.18:0/3798602313 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd7bc10fe10 con 0x7fd7bc105800 2024-04-12T21:19:02.759 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:02.764+0000 7fd7bbe00700 1 -- 172.21.15.18:0/3798602313 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd7bc1103d0 con 0x7fd7bc105800 2024-04-12T21:19:02.760 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:02.765+0000 7fd7bbe00700 1 -- 172.21.15.18:0/3798602313 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd7bc1005b0 con 0x7fd7bc105800 2024-04-12T21:19:02.761 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:02.766+0000 7fd7b2a00700 1 -- 172.21.15.18:0/3798602313 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7fd7a8009810 con 0x7fd7bc105800 2024-04-12T21:19:02.762 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:02.767+0000 7fd7b2a00700 1 --2- 172.21.15.18:0/3798602313 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fd7a006ec00 0x7fd7a00710b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:19:02.762 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:02.767+0000 7fd7b2a00700 1 -- 172.21.15.18:0/3798602313 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 6757+0+0 (secure 0 0 0) 0x7fd7a808e640 con 0x7fd7bc105800 2024-04-12T21:19:02.762 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:02.767+0000 7fd7baa00700 1 --2- 172.21.15.18:0/3798602313 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fd7a006ec00 0x7fd7a00710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:19:02.763 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:02.768+0000 7fd7baa00700 1 --2- 172.21.15.18:0/3798602313 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fd7a006ec00 0x7fd7a00710b0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fd7ac00b3b0 tx=0x7fd7ac009e30 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:19:02.767 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:02.772+0000 7fd7b2a00700 1 -- 172.21.15.18:0/3798602313 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7fd7a805ae90 con 0x7fd7bc105800 2024-04-12T21:19:03.009 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:03.014+0000 7fd7bbe00700 1 -- 172.21.15.18:0/3798602313 --> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fd7bc10bf60 con 0x7fd7a006ec00 2024-04-12T21:19:03.027 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:03.032+0000 7fd7b2a00700 1 -- 172.21.15.18:0/3798602313 <== mgr.34102 v2:172.21.15.18:6800/3812891919 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7fd7bc10bf60 con 0x7fd7a006ec00 2024-04-12T21:19:03.028 INFO:teuthology.orchestra.run.smithi018.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-12T21:19:03.029 INFO:teuthology.orchestra.run.smithi018.stdout:alertmanager.smithi018 smithi018 *:9093,9094 running (19m) 11m ago 34m 14.7M - 0.25.0 c8568f914cd2 c332090068e8 2024-04-12T21:19:03.029 INFO:teuthology.orchestra.run.smithi018.stdout:crash.smithi018 smithi018 running (15m) 11m ago 34m 6665k - 19.0.0-2879-g324baf8d 8717720dc5e5 b7f80d1f7982 2024-04-12T21:19:03.029 INFO:teuthology.orchestra.run.smithi018.stdout:crash.smithi186 smithi186 running (15m) 12m ago 33m 6660k - 19.0.0-2879-g324baf8d 8717720dc5e5 a4f55c81b623 2024-04-12T21:19:03.029 INFO:teuthology.orchestra.run.smithi018.stdout:grafana.smithi018 smithi018 *:3000 running (19m) 11m ago 33m 74.8M - 9.4.12 2bacad6d85d8 b08e3bf176dd 2024-04-12T21:19:03.029 INFO:teuthology.orchestra.run.smithi018.stdout:mds.cephfs.smithi018.acqbsc smithi018 running (11m) 11m ago 27m 11.1M - 19.0.0-2879-g324baf8d 8717720dc5e5 021f99ab24ae 2024-04-12T21:19:03.029 INFO:teuthology.orchestra.run.smithi018.stdout:mds.cephfs.smithi018.szjluo smithi018 running (11m) 11m ago 26m 10.9M - 19.0.0-2879-g324baf8d 8717720dc5e5 0003117479d4 2024-04-12T21:19:03.029 INFO:teuthology.orchestra.run.smithi018.stdout:mds.cephfs.smithi186.jdllyn smithi186 running (26m) 12m ago 26m 15.6M - 17.2.7-827-gd3b5060a 21de9b7369ad 2b2bc46bd29a 2024-04-12T21:19:03.029 INFO:teuthology.orchestra.run.smithi018.stdout:mds.cephfs.smithi186.wjnafs smithi186 running (26m) 12m ago 26m 15.4M - 17.2.7-827-gd3b5060a 21de9b7369ad 74b115d6d6fd 2024-04-12T21:19:03.029 INFO:teuthology.orchestra.run.smithi018.stdout:mgr.smithi018.xjzwga smithi018 *:8443,9283,8765 running (22m) 11m ago 35m 509M - 19.0.0-2879-g324baf8d 8717720dc5e5 56088305eab7 2024-04-12T21:19:03.029 INFO:teuthology.orchestra.run.smithi018.stdout:mgr.smithi186.kqcfxk smithi186 *:8443,9283,8765 running (21m) 12m ago 32m 452M - 19.0.0-2879-g324baf8d 8717720dc5e5 09cd766e5e0d 2024-04-12T21:19:03.029 INFO:teuthology.orchestra.run.smithi018.stdout:mon.smithi018 smithi018 running (17m) 11m ago 36m 57.0M 2048M 19.0.0-2879-g324baf8d 8717720dc5e5 df3e201dfed6 2024-04-12T21:19:03.029 INFO:teuthology.orchestra.run.smithi018.stdout:mon.smithi186 smithi186 running (16m) 12m ago 32m 37.7M 2048M 19.0.0-2879-g324baf8d 8717720dc5e5 c77498da9b2b 2024-04-12T21:19:03.029 INFO:teuthology.orchestra.run.smithi018.stdout:node-exporter.smithi018 smithi018 *:9100 running (20m) 11m ago 34m 7780k - 1.5.0 0da6a335fe13 326c3d9c6847 2024-04-12T21:19:03.029 INFO:teuthology.orchestra.run.smithi018.stdout:node-exporter.smithi186 smithi186 *:9100 running (20m) 12m ago 32m 7826k - 1.5.0 0da6a335fe13 ec2112baeecf 2024-04-12T21:19:03.030 INFO:teuthology.orchestra.run.smithi018.stdout:osd.0 smithi018 running (15m) 11m ago 31m 121M 1559M 19.0.0-2879-g324baf8d 8717720dc5e5 60b1273ca047 2024-04-12T21:19:03.030 INFO:teuthology.orchestra.run.smithi018.stdout:osd.1 smithi018 running (14m) 11m ago 31m 91.3M 1559M 19.0.0-2879-g324baf8d 8717720dc5e5 170ce1e5fb8d 2024-04-12T21:19:03.030 INFO:teuthology.orchestra.run.smithi018.stdout:osd.2 smithi018 running (14m) 11m ago 30m 90.3M 1559M 19.0.0-2879-g324baf8d 8717720dc5e5 c2cbc6aa2839 2024-04-12T21:19:03.030 INFO:teuthology.orchestra.run.smithi018.stdout:osd.3 smithi186 running (13m) 12m ago 29m 96.6M 2581M 19.0.0-2879-g324baf8d 8717720dc5e5 0138e8262e4b 2024-04-12T21:19:03.030 INFO:teuthology.orchestra.run.smithi018.stdout:osd.4 smithi186 running (13m) 12m ago 29m 87.2M 2581M 19.0.0-2879-g324baf8d 8717720dc5e5 b20cb63b25c9 2024-04-12T21:19:03.030 INFO:teuthology.orchestra.run.smithi018.stdout:osd.5 smithi186 running (12m) 12m ago 28m 11.1M 2581M 19.0.0-2879-g324baf8d 8717720dc5e5 2513bea2610e 2024-04-12T21:19:03.030 INFO:teuthology.orchestra.run.smithi018.stdout:prometheus.smithi018 smithi018 *:9095 running (20m) 11m ago 33m 60.6M - 2.43.0 a07b618ecd1d f58841127e21 2024-04-12T21:19:03.031 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:03.035+0000 7fd7bbe00700 1 -- 172.21.15.18:0/3798602313 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fd7a006ec00 msgr2=0x7fd7a00710b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:19:03.031 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:03.035+0000 7fd7bbe00700 1 --2- 172.21.15.18:0/3798602313 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fd7a006ec00 0x7fd7a00710b0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fd7ac00b3b0 tx=0x7fd7ac009e30 comp rx=0 tx=0).stop 2024-04-12T21:19:03.031 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:03.035+0000 7fd7bbe00700 1 -- 172.21.15.18:0/3798602313 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd7bc105800 msgr2=0x7fd7bc1160f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:19:03.031 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:03.035+0000 7fd7bbe00700 1 --2- 172.21.15.18:0/3798602313 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd7bc105800 0x7fd7bc1160f0 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7fd7a8002a50 tx=0x7fd7a8004600 comp rx=0 tx=0).stop 2024-04-12T21:19:03.031 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:03.035+0000 7fd7bbe00700 1 -- 172.21.15.18:0/3798602313 shutdown_connections 2024-04-12T21:19:03.031 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:03.035+0000 7fd7bbe00700 1 --2- 172.21.15.18:0/3798602313 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fd7a006ec00 0x7fd7a00710b0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:19:03.031 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:03.035+0000 7fd7bbe00700 1 --2- 172.21.15.18:0/3798602313 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd7bc104e60 0x7fd7bc115bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:19:03.031 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:03.035+0000 7fd7bbe00700 1 --2- 172.21.15.18:0/3798602313 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd7bc105800 0x7fd7bc1160f0 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:19:03.031 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:03.035+0000 7fd7bbe00700 1 -- 172.21.15.18:0/3798602313 >> 172.21.15.18:0/3798602313 conn(0x7fd7bc0fab10 msgr2=0x7fd7bc0fca30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:19:03.031 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:03.036+0000 7fd7bbe00700 1 -- 172.21.15.18:0/3798602313 shutdown_connections 2024-04-12T21:19:03.031 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:03.036+0000 7fd7bbe00700 1 -- 172.21.15.18:0/3798602313 wait complete. 2024-04-12T21:19:03.185 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:03.189+0000 7f9d71e00700 1 -- 172.21.15.18:0/2963341636 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9d6c105020 msgr2=0x7f9d6c1053f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:19:03.185 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:03.189+0000 7f9d71e00700 1 --2- 172.21.15.18:0/2963341636 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9d6c105020 0x7f9d6c1053f0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f9d5c004940 tx=0x7f9d5c009e30 comp rx=0 tx=0).stop 2024-04-12T21:19:03.185 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:03.189+0000 7f9d71e00700 1 -- 172.21.15.18:0/2963341636 shutdown_connections 2024-04-12T21:19:03.185 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:03.189+0000 7f9d71e00700 1 --2- 172.21.15.18:0/2963341636 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9d6c1059c0 0x7f9d6c0ff080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:19:03.185 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:03.189+0000 7f9d71e00700 1 --2- 172.21.15.18:0/2963341636 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9d6c105020 0x7f9d6c1053f0 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:19:03.185 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:03.189+0000 7f9d71e00700 1 -- 172.21.15.18:0/2963341636 >> 172.21.15.18:0/2963341636 conn(0x7f9d6c0faba0 msgr2=0x7f9d6c0fcfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:19:03.185 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:03.189+0000 7f9d71e00700 1 -- 172.21.15.18:0/2963341636 shutdown_connections 2024-04-12T21:19:03.185 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:03.189+0000 7f9d71e00700 1 -- 172.21.15.18:0/2963341636 wait complete. 2024-04-12T21:19:03.187 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:03.190+0000 7f9d71e00700 1 Processor -- start 2024-04-12T21:19:03.187 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:03.191+0000 7f9d71e00700 1 -- start start 2024-04-12T21:19:03.187 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:03.191+0000 7f9d71e00700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9d6c105020 0x7f9d6c115b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:19:03.187 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:03.191+0000 7f9d71e00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9d6c1059c0 0x7f9d6c1160d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:19:03.187 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:03.191+0000 7f9d71e00700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d6c116720 con 0x7f9d6c1059c0 2024-04-12T21:19:03.187 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:03.191+0000 7f9d71e00700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d6c10fd00 con 0x7f9d6c105020 2024-04-12T21:19:03.187 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:03.191+0000 7f9d63e00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9d6c1059c0 0x7f9d6c1160d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:19:03.188 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:03.191+0000 7f9d63e00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9d6c1059c0 0x7f9d6c1160d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:58996/0 (socket says 172.21.15.18:58996) 2024-04-12T21:19:03.188 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:03.191+0000 7f9d63e00700 1 -- 172.21.15.18:0/2812467083 learned_addr learned my addr 172.21.15.18:0/2812467083 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:19:03.188 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:03.192+0000 7f9d6aa00700 1 --2- 172.21.15.18:0/2812467083 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9d6c105020 0x7f9d6c115b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:19:03.188 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:03.192+0000 7f9d63e00700 1 -- 172.21.15.18:0/2812467083 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9d6c105020 msgr2=0x7f9d6c115b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:19:03.188 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:03.192+0000 7f9d63e00700 1 --2- 172.21.15.18:0/2812467083 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9d6c105020 0x7f9d6c115b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:19:03.188 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:03.192+0000 7f9d63e00700 1 -- 172.21.15.18:0/2812467083 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d5c005040 con 0x7f9d6c1059c0 2024-04-12T21:19:03.188 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:03.192+0000 7f9d63e00700 1 --2- 172.21.15.18:0/2812467083 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9d6c1059c0 0x7f9d6c1160d0 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f9d5400c800 tx=0x7f9d54004470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:19:03.189 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:03.192+0000 7f9d62a00700 1 -- 172.21.15.18:0/2812467083 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9d5400a760 con 0x7f9d6c1059c0 2024-04-12T21:19:03.189 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:03.192+0000 7f9d62a00700 1 -- 172.21.15.18:0/2812467083 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f9d5400ad90 con 0x7f9d6c1059c0 2024-04-12T21:19:03.189 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:03.192+0000 7f9d62a00700 1 -- 172.21.15.18:0/2812467083 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9d540113a0 con 0x7f9d6c1059c0 2024-04-12T21:19:03.189 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:03.193+0000 7f9d71e00700 1 -- 172.21.15.18:0/2812467083 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9d6c10ffe0 con 0x7f9d6c1059c0 2024-04-12T21:19:03.189 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:03.193+0000 7f9d71e00700 1 -- 172.21.15.18:0/2812467083 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9d6c1105a0 con 0x7f9d6c1059c0 2024-04-12T21:19:03.189 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:03.194+0000 7f9d71e00700 1 -- 172.21.15.18:0/2812467083 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9d6c100770 con 0x7f9d6c1059c0 2024-04-12T21:19:03.190 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:03.195+0000 7f9d62a00700 1 -- 172.21.15.18:0/2812467083 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7f9d54012400 con 0x7f9d6c1059c0 2024-04-12T21:19:03.191 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:03.196+0000 7f9d62a00700 1 --2- 172.21.15.18:0/2812467083 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f9d5006ec00 0x7f9d500710b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:19:03.191 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:03.197+0000 7f9d62a00700 1 -- 172.21.15.18:0/2812467083 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 6757+0+0 (secure 0 0 0) 0x7f9d5408e950 con 0x7f9d6c1059c0 2024-04-12T21:19:03.192 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:03.197+0000 7f9d6aa00700 1 --2- 172.21.15.18:0/2812467083 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f9d5006ec00 0x7f9d500710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:19:03.192 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:03.198+0000 7f9d6aa00700 1 --2- 172.21.15.18:0/2812467083 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f9d5006ec00 0x7f9d500710b0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f9d5c00b3b0 tx=0x7f9d5c009e30 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:19:03.197 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:03.203+0000 7f9d62a00700 1 -- 172.21.15.18:0/2812467083 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f9d5405b180 con 0x7f9d6c1059c0 2024-04-12T21:19:03.349 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:19:03 smithi018 ceph-mon[112590]: from='client.35032 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:19:03.350 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:19:03 smithi018 ceph-mon[112590]: pgmap v577: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:19:03.350 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:19:03 smithi018 ceph-mon[112590]: from='client.35036 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:19:03.515 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:03.520+0000 7f9d71e00700 1 -- 172.21.15.18:0/2812467083 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f9d6c10c150 con 0x7f9d6c1059c0 2024-04-12T21:19:03.516 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:03.521+0000 7f9d62a00700 1 -- 172.21.15.18:0/2812467083 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+837 (secure 0 0 0) 0x7f9d5405ad10 con 0x7f9d6c1059c0 2024-04-12T21:19:03.516 INFO:teuthology.orchestra.run.smithi018.stdout:{ 2024-04-12T21:19:03.516 INFO:teuthology.orchestra.run.smithi018.stdout: "mon": { 2024-04-12T21:19:03.516 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 19.0.0-2879-g324baf8d (324baf8ddb91dea46cac50cb535c306115791355) squid (dev)": 2 2024-04-12T21:19:03.516 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-12T21:19:03.517 INFO:teuthology.orchestra.run.smithi018.stdout: "mgr": { 2024-04-12T21:19:03.517 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 19.0.0-2879-g324baf8d (324baf8ddb91dea46cac50cb535c306115791355) squid (dev)": 2 2024-04-12T21:19:03.517 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-12T21:19:03.517 INFO:teuthology.orchestra.run.smithi018.stdout: "osd": { 2024-04-12T21:19:03.517 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 19.0.0-2879-g324baf8d (324baf8ddb91dea46cac50cb535c306115791355) squid (dev)": 6 2024-04-12T21:19:03.517 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-12T21:19:03.517 INFO:teuthology.orchestra.run.smithi018.stdout: "mds": { 2024-04-12T21:19:03.517 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 17.2.7-827-gd3b5060a (d3b5060a17449edfae9385cdf792afec109fca5d) quincy (stable)": 2, 2024-04-12T21:19:03.517 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 19.0.0-2879-g324baf8d (324baf8ddb91dea46cac50cb535c306115791355) squid (dev)": 2 2024-04-12T21:19:03.517 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-12T21:19:03.517 INFO:teuthology.orchestra.run.smithi018.stdout: "overall": { 2024-04-12T21:19:03.517 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 17.2.7-827-gd3b5060a (d3b5060a17449edfae9385cdf792afec109fca5d) quincy (stable)": 2, 2024-04-12T21:19:03.517 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 19.0.0-2879-g324baf8d (324baf8ddb91dea46cac50cb535c306115791355) squid (dev)": 12 2024-04-12T21:19:03.517 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-04-12T21:19:03.517 INFO:teuthology.orchestra.run.smithi018.stdout:} 2024-04-12T21:19:03.519 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:03.524+0000 7f9d71e00700 1 -- 172.21.15.18:0/2812467083 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f9d5006ec00 msgr2=0x7f9d500710b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:19:03.519 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:03.524+0000 7f9d71e00700 1 --2- 172.21.15.18:0/2812467083 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f9d5006ec00 0x7f9d500710b0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f9d5c00b3b0 tx=0x7f9d5c009e30 comp rx=0 tx=0).stop 2024-04-12T21:19:03.519 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:03.524+0000 7f9d71e00700 1 -- 172.21.15.18:0/2812467083 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9d6c1059c0 msgr2=0x7f9d6c1160d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:19:03.519 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:03.524+0000 7f9d71e00700 1 --2- 172.21.15.18:0/2812467083 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9d6c1059c0 0x7f9d6c1160d0 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f9d5400c800 tx=0x7f9d54004470 comp rx=0 tx=0).stop 2024-04-12T21:19:03.519 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:03.524+0000 7f9d71e00700 1 -- 172.21.15.18:0/2812467083 shutdown_connections 2024-04-12T21:19:03.520 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:03.524+0000 7f9d71e00700 1 --2- 172.21.15.18:0/2812467083 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f9d5006ec00 0x7f9d500710b0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:19:03.520 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:03.524+0000 7f9d71e00700 1 --2- 172.21.15.18:0/2812467083 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9d6c105020 0x7f9d6c115b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:19:03.520 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:03.524+0000 7f9d71e00700 1 --2- 172.21.15.18:0/2812467083 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9d6c1059c0 0x7f9d6c1160d0 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:19:03.520 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:03.524+0000 7f9d71e00700 1 -- 172.21.15.18:0/2812467083 >> 172.21.15.18:0/2812467083 conn(0x7f9d6c0faba0 msgr2=0x7f9d6c0fbb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:19:03.520 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:03.524+0000 7f9d71e00700 1 -- 172.21.15.18:0/2812467083 shutdown_connections 2024-04-12T21:19:03.520 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:03.524+0000 7f9d71e00700 1 -- 172.21.15.18:0/2812467083 wait complete. 2024-04-12T21:19:03.528 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:19:03 smithi186 ceph-mon[86379]: from='client.35032 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:19:03.528 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:19:03 smithi186 ceph-mon[86379]: pgmap v577: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:19:03.528 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:19:03 smithi186 ceph-mon[86379]: from='client.35036 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:19:03.673 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:03.676+0000 7fc126e00700 1 -- 172.21.15.18:0/2861994904 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc120109430 msgr2=0x7fc120109800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:19:03.673 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:03.676+0000 7fc126e00700 1 --2- 172.21.15.18:0/2861994904 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc120109430 0x7fc120109800 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7fc110004940 tx=0x7fc110009e30 comp rx=0 tx=0).stop 2024-04-12T21:19:03.673 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:03.676+0000 7fc126e00700 1 -- 172.21.15.18:0/2861994904 shutdown_connections 2024-04-12T21:19:03.673 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:03.676+0000 7fc126e00700 1 --2- 172.21.15.18:0/2861994904 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc120109dd0 0x7fc120103440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:19:03.673 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:03.676+0000 7fc126e00700 1 --2- 172.21.15.18:0/2861994904 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc120109430 0x7fc120109800 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:19:03.673 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:03.676+0000 7fc126e00700 1 -- 172.21.15.18:0/2861994904 >> 172.21.15.18:0/2861994904 conn(0x7fc120072c00 msgr2=0x7fc120075010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:19:03.673 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:03.676+0000 7fc126e00700 1 -- 172.21.15.18:0/2861994904 shutdown_connections 2024-04-12T21:19:03.674 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:03.676+0000 7fc126e00700 1 -- 172.21.15.18:0/2861994904 wait complete. 2024-04-12T21:19:03.674 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:03.677+0000 7fc126e00700 1 Processor -- start 2024-04-12T21:19:03.674 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:03.677+0000 7fc126e00700 1 -- start start 2024-04-12T21:19:03.674 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:03.677+0000 7fc126e00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc120109dd0 0x7fc12011a120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:19:03.674 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:03.677+0000 7fc126e00700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc12011a660 0x7fc120114290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:19:03.674 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:03.677+0000 7fc126e00700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc12011abe0 con 0x7fc120109dd0 2024-04-12T21:19:03.674 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:03.677+0000 7fc126e00700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc12011ad50 con 0x7fc12011a660 2024-04-12T21:19:03.674 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:03.678+0000 7fc11fe00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc120109dd0 0x7fc12011a120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:19:03.674 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:03.678+0000 7fc11fe00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc120109dd0 0x7fc12011a120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:59010/0 (socket says 172.21.15.18:59010) 2024-04-12T21:19:03.674 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:03.678+0000 7fc11fe00700 1 -- 172.21.15.18:0/446398365 learned_addr learned my addr 172.21.15.18:0/446398365 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:19:03.674 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:03.678+0000 7fc11f400700 1 --2- 172.21.15.18:0/446398365 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc12011a660 0x7fc120114290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:19:03.674 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:03.678+0000 7fc11fe00700 1 -- 172.21.15.18:0/446398365 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc12011a660 msgr2=0x7fc120114290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:19:03.674 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:03.678+0000 7fc11fe00700 1 --2- 172.21.15.18:0/446398365 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc12011a660 0x7fc120114290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:19:03.675 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:03.678+0000 7fc11fe00700 1 -- 172.21.15.18:0/446398365 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc110005040 con 0x7fc120109dd0 2024-04-12T21:19:03.675 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:03.678+0000 7fc11f400700 1 --2- 172.21.15.18:0/446398365 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc12011a660 0x7fc120114290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-12T21:19:03.675 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:03.678+0000 7fc11fe00700 1 --2- 172.21.15.18:0/446398365 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc120109dd0 0x7fc12011a120 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7fc110000c00 tx=0x7fc110009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:19:03.676 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:03.679+0000 7fc117e00700 1 -- 172.21.15.18:0/446398365 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc11000e040 con 0x7fc120109dd0 2024-04-12T21:19:03.676 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:03.679+0000 7fc117e00700 1 -- 172.21.15.18:0/446398365 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc110009430 con 0x7fc120109dd0 2024-04-12T21:19:03.676 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:03.679+0000 7fc126e00700 1 -- 172.21.15.18:0/446398365 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc120114890 con 0x7fc120109dd0 2024-04-12T21:19:03.676 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:03.679+0000 7fc117e00700 1 -- 172.21.15.18:0/446398365 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc11001c3f0 con 0x7fc120109dd0 2024-04-12T21:19:03.676 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:03.679+0000 7fc126e00700 1 -- 172.21.15.18:0/446398365 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc120114e50 con 0x7fc120109dd0 2024-04-12T21:19:03.676 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:03.680+0000 7fc126e00700 1 -- 172.21.15.18:0/446398365 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc1201150c0 con 0x7fc120109dd0 2024-04-12T21:19:03.676 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:03.681+0000 7fc117e00700 1 -- 172.21.15.18:0/446398365 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7fc110013840 con 0x7fc120109dd0 2024-04-12T21:19:03.677 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:03.682+0000 7fc117e00700 1 --2- 172.21.15.18:0/446398365 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fc10406ec00 0x7fc1040710b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:19:03.677 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:03.682+0000 7fc117e00700 1 -- 172.21.15.18:0/446398365 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 6757+0+0 (secure 0 0 0) 0x7fc11008fd10 con 0x7fc120109dd0 2024-04-12T21:19:03.677 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:03.683+0000 7fc11f400700 1 --2- 172.21.15.18:0/446398365 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fc10406ec00 0x7fc1040710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:19:03.678 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:03.683+0000 7fc11f400700 1 --2- 172.21.15.18:0/446398365 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fc10406ec00 0x7fc1040710b0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fc10c009fd0 tx=0x7fc10c00b040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:19:03.682 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:03.687+0000 7fc117e00700 1 -- 172.21.15.18:0/446398365 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7fc11005c540 con 0x7fc120109dd0 2024-04-12T21:19:03.959 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:03.964+0000 7fc126e00700 1 -- 172.21.15.18:0/446398365 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fc120115770 con 0x7fc120109dd0 2024-04-12T21:19:03.959 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:03.965+0000 7fc117e00700 1 -- 172.21.15.18:0/446398365 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 29 v29) v1 ==== 76+0+1872 (secure 0 0 0) 0x7fc11005c0d0 con 0x7fc120109dd0 2024-04-12T21:19:03.960 INFO:teuthology.orchestra.run.smithi018.stdout:e29 2024-04-12T21:19:03.960 INFO:teuthology.orchestra.run.smithi018.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-12T21:19:03.960 INFO:teuthology.orchestra.run.smithi018.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-12T21:19:03.960 INFO:teuthology.orchestra.run.smithi018.stdout:legacy client fscid: 1 2024-04-12T21:19:03.960 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-12T21:19:03.960 INFO:teuthology.orchestra.run.smithi018.stdout:Filesystem 'cephfs' (1) 2024-04-12T21:19:03.960 INFO:teuthology.orchestra.run.smithi018.stdout:fs_name cephfs 2024-04-12T21:19:03.960 INFO:teuthology.orchestra.run.smithi018.stdout:epoch 24 2024-04-12T21:19:03.960 INFO:teuthology.orchestra.run.smithi018.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-04-12T21:19:03.961 INFO:teuthology.orchestra.run.smithi018.stdout:created 2024-04-12T20:51:57.362850+0000 2024-04-12T21:19:03.961 INFO:teuthology.orchestra.run.smithi018.stdout:modified 2024-04-12T21:06:59.717190+0000 2024-04-12T21:19:03.961 INFO:teuthology.orchestra.run.smithi018.stdout:tableserver 0 2024-04-12T21:19:03.961 INFO:teuthology.orchestra.run.smithi018.stdout:root 0 2024-04-12T21:19:03.961 INFO:teuthology.orchestra.run.smithi018.stdout:session_timeout 60 2024-04-12T21:19:03.961 INFO:teuthology.orchestra.run.smithi018.stdout:session_autoclose 300 2024-04-12T21:19:03.961 INFO:teuthology.orchestra.run.smithi018.stdout:max_file_size 1099511627776 2024-04-12T21:19:03.961 INFO:teuthology.orchestra.run.smithi018.stdout:max_xattr_size 65536 2024-04-12T21:19:03.961 INFO:teuthology.orchestra.run.smithi018.stdout:required_client_features {} 2024-04-12T21:19:03.961 INFO:teuthology.orchestra.run.smithi018.stdout:last_failure 0 2024-04-12T21:19:03.961 INFO:teuthology.orchestra.run.smithi018.stdout:last_failure_osd_epoch 100 2024-04-12T21:19:03.961 INFO:teuthology.orchestra.run.smithi018.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-12T21:19:03.961 INFO:teuthology.orchestra.run.smithi018.stdout:max_mds 2 2024-04-12T21:19:03.961 INFO:teuthology.orchestra.run.smithi018.stdout:in 0,1 2024-04-12T21:19:03.961 INFO:teuthology.orchestra.run.smithi018.stdout:up {} 2024-04-12T21:19:03.962 INFO:teuthology.orchestra.run.smithi018.stdout:failed 0,1 2024-04-12T21:19:03.962 INFO:teuthology.orchestra.run.smithi018.stdout:damaged 2024-04-12T21:19:03.962 INFO:teuthology.orchestra.run.smithi018.stdout:stopped 2024-04-12T21:19:03.962 INFO:teuthology.orchestra.run.smithi018.stdout:data_pools [3] 2024-04-12T21:19:03.962 INFO:teuthology.orchestra.run.smithi018.stdout:metadata_pool 2 2024-04-12T21:19:03.962 INFO:teuthology.orchestra.run.smithi018.stdout:inline_data disabled 2024-04-12T21:19:03.962 INFO:teuthology.orchestra.run.smithi018.stdout:balancer 2024-04-12T21:19:03.962 INFO:teuthology.orchestra.run.smithi018.stdout:bal_rank_mask -1 2024-04-12T21:19:03.962 INFO:teuthology.orchestra.run.smithi018.stdout:standby_count_wanted 1 2024-04-12T21:19:03.962 INFO:teuthology.orchestra.run.smithi018.stdout:qdb_cluster leader: 0 members: 2024-04-12T21:19:03.962 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-12T21:19:03.962 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-12T21:19:03.963 INFO:teuthology.orchestra.run.smithi018.stdout:Standby daemons: 2024-04-12T21:19:03.963 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-12T21:19:03.963 INFO:teuthology.orchestra.run.smithi018.stdout:[mds.cephfs.smithi186.wjnafs{-1:24277} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.186:6826/438347207,v1:172.21.15.186:6827/438347207] compat {c=[1],r=[1],i=[7ff]}] 2024-04-12T21:19:03.963 INFO:teuthology.orchestra.run.smithi018.stdout:[mds.cephfs.smithi186.jdllyn{-1:24281} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.186:6824/1464359952,v1:172.21.15.186:6825/1464359952] compat {c=[1],r=[1],i=[7ff]}] 2024-04-12T21:19:03.963 INFO:teuthology.orchestra.run.smithi018.stdout:[mds.cephfs.smithi018.acqbsc{-1:34436} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.18:6828/755179764,v1:172.21.15.18:6829/755179764] compat {c=[1],r=[1],i=[1fff]}] 2024-04-12T21:19:03.963 INFO:teuthology.orchestra.run.smithi018.stdout:[mds.cephfs.smithi018.szjluo{-1:34468} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.18:6826/2407992653,v1:172.21.15.18:6827/2407992653] compat {c=[1],r=[1],i=[1fff]}] 2024-04-12T21:19:03.963 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:03.967+0000 7fc126e00700 1 -- 172.21.15.18:0/446398365 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fc10406ec00 msgr2=0x7fc1040710b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:19:03.964 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:03.967+0000 7fc126e00700 1 --2- 172.21.15.18:0/446398365 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fc10406ec00 0x7fc1040710b0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fc10c009fd0 tx=0x7fc10c00b040 comp rx=0 tx=0).stop 2024-04-12T21:19:03.964 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:03.967+0000 7fc126e00700 1 -- 172.21.15.18:0/446398365 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc120109dd0 msgr2=0x7fc12011a120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:19:03.964 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:03.967+0000 7fc126e00700 1 --2- 172.21.15.18:0/446398365 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc120109dd0 0x7fc12011a120 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7fc110000c00 tx=0x7fc110009e30 comp rx=0 tx=0).stop 2024-04-12T21:19:03.964 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:03.967+0000 7fc126e00700 1 -- 172.21.15.18:0/446398365 shutdown_connections 2024-04-12T21:19:03.964 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:03.967+0000 7fc126e00700 1 --2- 172.21.15.18:0/446398365 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fc10406ec00 0x7fc1040710b0 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:19:03.964 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:03.967+0000 7fc126e00700 1 --2- 172.21.15.18:0/446398365 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc120109dd0 0x7fc12011a120 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:19:03.964 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:03.967+0000 7fc126e00700 1 --2- 172.21.15.18:0/446398365 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc12011a660 0x7fc120114290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:19:03.964 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:03.967+0000 7fc126e00700 1 -- 172.21.15.18:0/446398365 >> 172.21.15.18:0/446398365 conn(0x7fc120072c00 msgr2=0x7fc120074cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:19:03.964 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:03.968+0000 7fc126e00700 1 -- 172.21.15.18:0/446398365 shutdown_connections 2024-04-12T21:19:03.964 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:03.968+0000 7fc126e00700 1 -- 172.21.15.18:0/446398365 wait complete. 2024-04-12T21:19:03.965 INFO:teuthology.orchestra.run.smithi018.stderr:dumped fsmap epoch 29 2024-04-12T21:19:04.123 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:04.128+0000 7f0716e00700 1 -- 172.21.15.18:0/3992772038 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0710072e00 msgr2=0x7f071010d770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:19:04.124 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:04.128+0000 7f0716e00700 1 --2- 172.21.15.18:0/3992772038 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0710072e00 0x7f071010d770 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f06f8004940 tx=0x7f06f8009e30 comp rx=0 tx=0).stop 2024-04-12T21:19:04.124 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:04.128+0000 7f0716e00700 1 -- 172.21.15.18:0/3992772038 shutdown_connections 2024-04-12T21:19:04.124 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:04.128+0000 7f0716e00700 1 --2- 172.21.15.18:0/3992772038 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0710072e00 0x7f071010d770 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:19:04.124 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:04.128+0000 7f0716e00700 1 --2- 172.21.15.18:0/3992772038 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f07100724f0 0x7f07100728c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:19:04.124 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:04.128+0000 7f0716e00700 1 -- 172.21.15.18:0/3992772038 >> 172.21.15.18:0/3992772038 conn(0x7f07100fabd0 msgr2=0x7f07100fcfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:19:04.124 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:04.128+0000 7f0716e00700 1 -- 172.21.15.18:0/3992772038 shutdown_connections 2024-04-12T21:19:04.124 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:04.128+0000 7f0716e00700 1 -- 172.21.15.18:0/3992772038 wait complete. 2024-04-12T21:19:04.125 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:04.129+0000 7f0716e00700 1 Processor -- start 2024-04-12T21:19:04.125 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:04.129+0000 7f0716e00700 1 -- start start 2024-04-12T21:19:04.125 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:04.130+0000 7f0716e00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f07100724f0 0x7f071011a120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:19:04.125 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:04.130+0000 7f0716e00700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0710072e00 0x7f071011a660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:19:04.125 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:04.130+0000 7f0716e00700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f071011acb0 con 0x7f07100724f0 2024-04-12T21:19:04.126 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:04.130+0000 7f0716e00700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07101140c0 con 0x7f0710072e00 2024-04-12T21:19:04.126 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:04.130+0000 7f070fe00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f07100724f0 0x7f071011a120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:19:04.126 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:04.130+0000 7f070fe00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f07100724f0 0x7f071011a120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:59030/0 (socket says 172.21.15.18:59030) 2024-04-12T21:19:04.126 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:04.130+0000 7f070fe00700 1 -- 172.21.15.18:0/1504174592 learned_addr learned my addr 172.21.15.18:0/1504174592 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:19:04.126 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:04.130+0000 7f0707e00700 1 --2- 172.21.15.18:0/1504174592 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0710072e00 0x7f071011a660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:19:04.126 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:04.130+0000 7f070fe00700 1 -- 172.21.15.18:0/1504174592 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0710072e00 msgr2=0x7f071011a660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:19:04.126 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:04.130+0000 7f070fe00700 1 --2- 172.21.15.18:0/1504174592 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0710072e00 0x7f071011a660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:19:04.126 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:04.130+0000 7f070fe00700 1 -- 172.21.15.18:0/1504174592 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06f8005040 con 0x7f07100724f0 2024-04-12T21:19:04.127 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:04.131+0000 7f070fe00700 1 --2- 172.21.15.18:0/1504174592 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f07100724f0 0x7f071011a120 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f0700002a50 tx=0x7f07000046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:19:04.127 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:04.131+0000 7f070cc00700 1 -- 172.21.15.18:0/1504174592 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0700007cc0 con 0x7f07100724f0 2024-04-12T21:19:04.127 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:04.131+0000 7f070cc00700 1 -- 172.21.15.18:0/1504174592 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f070000f040 con 0x7f07100724f0 2024-04-12T21:19:04.127 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:04.131+0000 7f070cc00700 1 -- 172.21.15.18:0/1504174592 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0700009640 con 0x7f07100724f0 2024-04-12T21:19:04.127 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:04.131+0000 7f0716e00700 1 -- 172.21.15.18:0/1504174592 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f07101143a0 con 0x7f07100724f0 2024-04-12T21:19:04.127 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:04.131+0000 7f0716e00700 1 -- 172.21.15.18:0/1504174592 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0710114960 con 0x7f07100724f0 2024-04-12T21:19:04.128 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:04.132+0000 7f0716e00700 1 -- 172.21.15.18:0/1504174592 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0710073c30 con 0x7f07100724f0 2024-04-12T21:19:04.128 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:04.134+0000 7f070cc00700 1 -- 172.21.15.18:0/1504174592 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7f07000097a0 con 0x7f07100724f0 2024-04-12T21:19:04.129 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:04.135+0000 7f070cc00700 1 --2- 172.21.15.18:0/1504174592 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f06f406ec10 0x7f06f40710c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:19:04.130 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:04.135+0000 7f070cc00700 1 -- 172.21.15.18:0/1504174592 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 6757+0+0 (secure 0 0 0) 0x7f070008e760 con 0x7f07100724f0 2024-04-12T21:19:04.130 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:04.135+0000 7f0707e00700 1 --2- 172.21.15.18:0/1504174592 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f06f406ec10 0x7f06f40710c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:19:04.130 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:04.136+0000 7f0707e00700 1 --2- 172.21.15.18:0/1504174592 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f06f406ec10 0x7f06f40710c0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f0710115a00 tx=0x7f06f8009e30 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:19:04.134 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:04.139+0000 7f070cc00700 1 -- 172.21.15.18:0/1504174592 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f07000573e0 con 0x7f07100724f0 2024-04-12T21:19:04.397 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:04.400+0000 7f0716e00700 1 -- 172.21.15.18:0/1504174592 --> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f071004bf40 con 0x7f06f406ec10 2024-04-12T21:19:04.399 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:04.404+0000 7f070cc00700 1 -- 172.21.15.18:0/1504174592 <== mgr.34102 v2:172.21.15.18:6800/3812891919 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f071004bf40 con 0x7f06f406ec10 2024-04-12T21:19:04.399 INFO:teuthology.orchestra.run.smithi018.stdout:{ 2024-04-12T21:19:04.399 INFO:teuthology.orchestra.run.smithi018.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:f4452def3be4d350f4a8e5feaf7ebb1511be9298fe68478ba7a2d6fbd7d69a63", 2024-04-12T21:19:04.399 INFO:teuthology.orchestra.run.smithi018.stdout: "in_progress": true, 2024-04-12T21:19:04.399 INFO:teuthology.orchestra.run.smithi018.stdout: "which": "Upgrading all daemon types on all hosts", 2024-04-12T21:19:04.399 INFO:teuthology.orchestra.run.smithi018.stdout: "services_complete": [ 2024-04-12T21:19:04.399 INFO:teuthology.orchestra.run.smithi018.stdout: "crash", 2024-04-12T21:19:04.399 INFO:teuthology.orchestra.run.smithi018.stdout: "osd", 2024-04-12T21:19:04.400 INFO:teuthology.orchestra.run.smithi018.stdout: "mon", 2024-04-12T21:19:04.400 INFO:teuthology.orchestra.run.smithi018.stdout: "mgr" 2024-04-12T21:19:04.400 INFO:teuthology.orchestra.run.smithi018.stdout: ], 2024-04-12T21:19:04.400 INFO:teuthology.orchestra.run.smithi018.stdout: "progress": "14/21 daemons upgraded", 2024-04-12T21:19:04.400 INFO:teuthology.orchestra.run.smithi018.stdout: "message": "Currently upgrading mds daemons", 2024-04-12T21:19:04.400 INFO:teuthology.orchestra.run.smithi018.stdout: "is_paused": false 2024-04-12T21:19:04.400 INFO:teuthology.orchestra.run.smithi018.stdout:} 2024-04-12T21:19:04.402 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:04.407+0000 7f0716e00700 1 -- 172.21.15.18:0/1504174592 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f06f406ec10 msgr2=0x7f06f40710c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:19:04.403 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:04.407+0000 7f0716e00700 1 --2- 172.21.15.18:0/1504174592 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f06f406ec10 0x7f06f40710c0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f0710115a00 tx=0x7f06f8009e30 comp rx=0 tx=0).stop 2024-04-12T21:19:04.403 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:04.407+0000 7f0716e00700 1 -- 172.21.15.18:0/1504174592 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f07100724f0 msgr2=0x7f071011a120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:19:04.403 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:04.407+0000 7f0716e00700 1 --2- 172.21.15.18:0/1504174592 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f07100724f0 0x7f071011a120 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f0700002a50 tx=0x7f07000046b0 comp rx=0 tx=0).stop 2024-04-12T21:19:04.403 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:04.407+0000 7f0716e00700 1 -- 172.21.15.18:0/1504174592 shutdown_connections 2024-04-12T21:19:04.403 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:04.407+0000 7f0716e00700 1 --2- 172.21.15.18:0/1504174592 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f06f406ec10 0x7f06f40710c0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:19:04.403 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:04.407+0000 7f0716e00700 1 --2- 172.21.15.18:0/1504174592 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f07100724f0 0x7f071011a120 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:19:04.403 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:04.407+0000 7f0716e00700 1 --2- 172.21.15.18:0/1504174592 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0710072e00 0x7f071011a660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:19:04.403 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:04.407+0000 7f0716e00700 1 -- 172.21.15.18:0/1504174592 >> 172.21.15.18:0/1504174592 conn(0x7f07100fabd0 msgr2=0x7f07100fafa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:19:04.403 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:04.408+0000 7f0716e00700 1 -- 172.21.15.18:0/1504174592 shutdown_connections 2024-04-12T21:19:04.403 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:04.408+0000 7f0716e00700 1 -- 172.21.15.18:0/1504174592 wait complete. 2024-04-12T21:19:04.528 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:19:04 smithi186 ceph-mon[86379]: from='client.35040 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:19:04.529 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:19:04 smithi186 ceph-mon[86379]: from='client.? 172.21.15.18:0/2812467083' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T21:19:04.529 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:19:04 smithi186 ceph-mon[86379]: from='client.? 172.21.15.18:0/446398365' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-12T21:19:04.555 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:04.560+0000 7f8ed6e00700 1 -- 172.21.15.18:0/1995099330 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8ed01059a0 msgr2=0x7f8ed00ff010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:19:04.555 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:04.560+0000 7f8ed6e00700 1 --2- 172.21.15.18:0/1995099330 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8ed01059a0 0x7f8ed00ff010 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f8ebc004940 tx=0x7f8ebc009e30 comp rx=0 tx=0).stop 2024-04-12T21:19:04.555 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:04.560+0000 7f8ed6e00700 1 -- 172.21.15.18:0/1995099330 shutdown_connections 2024-04-12T21:19:04.555 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:04.560+0000 7f8ed6e00700 1 --2- 172.21.15.18:0/1995099330 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8ed01059a0 0x7f8ed00ff010 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:19:04.556 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:04.560+0000 7f8ed6e00700 1 --2- 172.21.15.18:0/1995099330 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8ed0105000 0x7f8ed01053d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:19:04.556 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:04.560+0000 7f8ed6e00700 1 -- 172.21.15.18:0/1995099330 >> 172.21.15.18:0/1995099330 conn(0x7f8ed00fabd0 msgr2=0x7f8ed00fcfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:19:04.556 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:04.560+0000 7f8ed6e00700 1 -- 172.21.15.18:0/1995099330 shutdown_connections 2024-04-12T21:19:04.556 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:19:04 smithi018 ceph-mon[112590]: from='client.35040 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:19:04.556 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:19:04 smithi018 ceph-mon[112590]: from='client.? 172.21.15.18:0/2812467083' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T21:19:04.556 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:19:04 smithi018 ceph-mon[112590]: from='client.? 172.21.15.18:0/446398365' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-12T21:19:04.558 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:04.560+0000 7f8ed6e00700 1 -- 172.21.15.18:0/1995099330 wait complete. 2024-04-12T21:19:04.558 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:04.561+0000 7f8ed6e00700 1 Processor -- start 2024-04-12T21:19:04.558 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:04.561+0000 7f8ed6e00700 1 -- start start 2024-04-12T21:19:04.558 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:04.562+0000 7f8ed6e00700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8ed0105000 0x7f8ed0115d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:19:04.558 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:04.562+0000 7f8ed6e00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8ed01059a0 0x7f8ed01162c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:19:04.558 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:04.562+0000 7f8ed6e00700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ed0116910 con 0x7f8ed01059a0 2024-04-12T21:19:04.558 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:04.562+0000 7f8ed6e00700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ed010fce0 con 0x7f8ed0105000 2024-04-12T21:19:04.558 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:04.562+0000 7f8ecf400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8ed01059a0 0x7f8ed01162c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:19:04.558 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:04.562+0000 7f8ecf400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8ed01059a0 0x7f8ed01162c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:59062/0 (socket says 172.21.15.18:59062) 2024-04-12T21:19:04.558 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:04.562+0000 7f8ecf400700 1 -- 172.21.15.18:0/3172414058 learned_addr learned my addr 172.21.15.18:0/3172414058 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:19:04.558 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:04.562+0000 7f8ecfe00700 1 --2- 172.21.15.18:0/3172414058 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8ed0105000 0x7f8ed0115d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:19:04.558 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:04.563+0000 7f8ecf400700 1 -- 172.21.15.18:0/3172414058 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8ed0105000 msgr2=0x7f8ed0115d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:19:04.559 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:04.563+0000 7f8ecf400700 1 --2- 172.21.15.18:0/3172414058 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8ed0105000 0x7f8ed0115d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:19:04.559 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:04.563+0000 7f8ecf400700 1 -- 172.21.15.18:0/3172414058 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ebc005040 con 0x7f8ed01059a0 2024-04-12T21:19:04.559 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:04.563+0000 7f8ecf400700 1 --2- 172.21.15.18:0/3172414058 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8ed01059a0 0x7f8ed01162c0 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f8ebc009100 tx=0x7f8ebc009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:19:04.559 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:04.563+0000 7f8ec7e00700 1 -- 172.21.15.18:0/3172414058 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8ebc00e040 con 0x7f8ed01059a0 2024-04-12T21:19:04.559 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:04.563+0000 7f8ec7e00700 1 -- 172.21.15.18:0/3172414058 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f8ebc013420 con 0x7f8ed01059a0 2024-04-12T21:19:04.559 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:04.563+0000 7f8ed6e00700 1 -- 172.21.15.18:0/3172414058 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8ed010ff60 con 0x7f8ed01059a0 2024-04-12T21:19:04.560 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:04.563+0000 7f8ec7e00700 1 -- 172.21.15.18:0/3172414058 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8ebc01c630 con 0x7f8ed01059a0 2024-04-12T21:19:04.560 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:04.563+0000 7f8ed6e00700 1 -- 172.21.15.18:0/3172414058 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8ed0110520 con 0x7f8ed01059a0 2024-04-12T21:19:04.560 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:04.565+0000 7f8ed6e00700 1 -- 172.21.15.18:0/3172414058 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8ed0100750 con 0x7f8ed01059a0 2024-04-12T21:19:04.560 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:04.566+0000 7f8ec7e00700 1 -- 172.21.15.18:0/3172414058 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7f8ebc013590 con 0x7f8ed01059a0 2024-04-12T21:19:04.562 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:04.567+0000 7f8ec7e00700 1 --2- 172.21.15.18:0/3172414058 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f8eb406ec10 0x7f8eb40710c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:19:04.562 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:04.567+0000 7f8ec7e00700 1 -- 172.21.15.18:0/3172414058 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 6757+0+0 (secure 0 0 0) 0x7f8ebc08fba0 con 0x7f8ed01059a0 2024-04-12T21:19:04.562 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:04.567+0000 7f8ecfe00700 1 --2- 172.21.15.18:0/3172414058 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f8eb406ec10 0x7f8eb40710c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:19:04.562 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:04.568+0000 7f8ecfe00700 1 --2- 172.21.15.18:0/3172414058 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f8eb406ec10 0x7f8eb40710c0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f8ec0004990 tx=0x7f8ec0009ee0 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:19:04.566 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:04.572+0000 7f8ec7e00700 1 -- 172.21.15.18:0/3172414058 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f8ebc05c3d0 con 0x7f8ed01059a0 2024-04-12T21:19:04.890 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:04.893+0000 7f8ed6e00700 1 -- 172.21.15.18:0/3172414058 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f8ed010c0e0 con 0x7f8ed01059a0 2024-04-12T21:19:04.890 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:04.894+0000 7f8ec7e00700 1 -- 172.21.15.18:0/3172414058 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+233 (secure 0 0 0) 0x7f8ebc058030 con 0x7f8ed01059a0 2024-04-12T21:19:04.890 INFO:teuthology.orchestra.run.smithi018.stdout:HEALTH_ERR 1 filesystem is degraded; 1 filesystem is offline 2024-04-12T21:19:04.890 INFO:teuthology.orchestra.run.smithi018.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-04-12T21:19:04.890 INFO:teuthology.orchestra.run.smithi018.stdout: fs cephfs is degraded 2024-04-12T21:19:04.890 INFO:teuthology.orchestra.run.smithi018.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-04-12T21:19:04.890 INFO:teuthology.orchestra.run.smithi018.stdout: fs cephfs is offline because no MDS is active for it. 2024-04-12T21:19:04.891 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:04.896+0000 7f8ed6e00700 1 -- 172.21.15.18:0/3172414058 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f8eb406ec10 msgr2=0x7f8eb40710c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:19:04.892 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:04.897+0000 7f8ed6e00700 1 --2- 172.21.15.18:0/3172414058 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f8eb406ec10 0x7f8eb40710c0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f8ec0004990 tx=0x7f8ec0009ee0 comp rx=0 tx=0).stop 2024-04-12T21:19:04.892 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:04.897+0000 7f8ed6e00700 1 -- 172.21.15.18:0/3172414058 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8ed01059a0 msgr2=0x7f8ed01162c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:19:04.892 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:04.897+0000 7f8ed6e00700 1 --2- 172.21.15.18:0/3172414058 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8ed01059a0 0x7f8ed01162c0 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f8ebc009100 tx=0x7f8ebc009e30 comp rx=0 tx=0).stop 2024-04-12T21:19:04.892 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:04.897+0000 7f8ed6e00700 1 -- 172.21.15.18:0/3172414058 shutdown_connections 2024-04-12T21:19:04.892 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:04.897+0000 7f8ed6e00700 1 --2- 172.21.15.18:0/3172414058 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f8eb406ec10 0x7f8eb40710c0 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:19:04.892 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:04.897+0000 7f8ed6e00700 1 --2- 172.21.15.18:0/3172414058 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8ed0105000 0x7f8ed0115d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:19:04.892 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:04.897+0000 7f8ed6e00700 1 --2- 172.21.15.18:0/3172414058 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8ed01059a0 0x7f8ed01162c0 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:19:04.893 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:04.897+0000 7f8ed6e00700 1 -- 172.21.15.18:0/3172414058 >> 172.21.15.18:0/3172414058 conn(0x7f8ed00fabd0 msgr2=0x7f8ed00fbcc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:19:04.893 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:04.897+0000 7f8ed6e00700 1 -- 172.21.15.18:0/3172414058 shutdown_connections 2024-04-12T21:19:04.893 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:04.897+0000 7f8ed6e00700 1 -- 172.21.15.18:0/3172414058 wait complete. 2024-04-12T21:19:05.528 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:19:05 smithi186 ceph-mon[86379]: pgmap v578: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:19:05.528 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:19:05 smithi186 ceph-mon[86379]: from='client.35052 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:19:05.529 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:19:05 smithi186 ceph-mon[86379]: from='client.? 172.21.15.18:0/3172414058' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-12T21:19:05.599 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:19:05 smithi018 ceph-mon[112590]: pgmap v578: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:19:05.599 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:19:05 smithi018 ceph-mon[112590]: from='client.35052 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:19:05.600 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:19:05 smithi018 ceph-mon[112590]: from='client.? 172.21.15.18:0/3172414058' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-12T21:19:07.528 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:19:07 smithi186 ceph-mon[86379]: pgmap v579: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:19:07.599 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:19:07 smithi018 ceph-mon[112590]: pgmap v579: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:19:09.528 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:19:09 smithi186 ceph-mon[86379]: pgmap v580: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:19:09.602 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:19:09 smithi018 ceph-mon[112590]: pgmap v580: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:19:10.599 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:19:10 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T21:19:10.778 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:19:10 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T21:19:11.599 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:19:11 smithi018 ceph-mon[112590]: pgmap v581: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:19:11.778 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:19:11 smithi186 ceph-mon[86379]: pgmap v581: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:19:13.599 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:19:13 smithi018 ceph-mon[112590]: pgmap v582: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:19:13.778 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:19:13 smithi186 ceph-mon[86379]: pgmap v582: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:19:15.599 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:19:15 smithi018 ceph-mon[112590]: pgmap v583: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:19:15.778 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:19:15 smithi186 ceph-mon[86379]: pgmap v583: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:19:17.599 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:19:17 smithi018 ceph-mon[112590]: pgmap v584: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:19:17.779 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:19:17 smithi186 ceph-mon[86379]: pgmap v584: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:19:19.599 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:19:19 smithi018 ceph-mon[112590]: pgmap v585: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:19:19.778 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:19:19 smithi186 ceph-mon[86379]: pgmap v585: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:19:21.599 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:19:21 smithi018 ceph-mon[112590]: pgmap v586: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:19:21.778 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:19:21 smithi186 ceph-mon[86379]: pgmap v586: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:19:23.600 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:19:23 smithi018 ceph-mon[112590]: pgmap v587: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:19:23.778 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:19:23 smithi186 ceph-mon[86379]: pgmap v587: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:19:25.600 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:19:25 smithi018 ceph-mon[112590]: pgmap v588: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:19:25.600 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:19:25 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T21:19:25.778 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:19:25 smithi186 ceph-mon[86379]: pgmap v588: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:19:25.778 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:19:25 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T21:19:27.599 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:19:27 smithi018 ceph-mon[112590]: pgmap v589: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:19:27.778 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:19:27 smithi186 ceph-mon[86379]: pgmap v589: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:19:29.599 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:19:29 smithi018 ceph-mon[112590]: pgmap v590: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:19:29.778 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:19:29 smithi186 ceph-mon[86379]: pgmap v590: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:19:31.778 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:19:31 smithi186 ceph-mon[86379]: pgmap v591: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:19:31.849 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:19:31 smithi018 ceph-mon[112590]: pgmap v591: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:19:33.778 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:19:33 smithi186 ceph-mon[86379]: pgmap v592: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:19:33.849 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:19:33 smithi018 ceph-mon[112590]: pgmap v592: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:19:35.067 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:35.071+0000 7fbabd400700 1 -- 172.21.15.18:0/3146224355 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fbab80fde90 msgr2=0x7fbab80fe260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:19:35.068 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:35.071+0000 7fbabd400700 1 --2- 172.21.15.18:0/3146224355 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fbab80fde90 0x7fbab80fe260 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7fbaa8004940 tx=0x7fbaa8009e30 comp rx=0 tx=0).stop 2024-04-12T21:19:35.068 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:35.071+0000 7fbabd400700 1 -- 172.21.15.18:0/3146224355 shutdown_connections 2024-04-12T21:19:35.068 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:35.071+0000 7fbabd400700 1 --2- 172.21.15.18:0/3146224355 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbab80fe830 0x7fbab8106f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:19:35.068 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:35.071+0000 7fbabd400700 1 --2- 172.21.15.18:0/3146224355 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fbab80fde90 0x7fbab80fe260 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:19:35.068 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:35.071+0000 7fbabd400700 1 -- 172.21.15.18:0/3146224355 >> 172.21.15.18:0/3146224355 conn(0x7fbab8072bc0 msgr2=0x7fbab8072fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:19:35.068 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:35.071+0000 7fbabd400700 1 -- 172.21.15.18:0/3146224355 shutdown_connections 2024-04-12T21:19:35.068 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:35.071+0000 7fbabd400700 1 -- 172.21.15.18:0/3146224355 wait complete. 2024-04-12T21:19:35.069 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:35.072+0000 7fbabd400700 1 Processor -- start 2024-04-12T21:19:35.069 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:35.072+0000 7fbabd400700 1 -- start start 2024-04-12T21:19:35.069 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:35.073+0000 7fbabd400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fbab80fde90 0x7fbab8115d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:19:35.070 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:35.073+0000 7fbabd400700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbab80fe830 0x7fbab81162c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:19:35.070 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:35.073+0000 7fbabd400700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbab8116910 con 0x7fbab80fde90 2024-04-12T21:19:35.070 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:35.073+0000 7fbabd400700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbab810fd20 con 0x7fbab80fe830 2024-04-12T21:19:35.070 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:35.073+0000 7fbab6800700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fbab80fde90 0x7fbab8115d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:19:35.070 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:35.073+0000 7fbab6800700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fbab80fde90 0x7fbab8115d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:54512/0 (socket says 172.21.15.18:54512) 2024-04-12T21:19:35.070 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:35.073+0000 7fbab6800700 1 -- 172.21.15.18:0/4143112766 learned_addr learned my addr 172.21.15.18:0/4143112766 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:19:35.070 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:35.073+0000 7fbab5e00700 1 --2- 172.21.15.18:0/4143112766 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbab80fe830 0x7fbab81162c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:19:35.070 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:35.074+0000 7fbab6800700 1 -- 172.21.15.18:0/4143112766 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbab80fe830 msgr2=0x7fbab81162c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:19:35.070 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:35.074+0000 7fbab6800700 1 --2- 172.21.15.18:0/4143112766 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbab80fe830 0x7fbab81162c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:19:35.070 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:35.074+0000 7fbab6800700 1 -- 172.21.15.18:0/4143112766 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbaa8005040 con 0x7fbab80fde90 2024-04-12T21:19:35.070 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:35.074+0000 7fbab5e00700 1 --2- 172.21.15.18:0/4143112766 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbab80fe830 0x7fbab81162c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-12T21:19:35.071 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:35.074+0000 7fbab6800700 1 --2- 172.21.15.18:0/4143112766 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fbab80fde90 0x7fbab8115d80 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7fbaa80093b0 tx=0x7fbaa8009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:19:35.071 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:35.074+0000 7fbaaea00700 1 -- 172.21.15.18:0/4143112766 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbaa800e040 con 0x7fbab80fde90 2024-04-12T21:19:35.071 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:35.074+0000 7fbaaea00700 1 -- 172.21.15.18:0/4143112766 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fbaa8004290 con 0x7fbab80fde90 2024-04-12T21:19:35.071 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:35.074+0000 7fbabd400700 1 -- 172.21.15.18:0/4143112766 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbab810ffa0 con 0x7fbab80fde90 2024-04-12T21:19:35.071 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:35.074+0000 7fbaaea00700 1 -- 172.21.15.18:0/4143112766 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbaa801c3f0 con 0x7fbab80fde90 2024-04-12T21:19:35.071 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:35.075+0000 7fbabd400700 1 -- 172.21.15.18:0/4143112766 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbab81104e0 con 0x7fbab80fde90 2024-04-12T21:19:35.072 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:35.076+0000 7fbabd400700 1 -- 172.21.15.18:0/4143112766 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbab8106a20 con 0x7fbab80fde90 2024-04-12T21:19:35.073 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:35.077+0000 7fbaaea00700 1 -- 172.21.15.18:0/4143112766 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7fbaa8013890 con 0x7fbab80fde90 2024-04-12T21:19:35.074 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:35.078+0000 7fbaaea00700 1 --2- 172.21.15.18:0/4143112766 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fba9c06ec10 0x7fba9c0710c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:19:35.074 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:35.078+0000 7fbaaea00700 1 -- 172.21.15.18:0/4143112766 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 6757+0+0 (secure 0 0 0) 0x7fbaa808fcc0 con 0x7fbab80fde90 2024-04-12T21:19:35.074 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:35.078+0000 7fbab5e00700 1 --2- 172.21.15.18:0/4143112766 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fba9c06ec10 0x7fba9c0710c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:19:35.075 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:35.079+0000 7fbab5e00700 1 --2- 172.21.15.18:0/4143112766 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fba9c06ec10 0x7fba9c0710c0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fbaa4009fd0 tx=0x7fbaa400b040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:19:35.079 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:35.083+0000 7fbaaea00700 1 -- 172.21.15.18:0/4143112766 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7fbaa8095020 con 0x7fbab80fde90 2024-04-12T21:19:35.329 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:35.333+0000 7fbabd400700 1 -- 172.21.15.18:0/4143112766 --> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fbab8109f50 con 0x7fba9c06ec10 2024-04-12T21:19:35.331 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:35.335+0000 7fbaaea00700 1 -- 172.21.15.18:0/4143112766 <== mgr.34102 v2:172.21.15.18:6800/3812891919 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7fbab8109f50 con 0x7fba9c06ec10 2024-04-12T21:19:35.334 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:35.338+0000 7fbabd400700 1 -- 172.21.15.18:0/4143112766 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fba9c06ec10 msgr2=0x7fba9c0710c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:19:35.334 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:35.338+0000 7fbabd400700 1 --2- 172.21.15.18:0/4143112766 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fba9c06ec10 0x7fba9c0710c0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fbaa4009fd0 tx=0x7fbaa400b040 comp rx=0 tx=0).stop 2024-04-12T21:19:35.334 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:35.338+0000 7fbabd400700 1 -- 172.21.15.18:0/4143112766 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fbab80fde90 msgr2=0x7fbab8115d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:19:35.335 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:35.338+0000 7fbabd400700 1 --2- 172.21.15.18:0/4143112766 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fbab80fde90 0x7fbab8115d80 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7fbaa80093b0 tx=0x7fbaa8009e30 comp rx=0 tx=0).stop 2024-04-12T21:19:35.335 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:35.338+0000 7fbabd400700 1 -- 172.21.15.18:0/4143112766 shutdown_connections 2024-04-12T21:19:35.335 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:35.338+0000 7fbabd400700 1 --2- 172.21.15.18:0/4143112766 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fba9c06ec10 0x7fba9c0710c0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:19:35.335 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:35.338+0000 7fbabd400700 1 --2- 172.21.15.18:0/4143112766 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fbab80fde90 0x7fbab8115d80 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:19:35.335 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:35.338+0000 7fbabd400700 1 --2- 172.21.15.18:0/4143112766 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbab80fe830 0x7fbab81162c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:19:35.335 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:35.338+0000 7fbabd400700 1 -- 172.21.15.18:0/4143112766 >> 172.21.15.18:0/4143112766 conn(0x7fbab8072bc0 msgr2=0x7fbab80fb840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:19:35.335 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:35.339+0000 7fbabd400700 1 -- 172.21.15.18:0/4143112766 shutdown_connections 2024-04-12T21:19:35.335 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:35.339+0000 7fbabd400700 1 -- 172.21.15.18:0/4143112766 wait complete. 2024-04-12T21:19:35.350 INFO:teuthology.orchestra.run.smithi018.stdout:true 2024-04-12T21:19:35.482 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:35.485+0000 7f791b600700 1 -- 172.21.15.18:0/4139169299 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7914102e00 msgr2=0x7f79141031d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:19:35.482 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:35.485+0000 7f791b600700 1 --2- 172.21.15.18:0/4139169299 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7914102e00 0x7f79141031d0 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f7904004940 tx=0x7f7904009e30 comp rx=0 tx=0).stop 2024-04-12T21:19:35.482 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:35.486+0000 7f791b600700 1 -- 172.21.15.18:0/4139169299 shutdown_connections 2024-04-12T21:19:35.482 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:35.486+0000 7f791b600700 1 --2- 172.21.15.18:0/4139169299 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f79141037a0 0x7f79140fce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:19:35.482 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:35.486+0000 7f791b600700 1 --2- 172.21.15.18:0/4139169299 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7914102e00 0x7f79141031d0 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:19:35.482 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:35.486+0000 7f791b600700 1 -- 172.21.15.18:0/4139169299 >> 172.21.15.18:0/4139169299 conn(0x7f79140f89b0 msgr2=0x7f79140fadc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:19:35.482 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:35.486+0000 7f791b600700 1 -- 172.21.15.18:0/4139169299 shutdown_connections 2024-04-12T21:19:35.482 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:35.486+0000 7f791b600700 1 -- 172.21.15.18:0/4139169299 wait complete. 2024-04-12T21:19:35.484 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:35.486+0000 7f791b600700 1 Processor -- start 2024-04-12T21:19:35.484 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:35.487+0000 7f791b600700 1 -- start start 2024-04-12T21:19:35.484 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:35.487+0000 7f791b600700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7914102e00 0x7f7914113ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:19:35.484 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:35.487+0000 7f791b600700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f79141037a0 0x7f79141140e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:19:35.484 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:35.487+0000 7f791b600700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7914114730 con 0x7f7914102e00 2024-04-12T21:19:35.484 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:35.487+0000 7f791b600700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f791410db70 con 0x7f79141037a0 2024-04-12T21:19:35.484 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:35.487+0000 7f7918c00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7914102e00 0x7f7914113ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:19:35.484 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:35.487+0000 7f7918c00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7914102e00 0x7f7914113ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:54528/0 (socket says 172.21.15.18:54528) 2024-04-12T21:19:35.485 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:35.487+0000 7f7918c00700 1 -- 172.21.15.18:0/3701018336 learned_addr learned my addr 172.21.15.18:0/3701018336 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:19:35.485 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:35.488+0000 7f7918c00700 1 -- 172.21.15.18:0/3701018336 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f79141037a0 msgr2=0x7f79141140e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:19:35.485 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:35.488+0000 7f7918c00700 1 --2- 172.21.15.18:0/3701018336 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f79141037a0 0x7f79141140e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:19:35.485 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:35.488+0000 7f7918c00700 1 -- 172.21.15.18:0/3701018336 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7904005040 con 0x7f7914102e00 2024-04-12T21:19:35.485 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:35.488+0000 7f7918c00700 1 --2- 172.21.15.18:0/3701018336 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7914102e00 0x7f7914113ba0 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f7904009150 tx=0x7f7904009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:19:35.485 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:35.488+0000 7f7911600700 1 -- 172.21.15.18:0/3701018336 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f790400e040 con 0x7f7914102e00 2024-04-12T21:19:35.485 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:35.488+0000 7f7911600700 1 -- 172.21.15.18:0/3701018336 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f7904013420 con 0x7f7914102e00 2024-04-12T21:19:35.485 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:35.488+0000 7f7911600700 1 -- 172.21.15.18:0/3701018336 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f790401c630 con 0x7f7914102e00 2024-04-12T21:19:35.485 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:35.488+0000 7f791b600700 1 -- 172.21.15.18:0/3701018336 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f791410ddf0 con 0x7f7914102e00 2024-04-12T21:19:35.486 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:35.489+0000 7f791b600700 1 -- 172.21.15.18:0/3701018336 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f791410e3b0 con 0x7f7914102e00 2024-04-12T21:19:35.486 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:35.490+0000 7f791b600700 1 -- 172.21.15.18:0/3701018336 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7914100110 con 0x7f7914102e00 2024-04-12T21:19:35.487 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:35.491+0000 7f7911600700 1 -- 172.21.15.18:0/3701018336 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7f7904013590 con 0x7f7914102e00 2024-04-12T21:19:35.488 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:35.492+0000 7f7911600700 1 --2- 172.21.15.18:0/3701018336 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f78f806ec10 0x7f78f80710c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:19:35.488 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:35.492+0000 7f7911600700 1 -- 172.21.15.18:0/3701018336 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 6757+0+0 (secure 0 0 0) 0x7f790408fb50 con 0x7f7914102e00 2024-04-12T21:19:35.488 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:35.492+0000 7f790be00700 1 --2- 172.21.15.18:0/3701018336 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f78f806ec10 0x7f78f80710c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:19:35.488 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:35.493+0000 7f790be00700 1 --2- 172.21.15.18:0/3701018336 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f78f806ec10 0x7f78f80710c0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f791410ece0 tx=0x7f78fc009ce0 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:19:35.492 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:35.496+0000 7f7911600700 1 -- 172.21.15.18:0/3701018336 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f790405c440 con 0x7f7914102e00 2024-04-12T21:19:35.599 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:19:35 smithi018 ceph-mon[112590]: pgmap v593: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:19:35.732 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:35.736+0000 7f791b600700 1 -- 172.21.15.18:0/3701018336 --> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f7914109f00 con 0x7f78f806ec10 2024-04-12T21:19:35.734 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:35.738+0000 7f7911600700 1 -- 172.21.15.18:0/3701018336 <== mgr.34102 v2:172.21.15.18:6800/3812891919 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f7914109f00 con 0x7f78f806ec10 2024-04-12T21:19:35.737 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:35.741+0000 7f791b600700 1 -- 172.21.15.18:0/3701018336 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f78f806ec10 msgr2=0x7f78f80710c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:19:35.737 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:35.741+0000 7f791b600700 1 --2- 172.21.15.18:0/3701018336 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f78f806ec10 0x7f78f80710c0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f791410ece0 tx=0x7f78fc009ce0 comp rx=0 tx=0).stop 2024-04-12T21:19:35.737 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:35.741+0000 7f791b600700 1 -- 172.21.15.18:0/3701018336 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7914102e00 msgr2=0x7f7914113ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:19:35.737 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:35.741+0000 7f791b600700 1 --2- 172.21.15.18:0/3701018336 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7914102e00 0x7f7914113ba0 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f7904009150 tx=0x7f7904009e30 comp rx=0 tx=0).stop 2024-04-12T21:19:35.737 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:35.741+0000 7f791b600700 1 -- 172.21.15.18:0/3701018336 shutdown_connections 2024-04-12T21:19:35.737 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:35.741+0000 7f791b600700 1 --2- 172.21.15.18:0/3701018336 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f78f806ec10 0x7f78f80710c0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:19:35.738 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:35.741+0000 7f791b600700 1 --2- 172.21.15.18:0/3701018336 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7914102e00 0x7f7914113ba0 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:19:35.738 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:35.741+0000 7f791b600700 1 --2- 172.21.15.18:0/3701018336 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f79141037a0 0x7f79141140e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:19:35.738 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:35.741+0000 7f791b600700 1 -- 172.21.15.18:0/3701018336 >> 172.21.15.18:0/3701018336 conn(0x7f79140f89b0 msgr2=0x7f79141073a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:19:35.738 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:35.741+0000 7f791b600700 1 -- 172.21.15.18:0/3701018336 shutdown_connections 2024-04-12T21:19:35.738 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:35.741+0000 7f791b600700 1 -- 172.21.15.18:0/3701018336 wait complete. 2024-04-12T21:19:35.778 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:19:35 smithi186 ceph-mon[86379]: pgmap v593: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:19:35.893 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:35.897+0000 7f1274000700 1 -- 172.21.15.18:0/1440109279 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f126c1059c0 msgr2=0x7f126c0ff080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:19:35.894 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:35.897+0000 7f1274000700 1 --2- 172.21.15.18:0/1440109279 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f126c1059c0 0x7f126c0ff080 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f1258004940 tx=0x7f1258009e30 comp rx=0 tx=0).stop 2024-04-12T21:19:35.894 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:35.897+0000 7f1274000700 1 -- 172.21.15.18:0/1440109279 shutdown_connections 2024-04-12T21:19:35.894 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:35.897+0000 7f1274000700 1 --2- 172.21.15.18:0/1440109279 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f126c1059c0 0x7f126c0ff080 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:19:35.894 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:35.897+0000 7f1274000700 1 --2- 172.21.15.18:0/1440109279 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f126c105020 0x7f126c1053f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:19:35.894 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:35.897+0000 7f1274000700 1 -- 172.21.15.18:0/1440109279 >> 172.21.15.18:0/1440109279 conn(0x7f126c0faba0 msgr2=0x7f126c0fcfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:19:35.894 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:35.898+0000 7f1274000700 1 -- 172.21.15.18:0/1440109279 shutdown_connections 2024-04-12T21:19:35.894 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:35.898+0000 7f1274000700 1 -- 172.21.15.18:0/1440109279 wait complete. 2024-04-12T21:19:35.895 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:35.898+0000 7f1274000700 1 Processor -- start 2024-04-12T21:19:35.896 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:35.899+0000 7f1274000700 1 -- start start 2024-04-12T21:19:35.896 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:35.899+0000 7f1274000700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f126c105020 0x7f126c115ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:19:35.896 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:35.899+0000 7f1274000700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f126c1059c0 0x7f126c1160e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:19:35.896 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:35.899+0000 7f1274000700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f126c116730 con 0x7f126c1059c0 2024-04-12T21:19:35.896 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:35.899+0000 7f1274000700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f126c10fd10 con 0x7f126c105020 2024-04-12T21:19:35.896 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:35.899+0000 7f1270c00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f126c1059c0 0x7f126c1160e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:19:35.896 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:35.899+0000 7f1270c00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f126c1059c0 0x7f126c1160e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:54538/0 (socket says 172.21.15.18:54538) 2024-04-12T21:19:35.896 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:35.899+0000 7f1270c00700 1 -- 172.21.15.18:0/426917126 learned_addr learned my addr 172.21.15.18:0/426917126 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:19:35.896 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:35.900+0000 7f1271600700 1 --2- 172.21.15.18:0/426917126 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f126c105020 0x7f126c115ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:19:35.896 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:35.900+0000 7f1270c00700 1 -- 172.21.15.18:0/426917126 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f126c105020 msgr2=0x7f126c115ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:19:35.896 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:35.900+0000 7f1270c00700 1 --2- 172.21.15.18:0/426917126 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f126c105020 0x7f126c115ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:19:35.896 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:35.900+0000 7f1270c00700 1 -- 172.21.15.18:0/426917126 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1258005040 con 0x7f126c1059c0 2024-04-12T21:19:35.896 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:35.900+0000 7f1271600700 1 --2- 172.21.15.18:0/426917126 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f126c105020 0x7f126c115ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-12T21:19:35.897 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:35.900+0000 7f1270c00700 1 --2- 172.21.15.18:0/426917126 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f126c1059c0 0x7f126c1160e0 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f1258009180 tx=0x7f1258009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:19:35.897 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:35.901+0000 7f1269600700 1 -- 172.21.15.18:0/426917126 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f125800e040 con 0x7f126c1059c0 2024-04-12T21:19:35.897 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:35.901+0000 7f1269600700 1 -- 172.21.15.18:0/426917126 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f125800bbc0 con 0x7f126c1059c0 2024-04-12T21:19:35.898 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:35.901+0000 7f1269600700 1 -- 172.21.15.18:0/426917126 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f125801c3f0 con 0x7f126c1059c0 2024-04-12T21:19:35.898 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:35.901+0000 7f1274000700 1 -- 172.21.15.18:0/426917126 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f126c10ff90 con 0x7f126c1059c0 2024-04-12T21:19:35.898 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:35.901+0000 7f1274000700 1 -- 172.21.15.18:0/426917126 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f126c110550 con 0x7f126c1059c0 2024-04-12T21:19:35.898 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:35.902+0000 7f1274000700 1 -- 172.21.15.18:0/426917126 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f126c100770 con 0x7f126c1059c0 2024-04-12T21:19:35.899 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:35.903+0000 7f1269600700 1 -- 172.21.15.18:0/426917126 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7f125800bd30 con 0x7f126c1059c0 2024-04-12T21:19:35.900 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:35.904+0000 7f1269600700 1 --2- 172.21.15.18:0/426917126 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f125406ec00 0x7f12540710b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:19:35.900 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:35.904+0000 7f1269600700 1 -- 172.21.15.18:0/426917126 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 6757+0+0 (secure 0 0 0) 0x7f125808fdd0 con 0x7f126c1059c0 2024-04-12T21:19:35.900 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:35.904+0000 7f1271600700 1 --2- 172.21.15.18:0/426917126 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f125406ec00 0x7f12540710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:19:35.901 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:35.905+0000 7f1271600700 1 --2- 172.21.15.18:0/426917126 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f125406ec00 0x7f12540710b0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f1260009fd0 tx=0x7f126000b040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:19:35.905 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:35.909+0000 7f1269600700 1 -- 172.21.15.18:0/426917126 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f125805c580 con 0x7f126c1059c0 2024-04-12T21:19:36.149 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:36.153+0000 7f1274000700 1 -- 172.21.15.18:0/426917126 --> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f126c10c170 con 0x7f125406ec00 2024-04-12T21:19:36.180 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:36.184+0000 7f1269600700 1 -- 172.21.15.18:0/426917126 <== mgr.34102 v2:172.21.15.18:6800/3812891919 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7f126c10c170 con 0x7f125406ec00 2024-04-12T21:19:36.180 INFO:teuthology.orchestra.run.smithi018.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-12T21:19:36.180 INFO:teuthology.orchestra.run.smithi018.stdout:alertmanager.smithi018 smithi018 *:9093,9094 running (20m) 11m ago 34m 14.7M - 0.25.0 c8568f914cd2 c332090068e8 2024-04-12T21:19:36.180 INFO:teuthology.orchestra.run.smithi018.stdout:crash.smithi018 smithi018 running (16m) 11m ago 34m 6665k - 19.0.0-2879-g324baf8d 8717720dc5e5 b7f80d1f7982 2024-04-12T21:19:36.181 INFO:teuthology.orchestra.run.smithi018.stdout:crash.smithi186 smithi186 running (16m) 12m ago 33m 6660k - 19.0.0-2879-g324baf8d 8717720dc5e5 a4f55c81b623 2024-04-12T21:19:36.181 INFO:teuthology.orchestra.run.smithi018.stdout:grafana.smithi018 smithi018 *:3000 running (19m) 11m ago 34m 74.8M - 9.4.12 2bacad6d85d8 b08e3bf176dd 2024-04-12T21:19:36.181 INFO:teuthology.orchestra.run.smithi018.stdout:mds.cephfs.smithi018.acqbsc smithi018 running (12m) 11m ago 27m 11.1M - 19.0.0-2879-g324baf8d 8717720dc5e5 021f99ab24ae 2024-04-12T21:19:36.181 INFO:teuthology.orchestra.run.smithi018.stdout:mds.cephfs.smithi018.szjluo smithi018 running (11m) 11m ago 27m 10.9M - 19.0.0-2879-g324baf8d 8717720dc5e5 0003117479d4 2024-04-12T21:19:36.181 INFO:teuthology.orchestra.run.smithi018.stdout:mds.cephfs.smithi186.jdllyn smithi186 running (27m) 12m ago 27m 15.6M - 17.2.7-827-gd3b5060a 21de9b7369ad 2b2bc46bd29a 2024-04-12T21:19:36.181 INFO:teuthology.orchestra.run.smithi018.stdout:mds.cephfs.smithi186.wjnafs smithi186 running (27m) 12m ago 27m 15.4M - 17.2.7-827-gd3b5060a 21de9b7369ad 74b115d6d6fd 2024-04-12T21:19:36.181 INFO:teuthology.orchestra.run.smithi018.stdout:mgr.smithi018.xjzwga smithi018 *:8443,9283,8765 running (22m) 11m ago 36m 509M - 19.0.0-2879-g324baf8d 8717720dc5e5 56088305eab7 2024-04-12T21:19:36.181 INFO:teuthology.orchestra.run.smithi018.stdout:mgr.smithi186.kqcfxk smithi186 *:8443,9283,8765 running (21m) 12m ago 33m 452M - 19.0.0-2879-g324baf8d 8717720dc5e5 09cd766e5e0d 2024-04-12T21:19:36.181 INFO:teuthology.orchestra.run.smithi018.stdout:mon.smithi018 smithi018 running (18m) 11m ago 36m 57.0M 2048M 19.0.0-2879-g324baf8d 8717720dc5e5 df3e201dfed6 2024-04-12T21:19:36.181 INFO:teuthology.orchestra.run.smithi018.stdout:mon.smithi186 smithi186 running (17m) 12m ago 33m 37.7M 2048M 19.0.0-2879-g324baf8d 8717720dc5e5 c77498da9b2b 2024-04-12T21:19:36.181 INFO:teuthology.orchestra.run.smithi018.stdout:node-exporter.smithi018 smithi018 *:9100 running (21m) 11m ago 34m 7780k - 1.5.0 0da6a335fe13 326c3d9c6847 2024-04-12T21:19:36.181 INFO:teuthology.orchestra.run.smithi018.stdout:node-exporter.smithi186 smithi186 *:9100 running (21m) 12m ago 33m 7826k - 1.5.0 0da6a335fe13 ec2112baeecf 2024-04-12T21:19:36.181 INFO:teuthology.orchestra.run.smithi018.stdout:osd.0 smithi018 running (15m) 11m ago 32m 121M 1559M 19.0.0-2879-g324baf8d 8717720dc5e5 60b1273ca047 2024-04-12T21:19:36.181 INFO:teuthology.orchestra.run.smithi018.stdout:osd.1 smithi018 running (15m) 11m ago 31m 91.3M 1559M 19.0.0-2879-g324baf8d 8717720dc5e5 170ce1e5fb8d 2024-04-12T21:19:36.181 INFO:teuthology.orchestra.run.smithi018.stdout:osd.2 smithi018 running (14m) 11m ago 31m 90.3M 1559M 19.0.0-2879-g324baf8d 8717720dc5e5 c2cbc6aa2839 2024-04-12T21:19:36.182 INFO:teuthology.orchestra.run.smithi018.stdout:osd.3 smithi186 running (14m) 12m ago 30m 96.6M 2581M 19.0.0-2879-g324baf8d 8717720dc5e5 0138e8262e4b 2024-04-12T21:19:36.182 INFO:teuthology.orchestra.run.smithi018.stdout:osd.4 smithi186 running (13m) 12m ago 30m 87.2M 2581M 19.0.0-2879-g324baf8d 8717720dc5e5 b20cb63b25c9 2024-04-12T21:19:36.182 INFO:teuthology.orchestra.run.smithi018.stdout:osd.5 smithi186 running (12m) 12m ago 29m 11.1M 2581M 19.0.0-2879-g324baf8d 8717720dc5e5 2513bea2610e 2024-04-12T21:19:36.182 INFO:teuthology.orchestra.run.smithi018.stdout:prometheus.smithi018 smithi018 *:9095 running (20m) 11m ago 34m 60.6M - 2.43.0 a07b618ecd1d f58841127e21 2024-04-12T21:19:36.182 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:36.186+0000 7f1274000700 1 -- 172.21.15.18:0/426917126 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f125406ec00 msgr2=0x7f12540710b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:19:36.182 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:36.186+0000 7f1274000700 1 --2- 172.21.15.18:0/426917126 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f125406ec00 0x7f12540710b0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f1260009fd0 tx=0x7f126000b040 comp rx=0 tx=0).stop 2024-04-12T21:19:36.183 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:36.187+0000 7f1274000700 1 -- 172.21.15.18:0/426917126 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f126c1059c0 msgr2=0x7f126c1160e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:19:36.183 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:36.187+0000 7f1274000700 1 --2- 172.21.15.18:0/426917126 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f126c1059c0 0x7f126c1160e0 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f1258009180 tx=0x7f1258009e30 comp rx=0 tx=0).stop 2024-04-12T21:19:36.183 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:36.187+0000 7f1274000700 1 -- 172.21.15.18:0/426917126 shutdown_connections 2024-04-12T21:19:36.183 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:36.187+0000 7f1274000700 1 --2- 172.21.15.18:0/426917126 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f125406ec00 0x7f12540710b0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:19:36.183 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:36.187+0000 7f1274000700 1 --2- 172.21.15.18:0/426917126 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f126c105020 0x7f126c115ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:19:36.183 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:36.187+0000 7f1274000700 1 --2- 172.21.15.18:0/426917126 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f126c1059c0 0x7f126c1160e0 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:19:36.183 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:36.187+0000 7f1274000700 1 -- 172.21.15.18:0/426917126 >> 172.21.15.18:0/426917126 conn(0x7f126c0faba0 msgr2=0x7f126c0fbb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:19:36.184 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:36.188+0000 7f1274000700 1 -- 172.21.15.18:0/426917126 shutdown_connections 2024-04-12T21:19:36.184 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:36.188+0000 7f1274000700 1 -- 172.21.15.18:0/426917126 wait complete. 2024-04-12T21:19:36.341 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:36.344+0000 7f9e03e00700 1 -- 172.21.15.18:0/67775698 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9e040fc2b0 msgr2=0x7f9e04109010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:19:36.341 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:36.344+0000 7f9e03e00700 1 --2- 172.21.15.18:0/67775698 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9e040fc2b0 0x7f9e04109010 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f9df0004940 tx=0x7f9df0009e30 comp rx=0 tx=0).stop 2024-04-12T21:19:36.341 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:36.344+0000 7f9e03e00700 1 -- 172.21.15.18:0/67775698 shutdown_connections 2024-04-12T21:19:36.341 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:36.344+0000 7f9e03e00700 1 --2- 172.21.15.18:0/67775698 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9e040fc2b0 0x7f9e04109010 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:19:36.341 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:36.344+0000 7f9e03e00700 1 --2- 172.21.15.18:0/67775698 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9e040fb9a0 0x7f9e040fbd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:19:36.342 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:36.344+0000 7f9e03e00700 1 -- 172.21.15.18:0/67775698 >> 172.21.15.18:0/67775698 conn(0x7f9e040f7670 msgr2=0x7f9e040f9a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:19:36.342 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:36.345+0000 7f9e03e00700 1 -- 172.21.15.18:0/67775698 shutdown_connections 2024-04-12T21:19:36.342 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:36.345+0000 7f9e03e00700 1 -- 172.21.15.18:0/67775698 wait complete. 2024-04-12T21:19:36.342 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:36.345+0000 7f9e03e00700 1 Processor -- start 2024-04-12T21:19:36.343 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:36.346+0000 7f9e03e00700 1 -- start start 2024-04-12T21:19:36.343 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:36.346+0000 7f9e03e00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9e040fb9a0 0x7f9e04115b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:19:36.343 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:36.346+0000 7f9e03e00700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9e040fc2b0 0x7f9e041160a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:19:36.343 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:36.346+0000 7f9e03e00700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e041166f0 con 0x7f9e040fb9a0 2024-04-12T21:19:36.344 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:36.346+0000 7f9e03e00700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e0410fae0 con 0x7f9e040fc2b0 2024-04-12T21:19:36.344 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:36.346+0000 7f9e02a00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9e040fb9a0 0x7f9e04115b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:19:36.344 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:36.346+0000 7f9e02a00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9e040fb9a0 0x7f9e04115b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:54546/0 (socket says 172.21.15.18:54546) 2024-04-12T21:19:36.344 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:36.346+0000 7f9e02a00700 1 -- 172.21.15.18:0/1172115389 learned_addr learned my addr 172.21.15.18:0/1172115389 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:19:36.344 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:36.347+0000 7f9e02000700 1 --2- 172.21.15.18:0/1172115389 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9e040fc2b0 0x7f9e041160a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:19:36.344 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:36.347+0000 7f9e02a00700 1 -- 172.21.15.18:0/1172115389 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9e040fc2b0 msgr2=0x7f9e041160a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:19:36.344 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:36.347+0000 7f9e02a00700 1 --2- 172.21.15.18:0/1172115389 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9e040fc2b0 0x7f9e041160a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:19:36.344 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:36.347+0000 7f9e02a00700 1 -- 172.21.15.18:0/1172115389 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9df0005040 con 0x7f9e040fb9a0 2024-04-12T21:19:36.344 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:36.347+0000 7f9e02a00700 1 --2- 172.21.15.18:0/1172115389 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9e040fb9a0 0x7f9e04115b60 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f9df4002a50 tx=0x7f9df40046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:19:36.344 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:36.347+0000 7f9dfaa00700 1 -- 172.21.15.18:0/1172115389 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9df4007cc0 con 0x7f9e040fb9a0 2024-04-12T21:19:36.344 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:36.347+0000 7f9dfaa00700 1 -- 172.21.15.18:0/1172115389 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f9df400f040 con 0x7f9e040fb9a0 2024-04-12T21:19:36.344 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:36.347+0000 7f9e03e00700 1 -- 172.21.15.18:0/1172115389 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9e0410fdc0 con 0x7f9e040fb9a0 2024-04-12T21:19:36.344 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:36.348+0000 7f9dfaa00700 1 -- 172.21.15.18:0/1172115389 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9df4009640 con 0x7f9e040fb9a0 2024-04-12T21:19:36.345 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:36.348+0000 7f9e03e00700 1 -- 172.21.15.18:0/1172115389 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9e04110380 con 0x7f9e040fb9a0 2024-04-12T21:19:36.345 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:36.349+0000 7f9e03e00700 1 -- 172.21.15.18:0/1172115389 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9e040668b0 con 0x7f9e040fb9a0 2024-04-12T21:19:36.345 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:36.350+0000 7f9dfaa00700 1 -- 172.21.15.18:0/1172115389 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7f9df40097a0 con 0x7f9e040fb9a0 2024-04-12T21:19:36.347 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:36.351+0000 7f9dfaa00700 1 --2- 172.21.15.18:0/1172115389 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f9de806ec00 0x7f9de80710b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:19:36.347 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:36.351+0000 7f9dfaa00700 1 -- 172.21.15.18:0/1172115389 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 6757+0+0 (secure 0 0 0) 0x7f9df408e790 con 0x7f9e040fb9a0 2024-04-12T21:19:36.347 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:36.351+0000 7f9e02000700 1 --2- 172.21.15.18:0/1172115389 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f9de806ec00 0x7f9de80710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:19:36.347 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:36.352+0000 7f9e02000700 1 --2- 172.21.15.18:0/1172115389 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f9de806ec00 0x7f9de80710b0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f9e04110ca0 tx=0x7f9df0009e30 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:19:36.351 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:36.356+0000 7f9dfaa00700 1 -- 172.21.15.18:0/1172115389 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f9df405af60 con 0x7f9e040fb9a0 2024-04-12T21:19:36.599 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:19:36 smithi018 ceph-mon[112590]: from='client.35060 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:19:36.670 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:36.673+0000 7f9e03e00700 1 -- 172.21.15.18:0/1172115389 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f9e0410bed0 con 0x7f9e040fb9a0 2024-04-12T21:19:36.670 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:36.674+0000 7f9dfaa00700 1 -- 172.21.15.18:0/1172115389 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+837 (secure 0 0 0) 0x7f9df405aaf0 con 0x7f9e040fb9a0 2024-04-12T21:19:36.671 INFO:teuthology.orchestra.run.smithi018.stdout:{ 2024-04-12T21:19:36.671 INFO:teuthology.orchestra.run.smithi018.stdout: "mon": { 2024-04-12T21:19:36.671 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 19.0.0-2879-g324baf8d (324baf8ddb91dea46cac50cb535c306115791355) squid (dev)": 2 2024-04-12T21:19:36.671 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-12T21:19:36.671 INFO:teuthology.orchestra.run.smithi018.stdout: "mgr": { 2024-04-12T21:19:36.671 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 19.0.0-2879-g324baf8d (324baf8ddb91dea46cac50cb535c306115791355) squid (dev)": 2 2024-04-12T21:19:36.671 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-12T21:19:36.672 INFO:teuthology.orchestra.run.smithi018.stdout: "osd": { 2024-04-12T21:19:36.672 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 19.0.0-2879-g324baf8d (324baf8ddb91dea46cac50cb535c306115791355) squid (dev)": 6 2024-04-12T21:19:36.672 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-12T21:19:36.672 INFO:teuthology.orchestra.run.smithi018.stdout: "mds": { 2024-04-12T21:19:36.672 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 17.2.7-827-gd3b5060a (d3b5060a17449edfae9385cdf792afec109fca5d) quincy (stable)": 2, 2024-04-12T21:19:36.672 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 19.0.0-2879-g324baf8d (324baf8ddb91dea46cac50cb535c306115791355) squid (dev)": 2 2024-04-12T21:19:36.672 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-12T21:19:36.672 INFO:teuthology.orchestra.run.smithi018.stdout: "overall": { 2024-04-12T21:19:36.672 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 17.2.7-827-gd3b5060a (d3b5060a17449edfae9385cdf792afec109fca5d) quincy (stable)": 2, 2024-04-12T21:19:36.672 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 19.0.0-2879-g324baf8d (324baf8ddb91dea46cac50cb535c306115791355) squid (dev)": 12 2024-04-12T21:19:36.672 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-04-12T21:19:36.672 INFO:teuthology.orchestra.run.smithi018.stdout:} 2024-04-12T21:19:36.673 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:36.677+0000 7f9e03e00700 1 -- 172.21.15.18:0/1172115389 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f9de806ec00 msgr2=0x7f9de80710b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:19:36.673 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:36.677+0000 7f9e03e00700 1 --2- 172.21.15.18:0/1172115389 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f9de806ec00 0x7f9de80710b0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f9e04110ca0 tx=0x7f9df0009e30 comp rx=0 tx=0).stop 2024-04-12T21:19:36.673 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:36.677+0000 7f9e03e00700 1 -- 172.21.15.18:0/1172115389 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9e040fb9a0 msgr2=0x7f9e04115b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:19:36.673 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:36.677+0000 7f9e03e00700 1 --2- 172.21.15.18:0/1172115389 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9e040fb9a0 0x7f9e04115b60 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f9df4002a50 tx=0x7f9df40046b0 comp rx=0 tx=0).stop 2024-04-12T21:19:36.673 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:36.677+0000 7f9e03e00700 1 -- 172.21.15.18:0/1172115389 shutdown_connections 2024-04-12T21:19:36.674 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:36.677+0000 7f9e03e00700 1 --2- 172.21.15.18:0/1172115389 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f9de806ec00 0x7f9de80710b0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:19:36.674 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:36.677+0000 7f9e03e00700 1 --2- 172.21.15.18:0/1172115389 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9e040fb9a0 0x7f9e04115b60 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:19:36.674 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:36.677+0000 7f9e03e00700 1 --2- 172.21.15.18:0/1172115389 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9e040fc2b0 0x7f9e041160a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:19:36.674 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:36.677+0000 7f9e03e00700 1 -- 172.21.15.18:0/1172115389 >> 172.21.15.18:0/1172115389 conn(0x7f9e040f7670 msgr2=0x7f9e040f7a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:19:36.674 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:36.677+0000 7f9e03e00700 1 -- 172.21.15.18:0/1172115389 shutdown_connections 2024-04-12T21:19:36.674 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:36.678+0000 7f9e03e00700 1 -- 172.21.15.18:0/1172115389 wait complete. 2024-04-12T21:19:36.778 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:19:36 smithi186 ceph-mon[86379]: from='client.35060 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:19:36.831 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:36.834+0000 7fcffb600700 1 -- 172.21.15.18:0/909556829 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fcff4109430 msgr2=0x7fcff4109800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:19:36.831 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:36.834+0000 7fcffb600700 1 --2- 172.21.15.18:0/909556829 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fcff4109430 0x7fcff4109800 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7fcfe8004940 tx=0x7fcfe8009e30 comp rx=0 tx=0).stop 2024-04-12T21:19:36.831 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:36.835+0000 7fcffb600700 1 -- 172.21.15.18:0/909556829 shutdown_connections 2024-04-12T21:19:36.831 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:36.835+0000 7fcffb600700 1 --2- 172.21.15.18:0/909556829 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcff4109dd0 0x7fcff4103440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:19:36.831 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:36.835+0000 7fcffb600700 1 --2- 172.21.15.18:0/909556829 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fcff4109430 0x7fcff4109800 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:19:36.831 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:36.835+0000 7fcffb600700 1 -- 172.21.15.18:0/909556829 >> 172.21.15.18:0/909556829 conn(0x7fcff4072c00 msgr2=0x7fcff4075010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:19:36.831 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:36.835+0000 7fcffb600700 1 -- 172.21.15.18:0/909556829 shutdown_connections 2024-04-12T21:19:36.832 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:36.835+0000 7fcffb600700 1 -- 172.21.15.18:0/909556829 wait complete. 2024-04-12T21:19:36.833 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:36.836+0000 7fcffb600700 1 Processor -- start 2024-04-12T21:19:36.833 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:36.836+0000 7fcffb600700 1 -- start start 2024-04-12T21:19:36.833 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:36.836+0000 7fcffb600700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcff4109430 0x7fcff4119f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:19:36.833 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:36.836+0000 7fcffb600700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fcff4109dd0 0x7fcff411a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:19:36.833 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:36.836+0000 7fcffb600700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcff411aad0 con 0x7fcff4109dd0 2024-04-12T21:19:36.833 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:36.836+0000 7fcffb600700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcff4114100 con 0x7fcff4109430 2024-04-12T21:19:36.833 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:36.836+0000 7fcff3e00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fcff4109dd0 0x7fcff411a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:19:36.833 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:36.836+0000 7fcff3e00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fcff4109dd0 0x7fcff411a480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:54554/0 (socket says 172.21.15.18:54554) 2024-04-12T21:19:36.833 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:36.836+0000 7fcff3e00700 1 -- 172.21.15.18:0/590954270 learned_addr learned my addr 172.21.15.18:0/590954270 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:19:36.833 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:36.837+0000 7fcff3e00700 1 -- 172.21.15.18:0/590954270 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcff4109430 msgr2=0x7fcff4119f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:19:36.834 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:36.837+0000 7fcff3e00700 1 --2- 172.21.15.18:0/590954270 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcff4109430 0x7fcff4119f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:19:36.834 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:36.837+0000 7fcff3e00700 1 -- 172.21.15.18:0/590954270 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcfe8005040 con 0x7fcff4109dd0 2024-04-12T21:19:36.834 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:36.837+0000 7fcff3e00700 1 --2- 172.21.15.18:0/590954270 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fcff4109dd0 0x7fcff411a480 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7fcfe000c800 tx=0x7fcfe0004470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:19:36.834 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:36.837+0000 7fcff0c00700 1 -- 172.21.15.18:0/590954270 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcfe000a760 con 0x7fcff4109dd0 2024-04-12T21:19:36.835 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:36.837+0000 7fcff0c00700 1 -- 172.21.15.18:0/590954270 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fcfe000ad90 con 0x7fcff4109dd0 2024-04-12T21:19:36.835 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:36.837+0000 7fcffb600700 1 -- 172.21.15.18:0/590954270 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcff41143e0 con 0x7fcff4109dd0 2024-04-12T21:19:36.835 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:36.837+0000 7fcff0c00700 1 -- 172.21.15.18:0/590954270 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcfe00113a0 con 0x7fcff4109dd0 2024-04-12T21:19:36.835 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:36.838+0000 7fcffb600700 1 -- 172.21.15.18:0/590954270 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcff41149a0 con 0x7fcff4109dd0 2024-04-12T21:19:36.835 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:36.839+0000 7fcffb600700 1 -- 172.21.15.18:0/590954270 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcff4104b80 con 0x7fcff4109dd0 2024-04-12T21:19:36.836 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:36.840+0000 7fcff0c00700 1 -- 172.21.15.18:0/590954270 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7fcfe0012400 con 0x7fcff4109dd0 2024-04-12T21:19:36.837 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:36.841+0000 7fcff0c00700 1 --2- 172.21.15.18:0/590954270 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fcfdc06ec00 0x7fcfdc0710b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:19:36.837 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:36.841+0000 7fcff0c00700 1 -- 172.21.15.18:0/590954270 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 6757+0+0 (secure 0 0 0) 0x7fcfe008ea40 con 0x7fcff4109dd0 2024-04-12T21:19:36.837 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:36.842+0000 7fcff8c00700 1 --2- 172.21.15.18:0/590954270 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fcfdc06ec00 0x7fcfdc0710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:19:36.838 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:36.842+0000 7fcff8c00700 1 --2- 172.21.15.18:0/590954270 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fcfdc06ec00 0x7fcfdc0710b0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fcfe8004f90 tx=0x7fcfe8009e30 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:19:36.842 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:36.846+0000 7fcff0c00700 1 -- 172.21.15.18:0/590954270 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7fcfe005b1f0 con 0x7fcff4109dd0 2024-04-12T21:19:37.115 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:37.118+0000 7fcffb600700 1 -- 172.21.15.18:0/590954270 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fcff4110550 con 0x7fcff4109dd0 2024-04-12T21:19:37.116 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:37.120+0000 7fcff0c00700 1 -- 172.21.15.18:0/590954270 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 29 v29) v1 ==== 76+0+1872 (secure 0 0 0) 0x7fcfe005ad80 con 0x7fcff4109dd0 2024-04-12T21:19:37.117 INFO:teuthology.orchestra.run.smithi018.stdout:e29 2024-04-12T21:19:37.117 INFO:teuthology.orchestra.run.smithi018.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-12T21:19:37.117 INFO:teuthology.orchestra.run.smithi018.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-12T21:19:37.117 INFO:teuthology.orchestra.run.smithi018.stdout:legacy client fscid: 1 2024-04-12T21:19:37.117 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-12T21:19:37.117 INFO:teuthology.orchestra.run.smithi018.stdout:Filesystem 'cephfs' (1) 2024-04-12T21:19:37.117 INFO:teuthology.orchestra.run.smithi018.stdout:fs_name cephfs 2024-04-12T21:19:37.117 INFO:teuthology.orchestra.run.smithi018.stdout:epoch 24 2024-04-12T21:19:37.117 INFO:teuthology.orchestra.run.smithi018.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-04-12T21:19:37.117 INFO:teuthology.orchestra.run.smithi018.stdout:created 2024-04-12T20:51:57.362850+0000 2024-04-12T21:19:37.117 INFO:teuthology.orchestra.run.smithi018.stdout:modified 2024-04-12T21:06:59.717190+0000 2024-04-12T21:19:37.117 INFO:teuthology.orchestra.run.smithi018.stdout:tableserver 0 2024-04-12T21:19:37.117 INFO:teuthology.orchestra.run.smithi018.stdout:root 0 2024-04-12T21:19:37.118 INFO:teuthology.orchestra.run.smithi018.stdout:session_timeout 60 2024-04-12T21:19:37.118 INFO:teuthology.orchestra.run.smithi018.stdout:session_autoclose 300 2024-04-12T21:19:37.118 INFO:teuthology.orchestra.run.smithi018.stdout:max_file_size 1099511627776 2024-04-12T21:19:37.118 INFO:teuthology.orchestra.run.smithi018.stdout:max_xattr_size 65536 2024-04-12T21:19:37.118 INFO:teuthology.orchestra.run.smithi018.stdout:required_client_features {} 2024-04-12T21:19:37.118 INFO:teuthology.orchestra.run.smithi018.stdout:last_failure 0 2024-04-12T21:19:37.118 INFO:teuthology.orchestra.run.smithi018.stdout:last_failure_osd_epoch 100 2024-04-12T21:19:37.118 INFO:teuthology.orchestra.run.smithi018.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-12T21:19:37.118 INFO:teuthology.orchestra.run.smithi018.stdout:max_mds 2 2024-04-12T21:19:37.118 INFO:teuthology.orchestra.run.smithi018.stdout:in 0,1 2024-04-12T21:19:37.118 INFO:teuthology.orchestra.run.smithi018.stdout:up {} 2024-04-12T21:19:37.118 INFO:teuthology.orchestra.run.smithi018.stdout:failed 0,1 2024-04-12T21:19:37.118 INFO:teuthology.orchestra.run.smithi018.stdout:damaged 2024-04-12T21:19:37.119 INFO:teuthology.orchestra.run.smithi018.stdout:stopped 2024-04-12T21:19:37.119 INFO:teuthology.orchestra.run.smithi018.stdout:data_pools [3] 2024-04-12T21:19:37.119 INFO:teuthology.orchestra.run.smithi018.stdout:metadata_pool 2 2024-04-12T21:19:37.119 INFO:teuthology.orchestra.run.smithi018.stdout:inline_data disabled 2024-04-12T21:19:37.119 INFO:teuthology.orchestra.run.smithi018.stdout:balancer 2024-04-12T21:19:37.119 INFO:teuthology.orchestra.run.smithi018.stdout:bal_rank_mask -1 2024-04-12T21:19:37.119 INFO:teuthology.orchestra.run.smithi018.stdout:standby_count_wanted 1 2024-04-12T21:19:37.119 INFO:teuthology.orchestra.run.smithi018.stdout:qdb_cluster leader: 0 members: 2024-04-12T21:19:37.119 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-12T21:19:37.119 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-12T21:19:37.119 INFO:teuthology.orchestra.run.smithi018.stdout:Standby daemons: 2024-04-12T21:19:37.119 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-12T21:19:37.119 INFO:teuthology.orchestra.run.smithi018.stdout:[mds.cephfs.smithi186.wjnafs{-1:24277} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.186:6826/438347207,v1:172.21.15.186:6827/438347207] compat {c=[1],r=[1],i=[7ff]}] 2024-04-12T21:19:37.119 INFO:teuthology.orchestra.run.smithi018.stdout:[mds.cephfs.smithi186.jdllyn{-1:24281} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.186:6824/1464359952,v1:172.21.15.186:6825/1464359952] compat {c=[1],r=[1],i=[7ff]}] 2024-04-12T21:19:37.119 INFO:teuthology.orchestra.run.smithi018.stdout:[mds.cephfs.smithi018.acqbsc{-1:34436} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.18:6828/755179764,v1:172.21.15.18:6829/755179764] compat {c=[1],r=[1],i=[1fff]}] 2024-04-12T21:19:37.120 INFO:teuthology.orchestra.run.smithi018.stdout:[mds.cephfs.smithi018.szjluo{-1:34468} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.18:6826/2407992653,v1:172.21.15.18:6827/2407992653] compat {c=[1],r=[1],i=[1fff]}] 2024-04-12T21:19:37.120 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:37.122+0000 7fcffb600700 1 -- 172.21.15.18:0/590954270 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fcfdc06ec00 msgr2=0x7fcfdc0710b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:19:37.120 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:37.122+0000 7fcffb600700 1 --2- 172.21.15.18:0/590954270 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fcfdc06ec00 0x7fcfdc0710b0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fcfe8004f90 tx=0x7fcfe8009e30 comp rx=0 tx=0).stop 2024-04-12T21:19:37.121 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:37.122+0000 7fcffb600700 1 -- 172.21.15.18:0/590954270 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fcff4109dd0 msgr2=0x7fcff411a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:19:37.121 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:37.122+0000 7fcffb600700 1 --2- 172.21.15.18:0/590954270 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fcff4109dd0 0x7fcff411a480 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7fcfe000c800 tx=0x7fcfe0004470 comp rx=0 tx=0).stop 2024-04-12T21:19:37.121 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:37.123+0000 7fcffb600700 1 -- 172.21.15.18:0/590954270 shutdown_connections 2024-04-12T21:19:37.121 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:37.123+0000 7fcffb600700 1 --2- 172.21.15.18:0/590954270 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fcfdc06ec00 0x7fcfdc0710b0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:19:37.121 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:37.123+0000 7fcffb600700 1 --2- 172.21.15.18:0/590954270 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcff4109430 0x7fcff4119f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:19:37.121 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:37.123+0000 7fcffb600700 1 --2- 172.21.15.18:0/590954270 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fcff4109dd0 0x7fcff411a480 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:19:37.121 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:37.123+0000 7fcffb600700 1 -- 172.21.15.18:0/590954270 >> 172.21.15.18:0/590954270 conn(0x7fcff4072c00 msgr2=0x7fcff4074c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:19:37.121 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:37.123+0000 7fcffb600700 1 -- 172.21.15.18:0/590954270 shutdown_connections 2024-04-12T21:19:37.121 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:37.123+0000 7fcffb600700 1 -- 172.21.15.18:0/590954270 wait complete. 2024-04-12T21:19:37.121 INFO:teuthology.orchestra.run.smithi018.stderr:dumped fsmap epoch 29 2024-04-12T21:19:37.274 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:37.278+0000 7f9ce9c00700 1 -- 172.21.15.18:0/819290706 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9ce40fbbd0 msgr2=0x7f9ce40fbfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:19:37.275 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:37.278+0000 7f9ce9c00700 1 --2- 172.21.15.18:0/819290706 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9ce40fbbd0 0x7f9ce40fbfa0 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f9cd4004940 tx=0x7f9cd4009e30 comp rx=0 tx=0).stop 2024-04-12T21:19:37.275 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:37.278+0000 7f9ce9c00700 1 -- 172.21.15.18:0/819290706 shutdown_connections 2024-04-12T21:19:37.275 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:37.278+0000 7f9ce9c00700 1 --2- 172.21.15.18:0/819290706 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9ce40fc4e0 0x7f9ce4109240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:19:37.275 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:37.278+0000 7f9ce9c00700 1 --2- 172.21.15.18:0/819290706 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9ce40fbbd0 0x7f9ce40fbfa0 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:19:37.275 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:37.278+0000 7f9ce9c00700 1 -- 172.21.15.18:0/819290706 >> 172.21.15.18:0/819290706 conn(0x7f9ce40f7880 msgr2=0x7f9ce40f9c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:19:37.275 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:37.278+0000 7f9ce9c00700 1 -- 172.21.15.18:0/819290706 shutdown_connections 2024-04-12T21:19:37.275 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:37.278+0000 7f9ce9c00700 1 -- 172.21.15.18:0/819290706 wait complete. 2024-04-12T21:19:37.276 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:37.279+0000 7f9ce9c00700 1 Processor -- start 2024-04-12T21:19:37.277 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:37.279+0000 7f9ce9c00700 1 -- start start 2024-04-12T21:19:37.277 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:37.280+0000 7f9ce9c00700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9ce40fbbd0 0x7f9ce4115fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:19:37.277 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:37.280+0000 7f9ce9c00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9ce40fc4e0 0x7f9ce4116520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:19:37.277 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:37.280+0000 7f9ce9c00700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ce4116b70 con 0x7f9ce40fc4e0 2024-04-12T21:19:37.277 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:37.280+0000 7f9ce9c00700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ce410ff40 con 0x7f9ce40fbbd0 2024-04-12T21:19:37.277 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:37.280+0000 7f9ce2000700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9ce40fc4e0 0x7f9ce4116520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:19:37.277 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:37.280+0000 7f9ce2000700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9ce40fc4e0 0x7f9ce4116520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:54562/0 (socket says 172.21.15.18:54562) 2024-04-12T21:19:37.277 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:37.280+0000 7f9ce2000700 1 -- 172.21.15.18:0/2519515350 learned_addr learned my addr 172.21.15.18:0/2519515350 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:19:37.277 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:37.280+0000 7f9ce2a00700 1 --2- 172.21.15.18:0/2519515350 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9ce40fbbd0 0x7f9ce4115fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:19:37.277 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:37.280+0000 7f9ce2000700 1 -- 172.21.15.18:0/2519515350 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9ce40fbbd0 msgr2=0x7f9ce4115fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:19:37.277 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:37.280+0000 7f9ce2000700 1 --2- 172.21.15.18:0/2519515350 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9ce40fbbd0 0x7f9ce4115fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:19:37.277 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:37.280+0000 7f9ce2000700 1 -- 172.21.15.18:0/2519515350 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9cd4005040 con 0x7f9ce40fc4e0 2024-04-12T21:19:37.277 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:37.281+0000 7f9ce2000700 1 --2- 172.21.15.18:0/2519515350 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9ce40fc4e0 0x7f9ce4116520 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f9cd000c830 tx=0x7f9cd0004470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:19:37.278 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:37.281+0000 7f9cdaa00700 1 -- 172.21.15.18:0/2519515350 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9cd000a760 con 0x7f9ce40fc4e0 2024-04-12T21:19:37.278 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:37.281+0000 7f9cdaa00700 1 -- 172.21.15.18:0/2519515350 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f9cd000ad90 con 0x7f9ce40fc4e0 2024-04-12T21:19:37.278 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:37.281+0000 7f9ce9c00700 1 -- 172.21.15.18:0/2519515350 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9ce4110220 con 0x7f9ce40fc4e0 2024-04-12T21:19:37.278 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:37.281+0000 7f9cdaa00700 1 -- 172.21.15.18:0/2519515350 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9cd00113a0 con 0x7f9ce40fc4e0 2024-04-12T21:19:37.278 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:37.281+0000 7f9ce9c00700 1 -- 172.21.15.18:0/2519515350 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9ce41107e0 con 0x7f9ce40fc4e0 2024-04-12T21:19:37.279 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:37.282+0000 7f9ce9c00700 1 -- 172.21.15.18:0/2519515350 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9ce4106970 con 0x7f9ce40fc4e0 2024-04-12T21:19:37.279 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:37.283+0000 7f9cdaa00700 1 -- 172.21.15.18:0/2519515350 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7f9cd0012400 con 0x7f9ce40fc4e0 2024-04-12T21:19:37.280 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:37.284+0000 7f9cdaa00700 1 --2- 172.21.15.18:0/2519515350 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f9cc806ec00 0x7f9cc80710b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:19:37.280 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:37.284+0000 7f9cdaa00700 1 -- 172.21.15.18:0/2519515350 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 6757+0+0 (secure 0 0 0) 0x7f9cd008e9c0 con 0x7f9ce40fc4e0 2024-04-12T21:19:37.280 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:37.285+0000 7f9ce2a00700 1 --2- 172.21.15.18:0/2519515350 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f9cc806ec00 0x7f9cc80710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:19:37.281 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:37.285+0000 7f9ce2a00700 1 --2- 172.21.15.18:0/2519515350 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f9cc806ec00 0x7f9cc80710b0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f9cd4004f90 tx=0x7f9cd4009e30 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:19:37.285 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:37.289+0000 7f9cdaa00700 1 -- 172.21.15.18:0/2519515350 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f9cd005b1f0 con 0x7f9ce40fc4e0 2024-04-12T21:19:37.545 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:37.548+0000 7f9ce9c00700 1 -- 172.21.15.18:0/2519515350 --> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f9ce4111680 con 0x7f9cc806ec00 2024-04-12T21:19:37.547 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:37.551+0000 7f9cdaa00700 1 -- 172.21.15.18:0/2519515350 <== mgr.34102 v2:172.21.15.18:6800/3812891919 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f9ce4111680 con 0x7f9cc806ec00 2024-04-12T21:19:37.547 INFO:teuthology.orchestra.run.smithi018.stdout:{ 2024-04-12T21:19:37.547 INFO:teuthology.orchestra.run.smithi018.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:f4452def3be4d350f4a8e5feaf7ebb1511be9298fe68478ba7a2d6fbd7d69a63", 2024-04-12T21:19:37.548 INFO:teuthology.orchestra.run.smithi018.stdout: "in_progress": true, 2024-04-12T21:19:37.548 INFO:teuthology.orchestra.run.smithi018.stdout: "which": "Upgrading all daemon types on all hosts", 2024-04-12T21:19:37.548 INFO:teuthology.orchestra.run.smithi018.stdout: "services_complete": [ 2024-04-12T21:19:37.548 INFO:teuthology.orchestra.run.smithi018.stdout: "crash", 2024-04-12T21:19:37.548 INFO:teuthology.orchestra.run.smithi018.stdout: "osd", 2024-04-12T21:19:37.548 INFO:teuthology.orchestra.run.smithi018.stdout: "mon", 2024-04-12T21:19:37.548 INFO:teuthology.orchestra.run.smithi018.stdout: "mgr" 2024-04-12T21:19:37.548 INFO:teuthology.orchestra.run.smithi018.stdout: ], 2024-04-12T21:19:37.548 INFO:teuthology.orchestra.run.smithi018.stdout: "progress": "14/21 daemons upgraded", 2024-04-12T21:19:37.549 INFO:teuthology.orchestra.run.smithi018.stdout: "message": "Currently upgrading mds daemons", 2024-04-12T21:19:37.549 INFO:teuthology.orchestra.run.smithi018.stdout: "is_paused": false 2024-04-12T21:19:37.549 INFO:teuthology.orchestra.run.smithi018.stdout:} 2024-04-12T21:19:37.549 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:37.553+0000 7f9ce9c00700 1 -- 172.21.15.18:0/2519515350 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f9cc806ec00 msgr2=0x7f9cc80710b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:19:37.550 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:37.553+0000 7f9ce9c00700 1 --2- 172.21.15.18:0/2519515350 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f9cc806ec00 0x7f9cc80710b0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f9cd4004f90 tx=0x7f9cd4009e30 comp rx=0 tx=0).stop 2024-04-12T21:19:37.550 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:37.553+0000 7f9ce9c00700 1 -- 172.21.15.18:0/2519515350 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9ce40fc4e0 msgr2=0x7f9ce4116520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:19:37.550 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:37.553+0000 7f9ce9c00700 1 --2- 172.21.15.18:0/2519515350 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9ce40fc4e0 0x7f9ce4116520 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f9cd000c830 tx=0x7f9cd0004470 comp rx=0 tx=0).stop 2024-04-12T21:19:37.551 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:37.554+0000 7f9ce9c00700 1 -- 172.21.15.18:0/2519515350 shutdown_connections 2024-04-12T21:19:37.551 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:37.554+0000 7f9ce9c00700 1 --2- 172.21.15.18:0/2519515350 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f9cc806ec00 0x7f9cc80710b0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:19:37.551 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:37.554+0000 7f9ce9c00700 1 --2- 172.21.15.18:0/2519515350 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9ce40fbbd0 0x7f9ce4115fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:19:37.551 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:37.554+0000 7f9ce9c00700 1 --2- 172.21.15.18:0/2519515350 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9ce40fc4e0 0x7f9ce4116520 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:19:37.551 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:37.554+0000 7f9ce9c00700 1 -- 172.21.15.18:0/2519515350 >> 172.21.15.18:0/2519515350 conn(0x7f9ce40f7880 msgr2=0x7f9ce40f7f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:19:37.551 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:37.554+0000 7f9ce9c00700 1 -- 172.21.15.18:0/2519515350 shutdown_connections 2024-04-12T21:19:37.551 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:37.554+0000 7f9ce9c00700 1 -- 172.21.15.18:0/2519515350 wait complete. 2024-04-12T21:19:37.700 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:37.704+0000 7f1571c00700 1 -- 172.21.15.18:0/927695019 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f156c1059c0 msgr2=0x7f156c0ff080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:19:37.701 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:37.704+0000 7f1571c00700 1 --2- 172.21.15.18:0/927695019 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f156c1059c0 0x7f156c0ff080 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f1558004940 tx=0x7f1558009e30 comp rx=0 tx=0).stop 2024-04-12T21:19:37.701 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:37.704+0000 7f1571c00700 1 -- 172.21.15.18:0/927695019 shutdown_connections 2024-04-12T21:19:37.701 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:37.704+0000 7f1571c00700 1 --2- 172.21.15.18:0/927695019 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f156c1059c0 0x7f156c0ff080 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:19:37.701 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:37.704+0000 7f1571c00700 1 --2- 172.21.15.18:0/927695019 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f156c105020 0x7f156c1053f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:19:37.701 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:37.704+0000 7f1571c00700 1 -- 172.21.15.18:0/927695019 >> 172.21.15.18:0/927695019 conn(0x7f156c0faba0 msgr2=0x7f156c0fcfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:19:37.701 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:37.704+0000 7f1571c00700 1 -- 172.21.15.18:0/927695019 shutdown_connections 2024-04-12T21:19:37.701 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:37.705+0000 7f1571c00700 1 -- 172.21.15.18:0/927695019 wait complete. 2024-04-12T21:19:37.702 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:19:37 smithi018 ceph-mon[112590]: from='client.35064 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:19:37.702 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:19:37 smithi018 ceph-mon[112590]: from='client.35068 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:19:37.702 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:19:37 smithi018 ceph-mon[112590]: pgmap v594: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:19:37.702 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:19:37 smithi018 ceph-mon[112590]: from='client.? 172.21.15.18:0/1172115389' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T21:19:37.703 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:19:37 smithi018 ceph-mon[112590]: from='client.? 172.21.15.18:0/590954270' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-12T21:19:37.705 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:37.706+0000 7f1571c00700 1 Processor -- start 2024-04-12T21:19:37.705 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:37.706+0000 7f1571c00700 1 -- start start 2024-04-12T21:19:37.705 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:37.706+0000 7f1571c00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f156c105020 0x7f156c115b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:19:37.706 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:37.706+0000 7f1571c00700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f156c1059c0 0x7f156c116050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:19:37.706 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:37.706+0000 7f1571c00700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f156c116730 con 0x7f156c105020 2024-04-12T21:19:37.706 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:37.706+0000 7f1571c00700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f156c10fcc0 con 0x7f156c1059c0 2024-04-12T21:19:37.706 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:37.707+0000 7f156aa00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f156c105020 0x7f156c115b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:19:37.706 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:37.707+0000 7f156aa00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f156c105020 0x7f156c115b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:54588/0 (socket says 172.21.15.18:54588) 2024-04-12T21:19:37.707 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:37.707+0000 7f156aa00700 1 -- 172.21.15.18:0/995157906 learned_addr learned my addr 172.21.15.18:0/995157906 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:19:37.707 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:37.707+0000 7f156a000700 1 --2- 172.21.15.18:0/995157906 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f156c1059c0 0x7f156c116050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:19:37.707 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:37.707+0000 7f156aa00700 1 -- 172.21.15.18:0/995157906 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f156c1059c0 msgr2=0x7f156c116050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:19:37.707 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:37.707+0000 7f156aa00700 1 --2- 172.21.15.18:0/995157906 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f156c1059c0 0x7f156c116050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:19:37.707 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:37.707+0000 7f156aa00700 1 -- 172.21.15.18:0/995157906 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1558005040 con 0x7f156c105020 2024-04-12T21:19:37.707 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:37.707+0000 7f156a000700 1 --2- 172.21.15.18:0/995157906 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f156c1059c0 0x7f156c116050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-12T21:19:37.707 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:37.708+0000 7f156aa00700 1 --2- 172.21.15.18:0/995157906 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f156c105020 0x7f156c115b10 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f155c00b790 tx=0x7f155c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:19:37.707 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:37.708+0000 7f1562a00700 1 -- 172.21.15.18:0/995157906 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f155c00f770 con 0x7f156c105020 2024-04-12T21:19:37.707 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:37.708+0000 7f1562a00700 1 -- 172.21.15.18:0/995157906 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f155c00fda0 con 0x7f156c105020 2024-04-12T21:19:37.707 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:37.708+0000 7f1571c00700 1 -- 172.21.15.18:0/995157906 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f156c10ffa0 con 0x7f156c105020 2024-04-12T21:19:37.707 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:37.708+0000 7f1562a00700 1 -- 172.21.15.18:0/995157906 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f155c00d310 con 0x7f156c105020 2024-04-12T21:19:37.708 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:37.708+0000 7f1571c00700 1 -- 172.21.15.18:0/995157906 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f156c110560 con 0x7f156c105020 2024-04-12T21:19:37.708 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:37.709+0000 7f1571c00700 1 -- 172.21.15.18:0/995157906 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f154c0052b0 con 0x7f156c105020 2024-04-12T21:19:37.709 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:37.710+0000 7f1562a00700 1 -- 172.21.15.18:0/995157906 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7f155c010420 con 0x7f156c105020 2024-04-12T21:19:37.709 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:37.711+0000 7f1562a00700 1 --2- 172.21.15.18:0/995157906 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f155006ec00 0x7f15500710b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:19:37.709 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:37.711+0000 7f1562a00700 1 -- 172.21.15.18:0/995157906 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 6757+0+0 (secure 0 0 0) 0x7f155c08e9b0 con 0x7f156c105020 2024-04-12T21:19:37.709 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:37.712+0000 7f156a000700 1 --2- 172.21.15.18:0/995157906 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f155006ec00 0x7f15500710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:19:37.709 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:37.712+0000 7f156a000700 1 --2- 172.21.15.18:0/995157906 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f155006ec00 0x7f15500710b0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f155800b3b0 tx=0x7f1558009e30 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:19:37.712 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:37.716+0000 7f1562a00700 1 -- 172.21.15.18:0/995157906 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f155c0576d0 con 0x7f156c105020 2024-04-12T21:19:37.778 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:19:37 smithi186 ceph-mon[86379]: from='client.35064 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:19:37.778 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:19:37 smithi186 ceph-mon[86379]: from='client.35068 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:19:37.778 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:19:37 smithi186 ceph-mon[86379]: pgmap v594: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:19:37.779 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:19:37 smithi186 ceph-mon[86379]: from='client.? 172.21.15.18:0/1172115389' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T21:19:37.779 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:19:37 smithi186 ceph-mon[86379]: from='client.? 172.21.15.18:0/590954270' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-12T21:19:38.021 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:38.025+0000 7f1571c00700 1 -- 172.21.15.18:0/995157906 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f154c005560 con 0x7f156c105020 2024-04-12T21:19:38.022 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:38.026+0000 7f1562a00700 1 -- 172.21.15.18:0/995157906 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+233 (secure 0 0 0) 0x7f155c05ad90 con 0x7f156c105020 2024-04-12T21:19:38.022 INFO:teuthology.orchestra.run.smithi018.stdout:HEALTH_ERR 1 filesystem is degraded; 1 filesystem is offline 2024-04-12T21:19:38.022 INFO:teuthology.orchestra.run.smithi018.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-04-12T21:19:38.022 INFO:teuthology.orchestra.run.smithi018.stdout: fs cephfs is degraded 2024-04-12T21:19:38.022 INFO:teuthology.orchestra.run.smithi018.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-04-12T21:19:38.022 INFO:teuthology.orchestra.run.smithi018.stdout: fs cephfs is offline because no MDS is active for it. 2024-04-12T21:19:38.024 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:38.029+0000 7f1571c00700 1 -- 172.21.15.18:0/995157906 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f155006ec00 msgr2=0x7f15500710b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:19:38.025 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:38.029+0000 7f1571c00700 1 --2- 172.21.15.18:0/995157906 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f155006ec00 0x7f15500710b0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f155800b3b0 tx=0x7f1558009e30 comp rx=0 tx=0).stop 2024-04-12T21:19:38.025 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:38.029+0000 7f1571c00700 1 -- 172.21.15.18:0/995157906 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f156c105020 msgr2=0x7f156c115b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:19:38.025 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:38.029+0000 7f1571c00700 1 --2- 172.21.15.18:0/995157906 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f156c105020 0x7f156c115b10 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f155c00b790 tx=0x7f155c009250 comp rx=0 tx=0).stop 2024-04-12T21:19:38.025 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:38.029+0000 7f1571c00700 1 -- 172.21.15.18:0/995157906 shutdown_connections 2024-04-12T21:19:38.026 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:38.029+0000 7f1571c00700 1 --2- 172.21.15.18:0/995157906 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f155006ec00 0x7f15500710b0 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:19:38.026 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:38.029+0000 7f1571c00700 1 --2- 172.21.15.18:0/995157906 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f156c105020 0x7f156c115b10 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:19:38.026 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:38.029+0000 7f1571c00700 1 --2- 172.21.15.18:0/995157906 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f156c1059c0 0x7f156c116050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:19:38.026 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:38.029+0000 7f1571c00700 1 -- 172.21.15.18:0/995157906 >> 172.21.15.18:0/995157906 conn(0x7f156c0faba0 msgr2=0x7f156c0fbaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:19:38.026 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:38.029+0000 7f1571c00700 1 -- 172.21.15.18:0/995157906 shutdown_connections 2024-04-12T21:19:38.026 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:19:38.029+0000 7f1571c00700 1 -- 172.21.15.18:0/995157906 wait complete. 2024-04-12T21:19:38.778 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:19:38 smithi186 ceph-mon[86379]: from='client.35080 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:19:38.778 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:19:38 smithi186 ceph-mon[86379]: from='client.? 172.21.15.18:0/995157906' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-12T21:19:38.849 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:19:38 smithi018 ceph-mon[112590]: from='client.35080 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:19:38.849 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:19:38 smithi018 ceph-mon[112590]: from='client.? 172.21.15.18:0/995157906' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-12T21:19:39.680 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:19:39 smithi186 ceph-mon[86379]: pgmap v595: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:19:39.849 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:19:39 smithi018 ceph-mon[112590]: pgmap v595: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:19:40.779 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:19:40 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T21:19:40.849 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:19:40 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T21:19:41.751 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:19:41 smithi018 ceph-mon[112590]: pgmap v596: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:19:41.778 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:19:41 smithi186 ceph-mon[86379]: pgmap v596: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:19:43.778 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:19:43 smithi186 ceph-mon[86379]: pgmap v597: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:19:43.849 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:19:43 smithi018 ceph-mon[112590]: pgmap v597: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:19:45.778 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:19:45 smithi186 ceph-mon[86379]: pgmap v598: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:19:45.849 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:19:45 smithi018 ceph-mon[112590]: pgmap v598: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:19:47.778 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:19:47 smithi186 ceph-mon[86379]: pgmap v599: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:19:47.849 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:19:47 smithi018 ceph-mon[112590]: pgmap v599: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:19:49.778 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:19:49 smithi186 ceph-mon[86379]: pgmap v600: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:19:49.849 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:19:49 smithi018 ceph-mon[112590]: pgmap v600: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:19:51.779 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:19:51 smithi186 ceph-mon[86379]: pgmap v601: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:19:51.849 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:19:51 smithi018 ceph-mon[112590]: pgmap v601: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:19:53.778 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:19:53 smithi186 ceph-mon[86379]: pgmap v602: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:19:53.849 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:19:53 smithi018 ceph-mon[112590]: pgmap v602: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:19:55.778 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:19:55 smithi186 ceph-mon[86379]: pgmap v603: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:19:55.778 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:19:55 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T21:19:55.849 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:19:55 smithi018 ceph-mon[112590]: pgmap v603: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:19:55.849 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:19:55 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T21:19:57.778 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:19:57 smithi186 ceph-mon[86379]: pgmap v604: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:19:57.849 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:19:57 smithi018 ceph-mon[112590]: pgmap v604: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:19:59.778 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:19:59 smithi186 ceph-mon[86379]: pgmap v605: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:19:59.849 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:19:59 smithi018 ceph-mon[112590]: pgmap v605: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:20:00.349 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:20:00 smithi018 ceph-1cbe407a-f90d-11ee-bc8d-c7b262605968-mon-smithi018[112586]: 2024-04-12T21:19:59.999+0000 7fd45a600700 -1 log_channel(cluster) log [ERR] : overall HEALTH_ERR 1 filesystem is degraded; 1 filesystem is offline 2024-04-12T21:20:00.778 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:20:00 smithi186 ceph-mon[86379]: overall HEALTH_ERR 1 filesystem is degraded; 1 filesystem is offline 2024-04-12T21:20:00.849 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:20:00 smithi018 ceph-mon[112590]: overall HEALTH_ERR 1 filesystem is degraded; 1 filesystem is offline 2024-04-12T21:20:01.778 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:20:01 smithi186 ceph-mon[86379]: pgmap v606: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:20:01.849 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:20:01 smithi018 ceph-mon[112590]: pgmap v606: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:20:03.778 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:20:03 smithi186 ceph-mon[86379]: pgmap v607: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:20:03.849 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:20:03 smithi018 ceph-mon[112590]: pgmap v607: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:20:05.778 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:20:05 smithi186 ceph-mon[86379]: pgmap v608: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:20:05.849 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:20:05 smithi018 ceph-mon[112590]: pgmap v608: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:20:07.778 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:20:07 smithi186 ceph-mon[86379]: pgmap v609: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:20:07.849 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:20:07 smithi018 ceph-mon[112590]: pgmap v609: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:20:08.198 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:08.202+0000 7fed79e00700 1 -- 172.21.15.18:0/1814437718 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fed74105990 msgr2=0x7fed740ff000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:20:08.198 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:08.202+0000 7fed79e00700 1 --2- 172.21.15.18:0/1814437718 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fed74105990 0x7fed740ff000 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7fed60004940 tx=0x7fed60009e30 comp rx=0 tx=0).stop 2024-04-12T21:20:08.198 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:08.203+0000 7fed79e00700 1 -- 172.21.15.18:0/1814437718 shutdown_connections 2024-04-12T21:20:08.198 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:08.203+0000 7fed79e00700 1 --2- 172.21.15.18:0/1814437718 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fed74105990 0x7fed740ff000 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:20:08.198 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:08.203+0000 7fed79e00700 1 --2- 172.21.15.18:0/1814437718 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fed74104ff0 0x7fed741053c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:20:08.201 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:08.203+0000 7fed79e00700 1 -- 172.21.15.18:0/1814437718 >> 172.21.15.18:0/1814437718 conn(0x7fed740faba0 msgr2=0x7fed740fcfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:20:08.201 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:08.203+0000 7fed79e00700 1 -- 172.21.15.18:0/1814437718 shutdown_connections 2024-04-12T21:20:08.201 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:08.203+0000 7fed79e00700 1 -- 172.21.15.18:0/1814437718 wait complete. 2024-04-12T21:20:08.203 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:08.204+0000 7fed79e00700 1 Processor -- start 2024-04-12T21:20:08.203 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:08.204+0000 7fed79e00700 1 -- start start 2024-04-12T21:20:08.203 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:08.204+0000 7fed79e00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fed74104ff0 0x7fed74115b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:20:08.203 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:08.204+0000 7fed79e00700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fed74105990 0x7fed741160d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:20:08.203 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:08.204+0000 7fed79e00700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed741167b0 con 0x7fed74104ff0 2024-04-12T21:20:08.203 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:08.204+0000 7fed79e00700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed7410fd00 con 0x7fed74105990 2024-04-12T21:20:08.204 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:08.204+0000 7fed72a00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fed74104ff0 0x7fed74115b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:20:08.204 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:08.205+0000 7fed72a00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fed74104ff0 0x7fed74115b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:38794/0 (socket says 172.21.15.18:38794) 2024-04-12T21:20:08.204 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:08.205+0000 7fed72a00700 1 -- 172.21.15.18:0/4200816502 learned_addr learned my addr 172.21.15.18:0/4200816502 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:20:08.204 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:08.205+0000 7fed72000700 1 --2- 172.21.15.18:0/4200816502 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fed74105990 0x7fed741160d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:20:08.204 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:08.205+0000 7fed72a00700 1 -- 172.21.15.18:0/4200816502 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fed74105990 msgr2=0x7fed741160d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:20:08.204 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:08.205+0000 7fed72a00700 1 --2- 172.21.15.18:0/4200816502 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fed74105990 0x7fed741160d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:20:08.204 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:08.205+0000 7fed72a00700 1 -- 172.21.15.18:0/4200816502 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed60005040 con 0x7fed74104ff0 2024-04-12T21:20:08.204 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:08.206+0000 7fed72a00700 1 --2- 172.21.15.18:0/4200816502 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fed74104ff0 0x7fed74115b90 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7fed6400b790 tx=0x7fed64009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:20:08.204 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:08.206+0000 7fed6aa00700 1 -- 172.21.15.18:0/4200816502 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fed6400f770 con 0x7fed74104ff0 2024-04-12T21:20:08.204 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:08.206+0000 7fed6aa00700 1 -- 172.21.15.18:0/4200816502 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fed6400fda0 con 0x7fed74104ff0 2024-04-12T21:20:08.204 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:08.206+0000 7fed79e00700 1 -- 172.21.15.18:0/4200816502 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fed7410ff80 con 0x7fed74104ff0 2024-04-12T21:20:08.204 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:08.206+0000 7fed6aa00700 1 -- 172.21.15.18:0/4200816502 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fed6400d310 con 0x7fed74104ff0 2024-04-12T21:20:08.204 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:08.206+0000 7fed79e00700 1 -- 172.21.15.18:0/4200816502 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fed74110540 con 0x7fed74104ff0 2024-04-12T21:20:08.205 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:08.207+0000 7fed79e00700 1 -- 172.21.15.18:0/4200816502 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fed74100740 con 0x7fed74104ff0 2024-04-12T21:20:08.205 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:08.208+0000 7fed6aa00700 1 -- 172.21.15.18:0/4200816502 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7fed64010420 con 0x7fed74104ff0 2024-04-12T21:20:08.205 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:08.209+0000 7fed6aa00700 1 --2- 172.21.15.18:0/4200816502 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fed5806ec00 0x7fed580710b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:20:08.205 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:08.210+0000 7fed6aa00700 1 -- 172.21.15.18:0/4200816502 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 6757+0+0 (secure 0 0 0) 0x7fed6408e9c0 con 0x7fed74104ff0 2024-04-12T21:20:08.206 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:08.210+0000 7fed72000700 1 --2- 172.21.15.18:0/4200816502 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fed5806ec00 0x7fed580710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:20:08.206 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:08.211+0000 7fed72000700 1 --2- 172.21.15.18:0/4200816502 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fed5806ec00 0x7fed580710b0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fed6000b020 tx=0x7fed60009e30 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:20:08.210 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:08.215+0000 7fed6aa00700 1 -- 172.21.15.18:0/4200816502 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7fed6405b210 con 0x7fed74104ff0 2024-04-12T21:20:08.464 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:08.469+0000 7fed79e00700 1 -- 172.21.15.18:0/4200816502 --> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fed741107c0 con 0x7fed5806ec00 2024-04-12T21:20:08.466 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:08.471+0000 7fed6aa00700 1 -- 172.21.15.18:0/4200816502 <== mgr.34102 v2:172.21.15.18:6800/3812891919 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7fed741107c0 con 0x7fed5806ec00 2024-04-12T21:20:08.470 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:08.474+0000 7fed79e00700 1 -- 172.21.15.18:0/4200816502 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fed5806ec00 msgr2=0x7fed580710b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:20:08.470 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:08.474+0000 7fed79e00700 1 --2- 172.21.15.18:0/4200816502 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fed5806ec00 0x7fed580710b0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fed6000b020 tx=0x7fed60009e30 comp rx=0 tx=0).stop 2024-04-12T21:20:08.470 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:08.474+0000 7fed79e00700 1 -- 172.21.15.18:0/4200816502 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fed74104ff0 msgr2=0x7fed74115b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:20:08.470 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:08.474+0000 7fed79e00700 1 --2- 172.21.15.18:0/4200816502 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fed74104ff0 0x7fed74115b90 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7fed6400b790 tx=0x7fed64009250 comp rx=0 tx=0).stop 2024-04-12T21:20:08.470 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:08.474+0000 7fed79e00700 1 -- 172.21.15.18:0/4200816502 shutdown_connections 2024-04-12T21:20:08.470 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:08.474+0000 7fed79e00700 1 --2- 172.21.15.18:0/4200816502 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fed5806ec00 0x7fed580710b0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:20:08.470 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:08.474+0000 7fed79e00700 1 --2- 172.21.15.18:0/4200816502 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fed74104ff0 0x7fed74115b90 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:20:08.470 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:08.474+0000 7fed79e00700 1 --2- 172.21.15.18:0/4200816502 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fed74105990 0x7fed741160d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:20:08.470 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:08.474+0000 7fed79e00700 1 -- 172.21.15.18:0/4200816502 >> 172.21.15.18:0/4200816502 conn(0x7fed740faba0 msgr2=0x7fed740fbad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:20:08.471 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:08.474+0000 7fed79e00700 1 -- 172.21.15.18:0/4200816502 shutdown_connections 2024-04-12T21:20:08.471 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:08.475+0000 7fed79e00700 1 -- 172.21.15.18:0/4200816502 wait complete. 2024-04-12T21:20:08.486 INFO:teuthology.orchestra.run.smithi018.stdout:true 2024-04-12T21:20:08.636 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:08.640+0000 7ff7ed800700 1 -- 172.21.15.18:0/4129523499 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff7e81037a0 msgr2=0x7ff7e80fce10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:20:08.636 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:08.640+0000 7ff7ed800700 1 --2- 172.21.15.18:0/4129523499 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff7e81037a0 0x7ff7e80fce10 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7ff7d4004940 tx=0x7ff7d4009e30 comp rx=0 tx=0).stop 2024-04-12T21:20:08.636 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:08.640+0000 7ff7ed800700 1 -- 172.21.15.18:0/4129523499 shutdown_connections 2024-04-12T21:20:08.636 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:08.640+0000 7ff7ed800700 1 --2- 172.21.15.18:0/4129523499 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff7e81037a0 0x7ff7e80fce10 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:20:08.636 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:08.640+0000 7ff7ed800700 1 --2- 172.21.15.18:0/4129523499 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff7e8102e00 0x7ff7e81031d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:20:08.636 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:08.640+0000 7ff7ed800700 1 -- 172.21.15.18:0/4129523499 >> 172.21.15.18:0/4129523499 conn(0x7ff7e80f89b0 msgr2=0x7ff7e80fadc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:20:08.636 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:08.641+0000 7ff7ed800700 1 -- 172.21.15.18:0/4129523499 shutdown_connections 2024-04-12T21:20:08.636 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:08.641+0000 7ff7ed800700 1 -- 172.21.15.18:0/4129523499 wait complete. 2024-04-12T21:20:08.637 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:08.641+0000 7ff7ed800700 1 Processor -- start 2024-04-12T21:20:08.637 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:08.642+0000 7ff7ed800700 1 -- start start 2024-04-12T21:20:08.637 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:08.642+0000 7ff7ed800700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff7e8102e00 0x7ff7e8115de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:20:08.637 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:08.642+0000 7ff7ed800700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff7e81037a0 0x7ff7e8116320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:20:08.638 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:08.642+0000 7ff7ed800700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7e8116970 con 0x7ff7e81037a0 2024-04-12T21:20:08.638 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:08.642+0000 7ff7ed800700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7e810fd80 con 0x7ff7e8102e00 2024-04-12T21:20:08.638 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:08.642+0000 7ff7e6000700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff7e81037a0 0x7ff7e8116320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:20:08.638 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:08.642+0000 7ff7e6000700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff7e81037a0 0x7ff7e8116320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:38804/0 (socket says 172.21.15.18:38804) 2024-04-12T21:20:08.638 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:08.642+0000 7ff7e6000700 1 -- 172.21.15.18:0/1470207502 learned_addr learned my addr 172.21.15.18:0/1470207502 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:20:08.639 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:08.643+0000 7ff7e6a00700 1 --2- 172.21.15.18:0/1470207502 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff7e8102e00 0x7ff7e8115de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:20:08.639 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:08.643+0000 7ff7e6000700 1 -- 172.21.15.18:0/1470207502 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff7e8102e00 msgr2=0x7ff7e8115de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:20:08.639 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:08.643+0000 7ff7e6000700 1 --2- 172.21.15.18:0/1470207502 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff7e8102e00 0x7ff7e8115de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:20:08.639 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:08.643+0000 7ff7e6000700 1 -- 172.21.15.18:0/1470207502 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7d4005040 con 0x7ff7e81037a0 2024-04-12T21:20:08.639 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:08.643+0000 7ff7e6000700 1 --2- 172.21.15.18:0/1470207502 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff7e81037a0 0x7ff7e8116320 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7ff7d4009180 tx=0x7ff7d4009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:20:08.639 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:08.644+0000 7ff7dea00700 1 -- 172.21.15.18:0/1470207502 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff7d400e040 con 0x7ff7e81037a0 2024-04-12T21:20:08.639 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:08.644+0000 7ff7ed800700 1 -- 172.21.15.18:0/1470207502 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff7e8110000 con 0x7ff7e81037a0 2024-04-12T21:20:08.639 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:08.644+0000 7ff7dea00700 1 -- 172.21.15.18:0/1470207502 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff7d400bbc0 con 0x7ff7e81037a0 2024-04-12T21:20:08.640 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:08.644+0000 7ff7dea00700 1 -- 172.21.15.18:0/1470207502 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff7d401c3f0 con 0x7ff7e81037a0 2024-04-12T21:20:08.640 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:08.644+0000 7ff7ed800700 1 -- 172.21.15.18:0/1470207502 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff7e8110540 con 0x7ff7e81037a0 2024-04-12T21:20:08.640 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:08.645+0000 7ff7ed800700 1 -- 172.21.15.18:0/1470207502 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff7e8102120 con 0x7ff7e81037a0 2024-04-12T21:20:08.641 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:08.646+0000 7ff7dea00700 1 -- 172.21.15.18:0/1470207502 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7ff7d400bd30 con 0x7ff7e81037a0 2024-04-12T21:20:08.642 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:08.647+0000 7ff7dea00700 1 --2- 172.21.15.18:0/1470207502 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7ff7cc06ec00 0x7ff7cc0710b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:20:08.642 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:08.647+0000 7ff7dea00700 1 -- 172.21.15.18:0/1470207502 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 6757+0+0 (secure 0 0 0) 0x7ff7d408fdd0 con 0x7ff7e81037a0 2024-04-12T21:20:08.642 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:08.648+0000 7ff7e6a00700 1 --2- 172.21.15.18:0/1470207502 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7ff7cc06ec00 0x7ff7cc0710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:20:08.643 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:08.648+0000 7ff7e6a00700 1 --2- 172.21.15.18:0/1470207502 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7ff7cc06ec00 0x7ff7cc0710b0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7ff7d8004f30 tx=0x7ff7d800b040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:20:08.647 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:08.652+0000 7ff7dea00700 1 -- 172.21.15.18:0/1470207502 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7ff7d4058a50 con 0x7ff7e81037a0 2024-04-12T21:20:08.896 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:08.901+0000 7ff7ed800700 1 -- 172.21.15.18:0/1470207502 --> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7ff7e8111470 con 0x7ff7cc06ec00 2024-04-12T21:20:08.898 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:08.903+0000 7ff7dea00700 1 -- 172.21.15.18:0/1470207502 <== mgr.34102 v2:172.21.15.18:6800/3812891919 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7ff7e8111470 con 0x7ff7cc06ec00 2024-04-12T21:20:08.901 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:08.906+0000 7ff7ed800700 1 -- 172.21.15.18:0/1470207502 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7ff7cc06ec00 msgr2=0x7ff7cc0710b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:20:08.901 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:08.906+0000 7ff7ed800700 1 --2- 172.21.15.18:0/1470207502 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7ff7cc06ec00 0x7ff7cc0710b0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7ff7d8004f30 tx=0x7ff7d800b040 comp rx=0 tx=0).stop 2024-04-12T21:20:08.901 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:08.906+0000 7ff7ed800700 1 -- 172.21.15.18:0/1470207502 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff7e81037a0 msgr2=0x7ff7e8116320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:20:08.902 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:08.906+0000 7ff7ed800700 1 --2- 172.21.15.18:0/1470207502 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff7e81037a0 0x7ff7e8116320 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7ff7d4009180 tx=0x7ff7d4009e30 comp rx=0 tx=0).stop 2024-04-12T21:20:08.902 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:08.906+0000 7ff7ed800700 1 -- 172.21.15.18:0/1470207502 shutdown_connections 2024-04-12T21:20:08.902 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:08.906+0000 7ff7ed800700 1 --2- 172.21.15.18:0/1470207502 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7ff7cc06ec00 0x7ff7cc0710b0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:20:08.902 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:08.906+0000 7ff7ed800700 1 --2- 172.21.15.18:0/1470207502 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff7e8102e00 0x7ff7e8115de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:20:08.902 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:08.906+0000 7ff7ed800700 1 --2- 172.21.15.18:0/1470207502 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff7e81037a0 0x7ff7e8116320 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:20:08.902 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:08.906+0000 7ff7ed800700 1 -- 172.21.15.18:0/1470207502 >> 172.21.15.18:0/1470207502 conn(0x7ff7e80f89b0 msgr2=0x7ff7e8070bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:20:08.902 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:08.906+0000 7ff7ed800700 1 -- 172.21.15.18:0/1470207502 shutdown_connections 2024-04-12T21:20:08.902 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:08.906+0000 7ff7ed800700 1 -- 172.21.15.18:0/1470207502 wait complete. 2024-04-12T21:20:09.063 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:09.067+0000 7fbd15000700 1 -- 172.21.15.18:0/1795209797 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fbd100702e0 msgr2=0x7fbd100706b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:20:09.063 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:09.067+0000 7fbd15000700 1 --2- 172.21.15.18:0/1795209797 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fbd100702e0 0x7fbd100706b0 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7fbd00004940 tx=0x7fbd00009e30 comp rx=0 tx=0).stop 2024-04-12T21:20:09.063 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:09.067+0000 7fbd15000700 1 -- 172.21.15.18:0/1795209797 shutdown_connections 2024-04-12T21:20:09.063 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:09.067+0000 7fbd15000700 1 --2- 172.21.15.18:0/1795209797 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbd10070bf0 0x7fbd10109190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:20:09.063 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:09.067+0000 7fbd15000700 1 --2- 172.21.15.18:0/1795209797 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fbd100702e0 0x7fbd100706b0 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:20:09.063 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:09.067+0000 7fbd15000700 1 -- 172.21.15.18:0/1795209797 >> 172.21.15.18:0/1795209797 conn(0x7fbd100f89b0 msgr2=0x7fbd100fadc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:20:09.063 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:09.067+0000 7fbd15000700 1 -- 172.21.15.18:0/1795209797 shutdown_connections 2024-04-12T21:20:09.063 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:09.067+0000 7fbd15000700 1 -- 172.21.15.18:0/1795209797 wait complete. 2024-04-12T21:20:09.063 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:09.068+0000 7fbd15000700 1 Processor -- start 2024-04-12T21:20:09.065 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:09.068+0000 7fbd15000700 1 -- start start 2024-04-12T21:20:09.065 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:09.069+0000 7fbd15000700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fbd100702e0 0x7fbd10115be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:20:09.065 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:09.069+0000 7fbd15000700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbd10070bf0 0x7fbd10116120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:20:09.065 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:09.069+0000 7fbd15000700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd10116770 con 0x7fbd100702e0 2024-04-12T21:20:09.065 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:09.069+0000 7fbd15000700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd1010fd50 con 0x7fbd10070bf0 2024-04-12T21:20:09.065 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:09.069+0000 7fbd0de00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fbd100702e0 0x7fbd10115be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:20:09.065 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:09.069+0000 7fbd0de00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fbd100702e0 0x7fbd10115be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:38828/0 (socket says 172.21.15.18:38828) 2024-04-12T21:20:09.065 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:09.069+0000 7fbd0de00700 1 -- 172.21.15.18:0/3573863317 learned_addr learned my addr 172.21.15.18:0/3573863317 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:20:09.065 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:09.070+0000 7fbd0de00700 1 -- 172.21.15.18:0/3573863317 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbd10070bf0 msgr2=0x7fbd10116120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:20:09.065 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:09.070+0000 7fbd0d400700 1 --2- 172.21.15.18:0/3573863317 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbd10070bf0 0x7fbd10116120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:20:09.066 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:09.070+0000 7fbd0de00700 1 --2- 172.21.15.18:0/3573863317 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbd10070bf0 0x7fbd10116120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:20:09.066 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:09.070+0000 7fbd0de00700 1 -- 172.21.15.18:0/3573863317 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd00005040 con 0x7fbd100702e0 2024-04-12T21:20:09.066 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:09.070+0000 7fbd0de00700 1 --2- 172.21.15.18:0/3573863317 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fbd100702e0 0x7fbd10115be0 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7fbd00009150 tx=0x7fbd00009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:20:09.066 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:09.070+0000 7fbd06000700 1 -- 172.21.15.18:0/3573863317 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbd00021070 con 0x7fbd100702e0 2024-04-12T21:20:09.066 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:09.070+0000 7fbd06000700 1 -- 172.21.15.18:0/3573863317 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fbd00013c00 con 0x7fbd100702e0 2024-04-12T21:20:09.067 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:09.070+0000 7fbd06000700 1 -- 172.21.15.18:0/3573863317 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbd0000e070 con 0x7fbd100702e0 2024-04-12T21:20:09.067 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:09.070+0000 7fbd15000700 1 -- 172.21.15.18:0/3573863317 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbd1010ffd0 con 0x7fbd100702e0 2024-04-12T21:20:09.067 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:09.071+0000 7fbd15000700 1 -- 172.21.15.18:0/3573863317 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbd101104b0 con 0x7fbd100702e0 2024-04-12T21:20:09.067 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:09.072+0000 7fbd15000700 1 -- 172.21.15.18:0/3573863317 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbd10071a20 con 0x7fbd100702e0 2024-04-12T21:20:09.067 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:09.073+0000 7fbd06000700 1 -- 172.21.15.18:0/3573863317 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7fbd0001c400 con 0x7fbd100702e0 2024-04-12T21:20:09.069 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:09.074+0000 7fbd06000700 1 --2- 172.21.15.18:0/3573863317 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fbcf406ec00 0x7fbcf40710b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:20:09.069 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:09.074+0000 7fbd06000700 1 -- 172.21.15.18:0/3573863317 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 6757+0+0 (secure 0 0 0) 0x7fbd0008fd30 con 0x7fbd100702e0 2024-04-12T21:20:09.069 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:09.074+0000 7fbd0d400700 1 --2- 172.21.15.18:0/3573863317 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fbcf406ec00 0x7fbcf40710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:20:09.070 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:09.075+0000 7fbd0d400700 1 --2- 172.21.15.18:0/3573863317 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fbcf406ec00 0x7fbcf40710b0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fbcfc004600 tx=0x7fbcfc00b040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:20:09.073 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:09.079+0000 7fbd06000700 1 -- 172.21.15.18:0/3573863317 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7fbd0005c620 con 0x7fbd100702e0 2024-04-12T21:20:09.318 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:09.323+0000 7fbd15000700 1 -- 172.21.15.18:0/3573863317 --> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fbd10111280 con 0x7fbcf406ec00 2024-04-12T21:20:09.348 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:09.353+0000 7fbd06000700 1 -- 172.21.15.18:0/3573863317 <== mgr.34102 v2:172.21.15.18:6800/3812891919 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7fbd10111280 con 0x7fbcf406ec00 2024-04-12T21:20:09.349 INFO:teuthology.orchestra.run.smithi018.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-12T21:20:09.349 INFO:teuthology.orchestra.run.smithi018.stdout:alertmanager.smithi018 smithi018 *:9093,9094 running (20m) 12m ago 35m 14.7M - 0.25.0 c8568f914cd2 c332090068e8 2024-04-12T21:20:09.349 INFO:teuthology.orchestra.run.smithi018.stdout:crash.smithi018 smithi018 running (17m) 12m ago 35m 6665k - 19.0.0-2879-g324baf8d 8717720dc5e5 b7f80d1f7982 2024-04-12T21:20:09.349 INFO:teuthology.orchestra.run.smithi018.stdout:crash.smithi186 smithi186 running (16m) 13m ago 34m 6660k - 19.0.0-2879-g324baf8d 8717720dc5e5 a4f55c81b623 2024-04-12T21:20:09.349 INFO:teuthology.orchestra.run.smithi018.stdout:grafana.smithi018 smithi018 *:3000 running (20m) 12m ago 34m 74.8M - 9.4.12 2bacad6d85d8 b08e3bf176dd 2024-04-12T21:20:09.349 INFO:teuthology.orchestra.run.smithi018.stdout:mds.cephfs.smithi018.acqbsc smithi018 running (12m) 12m ago 28m 11.1M - 19.0.0-2879-g324baf8d 8717720dc5e5 021f99ab24ae 2024-04-12T21:20:09.349 INFO:teuthology.orchestra.run.smithi018.stdout:mds.cephfs.smithi018.szjluo smithi018 running (12m) 12m ago 27m 10.9M - 19.0.0-2879-g324baf8d 8717720dc5e5 0003117479d4 2024-04-12T21:20:09.349 INFO:teuthology.orchestra.run.smithi018.stdout:mds.cephfs.smithi186.jdllyn smithi186 running (28m) 13m ago 28m 15.6M - 17.2.7-827-gd3b5060a 21de9b7369ad 2b2bc46bd29a 2024-04-12T21:20:09.349 INFO:teuthology.orchestra.run.smithi018.stdout:mds.cephfs.smithi186.wjnafs smithi186 running (27m) 13m ago 27m 15.4M - 17.2.7-827-gd3b5060a 21de9b7369ad 74b115d6d6fd 2024-04-12T21:20:09.349 INFO:teuthology.orchestra.run.smithi018.stdout:mgr.smithi018.xjzwga smithi018 *:8443,9283,8765 running (23m) 12m ago 36m 509M - 19.0.0-2879-g324baf8d 8717720dc5e5 56088305eab7 2024-04-12T21:20:09.350 INFO:teuthology.orchestra.run.smithi018.stdout:mgr.smithi186.kqcfxk smithi186 *:8443,9283,8765 running (22m) 13m ago 33m 452M - 19.0.0-2879-g324baf8d 8717720dc5e5 09cd766e5e0d 2024-04-12T21:20:09.350 INFO:teuthology.orchestra.run.smithi018.stdout:mon.smithi018 smithi018 running (18m) 12m ago 37m 57.0M 2048M 19.0.0-2879-g324baf8d 8717720dc5e5 df3e201dfed6 2024-04-12T21:20:09.350 INFO:teuthology.orchestra.run.smithi018.stdout:mon.smithi186 smithi186 running (18m) 13m ago 33m 37.7M 2048M 19.0.0-2879-g324baf8d 8717720dc5e5 c77498da9b2b 2024-04-12T21:20:09.350 INFO:teuthology.orchestra.run.smithi018.stdout:node-exporter.smithi018 smithi018 *:9100 running (21m) 12m ago 35m 7780k - 1.5.0 0da6a335fe13 326c3d9c6847 2024-04-12T21:20:09.350 INFO:teuthology.orchestra.run.smithi018.stdout:node-exporter.smithi186 smithi186 *:9100 running (21m) 13m ago 34m 7826k - 1.5.0 0da6a335fe13 ec2112baeecf 2024-04-12T21:20:09.350 INFO:teuthology.orchestra.run.smithi018.stdout:osd.0 smithi018 running (16m) 12m ago 32m 121M 1559M 19.0.0-2879-g324baf8d 8717720dc5e5 60b1273ca047 2024-04-12T21:20:09.350 INFO:teuthology.orchestra.run.smithi018.stdout:osd.1 smithi018 running (15m) 12m ago 32m 91.3M 1559M 19.0.0-2879-g324baf8d 8717720dc5e5 170ce1e5fb8d 2024-04-12T21:20:09.350 INFO:teuthology.orchestra.run.smithi018.stdout:osd.2 smithi018 running (15m) 12m ago 31m 90.3M 1559M 19.0.0-2879-g324baf8d 8717720dc5e5 c2cbc6aa2839 2024-04-12T21:20:09.350 INFO:teuthology.orchestra.run.smithi018.stdout:osd.3 smithi186 running (14m) 13m ago 31m 96.6M 2581M 19.0.0-2879-g324baf8d 8717720dc5e5 0138e8262e4b 2024-04-12T21:20:09.350 INFO:teuthology.orchestra.run.smithi018.stdout:osd.4 smithi186 running (14m) 13m ago 30m 87.2M 2581M 19.0.0-2879-g324baf8d 8717720dc5e5 b20cb63b25c9 2024-04-12T21:20:09.350 INFO:teuthology.orchestra.run.smithi018.stdout:osd.5 smithi186 running (13m) 13m ago 29m 11.1M 2581M 19.0.0-2879-g324baf8d 8717720dc5e5 2513bea2610e 2024-04-12T21:20:09.350 INFO:teuthology.orchestra.run.smithi018.stdout:prometheus.smithi018 smithi018 *:9095 running (21m) 12m ago 34m 60.6M - 2.43.0 a07b618ecd1d f58841127e21 2024-04-12T21:20:09.351 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:09.356+0000 7fbd15000700 1 -- 172.21.15.18:0/3573863317 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fbcf406ec00 msgr2=0x7fbcf40710b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:20:09.352 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:09.356+0000 7fbd15000700 1 --2- 172.21.15.18:0/3573863317 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fbcf406ec00 0x7fbcf40710b0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fbcfc004600 tx=0x7fbcfc00b040 comp rx=0 tx=0).stop 2024-04-12T21:20:09.352 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:09.356+0000 7fbd15000700 1 -- 172.21.15.18:0/3573863317 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fbd100702e0 msgr2=0x7fbd10115be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:20:09.352 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:09.356+0000 7fbd15000700 1 --2- 172.21.15.18:0/3573863317 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fbd100702e0 0x7fbd10115be0 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7fbd00009150 tx=0x7fbd00009e30 comp rx=0 tx=0).stop 2024-04-12T21:20:09.352 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:09.356+0000 7fbd15000700 1 -- 172.21.15.18:0/3573863317 shutdown_connections 2024-04-12T21:20:09.352 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:09.356+0000 7fbd15000700 1 --2- 172.21.15.18:0/3573863317 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fbcf406ec00 0x7fbcf40710b0 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:20:09.352 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:09.356+0000 7fbd15000700 1 --2- 172.21.15.18:0/3573863317 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fbd100702e0 0x7fbd10115be0 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:20:09.352 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:09.356+0000 7fbd15000700 1 --2- 172.21.15.18:0/3573863317 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbd10070bf0 0x7fbd10116120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:20:09.352 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:09.356+0000 7fbd15000700 1 -- 172.21.15.18:0/3573863317 >> 172.21.15.18:0/3573863317 conn(0x7fbd100f89b0 msgr2=0x7fbd100f9d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:20:09.352 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:09.357+0000 7fbd15000700 1 -- 172.21.15.18:0/3573863317 shutdown_connections 2024-04-12T21:20:09.352 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:09.357+0000 7fbd15000700 1 -- 172.21.15.18:0/3573863317 wait complete. 2024-04-12T21:20:09.511 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:09.515+0000 7f4fcee00700 1 -- 172.21.15.18:0/3610379153 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4fc8101d10 msgr2=0x7f4fc81020e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:20:09.511 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:09.515+0000 7f4fcee00700 1 --2- 172.21.15.18:0/3610379153 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4fc8101d10 0x7f4fc81020e0 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f4fb8004940 tx=0x7f4fb8009e30 comp rx=0 tx=0).stop 2024-04-12T21:20:09.511 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:09.516+0000 7f4fcee00700 1 -- 172.21.15.18:0/3610379153 shutdown_connections 2024-04-12T21:20:09.511 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:09.516+0000 7f4fcee00700 1 --2- 172.21.15.18:0/3610379153 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4fc8102620 0x7f4fc80fc840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:20:09.511 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:09.516+0000 7f4fcee00700 1 --2- 172.21.15.18:0/3610379153 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4fc8101d10 0x7f4fc81020e0 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:20:09.511 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:09.516+0000 7f4fcee00700 1 -- 172.21.15.18:0/3610379153 >> 172.21.15.18:0/3610379153 conn(0x7f4fc80f6bd0 msgr2=0x7f4fc80f8fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:20:09.511 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:09.516+0000 7f4fcee00700 1 -- 172.21.15.18:0/3610379153 shutdown_connections 2024-04-12T21:20:09.511 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:09.516+0000 7f4fcee00700 1 -- 172.21.15.18:0/3610379153 wait complete. 2024-04-12T21:20:09.512 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:09.517+0000 7f4fcee00700 1 Processor -- start 2024-04-12T21:20:09.512 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:09.517+0000 7f4fcee00700 1 -- start start 2024-04-12T21:20:09.512 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:09.517+0000 7f4fcee00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4fc8101d10 0x7f4fc8115c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:20:09.513 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:09.517+0000 7f4fcee00700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4fc8102620 0x7f4fc8116190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:20:09.513 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:09.517+0000 7f4fcee00700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fc8116870 con 0x7f4fc8101d10 2024-04-12T21:20:09.513 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:09.517+0000 7f4fcee00700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fc810fdc0 con 0x7f4fc8102620 2024-04-12T21:20:09.513 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:09.518+0000 7f4fc7e00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4fc8101d10 0x7f4fc8115c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:20:09.513 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:09.518+0000 7f4fc7e00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4fc8101d10 0x7f4fc8115c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:38850/0 (socket says 172.21.15.18:38850) 2024-04-12T21:20:09.513 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:09.518+0000 7f4fc7e00700 1 -- 172.21.15.18:0/1894038056 learned_addr learned my addr 172.21.15.18:0/1894038056 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:20:09.514 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:09.518+0000 7f4fc7e00700 1 -- 172.21.15.18:0/1894038056 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4fc8102620 msgr2=0x7f4fc8116190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:20:09.514 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:09.518+0000 7f4fc7400700 1 --2- 172.21.15.18:0/1894038056 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4fc8102620 0x7f4fc8116190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:20:09.514 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:09.518+0000 7f4fc7e00700 1 --2- 172.21.15.18:0/1894038056 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4fc8102620 0x7f4fc8116190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:20:09.514 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:09.518+0000 7f4fc7e00700 1 -- 172.21.15.18:0/1894038056 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4fb8005040 con 0x7f4fc8101d10 2024-04-12T21:20:09.514 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:09.518+0000 7f4fc7400700 1 --2- 172.21.15.18:0/1894038056 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4fc8102620 0x7f4fc8116190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-12T21:20:09.514 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:09.518+0000 7f4fc7e00700 1 --2- 172.21.15.18:0/1894038056 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4fc8101d10 0x7f4fc8115c50 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f4fb8009100 tx=0x7f4fb8009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:20:09.514 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:09.519+0000 7f4fbfe00700 1 -- 172.21.15.18:0/1894038056 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4fb800e040 con 0x7f4fc8101d10 2024-04-12T21:20:09.515 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:09.519+0000 7f4fcee00700 1 -- 172.21.15.18:0/1894038056 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4fc810ffe0 con 0x7f4fc8101d10 2024-04-12T21:20:09.515 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:09.519+0000 7f4fbfe00700 1 -- 172.21.15.18:0/1894038056 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f4fb800bbc0 con 0x7f4fc8101d10 2024-04-12T21:20:09.515 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:09.519+0000 7f4fbfe00700 1 -- 172.21.15.18:0/1894038056 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4fb801c630 con 0x7f4fc8101d10 2024-04-12T21:20:09.515 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:09.519+0000 7f4fcee00700 1 -- 172.21.15.18:0/1894038056 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4fc8110520 con 0x7f4fc8101d10 2024-04-12T21:20:09.515 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:09.520+0000 7f4fcee00700 1 -- 172.21.15.18:0/1894038056 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4fc80fb310 con 0x7f4fc8101d10 2024-04-12T21:20:09.516 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:09.521+0000 7f4fbfe00700 1 -- 172.21.15.18:0/1894038056 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7f4fb800bd30 con 0x7f4fc8101d10 2024-04-12T21:20:09.517 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:09.522+0000 7f4fbfe00700 1 --2- 172.21.15.18:0/1894038056 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f4fac06ec00 0x7f4fac0710b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:20:09.517 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:09.522+0000 7f4fbfe00700 1 -- 172.21.15.18:0/1894038056 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 6757+0+0 (secure 0 0 0) 0x7f4fb808fcc0 con 0x7f4fc8101d10 2024-04-12T21:20:09.517 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:09.522+0000 7f4fc7400700 1 --2- 172.21.15.18:0/1894038056 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f4fac06ec00 0x7f4fac0710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:20:09.517 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:09.523+0000 7f4fc7400700 1 --2- 172.21.15.18:0/1894038056 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f4fac06ec00 0x7f4fac0710b0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f4fb4006fd0 tx=0x7f4fb4008040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:20:09.521 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:09.527+0000 7f4fbfe00700 1 -- 172.21.15.18:0/1894038056 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f4fb8095020 con 0x7f4fc8101d10 2024-04-12T21:20:09.599 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:20:09 smithi018 ceph-mon[112590]: pgmap v610: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:20:09.599 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:20:09 smithi018 ceph-mon[112590]: from='client.35088 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:20:09.778 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:20:09 smithi186 ceph-mon[86379]: pgmap v610: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:20:09.778 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:20:09 smithi186 ceph-mon[86379]: from='client.35088 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:20:09.833 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:09.838+0000 7f4fcee00700 1 -- 172.21.15.18:0/1894038056 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f4fc8111330 con 0x7f4fc8101d10 2024-04-12T21:20:09.834 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:09.839+0000 7f4fbfe00700 1 -- 172.21.15.18:0/1894038056 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+837 (secure 0 0 0) 0x7f4fb805c470 con 0x7f4fc8101d10 2024-04-12T21:20:09.838 INFO:teuthology.orchestra.run.smithi018.stdout:{ 2024-04-12T21:20:09.838 INFO:teuthology.orchestra.run.smithi018.stdout: "mon": { 2024-04-12T21:20:09.838 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 19.0.0-2879-g324baf8d (324baf8ddb91dea46cac50cb535c306115791355) squid (dev)": 2 2024-04-12T21:20:09.838 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-12T21:20:09.838 INFO:teuthology.orchestra.run.smithi018.stdout: "mgr": { 2024-04-12T21:20:09.838 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 19.0.0-2879-g324baf8d (324baf8ddb91dea46cac50cb535c306115791355) squid (dev)": 2 2024-04-12T21:20:09.838 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-12T21:20:09.838 INFO:teuthology.orchestra.run.smithi018.stdout: "osd": { 2024-04-12T21:20:09.839 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 19.0.0-2879-g324baf8d (324baf8ddb91dea46cac50cb535c306115791355) squid (dev)": 6 2024-04-12T21:20:09.839 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-12T21:20:09.839 INFO:teuthology.orchestra.run.smithi018.stdout: "mds": { 2024-04-12T21:20:09.839 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 17.2.7-827-gd3b5060a (d3b5060a17449edfae9385cdf792afec109fca5d) quincy (stable)": 2, 2024-04-12T21:20:09.839 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 19.0.0-2879-g324baf8d (324baf8ddb91dea46cac50cb535c306115791355) squid (dev)": 2 2024-04-12T21:20:09.839 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-12T21:20:09.839 INFO:teuthology.orchestra.run.smithi018.stdout: "overall": { 2024-04-12T21:20:09.839 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 17.2.7-827-gd3b5060a (d3b5060a17449edfae9385cdf792afec109fca5d) quincy (stable)": 2, 2024-04-12T21:20:09.839 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 19.0.0-2879-g324baf8d (324baf8ddb91dea46cac50cb535c306115791355) squid (dev)": 12 2024-04-12T21:20:09.839 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-04-12T21:20:09.839 INFO:teuthology.orchestra.run.smithi018.stdout:} 2024-04-12T21:20:09.840 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:09.841+0000 7f4fcee00700 1 -- 172.21.15.18:0/1894038056 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f4fac06ec00 msgr2=0x7f4fac0710b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:20:09.840 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:09.841+0000 7f4fcee00700 1 --2- 172.21.15.18:0/1894038056 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f4fac06ec00 0x7f4fac0710b0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f4fb4006fd0 tx=0x7f4fb4008040 comp rx=0 tx=0).stop 2024-04-12T21:20:09.840 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:09.841+0000 7f4fcee00700 1 -- 172.21.15.18:0/1894038056 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4fc8101d10 msgr2=0x7f4fc8115c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:20:09.840 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:09.841+0000 7f4fcee00700 1 --2- 172.21.15.18:0/1894038056 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4fc8101d10 0x7f4fc8115c50 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f4fb8009100 tx=0x7f4fb8009e30 comp rx=0 tx=0).stop 2024-04-12T21:20:09.840 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:09.842+0000 7f4fcee00700 1 -- 172.21.15.18:0/1894038056 shutdown_connections 2024-04-12T21:20:09.840 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:09.842+0000 7f4fcee00700 1 --2- 172.21.15.18:0/1894038056 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f4fac06ec00 0x7f4fac0710b0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:20:09.841 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:09.842+0000 7f4fcee00700 1 --2- 172.21.15.18:0/1894038056 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4fc8101d10 0x7f4fc8115c50 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:20:09.841 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:09.842+0000 7f4fcee00700 1 --2- 172.21.15.18:0/1894038056 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4fc8102620 0x7f4fc8116190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:20:09.841 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:09.842+0000 7f4fcee00700 1 -- 172.21.15.18:0/1894038056 >> 172.21.15.18:0/1894038056 conn(0x7f4fc80f6bd0 msgr2=0x7f4fc80f7b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:20:09.841 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:09.842+0000 7f4fcee00700 1 -- 172.21.15.18:0/1894038056 shutdown_connections 2024-04-12T21:20:09.841 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:09.842+0000 7f4fcee00700 1 -- 172.21.15.18:0/1894038056 wait complete. 2024-04-12T21:20:09.995 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:10.000+0000 7fd180400700 1 -- 172.21.15.18:0/2154154221 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd1781059b0 msgr2=0x7fd1780ff020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:20:09.996 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:10.000+0000 7fd180400700 1 --2- 172.21.15.18:0/2154154221 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd1781059b0 0x7fd1780ff020 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7fd164004940 tx=0x7fd164009e30 comp rx=0 tx=0).stop 2024-04-12T21:20:09.996 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:10.000+0000 7fd180400700 1 -- 172.21.15.18:0/2154154221 shutdown_connections 2024-04-12T21:20:09.996 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:10.000+0000 7fd180400700 1 --2- 172.21.15.18:0/2154154221 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd1781059b0 0x7fd1780ff020 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:20:09.996 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:10.000+0000 7fd180400700 1 --2- 172.21.15.18:0/2154154221 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd178105010 0x7fd1781053e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:20:09.996 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:10.000+0000 7fd180400700 1 -- 172.21.15.18:0/2154154221 >> 172.21.15.18:0/2154154221 conn(0x7fd1780faba0 msgr2=0x7fd1780fcfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:20:09.996 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:10.000+0000 7fd180400700 1 -- 172.21.15.18:0/2154154221 shutdown_connections 2024-04-12T21:20:09.996 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:10.000+0000 7fd180400700 1 -- 172.21.15.18:0/2154154221 wait complete. 2024-04-12T21:20:09.997 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:10.001+0000 7fd180400700 1 Processor -- start 2024-04-12T21:20:09.998 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:10.001+0000 7fd180400700 1 -- start start 2024-04-12T21:20:09.998 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:10.002+0000 7fd180400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd178105010 0x7fd178115d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:20:09.998 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:10.002+0000 7fd180400700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd1781059b0 0x7fd1781162b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:20:09.998 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:10.002+0000 7fd180400700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd178116900 con 0x7fd178105010 2024-04-12T21:20:09.998 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:10.002+0000 7fd180400700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd17810fcd0 con 0x7fd1781059b0 2024-04-12T21:20:09.998 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:10.002+0000 7fd17da00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd178105010 0x7fd178115d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:20:09.998 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:10.002+0000 7fd17da00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd178105010 0x7fd178115d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:38866/0 (socket says 172.21.15.18:38866) 2024-04-12T21:20:09.998 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:10.002+0000 7fd17da00700 1 -- 172.21.15.18:0/1186619539 learned_addr learned my addr 172.21.15.18:0/1186619539 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:20:09.998 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:10.002+0000 7fd17d000700 1 --2- 172.21.15.18:0/1186619539 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd1781059b0 0x7fd1781162b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:20:09.998 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:10.002+0000 7fd17da00700 1 -- 172.21.15.18:0/1186619539 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd1781059b0 msgr2=0x7fd1781162b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:20:09.998 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:10.002+0000 7fd17da00700 1 --2- 172.21.15.18:0/1186619539 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd1781059b0 0x7fd1781162b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:20:09.998 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:10.003+0000 7fd17da00700 1 -- 172.21.15.18:0/1186619539 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd164005040 con 0x7fd178105010 2024-04-12T21:20:09.999 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:10.003+0000 7fd17da00700 1 --2- 172.21.15.18:0/1186619539 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd178105010 0x7fd178115d70 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7fd16c001d80 tx=0x7fd16c009ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:20:09.999 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:10.003+0000 7fd175600700 1 -- 172.21.15.18:0/1186619539 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd16c007d10 con 0x7fd178105010 2024-04-12T21:20:09.999 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:10.003+0000 7fd175600700 1 -- 172.21.15.18:0/1186619539 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd16c007e70 con 0x7fd178105010 2024-04-12T21:20:10.000 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:10.003+0000 7fd175600700 1 -- 172.21.15.18:0/1186619539 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd16c014640 con 0x7fd178105010 2024-04-12T21:20:10.000 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:10.003+0000 7fd180400700 1 -- 172.21.15.18:0/1186619539 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd17810ffb0 con 0x7fd178105010 2024-04-12T21:20:10.000 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:10.003+0000 7fd180400700 1 -- 172.21.15.18:0/1186619539 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd178110570 con 0x7fd178105010 2024-04-12T21:20:10.000 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:10.005+0000 7fd180400700 1 -- 172.21.15.18:0/1186619539 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd178100760 con 0x7fd178105010 2024-04-12T21:20:10.000 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:10.006+0000 7fd175600700 1 -- 172.21.15.18:0/1186619539 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7fd16c00e960 con 0x7fd178105010 2024-04-12T21:20:10.002 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:10.007+0000 7fd175600700 1 --2- 172.21.15.18:0/1186619539 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fd16006ec10 0x7fd1600710c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:20:10.002 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:10.007+0000 7fd175600700 1 -- 172.21.15.18:0/1186619539 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 6757+0+0 (secure 0 0 0) 0x7fd16c08e6e0 con 0x7fd178105010 2024-04-12T21:20:10.002 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:10.007+0000 7fd17d000700 1 --2- 172.21.15.18:0/1186619539 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fd16006ec10 0x7fd1600710c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:20:10.002 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:10.008+0000 7fd17d000700 1 --2- 172.21.15.18:0/1186619539 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fd16006ec10 0x7fd1600710c0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fd16400b3b0 tx=0x7fd164009e30 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:20:10.007 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:10.012+0000 7fd175600700 1 -- 172.21.15.18:0/1186619539 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7fd16c05afe0 con 0x7fd178105010 2024-04-12T21:20:10.293 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:10.298+0000 7fd180400700 1 -- 172.21.15.18:0/1186619539 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fd17804bed0 con 0x7fd178105010 2024-04-12T21:20:10.294 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:10.299+0000 7fd175600700 1 -- 172.21.15.18:0/1186619539 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 29 v29) v1 ==== 76+0+1872 (secure 0 0 0) 0x7fd16c05ab70 con 0x7fd178105010 2024-04-12T21:20:10.295 INFO:teuthology.orchestra.run.smithi018.stdout:e29 2024-04-12T21:20:10.295 INFO:teuthology.orchestra.run.smithi018.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-12T21:20:10.296 INFO:teuthology.orchestra.run.smithi018.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-12T21:20:10.296 INFO:teuthology.orchestra.run.smithi018.stdout:legacy client fscid: 1 2024-04-12T21:20:10.296 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-12T21:20:10.296 INFO:teuthology.orchestra.run.smithi018.stdout:Filesystem 'cephfs' (1) 2024-04-12T21:20:10.296 INFO:teuthology.orchestra.run.smithi018.stdout:fs_name cephfs 2024-04-12T21:20:10.296 INFO:teuthology.orchestra.run.smithi018.stdout:epoch 24 2024-04-12T21:20:10.296 INFO:teuthology.orchestra.run.smithi018.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-04-12T21:20:10.296 INFO:teuthology.orchestra.run.smithi018.stdout:created 2024-04-12T20:51:57.362850+0000 2024-04-12T21:20:10.296 INFO:teuthology.orchestra.run.smithi018.stdout:modified 2024-04-12T21:06:59.717190+0000 2024-04-12T21:20:10.296 INFO:teuthology.orchestra.run.smithi018.stdout:tableserver 0 2024-04-12T21:20:10.296 INFO:teuthology.orchestra.run.smithi018.stdout:root 0 2024-04-12T21:20:10.296 INFO:teuthology.orchestra.run.smithi018.stdout:session_timeout 60 2024-04-12T21:20:10.296 INFO:teuthology.orchestra.run.smithi018.stdout:session_autoclose 300 2024-04-12T21:20:10.296 INFO:teuthology.orchestra.run.smithi018.stdout:max_file_size 1099511627776 2024-04-12T21:20:10.296 INFO:teuthology.orchestra.run.smithi018.stdout:max_xattr_size 65536 2024-04-12T21:20:10.297 INFO:teuthology.orchestra.run.smithi018.stdout:required_client_features {} 2024-04-12T21:20:10.297 INFO:teuthology.orchestra.run.smithi018.stdout:last_failure 0 2024-04-12T21:20:10.297 INFO:teuthology.orchestra.run.smithi018.stdout:last_failure_osd_epoch 100 2024-04-12T21:20:10.297 INFO:teuthology.orchestra.run.smithi018.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-12T21:20:10.297 INFO:teuthology.orchestra.run.smithi018.stdout:max_mds 2 2024-04-12T21:20:10.297 INFO:teuthology.orchestra.run.smithi018.stdout:in 0,1 2024-04-12T21:20:10.297 INFO:teuthology.orchestra.run.smithi018.stdout:up {} 2024-04-12T21:20:10.297 INFO:teuthology.orchestra.run.smithi018.stdout:failed 0,1 2024-04-12T21:20:10.297 INFO:teuthology.orchestra.run.smithi018.stdout:damaged 2024-04-12T21:20:10.297 INFO:teuthology.orchestra.run.smithi018.stdout:stopped 2024-04-12T21:20:10.297 INFO:teuthology.orchestra.run.smithi018.stdout:data_pools [3] 2024-04-12T21:20:10.298 INFO:teuthology.orchestra.run.smithi018.stdout:metadata_pool 2 2024-04-12T21:20:10.298 INFO:teuthology.orchestra.run.smithi018.stdout:inline_data disabled 2024-04-12T21:20:10.298 INFO:teuthology.orchestra.run.smithi018.stdout:balancer 2024-04-12T21:20:10.298 INFO:teuthology.orchestra.run.smithi018.stdout:bal_rank_mask -1 2024-04-12T21:20:10.298 INFO:teuthology.orchestra.run.smithi018.stdout:standby_count_wanted 1 2024-04-12T21:20:10.298 INFO:teuthology.orchestra.run.smithi018.stdout:qdb_cluster leader: 0 members: 2024-04-12T21:20:10.298 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-12T21:20:10.298 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-12T21:20:10.298 INFO:teuthology.orchestra.run.smithi018.stdout:Standby daemons: 2024-04-12T21:20:10.298 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-12T21:20:10.299 INFO:teuthology.orchestra.run.smithi018.stdout:[mds.cephfs.smithi186.wjnafs{-1:24277} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.186:6826/438347207,v1:172.21.15.186:6827/438347207] compat {c=[1],r=[1],i=[7ff]}] 2024-04-12T21:20:10.299 INFO:teuthology.orchestra.run.smithi018.stdout:[mds.cephfs.smithi186.jdllyn{-1:24281} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.186:6824/1464359952,v1:172.21.15.186:6825/1464359952] compat {c=[1],r=[1],i=[7ff]}] 2024-04-12T21:20:10.299 INFO:teuthology.orchestra.run.smithi018.stdout:[mds.cephfs.smithi018.acqbsc{-1:34436} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.18:6828/755179764,v1:172.21.15.18:6829/755179764] compat {c=[1],r=[1],i=[1fff]}] 2024-04-12T21:20:10.299 INFO:teuthology.orchestra.run.smithi018.stdout:[mds.cephfs.smithi018.szjluo{-1:34468} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.18:6826/2407992653,v1:172.21.15.18:6827/2407992653] compat {c=[1],r=[1],i=[1fff]}] 2024-04-12T21:20:10.299 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:10.303+0000 7fd180400700 1 -- 172.21.15.18:0/1186619539 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fd16006ec10 msgr2=0x7fd1600710c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:20:10.300 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:10.303+0000 7fd180400700 1 --2- 172.21.15.18:0/1186619539 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fd16006ec10 0x7fd1600710c0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fd16400b3b0 tx=0x7fd164009e30 comp rx=0 tx=0).stop 2024-04-12T21:20:10.300 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:10.303+0000 7fd180400700 1 -- 172.21.15.18:0/1186619539 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd178105010 msgr2=0x7fd178115d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:20:10.300 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:10.303+0000 7fd180400700 1 --2- 172.21.15.18:0/1186619539 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd178105010 0x7fd178115d70 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7fd16c001d80 tx=0x7fd16c009ec0 comp rx=0 tx=0).stop 2024-04-12T21:20:10.300 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:10.303+0000 7fd180400700 1 -- 172.21.15.18:0/1186619539 shutdown_connections 2024-04-12T21:20:10.300 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:10.303+0000 7fd180400700 1 --2- 172.21.15.18:0/1186619539 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fd16006ec10 0x7fd1600710c0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:20:10.300 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:10.303+0000 7fd180400700 1 --2- 172.21.15.18:0/1186619539 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd178105010 0x7fd178115d70 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:20:10.300 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:10.303+0000 7fd180400700 1 --2- 172.21.15.18:0/1186619539 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd1781059b0 0x7fd1781162b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:20:10.300 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:10.303+0000 7fd180400700 1 -- 172.21.15.18:0/1186619539 >> 172.21.15.18:0/1186619539 conn(0x7fd1780faba0 msgr2=0x7fd1780fbcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:20:10.300 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:10.303+0000 7fd180400700 1 -- 172.21.15.18:0/1186619539 shutdown_connections 2024-04-12T21:20:10.300 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:10.304+0000 7fd180400700 1 -- 172.21.15.18:0/1186619539 wait complete. 2024-04-12T21:20:10.301 INFO:teuthology.orchestra.run.smithi018.stderr:dumped fsmap epoch 29 2024-04-12T21:20:10.457 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:10.461+0000 7f50cc000700 1 -- 172.21.15.18:0/3772875643 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f50c4070360 msgr2=0x7f50c4070730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:20:10.457 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:10.461+0000 7f50cc000700 1 --2- 172.21.15.18:0/3772875643 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f50c4070360 0x7f50c4070730 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f50b8009d10 tx=0x7f50b8004d10 comp rx=0 tx=0).stop 2024-04-12T21:20:10.457 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:10.461+0000 7f50cc000700 1 -- 172.21.15.18:0/3772875643 shutdown_connections 2024-04-12T21:20:10.457 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:10.461+0000 7f50cc000700 1 --2- 172.21.15.18:0/3772875643 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f50c4070c70 0x7f50c4109110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:20:10.457 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:10.461+0000 7f50cc000700 1 --2- 172.21.15.18:0/3772875643 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f50c4070360 0x7f50c4070730 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:20:10.457 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:10.461+0000 7f50cc000700 1 -- 172.21.15.18:0/3772875643 >> 172.21.15.18:0/3772875643 conn(0x7f50c40f89b0 msgr2=0x7f50c40fadc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:20:10.457 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:10.461+0000 7f50cc000700 1 -- 172.21.15.18:0/3772875643 shutdown_connections 2024-04-12T21:20:10.457 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:10.462+0000 7f50cc000700 1 -- 172.21.15.18:0/3772875643 wait complete. 2024-04-12T21:20:10.459 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:10.462+0000 7f50cc000700 1 Processor -- start 2024-04-12T21:20:10.459 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:10.462+0000 7f50cc000700 1 -- start start 2024-04-12T21:20:10.459 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:10.463+0000 7f50cc000700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f50c4070360 0x7f50c4115fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:20:10.459 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:10.463+0000 7f50cc000700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f50c4070c70 0x7f50c41164e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:20:10.459 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:10.463+0000 7f50cc000700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50c4116b30 con 0x7f50c4070360 2024-04-12T21:20:10.460 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:10.463+0000 7f50cc000700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50c410ff00 con 0x7f50c4070c70 2024-04-12T21:20:10.460 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:10.463+0000 7f50c9600700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f50c4070360 0x7f50c4115fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:20:10.460 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:10.463+0000 7f50c9600700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f50c4070360 0x7f50c4115fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:38884/0 (socket says 172.21.15.18:38884) 2024-04-12T21:20:10.460 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:10.463+0000 7f50c9600700 1 -- 172.21.15.18:0/199929076 learned_addr learned my addr 172.21.15.18:0/199929076 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:20:10.460 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:10.463+0000 7f50c8c00700 1 --2- 172.21.15.18:0/199929076 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f50c4070c70 0x7f50c41164e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:20:10.460 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:10.463+0000 7f50c9600700 1 -- 172.21.15.18:0/199929076 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f50c4070c70 msgr2=0x7f50c41164e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:20:10.460 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:10.463+0000 7f50c9600700 1 --2- 172.21.15.18:0/199929076 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f50c4070c70 0x7f50c41164e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:20:10.460 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:10.463+0000 7f50c9600700 1 -- 172.21.15.18:0/199929076 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50b8009940 con 0x7f50c4070360 2024-04-12T21:20:10.460 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:10.464+0000 7f50c9600700 1 --2- 172.21.15.18:0/199929076 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f50c4070360 0x7f50c4115fa0 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f50b8008df0 tx=0x7f50b8004d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:20:10.460 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:10.464+0000 7f50c1600700 1 -- 172.21.15.18:0/199929076 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f50b800e040 con 0x7f50c4070360 2024-04-12T21:20:10.460 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:10.464+0000 7f50c1600700 1 -- 172.21.15.18:0/199929076 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f50b8004610 con 0x7f50c4070360 2024-04-12T21:20:10.460 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:10.464+0000 7f50cc000700 1 -- 172.21.15.18:0/199929076 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f50c4110180 con 0x7f50c4070360 2024-04-12T21:20:10.460 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:10.464+0000 7f50c1600700 1 -- 172.21.15.18:0/199929076 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f50b801c8a0 con 0x7f50c4070360 2024-04-12T21:20:10.460 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:10.464+0000 7f50cc000700 1 -- 172.21.15.18:0/199929076 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f50c41106c0 con 0x7f50c4070360 2024-04-12T21:20:10.461 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:10.465+0000 7f50cc000700 1 -- 172.21.15.18:0/199929076 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f50c4071aa0 con 0x7f50c4070360 2024-04-12T21:20:10.461 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:10.466+0000 7f50c1600700 1 -- 172.21.15.18:0/199929076 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7f50b8012410 con 0x7f50c4070360 2024-04-12T21:20:10.462 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:10.467+0000 7f50c1600700 1 --2- 172.21.15.18:0/199929076 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f50ac06ec10 0x7f50ac0710c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:20:10.462 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:10.467+0000 7f50c1600700 1 -- 172.21.15.18:0/199929076 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 6757+0+0 (secure 0 0 0) 0x7f50b8090690 con 0x7f50c4070360 2024-04-12T21:20:10.462 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:10.468+0000 7f50c8c00700 1 --2- 172.21.15.18:0/199929076 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f50ac06ec10 0x7f50ac0710c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:20:10.463 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:10.468+0000 7f50c8c00700 1 --2- 172.21.15.18:0/199929076 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f50ac06ec10 0x7f50ac0710c0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f50c4111840 tx=0x7f50b000b040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:20:10.467 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:10.472+0000 7f50c1600700 1 -- 172.21.15.18:0/199929076 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f50b805ce40 con 0x7f50c4070360 2024-04-12T21:20:10.599 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:20:10 smithi018 ceph-mon[112590]: from='client.35092 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:20:10.600 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:20:10 smithi018 ceph-mon[112590]: from='client.35096 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:20:10.600 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:20:10 smithi018 ceph-mon[112590]: from='client.? 172.21.15.18:0/1894038056' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T21:20:10.600 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:20:10 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T21:20:10.600 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:20:10 smithi018 ceph-mon[112590]: from='client.? 172.21.15.18:0/1186619539' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-12T21:20:10.713 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:10.718+0000 7f50cc000700 1 -- 172.21.15.18:0/199929076 --> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f50c404bed0 con 0x7f50ac06ec10 2024-04-12T21:20:10.715 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:10.720+0000 7f50c1600700 1 -- 172.21.15.18:0/199929076 <== mgr.34102 v2:172.21.15.18:6800/3812891919 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f50c404bed0 con 0x7f50ac06ec10 2024-04-12T21:20:10.716 INFO:teuthology.orchestra.run.smithi018.stdout:{ 2024-04-12T21:20:10.716 INFO:teuthology.orchestra.run.smithi018.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:f4452def3be4d350f4a8e5feaf7ebb1511be9298fe68478ba7a2d6fbd7d69a63", 2024-04-12T21:20:10.716 INFO:teuthology.orchestra.run.smithi018.stdout: "in_progress": true, 2024-04-12T21:20:10.716 INFO:teuthology.orchestra.run.smithi018.stdout: "which": "Upgrading all daemon types on all hosts", 2024-04-12T21:20:10.716 INFO:teuthology.orchestra.run.smithi018.stdout: "services_complete": [ 2024-04-12T21:20:10.716 INFO:teuthology.orchestra.run.smithi018.stdout: "crash", 2024-04-12T21:20:10.716 INFO:teuthology.orchestra.run.smithi018.stdout: "osd", 2024-04-12T21:20:10.716 INFO:teuthology.orchestra.run.smithi018.stdout: "mon", 2024-04-12T21:20:10.716 INFO:teuthology.orchestra.run.smithi018.stdout: "mgr" 2024-04-12T21:20:10.716 INFO:teuthology.orchestra.run.smithi018.stdout: ], 2024-04-12T21:20:10.716 INFO:teuthology.orchestra.run.smithi018.stdout: "progress": "14/21 daemons upgraded", 2024-04-12T21:20:10.716 INFO:teuthology.orchestra.run.smithi018.stdout: "message": "Currently upgrading mds daemons", 2024-04-12T21:20:10.716 INFO:teuthology.orchestra.run.smithi018.stdout: "is_paused": false 2024-04-12T21:20:10.716 INFO:teuthology.orchestra.run.smithi018.stdout:} 2024-04-12T21:20:10.718 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:10.723+0000 7f50cc000700 1 -- 172.21.15.18:0/199929076 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f50ac06ec10 msgr2=0x7f50ac0710c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:20:10.719 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:10.723+0000 7f50cc000700 1 --2- 172.21.15.18:0/199929076 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f50ac06ec10 0x7f50ac0710c0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f50c4111840 tx=0x7f50b000b040 comp rx=0 tx=0).stop 2024-04-12T21:20:10.719 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:10.723+0000 7f50cc000700 1 -- 172.21.15.18:0/199929076 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f50c4070360 msgr2=0x7f50c4115fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:20:10.719 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:10.723+0000 7f50cc000700 1 --2- 172.21.15.18:0/199929076 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f50c4070360 0x7f50c4115fa0 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f50b8008df0 tx=0x7f50b8004d10 comp rx=0 tx=0).stop 2024-04-12T21:20:10.719 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:10.723+0000 7f50cc000700 1 -- 172.21.15.18:0/199929076 shutdown_connections 2024-04-12T21:20:10.719 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:10.723+0000 7f50cc000700 1 --2- 172.21.15.18:0/199929076 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f50ac06ec10 0x7f50ac0710c0 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:20:10.719 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:10.723+0000 7f50cc000700 1 --2- 172.21.15.18:0/199929076 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f50c4070360 0x7f50c4115fa0 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:20:10.719 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:10.723+0000 7f50cc000700 1 --2- 172.21.15.18:0/199929076 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f50c4070c70 0x7f50c41164e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:20:10.719 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:10.723+0000 7f50cc000700 1 -- 172.21.15.18:0/199929076 >> 172.21.15.18:0/199929076 conn(0x7f50c40f89b0 msgr2=0x7f50c40f9f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:20:10.719 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:10.724+0000 7f50cc000700 1 -- 172.21.15.18:0/199929076 shutdown_connections 2024-04-12T21:20:10.719 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:10.724+0000 7f50cc000700 1 -- 172.21.15.18:0/199929076 wait complete. 2024-04-12T21:20:10.778 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:20:10 smithi186 ceph-mon[86379]: from='client.35092 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:20:10.779 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:20:10 smithi186 ceph-mon[86379]: from='client.35096 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:20:10.779 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:20:10 smithi186 ceph-mon[86379]: from='client.? 172.21.15.18:0/1894038056' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T21:20:10.779 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:20:10 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T21:20:10.779 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:20:10 smithi186 ceph-mon[86379]: from='client.? 172.21.15.18:0/1186619539' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-12T21:20:10.876 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:10.880+0000 7f5595800700 1 -- 172.21.15.18:0/3734829611 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5590109430 msgr2=0x7f5590109800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:20:10.876 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:10.880+0000 7f5595800700 1 --2- 172.21.15.18:0/3734829611 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5590109430 0x7f5590109800 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f5580004940 tx=0x7f5580009e30 comp rx=0 tx=0).stop 2024-04-12T21:20:10.876 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:10.881+0000 7f5595800700 1 -- 172.21.15.18:0/3734829611 shutdown_connections 2024-04-12T21:20:10.876 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:10.881+0000 7f5595800700 1 --2- 172.21.15.18:0/3734829611 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5590109dd0 0x7f5590103440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:20:10.876 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:10.881+0000 7f5595800700 1 --2- 172.21.15.18:0/3734829611 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5590109430 0x7f5590109800 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:20:10.876 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:10.881+0000 7f5595800700 1 -- 172.21.15.18:0/3734829611 >> 172.21.15.18:0/3734829611 conn(0x7f5590072c00 msgr2=0x7f5590075010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:20:10.876 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:10.881+0000 7f5595800700 1 -- 172.21.15.18:0/3734829611 shutdown_connections 2024-04-12T21:20:10.877 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:10.881+0000 7f5595800700 1 -- 172.21.15.18:0/3734829611 wait complete. 2024-04-12T21:20:10.878 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:10.882+0000 7f5595800700 1 Processor -- start 2024-04-12T21:20:10.878 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:10.882+0000 7f5595800700 1 -- start start 2024-04-12T21:20:10.878 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:10.882+0000 7f5595800700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5590109430 0x7f5590119f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:20:10.878 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:10.882+0000 7f5595800700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5590109dd0 0x7f559011a4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:20:10.878 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:10.882+0000 7f5595800700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f559011ab20 con 0x7f5590109430 2024-04-12T21:20:10.878 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:10.882+0000 7f5595800700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5590114100 con 0x7f5590109dd0 2024-04-12T21:20:10.878 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:10.882+0000 7f558ea00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5590109430 0x7f5590119f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:20:10.878 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:10.882+0000 7f558ea00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5590109430 0x7f5590119f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:52616/0 (socket says 172.21.15.18:52616) 2024-04-12T21:20:10.879 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:10.882+0000 7f558ea00700 1 -- 172.21.15.18:0/2210992645 learned_addr learned my addr 172.21.15.18:0/2210992645 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:20:10.879 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:10.883+0000 7f558e000700 1 --2- 172.21.15.18:0/2210992645 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5590109dd0 0x7f559011a4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:20:10.879 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:10.883+0000 7f558ea00700 1 -- 172.21.15.18:0/2210992645 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5590109dd0 msgr2=0x7f559011a4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:20:10.879 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:10.883+0000 7f558ea00700 1 --2- 172.21.15.18:0/2210992645 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5590109dd0 0x7f559011a4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:20:10.879 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:10.883+0000 7f558ea00700 1 -- 172.21.15.18:0/2210992645 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5580005040 con 0x7f5590109430 2024-04-12T21:20:10.879 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:10.883+0000 7f558ea00700 1 --2- 172.21.15.18:0/2210992645 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5590109430 0x7f5590119f90 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f558000b8b0 tx=0x7f5580009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:20:10.880 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:10.883+0000 7f5586a00700 1 -- 172.21.15.18:0/2210992645 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f558000e040 con 0x7f5590109430 2024-04-12T21:20:10.880 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:10.883+0000 7f5586a00700 1 -- 172.21.15.18:0/2210992645 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f5580004290 con 0x7f5590109430 2024-04-12T21:20:10.880 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:10.883+0000 7f5595800700 1 -- 172.21.15.18:0/2210992645 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5590114380 con 0x7f5590109430 2024-04-12T21:20:10.880 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:10.883+0000 7f5586a00700 1 -- 172.21.15.18:0/2210992645 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f558001c3f0 con 0x7f5590109430 2024-04-12T21:20:10.880 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:10.884+0000 7f5595800700 1 -- 172.21.15.18:0/2210992645 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5590114940 con 0x7f5590109430 2024-04-12T21:20:10.880 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:10.885+0000 7f5595800700 1 -- 172.21.15.18:0/2210992645 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5590104b80 con 0x7f5590109430 2024-04-12T21:20:10.880 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:10.886+0000 7f5586a00700 1 -- 172.21.15.18:0/2210992645 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7f55800137b0 con 0x7f5590109430 2024-04-12T21:20:10.882 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:10.887+0000 7f5586a00700 1 --2- 172.21.15.18:0/2210992645 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f557406ec00 0x7f55740710b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:20:10.882 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:10.887+0000 7f5586a00700 1 -- 172.21.15.18:0/2210992645 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 6757+0+0 (secure 0 0 0) 0x7f558008fd80 con 0x7f5590109430 2024-04-12T21:20:10.882 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:10.887+0000 7f558e000700 1 --2- 172.21.15.18:0/2210992645 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f557406ec00 0x7f55740710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:20:10.882 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:10.888+0000 7f558e000700 1 --2- 172.21.15.18:0/2210992645 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f557406ec00 0x7f55740710b0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f5590115830 tx=0x7f557c00b040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:20:10.886 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:10.892+0000 7f5586a00700 1 -- 172.21.15.18:0/2210992645 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f558005c530 con 0x7f5590109430 2024-04-12T21:20:11.210 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:11.215+0000 7f5595800700 1 -- 172.21.15.18:0/2210992645 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f55901155e0 con 0x7f5590109430 2024-04-12T21:20:11.210 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:11.215+0000 7f5586a00700 1 -- 172.21.15.18:0/2210992645 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+233 (secure 0 0 0) 0x7f558005c0c0 con 0x7f5590109430 2024-04-12T21:20:11.211 INFO:teuthology.orchestra.run.smithi018.stdout:HEALTH_ERR 1 filesystem is degraded; 1 filesystem is offline 2024-04-12T21:20:11.211 INFO:teuthology.orchestra.run.smithi018.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-04-12T21:20:11.211 INFO:teuthology.orchestra.run.smithi018.stdout: fs cephfs is degraded 2024-04-12T21:20:11.211 INFO:teuthology.orchestra.run.smithi018.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-04-12T21:20:11.211 INFO:teuthology.orchestra.run.smithi018.stdout: fs cephfs is offline because no MDS is active for it. 2024-04-12T21:20:11.214 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:11.218+0000 7f5595800700 1 -- 172.21.15.18:0/2210992645 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f557406ec00 msgr2=0x7f55740710b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:20:11.214 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:11.218+0000 7f5595800700 1 --2- 172.21.15.18:0/2210992645 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f557406ec00 0x7f55740710b0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f5590115830 tx=0x7f557c00b040 comp rx=0 tx=0).stop 2024-04-12T21:20:11.214 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:11.218+0000 7f5595800700 1 -- 172.21.15.18:0/2210992645 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5590109430 msgr2=0x7f5590119f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:20:11.214 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:11.218+0000 7f5595800700 1 --2- 172.21.15.18:0/2210992645 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5590109430 0x7f5590119f90 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f558000b8b0 tx=0x7f5580009e30 comp rx=0 tx=0).stop 2024-04-12T21:20:11.214 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:11.219+0000 7f5595800700 1 -- 172.21.15.18:0/2210992645 shutdown_connections 2024-04-12T21:20:11.214 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:11.219+0000 7f5595800700 1 --2- 172.21.15.18:0/2210992645 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f557406ec00 0x7f55740710b0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:20:11.214 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:11.219+0000 7f5595800700 1 --2- 172.21.15.18:0/2210992645 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5590109430 0x7f5590119f90 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:20:11.215 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:11.219+0000 7f5595800700 1 --2- 172.21.15.18:0/2210992645 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5590109dd0 0x7f559011a4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:20:11.215 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:11.219+0000 7f5595800700 1 -- 172.21.15.18:0/2210992645 >> 172.21.15.18:0/2210992645 conn(0x7f5590072c00 msgr2=0x7f5590074c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:20:11.215 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:11.219+0000 7f5595800700 1 -- 172.21.15.18:0/2210992645 shutdown_connections 2024-04-12T21:20:11.215 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:11.219+0000 7f5595800700 1 -- 172.21.15.18:0/2210992645 wait complete. 2024-04-12T21:20:11.778 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:20:11 smithi186 ceph-mon[86379]: pgmap v611: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:20:11.779 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:20:11 smithi186 ceph-mon[86379]: from='client.35108 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:20:11.779 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:20:11 smithi186 ceph-mon[86379]: from='client.? 172.21.15.18:0/2210992645' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-12T21:20:11.849 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:20:11 smithi018 ceph-mon[112590]: pgmap v611: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:20:11.850 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:20:11 smithi018 ceph-mon[112590]: from='client.35108 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:20:11.850 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:20:11 smithi018 ceph-mon[112590]: from='client.? 172.21.15.18:0/2210992645' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-12T21:20:13.849 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:20:13 smithi018 ceph-mon[112590]: pgmap v612: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:20:14.028 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:20:13 smithi186 ceph-mon[86379]: pgmap v612: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:20:15.849 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:20:15 smithi018 ceph-mon[112590]: pgmap v613: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:20:16.028 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:20:15 smithi186 ceph-mon[86379]: pgmap v613: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:20:17.849 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:20:17 smithi018 ceph-mon[112590]: pgmap v614: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:20:18.028 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:20:17 smithi186 ceph-mon[86379]: pgmap v614: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:20:19.849 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:20:19 smithi018 ceph-mon[112590]: pgmap v615: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:20:20.028 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:20:19 smithi186 ceph-mon[86379]: pgmap v615: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:20:21.849 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:20:21 smithi018 ceph-mon[112590]: pgmap v616: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:20:22.028 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:20:21 smithi186 ceph-mon[86379]: pgmap v616: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:20:23.849 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:20:23 smithi018 ceph-mon[112590]: pgmap v617: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:20:24.028 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:20:23 smithi186 ceph-mon[86379]: pgmap v617: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:20:25.849 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:20:25 smithi018 ceph-mon[112590]: pgmap v618: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:20:25.849 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:20:25 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T21:20:26.028 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:20:25 smithi186 ceph-mon[86379]: pgmap v618: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:20:26.028 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:20:25 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T21:20:27.849 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:20:27 smithi018 ceph-mon[112590]: pgmap v619: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:20:28.028 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:20:27 smithi186 ceph-mon[86379]: pgmap v619: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:20:29.849 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:20:29 smithi018 ceph-mon[112590]: pgmap v620: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:20:30.028 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:20:29 smithi186 ceph-mon[86379]: pgmap v620: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:20:31.849 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:20:31 smithi018 ceph-mon[112590]: pgmap v621: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:20:32.028 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:20:31 smithi186 ceph-mon[86379]: pgmap v621: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:20:34.028 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:20:33 smithi186 ceph-mon[86379]: pgmap v622: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:20:34.099 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:20:33 smithi018 ceph-mon[112590]: pgmap v622: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:20:35.849 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:20:35 smithi018 ceph-mon[112590]: pgmap v623: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:20:36.028 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:20:35 smithi186 ceph-mon[86379]: pgmap v623: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:20:38.028 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:20:37 smithi186 ceph-mon[86379]: pgmap v624: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:20:38.099 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:20:37 smithi018 ceph-mon[112590]: pgmap v624: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:20:40.028 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:20:39 smithi186 ceph-mon[86379]: pgmap v625: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:20:40.099 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:20:39 smithi018 ceph-mon[112590]: pgmap v625: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:20:41.001 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:20:40 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T21:20:41.028 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:20:40 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T21:20:41.388 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:41.392+0000 7f14a2e00700 1 -- 172.21.15.18:0/3933360202 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f149c104fc0 msgr2=0x7f149c105390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:20:41.388 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:41.392+0000 7f14a2e00700 1 --2- 172.21.15.18:0/3933360202 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f149c104fc0 0x7f149c105390 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f148c004940 tx=0x7f148c009e30 comp rx=0 tx=0).stop 2024-04-12T21:20:41.388 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:41.392+0000 7f14a2e00700 1 -- 172.21.15.18:0/3933360202 shutdown_connections 2024-04-12T21:20:41.389 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:41.392+0000 7f14a2e00700 1 --2- 172.21.15.18:0/3933360202 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f149c105960 0x7f149c0fefd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:20:41.389 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:41.392+0000 7f14a2e00700 1 --2- 172.21.15.18:0/3933360202 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f149c104fc0 0x7f149c105390 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:20:41.389 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:41.392+0000 7f14a2e00700 1 -- 172.21.15.18:0/3933360202 >> 172.21.15.18:0/3933360202 conn(0x7f149c0fab70 msgr2=0x7f149c0fcf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:20:41.389 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:41.392+0000 7f14a2e00700 1 -- 172.21.15.18:0/3933360202 shutdown_connections 2024-04-12T21:20:41.389 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:41.392+0000 7f14a2e00700 1 -- 172.21.15.18:0/3933360202 wait complete. 2024-04-12T21:20:41.390 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:41.393+0000 7f14a2e00700 1 Processor -- start 2024-04-12T21:20:41.390 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:41.393+0000 7f14a2e00700 1 -- start start 2024-04-12T21:20:41.390 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:41.393+0000 7f14a2e00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f149c104fc0 0x7f149c115d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:20:41.391 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:41.393+0000 7f14a2e00700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f149c105960 0x7f149c116280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:20:41.391 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:41.394+0000 7f14a2e00700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f149c116960 con 0x7f149c104fc0 2024-04-12T21:20:41.391 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:41.394+0000 7f14a2e00700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f149c10fca0 con 0x7f149c105960 2024-04-12T21:20:41.391 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:41.394+0000 7f149be00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f149c104fc0 0x7f149c115d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:20:41.391 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:41.394+0000 7f149be00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f149c104fc0 0x7f149c115d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:47148/0 (socket says 172.21.15.18:47148) 2024-04-12T21:20:41.391 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:41.394+0000 7f149be00700 1 -- 172.21.15.18:0/3577659365 learned_addr learned my addr 172.21.15.18:0/3577659365 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:20:41.391 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:41.394+0000 7f149b400700 1 --2- 172.21.15.18:0/3577659365 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f149c105960 0x7f149c116280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:20:41.391 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:41.394+0000 7f149be00700 1 -- 172.21.15.18:0/3577659365 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f149c105960 msgr2=0x7f149c116280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:20:41.391 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:41.394+0000 7f149be00700 1 --2- 172.21.15.18:0/3577659365 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f149c105960 0x7f149c116280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:20:41.391 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:41.394+0000 7f149be00700 1 -- 172.21.15.18:0/3577659365 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f148c005040 con 0x7f149c104fc0 2024-04-12T21:20:41.391 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:41.395+0000 7f149be00700 1 --2- 172.21.15.18:0/3577659365 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f149c104fc0 0x7f149c115d40 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f148c009180 tx=0x7f148c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:20:41.392 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:41.395+0000 7f1493e00700 1 -- 172.21.15.18:0/3577659365 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f148c00e040 con 0x7f149c104fc0 2024-04-12T21:20:41.392 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:41.395+0000 7f1493e00700 1 -- 172.21.15.18:0/3577659365 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f148c00bbc0 con 0x7f149c104fc0 2024-04-12T21:20:41.392 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:41.395+0000 7f14a2e00700 1 -- 172.21.15.18:0/3577659365 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f149c10fec0 con 0x7f149c104fc0 2024-04-12T21:20:41.392 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:41.395+0000 7f1493e00700 1 -- 172.21.15.18:0/3577659365 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f148c01c630 con 0x7f149c104fc0 2024-04-12T21:20:41.392 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:41.395+0000 7f14a2e00700 1 -- 172.21.15.18:0/3577659365 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f149c110480 con 0x7f149c104fc0 2024-04-12T21:20:41.392 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:41.396+0000 7f14a2e00700 1 -- 172.21.15.18:0/3577659365 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f149c100710 con 0x7f149c104fc0 2024-04-12T21:20:41.393 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:41.397+0000 7f1493e00700 1 -- 172.21.15.18:0/3577659365 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7f148c00bd30 con 0x7f149c104fc0 2024-04-12T21:20:41.394 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:41.398+0000 7f1493e00700 1 --2- 172.21.15.18:0/3577659365 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f148006ec00 0x7f14800710b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:20:41.394 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:41.399+0000 7f1493e00700 1 -- 172.21.15.18:0/3577659365 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 6757+0+0 (secure 0 0 0) 0x7f148c08fd90 con 0x7f149c104fc0 2024-04-12T21:20:41.395 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:41.399+0000 7f149b400700 1 --2- 172.21.15.18:0/3577659365 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f148006ec00 0x7f14800710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:20:41.395 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:41.400+0000 7f149b400700 1 --2- 172.21.15.18:0/3577659365 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f148006ec00 0x7f14800710b0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f1488006fd0 tx=0x7f1488008040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:20:41.400 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:41.404+0000 7f1493e00700 1 -- 172.21.15.18:0/3577659365 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f148c05c5e0 con 0x7f149c104fc0 2024-04-12T21:20:41.655 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:41.659+0000 7f14a2e00700 1 -- 172.21.15.18:0/3577659365 --> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f149c04bf40 con 0x7f148006ec00 2024-04-12T21:20:41.656 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:20:41 smithi018 ceph-mon[112590]: pgmap v626: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:20:41.658 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:41.662+0000 7f1493e00700 1 -- 172.21.15.18:0/3577659365 <== mgr.34102 v2:172.21.15.18:6800/3812891919 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f149c04bf40 con 0x7f148006ec00 2024-04-12T21:20:41.661 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:41.665+0000 7f14a2e00700 1 -- 172.21.15.18:0/3577659365 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f148006ec00 msgr2=0x7f14800710b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:20:41.661 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:41.665+0000 7f14a2e00700 1 --2- 172.21.15.18:0/3577659365 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f148006ec00 0x7f14800710b0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f1488006fd0 tx=0x7f1488008040 comp rx=0 tx=0).stop 2024-04-12T21:20:41.661 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:41.665+0000 7f14a2e00700 1 -- 172.21.15.18:0/3577659365 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f149c104fc0 msgr2=0x7f149c115d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:20:41.661 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:41.665+0000 7f14a2e00700 1 --2- 172.21.15.18:0/3577659365 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f149c104fc0 0x7f149c115d40 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f148c009180 tx=0x7f148c009e30 comp rx=0 tx=0).stop 2024-04-12T21:20:41.661 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:41.665+0000 7f14a2e00700 1 -- 172.21.15.18:0/3577659365 shutdown_connections 2024-04-12T21:20:41.661 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:41.665+0000 7f14a2e00700 1 --2- 172.21.15.18:0/3577659365 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f148006ec00 0x7f14800710b0 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:20:41.661 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:41.665+0000 7f14a2e00700 1 --2- 172.21.15.18:0/3577659365 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f149c104fc0 0x7f149c115d40 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:20:41.662 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:41.665+0000 7f14a2e00700 1 --2- 172.21.15.18:0/3577659365 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f149c105960 0x7f149c116280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:20:41.662 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:41.665+0000 7f14a2e00700 1 -- 172.21.15.18:0/3577659365 >> 172.21.15.18:0/3577659365 conn(0x7f149c0fab70 msgr2=0x7f149c0fbb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:20:41.662 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:41.666+0000 7f14a2e00700 1 -- 172.21.15.18:0/3577659365 shutdown_connections 2024-04-12T21:20:41.662 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:41.666+0000 7f14a2e00700 1 -- 172.21.15.18:0/3577659365 wait complete. 2024-04-12T21:20:41.679 INFO:teuthology.orchestra.run.smithi018.stdout:true 2024-04-12T21:20:41.826 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:41.829+0000 7f06a2c00700 1 -- 172.21.15.18:0/1719690418 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f069c071f90 msgr2=0x7f069c070360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:20:41.826 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:41.829+0000 7f06a2c00700 1 --2- 172.21.15.18:0/1719690418 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f069c071f90 0x7f069c070360 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f068c004970 tx=0x7f068c009e30 comp rx=0 tx=0).stop 2024-04-12T21:20:41.826 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:41.829+0000 7f06a2c00700 1 -- 172.21.15.18:0/1719690418 shutdown_connections 2024-04-12T21:20:41.826 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:41.829+0000 7f06a2c00700 1 --2- 172.21.15.18:0/1719690418 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f069c0708a0 0x7f069c070d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:20:41.826 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:41.829+0000 7f06a2c00700 1 --2- 172.21.15.18:0/1719690418 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f069c071f90 0x7f069c070360 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:20:41.826 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:41.829+0000 7f06a2c00700 1 -- 172.21.15.18:0/1719690418 >> 172.21.15.18:0/1719690418 conn(0x7f069c0f89b0 msgr2=0x7f069c0fadc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:20:41.826 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:41.830+0000 7f06a2c00700 1 -- 172.21.15.18:0/1719690418 shutdown_connections 2024-04-12T21:20:41.826 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:41.830+0000 7f06a2c00700 1 -- 172.21.15.18:0/1719690418 wait complete. 2024-04-12T21:20:41.828 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:41.830+0000 7f06a2c00700 1 Processor -- start 2024-04-12T21:20:41.828 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:41.831+0000 7f06a2c00700 1 -- start start 2024-04-12T21:20:41.828 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:41.831+0000 7f06a2c00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f069c0708a0 0x7f069c119ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:20:41.828 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:41.831+0000 7f06a2c00700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f069c071f90 0x7f069c11a420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:20:41.828 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:41.831+0000 7f06a2c00700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f069c11aa70 con 0x7f069c0708a0 2024-04-12T21:20:41.828 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:41.831+0000 7f06a2c00700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f069c114100 con 0x7f069c071f90 2024-04-12T21:20:41.828 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:41.831+0000 7f069be00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f069c0708a0 0x7f069c119ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:20:41.828 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:41.832+0000 7f069be00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f069c0708a0 0x7f069c119ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:47162/0 (socket says 172.21.15.18:47162) 2024-04-12T21:20:41.828 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:41.832+0000 7f069be00700 1 -- 172.21.15.18:0/2317916787 learned_addr learned my addr 172.21.15.18:0/2317916787 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:20:41.828 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:41.832+0000 7f069be00700 1 -- 172.21.15.18:0/2317916787 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f069c071f90 msgr2=0x7f069c11a420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:20:41.828 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:41.832+0000 7f069be00700 1 --2- 172.21.15.18:0/2317916787 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f069c071f90 0x7f069c11a420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:20:41.828 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:41.832+0000 7f069be00700 1 -- 172.21.15.18:0/2317916787 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f068c005040 con 0x7f069c0708a0 2024-04-12T21:20:41.829 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:41.832+0000 7f069be00700 1 --2- 172.21.15.18:0/2317916787 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f069c0708a0 0x7f069c119ee0 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f068c000c00 tx=0x7f068c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:20:41.829 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:41.832+0000 7f0693e00700 1 -- 172.21.15.18:0/2317916787 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f068c00e040 con 0x7f069c0708a0 2024-04-12T21:20:41.829 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:41.833+0000 7f0693e00700 1 -- 172.21.15.18:0/2317916787 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f068c00bce0 con 0x7f069c0708a0 2024-04-12T21:20:41.829 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:41.833+0000 7f06a2c00700 1 -- 172.21.15.18:0/2317916787 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f069c114380 con 0x7f069c0708a0 2024-04-12T21:20:41.829 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:41.833+0000 7f0693e00700 1 -- 172.21.15.18:0/2317916787 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f068c01c630 con 0x7f069c0708a0 2024-04-12T21:20:41.830 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:41.833+0000 7f06a2c00700 1 -- 172.21.15.18:0/2317916787 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f069c1148c0 con 0x7f069c0708a0 2024-04-12T21:20:41.830 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:41.834+0000 7f06a2c00700 1 -- 172.21.15.18:0/2317916787 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f069c071aa0 con 0x7f069c0708a0 2024-04-12T21:20:41.831 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:41.835+0000 7f0693e00700 1 -- 172.21.15.18:0/2317916787 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7f068c013980 con 0x7f069c0708a0 2024-04-12T21:20:41.832 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:41.836+0000 7f0693e00700 1 --2- 172.21.15.18:0/2317916787 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f068006ec00 0x7f06800710b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:20:41.832 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:41.836+0000 7f0693e00700 1 -- 172.21.15.18:0/2317916787 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 6757+0+0 (secure 0 0 0) 0x7f068c08fe10 con 0x7f069c0708a0 2024-04-12T21:20:41.832 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:41.836+0000 7f069b400700 1 --2- 172.21.15.18:0/2317916787 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f068006ec00 0x7f06800710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:20:41.833 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:41.837+0000 7f069b400700 1 --2- 172.21.15.18:0/2317916787 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f068006ec00 0x7f06800710b0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f069c10ce70 tx=0x7f0688009ce0 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:20:41.838 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:41.841+0000 7f0693e00700 1 -- 172.21.15.18:0/2317916787 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f068c058a90 con 0x7f069c0708a0 2024-04-12T21:20:42.028 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:20:41 smithi186 ceph-mon[86379]: pgmap v626: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:20:42.094 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:42.098+0000 7f06a2c00700 1 -- 172.21.15.18:0/2317916787 --> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f069c110540 con 0x7f068006ec00 2024-04-12T21:20:42.096 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:42.100+0000 7f0693e00700 1 -- 172.21.15.18:0/2317916787 <== mgr.34102 v2:172.21.15.18:6800/3812891919 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f069c110540 con 0x7f068006ec00 2024-04-12T21:20:42.099 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:42.103+0000 7f06a2c00700 1 -- 172.21.15.18:0/2317916787 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f068006ec00 msgr2=0x7f06800710b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:20:42.099 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:42.103+0000 7f06a2c00700 1 --2- 172.21.15.18:0/2317916787 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f068006ec00 0x7f06800710b0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f069c10ce70 tx=0x7f0688009ce0 comp rx=0 tx=0).stop 2024-04-12T21:20:42.099 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:42.103+0000 7f06a2c00700 1 -- 172.21.15.18:0/2317916787 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f069c0708a0 msgr2=0x7f069c119ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:20:42.099 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:42.103+0000 7f06a2c00700 1 --2- 172.21.15.18:0/2317916787 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f069c0708a0 0x7f069c119ee0 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f068c000c00 tx=0x7f068c009e30 comp rx=0 tx=0).stop 2024-04-12T21:20:42.099 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:42.103+0000 7f06a2c00700 1 -- 172.21.15.18:0/2317916787 shutdown_connections 2024-04-12T21:20:42.099 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:42.103+0000 7f06a2c00700 1 --2- 172.21.15.18:0/2317916787 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f068006ec00 0x7f06800710b0 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:20:42.100 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:42.103+0000 7f06a2c00700 1 --2- 172.21.15.18:0/2317916787 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f069c0708a0 0x7f069c119ee0 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:20:42.100 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:42.103+0000 7f06a2c00700 1 --2- 172.21.15.18:0/2317916787 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f069c071f90 0x7f069c11a420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:20:42.100 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:42.103+0000 7f06a2c00700 1 -- 172.21.15.18:0/2317916787 >> 172.21.15.18:0/2317916787 conn(0x7f069c0f89b0 msgr2=0x7f069c0f9a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:20:42.100 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:42.103+0000 7f06a2c00700 1 -- 172.21.15.18:0/2317916787 shutdown_connections 2024-04-12T21:20:42.100 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:42.103+0000 7f06a2c00700 1 -- 172.21.15.18:0/2317916787 wait complete. 2024-04-12T21:20:42.255 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:42.258+0000 7f3dcf600700 1 -- 172.21.15.18:0/1497512076 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3dc8105020 msgr2=0x7f3dc81053f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:20:42.255 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:42.258+0000 7f3dcf600700 1 --2- 172.21.15.18:0/1497512076 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3dc8105020 0x7f3dc81053f0 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f3dbc004940 tx=0x7f3dbc009e30 comp rx=0 tx=0).stop 2024-04-12T21:20:42.255 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:42.259+0000 7f3dcf600700 1 -- 172.21.15.18:0/1497512076 shutdown_connections 2024-04-12T21:20:42.256 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:42.259+0000 7f3dcf600700 1 --2- 172.21.15.18:0/1497512076 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3dc81059c0 0x7f3dc80ff080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:20:42.256 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:42.259+0000 7f3dcf600700 1 --2- 172.21.15.18:0/1497512076 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3dc8105020 0x7f3dc81053f0 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:20:42.256 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:42.259+0000 7f3dcf600700 1 -- 172.21.15.18:0/1497512076 >> 172.21.15.18:0/1497512076 conn(0x7f3dc80faba0 msgr2=0x7f3dc80fcfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:20:42.256 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:42.259+0000 7f3dcf600700 1 -- 172.21.15.18:0/1497512076 shutdown_connections 2024-04-12T21:20:42.256 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:42.259+0000 7f3dcf600700 1 -- 172.21.15.18:0/1497512076 wait complete. 2024-04-12T21:20:42.256 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:42.260+0000 7f3dcf600700 1 Processor -- start 2024-04-12T21:20:42.257 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:42.260+0000 7f3dcf600700 1 -- start start 2024-04-12T21:20:42.257 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:42.260+0000 7f3dcf600700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3dc8105020 0x7f3dc8115b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:20:42.257 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:42.260+0000 7f3dcf600700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3dc81059c0 0x7f3dc81160d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:20:42.258 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:42.260+0000 7f3dcf600700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3dc8116720 con 0x7f3dc81059c0 2024-04-12T21:20:42.258 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:42.260+0000 7f3dcf600700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3dc810fd00 con 0x7f3dc8105020 2024-04-12T21:20:42.258 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:42.261+0000 7f3dc7e00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3dc81059c0 0x7f3dc81160d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:20:42.258 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:42.261+0000 7f3dc7e00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3dc81059c0 0x7f3dc81160d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:47180/0 (socket says 172.21.15.18:47180) 2024-04-12T21:20:42.258 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:42.261+0000 7f3dc7e00700 1 -- 172.21.15.18:0/2358921840 learned_addr learned my addr 172.21.15.18:0/2358921840 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:20:42.258 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:42.261+0000 7f3dccc00700 1 --2- 172.21.15.18:0/2358921840 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3dc8105020 0x7f3dc8115b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:20:42.258 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:42.261+0000 7f3dc7e00700 1 -- 172.21.15.18:0/2358921840 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3dc8105020 msgr2=0x7f3dc8115b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:20:42.258 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:42.261+0000 7f3dc7e00700 1 --2- 172.21.15.18:0/2358921840 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3dc8105020 0x7f3dc8115b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:20:42.258 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:42.261+0000 7f3dc7e00700 1 -- 172.21.15.18:0/2358921840 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3dbc005040 con 0x7f3dc81059c0 2024-04-12T21:20:42.258 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:42.261+0000 7f3dccc00700 1 --2- 172.21.15.18:0/2358921840 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3dc8105020 0x7f3dc8115b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-12T21:20:42.258 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:42.261+0000 7f3dc7e00700 1 --2- 172.21.15.18:0/2358921840 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3dc81059c0 0x7f3dc81160d0 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f3db4002a50 tx=0x7f3db4004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:20:42.258 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:42.262+0000 7f3dc4c00700 1 -- 172.21.15.18:0/2358921840 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3db4007cc0 con 0x7f3dc81059c0 2024-04-12T21:20:42.259 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:42.262+0000 7f3dc4c00700 1 -- 172.21.15.18:0/2358921840 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f3db400f040 con 0x7f3dc81059c0 2024-04-12T21:20:42.259 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:42.262+0000 7f3dcf600700 1 -- 172.21.15.18:0/2358921840 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3dc810ffe0 con 0x7f3dc81059c0 2024-04-12T21:20:42.259 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:42.262+0000 7f3dc4c00700 1 -- 172.21.15.18:0/2358921840 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3db400da40 con 0x7f3dc81059c0 2024-04-12T21:20:42.259 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:42.262+0000 7f3dcf600700 1 -- 172.21.15.18:0/2358921840 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3dc81105a0 con 0x7f3dc81059c0 2024-04-12T21:20:42.259 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:42.263+0000 7f3dcf600700 1 -- 172.21.15.18:0/2358921840 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3dc8100770 con 0x7f3dc81059c0 2024-04-12T21:20:42.260 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:42.264+0000 7f3dc4c00700 1 -- 172.21.15.18:0/2358921840 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7f3db40097c0 con 0x7f3dc81059c0 2024-04-12T21:20:42.261 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:42.265+0000 7f3dc4c00700 1 --2- 172.21.15.18:0/2358921840 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f3db006ec00 0x7f3db00710b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:20:42.262 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:42.266+0000 7f3dc4c00700 1 -- 172.21.15.18:0/2358921840 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 6757+0+0 (secure 0 0 0) 0x7f3db408e7b0 con 0x7f3dc81059c0 2024-04-12T21:20:42.262 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:42.266+0000 7f3dccc00700 1 --2- 172.21.15.18:0/2358921840 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f3db006ec00 0x7f3db00710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:20:42.262 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:42.267+0000 7f3dccc00700 1 --2- 172.21.15.18:0/2358921840 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f3db006ec00 0x7f3db00710b0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f3dbc004560 tx=0x7f3dbc009e30 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:20:42.266 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:42.270+0000 7f3dc4c00700 1 -- 172.21.15.18:0/2358921840 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f3db405acd0 con 0x7f3dc81059c0 2024-04-12T21:20:42.505 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:42.509+0000 7f3dcf600700 1 -- 172.21.15.18:0/2358921840 --> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f3dc810c190 con 0x7f3db006ec00 2024-04-12T21:20:42.535 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:42.539+0000 7f3dc4c00700 1 -- 172.21.15.18:0/2358921840 <== mgr.34102 v2:172.21.15.18:6800/3812891919 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7f3dc810c190 con 0x7f3db006ec00 2024-04-12T21:20:42.536 INFO:teuthology.orchestra.run.smithi018.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-12T21:20:42.536 INFO:teuthology.orchestra.run.smithi018.stdout:alertmanager.smithi018 smithi018 *:9093,9094 running (21m) 12m ago 35m 14.7M - 0.25.0 c8568f914cd2 c332090068e8 2024-04-12T21:20:42.536 INFO:teuthology.orchestra.run.smithi018.stdout:crash.smithi018 smithi018 running (17m) 12m ago 36m 6665k - 19.0.0-2879-g324baf8d 8717720dc5e5 b7f80d1f7982 2024-04-12T21:20:42.536 INFO:teuthology.orchestra.run.smithi018.stdout:crash.smithi186 smithi186 running (17m) 13m ago 34m 6660k - 19.0.0-2879-g324baf8d 8717720dc5e5 a4f55c81b623 2024-04-12T21:20:42.536 INFO:teuthology.orchestra.run.smithi018.stdout:grafana.smithi018 smithi018 *:3000 running (20m) 12m ago 35m 74.8M - 9.4.12 2bacad6d85d8 b08e3bf176dd 2024-04-12T21:20:42.536 INFO:teuthology.orchestra.run.smithi018.stdout:mds.cephfs.smithi018.acqbsc smithi018 running (13m) 12m ago 28m 11.1M - 19.0.0-2879-g324baf8d 8717720dc5e5 021f99ab24ae 2024-04-12T21:20:42.536 INFO:teuthology.orchestra.run.smithi018.stdout:mds.cephfs.smithi018.szjluo smithi018 running (13m) 12m ago 28m 10.9M - 19.0.0-2879-g324baf8d 8717720dc5e5 0003117479d4 2024-04-12T21:20:42.536 INFO:teuthology.orchestra.run.smithi018.stdout:mds.cephfs.smithi186.jdllyn smithi186 running (28m) 13m ago 28m 15.6M - 17.2.7-827-gd3b5060a 21de9b7369ad 2b2bc46bd29a 2024-04-12T21:20:42.536 INFO:teuthology.orchestra.run.smithi018.stdout:mds.cephfs.smithi186.wjnafs smithi186 running (28m) 13m ago 28m 15.4M - 17.2.7-827-gd3b5060a 21de9b7369ad 74b115d6d6fd 2024-04-12T21:20:42.537 INFO:teuthology.orchestra.run.smithi018.stdout:mgr.smithi018.xjzwga smithi018 *:8443,9283,8765 running (23m) 12m ago 37m 509M - 19.0.0-2879-g324baf8d 8717720dc5e5 56088305eab7 2024-04-12T21:20:42.537 INFO:teuthology.orchestra.run.smithi018.stdout:mgr.smithi186.kqcfxk smithi186 *:8443,9283,8765 running (22m) 13m ago 34m 452M - 19.0.0-2879-g324baf8d 8717720dc5e5 09cd766e5e0d 2024-04-12T21:20:42.537 INFO:teuthology.orchestra.run.smithi018.stdout:mon.smithi018 smithi018 running (19m) 12m ago 37m 57.0M 2048M 19.0.0-2879-g324baf8d 8717720dc5e5 df3e201dfed6 2024-04-12T21:20:42.537 INFO:teuthology.orchestra.run.smithi018.stdout:mon.smithi186 smithi186 running (18m) 13m ago 34m 37.7M 2048M 19.0.0-2879-g324baf8d 8717720dc5e5 c77498da9b2b 2024-04-12T21:20:42.537 INFO:teuthology.orchestra.run.smithi018.stdout:node-exporter.smithi018 smithi018 *:9100 running (22m) 12m ago 35m 7780k - 1.5.0 0da6a335fe13 326c3d9c6847 2024-04-12T21:20:42.537 INFO:teuthology.orchestra.run.smithi018.stdout:node-exporter.smithi186 smithi186 *:9100 running (22m) 13m ago 34m 7826k - 1.5.0 0da6a335fe13 ec2112baeecf 2024-04-12T21:20:42.537 INFO:teuthology.orchestra.run.smithi018.stdout:osd.0 smithi018 running (16m) 12m ago 33m 121M 1559M 19.0.0-2879-g324baf8d 8717720dc5e5 60b1273ca047 2024-04-12T21:20:42.537 INFO:teuthology.orchestra.run.smithi018.stdout:osd.1 smithi018 running (16m) 12m ago 32m 91.3M 1559M 19.0.0-2879-g324baf8d 8717720dc5e5 170ce1e5fb8d 2024-04-12T21:20:42.537 INFO:teuthology.orchestra.run.smithi018.stdout:osd.2 smithi018 running (15m) 12m ago 32m 90.3M 1559M 19.0.0-2879-g324baf8d 8717720dc5e5 c2cbc6aa2839 2024-04-12T21:20:42.537 INFO:teuthology.orchestra.run.smithi018.stdout:osd.3 smithi186 running (15m) 13m ago 31m 96.6M 2581M 19.0.0-2879-g324baf8d 8717720dc5e5 0138e8262e4b 2024-04-12T21:20:42.537 INFO:teuthology.orchestra.run.smithi018.stdout:osd.4 smithi186 running (14m) 13m ago 31m 87.2M 2581M 19.0.0-2879-g324baf8d 8717720dc5e5 b20cb63b25c9 2024-04-12T21:20:42.537 INFO:teuthology.orchestra.run.smithi018.stdout:osd.5 smithi186 running (13m) 13m ago 30m 11.1M 2581M 19.0.0-2879-g324baf8d 8717720dc5e5 2513bea2610e 2024-04-12T21:20:42.537 INFO:teuthology.orchestra.run.smithi018.stdout:prometheus.smithi018 smithi018 *:9095 running (21m) 12m ago 35m 60.6M - 2.43.0 a07b618ecd1d f58841127e21 2024-04-12T21:20:42.538 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:42.542+0000 7f3dcf600700 1 -- 172.21.15.18:0/2358921840 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f3db006ec00 msgr2=0x7f3db00710b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:20:42.538 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:42.542+0000 7f3dcf600700 1 --2- 172.21.15.18:0/2358921840 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f3db006ec00 0x7f3db00710b0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f3dbc004560 tx=0x7f3dbc009e30 comp rx=0 tx=0).stop 2024-04-12T21:20:42.538 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:42.542+0000 7f3dcf600700 1 -- 172.21.15.18:0/2358921840 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3dc81059c0 msgr2=0x7f3dc81160d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:20:42.538 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:42.542+0000 7f3dcf600700 1 --2- 172.21.15.18:0/2358921840 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3dc81059c0 0x7f3dc81160d0 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f3db4002a50 tx=0x7f3db4004600 comp rx=0 tx=0).stop 2024-04-12T21:20:42.539 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:42.542+0000 7f3dcf600700 1 -- 172.21.15.18:0/2358921840 shutdown_connections 2024-04-12T21:20:42.539 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:42.542+0000 7f3dcf600700 1 --2- 172.21.15.18:0/2358921840 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f3db006ec00 0x7f3db00710b0 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:20:42.539 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:42.542+0000 7f3dcf600700 1 --2- 172.21.15.18:0/2358921840 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3dc8105020 0x7f3dc8115b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:20:42.539 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:42.542+0000 7f3dcf600700 1 --2- 172.21.15.18:0/2358921840 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3dc81059c0 0x7f3dc81160d0 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:20:42.539 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:42.542+0000 7f3dcf600700 1 -- 172.21.15.18:0/2358921840 >> 172.21.15.18:0/2358921840 conn(0x7f3dc80faba0 msgr2=0x7f3dc80fbb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:20:42.539 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:42.542+0000 7f3dcf600700 1 -- 172.21.15.18:0/2358921840 shutdown_connections 2024-04-12T21:20:42.539 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:42.543+0000 7f3dcf600700 1 -- 172.21.15.18:0/2358921840 wait complete. 2024-04-12T21:20:42.695 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:42.698+0000 7f4560a00700 1 -- 172.21.15.18:0/3857530445 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f455c072e00 msgr2=0x7f455c10d630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:20:42.695 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:42.698+0000 7f4560a00700 1 --2- 172.21.15.18:0/3857530445 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f455c072e00 0x7f455c10d630 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f4548004940 tx=0x7f4548009e30 comp rx=0 tx=0).stop 2024-04-12T21:20:42.695 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:42.698+0000 7f4560a00700 1 -- 172.21.15.18:0/3857530445 shutdown_connections 2024-04-12T21:20:42.695 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:42.698+0000 7f4560a00700 1 --2- 172.21.15.18:0/3857530445 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f455c072e00 0x7f455c10d630 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:20:42.695 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:42.698+0000 7f4560a00700 1 --2- 172.21.15.18:0/3857530445 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f455c0724f0 0x7f455c0728c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:20:42.695 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:42.698+0000 7f4560a00700 1 -- 172.21.15.18:0/3857530445 >> 172.21.15.18:0/3857530445 conn(0x7f455c0fab10 msgr2=0x7f455c0fcef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:20:42.696 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:42.699+0000 7f4560a00700 1 -- 172.21.15.18:0/3857530445 shutdown_connections 2024-04-12T21:20:42.696 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:42.699+0000 7f4560a00700 1 -- 172.21.15.18:0/3857530445 wait complete. 2024-04-12T21:20:42.697 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:42.700+0000 7f4560a00700 1 Processor -- start 2024-04-12T21:20:42.697 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:42.700+0000 7f4560a00700 1 -- start start 2024-04-12T21:20:42.697 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:42.700+0000 7f4560a00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f455c0724f0 0x7f455c119f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:20:42.697 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:42.700+0000 7f4560a00700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f455c072e00 0x7f455c11a4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:20:42.697 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:42.700+0000 7f4560a00700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f455c11ab10 con 0x7f455c0724f0 2024-04-12T21:20:42.697 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:42.700+0000 7f4560a00700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f455c113ee0 con 0x7f455c072e00 2024-04-12T21:20:42.697 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:42.701+0000 7f455b400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f455c0724f0 0x7f455c119f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:20:42.698 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:42.701+0000 7f455b400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f455c0724f0 0x7f455c119f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:47196/0 (socket says 172.21.15.18:47196) 2024-04-12T21:20:42.698 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:42.701+0000 7f455b400700 1 -- 172.21.15.18:0/3086567664 learned_addr learned my addr 172.21.15.18:0/3086567664 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:20:42.698 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:42.701+0000 7f455aa00700 1 --2- 172.21.15.18:0/3086567664 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f455c072e00 0x7f455c11a4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:20:42.698 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:42.701+0000 7f455b400700 1 -- 172.21.15.18:0/3086567664 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f455c072e00 msgr2=0x7f455c11a4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:20:42.698 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:42.701+0000 7f455b400700 1 --2- 172.21.15.18:0/3086567664 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f455c072e00 0x7f455c11a4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:20:42.698 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:42.701+0000 7f455b400700 1 -- 172.21.15.18:0/3086567664 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4548005040 con 0x7f455c0724f0 2024-04-12T21:20:42.698 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:42.701+0000 7f455b400700 1 --2- 172.21.15.18:0/3086567664 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f455c0724f0 0x7f455c119f80 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f454c002a50 tx=0x7f454c0046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:20:42.699 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:42.702+0000 7f4553400700 1 -- 172.21.15.18:0/3086567664 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f454c007cc0 con 0x7f455c0724f0 2024-04-12T21:20:42.699 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:42.702+0000 7f4553400700 1 -- 172.21.15.18:0/3086567664 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f454c00f040 con 0x7f455c0724f0 2024-04-12T21:20:42.699 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:42.702+0000 7f4560a00700 1 -- 172.21.15.18:0/3086567664 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f455c1141c0 con 0x7f455c0724f0 2024-04-12T21:20:42.699 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:42.702+0000 7f4553400700 1 -- 172.21.15.18:0/3086567664 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f454c009640 con 0x7f455c0724f0 2024-04-12T21:20:42.699 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:42.702+0000 7f4560a00700 1 -- 172.21.15.18:0/3086567664 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f455c114780 con 0x7f455c0724f0 2024-04-12T21:20:42.699 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:42.703+0000 7f4560a00700 1 -- 172.21.15.18:0/3086567664 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f455c073be0 con 0x7f455c0724f0 2024-04-12T21:20:42.699 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:42.704+0000 7f4553400700 1 -- 172.21.15.18:0/3086567664 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7f454c0097a0 con 0x7f455c0724f0 2024-04-12T21:20:42.701 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:42.705+0000 7f4553400700 1 --2- 172.21.15.18:0/3086567664 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f454006ec00 0x7f45400710b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:20:42.701 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:42.705+0000 7f4553400700 1 -- 172.21.15.18:0/3086567664 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 6757+0+0 (secure 0 0 0) 0x7f454c08e830 con 0x7f455c0724f0 2024-04-12T21:20:42.701 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:42.705+0000 7f455aa00700 1 --2- 172.21.15.18:0/3086567664 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f454006ec00 0x7f45400710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:20:42.701 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:42.706+0000 7f455aa00700 1 --2- 172.21.15.18:0/3086567664 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f454006ec00 0x7f45400710b0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f454800b3b0 tx=0x7f4548009e30 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:20:42.705 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:42.709+0000 7f4553400700 1 -- 172.21.15.18:0/3086567664 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f454c05af60 con 0x7f455c0724f0 2024-04-12T21:20:42.849 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:20:42 smithi018 ceph-mon[112590]: from='client.35116 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:20:43.028 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:20:42 smithi186 ceph-mon[86379]: from='client.35116 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:20:43.029 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:43.033+0000 7f4560a00700 1 -- 172.21.15.18:0/3086567664 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f455c04bed0 con 0x7f455c0724f0 2024-04-12T21:20:43.030 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:43.034+0000 7f4553400700 1 -- 172.21.15.18:0/3086567664 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+837 (secure 0 0 0) 0x7f454c05aaf0 con 0x7f455c0724f0 2024-04-12T21:20:43.030 INFO:teuthology.orchestra.run.smithi018.stdout:{ 2024-04-12T21:20:43.030 INFO:teuthology.orchestra.run.smithi018.stdout: "mon": { 2024-04-12T21:20:43.030 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 19.0.0-2879-g324baf8d (324baf8ddb91dea46cac50cb535c306115791355) squid (dev)": 2 2024-04-12T21:20:43.031 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-12T21:20:43.031 INFO:teuthology.orchestra.run.smithi018.stdout: "mgr": { 2024-04-12T21:20:43.031 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 19.0.0-2879-g324baf8d (324baf8ddb91dea46cac50cb535c306115791355) squid (dev)": 2 2024-04-12T21:20:43.031 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-12T21:20:43.031 INFO:teuthology.orchestra.run.smithi018.stdout: "osd": { 2024-04-12T21:20:43.031 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 19.0.0-2879-g324baf8d (324baf8ddb91dea46cac50cb535c306115791355) squid (dev)": 6 2024-04-12T21:20:43.031 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-12T21:20:43.031 INFO:teuthology.orchestra.run.smithi018.stdout: "mds": { 2024-04-12T21:20:43.031 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 17.2.7-827-gd3b5060a (d3b5060a17449edfae9385cdf792afec109fca5d) quincy (stable)": 2, 2024-04-12T21:20:43.031 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 19.0.0-2879-g324baf8d (324baf8ddb91dea46cac50cb535c306115791355) squid (dev)": 2 2024-04-12T21:20:43.031 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-12T21:20:43.031 INFO:teuthology.orchestra.run.smithi018.stdout: "overall": { 2024-04-12T21:20:43.031 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 17.2.7-827-gd3b5060a (d3b5060a17449edfae9385cdf792afec109fca5d) quincy (stable)": 2, 2024-04-12T21:20:43.031 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 19.0.0-2879-g324baf8d (324baf8ddb91dea46cac50cb535c306115791355) squid (dev)": 12 2024-04-12T21:20:43.031 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-04-12T21:20:43.032 INFO:teuthology.orchestra.run.smithi018.stdout:} 2024-04-12T21:20:43.033 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:43.037+0000 7f4560a00700 1 -- 172.21.15.18:0/3086567664 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f454006ec00 msgr2=0x7f45400710b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:20:43.033 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:43.037+0000 7f4560a00700 1 --2- 172.21.15.18:0/3086567664 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f454006ec00 0x7f45400710b0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f454800b3b0 tx=0x7f4548009e30 comp rx=0 tx=0).stop 2024-04-12T21:20:43.033 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:43.037+0000 7f4560a00700 1 -- 172.21.15.18:0/3086567664 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f455c0724f0 msgr2=0x7f455c119f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:20:43.033 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:43.037+0000 7f4560a00700 1 --2- 172.21.15.18:0/3086567664 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f455c0724f0 0x7f455c119f80 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f454c002a50 tx=0x7f454c0046b0 comp rx=0 tx=0).stop 2024-04-12T21:20:43.034 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:43.037+0000 7f4560a00700 1 -- 172.21.15.18:0/3086567664 shutdown_connections 2024-04-12T21:20:43.034 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:43.037+0000 7f4560a00700 1 --2- 172.21.15.18:0/3086567664 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f454006ec00 0x7f45400710b0 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:20:43.034 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:43.037+0000 7f4560a00700 1 --2- 172.21.15.18:0/3086567664 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f455c0724f0 0x7f455c119f80 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:20:43.034 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:43.037+0000 7f4560a00700 1 --2- 172.21.15.18:0/3086567664 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f455c072e00 0x7f455c11a4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:20:43.034 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:43.037+0000 7f4560a00700 1 -- 172.21.15.18:0/3086567664 >> 172.21.15.18:0/3086567664 conn(0x7f455c0fab10 msgr2=0x7f455c0faee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:20:43.034 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:43.037+0000 7f4560a00700 1 -- 172.21.15.18:0/3086567664 shutdown_connections 2024-04-12T21:20:43.034 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:43.037+0000 7f4560a00700 1 -- 172.21.15.18:0/3086567664 wait complete. 2024-04-12T21:20:43.187 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:43.190+0000 7fab31400700 1 -- 172.21.15.18:0/18912786 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fab2c109dd0 msgr2=0x7fab2c103440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:20:43.187 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:43.190+0000 7fab31400700 1 --2- 172.21.15.18:0/18912786 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fab2c109dd0 0x7fab2c103440 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7fab18004940 tx=0x7fab18009e30 comp rx=0 tx=0).stop 2024-04-12T21:20:43.187 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:43.191+0000 7fab31400700 1 -- 172.21.15.18:0/18912786 shutdown_connections 2024-04-12T21:20:43.187 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:43.191+0000 7fab31400700 1 --2- 172.21.15.18:0/18912786 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fab2c109dd0 0x7fab2c103440 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:20:43.187 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:43.191+0000 7fab31400700 1 --2- 172.21.15.18:0/18912786 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fab2c109430 0x7fab2c109800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:20:43.187 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:43.191+0000 7fab31400700 1 -- 172.21.15.18:0/18912786 >> 172.21.15.18:0/18912786 conn(0x7fab2c072c00 msgr2=0x7fab2c075010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:20:43.187 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:43.191+0000 7fab31400700 1 -- 172.21.15.18:0/18912786 shutdown_connections 2024-04-12T21:20:43.187 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:43.191+0000 7fab31400700 1 -- 172.21.15.18:0/18912786 wait complete. 2024-04-12T21:20:43.189 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:43.191+0000 7fab31400700 1 Processor -- start 2024-04-12T21:20:43.189 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:43.192+0000 7fab31400700 1 -- start start 2024-04-12T21:20:43.189 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:43.192+0000 7fab31400700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fab2c109430 0x7fab2c119fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:20:43.189 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:43.192+0000 7fab31400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fab2c109dd0 0x7fab2c11a520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:20:43.189 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:43.192+0000 7fab31400700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab2c11ab70 con 0x7fab2c109dd0 2024-04-12T21:20:43.190 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:43.192+0000 7fab31400700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab2c114150 con 0x7fab2c109430 2024-04-12T21:20:43.190 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:43.192+0000 7fab29e00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fab2c109dd0 0x7fab2c11a520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:20:43.190 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:43.192+0000 7fab29e00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fab2c109dd0 0x7fab2c11a520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:47214/0 (socket says 172.21.15.18:47214) 2024-04-12T21:20:43.190 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:43.192+0000 7fab29e00700 1 -- 172.21.15.18:0/3615187954 learned_addr learned my addr 172.21.15.18:0/3615187954 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:20:43.190 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:43.193+0000 7fab2a800700 1 --2- 172.21.15.18:0/3615187954 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fab2c109430 0x7fab2c119fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:20:43.190 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:43.193+0000 7fab29e00700 1 -- 172.21.15.18:0/3615187954 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fab2c109430 msgr2=0x7fab2c119fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:20:43.190 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:43.193+0000 7fab29e00700 1 --2- 172.21.15.18:0/3615187954 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fab2c109430 0x7fab2c119fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:20:43.190 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:43.193+0000 7fab29e00700 1 -- 172.21.15.18:0/3615187954 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab18005040 con 0x7fab2c109dd0 2024-04-12T21:20:43.190 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:43.193+0000 7fab29e00700 1 --2- 172.21.15.18:0/3615187954 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fab2c109dd0 0x7fab2c11a520 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7fab18009100 tx=0x7fab18009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:20:43.190 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:43.193+0000 7fab22a00700 1 -- 172.21.15.18:0/3615187954 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fab1800e040 con 0x7fab2c109dd0 2024-04-12T21:20:43.190 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:43.193+0000 7fab22a00700 1 -- 172.21.15.18:0/3615187954 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fab18004290 con 0x7fab2c109dd0 2024-04-12T21:20:43.190 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:43.193+0000 7fab31400700 1 -- 172.21.15.18:0/3615187954 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fab2c1143d0 con 0x7fab2c109dd0 2024-04-12T21:20:43.190 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:43.193+0000 7fab22a00700 1 -- 172.21.15.18:0/3615187954 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fab1801c3f0 con 0x7fab2c109dd0 2024-04-12T21:20:43.190 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:43.194+0000 7fab31400700 1 -- 172.21.15.18:0/3615187954 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fab2c114990 con 0x7fab2c109dd0 2024-04-12T21:20:43.191 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:43.194+0000 7fab31400700 1 -- 172.21.15.18:0/3615187954 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fab2c104b80 con 0x7fab2c109dd0 2024-04-12T21:20:43.191 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:43.196+0000 7fab22a00700 1 -- 172.21.15.18:0/3615187954 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7fab18013790 con 0x7fab2c109dd0 2024-04-12T21:20:43.193 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:43.197+0000 7fab22a00700 1 --2- 172.21.15.18:0/3615187954 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fab1006ec50 0x7fab10071100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:20:43.193 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:43.197+0000 7fab22a00700 1 -- 172.21.15.18:0/3615187954 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 6757+0+0 (secure 0 0 0) 0x7fab1808fd10 con 0x7fab2c109dd0 2024-04-12T21:20:43.193 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:43.197+0000 7fab2a800700 1 --2- 172.21.15.18:0/3615187954 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fab1006ec50 0x7fab10071100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:20:43.193 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:43.198+0000 7fab2a800700 1 --2- 172.21.15.18:0/3615187954 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fab1006ec50 0x7fab10071100 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fab1c0041d0 tx=0x7fab1c009040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:20:43.198 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:43.202+0000 7fab22a00700 1 -- 172.21.15.18:0/3615187954 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7fab1805c4c0 con 0x7fab2c109dd0 2024-04-12T21:20:43.479 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:43.483+0000 7fab31400700 1 -- 172.21.15.18:0/3615187954 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fab2c110520 con 0x7fab2c109dd0 2024-04-12T21:20:43.480 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:43.484+0000 7fab22a00700 1 -- 172.21.15.18:0/3615187954 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 29 v29) v1 ==== 76+0+1872 (secure 0 0 0) 0x7fab18013420 con 0x7fab2c109dd0 2024-04-12T21:20:43.481 INFO:teuthology.orchestra.run.smithi018.stdout:e29 2024-04-12T21:20:43.482 INFO:teuthology.orchestra.run.smithi018.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-12T21:20:43.482 INFO:teuthology.orchestra.run.smithi018.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-12T21:20:43.482 INFO:teuthology.orchestra.run.smithi018.stdout:legacy client fscid: 1 2024-04-12T21:20:43.482 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-12T21:20:43.482 INFO:teuthology.orchestra.run.smithi018.stdout:Filesystem 'cephfs' (1) 2024-04-12T21:20:43.482 INFO:teuthology.orchestra.run.smithi018.stdout:fs_name cephfs 2024-04-12T21:20:43.482 INFO:teuthology.orchestra.run.smithi018.stdout:epoch 24 2024-04-12T21:20:43.482 INFO:teuthology.orchestra.run.smithi018.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-04-12T21:20:43.482 INFO:teuthology.orchestra.run.smithi018.stdout:created 2024-04-12T20:51:57.362850+0000 2024-04-12T21:20:43.482 INFO:teuthology.orchestra.run.smithi018.stdout:modified 2024-04-12T21:06:59.717190+0000 2024-04-12T21:20:43.482 INFO:teuthology.orchestra.run.smithi018.stdout:tableserver 0 2024-04-12T21:20:43.482 INFO:teuthology.orchestra.run.smithi018.stdout:root 0 2024-04-12T21:20:43.483 INFO:teuthology.orchestra.run.smithi018.stdout:session_timeout 60 2024-04-12T21:20:43.483 INFO:teuthology.orchestra.run.smithi018.stdout:session_autoclose 300 2024-04-12T21:20:43.483 INFO:teuthology.orchestra.run.smithi018.stdout:max_file_size 1099511627776 2024-04-12T21:20:43.483 INFO:teuthology.orchestra.run.smithi018.stdout:max_xattr_size 65536 2024-04-12T21:20:43.483 INFO:teuthology.orchestra.run.smithi018.stdout:required_client_features {} 2024-04-12T21:20:43.483 INFO:teuthology.orchestra.run.smithi018.stdout:last_failure 0 2024-04-12T21:20:43.483 INFO:teuthology.orchestra.run.smithi018.stdout:last_failure_osd_epoch 100 2024-04-12T21:20:43.483 INFO:teuthology.orchestra.run.smithi018.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-12T21:20:43.483 INFO:teuthology.orchestra.run.smithi018.stdout:max_mds 2 2024-04-12T21:20:43.483 INFO:teuthology.orchestra.run.smithi018.stdout:in 0,1 2024-04-12T21:20:43.483 INFO:teuthology.orchestra.run.smithi018.stdout:up {} 2024-04-12T21:20:43.483 INFO:teuthology.orchestra.run.smithi018.stdout:failed 0,1 2024-04-12T21:20:43.483 INFO:teuthology.orchestra.run.smithi018.stdout:damaged 2024-04-12T21:20:43.483 INFO:teuthology.orchestra.run.smithi018.stdout:stopped 2024-04-12T21:20:43.484 INFO:teuthology.orchestra.run.smithi018.stdout:data_pools [3] 2024-04-12T21:20:43.484 INFO:teuthology.orchestra.run.smithi018.stdout:metadata_pool 2 2024-04-12T21:20:43.484 INFO:teuthology.orchestra.run.smithi018.stdout:inline_data disabled 2024-04-12T21:20:43.484 INFO:teuthology.orchestra.run.smithi018.stdout:balancer 2024-04-12T21:20:43.484 INFO:teuthology.orchestra.run.smithi018.stdout:bal_rank_mask -1 2024-04-12T21:20:43.484 INFO:teuthology.orchestra.run.smithi018.stdout:standby_count_wanted 1 2024-04-12T21:20:43.484 INFO:teuthology.orchestra.run.smithi018.stdout:qdb_cluster leader: 0 members: 2024-04-12T21:20:43.484 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-12T21:20:43.484 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-12T21:20:43.484 INFO:teuthology.orchestra.run.smithi018.stdout:Standby daemons: 2024-04-12T21:20:43.484 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-12T21:20:43.484 INFO:teuthology.orchestra.run.smithi018.stdout:[mds.cephfs.smithi186.wjnafs{-1:24277} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.186:6826/438347207,v1:172.21.15.186:6827/438347207] compat {c=[1],r=[1],i=[7ff]}] 2024-04-12T21:20:43.484 INFO:teuthology.orchestra.run.smithi018.stdout:[mds.cephfs.smithi186.jdllyn{-1:24281} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.186:6824/1464359952,v1:172.21.15.186:6825/1464359952] compat {c=[1],r=[1],i=[7ff]}] 2024-04-12T21:20:43.484 INFO:teuthology.orchestra.run.smithi018.stdout:[mds.cephfs.smithi018.acqbsc{-1:34436} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.18:6828/755179764,v1:172.21.15.18:6829/755179764] compat {c=[1],r=[1],i=[1fff]}] 2024-04-12T21:20:43.485 INFO:teuthology.orchestra.run.smithi018.stdout:[mds.cephfs.smithi018.szjluo{-1:34468} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.18:6826/2407992653,v1:172.21.15.18:6827/2407992653] compat {c=[1],r=[1],i=[1fff]}] 2024-04-12T21:20:43.485 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:43.486+0000 7fab31400700 1 -- 172.21.15.18:0/3615187954 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fab1006ec50 msgr2=0x7fab10071100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:20:43.486 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:43.486+0000 7fab31400700 1 --2- 172.21.15.18:0/3615187954 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fab1006ec50 0x7fab10071100 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fab1c0041d0 tx=0x7fab1c009040 comp rx=0 tx=0).stop 2024-04-12T21:20:43.486 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:43.486+0000 7fab31400700 1 -- 172.21.15.18:0/3615187954 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fab2c109dd0 msgr2=0x7fab2c11a520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:20:43.486 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:43.486+0000 7fab31400700 1 --2- 172.21.15.18:0/3615187954 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fab2c109dd0 0x7fab2c11a520 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7fab18009100 tx=0x7fab18009e30 comp rx=0 tx=0).stop 2024-04-12T21:20:43.486 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:43.486+0000 7fab31400700 1 -- 172.21.15.18:0/3615187954 shutdown_connections 2024-04-12T21:20:43.486 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:43.486+0000 7fab31400700 1 --2- 172.21.15.18:0/3615187954 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fab1006ec50 0x7fab10071100 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:20:43.486 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:43.486+0000 7fab31400700 1 --2- 172.21.15.18:0/3615187954 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fab2c109430 0x7fab2c119fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:20:43.486 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:43.486+0000 7fab31400700 1 --2- 172.21.15.18:0/3615187954 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fab2c109dd0 0x7fab2c11a520 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:20:43.486 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:43.486+0000 7fab31400700 1 -- 172.21.15.18:0/3615187954 >> 172.21.15.18:0/3615187954 conn(0x7fab2c072c00 msgr2=0x7fab2c074be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:20:43.486 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:43.487+0000 7fab31400700 1 -- 172.21.15.18:0/3615187954 shutdown_connections 2024-04-12T21:20:43.486 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:43.487+0000 7fab31400700 1 -- 172.21.15.18:0/3615187954 wait complete. 2024-04-12T21:20:43.486 INFO:teuthology.orchestra.run.smithi018.stderr:dumped fsmap epoch 29 2024-04-12T21:20:43.641 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:43.645+0000 7f5154400700 1 -- 172.21.15.18:0/2813986029 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f514c109430 msgr2=0x7f514c109800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:20:43.641 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:43.645+0000 7f5154400700 1 --2- 172.21.15.18:0/2813986029 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f514c109430 0x7f514c109800 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f5140009ce0 tx=0x7f5140004d10 comp rx=0 tx=0).stop 2024-04-12T21:20:43.642 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:43.645+0000 7f5154400700 1 -- 172.21.15.18:0/2813986029 shutdown_connections 2024-04-12T21:20:43.642 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:43.645+0000 7f5154400700 1 --2- 172.21.15.18:0/2813986029 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f514c109dd0 0x7f514c103440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:20:43.642 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:43.645+0000 7f5154400700 1 --2- 172.21.15.18:0/2813986029 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f514c109430 0x7f514c109800 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:20:43.642 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:43.645+0000 7f5154400700 1 -- 172.21.15.18:0/2813986029 >> 172.21.15.18:0/2813986029 conn(0x7f514c072c00 msgr2=0x7f514c075010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:20:43.642 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:43.645+0000 7f5154400700 1 -- 172.21.15.18:0/2813986029 shutdown_connections 2024-04-12T21:20:43.642 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:43.645+0000 7f5154400700 1 -- 172.21.15.18:0/2813986029 wait complete. 2024-04-12T21:20:43.643 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:43.646+0000 7f5154400700 1 Processor -- start 2024-04-12T21:20:43.643 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:43.646+0000 7f5154400700 1 -- start start 2024-04-12T21:20:43.643 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:43.647+0000 7f5154400700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f514c109430 0x7f514c11a140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:20:43.643 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:43.647+0000 7f5154400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f514c109dd0 0x7f514c11a680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:20:43.643 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:43.647+0000 7f5154400700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f514c11acd0 con 0x7f514c109dd0 2024-04-12T21:20:43.643 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:43.647+0000 7f5154400700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f514c1142b0 con 0x7f514c109430 2024-04-12T21:20:43.644 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:43.647+0000 7f5151000700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f514c109dd0 0x7f514c11a680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:20:43.644 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:43.647+0000 7f5151000700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f514c109dd0 0x7f514c11a680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:47232/0 (socket says 172.21.15.18:47232) 2024-04-12T21:20:43.645 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:43.647+0000 7f5151000700 1 -- 172.21.15.18:0/940422857 learned_addr learned my addr 172.21.15.18:0/940422857 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:20:43.645 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:43.647+0000 7f5151a00700 1 --2- 172.21.15.18:0/940422857 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f514c109430 0x7f514c11a140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:20:43.645 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:43.648+0000 7f5151000700 1 -- 172.21.15.18:0/940422857 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f514c109430 msgr2=0x7f514c11a140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:20:43.645 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:43.648+0000 7f5151000700 1 --2- 172.21.15.18:0/940422857 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f514c109430 0x7f514c11a140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:20:43.645 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:43.648+0000 7f5151000700 1 -- 172.21.15.18:0/940422857 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5140009940 con 0x7f514c109dd0 2024-04-12T21:20:43.645 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:43.648+0000 7f5151000700 1 --2- 172.21.15.18:0/940422857 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f514c109dd0 0x7f514c11a680 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f513800c7e0 tx=0x7f5138007f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:20:43.645 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:43.649+0000 7f5149600700 1 -- 172.21.15.18:0/940422857 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5138012810 con 0x7f514c109dd0 2024-04-12T21:20:43.645 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:43.649+0000 7f5149600700 1 -- 172.21.15.18:0/940422857 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f5138012e40 con 0x7f514c109dd0 2024-04-12T21:20:43.645 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:43.649+0000 7f5154400700 1 -- 172.21.15.18:0/940422857 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f514c114590 con 0x7f514c109dd0 2024-04-12T21:20:43.645 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:43.649+0000 7f5149600700 1 -- 172.21.15.18:0/940422857 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f513800a550 con 0x7f514c109dd0 2024-04-12T21:20:43.645 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:43.649+0000 7f5154400700 1 -- 172.21.15.18:0/940422857 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f514c114b50 con 0x7f514c109dd0 2024-04-12T21:20:43.646 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:43.650+0000 7f5154400700 1 -- 172.21.15.18:0/940422857 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f514c104b80 con 0x7f514c109dd0 2024-04-12T21:20:43.647 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:43.651+0000 7f5149600700 1 -- 172.21.15.18:0/940422857 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7f5138011380 con 0x7f514c109dd0 2024-04-12T21:20:43.648 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:43.653+0000 7f5149600700 1 --2- 172.21.15.18:0/940422857 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f513406ec00 0x7f51340710b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:20:43.649 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:43.653+0000 7f5149600700 1 -- 172.21.15.18:0/940422857 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 6757+0+0 (secure 0 0 0) 0x7f513808ee50 con 0x7f514c109dd0 2024-04-12T21:20:43.649 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:43.653+0000 7f5151a00700 1 --2- 172.21.15.18:0/940422857 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f513406ec00 0x7f51340710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:20:43.649 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:43.654+0000 7f5151a00700 1 --2- 172.21.15.18:0/940422857 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f513406ec00 0x7f51340710b0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f514000b740 tx=0x7f5140004d10 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:20:43.654 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:43.658+0000 7f5149600700 1 -- 172.21.15.18:0/940422857 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f513805b580 con 0x7f514c109dd0 2024-04-12T21:20:43.849 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:20:43 smithi018 ceph-mon[112590]: from='client.35120 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:20:43.850 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:20:43 smithi018 ceph-mon[112590]: pgmap v627: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:20:43.850 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:20:43 smithi018 ceph-mon[112590]: from='client.35124 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:20:43.850 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:20:43 smithi018 ceph-mon[112590]: from='client.? 172.21.15.18:0/3086567664' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T21:20:43.850 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:20:43 smithi018 ceph-mon[112590]: from='client.? 172.21.15.18:0/3615187954' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-12T21:20:43.911 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:43.914+0000 7f5154400700 1 -- 172.21.15.18:0/940422857 --> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f514c1105b0 con 0x7f513406ec00 2024-04-12T21:20:43.913 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:43.917+0000 7f5149600700 1 -- 172.21.15.18:0/940422857 <== mgr.34102 v2:172.21.15.18:6800/3812891919 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f514c1105b0 con 0x7f513406ec00 2024-04-12T21:20:43.914 INFO:teuthology.orchestra.run.smithi018.stdout:{ 2024-04-12T21:20:43.914 INFO:teuthology.orchestra.run.smithi018.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:f4452def3be4d350f4a8e5feaf7ebb1511be9298fe68478ba7a2d6fbd7d69a63", 2024-04-12T21:20:43.914 INFO:teuthology.orchestra.run.smithi018.stdout: "in_progress": true, 2024-04-12T21:20:43.914 INFO:teuthology.orchestra.run.smithi018.stdout: "which": "Upgrading all daemon types on all hosts", 2024-04-12T21:20:43.914 INFO:teuthology.orchestra.run.smithi018.stdout: "services_complete": [ 2024-04-12T21:20:43.914 INFO:teuthology.orchestra.run.smithi018.stdout: "crash", 2024-04-12T21:20:43.914 INFO:teuthology.orchestra.run.smithi018.stdout: "osd", 2024-04-12T21:20:43.914 INFO:teuthology.orchestra.run.smithi018.stdout: "mon", 2024-04-12T21:20:43.914 INFO:teuthology.orchestra.run.smithi018.stdout: "mgr" 2024-04-12T21:20:43.915 INFO:teuthology.orchestra.run.smithi018.stdout: ], 2024-04-12T21:20:43.915 INFO:teuthology.orchestra.run.smithi018.stdout: "progress": "14/21 daemons upgraded", 2024-04-12T21:20:43.915 INFO:teuthology.orchestra.run.smithi018.stdout: "message": "Currently upgrading mds daemons", 2024-04-12T21:20:43.915 INFO:teuthology.orchestra.run.smithi018.stdout: "is_paused": false 2024-04-12T21:20:43.915 INFO:teuthology.orchestra.run.smithi018.stdout:} 2024-04-12T21:20:43.917 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:43.921+0000 7f5154400700 1 -- 172.21.15.18:0/940422857 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f513406ec00 msgr2=0x7f51340710b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:20:43.917 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:43.921+0000 7f5154400700 1 --2- 172.21.15.18:0/940422857 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f513406ec00 0x7f51340710b0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f514000b740 tx=0x7f5140004d10 comp rx=0 tx=0).stop 2024-04-12T21:20:43.917 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:43.921+0000 7f5154400700 1 -- 172.21.15.18:0/940422857 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f514c109dd0 msgr2=0x7f514c11a680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:20:43.917 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:43.921+0000 7f5154400700 1 --2- 172.21.15.18:0/940422857 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f514c109dd0 0x7f514c11a680 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f513800c7e0 tx=0x7f5138007f90 comp rx=0 tx=0).stop 2024-04-12T21:20:43.918 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:43.921+0000 7f5154400700 1 -- 172.21.15.18:0/940422857 shutdown_connections 2024-04-12T21:20:43.918 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:43.921+0000 7f5154400700 1 --2- 172.21.15.18:0/940422857 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f513406ec00 0x7f51340710b0 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:20:43.918 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:43.921+0000 7f5154400700 1 --2- 172.21.15.18:0/940422857 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f514c109430 0x7f514c11a140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:20:43.918 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:43.921+0000 7f5154400700 1 --2- 172.21.15.18:0/940422857 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f514c109dd0 0x7f514c11a680 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:20:43.918 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:43.921+0000 7f5154400700 1 -- 172.21.15.18:0/940422857 >> 172.21.15.18:0/940422857 conn(0x7f514c072c00 msgr2=0x7f514c074cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:20:43.918 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:43.921+0000 7f5154400700 1 -- 172.21.15.18:0/940422857 shutdown_connections 2024-04-12T21:20:43.918 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:43.921+0000 7f5154400700 1 -- 172.21.15.18:0/940422857 wait complete. 2024-04-12T21:20:44.028 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:20:43 smithi186 ceph-mon[86379]: from='client.35120 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:20:44.028 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:20:43 smithi186 ceph-mon[86379]: pgmap v627: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:20:44.029 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:20:43 smithi186 ceph-mon[86379]: from='client.35124 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:20:44.029 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:20:43 smithi186 ceph-mon[86379]: from='client.? 172.21.15.18:0/3086567664' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T21:20:44.029 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:20:43 smithi186 ceph-mon[86379]: from='client.? 172.21.15.18:0/3615187954' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-12T21:20:44.076 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:44.080+0000 7f4d93200700 1 -- 172.21.15.18:0/2758067546 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4d94070c70 msgr2=0x7f4d94108f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:20:44.076 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:44.080+0000 7f4d93200700 1 --2- 172.21.15.18:0/2758067546 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4d94070c70 0x7f4d94108f50 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f4d80004970 tx=0x7f4d80009e30 comp rx=0 tx=0).stop 2024-04-12T21:20:44.076 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:44.080+0000 7f4d93200700 1 -- 172.21.15.18:0/2758067546 shutdown_connections 2024-04-12T21:20:44.076 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:44.080+0000 7f4d93200700 1 --2- 172.21.15.18:0/2758067546 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4d94070c70 0x7f4d94108f50 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:20:44.076 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:44.080+0000 7f4d93200700 1 --2- 172.21.15.18:0/2758067546 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4d94070360 0x7f4d94070730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:20:44.076 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:44.080+0000 7f4d93200700 1 -- 172.21.15.18:0/2758067546 >> 172.21.15.18:0/2758067546 conn(0x7f4d940f8790 msgr2=0x7f4d940faba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:20:44.076 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:44.080+0000 7f4d93200700 1 -- 172.21.15.18:0/2758067546 shutdown_connections 2024-04-12T21:20:44.076 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:44.080+0000 7f4d93200700 1 -- 172.21.15.18:0/2758067546 wait complete. 2024-04-12T21:20:44.078 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:44.081+0000 7f4d93200700 1 Processor -- start 2024-04-12T21:20:44.078 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:44.081+0000 7f4d93200700 1 -- start start 2024-04-12T21:20:44.078 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:44.081+0000 7f4d93200700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4d94070360 0x7f4d94115c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:20:44.078 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:44.081+0000 7f4d93200700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4d94070c70 0x7f4d941161b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:20:44.078 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:44.081+0000 7f4d93200700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d94116890 con 0x7f4d94070360 2024-04-12T21:20:44.078 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:44.081+0000 7f4d93200700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d9410fbd0 con 0x7f4d94070c70 2024-04-12T21:20:44.078 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:44.081+0000 7f4d91e00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4d94070360 0x7f4d94115c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:20:44.078 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:44.081+0000 7f4d91e00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4d94070360 0x7f4d94115c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:47260/0 (socket says 172.21.15.18:47260) 2024-04-12T21:20:44.078 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:44.082+0000 7f4d91e00700 1 -- 172.21.15.18:0/2571316805 learned_addr learned my addr 172.21.15.18:0/2571316805 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:20:44.078 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:44.082+0000 7f4d91e00700 1 -- 172.21.15.18:0/2571316805 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4d94070c70 msgr2=0x7f4d941161b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:20:44.078 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:44.082+0000 7f4d91e00700 1 --2- 172.21.15.18:0/2571316805 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4d94070c70 0x7f4d941161b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:20:44.079 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:44.082+0000 7f4d91e00700 1 -- 172.21.15.18:0/2571316805 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d80005040 con 0x7f4d94070360 2024-04-12T21:20:44.079 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:44.082+0000 7f4d91e00700 1 --2- 172.21.15.18:0/2571316805 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4d94070360 0x7f4d94115c70 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f4d8400b790 tx=0x7f4d84009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:20:44.079 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:44.082+0000 7f4d8a000700 1 -- 172.21.15.18:0/2571316805 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4d8400f770 con 0x7f4d94070360 2024-04-12T21:20:44.079 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:44.083+0000 7f4d8a000700 1 -- 172.21.15.18:0/2571316805 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f4d8400fda0 con 0x7f4d94070360 2024-04-12T21:20:44.080 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:44.083+0000 7f4d8a000700 1 -- 172.21.15.18:0/2571316805 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4d8400d310 con 0x7f4d94070360 2024-04-12T21:20:44.080 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:44.083+0000 7f4d93200700 1 -- 172.21.15.18:0/2571316805 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4d9410fe50 con 0x7f4d94070360 2024-04-12T21:20:44.080 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:44.083+0000 7f4d93200700 1 -- 172.21.15.18:0/2571316805 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4d94110340 con 0x7f4d94070360 2024-04-12T21:20:44.080 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:44.084+0000 7f4d93200700 1 -- 172.21.15.18:0/2571316805 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4d94071aa0 con 0x7f4d94070360 2024-04-12T21:20:44.080 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:44.085+0000 7f4d8a000700 1 -- 172.21.15.18:0/2571316805 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7f4d84010420 con 0x7f4d94070360 2024-04-12T21:20:44.081 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:44.086+0000 7f4d8a000700 1 --2- 172.21.15.18:0/2571316805 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f4d7806ec00 0x7f4d780710b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:20:44.082 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:44.086+0000 7f4d8a000700 1 -- 172.21.15.18:0/2571316805 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 6757+0+0 (secure 0 0 0) 0x7f4d8408ea80 con 0x7f4d94070360 2024-04-12T21:20:44.082 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:44.086+0000 7f4d91400700 1 --2- 172.21.15.18:0/2571316805 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f4d7806ec00 0x7f4d780710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:20:44.082 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:44.087+0000 7f4d91400700 1 --2- 172.21.15.18:0/2571316805 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f4d7806ec00 0x7f4d780710b0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f4d80009fd0 tx=0x7f4d80009e30 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:20:44.086 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:44.091+0000 7f4d8a000700 1 -- 172.21.15.18:0/2571316805 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f4d8405b230 con 0x7f4d94070360 2024-04-12T21:20:44.400 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:44.404+0000 7f4d93200700 1 -- 172.21.15.18:0/2571316805 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f4d9410c010 con 0x7f4d94070360 2024-04-12T21:20:44.401 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:44.405+0000 7f4d8a000700 1 -- 172.21.15.18:0/2571316805 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+233 (secure 0 0 0) 0x7f4d8405adc0 con 0x7f4d94070360 2024-04-12T21:20:44.401 INFO:teuthology.orchestra.run.smithi018.stdout:HEALTH_ERR 1 filesystem is degraded; 1 filesystem is offline 2024-04-12T21:20:44.401 INFO:teuthology.orchestra.run.smithi018.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-04-12T21:20:44.401 INFO:teuthology.orchestra.run.smithi018.stdout: fs cephfs is degraded 2024-04-12T21:20:44.401 INFO:teuthology.orchestra.run.smithi018.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-04-12T21:20:44.401 INFO:teuthology.orchestra.run.smithi018.stdout: fs cephfs is offline because no MDS is active for it. 2024-04-12T21:20:44.403 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:44.407+0000 7f4d93200700 1 -- 172.21.15.18:0/2571316805 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f4d7806ec00 msgr2=0x7f4d780710b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:20:44.403 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:44.407+0000 7f4d93200700 1 --2- 172.21.15.18:0/2571316805 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f4d7806ec00 0x7f4d780710b0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f4d80009fd0 tx=0x7f4d80009e30 comp rx=0 tx=0).stop 2024-04-12T21:20:44.404 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:44.407+0000 7f4d93200700 1 -- 172.21.15.18:0/2571316805 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4d94070360 msgr2=0x7f4d94115c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:20:44.404 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:44.407+0000 7f4d93200700 1 --2- 172.21.15.18:0/2571316805 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4d94070360 0x7f4d94115c70 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f4d8400b790 tx=0x7f4d84009250 comp rx=0 tx=0).stop 2024-04-12T21:20:44.404 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:44.407+0000 7f4d93200700 1 -- 172.21.15.18:0/2571316805 shutdown_connections 2024-04-12T21:20:44.404 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:44.407+0000 7f4d93200700 1 --2- 172.21.15.18:0/2571316805 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f4d7806ec00 0x7f4d780710b0 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:20:44.404 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:44.407+0000 7f4d93200700 1 --2- 172.21.15.18:0/2571316805 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4d94070360 0x7f4d94115c70 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:20:44.404 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:44.407+0000 7f4d93200700 1 --2- 172.21.15.18:0/2571316805 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4d94070c70 0x7f4d941161b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:20:44.404 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:44.407+0000 7f4d93200700 1 -- 172.21.15.18:0/2571316805 >> 172.21.15.18:0/2571316805 conn(0x7f4d940f8790 msgr2=0x7f4d940f9c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:20:44.404 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:44.408+0000 7f4d93200700 1 -- 172.21.15.18:0/2571316805 shutdown_connections 2024-04-12T21:20:44.404 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:20:44.408+0000 7f4d93200700 1 -- 172.21.15.18:0/2571316805 wait complete. 2024-04-12T21:20:45.028 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:20:44 smithi186 ceph-mon[86379]: from='client.? 172.21.15.18:0/2571316805' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-12T21:20:45.099 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:20:44 smithi018 ceph-mon[112590]: from='client.? 172.21.15.18:0/2571316805' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-12T21:20:46.028 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:20:45 smithi186 ceph-mon[86379]: from='client.35136 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:20:46.028 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:20:45 smithi186 ceph-mon[86379]: pgmap v628: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:20:46.099 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:20:45 smithi018 ceph-mon[112590]: from='client.35136 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:20:46.099 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:20:45 smithi018 ceph-mon[112590]: pgmap v628: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:20:48.028 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:20:47 smithi186 ceph-mon[86379]: pgmap v629: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:20:48.099 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:20:47 smithi018 ceph-mon[112590]: pgmap v629: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:20:50.028 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:20:49 smithi186 ceph-mon[86379]: pgmap v630: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:20:50.099 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:20:49 smithi018 ceph-mon[112590]: pgmap v630: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:20:52.028 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:20:51 smithi186 ceph-mon[86379]: pgmap v631: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:20:52.099 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:20:51 smithi018 ceph-mon[112590]: pgmap v631: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:20:54.028 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:20:53 smithi186 ceph-mon[86379]: pgmap v632: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:20:54.099 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:20:53 smithi018 ceph-mon[112590]: pgmap v632: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:20:56.028 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:20:55 smithi186 ceph-mon[86379]: pgmap v633: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:20:56.028 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:20:55 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T21:20:56.099 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:20:55 smithi018 ceph-mon[112590]: pgmap v633: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:20:56.099 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:20:55 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T21:20:58.028 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:20:57 smithi186 ceph-mon[86379]: pgmap v634: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:20:58.099 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:20:57 smithi018 ceph-mon[112590]: pgmap v634: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:21:00.030 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:20:59 smithi186 ceph-mon[86379]: pgmap v635: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:21:00.099 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:20:59 smithi018 ceph-mon[112590]: pgmap v635: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:21:02.028 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:21:01 smithi186 ceph-mon[86379]: pgmap v636: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:21:02.099 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:21:01 smithi018 ceph-mon[112590]: pgmap v636: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:21:04.028 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:21:03 smithi186 ceph-mon[86379]: pgmap v637: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:21:04.099 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:21:03 smithi018 ceph-mon[112590]: pgmap v637: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:21:06.028 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:21:05 smithi186 ceph-mon[86379]: pgmap v638: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:21:06.099 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:21:05 smithi018 ceph-mon[112590]: pgmap v638: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:21:08.028 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:21:07 smithi186 ceph-mon[86379]: pgmap v639: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:21:08.099 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:21:07 smithi018 ceph-mon[112590]: pgmap v639: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:21:09.099 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:21:08 smithi018 ceph-mon[112590]: pgmap v640: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:21:09.278 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:21:08 smithi186 ceph-mon[86379]: pgmap v640: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:21:10.278 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:21:09 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T21:21:10.349 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:21:09 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T21:21:11.251 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:21:10 smithi018 ceph-mon[112590]: pgmap v641: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:21:11.278 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:21:10 smithi186 ceph-mon[86379]: pgmap v641: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:21:13.599 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:21:13 smithi018 ceph-mon[112590]: pgmap v642: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:21:13.778 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:21:13 smithi186 ceph-mon[86379]: pgmap v642: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:21:14.578 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:14.581+0000 7f29d6800700 1 -- 172.21.15.18:0/3571707560 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f29d0072e00 msgr2=0x7f29d010d780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:21:14.578 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:14.581+0000 7f29d6800700 1 --2- 172.21.15.18:0/3571707560 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f29d0072e00 0x7f29d010d780 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f29bc004940 tx=0x7f29bc009e30 comp rx=0 tx=0).stop 2024-04-12T21:21:14.578 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:14.581+0000 7f29d6800700 1 -- 172.21.15.18:0/3571707560 shutdown_connections 2024-04-12T21:21:14.578 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:14.581+0000 7f29d6800700 1 --2- 172.21.15.18:0/3571707560 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f29d0072e00 0x7f29d010d780 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:21:14.578 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:14.581+0000 7f29d6800700 1 --2- 172.21.15.18:0/3571707560 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f29d00724f0 0x7f29d00728c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:21:14.578 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:14.581+0000 7f29d6800700 1 -- 172.21.15.18:0/3571707560 >> 172.21.15.18:0/3571707560 conn(0x7f29d00faba0 msgr2=0x7f29d00fcfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:21:14.578 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:14.582+0000 7f29d6800700 1 -- 172.21.15.18:0/3571707560 shutdown_connections 2024-04-12T21:21:14.579 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:14.582+0000 7f29d6800700 1 -- 172.21.15.18:0/3571707560 wait complete. 2024-04-12T21:21:14.579 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:14.583+0000 7f29d6800700 1 Processor -- start 2024-04-12T21:21:14.580 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:14.583+0000 7f29d6800700 1 -- start start 2024-04-12T21:21:14.580 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:14.583+0000 7f29d6800700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f29d00724f0 0x7f29d011a110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:21:14.580 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:14.583+0000 7f29d6800700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f29d0072e00 0x7f29d011a650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:21:14.580 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:14.583+0000 7f29d6800700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29d011aca0 con 0x7f29d0072e00 2024-04-12T21:21:14.580 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:14.583+0000 7f29d6800700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29d0114070 con 0x7f29d00724f0 2024-04-12T21:21:14.580 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:14.584+0000 7f29cea00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f29d0072e00 0x7f29d011a650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:21:14.580 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:14.584+0000 7f29cea00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f29d0072e00 0x7f29d011a650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:38508/0 (socket says 172.21.15.18:38508) 2024-04-12T21:21:14.580 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:14.584+0000 7f29cea00700 1 -- 172.21.15.18:0/1556753924 learned_addr learned my addr 172.21.15.18:0/1556753924 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:21:14.581 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:14.584+0000 7f29cf400700 1 --2- 172.21.15.18:0/1556753924 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f29d00724f0 0x7f29d011a110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:21:14.581 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:14.584+0000 7f29cea00700 1 -- 172.21.15.18:0/1556753924 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f29d00724f0 msgr2=0x7f29d011a110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:21:14.581 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:14.584+0000 7f29cea00700 1 --2- 172.21.15.18:0/1556753924 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f29d00724f0 0x7f29d011a110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:21:14.581 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:14.584+0000 7f29cea00700 1 -- 172.21.15.18:0/1556753924 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29bc005040 con 0x7f29d0072e00 2024-04-12T21:21:14.581 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:14.585+0000 7f29cea00700 1 --2- 172.21.15.18:0/1556753924 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f29d0072e00 0x7f29d011a650 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f29bc00b3b0 tx=0x7f29bc009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:21:14.581 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:14.585+0000 7f29c7400700 1 -- 172.21.15.18:0/1556753924 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f29bc00e040 con 0x7f29d0072e00 2024-04-12T21:21:14.582 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:14.585+0000 7f29c7400700 1 -- 172.21.15.18:0/1556753924 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f29bc013420 con 0x7f29d0072e00 2024-04-12T21:21:14.582 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:14.585+0000 7f29d6800700 1 -- 172.21.15.18:0/1556753924 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f29d01142f0 con 0x7f29d0072e00 2024-04-12T21:21:14.582 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:14.585+0000 7f29c7400700 1 -- 172.21.15.18:0/1556753924 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f29bc01c630 con 0x7f29d0072e00 2024-04-12T21:21:14.582 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:14.585+0000 7f29d6800700 1 -- 172.21.15.18:0/1556753924 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f29d01148b0 con 0x7f29d0072e00 2024-04-12T21:21:14.582 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:14.586+0000 7f29d6800700 1 -- 172.21.15.18:0/1556753924 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f29d010a5c0 con 0x7f29d0072e00 2024-04-12T21:21:14.583 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:14.587+0000 7f29c7400700 1 -- 172.21.15.18:0/1556753924 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7f29bc013590 con 0x7f29d0072e00 2024-04-12T21:21:14.585 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:14.589+0000 7f29c7400700 1 --2- 172.21.15.18:0/1556753924 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f29b406ec10 0x7f29b40710c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:21:14.585 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:14.589+0000 7f29c7400700 1 -- 172.21.15.18:0/1556753924 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 6757+0+0 (secure 0 0 0) 0x7f29bc08fcf0 con 0x7f29d0072e00 2024-04-12T21:21:14.585 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:14.589+0000 7f29cf400700 1 --2- 172.21.15.18:0/1556753924 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f29b406ec10 0x7f29b40710c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:21:14.586 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:14.590+0000 7f29cf400700 1 --2- 172.21.15.18:0/1556753924 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f29b406ec10 0x7f29b40710c0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f29c0004d60 tx=0x7f29c000b040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:21:14.590 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:14.594+0000 7f29c7400700 1 -- 172.21.15.18:0/1556753924 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f29bc05c4a0 con 0x7f29d0072e00 2024-04-12T21:21:14.847 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:14.851+0000 7f29d6800700 1 -- 172.21.15.18:0/1556753924 --> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f29d01157d0 con 0x7f29b406ec10 2024-04-12T21:21:14.849 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:14.853+0000 7f29c7400700 1 -- 172.21.15.18:0/1556753924 <== mgr.34102 v2:172.21.15.18:6800/3812891919 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f29d01157d0 con 0x7f29b406ec10 2024-04-12T21:21:14.852 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:14.856+0000 7f29d6800700 1 -- 172.21.15.18:0/1556753924 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f29b406ec10 msgr2=0x7f29b40710c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:21:14.853 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:14.856+0000 7f29d6800700 1 --2- 172.21.15.18:0/1556753924 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f29b406ec10 0x7f29b40710c0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f29c0004d60 tx=0x7f29c000b040 comp rx=0 tx=0).stop 2024-04-12T21:21:14.853 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:14.856+0000 7f29d6800700 1 -- 172.21.15.18:0/1556753924 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f29d0072e00 msgr2=0x7f29d011a650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:21:14.853 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:14.856+0000 7f29d6800700 1 --2- 172.21.15.18:0/1556753924 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f29d0072e00 0x7f29d011a650 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f29bc00b3b0 tx=0x7f29bc009e30 comp rx=0 tx=0).stop 2024-04-12T21:21:14.853 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:14.856+0000 7f29d6800700 1 -- 172.21.15.18:0/1556753924 shutdown_connections 2024-04-12T21:21:14.853 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:14.856+0000 7f29d6800700 1 --2- 172.21.15.18:0/1556753924 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f29b406ec10 0x7f29b40710c0 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:21:14.853 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:14.856+0000 7f29d6800700 1 --2- 172.21.15.18:0/1556753924 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f29d00724f0 0x7f29d011a110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:21:14.853 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:14.856+0000 7f29d6800700 1 --2- 172.21.15.18:0/1556753924 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f29d0072e00 0x7f29d011a650 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:21:14.853 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:14.856+0000 7f29d6800700 1 -- 172.21.15.18:0/1556753924 >> 172.21.15.18:0/1556753924 conn(0x7f29d00faba0 msgr2=0x7f29d0107740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:21:14.853 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:14.857+0000 7f29d6800700 1 -- 172.21.15.18:0/1556753924 shutdown_connections 2024-04-12T21:21:14.853 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:14.857+0000 7f29d6800700 1 -- 172.21.15.18:0/1556753924 wait complete. 2024-04-12T21:21:14.869 INFO:teuthology.orchestra.run.smithi018.stdout:true 2024-04-12T21:21:15.016 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:15.020+0000 7f71b5800700 1 -- 172.21.15.18:0/1168026586 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f71b0109dd0 msgr2=0x7f71b0103440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:21:15.016 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:15.020+0000 7f71b5800700 1 --2- 172.21.15.18:0/1168026586 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f71b0109dd0 0x7f71b0103440 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f719c004940 tx=0x7f719c009e30 comp rx=0 tx=0).stop 2024-04-12T21:21:15.016 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:15.020+0000 7f71b5800700 1 -- 172.21.15.18:0/1168026586 shutdown_connections 2024-04-12T21:21:15.016 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:15.020+0000 7f71b5800700 1 --2- 172.21.15.18:0/1168026586 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f71b0109dd0 0x7f71b0103440 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:21:15.016 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:15.020+0000 7f71b5800700 1 --2- 172.21.15.18:0/1168026586 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f71b0109430 0x7f71b0109800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:21:15.016 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:15.020+0000 7f71b5800700 1 -- 172.21.15.18:0/1168026586 >> 172.21.15.18:0/1168026586 conn(0x7f71b0072c00 msgr2=0x7f71b0075010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:21:15.016 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:15.020+0000 7f71b5800700 1 -- 172.21.15.18:0/1168026586 shutdown_connections 2024-04-12T21:21:15.016 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:15.020+0000 7f71b5800700 1 -- 172.21.15.18:0/1168026586 wait complete. 2024-04-12T21:21:15.018 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:15.021+0000 7f71b5800700 1 Processor -- start 2024-04-12T21:21:15.018 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:15.021+0000 7f71b5800700 1 -- start start 2024-04-12T21:21:15.018 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:15.021+0000 7f71b5800700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f71b0109430 0x7f71b0119f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:21:15.018 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:15.022+0000 7f71b5800700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f71b0109dd0 0x7f71b011a4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:21:15.018 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:15.022+0000 7f71b5800700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71b011ab10 con 0x7f71b0109dd0 2024-04-12T21:21:15.018 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:15.022+0000 7f71b5800700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71b01140f0 con 0x7f71b0109430 2024-04-12T21:21:15.018 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:15.022+0000 7f71ae000700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f71b0109dd0 0x7f71b011a4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:21:15.018 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:15.022+0000 7f71ae000700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f71b0109dd0 0x7f71b011a4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:38526/0 (socket says 172.21.15.18:38526) 2024-04-12T21:21:15.018 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:15.022+0000 7f71ae000700 1 -- 172.21.15.18:0/3248745170 learned_addr learned my addr 172.21.15.18:0/3248745170 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:21:15.019 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:15.022+0000 7f71aea00700 1 --2- 172.21.15.18:0/3248745170 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f71b0109430 0x7f71b0119f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:21:15.020 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:15.022+0000 7f71ae000700 1 -- 172.21.15.18:0/3248745170 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f71b0109430 msgr2=0x7f71b0119f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:21:15.020 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:15.022+0000 7f71ae000700 1 --2- 172.21.15.18:0/3248745170 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f71b0109430 0x7f71b0119f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:21:15.020 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:15.022+0000 7f71ae000700 1 -- 172.21.15.18:0/3248745170 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f719c005040 con 0x7f71b0109dd0 2024-04-12T21:21:15.020 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:15.023+0000 7f71ae000700 1 --2- 172.21.15.18:0/3248745170 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f71b0109dd0 0x7f71b011a4c0 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f719c009100 tx=0x7f719c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:21:15.020 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:15.023+0000 7f71a6a00700 1 -- 172.21.15.18:0/3248745170 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f719c00e040 con 0x7f71b0109dd0 2024-04-12T21:21:15.020 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:15.023+0000 7f71a6a00700 1 -- 172.21.15.18:0/3248745170 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f719c013420 con 0x7f71b0109dd0 2024-04-12T21:21:15.020 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:15.023+0000 7f71a6a00700 1 -- 172.21.15.18:0/3248745170 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f719c01c630 con 0x7f71b0109dd0 2024-04-12T21:21:15.020 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:15.023+0000 7f71b5800700 1 -- 172.21.15.18:0/3248745170 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f71b0114370 con 0x7f71b0109dd0 2024-04-12T21:21:15.020 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:15.023+0000 7f71b5800700 1 -- 172.21.15.18:0/3248745170 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f71b0114930 con 0x7f71b0109dd0 2024-04-12T21:21:15.021 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:15.024+0000 7f71b5800700 1 -- 172.21.15.18:0/3248745170 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f71b0104b80 con 0x7f71b0109dd0 2024-04-12T21:21:15.021 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:15.026+0000 7f71a6a00700 1 -- 172.21.15.18:0/3248745170 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7f719c013590 con 0x7f71b0109dd0 2024-04-12T21:21:15.023 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:15.027+0000 7f71a6a00700 1 --2- 172.21.15.18:0/3248745170 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f719406ec00 0x7f71940710b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:21:15.023 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:15.027+0000 7f71a6a00700 1 -- 172.21.15.18:0/3248745170 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 6757+0+0 (secure 0 0 0) 0x7f719c08fba0 con 0x7f71b0109dd0 2024-04-12T21:21:15.023 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:15.027+0000 7f71aea00700 1 --2- 172.21.15.18:0/3248745170 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f719406ec00 0x7f71940710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:21:15.024 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:15.028+0000 7f71aea00700 1 --2- 172.21.15.18:0/3248745170 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f719406ec00 0x7f71940710b0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f71a0004990 tx=0x7f71a0009ee0 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:21:15.028 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:15.032+0000 7f71a6a00700 1 -- 172.21.15.18:0/3248745170 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f719c05c3d0 con 0x7f71b0109dd0 2024-04-12T21:21:15.305 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:15.309+0000 7f71b5800700 1 -- 172.21.15.18:0/3248745170 --> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f71b0110570 con 0x7f719406ec00 2024-04-12T21:21:15.307 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:15.311+0000 7f71a6a00700 1 -- 172.21.15.18:0/3248745170 <== mgr.34102 v2:172.21.15.18:6800/3812891919 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f71b0110570 con 0x7f719406ec00 2024-04-12T21:21:15.311 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:15.315+0000 7f71b5800700 1 -- 172.21.15.18:0/3248745170 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f719406ec00 msgr2=0x7f71940710b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:21:15.311 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:15.315+0000 7f71b5800700 1 --2- 172.21.15.18:0/3248745170 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f719406ec00 0x7f71940710b0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f71a0004990 tx=0x7f71a0009ee0 comp rx=0 tx=0).stop 2024-04-12T21:21:15.311 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:15.315+0000 7f71b5800700 1 -- 172.21.15.18:0/3248745170 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f71b0109dd0 msgr2=0x7f71b011a4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:21:15.311 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:15.315+0000 7f71b5800700 1 --2- 172.21.15.18:0/3248745170 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f71b0109dd0 0x7f71b011a4c0 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f719c009100 tx=0x7f719c009e30 comp rx=0 tx=0).stop 2024-04-12T21:21:15.311 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:15.315+0000 7f71b5800700 1 -- 172.21.15.18:0/3248745170 shutdown_connections 2024-04-12T21:21:15.311 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:15.315+0000 7f71b5800700 1 --2- 172.21.15.18:0/3248745170 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f719406ec00 0x7f71940710b0 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:21:15.312 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:15.315+0000 7f71b5800700 1 --2- 172.21.15.18:0/3248745170 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f71b0109430 0x7f71b0119f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:21:15.312 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:15.315+0000 7f71b5800700 1 --2- 172.21.15.18:0/3248745170 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f71b0109dd0 0x7f71b011a4c0 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:21:15.312 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:15.315+0000 7f71b5800700 1 -- 172.21.15.18:0/3248745170 >> 172.21.15.18:0/3248745170 conn(0x7f71b0072c00 msgr2=0x7f71b0074c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:21:15.312 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:15.315+0000 7f71b5800700 1 -- 172.21.15.18:0/3248745170 shutdown_connections 2024-04-12T21:21:15.312 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:15.315+0000 7f71b5800700 1 -- 172.21.15.18:0/3248745170 wait complete. 2024-04-12T21:21:15.476 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:15.479+0000 7f9906400700 1 -- 172.21.15.18:0/3954697546 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9900064920 msgr2=0x7f99000fae70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:21:15.476 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:15.479+0000 7f9906400700 1 --2- 172.21.15.18:0/3954697546 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9900064920 0x7f99000fae70 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f98f4004a50 tx=0x7f98f400af40 comp rx=0 tx=0).stop 2024-04-12T21:21:15.476 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:15.480+0000 7f9906400700 1 -- 172.21.15.18:0/3954697546 shutdown_connections 2024-04-12T21:21:15.476 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:15.480+0000 7f9906400700 1 --2- 172.21.15.18:0/3954697546 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9900101780 0x7f99000fb3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:21:15.476 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:15.480+0000 7f9906400700 1 --2- 172.21.15.18:0/3954697546 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9900064920 0x7f99000fae70 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:21:15.476 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:15.480+0000 7f9906400700 1 -- 172.21.15.18:0/3954697546 >> 172.21.15.18:0/3954697546 conn(0x7f99000f69b0 msgr2=0x7f99000f8dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:21:15.476 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:15.480+0000 7f9906400700 1 -- 172.21.15.18:0/3954697546 shutdown_connections 2024-04-12T21:21:15.476 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:15.480+0000 7f9906400700 1 -- 172.21.15.18:0/3954697546 wait complete. 2024-04-12T21:21:15.478 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:15.481+0000 7f9906400700 1 Processor -- start 2024-04-12T21:21:15.478 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:15.481+0000 7f9906400700 1 -- start start 2024-04-12T21:21:15.478 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:15.481+0000 7f9906400700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9900064920 0x7f9900115b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:21:15.478 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:15.481+0000 7f9906400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9900101780 0x7f9900116070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:21:15.478 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:15.481+0000 7f9906400700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99001166c0 con 0x7f9900101780 2024-04-12T21:21:15.478 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:15.481+0000 7f9906400700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f990010fab0 con 0x7f9900064920 2024-04-12T21:21:15.478 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:15.481+0000 7f98ffe00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9900101780 0x7f9900116070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:21:15.478 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:15.482+0000 7f98ffe00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9900101780 0x7f9900116070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:38530/0 (socket says 172.21.15.18:38530) 2024-04-12T21:21:15.479 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:15.482+0000 7f98ffe00700 1 -- 172.21.15.18:0/3551367478 learned_addr learned my addr 172.21.15.18:0/3551367478 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:21:15.479 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:15.482+0000 7f9905000700 1 --2- 172.21.15.18:0/3551367478 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9900064920 0x7f9900115b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:21:15.479 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:15.482+0000 7f98ffe00700 1 -- 172.21.15.18:0/3551367478 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9900064920 msgr2=0x7f9900115b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:21:15.479 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:15.482+0000 7f98ffe00700 1 --2- 172.21.15.18:0/3551367478 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9900064920 0x7f9900115b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:21:15.479 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:15.482+0000 7f98ffe00700 1 -- 172.21.15.18:0/3551367478 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98f4005040 con 0x7f9900101780 2024-04-12T21:21:15.479 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:15.482+0000 7f98ffe00700 1 --2- 172.21.15.18:0/3551367478 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9900101780 0x7f9900116070 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f98ec002a50 tx=0x7f98ec0046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:21:15.480 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:15.483+0000 7f98fcc00700 1 -- 172.21.15.18:0/3551367478 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f98ec007cc0 con 0x7f9900101780 2024-04-12T21:21:15.480 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:15.483+0000 7f98fcc00700 1 -- 172.21.15.18:0/3551367478 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f98ec00f040 con 0x7f9900101780 2024-04-12T21:21:15.480 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:15.483+0000 7f9906400700 1 -- 172.21.15.18:0/3551367478 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f990010fd90 con 0x7f9900101780 2024-04-12T21:21:15.480 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:15.483+0000 7f98fcc00700 1 -- 172.21.15.18:0/3551367478 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f98ec009640 con 0x7f9900101780 2024-04-12T21:21:15.480 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:15.483+0000 7f9906400700 1 -- 172.21.15.18:0/3551367478 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f99001102d0 con 0x7f9900101780 2024-04-12T21:21:15.480 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:15.484+0000 7f9906400700 1 -- 172.21.15.18:0/3551367478 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f99000fc5b0 con 0x7f9900101780 2024-04-12T21:21:15.481 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:15.485+0000 7f98fcc00700 1 -- 172.21.15.18:0/3551367478 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7f98ec0097a0 con 0x7f9900101780 2024-04-12T21:21:15.482 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:15.487+0000 7f98fcc00700 1 --2- 172.21.15.18:0/3551367478 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f98e806ec00 0x7f98e80710b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:21:15.483 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:15.487+0000 7f98fcc00700 1 -- 172.21.15.18:0/3551367478 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 6757+0+0 (secure 0 0 0) 0x7f98ec08e420 con 0x7f9900101780 2024-04-12T21:21:15.483 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:15.487+0000 7f9905000700 1 --2- 172.21.15.18:0/3551367478 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f98e806ec00 0x7f98e80710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:21:15.483 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:15.488+0000 7f9905000700 1 --2- 172.21.15.18:0/3551367478 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f98e806ec00 0x7f98e80710b0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f98f400c3b0 tx=0x7f98f400af40 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:21:15.487 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:15.491+0000 7f98fcc00700 1 -- 172.21.15.18:0/3551367478 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f98ec05ac70 con 0x7f9900101780 2024-04-12T21:21:15.600 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:21:15 smithi018 ceph-mon[112590]: pgmap v643: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:21:15.727 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:15.731+0000 7f9906400700 1 -- 172.21.15.18:0/3551367478 --> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f9900107a30 con 0x7f98e806ec00 2024-04-12T21:21:15.757 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:15.761+0000 7f98fcc00700 1 -- 172.21.15.18:0/3551367478 <== mgr.34102 v2:172.21.15.18:6800/3812891919 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7f9900107a30 con 0x7f98e806ec00 2024-04-12T21:21:15.758 INFO:teuthology.orchestra.run.smithi018.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-12T21:21:15.758 INFO:teuthology.orchestra.run.smithi018.stdout:alertmanager.smithi018 smithi018 *:9093,9094 running (21m) 13m ago 36m 14.7M - 0.25.0 c8568f914cd2 c332090068e8 2024-04-12T21:21:15.758 INFO:teuthology.orchestra.run.smithi018.stdout:crash.smithi018 smithi018 running (18m) 13m ago 36m 6665k - 19.0.0-2879-g324baf8d 8717720dc5e5 b7f80d1f7982 2024-04-12T21:21:15.758 INFO:teuthology.orchestra.run.smithi018.stdout:crash.smithi186 smithi186 running (18m) 14m ago 35m 6660k - 19.0.0-2879-g324baf8d 8717720dc5e5 a4f55c81b623 2024-04-12T21:21:15.758 INFO:teuthology.orchestra.run.smithi018.stdout:grafana.smithi018 smithi018 *:3000 running (21m) 13m ago 35m 74.8M - 9.4.12 2bacad6d85d8 b08e3bf176dd 2024-04-12T21:21:15.758 INFO:teuthology.orchestra.run.smithi018.stdout:mds.cephfs.smithi018.acqbsc smithi018 running (14m) 13m ago 29m 11.1M - 19.0.0-2879-g324baf8d 8717720dc5e5 021f99ab24ae 2024-04-12T21:21:15.758 INFO:teuthology.orchestra.run.smithi018.stdout:mds.cephfs.smithi018.szjluo smithi018 running (13m) 13m ago 29m 10.9M - 19.0.0-2879-g324baf8d 8717720dc5e5 0003117479d4 2024-04-12T21:21:15.758 INFO:teuthology.orchestra.run.smithi018.stdout:mds.cephfs.smithi186.jdllyn smithi186 running (29m) 14m ago 29m 15.6M - 17.2.7-827-gd3b5060a 21de9b7369ad 2b2bc46bd29a 2024-04-12T21:21:15.758 INFO:teuthology.orchestra.run.smithi018.stdout:mds.cephfs.smithi186.wjnafs smithi186 running (29m) 14m ago 29m 15.4M - 17.2.7-827-gd3b5060a 21de9b7369ad 74b115d6d6fd 2024-04-12T21:21:15.758 INFO:teuthology.orchestra.run.smithi018.stdout:mgr.smithi018.xjzwga smithi018 *:8443,9283,8765 running (24m) 13m ago 38m 509M - 19.0.0-2879-g324baf8d 8717720dc5e5 56088305eab7 2024-04-12T21:21:15.758 INFO:teuthology.orchestra.run.smithi018.stdout:mgr.smithi186.kqcfxk smithi186 *:8443,9283,8765 running (23m) 14m ago 35m 452M - 19.0.0-2879-g324baf8d 8717720dc5e5 09cd766e5e0d 2024-04-12T21:21:15.758 INFO:teuthology.orchestra.run.smithi018.stdout:mon.smithi018 smithi018 running (19m) 13m ago 38m 57.0M 2048M 19.0.0-2879-g324baf8d 8717720dc5e5 df3e201dfed6 2024-04-12T21:21:15.758 INFO:teuthology.orchestra.run.smithi018.stdout:mon.smithi186 smithi186 running (19m) 14m ago 34m 37.7M 2048M 19.0.0-2879-g324baf8d 8717720dc5e5 c77498da9b2b 2024-04-12T21:21:15.759 INFO:teuthology.orchestra.run.smithi018.stdout:node-exporter.smithi018 smithi018 *:9100 running (22m) 13m ago 36m 7780k - 1.5.0 0da6a335fe13 326c3d9c6847 2024-04-12T21:21:15.759 INFO:teuthology.orchestra.run.smithi018.stdout:node-exporter.smithi186 smithi186 *:9100 running (22m) 14m ago 35m 7826k - 1.5.0 0da6a335fe13 ec2112baeecf 2024-04-12T21:21:15.759 INFO:teuthology.orchestra.run.smithi018.stdout:osd.0 smithi018 running (17m) 13m ago 33m 121M 1559M 19.0.0-2879-g324baf8d 8717720dc5e5 60b1273ca047 2024-04-12T21:21:15.759 INFO:teuthology.orchestra.run.smithi018.stdout:osd.1 smithi018 running (16m) 13m ago 33m 91.3M 1559M 19.0.0-2879-g324baf8d 8717720dc5e5 170ce1e5fb8d 2024-04-12T21:21:15.759 INFO:teuthology.orchestra.run.smithi018.stdout:osd.2 smithi018 running (16m) 13m ago 32m 90.3M 1559M 19.0.0-2879-g324baf8d 8717720dc5e5 c2cbc6aa2839 2024-04-12T21:21:15.759 INFO:teuthology.orchestra.run.smithi018.stdout:osd.3 smithi186 running (15m) 14m ago 32m 96.6M 2581M 19.0.0-2879-g324baf8d 8717720dc5e5 0138e8262e4b 2024-04-12T21:21:15.759 INFO:teuthology.orchestra.run.smithi018.stdout:osd.4 smithi186 running (15m) 14m ago 31m 87.2M 2581M 19.0.0-2879-g324baf8d 8717720dc5e5 b20cb63b25c9 2024-04-12T21:21:15.759 INFO:teuthology.orchestra.run.smithi018.stdout:osd.5 smithi186 running (14m) 14m ago 31m 11.1M 2581M 19.0.0-2879-g324baf8d 8717720dc5e5 2513bea2610e 2024-04-12T21:21:15.759 INFO:teuthology.orchestra.run.smithi018.stdout:prometheus.smithi018 smithi018 *:9095 running (22m) 13m ago 35m 60.6M - 2.43.0 a07b618ecd1d f58841127e21 2024-04-12T21:21:15.761 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:15.765+0000 7f9906400700 1 -- 172.21.15.18:0/3551367478 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f98e806ec00 msgr2=0x7f98e80710b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:21:15.761 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:15.765+0000 7f9906400700 1 --2- 172.21.15.18:0/3551367478 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f98e806ec00 0x7f98e80710b0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f98f400c3b0 tx=0x7f98f400af40 comp rx=0 tx=0).stop 2024-04-12T21:21:15.761 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:15.765+0000 7f9906400700 1 -- 172.21.15.18:0/3551367478 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9900101780 msgr2=0x7f9900116070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:21:15.761 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:15.765+0000 7f9906400700 1 --2- 172.21.15.18:0/3551367478 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9900101780 0x7f9900116070 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f98ec002a50 tx=0x7f98ec0046b0 comp rx=0 tx=0).stop 2024-04-12T21:21:15.761 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:15.765+0000 7f9906400700 1 -- 172.21.15.18:0/3551367478 shutdown_connections 2024-04-12T21:21:15.761 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:15.765+0000 7f9906400700 1 --2- 172.21.15.18:0/3551367478 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f98e806ec00 0x7f98e80710b0 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:21:15.762 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:15.765+0000 7f9906400700 1 --2- 172.21.15.18:0/3551367478 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9900064920 0x7f9900115b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:21:15.762 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:15.765+0000 7f9906400700 1 --2- 172.21.15.18:0/3551367478 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9900101780 0x7f9900116070 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:21:15.762 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:15.765+0000 7f9906400700 1 -- 172.21.15.18:0/3551367478 >> 172.21.15.18:0/3551367478 conn(0x7f99000f69b0 msgr2=0x7f99000f7830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:21:15.762 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:15.765+0000 7f9906400700 1 -- 172.21.15.18:0/3551367478 shutdown_connections 2024-04-12T21:21:15.762 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:15.765+0000 7f9906400700 1 -- 172.21.15.18:0/3551367478 wait complete. 2024-04-12T21:21:15.778 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:21:15 smithi186 ceph-mon[86379]: pgmap v643: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:21:15.922 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:15.925+0000 7f87e9400700 1 -- 172.21.15.18:0/2552654463 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f87e4104e70 msgr2=0x7f87e4105240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:21:15.922 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:15.925+0000 7f87e9400700 1 --2- 172.21.15.18:0/2552654463 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f87e4104e70 0x7f87e4105240 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f87d4004940 tx=0x7f87d4009e30 comp rx=0 tx=0).stop 2024-04-12T21:21:15.922 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:15.926+0000 7f87e9400700 1 -- 172.21.15.18:0/2552654463 shutdown_connections 2024-04-12T21:21:15.922 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:15.926+0000 7f87e9400700 1 --2- 172.21.15.18:0/2552654463 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f87e4105810 0x7f87e40feed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:21:15.922 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:15.926+0000 7f87e9400700 1 --2- 172.21.15.18:0/2552654463 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f87e4104e70 0x7f87e4105240 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:21:15.922 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:15.926+0000 7f87e9400700 1 -- 172.21.15.18:0/2552654463 >> 172.21.15.18:0/2552654463 conn(0x7f87e40fab10 msgr2=0x7f87e40fcef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:21:15.922 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:15.926+0000 7f87e9400700 1 -- 172.21.15.18:0/2552654463 shutdown_connections 2024-04-12T21:21:15.923 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:15.926+0000 7f87e9400700 1 -- 172.21.15.18:0/2552654463 wait complete. 2024-04-12T21:21:15.923 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:15.927+0000 7f87e9400700 1 Processor -- start 2024-04-12T21:21:15.923 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:15.927+0000 7f87e9400700 1 -- start start 2024-04-12T21:21:15.924 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:15.928+0000 7f87e9400700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f87e4104e70 0x7f87e4115bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:21:15.924 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:15.928+0000 7f87e9400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f87e4105810 0x7f87e41160f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:21:15.924 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:15.928+0000 7f87e9400700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87e41167d0 con 0x7f87e4105810 2024-04-12T21:21:15.924 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:15.928+0000 7f87e9400700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87e410fb30 con 0x7f87e4104e70 2024-04-12T21:21:15.924 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:15.928+0000 7f87e3400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f87e4105810 0x7f87e41160f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:21:15.924 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:15.928+0000 7f87e3400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f87e4105810 0x7f87e41160f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:38548/0 (socket says 172.21.15.18:38548) 2024-04-12T21:21:15.925 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:15.929+0000 7f87e3400700 1 -- 172.21.15.18:0/3652630368 learned_addr learned my addr 172.21.15.18:0/3652630368 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:21:15.926 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:15.929+0000 7f87e3e00700 1 --2- 172.21.15.18:0/3652630368 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f87e4104e70 0x7f87e4115bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:21:15.926 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:15.929+0000 7f87e3400700 1 -- 172.21.15.18:0/3652630368 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f87e4104e70 msgr2=0x7f87e4115bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:21:15.926 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:15.929+0000 7f87e3400700 1 --2- 172.21.15.18:0/3652630368 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f87e4104e70 0x7f87e4115bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:21:15.926 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:15.929+0000 7f87e3400700 1 -- 172.21.15.18:0/3652630368 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87d4005040 con 0x7f87e4105810 2024-04-12T21:21:15.926 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:15.929+0000 7f87e3400700 1 --2- 172.21.15.18:0/3652630368 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f87e4105810 0x7f87e41160f0 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f87d000b790 tx=0x7f87d0009220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:21:15.926 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:15.930+0000 7f87dbe00700 1 -- 172.21.15.18:0/3652630368 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f87d000f6f0 con 0x7f87e4105810 2024-04-12T21:21:15.926 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:15.930+0000 7f87dbe00700 1 -- 172.21.15.18:0/3652630368 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f87d000fd20 con 0x7f87e4105810 2024-04-12T21:21:15.926 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:15.930+0000 7f87e9400700 1 -- 172.21.15.18:0/3652630368 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f87e410fe10 con 0x7f87e4105810 2024-04-12T21:21:15.926 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:15.930+0000 7f87dbe00700 1 -- 172.21.15.18:0/3652630368 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f87d00103a0 con 0x7f87e4105810 2024-04-12T21:21:15.926 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:15.930+0000 7f87e9400700 1 -- 172.21.15.18:0/3652630368 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f87e41103d0 con 0x7f87e4105810 2024-04-12T21:21:15.927 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:15.931+0000 7f87e9400700 1 -- 172.21.15.18:0/3652630368 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f87e41005c0 con 0x7f87e4105810 2024-04-12T21:21:15.928 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:15.932+0000 7f87dbe00700 1 -- 172.21.15.18:0/3652630368 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7f87d0011400 con 0x7f87e4105810 2024-04-12T21:21:15.929 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:15.933+0000 7f87dbe00700 1 --2- 172.21.15.18:0/3652630368 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f87c806ec00 0x7f87c80710b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:21:15.929 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:15.933+0000 7f87dbe00700 1 -- 172.21.15.18:0/3652630368 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 6757+0+0 (secure 0 0 0) 0x7f87d008e780 con 0x7f87e4105810 2024-04-12T21:21:15.929 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:15.934+0000 7f87e3e00700 1 --2- 172.21.15.18:0/3652630368 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f87c806ec00 0x7f87c80710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:21:15.930 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:15.934+0000 7f87e3e00700 1 --2- 172.21.15.18:0/3652630368 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f87c806ec00 0x7f87c80710b0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f87d400b020 tx=0x7f87d4009e30 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:21:15.934 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:15.938+0000 7f87dbe00700 1 -- 172.21.15.18:0/3652630368 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f87d005afd0 con 0x7f87e4105810 2024-04-12T21:21:16.246 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:16.249+0000 7f87e9400700 1 -- 172.21.15.18:0/3652630368 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f87e410bfe0 con 0x7f87e4105810 2024-04-12T21:21:16.247 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:16.251+0000 7f87dbe00700 1 -- 172.21.15.18:0/3652630368 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+837 (secure 0 0 0) 0x7f87d005ab60 con 0x7f87e4105810 2024-04-12T21:21:16.247 INFO:teuthology.orchestra.run.smithi018.stdout:{ 2024-04-12T21:21:16.247 INFO:teuthology.orchestra.run.smithi018.stdout: "mon": { 2024-04-12T21:21:16.247 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 19.0.0-2879-g324baf8d (324baf8ddb91dea46cac50cb535c306115791355) squid (dev)": 2 2024-04-12T21:21:16.248 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-12T21:21:16.248 INFO:teuthology.orchestra.run.smithi018.stdout: "mgr": { 2024-04-12T21:21:16.248 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 19.0.0-2879-g324baf8d (324baf8ddb91dea46cac50cb535c306115791355) squid (dev)": 2 2024-04-12T21:21:16.248 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-12T21:21:16.248 INFO:teuthology.orchestra.run.smithi018.stdout: "osd": { 2024-04-12T21:21:16.248 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 19.0.0-2879-g324baf8d (324baf8ddb91dea46cac50cb535c306115791355) squid (dev)": 6 2024-04-12T21:21:16.248 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-12T21:21:16.248 INFO:teuthology.orchestra.run.smithi018.stdout: "mds": { 2024-04-12T21:21:16.248 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 17.2.7-827-gd3b5060a (d3b5060a17449edfae9385cdf792afec109fca5d) quincy (stable)": 2, 2024-04-12T21:21:16.248 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 19.0.0-2879-g324baf8d (324baf8ddb91dea46cac50cb535c306115791355) squid (dev)": 2 2024-04-12T21:21:16.248 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-12T21:21:16.248 INFO:teuthology.orchestra.run.smithi018.stdout: "overall": { 2024-04-12T21:21:16.248 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 17.2.7-827-gd3b5060a (d3b5060a17449edfae9385cdf792afec109fca5d) quincy (stable)": 2, 2024-04-12T21:21:16.248 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 19.0.0-2879-g324baf8d (324baf8ddb91dea46cac50cb535c306115791355) squid (dev)": 12 2024-04-12T21:21:16.248 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-04-12T21:21:16.249 INFO:teuthology.orchestra.run.smithi018.stdout:} 2024-04-12T21:21:16.250 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:16.254+0000 7f87e9400700 1 -- 172.21.15.18:0/3652630368 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f87c806ec00 msgr2=0x7f87c80710b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:21:16.250 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:16.254+0000 7f87e9400700 1 --2- 172.21.15.18:0/3652630368 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f87c806ec00 0x7f87c80710b0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f87d400b020 tx=0x7f87d4009e30 comp rx=0 tx=0).stop 2024-04-12T21:21:16.250 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:16.254+0000 7f87e9400700 1 -- 172.21.15.18:0/3652630368 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f87e4105810 msgr2=0x7f87e41160f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:21:16.250 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:16.254+0000 7f87e9400700 1 --2- 172.21.15.18:0/3652630368 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f87e4105810 0x7f87e41160f0 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f87d000b790 tx=0x7f87d0009220 comp rx=0 tx=0).stop 2024-04-12T21:21:16.251 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:16.254+0000 7f87e9400700 1 -- 172.21.15.18:0/3652630368 shutdown_connections 2024-04-12T21:21:16.251 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:16.254+0000 7f87e9400700 1 --2- 172.21.15.18:0/3652630368 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f87c806ec00 0x7f87c80710b0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:21:16.251 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:16.254+0000 7f87e9400700 1 --2- 172.21.15.18:0/3652630368 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f87e4104e70 0x7f87e4115bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:21:16.251 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:16.254+0000 7f87e9400700 1 --2- 172.21.15.18:0/3652630368 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f87e4105810 0x7f87e41160f0 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:21:16.251 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:16.254+0000 7f87e9400700 1 -- 172.21.15.18:0/3652630368 >> 172.21.15.18:0/3652630368 conn(0x7f87e40fab10 msgr2=0x7f87e40fca10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:21:16.251 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:16.254+0000 7f87e9400700 1 -- 172.21.15.18:0/3652630368 shutdown_connections 2024-04-12T21:21:16.251 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:16.254+0000 7f87e9400700 1 -- 172.21.15.18:0/3652630368 wait complete. 2024-04-12T21:21:16.409 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:16.413+0000 7fd727200700 1 -- 172.21.15.18:0/1435668378 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd7201037a0 msgr2=0x7fd7200fce10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:21:16.409 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:16.413+0000 7fd727200700 1 --2- 172.21.15.18:0/1435668378 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd7201037a0 0x7fd7200fce10 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7fd70c004940 tx=0x7fd70c009e30 comp rx=0 tx=0).stop 2024-04-12T21:21:16.410 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:16.413+0000 7fd727200700 1 -- 172.21.15.18:0/1435668378 shutdown_connections 2024-04-12T21:21:16.410 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:16.413+0000 7fd727200700 1 --2- 172.21.15.18:0/1435668378 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd7201037a0 0x7fd7200fce10 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:21:16.410 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:16.413+0000 7fd727200700 1 --2- 172.21.15.18:0/1435668378 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd720102e00 0x7fd7201031d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:21:16.410 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:16.413+0000 7fd727200700 1 -- 172.21.15.18:0/1435668378 >> 172.21.15.18:0/1435668378 conn(0x7fd7200f89b0 msgr2=0x7fd7200fadc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:21:16.410 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:16.413+0000 7fd727200700 1 -- 172.21.15.18:0/1435668378 shutdown_connections 2024-04-12T21:21:16.410 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:16.413+0000 7fd727200700 1 -- 172.21.15.18:0/1435668378 wait complete. 2024-04-12T21:21:16.411 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:16.414+0000 7fd727200700 1 Processor -- start 2024-04-12T21:21:16.411 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:16.414+0000 7fd727200700 1 -- start start 2024-04-12T21:21:16.412 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:16.415+0000 7fd727200700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd720102e00 0x7fd7200702c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:21:16.412 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:16.415+0000 7fd727200700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd7201037a0 0x7fd720070800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:21:16.412 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:16.415+0000 7fd727200700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd720070d40 con 0x7fd720102e00 2024-04-12T21:21:16.412 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:16.415+0000 7fd727200700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd720070e80 con 0x7fd7201037a0 2024-04-12T21:21:16.412 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:16.415+0000 7fd71fe00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd720102e00 0x7fd7200702c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:21:16.412 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:16.415+0000 7fd71fe00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd720102e00 0x7fd7200702c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:38574/0 (socket says 172.21.15.18:38574) 2024-04-12T21:21:16.412 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:16.415+0000 7fd71fe00700 1 -- 172.21.15.18:0/1278958250 learned_addr learned my addr 172.21.15.18:0/1278958250 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:21:16.412 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:16.416+0000 7fd71f400700 1 --2- 172.21.15.18:0/1278958250 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd7201037a0 0x7fd720070800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:21:16.413 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:16.416+0000 7fd71fe00700 1 -- 172.21.15.18:0/1278958250 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd7201037a0 msgr2=0x7fd720070800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:21:16.413 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:16.416+0000 7fd71fe00700 1 --2- 172.21.15.18:0/1278958250 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd7201037a0 0x7fd720070800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:21:16.413 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:16.416+0000 7fd71fe00700 1 -- 172.21.15.18:0/1278958250 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd70c005040 con 0x7fd720102e00 2024-04-12T21:21:16.413 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:16.416+0000 7fd71fe00700 1 --2- 172.21.15.18:0/1278958250 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd720102e00 0x7fd7200702c0 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7fd710002a50 tx=0x7fd7100046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:21:16.413 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:16.417+0000 7fd717e00700 1 -- 172.21.15.18:0/1278958250 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd710007cc0 con 0x7fd720102e00 2024-04-12T21:21:16.414 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:16.417+0000 7fd717e00700 1 -- 172.21.15.18:0/1278958250 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd71000f040 con 0x7fd720102e00 2024-04-12T21:21:16.414 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:16.417+0000 7fd717e00700 1 -- 172.21.15.18:0/1278958250 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd710009640 con 0x7fd720102e00 2024-04-12T21:21:16.414 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:16.417+0000 7fd727200700 1 -- 172.21.15.18:0/1278958250 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd720071130 con 0x7fd720102e00 2024-04-12T21:21:16.414 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:16.417+0000 7fd727200700 1 -- 172.21.15.18:0/1278958250 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd72011a340 con 0x7fd720102e00 2024-04-12T21:21:16.415 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:16.419+0000 7fd727200700 1 -- 172.21.15.18:0/1278958250 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd7201052d0 con 0x7fd720102e00 2024-04-12T21:21:16.415 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:16.419+0000 7fd717e00700 1 -- 172.21.15.18:0/1278958250 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7fd7100097c0 con 0x7fd720102e00 2024-04-12T21:21:16.416 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:16.420+0000 7fd717e00700 1 --2- 172.21.15.18:0/1278958250 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fd70406ec10 0x7fd7040710c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:21:16.416 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:16.420+0000 7fd717e00700 1 -- 172.21.15.18:0/1278958250 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 6757+0+0 (secure 0 0 0) 0x7fd71008e630 con 0x7fd720102e00 2024-04-12T21:21:16.416 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:16.421+0000 7fd71f400700 1 --2- 172.21.15.18:0/1278958250 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fd70406ec10 0x7fd7040710c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:21:16.417 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:16.422+0000 7fd71f400700 1 --2- 172.21.15.18:0/1278958250 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fd70406ec10 0x7fd7040710c0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fd70c00b020 tx=0x7fd70c009e30 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:21:16.421 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:16.425+0000 7fd717e00700 1 -- 172.21.15.18:0/1278958250 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7fd7100582f0 con 0x7fd720102e00 2024-04-12T21:21:16.599 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:21:16 smithi018 ceph-mon[112590]: from='client.35144 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:21:16.600 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:21:16 smithi018 ceph-mon[112590]: from='client.35148 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:21:16.600 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:21:16 smithi018 ceph-mon[112590]: from='client.35152 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:21:16.600 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:21:16 smithi018 ceph-mon[112590]: from='client.? 172.21.15.18:0/3652630368' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T21:21:16.705 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:16.709+0000 7fd727200700 1 -- 172.21.15.18:0/1278958250 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fd720109f70 con 0x7fd720102e00 2024-04-12T21:21:16.705 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:16.709+0000 7fd717e00700 1 -- 172.21.15.18:0/1278958250 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 29 v29) v1 ==== 76+0+1872 (secure 0 0 0) 0x7fd71005b9b0 con 0x7fd720102e00 2024-04-12T21:21:16.706 INFO:teuthology.orchestra.run.smithi018.stdout:e29 2024-04-12T21:21:16.706 INFO:teuthology.orchestra.run.smithi018.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-12T21:21:16.706 INFO:teuthology.orchestra.run.smithi018.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-12T21:21:16.706 INFO:teuthology.orchestra.run.smithi018.stdout:legacy client fscid: 1 2024-04-12T21:21:16.706 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-12T21:21:16.706 INFO:teuthology.orchestra.run.smithi018.stdout:Filesystem 'cephfs' (1) 2024-04-12T21:21:16.706 INFO:teuthology.orchestra.run.smithi018.stdout:fs_name cephfs 2024-04-12T21:21:16.707 INFO:teuthology.orchestra.run.smithi018.stdout:epoch 24 2024-04-12T21:21:16.707 INFO:teuthology.orchestra.run.smithi018.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-04-12T21:21:16.707 INFO:teuthology.orchestra.run.smithi018.stdout:created 2024-04-12T20:51:57.362850+0000 2024-04-12T21:21:16.707 INFO:teuthology.orchestra.run.smithi018.stdout:modified 2024-04-12T21:06:59.717190+0000 2024-04-12T21:21:16.707 INFO:teuthology.orchestra.run.smithi018.stdout:tableserver 0 2024-04-12T21:21:16.707 INFO:teuthology.orchestra.run.smithi018.stdout:root 0 2024-04-12T21:21:16.707 INFO:teuthology.orchestra.run.smithi018.stdout:session_timeout 60 2024-04-12T21:21:16.707 INFO:teuthology.orchestra.run.smithi018.stdout:session_autoclose 300 2024-04-12T21:21:16.707 INFO:teuthology.orchestra.run.smithi018.stdout:max_file_size 1099511627776 2024-04-12T21:21:16.707 INFO:teuthology.orchestra.run.smithi018.stdout:max_xattr_size 65536 2024-04-12T21:21:16.707 INFO:teuthology.orchestra.run.smithi018.stdout:required_client_features {} 2024-04-12T21:21:16.707 INFO:teuthology.orchestra.run.smithi018.stdout:last_failure 0 2024-04-12T21:21:16.707 INFO:teuthology.orchestra.run.smithi018.stdout:last_failure_osd_epoch 100 2024-04-12T21:21:16.708 INFO:teuthology.orchestra.run.smithi018.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-12T21:21:16.708 INFO:teuthology.orchestra.run.smithi018.stdout:max_mds 2 2024-04-12T21:21:16.708 INFO:teuthology.orchestra.run.smithi018.stdout:in 0,1 2024-04-12T21:21:16.708 INFO:teuthology.orchestra.run.smithi018.stdout:up {} 2024-04-12T21:21:16.708 INFO:teuthology.orchestra.run.smithi018.stdout:failed 0,1 2024-04-12T21:21:16.708 INFO:teuthology.orchestra.run.smithi018.stdout:damaged 2024-04-12T21:21:16.708 INFO:teuthology.orchestra.run.smithi018.stdout:stopped 2024-04-12T21:21:16.708 INFO:teuthology.orchestra.run.smithi018.stdout:data_pools [3] 2024-04-12T21:21:16.708 INFO:teuthology.orchestra.run.smithi018.stdout:metadata_pool 2 2024-04-12T21:21:16.708 INFO:teuthology.orchestra.run.smithi018.stdout:inline_data disabled 2024-04-12T21:21:16.708 INFO:teuthology.orchestra.run.smithi018.stdout:balancer 2024-04-12T21:21:16.708 INFO:teuthology.orchestra.run.smithi018.stdout:bal_rank_mask -1 2024-04-12T21:21:16.708 INFO:teuthology.orchestra.run.smithi018.stdout:standby_count_wanted 1 2024-04-12T21:21:16.709 INFO:teuthology.orchestra.run.smithi018.stdout:qdb_cluster leader: 0 members: 2024-04-12T21:21:16.709 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-12T21:21:16.709 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-12T21:21:16.709 INFO:teuthology.orchestra.run.smithi018.stdout:Standby daemons: 2024-04-12T21:21:16.709 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-12T21:21:16.709 INFO:teuthology.orchestra.run.smithi018.stdout:[mds.cephfs.smithi186.wjnafs{-1:24277} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.186:6826/438347207,v1:172.21.15.186:6827/438347207] compat {c=[1],r=[1],i=[7ff]}] 2024-04-12T21:21:16.709 INFO:teuthology.orchestra.run.smithi018.stdout:[mds.cephfs.smithi186.jdllyn{-1:24281} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.186:6824/1464359952,v1:172.21.15.186:6825/1464359952] compat {c=[1],r=[1],i=[7ff]}] 2024-04-12T21:21:16.709 INFO:teuthology.orchestra.run.smithi018.stdout:[mds.cephfs.smithi018.acqbsc{-1:34436} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.18:6828/755179764,v1:172.21.15.18:6829/755179764] compat {c=[1],r=[1],i=[1fff]}] 2024-04-12T21:21:16.709 INFO:teuthology.orchestra.run.smithi018.stdout:[mds.cephfs.smithi018.szjluo{-1:34468} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.18:6826/2407992653,v1:172.21.15.18:6827/2407992653] compat {c=[1],r=[1],i=[1fff]}] 2024-04-12T21:21:16.709 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:16.714+0000 7fd727200700 1 -- 172.21.15.18:0/1278958250 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fd70406ec10 msgr2=0x7fd7040710c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:21:16.710 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:16.714+0000 7fd727200700 1 --2- 172.21.15.18:0/1278958250 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fd70406ec10 0x7fd7040710c0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fd70c00b020 tx=0x7fd70c009e30 comp rx=0 tx=0).stop 2024-04-12T21:21:16.710 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:16.714+0000 7fd727200700 1 -- 172.21.15.18:0/1278958250 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd720102e00 msgr2=0x7fd7200702c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:21:16.710 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:16.714+0000 7fd727200700 1 --2- 172.21.15.18:0/1278958250 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd720102e00 0x7fd7200702c0 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7fd710002a50 tx=0x7fd7100046b0 comp rx=0 tx=0).stop 2024-04-12T21:21:16.711 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:16.714+0000 7fd727200700 1 -- 172.21.15.18:0/1278958250 shutdown_connections 2024-04-12T21:21:16.711 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:16.714+0000 7fd727200700 1 --2- 172.21.15.18:0/1278958250 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fd70406ec10 0x7fd7040710c0 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:21:16.711 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:16.714+0000 7fd727200700 1 --2- 172.21.15.18:0/1278958250 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd720102e00 0x7fd7200702c0 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:21:16.711 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:16.714+0000 7fd727200700 1 --2- 172.21.15.18:0/1278958250 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd7201037a0 0x7fd720070800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:21:16.711 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:16.714+0000 7fd727200700 1 -- 172.21.15.18:0/1278958250 >> 172.21.15.18:0/1278958250 conn(0x7fd7200f89b0 msgr2=0x7fd720102210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:21:16.712 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:16.714+0000 7fd727200700 1 -- 172.21.15.18:0/1278958250 shutdown_connections 2024-04-12T21:21:16.712 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:16.714+0000 7fd727200700 1 -- 172.21.15.18:0/1278958250 wait complete. 2024-04-12T21:21:16.712 INFO:teuthology.orchestra.run.smithi018.stderr:dumped fsmap epoch 29 2024-04-12T21:21:16.778 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:21:16 smithi186 ceph-mon[86379]: from='client.35144 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:21:16.778 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:21:16 smithi186 ceph-mon[86379]: from='client.35148 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:21:16.779 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:21:16 smithi186 ceph-mon[86379]: from='client.35152 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:21:16.779 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:21:16 smithi186 ceph-mon[86379]: from='client.? 172.21.15.18:0/3652630368' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T21:21:16.863 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:16.866+0000 7f39d4e00700 1 -- 172.21.15.18:0/2454138823 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f39d0104ff0 msgr2=0x7f39d01053c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:21:16.863 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:16.866+0000 7f39d4e00700 1 --2- 172.21.15.18:0/2454138823 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f39d0104ff0 0x7f39d01053c0 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f39c0009ce0 tx=0x7f39c0004d10 comp rx=0 tx=0).stop 2024-04-12T21:21:16.863 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:16.866+0000 7f39d4e00700 1 -- 172.21.15.18:0/2454138823 shutdown_connections 2024-04-12T21:21:16.863 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:16.866+0000 7f39d4e00700 1 --2- 172.21.15.18:0/2454138823 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f39d0105990 0x7f39d00ff000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:21:16.863 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:16.866+0000 7f39d4e00700 1 --2- 172.21.15.18:0/2454138823 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f39d0104ff0 0x7f39d01053c0 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:21:16.863 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:16.866+0000 7f39d4e00700 1 -- 172.21.15.18:0/2454138823 >> 172.21.15.18:0/2454138823 conn(0x7f39d00faba0 msgr2=0x7f39d00fcfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:21:16.863 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:16.866+0000 7f39d4e00700 1 -- 172.21.15.18:0/2454138823 shutdown_connections 2024-04-12T21:21:16.864 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:16.866+0000 7f39d4e00700 1 -- 172.21.15.18:0/2454138823 wait complete. 2024-04-12T21:21:16.864 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:16.867+0000 7f39d4e00700 1 Processor -- start 2024-04-12T21:21:16.864 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:16.867+0000 7f39d4e00700 1 -- start start 2024-04-12T21:21:16.865 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:16.867+0000 7f39d4e00700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f39d0104ff0 0x7f39d0115cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:21:16.865 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:16.867+0000 7f39d4e00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f39d0105990 0x7f39d01161f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:21:16.865 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:16.867+0000 7f39d4e00700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39d0116840 con 0x7f39d0105990 2024-04-12T21:21:16.865 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:16.867+0000 7f39d4e00700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39d010fe60 con 0x7f39d0104ff0 2024-04-12T21:21:16.865 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:16.868+0000 7f39c5400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f39d0105990 0x7f39d01161f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:21:16.866 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:16.868+0000 7f39c5400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f39d0105990 0x7f39d01161f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:38600/0 (socket says 172.21.15.18:38600) 2024-04-12T21:21:16.866 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:16.868+0000 7f39c5400700 1 -- 172.21.15.18:0/1664337131 learned_addr learned my addr 172.21.15.18:0/1664337131 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:21:16.866 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:16.868+0000 7f39cde00700 1 --2- 172.21.15.18:0/1664337131 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f39d0104ff0 0x7f39d0115cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:21:16.866 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:16.868+0000 7f39c5400700 1 -- 172.21.15.18:0/1664337131 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f39d0104ff0 msgr2=0x7f39d0115cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:21:16.866 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:16.868+0000 7f39c5400700 1 --2- 172.21.15.18:0/1664337131 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f39d0104ff0 0x7f39d0115cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:21:16.866 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:16.868+0000 7f39c5400700 1 -- 172.21.15.18:0/1664337131 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39c0009940 con 0x7f39d0105990 2024-04-12T21:21:16.866 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:16.869+0000 7f39c5400700 1 --2- 172.21.15.18:0/1664337131 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f39d0105990 0x7f39d01161f0 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f39b800c800 tx=0x7f39b8004470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:21:16.866 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:16.869+0000 7f39c6a00700 1 -- 172.21.15.18:0/1664337131 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f39b800a760 con 0x7f39d0105990 2024-04-12T21:21:16.866 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:16.869+0000 7f39c6a00700 1 -- 172.21.15.18:0/1664337131 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f39b800ad90 con 0x7f39d0105990 2024-04-12T21:21:16.866 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:16.869+0000 7f39c6a00700 1 -- 172.21.15.18:0/1664337131 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f39b80113a0 con 0x7f39d0105990 2024-04-12T21:21:16.866 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:16.869+0000 7f39d4e00700 1 -- 172.21.15.18:0/1664337131 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f39d0110140 con 0x7f39d0105990 2024-04-12T21:21:16.866 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:16.869+0000 7f39d4e00700 1 -- 172.21.15.18:0/1664337131 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f39d0110700 con 0x7f39d0105990 2024-04-12T21:21:16.867 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:16.870+0000 7f39d4e00700 1 -- 172.21.15.18:0/1664337131 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f39d0100740 con 0x7f39d0105990 2024-04-12T21:21:16.867 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:16.872+0000 7f39c6a00700 1 -- 172.21.15.18:0/1664337131 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7f39b8012400 con 0x7f39d0105990 2024-04-12T21:21:16.868 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:16.872+0000 7f39c6a00700 1 --2- 172.21.15.18:0/1664337131 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f39b406ec00 0x7f39b40710b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:21:16.868 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:16.873+0000 7f39c6a00700 1 -- 172.21.15.18:0/1664337131 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 6757+0+0 (secure 0 0 0) 0x7f39b808e8d0 con 0x7f39d0105990 2024-04-12T21:21:16.868 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:16.873+0000 7f39cde00700 1 --2- 172.21.15.18:0/1664337131 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f39b406ec00 0x7f39b40710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:21:16.869 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:16.873+0000 7f39cde00700 1 --2- 172.21.15.18:0/1664337131 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f39b406ec00 0x7f39b40710b0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f39c0005010 tx=0x7f39c0004d10 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:21:16.873 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:16.877+0000 7f39c6a00700 1 -- 172.21.15.18:0/1664337131 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f39b8057690 con 0x7f39d0105990 2024-04-12T21:21:17.112 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:17.115+0000 7f39d4e00700 1 -- 172.21.15.18:0/1664337131 --> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f39d010c0f0 con 0x7f39b406ec00 2024-04-12T21:21:17.114 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:17.118+0000 7f39c6a00700 1 -- 172.21.15.18:0/1664337131 <== mgr.34102 v2:172.21.15.18:6800/3812891919 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f39d010c0f0 con 0x7f39b406ec00 2024-04-12T21:21:17.114 INFO:teuthology.orchestra.run.smithi018.stdout:{ 2024-04-12T21:21:17.114 INFO:teuthology.orchestra.run.smithi018.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:f4452def3be4d350f4a8e5feaf7ebb1511be9298fe68478ba7a2d6fbd7d69a63", 2024-04-12T21:21:17.114 INFO:teuthology.orchestra.run.smithi018.stdout: "in_progress": true, 2024-04-12T21:21:17.114 INFO:teuthology.orchestra.run.smithi018.stdout: "which": "Upgrading all daemon types on all hosts", 2024-04-12T21:21:17.114 INFO:teuthology.orchestra.run.smithi018.stdout: "services_complete": [ 2024-04-12T21:21:17.114 INFO:teuthology.orchestra.run.smithi018.stdout: "crash", 2024-04-12T21:21:17.114 INFO:teuthology.orchestra.run.smithi018.stdout: "osd", 2024-04-12T21:21:17.114 INFO:teuthology.orchestra.run.smithi018.stdout: "mon", 2024-04-12T21:21:17.114 INFO:teuthology.orchestra.run.smithi018.stdout: "mgr" 2024-04-12T21:21:17.114 INFO:teuthology.orchestra.run.smithi018.stdout: ], 2024-04-12T21:21:17.115 INFO:teuthology.orchestra.run.smithi018.stdout: "progress": "14/21 daemons upgraded", 2024-04-12T21:21:17.115 INFO:teuthology.orchestra.run.smithi018.stdout: "message": "Currently upgrading mds daemons", 2024-04-12T21:21:17.115 INFO:teuthology.orchestra.run.smithi018.stdout: "is_paused": false 2024-04-12T21:21:17.115 INFO:teuthology.orchestra.run.smithi018.stdout:} 2024-04-12T21:21:17.116 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:17.120+0000 7f39d4e00700 1 -- 172.21.15.18:0/1664337131 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f39b406ec00 msgr2=0x7f39b40710b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:21:17.116 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:17.120+0000 7f39d4e00700 1 --2- 172.21.15.18:0/1664337131 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f39b406ec00 0x7f39b40710b0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f39c0005010 tx=0x7f39c0004d10 comp rx=0 tx=0).stop 2024-04-12T21:21:17.117 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:17.120+0000 7f39d4e00700 1 -- 172.21.15.18:0/1664337131 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f39d0105990 msgr2=0x7f39d01161f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:21:17.117 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:17.120+0000 7f39d4e00700 1 --2- 172.21.15.18:0/1664337131 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f39d0105990 0x7f39d01161f0 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f39b800c800 tx=0x7f39b8004470 comp rx=0 tx=0).stop 2024-04-12T21:21:17.117 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:17.121+0000 7f39d4e00700 1 -- 172.21.15.18:0/1664337131 shutdown_connections 2024-04-12T21:21:17.117 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:17.121+0000 7f39d4e00700 1 --2- 172.21.15.18:0/1664337131 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f39b406ec00 0x7f39b40710b0 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:21:17.117 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:17.121+0000 7f39d4e00700 1 --2- 172.21.15.18:0/1664337131 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f39d0104ff0 0x7f39d0115cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:21:17.117 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:17.121+0000 7f39d4e00700 1 --2- 172.21.15.18:0/1664337131 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f39d0105990 0x7f39d01161f0 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:21:17.118 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:17.121+0000 7f39d4e00700 1 -- 172.21.15.18:0/1664337131 >> 172.21.15.18:0/1664337131 conn(0x7f39d00faba0 msgr2=0x7f39d00fbb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:21:17.118 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:17.121+0000 7f39d4e00700 1 -- 172.21.15.18:0/1664337131 shutdown_connections 2024-04-12T21:21:17.118 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:17.121+0000 7f39d4e00700 1 -- 172.21.15.18:0/1664337131 wait complete. 2024-04-12T21:21:17.273 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:17.276+0000 7f5164a00700 1 -- 172.21.15.18:0/124347726 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f51600fde90 msgr2=0x7f51600fe260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:21:17.273 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:17.276+0000 7f5164a00700 1 --2- 172.21.15.18:0/124347726 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f51600fde90 0x7f51600fe260 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f5150004940 tx=0x7f5150009e30 comp rx=0 tx=0).stop 2024-04-12T21:21:17.273 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:17.276+0000 7f5164a00700 1 -- 172.21.15.18:0/124347726 shutdown_connections 2024-04-12T21:21:17.273 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:17.276+0000 7f5164a00700 1 --2- 172.21.15.18:0/124347726 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f51600fe830 0x7f5160106f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:21:17.273 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:17.276+0000 7f5164a00700 1 --2- 172.21.15.18:0/124347726 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f51600fde90 0x7f51600fe260 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:21:17.273 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:17.276+0000 7f5164a00700 1 -- 172.21.15.18:0/124347726 >> 172.21.15.18:0/124347726 conn(0x7f5160072bc0 msgr2=0x7f5160072fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:21:17.273 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:17.277+0000 7f5164a00700 1 -- 172.21.15.18:0/124347726 shutdown_connections 2024-04-12T21:21:17.273 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:17.277+0000 7f5164a00700 1 -- 172.21.15.18:0/124347726 wait complete. 2024-04-12T21:21:17.274 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:17.278+0000 7f5164a00700 1 Processor -- start 2024-04-12T21:21:17.274 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:17.278+0000 7f5164a00700 1 -- start start 2024-04-12T21:21:17.275 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:17.278+0000 7f5164a00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f51600fde90 0x7f5160115d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:21:17.275 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:17.278+0000 7f5164a00700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f51600fe830 0x7f5160116280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:21:17.275 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:17.279+0000 7f5164a00700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5160116960 con 0x7f51600fde90 2024-04-12T21:21:17.275 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:17.279+0000 7f5164a00700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f516010fce0 con 0x7f51600fe830 2024-04-12T21:21:17.275 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:17.279+0000 7f515de00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f51600fde90 0x7f5160115d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:21:17.275 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:17.279+0000 7f515de00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f51600fde90 0x7f5160115d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:38620/0 (socket says 172.21.15.18:38620) 2024-04-12T21:21:17.275 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:17.279+0000 7f515de00700 1 -- 172.21.15.18:0/122919963 learned_addr learned my addr 172.21.15.18:0/122919963 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:21:17.275 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:17.279+0000 7f515de00700 1 -- 172.21.15.18:0/122919963 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f51600fe830 msgr2=0x7f5160116280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:21:17.275 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:17.279+0000 7f515de00700 1 --2- 172.21.15.18:0/122919963 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f51600fe830 0x7f5160116280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:21:17.275 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:17.279+0000 7f515de00700 1 -- 172.21.15.18:0/122919963 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5150005040 con 0x7f51600fde90 2024-04-12T21:21:17.276 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:17.280+0000 7f515de00700 1 --2- 172.21.15.18:0/122919963 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f51600fde90 0x7f5160115d40 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f5150000c00 tx=0x7f5150009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:21:17.276 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:17.280+0000 7f5156000700 1 -- 172.21.15.18:0/122919963 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f515000e040 con 0x7f51600fde90 2024-04-12T21:21:17.276 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:17.280+0000 7f5156000700 1 -- 172.21.15.18:0/122919963 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f5150013420 con 0x7f51600fde90 2024-04-12T21:21:17.277 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:17.280+0000 7f5156000700 1 -- 172.21.15.18:0/122919963 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f515001c610 con 0x7f51600fde90 2024-04-12T21:21:17.277 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:17.280+0000 7f5164a00700 1 -- 172.21.15.18:0/122919963 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f516010ff60 con 0x7f51600fde90 2024-04-12T21:21:17.277 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:17.280+0000 7f5164a00700 1 -- 172.21.15.18:0/122919963 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f51601104a0 con 0x7f51600fde90 2024-04-12T21:21:17.277 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:17.281+0000 7f5164a00700 1 -- 172.21.15.18:0/122919963 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5160106a20 con 0x7f51600fde90 2024-04-12T21:21:17.278 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:17.282+0000 7f5156000700 1 -- 172.21.15.18:0/122919963 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7f5150013590 con 0x7f51600fde90 2024-04-12T21:21:17.279 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:17.283+0000 7f5156000700 1 --2- 172.21.15.18:0/122919963 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f514406ec10 0x7f51440710c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:21:17.279 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:17.283+0000 7f5156000700 1 -- 172.21.15.18:0/122919963 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 6757+0+0 (secure 0 0 0) 0x7f515008fb30 con 0x7f51600fde90 2024-04-12T21:21:17.279 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:17.284+0000 7f515d400700 1 --2- 172.21.15.18:0/122919963 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f514406ec10 0x7f51440710c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:21:17.280 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:17.284+0000 7f515d400700 1 --2- 172.21.15.18:0/122919963 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f514406ec10 0x7f51440710c0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f514c006fd0 tx=0x7f514c006db0 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:21:17.284 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:17.288+0000 7f5156000700 1 -- 172.21.15.18:0/122919963 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f5150058850 con 0x7f51600fde90 2024-04-12T21:21:17.599 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:21:17 smithi018 ceph-mon[112590]: pgmap v644: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:21:17.599 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:21:17 smithi018 ceph-mon[112590]: from='client.? 172.21.15.18:0/1278958250' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-12T21:21:17.607 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:17.611+0000 7f5164a00700 1 -- 172.21.15.18:0/122919963 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f5160065e30 con 0x7f51600fde90 2024-04-12T21:21:17.608 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:17.612+0000 7f5156000700 1 -- 172.21.15.18:0/122919963 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+233 (secure 0 0 0) 0x7f515005bf10 con 0x7f51600fde90 2024-04-12T21:21:17.608 INFO:teuthology.orchestra.run.smithi018.stdout:HEALTH_ERR 1 filesystem is degraded; 1 filesystem is offline 2024-04-12T21:21:17.608 INFO:teuthology.orchestra.run.smithi018.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-04-12T21:21:17.608 INFO:teuthology.orchestra.run.smithi018.stdout: fs cephfs is degraded 2024-04-12T21:21:17.608 INFO:teuthology.orchestra.run.smithi018.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-04-12T21:21:17.608 INFO:teuthology.orchestra.run.smithi018.stdout: fs cephfs is offline because no MDS is active for it. 2024-04-12T21:21:17.611 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:17.615+0000 7f5164a00700 1 -- 172.21.15.18:0/122919963 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f514406ec10 msgr2=0x7f51440710c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:21:17.612 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:17.615+0000 7f5164a00700 1 --2- 172.21.15.18:0/122919963 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f514406ec10 0x7f51440710c0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f514c006fd0 tx=0x7f514c006db0 comp rx=0 tx=0).stop 2024-04-12T21:21:17.612 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:17.615+0000 7f5164a00700 1 -- 172.21.15.18:0/122919963 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f51600fde90 msgr2=0x7f5160115d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:21:17.612 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:17.615+0000 7f5164a00700 1 --2- 172.21.15.18:0/122919963 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f51600fde90 0x7f5160115d40 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f5150000c00 tx=0x7f5150009e30 comp rx=0 tx=0).stop 2024-04-12T21:21:17.612 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:17.615+0000 7f5164a00700 1 -- 172.21.15.18:0/122919963 shutdown_connections 2024-04-12T21:21:17.612 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:17.615+0000 7f5164a00700 1 --2- 172.21.15.18:0/122919963 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f514406ec10 0x7f51440710c0 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:21:17.612 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:17.615+0000 7f5164a00700 1 --2- 172.21.15.18:0/122919963 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f51600fde90 0x7f5160115d40 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:21:17.612 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:17.615+0000 7f5164a00700 1 --2- 172.21.15.18:0/122919963 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f51600fe830 0x7f5160116280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:21:17.612 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:17.615+0000 7f5164a00700 1 -- 172.21.15.18:0/122919963 >> 172.21.15.18:0/122919963 conn(0x7f5160072bc0 msgr2=0x7f51600fb700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:21:17.612 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:17.616+0000 7f5164a00700 1 -- 172.21.15.18:0/122919963 shutdown_connections 2024-04-12T21:21:17.612 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:17.616+0000 7f5164a00700 1 -- 172.21.15.18:0/122919963 wait complete. 2024-04-12T21:21:17.778 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:21:17 smithi186 ceph-mon[86379]: pgmap v644: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:21:17.778 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:21:17 smithi186 ceph-mon[86379]: from='client.? 172.21.15.18:0/1278958250' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-12T21:21:18.599 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:21:18 smithi018 ceph-mon[112590]: from='client.35164 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:21:18.599 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:21:18 smithi018 ceph-mon[112590]: from='client.? 172.21.15.18:0/122919963' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-12T21:21:18.778 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:21:18 smithi186 ceph-mon[86379]: from='client.35164 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:21:18.778 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:21:18 smithi186 ceph-mon[86379]: from='client.? 172.21.15.18:0/122919963' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-12T21:21:19.599 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:21:19 smithi018 ceph-mon[112590]: pgmap v645: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:21:19.778 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:21:19 smithi186 ceph-mon[86379]: pgmap v645: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:21:21.599 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:21:21 smithi018 ceph-mon[112590]: pgmap v646: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:21:21.778 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:21:21 smithi186 ceph-mon[86379]: pgmap v646: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:21:23.600 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:21:23 smithi018 ceph-mon[112590]: pgmap v647: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:21:23.779 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:21:23 smithi186 ceph-mon[86379]: pgmap v647: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:21:25.599 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:21:25 smithi018 ceph-mon[112590]: pgmap v648: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:21:25.599 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:21:25 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T21:21:25.778 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:21:25 smithi186 ceph-mon[86379]: pgmap v648: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:21:25.778 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:21:25 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T21:21:27.599 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:21:27 smithi018 ceph-mon[112590]: pgmap v649: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:21:27.778 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:21:27 smithi186 ceph-mon[86379]: pgmap v649: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:21:29.778 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:21:29 smithi186 ceph-mon[86379]: pgmap v650: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:21:29.849 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:21:29 smithi018 ceph-mon[112590]: pgmap v650: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:21:31.778 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:21:31 smithi186 ceph-mon[86379]: pgmap v651: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:21:31.849 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:21:31 smithi018 ceph-mon[112590]: pgmap v651: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:21:33.778 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:21:33 smithi186 ceph-mon[86379]: pgmap v652: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:21:33.849 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:21:33 smithi018 ceph-mon[112590]: pgmap v652: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:21:35.778 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:21:35 smithi186 ceph-mon[86379]: pgmap v653: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:21:35.849 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:21:35 smithi018 ceph-mon[112590]: pgmap v653: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:21:37.778 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:21:37 smithi186 ceph-mon[86379]: pgmap v654: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:21:37.849 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:21:37 smithi018 ceph-mon[112590]: pgmap v654: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:21:39.778 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:21:39 smithi186 ceph-mon[86379]: pgmap v655: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:21:39.849 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:21:39 smithi018 ceph-mon[112590]: pgmap v655: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:21:40.680 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:21:40 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T21:21:40.849 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:21:40 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T21:21:41.751 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:21:41 smithi018 ceph-mon[112590]: pgmap v656: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:21:41.778 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:21:41 smithi186 ceph-mon[86379]: pgmap v656: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:21:43.778 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:21:43 smithi186 ceph-mon[86379]: pgmap v657: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:21:43.849 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:21:43 smithi018 ceph-mon[112590]: pgmap v657: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:21:45.778 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:21:45 smithi186 ceph-mon[86379]: pgmap v658: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:21:45.849 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:21:45 smithi018 ceph-mon[112590]: pgmap v658: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:21:47.778 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:21:47 smithi186 ceph-mon[86379]: pgmap v659: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:21:47.786 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:47.788+0000 7f6285e00700 1 -- 172.21.15.18:0/1797283616 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6280109dd0 msgr2=0x7f6280103440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:21:47.786 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:47.788+0000 7f6285e00700 1 --2- 172.21.15.18:0/1797283616 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6280109dd0 0x7f6280103440 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f626c004940 tx=0x7f626c009e30 comp rx=0 tx=0).stop 2024-04-12T21:21:47.786 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:21:47 smithi018 ceph-mon[112590]: pgmap v659: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:21:47.788 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:47.788+0000 7f6285e00700 1 -- 172.21.15.18:0/1797283616 shutdown_connections 2024-04-12T21:21:47.788 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:47.788+0000 7f6285e00700 1 --2- 172.21.15.18:0/1797283616 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6280109dd0 0x7f6280103440 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:21:47.788 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:47.788+0000 7f6285e00700 1 --2- 172.21.15.18:0/1797283616 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6280109430 0x7f6280109800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:21:47.788 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:47.788+0000 7f6285e00700 1 -- 172.21.15.18:0/1797283616 >> 172.21.15.18:0/1797283616 conn(0x7f6280072c00 msgr2=0x7f6280075010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:21:47.788 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:47.789+0000 7f6285e00700 1 -- 172.21.15.18:0/1797283616 shutdown_connections 2024-04-12T21:21:47.788 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:47.789+0000 7f6285e00700 1 -- 172.21.15.18:0/1797283616 wait complete. 2024-04-12T21:21:47.788 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:47.789+0000 7f6285e00700 1 Processor -- start 2024-04-12T21:21:47.788 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:47.790+0000 7f6285e00700 1 -- start start 2024-04-12T21:21:47.789 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:47.790+0000 7f6285e00700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6280109430 0x7f6280119f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:21:47.789 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:47.790+0000 7f6285e00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6280109dd0 0x7f628011a4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:21:47.789 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:47.790+0000 7f6285e00700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f628011ab10 con 0x7f6280109dd0 2024-04-12T21:21:47.789 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:47.790+0000 7f6285e00700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62801140f0 con 0x7f6280109430 2024-04-12T21:21:47.789 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:47.791+0000 7f627e000700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6280109dd0 0x7f628011a4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:21:47.789 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:47.791+0000 7f627e000700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6280109dd0 0x7f628011a4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:45532/0 (socket says 172.21.15.18:45532) 2024-04-12T21:21:47.789 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:47.791+0000 7f627e000700 1 -- 172.21.15.18:0/354463367 learned_addr learned my addr 172.21.15.18:0/354463367 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:21:47.789 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:47.791+0000 7f627ea00700 1 --2- 172.21.15.18:0/354463367 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6280109430 0x7f6280119f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:21:47.789 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:47.791+0000 7f627e000700 1 -- 172.21.15.18:0/354463367 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6280109430 msgr2=0x7f6280119f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:21:47.789 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:47.791+0000 7f627e000700 1 --2- 172.21.15.18:0/354463367 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6280109430 0x7f6280119f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:21:47.789 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:47.791+0000 7f627e000700 1 -- 172.21.15.18:0/354463367 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f626c005040 con 0x7f6280109dd0 2024-04-12T21:21:47.789 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:47.792+0000 7f627e000700 1 --2- 172.21.15.18:0/354463367 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6280109dd0 0x7f628011a4c0 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f626c009100 tx=0x7f626c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:21:47.789 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:47.792+0000 7f6276a00700 1 -- 172.21.15.18:0/354463367 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f626c00e040 con 0x7f6280109dd0 2024-04-12T21:21:47.789 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:47.792+0000 7f6276a00700 1 -- 172.21.15.18:0/354463367 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f626c013420 con 0x7f6280109dd0 2024-04-12T21:21:47.789 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:47.792+0000 7f6276a00700 1 -- 172.21.15.18:0/354463367 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f626c01c630 con 0x7f6280109dd0 2024-04-12T21:21:47.789 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:47.792+0000 7f6285e00700 1 -- 172.21.15.18:0/354463367 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6280114370 con 0x7f6280109dd0 2024-04-12T21:21:47.790 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:47.792+0000 7f6285e00700 1 -- 172.21.15.18:0/354463367 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6280114930 con 0x7f6280109dd0 2024-04-12T21:21:47.790 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:47.793+0000 7f6285e00700 1 -- 172.21.15.18:0/354463367 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6280104b80 con 0x7f6280109dd0 2024-04-12T21:21:47.790 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:47.794+0000 7f6276a00700 1 -- 172.21.15.18:0/354463367 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7f626c013590 con 0x7f6280109dd0 2024-04-12T21:21:47.790 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:47.795+0000 7f6276a00700 1 --2- 172.21.15.18:0/354463367 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f626406ec00 0x7f62640710b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:21:47.791 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:47.796+0000 7f6276a00700 1 -- 172.21.15.18:0/354463367 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 6757+0+0 (secure 0 0 0) 0x7f626c08fae0 con 0x7f6280109dd0 2024-04-12T21:21:47.791 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:47.796+0000 7f627ea00700 1 --2- 172.21.15.18:0/354463367 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f626406ec00 0x7f62640710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:21:47.791 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:47.797+0000 7f627ea00700 1 --2- 172.21.15.18:0/354463367 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f626406ec00 0x7f62640710b0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f6270007fd0 tx=0x7f6270009040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:21:47.798 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:47.803+0000 7f6276a00700 1 -- 172.21.15.18:0/354463367 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f626c05c3d0 con 0x7f6280109dd0 2024-04-12T21:21:48.051 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:48.056+0000 7f6285e00700 1 -- 172.21.15.18:0/354463367 --> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f6280110500 con 0x7f626406ec00 2024-04-12T21:21:48.054 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:48.060+0000 7f6276a00700 1 -- 172.21.15.18:0/354463367 <== mgr.34102 v2:172.21.15.18:6800/3812891919 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f6280110500 con 0x7f626406ec00 2024-04-12T21:21:48.057 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:48.062+0000 7f6285e00700 1 -- 172.21.15.18:0/354463367 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f626406ec00 msgr2=0x7f62640710b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:21:48.057 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:48.062+0000 7f6285e00700 1 --2- 172.21.15.18:0/354463367 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f626406ec00 0x7f62640710b0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f6270007fd0 tx=0x7f6270009040 comp rx=0 tx=0).stop 2024-04-12T21:21:48.057 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:48.062+0000 7f6285e00700 1 -- 172.21.15.18:0/354463367 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6280109dd0 msgr2=0x7f628011a4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:21:48.057 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:48.062+0000 7f6285e00700 1 --2- 172.21.15.18:0/354463367 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6280109dd0 0x7f628011a4c0 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f626c009100 tx=0x7f626c009e30 comp rx=0 tx=0).stop 2024-04-12T21:21:48.058 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:48.063+0000 7f6285e00700 1 -- 172.21.15.18:0/354463367 shutdown_connections 2024-04-12T21:21:48.058 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:48.063+0000 7f6285e00700 1 --2- 172.21.15.18:0/354463367 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f626406ec00 0x7f62640710b0 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:21:48.058 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:48.063+0000 7f6285e00700 1 --2- 172.21.15.18:0/354463367 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6280109430 0x7f6280119f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:21:48.058 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:48.063+0000 7f6285e00700 1 --2- 172.21.15.18:0/354463367 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6280109dd0 0x7f628011a4c0 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:21:48.058 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:48.063+0000 7f6285e00700 1 -- 172.21.15.18:0/354463367 >> 172.21.15.18:0/354463367 conn(0x7f6280072c00 msgr2=0x7f6280074c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:21:48.058 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:48.063+0000 7f6285e00700 1 -- 172.21.15.18:0/354463367 shutdown_connections 2024-04-12T21:21:48.059 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:48.063+0000 7f6285e00700 1 -- 172.21.15.18:0/354463367 wait complete. 2024-04-12T21:21:48.074 INFO:teuthology.orchestra.run.smithi018.stdout:true 2024-04-12T21:21:48.223 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:48.227+0000 7f75d2600700 1 -- 172.21.15.18:0/2845827913 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f75cc109430 msgr2=0x7f75cc109800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:21:48.223 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:48.227+0000 7f75d2600700 1 --2- 172.21.15.18:0/2845827913 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f75cc109430 0x7f75cc109800 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f75bc004940 tx=0x7f75bc009e30 comp rx=0 tx=0).stop 2024-04-12T21:21:48.223 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:48.228+0000 7f75d2600700 1 -- 172.21.15.18:0/2845827913 shutdown_connections 2024-04-12T21:21:48.223 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:48.228+0000 7f75d2600700 1 --2- 172.21.15.18:0/2845827913 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f75cc109dd0 0x7f75cc103440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:21:48.223 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:48.228+0000 7f75d2600700 1 --2- 172.21.15.18:0/2845827913 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f75cc109430 0x7f75cc109800 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:21:48.223 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:48.228+0000 7f75d2600700 1 -- 172.21.15.18:0/2845827913 >> 172.21.15.18:0/2845827913 conn(0x7f75cc072c00 msgr2=0x7f75cc075010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:21:48.224 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:48.228+0000 7f75d2600700 1 -- 172.21.15.18:0/2845827913 shutdown_connections 2024-04-12T21:21:48.224 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:48.228+0000 7f75d2600700 1 -- 172.21.15.18:0/2845827913 wait complete. 2024-04-12T21:21:48.225 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:48.229+0000 7f75d2600700 1 Processor -- start 2024-04-12T21:21:48.225 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:48.229+0000 7f75d2600700 1 -- start start 2024-04-12T21:21:48.225 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:48.229+0000 7f75d2600700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f75cc109430 0x7f75cc119f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:21:48.225 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:48.229+0000 7f75d2600700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f75cc109dd0 0x7f75cc11a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:21:48.226 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:48.229+0000 7f75d2600700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75cc11ab60 con 0x7f75cc109430 2024-04-12T21:21:48.226 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:48.229+0000 7f75d2600700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75cc1140b0 con 0x7f75cc109dd0 2024-04-12T21:21:48.226 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:48.230+0000 7f75cb400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f75cc109430 0x7f75cc119f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:21:48.226 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:48.230+0000 7f75cb400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f75cc109430 0x7f75cc119f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:45548/0 (socket says 172.21.15.18:45548) 2024-04-12T21:21:48.226 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:48.230+0000 7f75cb400700 1 -- 172.21.15.18:0/3870464429 learned_addr learned my addr 172.21.15.18:0/3870464429 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:21:48.226 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:48.230+0000 7f75caa00700 1 --2- 172.21.15.18:0/3870464429 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f75cc109dd0 0x7f75cc11a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:21:48.226 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:48.230+0000 7f75cb400700 1 -- 172.21.15.18:0/3870464429 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f75cc109dd0 msgr2=0x7f75cc11a480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:21:48.226 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:48.230+0000 7f75cb400700 1 --2- 172.21.15.18:0/3870464429 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f75cc109dd0 0x7f75cc11a480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:21:48.226 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:48.230+0000 7f75cb400700 1 -- 172.21.15.18:0/3870464429 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75bc005040 con 0x7f75cc109430 2024-04-12T21:21:48.226 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:48.230+0000 7f75cb400700 1 --2- 172.21.15.18:0/3870464429 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f75cc109430 0x7f75cc119f40 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f75bc009150 tx=0x7f75bc009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:21:48.227 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:48.231+0000 7f75c3400700 1 -- 172.21.15.18:0/3870464429 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f75bc00e040 con 0x7f75cc109430 2024-04-12T21:21:48.227 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:48.231+0000 7f75c3400700 1 -- 172.21.15.18:0/3870464429 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f75bc013420 con 0x7f75cc109430 2024-04-12T21:21:48.227 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:48.231+0000 7f75c3400700 1 -- 172.21.15.18:0/3870464429 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f75bc01c630 con 0x7f75cc109430 2024-04-12T21:21:48.228 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:48.231+0000 7f75d2600700 1 -- 172.21.15.18:0/3870464429 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f75cc1142d0 con 0x7f75cc109430 2024-04-12T21:21:48.228 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:48.231+0000 7f75d2600700 1 -- 172.21.15.18:0/3870464429 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f75cc114890 con 0x7f75cc109430 2024-04-12T21:21:48.228 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:48.232+0000 7f75d2600700 1 -- 172.21.15.18:0/3870464429 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f75cc104b80 con 0x7f75cc109430 2024-04-12T21:21:48.228 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:48.234+0000 7f75c3400700 1 -- 172.21.15.18:0/3870464429 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7f75bc013590 con 0x7f75cc109430 2024-04-12T21:21:48.229 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:48.234+0000 7f75c3400700 1 --2- 172.21.15.18:0/3870464429 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f75b006ec00 0x7f75b00710b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:21:48.229 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:48.235+0000 7f75c3400700 1 -- 172.21.15.18:0/3870464429 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 6757+0+0 (secure 0 0 0) 0x7f75bc08fb40 con 0x7f75cc109430 2024-04-12T21:21:48.230 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:48.235+0000 7f75caa00700 1 --2- 172.21.15.18:0/3870464429 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f75b006ec00 0x7f75b00710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:21:48.231 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:48.236+0000 7f75caa00700 1 --2- 172.21.15.18:0/3870464429 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f75b006ec00 0x7f75b00710b0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f75b8006fd0 tx=0x7f75b8008040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:21:48.234 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:48.240+0000 7f75c3400700 1 -- 172.21.15.18:0/3870464429 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f75bc05c430 con 0x7f75cc109430 2024-04-12T21:21:48.484 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:48.489+0000 7f75d2600700 1 -- 172.21.15.18:0/3870464429 --> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f75cc114bc0 con 0x7f75b006ec00 2024-04-12T21:21:48.486 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:48.492+0000 7f75c3400700 1 -- 172.21.15.18:0/3870464429 <== mgr.34102 v2:172.21.15.18:6800/3812891919 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f75cc114bc0 con 0x7f75b006ec00 2024-04-12T21:21:48.489 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:48.494+0000 7f75d2600700 1 -- 172.21.15.18:0/3870464429 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f75b006ec00 msgr2=0x7f75b00710b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:21:48.489 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:48.494+0000 7f75d2600700 1 --2- 172.21.15.18:0/3870464429 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f75b006ec00 0x7f75b00710b0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f75b8006fd0 tx=0x7f75b8008040 comp rx=0 tx=0).stop 2024-04-12T21:21:48.489 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:48.494+0000 7f75d2600700 1 -- 172.21.15.18:0/3870464429 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f75cc109430 msgr2=0x7f75cc119f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:21:48.490 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:48.494+0000 7f75d2600700 1 --2- 172.21.15.18:0/3870464429 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f75cc109430 0x7f75cc119f40 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f75bc009150 tx=0x7f75bc009e30 comp rx=0 tx=0).stop 2024-04-12T21:21:48.490 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:48.495+0000 7f75d2600700 1 -- 172.21.15.18:0/3870464429 shutdown_connections 2024-04-12T21:21:48.490 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:48.495+0000 7f75d2600700 1 --2- 172.21.15.18:0/3870464429 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f75b006ec00 0x7f75b00710b0 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:21:48.490 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:48.495+0000 7f75d2600700 1 --2- 172.21.15.18:0/3870464429 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f75cc109430 0x7f75cc119f40 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:21:48.490 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:48.495+0000 7f75d2600700 1 --2- 172.21.15.18:0/3870464429 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f75cc109dd0 0x7f75cc11a480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:21:48.490 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:48.495+0000 7f75d2600700 1 -- 172.21.15.18:0/3870464429 >> 172.21.15.18:0/3870464429 conn(0x7f75cc072c00 msgr2=0x7f75cc074c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:21:48.490 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:48.495+0000 7f75d2600700 1 -- 172.21.15.18:0/3870464429 shutdown_connections 2024-04-12T21:21:48.490 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:48.495+0000 7f75d2600700 1 -- 172.21.15.18:0/3870464429 wait complete. 2024-04-12T21:21:48.648 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:48.653+0000 7f18d9600700 1 -- 172.21.15.18:0/744859921 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f18d40fc620 msgr2=0x7f18d4100cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:21:48.648 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:48.653+0000 7f18d9600700 1 --2- 172.21.15.18:0/744859921 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f18d40fc620 0x7f18d4100cd0 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f18c0004940 tx=0x7f18c0009e30 comp rx=0 tx=0).stop 2024-04-12T21:21:48.648 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:48.653+0000 7f18d9600700 1 -- 172.21.15.18:0/744859921 shutdown_connections 2024-04-12T21:21:48.648 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:48.653+0000 7f18d9600700 1 --2- 172.21.15.18:0/744859921 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f18d40fc620 0x7f18d4100cd0 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:21:48.648 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:48.653+0000 7f18d9600700 1 --2- 172.21.15.18:0/744859921 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f18d40fbc80 0x7f18d40fc050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:21:48.648 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:48.653+0000 7f18d9600700 1 -- 172.21.15.18:0/744859921 >> 172.21.15.18:0/744859921 conn(0x7f18d4072bc0 msgr2=0x7f18d4072fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:21:48.648 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:48.653+0000 7f18d9600700 1 -- 172.21.15.18:0/744859921 shutdown_connections 2024-04-12T21:21:48.648 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:48.653+0000 7f18d9600700 1 -- 172.21.15.18:0/744859921 wait complete. 2024-04-12T21:21:48.649 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:48.654+0000 7f18d9600700 1 Processor -- start 2024-04-12T21:21:48.649 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:48.654+0000 7f18d9600700 1 -- start start 2024-04-12T21:21:48.649 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:48.654+0000 7f18d9600700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f18d40fbc80 0x7f18d411a1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:21:48.650 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:48.654+0000 7f18d9600700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f18d40fc620 0x7f18d411a710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:21:48.650 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:48.654+0000 7f18d9600700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18d411ad60 con 0x7f18d40fbc80 2024-04-12T21:21:48.650 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:48.654+0000 7f18d9600700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18d4114130 con 0x7f18d40fc620 2024-04-12T21:21:48.650 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:48.654+0000 7f18d2a00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f18d40fbc80 0x7f18d411a1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:21:48.650 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:48.654+0000 7f18d2a00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f18d40fbc80 0x7f18d411a1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:45570/0 (socket says 172.21.15.18:45570) 2024-04-12T21:21:48.650 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:48.654+0000 7f18d2a00700 1 -- 172.21.15.18:0/1691540961 learned_addr learned my addr 172.21.15.18:0/1691540961 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:21:48.651 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:48.655+0000 7f18d2000700 1 --2- 172.21.15.18:0/1691540961 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f18d40fc620 0x7f18d411a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:21:48.651 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:48.655+0000 7f18d2a00700 1 -- 172.21.15.18:0/1691540961 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f18d40fc620 msgr2=0x7f18d411a710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:21:48.651 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:48.655+0000 7f18d2a00700 1 --2- 172.21.15.18:0/1691540961 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f18d40fc620 0x7f18d411a710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:21:48.651 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:48.655+0000 7f18d2a00700 1 -- 172.21.15.18:0/1691540961 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18c0005040 con 0x7f18d40fbc80 2024-04-12T21:21:48.651 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:48.655+0000 7f18d2a00700 1 --2- 172.21.15.18:0/1691540961 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f18d40fbc80 0x7f18d411a1d0 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f18c4002a50 tx=0x7f18c40046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:21:48.651 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:48.656+0000 7f18caa00700 1 -- 172.21.15.18:0/1691540961 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f18c4007cc0 con 0x7f18d40fbc80 2024-04-12T21:21:48.651 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:48.656+0000 7f18caa00700 1 -- 172.21.15.18:0/1691540961 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f18c400f040 con 0x7f18d40fbc80 2024-04-12T21:21:48.651 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:48.656+0000 7f18caa00700 1 -- 172.21.15.18:0/1691540961 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f18c4009640 con 0x7f18d40fbc80 2024-04-12T21:21:48.652 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:48.656+0000 7f18d9600700 1 -- 172.21.15.18:0/1691540961 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f18d4114410 con 0x7f18d40fbc80 2024-04-12T21:21:48.652 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:48.656+0000 7f18d9600700 1 -- 172.21.15.18:0/1691540961 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f18d4114950 con 0x7f18d40fbc80 2024-04-12T21:21:48.652 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:48.657+0000 7f18d9600700 1 -- 172.21.15.18:0/1691540961 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f18d4065e30 con 0x7f18d40fbc80 2024-04-12T21:21:48.653 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:48.658+0000 7f18caa00700 1 -- 172.21.15.18:0/1691540961 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7f18c40097a0 con 0x7f18d40fbc80 2024-04-12T21:21:48.654 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:48.659+0000 7f18caa00700 1 --2- 172.21.15.18:0/1691540961 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f18b806ec10 0x7f18b80710c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:21:48.654 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:48.659+0000 7f18caa00700 1 -- 172.21.15.18:0/1691540961 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 6757+0+0 (secure 0 0 0) 0x7f18c408e830 con 0x7f18d40fbc80 2024-04-12T21:21:48.654 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:48.659+0000 7f18d2000700 1 --2- 172.21.15.18:0/1691540961 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f18b806ec10 0x7f18b80710c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:21:48.655 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:48.660+0000 7f18d2000700 1 --2- 172.21.15.18:0/1691540961 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f18b806ec10 0x7f18b80710c0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f18c000b3b0 tx=0x7f18c0009e30 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:21:48.659 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:48.664+0000 7f18caa00700 1 -- 172.21.15.18:0/1691540961 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f18c405af60 con 0x7f18d40fbc80 2024-04-12T21:21:48.909 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:48.914+0000 7f18d9600700 1 -- 172.21.15.18:0/1691540961 --> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f18d404bf70 con 0x7f18b806ec10 2024-04-12T21:21:48.939 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:48.944+0000 7f18caa00700 1 -- 172.21.15.18:0/1691540961 <== mgr.34102 v2:172.21.15.18:6800/3812891919 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7f18c0009710 con 0x7f18b806ec10 2024-04-12T21:21:48.940 INFO:teuthology.orchestra.run.smithi018.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-12T21:21:48.940 INFO:teuthology.orchestra.run.smithi018.stdout:alertmanager.smithi018 smithi018 *:9093,9094 running (22m) 14m ago 36m 14.7M - 0.25.0 c8568f914cd2 c332090068e8 2024-04-12T21:21:48.940 INFO:teuthology.orchestra.run.smithi018.stdout:crash.smithi018 smithi018 running (18m) 14m ago 37m 6665k - 19.0.0-2879-g324baf8d 8717720dc5e5 b7f80d1f7982 2024-04-12T21:21:48.940 INFO:teuthology.orchestra.run.smithi018.stdout:crash.smithi186 smithi186 running (18m) 14m ago 35m 6660k - 19.0.0-2879-g324baf8d 8717720dc5e5 a4f55c81b623 2024-04-12T21:21:48.940 INFO:teuthology.orchestra.run.smithi018.stdout:grafana.smithi018 smithi018 *:3000 running (21m) 14m ago 36m 74.8M - 9.4.12 2bacad6d85d8 b08e3bf176dd 2024-04-12T21:21:48.940 INFO:teuthology.orchestra.run.smithi018.stdout:mds.cephfs.smithi018.acqbsc smithi018 running (14m) 14m ago 29m 11.1M - 19.0.0-2879-g324baf8d 8717720dc5e5 021f99ab24ae 2024-04-12T21:21:48.940 INFO:teuthology.orchestra.run.smithi018.stdout:mds.cephfs.smithi018.szjluo smithi018 running (14m) 14m ago 29m 10.9M - 19.0.0-2879-g324baf8d 8717720dc5e5 0003117479d4 2024-04-12T21:21:48.940 INFO:teuthology.orchestra.run.smithi018.stdout:mds.cephfs.smithi186.jdllyn smithi186 running (29m) 14m ago 29m 15.6M - 17.2.7-827-gd3b5060a 21de9b7369ad 2b2bc46bd29a 2024-04-12T21:21:48.940 INFO:teuthology.orchestra.run.smithi018.stdout:mds.cephfs.smithi186.wjnafs smithi186 running (29m) 14m ago 29m 15.4M - 17.2.7-827-gd3b5060a 21de9b7369ad 74b115d6d6fd 2024-04-12T21:21:48.940 INFO:teuthology.orchestra.run.smithi018.stdout:mgr.smithi018.xjzwga smithi018 *:8443,9283,8765 running (25m) 14m ago 38m 509M - 19.0.0-2879-g324baf8d 8717720dc5e5 56088305eab7 2024-04-12T21:21:48.940 INFO:teuthology.orchestra.run.smithi018.stdout:mgr.smithi186.kqcfxk smithi186 *:8443,9283,8765 running (23m) 14m ago 35m 452M - 19.0.0-2879-g324baf8d 8717720dc5e5 09cd766e5e0d 2024-04-12T21:21:48.941 INFO:teuthology.orchestra.run.smithi018.stdout:mon.smithi018 smithi018 running (20m) 14m ago 38m 57.0M 2048M 19.0.0-2879-g324baf8d 8717720dc5e5 df3e201dfed6 2024-04-12T21:21:48.941 INFO:teuthology.orchestra.run.smithi018.stdout:mon.smithi186 smithi186 running (19m) 14m ago 35m 37.7M 2048M 19.0.0-2879-g324baf8d 8717720dc5e5 c77498da9b2b 2024-04-12T21:21:48.941 INFO:teuthology.orchestra.run.smithi018.stdout:node-exporter.smithi018 smithi018 *:9100 running (23m) 14m ago 37m 7780k - 1.5.0 0da6a335fe13 326c3d9c6847 2024-04-12T21:21:48.941 INFO:teuthology.orchestra.run.smithi018.stdout:node-exporter.smithi186 smithi186 *:9100 running (23m) 14m ago 35m 7826k - 1.5.0 0da6a335fe13 ec2112baeecf 2024-04-12T21:21:48.941 INFO:teuthology.orchestra.run.smithi018.stdout:osd.0 smithi018 running (18m) 14m ago 34m 121M 1559M 19.0.0-2879-g324baf8d 8717720dc5e5 60b1273ca047 2024-04-12T21:21:48.941 INFO:teuthology.orchestra.run.smithi018.stdout:osd.1 smithi018 running (17m) 14m ago 33m 91.3M 1559M 19.0.0-2879-g324baf8d 8717720dc5e5 170ce1e5fb8d 2024-04-12T21:21:48.941 INFO:teuthology.orchestra.run.smithi018.stdout:osd.2 smithi018 running (17m) 14m ago 33m 90.3M 1559M 19.0.0-2879-g324baf8d 8717720dc5e5 c2cbc6aa2839 2024-04-12T21:21:48.941 INFO:teuthology.orchestra.run.smithi018.stdout:osd.3 smithi186 running (16m) 14m ago 32m 96.6M 2581M 19.0.0-2879-g324baf8d 8717720dc5e5 0138e8262e4b 2024-04-12T21:21:48.941 INFO:teuthology.orchestra.run.smithi018.stdout:osd.4 smithi186 running (15m) 14m ago 32m 87.2M 2581M 19.0.0-2879-g324baf8d 8717720dc5e5 b20cb63b25c9 2024-04-12T21:21:48.941 INFO:teuthology.orchestra.run.smithi018.stdout:osd.5 smithi186 running (15m) 14m ago 31m 11.1M 2581M 19.0.0-2879-g324baf8d 8717720dc5e5 2513bea2610e 2024-04-12T21:21:48.941 INFO:teuthology.orchestra.run.smithi018.stdout:prometheus.smithi018 smithi018 *:9095 running (22m) 14m ago 36m 60.6M - 2.43.0 a07b618ecd1d f58841127e21 2024-04-12T21:21:48.942 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:48.947+0000 7f18d9600700 1 -- 172.21.15.18:0/1691540961 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f18b806ec10 msgr2=0x7f18b80710c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:21:48.943 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:48.947+0000 7f18d9600700 1 --2- 172.21.15.18:0/1691540961 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f18b806ec10 0x7f18b80710c0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f18c000b3b0 tx=0x7f18c0009e30 comp rx=0 tx=0).stop 2024-04-12T21:21:48.943 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:48.947+0000 7f18d9600700 1 -- 172.21.15.18:0/1691540961 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f18d40fbc80 msgr2=0x7f18d411a1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:21:48.943 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:48.947+0000 7f18d9600700 1 --2- 172.21.15.18:0/1691540961 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f18d40fbc80 0x7f18d411a1d0 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f18c4002a50 tx=0x7f18c40046b0 comp rx=0 tx=0).stop 2024-04-12T21:21:48.943 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:48.947+0000 7f18d9600700 1 -- 172.21.15.18:0/1691540961 shutdown_connections 2024-04-12T21:21:48.943 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:48.947+0000 7f18d9600700 1 --2- 172.21.15.18:0/1691540961 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f18b806ec10 0x7f18b80710c0 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:21:48.943 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:48.947+0000 7f18d9600700 1 --2- 172.21.15.18:0/1691540961 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f18d40fbc80 0x7f18d411a1d0 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:21:48.943 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:48.947+0000 7f18d9600700 1 --2- 172.21.15.18:0/1691540961 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f18d40fc620 0x7f18d411a710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:21:48.943 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:48.947+0000 7f18d9600700 1 -- 172.21.15.18:0/1691540961 >> 172.21.15.18:0/1691540961 conn(0x7f18d4072bc0 msgr2=0x7f18d410aab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:21:48.944 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:48.948+0000 7f18d9600700 1 -- 172.21.15.18:0/1691540961 shutdown_connections 2024-04-12T21:21:48.944 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:48.948+0000 7f18d9600700 1 -- 172.21.15.18:0/1691540961 wait complete. 2024-04-12T21:21:49.104 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:49.108+0000 7f6cca600700 1 -- 172.21.15.18:0/2621676729 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6cc4109430 msgr2=0x7f6cc4109800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:21:49.104 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:49.108+0000 7f6cca600700 1 --2- 172.21.15.18:0/2621676729 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6cc4109430 0x7f6cc4109800 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f6cb4004940 tx=0x7f6cb4009e30 comp rx=0 tx=0).stop 2024-04-12T21:21:49.104 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:49.108+0000 7f6cca600700 1 -- 172.21.15.18:0/2621676729 shutdown_connections 2024-04-12T21:21:49.104 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:49.108+0000 7f6cca600700 1 --2- 172.21.15.18:0/2621676729 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6cc4109dd0 0x7f6cc4103440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:21:49.104 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:49.108+0000 7f6cca600700 1 --2- 172.21.15.18:0/2621676729 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6cc4109430 0x7f6cc4109800 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:21:49.104 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:49.108+0000 7f6cca600700 1 -- 172.21.15.18:0/2621676729 >> 172.21.15.18:0/2621676729 conn(0x7f6cc4072c00 msgr2=0x7f6cc4075010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:21:49.104 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:49.108+0000 7f6cca600700 1 -- 172.21.15.18:0/2621676729 shutdown_connections 2024-04-12T21:21:49.105 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:49.108+0000 7f6cca600700 1 -- 172.21.15.18:0/2621676729 wait complete. 2024-04-12T21:21:49.106 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:49.109+0000 7f6cca600700 1 Processor -- start 2024-04-12T21:21:49.106 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:49.110+0000 7f6cca600700 1 -- start start 2024-04-12T21:21:49.106 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:49.110+0000 7f6cca600700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6cc4109430 0x7f6cc4119f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:21:49.106 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:49.110+0000 7f6cca600700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6cc4109dd0 0x7f6cc411a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:21:49.107 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:49.110+0000 7f6cca600700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6cc411ab60 con 0x7f6cc4109dd0 2024-04-12T21:21:49.107 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:49.110+0000 7f6cca600700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6cc41140b0 con 0x7f6cc4109430 2024-04-12T21:21:49.107 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:49.111+0000 7f6cc2a00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6cc4109dd0 0x7f6cc411a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:21:49.107 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:49.111+0000 7f6cc2a00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6cc4109dd0 0x7f6cc411a480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:45586/0 (socket says 172.21.15.18:45586) 2024-04-12T21:21:49.107 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:49.111+0000 7f6cc2a00700 1 -- 172.21.15.18:0/3648146333 learned_addr learned my addr 172.21.15.18:0/3648146333 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:21:49.107 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:49.111+0000 7f6cc3400700 1 --2- 172.21.15.18:0/3648146333 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6cc4109430 0x7f6cc4119f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:21:49.107 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:49.111+0000 7f6cc2a00700 1 -- 172.21.15.18:0/3648146333 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6cc4109430 msgr2=0x7f6cc4119f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:21:49.107 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:49.111+0000 7f6cc2a00700 1 --2- 172.21.15.18:0/3648146333 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6cc4109430 0x7f6cc4119f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:21:49.107 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:49.111+0000 7f6cc2a00700 1 -- 172.21.15.18:0/3648146333 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6cb4005040 con 0x7f6cc4109dd0 2024-04-12T21:21:49.107 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:49.111+0000 7f6cc3400700 1 --2- 172.21.15.18:0/3648146333 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6cc4109430 0x7f6cc4119f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-12T21:21:49.107 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:49.111+0000 7f6cc2a00700 1 --2- 172.21.15.18:0/3648146333 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6cc4109dd0 0x7f6cc411a480 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f6cb000b790 tx=0x7f6cb0009220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:21:49.108 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:49.112+0000 7f6cbb400700 1 -- 172.21.15.18:0/3648146333 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6cb000f6f0 con 0x7f6cc4109dd0 2024-04-12T21:21:49.108 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:49.112+0000 7f6cbb400700 1 -- 172.21.15.18:0/3648146333 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f6cb000fd20 con 0x7f6cc4109dd0 2024-04-12T21:21:49.108 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:49.112+0000 7f6cbb400700 1 -- 172.21.15.18:0/3648146333 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6cb00103a0 con 0x7f6cc4109dd0 2024-04-12T21:21:49.108 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:49.112+0000 7f6cca600700 1 -- 172.21.15.18:0/3648146333 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6cc4114330 con 0x7f6cc4109dd0 2024-04-12T21:21:49.108 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:49.112+0000 7f6cca600700 1 -- 172.21.15.18:0/3648146333 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6cc41148f0 con 0x7f6cc4109dd0 2024-04-12T21:21:49.108 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:49.113+0000 7f6cca600700 1 -- 172.21.15.18:0/3648146333 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6cc4104b80 con 0x7f6cc4109dd0 2024-04-12T21:21:49.109 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:49.114+0000 7f6cbb400700 1 -- 172.21.15.18:0/3648146333 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7f6cb0011400 con 0x7f6cc4109dd0 2024-04-12T21:21:49.110 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:49.115+0000 7f6cbb400700 1 --2- 172.21.15.18:0/3648146333 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f6ca806ec00 0x7f6ca80710b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:21:49.110 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:49.115+0000 7f6cbb400700 1 -- 172.21.15.18:0/3648146333 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 6757+0+0 (secure 0 0 0) 0x7f6cb008e820 con 0x7f6cc4109dd0 2024-04-12T21:21:49.110 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:49.116+0000 7f6cc3400700 1 --2- 172.21.15.18:0/3648146333 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f6ca806ec00 0x7f6ca80710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:21:49.111 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:49.116+0000 7f6cc3400700 1 --2- 172.21.15.18:0/3648146333 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f6ca806ec00 0x7f6ca80710b0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f6cb400b3b0 tx=0x7f6cb4009e30 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:21:49.116 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:49.120+0000 7f6cbb400700 1 -- 172.21.15.18:0/3648146333 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f6cb005afd0 con 0x7f6cc4109dd0 2024-04-12T21:21:49.443 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:49.448+0000 7f6cca600700 1 -- 172.21.15.18:0/3648146333 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f6cc404c710 con 0x7f6cc4109dd0 2024-04-12T21:21:49.444 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:49.449+0000 7f6cbb400700 1 -- 172.21.15.18:0/3648146333 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+837 (secure 0 0 0) 0x7f6cb005ab60 con 0x7f6cc4109dd0 2024-04-12T21:21:49.444 INFO:teuthology.orchestra.run.smithi018.stdout:{ 2024-04-12T21:21:49.444 INFO:teuthology.orchestra.run.smithi018.stdout: "mon": { 2024-04-12T21:21:49.444 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 19.0.0-2879-g324baf8d (324baf8ddb91dea46cac50cb535c306115791355) squid (dev)": 2 2024-04-12T21:21:49.444 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-12T21:21:49.444 INFO:teuthology.orchestra.run.smithi018.stdout: "mgr": { 2024-04-12T21:21:49.445 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 19.0.0-2879-g324baf8d (324baf8ddb91dea46cac50cb535c306115791355) squid (dev)": 2 2024-04-12T21:21:49.445 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-12T21:21:49.445 INFO:teuthology.orchestra.run.smithi018.stdout: "osd": { 2024-04-12T21:21:49.445 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 19.0.0-2879-g324baf8d (324baf8ddb91dea46cac50cb535c306115791355) squid (dev)": 6 2024-04-12T21:21:49.445 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-12T21:21:49.445 INFO:teuthology.orchestra.run.smithi018.stdout: "mds": { 2024-04-12T21:21:49.445 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 17.2.7-827-gd3b5060a (d3b5060a17449edfae9385cdf792afec109fca5d) quincy (stable)": 2, 2024-04-12T21:21:49.445 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 19.0.0-2879-g324baf8d (324baf8ddb91dea46cac50cb535c306115791355) squid (dev)": 2 2024-04-12T21:21:49.445 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-12T21:21:49.445 INFO:teuthology.orchestra.run.smithi018.stdout: "overall": { 2024-04-12T21:21:49.445 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 17.2.7-827-gd3b5060a (d3b5060a17449edfae9385cdf792afec109fca5d) quincy (stable)": 2, 2024-04-12T21:21:49.445 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 19.0.0-2879-g324baf8d (324baf8ddb91dea46cac50cb535c306115791355) squid (dev)": 12 2024-04-12T21:21:49.445 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-04-12T21:21:49.445 INFO:teuthology.orchestra.run.smithi018.stdout:} 2024-04-12T21:21:49.447 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:49.452+0000 7f6cca600700 1 -- 172.21.15.18:0/3648146333 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f6ca806ec00 msgr2=0x7f6ca80710b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:21:49.447 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:49.452+0000 7f6cca600700 1 --2- 172.21.15.18:0/3648146333 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f6ca806ec00 0x7f6ca80710b0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f6cb400b3b0 tx=0x7f6cb4009e30 comp rx=0 tx=0).stop 2024-04-12T21:21:49.447 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:49.452+0000 7f6cca600700 1 -- 172.21.15.18:0/3648146333 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6cc4109dd0 msgr2=0x7f6cc411a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:21:49.447 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:49.452+0000 7f6cca600700 1 --2- 172.21.15.18:0/3648146333 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6cc4109dd0 0x7f6cc411a480 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f6cb000b790 tx=0x7f6cb0009220 comp rx=0 tx=0).stop 2024-04-12T21:21:49.448 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:49.452+0000 7f6cca600700 1 -- 172.21.15.18:0/3648146333 shutdown_connections 2024-04-12T21:21:49.448 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:49.452+0000 7f6cca600700 1 --2- 172.21.15.18:0/3648146333 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f6ca806ec00 0x7f6ca80710b0 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:21:49.448 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:49.452+0000 7f6cca600700 1 --2- 172.21.15.18:0/3648146333 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6cc4109430 0x7f6cc4119f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:21:49.448 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:49.452+0000 7f6cca600700 1 --2- 172.21.15.18:0/3648146333 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6cc4109dd0 0x7f6cc411a480 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:21:49.448 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:49.452+0000 7f6cca600700 1 -- 172.21.15.18:0/3648146333 >> 172.21.15.18:0/3648146333 conn(0x7f6cc4072c00 msgr2=0x7f6cc4074c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:21:49.448 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:49.452+0000 7f6cca600700 1 -- 172.21.15.18:0/3648146333 shutdown_connections 2024-04-12T21:21:49.448 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:49.453+0000 7f6cca600700 1 -- 172.21.15.18:0/3648146333 wait complete. 2024-04-12T21:21:49.595 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:49.600+0000 7ff5fb400700 1 -- 172.21.15.18:0/4279044653 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff5f4071f90 msgr2=0x7ff5f4070360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:21:49.596 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:49.600+0000 7ff5fb400700 1 --2- 172.21.15.18:0/4279044653 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff5f4071f90 0x7ff5f4070360 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7ff5e8004970 tx=0x7ff5e8009e30 comp rx=0 tx=0).stop 2024-04-12T21:21:49.596 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:49.600+0000 7ff5fb400700 1 -- 172.21.15.18:0/4279044653 shutdown_connections 2024-04-12T21:21:49.596 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:49.600+0000 7ff5fb400700 1 --2- 172.21.15.18:0/4279044653 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff5f40708a0 0x7ff5f4070d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:21:49.596 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:49.600+0000 7ff5fb400700 1 --2- 172.21.15.18:0/4279044653 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff5f4071f90 0x7ff5f4070360 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:21:49.596 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:49.600+0000 7ff5fb400700 1 -- 172.21.15.18:0/4279044653 >> 172.21.15.18:0/4279044653 conn(0x7ff5f40f89b0 msgr2=0x7ff5f40fadc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:21:49.596 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:49.600+0000 7ff5fb400700 1 -- 172.21.15.18:0/4279044653 shutdown_connections 2024-04-12T21:21:49.596 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:49.601+0000 7ff5fb400700 1 -- 172.21.15.18:0/4279044653 wait complete. 2024-04-12T21:21:49.597 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:49.601+0000 7ff5fb400700 1 Processor -- start 2024-04-12T21:21:49.597 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:49.602+0000 7ff5fb400700 1 -- start start 2024-04-12T21:21:49.597 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:49.602+0000 7ff5fb400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff5f40708a0 0x7ff5f411a1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:21:49.597 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:49.602+0000 7ff5fb400700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff5f4071f90 0x7ff5f411a6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:21:49.597 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:49.602+0000 7ff5fb400700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5f411ad30 con 0x7ff5f40708a0 2024-04-12T21:21:49.597 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:49.602+0000 7ff5fb400700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5f4114100 con 0x7ff5f4071f90 2024-04-12T21:21:49.597 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:49.602+0000 7ff5f8a00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff5f40708a0 0x7ff5f411a1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:21:49.598 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:49.603+0000 7ff5f8a00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff5f40708a0 0x7ff5f411a1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:45592/0 (socket says 172.21.15.18:45592) 2024-04-12T21:21:49.598 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:49.603+0000 7ff5f8a00700 1 -- 172.21.15.18:0/2185812937 learned_addr learned my addr 172.21.15.18:0/2185812937 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:21:49.599 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:49.603+0000 7ff5f3e00700 1 --2- 172.21.15.18:0/2185812937 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff5f4071f90 0x7ff5f411a6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:21:49.599 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:49.603+0000 7ff5f8a00700 1 -- 172.21.15.18:0/2185812937 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff5f4071f90 msgr2=0x7ff5f411a6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:21:49.599 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:49.603+0000 7ff5f8a00700 1 --2- 172.21.15.18:0/2185812937 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff5f4071f90 0x7ff5f411a6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:21:49.599 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:49.603+0000 7ff5f8a00700 1 -- 172.21.15.18:0/2185812937 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5e8005040 con 0x7ff5f40708a0 2024-04-12T21:21:49.599 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:49.603+0000 7ff5f3e00700 1 --2- 172.21.15.18:0/2185812937 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff5f4071f90 0x7ff5f411a6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-12T21:21:49.599 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:49.603+0000 7ff5f8a00700 1 --2- 172.21.15.18:0/2185812937 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff5f40708a0 0x7ff5f411a1a0 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7ff5e8004f50 tx=0x7ff5e8009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:21:49.599 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:49.604+0000 7ff5f0c00700 1 -- 172.21.15.18:0/2185812937 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff5e800e040 con 0x7ff5f40708a0 2024-04-12T21:21:49.599 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:49.604+0000 7ff5f0c00700 1 -- 172.21.15.18:0/2185812937 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff5e8004290 con 0x7ff5f40708a0 2024-04-12T21:21:49.599 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:49.604+0000 7ff5f0c00700 1 -- 172.21.15.18:0/2185812937 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff5e801c630 con 0x7ff5f40708a0 2024-04-12T21:21:49.599 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:49.604+0000 7ff5fb400700 1 -- 172.21.15.18:0/2185812937 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff5f4114380 con 0x7ff5f40708a0 2024-04-12T21:21:49.599 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:49.604+0000 7ff5fb400700 1 -- 172.21.15.18:0/2185812937 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff5f41148c0 con 0x7ff5f40708a0 2024-04-12T21:21:49.600 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:49.605+0000 7ff5fb400700 1 -- 172.21.15.18:0/2185812937 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff5f4071aa0 con 0x7ff5f40708a0 2024-04-12T21:21:49.600 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:49.606+0000 7ff5f0c00700 1 -- 172.21.15.18:0/2185812937 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7ff5e800bc10 con 0x7ff5f40708a0 2024-04-12T21:21:49.602 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:49.607+0000 7ff5f0c00700 1 --2- 172.21.15.18:0/2185812937 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7ff5dc06ec10 0x7ff5dc0710c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:21:49.602 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:49.608+0000 7ff5f0c00700 1 -- 172.21.15.18:0/2185812937 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 6757+0+0 (secure 0 0 0) 0x7ff5e808fca0 con 0x7ff5f40708a0 2024-04-12T21:21:49.602 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:49.608+0000 7ff5f3e00700 1 --2- 172.21.15.18:0/2185812937 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7ff5dc06ec10 0x7ff5dc0710c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:21:49.603 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:49.608+0000 7ff5f3e00700 1 --2- 172.21.15.18:0/2185812937 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7ff5dc06ec10 0x7ff5dc0710c0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7ff5e0009fd0 tx=0x7ff5e000b040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:21:49.607 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:49.612+0000 7ff5f0c00700 1 -- 172.21.15.18:0/2185812937 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7ff5e805c4d0 con 0x7ff5f40708a0 2024-04-12T21:21:49.778 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:21:49 smithi186 ceph-mon[86379]: from='client.35172 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:21:49.778 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:21:49 smithi186 ceph-mon[86379]: pgmap v660: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:21:49.778 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:21:49 smithi186 ceph-mon[86379]: from='client.35176 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:21:49.849 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:21:49 smithi018 ceph-mon[112590]: from='client.35172 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:21:49.849 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:21:49 smithi018 ceph-mon[112590]: pgmap v660: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:21:49.850 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:21:49 smithi018 ceph-mon[112590]: from='client.35176 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:21:49.888 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:49.893+0000 7ff5fb400700 1 -- 172.21.15.18:0/2185812937 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7ff5f404bed0 con 0x7ff5f40708a0 2024-04-12T21:21:49.888 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:49.894+0000 7ff5f0c00700 1 -- 172.21.15.18:0/2185812937 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 29 v29) v1 ==== 76+0+1872 (secure 0 0 0) 0x7ff5e805c060 con 0x7ff5f40708a0 2024-04-12T21:21:49.889 INFO:teuthology.orchestra.run.smithi018.stdout:e29 2024-04-12T21:21:49.889 INFO:teuthology.orchestra.run.smithi018.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-12T21:21:49.889 INFO:teuthology.orchestra.run.smithi018.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-12T21:21:49.889 INFO:teuthology.orchestra.run.smithi018.stdout:legacy client fscid: 1 2024-04-12T21:21:49.889 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-12T21:21:49.889 INFO:teuthology.orchestra.run.smithi018.stdout:Filesystem 'cephfs' (1) 2024-04-12T21:21:49.890 INFO:teuthology.orchestra.run.smithi018.stdout:fs_name cephfs 2024-04-12T21:21:49.890 INFO:teuthology.orchestra.run.smithi018.stdout:epoch 24 2024-04-12T21:21:49.890 INFO:teuthology.orchestra.run.smithi018.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-04-12T21:21:49.890 INFO:teuthology.orchestra.run.smithi018.stdout:created 2024-04-12T20:51:57.362850+0000 2024-04-12T21:21:49.890 INFO:teuthology.orchestra.run.smithi018.stdout:modified 2024-04-12T21:06:59.717190+0000 2024-04-12T21:21:49.890 INFO:teuthology.orchestra.run.smithi018.stdout:tableserver 0 2024-04-12T21:21:49.890 INFO:teuthology.orchestra.run.smithi018.stdout:root 0 2024-04-12T21:21:49.890 INFO:teuthology.orchestra.run.smithi018.stdout:session_timeout 60 2024-04-12T21:21:49.890 INFO:teuthology.orchestra.run.smithi018.stdout:session_autoclose 300 2024-04-12T21:21:49.890 INFO:teuthology.orchestra.run.smithi018.stdout:max_file_size 1099511627776 2024-04-12T21:21:49.890 INFO:teuthology.orchestra.run.smithi018.stdout:max_xattr_size 65536 2024-04-12T21:21:49.890 INFO:teuthology.orchestra.run.smithi018.stdout:required_client_features {} 2024-04-12T21:21:49.890 INFO:teuthology.orchestra.run.smithi018.stdout:last_failure 0 2024-04-12T21:21:49.891 INFO:teuthology.orchestra.run.smithi018.stdout:last_failure_osd_epoch 100 2024-04-12T21:21:49.891 INFO:teuthology.orchestra.run.smithi018.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-12T21:21:49.891 INFO:teuthology.orchestra.run.smithi018.stdout:max_mds 2 2024-04-12T21:21:49.891 INFO:teuthology.orchestra.run.smithi018.stdout:in 0,1 2024-04-12T21:21:49.891 INFO:teuthology.orchestra.run.smithi018.stdout:up {} 2024-04-12T21:21:49.891 INFO:teuthology.orchestra.run.smithi018.stdout:failed 0,1 2024-04-12T21:21:49.891 INFO:teuthology.orchestra.run.smithi018.stdout:damaged 2024-04-12T21:21:49.891 INFO:teuthology.orchestra.run.smithi018.stdout:stopped 2024-04-12T21:21:49.891 INFO:teuthology.orchestra.run.smithi018.stdout:data_pools [3] 2024-04-12T21:21:49.891 INFO:teuthology.orchestra.run.smithi018.stdout:metadata_pool 2 2024-04-12T21:21:49.891 INFO:teuthology.orchestra.run.smithi018.stdout:inline_data disabled 2024-04-12T21:21:49.891 INFO:teuthology.orchestra.run.smithi018.stdout:balancer 2024-04-12T21:21:49.891 INFO:teuthology.orchestra.run.smithi018.stdout:bal_rank_mask -1 2024-04-12T21:21:49.891 INFO:teuthology.orchestra.run.smithi018.stdout:standby_count_wanted 1 2024-04-12T21:21:49.891 INFO:teuthology.orchestra.run.smithi018.stdout:qdb_cluster leader: 0 members: 2024-04-12T21:21:49.892 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-12T21:21:49.892 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-12T21:21:49.892 INFO:teuthology.orchestra.run.smithi018.stdout:Standby daemons: 2024-04-12T21:21:49.892 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-12T21:21:49.892 INFO:teuthology.orchestra.run.smithi018.stdout:[mds.cephfs.smithi186.wjnafs{-1:24277} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.186:6826/438347207,v1:172.21.15.186:6827/438347207] compat {c=[1],r=[1],i=[7ff]}] 2024-04-12T21:21:49.892 INFO:teuthology.orchestra.run.smithi018.stdout:[mds.cephfs.smithi186.jdllyn{-1:24281} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.186:6824/1464359952,v1:172.21.15.186:6825/1464359952] compat {c=[1],r=[1],i=[7ff]}] 2024-04-12T21:21:49.892 INFO:teuthology.orchestra.run.smithi018.stdout:[mds.cephfs.smithi018.acqbsc{-1:34436} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.18:6828/755179764,v1:172.21.15.18:6829/755179764] compat {c=[1],r=[1],i=[1fff]}] 2024-04-12T21:21:49.892 INFO:teuthology.orchestra.run.smithi018.stdout:[mds.cephfs.smithi018.szjluo{-1:34468} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.18:6826/2407992653,v1:172.21.15.18:6827/2407992653] compat {c=[1],r=[1],i=[1fff]}] 2024-04-12T21:21:49.893 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:49.896+0000 7ff5fb400700 1 -- 172.21.15.18:0/2185812937 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7ff5dc06ec10 msgr2=0x7ff5dc0710c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:21:49.893 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:49.896+0000 7ff5fb400700 1 --2- 172.21.15.18:0/2185812937 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7ff5dc06ec10 0x7ff5dc0710c0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7ff5e0009fd0 tx=0x7ff5e000b040 comp rx=0 tx=0).stop 2024-04-12T21:21:49.893 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:49.897+0000 7ff5fb400700 1 -- 172.21.15.18:0/2185812937 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff5f40708a0 msgr2=0x7ff5f411a1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:21:49.893 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:49.897+0000 7ff5fb400700 1 --2- 172.21.15.18:0/2185812937 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff5f40708a0 0x7ff5f411a1a0 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7ff5e8004f50 tx=0x7ff5e8009e30 comp rx=0 tx=0).stop 2024-04-12T21:21:49.893 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:49.897+0000 7ff5fb400700 1 -- 172.21.15.18:0/2185812937 shutdown_connections 2024-04-12T21:21:49.893 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:49.897+0000 7ff5fb400700 1 --2- 172.21.15.18:0/2185812937 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7ff5dc06ec10 0x7ff5dc0710c0 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:21:49.893 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:49.897+0000 7ff5fb400700 1 --2- 172.21.15.18:0/2185812937 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff5f40708a0 0x7ff5f411a1a0 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:21:49.893 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:49.897+0000 7ff5fb400700 1 --2- 172.21.15.18:0/2185812937 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff5f4071f90 0x7ff5f411a6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:21:49.893 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:49.897+0000 7ff5fb400700 1 -- 172.21.15.18:0/2185812937 >> 172.21.15.18:0/2185812937 conn(0x7ff5f40f89b0 msgr2=0x7ff5f40f9aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:21:49.893 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:49.897+0000 7ff5fb400700 1 -- 172.21.15.18:0/2185812937 shutdown_connections 2024-04-12T21:21:49.894 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:49.897+0000 7ff5fb400700 1 -- 172.21.15.18:0/2185812937 wait complete. 2024-04-12T21:21:49.894 INFO:teuthology.orchestra.run.smithi018.stderr:dumped fsmap epoch 29 2024-04-12T21:21:50.053 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:50.058+0000 7fee80600700 1 -- 172.21.15.18:0/1285373919 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fee78109dd0 msgr2=0x7fee78103440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:21:50.053 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:50.058+0000 7fee80600700 1 --2- 172.21.15.18:0/1285373919 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fee78109dd0 0x7fee78103440 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7fee64004940 tx=0x7fee64009e30 comp rx=0 tx=0).stop 2024-04-12T21:21:50.054 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:50.058+0000 7fee80600700 1 -- 172.21.15.18:0/1285373919 shutdown_connections 2024-04-12T21:21:50.054 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:50.058+0000 7fee80600700 1 --2- 172.21.15.18:0/1285373919 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fee78109dd0 0x7fee78103440 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:21:50.054 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:50.058+0000 7fee80600700 1 --2- 172.21.15.18:0/1285373919 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fee78109430 0x7fee78109800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:21:50.054 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:50.058+0000 7fee80600700 1 -- 172.21.15.18:0/1285373919 >> 172.21.15.18:0/1285373919 conn(0x7fee78072c00 msgr2=0x7fee78075010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:21:50.054 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:50.058+0000 7fee80600700 1 -- 172.21.15.18:0/1285373919 shutdown_connections 2024-04-12T21:21:50.054 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:50.058+0000 7fee80600700 1 -- 172.21.15.18:0/1285373919 wait complete. 2024-04-12T21:21:50.055 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:50.059+0000 7fee80600700 1 Processor -- start 2024-04-12T21:21:50.055 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:50.059+0000 7fee80600700 1 -- start start 2024-04-12T21:21:50.055 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:50.060+0000 7fee80600700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fee78109430 0x7fee78119f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:21:50.055 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:50.060+0000 7fee80600700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fee78109dd0 0x7fee7811a4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:21:50.055 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:50.060+0000 7fee80600700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee7811ab10 con 0x7fee78109dd0 2024-04-12T21:21:50.055 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:50.060+0000 7fee80600700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee781140f0 con 0x7fee78109430 2024-04-12T21:21:50.055 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:50.060+0000 7fee7d200700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fee78109dd0 0x7fee7811a4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:21:50.056 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:50.060+0000 7fee7d200700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fee78109dd0 0x7fee7811a4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:45602/0 (socket says 172.21.15.18:45602) 2024-04-12T21:21:50.056 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:50.060+0000 7fee7d200700 1 -- 172.21.15.18:0/3117034332 learned_addr learned my addr 172.21.15.18:0/3117034332 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:21:50.056 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:50.060+0000 7fee7d200700 1 -- 172.21.15.18:0/3117034332 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fee78109430 msgr2=0x7fee78119f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:21:50.056 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:50.060+0000 7fee7d200700 1 --2- 172.21.15.18:0/3117034332 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fee78109430 0x7fee78119f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:21:50.056 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:50.060+0000 7fee7d200700 1 -- 172.21.15.18:0/3117034332 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee64005040 con 0x7fee78109dd0 2024-04-12T21:21:50.057 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:50.061+0000 7fee7d200700 1 --2- 172.21.15.18:0/3117034332 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fee78109dd0 0x7fee7811a4c0 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7fee64009100 tx=0x7fee64009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:21:50.057 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:50.061+0000 7fee75600700 1 -- 172.21.15.18:0/3117034332 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fee6400e040 con 0x7fee78109dd0 2024-04-12T21:21:50.057 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:50.061+0000 7fee75600700 1 -- 172.21.15.18:0/3117034332 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fee64013420 con 0x7fee78109dd0 2024-04-12T21:21:50.057 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:50.061+0000 7fee80600700 1 -- 172.21.15.18:0/3117034332 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fee78114370 con 0x7fee78109dd0 2024-04-12T21:21:50.057 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:50.061+0000 7fee75600700 1 -- 172.21.15.18:0/3117034332 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fee6401c630 con 0x7fee78109dd0 2024-04-12T21:21:50.057 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:50.061+0000 7fee80600700 1 -- 172.21.15.18:0/3117034332 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fee78114930 con 0x7fee78109dd0 2024-04-12T21:21:50.057 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:50.062+0000 7fee80600700 1 -- 172.21.15.18:0/3117034332 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fee78104b80 con 0x7fee78109dd0 2024-04-12T21:21:50.058 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:50.064+0000 7fee75600700 1 -- 172.21.15.18:0/3117034332 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7fee64013590 con 0x7fee78109dd0 2024-04-12T21:21:50.060 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:50.065+0000 7fee75600700 1 --2- 172.21.15.18:0/3117034332 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fee6006ec00 0x7fee600710b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:21:50.060 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:50.065+0000 7fee75600700 1 -- 172.21.15.18:0/3117034332 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 6757+0+0 (secure 0 0 0) 0x7fee6408fba0 con 0x7fee78109dd0 2024-04-12T21:21:50.060 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:50.065+0000 7fee7dc00700 1 --2- 172.21.15.18:0/3117034332 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fee6006ec00 0x7fee600710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:21:50.061 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:50.066+0000 7fee7dc00700 1 --2- 172.21.15.18:0/3117034332 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fee6006ec00 0x7fee600710b0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fee6c007fd0 tx=0x7fee6c007bb0 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:21:50.064 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:50.070+0000 7fee75600700 1 -- 172.21.15.18:0/3117034332 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7fee6405c3d0 con 0x7fee78109dd0 2024-04-12T21:21:50.317 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:50.322+0000 7fee80600700 1 -- 172.21.15.18:0/3117034332 --> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fee78110570 con 0x7fee6006ec00 2024-04-12T21:21:50.319 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:50.324+0000 7fee75600700 1 -- 172.21.15.18:0/3117034332 <== mgr.34102 v2:172.21.15.18:6800/3812891919 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7fee78110570 con 0x7fee6006ec00 2024-04-12T21:21:50.319 INFO:teuthology.orchestra.run.smithi018.stdout:{ 2024-04-12T21:21:50.319 INFO:teuthology.orchestra.run.smithi018.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:f4452def3be4d350f4a8e5feaf7ebb1511be9298fe68478ba7a2d6fbd7d69a63", 2024-04-12T21:21:50.320 INFO:teuthology.orchestra.run.smithi018.stdout: "in_progress": true, 2024-04-12T21:21:50.320 INFO:teuthology.orchestra.run.smithi018.stdout: "which": "Upgrading all daemon types on all hosts", 2024-04-12T21:21:50.320 INFO:teuthology.orchestra.run.smithi018.stdout: "services_complete": [ 2024-04-12T21:21:50.320 INFO:teuthology.orchestra.run.smithi018.stdout: "crash", 2024-04-12T21:21:50.320 INFO:teuthology.orchestra.run.smithi018.stdout: "osd", 2024-04-12T21:21:50.320 INFO:teuthology.orchestra.run.smithi018.stdout: "mon", 2024-04-12T21:21:50.320 INFO:teuthology.orchestra.run.smithi018.stdout: "mgr" 2024-04-12T21:21:50.320 INFO:teuthology.orchestra.run.smithi018.stdout: ], 2024-04-12T21:21:50.320 INFO:teuthology.orchestra.run.smithi018.stdout: "progress": "14/21 daemons upgraded", 2024-04-12T21:21:50.320 INFO:teuthology.orchestra.run.smithi018.stdout: "message": "Currently upgrading mds daemons", 2024-04-12T21:21:50.320 INFO:teuthology.orchestra.run.smithi018.stdout: "is_paused": false 2024-04-12T21:21:50.320 INFO:teuthology.orchestra.run.smithi018.stdout:} 2024-04-12T21:21:50.322 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:50.327+0000 7fee80600700 1 -- 172.21.15.18:0/3117034332 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fee6006ec00 msgr2=0x7fee600710b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:21:50.322 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:50.327+0000 7fee80600700 1 --2- 172.21.15.18:0/3117034332 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fee6006ec00 0x7fee600710b0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fee6c007fd0 tx=0x7fee6c007bb0 comp rx=0 tx=0).stop 2024-04-12T21:21:50.323 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:50.327+0000 7fee80600700 1 -- 172.21.15.18:0/3117034332 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fee78109dd0 msgr2=0x7fee7811a4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:21:50.323 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:50.327+0000 7fee80600700 1 --2- 172.21.15.18:0/3117034332 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fee78109dd0 0x7fee7811a4c0 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7fee64009100 tx=0x7fee64009e30 comp rx=0 tx=0).stop 2024-04-12T21:21:50.323 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:50.327+0000 7fee80600700 1 -- 172.21.15.18:0/3117034332 shutdown_connections 2024-04-12T21:21:50.323 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:50.327+0000 7fee80600700 1 --2- 172.21.15.18:0/3117034332 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fee6006ec00 0x7fee600710b0 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:21:50.323 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:50.327+0000 7fee80600700 1 --2- 172.21.15.18:0/3117034332 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fee78109430 0x7fee78119f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:21:50.323 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:50.327+0000 7fee80600700 1 --2- 172.21.15.18:0/3117034332 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fee78109dd0 0x7fee7811a4c0 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:21:50.323 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:50.327+0000 7fee80600700 1 -- 172.21.15.18:0/3117034332 >> 172.21.15.18:0/3117034332 conn(0x7fee78072c00 msgr2=0x7fee78074c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:21:50.323 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:50.328+0000 7fee80600700 1 -- 172.21.15.18:0/3117034332 shutdown_connections 2024-04-12T21:21:50.323 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:50.328+0000 7fee80600700 1 -- 172.21.15.18:0/3117034332 wait complete. 2024-04-12T21:21:50.479 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:50.481+0000 7f5186600700 1 -- 172.21.15.18:0/648910355 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5180105020 msgr2=0x7f51801053f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:21:50.479 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:50.481+0000 7f5186600700 1 --2- 172.21.15.18:0/648910355 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5180105020 0x7f51801053f0 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f5170004940 tx=0x7f5170009e30 comp rx=0 tx=0).stop 2024-04-12T21:21:50.479 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:50.482+0000 7f5186600700 1 -- 172.21.15.18:0/648910355 shutdown_connections 2024-04-12T21:21:50.479 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:50.482+0000 7f5186600700 1 --2- 172.21.15.18:0/648910355 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f51801059c0 0x7f51800ff080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:21:50.479 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:50.482+0000 7f5186600700 1 --2- 172.21.15.18:0/648910355 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5180105020 0x7f51801053f0 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:21:50.480 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:50.482+0000 7f5186600700 1 -- 172.21.15.18:0/648910355 >> 172.21.15.18:0/648910355 conn(0x7f51800faba0 msgr2=0x7f51800fcfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:21:50.480 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:50.482+0000 7f5186600700 1 -- 172.21.15.18:0/648910355 shutdown_connections 2024-04-12T21:21:50.480 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:50.482+0000 7f5186600700 1 -- 172.21.15.18:0/648910355 wait complete. 2024-04-12T21:21:50.480 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:50.482+0000 7f5186600700 1 Processor -- start 2024-04-12T21:21:50.480 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:50.483+0000 7f5186600700 1 -- start start 2024-04-12T21:21:50.480 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:50.483+0000 7f5186600700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5180105020 0x7f5180115bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:21:50.480 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:50.483+0000 7f5186600700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f51801059c0 0x7f5180116130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:21:50.480 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:50.483+0000 7f5186600700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5180116810 con 0x7f5180105020 2024-04-12T21:21:50.480 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:50.483+0000 7f5186600700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f518010fd60 con 0x7f51801059c0 2024-04-12T21:21:50.480 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:50.483+0000 7f517f400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5180105020 0x7f5180115bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:21:50.480 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:50.483+0000 7f517f400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5180105020 0x7f5180115bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:45612/0 (socket says 172.21.15.18:45612) 2024-04-12T21:21:50.480 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:50.483+0000 7f517f400700 1 -- 172.21.15.18:0/760762720 learned_addr learned my addr 172.21.15.18:0/760762720 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:21:50.480 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:50.484+0000 7f517f400700 1 -- 172.21.15.18:0/760762720 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f51801059c0 msgr2=0x7f5180116130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:21:50.481 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:50.484+0000 7f517f400700 1 --2- 172.21.15.18:0/760762720 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f51801059c0 0x7f5180116130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:21:50.481 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:50.484+0000 7f517f400700 1 -- 172.21.15.18:0/760762720 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5170005040 con 0x7f5180105020 2024-04-12T21:21:50.481 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:50.484+0000 7f517f400700 1 --2- 172.21.15.18:0/760762720 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5180105020 0x7f5180115bf0 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f5170009150 tx=0x7f5170009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:21:50.481 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:50.484+0000 7f5177400700 1 -- 172.21.15.18:0/760762720 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f517000e040 con 0x7f5180105020 2024-04-12T21:21:50.481 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:50.484+0000 7f5186600700 1 -- 172.21.15.18:0/760762720 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f518010ff80 con 0x7f5180105020 2024-04-12T21:21:50.481 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:50.484+0000 7f5177400700 1 -- 172.21.15.18:0/760762720 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f5170013420 con 0x7f5180105020 2024-04-12T21:21:50.481 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:50.484+0000 7f5177400700 1 -- 172.21.15.18:0/760762720 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f517001c630 con 0x7f5180105020 2024-04-12T21:21:50.481 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:50.484+0000 7f5186600700 1 -- 172.21.15.18:0/760762720 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f51801104c0 con 0x7f5180105020 2024-04-12T21:21:50.482 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:50.485+0000 7f5186600700 1 -- 172.21.15.18:0/760762720 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5180100770 con 0x7f5180105020 2024-04-12T21:21:50.482 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:50.487+0000 7f5177400700 1 -- 172.21.15.18:0/760762720 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7f5170013590 con 0x7f5180105020 2024-04-12T21:21:50.482 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:50.488+0000 7f5177400700 1 --2- 172.21.15.18:0/760762720 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f516406ec00 0x7f51640710b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:21:50.483 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:50.488+0000 7f5177400700 1 -- 172.21.15.18:0/760762720 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 6757+0+0 (secure 0 0 0) 0x7f517008fc10 con 0x7f5180105020 2024-04-12T21:21:50.483 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:50.488+0000 7f517ea00700 1 --2- 172.21.15.18:0/760762720 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f516406ec00 0x7f51640710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:21:50.483 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:50.489+0000 7f517ea00700 1 --2- 172.21.15.18:0/760762720 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f516406ec00 0x7f51640710b0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f516c006fd0 tx=0x7f516c006db0 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:21:50.488 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:50.493+0000 7f5177400700 1 -- 172.21.15.18:0/760762720 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f517005c440 con 0x7f5180105020 2024-04-12T21:21:50.599 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:21:50 smithi018 ceph-mon[112590]: from='client.35180 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:21:50.599 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:21:50 smithi018 ceph-mon[112590]: from='client.? 172.21.15.18:0/3648146333' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T21:21:50.600 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:21:50 smithi018 ceph-mon[112590]: from='client.? 172.21.15.18:0/2185812937' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-12T21:21:50.778 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:21:50 smithi186 ceph-mon[86379]: from='client.35180 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:21:50.778 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:21:50 smithi186 ceph-mon[86379]: from='client.? 172.21.15.18:0/3648146333' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T21:21:50.779 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:21:50 smithi186 ceph-mon[86379]: from='client.? 172.21.15.18:0/2185812937' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-12T21:21:50.816 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:50.821+0000 7f5186600700 1 -- 172.21.15.18:0/760762720 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f518010c220 con 0x7f5180105020 2024-04-12T21:21:50.817 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:50.822+0000 7f5177400700 1 -- 172.21.15.18:0/760762720 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+233 (secure 0 0 0) 0x7f517005bfd0 con 0x7f5180105020 2024-04-12T21:21:50.817 INFO:teuthology.orchestra.run.smithi018.stdout:HEALTH_ERR 1 filesystem is degraded; 1 filesystem is offline 2024-04-12T21:21:50.817 INFO:teuthology.orchestra.run.smithi018.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-04-12T21:21:50.817 INFO:teuthology.orchestra.run.smithi018.stdout: fs cephfs is degraded 2024-04-12T21:21:50.817 INFO:teuthology.orchestra.run.smithi018.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-04-12T21:21:50.817 INFO:teuthology.orchestra.run.smithi018.stdout: fs cephfs is offline because no MDS is active for it. 2024-04-12T21:21:50.820 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:50.825+0000 7f5186600700 1 -- 172.21.15.18:0/760762720 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f516406ec00 msgr2=0x7f51640710b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:21:50.820 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:50.825+0000 7f5186600700 1 --2- 172.21.15.18:0/760762720 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f516406ec00 0x7f51640710b0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f516c006fd0 tx=0x7f516c006db0 comp rx=0 tx=0).stop 2024-04-12T21:21:50.820 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:50.825+0000 7f5186600700 1 -- 172.21.15.18:0/760762720 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5180105020 msgr2=0x7f5180115bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:21:50.821 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:50.825+0000 7f5186600700 1 --2- 172.21.15.18:0/760762720 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5180105020 0x7f5180115bf0 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f5170009150 tx=0x7f5170009e30 comp rx=0 tx=0).stop 2024-04-12T21:21:50.821 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:50.825+0000 7f5186600700 1 -- 172.21.15.18:0/760762720 shutdown_connections 2024-04-12T21:21:50.821 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:50.825+0000 7f5186600700 1 --2- 172.21.15.18:0/760762720 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f516406ec00 0x7f51640710b0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:21:50.821 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:50.825+0000 7f5186600700 1 --2- 172.21.15.18:0/760762720 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5180105020 0x7f5180115bf0 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:21:50.821 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:50.825+0000 7f5186600700 1 --2- 172.21.15.18:0/760762720 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f51801059c0 0x7f5180116130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:21:50.821 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:50.825+0000 7f5186600700 1 -- 172.21.15.18:0/760762720 >> 172.21.15.18:0/760762720 conn(0x7f51800faba0 msgr2=0x7f51800fbb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:21:50.821 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:50.826+0000 7f5186600700 1 -- 172.21.15.18:0/760762720 shutdown_connections 2024-04-12T21:21:50.821 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:21:50.826+0000 7f5186600700 1 -- 172.21.15.18:0/760762720 wait complete. 2024-04-12T21:21:51.778 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:21:51 smithi186 ceph-mon[86379]: pgmap v661: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:21:51.778 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:21:51 smithi186 ceph-mon[86379]: from='client.35192 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:21:51.779 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:21:51 smithi186 ceph-mon[86379]: from='client.? 172.21.15.18:0/760762720' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-12T21:21:51.849 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:21:51 smithi018 ceph-mon[112590]: pgmap v661: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:21:51.850 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:21:51 smithi018 ceph-mon[112590]: from='client.35192 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:21:51.850 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:21:51 smithi018 ceph-mon[112590]: from='client.? 172.21.15.18:0/760762720' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-12T21:21:53.778 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:21:53 smithi186 ceph-mon[86379]: pgmap v662: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:21:53.849 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:21:53 smithi018 ceph-mon[112590]: pgmap v662: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:21:55.778 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:21:55 smithi186 ceph-mon[86379]: pgmap v663: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:21:55.778 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:21:55 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T21:21:55.849 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:21:55 smithi018 ceph-mon[112590]: pgmap v663: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:21:55.849 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:21:55 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T21:21:57.778 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:21:57 smithi186 ceph-mon[86379]: pgmap v664: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:21:57.849 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:21:57 smithi018 ceph-mon[112590]: pgmap v664: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:21:59.778 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:21:59 smithi186 ceph-mon[86379]: pgmap v665: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:21:59.849 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:21:59 smithi018 ceph-mon[112590]: pgmap v665: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:22:01.778 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:22:01 smithi186 ceph-mon[86379]: pgmap v666: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:22:01.849 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:22:01 smithi018 ceph-mon[112590]: pgmap v666: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:22:03.778 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:22:03 smithi186 ceph-mon[86379]: pgmap v667: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:22:03.849 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:22:03 smithi018 ceph-mon[112590]: pgmap v667: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:22:05.778 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:22:05 smithi186 ceph-mon[86379]: pgmap v668: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:22:05.849 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:22:05 smithi018 ceph-mon[112590]: pgmap v668: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:22:07.778 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:22:07 smithi186 ceph-mon[86379]: pgmap v669: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:22:07.849 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:22:07 smithi018 ceph-mon[112590]: pgmap v669: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:22:09.778 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:22:09 smithi186 ceph-mon[86379]: pgmap v670: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:22:09.849 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:22:09 smithi018 ceph-mon[112590]: pgmap v670: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:22:10.778 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:22:10 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T21:22:10.849 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:22:10 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T21:22:11.778 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:22:11 smithi186 ceph-mon[86379]: pgmap v671: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:22:11.849 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:22:11 smithi018 ceph-mon[112590]: pgmap v671: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:22:13.781 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:22:13 smithi186 ceph-mon[86379]: pgmap v672: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:22:13.849 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:22:13 smithi018 ceph-mon[112590]: pgmap v672: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:22:15.778 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:22:15 smithi186 ceph-mon[86379]: pgmap v673: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:22:15.849 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:22:15 smithi018 ceph-mon[112590]: pgmap v673: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:22:17.778 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:22:17 smithi186 ceph-mon[86379]: pgmap v674: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:22:17.849 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:22:17 smithi018 ceph-mon[112590]: pgmap v674: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:22:19.849 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:22:19 smithi018 ceph-mon[112590]: pgmap v675: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:22:20.028 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:22:19 smithi186 ceph-mon[86379]: pgmap v675: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:22:20.990 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:20.994+0000 7f498fe00700 1 -- 172.21.15.18:0/4026964710 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4988109dd0 msgr2=0x7f4988103440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:22:20.991 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:20.994+0000 7f498fe00700 1 --2- 172.21.15.18:0/4026964710 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4988109dd0 0x7f4988103440 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f4974004940 tx=0x7f4974009e30 comp rx=0 tx=0).stop 2024-04-12T21:22:20.991 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:20.994+0000 7f498fe00700 1 -- 172.21.15.18:0/4026964710 shutdown_connections 2024-04-12T21:22:20.991 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:20.994+0000 7f498fe00700 1 --2- 172.21.15.18:0/4026964710 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4988109dd0 0x7f4988103440 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:22:20.991 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:20.994+0000 7f498fe00700 1 --2- 172.21.15.18:0/4026964710 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4988109430 0x7f4988109800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:22:20.991 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:20.994+0000 7f498fe00700 1 -- 172.21.15.18:0/4026964710 >> 172.21.15.18:0/4026964710 conn(0x7f4988072c00 msgr2=0x7f4988075010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:22:20.991 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:20.994+0000 7f498fe00700 1 -- 172.21.15.18:0/4026964710 shutdown_connections 2024-04-12T21:22:20.991 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:20.994+0000 7f498fe00700 1 -- 172.21.15.18:0/4026964710 wait complete. 2024-04-12T21:22:20.992 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:20.995+0000 7f498fe00700 1 Processor -- start 2024-04-12T21:22:20.993 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:20.995+0000 7f498fe00700 1 -- start start 2024-04-12T21:22:20.993 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:20.996+0000 7f498fe00700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4988109430 0x7f4988119fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:22:20.993 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:20.996+0000 7f498fe00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4988109dd0 0x7f498811a4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:22:20.993 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:20.996+0000 7f498fe00700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f498811abc0 con 0x7f4988109dd0 2024-04-12T21:22:20.993 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:20.996+0000 7f498fe00700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4988114110 con 0x7f4988109430 2024-04-12T21:22:20.993 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:20.996+0000 7f498ca00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4988109dd0 0x7f498811a4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:22:20.993 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:20.996+0000 7f498ca00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4988109dd0 0x7f498811a4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:55214/0 (socket says 172.21.15.18:55214) 2024-04-12T21:22:20.993 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:20.996+0000 7f498ca00700 1 -- 172.21.15.18:0/4081856566 learned_addr learned my addr 172.21.15.18:0/4081856566 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:22:20.993 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:20.996+0000 7f498d400700 1 --2- 172.21.15.18:0/4081856566 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4988109430 0x7f4988119fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:22:20.993 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:20.997+0000 7f498ca00700 1 -- 172.21.15.18:0/4081856566 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4988109430 msgr2=0x7f4988119fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:22:20.993 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:20.997+0000 7f498ca00700 1 --2- 172.21.15.18:0/4081856566 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4988109430 0x7f4988119fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:22:20.993 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:20.997+0000 7f498ca00700 1 -- 172.21.15.18:0/4081856566 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4974005040 con 0x7f4988109dd0 2024-04-12T21:22:20.993 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:20.997+0000 7f498ca00700 1 --2- 172.21.15.18:0/4081856566 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4988109dd0 0x7f498811a4e0 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f4974009100 tx=0x7f4974009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:22:20.994 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:20.997+0000 7f4985600700 1 -- 172.21.15.18:0/4081856566 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f497400e040 con 0x7f4988109dd0 2024-04-12T21:22:20.995 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:20.997+0000 7f4985600700 1 -- 172.21.15.18:0/4081856566 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f4974013420 con 0x7f4988109dd0 2024-04-12T21:22:20.995 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:20.997+0000 7f4985600700 1 -- 172.21.15.18:0/4081856566 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f497401c630 con 0x7f4988109dd0 2024-04-12T21:22:20.995 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:20.997+0000 7f498fe00700 1 -- 172.21.15.18:0/4081856566 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4988114330 con 0x7f4988109dd0 2024-04-12T21:22:20.995 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:20.998+0000 7f498fe00700 1 -- 172.21.15.18:0/4081856566 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f49881148f0 con 0x7f4988109dd0 2024-04-12T21:22:20.995 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:20.999+0000 7f498fe00700 1 -- 172.21.15.18:0/4081856566 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4988104b80 con 0x7f4988109dd0 2024-04-12T21:22:20.996 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:21.000+0000 7f4985600700 1 -- 172.21.15.18:0/4081856566 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7f4974013590 con 0x7f4988109dd0 2024-04-12T21:22:20.998 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:21.002+0000 7f4985600700 1 --2- 172.21.15.18:0/4081856566 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f497006ec00 0x7f49700710b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:22:20.998 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:21.002+0000 7f4985600700 1 -- 172.21.15.18:0/4081856566 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 6757+0+0 (secure 0 0 0) 0x7f497408fba0 con 0x7f4988109dd0 2024-04-12T21:22:20.998 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:21.003+0000 7f498d400700 1 --2- 172.21.15.18:0/4081856566 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f497006ec00 0x7f49700710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:22:20.999 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:21.003+0000 7f498d400700 1 --2- 172.21.15.18:0/4081856566 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f497006ec00 0x7f49700710b0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f497c006fd0 tx=0x7f497c008040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:22:21.003 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:21.007+0000 7f4985600700 1 -- 172.21.15.18:0/4081856566 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f497405c3d0 con 0x7f4988109dd0 2024-04-12T21:22:21.257 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:21.261+0000 7f498fe00700 1 -- 172.21.15.18:0/4081856566 --> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f4988110620 con 0x7f497006ec00 2024-04-12T21:22:21.258 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:21.263+0000 7f4985600700 1 -- 172.21.15.18:0/4081856566 <== mgr.34102 v2:172.21.15.18:6800/3812891919 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f497c00d660 con 0x7f497006ec00 2024-04-12T21:22:21.262 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:21.266+0000 7f498fe00700 1 -- 172.21.15.18:0/4081856566 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f497006ec00 msgr2=0x7f49700710b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:22:21.262 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:21.266+0000 7f498fe00700 1 --2- 172.21.15.18:0/4081856566 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f497006ec00 0x7f49700710b0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f497c006fd0 tx=0x7f497c008040 comp rx=0 tx=0).stop 2024-04-12T21:22:21.262 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:21.266+0000 7f498fe00700 1 -- 172.21.15.18:0/4081856566 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4988109dd0 msgr2=0x7f498811a4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:22:21.262 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:21.266+0000 7f498fe00700 1 --2- 172.21.15.18:0/4081856566 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4988109dd0 0x7f498811a4e0 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f4974009100 tx=0x7f4974009e30 comp rx=0 tx=0).stop 2024-04-12T21:22:21.262 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:21.266+0000 7f498fe00700 1 -- 172.21.15.18:0/4081856566 shutdown_connections 2024-04-12T21:22:21.262 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:21.266+0000 7f498fe00700 1 --2- 172.21.15.18:0/4081856566 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f497006ec00 0x7f49700710b0 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:22:21.263 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:21.266+0000 7f498fe00700 1 --2- 172.21.15.18:0/4081856566 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4988109430 0x7f4988119fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:22:21.263 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:21.266+0000 7f498fe00700 1 --2- 172.21.15.18:0/4081856566 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4988109dd0 0x7f498811a4e0 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:22:21.263 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:21.266+0000 7f498fe00700 1 -- 172.21.15.18:0/4081856566 >> 172.21.15.18:0/4081856566 conn(0x7f4988072c00 msgr2=0x7f4988074b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:22:21.263 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:21.266+0000 7f498fe00700 1 -- 172.21.15.18:0/4081856566 shutdown_connections 2024-04-12T21:22:21.263 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:21.267+0000 7f498fe00700 1 -- 172.21.15.18:0/4081856566 wait complete. 2024-04-12T21:22:21.280 INFO:teuthology.orchestra.run.smithi018.stdout:true 2024-04-12T21:22:21.421 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:21.425+0000 7f4e1f600700 1 -- 172.21.15.18:0/2224038773 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4e180708a0 msgr2=0x7f4e18070d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:22:21.422 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:21.425+0000 7f4e1f600700 1 --2- 172.21.15.18:0/2224038773 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4e180708a0 0x7f4e18070d10 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f4e04004970 tx=0x7f4e04009e30 comp rx=0 tx=0).stop 2024-04-12T21:22:21.422 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:21.426+0000 7f4e1f600700 1 -- 172.21.15.18:0/2224038773 shutdown_connections 2024-04-12T21:22:21.422 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:21.426+0000 7f4e1f600700 1 --2- 172.21.15.18:0/2224038773 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4e180708a0 0x7f4e18070d10 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:22:21.422 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:21.426+0000 7f4e1f600700 1 --2- 172.21.15.18:0/2224038773 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4e18071f90 0x7f4e18070360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:22:21.422 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:21.426+0000 7f4e1f600700 1 -- 172.21.15.18:0/2224038773 >> 172.21.15.18:0/2224038773 conn(0x7f4e180f8970 msgr2=0x7f4e180fad80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:22:21.422 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:21.426+0000 7f4e1f600700 1 -- 172.21.15.18:0/2224038773 shutdown_connections 2024-04-12T21:22:21.422 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:21.426+0000 7f4e1f600700 1 -- 172.21.15.18:0/2224038773 wait complete. 2024-04-12T21:22:21.423 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:21.427+0000 7f4e1f600700 1 Processor -- start 2024-04-12T21:22:21.424 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:21.427+0000 7f4e1f600700 1 -- start start 2024-04-12T21:22:21.424 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:21.427+0000 7f4e1f600700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4e180708a0 0x7f4e1811a160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:22:21.424 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:21.427+0000 7f4e1f600700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4e18071f90 0x7f4e1811a6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:22:21.424 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:21.427+0000 7f4e1f600700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e1811acf0 con 0x7f4e18071f90 2024-04-12T21:22:21.424 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:21.427+0000 7f4e1f600700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e181140c0 con 0x7f4e180708a0 2024-04-12T21:22:21.424 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:21.427+0000 7f4e17e00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4e18071f90 0x7f4e1811a6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:22:21.424 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:21.427+0000 7f4e17e00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4e18071f90 0x7f4e1811a6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:55234/0 (socket says 172.21.15.18:55234) 2024-04-12T21:22:21.424 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:21.428+0000 7f4e17e00700 1 -- 172.21.15.18:0/195116382 learned_addr learned my addr 172.21.15.18:0/195116382 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:22:21.424 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:21.428+0000 7f4e1cc00700 1 --2- 172.21.15.18:0/195116382 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4e180708a0 0x7f4e1811a160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:22:21.424 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:21.428+0000 7f4e17e00700 1 -- 172.21.15.18:0/195116382 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4e180708a0 msgr2=0x7f4e1811a160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:22:21.424 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:21.428+0000 7f4e17e00700 1 --2- 172.21.15.18:0/195116382 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4e180708a0 0x7f4e1811a160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:22:21.424 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:21.428+0000 7f4e17e00700 1 -- 172.21.15.18:0/195116382 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e04005040 con 0x7f4e18071f90 2024-04-12T21:22:21.425 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:21.428+0000 7f4e17e00700 1 --2- 172.21.15.18:0/195116382 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4e18071f90 0x7f4e1811a6a0 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f4e04005010 tx=0x7f4e04009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:22:21.425 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:21.429+0000 7f4e14c00700 1 -- 172.21.15.18:0/195116382 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4e0400e040 con 0x7f4e18071f90 2024-04-12T21:22:21.425 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:21.429+0000 7f4e14c00700 1 -- 172.21.15.18:0/195116382 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f4e04009950 con 0x7f4e18071f90 2024-04-12T21:22:21.425 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:21.429+0000 7f4e1f600700 1 -- 172.21.15.18:0/195116382 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4e18114340 con 0x7f4e18071f90 2024-04-12T21:22:21.425 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:21.429+0000 7f4e14c00700 1 -- 172.21.15.18:0/195116382 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4e0401c3f0 con 0x7f4e18071f90 2024-04-12T21:22:21.426 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:21.429+0000 7f4e1f600700 1 -- 172.21.15.18:0/195116382 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4e18114880 con 0x7f4e18071f90 2024-04-12T21:22:21.426 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:21.430+0000 7f4e1f600700 1 -- 172.21.15.18:0/195116382 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4e18071aa0 con 0x7f4e18071f90 2024-04-12T21:22:21.426 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:21.431+0000 7f4e14c00700 1 -- 172.21.15.18:0/195116382 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7f4e040137b0 con 0x7f4e18071f90 2024-04-12T21:22:21.428 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:21.432+0000 7f4e14c00700 1 --2- 172.21.15.18:0/195116382 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f4e0006ec10 0x7f4e000710c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:22:21.428 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:21.432+0000 7f4e14c00700 1 -- 172.21.15.18:0/195116382 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 6757+0+0 (secure 0 0 0) 0x7f4e0408fc60 con 0x7f4e18071f90 2024-04-12T21:22:21.428 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:21.433+0000 7f4e1cc00700 1 --2- 172.21.15.18:0/195116382 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f4e0006ec10 0x7f4e000710c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:22:21.429 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:21.433+0000 7f4e1cc00700 1 --2- 172.21.15.18:0/195116382 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f4e0006ec10 0x7f4e000710c0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f4e0c004600 tx=0x7f4e0c00b040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:22:21.432 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:21.437+0000 7f4e14c00700 1 -- 172.21.15.18:0/195116382 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f4e0405c490 con 0x7f4e18071f90 2024-04-12T21:22:21.687 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:21.691+0000 7f4e1f600700 1 -- 172.21.15.18:0/195116382 --> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f4e18071aa0 con 0x7f4e0006ec10 2024-04-12T21:22:21.689 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:21.694+0000 7f4e14c00700 1 -- 172.21.15.18:0/195116382 <== mgr.34102 v2:172.21.15.18:6800/3812891919 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f4e18071aa0 con 0x7f4e0006ec10 2024-04-12T21:22:21.693 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:21.696+0000 7f4e1f600700 1 -- 172.21.15.18:0/195116382 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f4e0006ec10 msgr2=0x7f4e000710c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:22:21.693 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:21.696+0000 7f4e1f600700 1 --2- 172.21.15.18:0/195116382 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f4e0006ec10 0x7f4e000710c0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f4e0c004600 tx=0x7f4e0c00b040 comp rx=0 tx=0).stop 2024-04-12T21:22:21.693 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:21.696+0000 7f4e1f600700 1 -- 172.21.15.18:0/195116382 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4e18071f90 msgr2=0x7f4e1811a6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:22:21.693 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:21.696+0000 7f4e1f600700 1 --2- 172.21.15.18:0/195116382 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4e18071f90 0x7f4e1811a6a0 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f4e04005010 tx=0x7f4e04009e30 comp rx=0 tx=0).stop 2024-04-12T21:22:21.693 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:21.697+0000 7f4e1f600700 1 -- 172.21.15.18:0/195116382 shutdown_connections 2024-04-12T21:22:21.693 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:21.697+0000 7f4e1f600700 1 --2- 172.21.15.18:0/195116382 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f4e0006ec10 0x7f4e000710c0 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:22:21.693 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:21.697+0000 7f4e1f600700 1 --2- 172.21.15.18:0/195116382 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4e180708a0 0x7f4e1811a160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:22:21.693 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:21.697+0000 7f4e1f600700 1 --2- 172.21.15.18:0/195116382 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4e18071f90 0x7f4e1811a6a0 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:22:21.693 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:21.697+0000 7f4e1f600700 1 -- 172.21.15.18:0/195116382 >> 172.21.15.18:0/195116382 conn(0x7f4e180f8970 msgr2=0x7f4e180f99f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:22:21.693 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:21.697+0000 7f4e1f600700 1 -- 172.21.15.18:0/195116382 shutdown_connections 2024-04-12T21:22:21.693 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:21.697+0000 7f4e1f600700 1 -- 172.21.15.18:0/195116382 wait complete. 2024-04-12T21:22:21.849 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:22:21 smithi018 ceph-mon[112590]: pgmap v676: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:22:21.852 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:21.856+0000 7fa08b400700 1 -- 172.21.15.18:0/2150162966 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa084109dd0 msgr2=0x7fa084103440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:22:21.852 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:21.856+0000 7fa08b400700 1 --2- 172.21.15.18:0/2150162966 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa084109dd0 0x7fa084103440 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7fa070004940 tx=0x7fa070009e30 comp rx=0 tx=0).stop 2024-04-12T21:22:21.852 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:21.856+0000 7fa08b400700 1 -- 172.21.15.18:0/2150162966 shutdown_connections 2024-04-12T21:22:21.852 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:21.856+0000 7fa08b400700 1 --2- 172.21.15.18:0/2150162966 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa084109dd0 0x7fa084103440 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:22:21.852 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:21.856+0000 7fa08b400700 1 --2- 172.21.15.18:0/2150162966 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa084109430 0x7fa084109800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:22:21.852 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:21.856+0000 7fa08b400700 1 -- 172.21.15.18:0/2150162966 >> 172.21.15.18:0/2150162966 conn(0x7fa084072c00 msgr2=0x7fa084075010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:22:21.852 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:21.856+0000 7fa08b400700 1 -- 172.21.15.18:0/2150162966 shutdown_connections 2024-04-12T21:22:21.852 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:21.856+0000 7fa08b400700 1 -- 172.21.15.18:0/2150162966 wait complete. 2024-04-12T21:22:21.853 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:21.857+0000 7fa08b400700 1 Processor -- start 2024-04-12T21:22:21.853 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:21.857+0000 7fa08b400700 1 -- start start 2024-04-12T21:22:21.854 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:21.858+0000 7fa08b400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa084109430 0x7fa084119f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:22:21.854 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:21.858+0000 7fa08b400700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa084109dd0 0x7fa08411a4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:22:21.854 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:21.858+0000 7fa08b400700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa08411ab10 con 0x7fa084109430 2024-04-12T21:22:21.854 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:21.858+0000 7fa08b400700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0841140f0 con 0x7fa084109dd0 2024-04-12T21:22:21.854 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:21.858+0000 7fa088a00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa084109430 0x7fa084119f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:22:21.854 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:21.858+0000 7fa088a00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa084109430 0x7fa084119f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:55254/0 (socket says 172.21.15.18:55254) 2024-04-12T21:22:21.854 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:21.858+0000 7fa088a00700 1 -- 172.21.15.18:0/2496721067 learned_addr learned my addr 172.21.15.18:0/2496721067 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:22:21.854 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:21.858+0000 7fa083e00700 1 --2- 172.21.15.18:0/2496721067 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa084109dd0 0x7fa08411a4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:22:21.854 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:21.858+0000 7fa088a00700 1 -- 172.21.15.18:0/2496721067 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa084109dd0 msgr2=0x7fa08411a4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:22:21.855 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:21.858+0000 7fa088a00700 1 --2- 172.21.15.18:0/2496721067 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa084109dd0 0x7fa08411a4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:22:21.855 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:21.858+0000 7fa088a00700 1 -- 172.21.15.18:0/2496721067 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa070005040 con 0x7fa084109430 2024-04-12T21:22:21.855 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:21.859+0000 7fa088a00700 1 --2- 172.21.15.18:0/2496721067 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa084109430 0x7fa084119f80 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7fa07800c8c0 tx=0x7fa0780044c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:22:21.855 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:21.859+0000 7fa080c00700 1 -- 172.21.15.18:0/2496721067 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa07800a980 con 0x7fa084109430 2024-04-12T21:22:21.856 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:21.859+0000 7fa080c00700 1 -- 172.21.15.18:0/2496721067 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa07800ce60 con 0x7fa084109430 2024-04-12T21:22:21.856 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:21.859+0000 7fa08b400700 1 -- 172.21.15.18:0/2496721067 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa0841143d0 con 0x7fa084109430 2024-04-12T21:22:21.856 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:21.859+0000 7fa080c00700 1 -- 172.21.15.18:0/2496721067 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa0780115e0 con 0x7fa084109430 2024-04-12T21:22:21.856 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:21.859+0000 7fa08b400700 1 -- 172.21.15.18:0/2496721067 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa084114990 con 0x7fa084109430 2024-04-12T21:22:21.856 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:21.860+0000 7fa08b400700 1 -- 172.21.15.18:0/2496721067 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa084104b80 con 0x7fa084109430 2024-04-12T21:22:21.857 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:21.861+0000 7fa080c00700 1 -- 172.21.15.18:0/2496721067 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7fa078012400 con 0x7fa084109430 2024-04-12T21:22:21.858 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:21.862+0000 7fa080c00700 1 --2- 172.21.15.18:0/2496721067 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fa06c06ec00 0x7fa06c0710b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:22:21.858 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:21.862+0000 7fa080c00700 1 -- 172.21.15.18:0/2496721067 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 6757+0+0 (secure 0 0 0) 0x7fa07808eb30 con 0x7fa084109430 2024-04-12T21:22:21.858 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:21.863+0000 7fa083e00700 1 --2- 172.21.15.18:0/2496721067 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fa06c06ec00 0x7fa06c0710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:22:21.859 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:21.863+0000 7fa083e00700 1 --2- 172.21.15.18:0/2496721067 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fa06c06ec00 0x7fa06c0710b0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fa07000b3b0 tx=0x7fa070009e30 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:22:21.863 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:21.867+0000 7fa080c00700 1 -- 172.21.15.18:0/2496721067 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7fa07805b2e0 con 0x7fa084109430 2024-04-12T21:22:22.028 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:22:21 smithi186 ceph-mon[86379]: pgmap v676: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:22:22.103 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:22.107+0000 7fa08b400700 1 -- 172.21.15.18:0/2496721067 --> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fa0841155d0 con 0x7fa06c06ec00 2024-04-12T21:22:22.133 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:22.137+0000 7fa080c00700 1 -- 172.21.15.18:0/2496721067 <== mgr.34102 v2:172.21.15.18:6800/3812891919 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7fa0841155d0 con 0x7fa06c06ec00 2024-04-12T21:22:22.133 INFO:teuthology.orchestra.run.smithi018.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-12T21:22:22.133 INFO:teuthology.orchestra.run.smithi018.stdout:alertmanager.smithi018 smithi018 *:9093,9094 running (22m) 14m ago 37m 14.7M - 0.25.0 c8568f914cd2 c332090068e8 2024-04-12T21:22:22.133 INFO:teuthology.orchestra.run.smithi018.stdout:crash.smithi018 smithi018 running (19m) 14m ago 37m 6665k - 19.0.0-2879-g324baf8d 8717720dc5e5 b7f80d1f7982 2024-04-12T21:22:22.133 INFO:teuthology.orchestra.run.smithi018.stdout:crash.smithi186 smithi186 running (19m) 15m ago 36m 6660k - 19.0.0-2879-g324baf8d 8717720dc5e5 a4f55c81b623 2024-04-12T21:22:22.133 INFO:teuthology.orchestra.run.smithi018.stdout:grafana.smithi018 smithi018 *:3000 running (22m) 14m ago 37m 74.8M - 9.4.12 2bacad6d85d8 b08e3bf176dd 2024-04-12T21:22:22.133 INFO:teuthology.orchestra.run.smithi018.stdout:mds.cephfs.smithi018.acqbsc smithi018 running (15m) 14m ago 30m 11.1M - 19.0.0-2879-g324baf8d 8717720dc5e5 021f99ab24ae 2024-04-12T21:22:22.134 INFO:teuthology.orchestra.run.smithi018.stdout:mds.cephfs.smithi018.szjluo smithi018 running (14m) 14m ago 30m 10.9M - 19.0.0-2879-g324baf8d 8717720dc5e5 0003117479d4 2024-04-12T21:22:22.134 INFO:teuthology.orchestra.run.smithi018.stdout:mds.cephfs.smithi186.jdllyn smithi186 running (30m) 15m ago 30m 15.6M - 17.2.7-827-gd3b5060a 21de9b7369ad 2b2bc46bd29a 2024-04-12T21:22:22.134 INFO:teuthology.orchestra.run.smithi018.stdout:mds.cephfs.smithi186.wjnafs smithi186 running (30m) 15m ago 30m 15.4M - 17.2.7-827-gd3b5060a 21de9b7369ad 74b115d6d6fd 2024-04-12T21:22:22.134 INFO:teuthology.orchestra.run.smithi018.stdout:mgr.smithi018.xjzwga smithi018 *:8443,9283,8765 running (25m) 14m ago 39m 509M - 19.0.0-2879-g324baf8d 8717720dc5e5 56088305eab7 2024-04-12T21:22:22.134 INFO:teuthology.orchestra.run.smithi018.stdout:mgr.smithi186.kqcfxk smithi186 *:8443,9283,8765 running (24m) 15m ago 36m 452M - 19.0.0-2879-g324baf8d 8717720dc5e5 09cd766e5e0d 2024-04-12T21:22:22.134 INFO:teuthology.orchestra.run.smithi018.stdout:mon.smithi018 smithi018 running (20m) 14m ago 39m 57.0M 2048M 19.0.0-2879-g324baf8d 8717720dc5e5 df3e201dfed6 2024-04-12T21:22:22.134 INFO:teuthology.orchestra.run.smithi018.stdout:mon.smithi186 smithi186 running (20m) 15m ago 36m 37.7M 2048M 19.0.0-2879-g324baf8d 8717720dc5e5 c77498da9b2b 2024-04-12T21:22:22.134 INFO:teuthology.orchestra.run.smithi018.stdout:node-exporter.smithi018 smithi018 *:9100 running (24m) 14m ago 37m 7780k - 1.5.0 0da6a335fe13 326c3d9c6847 2024-04-12T21:22:22.134 INFO:teuthology.orchestra.run.smithi018.stdout:node-exporter.smithi186 smithi186 *:9100 running (23m) 15m ago 36m 7826k - 1.5.0 0da6a335fe13 ec2112baeecf 2024-04-12T21:22:22.134 INFO:teuthology.orchestra.run.smithi018.stdout:osd.0 smithi018 running (18m) 14m ago 35m 121M 1559M 19.0.0-2879-g324baf8d 8717720dc5e5 60b1273ca047 2024-04-12T21:22:22.134 INFO:teuthology.orchestra.run.smithi018.stdout:osd.1 smithi018 running (18m) 14m ago 34m 91.3M 1559M 19.0.0-2879-g324baf8d 8717720dc5e5 170ce1e5fb8d 2024-04-12T21:22:22.134 INFO:teuthology.orchestra.run.smithi018.stdout:osd.2 smithi018 running (17m) 14m ago 33m 90.3M 1559M 19.0.0-2879-g324baf8d 8717720dc5e5 c2cbc6aa2839 2024-04-12T21:22:22.134 INFO:teuthology.orchestra.run.smithi018.stdout:osd.3 smithi186 running (17m) 15m ago 33m 96.6M 2581M 19.0.0-2879-g324baf8d 8717720dc5e5 0138e8262e4b 2024-04-12T21:22:22.135 INFO:teuthology.orchestra.run.smithi018.stdout:osd.4 smithi186 running (16m) 15m ago 32m 87.2M 2581M 19.0.0-2879-g324baf8d 8717720dc5e5 b20cb63b25c9 2024-04-12T21:22:22.135 INFO:teuthology.orchestra.run.smithi018.stdout:osd.5 smithi186 running (15m) 15m ago 32m 11.1M 2581M 19.0.0-2879-g324baf8d 8717720dc5e5 2513bea2610e 2024-04-12T21:22:22.135 INFO:teuthology.orchestra.run.smithi018.stdout:prometheus.smithi018 smithi018 *:9095 running (23m) 14m ago 36m 60.6M - 2.43.0 a07b618ecd1d f58841127e21 2024-04-12T21:22:22.136 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:22.140+0000 7fa08b400700 1 -- 172.21.15.18:0/2496721067 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fa06c06ec00 msgr2=0x7fa06c0710b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:22:22.136 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:22.140+0000 7fa08b400700 1 --2- 172.21.15.18:0/2496721067 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fa06c06ec00 0x7fa06c0710b0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fa07000b3b0 tx=0x7fa070009e30 comp rx=0 tx=0).stop 2024-04-12T21:22:22.136 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:22.140+0000 7fa08b400700 1 -- 172.21.15.18:0/2496721067 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa084109430 msgr2=0x7fa084119f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:22:22.136 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:22.140+0000 7fa08b400700 1 --2- 172.21.15.18:0/2496721067 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa084109430 0x7fa084119f80 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7fa07800c8c0 tx=0x7fa0780044c0 comp rx=0 tx=0).stop 2024-04-12T21:22:22.136 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:22.140+0000 7fa08b400700 1 -- 172.21.15.18:0/2496721067 shutdown_connections 2024-04-12T21:22:22.136 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:22.140+0000 7fa08b400700 1 --2- 172.21.15.18:0/2496721067 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fa06c06ec00 0x7fa06c0710b0 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:22:22.136 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:22.140+0000 7fa08b400700 1 --2- 172.21.15.18:0/2496721067 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa084109430 0x7fa084119f80 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:22:22.136 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:22.140+0000 7fa08b400700 1 --2- 172.21.15.18:0/2496721067 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa084109dd0 0x7fa08411a4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:22:22.136 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:22.140+0000 7fa08b400700 1 -- 172.21.15.18:0/2496721067 >> 172.21.15.18:0/2496721067 conn(0x7fa084072c00 msgr2=0x7fa084074c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:22:22.137 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:22.140+0000 7fa08b400700 1 -- 172.21.15.18:0/2496721067 shutdown_connections 2024-04-12T21:22:22.137 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:22.140+0000 7fa08b400700 1 -- 172.21.15.18:0/2496721067 wait complete. 2024-04-12T21:22:22.287 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:22.291+0000 7f5c7ba00700 1 -- 172.21.15.18:0/1267454103 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5c740fbc80 msgr2=0x7f5c740fc050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:22:22.287 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:22.291+0000 7f5c7ba00700 1 --2- 172.21.15.18:0/1267454103 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5c740fbc80 0x7f5c740fc050 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f5c68004940 tx=0x7f5c68009e30 comp rx=0 tx=0).stop 2024-04-12T21:22:22.287 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:22.291+0000 7f5c7ba00700 1 -- 172.21.15.18:0/1267454103 shutdown_connections 2024-04-12T21:22:22.287 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:22.291+0000 7f5c7ba00700 1 --2- 172.21.15.18:0/1267454103 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5c740fc620 0x7f5c74100cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:22:22.287 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:22.291+0000 7f5c7ba00700 1 --2- 172.21.15.18:0/1267454103 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5c740fbc80 0x7f5c740fc050 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:22:22.287 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:22.291+0000 7f5c7ba00700 1 -- 172.21.15.18:0/1267454103 >> 172.21.15.18:0/1267454103 conn(0x7f5c74072bc0 msgr2=0x7f5c74072fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:22:22.287 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:22.291+0000 7f5c7ba00700 1 -- 172.21.15.18:0/1267454103 shutdown_connections 2024-04-12T21:22:22.288 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:22.291+0000 7f5c7ba00700 1 -- 172.21.15.18:0/1267454103 wait complete. 2024-04-12T21:22:22.288 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:22.292+0000 7f5c7ba00700 1 Processor -- start 2024-04-12T21:22:22.289 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:22.292+0000 7f5c7ba00700 1 -- start start 2024-04-12T21:22:22.289 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:22.293+0000 7f5c7ba00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5c740fbc80 0x7f5c7411a1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:22:22.289 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:22.293+0000 7f5c7ba00700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5c740fc620 0x7f5c7411a710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:22:22.289 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:22.293+0000 7f5c7ba00700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c7411ad60 con 0x7f5c740fbc80 2024-04-12T21:22:22.289 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:22.293+0000 7f5c7ba00700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c74114130 con 0x7f5c740fc620 2024-04-12T21:22:22.289 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:22.293+0000 7f5c79000700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5c740fbc80 0x7f5c7411a1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:22:22.289 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:22.293+0000 7f5c79000700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5c740fbc80 0x7f5c7411a1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:55276/0 (socket says 172.21.15.18:55276) 2024-04-12T21:22:22.289 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:22.293+0000 7f5c79000700 1 -- 172.21.15.18:0/186198045 learned_addr learned my addr 172.21.15.18:0/186198045 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:22:22.290 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:22.293+0000 7f5c79000700 1 -- 172.21.15.18:0/186198045 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5c740fc620 msgr2=0x7f5c7411a710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:22:22.290 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:22.293+0000 7f5c79000700 1 --2- 172.21.15.18:0/186198045 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5c740fc620 0x7f5c7411a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:22:22.290 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:22.293+0000 7f5c79000700 1 -- 172.21.15.18:0/186198045 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c68005040 con 0x7f5c740fbc80 2024-04-12T21:22:22.290 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:22.294+0000 7f5c79000700 1 --2- 172.21.15.18:0/186198045 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5c740fbc80 0x7f5c7411a1d0 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f5c6800b3b0 tx=0x7f5c68009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:22:22.290 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:22.294+0000 7f5c70c00700 1 -- 172.21.15.18:0/186198045 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5c6800e040 con 0x7f5c740fbc80 2024-04-12T21:22:22.290 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:22.294+0000 7f5c70c00700 1 -- 172.21.15.18:0/186198045 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f5c6800bbc0 con 0x7f5c740fbc80 2024-04-12T21:22:22.290 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:22.294+0000 7f5c70c00700 1 -- 172.21.15.18:0/186198045 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5c6801c3f0 con 0x7f5c740fbc80 2024-04-12T21:22:22.291 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:22.294+0000 7f5c7ba00700 1 -- 172.21.15.18:0/186198045 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5c741143b0 con 0x7f5c740fbc80 2024-04-12T21:22:22.291 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:22.294+0000 7f5c7ba00700 1 -- 172.21.15.18:0/186198045 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5c741148f0 con 0x7f5c740fbc80 2024-04-12T21:22:22.291 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:22.295+0000 7f5c7ba00700 1 -- 172.21.15.18:0/186198045 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5c74065e30 con 0x7f5c740fbc80 2024-04-12T21:22:22.292 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:22.296+0000 7f5c70c00700 1 -- 172.21.15.18:0/186198045 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7f5c6800bd30 con 0x7f5c740fbc80 2024-04-12T21:22:22.293 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:22.297+0000 7f5c70c00700 1 --2- 172.21.15.18:0/186198045 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f5c5c06ec00 0x7f5c5c0710b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:22:22.293 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:22.297+0000 7f5c70c00700 1 -- 172.21.15.18:0/186198045 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 6757+0+0 (secure 0 0 0) 0x7f5c6808fcf0 con 0x7f5c740fbc80 2024-04-12T21:22:22.293 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:22.297+0000 7f5c73e00700 1 --2- 172.21.15.18:0/186198045 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f5c5c06ec00 0x7f5c5c0710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:22:22.294 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:22.298+0000 7f5c73e00700 1 --2- 172.21.15.18:0/186198045 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f5c5c06ec00 0x7f5c5c0710b0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f5c74115a70 tx=0x7f5c60009ce0 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:22:22.298 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:22.302+0000 7f5c70c00700 1 -- 172.21.15.18:0/186198045 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f5c6805c420 con 0x7f5c740fbc80 2024-04-12T21:22:22.624 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:22.627+0000 7f5c7ba00700 1 -- 172.21.15.18:0/186198045 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f5c7404bed0 con 0x7f5c740fbc80 2024-04-12T21:22:22.624 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:22.628+0000 7f5c70c00700 1 -- 172.21.15.18:0/186198045 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+837 (secure 0 0 0) 0x7f5c6805bfb0 con 0x7f5c740fbc80 2024-04-12T21:22:22.624 INFO:teuthology.orchestra.run.smithi018.stdout:{ 2024-04-12T21:22:22.625 INFO:teuthology.orchestra.run.smithi018.stdout: "mon": { 2024-04-12T21:22:22.625 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 19.0.0-2879-g324baf8d (324baf8ddb91dea46cac50cb535c306115791355) squid (dev)": 2 2024-04-12T21:22:22.625 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-12T21:22:22.625 INFO:teuthology.orchestra.run.smithi018.stdout: "mgr": { 2024-04-12T21:22:22.625 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 19.0.0-2879-g324baf8d (324baf8ddb91dea46cac50cb535c306115791355) squid (dev)": 2 2024-04-12T21:22:22.625 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-12T21:22:22.625 INFO:teuthology.orchestra.run.smithi018.stdout: "osd": { 2024-04-12T21:22:22.625 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 19.0.0-2879-g324baf8d (324baf8ddb91dea46cac50cb535c306115791355) squid (dev)": 6 2024-04-12T21:22:22.625 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-12T21:22:22.625 INFO:teuthology.orchestra.run.smithi018.stdout: "mds": { 2024-04-12T21:22:22.625 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 17.2.7-827-gd3b5060a (d3b5060a17449edfae9385cdf792afec109fca5d) quincy (stable)": 2, 2024-04-12T21:22:22.625 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 19.0.0-2879-g324baf8d (324baf8ddb91dea46cac50cb535c306115791355) squid (dev)": 2 2024-04-12T21:22:22.626 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-12T21:22:22.626 INFO:teuthology.orchestra.run.smithi018.stdout: "overall": { 2024-04-12T21:22:22.626 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 17.2.7-827-gd3b5060a (d3b5060a17449edfae9385cdf792afec109fca5d) quincy (stable)": 2, 2024-04-12T21:22:22.626 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 19.0.0-2879-g324baf8d (324baf8ddb91dea46cac50cb535c306115791355) squid (dev)": 12 2024-04-12T21:22:22.626 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-04-12T21:22:22.626 INFO:teuthology.orchestra.run.smithi018.stdout:} 2024-04-12T21:22:22.627 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:22.631+0000 7f5c7ba00700 1 -- 172.21.15.18:0/186198045 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f5c5c06ec00 msgr2=0x7f5c5c0710b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:22:22.627 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:22.631+0000 7f5c7ba00700 1 --2- 172.21.15.18:0/186198045 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f5c5c06ec00 0x7f5c5c0710b0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f5c74115a70 tx=0x7f5c60009ce0 comp rx=0 tx=0).stop 2024-04-12T21:22:22.627 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:22.631+0000 7f5c7ba00700 1 -- 172.21.15.18:0/186198045 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5c740fbc80 msgr2=0x7f5c7411a1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:22:22.627 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:22.631+0000 7f5c7ba00700 1 --2- 172.21.15.18:0/186198045 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5c740fbc80 0x7f5c7411a1d0 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f5c6800b3b0 tx=0x7f5c68009e30 comp rx=0 tx=0).stop 2024-04-12T21:22:22.628 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:22.631+0000 7f5c7ba00700 1 -- 172.21.15.18:0/186198045 shutdown_connections 2024-04-12T21:22:22.628 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:22.631+0000 7f5c7ba00700 1 --2- 172.21.15.18:0/186198045 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f5c5c06ec00 0x7f5c5c0710b0 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:22:22.628 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:22.631+0000 7f5c7ba00700 1 --2- 172.21.15.18:0/186198045 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5c740fbc80 0x7f5c7411a1d0 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:22:22.628 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:22.631+0000 7f5c7ba00700 1 --2- 172.21.15.18:0/186198045 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5c740fc620 0x7f5c7411a710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:22:22.628 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:22.631+0000 7f5c7ba00700 1 -- 172.21.15.18:0/186198045 >> 172.21.15.18:0/186198045 conn(0x7f5c74072bc0 msgr2=0x7f5c7410aab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:22:22.628 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:22.631+0000 7f5c7ba00700 1 -- 172.21.15.18:0/186198045 shutdown_connections 2024-04-12T21:22:22.628 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:22.632+0000 7f5c7ba00700 1 -- 172.21.15.18:0/186198045 wait complete. 2024-04-12T21:22:22.784 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:22.788+0000 7fabb7000700 1 -- 172.21.15.18:0/2364270980 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fabb0071f90 msgr2=0x7fabb0070360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:22:22.784 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:22.788+0000 7fabb7000700 1 --2- 172.21.15.18:0/2364270980 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fabb0071f90 0x7fabb0070360 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7faba0004970 tx=0x7faba0009e30 comp rx=0 tx=0).stop 2024-04-12T21:22:22.785 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:22.788+0000 7fabb7000700 1 -- 172.21.15.18:0/2364270980 shutdown_connections 2024-04-12T21:22:22.785 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:22.788+0000 7fabb7000700 1 --2- 172.21.15.18:0/2364270980 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fabb00708a0 0x7fabb0070d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:22:22.785 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:22.788+0000 7fabb7000700 1 --2- 172.21.15.18:0/2364270980 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fabb0071f90 0x7fabb0070360 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:22:22.785 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:22.788+0000 7fabb7000700 1 -- 172.21.15.18:0/2364270980 >> 172.21.15.18:0/2364270980 conn(0x7fabb00f89b0 msgr2=0x7fabb00fadc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:22:22.785 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:22.788+0000 7fabb7000700 1 -- 172.21.15.18:0/2364270980 shutdown_connections 2024-04-12T21:22:22.785 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:22.788+0000 7fabb7000700 1 -- 172.21.15.18:0/2364270980 wait complete. 2024-04-12T21:22:22.786 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:22.789+0000 7fabb7000700 1 Processor -- start 2024-04-12T21:22:22.786 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:22.790+0000 7fabb7000700 1 -- start start 2024-04-12T21:22:22.787 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:22.790+0000 7fabb7000700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fabb00708a0 0x7fabb011a1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:22:22.787 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:22.790+0000 7fabb7000700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fabb0071f90 0x7fabb011a6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:22:22.787 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:22.790+0000 7fabb7000700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabb011ad30 con 0x7fabb0071f90 2024-04-12T21:22:22.787 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:22.790+0000 7fabb7000700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabb0114100 con 0x7fabb00708a0 2024-04-12T21:22:22.787 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:22.791+0000 7fabaf400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fabb0071f90 0x7fabb011a6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:22:22.787 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:22.791+0000 7fabaf400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fabb0071f90 0x7fabb011a6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:55294/0 (socket says 172.21.15.18:55294) 2024-04-12T21:22:22.787 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:22.791+0000 7fabaf400700 1 -- 172.21.15.18:0/189099501 learned_addr learned my addr 172.21.15.18:0/189099501 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:22:22.788 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:22.791+0000 7fabafe00700 1 --2- 172.21.15.18:0/189099501 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fabb00708a0 0x7fabb011a1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:22:22.788 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:22.791+0000 7fabaf400700 1 -- 172.21.15.18:0/189099501 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fabb00708a0 msgr2=0x7fabb011a1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:22:22.788 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:22.791+0000 7fabaf400700 1 --2- 172.21.15.18:0/189099501 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fabb00708a0 0x7fabb011a1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:22:22.788 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:22.791+0000 7fabaf400700 1 -- 172.21.15.18:0/189099501 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faba0005040 con 0x7fabb0071f90 2024-04-12T21:22:22.788 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:22.791+0000 7fabafe00700 1 --2- 172.21.15.18:0/189099501 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fabb00708a0 0x7fabb011a1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-12T21:22:22.789 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:22.792+0000 7fabaf400700 1 --2- 172.21.15.18:0/189099501 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fabb0071f90 0x7fabb011a6e0 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7fab9c002a50 tx=0x7fab9c0046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:22:22.789 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:22.792+0000 7faba7e00700 1 -- 172.21.15.18:0/189099501 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fab9c007cc0 con 0x7fabb0071f90 2024-04-12T21:22:22.789 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:22.792+0000 7faba7e00700 1 -- 172.21.15.18:0/189099501 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fab9c00f040 con 0x7fabb0071f90 2024-04-12T21:22:22.789 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:22.792+0000 7faba7e00700 1 -- 172.21.15.18:0/189099501 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fab9c009640 con 0x7fabb0071f90 2024-04-12T21:22:22.789 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:22.792+0000 7fabb7000700 1 -- 172.21.15.18:0/189099501 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fabb01143e0 con 0x7fabb0071f90 2024-04-12T21:22:22.789 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:22.792+0000 7fabb7000700 1 -- 172.21.15.18:0/189099501 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fabb0114920 con 0x7fabb0071f90 2024-04-12T21:22:22.790 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:22.793+0000 7fabb7000700 1 -- 172.21.15.18:0/189099501 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fabb0071aa0 con 0x7fabb0071f90 2024-04-12T21:22:22.790 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:22.795+0000 7faba7e00700 1 -- 172.21.15.18:0/189099501 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7fab9c0097a0 con 0x7fabb0071f90 2024-04-12T21:22:22.792 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:22.796+0000 7faba7e00700 1 --2- 172.21.15.18:0/189099501 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fab9406ec10 0x7fab940710c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:22:22.792 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:22.796+0000 7faba7e00700 1 -- 172.21.15.18:0/189099501 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 6757+0+0 (secure 0 0 0) 0x7fab9c08e760 con 0x7fabb0071f90 2024-04-12T21:22:22.792 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:22.797+0000 7fabafe00700 1 --2- 172.21.15.18:0/189099501 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fab9406ec10 0x7fab940710c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:22:22.793 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:22.797+0000 7fabafe00700 1 --2- 172.21.15.18:0/189099501 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fab9406ec10 0x7fab940710c0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7faba0009fd0 tx=0x7faba0009e30 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:22:22.798 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:22.801+0000 7faba7e00700 1 -- 172.21.15.18:0/189099501 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7fab9c05af10 con 0x7fabb0071f90 2024-04-12T21:22:22.849 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:22:22 smithi018 ceph-mon[112590]: from='client.35200 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:22:22.849 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:22:22 smithi018 ceph-mon[112590]: from='client.35204 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:22:23.028 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:22:22 smithi186 ceph-mon[86379]: from='client.35200 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:22:23.028 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:22:22 smithi186 ceph-mon[86379]: from='client.35204 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:22:23.080 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:23.084+0000 7fabb7000700 1 -- 172.21.15.18:0/189099501 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fabb004bed0 con 0x7fabb0071f90 2024-04-12T21:22:23.081 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:23.085+0000 7faba7e00700 1 -- 172.21.15.18:0/189099501 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 29 v29) v1 ==== 76+0+1872 (secure 0 0 0) 0x7fab9c05aaa0 con 0x7fabb0071f90 2024-04-12T21:22:23.082 INFO:teuthology.orchestra.run.smithi018.stdout:e29 2024-04-12T21:22:23.082 INFO:teuthology.orchestra.run.smithi018.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-12T21:22:23.082 INFO:teuthology.orchestra.run.smithi018.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-12T21:22:23.082 INFO:teuthology.orchestra.run.smithi018.stdout:legacy client fscid: 1 2024-04-12T21:22:23.082 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-12T21:22:23.082 INFO:teuthology.orchestra.run.smithi018.stdout:Filesystem 'cephfs' (1) 2024-04-12T21:22:23.082 INFO:teuthology.orchestra.run.smithi018.stdout:fs_name cephfs 2024-04-12T21:22:23.083 INFO:teuthology.orchestra.run.smithi018.stdout:epoch 24 2024-04-12T21:22:23.083 INFO:teuthology.orchestra.run.smithi018.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-04-12T21:22:23.083 INFO:teuthology.orchestra.run.smithi018.stdout:created 2024-04-12T20:51:57.362850+0000 2024-04-12T21:22:23.083 INFO:teuthology.orchestra.run.smithi018.stdout:modified 2024-04-12T21:06:59.717190+0000 2024-04-12T21:22:23.083 INFO:teuthology.orchestra.run.smithi018.stdout:tableserver 0 2024-04-12T21:22:23.083 INFO:teuthology.orchestra.run.smithi018.stdout:root 0 2024-04-12T21:22:23.083 INFO:teuthology.orchestra.run.smithi018.stdout:session_timeout 60 2024-04-12T21:22:23.084 INFO:teuthology.orchestra.run.smithi018.stdout:session_autoclose 300 2024-04-12T21:22:23.084 INFO:teuthology.orchestra.run.smithi018.stdout:max_file_size 1099511627776 2024-04-12T21:22:23.085 INFO:teuthology.orchestra.run.smithi018.stdout:max_xattr_size 65536 2024-04-12T21:22:23.085 INFO:teuthology.orchestra.run.smithi018.stdout:required_client_features {} 2024-04-12T21:22:23.085 INFO:teuthology.orchestra.run.smithi018.stdout:last_failure 0 2024-04-12T21:22:23.085 INFO:teuthology.orchestra.run.smithi018.stdout:last_failure_osd_epoch 100 2024-04-12T21:22:23.085 INFO:teuthology.orchestra.run.smithi018.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-12T21:22:23.085 INFO:teuthology.orchestra.run.smithi018.stdout:max_mds 2 2024-04-12T21:22:23.085 INFO:teuthology.orchestra.run.smithi018.stdout:in 0,1 2024-04-12T21:22:23.085 INFO:teuthology.orchestra.run.smithi018.stdout:up {} 2024-04-12T21:22:23.085 INFO:teuthology.orchestra.run.smithi018.stdout:failed 0,1 2024-04-12T21:22:23.085 INFO:teuthology.orchestra.run.smithi018.stdout:damaged 2024-04-12T21:22:23.085 INFO:teuthology.orchestra.run.smithi018.stdout:stopped 2024-04-12T21:22:23.085 INFO:teuthology.orchestra.run.smithi018.stdout:data_pools [3] 2024-04-12T21:22:23.085 INFO:teuthology.orchestra.run.smithi018.stdout:metadata_pool 2 2024-04-12T21:22:23.085 INFO:teuthology.orchestra.run.smithi018.stdout:inline_data disabled 2024-04-12T21:22:23.086 INFO:teuthology.orchestra.run.smithi018.stdout:balancer 2024-04-12T21:22:23.086 INFO:teuthology.orchestra.run.smithi018.stdout:bal_rank_mask -1 2024-04-12T21:22:23.086 INFO:teuthology.orchestra.run.smithi018.stdout:standby_count_wanted 1 2024-04-12T21:22:23.086 INFO:teuthology.orchestra.run.smithi018.stdout:qdb_cluster leader: 0 members: 2024-04-12T21:22:23.086 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-12T21:22:23.086 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-12T21:22:23.086 INFO:teuthology.orchestra.run.smithi018.stdout:Standby daemons: 2024-04-12T21:22:23.086 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-12T21:22:23.086 INFO:teuthology.orchestra.run.smithi018.stdout:[mds.cephfs.smithi186.wjnafs{-1:24277} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.186:6826/438347207,v1:172.21.15.186:6827/438347207] compat {c=[1],r=[1],i=[7ff]}] 2024-04-12T21:22:23.086 INFO:teuthology.orchestra.run.smithi018.stdout:[mds.cephfs.smithi186.jdllyn{-1:24281} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.186:6824/1464359952,v1:172.21.15.186:6825/1464359952] compat {c=[1],r=[1],i=[7ff]}] 2024-04-12T21:22:23.086 INFO:teuthology.orchestra.run.smithi018.stdout:[mds.cephfs.smithi018.acqbsc{-1:34436} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.18:6828/755179764,v1:172.21.15.18:6829/755179764] compat {c=[1],r=[1],i=[1fff]}] 2024-04-12T21:22:23.086 INFO:teuthology.orchestra.run.smithi018.stdout:[mds.cephfs.smithi018.szjluo{-1:34468} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.18:6826/2407992653,v1:172.21.15.18:6827/2407992653] compat {c=[1],r=[1],i=[1fff]}] 2024-04-12T21:22:23.087 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:23.088+0000 7fabb7000700 1 -- 172.21.15.18:0/189099501 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fab9406ec10 msgr2=0x7fab940710c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:22:23.087 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:23.088+0000 7fabb7000700 1 --2- 172.21.15.18:0/189099501 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fab9406ec10 0x7fab940710c0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7faba0009fd0 tx=0x7faba0009e30 comp rx=0 tx=0).stop 2024-04-12T21:22:23.087 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:23.088+0000 7fabb7000700 1 -- 172.21.15.18:0/189099501 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fabb0071f90 msgr2=0x7fabb011a6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:22:23.087 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:23.088+0000 7fabb7000700 1 --2- 172.21.15.18:0/189099501 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fabb0071f90 0x7fabb011a6e0 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7fab9c002a50 tx=0x7fab9c0046b0 comp rx=0 tx=0).stop 2024-04-12T21:22:23.088 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:23.088+0000 7fabb7000700 1 -- 172.21.15.18:0/189099501 shutdown_connections 2024-04-12T21:22:23.088 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:23.088+0000 7fabb7000700 1 --2- 172.21.15.18:0/189099501 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fab9406ec10 0x7fab940710c0 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:22:23.088 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:23.088+0000 7fabb7000700 1 --2- 172.21.15.18:0/189099501 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fabb00708a0 0x7fabb011a1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:22:23.088 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:23.088+0000 7fabb7000700 1 --2- 172.21.15.18:0/189099501 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fabb0071f90 0x7fabb011a6e0 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:22:23.088 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:23.088+0000 7fabb7000700 1 -- 172.21.15.18:0/189099501 >> 172.21.15.18:0/189099501 conn(0x7fabb00f89b0 msgr2=0x7fabb00f9aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:22:23.088 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:23.088+0000 7fabb7000700 1 -- 172.21.15.18:0/189099501 shutdown_connections 2024-04-12T21:22:23.088 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:23.088+0000 7fabb7000700 1 -- 172.21.15.18:0/189099501 wait complete. 2024-04-12T21:22:23.088 INFO:teuthology.orchestra.run.smithi018.stderr:dumped fsmap epoch 29 2024-04-12T21:22:23.249 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:23.252+0000 7f9be2a00700 1 -- 172.21.15.18:0/1077376358 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9bdc1037a0 msgr2=0x7f9bdc0fce10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:22:23.249 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:23.252+0000 7f9be2a00700 1 --2- 172.21.15.18:0/1077376358 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9bdc1037a0 0x7f9bdc0fce10 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f9bc8004940 tx=0x7f9bc8009e30 comp rx=0 tx=0).stop 2024-04-12T21:22:23.249 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:23.252+0000 7f9be2a00700 1 -- 172.21.15.18:0/1077376358 shutdown_connections 2024-04-12T21:22:23.249 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:23.252+0000 7f9be2a00700 1 --2- 172.21.15.18:0/1077376358 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9bdc1037a0 0x7f9bdc0fce10 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:22:23.249 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:23.252+0000 7f9be2a00700 1 --2- 172.21.15.18:0/1077376358 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9bdc102e00 0x7f9bdc1031d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:22:23.249 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:23.252+0000 7f9be2a00700 1 -- 172.21.15.18:0/1077376358 >> 172.21.15.18:0/1077376358 conn(0x7f9bdc0f89b0 msgr2=0x7f9bdc0fadc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:22:23.249 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:23.252+0000 7f9be2a00700 1 -- 172.21.15.18:0/1077376358 shutdown_connections 2024-04-12T21:22:23.249 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:23.253+0000 7f9be2a00700 1 -- 172.21.15.18:0/1077376358 wait complete. 2024-04-12T21:22:23.252 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:23.253+0000 7f9be2a00700 1 Processor -- start 2024-04-12T21:22:23.252 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:23.254+0000 7f9be2a00700 1 -- start start 2024-04-12T21:22:23.252 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:23.254+0000 7f9be2a00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9bdc102e00 0x7f9bdc115da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:22:23.252 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:23.254+0000 7f9be2a00700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9bdc1037a0 0x7f9bdc1162e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:22:23.252 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:23.254+0000 7f9be2a00700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9bdc116930 con 0x7f9bdc102e00 2024-04-12T21:22:23.252 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:23.254+0000 7f9be2a00700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9bdc10fd00 con 0x7f9bdc1037a0 2024-04-12T21:22:23.252 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:23.254+0000 7f9bdbe00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9bdc102e00 0x7f9bdc115da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:22:23.252 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:23.254+0000 7f9bdbe00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9bdc102e00 0x7f9bdc115da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:55314/0 (socket says 172.21.15.18:55314) 2024-04-12T21:22:23.252 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:23.254+0000 7f9bdbe00700 1 -- 172.21.15.18:0/2864474450 learned_addr learned my addr 172.21.15.18:0/2864474450 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:22:23.253 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:23.255+0000 7f9bdb400700 1 --2- 172.21.15.18:0/2864474450 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9bdc1037a0 0x7f9bdc1162e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:22:23.253 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:23.255+0000 7f9bdbe00700 1 -- 172.21.15.18:0/2864474450 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9bdc1037a0 msgr2=0x7f9bdc1162e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:22:23.253 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:23.255+0000 7f9bdbe00700 1 --2- 172.21.15.18:0/2864474450 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9bdc1037a0 0x7f9bdc1162e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:22:23.253 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:23.255+0000 7f9bdbe00700 1 -- 172.21.15.18:0/2864474450 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9bc8005040 con 0x7f9bdc102e00 2024-04-12T21:22:23.253 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:23.255+0000 7f9bdbe00700 1 --2- 172.21.15.18:0/2864474450 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9bdc102e00 0x7f9bdc115da0 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f9bcc002a50 tx=0x7f9bcc0046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:22:23.253 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:23.255+0000 7f9bd3e00700 1 -- 172.21.15.18:0/2864474450 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9bcc007cc0 con 0x7f9bdc102e00 2024-04-12T21:22:23.253 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:23.256+0000 7f9bd3e00700 1 -- 172.21.15.18:0/2864474450 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f9bcc00f040 con 0x7f9bdc102e00 2024-04-12T21:22:23.253 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:23.256+0000 7f9be2a00700 1 -- 172.21.15.18:0/2864474450 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9bdc10ffe0 con 0x7f9bdc102e00 2024-04-12T21:22:23.253 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:23.256+0000 7f9bd3e00700 1 -- 172.21.15.18:0/2864474450 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9bcc00db70 con 0x7f9bdc102e00 2024-04-12T21:22:23.253 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:23.256+0000 7f9be2a00700 1 -- 172.21.15.18:0/2864474450 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9bdc1105a0 con 0x7f9bdc102e00 2024-04-12T21:22:23.254 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:23.257+0000 7f9be2a00700 1 -- 172.21.15.18:0/2864474450 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9bdc102120 con 0x7f9bdc102e00 2024-04-12T21:22:23.254 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:23.258+0000 7f9bd3e00700 1 -- 172.21.15.18:0/2864474450 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7f9bcc00dd10 con 0x7f9bdc102e00 2024-04-12T21:22:23.254 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:23.259+0000 7f9bd3e00700 1 --2- 172.21.15.18:0/2864474450 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f9bc006ec10 0x7f9bc00710c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:22:23.255 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:23.259+0000 7f9bd3e00700 1 -- 172.21.15.18:0/2864474450 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 6757+0+0 (secure 0 0 0) 0x7f9bcc08dff0 con 0x7f9bdc102e00 2024-04-12T21:22:23.255 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:23.259+0000 7f9bdb400700 1 --2- 172.21.15.18:0/2864474450 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f9bc006ec10 0x7f9bc00710c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:22:23.255 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:23.260+0000 7f9bdb400700 1 --2- 172.21.15.18:0/2864474450 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f9bc006ec10 0x7f9bc00710c0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f9bc800b020 tx=0x7f9bc8009e30 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:22:23.260 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:23.264+0000 7f9bd3e00700 1 -- 172.21.15.18:0/2864474450 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f9bcc05a800 con 0x7f9bdc102e00 2024-04-12T21:22:23.513 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:23.517+0000 7f9be2a00700 1 -- 172.21.15.18:0/2864474450 --> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f9bdc109f70 con 0x7f9bc006ec10 2024-04-12T21:22:23.516 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:23.520+0000 7f9bd3e00700 1 -- 172.21.15.18:0/2864474450 <== mgr.34102 v2:172.21.15.18:6800/3812891919 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f9bdc109f70 con 0x7f9bc006ec10 2024-04-12T21:22:23.516 INFO:teuthology.orchestra.run.smithi018.stdout:{ 2024-04-12T21:22:23.516 INFO:teuthology.orchestra.run.smithi018.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:f4452def3be4d350f4a8e5feaf7ebb1511be9298fe68478ba7a2d6fbd7d69a63", 2024-04-12T21:22:23.516 INFO:teuthology.orchestra.run.smithi018.stdout: "in_progress": true, 2024-04-12T21:22:23.516 INFO:teuthology.orchestra.run.smithi018.stdout: "which": "Upgrading all daemon types on all hosts", 2024-04-12T21:22:23.516 INFO:teuthology.orchestra.run.smithi018.stdout: "services_complete": [ 2024-04-12T21:22:23.516 INFO:teuthology.orchestra.run.smithi018.stdout: "crash", 2024-04-12T21:22:23.517 INFO:teuthology.orchestra.run.smithi018.stdout: "osd", 2024-04-12T21:22:23.517 INFO:teuthology.orchestra.run.smithi018.stdout: "mon", 2024-04-12T21:22:23.517 INFO:teuthology.orchestra.run.smithi018.stdout: "mgr" 2024-04-12T21:22:23.517 INFO:teuthology.orchestra.run.smithi018.stdout: ], 2024-04-12T21:22:23.517 INFO:teuthology.orchestra.run.smithi018.stdout: "progress": "14/21 daemons upgraded", 2024-04-12T21:22:23.517 INFO:teuthology.orchestra.run.smithi018.stdout: "message": "Currently upgrading mds daemons", 2024-04-12T21:22:23.517 INFO:teuthology.orchestra.run.smithi018.stdout: "is_paused": false 2024-04-12T21:22:23.517 INFO:teuthology.orchestra.run.smithi018.stdout:} 2024-04-12T21:22:23.519 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:23.523+0000 7f9be2a00700 1 -- 172.21.15.18:0/2864474450 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f9bc006ec10 msgr2=0x7f9bc00710c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:22:23.519 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:23.523+0000 7f9be2a00700 1 --2- 172.21.15.18:0/2864474450 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f9bc006ec10 0x7f9bc00710c0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f9bc800b020 tx=0x7f9bc8009e30 comp rx=0 tx=0).stop 2024-04-12T21:22:23.519 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:23.523+0000 7f9be2a00700 1 -- 172.21.15.18:0/2864474450 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9bdc102e00 msgr2=0x7f9bdc115da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:22:23.519 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:23.523+0000 7f9be2a00700 1 --2- 172.21.15.18:0/2864474450 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9bdc102e00 0x7f9bdc115da0 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f9bcc002a50 tx=0x7f9bcc0046b0 comp rx=0 tx=0).stop 2024-04-12T21:22:23.520 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:23.523+0000 7f9be2a00700 1 -- 172.21.15.18:0/2864474450 shutdown_connections 2024-04-12T21:22:23.520 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:23.523+0000 7f9be2a00700 1 --2- 172.21.15.18:0/2864474450 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f9bc006ec10 0x7f9bc00710c0 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:22:23.520 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:23.523+0000 7f9be2a00700 1 --2- 172.21.15.18:0/2864474450 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9bdc102e00 0x7f9bdc115da0 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:22:23.520 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:23.523+0000 7f9be2a00700 1 --2- 172.21.15.18:0/2864474450 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9bdc1037a0 0x7f9bdc1162e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:22:23.520 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:23.523+0000 7f9be2a00700 1 -- 172.21.15.18:0/2864474450 >> 172.21.15.18:0/2864474450 conn(0x7f9bdc0f89b0 msgr2=0x7f9bdc070b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:22:23.520 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:23.523+0000 7f9be2a00700 1 -- 172.21.15.18:0/2864474450 shutdown_connections 2024-04-12T21:22:23.520 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:23.523+0000 7f9be2a00700 1 -- 172.21.15.18:0/2864474450 wait complete. 2024-04-12T21:22:23.684 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:23.687+0000 7f7e49400700 1 -- 172.21.15.18:0/2869178887 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7e440702e0 msgr2=0x7f7e440706b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:22:23.684 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:23.687+0000 7f7e49400700 1 --2- 172.21.15.18:0/2869178887 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7e440702e0 0x7f7e440706b0 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f7e34004940 tx=0x7f7e34009e30 comp rx=0 tx=0).stop 2024-04-12T21:22:23.684 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:23.687+0000 7f7e49400700 1 -- 172.21.15.18:0/2869178887 shutdown_connections 2024-04-12T21:22:23.684 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:23.688+0000 7f7e49400700 1 --2- 172.21.15.18:0/2869178887 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7e44070bf0 0x7f7e44108fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:22:23.684 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:23.688+0000 7f7e49400700 1 --2- 172.21.15.18:0/2869178887 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7e440702e0 0x7f7e440706b0 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:22:23.684 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:23.688+0000 7f7e49400700 1 -- 172.21.15.18:0/2869178887 >> 172.21.15.18:0/2869178887 conn(0x7f7e440f8760 msgr2=0x7f7e440fab70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:22:23.684 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:23.688+0000 7f7e49400700 1 -- 172.21.15.18:0/2869178887 shutdown_connections 2024-04-12T21:22:23.684 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:23.688+0000 7f7e49400700 1 -- 172.21.15.18:0/2869178887 wait complete. 2024-04-12T21:22:23.685 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:23.689+0000 7f7e49400700 1 Processor -- start 2024-04-12T21:22:23.685 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:23.689+0000 7f7e49400700 1 -- start start 2024-04-12T21:22:23.685 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:23.690+0000 7f7e49400700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7e440702e0 0x7f7e44115b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:22:23.685 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:23.690+0000 7f7e49400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7e44070bf0 0x7f7e441160c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:22:23.685 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:23.690+0000 7f7e49400700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e44116710 con 0x7f7e44070bf0 2024-04-12T21:22:23.686 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:23.690+0000 7f7e49400700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e4410fb20 con 0x7f7e440702e0 2024-04-12T21:22:23.686 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:23.690+0000 7f7e3be00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7e44070bf0 0x7f7e441160c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:22:23.686 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:23.690+0000 7f7e3be00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7e44070bf0 0x7f7e441160c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:55328/0 (socket says 172.21.15.18:55328) 2024-04-12T21:22:23.686 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:23.690+0000 7f7e3be00700 1 -- 172.21.15.18:0/2271145589 learned_addr learned my addr 172.21.15.18:0/2271145589 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:22:23.687 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:23.690+0000 7f7e43e00700 1 --2- 172.21.15.18:0/2271145589 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7e440702e0 0x7f7e44115b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:22:23.687 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:23.691+0000 7f7e3be00700 1 -- 172.21.15.18:0/2271145589 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7e440702e0 msgr2=0x7f7e44115b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:22:23.687 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:23.691+0000 7f7e3be00700 1 --2- 172.21.15.18:0/2271145589 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7e440702e0 0x7f7e44115b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:22:23.687 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:23.691+0000 7f7e3be00700 1 -- 172.21.15.18:0/2271145589 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e34005040 con 0x7f7e44070bf0 2024-04-12T21:22:23.688 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:23.691+0000 7f7e3be00700 1 --2- 172.21.15.18:0/2271145589 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7e44070bf0 0x7f7e441160c0 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f7e2c002a50 tx=0x7f7e2c0046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:22:23.688 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:23.691+0000 7f7e40c00700 1 -- 172.21.15.18:0/2271145589 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7e2c007cc0 con 0x7f7e44070bf0 2024-04-12T21:22:23.688 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:23.691+0000 7f7e40c00700 1 -- 172.21.15.18:0/2271145589 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f7e2c00f040 con 0x7f7e44070bf0 2024-04-12T21:22:23.688 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:23.691+0000 7f7e49400700 1 -- 172.21.15.18:0/2271145589 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7e4410fe00 con 0x7f7e44070bf0 2024-04-12T21:22:23.688 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:23.691+0000 7f7e40c00700 1 -- 172.21.15.18:0/2271145589 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7e2c009640 con 0x7f7e44070bf0 2024-04-12T21:22:23.688 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:23.692+0000 7f7e49400700 1 -- 172.21.15.18:0/2271145589 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7e44110340 con 0x7f7e44070bf0 2024-04-12T21:22:23.688 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:23.693+0000 7f7e49400700 1 -- 172.21.15.18:0/2271145589 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7e44071a20 con 0x7f7e44070bf0 2024-04-12T21:22:23.689 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:23.694+0000 7f7e40c00700 1 -- 172.21.15.18:0/2271145589 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7f7e2c0097a0 con 0x7f7e44070bf0 2024-04-12T21:22:23.691 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:23.695+0000 7f7e40c00700 1 --2- 172.21.15.18:0/2271145589 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f7e2806ec00 0x7f7e280710b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:22:23.691 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:23.696+0000 7f7e40c00700 1 -- 172.21.15.18:0/2271145589 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 6757+0+0 (secure 0 0 0) 0x7f7e2c08e760 con 0x7f7e44070bf0 2024-04-12T21:22:23.691 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:23.696+0000 7f7e43e00700 1 --2- 172.21.15.18:0/2271145589 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f7e2806ec00 0x7f7e280710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:22:23.692 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:23.697+0000 7f7e43e00700 1 --2- 172.21.15.18:0/2271145589 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f7e2806ec00 0x7f7e280710b0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f7e3400b020 tx=0x7f7e34009e30 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:22:23.696 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:23.700+0000 7f7e40c00700 1 -- 172.21.15.18:0/2271145589 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f7e2c0573e0 con 0x7f7e44070bf0 2024-04-12T21:22:23.849 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:22:23 smithi018 ceph-mon[112590]: from='client.35208 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:22:23.849 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:22:23 smithi018 ceph-mon[112590]: pgmap v677: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:22:23.849 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:22:23 smithi018 ceph-mon[112590]: from='client.? 172.21.15.18:0/186198045' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T21:22:23.849 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:22:23 smithi018 ceph-mon[112590]: from='client.? 172.21.15.18:0/189099501' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-12T21:22:24.023 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:24.027+0000 7f7e49400700 1 -- 172.21.15.18:0/2271145589 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f7e4404bed0 con 0x7f7e44070bf0 2024-04-12T21:22:24.023 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:24.028+0000 7f7e40c00700 1 -- 172.21.15.18:0/2271145589 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+233 (secure 0 0 0) 0x7f7e2c05aaa0 con 0x7f7e44070bf0 2024-04-12T21:22:24.024 INFO:teuthology.orchestra.run.smithi018.stdout:HEALTH_ERR 1 filesystem is degraded; 1 filesystem is offline 2024-04-12T21:22:24.024 INFO:teuthology.orchestra.run.smithi018.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-04-12T21:22:24.024 INFO:teuthology.orchestra.run.smithi018.stdout: fs cephfs is degraded 2024-04-12T21:22:24.024 INFO:teuthology.orchestra.run.smithi018.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-04-12T21:22:24.024 INFO:teuthology.orchestra.run.smithi018.stdout: fs cephfs is offline because no MDS is active for it. 2024-04-12T21:22:24.027 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:24.031+0000 7f7e49400700 1 -- 172.21.15.18:0/2271145589 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f7e2806ec00 msgr2=0x7f7e280710b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:22:24.027 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:24.031+0000 7f7e49400700 1 --2- 172.21.15.18:0/2271145589 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f7e2806ec00 0x7f7e280710b0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f7e3400b020 tx=0x7f7e34009e30 comp rx=0 tx=0).stop 2024-04-12T21:22:24.027 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:24.031+0000 7f7e49400700 1 -- 172.21.15.18:0/2271145589 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7e44070bf0 msgr2=0x7f7e441160c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:22:24.027 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:24.031+0000 7f7e49400700 1 --2- 172.21.15.18:0/2271145589 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7e44070bf0 0x7f7e441160c0 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f7e2c002a50 tx=0x7f7e2c0046b0 comp rx=0 tx=0).stop 2024-04-12T21:22:24.027 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:24.031+0000 7f7e49400700 1 -- 172.21.15.18:0/2271145589 shutdown_connections 2024-04-12T21:22:24.027 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:24.031+0000 7f7e49400700 1 --2- 172.21.15.18:0/2271145589 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f7e2806ec00 0x7f7e280710b0 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:22:24.027 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:24.031+0000 7f7e49400700 1 --2- 172.21.15.18:0/2271145589 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7e440702e0 0x7f7e44115b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:22:24.027 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:24.031+0000 7f7e49400700 1 --2- 172.21.15.18:0/2271145589 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7e44070bf0 0x7f7e441160c0 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:22:24.028 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:24.031+0000 7f7e49400700 1 -- 172.21.15.18:0/2271145589 >> 172.21.15.18:0/2271145589 conn(0x7f7e440f8760 msgr2=0x7f7e441058e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:22:24.028 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:24.031+0000 7f7e49400700 1 -- 172.21.15.18:0/2271145589 shutdown_connections 2024-04-12T21:22:24.028 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:24.031+0000 7f7e49400700 1 -- 172.21.15.18:0/2271145589 wait complete. 2024-04-12T21:22:24.028 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:22:23 smithi186 ceph-mon[86379]: from='client.35208 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:22:24.028 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:22:23 smithi186 ceph-mon[86379]: pgmap v677: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:22:24.028 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:22:23 smithi186 ceph-mon[86379]: from='client.? 172.21.15.18:0/186198045' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T21:22:24.028 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:22:23 smithi186 ceph-mon[86379]: from='client.? 172.21.15.18:0/189099501' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-12T21:22:24.849 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:22:24 smithi018 ceph-mon[112590]: from='client.35220 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:22:24.850 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:22:24 smithi018 ceph-mon[112590]: from='client.? 172.21.15.18:0/2271145589' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-12T21:22:25.028 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:22:24 smithi186 ceph-mon[86379]: from='client.35220 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:22:25.028 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:22:24 smithi186 ceph-mon[86379]: from='client.? 172.21.15.18:0/2271145589' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-12T21:22:25.849 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:22:25 smithi018 ceph-mon[112590]: pgmap v678: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:22:25.849 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:22:25 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T21:22:26.028 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:22:25 smithi186 ceph-mon[86379]: pgmap v678: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:22:26.028 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:22:25 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T21:22:27.849 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:22:27 smithi018 ceph-mon[112590]: pgmap v679: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:22:28.034 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:22:27 smithi186 ceph-mon[86379]: pgmap v679: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:22:29.849 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:22:29 smithi018 ceph-mon[112590]: pgmap v680: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:22:30.028 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:22:29 smithi186 ceph-mon[86379]: pgmap v680: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:22:31.849 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:22:31 smithi018 ceph-mon[112590]: pgmap v681: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:22:32.028 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:22:31 smithi186 ceph-mon[86379]: pgmap v681: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:22:33.849 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:22:33 smithi018 ceph-mon[112590]: pgmap v682: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:22:34.028 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:22:33 smithi186 ceph-mon[86379]: pgmap v682: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:22:35.849 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:22:35 smithi018 ceph-mon[112590]: pgmap v683: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:22:36.028 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:22:35 smithi186 ceph-mon[86379]: pgmap v683: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:22:38.028 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:22:37 smithi186 ceph-mon[86379]: pgmap v684: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:22:38.099 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:22:37 smithi018 ceph-mon[112590]: pgmap v684: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:22:40.028 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:22:39 smithi186 ceph-mon[86379]: pgmap v685: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:22:40.099 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:22:39 smithi018 ceph-mon[112590]: pgmap v685: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:22:40.930 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:22:40 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T21:22:41.001 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:22:40 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T21:22:42.028 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:22:41 smithi186 ceph-mon[86379]: pgmap v686: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:22:42.099 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:22:41 smithi018 ceph-mon[112590]: pgmap v686: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:22:44.028 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:22:43 smithi186 ceph-mon[86379]: pgmap v687: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:22:44.099 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:22:43 smithi018 ceph-mon[112590]: pgmap v687: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:22:46.028 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:22:45 smithi186 ceph-mon[86379]: pgmap v688: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:22:46.028 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:22:45 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-12T21:22:46.029 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:22:45 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-12T21:22:46.029 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:22:45 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' 2024-04-12T21:22:46.099 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:22:45 smithi018 ceph-mon[112590]: pgmap v688: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:22:46.100 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:22:45 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-12T21:22:46.100 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:22:45 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-12T21:22:46.100 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:22:45 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' 2024-04-12T21:22:46.897 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:22:46 smithi186 ceph-mon[86379]: pgmap v689: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:22:46.897 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:22:46 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-12T21:22:46.897 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:22:46 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T21:22:46.897 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:22:46 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T21:22:46.897 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:22:46 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T21:22:46.897 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:22:46 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T21:22:46.897 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:22:46 smithi186 ceph-mon[86379]: Health check failed: failed to probe daemons or devices (CEPHADM_REFRESH_FAILED) 2024-04-12T21:22:47.099 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:22:46 smithi018 ceph-mon[112590]: pgmap v689: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:22:47.099 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:22:46 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-12T21:22:47.099 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:22:46 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T21:22:47.100 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:22:46 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T21:22:47.100 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:22:46 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T21:22:47.100 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:22:46 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T21:22:47.100 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:22:46 smithi018 ceph-mon[112590]: Health check failed: failed to probe daemons or devices (CEPHADM_REFRESH_FAILED) 2024-04-12T21:22:48.996 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:22:48 smithi186 ceph-mon[86379]: pgmap v690: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:22:48.996 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:22:48 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' 2024-04-12T21:22:48.997 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:22:48 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi186.jdllyn", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-12T21:22:48.997 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:22:48 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-12T21:22:49.099 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:22:48 smithi018 ceph-mon[112590]: pgmap v690: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:22:49.099 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:22:48 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' 2024-04-12T21:22:49.099 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:22:48 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi186.jdllyn", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-12T21:22:49.100 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:22:48 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-12T21:22:50.011 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:22:49 smithi186 ceph-mon[86379]: Upgrade: Updating mds.cephfs.smithi186.jdllyn 2024-04-12T21:22:50.011 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:22:49 smithi186 ceph-mon[86379]: Deploying daemon mds.cephfs.smithi186.jdllyn on smithi186 2024-04-12T21:22:50.099 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:22:49 smithi018 ceph-mon[112590]: Upgrade: Updating mds.cephfs.smithi186.jdllyn 2024-04-12T21:22:50.099 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:22:49 smithi018 ceph-mon[112590]: Deploying daemon mds.cephfs.smithi186.jdllyn on smithi186 2024-04-12T21:22:50.778 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:22:50 smithi186 ceph-mon[86379]: pgmap v691: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:22:51.099 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:22:50 smithi018 ceph-mon[112590]: pgmap v691: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:22:53.028 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:22:52 smithi186 ceph-mon[86379]: pgmap v692: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:22:53.028 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:22:52 smithi186 ceph-mon[86379]: osdmap e103: 6 total, 6 up, 6 in 2024-04-12T21:22:53.028 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:22:52 smithi186 ceph-mon[86379]: fsmap cephfs:0/2 3 up:standby, 2 failed 2024-04-12T21:22:53.100 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:22:52 smithi018 ceph-mon[112590]: pgmap v692: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:22:53.100 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:22:52 smithi018 ceph-mon[112590]: osdmap e103: 6 total, 6 up, 6 in 2024-04-12T21:22:53.100 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:22:52 smithi018 ceph-mon[112590]: fsmap cephfs:0/2 3 up:standby, 2 failed 2024-04-12T21:22:54.202 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:54.207+0000 7fad0fc00700 1 -- 172.21.15.18:0/982364586 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fad081037a0 msgr2=0x7fad080fce10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:22:54.203 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:54.207+0000 7fad0fc00700 1 --2- 172.21.15.18:0/982364586 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fad081037a0 0x7fad080fce10 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7facf4004940 tx=0x7facf4009e30 comp rx=0 tx=0).stop 2024-04-12T21:22:54.203 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:54.207+0000 7fad0fc00700 1 -- 172.21.15.18:0/982364586 shutdown_connections 2024-04-12T21:22:54.203 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:54.207+0000 7fad0fc00700 1 --2- 172.21.15.18:0/982364586 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fad081037a0 0x7fad080fce10 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:22:54.203 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:54.207+0000 7fad0fc00700 1 --2- 172.21.15.18:0/982364586 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fad08102e00 0x7fad081031d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:22:54.203 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:54.207+0000 7fad0fc00700 1 -- 172.21.15.18:0/982364586 >> 172.21.15.18:0/982364586 conn(0x7fad080f89b0 msgr2=0x7fad080fadc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:22:54.203 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:54.207+0000 7fad0fc00700 1 -- 172.21.15.18:0/982364586 shutdown_connections 2024-04-12T21:22:54.203 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:54.207+0000 7fad0fc00700 1 -- 172.21.15.18:0/982364586 wait complete. 2024-04-12T21:22:54.203 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:54.208+0000 7fad0fc00700 1 Processor -- start 2024-04-12T21:22:54.205 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:54.208+0000 7fad0fc00700 1 -- start start 2024-04-12T21:22:54.205 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:54.209+0000 7fad0fc00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fad08102e00 0x7fad080702c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:22:54.205 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:54.209+0000 7fad0fc00700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fad081037a0 0x7fad08070800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:22:54.205 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:54.209+0000 7fad0fc00700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad08072210 con 0x7fad08102e00 2024-04-12T21:22:54.205 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:54.209+0000 7fad0fc00700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad08070d40 con 0x7fad081037a0 2024-04-12T21:22:54.205 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:54.209+0000 7fad0d200700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fad08102e00 0x7fad080702c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:22:54.205 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:54.209+0000 7fad0d200700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fad08102e00 0x7fad080702c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:53274/0 (socket says 172.21.15.18:53274) 2024-04-12T21:22:54.206 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:54.209+0000 7fad0d200700 1 -- 172.21.15.18:0/2016585764 learned_addr learned my addr 172.21.15.18:0/2016585764 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:22:54.206 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:54.209+0000 7fad07e00700 1 --2- 172.21.15.18:0/2016585764 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fad081037a0 0x7fad08070800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:22:54.206 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:54.210+0000 7fad0d200700 1 -- 172.21.15.18:0/2016585764 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fad081037a0 msgr2=0x7fad08070800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:22:54.206 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:54.210+0000 7fad0d200700 1 --2- 172.21.15.18:0/2016585764 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fad081037a0 0x7fad08070800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:22:54.206 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:54.210+0000 7fad0d200700 1 -- 172.21.15.18:0/2016585764 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7facf4005040 con 0x7fad08102e00 2024-04-12T21:22:54.206 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:54.210+0000 7fad0d200700 1 --2- 172.21.15.18:0/2016585764 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fad08102e00 0x7fad080702c0 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7facfc00c8c0 tx=0x7facfc0044c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:22:54.206 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:54.210+0000 7fad04c00700 1 -- 172.21.15.18:0/2016585764 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7facfc00a980 con 0x7fad08102e00 2024-04-12T21:22:54.206 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:54.210+0000 7fad04c00700 1 -- 172.21.15.18:0/2016585764 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7facfc00ce60 con 0x7fad08102e00 2024-04-12T21:22:54.206 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:54.210+0000 7fad04c00700 1 -- 172.21.15.18:0/2016585764 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7facfc0115e0 con 0x7fad08102e00 2024-04-12T21:22:54.207 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:54.210+0000 7fad0fc00700 1 -- 172.21.15.18:0/2016585764 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fad08071020 con 0x7fad08102e00 2024-04-12T21:22:54.207 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:54.210+0000 7fad0fc00700 1 -- 172.21.15.18:0/2016585764 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fad0811a170 con 0x7fad08102e00 2024-04-12T21:22:54.207 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:54.212+0000 7fad0fc00700 1 -- 172.21.15.18:0/2016585764 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fad081052d0 con 0x7fad08102e00 2024-04-12T21:22:54.207 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:54.213+0000 7fad04c00700 1 -- 172.21.15.18:0/2016585764 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7facfc012400 con 0x7fad08102e00 2024-04-12T21:22:54.208 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:54.213+0000 7fad04c00700 1 --2- 172.21.15.18:0/2016585764 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7facf006ec00 0x7facf00710b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:22:54.208 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:54.214+0000 7fad04c00700 1 -- 172.21.15.18:0/2016585764 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 6757+0+0 (secure 0 0 0) 0x7facfc08ea60 con 0x7fad08102e00 2024-04-12T21:22:54.209 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:54.214+0000 7fad07e00700 1 --2- 172.21.15.18:0/2016585764 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7facf006ec00 0x7facf00710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:22:54.209 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:54.215+0000 7fad07e00700 1 --2- 172.21.15.18:0/2016585764 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7facf006ec00 0x7facf00710b0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7facf400b020 tx=0x7facf4009e30 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:22:54.214 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:54.219+0000 7fad04c00700 1 -- 172.21.15.18:0/2016585764 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7facfc057760 con 0x7fad08102e00 2024-04-12T21:22:54.468 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:54.473+0000 7fad0fc00700 1 -- 172.21.15.18:0/2016585764 --> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fad080717a0 con 0x7facf006ec00 2024-04-12T21:22:54.469 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:54.474+0000 7fad04c00700 1 -- 172.21.15.18:0/2016585764 <== mgr.34102 v2:172.21.15.18:6800/3812891919 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7fad080717a0 con 0x7facf006ec00 2024-04-12T21:22:54.472 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:54.477+0000 7fad0fc00700 1 -- 172.21.15.18:0/2016585764 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7facf006ec00 msgr2=0x7facf00710b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:22:54.472 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:54.477+0000 7fad0fc00700 1 --2- 172.21.15.18:0/2016585764 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7facf006ec00 0x7facf00710b0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7facf400b020 tx=0x7facf4009e30 comp rx=0 tx=0).stop 2024-04-12T21:22:54.472 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:54.477+0000 7fad0fc00700 1 -- 172.21.15.18:0/2016585764 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fad08102e00 msgr2=0x7fad080702c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:22:54.473 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:54.477+0000 7fad0fc00700 1 --2- 172.21.15.18:0/2016585764 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fad08102e00 0x7fad080702c0 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7facfc00c8c0 tx=0x7facfc0044c0 comp rx=0 tx=0).stop 2024-04-12T21:22:54.473 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:54.477+0000 7fad0fc00700 1 -- 172.21.15.18:0/2016585764 shutdown_connections 2024-04-12T21:22:54.473 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:54.477+0000 7fad0fc00700 1 --2- 172.21.15.18:0/2016585764 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7facf006ec00 0x7facf00710b0 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:22:54.473 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:54.477+0000 7fad0fc00700 1 --2- 172.21.15.18:0/2016585764 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fad08102e00 0x7fad080702c0 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:22:54.473 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:54.477+0000 7fad0fc00700 1 --2- 172.21.15.18:0/2016585764 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fad081037a0 0x7fad08070800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:22:54.473 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:54.477+0000 7fad0fc00700 1 -- 172.21.15.18:0/2016585764 >> 172.21.15.18:0/2016585764 conn(0x7fad080f89b0 msgr2=0x7fad081021c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:22:54.473 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:54.478+0000 7fad0fc00700 1 -- 172.21.15.18:0/2016585764 shutdown_connections 2024-04-12T21:22:54.473 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:54.478+0000 7fad0fc00700 1 -- 172.21.15.18:0/2016585764 wait complete. 2024-04-12T21:22:54.489 INFO:teuthology.orchestra.run.smithi018.stdout:true 2024-04-12T21:22:54.638 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:54.643+0000 7fdff5c00700 1 -- 172.21.15.18:0/3766772702 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fdff0071f90 msgr2=0x7fdff0070360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:22:54.639 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:54.643+0000 7fdff5c00700 1 --2- 172.21.15.18:0/3766772702 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fdff0071f90 0x7fdff0070360 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7fdfe0004970 tx=0x7fdfe0009e30 comp rx=0 tx=0).stop 2024-04-12T21:22:54.639 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:54.643+0000 7fdff5c00700 1 -- 172.21.15.18:0/3766772702 shutdown_connections 2024-04-12T21:22:54.639 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:54.643+0000 7fdff5c00700 1 --2- 172.21.15.18:0/3766772702 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdff00708a0 0x7fdff0070d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:22:54.639 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:54.643+0000 7fdff5c00700 1 --2- 172.21.15.18:0/3766772702 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fdff0071f90 0x7fdff0070360 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:22:54.639 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:54.643+0000 7fdff5c00700 1 -- 172.21.15.18:0/3766772702 >> 172.21.15.18:0/3766772702 conn(0x7fdff00f8830 msgr2=0x7fdff00fac40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:22:54.639 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:54.643+0000 7fdff5c00700 1 -- 172.21.15.18:0/3766772702 shutdown_connections 2024-04-12T21:22:54.639 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:54.643+0000 7fdff5c00700 1 -- 172.21.15.18:0/3766772702 wait complete. 2024-04-12T21:22:54.641 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:54.644+0000 7fdff5c00700 1 Processor -- start 2024-04-12T21:22:54.642 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:54.644+0000 7fdff5c00700 1 -- start start 2024-04-12T21:22:54.642 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:54.645+0000 7fdff5c00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fdff00708a0 0x7fdff0119ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:22:54.642 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:54.645+0000 7fdff5c00700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdff0071f90 0x7fdff011a530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:22:54.642 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:54.645+0000 7fdff5c00700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdff011ab80 con 0x7fdff00708a0 2024-04-12T21:22:54.642 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:54.645+0000 7fdff5c00700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdff0113f50 con 0x7fdff0071f90 2024-04-12T21:22:54.642 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:54.645+0000 7fdfefe00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fdff00708a0 0x7fdff0119ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:22:54.642 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:54.645+0000 7fdfefe00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fdff00708a0 0x7fdff0119ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:53286/0 (socket says 172.21.15.18:53286) 2024-04-12T21:22:54.642 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:54.645+0000 7fdfefe00700 1 -- 172.21.15.18:0/1805880473 learned_addr learned my addr 172.21.15.18:0/1805880473 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:22:54.642 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:54.645+0000 7fdfe7400700 1 --2- 172.21.15.18:0/1805880473 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdff0071f90 0x7fdff011a530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:22:54.642 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:54.645+0000 7fdfefe00700 1 -- 172.21.15.18:0/1805880473 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdff0071f90 msgr2=0x7fdff011a530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:22:54.643 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:54.645+0000 7fdfefe00700 1 --2- 172.21.15.18:0/1805880473 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdff0071f90 0x7fdff011a530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:22:54.643 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:54.645+0000 7fdfefe00700 1 -- 172.21.15.18:0/1805880473 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdfe0005040 con 0x7fdff00708a0 2024-04-12T21:22:54.643 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:54.645+0000 7fdfe7400700 1 --2- 172.21.15.18:0/1805880473 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdff0071f90 0x7fdff011a530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-12T21:22:54.643 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:54.646+0000 7fdfefe00700 1 --2- 172.21.15.18:0/1805880473 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fdff00708a0 0x7fdff0119ff0 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7fdfe0004f50 tx=0x7fdfe0009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:22:54.643 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:54.646+0000 7fdfecc00700 1 -- 172.21.15.18:0/1805880473 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdfe000e040 con 0x7fdff00708a0 2024-04-12T21:22:54.643 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:54.646+0000 7fdfecc00700 1 -- 172.21.15.18:0/1805880473 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fdfe000bc10 con 0x7fdff00708a0 2024-04-12T21:22:54.643 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:54.646+0000 7fdff5c00700 1 -- 172.21.15.18:0/1805880473 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdff01141d0 con 0x7fdff00708a0 2024-04-12T21:22:54.644 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:54.646+0000 7fdfecc00700 1 -- 172.21.15.18:0/1805880473 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdfe001c630 con 0x7fdff00708a0 2024-04-12T21:22:54.644 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:54.646+0000 7fdff5c00700 1 -- 172.21.15.18:0/1805880473 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdff0114710 con 0x7fdff00708a0 2024-04-12T21:22:54.645 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:54.647+0000 7fdff5c00700 1 -- 172.21.15.18:0/1805880473 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdff0071aa0 con 0x7fdff00708a0 2024-04-12T21:22:54.645 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:54.649+0000 7fdfecc00700 1 -- 172.21.15.18:0/1805880473 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7fdfe0013420 con 0x7fdff00708a0 2024-04-12T21:22:54.645 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:54.650+0000 7fdfecc00700 1 --2- 172.21.15.18:0/1805880473 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fdfd406ec10 0x7fdfd40710c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:22:54.645 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:54.650+0000 7fdfecc00700 1 -- 172.21.15.18:0/1805880473 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 6757+0+0 (secure 0 0 0) 0x7fdfe008fce0 con 0x7fdff00708a0 2024-04-12T21:22:54.645 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:54.650+0000 7fdfe7400700 1 --2- 172.21.15.18:0/1805880473 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fdfd406ec10 0x7fdfd40710c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:22:54.646 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:54.651+0000 7fdfe7400700 1 --2- 172.21.15.18:0/1805880473 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fdfd406ec10 0x7fdfd40710c0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fdfd8007fd0 tx=0x7fdfd8009040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:22:54.649 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:54.655+0000 7fdfecc00700 1 -- 172.21.15.18:0/1805880473 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7fdfe005c510 con 0x7fdff00708a0 2024-04-12T21:22:54.849 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:22:54 smithi018 ceph-mon[112590]: pgmap v694: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:22:54.908 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:54.913+0000 7fdff5c00700 1 -- 172.21.15.18:0/1805880473 --> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fdff004bf40 con 0x7fdfd406ec10 2024-04-12T21:22:54.914 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:54.920+0000 7fdfecc00700 1 -- 172.21.15.18:0/1805880473 <== mgr.34102 v2:172.21.15.18:6800/3812891919 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7fdff004bf40 con 0x7fdfd406ec10 2024-04-12T21:22:54.918 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:54.922+0000 7fdff5c00700 1 -- 172.21.15.18:0/1805880473 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fdfd406ec10 msgr2=0x7fdfd40710c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:22:54.918 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:54.922+0000 7fdff5c00700 1 --2- 172.21.15.18:0/1805880473 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fdfd406ec10 0x7fdfd40710c0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fdfd8007fd0 tx=0x7fdfd8009040 comp rx=0 tx=0).stop 2024-04-12T21:22:54.918 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:54.922+0000 7fdff5c00700 1 -- 172.21.15.18:0/1805880473 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fdff00708a0 msgr2=0x7fdff0119ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:22:54.918 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:54.922+0000 7fdff5c00700 1 --2- 172.21.15.18:0/1805880473 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fdff00708a0 0x7fdff0119ff0 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7fdfe0004f50 tx=0x7fdfe0009e30 comp rx=0 tx=0).stop 2024-04-12T21:22:54.918 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:54.923+0000 7fdff5c00700 1 -- 172.21.15.18:0/1805880473 shutdown_connections 2024-04-12T21:22:54.918 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:54.923+0000 7fdff5c00700 1 --2- 172.21.15.18:0/1805880473 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fdfd406ec10 0x7fdfd40710c0 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:22:54.918 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:54.923+0000 7fdff5c00700 1 --2- 172.21.15.18:0/1805880473 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fdff00708a0 0x7fdff0119ff0 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:22:54.918 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:54.923+0000 7fdff5c00700 1 --2- 172.21.15.18:0/1805880473 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdff0071f90 0x7fdff011a530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:22:54.918 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:54.923+0000 7fdff5c00700 1 -- 172.21.15.18:0/1805880473 >> 172.21.15.18:0/1805880473 conn(0x7fdff00f8830 msgr2=0x7fdff00f9930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:22:54.919 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:54.923+0000 7fdff5c00700 1 -- 172.21.15.18:0/1805880473 shutdown_connections 2024-04-12T21:22:54.919 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:54.923+0000 7fdff5c00700 1 -- 172.21.15.18:0/1805880473 wait complete. 2024-04-12T21:22:54.938 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:22:54 smithi186 ceph-mon[86379]: pgmap v694: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:22:55.078 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:55.083+0000 7f5567200700 1 -- 172.21.15.18:0/3766052452 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5568109c40 msgr2=0x7f55681032b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:22:55.078 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:55.083+0000 7f5567200700 1 --2- 172.21.15.18:0/3766052452 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5568109c40 0x7f55681032b0 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f5554004940 tx=0x7f5554009e30 comp rx=0 tx=0).stop 2024-04-12T21:22:55.079 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:55.083+0000 7f5567200700 1 -- 172.21.15.18:0/3766052452 shutdown_connections 2024-04-12T21:22:55.079 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:55.083+0000 7f5567200700 1 --2- 172.21.15.18:0/3766052452 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5568109c40 0x7f55681032b0 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:22:55.079 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:55.083+0000 7f5567200700 1 --2- 172.21.15.18:0/3766052452 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f55681092a0 0x7f5568109670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:22:55.079 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:55.083+0000 7f5567200700 1 -- 172.21.15.18:0/3766052452 >> 172.21.15.18:0/3766052452 conn(0x7f5568072a30 msgr2=0x7f5568074e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:22:55.079 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:55.083+0000 7f5567200700 1 -- 172.21.15.18:0/3766052452 shutdown_connections 2024-04-12T21:22:55.079 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:55.083+0000 7f5567200700 1 -- 172.21.15.18:0/3766052452 wait complete. 2024-04-12T21:22:55.080 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:55.084+0000 7f5567200700 1 Processor -- start 2024-04-12T21:22:55.080 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:55.084+0000 7f5567200700 1 -- start start 2024-04-12T21:22:55.080 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:55.085+0000 7f5567200700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f55681092a0 0x7f5568119ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:22:55.081 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:55.085+0000 7f5567200700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5568109c40 0x7f556811a530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:22:55.081 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:55.085+0000 7f5567200700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f556811ab80 con 0x7f55681092a0 2024-04-12T21:22:55.081 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:55.085+0000 7f5567200700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5568113f50 con 0x7f5568109c40 2024-04-12T21:22:55.081 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:55.085+0000 7f5565e00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f55681092a0 0x7f5568119ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:22:55.081 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:55.085+0000 7f5565e00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f55681092a0 0x7f5568119ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:53304/0 (socket says 172.21.15.18:53304) 2024-04-12T21:22:55.081 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:55.085+0000 7f5565e00700 1 -- 172.21.15.18:0/1871762873 learned_addr learned my addr 172.21.15.18:0/1871762873 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:22:55.081 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:55.085+0000 7f5565400700 1 --2- 172.21.15.18:0/1871762873 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5568109c40 0x7f556811a530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:22:55.081 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:55.085+0000 7f5565e00700 1 -- 172.21.15.18:0/1871762873 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5568109c40 msgr2=0x7f556811a530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:22:55.082 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:55.085+0000 7f5565e00700 1 --2- 172.21.15.18:0/1871762873 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5568109c40 0x7f556811a530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:22:55.082 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:55.085+0000 7f5565e00700 1 -- 172.21.15.18:0/1871762873 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5554005040 con 0x7f55681092a0 2024-04-12T21:22:55.082 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:55.086+0000 7f5565e00700 1 --2- 172.21.15.18:0/1871762873 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f55681092a0 0x7f5568119ff0 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f555800c830 tx=0x7f5558004470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:22:55.083 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:55.086+0000 7f555e000700 1 -- 172.21.15.18:0/1871762873 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f555800a760 con 0x7f55681092a0 2024-04-12T21:22:55.083 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:55.086+0000 7f5567200700 1 -- 172.21.15.18:0/1871762873 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5568114230 con 0x7f55681092a0 2024-04-12T21:22:55.084 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:55.086+0000 7f555e000700 1 -- 172.21.15.18:0/1871762873 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f555800ad90 con 0x7f55681092a0 2024-04-12T21:22:55.084 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:55.086+0000 7f555e000700 1 -- 172.21.15.18:0/1871762873 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f55580113a0 con 0x7f55681092a0 2024-04-12T21:22:55.084 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:55.086+0000 7f5567200700 1 -- 172.21.15.18:0/1871762873 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5568114770 con 0x7f55681092a0 2024-04-12T21:22:55.084 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:55.088+0000 7f5567200700 1 -- 172.21.15.18:0/1871762873 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f55681049f0 con 0x7f55681092a0 2024-04-12T21:22:55.084 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:55.089+0000 7f555e000700 1 -- 172.21.15.18:0/1871762873 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7f5558012400 con 0x7f55681092a0 2024-04-12T21:22:55.085 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:55.090+0000 7f555e000700 1 --2- 172.21.15.18:0/1871762873 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f554c06ec00 0x7f554c0710b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:22:55.085 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:55.090+0000 7f555e000700 1 -- 172.21.15.18:0/1871762873 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 6757+0+0 (secure 0 0 0) 0x7f555808ea00 con 0x7f55681092a0 2024-04-12T21:22:55.086 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:55.090+0000 7f5565400700 1 --2- 172.21.15.18:0/1871762873 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f554c06ec00 0x7f554c0710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:22:55.086 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:55.091+0000 7f5565400700 1 --2- 172.21.15.18:0/1871762873 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f554c06ec00 0x7f554c0710b0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f5554004f90 tx=0x7f5554009e30 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:22:55.090 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:55.095+0000 7f555e000700 1 -- 172.21.15.18:0/1871762873 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f555805b250 con 0x7f55681092a0 2024-04-12T21:22:55.326 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:55.331+0000 7f5567200700 1 -- 172.21.15.18:0/1871762873 --> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f556804bed0 con 0x7f554c06ec00 2024-04-12T21:22:55.358 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:55.363+0000 7f555e000700 1 -- 172.21.15.18:0/1871762873 <== mgr.34102 v2:172.21.15.18:6800/3812891919 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7f556804bed0 con 0x7f554c06ec00 2024-04-12T21:22:55.359 INFO:teuthology.orchestra.run.smithi018.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-12T21:22:55.359 INFO:teuthology.orchestra.run.smithi018.stdout:alertmanager.smithi018 smithi018 *:9093,9094 running (23m) 15m ago 37m 14.7M - 0.25.0 c8568f914cd2 c332090068e8 2024-04-12T21:22:55.359 INFO:teuthology.orchestra.run.smithi018.stdout:crash.smithi018 smithi018 running (19m) 15m ago 38m 6665k - 19.0.0-2879-g324baf8d 8717720dc5e5 b7f80d1f7982 2024-04-12T21:22:55.359 INFO:teuthology.orchestra.run.smithi018.stdout:crash.smithi186 smithi186 running (19m) 16m ago 36m 6660k - 19.0.0-2879-g324baf8d 8717720dc5e5 a4f55c81b623 2024-04-12T21:22:55.359 INFO:teuthology.orchestra.run.smithi018.stdout:grafana.smithi018 smithi018 *:3000 running (22m) 15m ago 37m 74.8M - 9.4.12 2bacad6d85d8 b08e3bf176dd 2024-04-12T21:22:55.359 INFO:teuthology.orchestra.run.smithi018.stdout:mds.cephfs.smithi018.acqbsc smithi018 running (15m) 15m ago 30m 11.1M - 19.0.0-2879-g324baf8d 8717720dc5e5 021f99ab24ae 2024-04-12T21:22:55.359 INFO:teuthology.orchestra.run.smithi018.stdout:mds.cephfs.smithi018.szjluo smithi018 running (15m) 15m ago 30m 10.9M - 19.0.0-2879-g324baf8d 8717720dc5e5 0003117479d4 2024-04-12T21:22:55.359 INFO:teuthology.orchestra.run.smithi018.stdout:mds.cephfs.smithi186.jdllyn smithi186 running (30m) 16m ago 30m 15.6M - 17.2.7-827-gd3b5060a 21de9b7369ad 2b2bc46bd29a 2024-04-12T21:22:55.359 INFO:teuthology.orchestra.run.smithi018.stdout:mds.cephfs.smithi186.wjnafs smithi186 running (30m) 16m ago 30m 15.4M - 17.2.7-827-gd3b5060a 21de9b7369ad 74b115d6d6fd 2024-04-12T21:22:55.359 INFO:teuthology.orchestra.run.smithi018.stdout:mgr.smithi018.xjzwga smithi018 *:8443,9283,8765 running (26m) 15m ago 39m 509M - 19.0.0-2879-g324baf8d 8717720dc5e5 56088305eab7 2024-04-12T21:22:55.359 INFO:teuthology.orchestra.run.smithi018.stdout:mgr.smithi186.kqcfxk smithi186 *:8443,9283,8765 running (24m) 16m ago 36m 452M - 19.0.0-2879-g324baf8d 8717720dc5e5 09cd766e5e0d 2024-04-12T21:22:55.359 INFO:teuthology.orchestra.run.smithi018.stdout:mon.smithi018 smithi018 running (21m) 15m ago 39m 57.0M 2048M 19.0.0-2879-g324baf8d 8717720dc5e5 df3e201dfed6 2024-04-12T21:22:55.359 INFO:teuthology.orchestra.run.smithi018.stdout:mon.smithi186 smithi186 running (20m) 16m ago 36m 37.7M 2048M 19.0.0-2879-g324baf8d 8717720dc5e5 c77498da9b2b 2024-04-12T21:22:55.360 INFO:teuthology.orchestra.run.smithi018.stdout:node-exporter.smithi018 smithi018 *:9100 running (24m) 15m ago 38m 7780k - 1.5.0 0da6a335fe13 326c3d9c6847 2024-04-12T21:22:55.360 INFO:teuthology.orchestra.run.smithi018.stdout:node-exporter.smithi186 smithi186 *:9100 running (24m) 16m ago 36m 7826k - 1.5.0 0da6a335fe13 ec2112baeecf 2024-04-12T21:22:55.360 INFO:teuthology.orchestra.run.smithi018.stdout:osd.0 smithi018 running (19m) 15m ago 35m 121M 1559M 19.0.0-2879-g324baf8d 8717720dc5e5 60b1273ca047 2024-04-12T21:22:55.360 INFO:teuthology.orchestra.run.smithi018.stdout:osd.1 smithi018 running (18m) 15m ago 35m 91.3M 1559M 19.0.0-2879-g324baf8d 8717720dc5e5 170ce1e5fb8d 2024-04-12T21:22:55.360 INFO:teuthology.orchestra.run.smithi018.stdout:osd.2 smithi018 running (18m) 15m ago 34m 90.3M 1559M 19.0.0-2879-g324baf8d 8717720dc5e5 c2cbc6aa2839 2024-04-12T21:22:55.360 INFO:teuthology.orchestra.run.smithi018.stdout:osd.3 smithi186 running (17m) 16m ago 33m 96.6M 2581M 19.0.0-2879-g324baf8d 8717720dc5e5 0138e8262e4b 2024-04-12T21:22:55.360 INFO:teuthology.orchestra.run.smithi018.stdout:osd.4 smithi186 running (16m) 16m ago 33m 87.2M 2581M 19.0.0-2879-g324baf8d 8717720dc5e5 b20cb63b25c9 2024-04-12T21:22:55.360 INFO:teuthology.orchestra.run.smithi018.stdout:osd.5 smithi186 running (16m) 16m ago 32m 11.1M 2581M 19.0.0-2879-g324baf8d 8717720dc5e5 2513bea2610e 2024-04-12T21:22:55.360 INFO:teuthology.orchestra.run.smithi018.stdout:prometheus.smithi018 smithi018 *:9095 running (23m) 15m ago 37m 60.6M - 2.43.0 a07b618ecd1d f58841127e21 2024-04-12T21:22:55.361 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:55.366+0000 7f5567200700 1 -- 172.21.15.18:0/1871762873 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f554c06ec00 msgr2=0x7f554c0710b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:22:55.361 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:55.366+0000 7f5567200700 1 --2- 172.21.15.18:0/1871762873 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f554c06ec00 0x7f554c0710b0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f5554004f90 tx=0x7f5554009e30 comp rx=0 tx=0).stop 2024-04-12T21:22:55.361 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:55.366+0000 7f5567200700 1 -- 172.21.15.18:0/1871762873 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f55681092a0 msgr2=0x7f5568119ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:22:55.361 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:55.366+0000 7f5567200700 1 --2- 172.21.15.18:0/1871762873 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f55681092a0 0x7f5568119ff0 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f555800c830 tx=0x7f5558004470 comp rx=0 tx=0).stop 2024-04-12T21:22:55.362 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:55.366+0000 7f5567200700 1 -- 172.21.15.18:0/1871762873 shutdown_connections 2024-04-12T21:22:55.362 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:55.366+0000 7f5567200700 1 --2- 172.21.15.18:0/1871762873 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f554c06ec00 0x7f554c0710b0 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:22:55.362 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:55.366+0000 7f5567200700 1 --2- 172.21.15.18:0/1871762873 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f55681092a0 0x7f5568119ff0 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:22:55.362 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:55.366+0000 7f5567200700 1 --2- 172.21.15.18:0/1871762873 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5568109c40 0x7f556811a530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:22:55.362 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:55.366+0000 7f5567200700 1 -- 172.21.15.18:0/1871762873 >> 172.21.15.18:0/1871762873 conn(0x7f5568072a30 msgr2=0x7f5568074850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:22:55.362 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:55.366+0000 7f5567200700 1 -- 172.21.15.18:0/1871762873 shutdown_connections 2024-04-12T21:22:55.362 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:55.367+0000 7f5567200700 1 -- 172.21.15.18:0/1871762873 wait complete. 2024-04-12T21:22:55.505 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:55.510+0000 7f55fc200700 1 -- 172.21.15.18:0/2742489277 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f55f4072e00 msgr2=0x7f55f410d770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:22:55.505 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:55.510+0000 7f55fc200700 1 --2- 172.21.15.18:0/2742489277 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f55f4072e00 0x7f55f410d770 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f55e0004940 tx=0x7f55e0009e30 comp rx=0 tx=0).stop 2024-04-12T21:22:55.505 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:55.510+0000 7f55fc200700 1 -- 172.21.15.18:0/2742489277 shutdown_connections 2024-04-12T21:22:55.505 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:55.510+0000 7f55fc200700 1 --2- 172.21.15.18:0/2742489277 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f55f4072e00 0x7f55f410d770 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:22:55.505 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:55.510+0000 7f55fc200700 1 --2- 172.21.15.18:0/2742489277 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f55f40724f0 0x7f55f40728c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:22:55.505 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:55.510+0000 7f55fc200700 1 -- 172.21.15.18:0/2742489277 >> 172.21.15.18:0/2742489277 conn(0x7f55f40fabd0 msgr2=0x7f55f40fcfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:22:55.505 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:55.510+0000 7f55fc200700 1 -- 172.21.15.18:0/2742489277 shutdown_connections 2024-04-12T21:22:55.505 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:55.510+0000 7f55fc200700 1 -- 172.21.15.18:0/2742489277 wait complete. 2024-04-12T21:22:55.507 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:55.511+0000 7f55fc200700 1 Processor -- start 2024-04-12T21:22:55.507 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:55.511+0000 7f55fc200700 1 -- start start 2024-04-12T21:22:55.507 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:55.511+0000 7f55fc200700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f55f40724f0 0x7f55f411a120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:22:55.507 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:55.511+0000 7f55fc200700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f55f4072e00 0x7f55f411a660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:22:55.507 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:55.511+0000 7f55fc200700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55f411acb0 con 0x7f55f40724f0 2024-04-12T21:22:55.507 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:55.511+0000 7f55fc200700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55f41140c0 con 0x7f55f4072e00 2024-04-12T21:22:55.507 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:55.511+0000 7f55f9800700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f55f40724f0 0x7f55f411a120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:22:55.507 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:55.511+0000 7f55f9800700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f55f40724f0 0x7f55f411a120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:53328/0 (socket says 172.21.15.18:53328) 2024-04-12T21:22:55.507 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:55.511+0000 7f55f9800700 1 -- 172.21.15.18:0/1019563989 learned_addr learned my addr 172.21.15.18:0/1019563989 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:22:55.507 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:55.512+0000 7f55f8e00700 1 --2- 172.21.15.18:0/1019563989 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f55f4072e00 0x7f55f411a660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:22:55.507 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:55.512+0000 7f55f9800700 1 -- 172.21.15.18:0/1019563989 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f55f4072e00 msgr2=0x7f55f411a660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:22:55.508 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:55.512+0000 7f55f9800700 1 --2- 172.21.15.18:0/1019563989 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f55f4072e00 0x7f55f411a660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:22:55.508 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:55.512+0000 7f55f9800700 1 -- 172.21.15.18:0/1019563989 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55e0005040 con 0x7f55f40724f0 2024-04-12T21:22:55.508 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:55.512+0000 7f55f9800700 1 --2- 172.21.15.18:0/1019563989 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f55f40724f0 0x7f55f411a120 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f55e800c8c0 tx=0x7f55e80044c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:22:55.508 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:55.512+0000 7f55f1600700 1 -- 172.21.15.18:0/1019563989 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f55e800a980 con 0x7f55f40724f0 2024-04-12T21:22:55.509 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:55.512+0000 7f55f1600700 1 -- 172.21.15.18:0/1019563989 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f55e800ce60 con 0x7f55f40724f0 2024-04-12T21:22:55.509 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:55.512+0000 7f55f1600700 1 -- 172.21.15.18:0/1019563989 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f55e80115e0 con 0x7f55f40724f0 2024-04-12T21:22:55.509 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:55.513+0000 7f55fc200700 1 -- 172.21.15.18:0/1019563989 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f55f41143a0 con 0x7f55f40724f0 2024-04-12T21:22:55.509 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:55.513+0000 7f55fc200700 1 -- 172.21.15.18:0/1019563989 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f55f4114910 con 0x7f55f40724f0 2024-04-12T21:22:55.509 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:55.514+0000 7f55fc200700 1 -- 172.21.15.18:0/1019563989 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f55f4073c30 con 0x7f55f40724f0 2024-04-12T21:22:55.510 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:55.515+0000 7f55f1600700 1 -- 172.21.15.18:0/1019563989 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7f55e8012400 con 0x7f55f40724f0 2024-04-12T21:22:55.511 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:55.516+0000 7f55f1600700 1 --2- 172.21.15.18:0/1019563989 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f55dc06ec10 0x7f55dc0710c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:22:55.511 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:55.516+0000 7f55f1600700 1 -- 172.21.15.18:0/1019563989 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 6757+0+0 (secure 0 0 0) 0x7f55e808eb30 con 0x7f55f40724f0 2024-04-12T21:22:55.511 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:55.516+0000 7f55f8e00700 1 --2- 172.21.15.18:0/1019563989 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f55dc06ec10 0x7f55dc0710c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:22:55.511 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:55.517+0000 7f55f8e00700 1 --2- 172.21.15.18:0/1019563989 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f55dc06ec10 0x7f55dc0710c0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f55f4115a00 tx=0x7f55e0009e30 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:22:55.516 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:55.521+0000 7f55f1600700 1 -- 172.21.15.18:0/1019563989 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f55e80577b0 con 0x7f55f40724f0 2024-04-12T21:22:55.845 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:55.850+0000 7f55fc200700 1 -- 172.21.15.18:0/1019563989 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f55f404bed0 con 0x7f55f40724f0 2024-04-12T21:22:55.846 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:55.851+0000 7f55f1600700 1 -- 172.21.15.18:0/1019563989 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+837 (secure 0 0 0) 0x7f55e80126b0 con 0x7f55f40724f0 2024-04-12T21:22:55.846 INFO:teuthology.orchestra.run.smithi018.stdout:{ 2024-04-12T21:22:55.847 INFO:teuthology.orchestra.run.smithi018.stdout: "mon": { 2024-04-12T21:22:55.847 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 19.0.0-2879-g324baf8d (324baf8ddb91dea46cac50cb535c306115791355) squid (dev)": 2 2024-04-12T21:22:55.847 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-12T21:22:55.847 INFO:teuthology.orchestra.run.smithi018.stdout: "mgr": { 2024-04-12T21:22:55.847 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 19.0.0-2879-g324baf8d (324baf8ddb91dea46cac50cb535c306115791355) squid (dev)": 2 2024-04-12T21:22:55.847 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-12T21:22:55.847 INFO:teuthology.orchestra.run.smithi018.stdout: "osd": { 2024-04-12T21:22:55.848 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 19.0.0-2879-g324baf8d (324baf8ddb91dea46cac50cb535c306115791355) squid (dev)": 6 2024-04-12T21:22:55.848 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-12T21:22:55.848 INFO:teuthology.orchestra.run.smithi018.stdout: "mds": { 2024-04-12T21:22:55.848 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 17.2.7-827-gd3b5060a (d3b5060a17449edfae9385cdf792afec109fca5d) quincy (stable)": 1, 2024-04-12T21:22:55.848 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 19.0.0-2879-g324baf8d (324baf8ddb91dea46cac50cb535c306115791355) squid (dev)": 2 2024-04-12T21:22:55.848 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-12T21:22:55.848 INFO:teuthology.orchestra.run.smithi018.stdout: "overall": { 2024-04-12T21:22:55.848 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 17.2.7-827-gd3b5060a (d3b5060a17449edfae9385cdf792afec109fca5d) quincy (stable)": 1, 2024-04-12T21:22:55.848 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 19.0.0-2879-g324baf8d (324baf8ddb91dea46cac50cb535c306115791355) squid (dev)": 12 2024-04-12T21:22:55.848 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-04-12T21:22:55.848 INFO:teuthology.orchestra.run.smithi018.stdout:} 2024-04-12T21:22:55.849 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:55.853+0000 7f55fc200700 1 -- 172.21.15.18:0/1019563989 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f55dc06ec10 msgr2=0x7f55dc0710c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:22:55.849 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:55.853+0000 7f55fc200700 1 --2- 172.21.15.18:0/1019563989 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f55dc06ec10 0x7f55dc0710c0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f55f4115a00 tx=0x7f55e0009e30 comp rx=0 tx=0).stop 2024-04-12T21:22:55.849 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:55.853+0000 7f55fc200700 1 -- 172.21.15.18:0/1019563989 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f55f40724f0 msgr2=0x7f55f411a120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:22:55.849 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:55.853+0000 7f55fc200700 1 --2- 172.21.15.18:0/1019563989 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f55f40724f0 0x7f55f411a120 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f55e800c8c0 tx=0x7f55e80044c0 comp rx=0 tx=0).stop 2024-04-12T21:22:55.850 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:55.854+0000 7f55fc200700 1 -- 172.21.15.18:0/1019563989 shutdown_connections 2024-04-12T21:22:55.850 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:55.854+0000 7f55fc200700 1 --2- 172.21.15.18:0/1019563989 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f55dc06ec10 0x7f55dc0710c0 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:22:55.850 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:55.854+0000 7f55fc200700 1 --2- 172.21.15.18:0/1019563989 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f55f40724f0 0x7f55f411a120 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:22:55.850 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:55.854+0000 7f55fc200700 1 --2- 172.21.15.18:0/1019563989 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f55f4072e00 0x7f55f411a660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:22:55.850 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:55.854+0000 7f55fc200700 1 -- 172.21.15.18:0/1019563989 >> 172.21.15.18:0/1019563989 conn(0x7f55f40fabd0 msgr2=0x7f55f40fafa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:22:55.850 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:55.854+0000 7f55fc200700 1 -- 172.21.15.18:0/1019563989 shutdown_connections 2024-04-12T21:22:55.850 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:55.854+0000 7f55fc200700 1 -- 172.21.15.18:0/1019563989 wait complete. 2024-04-12T21:22:55.996 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:56.001+0000 7f829ea00700 1 -- 172.21.15.18:0/3016622202 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8298109430 msgr2=0x7f8298109800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:22:55.996 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:56.001+0000 7f829ea00700 1 --2- 172.21.15.18:0/3016622202 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8298109430 0x7f8298109800 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f8288004940 tx=0x7f8288009e30 comp rx=0 tx=0).stop 2024-04-12T21:22:55.996 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:56.001+0000 7f829ea00700 1 -- 172.21.15.18:0/3016622202 shutdown_connections 2024-04-12T21:22:55.997 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:56.001+0000 7f829ea00700 1 --2- 172.21.15.18:0/3016622202 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8298109dd0 0x7f8298103440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:22:55.997 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:56.001+0000 7f829ea00700 1 --2- 172.21.15.18:0/3016622202 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8298109430 0x7f8298109800 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:22:55.997 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:56.001+0000 7f829ea00700 1 -- 172.21.15.18:0/3016622202 >> 172.21.15.18:0/3016622202 conn(0x7f8298072c00 msgr2=0x7f8298075010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:22:55.997 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:22:55 smithi018 ceph-mon[112590]: from='client.35228 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:22:55.997 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:22:55 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' 2024-04-12T21:22:55.997 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:22:55 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T21:22:55.998 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:56.002+0000 7f829ea00700 1 -- 172.21.15.18:0/3016622202 shutdown_connections 2024-04-12T21:22:55.998 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:56.002+0000 7f829ea00700 1 -- 172.21.15.18:0/3016622202 wait complete. 2024-04-12T21:22:55.998 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:56.003+0000 7f829ea00700 1 Processor -- start 2024-04-12T21:22:55.998 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:56.003+0000 7f829ea00700 1 -- start start 2024-04-12T21:22:55.998 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:56.003+0000 7f829ea00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8298109430 0x7f8298119f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:22:55.998 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:56.003+0000 7f829ea00700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8298109dd0 0x7f829811a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:22:55.998 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:56.004+0000 7f829ea00700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f829811aad0 con 0x7f8298109430 2024-04-12T21:22:55.999 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:56.004+0000 7f829ea00700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8298114100 con 0x7f8298109dd0 2024-04-12T21:22:55.999 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:56.004+0000 7f8297e00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8298109430 0x7f8298119f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:22:55.999 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:56.004+0000 7f8297e00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8298109430 0x7f8298119f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:53346/0 (socket says 172.21.15.18:53346) 2024-04-12T21:22:56.000 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:56.004+0000 7f8297e00700 1 -- 172.21.15.18:0/3422366002 learned_addr learned my addr 172.21.15.18:0/3422366002 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:22:56.000 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:56.004+0000 7f8297400700 1 --2- 172.21.15.18:0/3422366002 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8298109dd0 0x7f829811a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:22:56.000 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:56.004+0000 7f8297e00700 1 -- 172.21.15.18:0/3422366002 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8298109dd0 msgr2=0x7f829811a480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:22:56.000 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:56.004+0000 7f8297e00700 1 --2- 172.21.15.18:0/3422366002 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8298109dd0 0x7f829811a480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:22:56.000 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:56.004+0000 7f8297e00700 1 -- 172.21.15.18:0/3422366002 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8288005040 con 0x7f8298109430 2024-04-12T21:22:56.000 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:56.004+0000 7f8297400700 1 --2- 172.21.15.18:0/3422366002 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8298109dd0 0x7f829811a480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-12T21:22:56.000 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:56.005+0000 7f8297e00700 1 --2- 172.21.15.18:0/3422366002 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8298109430 0x7f8298119f40 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f8288009150 tx=0x7f8288009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:22:56.001 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:56.005+0000 7f828fe00700 1 -- 172.21.15.18:0/3422366002 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f828800e040 con 0x7f8298109430 2024-04-12T21:22:56.001 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:56.005+0000 7f828fe00700 1 -- 172.21.15.18:0/3422366002 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f8288013420 con 0x7f8298109430 2024-04-12T21:22:56.001 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:56.005+0000 7f829ea00700 1 -- 172.21.15.18:0/3422366002 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8298114380 con 0x7f8298109430 2024-04-12T21:22:56.001 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:56.005+0000 7f828fe00700 1 -- 172.21.15.18:0/3422366002 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f828801c630 con 0x7f8298109430 2024-04-12T21:22:56.001 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:56.005+0000 7f829ea00700 1 -- 172.21.15.18:0/3422366002 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8298114940 con 0x7f8298109430 2024-04-12T21:22:56.001 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:56.006+0000 7f829ea00700 1 -- 172.21.15.18:0/3422366002 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8298104b80 con 0x7f8298109430 2024-04-12T21:22:56.002 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:56.007+0000 7f828fe00700 1 -- 172.21.15.18:0/3422366002 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7f8288013590 con 0x7f8298109430 2024-04-12T21:22:56.003 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:56.008+0000 7f828fe00700 1 --2- 172.21.15.18:0/3422366002 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f827c06ec00 0x7f827c0710b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:22:56.003 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:56.008+0000 7f828fe00700 1 -- 172.21.15.18:0/3422366002 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 6757+0+0 (secure 0 0 0) 0x7f828808fc40 con 0x7f8298109430 2024-04-12T21:22:56.003 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:56.009+0000 7f8297400700 1 --2- 172.21.15.18:0/3422366002 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f827c06ec00 0x7f827c0710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:22:56.004 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:56.009+0000 7f8297400700 1 --2- 172.21.15.18:0/3422366002 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f827c06ec00 0x7f827c0710b0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f8284009fd0 tx=0x7f8284009ee0 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:22:56.008 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:56.014+0000 7f828fe00700 1 -- 172.21.15.18:0/3422366002 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f828805c490 con 0x7f8298109430 2024-04-12T21:22:56.029 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:22:55 smithi186 ceph-mon[86379]: from='client.35228 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:22:56.029 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:22:55 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' 2024-04-12T21:22:56.029 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:22:55 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T21:22:56.276 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:56.282+0000 7f829ea00700 1 -- 172.21.15.18:0/3422366002 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f8298115590 con 0x7f8298109430 2024-04-12T21:22:56.277 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:56.283+0000 7f828fe00700 1 -- 172.21.15.18:0/3422366002 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 30 v30) v1 ==== 76+0+1694 (secure 0 0 0) 0x7f828805c020 con 0x7f8298109430 2024-04-12T21:22:56.278 INFO:teuthology.orchestra.run.smithi018.stdout:e30 2024-04-12T21:22:56.278 INFO:teuthology.orchestra.run.smithi018.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-12T21:22:56.278 INFO:teuthology.orchestra.run.smithi018.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-12T21:22:56.278 INFO:teuthology.orchestra.run.smithi018.stdout:legacy client fscid: 1 2024-04-12T21:22:56.279 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-12T21:22:56.279 INFO:teuthology.orchestra.run.smithi018.stdout:Filesystem 'cephfs' (1) 2024-04-12T21:22:56.279 INFO:teuthology.orchestra.run.smithi018.stdout:fs_name cephfs 2024-04-12T21:22:56.279 INFO:teuthology.orchestra.run.smithi018.stdout:epoch 24 2024-04-12T21:22:56.279 INFO:teuthology.orchestra.run.smithi018.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-04-12T21:22:56.279 INFO:teuthology.orchestra.run.smithi018.stdout:created 2024-04-12T20:51:57.362850+0000 2024-04-12T21:22:56.279 INFO:teuthology.orchestra.run.smithi018.stdout:modified 2024-04-12T21:06:59.717190+0000 2024-04-12T21:22:56.279 INFO:teuthology.orchestra.run.smithi018.stdout:tableserver 0 2024-04-12T21:22:56.279 INFO:teuthology.orchestra.run.smithi018.stdout:root 0 2024-04-12T21:22:56.279 INFO:teuthology.orchestra.run.smithi018.stdout:session_timeout 60 2024-04-12T21:22:56.279 INFO:teuthology.orchestra.run.smithi018.stdout:session_autoclose 300 2024-04-12T21:22:56.279 INFO:teuthology.orchestra.run.smithi018.stdout:max_file_size 1099511627776 2024-04-12T21:22:56.279 INFO:teuthology.orchestra.run.smithi018.stdout:max_xattr_size 65536 2024-04-12T21:22:56.280 INFO:teuthology.orchestra.run.smithi018.stdout:required_client_features {} 2024-04-12T21:22:56.280 INFO:teuthology.orchestra.run.smithi018.stdout:last_failure 0 2024-04-12T21:22:56.280 INFO:teuthology.orchestra.run.smithi018.stdout:last_failure_osd_epoch 100 2024-04-12T21:22:56.280 INFO:teuthology.orchestra.run.smithi018.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-12T21:22:56.280 INFO:teuthology.orchestra.run.smithi018.stdout:max_mds 2 2024-04-12T21:22:56.280 INFO:teuthology.orchestra.run.smithi018.stdout:in 0,1 2024-04-12T21:22:56.280 INFO:teuthology.orchestra.run.smithi018.stdout:up {} 2024-04-12T21:22:56.280 INFO:teuthology.orchestra.run.smithi018.stdout:failed 0,1 2024-04-12T21:22:56.280 INFO:teuthology.orchestra.run.smithi018.stdout:damaged 2024-04-12T21:22:56.280 INFO:teuthology.orchestra.run.smithi018.stdout:stopped 2024-04-12T21:22:56.280 INFO:teuthology.orchestra.run.smithi018.stdout:data_pools [3] 2024-04-12T21:22:56.280 INFO:teuthology.orchestra.run.smithi018.stdout:metadata_pool 2 2024-04-12T21:22:56.280 INFO:teuthology.orchestra.run.smithi018.stdout:inline_data disabled 2024-04-12T21:22:56.280 INFO:teuthology.orchestra.run.smithi018.stdout:balancer 2024-04-12T21:22:56.280 INFO:teuthology.orchestra.run.smithi018.stdout:bal_rank_mask -1 2024-04-12T21:22:56.281 INFO:teuthology.orchestra.run.smithi018.stdout:standby_count_wanted 1 2024-04-12T21:22:56.281 INFO:teuthology.orchestra.run.smithi018.stdout:qdb_cluster leader: 0 members: 2024-04-12T21:22:56.281 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-12T21:22:56.281 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-12T21:22:56.281 INFO:teuthology.orchestra.run.smithi018.stdout:Standby daemons: 2024-04-12T21:22:56.281 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-12T21:22:56.281 INFO:teuthology.orchestra.run.smithi018.stdout:[mds.cephfs.smithi186.wjnafs{-1:24277} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.186:6826/438347207,v1:172.21.15.186:6827/438347207] compat {c=[1],r=[1],i=[7ff]}] 2024-04-12T21:22:56.281 INFO:teuthology.orchestra.run.smithi018.stdout:[mds.cephfs.smithi018.acqbsc{-1:34436} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.18:6828/755179764,v1:172.21.15.18:6829/755179764] compat {c=[1],r=[1],i=[1fff]}] 2024-04-12T21:22:56.281 INFO:teuthology.orchestra.run.smithi018.stdout:[mds.cephfs.smithi018.szjluo{-1:34468} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.18:6826/2407992653,v1:172.21.15.18:6827/2407992653] compat {c=[1],r=[1],i=[1fff]}] 2024-04-12T21:22:56.282 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:56.286+0000 7f829ea00700 1 -- 172.21.15.18:0/3422366002 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f827c06ec00 msgr2=0x7f827c0710b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:22:56.282 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:56.286+0000 7f829ea00700 1 --2- 172.21.15.18:0/3422366002 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f827c06ec00 0x7f827c0710b0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f8284009fd0 tx=0x7f8284009ee0 comp rx=0 tx=0).stop 2024-04-12T21:22:56.282 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:56.286+0000 7f829ea00700 1 -- 172.21.15.18:0/3422366002 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8298109430 msgr2=0x7f8298119f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:22:56.282 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:56.286+0000 7f829ea00700 1 --2- 172.21.15.18:0/3422366002 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8298109430 0x7f8298119f40 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f8288009150 tx=0x7f8288009e30 comp rx=0 tx=0).stop 2024-04-12T21:22:56.282 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:56.286+0000 7f829ea00700 1 -- 172.21.15.18:0/3422366002 shutdown_connections 2024-04-12T21:22:56.282 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:56.286+0000 7f829ea00700 1 --2- 172.21.15.18:0/3422366002 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f827c06ec00 0x7f827c0710b0 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:22:56.282 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:56.286+0000 7f829ea00700 1 --2- 172.21.15.18:0/3422366002 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8298109430 0x7f8298119f40 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:22:56.282 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:56.286+0000 7f829ea00700 1 --2- 172.21.15.18:0/3422366002 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8298109dd0 0x7f829811a480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:22:56.283 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:56.286+0000 7f829ea00700 1 -- 172.21.15.18:0/3422366002 >> 172.21.15.18:0/3422366002 conn(0x7f8298072c00 msgr2=0x7f8298074c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:22:56.283 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:56.286+0000 7f829ea00700 1 -- 172.21.15.18:0/3422366002 shutdown_connections 2024-04-12T21:22:56.283 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:56.286+0000 7f829ea00700 1 -- 172.21.15.18:0/3422366002 wait complete. 2024-04-12T21:22:56.283 INFO:teuthology.orchestra.run.smithi018.stderr:dumped fsmap epoch 30 2024-04-12T21:22:56.440 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:56.445+0000 7eff83400700 1 -- 172.21.15.18:0/2754018727 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7eff7c105010 msgr2=0x7eff7c1053e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:22:56.440 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:56.445+0000 7eff83400700 1 --2- 172.21.15.18:0/2754018727 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7eff7c105010 0x7eff7c1053e0 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7eff70004940 tx=0x7eff70009e30 comp rx=0 tx=0).stop 2024-04-12T21:22:56.440 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:56.445+0000 7eff83400700 1 -- 172.21.15.18:0/2754018727 shutdown_connections 2024-04-12T21:22:56.440 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:56.445+0000 7eff83400700 1 --2- 172.21.15.18:0/2754018727 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7eff7c1059b0 0x7eff7c0ff020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:22:56.441 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:56.445+0000 7eff83400700 1 --2- 172.21.15.18:0/2754018727 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7eff7c105010 0x7eff7c1053e0 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:22:56.441 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:56.445+0000 7eff83400700 1 -- 172.21.15.18:0/2754018727 >> 172.21.15.18:0/2754018727 conn(0x7eff7c0faba0 msgr2=0x7eff7c0fcfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:22:56.441 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:56.445+0000 7eff83400700 1 -- 172.21.15.18:0/2754018727 shutdown_connections 2024-04-12T21:22:56.441 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:56.445+0000 7eff83400700 1 -- 172.21.15.18:0/2754018727 wait complete. 2024-04-12T21:22:56.442 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:56.446+0000 7eff83400700 1 Processor -- start 2024-04-12T21:22:56.442 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:56.446+0000 7eff83400700 1 -- start start 2024-04-12T21:22:56.442 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:56.447+0000 7eff83400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7eff7c105010 0x7eff7c115d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:22:56.442 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:56.447+0000 7eff83400700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7eff7c1059b0 0x7eff7c1162d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:22:56.442 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:56.447+0000 7eff83400700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff7c1169b0 con 0x7eff7c105010 2024-04-12T21:22:56.442 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:56.447+0000 7eff83400700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff7c10fd30 con 0x7eff7c1059b0 2024-04-12T21:22:56.442 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:56.447+0000 7eff80a00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7eff7c105010 0x7eff7c115d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:22:56.442 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:56.447+0000 7eff80a00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7eff7c105010 0x7eff7c115d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:53360/0 (socket says 172.21.15.18:53360) 2024-04-12T21:22:56.442 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:56.447+0000 7eff80a00700 1 -- 172.21.15.18:0/2077724251 learned_addr learned my addr 172.21.15.18:0/2077724251 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:22:56.442 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:56.447+0000 7eff7be00700 1 --2- 172.21.15.18:0/2077724251 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7eff7c1059b0 0x7eff7c1162d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:22:56.443 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:56.447+0000 7eff80a00700 1 -- 172.21.15.18:0/2077724251 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7eff7c1059b0 msgr2=0x7eff7c1162d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:22:56.443 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:56.447+0000 7eff80a00700 1 --2- 172.21.15.18:0/2077724251 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7eff7c1059b0 0x7eff7c1162d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:22:56.443 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:56.447+0000 7eff80a00700 1 -- 172.21.15.18:0/2077724251 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff70005040 con 0x7eff7c105010 2024-04-12T21:22:56.444 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:56.448+0000 7eff80a00700 1 --2- 172.21.15.18:0/2077724251 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7eff7c105010 0x7eff7c115d90 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7eff70009180 tx=0x7eff70009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:22:56.444 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:56.448+0000 7eff78c00700 1 -- 172.21.15.18:0/2077724251 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7eff7000e040 con 0x7eff7c105010 2024-04-12T21:22:56.444 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:56.448+0000 7eff78c00700 1 -- 172.21.15.18:0/2077724251 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7eff7000bbc0 con 0x7eff7c105010 2024-04-12T21:22:56.444 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:56.448+0000 7eff78c00700 1 -- 172.21.15.18:0/2077724251 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7eff7001c630 con 0x7eff7c105010 2024-04-12T21:22:56.444 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:56.448+0000 7eff83400700 1 -- 172.21.15.18:0/2077724251 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7eff7c10ffb0 con 0x7eff7c105010 2024-04-12T21:22:56.444 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:56.448+0000 7eff83400700 1 -- 172.21.15.18:0/2077724251 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7eff7c110570 con 0x7eff7c105010 2024-04-12T21:22:56.444 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:56.449+0000 7eff83400700 1 -- 172.21.15.18:0/2077724251 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7eff7c100760 con 0x7eff7c105010 2024-04-12T21:22:56.445 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:56.450+0000 7eff78c00700 1 -- 172.21.15.18:0/2077724251 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7eff7000bd30 con 0x7eff7c105010 2024-04-12T21:22:56.446 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:56.451+0000 7eff78c00700 1 --2- 172.21.15.18:0/2077724251 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7eff6406ec10 0x7eff640710c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:22:56.446 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:56.451+0000 7eff78c00700 1 -- 172.21.15.18:0/2077724251 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 6757+0+0 (secure 0 0 0) 0x7eff7008fcd0 con 0x7eff7c105010 2024-04-12T21:22:56.446 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:56.452+0000 7eff7be00700 1 --2- 172.21.15.18:0/2077724251 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7eff6406ec10 0x7eff640710c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:22:56.447 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:56.452+0000 7eff7be00700 1 --2- 172.21.15.18:0/2077724251 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7eff6406ec10 0x7eff640710c0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7eff68006fd0 tx=0x7eff68008040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:22:56.451 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:56.456+0000 7eff78c00700 1 -- 172.21.15.18:0/2077724251 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7eff70058a90 con 0x7eff7c105010 2024-04-12T21:22:56.705 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:56.710+0000 7eff83400700 1 -- 172.21.15.18:0/2077724251 --> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7eff7c04bed0 con 0x7eff6406ec10 2024-04-12T21:22:56.706 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:56.712+0000 7eff78c00700 1 -- 172.21.15.18:0/2077724251 <== mgr.34102 v2:172.21.15.18:6800/3812891919 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7eff7c04bed0 con 0x7eff6406ec10 2024-04-12T21:22:56.707 INFO:teuthology.orchestra.run.smithi018.stdout:{ 2024-04-12T21:22:56.707 INFO:teuthology.orchestra.run.smithi018.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:f4452def3be4d350f4a8e5feaf7ebb1511be9298fe68478ba7a2d6fbd7d69a63", 2024-04-12T21:22:56.707 INFO:teuthology.orchestra.run.smithi018.stdout: "in_progress": true, 2024-04-12T21:22:56.707 INFO:teuthology.orchestra.run.smithi018.stdout: "which": "Upgrading all daemon types on all hosts", 2024-04-12T21:22:56.707 INFO:teuthology.orchestra.run.smithi018.stdout: "services_complete": [ 2024-04-12T21:22:56.707 INFO:teuthology.orchestra.run.smithi018.stdout: "crash", 2024-04-12T21:22:56.707 INFO:teuthology.orchestra.run.smithi018.stdout: "osd", 2024-04-12T21:22:56.707 INFO:teuthology.orchestra.run.smithi018.stdout: "mon", 2024-04-12T21:22:56.707 INFO:teuthology.orchestra.run.smithi018.stdout: "mgr" 2024-04-12T21:22:56.707 INFO:teuthology.orchestra.run.smithi018.stdout: ], 2024-04-12T21:22:56.707 INFO:teuthology.orchestra.run.smithi018.stdout: "progress": "14/21 daemons upgraded", 2024-04-12T21:22:56.708 INFO:teuthology.orchestra.run.smithi018.stdout: "message": "Currently upgrading mds daemons", 2024-04-12T21:22:56.708 INFO:teuthology.orchestra.run.smithi018.stdout: "is_paused": false 2024-04-12T21:22:56.708 INFO:teuthology.orchestra.run.smithi018.stdout:} 2024-04-12T21:22:56.710 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:56.715+0000 7eff83400700 1 -- 172.21.15.18:0/2077724251 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7eff6406ec10 msgr2=0x7eff640710c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:22:56.710 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:56.715+0000 7eff83400700 1 --2- 172.21.15.18:0/2077724251 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7eff6406ec10 0x7eff640710c0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7eff68006fd0 tx=0x7eff68008040 comp rx=0 tx=0).stop 2024-04-12T21:22:56.710 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:56.715+0000 7eff83400700 1 -- 172.21.15.18:0/2077724251 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7eff7c105010 msgr2=0x7eff7c115d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:22:56.710 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:56.715+0000 7eff83400700 1 --2- 172.21.15.18:0/2077724251 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7eff7c105010 0x7eff7c115d90 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7eff70009180 tx=0x7eff70009e30 comp rx=0 tx=0).stop 2024-04-12T21:22:56.710 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:56.715+0000 7eff83400700 1 -- 172.21.15.18:0/2077724251 shutdown_connections 2024-04-12T21:22:56.710 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:56.715+0000 7eff83400700 1 --2- 172.21.15.18:0/2077724251 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7eff6406ec10 0x7eff640710c0 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:22:56.710 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:56.715+0000 7eff83400700 1 --2- 172.21.15.18:0/2077724251 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7eff7c105010 0x7eff7c115d90 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:22:56.710 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:56.715+0000 7eff83400700 1 --2- 172.21.15.18:0/2077724251 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7eff7c1059b0 0x7eff7c1162d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:22:56.711 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:56.715+0000 7eff83400700 1 -- 172.21.15.18:0/2077724251 >> 172.21.15.18:0/2077724251 conn(0x7eff7c0faba0 msgr2=0x7eff7c0fbb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:22:56.711 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:56.715+0000 7eff83400700 1 -- 172.21.15.18:0/2077724251 shutdown_connections 2024-04-12T21:22:56.711 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:56.715+0000 7eff83400700 1 -- 172.21.15.18:0/2077724251 wait complete. 2024-04-12T21:22:56.778 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:22:56 smithi186 ceph-mon[86379]: from='client.35232 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:22:56.779 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:22:56 smithi186 ceph-mon[86379]: from='client.35236 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:22:56.779 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:22:56 smithi186 ceph-mon[86379]: pgmap v695: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:22:56.779 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:22:56 smithi186 ceph-mon[86379]: from='client.? 172.21.15.18:0/1019563989' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T21:22:56.779 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:22:56 smithi186 ceph-mon[86379]: from='client.? 172.21.15.18:0/3422366002' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-12T21:22:56.868 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:56.873+0000 7f0295800700 1 -- 172.21.15.18:0/2672400547 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f02900fde90 msgr2=0x7f02900fe260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:22:56.868 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:56.873+0000 7f0295800700 1 --2- 172.21.15.18:0/2672400547 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f02900fde90 0x7f02900fe260 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f0280004940 tx=0x7f0280009e30 comp rx=0 tx=0).stop 2024-04-12T21:22:56.869 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:56.873+0000 7f0295800700 1 -- 172.21.15.18:0/2672400547 shutdown_connections 2024-04-12T21:22:56.869 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:56.873+0000 7f0295800700 1 --2- 172.21.15.18:0/2672400547 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f02900fe830 0x7f0290106f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:22:56.869 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:56.873+0000 7f0295800700 1 --2- 172.21.15.18:0/2672400547 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f02900fde90 0x7f02900fe260 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:22:56.869 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:56.873+0000 7f0295800700 1 -- 172.21.15.18:0/2672400547 >> 172.21.15.18:0/2672400547 conn(0x7f0290072bc0 msgr2=0x7f0290072fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:22:56.869 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:56.873+0000 7f0295800700 1 -- 172.21.15.18:0/2672400547 shutdown_connections 2024-04-12T21:22:56.869 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:56.873+0000 7f0295800700 1 -- 172.21.15.18:0/2672400547 wait complete. 2024-04-12T21:22:56.870 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:56.874+0000 7f0295800700 1 Processor -- start 2024-04-12T21:22:56.870 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:56.874+0000 7f0295800700 1 -- start start 2024-04-12T21:22:56.870 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:56.875+0000 7f0295800700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f02900fde90 0x7f0290113b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:22:56.870 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:56.875+0000 7f0295800700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f02900fe830 0x7f02901140c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:22:56.871 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:56.875+0000 7f0295800700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02901147a0 con 0x7f02900fe830 2024-04-12T21:22:56.871 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:56.875+0000 7f0295800700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f029010db50 con 0x7f02900fde90 2024-04-12T21:22:56.871 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:56.875+0000 7f028e000700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f02900fe830 0x7f02901140c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:22:56.871 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:56.876+0000 7f028e000700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f02900fe830 0x7f02901140c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:53372/0 (socket says 172.21.15.18:53372) 2024-04-12T21:22:56.871 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:56.876+0000 7f028e000700 1 -- 172.21.15.18:0/1801539913 learned_addr learned my addr 172.21.15.18:0/1801539913 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:22:56.871 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:56.876+0000 7f028ea00700 1 --2- 172.21.15.18:0/1801539913 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f02900fde90 0x7f0290113b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:22:56.872 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:56.876+0000 7f028e000700 1 -- 172.21.15.18:0/1801539913 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f02900fde90 msgr2=0x7f0290113b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:22:56.872 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:56.876+0000 7f028e000700 1 --2- 172.21.15.18:0/1801539913 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f02900fde90 0x7f0290113b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:22:56.872 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:56.876+0000 7f028e000700 1 -- 172.21.15.18:0/1801539913 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0280005040 con 0x7f02900fe830 2024-04-12T21:22:56.872 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:56.876+0000 7f028e000700 1 --2- 172.21.15.18:0/1801539913 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f02900fe830 0x7f02901140c0 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f027c00b790 tx=0x7f027c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:22:56.872 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:56.877+0000 7f0286a00700 1 -- 172.21.15.18:0/1801539913 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f027c00f770 con 0x7f02900fe830 2024-04-12T21:22:56.872 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:56.877+0000 7f0286a00700 1 -- 172.21.15.18:0/1801539913 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f027c00fda0 con 0x7f02900fe830 2024-04-12T21:22:56.872 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:56.877+0000 7f0295800700 1 -- 172.21.15.18:0/1801539913 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f029010de30 con 0x7f02900fe830 2024-04-12T21:22:56.873 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:56.877+0000 7f0286a00700 1 -- 172.21.15.18:0/1801539913 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f027c00d310 con 0x7f02900fe830 2024-04-12T21:22:56.873 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:56.877+0000 7f0295800700 1 -- 172.21.15.18:0/1801539913 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f029010e370 con 0x7f02900fe830 2024-04-12T21:22:56.873 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:56.878+0000 7f0295800700 1 -- 172.21.15.18:0/1801539913 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0290065e30 con 0x7f02900fe830 2024-04-12T21:22:56.874 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:56.879+0000 7f0286a00700 1 -- 172.21.15.18:0/1801539913 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7f027c010420 con 0x7f02900fe830 2024-04-12T21:22:56.875 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:56.880+0000 7f0286a00700 1 --2- 172.21.15.18:0/1801539913 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f027406ec10 0x7f02740710c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:22:56.875 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:56.880+0000 7f0286a00700 1 -- 172.21.15.18:0/1801539913 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 6757+0+0 (secure 0 0 0) 0x7f027c08ea60 con 0x7f02900fe830 2024-04-12T21:22:56.875 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:56.880+0000 7f028ea00700 1 --2- 172.21.15.18:0/1801539913 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f027406ec10 0x7f02740710c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:22:56.876 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:56.881+0000 7f028ea00700 1 --2- 172.21.15.18:0/1801539913 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f027406ec10 0x7f02740710c0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f028000b3b0 tx=0x7f0280009e30 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:22:56.880 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:56.885+0000 7f0286a00700 1 -- 172.21.15.18:0/1801539913 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f027c05b210 con 0x7f02900fe830 2024-04-12T21:22:57.099 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:22:56 smithi018 ceph-mon[112590]: from='client.35232 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:22:57.099 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:22:56 smithi018 ceph-mon[112590]: from='client.35236 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:22:57.100 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:22:56 smithi018 ceph-mon[112590]: pgmap v695: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:22:57.100 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:22:56 smithi018 ceph-mon[112590]: from='client.? 172.21.15.18:0/1019563989' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T21:22:57.100 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:22:56 smithi018 ceph-mon[112590]: from='client.? 172.21.15.18:0/3422366002' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-12T21:22:57.206 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:57.210+0000 7f0295800700 1 -- 172.21.15.18:0/1801539913 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f029004bed0 con 0x7f02900fe830 2024-04-12T21:22:57.206 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:57.211+0000 7f0286a00700 1 -- 172.21.15.18:0/1801539913 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+4303 (secure 0 0 0) 0x7f027c05ada0 con 0x7f02900fe830 2024-04-12T21:22:57.206 INFO:teuthology.orchestra.run.smithi018.stdout:HEALTH_ERR failed to probe daemons or devices; 1 filesystem is degraded; 1 filesystem is offline 2024-04-12T21:22:57.206 INFO:teuthology.orchestra.run.smithi018.stdout:[WRN] CEPHADM_REFRESH_FAILED: failed to probe daemons or devices 2024-04-12T21:22:57.207 INFO:teuthology.orchestra.run.smithi018.stdout: host smithi018 `cephadm ceph-volume` failed: cephadm exited with an error code: 1, stderr: Inferring config /var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/mon.smithi018/config 2024-04-12T21:22:57.207 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/podman: timeout after 895 seconds 2024-04-12T21:22:57.207 INFO:teuthology.orchestra.run.smithi018.stdout:Non-zero exit code 124 from /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:f4452def3be4d350f4a8e5feaf7ebb1511be9298fe68478ba7a2d6fbd7d69a63 -e NODE_NAME=smithi018 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968:/var/run/ceph:z -v /var/log/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968:/var/log/ceph:z -v /var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmpxsmwkkuh:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:f4452def3be4d350f4a8e5feaf7ebb1511be9298fe68478ba7a2d6fbd7d69a63 inventory --format=json-pretty --filter-for-batch 2024-04-12T21:22:57.207 INFO:teuthology.orchestra.run.smithi018.stdout:Traceback (most recent call last): 2024-04-12T21:22:57.207 INFO:teuthology.orchestra.run.smithi018.stdout: File "/usr/lib64/python3.9/runpy.py", line 197, in _run_module_as_main 2024-04-12T21:22:57.207 INFO:teuthology.orchestra.run.smithi018.stdout: return _run_code(code, main_globals, None, 2024-04-12T21:22:57.207 INFO:teuthology.orchestra.run.smithi018.stdout: File "/usr/lib64/python3.9/runpy.py", line 87, in _run_code 2024-04-12T21:22:57.207 INFO:teuthology.orchestra.run.smithi018.stdout: exec(code, run_globals) 2024-04-12T21:22:57.207 INFO:teuthology.orchestra.run.smithi018.stdout: File "/var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/cephadm.e44f3ed5e34e4ef52eedef233d1190bff004470d8e8005127b5312bff357fa7f/__main__.py", line 5618, in 2024-04-12T21:22:57.207 INFO:teuthology.orchestra.run.smithi018.stdout: File "/var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/cephadm.e44f3ed5e34e4ef52eedef233d1190bff004470d8e8005127b5312bff357fa7f/__main__.py", line 5606, in main 2024-04-12T21:22:57.207 INFO:teuthology.orchestra.run.smithi018.stdout: File "/var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/cephadm.e44f3ed5e34e4ef52eedef233d1190bff004470d8e8005127b5312bff357fa7f/__main__.py", line 408, in _infer_config 2024-04-12T21:22:57.207 INFO:teuthology.orchestra.run.smithi018.stdout: File "/var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/cephadm.e44f3ed5e34e4ef52eedef233d1190bff004470d8e8005127b5312bff357fa7f/__main__.py", line 323, in _infer_fsid 2024-04-12T21:22:57.207 INFO:teuthology.orchestra.run.smithi018.stdout: File "/var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/cephadm.e44f3ed5e34e4ef52eedef233d1190bff004470d8e8005127b5312bff357fa7f/__main__.py", line 436, in _infer_image 2024-04-12T21:22:57.207 INFO:teuthology.orchestra.run.smithi018.stdout: File "/var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/cephadm.e44f3ed5e34e4ef52eedef233d1190bff004470d8e8005127b5312bff357fa7f/__main__.py", line 310, in _validate_fsid 2024-04-12T21:22:57.207 INFO:teuthology.orchestra.run.smithi018.stdout: File "/var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/cephadm.e44f3ed5e34e4ef52eedef233d1190bff004470d8e8005127b5312bff357fa7f/__main__.py", line 3359, in command_ceph_volume 2024-04-12T21:22:57.208 INFO:teuthology.orchestra.run.smithi018.stdout: File "/var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/cephadm.e44f3ed5e34e4ef52eedef233d1190bff004470d8e8005127b5312bff357fa7f/cephadmlib/call_wrappers.py", line 310, in call_throws 2024-04-12T21:22:57.208 INFO:teuthology.orchestra.run.smithi018.stdout:RuntimeError: Failed command: /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:f4452def3be4d350f4a8e5feaf7ebb1511be9298fe68478ba7a2d6fbd7d69a63 -e NODE_NAME=smithi018 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968:/var/run/ceph:z -v /var/log/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968:/var/log/ceph:z -v /var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmpxsmwkkuh:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:f4452def3be4d350f4a8e5feaf7ebb1511be9298fe68478ba7a2d6fbd7d69a63 inventory --format=json-pretty --filter-for-batch 2024-04-12T21:22:57.208 INFO:teuthology.orchestra.run.smithi018.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-04-12T21:22:57.208 INFO:teuthology.orchestra.run.smithi018.stdout: fs cephfs is degraded 2024-04-12T21:22:57.208 INFO:teuthology.orchestra.run.smithi018.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-04-12T21:22:57.208 INFO:teuthology.orchestra.run.smithi018.stdout: fs cephfs is offline because no MDS is active for it. 2024-04-12T21:22:57.209 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:57.214+0000 7f0295800700 1 -- 172.21.15.18:0/1801539913 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f027406ec10 msgr2=0x7f02740710c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:22:57.209 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:57.214+0000 7f0295800700 1 --2- 172.21.15.18:0/1801539913 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f027406ec10 0x7f02740710c0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f028000b3b0 tx=0x7f0280009e30 comp rx=0 tx=0).stop 2024-04-12T21:22:57.209 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:57.214+0000 7f0295800700 1 -- 172.21.15.18:0/1801539913 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f02900fe830 msgr2=0x7f02901140c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:22:57.209 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:57.214+0000 7f0295800700 1 --2- 172.21.15.18:0/1801539913 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f02900fe830 0x7f02901140c0 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f027c00b790 tx=0x7f027c009250 comp rx=0 tx=0).stop 2024-04-12T21:22:57.209 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:57.214+0000 7f0295800700 1 -- 172.21.15.18:0/1801539913 shutdown_connections 2024-04-12T21:22:57.209 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:57.214+0000 7f0295800700 1 --2- 172.21.15.18:0/1801539913 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f027406ec10 0x7f02740710c0 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:22:57.210 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:57.214+0000 7f0295800700 1 --2- 172.21.15.18:0/1801539913 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f02900fde90 0x7f0290113b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:22:57.210 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:57.214+0000 7f0295800700 1 --2- 172.21.15.18:0/1801539913 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f02900fe830 0x7f02901140c0 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:22:57.210 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:57.214+0000 7f0295800700 1 -- 172.21.15.18:0/1801539913 >> 172.21.15.18:0/1801539913 conn(0x7f0290072bc0 msgr2=0x7f02900ff440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:22:57.210 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:57.214+0000 7f0295800700 1 -- 172.21.15.18:0/1801539913 shutdown_connections 2024-04-12T21:22:57.210 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:22:57.214+0000 7f0295800700 1 -- 172.21.15.18:0/1801539913 wait complete. 2024-04-12T21:22:58.028 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:22:57 smithi186 ceph-mon[86379]: from='client.35248 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:22:58.028 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:22:57 smithi186 ceph-mon[86379]: from='client.? 172.21.15.18:0/1801539913' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-12T21:22:58.029 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:22:57 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' 2024-04-12T21:22:58.029 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:22:57 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' 2024-04-12T21:22:58.029 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:22:57 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-12T21:22:58.099 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:22:57 smithi018 ceph-mon[112590]: from='client.35248 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:22:58.099 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:22:57 smithi018 ceph-mon[112590]: from='client.? 172.21.15.18:0/1801539913' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-12T21:22:58.100 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:22:57 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' 2024-04-12T21:22:58.100 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:22:57 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' 2024-04-12T21:22:58.100 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:22:57 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-12T21:22:59.028 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:22:58 smithi186 ceph-mon[86379]: pgmap v696: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:22:59.028 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:22:58 smithi186 ceph-mon[86379]: mds.? [v2:172.21.15.186:6824/3523444202,v1:172.21.15.186:6825/3523444202] up:boot 2024-04-12T21:22:59.029 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:22:58 smithi186 ceph-mon[86379]: fsmap cephfs:0/2 4 up:standby, 2 failed 2024-04-12T21:22:59.029 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:22:58 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi186.jdllyn"}]: dispatch 2024-04-12T21:22:59.029 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:22:58 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' 2024-04-12T21:22:59.029 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:22:58 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' 2024-04-12T21:22:59.029 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:22:58 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' 2024-04-12T21:22:59.029 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:22:58 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' 2024-04-12T21:22:59.099 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:22:58 smithi018 ceph-mon[112590]: pgmap v696: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:22:59.100 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:22:58 smithi018 ceph-mon[112590]: mds.? [v2:172.21.15.186:6824/3523444202,v1:172.21.15.186:6825/3523444202] up:boot 2024-04-12T21:22:59.100 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:22:58 smithi018 ceph-mon[112590]: fsmap cephfs:0/2 4 up:standby, 2 failed 2024-04-12T21:22:59.100 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:22:58 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi186.jdllyn"}]: dispatch 2024-04-12T21:22:59.100 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:22:58 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' 2024-04-12T21:22:59.100 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:22:58 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' 2024-04-12T21:22:59.100 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:22:58 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' 2024-04-12T21:22:59.100 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:22:58 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' 2024-04-12T21:23:00.778 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:23:00 smithi186 ceph-mon[86379]: pgmap v697: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:23:01.099 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:23:00 smithi018 ceph-mon[112590]: pgmap v697: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:23:03.028 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:23:02 smithi186 ceph-mon[86379]: pgmap v698: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:23:03.028 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:23:02 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' 2024-04-12T21:23:03.028 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:23:02 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' 2024-04-12T21:23:03.091 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:23:02 smithi018 ceph-mon[112590]: pgmap v698: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:23:03.092 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:23:02 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' 2024-04-12T21:23:03.092 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:23:02 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' 2024-04-12T21:23:05.028 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:23:04 smithi186 ceph-mon[86379]: pgmap v699: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:23:05.028 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:23:04 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' 2024-04-12T21:23:05.028 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:23:04 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' 2024-04-12T21:23:05.099 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:23:04 smithi018 ceph-mon[112590]: pgmap v699: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:23:05.100 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:23:04 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' 2024-04-12T21:23:05.100 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:23:04 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' 2024-04-12T21:23:06.581 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:23:06 smithi186 ceph-mon[86379]: pgmap v700: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:23:06.581 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:23:06 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' 2024-04-12T21:23:06.582 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:23:06 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' 2024-04-12T21:23:06.849 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:23:06 smithi018 ceph-mon[112590]: pgmap v700: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:23:06.849 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:23:06 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' 2024-04-12T21:23:06.849 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:23:06 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' 2024-04-12T21:23:08.778 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:23:08 smithi186 ceph-mon[86379]: pgmap v701: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:23:08.849 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:23:08 smithi018 ceph-mon[112590]: pgmap v701: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:23:10.779 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:23:10 smithi186 ceph-mon[86379]: pgmap v702: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:23:10.779 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:23:10 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T21:23:10.849 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:23:10 smithi018 ceph-mon[112590]: pgmap v702: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:23:10.849 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:23:10 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T21:23:12.779 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:23:12 smithi186 ceph-mon[86379]: pgmap v703: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:23:12.849 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:23:12 smithi018 ceph-mon[112590]: pgmap v703: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:23:14.778 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:23:14 smithi186 ceph-mon[86379]: pgmap v704: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:23:14.849 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:23:14 smithi018 ceph-mon[112590]: pgmap v704: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:23:16.778 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:23:16 smithi186 ceph-mon[86379]: pgmap v705: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:23:16.849 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:23:16 smithi018 ceph-mon[112590]: pgmap v705: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:23:18.778 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:23:18 smithi186 ceph-mon[86379]: pgmap v706: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:23:18.849 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:23:18 smithi018 ceph-mon[112590]: pgmap v706: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:23:20.778 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:23:20 smithi186 ceph-mon[86379]: pgmap v707: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:23:20.849 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:23:20 smithi018 ceph-mon[112590]: pgmap v707: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:23:22.778 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:23:22 smithi186 ceph-mon[86379]: pgmap v708: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:23:22.849 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:23:22 smithi018 ceph-mon[112590]: pgmap v708: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:23:24.849 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:23:24 smithi018 ceph-mon[112590]: pgmap v709: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:23:25.028 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:23:24 smithi186 ceph-mon[86379]: pgmap v709: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:23:25.849 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:23:25 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T21:23:26.028 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:23:25 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T21:23:26.849 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:23:26 smithi018 ceph-mon[112590]: pgmap v710: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:23:27.028 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:23:26 smithi186 ceph-mon[86379]: pgmap v710: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:23:27.384 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:23:27.387+0000 7f3469600700 1 -- 172.21.15.18:0/308161260 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3464071f90 msgr2=0x7f3464070360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:23:27.384 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:23:27.387+0000 7f3469600700 1 --2- 172.21.15.18:0/308161260 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3464071f90 0x7f3464070360 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f3454004970 tx=0x7f3454009e30 comp rx=0 tx=0).stop 2024-04-12T21:23:27.384 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:23:27.387+0000 7f3469600700 1 -- 172.21.15.18:0/308161260 shutdown_connections 2024-04-12T21:23:27.384 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:23:27.387+0000 7f3469600700 1 --2- 172.21.15.18:0/308161260 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f34640708a0 0x7f3464070d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:23:27.384 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:23:27.387+0000 7f3469600700 1 --2- 172.21.15.18:0/308161260 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3464071f90 0x7f3464070360 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:23:27.384 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:23:27.387+0000 7f3469600700 1 -- 172.21.15.18:0/308161260 >> 172.21.15.18:0/308161260 conn(0x7f34640f8970 msgr2=0x7f34640fad80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:23:27.384 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:23:27.387+0000 7f3469600700 1 -- 172.21.15.18:0/308161260 shutdown_connections 2024-04-12T21:23:27.384 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:23:27.387+0000 7f3469600700 1 -- 172.21.15.18:0/308161260 wait complete. 2024-04-12T21:23:27.384 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:23:27.388+0000 7f3469600700 1 Processor -- start 2024-04-12T21:23:27.384 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:23:27.388+0000 7f3469600700 1 -- start start 2024-04-12T21:23:27.386 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:23:27.388+0000 7f3469600700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f34640708a0 0x7f346411a1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:23:27.386 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:23:27.388+0000 7f3469600700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3464071f90 0x7f346411a720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:23:27.386 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:23:27.388+0000 7f3469600700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f346411ae00 con 0x7f34640708a0 2024-04-12T21:23:27.386 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:23:27.389+0000 7f3469600700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3464114140 con 0x7f3464071f90 2024-04-12T21:23:27.386 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:23:27.389+0000 7f3462a00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f34640708a0 0x7f346411a1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:23:27.386 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:23:27.389+0000 7f3462a00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f34640708a0 0x7f346411a1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:52910/0 (socket says 172.21.15.18:52910) 2024-04-12T21:23:27.387 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:23:27.389+0000 7f3462a00700 1 -- 172.21.15.18:0/3375528397 learned_addr learned my addr 172.21.15.18:0/3375528397 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:23:27.387 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:23:27.389+0000 7f3462a00700 1 -- 172.21.15.18:0/3375528397 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3464071f90 msgr2=0x7f346411a720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:23:27.387 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:23:27.389+0000 7f3462a00700 1 --2- 172.21.15.18:0/3375528397 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3464071f90 0x7f346411a720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:23:27.387 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:23:27.389+0000 7f3462a00700 1 -- 172.21.15.18:0/3375528397 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3454005040 con 0x7f34640708a0 2024-04-12T21:23:27.387 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:23:27.390+0000 7f3462a00700 1 --2- 172.21.15.18:0/3375528397 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f34640708a0 0x7f346411a1e0 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f3454004f50 tx=0x7f3454009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:23:27.387 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:23:27.390+0000 7f345aa00700 1 -- 172.21.15.18:0/3375528397 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f345400e040 con 0x7f34640708a0 2024-04-12T21:23:27.387 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:23:27.390+0000 7f3469600700 1 -- 172.21.15.18:0/3375528397 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3464114360 con 0x7f34640708a0 2024-04-12T21:23:27.387 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:23:27.390+0000 7f345aa00700 1 -- 172.21.15.18:0/3375528397 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f345400bc10 con 0x7f34640708a0 2024-04-12T21:23:27.387 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:23:27.390+0000 7f3469600700 1 -- 172.21.15.18:0/3375528397 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f34641148a0 con 0x7f34640708a0 2024-04-12T21:23:27.387 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:23:27.390+0000 7f345aa00700 1 -- 172.21.15.18:0/3375528397 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f34540255e0 con 0x7f34640708a0 2024-04-12T21:23:27.388 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:23:27.391+0000 7f3469600700 1 -- 172.21.15.18:0/3375528397 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3464071aa0 con 0x7f34640708a0 2024-04-12T21:23:27.388 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:23:27.392+0000 7f345aa00700 1 -- 172.21.15.18:0/3375528397 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7f3454013420 con 0x7f34640708a0 2024-04-12T21:23:27.389 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:23:27.393+0000 7f345aa00700 1 --2- 172.21.15.18:0/3375528397 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f344806ec10 0x7f34480710c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:23:27.389 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:23:27.393+0000 7f345aa00700 1 -- 172.21.15.18:0/3375528397 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 6757+0+0 (secure 0 0 0) 0x7f345408fd80 con 0x7f34640708a0 2024-04-12T21:23:27.389 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:23:27.394+0000 7f3462000700 1 --2- 172.21.15.18:0/3375528397 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f344806ec10 0x7f34480710c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:23:27.390 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:23:27.394+0000 7f3462000700 1 --2- 172.21.15.18:0/3375528397 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f344806ec10 0x7f34480710c0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f3450006fd0 tx=0x7f3450006db0 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:23:27.394 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:23:27.399+0000 7f345aa00700 1 -- 172.21.15.18:0/3375528397 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f345405c530 con 0x7f34640708a0 2024-04-12T21:23:27.648 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:23:27.652+0000 7f3469600700 1 -- 172.21.15.18:0/3375528397 --> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f346404bed0 con 0x7f344806ec10 2024-04-12T21:23:27.651 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:23:27.655+0000 7f345aa00700 1 -- 172.21.15.18:0/3375528397 <== mgr.34102 v2:172.21.15.18:6800/3812891919 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f346404bed0 con 0x7f344806ec10 2024-04-12T21:23:27.655 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:23:27.658+0000 7f3469600700 1 -- 172.21.15.18:0/3375528397 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f344806ec10 msgr2=0x7f34480710c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:23:27.655 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:23:27.658+0000 7f3469600700 1 --2- 172.21.15.18:0/3375528397 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f344806ec10 0x7f34480710c0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f3450006fd0 tx=0x7f3450006db0 comp rx=0 tx=0).stop 2024-04-12T21:23:27.655 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:23:27.658+0000 7f3469600700 1 -- 172.21.15.18:0/3375528397 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f34640708a0 msgr2=0x7f346411a1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:23:27.655 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:23:27.658+0000 7f3469600700 1 --2- 172.21.15.18:0/3375528397 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f34640708a0 0x7f346411a1e0 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f3454004f50 tx=0x7f3454009e30 comp rx=0 tx=0).stop 2024-04-12T21:23:27.655 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:23:27.659+0000 7f3469600700 1 -- 172.21.15.18:0/3375528397 shutdown_connections 2024-04-12T21:23:27.655 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:23:27.659+0000 7f3469600700 1 --2- 172.21.15.18:0/3375528397 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f344806ec10 0x7f34480710c0 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:23:27.655 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:23:27.659+0000 7f3469600700 1 --2- 172.21.15.18:0/3375528397 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f34640708a0 0x7f346411a1e0 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:23:27.655 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:23:27.659+0000 7f3469600700 1 --2- 172.21.15.18:0/3375528397 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3464071f90 0x7f346411a720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:23:27.655 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:23:27.659+0000 7f3469600700 1 -- 172.21.15.18:0/3375528397 >> 172.21.15.18:0/3375528397 conn(0x7f34640f8970 msgr2=0x7f34640f9a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:23:27.655 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:23:27.659+0000 7f3469600700 1 -- 172.21.15.18:0/3375528397 shutdown_connections 2024-04-12T21:23:27.656 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:23:27.659+0000 7f3469600700 1 -- 172.21.15.18:0/3375528397 wait complete. 2024-04-12T21:23:27.672 INFO:teuthology.orchestra.run.smithi018.stdout:true 2024-04-12T21:23:27.821 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:23:27.825+0000 7fa71bc00700 1 -- 172.21.15.18:0/2649198131 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa714105020 msgr2=0x7fa7141053f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:23:27.821 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:23:27.825+0000 7fa71bc00700 1 --2- 172.21.15.18:0/2649198131 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa714105020 0x7fa7141053f0 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7fa708009ce0 tx=0x7fa708004d10 comp rx=0 tx=0).stop 2024-04-12T21:23:27.821 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:23:27.825+0000 7fa71bc00700 1 -- 172.21.15.18:0/2649198131 shutdown_connections 2024-04-12T21:23:27.821 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:23:27.825+0000 7fa71bc00700 1 --2- 172.21.15.18:0/2649198131 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa7141059c0 0x7fa7140ff080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:23:27.821 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:23:27.825+0000 7fa71bc00700 1 --2- 172.21.15.18:0/2649198131 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa714105020 0x7fa7141053f0 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:23:27.821 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:23:27.825+0000 7fa71bc00700 1 -- 172.21.15.18:0/2649198131 >> 172.21.15.18:0/2649198131 conn(0x7fa7140faba0 msgr2=0x7fa7140fcfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:23:27.821 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:23:27.825+0000 7fa71bc00700 1 -- 172.21.15.18:0/2649198131 shutdown_connections 2024-04-12T21:23:27.821 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:23:27.825+0000 7fa71bc00700 1 -- 172.21.15.18:0/2649198131 wait complete. 2024-04-12T21:23:27.822 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:23:27.826+0000 7fa71bc00700 1 Processor -- start 2024-04-12T21:23:27.823 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:23:27.826+0000 7fa71bc00700 1 -- start start 2024-04-12T21:23:27.823 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:23:27.826+0000 7fa71bc00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa714105020 0x7fa714115d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:23:27.823 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:23:27.826+0000 7fa71bc00700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa7141059c0 0x7fa7141162b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:23:27.823 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:23:27.826+0000 7fa71bc00700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa714116990 con 0x7fa714105020 2024-04-12T21:23:27.823 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:23:27.826+0000 7fa71bc00700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa71410fee0 con 0x7fa7141059c0 2024-04-12T21:23:27.823 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:23:27.827+0000 7fa719200700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa714105020 0x7fa714115d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:23:27.823 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:23:27.827+0000 7fa719200700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa714105020 0x7fa714115d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:52930/0 (socket says 172.21.15.18:52930) 2024-04-12T21:23:27.823 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:23:27.827+0000 7fa719200700 1 -- 172.21.15.18:0/345423767 learned_addr learned my addr 172.21.15.18:0/345423767 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:23:27.823 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:23:27.827+0000 7fa719200700 1 -- 172.21.15.18:0/345423767 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa7141059c0 msgr2=0x7fa7141162b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:23:27.823 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:23:27.827+0000 7fa719200700 1 --2- 172.21.15.18:0/345423767 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa7141059c0 0x7fa7141162b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:23:27.823 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:23:27.827+0000 7fa719200700 1 -- 172.21.15.18:0/345423767 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa708009940 con 0x7fa714105020 2024-04-12T21:23:27.824 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:23:27.827+0000 7fa719200700 1 --2- 172.21.15.18:0/345423767 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa714105020 0x7fa714115d70 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7fa708005010 tx=0x7fa708004d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:23:27.824 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:23:27.828+0000 7fa710c00700 1 -- 172.21.15.18:0/345423767 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa70800e040 con 0x7fa714105020 2024-04-12T21:23:27.824 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:23:27.828+0000 7fa710c00700 1 -- 172.21.15.18:0/345423767 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa708008e20 con 0x7fa714105020 2024-04-12T21:23:27.824 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:23:27.828+0000 7fa71bc00700 1 -- 172.21.15.18:0/345423767 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa714110100 con 0x7fa714105020 2024-04-12T21:23:27.824 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:23:27.828+0000 7fa710c00700 1 -- 172.21.15.18:0/345423767 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa70801c660 con 0x7fa714105020 2024-04-12T21:23:27.825 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:23:27.828+0000 7fa71bc00700 1 -- 172.21.15.18:0/345423767 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa7141106c0 con 0x7fa714105020 2024-04-12T21:23:27.825 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:23:27.829+0000 7fa71bc00700 1 -- 172.21.15.18:0/345423767 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa714100770 con 0x7fa714105020 2024-04-12T21:23:27.826 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:23:27.830+0000 7fa710c00700 1 -- 172.21.15.18:0/345423767 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7fa708012a70 con 0x7fa714105020 2024-04-12T21:23:27.827 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:23:27.831+0000 7fa710c00700 1 --2- 172.21.15.18:0/345423767 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fa6fc06ec00 0x7fa6fc0710b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:23:27.827 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:23:27.831+0000 7fa710c00700 1 -- 172.21.15.18:0/345423767 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 6757+0+0 (secure 0 0 0) 0x7fa708090510 con 0x7fa714105020 2024-04-12T21:23:27.827 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:23:27.832+0000 7fa713e00700 1 --2- 172.21.15.18:0/345423767 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fa6fc06ec00 0x7fa6fc0710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:23:27.828 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:23:27.832+0000 7fa713e00700 1 --2- 172.21.15.18:0/345423767 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fa6fc06ec00 0x7fa6fc0710b0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fa700006fd0 tx=0x7fa700006db0 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:23:27.833 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:23:27.837+0000 7fa710c00700 1 -- 172.21.15.18:0/345423767 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7fa70805ccc0 con 0x7fa714105020 2024-04-12T21:23:28.087 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:23:28.091+0000 7fa71bc00700 1 -- 172.21.15.18:0/345423767 --> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fa7141109f0 con 0x7fa6fc06ec00 2024-04-12T21:23:28.089 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:23:28.093+0000 7fa710c00700 1 -- 172.21.15.18:0/345423767 <== mgr.34102 v2:172.21.15.18:6800/3812891919 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7fa7141109f0 con 0x7fa6fc06ec00 2024-04-12T21:23:28.092 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:23:28.096+0000 7fa71bc00700 1 -- 172.21.15.18:0/345423767 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fa6fc06ec00 msgr2=0x7fa6fc0710b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:23:28.092 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:23:28.096+0000 7fa71bc00700 1 --2- 172.21.15.18:0/345423767 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fa6fc06ec00 0x7fa6fc0710b0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fa700006fd0 tx=0x7fa700006db0 comp rx=0 tx=0).stop 2024-04-12T21:23:28.092 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:23:28.096+0000 7fa71bc00700 1 -- 172.21.15.18:0/345423767 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa714105020 msgr2=0x7fa714115d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:23:28.092 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:23:28.096+0000 7fa71bc00700 1 --2- 172.21.15.18:0/345423767 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa714105020 0x7fa714115d70 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7fa708005010 tx=0x7fa708004d10 comp rx=0 tx=0).stop 2024-04-12T21:23:28.092 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:23:28.096+0000 7fa71bc00700 1 -- 172.21.15.18:0/345423767 shutdown_connections 2024-04-12T21:23:28.092 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:23:28.096+0000 7fa71bc00700 1 --2- 172.21.15.18:0/345423767 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fa6fc06ec00 0x7fa6fc0710b0 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:23:28.093 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:23:28.096+0000 7fa71bc00700 1 --2- 172.21.15.18:0/345423767 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa714105020 0x7fa714115d70 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:23:28.093 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:23:28.096+0000 7fa71bc00700 1 --2- 172.21.15.18:0/345423767 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa7141059c0 0x7fa7141162b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:23:28.093 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:23:28.096+0000 7fa71bc00700 1 -- 172.21.15.18:0/345423767 >> 172.21.15.18:0/345423767 conn(0x7fa7140faba0 msgr2=0x7fa7140fbb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:23:28.093 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:23:28.096+0000 7fa71bc00700 1 -- 172.21.15.18:0/345423767 shutdown_connections 2024-04-12T21:23:28.093 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:23:28.096+0000 7fa71bc00700 1 -- 172.21.15.18:0/345423767 wait complete. 2024-04-12T21:23:28.251 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:23:28.255+0000 7f5e68a00700 1 -- 172.21.15.18:0/3723276235 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5e641035d0 msgr2=0x7f5e640fcc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:23:28.251 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:23:28.255+0000 7f5e68a00700 1 --2- 172.21.15.18:0/3723276235 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5e641035d0 0x7f5e640fcc40 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f5e4c004940 tx=0x7f5e4c009e30 comp rx=0 tx=0).stop 2024-04-12T21:23:28.251 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:23:28.255+0000 7f5e68a00700 1 -- 172.21.15.18:0/3723276235 shutdown_connections 2024-04-12T21:23:28.252 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:23:28.255+0000 7f5e68a00700 1 --2- 172.21.15.18:0/3723276235 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5e641035d0 0x7f5e640fcc40 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:23:28.252 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:23:28.255+0000 7f5e68a00700 1 --2- 172.21.15.18:0/3723276235 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5e64102c30 0x7f5e64103000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:23:28.252 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:23:28.255+0000 7f5e68a00700 1 -- 172.21.15.18:0/3723276235 >> 172.21.15.18:0/3723276235 conn(0x7f5e640f8760 msgr2=0x7f5e640fab70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:23:28.252 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:23:28.255+0000 7f5e68a00700 1 -- 172.21.15.18:0/3723276235 shutdown_connections 2024-04-12T21:23:28.252 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:23:28.255+0000 7f5e68a00700 1 -- 172.21.15.18:0/3723276235 wait complete. 2024-04-12T21:23:28.253 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:23:28.256+0000 7f5e68a00700 1 Processor -- start 2024-04-12T21:23:28.253 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:23:28.256+0000 7f5e68a00700 1 -- start start 2024-04-12T21:23:28.253 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:23:28.257+0000 7f5e68a00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5e64102c30 0x7f5e64113a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:23:28.253 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:23:28.257+0000 7f5e68a00700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5e641035d0 0x7f5e64113f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:23:28.253 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:23:28.257+0000 7f5e68a00700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e64114590 con 0x7f5e64102c30 2024-04-12T21:23:28.253 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:23:28.257+0000 7f5e68a00700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e6410d980 con 0x7f5e641035d0 2024-04-12T21:23:28.253 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:23:28.257+0000 7f5e62800700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5e64102c30 0x7f5e64113a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:23:28.253 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:23:28.257+0000 7f5e62800700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5e64102c30 0x7f5e64113a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:52954/0 (socket says 172.21.15.18:52954) 2024-04-12T21:23:28.253 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:23:28.257+0000 7f5e62800700 1 -- 172.21.15.18:0/1839591849 learned_addr learned my addr 172.21.15.18:0/1839591849 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:23:28.255 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:23:28.258+0000 7f5e59e00700 1 --2- 172.21.15.18:0/1839591849 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5e641035d0 0x7f5e64113f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:23:28.255 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:23:28.258+0000 7f5e62800700 1 -- 172.21.15.18:0/1839591849 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5e641035d0 msgr2=0x7f5e64113f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:23:28.255 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:23:28.258+0000 7f5e62800700 1 --2- 172.21.15.18:0/1839591849 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5e641035d0 0x7f5e64113f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:23:28.255 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:23:28.258+0000 7f5e62800700 1 -- 172.21.15.18:0/1839591849 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e4c005040 con 0x7f5e64102c30 2024-04-12T21:23:28.255 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:23:28.258+0000 7f5e62800700 1 --2- 172.21.15.18:0/1839591849 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5e64102c30 0x7f5e64113a00 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f5e5400cb10 tx=0x7f5e54007f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:23:28.255 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:23:28.258+0000 7f5e5b400700 1 -- 172.21.15.18:0/1839591849 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5e54004200 con 0x7f5e64102c30 2024-04-12T21:23:28.255 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:23:28.258+0000 7f5e5b400700 1 -- 172.21.15.18:0/1839591849 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f5e5400acc0 con 0x7f5e64102c30 2024-04-12T21:23:28.255 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:23:28.258+0000 7f5e68a00700 1 -- 172.21.15.18:0/1839591849 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5e6410dc60 con 0x7f5e64102c30 2024-04-12T21:23:28.255 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:23:28.258+0000 7f5e5b400700 1 -- 172.21.15.18:0/1839591849 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5e54003d90 con 0x7f5e64102c30 2024-04-12T21:23:28.255 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:23:28.259+0000 7f5e68a00700 1 -- 172.21.15.18:0/1839591849 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5e6410e220 con 0x7f5e64102c30 2024-04-12T21:23:28.256 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:23:28.260+0000 7f5e68a00700 1 -- 172.21.15.18:0/1839591849 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5e640fe380 con 0x7f5e64102c30 2024-04-12T21:23:28.256 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:23:28.261+0000 7f5e5b400700 1 -- 172.21.15.18:0/1839591849 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7f5e54003430 con 0x7f5e64102c30 2024-04-12T21:23:28.258 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:23:28.262+0000 7f5e5b400700 1 --2- 172.21.15.18:0/1839591849 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f5e4806ec00 0x7f5e480710b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:23:28.258 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:23:28.262+0000 7f5e5b400700 1 -- 172.21.15.18:0/1839591849 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 6757+0+0 (secure 0 0 0) 0x7f5e5408ef70 con 0x7f5e64102c30 2024-04-12T21:23:28.258 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:23:28.262+0000 7f5e59e00700 1 --2- 172.21.15.18:0/1839591849 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f5e4806ec00 0x7f5e480710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:23:28.259 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:23:28.263+0000 7f5e59e00700 1 --2- 172.21.15.18:0/1839591849 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f5e4806ec00 0x7f5e480710b0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f5e6410eb40 tx=0x7f5e4c009e30 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:23:28.263 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:23:28.267+0000 7f5e5b400700 1 -- 172.21.15.18:0/1839591849 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f5e5405b720 con 0x7f5e64102c30 2024-04-12T21:23:28.509 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:23:28.513+0000 7f5e68a00700 1 -- 172.21.15.18:0/1839591849 --> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f5e64109d90 con 0x7f5e4806ec00 2024-04-12T21:23:28.536 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:23:28.540+0000 7f5e5b400700 1 -- 172.21.15.18:0/1839591849 <== mgr.34102 v2:172.21.15.18:6800/3812891919 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7f5e64109d90 con 0x7f5e4806ec00 2024-04-12T21:23:28.536 INFO:teuthology.orchestra.run.smithi018.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-12T21:23:28.536 INFO:teuthology.orchestra.run.smithi018.stdout:alertmanager.smithi018 smithi018 *:9093,9094 running (24m) 23s ago 38m 15.5M - 0.25.0 c8568f914cd2 c332090068e8 2024-04-12T21:23:28.536 INFO:teuthology.orchestra.run.smithi018.stdout:crash.smithi018 smithi018 running (20m) 23s ago 38m 6665k - 19.0.0-2879-g324baf8d 8717720dc5e5 b7f80d1f7982 2024-04-12T21:23:28.536 INFO:teuthology.orchestra.run.smithi018.stdout:crash.smithi186 smithi186 running (20m) 26s ago 37m 6660k - 19.0.0-2879-g324baf8d 8717720dc5e5 a4f55c81b623 2024-04-12T21:23:28.536 INFO:teuthology.orchestra.run.smithi018.stdout:grafana.smithi018 smithi018 *:3000 running (23m) 23s ago 38m 74.8M - 9.4.12 2bacad6d85d8 b08e3bf176dd 2024-04-12T21:23:28.536 INFO:teuthology.orchestra.run.smithi018.stdout:mds.cephfs.smithi018.acqbsc smithi018 running (16m) 23s ago 31m 16.7M - 19.0.0-2879-g324baf8d 8717720dc5e5 021f99ab24ae 2024-04-12T21:23:28.536 INFO:teuthology.orchestra.run.smithi018.stdout:mds.cephfs.smithi018.szjluo smithi018 running (15m) 23s ago 31m 16.5M - 19.0.0-2879-g324baf8d 8717720dc5e5 0003117479d4 2024-04-12T21:23:28.537 INFO:teuthology.orchestra.run.smithi018.stdout:mds.cephfs.smithi186.jdllyn smithi186 running (31s) 26s ago 31m 11.0M - 19.0.0-2879-g324baf8d 8717720dc5e5 21b1f46eef23 2024-04-12T21:23:28.537 INFO:teuthology.orchestra.run.smithi018.stdout:mds.cephfs.smithi186.wjnafs smithi186 running (31m) 26s ago 31m 19.5M - 17.2.7-827-gd3b5060a 21de9b7369ad 74b115d6d6fd 2024-04-12T21:23:28.537 INFO:teuthology.orchestra.run.smithi018.stdout:mgr.smithi018.xjzwga smithi018 *:8443,9283,8765 running (26m) 23s ago 40m 512M - 19.0.0-2879-g324baf8d 8717720dc5e5 56088305eab7 2024-04-12T21:23:28.537 INFO:teuthology.orchestra.run.smithi018.stdout:mgr.smithi186.kqcfxk smithi186 *:8443,9283,8765 running (25m) 26s ago 37m 459M - 19.0.0-2879-g324baf8d 8717720dc5e5 09cd766e5e0d 2024-04-12T21:23:28.537 INFO:teuthology.orchestra.run.smithi018.stdout:mon.smithi018 smithi018 running (21m) 23s ago 40m 73.1M 2048M 19.0.0-2879-g324baf8d 8717720dc5e5 df3e201dfed6 2024-04-12T21:23:28.537 INFO:teuthology.orchestra.run.smithi018.stdout:mon.smithi186 smithi186 running (21m) 26s ago 37m 47.2M 2048M 19.0.0-2879-g324baf8d 8717720dc5e5 c77498da9b2b 2024-04-12T21:23:28.537 INFO:teuthology.orchestra.run.smithi018.stdout:node-exporter.smithi018 smithi018 *:9100 running (25m) 23s ago 38m 8108k - 1.5.0 0da6a335fe13 326c3d9c6847 2024-04-12T21:23:28.537 INFO:teuthology.orchestra.run.smithi018.stdout:node-exporter.smithi186 smithi186 *:9100 running (25m) 26s ago 37m 8757k - 1.5.0 0da6a335fe13 ec2112baeecf 2024-04-12T21:23:28.537 INFO:teuthology.orchestra.run.smithi018.stdout:osd.0 smithi018 running (19m) 23s ago 36m 123M 1559M 19.0.0-2879-g324baf8d 8717720dc5e5 60b1273ca047 2024-04-12T21:23:28.537 INFO:teuthology.orchestra.run.smithi018.stdout:osd.1 smithi018 running (19m) 23s ago 35m 95.1M 1559M 19.0.0-2879-g324baf8d 8717720dc5e5 170ce1e5fb8d 2024-04-12T21:23:28.537 INFO:teuthology.orchestra.run.smithi018.stdout:osd.2 smithi018 running (18m) 23s ago 34m 94.3M 1559M 19.0.0-2879-g324baf8d 8717720dc5e5 c2cbc6aa2839 2024-04-12T21:23:28.537 INFO:teuthology.orchestra.run.smithi018.stdout:osd.3 smithi186 running (18m) 26s ago 34m 101M 2581M 19.0.0-2879-g324baf8d 8717720dc5e5 0138e8262e4b 2024-04-12T21:23:28.537 INFO:teuthology.orchestra.run.smithi018.stdout:osd.4 smithi186 running (17m) 26s ago 33m 91.0M 2581M 19.0.0-2879-g324baf8d 8717720dc5e5 b20cb63b25c9 2024-04-12T21:23:28.537 INFO:teuthology.orchestra.run.smithi018.stdout:osd.5 smithi186 running (16m) 26s ago 33m 92.0M 2581M 19.0.0-2879-g324baf8d 8717720dc5e5 2513bea2610e 2024-04-12T21:23:28.538 INFO:teuthology.orchestra.run.smithi018.stdout:prometheus.smithi018 smithi018 *:9095 running (24m) 23s ago 37m 67.9M - 2.43.0 a07b618ecd1d f58841127e21 2024-04-12T21:23:28.538 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:23:28.542+0000 7f5e68a00700 1 -- 172.21.15.18:0/1839591849 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f5e4806ec00 msgr2=0x7f5e480710b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:23:28.538 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:23:28.542+0000 7f5e68a00700 1 --2- 172.21.15.18:0/1839591849 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f5e4806ec00 0x7f5e480710b0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f5e6410eb40 tx=0x7f5e4c009e30 comp rx=0 tx=0).stop 2024-04-12T21:23:28.539 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:23:28.543+0000 7f5e68a00700 1 -- 172.21.15.18:0/1839591849 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5e64102c30 msgr2=0x7f5e64113a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:23:28.539 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:23:28.543+0000 7f5e68a00700 1 --2- 172.21.15.18:0/1839591849 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5e64102c30 0x7f5e64113a00 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f5e5400cb10 tx=0x7f5e54007f70 comp rx=0 tx=0).stop 2024-04-12T21:23:28.539 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:23:28.543+0000 7f5e68a00700 1 -- 172.21.15.18:0/1839591849 shutdown_connections 2024-04-12T21:23:28.539 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:23:28.543+0000 7f5e68a00700 1 --2- 172.21.15.18:0/1839591849 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f5e4806ec00 0x7f5e480710b0 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:23:28.539 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:23:28.543+0000 7f5e68a00700 1 --2- 172.21.15.18:0/1839591849 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5e64102c30 0x7f5e64113a00 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:23:28.539 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:23:28.543+0000 7f5e68a00700 1 --2- 172.21.15.18:0/1839591849 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5e641035d0 0x7f5e64113f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:23:28.540 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:23:28.543+0000 7f5e68a00700 1 -- 172.21.15.18:0/1839591849 >> 172.21.15.18:0/1839591849 conn(0x7f5e640f8760 msgr2=0x7f5e640f9740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:23:28.540 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:23:28.543+0000 7f5e68a00700 1 -- 172.21.15.18:0/1839591849 shutdown_connections 2024-04-12T21:23:28.540 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:23:28.543+0000 7f5e68a00700 1 -- 172.21.15.18:0/1839591849 wait complete. 2024-04-12T21:23:28.695 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:23:28.699+0000 7f4064e00700 1 -- 172.21.15.18:0/1068877310 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4060105800 msgr2=0x7f40600fee70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:23:28.695 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:23:28.699+0000 7f4064e00700 1 --2- 172.21.15.18:0/1068877310 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4060105800 0x7f40600fee70 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f4048004940 tx=0x7f4048009e30 comp rx=0 tx=0).stop 2024-04-12T21:23:28.695 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:23:28.699+0000 7f4064e00700 1 -- 172.21.15.18:0/1068877310 shutdown_connections 2024-04-12T21:23:28.696 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:23:28.699+0000 7f4064e00700 1 --2- 172.21.15.18:0/1068877310 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4060105800 0x7f40600fee70 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:23:28.696 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:23:28.699+0000 7f4064e00700 1 --2- 172.21.15.18:0/1068877310 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4060104e60 0x7f4060105230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:23:28.696 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:23:28.699+0000 7f4064e00700 1 -- 172.21.15.18:0/1068877310 >> 172.21.15.18:0/1068877310 conn(0x7f40600fab10 msgr2=0x7f40600fcef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:23:28.696 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:23:28.699+0000 7f4064e00700 1 -- 172.21.15.18:0/1068877310 shutdown_connections 2024-04-12T21:23:28.696 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:23:28.699+0000 7f4064e00700 1 -- 172.21.15.18:0/1068877310 wait complete. 2024-04-12T21:23:28.697 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:23:28.700+0000 7f4064e00700 1 Processor -- start 2024-04-12T21:23:28.697 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:23:28.700+0000 7f4064e00700 1 -- start start 2024-04-12T21:23:28.697 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:23:28.700+0000 7f4064e00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4060104e60 0x7f4060115ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:23:28.697 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:23:28.701+0000 7f4064e00700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4060105800 0x7f40601160e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:23:28.697 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:23:28.701+0000 7f4064e00700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40601167c0 con 0x7f4060104e60 2024-04-12T21:23:28.698 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:23:28.701+0000 7f4064e00700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f406010fb40 con 0x7f4060105800 2024-04-12T21:23:28.698 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:23:28.701+0000 7f405f400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4060104e60 0x7f4060115ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:23:28.698 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:23:28.701+0000 7f405f400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4060104e60 0x7f4060115ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:52966/0 (socket says 172.21.15.18:52966) 2024-04-12T21:23:28.698 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:23:28.701+0000 7f405f400700 1 -- 172.21.15.18:0/878455065 learned_addr learned my addr 172.21.15.18:0/878455065 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:23:28.698 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:23:28.701+0000 7f4057e00700 1 --2- 172.21.15.18:0/878455065 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4060105800 0x7f40601160e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:23:28.698 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:23:28.701+0000 7f405f400700 1 -- 172.21.15.18:0/878455065 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4060105800 msgr2=0x7f40601160e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:23:28.698 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:23:28.701+0000 7f405f400700 1 --2- 172.21.15.18:0/878455065 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4060105800 0x7f40601160e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:23:28.698 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:23:28.701+0000 7f405f400700 1 -- 172.21.15.18:0/878455065 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4048005040 con 0x7f4060104e60 2024-04-12T21:23:28.698 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:23:28.701+0000 7f4057e00700 1 --2- 172.21.15.18:0/878455065 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4060105800 0x7f40601160e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-12T21:23:28.698 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:23:28.702+0000 7f405f400700 1 --2- 172.21.15.18:0/878455065 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4060104e60 0x7f4060115ba0 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f405000b790 tx=0x7f4050009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:23:28.699 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:23:28.702+0000 7f4057400700 1 -- 172.21.15.18:0/878455065 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f405000f770 con 0x7f4060104e60 2024-04-12T21:23:28.699 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:23:28.702+0000 7f4057400700 1 -- 172.21.15.18:0/878455065 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f405000fda0 con 0x7f4060104e60 2024-04-12T21:23:28.699 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:23:28.702+0000 7f4064e00700 1 -- 172.21.15.18:0/878455065 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f406010fe20 con 0x7f4060104e60 2024-04-12T21:23:28.699 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:23:28.702+0000 7f4057400700 1 -- 172.21.15.18:0/878455065 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f405000d310 con 0x7f4060104e60 2024-04-12T21:23:28.699 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:23:28.702+0000 7f4064e00700 1 -- 172.21.15.18:0/878455065 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f40601103e0 con 0x7f4060104e60 2024-04-12T21:23:28.699 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:23:28.703+0000 7f4064e00700 1 -- 172.21.15.18:0/878455065 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f40601005b0 con 0x7f4060104e60 2024-04-12T21:23:28.700 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:23:28.704+0000 7f4057400700 1 -- 172.21.15.18:0/878455065 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7f4050010420 con 0x7f4060104e60 2024-04-12T21:23:28.701 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:23:28.705+0000 7f4057400700 1 --2- 172.21.15.18:0/878455065 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f404406ec00 0x7f40440710b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:23:28.701 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:23:28.705+0000 7f4057400700 1 -- 172.21.15.18:0/878455065 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 6757+0+0 (secure 0 0 0) 0x7f405008e9a0 con 0x7f4060104e60 2024-04-12T21:23:28.701 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:23:28.706+0000 7f4057e00700 1 --2- 172.21.15.18:0/878455065 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f404406ec00 0x7f40440710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:23:28.702 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:23:28.706+0000 7f4057e00700 1 --2- 172.21.15.18:0/878455065 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f404406ec00 0x7f40440710b0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f4048009fd0 tx=0x7f4048009e30 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:23:28.706 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:23:28.710+0000 7f4057400700 1 -- 172.21.15.18:0/878455065 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f40500576a0 con 0x7f4060104e60 2024-04-12T21:23:28.849 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:23:28 smithi018 ceph-mon[112590]: pgmap v711: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:23:28.849 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:23:28 smithi018 ceph-mon[112590]: from='client.35260 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:23:29.019 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:23:29.023+0000 7f4064e00700 1 -- 172.21.15.18:0/878455065 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f40600fe610 con 0x7f4060104e60 2024-04-12T21:23:29.020 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:23:29.024+0000 7f4057400700 1 -- 172.21.15.18:0/878455065 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+837 (secure 0 0 0) 0x7f405005ad60 con 0x7f4060104e60 2024-04-12T21:23:29.020 INFO:teuthology.orchestra.run.smithi018.stdout:{ 2024-04-12T21:23:29.021 INFO:teuthology.orchestra.run.smithi018.stdout: "mon": { 2024-04-12T21:23:29.021 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 19.0.0-2879-g324baf8d (324baf8ddb91dea46cac50cb535c306115791355) squid (dev)": 2 2024-04-12T21:23:29.021 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-12T21:23:29.021 INFO:teuthology.orchestra.run.smithi018.stdout: "mgr": { 2024-04-12T21:23:29.021 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 19.0.0-2879-g324baf8d (324baf8ddb91dea46cac50cb535c306115791355) squid (dev)": 2 2024-04-12T21:23:29.021 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-12T21:23:29.021 INFO:teuthology.orchestra.run.smithi018.stdout: "osd": { 2024-04-12T21:23:29.021 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 19.0.0-2879-g324baf8d (324baf8ddb91dea46cac50cb535c306115791355) squid (dev)": 6 2024-04-12T21:23:29.021 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-12T21:23:29.021 INFO:teuthology.orchestra.run.smithi018.stdout: "mds": { 2024-04-12T21:23:29.021 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 17.2.7-827-gd3b5060a (d3b5060a17449edfae9385cdf792afec109fca5d) quincy (stable)": 1, 2024-04-12T21:23:29.021 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 19.0.0-2879-g324baf8d (324baf8ddb91dea46cac50cb535c306115791355) squid (dev)": 3 2024-04-12T21:23:29.021 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-12T21:23:29.022 INFO:teuthology.orchestra.run.smithi018.stdout: "overall": { 2024-04-12T21:23:29.022 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 17.2.7-827-gd3b5060a (d3b5060a17449edfae9385cdf792afec109fca5d) quincy (stable)": 1, 2024-04-12T21:23:29.022 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 19.0.0-2879-g324baf8d (324baf8ddb91dea46cac50cb535c306115791355) squid (dev)": 13 2024-04-12T21:23:29.022 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-04-12T21:23:29.022 INFO:teuthology.orchestra.run.smithi018.stdout:} 2024-04-12T21:23:29.024 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:23:29.027+0000 7f4064e00700 1 -- 172.21.15.18:0/878455065 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f404406ec00 msgr2=0x7f40440710b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:23:29.024 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:23:29.027+0000 7f4064e00700 1 --2- 172.21.15.18:0/878455065 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f404406ec00 0x7f40440710b0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f4048009fd0 tx=0x7f4048009e30 comp rx=0 tx=0).stop 2024-04-12T21:23:29.024 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:23:29.028+0000 7f4064e00700 1 -- 172.21.15.18:0/878455065 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4060104e60 msgr2=0x7f4060115ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:23:29.024 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:23:29.028+0000 7f4064e00700 1 --2- 172.21.15.18:0/878455065 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4060104e60 0x7f4060115ba0 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f405000b790 tx=0x7f4050009250 comp rx=0 tx=0).stop 2024-04-12T21:23:29.024 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:23:29.028+0000 7f4064e00700 1 -- 172.21.15.18:0/878455065 shutdown_connections 2024-04-12T21:23:29.024 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:23:29.028+0000 7f4064e00700 1 --2- 172.21.15.18:0/878455065 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f404406ec00 0x7f40440710b0 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:23:29.024 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:23:29.028+0000 7f4064e00700 1 --2- 172.21.15.18:0/878455065 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4060104e60 0x7f4060115ba0 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:23:29.024 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:23:29.028+0000 7f4064e00700 1 --2- 172.21.15.18:0/878455065 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4060105800 0x7f40601160e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:23:29.024 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:23:29.028+0000 7f4064e00700 1 -- 172.21.15.18:0/878455065 >> 172.21.15.18:0/878455065 conn(0x7f40600fab10 msgr2=0x7f40600fca30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:23:29.024 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:23:29.028+0000 7f4064e00700 1 -- 172.21.15.18:0/878455065 shutdown_connections 2024-04-12T21:23:29.024 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:23:29.028+0000 7f4064e00700 1 -- 172.21.15.18:0/878455065 wait complete. 2024-04-12T21:23:29.028 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:23:28 smithi186 ceph-mon[86379]: pgmap v711: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:23:29.028 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:23:28 smithi186 ceph-mon[86379]: from='client.35260 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:23:29.175 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:23:29.179+0000 7f00bec00700 1 -- 172.21.15.18:0/591888935 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f00b8072e00 msgr2=0x7f00b810d770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:23:29.176 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:23:29.179+0000 7f00bec00700 1 --2- 172.21.15.18:0/591888935 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f00b8072e00 0x7f00b810d770 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f00a0004940 tx=0x7f00a0009e30 comp rx=0 tx=0).stop 2024-04-12T21:23:29.176 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:23:29.179+0000 7f00bec00700 1 -- 172.21.15.18:0/591888935 shutdown_connections 2024-04-12T21:23:29.176 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:23:29.179+0000 7f00bec00700 1 --2- 172.21.15.18:0/591888935 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f00b8072e00 0x7f00b810d770 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:23:29.176 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:23:29.179+0000 7f00bec00700 1 --2- 172.21.15.18:0/591888935 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f00b80724f0 0x7f00b80728c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:23:29.176 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:23:29.179+0000 7f00bec00700 1 -- 172.21.15.18:0/591888935 >> 172.21.15.18:0/591888935 conn(0x7f00b80fabd0 msgr2=0x7f00b80fcfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:23:29.176 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:23:29.179+0000 7f00bec00700 1 -- 172.21.15.18:0/591888935 shutdown_connections 2024-04-12T21:23:29.176 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:23:29.179+0000 7f00bec00700 1 -- 172.21.15.18:0/591888935 wait complete. 2024-04-12T21:23:29.176 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:23:29.180+0000 7f00bec00700 1 Processor -- start 2024-04-12T21:23:29.178 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:23:29.180+0000 7f00bec00700 1 -- start start 2024-04-12T21:23:29.178 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:23:29.180+0000 7f00bec00700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f00b80724f0 0x7f00b811a120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:23:29.178 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:23:29.181+0000 7f00bec00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f00b8072e00 0x7f00b811a660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:23:29.178 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:23:29.181+0000 7f00bec00700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00b811ad40 con 0x7f00b8072e00 2024-04-12T21:23:29.178 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:23:29.181+0000 7f00bec00700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00b8114080 con 0x7f00b80724f0 2024-04-12T21:23:29.178 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:23:29.181+0000 7f00af400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f00b8072e00 0x7f00b811a660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:23:29.178 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:23:29.181+0000 7f00af400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f00b8072e00 0x7f00b811a660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:52984/0 (socket says 172.21.15.18:52984) 2024-04-12T21:23:29.178 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:23:29.181+0000 7f00af400700 1 -- 172.21.15.18:0/1188530946 learned_addr learned my addr 172.21.15.18:0/1188530946 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:23:29.178 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:23:29.181+0000 7f00b7e00700 1 --2- 172.21.15.18:0/1188530946 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f00b80724f0 0x7f00b811a120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:23:29.179 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:23:29.181+0000 7f00af400700 1 -- 172.21.15.18:0/1188530946 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f00b80724f0 msgr2=0x7f00b811a120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:23:29.179 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:23:29.181+0000 7f00af400700 1 --2- 172.21.15.18:0/1188530946 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f00b80724f0 0x7f00b811a120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:23:29.179 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:23:29.181+0000 7f00af400700 1 -- 172.21.15.18:0/1188530946 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00a0005040 con 0x7f00b8072e00 2024-04-12T21:23:29.179 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:23:29.182+0000 7f00af400700 1 --2- 172.21.15.18:0/1188530946 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f00b8072e00 0x7f00b811a660 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f00a0009180 tx=0x7f00a0009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:23:29.179 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:23:29.182+0000 7f00b4c00700 1 -- 172.21.15.18:0/1188530946 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f00a000e040 con 0x7f00b8072e00 2024-04-12T21:23:29.179 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:23:29.182+0000 7f00b4c00700 1 -- 172.21.15.18:0/1188530946 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f00a0013420 con 0x7f00b8072e00 2024-04-12T21:23:29.179 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:23:29.182+0000 7f00bec00700 1 -- 172.21.15.18:0/1188530946 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f00b81142a0 con 0x7f00b8072e00 2024-04-12T21:23:29.179 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:23:29.182+0000 7f00b4c00700 1 -- 172.21.15.18:0/1188530946 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f00a001c630 con 0x7f00b8072e00 2024-04-12T21:23:29.179 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:23:29.182+0000 7f00bec00700 1 -- 172.21.15.18:0/1188530946 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f00b8114860 con 0x7f00b8072e00 2024-04-12T21:23:29.180 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:23:29.183+0000 7f00bec00700 1 -- 172.21.15.18:0/1188530946 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f00b810a5b0 con 0x7f00b8072e00 2024-04-12T21:23:29.180 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:23:29.184+0000 7f00b4c00700 1 -- 172.21.15.18:0/1188530946 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7f00a0013590 con 0x7f00b8072e00 2024-04-12T21:23:29.181 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:23:29.185+0000 7f00b4c00700 1 --2- 172.21.15.18:0/1188530946 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f009c06ec10 0x7f009c0710c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:23:29.181 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:23:29.185+0000 7f00b4c00700 1 -- 172.21.15.18:0/1188530946 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 6757+0+0 (secure 0 0 0) 0x7f00a008fde0 con 0x7f00b8072e00 2024-04-12T21:23:29.181 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:23:29.185+0000 7f00b7e00700 1 --2- 172.21.15.18:0/1188530946 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f009c06ec10 0x7f009c0710c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:23:29.182 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:23:29.186+0000 7f00b7e00700 1 --2- 172.21.15.18:0/1188530946 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f009c06ec10 0x7f009c0710c0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f00a8006fd0 tx=0x7f00a8008040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:23:29.185 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:23:29.190+0000 7f00b4c00700 1 -- 172.21.15.18:0/1188530946 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f00a005c590 con 0x7f00b8072e00 2024-04-12T21:23:29.467 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:23:29.471+0000 7f00bec00700 1 -- 172.21.15.18:0/1188530946 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f00b8073c30 con 0x7f00b8072e00 2024-04-12T21:23:29.468 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:23:29.472+0000 7f00b4c00700 1 -- 172.21.15.18:0/1188530946 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 31 v31) v1 ==== 76+0+1873 (secure 0 0 0) 0x7f00a005c120 con 0x7f00b8072e00 2024-04-12T21:23:29.468 INFO:teuthology.orchestra.run.smithi018.stdout:e31 2024-04-12T21:23:29.468 INFO:teuthology.orchestra.run.smithi018.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-12T21:23:29.469 INFO:teuthology.orchestra.run.smithi018.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-12T21:23:29.469 INFO:teuthology.orchestra.run.smithi018.stdout:legacy client fscid: 1 2024-04-12T21:23:29.469 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-12T21:23:29.469 INFO:teuthology.orchestra.run.smithi018.stdout:Filesystem 'cephfs' (1) 2024-04-12T21:23:29.469 INFO:teuthology.orchestra.run.smithi018.stdout:fs_name cephfs 2024-04-12T21:23:29.469 INFO:teuthology.orchestra.run.smithi018.stdout:epoch 24 2024-04-12T21:23:29.469 INFO:teuthology.orchestra.run.smithi018.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-04-12T21:23:29.469 INFO:teuthology.orchestra.run.smithi018.stdout:created 2024-04-12T20:51:57.362850+0000 2024-04-12T21:23:29.469 INFO:teuthology.orchestra.run.smithi018.stdout:modified 2024-04-12T21:06:59.717190+0000 2024-04-12T21:23:29.469 INFO:teuthology.orchestra.run.smithi018.stdout:tableserver 0 2024-04-12T21:23:29.469 INFO:teuthology.orchestra.run.smithi018.stdout:root 0 2024-04-12T21:23:29.469 INFO:teuthology.orchestra.run.smithi018.stdout:session_timeout 60 2024-04-12T21:23:29.469 INFO:teuthology.orchestra.run.smithi018.stdout:session_autoclose 300 2024-04-12T21:23:29.469 INFO:teuthology.orchestra.run.smithi018.stdout:max_file_size 1099511627776 2024-04-12T21:23:29.470 INFO:teuthology.orchestra.run.smithi018.stdout:max_xattr_size 65536 2024-04-12T21:23:29.470 INFO:teuthology.orchestra.run.smithi018.stdout:required_client_features {} 2024-04-12T21:23:29.470 INFO:teuthology.orchestra.run.smithi018.stdout:last_failure 0 2024-04-12T21:23:29.470 INFO:teuthology.orchestra.run.smithi018.stdout:last_failure_osd_epoch 100 2024-04-12T21:23:29.470 INFO:teuthology.orchestra.run.smithi018.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-12T21:23:29.470 INFO:teuthology.orchestra.run.smithi018.stdout:max_mds 2 2024-04-12T21:23:29.470 INFO:teuthology.orchestra.run.smithi018.stdout:in 0,1 2024-04-12T21:23:29.470 INFO:teuthology.orchestra.run.smithi018.stdout:up {} 2024-04-12T21:23:29.470 INFO:teuthology.orchestra.run.smithi018.stdout:failed 0,1 2024-04-12T21:23:29.470 INFO:teuthology.orchestra.run.smithi018.stdout:damaged 2024-04-12T21:23:29.470 INFO:teuthology.orchestra.run.smithi018.stdout:stopped 2024-04-12T21:23:29.470 INFO:teuthology.orchestra.run.smithi018.stdout:data_pools [3] 2024-04-12T21:23:29.470 INFO:teuthology.orchestra.run.smithi018.stdout:metadata_pool 2 2024-04-12T21:23:29.470 INFO:teuthology.orchestra.run.smithi018.stdout:inline_data disabled 2024-04-12T21:23:29.471 INFO:teuthology.orchestra.run.smithi018.stdout:balancer 2024-04-12T21:23:29.471 INFO:teuthology.orchestra.run.smithi018.stdout:bal_rank_mask -1 2024-04-12T21:23:29.471 INFO:teuthology.orchestra.run.smithi018.stdout:standby_count_wanted 1 2024-04-12T21:23:29.471 INFO:teuthology.orchestra.run.smithi018.stdout:qdb_cluster leader: 0 members: 2024-04-12T21:23:29.471 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-12T21:23:29.471 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-12T21:23:29.471 INFO:teuthology.orchestra.run.smithi018.stdout:Standby daemons: 2024-04-12T21:23:29.471 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-12T21:23:29.471 INFO:teuthology.orchestra.run.smithi018.stdout:[mds.cephfs.smithi186.wjnafs{-1:24277} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.186:6826/438347207,v1:172.21.15.186:6827/438347207] compat {c=[1],r=[1],i=[7ff]}] 2024-04-12T21:23:29.471 INFO:teuthology.orchestra.run.smithi018.stdout:[mds.cephfs.smithi018.acqbsc{-1:34436} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.18:6828/755179764,v1:172.21.15.18:6829/755179764] compat {c=[1],r=[1],i=[1fff]}] 2024-04-12T21:23:29.471 INFO:teuthology.orchestra.run.smithi018.stdout:[mds.cephfs.smithi018.szjluo{-1:34468} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.18:6826/2407992653,v1:172.21.15.18:6827/2407992653] compat {c=[1],r=[1],i=[1fff]}] 2024-04-12T21:23:29.471 INFO:teuthology.orchestra.run.smithi018.stdout:[mds.cephfs.smithi186.jdllyn{-1:44595} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.186:6824/3523444202,v1:172.21.15.186:6825/3523444202] compat {c=[1],r=[1],i=[1fff]}] 2024-04-12T21:23:29.472 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:23:29.476+0000 7f00bec00700 1 -- 172.21.15.18:0/1188530946 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f009c06ec10 msgr2=0x7f009c0710c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:23:29.472 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:23:29.476+0000 7f00bec00700 1 --2- 172.21.15.18:0/1188530946 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f009c06ec10 0x7f009c0710c0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f00a8006fd0 tx=0x7f00a8008040 comp rx=0 tx=0).stop 2024-04-12T21:23:29.472 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:23:29.476+0000 7f00bec00700 1 -- 172.21.15.18:0/1188530946 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f00b8072e00 msgr2=0x7f00b811a660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:23:29.472 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:23:29.476+0000 7f00bec00700 1 --2- 172.21.15.18:0/1188530946 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f00b8072e00 0x7f00b811a660 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f00a0009180 tx=0x7f00a0009e30 comp rx=0 tx=0).stop 2024-04-12T21:23:29.472 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:23:29.476+0000 7f00bec00700 1 -- 172.21.15.18:0/1188530946 shutdown_connections 2024-04-12T21:23:29.472 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:23:29.476+0000 7f00bec00700 1 --2- 172.21.15.18:0/1188530946 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f009c06ec10 0x7f009c0710c0 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:23:29.472 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:23:29.476+0000 7f00bec00700 1 --2- 172.21.15.18:0/1188530946 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f00b80724f0 0x7f00b811a120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:23:29.473 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:23:29.476+0000 7f00bec00700 1 --2- 172.21.15.18:0/1188530946 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f00b8072e00 0x7f00b811a660 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:23:29.473 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:23:29.476+0000 7f00bec00700 1 -- 172.21.15.18:0/1188530946 >> 172.21.15.18:0/1188530946 conn(0x7f00b80fabd0 msgr2=0x7f00b8107730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:23:29.473 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:23:29.476+0000 7f00bec00700 1 -- 172.21.15.18:0/1188530946 shutdown_connections 2024-04-12T21:23:29.473 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:23:29.476+0000 7f00bec00700 1 -- 172.21.15.18:0/1188530946 wait complete. 2024-04-12T21:23:29.474 INFO:teuthology.orchestra.run.smithi018.stderr:dumped fsmap epoch 31 2024-04-12T21:23:29.622 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:23:29.626+0000 7f333d800700 1 -- 172.21.15.18:0/4196399613 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f33380fddf0 msgr2=0x7f33380fe1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:23:29.623 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:23:29.626+0000 7f333d800700 1 --2- 172.21.15.18:0/4196399613 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f33380fddf0 0x7f33380fe1c0 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f3328004940 tx=0x7f3328009e30 comp rx=0 tx=0).stop 2024-04-12T21:23:29.623 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:23:29.626+0000 7f333d800700 1 -- 172.21.15.18:0/4196399613 shutdown_connections 2024-04-12T21:23:29.623 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:23:29.626+0000 7f333d800700 1 --2- 172.21.15.18:0/4196399613 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f33380fe790 0x7f3338106dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:23:29.623 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:23:29.626+0000 7f333d800700 1 --2- 172.21.15.18:0/4196399613 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f33380fddf0 0x7f33380fe1c0 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:23:29.623 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:23:29.626+0000 7f333d800700 1 -- 172.21.15.18:0/4196399613 >> 172.21.15.18:0/4196399613 conn(0x7f33380f7860 msgr2=0x7f33380f9c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:23:29.623 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:23:29.627+0000 7f333d800700 1 -- 172.21.15.18:0/4196399613 shutdown_connections 2024-04-12T21:23:29.623 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:23:29.627+0000 7f333d800700 1 -- 172.21.15.18:0/4196399613 wait complete. 2024-04-12T21:23:29.623 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:23:29.628+0000 7f333d800700 1 Processor -- start 2024-04-12T21:23:29.624 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:23:29.628+0000 7f333d800700 1 -- start start 2024-04-12T21:23:29.624 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:23:29.628+0000 7f333d800700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f33380fddf0 0x7f33380fbc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:23:29.624 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:23:29.628+0000 7f333d800700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f33380fe790 0x7f33380fc160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:23:29.624 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:23:29.628+0000 7f333d800700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33380fc6f0 con 0x7f33380fe790 2024-04-12T21:23:29.625 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:23:29.628+0000 7f333d800700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33380fc830 con 0x7f33380fddf0 2024-04-12T21:23:29.625 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:23:29.629+0000 7f3336000700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f33380fe790 0x7f33380fc160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:23:29.625 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:23:29.629+0000 7f3336000700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f33380fe790 0x7f33380fc160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:52996/0 (socket says 172.21.15.18:52996) 2024-04-12T21:23:29.625 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:23:29.629+0000 7f3336000700 1 -- 172.21.15.18:0/1941319807 learned_addr learned my addr 172.21.15.18:0/1941319807 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:23:29.626 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:23:29.629+0000 7f3336000700 1 -- 172.21.15.18:0/1941319807 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f33380fddf0 msgr2=0x7f33380fbc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:23:29.626 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:23:29.629+0000 7f3336a00700 1 --2- 172.21.15.18:0/1941319807 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f33380fddf0 0x7f33380fbc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:23:29.626 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:23:29.629+0000 7f3336000700 1 --2- 172.21.15.18:0/1941319807 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f33380fddf0 0x7f33380fbc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:23:29.626 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:23:29.629+0000 7f3336000700 1 -- 172.21.15.18:0/1941319807 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3328005040 con 0x7f33380fe790 2024-04-12T21:23:29.626 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:23:29.629+0000 7f3336000700 1 --2- 172.21.15.18:0/1941319807 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f33380fe790 0x7f33380fc160 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f3324002a50 tx=0x7f33240046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:23:29.626 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:23:29.630+0000 7f332ea00700 1 -- 172.21.15.18:0/1941319807 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3324007cc0 con 0x7f33380fe790 2024-04-12T21:23:29.626 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:23:29.630+0000 7f332ea00700 1 -- 172.21.15.18:0/1941319807 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f332400f040 con 0x7f33380fe790 2024-04-12T21:23:29.626 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:23:29.630+0000 7f333d800700 1 -- 172.21.15.18:0/1941319807 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3338119e90 con 0x7f33380fe790 2024-04-12T21:23:29.626 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:23:29.630+0000 7f332ea00700 1 -- 172.21.15.18:0/1941319807 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f332400db70 con 0x7f33380fe790 2024-04-12T21:23:29.626 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:23:29.630+0000 7f333d800700 1 -- 172.21.15.18:0/1941319807 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f333811a290 con 0x7f33380fe790 2024-04-12T21:23:29.627 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:23:29.631+0000 7f333d800700 1 -- 172.21.15.18:0/1941319807 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3338066770 con 0x7f33380fe790 2024-04-12T21:23:29.628 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:23:29.632+0000 7f332ea00700 1 -- 172.21.15.18:0/1941319807 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7f332400dd10 con 0x7f33380fe790 2024-04-12T21:23:29.629 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:23:29.633+0000 7f332ea00700 1 --2- 172.21.15.18:0/1941319807 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f331c06ec10 0x7f331c0710c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:23:29.629 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:23:29.633+0000 7f332ea00700 1 -- 172.21.15.18:0/1941319807 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 6757+0+0 (secure 0 0 0) 0x7f332408e090 con 0x7f33380fe790 2024-04-12T21:23:29.629 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:23:29.633+0000 7f3336a00700 1 --2- 172.21.15.18:0/1941319807 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f331c06ec10 0x7f331c0710c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:23:29.630 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:23:29.634+0000 7f3336a00700 1 --2- 172.21.15.18:0/1941319807 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f331c06ec10 0x7f331c0710c0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f3328009fd0 tx=0x7f3328009e30 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:23:29.633 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:23:29.638+0000 7f332ea00700 1 -- 172.21.15.18:0/1941319807 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f3324056d10 con 0x7f33380fe790 2024-04-12T21:23:29.849 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:23:29 smithi018 ceph-mon[112590]: from='client.35264 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:23:29.849 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:23:29 smithi018 ceph-mon[112590]: from='client.35268 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:23:29.849 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:23:29 smithi018 ceph-mon[112590]: from='client.? 172.21.15.18:0/878455065' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T21:23:29.849 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:23:29 smithi018 ceph-mon[112590]: from='client.? 172.21.15.18:0/1188530946' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-12T21:23:29.879 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:23:29.883+0000 7f333d800700 1 -- 172.21.15.18:0/1941319807 --> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f333804bed0 con 0x7f331c06ec10 2024-04-12T21:23:29.881 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:23:29.885+0000 7f332ea00700 1 -- 172.21.15.18:0/1941319807 <== mgr.34102 v2:172.21.15.18:6800/3812891919 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f333804bed0 con 0x7f331c06ec10 2024-04-12T21:23:29.881 INFO:teuthology.orchestra.run.smithi018.stdout:{ 2024-04-12T21:23:29.881 INFO:teuthology.orchestra.run.smithi018.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:f4452def3be4d350f4a8e5feaf7ebb1511be9298fe68478ba7a2d6fbd7d69a63", 2024-04-12T21:23:29.881 INFO:teuthology.orchestra.run.smithi018.stdout: "in_progress": true, 2024-04-12T21:23:29.881 INFO:teuthology.orchestra.run.smithi018.stdout: "which": "Upgrading all daemon types on all hosts", 2024-04-12T21:23:29.882 INFO:teuthology.orchestra.run.smithi018.stdout: "services_complete": [ 2024-04-12T21:23:29.882 INFO:teuthology.orchestra.run.smithi018.stdout: "crash", 2024-04-12T21:23:29.882 INFO:teuthology.orchestra.run.smithi018.stdout: "osd", 2024-04-12T21:23:29.882 INFO:teuthology.orchestra.run.smithi018.stdout: "mon", 2024-04-12T21:23:29.882 INFO:teuthology.orchestra.run.smithi018.stdout: "mgr" 2024-04-12T21:23:29.882 INFO:teuthology.orchestra.run.smithi018.stdout: ], 2024-04-12T21:23:29.882 INFO:teuthology.orchestra.run.smithi018.stdout: "progress": "15/21 daemons upgraded", 2024-04-12T21:23:29.882 INFO:teuthology.orchestra.run.smithi018.stdout: "message": "Currently upgrading mds daemons", 2024-04-12T21:23:29.882 INFO:teuthology.orchestra.run.smithi018.stdout: "is_paused": false 2024-04-12T21:23:29.882 INFO:teuthology.orchestra.run.smithi018.stdout:} 2024-04-12T21:23:29.884 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:23:29.887+0000 7f333d800700 1 -- 172.21.15.18:0/1941319807 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f331c06ec10 msgr2=0x7f331c0710c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:23:29.884 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:23:29.887+0000 7f333d800700 1 --2- 172.21.15.18:0/1941319807 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f331c06ec10 0x7f331c0710c0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f3328009fd0 tx=0x7f3328009e30 comp rx=0 tx=0).stop 2024-04-12T21:23:29.884 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:23:29.887+0000 7f333d800700 1 -- 172.21.15.18:0/1941319807 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f33380fe790 msgr2=0x7f33380fc160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:23:29.884 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:23:29.887+0000 7f333d800700 1 --2- 172.21.15.18:0/1941319807 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f33380fe790 0x7f33380fc160 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f3324002a50 tx=0x7f33240046b0 comp rx=0 tx=0).stop 2024-04-12T21:23:29.884 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:23:29.888+0000 7f333d800700 1 -- 172.21.15.18:0/1941319807 shutdown_connections 2024-04-12T21:23:29.884 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:23:29.888+0000 7f333d800700 1 --2- 172.21.15.18:0/1941319807 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f331c06ec10 0x7f331c0710c0 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:23:29.884 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:23:29.888+0000 7f333d800700 1 --2- 172.21.15.18:0/1941319807 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f33380fddf0 0x7f33380fbc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:23:29.884 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:23:29.888+0000 7f333d800700 1 --2- 172.21.15.18:0/1941319807 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f33380fe790 0x7f33380fc160 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:23:29.884 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:23:29.888+0000 7f333d800700 1 -- 172.21.15.18:0/1941319807 >> 172.21.15.18:0/1941319807 conn(0x7f33380f7860 msgr2=0x7f33380f8ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:23:29.884 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:23:29.888+0000 7f333d800700 1 -- 172.21.15.18:0/1941319807 shutdown_connections 2024-04-12T21:23:29.884 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:23:29.888+0000 7f333d800700 1 -- 172.21.15.18:0/1941319807 wait complete. 2024-04-12T21:23:30.028 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:23:29 smithi186 ceph-mon[86379]: from='client.35264 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:23:30.029 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:23:29 smithi186 ceph-mon[86379]: from='client.35268 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:23:30.029 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:23:29 smithi186 ceph-mon[86379]: from='client.? 172.21.15.18:0/878455065' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T21:23:30.029 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:23:29 smithi186 ceph-mon[86379]: from='client.? 172.21.15.18:0/1188530946' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-12T21:23:30.043 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:23:30.046+0000 7fdf8c400700 1 -- 172.21.15.18:0/821140570 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fdf840fc620 msgr2=0x7fdf84100cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:23:30.043 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:23:30.046+0000 7fdf8c400700 1 --2- 172.21.15.18:0/821140570 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fdf840fc620 0x7fdf84100cd0 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7fdf70004940 tx=0x7fdf70009e30 comp rx=0 tx=0).stop 2024-04-12T21:23:30.043 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:23:30.046+0000 7fdf8c400700 1 -- 172.21.15.18:0/821140570 shutdown_connections 2024-04-12T21:23:30.044 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:23:30.046+0000 7fdf8c400700 1 --2- 172.21.15.18:0/821140570 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fdf840fc620 0x7fdf84100cd0 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:23:30.044 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:23:30.046+0000 7fdf8c400700 1 --2- 172.21.15.18:0/821140570 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdf840fbc80 0x7fdf840fc050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:23:30.044 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:23:30.046+0000 7fdf8c400700 1 -- 172.21.15.18:0/821140570 >> 172.21.15.18:0/821140570 conn(0x7fdf84072bc0 msgr2=0x7fdf84072fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:23:30.044 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:23:30.046+0000 7fdf8c400700 1 -- 172.21.15.18:0/821140570 shutdown_connections 2024-04-12T21:23:30.044 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:23:30.046+0000 7fdf8c400700 1 -- 172.21.15.18:0/821140570 wait complete. 2024-04-12T21:23:30.044 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:23:30.047+0000 7fdf8c400700 1 Processor -- start 2024-04-12T21:23:30.045 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:23:30.047+0000 7fdf8c400700 1 -- start start 2024-04-12T21:23:30.047 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:23:30.048+0000 7fdf8c400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fdf840fbc80 0x7fdf8411a1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:23:30.047 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:23:30.048+0000 7fdf8c400700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdf840fc620 0x7fdf8411a710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:23:30.047 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:23:30.048+0000 7fdf8c400700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf8411ad60 con 0x7fdf840fbc80 2024-04-12T21:23:30.048 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:23:30.048+0000 7fdf8c400700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf84114130 con 0x7fdf840fc620 2024-04-12T21:23:30.048 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:23:30.048+0000 7fdf89a00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fdf840fbc80 0x7fdf8411a1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:23:30.048 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:23:30.048+0000 7fdf89a00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fdf840fbc80 0x7fdf8411a1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:53020/0 (socket says 172.21.15.18:53020) 2024-04-12T21:23:30.048 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:23:30.048+0000 7fdf89a00700 1 -- 172.21.15.18:0/3163644408 learned_addr learned my addr 172.21.15.18:0/3163644408 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:23:30.048 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:23:30.048+0000 7fdf89000700 1 --2- 172.21.15.18:0/3163644408 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdf840fc620 0x7fdf8411a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:23:30.048 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:23:30.048+0000 7fdf89a00700 1 -- 172.21.15.18:0/3163644408 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdf840fc620 msgr2=0x7fdf8411a710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:23:30.049 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:23:30.048+0000 7fdf89a00700 1 --2- 172.21.15.18:0/3163644408 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdf840fc620 0x7fdf8411a710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:23:30.049 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:23:30.048+0000 7fdf89a00700 1 -- 172.21.15.18:0/3163644408 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf70005040 con 0x7fdf840fbc80 2024-04-12T21:23:30.049 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:23:30.049+0000 7fdf89a00700 1 --2- 172.21.15.18:0/3163644408 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fdf840fbc80 0x7fdf8411a1d0 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7fdf78002a50 tx=0x7fdf780046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:23:30.049 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:23:30.049+0000 7fdf81600700 1 -- 172.21.15.18:0/3163644408 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdf78007cc0 con 0x7fdf840fbc80 2024-04-12T21:23:30.049 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:23:30.049+0000 7fdf81600700 1 -- 172.21.15.18:0/3163644408 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fdf7800f040 con 0x7fdf840fbc80 2024-04-12T21:23:30.049 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:23:30.049+0000 7fdf8c400700 1 -- 172.21.15.18:0/3163644408 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdf84114410 con 0x7fdf840fbc80 2024-04-12T21:23:30.050 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:23:30.049+0000 7fdf81600700 1 -- 172.21.15.18:0/3163644408 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdf78009640 con 0x7fdf840fbc80 2024-04-12T21:23:30.050 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:23:30.049+0000 7fdf8c400700 1 -- 172.21.15.18:0/3163644408 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdf84114950 con 0x7fdf840fbc80 2024-04-12T21:23:30.050 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:23:30.050+0000 7fdf8c400700 1 -- 172.21.15.18:0/3163644408 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdf84065e30 con 0x7fdf840fbc80 2024-04-12T21:23:30.050 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:23:30.051+0000 7fdf81600700 1 -- 172.21.15.18:0/3163644408 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7fdf780097a0 con 0x7fdf840fbc80 2024-04-12T21:23:30.051 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:23:30.052+0000 7fdf81600700 1 --2- 172.21.15.18:0/3163644408 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fdf6c06ec10 0x7fdf6c0710c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:23:30.052 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:23:30.053+0000 7fdf81600700 1 -- 172.21.15.18:0/3163644408 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 6757+0+0 (secure 0 0 0) 0x7fdf7808e760 con 0x7fdf840fbc80 2024-04-12T21:23:30.052 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:23:30.053+0000 7fdf89000700 1 --2- 172.21.15.18:0/3163644408 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fdf6c06ec10 0x7fdf6c0710c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:23:30.052 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:23:30.053+0000 7fdf89000700 1 --2- 172.21.15.18:0/3163644408 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fdf6c06ec10 0x7fdf6c0710c0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fdf7000b3b0 tx=0x7fdf70009e30 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:23:30.054 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:23:30.058+0000 7fdf81600700 1 -- 172.21.15.18:0/3163644408 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7fdf7805af10 con 0x7fdf840fbc80 2024-04-12T21:23:30.381 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:23:30.385+0000 7fdf8c400700 1 -- 172.21.15.18:0/3163644408 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7fdf8404bed0 con 0x7fdf840fbc80 2024-04-12T21:23:30.382 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:23:30.386+0000 7fdf81600700 1 -- 172.21.15.18:0/3163644408 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+4303 (secure 0 0 0) 0x7fdf7805aaa0 con 0x7fdf840fbc80 2024-04-12T21:23:30.382 INFO:teuthology.orchestra.run.smithi018.stdout:HEALTH_ERR failed to probe daemons or devices; 1 filesystem is degraded; 1 filesystem is offline 2024-04-12T21:23:30.382 INFO:teuthology.orchestra.run.smithi018.stdout:[WRN] CEPHADM_REFRESH_FAILED: failed to probe daemons or devices 2024-04-12T21:23:30.382 INFO:teuthology.orchestra.run.smithi018.stdout: host smithi018 `cephadm ceph-volume` failed: cephadm exited with an error code: 1, stderr: Inferring config /var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/mon.smithi018/config 2024-04-12T21:23:30.382 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/podman: timeout after 895 seconds 2024-04-12T21:23:30.383 INFO:teuthology.orchestra.run.smithi018.stdout:Non-zero exit code 124 from /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:f4452def3be4d350f4a8e5feaf7ebb1511be9298fe68478ba7a2d6fbd7d69a63 -e NODE_NAME=smithi018 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968:/var/run/ceph:z -v /var/log/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968:/var/log/ceph:z -v /var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmpxsmwkkuh:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:f4452def3be4d350f4a8e5feaf7ebb1511be9298fe68478ba7a2d6fbd7d69a63 inventory --format=json-pretty --filter-for-batch 2024-04-12T21:23:30.383 INFO:teuthology.orchestra.run.smithi018.stdout:Traceback (most recent call last): 2024-04-12T21:23:30.383 INFO:teuthology.orchestra.run.smithi018.stdout: File "/usr/lib64/python3.9/runpy.py", line 197, in _run_module_as_main 2024-04-12T21:23:30.383 INFO:teuthology.orchestra.run.smithi018.stdout: return _run_code(code, main_globals, None, 2024-04-12T21:23:30.383 INFO:teuthology.orchestra.run.smithi018.stdout: File "/usr/lib64/python3.9/runpy.py", line 87, in _run_code 2024-04-12T21:23:30.383 INFO:teuthology.orchestra.run.smithi018.stdout: exec(code, run_globals) 2024-04-12T21:23:30.383 INFO:teuthology.orchestra.run.smithi018.stdout: File "/var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/cephadm.e44f3ed5e34e4ef52eedef233d1190bff004470d8e8005127b5312bff357fa7f/__main__.py", line 5618, in 2024-04-12T21:23:30.383 INFO:teuthology.orchestra.run.smithi018.stdout: File "/var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/cephadm.e44f3ed5e34e4ef52eedef233d1190bff004470d8e8005127b5312bff357fa7f/__main__.py", line 5606, in main 2024-04-12T21:23:30.383 INFO:teuthology.orchestra.run.smithi018.stdout: File "/var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/cephadm.e44f3ed5e34e4ef52eedef233d1190bff004470d8e8005127b5312bff357fa7f/__main__.py", line 408, in _infer_config 2024-04-12T21:23:30.383 INFO:teuthology.orchestra.run.smithi018.stdout: File "/var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/cephadm.e44f3ed5e34e4ef52eedef233d1190bff004470d8e8005127b5312bff357fa7f/__main__.py", line 323, in _infer_fsid 2024-04-12T21:23:30.383 INFO:teuthology.orchestra.run.smithi018.stdout: File "/var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/cephadm.e44f3ed5e34e4ef52eedef233d1190bff004470d8e8005127b5312bff357fa7f/__main__.py", line 436, in _infer_image 2024-04-12T21:23:30.383 INFO:teuthology.orchestra.run.smithi018.stdout: File "/var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/cephadm.e44f3ed5e34e4ef52eedef233d1190bff004470d8e8005127b5312bff357fa7f/__main__.py", line 310, in _validate_fsid 2024-04-12T21:23:30.383 INFO:teuthology.orchestra.run.smithi018.stdout: File "/var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/cephadm.e44f3ed5e34e4ef52eedef233d1190bff004470d8e8005127b5312bff357fa7f/__main__.py", line 3359, in command_ceph_volume 2024-04-12T21:23:30.383 INFO:teuthology.orchestra.run.smithi018.stdout: File "/var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/cephadm.e44f3ed5e34e4ef52eedef233d1190bff004470d8e8005127b5312bff357fa7f/cephadmlib/call_wrappers.py", line 310, in call_throws 2024-04-12T21:23:30.384 INFO:teuthology.orchestra.run.smithi018.stdout:RuntimeError: Failed command: /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:f4452def3be4d350f4a8e5feaf7ebb1511be9298fe68478ba7a2d6fbd7d69a63 -e NODE_NAME=smithi018 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968:/var/run/ceph:z -v /var/log/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968:/var/log/ceph:z -v /var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmpxsmwkkuh:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:f4452def3be4d350f4a8e5feaf7ebb1511be9298fe68478ba7a2d6fbd7d69a63 inventory --format=json-pretty --filter-for-batch 2024-04-12T21:23:30.384 INFO:teuthology.orchestra.run.smithi018.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-04-12T21:23:30.384 INFO:teuthology.orchestra.run.smithi018.stdout: fs cephfs is degraded 2024-04-12T21:23:30.384 INFO:teuthology.orchestra.run.smithi018.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-04-12T21:23:30.384 INFO:teuthology.orchestra.run.smithi018.stdout: fs cephfs is offline because no MDS is active for it. 2024-04-12T21:23:30.385 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:23:30.389+0000 7fdf8c400700 1 -- 172.21.15.18:0/3163644408 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fdf6c06ec10 msgr2=0x7fdf6c0710c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:23:30.385 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:23:30.389+0000 7fdf8c400700 1 --2- 172.21.15.18:0/3163644408 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fdf6c06ec10 0x7fdf6c0710c0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fdf7000b3b0 tx=0x7fdf70009e30 comp rx=0 tx=0).stop 2024-04-12T21:23:30.385 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:23:30.389+0000 7fdf8c400700 1 -- 172.21.15.18:0/3163644408 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fdf840fbc80 msgr2=0x7fdf8411a1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:23:30.385 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:23:30.389+0000 7fdf8c400700 1 --2- 172.21.15.18:0/3163644408 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fdf840fbc80 0x7fdf8411a1d0 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7fdf78002a50 tx=0x7fdf780046b0 comp rx=0 tx=0).stop 2024-04-12T21:23:30.385 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:23:30.389+0000 7fdf8c400700 1 -- 172.21.15.18:0/3163644408 shutdown_connections 2024-04-12T21:23:30.385 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:23:30.389+0000 7fdf8c400700 1 --2- 172.21.15.18:0/3163644408 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fdf6c06ec10 0x7fdf6c0710c0 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:23:30.385 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:23:30.389+0000 7fdf8c400700 1 --2- 172.21.15.18:0/3163644408 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fdf840fbc80 0x7fdf8411a1d0 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:23:30.386 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:23:30.389+0000 7fdf8c400700 1 --2- 172.21.15.18:0/3163644408 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdf840fc620 0x7fdf8411a710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:23:30.386 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:23:30.389+0000 7fdf8c400700 1 -- 172.21.15.18:0/3163644408 >> 172.21.15.18:0/3163644408 conn(0x7fdf84072bc0 msgr2=0x7fdf8410aab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:23:30.386 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:23:30.389+0000 7fdf8c400700 1 -- 172.21.15.18:0/3163644408 shutdown_connections 2024-04-12T21:23:30.386 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:23:30.389+0000 7fdf8c400700 1 -- 172.21.15.18:0/3163644408 wait complete. 2024-04-12T21:23:30.849 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:23:30 smithi018 ceph-mon[112590]: pgmap v712: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:23:30.849 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:23:30 smithi018 ceph-mon[112590]: from='client.? 172.21.15.18:0/3163644408' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-12T21:23:31.028 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:23:30 smithi186 ceph-mon[86379]: pgmap v712: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:23:31.028 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:23:30 smithi186 ceph-mon[86379]: from='client.? 172.21.15.18:0/3163644408' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-12T21:23:31.849 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:23:31 smithi018 ceph-mon[112590]: from='client.35280 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:23:32.028 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:23:31 smithi186 ceph-mon[86379]: from='client.35280 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:23:32.849 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:23:32 smithi018 ceph-mon[112590]: pgmap v713: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:23:33.028 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:23:32 smithi186 ceph-mon[86379]: pgmap v713: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:23:34.849 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:23:34 smithi018 ceph-mon[112590]: pgmap v714: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:23:35.028 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:23:34 smithi186 ceph-mon[86379]: pgmap v714: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:23:36.849 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:23:36 smithi018 ceph-mon[112590]: pgmap v715: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:23:37.028 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:23:36 smithi186 ceph-mon[86379]: pgmap v715: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:23:38.849 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:23:38 smithi018 ceph-mon[112590]: pgmap v716: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:23:39.028 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:23:38 smithi186 ceph-mon[86379]: pgmap v716: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:23:41.028 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:23:40 smithi186 ceph-mon[86379]: pgmap v717: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:23:41.029 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:23:40 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T21:23:41.099 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:23:40 smithi018 ceph-mon[112590]: pgmap v717: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:23:41.099 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:23:40 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T21:23:42.778 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:23:42 smithi186 ceph-mon[86379]: pgmap v718: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:23:42.902 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:23:42 smithi018 ceph-mon[112590]: pgmap v718: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:23:45.028 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:23:44 smithi186 ceph-mon[86379]: pgmap v719: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:23:45.099 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:23:44 smithi018 ceph-mon[112590]: pgmap v719: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:23:47.028 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:23:46 smithi186 ceph-mon[86379]: pgmap v720: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:23:47.099 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:23:46 smithi018 ceph-mon[112590]: pgmap v720: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:23:49.028 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:23:48 smithi186 ceph-mon[86379]: pgmap v721: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:23:49.099 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:23:48 smithi018 ceph-mon[112590]: pgmap v721: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:23:51.028 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:23:50 smithi186 ceph-mon[86379]: pgmap v722: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:23:51.099 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:23:50 smithi018 ceph-mon[112590]: pgmap v722: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:23:53.028 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:23:52 smithi186 ceph-mon[86379]: pgmap v723: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:23:53.099 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:23:52 smithi018 ceph-mon[112590]: pgmap v723: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:23:55.028 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:23:54 smithi186 ceph-mon[86379]: pgmap v724: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:23:55.099 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:23:54 smithi018 ceph-mon[112590]: pgmap v724: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:23:56.028 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:23:55 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T21:23:56.099 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:23:55 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T21:23:57.028 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:23:56 smithi186 ceph-mon[86379]: pgmap v725: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:23:57.099 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:23:56 smithi018 ceph-mon[112590]: pgmap v725: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:23:59.028 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:23:58 smithi186 ceph-mon[86379]: pgmap v726: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:23:59.099 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:23:58 smithi018 ceph-mon[112590]: pgmap v726: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:24:00.563 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:00.565+0000 7f96ff800700 1 -- 172.21.15.18:0/2908004284 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f96f81059c0 msgr2=0x7f96f80ff030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:24:00.563 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:00.565+0000 7f96ff800700 1 --2- 172.21.15.18:0/2908004284 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f96f81059c0 0x7f96f80ff030 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f96e4004940 tx=0x7f96e4009e30 comp rx=0 tx=0).stop 2024-04-12T21:24:00.563 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:00.566+0000 7f96ff800700 1 -- 172.21.15.18:0/2908004284 shutdown_connections 2024-04-12T21:24:00.563 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:00.566+0000 7f96ff800700 1 --2- 172.21.15.18:0/2908004284 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f96f81059c0 0x7f96f80ff030 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:24:00.564 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:00.566+0000 7f96ff800700 1 --2- 172.21.15.18:0/2908004284 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f96f8105020 0x7f96f81053f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:24:00.564 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:00.566+0000 7f96ff800700 1 -- 172.21.15.18:0/2908004284 >> 172.21.15.18:0/2908004284 conn(0x7f96f80fab70 msgr2=0x7f96f80fcf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:24:00.564 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:00.566+0000 7f96ff800700 1 -- 172.21.15.18:0/2908004284 shutdown_connections 2024-04-12T21:24:00.564 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:00.566+0000 7f96ff800700 1 -- 172.21.15.18:0/2908004284 wait complete. 2024-04-12T21:24:00.564 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:00.567+0000 7f96ff800700 1 Processor -- start 2024-04-12T21:24:00.566 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:00.567+0000 7f96ff800700 1 -- start start 2024-04-12T21:24:00.566 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:00.567+0000 7f96ff800700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f96f8105020 0x7f96f8115dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:24:00.566 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:00.567+0000 7f96ff800700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f96f81059c0 0x7f96f8116300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:24:00.566 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:00.567+0000 7f96ff800700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96f8116950 con 0x7f96f8105020 2024-04-12T21:24:00.566 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:00.567+0000 7f96ff800700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96f810fd20 con 0x7f96f81059c0 2024-04-12T21:24:00.566 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:00.567+0000 7f96fce00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f96f8105020 0x7f96f8115dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:24:00.566 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:00.568+0000 7f96fce00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f96f8105020 0x7f96f8115dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:51636/0 (socket says 172.21.15.18:51636) 2024-04-12T21:24:00.566 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:00.568+0000 7f96fce00700 1 -- 172.21.15.18:0/2233517232 learned_addr learned my addr 172.21.15.18:0/2233517232 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:24:00.566 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:00.568+0000 7f96f7e00700 1 --2- 172.21.15.18:0/2233517232 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f96f81059c0 0x7f96f8116300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:24:00.567 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:00.568+0000 7f96fce00700 1 -- 172.21.15.18:0/2233517232 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f96f81059c0 msgr2=0x7f96f8116300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:24:00.567 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:00.568+0000 7f96fce00700 1 --2- 172.21.15.18:0/2233517232 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f96f81059c0 0x7f96f8116300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:24:00.567 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:00.568+0000 7f96fce00700 1 -- 172.21.15.18:0/2233517232 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96e4005040 con 0x7f96f8105020 2024-04-12T21:24:00.567 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:00.568+0000 7f96fce00700 1 --2- 172.21.15.18:0/2233517232 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f96f8105020 0x7f96f8115dc0 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f96ec00c8c0 tx=0x7f96ec0044c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:24:00.567 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:00.569+0000 7f96f4c00700 1 -- 172.21.15.18:0/2233517232 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f96ec00a980 con 0x7f96f8105020 2024-04-12T21:24:00.567 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:00.569+0000 7f96f4c00700 1 -- 172.21.15.18:0/2233517232 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f96ec00ce60 con 0x7f96f8105020 2024-04-12T21:24:00.567 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:00.569+0000 7f96ff800700 1 -- 172.21.15.18:0/2233517232 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f96f8110000 con 0x7f96f8105020 2024-04-12T21:24:00.567 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:00.569+0000 7f96f4c00700 1 -- 172.21.15.18:0/2233517232 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f96ec0115e0 con 0x7f96f8105020 2024-04-12T21:24:00.567 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:00.569+0000 7f96ff800700 1 -- 172.21.15.18:0/2233517232 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f96f81105c0 con 0x7f96f8105020 2024-04-12T21:24:00.567 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:00.570+0000 7f96ff800700 1 -- 172.21.15.18:0/2233517232 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f96f8100770 con 0x7f96f8105020 2024-04-12T21:24:00.568 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:00.571+0000 7f96f4c00700 1 -- 172.21.15.18:0/2233517232 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7f96ec012400 con 0x7f96f8105020 2024-04-12T21:24:00.568 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:00.572+0000 7f96f4c00700 1 --2- 172.21.15.18:0/2233517232 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f96e006ec10 0x7f96e00710c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:24:00.568 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:00.572+0000 7f96f4c00700 1 -- 172.21.15.18:0/2233517232 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 6757+0+0 (secure 0 0 0) 0x7f96ec08eb30 con 0x7f96f8105020 2024-04-12T21:24:00.568 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:00.573+0000 7f96f7e00700 1 --2- 172.21.15.18:0/2233517232 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f96e006ec10 0x7f96e00710c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:24:00.569 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:00.573+0000 7f96f7e00700 1 --2- 172.21.15.18:0/2233517232 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f96e006ec10 0x7f96e00710c0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f96e400b3b0 tx=0x7f96e4009e30 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:24:00.573 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:00.578+0000 7f96f4c00700 1 -- 172.21.15.18:0/2233517232 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f96ec05b2e0 con 0x7f96f8105020 2024-04-12T21:24:00.828 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:00.831+0000 7f96ff800700 1 -- 172.21.15.18:0/2233517232 --> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f96f8111410 con 0x7f96e006ec10 2024-04-12T21:24:00.828 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:24:00 smithi018 ceph-mon[112590]: pgmap v727: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:24:00.830 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:00.834+0000 7f96f4c00700 1 -- 172.21.15.18:0/2233517232 <== mgr.34102 v2:172.21.15.18:6800/3812891919 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f96f8111410 con 0x7f96e006ec10 2024-04-12T21:24:00.834 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:00.837+0000 7f96ff800700 1 -- 172.21.15.18:0/2233517232 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f96e006ec10 msgr2=0x7f96e00710c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:24:00.834 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:00.837+0000 7f96ff800700 1 --2- 172.21.15.18:0/2233517232 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f96e006ec10 0x7f96e00710c0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f96e400b3b0 tx=0x7f96e4009e30 comp rx=0 tx=0).stop 2024-04-12T21:24:00.834 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:00.837+0000 7f96ff800700 1 -- 172.21.15.18:0/2233517232 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f96f8105020 msgr2=0x7f96f8115dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:24:00.834 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:00.837+0000 7f96ff800700 1 --2- 172.21.15.18:0/2233517232 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f96f8105020 0x7f96f8115dc0 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f96ec00c8c0 tx=0x7f96ec0044c0 comp rx=0 tx=0).stop 2024-04-12T21:24:00.834 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:00.837+0000 7f96ff800700 1 -- 172.21.15.18:0/2233517232 shutdown_connections 2024-04-12T21:24:00.834 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:00.837+0000 7f96ff800700 1 --2- 172.21.15.18:0/2233517232 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f96e006ec10 0x7f96e00710c0 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:24:00.834 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:00.837+0000 7f96ff800700 1 --2- 172.21.15.18:0/2233517232 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f96f8105020 0x7f96f8115dc0 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:24:00.834 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:00.837+0000 7f96ff800700 1 --2- 172.21.15.18:0/2233517232 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f96f81059c0 0x7f96f8116300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:24:00.834 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:00.837+0000 7f96ff800700 1 -- 172.21.15.18:0/2233517232 >> 172.21.15.18:0/2233517232 conn(0x7f96f80fab70 msgr2=0x7f96f80fbbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:24:00.835 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:00.838+0000 7f96ff800700 1 -- 172.21.15.18:0/2233517232 shutdown_connections 2024-04-12T21:24:00.835 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:00.838+0000 7f96ff800700 1 -- 172.21.15.18:0/2233517232 wait complete. 2024-04-12T21:24:00.851 INFO:teuthology.orchestra.run.smithi018.stdout:true 2024-04-12T21:24:00.999 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:01.002+0000 7ff1cbe00700 1 -- 172.21.15.18:0/2767070042 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff1cc109c40 msgr2=0x7ff1cc1032b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:24:00.999 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:01.002+0000 7ff1cbe00700 1 --2- 172.21.15.18:0/2767070042 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff1cc109c40 0x7ff1cc1032b0 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7ff1b8004940 tx=0x7ff1b8009e30 comp rx=0 tx=0).stop 2024-04-12T21:24:00.999 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:01.003+0000 7ff1cbe00700 1 -- 172.21.15.18:0/2767070042 shutdown_connections 2024-04-12T21:24:00.999 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:01.003+0000 7ff1cbe00700 1 --2- 172.21.15.18:0/2767070042 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff1cc109c40 0x7ff1cc1032b0 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:24:01.000 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:01.003+0000 7ff1cbe00700 1 --2- 172.21.15.18:0/2767070042 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff1cc1092a0 0x7ff1cc109670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:24:01.000 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:01.003+0000 7ff1cbe00700 1 -- 172.21.15.18:0/2767070042 >> 172.21.15.18:0/2767070042 conn(0x7ff1cc072a30 msgr2=0x7ff1cc074e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:24:01.000 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:01.003+0000 7ff1cbe00700 1 -- 172.21.15.18:0/2767070042 shutdown_connections 2024-04-12T21:24:01.000 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:01.003+0000 7ff1cbe00700 1 -- 172.21.15.18:0/2767070042 wait complete. 2024-04-12T21:24:01.002 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:01.004+0000 7ff1cbe00700 1 Processor -- start 2024-04-12T21:24:01.002 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:01.004+0000 7ff1cbe00700 1 -- start start 2024-04-12T21:24:01.002 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:01.004+0000 7ff1cbe00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff1cc1092a0 0x7ff1cc119fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:24:01.002 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:01.004+0000 7ff1cbe00700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff1cc109c40 0x7ff1cc11a500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:24:01.002 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:01.004+0000 7ff1cbe00700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1cc11ab50 con 0x7ff1cc1092a0 2024-04-12T21:24:01.003 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:01.004+0000 7ff1cbe00700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1cc113f20 con 0x7ff1cc109c40 2024-04-12T21:24:01.003 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:01.004+0000 7ff1caa00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff1cc1092a0 0x7ff1cc119fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:24:01.003 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:01.005+0000 7ff1caa00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff1cc1092a0 0x7ff1cc119fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:52708/0 (socket says 172.21.15.18:52708) 2024-04-12T21:24:01.003 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:01.005+0000 7ff1caa00700 1 -- 172.21.15.18:0/2288317568 learned_addr learned my addr 172.21.15.18:0/2288317568 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:24:01.003 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:01.005+0000 7ff1ca000700 1 --2- 172.21.15.18:0/2288317568 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff1cc109c40 0x7ff1cc11a500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:24:01.003 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:01.005+0000 7ff1caa00700 1 -- 172.21.15.18:0/2288317568 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff1cc109c40 msgr2=0x7ff1cc11a500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:24:01.003 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:01.005+0000 7ff1caa00700 1 --2- 172.21.15.18:0/2288317568 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff1cc109c40 0x7ff1cc11a500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:24:01.003 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:01.005+0000 7ff1caa00700 1 -- 172.21.15.18:0/2288317568 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1b8005040 con 0x7ff1cc1092a0 2024-04-12T21:24:01.003 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:01.005+0000 7ff1caa00700 1 --2- 172.21.15.18:0/2288317568 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff1cc1092a0 0x7ff1cc119fc0 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7ff1bc00c830 tx=0x7ff1bc004470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:24:01.003 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:01.006+0000 7ff1c2a00700 1 -- 172.21.15.18:0/2288317568 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff1bc00a760 con 0x7ff1cc1092a0 2024-04-12T21:24:01.003 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:01.006+0000 7ff1c2a00700 1 -- 172.21.15.18:0/2288317568 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff1bc00ad90 con 0x7ff1cc1092a0 2024-04-12T21:24:01.004 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:01.006+0000 7ff1c2a00700 1 -- 172.21.15.18:0/2288317568 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff1bc0113a0 con 0x7ff1cc1092a0 2024-04-12T21:24:01.004 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:01.006+0000 7ff1cbe00700 1 -- 172.21.15.18:0/2288317568 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff1cc114200 con 0x7ff1cc1092a0 2024-04-12T21:24:01.004 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:01.006+0000 7ff1cbe00700 1 -- 172.21.15.18:0/2288317568 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff1cc1147c0 con 0x7ff1cc1092a0 2024-04-12T21:24:01.004 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:01.007+0000 7ff1cbe00700 1 -- 172.21.15.18:0/2288317568 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff1cc1049f0 con 0x7ff1cc1092a0 2024-04-12T21:24:01.004 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:01.008+0000 7ff1c2a00700 1 -- 172.21.15.18:0/2288317568 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7ff1bc012400 con 0x7ff1cc1092a0 2024-04-12T21:24:01.005 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:01.009+0000 7ff1c2a00700 1 --2- 172.21.15.18:0/2288317568 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7ff1b006ec00 0x7ff1b00710b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:24:01.005 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:01.009+0000 7ff1c2a00700 1 -- 172.21.15.18:0/2288317568 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 6757+0+0 (secure 0 0 0) 0x7ff1bc08eaa0 con 0x7ff1cc1092a0 2024-04-12T21:24:01.005 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:01.009+0000 7ff1ca000700 1 --2- 172.21.15.18:0/2288317568 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7ff1b006ec00 0x7ff1b00710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:24:01.006 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:01.010+0000 7ff1ca000700 1 --2- 172.21.15.18:0/2288317568 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7ff1b006ec00 0x7ff1b00710b0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7ff1b800b3b0 tx=0x7ff1b8009e30 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:24:01.011 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:01.014+0000 7ff1c2a00700 1 -- 172.21.15.18:0/2288317568 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7ff1bc05b250 con 0x7ff1cc1092a0 2024-04-12T21:24:01.028 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:24:00 smithi186 ceph-mon[86379]: pgmap v727: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:24:01.266 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:01.269+0000 7ff1cbe00700 1 -- 172.21.15.18:0/2288317568 --> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7ff1cc04bed0 con 0x7ff1b006ec00 2024-04-12T21:24:01.268 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:01.272+0000 7ff1c2a00700 1 -- 172.21.15.18:0/2288317568 <== mgr.34102 v2:172.21.15.18:6800/3812891919 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7ff1cc04bed0 con 0x7ff1b006ec00 2024-04-12T21:24:01.272 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:01.275+0000 7ff1cbe00700 1 -- 172.21.15.18:0/2288317568 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7ff1b006ec00 msgr2=0x7ff1b00710b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:24:01.272 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:01.275+0000 7ff1cbe00700 1 --2- 172.21.15.18:0/2288317568 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7ff1b006ec00 0x7ff1b00710b0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7ff1b800b3b0 tx=0x7ff1b8009e30 comp rx=0 tx=0).stop 2024-04-12T21:24:01.272 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:01.275+0000 7ff1cbe00700 1 -- 172.21.15.18:0/2288317568 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff1cc1092a0 msgr2=0x7ff1cc119fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:24:01.272 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:01.275+0000 7ff1cbe00700 1 --2- 172.21.15.18:0/2288317568 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff1cc1092a0 0x7ff1cc119fc0 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7ff1bc00c830 tx=0x7ff1bc004470 comp rx=0 tx=0).stop 2024-04-12T21:24:01.272 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:01.275+0000 7ff1cbe00700 1 -- 172.21.15.18:0/2288317568 shutdown_connections 2024-04-12T21:24:01.272 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:01.276+0000 7ff1cbe00700 1 --2- 172.21.15.18:0/2288317568 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7ff1b006ec00 0x7ff1b00710b0 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:24:01.272 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:01.276+0000 7ff1cbe00700 1 --2- 172.21.15.18:0/2288317568 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff1cc1092a0 0x7ff1cc119fc0 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:24:01.273 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:01.276+0000 7ff1cbe00700 1 --2- 172.21.15.18:0/2288317568 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff1cc109c40 0x7ff1cc11a500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:24:01.273 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:01.276+0000 7ff1cbe00700 1 -- 172.21.15.18:0/2288317568 >> 172.21.15.18:0/2288317568 conn(0x7ff1cc072a30 msgr2=0x7ff1cc074910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:24:01.273 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:01.276+0000 7ff1cbe00700 1 -- 172.21.15.18:0/2288317568 shutdown_connections 2024-04-12T21:24:01.273 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:01.276+0000 7ff1cbe00700 1 -- 172.21.15.18:0/2288317568 wait complete. 2024-04-12T21:24:01.435 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:01.439+0000 7f7166800700 1 -- 172.21.15.18:0/1145595549 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f71600fb4a0 msgr2=0x7f71600fb910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:24:01.436 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:01.439+0000 7f7166800700 1 --2- 172.21.15.18:0/1145595549 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f71600fb4a0 0x7f71600fb910 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f714c004940 tx=0x7f714c009e30 comp rx=0 tx=0).stop 2024-04-12T21:24:01.436 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:01.439+0000 7f7166800700 1 -- 172.21.15.18:0/1145595549 shutdown_connections 2024-04-12T21:24:01.436 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:01.439+0000 7f7166800700 1 --2- 172.21.15.18:0/1145595549 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f71600fb4a0 0x7f71600fb910 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:24:01.436 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:01.439+0000 7f7166800700 1 --2- 172.21.15.18:0/1145595549 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f71601014f0 0x7f71601018c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:24:01.436 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:01.439+0000 7f7166800700 1 -- 172.21.15.18:0/1145595549 >> 172.21.15.18:0/1145595549 conn(0x7f71600714c0 msgr2=0x7f71600718c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:24:01.436 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:01.439+0000 7f7166800700 1 -- 172.21.15.18:0/1145595549 shutdown_connections 2024-04-12T21:24:01.436 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:01.439+0000 7f7166800700 1 -- 172.21.15.18:0/1145595549 wait complete. 2024-04-12T21:24:01.437 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:01.440+0000 7f7166800700 1 Processor -- start 2024-04-12T21:24:01.437 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:01.440+0000 7f7166800700 1 -- start start 2024-04-12T21:24:01.437 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:01.441+0000 7f7166800700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f71600fb4a0 0x7f7160115d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:24:01.437 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:01.441+0000 7f7166800700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f71601014f0 0x7f71601162d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:24:01.437 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:01.441+0000 7f7166800700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7160116920 con 0x7f71601014f0 2024-04-12T21:24:01.437 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:01.441+0000 7f7166800700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f716010fd40 con 0x7f71600fb4a0 2024-04-12T21:24:01.437 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:01.441+0000 7f715ea00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f71601014f0 0x7f71601162d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:24:01.438 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:01.441+0000 7f715ea00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f71601014f0 0x7f71601162d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:52726/0 (socket says 172.21.15.18:52726) 2024-04-12T21:24:01.438 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:01.441+0000 7f715ea00700 1 -- 172.21.15.18:0/2377034788 learned_addr learned my addr 172.21.15.18:0/2377034788 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:24:01.438 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:01.441+0000 7f715f400700 1 --2- 172.21.15.18:0/2377034788 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f71600fb4a0 0x7f7160115d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:24:01.438 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:01.441+0000 7f715ea00700 1 -- 172.21.15.18:0/2377034788 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f71600fb4a0 msgr2=0x7f7160115d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:24:01.439 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:01.441+0000 7f715ea00700 1 --2- 172.21.15.18:0/2377034788 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f71600fb4a0 0x7f7160115d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:24:01.439 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:01.441+0000 7f715ea00700 1 -- 172.21.15.18:0/2377034788 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f714c005040 con 0x7f71601014f0 2024-04-12T21:24:01.439 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:01.442+0000 7f715ea00700 1 --2- 172.21.15.18:0/2377034788 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f71601014f0 0x7f71601162d0 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f714c009180 tx=0x7f714c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:24:01.439 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:01.442+0000 7f7157400700 1 -- 172.21.15.18:0/2377034788 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f714c00e040 con 0x7f71601014f0 2024-04-12T21:24:01.439 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:01.442+0000 7f7166800700 1 -- 172.21.15.18:0/2377034788 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f716010ffc0 con 0x7f71601014f0 2024-04-12T21:24:01.439 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:01.442+0000 7f7157400700 1 -- 172.21.15.18:0/2377034788 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f714c00bbc0 con 0x7f71601014f0 2024-04-12T21:24:01.439 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:01.442+0000 7f7157400700 1 -- 172.21.15.18:0/2377034788 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f714c01c630 con 0x7f71601014f0 2024-04-12T21:24:01.439 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:01.442+0000 7f7166800700 1 -- 172.21.15.18:0/2377034788 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7160110580 con 0x7f71601014f0 2024-04-12T21:24:01.440 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:01.444+0000 7f7166800700 1 -- 172.21.15.18:0/2377034788 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f71600fcbd0 con 0x7f71601014f0 2024-04-12T21:24:01.441 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:01.445+0000 7f7157400700 1 -- 172.21.15.18:0/2377034788 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7f714c00bd30 con 0x7f71601014f0 2024-04-12T21:24:01.442 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:01.446+0000 7f7157400700 1 --2- 172.21.15.18:0/2377034788 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f714406ec10 0x7f71440710c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:24:01.442 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:01.446+0000 7f7157400700 1 -- 172.21.15.18:0/2377034788 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 6757+0+0 (secure 0 0 0) 0x7f714c08fe30 con 0x7f71601014f0 2024-04-12T21:24:01.442 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:01.446+0000 7f715f400700 1 --2- 172.21.15.18:0/2377034788 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f714406ec10 0x7f71440710c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:24:01.443 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:01.447+0000 7f715f400700 1 --2- 172.21.15.18:0/2377034788 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f714406ec10 0x7f71440710c0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f7150007fd0 tx=0x7f7150009040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:24:01.447 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:01.451+0000 7f7157400700 1 -- 172.21.15.18:0/2377034788 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f714c05c5e0 con 0x7f71601014f0 2024-04-12T21:24:01.693 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:01.696+0000 7f7166800700 1 -- 172.21.15.18:0/2377034788 --> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f71601065e0 con 0x7f714406ec10 2024-04-12T21:24:01.725 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:01.728+0000 7f7157400700 1 -- 172.21.15.18:0/2377034788 <== mgr.34102 v2:172.21.15.18:6800/3812891919 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7f71601065e0 con 0x7f714406ec10 2024-04-12T21:24:01.725 INFO:teuthology.orchestra.run.smithi018.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-12T21:24:01.725 INFO:teuthology.orchestra.run.smithi018.stdout:alertmanager.smithi018 smithi018 *:9093,9094 running (24m) 56s ago 39m 15.5M - 0.25.0 c8568f914cd2 c332090068e8 2024-04-12T21:24:01.725 INFO:teuthology.orchestra.run.smithi018.stdout:crash.smithi018 smithi018 running (20m) 56s ago 39m 6665k - 19.0.0-2879-g324baf8d 8717720dc5e5 b7f80d1f7982 2024-04-12T21:24:01.725 INFO:teuthology.orchestra.run.smithi018.stdout:crash.smithi186 smithi186 running (20m) 59s ago 37m 6660k - 19.0.0-2879-g324baf8d 8717720dc5e5 a4f55c81b623 2024-04-12T21:24:01.725 INFO:teuthology.orchestra.run.smithi018.stdout:grafana.smithi018 smithi018 *:3000 running (24m) 56s ago 38m 74.8M - 9.4.12 2bacad6d85d8 b08e3bf176dd 2024-04-12T21:24:01.726 INFO:teuthology.orchestra.run.smithi018.stdout:mds.cephfs.smithi018.acqbsc smithi018 running (16m) 56s ago 31m 16.7M - 19.0.0-2879-g324baf8d 8717720dc5e5 021f99ab24ae 2024-04-12T21:24:01.726 INFO:teuthology.orchestra.run.smithi018.stdout:mds.cephfs.smithi018.szjluo smithi018 running (16m) 56s ago 31m 16.5M - 19.0.0-2879-g324baf8d 8717720dc5e5 0003117479d4 2024-04-12T21:24:01.726 INFO:teuthology.orchestra.run.smithi018.stdout:mds.cephfs.smithi186.jdllyn smithi186 running (65s) 59s ago 31m 11.0M - 19.0.0-2879-g324baf8d 8717720dc5e5 21b1f46eef23 2024-04-12T21:24:01.726 INFO:teuthology.orchestra.run.smithi018.stdout:mds.cephfs.smithi186.wjnafs smithi186 running (31m) 59s ago 31m 19.5M - 17.2.7-827-gd3b5060a 21de9b7369ad 74b115d6d6fd 2024-04-12T21:24:01.726 INFO:teuthology.orchestra.run.smithi018.stdout:mgr.smithi018.xjzwga smithi018 *:8443,9283,8765 running (27m) 56s ago 40m 512M - 19.0.0-2879-g324baf8d 8717720dc5e5 56088305eab7 2024-04-12T21:24:01.726 INFO:teuthology.orchestra.run.smithi018.stdout:mgr.smithi186.kqcfxk smithi186 *:8443,9283,8765 running (26m) 59s ago 37m 459M - 19.0.0-2879-g324baf8d 8717720dc5e5 09cd766e5e0d 2024-04-12T21:24:01.726 INFO:teuthology.orchestra.run.smithi018.stdout:mon.smithi018 smithi018 running (22m) 56s ago 41m 73.1M 2048M 19.0.0-2879-g324baf8d 8717720dc5e5 df3e201dfed6 2024-04-12T21:24:01.726 INFO:teuthology.orchestra.run.smithi018.stdout:mon.smithi186 smithi186 running (21m) 59s ago 37m 47.2M 2048M 19.0.0-2879-g324baf8d 8717720dc5e5 c77498da9b2b 2024-04-12T21:24:01.726 INFO:teuthology.orchestra.run.smithi018.stdout:node-exporter.smithi018 smithi018 *:9100 running (25m) 56s ago 39m 8108k - 1.5.0 0da6a335fe13 326c3d9c6847 2024-04-12T21:24:01.726 INFO:teuthology.orchestra.run.smithi018.stdout:node-exporter.smithi186 smithi186 *:9100 running (25m) 59s ago 37m 8757k - 1.5.0 0da6a335fe13 ec2112baeecf 2024-04-12T21:24:01.726 INFO:teuthology.orchestra.run.smithi018.stdout:osd.0 smithi018 running (20m) 56s ago 36m 123M 1559M 19.0.0-2879-g324baf8d 8717720dc5e5 60b1273ca047 2024-04-12T21:24:01.726 INFO:teuthology.orchestra.run.smithi018.stdout:osd.1 smithi018 running (19m) 56s ago 36m 95.1M 1559M 19.0.0-2879-g324baf8d 8717720dc5e5 170ce1e5fb8d 2024-04-12T21:24:01.727 INFO:teuthology.orchestra.run.smithi018.stdout:osd.2 smithi018 running (19m) 56s ago 35m 94.3M 1559M 19.0.0-2879-g324baf8d 8717720dc5e5 c2cbc6aa2839 2024-04-12T21:24:01.727 INFO:teuthology.orchestra.run.smithi018.stdout:osd.3 smithi186 running (18m) 59s ago 34m 101M 2581M 19.0.0-2879-g324baf8d 8717720dc5e5 0138e8262e4b 2024-04-12T21:24:01.727 INFO:teuthology.orchestra.run.smithi018.stdout:osd.4 smithi186 running (17m) 59s ago 34m 91.0M 2581M 19.0.0-2879-g324baf8d 8717720dc5e5 b20cb63b25c9 2024-04-12T21:24:01.727 INFO:teuthology.orchestra.run.smithi018.stdout:osd.5 smithi186 running (17m) 59s ago 33m 92.0M 2581M 19.0.0-2879-g324baf8d 8717720dc5e5 2513bea2610e 2024-04-12T21:24:01.727 INFO:teuthology.orchestra.run.smithi018.stdout:prometheus.smithi018 smithi018 *:9095 running (25m) 56s ago 38m 67.9M - 2.43.0 a07b618ecd1d f58841127e21 2024-04-12T21:24:01.728 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:01.732+0000 7f7166800700 1 -- 172.21.15.18:0/2377034788 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f714406ec10 msgr2=0x7f71440710c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:24:01.728 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:01.732+0000 7f7166800700 1 --2- 172.21.15.18:0/2377034788 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f714406ec10 0x7f71440710c0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f7150007fd0 tx=0x7f7150009040 comp rx=0 tx=0).stop 2024-04-12T21:24:01.728 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:01.732+0000 7f7166800700 1 -- 172.21.15.18:0/2377034788 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f71601014f0 msgr2=0x7f71601162d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:24:01.729 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:01.732+0000 7f7166800700 1 --2- 172.21.15.18:0/2377034788 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f71601014f0 0x7f71601162d0 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f714c009180 tx=0x7f714c009e30 comp rx=0 tx=0).stop 2024-04-12T21:24:01.729 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:01.732+0000 7f7166800700 1 -- 172.21.15.18:0/2377034788 shutdown_connections 2024-04-12T21:24:01.729 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:01.732+0000 7f7166800700 1 --2- 172.21.15.18:0/2377034788 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f714406ec10 0x7f71440710c0 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:24:01.729 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:01.732+0000 7f7166800700 1 --2- 172.21.15.18:0/2377034788 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f71600fb4a0 0x7f7160115d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:24:01.729 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:01.732+0000 7f7166800700 1 --2- 172.21.15.18:0/2377034788 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f71601014f0 0x7f71601162d0 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:24:01.729 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:01.732+0000 7f7166800700 1 -- 172.21.15.18:0/2377034788 >> 172.21.15.18:0/2377034788 conn(0x7f71600714c0 msgr2=0x7f71600f9d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:24:01.729 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:01.732+0000 7f7166800700 1 -- 172.21.15.18:0/2377034788 shutdown_connections 2024-04-12T21:24:01.729 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:01.732+0000 7f7166800700 1 -- 172.21.15.18:0/2377034788 wait complete. 2024-04-12T21:24:01.889 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:01.892+0000 7f7a9aa00700 1 -- 172.21.15.18:0/3503251724 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7a940fde90 msgr2=0x7f7a940fe260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:24:01.889 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:01.892+0000 7f7a9aa00700 1 --2- 172.21.15.18:0/3503251724 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7a940fde90 0x7f7a940fe260 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f7a84004940 tx=0x7f7a84009e30 comp rx=0 tx=0).stop 2024-04-12T21:24:01.889 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:01.892+0000 7f7a9aa00700 1 -- 172.21.15.18:0/3503251724 shutdown_connections 2024-04-12T21:24:01.889 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:01.892+0000 7f7a9aa00700 1 --2- 172.21.15.18:0/3503251724 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7a940fe830 0x7f7a94106f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:24:01.889 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:01.892+0000 7f7a9aa00700 1 --2- 172.21.15.18:0/3503251724 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7a940fde90 0x7f7a940fe260 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:24:01.889 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:01.892+0000 7f7a9aa00700 1 -- 172.21.15.18:0/3503251724 >> 172.21.15.18:0/3503251724 conn(0x7f7a94072bc0 msgr2=0x7f7a94072fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:24:01.890 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:01.893+0000 7f7a9aa00700 1 -- 172.21.15.18:0/3503251724 shutdown_connections 2024-04-12T21:24:01.890 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:01.893+0000 7f7a9aa00700 1 -- 172.21.15.18:0/3503251724 wait complete. 2024-04-12T21:24:01.890 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:01.894+0000 7f7a9aa00700 1 Processor -- start 2024-04-12T21:24:01.890 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:01.894+0000 7f7a9aa00700 1 -- start start 2024-04-12T21:24:01.891 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:01.894+0000 7f7a9aa00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7a940fde90 0x7f7a94115dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:24:01.891 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:01.894+0000 7f7a9aa00700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7a940fe830 0x7f7a94116310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:24:01.891 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:01.894+0000 7f7a9aa00700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a94116960 con 0x7f7a940fde90 2024-04-12T21:24:01.891 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:01.894+0000 7f7a9aa00700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a9410fd30 con 0x7f7a940fe830 2024-04-12T21:24:01.892 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:01.895+0000 7f7a93e00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7a940fde90 0x7f7a94115dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:24:01.892 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:01.895+0000 7f7a93e00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7a940fde90 0x7f7a94115dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:52752/0 (socket says 172.21.15.18:52752) 2024-04-12T21:24:01.892 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:01.895+0000 7f7a93e00700 1 -- 172.21.15.18:0/2811583914 learned_addr learned my addr 172.21.15.18:0/2811583914 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:24:01.892 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:01.895+0000 7f7a8b400700 1 --2- 172.21.15.18:0/2811583914 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7a940fe830 0x7f7a94116310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:24:01.892 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:01.895+0000 7f7a93e00700 1 -- 172.21.15.18:0/2811583914 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7a940fe830 msgr2=0x7f7a94116310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:24:01.892 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:01.895+0000 7f7a93e00700 1 --2- 172.21.15.18:0/2811583914 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7a940fe830 0x7f7a94116310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:24:01.892 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:01.895+0000 7f7a93e00700 1 -- 172.21.15.18:0/2811583914 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a84005040 con 0x7f7a940fde90 2024-04-12T21:24:01.892 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:01.896+0000 7f7a93e00700 1 --2- 172.21.15.18:0/2811583914 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7a940fde90 0x7f7a94115dd0 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f7a840093b0 tx=0x7f7a84009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:24:01.893 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:01.896+0000 7f7a90c00700 1 -- 172.21.15.18:0/2811583914 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7a8400e040 con 0x7f7a940fde90 2024-04-12T21:24:01.893 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:01.896+0000 7f7a90c00700 1 -- 172.21.15.18:0/2811583914 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f7a84004290 con 0x7f7a940fde90 2024-04-12T21:24:01.893 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:01.896+0000 7f7a90c00700 1 -- 172.21.15.18:0/2811583914 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7a8401c3f0 con 0x7f7a940fde90 2024-04-12T21:24:01.893 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:01.896+0000 7f7a9aa00700 1 -- 172.21.15.18:0/2811583914 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7a9410ffb0 con 0x7f7a940fde90 2024-04-12T21:24:01.893 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:01.896+0000 7f7a9aa00700 1 -- 172.21.15.18:0/2811583914 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7a941104f0 con 0x7f7a940fde90 2024-04-12T21:24:01.894 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:01.897+0000 7f7a9aa00700 1 -- 172.21.15.18:0/2811583914 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7a94106a20 con 0x7f7a940fde90 2024-04-12T21:24:01.895 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:01.899+0000 7f7a90c00700 1 -- 172.21.15.18:0/2811583914 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7f7a84013890 con 0x7f7a940fde90 2024-04-12T21:24:01.896 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:01.900+0000 7f7a90c00700 1 --2- 172.21.15.18:0/2811583914 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f7a7406ec10 0x7f7a740710c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:24:01.896 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:01.900+0000 7f7a90c00700 1 -- 172.21.15.18:0/2811583914 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 6757+0+0 (secure 0 0 0) 0x7f7a8408fc10 con 0x7f7a940fde90 2024-04-12T21:24:01.896 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:01.900+0000 7f7a8b400700 1 --2- 172.21.15.18:0/2811583914 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f7a7406ec10 0x7f7a740710c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:24:01.897 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:01.901+0000 7f7a8b400700 1 --2- 172.21.15.18:0/2811583914 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f7a7406ec10 0x7f7a740710c0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f7a94111670 tx=0x7f7a7c009ee0 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:24:01.900 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:01.904+0000 7f7a90c00700 1 -- 172.21.15.18:0/2811583914 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f7a8405c440 con 0x7f7a940fde90 2024-04-12T21:24:02.219 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:02.222+0000 7f7a9aa00700 1 -- 172.21.15.18:0/2811583914 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f7a9404bed0 con 0x7f7a940fde90 2024-04-12T21:24:02.220 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:02.224+0000 7f7a90c00700 1 -- 172.21.15.18:0/2811583914 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+837 (secure 0 0 0) 0x7f7a8405bfd0 con 0x7f7a940fde90 2024-04-12T21:24:02.220 INFO:teuthology.orchestra.run.smithi018.stdout:{ 2024-04-12T21:24:02.220 INFO:teuthology.orchestra.run.smithi018.stdout: "mon": { 2024-04-12T21:24:02.220 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 19.0.0-2879-g324baf8d (324baf8ddb91dea46cac50cb535c306115791355) squid (dev)": 2 2024-04-12T21:24:02.220 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-12T21:24:02.220 INFO:teuthology.orchestra.run.smithi018.stdout: "mgr": { 2024-04-12T21:24:02.221 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 19.0.0-2879-g324baf8d (324baf8ddb91dea46cac50cb535c306115791355) squid (dev)": 2 2024-04-12T21:24:02.221 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-12T21:24:02.221 INFO:teuthology.orchestra.run.smithi018.stdout: "osd": { 2024-04-12T21:24:02.221 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 19.0.0-2879-g324baf8d (324baf8ddb91dea46cac50cb535c306115791355) squid (dev)": 6 2024-04-12T21:24:02.221 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-12T21:24:02.221 INFO:teuthology.orchestra.run.smithi018.stdout: "mds": { 2024-04-12T21:24:02.221 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 17.2.7-827-gd3b5060a (d3b5060a17449edfae9385cdf792afec109fca5d) quincy (stable)": 1, 2024-04-12T21:24:02.221 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 19.0.0-2879-g324baf8d (324baf8ddb91dea46cac50cb535c306115791355) squid (dev)": 3 2024-04-12T21:24:02.221 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-12T21:24:02.221 INFO:teuthology.orchestra.run.smithi018.stdout: "overall": { 2024-04-12T21:24:02.221 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 17.2.7-827-gd3b5060a (d3b5060a17449edfae9385cdf792afec109fca5d) quincy (stable)": 1, 2024-04-12T21:24:02.221 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 19.0.0-2879-g324baf8d (324baf8ddb91dea46cac50cb535c306115791355) squid (dev)": 13 2024-04-12T21:24:02.221 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-04-12T21:24:02.221 INFO:teuthology.orchestra.run.smithi018.stdout:} 2024-04-12T21:24:02.223 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:02.227+0000 7f7a9aa00700 1 -- 172.21.15.18:0/2811583914 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f7a7406ec10 msgr2=0x7f7a740710c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:24:02.223 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:02.227+0000 7f7a9aa00700 1 --2- 172.21.15.18:0/2811583914 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f7a7406ec10 0x7f7a740710c0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f7a94111670 tx=0x7f7a7c009ee0 comp rx=0 tx=0).stop 2024-04-12T21:24:02.223 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:02.227+0000 7f7a9aa00700 1 -- 172.21.15.18:0/2811583914 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7a940fde90 msgr2=0x7f7a94115dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:24:02.223 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:02.227+0000 7f7a9aa00700 1 --2- 172.21.15.18:0/2811583914 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7a940fde90 0x7f7a94115dd0 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f7a840093b0 tx=0x7f7a84009e30 comp rx=0 tx=0).stop 2024-04-12T21:24:02.223 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:02.227+0000 7f7a9aa00700 1 -- 172.21.15.18:0/2811583914 shutdown_connections 2024-04-12T21:24:02.223 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:02.227+0000 7f7a9aa00700 1 --2- 172.21.15.18:0/2811583914 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f7a7406ec10 0x7f7a740710c0 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:24:02.223 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:02.227+0000 7f7a9aa00700 1 --2- 172.21.15.18:0/2811583914 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7a940fde90 0x7f7a94115dd0 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:24:02.223 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:02.227+0000 7f7a9aa00700 1 --2- 172.21.15.18:0/2811583914 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7a940fe830 0x7f7a94116310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:24:02.224 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:02.227+0000 7f7a9aa00700 1 -- 172.21.15.18:0/2811583914 >> 172.21.15.18:0/2811583914 conn(0x7f7a94072bc0 msgr2=0x7f7a940fd590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:24:02.224 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:02.227+0000 7f7a9aa00700 1 -- 172.21.15.18:0/2811583914 shutdown_connections 2024-04-12T21:24:02.224 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:02.227+0000 7f7a9aa00700 1 -- 172.21.15.18:0/2811583914 wait complete. 2024-04-12T21:24:02.368 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:02.372+0000 7f2462200700 1 -- 172.21.15.18:0/3669336770 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f245c071f90 msgr2=0x7f245c070360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:24:02.369 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:02.372+0000 7f2462200700 1 --2- 172.21.15.18:0/3669336770 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f245c071f90 0x7f245c070360 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f244c004970 tx=0x7f244c009e30 comp rx=0 tx=0).stop 2024-04-12T21:24:02.369 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:02.372+0000 7f2462200700 1 -- 172.21.15.18:0/3669336770 shutdown_connections 2024-04-12T21:24:02.369 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:02.372+0000 7f2462200700 1 --2- 172.21.15.18:0/3669336770 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f245c0708a0 0x7f245c070d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:24:02.369 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:02.372+0000 7f2462200700 1 --2- 172.21.15.18:0/3669336770 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f245c071f90 0x7f245c070360 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:24:02.369 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:02.372+0000 7f2462200700 1 -- 172.21.15.18:0/3669336770 >> 172.21.15.18:0/3669336770 conn(0x7f245c0f89b0 msgr2=0x7f245c0fadc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:24:02.369 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:02.372+0000 7f2462200700 1 -- 172.21.15.18:0/3669336770 shutdown_connections 2024-04-12T21:24:02.369 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:02.372+0000 7f2462200700 1 -- 172.21.15.18:0/3669336770 wait complete. 2024-04-12T21:24:02.370 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:02.373+0000 7f2462200700 1 Processor -- start 2024-04-12T21:24:02.370 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:02.373+0000 7f2462200700 1 -- start start 2024-04-12T21:24:02.370 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:02.374+0000 7f2462200700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f245c0708a0 0x7f245c119ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:24:02.370 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:02.374+0000 7f2462200700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f245c071f90 0x7f245c11a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:24:02.370 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:02.374+0000 7f2462200700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f245c11aa80 con 0x7f245c0708a0 2024-04-12T21:24:02.370 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:02.374+0000 7f2462200700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f245c114110 con 0x7f245c071f90 2024-04-12T21:24:02.370 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:02.374+0000 7f245b400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f245c0708a0 0x7f245c119ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:24:02.371 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:02.374+0000 7f245b400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f245c0708a0 0x7f245c119ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:52774/0 (socket says 172.21.15.18:52774) 2024-04-12T21:24:02.371 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:02.374+0000 7f245b400700 1 -- 172.21.15.18:0/1491610620 learned_addr learned my addr 172.21.15.18:0/1491610620 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:24:02.371 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:02.374+0000 7f245aa00700 1 --2- 172.21.15.18:0/1491610620 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f245c071f90 0x7f245c11a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:24:02.372 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:02.374+0000 7f245b400700 1 -- 172.21.15.18:0/1491610620 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f245c071f90 msgr2=0x7f245c11a430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:24:02.372 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:02.374+0000 7f245b400700 1 --2- 172.21.15.18:0/1491610620 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f245c071f90 0x7f245c11a430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:24:02.372 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:02.375+0000 7f245b400700 1 -- 172.21.15.18:0/1491610620 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f244c005040 con 0x7f245c0708a0 2024-04-12T21:24:02.372 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:02.375+0000 7f245b400700 1 --2- 172.21.15.18:0/1491610620 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f245c0708a0 0x7f245c119ef0 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f244c000c00 tx=0x7f244c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:24:02.372 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:02.375+0000 7f2453400700 1 -- 172.21.15.18:0/1491610620 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f244c00e040 con 0x7f245c0708a0 2024-04-12T21:24:02.372 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:02.375+0000 7f2453400700 1 -- 172.21.15.18:0/1491610620 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f244c00bc10 con 0x7f245c0708a0 2024-04-12T21:24:02.372 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:02.375+0000 7f2453400700 1 -- 172.21.15.18:0/1491610620 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f244c01c630 con 0x7f245c0708a0 2024-04-12T21:24:02.372 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:02.375+0000 7f2462200700 1 -- 172.21.15.18:0/1491610620 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f245c114390 con 0x7f245c0708a0 2024-04-12T21:24:02.372 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:02.375+0000 7f2462200700 1 -- 172.21.15.18:0/1491610620 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f245c114880 con 0x7f245c0708a0 2024-04-12T21:24:02.372 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:02.376+0000 7f2462200700 1 -- 172.21.15.18:0/1491610620 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f245c071aa0 con 0x7f245c0708a0 2024-04-12T21:24:02.373 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:02.377+0000 7f2453400700 1 -- 172.21.15.18:0/1491610620 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7f244c00bd80 con 0x7f245c0708a0 2024-04-12T21:24:02.374 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:02.378+0000 7f2453400700 1 --2- 172.21.15.18:0/1491610620 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f244006ec00 0x7f24400710b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:24:02.375 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:02.379+0000 7f2453400700 1 -- 172.21.15.18:0/1491610620 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 6757+0+0 (secure 0 0 0) 0x7f244c08fda0 con 0x7f245c0708a0 2024-04-12T21:24:02.375 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:02.379+0000 7f245aa00700 1 --2- 172.21.15.18:0/1491610620 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f244006ec00 0x7f24400710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:24:02.375 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:02.379+0000 7f245aa00700 1 --2- 172.21.15.18:0/1491610620 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f244006ec00 0x7f24400710b0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f245c10ce80 tx=0x7f2448009ee0 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:24:02.379 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:02.383+0000 7f2453400700 1 -- 172.21.15.18:0/1491610620 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f244c058ac0 con 0x7f245c0708a0 2024-04-12T21:24:02.662 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:02.666+0000 7f2462200700 1 -- 172.21.15.18:0/1491610620 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f245c04c710 con 0x7f245c0708a0 2024-04-12T21:24:02.662 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:02.667+0000 7f2453400700 1 -- 172.21.15.18:0/1491610620 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 31 v31) v1 ==== 76+0+1873 (secure 0 0 0) 0x7f244c05c180 con 0x7f245c0708a0 2024-04-12T21:24:02.663 INFO:teuthology.orchestra.run.smithi018.stdout:e31 2024-04-12T21:24:02.663 INFO:teuthology.orchestra.run.smithi018.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-12T21:24:02.663 INFO:teuthology.orchestra.run.smithi018.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-12T21:24:02.664 INFO:teuthology.orchestra.run.smithi018.stdout:legacy client fscid: 1 2024-04-12T21:24:02.664 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-12T21:24:02.664 INFO:teuthology.orchestra.run.smithi018.stdout:Filesystem 'cephfs' (1) 2024-04-12T21:24:02.664 INFO:teuthology.orchestra.run.smithi018.stdout:fs_name cephfs 2024-04-12T21:24:02.664 INFO:teuthology.orchestra.run.smithi018.stdout:epoch 24 2024-04-12T21:24:02.664 INFO:teuthology.orchestra.run.smithi018.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-04-12T21:24:02.664 INFO:teuthology.orchestra.run.smithi018.stdout:created 2024-04-12T20:51:57.362850+0000 2024-04-12T21:24:02.664 INFO:teuthology.orchestra.run.smithi018.stdout:modified 2024-04-12T21:06:59.717190+0000 2024-04-12T21:24:02.664 INFO:teuthology.orchestra.run.smithi018.stdout:tableserver 0 2024-04-12T21:24:02.664 INFO:teuthology.orchestra.run.smithi018.stdout:root 0 2024-04-12T21:24:02.664 INFO:teuthology.orchestra.run.smithi018.stdout:session_timeout 60 2024-04-12T21:24:02.664 INFO:teuthology.orchestra.run.smithi018.stdout:session_autoclose 300 2024-04-12T21:24:02.664 INFO:teuthology.orchestra.run.smithi018.stdout:max_file_size 1099511627776 2024-04-12T21:24:02.664 INFO:teuthology.orchestra.run.smithi018.stdout:max_xattr_size 65536 2024-04-12T21:24:02.664 INFO:teuthology.orchestra.run.smithi018.stdout:required_client_features {} 2024-04-12T21:24:02.665 INFO:teuthology.orchestra.run.smithi018.stdout:last_failure 0 2024-04-12T21:24:02.665 INFO:teuthology.orchestra.run.smithi018.stdout:last_failure_osd_epoch 100 2024-04-12T21:24:02.665 INFO:teuthology.orchestra.run.smithi018.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-12T21:24:02.665 INFO:teuthology.orchestra.run.smithi018.stdout:max_mds 2 2024-04-12T21:24:02.665 INFO:teuthology.orchestra.run.smithi018.stdout:in 0,1 2024-04-12T21:24:02.665 INFO:teuthology.orchestra.run.smithi018.stdout:up {} 2024-04-12T21:24:02.665 INFO:teuthology.orchestra.run.smithi018.stdout:failed 0,1 2024-04-12T21:24:02.665 INFO:teuthology.orchestra.run.smithi018.stdout:damaged 2024-04-12T21:24:02.665 INFO:teuthology.orchestra.run.smithi018.stdout:stopped 2024-04-12T21:24:02.665 INFO:teuthology.orchestra.run.smithi018.stdout:data_pools [3] 2024-04-12T21:24:02.665 INFO:teuthology.orchestra.run.smithi018.stdout:metadata_pool 2 2024-04-12T21:24:02.665 INFO:teuthology.orchestra.run.smithi018.stdout:inline_data disabled 2024-04-12T21:24:02.665 INFO:teuthology.orchestra.run.smithi018.stdout:balancer 2024-04-12T21:24:02.665 INFO:teuthology.orchestra.run.smithi018.stdout:bal_rank_mask -1 2024-04-12T21:24:02.666 INFO:teuthology.orchestra.run.smithi018.stdout:standby_count_wanted 1 2024-04-12T21:24:02.666 INFO:teuthology.orchestra.run.smithi018.stdout:qdb_cluster leader: 0 members: 2024-04-12T21:24:02.666 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-12T21:24:02.666 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-12T21:24:02.666 INFO:teuthology.orchestra.run.smithi018.stdout:Standby daemons: 2024-04-12T21:24:02.666 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-12T21:24:02.666 INFO:teuthology.orchestra.run.smithi018.stdout:[mds.cephfs.smithi186.wjnafs{-1:24277} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.186:6826/438347207,v1:172.21.15.186:6827/438347207] compat {c=[1],r=[1],i=[7ff]}] 2024-04-12T21:24:02.666 INFO:teuthology.orchestra.run.smithi018.stdout:[mds.cephfs.smithi018.acqbsc{-1:34436} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.18:6828/755179764,v1:172.21.15.18:6829/755179764] compat {c=[1],r=[1],i=[1fff]}] 2024-04-12T21:24:02.666 INFO:teuthology.orchestra.run.smithi018.stdout:[mds.cephfs.smithi018.szjluo{-1:34468} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.18:6826/2407992653,v1:172.21.15.18:6827/2407992653] compat {c=[1],r=[1],i=[1fff]}] 2024-04-12T21:24:02.666 INFO:teuthology.orchestra.run.smithi018.stdout:[mds.cephfs.smithi186.jdllyn{-1:44595} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.186:6824/3523444202,v1:172.21.15.186:6825/3523444202] compat {c=[1],r=[1],i=[1fff]}] 2024-04-12T21:24:02.667 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:02.671+0000 7f2462200700 1 -- 172.21.15.18:0/1491610620 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f244006ec00 msgr2=0x7f24400710b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:24:02.668 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:02.671+0000 7f2462200700 1 --2- 172.21.15.18:0/1491610620 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f244006ec00 0x7f24400710b0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f245c10ce80 tx=0x7f2448009ee0 comp rx=0 tx=0).stop 2024-04-12T21:24:02.668 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:02.671+0000 7f2462200700 1 -- 172.21.15.18:0/1491610620 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f245c0708a0 msgr2=0x7f245c119ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:24:02.668 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:02.671+0000 7f2462200700 1 --2- 172.21.15.18:0/1491610620 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f245c0708a0 0x7f245c119ef0 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f244c000c00 tx=0x7f244c009e30 comp rx=0 tx=0).stop 2024-04-12T21:24:02.668 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:02.671+0000 7f2462200700 1 -- 172.21.15.18:0/1491610620 shutdown_connections 2024-04-12T21:24:02.668 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:02.671+0000 7f2462200700 1 --2- 172.21.15.18:0/1491610620 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f244006ec00 0x7f24400710b0 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:24:02.668 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:02.671+0000 7f2462200700 1 --2- 172.21.15.18:0/1491610620 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f245c0708a0 0x7f245c119ef0 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:24:02.668 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:02.671+0000 7f2462200700 1 --2- 172.21.15.18:0/1491610620 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f245c071f90 0x7f245c11a430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:24:02.668 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:02.671+0000 7f2462200700 1 -- 172.21.15.18:0/1491610620 >> 172.21.15.18:0/1491610620 conn(0x7f245c0f89b0 msgr2=0x7f245c0f9a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:24:02.668 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:02.671+0000 7f2462200700 1 -- 172.21.15.18:0/1491610620 shutdown_connections 2024-04-12T21:24:02.668 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:02.671+0000 7f2462200700 1 -- 172.21.15.18:0/1491610620 wait complete. 2024-04-12T21:24:02.669 INFO:teuthology.orchestra.run.smithi018.stderr:dumped fsmap epoch 31 2024-04-12T21:24:02.823 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:02.826+0000 7f9b6f000700 1 -- 172.21.15.18:0/132986534 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9b68070360 msgr2=0x7f9b68070730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:24:02.823 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:02.826+0000 7f9b6f000700 1 --2- 172.21.15.18:0/132986534 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9b68070360 0x7f9b68070730 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f9b58004970 tx=0x7f9b58009e30 comp rx=0 tx=0).stop 2024-04-12T21:24:02.823 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:02.827+0000 7f9b6f000700 1 -- 172.21.15.18:0/132986534 shutdown_connections 2024-04-12T21:24:02.823 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:02.827+0000 7f9b6f000700 1 --2- 172.21.15.18:0/132986534 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9b68070c70 0x7f9b68109110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:24:02.823 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:02.827+0000 7f9b6f000700 1 --2- 172.21.15.18:0/132986534 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9b68070360 0x7f9b68070730 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:24:02.823 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:02.827+0000 7f9b6f000700 1 -- 172.21.15.18:0/132986534 >> 172.21.15.18:0/132986534 conn(0x7f9b680f89b0 msgr2=0x7f9b680fadc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:24:02.824 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:02.827+0000 7f9b6f000700 1 -- 172.21.15.18:0/132986534 shutdown_connections 2024-04-12T21:24:02.824 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:02.827+0000 7f9b6f000700 1 -- 172.21.15.18:0/132986534 wait complete. 2024-04-12T21:24:02.824 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:02.828+0000 7f9b6f000700 1 Processor -- start 2024-04-12T21:24:02.824 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:02.828+0000 7f9b6f000700 1 -- start start 2024-04-12T21:24:02.825 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:02.829+0000 7f9b6f000700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9b68070360 0x7f9b68115b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:24:02.825 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:02.829+0000 7f9b6f000700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9b68070c70 0x7f9b681160a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:24:02.825 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:02.829+0000 7f9b6f000700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b681166f0 con 0x7f9b68070c70 2024-04-12T21:24:02.825 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:02.829+0000 7f9b6f000700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b6810fcd0 con 0x7f9b68070360 2024-04-12T21:24:02.825 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:02.829+0000 7f9b67400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9b68070c70 0x7f9b681160a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:24:02.825 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:02.829+0000 7f9b67400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9b68070c70 0x7f9b681160a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:52800/0 (socket says 172.21.15.18:52800) 2024-04-12T21:24:02.825 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:02.829+0000 7f9b67400700 1 -- 172.21.15.18:0/1294565104 learned_addr learned my addr 172.21.15.18:0/1294565104 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:24:02.826 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:02.829+0000 7f9b67e00700 1 --2- 172.21.15.18:0/1294565104 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9b68070360 0x7f9b68115b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:24:02.826 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:02.830+0000 7f9b67400700 1 -- 172.21.15.18:0/1294565104 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9b68070360 msgr2=0x7f9b68115b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:24:02.826 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:02.830+0000 7f9b67400700 1 --2- 172.21.15.18:0/1294565104 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9b68070360 0x7f9b68115b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:24:02.826 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:02.830+0000 7f9b67400700 1 -- 172.21.15.18:0/1294565104 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b58005040 con 0x7f9b68070c70 2024-04-12T21:24:02.826 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:02.830+0000 7f9b67400700 1 --2- 172.21.15.18:0/1294565104 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9b68070c70 0x7f9b681160a0 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f9b5400c8c0 tx=0x7f9b540044c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:24:02.827 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:02.830+0000 7f9b5fe00700 1 -- 172.21.15.18:0/1294565104 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9b5400a980 con 0x7f9b68070c70 2024-04-12T21:24:02.828 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:02.830+0000 7f9b6f000700 1 -- 172.21.15.18:0/1294565104 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9b6810ffb0 con 0x7f9b68070c70 2024-04-12T21:24:02.828 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:02.830+0000 7f9b5fe00700 1 -- 172.21.15.18:0/1294565104 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f9b5400ce60 con 0x7f9b68070c70 2024-04-12T21:24:02.828 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:02.830+0000 7f9b5fe00700 1 -- 172.21.15.18:0/1294565104 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9b540115e0 con 0x7f9b68070c70 2024-04-12T21:24:02.828 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:02.831+0000 7f9b6f000700 1 -- 172.21.15.18:0/1294565104 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9b681104f0 con 0x7f9b68070c70 2024-04-12T21:24:02.828 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:02.832+0000 7f9b6f000700 1 -- 172.21.15.18:0/1294565104 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9b68071aa0 con 0x7f9b68070c70 2024-04-12T21:24:02.829 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:02.833+0000 7f9b5fe00700 1 -- 172.21.15.18:0/1294565104 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7f9b54012400 con 0x7f9b68070c70 2024-04-12T21:24:02.830 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:02.834+0000 7f9b5fe00700 1 --2- 172.21.15.18:0/1294565104 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f9b4c06ec00 0x7f9b4c0710b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:24:02.830 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:02.834+0000 7f9b5fe00700 1 -- 172.21.15.18:0/1294565104 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 6757+0+0 (secure 0 0 0) 0x7f9b5408ea40 con 0x7f9b68070c70 2024-04-12T21:24:02.831 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:02.835+0000 7f9b67e00700 1 --2- 172.21.15.18:0/1294565104 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f9b4c06ec00 0x7f9b4c0710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:24:02.831 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:02.835+0000 7f9b67e00700 1 --2- 172.21.15.18:0/1294565104 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f9b4c06ec00 0x7f9b4c0710b0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f9b58004f50 tx=0x7f9b58009e30 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:24:02.835 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:02.839+0000 7f9b5fe00700 1 -- 172.21.15.18:0/1294565104 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f9b5405b270 con 0x7f9b68070c70 2024-04-12T21:24:03.029 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:24:02 smithi186 ceph-mon[86379]: from='client.35288 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:24:03.029 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:24:02 smithi186 ceph-mon[86379]: from='client.35292 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:24:03.029 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:24:02 smithi186 ceph-mon[86379]: pgmap v728: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:24:03.029 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:24:02 smithi186 ceph-mon[86379]: from='client.35296 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:24:03.029 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:24:02 smithi186 ceph-mon[86379]: from='client.? 172.21.15.18:0/2811583914' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T21:24:03.029 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:24:02 smithi186 ceph-mon[86379]: from='client.? 172.21.15.18:0/1491610620' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-12T21:24:03.080 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:03.083+0000 7f9b6f000700 1 -- 172.21.15.18:0/1294565104 --> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f9b6810c160 con 0x7f9b4c06ec00 2024-04-12T21:24:03.080 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:24:02 smithi018 ceph-mon[112590]: from='client.35288 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:24:03.080 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:24:02 smithi018 ceph-mon[112590]: from='client.35292 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:24:03.080 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:24:02 smithi018 ceph-mon[112590]: pgmap v728: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:24:03.080 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:24:02 smithi018 ceph-mon[112590]: from='client.35296 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:24:03.080 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:24:02 smithi018 ceph-mon[112590]: from='client.? 172.21.15.18:0/2811583914' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T21:24:03.080 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:24:02 smithi018 ceph-mon[112590]: from='client.? 172.21.15.18:0/1491610620' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-12T21:24:03.082 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:03.086+0000 7f9b5fe00700 1 -- 172.21.15.18:0/1294565104 <== mgr.34102 v2:172.21.15.18:6800/3812891919 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f9b6810c160 con 0x7f9b4c06ec00 2024-04-12T21:24:03.082 INFO:teuthology.orchestra.run.smithi018.stdout:{ 2024-04-12T21:24:03.082 INFO:teuthology.orchestra.run.smithi018.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:f4452def3be4d350f4a8e5feaf7ebb1511be9298fe68478ba7a2d6fbd7d69a63", 2024-04-12T21:24:03.082 INFO:teuthology.orchestra.run.smithi018.stdout: "in_progress": true, 2024-04-12T21:24:03.082 INFO:teuthology.orchestra.run.smithi018.stdout: "which": "Upgrading all daemon types on all hosts", 2024-04-12T21:24:03.082 INFO:teuthology.orchestra.run.smithi018.stdout: "services_complete": [ 2024-04-12T21:24:03.082 INFO:teuthology.orchestra.run.smithi018.stdout: "crash", 2024-04-12T21:24:03.082 INFO:teuthology.orchestra.run.smithi018.stdout: "osd", 2024-04-12T21:24:03.083 INFO:teuthology.orchestra.run.smithi018.stdout: "mon", 2024-04-12T21:24:03.083 INFO:teuthology.orchestra.run.smithi018.stdout: "mgr" 2024-04-12T21:24:03.083 INFO:teuthology.orchestra.run.smithi018.stdout: ], 2024-04-12T21:24:03.083 INFO:teuthology.orchestra.run.smithi018.stdout: "progress": "15/21 daemons upgraded", 2024-04-12T21:24:03.083 INFO:teuthology.orchestra.run.smithi018.stdout: "message": "Currently upgrading mds daemons", 2024-04-12T21:24:03.083 INFO:teuthology.orchestra.run.smithi018.stdout: "is_paused": false 2024-04-12T21:24:03.083 INFO:teuthology.orchestra.run.smithi018.stdout:} 2024-04-12T21:24:03.086 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:03.089+0000 7f9b6f000700 1 -- 172.21.15.18:0/1294565104 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f9b4c06ec00 msgr2=0x7f9b4c0710b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:24:03.086 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:03.089+0000 7f9b6f000700 1 --2- 172.21.15.18:0/1294565104 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f9b4c06ec00 0x7f9b4c0710b0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f9b58004f50 tx=0x7f9b58009e30 comp rx=0 tx=0).stop 2024-04-12T21:24:03.086 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:03.089+0000 7f9b6f000700 1 -- 172.21.15.18:0/1294565104 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9b68070c70 msgr2=0x7f9b681160a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:24:03.086 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:03.089+0000 7f9b6f000700 1 --2- 172.21.15.18:0/1294565104 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9b68070c70 0x7f9b681160a0 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f9b5400c8c0 tx=0x7f9b540044c0 comp rx=0 tx=0).stop 2024-04-12T21:24:03.086 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:03.090+0000 7f9b6f000700 1 -- 172.21.15.18:0/1294565104 shutdown_connections 2024-04-12T21:24:03.086 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:03.090+0000 7f9b6f000700 1 --2- 172.21.15.18:0/1294565104 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f9b4c06ec00 0x7f9b4c0710b0 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:24:03.087 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:03.090+0000 7f9b6f000700 1 --2- 172.21.15.18:0/1294565104 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9b68070360 0x7f9b68115b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:24:03.087 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:03.090+0000 7f9b6f000700 1 --2- 172.21.15.18:0/1294565104 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9b68070c70 0x7f9b681160a0 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:24:03.087 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:03.090+0000 7f9b6f000700 1 -- 172.21.15.18:0/1294565104 >> 172.21.15.18:0/1294565104 conn(0x7f9b680f89b0 msgr2=0x7f9b680f9de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:24:03.087 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:03.090+0000 7f9b6f000700 1 -- 172.21.15.18:0/1294565104 shutdown_connections 2024-04-12T21:24:03.087 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:03.090+0000 7f9b6f000700 1 -- 172.21.15.18:0/1294565104 wait complete. 2024-04-12T21:24:03.243 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:03.246+0000 7f4460e00700 1 -- 172.21.15.18:0/84670197 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f445c1059d0 msgr2=0x7f445c0ff040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:24:03.243 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:03.246+0000 7f4460e00700 1 --2- 172.21.15.18:0/84670197 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f445c1059d0 0x7f445c0ff040 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f444c004940 tx=0x7f444c009e30 comp rx=0 tx=0).stop 2024-04-12T21:24:03.243 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:03.247+0000 7f4460e00700 1 -- 172.21.15.18:0/84670197 shutdown_connections 2024-04-12T21:24:03.243 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:03.247+0000 7f4460e00700 1 --2- 172.21.15.18:0/84670197 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f445c1059d0 0x7f445c0ff040 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:24:03.243 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:03.247+0000 7f4460e00700 1 --2- 172.21.15.18:0/84670197 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f445c105030 0x7f445c105400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:24:03.244 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:03.247+0000 7f4460e00700 1 -- 172.21.15.18:0/84670197 >> 172.21.15.18:0/84670197 conn(0x7f445c0faba0 msgr2=0x7f445c0fcfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:24:03.244 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:03.247+0000 7f4460e00700 1 -- 172.21.15.18:0/84670197 shutdown_connections 2024-04-12T21:24:03.244 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:03.247+0000 7f4460e00700 1 -- 172.21.15.18:0/84670197 wait complete. 2024-04-12T21:24:03.246 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:03.248+0000 7f4460e00700 1 Processor -- start 2024-04-12T21:24:03.246 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:03.248+0000 7f4460e00700 1 -- start start 2024-04-12T21:24:03.246 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:03.248+0000 7f4460e00700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f445c105030 0x7f445c115bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:24:03.246 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:03.248+0000 7f4460e00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f445c1059d0 0x7f445c116100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:24:03.246 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:03.248+0000 7f4460e00700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f445c116750 con 0x7f445c1059d0 2024-04-12T21:24:03.246 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:03.248+0000 7f4460e00700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f445c10fd30 con 0x7f445c105030 2024-04-12T21:24:03.246 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:03.249+0000 7f4451400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f445c1059d0 0x7f445c116100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:24:03.246 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:03.249+0000 7f4451400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f445c1059d0 0x7f445c116100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:52814/0 (socket says 172.21.15.18:52814) 2024-04-12T21:24:03.246 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:03.249+0000 7f4451400700 1 -- 172.21.15.18:0/4189461439 learned_addr learned my addr 172.21.15.18:0/4189461439 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:24:03.247 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:03.249+0000 7f4459e00700 1 --2- 172.21.15.18:0/4189461439 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f445c105030 0x7f445c115bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:24:03.247 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:03.249+0000 7f4451400700 1 -- 172.21.15.18:0/4189461439 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f445c105030 msgr2=0x7f445c115bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:24:03.247 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:03.249+0000 7f4451400700 1 --2- 172.21.15.18:0/4189461439 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f445c105030 0x7f445c115bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:24:03.247 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:03.249+0000 7f4451400700 1 -- 172.21.15.18:0/4189461439 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f444c005040 con 0x7f445c1059d0 2024-04-12T21:24:03.247 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:03.249+0000 7f4459e00700 1 --2- 172.21.15.18:0/4189461439 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f445c105030 0x7f445c115bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-12T21:24:03.247 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:03.249+0000 7f4451400700 1 --2- 172.21.15.18:0/4189461439 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f445c1059d0 0x7f445c116100 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f444c009100 tx=0x7f444c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:24:03.247 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:03.250+0000 7f4452a00700 1 -- 172.21.15.18:0/4189461439 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f444c00e040 con 0x7f445c1059d0 2024-04-12T21:24:03.247 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:03.250+0000 7f4452a00700 1 -- 172.21.15.18:0/4189461439 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f444c013420 con 0x7f445c1059d0 2024-04-12T21:24:03.247 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:03.250+0000 7f4460e00700 1 -- 172.21.15.18:0/4189461439 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f445c10ffb0 con 0x7f445c1059d0 2024-04-12T21:24:03.247 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:03.250+0000 7f4452a00700 1 -- 172.21.15.18:0/4189461439 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f444c01c630 con 0x7f445c1059d0 2024-04-12T21:24:03.247 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:03.250+0000 7f4460e00700 1 -- 172.21.15.18:0/4189461439 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f445c110570 con 0x7f445c1059d0 2024-04-12T21:24:03.248 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:03.251+0000 7f4460e00700 1 -- 172.21.15.18:0/4189461439 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f445c100780 con 0x7f445c1059d0 2024-04-12T21:24:03.248 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:03.252+0000 7f4452a00700 1 -- 172.21.15.18:0/4189461439 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7f444c013590 con 0x7f445c1059d0 2024-04-12T21:24:03.249 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:03.253+0000 7f4452a00700 1 --2- 172.21.15.18:0/4189461439 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f444006ec00 0x7f44400710b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:24:03.249 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:03.253+0000 7f4452a00700 1 -- 172.21.15.18:0/4189461439 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 6757+0+0 (secure 0 0 0) 0x7f444c08fba0 con 0x7f445c1059d0 2024-04-12T21:24:03.249 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:03.253+0000 7f4459e00700 1 --2- 172.21.15.18:0/4189461439 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f444006ec00 0x7f44400710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:24:03.250 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:03.254+0000 7f4459e00700 1 --2- 172.21.15.18:0/4189461439 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f444006ec00 0x7f44400710b0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f4444007fd0 tx=0x7f4444009040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:24:03.254 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:03.258+0000 7f4452a00700 1 -- 172.21.15.18:0/4189461439 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f444c05c3d0 con 0x7f445c1059d0 2024-04-12T21:24:03.575 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:03.578+0000 7f4460e00700 1 -- 172.21.15.18:0/4189461439 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f445c04c710 con 0x7f445c1059d0 2024-04-12T21:24:03.575 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:03.579+0000 7f4452a00700 1 -- 172.21.15.18:0/4189461439 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+4303 (secure 0 0 0) 0x7f444c058020 con 0x7f445c1059d0 2024-04-12T21:24:03.576 INFO:teuthology.orchestra.run.smithi018.stdout:HEALTH_ERR failed to probe daemons or devices; 1 filesystem is degraded; 1 filesystem is offline 2024-04-12T21:24:03.576 INFO:teuthology.orchestra.run.smithi018.stdout:[WRN] CEPHADM_REFRESH_FAILED: failed to probe daemons or devices 2024-04-12T21:24:03.576 INFO:teuthology.orchestra.run.smithi018.stdout: host smithi018 `cephadm ceph-volume` failed: cephadm exited with an error code: 1, stderr: Inferring config /var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/mon.smithi018/config 2024-04-12T21:24:03.576 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/podman: timeout after 895 seconds 2024-04-12T21:24:03.576 INFO:teuthology.orchestra.run.smithi018.stdout:Non-zero exit code 124 from /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:f4452def3be4d350f4a8e5feaf7ebb1511be9298fe68478ba7a2d6fbd7d69a63 -e NODE_NAME=smithi018 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968:/var/run/ceph:z -v /var/log/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968:/var/log/ceph:z -v /var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmpxsmwkkuh:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:f4452def3be4d350f4a8e5feaf7ebb1511be9298fe68478ba7a2d6fbd7d69a63 inventory --format=json-pretty --filter-for-batch 2024-04-12T21:24:03.576 INFO:teuthology.orchestra.run.smithi018.stdout:Traceback (most recent call last): 2024-04-12T21:24:03.576 INFO:teuthology.orchestra.run.smithi018.stdout: File "/usr/lib64/python3.9/runpy.py", line 197, in _run_module_as_main 2024-04-12T21:24:03.576 INFO:teuthology.orchestra.run.smithi018.stdout: return _run_code(code, main_globals, None, 2024-04-12T21:24:03.576 INFO:teuthology.orchestra.run.smithi018.stdout: File "/usr/lib64/python3.9/runpy.py", line 87, in _run_code 2024-04-12T21:24:03.576 INFO:teuthology.orchestra.run.smithi018.stdout: exec(code, run_globals) 2024-04-12T21:24:03.576 INFO:teuthology.orchestra.run.smithi018.stdout: File "/var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/cephadm.e44f3ed5e34e4ef52eedef233d1190bff004470d8e8005127b5312bff357fa7f/__main__.py", line 5618, in 2024-04-12T21:24:03.576 INFO:teuthology.orchestra.run.smithi018.stdout: File "/var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/cephadm.e44f3ed5e34e4ef52eedef233d1190bff004470d8e8005127b5312bff357fa7f/__main__.py", line 5606, in main 2024-04-12T21:24:03.577 INFO:teuthology.orchestra.run.smithi018.stdout: File "/var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/cephadm.e44f3ed5e34e4ef52eedef233d1190bff004470d8e8005127b5312bff357fa7f/__main__.py", line 408, in _infer_config 2024-04-12T21:24:03.577 INFO:teuthology.orchestra.run.smithi018.stdout: File "/var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/cephadm.e44f3ed5e34e4ef52eedef233d1190bff004470d8e8005127b5312bff357fa7f/__main__.py", line 323, in _infer_fsid 2024-04-12T21:24:03.577 INFO:teuthology.orchestra.run.smithi018.stdout: File "/var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/cephadm.e44f3ed5e34e4ef52eedef233d1190bff004470d8e8005127b5312bff357fa7f/__main__.py", line 436, in _infer_image 2024-04-12T21:24:03.577 INFO:teuthology.orchestra.run.smithi018.stdout: File "/var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/cephadm.e44f3ed5e34e4ef52eedef233d1190bff004470d8e8005127b5312bff357fa7f/__main__.py", line 310, in _validate_fsid 2024-04-12T21:24:03.577 INFO:teuthology.orchestra.run.smithi018.stdout: File "/var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/cephadm.e44f3ed5e34e4ef52eedef233d1190bff004470d8e8005127b5312bff357fa7f/__main__.py", line 3359, in command_ceph_volume 2024-04-12T21:24:03.577 INFO:teuthology.orchestra.run.smithi018.stdout: File "/var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/cephadm.e44f3ed5e34e4ef52eedef233d1190bff004470d8e8005127b5312bff357fa7f/cephadmlib/call_wrappers.py", line 310, in call_throws 2024-04-12T21:24:03.577 INFO:teuthology.orchestra.run.smithi018.stdout:RuntimeError: Failed command: /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:f4452def3be4d350f4a8e5feaf7ebb1511be9298fe68478ba7a2d6fbd7d69a63 -e NODE_NAME=smithi018 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968:/var/run/ceph:z -v /var/log/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968:/var/log/ceph:z -v /var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmpxsmwkkuh:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:f4452def3be4d350f4a8e5feaf7ebb1511be9298fe68478ba7a2d6fbd7d69a63 inventory --format=json-pretty --filter-for-batch 2024-04-12T21:24:03.577 INFO:teuthology.orchestra.run.smithi018.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-04-12T21:24:03.577 INFO:teuthology.orchestra.run.smithi018.stdout: fs cephfs is degraded 2024-04-12T21:24:03.577 INFO:teuthology.orchestra.run.smithi018.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-04-12T21:24:03.577 INFO:teuthology.orchestra.run.smithi018.stdout: fs cephfs is offline because no MDS is active for it. 2024-04-12T21:24:03.579 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:03.582+0000 7f4460e00700 1 -- 172.21.15.18:0/4189461439 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f444006ec00 msgr2=0x7f44400710b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:24:03.579 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:03.582+0000 7f4460e00700 1 --2- 172.21.15.18:0/4189461439 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f444006ec00 0x7f44400710b0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f4444007fd0 tx=0x7f4444009040 comp rx=0 tx=0).stop 2024-04-12T21:24:03.579 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:03.582+0000 7f4460e00700 1 -- 172.21.15.18:0/4189461439 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f445c1059d0 msgr2=0x7f445c116100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:24:03.579 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:03.582+0000 7f4460e00700 1 --2- 172.21.15.18:0/4189461439 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f445c1059d0 0x7f445c116100 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f444c009100 tx=0x7f444c009e30 comp rx=0 tx=0).stop 2024-04-12T21:24:03.579 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:03.583+0000 7f4460e00700 1 -- 172.21.15.18:0/4189461439 shutdown_connections 2024-04-12T21:24:03.579 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:03.583+0000 7f4460e00700 1 --2- 172.21.15.18:0/4189461439 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f444006ec00 0x7f44400710b0 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:24:03.579 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:03.583+0000 7f4460e00700 1 --2- 172.21.15.18:0/4189461439 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f445c105030 0x7f445c115bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:24:03.580 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:03.583+0000 7f4460e00700 1 --2- 172.21.15.18:0/4189461439 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f445c1059d0 0x7f445c116100 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:24:03.580 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:03.583+0000 7f4460e00700 1 -- 172.21.15.18:0/4189461439 >> 172.21.15.18:0/4189461439 conn(0x7f445c0faba0 msgr2=0x7f445c0fbb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:24:03.580 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:03.583+0000 7f4460e00700 1 -- 172.21.15.18:0/4189461439 shutdown_connections 2024-04-12T21:24:03.580 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:03.583+0000 7f4460e00700 1 -- 172.21.15.18:0/4189461439 wait complete. 2024-04-12T21:24:03.849 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:24:03 smithi018 ceph-mon[112590]: from='client.? 172.21.15.18:0/4189461439' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-12T21:24:04.028 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:24:03 smithi186 ceph-mon[86379]: from='client.? 172.21.15.18:0/4189461439' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-12T21:24:05.028 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:24:04 smithi186 ceph-mon[86379]: from='client.35308 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:24:05.029 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:24:04 smithi186 ceph-mon[86379]: pgmap v729: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:24:05.099 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:24:04 smithi018 ceph-mon[112590]: from='client.35308 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:24:05.100 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:24:04 smithi018 ceph-mon[112590]: pgmap v729: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:24:07.028 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:24:06 smithi186 ceph-mon[86379]: pgmap v730: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:24:07.099 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:24:06 smithi018 ceph-mon[112590]: pgmap v730: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:24:09.028 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:24:08 smithi186 ceph-mon[86379]: pgmap v731: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:24:09.099 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:24:08 smithi018 ceph-mon[112590]: pgmap v731: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:24:11.029 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:24:10 smithi186 ceph-mon[86379]: pgmap v732: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:24:11.029 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:24:10 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T21:24:11.100 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:24:10 smithi018 ceph-mon[112590]: pgmap v732: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:24:11.100 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:24:10 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T21:24:13.028 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:24:12 smithi186 ceph-mon[86379]: pgmap v733: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:24:13.099 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:24:12 smithi018 ceph-mon[112590]: pgmap v733: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:24:15.029 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:24:14 smithi186 ceph-mon[86379]: pgmap v734: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:24:15.099 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:24:14 smithi018 ceph-mon[112590]: pgmap v734: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:24:17.029 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:24:16 smithi186 ceph-mon[86379]: pgmap v735: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:24:17.099 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:24:16 smithi018 ceph-mon[112590]: pgmap v735: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:24:19.029 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:24:18 smithi186 ceph-mon[86379]: pgmap v736: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:24:19.099 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:24:18 smithi018 ceph-mon[112590]: pgmap v736: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:24:21.029 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:24:20 smithi186 ceph-mon[86379]: pgmap v737: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:24:21.100 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:24:20 smithi018 ceph-mon[112590]: pgmap v737: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:24:23.029 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:24:22 smithi186 ceph-mon[86379]: pgmap v738: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:24:23.099 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:24:22 smithi018 ceph-mon[112590]: pgmap v738: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:24:25.029 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:24:24 smithi186 ceph-mon[86379]: pgmap v739: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:24:25.100 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:24:24 smithi018 ceph-mon[112590]: pgmap v739: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:24:26.100 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:24:25 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T21:24:26.279 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:24:25 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T21:24:27.100 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:24:26 smithi018 ceph-mon[112590]: pgmap v740: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:24:27.279 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:24:26 smithi186 ceph-mon[86379]: pgmap v740: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:24:28.100 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:24:27 smithi018 ceph-mon[112590]: pgmap v741: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:24:28.279 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:24:27 smithi186 ceph-mon[86379]: pgmap v741: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:24:30.779 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:24:30 smithi186 ceph-mon[86379]: pgmap v742: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:24:30.850 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:24:30 smithi018 ceph-mon[112590]: pgmap v742: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:24:32.779 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:24:32 smithi186 ceph-mon[86379]: pgmap v743: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:24:32.850 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:24:32 smithi018 ceph-mon[112590]: pgmap v743: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:24:33.756 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:33.760+0000 7f5719000700 1 -- 172.21.15.18:0/26478188 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5714102e00 msgr2=0x7f57141031d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:24:33.756 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:33.760+0000 7f5719000700 1 --2- 172.21.15.18:0/26478188 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5714102e00 0x7f57141031d0 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f5704004940 tx=0x7f5704009e30 comp rx=0 tx=0).stop 2024-04-12T21:24:33.756 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:33.760+0000 7f5719000700 1 -- 172.21.15.18:0/26478188 shutdown_connections 2024-04-12T21:24:33.756 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:33.760+0000 7f5719000700 1 --2- 172.21.15.18:0/26478188 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f57141037a0 0x7f57140fce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:24:33.756 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:33.760+0000 7f5719000700 1 --2- 172.21.15.18:0/26478188 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5714102e00 0x7f57141031d0 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:24:33.756 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:33.760+0000 7f5719000700 1 -- 172.21.15.18:0/26478188 >> 172.21.15.18:0/26478188 conn(0x7f57140f89b0 msgr2=0x7f57140fadc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:24:33.756 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:33.760+0000 7f5719000700 1 -- 172.21.15.18:0/26478188 shutdown_connections 2024-04-12T21:24:33.756 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:33.760+0000 7f5719000700 1 -- 172.21.15.18:0/26478188 wait complete. 2024-04-12T21:24:33.758 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:33.761+0000 7f5719000700 1 Processor -- start 2024-04-12T21:24:33.758 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:33.761+0000 7f5719000700 1 -- start start 2024-04-12T21:24:33.759 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:33.761+0000 7f5719000700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5714102e00 0x7f5714115dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:24:33.759 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:33.761+0000 7f5719000700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f57141037a0 0x7f5714116310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:24:33.759 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:33.761+0000 7f5719000700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57141169f0 con 0x7f57141037a0 2024-04-12T21:24:33.759 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:33.761+0000 7f5719000700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f571410fd30 con 0x7f5714102e00 2024-04-12T21:24:33.759 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:33.762+0000 7f5709400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f57141037a0 0x7f5714116310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:24:33.759 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:33.762+0000 7f5709400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f57141037a0 0x7f5714116310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:34858/0 (socket says 172.21.15.18:34858) 2024-04-12T21:24:33.759 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:33.762+0000 7f5709400700 1 -- 172.21.15.18:0/2617412011 learned_addr learned my addr 172.21.15.18:0/2617412011 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:24:33.759 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:33.762+0000 7f5711e00700 1 --2- 172.21.15.18:0/2617412011 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5714102e00 0x7f5714115dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:24:33.759 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:33.762+0000 7f5709400700 1 -- 172.21.15.18:0/2617412011 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5714102e00 msgr2=0x7f5714115dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:24:33.760 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:33.762+0000 7f5709400700 1 --2- 172.21.15.18:0/2617412011 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5714102e00 0x7f5714115dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:24:33.760 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:33.762+0000 7f5709400700 1 -- 172.21.15.18:0/2617412011 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5704005040 con 0x7f57141037a0 2024-04-12T21:24:33.760 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:33.762+0000 7f5711e00700 1 --2- 172.21.15.18:0/2617412011 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5714102e00 0x7f5714115dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-12T21:24:33.760 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:33.763+0000 7f5709400700 1 --2- 172.21.15.18:0/2617412011 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f57141037a0 0x7f5714116310 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f56fc00b790 tx=0x7f56fc009220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:24:33.760 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:33.763+0000 7f570aa00700 1 -- 172.21.15.18:0/2617412011 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f56fc00f6f0 con 0x7f57141037a0 2024-04-12T21:24:33.760 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:33.763+0000 7f570aa00700 1 -- 172.21.15.18:0/2617412011 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f56fc00fd20 con 0x7f57141037a0 2024-04-12T21:24:33.760 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:33.763+0000 7f570aa00700 1 -- 172.21.15.18:0/2617412011 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f56fc0103a0 con 0x7f57141037a0 2024-04-12T21:24:33.760 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:33.763+0000 7f5719000700 1 -- 172.21.15.18:0/2617412011 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f571410ffb0 con 0x7f57141037a0 2024-04-12T21:24:33.760 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:33.763+0000 7f5719000700 1 -- 172.21.15.18:0/2617412011 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5714110570 con 0x7f57141037a0 2024-04-12T21:24:33.761 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:33.764+0000 7f5719000700 1 -- 172.21.15.18:0/2617412011 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f57140fe550 con 0x7f57141037a0 2024-04-12T21:24:33.761 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:33.765+0000 7f570aa00700 1 -- 172.21.15.18:0/2617412011 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7f56fc011400 con 0x7f57141037a0 2024-04-12T21:24:33.762 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:33.766+0000 7f570aa00700 1 --2- 172.21.15.18:0/2617412011 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f56f806ec10 0x7f56f80710c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:24:33.762 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:33.767+0000 7f570aa00700 1 -- 172.21.15.18:0/2617412011 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 6757+0+0 (secure 0 0 0) 0x7f56fc08e780 con 0x7f57141037a0 2024-04-12T21:24:33.763 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:33.767+0000 7f5711e00700 1 --2- 172.21.15.18:0/2617412011 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f56f806ec10 0x7f56f80710c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:24:33.763 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:33.768+0000 7f5711e00700 1 --2- 172.21.15.18:0/2617412011 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f56f806ec10 0x7f56f80710c0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f570400b3b0 tx=0x7f5704009e30 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:24:33.767 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:33.772+0000 7f570aa00700 1 -- 172.21.15.18:0/2617412011 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f56fc05afd0 con 0x7f57141037a0 2024-04-12T21:24:34.022 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:34.026+0000 7f5719000700 1 -- 172.21.15.18:0/2617412011 --> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f5714109fe0 con 0x7f56f806ec10 2024-04-12T21:24:34.024 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:34.028+0000 7f570aa00700 1 -- 172.21.15.18:0/2617412011 <== mgr.34102 v2:172.21.15.18:6800/3812891919 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f5714109fe0 con 0x7f56f806ec10 2024-04-12T21:24:34.026 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:34.031+0000 7f5719000700 1 -- 172.21.15.18:0/2617412011 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f56f806ec10 msgr2=0x7f56f80710c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:24:34.027 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:34.031+0000 7f5719000700 1 --2- 172.21.15.18:0/2617412011 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f56f806ec10 0x7f56f80710c0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f570400b3b0 tx=0x7f5704009e30 comp rx=0 tx=0).stop 2024-04-12T21:24:34.027 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:34.031+0000 7f5719000700 1 -- 172.21.15.18:0/2617412011 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f57141037a0 msgr2=0x7f5714116310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:24:34.027 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:34.031+0000 7f5719000700 1 --2- 172.21.15.18:0/2617412011 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f57141037a0 0x7f5714116310 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f56fc00b790 tx=0x7f56fc009220 comp rx=0 tx=0).stop 2024-04-12T21:24:34.027 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:34.031+0000 7f5719000700 1 -- 172.21.15.18:0/2617412011 shutdown_connections 2024-04-12T21:24:34.027 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:34.031+0000 7f5719000700 1 --2- 172.21.15.18:0/2617412011 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f56f806ec10 0x7f56f80710c0 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:24:34.027 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:34.031+0000 7f5719000700 1 --2- 172.21.15.18:0/2617412011 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5714102e00 0x7f5714115dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:24:34.027 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:34.031+0000 7f5719000700 1 --2- 172.21.15.18:0/2617412011 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f57141037a0 0x7f5714116310 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:24:34.027 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:34.031+0000 7f5719000700 1 -- 172.21.15.18:0/2617412011 >> 172.21.15.18:0/2617412011 conn(0x7f57140f89b0 msgr2=0x7f5714071c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:24:34.027 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:34.032+0000 7f5719000700 1 -- 172.21.15.18:0/2617412011 shutdown_connections 2024-04-12T21:24:34.028 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:34.032+0000 7f5719000700 1 -- 172.21.15.18:0/2617412011 wait complete. 2024-04-12T21:24:34.044 INFO:teuthology.orchestra.run.smithi018.stdout:true 2024-04-12T21:24:34.189 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:34.193+0000 7f3016200700 1 -- 172.21.15.18:0/2373744173 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3010109430 msgr2=0x7f3010109800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:24:34.189 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:34.193+0000 7f3016200700 1 --2- 172.21.15.18:0/2373744173 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3010109430 0x7f3010109800 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f3000004940 tx=0x7f3000009e30 comp rx=0 tx=0).stop 2024-04-12T21:24:34.189 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:34.193+0000 7f3016200700 1 -- 172.21.15.18:0/2373744173 shutdown_connections 2024-04-12T21:24:34.189 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:34.194+0000 7f3016200700 1 --2- 172.21.15.18:0/2373744173 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3010109dd0 0x7f3010103440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:24:34.189 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:34.194+0000 7f3016200700 1 --2- 172.21.15.18:0/2373744173 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3010109430 0x7f3010109800 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:24:34.190 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:34.194+0000 7f3016200700 1 -- 172.21.15.18:0/2373744173 >> 172.21.15.18:0/2373744173 conn(0x7f3010072c00 msgr2=0x7f3010075010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:24:34.190 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:34.194+0000 7f3016200700 1 -- 172.21.15.18:0/2373744173 shutdown_connections 2024-04-12T21:24:34.190 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:34.194+0000 7f3016200700 1 -- 172.21.15.18:0/2373744173 wait complete. 2024-04-12T21:24:34.191 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:34.194+0000 7f3016200700 1 Processor -- start 2024-04-12T21:24:34.191 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:34.195+0000 7f3016200700 1 -- start start 2024-04-12T21:24:34.191 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:34.195+0000 7f3016200700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3010109430 0x7f3010119f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:24:34.191 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:34.195+0000 7f3016200700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3010109dd0 0x7f301011a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:24:34.191 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:34.195+0000 7f3016200700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f301011aad0 con 0x7f3010109430 2024-04-12T21:24:34.191 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:34.195+0000 7f3016200700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3010114100 con 0x7f3010109dd0 2024-04-12T21:24:34.191 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:34.195+0000 7f300f400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3010109430 0x7f3010119f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:24:34.191 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:34.195+0000 7f300f400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3010109430 0x7f3010119f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:34890/0 (socket says 172.21.15.18:34890) 2024-04-12T21:24:34.191 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:34.195+0000 7f300f400700 1 -- 172.21.15.18:0/770969044 learned_addr learned my addr 172.21.15.18:0/770969044 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:24:34.192 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:34.196+0000 7f300f400700 1 -- 172.21.15.18:0/770969044 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3010109dd0 msgr2=0x7f301011a480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:24:34.192 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:34.196+0000 7f300f400700 1 --2- 172.21.15.18:0/770969044 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3010109dd0 0x7f301011a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:24:34.192 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:34.196+0000 7f300f400700 1 -- 172.21.15.18:0/770969044 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3000005040 con 0x7f3010109430 2024-04-12T21:24:34.192 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:34.196+0000 7f300f400700 1 --2- 172.21.15.18:0/770969044 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3010109430 0x7f3010119f40 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f3000009150 tx=0x7f3000009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:24:34.193 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:34.196+0000 7f3007400700 1 -- 172.21.15.18:0/770969044 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f300000e040 con 0x7f3010109430 2024-04-12T21:24:34.193 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:34.197+0000 7f3007400700 1 -- 172.21.15.18:0/770969044 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f3000013420 con 0x7f3010109430 2024-04-12T21:24:34.193 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:34.197+0000 7f3016200700 1 -- 172.21.15.18:0/770969044 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3010114380 con 0x7f3010109430 2024-04-12T21:24:34.193 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:34.197+0000 7f3007400700 1 -- 172.21.15.18:0/770969044 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f300001c630 con 0x7f3010109430 2024-04-12T21:24:34.193 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:34.197+0000 7f3016200700 1 -- 172.21.15.18:0/770969044 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3010114940 con 0x7f3010109430 2024-04-12T21:24:34.193 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:34.198+0000 7f3016200700 1 -- 172.21.15.18:0/770969044 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3010104b80 con 0x7f3010109430 2024-04-12T21:24:34.194 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:34.199+0000 7f3007400700 1 -- 172.21.15.18:0/770969044 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7f3000013590 con 0x7f3010109430 2024-04-12T21:24:34.195 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:34.200+0000 7f3007400700 1 --2- 172.21.15.18:0/770969044 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f2ff406ec00 0x7f2ff40710b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:24:34.195 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:34.200+0000 7f3007400700 1 -- 172.21.15.18:0/770969044 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 6757+0+0 (secure 0 0 0) 0x7f300008fc70 con 0x7f3010109430 2024-04-12T21:24:34.196 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:34.200+0000 7f3007e00700 1 --2- 172.21.15.18:0/770969044 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f2ff406ec00 0x7f2ff40710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:24:34.196 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:34.201+0000 7f3007e00700 1 --2- 172.21.15.18:0/770969044 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f2ff406ec00 0x7f2ff40710b0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f30101157e0 tx=0x7f2ff8009ce0 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:24:34.200 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:34.205+0000 7f3007400700 1 -- 172.21.15.18:0/770969044 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f3000096080 con 0x7f3010109430 2024-04-12T21:24:34.456 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:34.461+0000 7f3016200700 1 -- 172.21.15.18:0/770969044 --> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f30101105c0 con 0x7f2ff406ec00 2024-04-12T21:24:34.458 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:34.463+0000 7f3007400700 1 -- 172.21.15.18:0/770969044 <== mgr.34102 v2:172.21.15.18:6800/3812891919 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f2ff800f040 con 0x7f2ff406ec00 2024-04-12T21:24:34.461 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:34.466+0000 7f3016200700 1 -- 172.21.15.18:0/770969044 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f2ff406ec00 msgr2=0x7f2ff40710b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:24:34.462 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:34.466+0000 7f3016200700 1 --2- 172.21.15.18:0/770969044 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f2ff406ec00 0x7f2ff40710b0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f30101157e0 tx=0x7f2ff8009ce0 comp rx=0 tx=0).stop 2024-04-12T21:24:34.462 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:34.466+0000 7f3016200700 1 -- 172.21.15.18:0/770969044 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3010109430 msgr2=0x7f3010119f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:24:34.462 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:34.466+0000 7f3016200700 1 --2- 172.21.15.18:0/770969044 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3010109430 0x7f3010119f40 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f3000009150 tx=0x7f3000009e30 comp rx=0 tx=0).stop 2024-04-12T21:24:34.462 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:34.466+0000 7f3016200700 1 -- 172.21.15.18:0/770969044 shutdown_connections 2024-04-12T21:24:34.462 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:34.466+0000 7f3016200700 1 --2- 172.21.15.18:0/770969044 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f2ff406ec00 0x7f2ff40710b0 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:24:34.462 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:34.466+0000 7f3016200700 1 --2- 172.21.15.18:0/770969044 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3010109430 0x7f3010119f40 unknown :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:24:34.462 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:34.466+0000 7f3016200700 1 --2- 172.21.15.18:0/770969044 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3010109dd0 0x7f301011a480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:24:34.462 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:34.466+0000 7f3016200700 1 -- 172.21.15.18:0/770969044 >> 172.21.15.18:0/770969044 conn(0x7f3010072c00 msgr2=0x7f3010074c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:24:34.462 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:34.466+0000 7f3016200700 1 -- 172.21.15.18:0/770969044 shutdown_connections 2024-04-12T21:24:34.462 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:34.466+0000 7f3016200700 1 -- 172.21.15.18:0/770969044 wait complete. 2024-04-12T21:24:34.616 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:34.620+0000 7fc53c200700 1 -- 172.21.15.18:0/266264229 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc534105a30 msgr2=0x7fc5340ff0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:24:34.616 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:34.620+0000 7fc53c200700 1 --2- 172.21.15.18:0/266264229 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc534105a30 0x7fc5340ff0a0 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7fc520004940 tx=0x7fc520009e30 comp rx=0 tx=0).stop 2024-04-12T21:24:34.616 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:34.620+0000 7fc53c200700 1 -- 172.21.15.18:0/266264229 shutdown_connections 2024-04-12T21:24:34.616 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:34.620+0000 7fc53c200700 1 --2- 172.21.15.18:0/266264229 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc534105a30 0x7fc5340ff0a0 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:24:34.616 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:34.620+0000 7fc53c200700 1 --2- 172.21.15.18:0/266264229 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc534105090 0x7fc534105460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:24:34.616 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:34.620+0000 7fc53c200700 1 -- 172.21.15.18:0/266264229 >> 172.21.15.18:0/266264229 conn(0x7fc5340fac00 msgr2=0x7fc5340fd010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:24:34.616 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:34.620+0000 7fc53c200700 1 -- 172.21.15.18:0/266264229 shutdown_connections 2024-04-12T21:24:34.616 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:34.620+0000 7fc53c200700 1 -- 172.21.15.18:0/266264229 wait complete. 2024-04-12T21:24:34.617 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:34.621+0000 7fc53c200700 1 Processor -- start 2024-04-12T21:24:34.617 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:34.621+0000 7fc53c200700 1 -- start start 2024-04-12T21:24:34.617 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:34.622+0000 7fc53c200700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc534105090 0x7fc534115b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:24:34.618 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:34.622+0000 7fc53c200700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc534105a30 0x7fc534116090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:24:34.618 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:34.622+0000 7fc53c200700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5341166e0 con 0x7fc534105a30 2024-04-12T21:24:34.618 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:34.622+0000 7fc53c200700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc53410fcc0 con 0x7fc534105090 2024-04-12T21:24:34.618 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:34.622+0000 7fc538e00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc534105a30 0x7fc534116090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:24:34.618 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:34.622+0000 7fc538e00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc534105a30 0x7fc534116090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:34904/0 (socket says 172.21.15.18:34904) 2024-04-12T21:24:34.618 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:34.622+0000 7fc538e00700 1 -- 172.21.15.18:0/1545518051 learned_addr learned my addr 172.21.15.18:0/1545518051 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:24:34.619 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:34.622+0000 7fc539800700 1 --2- 172.21.15.18:0/1545518051 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc534105090 0x7fc534115b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:24:34.619 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:34.622+0000 7fc538e00700 1 -- 172.21.15.18:0/1545518051 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc534105090 msgr2=0x7fc534115b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:24:34.619 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:34.623+0000 7fc538e00700 1 --2- 172.21.15.18:0/1545518051 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc534105090 0x7fc534115b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:24:34.619 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:34.623+0000 7fc538e00700 1 -- 172.21.15.18:0/1545518051 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc520005040 con 0x7fc534105a30 2024-04-12T21:24:34.619 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:34.623+0000 7fc538e00700 1 --2- 172.21.15.18:0/1545518051 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc534105a30 0x7fc534116090 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7fc520009100 tx=0x7fc520009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:24:34.619 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:34.623+0000 7fc531600700 1 -- 172.21.15.18:0/1545518051 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc52000e040 con 0x7fc534105a30 2024-04-12T21:24:34.619 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:34.623+0000 7fc531600700 1 -- 172.21.15.18:0/1545518051 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc520013420 con 0x7fc534105a30 2024-04-12T21:24:34.620 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:34.623+0000 7fc531600700 1 -- 172.21.15.18:0/1545518051 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc52001c630 con 0x7fc534105a30 2024-04-12T21:24:34.620 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:34.623+0000 7fc53c200700 1 -- 172.21.15.18:0/1545518051 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc53410ff40 con 0x7fc534105a30 2024-04-12T21:24:34.620 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:34.623+0000 7fc53c200700 1 -- 172.21.15.18:0/1545518051 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc534110500 con 0x7fc534105a30 2024-04-12T21:24:34.620 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:34.624+0000 7fc53c200700 1 -- 172.21.15.18:0/1545518051 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc534107560 con 0x7fc534105a30 2024-04-12T21:24:34.621 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:34.626+0000 7fc531600700 1 -- 172.21.15.18:0/1545518051 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7fc520013590 con 0x7fc534105a30 2024-04-12T21:24:34.622 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:34.627+0000 7fc531600700 1 --2- 172.21.15.18:0/1545518051 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fc51c06ec00 0x7fc51c0710b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:24:34.622 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:34.627+0000 7fc531600700 1 -- 172.21.15.18:0/1545518051 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 6757+0+0 (secure 0 0 0) 0x7fc52008fba0 con 0x7fc534105a30 2024-04-12T21:24:34.622 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:34.627+0000 7fc539800700 1 --2- 172.21.15.18:0/1545518051 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fc51c06ec00 0x7fc51c0710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:24:34.623 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:34.628+0000 7fc539800700 1 --2- 172.21.15.18:0/1545518051 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fc51c06ec00 0x7fc51c0710b0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fc5280041d0 tx=0x7fc528009040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:24:34.626 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:34.631+0000 7fc531600700 1 -- 172.21.15.18:0/1545518051 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7fc52005c3d0 con 0x7fc534105a30 2024-04-12T21:24:34.779 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:24:34 smithi186 ceph-mon[86379]: pgmap v744: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:24:34.850 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:24:34 smithi018 ceph-mon[112590]: pgmap v744: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:24:34.870 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:34.874+0000 7fc53c200700 1 -- 172.21.15.18:0/1545518051 --> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fc534111240 con 0x7fc51c06ec00 2024-04-12T21:24:34.899 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:34.904+0000 7fc531600700 1 -- 172.21.15.18:0/1545518051 <== mgr.34102 v2:172.21.15.18:6800/3812891919 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7fc534111240 con 0x7fc51c06ec00 2024-04-12T21:24:34.900 INFO:teuthology.orchestra.run.smithi018.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-12T21:24:34.900 INFO:teuthology.orchestra.run.smithi018.stdout:alertmanager.smithi018 smithi018 *:9093,9094 running (25m) 89s ago 39m 15.5M - 0.25.0 c8568f914cd2 c332090068e8 2024-04-12T21:24:34.900 INFO:teuthology.orchestra.run.smithi018.stdout:crash.smithi018 smithi018 running (21m) 89s ago 39m 6665k - 19.0.0-2879-g324baf8d 8717720dc5e5 b7f80d1f7982 2024-04-12T21:24:34.900 INFO:teuthology.orchestra.run.smithi018.stdout:crash.smithi186 smithi186 running (21m) 92s ago 38m 6660k - 19.0.0-2879-g324baf8d 8717720dc5e5 a4f55c81b623 2024-04-12T21:24:34.900 INFO:teuthology.orchestra.run.smithi018.stdout:grafana.smithi018 smithi018 *:3000 running (24m) 89s ago 39m 74.8M - 9.4.12 2bacad6d85d8 b08e3bf176dd 2024-04-12T21:24:34.900 INFO:teuthology.orchestra.run.smithi018.stdout:mds.cephfs.smithi018.acqbsc smithi018 running (17m) 89s ago 32m 16.7M - 19.0.0-2879-g324baf8d 8717720dc5e5 021f99ab24ae 2024-04-12T21:24:34.900 INFO:teuthology.orchestra.run.smithi018.stdout:mds.cephfs.smithi018.szjluo smithi018 running (16m) 89s ago 32m 16.5M - 19.0.0-2879-g324baf8d 8717720dc5e5 0003117479d4 2024-04-12T21:24:34.901 INFO:teuthology.orchestra.run.smithi018.stdout:mds.cephfs.smithi186.jdllyn smithi186 running (98s) 92s ago 32m 11.0M - 19.0.0-2879-g324baf8d 8717720dc5e5 21b1f46eef23 2024-04-12T21:24:34.901 INFO:teuthology.orchestra.run.smithi018.stdout:mds.cephfs.smithi186.wjnafs smithi186 running (32m) 92s ago 32m 19.5M - 17.2.7-827-gd3b5060a 21de9b7369ad 74b115d6d6fd 2024-04-12T21:24:34.901 INFO:teuthology.orchestra.run.smithi018.stdout:mgr.smithi018.xjzwga smithi018 *:8443,9283,8765 running (27m) 89s ago 41m 512M - 19.0.0-2879-g324baf8d 8717720dc5e5 56088305eab7 2024-04-12T21:24:34.901 INFO:teuthology.orchestra.run.smithi018.stdout:mgr.smithi186.kqcfxk smithi186 *:8443,9283,8765 running (26m) 92s ago 38m 459M - 19.0.0-2879-g324baf8d 8717720dc5e5 09cd766e5e0d 2024-04-12T21:24:34.901 INFO:teuthology.orchestra.run.smithi018.stdout:mon.smithi018 smithi018 running (23m) 89s ago 41m 73.1M 2048M 19.0.0-2879-g324baf8d 8717720dc5e5 df3e201dfed6 2024-04-12T21:24:34.901 INFO:teuthology.orchestra.run.smithi018.stdout:mon.smithi186 smithi186 running (22m) 92s ago 38m 47.2M 2048M 19.0.0-2879-g324baf8d 8717720dc5e5 c77498da9b2b 2024-04-12T21:24:34.901 INFO:teuthology.orchestra.run.smithi018.stdout:node-exporter.smithi018 smithi018 *:9100 running (26m) 89s ago 39m 8108k - 1.5.0 0da6a335fe13 326c3d9c6847 2024-04-12T21:24:34.901 INFO:teuthology.orchestra.run.smithi018.stdout:node-exporter.smithi186 smithi186 *:9100 running (26m) 92s ago 38m 8757k - 1.5.0 0da6a335fe13 ec2112baeecf 2024-04-12T21:24:34.901 INFO:teuthology.orchestra.run.smithi018.stdout:osd.0 smithi018 running (20m) 89s ago 37m 123M 1559M 19.0.0-2879-g324baf8d 8717720dc5e5 60b1273ca047 2024-04-12T21:24:34.901 INFO:teuthology.orchestra.run.smithi018.stdout:osd.1 smithi018 running (20m) 89s ago 36m 95.1M 1559M 19.0.0-2879-g324baf8d 8717720dc5e5 170ce1e5fb8d 2024-04-12T21:24:34.902 INFO:teuthology.orchestra.run.smithi018.stdout:osd.2 smithi018 running (19m) 89s ago 36m 94.3M 1559M 19.0.0-2879-g324baf8d 8717720dc5e5 c2cbc6aa2839 2024-04-12T21:24:34.902 INFO:teuthology.orchestra.run.smithi018.stdout:osd.3 smithi186 running (19m) 92s ago 35m 101M 2581M 19.0.0-2879-g324baf8d 8717720dc5e5 0138e8262e4b 2024-04-12T21:24:34.902 INFO:teuthology.orchestra.run.smithi018.stdout:osd.4 smithi186 running (18m) 92s ago 34m 91.0M 2581M 19.0.0-2879-g324baf8d 8717720dc5e5 b20cb63b25c9 2024-04-12T21:24:34.902 INFO:teuthology.orchestra.run.smithi018.stdout:osd.5 smithi186 running (17m) 92s ago 34m 92.0M 2581M 19.0.0-2879-g324baf8d 8717720dc5e5 2513bea2610e 2024-04-12T21:24:34.902 INFO:teuthology.orchestra.run.smithi018.stdout:prometheus.smithi018 smithi018 *:9095 running (25m) 89s ago 39m 67.9M - 2.43.0 a07b618ecd1d f58841127e21 2024-04-12T21:24:34.903 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:34.908+0000 7fc53c200700 1 -- 172.21.15.18:0/1545518051 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fc51c06ec00 msgr2=0x7fc51c0710b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:24:34.904 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:34.908+0000 7fc53c200700 1 --2- 172.21.15.18:0/1545518051 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fc51c06ec00 0x7fc51c0710b0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fc5280041d0 tx=0x7fc528009040 comp rx=0 tx=0).stop 2024-04-12T21:24:34.904 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:34.908+0000 7fc53c200700 1 -- 172.21.15.18:0/1545518051 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc534105a30 msgr2=0x7fc534116090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:24:34.904 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:34.908+0000 7fc53c200700 1 --2- 172.21.15.18:0/1545518051 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc534105a30 0x7fc534116090 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7fc520009100 tx=0x7fc520009e30 comp rx=0 tx=0).stop 2024-04-12T21:24:34.904 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:34.908+0000 7fc53c200700 1 -- 172.21.15.18:0/1545518051 shutdown_connections 2024-04-12T21:24:34.904 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:34.908+0000 7fc53c200700 1 --2- 172.21.15.18:0/1545518051 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fc51c06ec00 0x7fc51c0710b0 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:24:34.904 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:34.908+0000 7fc53c200700 1 --2- 172.21.15.18:0/1545518051 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc534105090 0x7fc534115b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:24:34.904 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:34.908+0000 7fc53c200700 1 --2- 172.21.15.18:0/1545518051 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc534105a30 0x7fc534116090 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:24:34.904 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:34.908+0000 7fc53c200700 1 -- 172.21.15.18:0/1545518051 >> 172.21.15.18:0/1545518051 conn(0x7fc5340fac00 msgr2=0x7fc5340fc9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:24:34.904 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:34.908+0000 7fc53c200700 1 -- 172.21.15.18:0/1545518051 shutdown_connections 2024-04-12T21:24:34.904 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:34.908+0000 7fc53c200700 1 -- 172.21.15.18:0/1545518051 wait complete. 2024-04-12T21:24:35.051 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:35.055+0000 7f1bdec00700 1 -- 172.21.15.18:0/4081555858 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1bd81059c0 msgr2=0x7f1bd80ff080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:24:35.051 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:35.055+0000 7f1bdec00700 1 --2- 172.21.15.18:0/4081555858 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1bd81059c0 0x7f1bd80ff080 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f1bc0004940 tx=0x7f1bc0009e30 comp rx=0 tx=0).stop 2024-04-12T21:24:35.051 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:35.055+0000 7f1bdec00700 1 -- 172.21.15.18:0/4081555858 shutdown_connections 2024-04-12T21:24:35.051 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:35.055+0000 7f1bdec00700 1 --2- 172.21.15.18:0/4081555858 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1bd81059c0 0x7f1bd80ff080 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:24:35.051 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:35.055+0000 7f1bdec00700 1 --2- 172.21.15.18:0/4081555858 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1bd8105020 0x7f1bd81053f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:24:35.051 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:35.055+0000 7f1bdec00700 1 -- 172.21.15.18:0/4081555858 >> 172.21.15.18:0/4081555858 conn(0x7f1bd80faba0 msgr2=0x7f1bd80fcfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:24:35.052 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:35.056+0000 7f1bdec00700 1 -- 172.21.15.18:0/4081555858 shutdown_connections 2024-04-12T21:24:35.052 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:35.056+0000 7f1bdec00700 1 -- 172.21.15.18:0/4081555858 wait complete. 2024-04-12T21:24:35.052 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:35.056+0000 7f1bdec00700 1 Processor -- start 2024-04-12T21:24:35.053 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:35.057+0000 7f1bdec00700 1 -- start start 2024-04-12T21:24:35.053 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:35.057+0000 7f1bdec00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1bd8105020 0x7f1bd8115ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:24:35.053 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:35.057+0000 7f1bdec00700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1bd81059c0 0x7f1bd81160e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:24:35.053 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:35.057+0000 7f1bdec00700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1bd8116730 con 0x7f1bd8105020 2024-04-12T21:24:35.053 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:35.057+0000 7f1bdec00700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1bd810fd10 con 0x7f1bd81059c0 2024-04-12T21:24:35.053 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:35.057+0000 7f1bd7e00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1bd8105020 0x7f1bd8115ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:24:35.053 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:35.057+0000 7f1bd7e00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1bd8105020 0x7f1bd8115ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:34922/0 (socket says 172.21.15.18:34922) 2024-04-12T21:24:35.053 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:35.057+0000 7f1bd7e00700 1 -- 172.21.15.18:0/3928728454 learned_addr learned my addr 172.21.15.18:0/3928728454 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:24:35.054 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:35.058+0000 7f1bcfe00700 1 --2- 172.21.15.18:0/3928728454 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1bd81059c0 0x7f1bd81160e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:24:35.054 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:35.058+0000 7f1bd7e00700 1 -- 172.21.15.18:0/3928728454 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1bd81059c0 msgr2=0x7f1bd81160e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:24:35.054 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:35.058+0000 7f1bd7e00700 1 --2- 172.21.15.18:0/3928728454 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1bd81059c0 0x7f1bd81160e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:24:35.054 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:35.058+0000 7f1bd7e00700 1 -- 172.21.15.18:0/3928728454 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1bc0005040 con 0x7f1bd8105020 2024-04-12T21:24:35.055 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:35.058+0000 7f1bcfe00700 1 --2- 172.21.15.18:0/3928728454 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1bd81059c0 0x7f1bd81160e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-12T21:24:35.055 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:35.058+0000 7f1bd7e00700 1 --2- 172.21.15.18:0/3928728454 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1bd8105020 0x7f1bd8115ba0 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f1bc800c8c0 tx=0x7f1bc80044c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:24:35.055 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:35.058+0000 7f1bd4c00700 1 -- 172.21.15.18:0/3928728454 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1bc800a980 con 0x7f1bd8105020 2024-04-12T21:24:35.055 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:35.058+0000 7f1bd4c00700 1 -- 172.21.15.18:0/3928728454 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f1bc800ce60 con 0x7f1bd8105020 2024-04-12T21:24:35.055 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:35.058+0000 7f1bdec00700 1 -- 172.21.15.18:0/3928728454 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1bd810fff0 con 0x7f1bd8105020 2024-04-12T21:24:35.055 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:35.058+0000 7f1bd4c00700 1 -- 172.21.15.18:0/3928728454 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1bc80115e0 con 0x7f1bd8105020 2024-04-12T21:24:35.055 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:35.059+0000 7f1bdec00700 1 -- 172.21.15.18:0/3928728454 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1bd81105b0 con 0x7f1bd8105020 2024-04-12T21:24:35.055 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:35.060+0000 7f1bdec00700 1 -- 172.21.15.18:0/3928728454 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1bd8100770 con 0x7f1bd8105020 2024-04-12T21:24:35.056 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:35.061+0000 7f1bd4c00700 1 -- 172.21.15.18:0/3928728454 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7f1bc8012400 con 0x7f1bd8105020 2024-04-12T21:24:35.057 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:35.061+0000 7f1bd4c00700 1 --2- 172.21.15.18:0/3928728454 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f1bbc06ec00 0x7f1bbc0710b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:24:35.057 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:35.062+0000 7f1bd4c00700 1 -- 172.21.15.18:0/3928728454 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 6757+0+0 (secure 0 0 0) 0x7f1bc808eb30 con 0x7f1bd8105020 2024-04-12T21:24:35.057 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:35.062+0000 7f1bcfe00700 1 --2- 172.21.15.18:0/3928728454 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f1bbc06ec00 0x7f1bbc0710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:24:35.058 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:35.063+0000 7f1bcfe00700 1 --2- 172.21.15.18:0/3928728454 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f1bbc06ec00 0x7f1bbc0710b0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f1bc000b3b0 tx=0x7f1bc0009e30 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:24:35.062 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:35.066+0000 7f1bd4c00700 1 -- 172.21.15.18:0/3928728454 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f1bc805b2e0 con 0x7f1bd8105020 2024-04-12T21:24:35.375 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:35.379+0000 7f1bdec00700 1 -- 172.21.15.18:0/3928728454 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f1bd81111f0 con 0x7f1bd8105020 2024-04-12T21:24:35.375 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:35.380+0000 7f1bd4c00700 1 -- 172.21.15.18:0/3928728454 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+837 (secure 0 0 0) 0x7f1bc80126b0 con 0x7f1bd8105020 2024-04-12T21:24:35.376 INFO:teuthology.orchestra.run.smithi018.stdout:{ 2024-04-12T21:24:35.376 INFO:teuthology.orchestra.run.smithi018.stdout: "mon": { 2024-04-12T21:24:35.376 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 19.0.0-2879-g324baf8d (324baf8ddb91dea46cac50cb535c306115791355) squid (dev)": 2 2024-04-12T21:24:35.376 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-12T21:24:35.376 INFO:teuthology.orchestra.run.smithi018.stdout: "mgr": { 2024-04-12T21:24:35.376 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 19.0.0-2879-g324baf8d (324baf8ddb91dea46cac50cb535c306115791355) squid (dev)": 2 2024-04-12T21:24:35.376 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-12T21:24:35.376 INFO:teuthology.orchestra.run.smithi018.stdout: "osd": { 2024-04-12T21:24:35.376 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 19.0.0-2879-g324baf8d (324baf8ddb91dea46cac50cb535c306115791355) squid (dev)": 6 2024-04-12T21:24:35.377 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-12T21:24:35.377 INFO:teuthology.orchestra.run.smithi018.stdout: "mds": { 2024-04-12T21:24:35.377 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 17.2.7-827-gd3b5060a (d3b5060a17449edfae9385cdf792afec109fca5d) quincy (stable)": 1, 2024-04-12T21:24:35.377 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 19.0.0-2879-g324baf8d (324baf8ddb91dea46cac50cb535c306115791355) squid (dev)": 3 2024-04-12T21:24:35.377 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-12T21:24:35.377 INFO:teuthology.orchestra.run.smithi018.stdout: "overall": { 2024-04-12T21:24:35.377 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 17.2.7-827-gd3b5060a (d3b5060a17449edfae9385cdf792afec109fca5d) quincy (stable)": 1, 2024-04-12T21:24:35.377 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 19.0.0-2879-g324baf8d (324baf8ddb91dea46cac50cb535c306115791355) squid (dev)": 13 2024-04-12T21:24:35.377 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-04-12T21:24:35.377 INFO:teuthology.orchestra.run.smithi018.stdout:} 2024-04-12T21:24:35.378 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:35.383+0000 7f1bdec00700 1 -- 172.21.15.18:0/3928728454 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f1bbc06ec00 msgr2=0x7f1bbc0710b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:24:35.378 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:35.383+0000 7f1bdec00700 1 --2- 172.21.15.18:0/3928728454 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f1bbc06ec00 0x7f1bbc0710b0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f1bc000b3b0 tx=0x7f1bc0009e30 comp rx=0 tx=0).stop 2024-04-12T21:24:35.378 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:35.383+0000 7f1bdec00700 1 -- 172.21.15.18:0/3928728454 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1bd8105020 msgr2=0x7f1bd8115ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:24:35.378 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:35.383+0000 7f1bdec00700 1 --2- 172.21.15.18:0/3928728454 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1bd8105020 0x7f1bd8115ba0 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f1bc800c8c0 tx=0x7f1bc80044c0 comp rx=0 tx=0).stop 2024-04-12T21:24:35.379 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:35.383+0000 7f1bdec00700 1 -- 172.21.15.18:0/3928728454 shutdown_connections 2024-04-12T21:24:35.379 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:35.383+0000 7f1bdec00700 1 --2- 172.21.15.18:0/3928728454 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f1bbc06ec00 0x7f1bbc0710b0 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:24:35.379 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:35.383+0000 7f1bdec00700 1 --2- 172.21.15.18:0/3928728454 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1bd8105020 0x7f1bd8115ba0 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:24:35.379 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:35.383+0000 7f1bdec00700 1 --2- 172.21.15.18:0/3928728454 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1bd81059c0 0x7f1bd81160e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:24:35.380 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:35.383+0000 7f1bdec00700 1 -- 172.21.15.18:0/3928728454 >> 172.21.15.18:0/3928728454 conn(0x7f1bd80faba0 msgr2=0x7f1bd80fbb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:24:35.380 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:35.384+0000 7f1bdec00700 1 -- 172.21.15.18:0/3928728454 shutdown_connections 2024-04-12T21:24:35.380 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:35.384+0000 7f1bdec00700 1 -- 172.21.15.18:0/3928728454 wait complete. 2024-04-12T21:24:35.528 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:35.533+0000 7f1414800700 1 -- 172.21.15.18:0/2709635575 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f140c1059c0 msgr2=0x7f140c0ff080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:24:35.528 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:35.533+0000 7f1414800700 1 --2- 172.21.15.18:0/2709635575 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f140c1059c0 0x7f140c0ff080 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f13fc004940 tx=0x7f13fc009e30 comp rx=0 tx=0).stop 2024-04-12T21:24:35.529 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:35.533+0000 7f1414800700 1 -- 172.21.15.18:0/2709635575 shutdown_connections 2024-04-12T21:24:35.529 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:35.533+0000 7f1414800700 1 --2- 172.21.15.18:0/2709635575 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f140c1059c0 0x7f140c0ff080 unknown :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:24:35.529 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:35.533+0000 7f1414800700 1 --2- 172.21.15.18:0/2709635575 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f140c105020 0x7f140c1053f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:24:35.529 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:35.533+0000 7f1414800700 1 -- 172.21.15.18:0/2709635575 >> 172.21.15.18:0/2709635575 conn(0x7f140c0faba0 msgr2=0x7f140c0fcfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:24:35.529 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:35.533+0000 7f1414800700 1 -- 172.21.15.18:0/2709635575 shutdown_connections 2024-04-12T21:24:35.529 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:35.533+0000 7f1414800700 1 -- 172.21.15.18:0/2709635575 wait complete. 2024-04-12T21:24:35.530 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:35.534+0000 7f1414800700 1 Processor -- start 2024-04-12T21:24:35.530 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:35.534+0000 7f1414800700 1 -- start start 2024-04-12T21:24:35.530 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:35.534+0000 7f1414800700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f140c105020 0x7f140c115b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:24:35.530 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:35.534+0000 7f1414800700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f140c1059c0 0x7f140c116050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:24:35.530 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:35.534+0000 7f1414800700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f140c116730 con 0x7f140c105020 2024-04-12T21:24:35.531 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:35.534+0000 7f1414800700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f140c10fcc0 con 0x7f140c1059c0 2024-04-12T21:24:35.531 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:35.535+0000 7f1411e00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f140c105020 0x7f140c115b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:24:35.531 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:35.535+0000 7f1411e00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f140c105020 0x7f140c115b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:34940/0 (socket says 172.21.15.18:34940) 2024-04-12T21:24:35.531 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:35.535+0000 7f1411e00700 1 -- 172.21.15.18:0/885156589 learned_addr learned my addr 172.21.15.18:0/885156589 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:24:35.531 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:35.535+0000 7f1411e00700 1 -- 172.21.15.18:0/885156589 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f140c1059c0 msgr2=0x7f140c116050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:24:35.531 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:35.535+0000 7f1411e00700 1 --2- 172.21.15.18:0/885156589 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f140c1059c0 0x7f140c116050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:24:35.531 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:35.535+0000 7f1411e00700 1 -- 172.21.15.18:0/885156589 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13fc005040 con 0x7f140c105020 2024-04-12T21:24:35.532 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:35.535+0000 7f1411e00700 1 --2- 172.21.15.18:0/885156589 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f140c105020 0x7f140c115b10 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f140000b790 tx=0x7f1400009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:24:35.532 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:35.536+0000 7f1406000700 1 -- 172.21.15.18:0/885156589 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f140000f770 con 0x7f140c105020 2024-04-12T21:24:35.532 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:35.536+0000 7f1406000700 1 -- 172.21.15.18:0/885156589 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f140000fda0 con 0x7f140c105020 2024-04-12T21:24:35.532 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:35.536+0000 7f1414800700 1 -- 172.21.15.18:0/885156589 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f140c10ffa0 con 0x7f140c105020 2024-04-12T21:24:35.532 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:35.536+0000 7f1406000700 1 -- 172.21.15.18:0/885156589 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f140000d310 con 0x7f140c105020 2024-04-12T21:24:35.532 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:35.536+0000 7f1414800700 1 -- 172.21.15.18:0/885156589 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f140c110560 con 0x7f140c105020 2024-04-12T21:24:35.532 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:35.537+0000 7f1414800700 1 -- 172.21.15.18:0/885156589 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f140c100770 con 0x7f140c105020 2024-04-12T21:24:35.533 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:35.538+0000 7f1406000700 1 -- 172.21.15.18:0/885156589 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7f1400010420 con 0x7f140c105020 2024-04-12T21:24:35.534 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:35.539+0000 7f1406000700 1 --2- 172.21.15.18:0/885156589 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f13f406ec00 0x7f13f40710b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:24:35.534 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:35.539+0000 7f1406000700 1 -- 172.21.15.18:0/885156589 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 6757+0+0 (secure 0 0 0) 0x7f140008e9a0 con 0x7f140c105020 2024-04-12T21:24:35.534 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:35.539+0000 7f1411400700 1 --2- 172.21.15.18:0/885156589 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f13f406ec00 0x7f13f40710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:24:35.535 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:35.540+0000 7f1411400700 1 --2- 172.21.15.18:0/885156589 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f13f406ec00 0x7f13f40710b0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f13fc004f90 tx=0x7f13fc009e30 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:24:35.539 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:35.544+0000 7f1406000700 1 -- 172.21.15.18:0/885156589 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f14000576a0 con 0x7f140c105020 2024-04-12T21:24:35.779 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:24:35 smithi186 ceph-mon[86379]: from='client.35316 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:24:35.779 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:24:35 smithi186 ceph-mon[86379]: from='client.35320 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:24:35.779 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:24:35 smithi186 ceph-mon[86379]: from='client.? 172.21.15.18:0/3928728454' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T21:24:35.818 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:35.822+0000 7f1414800700 1 -- 172.21.15.18:0/885156589 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f140c04c710 con 0x7f140c105020 2024-04-12T21:24:35.818 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:24:35 smithi018 ceph-mon[112590]: from='client.35316 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:24:35.818 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:24:35 smithi018 ceph-mon[112590]: from='client.35320 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:24:35.818 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:24:35 smithi018 ceph-mon[112590]: from='client.? 172.21.15.18:0/3928728454' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T21:24:35.819 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:35.823+0000 7f1406000700 1 -- 172.21.15.18:0/885156589 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 31 v31) v1 ==== 76+0+1873 (secure 0 0 0) 0x7f140005ad60 con 0x7f140c105020 2024-04-12T21:24:35.819 INFO:teuthology.orchestra.run.smithi018.stdout:e31 2024-04-12T21:24:35.819 INFO:teuthology.orchestra.run.smithi018.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-12T21:24:35.819 INFO:teuthology.orchestra.run.smithi018.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-12T21:24:35.819 INFO:teuthology.orchestra.run.smithi018.stdout:legacy client fscid: 1 2024-04-12T21:24:35.819 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-12T21:24:35.819 INFO:teuthology.orchestra.run.smithi018.stdout:Filesystem 'cephfs' (1) 2024-04-12T21:24:35.819 INFO:teuthology.orchestra.run.smithi018.stdout:fs_name cephfs 2024-04-12T21:24:35.819 INFO:teuthology.orchestra.run.smithi018.stdout:epoch 24 2024-04-12T21:24:35.820 INFO:teuthology.orchestra.run.smithi018.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-04-12T21:24:35.820 INFO:teuthology.orchestra.run.smithi018.stdout:created 2024-04-12T20:51:57.362850+0000 2024-04-12T21:24:35.820 INFO:teuthology.orchestra.run.smithi018.stdout:modified 2024-04-12T21:06:59.717190+0000 2024-04-12T21:24:35.820 INFO:teuthology.orchestra.run.smithi018.stdout:tableserver 0 2024-04-12T21:24:35.820 INFO:teuthology.orchestra.run.smithi018.stdout:root 0 2024-04-12T21:24:35.820 INFO:teuthology.orchestra.run.smithi018.stdout:session_timeout 60 2024-04-12T21:24:35.820 INFO:teuthology.orchestra.run.smithi018.stdout:session_autoclose 300 2024-04-12T21:24:35.820 INFO:teuthology.orchestra.run.smithi018.stdout:max_file_size 1099511627776 2024-04-12T21:24:35.820 INFO:teuthology.orchestra.run.smithi018.stdout:max_xattr_size 65536 2024-04-12T21:24:35.820 INFO:teuthology.orchestra.run.smithi018.stdout:required_client_features {} 2024-04-12T21:24:35.820 INFO:teuthology.orchestra.run.smithi018.stdout:last_failure 0 2024-04-12T21:24:35.820 INFO:teuthology.orchestra.run.smithi018.stdout:last_failure_osd_epoch 100 2024-04-12T21:24:35.820 INFO:teuthology.orchestra.run.smithi018.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-12T21:24:35.820 INFO:teuthology.orchestra.run.smithi018.stdout:max_mds 2 2024-04-12T21:24:35.820 INFO:teuthology.orchestra.run.smithi018.stdout:in 0,1 2024-04-12T21:24:35.821 INFO:teuthology.orchestra.run.smithi018.stdout:up {} 2024-04-12T21:24:35.821 INFO:teuthology.orchestra.run.smithi018.stdout:failed 0,1 2024-04-12T21:24:35.821 INFO:teuthology.orchestra.run.smithi018.stdout:damaged 2024-04-12T21:24:35.821 INFO:teuthology.orchestra.run.smithi018.stdout:stopped 2024-04-12T21:24:35.821 INFO:teuthology.orchestra.run.smithi018.stdout:data_pools [3] 2024-04-12T21:24:35.821 INFO:teuthology.orchestra.run.smithi018.stdout:metadata_pool 2 2024-04-12T21:24:35.821 INFO:teuthology.orchestra.run.smithi018.stdout:inline_data disabled 2024-04-12T21:24:35.821 INFO:teuthology.orchestra.run.smithi018.stdout:balancer 2024-04-12T21:24:35.821 INFO:teuthology.orchestra.run.smithi018.stdout:bal_rank_mask -1 2024-04-12T21:24:35.821 INFO:teuthology.orchestra.run.smithi018.stdout:standby_count_wanted 1 2024-04-12T21:24:35.821 INFO:teuthology.orchestra.run.smithi018.stdout:qdb_cluster leader: 0 members: 2024-04-12T21:24:35.821 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-12T21:24:35.821 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-12T21:24:35.821 INFO:teuthology.orchestra.run.smithi018.stdout:Standby daemons: 2024-04-12T21:24:35.821 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-12T21:24:35.821 INFO:teuthology.orchestra.run.smithi018.stdout:[mds.cephfs.smithi186.wjnafs{-1:24277} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.186:6826/438347207,v1:172.21.15.186:6827/438347207] compat {c=[1],r=[1],i=[7ff]}] 2024-04-12T21:24:35.822 INFO:teuthology.orchestra.run.smithi018.stdout:[mds.cephfs.smithi018.acqbsc{-1:34436} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.18:6828/755179764,v1:172.21.15.18:6829/755179764] compat {c=[1],r=[1],i=[1fff]}] 2024-04-12T21:24:35.822 INFO:teuthology.orchestra.run.smithi018.stdout:[mds.cephfs.smithi018.szjluo{-1:34468} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.18:6826/2407992653,v1:172.21.15.18:6827/2407992653] compat {c=[1],r=[1],i=[1fff]}] 2024-04-12T21:24:35.822 INFO:teuthology.orchestra.run.smithi018.stdout:[mds.cephfs.smithi186.jdllyn{-1:44595} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.186:6824/3523444202,v1:172.21.15.186:6825/3523444202] compat {c=[1],r=[1],i=[1fff]}] 2024-04-12T21:24:35.822 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:35.827+0000 7f1414800700 1 -- 172.21.15.18:0/885156589 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f13f406ec00 msgr2=0x7f13f40710b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:24:35.822 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:35.827+0000 7f1414800700 1 --2- 172.21.15.18:0/885156589 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f13f406ec00 0x7f13f40710b0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f13fc004f90 tx=0x7f13fc009e30 comp rx=0 tx=0).stop 2024-04-12T21:24:35.823 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:35.827+0000 7f1414800700 1 -- 172.21.15.18:0/885156589 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f140c105020 msgr2=0x7f140c115b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:24:35.823 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:35.827+0000 7f1414800700 1 --2- 172.21.15.18:0/885156589 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f140c105020 0x7f140c115b10 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f140000b790 tx=0x7f1400009250 comp rx=0 tx=0).stop 2024-04-12T21:24:35.823 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:35.827+0000 7f1414800700 1 -- 172.21.15.18:0/885156589 shutdown_connections 2024-04-12T21:24:35.823 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:35.827+0000 7f1414800700 1 --2- 172.21.15.18:0/885156589 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f13f406ec00 0x7f13f40710b0 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:24:35.823 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:35.827+0000 7f1414800700 1 --2- 172.21.15.18:0/885156589 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f140c105020 0x7f140c115b10 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:24:35.823 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:35.827+0000 7f1414800700 1 --2- 172.21.15.18:0/885156589 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f140c1059c0 0x7f140c116050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:24:35.823 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:35.827+0000 7f1414800700 1 -- 172.21.15.18:0/885156589 >> 172.21.15.18:0/885156589 conn(0x7f140c0faba0 msgr2=0x7f140c0fbaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:24:35.824 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:35.828+0000 7f1414800700 1 -- 172.21.15.18:0/885156589 shutdown_connections 2024-04-12T21:24:35.824 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:35.828+0000 7f1414800700 1 -- 172.21.15.18:0/885156589 wait complete. 2024-04-12T21:24:35.825 INFO:teuthology.orchestra.run.smithi018.stderr:dumped fsmap epoch 31 2024-04-12T21:24:35.983 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:35.988+0000 7fe303e00700 1 -- 172.21.15.18:0/1286122792 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe2fc105020 msgr2=0x7fe2fc1053f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:24:35.984 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:35.988+0000 7fe303e00700 1 --2- 172.21.15.18:0/1286122792 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe2fc105020 0x7fe2fc1053f0 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7fe2f0004940 tx=0x7fe2f0009e30 comp rx=0 tx=0).stop 2024-04-12T21:24:35.984 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:35.988+0000 7fe303e00700 1 -- 172.21.15.18:0/1286122792 shutdown_connections 2024-04-12T21:24:35.984 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:35.988+0000 7fe303e00700 1 --2- 172.21.15.18:0/1286122792 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe2fc1059c0 0x7fe2fc0ff080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:24:35.984 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:35.988+0000 7fe303e00700 1 --2- 172.21.15.18:0/1286122792 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe2fc105020 0x7fe2fc1053f0 unknown :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:24:35.984 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:35.988+0000 7fe303e00700 1 -- 172.21.15.18:0/1286122792 >> 172.21.15.18:0/1286122792 conn(0x7fe2fc0faba0 msgr2=0x7fe2fc0fcfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:24:35.984 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:35.988+0000 7fe303e00700 1 -- 172.21.15.18:0/1286122792 shutdown_connections 2024-04-12T21:24:35.984 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:35.988+0000 7fe303e00700 1 -- 172.21.15.18:0/1286122792 wait complete. 2024-04-12T21:24:35.986 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:35.989+0000 7fe303e00700 1 Processor -- start 2024-04-12T21:24:35.986 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:35.989+0000 7fe303e00700 1 -- start start 2024-04-12T21:24:35.986 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:35.989+0000 7fe303e00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe2fc105020 0x7fe2fc115b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:24:35.986 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:35.989+0000 7fe303e00700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe2fc1059c0 0x7fe2fc1160d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:24:35.986 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:35.989+0000 7fe303e00700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2fc116720 con 0x7fe2fc105020 2024-04-12T21:24:35.986 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:35.989+0000 7fe303e00700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2fc10fd00 con 0x7fe2fc1059c0 2024-04-12T21:24:35.986 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:35.990+0000 7fe301400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe2fc105020 0x7fe2fc115b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:24:35.986 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:35.990+0000 7fe301400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe2fc105020 0x7fe2fc115b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:34962/0 (socket says 172.21.15.18:34962) 2024-04-12T21:24:35.986 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:35.990+0000 7fe301400700 1 -- 172.21.15.18:0/3884081766 learned_addr learned my addr 172.21.15.18:0/3884081766 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:24:35.986 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:35.990+0000 7fe301400700 1 -- 172.21.15.18:0/3884081766 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe2fc1059c0 msgr2=0x7fe2fc1160d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:24:35.986 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:35.990+0000 7fe300a00700 1 --2- 172.21.15.18:0/3884081766 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe2fc1059c0 0x7fe2fc1160d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:24:35.986 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:35.990+0000 7fe301400700 1 --2- 172.21.15.18:0/3884081766 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe2fc1059c0 0x7fe2fc1160d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:24:35.987 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:35.990+0000 7fe301400700 1 -- 172.21.15.18:0/3884081766 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2f0005040 con 0x7fe2fc105020 2024-04-12T21:24:35.987 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:35.990+0000 7fe300a00700 1 --2- 172.21.15.18:0/3884081766 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe2fc1059c0 0x7fe2fc1160d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-12T21:24:35.987 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:35.991+0000 7fe301400700 1 --2- 172.21.15.18:0/3884081766 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe2fc105020 0x7fe2fc115b90 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7fe2f000b8b0 tx=0x7fe2f0009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:24:35.987 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:35.991+0000 7fe2f9600700 1 -- 172.21.15.18:0/3884081766 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe2f000e040 con 0x7fe2fc105020 2024-04-12T21:24:35.988 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:35.991+0000 7fe2f9600700 1 -- 172.21.15.18:0/3884081766 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe2f0004290 con 0x7fe2fc105020 2024-04-12T21:24:35.988 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:35.991+0000 7fe303e00700 1 -- 172.21.15.18:0/3884081766 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe2fc10ff80 con 0x7fe2fc105020 2024-04-12T21:24:35.988 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:35.991+0000 7fe2f9600700 1 -- 172.21.15.18:0/3884081766 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe2f001c3f0 con 0x7fe2fc105020 2024-04-12T21:24:35.988 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:35.991+0000 7fe303e00700 1 -- 172.21.15.18:0/3884081766 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe2fc110540 con 0x7fe2fc105020 2024-04-12T21:24:35.988 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:35.992+0000 7fe303e00700 1 -- 172.21.15.18:0/3884081766 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe2fc100770 con 0x7fe2fc105020 2024-04-12T21:24:35.988 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:35.993+0000 7fe2f9600700 1 -- 172.21.15.18:0/3884081766 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7fe2f00137b0 con 0x7fe2fc105020 2024-04-12T21:24:35.990 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:35.995+0000 7fe2f9600700 1 --2- 172.21.15.18:0/3884081766 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fe2e406ec00 0x7fe2e40710b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:24:35.990 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:35.995+0000 7fe2f9600700 1 -- 172.21.15.18:0/3884081766 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 6757+0+0 (secure 0 0 0) 0x7fe2f008fbe0 con 0x7fe2fc105020 2024-04-12T21:24:35.990 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:35.995+0000 7fe300a00700 1 --2- 172.21.15.18:0/3884081766 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fe2e406ec00 0x7fe2e40710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:24:35.991 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:35.996+0000 7fe300a00700 1 --2- 172.21.15.18:0/3884081766 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fe2e406ec00 0x7fe2e40710b0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fe2e8007fd0 tx=0x7fe2e8009040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:24:35.995 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:36.000+0000 7fe2f9600700 1 -- 172.21.15.18:0/3884081766 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7fe2f005c4d0 con 0x7fe2fc105020 2024-04-12T21:24:36.249 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:36.253+0000 7fe303e00700 1 -- 172.21.15.18:0/3884081766 --> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fe2fc111280 con 0x7fe2e406ec00 2024-04-12T21:24:36.251 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:36.256+0000 7fe2f9600700 1 -- 172.21.15.18:0/3884081766 <== mgr.34102 v2:172.21.15.18:6800/3812891919 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7fe2fc111280 con 0x7fe2e406ec00 2024-04-12T21:24:36.252 INFO:teuthology.orchestra.run.smithi018.stdout:{ 2024-04-12T21:24:36.252 INFO:teuthology.orchestra.run.smithi018.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:f4452def3be4d350f4a8e5feaf7ebb1511be9298fe68478ba7a2d6fbd7d69a63", 2024-04-12T21:24:36.252 INFO:teuthology.orchestra.run.smithi018.stdout: "in_progress": true, 2024-04-12T21:24:36.252 INFO:teuthology.orchestra.run.smithi018.stdout: "which": "Upgrading all daemon types on all hosts", 2024-04-12T21:24:36.252 INFO:teuthology.orchestra.run.smithi018.stdout: "services_complete": [ 2024-04-12T21:24:36.252 INFO:teuthology.orchestra.run.smithi018.stdout: "crash", 2024-04-12T21:24:36.252 INFO:teuthology.orchestra.run.smithi018.stdout: "osd", 2024-04-12T21:24:36.252 INFO:teuthology.orchestra.run.smithi018.stdout: "mon", 2024-04-12T21:24:36.252 INFO:teuthology.orchestra.run.smithi018.stdout: "mgr" 2024-04-12T21:24:36.252 INFO:teuthology.orchestra.run.smithi018.stdout: ], 2024-04-12T21:24:36.253 INFO:teuthology.orchestra.run.smithi018.stdout: "progress": "15/21 daemons upgraded", 2024-04-12T21:24:36.253 INFO:teuthology.orchestra.run.smithi018.stdout: "message": "Currently upgrading mds daemons", 2024-04-12T21:24:36.253 INFO:teuthology.orchestra.run.smithi018.stdout: "is_paused": false 2024-04-12T21:24:36.253 INFO:teuthology.orchestra.run.smithi018.stdout:} 2024-04-12T21:24:36.255 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:36.259+0000 7fe303e00700 1 -- 172.21.15.18:0/3884081766 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fe2e406ec00 msgr2=0x7fe2e40710b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:24:36.255 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:36.259+0000 7fe303e00700 1 --2- 172.21.15.18:0/3884081766 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fe2e406ec00 0x7fe2e40710b0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fe2e8007fd0 tx=0x7fe2e8009040 comp rx=0 tx=0).stop 2024-04-12T21:24:36.255 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:36.259+0000 7fe303e00700 1 -- 172.21.15.18:0/3884081766 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe2fc105020 msgr2=0x7fe2fc115b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:24:36.255 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:36.259+0000 7fe303e00700 1 --2- 172.21.15.18:0/3884081766 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe2fc105020 0x7fe2fc115b90 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7fe2f000b8b0 tx=0x7fe2f0009e30 comp rx=0 tx=0).stop 2024-04-12T21:24:36.255 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:36.259+0000 7fe303e00700 1 -- 172.21.15.18:0/3884081766 shutdown_connections 2024-04-12T21:24:36.255 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:36.259+0000 7fe303e00700 1 --2- 172.21.15.18:0/3884081766 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fe2e406ec00 0x7fe2e40710b0 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:24:36.255 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:36.259+0000 7fe303e00700 1 --2- 172.21.15.18:0/3884081766 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe2fc105020 0x7fe2fc115b90 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:24:36.255 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:36.259+0000 7fe303e00700 1 --2- 172.21.15.18:0/3884081766 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe2fc1059c0 0x7fe2fc1160d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:24:36.255 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:36.259+0000 7fe303e00700 1 -- 172.21.15.18:0/3884081766 >> 172.21.15.18:0/3884081766 conn(0x7fe2fc0faba0 msgr2=0x7fe2fc0fbb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:24:36.255 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:36.259+0000 7fe303e00700 1 -- 172.21.15.18:0/3884081766 shutdown_connections 2024-04-12T21:24:36.255 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:36.259+0000 7fe303e00700 1 -- 172.21.15.18:0/3884081766 wait complete. 2024-04-12T21:24:36.409 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:36.413+0000 7fc1ab200700 1 -- 172.21.15.18:0/605615204 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc1ac104e70 msgr2=0x7fc1ac105240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:24:36.409 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:36.413+0000 7fc1ab200700 1 --2- 172.21.15.18:0/605615204 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc1ac104e70 0x7fc1ac105240 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7fc19c004940 tx=0x7fc19c009e30 comp rx=0 tx=0).stop 2024-04-12T21:24:36.409 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:36.413+0000 7fc1ab200700 1 -- 172.21.15.18:0/605615204 shutdown_connections 2024-04-12T21:24:36.409 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:36.413+0000 7fc1ab200700 1 --2- 172.21.15.18:0/605615204 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc1ac105810 0x7fc1ac0feed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:24:36.410 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:36.413+0000 7fc1ab200700 1 --2- 172.21.15.18:0/605615204 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc1ac104e70 0x7fc1ac105240 unknown :-1 s=CLOSED pgs=638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:24:36.410 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:36.413+0000 7fc1ab200700 1 -- 172.21.15.18:0/605615204 >> 172.21.15.18:0/605615204 conn(0x7fc1ac0fab10 msgr2=0x7fc1ac0fcef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:24:36.410 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:36.414+0000 7fc1ab200700 1 -- 172.21.15.18:0/605615204 shutdown_connections 2024-04-12T21:24:36.410 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:36.414+0000 7fc1ab200700 1 -- 172.21.15.18:0/605615204 wait complete. 2024-04-12T21:24:36.412 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:36.414+0000 7fc1ab200700 1 Processor -- start 2024-04-12T21:24:36.412 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:36.415+0000 7fc1ab200700 1 -- start start 2024-04-12T21:24:36.412 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:36.415+0000 7fc1ab200700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc1ac105810 0x7fc1ac115d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:24:36.412 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:36.415+0000 7fc1ab200700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc1ac116280 0x7fc1ac10fca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:24:36.412 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:36.415+0000 7fc1ab200700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1ac116800 con 0x7fc1ac116280 2024-04-12T21:24:36.412 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:36.415+0000 7fc1ab200700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1ac116970 con 0x7fc1ac105810 2024-04-12T21:24:36.412 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:36.415+0000 7fc1a9400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc1ac116280 0x7fc1ac10fca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:24:36.412 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:36.415+0000 7fc1a9400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc1ac116280 0x7fc1ac10fca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:34986/0 (socket says 172.21.15.18:34986) 2024-04-12T21:24:36.412 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:36.415+0000 7fc1a9400700 1 -- 172.21.15.18:0/1645308055 learned_addr learned my addr 172.21.15.18:0/1645308055 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:24:36.412 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:36.416+0000 7fc1a9e00700 1 --2- 172.21.15.18:0/1645308055 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc1ac105810 0x7fc1ac115d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:24:36.412 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:36.416+0000 7fc1a9400700 1 -- 172.21.15.18:0/1645308055 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc1ac105810 msgr2=0x7fc1ac115d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:24:36.412 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:36.416+0000 7fc1a9400700 1 --2- 172.21.15.18:0/1645308055 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc1ac105810 0x7fc1ac115d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:24:36.413 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:36.416+0000 7fc1a9400700 1 -- 172.21.15.18:0/1645308055 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc19c005040 con 0x7fc1ac116280 2024-04-12T21:24:36.413 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:36.416+0000 7fc1a9400700 1 --2- 172.21.15.18:0/1645308055 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc1ac116280 0x7fc1ac10fca0 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7fc198002a50 tx=0x7fc198004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:24:36.413 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:36.416+0000 7fc1a2000700 1 -- 172.21.15.18:0/1645308055 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc198007cc0 con 0x7fc1ac116280 2024-04-12T21:24:36.413 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:36.416+0000 7fc1a2000700 1 -- 172.21.15.18:0/1645308055 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc19800f040 con 0x7fc1ac116280 2024-04-12T21:24:36.413 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:36.416+0000 7fc1ab200700 1 -- 172.21.15.18:0/1645308055 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc1ac1102a0 con 0x7fc1ac116280 2024-04-12T21:24:36.413 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:36.416+0000 7fc1a2000700 1 -- 172.21.15.18:0/1645308055 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc19800daf0 con 0x7fc1ac116280 2024-04-12T21:24:36.413 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:36.416+0000 7fc1ab200700 1 -- 172.21.15.18:0/1645308055 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc1ac110860 con 0x7fc1ac116280 2024-04-12T21:24:36.413 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:36.417+0000 7fc1ab200700 1 -- 172.21.15.18:0/1645308055 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc1ac1005c0 con 0x7fc1ac116280 2024-04-12T21:24:36.414 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:36.419+0000 7fc1a2000700 1 -- 172.21.15.18:0/1645308055 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7fc198009810 con 0x7fc1ac116280 2024-04-12T21:24:36.415 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:36.420+0000 7fc1a2000700 1 --2- 172.21.15.18:0/1645308055 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fc19006ec00 0x7fc1900710b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:24:36.415 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:36.420+0000 7fc1a2000700 1 -- 172.21.15.18:0/1645308055 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 6757+0+0 (secure 0 0 0) 0x7fc19808e700 con 0x7fc1ac116280 2024-04-12T21:24:36.415 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:36.420+0000 7fc1a9e00700 1 --2- 172.21.15.18:0/1645308055 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fc19006ec00 0x7fc1900710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:24:36.416 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:36.421+0000 7fc1a9e00700 1 --2- 172.21.15.18:0/1645308055 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fc19006ec00 0x7fc1900710b0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fc19c000c00 tx=0x7fc19c009e30 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:24:36.420 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:36.425+0000 7fc1a2000700 1 -- 172.21.15.18:0/1645308055 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7fc19805aeb0 con 0x7fc1ac116280 2024-04-12T21:24:36.730 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:36.735+0000 7fc1ab200700 1 -- 172.21.15.18:0/1645308055 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7fc1ac10bf70 con 0x7fc1ac116280 2024-04-12T21:24:36.731 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:36.736+0000 7fc1a2000700 1 -- 172.21.15.18:0/1645308055 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+4303 (secure 0 0 0) 0x7fc19805aa40 con 0x7fc1ac116280 2024-04-12T21:24:36.731 INFO:teuthology.orchestra.run.smithi018.stdout:HEALTH_ERR failed to probe daemons or devices; 1 filesystem is degraded; 1 filesystem is offline 2024-04-12T21:24:36.732 INFO:teuthology.orchestra.run.smithi018.stdout:[WRN] CEPHADM_REFRESH_FAILED: failed to probe daemons or devices 2024-04-12T21:24:36.732 INFO:teuthology.orchestra.run.smithi018.stdout: host smithi018 `cephadm ceph-volume` failed: cephadm exited with an error code: 1, stderr: Inferring config /var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/mon.smithi018/config 2024-04-12T21:24:36.732 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/podman: timeout after 895 seconds 2024-04-12T21:24:36.732 INFO:teuthology.orchestra.run.smithi018.stdout:Non-zero exit code 124 from /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:f4452def3be4d350f4a8e5feaf7ebb1511be9298fe68478ba7a2d6fbd7d69a63 -e NODE_NAME=smithi018 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968:/var/run/ceph:z -v /var/log/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968:/var/log/ceph:z -v /var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmpxsmwkkuh:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:f4452def3be4d350f4a8e5feaf7ebb1511be9298fe68478ba7a2d6fbd7d69a63 inventory --format=json-pretty --filter-for-batch 2024-04-12T21:24:36.732 INFO:teuthology.orchestra.run.smithi018.stdout:Traceback (most recent call last): 2024-04-12T21:24:36.732 INFO:teuthology.orchestra.run.smithi018.stdout: File "/usr/lib64/python3.9/runpy.py", line 197, in _run_module_as_main 2024-04-12T21:24:36.732 INFO:teuthology.orchestra.run.smithi018.stdout: return _run_code(code, main_globals, None, 2024-04-12T21:24:36.732 INFO:teuthology.orchestra.run.smithi018.stdout: File "/usr/lib64/python3.9/runpy.py", line 87, in _run_code 2024-04-12T21:24:36.732 INFO:teuthology.orchestra.run.smithi018.stdout: exec(code, run_globals) 2024-04-12T21:24:36.732 INFO:teuthology.orchestra.run.smithi018.stdout: File "/var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/cephadm.e44f3ed5e34e4ef52eedef233d1190bff004470d8e8005127b5312bff357fa7f/__main__.py", line 5618, in 2024-04-12T21:24:36.732 INFO:teuthology.orchestra.run.smithi018.stdout: File "/var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/cephadm.e44f3ed5e34e4ef52eedef233d1190bff004470d8e8005127b5312bff357fa7f/__main__.py", line 5606, in main 2024-04-12T21:24:36.732 INFO:teuthology.orchestra.run.smithi018.stdout: File "/var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/cephadm.e44f3ed5e34e4ef52eedef233d1190bff004470d8e8005127b5312bff357fa7f/__main__.py", line 408, in _infer_config 2024-04-12T21:24:36.732 INFO:teuthology.orchestra.run.smithi018.stdout: File "/var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/cephadm.e44f3ed5e34e4ef52eedef233d1190bff004470d8e8005127b5312bff357fa7f/__main__.py", line 323, in _infer_fsid 2024-04-12T21:24:36.732 INFO:teuthology.orchestra.run.smithi018.stdout: File "/var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/cephadm.e44f3ed5e34e4ef52eedef233d1190bff004470d8e8005127b5312bff357fa7f/__main__.py", line 436, in _infer_image 2024-04-12T21:24:36.733 INFO:teuthology.orchestra.run.smithi018.stdout: File "/var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/cephadm.e44f3ed5e34e4ef52eedef233d1190bff004470d8e8005127b5312bff357fa7f/__main__.py", line 310, in _validate_fsid 2024-04-12T21:24:36.733 INFO:teuthology.orchestra.run.smithi018.stdout: File "/var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/cephadm.e44f3ed5e34e4ef52eedef233d1190bff004470d8e8005127b5312bff357fa7f/__main__.py", line 3359, in command_ceph_volume 2024-04-12T21:24:36.733 INFO:teuthology.orchestra.run.smithi018.stdout: File "/var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/cephadm.e44f3ed5e34e4ef52eedef233d1190bff004470d8e8005127b5312bff357fa7f/cephadmlib/call_wrappers.py", line 310, in call_throws 2024-04-12T21:24:36.733 INFO:teuthology.orchestra.run.smithi018.stdout:RuntimeError: Failed command: /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:f4452def3be4d350f4a8e5feaf7ebb1511be9298fe68478ba7a2d6fbd7d69a63 -e NODE_NAME=smithi018 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968:/var/run/ceph:z -v /var/log/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968:/var/log/ceph:z -v /var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmpxsmwkkuh:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:f4452def3be4d350f4a8e5feaf7ebb1511be9298fe68478ba7a2d6fbd7d69a63 inventory --format=json-pretty --filter-for-batch 2024-04-12T21:24:36.733 INFO:teuthology.orchestra.run.smithi018.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-04-12T21:24:36.733 INFO:teuthology.orchestra.run.smithi018.stdout: fs cephfs is degraded 2024-04-12T21:24:36.733 INFO:teuthology.orchestra.run.smithi018.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-04-12T21:24:36.733 INFO:teuthology.orchestra.run.smithi018.stdout: fs cephfs is offline because no MDS is active for it. 2024-04-12T21:24:36.735 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:36.740+0000 7fc1ab200700 1 -- 172.21.15.18:0/1645308055 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fc19006ec00 msgr2=0x7fc1900710b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:24:36.735 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:36.740+0000 7fc1ab200700 1 --2- 172.21.15.18:0/1645308055 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fc19006ec00 0x7fc1900710b0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fc19c000c00 tx=0x7fc19c009e30 comp rx=0 tx=0).stop 2024-04-12T21:24:36.735 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:36.740+0000 7fc1ab200700 1 -- 172.21.15.18:0/1645308055 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc1ac116280 msgr2=0x7fc1ac10fca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:24:36.736 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:36.740+0000 7fc1ab200700 1 --2- 172.21.15.18:0/1645308055 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc1ac116280 0x7fc1ac10fca0 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7fc198002a50 tx=0x7fc198004600 comp rx=0 tx=0).stop 2024-04-12T21:24:36.736 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:36.740+0000 7fc1ab200700 1 -- 172.21.15.18:0/1645308055 shutdown_connections 2024-04-12T21:24:36.736 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:36.740+0000 7fc1ab200700 1 --2- 172.21.15.18:0/1645308055 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fc19006ec00 0x7fc1900710b0 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:24:36.736 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:36.740+0000 7fc1ab200700 1 --2- 172.21.15.18:0/1645308055 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc1ac105810 0x7fc1ac115d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:24:36.736 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:36.740+0000 7fc1ab200700 1 --2- 172.21.15.18:0/1645308055 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc1ac116280 0x7fc1ac10fca0 unknown :-1 s=CLOSED pgs=639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:24:36.736 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:36.740+0000 7fc1ab200700 1 -- 172.21.15.18:0/1645308055 >> 172.21.15.18:0/1645308055 conn(0x7fc1ac0fab10 msgr2=0x7fc1ac0fcb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:24:36.736 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:36.740+0000 7fc1ab200700 1 -- 172.21.15.18:0/1645308055 shutdown_connections 2024-04-12T21:24:36.736 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:24:36.740+0000 7fc1ab200700 1 -- 172.21.15.18:0/1645308055 wait complete. 2024-04-12T21:24:36.779 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:24:36 smithi186 ceph-mon[86379]: from='client.35324 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:24:36.779 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:24:36 smithi186 ceph-mon[86379]: pgmap v745: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:24:36.779 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:24:36 smithi186 ceph-mon[86379]: from='client.? 172.21.15.18:0/885156589' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-12T21:24:36.850 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:24:36 smithi018 ceph-mon[112590]: from='client.35324 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:24:36.850 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:24:36 smithi018 ceph-mon[112590]: pgmap v745: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:24:36.850 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:24:36 smithi018 ceph-mon[112590]: from='client.? 172.21.15.18:0/885156589' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-12T21:24:37.779 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:24:37 smithi186 ceph-mon[86379]: from='client.35336 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:24:37.779 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:24:37 smithi186 ceph-mon[86379]: from='client.? 172.21.15.18:0/1645308055' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-12T21:24:37.850 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:24:37 smithi018 ceph-mon[112590]: from='client.35336 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:24:37.850 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:24:37 smithi018 ceph-mon[112590]: from='client.? 172.21.15.18:0/1645308055' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-12T21:24:38.779 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:24:38 smithi186 ceph-mon[86379]: pgmap v746: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:24:38.850 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:24:38 smithi018 ceph-mon[112590]: pgmap v746: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:24:40.779 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:24:40 smithi186 ceph-mon[86379]: pgmap v747: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:24:40.779 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:24:40 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T21:24:40.850 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:24:40 smithi018 ceph-mon[112590]: pgmap v747: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:24:40.850 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:24:40 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T21:24:42.850 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:24:42 smithi018 ceph-mon[112590]: pgmap v748: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:24:43.029 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:24:42 smithi186 ceph-mon[86379]: pgmap v748: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:24:44.850 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:24:44 smithi018 ceph-mon[112590]: pgmap v749: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:24:45.029 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:24:44 smithi186 ceph-mon[86379]: pgmap v749: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:24:46.850 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:24:46 smithi018 ceph-mon[112590]: pgmap v750: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:24:47.029 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:24:46 smithi186 ceph-mon[86379]: pgmap v750: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:24:48.850 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:24:48 smithi018 ceph-mon[112590]: pgmap v751: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:24:49.029 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:24:48 smithi186 ceph-mon[86379]: pgmap v751: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:24:50.850 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:24:50 smithi018 ceph-mon[112590]: pgmap v752: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:24:51.029 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:24:50 smithi186 ceph-mon[86379]: pgmap v752: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:24:52.850 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:24:52 smithi018 ceph-mon[112590]: pgmap v753: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:24:53.029 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:24:52 smithi186 ceph-mon[86379]: pgmap v753: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:24:54.850 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:24:54 smithi018 ceph-mon[112590]: pgmap v754: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:24:55.029 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:24:54 smithi186 ceph-mon[86379]: pgmap v754: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:24:55.850 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:24:55 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T21:24:56.029 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:24:55 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T21:24:56.850 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:24:56 smithi018 ceph-mon[112590]: pgmap v755: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:24:57.029 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:24:56 smithi186 ceph-mon[86379]: pgmap v755: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:24:58.850 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:24:58 smithi018 ceph-mon[112590]: pgmap v756: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:24:59.029 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:24:58 smithi186 ceph-mon[86379]: pgmap v756: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:25:00.850 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:25:00 smithi018 ceph-mon[112590]: pgmap v757: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:25:01.029 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:25:00 smithi186 ceph-mon[86379]: pgmap v757: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:25:03.029 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:25:02 smithi186 ceph-mon[86379]: pgmap v758: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:25:03.100 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:25:02 smithi018 ceph-mon[112590]: pgmap v758: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:25:05.030 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:25:04 smithi186 ceph-mon[86379]: pgmap v759: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:25:05.100 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:25:04 smithi018 ceph-mon[112590]: pgmap v759: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:25:06.909 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:06.912+0000 7f587e200700 1 -- 172.21.15.18:0/476010808 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5878105020 msgr2=0x7f58781053f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:25:06.909 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:06.912+0000 7f587e200700 1 --2- 172.21.15.18:0/476010808 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5878105020 0x7f58781053f0 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f5868004940 tx=0x7f5868009e30 comp rx=0 tx=0).stop 2024-04-12T21:25:06.909 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:06.912+0000 7f587e200700 1 -- 172.21.15.18:0/476010808 shutdown_connections 2024-04-12T21:25:06.909 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:06.912+0000 7f587e200700 1 --2- 172.21.15.18:0/476010808 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f58781059c0 0x7f58780ff080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:25:06.909 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:06.912+0000 7f587e200700 1 --2- 172.21.15.18:0/476010808 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5878105020 0x7f58781053f0 unknown :-1 s=CLOSED pgs=640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:25:06.910 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:06.912+0000 7f587e200700 1 -- 172.21.15.18:0/476010808 >> 172.21.15.18:0/476010808 conn(0x7f58780faba0 msgr2=0x7f58780fcfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:25:06.910 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:25:06 smithi018 ceph-mon[112590]: pgmap v760: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:25:06.912 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:06.912+0000 7f587e200700 1 -- 172.21.15.18:0/476010808 shutdown_connections 2024-04-12T21:25:06.912 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:06.912+0000 7f587e200700 1 -- 172.21.15.18:0/476010808 wait complete. 2024-04-12T21:25:06.912 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:06.913+0000 7f587e200700 1 Processor -- start 2024-04-12T21:25:06.912 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:06.913+0000 7f587e200700 1 -- start start 2024-04-12T21:25:06.912 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:06.914+0000 7f587e200700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5878105020 0x7f5878115b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:25:06.912 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:06.914+0000 7f587e200700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f58781059c0 0x7f58781160d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:25:06.913 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:06.914+0000 7f587e200700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5878116720 con 0x7f5878105020 2024-04-12T21:25:06.913 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:06.914+0000 7f587e200700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f587810fd00 con 0x7f58781059c0 2024-04-12T21:25:06.913 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:06.914+0000 7f5877400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5878105020 0x7f5878115b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:25:06.913 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:06.914+0000 7f5877400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5878105020 0x7f5878115b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:59194/0 (socket says 172.21.15.18:59194) 2024-04-12T21:25:06.913 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:06.914+0000 7f5877400700 1 -- 172.21.15.18:0/315057758 learned_addr learned my addr 172.21.15.18:0/315057758 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:25:06.913 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:06.914+0000 7f5876a00700 1 --2- 172.21.15.18:0/315057758 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f58781059c0 0x7f58781160d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:25:06.913 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:06.914+0000 7f5877400700 1 -- 172.21.15.18:0/315057758 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f58781059c0 msgr2=0x7f58781160d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:25:06.913 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:06.914+0000 7f5877400700 1 --2- 172.21.15.18:0/315057758 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f58781059c0 0x7f58781160d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:25:06.913 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:06.915+0000 7f5877400700 1 -- 172.21.15.18:0/315057758 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5868005040 con 0x7f5878105020 2024-04-12T21:25:06.913 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:06.915+0000 7f5877400700 1 --2- 172.21.15.18:0/315057758 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5878105020 0x7f5878115b90 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f5868009180 tx=0x7f5868009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:25:06.913 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:06.915+0000 7f586f400700 1 -- 172.21.15.18:0/315057758 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f586800e040 con 0x7f5878105020 2024-04-12T21:25:06.913 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:06.915+0000 7f586f400700 1 -- 172.21.15.18:0/315057758 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f586800bbc0 con 0x7f5878105020 2024-04-12T21:25:06.913 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:06.915+0000 7f587e200700 1 -- 172.21.15.18:0/315057758 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f587810ff80 con 0x7f5878105020 2024-04-12T21:25:06.914 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:06.915+0000 7f586f400700 1 -- 172.21.15.18:0/315057758 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f586801c630 con 0x7f5878105020 2024-04-12T21:25:06.914 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:06.915+0000 7f587e200700 1 -- 172.21.15.18:0/315057758 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5878110540 con 0x7f5878105020 2024-04-12T21:25:06.914 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:06.916+0000 7f587e200700 1 -- 172.21.15.18:0/315057758 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5878100770 con 0x7f5878105020 2024-04-12T21:25:06.914 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:06.918+0000 7f586f400700 1 -- 172.21.15.18:0/315057758 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7f586800bd30 con 0x7f5878105020 2024-04-12T21:25:06.915 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:06.918+0000 7f586f400700 1 --2- 172.21.15.18:0/315057758 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f585c06ec00 0x7f585c0710b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:25:06.915 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:06.919+0000 7f586f400700 1 -- 172.21.15.18:0/315057758 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 6757+0+0 (secure 0 0 0) 0x7f586808fd40 con 0x7f5878105020 2024-04-12T21:25:06.915 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:06.919+0000 7f5876a00700 1 --2- 172.21.15.18:0/315057758 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f585c06ec00 0x7f585c0710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:25:06.916 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:06.920+0000 7f5876a00700 1 --2- 172.21.15.18:0/315057758 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f585c06ec00 0x7f585c0710b0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f5878111430 tx=0x7f586000b040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:25:06.920 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:06.924+0000 7f586f400700 1 -- 172.21.15.18:0/315057758 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f586805c570 con 0x7f5878105020 2024-04-12T21:25:07.029 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:25:06 smithi186 ceph-mon[86379]: pgmap v760: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:25:07.174 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:07.177+0000 7f587e200700 1 -- 172.21.15.18:0/315057758 --> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f58781107b0 con 0x7f585c06ec00 2024-04-12T21:25:07.177 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:07.180+0000 7f586f400700 1 -- 172.21.15.18:0/315057758 <== mgr.34102 v2:172.21.15.18:6800/3812891919 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f58781107b0 con 0x7f585c06ec00 2024-04-12T21:25:07.180 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:07.183+0000 7f587e200700 1 -- 172.21.15.18:0/315057758 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f585c06ec00 msgr2=0x7f585c0710b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:25:07.180 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:07.183+0000 7f587e200700 1 --2- 172.21.15.18:0/315057758 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f585c06ec00 0x7f585c0710b0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f5878111430 tx=0x7f586000b040 comp rx=0 tx=0).stop 2024-04-12T21:25:07.180 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:07.183+0000 7f587e200700 1 -- 172.21.15.18:0/315057758 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5878105020 msgr2=0x7f5878115b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:25:07.180 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:07.183+0000 7f587e200700 1 --2- 172.21.15.18:0/315057758 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5878105020 0x7f5878115b90 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f5868009180 tx=0x7f5868009e30 comp rx=0 tx=0).stop 2024-04-12T21:25:07.180 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:07.183+0000 7f587e200700 1 -- 172.21.15.18:0/315057758 shutdown_connections 2024-04-12T21:25:07.180 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:07.183+0000 7f587e200700 1 --2- 172.21.15.18:0/315057758 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f585c06ec00 0x7f585c0710b0 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:25:07.180 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:07.183+0000 7f587e200700 1 --2- 172.21.15.18:0/315057758 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5878105020 0x7f5878115b90 unknown :-1 s=CLOSED pgs=641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:25:07.181 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:07.183+0000 7f587e200700 1 --2- 172.21.15.18:0/315057758 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f58781059c0 0x7f58781160d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:25:07.181 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:07.183+0000 7f587e200700 1 -- 172.21.15.18:0/315057758 >> 172.21.15.18:0/315057758 conn(0x7f58780faba0 msgr2=0x7f58780fbb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:25:07.181 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:07.184+0000 7f587e200700 1 -- 172.21.15.18:0/315057758 shutdown_connections 2024-04-12T21:25:07.181 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:07.184+0000 7f587e200700 1 -- 172.21.15.18:0/315057758 wait complete. 2024-04-12T21:25:07.198 INFO:teuthology.orchestra.run.smithi018.stdout:true 2024-04-12T21:25:07.343 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:07.346+0000 7fbe01800700 1 -- 172.21.15.18:0/2028797510 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fbdfc103780 msgr2=0x7fbdfc0fcdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:25:07.343 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:07.346+0000 7fbe01800700 1 --2- 172.21.15.18:0/2028797510 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fbdfc103780 0x7fbdfc0fcdf0 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7fbde8004940 tx=0x7fbde8009e30 comp rx=0 tx=0).stop 2024-04-12T21:25:07.344 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:07.346+0000 7fbe01800700 1 -- 172.21.15.18:0/2028797510 shutdown_connections 2024-04-12T21:25:07.344 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:07.346+0000 7fbe01800700 1 --2- 172.21.15.18:0/2028797510 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fbdfc103780 0x7fbdfc0fcdf0 unknown :-1 s=CLOSED pgs=642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:25:07.344 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:07.346+0000 7fbe01800700 1 --2- 172.21.15.18:0/2028797510 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbdfc102de0 0x7fbdfc1031b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:25:07.344 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:07.346+0000 7fbe01800700 1 -- 172.21.15.18:0/2028797510 >> 172.21.15.18:0/2028797510 conn(0x7fbdfc0f8950 msgr2=0x7fbdfc0fad60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:25:07.344 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:07.346+0000 7fbe01800700 1 -- 172.21.15.18:0/2028797510 shutdown_connections 2024-04-12T21:25:07.344 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:07.347+0000 7fbe01800700 1 -- 172.21.15.18:0/2028797510 wait complete. 2024-04-12T21:25:07.345 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:07.347+0000 7fbe01800700 1 Processor -- start 2024-04-12T21:25:07.345 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:07.348+0000 7fbe01800700 1 -- start start 2024-04-12T21:25:07.345 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:07.348+0000 7fbe01800700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fbdfc102de0 0x7fbdfc0702c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:25:07.345 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:07.348+0000 7fbe01800700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbdfc103780 0x7fbdfc070800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:25:07.345 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:07.348+0000 7fbe01800700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbdfc072210 con 0x7fbdfc102de0 2024-04-12T21:25:07.346 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:07.348+0000 7fbe01800700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbdfc070d40 con 0x7fbdfc103780 2024-04-12T21:25:07.346 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:07.348+0000 7fbdfaa00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fbdfc102de0 0x7fbdfc0702c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:25:07.346 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:07.349+0000 7fbdfaa00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fbdfc102de0 0x7fbdfc0702c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:59214/0 (socket says 172.21.15.18:59214) 2024-04-12T21:25:07.346 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:07.349+0000 7fbdfaa00700 1 -- 172.21.15.18:0/3556185523 learned_addr learned my addr 172.21.15.18:0/3556185523 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:25:07.346 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:07.349+0000 7fbdfa000700 1 --2- 172.21.15.18:0/3556185523 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbdfc103780 0x7fbdfc070800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:25:07.347 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:07.349+0000 7fbdfaa00700 1 -- 172.21.15.18:0/3556185523 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbdfc103780 msgr2=0x7fbdfc070800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:25:07.347 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:07.349+0000 7fbdfaa00700 1 --2- 172.21.15.18:0/3556185523 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbdfc103780 0x7fbdfc070800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:25:07.347 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:07.349+0000 7fbdfaa00700 1 -- 172.21.15.18:0/3556185523 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbde8005040 con 0x7fbdfc102de0 2024-04-12T21:25:07.347 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:07.349+0000 7fbdfaa00700 1 --2- 172.21.15.18:0/3556185523 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fbdfc102de0 0x7fbdfc0702c0 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7fbdec00c830 tx=0x7fbdec004470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:25:07.347 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:07.350+0000 7fbdf2a00700 1 -- 172.21.15.18:0/3556185523 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbdec00a760 con 0x7fbdfc102de0 2024-04-12T21:25:07.347 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:07.350+0000 7fbdf2a00700 1 -- 172.21.15.18:0/3556185523 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fbdec00ad90 con 0x7fbdfc102de0 2024-04-12T21:25:07.347 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:07.350+0000 7fbdf2a00700 1 -- 172.21.15.18:0/3556185523 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbdec0113a0 con 0x7fbdfc102de0 2024-04-12T21:25:07.347 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:07.350+0000 7fbe01800700 1 -- 172.21.15.18:0/3556185523 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbdfc071020 con 0x7fbdfc102de0 2024-04-12T21:25:07.347 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:07.350+0000 7fbe01800700 1 -- 172.21.15.18:0/3556185523 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbdfc11a120 con 0x7fbdfc102de0 2024-04-12T21:25:07.348 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:07.351+0000 7fbe01800700 1 -- 172.21.15.18:0/3556185523 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbdfc0fe530 con 0x7fbdfc102de0 2024-04-12T21:25:07.349 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:07.352+0000 7fbdf2a00700 1 -- 172.21.15.18:0/3556185523 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7fbdec012400 con 0x7fbdfc102de0 2024-04-12T21:25:07.350 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:07.353+0000 7fbdf2a00700 1 --2- 172.21.15.18:0/3556185523 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fbde006ec00 0x7fbde00710b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:25:07.350 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:07.353+0000 7fbdf2a00700 1 -- 172.21.15.18:0/3556185523 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 6757+0+0 (secure 0 0 0) 0x7fbdec08e900 con 0x7fbdfc102de0 2024-04-12T21:25:07.350 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:07.353+0000 7fbdfa000700 1 --2- 172.21.15.18:0/3556185523 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fbde006ec00 0x7fbde00710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:25:07.351 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:07.354+0000 7fbdfa000700 1 --2- 172.21.15.18:0/3556185523 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fbde006ec00 0x7fbde00710b0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fbdfc0719f0 tx=0x7fbde8009e30 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:25:07.355 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:07.358+0000 7fbdf2a00700 1 -- 172.21.15.18:0/3556185523 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7fbdec0576c0 con 0x7fbdfc102de0 2024-04-12T21:25:07.614 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:07.617+0000 7fbe01800700 1 -- 172.21.15.18:0/3556185523 --> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fbdfc0f9fd0 con 0x7fbde006ec00 2024-04-12T21:25:07.617 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:07.620+0000 7fbdf2a00700 1 -- 172.21.15.18:0/3556185523 <== mgr.34102 v2:172.21.15.18:6800/3812891919 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7fbdfc0f9fd0 con 0x7fbde006ec00 2024-04-12T21:25:07.620 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:07.624+0000 7fbe01800700 1 -- 172.21.15.18:0/3556185523 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fbde006ec00 msgr2=0x7fbde00710b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:25:07.621 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:07.624+0000 7fbe01800700 1 --2- 172.21.15.18:0/3556185523 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fbde006ec00 0x7fbde00710b0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fbdfc0719f0 tx=0x7fbde8009e30 comp rx=0 tx=0).stop 2024-04-12T21:25:07.621 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:07.624+0000 7fbe01800700 1 -- 172.21.15.18:0/3556185523 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fbdfc102de0 msgr2=0x7fbdfc0702c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:25:07.621 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:07.624+0000 7fbe01800700 1 --2- 172.21.15.18:0/3556185523 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fbdfc102de0 0x7fbdfc0702c0 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7fbdec00c830 tx=0x7fbdec004470 comp rx=0 tx=0).stop 2024-04-12T21:25:07.621 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:07.624+0000 7fbe01800700 1 -- 172.21.15.18:0/3556185523 shutdown_connections 2024-04-12T21:25:07.621 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:07.624+0000 7fbe01800700 1 --2- 172.21.15.18:0/3556185523 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fbde006ec00 0x7fbde00710b0 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:25:07.621 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:07.624+0000 7fbe01800700 1 --2- 172.21.15.18:0/3556185523 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fbdfc102de0 0x7fbdfc0702c0 unknown :-1 s=CLOSED pgs=643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:25:07.621 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:07.624+0000 7fbe01800700 1 --2- 172.21.15.18:0/3556185523 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbdfc103780 0x7fbdfc070800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:25:07.621 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:07.624+0000 7fbe01800700 1 -- 172.21.15.18:0/3556185523 >> 172.21.15.18:0/3556185523 conn(0x7fbdfc0f8950 msgr2=0x7fbdfc0f98c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:25:07.621 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:07.624+0000 7fbe01800700 1 -- 172.21.15.18:0/3556185523 shutdown_connections 2024-04-12T21:25:07.622 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:07.624+0000 7fbe01800700 1 -- 172.21.15.18:0/3556185523 wait complete. 2024-04-12T21:25:07.785 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:07.787+0000 7f9349800700 1 -- 172.21.15.18:0/288376767 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f93440708a0 msgr2=0x7f9344070d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:25:07.785 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:07.787+0000 7f9349800700 1 --2- 172.21.15.18:0/288376767 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f93440708a0 0x7f9344070d10 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f9330004970 tx=0x7f9330009e30 comp rx=0 tx=0).stop 2024-04-12T21:25:07.785 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:07.788+0000 7f9349800700 1 -- 172.21.15.18:0/288376767 shutdown_connections 2024-04-12T21:25:07.785 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:07.788+0000 7f9349800700 1 --2- 172.21.15.18:0/288376767 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f93440708a0 0x7f9344070d10 unknown :-1 s=CLOSED pgs=644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:25:07.785 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:07.788+0000 7f9349800700 1 --2- 172.21.15.18:0/288376767 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9344071f90 0x7f9344070360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:25:07.785 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:07.788+0000 7f9349800700 1 -- 172.21.15.18:0/288376767 >> 172.21.15.18:0/288376767 conn(0x7f93440f8970 msgr2=0x7f93440fad80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:25:07.785 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:07.788+0000 7f9349800700 1 -- 172.21.15.18:0/288376767 shutdown_connections 2024-04-12T21:25:07.785 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:07.788+0000 7f9349800700 1 -- 172.21.15.18:0/288376767 wait complete. 2024-04-12T21:25:07.787 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:07.789+0000 7f9349800700 1 Processor -- start 2024-04-12T21:25:07.787 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:07.789+0000 7f9349800700 1 -- start start 2024-04-12T21:25:07.787 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:07.789+0000 7f9349800700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f93440708a0 0x7f934411a160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:25:07.787 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:07.789+0000 7f9349800700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9344071f90 0x7f934411a6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:25:07.787 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:07.789+0000 7f9349800700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f934411acf0 con 0x7f93440708a0 2024-04-12T21:25:07.788 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:07.789+0000 7f9349800700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93441140c0 con 0x7f9344071f90 2024-04-12T21:25:07.788 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:07.789+0000 7f9342a00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f93440708a0 0x7f934411a160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:25:07.788 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:07.790+0000 7f9342a00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f93440708a0 0x7f934411a160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:59220/0 (socket says 172.21.15.18:59220) 2024-04-12T21:25:07.788 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:07.790+0000 7f9342a00700 1 -- 172.21.15.18:0/2404597230 learned_addr learned my addr 172.21.15.18:0/2404597230 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:25:07.788 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:07.790+0000 7f9342000700 1 --2- 172.21.15.18:0/2404597230 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9344071f90 0x7f934411a6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:25:07.788 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:07.790+0000 7f9342a00700 1 -- 172.21.15.18:0/2404597230 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9344071f90 msgr2=0x7f934411a6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:25:07.788 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:07.790+0000 7f9342a00700 1 --2- 172.21.15.18:0/2404597230 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9344071f90 0x7f934411a6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:25:07.788 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:07.790+0000 7f9342a00700 1 -- 172.21.15.18:0/2404597230 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9330005040 con 0x7f93440708a0 2024-04-12T21:25:07.788 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:07.790+0000 7f9342a00700 1 --2- 172.21.15.18:0/2404597230 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f93440708a0 0x7f934411a160 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f933400c8c0 tx=0x7f93340044c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:25:07.788 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:07.791+0000 7f933aa00700 1 -- 172.21.15.18:0/2404597230 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f933400a980 con 0x7f93440708a0 2024-04-12T21:25:07.788 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:07.791+0000 7f933aa00700 1 -- 172.21.15.18:0/2404597230 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f933400ce60 con 0x7f93440708a0 2024-04-12T21:25:07.788 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:07.791+0000 7f933aa00700 1 -- 172.21.15.18:0/2404597230 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f93340115e0 con 0x7f93440708a0 2024-04-12T21:25:07.788 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:07.791+0000 7f9349800700 1 -- 172.21.15.18:0/2404597230 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f93441143a0 con 0x7f93440708a0 2024-04-12T21:25:07.789 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:07.791+0000 7f9349800700 1 -- 172.21.15.18:0/2404597230 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f93441148e0 con 0x7f93440708a0 2024-04-12T21:25:07.789 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:07.792+0000 7f9349800700 1 -- 172.21.15.18:0/2404597230 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9344071aa0 con 0x7f93440708a0 2024-04-12T21:25:07.790 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:07.793+0000 7f933aa00700 1 -- 172.21.15.18:0/2404597230 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7f9334012400 con 0x7f93440708a0 2024-04-12T21:25:07.791 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:07.794+0000 7f933aa00700 1 --2- 172.21.15.18:0/2404597230 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f932806ec10 0x7f93280710c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:25:07.791 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:07.794+0000 7f933aa00700 1 -- 172.21.15.18:0/2404597230 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 6757+0+0 (secure 0 0 0) 0x7f933408ea40 con 0x7f93440708a0 2024-04-12T21:25:07.791 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:07.795+0000 7f9342000700 1 --2- 172.21.15.18:0/2404597230 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f932806ec10 0x7f93280710c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:25:07.792 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:07.795+0000 7f9342000700 1 --2- 172.21.15.18:0/2404597230 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f932806ec10 0x7f93280710c0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f9330000c00 tx=0x7f9330009e30 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:25:07.796 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:07.799+0000 7f933aa00700 1 -- 172.21.15.18:0/2404597230 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f933405b270 con 0x7f93440708a0 2024-04-12T21:25:08.037 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:08.040+0000 7f9349800700 1 -- 172.21.15.18:0/2404597230 --> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f934404bf40 con 0x7f932806ec10 2024-04-12T21:25:08.049 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:08.053+0000 7f933aa00700 1 -- 172.21.15.18:0/2404597230 <== mgr.34102 v2:172.21.15.18:6800/3812891919 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7f934404bf40 con 0x7f932806ec10 2024-04-12T21:25:08.050 INFO:teuthology.orchestra.run.smithi018.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-12T21:25:08.050 INFO:teuthology.orchestra.run.smithi018.stdout:alertmanager.smithi018 smithi018 *:9093,9094 running (25m) 2m ago 40m 15.5M - 0.25.0 c8568f914cd2 c332090068e8 2024-04-12T21:25:08.050 INFO:teuthology.orchestra.run.smithi018.stdout:crash.smithi018 smithi018 running (22m) 2m ago 40m 6665k - 19.0.0-2879-g324baf8d 8717720dc5e5 b7f80d1f7982 2024-04-12T21:25:08.050 INFO:teuthology.orchestra.run.smithi018.stdout:crash.smithi186 smithi186 running (21m) 2m ago 39m 6660k - 19.0.0-2879-g324baf8d 8717720dc5e5 a4f55c81b623 2024-04-12T21:25:08.050 INFO:teuthology.orchestra.run.smithi018.stdout:grafana.smithi018 smithi018 *:3000 running (25m) 2m ago 39m 74.8M - 9.4.12 2bacad6d85d8 b08e3bf176dd 2024-04-12T21:25:08.050 INFO:teuthology.orchestra.run.smithi018.stdout:mds.cephfs.smithi018.acqbsc smithi018 running (17m) 2m ago 33m 16.7M - 19.0.0-2879-g324baf8d 8717720dc5e5 021f99ab24ae 2024-04-12T21:25:08.050 INFO:teuthology.orchestra.run.smithi018.stdout:mds.cephfs.smithi018.szjluo smithi018 running (17m) 2m ago 32m 16.5M - 19.0.0-2879-g324baf8d 8717720dc5e5 0003117479d4 2024-04-12T21:25:08.051 INFO:teuthology.orchestra.run.smithi018.stdout:mds.cephfs.smithi186.jdllyn smithi186 running (2m) 2m ago 33m 11.0M - 19.0.0-2879-g324baf8d 8717720dc5e5 21b1f46eef23 2024-04-12T21:25:08.051 INFO:teuthology.orchestra.run.smithi018.stdout:mds.cephfs.smithi186.wjnafs smithi186 running (32m) 2m ago 32m 19.5M - 17.2.7-827-gd3b5060a 21de9b7369ad 74b115d6d6fd 2024-04-12T21:25:08.051 INFO:teuthology.orchestra.run.smithi018.stdout:mgr.smithi018.xjzwga smithi018 *:8443,9283,8765 running (28m) 2m ago 41m 512M - 19.0.0-2879-g324baf8d 8717720dc5e5 56088305eab7 2024-04-12T21:25:08.051 INFO:teuthology.orchestra.run.smithi018.stdout:mgr.smithi186.kqcfxk smithi186 *:8443,9283,8765 running (27m) 2m ago 38m 459M - 19.0.0-2879-g324baf8d 8717720dc5e5 09cd766e5e0d 2024-04-12T21:25:08.051 INFO:teuthology.orchestra.run.smithi018.stdout:mon.smithi018 smithi018 running (23m) 2m ago 42m 73.1M 2048M 19.0.0-2879-g324baf8d 8717720dc5e5 df3e201dfed6 2024-04-12T21:25:08.051 INFO:teuthology.orchestra.run.smithi018.stdout:mon.smithi186 smithi186 running (23m) 2m ago 38m 47.2M 2048M 19.0.0-2879-g324baf8d 8717720dc5e5 c77498da9b2b 2024-04-12T21:25:08.051 INFO:teuthology.orchestra.run.smithi018.stdout:node-exporter.smithi018 smithi018 *:9100 running (26m) 2m ago 40m 8108k - 1.5.0 0da6a335fe13 326c3d9c6847 2024-04-12T21:25:08.051 INFO:teuthology.orchestra.run.smithi018.stdout:node-exporter.smithi186 smithi186 *:9100 running (26m) 2m ago 38m 8757k - 1.5.0 0da6a335fe13 ec2112baeecf 2024-04-12T21:25:08.051 INFO:teuthology.orchestra.run.smithi018.stdout:osd.0 smithi018 running (21m) 2m ago 37m 123M 1559M 19.0.0-2879-g324baf8d 8717720dc5e5 60b1273ca047 2024-04-12T21:25:08.051 INFO:teuthology.orchestra.run.smithi018.stdout:osd.1 smithi018 running (20m) 2m ago 37m 95.1M 1559M 19.0.0-2879-g324baf8d 8717720dc5e5 170ce1e5fb8d 2024-04-12T21:25:08.051 INFO:teuthology.orchestra.run.smithi018.stdout:osd.2 smithi018 running (20m) 2m ago 36m 94.3M 1559M 19.0.0-2879-g324baf8d 8717720dc5e5 c2cbc6aa2839 2024-04-12T21:25:08.051 INFO:teuthology.orchestra.run.smithi018.stdout:osd.3 smithi186 running (19m) 2m ago 36m 101M 2581M 19.0.0-2879-g324baf8d 8717720dc5e5 0138e8262e4b 2024-04-12T21:25:08.051 INFO:teuthology.orchestra.run.smithi018.stdout:osd.4 smithi186 running (19m) 2m ago 35m 91.0M 2581M 19.0.0-2879-g324baf8d 8717720dc5e5 b20cb63b25c9 2024-04-12T21:25:08.051 INFO:teuthology.orchestra.run.smithi018.stdout:osd.5 smithi186 running (18m) 2m ago 34m 92.0M 2581M 19.0.0-2879-g324baf8d 8717720dc5e5 2513bea2610e 2024-04-12T21:25:08.051 INFO:teuthology.orchestra.run.smithi018.stdout:prometheus.smithi018 smithi018 *:9095 running (26m) 2m ago 39m 67.9M - 2.43.0 a07b618ecd1d f58841127e21 2024-04-12T21:25:08.052 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:08.055+0000 7f9349800700 1 -- 172.21.15.18:0/2404597230 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f932806ec10 msgr2=0x7f93280710c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:25:08.052 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:08.056+0000 7f9349800700 1 --2- 172.21.15.18:0/2404597230 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f932806ec10 0x7f93280710c0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f9330000c00 tx=0x7f9330009e30 comp rx=0 tx=0).stop 2024-04-12T21:25:08.052 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:08.056+0000 7f9349800700 1 -- 172.21.15.18:0/2404597230 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f93440708a0 msgr2=0x7f934411a160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:25:08.052 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:08.056+0000 7f9349800700 1 --2- 172.21.15.18:0/2404597230 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f93440708a0 0x7f934411a160 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f933400c8c0 tx=0x7f93340044c0 comp rx=0 tx=0).stop 2024-04-12T21:25:08.053 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:08.056+0000 7f9349800700 1 -- 172.21.15.18:0/2404597230 shutdown_connections 2024-04-12T21:25:08.053 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:08.056+0000 7f9349800700 1 --2- 172.21.15.18:0/2404597230 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f932806ec10 0x7f93280710c0 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:25:08.053 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:08.056+0000 7f9349800700 1 --2- 172.21.15.18:0/2404597230 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f93440708a0 0x7f934411a160 unknown :-1 s=CLOSED pgs=645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:25:08.053 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:08.056+0000 7f9349800700 1 --2- 172.21.15.18:0/2404597230 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9344071f90 0x7f934411a6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:25:08.053 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:08.056+0000 7f9349800700 1 -- 172.21.15.18:0/2404597230 >> 172.21.15.18:0/2404597230 conn(0x7f93440f8970 msgr2=0x7f93440f99f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:25:08.053 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:08.056+0000 7f9349800700 1 -- 172.21.15.18:0/2404597230 shutdown_connections 2024-04-12T21:25:08.053 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:08.056+0000 7f9349800700 1 -- 172.21.15.18:0/2404597230 wait complete. 2024-04-12T21:25:08.211 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:08.213+0000 7f74b5e00700 1 -- 172.21.15.18:0/2171331045 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f74b0072e00 msgr2=0x7f74b010d770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:25:08.211 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:08.213+0000 7f74b5e00700 1 --2- 172.21.15.18:0/2171331045 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f74b0072e00 0x7f74b010d770 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f749c004940 tx=0x7f749c009e30 comp rx=0 tx=0).stop 2024-04-12T21:25:08.211 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:08.214+0000 7f74b5e00700 1 -- 172.21.15.18:0/2171331045 shutdown_connections 2024-04-12T21:25:08.211 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:08.214+0000 7f74b5e00700 1 --2- 172.21.15.18:0/2171331045 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f74b0072e00 0x7f74b010d770 unknown :-1 s=CLOSED pgs=646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:25:08.211 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:08.214+0000 7f74b5e00700 1 --2- 172.21.15.18:0/2171331045 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f74b00724f0 0x7f74b00728c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:25:08.211 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:08.214+0000 7f74b5e00700 1 -- 172.21.15.18:0/2171331045 >> 172.21.15.18:0/2171331045 conn(0x7f74b00fabd0 msgr2=0x7f74b00fcfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:25:08.211 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:08.214+0000 7f74b5e00700 1 -- 172.21.15.18:0/2171331045 shutdown_connections 2024-04-12T21:25:08.211 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:08.214+0000 7f74b5e00700 1 -- 172.21.15.18:0/2171331045 wait complete. 2024-04-12T21:25:08.212 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:08.214+0000 7f74b5e00700 1 Processor -- start 2024-04-12T21:25:08.212 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:08.215+0000 7f74b5e00700 1 -- start start 2024-04-12T21:25:08.213 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:08.215+0000 7f74b5e00700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f74b00724f0 0x7f74b011a120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:25:08.213 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:08.215+0000 7f74b5e00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f74b0072e00 0x7f74b011a660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:25:08.213 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:08.215+0000 7f74b5e00700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74b011acb0 con 0x7f74b0072e00 2024-04-12T21:25:08.213 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:08.215+0000 7f74b5e00700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74b01140c0 con 0x7f74b00724f0 2024-04-12T21:25:08.213 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:08.215+0000 7f74ae000700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f74b0072e00 0x7f74b011a660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:25:08.213 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:08.215+0000 7f74ae000700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f74b0072e00 0x7f74b011a660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:59230/0 (socket says 172.21.15.18:59230) 2024-04-12T21:25:08.213 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:08.215+0000 7f74ae000700 1 -- 172.21.15.18:0/629749514 learned_addr learned my addr 172.21.15.18:0/629749514 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:25:08.213 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:08.216+0000 7f74aea00700 1 --2- 172.21.15.18:0/629749514 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f74b00724f0 0x7f74b011a120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:25:08.213 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:08.216+0000 7f74ae000700 1 -- 172.21.15.18:0/629749514 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f74b00724f0 msgr2=0x7f74b011a120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:25:08.213 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:08.216+0000 7f74ae000700 1 --2- 172.21.15.18:0/629749514 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f74b00724f0 0x7f74b011a120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:25:08.213 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:08.216+0000 7f74ae000700 1 -- 172.21.15.18:0/629749514 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f749c005040 con 0x7f74b0072e00 2024-04-12T21:25:08.214 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:08.216+0000 7f74ae000700 1 --2- 172.21.15.18:0/629749514 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f74b0072e00 0x7f74b011a660 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f749c004890 tx=0x7f749c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:25:08.214 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:08.216+0000 7f74a6a00700 1 -- 172.21.15.18:0/629749514 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f749c00e040 con 0x7f74b0072e00 2024-04-12T21:25:08.214 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:08.217+0000 7f74a6a00700 1 -- 172.21.15.18:0/629749514 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f749c009af0 con 0x7f74b0072e00 2024-04-12T21:25:08.215 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:08.217+0000 7f74a6a00700 1 -- 172.21.15.18:0/629749514 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f749c01c630 con 0x7f74b0072e00 2024-04-12T21:25:08.215 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:08.217+0000 7f74b5e00700 1 -- 172.21.15.18:0/629749514 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f74b0114340 con 0x7f74b0072e00 2024-04-12T21:25:08.215 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:08.217+0000 7f74b5e00700 1 -- 172.21.15.18:0/629749514 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f74b0114900 con 0x7f74b0072e00 2024-04-12T21:25:08.215 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:08.218+0000 7f74b5e00700 1 -- 172.21.15.18:0/629749514 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f74b0073c30 con 0x7f74b0072e00 2024-04-12T21:25:08.215 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:08.219+0000 7f74a6a00700 1 -- 172.21.15.18:0/629749514 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7f749c00bbc0 con 0x7f74b0072e00 2024-04-12T21:25:08.217 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:08.220+0000 7f74a6a00700 1 --2- 172.21.15.18:0/629749514 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f749406ec00 0x7f74940710b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:25:08.217 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:08.220+0000 7f74a6a00700 1 -- 172.21.15.18:0/629749514 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 6757+0+0 (secure 0 0 0) 0x7f749c08fc10 con 0x7f74b0072e00 2024-04-12T21:25:08.217 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:08.220+0000 7f74aea00700 1 --2- 172.21.15.18:0/629749514 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f749406ec00 0x7f74940710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:25:08.217 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:08.221+0000 7f74aea00700 1 --2- 172.21.15.18:0/629749514 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f749406ec00 0x7f74940710b0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f74a0004d60 tx=0x7f74a000b040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:25:08.221 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:08.225+0000 7f74a6a00700 1 -- 172.21.15.18:0/629749514 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f749c0589d0 con 0x7f74b0072e00 2024-04-12T21:25:08.542 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:08.545+0000 7f74b5e00700 1 -- 172.21.15.18:0/629749514 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f74b0110480 con 0x7f74b0072e00 2024-04-12T21:25:08.543 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:08.546+0000 7f74a6a00700 1 -- 172.21.15.18:0/629749514 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+837 (secure 0 0 0) 0x7f749c05c090 con 0x7f74b0072e00 2024-04-12T21:25:08.543 INFO:teuthology.orchestra.run.smithi018.stdout:{ 2024-04-12T21:25:08.543 INFO:teuthology.orchestra.run.smithi018.stdout: "mon": { 2024-04-12T21:25:08.543 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 19.0.0-2879-g324baf8d (324baf8ddb91dea46cac50cb535c306115791355) squid (dev)": 2 2024-04-12T21:25:08.544 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-12T21:25:08.544 INFO:teuthology.orchestra.run.smithi018.stdout: "mgr": { 2024-04-12T21:25:08.544 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 19.0.0-2879-g324baf8d (324baf8ddb91dea46cac50cb535c306115791355) squid (dev)": 2 2024-04-12T21:25:08.544 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-12T21:25:08.544 INFO:teuthology.orchestra.run.smithi018.stdout: "osd": { 2024-04-12T21:25:08.544 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 19.0.0-2879-g324baf8d (324baf8ddb91dea46cac50cb535c306115791355) squid (dev)": 6 2024-04-12T21:25:08.544 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-12T21:25:08.544 INFO:teuthology.orchestra.run.smithi018.stdout: "mds": { 2024-04-12T21:25:08.544 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 17.2.7-827-gd3b5060a (d3b5060a17449edfae9385cdf792afec109fca5d) quincy (stable)": 1, 2024-04-12T21:25:08.544 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 19.0.0-2879-g324baf8d (324baf8ddb91dea46cac50cb535c306115791355) squid (dev)": 3 2024-04-12T21:25:08.544 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-12T21:25:08.544 INFO:teuthology.orchestra.run.smithi018.stdout: "overall": { 2024-04-12T21:25:08.544 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 17.2.7-827-gd3b5060a (d3b5060a17449edfae9385cdf792afec109fca5d) quincy (stable)": 1, 2024-04-12T21:25:08.544 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 19.0.0-2879-g324baf8d (324baf8ddb91dea46cac50cb535c306115791355) squid (dev)": 13 2024-04-12T21:25:08.544 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-04-12T21:25:08.545 INFO:teuthology.orchestra.run.smithi018.stdout:} 2024-04-12T21:25:08.546 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:08.549+0000 7f74b5e00700 1 -- 172.21.15.18:0/629749514 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f749406ec00 msgr2=0x7f74940710b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:25:08.546 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:08.549+0000 7f74b5e00700 1 --2- 172.21.15.18:0/629749514 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f749406ec00 0x7f74940710b0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f74a0004d60 tx=0x7f74a000b040 comp rx=0 tx=0).stop 2024-04-12T21:25:08.546 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:08.549+0000 7f74b5e00700 1 -- 172.21.15.18:0/629749514 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f74b0072e00 msgr2=0x7f74b011a660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:25:08.546 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:08.549+0000 7f74b5e00700 1 --2- 172.21.15.18:0/629749514 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f74b0072e00 0x7f74b011a660 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f749c004890 tx=0x7f749c009e30 comp rx=0 tx=0).stop 2024-04-12T21:25:08.546 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:08.549+0000 7f74b5e00700 1 -- 172.21.15.18:0/629749514 shutdown_connections 2024-04-12T21:25:08.546 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:08.549+0000 7f74b5e00700 1 --2- 172.21.15.18:0/629749514 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f749406ec00 0x7f74940710b0 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:25:08.546 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:08.549+0000 7f74b5e00700 1 --2- 172.21.15.18:0/629749514 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f74b00724f0 0x7f74b011a120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:25:08.546 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:08.549+0000 7f74b5e00700 1 --2- 172.21.15.18:0/629749514 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f74b0072e00 0x7f74b011a660 unknown :-1 s=CLOSED pgs=647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:25:08.546 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:08.549+0000 7f74b5e00700 1 -- 172.21.15.18:0/629749514 >> 172.21.15.18:0/629749514 conn(0x7f74b00fabd0 msgr2=0x7f74b00fafa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:25:08.547 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:08.549+0000 7f74b5e00700 1 -- 172.21.15.18:0/629749514 shutdown_connections 2024-04-12T21:25:08.547 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:08.549+0000 7f74b5e00700 1 -- 172.21.15.18:0/629749514 wait complete. 2024-04-12T21:25:08.703 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:08.706+0000 7f9005000700 1 -- 172.21.15.18:0/1783133471 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9000072e00 msgr2=0x7f900010d770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:25:08.703 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:08.706+0000 7f9005000700 1 --2- 172.21.15.18:0/1783133471 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9000072e00 0x7f900010d770 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f8fec004940 tx=0x7f8fec009e30 comp rx=0 tx=0).stop 2024-04-12T21:25:08.704 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:08.706+0000 7f9005000700 1 -- 172.21.15.18:0/1783133471 shutdown_connections 2024-04-12T21:25:08.704 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:08.706+0000 7f9005000700 1 --2- 172.21.15.18:0/1783133471 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9000072e00 0x7f900010d770 unknown :-1 s=CLOSED pgs=648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:25:08.704 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:08.706+0000 7f9005000700 1 --2- 172.21.15.18:0/1783133471 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f90000724f0 0x7f90000728c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:25:08.704 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:08.706+0000 7f9005000700 1 -- 172.21.15.18:0/1783133471 >> 172.21.15.18:0/1783133471 conn(0x7f90000fabd0 msgr2=0x7f90000fcfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:25:08.704 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:08.706+0000 7f9005000700 1 -- 172.21.15.18:0/1783133471 shutdown_connections 2024-04-12T21:25:08.704 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:08.706+0000 7f9005000700 1 -- 172.21.15.18:0/1783133471 wait complete. 2024-04-12T21:25:08.705 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:08.707+0000 7f9005000700 1 Processor -- start 2024-04-12T21:25:08.705 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:08.707+0000 7f9005000700 1 -- start start 2024-04-12T21:25:08.705 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:08.708+0000 7f9005000700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f90000724f0 0x7f900011a120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:25:08.705 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:08.708+0000 7f9005000700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9000072e00 0x7f900011a660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:25:08.705 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:08.708+0000 7f9005000700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f900011acb0 con 0x7f90000724f0 2024-04-12T21:25:08.705 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:08.708+0000 7f9005000700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90001140c0 con 0x7f9000072e00 2024-04-12T21:25:08.705 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:08.708+0000 7f8ffde00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f90000724f0 0x7f900011a120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:25:08.705 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:08.708+0000 7f8ffde00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f90000724f0 0x7f900011a120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:59252/0 (socket says 172.21.15.18:59252) 2024-04-12T21:25:08.705 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:08.708+0000 7f8ffde00700 1 -- 172.21.15.18:0/3915079678 learned_addr learned my addr 172.21.15.18:0/3915079678 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:25:08.706 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:08.708+0000 7f8ffd400700 1 --2- 172.21.15.18:0/3915079678 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9000072e00 0x7f900011a660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:25:08.706 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:08.709+0000 7f8ffde00700 1 -- 172.21.15.18:0/3915079678 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9000072e00 msgr2=0x7f900011a660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:25:08.706 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:08.709+0000 7f8ffde00700 1 --2- 172.21.15.18:0/3915079678 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9000072e00 0x7f900011a660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:25:08.706 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:08.709+0000 7f8ffde00700 1 -- 172.21.15.18:0/3915079678 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8fec005040 con 0x7f90000724f0 2024-04-12T21:25:08.707 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:08.709+0000 7f8ffde00700 1 --2- 172.21.15.18:0/3915079678 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f90000724f0 0x7f900011a120 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f8ff0002a50 tx=0x7f8ff00046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:25:08.707 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:08.709+0000 7f8ff6000700 1 -- 172.21.15.18:0/3915079678 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8ff0007cc0 con 0x7f90000724f0 2024-04-12T21:25:08.707 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:08.709+0000 7f8ff6000700 1 -- 172.21.15.18:0/3915079678 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f8ff000f040 con 0x7f90000724f0 2024-04-12T21:25:08.707 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:08.709+0000 7f8ff6000700 1 -- 172.21.15.18:0/3915079678 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8ff0009640 con 0x7f90000724f0 2024-04-12T21:25:08.707 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:08.709+0000 7f9005000700 1 -- 172.21.15.18:0/3915079678 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f90001143a0 con 0x7f90000724f0 2024-04-12T21:25:08.707 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:08.709+0000 7f9005000700 1 -- 172.21.15.18:0/3915079678 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9000114960 con 0x7f90000724f0 2024-04-12T21:25:08.707 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:08.710+0000 7f9005000700 1 -- 172.21.15.18:0/3915079678 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9000073c30 con 0x7f90000724f0 2024-04-12T21:25:08.708 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:08.712+0000 7f8ff6000700 1 -- 172.21.15.18:0/3915079678 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7f8ff00097a0 con 0x7f90000724f0 2024-04-12T21:25:08.709 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:08.713+0000 7f8ff6000700 1 --2- 172.21.15.18:0/3915079678 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f8fe406ec00 0x7f8fe40710b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:25:08.709 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:08.713+0000 7f8ff6000700 1 -- 172.21.15.18:0/3915079678 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 6757+0+0 (secure 0 0 0) 0x7f8ff008e6c0 con 0x7f90000724f0 2024-04-12T21:25:08.710 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:08.713+0000 7f8ffd400700 1 --2- 172.21.15.18:0/3915079678 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f8fe406ec00 0x7f8fe40710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:25:08.710 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:08.714+0000 7f8ffd400700 1 --2- 172.21.15.18:0/3915079678 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f8fe406ec00 0x7f8fe40710b0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f8fec004f90 tx=0x7f8fec009e30 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:25:08.714 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:08.718+0000 7f8ff6000700 1 -- 172.21.15.18:0/3915079678 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f8ff00573e0 con 0x7f90000724f0 2024-04-12T21:25:08.850 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:25:08 smithi018 ceph-mon[112590]: from='client.35344 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:25:08.850 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:25:08 smithi018 ceph-mon[112590]: pgmap v761: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:25:08.851 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:25:08 smithi018 ceph-mon[112590]: from='client.35348 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:25:08.851 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:25:08 smithi018 ceph-mon[112590]: from='client.? 172.21.15.18:0/629749514' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T21:25:09.001 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:09.004+0000 7f9005000700 1 -- 172.21.15.18:0/3915079678 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f900004bed0 con 0x7f90000724f0 2024-04-12T21:25:09.002 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:09.005+0000 7f8ff6000700 1 -- 172.21.15.18:0/3915079678 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 31 v31) v1 ==== 76+0+1873 (secure 0 0 0) 0x7f8ff005aaa0 con 0x7f90000724f0 2024-04-12T21:25:09.003 INFO:teuthology.orchestra.run.smithi018.stdout:e31 2024-04-12T21:25:09.003 INFO:teuthology.orchestra.run.smithi018.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-12T21:25:09.003 INFO:teuthology.orchestra.run.smithi018.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-12T21:25:09.003 INFO:teuthology.orchestra.run.smithi018.stdout:legacy client fscid: 1 2024-04-12T21:25:09.003 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-12T21:25:09.003 INFO:teuthology.orchestra.run.smithi018.stdout:Filesystem 'cephfs' (1) 2024-04-12T21:25:09.003 INFO:teuthology.orchestra.run.smithi018.stdout:fs_name cephfs 2024-04-12T21:25:09.003 INFO:teuthology.orchestra.run.smithi018.stdout:epoch 24 2024-04-12T21:25:09.003 INFO:teuthology.orchestra.run.smithi018.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-04-12T21:25:09.003 INFO:teuthology.orchestra.run.smithi018.stdout:created 2024-04-12T20:51:57.362850+0000 2024-04-12T21:25:09.003 INFO:teuthology.orchestra.run.smithi018.stdout:modified 2024-04-12T21:06:59.717190+0000 2024-04-12T21:25:09.003 INFO:teuthology.orchestra.run.smithi018.stdout:tableserver 0 2024-04-12T21:25:09.003 INFO:teuthology.orchestra.run.smithi018.stdout:root 0 2024-04-12T21:25:09.003 INFO:teuthology.orchestra.run.smithi018.stdout:session_timeout 60 2024-04-12T21:25:09.004 INFO:teuthology.orchestra.run.smithi018.stdout:session_autoclose 300 2024-04-12T21:25:09.004 INFO:teuthology.orchestra.run.smithi018.stdout:max_file_size 1099511627776 2024-04-12T21:25:09.004 INFO:teuthology.orchestra.run.smithi018.stdout:max_xattr_size 65536 2024-04-12T21:25:09.004 INFO:teuthology.orchestra.run.smithi018.stdout:required_client_features {} 2024-04-12T21:25:09.004 INFO:teuthology.orchestra.run.smithi018.stdout:last_failure 0 2024-04-12T21:25:09.004 INFO:teuthology.orchestra.run.smithi018.stdout:last_failure_osd_epoch 100 2024-04-12T21:25:09.004 INFO:teuthology.orchestra.run.smithi018.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-12T21:25:09.004 INFO:teuthology.orchestra.run.smithi018.stdout:max_mds 2 2024-04-12T21:25:09.004 INFO:teuthology.orchestra.run.smithi018.stdout:in 0,1 2024-04-12T21:25:09.004 INFO:teuthology.orchestra.run.smithi018.stdout:up {} 2024-04-12T21:25:09.004 INFO:teuthology.orchestra.run.smithi018.stdout:failed 0,1 2024-04-12T21:25:09.004 INFO:teuthology.orchestra.run.smithi018.stdout:damaged 2024-04-12T21:25:09.004 INFO:teuthology.orchestra.run.smithi018.stdout:stopped 2024-04-12T21:25:09.004 INFO:teuthology.orchestra.run.smithi018.stdout:data_pools [3] 2024-04-12T21:25:09.004 INFO:teuthology.orchestra.run.smithi018.stdout:metadata_pool 2 2024-04-12T21:25:09.004 INFO:teuthology.orchestra.run.smithi018.stdout:inline_data disabled 2024-04-12T21:25:09.004 INFO:teuthology.orchestra.run.smithi018.stdout:balancer 2024-04-12T21:25:09.005 INFO:teuthology.orchestra.run.smithi018.stdout:bal_rank_mask -1 2024-04-12T21:25:09.005 INFO:teuthology.orchestra.run.smithi018.stdout:standby_count_wanted 1 2024-04-12T21:25:09.005 INFO:teuthology.orchestra.run.smithi018.stdout:qdb_cluster leader: 0 members: 2024-04-12T21:25:09.005 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-12T21:25:09.005 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-12T21:25:09.005 INFO:teuthology.orchestra.run.smithi018.stdout:Standby daemons: 2024-04-12T21:25:09.005 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-12T21:25:09.005 INFO:teuthology.orchestra.run.smithi018.stdout:[mds.cephfs.smithi186.wjnafs{-1:24277} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.186:6826/438347207,v1:172.21.15.186:6827/438347207] compat {c=[1],r=[1],i=[7ff]}] 2024-04-12T21:25:09.005 INFO:teuthology.orchestra.run.smithi018.stdout:[mds.cephfs.smithi018.acqbsc{-1:34436} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.18:6828/755179764,v1:172.21.15.18:6829/755179764] compat {c=[1],r=[1],i=[1fff]}] 2024-04-12T21:25:09.005 INFO:teuthology.orchestra.run.smithi018.stdout:[mds.cephfs.smithi018.szjluo{-1:34468} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.18:6826/2407992653,v1:172.21.15.18:6827/2407992653] compat {c=[1],r=[1],i=[1fff]}] 2024-04-12T21:25:09.005 INFO:teuthology.orchestra.run.smithi018.stdout:[mds.cephfs.smithi186.jdllyn{-1:44595} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.186:6824/3523444202,v1:172.21.15.186:6825/3523444202] compat {c=[1],r=[1],i=[1fff]}] 2024-04-12T21:25:09.006 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:09.008+0000 7f9005000700 1 -- 172.21.15.18:0/3915079678 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f8fe406ec00 msgr2=0x7f8fe40710b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:25:09.006 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:09.008+0000 7f9005000700 1 --2- 172.21.15.18:0/3915079678 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f8fe406ec00 0x7f8fe40710b0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f8fec004f90 tx=0x7f8fec009e30 comp rx=0 tx=0).stop 2024-04-12T21:25:09.006 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:09.008+0000 7f9005000700 1 -- 172.21.15.18:0/3915079678 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f90000724f0 msgr2=0x7f900011a120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:25:09.006 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:09.008+0000 7f9005000700 1 --2- 172.21.15.18:0/3915079678 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f90000724f0 0x7f900011a120 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f8ff0002a50 tx=0x7f8ff00046b0 comp rx=0 tx=0).stop 2024-04-12T21:25:09.006 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:09.009+0000 7f9005000700 1 -- 172.21.15.18:0/3915079678 shutdown_connections 2024-04-12T21:25:09.006 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:09.009+0000 7f9005000700 1 --2- 172.21.15.18:0/3915079678 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f8fe406ec00 0x7f8fe40710b0 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:25:09.006 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:09.009+0000 7f9005000700 1 --2- 172.21.15.18:0/3915079678 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f90000724f0 0x7f900011a120 unknown :-1 s=CLOSED pgs=649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:25:09.006 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:09.009+0000 7f9005000700 1 --2- 172.21.15.18:0/3915079678 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9000072e00 0x7f900011a660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:25:09.007 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:09.009+0000 7f9005000700 1 -- 172.21.15.18:0/3915079678 >> 172.21.15.18:0/3915079678 conn(0x7f90000fabd0 msgr2=0x7f90000fafa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:25:09.007 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:09.009+0000 7f9005000700 1 -- 172.21.15.18:0/3915079678 shutdown_connections 2024-04-12T21:25:09.007 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:09.009+0000 7f9005000700 1 -- 172.21.15.18:0/3915079678 wait complete. 2024-04-12T21:25:09.007 INFO:teuthology.orchestra.run.smithi018.stderr:dumped fsmap epoch 31 2024-04-12T21:25:09.029 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:25:08 smithi186 ceph-mon[86379]: from='client.35344 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:25:09.029 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:25:08 smithi186 ceph-mon[86379]: pgmap v761: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:25:09.029 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:25:08 smithi186 ceph-mon[86379]: from='client.35348 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:25:09.029 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:25:08 smithi186 ceph-mon[86379]: from='client.? 172.21.15.18:0/629749514' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T21:25:09.169 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:09.171+0000 7fcc7fe00700 1 -- 172.21.15.18:0/2941475226 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fcc781059b0 msgr2=0x7fcc780ff070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:25:09.169 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:09.171+0000 7fcc7fe00700 1 --2- 172.21.15.18:0/2941475226 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fcc781059b0 0x7fcc780ff070 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7fcc64004940 tx=0x7fcc64009e30 comp rx=0 tx=0).stop 2024-04-12T21:25:09.169 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:09.172+0000 7fcc7fe00700 1 -- 172.21.15.18:0/2941475226 shutdown_connections 2024-04-12T21:25:09.169 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:09.172+0000 7fcc7fe00700 1 --2- 172.21.15.18:0/2941475226 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fcc781059b0 0x7fcc780ff070 unknown :-1 s=CLOSED pgs=650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:25:09.169 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:09.172+0000 7fcc7fe00700 1 --2- 172.21.15.18:0/2941475226 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcc78105010 0x7fcc781053e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:25:09.169 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:09.172+0000 7fcc7fe00700 1 -- 172.21.15.18:0/2941475226 >> 172.21.15.18:0/2941475226 conn(0x7fcc780fabd0 msgr2=0x7fcc780fcfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:25:09.169 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:09.172+0000 7fcc7fe00700 1 -- 172.21.15.18:0/2941475226 shutdown_connections 2024-04-12T21:25:09.169 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:09.172+0000 7fcc7fe00700 1 -- 172.21.15.18:0/2941475226 wait complete. 2024-04-12T21:25:09.170 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:09.172+0000 7fcc7fe00700 1 Processor -- start 2024-04-12T21:25:09.171 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:09.173+0000 7fcc7fe00700 1 -- start start 2024-04-12T21:25:09.171 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:09.173+0000 7fcc7fe00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fcc78105010 0x7fcc78115da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:25:09.171 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:09.173+0000 7fcc7fe00700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcc781059b0 0x7fcc781162e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:25:09.171 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:09.173+0000 7fcc7fe00700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc78116930 con 0x7fcc78105010 2024-04-12T21:25:09.171 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:09.173+0000 7fcc7fe00700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc7810fd00 con 0x7fcc781059b0 2024-04-12T21:25:09.171 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:09.173+0000 7fcc7d400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fcc78105010 0x7fcc78115da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:25:09.171 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:09.173+0000 7fcc7d400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fcc78105010 0x7fcc78115da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:59258/0 (socket says 172.21.15.18:59258) 2024-04-12T21:25:09.171 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:09.173+0000 7fcc7d400700 1 -- 172.21.15.18:0/3443752490 learned_addr learned my addr 172.21.15.18:0/3443752490 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:25:09.171 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:09.174+0000 7fcc7ca00700 1 --2- 172.21.15.18:0/3443752490 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcc781059b0 0x7fcc781162e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:25:09.171 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:09.174+0000 7fcc7d400700 1 -- 172.21.15.18:0/3443752490 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcc781059b0 msgr2=0x7fcc781162e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:25:09.171 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:09.174+0000 7fcc7d400700 1 --2- 172.21.15.18:0/3443752490 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcc781059b0 0x7fcc781162e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:25:09.171 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:09.174+0000 7fcc7d400700 1 -- 172.21.15.18:0/3443752490 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc64005040 con 0x7fcc78105010 2024-04-12T21:25:09.172 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:09.174+0000 7fcc7d400700 1 --2- 172.21.15.18:0/3443752490 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fcc78105010 0x7fcc78115da0 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7fcc6c002a50 tx=0x7fcc6c0046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:25:09.172 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:09.175+0000 7fcc75600700 1 -- 172.21.15.18:0/3443752490 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcc6c007cc0 con 0x7fcc78105010 2024-04-12T21:25:09.172 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:09.175+0000 7fcc75600700 1 -- 172.21.15.18:0/3443752490 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fcc6c00f040 con 0x7fcc78105010 2024-04-12T21:25:09.172 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:09.175+0000 7fcc75600700 1 -- 172.21.15.18:0/3443752490 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcc6c009640 con 0x7fcc78105010 2024-04-12T21:25:09.172 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:09.175+0000 7fcc7fe00700 1 -- 172.21.15.18:0/3443752490 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcc7810ffe0 con 0x7fcc78105010 2024-04-12T21:25:09.173 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:09.175+0000 7fcc7fe00700 1 -- 172.21.15.18:0/3443752490 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcc781105a0 con 0x7fcc78105010 2024-04-12T21:25:09.173 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:09.176+0000 7fcc7fe00700 1 -- 172.21.15.18:0/3443752490 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcc78100760 con 0x7fcc78105010 2024-04-12T21:25:09.174 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:09.177+0000 7fcc75600700 1 -- 172.21.15.18:0/3443752490 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7fcc6c0097a0 con 0x7fcc78105010 2024-04-12T21:25:09.175 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:09.178+0000 7fcc75600700 1 --2- 172.21.15.18:0/3443752490 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fcc6006ec10 0x7fcc600710c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:25:09.175 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:09.178+0000 7fcc75600700 1 -- 172.21.15.18:0/3443752490 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 6757+0+0 (secure 0 0 0) 0x7fcc6c08e760 con 0x7fcc78105010 2024-04-12T21:25:09.175 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:09.178+0000 7fcc7ca00700 1 --2- 172.21.15.18:0/3443752490 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fcc6006ec10 0x7fcc600710c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:25:09.176 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:09.179+0000 7fcc7ca00700 1 --2- 172.21.15.18:0/3443752490 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fcc6006ec10 0x7fcc600710c0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fcc6400b020 tx=0x7fcc64009e30 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:25:09.179 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:09.183+0000 7fcc75600700 1 -- 172.21.15.18:0/3443752490 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7fcc6c05af10 con 0x7fcc78105010 2024-04-12T21:25:09.435 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:09.438+0000 7fcc7fe00700 1 -- 172.21.15.18:0/3443752490 --> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fcc7804bf40 con 0x7fcc6006ec10 2024-04-12T21:25:09.437 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:09.441+0000 7fcc75600700 1 -- 172.21.15.18:0/3443752490 <== mgr.34102 v2:172.21.15.18:6800/3812891919 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7fcc7804bf40 con 0x7fcc6006ec10 2024-04-12T21:25:09.439 INFO:teuthology.orchestra.run.smithi018.stdout:{ 2024-04-12T21:25:09.439 INFO:teuthology.orchestra.run.smithi018.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:f4452def3be4d350f4a8e5feaf7ebb1511be9298fe68478ba7a2d6fbd7d69a63", 2024-04-12T21:25:09.439 INFO:teuthology.orchestra.run.smithi018.stdout: "in_progress": true, 2024-04-12T21:25:09.439 INFO:teuthology.orchestra.run.smithi018.stdout: "which": "Upgrading all daemon types on all hosts", 2024-04-12T21:25:09.439 INFO:teuthology.orchestra.run.smithi018.stdout: "services_complete": [ 2024-04-12T21:25:09.439 INFO:teuthology.orchestra.run.smithi018.stdout: "crash", 2024-04-12T21:25:09.439 INFO:teuthology.orchestra.run.smithi018.stdout: "osd", 2024-04-12T21:25:09.439 INFO:teuthology.orchestra.run.smithi018.stdout: "mon", 2024-04-12T21:25:09.439 INFO:teuthology.orchestra.run.smithi018.stdout: "mgr" 2024-04-12T21:25:09.439 INFO:teuthology.orchestra.run.smithi018.stdout: ], 2024-04-12T21:25:09.439 INFO:teuthology.orchestra.run.smithi018.stdout: "progress": "15/21 daemons upgraded", 2024-04-12T21:25:09.439 INFO:teuthology.orchestra.run.smithi018.stdout: "message": "Currently upgrading mds daemons", 2024-04-12T21:25:09.439 INFO:teuthology.orchestra.run.smithi018.stdout: "is_paused": false 2024-04-12T21:25:09.440 INFO:teuthology.orchestra.run.smithi018.stdout:} 2024-04-12T21:25:09.441 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:09.444+0000 7fcc7fe00700 1 -- 172.21.15.18:0/3443752490 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fcc6006ec10 msgr2=0x7fcc600710c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:25:09.441 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:09.444+0000 7fcc7fe00700 1 --2- 172.21.15.18:0/3443752490 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fcc6006ec10 0x7fcc600710c0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fcc6400b020 tx=0x7fcc64009e30 comp rx=0 tx=0).stop 2024-04-12T21:25:09.441 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:09.444+0000 7fcc7fe00700 1 -- 172.21.15.18:0/3443752490 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fcc78105010 msgr2=0x7fcc78115da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:25:09.441 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:09.444+0000 7fcc7fe00700 1 --2- 172.21.15.18:0/3443752490 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fcc78105010 0x7fcc78115da0 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7fcc6c002a50 tx=0x7fcc6c0046b0 comp rx=0 tx=0).stop 2024-04-12T21:25:09.441 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:09.444+0000 7fcc7fe00700 1 -- 172.21.15.18:0/3443752490 shutdown_connections 2024-04-12T21:25:09.441 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:09.444+0000 7fcc7fe00700 1 --2- 172.21.15.18:0/3443752490 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fcc6006ec10 0x7fcc600710c0 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:25:09.441 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:09.444+0000 7fcc7fe00700 1 --2- 172.21.15.18:0/3443752490 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fcc78105010 0x7fcc78115da0 unknown :-1 s=CLOSED pgs=651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:25:09.441 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:09.444+0000 7fcc7fe00700 1 --2- 172.21.15.18:0/3443752490 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcc781059b0 0x7fcc781162e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:25:09.441 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:09.444+0000 7fcc7fe00700 1 -- 172.21.15.18:0/3443752490 >> 172.21.15.18:0/3443752490 conn(0x7fcc780fabd0 msgr2=0x7fcc780fbbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:25:09.441 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:09.444+0000 7fcc7fe00700 1 -- 172.21.15.18:0/3443752490 shutdown_connections 2024-04-12T21:25:09.441 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:09.444+0000 7fcc7fe00700 1 -- 172.21.15.18:0/3443752490 wait complete. 2024-04-12T21:25:09.599 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:09.602+0000 7f5bcfe00700 1 -- 172.21.15.18:0/2362766528 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5bd0070360 msgr2=0x7f5bd0070730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:25:09.599 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:09.602+0000 7f5bcfe00700 1 --2- 172.21.15.18:0/2362766528 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5bd0070360 0x7f5bd0070730 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f5bc0009d10 tx=0x7f5bc0004d10 comp rx=0 tx=0).stop 2024-04-12T21:25:09.599 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:09.602+0000 7f5bcfe00700 1 -- 172.21.15.18:0/2362766528 shutdown_connections 2024-04-12T21:25:09.599 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:09.602+0000 7f5bcfe00700 1 --2- 172.21.15.18:0/2362766528 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5bd0070c70 0x7f5bd0108f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:25:09.600 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:09.602+0000 7f5bcfe00700 1 --2- 172.21.15.18:0/2362766528 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5bd0070360 0x7f5bd0070730 unknown :-1 s=CLOSED pgs=652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:25:09.600 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:09.602+0000 7f5bcfe00700 1 -- 172.21.15.18:0/2362766528 >> 172.21.15.18:0/2362766528 conn(0x7f5bd00f8830 msgr2=0x7f5bd00fac40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:25:09.600 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:09.602+0000 7f5bcfe00700 1 -- 172.21.15.18:0/2362766528 shutdown_connections 2024-04-12T21:25:09.600 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:09.602+0000 7f5bcfe00700 1 -- 172.21.15.18:0/2362766528 wait complete. 2024-04-12T21:25:09.601 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:09.603+0000 7f5bcfe00700 1 Processor -- start 2024-04-12T21:25:09.602 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:09.603+0000 7f5bcfe00700 1 -- start start 2024-04-12T21:25:09.602 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:09.604+0000 7f5bcfe00700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5bd0070360 0x7f5bd0115e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:25:09.602 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:09.604+0000 7f5bcfe00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5bd0070c70 0x7f5bd0116340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:25:09.602 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:09.604+0000 7f5bcfe00700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5bd0116a20 con 0x7f5bd0070c70 2024-04-12T21:25:09.602 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:09.604+0000 7f5bcfe00700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5bd010fda0 con 0x7f5bd0070360 2024-04-12T21:25:09.602 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:09.604+0000 7f5bce000700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5bd0070c70 0x7f5bd0116340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:25:09.602 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:09.604+0000 7f5bce000700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5bd0070c70 0x7f5bd0116340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:59276/0 (socket says 172.21.15.18:59276) 2024-04-12T21:25:09.602 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:09.604+0000 7f5bce000700 1 -- 172.21.15.18:0/2946009864 learned_addr learned my addr 172.21.15.18:0/2946009864 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:25:09.602 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:09.604+0000 7f5bcea00700 1 --2- 172.21.15.18:0/2946009864 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5bd0070360 0x7f5bd0115e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:25:09.602 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:09.604+0000 7f5bce000700 1 -- 172.21.15.18:0/2946009864 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5bd0070360 msgr2=0x7f5bd0115e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:25:09.602 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:09.604+0000 7f5bce000700 1 --2- 172.21.15.18:0/2946009864 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5bd0070360 0x7f5bd0115e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:25:09.602 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:09.604+0000 7f5bce000700 1 -- 172.21.15.18:0/2946009864 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5bc0009940 con 0x7f5bd0070c70 2024-04-12T21:25:09.602 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:09.605+0000 7f5bce000700 1 --2- 172.21.15.18:0/2946009864 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5bd0070c70 0x7f5bd0116340 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f5bbc00b790 tx=0x7f5bbc009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:25:09.602 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:09.605+0000 7f5bc6a00700 1 -- 172.21.15.18:0/2946009864 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5bbc00f770 con 0x7f5bd0070c70 2024-04-12T21:25:09.603 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:09.605+0000 7f5bc6a00700 1 -- 172.21.15.18:0/2946009864 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f5bbc00fda0 con 0x7f5bd0070c70 2024-04-12T21:25:09.603 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:09.605+0000 7f5bcfe00700 1 -- 172.21.15.18:0/2946009864 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5bd0110080 con 0x7f5bd0070c70 2024-04-12T21:25:09.603 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:09.605+0000 7f5bc6a00700 1 -- 172.21.15.18:0/2946009864 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5bbc00d310 con 0x7f5bd0070c70 2024-04-12T21:25:09.603 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:09.605+0000 7f5bcfe00700 1 -- 172.21.15.18:0/2946009864 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5bd01105c0 con 0x7f5bd0070c70 2024-04-12T21:25:09.603 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:09.606+0000 7f5bcfe00700 1 -- 172.21.15.18:0/2946009864 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5bd0071aa0 con 0x7f5bd0070c70 2024-04-12T21:25:09.603 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:09.607+0000 7f5bc6a00700 1 -- 172.21.15.18:0/2946009864 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7f5bbc010420 con 0x7f5bd0070c70 2024-04-12T21:25:09.604 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:09.608+0000 7f5bc6a00700 1 --2- 172.21.15.18:0/2946009864 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f5bb406ec10 0x7f5bb40710c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:25:09.604 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:09.608+0000 7f5bc6a00700 1 -- 172.21.15.18:0/2946009864 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 6757+0+0 (secure 0 0 0) 0x7f5bbc08ea00 con 0x7f5bd0070c70 2024-04-12T21:25:09.605 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:09.608+0000 7f5bcea00700 1 --2- 172.21.15.18:0/2946009864 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f5bb406ec10 0x7f5bb40710c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:25:09.605 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:09.609+0000 7f5bcea00700 1 --2- 172.21.15.18:0/2946009864 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f5bb406ec10 0x7f5bb40710c0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f5bc0005010 tx=0x7f5bc0004d10 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:25:09.609 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:09.613+0000 7f5bc6a00700 1 -- 172.21.15.18:0/2946009864 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f5bbc057700 con 0x7f5bd0070c70 2024-04-12T21:25:09.850 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:25:09 smithi018 ceph-mon[112590]: from='client.35352 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:25:09.850 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:25:09 smithi018 ceph-mon[112590]: from='client.? 172.21.15.18:0/3915079678' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-12T21:25:09.919 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:09.922+0000 7f5bcfe00700 1 -- 172.21.15.18:0/2946009864 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f5bd005f5f0 con 0x7f5bd0070c70 2024-04-12T21:25:09.919 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:09.923+0000 7f5bc6a00700 1 -- 172.21.15.18:0/2946009864 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+4303 (secure 0 0 0) 0x7f5bbc05adc0 con 0x7f5bd0070c70 2024-04-12T21:25:09.920 INFO:teuthology.orchestra.run.smithi018.stdout:HEALTH_ERR failed to probe daemons or devices; 1 filesystem is degraded; 1 filesystem is offline 2024-04-12T21:25:09.920 INFO:teuthology.orchestra.run.smithi018.stdout:[WRN] CEPHADM_REFRESH_FAILED: failed to probe daemons or devices 2024-04-12T21:25:09.920 INFO:teuthology.orchestra.run.smithi018.stdout: host smithi018 `cephadm ceph-volume` failed: cephadm exited with an error code: 1, stderr: Inferring config /var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/mon.smithi018/config 2024-04-12T21:25:09.920 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/podman: timeout after 895 seconds 2024-04-12T21:25:09.920 INFO:teuthology.orchestra.run.smithi018.stdout:Non-zero exit code 124 from /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:f4452def3be4d350f4a8e5feaf7ebb1511be9298fe68478ba7a2d6fbd7d69a63 -e NODE_NAME=smithi018 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968:/var/run/ceph:z -v /var/log/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968:/var/log/ceph:z -v /var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmpxsmwkkuh:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:f4452def3be4d350f4a8e5feaf7ebb1511be9298fe68478ba7a2d6fbd7d69a63 inventory --format=json-pretty --filter-for-batch 2024-04-12T21:25:09.920 INFO:teuthology.orchestra.run.smithi018.stdout:Traceback (most recent call last): 2024-04-12T21:25:09.920 INFO:teuthology.orchestra.run.smithi018.stdout: File "/usr/lib64/python3.9/runpy.py", line 197, in _run_module_as_main 2024-04-12T21:25:09.920 INFO:teuthology.orchestra.run.smithi018.stdout: return _run_code(code, main_globals, None, 2024-04-12T21:25:09.921 INFO:teuthology.orchestra.run.smithi018.stdout: File "/usr/lib64/python3.9/runpy.py", line 87, in _run_code 2024-04-12T21:25:09.921 INFO:teuthology.orchestra.run.smithi018.stdout: exec(code, run_globals) 2024-04-12T21:25:09.921 INFO:teuthology.orchestra.run.smithi018.stdout: File "/var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/cephadm.e44f3ed5e34e4ef52eedef233d1190bff004470d8e8005127b5312bff357fa7f/__main__.py", line 5618, in 2024-04-12T21:25:09.921 INFO:teuthology.orchestra.run.smithi018.stdout: File "/var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/cephadm.e44f3ed5e34e4ef52eedef233d1190bff004470d8e8005127b5312bff357fa7f/__main__.py", line 5606, in main 2024-04-12T21:25:09.921 INFO:teuthology.orchestra.run.smithi018.stdout: File "/var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/cephadm.e44f3ed5e34e4ef52eedef233d1190bff004470d8e8005127b5312bff357fa7f/__main__.py", line 408, in _infer_config 2024-04-12T21:25:09.921 INFO:teuthology.orchestra.run.smithi018.stdout: File "/var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/cephadm.e44f3ed5e34e4ef52eedef233d1190bff004470d8e8005127b5312bff357fa7f/__main__.py", line 323, in _infer_fsid 2024-04-12T21:25:09.921 INFO:teuthology.orchestra.run.smithi018.stdout: File "/var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/cephadm.e44f3ed5e34e4ef52eedef233d1190bff004470d8e8005127b5312bff357fa7f/__main__.py", line 436, in _infer_image 2024-04-12T21:25:09.921 INFO:teuthology.orchestra.run.smithi018.stdout: File "/var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/cephadm.e44f3ed5e34e4ef52eedef233d1190bff004470d8e8005127b5312bff357fa7f/__main__.py", line 310, in _validate_fsid 2024-04-12T21:25:09.921 INFO:teuthology.orchestra.run.smithi018.stdout: File "/var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/cephadm.e44f3ed5e34e4ef52eedef233d1190bff004470d8e8005127b5312bff357fa7f/__main__.py", line 3359, in command_ceph_volume 2024-04-12T21:25:09.921 INFO:teuthology.orchestra.run.smithi018.stdout: File "/var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/cephadm.e44f3ed5e34e4ef52eedef233d1190bff004470d8e8005127b5312bff357fa7f/cephadmlib/call_wrappers.py", line 310, in call_throws 2024-04-12T21:25:09.921 INFO:teuthology.orchestra.run.smithi018.stdout:RuntimeError: Failed command: /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:f4452def3be4d350f4a8e5feaf7ebb1511be9298fe68478ba7a2d6fbd7d69a63 -e NODE_NAME=smithi018 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968:/var/run/ceph:z -v /var/log/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968:/var/log/ceph:z -v /var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmpxsmwkkuh:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:f4452def3be4d350f4a8e5feaf7ebb1511be9298fe68478ba7a2d6fbd7d69a63 inventory --format=json-pretty --filter-for-batch 2024-04-12T21:25:09.921 INFO:teuthology.orchestra.run.smithi018.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-04-12T21:25:09.921 INFO:teuthology.orchestra.run.smithi018.stdout: fs cephfs is degraded 2024-04-12T21:25:09.921 INFO:teuthology.orchestra.run.smithi018.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-04-12T21:25:09.922 INFO:teuthology.orchestra.run.smithi018.stdout: fs cephfs is offline because no MDS is active for it. 2024-04-12T21:25:09.923 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:09.926+0000 7f5bcfe00700 1 -- 172.21.15.18:0/2946009864 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f5bb406ec10 msgr2=0x7f5bb40710c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:25:09.924 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:09.926+0000 7f5bcfe00700 1 --2- 172.21.15.18:0/2946009864 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f5bb406ec10 0x7f5bb40710c0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f5bc0005010 tx=0x7f5bc0004d10 comp rx=0 tx=0).stop 2024-04-12T21:25:09.924 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:09.926+0000 7f5bcfe00700 1 -- 172.21.15.18:0/2946009864 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5bd0070c70 msgr2=0x7f5bd0116340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:25:09.924 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:09.926+0000 7f5bcfe00700 1 --2- 172.21.15.18:0/2946009864 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5bd0070c70 0x7f5bd0116340 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f5bbc00b790 tx=0x7f5bbc009250 comp rx=0 tx=0).stop 2024-04-12T21:25:09.924 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:09.927+0000 7f5bcfe00700 1 -- 172.21.15.18:0/2946009864 shutdown_connections 2024-04-12T21:25:09.924 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:09.927+0000 7f5bcfe00700 1 --2- 172.21.15.18:0/2946009864 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f5bb406ec10 0x7f5bb40710c0 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:25:09.924 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:09.927+0000 7f5bcfe00700 1 --2- 172.21.15.18:0/2946009864 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5bd0070360 0x7f5bd0115e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:25:09.924 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:09.927+0000 7f5bcfe00700 1 --2- 172.21.15.18:0/2946009864 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5bd0070c70 0x7f5bd0116340 unknown :-1 s=CLOSED pgs=653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:25:09.924 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:09.927+0000 7f5bcfe00700 1 -- 172.21.15.18:0/2946009864 >> 172.21.15.18:0/2946009864 conn(0x7f5bd00f8830 msgr2=0x7f5bd00f9d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:25:09.924 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:09.927+0000 7f5bcfe00700 1 -- 172.21.15.18:0/2946009864 shutdown_connections 2024-04-12T21:25:09.924 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:09.927+0000 7f5bcfe00700 1 -- 172.21.15.18:0/2946009864 wait complete. 2024-04-12T21:25:10.029 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:25:09 smithi186 ceph-mon[86379]: from='client.35352 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:25:10.029 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:25:09 smithi186 ceph-mon[86379]: from='client.? 172.21.15.18:0/3915079678' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-12T21:25:10.931 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:25:10 smithi186 ceph-mon[86379]: from='client.35364 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:25:10.931 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:25:10 smithi186 ceph-mon[86379]: pgmap v762: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:25:10.931 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:25:10 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T21:25:10.931 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:25:10 smithi186 ceph-mon[86379]: from='client.? 172.21.15.18:0/2946009864' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-12T21:25:11.100 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:25:10 smithi018 ceph-mon[112590]: from='client.35364 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:25:11.101 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:25:10 smithi018 ceph-mon[112590]: pgmap v762: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:25:11.101 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:25:10 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T21:25:11.101 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:25:10 smithi018 ceph-mon[112590]: from='client.? 172.21.15.18:0/2946009864' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-12T21:25:12.905 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:25:12 smithi018 ceph-mon[112590]: pgmap v763: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:25:13.029 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:25:12 smithi186 ceph-mon[86379]: pgmap v763: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:25:15.029 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:25:14 smithi186 ceph-mon[86379]: pgmap v764: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:25:15.100 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:25:14 smithi018 ceph-mon[112590]: pgmap v764: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:25:17.029 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:25:16 smithi186 ceph-mon[86379]: pgmap v765: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:25:17.100 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:25:16 smithi018 ceph-mon[112590]: pgmap v765: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:25:19.029 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:25:18 smithi186 ceph-mon[86379]: pgmap v766: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:25:19.100 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:25:18 smithi018 ceph-mon[112590]: pgmap v766: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:25:21.029 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:25:20 smithi186 ceph-mon[86379]: pgmap v767: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:25:21.100 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:25:20 smithi018 ceph-mon[112590]: pgmap v767: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:25:23.029 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:25:22 smithi186 ceph-mon[86379]: pgmap v768: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:25:23.100 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:25:22 smithi018 ceph-mon[112590]: pgmap v768: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:25:25.029 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:25:24 smithi186 ceph-mon[86379]: pgmap v769: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:25:25.100 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:25:24 smithi018 ceph-mon[112590]: pgmap v769: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:25:26.029 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:25:25 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T21:25:26.100 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:25:25 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T21:25:27.029 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:25:26 smithi186 ceph-mon[86379]: pgmap v770: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:25:27.100 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:25:26 smithi018 ceph-mon[112590]: pgmap v770: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:25:29.029 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:25:28 smithi186 ceph-mon[86379]: pgmap v771: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:25:29.100 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:25:28 smithi018 ceph-mon[112590]: pgmap v771: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:25:31.029 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:25:30 smithi186 ceph-mon[86379]: pgmap v772: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:25:31.100 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:25:30 smithi018 ceph-mon[112590]: pgmap v772: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:25:33.029 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:25:32 smithi186 ceph-mon[86379]: pgmap v773: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:25:33.100 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:25:32 smithi018 ceph-mon[112590]: pgmap v773: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:25:35.029 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:25:34 smithi186 ceph-mon[86379]: pgmap v774: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:25:35.100 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:25:34 smithi018 ceph-mon[112590]: pgmap v774: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:25:37.029 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:25:36 smithi186 ceph-mon[86379]: pgmap v775: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:25:37.100 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:25:36 smithi018 ceph-mon[112590]: pgmap v775: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:25:39.029 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:25:38 smithi186 ceph-mon[86379]: pgmap v776: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:25:39.100 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:25:38 smithi018 ceph-mon[112590]: pgmap v776: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:25:40.097 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:40.100+0000 7f6311000700 1 -- 172.21.15.18:0/3016820875 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f630c109430 msgr2=0x7f630c109800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:25:40.097 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:40.101+0000 7f6311000700 1 --2- 172.21.15.18:0/3016820875 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f630c109430 0x7f630c109800 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f62fc009ce0 tx=0x7f62fc004d10 comp rx=0 tx=0).stop 2024-04-12T21:25:40.097 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:40.101+0000 7f6311000700 1 -- 172.21.15.18:0/3016820875 shutdown_connections 2024-04-12T21:25:40.097 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:40.101+0000 7f6311000700 1 --2- 172.21.15.18:0/3016820875 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f630c109dd0 0x7f630c103440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:25:40.097 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:40.101+0000 7f6311000700 1 --2- 172.21.15.18:0/3016820875 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f630c109430 0x7f630c109800 unknown :-1 s=CLOSED pgs=654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:25:40.097 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:40.101+0000 7f6311000700 1 -- 172.21.15.18:0/3016820875 >> 172.21.15.18:0/3016820875 conn(0x7f630c072c00 msgr2=0x7f630c075010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:25:40.098 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:40.101+0000 7f6311000700 1 -- 172.21.15.18:0/3016820875 shutdown_connections 2024-04-12T21:25:40.098 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:40.101+0000 7f6311000700 1 -- 172.21.15.18:0/3016820875 wait complete. 2024-04-12T21:25:40.099 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:40.102+0000 7f6311000700 1 Processor -- start 2024-04-12T21:25:40.099 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:40.102+0000 7f6311000700 1 -- start start 2024-04-12T21:25:40.099 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:40.102+0000 7f6311000700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f630c109430 0x7f630c11a130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:25:40.099 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:40.102+0000 7f6311000700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f630c109dd0 0x7f630c11a670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:25:40.099 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:40.102+0000 7f6311000700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f630c11ad50 con 0x7f630c109dd0 2024-04-12T21:25:40.099 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:40.102+0000 7f6311000700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f630c1142a0 con 0x7f630c109430 2024-04-12T21:25:40.099 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:40.103+0000 7f6301400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f630c109dd0 0x7f630c11a670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:25:40.099 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:40.103+0000 7f6301400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f630c109dd0 0x7f630c11a670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:44384/0 (socket says 172.21.15.18:44384) 2024-04-12T21:25:40.099 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:40.103+0000 7f6301400700 1 -- 172.21.15.18:0/3657215849 learned_addr learned my addr 172.21.15.18:0/3657215849 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:25:40.100 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:40.103+0000 7f6309e00700 1 --2- 172.21.15.18:0/3657215849 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f630c109430 0x7f630c11a130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:25:40.100 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:40.103+0000 7f6301400700 1 -- 172.21.15.18:0/3657215849 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f630c109430 msgr2=0x7f630c11a130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:25:40.100 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:40.103+0000 7f6301400700 1 --2- 172.21.15.18:0/3657215849 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f630c109430 0x7f630c11a130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:25:40.100 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:40.103+0000 7f6301400700 1 -- 172.21.15.18:0/3657215849 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62fc009940 con 0x7f630c109dd0 2024-04-12T21:25:40.100 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:40.104+0000 7f6301400700 1 --2- 172.21.15.18:0/3657215849 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f630c109dd0 0x7f630c11a670 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f62f400b790 tx=0x7f62f4009220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:25:40.101 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:40.104+0000 7f6302a00700 1 -- 172.21.15.18:0/3657215849 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f62f400f6f0 con 0x7f630c109dd0 2024-04-12T21:25:40.101 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:40.104+0000 7f6302a00700 1 -- 172.21.15.18:0/3657215849 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f62f400fd20 con 0x7f630c109dd0 2024-04-12T21:25:40.101 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:40.104+0000 7f6311000700 1 -- 172.21.15.18:0/3657215849 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f630c114520 con 0x7f630c109dd0 2024-04-12T21:25:40.101 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:40.104+0000 7f6302a00700 1 -- 172.21.15.18:0/3657215849 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f62f40103a0 con 0x7f630c109dd0 2024-04-12T21:25:40.101 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:40.104+0000 7f6311000700 1 -- 172.21.15.18:0/3657215849 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f630c114ae0 con 0x7f630c109dd0 2024-04-12T21:25:40.101 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:40.105+0000 7f6311000700 1 -- 172.21.15.18:0/3657215849 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f630c104b80 con 0x7f630c109dd0 2024-04-12T21:25:40.102 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:40.106+0000 7f6302a00700 1 -- 172.21.15.18:0/3657215849 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7f62f4011400 con 0x7f630c109dd0 2024-04-12T21:25:40.104 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:40.108+0000 7f6302a00700 1 --2- 172.21.15.18:0/3657215849 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f62f006ec00 0x7f62f00710b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:25:40.104 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:40.108+0000 7f6302a00700 1 -- 172.21.15.18:0/3657215849 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 6757+0+0 (secure 0 0 0) 0x7f62f408e780 con 0x7f630c109dd0 2024-04-12T21:25:40.104 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:40.108+0000 7f6309e00700 1 --2- 172.21.15.18:0/3657215849 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f62f006ec00 0x7f62f00710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:25:40.105 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:40.109+0000 7f6309e00700 1 --2- 172.21.15.18:0/3657215849 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f62f006ec00 0x7f62f00710b0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f62fc00b740 tx=0x7f62fc004d10 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:25:40.109 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:40.113+0000 7f6302a00700 1 -- 172.21.15.18:0/3657215849 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f62f405afd0 con 0x7f630c109dd0 2024-04-12T21:25:40.363 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:40.367+0000 7f6311000700 1 -- 172.21.15.18:0/3657215849 --> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f630c110610 con 0x7f62f006ec00 2024-04-12T21:25:40.365 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:40.369+0000 7f6302a00700 1 -- 172.21.15.18:0/3657215849 <== mgr.34102 v2:172.21.15.18:6800/3812891919 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f630c110610 con 0x7f62f006ec00 2024-04-12T21:25:40.368 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:40.372+0000 7f6311000700 1 -- 172.21.15.18:0/3657215849 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f62f006ec00 msgr2=0x7f62f00710b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:25:40.369 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:40.372+0000 7f6311000700 1 --2- 172.21.15.18:0/3657215849 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f62f006ec00 0x7f62f00710b0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f62fc00b740 tx=0x7f62fc004d10 comp rx=0 tx=0).stop 2024-04-12T21:25:40.369 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:40.372+0000 7f6311000700 1 -- 172.21.15.18:0/3657215849 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f630c109dd0 msgr2=0x7f630c11a670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:25:40.369 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:40.372+0000 7f6311000700 1 --2- 172.21.15.18:0/3657215849 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f630c109dd0 0x7f630c11a670 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f62f400b790 tx=0x7f62f4009220 comp rx=0 tx=0).stop 2024-04-12T21:25:40.369 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:40.373+0000 7f6311000700 1 -- 172.21.15.18:0/3657215849 shutdown_connections 2024-04-12T21:25:40.369 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:40.373+0000 7f6311000700 1 --2- 172.21.15.18:0/3657215849 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f62f006ec00 0x7f62f00710b0 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:25:40.369 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:40.373+0000 7f6311000700 1 --2- 172.21.15.18:0/3657215849 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f630c109430 0x7f630c11a130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:25:40.369 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:40.373+0000 7f6311000700 1 --2- 172.21.15.18:0/3657215849 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f630c109dd0 0x7f630c11a670 unknown :-1 s=CLOSED pgs=655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:25:40.369 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:40.373+0000 7f6311000700 1 -- 172.21.15.18:0/3657215849 >> 172.21.15.18:0/3657215849 conn(0x7f630c072c00 msgr2=0x7f630c074c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:25:40.369 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:40.373+0000 7f6311000700 1 -- 172.21.15.18:0/3657215849 shutdown_connections 2024-04-12T21:25:40.370 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:40.373+0000 7f6311000700 1 -- 172.21.15.18:0/3657215849 wait complete. 2024-04-12T21:25:40.389 INFO:teuthology.orchestra.run.smithi018.stdout:true 2024-04-12T21:25:40.534 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:40.537+0000 7fd3fce00700 1 -- 172.21.15.18:0/1702837829 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd3f81014f0 msgr2=0x7fd3f81018c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:25:40.534 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:40.537+0000 7fd3fce00700 1 --2- 172.21.15.18:0/1702837829 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd3f81014f0 0x7fd3f81018c0 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7fd3e8004940 tx=0x7fd3e8009e30 comp rx=0 tx=0).stop 2024-04-12T21:25:40.534 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:40.537+0000 7fd3fce00700 1 -- 172.21.15.18:0/1702837829 shutdown_connections 2024-04-12T21:25:40.534 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:40.537+0000 7fd3fce00700 1 --2- 172.21.15.18:0/1702837829 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd3f80fb4a0 0x7fd3f80fb910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:25:40.534 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:40.537+0000 7fd3fce00700 1 --2- 172.21.15.18:0/1702837829 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd3f81014f0 0x7fd3f81018c0 unknown :-1 s=CLOSED pgs=656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:25:40.534 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:40.537+0000 7fd3fce00700 1 -- 172.21.15.18:0/1702837829 >> 172.21.15.18:0/1702837829 conn(0x7fd3f80714c0 msgr2=0x7fd3f80718c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:25:40.534 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:40.538+0000 7fd3fce00700 1 -- 172.21.15.18:0/1702837829 shutdown_connections 2024-04-12T21:25:40.534 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:40.538+0000 7fd3fce00700 1 -- 172.21.15.18:0/1702837829 wait complete. 2024-04-12T21:25:40.535 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:40.538+0000 7fd3fce00700 1 Processor -- start 2024-04-12T21:25:40.535 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:40.539+0000 7fd3fce00700 1 -- start start 2024-04-12T21:25:40.535 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:40.539+0000 7fd3fce00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd3f80fb4a0 0x7fd3f8115d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:25:40.536 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:40.539+0000 7fd3fce00700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd3f81014f0 0x7fd3f81162a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:25:40.536 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:40.539+0000 7fd3fce00700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3f81168f0 con 0x7fd3f80fb4a0 2024-04-12T21:25:40.536 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:40.539+0000 7fd3fce00700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3f810fcc0 con 0x7fd3f81014f0 2024-04-12T21:25:40.536 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:40.539+0000 7fd3f5e00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd3f80fb4a0 0x7fd3f8115d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:25:40.536 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:40.539+0000 7fd3f5e00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd3f80fb4a0 0x7fd3f8115d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:44400/0 (socket says 172.21.15.18:44400) 2024-04-12T21:25:40.536 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:40.539+0000 7fd3f5e00700 1 -- 172.21.15.18:0/248629977 learned_addr learned my addr 172.21.15.18:0/248629977 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:25:40.537 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:40.540+0000 7fd3f5e00700 1 -- 172.21.15.18:0/248629977 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd3f81014f0 msgr2=0x7fd3f81162a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:25:40.537 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:40.540+0000 7fd3f5400700 1 --2- 172.21.15.18:0/248629977 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd3f81014f0 0x7fd3f81162a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:25:40.537 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:40.540+0000 7fd3f5e00700 1 --2- 172.21.15.18:0/248629977 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd3f81014f0 0x7fd3f81162a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:25:40.537 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:40.540+0000 7fd3f5e00700 1 -- 172.21.15.18:0/248629977 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3e8005040 con 0x7fd3f80fb4a0 2024-04-12T21:25:40.537 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:40.540+0000 7fd3f5e00700 1 --2- 172.21.15.18:0/248629977 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd3f80fb4a0 0x7fd3f8115d60 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7fd3e8009150 tx=0x7fd3e8009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:25:40.537 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:40.540+0000 7fd3ee000700 1 -- 172.21.15.18:0/248629977 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd3e800e040 con 0x7fd3f80fb4a0 2024-04-12T21:25:40.537 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:40.541+0000 7fd3ee000700 1 -- 172.21.15.18:0/248629977 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd3e8013420 con 0x7fd3f80fb4a0 2024-04-12T21:25:40.537 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:40.541+0000 7fd3fce00700 1 -- 172.21.15.18:0/248629977 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd3f810ff40 con 0x7fd3f80fb4a0 2024-04-12T21:25:40.537 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:40.541+0000 7fd3ee000700 1 -- 172.21.15.18:0/248629977 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd3e801c630 con 0x7fd3f80fb4a0 2024-04-12T21:25:40.537 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:40.541+0000 7fd3fce00700 1 -- 172.21.15.18:0/248629977 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd3f8110500 con 0x7fd3f80fb4a0 2024-04-12T21:25:40.538 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:40.542+0000 7fd3fce00700 1 -- 172.21.15.18:0/248629977 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd3f80fcbd0 con 0x7fd3f80fb4a0 2024-04-12T21:25:40.539 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:40.543+0000 7fd3ee000700 1 -- 172.21.15.18:0/248629977 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7fd3e8013590 con 0x7fd3f80fb4a0 2024-04-12T21:25:40.540 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:40.544+0000 7fd3ee000700 1 --2- 172.21.15.18:0/248629977 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fd3dc06ec10 0x7fd3dc0710c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:25:40.540 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:40.544+0000 7fd3ee000700 1 -- 172.21.15.18:0/248629977 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 6757+0+0 (secure 0 0 0) 0x7fd3e808fcc0 con 0x7fd3f80fb4a0 2024-04-12T21:25:40.540 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:40.544+0000 7fd3f5400700 1 --2- 172.21.15.18:0/248629977 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fd3dc06ec10 0x7fd3dc0710c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:25:40.541 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:40.545+0000 7fd3f5400700 1 --2- 172.21.15.18:0/248629977 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fd3dc06ec10 0x7fd3dc0710c0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fd3f8111600 tx=0x7fd3e400b040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:25:40.545 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:40.549+0000 7fd3ee000700 1 -- 172.21.15.18:0/248629977 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7fd3e8095020 con 0x7fd3f80fb4a0 2024-04-12T21:25:40.798 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:40.802+0000 7fd3fce00700 1 -- 172.21.15.18:0/248629977 --> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fd3f804bf70 con 0x7fd3dc06ec10 2024-04-12T21:25:40.799 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:40.803+0000 7fd3ee000700 1 -- 172.21.15.18:0/248629977 <== mgr.34102 v2:172.21.15.18:6800/3812891919 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7fd3e400f040 con 0x7fd3dc06ec10 2024-04-12T21:25:40.803 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:40.807+0000 7fd3fce00700 1 -- 172.21.15.18:0/248629977 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fd3dc06ec10 msgr2=0x7fd3dc0710c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:25:40.803 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:40.807+0000 7fd3fce00700 1 --2- 172.21.15.18:0/248629977 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fd3dc06ec10 0x7fd3dc0710c0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fd3f8111600 tx=0x7fd3e400b040 comp rx=0 tx=0).stop 2024-04-12T21:25:40.804 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:40.807+0000 7fd3fce00700 1 -- 172.21.15.18:0/248629977 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd3f80fb4a0 msgr2=0x7fd3f8115d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:25:40.804 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:40.807+0000 7fd3fce00700 1 --2- 172.21.15.18:0/248629977 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd3f80fb4a0 0x7fd3f8115d60 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7fd3e8009150 tx=0x7fd3e8009e30 comp rx=0 tx=0).stop 2024-04-12T21:25:40.804 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:40.807+0000 7fd3fce00700 1 -- 172.21.15.18:0/248629977 shutdown_connections 2024-04-12T21:25:40.804 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:40.807+0000 7fd3fce00700 1 --2- 172.21.15.18:0/248629977 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fd3dc06ec10 0x7fd3dc0710c0 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:25:40.804 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:40.807+0000 7fd3fce00700 1 --2- 172.21.15.18:0/248629977 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd3f80fb4a0 0x7fd3f8115d60 unknown :-1 s=CLOSED pgs=657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:25:40.804 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:40.807+0000 7fd3fce00700 1 --2- 172.21.15.18:0/248629977 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd3f81014f0 0x7fd3f81162a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:25:40.804 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:40.807+0000 7fd3fce00700 1 -- 172.21.15.18:0/248629977 >> 172.21.15.18:0/248629977 conn(0x7fd3f80714c0 msgr2=0x7fd3f80fcf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:25:40.804 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:40.808+0000 7fd3fce00700 1 -- 172.21.15.18:0/248629977 shutdown_connections 2024-04-12T21:25:40.804 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:40.808+0000 7fd3fce00700 1 -- 172.21.15.18:0/248629977 wait complete. 2024-04-12T21:25:40.950 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:40.954+0000 7f74b5200700 1 -- 172.21.15.18:0/630707833 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f74b0065630 msgr2=0x7f74b0065a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:25:40.950 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:40.954+0000 7f74b5200700 1 --2- 172.21.15.18:0/630707833 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f74b0065630 0x7f74b0065a80 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f7498004940 tx=0x7f7498009e30 comp rx=0 tx=0).stop 2024-04-12T21:25:40.951 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:40.954+0000 7f74b5200700 1 -- 172.21.15.18:0/630707833 shutdown_connections 2024-04-12T21:25:40.951 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:40.954+0000 7f74b5200700 1 --2- 172.21.15.18:0/630707833 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f74b0065630 0x7f74b0065a80 unknown :-1 s=CLOSED pgs=658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:25:40.951 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:40.954+0000 7f74b5200700 1 --2- 172.21.15.18:0/630707833 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f74b0102ca0 0x7f74b0103070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:25:40.951 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:40.954+0000 7f74b5200700 1 -- 172.21.15.18:0/630707833 >> 172.21.15.18:0/630707833 conn(0x7f74b0071b70 msgr2=0x7f74b0071f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:25:40.951 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:40.954+0000 7f74b5200700 1 -- 172.21.15.18:0/630707833 shutdown_connections 2024-04-12T21:25:40.951 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:40.955+0000 7f74b5200700 1 -- 172.21.15.18:0/630707833 wait complete. 2024-04-12T21:25:40.952 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:40.955+0000 7f74b5200700 1 Processor -- start 2024-04-12T21:25:40.952 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:40.955+0000 7f74b5200700 1 -- start start 2024-04-12T21:25:40.952 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:40.956+0000 7f74b5200700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f74b0065630 0x7f74b0113990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:25:40.952 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:40.956+0000 7f74b5200700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f74b0102ca0 0x7f74b0113ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:25:40.952 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:40.956+0000 7f74b5200700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74b0114520 con 0x7f74b0065630 2024-04-12T21:25:40.952 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:40.956+0000 7f74b5200700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74b010d910 con 0x7f74b0102ca0 2024-04-12T21:25:40.952 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:40.956+0000 7f74af400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f74b0065630 0x7f74b0113990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:25:40.952 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:40.956+0000 7f74af400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f74b0065630 0x7f74b0113990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:36898/0 (socket says 172.21.15.18:36898) 2024-04-12T21:25:40.952 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:40.956+0000 7f74af400700 1 -- 172.21.15.18:0/114327023 learned_addr learned my addr 172.21.15.18:0/114327023 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:25:40.953 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:40.957+0000 7f74a7e00700 1 --2- 172.21.15.18:0/114327023 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f74b0102ca0 0x7f74b0113ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:25:40.953 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:40.957+0000 7f74af400700 1 -- 172.21.15.18:0/114327023 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f74b0102ca0 msgr2=0x7f74b0113ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:25:40.953 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:40.957+0000 7f74af400700 1 --2- 172.21.15.18:0/114327023 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f74b0102ca0 0x7f74b0113ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:25:40.954 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:40.957+0000 7f74af400700 1 -- 172.21.15.18:0/114327023 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7498005040 con 0x7f74b0065630 2024-04-12T21:25:40.954 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:40.957+0000 7f74af400700 1 --2- 172.21.15.18:0/114327023 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f74b0065630 0x7f74b0113990 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f74a0002a50 tx=0x7f74a00046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:25:40.954 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:40.957+0000 7f74a7400700 1 -- 172.21.15.18:0/114327023 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f74a0007cc0 con 0x7f74b0065630 2024-04-12T21:25:40.954 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:40.957+0000 7f74a7400700 1 -- 172.21.15.18:0/114327023 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f74a000f040 con 0x7f74b0065630 2024-04-12T21:25:40.954 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:40.957+0000 7f74a7400700 1 -- 172.21.15.18:0/114327023 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f74a0009640 con 0x7f74b0065630 2024-04-12T21:25:40.954 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:40.957+0000 7f74b5200700 1 -- 172.21.15.18:0/114327023 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f74b010dbf0 con 0x7f74b0065630 2024-04-12T21:25:40.955 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:40.958+0000 7f74b5200700 1 -- 172.21.15.18:0/114327023 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f74b010e1b0 con 0x7f74b0065630 2024-04-12T21:25:40.955 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:40.958+0000 7f74b5200700 1 -- 172.21.15.18:0/114327023 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f74b00fe380 con 0x7f74b0065630 2024-04-12T21:25:40.955 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:40.960+0000 7f74a7400700 1 -- 172.21.15.18:0/114327023 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7f74a00097a0 con 0x7f74b0065630 2024-04-12T21:25:40.957 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:40.961+0000 7f74a7400700 1 --2- 172.21.15.18:0/114327023 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f749406ec10 0x7f74940710c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:25:40.957 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:40.961+0000 7f74a7400700 1 -- 172.21.15.18:0/114327023 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 6757+0+0 (secure 0 0 0) 0x7f74a008e760 con 0x7f74b0065630 2024-04-12T21:25:40.957 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:40.961+0000 7f74a7e00700 1 --2- 172.21.15.18:0/114327023 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f749406ec10 0x7f74940710c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:25:40.957 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:40.962+0000 7f74a7e00700 1 --2- 172.21.15.18:0/114327023 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f749406ec10 0x7f74940710c0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f749800b020 tx=0x7f7498009e30 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:25:40.961 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:40.965+0000 7f74a7400700 1 -- 172.21.15.18:0/114327023 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f74a005af10 con 0x7f74b0065630 2024-04-12T21:25:41.029 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:25:40 smithi186 ceph-mon[86379]: pgmap v777: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:25:41.030 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:25:40 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T21:25:41.100 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:25:40 smithi018 ceph-mon[112590]: pgmap v777: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:25:41.101 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:25:40 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T21:25:41.209 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:41.212+0000 7f74b5200700 1 -- 172.21.15.18:0/114327023 --> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f74b0109d80 con 0x7f749406ec10 2024-04-12T21:25:41.224 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:41.228+0000 7f74a7400700 1 -- 172.21.15.18:0/114327023 <== mgr.34102 v2:172.21.15.18:6800/3812891919 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7f749800b5c0 con 0x7f749406ec10 2024-04-12T21:25:41.224 INFO:teuthology.orchestra.run.smithi018.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-12T21:25:41.224 INFO:teuthology.orchestra.run.smithi018.stdout:alertmanager.smithi018 smithi018 *:9093,9094 running (26m) 2m ago 40m 15.5M - 0.25.0 c8568f914cd2 c332090068e8 2024-04-12T21:25:41.225 INFO:teuthology.orchestra.run.smithi018.stdout:crash.smithi018 smithi018 running (22m) 2m ago 41m 6665k - 19.0.0-2879-g324baf8d 8717720dc5e5 b7f80d1f7982 2024-04-12T21:25:41.225 INFO:teuthology.orchestra.run.smithi018.stdout:crash.smithi186 smithi186 running (22m) 2m ago 39m 6660k - 19.0.0-2879-g324baf8d 8717720dc5e5 a4f55c81b623 2024-04-12T21:25:41.225 INFO:teuthology.orchestra.run.smithi018.stdout:grafana.smithi018 smithi018 *:3000 running (25m) 2m ago 40m 74.8M - 9.4.12 2bacad6d85d8 b08e3bf176dd 2024-04-12T21:25:41.225 INFO:teuthology.orchestra.run.smithi018.stdout:mds.cephfs.smithi018.acqbsc smithi018 running (18m) 2m ago 33m 16.7M - 19.0.0-2879-g324baf8d 8717720dc5e5 021f99ab24ae 2024-04-12T21:25:41.225 INFO:teuthology.orchestra.run.smithi018.stdout:mds.cephfs.smithi018.szjluo smithi018 running (18m) 2m ago 33m 16.5M - 19.0.0-2879-g324baf8d 8717720dc5e5 0003117479d4 2024-04-12T21:25:41.225 INFO:teuthology.orchestra.run.smithi018.stdout:mds.cephfs.smithi186.jdllyn smithi186 running (2m) 2m ago 33m 11.0M - 19.0.0-2879-g324baf8d 8717720dc5e5 21b1f46eef23 2024-04-12T21:25:41.225 INFO:teuthology.orchestra.run.smithi018.stdout:mds.cephfs.smithi186.wjnafs smithi186 running (33m) 2m ago 33m 19.5M - 17.2.7-827-gd3b5060a 21de9b7369ad 74b115d6d6fd 2024-04-12T21:25:41.225 INFO:teuthology.orchestra.run.smithi018.stdout:mgr.smithi018.xjzwga smithi018 *:8443,9283,8765 running (28m) 2m ago 42m 512M - 19.0.0-2879-g324baf8d 8717720dc5e5 56088305eab7 2024-04-12T21:25:41.225 INFO:teuthology.orchestra.run.smithi018.stdout:mgr.smithi186.kqcfxk smithi186 *:8443,9283,8765 running (27m) 2m ago 39m 459M - 19.0.0-2879-g324baf8d 8717720dc5e5 09cd766e5e0d 2024-04-12T21:25:41.225 INFO:teuthology.orchestra.run.smithi018.stdout:mon.smithi018 smithi018 running (24m) 2m ago 42m 73.1M 2048M 19.0.0-2879-g324baf8d 8717720dc5e5 df3e201dfed6 2024-04-12T21:25:41.225 INFO:teuthology.orchestra.run.smithi018.stdout:mon.smithi186 smithi186 running (23m) 2m ago 39m 47.2M 2048M 19.0.0-2879-g324baf8d 8717720dc5e5 c77498da9b2b 2024-04-12T21:25:41.225 INFO:teuthology.orchestra.run.smithi018.stdout:node-exporter.smithi018 smithi018 *:9100 running (27m) 2m ago 40m 8108k - 1.5.0 0da6a335fe13 326c3d9c6847 2024-04-12T21:25:41.225 INFO:teuthology.orchestra.run.smithi018.stdout:node-exporter.smithi186 smithi186 *:9100 running (27m) 2m ago 39m 8757k - 1.5.0 0da6a335fe13 ec2112baeecf 2024-04-12T21:25:41.225 INFO:teuthology.orchestra.run.smithi018.stdout:osd.0 smithi018 running (21m) 2m ago 38m 123M 1559M 19.0.0-2879-g324baf8d 8717720dc5e5 60b1273ca047 2024-04-12T21:25:41.225 INFO:teuthology.orchestra.run.smithi018.stdout:osd.1 smithi018 running (21m) 2m ago 37m 95.1M 1559M 19.0.0-2879-g324baf8d 8717720dc5e5 170ce1e5fb8d 2024-04-12T21:25:41.226 INFO:teuthology.orchestra.run.smithi018.stdout:osd.2 smithi018 running (20m) 2m ago 37m 94.3M 1559M 19.0.0-2879-g324baf8d 8717720dc5e5 c2cbc6aa2839 2024-04-12T21:25:41.226 INFO:teuthology.orchestra.run.smithi018.stdout:osd.3 smithi186 running (20m) 2m ago 36m 101M 2581M 19.0.0-2879-g324baf8d 8717720dc5e5 0138e8262e4b 2024-04-12T21:25:41.226 INFO:teuthology.orchestra.run.smithi018.stdout:osd.4 smithi186 running (19m) 2m ago 36m 91.0M 2581M 19.0.0-2879-g324baf8d 8717720dc5e5 b20cb63b25c9 2024-04-12T21:25:41.226 INFO:teuthology.orchestra.run.smithi018.stdout:osd.5 smithi186 running (18m) 2m ago 35m 92.0M 2581M 19.0.0-2879-g324baf8d 8717720dc5e5 2513bea2610e 2024-04-12T21:25:41.226 INFO:teuthology.orchestra.run.smithi018.stdout:prometheus.smithi018 smithi018 *:9095 running (26m) 2m ago 40m 67.9M - 2.43.0 a07b618ecd1d f58841127e21 2024-04-12T21:25:41.226 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:41.230+0000 7f74b5200700 1 -- 172.21.15.18:0/114327023 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f749406ec10 msgr2=0x7f74940710c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:25:41.226 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:41.231+0000 7f74b5200700 1 --2- 172.21.15.18:0/114327023 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f749406ec10 0x7f74940710c0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f749800b020 tx=0x7f7498009e30 comp rx=0 tx=0).stop 2024-04-12T21:25:41.227 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:41.231+0000 7f74b5200700 1 -- 172.21.15.18:0/114327023 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f74b0065630 msgr2=0x7f74b0113990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:25:41.227 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:41.231+0000 7f74b5200700 1 --2- 172.21.15.18:0/114327023 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f74b0065630 0x7f74b0113990 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f74a0002a50 tx=0x7f74a00046b0 comp rx=0 tx=0).stop 2024-04-12T21:25:41.227 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:41.231+0000 7f74b5200700 1 -- 172.21.15.18:0/114327023 shutdown_connections 2024-04-12T21:25:41.227 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:41.231+0000 7f74b5200700 1 --2- 172.21.15.18:0/114327023 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f749406ec10 0x7f74940710c0 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:25:41.227 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:41.231+0000 7f74b5200700 1 --2- 172.21.15.18:0/114327023 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f74b0065630 0x7f74b0113990 unknown :-1 s=CLOSED pgs=659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:25:41.227 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:41.231+0000 7f74b5200700 1 --2- 172.21.15.18:0/114327023 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f74b0102ca0 0x7f74b0113ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:25:41.228 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:41.231+0000 7f74b5200700 1 -- 172.21.15.18:0/114327023 >> 172.21.15.18:0/114327023 conn(0x7f74b0071b70 msgr2=0x7f74b01007f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:25:41.228 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:41.231+0000 7f74b5200700 1 -- 172.21.15.18:0/114327023 shutdown_connections 2024-04-12T21:25:41.228 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:41.231+0000 7f74b5200700 1 -- 172.21.15.18:0/114327023 wait complete. 2024-04-12T21:25:41.383 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:41.386+0000 7fd9e9e00700 1 -- 172.21.15.18:0/2160760420 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd9e40fc5c0 msgr2=0x7fd9e4100be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:25:41.383 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:41.386+0000 7fd9e9e00700 1 --2- 172.21.15.18:0/2160760420 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd9e40fc5c0 0x7fd9e4100be0 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7fd9d0004940 tx=0x7fd9d0009e30 comp rx=0 tx=0).stop 2024-04-12T21:25:41.383 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:41.386+0000 7fd9e9e00700 1 -- 172.21.15.18:0/2160760420 shutdown_connections 2024-04-12T21:25:41.383 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:41.386+0000 7fd9e9e00700 1 --2- 172.21.15.18:0/2160760420 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd9e40fc5c0 0x7fd9e4100be0 unknown :-1 s=CLOSED pgs=660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:25:41.383 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:41.386+0000 7fd9e9e00700 1 --2- 172.21.15.18:0/2160760420 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd9e40fbc20 0x7fd9e40fbff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:25:41.383 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:41.386+0000 7fd9e9e00700 1 -- 172.21.15.18:0/2160760420 >> 172.21.15.18:0/2160760420 conn(0x7fd9e40f78c0 msgr2=0x7fd9e40f9cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:25:41.383 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:41.386+0000 7fd9e9e00700 1 -- 172.21.15.18:0/2160760420 shutdown_connections 2024-04-12T21:25:41.383 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:41.387+0000 7fd9e9e00700 1 -- 172.21.15.18:0/2160760420 wait complete. 2024-04-12T21:25:41.384 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:41.387+0000 7fd9e9e00700 1 Processor -- start 2024-04-12T21:25:41.384 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:41.387+0000 7fd9e9e00700 1 -- start start 2024-04-12T21:25:41.385 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:41.388+0000 7fd9e9e00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd9e40fbc20 0x7fd9e411a000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:25:41.385 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:41.388+0000 7fd9e9e00700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd9e40fc5c0 0x7fd9e411a540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:25:41.385 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:41.388+0000 7fd9e9e00700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9e411ab90 con 0x7fd9e40fbc20 2024-04-12T21:25:41.385 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:41.388+0000 7fd9e9e00700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9e4114170 con 0x7fd9e40fc5c0 2024-04-12T21:25:41.385 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:41.388+0000 7fd9e2a00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd9e40fbc20 0x7fd9e411a000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:25:41.385 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:41.388+0000 7fd9e2a00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd9e40fbc20 0x7fd9e411a000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:36922/0 (socket says 172.21.15.18:36922) 2024-04-12T21:25:41.385 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:41.388+0000 7fd9e2a00700 1 -- 172.21.15.18:0/3831802525 learned_addr learned my addr 172.21.15.18:0/3831802525 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:25:41.385 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:41.389+0000 7fd9e2000700 1 --2- 172.21.15.18:0/3831802525 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd9e40fc5c0 0x7fd9e411a540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:25:41.385 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:41.389+0000 7fd9e2a00700 1 -- 172.21.15.18:0/3831802525 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd9e40fc5c0 msgr2=0x7fd9e411a540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:25:41.385 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:41.389+0000 7fd9e2a00700 1 --2- 172.21.15.18:0/3831802525 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd9e40fc5c0 0x7fd9e411a540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:25:41.385 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:41.389+0000 7fd9e2a00700 1 -- 172.21.15.18:0/3831802525 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9d0005040 con 0x7fd9e40fbc20 2024-04-12T21:25:41.386 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:41.389+0000 7fd9e2000700 1 --2- 172.21.15.18:0/3831802525 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd9e40fc5c0 0x7fd9e411a540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-12T21:25:41.386 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:41.389+0000 7fd9e2a00700 1 --2- 172.21.15.18:0/3831802525 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd9e40fbc20 0x7fd9e411a000 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7fd9d400c830 tx=0x7fd9d4004470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:25:41.386 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:41.389+0000 7fd9daa00700 1 -- 172.21.15.18:0/3831802525 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd9d400a760 con 0x7fd9e40fbc20 2024-04-12T21:25:41.387 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:41.389+0000 7fd9daa00700 1 -- 172.21.15.18:0/3831802525 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd9d400ad90 con 0x7fd9e40fbc20 2024-04-12T21:25:41.387 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:41.389+0000 7fd9daa00700 1 -- 172.21.15.18:0/3831802525 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd9d40113a0 con 0x7fd9e40fbc20 2024-04-12T21:25:41.387 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:41.389+0000 7fd9e9e00700 1 -- 172.21.15.18:0/3831802525 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd9e4114450 con 0x7fd9e40fbc20 2024-04-12T21:25:41.387 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:41.389+0000 7fd9e9e00700 1 -- 172.21.15.18:0/3831802525 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd9e4114940 con 0x7fd9e40fbc20 2024-04-12T21:25:41.387 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:41.390+0000 7fd9e9e00700 1 -- 172.21.15.18:0/3831802525 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd9e4066770 con 0x7fd9e40fbc20 2024-04-12T21:25:41.387 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:41.392+0000 7fd9daa00700 1 -- 172.21.15.18:0/3831802525 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7fd9d4012400 con 0x7fd9e40fbc20 2024-04-12T21:25:41.388 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:41.392+0000 7fd9daa00700 1 --2- 172.21.15.18:0/3831802525 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fd9c806ec00 0x7fd9c80710b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:25:41.389 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:41.393+0000 7fd9daa00700 1 -- 172.21.15.18:0/3831802525 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 6757+0+0 (secure 0 0 0) 0x7fd9d408ea80 con 0x7fd9e40fbc20 2024-04-12T21:25:41.389 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:41.393+0000 7fd9e2000700 1 --2- 172.21.15.18:0/3831802525 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fd9c806ec00 0x7fd9c80710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:25:41.389 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:41.393+0000 7fd9e2000700 1 --2- 172.21.15.18:0/3831802525 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fd9c806ec00 0x7fd9c80710b0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fd9d000b3b0 tx=0x7fd9d0009e30 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:25:41.393 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:41.397+0000 7fd9daa00700 1 -- 172.21.15.18:0/3831802525 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7fd9d405b230 con 0x7fd9e40fbc20 2024-04-12T21:25:41.707 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:41.711+0000 7fd9e9e00700 1 -- 172.21.15.18:0/3831802525 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fd9e4115680 con 0x7fd9e40fbc20 2024-04-12T21:25:41.708 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:41.712+0000 7fd9daa00700 1 -- 172.21.15.18:0/3831802525 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+837 (secure 0 0 0) 0x7fd9d405adc0 con 0x7fd9e40fbc20 2024-04-12T21:25:41.708 INFO:teuthology.orchestra.run.smithi018.stdout:{ 2024-04-12T21:25:41.708 INFO:teuthology.orchestra.run.smithi018.stdout: "mon": { 2024-04-12T21:25:41.708 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 19.0.0-2879-g324baf8d (324baf8ddb91dea46cac50cb535c306115791355) squid (dev)": 2 2024-04-12T21:25:41.708 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-12T21:25:41.709 INFO:teuthology.orchestra.run.smithi018.stdout: "mgr": { 2024-04-12T21:25:41.709 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 19.0.0-2879-g324baf8d (324baf8ddb91dea46cac50cb535c306115791355) squid (dev)": 2 2024-04-12T21:25:41.709 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-12T21:25:41.709 INFO:teuthology.orchestra.run.smithi018.stdout: "osd": { 2024-04-12T21:25:41.709 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 19.0.0-2879-g324baf8d (324baf8ddb91dea46cac50cb535c306115791355) squid (dev)": 6 2024-04-12T21:25:41.709 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-12T21:25:41.709 INFO:teuthology.orchestra.run.smithi018.stdout: "mds": { 2024-04-12T21:25:41.709 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 17.2.7-827-gd3b5060a (d3b5060a17449edfae9385cdf792afec109fca5d) quincy (stable)": 1, 2024-04-12T21:25:41.709 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 19.0.0-2879-g324baf8d (324baf8ddb91dea46cac50cb535c306115791355) squid (dev)": 3 2024-04-12T21:25:41.709 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-12T21:25:41.709 INFO:teuthology.orchestra.run.smithi018.stdout: "overall": { 2024-04-12T21:25:41.709 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 17.2.7-827-gd3b5060a (d3b5060a17449edfae9385cdf792afec109fca5d) quincy (stable)": 1, 2024-04-12T21:25:41.709 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 19.0.0-2879-g324baf8d (324baf8ddb91dea46cac50cb535c306115791355) squid (dev)": 13 2024-04-12T21:25:41.709 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-04-12T21:25:41.710 INFO:teuthology.orchestra.run.smithi018.stdout:} 2024-04-12T21:25:41.711 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:41.715+0000 7fd9e9e00700 1 -- 172.21.15.18:0/3831802525 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fd9c806ec00 msgr2=0x7fd9c80710b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:25:41.711 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:41.715+0000 7fd9e9e00700 1 --2- 172.21.15.18:0/3831802525 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fd9c806ec00 0x7fd9c80710b0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fd9d000b3b0 tx=0x7fd9d0009e30 comp rx=0 tx=0).stop 2024-04-12T21:25:41.711 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:41.715+0000 7fd9e9e00700 1 -- 172.21.15.18:0/3831802525 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd9e40fbc20 msgr2=0x7fd9e411a000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:25:41.712 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:41.715+0000 7fd9e9e00700 1 --2- 172.21.15.18:0/3831802525 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd9e40fbc20 0x7fd9e411a000 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7fd9d400c830 tx=0x7fd9d4004470 comp rx=0 tx=0).stop 2024-04-12T21:25:41.712 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:41.715+0000 7fd9e9e00700 1 -- 172.21.15.18:0/3831802525 shutdown_connections 2024-04-12T21:25:41.712 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:41.715+0000 7fd9e9e00700 1 --2- 172.21.15.18:0/3831802525 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fd9c806ec00 0x7fd9c80710b0 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:25:41.712 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:41.715+0000 7fd9e9e00700 1 --2- 172.21.15.18:0/3831802525 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd9e40fbc20 0x7fd9e411a000 unknown :-1 s=CLOSED pgs=661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:25:41.712 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:41.715+0000 7fd9e9e00700 1 --2- 172.21.15.18:0/3831802525 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd9e40fc5c0 0x7fd9e411a540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:25:41.712 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:41.715+0000 7fd9e9e00700 1 -- 172.21.15.18:0/3831802525 >> 172.21.15.18:0/3831802525 conn(0x7fd9e40f78c0 msgr2=0x7fd9e40f8d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:25:41.712 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:41.715+0000 7fd9e9e00700 1 -- 172.21.15.18:0/3831802525 shutdown_connections 2024-04-12T21:25:41.712 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:41.716+0000 7fd9e9e00700 1 -- 172.21.15.18:0/3831802525 wait complete. 2024-04-12T21:25:41.856 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:41.859+0000 7efc28e00700 1 -- 172.21.15.18:0/523659188 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7efc240fe830 msgr2=0x7efc24106f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:25:41.856 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:41.859+0000 7efc28e00700 1 --2- 172.21.15.18:0/523659188 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7efc240fe830 0x7efc24106f00 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7efc10004940 tx=0x7efc10009e30 comp rx=0 tx=0).stop 2024-04-12T21:25:41.856 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:41.860+0000 7efc28e00700 1 -- 172.21.15.18:0/523659188 shutdown_connections 2024-04-12T21:25:41.857 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:41.860+0000 7efc28e00700 1 --2- 172.21.15.18:0/523659188 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7efc240fe830 0x7efc24106f00 unknown :-1 s=CLOSED pgs=662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:25:41.857 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:41.860+0000 7efc28e00700 1 --2- 172.21.15.18:0/523659188 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7efc240fde90 0x7efc240fe260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:25:41.857 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:41.860+0000 7efc28e00700 1 -- 172.21.15.18:0/523659188 >> 172.21.15.18:0/523659188 conn(0x7efc24072bc0 msgr2=0x7efc24072fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:25:41.857 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:41.860+0000 7efc28e00700 1 -- 172.21.15.18:0/523659188 shutdown_connections 2024-04-12T21:25:41.857 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:41.860+0000 7efc28e00700 1 -- 172.21.15.18:0/523659188 wait complete. 2024-04-12T21:25:41.859 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:41.861+0000 7efc28e00700 1 Processor -- start 2024-04-12T21:25:41.859 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:41.861+0000 7efc28e00700 1 -- start start 2024-04-12T21:25:41.859 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:41.861+0000 7efc28e00700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7efc240fde90 0x7efc240fbd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:25:41.859 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:41.861+0000 7efc28e00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7efc240fe830 0x7efc240fc290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:25:41.859 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:41.861+0000 7efc28e00700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc240fc820 con 0x7efc240fe830 2024-04-12T21:25:41.859 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:41.861+0000 7efc28e00700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc240fc960 con 0x7efc240fde90 2024-04-12T21:25:41.859 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:41.861+0000 7efc21400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7efc240fe830 0x7efc240fc290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:25:41.859 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:41.862+0000 7efc21400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7efc240fe830 0x7efc240fc290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:36940/0 (socket says 172.21.15.18:36940) 2024-04-12T21:25:41.859 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:41.862+0000 7efc21400700 1 -- 172.21.15.18:0/1306931456 learned_addr learned my addr 172.21.15.18:0/1306931456 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:25:41.859 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:41.862+0000 7efc21400700 1 -- 172.21.15.18:0/1306931456 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7efc240fde90 msgr2=0x7efc240fbd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:25:41.859 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:41.862+0000 7efc21400700 1 --2- 172.21.15.18:0/1306931456 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7efc240fde90 0x7efc240fbd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:25:41.859 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:41.862+0000 7efc21400700 1 -- 172.21.15.18:0/1306931456 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc10005040 con 0x7efc240fe830 2024-04-12T21:25:41.859 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:41.862+0000 7efc21400700 1 --2- 172.21.15.18:0/1306931456 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7efc240fe830 0x7efc240fc290 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7efc10000c00 tx=0x7efc10009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:25:41.860 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:41.862+0000 7efc1a000700 1 -- 172.21.15.18:0/1306931456 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7efc10021070 con 0x7efc240fe830 2024-04-12T21:25:41.860 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:41.862+0000 7efc1a000700 1 -- 172.21.15.18:0/1306931456 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7efc10013cd0 con 0x7efc240fe830 2024-04-12T21:25:41.860 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:41.863+0000 7efc28e00700 1 -- 172.21.15.18:0/1306931456 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efc240fcbb0 con 0x7efc240fe830 2024-04-12T21:25:41.860 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:41.863+0000 7efc1a000700 1 -- 172.21.15.18:0/1306931456 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7efc1000e070 con 0x7efc240fe830 2024-04-12T21:25:41.860 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:41.863+0000 7efc28e00700 1 -- 172.21.15.18:0/1306931456 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efc2411a1d0 con 0x7efc240fe830 2024-04-12T21:25:41.860 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:41.864+0000 7efc28e00700 1 -- 172.21.15.18:0/1306931456 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efc24065e30 con 0x7efc240fe830 2024-04-12T21:25:41.861 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:41.865+0000 7efc1a000700 1 -- 172.21.15.18:0/1306931456 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7efc1001ca60 con 0x7efc240fe830 2024-04-12T21:25:41.862 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:41.866+0000 7efc1a000700 1 --2- 172.21.15.18:0/1306931456 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7efc0806ec10 0x7efc080710c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:25:41.862 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:41.866+0000 7efc1a000700 1 -- 172.21.15.18:0/1306931456 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 6757+0+0 (secure 0 0 0) 0x7efc1008fde0 con 0x7efc240fe830 2024-04-12T21:25:41.862 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:41.866+0000 7efc21e00700 1 --2- 172.21.15.18:0/1306931456 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7efc0806ec10 0x7efc080710c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:25:41.862 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:41.867+0000 7efc21e00700 1 --2- 172.21.15.18:0/1306931456 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7efc0806ec10 0x7efc080710c0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7efc14009fd0 tx=0x7efc1400b040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:25:41.867 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:41.871+0000 7efc1a000700 1 -- 172.21.15.18:0/1306931456 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7efc10058a60 con 0x7efc240fe830 2024-04-12T21:25:42.029 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:25:41 smithi186 ceph-mon[86379]: from='client.35372 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:25:42.029 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:25:41 smithi186 ceph-mon[86379]: from='client.? 172.21.15.18:0/3831802525' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T21:25:42.100 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:25:41 smithi018 ceph-mon[112590]: from='client.35372 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:25:42.101 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:25:41 smithi018 ceph-mon[112590]: from='client.? 172.21.15.18:0/3831802525' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T21:25:42.134 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:42.138+0000 7efc28e00700 1 -- 172.21.15.18:0/1306931456 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7efc24109fa0 con 0x7efc240fe830 2024-04-12T21:25:42.135 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:42.139+0000 7efc1a000700 1 -- 172.21.15.18:0/1306931456 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 31 v31) v1 ==== 76+0+1873 (secure 0 0 0) 0x7efc1005c120 con 0x7efc240fe830 2024-04-12T21:25:42.136 INFO:teuthology.orchestra.run.smithi018.stdout:e31 2024-04-12T21:25:42.136 INFO:teuthology.orchestra.run.smithi018.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-12T21:25:42.136 INFO:teuthology.orchestra.run.smithi018.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-12T21:25:42.136 INFO:teuthology.orchestra.run.smithi018.stdout:legacy client fscid: 1 2024-04-12T21:25:42.136 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-12T21:25:42.136 INFO:teuthology.orchestra.run.smithi018.stdout:Filesystem 'cephfs' (1) 2024-04-12T21:25:42.136 INFO:teuthology.orchestra.run.smithi018.stdout:fs_name cephfs 2024-04-12T21:25:42.136 INFO:teuthology.orchestra.run.smithi018.stdout:epoch 24 2024-04-12T21:25:42.137 INFO:teuthology.orchestra.run.smithi018.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-04-12T21:25:42.137 INFO:teuthology.orchestra.run.smithi018.stdout:created 2024-04-12T20:51:57.362850+0000 2024-04-12T21:25:42.137 INFO:teuthology.orchestra.run.smithi018.stdout:modified 2024-04-12T21:06:59.717190+0000 2024-04-12T21:25:42.137 INFO:teuthology.orchestra.run.smithi018.stdout:tableserver 0 2024-04-12T21:25:42.137 INFO:teuthology.orchestra.run.smithi018.stdout:root 0 2024-04-12T21:25:42.137 INFO:teuthology.orchestra.run.smithi018.stdout:session_timeout 60 2024-04-12T21:25:42.137 INFO:teuthology.orchestra.run.smithi018.stdout:session_autoclose 300 2024-04-12T21:25:42.137 INFO:teuthology.orchestra.run.smithi018.stdout:max_file_size 1099511627776 2024-04-12T21:25:42.137 INFO:teuthology.orchestra.run.smithi018.stdout:max_xattr_size 65536 2024-04-12T21:25:42.137 INFO:teuthology.orchestra.run.smithi018.stdout:required_client_features {} 2024-04-12T21:25:42.137 INFO:teuthology.orchestra.run.smithi018.stdout:last_failure 0 2024-04-12T21:25:42.137 INFO:teuthology.orchestra.run.smithi018.stdout:last_failure_osd_epoch 100 2024-04-12T21:25:42.137 INFO:teuthology.orchestra.run.smithi018.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-12T21:25:42.137 INFO:teuthology.orchestra.run.smithi018.stdout:max_mds 2 2024-04-12T21:25:42.137 INFO:teuthology.orchestra.run.smithi018.stdout:in 0,1 2024-04-12T21:25:42.138 INFO:teuthology.orchestra.run.smithi018.stdout:up {} 2024-04-12T21:25:42.138 INFO:teuthology.orchestra.run.smithi018.stdout:failed 0,1 2024-04-12T21:25:42.138 INFO:teuthology.orchestra.run.smithi018.stdout:damaged 2024-04-12T21:25:42.138 INFO:teuthology.orchestra.run.smithi018.stdout:stopped 2024-04-12T21:25:42.138 INFO:teuthology.orchestra.run.smithi018.stdout:data_pools [3] 2024-04-12T21:25:42.138 INFO:teuthology.orchestra.run.smithi018.stdout:metadata_pool 2 2024-04-12T21:25:42.138 INFO:teuthology.orchestra.run.smithi018.stdout:inline_data disabled 2024-04-12T21:25:42.138 INFO:teuthology.orchestra.run.smithi018.stdout:balancer 2024-04-12T21:25:42.138 INFO:teuthology.orchestra.run.smithi018.stdout:bal_rank_mask -1 2024-04-12T21:25:42.138 INFO:teuthology.orchestra.run.smithi018.stdout:standby_count_wanted 1 2024-04-12T21:25:42.138 INFO:teuthology.orchestra.run.smithi018.stdout:qdb_cluster leader: 0 members: 2024-04-12T21:25:42.138 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-12T21:25:42.138 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-12T21:25:42.138 INFO:teuthology.orchestra.run.smithi018.stdout:Standby daemons: 2024-04-12T21:25:42.139 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-12T21:25:42.139 INFO:teuthology.orchestra.run.smithi018.stdout:[mds.cephfs.smithi186.wjnafs{-1:24277} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.186:6826/438347207,v1:172.21.15.186:6827/438347207] compat {c=[1],r=[1],i=[7ff]}] 2024-04-12T21:25:42.139 INFO:teuthology.orchestra.run.smithi018.stdout:[mds.cephfs.smithi018.acqbsc{-1:34436} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.18:6828/755179764,v1:172.21.15.18:6829/755179764] compat {c=[1],r=[1],i=[1fff]}] 2024-04-12T21:25:42.139 INFO:teuthology.orchestra.run.smithi018.stdout:[mds.cephfs.smithi018.szjluo{-1:34468} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.18:6826/2407992653,v1:172.21.15.18:6827/2407992653] compat {c=[1],r=[1],i=[1fff]}] 2024-04-12T21:25:42.139 INFO:teuthology.orchestra.run.smithi018.stdout:[mds.cephfs.smithi186.jdllyn{-1:44595} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.186:6824/3523444202,v1:172.21.15.186:6825/3523444202] compat {c=[1],r=[1],i=[1fff]}] 2024-04-12T21:25:42.139 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:42.142+0000 7efc28e00700 1 -- 172.21.15.18:0/1306931456 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7efc0806ec10 msgr2=0x7efc080710c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:25:42.139 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:42.142+0000 7efc28e00700 1 --2- 172.21.15.18:0/1306931456 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7efc0806ec10 0x7efc080710c0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7efc14009fd0 tx=0x7efc1400b040 comp rx=0 tx=0).stop 2024-04-12T21:25:42.140 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:42.142+0000 7efc28e00700 1 -- 172.21.15.18:0/1306931456 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7efc240fe830 msgr2=0x7efc240fc290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:25:42.140 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:42.142+0000 7efc28e00700 1 --2- 172.21.15.18:0/1306931456 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7efc240fe830 0x7efc240fc290 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7efc10000c00 tx=0x7efc10009e30 comp rx=0 tx=0).stop 2024-04-12T21:25:42.140 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:42.142+0000 7efc28e00700 1 -- 172.21.15.18:0/1306931456 shutdown_connections 2024-04-12T21:25:42.140 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:42.142+0000 7efc28e00700 1 --2- 172.21.15.18:0/1306931456 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7efc0806ec10 0x7efc080710c0 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:25:42.140 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:42.142+0000 7efc28e00700 1 --2- 172.21.15.18:0/1306931456 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7efc240fde90 0x7efc240fbd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:25:42.140 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:42.142+0000 7efc28e00700 1 --2- 172.21.15.18:0/1306931456 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7efc240fe830 0x7efc240fc290 unknown :-1 s=CLOSED pgs=663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:25:42.140 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:42.142+0000 7efc28e00700 1 -- 172.21.15.18:0/1306931456 >> 172.21.15.18:0/1306931456 conn(0x7efc24072bc0 msgr2=0x7efc24103740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:25:42.140 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:42.142+0000 7efc28e00700 1 -- 172.21.15.18:0/1306931456 shutdown_connections 2024-04-12T21:25:42.140 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:42.142+0000 7efc28e00700 1 -- 172.21.15.18:0/1306931456 wait complete. 2024-04-12T21:25:42.140 INFO:teuthology.orchestra.run.smithi018.stderr:dumped fsmap epoch 31 2024-04-12T21:25:42.297 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:42.301+0000 7f146d600700 1 -- 172.21.15.18:0/980754504 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f14680fb2d0 msgr2=0x7f14680fb720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:25:42.297 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:42.301+0000 7f146d600700 1 --2- 172.21.15.18:0/980754504 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f14680fb2d0 0x7f14680fb720 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f1454004940 tx=0x7f1454009e30 comp rx=0 tx=0).stop 2024-04-12T21:25:42.297 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:42.301+0000 7f146d600700 1 -- 172.21.15.18:0/980754504 shutdown_connections 2024-04-12T21:25:42.297 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:42.301+0000 7f146d600700 1 --2- 172.21.15.18:0/980754504 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f14680fb2d0 0x7f14680fb720 unknown :-1 s=CLOSED pgs=664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:25:42.298 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:42.301+0000 7f146d600700 1 --2- 172.21.15.18:0/980754504 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1468102e30 0x7f1468103200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:25:42.298 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:42.301+0000 7f146d600700 1 -- 172.21.15.18:0/980754504 >> 172.21.15.18:0/980754504 conn(0x7f14680721c0 msgr2=0x7f14680725c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:25:42.298 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:42.301+0000 7f146d600700 1 -- 172.21.15.18:0/980754504 shutdown_connections 2024-04-12T21:25:42.298 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:42.301+0000 7f146d600700 1 -- 172.21.15.18:0/980754504 wait complete. 2024-04-12T21:25:42.299 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:42.302+0000 7f146d600700 1 Processor -- start 2024-04-12T21:25:42.299 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:42.302+0000 7f146d600700 1 -- start start 2024-04-12T21:25:42.299 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:42.302+0000 7f146d600700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f14680fb2d0 0x7f14680fc4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:25:42.300 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:42.302+0000 7f146d600700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1468102e30 0x7f14680fc9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:25:42.300 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:42.303+0000 7f146d600700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14680fcf80 con 0x7f1468102e30 2024-04-12T21:25:42.300 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:42.303+0000 7f146d600700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14680fd0c0 con 0x7f14680fb2d0 2024-04-12T21:25:42.300 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:42.303+0000 7f1467400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1468102e30 0x7f14680fc9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:25:42.300 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:42.303+0000 7f1467400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1468102e30 0x7f14680fc9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:36954/0 (socket says 172.21.15.18:36954) 2024-04-12T21:25:42.300 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:42.303+0000 7f1467400700 1 -- 172.21.15.18:0/2613231942 learned_addr learned my addr 172.21.15.18:0/2613231942 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:25:42.300 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:42.303+0000 7f1467e00700 1 --2- 172.21.15.18:0/2613231942 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f14680fb2d0 0x7f14680fc4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:25:42.300 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:42.303+0000 7f1467400700 1 -- 172.21.15.18:0/2613231942 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f14680fb2d0 msgr2=0x7f14680fc4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:25:42.300 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:42.303+0000 7f1467400700 1 --2- 172.21.15.18:0/2613231942 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f14680fb2d0 0x7f14680fc4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:25:42.300 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:42.303+0000 7f1467400700 1 -- 172.21.15.18:0/2613231942 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1454005040 con 0x7f1468102e30 2024-04-12T21:25:42.300 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:42.303+0000 7f1467e00700 1 --2- 172.21.15.18:0/2613231942 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f14680fb2d0 0x7f14680fc4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-12T21:25:42.300 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:42.304+0000 7f1467400700 1 --2- 172.21.15.18:0/2613231942 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1468102e30 0x7f14680fc9f0 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f1454009180 tx=0x7f1454009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:25:42.301 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:42.304+0000 7f145fe00700 1 -- 172.21.15.18:0/2613231942 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f145400e040 con 0x7f1468102e30 2024-04-12T21:25:42.301 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:42.304+0000 7f145fe00700 1 -- 172.21.15.18:0/2613231942 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f145400bbc0 con 0x7f1468102e30 2024-04-12T21:25:42.301 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:42.304+0000 7f146d600700 1 -- 172.21.15.18:0/2613231942 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f14680fd310 con 0x7f1468102e30 2024-04-12T21:25:42.301 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:42.304+0000 7f145fe00700 1 -- 172.21.15.18:0/2613231942 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f145401c630 con 0x7f1468102e30 2024-04-12T21:25:42.301 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:42.304+0000 7f146d600700 1 -- 172.21.15.18:0/2613231942 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f146811a130 con 0x7f1468102e30 2024-04-12T21:25:42.301 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:42.305+0000 7f146d600700 1 -- 172.21.15.18:0/2613231942 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f14681051c0 con 0x7f1468102e30 2024-04-12T21:25:42.303 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:42.306+0000 7f145fe00700 1 -- 172.21.15.18:0/2613231942 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7f145400bd30 con 0x7f1468102e30 2024-04-12T21:25:42.304 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:42.307+0000 7f145fe00700 1 --2- 172.21.15.18:0/2613231942 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f144c06ec00 0x7f144c0710b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:25:42.304 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:42.307+0000 7f145fe00700 1 -- 172.21.15.18:0/2613231942 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 6757+0+0 (secure 0 0 0) 0x7f145408fca0 con 0x7f1468102e30 2024-04-12T21:25:42.304 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:42.308+0000 7f1467e00700 1 --2- 172.21.15.18:0/2613231942 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f144c06ec00 0x7f144c0710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:25:42.304 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:42.308+0000 7f1467e00700 1 --2- 172.21.15.18:0/2613231942 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f144c06ec00 0x7f144c0710b0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f1458007fd0 tx=0x7f1458009040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:25:42.308 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:42.312+0000 7f145fe00700 1 -- 172.21.15.18:0/2613231942 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f1454094070 con 0x7f1468102e30 2024-04-12T21:25:42.558 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:42.561+0000 7f146d600700 1 -- 172.21.15.18:0/2613231942 --> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f14680fdba0 con 0x7f144c06ec00 2024-04-12T21:25:42.560 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:42.564+0000 7f145fe00700 1 -- 172.21.15.18:0/2613231942 <== mgr.34102 v2:172.21.15.18:6800/3812891919 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f14680fdba0 con 0x7f144c06ec00 2024-04-12T21:25:42.560 INFO:teuthology.orchestra.run.smithi018.stdout:{ 2024-04-12T21:25:42.560 INFO:teuthology.orchestra.run.smithi018.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:f4452def3be4d350f4a8e5feaf7ebb1511be9298fe68478ba7a2d6fbd7d69a63", 2024-04-12T21:25:42.560 INFO:teuthology.orchestra.run.smithi018.stdout: "in_progress": true, 2024-04-12T21:25:42.561 INFO:teuthology.orchestra.run.smithi018.stdout: "which": "Upgrading all daemon types on all hosts", 2024-04-12T21:25:42.561 INFO:teuthology.orchestra.run.smithi018.stdout: "services_complete": [ 2024-04-12T21:25:42.561 INFO:teuthology.orchestra.run.smithi018.stdout: "crash", 2024-04-12T21:25:42.561 INFO:teuthology.orchestra.run.smithi018.stdout: "osd", 2024-04-12T21:25:42.561 INFO:teuthology.orchestra.run.smithi018.stdout: "mon", 2024-04-12T21:25:42.561 INFO:teuthology.orchestra.run.smithi018.stdout: "mgr" 2024-04-12T21:25:42.561 INFO:teuthology.orchestra.run.smithi018.stdout: ], 2024-04-12T21:25:42.561 INFO:teuthology.orchestra.run.smithi018.stdout: "progress": "15/21 daemons upgraded", 2024-04-12T21:25:42.561 INFO:teuthology.orchestra.run.smithi018.stdout: "message": "Currently upgrading mds daemons", 2024-04-12T21:25:42.561 INFO:teuthology.orchestra.run.smithi018.stdout: "is_paused": false 2024-04-12T21:25:42.561 INFO:teuthology.orchestra.run.smithi018.stdout:} 2024-04-12T21:25:42.564 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:42.568+0000 7f146d600700 1 -- 172.21.15.18:0/2613231942 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f144c06ec00 msgr2=0x7f144c0710b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:25:42.564 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:42.568+0000 7f146d600700 1 --2- 172.21.15.18:0/2613231942 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f144c06ec00 0x7f144c0710b0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f1458007fd0 tx=0x7f1458009040 comp rx=0 tx=0).stop 2024-04-12T21:25:42.564 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:42.568+0000 7f146d600700 1 -- 172.21.15.18:0/2613231942 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1468102e30 msgr2=0x7f14680fc9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:25:42.564 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:42.568+0000 7f146d600700 1 --2- 172.21.15.18:0/2613231942 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1468102e30 0x7f14680fc9f0 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f1454009180 tx=0x7f1454009e30 comp rx=0 tx=0).stop 2024-04-12T21:25:42.565 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:42.568+0000 7f146d600700 1 -- 172.21.15.18:0/2613231942 shutdown_connections 2024-04-12T21:25:42.565 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:42.568+0000 7f146d600700 1 --2- 172.21.15.18:0/2613231942 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f144c06ec00 0x7f144c0710b0 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:25:42.565 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:42.568+0000 7f146d600700 1 --2- 172.21.15.18:0/2613231942 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f14680fb2d0 0x7f14680fc4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:25:42.565 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:42.568+0000 7f146d600700 1 --2- 172.21.15.18:0/2613231942 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1468102e30 0x7f14680fc9f0 unknown :-1 s=CLOSED pgs=665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:25:42.565 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:42.568+0000 7f146d600700 1 -- 172.21.15.18:0/2613231942 >> 172.21.15.18:0/2613231942 conn(0x7f14680721c0 msgr2=0x7f14680faaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:25:42.566 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:42.569+0000 7f146d600700 1 -- 172.21.15.18:0/2613231942 shutdown_connections 2024-04-12T21:25:42.566 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:42.569+0000 7f146d600700 1 -- 172.21.15.18:0/2613231942 wait complete. 2024-04-12T21:25:42.728 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:42.731+0000 7f4902800700 1 -- 172.21.15.18:0/1385790071 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f48fc0fbc80 msgr2=0x7f48fc0fc050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:25:42.728 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:42.731+0000 7f4902800700 1 --2- 172.21.15.18:0/1385790071 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f48fc0fbc80 0x7f48fc0fc050 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f48ec009ce0 tx=0x7f48ec004d10 comp rx=0 tx=0).stop 2024-04-12T21:25:42.728 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:42.731+0000 7f4902800700 1 -- 172.21.15.18:0/1385790071 shutdown_connections 2024-04-12T21:25:42.728 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:42.731+0000 7f4902800700 1 --2- 172.21.15.18:0/1385790071 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f48fc0fc590 0x7f48fc1092f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:25:42.728 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:42.731+0000 7f4902800700 1 --2- 172.21.15.18:0/1385790071 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f48fc0fbc80 0x7f48fc0fc050 unknown :-1 s=CLOSED pgs=666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:25:42.728 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:42.731+0000 7f4902800700 1 -- 172.21.15.18:0/1385790071 >> 172.21.15.18:0/1385790071 conn(0x7f48fc072bc0 msgr2=0x7f48fc072fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:25:42.728 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:42.731+0000 7f4902800700 1 -- 172.21.15.18:0/1385790071 shutdown_connections 2024-04-12T21:25:42.728 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:42.731+0000 7f4902800700 1 -- 172.21.15.18:0/1385790071 wait complete. 2024-04-12T21:25:42.729 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:42.732+0000 7f4902800700 1 Processor -- start 2024-04-12T21:25:42.729 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:42.733+0000 7f4902800700 1 -- start start 2024-04-12T21:25:42.729 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:42.733+0000 7f4902800700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f48fc0fbc80 0x7f48fc115f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:25:42.729 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:42.733+0000 7f4902800700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f48fc0fc590 0x7f48fc116450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:25:42.729 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:42.733+0000 7f4902800700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48fc116aa0 con 0x7f48fc0fbc80 2024-04-12T21:25:42.730 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:42.733+0000 7f4902800700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48fc10fe70 con 0x7f48fc0fc590 2024-04-12T21:25:42.730 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:42.733+0000 7f48fb400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f48fc0fbc80 0x7f48fc115f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:25:42.730 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:42.733+0000 7f48fb400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f48fc0fbc80 0x7f48fc115f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:36970/0 (socket says 172.21.15.18:36970) 2024-04-12T21:25:42.730 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:42.733+0000 7f48fb400700 1 -- 172.21.15.18:0/1248594950 learned_addr learned my addr 172.21.15.18:0/1248594950 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:25:42.731 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:42.734+0000 7f48f2a00700 1 --2- 172.21.15.18:0/1248594950 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f48fc0fc590 0x7f48fc116450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:25:42.731 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:42.734+0000 7f48fb400700 1 -- 172.21.15.18:0/1248594950 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f48fc0fc590 msgr2=0x7f48fc116450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:25:42.731 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:42.734+0000 7f48fb400700 1 --2- 172.21.15.18:0/1248594950 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f48fc0fc590 0x7f48fc116450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:25:42.731 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:42.734+0000 7f48fb400700 1 -- 172.21.15.18:0/1248594950 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48ec009940 con 0x7f48fc0fbc80 2024-04-12T21:25:42.731 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:42.734+0000 7f48fb400700 1 --2- 172.21.15.18:0/1248594950 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f48fc0fbc80 0x7f48fc115f10 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7f48ec005010 tx=0x7f48ec004d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:25:42.731 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:42.734+0000 7f48f3e00700 1 -- 172.21.15.18:0/1248594950 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f48ec00e040 con 0x7f48fc0fbc80 2024-04-12T21:25:42.731 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:42.734+0000 7f48f3e00700 1 -- 172.21.15.18:0/1248594950 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f48ec008e20 con 0x7f48fc0fbc80 2024-04-12T21:25:42.731 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:42.734+0000 7f48f3e00700 1 -- 172.21.15.18:0/1248594950 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f48ec01c660 con 0x7f48fc0fbc80 2024-04-12T21:25:42.731 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:42.734+0000 7f4902800700 1 -- 172.21.15.18:0/1248594950 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f48fc1100f0 con 0x7f48fc0fbc80 2024-04-12T21:25:42.731 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:42.735+0000 7f4902800700 1 -- 172.21.15.18:0/1248594950 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f48fc1106b0 con 0x7f48fc0fbc80 2024-04-12T21:25:42.732 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:42.736+0000 7f4902800700 1 -- 172.21.15.18:0/1248594950 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f48fc065e30 con 0x7f48fc0fbc80 2024-04-12T21:25:42.732 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:42.737+0000 7f48f3e00700 1 -- 172.21.15.18:0/1248594950 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7f48ec012a70 con 0x7f48fc0fbc80 2024-04-12T21:25:42.734 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:42.738+0000 7f48f3e00700 1 --2- 172.21.15.18:0/1248594950 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f48e006ec10 0x7f48e00710c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:25:42.734 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:42.738+0000 7f48f3e00700 1 -- 172.21.15.18:0/1248594950 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 6757+0+0 (secure 0 0 0) 0x7f48ec090490 con 0x7f48fc0fbc80 2024-04-12T21:25:42.734 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:42.738+0000 7f48f2a00700 1 --2- 172.21.15.18:0/1248594950 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f48e006ec10 0x7f48e00710c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:25:42.734 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:42.739+0000 7f48f2a00700 1 --2- 172.21.15.18:0/1248594950 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f48e006ec10 0x7f48e00710c0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f48fc1117b0 tx=0x7f48e4009040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:25:42.738 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:42.742+0000 7f48f3e00700 1 -- 172.21.15.18:0/1248594950 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f48ec05cce0 con 0x7f48fc0fbc80 2024-04-12T21:25:42.850 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:25:42 smithi018 ceph-mon[112590]: from='client.35376 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:25:42.851 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:25:42 smithi018 ceph-mon[112590]: from='client.35380 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:25:42.851 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:25:42 smithi018 ceph-mon[112590]: pgmap v778: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:25:42.851 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:25:42 smithi018 ceph-mon[112590]: from='client.? 172.21.15.18:0/1306931456' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-12T21:25:43.029 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:25:42 smithi186 ceph-mon[86379]: from='client.35376 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:25:43.030 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:25:42 smithi186 ceph-mon[86379]: from='client.35380 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:25:43.030 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:25:42 smithi186 ceph-mon[86379]: pgmap v778: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:25:43.030 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:25:42 smithi186 ceph-mon[86379]: from='client.? 172.21.15.18:0/1306931456' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-12T21:25:43.058 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:43.061+0000 7f4902800700 1 -- 172.21.15.18:0/1248594950 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f48fc05f5f0 con 0x7f48fc0fbc80 2024-04-12T21:25:43.058 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:43.062+0000 7f48f3e00700 1 -- 172.21.15.18:0/1248594950 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+4303 (secure 0 0 0) 0x7f48ec05c870 con 0x7f48fc0fbc80 2024-04-12T21:25:43.059 INFO:teuthology.orchestra.run.smithi018.stdout:HEALTH_ERR failed to probe daemons or devices; 1 filesystem is degraded; 1 filesystem is offline 2024-04-12T21:25:43.059 INFO:teuthology.orchestra.run.smithi018.stdout:[WRN] CEPHADM_REFRESH_FAILED: failed to probe daemons or devices 2024-04-12T21:25:43.059 INFO:teuthology.orchestra.run.smithi018.stdout: host smithi018 `cephadm ceph-volume` failed: cephadm exited with an error code: 1, stderr: Inferring config /var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/mon.smithi018/config 2024-04-12T21:25:43.059 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/podman: timeout after 895 seconds 2024-04-12T21:25:43.059 INFO:teuthology.orchestra.run.smithi018.stdout:Non-zero exit code 124 from /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:f4452def3be4d350f4a8e5feaf7ebb1511be9298fe68478ba7a2d6fbd7d69a63 -e NODE_NAME=smithi018 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968:/var/run/ceph:z -v /var/log/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968:/var/log/ceph:z -v /var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmpxsmwkkuh:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:f4452def3be4d350f4a8e5feaf7ebb1511be9298fe68478ba7a2d6fbd7d69a63 inventory --format=json-pretty --filter-for-batch 2024-04-12T21:25:43.059 INFO:teuthology.orchestra.run.smithi018.stdout:Traceback (most recent call last): 2024-04-12T21:25:43.059 INFO:teuthology.orchestra.run.smithi018.stdout: File "/usr/lib64/python3.9/runpy.py", line 197, in _run_module_as_main 2024-04-12T21:25:43.059 INFO:teuthology.orchestra.run.smithi018.stdout: return _run_code(code, main_globals, None, 2024-04-12T21:25:43.059 INFO:teuthology.orchestra.run.smithi018.stdout: File "/usr/lib64/python3.9/runpy.py", line 87, in _run_code 2024-04-12T21:25:43.059 INFO:teuthology.orchestra.run.smithi018.stdout: exec(code, run_globals) 2024-04-12T21:25:43.059 INFO:teuthology.orchestra.run.smithi018.stdout: File "/var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/cephadm.e44f3ed5e34e4ef52eedef233d1190bff004470d8e8005127b5312bff357fa7f/__main__.py", line 5618, in 2024-04-12T21:25:43.059 INFO:teuthology.orchestra.run.smithi018.stdout: File "/var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/cephadm.e44f3ed5e34e4ef52eedef233d1190bff004470d8e8005127b5312bff357fa7f/__main__.py", line 5606, in main 2024-04-12T21:25:43.060 INFO:teuthology.orchestra.run.smithi018.stdout: File "/var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/cephadm.e44f3ed5e34e4ef52eedef233d1190bff004470d8e8005127b5312bff357fa7f/__main__.py", line 408, in _infer_config 2024-04-12T21:25:43.060 INFO:teuthology.orchestra.run.smithi018.stdout: File "/var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/cephadm.e44f3ed5e34e4ef52eedef233d1190bff004470d8e8005127b5312bff357fa7f/__main__.py", line 323, in _infer_fsid 2024-04-12T21:25:43.060 INFO:teuthology.orchestra.run.smithi018.stdout: File "/var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/cephadm.e44f3ed5e34e4ef52eedef233d1190bff004470d8e8005127b5312bff357fa7f/__main__.py", line 436, in _infer_image 2024-04-12T21:25:43.060 INFO:teuthology.orchestra.run.smithi018.stdout: File "/var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/cephadm.e44f3ed5e34e4ef52eedef233d1190bff004470d8e8005127b5312bff357fa7f/__main__.py", line 310, in _validate_fsid 2024-04-12T21:25:43.060 INFO:teuthology.orchestra.run.smithi018.stdout: File "/var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/cephadm.e44f3ed5e34e4ef52eedef233d1190bff004470d8e8005127b5312bff357fa7f/__main__.py", line 3359, in command_ceph_volume 2024-04-12T21:25:43.060 INFO:teuthology.orchestra.run.smithi018.stdout: File "/var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/cephadm.e44f3ed5e34e4ef52eedef233d1190bff004470d8e8005127b5312bff357fa7f/cephadmlib/call_wrappers.py", line 310, in call_throws 2024-04-12T21:25:43.060 INFO:teuthology.orchestra.run.smithi018.stdout:RuntimeError: Failed command: /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:f4452def3be4d350f4a8e5feaf7ebb1511be9298fe68478ba7a2d6fbd7d69a63 -e NODE_NAME=smithi018 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968:/var/run/ceph:z -v /var/log/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968:/var/log/ceph:z -v /var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmpxsmwkkuh:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:f4452def3be4d350f4a8e5feaf7ebb1511be9298fe68478ba7a2d6fbd7d69a63 inventory --format=json-pretty --filter-for-batch 2024-04-12T21:25:43.060 INFO:teuthology.orchestra.run.smithi018.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-04-12T21:25:43.060 INFO:teuthology.orchestra.run.smithi018.stdout: fs cephfs is degraded 2024-04-12T21:25:43.060 INFO:teuthology.orchestra.run.smithi018.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-04-12T21:25:43.060 INFO:teuthology.orchestra.run.smithi018.stdout: fs cephfs is offline because no MDS is active for it. 2024-04-12T21:25:43.061 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:43.065+0000 7f4902800700 1 -- 172.21.15.18:0/1248594950 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f48e006ec10 msgr2=0x7f48e00710c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:25:43.061 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:43.065+0000 7f4902800700 1 --2- 172.21.15.18:0/1248594950 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f48e006ec10 0x7f48e00710c0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f48fc1117b0 tx=0x7f48e4009040 comp rx=0 tx=0).stop 2024-04-12T21:25:43.061 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:43.065+0000 7f4902800700 1 -- 172.21.15.18:0/1248594950 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f48fc0fbc80 msgr2=0x7f48fc115f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:25:43.061 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:43.065+0000 7f4902800700 1 --2- 172.21.15.18:0/1248594950 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f48fc0fbc80 0x7f48fc115f10 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7f48ec005010 tx=0x7f48ec004d10 comp rx=0 tx=0).stop 2024-04-12T21:25:43.062 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:43.065+0000 7f4902800700 1 -- 172.21.15.18:0/1248594950 shutdown_connections 2024-04-12T21:25:43.062 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:43.065+0000 7f4902800700 1 --2- 172.21.15.18:0/1248594950 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f48e006ec10 0x7f48e00710c0 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:25:43.062 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:43.065+0000 7f4902800700 1 --2- 172.21.15.18:0/1248594950 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f48fc0fbc80 0x7f48fc115f10 unknown :-1 s=CLOSED pgs=667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:25:43.062 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:43.065+0000 7f4902800700 1 --2- 172.21.15.18:0/1248594950 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f48fc0fc590 0x7f48fc116450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:25:43.062 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:43.065+0000 7f4902800700 1 -- 172.21.15.18:0/1248594950 >> 172.21.15.18:0/1248594950 conn(0x7f48fc072bc0 msgr2=0x7f48fc105bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:25:43.062 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:43.066+0000 7f4902800700 1 -- 172.21.15.18:0/1248594950 shutdown_connections 2024-04-12T21:25:43.062 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:25:43.066+0000 7f4902800700 1 -- 172.21.15.18:0/1248594950 wait complete. 2024-04-12T21:25:44.029 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:25:43 smithi186 ceph-mon[86379]: from='client.35392 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:25:44.030 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:25:43 smithi186 ceph-mon[86379]: from='client.? 172.21.15.18:0/1248594950' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-12T21:25:44.100 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:25:43 smithi018 ceph-mon[112590]: from='client.35392 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:25:44.101 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:25:43 smithi018 ceph-mon[112590]: from='client.? 172.21.15.18:0/1248594950' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-12T21:25:45.029 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:25:44 smithi186 ceph-mon[86379]: pgmap v779: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:25:45.100 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:25:44 smithi018 ceph-mon[112590]: pgmap v779: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:25:47.029 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:25:46 smithi186 ceph-mon[86379]: pgmap v780: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:25:47.100 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:25:46 smithi018 ceph-mon[112590]: pgmap v780: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:25:49.101 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:25:48 smithi018 ceph-mon[112590]: pgmap v781: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:25:49.279 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:25:48 smithi186 ceph-mon[86379]: pgmap v781: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:25:51.100 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:25:50 smithi018 ceph-mon[112590]: pgmap v782: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:25:51.279 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:25:50 smithi186 ceph-mon[86379]: pgmap v782: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:25:53.100 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:25:52 smithi018 ceph-mon[112590]: pgmap v783: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:25:53.279 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:25:52 smithi186 ceph-mon[86379]: pgmap v783: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:25:54.100 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:25:53 smithi018 ceph-mon[112590]: pgmap v784: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:25:54.279 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:25:53 smithi186 ceph-mon[86379]: pgmap v784: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:25:55.279 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:25:54 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T21:25:55.350 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:25:54 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T21:25:56.279 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:25:55 smithi186 ceph-mon[86379]: pgmap v785: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:25:56.350 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:25:55 smithi018 ceph-mon[112590]: pgmap v785: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:25:58.779 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:25:58 smithi186 ceph-mon[86379]: pgmap v786: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:25:58.850 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:25:58 smithi018 ceph-mon[112590]: pgmap v786: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:26:00.850 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:26:00 smithi018 ceph-mon[112590]: pgmap v787: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:26:01.029 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:26:00 smithi186 ceph-mon[86379]: pgmap v787: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:26:02.850 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:26:02 smithi018 ceph-mon[112590]: pgmap v788: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:26:03.030 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:26:02 smithi186 ceph-mon[86379]: pgmap v788: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:26:04.850 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:26:04 smithi018 ceph-mon[112590]: pgmap v789: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:26:05.029 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:26:04 smithi186 ceph-mon[86379]: pgmap v789: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:26:06.850 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:26:06 smithi018 ceph-mon[112590]: pgmap v790: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:26:07.029 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:26:06 smithi186 ceph-mon[86379]: pgmap v790: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:26:08.851 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:26:08 smithi018 ceph-mon[112590]: pgmap v791: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:26:09.029 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:26:08 smithi186 ceph-mon[86379]: pgmap v791: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:26:10.833 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:26:10 smithi186 ceph-mon[86379]: pgmap v792: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:26:10.833 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:26:10 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T21:26:10.850 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:26:10 smithi018 ceph-mon[112590]: pgmap v792: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:26:10.850 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:26:10 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T21:26:12.850 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:26:12 smithi018 ceph-mon[112590]: pgmap v793: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:26:13.030 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:26:12 smithi186 ceph-mon[86379]: pgmap v793: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:26:13.236 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:13.239+0000 7f7801800700 1 -- 172.21.15.18:0/870887525 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f77fc1026c0 msgr2=0x7f77fc0fbd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:26:13.237 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:13.239+0000 7f7801800700 1 --2- 172.21.15.18:0/870887525 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f77fc1026c0 0x7f77fc0fbd30 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7f77e8004940 tx=0x7f77e8009e30 comp rx=0 tx=0).stop 2024-04-12T21:26:13.237 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:13.239+0000 7f7801800700 1 -- 172.21.15.18:0/870887525 shutdown_connections 2024-04-12T21:26:13.237 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:13.239+0000 7f7801800700 1 --2- 172.21.15.18:0/870887525 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f77fc1026c0 0x7f77fc0fbd30 unknown :-1 s=CLOSED pgs=668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:26:13.237 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:13.239+0000 7f7801800700 1 --2- 172.21.15.18:0/870887525 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f77fc101d20 0x7f77fc1020f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:26:13.237 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:13.239+0000 7f7801800700 1 -- 172.21.15.18:0/870887525 >> 172.21.15.18:0/870887525 conn(0x7f77fc0f7870 msgr2=0x7f77fc0f9c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:26:13.237 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:13.239+0000 7f7801800700 1 -- 172.21.15.18:0/870887525 shutdown_connections 2024-04-12T21:26:13.237 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:13.239+0000 7f7801800700 1 -- 172.21.15.18:0/870887525 wait complete. 2024-04-12T21:26:13.238 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:13.240+0000 7f7801800700 1 Processor -- start 2024-04-12T21:26:13.238 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:13.240+0000 7f7801800700 1 -- start start 2024-04-12T21:26:13.238 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:13.240+0000 7f7801800700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f77fc101d20 0x7f77fc115bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:26:13.238 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:13.240+0000 7f7801800700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f77fc1026c0 0x7f77fc116100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:26:13.238 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:13.241+0000 7f7801800700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77fc1167e0 con 0x7f77fc1026c0 2024-04-12T21:26:13.239 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:13.241+0000 7f7801800700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77fc10fd30 con 0x7f77fc101d20 2024-04-12T21:26:13.239 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:13.241+0000 7f77fa000700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f77fc1026c0 0x7f77fc116100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:26:13.239 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:13.241+0000 7f77fa000700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f77fc1026c0 0x7f77fc116100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:48154/0 (socket says 172.21.15.18:48154) 2024-04-12T21:26:13.239 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:13.241+0000 7f77fa000700 1 -- 172.21.15.18:0/740421327 learned_addr learned my addr 172.21.15.18:0/740421327 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:26:13.240 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:13.241+0000 7f77faa00700 1 --2- 172.21.15.18:0/740421327 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f77fc101d20 0x7f77fc115bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:26:13.240 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:13.241+0000 7f77fa000700 1 -- 172.21.15.18:0/740421327 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f77fc101d20 msgr2=0x7f77fc115bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:26:13.240 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:13.241+0000 7f77fa000700 1 --2- 172.21.15.18:0/740421327 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f77fc101d20 0x7f77fc115bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:26:13.240 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:13.241+0000 7f77fa000700 1 -- 172.21.15.18:0/740421327 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77e8005040 con 0x7f77fc1026c0 2024-04-12T21:26:13.240 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:13.242+0000 7f77fa000700 1 --2- 172.21.15.18:0/740421327 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f77fc1026c0 0x7f77fc116100 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f77e8009100 tx=0x7f77e8009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:26:13.240 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:13.242+0000 7f77f2a00700 1 -- 172.21.15.18:0/740421327 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f77e800e040 con 0x7f77fc1026c0 2024-04-12T21:26:13.240 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:13.242+0000 7f77f2a00700 1 -- 172.21.15.18:0/740421327 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f77e8013420 con 0x7f77fc1026c0 2024-04-12T21:26:13.240 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:13.242+0000 7f7801800700 1 -- 172.21.15.18:0/740421327 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f77fc10ff50 con 0x7f77fc1026c0 2024-04-12T21:26:13.240 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:13.242+0000 7f77f2a00700 1 -- 172.21.15.18:0/740421327 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f77e801c610 con 0x7f77fc1026c0 2024-04-12T21:26:13.240 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:13.242+0000 7f7801800700 1 -- 172.21.15.18:0/740421327 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f77fc110510 con 0x7f77fc1026c0 2024-04-12T21:26:13.241 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:13.244+0000 7f7801800700 1 -- 172.21.15.18:0/740421327 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f77fc0fd470 con 0x7f77fc1026c0 2024-04-12T21:26:13.242 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:13.245+0000 7f77f2a00700 1 -- 172.21.15.18:0/740421327 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7f77e8013590 con 0x7f77fc1026c0 2024-04-12T21:26:13.243 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:13.246+0000 7f77f2a00700 1 --2- 172.21.15.18:0/740421327 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f77e006ec00 0x7f77e00710b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:26:13.244 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:13.246+0000 7f77f2a00700 1 -- 172.21.15.18:0/740421327 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 6757+0+0 (secure 0 0 0) 0x7f77e808faf0 con 0x7f77fc1026c0 2024-04-12T21:26:13.244 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:13.247+0000 7f77faa00700 1 --2- 172.21.15.18:0/740421327 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f77e006ec00 0x7f77e00710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:26:13.244 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:13.247+0000 7f77faa00700 1 --2- 172.21.15.18:0/740421327 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f77e006ec00 0x7f77e00710b0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f77ec006fd0 tx=0x7f77ec008040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:26:13.249 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:13.252+0000 7f77f2a00700 1 -- 172.21.15.18:0/740421327 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f77e805c3e0 con 0x7f77fc1026c0 2024-04-12T21:26:13.502 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:13.504+0000 7f7801800700 1 -- 172.21.15.18:0/740421327 --> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f77fc110840 con 0x7f77e006ec00 2024-04-12T21:26:13.505 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:13.508+0000 7f77f2a00700 1 -- 172.21.15.18:0/740421327 <== mgr.34102 v2:172.21.15.18:6800/3812891919 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f77fc110840 con 0x7f77e006ec00 2024-04-12T21:26:13.508 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:13.511+0000 7f7801800700 1 -- 172.21.15.18:0/740421327 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f77e006ec00 msgr2=0x7f77e00710b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:26:13.508 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:13.511+0000 7f7801800700 1 --2- 172.21.15.18:0/740421327 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f77e006ec00 0x7f77e00710b0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f77ec006fd0 tx=0x7f77ec008040 comp rx=0 tx=0).stop 2024-04-12T21:26:13.508 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:13.511+0000 7f7801800700 1 -- 172.21.15.18:0/740421327 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f77fc1026c0 msgr2=0x7f77fc116100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:26:13.508 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:13.511+0000 7f7801800700 1 --2- 172.21.15.18:0/740421327 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f77fc1026c0 0x7f77fc116100 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f77e8009100 tx=0x7f77e8009e30 comp rx=0 tx=0).stop 2024-04-12T21:26:13.509 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:13.511+0000 7f7801800700 1 -- 172.21.15.18:0/740421327 shutdown_connections 2024-04-12T21:26:13.509 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:13.511+0000 7f7801800700 1 --2- 172.21.15.18:0/740421327 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f77e006ec00 0x7f77e00710b0 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:26:13.509 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:13.511+0000 7f7801800700 1 --2- 172.21.15.18:0/740421327 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f77fc101d20 0x7f77fc115bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:26:13.509 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:13.511+0000 7f7801800700 1 --2- 172.21.15.18:0/740421327 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f77fc1026c0 0x7f77fc116100 unknown :-1 s=CLOSED pgs=669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:26:13.509 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:13.511+0000 7f7801800700 1 -- 172.21.15.18:0/740421327 >> 172.21.15.18:0/740421327 conn(0x7f77fc0f7870 msgr2=0x7f77fc0f8ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:26:13.509 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:13.511+0000 7f7801800700 1 -- 172.21.15.18:0/740421327 shutdown_connections 2024-04-12T21:26:13.509 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:13.512+0000 7f7801800700 1 -- 172.21.15.18:0/740421327 wait complete. 2024-04-12T21:26:13.525 INFO:teuthology.orchestra.run.smithi018.stdout:true 2024-04-12T21:26:13.665 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:13.667+0000 7f125bc00700 1 -- 172.21.15.18:0/1804895627 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1254105020 msgr2=0x7f12541053f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:26:13.665 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:13.667+0000 7f125bc00700 1 --2- 172.21.15.18:0/1804895627 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1254105020 0x7f12541053f0 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f1248004940 tx=0x7f1248009e30 comp rx=0 tx=0).stop 2024-04-12T21:26:13.665 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:13.667+0000 7f125bc00700 1 -- 172.21.15.18:0/1804895627 shutdown_connections 2024-04-12T21:26:13.665 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:13.667+0000 7f125bc00700 1 --2- 172.21.15.18:0/1804895627 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f12541059c0 0x7f12540ff080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:26:13.665 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:13.667+0000 7f125bc00700 1 --2- 172.21.15.18:0/1804895627 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1254105020 0x7f12541053f0 unknown :-1 s=CLOSED pgs=670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:26:13.665 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:13.667+0000 7f125bc00700 1 -- 172.21.15.18:0/1804895627 >> 172.21.15.18:0/1804895627 conn(0x7f12540faba0 msgr2=0x7f12540fcfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:26:13.665 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:13.668+0000 7f125bc00700 1 -- 172.21.15.18:0/1804895627 shutdown_connections 2024-04-12T21:26:13.665 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:13.668+0000 7f125bc00700 1 -- 172.21.15.18:0/1804895627 wait complete. 2024-04-12T21:26:13.667 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:13.668+0000 7f125bc00700 1 Processor -- start 2024-04-12T21:26:13.667 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:13.669+0000 7f125bc00700 1 -- start start 2024-04-12T21:26:13.667 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:13.669+0000 7f125bc00700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1254105020 0x7f1254115bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:26:13.667 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:13.669+0000 7f125bc00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f12541059c0 0x7f1254116100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:26:13.667 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:13.669+0000 7f125bc00700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1254116750 con 0x7f12541059c0 2024-04-12T21:26:13.667 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:13.669+0000 7f125bc00700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f125410fd30 con 0x7f1254105020 2024-04-12T21:26:13.667 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:13.669+0000 7f1253e00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f12541059c0 0x7f1254116100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:26:13.667 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:13.669+0000 7f1253e00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f12541059c0 0x7f1254116100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:48168/0 (socket says 172.21.15.18:48168) 2024-04-12T21:26:13.667 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:13.669+0000 7f1253e00700 1 -- 172.21.15.18:0/931281371 learned_addr learned my addr 172.21.15.18:0/931281371 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:26:13.667 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:13.670+0000 7f1259200700 1 --2- 172.21.15.18:0/931281371 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1254105020 0x7f1254115bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:26:13.667 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:13.670+0000 7f1253e00700 1 -- 172.21.15.18:0/931281371 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1254105020 msgr2=0x7f1254115bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:26:13.668 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:13.670+0000 7f1253e00700 1 --2- 172.21.15.18:0/931281371 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1254105020 0x7f1254115bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:26:13.668 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:13.670+0000 7f1253e00700 1 -- 172.21.15.18:0/931281371 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1248005040 con 0x7f12541059c0 2024-04-12T21:26:13.668 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:13.670+0000 7f1253e00700 1 --2- 172.21.15.18:0/931281371 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f12541059c0 0x7f1254116100 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f1240002a80 tx=0x7f1240004630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:26:13.668 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:13.670+0000 7f1250c00700 1 -- 172.21.15.18:0/931281371 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1240007cc0 con 0x7f12541059c0 2024-04-12T21:26:13.669 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:13.670+0000 7f125bc00700 1 -- 172.21.15.18:0/931281371 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1254110010 con 0x7f12541059c0 2024-04-12T21:26:13.669 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:13.670+0000 7f1250c00700 1 -- 172.21.15.18:0/931281371 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f124000f040 con 0x7f12541059c0 2024-04-12T21:26:13.669 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:13.670+0000 7f1250c00700 1 -- 172.21.15.18:0/931281371 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f124000daf0 con 0x7f12541059c0 2024-04-12T21:26:13.669 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:13.671+0000 7f125bc00700 1 -- 172.21.15.18:0/931281371 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f12541105d0 con 0x7f12541059c0 2024-04-12T21:26:13.669 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:13.672+0000 7f125bc00700 1 -- 172.21.15.18:0/931281371 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1254100770 con 0x7f12541059c0 2024-04-12T21:26:13.670 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:13.673+0000 7f1250c00700 1 -- 172.21.15.18:0/931281371 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7f124000dc90 con 0x7f12541059c0 2024-04-12T21:26:13.672 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:13.675+0000 7f1250c00700 1 --2- 172.21.15.18:0/931281371 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f123c06ec00 0x7f123c0710b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:26:13.672 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:13.675+0000 7f1250c00700 1 -- 172.21.15.18:0/931281371 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 6757+0+0 (secure 0 0 0) 0x7f124008e070 con 0x7f12541059c0 2024-04-12T21:26:13.672 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:13.675+0000 7f1259200700 1 --2- 172.21.15.18:0/931281371 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f123c06ec00 0x7f123c0710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:26:13.673 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:13.676+0000 7f1259200700 1 --2- 172.21.15.18:0/931281371 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f123c06ec00 0x7f123c0710b0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f124800b020 tx=0x7f1248009e30 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:26:13.679 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:13.680+0000 7f1250c00700 1 -- 172.21.15.18:0/931281371 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f124005a820 con 0x7f12541059c0 2024-04-12T21:26:13.932 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:13.934+0000 7f125bc00700 1 -- 172.21.15.18:0/931281371 --> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f125410c140 con 0x7f123c06ec00 2024-04-12T21:26:13.934 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:13.936+0000 7f1250c00700 1 -- 172.21.15.18:0/931281371 <== mgr.34102 v2:172.21.15.18:6800/3812891919 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f125410c140 con 0x7f123c06ec00 2024-04-12T21:26:13.937 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:13.939+0000 7f125bc00700 1 -- 172.21.15.18:0/931281371 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f123c06ec00 msgr2=0x7f123c0710b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:26:13.937 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:13.939+0000 7f125bc00700 1 --2- 172.21.15.18:0/931281371 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f123c06ec00 0x7f123c0710b0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f124800b020 tx=0x7f1248009e30 comp rx=0 tx=0).stop 2024-04-12T21:26:13.937 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:13.939+0000 7f125bc00700 1 -- 172.21.15.18:0/931281371 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f12541059c0 msgr2=0x7f1254116100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:26:13.937 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:13.939+0000 7f125bc00700 1 --2- 172.21.15.18:0/931281371 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f12541059c0 0x7f1254116100 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f1240002a80 tx=0x7f1240004630 comp rx=0 tx=0).stop 2024-04-12T21:26:13.937 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:13.939+0000 7f125bc00700 1 -- 172.21.15.18:0/931281371 shutdown_connections 2024-04-12T21:26:13.937 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:13.939+0000 7f125bc00700 1 --2- 172.21.15.18:0/931281371 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f123c06ec00 0x7f123c0710b0 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:26:13.937 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:13.939+0000 7f125bc00700 1 --2- 172.21.15.18:0/931281371 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1254105020 0x7f1254115bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:26:13.937 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:13.940+0000 7f125bc00700 1 --2- 172.21.15.18:0/931281371 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f12541059c0 0x7f1254116100 unknown :-1 s=CLOSED pgs=671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:26:13.937 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:13.940+0000 7f125bc00700 1 -- 172.21.15.18:0/931281371 >> 172.21.15.18:0/931281371 conn(0x7f12540faba0 msgr2=0x7f12540fbba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:26:13.937 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:13.940+0000 7f125bc00700 1 -- 172.21.15.18:0/931281371 shutdown_connections 2024-04-12T21:26:13.937 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:13.940+0000 7f125bc00700 1 -- 172.21.15.18:0/931281371 wait complete. 2024-04-12T21:26:14.094 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:14.097+0000 7f3415a00700 1 -- 172.21.15.18:0/4125469072 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f34101037a0 msgr2=0x7f34100fce10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:26:14.094 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:14.097+0000 7f3415a00700 1 --2- 172.21.15.18:0/4125469072 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f34101037a0 0x7f34100fce10 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f33fc004940 tx=0x7f33fc009e30 comp rx=0 tx=0).stop 2024-04-12T21:26:14.095 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:14.097+0000 7f3415a00700 1 -- 172.21.15.18:0/4125469072 shutdown_connections 2024-04-12T21:26:14.095 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:14.097+0000 7f3415a00700 1 --2- 172.21.15.18:0/4125469072 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f34101037a0 0x7f34100fce10 unknown :-1 s=CLOSED pgs=672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:26:14.095 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:14.097+0000 7f3415a00700 1 --2- 172.21.15.18:0/4125469072 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3410102e00 0x7f34101031d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:26:14.095 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:14.097+0000 7f3415a00700 1 -- 172.21.15.18:0/4125469072 >> 172.21.15.18:0/4125469072 conn(0x7f34100f89b0 msgr2=0x7f34100fadc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:26:14.095 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:14.097+0000 7f3415a00700 1 -- 172.21.15.18:0/4125469072 shutdown_connections 2024-04-12T21:26:14.095 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:14.097+0000 7f3415a00700 1 -- 172.21.15.18:0/4125469072 wait complete. 2024-04-12T21:26:14.096 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:14.098+0000 7f3415a00700 1 Processor -- start 2024-04-12T21:26:14.096 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:14.098+0000 7f3415a00700 1 -- start start 2024-04-12T21:26:14.096 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:14.098+0000 7f3415a00700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3410102e00 0x7f34100702c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:26:14.096 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:14.098+0000 7f3415a00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f34101037a0 0x7f3410070800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:26:14.096 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:14.098+0000 7f3415a00700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3410070e20 con 0x7f34101037a0 2024-04-12T21:26:14.096 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:14.098+0000 7f3415a00700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3410070f60 con 0x7f3410102e00 2024-04-12T21:26:14.096 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:14.099+0000 7f340e000700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f34101037a0 0x7f3410070800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:26:14.096 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:14.099+0000 7f340e000700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f34101037a0 0x7f3410070800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:48190/0 (socket says 172.21.15.18:48190) 2024-04-12T21:26:14.097 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:14.099+0000 7f340e000700 1 -- 172.21.15.18:0/2190701539 learned_addr learned my addr 172.21.15.18:0/2190701539 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:26:14.098 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:14.099+0000 7f340ea00700 1 --2- 172.21.15.18:0/2190701539 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3410102e00 0x7f34100702c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:26:14.098 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:14.099+0000 7f340e000700 1 -- 172.21.15.18:0/2190701539 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3410102e00 msgr2=0x7f34100702c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:26:14.098 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:14.099+0000 7f340e000700 1 --2- 172.21.15.18:0/2190701539 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3410102e00 0x7f34100702c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:26:14.098 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:14.099+0000 7f340e000700 1 -- 172.21.15.18:0/2190701539 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33fc005040 con 0x7f34101037a0 2024-04-12T21:26:14.098 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:14.100+0000 7f340e000700 1 --2- 172.21.15.18:0/2190701539 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f34101037a0 0x7f3410070800 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f33fc009100 tx=0x7f33fc009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:26:14.098 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:14.100+0000 7f3406a00700 1 -- 172.21.15.18:0/2190701539 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f33fc00e040 con 0x7f34101037a0 2024-04-12T21:26:14.098 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:14.100+0000 7f3406a00700 1 -- 172.21.15.18:0/2190701539 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f33fc013420 con 0x7f34101037a0 2024-04-12T21:26:14.098 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:14.100+0000 7f3415a00700 1 -- 172.21.15.18:0/2190701539 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3410119ee0 con 0x7f34101037a0 2024-04-12T21:26:14.098 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:14.100+0000 7f3406a00700 1 -- 172.21.15.18:0/2190701539 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f33fc01c630 con 0x7f34101037a0 2024-04-12T21:26:14.098 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:14.100+0000 7f3415a00700 1 -- 172.21.15.18:0/2190701539 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f341011a3c0 con 0x7f34101037a0 2024-04-12T21:26:14.099 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:14.101+0000 7f3415a00700 1 -- 172.21.15.18:0/2190701539 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3410100110 con 0x7f34101037a0 2024-04-12T21:26:14.100 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:14.103+0000 7f3406a00700 1 -- 172.21.15.18:0/2190701539 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7f33fc013590 con 0x7f34101037a0 2024-04-12T21:26:14.101 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:14.104+0000 7f3406a00700 1 --2- 172.21.15.18:0/2190701539 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f33f406ec10 0x7f33f40710c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:26:14.101 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:14.104+0000 7f3406a00700 1 -- 172.21.15.18:0/2190701539 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 6757+0+0 (secure 0 0 0) 0x7f33fc08fba0 con 0x7f34101037a0 2024-04-12T21:26:14.101 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:14.104+0000 7f340ea00700 1 --2- 172.21.15.18:0/2190701539 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f33f406ec10 0x7f33f40710c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:26:14.101 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:14.105+0000 7f340ea00700 1 --2- 172.21.15.18:0/2190701539 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f33f406ec10 0x7f33f40710c0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f3400006fd0 tx=0x7f3400008040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:26:14.106 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:14.109+0000 7f3406a00700 1 -- 172.21.15.18:0/2190701539 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f33fc0588a0 con 0x7f34101037a0 2024-04-12T21:26:14.338 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:14.340+0000 7f3415a00700 1 -- 172.21.15.18:0/2190701539 --> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f3410071a20 con 0x7f33f406ec10 2024-04-12T21:26:14.348 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:14.351+0000 7f3406a00700 1 -- 172.21.15.18:0/2190701539 <== mgr.34102 v2:172.21.15.18:6800/3812891919 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7f3410071a20 con 0x7f33f406ec10 2024-04-12T21:26:14.349 INFO:teuthology.orchestra.run.smithi018.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-12T21:26:14.349 INFO:teuthology.orchestra.run.smithi018.stdout:alertmanager.smithi018 smithi018 *:9093,9094 running (26m) 3m ago 41m 15.5M - 0.25.0 c8568f914cd2 c332090068e8 2024-04-12T21:26:14.349 INFO:teuthology.orchestra.run.smithi018.stdout:crash.smithi018 smithi018 running (23m) 3m ago 41m 6665k - 19.0.0-2879-g324baf8d 8717720dc5e5 b7f80d1f7982 2024-04-12T21:26:14.349 INFO:teuthology.orchestra.run.smithi018.stdout:crash.smithi186 smithi186 running (23m) 3m ago 40m 6660k - 19.0.0-2879-g324baf8d 8717720dc5e5 a4f55c81b623 2024-04-12T21:26:14.349 INFO:teuthology.orchestra.run.smithi018.stdout:grafana.smithi018 smithi018 *:3000 running (26m) 3m ago 40m 74.8M - 9.4.12 2bacad6d85d8 b08e3bf176dd 2024-04-12T21:26:14.349 INFO:teuthology.orchestra.run.smithi018.stdout:mds.cephfs.smithi018.acqbsc smithi018 running (19m) 3m ago 34m 16.7M - 19.0.0-2879-g324baf8d 8717720dc5e5 021f99ab24ae 2024-04-12T21:26:14.349 INFO:teuthology.orchestra.run.smithi018.stdout:mds.cephfs.smithi018.szjluo smithi018 running (18m) 3m ago 34m 16.5M - 19.0.0-2879-g324baf8d 8717720dc5e5 0003117479d4 2024-04-12T21:26:14.349 INFO:teuthology.orchestra.run.smithi018.stdout:mds.cephfs.smithi186.jdllyn smithi186 running (3m) 3m ago 34m 11.0M - 19.0.0-2879-g324baf8d 8717720dc5e5 21b1f46eef23 2024-04-12T21:26:14.349 INFO:teuthology.orchestra.run.smithi018.stdout:mds.cephfs.smithi186.wjnafs smithi186 running (34m) 3m ago 34m 19.5M - 17.2.7-827-gd3b5060a 21de9b7369ad 74b115d6d6fd 2024-04-12T21:26:14.350 INFO:teuthology.orchestra.run.smithi018.stdout:mgr.smithi018.xjzwga smithi018 *:8443,9283,8765 running (29m) 3m ago 43m 512M - 19.0.0-2879-g324baf8d 8717720dc5e5 56088305eab7 2024-04-12T21:26:14.350 INFO:teuthology.orchestra.run.smithi018.stdout:mgr.smithi186.kqcfxk smithi186 *:8443,9283,8765 running (28m) 3m ago 40m 459M - 19.0.0-2879-g324baf8d 8717720dc5e5 09cd766e5e0d 2024-04-12T21:26:14.350 INFO:teuthology.orchestra.run.smithi018.stdout:mon.smithi018 smithi018 running (24m) 3m ago 43m 73.1M 2048M 19.0.0-2879-g324baf8d 8717720dc5e5 df3e201dfed6 2024-04-12T21:26:14.350 INFO:teuthology.orchestra.run.smithi018.stdout:mon.smithi186 smithi186 running (24m) 3m ago 39m 47.2M 2048M 19.0.0-2879-g324baf8d 8717720dc5e5 c77498da9b2b 2024-04-12T21:26:14.350 INFO:teuthology.orchestra.run.smithi018.stdout:node-exporter.smithi018 smithi018 *:9100 running (27m) 3m ago 41m 8108k - 1.5.0 0da6a335fe13 326c3d9c6847 2024-04-12T21:26:14.350 INFO:teuthology.orchestra.run.smithi018.stdout:node-exporter.smithi186 smithi186 *:9100 running (27m) 3m ago 40m 8757k - 1.5.0 0da6a335fe13 ec2112baeecf 2024-04-12T21:26:14.350 INFO:teuthology.orchestra.run.smithi018.stdout:osd.0 smithi018 running (22m) 3m ago 38m 123M 1559M 19.0.0-2879-g324baf8d 8717720dc5e5 60b1273ca047 2024-04-12T21:26:14.350 INFO:teuthology.orchestra.run.smithi018.stdout:osd.1 smithi018 running (21m) 3m ago 38m 95.1M 1559M 19.0.0-2879-g324baf8d 8717720dc5e5 170ce1e5fb8d 2024-04-12T21:26:14.350 INFO:teuthology.orchestra.run.smithi018.stdout:osd.2 smithi018 running (21m) 3m ago 37m 94.3M 1559M 19.0.0-2879-g324baf8d 8717720dc5e5 c2cbc6aa2839 2024-04-12T21:26:14.350 INFO:teuthology.orchestra.run.smithi018.stdout:osd.3 smithi186 running (20m) 3m ago 37m 101M 2581M 19.0.0-2879-g324baf8d 8717720dc5e5 0138e8262e4b 2024-04-12T21:26:14.350 INFO:teuthology.orchestra.run.smithi018.stdout:osd.4 smithi186 running (20m) 3m ago 36m 91.0M 2581M 19.0.0-2879-g324baf8d 8717720dc5e5 b20cb63b25c9 2024-04-12T21:26:14.350 INFO:teuthology.orchestra.run.smithi018.stdout:osd.5 smithi186 running (19m) 3m ago 36m 92.0M 2581M 19.0.0-2879-g324baf8d 8717720dc5e5 2513bea2610e 2024-04-12T21:26:14.350 INFO:teuthology.orchestra.run.smithi018.stdout:prometheus.smithi018 smithi018 *:9095 running (27m) 3m ago 40m 67.9M - 2.43.0 a07b618ecd1d f58841127e21 2024-04-12T21:26:14.352 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:14.354+0000 7f3415a00700 1 -- 172.21.15.18:0/2190701539 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f33f406ec10 msgr2=0x7f33f40710c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:26:14.352 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:14.354+0000 7f3415a00700 1 --2- 172.21.15.18:0/2190701539 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f33f406ec10 0x7f33f40710c0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f3400006fd0 tx=0x7f3400008040 comp rx=0 tx=0).stop 2024-04-12T21:26:14.352 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:14.354+0000 7f3415a00700 1 -- 172.21.15.18:0/2190701539 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f34101037a0 msgr2=0x7f3410070800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:26:14.352 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:14.354+0000 7f3415a00700 1 --2- 172.21.15.18:0/2190701539 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f34101037a0 0x7f3410070800 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f33fc009100 tx=0x7f33fc009e30 comp rx=0 tx=0).stop 2024-04-12T21:26:14.352 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:14.354+0000 7f3415a00700 1 -- 172.21.15.18:0/2190701539 shutdown_connections 2024-04-12T21:26:14.352 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:14.354+0000 7f3415a00700 1 --2- 172.21.15.18:0/2190701539 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f33f406ec10 0x7f33f40710c0 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:26:14.352 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:14.354+0000 7f3415a00700 1 --2- 172.21.15.18:0/2190701539 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3410102e00 0x7f34100702c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:26:14.352 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:14.354+0000 7f3415a00700 1 --2- 172.21.15.18:0/2190701539 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f34101037a0 0x7f3410070800 unknown :-1 s=CLOSED pgs=673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:26:14.352 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:14.354+0000 7f3415a00700 1 -- 172.21.15.18:0/2190701539 >> 172.21.15.18:0/2190701539 conn(0x7f34100f89b0 msgr2=0x7f34100fa7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:26:14.352 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:14.354+0000 7f3415a00700 1 -- 172.21.15.18:0/2190701539 shutdown_connections 2024-04-12T21:26:14.352 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:14.355+0000 7f3415a00700 1 -- 172.21.15.18:0/2190701539 wait complete. 2024-04-12T21:26:14.504 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:14.506+0000 7f00a6400700 1 -- 172.21.15.18:0/883473772 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f00a0102c40 msgr2=0x7f00a0103010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:26:14.504 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:14.506+0000 7f00a6400700 1 --2- 172.21.15.18:0/883473772 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f00a0102c40 0x7f00a0103010 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f0094004940 tx=0x7f0094009e30 comp rx=0 tx=0).stop 2024-04-12T21:26:14.504 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:14.506+0000 7f00a6400700 1 -- 172.21.15.18:0/883473772 shutdown_connections 2024-04-12T21:26:14.504 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:14.506+0000 7f00a6400700 1 --2- 172.21.15.18:0/883473772 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f00a01035e0 0x7f00a00fcc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:26:14.504 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:14.506+0000 7f00a6400700 1 --2- 172.21.15.18:0/883473772 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f00a0102c40 0x7f00a0103010 unknown :-1 s=CLOSED pgs=674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:26:14.504 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:14.506+0000 7f00a6400700 1 -- 172.21.15.18:0/883473772 >> 172.21.15.18:0/883473772 conn(0x7f00a00f87b0 msgr2=0x7f00a00fabc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:26:14.504 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:14.506+0000 7f00a6400700 1 -- 172.21.15.18:0/883473772 shutdown_connections 2024-04-12T21:26:14.504 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:14.506+0000 7f00a6400700 1 -- 172.21.15.18:0/883473772 wait complete. 2024-04-12T21:26:14.506 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:14.507+0000 7f00a6400700 1 Processor -- start 2024-04-12T21:26:14.506 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:14.507+0000 7f00a6400700 1 -- start start 2024-04-12T21:26:14.506 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:14.508+0000 7f00a6400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f00a0102c40 0x7f00a0115bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:26:14.506 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:14.508+0000 7f00a6400700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f00a01035e0 0x7f00a01160f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:26:14.506 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:14.508+0000 7f00a6400700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00a0116740 con 0x7f00a0102c40 2024-04-12T21:26:14.506 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:14.508+0000 7f00a6400700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00a010fb50 con 0x7f00a01035e0 2024-04-12T21:26:14.506 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:14.508+0000 7f00a5000700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f00a0102c40 0x7f00a0115bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:26:14.506 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:14.508+0000 7f00a5000700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f00a0102c40 0x7f00a0115bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:48206/0 (socket says 172.21.15.18:48206) 2024-04-12T21:26:14.506 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:14.508+0000 7f00a5000700 1 -- 172.21.15.18:0/366602254 learned_addr learned my addr 172.21.15.18:0/366602254 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:26:14.506 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:14.508+0000 7f00a5000700 1 -- 172.21.15.18:0/366602254 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f00a01035e0 msgr2=0x7f00a01160f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:26:14.506 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:14.508+0000 7f00a5000700 1 --2- 172.21.15.18:0/366602254 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f00a01035e0 0x7f00a01160f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:26:14.506 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:14.508+0000 7f00a5000700 1 -- 172.21.15.18:0/366602254 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0094005040 con 0x7f00a0102c40 2024-04-12T21:26:14.507 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:14.509+0000 7f00a5000700 1 --2- 172.21.15.18:0/366602254 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f00a0102c40 0x7f00a0115bb0 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f0094009150 tx=0x7f0094009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:26:14.507 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:14.509+0000 7f009cc00700 1 -- 172.21.15.18:0/366602254 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f009400e040 con 0x7f00a0102c40 2024-04-12T21:26:14.507 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:14.509+0000 7f00a6400700 1 -- 172.21.15.18:0/366602254 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f00a010fdd0 con 0x7f00a0102c40 2024-04-12T21:26:14.507 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:14.509+0000 7f009cc00700 1 -- 172.21.15.18:0/366602254 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f0094013420 con 0x7f00a0102c40 2024-04-12T21:26:14.507 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:14.509+0000 7f009cc00700 1 -- 172.21.15.18:0/366602254 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f009401c630 con 0x7f00a0102c40 2024-04-12T21:26:14.508 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:14.509+0000 7f00a6400700 1 -- 172.21.15.18:0/366602254 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f00a0110390 con 0x7f00a0102c40 2024-04-12T21:26:14.508 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:14.510+0000 7f00a6400700 1 -- 172.21.15.18:0/366602254 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f00a00fe390 con 0x7f00a0102c40 2024-04-12T21:26:14.508 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:14.511+0000 7f009cc00700 1 -- 172.21.15.18:0/366602254 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7f0094013590 con 0x7f00a0102c40 2024-04-12T21:26:14.509 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:14.512+0000 7f009cc00700 1 --2- 172.21.15.18:0/366602254 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f008806ec10 0x7f00880710c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:26:14.509 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:14.512+0000 7f009cc00700 1 -- 172.21.15.18:0/366602254 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 6757+0+0 (secure 0 0 0) 0x7f009408fce0 con 0x7f00a0102c40 2024-04-12T21:26:14.510 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:14.512+0000 7f009fe00700 1 --2- 172.21.15.18:0/366602254 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f008806ec10 0x7f00880710c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:26:14.510 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:14.513+0000 7f009fe00700 1 --2- 172.21.15.18:0/366602254 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f008806ec10 0x7f00880710c0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f00a0111450 tx=0x7f008c007d40 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:26:14.514 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:14.517+0000 7f009cc00700 1 -- 172.21.15.18:0/366602254 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f0094058960 con 0x7f00a0102c40 2024-04-12T21:26:14.600 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:26:14 smithi018 ceph-mon[112590]: pgmap v794: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:26:14.601 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:26:14 smithi018 ceph-mon[112590]: from='client.35400 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:26:14.829 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:14.831+0000 7f00a6400700 1 -- 172.21.15.18:0/366602254 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f00a004bed0 con 0x7f00a0102c40 2024-04-12T21:26:14.830 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:14.832+0000 7f009cc00700 1 -- 172.21.15.18:0/366602254 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+837 (secure 0 0 0) 0x7f009405c020 con 0x7f00a0102c40 2024-04-12T21:26:14.831 INFO:teuthology.orchestra.run.smithi018.stdout:{ 2024-04-12T21:26:14.831 INFO:teuthology.orchestra.run.smithi018.stdout: "mon": { 2024-04-12T21:26:14.831 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 19.0.0-2879-g324baf8d (324baf8ddb91dea46cac50cb535c306115791355) squid (dev)": 2 2024-04-12T21:26:14.831 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-12T21:26:14.831 INFO:teuthology.orchestra.run.smithi018.stdout: "mgr": { 2024-04-12T21:26:14.831 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 19.0.0-2879-g324baf8d (324baf8ddb91dea46cac50cb535c306115791355) squid (dev)": 2 2024-04-12T21:26:14.831 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-12T21:26:14.831 INFO:teuthology.orchestra.run.smithi018.stdout: "osd": { 2024-04-12T21:26:14.831 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 19.0.0-2879-g324baf8d (324baf8ddb91dea46cac50cb535c306115791355) squid (dev)": 6 2024-04-12T21:26:14.831 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-12T21:26:14.831 INFO:teuthology.orchestra.run.smithi018.stdout: "mds": { 2024-04-12T21:26:14.831 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 17.2.7-827-gd3b5060a (d3b5060a17449edfae9385cdf792afec109fca5d) quincy (stable)": 1, 2024-04-12T21:26:14.832 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 19.0.0-2879-g324baf8d (324baf8ddb91dea46cac50cb535c306115791355) squid (dev)": 3 2024-04-12T21:26:14.832 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-12T21:26:14.832 INFO:teuthology.orchestra.run.smithi018.stdout: "overall": { 2024-04-12T21:26:14.832 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 17.2.7-827-gd3b5060a (d3b5060a17449edfae9385cdf792afec109fca5d) quincy (stable)": 1, 2024-04-12T21:26:14.832 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 19.0.0-2879-g324baf8d (324baf8ddb91dea46cac50cb535c306115791355) squid (dev)": 13 2024-04-12T21:26:14.832 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-04-12T21:26:14.832 INFO:teuthology.orchestra.run.smithi018.stdout:} 2024-04-12T21:26:14.833 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:14.835+0000 7f00a6400700 1 -- 172.21.15.18:0/366602254 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f008806ec10 msgr2=0x7f00880710c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:26:14.833 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:14.836+0000 7f00a6400700 1 --2- 172.21.15.18:0/366602254 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f008806ec10 0x7f00880710c0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f00a0111450 tx=0x7f008c007d40 comp rx=0 tx=0).stop 2024-04-12T21:26:14.833 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:14.836+0000 7f00a6400700 1 -- 172.21.15.18:0/366602254 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f00a0102c40 msgr2=0x7f00a0115bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:26:14.833 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:14.836+0000 7f00a6400700 1 --2- 172.21.15.18:0/366602254 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f00a0102c40 0x7f00a0115bb0 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f0094009150 tx=0x7f0094009e30 comp rx=0 tx=0).stop 2024-04-12T21:26:14.834 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:14.836+0000 7f00a6400700 1 -- 172.21.15.18:0/366602254 shutdown_connections 2024-04-12T21:26:14.834 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:14.836+0000 7f00a6400700 1 --2- 172.21.15.18:0/366602254 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f008806ec10 0x7f00880710c0 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:26:14.834 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:14.836+0000 7f00a6400700 1 --2- 172.21.15.18:0/366602254 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f00a0102c40 0x7f00a0115bb0 unknown :-1 s=CLOSED pgs=675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:26:14.834 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:14.836+0000 7f00a6400700 1 --2- 172.21.15.18:0/366602254 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f00a01035e0 0x7f00a01160f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:26:14.834 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:14.836+0000 7f00a6400700 1 -- 172.21.15.18:0/366602254 >> 172.21.15.18:0/366602254 conn(0x7f00a00f87b0 msgr2=0x7f00a00f8f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:26:14.834 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:14.836+0000 7f00a6400700 1 -- 172.21.15.18:0/366602254 shutdown_connections 2024-04-12T21:26:14.834 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:14.836+0000 7f00a6400700 1 -- 172.21.15.18:0/366602254 wait complete. 2024-04-12T21:26:14.993 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:14.995+0000 7f3eb5e00700 1 -- 172.21.15.18:0/2108606742 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3eb01059c0 msgr2=0x7f3eb00ff080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:26:14.993 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:14.995+0000 7f3eb5e00700 1 --2- 172.21.15.18:0/2108606742 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3eb01059c0 0x7f3eb00ff080 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f3e98004940 tx=0x7f3e98009e30 comp rx=0 tx=0).stop 2024-04-12T21:26:14.993 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:14.995+0000 7f3eb5e00700 1 -- 172.21.15.18:0/2108606742 shutdown_connections 2024-04-12T21:26:14.993 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:14.995+0000 7f3eb5e00700 1 --2- 172.21.15.18:0/2108606742 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3eb01059c0 0x7f3eb00ff080 unknown :-1 s=CLOSED pgs=676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:26:14.993 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:14.995+0000 7f3eb5e00700 1 --2- 172.21.15.18:0/2108606742 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3eb0105020 0x7f3eb01053f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:26:14.993 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:14.995+0000 7f3eb5e00700 1 -- 172.21.15.18:0/2108606742 >> 172.21.15.18:0/2108606742 conn(0x7f3eb00faba0 msgr2=0x7f3eb00fcfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:26:14.993 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:14.995+0000 7f3eb5e00700 1 -- 172.21.15.18:0/2108606742 shutdown_connections 2024-04-12T21:26:14.993 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:14.996+0000 7f3eb5e00700 1 -- 172.21.15.18:0/2108606742 wait complete. 2024-04-12T21:26:14.995 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:14.996+0000 7f3eb5e00700 1 Processor -- start 2024-04-12T21:26:14.995 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:14.997+0000 7f3eb5e00700 1 -- start start 2024-04-12T21:26:14.995 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:14.997+0000 7f3eb5e00700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3eb0105020 0x7f3eb0115b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:26:14.995 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:14.997+0000 7f3eb5e00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3eb01059c0 0x7f3eb0116050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:26:14.995 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:14.997+0000 7f3eb5e00700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3eb0116730 con 0x7f3eb01059c0 2024-04-12T21:26:14.995 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:14.997+0000 7f3eb5e00700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3eb010fcc0 con 0x7f3eb0105020 2024-04-12T21:26:14.995 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:14.997+0000 7f3ea6000700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3eb01059c0 0x7f3eb0116050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:26:14.995 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:14.997+0000 7f3ea6000700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3eb01059c0 0x7f3eb0116050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:48232/0 (socket says 172.21.15.18:48232) 2024-04-12T21:26:14.995 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:14.997+0000 7f3ea6000700 1 -- 172.21.15.18:0/2968449579 learned_addr learned my addr 172.21.15.18:0/2968449579 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:26:14.995 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:14.998+0000 7f3ea6000700 1 -- 172.21.15.18:0/2968449579 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3eb0105020 msgr2=0x7f3eb0115b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:26:14.995 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:14.998+0000 7f3eaea00700 1 --2- 172.21.15.18:0/2968449579 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3eb0105020 0x7f3eb0115b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:26:14.996 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:14.998+0000 7f3ea6000700 1 --2- 172.21.15.18:0/2968449579 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3eb0105020 0x7f3eb0115b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:26:14.996 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:14.998+0000 7f3ea6000700 1 -- 172.21.15.18:0/2968449579 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e98005040 con 0x7f3eb01059c0 2024-04-12T21:26:14.997 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:14.998+0000 7f3ea6000700 1 --2- 172.21.15.18:0/2968449579 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3eb01059c0 0x7f3eb0116050 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f3e98009100 tx=0x7f3e98009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:26:14.997 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:14.998+0000 7f3ea7400700 1 -- 172.21.15.18:0/2968449579 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3e9800e040 con 0x7f3eb01059c0 2024-04-12T21:26:14.997 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:14.998+0000 7f3ea7400700 1 -- 172.21.15.18:0/2968449579 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f3e98004290 con 0x7f3eb01059c0 2024-04-12T21:26:14.997 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:14.998+0000 7f3eb5e00700 1 -- 172.21.15.18:0/2968449579 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3eb010ff40 con 0x7f3eb01059c0 2024-04-12T21:26:14.997 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:14.998+0000 7f3ea7400700 1 -- 172.21.15.18:0/2968449579 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3e9801c3f0 con 0x7f3eb01059c0 2024-04-12T21:26:14.997 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:14.999+0000 7f3eb5e00700 1 -- 172.21.15.18:0/2968449579 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3eb0110500 con 0x7f3eb01059c0 2024-04-12T21:26:14.997 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:15.000+0000 7f3eb5e00700 1 -- 172.21.15.18:0/2968449579 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3eb0100770 con 0x7f3eb01059c0 2024-04-12T21:26:14.998 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:15.001+0000 7f3ea7400700 1 -- 172.21.15.18:0/2968449579 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7f3e98013790 con 0x7f3eb01059c0 2024-04-12T21:26:14.999 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:15.002+0000 7f3ea7400700 1 --2- 172.21.15.18:0/2968449579 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f3e9406ec00 0x7f3e940710b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:26:15.000 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:15.002+0000 7f3ea7400700 1 -- 172.21.15.18:0/2968449579 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 6757+0+0 (secure 0 0 0) 0x7f3e9808fc30 con 0x7f3eb01059c0 2024-04-12T21:26:15.000 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:15.003+0000 7f3eaea00700 1 --2- 172.21.15.18:0/2968449579 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f3e9406ec00 0x7f3e940710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:26:15.000 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:15.003+0000 7f3eaea00700 1 --2- 172.21.15.18:0/2968449579 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f3e9406ec00 0x7f3e940710b0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f3ea0006fd0 tx=0x7f3ea0008040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:26:15.004 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:15.007+0000 7f3ea7400700 1 -- 172.21.15.18:0/2968449579 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f3e980589e0 con 0x7f3eb01059c0 2024-04-12T21:26:15.029 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:26:14 smithi186 ceph-mon[86379]: pgmap v794: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:26:15.029 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:26:14 smithi186 ceph-mon[86379]: from='client.35400 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:26:15.274 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:15.276+0000 7f3eb5e00700 1 -- 172.21.15.18:0/2968449579 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f3eb004c710 con 0x7f3eb01059c0 2024-04-12T21:26:15.274 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:15.277+0000 7f3ea7400700 1 -- 172.21.15.18:0/2968449579 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 31 v31) v1 ==== 76+0+1873 (secure 0 0 0) 0x7f3e98013420 con 0x7f3eb01059c0 2024-04-12T21:26:15.274 INFO:teuthology.orchestra.run.smithi018.stdout:e31 2024-04-12T21:26:15.275 INFO:teuthology.orchestra.run.smithi018.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-12T21:26:15.275 INFO:teuthology.orchestra.run.smithi018.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-12T21:26:15.275 INFO:teuthology.orchestra.run.smithi018.stdout:legacy client fscid: 1 2024-04-12T21:26:15.275 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-12T21:26:15.275 INFO:teuthology.orchestra.run.smithi018.stdout:Filesystem 'cephfs' (1) 2024-04-12T21:26:15.275 INFO:teuthology.orchestra.run.smithi018.stdout:fs_name cephfs 2024-04-12T21:26:15.275 INFO:teuthology.orchestra.run.smithi018.stdout:epoch 24 2024-04-12T21:26:15.275 INFO:teuthology.orchestra.run.smithi018.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-04-12T21:26:15.275 INFO:teuthology.orchestra.run.smithi018.stdout:created 2024-04-12T20:51:57.362850+0000 2024-04-12T21:26:15.275 INFO:teuthology.orchestra.run.smithi018.stdout:modified 2024-04-12T21:06:59.717190+0000 2024-04-12T21:26:15.275 INFO:teuthology.orchestra.run.smithi018.stdout:tableserver 0 2024-04-12T21:26:15.275 INFO:teuthology.orchestra.run.smithi018.stdout:root 0 2024-04-12T21:26:15.275 INFO:teuthology.orchestra.run.smithi018.stdout:session_timeout 60 2024-04-12T21:26:15.275 INFO:teuthology.orchestra.run.smithi018.stdout:session_autoclose 300 2024-04-12T21:26:15.276 INFO:teuthology.orchestra.run.smithi018.stdout:max_file_size 1099511627776 2024-04-12T21:26:15.276 INFO:teuthology.orchestra.run.smithi018.stdout:max_xattr_size 65536 2024-04-12T21:26:15.276 INFO:teuthology.orchestra.run.smithi018.stdout:required_client_features {} 2024-04-12T21:26:15.276 INFO:teuthology.orchestra.run.smithi018.stdout:last_failure 0 2024-04-12T21:26:15.276 INFO:teuthology.orchestra.run.smithi018.stdout:last_failure_osd_epoch 100 2024-04-12T21:26:15.276 INFO:teuthology.orchestra.run.smithi018.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-12T21:26:15.276 INFO:teuthology.orchestra.run.smithi018.stdout:max_mds 2 2024-04-12T21:26:15.276 INFO:teuthology.orchestra.run.smithi018.stdout:in 0,1 2024-04-12T21:26:15.276 INFO:teuthology.orchestra.run.smithi018.stdout:up {} 2024-04-12T21:26:15.276 INFO:teuthology.orchestra.run.smithi018.stdout:failed 0,1 2024-04-12T21:26:15.276 INFO:teuthology.orchestra.run.smithi018.stdout:damaged 2024-04-12T21:26:15.276 INFO:teuthology.orchestra.run.smithi018.stdout:stopped 2024-04-12T21:26:15.276 INFO:teuthology.orchestra.run.smithi018.stdout:data_pools [3] 2024-04-12T21:26:15.276 INFO:teuthology.orchestra.run.smithi018.stdout:metadata_pool 2 2024-04-12T21:26:15.277 INFO:teuthology.orchestra.run.smithi018.stdout:inline_data disabled 2024-04-12T21:26:15.277 INFO:teuthology.orchestra.run.smithi018.stdout:balancer 2024-04-12T21:26:15.277 INFO:teuthology.orchestra.run.smithi018.stdout:bal_rank_mask -1 2024-04-12T21:26:15.277 INFO:teuthology.orchestra.run.smithi018.stdout:standby_count_wanted 1 2024-04-12T21:26:15.277 INFO:teuthology.orchestra.run.smithi018.stdout:qdb_cluster leader: 0 members: 2024-04-12T21:26:15.277 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-12T21:26:15.277 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-12T21:26:15.277 INFO:teuthology.orchestra.run.smithi018.stdout:Standby daemons: 2024-04-12T21:26:15.277 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-12T21:26:15.277 INFO:teuthology.orchestra.run.smithi018.stdout:[mds.cephfs.smithi186.wjnafs{-1:24277} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.186:6826/438347207,v1:172.21.15.186:6827/438347207] compat {c=[1],r=[1],i=[7ff]}] 2024-04-12T21:26:15.277 INFO:teuthology.orchestra.run.smithi018.stdout:[mds.cephfs.smithi018.acqbsc{-1:34436} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.18:6828/755179764,v1:172.21.15.18:6829/755179764] compat {c=[1],r=[1],i=[1fff]}] 2024-04-12T21:26:15.277 INFO:teuthology.orchestra.run.smithi018.stdout:[mds.cephfs.smithi018.szjluo{-1:34468} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.18:6826/2407992653,v1:172.21.15.18:6827/2407992653] compat {c=[1],r=[1],i=[1fff]}] 2024-04-12T21:26:15.277 INFO:teuthology.orchestra.run.smithi018.stdout:[mds.cephfs.smithi186.jdllyn{-1:44595} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.186:6824/3523444202,v1:172.21.15.186:6825/3523444202] compat {c=[1],r=[1],i=[1fff]}] 2024-04-12T21:26:15.280 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:15.283+0000 7f3eb5e00700 1 -- 172.21.15.18:0/2968449579 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f3e9406ec00 msgr2=0x7f3e940710b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:26:15.280 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:15.283+0000 7f3eb5e00700 1 --2- 172.21.15.18:0/2968449579 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f3e9406ec00 0x7f3e940710b0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f3ea0006fd0 tx=0x7f3ea0008040 comp rx=0 tx=0).stop 2024-04-12T21:26:15.280 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:15.283+0000 7f3eb5e00700 1 -- 172.21.15.18:0/2968449579 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3eb01059c0 msgr2=0x7f3eb0116050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:26:15.280 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:15.283+0000 7f3eb5e00700 1 --2- 172.21.15.18:0/2968449579 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3eb01059c0 0x7f3eb0116050 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f3e98009100 tx=0x7f3e98009e30 comp rx=0 tx=0).stop 2024-04-12T21:26:15.281 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:15.283+0000 7f3eb5e00700 1 -- 172.21.15.18:0/2968449579 shutdown_connections 2024-04-12T21:26:15.281 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:15.283+0000 7f3eb5e00700 1 --2- 172.21.15.18:0/2968449579 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f3e9406ec00 0x7f3e940710b0 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:26:15.281 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:15.283+0000 7f3eb5e00700 1 --2- 172.21.15.18:0/2968449579 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3eb0105020 0x7f3eb0115b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:26:15.281 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:15.283+0000 7f3eb5e00700 1 --2- 172.21.15.18:0/2968449579 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3eb01059c0 0x7f3eb0116050 unknown :-1 s=CLOSED pgs=677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:26:15.281 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:15.283+0000 7f3eb5e00700 1 -- 172.21.15.18:0/2968449579 >> 172.21.15.18:0/2968449579 conn(0x7f3eb00faba0 msgr2=0x7f3eb00fbaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:26:15.281 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:15.283+0000 7f3eb5e00700 1 -- 172.21.15.18:0/2968449579 shutdown_connections 2024-04-12T21:26:15.281 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:15.284+0000 7f3eb5e00700 1 -- 172.21.15.18:0/2968449579 wait complete. 2024-04-12T21:26:15.282 INFO:teuthology.orchestra.run.smithi018.stderr:dumped fsmap epoch 31 2024-04-12T21:26:15.438 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:15.441+0000 7f46a0800700 1 -- 172.21.15.18:0/2024654780 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4698105090 msgr2=0x7f4698105460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:26:15.439 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:15.441+0000 7f46a0800700 1 --2- 172.21.15.18:0/2024654780 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4698105090 0x7f4698105460 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f468c004940 tx=0x7f468c009e30 comp rx=0 tx=0).stop 2024-04-12T21:26:15.439 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:15.441+0000 7f46a0800700 1 -- 172.21.15.18:0/2024654780 shutdown_connections 2024-04-12T21:26:15.439 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:15.441+0000 7f46a0800700 1 --2- 172.21.15.18:0/2024654780 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4698105a30 0x7f46980ff0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:26:15.439 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:15.441+0000 7f46a0800700 1 --2- 172.21.15.18:0/2024654780 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4698105090 0x7f4698105460 unknown :-1 s=CLOSED pgs=678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:26:15.439 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:15.441+0000 7f46a0800700 1 -- 172.21.15.18:0/2024654780 >> 172.21.15.18:0/2024654780 conn(0x7f46980fac00 msgr2=0x7f46980fd010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:26:15.439 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:15.441+0000 7f46a0800700 1 -- 172.21.15.18:0/2024654780 shutdown_connections 2024-04-12T21:26:15.439 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:15.441+0000 7f46a0800700 1 -- 172.21.15.18:0/2024654780 wait complete. 2024-04-12T21:26:15.440 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:15.442+0000 7f46a0800700 1 Processor -- start 2024-04-12T21:26:15.440 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:15.442+0000 7f46a0800700 1 -- start start 2024-04-12T21:26:15.440 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:15.443+0000 7f46a0800700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4698105090 0x7f4698115b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:26:15.440 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:15.443+0000 7f46a0800700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4698105a30 0x7f4698116090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:26:15.440 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:15.443+0000 7f46a0800700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46981166e0 con 0x7f4698105a30 2024-04-12T21:26:15.440 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:15.443+0000 7f46a0800700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f469810fcc0 con 0x7f4698105090 2024-04-12T21:26:15.440 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:15.443+0000 7f469d400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4698105a30 0x7f4698116090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:26:15.441 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:15.443+0000 7f469d400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4698105a30 0x7f4698116090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:48256/0 (socket says 172.21.15.18:48256) 2024-04-12T21:26:15.441 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:15.443+0000 7f469d400700 1 -- 172.21.15.18:0/233642252 learned_addr learned my addr 172.21.15.18:0/233642252 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:26:15.442 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:15.443+0000 7f469de00700 1 --2- 172.21.15.18:0/233642252 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4698105090 0x7f4698115b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:26:15.442 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:15.443+0000 7f469d400700 1 -- 172.21.15.18:0/233642252 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4698105090 msgr2=0x7f4698115b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:26:15.442 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:15.443+0000 7f469d400700 1 --2- 172.21.15.18:0/233642252 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4698105090 0x7f4698115b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:26:15.442 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:15.444+0000 7f469d400700 1 -- 172.21.15.18:0/233642252 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f468c005040 con 0x7f4698105a30 2024-04-12T21:26:15.442 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:15.444+0000 7f469de00700 1 --2- 172.21.15.18:0/233642252 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4698105090 0x7f4698115b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-12T21:26:15.442 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:15.444+0000 7f469d400700 1 --2- 172.21.15.18:0/233642252 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4698105a30 0x7f4698116090 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f468400c890 tx=0x7f46840044c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:26:15.442 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:15.444+0000 7f4692000700 1 -- 172.21.15.18:0/233642252 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f468400a7d0 con 0x7f4698105a30 2024-04-12T21:26:15.442 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:15.444+0000 7f4692000700 1 -- 172.21.15.18:0/233642252 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f468400ae00 con 0x7f4698105a30 2024-04-12T21:26:15.442 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:15.444+0000 7f4692000700 1 -- 172.21.15.18:0/233642252 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f46840113a0 con 0x7f4698105a30 2024-04-12T21:26:15.442 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:15.444+0000 7f46a0800700 1 -- 172.21.15.18:0/233642252 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f469810ffa0 con 0x7f4698105a30 2024-04-12T21:26:15.443 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:15.444+0000 7f46a0800700 1 -- 172.21.15.18:0/233642252 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4698110560 con 0x7f4698105a30 2024-04-12T21:26:15.443 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:15.446+0000 7f46a0800700 1 -- 172.21.15.18:0/233642252 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f46981023a0 con 0x7f4698105a30 2024-04-12T21:26:15.443 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:15.446+0000 7f4692000700 1 -- 172.21.15.18:0/233642252 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7f4684012400 con 0x7f4698105a30 2024-04-12T21:26:15.444 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:15.447+0000 7f4692000700 1 --2- 172.21.15.18:0/233642252 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f468006ec00 0x7f46800710b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:26:15.444 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:15.447+0000 7f4692000700 1 -- 172.21.15.18:0/233642252 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 6757+0+0 (secure 0 0 0) 0x7f468408eae0 con 0x7f4698105a30 2024-04-12T21:26:15.445 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:15.448+0000 7f469de00700 1 --2- 172.21.15.18:0/233642252 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f468006ec00 0x7f46800710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:26:15.445 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:15.448+0000 7f469de00700 1 --2- 172.21.15.18:0/233642252 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f468006ec00 0x7f46800710b0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f468c00b3b0 tx=0x7f468c009e30 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:26:15.449 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:15.452+0000 7f4692000700 1 -- 172.21.15.18:0/233642252 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f468405b290 con 0x7f4698105a30 2024-04-12T21:26:15.699 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:15.701+0000 7f46a0800700 1 -- 172.21.15.18:0/233642252 --> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f4698111210 con 0x7f468006ec00 2024-04-12T21:26:15.701 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:15.704+0000 7f4692000700 1 -- 172.21.15.18:0/233642252 <== mgr.34102 v2:172.21.15.18:6800/3812891919 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f4698111210 con 0x7f468006ec00 2024-04-12T21:26:15.702 INFO:teuthology.orchestra.run.smithi018.stdout:{ 2024-04-12T21:26:15.702 INFO:teuthology.orchestra.run.smithi018.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:f4452def3be4d350f4a8e5feaf7ebb1511be9298fe68478ba7a2d6fbd7d69a63", 2024-04-12T21:26:15.703 INFO:teuthology.orchestra.run.smithi018.stdout: "in_progress": true, 2024-04-12T21:26:15.703 INFO:teuthology.orchestra.run.smithi018.stdout: "which": "Upgrading all daemon types on all hosts", 2024-04-12T21:26:15.703 INFO:teuthology.orchestra.run.smithi018.stdout: "services_complete": [ 2024-04-12T21:26:15.703 INFO:teuthology.orchestra.run.smithi018.stdout: "crash", 2024-04-12T21:26:15.703 INFO:teuthology.orchestra.run.smithi018.stdout: "osd", 2024-04-12T21:26:15.703 INFO:teuthology.orchestra.run.smithi018.stdout: "mon", 2024-04-12T21:26:15.703 INFO:teuthology.orchestra.run.smithi018.stdout: "mgr" 2024-04-12T21:26:15.703 INFO:teuthology.orchestra.run.smithi018.stdout: ], 2024-04-12T21:26:15.703 INFO:teuthology.orchestra.run.smithi018.stdout: "progress": "15/21 daemons upgraded", 2024-04-12T21:26:15.703 INFO:teuthology.orchestra.run.smithi018.stdout: "message": "Currently upgrading mds daemons", 2024-04-12T21:26:15.703 INFO:teuthology.orchestra.run.smithi018.stdout: "is_paused": false 2024-04-12T21:26:15.703 INFO:teuthology.orchestra.run.smithi018.stdout:} 2024-04-12T21:26:15.704 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:15.707+0000 7f46a0800700 1 -- 172.21.15.18:0/233642252 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f468006ec00 msgr2=0x7f46800710b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:26:15.704 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:15.707+0000 7f46a0800700 1 --2- 172.21.15.18:0/233642252 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f468006ec00 0x7f46800710b0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f468c00b3b0 tx=0x7f468c009e30 comp rx=0 tx=0).stop 2024-04-12T21:26:15.704 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:15.707+0000 7f46a0800700 1 -- 172.21.15.18:0/233642252 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4698105a30 msgr2=0x7f4698116090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:26:15.704 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:15.707+0000 7f46a0800700 1 --2- 172.21.15.18:0/233642252 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4698105a30 0x7f4698116090 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f468400c890 tx=0x7f46840044c0 comp rx=0 tx=0).stop 2024-04-12T21:26:15.705 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:15.707+0000 7f46a0800700 1 -- 172.21.15.18:0/233642252 shutdown_connections 2024-04-12T21:26:15.705 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:15.707+0000 7f46a0800700 1 --2- 172.21.15.18:0/233642252 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f468006ec00 0x7f46800710b0 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:26:15.705 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:15.707+0000 7f46a0800700 1 --2- 172.21.15.18:0/233642252 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4698105090 0x7f4698115b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:26:15.705 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:15.707+0000 7f46a0800700 1 --2- 172.21.15.18:0/233642252 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4698105a30 0x7f4698116090 unknown :-1 s=CLOSED pgs=679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:26:15.705 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:15.707+0000 7f46a0800700 1 -- 172.21.15.18:0/233642252 >> 172.21.15.18:0/233642252 conn(0x7f46980fac00 msgr2=0x7f46980fc900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:26:15.705 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:15.707+0000 7f46a0800700 1 -- 172.21.15.18:0/233642252 shutdown_connections 2024-04-12T21:26:15.705 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:15.707+0000 7f46a0800700 1 -- 172.21.15.18:0/233642252 wait complete. 2024-04-12T21:26:15.850 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:26:15 smithi018 ceph-mon[112590]: from='client.35404 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:26:15.850 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:26:15 smithi018 ceph-mon[112590]: from='client.35408 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:26:15.851 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:26:15 smithi018 ceph-mon[112590]: from='client.? 172.21.15.18:0/366602254' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T21:26:15.851 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:26:15 smithi018 ceph-mon[112590]: from='client.? 172.21.15.18:0/2968449579' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-12T21:26:15.862 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:15.865+0000 7f87f5800700 1 -- 172.21.15.18:0/3723749339 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f87f0109dd0 msgr2=0x7f87f0103440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:26:15.862 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:15.865+0000 7f87f5800700 1 --2- 172.21.15.18:0/3723749339 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f87f0109dd0 0x7f87f0103440 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f87dc004940 tx=0x7f87dc009e30 comp rx=0 tx=0).stop 2024-04-12T21:26:15.862 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:15.865+0000 7f87f5800700 1 -- 172.21.15.18:0/3723749339 shutdown_connections 2024-04-12T21:26:15.863 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:15.865+0000 7f87f5800700 1 --2- 172.21.15.18:0/3723749339 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f87f0109dd0 0x7f87f0103440 unknown :-1 s=CLOSED pgs=680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:26:15.863 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:15.865+0000 7f87f5800700 1 --2- 172.21.15.18:0/3723749339 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f87f0109430 0x7f87f0109800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:26:15.863 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:15.865+0000 7f87f5800700 1 -- 172.21.15.18:0/3723749339 >> 172.21.15.18:0/3723749339 conn(0x7f87f0072c00 msgr2=0x7f87f0075010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:26:15.863 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:15.865+0000 7f87f5800700 1 -- 172.21.15.18:0/3723749339 shutdown_connections 2024-04-12T21:26:15.863 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:15.865+0000 7f87f5800700 1 -- 172.21.15.18:0/3723749339 wait complete. 2024-04-12T21:26:15.865 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:15.866+0000 7f87f5800700 1 Processor -- start 2024-04-12T21:26:15.865 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:15.866+0000 7f87f5800700 1 -- start start 2024-04-12T21:26:15.865 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:15.866+0000 7f87f5800700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f87f0109430 0x7f87f0119f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:26:15.865 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:15.866+0000 7f87f5800700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f87f0109dd0 0x7f87f011a4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:26:15.865 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:15.866+0000 7f87f5800700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87f011ab10 con 0x7f87f0109430 2024-04-12T21:26:15.865 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:15.866+0000 7f87f5800700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87f01140f0 con 0x7f87f0109dd0 2024-04-12T21:26:15.865 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:15.867+0000 7f87eea00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f87f0109430 0x7f87f0119f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:26:15.865 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:15.867+0000 7f87eea00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f87f0109430 0x7f87f0119f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:48268/0 (socket says 172.21.15.18:48268) 2024-04-12T21:26:15.865 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:15.867+0000 7f87eea00700 1 -- 172.21.15.18:0/3773082529 learned_addr learned my addr 172.21.15.18:0/3773082529 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:26:15.866 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:15.867+0000 7f87eea00700 1 -- 172.21.15.18:0/3773082529 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f87f0109dd0 msgr2=0x7f87f011a4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:26:15.866 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:15.867+0000 7f87eea00700 1 --2- 172.21.15.18:0/3773082529 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f87f0109dd0 0x7f87f011a4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:26:15.866 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:15.867+0000 7f87eea00700 1 -- 172.21.15.18:0/3773082529 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87dc005040 con 0x7f87f0109430 2024-04-12T21:26:15.866 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:15.867+0000 7f87eea00700 1 --2- 172.21.15.18:0/3773082529 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f87f0109430 0x7f87f0119f80 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f87e0002a50 tx=0x7f87e00046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:26:15.866 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:15.868+0000 7f87e6a00700 1 -- 172.21.15.18:0/3773082529 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f87e0007cc0 con 0x7f87f0109430 2024-04-12T21:26:15.866 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:15.868+0000 7f87e6a00700 1 -- 172.21.15.18:0/3773082529 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f87e000f040 con 0x7f87f0109430 2024-04-12T21:26:15.866 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:15.868+0000 7f87f5800700 1 -- 172.21.15.18:0/3773082529 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f87f01143d0 con 0x7f87f0109430 2024-04-12T21:26:15.866 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:15.868+0000 7f87e6a00700 1 -- 172.21.15.18:0/3773082529 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f87e0009640 con 0x7f87f0109430 2024-04-12T21:26:15.866 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:15.868+0000 7f87f5800700 1 -- 172.21.15.18:0/3773082529 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f87f0114990 con 0x7f87f0109430 2024-04-12T21:26:15.867 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:15.869+0000 7f87f5800700 1 -- 172.21.15.18:0/3773082529 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f87f0104b80 con 0x7f87f0109430 2024-04-12T21:26:15.867 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:15.870+0000 7f87e6a00700 1 -- 172.21.15.18:0/3773082529 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7f87e00097a0 con 0x7f87f0109430 2024-04-12T21:26:15.868 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:15.871+0000 7f87e6a00700 1 --2- 172.21.15.18:0/3773082529 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f87d406ec00 0x7f87d40710b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:26:15.868 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:15.871+0000 7f87e6a00700 1 -- 172.21.15.18:0/3773082529 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 6757+0+0 (secure 0 0 0) 0x7f87e008e6c0 con 0x7f87f0109430 2024-04-12T21:26:15.868 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:15.871+0000 7f87ee000700 1 --2- 172.21.15.18:0/3773082529 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f87d406ec00 0x7f87d40710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:26:15.869 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:15.872+0000 7f87ee000700 1 --2- 172.21.15.18:0/3773082529 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f87d406ec00 0x7f87d40710b0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f87dc004f90 tx=0x7f87dc009e30 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:26:15.873 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:15.876+0000 7f87e6a00700 1 -- 172.21.15.18:0/3773082529 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f87e005af10 con 0x7f87f0109430 2024-04-12T21:26:16.030 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:26:15 smithi186 ceph-mon[86379]: from='client.35404 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:26:16.030 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:26:15 smithi186 ceph-mon[86379]: from='client.35408 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:26:16.030 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:26:15 smithi186 ceph-mon[86379]: from='client.? 172.21.15.18:0/366602254' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T21:26:16.030 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:26:15 smithi186 ceph-mon[86379]: from='client.? 172.21.15.18:0/2968449579' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-12T21:26:16.190 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:16.192+0000 7f87f5800700 1 -- 172.21.15.18:0/3773082529 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f87f0110500 con 0x7f87f0109430 2024-04-12T21:26:16.190 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:16.193+0000 7f87e6a00700 1 -- 172.21.15.18:0/3773082529 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+4303 (secure 0 0 0) 0x7f87e005aaa0 con 0x7f87f0109430 2024-04-12T21:26:16.191 INFO:teuthology.orchestra.run.smithi018.stdout:HEALTH_ERR failed to probe daemons or devices; 1 filesystem is degraded; 1 filesystem is offline 2024-04-12T21:26:16.191 INFO:teuthology.orchestra.run.smithi018.stdout:[WRN] CEPHADM_REFRESH_FAILED: failed to probe daemons or devices 2024-04-12T21:26:16.191 INFO:teuthology.orchestra.run.smithi018.stdout: host smithi018 `cephadm ceph-volume` failed: cephadm exited with an error code: 1, stderr: Inferring config /var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/mon.smithi018/config 2024-04-12T21:26:16.191 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/podman: timeout after 895 seconds 2024-04-12T21:26:16.192 INFO:teuthology.orchestra.run.smithi018.stdout:Non-zero exit code 124 from /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:f4452def3be4d350f4a8e5feaf7ebb1511be9298fe68478ba7a2d6fbd7d69a63 -e NODE_NAME=smithi018 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968:/var/run/ceph:z -v /var/log/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968:/var/log/ceph:z -v /var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmpxsmwkkuh:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:f4452def3be4d350f4a8e5feaf7ebb1511be9298fe68478ba7a2d6fbd7d69a63 inventory --format=json-pretty --filter-for-batch 2024-04-12T21:26:16.192 INFO:teuthology.orchestra.run.smithi018.stdout:Traceback (most recent call last): 2024-04-12T21:26:16.192 INFO:teuthology.orchestra.run.smithi018.stdout: File "/usr/lib64/python3.9/runpy.py", line 197, in _run_module_as_main 2024-04-12T21:26:16.192 INFO:teuthology.orchestra.run.smithi018.stdout: return _run_code(code, main_globals, None, 2024-04-12T21:26:16.192 INFO:teuthology.orchestra.run.smithi018.stdout: File "/usr/lib64/python3.9/runpy.py", line 87, in _run_code 2024-04-12T21:26:16.192 INFO:teuthology.orchestra.run.smithi018.stdout: exec(code, run_globals) 2024-04-12T21:26:16.192 INFO:teuthology.orchestra.run.smithi018.stdout: File "/var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/cephadm.e44f3ed5e34e4ef52eedef233d1190bff004470d8e8005127b5312bff357fa7f/__main__.py", line 5618, in 2024-04-12T21:26:16.192 INFO:teuthology.orchestra.run.smithi018.stdout: File "/var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/cephadm.e44f3ed5e34e4ef52eedef233d1190bff004470d8e8005127b5312bff357fa7f/__main__.py", line 5606, in main 2024-04-12T21:26:16.192 INFO:teuthology.orchestra.run.smithi018.stdout: File "/var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/cephadm.e44f3ed5e34e4ef52eedef233d1190bff004470d8e8005127b5312bff357fa7f/__main__.py", line 408, in _infer_config 2024-04-12T21:26:16.192 INFO:teuthology.orchestra.run.smithi018.stdout: File "/var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/cephadm.e44f3ed5e34e4ef52eedef233d1190bff004470d8e8005127b5312bff357fa7f/__main__.py", line 323, in _infer_fsid 2024-04-12T21:26:16.192 INFO:teuthology.orchestra.run.smithi018.stdout: File "/var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/cephadm.e44f3ed5e34e4ef52eedef233d1190bff004470d8e8005127b5312bff357fa7f/__main__.py", line 436, in _infer_image 2024-04-12T21:26:16.192 INFO:teuthology.orchestra.run.smithi018.stdout: File "/var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/cephadm.e44f3ed5e34e4ef52eedef233d1190bff004470d8e8005127b5312bff357fa7f/__main__.py", line 310, in _validate_fsid 2024-04-12T21:26:16.192 INFO:teuthology.orchestra.run.smithi018.stdout: File "/var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/cephadm.e44f3ed5e34e4ef52eedef233d1190bff004470d8e8005127b5312bff357fa7f/__main__.py", line 3359, in command_ceph_volume 2024-04-12T21:26:16.193 INFO:teuthology.orchestra.run.smithi018.stdout: File "/var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/cephadm.e44f3ed5e34e4ef52eedef233d1190bff004470d8e8005127b5312bff357fa7f/cephadmlib/call_wrappers.py", line 310, in call_throws 2024-04-12T21:26:16.193 INFO:teuthology.orchestra.run.smithi018.stdout:RuntimeError: Failed command: /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:f4452def3be4d350f4a8e5feaf7ebb1511be9298fe68478ba7a2d6fbd7d69a63 -e NODE_NAME=smithi018 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968:/var/run/ceph:z -v /var/log/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968:/var/log/ceph:z -v /var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmpxsmwkkuh:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:f4452def3be4d350f4a8e5feaf7ebb1511be9298fe68478ba7a2d6fbd7d69a63 inventory --format=json-pretty --filter-for-batch 2024-04-12T21:26:16.193 INFO:teuthology.orchestra.run.smithi018.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-04-12T21:26:16.193 INFO:teuthology.orchestra.run.smithi018.stdout: fs cephfs is degraded 2024-04-12T21:26:16.193 INFO:teuthology.orchestra.run.smithi018.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-04-12T21:26:16.193 INFO:teuthology.orchestra.run.smithi018.stdout: fs cephfs is offline because no MDS is active for it. 2024-04-12T21:26:16.194 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:16.196+0000 7f87f5800700 1 -- 172.21.15.18:0/3773082529 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f87d406ec00 msgr2=0x7f87d40710b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:26:16.194 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:16.196+0000 7f87f5800700 1 --2- 172.21.15.18:0/3773082529 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f87d406ec00 0x7f87d40710b0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f87dc004f90 tx=0x7f87dc009e30 comp rx=0 tx=0).stop 2024-04-12T21:26:16.194 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:16.196+0000 7f87f5800700 1 -- 172.21.15.18:0/3773082529 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f87f0109430 msgr2=0x7f87f0119f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:26:16.194 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:16.196+0000 7f87f5800700 1 --2- 172.21.15.18:0/3773082529 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f87f0109430 0x7f87f0119f80 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f87e0002a50 tx=0x7f87e00046b0 comp rx=0 tx=0).stop 2024-04-12T21:26:16.194 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:16.196+0000 7f87f5800700 1 -- 172.21.15.18:0/3773082529 shutdown_connections 2024-04-12T21:26:16.194 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:16.196+0000 7f87f5800700 1 --2- 172.21.15.18:0/3773082529 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f87d406ec00 0x7f87d40710b0 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:26:16.194 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:16.196+0000 7f87f5800700 1 --2- 172.21.15.18:0/3773082529 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f87f0109430 0x7f87f0119f80 unknown :-1 s=CLOSED pgs=681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:26:16.194 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:16.196+0000 7f87f5800700 1 --2- 172.21.15.18:0/3773082529 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f87f0109dd0 0x7f87f011a4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:26:16.194 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:16.196+0000 7f87f5800700 1 -- 172.21.15.18:0/3773082529 >> 172.21.15.18:0/3773082529 conn(0x7f87f0072c00 msgr2=0x7f87f0074c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:26:16.194 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:16.196+0000 7f87f5800700 1 -- 172.21.15.18:0/3773082529 shutdown_connections 2024-04-12T21:26:16.195 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:16.197+0000 7f87f5800700 1 -- 172.21.15.18:0/3773082529 wait complete. 2024-04-12T21:26:16.851 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:26:16 smithi018 ceph-mon[112590]: pgmap v795: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:26:16.851 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:26:16 smithi018 ceph-mon[112590]: from='client.35420 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:26:16.851 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:26:16 smithi018 ceph-mon[112590]: from='client.? 172.21.15.18:0/3773082529' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-12T21:26:17.030 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:26:16 smithi186 ceph-mon[86379]: pgmap v795: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:26:17.030 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:26:16 smithi186 ceph-mon[86379]: from='client.35420 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:26:17.030 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:26:16 smithi186 ceph-mon[86379]: from='client.? 172.21.15.18:0/3773082529' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-12T21:26:18.850 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:26:18 smithi018 ceph-mon[112590]: pgmap v796: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:26:19.030 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:26:18 smithi186 ceph-mon[86379]: pgmap v796: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:26:21.029 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:26:20 smithi186 ceph-mon[86379]: pgmap v797: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:26:21.100 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:26:20 smithi018 ceph-mon[112590]: pgmap v797: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:26:23.030 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:26:22 smithi186 ceph-mon[86379]: pgmap v798: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:26:23.101 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:26:22 smithi018 ceph-mon[112590]: pgmap v798: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:26:25.030 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:26:24 smithi186 ceph-mon[86379]: pgmap v799: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:26:25.101 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:26:24 smithi018 ceph-mon[112590]: pgmap v799: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:26:25.851 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:26:25 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T21:26:26.030 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:26:25 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T21:26:27.030 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:26:26 smithi186 ceph-mon[86379]: pgmap v800: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:26:27.101 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:26:26 smithi018 ceph-mon[112590]: pgmap v800: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:26:29.030 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:26:28 smithi186 ceph-mon[86379]: pgmap v801: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:26:29.101 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:26:28 smithi018 ceph-mon[112590]: pgmap v801: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:26:31.030 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:26:30 smithi186 ceph-mon[86379]: pgmap v802: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:26:31.101 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:26:30 smithi018 ceph-mon[112590]: pgmap v802: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:26:33.030 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:26:32 smithi186 ceph-mon[86379]: pgmap v803: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:26:33.101 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:26:32 smithi018 ceph-mon[112590]: pgmap v803: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:26:35.030 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:26:34 smithi186 ceph-mon[86379]: pgmap v804: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:26:35.101 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:26:34 smithi018 ceph-mon[112590]: pgmap v804: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:26:37.030 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:26:36 smithi186 ceph-mon[86379]: pgmap v805: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:26:37.101 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:26:36 smithi018 ceph-mon[112590]: pgmap v805: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:26:39.030 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:26:38 smithi186 ceph-mon[86379]: pgmap v806: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:26:39.101 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:26:38 smithi018 ceph-mon[112590]: pgmap v806: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:26:41.030 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:26:40 smithi186 ceph-mon[86379]: pgmap v807: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:26:41.030 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:26:40 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T21:26:41.101 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:26:40 smithi018 ceph-mon[112590]: pgmap v807: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:26:41.101 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:26:40 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T21:26:43.030 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:26:42 smithi186 ceph-mon[86379]: pgmap v808: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:26:43.101 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:26:42 smithi018 ceph-mon[112590]: pgmap v808: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:26:45.030 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:26:44 smithi186 ceph-mon[86379]: pgmap v809: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:26:45.101 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:26:44 smithi018 ceph-mon[112590]: pgmap v809: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:26:46.367 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:46.370+0000 7fae0aa00700 1 -- 172.21.15.18:0/474852630 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fae04109430 msgr2=0x7fae04109800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:26:46.368 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:46.370+0000 7fae0aa00700 1 --2- 172.21.15.18:0/474852630 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fae04109430 0x7fae04109800 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7fadf4004940 tx=0x7fadf4009e30 comp rx=0 tx=0).stop 2024-04-12T21:26:46.368 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:46.370+0000 7fae0aa00700 1 -- 172.21.15.18:0/474852630 shutdown_connections 2024-04-12T21:26:46.368 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:46.370+0000 7fae0aa00700 1 --2- 172.21.15.18:0/474852630 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fae04109dd0 0x7fae04103440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:26:46.368 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:46.370+0000 7fae0aa00700 1 --2- 172.21.15.18:0/474852630 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fae04109430 0x7fae04109800 unknown :-1 s=CLOSED pgs=682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:26:46.368 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:46.370+0000 7fae0aa00700 1 -- 172.21.15.18:0/474852630 >> 172.21.15.18:0/474852630 conn(0x7fae04072c00 msgr2=0x7fae04075010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:26:46.368 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:46.370+0000 7fae0aa00700 1 -- 172.21.15.18:0/474852630 shutdown_connections 2024-04-12T21:26:46.368 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:46.370+0000 7fae0aa00700 1 -- 172.21.15.18:0/474852630 wait complete. 2024-04-12T21:26:46.368 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:46.371+0000 7fae0aa00700 1 Processor -- start 2024-04-12T21:26:46.369 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:46.371+0000 7fae0aa00700 1 -- start start 2024-04-12T21:26:46.369 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:46.372+0000 7fae0aa00700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fae04109430 0x7fae04119f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:26:46.369 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:46.372+0000 7fae0aa00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fae04109dd0 0x7fae0411a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:26:46.370 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:46.372+0000 7fae0aa00700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae0411aad0 con 0x7fae04109dd0 2024-04-12T21:26:46.370 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:46.372+0000 7fae0aa00700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae04114100 con 0x7fae04109430 2024-04-12T21:26:46.370 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:46.372+0000 7fae03400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fae04109dd0 0x7fae0411a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:26:46.370 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:46.372+0000 7fae03400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fae04109dd0 0x7fae0411a480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:39736/0 (socket says 172.21.15.18:39736) 2024-04-12T21:26:46.370 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:46.372+0000 7fae03400700 1 -- 172.21.15.18:0/3668508289 learned_addr learned my addr 172.21.15.18:0/3668508289 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:26:46.371 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:46.373+0000 7fae03e00700 1 --2- 172.21.15.18:0/3668508289 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fae04109430 0x7fae04119f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:26:46.371 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:46.373+0000 7fae03400700 1 -- 172.21.15.18:0/3668508289 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fae04109430 msgr2=0x7fae04119f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:26:46.371 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:46.373+0000 7fae03400700 1 --2- 172.21.15.18:0/3668508289 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fae04109430 0x7fae04119f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:26:46.371 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:46.373+0000 7fae03400700 1 -- 172.21.15.18:0/3668508289 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fadf4005040 con 0x7fae04109dd0 2024-04-12T21:26:46.371 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:46.373+0000 7fae03e00700 1 --2- 172.21.15.18:0/3668508289 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fae04109430 0x7fae04119f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-12T21:26:46.371 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:46.373+0000 7fae03400700 1 --2- 172.21.15.18:0/3668508289 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fae04109dd0 0x7fae0411a480 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7fadf0002a50 tx=0x7fadf0004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:26:46.371 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:46.373+0000 7fadfbe00700 1 -- 172.21.15.18:0/3668508289 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fadf0007cc0 con 0x7fae04109dd0 2024-04-12T21:26:46.371 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:46.373+0000 7fae0aa00700 1 -- 172.21.15.18:0/3668508289 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fae041143e0 con 0x7fae04109dd0 2024-04-12T21:26:46.371 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:46.373+0000 7fadfbe00700 1 -- 172.21.15.18:0/3668508289 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fadf000f040 con 0x7fae04109dd0 2024-04-12T21:26:46.371 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:46.373+0000 7fadfbe00700 1 -- 172.21.15.18:0/3668508289 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fadf000da40 con 0x7fae04109dd0 2024-04-12T21:26:46.371 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:46.374+0000 7fae0aa00700 1 -- 172.21.15.18:0/3668508289 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fae041149a0 con 0x7fae04109dd0 2024-04-12T21:26:46.372 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:46.374+0000 7fae0aa00700 1 -- 172.21.15.18:0/3668508289 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fae04104b80 con 0x7fae04109dd0 2024-04-12T21:26:46.373 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:46.376+0000 7fadfbe00700 1 -- 172.21.15.18:0/3668508289 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7fadf00097c0 con 0x7fae04109dd0 2024-04-12T21:26:46.374 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:46.377+0000 7fadfbe00700 1 --2- 172.21.15.18:0/3668508289 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fade806ec00 0x7fade80710b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:26:46.374 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:46.377+0000 7fadfbe00700 1 -- 172.21.15.18:0/3668508289 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 6757+0+0 (secure 0 0 0) 0x7fadf008e7b0 con 0x7fae04109dd0 2024-04-12T21:26:46.375 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:46.377+0000 7fae03e00700 1 --2- 172.21.15.18:0/3668508289 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fade806ec00 0x7fade80710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:26:46.375 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:46.378+0000 7fae03e00700 1 --2- 172.21.15.18:0/3668508289 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fade806ec00 0x7fade80710b0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fadf400b3b0 tx=0x7fadf4009e30 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:26:46.380 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:46.382+0000 7fadfbe00700 1 -- 172.21.15.18:0/3668508289 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7fadf005acd0 con 0x7fae04109dd0 2024-04-12T21:26:46.627 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:46.630+0000 7fae0aa00700 1 -- 172.21.15.18:0/3668508289 --> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fae04110590 con 0x7fade806ec00 2024-04-12T21:26:46.630 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:46.633+0000 7fadfbe00700 1 -- 172.21.15.18:0/3668508289 <== mgr.34102 v2:172.21.15.18:6800/3812891919 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7fae04110590 con 0x7fade806ec00 2024-04-12T21:26:46.634 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:46.636+0000 7fae0aa00700 1 -- 172.21.15.18:0/3668508289 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fade806ec00 msgr2=0x7fade80710b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:26:46.634 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:46.636+0000 7fae0aa00700 1 --2- 172.21.15.18:0/3668508289 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fade806ec00 0x7fade80710b0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fadf400b3b0 tx=0x7fadf4009e30 comp rx=0 tx=0).stop 2024-04-12T21:26:46.634 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:46.636+0000 7fae0aa00700 1 -- 172.21.15.18:0/3668508289 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fae04109dd0 msgr2=0x7fae0411a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:26:46.634 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:46.636+0000 7fae0aa00700 1 --2- 172.21.15.18:0/3668508289 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fae04109dd0 0x7fae0411a480 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7fadf0002a50 tx=0x7fadf0004600 comp rx=0 tx=0).stop 2024-04-12T21:26:46.634 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:46.636+0000 7fae0aa00700 1 -- 172.21.15.18:0/3668508289 shutdown_connections 2024-04-12T21:26:46.634 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:46.636+0000 7fae0aa00700 1 --2- 172.21.15.18:0/3668508289 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fade806ec00 0x7fade80710b0 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:26:46.634 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:46.636+0000 7fae0aa00700 1 --2- 172.21.15.18:0/3668508289 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fae04109430 0x7fae04119f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:26:46.634 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:46.636+0000 7fae0aa00700 1 --2- 172.21.15.18:0/3668508289 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fae04109dd0 0x7fae0411a480 unknown :-1 s=CLOSED pgs=683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:26:46.634 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:46.636+0000 7fae0aa00700 1 -- 172.21.15.18:0/3668508289 >> 172.21.15.18:0/3668508289 conn(0x7fae04072c00 msgr2=0x7fae04074c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:26:46.635 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:46.637+0000 7fae0aa00700 1 -- 172.21.15.18:0/3668508289 shutdown_connections 2024-04-12T21:26:46.635 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:46.637+0000 7fae0aa00700 1 -- 172.21.15.18:0/3668508289 wait complete. 2024-04-12T21:26:46.649 INFO:teuthology.orchestra.run.smithi018.stdout:true 2024-04-12T21:26:46.793 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:46.795+0000 7f3499c00700 1 -- 172.21.15.18:0/2950934493 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f34941059c0 msgr2=0x7f34940ff080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:26:46.793 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:46.795+0000 7f3499c00700 1 --2- 172.21.15.18:0/2950934493 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f34941059c0 0x7f34940ff080 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7f3480004940 tx=0x7f3480009e30 comp rx=0 tx=0).stop 2024-04-12T21:26:46.793 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:46.795+0000 7f3499c00700 1 -- 172.21.15.18:0/2950934493 shutdown_connections 2024-04-12T21:26:46.793 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:46.795+0000 7f3499c00700 1 --2- 172.21.15.18:0/2950934493 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f34941059c0 0x7f34940ff080 unknown :-1 s=CLOSED pgs=684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:26:46.793 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:46.795+0000 7f3499c00700 1 --2- 172.21.15.18:0/2950934493 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3494105020 0x7f34941053f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:26:46.794 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:46.795+0000 7f3499c00700 1 -- 172.21.15.18:0/2950934493 >> 172.21.15.18:0/2950934493 conn(0x7f34940faba0 msgr2=0x7f34940fcfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:26:46.794 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:46.796+0000 7f3499c00700 1 -- 172.21.15.18:0/2950934493 shutdown_connections 2024-04-12T21:26:46.794 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:46.796+0000 7f3499c00700 1 -- 172.21.15.18:0/2950934493 wait complete. 2024-04-12T21:26:46.795 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:46.796+0000 7f3499c00700 1 Processor -- start 2024-04-12T21:26:46.795 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:46.797+0000 7f3499c00700 1 -- start start 2024-04-12T21:26:46.795 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:46.797+0000 7f3499c00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3494105020 0x7f3494115ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:26:46.795 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:46.797+0000 7f3499c00700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f34941059c0 0x7f34941160e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:26:46.795 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:46.797+0000 7f3499c00700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3494116730 con 0x7f3494105020 2024-04-12T21:26:46.795 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:46.797+0000 7f3499c00700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f349410fd10 con 0x7f34941059c0 2024-04-12T21:26:46.795 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:46.797+0000 7f3492a00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3494105020 0x7f3494115ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:26:46.795 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:46.797+0000 7f3492a00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3494105020 0x7f3494115ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:39760/0 (socket says 172.21.15.18:39760) 2024-04-12T21:26:46.795 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:46.798+0000 7f3492a00700 1 -- 172.21.15.18:0/3969605012 learned_addr learned my addr 172.21.15.18:0/3969605012 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:26:46.796 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:46.798+0000 7f3492a00700 1 -- 172.21.15.18:0/3969605012 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f34941059c0 msgr2=0x7f34941160e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:26:46.796 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:46.798+0000 7f3492000700 1 --2- 172.21.15.18:0/3969605012 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f34941059c0 0x7f34941160e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:26:46.796 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:46.798+0000 7f3492a00700 1 --2- 172.21.15.18:0/3969605012 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f34941059c0 0x7f34941160e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:26:46.797 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:46.798+0000 7f3492a00700 1 -- 172.21.15.18:0/3969605012 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3480005040 con 0x7f3494105020 2024-04-12T21:26:46.797 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:46.798+0000 7f3492a00700 1 --2- 172.21.15.18:0/3969605012 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3494105020 0x7f3494115ba0 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7f3484002a50 tx=0x7f34840046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:26:46.797 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:46.799+0000 7f348aa00700 1 -- 172.21.15.18:0/3969605012 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3484007cc0 con 0x7f3494105020 2024-04-12T21:26:46.797 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:46.799+0000 7f348aa00700 1 -- 172.21.15.18:0/3969605012 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f348400f040 con 0x7f3494105020 2024-04-12T21:26:46.797 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:46.799+0000 7f348aa00700 1 -- 172.21.15.18:0/3969605012 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3484009640 con 0x7f3494105020 2024-04-12T21:26:46.797 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:46.799+0000 7f3499c00700 1 -- 172.21.15.18:0/3969605012 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f349410fff0 con 0x7f3494105020 2024-04-12T21:26:46.797 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:46.799+0000 7f3499c00700 1 -- 172.21.15.18:0/3969605012 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f34941105b0 con 0x7f3494105020 2024-04-12T21:26:46.797 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:46.800+0000 7f3499c00700 1 -- 172.21.15.18:0/3969605012 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3494100770 con 0x7f3494105020 2024-04-12T21:26:46.798 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:46.801+0000 7f348aa00700 1 -- 172.21.15.18:0/3969605012 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7f34840097a0 con 0x7f3494105020 2024-04-12T21:26:46.799 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:46.802+0000 7f348aa00700 1 --2- 172.21.15.18:0/3969605012 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f347806ec00 0x7f34780710b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:26:46.800 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:46.802+0000 7f348aa00700 1 -- 172.21.15.18:0/3969605012 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 6757+0+0 (secure 0 0 0) 0x7f348408e760 con 0x7f3494105020 2024-04-12T21:26:46.800 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:46.802+0000 7f3492000700 1 --2- 172.21.15.18:0/3969605012 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f347806ec00 0x7f34780710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:26:46.800 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:46.803+0000 7f3492000700 1 --2- 172.21.15.18:0/3969605012 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f347806ec00 0x7f34780710b0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f348000b3b0 tx=0x7f3480009e30 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:26:46.805 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:46.807+0000 7f348aa00700 1 -- 172.21.15.18:0/3969605012 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f348405af10 con 0x7f3494105020 2024-04-12T21:26:47.030 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:26:46 smithi186 ceph-mon[86379]: pgmap v810: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:26:47.046 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:47.049+0000 7f3499c00700 1 -- 172.21.15.18:0/3969605012 --> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f3494111260 con 0x7f347806ec00 2024-04-12T21:26:47.047 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:26:46 smithi018 ceph-mon[112590]: pgmap v810: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:26:47.048 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:47.051+0000 7f348aa00700 1 -- 172.21.15.18:0/3969605012 <== mgr.34102 v2:172.21.15.18:6800/3812891919 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f3494111260 con 0x7f347806ec00 2024-04-12T21:26:47.053 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:47.055+0000 7f3499c00700 1 -- 172.21.15.18:0/3969605012 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f347806ec00 msgr2=0x7f34780710b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:26:47.053 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:47.055+0000 7f3499c00700 1 --2- 172.21.15.18:0/3969605012 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f347806ec00 0x7f34780710b0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f348000b3b0 tx=0x7f3480009e30 comp rx=0 tx=0).stop 2024-04-12T21:26:47.053 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:47.055+0000 7f3499c00700 1 -- 172.21.15.18:0/3969605012 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3494105020 msgr2=0x7f3494115ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:26:47.054 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:47.055+0000 7f3499c00700 1 --2- 172.21.15.18:0/3969605012 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3494105020 0x7f3494115ba0 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7f3484002a50 tx=0x7f34840046b0 comp rx=0 tx=0).stop 2024-04-12T21:26:47.054 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:47.055+0000 7f3499c00700 1 -- 172.21.15.18:0/3969605012 shutdown_connections 2024-04-12T21:26:47.054 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:47.055+0000 7f3499c00700 1 --2- 172.21.15.18:0/3969605012 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f347806ec00 0x7f34780710b0 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:26:47.054 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:47.055+0000 7f3499c00700 1 --2- 172.21.15.18:0/3969605012 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3494105020 0x7f3494115ba0 unknown :-1 s=CLOSED pgs=685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:26:47.054 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:47.055+0000 7f3499c00700 1 --2- 172.21.15.18:0/3969605012 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f34941059c0 0x7f34941160e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:26:47.054 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:47.055+0000 7f3499c00700 1 -- 172.21.15.18:0/3969605012 >> 172.21.15.18:0/3969605012 conn(0x7f34940faba0 msgr2=0x7f34940fbb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:26:47.054 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:47.056+0000 7f3499c00700 1 -- 172.21.15.18:0/3969605012 shutdown_connections 2024-04-12T21:26:47.054 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:47.056+0000 7f3499c00700 1 -- 172.21.15.18:0/3969605012 wait complete. 2024-04-12T21:26:47.208 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:47.211+0000 7f1b40a00700 1 -- 172.21.15.18:0/3679074048 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1b3c0fde90 msgr2=0x7f1b3c0fe260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:26:47.209 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:47.211+0000 7f1b40a00700 1 --2- 172.21.15.18:0/3679074048 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1b3c0fde90 0x7f1b3c0fe260 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7f1b2c004940 tx=0x7f1b2c009e30 comp rx=0 tx=0).stop 2024-04-12T21:26:47.209 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:47.211+0000 7f1b40a00700 1 -- 172.21.15.18:0/3679074048 shutdown_connections 2024-04-12T21:26:47.209 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:47.211+0000 7f1b40a00700 1 --2- 172.21.15.18:0/3679074048 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1b3c0fe830 0x7f1b3c106f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:26:47.209 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:47.211+0000 7f1b40a00700 1 --2- 172.21.15.18:0/3679074048 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1b3c0fde90 0x7f1b3c0fe260 unknown :-1 s=CLOSED pgs=686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:26:47.209 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:47.211+0000 7f1b40a00700 1 -- 172.21.15.18:0/3679074048 >> 172.21.15.18:0/3679074048 conn(0x7f1b3c072bc0 msgr2=0x7f1b3c072fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:26:47.209 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:47.211+0000 7f1b40a00700 1 -- 172.21.15.18:0/3679074048 shutdown_connections 2024-04-12T21:26:47.209 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:47.211+0000 7f1b40a00700 1 -- 172.21.15.18:0/3679074048 wait complete. 2024-04-12T21:26:47.210 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:47.212+0000 7f1b40a00700 1 Processor -- start 2024-04-12T21:26:47.211 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:47.212+0000 7f1b40a00700 1 -- start start 2024-04-12T21:26:47.211 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:47.212+0000 7f1b40a00700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1b3c0fde90 0x7f1b3c115d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:26:47.211 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:47.212+0000 7f1b40a00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1b3c0fe830 0x7f1b3c1162c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:26:47.211 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:47.212+0000 7f1b40a00700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b3c116910 con 0x7f1b3c0fe830 2024-04-12T21:26:47.211 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:47.212+0000 7f1b40a00700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b3c10fd20 con 0x7f1b3c0fde90 2024-04-12T21:26:47.211 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:47.212+0000 7f1b39400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1b3c0fe830 0x7f1b3c1162c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:26:47.211 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:47.213+0000 7f1b39400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1b3c0fe830 0x7f1b3c1162c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:39778/0 (socket says 172.21.15.18:39778) 2024-04-12T21:26:47.211 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:47.213+0000 7f1b39400700 1 -- 172.21.15.18:0/2328613819 learned_addr learned my addr 172.21.15.18:0/2328613819 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:26:47.211 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:47.213+0000 7f1b39e00700 1 --2- 172.21.15.18:0/2328613819 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1b3c0fde90 0x7f1b3c115d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:26:47.211 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:47.213+0000 7f1b39400700 1 -- 172.21.15.18:0/2328613819 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1b3c0fde90 msgr2=0x7f1b3c115d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:26:47.211 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:47.213+0000 7f1b39400700 1 --2- 172.21.15.18:0/2328613819 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1b3c0fde90 0x7f1b3c115d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:26:47.212 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:47.213+0000 7f1b39400700 1 -- 172.21.15.18:0/2328613819 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b2c005040 con 0x7f1b3c0fe830 2024-04-12T21:26:47.212 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:47.213+0000 7f1b39400700 1 --2- 172.21.15.18:0/2328613819 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1b3c0fe830 0x7f1b3c1162c0 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f1b2800c830 tx=0x7f1b28004470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:26:47.212 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:47.214+0000 7f1b32000700 1 -- 172.21.15.18:0/2328613819 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1b2800a760 con 0x7f1b3c0fe830 2024-04-12T21:26:47.212 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:47.214+0000 7f1b32000700 1 -- 172.21.15.18:0/2328613819 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f1b2800ad90 con 0x7f1b3c0fe830 2024-04-12T21:26:47.212 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:47.214+0000 7f1b40a00700 1 -- 172.21.15.18:0/2328613819 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1b3c110000 con 0x7f1b3c0fe830 2024-04-12T21:26:47.213 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:47.214+0000 7f1b32000700 1 -- 172.21.15.18:0/2328613819 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1b280113a0 con 0x7f1b3c0fe830 2024-04-12T21:26:47.213 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:47.214+0000 7f1b40a00700 1 -- 172.21.15.18:0/2328613819 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1b3c110540 con 0x7f1b3c0fe830 2024-04-12T21:26:47.213 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:47.215+0000 7f1b40a00700 1 -- 172.21.15.18:0/2328613819 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1b3c106a20 con 0x7f1b3c0fe830 2024-04-12T21:26:47.213 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:47.216+0000 7f1b32000700 1 -- 172.21.15.18:0/2328613819 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7f1b28012400 con 0x7f1b3c0fe830 2024-04-12T21:26:47.215 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:47.217+0000 7f1b32000700 1 --2- 172.21.15.18:0/2328613819 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f1b2006ec00 0x7f1b200710b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:26:47.215 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:47.218+0000 7f1b32000700 1 -- 172.21.15.18:0/2328613819 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 6757+0+0 (secure 0 0 0) 0x7f1b2808ea80 con 0x7f1b3c0fe830 2024-04-12T21:26:47.215 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:47.218+0000 7f1b39e00700 1 --2- 172.21.15.18:0/2328613819 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f1b2006ec00 0x7f1b200710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:26:47.216 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:47.218+0000 7f1b39e00700 1 --2- 172.21.15.18:0/2328613819 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f1b2006ec00 0x7f1b200710b0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f1b2c00b3b0 tx=0x7f1b2c009e30 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:26:47.220 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:47.222+0000 7f1b32000700 1 -- 172.21.15.18:0/2328613819 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f1b28057700 con 0x7f1b3c0fe830 2024-04-12T21:26:47.449 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:47.451+0000 7f1b40a00700 1 -- 172.21.15.18:0/2328613819 --> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f1b3c111480 con 0x7f1b2006ec00 2024-04-12T21:26:47.467 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:47.470+0000 7f1b32000700 1 -- 172.21.15.18:0/2328613819 <== mgr.34102 v2:172.21.15.18:6800/3812891919 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7f1b3c111480 con 0x7f1b2006ec00 2024-04-12T21:26:47.468 INFO:teuthology.orchestra.run.smithi018.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-12T21:26:47.468 INFO:teuthology.orchestra.run.smithi018.stdout:alertmanager.smithi018 smithi018 *:9093,9094 running (27m) 3m ago 41m 15.5M - 0.25.0 c8568f914cd2 c332090068e8 2024-04-12T21:26:47.468 INFO:teuthology.orchestra.run.smithi018.stdout:crash.smithi018 smithi018 running (23m) 3m ago 42m 6665k - 19.0.0-2879-g324baf8d 8717720dc5e5 b7f80d1f7982 2024-04-12T21:26:47.468 INFO:teuthology.orchestra.run.smithi018.stdout:crash.smithi186 smithi186 running (23m) 3m ago 40m 6660k - 19.0.0-2879-g324baf8d 8717720dc5e5 a4f55c81b623 2024-04-12T21:26:47.468 INFO:teuthology.orchestra.run.smithi018.stdout:grafana.smithi018 smithi018 *:3000 running (26m) 3m ago 41m 74.8M - 9.4.12 2bacad6d85d8 b08e3bf176dd 2024-04-12T21:26:47.468 INFO:teuthology.orchestra.run.smithi018.stdout:mds.cephfs.smithi018.acqbsc smithi018 running (19m) 3m ago 34m 16.7M - 19.0.0-2879-g324baf8d 8717720dc5e5 021f99ab24ae 2024-04-12T21:26:47.468 INFO:teuthology.orchestra.run.smithi018.stdout:mds.cephfs.smithi018.szjluo smithi018 running (19m) 3m ago 34m 16.5M - 19.0.0-2879-g324baf8d 8717720dc5e5 0003117479d4 2024-04-12T21:26:47.468 INFO:teuthology.orchestra.run.smithi018.stdout:mds.cephfs.smithi186.jdllyn smithi186 running (3m) 3m ago 34m 11.0M - 19.0.0-2879-g324baf8d 8717720dc5e5 21b1f46eef23 2024-04-12T21:26:47.468 INFO:teuthology.orchestra.run.smithi018.stdout:mds.cephfs.smithi186.wjnafs smithi186 running (34m) 3m ago 34m 19.5M - 17.2.7-827-gd3b5060a 21de9b7369ad 74b115d6d6fd 2024-04-12T21:26:47.468 INFO:teuthology.orchestra.run.smithi018.stdout:mgr.smithi018.xjzwga smithi018 *:8443,9283,8765 running (30m) 3m ago 43m 512M - 19.0.0-2879-g324baf8d 8717720dc5e5 56088305eab7 2024-04-12T21:26:47.468 INFO:teuthology.orchestra.run.smithi018.stdout:mgr.smithi186.kqcfxk smithi186 *:8443,9283,8765 running (28m) 3m ago 40m 459M - 19.0.0-2879-g324baf8d 8717720dc5e5 09cd766e5e0d 2024-04-12T21:26:47.469 INFO:teuthology.orchestra.run.smithi018.stdout:mon.smithi018 smithi018 running (25m) 3m ago 43m 73.1M 2048M 19.0.0-2879-g324baf8d 8717720dc5e5 df3e201dfed6 2024-04-12T21:26:47.469 INFO:teuthology.orchestra.run.smithi018.stdout:mon.smithi186 smithi186 running (24m) 3m ago 40m 47.2M 2048M 19.0.0-2879-g324baf8d 8717720dc5e5 c77498da9b2b 2024-04-12T21:26:47.469 INFO:teuthology.orchestra.run.smithi018.stdout:node-exporter.smithi018 smithi018 *:9100 running (28m) 3m ago 42m 8108k - 1.5.0 0da6a335fe13 326c3d9c6847 2024-04-12T21:26:47.469 INFO:teuthology.orchestra.run.smithi018.stdout:node-exporter.smithi186 smithi186 *:9100 running (28m) 3m ago 40m 8757k - 1.5.0 0da6a335fe13 ec2112baeecf 2024-04-12T21:26:47.469 INFO:teuthology.orchestra.run.smithi018.stdout:osd.0 smithi018 running (23m) 3m ago 39m 123M 1559M 19.0.0-2879-g324baf8d 8717720dc5e5 60b1273ca047 2024-04-12T21:26:47.469 INFO:teuthology.orchestra.run.smithi018.stdout:osd.1 smithi018 running (22m) 3m ago 38m 95.1M 1559M 19.0.0-2879-g324baf8d 8717720dc5e5 170ce1e5fb8d 2024-04-12T21:26:47.469 INFO:teuthology.orchestra.run.smithi018.stdout:osd.2 smithi018 running (21m) 3m ago 38m 94.3M 1559M 19.0.0-2879-g324baf8d 8717720dc5e5 c2cbc6aa2839 2024-04-12T21:26:47.469 INFO:teuthology.orchestra.run.smithi018.stdout:osd.3 smithi186 running (21m) 3m ago 37m 101M 2581M 19.0.0-2879-g324baf8d 8717720dc5e5 0138e8262e4b 2024-04-12T21:26:47.469 INFO:teuthology.orchestra.run.smithi018.stdout:osd.4 smithi186 running (20m) 3m ago 37m 91.0M 2581M 19.0.0-2879-g324baf8d 8717720dc5e5 b20cb63b25c9 2024-04-12T21:26:47.469 INFO:teuthology.orchestra.run.smithi018.stdout:osd.5 smithi186 running (20m) 3m ago 36m 92.0M 2581M 19.0.0-2879-g324baf8d 8717720dc5e5 2513bea2610e 2024-04-12T21:26:47.469 INFO:teuthology.orchestra.run.smithi018.stdout:prometheus.smithi018 smithi018 *:9095 running (27m) 3m ago 41m 67.9M - 2.43.0 a07b618ecd1d f58841127e21 2024-04-12T21:26:47.472 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:47.473+0000 7f1b40a00700 1 -- 172.21.15.18:0/2328613819 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f1b2006ec00 msgr2=0x7f1b200710b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:26:47.472 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:47.473+0000 7f1b40a00700 1 --2- 172.21.15.18:0/2328613819 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f1b2006ec00 0x7f1b200710b0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f1b2c00b3b0 tx=0x7f1b2c009e30 comp rx=0 tx=0).stop 2024-04-12T21:26:47.472 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:47.473+0000 7f1b40a00700 1 -- 172.21.15.18:0/2328613819 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1b3c0fe830 msgr2=0x7f1b3c1162c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:26:47.472 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:47.474+0000 7f1b40a00700 1 --2- 172.21.15.18:0/2328613819 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1b3c0fe830 0x7f1b3c1162c0 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f1b2800c830 tx=0x7f1b28004470 comp rx=0 tx=0).stop 2024-04-12T21:26:47.472 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:47.474+0000 7f1b40a00700 1 -- 172.21.15.18:0/2328613819 shutdown_connections 2024-04-12T21:26:47.472 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:47.474+0000 7f1b40a00700 1 --2- 172.21.15.18:0/2328613819 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f1b2006ec00 0x7f1b200710b0 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:26:47.472 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:47.474+0000 7f1b40a00700 1 --2- 172.21.15.18:0/2328613819 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1b3c0fde90 0x7f1b3c115d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:26:47.473 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:47.474+0000 7f1b40a00700 1 --2- 172.21.15.18:0/2328613819 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1b3c0fe830 0x7f1b3c1162c0 unknown :-1 s=CLOSED pgs=687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:26:47.473 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:47.474+0000 7f1b40a00700 1 -- 172.21.15.18:0/2328613819 >> 172.21.15.18:0/2328613819 conn(0x7f1b3c072bc0 msgr2=0x7f1b3c0fb840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:26:47.473 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:47.474+0000 7f1b40a00700 1 -- 172.21.15.18:0/2328613819 shutdown_connections 2024-04-12T21:26:47.473 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:47.474+0000 7f1b40a00700 1 -- 172.21.15.18:0/2328613819 wait complete. 2024-04-12T21:26:47.627 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:47.629+0000 7ff3de400700 1 -- 172.21.15.18:0/3091617190 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff3d8070c10 msgr2=0x7ff3d8109190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:26:47.628 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:47.629+0000 7ff3de400700 1 --2- 172.21.15.18:0/3091617190 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff3d8070c10 0x7ff3d8109190 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7ff3c4004970 tx=0x7ff3c4009e30 comp rx=0 tx=0).stop 2024-04-12T21:26:47.628 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:47.629+0000 7ff3de400700 1 -- 172.21.15.18:0/3091617190 shutdown_connections 2024-04-12T21:26:47.628 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:47.629+0000 7ff3de400700 1 --2- 172.21.15.18:0/3091617190 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff3d8070c10 0x7ff3d8109190 unknown :-1 s=CLOSED pgs=688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:26:47.628 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:47.629+0000 7ff3de400700 1 --2- 172.21.15.18:0/3091617190 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff3d8070300 0x7ff3d80706d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:26:47.628 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:47.630+0000 7ff3de400700 1 -- 172.21.15.18:0/3091617190 >> 172.21.15.18:0/3091617190 conn(0x7ff3d80f8970 msgr2=0x7ff3d80fad80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:26:47.628 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:47.630+0000 7ff3de400700 1 -- 172.21.15.18:0/3091617190 shutdown_connections 2024-04-12T21:26:47.628 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:47.630+0000 7ff3de400700 1 -- 172.21.15.18:0/3091617190 wait complete. 2024-04-12T21:26:47.629 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:47.631+0000 7ff3de400700 1 Processor -- start 2024-04-12T21:26:47.629 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:47.631+0000 7ff3de400700 1 -- start start 2024-04-12T21:26:47.629 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:47.631+0000 7ff3de400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff3d8070300 0x7ff3d8115e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:26:47.629 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:47.631+0000 7ff3de400700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff3d8070c10 0x7ff3d8116350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:26:47.629 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:47.631+0000 7ff3de400700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3d8116a30 con 0x7ff3d8070300 2024-04-12T21:26:47.629 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:47.631+0000 7ff3de400700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3d810fd70 con 0x7ff3d8070c10 2024-04-12T21:26:47.629 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:47.631+0000 7ff3d7400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff3d8070300 0x7ff3d8115e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:26:47.629 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:47.632+0000 7ff3d7400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff3d8070300 0x7ff3d8115e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:39790/0 (socket says 172.21.15.18:39790) 2024-04-12T21:26:47.629 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:47.632+0000 7ff3d7400700 1 -- 172.21.15.18:0/853839581 learned_addr learned my addr 172.21.15.18:0/853839581 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:26:47.630 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:47.632+0000 7ff3d6a00700 1 --2- 172.21.15.18:0/853839581 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff3d8070c10 0x7ff3d8116350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:26:47.630 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:47.632+0000 7ff3d7400700 1 -- 172.21.15.18:0/853839581 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff3d8070c10 msgr2=0x7ff3d8116350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:26:47.630 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:47.632+0000 7ff3d7400700 1 --2- 172.21.15.18:0/853839581 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff3d8070c10 0x7ff3d8116350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:26:47.631 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:47.632+0000 7ff3d7400700 1 -- 172.21.15.18:0/853839581 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3c4005040 con 0x7ff3d8070300 2024-04-12T21:26:47.631 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:47.632+0000 7ff3d7400700 1 --2- 172.21.15.18:0/853839581 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff3d8070300 0x7ff3d8115e10 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7ff3c800b790 tx=0x7ff3c8009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:26:47.631 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:47.633+0000 7ff3cf400700 1 -- 172.21.15.18:0/853839581 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff3c800f770 con 0x7ff3d8070300 2024-04-12T21:26:47.631 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:47.633+0000 7ff3cf400700 1 -- 172.21.15.18:0/853839581 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff3c800fda0 con 0x7ff3d8070300 2024-04-12T21:26:47.631 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:47.633+0000 7ff3de400700 1 -- 172.21.15.18:0/853839581 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff3d810fff0 con 0x7ff3d8070300 2024-04-12T21:26:47.631 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:47.633+0000 7ff3cf400700 1 -- 172.21.15.18:0/853839581 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff3c800d310 con 0x7ff3d8070300 2024-04-12T21:26:47.631 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:47.633+0000 7ff3de400700 1 -- 172.21.15.18:0/853839581 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff3d8110530 con 0x7ff3d8070300 2024-04-12T21:26:47.631 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:47.634+0000 7ff3de400700 1 -- 172.21.15.18:0/853839581 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff3d8071a40 con 0x7ff3d8070300 2024-04-12T21:26:47.632 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:47.635+0000 7ff3cf400700 1 -- 172.21.15.18:0/853839581 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7ff3c8010420 con 0x7ff3d8070300 2024-04-12T21:26:47.633 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:47.636+0000 7ff3cf400700 1 --2- 172.21.15.18:0/853839581 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7ff3bc06ec10 0x7ff3bc0710c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:26:47.633 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:47.636+0000 7ff3cf400700 1 -- 172.21.15.18:0/853839581 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 6757+0+0 (secure 0 0 0) 0x7ff3c808e9a0 con 0x7ff3d8070300 2024-04-12T21:26:47.634 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:47.636+0000 7ff3d6a00700 1 --2- 172.21.15.18:0/853839581 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7ff3bc06ec10 0x7ff3bc0710c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:26:47.634 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:47.637+0000 7ff3d6a00700 1 --2- 172.21.15.18:0/853839581 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7ff3bc06ec10 0x7ff3bc0710c0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7ff3c4000c00 tx=0x7ff3c4009e30 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:26:47.638 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:47.641+0000 7ff3cf400700 1 -- 172.21.15.18:0/853839581 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7ff3c805b1d0 con 0x7ff3d8070300 2024-04-12T21:26:47.959 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:47.962+0000 7ff3de400700 1 -- 172.21.15.18:0/853839581 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7ff3d8111460 con 0x7ff3d8070300 2024-04-12T21:26:47.960 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:26:47 smithi018 ceph-mon[112590]: from='client.35428 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:26:47.960 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:47.963+0000 7ff3cf400700 1 -- 172.21.15.18:0/853839581 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+837 (secure 0 0 0) 0x7ff3c805ad60 con 0x7ff3d8070300 2024-04-12T21:26:47.961 INFO:teuthology.orchestra.run.smithi018.stdout:{ 2024-04-12T21:26:47.961 INFO:teuthology.orchestra.run.smithi018.stdout: "mon": { 2024-04-12T21:26:47.961 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 19.0.0-2879-g324baf8d (324baf8ddb91dea46cac50cb535c306115791355) squid (dev)": 2 2024-04-12T21:26:47.961 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-12T21:26:47.961 INFO:teuthology.orchestra.run.smithi018.stdout: "mgr": { 2024-04-12T21:26:47.961 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 19.0.0-2879-g324baf8d (324baf8ddb91dea46cac50cb535c306115791355) squid (dev)": 2 2024-04-12T21:26:47.961 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-12T21:26:47.961 INFO:teuthology.orchestra.run.smithi018.stdout: "osd": { 2024-04-12T21:26:47.961 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 19.0.0-2879-g324baf8d (324baf8ddb91dea46cac50cb535c306115791355) squid (dev)": 6 2024-04-12T21:26:47.961 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-12T21:26:47.961 INFO:teuthology.orchestra.run.smithi018.stdout: "mds": { 2024-04-12T21:26:47.962 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 17.2.7-827-gd3b5060a (d3b5060a17449edfae9385cdf792afec109fca5d) quincy (stable)": 1, 2024-04-12T21:26:47.962 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 19.0.0-2879-g324baf8d (324baf8ddb91dea46cac50cb535c306115791355) squid (dev)": 3 2024-04-12T21:26:47.962 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-12T21:26:47.962 INFO:teuthology.orchestra.run.smithi018.stdout: "overall": { 2024-04-12T21:26:47.962 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 17.2.7-827-gd3b5060a (d3b5060a17449edfae9385cdf792afec109fca5d) quincy (stable)": 1, 2024-04-12T21:26:47.962 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 19.0.0-2879-g324baf8d (324baf8ddb91dea46cac50cb535c306115791355) squid (dev)": 13 2024-04-12T21:26:47.962 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-04-12T21:26:47.962 INFO:teuthology.orchestra.run.smithi018.stdout:} 2024-04-12T21:26:47.963 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:47.966+0000 7ff3de400700 1 -- 172.21.15.18:0/853839581 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7ff3bc06ec10 msgr2=0x7ff3bc0710c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:26:47.964 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:47.966+0000 7ff3de400700 1 --2- 172.21.15.18:0/853839581 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7ff3bc06ec10 0x7ff3bc0710c0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7ff3c4000c00 tx=0x7ff3c4009e30 comp rx=0 tx=0).stop 2024-04-12T21:26:47.964 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:47.966+0000 7ff3de400700 1 -- 172.21.15.18:0/853839581 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff3d8070300 msgr2=0x7ff3d8115e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:26:47.964 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:47.966+0000 7ff3de400700 1 --2- 172.21.15.18:0/853839581 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff3d8070300 0x7ff3d8115e10 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7ff3c800b790 tx=0x7ff3c8009250 comp rx=0 tx=0).stop 2024-04-12T21:26:47.964 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:47.966+0000 7ff3de400700 1 -- 172.21.15.18:0/853839581 shutdown_connections 2024-04-12T21:26:47.964 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:47.966+0000 7ff3de400700 1 --2- 172.21.15.18:0/853839581 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7ff3bc06ec10 0x7ff3bc0710c0 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:26:47.964 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:47.966+0000 7ff3de400700 1 --2- 172.21.15.18:0/853839581 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff3d8070300 0x7ff3d8115e10 unknown :-1 s=CLOSED pgs=689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:26:47.964 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:47.966+0000 7ff3de400700 1 --2- 172.21.15.18:0/853839581 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff3d8070c10 0x7ff3d8116350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:26:47.964 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:47.966+0000 7ff3de400700 1 -- 172.21.15.18:0/853839581 >> 172.21.15.18:0/853839581 conn(0x7ff3d80f8970 msgr2=0x7ff3d80f9d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:26:47.964 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:47.966+0000 7ff3de400700 1 -- 172.21.15.18:0/853839581 shutdown_connections 2024-04-12T21:26:47.964 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:47.966+0000 7ff3de400700 1 -- 172.21.15.18:0/853839581 wait complete. 2024-04-12T21:26:48.030 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:26:47 smithi186 ceph-mon[86379]: from='client.35428 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:26:48.121 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:48.123+0000 7fd486600700 1 -- 172.21.15.18:0/132995208 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd4801057d0 msgr2=0x7fd4800fee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:26:48.121 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:48.123+0000 7fd486600700 1 --2- 172.21.15.18:0/132995208 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd4801057d0 0x7fd4800fee40 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7fd46c004940 tx=0x7fd46c009e30 comp rx=0 tx=0).stop 2024-04-12T21:26:48.121 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:48.124+0000 7fd486600700 1 -- 172.21.15.18:0/132995208 shutdown_connections 2024-04-12T21:26:48.121 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:48.124+0000 7fd486600700 1 --2- 172.21.15.18:0/132995208 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd4801057d0 0x7fd4800fee40 unknown :-1 s=CLOSED pgs=690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:26:48.121 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:48.124+0000 7fd486600700 1 --2- 172.21.15.18:0/132995208 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd480104e30 0x7fd480105200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:26:48.121 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:48.124+0000 7fd486600700 1 -- 172.21.15.18:0/132995208 >> 172.21.15.18:0/132995208 conn(0x7fd4800fab10 msgr2=0x7fd4800fcef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:26:48.122 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:48.124+0000 7fd486600700 1 -- 172.21.15.18:0/132995208 shutdown_connections 2024-04-12T21:26:48.122 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:48.124+0000 7fd486600700 1 -- 172.21.15.18:0/132995208 wait complete. 2024-04-12T21:26:48.122 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:48.124+0000 7fd486600700 1 Processor -- start 2024-04-12T21:26:48.122 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:48.125+0000 7fd486600700 1 -- start start 2024-04-12T21:26:48.123 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:48.125+0000 7fd486600700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd480104e30 0x7fd480115ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:26:48.123 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:48.125+0000 7fd486600700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd4801057d0 0x7fd4801160e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:26:48.124 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:48.125+0000 7fd486600700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd480116730 con 0x7fd4801057d0 2024-04-12T21:26:48.124 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:48.125+0000 7fd486600700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd48010fb20 con 0x7fd480104e30 2024-04-12T21:26:48.124 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:48.125+0000 7fd47fe00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd4801057d0 0x7fd4801160e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:26:48.124 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:48.126+0000 7fd47fe00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd4801057d0 0x7fd4801160e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:39810/0 (socket says 172.21.15.18:39810) 2024-04-12T21:26:48.124 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:48.126+0000 7fd47fe00700 1 -- 172.21.15.18:0/1030845450 learned_addr learned my addr 172.21.15.18:0/1030845450 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:26:48.124 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:48.126+0000 7fd47fe00700 1 -- 172.21.15.18:0/1030845450 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd480104e30 msgr2=0x7fd480115ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:26:48.124 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:48.126+0000 7fd485200700 1 --2- 172.21.15.18:0/1030845450 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd480104e30 0x7fd480115ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:26:48.124 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:48.126+0000 7fd47fe00700 1 --2- 172.21.15.18:0/1030845450 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd480104e30 0x7fd480115ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:26:48.124 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:48.126+0000 7fd47fe00700 1 -- 172.21.15.18:0/1030845450 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd46c005040 con 0x7fd4801057d0 2024-04-12T21:26:48.124 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:48.126+0000 7fd485200700 1 --2- 172.21.15.18:0/1030845450 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd480104e30 0x7fd480115ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-12T21:26:48.124 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:48.126+0000 7fd47fe00700 1 --2- 172.21.15.18:0/1030845450 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd4801057d0 0x7fd4801160e0 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7fd46c009100 tx=0x7fd46c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:26:48.125 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:48.127+0000 7fd47cc00700 1 -- 172.21.15.18:0/1030845450 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd46c00e040 con 0x7fd4801057d0 2024-04-12T21:26:48.125 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:48.127+0000 7fd47cc00700 1 -- 172.21.15.18:0/1030845450 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd46c004290 con 0x7fd4801057d0 2024-04-12T21:26:48.125 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:48.127+0000 7fd486600700 1 -- 172.21.15.18:0/1030845450 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd48010fda0 con 0x7fd4801057d0 2024-04-12T21:26:48.125 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:48.127+0000 7fd47cc00700 1 -- 172.21.15.18:0/1030845450 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd46c01c3f0 con 0x7fd4801057d0 2024-04-12T21:26:48.125 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:48.127+0000 7fd486600700 1 -- 172.21.15.18:0/1030845450 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd480110360 con 0x7fd4801057d0 2024-04-12T21:26:48.126 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:48.128+0000 7fd486600700 1 -- 172.21.15.18:0/1030845450 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd480107300 con 0x7fd4801057d0 2024-04-12T21:26:48.126 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:48.129+0000 7fd47cc00700 1 -- 172.21.15.18:0/1030845450 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7fd46c013790 con 0x7fd4801057d0 2024-04-12T21:26:48.128 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:48.130+0000 7fd47cc00700 1 --2- 172.21.15.18:0/1030845450 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fd46806ec00 0x7fd4680710b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:26:48.128 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:48.131+0000 7fd47cc00700 1 -- 172.21.15.18:0/1030845450 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 6757+0+0 (secure 0 0 0) 0x7fd46c08fd10 con 0x7fd4801057d0 2024-04-12T21:26:48.128 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:48.131+0000 7fd485200700 1 --2- 172.21.15.18:0/1030845450 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fd46806ec00 0x7fd4680710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:26:48.129 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:48.132+0000 7fd485200700 1 --2- 172.21.15.18:0/1030845450 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fd46806ec00 0x7fd4680710b0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fd474007fd0 tx=0x7fd474009040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:26:48.133 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:48.136+0000 7fd47cc00700 1 -- 172.21.15.18:0/1030845450 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7fd46c05c4c0 con 0x7fd4801057d0 2024-04-12T21:26:48.416 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:48.418+0000 7fd486600700 1 -- 172.21.15.18:0/1030845450 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fd48010bf10 con 0x7fd4801057d0 2024-04-12T21:26:48.417 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:48.419+0000 7fd47cc00700 1 -- 172.21.15.18:0/1030845450 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 31 v31) v1 ==== 76+0+1873 (secure 0 0 0) 0x7fd46c013420 con 0x7fd4801057d0 2024-04-12T21:26:48.417 INFO:teuthology.orchestra.run.smithi018.stdout:e31 2024-04-12T21:26:48.418 INFO:teuthology.orchestra.run.smithi018.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-12T21:26:48.418 INFO:teuthology.orchestra.run.smithi018.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-12T21:26:48.418 INFO:teuthology.orchestra.run.smithi018.stdout:legacy client fscid: 1 2024-04-12T21:26:48.418 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-12T21:26:48.418 INFO:teuthology.orchestra.run.smithi018.stdout:Filesystem 'cephfs' (1) 2024-04-12T21:26:48.418 INFO:teuthology.orchestra.run.smithi018.stdout:fs_name cephfs 2024-04-12T21:26:48.418 INFO:teuthology.orchestra.run.smithi018.stdout:epoch 24 2024-04-12T21:26:48.418 INFO:teuthology.orchestra.run.smithi018.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-04-12T21:26:48.418 INFO:teuthology.orchestra.run.smithi018.stdout:created 2024-04-12T20:51:57.362850+0000 2024-04-12T21:26:48.418 INFO:teuthology.orchestra.run.smithi018.stdout:modified 2024-04-12T21:06:59.717190+0000 2024-04-12T21:26:48.418 INFO:teuthology.orchestra.run.smithi018.stdout:tableserver 0 2024-04-12T21:26:48.418 INFO:teuthology.orchestra.run.smithi018.stdout:root 0 2024-04-12T21:26:48.418 INFO:teuthology.orchestra.run.smithi018.stdout:session_timeout 60 2024-04-12T21:26:48.419 INFO:teuthology.orchestra.run.smithi018.stdout:session_autoclose 300 2024-04-12T21:26:48.419 INFO:teuthology.orchestra.run.smithi018.stdout:max_file_size 1099511627776 2024-04-12T21:26:48.419 INFO:teuthology.orchestra.run.smithi018.stdout:max_xattr_size 65536 2024-04-12T21:26:48.419 INFO:teuthology.orchestra.run.smithi018.stdout:required_client_features {} 2024-04-12T21:26:48.419 INFO:teuthology.orchestra.run.smithi018.stdout:last_failure 0 2024-04-12T21:26:48.419 INFO:teuthology.orchestra.run.smithi018.stdout:last_failure_osd_epoch 100 2024-04-12T21:26:48.419 INFO:teuthology.orchestra.run.smithi018.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-12T21:26:48.419 INFO:teuthology.orchestra.run.smithi018.stdout:max_mds 2 2024-04-12T21:26:48.419 INFO:teuthology.orchestra.run.smithi018.stdout:in 0,1 2024-04-12T21:26:48.419 INFO:teuthology.orchestra.run.smithi018.stdout:up {} 2024-04-12T21:26:48.419 INFO:teuthology.orchestra.run.smithi018.stdout:failed 0,1 2024-04-12T21:26:48.419 INFO:teuthology.orchestra.run.smithi018.stdout:damaged 2024-04-12T21:26:48.419 INFO:teuthology.orchestra.run.smithi018.stdout:stopped 2024-04-12T21:26:48.419 INFO:teuthology.orchestra.run.smithi018.stdout:data_pools [3] 2024-04-12T21:26:48.419 INFO:teuthology.orchestra.run.smithi018.stdout:metadata_pool 2 2024-04-12T21:26:48.419 INFO:teuthology.orchestra.run.smithi018.stdout:inline_data disabled 2024-04-12T21:26:48.420 INFO:teuthology.orchestra.run.smithi018.stdout:balancer 2024-04-12T21:26:48.420 INFO:teuthology.orchestra.run.smithi018.stdout:bal_rank_mask -1 2024-04-12T21:26:48.420 INFO:teuthology.orchestra.run.smithi018.stdout:standby_count_wanted 1 2024-04-12T21:26:48.420 INFO:teuthology.orchestra.run.smithi018.stdout:qdb_cluster leader: 0 members: 2024-04-12T21:26:48.420 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-12T21:26:48.420 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-12T21:26:48.420 INFO:teuthology.orchestra.run.smithi018.stdout:Standby daemons: 2024-04-12T21:26:48.420 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-12T21:26:48.420 INFO:teuthology.orchestra.run.smithi018.stdout:[mds.cephfs.smithi186.wjnafs{-1:24277} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.186:6826/438347207,v1:172.21.15.186:6827/438347207] compat {c=[1],r=[1],i=[7ff]}] 2024-04-12T21:26:48.420 INFO:teuthology.orchestra.run.smithi018.stdout:[mds.cephfs.smithi018.acqbsc{-1:34436} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.18:6828/755179764,v1:172.21.15.18:6829/755179764] compat {c=[1],r=[1],i=[1fff]}] 2024-04-12T21:26:48.420 INFO:teuthology.orchestra.run.smithi018.stdout:[mds.cephfs.smithi018.szjluo{-1:34468} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.18:6826/2407992653,v1:172.21.15.18:6827/2407992653] compat {c=[1],r=[1],i=[1fff]}] 2024-04-12T21:26:48.420 INFO:teuthology.orchestra.run.smithi018.stdout:[mds.cephfs.smithi186.jdllyn{-1:44595} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.186:6824/3523444202,v1:172.21.15.186:6825/3523444202] compat {c=[1],r=[1],i=[1fff]}] 2024-04-12T21:26:48.421 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:48.422+0000 7fd486600700 1 -- 172.21.15.18:0/1030845450 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fd46806ec00 msgr2=0x7fd4680710b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:26:48.421 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:48.422+0000 7fd486600700 1 --2- 172.21.15.18:0/1030845450 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fd46806ec00 0x7fd4680710b0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fd474007fd0 tx=0x7fd474009040 comp rx=0 tx=0).stop 2024-04-12T21:26:48.421 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:48.422+0000 7fd486600700 1 -- 172.21.15.18:0/1030845450 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd4801057d0 msgr2=0x7fd4801160e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:26:48.421 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:48.422+0000 7fd486600700 1 --2- 172.21.15.18:0/1030845450 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd4801057d0 0x7fd4801160e0 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7fd46c009100 tx=0x7fd46c009e30 comp rx=0 tx=0).stop 2024-04-12T21:26:48.421 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:48.422+0000 7fd486600700 1 -- 172.21.15.18:0/1030845450 shutdown_connections 2024-04-12T21:26:48.421 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:48.422+0000 7fd486600700 1 --2- 172.21.15.18:0/1030845450 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fd46806ec00 0x7fd4680710b0 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:26:48.421 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:48.422+0000 7fd486600700 1 --2- 172.21.15.18:0/1030845450 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd480104e30 0x7fd480115ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:26:48.421 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:48.422+0000 7fd486600700 1 --2- 172.21.15.18:0/1030845450 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd4801057d0 0x7fd4801160e0 unknown :-1 s=CLOSED pgs=691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:26:48.422 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:48.423+0000 7fd486600700 1 -- 172.21.15.18:0/1030845450 >> 172.21.15.18:0/1030845450 conn(0x7fd4800fab10 msgr2=0x7fd4800fda00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:26:48.422 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:48.423+0000 7fd486600700 1 -- 172.21.15.18:0/1030845450 shutdown_connections 2024-04-12T21:26:48.422 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:48.423+0000 7fd486600700 1 -- 172.21.15.18:0/1030845450 wait complete. 2024-04-12T21:26:48.422 INFO:teuthology.orchestra.run.smithi018.stderr:dumped fsmap epoch 31 2024-04-12T21:26:48.573 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:48.575+0000 7f8afd800700 1 -- 172.21.15.18:0/1992824788 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8af80fba90 msgr2=0x7f8af80fbe60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:26:48.573 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:48.575+0000 7f8afd800700 1 --2- 172.21.15.18:0/1992824788 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8af80fba90 0x7f8af80fbe60 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7f8ae8004940 tx=0x7f8ae8009e30 comp rx=0 tx=0).stop 2024-04-12T21:26:48.573 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:48.575+0000 7f8afd800700 1 -- 172.21.15.18:0/1992824788 shutdown_connections 2024-04-12T21:26:48.573 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:48.575+0000 7f8afd800700 1 --2- 172.21.15.18:0/1992824788 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8af80fc430 0x7f8af8100a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:26:48.573 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:48.575+0000 7f8afd800700 1 --2- 172.21.15.18:0/1992824788 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8af80fba90 0x7f8af80fbe60 unknown :-1 s=CLOSED pgs=692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:26:48.573 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:48.575+0000 7f8afd800700 1 -- 172.21.15.18:0/1992824788 >> 172.21.15.18:0/1992824788 conn(0x7f8af80f7710 msgr2=0x7f8af80f9b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:26:48.573 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:48.575+0000 7f8afd800700 1 -- 172.21.15.18:0/1992824788 shutdown_connections 2024-04-12T21:26:48.573 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:48.575+0000 7f8afd800700 1 -- 172.21.15.18:0/1992824788 wait complete. 2024-04-12T21:26:48.575 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:48.576+0000 7f8afd800700 1 Processor -- start 2024-04-12T21:26:48.575 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:48.576+0000 7f8afd800700 1 -- start start 2024-04-12T21:26:48.575 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:48.577+0000 7f8afd800700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8af80fba90 0x7f8af8119fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:26:48.575 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:48.577+0000 7f8afd800700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8af80fc430 0x7f8af811a520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:26:48.575 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:48.577+0000 7f8afd800700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8af811ab70 con 0x7f8af80fba90 2024-04-12T21:26:48.575 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:48.577+0000 7f8afd800700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8af8113ff0 con 0x7f8af80fc430 2024-04-12T21:26:48.575 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:48.577+0000 7f8af7e00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8af80fba90 0x7f8af8119fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:26:48.575 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:48.577+0000 7f8af7e00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8af80fba90 0x7f8af8119fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:39836/0 (socket says 172.21.15.18:39836) 2024-04-12T21:26:48.576 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:48.577+0000 7f8af7e00700 1 -- 172.21.15.18:0/3609724503 learned_addr learned my addr 172.21.15.18:0/3609724503 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:26:48.576 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:48.577+0000 7f8af7400700 1 --2- 172.21.15.18:0/3609724503 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8af80fc430 0x7f8af811a520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:26:48.576 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:48.577+0000 7f8af7e00700 1 -- 172.21.15.18:0/3609724503 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8af80fc430 msgr2=0x7f8af811a520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:26:48.576 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:48.577+0000 7f8af7e00700 1 --2- 172.21.15.18:0/3609724503 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8af80fc430 0x7f8af811a520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:26:48.576 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:48.577+0000 7f8af7e00700 1 -- 172.21.15.18:0/3609724503 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ae8005040 con 0x7f8af80fba90 2024-04-12T21:26:48.576 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:48.578+0000 7f8af7e00700 1 --2- 172.21.15.18:0/3609724503 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8af80fba90 0x7f8af8119fe0 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7f8ae8009150 tx=0x7f8ae8009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:26:48.577 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:48.578+0000 7f8aefe00700 1 -- 172.21.15.18:0/3609724503 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8ae800e040 con 0x7f8af80fba90 2024-04-12T21:26:48.577 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:48.578+0000 7f8afd800700 1 -- 172.21.15.18:0/3609724503 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8af8114270 con 0x7f8af80fba90 2024-04-12T21:26:48.577 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:48.578+0000 7f8aefe00700 1 -- 172.21.15.18:0/3609724503 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f8ae8013420 con 0x7f8af80fba90 2024-04-12T21:26:48.577 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:48.578+0000 7f8aefe00700 1 -- 172.21.15.18:0/3609724503 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8ae801c630 con 0x7f8af80fba90 2024-04-12T21:26:48.577 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:48.578+0000 7f8afd800700 1 -- 172.21.15.18:0/3609724503 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8af81147b0 con 0x7f8af80fba90 2024-04-12T21:26:48.577 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:48.579+0000 7f8afd800700 1 -- 172.21.15.18:0/3609724503 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8af80669b0 con 0x7f8af80fba90 2024-04-12T21:26:48.577 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:48.580+0000 7f8aefe00700 1 -- 172.21.15.18:0/3609724503 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7f8ae8013590 con 0x7f8af80fba90 2024-04-12T21:26:48.579 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:48.581+0000 7f8aefe00700 1 --2- 172.21.15.18:0/3609724503 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f8adc06ec00 0x7f8adc0710b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:26:48.579 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:48.581+0000 7f8aefe00700 1 -- 172.21.15.18:0/3609724503 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 6757+0+0 (secure 0 0 0) 0x7f8ae808fc10 con 0x7f8af80fba90 2024-04-12T21:26:48.579 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:48.581+0000 7f8af7400700 1 --2- 172.21.15.18:0/3609724503 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f8adc06ec00 0x7f8adc0710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:26:48.579 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:48.582+0000 7f8af7400700 1 --2- 172.21.15.18:0/3609724503 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f8adc06ec00 0x7f8adc0710b0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f8af810dc30 tx=0x7f8ae400b040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:26:48.583 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:48.586+0000 7f8aefe00700 1 -- 172.21.15.18:0/3609724503 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f8ae8058910 con 0x7f8af80fba90 2024-04-12T21:26:48.826 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:48.828+0000 7f8afd800700 1 -- 172.21.15.18:0/3609724503 --> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f8af81103a0 con 0x7f8adc06ec00 2024-04-12T21:26:48.828 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:48.831+0000 7f8aefe00700 1 -- 172.21.15.18:0/3609724503 <== mgr.34102 v2:172.21.15.18:6800/3812891919 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f8af81103a0 con 0x7f8adc06ec00 2024-04-12T21:26:48.829 INFO:teuthology.orchestra.run.smithi018.stdout:{ 2024-04-12T21:26:48.829 INFO:teuthology.orchestra.run.smithi018.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:f4452def3be4d350f4a8e5feaf7ebb1511be9298fe68478ba7a2d6fbd7d69a63", 2024-04-12T21:26:48.829 INFO:teuthology.orchestra.run.smithi018.stdout: "in_progress": true, 2024-04-12T21:26:48.830 INFO:teuthology.orchestra.run.smithi018.stdout: "which": "Upgrading all daemon types on all hosts", 2024-04-12T21:26:48.830 INFO:teuthology.orchestra.run.smithi018.stdout: "services_complete": [ 2024-04-12T21:26:48.830 INFO:teuthology.orchestra.run.smithi018.stdout: "crash", 2024-04-12T21:26:48.830 INFO:teuthology.orchestra.run.smithi018.stdout: "osd", 2024-04-12T21:26:48.830 INFO:teuthology.orchestra.run.smithi018.stdout: "mon", 2024-04-12T21:26:48.830 INFO:teuthology.orchestra.run.smithi018.stdout: "mgr" 2024-04-12T21:26:48.830 INFO:teuthology.orchestra.run.smithi018.stdout: ], 2024-04-12T21:26:48.830 INFO:teuthology.orchestra.run.smithi018.stdout: "progress": "15/21 daemons upgraded", 2024-04-12T21:26:48.830 INFO:teuthology.orchestra.run.smithi018.stdout: "message": "Currently upgrading mds daemons", 2024-04-12T21:26:48.830 INFO:teuthology.orchestra.run.smithi018.stdout: "is_paused": false 2024-04-12T21:26:48.830 INFO:teuthology.orchestra.run.smithi018.stdout:} 2024-04-12T21:26:48.831 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:48.833+0000 7f8afd800700 1 -- 172.21.15.18:0/3609724503 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f8adc06ec00 msgr2=0x7f8adc0710b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:26:48.831 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:48.833+0000 7f8afd800700 1 --2- 172.21.15.18:0/3609724503 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f8adc06ec00 0x7f8adc0710b0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f8af810dc30 tx=0x7f8ae400b040 comp rx=0 tx=0).stop 2024-04-12T21:26:48.832 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:48.833+0000 7f8afd800700 1 -- 172.21.15.18:0/3609724503 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8af80fba90 msgr2=0x7f8af8119fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:26:48.832 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:48.833+0000 7f8afd800700 1 --2- 172.21.15.18:0/3609724503 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8af80fba90 0x7f8af8119fe0 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7f8ae8009150 tx=0x7f8ae8009e30 comp rx=0 tx=0).stop 2024-04-12T21:26:48.832 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:48.834+0000 7f8afd800700 1 -- 172.21.15.18:0/3609724503 shutdown_connections 2024-04-12T21:26:48.832 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:48.834+0000 7f8afd800700 1 --2- 172.21.15.18:0/3609724503 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f8adc06ec00 0x7f8adc0710b0 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:26:48.832 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:48.834+0000 7f8afd800700 1 --2- 172.21.15.18:0/3609724503 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8af80fba90 0x7f8af8119fe0 unknown :-1 s=CLOSED pgs=693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:26:48.832 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:48.834+0000 7f8afd800700 1 --2- 172.21.15.18:0/3609724503 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8af80fc430 0x7f8af811a520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:26:48.832 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:48.834+0000 7f8afd800700 1 -- 172.21.15.18:0/3609724503 >> 172.21.15.18:0/3609724503 conn(0x7f8af80f7710 msgr2=0x7f8af810a830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:26:48.832 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:48.834+0000 7f8afd800700 1 -- 172.21.15.18:0/3609724503 shutdown_connections 2024-04-12T21:26:48.832 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:48.834+0000 7f8afd800700 1 -- 172.21.15.18:0/3609724503 wait complete. 2024-04-12T21:26:48.977 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:48.979+0000 7f3b63000700 1 -- 172.21.15.18:0/1309944045 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3b5c0724f0 msgr2=0x7f3b5c0728c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:26:48.977 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:48.979+0000 7f3b63000700 1 --2- 172.21.15.18:0/1309944045 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3b5c0724f0 0x7f3b5c0728c0 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f3b4c004940 tx=0x7f3b4c009e30 comp rx=0 tx=0).stop 2024-04-12T21:26:48.977 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:48.979+0000 7f3b63000700 1 -- 172.21.15.18:0/1309944045 shutdown_connections 2024-04-12T21:26:48.977 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:48.979+0000 7f3b63000700 1 --2- 172.21.15.18:0/1309944045 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3b5c072e00 0x7f3b5c10d760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:26:48.977 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:48.979+0000 7f3b63000700 1 --2- 172.21.15.18:0/1309944045 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3b5c0724f0 0x7f3b5c0728c0 unknown :-1 s=CLOSED pgs=694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:26:48.977 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:48.979+0000 7f3b63000700 1 -- 172.21.15.18:0/1309944045 >> 172.21.15.18:0/1309944045 conn(0x7f3b5c0faba0 msgr2=0x7f3b5c0fcfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:26:48.977 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:48.979+0000 7f3b63000700 1 -- 172.21.15.18:0/1309944045 shutdown_connections 2024-04-12T21:26:48.977 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:26:48 smithi018 ceph-mon[112590]: from='client.35432 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:26:48.978 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:26:48 smithi018 ceph-mon[112590]: from='client.35436 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:26:48.978 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:26:48 smithi018 ceph-mon[112590]: pgmap v811: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:26:48.978 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:26:48 smithi018 ceph-mon[112590]: from='client.? 172.21.15.18:0/853839581' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T21:26:48.978 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:26:48 smithi018 ceph-mon[112590]: from='client.? 172.21.15.18:0/1030845450' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-12T21:26:48.980 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:48.979+0000 7f3b63000700 1 -- 172.21.15.18:0/1309944045 wait complete. 2024-04-12T21:26:48.980 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:48.980+0000 7f3b63000700 1 Processor -- start 2024-04-12T21:26:48.980 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:48.980+0000 7f3b63000700 1 -- start start 2024-04-12T21:26:48.980 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:48.980+0000 7f3b63000700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3b5c0724f0 0x7f3b5c119ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:26:48.980 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:48.980+0000 7f3b63000700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3b5c072e00 0x7f3b5c11a420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:26:48.980 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:48.980+0000 7f3b63000700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b5c11ab00 con 0x7f3b5c072e00 2024-04-12T21:26:48.980 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:48.980+0000 7f3b63000700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b5c114050 con 0x7f3b5c0724f0 2024-04-12T21:26:48.980 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:48.981+0000 7f3b53e00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3b5c072e00 0x7f3b5c11a420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:26:48.980 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:48.981+0000 7f3b53e00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3b5c072e00 0x7f3b5c11a420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:39864/0 (socket says 172.21.15.18:39864) 2024-04-12T21:26:48.980 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:48.981+0000 7f3b53e00700 1 -- 172.21.15.18:0/1912232942 learned_addr learned my addr 172.21.15.18:0/1912232942 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:26:48.980 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:48.981+0000 7f3b5be00700 1 --2- 172.21.15.18:0/1912232942 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3b5c0724f0 0x7f3b5c119ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:26:48.980 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:48.981+0000 7f3b53e00700 1 -- 172.21.15.18:0/1912232942 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3b5c0724f0 msgr2=0x7f3b5c119ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:26:48.981 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:48.981+0000 7f3b53e00700 1 --2- 172.21.15.18:0/1912232942 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3b5c0724f0 0x7f3b5c119ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:26:48.981 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:48.981+0000 7f3b53e00700 1 -- 172.21.15.18:0/1912232942 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b4c005040 con 0x7f3b5c072e00 2024-04-12T21:26:48.981 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:48.981+0000 7f3b5be00700 1 --2- 172.21.15.18:0/1912232942 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3b5c0724f0 0x7f3b5c119ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-12T21:26:48.981 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:48.982+0000 7f3b53e00700 1 --2- 172.21.15.18:0/1912232942 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3b5c072e00 0x7f3b5c11a420 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7f3b4400b790 tx=0x7f3b44009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:26:48.981 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:48.982+0000 7f3b58c00700 1 -- 172.21.15.18:0/1912232942 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3b4400f770 con 0x7f3b5c072e00 2024-04-12T21:26:48.981 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:48.982+0000 7f3b63000700 1 -- 172.21.15.18:0/1912232942 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3b5c1142d0 con 0x7f3b5c072e00 2024-04-12T21:26:48.981 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:48.982+0000 7f3b63000700 1 -- 172.21.15.18:0/1912232942 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3b5c114890 con 0x7f3b5c072e00 2024-04-12T21:26:48.981 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:48.982+0000 7f3b58c00700 1 -- 172.21.15.18:0/1912232942 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f3b4400fda0 con 0x7f3b5c072e00 2024-04-12T21:26:48.982 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:48.982+0000 7f3b58c00700 1 -- 172.21.15.18:0/1912232942 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3b44010420 con 0x7f3b5c072e00 2024-04-12T21:26:48.982 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:48.983+0000 7f3b63000700 1 -- 172.21.15.18:0/1912232942 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3b380052b0 con 0x7f3b5c072e00 2024-04-12T21:26:48.982 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:48.984+0000 7f3b58c00700 1 -- 172.21.15.18:0/1912232942 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7f3b4400f8d0 con 0x7f3b5c072e00 2024-04-12T21:26:48.983 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:48.985+0000 7f3b58c00700 1 --2- 172.21.15.18:0/1912232942 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f3b4006ec00 0x7f3b400710b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:26:48.983 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:48.986+0000 7f3b58c00700 1 -- 172.21.15.18:0/1912232942 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 6757+0+0 (secure 0 0 0) 0x7f3b4408e820 con 0x7f3b5c072e00 2024-04-12T21:26:48.983 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:48.986+0000 7f3b5be00700 1 --2- 172.21.15.18:0/1912232942 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f3b4006ec00 0x7f3b400710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:26:48.984 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:48.986+0000 7f3b5be00700 1 --2- 172.21.15.18:0/1912232942 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f3b4006ec00 0x7f3b400710b0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f3b4c004f90 tx=0x7f3b4c009e30 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:26:48.988 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:48.991+0000 7f3b58c00700 1 -- 172.21.15.18:0/1912232942 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f3b4405b110 con 0x7f3b5c072e00 2024-04-12T21:26:49.030 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:26:48 smithi186 ceph-mon[86379]: from='client.35432 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:26:49.030 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:26:48 smithi186 ceph-mon[86379]: from='client.35436 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:26:49.030 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:26:48 smithi186 ceph-mon[86379]: pgmap v811: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:26:49.030 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:26:48 smithi186 ceph-mon[86379]: from='client.? 172.21.15.18:0/853839581' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T21:26:49.030 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:26:48 smithi186 ceph-mon[86379]: from='client.? 172.21.15.18:0/1030845450' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-12T21:26:49.298 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:49.300+0000 7f3b63000700 1 -- 172.21.15.18:0/1912232942 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f3b38005560 con 0x7f3b5c072e00 2024-04-12T21:26:49.298 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:49.301+0000 7f3b58c00700 1 -- 172.21.15.18:0/1912232942 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+4303 (secure 0 0 0) 0x7f3b4405aca0 con 0x7f3b5c072e00 2024-04-12T21:26:49.299 INFO:teuthology.orchestra.run.smithi018.stdout:HEALTH_ERR failed to probe daemons or devices; 1 filesystem is degraded; 1 filesystem is offline 2024-04-12T21:26:49.299 INFO:teuthology.orchestra.run.smithi018.stdout:[WRN] CEPHADM_REFRESH_FAILED: failed to probe daemons or devices 2024-04-12T21:26:49.299 INFO:teuthology.orchestra.run.smithi018.stdout: host smithi018 `cephadm ceph-volume` failed: cephadm exited with an error code: 1, stderr: Inferring config /var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/mon.smithi018/config 2024-04-12T21:26:49.299 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/podman: timeout after 895 seconds 2024-04-12T21:26:49.299 INFO:teuthology.orchestra.run.smithi018.stdout:Non-zero exit code 124 from /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:f4452def3be4d350f4a8e5feaf7ebb1511be9298fe68478ba7a2d6fbd7d69a63 -e NODE_NAME=smithi018 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968:/var/run/ceph:z -v /var/log/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968:/var/log/ceph:z -v /var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmpxsmwkkuh:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:f4452def3be4d350f4a8e5feaf7ebb1511be9298fe68478ba7a2d6fbd7d69a63 inventory --format=json-pretty --filter-for-batch 2024-04-12T21:26:49.299 INFO:teuthology.orchestra.run.smithi018.stdout:Traceback (most recent call last): 2024-04-12T21:26:49.299 INFO:teuthology.orchestra.run.smithi018.stdout: File "/usr/lib64/python3.9/runpy.py", line 197, in _run_module_as_main 2024-04-12T21:26:49.299 INFO:teuthology.orchestra.run.smithi018.stdout: return _run_code(code, main_globals, None, 2024-04-12T21:26:49.299 INFO:teuthology.orchestra.run.smithi018.stdout: File "/usr/lib64/python3.9/runpy.py", line 87, in _run_code 2024-04-12T21:26:49.299 INFO:teuthology.orchestra.run.smithi018.stdout: exec(code, run_globals) 2024-04-12T21:26:49.300 INFO:teuthology.orchestra.run.smithi018.stdout: File "/var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/cephadm.e44f3ed5e34e4ef52eedef233d1190bff004470d8e8005127b5312bff357fa7f/__main__.py", line 5618, in 2024-04-12T21:26:49.300 INFO:teuthology.orchestra.run.smithi018.stdout: File "/var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/cephadm.e44f3ed5e34e4ef52eedef233d1190bff004470d8e8005127b5312bff357fa7f/__main__.py", line 5606, in main 2024-04-12T21:26:49.300 INFO:teuthology.orchestra.run.smithi018.stdout: File "/var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/cephadm.e44f3ed5e34e4ef52eedef233d1190bff004470d8e8005127b5312bff357fa7f/__main__.py", line 408, in _infer_config 2024-04-12T21:26:49.300 INFO:teuthology.orchestra.run.smithi018.stdout: File "/var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/cephadm.e44f3ed5e34e4ef52eedef233d1190bff004470d8e8005127b5312bff357fa7f/__main__.py", line 323, in _infer_fsid 2024-04-12T21:26:49.300 INFO:teuthology.orchestra.run.smithi018.stdout: File "/var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/cephadm.e44f3ed5e34e4ef52eedef233d1190bff004470d8e8005127b5312bff357fa7f/__main__.py", line 436, in _infer_image 2024-04-12T21:26:49.300 INFO:teuthology.orchestra.run.smithi018.stdout: File "/var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/cephadm.e44f3ed5e34e4ef52eedef233d1190bff004470d8e8005127b5312bff357fa7f/__main__.py", line 310, in _validate_fsid 2024-04-12T21:26:49.300 INFO:teuthology.orchestra.run.smithi018.stdout: File "/var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/cephadm.e44f3ed5e34e4ef52eedef233d1190bff004470d8e8005127b5312bff357fa7f/__main__.py", line 3359, in command_ceph_volume 2024-04-12T21:26:49.300 INFO:teuthology.orchestra.run.smithi018.stdout: File "/var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/cephadm.e44f3ed5e34e4ef52eedef233d1190bff004470d8e8005127b5312bff357fa7f/cephadmlib/call_wrappers.py", line 310, in call_throws 2024-04-12T21:26:49.300 INFO:teuthology.orchestra.run.smithi018.stdout:RuntimeError: Failed command: /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:f4452def3be4d350f4a8e5feaf7ebb1511be9298fe68478ba7a2d6fbd7d69a63 -e NODE_NAME=smithi018 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968:/var/run/ceph:z -v /var/log/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968:/var/log/ceph:z -v /var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmpxsmwkkuh:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:f4452def3be4d350f4a8e5feaf7ebb1511be9298fe68478ba7a2d6fbd7d69a63 inventory --format=json-pretty --filter-for-batch 2024-04-12T21:26:49.300 INFO:teuthology.orchestra.run.smithi018.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-04-12T21:26:49.300 INFO:teuthology.orchestra.run.smithi018.stdout: fs cephfs is degraded 2024-04-12T21:26:49.300 INFO:teuthology.orchestra.run.smithi018.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-04-12T21:26:49.300 INFO:teuthology.orchestra.run.smithi018.stdout: fs cephfs is offline because no MDS is active for it. 2024-04-12T21:26:49.301 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:49.303+0000 7f3b63000700 1 -- 172.21.15.18:0/1912232942 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f3b4006ec00 msgr2=0x7f3b400710b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:26:49.301 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:49.303+0000 7f3b63000700 1 --2- 172.21.15.18:0/1912232942 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f3b4006ec00 0x7f3b400710b0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f3b4c004f90 tx=0x7f3b4c009e30 comp rx=0 tx=0).stop 2024-04-12T21:26:49.301 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:49.303+0000 7f3b63000700 1 -- 172.21.15.18:0/1912232942 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3b5c072e00 msgr2=0x7f3b5c11a420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:26:49.301 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:49.303+0000 7f3b63000700 1 --2- 172.21.15.18:0/1912232942 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3b5c072e00 0x7f3b5c11a420 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7f3b4400b790 tx=0x7f3b44009250 comp rx=0 tx=0).stop 2024-04-12T21:26:49.301 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:49.304+0000 7f3b63000700 1 -- 172.21.15.18:0/1912232942 shutdown_connections 2024-04-12T21:26:49.302 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:49.304+0000 7f3b63000700 1 --2- 172.21.15.18:0/1912232942 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f3b4006ec00 0x7f3b400710b0 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:26:49.302 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:49.304+0000 7f3b63000700 1 --2- 172.21.15.18:0/1912232942 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3b5c0724f0 0x7f3b5c119ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:26:49.302 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:49.304+0000 7f3b63000700 1 --2- 172.21.15.18:0/1912232942 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3b5c072e00 0x7f3b5c11a420 unknown :-1 s=CLOSED pgs=695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:26:49.302 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:49.304+0000 7f3b63000700 1 -- 172.21.15.18:0/1912232942 >> 172.21.15.18:0/1912232942 conn(0x7f3b5c0faba0 msgr2=0x7f3b5c075030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:26:49.302 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:49.304+0000 7f3b63000700 1 -- 172.21.15.18:0/1912232942 shutdown_connections 2024-04-12T21:26:49.302 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:26:49.304+0000 7f3b63000700 1 -- 172.21.15.18:0/1912232942 wait complete. 2024-04-12T21:26:50.030 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:26:49 smithi186 ceph-mon[86379]: from='client.? 172.21.15.18:0/1912232942' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-12T21:26:50.101 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:26:49 smithi018 ceph-mon[112590]: from='client.? 172.21.15.18:0/1912232942' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-12T21:26:51.030 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:26:50 smithi186 ceph-mon[86379]: from='client.35448 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:26:51.030 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:26:50 smithi186 ceph-mon[86379]: pgmap v812: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:26:51.101 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:26:50 smithi018 ceph-mon[112590]: from='client.35448 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:26:51.101 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:26:50 smithi018 ceph-mon[112590]: pgmap v812: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:26:53.030 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:26:52 smithi186 ceph-mon[86379]: pgmap v813: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:26:53.101 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:26:52 smithi018 ceph-mon[112590]: pgmap v813: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:26:55.030 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:26:54 smithi186 ceph-mon[86379]: pgmap v814: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:26:55.101 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:26:54 smithi018 ceph-mon[112590]: pgmap v814: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:26:56.030 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:26:55 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T21:26:56.101 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:26:55 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T21:26:57.030 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:26:56 smithi186 ceph-mon[86379]: pgmap v815: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:26:57.101 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:26:56 smithi018 ceph-mon[112590]: pgmap v815: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:26:59.030 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:26:58 smithi186 ceph-mon[86379]: pgmap v816: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:26:59.101 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:26:58 smithi018 ceph-mon[112590]: pgmap v816: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:27:01.030 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:27:00 smithi186 ceph-mon[86379]: pgmap v817: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:27:01.101 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:27:00 smithi018 ceph-mon[112590]: pgmap v817: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:27:03.030 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:27:02 smithi186 ceph-mon[86379]: pgmap v818: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:27:03.101 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:27:02 smithi018 ceph-mon[112590]: pgmap v818: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:27:05.030 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:27:04 smithi186 ceph-mon[86379]: pgmap v819: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:27:05.101 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:27:04 smithi018 ceph-mon[112590]: pgmap v819: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:27:06.530 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:27:06 smithi186 ceph-mon[86379]: pgmap v820: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:27:06.601 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:27:06 smithi018 ceph-mon[112590]: pgmap v820: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:27:08.851 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:27:08 smithi018 ceph-mon[112590]: pgmap v821: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:27:09.030 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:27:08 smithi186 ceph-mon[86379]: pgmap v821: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:27:10.833 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:27:10 smithi186 ceph-mon[86379]: pgmap v822: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:27:10.833 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:27:10 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T21:27:10.851 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:27:10 smithi018 ceph-mon[112590]: pgmap v822: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:27:10.851 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:27:10 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T21:27:12.851 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:27:12 smithi018 ceph-mon[112590]: pgmap v823: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:27:13.030 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:27:12 smithi186 ceph-mon[86379]: pgmap v823: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:27:14.851 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:27:14 smithi018 ceph-mon[112590]: pgmap v824: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:27:15.030 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:27:14 smithi186 ceph-mon[86379]: pgmap v824: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:27:16.851 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:27:16 smithi018 ceph-mon[112590]: pgmap v825: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:27:17.030 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:27:16 smithi186 ceph-mon[86379]: pgmap v825: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:27:18.851 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:27:18 smithi018 ceph-mon[112590]: pgmap v826: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:27:19.030 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:27:18 smithi186 ceph-mon[86379]: pgmap v826: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:27:19.477 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:19.480+0000 7fceae600700 1 -- 172.21.15.18:0/999092022 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fcea81059b0 msgr2=0x7fcea80ff020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:27:19.477 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:19.480+0000 7fceae600700 1 --2- 172.21.15.18:0/999092022 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fcea81059b0 0x7fcea80ff020 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7fce90004940 tx=0x7fce90009e30 comp rx=0 tx=0).stop 2024-04-12T21:27:19.477 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:19.480+0000 7fceae600700 1 -- 172.21.15.18:0/999092022 shutdown_connections 2024-04-12T21:27:19.477 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:19.480+0000 7fceae600700 1 --2- 172.21.15.18:0/999092022 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fcea81059b0 0x7fcea80ff020 unknown :-1 s=CLOSED pgs=696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:27:19.477 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:19.480+0000 7fceae600700 1 --2- 172.21.15.18:0/999092022 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcea8105010 0x7fcea81053e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:27:19.477 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:19.480+0000 7fceae600700 1 -- 172.21.15.18:0/999092022 >> 172.21.15.18:0/999092022 conn(0x7fcea80faba0 msgr2=0x7fcea80fcfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:27:19.478 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:19.480+0000 7fceae600700 1 -- 172.21.15.18:0/999092022 shutdown_connections 2024-04-12T21:27:19.478 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:19.480+0000 7fceae600700 1 -- 172.21.15.18:0/999092022 wait complete. 2024-04-12T21:27:19.479 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:19.481+0000 7fceae600700 1 Processor -- start 2024-04-12T21:27:19.479 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:19.481+0000 7fceae600700 1 -- start start 2024-04-12T21:27:19.479 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:19.481+0000 7fceae600700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fcea8105010 0x7fcea8115e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:27:19.479 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:19.481+0000 7fceae600700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcea81059b0 0x7fcea8116340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:27:19.479 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:19.481+0000 7fceae600700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcea8116990 con 0x7fcea8105010 2024-04-12T21:27:19.479 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:19.481+0000 7fceae600700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcea810fd60 con 0x7fcea81059b0 2024-04-12T21:27:19.479 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:19.482+0000 7fcea7400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fcea8105010 0x7fcea8115e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:27:19.479 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:19.482+0000 7fcea7400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fcea8105010 0x7fcea8115e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:58580/0 (socket says 172.21.15.18:58580) 2024-04-12T21:27:19.479 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:19.482+0000 7fcea7400700 1 -- 172.21.15.18:0/877531865 learned_addr learned my addr 172.21.15.18:0/877531865 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:27:19.480 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:19.482+0000 7fce9ea00700 1 --2- 172.21.15.18:0/877531865 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcea81059b0 0x7fcea8116340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:27:19.480 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:19.482+0000 7fcea7400700 1 -- 172.21.15.18:0/877531865 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcea81059b0 msgr2=0x7fcea8116340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:27:19.480 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:19.482+0000 7fcea7400700 1 --2- 172.21.15.18:0/877531865 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcea81059b0 0x7fcea8116340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:27:19.480 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:19.482+0000 7fcea7400700 1 -- 172.21.15.18:0/877531865 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce90005040 con 0x7fcea8105010 2024-04-12T21:27:19.481 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:19.483+0000 7fcea7400700 1 --2- 172.21.15.18:0/877531865 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fcea8105010 0x7fcea8115e00 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7fce9800c830 tx=0x7fce98004470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:27:19.481 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:19.483+0000 7fce9fe00700 1 -- 172.21.15.18:0/877531865 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fce9800a760 con 0x7fcea8105010 2024-04-12T21:27:19.481 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:19.483+0000 7fce9fe00700 1 -- 172.21.15.18:0/877531865 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fce9800ad90 con 0x7fcea8105010 2024-04-12T21:27:19.481 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:19.483+0000 7fceae600700 1 -- 172.21.15.18:0/877531865 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcea8110040 con 0x7fcea8105010 2024-04-12T21:27:19.481 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:19.483+0000 7fce9fe00700 1 -- 172.21.15.18:0/877531865 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fce980113a0 con 0x7fcea8105010 2024-04-12T21:27:19.481 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:19.483+0000 7fceae600700 1 -- 172.21.15.18:0/877531865 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcea8110600 con 0x7fcea8105010 2024-04-12T21:27:19.481 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:19.484+0000 7fceae600700 1 -- 172.21.15.18:0/877531865 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcea8100760 con 0x7fcea8105010 2024-04-12T21:27:19.482 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:19.485+0000 7fce9fe00700 1 -- 172.21.15.18:0/877531865 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7fce98012400 con 0x7fcea8105010 2024-04-12T21:27:19.484 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:19.487+0000 7fce9fe00700 1 --2- 172.21.15.18:0/877531865 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fce8c06ec10 0x7fce8c0710c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:27:19.484 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:19.487+0000 7fce9fe00700 1 -- 172.21.15.18:0/877531865 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 6757+0+0 (secure 0 0 0) 0x7fce9808eaa0 con 0x7fcea8105010 2024-04-12T21:27:19.484 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:19.487+0000 7fce9ea00700 1 --2- 172.21.15.18:0/877531865 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fce8c06ec10 0x7fce8c0710c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:27:19.484 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:19.488+0000 7fce9ea00700 1 --2- 172.21.15.18:0/877531865 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fce8c06ec10 0x7fce8c0710c0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fce9000b3b0 tx=0x7fce90009e30 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:27:19.489 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:19.492+0000 7fce9fe00700 1 -- 172.21.15.18:0/877531865 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7fce9805b250 con 0x7fcea8105010 2024-04-12T21:27:19.741 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:19.744+0000 7fceae600700 1 -- 172.21.15.18:0/877531865 --> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fcea804bf40 con 0x7fce8c06ec10 2024-04-12T21:27:19.743 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:19.746+0000 7fce9fe00700 1 -- 172.21.15.18:0/877531865 <== mgr.34102 v2:172.21.15.18:6800/3812891919 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7fcea804bf40 con 0x7fce8c06ec10 2024-04-12T21:27:19.746 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:19.749+0000 7fceae600700 1 -- 172.21.15.18:0/877531865 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fce8c06ec10 msgr2=0x7fce8c0710c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:27:19.746 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:19.749+0000 7fceae600700 1 --2- 172.21.15.18:0/877531865 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fce8c06ec10 0x7fce8c0710c0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fce9000b3b0 tx=0x7fce90009e30 comp rx=0 tx=0).stop 2024-04-12T21:27:19.746 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:19.749+0000 7fceae600700 1 -- 172.21.15.18:0/877531865 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fcea8105010 msgr2=0x7fcea8115e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:27:19.746 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:19.749+0000 7fceae600700 1 --2- 172.21.15.18:0/877531865 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fcea8105010 0x7fcea8115e00 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7fce9800c830 tx=0x7fce98004470 comp rx=0 tx=0).stop 2024-04-12T21:27:19.746 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:19.749+0000 7fceae600700 1 -- 172.21.15.18:0/877531865 shutdown_connections 2024-04-12T21:27:19.747 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:19.749+0000 7fceae600700 1 --2- 172.21.15.18:0/877531865 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fce8c06ec10 0x7fce8c0710c0 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:27:19.747 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:19.749+0000 7fceae600700 1 --2- 172.21.15.18:0/877531865 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fcea8105010 0x7fcea8115e00 unknown :-1 s=CLOSED pgs=697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:27:19.747 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:19.749+0000 7fceae600700 1 --2- 172.21.15.18:0/877531865 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcea81059b0 0x7fcea8116340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:27:19.747 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:19.749+0000 7fceae600700 1 -- 172.21.15.18:0/877531865 >> 172.21.15.18:0/877531865 conn(0x7fcea80faba0 msgr2=0x7fcea80fbc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:27:19.747 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:19.750+0000 7fceae600700 1 -- 172.21.15.18:0/877531865 shutdown_connections 2024-04-12T21:27:19.747 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:19.750+0000 7fceae600700 1 -- 172.21.15.18:0/877531865 wait complete. 2024-04-12T21:27:19.764 INFO:teuthology.orchestra.run.smithi018.stdout:true 2024-04-12T21:27:19.912 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:19.915+0000 7f91f3200700 1 -- 172.21.15.18:0/1060671967 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f91f4101ba0 msgr2=0x7f91f4101f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:27:19.912 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:19.915+0000 7f91f3200700 1 --2- 172.21.15.18:0/1060671967 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f91f4101ba0 0x7f91f4101f70 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7f91e4004940 tx=0x7f91e4009e30 comp rx=0 tx=0).stop 2024-04-12T21:27:19.912 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:19.915+0000 7f91f3200700 1 -- 172.21.15.18:0/1060671967 shutdown_connections 2024-04-12T21:27:19.912 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:19.915+0000 7f91f3200700 1 --2- 172.21.15.18:0/1060671967 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f91f40fbb70 0x7f91f40fbfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:27:19.912 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:19.915+0000 7f91f3200700 1 --2- 172.21.15.18:0/1060671967 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f91f4101ba0 0x7f91f4101f70 unknown :-1 s=CLOSED pgs=698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:27:19.912 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:19.915+0000 7f91f3200700 1 -- 172.21.15.18:0/1060671967 >> 172.21.15.18:0/1060671967 conn(0x7f91f4071b70 msgr2=0x7f91f4071f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:27:19.912 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:19.915+0000 7f91f3200700 1 -- 172.21.15.18:0/1060671967 shutdown_connections 2024-04-12T21:27:19.912 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:19.915+0000 7f91f3200700 1 -- 172.21.15.18:0/1060671967 wait complete. 2024-04-12T21:27:19.913 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:19.916+0000 7f91f3200700 1 Processor -- start 2024-04-12T21:27:19.913 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:19.916+0000 7f91f3200700 1 -- start start 2024-04-12T21:27:19.913 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:19.916+0000 7f91f3200700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f91f40fbb70 0x7f91f4113a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:27:19.913 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:19.916+0000 7f91f3200700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f91f4101ba0 0x7f91f4113f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:27:19.913 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:19.916+0000 7f91f3200700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91f4114590 con 0x7f91f4101ba0 2024-04-12T21:27:19.914 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:19.916+0000 7f91f3200700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91f410d980 con 0x7f91f40fbb70 2024-04-12T21:27:19.914 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:19.917+0000 7f91ebe00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f91f4101ba0 0x7f91f4113f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:27:19.914 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:19.917+0000 7f91ebe00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f91f4101ba0 0x7f91f4113f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:58602/0 (socket says 172.21.15.18:58602) 2024-04-12T21:27:19.914 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:19.917+0000 7f91ebe00700 1 -- 172.21.15.18:0/3140733701 learned_addr learned my addr 172.21.15.18:0/3140733701 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:27:19.914 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:19.917+0000 7f91f1e00700 1 --2- 172.21.15.18:0/3140733701 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f91f40fbb70 0x7f91f4113a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:27:19.914 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:19.917+0000 7f91ebe00700 1 -- 172.21.15.18:0/3140733701 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f91f40fbb70 msgr2=0x7f91f4113a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:27:19.915 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:19.917+0000 7f91ebe00700 1 --2- 172.21.15.18:0/3140733701 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f91f40fbb70 0x7f91f4113a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:27:19.915 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:19.917+0000 7f91ebe00700 1 -- 172.21.15.18:0/3140733701 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91e4005040 con 0x7f91f4101ba0 2024-04-12T21:27:19.915 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:19.918+0000 7f91ebe00700 1 --2- 172.21.15.18:0/3140733701 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f91f4101ba0 0x7f91f4113f40 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7f91dc002a50 tx=0x7f91dc0046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:27:19.915 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:19.918+0000 7f91ea000700 1 -- 172.21.15.18:0/3140733701 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f91dc007cc0 con 0x7f91f4101ba0 2024-04-12T21:27:19.915 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:19.918+0000 7f91ea000700 1 -- 172.21.15.18:0/3140733701 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f91dc00f040 con 0x7f91f4101ba0 2024-04-12T21:27:19.915 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:19.918+0000 7f91f3200700 1 -- 172.21.15.18:0/3140733701 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f91f410dc60 con 0x7f91f4101ba0 2024-04-12T21:27:19.915 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:19.918+0000 7f91ea000700 1 -- 172.21.15.18:0/3140733701 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f91dc009640 con 0x7f91f4101ba0 2024-04-12T21:27:19.916 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:19.918+0000 7f91f3200700 1 -- 172.21.15.18:0/3140733701 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f91f410e220 con 0x7f91f4101ba0 2024-04-12T21:27:19.916 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:19.920+0000 7f91f3200700 1 -- 172.21.15.18:0/3140733701 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f91f40fd200 con 0x7f91f4101ba0 2024-04-12T21:27:19.918 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:19.921+0000 7f91ea000700 1 -- 172.21.15.18:0/3140733701 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7f91dc0097a0 con 0x7f91f4101ba0 2024-04-12T21:27:19.919 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:19.922+0000 7f91ea000700 1 --2- 172.21.15.18:0/3140733701 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f91d806ec10 0x7f91d80710c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:27:19.919 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:19.922+0000 7f91ea000700 1 -- 172.21.15.18:0/3140733701 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 6757+0+0 (secure 0 0 0) 0x7f91dc08e830 con 0x7f91f4101ba0 2024-04-12T21:27:19.919 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:19.923+0000 7f91f1e00700 1 --2- 172.21.15.18:0/3140733701 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f91d806ec10 0x7f91d80710c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:27:19.920 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:19.923+0000 7f91f1e00700 1 --2- 172.21.15.18:0/3140733701 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f91d806ec10 0x7f91d80710c0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f91e400b3b0 tx=0x7f91e4009e30 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:27:19.924 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:19.927+0000 7f91ea000700 1 -- 172.21.15.18:0/3140733701 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f91dc05af60 con 0x7f91f4101ba0 2024-04-12T21:27:20.181 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:20.185+0000 7f91f3200700 1 -- 172.21.15.18:0/3140733701 --> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f91f41136d0 con 0x7f91d806ec10 2024-04-12T21:27:20.185 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:20.188+0000 7f91ea000700 1 -- 172.21.15.18:0/3140733701 <== mgr.34102 v2:172.21.15.18:6800/3812891919 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f91f41136d0 con 0x7f91d806ec10 2024-04-12T21:27:20.188 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:20.191+0000 7f91f3200700 1 -- 172.21.15.18:0/3140733701 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f91d806ec10 msgr2=0x7f91d80710c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:27:20.188 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:20.191+0000 7f91f3200700 1 --2- 172.21.15.18:0/3140733701 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f91d806ec10 0x7f91d80710c0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f91e400b3b0 tx=0x7f91e4009e30 comp rx=0 tx=0).stop 2024-04-12T21:27:20.188 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:20.191+0000 7f91f3200700 1 -- 172.21.15.18:0/3140733701 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f91f4101ba0 msgr2=0x7f91f4113f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:27:20.188 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:20.191+0000 7f91f3200700 1 --2- 172.21.15.18:0/3140733701 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f91f4101ba0 0x7f91f4113f40 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7f91dc002a50 tx=0x7f91dc0046b0 comp rx=0 tx=0).stop 2024-04-12T21:27:20.188 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:20.191+0000 7f91f3200700 1 -- 172.21.15.18:0/3140733701 shutdown_connections 2024-04-12T21:27:20.188 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:20.191+0000 7f91f3200700 1 --2- 172.21.15.18:0/3140733701 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f91d806ec10 0x7f91d80710c0 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:27:20.189 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:20.191+0000 7f91f3200700 1 --2- 172.21.15.18:0/3140733701 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f91f40fbb70 0x7f91f4113a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:27:20.189 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:20.191+0000 7f91f3200700 1 --2- 172.21.15.18:0/3140733701 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f91f4101ba0 0x7f91f4113f40 unknown :-1 s=CLOSED pgs=699 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:27:20.189 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:20.191+0000 7f91f3200700 1 -- 172.21.15.18:0/3140733701 >> 172.21.15.18:0/3140733701 conn(0x7f91f4071b70 msgr2=0x7f91f40fa6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:27:20.189 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:20.191+0000 7f91f3200700 1 -- 172.21.15.18:0/3140733701 shutdown_connections 2024-04-12T21:27:20.189 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:20.192+0000 7f91f3200700 1 -- 172.21.15.18:0/3140733701 wait complete. 2024-04-12T21:27:20.352 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:20.355+0000 7f61f3800700 1 -- 172.21.15.18:0/1998408016 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f61ec1037a0 msgr2=0x7f61ec0fce10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:27:20.352 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:20.355+0000 7f61f3800700 1 --2- 172.21.15.18:0/1998408016 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f61ec1037a0 0x7f61ec0fce10 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7f61d8004940 tx=0x7f61d8009e30 comp rx=0 tx=0).stop 2024-04-12T21:27:20.352 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:20.355+0000 7f61f3800700 1 -- 172.21.15.18:0/1998408016 shutdown_connections 2024-04-12T21:27:20.352 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:20.355+0000 7f61f3800700 1 --2- 172.21.15.18:0/1998408016 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f61ec1037a0 0x7f61ec0fce10 unknown :-1 s=CLOSED pgs=700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:27:20.352 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:20.355+0000 7f61f3800700 1 --2- 172.21.15.18:0/1998408016 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f61ec102e00 0x7f61ec1031d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:27:20.352 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:20.355+0000 7f61f3800700 1 -- 172.21.15.18:0/1998408016 >> 172.21.15.18:0/1998408016 conn(0x7f61ec0f89b0 msgr2=0x7f61ec0fadc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:27:20.352 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:20.355+0000 7f61f3800700 1 -- 172.21.15.18:0/1998408016 shutdown_connections 2024-04-12T21:27:20.352 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:20.355+0000 7f61f3800700 1 -- 172.21.15.18:0/1998408016 wait complete. 2024-04-12T21:27:20.353 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:20.356+0000 7f61f3800700 1 Processor -- start 2024-04-12T21:27:20.353 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:20.356+0000 7f61f3800700 1 -- start start 2024-04-12T21:27:20.354 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:20.357+0000 7f61f3800700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f61ec102e00 0x7f61ec115de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:27:20.354 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:20.357+0000 7f61f3800700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f61ec1037a0 0x7f61ec116320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:27:20.354 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:20.357+0000 7f61f3800700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61ec116970 con 0x7f61ec1037a0 2024-04-12T21:27:20.354 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:20.357+0000 7f61f3800700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61ec10fd40 con 0x7f61ec102e00 2024-04-12T21:27:20.355 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:20.357+0000 7f61ebe00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f61ec1037a0 0x7f61ec116320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:27:20.355 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:20.357+0000 7f61ebe00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f61ec1037a0 0x7f61ec116320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:58608/0 (socket says 172.21.15.18:58608) 2024-04-12T21:27:20.355 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:20.357+0000 7f61ebe00700 1 -- 172.21.15.18:0/2685995017 learned_addr learned my addr 172.21.15.18:0/2685995017 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:27:20.355 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:20.358+0000 7f61f0e00700 1 --2- 172.21.15.18:0/2685995017 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f61ec102e00 0x7f61ec115de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:27:20.355 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:20.358+0000 7f61ebe00700 1 -- 172.21.15.18:0/2685995017 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f61ec102e00 msgr2=0x7f61ec115de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:27:20.355 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:20.358+0000 7f61ebe00700 1 --2- 172.21.15.18:0/2685995017 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f61ec102e00 0x7f61ec115de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:27:20.355 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:20.358+0000 7f61ebe00700 1 -- 172.21.15.18:0/2685995017 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61d8005040 con 0x7f61ec1037a0 2024-04-12T21:27:20.355 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:20.358+0000 7f61ebe00700 1 --2- 172.21.15.18:0/2685995017 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f61ec1037a0 0x7f61ec116320 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7f61d8009180 tx=0x7f61d8009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:27:20.356 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:20.359+0000 7f61e8c00700 1 -- 172.21.15.18:0/2685995017 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f61d800e040 con 0x7f61ec1037a0 2024-04-12T21:27:20.356 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:20.359+0000 7f61e8c00700 1 -- 172.21.15.18:0/2685995017 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f61d800bbc0 con 0x7f61ec1037a0 2024-04-12T21:27:20.356 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:20.359+0000 7f61f3800700 1 -- 172.21.15.18:0/2685995017 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f61ec10ffc0 con 0x7f61ec1037a0 2024-04-12T21:27:20.356 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:20.359+0000 7f61e8c00700 1 -- 172.21.15.18:0/2685995017 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f61d801c3f0 con 0x7f61ec1037a0 2024-04-12T21:27:20.356 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:20.359+0000 7f61f3800700 1 -- 172.21.15.18:0/2685995017 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f61ec110580 con 0x7f61ec1037a0 2024-04-12T21:27:20.357 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:20.360+0000 7f61f3800700 1 -- 172.21.15.18:0/2685995017 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f61ec1018c0 con 0x7f61ec1037a0 2024-04-12T21:27:20.358 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:20.361+0000 7f61e8c00700 1 -- 172.21.15.18:0/2685995017 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7f61d800bd30 con 0x7f61ec1037a0 2024-04-12T21:27:20.359 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:20.362+0000 7f61e8c00700 1 --2- 172.21.15.18:0/2685995017 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f61d406ec00 0x7f61d40710b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:27:20.359 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:20.362+0000 7f61e8c00700 1 -- 172.21.15.18:0/2685995017 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 6757+0+0 (secure 0 0 0) 0x7f61d808fd00 con 0x7f61ec1037a0 2024-04-12T21:27:20.359 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:20.363+0000 7f61f0e00700 1 --2- 172.21.15.18:0/2685995017 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f61d406ec00 0x7f61d40710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:27:20.360 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:20.363+0000 7f61f0e00700 1 --2- 172.21.15.18:0/2685995017 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f61d406ec00 0x7f61d40710b0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f61e0004990 tx=0x7f61e0009ee0 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:27:20.364 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:20.367+0000 7f61e8c00700 1 -- 172.21.15.18:0/2685995017 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f61d805c530 con 0x7f61ec1037a0 2024-04-12T21:27:20.600 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:20.603+0000 7f61f3800700 1 -- 172.21.15.18:0/2685995017 --> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f61ec111430 con 0x7f61d406ec00 2024-04-12T21:27:20.637 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:20.640+0000 7f61e8c00700 1 -- 172.21.15.18:0/2685995017 <== mgr.34102 v2:172.21.15.18:6800/3812891919 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7f61ec111430 con 0x7f61d406ec00 2024-04-12T21:27:20.638 INFO:teuthology.orchestra.run.smithi018.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-12T21:27:20.638 INFO:teuthology.orchestra.run.smithi018.stdout:alertmanager.smithi018 smithi018 *:9093,9094 running (27m) 4m ago 42m 15.5M - 0.25.0 c8568f914cd2 c332090068e8 2024-04-12T21:27:20.638 INFO:teuthology.orchestra.run.smithi018.stdout:crash.smithi018 smithi018 running (24m) 4m ago 42m 6665k - 19.0.0-2879-g324baf8d 8717720dc5e5 b7f80d1f7982 2024-04-12T21:27:20.638 INFO:teuthology.orchestra.run.smithi018.stdout:crash.smithi186 smithi186 running (24m) 4m ago 41m 6660k - 19.0.0-2879-g324baf8d 8717720dc5e5 a4f55c81b623 2024-04-12T21:27:20.639 INFO:teuthology.orchestra.run.smithi018.stdout:grafana.smithi018 smithi018 *:3000 running (27m) 4m ago 42m 74.8M - 9.4.12 2bacad6d85d8 b08e3bf176dd 2024-04-12T21:27:20.639 INFO:teuthology.orchestra.run.smithi018.stdout:mds.cephfs.smithi018.acqbsc smithi018 running (20m) 4m ago 35m 16.7M - 19.0.0-2879-g324baf8d 8717720dc5e5 021f99ab24ae 2024-04-12T21:27:20.639 INFO:teuthology.orchestra.run.smithi018.stdout:mds.cephfs.smithi018.szjluo smithi018 running (19m) 4m ago 35m 16.5M - 19.0.0-2879-g324baf8d 8717720dc5e5 0003117479d4 2024-04-12T21:27:20.639 INFO:teuthology.orchestra.run.smithi018.stdout:mds.cephfs.smithi186.jdllyn smithi186 running (4m) 4m ago 35m 11.0M - 19.0.0-2879-g324baf8d 8717720dc5e5 21b1f46eef23 2024-04-12T21:27:20.639 INFO:teuthology.orchestra.run.smithi018.stdout:mds.cephfs.smithi186.wjnafs smithi186 running (35m) 4m ago 35m 19.5M - 17.2.7-827-gd3b5060a 21de9b7369ad 74b115d6d6fd 2024-04-12T21:27:20.639 INFO:teuthology.orchestra.run.smithi018.stdout:mgr.smithi018.xjzwga smithi018 *:8443,9283,8765 running (30m) 4m ago 44m 512M - 19.0.0-2879-g324baf8d 8717720dc5e5 56088305eab7 2024-04-12T21:27:20.639 INFO:teuthology.orchestra.run.smithi018.stdout:mgr.smithi186.kqcfxk smithi186 *:8443,9283,8765 running (29m) 4m ago 41m 459M - 19.0.0-2879-g324baf8d 8717720dc5e5 09cd766e5e0d 2024-04-12T21:27:20.640 INFO:teuthology.orchestra.run.smithi018.stdout:mon.smithi018 smithi018 running (25m) 4m ago 44m 73.1M 2048M 19.0.0-2879-g324baf8d 8717720dc5e5 df3e201dfed6 2024-04-12T21:27:20.640 INFO:teuthology.orchestra.run.smithi018.stdout:mon.smithi186 smithi186 running (25m) 4m ago 41m 47.2M 2048M 19.0.0-2879-g324baf8d 8717720dc5e5 c77498da9b2b 2024-04-12T21:27:20.640 INFO:teuthology.orchestra.run.smithi018.stdout:node-exporter.smithi018 smithi018 *:9100 running (28m) 4m ago 42m 8108k - 1.5.0 0da6a335fe13 326c3d9c6847 2024-04-12T21:27:20.640 INFO:teuthology.orchestra.run.smithi018.stdout:node-exporter.smithi186 smithi186 *:9100 running (28m) 4m ago 41m 8757k - 1.5.0 0da6a335fe13 ec2112baeecf 2024-04-12T21:27:20.640 INFO:teuthology.orchestra.run.smithi018.stdout:osd.0 smithi018 running (23m) 4m ago 40m 123M 1559M 19.0.0-2879-g324baf8d 8717720dc5e5 60b1273ca047 2024-04-12T21:27:20.640 INFO:teuthology.orchestra.run.smithi018.stdout:osd.1 smithi018 running (23m) 4m ago 39m 95.1M 1559M 19.0.0-2879-g324baf8d 8717720dc5e5 170ce1e5fb8d 2024-04-12T21:27:20.640 INFO:teuthology.orchestra.run.smithi018.stdout:osd.2 smithi018 running (22m) 4m ago 38m 94.3M 1559M 19.0.0-2879-g324baf8d 8717720dc5e5 c2cbc6aa2839 2024-04-12T21:27:20.640 INFO:teuthology.orchestra.run.smithi018.stdout:osd.3 smithi186 running (22m) 4m ago 38m 101M 2581M 19.0.0-2879-g324baf8d 8717720dc5e5 0138e8262e4b 2024-04-12T21:27:20.640 INFO:teuthology.orchestra.run.smithi018.stdout:osd.4 smithi186 running (21m) 4m ago 37m 91.0M 2581M 19.0.0-2879-g324baf8d 8717720dc5e5 b20cb63b25c9 2024-04-12T21:27:20.640 INFO:teuthology.orchestra.run.smithi018.stdout:osd.5 smithi186 running (20m) 4m ago 37m 92.0M 2581M 19.0.0-2879-g324baf8d 8717720dc5e5 2513bea2610e 2024-04-12T21:27:20.640 INFO:teuthology.orchestra.run.smithi018.stdout:prometheus.smithi018 smithi018 *:9095 running (28m) 4m ago 41m 67.9M - 2.43.0 a07b618ecd1d f58841127e21 2024-04-12T21:27:20.641 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:20.643+0000 7f61f3800700 1 -- 172.21.15.18:0/2685995017 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f61d406ec00 msgr2=0x7f61d40710b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:27:20.641 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:20.643+0000 7f61f3800700 1 --2- 172.21.15.18:0/2685995017 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f61d406ec00 0x7f61d40710b0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f61e0004990 tx=0x7f61e0009ee0 comp rx=0 tx=0).stop 2024-04-12T21:27:20.642 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:20.643+0000 7f61f3800700 1 -- 172.21.15.18:0/2685995017 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f61ec1037a0 msgr2=0x7f61ec116320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:27:20.642 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:20.643+0000 7f61f3800700 1 --2- 172.21.15.18:0/2685995017 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f61ec1037a0 0x7f61ec116320 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7f61d8009180 tx=0x7f61d8009e30 comp rx=0 tx=0).stop 2024-04-12T21:27:20.642 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:20.643+0000 7f61f3800700 1 -- 172.21.15.18:0/2685995017 shutdown_connections 2024-04-12T21:27:20.642 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:20.643+0000 7f61f3800700 1 --2- 172.21.15.18:0/2685995017 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f61d406ec00 0x7f61d40710b0 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:27:20.642 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:20.643+0000 7f61f3800700 1 --2- 172.21.15.18:0/2685995017 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f61ec102e00 0x7f61ec115de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:27:20.642 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:20.643+0000 7f61f3800700 1 --2- 172.21.15.18:0/2685995017 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f61ec1037a0 0x7f61ec116320 unknown :-1 s=CLOSED pgs=701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:27:20.642 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:20.643+0000 7f61f3800700 1 -- 172.21.15.18:0/2685995017 >> 172.21.15.18:0/2685995017 conn(0x7f61ec0f89b0 msgr2=0x7f61ec102210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:27:20.642 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:20.643+0000 7f61f3800700 1 -- 172.21.15.18:0/2685995017 shutdown_connections 2024-04-12T21:27:20.642 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:20.643+0000 7f61f3800700 1 -- 172.21.15.18:0/2685995017 wait complete. 2024-04-12T21:27:20.798 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:20.801+0000 7f6939a00700 1 -- 172.21.15.18:0/1248453165 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6934109430 msgr2=0x7f6934109800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:27:20.799 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:20.801+0000 7f6939a00700 1 --2- 172.21.15.18:0/1248453165 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6934109430 0x7f6934109800 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7f6924004940 tx=0x7f6924009e30 comp rx=0 tx=0).stop 2024-04-12T21:27:20.799 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:20.801+0000 7f6939a00700 1 -- 172.21.15.18:0/1248453165 shutdown_connections 2024-04-12T21:27:20.799 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:20.801+0000 7f6939a00700 1 --2- 172.21.15.18:0/1248453165 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6934109dd0 0x7f6934103440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:27:20.799 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:20.801+0000 7f6939a00700 1 --2- 172.21.15.18:0/1248453165 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6934109430 0x7f6934109800 unknown :-1 s=CLOSED pgs=702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:27:20.799 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:20.802+0000 7f6939a00700 1 -- 172.21.15.18:0/1248453165 >> 172.21.15.18:0/1248453165 conn(0x7f6934072c00 msgr2=0x7f6934075010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:27:20.799 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:20.802+0000 7f6939a00700 1 -- 172.21.15.18:0/1248453165 shutdown_connections 2024-04-12T21:27:20.799 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:20.802+0000 7f6939a00700 1 -- 172.21.15.18:0/1248453165 wait complete. 2024-04-12T21:27:20.800 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:20.802+0000 7f6939a00700 1 Processor -- start 2024-04-12T21:27:20.800 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:20.803+0000 7f6939a00700 1 -- start start 2024-04-12T21:27:20.800 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:20.803+0000 7f6939a00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6934109430 0x7f6934119f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:27:20.801 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:20.803+0000 7f6939a00700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6934109dd0 0x7f693411a4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:27:20.801 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:20.803+0000 7f6939a00700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f693411ab20 con 0x7f6934109430 2024-04-12T21:27:20.801 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:20.803+0000 7f6939a00700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6934114100 con 0x7f6934109dd0 2024-04-12T21:27:20.801 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:20.803+0000 7f6932a00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6934109430 0x7f6934119f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:27:20.801 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:20.803+0000 7f6932a00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6934109430 0x7f6934119f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:58618/0 (socket says 172.21.15.18:58618) 2024-04-12T21:27:20.801 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:20.803+0000 7f6932a00700 1 -- 172.21.15.18:0/1017957372 learned_addr learned my addr 172.21.15.18:0/1017957372 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:27:20.801 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:20.804+0000 7f6932a00700 1 -- 172.21.15.18:0/1017957372 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6934109dd0 msgr2=0x7f693411a4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:27:20.801 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:20.804+0000 7f6932a00700 1 --2- 172.21.15.18:0/1017957372 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6934109dd0 0x7f693411a4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:27:20.801 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:20.804+0000 7f6932a00700 1 -- 172.21.15.18:0/1017957372 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6924005040 con 0x7f6934109430 2024-04-12T21:27:20.802 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:20.804+0000 7f6932a00700 1 --2- 172.21.15.18:0/1017957372 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6934109430 0x7f6934119f90 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7f6924009180 tx=0x7f6924009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:27:20.802 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:20.804+0000 7f692aa00700 1 -- 172.21.15.18:0/1017957372 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f692400e040 con 0x7f6934109430 2024-04-12T21:27:20.802 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:20.805+0000 7f692aa00700 1 -- 172.21.15.18:0/1017957372 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f692400bbc0 con 0x7f6934109430 2024-04-12T21:27:20.802 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:20.805+0000 7f6939a00700 1 -- 172.21.15.18:0/1017957372 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6934114380 con 0x7f6934109430 2024-04-12T21:27:20.802 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:20.805+0000 7f692aa00700 1 -- 172.21.15.18:0/1017957372 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f692401c630 con 0x7f6934109430 2024-04-12T21:27:20.802 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:20.805+0000 7f6939a00700 1 -- 172.21.15.18:0/1017957372 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6934114940 con 0x7f6934109430 2024-04-12T21:27:20.803 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:20.806+0000 7f6939a00700 1 -- 172.21.15.18:0/1017957372 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6934104b80 con 0x7f6934109430 2024-04-12T21:27:20.803 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:20.807+0000 7f692aa00700 1 -- 172.21.15.18:0/1017957372 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7f692400bd30 con 0x7f6934109430 2024-04-12T21:27:20.805 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:20.808+0000 7f692aa00700 1 --2- 172.21.15.18:0/1017957372 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f691806ec00 0x7f69180710b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:27:20.805 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:20.808+0000 7f692aa00700 1 -- 172.21.15.18:0/1017957372 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 6757+0+0 (secure 0 0 0) 0x7f692408fe30 con 0x7f6934109430 2024-04-12T21:27:20.805 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:20.808+0000 7f692be00700 1 --2- 172.21.15.18:0/1017957372 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f691806ec00 0x7f69180710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:27:20.805 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:20.809+0000 7f692be00700 1 --2- 172.21.15.18:0/1017957372 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f691806ec00 0x7f69180710b0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f6934115830 tx=0x7f691c009ce0 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:27:20.810 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:20.813+0000 7f692aa00700 1 -- 172.21.15.18:0/1017957372 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f692405c5e0 con 0x7f6934109430 2024-04-12T21:27:20.851 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:27:20 smithi018 ceph-mon[112590]: pgmap v827: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:27:20.851 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:27:20 smithi018 ceph-mon[112590]: from='client.35456 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:27:21.030 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:27:20 smithi186 ceph-mon[86379]: pgmap v827: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:27:21.030 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:27:20 smithi186 ceph-mon[86379]: from='client.35456 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:27:21.137 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:21.140+0000 7f6939a00700 1 -- 172.21.15.18:0/1017957372 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f6934110540 con 0x7f6934109430 2024-04-12T21:27:21.137 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:21.141+0000 7f692aa00700 1 -- 172.21.15.18:0/1017957372 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+837 (secure 0 0 0) 0x7f692405c170 con 0x7f6934109430 2024-04-12T21:27:21.138 INFO:teuthology.orchestra.run.smithi018.stdout:{ 2024-04-12T21:27:21.138 INFO:teuthology.orchestra.run.smithi018.stdout: "mon": { 2024-04-12T21:27:21.138 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 19.0.0-2879-g324baf8d (324baf8ddb91dea46cac50cb535c306115791355) squid (dev)": 2 2024-04-12T21:27:21.138 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-12T21:27:21.138 INFO:teuthology.orchestra.run.smithi018.stdout: "mgr": { 2024-04-12T21:27:21.138 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 19.0.0-2879-g324baf8d (324baf8ddb91dea46cac50cb535c306115791355) squid (dev)": 2 2024-04-12T21:27:21.138 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-12T21:27:21.138 INFO:teuthology.orchestra.run.smithi018.stdout: "osd": { 2024-04-12T21:27:21.138 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 19.0.0-2879-g324baf8d (324baf8ddb91dea46cac50cb535c306115791355) squid (dev)": 6 2024-04-12T21:27:21.138 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-12T21:27:21.138 INFO:teuthology.orchestra.run.smithi018.stdout: "mds": { 2024-04-12T21:27:21.138 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 17.2.7-827-gd3b5060a (d3b5060a17449edfae9385cdf792afec109fca5d) quincy (stable)": 1, 2024-04-12T21:27:21.139 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 19.0.0-2879-g324baf8d (324baf8ddb91dea46cac50cb535c306115791355) squid (dev)": 3 2024-04-12T21:27:21.139 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-12T21:27:21.139 INFO:teuthology.orchestra.run.smithi018.stdout: "overall": { 2024-04-12T21:27:21.139 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 17.2.7-827-gd3b5060a (d3b5060a17449edfae9385cdf792afec109fca5d) quincy (stable)": 1, 2024-04-12T21:27:21.139 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 19.0.0-2879-g324baf8d (324baf8ddb91dea46cac50cb535c306115791355) squid (dev)": 13 2024-04-12T21:27:21.139 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-04-12T21:27:21.139 INFO:teuthology.orchestra.run.smithi018.stdout:} 2024-04-12T21:27:21.140 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:21.144+0000 7f6939a00700 1 -- 172.21.15.18:0/1017957372 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f691806ec00 msgr2=0x7f69180710b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:27:21.140 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:21.144+0000 7f6939a00700 1 --2- 172.21.15.18:0/1017957372 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f691806ec00 0x7f69180710b0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f6934115830 tx=0x7f691c009ce0 comp rx=0 tx=0).stop 2024-04-12T21:27:21.140 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:21.144+0000 7f6939a00700 1 -- 172.21.15.18:0/1017957372 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6934109430 msgr2=0x7f6934119f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:27:21.140 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:21.144+0000 7f6939a00700 1 --2- 172.21.15.18:0/1017957372 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6934109430 0x7f6934119f90 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7f6924009180 tx=0x7f6924009e30 comp rx=0 tx=0).stop 2024-04-12T21:27:21.141 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:21.144+0000 7f6939a00700 1 -- 172.21.15.18:0/1017957372 shutdown_connections 2024-04-12T21:27:21.141 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:21.144+0000 7f6939a00700 1 --2- 172.21.15.18:0/1017957372 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f691806ec00 0x7f69180710b0 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:27:21.141 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:21.144+0000 7f6939a00700 1 --2- 172.21.15.18:0/1017957372 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6934109430 0x7f6934119f90 unknown :-1 s=CLOSED pgs=703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:27:21.141 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:21.144+0000 7f6939a00700 1 --2- 172.21.15.18:0/1017957372 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6934109dd0 0x7f693411a4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:27:21.141 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:21.144+0000 7f6939a00700 1 -- 172.21.15.18:0/1017957372 >> 172.21.15.18:0/1017957372 conn(0x7f6934072c00 msgr2=0x7f6934074c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:27:21.141 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:21.144+0000 7f6939a00700 1 -- 172.21.15.18:0/1017957372 shutdown_connections 2024-04-12T21:27:21.142 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:21.144+0000 7f6939a00700 1 -- 172.21.15.18:0/1017957372 wait complete. 2024-04-12T21:27:21.303 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:21.305+0000 7ff666a00700 1 -- 172.21.15.18:0/1621152376 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff6600705d0 msgr2=0x7ff6600709a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:27:21.303 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:21.305+0000 7ff666a00700 1 --2- 172.21.15.18:0/1621152376 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff6600705d0 0x7ff6600709a0 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7ff650004970 tx=0x7ff650009e30 comp rx=0 tx=0).stop 2024-04-12T21:27:21.303 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:21.306+0000 7ff666a00700 1 -- 172.21.15.18:0/1621152376 shutdown_connections 2024-04-12T21:27:21.303 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:21.306+0000 7ff666a00700 1 --2- 172.21.15.18:0/1621152376 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff660070ee0 0x7ff660109130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:27:21.303 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:21.306+0000 7ff666a00700 1 --2- 172.21.15.18:0/1621152376 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff6600705d0 0x7ff6600709a0 unknown :-1 s=CLOSED pgs=704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:27:21.303 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:21.306+0000 7ff666a00700 1 -- 172.21.15.18:0/1621152376 >> 172.21.15.18:0/1621152376 conn(0x7ff6600f89b0 msgr2=0x7ff6600fadc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:27:21.303 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:21.306+0000 7ff666a00700 1 -- 172.21.15.18:0/1621152376 shutdown_connections 2024-04-12T21:27:21.303 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:21.306+0000 7ff666a00700 1 -- 172.21.15.18:0/1621152376 wait complete. 2024-04-12T21:27:21.305 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:21.307+0000 7ff666a00700 1 Processor -- start 2024-04-12T21:27:21.305 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:21.307+0000 7ff666a00700 1 -- start start 2024-04-12T21:27:21.305 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:21.307+0000 7ff666a00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff6600705d0 0x7ff660115a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:27:21.305 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:21.307+0000 7ff666a00700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff660070ee0 0x7ff660115f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:27:21.305 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:21.307+0000 7ff666a00700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6601165d0 con 0x7ff6600705d0 2024-04-12T21:27:21.305 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:21.307+0000 7ff666a00700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff66010fc60 con 0x7ff660070ee0 2024-04-12T21:27:21.305 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:21.307+0000 7ff65fe00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff6600705d0 0x7ff660115a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:27:21.305 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:21.307+0000 7ff65fe00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff6600705d0 0x7ff660115a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:55686/0 (socket says 172.21.15.18:55686) 2024-04-12T21:27:21.305 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:21.307+0000 7ff65fe00700 1 -- 172.21.15.18:0/2422163320 learned_addr learned my addr 172.21.15.18:0/2422163320 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:27:21.305 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:21.308+0000 7ff65fe00700 1 -- 172.21.15.18:0/2422163320 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff660070ee0 msgr2=0x7ff660115f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:27:21.305 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:21.308+0000 7ff65fe00700 1 --2- 172.21.15.18:0/2422163320 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff660070ee0 0x7ff660115f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:27:21.305 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:21.308+0000 7ff65fe00700 1 -- 172.21.15.18:0/2422163320 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff650005040 con 0x7ff6600705d0 2024-04-12T21:27:21.305 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:21.308+0000 7ff65fe00700 1 --2- 172.21.15.18:0/2422163320 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff6600705d0 0x7ff660115a40 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7ff650009150 tx=0x7ff650009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:27:21.306 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:21.308+0000 7ff657e00700 1 -- 172.21.15.18:0/2422163320 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff65000e040 con 0x7ff6600705d0 2024-04-12T21:27:21.306 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:21.308+0000 7ff657e00700 1 -- 172.21.15.18:0/2422163320 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff65000bc10 con 0x7ff6600705d0 2024-04-12T21:27:21.306 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:21.309+0000 7ff666a00700 1 -- 172.21.15.18:0/2422163320 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff66010fee0 con 0x7ff6600705d0 2024-04-12T21:27:21.306 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:21.309+0000 7ff657e00700 1 -- 172.21.15.18:0/2422163320 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff65001c630 con 0x7ff6600705d0 2024-04-12T21:27:21.307 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:21.309+0000 7ff666a00700 1 -- 172.21.15.18:0/2422163320 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff660110420 con 0x7ff6600705d0 2024-04-12T21:27:21.307 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:21.310+0000 7ff666a00700 1 -- 172.21.15.18:0/2422163320 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff660071aa0 con 0x7ff6600705d0 2024-04-12T21:27:21.307 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:21.311+0000 7ff657e00700 1 -- 172.21.15.18:0/2422163320 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7ff65000bd80 con 0x7ff6600705d0 2024-04-12T21:27:21.309 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:21.312+0000 7ff657e00700 1 --2- 172.21.15.18:0/2422163320 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7ff64406ec00 0x7ff6440710b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:27:21.309 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:21.313+0000 7ff657e00700 1 -- 172.21.15.18:0/2422163320 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 6757+0+0 (secure 0 0 0) 0x7ff65008fdc0 con 0x7ff6600705d0 2024-04-12T21:27:21.309 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:21.313+0000 7ff65f400700 1 --2- 172.21.15.18:0/2422163320 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7ff64406ec00 0x7ff6440710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:27:21.310 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:21.313+0000 7ff65f400700 1 --2- 172.21.15.18:0/2422163320 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7ff64406ec00 0x7ff6440710b0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7ff660104980 tx=0x7ff64c007d40 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:27:21.314 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:21.318+0000 7ff657e00700 1 -- 172.21.15.18:0/2422163320 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7ff650093030 con 0x7ff6600705d0 2024-04-12T21:27:21.597 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:21.600+0000 7ff666a00700 1 -- 172.21.15.18:0/2422163320 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7ff660111140 con 0x7ff6600705d0 2024-04-12T21:27:21.598 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:21.601+0000 7ff657e00700 1 -- 172.21.15.18:0/2422163320 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 31 v31) v1 ==== 76+0+1873 (secure 0 0 0) 0x7ff650058a40 con 0x7ff6600705d0 2024-04-12T21:27:21.599 INFO:teuthology.orchestra.run.smithi018.stdout:e31 2024-04-12T21:27:21.599 INFO:teuthology.orchestra.run.smithi018.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-12T21:27:21.600 INFO:teuthology.orchestra.run.smithi018.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-12T21:27:21.600 INFO:teuthology.orchestra.run.smithi018.stdout:legacy client fscid: 1 2024-04-12T21:27:21.600 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-12T21:27:21.600 INFO:teuthology.orchestra.run.smithi018.stdout:Filesystem 'cephfs' (1) 2024-04-12T21:27:21.600 INFO:teuthology.orchestra.run.smithi018.stdout:fs_name cephfs 2024-04-12T21:27:21.600 INFO:teuthology.orchestra.run.smithi018.stdout:epoch 24 2024-04-12T21:27:21.600 INFO:teuthology.orchestra.run.smithi018.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-04-12T21:27:21.600 INFO:teuthology.orchestra.run.smithi018.stdout:created 2024-04-12T20:51:57.362850+0000 2024-04-12T21:27:21.600 INFO:teuthology.orchestra.run.smithi018.stdout:modified 2024-04-12T21:06:59.717190+0000 2024-04-12T21:27:21.600 INFO:teuthology.orchestra.run.smithi018.stdout:tableserver 0 2024-04-12T21:27:21.600 INFO:teuthology.orchestra.run.smithi018.stdout:root 0 2024-04-12T21:27:21.600 INFO:teuthology.orchestra.run.smithi018.stdout:session_timeout 60 2024-04-12T21:27:21.600 INFO:teuthology.orchestra.run.smithi018.stdout:session_autoclose 300 2024-04-12T21:27:21.600 INFO:teuthology.orchestra.run.smithi018.stdout:max_file_size 1099511627776 2024-04-12T21:27:21.601 INFO:teuthology.orchestra.run.smithi018.stdout:max_xattr_size 65536 2024-04-12T21:27:21.601 INFO:teuthology.orchestra.run.smithi018.stdout:required_client_features {} 2024-04-12T21:27:21.601 INFO:teuthology.orchestra.run.smithi018.stdout:last_failure 0 2024-04-12T21:27:21.601 INFO:teuthology.orchestra.run.smithi018.stdout:last_failure_osd_epoch 100 2024-04-12T21:27:21.601 INFO:teuthology.orchestra.run.smithi018.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-12T21:27:21.601 INFO:teuthology.orchestra.run.smithi018.stdout:max_mds 2 2024-04-12T21:27:21.601 INFO:teuthology.orchestra.run.smithi018.stdout:in 0,1 2024-04-12T21:27:21.601 INFO:teuthology.orchestra.run.smithi018.stdout:up {} 2024-04-12T21:27:21.601 INFO:teuthology.orchestra.run.smithi018.stdout:failed 0,1 2024-04-12T21:27:21.601 INFO:teuthology.orchestra.run.smithi018.stdout:damaged 2024-04-12T21:27:21.601 INFO:teuthology.orchestra.run.smithi018.stdout:stopped 2024-04-12T21:27:21.601 INFO:teuthology.orchestra.run.smithi018.stdout:data_pools [3] 2024-04-12T21:27:21.601 INFO:teuthology.orchestra.run.smithi018.stdout:metadata_pool 2 2024-04-12T21:27:21.601 INFO:teuthology.orchestra.run.smithi018.stdout:inline_data disabled 2024-04-12T21:27:21.601 INFO:teuthology.orchestra.run.smithi018.stdout:balancer 2024-04-12T21:27:21.601 INFO:teuthology.orchestra.run.smithi018.stdout:bal_rank_mask -1 2024-04-12T21:27:21.602 INFO:teuthology.orchestra.run.smithi018.stdout:standby_count_wanted 1 2024-04-12T21:27:21.602 INFO:teuthology.orchestra.run.smithi018.stdout:qdb_cluster leader: 0 members: 2024-04-12T21:27:21.602 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-12T21:27:21.602 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-12T21:27:21.602 INFO:teuthology.orchestra.run.smithi018.stdout:Standby daemons: 2024-04-12T21:27:21.602 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-12T21:27:21.602 INFO:teuthology.orchestra.run.smithi018.stdout:[mds.cephfs.smithi186.wjnafs{-1:24277} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.186:6826/438347207,v1:172.21.15.186:6827/438347207] compat {c=[1],r=[1],i=[7ff]}] 2024-04-12T21:27:21.602 INFO:teuthology.orchestra.run.smithi018.stdout:[mds.cephfs.smithi018.acqbsc{-1:34436} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.18:6828/755179764,v1:172.21.15.18:6829/755179764] compat {c=[1],r=[1],i=[1fff]}] 2024-04-12T21:27:21.602 INFO:teuthology.orchestra.run.smithi018.stdout:[mds.cephfs.smithi018.szjluo{-1:34468} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.18:6826/2407992653,v1:172.21.15.18:6827/2407992653] compat {c=[1],r=[1],i=[1fff]}] 2024-04-12T21:27:21.602 INFO:teuthology.orchestra.run.smithi018.stdout:[mds.cephfs.smithi186.jdllyn{-1:44595} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.186:6824/3523444202,v1:172.21.15.186:6825/3523444202] compat {c=[1],r=[1],i=[1fff]}] 2024-04-12T21:27:21.603 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:21.605+0000 7ff666a00700 1 -- 172.21.15.18:0/2422163320 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7ff64406ec00 msgr2=0x7ff6440710b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:27:21.603 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:21.605+0000 7ff666a00700 1 --2- 172.21.15.18:0/2422163320 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7ff64406ec00 0x7ff6440710b0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7ff660104980 tx=0x7ff64c007d40 comp rx=0 tx=0).stop 2024-04-12T21:27:21.603 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:21.605+0000 7ff666a00700 1 -- 172.21.15.18:0/2422163320 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff6600705d0 msgr2=0x7ff660115a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:27:21.603 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:21.605+0000 7ff666a00700 1 --2- 172.21.15.18:0/2422163320 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff6600705d0 0x7ff660115a40 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7ff650009150 tx=0x7ff650009e30 comp rx=0 tx=0).stop 2024-04-12T21:27:21.603 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:21.605+0000 7ff666a00700 1 -- 172.21.15.18:0/2422163320 shutdown_connections 2024-04-12T21:27:21.603 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:21.605+0000 7ff666a00700 1 --2- 172.21.15.18:0/2422163320 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7ff64406ec00 0x7ff6440710b0 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:27:21.603 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:21.605+0000 7ff666a00700 1 --2- 172.21.15.18:0/2422163320 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff6600705d0 0x7ff660115a40 unknown :-1 s=CLOSED pgs=705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:27:21.603 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:21.605+0000 7ff666a00700 1 --2- 172.21.15.18:0/2422163320 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff660070ee0 0x7ff660115f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:27:21.604 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:21.605+0000 7ff666a00700 1 -- 172.21.15.18:0/2422163320 >> 172.21.15.18:0/2422163320 conn(0x7ff6600f89b0 msgr2=0x7ff6600f9d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:27:21.604 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:21.606+0000 7ff666a00700 1 -- 172.21.15.18:0/2422163320 shutdown_connections 2024-04-12T21:27:21.604 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:21.606+0000 7ff666a00700 1 -- 172.21.15.18:0/2422163320 wait complete. 2024-04-12T21:27:21.604 INFO:teuthology.orchestra.run.smithi018.stderr:dumped fsmap epoch 31 2024-04-12T21:27:21.761 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:21.763+0000 7f685ec00700 1 -- 172.21.15.18:0/1305936519 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6858105010 msgr2=0x7f68581053e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:27:21.761 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:21.763+0000 7f685ec00700 1 --2- 172.21.15.18:0/1305936519 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6858105010 0x7f68581053e0 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7f6848004940 tx=0x7f6848009e30 comp rx=0 tx=0).stop 2024-04-12T21:27:21.761 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:21.763+0000 7f685ec00700 1 -- 172.21.15.18:0/1305936519 shutdown_connections 2024-04-12T21:27:21.761 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:21.763+0000 7f685ec00700 1 --2- 172.21.15.18:0/1305936519 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f68581059b0 0x7f68580ff020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:27:21.761 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:21.763+0000 7f685ec00700 1 --2- 172.21.15.18:0/1305936519 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6858105010 0x7f68581053e0 unknown :-1 s=CLOSED pgs=706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:27:21.761 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:21.764+0000 7f685ec00700 1 -- 172.21.15.18:0/1305936519 >> 172.21.15.18:0/1305936519 conn(0x7f68580faba0 msgr2=0x7f68580fcfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:27:21.761 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:21.764+0000 7f685ec00700 1 -- 172.21.15.18:0/1305936519 shutdown_connections 2024-04-12T21:27:21.761 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:21.764+0000 7f685ec00700 1 -- 172.21.15.18:0/1305936519 wait complete. 2024-04-12T21:27:21.762 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:21.764+0000 7f685ec00700 1 Processor -- start 2024-04-12T21:27:21.763 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:21.765+0000 7f685ec00700 1 -- start start 2024-04-12T21:27:21.763 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:21.765+0000 7f685ec00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6858105010 0x7f6858115da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:27:21.763 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:21.765+0000 7f685ec00700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f68581059b0 0x7f68581162e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:27:21.763 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:21.765+0000 7f685ec00700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6858116930 con 0x7f6858105010 2024-04-12T21:27:21.763 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:21.765+0000 7f685ec00700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f685810fd00 con 0x7f68581059b0 2024-04-12T21:27:21.763 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:21.765+0000 7f6857e00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6858105010 0x7f6858115da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:27:21.763 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:21.765+0000 7f6857e00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6858105010 0x7f6858115da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:55706/0 (socket says 172.21.15.18:55706) 2024-04-12T21:27:21.763 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:21.765+0000 7f6857e00700 1 -- 172.21.15.18:0/917466267 learned_addr learned my addr 172.21.15.18:0/917466267 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:27:21.763 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:21.766+0000 7f6857e00700 1 -- 172.21.15.18:0/917466267 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f68581059b0 msgr2=0x7f68581162e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:27:21.763 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:21.766+0000 7f6857e00700 1 --2- 172.21.15.18:0/917466267 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f68581059b0 0x7f68581162e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:27:21.763 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:21.766+0000 7f6857e00700 1 -- 172.21.15.18:0/917466267 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6848005040 con 0x7f6858105010 2024-04-12T21:27:21.763 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:21.766+0000 7f6857e00700 1 --2- 172.21.15.18:0/917466267 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6858105010 0x7f6858115da0 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7f6848009180 tx=0x7f6848009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:27:21.764 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:21.766+0000 7f684fe00700 1 -- 172.21.15.18:0/917466267 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f684800e040 con 0x7f6858105010 2024-04-12T21:27:21.764 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:21.766+0000 7f684fe00700 1 -- 172.21.15.18:0/917466267 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f684800bbc0 con 0x7f6858105010 2024-04-12T21:27:21.764 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:21.767+0000 7f684fe00700 1 -- 172.21.15.18:0/917466267 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f684801c630 con 0x7f6858105010 2024-04-12T21:27:21.764 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:21.767+0000 7f685ec00700 1 -- 172.21.15.18:0/917466267 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f685810ff80 con 0x7f6858105010 2024-04-12T21:27:21.764 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:21.767+0000 7f685ec00700 1 -- 172.21.15.18:0/917466267 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6858110540 con 0x7f6858105010 2024-04-12T21:27:21.765 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:21.768+0000 7f685ec00700 1 -- 172.21.15.18:0/917466267 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6858100760 con 0x7f6858105010 2024-04-12T21:27:21.765 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:21.769+0000 7f684fe00700 1 -- 172.21.15.18:0/917466267 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7f684800bd30 con 0x7f6858105010 2024-04-12T21:27:21.766 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:21.770+0000 7f684fe00700 1 --2- 172.21.15.18:0/917466267 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f683c06ec10 0x7f683c0710c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:27:21.766 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:21.770+0000 7f684fe00700 1 -- 172.21.15.18:0/917466267 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 6757+0+0 (secure 0 0 0) 0x7f684808fd40 con 0x7f6858105010 2024-04-12T21:27:21.767 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:21.770+0000 7f6857400700 1 --2- 172.21.15.18:0/917466267 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f683c06ec10 0x7f683c0710c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:27:21.767 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:21.771+0000 7f6857400700 1 --2- 172.21.15.18:0/917466267 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f683c06ec10 0x7f683c0710c0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f6858111640 tx=0x7f6844007d40 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:27:21.771 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:21.774+0000 7f684fe00700 1 -- 172.21.15.18:0/917466267 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f684805c570 con 0x7f6858105010 2024-04-12T21:27:21.851 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:27:21 smithi018 ceph-mon[112590]: from='client.35460 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:27:21.851 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:27:21 smithi018 ceph-mon[112590]: from='client.35464 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:27:21.851 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:27:21 smithi018 ceph-mon[112590]: from='client.? 172.21.15.18:0/1017957372' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T21:27:22.026 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:22.029+0000 7f685ec00700 1 -- 172.21.15.18:0/917466267 --> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f685810c180 con 0x7f683c06ec10 2024-04-12T21:27:22.028 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:22.031+0000 7f684fe00700 1 -- 172.21.15.18:0/917466267 <== mgr.34102 v2:172.21.15.18:6800/3812891919 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f685810c180 con 0x7f683c06ec10 2024-04-12T21:27:22.029 INFO:teuthology.orchestra.run.smithi018.stdout:{ 2024-04-12T21:27:22.029 INFO:teuthology.orchestra.run.smithi018.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:f4452def3be4d350f4a8e5feaf7ebb1511be9298fe68478ba7a2d6fbd7d69a63", 2024-04-12T21:27:22.029 INFO:teuthology.orchestra.run.smithi018.stdout: "in_progress": true, 2024-04-12T21:27:22.029 INFO:teuthology.orchestra.run.smithi018.stdout: "which": "Upgrading all daemon types on all hosts", 2024-04-12T21:27:22.029 INFO:teuthology.orchestra.run.smithi018.stdout: "services_complete": [ 2024-04-12T21:27:22.029 INFO:teuthology.orchestra.run.smithi018.stdout: "crash", 2024-04-12T21:27:22.029 INFO:teuthology.orchestra.run.smithi018.stdout: "osd", 2024-04-12T21:27:22.029 INFO:teuthology.orchestra.run.smithi018.stdout: "mon", 2024-04-12T21:27:22.029 INFO:teuthology.orchestra.run.smithi018.stdout: "mgr" 2024-04-12T21:27:22.030 INFO:teuthology.orchestra.run.smithi018.stdout: ], 2024-04-12T21:27:22.030 INFO:teuthology.orchestra.run.smithi018.stdout: "progress": "15/21 daemons upgraded", 2024-04-12T21:27:22.030 INFO:teuthology.orchestra.run.smithi018.stdout: "message": "Currently upgrading mds daemons", 2024-04-12T21:27:22.030 INFO:teuthology.orchestra.run.smithi018.stdout: "is_paused": false 2024-04-12T21:27:22.030 INFO:teuthology.orchestra.run.smithi018.stdout:} 2024-04-12T21:27:22.030 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:27:21 smithi186 ceph-mon[86379]: from='client.35460 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:27:22.030 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:27:21 smithi186 ceph-mon[86379]: from='client.35464 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:27:22.030 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:27:21 smithi186 ceph-mon[86379]: from='client.? 172.21.15.18:0/1017957372' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T21:27:22.031 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:22.034+0000 7f685ec00700 1 -- 172.21.15.18:0/917466267 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f683c06ec10 msgr2=0x7f683c0710c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:27:22.031 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:22.034+0000 7f685ec00700 1 --2- 172.21.15.18:0/917466267 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f683c06ec10 0x7f683c0710c0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f6858111640 tx=0x7f6844007d40 comp rx=0 tx=0).stop 2024-04-12T21:27:22.031 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:22.034+0000 7f685ec00700 1 -- 172.21.15.18:0/917466267 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6858105010 msgr2=0x7f6858115da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:27:22.031 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:22.034+0000 7f685ec00700 1 --2- 172.21.15.18:0/917466267 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6858105010 0x7f6858115da0 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7f6848009180 tx=0x7f6848009e30 comp rx=0 tx=0).stop 2024-04-12T21:27:22.032 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:22.034+0000 7f685ec00700 1 -- 172.21.15.18:0/917466267 shutdown_connections 2024-04-12T21:27:22.032 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:22.034+0000 7f685ec00700 1 --2- 172.21.15.18:0/917466267 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f683c06ec10 0x7f683c0710c0 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:27:22.032 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:22.034+0000 7f685ec00700 1 --2- 172.21.15.18:0/917466267 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6858105010 0x7f6858115da0 unknown :-1 s=CLOSED pgs=707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:27:22.032 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:22.034+0000 7f685ec00700 1 --2- 172.21.15.18:0/917466267 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f68581059b0 0x7f68581162e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:27:22.032 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:22.034+0000 7f685ec00700 1 -- 172.21.15.18:0/917466267 >> 172.21.15.18:0/917466267 conn(0x7f68580faba0 msgr2=0x7f68580fbc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:27:22.032 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:22.035+0000 7f685ec00700 1 -- 172.21.15.18:0/917466267 shutdown_connections 2024-04-12T21:27:22.032 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:22.035+0000 7f685ec00700 1 -- 172.21.15.18:0/917466267 wait complete. 2024-04-12T21:27:22.193 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:22.196+0000 7f913e400700 1 -- 172.21.15.18:0/1167116041 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f91381057d0 msgr2=0x7f91380fee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:27:22.194 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:22.196+0000 7f913e400700 1 --2- 172.21.15.18:0/1167116041 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f91381057d0 0x7f91380fee40 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7f9124004940 tx=0x7f9124009e30 comp rx=0 tx=0).stop 2024-04-12T21:27:22.194 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:22.196+0000 7f913e400700 1 -- 172.21.15.18:0/1167116041 shutdown_connections 2024-04-12T21:27:22.194 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:22.196+0000 7f913e400700 1 --2- 172.21.15.18:0/1167116041 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f91381057d0 0x7f91380fee40 unknown :-1 s=CLOSED pgs=708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:27:22.194 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:22.196+0000 7f913e400700 1 --2- 172.21.15.18:0/1167116041 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9138104e30 0x7f9138105200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:27:22.194 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:22.196+0000 7f913e400700 1 -- 172.21.15.18:0/1167116041 >> 172.21.15.18:0/1167116041 conn(0x7f91380fab10 msgr2=0x7f91380fcef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:27:22.194 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:22.197+0000 7f913e400700 1 -- 172.21.15.18:0/1167116041 shutdown_connections 2024-04-12T21:27:22.194 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:22.197+0000 7f913e400700 1 -- 172.21.15.18:0/1167116041 wait complete. 2024-04-12T21:27:22.195 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:22.197+0000 7f913e400700 1 Processor -- start 2024-04-12T21:27:22.196 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:22.198+0000 7f913e400700 1 -- start start 2024-04-12T21:27:22.196 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:22.198+0000 7f913e400700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9138104e30 0x7f9138115ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:27:22.196 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:22.198+0000 7f913e400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f91381057d0 0x7f91381160e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:27:22.196 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:22.198+0000 7f913e400700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9138116730 con 0x7f91381057d0 2024-04-12T21:27:22.196 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:22.198+0000 7f913e400700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f913810fb20 con 0x7f9138104e30 2024-04-12T21:27:22.196 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:22.198+0000 7f9137e00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f91381057d0 0x7f91381160e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:27:22.196 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:22.198+0000 7f9137e00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f91381057d0 0x7f91381160e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:55728/0 (socket says 172.21.15.18:55728) 2024-04-12T21:27:22.196 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:22.198+0000 7f9137e00700 1 -- 172.21.15.18:0/4148896846 learned_addr learned my addr 172.21.15.18:0/4148896846 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:27:22.196 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:22.199+0000 7f913d000700 1 --2- 172.21.15.18:0/4148896846 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9138104e30 0x7f9138115ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:27:22.196 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:22.199+0000 7f9137e00700 1 -- 172.21.15.18:0/4148896846 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9138104e30 msgr2=0x7f9138115ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:27:22.196 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:22.199+0000 7f9137e00700 1 --2- 172.21.15.18:0/4148896846 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9138104e30 0x7f9138115ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:27:22.196 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:22.199+0000 7f9137e00700 1 -- 172.21.15.18:0/4148896846 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9124005040 con 0x7f91381057d0 2024-04-12T21:27:22.196 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:22.199+0000 7f913d000700 1 --2- 172.21.15.18:0/4148896846 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9138104e30 0x7f9138115ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-12T21:27:22.196 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:22.199+0000 7f9137e00700 1 --2- 172.21.15.18:0/4148896846 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f91381057d0 0x7f91381160e0 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7f9124009100 tx=0x7f9124009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:27:22.197 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:22.199+0000 7f9134c00700 1 -- 172.21.15.18:0/4148896846 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f912400e040 con 0x7f91381057d0 2024-04-12T21:27:22.197 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:22.199+0000 7f9134c00700 1 -- 172.21.15.18:0/4148896846 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f9124013420 con 0x7f91381057d0 2024-04-12T21:27:22.197 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:22.199+0000 7f913e400700 1 -- 172.21.15.18:0/4148896846 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f913810fda0 con 0x7f91381057d0 2024-04-12T21:27:22.197 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:22.200+0000 7f9134c00700 1 -- 172.21.15.18:0/4148896846 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f912401c630 con 0x7f91381057d0 2024-04-12T21:27:22.198 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:22.200+0000 7f913e400700 1 -- 172.21.15.18:0/4148896846 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9138110360 con 0x7f91381057d0 2024-04-12T21:27:22.198 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:22.201+0000 7f913e400700 1 -- 172.21.15.18:0/4148896846 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9138107300 con 0x7f91381057d0 2024-04-12T21:27:22.198 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:22.202+0000 7f9134c00700 1 -- 172.21.15.18:0/4148896846 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7f9124013590 con 0x7f91381057d0 2024-04-12T21:27:22.200 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:22.203+0000 7f9134c00700 1 --2- 172.21.15.18:0/4148896846 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f912006ec00 0x7f91200710b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:27:22.200 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:22.203+0000 7f9134c00700 1 -- 172.21.15.18:0/4148896846 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 6757+0+0 (secure 0 0 0) 0x7f912408fba0 con 0x7f91381057d0 2024-04-12T21:27:22.200 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:22.204+0000 7f913d000700 1 --2- 172.21.15.18:0/4148896846 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f912006ec00 0x7f91200710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:27:22.201 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:22.204+0000 7f913d000700 1 --2- 172.21.15.18:0/4148896846 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f912006ec00 0x7f91200710b0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f912c007fd0 tx=0x7f912c009040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:27:22.204 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:22.208+0000 7f9134c00700 1 -- 172.21.15.18:0/4148896846 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f912405c3d0 con 0x7f91381057d0 2024-04-12T21:27:22.529 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:22.532+0000 7f913e400700 1 -- 172.21.15.18:0/4148896846 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f91381111f0 con 0x7f91381057d0 2024-04-12T21:27:22.529 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:22.533+0000 7f9134c00700 1 -- 172.21.15.18:0/4148896846 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+4303 (secure 0 0 0) 0x7f9124058030 con 0x7f91381057d0 2024-04-12T21:27:22.530 INFO:teuthology.orchestra.run.smithi018.stdout:HEALTH_ERR failed to probe daemons or devices; 1 filesystem is degraded; 1 filesystem is offline 2024-04-12T21:27:22.530 INFO:teuthology.orchestra.run.smithi018.stdout:[WRN] CEPHADM_REFRESH_FAILED: failed to probe daemons or devices 2024-04-12T21:27:22.530 INFO:teuthology.orchestra.run.smithi018.stdout: host smithi018 `cephadm ceph-volume` failed: cephadm exited with an error code: 1, stderr: Inferring config /var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/mon.smithi018/config 2024-04-12T21:27:22.530 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/podman: timeout after 895 seconds 2024-04-12T21:27:22.530 INFO:teuthology.orchestra.run.smithi018.stdout:Non-zero exit code 124 from /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:f4452def3be4d350f4a8e5feaf7ebb1511be9298fe68478ba7a2d6fbd7d69a63 -e NODE_NAME=smithi018 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968:/var/run/ceph:z -v /var/log/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968:/var/log/ceph:z -v /var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmpxsmwkkuh:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:f4452def3be4d350f4a8e5feaf7ebb1511be9298fe68478ba7a2d6fbd7d69a63 inventory --format=json-pretty --filter-for-batch 2024-04-12T21:27:22.530 INFO:teuthology.orchestra.run.smithi018.stdout:Traceback (most recent call last): 2024-04-12T21:27:22.530 INFO:teuthology.orchestra.run.smithi018.stdout: File "/usr/lib64/python3.9/runpy.py", line 197, in _run_module_as_main 2024-04-12T21:27:22.530 INFO:teuthology.orchestra.run.smithi018.stdout: return _run_code(code, main_globals, None, 2024-04-12T21:27:22.530 INFO:teuthology.orchestra.run.smithi018.stdout: File "/usr/lib64/python3.9/runpy.py", line 87, in _run_code 2024-04-12T21:27:22.530 INFO:teuthology.orchestra.run.smithi018.stdout: exec(code, run_globals) 2024-04-12T21:27:22.530 INFO:teuthology.orchestra.run.smithi018.stdout: File "/var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/cephadm.e44f3ed5e34e4ef52eedef233d1190bff004470d8e8005127b5312bff357fa7f/__main__.py", line 5618, in 2024-04-12T21:27:22.531 INFO:teuthology.orchestra.run.smithi018.stdout: File "/var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/cephadm.e44f3ed5e34e4ef52eedef233d1190bff004470d8e8005127b5312bff357fa7f/__main__.py", line 5606, in main 2024-04-12T21:27:22.531 INFO:teuthology.orchestra.run.smithi018.stdout: File "/var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/cephadm.e44f3ed5e34e4ef52eedef233d1190bff004470d8e8005127b5312bff357fa7f/__main__.py", line 408, in _infer_config 2024-04-12T21:27:22.531 INFO:teuthology.orchestra.run.smithi018.stdout: File "/var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/cephadm.e44f3ed5e34e4ef52eedef233d1190bff004470d8e8005127b5312bff357fa7f/__main__.py", line 323, in _infer_fsid 2024-04-12T21:27:22.531 INFO:teuthology.orchestra.run.smithi018.stdout: File "/var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/cephadm.e44f3ed5e34e4ef52eedef233d1190bff004470d8e8005127b5312bff357fa7f/__main__.py", line 436, in _infer_image 2024-04-12T21:27:22.531 INFO:teuthology.orchestra.run.smithi018.stdout: File "/var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/cephadm.e44f3ed5e34e4ef52eedef233d1190bff004470d8e8005127b5312bff357fa7f/__main__.py", line 310, in _validate_fsid 2024-04-12T21:27:22.531 INFO:teuthology.orchestra.run.smithi018.stdout: File "/var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/cephadm.e44f3ed5e34e4ef52eedef233d1190bff004470d8e8005127b5312bff357fa7f/__main__.py", line 3359, in command_ceph_volume 2024-04-12T21:27:22.531 INFO:teuthology.orchestra.run.smithi018.stdout: File "/var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/cephadm.e44f3ed5e34e4ef52eedef233d1190bff004470d8e8005127b5312bff357fa7f/cephadmlib/call_wrappers.py", line 310, in call_throws 2024-04-12T21:27:22.531 INFO:teuthology.orchestra.run.smithi018.stdout:RuntimeError: Failed command: /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:f4452def3be4d350f4a8e5feaf7ebb1511be9298fe68478ba7a2d6fbd7d69a63 -e NODE_NAME=smithi018 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968:/var/run/ceph:z -v /var/log/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968:/var/log/ceph:z -v /var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmpxsmwkkuh:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:f4452def3be4d350f4a8e5feaf7ebb1511be9298fe68478ba7a2d6fbd7d69a63 inventory --format=json-pretty --filter-for-batch 2024-04-12T21:27:22.531 INFO:teuthology.orchestra.run.smithi018.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-04-12T21:27:22.531 INFO:teuthology.orchestra.run.smithi018.stdout: fs cephfs is degraded 2024-04-12T21:27:22.531 INFO:teuthology.orchestra.run.smithi018.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-04-12T21:27:22.531 INFO:teuthology.orchestra.run.smithi018.stdout: fs cephfs is offline because no MDS is active for it. 2024-04-12T21:27:22.532 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:22.535+0000 7f913e400700 1 -- 172.21.15.18:0/4148896846 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f912006ec00 msgr2=0x7f91200710b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:27:22.532 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:22.535+0000 7f913e400700 1 --2- 172.21.15.18:0/4148896846 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f912006ec00 0x7f91200710b0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f912c007fd0 tx=0x7f912c009040 comp rx=0 tx=0).stop 2024-04-12T21:27:22.532 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:22.535+0000 7f913e400700 1 -- 172.21.15.18:0/4148896846 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f91381057d0 msgr2=0x7f91381160e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:27:22.533 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:22.535+0000 7f913e400700 1 --2- 172.21.15.18:0/4148896846 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f91381057d0 0x7f91381160e0 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7f9124009100 tx=0x7f9124009e30 comp rx=0 tx=0).stop 2024-04-12T21:27:22.533 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:22.536+0000 7f913e400700 1 -- 172.21.15.18:0/4148896846 shutdown_connections 2024-04-12T21:27:22.533 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:22.536+0000 7f913e400700 1 --2- 172.21.15.18:0/4148896846 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f912006ec00 0x7f91200710b0 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:27:22.533 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:22.536+0000 7f913e400700 1 --2- 172.21.15.18:0/4148896846 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9138104e30 0x7f9138115ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:27:22.533 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:22.536+0000 7f913e400700 1 --2- 172.21.15.18:0/4148896846 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f91381057d0 0x7f91381160e0 unknown :-1 s=CLOSED pgs=709 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:27:22.533 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:22.536+0000 7f913e400700 1 -- 172.21.15.18:0/4148896846 >> 172.21.15.18:0/4148896846 conn(0x7f91380fab10 msgr2=0x7f91380fda00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:27:22.533 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:22.536+0000 7f913e400700 1 -- 172.21.15.18:0/4148896846 shutdown_connections 2024-04-12T21:27:22.533 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:22.536+0000 7f913e400700 1 -- 172.21.15.18:0/4148896846 wait complete. 2024-04-12T21:27:22.851 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:27:22 smithi018 ceph-mon[112590]: pgmap v828: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:27:22.851 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:27:22 smithi018 ceph-mon[112590]: from='client.? 172.21.15.18:0/2422163320' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-12T21:27:22.851 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:27:22 smithi018 ceph-mon[112590]: from='client.? 172.21.15.18:0/4148896846' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-12T21:27:23.030 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:27:22 smithi186 ceph-mon[86379]: pgmap v828: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:27:23.030 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:27:22 smithi186 ceph-mon[86379]: from='client.? 172.21.15.18:0/2422163320' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-12T21:27:23.030 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:27:22 smithi186 ceph-mon[86379]: from='client.? 172.21.15.18:0/4148896846' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-12T21:27:24.030 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:27:23 smithi186 ceph-mon[86379]: from='client.35476 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:27:24.101 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:27:23 smithi018 ceph-mon[112590]: from='client.35476 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:27:25.030 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:27:24 smithi186 ceph-mon[86379]: pgmap v829: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:27:25.101 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:27:24 smithi018 ceph-mon[112590]: pgmap v829: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:27:25.851 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:27:25 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T21:27:26.030 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:27:25 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T21:27:27.030 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:27:26 smithi186 ceph-mon[86379]: pgmap v830: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:27:27.101 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:27:26 smithi018 ceph-mon[112590]: pgmap v830: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:27:29.030 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:27:28 smithi186 ceph-mon[86379]: pgmap v831: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:27:29.101 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:27:28 smithi018 ceph-mon[112590]: pgmap v831: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:27:30.901 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:27:30 smithi186 ceph-mon[86379]: pgmap v832: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:27:31.101 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:27:30 smithi018 ceph-mon[112590]: pgmap v832: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:27:33.030 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:27:32 smithi186 ceph-mon[86379]: pgmap v833: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:27:33.101 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:27:32 smithi018 ceph-mon[112590]: pgmap v833: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:27:35.030 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:27:34 smithi186 ceph-mon[86379]: pgmap v834: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:27:35.101 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:27:34 smithi018 ceph-mon[112590]: pgmap v834: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:27:37.030 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:27:36 smithi186 ceph-mon[86379]: pgmap v835: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:27:37.101 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:27:36 smithi018 ceph-mon[112590]: pgmap v835: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:27:39.030 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:27:38 smithi186 ceph-mon[86379]: pgmap v836: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:27:39.101 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:27:38 smithi018 ceph-mon[112590]: pgmap v836: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:27:41.030 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:27:40 smithi186 ceph-mon[86379]: pgmap v837: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:27:41.031 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:27:40 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T21:27:41.101 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:27:40 smithi018 ceph-mon[112590]: pgmap v837: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:27:41.101 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:27:40 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T21:27:43.030 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:27:42 smithi186 ceph-mon[86379]: pgmap v838: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:27:43.101 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:27:42 smithi018 ceph-mon[112590]: pgmap v838: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:27:45.030 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:27:44 smithi186 ceph-mon[86379]: pgmap v839: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:27:45.101 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:27:44 smithi018 ceph-mon[112590]: pgmap v839: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:27:47.030 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:27:46 smithi186 ceph-mon[86379]: pgmap v840: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:27:47.101 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:27:46 smithi018 ceph-mon[112590]: pgmap v840: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:27:49.030 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:27:48 smithi186 ceph-mon[86379]: pgmap v841: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:27:49.101 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:27:48 smithi018 ceph-mon[112590]: pgmap v841: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:27:51.030 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:27:50 smithi186 ceph-mon[86379]: pgmap v842: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:27:51.101 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:27:50 smithi018 ceph-mon[112590]: pgmap v842: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:27:52.707 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:52.708+0000 7f03f5600700 1 -- 172.21.15.18:0/2454873523 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f03f0070c10 msgr2=0x7f03f0109190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:27:52.707 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:52.708+0000 7f03f5600700 1 --2- 172.21.15.18:0/2454873523 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f03f0070c10 0x7f03f0109190 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7f03d8004970 tx=0x7f03d8009e30 comp rx=0 tx=0).stop 2024-04-12T21:27:52.707 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:52.709+0000 7f03f5600700 1 -- 172.21.15.18:0/2454873523 shutdown_connections 2024-04-12T21:27:52.707 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:52.709+0000 7f03f5600700 1 --2- 172.21.15.18:0/2454873523 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f03f0070c10 0x7f03f0109190 unknown :-1 s=CLOSED pgs=710 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:27:52.708 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:52.709+0000 7f03f5600700 1 --2- 172.21.15.18:0/2454873523 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f03f0070300 0x7f03f00706d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:27:52.708 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:52.709+0000 7f03f5600700 1 -- 172.21.15.18:0/2454873523 >> 172.21.15.18:0/2454873523 conn(0x7f03f00f8970 msgr2=0x7f03f00fad80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:27:52.708 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:52.709+0000 7f03f5600700 1 -- 172.21.15.18:0/2454873523 shutdown_connections 2024-04-12T21:27:52.708 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:52.709+0000 7f03f5600700 1 -- 172.21.15.18:0/2454873523 wait complete. 2024-04-12T21:27:52.708 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:52.710+0000 7f03f5600700 1 Processor -- start 2024-04-12T21:27:52.708 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:52.710+0000 7f03f5600700 1 -- start start 2024-04-12T21:27:52.709 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:52.711+0000 7f03f5600700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f03f0070300 0x7f03f0115da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:27:52.710 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:52.711+0000 7f03f5600700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f03f0070c10 0x7f03f01162e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:27:52.710 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:52.711+0000 7f03f5600700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03f0116930 con 0x7f03f0070300 2024-04-12T21:27:52.710 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:52.711+0000 7f03f5600700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03f010fd00 con 0x7f03f0070c10 2024-04-12T21:27:52.710 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:52.711+0000 7f03eea00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f03f0070300 0x7f03f0115da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:27:52.710 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:52.711+0000 7f03eea00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f03f0070300 0x7f03f0115da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:51580/0 (socket says 172.21.15.18:51580) 2024-04-12T21:27:52.710 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:52.711+0000 7f03eea00700 1 -- 172.21.15.18:0/1585740786 learned_addr learned my addr 172.21.15.18:0/1585740786 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:27:52.710 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:52.712+0000 7f03e6000700 1 --2- 172.21.15.18:0/1585740786 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f03f0070c10 0x7f03f01162e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:27:52.710 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:52.712+0000 7f03eea00700 1 -- 172.21.15.18:0/1585740786 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f03f0070c10 msgr2=0x7f03f01162e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:27:52.710 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:52.712+0000 7f03eea00700 1 --2- 172.21.15.18:0/1585740786 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f03f0070c10 0x7f03f01162e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:27:52.710 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:52.712+0000 7f03eea00700 1 -- 172.21.15.18:0/1585740786 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03d8005040 con 0x7f03f0070300 2024-04-12T21:27:52.711 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:52.712+0000 7f03eea00700 1 --2- 172.21.15.18:0/1585740786 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f03f0070300 0x7f03f0115da0 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7f03e000c8c0 tx=0x7f03e00044c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:27:52.711 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:52.712+0000 7f03e7400700 1 -- 172.21.15.18:0/1585740786 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f03e000a980 con 0x7f03f0070300 2024-04-12T21:27:52.711 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:52.713+0000 7f03f5600700 1 -- 172.21.15.18:0/1585740786 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f03f010ffe0 con 0x7f03f0070300 2024-04-12T21:27:52.711 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:52.713+0000 7f03e7400700 1 -- 172.21.15.18:0/1585740786 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f03e000ce60 con 0x7f03f0070300 2024-04-12T21:27:52.711 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:52.713+0000 7f03e7400700 1 -- 172.21.15.18:0/1585740786 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f03e001b5e0 con 0x7f03f0070300 2024-04-12T21:27:52.711 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:52.713+0000 7f03f5600700 1 -- 172.21.15.18:0/1585740786 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f03f01104f0 con 0x7f03f0070300 2024-04-12T21:27:52.712 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:52.714+0000 7f03f5600700 1 -- 172.21.15.18:0/1585740786 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f03f0071a40 con 0x7f03f0070300 2024-04-12T21:27:52.713 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:52.715+0000 7f03e7400700 1 -- 172.21.15.18:0/1585740786 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7f03e0012400 con 0x7f03f0070300 2024-04-12T21:27:52.714 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:52.716+0000 7f03e7400700 1 --2- 172.21.15.18:0/1585740786 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f03d406ec10 0x7f03d40710c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:27:52.714 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:52.716+0000 7f03e7400700 1 -- 172.21.15.18:0/1585740786 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 6757+0+0 (secure 0 0 0) 0x7f03e008eb30 con 0x7f03f0070300 2024-04-12T21:27:52.714 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:52.716+0000 7f03e6000700 1 --2- 172.21.15.18:0/1585740786 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f03d406ec10 0x7f03d40710c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:27:52.715 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:52.717+0000 7f03e6000700 1 --2- 172.21.15.18:0/1585740786 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f03d406ec10 0x7f03d40710c0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f03d8009fd0 tx=0x7f03d8009e30 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:27:52.719 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:52.721+0000 7f03e7400700 1 -- 172.21.15.18:0/1585740786 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f03e005b2e0 con 0x7f03f0070300 2024-04-12T21:27:52.973 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:52.975+0000 7f03f5600700 1 -- 172.21.15.18:0/1585740786 --> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f03f004bf40 con 0x7f03d406ec10 2024-04-12T21:27:52.973 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:27:52 smithi018 ceph-mon[112590]: pgmap v843: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:27:52.976 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:52.978+0000 7f03e7400700 1 -- 172.21.15.18:0/1585740786 <== mgr.34102 v2:172.21.15.18:6800/3812891919 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f03f004bf40 con 0x7f03d406ec10 2024-04-12T21:27:52.979 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:52.980+0000 7f03f5600700 1 -- 172.21.15.18:0/1585740786 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f03d406ec10 msgr2=0x7f03d40710c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:27:52.979 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:52.980+0000 7f03f5600700 1 --2- 172.21.15.18:0/1585740786 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f03d406ec10 0x7f03d40710c0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f03d8009fd0 tx=0x7f03d8009e30 comp rx=0 tx=0).stop 2024-04-12T21:27:52.979 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:52.980+0000 7f03f5600700 1 -- 172.21.15.18:0/1585740786 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f03f0070300 msgr2=0x7f03f0115da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:27:52.979 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:52.980+0000 7f03f5600700 1 --2- 172.21.15.18:0/1585740786 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f03f0070300 0x7f03f0115da0 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7f03e000c8c0 tx=0x7f03e00044c0 comp rx=0 tx=0).stop 2024-04-12T21:27:52.979 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:52.981+0000 7f03f5600700 1 -- 172.21.15.18:0/1585740786 shutdown_connections 2024-04-12T21:27:52.979 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:52.981+0000 7f03f5600700 1 --2- 172.21.15.18:0/1585740786 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f03d406ec10 0x7f03d40710c0 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:27:52.979 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:52.981+0000 7f03f5600700 1 --2- 172.21.15.18:0/1585740786 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f03f0070300 0x7f03f0115da0 unknown :-1 s=CLOSED pgs=711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:27:52.979 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:52.981+0000 7f03f5600700 1 --2- 172.21.15.18:0/1585740786 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f03f0070c10 0x7f03f01162e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:27:52.980 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:52.981+0000 7f03f5600700 1 -- 172.21.15.18:0/1585740786 >> 172.21.15.18:0/1585740786 conn(0x7f03f00f8970 msgr2=0x7f03f00f9d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:27:52.980 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:52.981+0000 7f03f5600700 1 -- 172.21.15.18:0/1585740786 shutdown_connections 2024-04-12T21:27:52.980 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:52.981+0000 7f03f5600700 1 -- 172.21.15.18:0/1585740786 wait complete. 2024-04-12T21:27:52.995 INFO:teuthology.orchestra.run.smithi018.stdout:true 2024-04-12T21:27:53.030 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:27:52 smithi186 ceph-mon[86379]: pgmap v843: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:27:53.128 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:53.129+0000 7f7a71000700 1 -- 172.21.15.18:0/3784767601 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7a6c1035e0 msgr2=0x7f7a6c0fcc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:27:53.128 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:53.129+0000 7f7a71000700 1 --2- 172.21.15.18:0/3784767601 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7a6c1035e0 0x7f7a6c0fcc50 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7f7a58004940 tx=0x7f7a58009e30 comp rx=0 tx=0).stop 2024-04-12T21:27:53.128 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:53.129+0000 7f7a71000700 1 -- 172.21.15.18:0/3784767601 shutdown_connections 2024-04-12T21:27:53.128 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:53.129+0000 7f7a71000700 1 --2- 172.21.15.18:0/3784767601 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7a6c1035e0 0x7f7a6c0fcc50 unknown :-1 s=CLOSED pgs=712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:27:53.128 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:53.129+0000 7f7a71000700 1 --2- 172.21.15.18:0/3784767601 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7a6c102c40 0x7f7a6c103010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:27:53.128 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:53.129+0000 7f7a71000700 1 -- 172.21.15.18:0/3784767601 >> 172.21.15.18:0/3784767601 conn(0x7f7a6c0f8830 msgr2=0x7f7a6c0fac40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:27:53.128 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:53.130+0000 7f7a71000700 1 -- 172.21.15.18:0/3784767601 shutdown_connections 2024-04-12T21:27:53.128 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:53.130+0000 7f7a71000700 1 -- 172.21.15.18:0/3784767601 wait complete. 2024-04-12T21:27:53.130 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:53.130+0000 7f7a71000700 1 Processor -- start 2024-04-12T21:27:53.130 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:53.131+0000 7f7a71000700 1 -- start start 2024-04-12T21:27:53.130 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:53.131+0000 7f7a71000700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7a6c102c40 0x7f7a6c115c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:27:53.130 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:53.131+0000 7f7a71000700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7a6c1035e0 0x7f7a6c116150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:27:53.130 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:53.131+0000 7f7a71000700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a6c1167a0 con 0x7f7a6c1035e0 2024-04-12T21:27:53.130 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:53.131+0000 7f7a71000700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a6c10fb90 con 0x7f7a6c102c40 2024-04-12T21:27:53.130 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:53.131+0000 7f7a6aa00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7a6c1035e0 0x7f7a6c116150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:27:53.130 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:53.131+0000 7f7a6aa00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7a6c1035e0 0x7f7a6c116150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:51608/0 (socket says 172.21.15.18:51608) 2024-04-12T21:27:53.130 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:53.131+0000 7f7a6aa00700 1 -- 172.21.15.18:0/1169083912 learned_addr learned my addr 172.21.15.18:0/1169083912 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:27:53.130 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:53.132+0000 7f7a6b400700 1 --2- 172.21.15.18:0/1169083912 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7a6c102c40 0x7f7a6c115c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:27:53.131 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:53.132+0000 7f7a6aa00700 1 -- 172.21.15.18:0/1169083912 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7a6c102c40 msgr2=0x7f7a6c115c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:27:53.131 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:53.132+0000 7f7a6aa00700 1 --2- 172.21.15.18:0/1169083912 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7a6c102c40 0x7f7a6c115c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:27:53.131 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:53.132+0000 7f7a6aa00700 1 -- 172.21.15.18:0/1169083912 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a58005040 con 0x7f7a6c1035e0 2024-04-12T21:27:53.131 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:53.132+0000 7f7a6aa00700 1 --2- 172.21.15.18:0/1169083912 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7a6c1035e0 0x7f7a6c116150 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7f7a58009100 tx=0x7f7a58009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:27:53.132 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:53.132+0000 7f7a63400700 1 -- 172.21.15.18:0/1169083912 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7a5800e040 con 0x7f7a6c1035e0 2024-04-12T21:27:53.132 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:53.132+0000 7f7a63400700 1 -- 172.21.15.18:0/1169083912 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f7a58004290 con 0x7f7a6c1035e0 2024-04-12T21:27:53.132 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:53.132+0000 7f7a63400700 1 -- 172.21.15.18:0/1169083912 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7a5801c3f0 con 0x7f7a6c1035e0 2024-04-12T21:27:53.132 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:53.132+0000 7f7a71000700 1 -- 172.21.15.18:0/1169083912 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7a6c10fe10 con 0x7f7a6c1035e0 2024-04-12T21:27:53.132 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:53.133+0000 7f7a71000700 1 -- 172.21.15.18:0/1169083912 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7a6c1103d0 con 0x7f7a6c1035e0 2024-04-12T21:27:53.132 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:53.134+0000 7f7a71000700 1 -- 172.21.15.18:0/1169083912 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7a6c0fe390 con 0x7f7a6c1035e0 2024-04-12T21:27:53.133 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:53.135+0000 7f7a63400700 1 -- 172.21.15.18:0/1169083912 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7f7a58013790 con 0x7f7a6c1035e0 2024-04-12T21:27:53.134 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:53.136+0000 7f7a63400700 1 --2- 172.21.15.18:0/1169083912 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f7a5006ec10 0x7f7a500710c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:27:53.134 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:53.136+0000 7f7a63400700 1 -- 172.21.15.18:0/1169083912 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 6757+0+0 (secure 0 0 0) 0x7f7a5808fc30 con 0x7f7a6c1035e0 2024-04-12T21:27:53.134 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:53.136+0000 7f7a6b400700 1 --2- 172.21.15.18:0/1169083912 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f7a5006ec10 0x7f7a500710c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:27:53.135 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:53.137+0000 7f7a6b400700 1 --2- 172.21.15.18:0/1169083912 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f7a5006ec10 0x7f7a500710c0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f7a5c004ec0 tx=0x7f7a5c009040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:27:53.139 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:53.141+0000 7f7a63400700 1 -- 172.21.15.18:0/1169083912 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f7a5805c510 con 0x7f7a6c1035e0 2024-04-12T21:27:53.381 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:53.383+0000 7f7a71000700 1 -- 172.21.15.18:0/1169083912 --> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f7a6c109d10 con 0x7f7a5006ec10 2024-04-12T21:27:53.384 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:53.386+0000 7f7a63400700 1 -- 172.21.15.18:0/1169083912 <== mgr.34102 v2:172.21.15.18:6800/3812891919 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f7a6c109d10 con 0x7f7a5006ec10 2024-04-12T21:27:53.387 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:53.388+0000 7f7a71000700 1 -- 172.21.15.18:0/1169083912 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f7a5006ec10 msgr2=0x7f7a500710c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:27:53.387 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:53.388+0000 7f7a71000700 1 --2- 172.21.15.18:0/1169083912 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f7a5006ec10 0x7f7a500710c0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f7a5c004ec0 tx=0x7f7a5c009040 comp rx=0 tx=0).stop 2024-04-12T21:27:53.387 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:53.388+0000 7f7a71000700 1 -- 172.21.15.18:0/1169083912 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7a6c1035e0 msgr2=0x7f7a6c116150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:27:53.387 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:53.388+0000 7f7a71000700 1 --2- 172.21.15.18:0/1169083912 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7a6c1035e0 0x7f7a6c116150 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7f7a58009100 tx=0x7f7a58009e30 comp rx=0 tx=0).stop 2024-04-12T21:27:53.387 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:53.388+0000 7f7a71000700 1 -- 172.21.15.18:0/1169083912 shutdown_connections 2024-04-12T21:27:53.387 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:53.388+0000 7f7a71000700 1 --2- 172.21.15.18:0/1169083912 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f7a5006ec10 0x7f7a500710c0 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:27:53.387 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:53.388+0000 7f7a71000700 1 --2- 172.21.15.18:0/1169083912 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7a6c102c40 0x7f7a6c115c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:27:53.387 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:53.388+0000 7f7a71000700 1 --2- 172.21.15.18:0/1169083912 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7a6c1035e0 0x7f7a6c116150 unknown :-1 s=CLOSED pgs=713 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:27:53.388 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:53.388+0000 7f7a71000700 1 -- 172.21.15.18:0/1169083912 >> 172.21.15.18:0/1169083912 conn(0x7f7a6c0f8830 msgr2=0x7f7a6c0f97f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:27:53.388 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:53.389+0000 7f7a71000700 1 -- 172.21.15.18:0/1169083912 shutdown_connections 2024-04-12T21:27:53.388 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:53.389+0000 7f7a71000700 1 -- 172.21.15.18:0/1169083912 wait complete. 2024-04-12T21:27:53.532 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:53.534+0000 7f76b0a00700 1 -- 172.21.15.18:0/151019976 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f76ac070c70 msgr2=0x7f76ac108f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:27:53.532 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:53.534+0000 7f76b0a00700 1 --2- 172.21.15.18:0/151019976 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f76ac070c70 0x7f76ac108f40 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7f7698004970 tx=0x7f7698009e30 comp rx=0 tx=0).stop 2024-04-12T21:27:53.533 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:53.534+0000 7f76b0a00700 1 -- 172.21.15.18:0/151019976 shutdown_connections 2024-04-12T21:27:53.533 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:53.534+0000 7f76b0a00700 1 --2- 172.21.15.18:0/151019976 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f76ac070c70 0x7f76ac108f40 unknown :-1 s=CLOSED pgs=714 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:27:53.533 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:53.534+0000 7f76b0a00700 1 --2- 172.21.15.18:0/151019976 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f76ac070360 0x7f76ac070730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:27:53.533 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:53.534+0000 7f76b0a00700 1 -- 172.21.15.18:0/151019976 >> 172.21.15.18:0/151019976 conn(0x7f76ac0f8760 msgr2=0x7f76ac0fab70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:27:53.533 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:53.534+0000 7f76b0a00700 1 -- 172.21.15.18:0/151019976 shutdown_connections 2024-04-12T21:27:53.533 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:53.534+0000 7f76b0a00700 1 -- 172.21.15.18:0/151019976 wait complete. 2024-04-12T21:27:53.535 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:53.535+0000 7f76b0a00700 1 Processor -- start 2024-04-12T21:27:53.535 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:53.535+0000 7f76b0a00700 1 -- start start 2024-04-12T21:27:53.535 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:53.535+0000 7f76b0a00700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f76ac070360 0x7f76ac115c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:27:53.535 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:53.535+0000 7f76b0a00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f76ac070c70 0x7f76ac116140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:27:53.535 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:53.536+0000 7f76b0a00700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76ac116820 con 0x7f76ac070c70 2024-04-12T21:27:53.535 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:53.536+0000 7f76b0a00700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76ac10fb80 con 0x7f76ac070360 2024-04-12T21:27:53.535 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:53.536+0000 7f76a9e00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f76ac070c70 0x7f76ac116140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:27:53.535 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:53.536+0000 7f76a9e00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f76ac070c70 0x7f76ac116140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:51636/0 (socket says 172.21.15.18:51636) 2024-04-12T21:27:53.536 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:53.536+0000 7f76a9e00700 1 -- 172.21.15.18:0/2388335840 learned_addr learned my addr 172.21.15.18:0/2388335840 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:27:53.536 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:53.536+0000 7f76aa800700 1 --2- 172.21.15.18:0/2388335840 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f76ac070360 0x7f76ac115c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:27:53.536 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:53.536+0000 7f76a9e00700 1 -- 172.21.15.18:0/2388335840 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f76ac070360 msgr2=0x7f76ac115c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:27:53.536 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:53.536+0000 7f76a9e00700 1 --2- 172.21.15.18:0/2388335840 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f76ac070360 0x7f76ac115c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:27:53.536 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:53.536+0000 7f76a9e00700 1 -- 172.21.15.18:0/2388335840 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7698005040 con 0x7f76ac070c70 2024-04-12T21:27:53.536 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:53.537+0000 7f76a9e00700 1 --2- 172.21.15.18:0/2388335840 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f76ac070c70 0x7f76ac116140 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7f7698004f50 tx=0x7f7698009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:27:53.536 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:53.537+0000 7f76a2a00700 1 -- 172.21.15.18:0/2388335840 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f769800e040 con 0x7f76ac070c70 2024-04-12T21:27:53.536 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:53.537+0000 7f76a2a00700 1 -- 172.21.15.18:0/2388335840 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f769800bc10 con 0x7f76ac070c70 2024-04-12T21:27:53.536 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:53.537+0000 7f76b0a00700 1 -- 172.21.15.18:0/2388335840 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f76ac10fe00 con 0x7f76ac070c70 2024-04-12T21:27:53.536 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:53.537+0000 7f76a2a00700 1 -- 172.21.15.18:0/2388335840 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f769801c630 con 0x7f76ac070c70 2024-04-12T21:27:53.536 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:53.537+0000 7f76b0a00700 1 -- 172.21.15.18:0/2388335840 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f76ac110340 con 0x7f76ac070c70 2024-04-12T21:27:53.537 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:53.538+0000 7f76b0a00700 1 -- 172.21.15.18:0/2388335840 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f76ac071aa0 con 0x7f76ac070c70 2024-04-12T21:27:53.537 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:53.539+0000 7f76a2a00700 1 -- 172.21.15.18:0/2388335840 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7f7698013420 con 0x7f76ac070c70 2024-04-12T21:27:53.538 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:53.540+0000 7f76a2a00700 1 --2- 172.21.15.18:0/2388335840 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f769006ec00 0x7f76900710b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:27:53.538 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:53.540+0000 7f76a2a00700 1 -- 172.21.15.18:0/2388335840 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 6757+0+0 (secure 0 0 0) 0x7f769808fcb0 con 0x7f76ac070c70 2024-04-12T21:27:53.538 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:53.540+0000 7f76aa800700 1 --2- 172.21.15.18:0/2388335840 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f769006ec00 0x7f76900710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:27:53.541 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:53.541+0000 7f76aa800700 1 --2- 172.21.15.18:0/2388335840 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f769006ec00 0x7f76900710b0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f769c006fd0 tx=0x7f769c008040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:27:53.543 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:53.545+0000 7f76a2a00700 1 -- 172.21.15.18:0/2388335840 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f769805c460 con 0x7f76ac070c70 2024-04-12T21:27:53.784 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:53.785+0000 7f76b0a00700 1 -- 172.21.15.18:0/2388335840 --> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f76ac071aa0 con 0x7f769006ec00 2024-04-12T21:27:53.816 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:53.818+0000 7f76a2a00700 1 -- 172.21.15.18:0/2388335840 <== mgr.34102 v2:172.21.15.18:6800/3812891919 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7f76ac071aa0 con 0x7f769006ec00 2024-04-12T21:27:53.816 INFO:teuthology.orchestra.run.smithi018.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-12T21:27:53.816 INFO:teuthology.orchestra.run.smithi018.stdout:alertmanager.smithi018 smithi018 *:9093,9094 running (28m) 4m ago 42m 15.5M - 0.25.0 c8568f914cd2 c332090068e8 2024-04-12T21:27:53.816 INFO:teuthology.orchestra.run.smithi018.stdout:crash.smithi018 smithi018 running (24m) 4m ago 43m 6665k - 19.0.0-2879-g324baf8d 8717720dc5e5 b7f80d1f7982 2024-04-12T21:27:53.817 INFO:teuthology.orchestra.run.smithi018.stdout:crash.smithi186 smithi186 running (24m) 4m ago 41m 6660k - 19.0.0-2879-g324baf8d 8717720dc5e5 a4f55c81b623 2024-04-12T21:27:53.817 INFO:teuthology.orchestra.run.smithi018.stdout:grafana.smithi018 smithi018 *:3000 running (27m) 4m ago 42m 74.8M - 9.4.12 2bacad6d85d8 b08e3bf176dd 2024-04-12T21:27:53.817 INFO:teuthology.orchestra.run.smithi018.stdout:mds.cephfs.smithi018.acqbsc smithi018 running (20m) 4m ago 35m 16.7M - 19.0.0-2879-g324baf8d 8717720dc5e5 021f99ab24ae 2024-04-12T21:27:53.817 INFO:teuthology.orchestra.run.smithi018.stdout:mds.cephfs.smithi018.szjluo smithi018 running (20m) 4m ago 35m 16.5M - 19.0.0-2879-g324baf8d 8717720dc5e5 0003117479d4 2024-04-12T21:27:53.817 INFO:teuthology.orchestra.run.smithi018.stdout:mds.cephfs.smithi186.jdllyn smithi186 running (4m) 4m ago 35m 11.0M - 19.0.0-2879-g324baf8d 8717720dc5e5 21b1f46eef23 2024-04-12T21:27:53.817 INFO:teuthology.orchestra.run.smithi018.stdout:mds.cephfs.smithi186.wjnafs smithi186 running (35m) 4m ago 35m 19.5M - 17.2.7-827-gd3b5060a 21de9b7369ad 74b115d6d6fd 2024-04-12T21:27:53.817 INFO:teuthology.orchestra.run.smithi018.stdout:mgr.smithi018.xjzwga smithi018 *:8443,9283,8765 running (31m) 4m ago 44m 512M - 19.0.0-2879-g324baf8d 8717720dc5e5 56088305eab7 2024-04-12T21:27:53.817 INFO:teuthology.orchestra.run.smithi018.stdout:mgr.smithi186.kqcfxk smithi186 *:8443,9283,8765 running (29m) 4m ago 41m 459M - 19.0.0-2879-g324baf8d 8717720dc5e5 09cd766e5e0d 2024-04-12T21:27:53.817 INFO:teuthology.orchestra.run.smithi018.stdout:mon.smithi018 smithi018 running (26m) 4m ago 44m 73.1M 2048M 19.0.0-2879-g324baf8d 8717720dc5e5 df3e201dfed6 2024-04-12T21:27:53.817 INFO:teuthology.orchestra.run.smithi018.stdout:mon.smithi186 smithi186 running (25m) 4m ago 41m 47.2M 2048M 19.0.0-2879-g324baf8d 8717720dc5e5 c77498da9b2b 2024-04-12T21:27:53.817 INFO:teuthology.orchestra.run.smithi018.stdout:node-exporter.smithi018 smithi018 *:9100 running (29m) 4m ago 43m 8108k - 1.5.0 0da6a335fe13 326c3d9c6847 2024-04-12T21:27:53.817 INFO:teuthology.orchestra.run.smithi018.stdout:node-exporter.smithi186 smithi186 *:9100 running (29m) 4m ago 41m 8757k - 1.5.0 0da6a335fe13 ec2112baeecf 2024-04-12T21:27:53.817 INFO:teuthology.orchestra.run.smithi018.stdout:osd.0 smithi018 running (24m) 4m ago 40m 123M 1559M 19.0.0-2879-g324baf8d 8717720dc5e5 60b1273ca047 2024-04-12T21:27:53.817 INFO:teuthology.orchestra.run.smithi018.stdout:osd.1 smithi018 running (23m) 4m ago 40m 95.1M 1559M 19.0.0-2879-g324baf8d 8717720dc5e5 170ce1e5fb8d 2024-04-12T21:27:53.817 INFO:teuthology.orchestra.run.smithi018.stdout:osd.2 smithi018 running (23m) 4m ago 39m 94.3M 1559M 19.0.0-2879-g324baf8d 8717720dc5e5 c2cbc6aa2839 2024-04-12T21:27:53.818 INFO:teuthology.orchestra.run.smithi018.stdout:osd.3 smithi186 running (22m) 4m ago 38m 101M 2581M 19.0.0-2879-g324baf8d 8717720dc5e5 0138e8262e4b 2024-04-12T21:27:53.818 INFO:teuthology.orchestra.run.smithi018.stdout:osd.4 smithi186 running (21m) 4m ago 38m 91.0M 2581M 19.0.0-2879-g324baf8d 8717720dc5e5 b20cb63b25c9 2024-04-12T21:27:53.818 INFO:teuthology.orchestra.run.smithi018.stdout:osd.5 smithi186 running (21m) 4m ago 37m 92.0M 2581M 19.0.0-2879-g324baf8d 8717720dc5e5 2513bea2610e 2024-04-12T21:27:53.818 INFO:teuthology.orchestra.run.smithi018.stdout:prometheus.smithi018 smithi018 *:9095 running (28m) 4m ago 42m 67.9M - 2.43.0 a07b618ecd1d f58841127e21 2024-04-12T21:27:53.821 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:53.823+0000 7f76b0a00700 1 -- 172.21.15.18:0/2388335840 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f769006ec00 msgr2=0x7f76900710b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:27:53.821 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:53.823+0000 7f76b0a00700 1 --2- 172.21.15.18:0/2388335840 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f769006ec00 0x7f76900710b0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f769c006fd0 tx=0x7f769c008040 comp rx=0 tx=0).stop 2024-04-12T21:27:53.821 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:53.823+0000 7f76b0a00700 1 -- 172.21.15.18:0/2388335840 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f76ac070c70 msgr2=0x7f76ac116140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:27:53.821 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:53.823+0000 7f76b0a00700 1 --2- 172.21.15.18:0/2388335840 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f76ac070c70 0x7f76ac116140 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7f7698004f50 tx=0x7f7698009e30 comp rx=0 tx=0).stop 2024-04-12T21:27:53.822 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:53.823+0000 7f76b0a00700 1 -- 172.21.15.18:0/2388335840 shutdown_connections 2024-04-12T21:27:53.822 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:53.823+0000 7f76b0a00700 1 --2- 172.21.15.18:0/2388335840 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f769006ec00 0x7f76900710b0 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:27:53.822 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:53.823+0000 7f76b0a00700 1 --2- 172.21.15.18:0/2388335840 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f76ac070360 0x7f76ac115c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:27:53.822 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:53.823+0000 7f76b0a00700 1 --2- 172.21.15.18:0/2388335840 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f76ac070c70 0x7f76ac116140 unknown :-1 s=CLOSED pgs=715 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:27:53.822 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:53.823+0000 7f76b0a00700 1 -- 172.21.15.18:0/2388335840 >> 172.21.15.18:0/2388335840 conn(0x7f76ac0f8760 msgr2=0x7f76ac0f9bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:27:53.822 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:53.824+0000 7f76b0a00700 1 -- 172.21.15.18:0/2388335840 shutdown_connections 2024-04-12T21:27:53.822 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:53.824+0000 7f76b0a00700 1 -- 172.21.15.18:0/2388335840 wait complete. 2024-04-12T21:27:53.979 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:53.981+0000 7efdd1c00700 1 -- 172.21.15.18:0/2097763183 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7efdcc1059c0 msgr2=0x7efdcc0ff080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:27:53.979 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:53.981+0000 7efdd1c00700 1 --2- 172.21.15.18:0/2097763183 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7efdcc1059c0 0x7efdcc0ff080 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7efdb8004940 tx=0x7efdb8009e30 comp rx=0 tx=0).stop 2024-04-12T21:27:53.979 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:53.981+0000 7efdd1c00700 1 -- 172.21.15.18:0/2097763183 shutdown_connections 2024-04-12T21:27:53.980 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:53.981+0000 7efdd1c00700 1 --2- 172.21.15.18:0/2097763183 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7efdcc1059c0 0x7efdcc0ff080 unknown :-1 s=CLOSED pgs=716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:27:53.980 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:53.981+0000 7efdd1c00700 1 --2- 172.21.15.18:0/2097763183 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7efdcc105020 0x7efdcc1053f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:27:53.980 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:53.981+0000 7efdd1c00700 1 -- 172.21.15.18:0/2097763183 >> 172.21.15.18:0/2097763183 conn(0x7efdcc0faba0 msgr2=0x7efdcc0fcfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:27:53.980 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:53.981+0000 7efdd1c00700 1 -- 172.21.15.18:0/2097763183 shutdown_connections 2024-04-12T21:27:53.980 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:53.981+0000 7efdd1c00700 1 -- 172.21.15.18:0/2097763183 wait complete. 2024-04-12T21:27:53.981 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:53.982+0000 7efdd1c00700 1 Processor -- start 2024-04-12T21:27:53.981 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:53.982+0000 7efdd1c00700 1 -- start start 2024-04-12T21:27:53.981 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:53.982+0000 7efdd1c00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7efdcc105020 0x7efdcc115ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:27:53.981 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:53.982+0000 7efdd1c00700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7efdcc1059c0 0x7efdcc1160e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:27:53.981 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:53.982+0000 7efdd1c00700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efdcc116730 con 0x7efdcc105020 2024-04-12T21:27:53.982 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:53.982+0000 7efdd1c00700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efdcc10fd10 con 0x7efdcc1059c0 2024-04-12T21:27:53.982 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:53.983+0000 7efdcaa00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7efdcc105020 0x7efdcc115ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:27:53.982 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:53.983+0000 7efdcaa00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7efdcc105020 0x7efdcc115ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:51652/0 (socket says 172.21.15.18:51652) 2024-04-12T21:27:53.982 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:53.983+0000 7efdcaa00700 1 -- 172.21.15.18:0/480048984 learned_addr learned my addr 172.21.15.18:0/480048984 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:27:53.982 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:53.983+0000 7efdcaa00700 1 -- 172.21.15.18:0/480048984 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7efdcc1059c0 msgr2=0x7efdcc1160e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:27:53.982 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:53.983+0000 7efdca000700 1 --2- 172.21.15.18:0/480048984 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7efdcc1059c0 0x7efdcc1160e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:27:53.982 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:53.983+0000 7efdcaa00700 1 --2- 172.21.15.18:0/480048984 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7efdcc1059c0 0x7efdcc1160e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:27:53.982 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:53.983+0000 7efdcaa00700 1 -- 172.21.15.18:0/480048984 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efdb8005040 con 0x7efdcc105020 2024-04-12T21:27:53.982 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:53.984+0000 7efdcaa00700 1 --2- 172.21.15.18:0/480048984 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7efdcc105020 0x7efdcc115ba0 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7efdbc00c8c0 tx=0x7efdbc0044c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:27:53.983 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:53.984+0000 7efdc2a00700 1 -- 172.21.15.18:0/480048984 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7efdbc00a980 con 0x7efdcc105020 2024-04-12T21:27:53.983 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:53.984+0000 7efdc2a00700 1 -- 172.21.15.18:0/480048984 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7efdbc00ce60 con 0x7efdcc105020 2024-04-12T21:27:53.983 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:53.984+0000 7efdd1c00700 1 -- 172.21.15.18:0/480048984 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efdcc10fff0 con 0x7efdcc105020 2024-04-12T21:27:53.983 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:53.984+0000 7efdc2a00700 1 -- 172.21.15.18:0/480048984 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7efdbc0115e0 con 0x7efdcc105020 2024-04-12T21:27:53.983 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:53.984+0000 7efdd1c00700 1 -- 172.21.15.18:0/480048984 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efdcc1105b0 con 0x7efdcc105020 2024-04-12T21:27:53.983 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:53.985+0000 7efdd1c00700 1 -- 172.21.15.18:0/480048984 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efdcc100770 con 0x7efdcc105020 2024-04-12T21:27:53.984 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:53.986+0000 7efdc2a00700 1 -- 172.21.15.18:0/480048984 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7efdbc012400 con 0x7efdcc105020 2024-04-12T21:27:53.985 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:53.987+0000 7efdc2a00700 1 --2- 172.21.15.18:0/480048984 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7efdb006ec00 0x7efdb00710b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:27:53.985 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:53.987+0000 7efdc2a00700 1 -- 172.21.15.18:0/480048984 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 6757+0+0 (secure 0 0 0) 0x7efdbc08ea40 con 0x7efdcc105020 2024-04-12T21:27:53.985 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:53.987+0000 7efdca000700 1 --2- 172.21.15.18:0/480048984 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7efdb006ec00 0x7efdb00710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:27:53.986 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:53.988+0000 7efdca000700 1 --2- 172.21.15.18:0/480048984 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7efdb006ec00 0x7efdb00710b0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7efdb800b3b0 tx=0x7efdb8009e30 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:27:53.989 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:53.992+0000 7efdc2a00700 1 -- 172.21.15.18:0/480048984 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7efdbc05b270 con 0x7efdcc105020 2024-04-12T21:27:54.298 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:54.299+0000 7efdd1c00700 1 -- 172.21.15.18:0/480048984 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7efdcc1111f0 con 0x7efdcc105020 2024-04-12T21:27:54.298 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:54.300+0000 7efdc2a00700 1 -- 172.21.15.18:0/480048984 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+837 (secure 0 0 0) 0x7efdbc0126b0 con 0x7efdcc105020 2024-04-12T21:27:54.299 INFO:teuthology.orchestra.run.smithi018.stdout:{ 2024-04-12T21:27:54.299 INFO:teuthology.orchestra.run.smithi018.stdout: "mon": { 2024-04-12T21:27:54.299 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 19.0.0-2879-g324baf8d (324baf8ddb91dea46cac50cb535c306115791355) squid (dev)": 2 2024-04-12T21:27:54.299 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-12T21:27:54.299 INFO:teuthology.orchestra.run.smithi018.stdout: "mgr": { 2024-04-12T21:27:54.299 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 19.0.0-2879-g324baf8d (324baf8ddb91dea46cac50cb535c306115791355) squid (dev)": 2 2024-04-12T21:27:54.299 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-12T21:27:54.299 INFO:teuthology.orchestra.run.smithi018.stdout: "osd": { 2024-04-12T21:27:54.299 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 19.0.0-2879-g324baf8d (324baf8ddb91dea46cac50cb535c306115791355) squid (dev)": 6 2024-04-12T21:27:54.299 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-12T21:27:54.299 INFO:teuthology.orchestra.run.smithi018.stdout: "mds": { 2024-04-12T21:27:54.299 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 17.2.7-827-gd3b5060a (d3b5060a17449edfae9385cdf792afec109fca5d) quincy (stable)": 1, 2024-04-12T21:27:54.299 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 19.0.0-2879-g324baf8d (324baf8ddb91dea46cac50cb535c306115791355) squid (dev)": 3 2024-04-12T21:27:54.300 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-12T21:27:54.300 INFO:teuthology.orchestra.run.smithi018.stdout: "overall": { 2024-04-12T21:27:54.300 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 17.2.7-827-gd3b5060a (d3b5060a17449edfae9385cdf792afec109fca5d) quincy (stable)": 1, 2024-04-12T21:27:54.300 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 19.0.0-2879-g324baf8d (324baf8ddb91dea46cac50cb535c306115791355) squid (dev)": 13 2024-04-12T21:27:54.300 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-04-12T21:27:54.300 INFO:teuthology.orchestra.run.smithi018.stdout:} 2024-04-12T21:27:54.301 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:54.303+0000 7efdd1c00700 1 -- 172.21.15.18:0/480048984 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7efdb006ec00 msgr2=0x7efdb00710b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:27:54.301 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:54.303+0000 7efdd1c00700 1 --2- 172.21.15.18:0/480048984 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7efdb006ec00 0x7efdb00710b0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7efdb800b3b0 tx=0x7efdb8009e30 comp rx=0 tx=0).stop 2024-04-12T21:27:54.302 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:54.303+0000 7efdd1c00700 1 -- 172.21.15.18:0/480048984 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7efdcc105020 msgr2=0x7efdcc115ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:27:54.302 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:54.303+0000 7efdd1c00700 1 --2- 172.21.15.18:0/480048984 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7efdcc105020 0x7efdcc115ba0 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7efdbc00c8c0 tx=0x7efdbc0044c0 comp rx=0 tx=0).stop 2024-04-12T21:27:54.302 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:54.303+0000 7efdd1c00700 1 -- 172.21.15.18:0/480048984 shutdown_connections 2024-04-12T21:27:54.302 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:54.303+0000 7efdd1c00700 1 --2- 172.21.15.18:0/480048984 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7efdb006ec00 0x7efdb00710b0 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:27:54.302 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:54.303+0000 7efdd1c00700 1 --2- 172.21.15.18:0/480048984 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7efdcc105020 0x7efdcc115ba0 unknown :-1 s=CLOSED pgs=717 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:27:54.302 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:54.303+0000 7efdd1c00700 1 --2- 172.21.15.18:0/480048984 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7efdcc1059c0 0x7efdcc1160e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:27:54.302 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:54.303+0000 7efdd1c00700 1 -- 172.21.15.18:0/480048984 >> 172.21.15.18:0/480048984 conn(0x7efdcc0faba0 msgr2=0x7efdcc0fbb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:27:54.302 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:54.303+0000 7efdd1c00700 1 -- 172.21.15.18:0/480048984 shutdown_connections 2024-04-12T21:27:54.302 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:54.303+0000 7efdd1c00700 1 -- 172.21.15.18:0/480048984 wait complete. 2024-04-12T21:27:54.461 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:54.463+0000 7fb3d7200700 1 -- 172.21.15.18:0/3245708526 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb3d01037c0 msgr2=0x7fb3d00fce30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:27:54.461 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:54.463+0000 7fb3d7200700 1 --2- 172.21.15.18:0/3245708526 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb3d01037c0 0x7fb3d00fce30 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7fb3bc004940 tx=0x7fb3bc009e30 comp rx=0 tx=0).stop 2024-04-12T21:27:54.461 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:54.463+0000 7fb3d7200700 1 -- 172.21.15.18:0/3245708526 shutdown_connections 2024-04-12T21:27:54.461 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:54.463+0000 7fb3d7200700 1 --2- 172.21.15.18:0/3245708526 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb3d01037c0 0x7fb3d00fce30 unknown :-1 s=CLOSED pgs=718 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:27:54.461 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:54.463+0000 7fb3d7200700 1 --2- 172.21.15.18:0/3245708526 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb3d0102e20 0x7fb3d01031f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:27:54.461 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:54.463+0000 7fb3d7200700 1 -- 172.21.15.18:0/3245708526 >> 172.21.15.18:0/3245708526 conn(0x7fb3d00f8970 msgr2=0x7fb3d00fad80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:27:54.462 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:54.463+0000 7fb3d7200700 1 -- 172.21.15.18:0/3245708526 shutdown_connections 2024-04-12T21:27:54.462 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:54.463+0000 7fb3d7200700 1 -- 172.21.15.18:0/3245708526 wait complete. 2024-04-12T21:27:54.462 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:54.464+0000 7fb3d7200700 1 Processor -- start 2024-04-12T21:27:54.463 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:54.464+0000 7fb3d7200700 1 -- start start 2024-04-12T21:27:54.463 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:54.464+0000 7fb3d7200700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb3d0102e20 0x7fb3d0115de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:27:54.463 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:54.464+0000 7fb3d7200700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb3d01037c0 0x7fb3d0116320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:27:54.463 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:54.464+0000 7fb3d7200700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3d0116970 con 0x7fb3d0102e20 2024-04-12T21:27:54.463 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:54.464+0000 7fb3d7200700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3d010fd40 con 0x7fb3d01037c0 2024-04-12T21:27:54.463 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:54.465+0000 7fb3cfe00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb3d0102e20 0x7fb3d0115de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:27:54.463 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:54.465+0000 7fb3cfe00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb3d0102e20 0x7fb3d0115de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:51666/0 (socket says 172.21.15.18:51666) 2024-04-12T21:27:54.463 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:54.465+0000 7fb3cfe00700 1 -- 172.21.15.18:0/489063285 learned_addr learned my addr 172.21.15.18:0/489063285 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:27:54.464 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:54.465+0000 7fb3cf400700 1 --2- 172.21.15.18:0/489063285 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb3d01037c0 0x7fb3d0116320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:27:54.464 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:54.465+0000 7fb3cfe00700 1 -- 172.21.15.18:0/489063285 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb3d01037c0 msgr2=0x7fb3d0116320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:27:54.464 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:54.465+0000 7fb3cfe00700 1 --2- 172.21.15.18:0/489063285 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb3d01037c0 0x7fb3d0116320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:27:54.464 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:54.465+0000 7fb3cfe00700 1 -- 172.21.15.18:0/489063285 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3bc005040 con 0x7fb3d0102e20 2024-04-12T21:27:54.464 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:54.466+0000 7fb3cfe00700 1 --2- 172.21.15.18:0/489063285 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb3d0102e20 0x7fb3d0115de0 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7fb3c0002a50 tx=0x7fb3c00046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:27:54.465 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:54.466+0000 7fb3c7e00700 1 -- 172.21.15.18:0/489063285 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb3c0007cc0 con 0x7fb3d0102e20 2024-04-12T21:27:54.465 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:54.466+0000 7fb3c7e00700 1 -- 172.21.15.18:0/489063285 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb3c000f040 con 0x7fb3d0102e20 2024-04-12T21:27:54.465 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:54.466+0000 7fb3c7e00700 1 -- 172.21.15.18:0/489063285 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb3c0009640 con 0x7fb3d0102e20 2024-04-12T21:27:54.465 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:54.466+0000 7fb3d7200700 1 -- 172.21.15.18:0/489063285 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb3d0110020 con 0x7fb3d0102e20 2024-04-12T21:27:54.465 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:54.466+0000 7fb3d7200700 1 -- 172.21.15.18:0/489063285 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb3d01105e0 con 0x7fb3d0102e20 2024-04-12T21:27:54.465 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:54.467+0000 7fb3d7200700 1 -- 172.21.15.18:0/489063285 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb3d0102140 con 0x7fb3d0102e20 2024-04-12T21:27:54.466 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:54.468+0000 7fb3c7e00700 1 -- 172.21.15.18:0/489063285 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7fb3c00097a0 con 0x7fb3d0102e20 2024-04-12T21:27:54.467 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:54.469+0000 7fb3c7e00700 1 --2- 172.21.15.18:0/489063285 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fb3b406ec10 0x7fb3b40710c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:27:54.467 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:54.469+0000 7fb3c7e00700 1 -- 172.21.15.18:0/489063285 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 6757+0+0 (secure 0 0 0) 0x7fb3c008e6c0 con 0x7fb3d0102e20 2024-04-12T21:27:54.467 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:54.469+0000 7fb3cf400700 1 --2- 172.21.15.18:0/489063285 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fb3b406ec10 0x7fb3b40710c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:27:54.468 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:54.470+0000 7fb3cf400700 1 --2- 172.21.15.18:0/489063285 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fb3b406ec10 0x7fb3b40710c0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fb3bc00b3b0 tx=0x7fb3bc009e30 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:27:54.472 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:54.474+0000 7fb3c7e00700 1 -- 172.21.15.18:0/489063285 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7fb3c005af10 con 0x7fb3d0102e20 2024-04-12T21:27:54.757 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:54.759+0000 7fb3d7200700 1 -- 172.21.15.18:0/489063285 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fb3d004bed0 con 0x7fb3d0102e20 2024-04-12T21:27:54.758 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:54.760+0000 7fb3c7e00700 1 -- 172.21.15.18:0/489063285 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 31 v31) v1 ==== 76+0+1873 (secure 0 0 0) 0x7fb3c005aaa0 con 0x7fb3d0102e20 2024-04-12T21:27:54.758 INFO:teuthology.orchestra.run.smithi018.stdout:e31 2024-04-12T21:27:54.758 INFO:teuthology.orchestra.run.smithi018.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-12T21:27:54.759 INFO:teuthology.orchestra.run.smithi018.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-12T21:27:54.759 INFO:teuthology.orchestra.run.smithi018.stdout:legacy client fscid: 1 2024-04-12T21:27:54.759 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-12T21:27:54.759 INFO:teuthology.orchestra.run.smithi018.stdout:Filesystem 'cephfs' (1) 2024-04-12T21:27:54.759 INFO:teuthology.orchestra.run.smithi018.stdout:fs_name cephfs 2024-04-12T21:27:54.759 INFO:teuthology.orchestra.run.smithi018.stdout:epoch 24 2024-04-12T21:27:54.759 INFO:teuthology.orchestra.run.smithi018.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-04-12T21:27:54.759 INFO:teuthology.orchestra.run.smithi018.stdout:created 2024-04-12T20:51:57.362850+0000 2024-04-12T21:27:54.759 INFO:teuthology.orchestra.run.smithi018.stdout:modified 2024-04-12T21:06:59.717190+0000 2024-04-12T21:27:54.759 INFO:teuthology.orchestra.run.smithi018.stdout:tableserver 0 2024-04-12T21:27:54.759 INFO:teuthology.orchestra.run.smithi018.stdout:root 0 2024-04-12T21:27:54.759 INFO:teuthology.orchestra.run.smithi018.stdout:session_timeout 60 2024-04-12T21:27:54.759 INFO:teuthology.orchestra.run.smithi018.stdout:session_autoclose 300 2024-04-12T21:27:54.759 INFO:teuthology.orchestra.run.smithi018.stdout:max_file_size 1099511627776 2024-04-12T21:27:54.759 INFO:teuthology.orchestra.run.smithi018.stdout:max_xattr_size 65536 2024-04-12T21:27:54.760 INFO:teuthology.orchestra.run.smithi018.stdout:required_client_features {} 2024-04-12T21:27:54.760 INFO:teuthology.orchestra.run.smithi018.stdout:last_failure 0 2024-04-12T21:27:54.760 INFO:teuthology.orchestra.run.smithi018.stdout:last_failure_osd_epoch 100 2024-04-12T21:27:54.760 INFO:teuthology.orchestra.run.smithi018.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-12T21:27:54.760 INFO:teuthology.orchestra.run.smithi018.stdout:max_mds 2 2024-04-12T21:27:54.760 INFO:teuthology.orchestra.run.smithi018.stdout:in 0,1 2024-04-12T21:27:54.760 INFO:teuthology.orchestra.run.smithi018.stdout:up {} 2024-04-12T21:27:54.760 INFO:teuthology.orchestra.run.smithi018.stdout:failed 0,1 2024-04-12T21:27:54.760 INFO:teuthology.orchestra.run.smithi018.stdout:damaged 2024-04-12T21:27:54.760 INFO:teuthology.orchestra.run.smithi018.stdout:stopped 2024-04-12T21:27:54.760 INFO:teuthology.orchestra.run.smithi018.stdout:data_pools [3] 2024-04-12T21:27:54.760 INFO:teuthology.orchestra.run.smithi018.stdout:metadata_pool 2 2024-04-12T21:27:54.760 INFO:teuthology.orchestra.run.smithi018.stdout:inline_data disabled 2024-04-12T21:27:54.761 INFO:teuthology.orchestra.run.smithi018.stdout:balancer 2024-04-12T21:27:54.761 INFO:teuthology.orchestra.run.smithi018.stdout:bal_rank_mask -1 2024-04-12T21:27:54.761 INFO:teuthology.orchestra.run.smithi018.stdout:standby_count_wanted 1 2024-04-12T21:27:54.761 INFO:teuthology.orchestra.run.smithi018.stdout:qdb_cluster leader: 0 members: 2024-04-12T21:27:54.761 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-12T21:27:54.761 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-12T21:27:54.761 INFO:teuthology.orchestra.run.smithi018.stdout:Standby daemons: 2024-04-12T21:27:54.761 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-12T21:27:54.761 INFO:teuthology.orchestra.run.smithi018.stdout:[mds.cephfs.smithi186.wjnafs{-1:24277} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.186:6826/438347207,v1:172.21.15.186:6827/438347207] compat {c=[1],r=[1],i=[7ff]}] 2024-04-12T21:27:54.761 INFO:teuthology.orchestra.run.smithi018.stdout:[mds.cephfs.smithi018.acqbsc{-1:34436} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.18:6828/755179764,v1:172.21.15.18:6829/755179764] compat {c=[1],r=[1],i=[1fff]}] 2024-04-12T21:27:54.761 INFO:teuthology.orchestra.run.smithi018.stdout:[mds.cephfs.smithi018.szjluo{-1:34468} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.18:6826/2407992653,v1:172.21.15.18:6827/2407992653] compat {c=[1],r=[1],i=[1fff]}] 2024-04-12T21:27:54.761 INFO:teuthology.orchestra.run.smithi018.stdout:[mds.cephfs.smithi186.jdllyn{-1:44595} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.186:6824/3523444202,v1:172.21.15.186:6825/3523444202] compat {c=[1],r=[1],i=[1fff]}] 2024-04-12T21:27:54.762 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:54.763+0000 7fb3d7200700 1 -- 172.21.15.18:0/489063285 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fb3b406ec10 msgr2=0x7fb3b40710c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:27:54.762 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:54.763+0000 7fb3d7200700 1 --2- 172.21.15.18:0/489063285 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fb3b406ec10 0x7fb3b40710c0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fb3bc00b3b0 tx=0x7fb3bc009e30 comp rx=0 tx=0).stop 2024-04-12T21:27:54.762 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:54.763+0000 7fb3d7200700 1 -- 172.21.15.18:0/489063285 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb3d0102e20 msgr2=0x7fb3d0115de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:27:54.762 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:54.763+0000 7fb3d7200700 1 --2- 172.21.15.18:0/489063285 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb3d0102e20 0x7fb3d0115de0 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7fb3c0002a50 tx=0x7fb3c00046b0 comp rx=0 tx=0).stop 2024-04-12T21:27:54.762 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:54.763+0000 7fb3d7200700 1 -- 172.21.15.18:0/489063285 shutdown_connections 2024-04-12T21:27:54.762 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:54.763+0000 7fb3d7200700 1 --2- 172.21.15.18:0/489063285 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fb3b406ec10 0x7fb3b40710c0 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:27:54.763 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:54.763+0000 7fb3d7200700 1 --2- 172.21.15.18:0/489063285 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb3d0102e20 0x7fb3d0115de0 unknown :-1 s=CLOSED pgs=719 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:27:54.763 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:54.763+0000 7fb3d7200700 1 --2- 172.21.15.18:0/489063285 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb3d01037c0 0x7fb3d0116320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:27:54.763 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:54.763+0000 7fb3d7200700 1 -- 172.21.15.18:0/489063285 >> 172.21.15.18:0/489063285 conn(0x7fb3d00f8970 msgr2=0x7fb3d00fa6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:27:54.763 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:54.763+0000 7fb3d7200700 1 -- 172.21.15.18:0/489063285 shutdown_connections 2024-04-12T21:27:54.763 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:54.763+0000 7fb3d7200700 1 -- 172.21.15.18:0/489063285 wait complete. 2024-04-12T21:27:54.763 INFO:teuthology.orchestra.run.smithi018.stderr:dumped fsmap epoch 31 2024-04-12T21:27:54.929 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:54.930+0000 7ffb32200700 1 -- 172.21.15.18:0/314813493 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ffb2c101ba0 msgr2=0x7ffb2c101f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:27:54.929 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:54.930+0000 7ffb32200700 1 --2- 172.21.15.18:0/314813493 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ffb2c101ba0 0x7ffb2c101f70 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7ffb20004940 tx=0x7ffb20009e30 comp rx=0 tx=0).stop 2024-04-12T21:27:54.929 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:54.930+0000 7ffb32200700 1 -- 172.21.15.18:0/314813493 shutdown_connections 2024-04-12T21:27:54.929 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:54.930+0000 7ffb32200700 1 --2- 172.21.15.18:0/314813493 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ffb2c0fbb70 0x7ffb2c0fbfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:27:54.929 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:54.930+0000 7ffb32200700 1 --2- 172.21.15.18:0/314813493 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ffb2c101ba0 0x7ffb2c101f70 unknown :-1 s=CLOSED pgs=720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:27:54.929 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:54.930+0000 7ffb32200700 1 -- 172.21.15.18:0/314813493 >> 172.21.15.18:0/314813493 conn(0x7ffb2c071b70 msgr2=0x7ffb2c071f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:27:54.929 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:54.931+0000 7ffb32200700 1 -- 172.21.15.18:0/314813493 shutdown_connections 2024-04-12T21:27:54.929 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:54.931+0000 7ffb32200700 1 -- 172.21.15.18:0/314813493 wait complete. 2024-04-12T21:27:54.931 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:54.932+0000 7ffb32200700 1 Processor -- start 2024-04-12T21:27:54.931 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:54.932+0000 7ffb32200700 1 -- start start 2024-04-12T21:27:54.931 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:54.932+0000 7ffb32200700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ffb2c0fbb70 0x7ffb2c115b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:27:54.931 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:54.932+0000 7ffb32200700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ffb2c101ba0 0x7ffb2c1160d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:27:54.931 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:54.932+0000 7ffb32200700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb2c116720 con 0x7ffb2c0fbb70 2024-04-12T21:27:54.931 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:54.932+0000 7ffb32200700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb2c10fb80 con 0x7ffb2c101ba0 2024-04-12T21:27:54.931 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:54.933+0000 7ffb30e00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ffb2c0fbb70 0x7ffb2c115b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:27:54.931 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:54.933+0000 7ffb2be00700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ffb2c101ba0 0x7ffb2c1160d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:27:54.931 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:54.933+0000 7ffb2be00700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ffb2c101ba0 0x7ffb2c1160d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.18:49518/0 (socket says 172.21.15.18:49518) 2024-04-12T21:27:54.931 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:54.933+0000 7ffb2be00700 1 -- 172.21.15.18:0/2740139543 learned_addr learned my addr 172.21.15.18:0/2740139543 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:27:54.932 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:54.934+0000 7ffb2be00700 1 -- 172.21.15.18:0/2740139543 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ffb2c0fbb70 msgr2=0x7ffb2c115b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:27:54.932 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:54.934+0000 7ffb2be00700 1 --2- 172.21.15.18:0/2740139543 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ffb2c0fbb70 0x7ffb2c115b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:27:54.932 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:54.934+0000 7ffb2be00700 1 -- 172.21.15.18:0/2740139543 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb20005040 con 0x7ffb2c101ba0 2024-04-12T21:27:54.932 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:54.934+0000 7ffb2be00700 1 --2- 172.21.15.18:0/2740139543 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ffb2c101ba0 0x7ffb2c1160d0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7ffb1800c830 tx=0x7ffb18004470 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:27:54.933 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:54.935+0000 7ffb28c00700 1 -- 172.21.15.18:0/2740139543 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ffb180106f0 con 0x7ffb2c101ba0 2024-04-12T21:27:54.933 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:54.935+0000 7ffb32200700 1 -- 172.21.15.18:0/2740139543 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffb2c10fe60 con 0x7ffb2c101ba0 2024-04-12T21:27:54.933 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:54.935+0000 7ffb32200700 1 -- 172.21.15.18:0/2740139543 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffb2c110420 con 0x7ffb2c101ba0 2024-04-12T21:27:54.933 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:54.935+0000 7ffb28c00700 1 -- 172.21.15.18:0/2740139543 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ffb18010d20 con 0x7ffb2c101ba0 2024-04-12T21:27:54.933 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:54.935+0000 7ffb28c00700 1 -- 172.21.15.18:0/2740139543 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ffb18012400 con 0x7ffb2c101ba0 2024-04-12T21:27:54.934 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:54.936+0000 7ffb32200700 1 -- 172.21.15.18:0/2740139543 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffb0c0052b0 con 0x7ffb2c101ba0 2024-04-12T21:27:54.936 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:54.938+0000 7ffb28c00700 1 -- 172.21.15.18:0/2740139543 <== mon.1 v2:172.21.15.186:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7ffb18010850 con 0x7ffb2c101ba0 2024-04-12T21:27:54.937 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:54.939+0000 7ffb28c00700 1 --2- 172.21.15.18:0/2740139543 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7ffb1406ec10 0x7ffb140710c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:27:54.937 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:54.939+0000 7ffb28c00700 1 -- 172.21.15.18:0/2740139543 <== mon.1 v2:172.21.15.186:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 6757+0+0 (secure 0 0 0) 0x7ffb1808efb0 con 0x7ffb2c101ba0 2024-04-12T21:27:54.937 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:54.940+0000 7ffb30e00700 1 --2- 172.21.15.18:0/2740139543 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7ffb1406ec10 0x7ffb140710c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:27:54.938 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:54.940+0000 7ffb30e00700 1 --2- 172.21.15.18:0/2740139543 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7ffb1406ec10 0x7ffb140710c0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7ffb20009fd0 tx=0x7ffb20009e30 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:27:54.954 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:54.956+0000 7ffb28c00700 1 -- 172.21.15.18:0/2740139543 <== mon.1 v2:172.21.15.186:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7ffb18057cb0 con 0x7ffb2c101ba0 2024-04-12T21:27:55.031 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:27:54 smithi186 ceph-mon[86379]: from='client.35484 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:27:55.031 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:27:54 smithi186 ceph-mon[86379]: from='client.35488 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:27:55.031 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:27:54 smithi186 ceph-mon[86379]: pgmap v844: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:27:55.031 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:27:54 smithi186 ceph-mon[86379]: from='client.35492 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:27:55.031 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:27:54 smithi186 ceph-mon[86379]: from='client.? 172.21.15.18:0/480048984' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T21:27:55.101 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:27:54 smithi018 ceph-mon[112590]: from='client.35484 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:27:55.102 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:27:54 smithi018 ceph-mon[112590]: from='client.35488 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:27:55.102 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:27:54 smithi018 ceph-mon[112590]: pgmap v844: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:27:55.102 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:27:54 smithi018 ceph-mon[112590]: from='client.35492 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:27:55.102 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:27:54 smithi018 ceph-mon[112590]: from='client.? 172.21.15.18:0/480048984' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T21:27:55.211 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:55.213+0000 7ffb32200700 1 -- 172.21.15.18:0/2740139543 --> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7ffb0c000bc0 con 0x7ffb1406ec10 2024-04-12T21:27:55.214 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:55.216+0000 7ffb28c00700 1 -- 172.21.15.18:0/2740139543 <== mgr.34102 v2:172.21.15.18:6800/3812891919 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7ffb0c000bc0 con 0x7ffb1406ec10 2024-04-12T21:27:55.214 INFO:teuthology.orchestra.run.smithi018.stdout:{ 2024-04-12T21:27:55.214 INFO:teuthology.orchestra.run.smithi018.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:f4452def3be4d350f4a8e5feaf7ebb1511be9298fe68478ba7a2d6fbd7d69a63", 2024-04-12T21:27:55.214 INFO:teuthology.orchestra.run.smithi018.stdout: "in_progress": true, 2024-04-12T21:27:55.215 INFO:teuthology.orchestra.run.smithi018.stdout: "which": "Upgrading all daemon types on all hosts", 2024-04-12T21:27:55.215 INFO:teuthology.orchestra.run.smithi018.stdout: "services_complete": [ 2024-04-12T21:27:55.215 INFO:teuthology.orchestra.run.smithi018.stdout: "crash", 2024-04-12T21:27:55.215 INFO:teuthology.orchestra.run.smithi018.stdout: "osd", 2024-04-12T21:27:55.216 INFO:teuthology.orchestra.run.smithi018.stdout: "mon", 2024-04-12T21:27:55.216 INFO:teuthology.orchestra.run.smithi018.stdout: "mgr" 2024-04-12T21:27:55.216 INFO:teuthology.orchestra.run.smithi018.stdout: ], 2024-04-12T21:27:55.216 INFO:teuthology.orchestra.run.smithi018.stdout: "progress": "15/21 daemons upgraded", 2024-04-12T21:27:55.216 INFO:teuthology.orchestra.run.smithi018.stdout: "message": "Currently upgrading mds daemons", 2024-04-12T21:27:55.216 INFO:teuthology.orchestra.run.smithi018.stdout: "is_paused": false 2024-04-12T21:27:55.216 INFO:teuthology.orchestra.run.smithi018.stdout:} 2024-04-12T21:27:55.217 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:55.219+0000 7ffb32200700 1 -- 172.21.15.18:0/2740139543 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7ffb1406ec10 msgr2=0x7ffb140710c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:27:55.217 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:55.219+0000 7ffb32200700 1 --2- 172.21.15.18:0/2740139543 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7ffb1406ec10 0x7ffb140710c0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7ffb20009fd0 tx=0x7ffb20009e30 comp rx=0 tx=0).stop 2024-04-12T21:27:55.218 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:55.219+0000 7ffb32200700 1 -- 172.21.15.18:0/2740139543 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ffb2c101ba0 msgr2=0x7ffb2c1160d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:27:55.218 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:55.219+0000 7ffb32200700 1 --2- 172.21.15.18:0/2740139543 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ffb2c101ba0 0x7ffb2c1160d0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7ffb1800c830 tx=0x7ffb18004470 comp rx=0 tx=0).stop 2024-04-12T21:27:55.218 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:55.219+0000 7ffb32200700 1 -- 172.21.15.18:0/2740139543 shutdown_connections 2024-04-12T21:27:55.218 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:55.219+0000 7ffb32200700 1 --2- 172.21.15.18:0/2740139543 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7ffb1406ec10 0x7ffb140710c0 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:27:55.218 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:55.219+0000 7ffb32200700 1 --2- 172.21.15.18:0/2740139543 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ffb2c0fbb70 0x7ffb2c115b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:27:55.218 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:55.219+0000 7ffb32200700 1 --2- 172.21.15.18:0/2740139543 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ffb2c101ba0 0x7ffb2c1160d0 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:27:55.218 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:55.219+0000 7ffb32200700 1 -- 172.21.15.18:0/2740139543 >> 172.21.15.18:0/2740139543 conn(0x7ffb2c071b70 msgr2=0x7ffb2c0f9d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:27:55.218 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:55.220+0000 7ffb32200700 1 -- 172.21.15.18:0/2740139543 shutdown_connections 2024-04-12T21:27:55.218 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:55.220+0000 7ffb32200700 1 -- 172.21.15.18:0/2740139543 wait complete. 2024-04-12T21:27:55.380 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:55.382+0000 7f42d8a00700 1 -- 172.21.15.18:0/3622684828 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f42d4102c40 msgr2=0x7f42d4103010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:27:55.381 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:55.382+0000 7f42d8a00700 1 --2- 172.21.15.18:0/3622684828 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f42d4102c40 0x7f42d4103010 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7f42c4004940 tx=0x7f42c4009e30 comp rx=0 tx=0).stop 2024-04-12T21:27:55.381 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:55.382+0000 7f42d8a00700 1 -- 172.21.15.18:0/3622684828 shutdown_connections 2024-04-12T21:27:55.381 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:55.382+0000 7f42d8a00700 1 --2- 172.21.15.18:0/3622684828 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f42d41035e0 0x7f42d40fcc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:27:55.381 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:55.382+0000 7f42d8a00700 1 --2- 172.21.15.18:0/3622684828 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f42d4102c40 0x7f42d4103010 unknown :-1 s=CLOSED pgs=721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:27:55.381 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:55.382+0000 7f42d8a00700 1 -- 172.21.15.18:0/3622684828 >> 172.21.15.18:0/3622684828 conn(0x7f42d40f87f0 msgr2=0x7f42d40fac00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:27:55.381 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:55.383+0000 7f42d8a00700 1 -- 172.21.15.18:0/3622684828 shutdown_connections 2024-04-12T21:27:55.381 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:55.383+0000 7f42d8a00700 1 -- 172.21.15.18:0/3622684828 wait complete. 2024-04-12T21:27:55.382 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:55.383+0000 7f42d8a00700 1 Processor -- start 2024-04-12T21:27:55.382 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:55.384+0000 7f42d8a00700 1 -- start start 2024-04-12T21:27:55.382 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:55.384+0000 7f42d8a00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f42d4102c40 0x7f42d4115bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:27:55.382 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:55.384+0000 7f42d8a00700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f42d41035e0 0x7f42d41160f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:27:55.382 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:55.384+0000 7f42d8a00700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42d4116740 con 0x7f42d4102c40 2024-04-12T21:27:55.382 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:55.384+0000 7f42d8a00700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42d410fb50 con 0x7f42d41035e0 2024-04-12T21:27:55.383 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:55.384+0000 7f42d2800700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f42d4102c40 0x7f42d4115bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:27:55.383 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:55.384+0000 7f42d2800700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f42d4102c40 0x7f42d4115bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:51686/0 (socket says 172.21.15.18:51686) 2024-04-12T21:27:55.383 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:55.384+0000 7f42d2800700 1 -- 172.21.15.18:0/3446141448 learned_addr learned my addr 172.21.15.18:0/3446141448 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:27:55.383 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:55.385+0000 7f42cbe00700 1 --2- 172.21.15.18:0/3446141448 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f42d41035e0 0x7f42d41160f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:27:55.387 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:55.385+0000 7f42d2800700 1 -- 172.21.15.18:0/3446141448 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f42d41035e0 msgr2=0x7f42d41160f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:27:55.387 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:55.385+0000 7f42d2800700 1 --2- 172.21.15.18:0/3446141448 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f42d41035e0 0x7f42d41160f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:27:55.387 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:55.385+0000 7f42d2800700 1 -- 172.21.15.18:0/3446141448 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42c4005040 con 0x7f42d4102c40 2024-04-12T21:27:55.387 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:55.385+0000 7f42d2800700 1 --2- 172.21.15.18:0/3446141448 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f42d4102c40 0x7f42d4115bb0 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7f42c4009180 tx=0x7f42c4009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:27:55.387 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:55.386+0000 7f42caa00700 1 -- 172.21.15.18:0/3446141448 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f42c400e040 con 0x7f42d4102c40 2024-04-12T21:27:55.387 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:55.386+0000 7f42caa00700 1 -- 172.21.15.18:0/3446141448 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f42c400bbc0 con 0x7f42d4102c40 2024-04-12T21:27:55.387 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:55.386+0000 7f42d8a00700 1 -- 172.21.15.18:0/3446141448 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f42d410fdd0 con 0x7f42d4102c40 2024-04-12T21:27:55.388 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:55.386+0000 7f42caa00700 1 -- 172.21.15.18:0/3446141448 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f42c401c630 con 0x7f42d4102c40 2024-04-12T21:27:55.388 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:55.386+0000 7f42d8a00700 1 -- 172.21.15.18:0/3446141448 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f42d4110390 con 0x7f42d4102c40 2024-04-12T21:27:55.388 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:55.387+0000 7f42d8a00700 1 -- 172.21.15.18:0/3446141448 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f42d40fe390 con 0x7f42d4102c40 2024-04-12T21:27:55.388 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:55.388+0000 7f42caa00700 1 -- 172.21.15.18:0/3446141448 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7f42c400bd30 con 0x7f42d4102c40 2024-04-12T21:27:55.389 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:55.389+0000 7f42caa00700 1 --2- 172.21.15.18:0/3446141448 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f42b806ec10 0x7f42b80710c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:27:55.389 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:55.389+0000 7f42caa00700 1 -- 172.21.15.18:0/3446141448 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 6757+0+0 (secure 0 0 0) 0x7f42c408fd70 con 0x7f42d4102c40 2024-04-12T21:27:55.389 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:55.390+0000 7f42cbe00700 1 --2- 172.21.15.18:0/3446141448 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f42b806ec10 0x7f42b80710c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:27:55.389 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:55.391+0000 7f42cbe00700 1 --2- 172.21.15.18:0/3446141448 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f42b806ec10 0x7f42b80710c0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f42d4111450 tx=0x7f42bc009ee0 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:27:55.392 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:55.395+0000 7f42caa00700 1 -- 172.21.15.18:0/3446141448 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f42c4058a90 con 0x7f42d4102c40 2024-04-12T21:27:55.719 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:55.721+0000 7f42d8a00700 1 -- 172.21.15.18:0/3446141448 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f42d405f5f0 con 0x7f42d4102c40 2024-04-12T21:27:55.720 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:55.722+0000 7f42caa00700 1 -- 172.21.15.18:0/3446141448 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+4303 (secure 0 0 0) 0x7f42d405f5f0 con 0x7f42d4102c40 2024-04-12T21:27:55.721 INFO:teuthology.orchestra.run.smithi018.stdout:HEALTH_ERR failed to probe daemons or devices; 1 filesystem is degraded; 1 filesystem is offline 2024-04-12T21:27:55.721 INFO:teuthology.orchestra.run.smithi018.stdout:[WRN] CEPHADM_REFRESH_FAILED: failed to probe daemons or devices 2024-04-12T21:27:55.721 INFO:teuthology.orchestra.run.smithi018.stdout: host smithi018 `cephadm ceph-volume` failed: cephadm exited with an error code: 1, stderr: Inferring config /var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/mon.smithi018/config 2024-04-12T21:27:55.721 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/podman: timeout after 895 seconds 2024-04-12T21:27:55.721 INFO:teuthology.orchestra.run.smithi018.stdout:Non-zero exit code 124 from /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:f4452def3be4d350f4a8e5feaf7ebb1511be9298fe68478ba7a2d6fbd7d69a63 -e NODE_NAME=smithi018 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968:/var/run/ceph:z -v /var/log/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968:/var/log/ceph:z -v /var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmpxsmwkkuh:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:f4452def3be4d350f4a8e5feaf7ebb1511be9298fe68478ba7a2d6fbd7d69a63 inventory --format=json-pretty --filter-for-batch 2024-04-12T21:27:55.721 INFO:teuthology.orchestra.run.smithi018.stdout:Traceback (most recent call last): 2024-04-12T21:27:55.721 INFO:teuthology.orchestra.run.smithi018.stdout: File "/usr/lib64/python3.9/runpy.py", line 197, in _run_module_as_main 2024-04-12T21:27:55.721 INFO:teuthology.orchestra.run.smithi018.stdout: return _run_code(code, main_globals, None, 2024-04-12T21:27:55.721 INFO:teuthology.orchestra.run.smithi018.stdout: File "/usr/lib64/python3.9/runpy.py", line 87, in _run_code 2024-04-12T21:27:55.721 INFO:teuthology.orchestra.run.smithi018.stdout: exec(code, run_globals) 2024-04-12T21:27:55.721 INFO:teuthology.orchestra.run.smithi018.stdout: File "/var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/cephadm.e44f3ed5e34e4ef52eedef233d1190bff004470d8e8005127b5312bff357fa7f/__main__.py", line 5618, in 2024-04-12T21:27:55.721 INFO:teuthology.orchestra.run.smithi018.stdout: File "/var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/cephadm.e44f3ed5e34e4ef52eedef233d1190bff004470d8e8005127b5312bff357fa7f/__main__.py", line 5606, in main 2024-04-12T21:27:55.721 INFO:teuthology.orchestra.run.smithi018.stdout: File "/var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/cephadm.e44f3ed5e34e4ef52eedef233d1190bff004470d8e8005127b5312bff357fa7f/__main__.py", line 408, in _infer_config 2024-04-12T21:27:55.722 INFO:teuthology.orchestra.run.smithi018.stdout: File "/var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/cephadm.e44f3ed5e34e4ef52eedef233d1190bff004470d8e8005127b5312bff357fa7f/__main__.py", line 323, in _infer_fsid 2024-04-12T21:27:55.722 INFO:teuthology.orchestra.run.smithi018.stdout: File "/var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/cephadm.e44f3ed5e34e4ef52eedef233d1190bff004470d8e8005127b5312bff357fa7f/__main__.py", line 436, in _infer_image 2024-04-12T21:27:55.722 INFO:teuthology.orchestra.run.smithi018.stdout: File "/var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/cephadm.e44f3ed5e34e4ef52eedef233d1190bff004470d8e8005127b5312bff357fa7f/__main__.py", line 310, in _validate_fsid 2024-04-12T21:27:55.722 INFO:teuthology.orchestra.run.smithi018.stdout: File "/var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/cephadm.e44f3ed5e34e4ef52eedef233d1190bff004470d8e8005127b5312bff357fa7f/__main__.py", line 3359, in command_ceph_volume 2024-04-12T21:27:55.722 INFO:teuthology.orchestra.run.smithi018.stdout: File "/var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/cephadm.e44f3ed5e34e4ef52eedef233d1190bff004470d8e8005127b5312bff357fa7f/cephadmlib/call_wrappers.py", line 310, in call_throws 2024-04-12T21:27:55.722 INFO:teuthology.orchestra.run.smithi018.stdout:RuntimeError: Failed command: /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:f4452def3be4d350f4a8e5feaf7ebb1511be9298fe68478ba7a2d6fbd7d69a63 -e NODE_NAME=smithi018 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968:/var/run/ceph:z -v /var/log/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968:/var/log/ceph:z -v /var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmpxsmwkkuh:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:f4452def3be4d350f4a8e5feaf7ebb1511be9298fe68478ba7a2d6fbd7d69a63 inventory --format=json-pretty --filter-for-batch 2024-04-12T21:27:55.722 INFO:teuthology.orchestra.run.smithi018.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-04-12T21:27:55.722 INFO:teuthology.orchestra.run.smithi018.stdout: fs cephfs is degraded 2024-04-12T21:27:55.722 INFO:teuthology.orchestra.run.smithi018.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-04-12T21:27:55.723 INFO:teuthology.orchestra.run.smithi018.stdout: fs cephfs is offline because no MDS is active for it. 2024-04-12T21:27:55.723 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:55.724+0000 7f42d8a00700 1 -- 172.21.15.18:0/3446141448 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f42b806ec10 msgr2=0x7f42b80710c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:27:55.723 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:55.724+0000 7f42d8a00700 1 --2- 172.21.15.18:0/3446141448 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f42b806ec10 0x7f42b80710c0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f42d4111450 tx=0x7f42bc009ee0 comp rx=0 tx=0).stop 2024-04-12T21:27:55.723 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:55.724+0000 7f42d8a00700 1 -- 172.21.15.18:0/3446141448 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f42d4102c40 msgr2=0x7f42d4115bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:27:55.723 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:55.724+0000 7f42d8a00700 1 --2- 172.21.15.18:0/3446141448 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f42d4102c40 0x7f42d4115bb0 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7f42c4009180 tx=0x7f42c4009e30 comp rx=0 tx=0).stop 2024-04-12T21:27:55.723 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:55.725+0000 7f42d8a00700 1 -- 172.21.15.18:0/3446141448 shutdown_connections 2024-04-12T21:27:55.723 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:55.725+0000 7f42d8a00700 1 --2- 172.21.15.18:0/3446141448 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f42b806ec10 0x7f42b80710c0 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:27:55.723 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:55.725+0000 7f42d8a00700 1 --2- 172.21.15.18:0/3446141448 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f42d4102c40 0x7f42d4115bb0 unknown :-1 s=CLOSED pgs=722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:27:55.724 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:55.725+0000 7f42d8a00700 1 --2- 172.21.15.18:0/3446141448 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f42d41035e0 0x7f42d41160f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:27:55.724 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:55.725+0000 7f42d8a00700 1 -- 172.21.15.18:0/3446141448 >> 172.21.15.18:0/3446141448 conn(0x7f42d40f87f0 msgr2=0x7f42d40f9820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:27:55.724 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:55.725+0000 7f42d8a00700 1 -- 172.21.15.18:0/3446141448 shutdown_connections 2024-04-12T21:27:55.724 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:27:55.725+0000 7f42d8a00700 1 -- 172.21.15.18:0/3446141448 wait complete. 2024-04-12T21:27:56.030 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:27:55 smithi186 ceph-mon[86379]: from='client.? 172.21.15.18:0/489063285' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-12T21:27:56.030 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:27:55 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T21:27:56.031 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:27:55 smithi186 ceph-mon[86379]: from='client.? 172.21.15.18:0/3446141448' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-12T21:27:56.101 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:27:55 smithi018 ceph-mon[112590]: from='client.? 172.21.15.18:0/489063285' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-12T21:27:56.102 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:27:55 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T21:27:56.102 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:27:55 smithi018 ceph-mon[112590]: from='client.? 172.21.15.18:0/3446141448' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-12T21:27:57.030 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:27:56 smithi186 ceph-mon[86379]: from='client.44671 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:27:57.030 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:27:56 smithi186 ceph-mon[86379]: pgmap v845: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:27:57.101 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:27:56 smithi018 ceph-mon[112590]: from='client.44671 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:27:57.101 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:27:56 smithi018 ceph-mon[112590]: pgmap v845: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:27:59.030 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:27:58 smithi186 ceph-mon[86379]: pgmap v846: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:27:59.101 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:27:58 smithi018 ceph-mon[112590]: pgmap v846: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:28:01.030 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:28:00 smithi186 ceph-mon[86379]: pgmap v847: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:28:01.101 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:28:00 smithi018 ceph-mon[112590]: pgmap v847: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:28:03.030 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:28:02 smithi186 ceph-mon[86379]: pgmap v848: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:28:03.101 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:28:02 smithi018 ceph-mon[112590]: pgmap v848: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:28:05.030 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:28:04 smithi186 ceph-mon[86379]: pgmap v849: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:28:05.101 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:28:04 smithi018 ceph-mon[112590]: pgmap v849: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:28:07.101 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:28:06 smithi018 ceph-mon[112590]: pgmap v850: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:28:07.280 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:28:06 smithi186 ceph-mon[86379]: pgmap v850: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:28:09.101 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:28:08 smithi018 ceph-mon[112590]: pgmap v851: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:28:09.280 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:28:08 smithi186 ceph-mon[86379]: pgmap v851: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:28:11.083 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:28:10 smithi186 ceph-mon[86379]: pgmap v852: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:28:11.083 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:28:10 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T21:28:11.101 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:28:10 smithi018 ceph-mon[112590]: pgmap v852: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:28:11.101 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:28:10 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T21:28:13.101 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:28:12 smithi018 ceph-mon[112590]: pgmap v853: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:28:13.280 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:28:12 smithi186 ceph-mon[86379]: pgmap v853: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:28:15.101 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:28:14 smithi018 ceph-mon[112590]: pgmap v854: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:28:15.280 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:28:14 smithi186 ceph-mon[86379]: pgmap v854: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:28:17.101 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:28:16 smithi018 ceph-mon[112590]: pgmap v855: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:28:17.280 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:28:16 smithi186 ceph-mon[86379]: pgmap v855: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:28:19.101 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:28:18 smithi018 ceph-mon[112590]: pgmap v856: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:28:19.280 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:28:18 smithi186 ceph-mon[86379]: pgmap v856: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:28:21.101 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:28:20 smithi018 ceph-mon[112590]: pgmap v857: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:28:21.280 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:28:20 smithi186 ceph-mon[86379]: pgmap v857: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:28:23.101 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:28:22 smithi018 ceph-mon[112590]: pgmap v858: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:28:23.280 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:28:22 smithi186 ceph-mon[86379]: pgmap v858: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:28:24.101 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:28:23 smithi018 ceph-mon[112590]: pgmap v859: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:28:24.280 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:28:23 smithi186 ceph-mon[86379]: pgmap v859: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:28:25.282 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:28:24 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T21:28:25.351 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:28:24 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T21:28:25.898 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:28:25.900+0000 7f4b32600700 1 -- 172.21.15.18:0/1676077762 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4b2c109410 msgr2=0x7f4b2c1097e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:28:25.898 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:28:25.900+0000 7f4b32600700 1 --2- 172.21.15.18:0/1676077762 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4b2c109410 0x7f4b2c1097e0 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f4b1c004940 tx=0x7f4b1c009e30 comp rx=0 tx=0).stop 2024-04-12T21:28:25.898 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:28:25.900+0000 7f4b32600700 1 -- 172.21.15.18:0/1676077762 shutdown_connections 2024-04-12T21:28:25.898 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:28:25.900+0000 7f4b32600700 1 --2- 172.21.15.18:0/1676077762 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4b2c109db0 0x7f4b2c103420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:28:25.898 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:28:25.900+0000 7f4b32600700 1 --2- 172.21.15.18:0/1676077762 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4b2c109410 0x7f4b2c1097e0 unknown :-1 s=CLOSED pgs=723 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:28:25.898 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:28:25.900+0000 7f4b32600700 1 -- 172.21.15.18:0/1676077762 >> 172.21.15.18:0/1676077762 conn(0x7f4b2c072ba0 msgr2=0x7f4b2c074fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:28:25.898 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:28:25.900+0000 7f4b32600700 1 -- 172.21.15.18:0/1676077762 shutdown_connections 2024-04-12T21:28:25.899 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:28:25.901+0000 7f4b32600700 1 -- 172.21.15.18:0/1676077762 wait complete. 2024-04-12T21:28:25.900 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:28:25.901+0000 7f4b32600700 1 Processor -- start 2024-04-12T21:28:25.900 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:28:25.901+0000 7f4b32600700 1 -- start start 2024-04-12T21:28:25.901 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:28:25.902+0000 7f4b32600700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4b2c109410 0x7f4b2c119f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:28:25.901 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:28:25.902+0000 7f4b32600700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4b2c109db0 0x7f4b2c11a4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:28:25.901 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:28:25.902+0000 7f4b32600700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b2c11ab10 con 0x7f4b2c109db0 2024-04-12T21:28:25.901 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:28:25.902+0000 7f4b32600700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b2c1140f0 con 0x7f4b2c109410 2024-04-12T21:28:25.901 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:28:25.902+0000 7f4b2aa00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4b2c109db0 0x7f4b2c11a4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:28:25.901 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:28:25.902+0000 7f4b2aa00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4b2c109db0 0x7f4b2c11a4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:37914/0 (socket says 172.21.15.18:37914) 2024-04-12T21:28:25.901 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:28:25.902+0000 7f4b2aa00700 1 -- 172.21.15.18:0/2342266784 learned_addr learned my addr 172.21.15.18:0/2342266784 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:28:25.901 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:28:25.903+0000 7f4b2b400700 1 --2- 172.21.15.18:0/2342266784 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4b2c109410 0x7f4b2c119f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:28:25.901 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:28:25.903+0000 7f4b2aa00700 1 -- 172.21.15.18:0/2342266784 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4b2c109410 msgr2=0x7f4b2c119f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:28:25.901 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:28:25.903+0000 7f4b2aa00700 1 --2- 172.21.15.18:0/2342266784 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4b2c109410 0x7f4b2c119f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:28:25.901 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:28:25.903+0000 7f4b2aa00700 1 -- 172.21.15.18:0/2342266784 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b1c005040 con 0x7f4b2c109db0 2024-04-12T21:28:25.902 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:28:25.903+0000 7f4b2aa00700 1 --2- 172.21.15.18:0/2342266784 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4b2c109db0 0x7f4b2c11a4c0 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7f4b1800c890 tx=0x7f4b180044c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:28:25.902 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:28:25.903+0000 7f4b23400700 1 -- 172.21.15.18:0/2342266784 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4b1800a7d0 con 0x7f4b2c109db0 2024-04-12T21:28:25.902 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:28:25.904+0000 7f4b23400700 1 -- 172.21.15.18:0/2342266784 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f4b1800ae00 con 0x7f4b2c109db0 2024-04-12T21:28:25.902 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:28:25.904+0000 7f4b32600700 1 -- 172.21.15.18:0/2342266784 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4b2c1143d0 con 0x7f4b2c109db0 2024-04-12T21:28:25.902 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:28:25.904+0000 7f4b23400700 1 -- 172.21.15.18:0/2342266784 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4b180113a0 con 0x7f4b2c109db0 2024-04-12T21:28:25.903 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:28:25.904+0000 7f4b32600700 1 -- 172.21.15.18:0/2342266784 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4b2c114990 con 0x7f4b2c109db0 2024-04-12T21:28:25.903 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:28:25.905+0000 7f4b32600700 1 -- 172.21.15.18:0/2342266784 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4b2c104b60 con 0x7f4b2c109db0 2024-04-12T21:28:25.903 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:28:25.906+0000 7f4b23400700 1 -- 172.21.15.18:0/2342266784 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7f4b18012400 con 0x7f4b2c109db0 2024-04-12T21:28:25.904 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:28:25.907+0000 7f4b23400700 1 --2- 172.21.15.18:0/2342266784 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f4b1006ec00 0x7f4b100710b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:28:25.904 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:28:25.907+0000 7f4b23400700 1 -- 172.21.15.18:0/2342266784 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 6757+0+0 (secure 0 0 0) 0x7f4b1808ea40 con 0x7f4b2c109db0 2024-04-12T21:28:25.904 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:28:25.907+0000 7f4b2b400700 1 --2- 172.21.15.18:0/2342266784 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f4b1006ec00 0x7f4b100710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:28:25.905 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:28:25.908+0000 7f4b2b400700 1 --2- 172.21.15.18:0/2342266784 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f4b1006ec00 0x7f4b100710b0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f4b1c00b3b0 tx=0x7f4b1c009e30 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:28:25.909 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:28:25.912+0000 7f4b23400700 1 -- 172.21.15.18:0/2342266784 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f4b1805b270 con 0x7f4b2c109db0 2024-04-12T21:28:26.169 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:28:26.171+0000 7f4b32600700 1 -- 172.21.15.18:0/2342266784 --> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f4b2c110570 con 0x7f4b1006ec00 2024-04-12T21:28:26.169 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:28:25 smithi018 ceph-mon[112590]: pgmap v860: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:28:26.171 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:28:26.174+0000 7f4b23400700 1 -- 172.21.15.18:0/2342266784 <== mgr.34102 v2:172.21.15.18:6800/3812891919 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f4b2c110570 con 0x7f4b1006ec00 2024-04-12T21:28:26.174 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:28:26.177+0000 7f4b32600700 1 -- 172.21.15.18:0/2342266784 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f4b1006ec00 msgr2=0x7f4b100710b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:28:26.174 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:28:26.177+0000 7f4b32600700 1 --2- 172.21.15.18:0/2342266784 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f4b1006ec00 0x7f4b100710b0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f4b1c00b3b0 tx=0x7f4b1c009e30 comp rx=0 tx=0).stop 2024-04-12T21:28:26.175 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:28:26.177+0000 7f4b32600700 1 -- 172.21.15.18:0/2342266784 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4b2c109db0 msgr2=0x7f4b2c11a4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:28:26.175 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:28:26.177+0000 7f4b32600700 1 --2- 172.21.15.18:0/2342266784 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4b2c109db0 0x7f4b2c11a4c0 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7f4b1800c890 tx=0x7f4b180044c0 comp rx=0 tx=0).stop 2024-04-12T21:28:26.175 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:28:26.177+0000 7f4b32600700 1 -- 172.21.15.18:0/2342266784 shutdown_connections 2024-04-12T21:28:26.175 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:28:26.177+0000 7f4b32600700 1 --2- 172.21.15.18:0/2342266784 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f4b1006ec00 0x7f4b100710b0 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:28:26.175 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:28:26.177+0000 7f4b32600700 1 --2- 172.21.15.18:0/2342266784 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4b2c109410 0x7f4b2c119f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:28:26.175 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:28:26.177+0000 7f4b32600700 1 --2- 172.21.15.18:0/2342266784 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4b2c109db0 0x7f4b2c11a4c0 unknown :-1 s=CLOSED pgs=724 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:28:26.175 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:28:26.177+0000 7f4b32600700 1 -- 172.21.15.18:0/2342266784 >> 172.21.15.18:0/2342266784 conn(0x7f4b2c072ba0 msgr2=0x7f4b2c074d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:28:26.175 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:28:26.177+0000 7f4b32600700 1 -- 172.21.15.18:0/2342266784 shutdown_connections 2024-04-12T21:28:26.175 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:28:26.178+0000 7f4b32600700 1 -- 172.21.15.18:0/2342266784 wait complete. 2024-04-12T21:28:26.191 INFO:teuthology.orchestra.run.smithi018.stdout:true 2024-04-12T21:28:26.280 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:28:25 smithi186 ceph-mon[86379]: pgmap v860: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:28:26.331 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:28:26.334+0000 7ffb05a00700 1 -- 172.21.15.18:0/2742178348 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ffb000fde90 msgr2=0x7ffb000fe260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:28:26.331 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:28:26.334+0000 7ffb05a00700 1 --2- 172.21.15.18:0/2742178348 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ffb000fde90 0x7ffb000fe260 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7ffaf0009ce0 tx=0x7ffaf0004d10 comp rx=0 tx=0).stop 2024-04-12T21:28:26.332 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:28:26.334+0000 7ffb05a00700 1 -- 172.21.15.18:0/2742178348 shutdown_connections 2024-04-12T21:28:26.332 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:28:26.334+0000 7ffb05a00700 1 --2- 172.21.15.18:0/2742178348 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ffb000fe830 0x7ffb00106f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:28:26.332 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:28:26.334+0000 7ffb05a00700 1 --2- 172.21.15.18:0/2742178348 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ffb000fde90 0x7ffb000fe260 unknown :-1 s=CLOSED pgs=725 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:28:26.332 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:28:26.334+0000 7ffb05a00700 1 -- 172.21.15.18:0/2742178348 >> 172.21.15.18:0/2742178348 conn(0x7ffb00072bc0 msgr2=0x7ffb00072fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:28:26.332 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:28:26.334+0000 7ffb05a00700 1 -- 172.21.15.18:0/2742178348 shutdown_connections 2024-04-12T21:28:26.332 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:28:26.334+0000 7ffb05a00700 1 -- 172.21.15.18:0/2742178348 wait complete. 2024-04-12T21:28:26.334 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:28:26.335+0000 7ffb05a00700 1 Processor -- start 2024-04-12T21:28:26.334 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:28:26.335+0000 7ffb05a00700 1 -- start start 2024-04-12T21:28:26.334 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:28:26.336+0000 7ffb05a00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ffb000fde90 0x7ffb00115d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:28:26.335 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:28:26.336+0000 7ffb05a00700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ffb000fe830 0x7ffb00116250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:28:26.335 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:28:26.336+0000 7ffb05a00700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb001168a0 con 0x7ffb000fde90 2024-04-12T21:28:26.335 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:28:26.336+0000 7ffb05a00700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb0010fd20 con 0x7ffb000fe830 2024-04-12T21:28:26.335 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:28:26.336+0000 7ffafea00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ffb000fde90 0x7ffb00115d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:28:26.335 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:28:26.336+0000 7ffafea00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ffb000fde90 0x7ffb00115d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:37934/0 (socket says 172.21.15.18:37934) 2024-04-12T21:28:26.335 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:28:26.336+0000 7ffafea00700 1 -- 172.21.15.18:0/4010525991 learned_addr learned my addr 172.21.15.18:0/4010525991 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:28:26.335 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:28:26.336+0000 7ffafea00700 1 -- 172.21.15.18:0/4010525991 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ffb000fe830 msgr2=0x7ffb00116250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:28:26.335 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:28:26.336+0000 7ffafe000700 1 --2- 172.21.15.18:0/4010525991 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ffb000fe830 0x7ffb00116250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:28:26.335 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:28:26.336+0000 7ffafea00700 1 --2- 172.21.15.18:0/4010525991 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ffb000fe830 0x7ffb00116250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:28:26.335 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:28:26.336+0000 7ffafea00700 1 -- 172.21.15.18:0/4010525991 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffaf0009940 con 0x7ffb000fde90 2024-04-12T21:28:26.335 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:28:26.337+0000 7ffafea00700 1 --2- 172.21.15.18:0/4010525991 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ffb000fde90 0x7ffb00115d10 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7ffaf000b740 tx=0x7ffaf0004d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:28:26.335 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:28:26.337+0000 7ffaf6a00700 1 -- 172.21.15.18:0/4010525991 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ffaf000e040 con 0x7ffb000fde90 2024-04-12T21:28:26.336 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:28:26.337+0000 7ffaf6a00700 1 -- 172.21.15.18:0/4010525991 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ffaf00128e0 con 0x7ffb000fde90 2024-04-12T21:28:26.336 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:28:26.337+0000 7ffb05a00700 1 -- 172.21.15.18:0/4010525991 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffb0010ffa0 con 0x7ffb000fde90 2024-04-12T21:28:26.336 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:28:26.337+0000 7ffaf6a00700 1 -- 172.21.15.18:0/4010525991 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ffaf001c7c0 con 0x7ffb000fde90 2024-04-12T21:28:26.336 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:28:26.337+0000 7ffb05a00700 1 -- 172.21.15.18:0/4010525991 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffb001104e0 con 0x7ffb000fde90 2024-04-12T21:28:26.337 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:28:26.339+0000 7ffb05a00700 1 -- 172.21.15.18:0/4010525991 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffb00106a20 con 0x7ffb000fde90 2024-04-12T21:28:26.337 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:28:26.340+0000 7ffaf6a00700 1 -- 172.21.15.18:0/4010525991 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7ffaf0012410 con 0x7ffb000fde90 2024-04-12T21:28:26.338 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:28:26.341+0000 7ffaf6a00700 1 --2- 172.21.15.18:0/4010525991 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7ffae406ec00 0x7ffae40710b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:28:26.338 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:28:26.341+0000 7ffaf6a00700 1 -- 172.21.15.18:0/4010525991 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 6757+0+0 (secure 0 0 0) 0x7ffaf0090820 con 0x7ffb000fde90 2024-04-12T21:28:26.338 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:28:26.341+0000 7ffafe000700 1 --2- 172.21.15.18:0/4010525991 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7ffae406ec00 0x7ffae40710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:28:26.339 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:28:26.342+0000 7ffafe000700 1 --2- 172.21.15.18:0/4010525991 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7ffae406ec00 0x7ffae40710b0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7ffb00107330 tx=0x7ffaec009040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:28:26.342 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:28:26.346+0000 7ffaf6a00700 1 -- 172.21.15.18:0/4010525991 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7ffaf0059420 con 0x7ffb000fde90 2024-04-12T21:28:26.596 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:28:26.598+0000 7ffb05a00700 1 -- 172.21.15.18:0/4010525991 --> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7ffb00111410 con 0x7ffae406ec00 2024-04-12T21:28:26.597 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:28:26.600+0000 7ffaf6a00700 1 -- 172.21.15.18:0/4010525991 <== mgr.34102 v2:172.21.15.18:6800/3812891919 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7ffb00111410 con 0x7ffae406ec00 2024-04-12T21:28:26.601 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:28:26.603+0000 7ffb05a00700 1 -- 172.21.15.18:0/4010525991 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7ffae406ec00 msgr2=0x7ffae40710b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:28:26.601 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:28:26.603+0000 7ffb05a00700 1 --2- 172.21.15.18:0/4010525991 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7ffae406ec00 0x7ffae40710b0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7ffb00107330 tx=0x7ffaec009040 comp rx=0 tx=0).stop 2024-04-12T21:28:26.601 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:28:26.603+0000 7ffb05a00700 1 -- 172.21.15.18:0/4010525991 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ffb000fde90 msgr2=0x7ffb00115d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:28:26.601 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:28:26.603+0000 7ffb05a00700 1 --2- 172.21.15.18:0/4010525991 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ffb000fde90 0x7ffb00115d10 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7ffaf000b740 tx=0x7ffaf0004d10 comp rx=0 tx=0).stop 2024-04-12T21:28:26.601 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:28:26.603+0000 7ffb05a00700 1 -- 172.21.15.18:0/4010525991 shutdown_connections 2024-04-12T21:28:26.601 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:28:26.603+0000 7ffb05a00700 1 --2- 172.21.15.18:0/4010525991 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7ffae406ec00 0x7ffae40710b0 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:28:26.601 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:28:26.603+0000 7ffb05a00700 1 --2- 172.21.15.18:0/4010525991 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ffb000fde90 0x7ffb00115d10 unknown :-1 s=CLOSED pgs=726 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:28:26.601 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:28:26.603+0000 7ffb05a00700 1 --2- 172.21.15.18:0/4010525991 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ffb000fe830 0x7ffb00116250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:28:26.601 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:28:26.603+0000 7ffb05a00700 1 -- 172.21.15.18:0/4010525991 >> 172.21.15.18:0/4010525991 conn(0x7ffb00072bc0 msgr2=0x7ffb000fb840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:28:26.601 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:28:26.604+0000 7ffb05a00700 1 -- 172.21.15.18:0/4010525991 shutdown_connections 2024-04-12T21:28:26.602 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:28:26.604+0000 7ffb05a00700 1 -- 172.21.15.18:0/4010525991 wait complete. 2024-04-12T21:28:26.749 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:28:26.751+0000 7fbf25800700 1 -- 172.21.15.18:0/2060218730 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fbf200fb570 msgr2=0x7fbf200fb9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:28:26.749 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:28:26.751+0000 7fbf25800700 1 --2- 172.21.15.18:0/2060218730 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fbf200fb570 0x7fbf200fb9e0 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7fbf0c004940 tx=0x7fbf0c009e30 comp rx=0 tx=0).stop 2024-04-12T21:28:26.749 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:28:26.752+0000 7fbf25800700 1 -- 172.21.15.18:0/2060218730 shutdown_connections 2024-04-12T21:28:26.749 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:28:26.752+0000 7fbf25800700 1 --2- 172.21.15.18:0/2060218730 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fbf200fb570 0x7fbf200fb9e0 unknown :-1 s=CLOSED pgs=727 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:28:26.749 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:28:26.752+0000 7fbf25800700 1 --2- 172.21.15.18:0/2060218730 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbf20101530 0x7fbf20101900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:28:26.750 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:28:26.752+0000 7fbf25800700 1 -- 172.21.15.18:0/2060218730 >> 172.21.15.18:0/2060218730 conn(0x7fbf200714c0 msgr2=0x7fbf200718c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:28:26.750 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:28:26.752+0000 7fbf25800700 1 -- 172.21.15.18:0/2060218730 shutdown_connections 2024-04-12T21:28:26.750 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:28:26.752+0000 7fbf25800700 1 -- 172.21.15.18:0/2060218730 wait complete. 2024-04-12T21:28:26.751 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:28:26.753+0000 7fbf25800700 1 Processor -- start 2024-04-12T21:28:26.752 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:28:26.753+0000 7fbf25800700 1 -- start start 2024-04-12T21:28:26.752 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:28:26.753+0000 7fbf25800700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fbf200fb570 0x7fbf200fd150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:28:26.752 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:28:26.753+0000 7fbf25800700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbf20101530 0x7fbf200fd690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:28:26.752 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:28:26.753+0000 7fbf25800700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf200ff0f0 con 0x7fbf200fb570 2024-04-12T21:28:26.752 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:28:26.753+0000 7fbf25800700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf200fdbd0 con 0x7fbf20101530 2024-04-12T21:28:26.752 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:28:26.754+0000 7fbf1ea00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fbf200fb570 0x7fbf200fd150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:28:26.752 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:28:26.754+0000 7fbf1ea00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fbf200fb570 0x7fbf200fd150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:37946/0 (socket says 172.21.15.18:37946) 2024-04-12T21:28:26.752 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:28:26.754+0000 7fbf1ea00700 1 -- 172.21.15.18:0/2162679676 learned_addr learned my addr 172.21.15.18:0/2162679676 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:28:26.752 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:28:26.754+0000 7fbf1e000700 1 --2- 172.21.15.18:0/2162679676 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbf20101530 0x7fbf200fd690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:28:26.752 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:28:26.754+0000 7fbf1ea00700 1 -- 172.21.15.18:0/2162679676 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbf20101530 msgr2=0x7fbf200fd690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:28:26.752 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:28:26.754+0000 7fbf1ea00700 1 --2- 172.21.15.18:0/2162679676 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbf20101530 0x7fbf200fd690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:28:26.752 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:28:26.754+0000 7fbf1ea00700 1 -- 172.21.15.18:0/2162679676 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf0c005040 con 0x7fbf200fb570 2024-04-12T21:28:26.753 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:28:26.755+0000 7fbf1ea00700 1 --2- 172.21.15.18:0/2162679676 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fbf200fb570 0x7fbf200fd150 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7fbf1000c810 tx=0x7fbf10004470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:28:26.753 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:28:26.755+0000 7fbf16a00700 1 -- 172.21.15.18:0/2162679676 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbf10012810 con 0x7fbf200fb570 2024-04-12T21:28:26.753 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:28:26.755+0000 7fbf16a00700 1 -- 172.21.15.18:0/2162679676 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fbf10012e40 con 0x7fbf200fb570 2024-04-12T21:28:26.754 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:28:26.755+0000 7fbf16a00700 1 -- 172.21.15.18:0/2162679676 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbf1000a370 con 0x7fbf200fb570 2024-04-12T21:28:26.754 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:28:26.755+0000 7fbf25800700 1 -- 172.21.15.18:0/2162679676 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbf200fde80 con 0x7fbf200fb570 2024-04-12T21:28:26.754 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:28:26.755+0000 7fbf25800700 1 -- 172.21.15.18:0/2162679676 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbf2011a0f0 con 0x7fbf200fb570 2024-04-12T21:28:26.754 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:28:26.756+0000 7fbf25800700 1 -- 172.21.15.18:0/2162679676 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbf201059f0 con 0x7fbf200fb570 2024-04-12T21:28:26.754 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:28:26.757+0000 7fbf16a00700 1 -- 172.21.15.18:0/2162679676 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7fbf10011810 con 0x7fbf200fb570 2024-04-12T21:28:26.755 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:28:26.758+0000 7fbf16a00700 1 --2- 172.21.15.18:0/2162679676 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fbf0406ec00 0x7fbf040710b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:28:26.755 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:28:26.758+0000 7fbf16a00700 1 -- 172.21.15.18:0/2162679676 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 6757+0+0 (secure 0 0 0) 0x7fbf1008ee90 con 0x7fbf200fb570 2024-04-12T21:28:26.755 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:28:26.758+0000 7fbf1e000700 1 --2- 172.21.15.18:0/2162679676 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fbf0406ec00 0x7fbf040710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:28:26.756 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:28:26.759+0000 7fbf1e000700 1 --2- 172.21.15.18:0/2162679676 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fbf0406ec00 0x7fbf040710b0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7fbf0c00b3b0 tx=0x7fbf0c009e30 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:28:26.760 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:28:26.763+0000 7fbf16a00700 1 -- 172.21.15.18:0/2162679676 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7fbf10057a90 con 0x7fbf200fb570 2024-04-12T21:28:27.001 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:28:27.003+0000 7fbf25800700 1 -- 172.21.15.18:0/2162679676 --> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fbf200fe630 con 0x7fbf0406ec00 2024-04-12T21:28:27.012 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:28:27.015+0000 7fbf16a00700 1 -- 172.21.15.18:0/2162679676 <== mgr.34102 v2:172.21.15.18:6800/3812891919 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7fbf200fe630 con 0x7fbf0406ec00 2024-04-12T21:28:27.013 INFO:teuthology.orchestra.run.smithi018.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-12T21:28:27.013 INFO:teuthology.orchestra.run.smithi018.stdout:alertmanager.smithi018 smithi018 *:9093,9094 running (29m) 5m ago 43m 15.5M - 0.25.0 c8568f914cd2 c332090068e8 2024-04-12T21:28:27.014 INFO:teuthology.orchestra.run.smithi018.stdout:crash.smithi018 smithi018 running (25m) 5m ago 43m 6665k - 19.0.0-2879-g324baf8d 8717720dc5e5 b7f80d1f7982 2024-04-12T21:28:27.014 INFO:teuthology.orchestra.run.smithi018.stdout:crash.smithi186 smithi186 running (25m) 5m ago 42m 6660k - 19.0.0-2879-g324baf8d 8717720dc5e5 a4f55c81b623 2024-04-12T21:28:27.014 INFO:teuthology.orchestra.run.smithi018.stdout:grafana.smithi018 smithi018 *:3000 running (28m) 5m ago 43m 74.8M - 9.4.12 2bacad6d85d8 b08e3bf176dd 2024-04-12T21:28:27.014 INFO:teuthology.orchestra.run.smithi018.stdout:mds.cephfs.smithi018.acqbsc smithi018 running (21m) 5m ago 36m 16.7M - 19.0.0-2879-g324baf8d 8717720dc5e5 021f99ab24ae 2024-04-12T21:28:27.014 INFO:teuthology.orchestra.run.smithi018.stdout:mds.cephfs.smithi018.szjluo smithi018 running (20m) 5m ago 36m 16.5M - 19.0.0-2879-g324baf8d 8717720dc5e5 0003117479d4 2024-04-12T21:28:27.014 INFO:teuthology.orchestra.run.smithi018.stdout:mds.cephfs.smithi186.jdllyn smithi186 running (5m) 5m ago 36m 11.0M - 19.0.0-2879-g324baf8d 8717720dc5e5 21b1f46eef23 2024-04-12T21:28:27.014 INFO:teuthology.orchestra.run.smithi018.stdout:mds.cephfs.smithi186.wjnafs smithi186 running (36m) 5m ago 36m 19.5M - 17.2.7-827-gd3b5060a 21de9b7369ad 74b115d6d6fd 2024-04-12T21:28:27.014 INFO:teuthology.orchestra.run.smithi018.stdout:mgr.smithi018.xjzwga smithi018 *:8443,9283,8765 running (31m) 5m ago 45m 512M - 19.0.0-2879-g324baf8d 8717720dc5e5 56088305eab7 2024-04-12T21:28:27.014 INFO:teuthology.orchestra.run.smithi018.stdout:mgr.smithi186.kqcfxk smithi186 *:8443,9283,8765 running (30m) 5m ago 42m 459M - 19.0.0-2879-g324baf8d 8717720dc5e5 09cd766e5e0d 2024-04-12T21:28:27.014 INFO:teuthology.orchestra.run.smithi018.stdout:mon.smithi018 smithi018 running (26m) 5m ago 45m 73.1M 2048M 19.0.0-2879-g324baf8d 8717720dc5e5 df3e201dfed6 2024-04-12T21:28:27.014 INFO:teuthology.orchestra.run.smithi018.stdout:mon.smithi186 smithi186 running (26m) 5m ago 42m 47.2M 2048M 19.0.0-2879-g324baf8d 8717720dc5e5 c77498da9b2b 2024-04-12T21:28:27.014 INFO:teuthology.orchestra.run.smithi018.stdout:node-exporter.smithi018 smithi018 *:9100 running (30m) 5m ago 43m 8108k - 1.5.0 0da6a335fe13 326c3d9c6847 2024-04-12T21:28:27.014 INFO:teuthology.orchestra.run.smithi018.stdout:node-exporter.smithi186 smithi186 *:9100 running (29m) 5m ago 42m 8757k - 1.5.0 0da6a335fe13 ec2112baeecf 2024-04-12T21:28:27.015 INFO:teuthology.orchestra.run.smithi018.stdout:osd.0 smithi018 running (24m) 5m ago 41m 123M 1559M 19.0.0-2879-g324baf8d 8717720dc5e5 60b1273ca047 2024-04-12T21:28:27.015 INFO:teuthology.orchestra.run.smithi018.stdout:osd.1 smithi018 running (24m) 5m ago 40m 95.1M 1559M 19.0.0-2879-g324baf8d 8717720dc5e5 170ce1e5fb8d 2024-04-12T21:28:27.015 INFO:teuthology.orchestra.run.smithi018.stdout:osd.2 smithi018 running (23m) 5m ago 39m 94.3M 1559M 19.0.0-2879-g324baf8d 8717720dc5e5 c2cbc6aa2839 2024-04-12T21:28:27.015 INFO:teuthology.orchestra.run.smithi018.stdout:osd.3 smithi186 running (23m) 5m ago 39m 101M 2581M 19.0.0-2879-g324baf8d 8717720dc5e5 0138e8262e4b 2024-04-12T21:28:27.015 INFO:teuthology.orchestra.run.smithi018.stdout:osd.4 smithi186 running (22m) 5m ago 38m 91.0M 2581M 19.0.0-2879-g324baf8d 8717720dc5e5 b20cb63b25c9 2024-04-12T21:28:27.015 INFO:teuthology.orchestra.run.smithi018.stdout:osd.5 smithi186 running (21m) 5m ago 38m 92.0M 2581M 19.0.0-2879-g324baf8d 8717720dc5e5 2513bea2610e 2024-04-12T21:28:27.015 INFO:teuthology.orchestra.run.smithi018.stdout:prometheus.smithi018 smithi018 *:9095 running (29m) 5m ago 42m 67.9M - 2.43.0 a07b618ecd1d f58841127e21 2024-04-12T21:28:27.015 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:28:27.018+0000 7fbf25800700 1 -- 172.21.15.18:0/2162679676 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fbf0406ec00 msgr2=0x7fbf040710b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:28:27.015 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:28:27.018+0000 7fbf25800700 1 --2- 172.21.15.18:0/2162679676 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fbf0406ec00 0x7fbf040710b0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7fbf0c00b3b0 tx=0x7fbf0c009e30 comp rx=0 tx=0).stop 2024-04-12T21:28:27.015 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:28:27.018+0000 7fbf25800700 1 -- 172.21.15.18:0/2162679676 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fbf200fb570 msgr2=0x7fbf200fd150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:28:27.015 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:28:27.018+0000 7fbf25800700 1 --2- 172.21.15.18:0/2162679676 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fbf200fb570 0x7fbf200fd150 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7fbf1000c810 tx=0x7fbf10004470 comp rx=0 tx=0).stop 2024-04-12T21:28:27.016 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:28:27.018+0000 7fbf25800700 1 -- 172.21.15.18:0/2162679676 shutdown_connections 2024-04-12T21:28:27.016 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:28:27.018+0000 7fbf25800700 1 --2- 172.21.15.18:0/2162679676 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fbf0406ec00 0x7fbf040710b0 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:28:27.016 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:28:27.018+0000 7fbf25800700 1 --2- 172.21.15.18:0/2162679676 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fbf200fb570 0x7fbf200fd150 unknown :-1 s=CLOSED pgs=728 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:28:27.016 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:28:27.018+0000 7fbf25800700 1 --2- 172.21.15.18:0/2162679676 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbf20101530 0x7fbf200fd690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:28:27.016 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:28:27.018+0000 7fbf25800700 1 -- 172.21.15.18:0/2162679676 >> 172.21.15.18:0/2162679676 conn(0x7fbf200714c0 msgr2=0x7fbf200fa830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:28:27.016 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:28:27.018+0000 7fbf25800700 1 -- 172.21.15.18:0/2162679676 shutdown_connections 2024-04-12T21:28:27.016 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:28:27.019+0000 7fbf25800700 1 -- 172.21.15.18:0/2162679676 wait complete. 2024-04-12T21:28:27.174 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:28:27.176+0000 7f1fa0000700 1 -- 172.21.15.18:0/1542042894 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1f981037a0 msgr2=0x7f1f980fce10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:28:27.174 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:28:27.176+0000 7f1fa0000700 1 --2- 172.21.15.18:0/1542042894 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1f981037a0 0x7f1f980fce10 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7f1f84004940 tx=0x7f1f84009e30 comp rx=0 tx=0).stop 2024-04-12T21:28:27.174 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:28:27.176+0000 7f1fa0000700 1 -- 172.21.15.18:0/1542042894 shutdown_connections 2024-04-12T21:28:27.174 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:28:27.176+0000 7f1fa0000700 1 --2- 172.21.15.18:0/1542042894 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1f981037a0 0x7f1f980fce10 unknown :-1 s=CLOSED pgs=729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:28:27.174 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:28:27.176+0000 7f1fa0000700 1 --2- 172.21.15.18:0/1542042894 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1f98102e00 0x7f1f981031d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:28:27.174 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:28:27.176+0000 7f1fa0000700 1 -- 172.21.15.18:0/1542042894 >> 172.21.15.18:0/1542042894 conn(0x7f1f980f89b0 msgr2=0x7f1f980fadc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:28:27.174 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:28:27.177+0000 7f1fa0000700 1 -- 172.21.15.18:0/1542042894 shutdown_connections 2024-04-12T21:28:27.174 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:28:27.177+0000 7f1fa0000700 1 -- 172.21.15.18:0/1542042894 wait complete. 2024-04-12T21:28:27.175 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:28:27.177+0000 7f1fa0000700 1 Processor -- start 2024-04-12T21:28:27.175 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:28:27.178+0000 7f1fa0000700 1 -- start start 2024-04-12T21:28:27.175 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:28:27.178+0000 7f1fa0000700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1f98102e00 0x7f1f98115bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:28:27.175 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:28:27.178+0000 7f1fa0000700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1f981037a0 0x7f1f981160f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:28:27.175 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:28:27.178+0000 7f1fa0000700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f98116740 con 0x7f1f981037a0 2024-04-12T21:28:27.175 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:28:27.178+0000 7f1fa0000700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f9810fd20 con 0x7f1f98102e00 2024-04-12T21:28:27.176 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:28:27.178+0000 7f1f9cc00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1f981037a0 0x7f1f981160f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:28:27.176 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:28:27.178+0000 7f1f9cc00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1f981037a0 0x7f1f981160f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:37958/0 (socket says 172.21.15.18:37958) 2024-04-12T21:28:27.176 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:28:27.178+0000 7f1f9cc00700 1 -- 172.21.15.18:0/436751468 learned_addr learned my addr 172.21.15.18:0/436751468 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:28:27.176 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:28:27.179+0000 7f1f9d600700 1 --2- 172.21.15.18:0/436751468 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1f98102e00 0x7f1f98115bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:28:27.176 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:28:27.179+0000 7f1f9cc00700 1 -- 172.21.15.18:0/436751468 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1f98102e00 msgr2=0x7f1f98115bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:28:27.176 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:28:27.179+0000 7f1f9cc00700 1 --2- 172.21.15.18:0/436751468 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1f98102e00 0x7f1f98115bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:28:27.177 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:28:27.179+0000 7f1f9cc00700 1 -- 172.21.15.18:0/436751468 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f84005040 con 0x7f1f981037a0 2024-04-12T21:28:27.177 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:28:27.179+0000 7f1f9cc00700 1 --2- 172.21.15.18:0/436751468 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1f981037a0 0x7f1f981160f0 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7f1f84009100 tx=0x7f1f84009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:28:27.177 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:28:27.180+0000 7f1f95600700 1 -- 172.21.15.18:0/436751468 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1f8400e040 con 0x7f1f981037a0 2024-04-12T21:28:27.177 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:28:27.180+0000 7f1f95600700 1 -- 172.21.15.18:0/436751468 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f1f84013420 con 0x7f1f981037a0 2024-04-12T21:28:27.177 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:28:27.180+0000 7f1fa0000700 1 -- 172.21.15.18:0/436751468 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1f9810ffa0 con 0x7f1f981037a0 2024-04-12T21:28:27.177 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:28:27.180+0000 7f1f95600700 1 -- 172.21.15.18:0/436751468 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1f8401c610 con 0x7f1f981037a0 2024-04-12T21:28:27.177 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:28:27.180+0000 7f1fa0000700 1 -- 172.21.15.18:0/436751468 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1f98110560 con 0x7f1f981037a0 2024-04-12T21:28:27.178 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:28:27.181+0000 7f1fa0000700 1 -- 172.21.15.18:0/436751468 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1f98101870 con 0x7f1f981037a0 2024-04-12T21:28:27.179 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:28:27.182+0000 7f1f95600700 1 -- 172.21.15.18:0/436751468 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7f1f84013590 con 0x7f1f981037a0 2024-04-12T21:28:27.180 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:28:27.183+0000 7f1f95600700 1 --2- 172.21.15.18:0/436751468 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f1f8006ec00 0x7f1f800710b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:28:27.180 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:28:27.183+0000 7f1f95600700 1 -- 172.21.15.18:0/436751468 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 6757+0+0 (secure 0 0 0) 0x7f1f8408fc30 con 0x7f1f981037a0 2024-04-12T21:28:27.180 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:28:27.183+0000 7f1f9d600700 1 --2- 172.21.15.18:0/436751468 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f1f8006ec00 0x7f1f800710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:28:27.181 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:28:27.184+0000 7f1f9d600700 1 --2- 172.21.15.18:0/436751468 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f1f8006ec00 0x7f1f800710b0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f1f8c004cc0 tx=0x7f1f8c00b040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:28:27.186 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:28:27.189+0000 7f1f95600700 1 -- 172.21.15.18:0/436751468 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f1f8405c480 con 0x7f1f981037a0 2024-04-12T21:28:27.280 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:28:26 smithi186 ceph-mon[86379]: from='client.35512 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:28:27.280 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:28:26 smithi186 ceph-mon[86379]: from='client.35516 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:28:27.351 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:28:26 smithi018 ceph-mon[112590]: from='client.35512 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:28:27.351 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:28:26 smithi018 ceph-mon[112590]: from='client.35516 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:28:27.503 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:28:27.506+0000 7f1fa0000700 1 -- 172.21.15.18:0/436751468 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f1f98111200 con 0x7f1f981037a0 2024-04-12T21:28:27.504 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:28:27.507+0000 7f1f95600700 1 -- 172.21.15.18:0/436751468 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+837 (secure 0 0 0) 0x7f1f8405c010 con 0x7f1f981037a0 2024-04-12T21:28:27.504 INFO:teuthology.orchestra.run.smithi018.stdout:{ 2024-04-12T21:28:27.504 INFO:teuthology.orchestra.run.smithi018.stdout: "mon": { 2024-04-12T21:28:27.504 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 19.0.0-2879-g324baf8d (324baf8ddb91dea46cac50cb535c306115791355) squid (dev)": 2 2024-04-12T21:28:27.504 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-12T21:28:27.505 INFO:teuthology.orchestra.run.smithi018.stdout: "mgr": { 2024-04-12T21:28:27.505 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 19.0.0-2879-g324baf8d (324baf8ddb91dea46cac50cb535c306115791355) squid (dev)": 2 2024-04-12T21:28:27.505 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-12T21:28:27.505 INFO:teuthology.orchestra.run.smithi018.stdout: "osd": { 2024-04-12T21:28:27.505 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 19.0.0-2879-g324baf8d (324baf8ddb91dea46cac50cb535c306115791355) squid (dev)": 6 2024-04-12T21:28:27.505 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-12T21:28:27.505 INFO:teuthology.orchestra.run.smithi018.stdout: "mds": { 2024-04-12T21:28:27.505 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 17.2.7-827-gd3b5060a (d3b5060a17449edfae9385cdf792afec109fca5d) quincy (stable)": 1, 2024-04-12T21:28:27.505 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 19.0.0-2879-g324baf8d (324baf8ddb91dea46cac50cb535c306115791355) squid (dev)": 3 2024-04-12T21:28:27.505 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-12T21:28:27.505 INFO:teuthology.orchestra.run.smithi018.stdout: "overall": { 2024-04-12T21:28:27.505 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 17.2.7-827-gd3b5060a (d3b5060a17449edfae9385cdf792afec109fca5d) quincy (stable)": 1, 2024-04-12T21:28:27.505 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 19.0.0-2879-g324baf8d (324baf8ddb91dea46cac50cb535c306115791355) squid (dev)": 13 2024-04-12T21:28:27.505 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-04-12T21:28:27.505 INFO:teuthology.orchestra.run.smithi018.stdout:} 2024-04-12T21:28:27.507 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:28:27.509+0000 7f1fa0000700 1 -- 172.21.15.18:0/436751468 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f1f8006ec00 msgr2=0x7f1f800710b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:28:27.507 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:28:27.509+0000 7f1fa0000700 1 --2- 172.21.15.18:0/436751468 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f1f8006ec00 0x7f1f800710b0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f1f8c004cc0 tx=0x7f1f8c00b040 comp rx=0 tx=0).stop 2024-04-12T21:28:27.507 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:28:27.510+0000 7f1fa0000700 1 -- 172.21.15.18:0/436751468 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1f981037a0 msgr2=0x7f1f981160f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:28:27.507 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:28:27.510+0000 7f1fa0000700 1 --2- 172.21.15.18:0/436751468 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1f981037a0 0x7f1f981160f0 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7f1f84009100 tx=0x7f1f84009e30 comp rx=0 tx=0).stop 2024-04-12T21:28:27.508 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:28:27.510+0000 7f1fa0000700 1 -- 172.21.15.18:0/436751468 shutdown_connections 2024-04-12T21:28:27.508 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:28:27.510+0000 7f1fa0000700 1 --2- 172.21.15.18:0/436751468 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f1f8006ec00 0x7f1f800710b0 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:28:27.508 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:28:27.510+0000 7f1fa0000700 1 --2- 172.21.15.18:0/436751468 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1f98102e00 0x7f1f98115bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:28:27.508 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:28:27.510+0000 7f1fa0000700 1 --2- 172.21.15.18:0/436751468 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1f981037a0 0x7f1f981160f0 unknown :-1 s=CLOSED pgs=730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:28:27.508 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:28:27.510+0000 7f1fa0000700 1 -- 172.21.15.18:0/436751468 >> 172.21.15.18:0/436751468 conn(0x7f1f980f89b0 msgr2=0x7f1f981021c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:28:27.508 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:28:27.510+0000 7f1fa0000700 1 -- 172.21.15.18:0/436751468 shutdown_connections 2024-04-12T21:28:27.508 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:28:27.510+0000 7f1fa0000700 1 -- 172.21.15.18:0/436751468 wait complete. 2024-04-12T21:28:27.662 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:28:27.665+0000 7f1617000700 1 -- 172.21.15.18:0/2800762210 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1610070300 msgr2=0x7f16100706d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:28:27.662 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:28:27.665+0000 7f1617000700 1 --2- 172.21.15.18:0/2800762210 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1610070300 0x7f16100706d0 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7f1600004970 tx=0x7f1600009e30 comp rx=0 tx=0).stop 2024-04-12T21:28:27.662 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:28:27.665+0000 7f1617000700 1 -- 172.21.15.18:0/2800762210 shutdown_connections 2024-04-12T21:28:27.662 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:28:27.665+0000 7f1617000700 1 --2- 172.21.15.18:0/2800762210 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1610070c10 0x7f1610109170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:28:27.662 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:28:27.665+0000 7f1617000700 1 --2- 172.21.15.18:0/2800762210 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1610070300 0x7f16100706d0 unknown :-1 s=CLOSED pgs=731 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:28:27.662 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:28:27.665+0000 7f1617000700 1 -- 172.21.15.18:0/2800762210 >> 172.21.15.18:0/2800762210 conn(0x7f16100f89b0 msgr2=0x7f16100fadc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:28:27.662 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:28:27.665+0000 7f1617000700 1 -- 172.21.15.18:0/2800762210 shutdown_connections 2024-04-12T21:28:27.663 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:28:27.665+0000 7f1617000700 1 -- 172.21.15.18:0/2800762210 wait complete. 2024-04-12T21:28:27.663 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:28:27.666+0000 7f1617000700 1 Processor -- start 2024-04-12T21:28:27.663 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:28:27.666+0000 7f1617000700 1 -- start start 2024-04-12T21:28:27.664 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:28:27.667+0000 7f1617000700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1610070300 0x7f1610115af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:28:27.664 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:28:27.667+0000 7f1617000700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1610070c10 0x7f1610116030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:28:27.664 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:28:27.667+0000 7f1617000700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1610116680 con 0x7f1610070300 2024-04-12T21:28:27.664 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:28:27.667+0000 7f1617000700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f161010fca0 con 0x7f1610070c10 2024-04-12T21:28:27.664 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:28:27.667+0000 7f160fe00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1610070300 0x7f1610115af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:28:27.664 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:28:27.667+0000 7f160fe00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1610070300 0x7f1610115af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:37980/0 (socket says 172.21.15.18:37980) 2024-04-12T21:28:27.664 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:28:27.667+0000 7f160fe00700 1 -- 172.21.15.18:0/3590163251 learned_addr learned my addr 172.21.15.18:0/3590163251 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:28:27.665 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:28:27.667+0000 7f160fe00700 1 -- 172.21.15.18:0/3590163251 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1610070c10 msgr2=0x7f1610116030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:28:27.665 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:28:27.667+0000 7f160f400700 1 --2- 172.21.15.18:0/3590163251 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1610070c10 0x7f1610116030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:28:27.665 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:28:27.667+0000 7f160fe00700 1 --2- 172.21.15.18:0/3590163251 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1610070c10 0x7f1610116030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:28:27.665 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:28:27.667+0000 7f160fe00700 1 -- 172.21.15.18:0/3590163251 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1600005040 con 0x7f1610070300 2024-04-12T21:28:27.665 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:28:27.668+0000 7f160fe00700 1 --2- 172.21.15.18:0/3590163251 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1610070300 0x7f1610115af0 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7f1600000c00 tx=0x7f1600009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:28:27.665 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:28:27.668+0000 7f1607e00700 1 -- 172.21.15.18:0/3590163251 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f160000e040 con 0x7f1610070300 2024-04-12T21:28:27.666 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:28:27.668+0000 7f1607e00700 1 -- 172.21.15.18:0/3590163251 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f160000bc10 con 0x7f1610070300 2024-04-12T21:28:27.666 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:28:27.668+0000 7f1617000700 1 -- 172.21.15.18:0/3590163251 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f161010ff20 con 0x7f1610070300 2024-04-12T21:28:27.666 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:28:27.668+0000 7f1607e00700 1 -- 172.21.15.18:0/3590163251 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f160001c630 con 0x7f1610070300 2024-04-12T21:28:27.666 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:28:27.668+0000 7f1617000700 1 -- 172.21.15.18:0/3590163251 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1610110460 con 0x7f1610070300 2024-04-12T21:28:27.666 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:28:27.669+0000 7f1617000700 1 -- 172.21.15.18:0/3590163251 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1610071a40 con 0x7f1610070300 2024-04-12T21:28:27.667 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:28:27.670+0000 7f1607e00700 1 -- 172.21.15.18:0/3590163251 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7f160000bd80 con 0x7f1610070300 2024-04-12T21:28:27.668 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:28:27.671+0000 7f1607e00700 1 --2- 172.21.15.18:0/3590163251 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f15f406ec00 0x7f15f40710b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:28:27.668 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:28:27.671+0000 7f1607e00700 1 -- 172.21.15.18:0/3590163251 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 6757+0+0 (secure 0 0 0) 0x7f160008fd40 con 0x7f1610070300 2024-04-12T21:28:27.669 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:28:27.672+0000 7f160f400700 1 --2- 172.21.15.18:0/3590163251 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f15f406ec00 0x7f15f40710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:28:27.669 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:28:27.672+0000 7f160f400700 1 --2- 172.21.15.18:0/3590163251 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f15f406ec00 0x7f15f40710b0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f16101113d0 tx=0x7f15fc009040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:28:27.673 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:28:27.676+0000 7f1607e00700 1 -- 172.21.15.18:0/3590163251 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f16000589c0 con 0x7f1610070300 2024-04-12T21:28:27.942 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:28:27.945+0000 7f1617000700 1 -- 172.21.15.18:0/3590163251 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f1610111180 con 0x7f1610070300 2024-04-12T21:28:27.943 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:28:27.946+0000 7f1607e00700 1 -- 172.21.15.18:0/3590163251 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 31 v31) v1 ==== 76+0+1873 (secure 0 0 0) 0x7f1600013e60 con 0x7f1610070300 2024-04-12T21:28:27.944 INFO:teuthology.orchestra.run.smithi018.stdout:e31 2024-04-12T21:28:27.944 INFO:teuthology.orchestra.run.smithi018.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-12T21:28:27.944 INFO:teuthology.orchestra.run.smithi018.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-12T21:28:27.944 INFO:teuthology.orchestra.run.smithi018.stdout:legacy client fscid: 1 2024-04-12T21:28:27.944 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-12T21:28:27.944 INFO:teuthology.orchestra.run.smithi018.stdout:Filesystem 'cephfs' (1) 2024-04-12T21:28:27.944 INFO:teuthology.orchestra.run.smithi018.stdout:fs_name cephfs 2024-04-12T21:28:27.945 INFO:teuthology.orchestra.run.smithi018.stdout:epoch 24 2024-04-12T21:28:27.945 INFO:teuthology.orchestra.run.smithi018.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-04-12T21:28:27.945 INFO:teuthology.orchestra.run.smithi018.stdout:created 2024-04-12T20:51:57.362850+0000 2024-04-12T21:28:27.945 INFO:teuthology.orchestra.run.smithi018.stdout:modified 2024-04-12T21:06:59.717190+0000 2024-04-12T21:28:27.945 INFO:teuthology.orchestra.run.smithi018.stdout:tableserver 0 2024-04-12T21:28:27.945 INFO:teuthology.orchestra.run.smithi018.stdout:root 0 2024-04-12T21:28:27.945 INFO:teuthology.orchestra.run.smithi018.stdout:session_timeout 60 2024-04-12T21:28:27.945 INFO:teuthology.orchestra.run.smithi018.stdout:session_autoclose 300 2024-04-12T21:28:27.945 INFO:teuthology.orchestra.run.smithi018.stdout:max_file_size 1099511627776 2024-04-12T21:28:27.945 INFO:teuthology.orchestra.run.smithi018.stdout:max_xattr_size 65536 2024-04-12T21:28:27.945 INFO:teuthology.orchestra.run.smithi018.stdout:required_client_features {} 2024-04-12T21:28:27.945 INFO:teuthology.orchestra.run.smithi018.stdout:last_failure 0 2024-04-12T21:28:27.945 INFO:teuthology.orchestra.run.smithi018.stdout:last_failure_osd_epoch 100 2024-04-12T21:28:27.945 INFO:teuthology.orchestra.run.smithi018.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-12T21:28:27.945 INFO:teuthology.orchestra.run.smithi018.stdout:max_mds 2 2024-04-12T21:28:27.946 INFO:teuthology.orchestra.run.smithi018.stdout:in 0,1 2024-04-12T21:28:27.946 INFO:teuthology.orchestra.run.smithi018.stdout:up {} 2024-04-12T21:28:27.946 INFO:teuthology.orchestra.run.smithi018.stdout:failed 0,1 2024-04-12T21:28:27.946 INFO:teuthology.orchestra.run.smithi018.stdout:damaged 2024-04-12T21:28:27.946 INFO:teuthology.orchestra.run.smithi018.stdout:stopped 2024-04-12T21:28:27.946 INFO:teuthology.orchestra.run.smithi018.stdout:data_pools [3] 2024-04-12T21:28:27.946 INFO:teuthology.orchestra.run.smithi018.stdout:metadata_pool 2 2024-04-12T21:28:27.946 INFO:teuthology.orchestra.run.smithi018.stdout:inline_data disabled 2024-04-12T21:28:27.946 INFO:teuthology.orchestra.run.smithi018.stdout:balancer 2024-04-12T21:28:27.946 INFO:teuthology.orchestra.run.smithi018.stdout:bal_rank_mask -1 2024-04-12T21:28:27.946 INFO:teuthology.orchestra.run.smithi018.stdout:standby_count_wanted 1 2024-04-12T21:28:27.946 INFO:teuthology.orchestra.run.smithi018.stdout:qdb_cluster leader: 0 members: 2024-04-12T21:28:27.946 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-12T21:28:27.946 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-12T21:28:27.946 INFO:teuthology.orchestra.run.smithi018.stdout:Standby daemons: 2024-04-12T21:28:27.947 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-12T21:28:27.947 INFO:teuthology.orchestra.run.smithi018.stdout:[mds.cephfs.smithi186.wjnafs{-1:24277} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.186:6826/438347207,v1:172.21.15.186:6827/438347207] compat {c=[1],r=[1],i=[7ff]}] 2024-04-12T21:28:27.947 INFO:teuthology.orchestra.run.smithi018.stdout:[mds.cephfs.smithi018.acqbsc{-1:34436} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.18:6828/755179764,v1:172.21.15.18:6829/755179764] compat {c=[1],r=[1],i=[1fff]}] 2024-04-12T21:28:27.947 INFO:teuthology.orchestra.run.smithi018.stdout:[mds.cephfs.smithi018.szjluo{-1:34468} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.18:6826/2407992653,v1:172.21.15.18:6827/2407992653] compat {c=[1],r=[1],i=[1fff]}] 2024-04-12T21:28:27.947 INFO:teuthology.orchestra.run.smithi018.stdout:[mds.cephfs.smithi186.jdllyn{-1:44595} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.186:6824/3523444202,v1:172.21.15.186:6825/3523444202] compat {c=[1],r=[1],i=[1fff]}] 2024-04-12T21:28:27.947 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:28:27.950+0000 7f1617000700 1 -- 172.21.15.18:0/3590163251 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f15f406ec00 msgr2=0x7f15f40710b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:28:27.947 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:28:27.950+0000 7f1617000700 1 --2- 172.21.15.18:0/3590163251 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f15f406ec00 0x7f15f40710b0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f16101113d0 tx=0x7f15fc009040 comp rx=0 tx=0).stop 2024-04-12T21:28:27.948 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:28:27.950+0000 7f1617000700 1 -- 172.21.15.18:0/3590163251 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1610070300 msgr2=0x7f1610115af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:28:27.948 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:28:27.950+0000 7f1617000700 1 --2- 172.21.15.18:0/3590163251 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1610070300 0x7f1610115af0 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7f1600000c00 tx=0x7f1600009e30 comp rx=0 tx=0).stop 2024-04-12T21:28:27.948 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:28:27.950+0000 7f1617000700 1 -- 172.21.15.18:0/3590163251 shutdown_connections 2024-04-12T21:28:27.948 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:28:27.950+0000 7f1617000700 1 --2- 172.21.15.18:0/3590163251 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f15f406ec00 0x7f15f40710b0 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:28:27.948 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:28:27.950+0000 7f1617000700 1 --2- 172.21.15.18:0/3590163251 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1610070300 0x7f1610115af0 unknown :-1 s=CLOSED pgs=732 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:28:27.948 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:28:27.950+0000 7f1617000700 1 --2- 172.21.15.18:0/3590163251 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1610070c10 0x7f1610116030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:28:27.948 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:28:27.950+0000 7f1617000700 1 -- 172.21.15.18:0/3590163251 >> 172.21.15.18:0/3590163251 conn(0x7f16100f89b0 msgr2=0x7f16100f9de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:28:27.948 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:28:27.950+0000 7f1617000700 1 -- 172.21.15.18:0/3590163251 shutdown_connections 2024-04-12T21:28:27.948 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:28:27.950+0000 7f1617000700 1 -- 172.21.15.18:0/3590163251 wait complete. 2024-04-12T21:28:27.949 INFO:teuthology.orchestra.run.smithi018.stderr:dumped fsmap epoch 31 2024-04-12T21:28:28.107 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:28:28.109+0000 7f36e1200700 1 -- 172.21.15.18:0/1741426148 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f36dc070bf0 msgr2=0x7f36dc1091b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:28:28.107 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:28:28.109+0000 7f36e1200700 1 --2- 172.21.15.18:0/1741426148 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f36dc070bf0 0x7f36dc1091b0 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7f36c8004940 tx=0x7f36c8009e30 comp rx=0 tx=0).stop 2024-04-12T21:28:28.107 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:28:28.109+0000 7f36e1200700 1 -- 172.21.15.18:0/1741426148 shutdown_connections 2024-04-12T21:28:28.107 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:28:28.109+0000 7f36e1200700 1 --2- 172.21.15.18:0/1741426148 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f36dc070bf0 0x7f36dc1091b0 unknown :-1 s=CLOSED pgs=733 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:28:28.107 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:28:28.109+0000 7f36e1200700 1 --2- 172.21.15.18:0/1741426148 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f36dc0702e0 0x7f36dc0706b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:28:28.107 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:28:28.109+0000 7f36e1200700 1 -- 172.21.15.18:0/1741426148 >> 172.21.15.18:0/1741426148 conn(0x7f36dc0f8970 msgr2=0x7f36dc0fad80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:28:28.107 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:28:28.109+0000 7f36e1200700 1 -- 172.21.15.18:0/1741426148 shutdown_connections 2024-04-12T21:28:28.107 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:28:28.109+0000 7f36e1200700 1 -- 172.21.15.18:0/1741426148 wait complete. 2024-04-12T21:28:28.108 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:28:28.111+0000 7f36e1200700 1 Processor -- start 2024-04-12T21:28:28.108 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:28:28.111+0000 7f36e1200700 1 -- start start 2024-04-12T21:28:28.108 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:28:28.111+0000 7f36e1200700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f36dc0702e0 0x7f36dc115e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:28:28.109 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:28:28.111+0000 7f36e1200700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f36dc070bf0 0x7f36dc116370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:28:28.109 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:28:28.111+0000 7f36e1200700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36dc1169c0 con 0x7f36dc070bf0 2024-04-12T21:28:28.109 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:28:28.111+0000 7f36e1200700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36dc10fd90 con 0x7f36dc0702e0 2024-04-12T21:28:28.109 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:28:28.112+0000 7f36d9400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f36dc070bf0 0x7f36dc116370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:28:28.109 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:28:28.112+0000 7f36d9400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f36dc070bf0 0x7f36dc116370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:38006/0 (socket says 172.21.15.18:38006) 2024-04-12T21:28:28.109 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:28:28.112+0000 7f36d9400700 1 -- 172.21.15.18:0/1061948060 learned_addr learned my addr 172.21.15.18:0/1061948060 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:28:28.110 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:28:28.112+0000 7f36d9e00700 1 --2- 172.21.15.18:0/1061948060 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f36dc0702e0 0x7f36dc115e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:28:28.110 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:28:28.112+0000 7f36d9400700 1 -- 172.21.15.18:0/1061948060 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f36dc0702e0 msgr2=0x7f36dc115e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:28:28.110 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:28:28.112+0000 7f36d9400700 1 --2- 172.21.15.18:0/1061948060 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f36dc0702e0 0x7f36dc115e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:28:28.110 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:28:28.112+0000 7f36d9400700 1 -- 172.21.15.18:0/1061948060 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36c8005040 con 0x7f36dc070bf0 2024-04-12T21:28:28.110 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:28:28.112+0000 7f36d9400700 1 --2- 172.21.15.18:0/1061948060 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f36dc070bf0 0x7f36dc116370 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7f36c800b8b0 tx=0x7f36c8009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:28:28.110 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:28:28.113+0000 7f36d2000700 1 -- 172.21.15.18:0/1061948060 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f36c800e040 con 0x7f36dc070bf0 2024-04-12T21:28:28.111 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:28:28.113+0000 7f36d2000700 1 -- 172.21.15.18:0/1061948060 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f36c8004290 con 0x7f36dc070bf0 2024-04-12T21:28:28.111 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:28:28.113+0000 7f36e1200700 1 -- 172.21.15.18:0/1061948060 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f36dc110010 con 0x7f36dc070bf0 2024-04-12T21:28:28.111 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:28:28.113+0000 7f36d2000700 1 -- 172.21.15.18:0/1061948060 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f36c801c3f0 con 0x7f36dc070bf0 2024-04-12T21:28:28.111 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:28:28.113+0000 7f36e1200700 1 -- 172.21.15.18:0/1061948060 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f36dc110550 con 0x7f36dc070bf0 2024-04-12T21:28:28.111 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:28:28.114+0000 7f36e1200700 1 -- 172.21.15.18:0/1061948060 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f36dc071a20 con 0x7f36dc070bf0 2024-04-12T21:28:28.112 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:28:28.115+0000 7f36d2000700 1 -- 172.21.15.18:0/1061948060 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7f36c80137b0 con 0x7f36dc070bf0 2024-04-12T21:28:28.113 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:28:28.116+0000 7f36d2000700 1 --2- 172.21.15.18:0/1061948060 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f36c006ec00 0x7f36c00710b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:28:28.113 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:28:28.116+0000 7f36d2000700 1 -- 172.21.15.18:0/1061948060 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 6757+0+0 (secure 0 0 0) 0x7f36c808fbe0 con 0x7f36dc070bf0 2024-04-12T21:28:28.113 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:28:28.116+0000 7f36d9e00700 1 --2- 172.21.15.18:0/1061948060 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f36c006ec00 0x7f36c00710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:28:28.114 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:28:28.117+0000 7f36d9e00700 1 --2- 172.21.15.18:0/1061948060 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f36c006ec00 0x7f36c00710b0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f36cc004c40 tx=0x7f36cc009040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:28:28.118 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:28:28.121+0000 7f36d2000700 1 -- 172.21.15.18:0/1061948060 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f36c805c4d0 con 0x7f36dc070bf0 2024-04-12T21:28:28.280 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:28:27 smithi186 ceph-mon[86379]: from='client.35520 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:28:28.280 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:28:27 smithi186 ceph-mon[86379]: from='client.? 172.21.15.18:0/436751468' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T21:28:28.281 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:28:27 smithi186 ceph-mon[86379]: pgmap v861: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:28:28.281 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:28:27 smithi186 ceph-mon[86379]: from='client.? 172.21.15.18:0/3590163251' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-12T21:28:28.351 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:28:27 smithi018 ceph-mon[112590]: from='client.35520 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:28:28.352 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:28:27 smithi018 ceph-mon[112590]: from='client.? 172.21.15.18:0/436751468' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T21:28:28.352 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:28:27 smithi018 ceph-mon[112590]: pgmap v861: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:28:28.352 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:28:27 smithi018 ceph-mon[112590]: from='client.? 172.21.15.18:0/3590163251' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-12T21:28:28.372 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:28:28.374+0000 7f36e1200700 1 -- 172.21.15.18:0/1061948060 --> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f36dc10c100 con 0x7f36c006ec00 2024-04-12T21:28:28.373 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:28:28.376+0000 7f36d2000700 1 -- 172.21.15.18:0/1061948060 <== mgr.34102 v2:172.21.15.18:6800/3812891919 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f36dc10c100 con 0x7f36c006ec00 2024-04-12T21:28:28.374 INFO:teuthology.orchestra.run.smithi018.stdout:{ 2024-04-12T21:28:28.374 INFO:teuthology.orchestra.run.smithi018.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:f4452def3be4d350f4a8e5feaf7ebb1511be9298fe68478ba7a2d6fbd7d69a63", 2024-04-12T21:28:28.374 INFO:teuthology.orchestra.run.smithi018.stdout: "in_progress": true, 2024-04-12T21:28:28.374 INFO:teuthology.orchestra.run.smithi018.stdout: "which": "Upgrading all daemon types on all hosts", 2024-04-12T21:28:28.374 INFO:teuthology.orchestra.run.smithi018.stdout: "services_complete": [ 2024-04-12T21:28:28.374 INFO:teuthology.orchestra.run.smithi018.stdout: "crash", 2024-04-12T21:28:28.374 INFO:teuthology.orchestra.run.smithi018.stdout: "osd", 2024-04-12T21:28:28.374 INFO:teuthology.orchestra.run.smithi018.stdout: "mon", 2024-04-12T21:28:28.374 INFO:teuthology.orchestra.run.smithi018.stdout: "mgr" 2024-04-12T21:28:28.374 INFO:teuthology.orchestra.run.smithi018.stdout: ], 2024-04-12T21:28:28.374 INFO:teuthology.orchestra.run.smithi018.stdout: "progress": "15/21 daemons upgraded", 2024-04-12T21:28:28.375 INFO:teuthology.orchestra.run.smithi018.stdout: "message": "Currently upgrading mds daemons", 2024-04-12T21:28:28.375 INFO:teuthology.orchestra.run.smithi018.stdout: "is_paused": false 2024-04-12T21:28:28.375 INFO:teuthology.orchestra.run.smithi018.stdout:} 2024-04-12T21:28:28.376 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:28:28.379+0000 7f36e1200700 1 -- 172.21.15.18:0/1061948060 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f36c006ec00 msgr2=0x7f36c00710b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:28:28.376 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:28:28.379+0000 7f36e1200700 1 --2- 172.21.15.18:0/1061948060 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f36c006ec00 0x7f36c00710b0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f36cc004c40 tx=0x7f36cc009040 comp rx=0 tx=0).stop 2024-04-12T21:28:28.376 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:28:28.379+0000 7f36e1200700 1 -- 172.21.15.18:0/1061948060 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f36dc070bf0 msgr2=0x7f36dc116370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:28:28.376 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:28:28.379+0000 7f36e1200700 1 --2- 172.21.15.18:0/1061948060 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f36dc070bf0 0x7f36dc116370 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7f36c800b8b0 tx=0x7f36c8009e30 comp rx=0 tx=0).stop 2024-04-12T21:28:28.377 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:28:28.379+0000 7f36e1200700 1 -- 172.21.15.18:0/1061948060 shutdown_connections 2024-04-12T21:28:28.377 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:28:28.379+0000 7f36e1200700 1 --2- 172.21.15.18:0/1061948060 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f36c006ec00 0x7f36c00710b0 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:28:28.377 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:28:28.379+0000 7f36e1200700 1 --2- 172.21.15.18:0/1061948060 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f36dc0702e0 0x7f36dc115e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:28:28.377 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:28:28.379+0000 7f36e1200700 1 --2- 172.21.15.18:0/1061948060 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f36dc070bf0 0x7f36dc116370 unknown :-1 s=CLOSED pgs=734 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:28:28.377 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:28:28.379+0000 7f36e1200700 1 -- 172.21.15.18:0/1061948060 >> 172.21.15.18:0/1061948060 conn(0x7f36dc0f8970 msgr2=0x7f36dc0f9ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:28:28.377 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:28:28.380+0000 7f36e1200700 1 -- 172.21.15.18:0/1061948060 shutdown_connections 2024-04-12T21:28:28.377 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:28:28.380+0000 7f36e1200700 1 -- 172.21.15.18:0/1061948060 wait complete. 2024-04-12T21:28:28.537 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:28:28.540+0000 7fb2a1c00700 1 -- 172.21.15.18:0/2814029187 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb29c072e00 msgr2=0x7fb29c10d760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:28:28.537 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:28:28.540+0000 7fb2a1c00700 1 --2- 172.21.15.18:0/2814029187 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb29c072e00 0x7fb29c10d760 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7fb288004940 tx=0x7fb288009e30 comp rx=0 tx=0).stop 2024-04-12T21:28:28.537 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:28:28.540+0000 7fb2a1c00700 1 -- 172.21.15.18:0/2814029187 shutdown_connections 2024-04-12T21:28:28.538 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:28:28.540+0000 7fb2a1c00700 1 --2- 172.21.15.18:0/2814029187 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb29c072e00 0x7fb29c10d760 unknown :-1 s=CLOSED pgs=735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:28:28.538 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:28:28.540+0000 7fb2a1c00700 1 --2- 172.21.15.18:0/2814029187 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb29c0724f0 0x7fb29c0728c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:28:28.538 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:28:28.540+0000 7fb2a1c00700 1 -- 172.21.15.18:0/2814029187 >> 172.21.15.18:0/2814029187 conn(0x7fb29c0faba0 msgr2=0x7fb29c0fcfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:28:28.538 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:28:28.540+0000 7fb2a1c00700 1 -- 172.21.15.18:0/2814029187 shutdown_connections 2024-04-12T21:28:28.538 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:28:28.540+0000 7fb2a1c00700 1 -- 172.21.15.18:0/2814029187 wait complete. 2024-04-12T21:28:28.539 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:28:28.541+0000 7fb2a1c00700 1 Processor -- start 2024-04-12T21:28:28.539 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:28:28.541+0000 7fb2a1c00700 1 -- start start 2024-04-12T21:28:28.539 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:28:28.541+0000 7fb2a1c00700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb29c0724f0 0x7fb29c119f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:28:28.539 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:28:28.541+0000 7fb2a1c00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb29c072e00 0x7fb29c11a450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:28:28.539 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:28:28.541+0000 7fb2a1c00700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb29c11aaa0 con 0x7fb29c072e00 2024-04-12T21:28:28.539 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:28:28.541+0000 7fb2a1c00700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb29c114080 con 0x7fb29c0724f0 2024-04-12T21:28:28.539 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:28:28.542+0000 7fb29a000700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb29c072e00 0x7fb29c11a450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:28:28.540 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:28:28.542+0000 7fb29a000700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb29c072e00 0x7fb29c11a450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:38030/0 (socket says 172.21.15.18:38030) 2024-04-12T21:28:28.540 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:28:28.542+0000 7fb29a000700 1 -- 172.21.15.18:0/1629079200 learned_addr learned my addr 172.21.15.18:0/1629079200 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:28:28.540 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:28:28.542+0000 7fb29aa00700 1 --2- 172.21.15.18:0/1629079200 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb29c0724f0 0x7fb29c119f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:28:28.540 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:28:28.542+0000 7fb29a000700 1 -- 172.21.15.18:0/1629079200 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb29c0724f0 msgr2=0x7fb29c119f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:28:28.540 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:28:28.542+0000 7fb29a000700 1 --2- 172.21.15.18:0/1629079200 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb29c0724f0 0x7fb29c119f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:28:28.540 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:28:28.542+0000 7fb29a000700 1 -- 172.21.15.18:0/1629079200 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb288005040 con 0x7fb29c072e00 2024-04-12T21:28:28.541 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:28:28.542+0000 7fb29a000700 1 --2- 172.21.15.18:0/1629079200 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb29c072e00 0x7fb29c11a450 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7fb288004890 tx=0x7fb288009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:28:28.541 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:28:28.543+0000 7fb292a00700 1 -- 172.21.15.18:0/1629079200 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb28800e040 con 0x7fb29c072e00 2024-04-12T21:28:28.541 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:28:28.543+0000 7fb292a00700 1 -- 172.21.15.18:0/1629079200 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb288009af0 con 0x7fb29c072e00 2024-04-12T21:28:28.541 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:28:28.543+0000 7fb2a1c00700 1 -- 172.21.15.18:0/1629079200 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb29c114300 con 0x7fb29c072e00 2024-04-12T21:28:28.541 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:28:28.543+0000 7fb292a00700 1 -- 172.21.15.18:0/1629079200 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb28801c630 con 0x7fb29c072e00 2024-04-12T21:28:28.541 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:28:28.543+0000 7fb2a1c00700 1 -- 172.21.15.18:0/1629079200 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb29c1148c0 con 0x7fb29c072e00 2024-04-12T21:28:28.541 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:28:28.544+0000 7fb2a1c00700 1 -- 172.21.15.18:0/1629079200 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb29c10a5a0 con 0x7fb29c072e00 2024-04-12T21:28:28.542 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:28:28.545+0000 7fb292a00700 1 -- 172.21.15.18:0/1629079200 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7fb28800bbc0 con 0x7fb29c072e00 2024-04-12T21:28:28.543 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:28:28.546+0000 7fb292a00700 1 --2- 172.21.15.18:0/1629079200 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fb28006ec00 0x7fb2800710b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:28:28.543 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:28:28.546+0000 7fb292a00700 1 -- 172.21.15.18:0/1629079200 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 6757+0+0 (secure 0 0 0) 0x7fb28808fd50 con 0x7fb29c072e00 2024-04-12T21:28:28.543 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:28:28.546+0000 7fb29aa00700 1 --2- 172.21.15.18:0/1629079200 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fb28006ec00 0x7fb2800710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:28:28.544 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:28:28.547+0000 7fb29aa00700 1 --2- 172.21.15.18:0/1629079200 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fb28006ec00 0x7fb2800710b0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7fb28c004d60 tx=0x7fb28c00b040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:28:28.548 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:28:28.551+0000 7fb292a00700 1 -- 172.21.15.18:0/1629079200 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7fb28805c500 con 0x7fb29c072e00 2024-04-12T21:28:28.872 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:28:28.875+0000 7fb2a1c00700 1 -- 172.21.15.18:0/1629079200 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7fb29c10a5a0 con 0x7fb29c072e00 2024-04-12T21:28:28.873 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:28:28.875+0000 7fb292a00700 1 -- 172.21.15.18:0/1629079200 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+4303 (secure 0 0 0) 0x7fb28805c090 con 0x7fb29c072e00 2024-04-12T21:28:28.873 INFO:teuthology.orchestra.run.smithi018.stdout:HEALTH_ERR failed to probe daemons or devices; 1 filesystem is degraded; 1 filesystem is offline 2024-04-12T21:28:28.873 INFO:teuthology.orchestra.run.smithi018.stdout:[WRN] CEPHADM_REFRESH_FAILED: failed to probe daemons or devices 2024-04-12T21:28:28.873 INFO:teuthology.orchestra.run.smithi018.stdout: host smithi018 `cephadm ceph-volume` failed: cephadm exited with an error code: 1, stderr: Inferring config /var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/mon.smithi018/config 2024-04-12T21:28:28.873 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/podman: timeout after 895 seconds 2024-04-12T21:28:28.873 INFO:teuthology.orchestra.run.smithi018.stdout:Non-zero exit code 124 from /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:f4452def3be4d350f4a8e5feaf7ebb1511be9298fe68478ba7a2d6fbd7d69a63 -e NODE_NAME=smithi018 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968:/var/run/ceph:z -v /var/log/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968:/var/log/ceph:z -v /var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmpxsmwkkuh:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:f4452def3be4d350f4a8e5feaf7ebb1511be9298fe68478ba7a2d6fbd7d69a63 inventory --format=json-pretty --filter-for-batch 2024-04-12T21:28:28.873 INFO:teuthology.orchestra.run.smithi018.stdout:Traceback (most recent call last): 2024-04-12T21:28:28.874 INFO:teuthology.orchestra.run.smithi018.stdout: File "/usr/lib64/python3.9/runpy.py", line 197, in _run_module_as_main 2024-04-12T21:28:28.874 INFO:teuthology.orchestra.run.smithi018.stdout: return _run_code(code, main_globals, None, 2024-04-12T21:28:28.874 INFO:teuthology.orchestra.run.smithi018.stdout: File "/usr/lib64/python3.9/runpy.py", line 87, in _run_code 2024-04-12T21:28:28.874 INFO:teuthology.orchestra.run.smithi018.stdout: exec(code, run_globals) 2024-04-12T21:28:28.874 INFO:teuthology.orchestra.run.smithi018.stdout: File "/var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/cephadm.e44f3ed5e34e4ef52eedef233d1190bff004470d8e8005127b5312bff357fa7f/__main__.py", line 5618, in 2024-04-12T21:28:28.874 INFO:teuthology.orchestra.run.smithi018.stdout: File "/var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/cephadm.e44f3ed5e34e4ef52eedef233d1190bff004470d8e8005127b5312bff357fa7f/__main__.py", line 5606, in main 2024-04-12T21:28:28.874 INFO:teuthology.orchestra.run.smithi018.stdout: File "/var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/cephadm.e44f3ed5e34e4ef52eedef233d1190bff004470d8e8005127b5312bff357fa7f/__main__.py", line 408, in _infer_config 2024-04-12T21:28:28.874 INFO:teuthology.orchestra.run.smithi018.stdout: File "/var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/cephadm.e44f3ed5e34e4ef52eedef233d1190bff004470d8e8005127b5312bff357fa7f/__main__.py", line 323, in _infer_fsid 2024-04-12T21:28:28.874 INFO:teuthology.orchestra.run.smithi018.stdout: File "/var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/cephadm.e44f3ed5e34e4ef52eedef233d1190bff004470d8e8005127b5312bff357fa7f/__main__.py", line 436, in _infer_image 2024-04-12T21:28:28.874 INFO:teuthology.orchestra.run.smithi018.stdout: File "/var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/cephadm.e44f3ed5e34e4ef52eedef233d1190bff004470d8e8005127b5312bff357fa7f/__main__.py", line 310, in _validate_fsid 2024-04-12T21:28:28.874 INFO:teuthology.orchestra.run.smithi018.stdout: File "/var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/cephadm.e44f3ed5e34e4ef52eedef233d1190bff004470d8e8005127b5312bff357fa7f/__main__.py", line 3359, in command_ceph_volume 2024-04-12T21:28:28.874 INFO:teuthology.orchestra.run.smithi018.stdout: File "/var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/cephadm.e44f3ed5e34e4ef52eedef233d1190bff004470d8e8005127b5312bff357fa7f/cephadmlib/call_wrappers.py", line 310, in call_throws 2024-04-12T21:28:28.874 INFO:teuthology.orchestra.run.smithi018.stdout:RuntimeError: Failed command: /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:f4452def3be4d350f4a8e5feaf7ebb1511be9298fe68478ba7a2d6fbd7d69a63 -e NODE_NAME=smithi018 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968:/var/run/ceph:z -v /var/log/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968:/var/log/ceph:z -v /var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmpxsmwkkuh:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:f4452def3be4d350f4a8e5feaf7ebb1511be9298fe68478ba7a2d6fbd7d69a63 inventory --format=json-pretty --filter-for-batch 2024-04-12T21:28:28.874 INFO:teuthology.orchestra.run.smithi018.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-04-12T21:28:28.875 INFO:teuthology.orchestra.run.smithi018.stdout: fs cephfs is degraded 2024-04-12T21:28:28.875 INFO:teuthology.orchestra.run.smithi018.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-04-12T21:28:28.875 INFO:teuthology.orchestra.run.smithi018.stdout: fs cephfs is offline because no MDS is active for it. 2024-04-12T21:28:28.875 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:28:28.878+0000 7fb2a1c00700 1 -- 172.21.15.18:0/1629079200 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fb28006ec00 msgr2=0x7fb2800710b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:28:28.875 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:28:28.878+0000 7fb2a1c00700 1 --2- 172.21.15.18:0/1629079200 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fb28006ec00 0x7fb2800710b0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7fb28c004d60 tx=0x7fb28c00b040 comp rx=0 tx=0).stop 2024-04-12T21:28:28.875 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:28:28.878+0000 7fb2a1c00700 1 -- 172.21.15.18:0/1629079200 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb29c072e00 msgr2=0x7fb29c11a450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:28:28.875 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:28:28.878+0000 7fb2a1c00700 1 --2- 172.21.15.18:0/1629079200 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb29c072e00 0x7fb29c11a450 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7fb288004890 tx=0x7fb288009e30 comp rx=0 tx=0).stop 2024-04-12T21:28:28.876 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:28:28.878+0000 7fb2a1c00700 1 -- 172.21.15.18:0/1629079200 shutdown_connections 2024-04-12T21:28:28.876 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:28:28.878+0000 7fb2a1c00700 1 --2- 172.21.15.18:0/1629079200 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fb28006ec00 0x7fb2800710b0 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:28:28.876 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:28:28.878+0000 7fb2a1c00700 1 --2- 172.21.15.18:0/1629079200 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb29c0724f0 0x7fb29c119f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:28:28.876 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:28:28.878+0000 7fb2a1c00700 1 --2- 172.21.15.18:0/1629079200 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb29c072e00 0x7fb29c11a450 unknown :-1 s=CLOSED pgs=736 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:28:28.876 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:28:28.878+0000 7fb2a1c00700 1 -- 172.21.15.18:0/1629079200 >> 172.21.15.18:0/1629079200 conn(0x7fb29c0faba0 msgr2=0x7fb29c107720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:28:28.876 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:28:28.879+0000 7fb2a1c00700 1 -- 172.21.15.18:0/1629079200 shutdown_connections 2024-04-12T21:28:28.876 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:28:28.879+0000 7fb2a1c00700 1 -- 172.21.15.18:0/1629079200 wait complete. 2024-04-12T21:28:29.280 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:28:28 smithi186 ceph-mon[86379]: from='client.35532 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:28:29.280 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:28:28 smithi186 ceph-mon[86379]: from='client.? 172.21.15.18:0/1629079200' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-12T21:28:29.351 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:28:28 smithi018 ceph-mon[112590]: from='client.35532 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:28:29.352 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:28:28 smithi018 ceph-mon[112590]: from='client.? 172.21.15.18:0/1629079200' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-12T21:28:30.280 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:28:29 smithi186 ceph-mon[86379]: pgmap v862: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:28:30.351 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:28:29 smithi018 ceph-mon[112590]: pgmap v862: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:28:33.030 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:28:32 smithi186 ceph-mon[86379]: pgmap v863: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:28:33.101 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:28:32 smithi018 ceph-mon[112590]: pgmap v863: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:28:35.031 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:28:34 smithi186 ceph-mon[86379]: pgmap v864: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:28:35.101 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:28:34 smithi018 ceph-mon[112590]: pgmap v864: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:28:37.030 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:28:36 smithi186 ceph-mon[86379]: pgmap v865: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:28:37.101 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:28:36 smithi018 ceph-mon[112590]: pgmap v865: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:28:39.030 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:28:38 smithi186 ceph-mon[86379]: pgmap v866: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:28:39.101 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:28:38 smithi018 ceph-mon[112590]: pgmap v866: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:28:41.031 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:28:40 smithi186 ceph-mon[86379]: pgmap v867: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:28:41.031 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:28:40 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T21:28:41.101 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:28:40 smithi018 ceph-mon[112590]: pgmap v867: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:28:41.102 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:28:40 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T21:28:42.904 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:28:42 smithi018 ceph-mon[112590]: pgmap v868: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:28:43.030 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:28:42 smithi186 ceph-mon[86379]: pgmap v868: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:28:45.030 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:28:44 smithi186 ceph-mon[86379]: pgmap v869: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:28:45.102 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:28:44 smithi018 ceph-mon[112590]: pgmap v869: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:28:47.030 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:28:46 smithi186 ceph-mon[86379]: pgmap v870: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:28:47.101 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:28:46 smithi018 ceph-mon[112590]: pgmap v870: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:28:49.030 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:28:48 smithi186 ceph-mon[86379]: pgmap v871: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:28:49.102 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:28:48 smithi018 ceph-mon[112590]: pgmap v871: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:28:51.030 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:28:50 smithi186 ceph-mon[86379]: pgmap v872: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:28:51.102 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:28:50 smithi018 ceph-mon[112590]: pgmap v872: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:28:53.031 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:28:52 smithi186 ceph-mon[86379]: pgmap v873: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:28:53.102 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:28:52 smithi018 ceph-mon[112590]: pgmap v873: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:28:55.030 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:28:54 smithi186 ceph-mon[86379]: pgmap v874: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:28:55.102 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:28:54 smithi018 ceph-mon[112590]: pgmap v874: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:28:56.030 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:28:55 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T21:28:56.101 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:28:55 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T21:28:57.030 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:28:56 smithi186 ceph-mon[86379]: pgmap v875: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:28:57.102 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:28:56 smithi018 ceph-mon[112590]: pgmap v875: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:28:59.031 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:28:58 smithi186 ceph-mon[86379]: pgmap v876: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:28:59.048 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:28:59.050+0000 7f059aa00700 1 -- 172.21.15.18:0/3677130465 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0594102c30 msgr2=0x7f0594103000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:28:59.048 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:28:59.050+0000 7f059aa00700 1 --2- 172.21.15.18:0/3677130465 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0594102c30 0x7f0594103000 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7f0588004940 tx=0x7f0588009e30 comp rx=0 tx=0).stop 2024-04-12T21:28:59.048 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:28:58 smithi018 ceph-mon[112590]: pgmap v876: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:28:59.049 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:28:59.050+0000 7f059aa00700 1 -- 172.21.15.18:0/3677130465 shutdown_connections 2024-04-12T21:28:59.049 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:28:59.050+0000 7f059aa00700 1 --2- 172.21.15.18:0/3677130465 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f05941035d0 0x7f05940fcc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:28:59.049 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:28:59.050+0000 7f059aa00700 1 --2- 172.21.15.18:0/3677130465 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0594102c30 0x7f0594103000 unknown :-1 s=CLOSED pgs=737 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:28:59.050 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:28:59.050+0000 7f059aa00700 1 -- 172.21.15.18:0/3677130465 >> 172.21.15.18:0/3677130465 conn(0x7f05940f8760 msgr2=0x7f05940fab70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:28:59.050 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:28:59.050+0000 7f059aa00700 1 -- 172.21.15.18:0/3677130465 shutdown_connections 2024-04-12T21:28:59.050 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:28:59.050+0000 7f059aa00700 1 -- 172.21.15.18:0/3677130465 wait complete. 2024-04-12T21:28:59.050 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:28:59.051+0000 7f059aa00700 1 Processor -- start 2024-04-12T21:28:59.050 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:28:59.051+0000 7f059aa00700 1 -- start start 2024-04-12T21:28:59.051 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:28:59.052+0000 7f059aa00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0594102c30 0x7f0594115c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:28:59.051 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:28:59.052+0000 7f059aa00700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f05941035d0 0x7f0594116140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:28:59.051 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:28:59.052+0000 7f059aa00700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0594116790 con 0x7f0594102c30 2024-04-12T21:28:59.051 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:28:59.052+0000 7f059aa00700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f059410fb80 con 0x7f05941035d0 2024-04-12T21:28:59.052 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:28:59.052+0000 7f0599600700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0594102c30 0x7f0594115c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:28:59.052 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:28:59.052+0000 7f0599600700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0594102c30 0x7f0594115c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:37864/0 (socket says 172.21.15.18:37864) 2024-04-12T21:28:59.052 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:28:59.052+0000 7f0599600700 1 -- 172.21.15.18:0/3109210173 learned_addr learned my addr 172.21.15.18:0/3109210173 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:28:59.052 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:28:59.052+0000 7f0599600700 1 -- 172.21.15.18:0/3109210173 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f05941035d0 msgr2=0x7f0594116140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:28:59.052 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:28:59.052+0000 7f0599600700 1 --2- 172.21.15.18:0/3109210173 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f05941035d0 0x7f0594116140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:28:59.052 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:28:59.052+0000 7f0599600700 1 -- 172.21.15.18:0/3109210173 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0588005040 con 0x7f0594102c30 2024-04-12T21:28:59.052 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:28:59.053+0000 7f0599600700 1 --2- 172.21.15.18:0/3109210173 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0594102c30 0x7f0594115c00 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7f0588009150 tx=0x7f0588009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:28:59.052 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:28:59.053+0000 7f0591600700 1 -- 172.21.15.18:0/3109210173 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f058800e040 con 0x7f0594102c30 2024-04-12T21:28:59.052 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:28:59.053+0000 7f0591600700 1 -- 172.21.15.18:0/3109210173 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f0588013420 con 0x7f0594102c30 2024-04-12T21:28:59.052 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:28:59.053+0000 7f059aa00700 1 -- 172.21.15.18:0/3109210173 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f059410fe00 con 0x7f0594102c30 2024-04-12T21:28:59.052 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:28:59.053+0000 7f0591600700 1 -- 172.21.15.18:0/3109210173 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f058801c630 con 0x7f0594102c30 2024-04-12T21:28:59.052 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:28:59.053+0000 7f059aa00700 1 -- 172.21.15.18:0/3109210173 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f05941103c0 con 0x7f0594102c30 2024-04-12T21:28:59.053 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:28:59.054+0000 7f059aa00700 1 -- 172.21.15.18:0/3109210173 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f05940fe380 con 0x7f0594102c30 2024-04-12T21:28:59.053 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:28:59.055+0000 7f0591600700 1 -- 172.21.15.18:0/3109210173 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7f0588013590 con 0x7f0594102c30 2024-04-12T21:28:59.054 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:28:59.056+0000 7f0591600700 1 --2- 172.21.15.18:0/3109210173 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f057c06ec00 0x7f057c0710b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:28:59.055 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:28:59.056+0000 7f0591600700 1 -- 172.21.15.18:0/3109210173 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 6757+0+0 (secure 0 0 0) 0x7f058808fce0 con 0x7f0594102c30 2024-04-12T21:28:59.055 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:28:59.057+0000 7f0598c00700 1 --2- 172.21.15.18:0/3109210173 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f057c06ec00 0x7f057c0710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:28:59.055 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:28:59.057+0000 7f0598c00700 1 --2- 172.21.15.18:0/3109210173 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f057c06ec00 0x7f057c0710b0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f0594111430 tx=0x7f0580007d40 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:28:59.060 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:28:59.062+0000 7f0591600700 1 -- 172.21.15.18:0/3109210173 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f058805c490 con 0x7f0594102c30 2024-04-12T21:28:59.313 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:28:59.315+0000 7f059aa00700 1 -- 172.21.15.18:0/3109210173 --> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f0594109d90 con 0x7f057c06ec00 2024-04-12T21:28:59.315 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:28:59.317+0000 7f0591600700 1 -- 172.21.15.18:0/3109210173 <== mgr.34102 v2:172.21.15.18:6800/3812891919 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f0594109d90 con 0x7f057c06ec00 2024-04-12T21:28:59.318 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:28:59.320+0000 7f059aa00700 1 -- 172.21.15.18:0/3109210173 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f057c06ec00 msgr2=0x7f057c0710b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:28:59.318 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:28:59.320+0000 7f059aa00700 1 --2- 172.21.15.18:0/3109210173 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f057c06ec00 0x7f057c0710b0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f0594111430 tx=0x7f0580007d40 comp rx=0 tx=0).stop 2024-04-12T21:28:59.318 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:28:59.320+0000 7f059aa00700 1 -- 172.21.15.18:0/3109210173 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0594102c30 msgr2=0x7f0594115c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:28:59.319 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:28:59.320+0000 7f059aa00700 1 --2- 172.21.15.18:0/3109210173 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0594102c30 0x7f0594115c00 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7f0588009150 tx=0x7f0588009e30 comp rx=0 tx=0).stop 2024-04-12T21:28:59.319 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:28:59.320+0000 7f059aa00700 1 -- 172.21.15.18:0/3109210173 shutdown_connections 2024-04-12T21:28:59.319 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:28:59.320+0000 7f059aa00700 1 --2- 172.21.15.18:0/3109210173 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f057c06ec00 0x7f057c0710b0 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:28:59.319 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:28:59.320+0000 7f059aa00700 1 --2- 172.21.15.18:0/3109210173 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0594102c30 0x7f0594115c00 unknown :-1 s=CLOSED pgs=738 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:28:59.320 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:28:59.320+0000 7f059aa00700 1 --2- 172.21.15.18:0/3109210173 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f05941035d0 0x7f0594116140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:28:59.320 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:28:59.321+0000 7f059aa00700 1 -- 172.21.15.18:0/3109210173 >> 172.21.15.18:0/3109210173 conn(0x7f05940f8760 msgr2=0x7f0594071cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:28:59.320 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:28:59.321+0000 7f059aa00700 1 -- 172.21.15.18:0/3109210173 shutdown_connections 2024-04-12T21:28:59.320 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:28:59.321+0000 7f059aa00700 1 -- 172.21.15.18:0/3109210173 wait complete. 2024-04-12T21:28:59.336 INFO:teuthology.orchestra.run.smithi018.stdout:true 2024-04-12T21:28:59.479 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:28:59.481+0000 7fe1d0e00700 1 -- 172.21.15.18:0/1266918946 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe1cc1037c0 msgr2=0x7fe1cc0fce30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:28:59.480 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:28:59.481+0000 7fe1d0e00700 1 --2- 172.21.15.18:0/1266918946 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe1cc1037c0 0x7fe1cc0fce30 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7fe1b8004940 tx=0x7fe1b8009e30 comp rx=0 tx=0).stop 2024-04-12T21:28:59.480 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:28:59.481+0000 7fe1d0e00700 1 -- 172.21.15.18:0/1266918946 shutdown_connections 2024-04-12T21:28:59.480 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:28:59.481+0000 7fe1d0e00700 1 --2- 172.21.15.18:0/1266918946 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe1cc1037c0 0x7fe1cc0fce30 unknown :-1 s=CLOSED pgs=739 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:28:59.480 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:28:59.481+0000 7fe1d0e00700 1 --2- 172.21.15.18:0/1266918946 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe1cc102e20 0x7fe1cc1031f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:28:59.480 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:28:59.481+0000 7fe1d0e00700 1 -- 172.21.15.18:0/1266918946 >> 172.21.15.18:0/1266918946 conn(0x7fe1cc0f8970 msgr2=0x7fe1cc0fad80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:28:59.480 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:28:59.481+0000 7fe1d0e00700 1 -- 172.21.15.18:0/1266918946 shutdown_connections 2024-04-12T21:28:59.480 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:28:59.481+0000 7fe1d0e00700 1 -- 172.21.15.18:0/1266918946 wait complete. 2024-04-12T21:28:59.481 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:28:59.482+0000 7fe1d0e00700 1 Processor -- start 2024-04-12T21:28:59.481 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:28:59.482+0000 7fe1d0e00700 1 -- start start 2024-04-12T21:28:59.481 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:28:59.483+0000 7fe1d0e00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe1cc102e20 0x7fe1cc10b910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:28:59.481 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:28:59.483+0000 7fe1d0e00700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe1cc1037c0 0x7fe1cc10be50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:28:59.481 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:28:59.483+0000 7fe1d0e00700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1cc10c4a0 con 0x7fe1cc102e20 2024-04-12T21:28:59.481 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:28:59.483+0000 7fe1d0e00700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1cc10c5e0 con 0x7fe1cc1037c0 2024-04-12T21:28:59.481 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:28:59.483+0000 7fe1c9e00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe1cc102e20 0x7fe1cc10b910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:28:59.482 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:28:59.483+0000 7fe1c9e00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe1cc102e20 0x7fe1cc10b910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:37888/0 (socket says 172.21.15.18:37888) 2024-04-12T21:28:59.482 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:28:59.483+0000 7fe1c9e00700 1 -- 172.21.15.18:0/3884470209 learned_addr learned my addr 172.21.15.18:0/3884470209 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:28:59.483 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:28:59.483+0000 7fe1c9e00700 1 -- 172.21.15.18:0/3884470209 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe1cc1037c0 msgr2=0x7fe1cc10be50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:28:59.483 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:28:59.483+0000 7fe1c9e00700 1 --2- 172.21.15.18:0/3884470209 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe1cc1037c0 0x7fe1cc10be50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:28:59.483 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:28:59.483+0000 7fe1c9e00700 1 -- 172.21.15.18:0/3884470209 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1b8005040 con 0x7fe1cc102e20 2024-04-12T21:28:59.483 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:28:59.484+0000 7fe1c9e00700 1 --2- 172.21.15.18:0/3884470209 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe1cc102e20 0x7fe1cc10b910 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7fe1bc002a50 tx=0x7fe1bc0046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:28:59.483 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:28:59.484+0000 7fe1c2000700 1 -- 172.21.15.18:0/3884470209 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe1bc007cc0 con 0x7fe1cc102e20 2024-04-12T21:28:59.483 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:28:59.484+0000 7fe1c2000700 1 -- 172.21.15.18:0/3884470209 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe1bc00f040 con 0x7fe1cc102e20 2024-04-12T21:28:59.483 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:28:59.484+0000 7fe1d0e00700 1 -- 172.21.15.18:0/3884470209 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe1cc11e270 con 0x7fe1cc102e20 2024-04-12T21:28:59.484 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:28:59.484+0000 7fe1c2000700 1 -- 172.21.15.18:0/3884470209 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe1bc009640 con 0x7fe1cc102e20 2024-04-12T21:28:59.484 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:28:59.484+0000 7fe1d0e00700 1 -- 172.21.15.18:0/3884470209 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe1cc11e750 con 0x7fe1cc102e20 2024-04-12T21:28:59.484 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:28:59.485+0000 7fe1d0e00700 1 -- 172.21.15.18:0/3884470209 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe1cc102140 con 0x7fe1cc102e20 2024-04-12T21:28:59.485 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:28:59.487+0000 7fe1c2000700 1 -- 172.21.15.18:0/3884470209 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7fe1bc0097a0 con 0x7fe1cc102e20 2024-04-12T21:28:59.486 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:28:59.488+0000 7fe1c2000700 1 --2- 172.21.15.18:0/3884470209 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fe1b006ec00 0x7fe1b00710b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:28:59.486 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:28:59.488+0000 7fe1c2000700 1 -- 172.21.15.18:0/3884470209 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 6757+0+0 (secure 0 0 0) 0x7fe1bc08e810 con 0x7fe1cc102e20 2024-04-12T21:28:59.486 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:28:59.488+0000 7fe1c9400700 1 --2- 172.21.15.18:0/3884470209 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fe1b006ec00 0x7fe1b00710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:28:59.487 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:28:59.489+0000 7fe1c9400700 1 --2- 172.21.15.18:0/3884470209 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fe1b006ec00 0x7fe1b00710b0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7fe1b8004f90 tx=0x7fe1b8009e30 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:28:59.491 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:28:59.493+0000 7fe1c2000700 1 -- 172.21.15.18:0/3884470209 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7fe1bc057410 con 0x7fe1cc102e20 2024-04-12T21:28:59.745 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:28:59.747+0000 7fe1d0e00700 1 -- 172.21.15.18:0/3884470209 --> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fe1cc109f00 con 0x7fe1b006ec00 2024-04-12T21:28:59.748 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:28:59.750+0000 7fe1c2000700 1 -- 172.21.15.18:0/3884470209 <== mgr.34102 v2:172.21.15.18:6800/3812891919 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7fe1cc109f00 con 0x7fe1b006ec00 2024-04-12T21:28:59.751 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:28:59.753+0000 7fe1d0e00700 1 -- 172.21.15.18:0/3884470209 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fe1b006ec00 msgr2=0x7fe1b00710b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:28:59.752 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:28:59.753+0000 7fe1d0e00700 1 --2- 172.21.15.18:0/3884470209 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fe1b006ec00 0x7fe1b00710b0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7fe1b8004f90 tx=0x7fe1b8009e30 comp rx=0 tx=0).stop 2024-04-12T21:28:59.752 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:28:59.753+0000 7fe1d0e00700 1 -- 172.21.15.18:0/3884470209 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe1cc102e20 msgr2=0x7fe1cc10b910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:28:59.752 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:28:59.753+0000 7fe1d0e00700 1 --2- 172.21.15.18:0/3884470209 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe1cc102e20 0x7fe1cc10b910 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7fe1bc002a50 tx=0x7fe1bc0046b0 comp rx=0 tx=0).stop 2024-04-12T21:28:59.752 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:28:59.753+0000 7fe1d0e00700 1 -- 172.21.15.18:0/3884470209 shutdown_connections 2024-04-12T21:28:59.752 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:28:59.753+0000 7fe1d0e00700 1 --2- 172.21.15.18:0/3884470209 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fe1b006ec00 0x7fe1b00710b0 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:28:59.752 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:28:59.753+0000 7fe1d0e00700 1 --2- 172.21.15.18:0/3884470209 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe1cc102e20 0x7fe1cc10b910 unknown :-1 s=CLOSED pgs=740 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:28:59.752 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:28:59.753+0000 7fe1d0e00700 1 --2- 172.21.15.18:0/3884470209 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe1cc1037c0 0x7fe1cc10be50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:28:59.752 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:28:59.753+0000 7fe1d0e00700 1 -- 172.21.15.18:0/3884470209 >> 172.21.15.18:0/3884470209 conn(0x7fe1cc0f8970 msgr2=0x7fe1cc070b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:28:59.752 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:28:59.753+0000 7fe1d0e00700 1 -- 172.21.15.18:0/3884470209 shutdown_connections 2024-04-12T21:28:59.752 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:28:59.753+0000 7fe1d0e00700 1 -- 172.21.15.18:0/3884470209 wait complete. 2024-04-12T21:28:59.910 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:28:59.911+0000 7f1042000700 1 -- 172.21.15.18:0/4193996479 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f103c1059a0 msgr2=0x7f103c0ff010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:28:59.910 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:28:59.911+0000 7f1042000700 1 --2- 172.21.15.18:0/4193996479 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f103c1059a0 0x7f103c0ff010 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7f1028004880 tx=0x7f1028009e30 comp rx=0 tx=0).stop 2024-04-12T21:28:59.910 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:28:59.911+0000 7f1042000700 1 -- 172.21.15.18:0/4193996479 shutdown_connections 2024-04-12T21:28:59.910 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:28:59.911+0000 7f1042000700 1 --2- 172.21.15.18:0/4193996479 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f103c1059a0 0x7f103c0ff010 unknown :-1 s=CLOSED pgs=741 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:28:59.910 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:28:59.911+0000 7f1042000700 1 --2- 172.21.15.18:0/4193996479 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f103c105000 0x7f103c1053d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:28:59.911 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:28:59.911+0000 7f1042000700 1 -- 172.21.15.18:0/4193996479 >> 172.21.15.18:0/4193996479 conn(0x7f103c0fab70 msgr2=0x7f103c0fcf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:28:59.911 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:28:59.912+0000 7f1042000700 1 -- 172.21.15.18:0/4193996479 shutdown_connections 2024-04-12T21:28:59.911 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:28:59.912+0000 7f1042000700 1 -- 172.21.15.18:0/4193996479 wait complete. 2024-04-12T21:28:59.912 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:28:59.912+0000 7f1042000700 1 Processor -- start 2024-04-12T21:28:59.912 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:28:59.913+0000 7f1042000700 1 -- start start 2024-04-12T21:28:59.912 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:28:59.913+0000 7f1042000700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f103c105000 0x7f103c115bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:28:59.912 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:28:59.913+0000 7f1042000700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f103c1059a0 0x7f103c116110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:28:59.912 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:28:59.913+0000 7f1042000700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f103c116760 con 0x7f103c1059a0 2024-04-12T21:28:59.912 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:28:59.913+0000 7f1042000700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f103c10fd40 con 0x7f103c105000 2024-04-12T21:28:59.912 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:28:59.913+0000 7f103aa00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f103c1059a0 0x7f103c116110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:28:59.912 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:28:59.914+0000 7f103aa00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f103c1059a0 0x7f103c116110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:37904/0 (socket says 172.21.15.18:37904) 2024-04-12T21:28:59.912 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:28:59.914+0000 7f103aa00700 1 -- 172.21.15.18:0/999761178 learned_addr learned my addr 172.21.15.18:0/999761178 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:28:59.913 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:28:59.914+0000 7f103b400700 1 --2- 172.21.15.18:0/999761178 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f103c105000 0x7f103c115bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:28:59.914 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:28:59.914+0000 7f103aa00700 1 -- 172.21.15.18:0/999761178 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f103c105000 msgr2=0x7f103c115bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:28:59.914 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:28:59.914+0000 7f103aa00700 1 --2- 172.21.15.18:0/999761178 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f103c105000 0x7f103c115bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:28:59.914 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:28:59.914+0000 7f103aa00700 1 -- 172.21.15.18:0/999761178 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1028005040 con 0x7f103c1059a0 2024-04-12T21:28:59.914 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:28:59.914+0000 7f103b400700 1 --2- 172.21.15.18:0/999761178 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f103c105000 0x7f103c115bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-12T21:28:59.914 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:28:59.914+0000 7f103aa00700 1 --2- 172.21.15.18:0/999761178 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f103c1059a0 0x7f103c116110 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7f10280041d0 tx=0x7f1028009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:28:59.914 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:28:59.915+0000 7f1033400700 1 -- 172.21.15.18:0/999761178 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f102800e070 con 0x7f103c1059a0 2024-04-12T21:28:59.914 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:28:59.915+0000 7f1042000700 1 -- 172.21.15.18:0/999761178 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f103c10ffc0 con 0x7f103c1059a0 2024-04-12T21:28:59.914 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:28:59.915+0000 7f1033400700 1 -- 172.21.15.18:0/999761178 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f1028013d60 con 0x7f103c1059a0 2024-04-12T21:28:59.914 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:28:59.915+0000 7f1033400700 1 -- 172.21.15.18:0/999761178 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f102801cda0 con 0x7f103c1059a0 2024-04-12T21:28:59.914 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:28:59.915+0000 7f1042000700 1 -- 172.21.15.18:0/999761178 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f103c110520 con 0x7f103c1059a0 2024-04-12T21:28:59.915 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:28:59.916+0000 7f1042000700 1 -- 172.21.15.18:0/999761178 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f103c100750 con 0x7f103c1059a0 2024-04-12T21:28:59.915 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:28:59.917+0000 7f1033400700 1 -- 172.21.15.18:0/999761178 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7f102801c590 con 0x7f103c1059a0 2024-04-12T21:28:59.916 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:28:59.918+0000 7f1033400700 1 --2- 172.21.15.18:0/999761178 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f102006ec00 0x7f10200710b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:28:59.916 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:28:59.918+0000 7f1033400700 1 -- 172.21.15.18:0/999761178 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 6757+0+0 (secure 0 0 0) 0x7f102808fe80 con 0x7f103c1059a0 2024-04-12T21:28:59.916 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:28:59.918+0000 7f103b400700 1 --2- 172.21.15.18:0/999761178 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f102006ec00 0x7f10200710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:28:59.917 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:28:59.919+0000 7f103b400700 1 --2- 172.21.15.18:0/999761178 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f102006ec00 0x7f10200710b0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f102c004a60 tx=0x7f102c00b040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:28:59.921 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:28:59.923+0000 7f1033400700 1 -- 172.21.15.18:0/999761178 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f102805c6b0 con 0x7f103c1059a0 2024-04-12T21:29:00.154 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:29:00.155+0000 7f1042000700 1 -- 172.21.15.18:0/999761178 --> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f103c10c150 con 0x7f102006ec00 2024-04-12T21:29:00.170 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:29:00.172+0000 7f1033400700 1 -- 172.21.15.18:0/999761178 <== mgr.34102 v2:172.21.15.18:6800/3812891919 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7f103c10c150 con 0x7f102006ec00 2024-04-12T21:29:00.171 INFO:teuthology.orchestra.run.smithi018.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-12T21:29:00.171 INFO:teuthology.orchestra.run.smithi018.stdout:alertmanager.smithi018 smithi018 *:9093,9094 running (29m) 5m ago 44m 15.5M - 0.25.0 c8568f914cd2 c332090068e8 2024-04-12T21:29:00.171 INFO:teuthology.orchestra.run.smithi018.stdout:crash.smithi018 smithi018 running (25m) 5m ago 44m 6665k - 19.0.0-2879-g324baf8d 8717720dc5e5 b7f80d1f7982 2024-04-12T21:29:00.171 INFO:teuthology.orchestra.run.smithi018.stdout:crash.smithi186 smithi186 running (25m) 5m ago 42m 6660k - 19.0.0-2879-g324baf8d 8717720dc5e5 a4f55c81b623 2024-04-12T21:29:00.171 INFO:teuthology.orchestra.run.smithi018.stdout:grafana.smithi018 smithi018 *:3000 running (29m) 5m ago 43m 74.8M - 9.4.12 2bacad6d85d8 b08e3bf176dd 2024-04-12T21:29:00.171 INFO:teuthology.orchestra.run.smithi018.stdout:mds.cephfs.smithi018.acqbsc smithi018 running (21m) 5m ago 36m 16.7M - 19.0.0-2879-g324baf8d 8717720dc5e5 021f99ab24ae 2024-04-12T21:29:00.171 INFO:teuthology.orchestra.run.smithi018.stdout:mds.cephfs.smithi018.szjluo smithi018 running (21m) 5m ago 36m 16.5M - 19.0.0-2879-g324baf8d 8717720dc5e5 0003117479d4 2024-04-12T21:29:00.171 INFO:teuthology.orchestra.run.smithi018.stdout:mds.cephfs.smithi186.jdllyn smithi186 running (6m) 5m ago 36m 11.0M - 19.0.0-2879-g324baf8d 8717720dc5e5 21b1f46eef23 2024-04-12T21:29:00.171 INFO:teuthology.orchestra.run.smithi018.stdout:mds.cephfs.smithi186.wjnafs smithi186 running (36m) 5m ago 36m 19.5M - 17.2.7-827-gd3b5060a 21de9b7369ad 74b115d6d6fd 2024-04-12T21:29:00.172 INFO:teuthology.orchestra.run.smithi018.stdout:mgr.smithi018.xjzwga smithi018 *:8443,9283,8765 running (32m) 5m ago 45m 512M - 19.0.0-2879-g324baf8d 8717720dc5e5 56088305eab7 2024-04-12T21:29:00.172 INFO:teuthology.orchestra.run.smithi018.stdout:mgr.smithi186.kqcfxk smithi186 *:8443,9283,8765 running (30m) 5m ago 42m 459M - 19.0.0-2879-g324baf8d 8717720dc5e5 09cd766e5e0d 2024-04-12T21:29:00.172 INFO:teuthology.orchestra.run.smithi018.stdout:mon.smithi018 smithi018 running (27m) 5m ago 45m 73.1M 2048M 19.0.0-2879-g324baf8d 8717720dc5e5 df3e201dfed6 2024-04-12T21:29:00.172 INFO:teuthology.orchestra.run.smithi018.stdout:mon.smithi186 smithi186 running (26m) 5m ago 42m 47.2M 2048M 19.0.0-2879-g324baf8d 8717720dc5e5 c77498da9b2b 2024-04-12T21:29:00.172 INFO:teuthology.orchestra.run.smithi018.stdout:node-exporter.smithi018 smithi018 *:9100 running (30m) 5m ago 44m 8108k - 1.5.0 0da6a335fe13 326c3d9c6847 2024-04-12T21:29:00.172 INFO:teuthology.orchestra.run.smithi018.stdout:node-exporter.smithi186 smithi186 *:9100 running (30m) 5m ago 42m 8757k - 1.5.0 0da6a335fe13 ec2112baeecf 2024-04-12T21:29:00.172 INFO:teuthology.orchestra.run.smithi018.stdout:osd.0 smithi018 running (25m) 5m ago 41m 123M 1559M 19.0.0-2879-g324baf8d 8717720dc5e5 60b1273ca047 2024-04-12T21:29:00.172 INFO:teuthology.orchestra.run.smithi018.stdout:osd.1 smithi018 running (24m) 5m ago 41m 95.1M 1559M 19.0.0-2879-g324baf8d 8717720dc5e5 170ce1e5fb8d 2024-04-12T21:29:00.172 INFO:teuthology.orchestra.run.smithi018.stdout:osd.2 smithi018 running (24m) 5m ago 40m 94.3M 1559M 19.0.0-2879-g324baf8d 8717720dc5e5 c2cbc6aa2839 2024-04-12T21:29:00.172 INFO:teuthology.orchestra.run.smithi018.stdout:osd.3 smithi186 running (23m) 5m ago 39m 101M 2581M 19.0.0-2879-g324baf8d 8717720dc5e5 0138e8262e4b 2024-04-12T21:29:00.172 INFO:teuthology.orchestra.run.smithi018.stdout:osd.4 smithi186 running (22m) 5m ago 39m 91.0M 2581M 19.0.0-2879-g324baf8d 8717720dc5e5 b20cb63b25c9 2024-04-12T21:29:00.172 INFO:teuthology.orchestra.run.smithi018.stdout:osd.5 smithi186 running (22m) 5m ago 38m 92.0M 2581M 19.0.0-2879-g324baf8d 8717720dc5e5 2513bea2610e 2024-04-12T21:29:00.172 INFO:teuthology.orchestra.run.smithi018.stdout:prometheus.smithi018 smithi018 *:9095 running (30m) 5m ago 43m 67.9M - 2.43.0 a07b618ecd1d f58841127e21 2024-04-12T21:29:00.173 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:29:00.175+0000 7f1042000700 1 -- 172.21.15.18:0/999761178 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f102006ec00 msgr2=0x7f10200710b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:29:00.174 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:29:00.175+0000 7f1042000700 1 --2- 172.21.15.18:0/999761178 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f102006ec00 0x7f10200710b0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f102c004a60 tx=0x7f102c00b040 comp rx=0 tx=0).stop 2024-04-12T21:29:00.174 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:29:00.175+0000 7f1042000700 1 -- 172.21.15.18:0/999761178 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f103c1059a0 msgr2=0x7f103c116110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:29:00.174 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:29:00.175+0000 7f1042000700 1 --2- 172.21.15.18:0/999761178 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f103c1059a0 0x7f103c116110 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7f10280041d0 tx=0x7f1028009e30 comp rx=0 tx=0).stop 2024-04-12T21:29:00.174 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:29:00.175+0000 7f1042000700 1 -- 172.21.15.18:0/999761178 shutdown_connections 2024-04-12T21:29:00.174 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:29:00.175+0000 7f1042000700 1 --2- 172.21.15.18:0/999761178 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f102006ec00 0x7f10200710b0 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:29:00.174 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:29:00.175+0000 7f1042000700 1 --2- 172.21.15.18:0/999761178 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f103c105000 0x7f103c115bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:29:00.174 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:29:00.175+0000 7f1042000700 1 --2- 172.21.15.18:0/999761178 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f103c1059a0 0x7f103c116110 unknown :-1 s=CLOSED pgs=742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:29:00.174 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:29:00.175+0000 7f1042000700 1 -- 172.21.15.18:0/999761178 >> 172.21.15.18:0/999761178 conn(0x7f103c0fab70 msgr2=0x7f103c0fbb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:29:00.174 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:29:00.175+0000 7f1042000700 1 -- 172.21.15.18:0/999761178 shutdown_connections 2024-04-12T21:29:00.174 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:29:00.175+0000 7f1042000700 1 -- 172.21.15.18:0/999761178 wait complete. 2024-04-12T21:29:00.333 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:29:00.334+0000 7ff5e7c00700 1 -- 172.21.15.18:0/2421113717 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff5e00fde80 msgr2=0x7ff5e00fe250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:29:00.333 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:29:00.334+0000 7ff5e7c00700 1 --2- 172.21.15.18:0/2421113717 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff5e00fde80 0x7ff5e00fe250 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7ff5d4004940 tx=0x7ff5d4009e30 comp rx=0 tx=0).stop 2024-04-12T21:29:00.334 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:29:00.334+0000 7ff5e7c00700 1 -- 172.21.15.18:0/2421113717 shutdown_connections 2024-04-12T21:29:00.334 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:29:00.334+0000 7ff5e7c00700 1 --2- 172.21.15.18:0/2421113717 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff5e00fe820 0x7ff5e0106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:29:00.334 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:29:00.334+0000 7ff5e7c00700 1 --2- 172.21.15.18:0/2421113717 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff5e00fde80 0x7ff5e00fe250 unknown :-1 s=CLOSED pgs=743 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:29:00.334 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:29:00.334+0000 7ff5e7c00700 1 -- 172.21.15.18:0/2421113717 >> 172.21.15.18:0/2421113717 conn(0x7ff5e00f78f0 msgr2=0x7ff5e00f9d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:29:00.334 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:29:00.334+0000 7ff5e7c00700 1 -- 172.21.15.18:0/2421113717 shutdown_connections 2024-04-12T21:29:00.334 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:29:00.334+0000 7ff5e7c00700 1 -- 172.21.15.18:0/2421113717 wait complete. 2024-04-12T21:29:00.334 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:29:00.335+0000 7ff5e7c00700 1 Processor -- start 2024-04-12T21:29:00.336 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:29:00.335+0000 7ff5e7c00700 1 -- start start 2024-04-12T21:29:00.336 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:29:00.336+0000 7ff5e7c00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff5e00fde80 0x7ff5e0115b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:29:00.336 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:29:00.336+0000 7ff5e7c00700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff5e00fe820 0x7ff5e0116090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:29:00.336 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:29:00.336+0000 7ff5e7c00700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5e01166e0 con 0x7ff5e00fde80 2024-04-12T21:29:00.336 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:29:00.336+0000 7ff5e7c00700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5e010fcc0 con 0x7ff5e00fe820 2024-04-12T21:29:00.336 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:29:00.336+0000 7ff5e5200700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff5e00fde80 0x7ff5e0115b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:29:00.336 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:29:00.336+0000 7ff5e5200700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff5e00fde80 0x7ff5e0115b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:37928/0 (socket says 172.21.15.18:37928) 2024-04-12T21:29:00.336 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:29:00.336+0000 7ff5e5200700 1 -- 172.21.15.18:0/1344593175 learned_addr learned my addr 172.21.15.18:0/1344593175 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:29:00.336 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:29:00.336+0000 7ff5dfe00700 1 --2- 172.21.15.18:0/1344593175 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff5e00fe820 0x7ff5e0116090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:29:00.336 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:29:00.337+0000 7ff5e5200700 1 -- 172.21.15.18:0/1344593175 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff5e00fe820 msgr2=0x7ff5e0116090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:29:00.336 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:29:00.337+0000 7ff5e5200700 1 --2- 172.21.15.18:0/1344593175 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff5e00fe820 0x7ff5e0116090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:29:00.336 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:29:00.337+0000 7ff5e5200700 1 -- 172.21.15.18:0/1344593175 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5d4005040 con 0x7ff5e00fde80 2024-04-12T21:29:00.337 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:29:00.337+0000 7ff5e5200700 1 --2- 172.21.15.18:0/1344593175 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff5e00fde80 0x7ff5e0115b50 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7ff5d4004890 tx=0x7ff5d4009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:29:00.337 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:29:00.337+0000 7ff5dcc00700 1 -- 172.21.15.18:0/1344593175 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff5d400e040 con 0x7ff5e00fde80 2024-04-12T21:29:00.337 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:29:00.337+0000 7ff5dcc00700 1 -- 172.21.15.18:0/1344593175 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff5d4009af0 con 0x7ff5e00fde80 2024-04-12T21:29:00.337 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:29:00.337+0000 7ff5e7c00700 1 -- 172.21.15.18:0/1344593175 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff5e010ff40 con 0x7ff5e00fde80 2024-04-12T21:29:00.337 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:29:00.337+0000 7ff5dcc00700 1 -- 172.21.15.18:0/1344593175 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff5d401c630 con 0x7ff5e00fde80 2024-04-12T21:29:00.337 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:29:00.338+0000 7ff5e7c00700 1 -- 172.21.15.18:0/1344593175 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff5e0110480 con 0x7ff5e00fde80 2024-04-12T21:29:00.337 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:29:00.339+0000 7ff5e7c00700 1 -- 172.21.15.18:0/1344593175 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff5e01068d0 con 0x7ff5e00fde80 2024-04-12T21:29:00.338 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:29:00.340+0000 7ff5dcc00700 1 -- 172.21.15.18:0/1344593175 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7ff5d400bbc0 con 0x7ff5e00fde80 2024-04-12T21:29:00.339 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:29:00.341+0000 7ff5dcc00700 1 --2- 172.21.15.18:0/1344593175 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7ff5c806ec00 0x7ff5c80710b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:29:00.339 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:29:00.341+0000 7ff5dcc00700 1 -- 172.21.15.18:0/1344593175 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 6757+0+0 (secure 0 0 0) 0x7ff5d408fd50 con 0x7ff5e00fde80 2024-04-12T21:29:00.339 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:29:00.341+0000 7ff5dfe00700 1 --2- 172.21.15.18:0/1344593175 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7ff5c806ec00 0x7ff5c80710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:29:00.340 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:29:00.342+0000 7ff5dfe00700 1 --2- 172.21.15.18:0/1344593175 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7ff5c806ec00 0x7ff5c80710b0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7ff5cc007fd0 tx=0x7ff5cc009040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:29:00.344 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:29:00.346+0000 7ff5dcc00700 1 -- 172.21.15.18:0/1344593175 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7ff5d405c500 con 0x7ff5e00fde80 2024-04-12T21:29:00.669 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:29:00.671+0000 7ff5e7c00700 1 -- 172.21.15.18:0/1344593175 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7ff5e01068d0 con 0x7ff5e00fde80 2024-04-12T21:29:00.670 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:29:00.672+0000 7ff5dcc00700 1 -- 172.21.15.18:0/1344593175 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+837 (secure 0 0 0) 0x7ff5d405c090 con 0x7ff5e00fde80 2024-04-12T21:29:00.671 INFO:teuthology.orchestra.run.smithi018.stdout:{ 2024-04-12T21:29:00.671 INFO:teuthology.orchestra.run.smithi018.stdout: "mon": { 2024-04-12T21:29:00.671 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 19.0.0-2879-g324baf8d (324baf8ddb91dea46cac50cb535c306115791355) squid (dev)": 2 2024-04-12T21:29:00.671 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-12T21:29:00.671 INFO:teuthology.orchestra.run.smithi018.stdout: "mgr": { 2024-04-12T21:29:00.671 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 19.0.0-2879-g324baf8d (324baf8ddb91dea46cac50cb535c306115791355) squid (dev)": 2 2024-04-12T21:29:00.671 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-12T21:29:00.671 INFO:teuthology.orchestra.run.smithi018.stdout: "osd": { 2024-04-12T21:29:00.671 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 19.0.0-2879-g324baf8d (324baf8ddb91dea46cac50cb535c306115791355) squid (dev)": 6 2024-04-12T21:29:00.671 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-12T21:29:00.671 INFO:teuthology.orchestra.run.smithi018.stdout: "mds": { 2024-04-12T21:29:00.671 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 17.2.7-827-gd3b5060a (d3b5060a17449edfae9385cdf792afec109fca5d) quincy (stable)": 1, 2024-04-12T21:29:00.672 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 19.0.0-2879-g324baf8d (324baf8ddb91dea46cac50cb535c306115791355) squid (dev)": 3 2024-04-12T21:29:00.672 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-12T21:29:00.672 INFO:teuthology.orchestra.run.smithi018.stdout: "overall": { 2024-04-12T21:29:00.672 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 17.2.7-827-gd3b5060a (d3b5060a17449edfae9385cdf792afec109fca5d) quincy (stable)": 1, 2024-04-12T21:29:00.672 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 19.0.0-2879-g324baf8d (324baf8ddb91dea46cac50cb535c306115791355) squid (dev)": 13 2024-04-12T21:29:00.672 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-04-12T21:29:00.672 INFO:teuthology.orchestra.run.smithi018.stdout:} 2024-04-12T21:29:00.673 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:29:00.675+0000 7ff5e7c00700 1 -- 172.21.15.18:0/1344593175 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7ff5c806ec00 msgr2=0x7ff5c80710b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:29:00.673 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:29:00.675+0000 7ff5e7c00700 1 --2- 172.21.15.18:0/1344593175 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7ff5c806ec00 0x7ff5c80710b0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7ff5cc007fd0 tx=0x7ff5cc009040 comp rx=0 tx=0).stop 2024-04-12T21:29:00.673 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:29:00.675+0000 7ff5e7c00700 1 -- 172.21.15.18:0/1344593175 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff5e00fde80 msgr2=0x7ff5e0115b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:29:00.673 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:29:00.675+0000 7ff5e7c00700 1 --2- 172.21.15.18:0/1344593175 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff5e00fde80 0x7ff5e0115b50 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7ff5d4004890 tx=0x7ff5d4009e30 comp rx=0 tx=0).stop 2024-04-12T21:29:00.674 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:29:00.675+0000 7ff5e7c00700 1 -- 172.21.15.18:0/1344593175 shutdown_connections 2024-04-12T21:29:00.674 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:29:00.675+0000 7ff5e7c00700 1 --2- 172.21.15.18:0/1344593175 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7ff5c806ec00 0x7ff5c80710b0 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:29:00.674 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:29:00.675+0000 7ff5e7c00700 1 --2- 172.21.15.18:0/1344593175 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff5e00fde80 0x7ff5e0115b50 unknown :-1 s=CLOSED pgs=744 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:29:00.674 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:29:00.675+0000 7ff5e7c00700 1 --2- 172.21.15.18:0/1344593175 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff5e00fe820 0x7ff5e0116090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:29:00.674 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:29:00.675+0000 7ff5e7c00700 1 -- 172.21.15.18:0/1344593175 >> 172.21.15.18:0/1344593175 conn(0x7ff5e00f78f0 msgr2=0x7ff5e00fd270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:29:00.674 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:29:00.675+0000 7ff5e7c00700 1 -- 172.21.15.18:0/1344593175 shutdown_connections 2024-04-12T21:29:00.674 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:29:00.675+0000 7ff5e7c00700 1 -- 172.21.15.18:0/1344593175 wait complete. 2024-04-12T21:29:00.828 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:29:00.829+0000 7f6df5c00700 1 -- 172.21.15.18:0/3916570867 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6df0102ca0 msgr2=0x7f6df0103070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:29:00.828 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:29:00.829+0000 7f6df5c00700 1 --2- 172.21.15.18:0/3916570867 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6df0102ca0 0x7f6df0103070 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7f6de0004940 tx=0x7f6de0009e30 comp rx=0 tx=0).stop 2024-04-12T21:29:00.828 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:29:00.829+0000 7f6df5c00700 1 -- 172.21.15.18:0/3916570867 shutdown_connections 2024-04-12T21:29:00.828 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:29:00.829+0000 7f6df5c00700 1 --2- 172.21.15.18:0/3916570867 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6df0065630 0x7f6df0065a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:29:00.828 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:29:00.829+0000 7f6df5c00700 1 --2- 172.21.15.18:0/3916570867 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6df0102ca0 0x7f6df0103070 unknown :-1 s=CLOSED pgs=745 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:29:00.828 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:29:00.829+0000 7f6df5c00700 1 -- 172.21.15.18:0/3916570867 >> 172.21.15.18:0/3916570867 conn(0x7f6df0071b70 msgr2=0x7f6df0071f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:29:00.828 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:29:00.829+0000 7f6df5c00700 1 -- 172.21.15.18:0/3916570867 shutdown_connections 2024-04-12T21:29:00.828 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:29:00.829+0000 7f6df5c00700 1 -- 172.21.15.18:0/3916570867 wait complete. 2024-04-12T21:29:00.831 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:29:00.830+0000 7f6df5c00700 1 Processor -- start 2024-04-12T21:29:00.831 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:29:00.830+0000 7f6df5c00700 1 -- start start 2024-04-12T21:29:00.831 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:29:00.830+0000 7f6df5c00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6df0065630 0x7f6df0115b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:29:00.831 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:29:00.830+0000 7f6df5c00700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6df0102ca0 0x7f6df01160d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:29:00.831 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:29:00.831+0000 7f6df5c00700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6df0116720 con 0x7f6df0065630 2024-04-12T21:29:00.831 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:29:00.831+0000 7f6df5c00700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6df010fb60 con 0x7f6df0102ca0 2024-04-12T21:29:00.831 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:29:00.831+0000 7f6defe00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6df0065630 0x7f6df0115b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:29:00.831 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:29:00.831+0000 7f6defe00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6df0065630 0x7f6df0115b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:39848/0 (socket says 172.21.15.18:39848) 2024-04-12T21:29:00.831 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:29:00.831+0000 7f6defe00700 1 -- 172.21.15.18:0/3419652240 learned_addr learned my addr 172.21.15.18:0/3419652240 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:29:00.831 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:29:00.831+0000 7f6defe00700 1 -- 172.21.15.18:0/3419652240 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6df0102ca0 msgr2=0x7f6df01160d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:29:00.832 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:29:00.831+0000 7f6def400700 1 --2- 172.21.15.18:0/3419652240 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6df0102ca0 0x7f6df01160d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:29:00.832 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:29:00.831+0000 7f6defe00700 1 --2- 172.21.15.18:0/3419652240 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6df0102ca0 0x7f6df01160d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:29:00.832 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:29:00.831+0000 7f6defe00700 1 -- 172.21.15.18:0/3419652240 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6de0005040 con 0x7f6df0065630 2024-04-12T21:29:00.832 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:29:00.831+0000 7f6def400700 1 --2- 172.21.15.18:0/3419652240 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6df0102ca0 0x7f6df01160d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-12T21:29:00.832 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:29:00.832+0000 7f6defe00700 1 --2- 172.21.15.18:0/3419652240 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6df0065630 0x7f6df0115b90 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7f6de0009180 tx=0x7f6de0009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:29:00.832 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:29:00.832+0000 7f6de7e00700 1 -- 172.21.15.18:0/3419652240 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6de000e040 con 0x7f6df0065630 2024-04-12T21:29:00.832 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:29:00.832+0000 7f6de7e00700 1 -- 172.21.15.18:0/3419652240 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f6de000bbc0 con 0x7f6df0065630 2024-04-12T21:29:00.832 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:29:00.832+0000 7f6df5c00700 1 -- 172.21.15.18:0/3419652240 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6df010fde0 con 0x7f6df0065630 2024-04-12T21:29:00.832 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:29:00.832+0000 7f6de7e00700 1 -- 172.21.15.18:0/3419652240 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6de001c630 con 0x7f6df0065630 2024-04-12T21:29:00.832 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:29:00.832+0000 7f6df5c00700 1 -- 172.21.15.18:0/3419652240 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6df01103a0 con 0x7f6df0065630 2024-04-12T21:29:00.833 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:29:00.833+0000 7f6df5c00700 1 -- 172.21.15.18:0/3419652240 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6df0106840 con 0x7f6df0065630 2024-04-12T21:29:00.833 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:29:00.834+0000 7f6de7e00700 1 -- 172.21.15.18:0/3419652240 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7f6de000bd30 con 0x7f6df0065630 2024-04-12T21:29:00.833 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:29:00.835+0000 7f6de7e00700 1 --2- 172.21.15.18:0/3419652240 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f6dd406ec10 0x7f6dd40710c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:29:00.834 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:29:00.835+0000 7f6de7e00700 1 -- 172.21.15.18:0/3419652240 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 6757+0+0 (secure 0 0 0) 0x7f6de008fd90 con 0x7f6df0065630 2024-04-12T21:29:00.834 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:29:00.836+0000 7f6def400700 1 --2- 172.21.15.18:0/3419652240 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f6dd406ec10 0x7f6dd40710c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:29:00.834 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:29:00.836+0000 7f6def400700 1 --2- 172.21.15.18:0/3419652240 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f6dd406ec10 0x7f6dd40710c0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f6ddc007fd0 tx=0x7f6ddc009040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:29:00.838 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:29:00.840+0000 7f6de7e00700 1 -- 172.21.15.18:0/3419652240 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f6de005c5c0 con 0x7f6df0065630 2024-04-12T21:29:01.031 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:29:00 smithi186 ceph-mon[86379]: from='client.35540 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:29:01.031 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:29:00 smithi186 ceph-mon[86379]: pgmap v877: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:29:01.031 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:29:00 smithi186 ceph-mon[86379]: from='client.35544 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:29:01.031 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:29:00 smithi186 ceph-mon[86379]: from='client.? 172.21.15.18:0/1344593175' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T21:29:01.102 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:29:00 smithi018 ceph-mon[112590]: from='client.35540 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:29:01.102 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:29:00 smithi018 ceph-mon[112590]: pgmap v877: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:29:01.102 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:29:00 smithi018 ceph-mon[112590]: from='client.35544 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:29:01.102 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:29:00 smithi018 ceph-mon[112590]: from='client.? 172.21.15.18:0/1344593175' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T21:29:01.120 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:29:01.121+0000 7f6df5c00700 1 -- 172.21.15.18:0/3419652240 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f6df01111e0 con 0x7f6df0065630 2024-04-12T21:29:01.121 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:29:01.122+0000 7f6de7e00700 1 -- 172.21.15.18:0/3419652240 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 31 v31) v1 ==== 76+0+1873 (secure 0 0 0) 0x7f6de005c150 con 0x7f6df0065630 2024-04-12T21:29:01.121 INFO:teuthology.orchestra.run.smithi018.stdout:e31 2024-04-12T21:29:01.121 INFO:teuthology.orchestra.run.smithi018.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-12T21:29:01.121 INFO:teuthology.orchestra.run.smithi018.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-12T21:29:01.121 INFO:teuthology.orchestra.run.smithi018.stdout:legacy client fscid: 1 2024-04-12T21:29:01.121 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-12T21:29:01.121 INFO:teuthology.orchestra.run.smithi018.stdout:Filesystem 'cephfs' (1) 2024-04-12T21:29:01.121 INFO:teuthology.orchestra.run.smithi018.stdout:fs_name cephfs 2024-04-12T21:29:01.122 INFO:teuthology.orchestra.run.smithi018.stdout:epoch 24 2024-04-12T21:29:01.122 INFO:teuthology.orchestra.run.smithi018.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-04-12T21:29:01.122 INFO:teuthology.orchestra.run.smithi018.stdout:created 2024-04-12T20:51:57.362850+0000 2024-04-12T21:29:01.122 INFO:teuthology.orchestra.run.smithi018.stdout:modified 2024-04-12T21:06:59.717190+0000 2024-04-12T21:29:01.122 INFO:teuthology.orchestra.run.smithi018.stdout:tableserver 0 2024-04-12T21:29:01.122 INFO:teuthology.orchestra.run.smithi018.stdout:root 0 2024-04-12T21:29:01.122 INFO:teuthology.orchestra.run.smithi018.stdout:session_timeout 60 2024-04-12T21:29:01.122 INFO:teuthology.orchestra.run.smithi018.stdout:session_autoclose 300 2024-04-12T21:29:01.122 INFO:teuthology.orchestra.run.smithi018.stdout:max_file_size 1099511627776 2024-04-12T21:29:01.122 INFO:teuthology.orchestra.run.smithi018.stdout:max_xattr_size 65536 2024-04-12T21:29:01.122 INFO:teuthology.orchestra.run.smithi018.stdout:required_client_features {} 2024-04-12T21:29:01.122 INFO:teuthology.orchestra.run.smithi018.stdout:last_failure 0 2024-04-12T21:29:01.122 INFO:teuthology.orchestra.run.smithi018.stdout:last_failure_osd_epoch 100 2024-04-12T21:29:01.122 INFO:teuthology.orchestra.run.smithi018.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-12T21:29:01.123 INFO:teuthology.orchestra.run.smithi018.stdout:max_mds 2 2024-04-12T21:29:01.123 INFO:teuthology.orchestra.run.smithi018.stdout:in 0,1 2024-04-12T21:29:01.123 INFO:teuthology.orchestra.run.smithi018.stdout:up {} 2024-04-12T21:29:01.123 INFO:teuthology.orchestra.run.smithi018.stdout:failed 0,1 2024-04-12T21:29:01.123 INFO:teuthology.orchestra.run.smithi018.stdout:damaged 2024-04-12T21:29:01.123 INFO:teuthology.orchestra.run.smithi018.stdout:stopped 2024-04-12T21:29:01.123 INFO:teuthology.orchestra.run.smithi018.stdout:data_pools [3] 2024-04-12T21:29:01.123 INFO:teuthology.orchestra.run.smithi018.stdout:metadata_pool 2 2024-04-12T21:29:01.123 INFO:teuthology.orchestra.run.smithi018.stdout:inline_data disabled 2024-04-12T21:29:01.123 INFO:teuthology.orchestra.run.smithi018.stdout:balancer 2024-04-12T21:29:01.123 INFO:teuthology.orchestra.run.smithi018.stdout:bal_rank_mask -1 2024-04-12T21:29:01.123 INFO:teuthology.orchestra.run.smithi018.stdout:standby_count_wanted 1 2024-04-12T21:29:01.123 INFO:teuthology.orchestra.run.smithi018.stdout:qdb_cluster leader: 0 members: 2024-04-12T21:29:01.123 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-12T21:29:01.123 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-12T21:29:01.124 INFO:teuthology.orchestra.run.smithi018.stdout:Standby daemons: 2024-04-12T21:29:01.124 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-12T21:29:01.124 INFO:teuthology.orchestra.run.smithi018.stdout:[mds.cephfs.smithi186.wjnafs{-1:24277} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.186:6826/438347207,v1:172.21.15.186:6827/438347207] compat {c=[1],r=[1],i=[7ff]}] 2024-04-12T21:29:01.124 INFO:teuthology.orchestra.run.smithi018.stdout:[mds.cephfs.smithi018.acqbsc{-1:34436} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.18:6828/755179764,v1:172.21.15.18:6829/755179764] compat {c=[1],r=[1],i=[1fff]}] 2024-04-12T21:29:01.124 INFO:teuthology.orchestra.run.smithi018.stdout:[mds.cephfs.smithi018.szjluo{-1:34468} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.18:6826/2407992653,v1:172.21.15.18:6827/2407992653] compat {c=[1],r=[1],i=[1fff]}] 2024-04-12T21:29:01.124 INFO:teuthology.orchestra.run.smithi018.stdout:[mds.cephfs.smithi186.jdllyn{-1:44595} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.186:6824/3523444202,v1:172.21.15.186:6825/3523444202] compat {c=[1],r=[1],i=[1fff]}] 2024-04-12T21:29:01.126 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:29:01.127+0000 7f6df5c00700 1 -- 172.21.15.18:0/3419652240 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f6dd406ec10 msgr2=0x7f6dd40710c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:29:01.126 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:29:01.127+0000 7f6df5c00700 1 --2- 172.21.15.18:0/3419652240 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f6dd406ec10 0x7f6dd40710c0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f6ddc007fd0 tx=0x7f6ddc009040 comp rx=0 tx=0).stop 2024-04-12T21:29:01.126 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:29:01.127+0000 7f6df5c00700 1 -- 172.21.15.18:0/3419652240 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6df0065630 msgr2=0x7f6df0115b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:29:01.126 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:29:01.127+0000 7f6df5c00700 1 --2- 172.21.15.18:0/3419652240 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6df0065630 0x7f6df0115b90 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7f6de0009180 tx=0x7f6de0009e30 comp rx=0 tx=0).stop 2024-04-12T21:29:01.126 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:29:01.127+0000 7f6df5c00700 1 -- 172.21.15.18:0/3419652240 shutdown_connections 2024-04-12T21:29:01.126 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:29:01.127+0000 7f6df5c00700 1 --2- 172.21.15.18:0/3419652240 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f6dd406ec10 0x7f6dd40710c0 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:29:01.126 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:29:01.127+0000 7f6df5c00700 1 --2- 172.21.15.18:0/3419652240 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6df0065630 0x7f6df0115b90 unknown :-1 s=CLOSED pgs=746 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:29:01.126 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:29:01.127+0000 7f6df5c00700 1 --2- 172.21.15.18:0/3419652240 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6df0102ca0 0x7f6df01160d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:29:01.126 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:29:01.127+0000 7f6df5c00700 1 -- 172.21.15.18:0/3419652240 >> 172.21.15.18:0/3419652240 conn(0x7f6df0071b70 msgr2=0x7f6df01039c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:29:01.127 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:29:01.128+0000 7f6df5c00700 1 -- 172.21.15.18:0/3419652240 shutdown_connections 2024-04-12T21:29:01.127 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:29:01.128+0000 7f6df5c00700 1 -- 172.21.15.18:0/3419652240 wait complete. 2024-04-12T21:29:01.127 INFO:teuthology.orchestra.run.smithi018.stderr:dumped fsmap epoch 31 2024-04-12T21:29:01.282 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:29:01.283+0000 7fc08f000700 1 -- 172.21.15.18:0/321701590 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc088072e00 msgr2=0x7fc08810d5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:29:01.282 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:29:01.283+0000 7fc08f000700 1 --2- 172.21.15.18:0/321701590 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc088072e00 0x7fc08810d5a0 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7fc074004940 tx=0x7fc074009e30 comp rx=0 tx=0).stop 2024-04-12T21:29:01.282 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:29:01.283+0000 7fc08f000700 1 -- 172.21.15.18:0/321701590 shutdown_connections 2024-04-12T21:29:01.282 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:29:01.283+0000 7fc08f000700 1 --2- 172.21.15.18:0/321701590 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc088072e00 0x7fc08810d5a0 unknown :-1 s=CLOSED pgs=747 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:29:01.282 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:29:01.283+0000 7fc08f000700 1 --2- 172.21.15.18:0/321701590 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc0880724f0 0x7fc0880728c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:29:01.282 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:29:01.283+0000 7fc08f000700 1 -- 172.21.15.18:0/321701590 >> 172.21.15.18:0/321701590 conn(0x7fc0880fab10 msgr2=0x7fc0880fcef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:29:01.282 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:29:01.283+0000 7fc08f000700 1 -- 172.21.15.18:0/321701590 shutdown_connections 2024-04-12T21:29:01.282 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:29:01.283+0000 7fc08f000700 1 -- 172.21.15.18:0/321701590 wait complete. 2024-04-12T21:29:01.284 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:29:01.284+0000 7fc08f000700 1 Processor -- start 2024-04-12T21:29:01.284 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:29:01.284+0000 7fc08f000700 1 -- start start 2024-04-12T21:29:01.284 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:29:01.285+0000 7fc08f000700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc0880724f0 0x7fc088119f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:29:01.284 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:29:01.285+0000 7fc08f000700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc088072e00 0x7fc08811a490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:29:01.284 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:29:01.285+0000 7fc08f000700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc08811aae0 con 0x7fc088072e00 2024-04-12T21:29:01.284 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:29:01.285+0000 7fc08f000700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc088113eb0 con 0x7fc0880724f0 2024-04-12T21:29:01.284 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:29:01.285+0000 7fc08d200700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc088072e00 0x7fc08811a490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:29:01.284 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:29:01.285+0000 7fc08d200700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc088072e00 0x7fc08811a490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:39868/0 (socket says 172.21.15.18:39868) 2024-04-12T21:29:01.284 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:29:01.285+0000 7fc08d200700 1 -- 172.21.15.18:0/2161552367 learned_addr learned my addr 172.21.15.18:0/2161552367 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:29:01.284 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:29:01.285+0000 7fc08d200700 1 -- 172.21.15.18:0/2161552367 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc0880724f0 msgr2=0x7fc088119f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:29:01.284 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:29:01.285+0000 7fc08dc00700 1 --2- 172.21.15.18:0/2161552367 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc0880724f0 0x7fc088119f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:29:01.284 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:29:01.285+0000 7fc08d200700 1 --2- 172.21.15.18:0/2161552367 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc0880724f0 0x7fc088119f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:29:01.285 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:29:01.285+0000 7fc08d200700 1 -- 172.21.15.18:0/2161552367 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc074005040 con 0x7fc088072e00 2024-04-12T21:29:01.285 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:29:01.286+0000 7fc08d200700 1 --2- 172.21.15.18:0/2161552367 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc088072e00 0x7fc08811a490 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7fc074004890 tx=0x7fc074009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:29:01.285 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:29:01.286+0000 7fc085600700 1 -- 172.21.15.18:0/2161552367 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc07400e040 con 0x7fc088072e00 2024-04-12T21:29:01.286 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:29:01.286+0000 7fc085600700 1 -- 172.21.15.18:0/2161552367 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc074009af0 con 0x7fc088072e00 2024-04-12T21:29:01.286 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:29:01.286+0000 7fc085600700 1 -- 172.21.15.18:0/2161552367 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc07401c630 con 0x7fc088072e00 2024-04-12T21:29:01.286 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:29:01.286+0000 7fc08f000700 1 -- 172.21.15.18:0/2161552367 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc088114130 con 0x7fc088072e00 2024-04-12T21:29:01.286 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:29:01.286+0000 7fc08f000700 1 -- 172.21.15.18:0/2161552367 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc0881146f0 con 0x7fc088072e00 2024-04-12T21:29:01.286 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:29:01.287+0000 7fc08f000700 1 -- 172.21.15.18:0/2161552367 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc088073c30 con 0x7fc088072e00 2024-04-12T21:29:01.286 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:29:01.288+0000 7fc085600700 1 -- 172.21.15.18:0/2161552367 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7fc07400bbc0 con 0x7fc088072e00 2024-04-12T21:29:01.287 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:29:01.289+0000 7fc085600700 1 --2- 172.21.15.18:0/2161552367 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fc07006ec00 0x7fc0700710b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:29:01.288 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:29:01.289+0000 7fc085600700 1 -- 172.21.15.18:0/2161552367 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 6757+0+0 (secure 0 0 0) 0x7fc07408fcd0 con 0x7fc088072e00 2024-04-12T21:29:01.288 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:29:01.290+0000 7fc08dc00700 1 --2- 172.21.15.18:0/2161552367 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fc07006ec00 0x7fc0700710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:29:01.289 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:29:01.291+0000 7fc08dc00700 1 --2- 172.21.15.18:0/2161552367 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fc07006ec00 0x7fc0700710b0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7fc07c0041d0 tx=0x7fc07c009040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:29:01.293 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:29:01.295+0000 7fc085600700 1 -- 172.21.15.18:0/2161552367 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7fc07405c500 con 0x7fc088072e00 2024-04-12T21:29:01.536 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:29:01.538+0000 7fc08f000700 1 -- 172.21.15.18:0/2161552367 --> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fc088110320 con 0x7fc07006ec00 2024-04-12T21:29:01.540 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:29:01.542+0000 7fc085600700 1 -- 172.21.15.18:0/2161552367 <== mgr.34102 v2:172.21.15.18:6800/3812891919 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7fc088110320 con 0x7fc07006ec00 2024-04-12T21:29:01.540 INFO:teuthology.orchestra.run.smithi018.stdout:{ 2024-04-12T21:29:01.540 INFO:teuthology.orchestra.run.smithi018.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:f4452def3be4d350f4a8e5feaf7ebb1511be9298fe68478ba7a2d6fbd7d69a63", 2024-04-12T21:29:01.540 INFO:teuthology.orchestra.run.smithi018.stdout: "in_progress": true, 2024-04-12T21:29:01.541 INFO:teuthology.orchestra.run.smithi018.stdout: "which": "Upgrading all daemon types on all hosts", 2024-04-12T21:29:01.541 INFO:teuthology.orchestra.run.smithi018.stdout: "services_complete": [ 2024-04-12T21:29:01.541 INFO:teuthology.orchestra.run.smithi018.stdout: "crash", 2024-04-12T21:29:01.541 INFO:teuthology.orchestra.run.smithi018.stdout: "osd", 2024-04-12T21:29:01.541 INFO:teuthology.orchestra.run.smithi018.stdout: "mon", 2024-04-12T21:29:01.541 INFO:teuthology.orchestra.run.smithi018.stdout: "mgr" 2024-04-12T21:29:01.541 INFO:teuthology.orchestra.run.smithi018.stdout: ], 2024-04-12T21:29:01.541 INFO:teuthology.orchestra.run.smithi018.stdout: "progress": "15/21 daemons upgraded", 2024-04-12T21:29:01.541 INFO:teuthology.orchestra.run.smithi018.stdout: "message": "Currently upgrading mds daemons", 2024-04-12T21:29:01.541 INFO:teuthology.orchestra.run.smithi018.stdout: "is_paused": false 2024-04-12T21:29:01.541 INFO:teuthology.orchestra.run.smithi018.stdout:} 2024-04-12T21:29:01.543 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:29:01.544+0000 7fc08f000700 1 -- 172.21.15.18:0/2161552367 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fc07006ec00 msgr2=0x7fc0700710b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:29:01.543 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:29:01.544+0000 7fc08f000700 1 --2- 172.21.15.18:0/2161552367 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fc07006ec00 0x7fc0700710b0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7fc07c0041d0 tx=0x7fc07c009040 comp rx=0 tx=0).stop 2024-04-12T21:29:01.543 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:29:01.544+0000 7fc08f000700 1 -- 172.21.15.18:0/2161552367 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc088072e00 msgr2=0x7fc08811a490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:29:01.543 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:29:01.544+0000 7fc08f000700 1 --2- 172.21.15.18:0/2161552367 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc088072e00 0x7fc08811a490 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7fc074004890 tx=0x7fc074009e30 comp rx=0 tx=0).stop 2024-04-12T21:29:01.543 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:29:01.544+0000 7fc08f000700 1 -- 172.21.15.18:0/2161552367 shutdown_connections 2024-04-12T21:29:01.543 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:29:01.544+0000 7fc08f000700 1 --2- 172.21.15.18:0/2161552367 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fc07006ec00 0x7fc0700710b0 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:29:01.543 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:29:01.544+0000 7fc08f000700 1 --2- 172.21.15.18:0/2161552367 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc0880724f0 0x7fc088119f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:29:01.543 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:29:01.544+0000 7fc08f000700 1 --2- 172.21.15.18:0/2161552367 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc088072e00 0x7fc08811a490 unknown :-1 s=CLOSED pgs=748 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:29:01.543 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:29:01.544+0000 7fc08f000700 1 -- 172.21.15.18:0/2161552367 >> 172.21.15.18:0/2161552367 conn(0x7fc0880fab10 msgr2=0x7fc0880faee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:29:01.543 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:29:01.545+0000 7fc08f000700 1 -- 172.21.15.18:0/2161552367 shutdown_connections 2024-04-12T21:29:01.544 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:29:01.545+0000 7fc08f000700 1 -- 172.21.15.18:0/2161552367 wait complete. 2024-04-12T21:29:01.702 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:29:01.703+0000 7f7be7400700 1 -- 172.21.15.18:0/2701907275 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7be00708a0 msgr2=0x7f7be0070d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:29:01.702 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:29:01.703+0000 7f7be7400700 1 --2- 172.21.15.18:0/2701907275 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7be00708a0 0x7f7be0070d10 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7f7bcc004970 tx=0x7f7bcc009e30 comp rx=0 tx=0).stop 2024-04-12T21:29:01.702 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:29:01.703+0000 7f7be7400700 1 -- 172.21.15.18:0/2701907275 shutdown_connections 2024-04-12T21:29:01.702 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:29:01.703+0000 7f7be7400700 1 --2- 172.21.15.18:0/2701907275 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7be00708a0 0x7f7be0070d10 unknown :-1 s=CLOSED pgs=749 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:29:01.702 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:29:01.703+0000 7f7be7400700 1 --2- 172.21.15.18:0/2701907275 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7be0071f90 0x7f7be0070360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:29:01.702 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:29:01.703+0000 7f7be7400700 1 -- 172.21.15.18:0/2701907275 >> 172.21.15.18:0/2701907275 conn(0x7f7be00f89b0 msgr2=0x7f7be00fadc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:29:01.702 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:29:01.703+0000 7f7be7400700 1 -- 172.21.15.18:0/2701907275 shutdown_connections 2024-04-12T21:29:01.702 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:29:01.704+0000 7f7be7400700 1 -- 172.21.15.18:0/2701907275 wait complete. 2024-04-12T21:29:01.703 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:29:01.704+0000 7f7be7400700 1 Processor -- start 2024-04-12T21:29:01.703 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:29:01.705+0000 7f7be7400700 1 -- start start 2024-04-12T21:29:01.703 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:29:01.705+0000 7f7be7400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7be00708a0 0x7f7be0119fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:29:01.703 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:29:01.705+0000 7f7be7400700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7be0071f90 0x7f7be011a520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:29:01.703 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:29:01.705+0000 7f7be7400700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7be011ac00 con 0x7f7be00708a0 2024-04-12T21:29:01.704 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:29:01.705+0000 7f7be7400700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7be0114150 con 0x7f7be0071f90 2024-04-12T21:29:01.704 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:29:01.705+0000 7f7be4a00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7be00708a0 0x7f7be0119fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:29:01.704 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:29:01.705+0000 7f7be4a00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7be00708a0 0x7f7be0119fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:39886/0 (socket says 172.21.15.18:39886) 2024-04-12T21:29:01.705 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:29:01.705+0000 7f7be4a00700 1 -- 172.21.15.18:0/48611879 learned_addr learned my addr 172.21.15.18:0/48611879 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:29:01.705 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:29:01.706+0000 7f7bdfe00700 1 --2- 172.21.15.18:0/48611879 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7be0071f90 0x7f7be011a520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:29:01.705 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:29:01.706+0000 7f7be4a00700 1 -- 172.21.15.18:0/48611879 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7be0071f90 msgr2=0x7f7be011a520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:29:01.705 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:29:01.706+0000 7f7be4a00700 1 --2- 172.21.15.18:0/48611879 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7be0071f90 0x7f7be011a520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:29:01.705 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:29:01.706+0000 7f7be4a00700 1 -- 172.21.15.18:0/48611879 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7bcc005040 con 0x7f7be00708a0 2024-04-12T21:29:01.705 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:29:01.706+0000 7f7be4a00700 1 --2- 172.21.15.18:0/48611879 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7be00708a0 0x7f7be0119fe0 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7f7bd400b790 tx=0x7f7bd4009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:29:01.707 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:29:01.708+0000 7f7bdcc00700 1 -- 172.21.15.18:0/48611879 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7bd400f770 con 0x7f7be00708a0 2024-04-12T21:29:01.707 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:29:01.709+0000 7f7bdcc00700 1 -- 172.21.15.18:0/48611879 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f7bd400fda0 con 0x7f7be00708a0 2024-04-12T21:29:01.707 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:29:01.709+0000 7f7bdcc00700 1 -- 172.21.15.18:0/48611879 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7bd400d310 con 0x7f7be00708a0 2024-04-12T21:29:01.707 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:29:01.709+0000 7f7be7400700 1 -- 172.21.15.18:0/48611879 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7be01143d0 con 0x7f7be00708a0 2024-04-12T21:29:01.707 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:29:01.709+0000 7f7be7400700 1 -- 172.21.15.18:0/48611879 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7be01148e0 con 0x7f7be00708a0 2024-04-12T21:29:01.708 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:29:01.710+0000 7f7be7400700 1 -- 172.21.15.18:0/48611879 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7be0071aa0 con 0x7f7be00708a0 2024-04-12T21:29:01.709 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:29:01.711+0000 7f7bdcc00700 1 -- 172.21.15.18:0/48611879 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7f7bd4010420 con 0x7f7be00708a0 2024-04-12T21:29:01.711 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:29:01.712+0000 7f7bdcc00700 1 --2- 172.21.15.18:0/48611879 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f7bc806ec00 0x7f7bc80710b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:29:01.711 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:29:01.713+0000 7f7bdcc00700 1 -- 172.21.15.18:0/48611879 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 6757+0+0 (secure 0 0 0) 0x7f7bd408e8e0 con 0x7f7be00708a0 2024-04-12T21:29:01.711 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:29:01.713+0000 7f7bdfe00700 1 --2- 172.21.15.18:0/48611879 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f7bc806ec00 0x7f7bc80710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:29:01.712 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:29:01.714+0000 7f7bdfe00700 1 --2- 172.21.15.18:0/48611879 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f7bc806ec00 0x7f7bc80710b0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f7bcc009fd0 tx=0x7f7bcc009e30 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:29:01.716 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:29:01.718+0000 7f7bdcc00700 1 -- 172.21.15.18:0/48611879 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f7bd405b1d0 con 0x7f7be00708a0 2024-04-12T21:29:01.852 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:29:01 smithi018 ceph-mon[112590]: from='client.35548 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:29:01.852 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:29:01 smithi018 ceph-mon[112590]: from='client.? 172.21.15.18:0/3419652240' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-12T21:29:02.031 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:29:01 smithi186 ceph-mon[86379]: from='client.35548 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:29:02.031 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:29:01 smithi186 ceph-mon[86379]: from='client.? 172.21.15.18:0/3419652240' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-12T21:29:02.042 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:29:02.043+0000 7f7be7400700 1 -- 172.21.15.18:0/48611879 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f7be0115660 con 0x7f7be00708a0 2024-04-12T21:29:02.043 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:29:02.044+0000 7f7bdcc00700 1 -- 172.21.15.18:0/48611879 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+4303 (secure 0 0 0) 0x7f7bd405ad60 con 0x7f7be00708a0 2024-04-12T21:29:02.043 INFO:teuthology.orchestra.run.smithi018.stdout:HEALTH_ERR failed to probe daemons or devices; 1 filesystem is degraded; 1 filesystem is offline 2024-04-12T21:29:02.043 INFO:teuthology.orchestra.run.smithi018.stdout:[WRN] CEPHADM_REFRESH_FAILED: failed to probe daemons or devices 2024-04-12T21:29:02.043 INFO:teuthology.orchestra.run.smithi018.stdout: host smithi018 `cephadm ceph-volume` failed: cephadm exited with an error code: 1, stderr: Inferring config /var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/mon.smithi018/config 2024-04-12T21:29:02.044 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/podman: timeout after 895 seconds 2024-04-12T21:29:02.044 INFO:teuthology.orchestra.run.smithi018.stdout:Non-zero exit code 124 from /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:f4452def3be4d350f4a8e5feaf7ebb1511be9298fe68478ba7a2d6fbd7d69a63 -e NODE_NAME=smithi018 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968:/var/run/ceph:z -v /var/log/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968:/var/log/ceph:z -v /var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmpxsmwkkuh:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:f4452def3be4d350f4a8e5feaf7ebb1511be9298fe68478ba7a2d6fbd7d69a63 inventory --format=json-pretty --filter-for-batch 2024-04-12T21:29:02.044 INFO:teuthology.orchestra.run.smithi018.stdout:Traceback (most recent call last): 2024-04-12T21:29:02.044 INFO:teuthology.orchestra.run.smithi018.stdout: File "/usr/lib64/python3.9/runpy.py", line 197, in _run_module_as_main 2024-04-12T21:29:02.044 INFO:teuthology.orchestra.run.smithi018.stdout: return _run_code(code, main_globals, None, 2024-04-12T21:29:02.044 INFO:teuthology.orchestra.run.smithi018.stdout: File "/usr/lib64/python3.9/runpy.py", line 87, in _run_code 2024-04-12T21:29:02.044 INFO:teuthology.orchestra.run.smithi018.stdout: exec(code, run_globals) 2024-04-12T21:29:02.044 INFO:teuthology.orchestra.run.smithi018.stdout: File "/var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/cephadm.e44f3ed5e34e4ef52eedef233d1190bff004470d8e8005127b5312bff357fa7f/__main__.py", line 5618, in 2024-04-12T21:29:02.044 INFO:teuthology.orchestra.run.smithi018.stdout: File "/var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/cephadm.e44f3ed5e34e4ef52eedef233d1190bff004470d8e8005127b5312bff357fa7f/__main__.py", line 5606, in main 2024-04-12T21:29:02.044 INFO:teuthology.orchestra.run.smithi018.stdout: File "/var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/cephadm.e44f3ed5e34e4ef52eedef233d1190bff004470d8e8005127b5312bff357fa7f/__main__.py", line 408, in _infer_config 2024-04-12T21:29:02.044 INFO:teuthology.orchestra.run.smithi018.stdout: File "/var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/cephadm.e44f3ed5e34e4ef52eedef233d1190bff004470d8e8005127b5312bff357fa7f/__main__.py", line 323, in _infer_fsid 2024-04-12T21:29:02.044 INFO:teuthology.orchestra.run.smithi018.stdout: File "/var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/cephadm.e44f3ed5e34e4ef52eedef233d1190bff004470d8e8005127b5312bff357fa7f/__main__.py", line 436, in _infer_image 2024-04-12T21:29:02.045 INFO:teuthology.orchestra.run.smithi018.stdout: File "/var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/cephadm.e44f3ed5e34e4ef52eedef233d1190bff004470d8e8005127b5312bff357fa7f/__main__.py", line 310, in _validate_fsid 2024-04-12T21:29:02.045 INFO:teuthology.orchestra.run.smithi018.stdout: File "/var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/cephadm.e44f3ed5e34e4ef52eedef233d1190bff004470d8e8005127b5312bff357fa7f/__main__.py", line 3359, in command_ceph_volume 2024-04-12T21:29:02.045 INFO:teuthology.orchestra.run.smithi018.stdout: File "/var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/cephadm.e44f3ed5e34e4ef52eedef233d1190bff004470d8e8005127b5312bff357fa7f/cephadmlib/call_wrappers.py", line 310, in call_throws 2024-04-12T21:29:02.045 INFO:teuthology.orchestra.run.smithi018.stdout:RuntimeError: Failed command: /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:f4452def3be4d350f4a8e5feaf7ebb1511be9298fe68478ba7a2d6fbd7d69a63 -e NODE_NAME=smithi018 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968:/var/run/ceph:z -v /var/log/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968:/var/log/ceph:z -v /var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmpxsmwkkuh:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:f4452def3be4d350f4a8e5feaf7ebb1511be9298fe68478ba7a2d6fbd7d69a63 inventory --format=json-pretty --filter-for-batch 2024-04-12T21:29:02.045 INFO:teuthology.orchestra.run.smithi018.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-04-12T21:29:02.045 INFO:teuthology.orchestra.run.smithi018.stdout: fs cephfs is degraded 2024-04-12T21:29:02.045 INFO:teuthology.orchestra.run.smithi018.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-04-12T21:29:02.045 INFO:teuthology.orchestra.run.smithi018.stdout: fs cephfs is offline because no MDS is active for it. 2024-04-12T21:29:02.047 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:29:02.048+0000 7f7be7400700 1 -- 172.21.15.18:0/48611879 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f7bc806ec00 msgr2=0x7f7bc80710b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:29:02.047 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:29:02.048+0000 7f7be7400700 1 --2- 172.21.15.18:0/48611879 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f7bc806ec00 0x7f7bc80710b0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f7bcc009fd0 tx=0x7f7bcc009e30 comp rx=0 tx=0).stop 2024-04-12T21:29:02.047 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:29:02.048+0000 7f7be7400700 1 -- 172.21.15.18:0/48611879 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7be00708a0 msgr2=0x7f7be0119fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:29:02.047 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:29:02.048+0000 7f7be7400700 1 --2- 172.21.15.18:0/48611879 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7be00708a0 0x7f7be0119fe0 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7f7bd400b790 tx=0x7f7bd4009250 comp rx=0 tx=0).stop 2024-04-12T21:29:02.047 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:29:02.048+0000 7f7be7400700 1 -- 172.21.15.18:0/48611879 shutdown_connections 2024-04-12T21:29:02.047 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:29:02.048+0000 7f7be7400700 1 --2- 172.21.15.18:0/48611879 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f7bc806ec00 0x7f7bc80710b0 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:29:02.047 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:29:02.048+0000 7f7be7400700 1 --2- 172.21.15.18:0/48611879 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7be00708a0 0x7f7be0119fe0 unknown :-1 s=CLOSED pgs=750 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:29:02.047 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:29:02.048+0000 7f7be7400700 1 --2- 172.21.15.18:0/48611879 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7be0071f90 0x7f7be011a520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:29:02.047 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:29:02.048+0000 7f7be7400700 1 -- 172.21.15.18:0/48611879 >> 172.21.15.18:0/48611879 conn(0x7f7be00f89b0 msgr2=0x7f7be00f9ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:29:02.047 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:29:02.048+0000 7f7be7400700 1 -- 172.21.15.18:0/48611879 shutdown_connections 2024-04-12T21:29:02.047 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:29:02.048+0000 7f7be7400700 1 -- 172.21.15.18:0/48611879 wait complete. 2024-04-12T21:29:03.031 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:29:02 smithi186 ceph-mon[86379]: from='client.35560 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:29:03.031 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:29:02 smithi186 ceph-mon[86379]: pgmap v878: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:29:03.031 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:29:02 smithi186 ceph-mon[86379]: from='client.? 172.21.15.18:0/48611879' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-12T21:29:03.102 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:29:02 smithi018 ceph-mon[112590]: from='client.35560 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:29:03.102 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:29:02 smithi018 ceph-mon[112590]: pgmap v878: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:29:03.102 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:29:02 smithi018 ceph-mon[112590]: from='client.? 172.21.15.18:0/48611879' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-12T21:29:05.030 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:29:04 smithi186 ceph-mon[86379]: pgmap v879: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:29:05.102 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:29:04 smithi018 ceph-mon[112590]: pgmap v879: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:29:07.031 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:29:06 smithi186 ceph-mon[86379]: pgmap v880: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:29:07.102 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:29:06 smithi018 ceph-mon[112590]: pgmap v880: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:29:09.031 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:29:08 smithi186 ceph-mon[86379]: pgmap v881: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:29:09.102 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:29:08 smithi018 ceph-mon[112590]: pgmap v881: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:29:11.031 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:29:10 smithi186 ceph-mon[86379]: pgmap v882: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:29:11.031 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:29:10 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T21:29:11.102 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:29:10 smithi018 ceph-mon[112590]: pgmap v882: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:29:11.102 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:29:10 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T21:29:13.031 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:29:12 smithi186 ceph-mon[86379]: pgmap v883: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:29:13.102 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:29:12 smithi018 ceph-mon[112590]: pgmap v883: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:29:15.031 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:29:14 smithi186 ceph-mon[86379]: pgmap v884: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:29:15.102 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:29:14 smithi018 ceph-mon[112590]: pgmap v884: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:29:17.031 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:29:16 smithi186 ceph-mon[86379]: pgmap v885: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:29:17.102 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:29:16 smithi018 ceph-mon[112590]: pgmap v885: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:29:19.031 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:29:18 smithi186 ceph-mon[86379]: pgmap v886: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:29:19.102 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:29:18 smithi018 ceph-mon[112590]: pgmap v886: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:29:21.102 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:29:20 smithi018 ceph-mon[112590]: pgmap v887: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:29:21.281 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:29:20 smithi186 ceph-mon[86379]: pgmap v887: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:29:23.102 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:29:22 smithi018 ceph-mon[112590]: pgmap v888: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:29:23.281 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:29:22 smithi186 ceph-mon[86379]: pgmap v888: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:29:25.102 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:29:24 smithi018 ceph-mon[112590]: pgmap v889: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:29:25.281 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:29:24 smithi186 ceph-mon[86379]: pgmap v889: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:29:26.102 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:29:25 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T21:29:26.281 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:29:25 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T21:29:27.102 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:29:26 smithi018 ceph-mon[112590]: pgmap v890: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:29:27.281 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:29:26 smithi186 ceph-mon[86379]: pgmap v890: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:29:29.102 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:29:28 smithi018 ceph-mon[112590]: pgmap v891: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:29:29.281 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:29:28 smithi186 ceph-mon[86379]: pgmap v891: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:29:31.102 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:29:30 smithi018 ceph-mon[112590]: pgmap v892: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:29:31.281 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:29:30 smithi186 ceph-mon[86379]: pgmap v892: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:29:32.219 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:29:32.220+0000 7f23e9400700 1 -- 172.21.15.18:0/3840881796 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f23e40fe7b0 msgr2=0x7f23e4106e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:29:32.219 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:29:32.220+0000 7f23e9400700 1 --2- 172.21.15.18:0/3840881796 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f23e40fe7b0 0x7f23e4106e80 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7f23d0004940 tx=0x7f23d0009e30 comp rx=0 tx=0).stop 2024-04-12T21:29:32.219 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:29:32.220+0000 7f23e9400700 1 -- 172.21.15.18:0/3840881796 shutdown_connections 2024-04-12T21:29:32.219 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:29:32.220+0000 7f23e9400700 1 --2- 172.21.15.18:0/3840881796 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f23e40fe7b0 0x7f23e4106e80 unknown :-1 s=CLOSED pgs=751 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:29:32.219 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:29:32.220+0000 7f23e9400700 1 --2- 172.21.15.18:0/3840881796 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f23e40fde10 0x7f23e40fe1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:29:32.219 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:29:32.220+0000 7f23e9400700 1 -- 172.21.15.18:0/3840881796 >> 172.21.15.18:0/3840881796 conn(0x7f23e40f7870 msgr2=0x7f23e40f9c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:29:32.219 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:29:32.220+0000 7f23e9400700 1 -- 172.21.15.18:0/3840881796 shutdown_connections 2024-04-12T21:29:32.220 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:29:32.220+0000 7f23e9400700 1 -- 172.21.15.18:0/3840881796 wait complete. 2024-04-12T21:29:32.221 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:29:32.221+0000 7f23e9400700 1 Processor -- start 2024-04-12T21:29:32.221 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:29:32.221+0000 7f23e9400700 1 -- start start 2024-04-12T21:29:32.221 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:29:32.221+0000 7f23e9400700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f23e40fde10 0x7f23e4115b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:29:32.221 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:29:32.222+0000 7f23e9400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f23e40fe7b0 0x7f23e41160c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:29:32.221 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:29:32.222+0000 7f23e9400700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23e41167a0 con 0x7f23e40fe7b0 2024-04-12T21:29:32.221 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:29:32.222+0000 7f23e9400700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23e410fcf0 con 0x7f23e40fde10 2024-04-12T21:29:32.221 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:29:32.222+0000 7f23e1e00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f23e40fe7b0 0x7f23e41160c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:29:32.222 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:29:32.222+0000 7f23e1e00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f23e40fe7b0 0x7f23e41160c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:55832/0 (socket says 172.21.15.18:55832) 2024-04-12T21:29:32.222 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:29:32.222+0000 7f23e1e00700 1 -- 172.21.15.18:0/178905150 learned_addr learned my addr 172.21.15.18:0/178905150 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:29:32.222 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:29:32.222+0000 7f23e2800700 1 --2- 172.21.15.18:0/178905150 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f23e40fde10 0x7f23e4115b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:29:32.222 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:29:32.222+0000 7f23e1e00700 1 -- 172.21.15.18:0/178905150 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f23e40fde10 msgr2=0x7f23e4115b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:29:32.222 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:29:32.222+0000 7f23e1e00700 1 --2- 172.21.15.18:0/178905150 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f23e40fde10 0x7f23e4115b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:29:32.222 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:29:32.222+0000 7f23e1e00700 1 -- 172.21.15.18:0/178905150 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23d0005040 con 0x7f23e40fe7b0 2024-04-12T21:29:32.222 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:29:32.222+0000 7f23e2800700 1 --2- 172.21.15.18:0/178905150 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f23e40fde10 0x7f23e4115b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-12T21:29:32.222 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:29:32.223+0000 7f23e1e00700 1 --2- 172.21.15.18:0/178905150 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f23e40fe7b0 0x7f23e41160c0 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7f23d0009180 tx=0x7f23d0009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:29:32.223 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:29:32.223+0000 7f23daa00700 1 -- 172.21.15.18:0/178905150 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f23d000e040 con 0x7f23e40fe7b0 2024-04-12T21:29:32.223 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:29:32.223+0000 7f23daa00700 1 -- 172.21.15.18:0/178905150 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f23d0013420 con 0x7f23e40fe7b0 2024-04-12T21:29:32.223 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:29:32.223+0000 7f23e9400700 1 -- 172.21.15.18:0/178905150 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f23e410ff10 con 0x7f23e40fe7b0 2024-04-12T21:29:32.223 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:29:32.223+0000 7f23daa00700 1 -- 172.21.15.18:0/178905150 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f23d001c630 con 0x7f23e40fe7b0 2024-04-12T21:29:32.223 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:29:32.223+0000 7f23e9400700 1 -- 172.21.15.18:0/178905150 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f23e4110450 con 0x7f23e40fe7b0 2024-04-12T21:29:32.223 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:29:32.224+0000 7f23e9400700 1 -- 172.21.15.18:0/178905150 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f23e4106940 con 0x7f23e40fe7b0 2024-04-12T21:29:32.224 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:29:32.226+0000 7f23daa00700 1 -- 172.21.15.18:0/178905150 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7f23d0013590 con 0x7f23e40fe7b0 2024-04-12T21:29:32.225 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:29:32.227+0000 7f23daa00700 1 --2- 172.21.15.18:0/178905150 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f23c806ec00 0x7f23c80710b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:29:32.225 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:29:32.227+0000 7f23daa00700 1 -- 172.21.15.18:0/178905150 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 6757+0+0 (secure 0 0 0) 0x7f23d008fde0 con 0x7f23e40fe7b0 2024-04-12T21:29:32.225 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:29:32.227+0000 7f23e2800700 1 --2- 172.21.15.18:0/178905150 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f23c806ec00 0x7f23c80710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:29:32.226 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:29:32.228+0000 7f23e2800700 1 --2- 172.21.15.18:0/178905150 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f23c806ec00 0x7f23c80710b0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f23d4006fd0 tx=0x7f23d4008040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:29:32.231 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:29:32.232+0000 7f23daa00700 1 -- 172.21.15.18:0/178905150 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f23d005c590 con 0x7f23e40fe7b0 2024-04-12T21:29:32.483 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:29:32.485+0000 7f23e9400700 1 -- 172.21.15.18:0/178905150 --> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f23e4109f20 con 0x7f23c806ec00 2024-04-12T21:29:32.485 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:29:32.487+0000 7f23daa00700 1 -- 172.21.15.18:0/178905150 <== mgr.34102 v2:172.21.15.18:6800/3812891919 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f23e4109f20 con 0x7f23c806ec00 2024-04-12T21:29:32.488 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:29:32.489+0000 7f23e9400700 1 -- 172.21.15.18:0/178905150 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f23c806ec00 msgr2=0x7f23c80710b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:29:32.488 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:29:32.489+0000 7f23e9400700 1 --2- 172.21.15.18:0/178905150 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f23c806ec00 0x7f23c80710b0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f23d4006fd0 tx=0x7f23d4008040 comp rx=0 tx=0).stop 2024-04-12T21:29:32.488 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:29:32.490+0000 7f23e9400700 1 -- 172.21.15.18:0/178905150 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f23e40fe7b0 msgr2=0x7f23e41160c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:29:32.488 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:29:32.490+0000 7f23e9400700 1 --2- 172.21.15.18:0/178905150 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f23e40fe7b0 0x7f23e41160c0 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7f23d0009180 tx=0x7f23d0009e30 comp rx=0 tx=0).stop 2024-04-12T21:29:32.489 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:29:32.490+0000 7f23e9400700 1 -- 172.21.15.18:0/178905150 shutdown_connections 2024-04-12T21:29:32.489 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:29:32.490+0000 7f23e9400700 1 --2- 172.21.15.18:0/178905150 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f23c806ec00 0x7f23c80710b0 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:29:32.489 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:29:32.490+0000 7f23e9400700 1 --2- 172.21.15.18:0/178905150 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f23e40fde10 0x7f23e4115b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:29:32.489 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:29:32.490+0000 7f23e9400700 1 --2- 172.21.15.18:0/178905150 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f23e40fe7b0 0x7f23e41160c0 unknown :-1 s=CLOSED pgs=752 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:29:32.489 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:29:32.490+0000 7f23e9400700 1 -- 172.21.15.18:0/178905150 >> 172.21.15.18:0/178905150 conn(0x7f23e40f7870 msgr2=0x7f23e40fd480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:29:32.489 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:29:32.490+0000 7f23e9400700 1 -- 172.21.15.18:0/178905150 shutdown_connections 2024-04-12T21:29:32.489 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:29:32.490+0000 7f23e9400700 1 -- 172.21.15.18:0/178905150 wait complete. 2024-04-12T21:29:32.504 INFO:teuthology.orchestra.run.smithi018.stdout:true 2024-04-12T21:29:32.649 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:29:32.650+0000 7f32bde00700 1 -- 172.21.15.18:0/486009931 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f32b8109430 msgr2=0x7f32b8109800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:29:32.649 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:29:32.650+0000 7f32bde00700 1 --2- 172.21.15.18:0/486009931 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f32b8109430 0x7f32b8109800 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7f32a8004940 tx=0x7f32a8009e30 comp rx=0 tx=0).stop 2024-04-12T21:29:32.649 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:29:32.650+0000 7f32bde00700 1 -- 172.21.15.18:0/486009931 shutdown_connections 2024-04-12T21:29:32.649 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:29:32.650+0000 7f32bde00700 1 --2- 172.21.15.18:0/486009931 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f32b8109dd0 0x7f32b8103440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:29:32.649 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:29:32.650+0000 7f32bde00700 1 --2- 172.21.15.18:0/486009931 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f32b8109430 0x7f32b8109800 unknown :-1 s=CLOSED pgs=753 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:29:32.649 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:29:32.650+0000 7f32bde00700 1 -- 172.21.15.18:0/486009931 >> 172.21.15.18:0/486009931 conn(0x7f32b8072c00 msgr2=0x7f32b8075010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:29:32.649 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:29:32.650+0000 7f32bde00700 1 -- 172.21.15.18:0/486009931 shutdown_connections 2024-04-12T21:29:32.649 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:29:32.651+0000 7f32bde00700 1 -- 172.21.15.18:0/486009931 wait complete. 2024-04-12T21:29:32.650 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:29:32.651+0000 7f32bde00700 1 Processor -- start 2024-04-12T21:29:32.650 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:29:32.652+0000 7f32bde00700 1 -- start start 2024-04-12T21:29:32.650 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:29:32.652+0000 7f32bde00700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f32b8109430 0x7f32b8119f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:29:32.651 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:29:32.652+0000 7f32bde00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f32b8109dd0 0x7f32b811a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:29:32.651 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:29:32.652+0000 7f32bde00700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32b811aad0 con 0x7f32b8109dd0 2024-04-12T21:29:32.651 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:29:32.652+0000 7f32bde00700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32b8114100 con 0x7f32b8109430 2024-04-12T21:29:32.651 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:29:32.652+0000 7f32afe00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f32b8109dd0 0x7f32b811a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:29:32.652 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:29:32.652+0000 7f32afe00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f32b8109dd0 0x7f32b811a480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:55854/0 (socket says 172.21.15.18:55854) 2024-04-12T21:29:32.652 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:29:32.652+0000 7f32afe00700 1 -- 172.21.15.18:0/649056369 learned_addr learned my addr 172.21.15.18:0/649056369 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:29:32.652 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:29:32.653+0000 7f32b6a00700 1 --2- 172.21.15.18:0/649056369 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f32b8109430 0x7f32b8119f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:29:32.652 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:29:32.653+0000 7f32afe00700 1 -- 172.21.15.18:0/649056369 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f32b8109430 msgr2=0x7f32b8119f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:29:32.652 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:29:32.653+0000 7f32afe00700 1 --2- 172.21.15.18:0/649056369 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f32b8109430 0x7f32b8119f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:29:32.652 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:29:32.653+0000 7f32afe00700 1 -- 172.21.15.18:0/649056369 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32a8005040 con 0x7f32b8109dd0 2024-04-12T21:29:32.652 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:29:32.653+0000 7f32afe00700 1 --2- 172.21.15.18:0/649056369 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f32b8109dd0 0x7f32b811a480 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7f32a0002a50 tx=0x7f32a0004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:29:32.653 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:29:32.653+0000 7f32aea00700 1 -- 172.21.15.18:0/649056369 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f32a0007cc0 con 0x7f32b8109dd0 2024-04-12T21:29:32.653 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:29:32.654+0000 7f32bde00700 1 -- 172.21.15.18:0/649056369 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f32b81143e0 con 0x7f32b8109dd0 2024-04-12T21:29:32.653 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:29:32.654+0000 7f32aea00700 1 -- 172.21.15.18:0/649056369 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f32a000f040 con 0x7f32b8109dd0 2024-04-12T21:29:32.653 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:29:32.654+0000 7f32aea00700 1 -- 172.21.15.18:0/649056369 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f32a000da40 con 0x7f32b8109dd0 2024-04-12T21:29:32.653 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:29:32.654+0000 7f32bde00700 1 -- 172.21.15.18:0/649056369 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f32b81149a0 con 0x7f32b8109dd0 2024-04-12T21:29:32.653 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:29:32.655+0000 7f32bde00700 1 -- 172.21.15.18:0/649056369 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f32b8104b80 con 0x7f32b8109dd0 2024-04-12T21:29:32.654 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:29:32.656+0000 7f32aea00700 1 -- 172.21.15.18:0/649056369 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7f32a00097c0 con 0x7f32b8109dd0 2024-04-12T21:29:32.655 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:29:32.657+0000 7f32aea00700 1 --2- 172.21.15.18:0/649056369 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f329c06ec00 0x7f329c0710b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:29:32.655 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:29:32.657+0000 7f32aea00700 1 -- 172.21.15.18:0/649056369 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 6757+0+0 (secure 0 0 0) 0x7f32a008e7b0 con 0x7f32b8109dd0 2024-04-12T21:29:32.656 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:29:32.657+0000 7f32b6a00700 1 --2- 172.21.15.18:0/649056369 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f329c06ec00 0x7f329c0710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:29:32.656 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:29:32.658+0000 7f32b6a00700 1 --2- 172.21.15.18:0/649056369 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f329c06ec00 0x7f329c0710b0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f32a800b3b0 tx=0x7f32a8009e30 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:29:32.660 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:29:32.662+0000 7f32aea00700 1 -- 172.21.15.18:0/649056369 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f32a005acd0 con 0x7f32b8109dd0 2024-04-12T21:29:32.917 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:29:32.918+0000 7f32bde00700 1 -- 172.21.15.18:0/649056369 --> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f32b8110590 con 0x7f329c06ec00 2024-04-12T21:29:32.918 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:29:32.920+0000 7f32aea00700 1 -- 172.21.15.18:0/649056369 <== mgr.34102 v2:172.21.15.18:6800/3812891919 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f32b8110590 con 0x7f329c06ec00 2024-04-12T21:29:32.921 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:29:32.923+0000 7f32bde00700 1 -- 172.21.15.18:0/649056369 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f329c06ec00 msgr2=0x7f329c0710b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:29:32.921 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:29:32.923+0000 7f32bde00700 1 --2- 172.21.15.18:0/649056369 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f329c06ec00 0x7f329c0710b0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f32a800b3b0 tx=0x7f32a8009e30 comp rx=0 tx=0).stop 2024-04-12T21:29:32.921 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:29:32.923+0000 7f32bde00700 1 -- 172.21.15.18:0/649056369 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f32b8109dd0 msgr2=0x7f32b811a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:29:32.921 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:29:32.923+0000 7f32bde00700 1 --2- 172.21.15.18:0/649056369 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f32b8109dd0 0x7f32b811a480 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7f32a0002a50 tx=0x7f32a0004600 comp rx=0 tx=0).stop 2024-04-12T21:29:32.922 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:29:32.923+0000 7f32bde00700 1 -- 172.21.15.18:0/649056369 shutdown_connections 2024-04-12T21:29:32.922 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:29:32.923+0000 7f32bde00700 1 --2- 172.21.15.18:0/649056369 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f329c06ec00 0x7f329c0710b0 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:29:32.922 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:29:32.923+0000 7f32bde00700 1 --2- 172.21.15.18:0/649056369 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f32b8109430 0x7f32b8119f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:29:32.922 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:29:32.923+0000 7f32bde00700 1 --2- 172.21.15.18:0/649056369 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f32b8109dd0 0x7f32b811a480 unknown :-1 s=CLOSED pgs=754 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:29:32.922 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:29:32.923+0000 7f32bde00700 1 -- 172.21.15.18:0/649056369 >> 172.21.15.18:0/649056369 conn(0x7f32b8072c00 msgr2=0x7f32b8074c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:29:32.922 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:29:32.923+0000 7f32bde00700 1 -- 172.21.15.18:0/649056369 shutdown_connections 2024-04-12T21:29:32.922 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:29:32.923+0000 7f32bde00700 1 -- 172.21.15.18:0/649056369 wait complete. 2024-04-12T21:29:33.079 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:29:33.080+0000 7f8caca00700 1 -- 172.21.15.18:0/1056465343 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8ca80fe5e0 msgr2=0x7f8ca8106c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:29:33.079 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:29:33.080+0000 7f8caca00700 1 --2- 172.21.15.18:0/1056465343 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8ca80fe5e0 0x7f8ca8106c20 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7f8c94004940 tx=0x7f8c94009e30 comp rx=0 tx=0).stop 2024-04-12T21:29:33.079 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:29:32 smithi018 ceph-mon[112590]: pgmap v893: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:29:33.080 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:29:33.080+0000 7f8caca00700 1 -- 172.21.15.18:0/1056465343 shutdown_connections 2024-04-12T21:29:33.080 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:29:33.080+0000 7f8caca00700 1 --2- 172.21.15.18:0/1056465343 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8ca80fe5e0 0x7f8ca8106c20 unknown :-1 s=CLOSED pgs=755 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:29:33.080 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:29:33.080+0000 7f8caca00700 1 --2- 172.21.15.18:0/1056465343 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8ca80fdc40 0x7f8ca80fe010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:29:33.080 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:29:33.080+0000 7f8caca00700 1 -- 172.21.15.18:0/1056465343 >> 172.21.15.18:0/1056465343 conn(0x7f8ca80f7670 msgr2=0x7f8ca80f9a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:29:33.080 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:29:33.081+0000 7f8caca00700 1 -- 172.21.15.18:0/1056465343 shutdown_connections 2024-04-12T21:29:33.080 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:29:33.081+0000 7f8caca00700 1 -- 172.21.15.18:0/1056465343 wait complete. 2024-04-12T21:29:33.081 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:29:33.082+0000 7f8caca00700 1 Processor -- start 2024-04-12T21:29:33.081 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:29:33.082+0000 7f8caca00700 1 -- start start 2024-04-12T21:29:33.082 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:29:33.082+0000 7f8caca00700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8ca80fdc40 0x7f8ca8115e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:29:33.082 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:29:33.082+0000 7f8caca00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8ca8116350 0x7f8ca810fd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:29:33.082 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:29:33.082+0000 7f8caca00700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ca81168d0 con 0x7f8ca8116350 2024-04-12T21:29:33.082 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:29:33.082+0000 7f8caca00700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ca8116a40 con 0x7f8ca80fdc40 2024-04-12T21:29:33.082 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:29:33.082+0000 7f8ca6a00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8ca8116350 0x7f8ca810fd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:29:33.082 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:29:33.082+0000 7f8ca6a00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8ca8116350 0x7f8ca810fd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:55864/0 (socket says 172.21.15.18:55864) 2024-04-12T21:29:33.082 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:29:33.082+0000 7f8ca6a00700 1 -- 172.21.15.18:0/2951051952 learned_addr learned my addr 172.21.15.18:0/2951051952 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:29:33.082 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:29:33.083+0000 7f8ca7400700 1 --2- 172.21.15.18:0/2951051952 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8ca80fdc40 0x7f8ca8115e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:29:33.082 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:29:33.083+0000 7f8ca6a00700 1 -- 172.21.15.18:0/2951051952 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8ca80fdc40 msgr2=0x7f8ca8115e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:29:33.082 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:29:33.083+0000 7f8ca6a00700 1 --2- 172.21.15.18:0/2951051952 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8ca80fdc40 0x7f8ca8115e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:29:33.082 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:29:33.083+0000 7f8ca6a00700 1 -- 172.21.15.18:0/2951051952 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c94005040 con 0x7f8ca8116350 2024-04-12T21:29:33.083 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:29:33.083+0000 7f8ca6a00700 1 --2- 172.21.15.18:0/2951051952 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8ca8116350 0x7f8ca810fd70 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7f8c9400b3b0 tx=0x7f8c94009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:29:33.083 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:29:33.084+0000 7f8c9f400700 1 -- 172.21.15.18:0/2951051952 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8c9400e040 con 0x7f8ca8116350 2024-04-12T21:29:33.083 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:29:33.084+0000 7f8c9f400700 1 -- 172.21.15.18:0/2951051952 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f8c94013420 con 0x7f8ca8116350 2024-04-12T21:29:33.083 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:29:33.084+0000 7f8caca00700 1 -- 172.21.15.18:0/2951051952 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8ca8110370 con 0x7f8ca8116350 2024-04-12T21:29:33.083 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:29:33.084+0000 7f8c9f400700 1 -- 172.21.15.18:0/2951051952 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8c9401c630 con 0x7f8ca8116350 2024-04-12T21:29:33.083 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:29:33.084+0000 7f8caca00700 1 -- 172.21.15.18:0/2951051952 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8ca81108b0 con 0x7f8ca8116350 2024-04-12T21:29:33.084 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:29:33.085+0000 7f8caca00700 1 -- 172.21.15.18:0/2951051952 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8ca8110be0 con 0x7f8ca8116350 2024-04-12T21:29:33.084 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:29:33.086+0000 7f8c9f400700 1 -- 172.21.15.18:0/2951051952 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7f8c94013590 con 0x7f8ca8116350 2024-04-12T21:29:33.086 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:29:33.087+0000 7f8c9f400700 1 --2- 172.21.15.18:0/2951051952 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f8c8c06ec00 0x7f8c8c0710b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:29:33.086 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:29:33.087+0000 7f8c9f400700 1 -- 172.21.15.18:0/2951051952 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 6757+0+0 (secure 0 0 0) 0x7f8c9408fad0 con 0x7f8ca8116350 2024-04-12T21:29:33.086 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:29:33.087+0000 7f8ca7400700 1 --2- 172.21.15.18:0/2951051952 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f8c8c06ec00 0x7f8c8c0710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:29:33.086 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:29:33.088+0000 7f8ca7400700 1 --2- 172.21.15.18:0/2951051952 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f8c8c06ec00 0x7f8c8c0710b0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f8c980041d0 tx=0x7f8c98009040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:29:33.090 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:29:33.092+0000 7f8c9f400700 1 -- 172.21.15.18:0/2951051952 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f8ca8110be0 con 0x7f8ca8116350 2024-04-12T21:29:33.281 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:29:32 smithi186 ceph-mon[86379]: pgmap v893: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:29:33.330 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:29:33.331+0000 7f8caca00700 1 -- 172.21.15.18:0/2951051952 --> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f8ca8110be0 con 0x7f8c8c06ec00 2024-04-12T21:29:33.360 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:29:33.361+0000 7f8c9f400700 1 -- 172.21.15.18:0/2951051952 <== mgr.34102 v2:172.21.15.18:6800/3812891919 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7f8ca8110be0 con 0x7f8c8c06ec00 2024-04-12T21:29:33.360 INFO:teuthology.orchestra.run.smithi018.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-12T21:29:33.360 INFO:teuthology.orchestra.run.smithi018.stdout:alertmanager.smithi018 smithi018 *:9093,9094 running (30m) 6m ago 44m 15.5M - 0.25.0 c8568f914cd2 c332090068e8 2024-04-12T21:29:33.361 INFO:teuthology.orchestra.run.smithi018.stdout:crash.smithi018 smithi018 running (26m) 6m ago 44m 6665k - 19.0.0-2879-g324baf8d 8717720dc5e5 b7f80d1f7982 2024-04-12T21:29:33.361 INFO:teuthology.orchestra.run.smithi018.stdout:crash.smithi186 smithi186 running (26m) 6m ago 43m 6660k - 19.0.0-2879-g324baf8d 8717720dc5e5 a4f55c81b623 2024-04-12T21:29:33.361 INFO:teuthology.orchestra.run.smithi018.stdout:grafana.smithi018 smithi018 *:3000 running (29m) 6m ago 44m 74.8M - 9.4.12 2bacad6d85d8 b08e3bf176dd 2024-04-12T21:29:33.361 INFO:teuthology.orchestra.run.smithi018.stdout:mds.cephfs.smithi018.acqbsc smithi018 running (22m) 6m ago 37m 16.7M - 19.0.0-2879-g324baf8d 8717720dc5e5 021f99ab24ae 2024-04-12T21:29:33.361 INFO:teuthology.orchestra.run.smithi018.stdout:mds.cephfs.smithi018.szjluo smithi018 running (21m) 6m ago 37m 16.5M - 19.0.0-2879-g324baf8d 8717720dc5e5 0003117479d4 2024-04-12T21:29:33.361 INFO:teuthology.orchestra.run.smithi018.stdout:mds.cephfs.smithi186.jdllyn smithi186 running (6m) 6m ago 37m 11.0M - 19.0.0-2879-g324baf8d 8717720dc5e5 21b1f46eef23 2024-04-12T21:29:33.361 INFO:teuthology.orchestra.run.smithi018.stdout:mds.cephfs.smithi186.wjnafs smithi186 running (37m) 6m ago 37m 19.5M - 17.2.7-827-gd3b5060a 21de9b7369ad 74b115d6d6fd 2024-04-12T21:29:33.361 INFO:teuthology.orchestra.run.smithi018.stdout:mgr.smithi018.xjzwga smithi018 *:8443,9283,8765 running (32m) 6m ago 46m 512M - 19.0.0-2879-g324baf8d 8717720dc5e5 56088305eab7 2024-04-12T21:29:33.361 INFO:teuthology.orchestra.run.smithi018.stdout:mgr.smithi186.kqcfxk smithi186 *:8443,9283,8765 running (31m) 6m ago 43m 459M - 19.0.0-2879-g324baf8d 8717720dc5e5 09cd766e5e0d 2024-04-12T21:29:33.361 INFO:teuthology.orchestra.run.smithi018.stdout:mon.smithi018 smithi018 running (28m) 6m ago 46m 73.1M 2048M 19.0.0-2879-g324baf8d 8717720dc5e5 df3e201dfed6 2024-04-12T21:29:33.361 INFO:teuthology.orchestra.run.smithi018.stdout:mon.smithi186 smithi186 running (27m) 6m ago 43m 47.2M 2048M 19.0.0-2879-g324baf8d 8717720dc5e5 c77498da9b2b 2024-04-12T21:29:33.361 INFO:teuthology.orchestra.run.smithi018.stdout:node-exporter.smithi018 smithi018 *:9100 running (31m) 6m ago 44m 8108k - 1.5.0 0da6a335fe13 326c3d9c6847 2024-04-12T21:29:33.361 INFO:teuthology.orchestra.run.smithi018.stdout:node-exporter.smithi186 smithi186 *:9100 running (31m) 6m ago 43m 8757k - 1.5.0 0da6a335fe13 ec2112baeecf 2024-04-12T21:29:33.361 INFO:teuthology.orchestra.run.smithi018.stdout:osd.0 smithi018 running (25m) 6m ago 42m 123M 1559M 19.0.0-2879-g324baf8d 8717720dc5e5 60b1273ca047 2024-04-12T21:29:33.361 INFO:teuthology.orchestra.run.smithi018.stdout:osd.1 smithi018 running (25m) 6m ago 41m 95.1M 1559M 19.0.0-2879-g324baf8d 8717720dc5e5 170ce1e5fb8d 2024-04-12T21:29:33.361 INFO:teuthology.orchestra.run.smithi018.stdout:osd.2 smithi018 running (24m) 6m ago 41m 94.3M 1559M 19.0.0-2879-g324baf8d 8717720dc5e5 c2cbc6aa2839 2024-04-12T21:29:33.362 INFO:teuthology.orchestra.run.smithi018.stdout:osd.3 smithi186 running (24m) 6m ago 40m 101M 2581M 19.0.0-2879-g324baf8d 8717720dc5e5 0138e8262e4b 2024-04-12T21:29:33.362 INFO:teuthology.orchestra.run.smithi018.stdout:osd.4 smithi186 running (23m) 6m ago 39m 91.0M 2581M 19.0.0-2879-g324baf8d 8717720dc5e5 b20cb63b25c9 2024-04-12T21:29:33.362 INFO:teuthology.orchestra.run.smithi018.stdout:osd.5 smithi186 running (22m) 6m ago 39m 92.0M 2581M 19.0.0-2879-g324baf8d 8717720dc5e5 2513bea2610e 2024-04-12T21:29:33.362 INFO:teuthology.orchestra.run.smithi018.stdout:prometheus.smithi018 smithi018 *:9095 running (30m) 6m ago 43m 67.9M - 2.43.0 a07b618ecd1d f58841127e21 2024-04-12T21:29:33.363 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:29:33.364+0000 7f8caca00700 1 -- 172.21.15.18:0/2951051952 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f8c8c06ec00 msgr2=0x7f8c8c0710b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:29:33.363 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:29:33.364+0000 7f8caca00700 1 --2- 172.21.15.18:0/2951051952 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f8c8c06ec00 0x7f8c8c0710b0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f8c980041d0 tx=0x7f8c98009040 comp rx=0 tx=0).stop 2024-04-12T21:29:33.363 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:29:33.364+0000 7f8caca00700 1 -- 172.21.15.18:0/2951051952 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8ca8116350 msgr2=0x7f8ca810fd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:29:33.363 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:29:33.364+0000 7f8caca00700 1 --2- 172.21.15.18:0/2951051952 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8ca8116350 0x7f8ca810fd70 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7f8c9400b3b0 tx=0x7f8c94009e30 comp rx=0 tx=0).stop 2024-04-12T21:29:33.363 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:29:33.364+0000 7f8caca00700 1 -- 172.21.15.18:0/2951051952 shutdown_connections 2024-04-12T21:29:33.364 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:29:33.364+0000 7f8caca00700 1 --2- 172.21.15.18:0/2951051952 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f8c8c06ec00 0x7f8c8c0710b0 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:29:33.364 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:29:33.364+0000 7f8caca00700 1 --2- 172.21.15.18:0/2951051952 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8ca80fdc40 0x7f8ca8115e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:29:33.364 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:29:33.364+0000 7f8caca00700 1 --2- 172.21.15.18:0/2951051952 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8ca8116350 0x7f8ca810fd70 unknown :-1 s=CLOSED pgs=756 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:29:33.364 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:29:33.365+0000 7f8caca00700 1 -- 172.21.15.18:0/2951051952 >> 172.21.15.18:0/2951051952 conn(0x7f8ca80f7670 msgr2=0x7f8ca80fbd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:29:33.364 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:29:33.365+0000 7f8caca00700 1 -- 172.21.15.18:0/2951051952 shutdown_connections 2024-04-12T21:29:33.364 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:29:33.365+0000 7f8caca00700 1 -- 172.21.15.18:0/2951051952 wait complete. 2024-04-12T21:29:33.527 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:29:33.528+0000 7f0cdd600700 1 -- 172.21.15.18:0/2433775006 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0cd8105020 msgr2=0x7f0cd81053f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:29:33.527 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:29:33.528+0000 7f0cdd600700 1 --2- 172.21.15.18:0/2433775006 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0cd8105020 0x7f0cd81053f0 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7f0cc8004940 tx=0x7f0cc8009e30 comp rx=0 tx=0).stop 2024-04-12T21:29:33.527 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:29:33.528+0000 7f0cdd600700 1 -- 172.21.15.18:0/2433775006 shutdown_connections 2024-04-12T21:29:33.527 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:29:33.528+0000 7f0cdd600700 1 --2- 172.21.15.18:0/2433775006 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0cd81059c0 0x7f0cd80ff080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:29:33.527 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:29:33.528+0000 7f0cdd600700 1 --2- 172.21.15.18:0/2433775006 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0cd8105020 0x7f0cd81053f0 unknown :-1 s=CLOSED pgs=757 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:29:33.527 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:29:33.528+0000 7f0cdd600700 1 -- 172.21.15.18:0/2433775006 >> 172.21.15.18:0/2433775006 conn(0x7f0cd80faba0 msgr2=0x7f0cd80fcfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:29:33.527 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:29:33.528+0000 7f0cdd600700 1 -- 172.21.15.18:0/2433775006 shutdown_connections 2024-04-12T21:29:33.527 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:29:33.528+0000 7f0cdd600700 1 -- 172.21.15.18:0/2433775006 wait complete. 2024-04-12T21:29:33.529 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:29:33.529+0000 7f0cdd600700 1 Processor -- start 2024-04-12T21:29:33.529 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:29:33.529+0000 7f0cdd600700 1 -- start start 2024-04-12T21:29:33.529 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:29:33.530+0000 7f0cdd600700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0cd8105020 0x7f0cd8115bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:29:33.529 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:29:33.530+0000 7f0cdd600700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0cd81059c0 0x7f0cd8116100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:29:33.529 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:29:33.530+0000 7f0cdd600700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0cd8116750 con 0x7f0cd8105020 2024-04-12T21:29:33.529 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:29:33.530+0000 7f0cdd600700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0cd810fd30 con 0x7f0cd81059c0 2024-04-12T21:29:33.529 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:29:33.530+0000 7f0cd6a00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0cd8105020 0x7f0cd8115bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:29:33.529 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:29:33.530+0000 7f0cd6a00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0cd8105020 0x7f0cd8115bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:55878/0 (socket says 172.21.15.18:55878) 2024-04-12T21:29:33.529 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:29:33.530+0000 7f0cd6a00700 1 -- 172.21.15.18:0/627136014 learned_addr learned my addr 172.21.15.18:0/627136014 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:29:33.530 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:29:33.530+0000 7f0ccfe00700 1 --2- 172.21.15.18:0/627136014 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0cd81059c0 0x7f0cd8116100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:29:33.530 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:29:33.530+0000 7f0cd6a00700 1 -- 172.21.15.18:0/627136014 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0cd81059c0 msgr2=0x7f0cd8116100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:29:33.530 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:29:33.530+0000 7f0cd6a00700 1 --2- 172.21.15.18:0/627136014 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0cd81059c0 0x7f0cd8116100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:29:33.530 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:29:33.531+0000 7f0cd6a00700 1 -- 172.21.15.18:0/627136014 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0cc8005040 con 0x7f0cd8105020 2024-04-12T21:29:33.530 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:29:33.531+0000 7f0cd6a00700 1 --2- 172.21.15.18:0/627136014 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0cd8105020 0x7f0cd8115bc0 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7f0cc8009180 tx=0x7f0cc8009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:29:33.531 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:29:33.531+0000 7f0ccea00700 1 -- 172.21.15.18:0/627136014 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0cc800e040 con 0x7f0cd8105020 2024-04-12T21:29:33.531 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:29:33.531+0000 7f0cdd600700 1 -- 172.21.15.18:0/627136014 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0cd810ffb0 con 0x7f0cd8105020 2024-04-12T21:29:33.531 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:29:33.531+0000 7f0ccea00700 1 -- 172.21.15.18:0/627136014 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f0cc800bbc0 con 0x7f0cd8105020 2024-04-12T21:29:33.531 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:29:33.531+0000 7f0ccea00700 1 -- 172.21.15.18:0/627136014 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0cc801c630 con 0x7f0cd8105020 2024-04-12T21:29:33.531 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:29:33.531+0000 7f0cdd600700 1 -- 172.21.15.18:0/627136014 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0cd8110570 con 0x7f0cd8105020 2024-04-12T21:29:33.531 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:29:33.533+0000 7f0cdd600700 1 -- 172.21.15.18:0/627136014 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0cd8100770 con 0x7f0cd8105020 2024-04-12T21:29:33.532 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:29:33.534+0000 7f0ccea00700 1 -- 172.21.15.18:0/627136014 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7f0cc800bd30 con 0x7f0cd8105020 2024-04-12T21:29:33.534 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:29:33.535+0000 7f0ccea00700 1 --2- 172.21.15.18:0/627136014 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f0cbc06ec00 0x7f0cbc0710b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:29:33.534 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:29:33.536+0000 7f0ccea00700 1 -- 172.21.15.18:0/627136014 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 6757+0+0 (secure 0 0 0) 0x7f0cc808fe20 con 0x7f0cd8105020 2024-04-12T21:29:33.535 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:29:33.536+0000 7f0ccfe00700 1 --2- 172.21.15.18:0/627136014 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f0cbc06ec00 0x7f0cbc0710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:29:33.535 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:29:33.536+0000 7f0ccfe00700 1 --2- 172.21.15.18:0/627136014 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f0cbc06ec00 0x7f0cbc0710b0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f0cc0009fd0 tx=0x7f0cc0009f20 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:29:33.539 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:29:33.540+0000 7f0ccea00700 1 -- 172.21.15.18:0/627136014 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f0cc805c5d0 con 0x7f0cd8105020 2024-04-12T21:29:33.881 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:29:33.882+0000 7f0cdd600700 1 -- 172.21.15.18:0/627136014 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f0cd8111210 con 0x7f0cd8105020 2024-04-12T21:29:33.882 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:29:33.883+0000 7f0ccea00700 1 -- 172.21.15.18:0/627136014 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+837 (secure 0 0 0) 0x7f0cc805c160 con 0x7f0cd8105020 2024-04-12T21:29:33.882 INFO:teuthology.orchestra.run.smithi018.stdout:{ 2024-04-12T21:29:33.882 INFO:teuthology.orchestra.run.smithi018.stdout: "mon": { 2024-04-12T21:29:33.882 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 19.0.0-2879-g324baf8d (324baf8ddb91dea46cac50cb535c306115791355) squid (dev)": 2 2024-04-12T21:29:33.882 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-12T21:29:33.882 INFO:teuthology.orchestra.run.smithi018.stdout: "mgr": { 2024-04-12T21:29:33.882 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 19.0.0-2879-g324baf8d (324baf8ddb91dea46cac50cb535c306115791355) squid (dev)": 2 2024-04-12T21:29:33.883 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-12T21:29:33.883 INFO:teuthology.orchestra.run.smithi018.stdout: "osd": { 2024-04-12T21:29:33.883 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 19.0.0-2879-g324baf8d (324baf8ddb91dea46cac50cb535c306115791355) squid (dev)": 6 2024-04-12T21:29:33.883 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-12T21:29:33.883 INFO:teuthology.orchestra.run.smithi018.stdout: "mds": { 2024-04-12T21:29:33.883 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 17.2.7-827-gd3b5060a (d3b5060a17449edfae9385cdf792afec109fca5d) quincy (stable)": 1, 2024-04-12T21:29:33.883 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 19.0.0-2879-g324baf8d (324baf8ddb91dea46cac50cb535c306115791355) squid (dev)": 3 2024-04-12T21:29:33.883 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-12T21:29:33.883 INFO:teuthology.orchestra.run.smithi018.stdout: "overall": { 2024-04-12T21:29:33.883 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 17.2.7-827-gd3b5060a (d3b5060a17449edfae9385cdf792afec109fca5d) quincy (stable)": 1, 2024-04-12T21:29:33.883 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 19.0.0-2879-g324baf8d (324baf8ddb91dea46cac50cb535c306115791355) squid (dev)": 13 2024-04-12T21:29:33.883 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-04-12T21:29:33.883 INFO:teuthology.orchestra.run.smithi018.stdout:} 2024-04-12T21:29:33.885 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:29:33.886+0000 7f0cdd600700 1 -- 172.21.15.18:0/627136014 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f0cbc06ec00 msgr2=0x7f0cbc0710b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:29:33.885 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:29:33.886+0000 7f0cdd600700 1 --2- 172.21.15.18:0/627136014 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f0cbc06ec00 0x7f0cbc0710b0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f0cc0009fd0 tx=0x7f0cc0009f20 comp rx=0 tx=0).stop 2024-04-12T21:29:33.886 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:29:33.886+0000 7f0cdd600700 1 -- 172.21.15.18:0/627136014 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0cd8105020 msgr2=0x7f0cd8115bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:29:33.886 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:29:33.886+0000 7f0cdd600700 1 --2- 172.21.15.18:0/627136014 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0cd8105020 0x7f0cd8115bc0 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7f0cc8009180 tx=0x7f0cc8009e30 comp rx=0 tx=0).stop 2024-04-12T21:29:33.886 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:29:33.887+0000 7f0cdd600700 1 -- 172.21.15.18:0/627136014 shutdown_connections 2024-04-12T21:29:33.886 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:29:33.887+0000 7f0cdd600700 1 --2- 172.21.15.18:0/627136014 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f0cbc06ec00 0x7f0cbc0710b0 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:29:33.886 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:29:33.887+0000 7f0cdd600700 1 --2- 172.21.15.18:0/627136014 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0cd8105020 0x7f0cd8115bc0 unknown :-1 s=CLOSED pgs=758 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:29:33.886 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:29:33.887+0000 7f0cdd600700 1 --2- 172.21.15.18:0/627136014 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0cd81059c0 0x7f0cd8116100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:29:33.886 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:29:33.887+0000 7f0cdd600700 1 -- 172.21.15.18:0/627136014 >> 172.21.15.18:0/627136014 conn(0x7f0cd80faba0 msgr2=0x7f0cd80fbba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:29:33.886 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:29:33.887+0000 7f0cdd600700 1 -- 172.21.15.18:0/627136014 shutdown_connections 2024-04-12T21:29:33.886 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:29:33.887+0000 7f0cdd600700 1 -- 172.21.15.18:0/627136014 wait complete. 2024-04-12T21:29:34.029 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:29:33 smithi186 ceph-mon[86379]: from='client.35568 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:29:34.046 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:29:34.047+0000 7f969ce00700 1 -- 172.21.15.18:0/3565693025 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f96980708a0 msgr2=0x7f9698070d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:29:34.046 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:29:34.047+0000 7f969ce00700 1 --2- 172.21.15.18:0/3565693025 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f96980708a0 0x7f9698070d10 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f9684004940 tx=0x7f9684009e30 comp rx=0 tx=0).stop 2024-04-12T21:29:34.046 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:29:34.047+0000 7f969ce00700 1 -- 172.21.15.18:0/3565693025 shutdown_connections 2024-04-12T21:29:34.046 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:29:34.047+0000 7f969ce00700 1 --2- 172.21.15.18:0/3565693025 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f96980708a0 0x7f9698070d10 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:29:34.046 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:29:34.047+0000 7f969ce00700 1 --2- 172.21.15.18:0/3565693025 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9698071f90 0x7f9698070360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:29:34.046 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:29:34.047+0000 7f969ce00700 1 -- 172.21.15.18:0/3565693025 >> 172.21.15.18:0/3565693025 conn(0x7f96980f8760 msgr2=0x7f96980fab70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:29:34.046 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:29:34.047+0000 7f969ce00700 1 -- 172.21.15.18:0/3565693025 shutdown_connections 2024-04-12T21:29:34.046 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:29:34.047+0000 7f969ce00700 1 -- 172.21.15.18:0/3565693025 wait complete. 2024-04-12T21:29:34.047 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:29:34.048+0000 7f969ce00700 1 Processor -- start 2024-04-12T21:29:34.048 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:29:34.048+0000 7f969ce00700 1 -- start start 2024-04-12T21:29:34.048 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:29:34.049+0000 7f969ce00700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f96980708a0 0x7f9698119fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:29:34.048 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:29:34.049+0000 7f969ce00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9698071f90 0x7f969811a4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:29:34.048 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:29:34.049+0000 7f969ce00700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f969811ab30 con 0x7f9698071f90 2024-04-12T21:29:34.048 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:29:34.049+0000 7f969ce00700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9698113f00 con 0x7f96980708a0 2024-04-12T21:29:34.048 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:29:34.049+0000 7f9696a00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9698071f90 0x7f969811a4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:29:34.048 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:29:34.049+0000 7f9696a00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9698071f90 0x7f969811a4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:55894/0 (socket says 172.21.15.18:55894) 2024-04-12T21:29:34.048 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:29:34.049+0000 7f9696a00700 1 -- 172.21.15.18:0/1438014835 learned_addr learned my addr 172.21.15.18:0/1438014835 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:29:34.048 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:29:34.049+0000 7f9697400700 1 --2- 172.21.15.18:0/1438014835 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f96980708a0 0x7f9698119fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:29:34.049 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:29:34.050+0000 7f9696a00700 1 -- 172.21.15.18:0/1438014835 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f96980708a0 msgr2=0x7f9698119fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:29:34.049 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:29:34.050+0000 7f9696a00700 1 --2- 172.21.15.18:0/1438014835 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f96980708a0 0x7f9698119fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:29:34.049 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:29:34.050+0000 7f9696a00700 1 -- 172.21.15.18:0/1438014835 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9684005040 con 0x7f9698071f90 2024-04-12T21:29:34.049 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:29:34.050+0000 7f9696a00700 1 --2- 172.21.15.18:0/1438014835 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9698071f90 0x7f969811a4e0 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7f9684009fd0 tx=0x7f9684009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:29:34.049 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:29:34.050+0000 7f968f400700 1 -- 172.21.15.18:0/1438014835 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f968400e070 con 0x7f9698071f90 2024-04-12T21:29:34.049 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:29:34.050+0000 7f968f400700 1 -- 172.21.15.18:0/1438014835 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f9684013c00 con 0x7f9698071f90 2024-04-12T21:29:34.049 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:29:34.050+0000 7f968f400700 1 -- 172.21.15.18:0/1438014835 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f968400b630 con 0x7f9698071f90 2024-04-12T21:29:34.050 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:29:34.050+0000 7f969ce00700 1 -- 172.21.15.18:0/1438014835 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9698114180 con 0x7f9698071f90 2024-04-12T21:29:34.050 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:29:34.051+0000 7f969ce00700 1 -- 172.21.15.18:0/1438014835 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9698114660 con 0x7f9698071f90 2024-04-12T21:29:34.050 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:29:34.052+0000 7f969ce00700 1 -- 172.21.15.18:0/1438014835 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9698071aa0 con 0x7f9698071f90 2024-04-12T21:29:34.051 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:29:34.053+0000 7f968f400700 1 -- 172.21.15.18:0/1438014835 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7f968401c400 con 0x7f9698071f90 2024-04-12T21:29:34.052 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:29:34.054+0000 7f968f400700 1 --2- 172.21.15.18:0/1438014835 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f967c06ec00 0x7f967c0710b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:29:34.053 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:29:34.054+0000 7f968f400700 1 -- 172.21.15.18:0/1438014835 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 6757+0+0 (secure 0 0 0) 0x7f9684090390 con 0x7f9698071f90 2024-04-12T21:29:34.053 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:29:34.054+0000 7f9697400700 1 --2- 172.21.15.18:0/1438014835 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f967c06ec00 0x7f967c0710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:29:34.053 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:29:34.055+0000 7f9697400700 1 --2- 172.21.15.18:0/1438014835 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f967c06ec00 0x7f967c0710b0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f9688009b80 tx=0x7f968800b040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:29:34.057 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:29:34.059+0000 7f968f400700 1 -- 172.21.15.18:0/1438014835 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f968405cbe0 con 0x7f9698071f90 2024-04-12T21:29:34.102 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:29:33 smithi018 ceph-mon[112590]: from='client.35568 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:29:34.339 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:29:34.340+0000 7f969ce00700 1 -- 172.21.15.18:0/1438014835 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f969804bed0 con 0x7f9698071f90 2024-04-12T21:29:34.340 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:29:34.341+0000 7f968f400700 1 -- 172.21.15.18:0/1438014835 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 31 v31) v1 ==== 76+0+1873 (secure 0 0 0) 0x7f968405c770 con 0x7f9698071f90 2024-04-12T21:29:34.341 INFO:teuthology.orchestra.run.smithi018.stdout:e31 2024-04-12T21:29:34.341 INFO:teuthology.orchestra.run.smithi018.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-12T21:29:34.341 INFO:teuthology.orchestra.run.smithi018.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-12T21:29:34.341 INFO:teuthology.orchestra.run.smithi018.stdout:legacy client fscid: 1 2024-04-12T21:29:34.341 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-12T21:29:34.341 INFO:teuthology.orchestra.run.smithi018.stdout:Filesystem 'cephfs' (1) 2024-04-12T21:29:34.341 INFO:teuthology.orchestra.run.smithi018.stdout:fs_name cephfs 2024-04-12T21:29:34.341 INFO:teuthology.orchestra.run.smithi018.stdout:epoch 24 2024-04-12T21:29:34.341 INFO:teuthology.orchestra.run.smithi018.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-04-12T21:29:34.341 INFO:teuthology.orchestra.run.smithi018.stdout:created 2024-04-12T20:51:57.362850+0000 2024-04-12T21:29:34.341 INFO:teuthology.orchestra.run.smithi018.stdout:modified 2024-04-12T21:06:59.717190+0000 2024-04-12T21:29:34.341 INFO:teuthology.orchestra.run.smithi018.stdout:tableserver 0 2024-04-12T21:29:34.342 INFO:teuthology.orchestra.run.smithi018.stdout:root 0 2024-04-12T21:29:34.342 INFO:teuthology.orchestra.run.smithi018.stdout:session_timeout 60 2024-04-12T21:29:34.342 INFO:teuthology.orchestra.run.smithi018.stdout:session_autoclose 300 2024-04-12T21:29:34.342 INFO:teuthology.orchestra.run.smithi018.stdout:max_file_size 1099511627776 2024-04-12T21:29:34.342 INFO:teuthology.orchestra.run.smithi018.stdout:max_xattr_size 65536 2024-04-12T21:29:34.342 INFO:teuthology.orchestra.run.smithi018.stdout:required_client_features {} 2024-04-12T21:29:34.342 INFO:teuthology.orchestra.run.smithi018.stdout:last_failure 0 2024-04-12T21:29:34.342 INFO:teuthology.orchestra.run.smithi018.stdout:last_failure_osd_epoch 100 2024-04-12T21:29:34.342 INFO:teuthology.orchestra.run.smithi018.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-12T21:29:34.342 INFO:teuthology.orchestra.run.smithi018.stdout:max_mds 2 2024-04-12T21:29:34.342 INFO:teuthology.orchestra.run.smithi018.stdout:in 0,1 2024-04-12T21:29:34.342 INFO:teuthology.orchestra.run.smithi018.stdout:up {} 2024-04-12T21:29:34.342 INFO:teuthology.orchestra.run.smithi018.stdout:failed 0,1 2024-04-12T21:29:34.343 INFO:teuthology.orchestra.run.smithi018.stdout:damaged 2024-04-12T21:29:34.343 INFO:teuthology.orchestra.run.smithi018.stdout:stopped 2024-04-12T21:29:34.343 INFO:teuthology.orchestra.run.smithi018.stdout:data_pools [3] 2024-04-12T21:29:34.343 INFO:teuthology.orchestra.run.smithi018.stdout:metadata_pool 2 2024-04-12T21:29:34.343 INFO:teuthology.orchestra.run.smithi018.stdout:inline_data disabled 2024-04-12T21:29:34.343 INFO:teuthology.orchestra.run.smithi018.stdout:balancer 2024-04-12T21:29:34.343 INFO:teuthology.orchestra.run.smithi018.stdout:bal_rank_mask -1 2024-04-12T21:29:34.343 INFO:teuthology.orchestra.run.smithi018.stdout:standby_count_wanted 1 2024-04-12T21:29:34.343 INFO:teuthology.orchestra.run.smithi018.stdout:qdb_cluster leader: 0 members: 2024-04-12T21:29:34.343 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-12T21:29:34.343 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-12T21:29:34.343 INFO:teuthology.orchestra.run.smithi018.stdout:Standby daemons: 2024-04-12T21:29:34.343 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-12T21:29:34.343 INFO:teuthology.orchestra.run.smithi018.stdout:[mds.cephfs.smithi186.wjnafs{-1:24277} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.186:6826/438347207,v1:172.21.15.186:6827/438347207] compat {c=[1],r=[1],i=[7ff]}] 2024-04-12T21:29:34.343 INFO:teuthology.orchestra.run.smithi018.stdout:[mds.cephfs.smithi018.acqbsc{-1:34436} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.18:6828/755179764,v1:172.21.15.18:6829/755179764] compat {c=[1],r=[1],i=[1fff]}] 2024-04-12T21:29:34.344 INFO:teuthology.orchestra.run.smithi018.stdout:[mds.cephfs.smithi018.szjluo{-1:34468} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.18:6826/2407992653,v1:172.21.15.18:6827/2407992653] compat {c=[1],r=[1],i=[1fff]}] 2024-04-12T21:29:34.344 INFO:teuthology.orchestra.run.smithi018.stdout:[mds.cephfs.smithi186.jdllyn{-1:44595} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.186:6824/3523444202,v1:172.21.15.186:6825/3523444202] compat {c=[1],r=[1],i=[1fff]}] 2024-04-12T21:29:34.344 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:29:34.344+0000 7f969ce00700 1 -- 172.21.15.18:0/1438014835 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f967c06ec00 msgr2=0x7f967c0710b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:29:34.344 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:29:34.345+0000 7f969ce00700 1 --2- 172.21.15.18:0/1438014835 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f967c06ec00 0x7f967c0710b0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f9688009b80 tx=0x7f968800b040 comp rx=0 tx=0).stop 2024-04-12T21:29:34.345 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:29:34.345+0000 7f969ce00700 1 -- 172.21.15.18:0/1438014835 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9698071f90 msgr2=0x7f969811a4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:29:34.345 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:29:34.345+0000 7f969ce00700 1 --2- 172.21.15.18:0/1438014835 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9698071f90 0x7f969811a4e0 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7f9684009fd0 tx=0x7f9684009e30 comp rx=0 tx=0).stop 2024-04-12T21:29:34.345 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:29:34.345+0000 7f969ce00700 1 -- 172.21.15.18:0/1438014835 shutdown_connections 2024-04-12T21:29:34.345 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:29:34.345+0000 7f969ce00700 1 --2- 172.21.15.18:0/1438014835 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f967c06ec00 0x7f967c0710b0 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:29:34.345 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:29:34.345+0000 7f969ce00700 1 --2- 172.21.15.18:0/1438014835 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f96980708a0 0x7f9698119fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:29:34.345 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:29:34.345+0000 7f969ce00700 1 --2- 172.21.15.18:0/1438014835 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9698071f90 0x7f969811a4e0 unknown :-1 s=CLOSED pgs=759 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:29:34.345 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:29:34.345+0000 7f969ce00700 1 -- 172.21.15.18:0/1438014835 >> 172.21.15.18:0/1438014835 conn(0x7f96980f8760 msgr2=0x7f96980f9850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:29:34.345 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:29:34.345+0000 7f969ce00700 1 -- 172.21.15.18:0/1438014835 shutdown_connections 2024-04-12T21:29:34.345 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:29:34.345+0000 7f969ce00700 1 -- 172.21.15.18:0/1438014835 wait complete. 2024-04-12T21:29:34.345 INFO:teuthology.orchestra.run.smithi018.stderr:dumped fsmap epoch 31 2024-04-12T21:29:34.516 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:29:34.517+0000 7fba14c00700 1 -- 172.21.15.18:0/2984194410 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fba100fdde0 msgr2=0x7fba100fe1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:29:34.516 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:29:34.517+0000 7fba14c00700 1 --2- 172.21.15.18:0/2984194410 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fba100fdde0 0x7fba100fe1b0 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7fba00004940 tx=0x7fba00009e30 comp rx=0 tx=0).stop 2024-04-12T21:29:34.516 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:29:34.517+0000 7fba14c00700 1 -- 172.21.15.18:0/2984194410 shutdown_connections 2024-04-12T21:29:34.516 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:29:34.517+0000 7fba14c00700 1 --2- 172.21.15.18:0/2984194410 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fba100fe780 0x7fba10106dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:29:34.516 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:29:34.517+0000 7fba14c00700 1 --2- 172.21.15.18:0/2984194410 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fba100fdde0 0x7fba100fe1b0 unknown :-1 s=CLOSED pgs=760 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:29:34.516 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:29:34.517+0000 7fba14c00700 1 -- 172.21.15.18:0/2984194410 >> 172.21.15.18:0/2984194410 conn(0x7fba100f7890 msgr2=0x7fba100f9ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:29:34.516 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:29:34.517+0000 7fba14c00700 1 -- 172.21.15.18:0/2984194410 shutdown_connections 2024-04-12T21:29:34.516 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:29:34.517+0000 7fba14c00700 1 -- 172.21.15.18:0/2984194410 wait complete. 2024-04-12T21:29:34.517 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:29:34.518+0000 7fba14c00700 1 Processor -- start 2024-04-12T21:29:34.517 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:29:34.519+0000 7fba14c00700 1 -- start start 2024-04-12T21:29:34.518 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:29:34.519+0000 7fba14c00700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fba100fdde0 0x7fba10114a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:29:34.518 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:29:34.519+0000 7fba14c00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fba100fe780 0x7fba10114fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:29:34.518 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:29:34.519+0000 7fba14c00700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba101155c0 con 0x7fba100fe780 2024-04-12T21:29:34.518 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:29:34.519+0000 7fba14c00700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba1010fda0 con 0x7fba100fdde0 2024-04-12T21:29:34.519 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:29:34.520+0000 7fba0d400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fba100fe780 0x7fba10114fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:29:34.519 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:29:34.520+0000 7fba0d400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fba100fe780 0x7fba10114fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:55906/0 (socket says 172.21.15.18:55906) 2024-04-12T21:29:34.519 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:29:34.520+0000 7fba0d400700 1 -- 172.21.15.18:0/845788844 learned_addr learned my addr 172.21.15.18:0/845788844 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:29:34.519 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:29:34.520+0000 7fba0de00700 1 --2- 172.21.15.18:0/845788844 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fba100fdde0 0x7fba10114a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:29:34.519 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:29:34.520+0000 7fba0d400700 1 -- 172.21.15.18:0/845788844 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fba100fdde0 msgr2=0x7fba10114a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:29:34.519 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:29:34.520+0000 7fba0d400700 1 --2- 172.21.15.18:0/845788844 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fba100fdde0 0x7fba10114a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:29:34.519 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:29:34.520+0000 7fba0d400700 1 -- 172.21.15.18:0/845788844 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba00005040 con 0x7fba100fe780 2024-04-12T21:29:34.519 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:29:34.520+0000 7fba0d400700 1 --2- 172.21.15.18:0/845788844 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fba100fe780 0x7fba10114fc0 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7fb9fc002a50 tx=0x7fb9fc0046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:29:34.520 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:29:34.521+0000 7fba06000700 1 -- 172.21.15.18:0/845788844 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb9fc007cc0 con 0x7fba100fe780 2024-04-12T21:29:34.520 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:29:34.521+0000 7fba06000700 1 -- 172.21.15.18:0/845788844 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb9fc00f040 con 0x7fba100fe780 2024-04-12T21:29:34.520 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:29:34.521+0000 7fba14c00700 1 -- 172.21.15.18:0/845788844 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fba10110080 con 0x7fba100fe780 2024-04-12T21:29:34.520 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:29:34.521+0000 7fba06000700 1 -- 172.21.15.18:0/845788844 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb9fc009640 con 0x7fba100fe780 2024-04-12T21:29:34.520 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:29:34.521+0000 7fba14c00700 1 -- 172.21.15.18:0/845788844 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fba101105c0 con 0x7fba100fe780 2024-04-12T21:29:34.521 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:29:34.522+0000 7fba14c00700 1 -- 172.21.15.18:0/845788844 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fba100fd1f0 con 0x7fba100fe780 2024-04-12T21:29:34.521 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:29:34.523+0000 7fba06000700 1 -- 172.21.15.18:0/845788844 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7fb9fc0097a0 con 0x7fba100fe780 2024-04-12T21:29:34.523 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:29:34.525+0000 7fba06000700 1 --2- 172.21.15.18:0/845788844 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fb9f406ec00 0x7fb9f40710b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:29:34.523 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:29:34.525+0000 7fba06000700 1 -- 172.21.15.18:0/845788844 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 6757+0+0 (secure 0 0 0) 0x7fb9fc08e760 con 0x7fba100fe780 2024-04-12T21:29:34.523 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:29:34.525+0000 7fba0de00700 1 --2- 172.21.15.18:0/845788844 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fb9f406ec00 0x7fb9f40710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:29:34.524 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:29:34.526+0000 7fba0de00700 1 --2- 172.21.15.18:0/845788844 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fb9f406ec00 0x7fb9f40710b0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7fba00009fd0 tx=0x7fba00009e30 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:29:34.528 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:29:34.530+0000 7fba06000700 1 -- 172.21.15.18:0/845788844 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7fb9fc0573e0 con 0x7fba100fe780 2024-04-12T21:29:34.782 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:29:34.783+0000 7fba14c00700 1 -- 172.21.15.18:0/845788844 --> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fba10109e70 con 0x7fb9f406ec00 2024-04-12T21:29:34.784 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:29:34.785+0000 7fba06000700 1 -- 172.21.15.18:0/845788844 <== mgr.34102 v2:172.21.15.18:6800/3812891919 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7fba10109e70 con 0x7fb9f406ec00 2024-04-12T21:29:34.784 INFO:teuthology.orchestra.run.smithi018.stdout:{ 2024-04-12T21:29:34.785 INFO:teuthology.orchestra.run.smithi018.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:f4452def3be4d350f4a8e5feaf7ebb1511be9298fe68478ba7a2d6fbd7d69a63", 2024-04-12T21:29:34.785 INFO:teuthology.orchestra.run.smithi018.stdout: "in_progress": true, 2024-04-12T21:29:34.785 INFO:teuthology.orchestra.run.smithi018.stdout: "which": "Upgrading all daemon types on all hosts", 2024-04-12T21:29:34.785 INFO:teuthology.orchestra.run.smithi018.stdout: "services_complete": [ 2024-04-12T21:29:34.785 INFO:teuthology.orchestra.run.smithi018.stdout: "crash", 2024-04-12T21:29:34.785 INFO:teuthology.orchestra.run.smithi018.stdout: "osd", 2024-04-12T21:29:34.785 INFO:teuthology.orchestra.run.smithi018.stdout: "mon", 2024-04-12T21:29:34.785 INFO:teuthology.orchestra.run.smithi018.stdout: "mgr" 2024-04-12T21:29:34.785 INFO:teuthology.orchestra.run.smithi018.stdout: ], 2024-04-12T21:29:34.785 INFO:teuthology.orchestra.run.smithi018.stdout: "progress": "15/21 daemons upgraded", 2024-04-12T21:29:34.785 INFO:teuthology.orchestra.run.smithi018.stdout: "message": "Currently upgrading mds daemons", 2024-04-12T21:29:34.785 INFO:teuthology.orchestra.run.smithi018.stdout: "is_paused": false 2024-04-12T21:29:34.785 INFO:teuthology.orchestra.run.smithi018.stdout:} 2024-04-12T21:29:34.787 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:29:34.788+0000 7fba14c00700 1 -- 172.21.15.18:0/845788844 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fb9f406ec00 msgr2=0x7fb9f40710b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:29:34.787 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:29:34.788+0000 7fba14c00700 1 --2- 172.21.15.18:0/845788844 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fb9f406ec00 0x7fb9f40710b0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7fba00009fd0 tx=0x7fba00009e30 comp rx=0 tx=0).stop 2024-04-12T21:29:34.787 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:29:34.788+0000 7fba14c00700 1 -- 172.21.15.18:0/845788844 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fba100fe780 msgr2=0x7fba10114fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:29:34.787 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:29:34.788+0000 7fba14c00700 1 --2- 172.21.15.18:0/845788844 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fba100fe780 0x7fba10114fc0 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7fb9fc002a50 tx=0x7fb9fc0046b0 comp rx=0 tx=0).stop 2024-04-12T21:29:34.787 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:29:34.788+0000 7fba14c00700 1 -- 172.21.15.18:0/845788844 shutdown_connections 2024-04-12T21:29:34.788 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:29:34.788+0000 7fba14c00700 1 --2- 172.21.15.18:0/845788844 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fb9f406ec00 0x7fb9f40710b0 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:29:34.788 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:29:34.788+0000 7fba14c00700 1 --2- 172.21.15.18:0/845788844 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fba100fdde0 0x7fba10114a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:29:34.788 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:29:34.788+0000 7fba14c00700 1 --2- 172.21.15.18:0/845788844 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fba100fe780 0x7fba10114fc0 unknown :-1 s=CLOSED pgs=761 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:29:34.788 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:29:34.788+0000 7fba14c00700 1 -- 172.21.15.18:0/845788844 >> 172.21.15.18:0/845788844 conn(0x7fba100f7890 msgr2=0x7fba100fbdd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:29:34.788 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:29:34.789+0000 7fba14c00700 1 -- 172.21.15.18:0/845788844 shutdown_connections 2024-04-12T21:29:34.788 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:29:34.789+0000 7fba14c00700 1 -- 172.21.15.18:0/845788844 wait complete. 2024-04-12T21:29:34.943 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:29:34.945+0000 7f94a1800700 1 -- 172.21.15.18:0/1041290050 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f949c105000 msgr2=0x7f949c1053d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:29:34.944 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:29:34.945+0000 7f94a1800700 1 --2- 172.21.15.18:0/1041290050 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f949c105000 0x7f949c1053d0 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7f948c004940 tx=0x7f948c009e30 comp rx=0 tx=0).stop 2024-04-12T21:29:34.944 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:29:34.945+0000 7f94a1800700 1 -- 172.21.15.18:0/1041290050 shutdown_connections 2024-04-12T21:29:34.944 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:29:34.945+0000 7f94a1800700 1 --2- 172.21.15.18:0/1041290050 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f949c1059a0 0x7f949c0ff010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:29:34.944 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:29:34.945+0000 7f94a1800700 1 --2- 172.21.15.18:0/1041290050 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f949c105000 0x7f949c1053d0 unknown :-1 s=CLOSED pgs=762 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:29:34.944 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:29:34.945+0000 7f94a1800700 1 -- 172.21.15.18:0/1041290050 >> 172.21.15.18:0/1041290050 conn(0x7f949c0fab70 msgr2=0x7f949c0fcf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:29:34.944 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:29:34.945+0000 7f94a1800700 1 -- 172.21.15.18:0/1041290050 shutdown_connections 2024-04-12T21:29:34.944 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:29:34.945+0000 7f94a1800700 1 -- 172.21.15.18:0/1041290050 wait complete. 2024-04-12T21:29:34.945 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:29:34.946+0000 7f94a1800700 1 Processor -- start 2024-04-12T21:29:34.945 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:29:34.946+0000 7f94a1800700 1 -- start start 2024-04-12T21:29:34.945 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:29:34.946+0000 7f94a1800700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f949c105000 0x7f949c115b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:29:34.945 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:29:34.946+0000 7f94a1800700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f949c1059a0 0x7f949c1160b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:29:34.945 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:29:34.946+0000 7f94a1800700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f949c116700 con 0x7f949c105000 2024-04-12T21:29:34.945 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:29:34.946+0000 7f94a1800700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f949c10fce0 con 0x7f949c1059a0 2024-04-12T21:29:34.945 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:29:34.947+0000 7f949aa00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f949c105000 0x7f949c115b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:29:34.945 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:29:34.947+0000 7f949aa00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f949c105000 0x7f949c115b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:55920/0 (socket says 172.21.15.18:55920) 2024-04-12T21:29:34.945 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:29:34.947+0000 7f949aa00700 1 -- 172.21.15.18:0/4092809793 learned_addr learned my addr 172.21.15.18:0/4092809793 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:29:34.946 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:29:34.947+0000 7f949a000700 1 --2- 172.21.15.18:0/4092809793 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f949c1059a0 0x7f949c1160b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:29:34.947 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:29:34.947+0000 7f949aa00700 1 -- 172.21.15.18:0/4092809793 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f949c1059a0 msgr2=0x7f949c1160b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:29:34.947 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:29:34.947+0000 7f949aa00700 1 --2- 172.21.15.18:0/4092809793 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f949c1059a0 0x7f949c1160b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:29:34.947 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:29:34.947+0000 7f949aa00700 1 -- 172.21.15.18:0/4092809793 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f948c005040 con 0x7f949c105000 2024-04-12T21:29:34.947 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:29:34.947+0000 7f949aa00700 1 --2- 172.21.15.18:0/4092809793 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f949c105000 0x7f949c115b70 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7f948c009150 tx=0x7f948c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:29:34.947 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:29:34.948+0000 7f9492a00700 1 -- 172.21.15.18:0/4092809793 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f948c00e040 con 0x7f949c105000 2024-04-12T21:29:34.947 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:29:34.948+0000 7f9492a00700 1 -- 172.21.15.18:0/4092809793 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f948c013420 con 0x7f949c105000 2024-04-12T21:29:34.947 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:29:34.948+0000 7f9492a00700 1 -- 172.21.15.18:0/4092809793 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f948c01c630 con 0x7f949c105000 2024-04-12T21:29:34.947 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:29:34.948+0000 7f94a1800700 1 -- 172.21.15.18:0/4092809793 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f949c10ff60 con 0x7f949c105000 2024-04-12T21:29:34.947 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:29:34.948+0000 7f94a1800700 1 -- 172.21.15.18:0/4092809793 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f949c110520 con 0x7f949c105000 2024-04-12T21:29:34.947 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:29:34.949+0000 7f94a1800700 1 -- 172.21.15.18:0/4092809793 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f949c100750 con 0x7f949c105000 2024-04-12T21:29:34.949 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:29:34.950+0000 7f9492a00700 1 -- 172.21.15.18:0/4092809793 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7f948c013590 con 0x7f949c105000 2024-04-12T21:29:34.950 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:29:34.951+0000 7f9492a00700 1 --2- 172.21.15.18:0/4092809793 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f948006ec00 0x7f94800710b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:29:34.950 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:29:34.951+0000 7f9492a00700 1 -- 172.21.15.18:0/4092809793 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 6757+0+0 (secure 0 0 0) 0x7f948c08fce0 con 0x7f949c105000 2024-04-12T21:29:34.951 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:29:34.952+0000 7f949a000700 1 --2- 172.21.15.18:0/4092809793 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f948006ec00 0x7f94800710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:29:34.951 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:29:34.952+0000 7f949a000700 1 --2- 172.21.15.18:0/4092809793 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f948006ec00 0x7f94800710b0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f949c111410 tx=0x7f948800b040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:29:34.954 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:29:34.956+0000 7f9492a00700 1 -- 172.21.15.18:0/4092809793 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f948c05c490 con 0x7f949c105000 2024-04-12T21:29:35.102 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:29:34 smithi018 ceph-mon[112590]: from='client.35572 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:29:35.102 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:29:34 smithi018 ceph-mon[112590]: from='client.35576 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:29:35.102 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:29:34 smithi018 ceph-mon[112590]: pgmap v894: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:29:35.102 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:29:34 smithi018 ceph-mon[112590]: from='client.? 172.21.15.18:0/627136014' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T21:29:35.102 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:29:34 smithi018 ceph-mon[112590]: from='client.? 172.21.15.18:0/1438014835' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-12T21:29:35.272 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:29:35.273+0000 7f94a1800700 1 -- 172.21.15.18:0/4092809793 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f949c04c710 con 0x7f949c105000 2024-04-12T21:29:35.273 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:29:35.274+0000 7f9492a00700 1 -- 172.21.15.18:0/4092809793 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+4303 (secure 0 0 0) 0x7f948c05c020 con 0x7f949c105000 2024-04-12T21:29:35.273 INFO:teuthology.orchestra.run.smithi018.stdout:HEALTH_ERR failed to probe daemons or devices; 1 filesystem is degraded; 1 filesystem is offline 2024-04-12T21:29:35.273 INFO:teuthology.orchestra.run.smithi018.stdout:[WRN] CEPHADM_REFRESH_FAILED: failed to probe daemons or devices 2024-04-12T21:29:35.273 INFO:teuthology.orchestra.run.smithi018.stdout: host smithi018 `cephadm ceph-volume` failed: cephadm exited with an error code: 1, stderr: Inferring config /var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/mon.smithi018/config 2024-04-12T21:29:35.274 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/podman: timeout after 895 seconds 2024-04-12T21:29:35.274 INFO:teuthology.orchestra.run.smithi018.stdout:Non-zero exit code 124 from /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:f4452def3be4d350f4a8e5feaf7ebb1511be9298fe68478ba7a2d6fbd7d69a63 -e NODE_NAME=smithi018 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968:/var/run/ceph:z -v /var/log/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968:/var/log/ceph:z -v /var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmpxsmwkkuh:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:f4452def3be4d350f4a8e5feaf7ebb1511be9298fe68478ba7a2d6fbd7d69a63 inventory --format=json-pretty --filter-for-batch 2024-04-12T21:29:35.274 INFO:teuthology.orchestra.run.smithi018.stdout:Traceback (most recent call last): 2024-04-12T21:29:35.274 INFO:teuthology.orchestra.run.smithi018.stdout: File "/usr/lib64/python3.9/runpy.py", line 197, in _run_module_as_main 2024-04-12T21:29:35.274 INFO:teuthology.orchestra.run.smithi018.stdout: return _run_code(code, main_globals, None, 2024-04-12T21:29:35.274 INFO:teuthology.orchestra.run.smithi018.stdout: File "/usr/lib64/python3.9/runpy.py", line 87, in _run_code 2024-04-12T21:29:35.274 INFO:teuthology.orchestra.run.smithi018.stdout: exec(code, run_globals) 2024-04-12T21:29:35.274 INFO:teuthology.orchestra.run.smithi018.stdout: File "/var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/cephadm.e44f3ed5e34e4ef52eedef233d1190bff004470d8e8005127b5312bff357fa7f/__main__.py", line 5618, in 2024-04-12T21:29:35.274 INFO:teuthology.orchestra.run.smithi018.stdout: File "/var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/cephadm.e44f3ed5e34e4ef52eedef233d1190bff004470d8e8005127b5312bff357fa7f/__main__.py", line 5606, in main 2024-04-12T21:29:35.274 INFO:teuthology.orchestra.run.smithi018.stdout: File "/var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/cephadm.e44f3ed5e34e4ef52eedef233d1190bff004470d8e8005127b5312bff357fa7f/__main__.py", line 408, in _infer_config 2024-04-12T21:29:35.274 INFO:teuthology.orchestra.run.smithi018.stdout: File "/var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/cephadm.e44f3ed5e34e4ef52eedef233d1190bff004470d8e8005127b5312bff357fa7f/__main__.py", line 323, in _infer_fsid 2024-04-12T21:29:35.274 INFO:teuthology.orchestra.run.smithi018.stdout: File "/var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/cephadm.e44f3ed5e34e4ef52eedef233d1190bff004470d8e8005127b5312bff357fa7f/__main__.py", line 436, in _infer_image 2024-04-12T21:29:35.274 INFO:teuthology.orchestra.run.smithi018.stdout: File "/var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/cephadm.e44f3ed5e34e4ef52eedef233d1190bff004470d8e8005127b5312bff357fa7f/__main__.py", line 310, in _validate_fsid 2024-04-12T21:29:35.274 INFO:teuthology.orchestra.run.smithi018.stdout: File "/var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/cephadm.e44f3ed5e34e4ef52eedef233d1190bff004470d8e8005127b5312bff357fa7f/__main__.py", line 3359, in command_ceph_volume 2024-04-12T21:29:35.274 INFO:teuthology.orchestra.run.smithi018.stdout: File "/var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/cephadm.e44f3ed5e34e4ef52eedef233d1190bff004470d8e8005127b5312bff357fa7f/cephadmlib/call_wrappers.py", line 310, in call_throws 2024-04-12T21:29:35.275 INFO:teuthology.orchestra.run.smithi018.stdout:RuntimeError: Failed command: /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:f4452def3be4d350f4a8e5feaf7ebb1511be9298fe68478ba7a2d6fbd7d69a63 -e NODE_NAME=smithi018 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968:/var/run/ceph:z -v /var/log/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968:/var/log/ceph:z -v /var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmpxsmwkkuh:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:f4452def3be4d350f4a8e5feaf7ebb1511be9298fe68478ba7a2d6fbd7d69a63 inventory --format=json-pretty --filter-for-batch 2024-04-12T21:29:35.275 INFO:teuthology.orchestra.run.smithi018.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-04-12T21:29:35.275 INFO:teuthology.orchestra.run.smithi018.stdout: fs cephfs is degraded 2024-04-12T21:29:35.275 INFO:teuthology.orchestra.run.smithi018.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-04-12T21:29:35.275 INFO:teuthology.orchestra.run.smithi018.stdout: fs cephfs is offline because no MDS is active for it. 2024-04-12T21:29:35.275 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:29:35.277+0000 7f94a1800700 1 -- 172.21.15.18:0/4092809793 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f948006ec00 msgr2=0x7f94800710b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:29:35.275 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:29:35.277+0000 7f94a1800700 1 --2- 172.21.15.18:0/4092809793 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f948006ec00 0x7f94800710b0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f949c111410 tx=0x7f948800b040 comp rx=0 tx=0).stop 2024-04-12T21:29:35.276 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:29:35.277+0000 7f94a1800700 1 -- 172.21.15.18:0/4092809793 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f949c105000 msgr2=0x7f949c115b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:29:35.276 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:29:35.277+0000 7f94a1800700 1 --2- 172.21.15.18:0/4092809793 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f949c105000 0x7f949c115b70 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7f948c009150 tx=0x7f948c009e30 comp rx=0 tx=0).stop 2024-04-12T21:29:35.276 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:29:35.277+0000 7f94a1800700 1 -- 172.21.15.18:0/4092809793 shutdown_connections 2024-04-12T21:29:35.276 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:29:35.277+0000 7f94a1800700 1 --2- 172.21.15.18:0/4092809793 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f948006ec00 0x7f94800710b0 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:29:35.276 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:29:35.277+0000 7f94a1800700 1 --2- 172.21.15.18:0/4092809793 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f949c105000 0x7f949c115b70 unknown :-1 s=CLOSED pgs=763 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:29:35.276 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:29:35.277+0000 7f94a1800700 1 --2- 172.21.15.18:0/4092809793 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f949c1059a0 0x7f949c1160b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:29:35.276 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:29:35.277+0000 7f94a1800700 1 -- 172.21.15.18:0/4092809793 >> 172.21.15.18:0/4092809793 conn(0x7f949c0fab70 msgr2=0x7f949c0fbb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:29:35.276 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:29:35.277+0000 7f94a1800700 1 -- 172.21.15.18:0/4092809793 shutdown_connections 2024-04-12T21:29:35.276 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:29:35.277+0000 7f94a1800700 1 -- 172.21.15.18:0/4092809793 wait complete. 2024-04-12T21:29:35.281 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:29:34 smithi186 ceph-mon[86379]: from='client.35572 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:29:35.281 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:29:34 smithi186 ceph-mon[86379]: from='client.35576 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:29:35.281 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:29:34 smithi186 ceph-mon[86379]: pgmap v894: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:29:35.281 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:29:34 smithi186 ceph-mon[86379]: from='client.? 172.21.15.18:0/627136014' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T21:29:35.281 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:29:34 smithi186 ceph-mon[86379]: from='client.? 172.21.15.18:0/1438014835' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-12T21:29:36.281 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:29:35 smithi186 ceph-mon[86379]: from='client.35588 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:29:36.281 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:29:35 smithi186 ceph-mon[86379]: from='client.? 172.21.15.18:0/4092809793' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-12T21:29:36.352 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:29:35 smithi018 ceph-mon[112590]: from='client.35588 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:29:36.352 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:29:35 smithi018 ceph-mon[112590]: from='client.? 172.21.15.18:0/4092809793' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-12T21:29:37.282 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:29:36 smithi186 ceph-mon[86379]: pgmap v895: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:29:37.352 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:29:36 smithi018 ceph-mon[112590]: pgmap v895: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:29:38.281 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:29:37 smithi186 ceph-mon[86379]: pgmap v896: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:29:38.352 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:29:37 smithi018 ceph-mon[112590]: pgmap v896: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:29:41.031 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:29:40 smithi186 ceph-mon[86379]: pgmap v897: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:29:41.031 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:29:40 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T21:29:41.102 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:29:40 smithi018 ceph-mon[112590]: pgmap v897: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:29:41.102 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:29:40 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T21:29:42.905 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:29:42 smithi018 ceph-mon[112590]: pgmap v898: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:29:43.031 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:29:42 smithi186 ceph-mon[86379]: pgmap v898: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:29:45.031 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:29:44 smithi186 ceph-mon[86379]: pgmap v899: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:29:45.102 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:29:44 smithi018 ceph-mon[112590]: pgmap v899: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:29:47.031 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:29:46 smithi186 ceph-mon[86379]: pgmap v900: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:29:47.102 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:29:46 smithi018 ceph-mon[112590]: pgmap v900: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:29:49.031 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:29:48 smithi186 ceph-mon[86379]: pgmap v901: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:29:49.102 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:29:48 smithi018 ceph-mon[112590]: pgmap v901: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:29:51.031 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:29:50 smithi186 ceph-mon[86379]: pgmap v902: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:29:51.102 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:29:50 smithi018 ceph-mon[112590]: pgmap v902: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:29:53.031 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:29:52 smithi186 ceph-mon[86379]: pgmap v903: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:29:53.102 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:29:52 smithi018 ceph-mon[112590]: pgmap v903: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:29:55.031 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:29:54 smithi186 ceph-mon[86379]: pgmap v904: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:29:55.102 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:29:54 smithi018 ceph-mon[112590]: pgmap v904: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:29:56.031 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:29:55 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T21:29:56.102 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:29:55 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T21:29:57.031 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:29:56 smithi186 ceph-mon[86379]: pgmap v905: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:29:57.102 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:29:56 smithi018 ceph-mon[112590]: pgmap v905: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:29:59.031 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:29:58 smithi186 ceph-mon[86379]: pgmap v906: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:29:59.102 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:29:58 smithi018 ceph-mon[112590]: pgmap v906: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:30:00.353 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:30:00 smithi018 ceph-1cbe407a-f90d-11ee-bc8d-c7b262605968-mon-smithi018[112586]: 2024-04-12T21:29:59.999+0000 7fd45a600700 -1 log_channel(cluster) log [ERR] : Health detail: HEALTH_ERR failed to probe daemons or devices; 1 filesystem is degraded; 1 filesystem is offline 2024-04-12T21:30:00.353 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:30:00 smithi018 ceph-1cbe407a-f90d-11ee-bc8d-c7b262605968-mon-smithi018[112586]: 2024-04-12T21:29:59.999+0000 7fd45a600700 -1 log_channel(cluster) log [ERR] : [WRN] CEPHADM_REFRESH_FAILED: failed to probe daemons or devices 2024-04-12T21:30:00.353 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:30:00 smithi018 ceph-1cbe407a-f90d-11ee-bc8d-c7b262605968-mon-smithi018[112586]: 2024-04-12T21:29:59.999+0000 7fd45a600700 -1 log_channel(cluster) log [ERR] : host smithi018 `cephadm ceph-volume` failed: cephadm exited with an error code: 1, stderr: Inferring config /var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/mon.smithi018/config 2024-04-12T21:30:00.353 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:30:00 smithi018 ceph-1cbe407a-f90d-11ee-bc8d-c7b262605968-mon-smithi018[112586]: 2024-04-12T21:29:59.999+0000 7fd45a600700 -1 log_channel(cluster) log [ERR] : /usr/bin/podman: timeout after 895 seconds 2024-04-12T21:30:00.353 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:30:00 smithi018 ceph-1cbe407a-f90d-11ee-bc8d-c7b262605968-mon-smithi018[112586]: 2024-04-12T21:29:59.999+0000 7fd45a600700 -1 log_channel(cluster) log [ERR] : Non-zero exit code 124 from /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:f4452def3be4d350f4a8e5feaf7ebb1511be9298fe68478ba7a2d6fbd7d69a63 -e NODE_NAME=smithi018 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968:/var/run/ceph:z -v /var/log/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968:/var/log/ceph:z -v /var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmpxsmwkkuh:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:f4452def3be4d350f4a8e5feaf7ebb1511be9298fe68478ba7a2d6fbd7d69a63 inventory --format=json-pretty --filter-for-batch 2024-04-12T21:30:00.353 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:30:00 smithi018 ceph-1cbe407a-f90d-11ee-bc8d-c7b262605968-mon-smithi018[112586]: 2024-04-12T21:30:00.000+0000 7fd45a600700 -1 log_channel(cluster) log [ERR] : Traceback (most recent call last): 2024-04-12T21:30:00.354 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:30:00 smithi018 ceph-1cbe407a-f90d-11ee-bc8d-c7b262605968-mon-smithi018[112586]: 2024-04-12T21:30:00.000+0000 7fd45a600700 -1 log_channel(cluster) log [ERR] : File "/usr/lib64/python3.9/runpy.py", line 197, in _run_module_as_main 2024-04-12T21:30:00.354 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:30:00 smithi018 ceph-1cbe407a-f90d-11ee-bc8d-c7b262605968-mon-smithi018[112586]: 2024-04-12T21:30:00.000+0000 7fd45a600700 -1 log_channel(cluster) log [ERR] : return _run_code(code, main_globals, None, 2024-04-12T21:30:00.354 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:30:00 smithi018 ceph-1cbe407a-f90d-11ee-bc8d-c7b262605968-mon-smithi018[112586]: 2024-04-12T21:30:00.000+0000 7fd45a600700 -1 log_channel(cluster) log [ERR] : File "/usr/lib64/python3.9/runpy.py", line 87, in _run_code 2024-04-12T21:30:00.354 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:30:00 smithi018 ceph-1cbe407a-f90d-11ee-bc8d-c7b262605968-mon-smithi018[112586]: 2024-04-12T21:30:00.000+0000 7fd45a600700 -1 log_channel(cluster) log [ERR] : exec(code, run_globals) 2024-04-12T21:30:00.354 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:30:00 smithi018 ceph-1cbe407a-f90d-11ee-bc8d-c7b262605968-mon-smithi018[112586]: 2024-04-12T21:30:00.000+0000 7fd45a600700 -1 log_channel(cluster) log [ERR] : File "/var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/cephadm.e44f3ed5e34e4ef52eedef233d1190bff004470d8e8005127b5312bff357fa7f/__main__.py", line 5618, in 2024-04-12T21:30:00.354 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:30:00 smithi018 ceph-1cbe407a-f90d-11ee-bc8d-c7b262605968-mon-smithi018[112586]: 2024-04-12T21:30:00.000+0000 7fd45a600700 -1 log_channel(cluster) log [ERR] : File "/var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/cephadm.e44f3ed5e34e4ef52eedef233d1190bff004470d8e8005127b5312bff357fa7f/__main__.py", line 5606, in main 2024-04-12T21:30:00.354 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:30:00 smithi018 ceph-1cbe407a-f90d-11ee-bc8d-c7b262605968-mon-smithi018[112586]: 2024-04-12T21:30:00.000+0000 7fd45a600700 -1 log_channel(cluster) log [ERR] : File "/var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/cephadm.e44f3ed5e34e4ef52eedef233d1190bff004470d8e8005127b5312bff357fa7f/__main__.py", line 408, in _infer_config 2024-04-12T21:30:00.354 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:30:00 smithi018 ceph-1cbe407a-f90d-11ee-bc8d-c7b262605968-mon-smithi018[112586]: 2024-04-12T21:30:00.000+0000 7fd45a600700 -1 log_channel(cluster) log [ERR] : File "/var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/cephadm.e44f3ed5e34e4ef52eedef233d1190bff004470d8e8005127b5312bff357fa7f/__main__.py", line 323, in _infer_fsid 2024-04-12T21:30:00.355 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:30:00 smithi018 ceph-1cbe407a-f90d-11ee-bc8d-c7b262605968-mon-smithi018[112586]: 2024-04-12T21:30:00.000+0000 7fd45a600700 -1 log_channel(cluster) log [ERR] : File "/var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/cephadm.e44f3ed5e34e4ef52eedef233d1190bff004470d8e8005127b5312bff357fa7f/__main__.py", line 436, in _infer_image 2024-04-12T21:30:00.355 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:30:00 smithi018 ceph-1cbe407a-f90d-11ee-bc8d-c7b262605968-mon-smithi018[112586]: 2024-04-12T21:30:00.000+0000 7fd45a600700 -1 log_channel(cluster) log [ERR] : File "/var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/cephadm.e44f3ed5e34e4ef52eedef233d1190bff004470d8e8005127b5312bff357fa7f/__main__.py", line 310, in _validate_fsid 2024-04-12T21:30:00.355 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:30:00 smithi018 ceph-1cbe407a-f90d-11ee-bc8d-c7b262605968-mon-smithi018[112586]: 2024-04-12T21:30:00.000+0000 7fd45a600700 -1 log_channel(cluster) log [ERR] : File "/var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/cephadm.e44f3ed5e34e4ef52eedef233d1190bff004470d8e8005127b5312bff357fa7f/__main__.py", line 3359, in command_ceph_volume 2024-04-12T21:30:00.355 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:30:00 smithi018 ceph-1cbe407a-f90d-11ee-bc8d-c7b262605968-mon-smithi018[112586]: 2024-04-12T21:30:00.000+0000 7fd45a600700 -1 log_channel(cluster) log [ERR] : File "/var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/cephadm.e44f3ed5e34e4ef52eedef233d1190bff004470d8e8005127b5312bff357fa7f/cephadmlib/call_wrappers.py", line 310, in call_throws 2024-04-12T21:30:00.355 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:30:00 smithi018 ceph-1cbe407a-f90d-11ee-bc8d-c7b262605968-mon-smithi018[112586]: 2024-04-12T21:30:00.000+0000 7fd45a600700 -1 log_channel(cluster) log [ERR] : RuntimeError: Failed command: /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:f4452def3be4d350f4a8e5feaf7ebb1511be9298fe68478ba7a2d6fbd7d69a63 -e NODE_NAME=smithi018 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968:/var/run/ceph:z -v /var/log/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968:/var/log/ceph:z -v /var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmpxsmwkkuh:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:f4452def3be4d350f4a8e5feaf7ebb1511be9298fe68478ba7a2d6fbd7d69a63 inventory --format=json-pretty --filter-for-batch 2024-04-12T21:30:00.355 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:30:00 smithi018 ceph-1cbe407a-f90d-11ee-bc8d-c7b262605968-mon-smithi018[112586]: 2024-04-12T21:30:00.000+0000 7fd45a600700 -1 log_channel(cluster) log [ERR] : [WRN] FS_DEGRADED: 1 filesystem is degraded 2024-04-12T21:30:00.355 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:30:00 smithi018 ceph-1cbe407a-f90d-11ee-bc8d-c7b262605968-mon-smithi018[112586]: 2024-04-12T21:30:00.000+0000 7fd45a600700 -1 log_channel(cluster) log [ERR] : fs cephfs is degraded 2024-04-12T21:30:00.355 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:30:00 smithi018 ceph-1cbe407a-f90d-11ee-bc8d-c7b262605968-mon-smithi018[112586]: 2024-04-12T21:30:00.000+0000 7fd45a600700 -1 log_channel(cluster) log [ERR] : [ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-04-12T21:30:00.355 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:30:00 smithi018 ceph-1cbe407a-f90d-11ee-bc8d-c7b262605968-mon-smithi018[112586]: 2024-04-12T21:30:00.000+0000 7fd45a600700 -1 log_channel(cluster) log [ERR] : fs cephfs is offline because no MDS is active for it. 2024-04-12T21:30:01.032 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:30:00 smithi186 ceph-mon[86379]: pgmap v907: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:30:01.032 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:30:00 smithi186 ceph-mon[86379]: Health detail: HEALTH_ERR failed to probe daemons or devices; 1 filesystem is degraded; 1 filesystem is offline 2024-04-12T21:30:01.032 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:30:00 smithi186 ceph-mon[86379]: [WRN] CEPHADM_REFRESH_FAILED: failed to probe daemons or devices 2024-04-12T21:30:01.032 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:30:00 smithi186 ceph-mon[86379]: host smithi018 `cephadm ceph-volume` failed: cephadm exited with an error code: 1, stderr: Inferring config /var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/mon.smithi018/config 2024-04-12T21:30:01.033 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:30:00 smithi186 ceph-mon[86379]: /usr/bin/podman: timeout after 895 seconds 2024-04-12T21:30:01.033 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:30:00 smithi186 ceph-mon[86379]: Non-zero exit code 124 from /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:f4452def3be4d350f4a8e5feaf7ebb1511be9298fe68478ba7a2d6fbd7d69a63 -e NODE_NAME=smithi018 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968:/var/run/ceph:z -v /var/log/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968:/var/log/ceph:z -v /var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmpxsmwkkuh:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:f4452def3be4d350f4a8e5feaf7ebb1511be9298fe68478ba7a2d6fbd7d69a63 inventory --format=json-pretty --filter-for-batch 2024-04-12T21:30:01.033 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:30:00 smithi186 ceph-mon[86379]: Traceback (most recent call last): 2024-04-12T21:30:01.033 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:30:00 smithi186 ceph-mon[86379]: File "/usr/lib64/python3.9/runpy.py", line 197, in _run_module_as_main 2024-04-12T21:30:01.033 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:30:00 smithi186 ceph-mon[86379]: return _run_code(code, main_globals, None, 2024-04-12T21:30:01.033 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:30:00 smithi186 ceph-mon[86379]: File "/usr/lib64/python3.9/runpy.py", line 87, in _run_code 2024-04-12T21:30:01.033 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:30:00 smithi186 ceph-mon[86379]: exec(code, run_globals) 2024-04-12T21:30:01.033 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:30:00 smithi186 ceph-mon[86379]: File "/var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/cephadm.e44f3ed5e34e4ef52eedef233d1190bff004470d8e8005127b5312bff357fa7f/__main__.py", line 5618, in 2024-04-12T21:30:01.033 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:30:00 smithi186 ceph-mon[86379]: File "/var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/cephadm.e44f3ed5e34e4ef52eedef233d1190bff004470d8e8005127b5312bff357fa7f/__main__.py", line 5606, in main 2024-04-12T21:30:01.033 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:30:00 smithi186 ceph-mon[86379]: File "/var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/cephadm.e44f3ed5e34e4ef52eedef233d1190bff004470d8e8005127b5312bff357fa7f/__main__.py", line 408, in _infer_config 2024-04-12T21:30:01.033 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:30:00 smithi186 ceph-mon[86379]: File "/var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/cephadm.e44f3ed5e34e4ef52eedef233d1190bff004470d8e8005127b5312bff357fa7f/__main__.py", line 323, in _infer_fsid 2024-04-12T21:30:01.033 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:30:00 smithi186 ceph-mon[86379]: File "/var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/cephadm.e44f3ed5e34e4ef52eedef233d1190bff004470d8e8005127b5312bff357fa7f/__main__.py", line 436, in _infer_image 2024-04-12T21:30:01.033 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:30:00 smithi186 ceph-mon[86379]: File "/var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/cephadm.e44f3ed5e34e4ef52eedef233d1190bff004470d8e8005127b5312bff357fa7f/__main__.py", line 310, in _validate_fsid 2024-04-12T21:30:01.033 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:30:00 smithi186 ceph-mon[86379]: File "/var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/cephadm.e44f3ed5e34e4ef52eedef233d1190bff004470d8e8005127b5312bff357fa7f/__main__.py", line 3359, in command_ceph_volume 2024-04-12T21:30:01.034 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:30:00 smithi186 ceph-mon[86379]: File "/var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/cephadm.e44f3ed5e34e4ef52eedef233d1190bff004470d8e8005127b5312bff357fa7f/cephadmlib/call_wrappers.py", line 310, in call_throws 2024-04-12T21:30:01.034 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:30:00 smithi186 ceph-mon[86379]: RuntimeError: Failed command: /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:f4452def3be4d350f4a8e5feaf7ebb1511be9298fe68478ba7a2d6fbd7d69a63 -e NODE_NAME=smithi018 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968:/var/run/ceph:z -v /var/log/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968:/var/log/ceph:z -v /var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmpxsmwkkuh:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:f4452def3be4d350f4a8e5feaf7ebb1511be9298fe68478ba7a2d6fbd7d69a63 inventory --format=json-pretty --filter-for-batch 2024-04-12T21:30:01.034 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:30:00 smithi186 ceph-mon[86379]: [WRN] FS_DEGRADED: 1 filesystem is degraded 2024-04-12T21:30:01.034 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:30:00 smithi186 ceph-mon[86379]: fs cephfs is degraded 2024-04-12T21:30:01.034 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:30:00 smithi186 ceph-mon[86379]: [ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-04-12T21:30:01.034 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:30:00 smithi186 ceph-mon[86379]: fs cephfs is offline because no MDS is active for it. 2024-04-12T21:30:01.103 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:30:00 smithi018 ceph-mon[112590]: pgmap v907: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:30:01.103 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:30:00 smithi018 ceph-mon[112590]: Health detail: HEALTH_ERR failed to probe daemons or devices; 1 filesystem is degraded; 1 filesystem is offline 2024-04-12T21:30:01.103 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:30:00 smithi018 ceph-mon[112590]: [WRN] CEPHADM_REFRESH_FAILED: failed to probe daemons or devices 2024-04-12T21:30:01.103 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:30:00 smithi018 ceph-mon[112590]: host smithi018 `cephadm ceph-volume` failed: cephadm exited with an error code: 1, stderr: Inferring config /var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/mon.smithi018/config 2024-04-12T21:30:01.103 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:30:00 smithi018 ceph-mon[112590]: /usr/bin/podman: timeout after 895 seconds 2024-04-12T21:30:01.103 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:30:00 smithi018 ceph-mon[112590]: Non-zero exit code 124 from /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:f4452def3be4d350f4a8e5feaf7ebb1511be9298fe68478ba7a2d6fbd7d69a63 -e NODE_NAME=smithi018 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968:/var/run/ceph:z -v /var/log/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968:/var/log/ceph:z -v /var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmpxsmwkkuh:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:f4452def3be4d350f4a8e5feaf7ebb1511be9298fe68478ba7a2d6fbd7d69a63 inventory --format=json-pretty --filter-for-batch 2024-04-12T21:30:01.103 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:30:00 smithi018 ceph-mon[112590]: Traceback (most recent call last): 2024-04-12T21:30:01.103 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:30:00 smithi018 ceph-mon[112590]: File "/usr/lib64/python3.9/runpy.py", line 197, in _run_module_as_main 2024-04-12T21:30:01.103 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:30:00 smithi018 ceph-mon[112590]: return _run_code(code, main_globals, None, 2024-04-12T21:30:01.104 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:30:00 smithi018 ceph-mon[112590]: File "/usr/lib64/python3.9/runpy.py", line 87, in _run_code 2024-04-12T21:30:01.104 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:30:00 smithi018 ceph-mon[112590]: exec(code, run_globals) 2024-04-12T21:30:01.104 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:30:00 smithi018 ceph-mon[112590]: File "/var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/cephadm.e44f3ed5e34e4ef52eedef233d1190bff004470d8e8005127b5312bff357fa7f/__main__.py", line 5618, in 2024-04-12T21:30:01.104 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:30:00 smithi018 ceph-mon[112590]: File "/var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/cephadm.e44f3ed5e34e4ef52eedef233d1190bff004470d8e8005127b5312bff357fa7f/__main__.py", line 5606, in main 2024-04-12T21:30:01.104 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:30:00 smithi018 ceph-mon[112590]: File "/var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/cephadm.e44f3ed5e34e4ef52eedef233d1190bff004470d8e8005127b5312bff357fa7f/__main__.py", line 408, in _infer_config 2024-04-12T21:30:01.104 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:30:00 smithi018 ceph-mon[112590]: File "/var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/cephadm.e44f3ed5e34e4ef52eedef233d1190bff004470d8e8005127b5312bff357fa7f/__main__.py", line 323, in _infer_fsid 2024-04-12T21:30:01.104 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:30:00 smithi018 ceph-mon[112590]: File "/var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/cephadm.e44f3ed5e34e4ef52eedef233d1190bff004470d8e8005127b5312bff357fa7f/__main__.py", line 436, in _infer_image 2024-04-12T21:30:01.104 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:30:00 smithi018 ceph-mon[112590]: File "/var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/cephadm.e44f3ed5e34e4ef52eedef233d1190bff004470d8e8005127b5312bff357fa7f/__main__.py", line 310, in _validate_fsid 2024-04-12T21:30:01.104 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:30:00 smithi018 ceph-mon[112590]: File "/var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/cephadm.e44f3ed5e34e4ef52eedef233d1190bff004470d8e8005127b5312bff357fa7f/__main__.py", line 3359, in command_ceph_volume 2024-04-12T21:30:01.104 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:30:00 smithi018 ceph-mon[112590]: File "/var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/cephadm.e44f3ed5e34e4ef52eedef233d1190bff004470d8e8005127b5312bff357fa7f/cephadmlib/call_wrappers.py", line 310, in call_throws 2024-04-12T21:30:01.104 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:30:00 smithi018 ceph-mon[112590]: RuntimeError: Failed command: /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:f4452def3be4d350f4a8e5feaf7ebb1511be9298fe68478ba7a2d6fbd7d69a63 -e NODE_NAME=smithi018 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968:/var/run/ceph:z -v /var/log/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968:/var/log/ceph:z -v /var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmpxsmwkkuh:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:f4452def3be4d350f4a8e5feaf7ebb1511be9298fe68478ba7a2d6fbd7d69a63 inventory --format=json-pretty --filter-for-batch 2024-04-12T21:30:01.104 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:30:00 smithi018 ceph-mon[112590]: [WRN] FS_DEGRADED: 1 filesystem is degraded 2024-04-12T21:30:01.104 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:30:00 smithi018 ceph-mon[112590]: fs cephfs is degraded 2024-04-12T21:30:01.104 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:30:00 smithi018 ceph-mon[112590]: [ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-04-12T21:30:01.104 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:30:00 smithi018 ceph-mon[112590]: fs cephfs is offline because no MDS is active for it. 2024-04-12T21:30:03.031 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:30:02 smithi186 ceph-mon[86379]: pgmap v908: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:30:03.102 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:30:02 smithi018 ceph-mon[112590]: pgmap v908: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:30:05.031 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:30:04 smithi186 ceph-mon[86379]: pgmap v909: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:30:05.102 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:30:04 smithi018 ceph-mon[112590]: pgmap v909: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:30:05.448 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:05.450+0000 7fcc14c00700 1 -- 172.21.15.18:0/3679881229 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fcc10109db0 msgr2=0x7fcc10103420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:30:05.448 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:05.450+0000 7fcc14c00700 1 --2- 172.21.15.18:0/3679881229 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fcc10109db0 0x7fcc10103420 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7fcbfc004940 tx=0x7fcbfc009e30 comp rx=0 tx=0).stop 2024-04-12T21:30:05.448 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:05.450+0000 7fcc14c00700 1 -- 172.21.15.18:0/3679881229 shutdown_connections 2024-04-12T21:30:05.449 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:05.450+0000 7fcc14c00700 1 --2- 172.21.15.18:0/3679881229 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fcc10109db0 0x7fcc10103420 unknown :-1 s=CLOSED pgs=764 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:30:05.449 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:05.450+0000 7fcc14c00700 1 --2- 172.21.15.18:0/3679881229 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcc10109410 0x7fcc101097e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:30:05.449 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:05.450+0000 7fcc14c00700 1 -- 172.21.15.18:0/3679881229 >> 172.21.15.18:0/3679881229 conn(0x7fcc10072ba0 msgr2=0x7fcc10074fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:30:05.449 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:05.450+0000 7fcc14c00700 1 -- 172.21.15.18:0/3679881229 shutdown_connections 2024-04-12T21:30:05.449 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:05.451+0000 7fcc14c00700 1 -- 172.21.15.18:0/3679881229 wait complete. 2024-04-12T21:30:05.449 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:05.451+0000 7fcc14c00700 1 Processor -- start 2024-04-12T21:30:05.449 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:05.452+0000 7fcc14c00700 1 -- start start 2024-04-12T21:30:05.450 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:05.452+0000 7fcc14c00700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcc10109410 0x7fcc10119fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:30:05.450 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:05.452+0000 7fcc14c00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fcc10109db0 0x7fcc1011a510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:30:05.450 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:05.452+0000 7fcc14c00700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc1011ab60 con 0x7fcc10109db0 2024-04-12T21:30:05.450 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:05.452+0000 7fcc14c00700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc10114140 con 0x7fcc10109410 2024-04-12T21:30:05.451 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:05.453+0000 7fcc0d400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fcc10109db0 0x7fcc1011a510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:30:05.451 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:05.453+0000 7fcc0d400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fcc10109db0 0x7fcc1011a510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:50692/0 (socket says 172.21.15.18:50692) 2024-04-12T21:30:05.451 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:05.453+0000 7fcc0d400700 1 -- 172.21.15.18:0/3507178538 learned_addr learned my addr 172.21.15.18:0/3507178538 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:30:05.451 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:05.453+0000 7fcc0de00700 1 --2- 172.21.15.18:0/3507178538 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcc10109410 0x7fcc10119fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:30:05.451 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:05.453+0000 7fcc0d400700 1 -- 172.21.15.18:0/3507178538 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcc10109410 msgr2=0x7fcc10119fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:30:05.451 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:05.453+0000 7fcc0d400700 1 --2- 172.21.15.18:0/3507178538 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcc10109410 0x7fcc10119fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:30:05.451 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:05.453+0000 7fcc0d400700 1 -- 172.21.15.18:0/3507178538 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcbfc005040 con 0x7fcc10109db0 2024-04-12T21:30:05.451 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:05.454+0000 7fcc0d400700 1 --2- 172.21.15.18:0/3507178538 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fcc10109db0 0x7fcc1011a510 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7fcbfc009100 tx=0x7fcbfc009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:30:05.452 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:05.454+0000 7fcc06000700 1 -- 172.21.15.18:0/3507178538 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcbfc00e040 con 0x7fcc10109db0 2024-04-12T21:30:05.452 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:05.454+0000 7fcc06000700 1 -- 172.21.15.18:0/3507178538 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fcbfc013420 con 0x7fcc10109db0 2024-04-12T21:30:05.452 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:05.454+0000 7fcc14c00700 1 -- 172.21.15.18:0/3507178538 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcc101143c0 con 0x7fcc10109db0 2024-04-12T21:30:05.452 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:05.454+0000 7fcc06000700 1 -- 172.21.15.18:0/3507178538 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcbfc01c630 con 0x7fcc10109db0 2024-04-12T21:30:05.452 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:05.454+0000 7fcc14c00700 1 -- 172.21.15.18:0/3507178538 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcc10114980 con 0x7fcc10109db0 2024-04-12T21:30:05.453 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:05.455+0000 7fcc14c00700 1 -- 172.21.15.18:0/3507178538 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcc10104b60 con 0x7fcc10109db0 2024-04-12T21:30:05.454 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:05.456+0000 7fcc06000700 1 -- 172.21.15.18:0/3507178538 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7fcbfc013590 con 0x7fcc10109db0 2024-04-12T21:30:05.455 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:05.457+0000 7fcc06000700 1 --2- 172.21.15.18:0/3507178538 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fcbf406ec00 0x7fcbf40710b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:30:05.455 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:05.458+0000 7fcc06000700 1 -- 172.21.15.18:0/3507178538 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 6757+0+0 (secure 0 0 0) 0x7fcbfc08fba0 con 0x7fcc10109db0 2024-04-12T21:30:05.455 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:05.458+0000 7fcc0de00700 1 --2- 172.21.15.18:0/3507178538 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fcbf406ec00 0x7fcbf40710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:30:05.456 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:05.459+0000 7fcc0de00700 1 --2- 172.21.15.18:0/3507178538 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fcbf406ec00 0x7fcbf40710b0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7fcc00007fd0 tx=0x7fcc00009040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:30:05.460 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:05.463+0000 7fcc06000700 1 -- 172.21.15.18:0/3507178538 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7fcbfc05c3d0 con 0x7fcc10109db0 2024-04-12T21:30:05.714 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:05.716+0000 7fcc14c00700 1 -- 172.21.15.18:0/3507178538 --> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fcc10110550 con 0x7fcbf406ec00 2024-04-12T21:30:05.716 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:05.719+0000 7fcc06000700 1 -- 172.21.15.18:0/3507178538 <== mgr.34102 v2:172.21.15.18:6800/3812891919 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7fcc10110550 con 0x7fcbf406ec00 2024-04-12T21:30:05.719 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:05.722+0000 7fcc14c00700 1 -- 172.21.15.18:0/3507178538 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fcbf406ec00 msgr2=0x7fcbf40710b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:30:05.719 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:05.722+0000 7fcc14c00700 1 --2- 172.21.15.18:0/3507178538 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fcbf406ec00 0x7fcbf40710b0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7fcc00007fd0 tx=0x7fcc00009040 comp rx=0 tx=0).stop 2024-04-12T21:30:05.719 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:05.722+0000 7fcc14c00700 1 -- 172.21.15.18:0/3507178538 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fcc10109db0 msgr2=0x7fcc1011a510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:30:05.719 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:05.722+0000 7fcc14c00700 1 --2- 172.21.15.18:0/3507178538 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fcc10109db0 0x7fcc1011a510 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7fcbfc009100 tx=0x7fcbfc009e30 comp rx=0 tx=0).stop 2024-04-12T21:30:05.720 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:05.722+0000 7fcc14c00700 1 -- 172.21.15.18:0/3507178538 shutdown_connections 2024-04-12T21:30:05.720 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:05.722+0000 7fcc14c00700 1 --2- 172.21.15.18:0/3507178538 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fcbf406ec00 0x7fcbf40710b0 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:30:05.720 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:05.722+0000 7fcc14c00700 1 --2- 172.21.15.18:0/3507178538 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcc10109410 0x7fcc10119fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:30:05.720 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:05.722+0000 7fcc14c00700 1 --2- 172.21.15.18:0/3507178538 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fcc10109db0 0x7fcc1011a510 unknown :-1 s=CLOSED pgs=765 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:30:05.720 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:05.722+0000 7fcc14c00700 1 -- 172.21.15.18:0/3507178538 >> 172.21.15.18:0/3507178538 conn(0x7fcc10072ba0 msgr2=0x7fcc10074cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:30:05.720 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:05.722+0000 7fcc14c00700 1 -- 172.21.15.18:0/3507178538 shutdown_connections 2024-04-12T21:30:05.720 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:05.722+0000 7fcc14c00700 1 -- 172.21.15.18:0/3507178538 wait complete. 2024-04-12T21:30:05.736 INFO:teuthology.orchestra.run.smithi018.stdout:true 2024-04-12T21:30:05.872 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:05.874+0000 7f1a33400700 1 -- 172.21.15.18:0/320647140 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1a2c0fde10 msgr2=0x7f1a2c0fe1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:30:05.872 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:05.874+0000 7f1a33400700 1 --2- 172.21.15.18:0/320647140 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1a2c0fde10 0x7f1a2c0fe1e0 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7f1a20004940 tx=0x7f1a20009e30 comp rx=0 tx=0).stop 2024-04-12T21:30:05.872 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:05.874+0000 7f1a33400700 1 -- 172.21.15.18:0/320647140 shutdown_connections 2024-04-12T21:30:05.872 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:05.874+0000 7f1a33400700 1 --2- 172.21.15.18:0/320647140 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1a2c0fe7b0 0x7f1a2c106df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:30:05.872 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:05.874+0000 7f1a33400700 1 --2- 172.21.15.18:0/320647140 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1a2c0fde10 0x7f1a2c0fe1e0 unknown :-1 s=CLOSED pgs=766 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:30:05.873 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:05.874+0000 7f1a33400700 1 -- 172.21.15.18:0/320647140 >> 172.21.15.18:0/320647140 conn(0x7f1a2c0f7880 msgr2=0x7f1a2c0f9c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:30:05.873 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:05.874+0000 7f1a33400700 1 -- 172.21.15.18:0/320647140 shutdown_connections 2024-04-12T21:30:05.873 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:05.874+0000 7f1a33400700 1 -- 172.21.15.18:0/320647140 wait complete. 2024-04-12T21:30:05.874 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:05.875+0000 7f1a33400700 1 Processor -- start 2024-04-12T21:30:05.874 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:05.875+0000 7f1a33400700 1 -- start start 2024-04-12T21:30:05.874 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:05.876+0000 7f1a33400700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1a2c0fde10 0x7f1a2c0fbc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:30:05.874 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:05.876+0000 7f1a33400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1a2c0fe7b0 0x7f1a2c0fc180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:30:05.874 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:05.876+0000 7f1a33400700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a2c0fc6c0 con 0x7f1a2c0fe7b0 2024-04-12T21:30:05.874 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:05.876+0000 7f1a33400700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a2c0fc800 con 0x7f1a2c0fde10 2024-04-12T21:30:05.874 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:05.876+0000 7f1a2be00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1a2c0fe7b0 0x7f1a2c0fc180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:30:05.874 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:05.876+0000 7f1a2be00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1a2c0fe7b0 0x7f1a2c0fc180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:50706/0 (socket says 172.21.15.18:50706) 2024-04-12T21:30:05.874 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:05.876+0000 7f1a2be00700 1 -- 172.21.15.18:0/2141006836 learned_addr learned my addr 172.21.15.18:0/2141006836 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:30:05.875 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:05.876+0000 7f1a30a00700 1 --2- 172.21.15.18:0/2141006836 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1a2c0fde10 0x7f1a2c0fbc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:30:05.875 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:05.877+0000 7f1a2be00700 1 -- 172.21.15.18:0/2141006836 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1a2c0fde10 msgr2=0x7f1a2c0fbc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:30:05.875 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:05.877+0000 7f1a2be00700 1 --2- 172.21.15.18:0/2141006836 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1a2c0fde10 0x7f1a2c0fbc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:30:05.875 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:05.877+0000 7f1a2be00700 1 -- 172.21.15.18:0/2141006836 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a20005040 con 0x7f1a2c0fe7b0 2024-04-12T21:30:05.875 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:05.877+0000 7f1a2be00700 1 --2- 172.21.15.18:0/2141006836 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1a2c0fe7b0 0x7f1a2c0fc180 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7f1a18002a50 tx=0x7f1a180046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:30:05.876 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:05.877+0000 7f1a28c00700 1 -- 172.21.15.18:0/2141006836 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1a18007cc0 con 0x7f1a2c0fe7b0 2024-04-12T21:30:05.876 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:05.877+0000 7f1a33400700 1 -- 172.21.15.18:0/2141006836 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1a2c0fcab0 con 0x7f1a2c0fe7b0 2024-04-12T21:30:05.876 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:05.877+0000 7f1a28c00700 1 -- 172.21.15.18:0/2141006836 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f1a1800f040 con 0x7f1a2c0fe7b0 2024-04-12T21:30:05.876 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:05.877+0000 7f1a28c00700 1 -- 172.21.15.18:0/2141006836 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1a1800db70 con 0x7f1a2c0fe7b0 2024-04-12T21:30:05.876 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:05.877+0000 7f1a33400700 1 -- 172.21.15.18:0/2141006836 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1a2c11a5c0 con 0x7f1a2c0fe7b0 2024-04-12T21:30:05.876 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:05.878+0000 7f1a33400700 1 -- 172.21.15.18:0/2141006836 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1a2c066770 con 0x7f1a2c0fe7b0 2024-04-12T21:30:05.877 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:05.880+0000 7f1a28c00700 1 -- 172.21.15.18:0/2141006836 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7f1a1800dd10 con 0x7f1a2c0fe7b0 2024-04-12T21:30:05.879 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:05.881+0000 7f1a28c00700 1 --2- 172.21.15.18:0/2141006836 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f1a1406ec00 0x7f1a140710b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:30:05.879 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:05.881+0000 7f1a28c00700 1 -- 172.21.15.18:0/2141006836 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 6757+0+0 (secure 0 0 0) 0x7f1a1808dfc0 con 0x7f1a2c0fe7b0 2024-04-12T21:30:05.879 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:05.881+0000 7f1a30a00700 1 --2- 172.21.15.18:0/2141006836 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f1a1406ec00 0x7f1a140710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:30:05.879 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:05.882+0000 7f1a30a00700 1 --2- 172.21.15.18:0/2141006836 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f1a1406ec00 0x7f1a140710b0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f1a20004f90 tx=0x7f1a20009e30 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:30:05.883 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:05.885+0000 7f1a28c00700 1 -- 172.21.15.18:0/2141006836 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f1a180c2ff0 con 0x7f1a2c0fe7b0 2024-04-12T21:30:06.131 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:06.133+0000 7f1a33400700 1 -- 172.21.15.18:0/2141006836 --> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f1a2c0fcab0 con 0x7f1a1406ec00 2024-04-12T21:30:06.134 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:06.136+0000 7f1a28c00700 1 -- 172.21.15.18:0/2141006836 <== mgr.34102 v2:172.21.15.18:6800/3812891919 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f1a2c0fcab0 con 0x7f1a1406ec00 2024-04-12T21:30:06.138 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:06.139+0000 7f1a33400700 1 -- 172.21.15.18:0/2141006836 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f1a1406ec00 msgr2=0x7f1a140710b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:30:06.138 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:06.139+0000 7f1a33400700 1 --2- 172.21.15.18:0/2141006836 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f1a1406ec00 0x7f1a140710b0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f1a20004f90 tx=0x7f1a20009e30 comp rx=0 tx=0).stop 2024-04-12T21:30:06.138 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:06.140+0000 7f1a33400700 1 -- 172.21.15.18:0/2141006836 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1a2c0fe7b0 msgr2=0x7f1a2c0fc180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:30:06.138 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:06.140+0000 7f1a33400700 1 --2- 172.21.15.18:0/2141006836 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1a2c0fe7b0 0x7f1a2c0fc180 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7f1a18002a50 tx=0x7f1a180046b0 comp rx=0 tx=0).stop 2024-04-12T21:30:06.138 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:06.140+0000 7f1a33400700 1 -- 172.21.15.18:0/2141006836 shutdown_connections 2024-04-12T21:30:06.138 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:06.140+0000 7f1a33400700 1 --2- 172.21.15.18:0/2141006836 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f1a1406ec00 0x7f1a140710b0 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:30:06.138 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:06.140+0000 7f1a33400700 1 --2- 172.21.15.18:0/2141006836 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1a2c0fde10 0x7f1a2c0fbc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:30:06.138 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:06.140+0000 7f1a33400700 1 --2- 172.21.15.18:0/2141006836 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1a2c0fe7b0 0x7f1a2c0fc180 unknown :-1 s=CLOSED pgs=767 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:30:06.138 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:06.140+0000 7f1a33400700 1 -- 172.21.15.18:0/2141006836 >> 172.21.15.18:0/2141006836 conn(0x7f1a2c0f7880 msgr2=0x7f1a2c0f8ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:30:06.139 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:06.140+0000 7f1a33400700 1 -- 172.21.15.18:0/2141006836 shutdown_connections 2024-04-12T21:30:06.139 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:06.140+0000 7f1a33400700 1 -- 172.21.15.18:0/2141006836 wait complete. 2024-04-12T21:30:06.296 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:06.298+0000 7fba36e00700 1 -- 172.21.15.18:0/2846247176 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fba300724f0 msgr2=0x7fba300728c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:30:06.297 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:06.298+0000 7fba36e00700 1 --2- 172.21.15.18:0/2846247176 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fba300724f0 0x7fba300728c0 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7fba20004940 tx=0x7fba20009e30 comp rx=0 tx=0).stop 2024-04-12T21:30:06.297 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:06.298+0000 7fba36e00700 1 -- 172.21.15.18:0/2846247176 shutdown_connections 2024-04-12T21:30:06.297 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:06.298+0000 7fba36e00700 1 --2- 172.21.15.18:0/2846247176 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fba30072e00 0x7fba3010d770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:30:06.297 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:06.298+0000 7fba36e00700 1 --2- 172.21.15.18:0/2846247176 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fba300724f0 0x7fba300728c0 unknown :-1 s=CLOSED pgs=768 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:30:06.297 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:06.298+0000 7fba36e00700 1 -- 172.21.15.18:0/2846247176 >> 172.21.15.18:0/2846247176 conn(0x7fba300fabd0 msgr2=0x7fba300fcfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:30:06.297 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:06.299+0000 7fba36e00700 1 -- 172.21.15.18:0/2846247176 shutdown_connections 2024-04-12T21:30:06.297 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:06.299+0000 7fba36e00700 1 -- 172.21.15.18:0/2846247176 wait complete. 2024-04-12T21:30:06.298 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:06.299+0000 7fba36e00700 1 Processor -- start 2024-04-12T21:30:06.298 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:06.300+0000 7fba36e00700 1 -- start start 2024-04-12T21:30:06.298 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:06.300+0000 7fba36e00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fba300724f0 0x7fba3011a120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:30:06.298 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:06.300+0000 7fba36e00700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fba30072e00 0x7fba3011a660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:30:06.298 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:06.300+0000 7fba36e00700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba3011acb0 con 0x7fba300724f0 2024-04-12T21:30:06.298 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:06.300+0000 7fba36e00700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba30114080 con 0x7fba30072e00 2024-04-12T21:30:06.299 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:06.300+0000 7fba2fe00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fba300724f0 0x7fba3011a120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:30:06.299 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:06.300+0000 7fba2fe00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fba300724f0 0x7fba3011a120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:50734/0 (socket says 172.21.15.18:50734) 2024-04-12T21:30:06.299 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:06.300+0000 7fba2fe00700 1 -- 172.21.15.18:0/384284023 learned_addr learned my addr 172.21.15.18:0/384284023 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:30:06.299 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:06.301+0000 7fba2f400700 1 --2- 172.21.15.18:0/384284023 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fba30072e00 0x7fba3011a660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:30:06.299 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:06.301+0000 7fba2fe00700 1 -- 172.21.15.18:0/384284023 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fba30072e00 msgr2=0x7fba3011a660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:30:06.299 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:06.301+0000 7fba2fe00700 1 --2- 172.21.15.18:0/384284023 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fba30072e00 0x7fba3011a660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:30:06.299 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:06.301+0000 7fba2fe00700 1 -- 172.21.15.18:0/384284023 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba20005040 con 0x7fba300724f0 2024-04-12T21:30:06.299 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:06.301+0000 7fba2fe00700 1 --2- 172.21.15.18:0/384284023 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fba300724f0 0x7fba3011a120 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7fba20004890 tx=0x7fba20009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:30:06.300 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:06.302+0000 7fba27e00700 1 -- 172.21.15.18:0/384284023 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fba2000e040 con 0x7fba300724f0 2024-04-12T21:30:06.300 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:06.302+0000 7fba27e00700 1 -- 172.21.15.18:0/384284023 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fba20009af0 con 0x7fba300724f0 2024-04-12T21:30:06.300 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:06.302+0000 7fba36e00700 1 -- 172.21.15.18:0/384284023 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fba30114300 con 0x7fba300724f0 2024-04-12T21:30:06.300 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:06.302+0000 7fba27e00700 1 -- 172.21.15.18:0/384284023 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fba2001c630 con 0x7fba300724f0 2024-04-12T21:30:06.300 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:06.302+0000 7fba36e00700 1 -- 172.21.15.18:0/384284023 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fba301148c0 con 0x7fba300724f0 2024-04-12T21:30:06.301 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:06.303+0000 7fba36e00700 1 -- 172.21.15.18:0/384284023 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fba3010a5b0 con 0x7fba300724f0 2024-04-12T21:30:06.302 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:06.304+0000 7fba27e00700 1 -- 172.21.15.18:0/384284023 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7fba2000bbc0 con 0x7fba300724f0 2024-04-12T21:30:06.303 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:06.306+0000 7fba27e00700 1 --2- 172.21.15.18:0/384284023 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fba1406ec00 0x7fba140710b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:30:06.303 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:06.306+0000 7fba27e00700 1 -- 172.21.15.18:0/384284023 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 6757+0+0 (secure 0 0 0) 0x7fba20090710 con 0x7fba300724f0 2024-04-12T21:30:06.304 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:06.306+0000 7fba2f400700 1 --2- 172.21.15.18:0/384284023 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fba1406ec00 0x7fba140710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:30:06.304 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:06.307+0000 7fba2f400700 1 --2- 172.21.15.18:0/384284023 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fba1406ec00 0x7fba140710b0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7fba1c007fd0 tx=0x7fba1c009040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:30:06.307 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:06.310+0000 7fba27e00700 1 -- 172.21.15.18:0/384284023 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7fba2005d000 con 0x7fba300724f0 2024-04-12T21:30:06.536 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:06.539+0000 7fba36e00700 1 -- 172.21.15.18:0/384284023 --> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fba3004bed0 con 0x7fba1406ec00 2024-04-12T21:30:06.568 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:06.570+0000 7fba27e00700 1 -- 172.21.15.18:0/384284023 <== mgr.34102 v2:172.21.15.18:6800/3812891919 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7fba3004bed0 con 0x7fba1406ec00 2024-04-12T21:30:06.568 INFO:teuthology.orchestra.run.smithi018.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-12T21:30:06.569 INFO:teuthology.orchestra.run.smithi018.stdout:alertmanager.smithi018 smithi018 *:9093,9094 running (30m) 7m ago 45m 15.5M - 0.25.0 c8568f914cd2 c332090068e8 2024-04-12T21:30:06.569 INFO:teuthology.orchestra.run.smithi018.stdout:crash.smithi018 smithi018 running (27m) 7m ago 45m 6665k - 19.0.0-2879-g324baf8d 8717720dc5e5 b7f80d1f7982 2024-04-12T21:30:06.569 INFO:teuthology.orchestra.run.smithi018.stdout:crash.smithi186 smithi186 running (26m) 7m ago 44m 6660k - 19.0.0-2879-g324baf8d 8717720dc5e5 a4f55c81b623 2024-04-12T21:30:06.569 INFO:teuthology.orchestra.run.smithi018.stdout:grafana.smithi018 smithi018 *:3000 running (30m) 7m ago 44m 74.8M - 9.4.12 2bacad6d85d8 b08e3bf176dd 2024-04-12T21:30:06.569 INFO:teuthology.orchestra.run.smithi018.stdout:mds.cephfs.smithi018.acqbsc smithi018 running (22m) 7m ago 38m 16.7M - 19.0.0-2879-g324baf8d 8717720dc5e5 021f99ab24ae 2024-04-12T21:30:06.569 INFO:teuthology.orchestra.run.smithi018.stdout:mds.cephfs.smithi018.szjluo smithi018 running (22m) 7m ago 37m 16.5M - 19.0.0-2879-g324baf8d 8717720dc5e5 0003117479d4 2024-04-12T21:30:06.569 INFO:teuthology.orchestra.run.smithi018.stdout:mds.cephfs.smithi186.jdllyn smithi186 running (7m) 7m ago 38m 11.0M - 19.0.0-2879-g324baf8d 8717720dc5e5 21b1f46eef23 2024-04-12T21:30:06.569 INFO:teuthology.orchestra.run.smithi018.stdout:mds.cephfs.smithi186.wjnafs smithi186 running (37m) 7m ago 37m 19.5M - 17.2.7-827-gd3b5060a 21de9b7369ad 74b115d6d6fd 2024-04-12T21:30:06.569 INFO:teuthology.orchestra.run.smithi018.stdout:mgr.smithi018.xjzwga smithi018 *:8443,9283,8765 running (33m) 7m ago 46m 512M - 19.0.0-2879-g324baf8d 8717720dc5e5 56088305eab7 2024-04-12T21:30:06.569 INFO:teuthology.orchestra.run.smithi018.stdout:mgr.smithi186.kqcfxk smithi186 *:8443,9283,8765 running (32m) 7m ago 43m 459M - 19.0.0-2879-g324baf8d 8717720dc5e5 09cd766e5e0d 2024-04-12T21:30:06.569 INFO:teuthology.orchestra.run.smithi018.stdout:mon.smithi018 smithi018 running (28m) 7m ago 47m 73.1M 2048M 19.0.0-2879-g324baf8d 8717720dc5e5 df3e201dfed6 2024-04-12T21:30:06.570 INFO:teuthology.orchestra.run.smithi018.stdout:mon.smithi186 smithi186 running (28m) 7m ago 43m 47.2M 2048M 19.0.0-2879-g324baf8d 8717720dc5e5 c77498da9b2b 2024-04-12T21:30:06.570 INFO:teuthology.orchestra.run.smithi018.stdout:node-exporter.smithi018 smithi018 *:9100 running (31m) 7m ago 45m 8108k - 1.5.0 0da6a335fe13 326c3d9c6847 2024-04-12T21:30:06.570 INFO:teuthology.orchestra.run.smithi018.stdout:node-exporter.smithi186 smithi186 *:9100 running (31m) 7m ago 43m 8757k - 1.5.0 0da6a335fe13 ec2112baeecf 2024-04-12T21:30:06.570 INFO:teuthology.orchestra.run.smithi018.stdout:osd.0 smithi018 running (26m) 7m ago 42m 123M 1559M 19.0.0-2879-g324baf8d 8717720dc5e5 60b1273ca047 2024-04-12T21:30:06.570 INFO:teuthology.orchestra.run.smithi018.stdout:osd.1 smithi018 running (25m) 7m ago 42m 95.1M 1559M 19.0.0-2879-g324baf8d 8717720dc5e5 170ce1e5fb8d 2024-04-12T21:30:06.570 INFO:teuthology.orchestra.run.smithi018.stdout:osd.2 smithi018 running (25m) 7m ago 41m 94.3M 1559M 19.0.0-2879-g324baf8d 8717720dc5e5 c2cbc6aa2839 2024-04-12T21:30:06.570 INFO:teuthology.orchestra.run.smithi018.stdout:osd.3 smithi186 running (24m) 7m ago 41m 101M 2581M 19.0.0-2879-g324baf8d 8717720dc5e5 0138e8262e4b 2024-04-12T21:30:06.570 INFO:teuthology.orchestra.run.smithi018.stdout:osd.4 smithi186 running (24m) 7m ago 40m 91.0M 2581M 19.0.0-2879-g324baf8d 8717720dc5e5 b20cb63b25c9 2024-04-12T21:30:06.570 INFO:teuthology.orchestra.run.smithi018.stdout:osd.5 smithi186 running (23m) 7m ago 39m 92.0M 2581M 19.0.0-2879-g324baf8d 8717720dc5e5 2513bea2610e 2024-04-12T21:30:06.570 INFO:teuthology.orchestra.run.smithi018.stdout:prometheus.smithi018 smithi018 *:9095 running (31m) 7m ago 44m 67.9M - 2.43.0 a07b618ecd1d f58841127e21 2024-04-12T21:30:06.571 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:06.573+0000 7fba36e00700 1 -- 172.21.15.18:0/384284023 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fba1406ec00 msgr2=0x7fba140710b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:30:06.571 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:06.573+0000 7fba36e00700 1 --2- 172.21.15.18:0/384284023 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fba1406ec00 0x7fba140710b0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7fba1c007fd0 tx=0x7fba1c009040 comp rx=0 tx=0).stop 2024-04-12T21:30:06.571 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:06.573+0000 7fba36e00700 1 -- 172.21.15.18:0/384284023 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fba300724f0 msgr2=0x7fba3011a120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:30:06.571 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:06.573+0000 7fba36e00700 1 --2- 172.21.15.18:0/384284023 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fba300724f0 0x7fba3011a120 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7fba20004890 tx=0x7fba20009e30 comp rx=0 tx=0).stop 2024-04-12T21:30:06.571 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:06.573+0000 7fba36e00700 1 -- 172.21.15.18:0/384284023 shutdown_connections 2024-04-12T21:30:06.571 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:06.573+0000 7fba36e00700 1 --2- 172.21.15.18:0/384284023 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fba1406ec00 0x7fba140710b0 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:30:06.572 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:06.573+0000 7fba36e00700 1 --2- 172.21.15.18:0/384284023 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fba300724f0 0x7fba3011a120 unknown :-1 s=CLOSED pgs=769 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:30:06.572 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:06.573+0000 7fba36e00700 1 --2- 172.21.15.18:0/384284023 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fba30072e00 0x7fba3011a660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:30:06.572 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:06.573+0000 7fba36e00700 1 -- 172.21.15.18:0/384284023 >> 172.21.15.18:0/384284023 conn(0x7fba300fabd0 msgr2=0x7fba30107730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:30:06.572 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:06.573+0000 7fba36e00700 1 -- 172.21.15.18:0/384284023 shutdown_connections 2024-04-12T21:30:06.572 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:06.573+0000 7fba36e00700 1 -- 172.21.15.18:0/384284023 wait complete. 2024-04-12T21:30:06.715 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:06.717+0000 7f5ee8c00700 1 -- 172.21.15.18:0/1459954509 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5ee4102ca0 msgr2=0x7f5ee4103070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:30:06.715 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:06.717+0000 7f5ee8c00700 1 --2- 172.21.15.18:0/1459954509 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5ee4102ca0 0x7f5ee4103070 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7f5ed4004940 tx=0x7f5ed4009e30 comp rx=0 tx=0).stop 2024-04-12T21:30:06.715 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:06.717+0000 7f5ee8c00700 1 -- 172.21.15.18:0/1459954509 shutdown_connections 2024-04-12T21:30:06.715 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:06.717+0000 7f5ee8c00700 1 --2- 172.21.15.18:0/1459954509 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5ee4065630 0x7f5ee4065a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:30:06.716 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:06.717+0000 7f5ee8c00700 1 --2- 172.21.15.18:0/1459954509 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5ee4102ca0 0x7f5ee4103070 unknown :-1 s=CLOSED pgs=770 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:30:06.716 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:06.717+0000 7f5ee8c00700 1 -- 172.21.15.18:0/1459954509 >> 172.21.15.18:0/1459954509 conn(0x7f5ee4071b70 msgr2=0x7f5ee4071f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:30:06.716 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:06.718+0000 7f5ee8c00700 1 -- 172.21.15.18:0/1459954509 shutdown_connections 2024-04-12T21:30:06.716 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:06.718+0000 7f5ee8c00700 1 -- 172.21.15.18:0/1459954509 wait complete. 2024-04-12T21:30:06.716 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:06.719+0000 7f5ee8c00700 1 Processor -- start 2024-04-12T21:30:06.716 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:06.719+0000 7f5ee8c00700 1 -- start start 2024-04-12T21:30:06.717 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:06.719+0000 7f5ee8c00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5ee4065630 0x7f5ee4113990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:30:06.717 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:06.719+0000 7f5ee8c00700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5ee4102ca0 0x7f5ee4113ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:30:06.718 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:06.719+0000 7f5ee8c00700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ee41145b0 con 0x7f5ee4065630 2024-04-12T21:30:06.718 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:06.719+0000 7f5ee8c00700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ee410d910 con 0x7f5ee4102ca0 2024-04-12T21:30:06.718 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:06.719+0000 7f5ee3400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5ee4065630 0x7f5ee4113990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:30:06.718 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:06.719+0000 7f5ee3400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5ee4065630 0x7f5ee4113990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:50752/0 (socket says 172.21.15.18:50752) 2024-04-12T21:30:06.718 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:06.719+0000 7f5ee3400700 1 -- 172.21.15.18:0/1924619031 learned_addr learned my addr 172.21.15.18:0/1924619031 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:30:06.718 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:06.720+0000 7f5ee2a00700 1 --2- 172.21.15.18:0/1924619031 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5ee4102ca0 0x7f5ee4113ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:30:06.718 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:06.720+0000 7f5ee3400700 1 -- 172.21.15.18:0/1924619031 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5ee4102ca0 msgr2=0x7f5ee4113ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:30:06.718 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:06.720+0000 7f5ee3400700 1 --2- 172.21.15.18:0/1924619031 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5ee4102ca0 0x7f5ee4113ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:30:06.718 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:06.720+0000 7f5ee3400700 1 -- 172.21.15.18:0/1924619031 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ed4005040 con 0x7f5ee4065630 2024-04-12T21:30:06.719 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:06.720+0000 7f5ee3400700 1 --2- 172.21.15.18:0/1924619031 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5ee4065630 0x7f5ee4113990 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7f5ed4009180 tx=0x7f5ed4009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:30:06.719 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:06.721+0000 7f5edb400700 1 -- 172.21.15.18:0/1924619031 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5ed400e040 con 0x7f5ee4065630 2024-04-12T21:30:06.719 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:06.721+0000 7f5ee8c00700 1 -- 172.21.15.18:0/1924619031 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5ee410db90 con 0x7f5ee4065630 2024-04-12T21:30:06.719 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:06.721+0000 7f5edb400700 1 -- 172.21.15.18:0/1924619031 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f5ed400bbc0 con 0x7f5ee4065630 2024-04-12T21:30:06.719 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:06.721+0000 7f5edb400700 1 -- 172.21.15.18:0/1924619031 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5ed401c630 con 0x7f5ee4065630 2024-04-12T21:30:06.719 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:06.721+0000 7f5ee8c00700 1 -- 172.21.15.18:0/1924619031 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5ee410e150 con 0x7f5ee4065630 2024-04-12T21:30:06.720 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:06.722+0000 7f5ee8c00700 1 -- 172.21.15.18:0/1924619031 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5ee40fe380 con 0x7f5ee4065630 2024-04-12T21:30:06.721 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:06.724+0000 7f5edb400700 1 -- 172.21.15.18:0/1924619031 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7f5ed400bd30 con 0x7f5ee4065630 2024-04-12T21:30:06.722 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:06.725+0000 7f5edb400700 1 --2- 172.21.15.18:0/1924619031 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f5ec806ec10 0x7f5ec80710c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:30:06.722 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:06.725+0000 7f5edb400700 1 -- 172.21.15.18:0/1924619031 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 6757+0+0 (secure 0 0 0) 0x7f5ed408fe30 con 0x7f5ee4065630 2024-04-12T21:30:06.723 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:06.725+0000 7f5ee2a00700 1 --2- 172.21.15.18:0/1924619031 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f5ec806ec10 0x7f5ec80710c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:30:06.723 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:06.726+0000 7f5ee2a00700 1 --2- 172.21.15.18:0/1924619031 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f5ec806ec10 0x7f5ec80710c0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f5ed0006fd0 tx=0x7f5ed0008040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:30:06.727 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:06.730+0000 7f5edb400700 1 -- 172.21.15.18:0/1924619031 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f5ed405c5e0 con 0x7f5ee4065630 2024-04-12T21:30:06.852 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:30:06 smithi018 ceph-mon[112590]: pgmap v910: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:30:06.852 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:30:06 smithi018 ceph-mon[112590]: from='client.35596 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:30:07.031 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:30:06 smithi186 ceph-mon[86379]: pgmap v910: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:30:07.031 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:30:06 smithi186 ceph-mon[86379]: from='client.35596 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:30:07.047 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:07.049+0000 7f5ee8c00700 1 -- 172.21.15.18:0/1924619031 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f5ee404bed0 con 0x7f5ee4065630 2024-04-12T21:30:07.048 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:07.050+0000 7f5edb400700 1 -- 172.21.15.18:0/1924619031 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+837 (secure 0 0 0) 0x7f5ed405c170 con 0x7f5ee4065630 2024-04-12T21:30:07.048 INFO:teuthology.orchestra.run.smithi018.stdout:{ 2024-04-12T21:30:07.048 INFO:teuthology.orchestra.run.smithi018.stdout: "mon": { 2024-04-12T21:30:07.048 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 19.0.0-2879-g324baf8d (324baf8ddb91dea46cac50cb535c306115791355) squid (dev)": 2 2024-04-12T21:30:07.049 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-12T21:30:07.049 INFO:teuthology.orchestra.run.smithi018.stdout: "mgr": { 2024-04-12T21:30:07.049 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 19.0.0-2879-g324baf8d (324baf8ddb91dea46cac50cb535c306115791355) squid (dev)": 2 2024-04-12T21:30:07.049 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-12T21:30:07.049 INFO:teuthology.orchestra.run.smithi018.stdout: "osd": { 2024-04-12T21:30:07.049 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 19.0.0-2879-g324baf8d (324baf8ddb91dea46cac50cb535c306115791355) squid (dev)": 6 2024-04-12T21:30:07.049 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-12T21:30:07.049 INFO:teuthology.orchestra.run.smithi018.stdout: "mds": { 2024-04-12T21:30:07.049 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 17.2.7-827-gd3b5060a (d3b5060a17449edfae9385cdf792afec109fca5d) quincy (stable)": 1, 2024-04-12T21:30:07.049 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 19.0.0-2879-g324baf8d (324baf8ddb91dea46cac50cb535c306115791355) squid (dev)": 3 2024-04-12T21:30:07.049 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-12T21:30:07.049 INFO:teuthology.orchestra.run.smithi018.stdout: "overall": { 2024-04-12T21:30:07.049 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 17.2.7-827-gd3b5060a (d3b5060a17449edfae9385cdf792afec109fca5d) quincy (stable)": 1, 2024-04-12T21:30:07.050 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 19.0.0-2879-g324baf8d (324baf8ddb91dea46cac50cb535c306115791355) squid (dev)": 13 2024-04-12T21:30:07.050 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-04-12T21:30:07.050 INFO:teuthology.orchestra.run.smithi018.stdout:} 2024-04-12T21:30:07.051 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:07.053+0000 7f5ee8c00700 1 -- 172.21.15.18:0/1924619031 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f5ec806ec10 msgr2=0x7f5ec80710c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:30:07.051 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:07.053+0000 7f5ee8c00700 1 --2- 172.21.15.18:0/1924619031 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f5ec806ec10 0x7f5ec80710c0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f5ed0006fd0 tx=0x7f5ed0008040 comp rx=0 tx=0).stop 2024-04-12T21:30:07.051 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:07.053+0000 7f5ee8c00700 1 -- 172.21.15.18:0/1924619031 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5ee4065630 msgr2=0x7f5ee4113990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:30:07.051 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:07.053+0000 7f5ee8c00700 1 --2- 172.21.15.18:0/1924619031 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5ee4065630 0x7f5ee4113990 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7f5ed4009180 tx=0x7f5ed4009e30 comp rx=0 tx=0).stop 2024-04-12T21:30:07.052 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:07.053+0000 7f5ee8c00700 1 -- 172.21.15.18:0/1924619031 shutdown_connections 2024-04-12T21:30:07.052 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:07.053+0000 7f5ee8c00700 1 --2- 172.21.15.18:0/1924619031 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f5ec806ec10 0x7f5ec80710c0 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:30:07.052 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:07.053+0000 7f5ee8c00700 1 --2- 172.21.15.18:0/1924619031 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5ee4065630 0x7f5ee4113990 unknown :-1 s=CLOSED pgs=771 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:30:07.052 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:07.053+0000 7f5ee8c00700 1 --2- 172.21.15.18:0/1924619031 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5ee4102ca0 0x7f5ee4113ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:30:07.052 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:07.053+0000 7f5ee8c00700 1 -- 172.21.15.18:0/1924619031 >> 172.21.15.18:0/1924619031 conn(0x7f5ee4071b70 msgr2=0x7f5ee40f9ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:30:07.052 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:07.054+0000 7f5ee8c00700 1 -- 172.21.15.18:0/1924619031 shutdown_connections 2024-04-12T21:30:07.052 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:07.054+0000 7f5ee8c00700 1 -- 172.21.15.18:0/1924619031 wait complete. 2024-04-12T21:30:07.204 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:07.206+0000 7fae36000700 1 -- 172.21.15.18:0/1453654666 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fae30102e00 msgr2=0x7fae301031d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:30:07.205 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:07.206+0000 7fae36000700 1 --2- 172.21.15.18:0/1453654666 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fae30102e00 0x7fae301031d0 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7fae20004940 tx=0x7fae20009e30 comp rx=0 tx=0).stop 2024-04-12T21:30:07.205 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:07.206+0000 7fae36000700 1 -- 172.21.15.18:0/1453654666 shutdown_connections 2024-04-12T21:30:07.205 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:07.206+0000 7fae36000700 1 --2- 172.21.15.18:0/1453654666 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fae301037a0 0x7fae300fce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:30:07.205 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:07.206+0000 7fae36000700 1 --2- 172.21.15.18:0/1453654666 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fae30102e00 0x7fae301031d0 unknown :-1 s=CLOSED pgs=772 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:30:07.205 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:07.206+0000 7fae36000700 1 -- 172.21.15.18:0/1453654666 >> 172.21.15.18:0/1453654666 conn(0x7fae300f8970 msgr2=0x7fae300fad80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:30:07.205 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:07.207+0000 7fae36000700 1 -- 172.21.15.18:0/1453654666 shutdown_connections 2024-04-12T21:30:07.205 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:07.207+0000 7fae36000700 1 -- 172.21.15.18:0/1453654666 wait complete. 2024-04-12T21:30:07.207 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:07.207+0000 7fae36000700 1 Processor -- start 2024-04-12T21:30:07.207 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:07.207+0000 7fae36000700 1 -- start start 2024-04-12T21:30:07.207 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:07.208+0000 7fae36000700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fae30102e00 0x7fae30113b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:30:07.207 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:07.208+0000 7fae36000700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fae301037a0 0x7fae301140c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:30:07.207 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:07.208+0000 7fae36000700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae301147a0 con 0x7fae30102e00 2024-04-12T21:30:07.207 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:07.208+0000 7fae36000700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae3010db00 con 0x7fae301037a0 2024-04-12T21:30:07.207 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:07.208+0000 7fae2f400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fae30102e00 0x7fae30113b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:30:07.207 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:07.208+0000 7fae2f400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fae30102e00 0x7fae30113b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:50778/0 (socket says 172.21.15.18:50778) 2024-04-12T21:30:07.207 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:07.208+0000 7fae2f400700 1 -- 172.21.15.18:0/3359997999 learned_addr learned my addr 172.21.15.18:0/3359997999 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:30:07.207 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:07.209+0000 7fae2f400700 1 -- 172.21.15.18:0/3359997999 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fae301037a0 msgr2=0x7fae301140c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:30:07.208 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:07.209+0000 7fae2ea00700 1 --2- 172.21.15.18:0/3359997999 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fae301037a0 0x7fae301140c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:30:07.208 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:07.209+0000 7fae2f400700 1 --2- 172.21.15.18:0/3359997999 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fae301037a0 0x7fae301140c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:30:07.208 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:07.209+0000 7fae2f400700 1 -- 172.21.15.18:0/3359997999 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae20005040 con 0x7fae30102e00 2024-04-12T21:30:07.208 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:07.209+0000 7fae2ea00700 1 --2- 172.21.15.18:0/3359997999 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fae301037a0 0x7fae301140c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-12T21:30:07.208 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:07.209+0000 7fae2f400700 1 --2- 172.21.15.18:0/3359997999 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fae30102e00 0x7fae30113b80 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7fae20009180 tx=0x7fae20009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:30:07.208 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:07.209+0000 7fae27400700 1 -- 172.21.15.18:0/3359997999 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fae2000e040 con 0x7fae30102e00 2024-04-12T21:30:07.208 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:07.209+0000 7fae27400700 1 -- 172.21.15.18:0/3359997999 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fae2000bbc0 con 0x7fae30102e00 2024-04-12T21:30:07.208 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:07.209+0000 7fae27400700 1 -- 172.21.15.18:0/3359997999 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fae2001c630 con 0x7fae30102e00 2024-04-12T21:30:07.208 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:07.209+0000 7fae36000700 1 -- 172.21.15.18:0/3359997999 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fae3010dd80 con 0x7fae30102e00 2024-04-12T21:30:07.209 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:07.209+0000 7fae36000700 1 -- 172.21.15.18:0/3359997999 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fae3010e340 con 0x7fae30102e00 2024-04-12T21:30:07.209 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:07.210+0000 7fae36000700 1 -- 172.21.15.18:0/3359997999 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fae300fe550 con 0x7fae30102e00 2024-04-12T21:30:07.209 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:07.212+0000 7fae27400700 1 -- 172.21.15.18:0/3359997999 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7fae2000bd30 con 0x7fae30102e00 2024-04-12T21:30:07.210 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:07.212+0000 7fae27400700 1 --2- 172.21.15.18:0/3359997999 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fae1406ec10 0x7fae140710c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:30:07.210 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:07.213+0000 7fae27400700 1 -- 172.21.15.18:0/3359997999 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 6757+0+0 (secure 0 0 0) 0x7fae2008fe20 con 0x7fae30102e00 2024-04-12T21:30:07.210 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:07.213+0000 7fae2ea00700 1 --2- 172.21.15.18:0/3359997999 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fae1406ec10 0x7fae140710c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:30:07.211 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:07.214+0000 7fae2ea00700 1 --2- 172.21.15.18:0/3359997999 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fae1406ec10 0x7fae140710c0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7fae1c006fd0 tx=0x7fae1c008040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:30:07.215 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:07.218+0000 7fae27400700 1 -- 172.21.15.18:0/3359997999 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7fae2005c5d0 con 0x7fae30102e00 2024-04-12T21:30:07.487 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:07.489+0000 7fae36000700 1 -- 172.21.15.18:0/3359997999 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fae3004bed0 con 0x7fae30102e00 2024-04-12T21:30:07.488 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:07.490+0000 7fae27400700 1 -- 172.21.15.18:0/3359997999 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 31 v31) v1 ==== 76+0+1873 (secure 0 0 0) 0x7fae2005c160 con 0x7fae30102e00 2024-04-12T21:30:07.489 INFO:teuthology.orchestra.run.smithi018.stdout:e31 2024-04-12T21:30:07.489 INFO:teuthology.orchestra.run.smithi018.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-12T21:30:07.489 INFO:teuthology.orchestra.run.smithi018.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-12T21:30:07.489 INFO:teuthology.orchestra.run.smithi018.stdout:legacy client fscid: 1 2024-04-12T21:30:07.489 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-12T21:30:07.490 INFO:teuthology.orchestra.run.smithi018.stdout:Filesystem 'cephfs' (1) 2024-04-12T21:30:07.490 INFO:teuthology.orchestra.run.smithi018.stdout:fs_name cephfs 2024-04-12T21:30:07.490 INFO:teuthology.orchestra.run.smithi018.stdout:epoch 24 2024-04-12T21:30:07.490 INFO:teuthology.orchestra.run.smithi018.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-04-12T21:30:07.490 INFO:teuthology.orchestra.run.smithi018.stdout:created 2024-04-12T20:51:57.362850+0000 2024-04-12T21:30:07.490 INFO:teuthology.orchestra.run.smithi018.stdout:modified 2024-04-12T21:06:59.717190+0000 2024-04-12T21:30:07.490 INFO:teuthology.orchestra.run.smithi018.stdout:tableserver 0 2024-04-12T21:30:07.490 INFO:teuthology.orchestra.run.smithi018.stdout:root 0 2024-04-12T21:30:07.490 INFO:teuthology.orchestra.run.smithi018.stdout:session_timeout 60 2024-04-12T21:30:07.490 INFO:teuthology.orchestra.run.smithi018.stdout:session_autoclose 300 2024-04-12T21:30:07.490 INFO:teuthology.orchestra.run.smithi018.stdout:max_file_size 1099511627776 2024-04-12T21:30:07.490 INFO:teuthology.orchestra.run.smithi018.stdout:max_xattr_size 65536 2024-04-12T21:30:07.490 INFO:teuthology.orchestra.run.smithi018.stdout:required_client_features {} 2024-04-12T21:30:07.490 INFO:teuthology.orchestra.run.smithi018.stdout:last_failure 0 2024-04-12T21:30:07.490 INFO:teuthology.orchestra.run.smithi018.stdout:last_failure_osd_epoch 100 2024-04-12T21:30:07.491 INFO:teuthology.orchestra.run.smithi018.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-12T21:30:07.491 INFO:teuthology.orchestra.run.smithi018.stdout:max_mds 2 2024-04-12T21:30:07.491 INFO:teuthology.orchestra.run.smithi018.stdout:in 0,1 2024-04-12T21:30:07.491 INFO:teuthology.orchestra.run.smithi018.stdout:up {} 2024-04-12T21:30:07.491 INFO:teuthology.orchestra.run.smithi018.stdout:failed 0,1 2024-04-12T21:30:07.491 INFO:teuthology.orchestra.run.smithi018.stdout:damaged 2024-04-12T21:30:07.491 INFO:teuthology.orchestra.run.smithi018.stdout:stopped 2024-04-12T21:30:07.491 INFO:teuthology.orchestra.run.smithi018.stdout:data_pools [3] 2024-04-12T21:30:07.491 INFO:teuthology.orchestra.run.smithi018.stdout:metadata_pool 2 2024-04-12T21:30:07.491 INFO:teuthology.orchestra.run.smithi018.stdout:inline_data disabled 2024-04-12T21:30:07.491 INFO:teuthology.orchestra.run.smithi018.stdout:balancer 2024-04-12T21:30:07.491 INFO:teuthology.orchestra.run.smithi018.stdout:bal_rank_mask -1 2024-04-12T21:30:07.491 INFO:teuthology.orchestra.run.smithi018.stdout:standby_count_wanted 1 2024-04-12T21:30:07.491 INFO:teuthology.orchestra.run.smithi018.stdout:qdb_cluster leader: 0 members: 2024-04-12T21:30:07.492 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-12T21:30:07.492 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-12T21:30:07.492 INFO:teuthology.orchestra.run.smithi018.stdout:Standby daemons: 2024-04-12T21:30:07.492 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-12T21:30:07.492 INFO:teuthology.orchestra.run.smithi018.stdout:[mds.cephfs.smithi186.wjnafs{-1:24277} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.186:6826/438347207,v1:172.21.15.186:6827/438347207] compat {c=[1],r=[1],i=[7ff]}] 2024-04-12T21:30:07.492 INFO:teuthology.orchestra.run.smithi018.stdout:[mds.cephfs.smithi018.acqbsc{-1:34436} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.18:6828/755179764,v1:172.21.15.18:6829/755179764] compat {c=[1],r=[1],i=[1fff]}] 2024-04-12T21:30:07.492 INFO:teuthology.orchestra.run.smithi018.stdout:[mds.cephfs.smithi018.szjluo{-1:34468} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.18:6826/2407992653,v1:172.21.15.18:6827/2407992653] compat {c=[1],r=[1],i=[1fff]}] 2024-04-12T21:30:07.492 INFO:teuthology.orchestra.run.smithi018.stdout:[mds.cephfs.smithi186.jdllyn{-1:44595} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.186:6824/3523444202,v1:172.21.15.186:6825/3523444202] compat {c=[1],r=[1],i=[1fff]}] 2024-04-12T21:30:07.493 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:07.493+0000 7fae36000700 1 -- 172.21.15.18:0/3359997999 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fae1406ec10 msgr2=0x7fae140710c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:30:07.493 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:07.493+0000 7fae36000700 1 --2- 172.21.15.18:0/3359997999 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fae1406ec10 0x7fae140710c0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7fae1c006fd0 tx=0x7fae1c008040 comp rx=0 tx=0).stop 2024-04-12T21:30:07.493 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:07.493+0000 7fae36000700 1 -- 172.21.15.18:0/3359997999 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fae30102e00 msgr2=0x7fae30113b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:30:07.493 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:07.493+0000 7fae36000700 1 --2- 172.21.15.18:0/3359997999 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fae30102e00 0x7fae30113b80 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7fae20009180 tx=0x7fae20009e30 comp rx=0 tx=0).stop 2024-04-12T21:30:07.493 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:07.493+0000 7fae36000700 1 -- 172.21.15.18:0/3359997999 shutdown_connections 2024-04-12T21:30:07.493 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:07.493+0000 7fae36000700 1 --2- 172.21.15.18:0/3359997999 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fae1406ec10 0x7fae140710c0 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:30:07.493 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:07.493+0000 7fae36000700 1 --2- 172.21.15.18:0/3359997999 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fae30102e00 0x7fae30113b80 unknown :-1 s=CLOSED pgs=773 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:30:07.493 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:07.493+0000 7fae36000700 1 --2- 172.21.15.18:0/3359997999 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fae301037a0 0x7fae301140c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:30:07.494 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:07.493+0000 7fae36000700 1 -- 172.21.15.18:0/3359997999 >> 172.21.15.18:0/3359997999 conn(0x7fae300f8970 msgr2=0x7fae300f9940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:30:07.494 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:07.493+0000 7fae36000700 1 -- 172.21.15.18:0/3359997999 shutdown_connections 2024-04-12T21:30:07.494 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:07.493+0000 7fae36000700 1 -- 172.21.15.18:0/3359997999 wait complete. 2024-04-12T21:30:07.494 INFO:teuthology.orchestra.run.smithi018.stderr:dumped fsmap epoch 31 2024-04-12T21:30:07.645 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:07.647+0000 7fddd5400700 1 -- 172.21.15.18:0/3755834556 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fddd0105020 msgr2=0x7fddd01053f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:30:07.645 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:07.647+0000 7fddd5400700 1 --2- 172.21.15.18:0/3755834556 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fddd0105020 0x7fddd01053f0 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7fddc0004940 tx=0x7fddc0009e30 comp rx=0 tx=0).stop 2024-04-12T21:30:07.645 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:07.647+0000 7fddd5400700 1 -- 172.21.15.18:0/3755834556 shutdown_connections 2024-04-12T21:30:07.645 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:07.647+0000 7fddd5400700 1 --2- 172.21.15.18:0/3755834556 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fddd01059c0 0x7fddd00ff080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:30:07.645 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:07.647+0000 7fddd5400700 1 --2- 172.21.15.18:0/3755834556 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fddd0105020 0x7fddd01053f0 unknown :-1 s=CLOSED pgs=774 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:30:07.645 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:07.647+0000 7fddd5400700 1 -- 172.21.15.18:0/3755834556 >> 172.21.15.18:0/3755834556 conn(0x7fddd00faba0 msgr2=0x7fddd00fcfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:30:07.646 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:07.647+0000 7fddd5400700 1 -- 172.21.15.18:0/3755834556 shutdown_connections 2024-04-12T21:30:07.646 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:07.647+0000 7fddd5400700 1 -- 172.21.15.18:0/3755834556 wait complete. 2024-04-12T21:30:07.646 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:07.648+0000 7fddd5400700 1 Processor -- start 2024-04-12T21:30:07.647 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:07.648+0000 7fddd5400700 1 -- start start 2024-04-12T21:30:07.647 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:07.649+0000 7fddd5400700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fddd0105020 0x7fddd0115bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:30:07.647 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:07.649+0000 7fddd5400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fddd01059c0 0x7fddd0116130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:30:07.647 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:07.649+0000 7fddd5400700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fddd0116810 con 0x7fddd01059c0 2024-04-12T21:30:07.647 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:07.649+0000 7fddd5400700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fddd010fd60 con 0x7fddd0105020 2024-04-12T21:30:07.648 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:07.649+0000 7fddcde00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fddd01059c0 0x7fddd0116130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:30:07.648 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:07.649+0000 7fddcde00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fddd01059c0 0x7fddd0116130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:50806/0 (socket says 172.21.15.18:50806) 2024-04-12T21:30:07.648 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:07.649+0000 7fddcde00700 1 -- 172.21.15.18:0/1539405545 learned_addr learned my addr 172.21.15.18:0/1539405545 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:30:07.648 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:07.649+0000 7fddce800700 1 --2- 172.21.15.18:0/1539405545 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fddd0105020 0x7fddd0115bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:30:07.648 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:07.650+0000 7fddcde00700 1 -- 172.21.15.18:0/1539405545 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fddd0105020 msgr2=0x7fddd0115bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:30:07.648 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:07.650+0000 7fddcde00700 1 --2- 172.21.15.18:0/1539405545 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fddd0105020 0x7fddd0115bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:30:07.648 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:07.650+0000 7fddcde00700 1 -- 172.21.15.18:0/1539405545 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fddc0005040 con 0x7fddd01059c0 2024-04-12T21:30:07.648 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:07.650+0000 7fddce800700 1 --2- 172.21.15.18:0/1539405545 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fddd0105020 0x7fddd0115bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-12T21:30:07.648 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:07.650+0000 7fddcde00700 1 --2- 172.21.15.18:0/1539405545 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fddd01059c0 0x7fddd0116130 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7fddbc00b790 tx=0x7fddbc009220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:30:07.649 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:07.650+0000 7fddc6a00700 1 -- 172.21.15.18:0/1539405545 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fddbc00f6f0 con 0x7fddd01059c0 2024-04-12T21:30:07.649 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:07.650+0000 7fddc6a00700 1 -- 172.21.15.18:0/1539405545 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fddbc00fd20 con 0x7fddd01059c0 2024-04-12T21:30:07.649 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:07.650+0000 7fddc6a00700 1 -- 172.21.15.18:0/1539405545 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fddbc0103a0 con 0x7fddd01059c0 2024-04-12T21:30:07.649 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:07.650+0000 7fddd5400700 1 -- 172.21.15.18:0/1539405545 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fddd010ffe0 con 0x7fddd01059c0 2024-04-12T21:30:07.649 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:07.651+0000 7fddd5400700 1 -- 172.21.15.18:0/1539405545 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fddd0110520 con 0x7fddd01059c0 2024-04-12T21:30:07.649 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:07.652+0000 7fddd5400700 1 -- 172.21.15.18:0/1539405545 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fddd0100770 con 0x7fddd01059c0 2024-04-12T21:30:07.650 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:07.653+0000 7fddc6a00700 1 -- 172.21.15.18:0/1539405545 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7fddbc011400 con 0x7fddd01059c0 2024-04-12T21:30:07.652 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:07.654+0000 7fddc6a00700 1 --2- 172.21.15.18:0/1539405545 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fddb406ec00 0x7fddb40710b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:30:07.652 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:07.654+0000 7fddc6a00700 1 -- 172.21.15.18:0/1539405545 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 6757+0+0 (secure 0 0 0) 0x7fddbc08e850 con 0x7fddd01059c0 2024-04-12T21:30:07.652 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:07.654+0000 7fddce800700 1 --2- 172.21.15.18:0/1539405545 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fddb406ec00 0x7fddb40710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:30:07.652 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:07.655+0000 7fddce800700 1 --2- 172.21.15.18:0/1539405545 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fddb406ec00 0x7fddb40710b0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7fddc0000c00 tx=0x7fddc0009e30 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:30:07.656 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:07.659+0000 7fddc6a00700 1 -- 172.21.15.18:0/1539405545 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7fddbc05b020 con 0x7fddd01059c0 2024-04-12T21:30:07.904 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:07.906+0000 7fddd5400700 1 -- 172.21.15.18:0/1539405545 --> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fddd010c200 con 0x7fddb406ec00 2024-04-12T21:30:07.906 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:07.908+0000 7fddc6a00700 1 -- 172.21.15.18:0/1539405545 <== mgr.34102 v2:172.21.15.18:6800/3812891919 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7fddd010c200 con 0x7fddb406ec00 2024-04-12T21:30:07.906 INFO:teuthology.orchestra.run.smithi018.stdout:{ 2024-04-12T21:30:07.906 INFO:teuthology.orchestra.run.smithi018.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:f4452def3be4d350f4a8e5feaf7ebb1511be9298fe68478ba7a2d6fbd7d69a63", 2024-04-12T21:30:07.906 INFO:teuthology.orchestra.run.smithi018.stdout: "in_progress": true, 2024-04-12T21:30:07.906 INFO:teuthology.orchestra.run.smithi018.stdout: "which": "Upgrading all daemon types on all hosts", 2024-04-12T21:30:07.907 INFO:teuthology.orchestra.run.smithi018.stdout: "services_complete": [ 2024-04-12T21:30:07.907 INFO:teuthology.orchestra.run.smithi018.stdout: "crash", 2024-04-12T21:30:07.907 INFO:teuthology.orchestra.run.smithi018.stdout: "osd", 2024-04-12T21:30:07.907 INFO:teuthology.orchestra.run.smithi018.stdout: "mon", 2024-04-12T21:30:07.907 INFO:teuthology.orchestra.run.smithi018.stdout: "mgr" 2024-04-12T21:30:07.907 INFO:teuthology.orchestra.run.smithi018.stdout: ], 2024-04-12T21:30:07.907 INFO:teuthology.orchestra.run.smithi018.stdout: "progress": "15/21 daemons upgraded", 2024-04-12T21:30:07.907 INFO:teuthology.orchestra.run.smithi018.stdout: "message": "Currently upgrading mds daemons", 2024-04-12T21:30:07.907 INFO:teuthology.orchestra.run.smithi018.stdout: "is_paused": false 2024-04-12T21:30:07.907 INFO:teuthology.orchestra.run.smithi018.stdout:} 2024-04-12T21:30:07.909 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:07.911+0000 7fddd5400700 1 -- 172.21.15.18:0/1539405545 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fddb406ec00 msgr2=0x7fddb40710b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:30:07.909 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:07.911+0000 7fddd5400700 1 --2- 172.21.15.18:0/1539405545 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fddb406ec00 0x7fddb40710b0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7fddc0000c00 tx=0x7fddc0009e30 comp rx=0 tx=0).stop 2024-04-12T21:30:07.909 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:07.911+0000 7fddd5400700 1 -- 172.21.15.18:0/1539405545 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fddd01059c0 msgr2=0x7fddd0116130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:30:07.909 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:07.911+0000 7fddd5400700 1 --2- 172.21.15.18:0/1539405545 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fddd01059c0 0x7fddd0116130 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7fddbc00b790 tx=0x7fddbc009220 comp rx=0 tx=0).stop 2024-04-12T21:30:07.910 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:07.911+0000 7fddd5400700 1 -- 172.21.15.18:0/1539405545 shutdown_connections 2024-04-12T21:30:07.910 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:07.911+0000 7fddd5400700 1 --2- 172.21.15.18:0/1539405545 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fddb406ec00 0x7fddb40710b0 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:30:07.910 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:07.911+0000 7fddd5400700 1 --2- 172.21.15.18:0/1539405545 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fddd0105020 0x7fddd0115bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:30:07.910 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:07.911+0000 7fddd5400700 1 --2- 172.21.15.18:0/1539405545 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fddd01059c0 0x7fddd0116130 unknown :-1 s=CLOSED pgs=775 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:30:07.910 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:07.911+0000 7fddd5400700 1 -- 172.21.15.18:0/1539405545 >> 172.21.15.18:0/1539405545 conn(0x7fddd00faba0 msgr2=0x7fddd00fbb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:30:07.910 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:07.911+0000 7fddd5400700 1 -- 172.21.15.18:0/1539405545 shutdown_connections 2024-04-12T21:30:07.910 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:07.912+0000 7fddd5400700 1 -- 172.21.15.18:0/1539405545 wait complete. 2024-04-12T21:30:08.031 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:30:07 smithi186 ceph-mon[86379]: from='client.35600 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:30:08.031 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:30:07 smithi186 ceph-mon[86379]: from='client.35604 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:30:08.031 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:30:07 smithi186 ceph-mon[86379]: from='client.? 172.21.15.18:0/1924619031' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T21:30:08.031 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:30:07 smithi186 ceph-mon[86379]: from='client.? 172.21.15.18:0/3359997999' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-12T21:30:08.063 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:08.065+0000 7fbffb800700 1 -- 172.21.15.18:0/413487142 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fbff4105010 msgr2=0x7fbff41053e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:30:08.063 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:08.065+0000 7fbffb800700 1 --2- 172.21.15.18:0/413487142 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fbff4105010 0x7fbff41053e0 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7fbfe8004940 tx=0x7fbfe8009e30 comp rx=0 tx=0).stop 2024-04-12T21:30:08.064 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:30:07 smithi018 ceph-mon[112590]: from='client.35600 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:30:08.064 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:30:07 smithi018 ceph-mon[112590]: from='client.35604 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:30:08.064 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:30:07 smithi018 ceph-mon[112590]: from='client.? 172.21.15.18:0/1924619031' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T21:30:08.064 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:30:07 smithi018 ceph-mon[112590]: from='client.? 172.21.15.18:0/3359997999' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-12T21:30:08.065 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:08.066+0000 7fbffb800700 1 -- 172.21.15.18:0/413487142 shutdown_connections 2024-04-12T21:30:08.065 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:08.066+0000 7fbffb800700 1 --2- 172.21.15.18:0/413487142 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbff41059b0 0x7fbff40ff020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:30:08.065 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:08.066+0000 7fbffb800700 1 --2- 172.21.15.18:0/413487142 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fbff4105010 0x7fbff41053e0 unknown :-1 s=CLOSED pgs=776 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:30:08.065 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:08.066+0000 7fbffb800700 1 -- 172.21.15.18:0/413487142 >> 172.21.15.18:0/413487142 conn(0x7fbff40faba0 msgr2=0x7fbff40fcfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:30:08.065 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:08.066+0000 7fbffb800700 1 -- 172.21.15.18:0/413487142 shutdown_connections 2024-04-12T21:30:08.065 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:08.066+0000 7fbffb800700 1 -- 172.21.15.18:0/413487142 wait complete. 2024-04-12T21:30:08.065 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:08.067+0000 7fbffb800700 1 Processor -- start 2024-04-12T21:30:08.065 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:08.067+0000 7fbffb800700 1 -- start start 2024-04-12T21:30:08.067 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:08.068+0000 7fbffb800700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbff4105010 0x7fbff4115dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:30:08.067 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:08.068+0000 7fbffb800700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fbff41059b0 0x7fbff4116300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:30:08.067 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:08.068+0000 7fbffb800700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbff4116950 con 0x7fbff41059b0 2024-04-12T21:30:08.067 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:08.068+0000 7fbffb800700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbff410fd20 con 0x7fbff4105010 2024-04-12T21:30:08.067 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:08.068+0000 7fbff3e00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fbff41059b0 0x7fbff4116300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:30:08.067 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:08.068+0000 7fbff3e00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fbff41059b0 0x7fbff4116300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:50820/0 (socket says 172.21.15.18:50820) 2024-04-12T21:30:08.068 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:08.068+0000 7fbff3e00700 1 -- 172.21.15.18:0/3384303128 learned_addr learned my addr 172.21.15.18:0/3384303128 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:30:08.068 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:08.068+0000 7fbff8e00700 1 --2- 172.21.15.18:0/3384303128 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbff4105010 0x7fbff4115dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:30:08.068 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:08.068+0000 7fbff3e00700 1 -- 172.21.15.18:0/3384303128 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbff4105010 msgr2=0x7fbff4115dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:30:08.068 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:08.068+0000 7fbff3e00700 1 --2- 172.21.15.18:0/3384303128 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbff4105010 0x7fbff4115dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:30:08.068 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:08.068+0000 7fbff3e00700 1 -- 172.21.15.18:0/3384303128 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbfe8005040 con 0x7fbff41059b0 2024-04-12T21:30:08.068 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:08.069+0000 7fbff8e00700 1 --2- 172.21.15.18:0/3384303128 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbff4105010 0x7fbff4115dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-12T21:30:08.068 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:08.069+0000 7fbff3e00700 1 --2- 172.21.15.18:0/3384303128 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fbff41059b0 0x7fbff4116300 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7fbfe0002a80 tx=0x7fbfe0004630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:30:08.068 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:08.069+0000 7fbff0c00700 1 -- 172.21.15.18:0/3384303128 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbfe0007cc0 con 0x7fbff41059b0 2024-04-12T21:30:08.068 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:08.069+0000 7fbff0c00700 1 -- 172.21.15.18:0/3384303128 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fbfe000f040 con 0x7fbff41059b0 2024-04-12T21:30:08.068 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:08.069+0000 7fbffb800700 1 -- 172.21.15.18:0/3384303128 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbff4110000 con 0x7fbff41059b0 2024-04-12T21:30:08.068 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:08.069+0000 7fbff0c00700 1 -- 172.21.15.18:0/3384303128 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbfe000daf0 con 0x7fbff41059b0 2024-04-12T21:30:08.068 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:08.069+0000 7fbffb800700 1 -- 172.21.15.18:0/3384303128 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbff41105c0 con 0x7fbff41059b0 2024-04-12T21:30:08.069 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:08.070+0000 7fbffb800700 1 -- 172.21.15.18:0/3384303128 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbff4100760 con 0x7fbff41059b0 2024-04-12T21:30:08.069 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:08.072+0000 7fbff0c00700 1 -- 172.21.15.18:0/3384303128 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7fbfe000dc50 con 0x7fbff41059b0 2024-04-12T21:30:08.071 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:08.073+0000 7fbff0c00700 1 --2- 172.21.15.18:0/3384303128 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fbfdc06ec10 0x7fbfdc0710c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:30:08.071 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:08.073+0000 7fbff0c00700 1 -- 172.21.15.18:0/3384303128 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 6757+0+0 (secure 0 0 0) 0x7fbfe008e210 con 0x7fbff41059b0 2024-04-12T21:30:08.071 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:08.074+0000 7fbff8e00700 1 --2- 172.21.15.18:0/3384303128 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fbfdc06ec10 0x7fbfdc0710c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:30:08.072 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:08.074+0000 7fbff8e00700 1 --2- 172.21.15.18:0/3384303128 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fbfdc06ec10 0x7fbfdc0710c0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7fbfe8004f90 tx=0x7fbfe8009e30 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:30:08.074 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:08.077+0000 7fbff0c00700 1 -- 172.21.15.18:0/3384303128 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7fbfe005ab00 con 0x7fbff41059b0 2024-04-12T21:30:08.394 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:08.396+0000 7fbffb800700 1 -- 172.21.15.18:0/3384303128 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7fbff405f5f0 con 0x7fbff41059b0 2024-04-12T21:30:08.394 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:08.397+0000 7fbff0c00700 1 -- 172.21.15.18:0/3384303128 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+4303 (secure 0 0 0) 0x7fbfe005a690 con 0x7fbff41059b0 2024-04-12T21:30:08.395 INFO:teuthology.orchestra.run.smithi018.stdout:HEALTH_ERR failed to probe daemons or devices; 1 filesystem is degraded; 1 filesystem is offline 2024-04-12T21:30:08.395 INFO:teuthology.orchestra.run.smithi018.stdout:[WRN] CEPHADM_REFRESH_FAILED: failed to probe daemons or devices 2024-04-12T21:30:08.395 INFO:teuthology.orchestra.run.smithi018.stdout: host smithi018 `cephadm ceph-volume` failed: cephadm exited with an error code: 1, stderr: Inferring config /var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/mon.smithi018/config 2024-04-12T21:30:08.395 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/podman: timeout after 895 seconds 2024-04-12T21:30:08.395 INFO:teuthology.orchestra.run.smithi018.stdout:Non-zero exit code 124 from /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:f4452def3be4d350f4a8e5feaf7ebb1511be9298fe68478ba7a2d6fbd7d69a63 -e NODE_NAME=smithi018 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968:/var/run/ceph:z -v /var/log/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968:/var/log/ceph:z -v /var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmpxsmwkkuh:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:f4452def3be4d350f4a8e5feaf7ebb1511be9298fe68478ba7a2d6fbd7d69a63 inventory --format=json-pretty --filter-for-batch 2024-04-12T21:30:08.395 INFO:teuthology.orchestra.run.smithi018.stdout:Traceback (most recent call last): 2024-04-12T21:30:08.395 INFO:teuthology.orchestra.run.smithi018.stdout: File "/usr/lib64/python3.9/runpy.py", line 197, in _run_module_as_main 2024-04-12T21:30:08.395 INFO:teuthology.orchestra.run.smithi018.stdout: return _run_code(code, main_globals, None, 2024-04-12T21:30:08.395 INFO:teuthology.orchestra.run.smithi018.stdout: File "/usr/lib64/python3.9/runpy.py", line 87, in _run_code 2024-04-12T21:30:08.395 INFO:teuthology.orchestra.run.smithi018.stdout: exec(code, run_globals) 2024-04-12T21:30:08.395 INFO:teuthology.orchestra.run.smithi018.stdout: File "/var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/cephadm.e44f3ed5e34e4ef52eedef233d1190bff004470d8e8005127b5312bff357fa7f/__main__.py", line 5618, in 2024-04-12T21:30:08.395 INFO:teuthology.orchestra.run.smithi018.stdout: File "/var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/cephadm.e44f3ed5e34e4ef52eedef233d1190bff004470d8e8005127b5312bff357fa7f/__main__.py", line 5606, in main 2024-04-12T21:30:08.396 INFO:teuthology.orchestra.run.smithi018.stdout: File "/var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/cephadm.e44f3ed5e34e4ef52eedef233d1190bff004470d8e8005127b5312bff357fa7f/__main__.py", line 408, in _infer_config 2024-04-12T21:30:08.396 INFO:teuthology.orchestra.run.smithi018.stdout: File "/var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/cephadm.e44f3ed5e34e4ef52eedef233d1190bff004470d8e8005127b5312bff357fa7f/__main__.py", line 323, in _infer_fsid 2024-04-12T21:30:08.396 INFO:teuthology.orchestra.run.smithi018.stdout: File "/var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/cephadm.e44f3ed5e34e4ef52eedef233d1190bff004470d8e8005127b5312bff357fa7f/__main__.py", line 436, in _infer_image 2024-04-12T21:30:08.396 INFO:teuthology.orchestra.run.smithi018.stdout: File "/var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/cephadm.e44f3ed5e34e4ef52eedef233d1190bff004470d8e8005127b5312bff357fa7f/__main__.py", line 310, in _validate_fsid 2024-04-12T21:30:08.396 INFO:teuthology.orchestra.run.smithi018.stdout: File "/var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/cephadm.e44f3ed5e34e4ef52eedef233d1190bff004470d8e8005127b5312bff357fa7f/__main__.py", line 3359, in command_ceph_volume 2024-04-12T21:30:08.396 INFO:teuthology.orchestra.run.smithi018.stdout: File "/var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/cephadm.e44f3ed5e34e4ef52eedef233d1190bff004470d8e8005127b5312bff357fa7f/cephadmlib/call_wrappers.py", line 310, in call_throws 2024-04-12T21:30:08.396 INFO:teuthology.orchestra.run.smithi018.stdout:RuntimeError: Failed command: /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:f4452def3be4d350f4a8e5feaf7ebb1511be9298fe68478ba7a2d6fbd7d69a63 -e NODE_NAME=smithi018 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968:/var/run/ceph:z -v /var/log/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968:/var/log/ceph:z -v /var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmpxsmwkkuh:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:f4452def3be4d350f4a8e5feaf7ebb1511be9298fe68478ba7a2d6fbd7d69a63 inventory --format=json-pretty --filter-for-batch 2024-04-12T21:30:08.396 INFO:teuthology.orchestra.run.smithi018.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-04-12T21:30:08.396 INFO:teuthology.orchestra.run.smithi018.stdout: fs cephfs is degraded 2024-04-12T21:30:08.396 INFO:teuthology.orchestra.run.smithi018.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-04-12T21:30:08.396 INFO:teuthology.orchestra.run.smithi018.stdout: fs cephfs is offline because no MDS is active for it. 2024-04-12T21:30:08.398 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:08.400+0000 7fbffb800700 1 -- 172.21.15.18:0/3384303128 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fbfdc06ec10 msgr2=0x7fbfdc0710c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:30:08.398 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:08.400+0000 7fbffb800700 1 --2- 172.21.15.18:0/3384303128 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fbfdc06ec10 0x7fbfdc0710c0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7fbfe8004f90 tx=0x7fbfe8009e30 comp rx=0 tx=0).stop 2024-04-12T21:30:08.398 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:08.400+0000 7fbffb800700 1 -- 172.21.15.18:0/3384303128 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fbff41059b0 msgr2=0x7fbff4116300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:30:08.398 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:08.400+0000 7fbffb800700 1 --2- 172.21.15.18:0/3384303128 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fbff41059b0 0x7fbff4116300 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7fbfe0002a80 tx=0x7fbfe0004630 comp rx=0 tx=0).stop 2024-04-12T21:30:08.398 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:08.400+0000 7fbffb800700 1 -- 172.21.15.18:0/3384303128 shutdown_connections 2024-04-12T21:30:08.399 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:08.400+0000 7fbffb800700 1 --2- 172.21.15.18:0/3384303128 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fbfdc06ec10 0x7fbfdc0710c0 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:30:08.399 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:08.400+0000 7fbffb800700 1 --2- 172.21.15.18:0/3384303128 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbff4105010 0x7fbff4115dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:30:08.399 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:08.400+0000 7fbffb800700 1 --2- 172.21.15.18:0/3384303128 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fbff41059b0 0x7fbff4116300 unknown :-1 s=CLOSED pgs=777 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:30:08.399 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:08.400+0000 7fbffb800700 1 -- 172.21.15.18:0/3384303128 >> 172.21.15.18:0/3384303128 conn(0x7fbff40faba0 msgr2=0x7fbff40fbc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:30:08.399 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:08.400+0000 7fbffb800700 1 -- 172.21.15.18:0/3384303128 shutdown_connections 2024-04-12T21:30:08.399 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:08.401+0000 7fbffb800700 1 -- 172.21.15.18:0/3384303128 wait complete. 2024-04-12T21:30:09.031 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:30:08 smithi186 ceph-mon[86379]: pgmap v911: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:30:09.031 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:30:08 smithi186 ceph-mon[86379]: from='client.? 172.21.15.18:0/3384303128' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-12T21:30:09.102 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:30:08 smithi018 ceph-mon[112590]: pgmap v911: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:30:09.102 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:30:08 smithi018 ceph-mon[112590]: from='client.? 172.21.15.18:0/3384303128' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-12T21:30:10.031 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:30:09 smithi186 ceph-mon[86379]: from='client.35616 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:30:10.102 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:30:09 smithi018 ceph-mon[112590]: from='client.35616 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:30:11.031 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:30:10 smithi186 ceph-mon[86379]: pgmap v912: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:30:11.031 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:30:10 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T21:30:11.102 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:30:10 smithi018 ceph-mon[112590]: pgmap v912: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:30:11.102 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:30:10 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T21:30:13.031 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:30:12 smithi186 ceph-mon[86379]: pgmap v913: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:30:13.102 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:30:12 smithi018 ceph-mon[112590]: pgmap v913: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:30:15.031 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:30:14 smithi186 ceph-mon[86379]: pgmap v914: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:30:15.103 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:30:14 smithi018 ceph-mon[112590]: pgmap v914: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:30:17.031 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:30:16 smithi186 ceph-mon[86379]: pgmap v915: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:30:17.102 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:30:16 smithi018 ceph-mon[112590]: pgmap v915: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:30:19.102 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:30:18 smithi018 ceph-mon[112590]: pgmap v916: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:30:19.281 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:30:18 smithi186 ceph-mon[86379]: pgmap v916: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:30:21.102 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:30:20 smithi018 ceph-mon[112590]: pgmap v917: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:30:21.281 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:30:20 smithi186 ceph-mon[86379]: pgmap v917: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:30:23.102 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:30:22 smithi018 ceph-mon[112590]: pgmap v918: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:30:23.281 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:30:22 smithi186 ceph-mon[86379]: pgmap v918: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:30:25.102 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:30:24 smithi018 ceph-mon[112590]: pgmap v919: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:30:25.281 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:30:24 smithi186 ceph-mon[86379]: pgmap v919: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:30:26.102 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:30:25 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T21:30:26.281 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:30:25 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T21:30:27.102 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:30:26 smithi018 ceph-mon[112590]: pgmap v920: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:30:27.281 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:30:26 smithi186 ceph-mon[86379]: pgmap v920: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:30:29.102 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:30:28 smithi018 ceph-mon[112590]: pgmap v921: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:30:29.281 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:30:28 smithi186 ceph-mon[86379]: pgmap v921: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:30:31.102 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:30:30 smithi018 ceph-mon[112590]: pgmap v922: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:30:31.281 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:30:30 smithi186 ceph-mon[86379]: pgmap v922: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:30:33.102 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:30:32 smithi018 ceph-mon[112590]: pgmap v923: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:30:33.281 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:30:32 smithi186 ceph-mon[86379]: pgmap v923: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:30:35.102 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:30:34 smithi018 ceph-mon[112590]: pgmap v924: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:30:35.281 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:30:34 smithi186 ceph-mon[86379]: pgmap v924: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:30:37.102 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:30:36 smithi018 ceph-mon[112590]: pgmap v925: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:30:37.281 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:30:36 smithi186 ceph-mon[86379]: pgmap v925: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:30:38.573 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:38.574+0000 7fb443e00700 1 -- 172.21.15.18:0/1282930045 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb4441092a0 msgr2=0x7fb444109670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:30:38.574 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:38.574+0000 7fb443e00700 1 --2- 172.21.15.18:0/1282930045 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb4441092a0 0x7fb444109670 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7fb434004940 tx=0x7fb434009e30 comp rx=0 tx=0).stop 2024-04-12T21:30:38.574 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:38.574+0000 7fb443e00700 1 -- 172.21.15.18:0/1282930045 shutdown_connections 2024-04-12T21:30:38.574 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:38.574+0000 7fb443e00700 1 --2- 172.21.15.18:0/1282930045 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb444109c40 0x7fb4441032b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:30:38.574 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:38.574+0000 7fb443e00700 1 --2- 172.21.15.18:0/1282930045 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb4441092a0 0x7fb444109670 unknown :-1 s=CLOSED pgs=778 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:30:38.574 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:38.574+0000 7fb443e00700 1 -- 172.21.15.18:0/1282930045 >> 172.21.15.18:0/1282930045 conn(0x7fb444072a30 msgr2=0x7fb444074e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:30:38.574 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:38.574+0000 7fb443e00700 1 -- 172.21.15.18:0/1282930045 shutdown_connections 2024-04-12T21:30:38.574 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:38.574+0000 7fb443e00700 1 -- 172.21.15.18:0/1282930045 wait complete. 2024-04-12T21:30:38.574 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:38.575+0000 7fb443e00700 1 Processor -- start 2024-04-12T21:30:38.574 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:38.575+0000 7fb443e00700 1 -- start start 2024-04-12T21:30:38.576 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:38.575+0000 7fb443e00700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb4441092a0 0x7fb444119ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:30:38.576 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:38.575+0000 7fb443e00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb444109c40 0x7fb44411a530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:30:38.576 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:38.575+0000 7fb443e00700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb44411ab80 con 0x7fb444109c40 2024-04-12T21:30:38.576 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:38.575+0000 7fb443e00700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb444113f50 con 0x7fb4441092a0 2024-04-12T21:30:38.576 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:38.576+0000 7fb442000700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb444109c40 0x7fb44411a530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:30:38.576 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:38.576+0000 7fb442000700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb444109c40 0x7fb44411a530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:55254/0 (socket says 172.21.15.18:55254) 2024-04-12T21:30:38.576 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:38.576+0000 7fb442000700 1 -- 172.21.15.18:0/3590235415 learned_addr learned my addr 172.21.15.18:0/3590235415 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:30:38.576 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:38.576+0000 7fb442a00700 1 --2- 172.21.15.18:0/3590235415 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb4441092a0 0x7fb444119ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:30:38.576 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:38.576+0000 7fb442000700 1 -- 172.21.15.18:0/3590235415 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb4441092a0 msgr2=0x7fb444119ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:30:38.576 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:38.576+0000 7fb442000700 1 --2- 172.21.15.18:0/3590235415 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb4441092a0 0x7fb444119ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:30:38.576 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:38.576+0000 7fb442000700 1 -- 172.21.15.18:0/3590235415 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb434005040 con 0x7fb444109c40 2024-04-12T21:30:38.576 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:38.577+0000 7fb442000700 1 --2- 172.21.15.18:0/3590235415 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb444109c40 0x7fb44411a530 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7fb43000c800 tx=0x7fb430004470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:30:38.576 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:38.577+0000 7fb43aa00700 1 -- 172.21.15.18:0/3590235415 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb43000a760 con 0x7fb444109c40 2024-04-12T21:30:38.577 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:38.577+0000 7fb43aa00700 1 -- 172.21.15.18:0/3590235415 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb43000ad90 con 0x7fb444109c40 2024-04-12T21:30:38.577 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:38.577+0000 7fb443e00700 1 -- 172.21.15.18:0/3590235415 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb444114230 con 0x7fb444109c40 2024-04-12T21:30:38.577 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:38.577+0000 7fb43aa00700 1 -- 172.21.15.18:0/3590235415 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb4300113a0 con 0x7fb444109c40 2024-04-12T21:30:38.577 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:38.577+0000 7fb443e00700 1 -- 172.21.15.18:0/3590235415 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb4441147f0 con 0x7fb444109c40 2024-04-12T21:30:38.577 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:38.579+0000 7fb443e00700 1 -- 172.21.15.18:0/3590235415 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb4441049f0 con 0x7fb444109c40 2024-04-12T21:30:38.578 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:38.579+0000 7fb43aa00700 1 -- 172.21.15.18:0/3590235415 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7fb430012400 con 0x7fb444109c40 2024-04-12T21:30:38.579 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:38.580+0000 7fb43aa00700 1 --2- 172.21.15.18:0/3590235415 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fb42806ec00 0x7fb4280710b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:30:38.580 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:38.581+0000 7fb43aa00700 1 -- 172.21.15.18:0/3590235415 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 6757+0+0 (secure 0 0 0) 0x7fb43008e8d0 con 0x7fb444109c40 2024-04-12T21:30:38.580 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:38.581+0000 7fb442a00700 1 --2- 172.21.15.18:0/3590235415 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fb42806ec00 0x7fb4280710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:30:38.580 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:38.582+0000 7fb442a00700 1 --2- 172.21.15.18:0/3590235415 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fb42806ec00 0x7fb4280710b0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7fb43400b020 tx=0x7fb434009e30 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:30:38.584 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:38.586+0000 7fb43aa00700 1 -- 172.21.15.18:0/3590235415 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7fb43005b1c0 con 0x7fb444109c40 2024-04-12T21:30:38.842 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:38.843+0000 7fb443e00700 1 -- 172.21.15.18:0/3590235415 --> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fb444110400 con 0x7fb42806ec00 2024-04-12T21:30:38.845 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:38.846+0000 7fb43aa00700 1 -- 172.21.15.18:0/3590235415 <== mgr.34102 v2:172.21.15.18:6800/3812891919 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7fb444110400 con 0x7fb42806ec00 2024-04-12T21:30:38.848 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:38.849+0000 7fb443e00700 1 -- 172.21.15.18:0/3590235415 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fb42806ec00 msgr2=0x7fb4280710b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:30:38.849 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:38.849+0000 7fb443e00700 1 --2- 172.21.15.18:0/3590235415 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fb42806ec00 0x7fb4280710b0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7fb43400b020 tx=0x7fb434009e30 comp rx=0 tx=0).stop 2024-04-12T21:30:38.849 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:38.850+0000 7fb443e00700 1 -- 172.21.15.18:0/3590235415 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb444109c40 msgr2=0x7fb44411a530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:30:38.849 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:38.850+0000 7fb443e00700 1 --2- 172.21.15.18:0/3590235415 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb444109c40 0x7fb44411a530 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7fb43000c800 tx=0x7fb430004470 comp rx=0 tx=0).stop 2024-04-12T21:30:38.849 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:38.850+0000 7fb443e00700 1 -- 172.21.15.18:0/3590235415 shutdown_connections 2024-04-12T21:30:38.849 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:38.850+0000 7fb443e00700 1 --2- 172.21.15.18:0/3590235415 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fb42806ec00 0x7fb4280710b0 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:30:38.849 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:38.850+0000 7fb443e00700 1 --2- 172.21.15.18:0/3590235415 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb4441092a0 0x7fb444119ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:30:38.849 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:38.850+0000 7fb443e00700 1 --2- 172.21.15.18:0/3590235415 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb444109c40 0x7fb44411a530 unknown :-1 s=CLOSED pgs=779 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:30:38.850 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:38.850+0000 7fb443e00700 1 -- 172.21.15.18:0/3590235415 >> 172.21.15.18:0/3590235415 conn(0x7fb444072a30 msgr2=0x7fb4440748d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:30:38.850 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:38.850+0000 7fb443e00700 1 -- 172.21.15.18:0/3590235415 shutdown_connections 2024-04-12T21:30:38.850 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:38.850+0000 7fb443e00700 1 -- 172.21.15.18:0/3590235415 wait complete. 2024-04-12T21:30:38.867 INFO:teuthology.orchestra.run.smithi018.stdout:true 2024-04-12T21:30:39.016 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:39.017+0000 7f0788000700 1 -- 172.21.15.18:0/4105866651 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f07800fe830 msgr2=0x7f0780106f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:30:39.016 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:39.017+0000 7f0788000700 1 --2- 172.21.15.18:0/4105866651 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f07800fe830 0x7f0780106f00 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7f076c004940 tx=0x7f076c009e30 comp rx=0 tx=0).stop 2024-04-12T21:30:39.016 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:39.017+0000 7f0788000700 1 -- 172.21.15.18:0/4105866651 shutdown_connections 2024-04-12T21:30:39.016 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:39.017+0000 7f0788000700 1 --2- 172.21.15.18:0/4105866651 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f07800fe830 0x7f0780106f00 unknown :-1 s=CLOSED pgs=780 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:30:39.016 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:39.017+0000 7f0788000700 1 --2- 172.21.15.18:0/4105866651 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f07800fde90 0x7f07800fe260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:30:39.016 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:39.017+0000 7f0788000700 1 -- 172.21.15.18:0/4105866651 >> 172.21.15.18:0/4105866651 conn(0x7f0780072bc0 msgr2=0x7f0780072fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:30:39.017 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:39.017+0000 7f0788000700 1 -- 172.21.15.18:0/4105866651 shutdown_connections 2024-04-12T21:30:39.017 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:39.017+0000 7f0788000700 1 -- 172.21.15.18:0/4105866651 wait complete. 2024-04-12T21:30:39.018 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:39.018+0000 7f0788000700 1 Processor -- start 2024-04-12T21:30:39.018 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:39.018+0000 7f0788000700 1 -- start start 2024-04-12T21:30:39.018 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:39.018+0000 7f0788000700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f07800fde90 0x7f07800fbd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:30:39.018 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:39.018+0000 7f0788000700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f07800fe830 0x7f07800fc290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:30:39.018 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:39.018+0000 7f0788000700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07800fc820 con 0x7f07800fde90 2024-04-12T21:30:39.018 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:39.018+0000 7f0788000700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07800fc960 con 0x7f07800fe830 2024-04-12T21:30:39.018 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:39.019+0000 7f0785600700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f07800fde90 0x7f07800fbd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:30:39.018 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:39.019+0000 7f0785600700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f07800fde90 0x7f07800fbd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:55278/0 (socket says 172.21.15.18:55278) 2024-04-12T21:30:39.018 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:39.019+0000 7f0785600700 1 -- 172.21.15.18:0/1447091460 learned_addr learned my addr 172.21.15.18:0/1447091460 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:30:39.019 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:39.019+0000 7f0784c00700 1 --2- 172.21.15.18:0/1447091460 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f07800fe830 0x7f07800fc290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:30:39.019 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:39.019+0000 7f0785600700 1 -- 172.21.15.18:0/1447091460 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f07800fe830 msgr2=0x7f07800fc290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:30:39.019 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:39.019+0000 7f0785600700 1 --2- 172.21.15.18:0/1447091460 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f07800fe830 0x7f07800fc290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:30:39.019 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:39.019+0000 7f0785600700 1 -- 172.21.15.18:0/1447091460 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f076c005040 con 0x7f07800fde90 2024-04-12T21:30:39.019 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:39.020+0000 7f0785600700 1 --2- 172.21.15.18:0/1447091460 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f07800fde90 0x7f07800fbd50 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7f0774002a50 tx=0x7f07740046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:30:39.020 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:39.020+0000 7f077d600700 1 -- 172.21.15.18:0/1447091460 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0774007cc0 con 0x7f07800fde90 2024-04-12T21:30:39.020 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:39.020+0000 7f077d600700 1 -- 172.21.15.18:0/1447091460 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f077400f040 con 0x7f07800fde90 2024-04-12T21:30:39.020 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:39.020+0000 7f0788000700 1 -- 172.21.15.18:0/1447091460 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0780119ea0 con 0x7f07800fde90 2024-04-12T21:30:39.020 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:39.020+0000 7f077d600700 1 -- 172.21.15.18:0/1447091460 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0774009640 con 0x7f07800fde90 2024-04-12T21:30:39.020 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:39.020+0000 7f0788000700 1 -- 172.21.15.18:0/1447091460 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f078011a2a0 con 0x7f07800fde90 2024-04-12T21:30:39.020 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:39.021+0000 7f0788000700 1 -- 172.21.15.18:0/1447091460 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0780065e30 con 0x7f07800fde90 2024-04-12T21:30:39.021 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:39.022+0000 7f077d600700 1 -- 172.21.15.18:0/1447091460 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7f07740097a0 con 0x7f07800fde90 2024-04-12T21:30:39.022 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:39.023+0000 7f077d600700 1 --2- 172.21.15.18:0/1447091460 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f076806ec00 0x7f07680710b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:30:39.022 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:39.024+0000 7f077d600700 1 -- 172.21.15.18:0/1447091460 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 6757+0+0 (secure 0 0 0) 0x7f077408e810 con 0x7f07800fde90 2024-04-12T21:30:39.023 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:39.024+0000 7f0784c00700 1 --2- 172.21.15.18:0/1447091460 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f076806ec00 0x7f07680710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:30:39.023 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:39.024+0000 7f0784c00700 1 --2- 172.21.15.18:0/1447091460 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f076806ec00 0x7f07680710b0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f07800fd690 tx=0x7f076c009e30 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:30:39.027 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:39.028+0000 7f077d600700 1 -- 172.21.15.18:0/1447091460 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f0774057410 con 0x7f07800fde90 2024-04-12T21:30:39.102 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:30:38 smithi018 ceph-mon[112590]: pgmap v926: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:30:39.281 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:30:38 smithi186 ceph-mon[86379]: pgmap v926: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:30:39.286 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:39.284+0000 7f0788000700 1 -- 172.21.15.18:0/1447091460 --> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f07800fd440 con 0x7f076806ec00 2024-04-12T21:30:39.286 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:39.287+0000 7f077d600700 1 -- 172.21.15.18:0/1447091460 <== mgr.34102 v2:172.21.15.18:6800/3812891919 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f07800fd440 con 0x7f076806ec00 2024-04-12T21:30:39.289 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:39.290+0000 7f0788000700 1 -- 172.21.15.18:0/1447091460 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f076806ec00 msgr2=0x7f07680710b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:30:39.289 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:39.290+0000 7f0788000700 1 --2- 172.21.15.18:0/1447091460 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f076806ec00 0x7f07680710b0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f07800fd690 tx=0x7f076c009e30 comp rx=0 tx=0).stop 2024-04-12T21:30:39.289 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:39.290+0000 7f0788000700 1 -- 172.21.15.18:0/1447091460 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f07800fde90 msgr2=0x7f07800fbd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:30:39.289 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:39.290+0000 7f0788000700 1 --2- 172.21.15.18:0/1447091460 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f07800fde90 0x7f07800fbd50 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7f0774002a50 tx=0x7f07740046b0 comp rx=0 tx=0).stop 2024-04-12T21:30:39.289 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:39.290+0000 7f0788000700 1 -- 172.21.15.18:0/1447091460 shutdown_connections 2024-04-12T21:30:39.289 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:39.290+0000 7f0788000700 1 --2- 172.21.15.18:0/1447091460 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f076806ec00 0x7f07680710b0 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:30:39.289 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:39.290+0000 7f0788000700 1 --2- 172.21.15.18:0/1447091460 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f07800fde90 0x7f07800fbd50 unknown :-1 s=CLOSED pgs=781 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:30:39.289 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:39.290+0000 7f0788000700 1 --2- 172.21.15.18:0/1447091460 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f07800fe830 0x7f07800fc290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:30:39.289 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:39.290+0000 7f0788000700 1 -- 172.21.15.18:0/1447091460 >> 172.21.15.18:0/1447091460 conn(0x7f0780072bc0 msgr2=0x7f07801037b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:30:39.289 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:39.290+0000 7f0788000700 1 -- 172.21.15.18:0/1447091460 shutdown_connections 2024-04-12T21:30:39.290 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:39.290+0000 7f0788000700 1 -- 172.21.15.18:0/1447091460 wait complete. 2024-04-12T21:30:39.447 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:39.448+0000 7fb26e400700 1 -- 172.21.15.18:0/2478351475 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb268102e10 msgr2=0x7fb2681031e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:30:39.447 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:39.448+0000 7fb26e400700 1 --2- 172.21.15.18:0/2478351475 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb268102e10 0x7fb2681031e0 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7fb258004940 tx=0x7fb258009e30 comp rx=0 tx=0).stop 2024-04-12T21:30:39.448 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:39.448+0000 7fb26e400700 1 -- 172.21.15.18:0/2478351475 shutdown_connections 2024-04-12T21:30:39.448 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:39.448+0000 7fb26e400700 1 --2- 172.21.15.18:0/2478351475 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb2681037b0 0x7fb2680fce20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:30:39.448 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:39.448+0000 7fb26e400700 1 --2- 172.21.15.18:0/2478351475 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb268102e10 0x7fb2681031e0 unknown :-1 s=CLOSED pgs=782 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:30:39.448 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:39.448+0000 7fb26e400700 1 -- 172.21.15.18:0/2478351475 >> 172.21.15.18:0/2478351475 conn(0x7fb2680f8980 msgr2=0x7fb2680fad90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:30:39.448 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:39.448+0000 7fb26e400700 1 -- 172.21.15.18:0/2478351475 shutdown_connections 2024-04-12T21:30:39.448 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:39.448+0000 7fb26e400700 1 -- 172.21.15.18:0/2478351475 wait complete. 2024-04-12T21:30:39.449 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:39.449+0000 7fb26e400700 1 Processor -- start 2024-04-12T21:30:39.449 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:39.449+0000 7fb26e400700 1 -- start start 2024-04-12T21:30:39.449 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:39.450+0000 7fb26e400700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb268102e10 0x7fb268115de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:30:39.449 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:39.450+0000 7fb26e400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb2681037b0 0x7fb268116320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:30:39.449 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:39.450+0000 7fb26e400700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb268116970 con 0x7fb2681037b0 2024-04-12T21:30:39.450 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:39.450+0000 7fb26e400700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb26810fd90 con 0x7fb268102e10 2024-04-12T21:30:39.450 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:39.450+0000 7fb266a00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb2681037b0 0x7fb268116320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:30:39.450 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:39.450+0000 7fb266a00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb2681037b0 0x7fb268116320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:55298/0 (socket says 172.21.15.18:55298) 2024-04-12T21:30:39.450 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:39.450+0000 7fb266a00700 1 -- 172.21.15.18:0/2887189547 learned_addr learned my addr 172.21.15.18:0/2887189547 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:30:39.450 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:39.450+0000 7fb267400700 1 --2- 172.21.15.18:0/2887189547 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb268102e10 0x7fb268115de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:30:39.450 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:39.450+0000 7fb266a00700 1 -- 172.21.15.18:0/2887189547 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb268102e10 msgr2=0x7fb268115de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:30:39.450 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:39.451+0000 7fb266a00700 1 --2- 172.21.15.18:0/2887189547 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb268102e10 0x7fb268115de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:30:39.450 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:39.451+0000 7fb266a00700 1 -- 172.21.15.18:0/2887189547 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb258005040 con 0x7fb2681037b0 2024-04-12T21:30:39.451 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:39.451+0000 7fb266a00700 1 --2- 172.21.15.18:0/2887189547 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb2681037b0 0x7fb268116320 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7fb254002a50 tx=0x7fb254004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:30:39.451 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:39.451+0000 7fb25f400700 1 -- 172.21.15.18:0/2887189547 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb254007cc0 con 0x7fb2681037b0 2024-04-12T21:30:39.451 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:39.451+0000 7fb25f400700 1 -- 172.21.15.18:0/2887189547 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb25400f040 con 0x7fb2681037b0 2024-04-12T21:30:39.451 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:39.451+0000 7fb26e400700 1 -- 172.21.15.18:0/2887189547 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb268110070 con 0x7fb2681037b0 2024-04-12T21:30:39.451 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:39.451+0000 7fb25f400700 1 -- 172.21.15.18:0/2887189547 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb25400da40 con 0x7fb2681037b0 2024-04-12T21:30:39.451 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:39.451+0000 7fb26e400700 1 -- 172.21.15.18:0/2887189547 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb268110630 con 0x7fb2681037b0 2024-04-12T21:30:39.451 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:39.452+0000 7fb26e400700 1 -- 172.21.15.18:0/2887189547 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb268106ab0 con 0x7fb2681037b0 2024-04-12T21:30:39.452 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:39.454+0000 7fb25f400700 1 -- 172.21.15.18:0/2887189547 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7fb2540097c0 con 0x7fb2681037b0 2024-04-12T21:30:39.453 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:39.455+0000 7fb25f400700 1 --2- 172.21.15.18:0/2887189547 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fb24c06ec10 0x7fb24c0710c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:30:39.453 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:39.455+0000 7fb25f400700 1 -- 172.21.15.18:0/2887189547 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 6757+0+0 (secure 0 0 0) 0x7fb25408e7b0 con 0x7fb2681037b0 2024-04-12T21:30:39.454 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:39.455+0000 7fb267400700 1 --2- 172.21.15.18:0/2887189547 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fb24c06ec10 0x7fb24c0710c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:30:39.454 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:39.456+0000 7fb267400700 1 --2- 172.21.15.18:0/2887189547 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fb24c06ec10 0x7fb24c0710c0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7fb25800b020 tx=0x7fb258009e30 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:30:39.458 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:39.459+0000 7fb25f400700 1 -- 172.21.15.18:0/2887189547 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7fb25405acd0 con 0x7fb2681037b0 2024-04-12T21:30:39.700 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:39.701+0000 7fb26e400700 1 -- 172.21.15.18:0/2887189547 --> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fb268109f10 con 0x7fb24c06ec10 2024-04-12T21:30:39.716 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:39.717+0000 7fb25f400700 1 -- 172.21.15.18:0/2887189547 <== mgr.34102 v2:172.21.15.18:6800/3812891919 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7fb268109f10 con 0x7fb24c06ec10 2024-04-12T21:30:39.717 INFO:teuthology.orchestra.run.smithi018.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-12T21:30:39.717 INFO:teuthology.orchestra.run.smithi018.stdout:alertmanager.smithi018 smithi018 *:9093,9094 running (31m) 7m ago 45m 15.5M - 0.25.0 c8568f914cd2 c332090068e8 2024-04-12T21:30:39.717 INFO:teuthology.orchestra.run.smithi018.stdout:crash.smithi018 smithi018 running (27m) 7m ago 46m 6665k - 19.0.0-2879-g324baf8d 8717720dc5e5 b7f80d1f7982 2024-04-12T21:30:39.717 INFO:teuthology.orchestra.run.smithi018.stdout:crash.smithi186 smithi186 running (27m) 7m ago 44m 6660k - 19.0.0-2879-g324baf8d 8717720dc5e5 a4f55c81b623 2024-04-12T21:30:39.717 INFO:teuthology.orchestra.run.smithi018.stdout:grafana.smithi018 smithi018 *:3000 running (30m) 7m ago 45m 74.8M - 9.4.12 2bacad6d85d8 b08e3bf176dd 2024-04-12T21:30:39.717 INFO:teuthology.orchestra.run.smithi018.stdout:mds.cephfs.smithi018.acqbsc smithi018 running (23m) 7m ago 38m 16.7M - 19.0.0-2879-g324baf8d 8717720dc5e5 021f99ab24ae 2024-04-12T21:30:39.717 INFO:teuthology.orchestra.run.smithi018.stdout:mds.cephfs.smithi018.szjluo smithi018 running (23m) 7m ago 38m 16.5M - 19.0.0-2879-g324baf8d 8717720dc5e5 0003117479d4 2024-04-12T21:30:39.717 INFO:teuthology.orchestra.run.smithi018.stdout:mds.cephfs.smithi186.jdllyn smithi186 running (7m) 7m ago 38m 11.0M - 19.0.0-2879-g324baf8d 8717720dc5e5 21b1f46eef23 2024-04-12T21:30:39.717 INFO:teuthology.orchestra.run.smithi018.stdout:mds.cephfs.smithi186.wjnafs smithi186 running (38m) 7m ago 38m 19.5M - 17.2.7-827-gd3b5060a 21de9b7369ad 74b115d6d6fd 2024-04-12T21:30:39.717 INFO:teuthology.orchestra.run.smithi018.stdout:mgr.smithi018.xjzwga smithi018 *:8443,9283,8765 running (33m) 7m ago 47m 512M - 19.0.0-2879-g324baf8d 8717720dc5e5 56088305eab7 2024-04-12T21:30:39.717 INFO:teuthology.orchestra.run.smithi018.stdout:mgr.smithi186.kqcfxk smithi186 *:8443,9283,8765 running (32m) 7m ago 44m 459M - 19.0.0-2879-g324baf8d 8717720dc5e5 09cd766e5e0d 2024-04-12T21:30:39.717 INFO:teuthology.orchestra.run.smithi018.stdout:mon.smithi018 smithi018 running (29m) 7m ago 47m 73.1M 2048M 19.0.0-2879-g324baf8d 8717720dc5e5 df3e201dfed6 2024-04-12T21:30:39.717 INFO:teuthology.orchestra.run.smithi018.stdout:mon.smithi186 smithi186 running (28m) 7m ago 44m 47.2M 2048M 19.0.0-2879-g324baf8d 8717720dc5e5 c77498da9b2b 2024-04-12T21:30:39.718 INFO:teuthology.orchestra.run.smithi018.stdout:node-exporter.smithi018 smithi018 *:9100 running (32m) 7m ago 45m 8108k - 1.5.0 0da6a335fe13 326c3d9c6847 2024-04-12T21:30:39.718 INFO:teuthology.orchestra.run.smithi018.stdout:node-exporter.smithi186 smithi186 *:9100 running (32m) 7m ago 44m 8757k - 1.5.0 0da6a335fe13 ec2112baeecf 2024-04-12T21:30:39.718 INFO:teuthology.orchestra.run.smithi018.stdout:osd.0 smithi018 running (26m) 7m ago 43m 123M 1559M 19.0.0-2879-g324baf8d 8717720dc5e5 60b1273ca047 2024-04-12T21:30:39.718 INFO:teuthology.orchestra.run.smithi018.stdout:osd.1 smithi018 running (26m) 7m ago 42m 95.1M 1559M 19.0.0-2879-g324baf8d 8717720dc5e5 170ce1e5fb8d 2024-04-12T21:30:39.718 INFO:teuthology.orchestra.run.smithi018.stdout:osd.2 smithi018 running (25m) 7m ago 42m 94.3M 1559M 19.0.0-2879-g324baf8d 8717720dc5e5 c2cbc6aa2839 2024-04-12T21:30:39.718 INFO:teuthology.orchestra.run.smithi018.stdout:osd.3 smithi186 running (25m) 7m ago 41m 101M 2581M 19.0.0-2879-g324baf8d 8717720dc5e5 0138e8262e4b 2024-04-12T21:30:39.718 INFO:teuthology.orchestra.run.smithi018.stdout:osd.4 smithi186 running (24m) 7m ago 41m 91.0M 2581M 19.0.0-2879-g324baf8d 8717720dc5e5 b20cb63b25c9 2024-04-12T21:30:39.718 INFO:teuthology.orchestra.run.smithi018.stdout:osd.5 smithi186 running (23m) 7m ago 40m 92.0M 2581M 19.0.0-2879-g324baf8d 8717720dc5e5 2513bea2610e 2024-04-12T21:30:39.718 INFO:teuthology.orchestra.run.smithi018.stdout:prometheus.smithi018 smithi018 *:9095 running (31m) 7m ago 45m 67.9M - 2.43.0 a07b618ecd1d f58841127e21 2024-04-12T21:30:39.719 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:39.720+0000 7fb26e400700 1 -- 172.21.15.18:0/2887189547 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fb24c06ec10 msgr2=0x7fb24c0710c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:30:39.719 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:39.720+0000 7fb26e400700 1 --2- 172.21.15.18:0/2887189547 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fb24c06ec10 0x7fb24c0710c0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7fb25800b020 tx=0x7fb258009e30 comp rx=0 tx=0).stop 2024-04-12T21:30:39.719 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:39.720+0000 7fb26e400700 1 -- 172.21.15.18:0/2887189547 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb2681037b0 msgr2=0x7fb268116320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:30:39.720 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:39.720+0000 7fb26e400700 1 --2- 172.21.15.18:0/2887189547 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb2681037b0 0x7fb268116320 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7fb254002a50 tx=0x7fb254004600 comp rx=0 tx=0).stop 2024-04-12T21:30:39.720 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:39.720+0000 7fb26e400700 1 -- 172.21.15.18:0/2887189547 shutdown_connections 2024-04-12T21:30:39.720 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:39.720+0000 7fb26e400700 1 --2- 172.21.15.18:0/2887189547 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fb24c06ec10 0x7fb24c0710c0 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:30:39.720 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:39.720+0000 7fb26e400700 1 --2- 172.21.15.18:0/2887189547 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb268102e10 0x7fb268115de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:30:39.720 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:39.720+0000 7fb26e400700 1 --2- 172.21.15.18:0/2887189547 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb2681037b0 0x7fb268116320 unknown :-1 s=CLOSED pgs=783 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:30:39.720 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:39.720+0000 7fb26e400700 1 -- 172.21.15.18:0/2887189547 >> 172.21.15.18:0/2887189547 conn(0x7fb2680f8980 msgr2=0x7fb268107400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:30:39.720 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:39.721+0000 7fb26e400700 1 -- 172.21.15.18:0/2887189547 shutdown_connections 2024-04-12T21:30:39.720 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:39.721+0000 7fb26e400700 1 -- 172.21.15.18:0/2887189547 wait complete. 2024-04-12T21:30:39.873 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:39.874+0000 7f5380a00700 1 -- 172.21.15.18:0/622452030 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f537c1037c0 msgr2=0x7f537c0fce30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:30:39.874 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:39.874+0000 7f5380a00700 1 --2- 172.21.15.18:0/622452030 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f537c1037c0 0x7f537c0fce30 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7f5368004940 tx=0x7f5368009e30 comp rx=0 tx=0).stop 2024-04-12T21:30:39.874 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:39.874+0000 7f5380a00700 1 -- 172.21.15.18:0/622452030 shutdown_connections 2024-04-12T21:30:39.874 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:39.874+0000 7f5380a00700 1 --2- 172.21.15.18:0/622452030 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f537c1037c0 0x7f537c0fce30 unknown :-1 s=CLOSED pgs=784 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:30:39.874 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:39.874+0000 7f5380a00700 1 --2- 172.21.15.18:0/622452030 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f537c102e20 0x7f537c1031f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:30:39.874 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:39.874+0000 7f5380a00700 1 -- 172.21.15.18:0/622452030 >> 172.21.15.18:0/622452030 conn(0x7f537c0f8970 msgr2=0x7f537c0fad80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:30:39.874 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:39.874+0000 7f5380a00700 1 -- 172.21.15.18:0/622452030 shutdown_connections 2024-04-12T21:30:39.874 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:39.875+0000 7f5380a00700 1 -- 172.21.15.18:0/622452030 wait complete. 2024-04-12T21:30:39.875 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:39.875+0000 7f5380a00700 1 Processor -- start 2024-04-12T21:30:39.875 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:39.875+0000 7f5380a00700 1 -- start start 2024-04-12T21:30:39.875 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:39.876+0000 7f5380a00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f537c102e20 0x7f537c115de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:30:39.876 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:39.876+0000 7f5380a00700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f537c1037c0 0x7f537c116320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:30:39.876 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:39.876+0000 7f5380a00700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f537c116970 con 0x7f537c102e20 2024-04-12T21:30:39.876 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:39.876+0000 7f5380a00700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f537c10fd40 con 0x7f537c1037c0 2024-04-12T21:30:39.876 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:39.876+0000 7f5379e00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f537c102e20 0x7f537c115de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:30:39.876 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:39.876+0000 7f5379e00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f537c102e20 0x7f537c115de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:55314/0 (socket says 172.21.15.18:55314) 2024-04-12T21:30:39.876 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:39.876+0000 7f5379e00700 1 -- 172.21.15.18:0/1935045714 learned_addr learned my addr 172.21.15.18:0/1935045714 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:30:39.876 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:39.877+0000 7f5379400700 1 --2- 172.21.15.18:0/1935045714 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f537c1037c0 0x7f537c116320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:30:39.876 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:39.877+0000 7f5379e00700 1 -- 172.21.15.18:0/1935045714 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f537c1037c0 msgr2=0x7f537c116320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:30:39.876 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:39.877+0000 7f5379e00700 1 --2- 172.21.15.18:0/1935045714 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f537c1037c0 0x7f537c116320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:30:39.876 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:39.877+0000 7f5379e00700 1 -- 172.21.15.18:0/1935045714 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5368005040 con 0x7f537c102e20 2024-04-12T21:30:39.877 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:39.877+0000 7f5379e00700 1 --2- 172.21.15.18:0/1935045714 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f537c102e20 0x7f537c115de0 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7f536c00c8c0 tx=0x7f536c0044c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:30:39.877 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:39.877+0000 7f5372000700 1 -- 172.21.15.18:0/1935045714 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f536c00a980 con 0x7f537c102e20 2024-04-12T21:30:39.877 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:39.877+0000 7f5372000700 1 -- 172.21.15.18:0/1935045714 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f536c00ce60 con 0x7f537c102e20 2024-04-12T21:30:39.877 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:39.877+0000 7f5380a00700 1 -- 172.21.15.18:0/1935045714 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f537c110020 con 0x7f537c102e20 2024-04-12T21:30:39.877 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:39.878+0000 7f5372000700 1 -- 172.21.15.18:0/1935045714 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f536c0115e0 con 0x7f537c102e20 2024-04-12T21:30:39.877 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:39.878+0000 7f5380a00700 1 -- 172.21.15.18:0/1935045714 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f537c1105e0 con 0x7f537c102e20 2024-04-12T21:30:39.877 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:39.879+0000 7f5380a00700 1 -- 172.21.15.18:0/1935045714 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f537c102140 con 0x7f537c102e20 2024-04-12T21:30:39.878 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:39.880+0000 7f5372000700 1 -- 172.21.15.18:0/1935045714 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7f536c012400 con 0x7f537c102e20 2024-04-12T21:30:39.879 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:39.880+0000 7f5372000700 1 --2- 172.21.15.18:0/1935045714 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f536006ec00 0x7f53600710b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:30:39.879 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:39.881+0000 7f5372000700 1 -- 172.21.15.18:0/1935045714 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 6757+0+0 (secure 0 0 0) 0x7f536c08ea40 con 0x7f537c102e20 2024-04-12T21:30:39.880 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:39.881+0000 7f5379400700 1 --2- 172.21.15.18:0/1935045714 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f536006ec00 0x7f53600710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:30:39.880 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:39.881+0000 7f5379400700 1 --2- 172.21.15.18:0/1935045714 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f536006ec00 0x7f53600710b0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f536800b3b0 tx=0x7f5368009e30 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:30:39.884 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:39.886+0000 7f5372000700 1 -- 172.21.15.18:0/1935045714 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f536c05b270 con 0x7f537c102e20 2024-04-12T21:30:40.102 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:30:39 smithi018 ceph-mon[112590]: from='client.35624 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:30:40.217 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:40.217+0000 7f5380a00700 1 -- 172.21.15.18:0/1935045714 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f537c109f00 con 0x7f537c102e20 2024-04-12T21:30:40.217 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:40.218+0000 7f5372000700 1 -- 172.21.15.18:0/1935045714 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+837 (secure 0 0 0) 0x7f536c0126b0 con 0x7f537c102e20 2024-04-12T21:30:40.218 INFO:teuthology.orchestra.run.smithi018.stdout:{ 2024-04-12T21:30:40.218 INFO:teuthology.orchestra.run.smithi018.stdout: "mon": { 2024-04-12T21:30:40.218 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 19.0.0-2879-g324baf8d (324baf8ddb91dea46cac50cb535c306115791355) squid (dev)": 2 2024-04-12T21:30:40.218 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-12T21:30:40.218 INFO:teuthology.orchestra.run.smithi018.stdout: "mgr": { 2024-04-12T21:30:40.218 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 19.0.0-2879-g324baf8d (324baf8ddb91dea46cac50cb535c306115791355) squid (dev)": 2 2024-04-12T21:30:40.218 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-12T21:30:40.218 INFO:teuthology.orchestra.run.smithi018.stdout: "osd": { 2024-04-12T21:30:40.218 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 19.0.0-2879-g324baf8d (324baf8ddb91dea46cac50cb535c306115791355) squid (dev)": 6 2024-04-12T21:30:40.218 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-12T21:30:40.218 INFO:teuthology.orchestra.run.smithi018.stdout: "mds": { 2024-04-12T21:30:40.218 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 17.2.7-827-gd3b5060a (d3b5060a17449edfae9385cdf792afec109fca5d) quincy (stable)": 1, 2024-04-12T21:30:40.219 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 19.0.0-2879-g324baf8d (324baf8ddb91dea46cac50cb535c306115791355) squid (dev)": 3 2024-04-12T21:30:40.219 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-12T21:30:40.219 INFO:teuthology.orchestra.run.smithi018.stdout: "overall": { 2024-04-12T21:30:40.219 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 17.2.7-827-gd3b5060a (d3b5060a17449edfae9385cdf792afec109fca5d) quincy (stable)": 1, 2024-04-12T21:30:40.219 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 19.0.0-2879-g324baf8d (324baf8ddb91dea46cac50cb535c306115791355) squid (dev)": 13 2024-04-12T21:30:40.219 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-04-12T21:30:40.219 INFO:teuthology.orchestra.run.smithi018.stdout:} 2024-04-12T21:30:40.220 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:40.221+0000 7f5380a00700 1 -- 172.21.15.18:0/1935045714 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f536006ec00 msgr2=0x7f53600710b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:30:40.220 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:40.221+0000 7f5380a00700 1 --2- 172.21.15.18:0/1935045714 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f536006ec00 0x7f53600710b0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f536800b3b0 tx=0x7f5368009e30 comp rx=0 tx=0).stop 2024-04-12T21:30:40.220 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:40.221+0000 7f5380a00700 1 -- 172.21.15.18:0/1935045714 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f537c102e20 msgr2=0x7f537c115de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:30:40.221 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:40.221+0000 7f5380a00700 1 --2- 172.21.15.18:0/1935045714 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f537c102e20 0x7f537c115de0 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7f536c00c8c0 tx=0x7f536c0044c0 comp rx=0 tx=0).stop 2024-04-12T21:30:40.221 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:40.222+0000 7f5380a00700 1 -- 172.21.15.18:0/1935045714 shutdown_connections 2024-04-12T21:30:40.221 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:40.222+0000 7f5380a00700 1 --2- 172.21.15.18:0/1935045714 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f536006ec00 0x7f53600710b0 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:30:40.221 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:40.222+0000 7f5380a00700 1 --2- 172.21.15.18:0/1935045714 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f537c102e20 0x7f537c115de0 unknown :-1 s=CLOSED pgs=785 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:30:40.221 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:40.222+0000 7f5380a00700 1 --2- 172.21.15.18:0/1935045714 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f537c1037c0 0x7f537c116320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:30:40.221 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:40.222+0000 7f5380a00700 1 -- 172.21.15.18:0/1935045714 >> 172.21.15.18:0/1935045714 conn(0x7f537c0f8970 msgr2=0x7f537c0fa6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:30:40.222 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:40.222+0000 7f5380a00700 1 -- 172.21.15.18:0/1935045714 shutdown_connections 2024-04-12T21:30:40.222 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:40.222+0000 7f5380a00700 1 -- 172.21.15.18:0/1935045714 wait complete. 2024-04-12T21:30:40.281 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:30:39 smithi186 ceph-mon[86379]: from='client.35624 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:30:40.374 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:40.374+0000 7f683e400700 1 -- 172.21.15.18:0/1024731525 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6838105a30 msgr2=0x7f68380ff0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:30:40.374 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:40.374+0000 7f683e400700 1 --2- 172.21.15.18:0/1024731525 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6838105a30 0x7f68380ff0a0 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7f6824004940 tx=0x7f6824009e30 comp rx=0 tx=0).stop 2024-04-12T21:30:40.374 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:40.375+0000 7f683e400700 1 -- 172.21.15.18:0/1024731525 shutdown_connections 2024-04-12T21:30:40.374 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:40.375+0000 7f683e400700 1 --2- 172.21.15.18:0/1024731525 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6838105a30 0x7f68380ff0a0 unknown :-1 s=CLOSED pgs=786 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:30:40.374 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:40.375+0000 7f683e400700 1 --2- 172.21.15.18:0/1024731525 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6838105090 0x7f6838105460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:30:40.374 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:40.375+0000 7f683e400700 1 -- 172.21.15.18:0/1024731525 >> 172.21.15.18:0/1024731525 conn(0x7f68380fac00 msgr2=0x7f68380fd010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:30:40.374 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:40.375+0000 7f683e400700 1 -- 172.21.15.18:0/1024731525 shutdown_connections 2024-04-12T21:30:40.374 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:40.375+0000 7f683e400700 1 -- 172.21.15.18:0/1024731525 wait complete. 2024-04-12T21:30:40.375 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:40.376+0000 7f683e400700 1 Processor -- start 2024-04-12T21:30:40.375 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:40.376+0000 7f683e400700 1 -- start start 2024-04-12T21:30:40.376 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:40.376+0000 7f683e400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6838105090 0x7f6838115b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:30:40.376 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:40.376+0000 7f683e400700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6838105a30 0x7f6838116080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:30:40.376 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:40.376+0000 7f683e400700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6838116760 con 0x7f6838105090 2024-04-12T21:30:40.376 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:40.376+0000 7f683e400700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f683810fcb0 con 0x7f6838105a30 2024-04-12T21:30:40.376 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:40.376+0000 7f6837400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6838105090 0x7f6838115b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:30:40.376 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:40.376+0000 7f6837400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6838105090 0x7f6838115b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:55338/0 (socket says 172.21.15.18:55338) 2024-04-12T21:30:40.376 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:40.376+0000 7f6837400700 1 -- 172.21.15.18:0/1239958809 learned_addr learned my addr 172.21.15.18:0/1239958809 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:30:40.376 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:40.377+0000 7f6837400700 1 -- 172.21.15.18:0/1239958809 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6838105a30 msgr2=0x7f6838116080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:30:40.376 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:40.377+0000 7f6837400700 1 --2- 172.21.15.18:0/1239958809 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6838105a30 0x7f6838116080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:30:40.376 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:40.377+0000 7f6837400700 1 -- 172.21.15.18:0/1239958809 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6824005040 con 0x7f6838105090 2024-04-12T21:30:40.377 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:40.377+0000 7f6837400700 1 --2- 172.21.15.18:0/1239958809 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6838105090 0x7f6838115b40 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7f682800b790 tx=0x7f6828009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:30:40.377 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:40.377+0000 7f682f400700 1 -- 172.21.15.18:0/1239958809 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f682800f770 con 0x7f6838105090 2024-04-12T21:30:40.377 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:40.377+0000 7f682f400700 1 -- 172.21.15.18:0/1239958809 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f682800fda0 con 0x7f6838105090 2024-04-12T21:30:40.377 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:40.377+0000 7f683e400700 1 -- 172.21.15.18:0/1239958809 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f683810ff30 con 0x7f6838105090 2024-04-12T21:30:40.377 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:40.377+0000 7f682f400700 1 -- 172.21.15.18:0/1239958809 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f682800d310 con 0x7f6838105090 2024-04-12T21:30:40.377 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:40.378+0000 7f683e400700 1 -- 172.21.15.18:0/1239958809 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f68381104f0 con 0x7f6838105090 2024-04-12T21:30:40.378 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:40.379+0000 7f683e400700 1 -- 172.21.15.18:0/1239958809 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f68381023a0 con 0x7f6838105090 2024-04-12T21:30:40.378 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:40.380+0000 7f682f400700 1 -- 172.21.15.18:0/1239958809 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7f6828010420 con 0x7f6838105090 2024-04-12T21:30:40.379 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:40.381+0000 7f682f400700 1 --2- 172.21.15.18:0/1239958809 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f681c06ec00 0x7f681c0710b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:30:40.380 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:40.381+0000 7f682f400700 1 -- 172.21.15.18:0/1239958809 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 6757+0+0 (secure 0 0 0) 0x7f682808ea80 con 0x7f6838105090 2024-04-12T21:30:40.380 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:40.381+0000 7f6836a00700 1 --2- 172.21.15.18:0/1239958809 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f681c06ec00 0x7f681c0710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:30:40.381 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:40.382+0000 7f6836a00700 1 --2- 172.21.15.18:0/1239958809 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f681c06ec00 0x7f681c0710b0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f6824004f90 tx=0x7f6824009e30 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:30:40.385 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:40.386+0000 7f682f400700 1 -- 172.21.15.18:0/1239958809 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f682805b230 con 0x7f6838105090 2024-04-12T21:30:40.653 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:40.654+0000 7f683e400700 1 -- 172.21.15.18:0/1239958809 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f683810c1e0 con 0x7f6838105090 2024-04-12T21:30:40.654 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:40.655+0000 7f682f400700 1 -- 172.21.15.18:0/1239958809 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 31 v31) v1 ==== 76+0+1873 (secure 0 0 0) 0x7f682805adc0 con 0x7f6838105090 2024-04-12T21:30:40.654 INFO:teuthology.orchestra.run.smithi018.stdout:e31 2024-04-12T21:30:40.654 INFO:teuthology.orchestra.run.smithi018.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-12T21:30:40.655 INFO:teuthology.orchestra.run.smithi018.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-12T21:30:40.655 INFO:teuthology.orchestra.run.smithi018.stdout:legacy client fscid: 1 2024-04-12T21:30:40.655 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-12T21:30:40.655 INFO:teuthology.orchestra.run.smithi018.stdout:Filesystem 'cephfs' (1) 2024-04-12T21:30:40.655 INFO:teuthology.orchestra.run.smithi018.stdout:fs_name cephfs 2024-04-12T21:30:40.655 INFO:teuthology.orchestra.run.smithi018.stdout:epoch 24 2024-04-12T21:30:40.655 INFO:teuthology.orchestra.run.smithi018.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-04-12T21:30:40.655 INFO:teuthology.orchestra.run.smithi018.stdout:created 2024-04-12T20:51:57.362850+0000 2024-04-12T21:30:40.655 INFO:teuthology.orchestra.run.smithi018.stdout:modified 2024-04-12T21:06:59.717190+0000 2024-04-12T21:30:40.655 INFO:teuthology.orchestra.run.smithi018.stdout:tableserver 0 2024-04-12T21:30:40.655 INFO:teuthology.orchestra.run.smithi018.stdout:root 0 2024-04-12T21:30:40.655 INFO:teuthology.orchestra.run.smithi018.stdout:session_timeout 60 2024-04-12T21:30:40.655 INFO:teuthology.orchestra.run.smithi018.stdout:session_autoclose 300 2024-04-12T21:30:40.655 INFO:teuthology.orchestra.run.smithi018.stdout:max_file_size 1099511627776 2024-04-12T21:30:40.655 INFO:teuthology.orchestra.run.smithi018.stdout:max_xattr_size 65536 2024-04-12T21:30:40.655 INFO:teuthology.orchestra.run.smithi018.stdout:required_client_features {} 2024-04-12T21:30:40.656 INFO:teuthology.orchestra.run.smithi018.stdout:last_failure 0 2024-04-12T21:30:40.656 INFO:teuthology.orchestra.run.smithi018.stdout:last_failure_osd_epoch 100 2024-04-12T21:30:40.656 INFO:teuthology.orchestra.run.smithi018.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-12T21:30:40.656 INFO:teuthology.orchestra.run.smithi018.stdout:max_mds 2 2024-04-12T21:30:40.656 INFO:teuthology.orchestra.run.smithi018.stdout:in 0,1 2024-04-12T21:30:40.656 INFO:teuthology.orchestra.run.smithi018.stdout:up {} 2024-04-12T21:30:40.656 INFO:teuthology.orchestra.run.smithi018.stdout:failed 0,1 2024-04-12T21:30:40.656 INFO:teuthology.orchestra.run.smithi018.stdout:damaged 2024-04-12T21:30:40.656 INFO:teuthology.orchestra.run.smithi018.stdout:stopped 2024-04-12T21:30:40.656 INFO:teuthology.orchestra.run.smithi018.stdout:data_pools [3] 2024-04-12T21:30:40.656 INFO:teuthology.orchestra.run.smithi018.stdout:metadata_pool 2 2024-04-12T21:30:40.656 INFO:teuthology.orchestra.run.smithi018.stdout:inline_data disabled 2024-04-12T21:30:40.656 INFO:teuthology.orchestra.run.smithi018.stdout:balancer 2024-04-12T21:30:40.656 INFO:teuthology.orchestra.run.smithi018.stdout:bal_rank_mask -1 2024-04-12T21:30:40.656 INFO:teuthology.orchestra.run.smithi018.stdout:standby_count_wanted 1 2024-04-12T21:30:40.656 INFO:teuthology.orchestra.run.smithi018.stdout:qdb_cluster leader: 0 members: 2024-04-12T21:30:40.657 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-12T21:30:40.657 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-12T21:30:40.657 INFO:teuthology.orchestra.run.smithi018.stdout:Standby daemons: 2024-04-12T21:30:40.657 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-12T21:30:40.657 INFO:teuthology.orchestra.run.smithi018.stdout:[mds.cephfs.smithi186.wjnafs{-1:24277} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.186:6826/438347207,v1:172.21.15.186:6827/438347207] compat {c=[1],r=[1],i=[7ff]}] 2024-04-12T21:30:40.657 INFO:teuthology.orchestra.run.smithi018.stdout:[mds.cephfs.smithi018.acqbsc{-1:34436} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.18:6828/755179764,v1:172.21.15.18:6829/755179764] compat {c=[1],r=[1],i=[1fff]}] 2024-04-12T21:30:40.657 INFO:teuthology.orchestra.run.smithi018.stdout:[mds.cephfs.smithi018.szjluo{-1:34468} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.18:6826/2407992653,v1:172.21.15.18:6827/2407992653] compat {c=[1],r=[1],i=[1fff]}] 2024-04-12T21:30:40.657 INFO:teuthology.orchestra.run.smithi018.stdout:[mds.cephfs.smithi186.jdllyn{-1:44595} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.186:6824/3523444202,v1:172.21.15.186:6825/3523444202] compat {c=[1],r=[1],i=[1fff]}] 2024-04-12T21:30:40.658 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:40.659+0000 7f683e400700 1 -- 172.21.15.18:0/1239958809 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f681c06ec00 msgr2=0x7f681c0710b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:30:40.658 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:40.659+0000 7f683e400700 1 --2- 172.21.15.18:0/1239958809 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f681c06ec00 0x7f681c0710b0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f6824004f90 tx=0x7f6824009e30 comp rx=0 tx=0).stop 2024-04-12T21:30:40.658 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:40.659+0000 7f683e400700 1 -- 172.21.15.18:0/1239958809 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6838105090 msgr2=0x7f6838115b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:30:40.659 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:40.659+0000 7f683e400700 1 --2- 172.21.15.18:0/1239958809 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6838105090 0x7f6838115b40 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7f682800b790 tx=0x7f6828009250 comp rx=0 tx=0).stop 2024-04-12T21:30:40.659 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:40.660+0000 7f683e400700 1 -- 172.21.15.18:0/1239958809 shutdown_connections 2024-04-12T21:30:40.659 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:40.660+0000 7f683e400700 1 --2- 172.21.15.18:0/1239958809 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f681c06ec00 0x7f681c0710b0 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:30:40.659 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:40.660+0000 7f683e400700 1 --2- 172.21.15.18:0/1239958809 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6838105090 0x7f6838115b40 unknown :-1 s=CLOSED pgs=787 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:30:40.659 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:40.660+0000 7f683e400700 1 --2- 172.21.15.18:0/1239958809 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6838105a30 0x7f6838116080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:30:40.659 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:40.660+0000 7f683e400700 1 -- 172.21.15.18:0/1239958809 >> 172.21.15.18:0/1239958809 conn(0x7f68380fac00 msgr2=0x7f68380fc870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:30:40.660 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:40.660+0000 7f683e400700 1 -- 172.21.15.18:0/1239958809 shutdown_connections 2024-04-12T21:30:40.660 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:40.660+0000 7f683e400700 1 -- 172.21.15.18:0/1239958809 wait complete. 2024-04-12T21:30:40.660 INFO:teuthology.orchestra.run.smithi018.stderr:dumped fsmap epoch 31 2024-04-12T21:30:40.816 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:40.817+0000 7f1771200700 1 -- 172.21.15.18:0/818129993 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f176c102cd0 msgr2=0x7f176c1030a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:30:40.816 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:40.817+0000 7f1771200700 1 --2- 172.21.15.18:0/818129993 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f176c102cd0 0x7f176c1030a0 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7f175c004940 tx=0x7f175c009e30 comp rx=0 tx=0).stop 2024-04-12T21:30:40.816 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:40.817+0000 7f1771200700 1 -- 172.21.15.18:0/818129993 shutdown_connections 2024-04-12T21:30:40.816 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:40.817+0000 7f1771200700 1 --2- 172.21.15.18:0/818129993 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f176c0fb2a0 0x7f176c0fb710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:30:40.816 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:40.817+0000 7f1771200700 1 --2- 172.21.15.18:0/818129993 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f176c102cd0 0x7f176c1030a0 unknown :-1 s=CLOSED pgs=788 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:30:40.816 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:40.817+0000 7f1771200700 1 -- 172.21.15.18:0/818129993 >> 172.21.15.18:0/818129993 conn(0x7f176c0721c0 msgr2=0x7f176c0725c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:30:40.816 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:40.817+0000 7f1771200700 1 -- 172.21.15.18:0/818129993 shutdown_connections 2024-04-12T21:30:40.816 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:40.817+0000 7f1771200700 1 -- 172.21.15.18:0/818129993 wait complete. 2024-04-12T21:30:40.817 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:40.818+0000 7f1771200700 1 Processor -- start 2024-04-12T21:30:40.818 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:40.818+0000 7f1771200700 1 -- start start 2024-04-12T21:30:40.818 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:40.818+0000 7f1771200700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f176c0fb2a0 0x7f176c1139f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:30:40.818 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:40.818+0000 7f1771200700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f176c102cd0 0x7f176c113f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:30:40.818 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:40.818+0000 7f1771200700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f176c114610 con 0x7f176c102cd0 2024-04-12T21:30:40.818 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:40.818+0000 7f1771200700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f176c10d970 con 0x7f176c0fb2a0 2024-04-12T21:30:40.818 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:40.818+0000 7f176aa00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f176c102cd0 0x7f176c113f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:30:40.818 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:40.819+0000 7f176aa00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f176c102cd0 0x7f176c113f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:55352/0 (socket says 172.21.15.18:55352) 2024-04-12T21:30:40.818 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:40.819+0000 7f176aa00700 1 -- 172.21.15.18:0/290589097 learned_addr learned my addr 172.21.15.18:0/290589097 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:30:40.818 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:40.819+0000 7f176b400700 1 --2- 172.21.15.18:0/290589097 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f176c0fb2a0 0x7f176c1139f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:30:40.818 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:40.819+0000 7f176aa00700 1 -- 172.21.15.18:0/290589097 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f176c0fb2a0 msgr2=0x7f176c1139f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:30:40.818 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:40.819+0000 7f176aa00700 1 --2- 172.21.15.18:0/290589097 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f176c0fb2a0 0x7f176c1139f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:30:40.818 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:40.819+0000 7f176aa00700 1 -- 172.21.15.18:0/290589097 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f175c005040 con 0x7f176c102cd0 2024-04-12T21:30:40.819 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:40.819+0000 7f176aa00700 1 --2- 172.21.15.18:0/290589097 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f176c102cd0 0x7f176c113f30 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7f175800b790 tx=0x7f1758009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:30:40.819 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:40.820+0000 7f1763400700 1 -- 172.21.15.18:0/290589097 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f175800f770 con 0x7f176c102cd0 2024-04-12T21:30:40.819 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:40.820+0000 7f1763400700 1 -- 172.21.15.18:0/290589097 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f175800fda0 con 0x7f176c102cd0 2024-04-12T21:30:40.819 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:40.820+0000 7f1763400700 1 -- 172.21.15.18:0/290589097 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f175800d310 con 0x7f176c102cd0 2024-04-12T21:30:40.819 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:40.820+0000 7f1771200700 1 -- 172.21.15.18:0/290589097 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f176c10dc50 con 0x7f176c102cd0 2024-04-12T21:30:40.819 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:40.820+0000 7f1771200700 1 -- 172.21.15.18:0/290589097 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f176c10e190 con 0x7f176c102cd0 2024-04-12T21:30:40.820 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:40.821+0000 7f1771200700 1 -- 172.21.15.18:0/290589097 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f176c0fffd0 con 0x7f176c102cd0 2024-04-12T21:30:40.821 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:40.822+0000 7f1763400700 1 -- 172.21.15.18:0/290589097 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7f1758010420 con 0x7f176c102cd0 2024-04-12T21:30:40.823 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:40.824+0000 7f1763400700 1 --2- 172.21.15.18:0/290589097 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f175006ec00 0x7f17500710b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:30:40.823 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:40.824+0000 7f1763400700 1 -- 172.21.15.18:0/290589097 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 6757+0+0 (secure 0 0 0) 0x7f175808e9a0 con 0x7f176c102cd0 2024-04-12T21:30:40.823 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:40.824+0000 7f176b400700 1 --2- 172.21.15.18:0/290589097 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f175006ec00 0x7f17500710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:30:40.823 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:40.825+0000 7f176b400700 1 --2- 172.21.15.18:0/290589097 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f175006ec00 0x7f17500710b0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f175c00b020 tx=0x7f175c009e30 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:30:40.827 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:40.829+0000 7f1763400700 1 -- 172.21.15.18:0/290589097 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f175805b1d0 con 0x7f176c102cd0 2024-04-12T21:30:41.070 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:41.070+0000 7f1771200700 1 -- 172.21.15.18:0/290589097 --> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f176c10f0e0 con 0x7f175006ec00 2024-04-12T21:30:41.070 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:30:40 smithi018 ceph-mon[112590]: from='client.35628 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:30:41.070 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:30:40 smithi018 ceph-mon[112590]: pgmap v927: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:30:41.070 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:30:40 smithi018 ceph-mon[112590]: from='client.35632 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:30:41.071 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:30:40 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T21:30:41.071 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:30:40 smithi018 ceph-mon[112590]: from='client.? 172.21.15.18:0/1935045714' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T21:30:41.071 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:30:40 smithi018 ceph-mon[112590]: from='client.? 172.21.15.18:0/1239958809' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-12T21:30:41.071 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:41.072+0000 7f1763400700 1 -- 172.21.15.18:0/290589097 <== mgr.34102 v2:172.21.15.18:6800/3812891919 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f176c10f0e0 con 0x7f175006ec00 2024-04-12T21:30:41.072 INFO:teuthology.orchestra.run.smithi018.stdout:{ 2024-04-12T21:30:41.072 INFO:teuthology.orchestra.run.smithi018.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:f4452def3be4d350f4a8e5feaf7ebb1511be9298fe68478ba7a2d6fbd7d69a63", 2024-04-12T21:30:41.072 INFO:teuthology.orchestra.run.smithi018.stdout: "in_progress": true, 2024-04-12T21:30:41.072 INFO:teuthology.orchestra.run.smithi018.stdout: "which": "Upgrading all daemon types on all hosts", 2024-04-12T21:30:41.072 INFO:teuthology.orchestra.run.smithi018.stdout: "services_complete": [ 2024-04-12T21:30:41.072 INFO:teuthology.orchestra.run.smithi018.stdout: "crash", 2024-04-12T21:30:41.072 INFO:teuthology.orchestra.run.smithi018.stdout: "osd", 2024-04-12T21:30:41.073 INFO:teuthology.orchestra.run.smithi018.stdout: "mon", 2024-04-12T21:30:41.073 INFO:teuthology.orchestra.run.smithi018.stdout: "mgr" 2024-04-12T21:30:41.073 INFO:teuthology.orchestra.run.smithi018.stdout: ], 2024-04-12T21:30:41.073 INFO:teuthology.orchestra.run.smithi018.stdout: "progress": "15/21 daemons upgraded", 2024-04-12T21:30:41.073 INFO:teuthology.orchestra.run.smithi018.stdout: "message": "Currently upgrading mds daemons", 2024-04-12T21:30:41.073 INFO:teuthology.orchestra.run.smithi018.stdout: "is_paused": false 2024-04-12T21:30:41.073 INFO:teuthology.orchestra.run.smithi018.stdout:} 2024-04-12T21:30:41.074 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:41.075+0000 7f1771200700 1 -- 172.21.15.18:0/290589097 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f175006ec00 msgr2=0x7f17500710b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:30:41.074 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:41.075+0000 7f1771200700 1 --2- 172.21.15.18:0/290589097 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f175006ec00 0x7f17500710b0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f175c00b020 tx=0x7f175c009e30 comp rx=0 tx=0).stop 2024-04-12T21:30:41.074 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:41.075+0000 7f1771200700 1 -- 172.21.15.18:0/290589097 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f176c102cd0 msgr2=0x7f176c113f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:30:41.075 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:41.075+0000 7f1771200700 1 --2- 172.21.15.18:0/290589097 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f176c102cd0 0x7f176c113f30 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7f175800b790 tx=0x7f1758009250 comp rx=0 tx=0).stop 2024-04-12T21:30:41.075 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:41.076+0000 7f1771200700 1 -- 172.21.15.18:0/290589097 shutdown_connections 2024-04-12T21:30:41.075 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:41.076+0000 7f1771200700 1 --2- 172.21.15.18:0/290589097 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f175006ec00 0x7f17500710b0 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:30:41.075 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:41.076+0000 7f1771200700 1 --2- 172.21.15.18:0/290589097 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f176c0fb2a0 0x7f176c1139f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:30:41.076 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:41.076+0000 7f1771200700 1 --2- 172.21.15.18:0/290589097 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f176c102cd0 0x7f176c113f30 unknown :-1 s=CLOSED pgs=789 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:30:41.076 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:41.076+0000 7f1771200700 1 -- 172.21.15.18:0/290589097 >> 172.21.15.18:0/290589097 conn(0x7f176c0721c0 msgr2=0x7f176c0fa520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:30:41.076 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:41.076+0000 7f1771200700 1 -- 172.21.15.18:0/290589097 shutdown_connections 2024-04-12T21:30:41.076 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:41.076+0000 7f1771200700 1 -- 172.21.15.18:0/290589097 wait complete. 2024-04-12T21:30:41.226 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:41.226+0000 7f2f3e600700 1 -- 172.21.15.18:0/3597457926 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2f38105800 msgr2=0x7f2f380fee70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:30:41.226 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:41.226+0000 7f2f3e600700 1 --2- 172.21.15.18:0/3597457926 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2f38105800 0x7f2f380fee70 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7f2f24004940 tx=0x7f2f24009e30 comp rx=0 tx=0).stop 2024-04-12T21:30:41.226 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:41.227+0000 7f2f3e600700 1 -- 172.21.15.18:0/3597457926 shutdown_connections 2024-04-12T21:30:41.226 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:41.227+0000 7f2f3e600700 1 --2- 172.21.15.18:0/3597457926 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2f38105800 0x7f2f380fee70 unknown :-1 s=CLOSED pgs=790 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:30:41.226 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:41.227+0000 7f2f3e600700 1 --2- 172.21.15.18:0/3597457926 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2f38104e60 0x7f2f38105230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:30:41.226 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:41.227+0000 7f2f3e600700 1 -- 172.21.15.18:0/3597457926 >> 172.21.15.18:0/3597457926 conn(0x7f2f380fab10 msgr2=0x7f2f380fcef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:30:41.226 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:41.227+0000 7f2f3e600700 1 -- 172.21.15.18:0/3597457926 shutdown_connections 2024-04-12T21:30:41.227 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:41.227+0000 7f2f3e600700 1 -- 172.21.15.18:0/3597457926 wait complete. 2024-04-12T21:30:41.228 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:41.228+0000 7f2f3e600700 1 Processor -- start 2024-04-12T21:30:41.228 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:41.228+0000 7f2f3e600700 1 -- start start 2024-04-12T21:30:41.228 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:41.228+0000 7f2f3e600700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2f38104e60 0x7f2f38115bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:30:41.228 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:41.228+0000 7f2f3e600700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2f38105800 0x7f2f381160f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:30:41.228 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:41.228+0000 7f2f3e600700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f38116740 con 0x7f2f38104e60 2024-04-12T21:30:41.228 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:41.228+0000 7f2f3e600700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f3810fb30 con 0x7f2f38105800 2024-04-12T21:30:41.228 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:41.229+0000 7f2f3d200700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2f38104e60 0x7f2f38115bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:30:41.228 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:41.229+0000 7f2f3d200700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2f38104e60 0x7f2f38115bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:48738/0 (socket says 172.21.15.18:48738) 2024-04-12T21:30:41.228 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:41.229+0000 7f2f3d200700 1 -- 172.21.15.18:0/362110514 learned_addr learned my addr 172.21.15.18:0/362110514 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:30:41.229 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:41.229+0000 7f2f3d200700 1 -- 172.21.15.18:0/362110514 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2f38105800 msgr2=0x7f2f381160f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:30:41.229 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:41.229+0000 7f2f3d200700 1 --2- 172.21.15.18:0/362110514 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2f38105800 0x7f2f381160f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:30:41.229 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:41.229+0000 7f2f3d200700 1 -- 172.21.15.18:0/362110514 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f24005040 con 0x7f2f38104e60 2024-04-12T21:30:41.229 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:41.229+0000 7f2f3d200700 1 --2- 172.21.15.18:0/362110514 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2f38104e60 0x7f2f38115bb0 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7f2f2c00c8c0 tx=0x7f2f2c0044c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:30:41.229 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:41.229+0000 7f2f34c00700 1 -- 172.21.15.18:0/362110514 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2f2c00a980 con 0x7f2f38104e60 2024-04-12T21:30:41.229 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:41.230+0000 7f2f34c00700 1 -- 172.21.15.18:0/362110514 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f2f2c00ce60 con 0x7f2f38104e60 2024-04-12T21:30:41.229 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:41.230+0000 7f2f3e600700 1 -- 172.21.15.18:0/362110514 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2f3810fe10 con 0x7f2f38104e60 2024-04-12T21:30:41.230 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:41.230+0000 7f2f34c00700 1 -- 172.21.15.18:0/362110514 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2f2c0115e0 con 0x7f2f38104e60 2024-04-12T21:30:41.230 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:41.230+0000 7f2f3e600700 1 -- 172.21.15.18:0/362110514 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2f381103d0 con 0x7f2f38104e60 2024-04-12T21:30:41.230 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:41.231+0000 7f2f3e600700 1 -- 172.21.15.18:0/362110514 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2f38107330 con 0x7f2f38104e60 2024-04-12T21:30:41.230 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:41.232+0000 7f2f34c00700 1 -- 172.21.15.18:0/362110514 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7f2f2c012400 con 0x7f2f38104e60 2024-04-12T21:30:41.232 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:41.233+0000 7f2f34c00700 1 --2- 172.21.15.18:0/362110514 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f2f2006ec00 0x7f2f200710b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:30:41.232 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:41.233+0000 7f2f34c00700 1 -- 172.21.15.18:0/362110514 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 6757+0+0 (secure 0 0 0) 0x7f2f2c08ea80 con 0x7f2f38104e60 2024-04-12T21:30:41.232 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:41.233+0000 7f2f37e00700 1 --2- 172.21.15.18:0/362110514 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f2f2006ec00 0x7f2f200710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:30:41.232 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:41.234+0000 7f2f37e00700 1 --2- 172.21.15.18:0/362110514 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f2f2006ec00 0x7f2f200710b0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f2f24004f90 tx=0x7f2f24009e30 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:30:41.236 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:41.238+0000 7f2f34c00700 1 -- 172.21.15.18:0/362110514 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f2f2c05b2d0 con 0x7f2f38104e60 2024-04-12T21:30:41.281 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:30:40 smithi186 ceph-mon[86379]: from='client.35628 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:30:41.281 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:30:40 smithi186 ceph-mon[86379]: pgmap v927: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:30:41.281 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:30:40 smithi186 ceph-mon[86379]: from='client.35632 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:30:41.281 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:30:40 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T21:30:41.282 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:30:40 smithi186 ceph-mon[86379]: from='client.? 172.21.15.18:0/1935045714' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T21:30:41.282 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:30:40 smithi186 ceph-mon[86379]: from='client.? 172.21.15.18:0/1239958809' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-12T21:30:41.548 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:41.548+0000 7f2f3e600700 1 -- 172.21.15.18:0/362110514 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f2f38111250 con 0x7f2f38104e60 2024-04-12T21:30:41.548 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:41.550+0000 7f2f34c00700 1 -- 172.21.15.18:0/362110514 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+4303 (secure 0 0 0) 0x7f2f2c0126b0 con 0x7f2f38104e60 2024-04-12T21:30:41.549 INFO:teuthology.orchestra.run.smithi018.stdout:HEALTH_ERR failed to probe daemons or devices; 1 filesystem is degraded; 1 filesystem is offline 2024-04-12T21:30:41.549 INFO:teuthology.orchestra.run.smithi018.stdout:[WRN] CEPHADM_REFRESH_FAILED: failed to probe daemons or devices 2024-04-12T21:30:41.549 INFO:teuthology.orchestra.run.smithi018.stdout: host smithi018 `cephadm ceph-volume` failed: cephadm exited with an error code: 1, stderr: Inferring config /var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/mon.smithi018/config 2024-04-12T21:30:41.549 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/podman: timeout after 895 seconds 2024-04-12T21:30:41.549 INFO:teuthology.orchestra.run.smithi018.stdout:Non-zero exit code 124 from /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:f4452def3be4d350f4a8e5feaf7ebb1511be9298fe68478ba7a2d6fbd7d69a63 -e NODE_NAME=smithi018 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968:/var/run/ceph:z -v /var/log/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968:/var/log/ceph:z -v /var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmpxsmwkkuh:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:f4452def3be4d350f4a8e5feaf7ebb1511be9298fe68478ba7a2d6fbd7d69a63 inventory --format=json-pretty --filter-for-batch 2024-04-12T21:30:41.549 INFO:teuthology.orchestra.run.smithi018.stdout:Traceback (most recent call last): 2024-04-12T21:30:41.549 INFO:teuthology.orchestra.run.smithi018.stdout: File "/usr/lib64/python3.9/runpy.py", line 197, in _run_module_as_main 2024-04-12T21:30:41.549 INFO:teuthology.orchestra.run.smithi018.stdout: return _run_code(code, main_globals, None, 2024-04-12T21:30:41.550 INFO:teuthology.orchestra.run.smithi018.stdout: File "/usr/lib64/python3.9/runpy.py", line 87, in _run_code 2024-04-12T21:30:41.550 INFO:teuthology.orchestra.run.smithi018.stdout: exec(code, run_globals) 2024-04-12T21:30:41.550 INFO:teuthology.orchestra.run.smithi018.stdout: File "/var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/cephadm.e44f3ed5e34e4ef52eedef233d1190bff004470d8e8005127b5312bff357fa7f/__main__.py", line 5618, in 2024-04-12T21:30:41.550 INFO:teuthology.orchestra.run.smithi018.stdout: File "/var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/cephadm.e44f3ed5e34e4ef52eedef233d1190bff004470d8e8005127b5312bff357fa7f/__main__.py", line 5606, in main 2024-04-12T21:30:41.550 INFO:teuthology.orchestra.run.smithi018.stdout: File "/var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/cephadm.e44f3ed5e34e4ef52eedef233d1190bff004470d8e8005127b5312bff357fa7f/__main__.py", line 408, in _infer_config 2024-04-12T21:30:41.550 INFO:teuthology.orchestra.run.smithi018.stdout: File "/var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/cephadm.e44f3ed5e34e4ef52eedef233d1190bff004470d8e8005127b5312bff357fa7f/__main__.py", line 323, in _infer_fsid 2024-04-12T21:30:41.550 INFO:teuthology.orchestra.run.smithi018.stdout: File "/var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/cephadm.e44f3ed5e34e4ef52eedef233d1190bff004470d8e8005127b5312bff357fa7f/__main__.py", line 436, in _infer_image 2024-04-12T21:30:41.550 INFO:teuthology.orchestra.run.smithi018.stdout: File "/var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/cephadm.e44f3ed5e34e4ef52eedef233d1190bff004470d8e8005127b5312bff357fa7f/__main__.py", line 310, in _validate_fsid 2024-04-12T21:30:41.550 INFO:teuthology.orchestra.run.smithi018.stdout: File "/var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/cephadm.e44f3ed5e34e4ef52eedef233d1190bff004470d8e8005127b5312bff357fa7f/__main__.py", line 3359, in command_ceph_volume 2024-04-12T21:30:41.550 INFO:teuthology.orchestra.run.smithi018.stdout: File "/var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/cephadm.e44f3ed5e34e4ef52eedef233d1190bff004470d8e8005127b5312bff357fa7f/cephadmlib/call_wrappers.py", line 310, in call_throws 2024-04-12T21:30:41.550 INFO:teuthology.orchestra.run.smithi018.stdout:RuntimeError: Failed command: /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:f4452def3be4d350f4a8e5feaf7ebb1511be9298fe68478ba7a2d6fbd7d69a63 -e NODE_NAME=smithi018 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968:/var/run/ceph:z -v /var/log/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968:/var/log/ceph:z -v /var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmpxsmwkkuh:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:f4452def3be4d350f4a8e5feaf7ebb1511be9298fe68478ba7a2d6fbd7d69a63 inventory --format=json-pretty --filter-for-batch 2024-04-12T21:30:41.550 INFO:teuthology.orchestra.run.smithi018.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-04-12T21:30:41.550 INFO:teuthology.orchestra.run.smithi018.stdout: fs cephfs is degraded 2024-04-12T21:30:41.550 INFO:teuthology.orchestra.run.smithi018.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-04-12T21:30:41.551 INFO:teuthology.orchestra.run.smithi018.stdout: fs cephfs is offline because no MDS is active for it. 2024-04-12T21:30:41.551 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:41.552+0000 7f2f3e600700 1 -- 172.21.15.18:0/362110514 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f2f2006ec00 msgr2=0x7f2f200710b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:30:41.551 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:41.552+0000 7f2f3e600700 1 --2- 172.21.15.18:0/362110514 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f2f2006ec00 0x7f2f200710b0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f2f24004f90 tx=0x7f2f24009e30 comp rx=0 tx=0).stop 2024-04-12T21:30:41.551 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:41.552+0000 7f2f3e600700 1 -- 172.21.15.18:0/362110514 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2f38104e60 msgr2=0x7f2f38115bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:30:41.552 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:41.552+0000 7f2f3e600700 1 --2- 172.21.15.18:0/362110514 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2f38104e60 0x7f2f38115bb0 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7f2f2c00c8c0 tx=0x7f2f2c0044c0 comp rx=0 tx=0).stop 2024-04-12T21:30:41.552 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:41.553+0000 7f2f3e600700 1 -- 172.21.15.18:0/362110514 shutdown_connections 2024-04-12T21:30:41.552 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:41.553+0000 7f2f3e600700 1 --2- 172.21.15.18:0/362110514 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f2f2006ec00 0x7f2f200710b0 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:30:41.552 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:41.553+0000 7f2f3e600700 1 --2- 172.21.15.18:0/362110514 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2f38104e60 0x7f2f38115bb0 unknown :-1 s=CLOSED pgs=791 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:30:41.552 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:41.553+0000 7f2f3e600700 1 --2- 172.21.15.18:0/362110514 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2f38105800 0x7f2f381160f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:30:41.552 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:41.553+0000 7f2f3e600700 1 -- 172.21.15.18:0/362110514 >> 172.21.15.18:0/362110514 conn(0x7f2f380fab10 msgr2=0x7f2f380fd940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:30:41.552 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:41.553+0000 7f2f3e600700 1 -- 172.21.15.18:0/362110514 shutdown_connections 2024-04-12T21:30:41.552 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:30:41.553+0000 7f2f3e600700 1 -- 172.21.15.18:0/362110514 wait complete. 2024-04-12T21:30:42.183 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:30:41 smithi186 ceph-mon[86379]: from='client.35644 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:30:42.183 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:30:41 smithi186 ceph-mon[86379]: from='client.? 172.21.15.18:0/362110514' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-12T21:30:42.183 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:30:41 smithi186 ceph-mon[86379]: pgmap v928: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:30:42.352 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:30:41 smithi018 ceph-mon[112590]: from='client.35644 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:30:42.352 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:30:41 smithi018 ceph-mon[112590]: from='client.? 172.21.15.18:0/362110514' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-12T21:30:42.353 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:30:41 smithi018 ceph-mon[112590]: pgmap v928: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:30:45.031 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:30:44 smithi186 ceph-mon[86379]: pgmap v929: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:30:45.056 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:30:44 smithi018 ceph-mon[112590]: pgmap v929: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:30:47.031 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:30:46 smithi186 ceph-mon[86379]: pgmap v930: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:30:47.102 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:30:46 smithi018 ceph-mon[112590]: pgmap v930: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:30:49.031 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:30:48 smithi186 ceph-mon[86379]: pgmap v931: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:30:49.102 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:30:48 smithi018 ceph-mon[112590]: pgmap v931: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:30:51.031 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:30:50 smithi186 ceph-mon[86379]: pgmap v932: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:30:51.102 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:30:50 smithi018 ceph-mon[112590]: pgmap v932: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:30:53.031 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:30:52 smithi186 ceph-mon[86379]: pgmap v933: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:30:53.102 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:30:52 smithi018 ceph-mon[112590]: pgmap v933: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:30:55.031 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:30:54 smithi186 ceph-mon[86379]: pgmap v934: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:30:55.102 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:30:54 smithi018 ceph-mon[112590]: pgmap v934: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:30:56.031 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:30:55 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T21:30:56.102 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:30:55 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T21:30:57.031 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:30:56 smithi186 ceph-mon[86379]: pgmap v935: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:30:57.102 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:30:56 smithi018 ceph-mon[112590]: pgmap v935: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:30:59.031 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:30:58 smithi186 ceph-mon[86379]: pgmap v936: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:30:59.102 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:30:58 smithi018 ceph-mon[112590]: pgmap v936: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:31:01.031 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:31:00 smithi186 ceph-mon[86379]: pgmap v937: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:31:01.103 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:31:00 smithi018 ceph-mon[112590]: pgmap v937: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:31:03.031 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:31:02 smithi186 ceph-mon[86379]: pgmap v938: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:31:03.102 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:31:02 smithi018 ceph-mon[112590]: pgmap v938: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:31:05.031 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:31:04 smithi186 ceph-mon[86379]: pgmap v939: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:31:05.102 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:31:04 smithi018 ceph-mon[112590]: pgmap v939: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:31:06.531 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:31:06 smithi186 ceph-mon[86379]: pgmap v940: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:31:06.602 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:31:06 smithi018 ceph-mon[112590]: pgmap v940: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:31:09.031 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:31:08 smithi186 ceph-mon[86379]: pgmap v941: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:31:09.102 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:31:08 smithi018 ceph-mon[112590]: pgmap v941: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:31:11.031 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:31:10 smithi186 ceph-mon[86379]: pgmap v942: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:31:11.031 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:31:10 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T21:31:11.102 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:31:10 smithi018 ceph-mon[112590]: pgmap v942: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:31:11.103 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:31:10 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T21:31:11.728 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:11.730+0000 7f0b92200700 1 -- 172.21.15.18:0/3055209613 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0b8c105020 msgr2=0x7f0b8c1053f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:31:11.729 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:11.730+0000 7f0b92200700 1 --2- 172.21.15.18:0/3055209613 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0b8c105020 0x7f0b8c1053f0 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7f0b7c004940 tx=0x7f0b7c009e30 comp rx=0 tx=0).stop 2024-04-12T21:31:11.729 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:11.730+0000 7f0b92200700 1 -- 172.21.15.18:0/3055209613 shutdown_connections 2024-04-12T21:31:11.729 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:11.730+0000 7f0b92200700 1 --2- 172.21.15.18:0/3055209613 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0b8c1059c0 0x7f0b8c0ff080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:31:11.729 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:11.730+0000 7f0b92200700 1 --2- 172.21.15.18:0/3055209613 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0b8c105020 0x7f0b8c1053f0 unknown :-1 s=CLOSED pgs=792 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:31:11.729 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:11.730+0000 7f0b92200700 1 -- 172.21.15.18:0/3055209613 >> 172.21.15.18:0/3055209613 conn(0x7f0b8c0faba0 msgr2=0x7f0b8c0fcfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:31:11.729 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:11.730+0000 7f0b92200700 1 -- 172.21.15.18:0/3055209613 shutdown_connections 2024-04-12T21:31:11.729 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:11.730+0000 7f0b92200700 1 -- 172.21.15.18:0/3055209613 wait complete. 2024-04-12T21:31:11.731 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:11.731+0000 7f0b92200700 1 Processor -- start 2024-04-12T21:31:11.731 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:11.731+0000 7f0b92200700 1 -- start start 2024-04-12T21:31:11.731 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:11.732+0000 7f0b92200700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0b8c105020 0x7f0b8c115bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:31:11.731 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:11.732+0000 7f0b92200700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0b8c1059c0 0x7f0b8c116100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:31:11.731 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:11.732+0000 7f0b92200700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b8c116750 con 0x7f0b8c105020 2024-04-12T21:31:11.731 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:11.732+0000 7f0b92200700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b8c10fd30 con 0x7f0b8c1059c0 2024-04-12T21:31:11.731 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:11.732+0000 7f0b8b400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0b8c105020 0x7f0b8c115bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:31:11.732 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:11.732+0000 7f0b8b400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0b8c105020 0x7f0b8c115bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:39554/0 (socket says 172.21.15.18:39554) 2024-04-12T21:31:11.732 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:11.732+0000 7f0b8b400700 1 -- 172.21.15.18:0/1535430454 learned_addr learned my addr 172.21.15.18:0/1535430454 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:31:11.732 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:11.732+0000 7f0b8aa00700 1 --2- 172.21.15.18:0/1535430454 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0b8c1059c0 0x7f0b8c116100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:31:11.732 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:11.733+0000 7f0b8b400700 1 -- 172.21.15.18:0/1535430454 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0b8c1059c0 msgr2=0x7f0b8c116100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:31:11.732 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:11.733+0000 7f0b8b400700 1 --2- 172.21.15.18:0/1535430454 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0b8c1059c0 0x7f0b8c116100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:31:11.732 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:11.733+0000 7f0b8b400700 1 -- 172.21.15.18:0/1535430454 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b7c005040 con 0x7f0b8c105020 2024-04-12T21:31:11.732 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:11.733+0000 7f0b8b400700 1 --2- 172.21.15.18:0/1535430454 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0b8c105020 0x7f0b8c115bc0 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7f0b7c009180 tx=0x7f0b7c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:31:11.733 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:11.733+0000 7f0b83400700 1 -- 172.21.15.18:0/1535430454 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0b7c00e040 con 0x7f0b8c105020 2024-04-12T21:31:11.733 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:11.733+0000 7f0b83400700 1 -- 172.21.15.18:0/1535430454 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f0b7c00bbc0 con 0x7f0b8c105020 2024-04-12T21:31:11.733 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:11.733+0000 7f0b83400700 1 -- 172.21.15.18:0/1535430454 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0b7c01c630 con 0x7f0b8c105020 2024-04-12T21:31:11.733 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:11.734+0000 7f0b92200700 1 -- 172.21.15.18:0/1535430454 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0b8c10ffb0 con 0x7f0b8c105020 2024-04-12T21:31:11.733 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:11.734+0000 7f0b92200700 1 -- 172.21.15.18:0/1535430454 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0b8c110570 con 0x7f0b8c105020 2024-04-12T21:31:11.733 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:11.735+0000 7f0b92200700 1 -- 172.21.15.18:0/1535430454 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0b8c100770 con 0x7f0b8c105020 2024-04-12T21:31:11.734 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:11.736+0000 7f0b83400700 1 -- 172.21.15.18:0/1535430454 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7f0b7c00bd30 con 0x7f0b8c105020 2024-04-12T21:31:11.735 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:11.737+0000 7f0b83400700 1 --2- 172.21.15.18:0/1535430454 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f0b7006ec00 0x7f0b700710b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:31:11.735 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:11.737+0000 7f0b83400700 1 -- 172.21.15.18:0/1535430454 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 6757+0+0 (secure 0 0 0) 0x7f0b7c08fd70 con 0x7f0b8c105020 2024-04-12T21:31:11.735 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:11.737+0000 7f0b8aa00700 1 --2- 172.21.15.18:0/1535430454 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f0b7006ec00 0x7f0b700710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:31:11.736 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:11.738+0000 7f0b8aa00700 1 --2- 172.21.15.18:0/1535430454 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f0b7006ec00 0x7f0b700710b0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f0b78009fd0 tx=0x7f0b78009f20 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:31:11.740 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:11.742+0000 7f0b83400700 1 -- 172.21.15.18:0/1535430454 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f0b7c05c5a0 con 0x7f0b8c105020 2024-04-12T21:31:11.995 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:11.996+0000 7f0b92200700 1 -- 172.21.15.18:0/1535430454 --> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f0b8c111210 con 0x7f0b7006ec00 2024-04-12T21:31:11.998 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:12.000+0000 7f0b83400700 1 -- 172.21.15.18:0/1535430454 <== mgr.34102 v2:172.21.15.18:6800/3812891919 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f0b8c111210 con 0x7f0b7006ec00 2024-04-12T21:31:12.001 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:12.003+0000 7f0b92200700 1 -- 172.21.15.18:0/1535430454 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f0b7006ec00 msgr2=0x7f0b700710b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:31:12.001 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:12.003+0000 7f0b92200700 1 --2- 172.21.15.18:0/1535430454 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f0b7006ec00 0x7f0b700710b0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f0b78009fd0 tx=0x7f0b78009f20 comp rx=0 tx=0).stop 2024-04-12T21:31:12.001 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:12.003+0000 7f0b92200700 1 -- 172.21.15.18:0/1535430454 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0b8c105020 msgr2=0x7f0b8c115bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:31:12.001 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:12.003+0000 7f0b92200700 1 --2- 172.21.15.18:0/1535430454 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0b8c105020 0x7f0b8c115bc0 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7f0b7c009180 tx=0x7f0b7c009e30 comp rx=0 tx=0).stop 2024-04-12T21:31:12.001 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:12.003+0000 7f0b92200700 1 -- 172.21.15.18:0/1535430454 shutdown_connections 2024-04-12T21:31:12.001 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:12.003+0000 7f0b92200700 1 --2- 172.21.15.18:0/1535430454 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f0b7006ec00 0x7f0b700710b0 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:31:12.001 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:12.003+0000 7f0b92200700 1 --2- 172.21.15.18:0/1535430454 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0b8c105020 0x7f0b8c115bc0 unknown :-1 s=CLOSED pgs=793 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:31:12.001 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:12.003+0000 7f0b92200700 1 --2- 172.21.15.18:0/1535430454 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0b8c1059c0 0x7f0b8c116100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:31:12.002 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:12.003+0000 7f0b92200700 1 -- 172.21.15.18:0/1535430454 >> 172.21.15.18:0/1535430454 conn(0x7f0b8c0faba0 msgr2=0x7f0b8c0fbba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:31:12.002 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:12.003+0000 7f0b92200700 1 -- 172.21.15.18:0/1535430454 shutdown_connections 2024-04-12T21:31:12.002 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:12.003+0000 7f0b92200700 1 -- 172.21.15.18:0/1535430454 wait complete. 2024-04-12T21:31:12.019 INFO:teuthology.orchestra.run.smithi018.stdout:true 2024-04-12T21:31:12.166 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:12.167+0000 7f1ce2c00700 1 -- 172.21.15.18:0/2110372762 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1cdc071f90 msgr2=0x7f1cdc070360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:31:12.166 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:12.167+0000 7f1ce2c00700 1 --2- 172.21.15.18:0/2110372762 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1cdc071f90 0x7f1cdc070360 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7f1cd0004970 tx=0x7f1cd0009e30 comp rx=0 tx=0).stop 2024-04-12T21:31:12.166 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:12.168+0000 7f1ce2c00700 1 -- 172.21.15.18:0/2110372762 shutdown_connections 2024-04-12T21:31:12.166 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:12.168+0000 7f1ce2c00700 1 --2- 172.21.15.18:0/2110372762 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1cdc0708a0 0x7f1cdc070d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:31:12.166 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:12.168+0000 7f1ce2c00700 1 --2- 172.21.15.18:0/2110372762 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1cdc071f90 0x7f1cdc070360 unknown :-1 s=CLOSED pgs=794 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:31:12.166 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:12.168+0000 7f1ce2c00700 1 -- 172.21.15.18:0/2110372762 >> 172.21.15.18:0/2110372762 conn(0x7f1cdc0f87f0 msgr2=0x7f1cdc0fac00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:31:12.166 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:12.168+0000 7f1ce2c00700 1 -- 172.21.15.18:0/2110372762 shutdown_connections 2024-04-12T21:31:12.166 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:12.168+0000 7f1ce2c00700 1 -- 172.21.15.18:0/2110372762 wait complete. 2024-04-12T21:31:12.167 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:12.169+0000 7f1ce2c00700 1 Processor -- start 2024-04-12T21:31:12.167 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:12.169+0000 7f1ce2c00700 1 -- start start 2024-04-12T21:31:12.167 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:12.170+0000 7f1ce2c00700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1cdc0708a0 0x7f1cdc119ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:31:12.167 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:12.170+0000 7f1ce2c00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1cdc071f90 0x7f1cdc11a530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:31:12.168 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:12.170+0000 7f1ce2c00700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1cdc11ab80 con 0x7f1cdc071f90 2024-04-12T21:31:12.168 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:12.170+0000 7f1ce2c00700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1cdc113f50 con 0x7f1cdc0708a0 2024-04-12T21:31:12.168 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:12.170+0000 7f1ce0e00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1cdc071f90 0x7f1cdc11a530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:31:12.169 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:12.170+0000 7f1ce0e00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1cdc071f90 0x7f1cdc11a530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:39578/0 (socket says 172.21.15.18:39578) 2024-04-12T21:31:12.169 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:12.170+0000 7f1ce0e00700 1 -- 172.21.15.18:0/2730356951 learned_addr learned my addr 172.21.15.18:0/2730356951 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:31:12.169 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:12.170+0000 7f1ce1800700 1 --2- 172.21.15.18:0/2730356951 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1cdc0708a0 0x7f1cdc119ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:31:12.169 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:12.171+0000 7f1ce0e00700 1 -- 172.21.15.18:0/2730356951 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1cdc0708a0 msgr2=0x7f1cdc119ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:31:12.169 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:12.171+0000 7f1ce0e00700 1 --2- 172.21.15.18:0/2730356951 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1cdc0708a0 0x7f1cdc119ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:31:12.169 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:12.171+0000 7f1ce0e00700 1 -- 172.21.15.18:0/2730356951 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1cd0005040 con 0x7f1cdc071f90 2024-04-12T21:31:12.170 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:12.171+0000 7f1ce0e00700 1 --2- 172.21.15.18:0/2730356951 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1cdc071f90 0x7f1cdc11a530 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7f1cc800eb40 tx=0x7f1cc800b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:31:12.170 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:12.171+0000 7f1cd9600700 1 -- 172.21.15.18:0/2730356951 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1cc8007cc0 con 0x7f1cdc071f90 2024-04-12T21:31:12.170 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:12.171+0000 7f1cd9600700 1 -- 172.21.15.18:0/2730356951 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f1cc800f040 con 0x7f1cdc071f90 2024-04-12T21:31:12.170 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:12.171+0000 7f1cd9600700 1 -- 172.21.15.18:0/2730356951 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1cc8009620 con 0x7f1cdc071f90 2024-04-12T21:31:12.170 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:12.171+0000 7f1ce2c00700 1 -- 172.21.15.18:0/2730356951 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1cdc1141d0 con 0x7f1cdc071f90 2024-04-12T21:31:12.170 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:12.171+0000 7f1ce2c00700 1 -- 172.21.15.18:0/2730356951 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1cdc114710 con 0x7f1cdc071f90 2024-04-12T21:31:12.171 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:12.173+0000 7f1ce2c00700 1 -- 172.21.15.18:0/2730356951 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1cdc071aa0 con 0x7f1cdc071f90 2024-04-12T21:31:12.172 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:12.174+0000 7f1cd9600700 1 -- 172.21.15.18:0/2730356951 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7f1cc8009780 con 0x7f1cdc071f90 2024-04-12T21:31:12.173 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:12.175+0000 7f1cd9600700 1 --2- 172.21.15.18:0/2730356951 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f1cc406ec00 0x7f1cc40710b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:31:12.173 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:12.175+0000 7f1cd9600700 1 -- 172.21.15.18:0/2730356951 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 6757+0+0 (secure 0 0 0) 0x7f1cc808e650 con 0x7f1cdc071f90 2024-04-12T21:31:12.173 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:12.175+0000 7f1ce1800700 1 --2- 172.21.15.18:0/2730356951 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f1cc406ec00 0x7f1cc40710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:31:12.174 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:12.176+0000 7f1ce1800700 1 --2- 172.21.15.18:0/2730356951 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f1cc406ec00 0x7f1cc40710b0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f1cd0000c00 tx=0x7f1cd0009e30 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:31:12.178 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:12.180+0000 7f1cd9600700 1 -- 172.21.15.18:0/2730356951 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f1cc805aeb0 con 0x7f1cdc071f90 2024-04-12T21:31:12.434 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:12.436+0000 7f1ce2c00700 1 -- 172.21.15.18:0/2730356951 --> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f1cdc110410 con 0x7f1cc406ec00 2024-04-12T21:31:12.437 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:12.439+0000 7f1cd9600700 1 -- 172.21.15.18:0/2730356951 <== mgr.34102 v2:172.21.15.18:6800/3812891919 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f1cdc110410 con 0x7f1cc406ec00 2024-04-12T21:31:12.440 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:12.442+0000 7f1ce2c00700 1 -- 172.21.15.18:0/2730356951 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f1cc406ec00 msgr2=0x7f1cc40710b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:31:12.440 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:12.442+0000 7f1ce2c00700 1 --2- 172.21.15.18:0/2730356951 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f1cc406ec00 0x7f1cc40710b0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f1cd0000c00 tx=0x7f1cd0009e30 comp rx=0 tx=0).stop 2024-04-12T21:31:12.440 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:12.442+0000 7f1ce2c00700 1 -- 172.21.15.18:0/2730356951 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1cdc071f90 msgr2=0x7f1cdc11a530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:31:12.440 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:12.442+0000 7f1ce2c00700 1 --2- 172.21.15.18:0/2730356951 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1cdc071f90 0x7f1cdc11a530 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7f1cc800eb40 tx=0x7f1cc800b040 comp rx=0 tx=0).stop 2024-04-12T21:31:12.441 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:12.442+0000 7f1ce2c00700 1 -- 172.21.15.18:0/2730356951 shutdown_connections 2024-04-12T21:31:12.441 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:12.442+0000 7f1ce2c00700 1 --2- 172.21.15.18:0/2730356951 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f1cc406ec00 0x7f1cc40710b0 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:31:12.441 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:12.442+0000 7f1ce2c00700 1 --2- 172.21.15.18:0/2730356951 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1cdc0708a0 0x7f1cdc119ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:31:12.441 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:12.442+0000 7f1ce2c00700 1 --2- 172.21.15.18:0/2730356951 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1cdc071f90 0x7f1cdc11a530 unknown :-1 s=CLOSED pgs=795 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:31:12.441 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:12.442+0000 7f1ce2c00700 1 -- 172.21.15.18:0/2730356951 >> 172.21.15.18:0/2730356951 conn(0x7f1cdc0f87f0 msgr2=0x7f1cdc0f9980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:31:12.441 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:12.443+0000 7f1ce2c00700 1 -- 172.21.15.18:0/2730356951 shutdown_connections 2024-04-12T21:31:12.441 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:12.443+0000 7f1ce2c00700 1 -- 172.21.15.18:0/2730356951 wait complete. 2024-04-12T21:31:12.599 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:12.601+0000 7ffabd800700 1 -- 172.21.15.18:0/3697111322 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ffab8105800 msgr2=0x7ffab80fee70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:31:12.600 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:12.601+0000 7ffabd800700 1 --2- 172.21.15.18:0/3697111322 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ffab8105800 0x7ffab80fee70 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7ffaa4004940 tx=0x7ffaa4009e30 comp rx=0 tx=0).stop 2024-04-12T21:31:12.600 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:12.601+0000 7ffabd800700 1 -- 172.21.15.18:0/3697111322 shutdown_connections 2024-04-12T21:31:12.600 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:12.601+0000 7ffabd800700 1 --2- 172.21.15.18:0/3697111322 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ffab8105800 0x7ffab80fee70 unknown :-1 s=CLOSED pgs=796 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:31:12.600 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:12.601+0000 7ffabd800700 1 --2- 172.21.15.18:0/3697111322 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ffab8104e60 0x7ffab8105230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:31:12.600 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:12.601+0000 7ffabd800700 1 -- 172.21.15.18:0/3697111322 >> 172.21.15.18:0/3697111322 conn(0x7ffab80fab10 msgr2=0x7ffab80fcef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:31:12.600 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:12.601+0000 7ffabd800700 1 -- 172.21.15.18:0/3697111322 shutdown_connections 2024-04-12T21:31:12.600 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:12.601+0000 7ffabd800700 1 -- 172.21.15.18:0/3697111322 wait complete. 2024-04-12T21:31:12.601 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:12.602+0000 7ffabd800700 1 Processor -- start 2024-04-12T21:31:12.601 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:12.602+0000 7ffabd800700 1 -- start start 2024-04-12T21:31:12.601 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:12.603+0000 7ffabd800700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ffab8104e60 0x7ffab8115b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:31:12.601 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:12.603+0000 7ffabd800700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ffab8105800 0x7ffab81160d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:31:12.601 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:12.603+0000 7ffabd800700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffab8116720 con 0x7ffab8105800 2024-04-12T21:31:12.602 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:12.603+0000 7ffabd800700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffab810fb10 con 0x7ffab8104e60 2024-04-12T21:31:12.602 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:12.603+0000 7ffab7400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ffab8105800 0x7ffab81160d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:31:12.602 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:12.603+0000 7ffab7400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ffab8105800 0x7ffab81160d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:39594/0 (socket says 172.21.15.18:39594) 2024-04-12T21:31:12.602 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:12.603+0000 7ffab7400700 1 -- 172.21.15.18:0/2971901066 learned_addr learned my addr 172.21.15.18:0/2971901066 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:31:12.603 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:12.603+0000 7ffab7e00700 1 --2- 172.21.15.18:0/2971901066 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ffab8104e60 0x7ffab8115b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:31:12.603 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:12.604+0000 7ffab7400700 1 -- 172.21.15.18:0/2971901066 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ffab8104e60 msgr2=0x7ffab8115b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:31:12.603 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:12.604+0000 7ffab7400700 1 --2- 172.21.15.18:0/2971901066 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ffab8104e60 0x7ffab8115b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:31:12.603 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:12.604+0000 7ffab7400700 1 -- 172.21.15.18:0/2971901066 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffaa4005040 con 0x7ffab8105800 2024-04-12T21:31:12.603 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:12.604+0000 7ffab7400700 1 --2- 172.21.15.18:0/2971901066 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ffab8105800 0x7ffab81160d0 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7ffaa4009180 tx=0x7ffaa4009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:31:12.603 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:12.604+0000 7ffaafe00700 1 -- 172.21.15.18:0/2971901066 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ffaa400e040 con 0x7ffab8105800 2024-04-12T21:31:12.603 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:12.604+0000 7ffaafe00700 1 -- 172.21.15.18:0/2971901066 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ffaa400bbc0 con 0x7ffab8105800 2024-04-12T21:31:12.603 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:12.604+0000 7ffabd800700 1 -- 172.21.15.18:0/2971901066 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffab810fd90 con 0x7ffab8105800 2024-04-12T21:31:12.603 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:12.604+0000 7ffaafe00700 1 -- 172.21.15.18:0/2971901066 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ffaa401c3f0 con 0x7ffab8105800 2024-04-12T21:31:12.603 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:12.605+0000 7ffabd800700 1 -- 172.21.15.18:0/2971901066 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffab8110350 con 0x7ffab8105800 2024-04-12T21:31:12.604 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:12.606+0000 7ffabd800700 1 -- 172.21.15.18:0/2971901066 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffab8107330 con 0x7ffab8105800 2024-04-12T21:31:12.604 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:12.607+0000 7ffaafe00700 1 -- 172.21.15.18:0/2971901066 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7ffaa400bd30 con 0x7ffab8105800 2024-04-12T21:31:12.606 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:12.608+0000 7ffaafe00700 1 --2- 172.21.15.18:0/2971901066 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7ffa9c06ec00 0x7ffa9c0710b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:31:12.606 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:12.608+0000 7ffaafe00700 1 -- 172.21.15.18:0/2971901066 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 6757+0+0 (secure 0 0 0) 0x7ffaa408fd00 con 0x7ffab8105800 2024-04-12T21:31:12.606 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:12.608+0000 7ffab7e00700 1 --2- 172.21.15.18:0/2971901066 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7ffa9c06ec00 0x7ffa9c0710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:31:12.606 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:12.609+0000 7ffab7e00700 1 --2- 172.21.15.18:0/2971901066 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7ffa9c06ec00 0x7ffa9c0710b0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7ffaa8007fd0 tx=0x7ffaa8009040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:31:12.610 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:12.612+0000 7ffaafe00700 1 -- 172.21.15.18:0/2971901066 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7ffaa405c530 con 0x7ffab8105800 2024-04-12T21:31:12.854 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:12.856+0000 7ffabd800700 1 -- 172.21.15.18:0/2971901066 --> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7ffab81111e0 con 0x7ffa9c06ec00 2024-04-12T21:31:12.885 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:12.887+0000 7ffaafe00700 1 -- 172.21.15.18:0/2971901066 <== mgr.34102 v2:172.21.15.18:6800/3812891919 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7ffab81111e0 con 0x7ffa9c06ec00 2024-04-12T21:31:12.886 INFO:teuthology.orchestra.run.smithi018.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-12T21:31:12.886 INFO:teuthology.orchestra.run.smithi018.stdout:alertmanager.smithi018 smithi018 *:9093,9094 running (31m) 8m ago 46m 15.5M - 0.25.0 c8568f914cd2 c332090068e8 2024-04-12T21:31:12.886 INFO:teuthology.orchestra.run.smithi018.stdout:crash.smithi018 smithi018 running (28m) 8m ago 46m 6665k - 19.0.0-2879-g324baf8d 8717720dc5e5 b7f80d1f7982 2024-04-12T21:31:12.886 INFO:teuthology.orchestra.run.smithi018.stdout:crash.smithi186 smithi186 running (28m) 8m ago 45m 6660k - 19.0.0-2879-g324baf8d 8717720dc5e5 a4f55c81b623 2024-04-12T21:31:12.886 INFO:teuthology.orchestra.run.smithi018.stdout:grafana.smithi018 smithi018 *:3000 running (31m) 8m ago 45m 74.8M - 9.4.12 2bacad6d85d8 b08e3bf176dd 2024-04-12T21:31:12.886 INFO:teuthology.orchestra.run.smithi018.stdout:mds.cephfs.smithi018.acqbsc smithi018 running (24m) 8m ago 39m 16.7M - 19.0.0-2879-g324baf8d 8717720dc5e5 021f99ab24ae 2024-04-12T21:31:12.886 INFO:teuthology.orchestra.run.smithi018.stdout:mds.cephfs.smithi018.szjluo smithi018 running (23m) 8m ago 39m 16.5M - 19.0.0-2879-g324baf8d 8717720dc5e5 0003117479d4 2024-04-12T21:31:12.886 INFO:teuthology.orchestra.run.smithi018.stdout:mds.cephfs.smithi186.jdllyn smithi186 running (8m) 8m ago 39m 11.0M - 19.0.0-2879-g324baf8d 8717720dc5e5 21b1f46eef23 2024-04-12T21:31:12.886 INFO:teuthology.orchestra.run.smithi018.stdout:mds.cephfs.smithi186.wjnafs smithi186 running (38m) 8m ago 38m 19.5M - 17.2.7-827-gd3b5060a 21de9b7369ad 74b115d6d6fd 2024-04-12T21:31:12.886 INFO:teuthology.orchestra.run.smithi018.stdout:mgr.smithi018.xjzwga smithi018 *:8443,9283,8765 running (34m) 8m ago 48m 512M - 19.0.0-2879-g324baf8d 8717720dc5e5 56088305eab7 2024-04-12T21:31:12.887 INFO:teuthology.orchestra.run.smithi018.stdout:mgr.smithi186.kqcfxk smithi186 *:8443,9283,8765 running (33m) 8m ago 45m 459M - 19.0.0-2879-g324baf8d 8717720dc5e5 09cd766e5e0d 2024-04-12T21:31:12.887 INFO:teuthology.orchestra.run.smithi018.stdout:mon.smithi018 smithi018 running (29m) 8m ago 48m 73.1M 2048M 19.0.0-2879-g324baf8d 8717720dc5e5 df3e201dfed6 2024-04-12T21:31:12.887 INFO:teuthology.orchestra.run.smithi018.stdout:mon.smithi186 smithi186 running (29m) 8m ago 44m 47.2M 2048M 19.0.0-2879-g324baf8d 8717720dc5e5 c77498da9b2b 2024-04-12T21:31:12.887 INFO:teuthology.orchestra.run.smithi018.stdout:node-exporter.smithi018 smithi018 *:9100 running (32m) 8m ago 46m 8108k - 1.5.0 0da6a335fe13 326c3d9c6847 2024-04-12T21:31:12.887 INFO:teuthology.orchestra.run.smithi018.stdout:node-exporter.smithi186 smithi186 *:9100 running (32m) 8m ago 45m 8757k - 1.5.0 0da6a335fe13 ec2112baeecf 2024-04-12T21:31:12.887 INFO:teuthology.orchestra.run.smithi018.stdout:osd.0 smithi018 running (27m) 8m ago 43m 123M 1559M 19.0.0-2879-g324baf8d 8717720dc5e5 60b1273ca047 2024-04-12T21:31:12.887 INFO:teuthology.orchestra.run.smithi018.stdout:osd.1 smithi018 running (26m) 8m ago 43m 95.1M 1559M 19.0.0-2879-g324baf8d 8717720dc5e5 170ce1e5fb8d 2024-04-12T21:31:12.887 INFO:teuthology.orchestra.run.smithi018.stdout:osd.2 smithi018 running (26m) 8m ago 42m 94.3M 1559M 19.0.0-2879-g324baf8d 8717720dc5e5 c2cbc6aa2839 2024-04-12T21:31:12.887 INFO:teuthology.orchestra.run.smithi018.stdout:osd.3 smithi186 running (25m) 8m ago 42m 101M 2581M 19.0.0-2879-g324baf8d 8717720dc5e5 0138e8262e4b 2024-04-12T21:31:12.887 INFO:teuthology.orchestra.run.smithi018.stdout:osd.4 smithi186 running (25m) 8m ago 41m 91.0M 2581M 19.0.0-2879-g324baf8d 8717720dc5e5 b20cb63b25c9 2024-04-12T21:31:12.887 INFO:teuthology.orchestra.run.smithi018.stdout:osd.5 smithi186 running (24m) 8m ago 41m 92.0M 2581M 19.0.0-2879-g324baf8d 8717720dc5e5 2513bea2610e 2024-04-12T21:31:12.887 INFO:teuthology.orchestra.run.smithi018.stdout:prometheus.smithi018 smithi018 *:9095 running (32m) 8m ago 45m 67.9M - 2.43.0 a07b618ecd1d f58841127e21 2024-04-12T21:31:12.888 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:12.890+0000 7ffabd800700 1 -- 172.21.15.18:0/2971901066 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7ffa9c06ec00 msgr2=0x7ffa9c0710b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:31:12.888 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:12.890+0000 7ffabd800700 1 --2- 172.21.15.18:0/2971901066 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7ffa9c06ec00 0x7ffa9c0710b0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7ffaa8007fd0 tx=0x7ffaa8009040 comp rx=0 tx=0).stop 2024-04-12T21:31:12.888 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:12.890+0000 7ffabd800700 1 -- 172.21.15.18:0/2971901066 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ffab8105800 msgr2=0x7ffab81160d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:31:12.888 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:12.890+0000 7ffabd800700 1 --2- 172.21.15.18:0/2971901066 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ffab8105800 0x7ffab81160d0 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7ffaa4009180 tx=0x7ffaa4009e30 comp rx=0 tx=0).stop 2024-04-12T21:31:12.889 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:12.890+0000 7ffabd800700 1 -- 172.21.15.18:0/2971901066 shutdown_connections 2024-04-12T21:31:12.889 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:12.890+0000 7ffabd800700 1 --2- 172.21.15.18:0/2971901066 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7ffa9c06ec00 0x7ffa9c0710b0 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:31:12.889 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:12.890+0000 7ffabd800700 1 --2- 172.21.15.18:0/2971901066 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ffab8104e60 0x7ffab8115b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:31:12.889 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:12.890+0000 7ffabd800700 1 --2- 172.21.15.18:0/2971901066 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ffab8105800 0x7ffab81160d0 unknown :-1 s=CLOSED pgs=797 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:31:12.889 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:12.890+0000 7ffabd800700 1 -- 172.21.15.18:0/2971901066 >> 172.21.15.18:0/2971901066 conn(0x7ffab80fab10 msgr2=0x7ffab80fd930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:31:12.889 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:12.890+0000 7ffabd800700 1 -- 172.21.15.18:0/2971901066 shutdown_connections 2024-04-12T21:31:12.889 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:12.890+0000 7ffabd800700 1 -- 172.21.15.18:0/2971901066 wait complete. 2024-04-12T21:31:13.031 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:31:12 smithi186 ceph-mon[86379]: pgmap v943: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:31:13.042 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:13.044+0000 7fb857000700 1 -- 172.21.15.18:0/2958690342 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb850065630 msgr2=0x7fb850065a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:31:13.042 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:13.044+0000 7fb857000700 1 --2- 172.21.15.18:0/2958690342 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb850065630 0x7fb850065a80 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7fb83c004940 tx=0x7fb83c009e30 comp rx=0 tx=0).stop 2024-04-12T21:31:13.042 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:31:12 smithi018 ceph-mon[112590]: pgmap v943: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:31:13.043 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:13.044+0000 7fb857000700 1 -- 172.21.15.18:0/2958690342 shutdown_connections 2024-04-12T21:31:13.043 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:13.044+0000 7fb857000700 1 --2- 172.21.15.18:0/2958690342 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb850065630 0x7fb850065a80 unknown :-1 s=CLOSED pgs=798 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:31:13.043 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:13.044+0000 7fb857000700 1 --2- 172.21.15.18:0/2958690342 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb850102ca0 0x7fb850103070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:31:13.043 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:13.044+0000 7fb857000700 1 -- 172.21.15.18:0/2958690342 >> 172.21.15.18:0/2958690342 conn(0x7fb850071b70 msgr2=0x7fb850071f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:31:13.043 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:13.044+0000 7fb857000700 1 -- 172.21.15.18:0/2958690342 shutdown_connections 2024-04-12T21:31:13.043 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:13.044+0000 7fb857000700 1 -- 172.21.15.18:0/2958690342 wait complete. 2024-04-12T21:31:13.043 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:13.045+0000 7fb857000700 1 Processor -- start 2024-04-12T21:31:13.044 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:13.045+0000 7fb857000700 1 -- start start 2024-04-12T21:31:13.044 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:13.046+0000 7fb857000700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb850065630 0x7fb850115bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:31:13.044 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:13.046+0000 7fb857000700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb850102ca0 0x7fb8501160f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:31:13.045 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:13.046+0000 7fb857000700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb850116740 con 0x7fb850065630 2024-04-12T21:31:13.045 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:13.046+0000 7fb857000700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb85010fb80 con 0x7fb850102ca0 2024-04-12T21:31:13.045 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:13.046+0000 7fb855c00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb850065630 0x7fb850115bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:31:13.045 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:13.046+0000 7fb855c00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb850065630 0x7fb850115bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:39606/0 (socket says 172.21.15.18:39606) 2024-04-12T21:31:13.045 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:13.046+0000 7fb855c00700 1 -- 172.21.15.18:0/3454466071 learned_addr learned my addr 172.21.15.18:0/3454466071 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:31:13.045 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:13.046+0000 7fb855200700 1 --2- 172.21.15.18:0/3454466071 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb850102ca0 0x7fb8501160f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:31:13.046 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:13.047+0000 7fb855c00700 1 -- 172.21.15.18:0/3454466071 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb850102ca0 msgr2=0x7fb8501160f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:31:13.046 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:13.047+0000 7fb855c00700 1 --2- 172.21.15.18:0/3454466071 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb850102ca0 0x7fb8501160f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:31:13.046 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:13.047+0000 7fb855c00700 1 -- 172.21.15.18:0/3454466071 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb83c005040 con 0x7fb850065630 2024-04-12T21:31:13.046 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:13.047+0000 7fb855c00700 1 --2- 172.21.15.18:0/3454466071 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb850065630 0x7fb850115bb0 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7fb84400c830 tx=0x7fb844004470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:31:13.046 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:13.047+0000 7fb84d600700 1 -- 172.21.15.18:0/3454466071 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb84400a760 con 0x7fb850065630 2024-04-12T21:31:13.046 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:13.047+0000 7fb84d600700 1 -- 172.21.15.18:0/3454466071 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb84400ad90 con 0x7fb850065630 2024-04-12T21:31:13.046 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:13.047+0000 7fb857000700 1 -- 172.21.15.18:0/3454466071 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb85010fe60 con 0x7fb850065630 2024-04-12T21:31:13.046 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:13.047+0000 7fb84d600700 1 -- 172.21.15.18:0/3454466071 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb8440113a0 con 0x7fb850065630 2024-04-12T21:31:13.046 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:13.048+0000 7fb857000700 1 -- 172.21.15.18:0/3454466071 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb850110420 con 0x7fb850065630 2024-04-12T21:31:13.047 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:13.049+0000 7fb857000700 1 -- 172.21.15.18:0/3454466071 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb850101660 con 0x7fb850065630 2024-04-12T21:31:13.048 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:13.050+0000 7fb84d600700 1 -- 172.21.15.18:0/3454466071 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7fb844012400 con 0x7fb850065630 2024-04-12T21:31:13.049 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:13.051+0000 7fb84d600700 1 --2- 172.21.15.18:0/3454466071 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fb83806ec10 0x7fb8380710c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:31:13.049 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:13.051+0000 7fb84d600700 1 -- 172.21.15.18:0/3454466071 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 6757+0+0 (secure 0 0 0) 0x7fb84408ea00 con 0x7fb850065630 2024-04-12T21:31:13.049 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:13.052+0000 7fb855200700 1 --2- 172.21.15.18:0/3454466071 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fb83806ec10 0x7fb8380710c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:31:13.050 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:13.052+0000 7fb855200700 1 --2- 172.21.15.18:0/3454466071 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fb83806ec10 0x7fb8380710c0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7fb850110cf0 tx=0x7fb83c009e30 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:31:13.054 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:13.056+0000 7fb84d600700 1 -- 172.21.15.18:0/3454466071 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7fb84405b250 con 0x7fb850065630 2024-04-12T21:31:13.377 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:13.379+0000 7fb857000700 1 -- 172.21.15.18:0/3454466071 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fb850111200 con 0x7fb850065630 2024-04-12T21:31:13.378 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:13.380+0000 7fb84d600700 1 -- 172.21.15.18:0/3454466071 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+837 (secure 0 0 0) 0x7fb84405ade0 con 0x7fb850065630 2024-04-12T21:31:13.379 INFO:teuthology.orchestra.run.smithi018.stdout:{ 2024-04-12T21:31:13.379 INFO:teuthology.orchestra.run.smithi018.stdout: "mon": { 2024-04-12T21:31:13.379 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 19.0.0-2879-g324baf8d (324baf8ddb91dea46cac50cb535c306115791355) squid (dev)": 2 2024-04-12T21:31:13.379 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-12T21:31:13.379 INFO:teuthology.orchestra.run.smithi018.stdout: "mgr": { 2024-04-12T21:31:13.379 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 19.0.0-2879-g324baf8d (324baf8ddb91dea46cac50cb535c306115791355) squid (dev)": 2 2024-04-12T21:31:13.379 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-12T21:31:13.379 INFO:teuthology.orchestra.run.smithi018.stdout: "osd": { 2024-04-12T21:31:13.379 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 19.0.0-2879-g324baf8d (324baf8ddb91dea46cac50cb535c306115791355) squid (dev)": 6 2024-04-12T21:31:13.379 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-12T21:31:13.379 INFO:teuthology.orchestra.run.smithi018.stdout: "mds": { 2024-04-12T21:31:13.380 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 17.2.7-827-gd3b5060a (d3b5060a17449edfae9385cdf792afec109fca5d) quincy (stable)": 1, 2024-04-12T21:31:13.380 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 19.0.0-2879-g324baf8d (324baf8ddb91dea46cac50cb535c306115791355) squid (dev)": 3 2024-04-12T21:31:13.380 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-12T21:31:13.380 INFO:teuthology.orchestra.run.smithi018.stdout: "overall": { 2024-04-12T21:31:13.380 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 17.2.7-827-gd3b5060a (d3b5060a17449edfae9385cdf792afec109fca5d) quincy (stable)": 1, 2024-04-12T21:31:13.380 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 19.0.0-2879-g324baf8d (324baf8ddb91dea46cac50cb535c306115791355) squid (dev)": 13 2024-04-12T21:31:13.380 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-04-12T21:31:13.380 INFO:teuthology.orchestra.run.smithi018.stdout:} 2024-04-12T21:31:13.382 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:13.384+0000 7fb857000700 1 -- 172.21.15.18:0/3454466071 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fb83806ec10 msgr2=0x7fb8380710c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:31:13.382 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:13.384+0000 7fb857000700 1 --2- 172.21.15.18:0/3454466071 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fb83806ec10 0x7fb8380710c0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7fb850110cf0 tx=0x7fb83c009e30 comp rx=0 tx=0).stop 2024-04-12T21:31:13.382 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:13.384+0000 7fb857000700 1 -- 172.21.15.18:0/3454466071 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb850065630 msgr2=0x7fb850115bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:31:13.383 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:13.384+0000 7fb857000700 1 --2- 172.21.15.18:0/3454466071 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb850065630 0x7fb850115bb0 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7fb84400c830 tx=0x7fb844004470 comp rx=0 tx=0).stop 2024-04-12T21:31:13.383 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:13.384+0000 7fb857000700 1 -- 172.21.15.18:0/3454466071 shutdown_connections 2024-04-12T21:31:13.383 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:13.384+0000 7fb857000700 1 --2- 172.21.15.18:0/3454466071 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fb83806ec10 0x7fb8380710c0 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:31:13.383 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:13.384+0000 7fb857000700 1 --2- 172.21.15.18:0/3454466071 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb850065630 0x7fb850115bb0 unknown :-1 s=CLOSED pgs=799 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:31:13.383 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:13.384+0000 7fb857000700 1 --2- 172.21.15.18:0/3454466071 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb850102ca0 0x7fb8501160f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:31:13.383 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:13.384+0000 7fb857000700 1 -- 172.21.15.18:0/3454466071 >> 172.21.15.18:0/3454466071 conn(0x7fb850071b70 msgr2=0x7fb8500fe7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:31:13.383 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:13.384+0000 7fb857000700 1 -- 172.21.15.18:0/3454466071 shutdown_connections 2024-04-12T21:31:13.383 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:13.384+0000 7fb857000700 1 -- 172.21.15.18:0/3454466071 wait complete. 2024-04-12T21:31:13.542 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:13.543+0000 7fd810e00700 1 -- 172.21.15.18:0/3627392228 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd80c109430 msgr2=0x7fd80c109800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:31:13.542 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:13.543+0000 7fd810e00700 1 --2- 172.21.15.18:0/3627392228 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd80c109430 0x7fd80c109800 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7fd7fc004940 tx=0x7fd7fc009e30 comp rx=0 tx=0).stop 2024-04-12T21:31:13.542 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:13.543+0000 7fd810e00700 1 -- 172.21.15.18:0/3627392228 shutdown_connections 2024-04-12T21:31:13.542 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:13.543+0000 7fd810e00700 1 --2- 172.21.15.18:0/3627392228 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd80c109dd0 0x7fd80c103440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:31:13.542 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:13.543+0000 7fd810e00700 1 --2- 172.21.15.18:0/3627392228 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd80c109430 0x7fd80c109800 unknown :-1 s=CLOSED pgs=800 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:31:13.542 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:13.543+0000 7fd810e00700 1 -- 172.21.15.18:0/3627392228 >> 172.21.15.18:0/3627392228 conn(0x7fd80c072c00 msgr2=0x7fd80c075010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:31:13.542 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:13.544+0000 7fd810e00700 1 -- 172.21.15.18:0/3627392228 shutdown_connections 2024-04-12T21:31:13.542 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:13.544+0000 7fd810e00700 1 -- 172.21.15.18:0/3627392228 wait complete. 2024-04-12T21:31:13.543 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:13.544+0000 7fd810e00700 1 Processor -- start 2024-04-12T21:31:13.544 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:13.545+0000 7fd810e00700 1 -- start start 2024-04-12T21:31:13.544 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:13.545+0000 7fd810e00700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd80c109430 0x7fd80c119f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:31:13.544 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:13.545+0000 7fd810e00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd80c109dd0 0x7fd80c11a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:31:13.544 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:13.545+0000 7fd810e00700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd80c11ab60 con 0x7fd80c109dd0 2024-04-12T21:31:13.544 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:13.545+0000 7fd810e00700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd80c1140b0 con 0x7fd80c109430 2024-04-12T21:31:13.544 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:13.545+0000 7fd809400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd80c109dd0 0x7fd80c11a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:31:13.544 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:13.545+0000 7fd809400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd80c109dd0 0x7fd80c11a480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:39626/0 (socket says 172.21.15.18:39626) 2024-04-12T21:31:13.544 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:13.545+0000 7fd809400700 1 -- 172.21.15.18:0/1357354613 learned_addr learned my addr 172.21.15.18:0/1357354613 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:31:13.544 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:13.546+0000 7fd809e00700 1 --2- 172.21.15.18:0/1357354613 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd80c109430 0x7fd80c119f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:31:13.544 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:13.546+0000 7fd809400700 1 -- 172.21.15.18:0/1357354613 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd80c109430 msgr2=0x7fd80c119f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:31:13.544 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:13.546+0000 7fd809400700 1 --2- 172.21.15.18:0/1357354613 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd80c109430 0x7fd80c119f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:31:13.544 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:13.546+0000 7fd809400700 1 -- 172.21.15.18:0/1357354613 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7fc005040 con 0x7fd80c109dd0 2024-04-12T21:31:13.545 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:13.546+0000 7fd809400700 1 --2- 172.21.15.18:0/1357354613 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd80c109dd0 0x7fd80c11a480 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7fd7f800b790 tx=0x7fd7f8009220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:31:13.545 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:13.546+0000 7fd802000700 1 -- 172.21.15.18:0/1357354613 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd7f800f6f0 con 0x7fd80c109dd0 2024-04-12T21:31:13.545 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:13.546+0000 7fd802000700 1 -- 172.21.15.18:0/1357354613 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd7f800fd20 con 0x7fd80c109dd0 2024-04-12T21:31:13.545 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:13.546+0000 7fd810e00700 1 -- 172.21.15.18:0/1357354613 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd80c114330 con 0x7fd80c109dd0 2024-04-12T21:31:13.545 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:13.546+0000 7fd802000700 1 -- 172.21.15.18:0/1357354613 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd7f80103a0 con 0x7fd80c109dd0 2024-04-12T21:31:13.545 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:13.547+0000 7fd810e00700 1 -- 172.21.15.18:0/1357354613 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd80c1148f0 con 0x7fd80c109dd0 2024-04-12T21:31:13.546 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:13.548+0000 7fd810e00700 1 -- 172.21.15.18:0/1357354613 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd80c104b80 con 0x7fd80c109dd0 2024-04-12T21:31:13.546 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:13.549+0000 7fd802000700 1 -- 172.21.15.18:0/1357354613 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7fd7f8011400 con 0x7fd80c109dd0 2024-04-12T21:31:13.547 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:13.550+0000 7fd802000700 1 --2- 172.21.15.18:0/1357354613 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fd7f006ec00 0x7fd7f00710b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:31:13.548 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:13.550+0000 7fd802000700 1 -- 172.21.15.18:0/1357354613 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 6757+0+0 (secure 0 0 0) 0x7fd7f808e820 con 0x7fd80c109dd0 2024-04-12T21:31:13.548 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:13.550+0000 7fd809e00700 1 --2- 172.21.15.18:0/1357354613 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fd7f006ec00 0x7fd7f00710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:31:13.548 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:13.550+0000 7fd809e00700 1 --2- 172.21.15.18:0/1357354613 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fd7f006ec00 0x7fd7f00710b0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7fd7fc00b020 tx=0x7fd7fc009e30 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:31:13.552 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:13.554+0000 7fd802000700 1 -- 172.21.15.18:0/1357354613 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7fd7f805afd0 con 0x7fd80c109dd0 2024-04-12T21:31:13.835 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:13.837+0000 7fd810e00700 1 -- 172.21.15.18:0/1357354613 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fd80c1105a0 con 0x7fd80c109dd0 2024-04-12T21:31:13.835 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:13.837+0000 7fd802000700 1 -- 172.21.15.18:0/1357354613 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 31 v31) v1 ==== 76+0+1873 (secure 0 0 0) 0x7fd7f805ab60 con 0x7fd80c109dd0 2024-04-12T21:31:13.836 INFO:teuthology.orchestra.run.smithi018.stdout:e31 2024-04-12T21:31:13.836 INFO:teuthology.orchestra.run.smithi018.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-12T21:31:13.836 INFO:teuthology.orchestra.run.smithi018.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-12T21:31:13.836 INFO:teuthology.orchestra.run.smithi018.stdout:legacy client fscid: 1 2024-04-12T21:31:13.837 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-12T21:31:13.837 INFO:teuthology.orchestra.run.smithi018.stdout:Filesystem 'cephfs' (1) 2024-04-12T21:31:13.837 INFO:teuthology.orchestra.run.smithi018.stdout:fs_name cephfs 2024-04-12T21:31:13.837 INFO:teuthology.orchestra.run.smithi018.stdout:epoch 24 2024-04-12T21:31:13.837 INFO:teuthology.orchestra.run.smithi018.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-04-12T21:31:13.837 INFO:teuthology.orchestra.run.smithi018.stdout:created 2024-04-12T20:51:57.362850+0000 2024-04-12T21:31:13.837 INFO:teuthology.orchestra.run.smithi018.stdout:modified 2024-04-12T21:06:59.717190+0000 2024-04-12T21:31:13.837 INFO:teuthology.orchestra.run.smithi018.stdout:tableserver 0 2024-04-12T21:31:13.837 INFO:teuthology.orchestra.run.smithi018.stdout:root 0 2024-04-12T21:31:13.837 INFO:teuthology.orchestra.run.smithi018.stdout:session_timeout 60 2024-04-12T21:31:13.837 INFO:teuthology.orchestra.run.smithi018.stdout:session_autoclose 300 2024-04-12T21:31:13.837 INFO:teuthology.orchestra.run.smithi018.stdout:max_file_size 1099511627776 2024-04-12T21:31:13.837 INFO:teuthology.orchestra.run.smithi018.stdout:max_xattr_size 65536 2024-04-12T21:31:13.837 INFO:teuthology.orchestra.run.smithi018.stdout:required_client_features {} 2024-04-12T21:31:13.838 INFO:teuthology.orchestra.run.smithi018.stdout:last_failure 0 2024-04-12T21:31:13.838 INFO:teuthology.orchestra.run.smithi018.stdout:last_failure_osd_epoch 100 2024-04-12T21:31:13.838 INFO:teuthology.orchestra.run.smithi018.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-12T21:31:13.838 INFO:teuthology.orchestra.run.smithi018.stdout:max_mds 2 2024-04-12T21:31:13.838 INFO:teuthology.orchestra.run.smithi018.stdout:in 0,1 2024-04-12T21:31:13.838 INFO:teuthology.orchestra.run.smithi018.stdout:up {} 2024-04-12T21:31:13.838 INFO:teuthology.orchestra.run.smithi018.stdout:failed 0,1 2024-04-12T21:31:13.838 INFO:teuthology.orchestra.run.smithi018.stdout:damaged 2024-04-12T21:31:13.838 INFO:teuthology.orchestra.run.smithi018.stdout:stopped 2024-04-12T21:31:13.838 INFO:teuthology.orchestra.run.smithi018.stdout:data_pools [3] 2024-04-12T21:31:13.838 INFO:teuthology.orchestra.run.smithi018.stdout:metadata_pool 2 2024-04-12T21:31:13.838 INFO:teuthology.orchestra.run.smithi018.stdout:inline_data disabled 2024-04-12T21:31:13.838 INFO:teuthology.orchestra.run.smithi018.stdout:balancer 2024-04-12T21:31:13.838 INFO:teuthology.orchestra.run.smithi018.stdout:bal_rank_mask -1 2024-04-12T21:31:13.838 INFO:teuthology.orchestra.run.smithi018.stdout:standby_count_wanted 1 2024-04-12T21:31:13.839 INFO:teuthology.orchestra.run.smithi018.stdout:qdb_cluster leader: 0 members: 2024-04-12T21:31:13.839 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-12T21:31:13.839 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-12T21:31:13.839 INFO:teuthology.orchestra.run.smithi018.stdout:Standby daemons: 2024-04-12T21:31:13.839 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-12T21:31:13.839 INFO:teuthology.orchestra.run.smithi018.stdout:[mds.cephfs.smithi186.wjnafs{-1:24277} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.186:6826/438347207,v1:172.21.15.186:6827/438347207] compat {c=[1],r=[1],i=[7ff]}] 2024-04-12T21:31:13.839 INFO:teuthology.orchestra.run.smithi018.stdout:[mds.cephfs.smithi018.acqbsc{-1:34436} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.18:6828/755179764,v1:172.21.15.18:6829/755179764] compat {c=[1],r=[1],i=[1fff]}] 2024-04-12T21:31:13.839 INFO:teuthology.orchestra.run.smithi018.stdout:[mds.cephfs.smithi018.szjluo{-1:34468} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.18:6826/2407992653,v1:172.21.15.18:6827/2407992653] compat {c=[1],r=[1],i=[1fff]}] 2024-04-12T21:31:13.839 INFO:teuthology.orchestra.run.smithi018.stdout:[mds.cephfs.smithi186.jdllyn{-1:44595} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.186:6824/3523444202,v1:172.21.15.186:6825/3523444202] compat {c=[1],r=[1],i=[1fff]}] 2024-04-12T21:31:13.840 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:13.841+0000 7fd810e00700 1 -- 172.21.15.18:0/1357354613 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fd7f006ec00 msgr2=0x7fd7f00710b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:31:13.840 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:13.841+0000 7fd810e00700 1 --2- 172.21.15.18:0/1357354613 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fd7f006ec00 0x7fd7f00710b0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7fd7fc00b020 tx=0x7fd7fc009e30 comp rx=0 tx=0).stop 2024-04-12T21:31:13.840 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:13.841+0000 7fd810e00700 1 -- 172.21.15.18:0/1357354613 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd80c109dd0 msgr2=0x7fd80c11a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:31:13.840 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:13.841+0000 7fd810e00700 1 --2- 172.21.15.18:0/1357354613 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd80c109dd0 0x7fd80c11a480 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7fd7f800b790 tx=0x7fd7f8009220 comp rx=0 tx=0).stop 2024-04-12T21:31:13.840 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:13.841+0000 7fd810e00700 1 -- 172.21.15.18:0/1357354613 shutdown_connections 2024-04-12T21:31:13.840 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:13.841+0000 7fd810e00700 1 --2- 172.21.15.18:0/1357354613 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fd7f006ec00 0x7fd7f00710b0 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:31:13.840 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:13.841+0000 7fd810e00700 1 --2- 172.21.15.18:0/1357354613 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd80c109430 0x7fd80c119f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:31:13.840 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:13.841+0000 7fd810e00700 1 --2- 172.21.15.18:0/1357354613 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd80c109dd0 0x7fd80c11a480 unknown :-1 s=CLOSED pgs=801 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:31:13.840 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:13.841+0000 7fd810e00700 1 -- 172.21.15.18:0/1357354613 >> 172.21.15.18:0/1357354613 conn(0x7fd80c072c00 msgr2=0x7fd80c074c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:31:13.840 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:13.842+0000 7fd810e00700 1 -- 172.21.15.18:0/1357354613 shutdown_connections 2024-04-12T21:31:13.840 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:13.842+0000 7fd810e00700 1 -- 172.21.15.18:0/1357354613 wait complete. 2024-04-12T21:31:13.841 INFO:teuthology.orchestra.run.smithi018.stderr:dumped fsmap epoch 31 2024-04-12T21:31:14.002 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:14.004+0000 7f7d81a00700 1 -- 172.21.15.18:0/2393815429 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7d7c070c70 msgr2=0x7f7d7c1090f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:31:14.002 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:14.004+0000 7f7d81a00700 1 --2- 172.21.15.18:0/2393815429 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7d7c070c70 0x7f7d7c1090f0 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7f7d68004970 tx=0x7f7d68009e30 comp rx=0 tx=0).stop 2024-04-12T21:31:14.002 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:14.004+0000 7f7d81a00700 1 -- 172.21.15.18:0/2393815429 shutdown_connections 2024-04-12T21:31:14.003 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:14.004+0000 7f7d81a00700 1 --2- 172.21.15.18:0/2393815429 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7d7c070c70 0x7f7d7c1090f0 unknown :-1 s=CLOSED pgs=802 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:31:14.003 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:14.004+0000 7f7d81a00700 1 --2- 172.21.15.18:0/2393815429 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7d7c070360 0x7f7d7c070730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:31:14.003 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:14.004+0000 7f7d81a00700 1 -- 172.21.15.18:0/2393815429 >> 172.21.15.18:0/2393815429 conn(0x7f7d7c0f8970 msgr2=0x7f7d7c0fad80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:31:14.003 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:31:13 smithi018 ceph-mon[112590]: from='client.35652 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:31:14.003 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:31:13 smithi018 ceph-mon[112590]: from='client.35656 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:31:14.003 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:31:13 smithi018 ceph-mon[112590]: from='client.35660 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:31:14.003 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:31:13 smithi018 ceph-mon[112590]: from='client.? 172.21.15.18:0/3454466071' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T21:31:14.004 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:14.004+0000 7f7d81a00700 1 -- 172.21.15.18:0/2393815429 shutdown_connections 2024-04-12T21:31:14.004 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:14.004+0000 7f7d81a00700 1 -- 172.21.15.18:0/2393815429 wait complete. 2024-04-12T21:31:14.004 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:14.006+0000 7f7d81a00700 1 Processor -- start 2024-04-12T21:31:14.004 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:14.006+0000 7f7d81a00700 1 -- start start 2024-04-12T21:31:14.005 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:14.006+0000 7f7d81a00700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7d7c070360 0x7f7d7c115d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:31:14.005 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:14.006+0000 7f7d81a00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7d7c070c70 0x7f7d7c116290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:31:14.005 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:14.006+0000 7f7d81a00700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d7c1168e0 con 0x7f7d7c070c70 2024-04-12T21:31:14.005 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:14.006+0000 7f7d81a00700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d7c10fcb0 con 0x7f7d7c070360 2024-04-12T21:31:14.005 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:14.007+0000 7f7d7a000700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7d7c070c70 0x7f7d7c116290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:31:14.005 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:14.007+0000 7f7d7a000700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7d7c070c70 0x7f7d7c116290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:39638/0 (socket says 172.21.15.18:39638) 2024-04-12T21:31:14.005 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:14.007+0000 7f7d7a000700 1 -- 172.21.15.18:0/511467732 learned_addr learned my addr 172.21.15.18:0/511467732 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:31:14.006 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:14.007+0000 7f7d7aa00700 1 --2- 172.21.15.18:0/511467732 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7d7c070360 0x7f7d7c115d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:31:14.006 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:14.007+0000 7f7d7a000700 1 -- 172.21.15.18:0/511467732 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7d7c070360 msgr2=0x7f7d7c115d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:31:14.006 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:14.007+0000 7f7d7a000700 1 --2- 172.21.15.18:0/511467732 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7d7c070360 0x7f7d7c115d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:31:14.006 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:14.007+0000 7f7d7a000700 1 -- 172.21.15.18:0/511467732 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d68005040 con 0x7f7d7c070c70 2024-04-12T21:31:14.006 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:14.008+0000 7f7d7a000700 1 --2- 172.21.15.18:0/511467732 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7d7c070c70 0x7f7d7c116290 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7f7d68004f50 tx=0x7f7d68009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:31:14.007 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:14.008+0000 7f7d72a00700 1 -- 172.21.15.18:0/511467732 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7d6800e040 con 0x7f7d7c070c70 2024-04-12T21:31:14.007 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:14.008+0000 7f7d72a00700 1 -- 172.21.15.18:0/511467732 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f7d6800bc10 con 0x7f7d7c070c70 2024-04-12T21:31:14.007 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:14.008+0000 7f7d72a00700 1 -- 172.21.15.18:0/511467732 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7d6801c630 con 0x7f7d7c070c70 2024-04-12T21:31:14.007 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:14.008+0000 7f7d81a00700 1 -- 172.21.15.18:0/511467732 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7d7c10ff30 con 0x7f7d7c070c70 2024-04-12T21:31:14.007 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:14.008+0000 7f7d81a00700 1 -- 172.21.15.18:0/511467732 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7d7c110420 con 0x7f7d7c070c70 2024-04-12T21:31:14.008 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:14.010+0000 7f7d81a00700 1 -- 172.21.15.18:0/511467732 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7d7c071aa0 con 0x7f7d7c070c70 2024-04-12T21:31:14.009 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:14.011+0000 7f7d72a00700 1 -- 172.21.15.18:0/511467732 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7f7d68013420 con 0x7f7d7c070c70 2024-04-12T21:31:14.010 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:14.012+0000 7f7d72a00700 1 --2- 172.21.15.18:0/511467732 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f7d6006ec10 0x7f7d600710c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:31:14.011 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:14.013+0000 7f7d72a00700 1 -- 172.21.15.18:0/511467732 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 6757+0+0 (secure 0 0 0) 0x7f7d6808fd40 con 0x7f7d7c070c70 2024-04-12T21:31:14.011 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:14.013+0000 7f7d7aa00700 1 --2- 172.21.15.18:0/511467732 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f7d6006ec10 0x7f7d600710c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:31:14.011 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:14.014+0000 7f7d7aa00700 1 --2- 172.21.15.18:0/511467732 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f7d6006ec10 0x7f7d600710c0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f7d6c004990 tx=0x7f7d6c009ee0 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:31:14.015 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:14.017+0000 7f7d72a00700 1 -- 172.21.15.18:0/511467732 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f7d6805c4f0 con 0x7f7d7c070c70 2024-04-12T21:31:14.031 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:31:13 smithi186 ceph-mon[86379]: from='client.35652 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:31:14.031 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:31:13 smithi186 ceph-mon[86379]: from='client.35656 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:31:14.031 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:31:13 smithi186 ceph-mon[86379]: from='client.35660 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:31:14.031 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:31:13 smithi186 ceph-mon[86379]: from='client.? 172.21.15.18:0/3454466071' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T21:31:14.272 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:14.273+0000 7f7d81a00700 1 -- 172.21.15.18:0/511467732 --> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f7d7c10c0d0 con 0x7f7d6006ec10 2024-04-12T21:31:14.274 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:14.276+0000 7f7d72a00700 1 -- 172.21.15.18:0/511467732 <== mgr.34102 v2:172.21.15.18:6800/3812891919 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f7d7c10c0d0 con 0x7f7d6006ec10 2024-04-12T21:31:14.274 INFO:teuthology.orchestra.run.smithi018.stdout:{ 2024-04-12T21:31:14.274 INFO:teuthology.orchestra.run.smithi018.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:f4452def3be4d350f4a8e5feaf7ebb1511be9298fe68478ba7a2d6fbd7d69a63", 2024-04-12T21:31:14.274 INFO:teuthology.orchestra.run.smithi018.stdout: "in_progress": true, 2024-04-12T21:31:14.274 INFO:teuthology.orchestra.run.smithi018.stdout: "which": "Upgrading all daemon types on all hosts", 2024-04-12T21:31:14.274 INFO:teuthology.orchestra.run.smithi018.stdout: "services_complete": [ 2024-04-12T21:31:14.274 INFO:teuthology.orchestra.run.smithi018.stdout: "crash", 2024-04-12T21:31:14.275 INFO:teuthology.orchestra.run.smithi018.stdout: "osd", 2024-04-12T21:31:14.275 INFO:teuthology.orchestra.run.smithi018.stdout: "mon", 2024-04-12T21:31:14.275 INFO:teuthology.orchestra.run.smithi018.stdout: "mgr" 2024-04-12T21:31:14.275 INFO:teuthology.orchestra.run.smithi018.stdout: ], 2024-04-12T21:31:14.275 INFO:teuthology.orchestra.run.smithi018.stdout: "progress": "15/21 daemons upgraded", 2024-04-12T21:31:14.275 INFO:teuthology.orchestra.run.smithi018.stdout: "message": "Currently upgrading mds daemons", 2024-04-12T21:31:14.275 INFO:teuthology.orchestra.run.smithi018.stdout: "is_paused": false 2024-04-12T21:31:14.275 INFO:teuthology.orchestra.run.smithi018.stdout:} 2024-04-12T21:31:14.277 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:14.279+0000 7f7d81a00700 1 -- 172.21.15.18:0/511467732 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f7d6006ec10 msgr2=0x7f7d600710c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:31:14.277 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:14.279+0000 7f7d81a00700 1 --2- 172.21.15.18:0/511467732 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f7d6006ec10 0x7f7d600710c0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f7d6c004990 tx=0x7f7d6c009ee0 comp rx=0 tx=0).stop 2024-04-12T21:31:14.278 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:14.279+0000 7f7d81a00700 1 -- 172.21.15.18:0/511467732 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7d7c070c70 msgr2=0x7f7d7c116290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:31:14.278 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:14.279+0000 7f7d81a00700 1 --2- 172.21.15.18:0/511467732 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7d7c070c70 0x7f7d7c116290 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7f7d68004f50 tx=0x7f7d68009e30 comp rx=0 tx=0).stop 2024-04-12T21:31:14.278 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:14.279+0000 7f7d81a00700 1 -- 172.21.15.18:0/511467732 shutdown_connections 2024-04-12T21:31:14.278 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:14.279+0000 7f7d81a00700 1 --2- 172.21.15.18:0/511467732 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f7d6006ec10 0x7f7d600710c0 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:31:14.278 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:14.279+0000 7f7d81a00700 1 --2- 172.21.15.18:0/511467732 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7d7c070360 0x7f7d7c115d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:31:14.278 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:14.279+0000 7f7d81a00700 1 --2- 172.21.15.18:0/511467732 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7d7c070c70 0x7f7d7c116290 unknown :-1 s=CLOSED pgs=803 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:31:14.278 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:14.279+0000 7f7d81a00700 1 -- 172.21.15.18:0/511467732 >> 172.21.15.18:0/511467732 conn(0x7f7d7c0f8970 msgr2=0x7f7d7c0f9d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:31:14.278 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:14.279+0000 7f7d81a00700 1 -- 172.21.15.18:0/511467732 shutdown_connections 2024-04-12T21:31:14.278 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:14.279+0000 7f7d81a00700 1 -- 172.21.15.18:0/511467732 wait complete. 2024-04-12T21:31:14.440 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:14.441+0000 7fc5f1c00700 1 -- 172.21.15.18:0/3199201524 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc5ec105800 msgr2=0x7fc5ec0fee70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:31:14.440 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:14.441+0000 7fc5f1c00700 1 --2- 172.21.15.18:0/3199201524 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc5ec105800 0x7fc5ec0fee70 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7fc5d4004940 tx=0x7fc5d4009e30 comp rx=0 tx=0).stop 2024-04-12T21:31:14.440 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:14.442+0000 7fc5f1c00700 1 -- 172.21.15.18:0/3199201524 shutdown_connections 2024-04-12T21:31:14.441 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:14.442+0000 7fc5f1c00700 1 --2- 172.21.15.18:0/3199201524 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc5ec105800 0x7fc5ec0fee70 unknown :-1 s=CLOSED pgs=804 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:31:14.441 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:14.442+0000 7fc5f1c00700 1 --2- 172.21.15.18:0/3199201524 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc5ec104e60 0x7fc5ec105230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:31:14.441 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:14.442+0000 7fc5f1c00700 1 -- 172.21.15.18:0/3199201524 >> 172.21.15.18:0/3199201524 conn(0x7fc5ec0fab10 msgr2=0x7fc5ec0fcef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:31:14.441 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:14.442+0000 7fc5f1c00700 1 -- 172.21.15.18:0/3199201524 shutdown_connections 2024-04-12T21:31:14.441 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:14.442+0000 7fc5f1c00700 1 -- 172.21.15.18:0/3199201524 wait complete. 2024-04-12T21:31:14.442 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:14.443+0000 7fc5f1c00700 1 Processor -- start 2024-04-12T21:31:14.442 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:14.443+0000 7fc5f1c00700 1 -- start start 2024-04-12T21:31:14.442 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:14.443+0000 7fc5f1c00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc5ec104e60 0x7fc5ec115c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:31:14.442 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:14.444+0000 7fc5f1c00700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc5ec105800 0x7fc5ec116140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:31:14.442 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:14.444+0000 7fc5f1c00700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5ec116820 con 0x7fc5ec104e60 2024-04-12T21:31:14.442 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:14.444+0000 7fc5f1c00700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5ec10fb80 con 0x7fc5ec105800 2024-04-12T21:31:14.442 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:14.444+0000 7fc5ebe00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc5ec104e60 0x7fc5ec115c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:31:14.443 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:14.444+0000 7fc5ebe00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc5ec104e60 0x7fc5ec115c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:39666/0 (socket says 172.21.15.18:39666) 2024-04-12T21:31:14.443 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:14.444+0000 7fc5ebe00700 1 -- 172.21.15.18:0/2141476669 learned_addr learned my addr 172.21.15.18:0/2141476669 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:31:14.443 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:14.444+0000 7fc5e3e00700 1 --2- 172.21.15.18:0/2141476669 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc5ec105800 0x7fc5ec116140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:31:14.444 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:14.444+0000 7fc5ebe00700 1 -- 172.21.15.18:0/2141476669 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc5ec105800 msgr2=0x7fc5ec116140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:31:14.444 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:14.444+0000 7fc5ebe00700 1 --2- 172.21.15.18:0/2141476669 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc5ec105800 0x7fc5ec116140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:31:14.444 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:14.444+0000 7fc5ebe00700 1 -- 172.21.15.18:0/2141476669 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5d4005040 con 0x7fc5ec104e60 2024-04-12T21:31:14.444 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:14.445+0000 7fc5ebe00700 1 --2- 172.21.15.18:0/2141476669 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc5ec104e60 0x7fc5ec115c00 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7fc5dc00b790 tx=0x7fc5dc009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:31:14.444 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:14.445+0000 7fc5e8c00700 1 -- 172.21.15.18:0/2141476669 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc5dc00f770 con 0x7fc5ec104e60 2024-04-12T21:31:14.444 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:14.445+0000 7fc5e8c00700 1 -- 172.21.15.18:0/2141476669 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc5dc00fda0 con 0x7fc5ec104e60 2024-04-12T21:31:14.444 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:14.445+0000 7fc5f1c00700 1 -- 172.21.15.18:0/2141476669 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc5ec10fe60 con 0x7fc5ec104e60 2024-04-12T21:31:14.444 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:14.445+0000 7fc5e8c00700 1 -- 172.21.15.18:0/2141476669 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc5dc00d310 con 0x7fc5ec104e60 2024-04-12T21:31:14.444 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:14.445+0000 7fc5f1c00700 1 -- 172.21.15.18:0/2141476669 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc5ec110420 con 0x7fc5ec104e60 2024-04-12T21:31:14.445 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:14.446+0000 7fc5f1c00700 1 -- 172.21.15.18:0/2141476669 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc5ec1005b0 con 0x7fc5ec104e60 2024-04-12T21:31:14.445 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:14.447+0000 7fc5e8c00700 1 -- 172.21.15.18:0/2141476669 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7fc5dc010420 con 0x7fc5ec104e60 2024-04-12T21:31:14.446 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:14.448+0000 7fc5e8c00700 1 --2- 172.21.15.18:0/2141476669 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fc5d006ec00 0x7fc5d00710b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:31:14.447 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:14.449+0000 7fc5e8c00700 1 -- 172.21.15.18:0/2141476669 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 6757+0+0 (secure 0 0 0) 0x7fc5dc08e9a0 con 0x7fc5ec104e60 2024-04-12T21:31:14.447 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:14.449+0000 7fc5e3e00700 1 --2- 172.21.15.18:0/2141476669 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fc5d006ec00 0x7fc5d00710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:31:14.447 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:14.449+0000 7fc5e3e00700 1 --2- 172.21.15.18:0/2141476669 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fc5d006ec00 0x7fc5d00710b0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7fc5ec111430 tx=0x7fc5d4009e30 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:31:14.451 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:14.454+0000 7fc5e8c00700 1 -- 172.21.15.18:0/2141476669 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7fc5dc05b1d0 con 0x7fc5ec104e60 2024-04-12T21:31:14.778 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:14.780+0000 7fc5f1c00700 1 -- 172.21.15.18:0/2141476669 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7fc5ec04bf60 con 0x7fc5ec104e60 2024-04-12T21:31:14.778 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:14.780+0000 7fc5e8c00700 1 -- 172.21.15.18:0/2141476669 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+4303 (secure 0 0 0) 0x7fc5dc05ad60 con 0x7fc5ec104e60 2024-04-12T21:31:14.779 INFO:teuthology.orchestra.run.smithi018.stdout:HEALTH_ERR failed to probe daemons or devices; 1 filesystem is degraded; 1 filesystem is offline 2024-04-12T21:31:14.779 INFO:teuthology.orchestra.run.smithi018.stdout:[WRN] CEPHADM_REFRESH_FAILED: failed to probe daemons or devices 2024-04-12T21:31:14.779 INFO:teuthology.orchestra.run.smithi018.stdout: host smithi018 `cephadm ceph-volume` failed: cephadm exited with an error code: 1, stderr: Inferring config /var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/mon.smithi018/config 2024-04-12T21:31:14.779 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/podman: timeout after 895 seconds 2024-04-12T21:31:14.779 INFO:teuthology.orchestra.run.smithi018.stdout:Non-zero exit code 124 from /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:f4452def3be4d350f4a8e5feaf7ebb1511be9298fe68478ba7a2d6fbd7d69a63 -e NODE_NAME=smithi018 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968:/var/run/ceph:z -v /var/log/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968:/var/log/ceph:z -v /var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmpxsmwkkuh:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:f4452def3be4d350f4a8e5feaf7ebb1511be9298fe68478ba7a2d6fbd7d69a63 inventory --format=json-pretty --filter-for-batch 2024-04-12T21:31:14.779 INFO:teuthology.orchestra.run.smithi018.stdout:Traceback (most recent call last): 2024-04-12T21:31:14.779 INFO:teuthology.orchestra.run.smithi018.stdout: File "/usr/lib64/python3.9/runpy.py", line 197, in _run_module_as_main 2024-04-12T21:31:14.779 INFO:teuthology.orchestra.run.smithi018.stdout: return _run_code(code, main_globals, None, 2024-04-12T21:31:14.779 INFO:teuthology.orchestra.run.smithi018.stdout: File "/usr/lib64/python3.9/runpy.py", line 87, in _run_code 2024-04-12T21:31:14.779 INFO:teuthology.orchestra.run.smithi018.stdout: exec(code, run_globals) 2024-04-12T21:31:14.779 INFO:teuthology.orchestra.run.smithi018.stdout: File "/var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/cephadm.e44f3ed5e34e4ef52eedef233d1190bff004470d8e8005127b5312bff357fa7f/__main__.py", line 5618, in 2024-04-12T21:31:14.779 INFO:teuthology.orchestra.run.smithi018.stdout: File "/var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/cephadm.e44f3ed5e34e4ef52eedef233d1190bff004470d8e8005127b5312bff357fa7f/__main__.py", line 5606, in main 2024-04-12T21:31:14.779 INFO:teuthology.orchestra.run.smithi018.stdout: File "/var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/cephadm.e44f3ed5e34e4ef52eedef233d1190bff004470d8e8005127b5312bff357fa7f/__main__.py", line 408, in _infer_config 2024-04-12T21:31:14.780 INFO:teuthology.orchestra.run.smithi018.stdout: File "/var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/cephadm.e44f3ed5e34e4ef52eedef233d1190bff004470d8e8005127b5312bff357fa7f/__main__.py", line 323, in _infer_fsid 2024-04-12T21:31:14.780 INFO:teuthology.orchestra.run.smithi018.stdout: File "/var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/cephadm.e44f3ed5e34e4ef52eedef233d1190bff004470d8e8005127b5312bff357fa7f/__main__.py", line 436, in _infer_image 2024-04-12T21:31:14.780 INFO:teuthology.orchestra.run.smithi018.stdout: File "/var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/cephadm.e44f3ed5e34e4ef52eedef233d1190bff004470d8e8005127b5312bff357fa7f/__main__.py", line 310, in _validate_fsid 2024-04-12T21:31:14.780 INFO:teuthology.orchestra.run.smithi018.stdout: File "/var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/cephadm.e44f3ed5e34e4ef52eedef233d1190bff004470d8e8005127b5312bff357fa7f/__main__.py", line 3359, in command_ceph_volume 2024-04-12T21:31:14.780 INFO:teuthology.orchestra.run.smithi018.stdout: File "/var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/cephadm.e44f3ed5e34e4ef52eedef233d1190bff004470d8e8005127b5312bff357fa7f/cephadmlib/call_wrappers.py", line 310, in call_throws 2024-04-12T21:31:14.780 INFO:teuthology.orchestra.run.smithi018.stdout:RuntimeError: Failed command: /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:f4452def3be4d350f4a8e5feaf7ebb1511be9298fe68478ba7a2d6fbd7d69a63 -e NODE_NAME=smithi018 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968:/var/run/ceph:z -v /var/log/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968:/var/log/ceph:z -v /var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmpxsmwkkuh:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:f4452def3be4d350f4a8e5feaf7ebb1511be9298fe68478ba7a2d6fbd7d69a63 inventory --format=json-pretty --filter-for-batch 2024-04-12T21:31:14.780 INFO:teuthology.orchestra.run.smithi018.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-04-12T21:31:14.780 INFO:teuthology.orchestra.run.smithi018.stdout: fs cephfs is degraded 2024-04-12T21:31:14.780 INFO:teuthology.orchestra.run.smithi018.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-04-12T21:31:14.780 INFO:teuthology.orchestra.run.smithi018.stdout: fs cephfs is offline because no MDS is active for it. 2024-04-12T21:31:14.782 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:14.783+0000 7fc5f1c00700 1 -- 172.21.15.18:0/2141476669 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fc5d006ec00 msgr2=0x7fc5d00710b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:31:14.782 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:14.783+0000 7fc5f1c00700 1 --2- 172.21.15.18:0/2141476669 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fc5d006ec00 0x7fc5d00710b0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7fc5ec111430 tx=0x7fc5d4009e30 comp rx=0 tx=0).stop 2024-04-12T21:31:14.782 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:14.783+0000 7fc5f1c00700 1 -- 172.21.15.18:0/2141476669 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc5ec104e60 msgr2=0x7fc5ec115c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:31:14.782 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:14.783+0000 7fc5f1c00700 1 --2- 172.21.15.18:0/2141476669 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc5ec104e60 0x7fc5ec115c00 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7fc5dc00b790 tx=0x7fc5dc009250 comp rx=0 tx=0).stop 2024-04-12T21:31:14.782 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:14.784+0000 7fc5f1c00700 1 -- 172.21.15.18:0/2141476669 shutdown_connections 2024-04-12T21:31:14.782 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:14.784+0000 7fc5f1c00700 1 --2- 172.21.15.18:0/2141476669 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fc5d006ec00 0x7fc5d00710b0 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:31:14.782 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:14.784+0000 7fc5f1c00700 1 --2- 172.21.15.18:0/2141476669 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc5ec104e60 0x7fc5ec115c00 unknown :-1 s=CLOSED pgs=805 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:31:14.782 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:14.784+0000 7fc5f1c00700 1 --2- 172.21.15.18:0/2141476669 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc5ec105800 0x7fc5ec116140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:31:14.782 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:14.784+0000 7fc5f1c00700 1 -- 172.21.15.18:0/2141476669 >> 172.21.15.18:0/2141476669 conn(0x7fc5ec0fab10 msgr2=0x7fc5ec0fca90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:31:14.783 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:14.784+0000 7fc5f1c00700 1 -- 172.21.15.18:0/2141476669 shutdown_connections 2024-04-12T21:31:14.783 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:14.784+0000 7fc5f1c00700 1 -- 172.21.15.18:0/2141476669 wait complete. 2024-04-12T21:31:15.031 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:31:14 smithi186 ceph-mon[86379]: pgmap v944: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:31:15.031 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:31:14 smithi186 ceph-mon[86379]: from='client.? 172.21.15.18:0/1357354613' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-12T21:31:15.102 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:31:14 smithi018 ceph-mon[112590]: pgmap v944: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:31:15.103 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:31:14 smithi018 ceph-mon[112590]: from='client.? 172.21.15.18:0/1357354613' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-12T21:31:16.031 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:31:15 smithi186 ceph-mon[86379]: from='client.35672 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:31:16.031 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:31:15 smithi186 ceph-mon[86379]: from='client.? 172.21.15.18:0/2141476669' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-12T21:31:16.102 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:31:15 smithi018 ceph-mon[112590]: from='client.35672 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:31:16.103 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:31:15 smithi018 ceph-mon[112590]: from='client.? 172.21.15.18:0/2141476669' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-12T21:31:17.031 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:31:16 smithi186 ceph-mon[86379]: pgmap v945: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:31:17.102 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:31:16 smithi018 ceph-mon[112590]: pgmap v945: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:31:19.031 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:31:18 smithi186 ceph-mon[86379]: pgmap v946: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:31:19.102 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:31:18 smithi018 ceph-mon[112590]: pgmap v946: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:31:21.031 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:31:20 smithi186 ceph-mon[86379]: pgmap v947: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:31:21.102 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:31:20 smithi018 ceph-mon[112590]: pgmap v947: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:31:23.031 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:31:22 smithi186 ceph-mon[86379]: pgmap v948: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:31:23.102 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:31:22 smithi018 ceph-mon[112590]: pgmap v948: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:31:25.031 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:31:24 smithi186 ceph-mon[86379]: pgmap v949: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:31:25.102 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:31:24 smithi018 ceph-mon[112590]: pgmap v949: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:31:26.033 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:31:25 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T21:31:26.102 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:31:25 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T21:31:27.031 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:31:26 smithi186 ceph-mon[86379]: pgmap v950: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:31:27.102 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:31:26 smithi018 ceph-mon[112590]: pgmap v950: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:31:29.031 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:31:28 smithi186 ceph-mon[86379]: pgmap v951: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:31:29.102 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:31:28 smithi018 ceph-mon[112590]: pgmap v951: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:31:31.031 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:31:30 smithi186 ceph-mon[86379]: pgmap v952: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:31:31.102 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:31:30 smithi018 ceph-mon[112590]: pgmap v952: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:31:33.102 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:31:32 smithi018 ceph-mon[112590]: pgmap v953: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:31:33.281 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:31:32 smithi186 ceph-mon[86379]: pgmap v953: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:31:35.102 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:31:34 smithi018 ceph-mon[112590]: pgmap v954: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:31:35.282 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:31:34 smithi186 ceph-mon[86379]: pgmap v954: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:31:36.911 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:31:36 smithi186 ceph-mon[86379]: pgmap v955: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:31:37.102 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:31:36 smithi018 ceph-mon[112590]: pgmap v955: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:31:39.102 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:31:38 smithi018 ceph-mon[112590]: pgmap v956: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:31:39.281 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:31:38 smithi186 ceph-mon[86379]: pgmap v956: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:31:41.102 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:31:40 smithi018 ceph-mon[112590]: pgmap v957: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:31:41.103 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:31:40 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T21:31:41.281 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:31:40 smithi186 ceph-mon[86379]: pgmap v957: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:31:41.282 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:31:40 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T21:31:43.084 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:31:42 smithi186 ceph-mon[86379]: pgmap v958: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:31:43.102 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:31:42 smithi018 ceph-mon[112590]: pgmap v958: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:31:44.956 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:44.957+0000 7f41c1400700 1 -- 172.21.15.18:0/1303208733 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f41bc109430 msgr2=0x7f41bc109800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:31:44.957 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:44.957+0000 7f41c1400700 1 --2- 172.21.15.18:0/1303208733 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f41bc109430 0x7f41bc109800 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7f41ac004940 tx=0x7f41ac009e30 comp rx=0 tx=0).stop 2024-04-12T21:31:44.957 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:44.957+0000 7f41c1400700 1 -- 172.21.15.18:0/1303208733 shutdown_connections 2024-04-12T21:31:44.957 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:44.957+0000 7f41c1400700 1 --2- 172.21.15.18:0/1303208733 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f41bc109dd0 0x7f41bc103440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:31:44.957 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:44.957+0000 7f41c1400700 1 --2- 172.21.15.18:0/1303208733 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f41bc109430 0x7f41bc109800 unknown :-1 s=CLOSED pgs=806 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:31:44.957 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:44.957+0000 7f41c1400700 1 -- 172.21.15.18:0/1303208733 >> 172.21.15.18:0/1303208733 conn(0x7f41bc072c00 msgr2=0x7f41bc075010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:31:44.957 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:44.957+0000 7f41c1400700 1 -- 172.21.15.18:0/1303208733 shutdown_connections 2024-04-12T21:31:44.957 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:44.957+0000 7f41c1400700 1 -- 172.21.15.18:0/1303208733 wait complete. 2024-04-12T21:31:44.958 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:44.958+0000 7f41c1400700 1 Processor -- start 2024-04-12T21:31:44.958 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:44.958+0000 7f41c1400700 1 -- start start 2024-04-12T21:31:44.958 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:44.959+0000 7f41c1400700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f41bc109430 0x7f41bc119f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:31:44.958 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:44.959+0000 7f41c1400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f41bc109dd0 0x7f41bc11a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:31:44.959 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:44.959+0000 7f41c1400700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41bc11aad0 con 0x7f41bc109dd0 2024-04-12T21:31:44.959 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:44.959+0000 7f41c1400700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41bc114100 con 0x7f41bc109430 2024-04-12T21:31:44.959 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:44.959+0000 7f41b1e00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f41bc109dd0 0x7f41bc11a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:31:44.959 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:44.959+0000 7f41b1e00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f41bc109dd0 0x7f41bc11a480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:40120/0 (socket says 172.21.15.18:40120) 2024-04-12T21:31:44.959 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:44.959+0000 7f41b1e00700 1 -- 172.21.15.18:0/63979583 learned_addr learned my addr 172.21.15.18:0/63979583 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:31:44.959 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:44.959+0000 7f41ba800700 1 --2- 172.21.15.18:0/63979583 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f41bc109430 0x7f41bc119f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:31:44.960 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:44.960+0000 7f41b1e00700 1 -- 172.21.15.18:0/63979583 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f41bc109430 msgr2=0x7f41bc119f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:31:44.960 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:44.960+0000 7f41b1e00700 1 --2- 172.21.15.18:0/63979583 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f41bc109430 0x7f41bc119f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:31:44.960 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:44.960+0000 7f41b1e00700 1 -- 172.21.15.18:0/63979583 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41ac005040 con 0x7f41bc109dd0 2024-04-12T21:31:44.960 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:44.960+0000 7f41b1e00700 1 --2- 172.21.15.18:0/63979583 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f41bc109dd0 0x7f41bc11a480 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7f41a4002a50 tx=0x7f41a4004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:31:44.960 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:44.960+0000 7f41b3400700 1 -- 172.21.15.18:0/63979583 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f41a4007cc0 con 0x7f41bc109dd0 2024-04-12T21:31:44.960 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:44.961+0000 7f41b3400700 1 -- 172.21.15.18:0/63979583 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f41a400f040 con 0x7f41bc109dd0 2024-04-12T21:31:44.960 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:44.961+0000 7f41c1400700 1 -- 172.21.15.18:0/63979583 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f41bc1143e0 con 0x7f41bc109dd0 2024-04-12T21:31:44.960 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:44.961+0000 7f41b3400700 1 -- 172.21.15.18:0/63979583 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f41a400d9c0 con 0x7f41bc109dd0 2024-04-12T21:31:44.960 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:44.961+0000 7f41c1400700 1 -- 172.21.15.18:0/63979583 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f41bc1149a0 con 0x7f41bc109dd0 2024-04-12T21:31:44.961 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:44.962+0000 7f41c1400700 1 -- 172.21.15.18:0/63979583 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f41bc104b80 con 0x7f41bc109dd0 2024-04-12T21:31:44.962 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:44.963+0000 7f41b3400700 1 -- 172.21.15.18:0/63979583 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7f41a4009730 con 0x7f41bc109dd0 2024-04-12T21:31:44.963 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:44.964+0000 7f41b3400700 1 --2- 172.21.15.18:0/63979583 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f41a006ec00 0x7f41a00710b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:31:44.963 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:44.964+0000 7f41b3400700 1 -- 172.21.15.18:0/63979583 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 6757+0+0 (secure 0 0 0) 0x7f41a408e630 con 0x7f41bc109dd0 2024-04-12T21:31:44.963 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:44.964+0000 7f41ba800700 1 --2- 172.21.15.18:0/63979583 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f41a006ec00 0x7f41a00710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:31:44.964 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:44.965+0000 7f41ba800700 1 --2- 172.21.15.18:0/63979583 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f41a006ec00 0x7f41a00710b0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f41ac009fd0 tx=0x7f41ac009e30 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:31:44.968 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:44.969+0000 7f41b3400700 1 -- 172.21.15.18:0/63979583 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f41a405ade0 con 0x7f41bc109dd0 2024-04-12T21:31:45.102 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:31:44 smithi018 ceph-mon[112590]: pgmap v959: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:31:45.218 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:45.219+0000 7f41c1400700 1 -- 172.21.15.18:0/63979583 --> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f41bc110590 con 0x7f41a006ec00 2024-04-12T21:31:45.221 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:45.221+0000 7f41b3400700 1 -- 172.21.15.18:0/63979583 <== mgr.34102 v2:172.21.15.18:6800/3812891919 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f41bc110590 con 0x7f41a006ec00 2024-04-12T21:31:45.223 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:45.224+0000 7f41c1400700 1 -- 172.21.15.18:0/63979583 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f41a006ec00 msgr2=0x7f41a00710b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:31:45.223 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:45.224+0000 7f41c1400700 1 --2- 172.21.15.18:0/63979583 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f41a006ec00 0x7f41a00710b0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f41ac009fd0 tx=0x7f41ac009e30 comp rx=0 tx=0).stop 2024-04-12T21:31:45.223 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:45.224+0000 7f41c1400700 1 -- 172.21.15.18:0/63979583 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f41bc109dd0 msgr2=0x7f41bc11a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:31:45.223 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:45.224+0000 7f41c1400700 1 --2- 172.21.15.18:0/63979583 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f41bc109dd0 0x7f41bc11a480 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7f41a4002a50 tx=0x7f41a4004600 comp rx=0 tx=0).stop 2024-04-12T21:31:45.224 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:45.224+0000 7f41c1400700 1 -- 172.21.15.18:0/63979583 shutdown_connections 2024-04-12T21:31:45.224 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:45.224+0000 7f41c1400700 1 --2- 172.21.15.18:0/63979583 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f41a006ec00 0x7f41a00710b0 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:31:45.224 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:45.224+0000 7f41c1400700 1 --2- 172.21.15.18:0/63979583 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f41bc109430 0x7f41bc119f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:31:45.224 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:45.224+0000 7f41c1400700 1 --2- 172.21.15.18:0/63979583 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f41bc109dd0 0x7f41bc11a480 unknown :-1 s=CLOSED pgs=807 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:31:45.224 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:45.224+0000 7f41c1400700 1 -- 172.21.15.18:0/63979583 >> 172.21.15.18:0/63979583 conn(0x7f41bc072c00 msgr2=0x7f41bc074c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:31:45.224 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:45.224+0000 7f41c1400700 1 -- 172.21.15.18:0/63979583 shutdown_connections 2024-04-12T21:31:45.224 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:45.224+0000 7f41c1400700 1 -- 172.21.15.18:0/63979583 wait complete. 2024-04-12T21:31:45.239 INFO:teuthology.orchestra.run.smithi018.stdout:true 2024-04-12T21:31:45.281 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:31:44 smithi186 ceph-mon[86379]: pgmap v959: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:31:45.369 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:45.369+0000 7f82ef400700 1 -- 172.21.15.18:0/2033871984 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f82e8109430 msgr2=0x7f82e8109800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:31:45.369 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:45.369+0000 7f82ef400700 1 --2- 172.21.15.18:0/2033871984 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f82e8109430 0x7f82e8109800 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7f82dc004940 tx=0x7f82dc009e30 comp rx=0 tx=0).stop 2024-04-12T21:31:45.369 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:45.369+0000 7f82ef400700 1 -- 172.21.15.18:0/2033871984 shutdown_connections 2024-04-12T21:31:45.369 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:45.370+0000 7f82ef400700 1 --2- 172.21.15.18:0/2033871984 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f82e8109dd0 0x7f82e8103440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:31:45.369 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:45.370+0000 7f82ef400700 1 --2- 172.21.15.18:0/2033871984 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f82e8109430 0x7f82e8109800 unknown :-1 s=CLOSED pgs=808 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:31:45.369 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:45.370+0000 7f82ef400700 1 -- 172.21.15.18:0/2033871984 >> 172.21.15.18:0/2033871984 conn(0x7f82e8072c00 msgr2=0x7f82e8075010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:31:45.369 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:45.370+0000 7f82ef400700 1 -- 172.21.15.18:0/2033871984 shutdown_connections 2024-04-12T21:31:45.369 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:45.370+0000 7f82ef400700 1 -- 172.21.15.18:0/2033871984 wait complete. 2024-04-12T21:31:45.370 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:45.371+0000 7f82ef400700 1 Processor -- start 2024-04-12T21:31:45.370 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:45.371+0000 7f82ef400700 1 -- start start 2024-04-12T21:31:45.370 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:45.371+0000 7f82ef400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f82e8109430 0x7f82e8119f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:31:45.371 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:45.371+0000 7f82ef400700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f82e8109dd0 0x7f82e811a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:31:45.371 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:45.371+0000 7f82ef400700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82e811ab60 con 0x7f82e8109430 2024-04-12T21:31:45.371 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:45.371+0000 7f82ef400700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82e81140b0 con 0x7f82e8109dd0 2024-04-12T21:31:45.371 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:45.371+0000 7f82eca00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f82e8109430 0x7f82e8119f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:31:45.371 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:45.372+0000 7f82eca00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f82e8109430 0x7f82e8119f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:40132/0 (socket says 172.21.15.18:40132) 2024-04-12T21:31:45.371 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:45.372+0000 7f82eca00700 1 -- 172.21.15.18:0/1218452003 learned_addr learned my addr 172.21.15.18:0/1218452003 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:31:45.372 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:45.372+0000 7f82eca00700 1 -- 172.21.15.18:0/1218452003 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f82e8109dd0 msgr2=0x7f82e811a480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:31:45.372 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:45.372+0000 7f82eca00700 1 --2- 172.21.15.18:0/1218452003 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f82e8109dd0 0x7f82e811a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:31:45.372 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:45.372+0000 7f82eca00700 1 -- 172.21.15.18:0/1218452003 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82dc005040 con 0x7f82e8109430 2024-04-12T21:31:45.372 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:45.372+0000 7f82eca00700 1 --2- 172.21.15.18:0/1218452003 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f82e8109430 0x7f82e8119f40 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7f82dc009150 tx=0x7f82dc009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:31:45.372 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:45.373+0000 7f82e4c00700 1 -- 172.21.15.18:0/1218452003 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f82dc00e040 con 0x7f82e8109430 2024-04-12T21:31:45.372 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:45.373+0000 7f82e4c00700 1 -- 172.21.15.18:0/1218452003 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f82dc013420 con 0x7f82e8109430 2024-04-12T21:31:45.372 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:45.373+0000 7f82e4c00700 1 -- 172.21.15.18:0/1218452003 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f82dc01c630 con 0x7f82e8109430 2024-04-12T21:31:45.372 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:45.373+0000 7f82ef400700 1 -- 172.21.15.18:0/1218452003 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f82e81142d0 con 0x7f82e8109430 2024-04-12T21:31:45.372 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:45.373+0000 7f82ef400700 1 -- 172.21.15.18:0/1218452003 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f82e8114890 con 0x7f82e8109430 2024-04-12T21:31:45.373 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:45.374+0000 7f82ef400700 1 -- 172.21.15.18:0/1218452003 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f82e8104b80 con 0x7f82e8109430 2024-04-12T21:31:45.374 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:45.375+0000 7f82e4c00700 1 -- 172.21.15.18:0/1218452003 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7f82dc013590 con 0x7f82e8109430 2024-04-12T21:31:45.375 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:45.376+0000 7f82e4c00700 1 --2- 172.21.15.18:0/1218452003 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f82d006ec00 0x7f82d00710b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:31:45.375 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:45.376+0000 7f82e4c00700 1 -- 172.21.15.18:0/1218452003 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 6757+0+0 (secure 0 0 0) 0x7f82dc08fc10 con 0x7f82e8109430 2024-04-12T21:31:45.375 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:45.376+0000 7f82e7e00700 1 --2- 172.21.15.18:0/1218452003 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f82d006ec00 0x7f82d00710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:31:45.376 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:45.377+0000 7f82e7e00700 1 --2- 172.21.15.18:0/1218452003 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f82d006ec00 0x7f82d00710b0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f82d4006fd0 tx=0x7f82d4006db0 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:31:45.381 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:45.382+0000 7f82e4c00700 1 -- 172.21.15.18:0/1218452003 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f82dc05c440 con 0x7f82e8109430 2024-04-12T21:31:45.628 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:45.629+0000 7f82ef400700 1 -- 172.21.15.18:0/1218452003 --> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f82e8114bc0 con 0x7f82d006ec00 2024-04-12T21:31:45.631 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:45.632+0000 7f82e4c00700 1 -- 172.21.15.18:0/1218452003 <== mgr.34102 v2:172.21.15.18:6800/3812891919 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f82e8114bc0 con 0x7f82d006ec00 2024-04-12T21:31:45.635 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:45.635+0000 7f82ef400700 1 -- 172.21.15.18:0/1218452003 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f82d006ec00 msgr2=0x7f82d00710b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:31:45.635 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:45.635+0000 7f82ef400700 1 --2- 172.21.15.18:0/1218452003 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f82d006ec00 0x7f82d00710b0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f82d4006fd0 tx=0x7f82d4006db0 comp rx=0 tx=0).stop 2024-04-12T21:31:45.635 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:45.635+0000 7f82ef400700 1 -- 172.21.15.18:0/1218452003 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f82e8109430 msgr2=0x7f82e8119f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:31:45.635 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:45.635+0000 7f82ef400700 1 --2- 172.21.15.18:0/1218452003 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f82e8109430 0x7f82e8119f40 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7f82dc009150 tx=0x7f82dc009e30 comp rx=0 tx=0).stop 2024-04-12T21:31:45.635 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:45.635+0000 7f82ef400700 1 -- 172.21.15.18:0/1218452003 shutdown_connections 2024-04-12T21:31:45.635 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:45.635+0000 7f82ef400700 1 --2- 172.21.15.18:0/1218452003 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f82d006ec00 0x7f82d00710b0 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:31:45.635 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:45.635+0000 7f82ef400700 1 --2- 172.21.15.18:0/1218452003 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f82e8109430 0x7f82e8119f40 unknown :-1 s=CLOSED pgs=809 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:31:45.635 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:45.635+0000 7f82ef400700 1 --2- 172.21.15.18:0/1218452003 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f82e8109dd0 0x7f82e811a480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:31:45.635 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:45.635+0000 7f82ef400700 1 -- 172.21.15.18:0/1218452003 >> 172.21.15.18:0/1218452003 conn(0x7f82e8072c00 msgr2=0x7f82e8074c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:31:45.635 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:45.636+0000 7f82ef400700 1 -- 172.21.15.18:0/1218452003 shutdown_connections 2024-04-12T21:31:45.636 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:45.636+0000 7f82ef400700 1 -- 172.21.15.18:0/1218452003 wait complete. 2024-04-12T21:31:45.793 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:45.793+0000 7f43fac00700 1 -- 172.21.15.18:0/3986513006 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f43f4071f90 msgr2=0x7f43f4070360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:31:45.793 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:45.793+0000 7f43fac00700 1 --2- 172.21.15.18:0/3986513006 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f43f4071f90 0x7f43f4070360 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7f43e4004970 tx=0x7f43e4009e30 comp rx=0 tx=0).stop 2024-04-12T21:31:45.793 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:45.794+0000 7f43fac00700 1 -- 172.21.15.18:0/3986513006 shutdown_connections 2024-04-12T21:31:45.793 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:45.794+0000 7f43fac00700 1 --2- 172.21.15.18:0/3986513006 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f43f40708a0 0x7f43f4070d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:31:45.793 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:45.794+0000 7f43fac00700 1 --2- 172.21.15.18:0/3986513006 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f43f4071f90 0x7f43f4070360 unknown :-1 s=CLOSED pgs=810 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:31:45.793 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:45.794+0000 7f43fac00700 1 -- 172.21.15.18:0/3986513006 >> 172.21.15.18:0/3986513006 conn(0x7f43f40f89b0 msgr2=0x7f43f40fadc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:31:45.794 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:45.794+0000 7f43fac00700 1 -- 172.21.15.18:0/3986513006 shutdown_connections 2024-04-12T21:31:45.794 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:45.794+0000 7f43fac00700 1 -- 172.21.15.18:0/3986513006 wait complete. 2024-04-12T21:31:45.795 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:45.795+0000 7f43fac00700 1 Processor -- start 2024-04-12T21:31:45.795 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:45.795+0000 7f43fac00700 1 -- start start 2024-04-12T21:31:45.795 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:45.795+0000 7f43fac00700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f43f40708a0 0x7f43f411a1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:31:45.795 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:45.795+0000 7f43fac00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f43f4071f90 0x7f43f411a6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:31:45.795 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:45.795+0000 7f43fac00700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43f411ad30 con 0x7f43f4071f90 2024-04-12T21:31:45.795 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:45.795+0000 7f43fac00700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43f4114100 con 0x7f43f40708a0 2024-04-12T21:31:45.795 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:45.796+0000 7f43eb400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f43f4071f90 0x7f43f411a6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:31:45.795 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:45.796+0000 7f43eb400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f43f4071f90 0x7f43f411a6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:40152/0 (socket says 172.21.15.18:40152) 2024-04-12T21:31:45.795 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:45.796+0000 7f43eb400700 1 -- 172.21.15.18:0/2242151551 learned_addr learned my addr 172.21.15.18:0/2242151551 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:31:45.796 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:45.796+0000 7f43f3e00700 1 --2- 172.21.15.18:0/2242151551 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f43f40708a0 0x7f43f411a1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:31:45.796 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:45.796+0000 7f43eb400700 1 -- 172.21.15.18:0/2242151551 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f43f40708a0 msgr2=0x7f43f411a1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:31:45.797 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:45.796+0000 7f43eb400700 1 --2- 172.21.15.18:0/2242151551 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f43f40708a0 0x7f43f411a1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:31:45.797 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:45.796+0000 7f43eb400700 1 -- 172.21.15.18:0/2242151551 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43e4005040 con 0x7f43f4071f90 2024-04-12T21:31:45.797 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:45.796+0000 7f43f3e00700 1 --2- 172.21.15.18:0/2242151551 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f43f40708a0 0x7f43f411a1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-12T21:31:45.797 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:45.796+0000 7f43eb400700 1 --2- 172.21.15.18:0/2242151551 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f43f4071f90 0x7f43f411a6e0 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7f43dc00c8c0 tx=0x7f43dc0044c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:31:45.797 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:45.797+0000 7f43f0c00700 1 -- 172.21.15.18:0/2242151551 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f43dc00a980 con 0x7f43f4071f90 2024-04-12T21:31:45.797 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:45.797+0000 7f43f0c00700 1 -- 172.21.15.18:0/2242151551 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f43dc00ce60 con 0x7f43f4071f90 2024-04-12T21:31:45.797 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:45.797+0000 7f43f0c00700 1 -- 172.21.15.18:0/2242151551 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f43dc0115e0 con 0x7f43f4071f90 2024-04-12T21:31:45.797 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:45.797+0000 7f43fac00700 1 -- 172.21.15.18:0/2242151551 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f43f41143e0 con 0x7f43f4071f90 2024-04-12T21:31:45.797 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:45.797+0000 7f43fac00700 1 -- 172.21.15.18:0/2242151551 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f43f4114920 con 0x7f43f4071f90 2024-04-12T21:31:45.797 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:45.798+0000 7f43fac00700 1 -- 172.21.15.18:0/2242151551 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f43f4071aa0 con 0x7f43f4071f90 2024-04-12T21:31:45.798 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:45.799+0000 7f43f0c00700 1 -- 172.21.15.18:0/2242151551 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7f43dc012400 con 0x7f43f4071f90 2024-04-12T21:31:45.799 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:45.800+0000 7f43f0c00700 1 --2- 172.21.15.18:0/2242151551 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f43d806ec10 0x7f43d80710c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:31:45.799 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:45.800+0000 7f43f0c00700 1 -- 172.21.15.18:0/2242151551 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 6757+0+0 (secure 0 0 0) 0x7f43dc08ea40 con 0x7f43f4071f90 2024-04-12T21:31:45.799 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:45.800+0000 7f43f3e00700 1 --2- 172.21.15.18:0/2242151551 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f43d806ec10 0x7f43d80710c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:31:45.800 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:45.801+0000 7f43f3e00700 1 --2- 172.21.15.18:0/2242151551 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f43d806ec10 0x7f43d80710c0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f43e400b9c0 tx=0x7f43e4009e30 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:31:45.804 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:45.805+0000 7f43f0c00700 1 -- 172.21.15.18:0/2242151551 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f43dc05b270 con 0x7f43f4071f90 2024-04-12T21:31:46.035 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:46.036+0000 7f43fac00700 1 -- 172.21.15.18:0/2242151551 --> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f43f4110520 con 0x7f43d806ec10 2024-04-12T21:31:46.046 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:46.047+0000 7f43f0c00700 1 -- 172.21.15.18:0/2242151551 <== mgr.34102 v2:172.21.15.18:6800/3812891919 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7f43f4110520 con 0x7f43d806ec10 2024-04-12T21:31:46.046 INFO:teuthology.orchestra.run.smithi018.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-12T21:31:46.046 INFO:teuthology.orchestra.run.smithi018.stdout:alertmanager.smithi018 smithi018 *:9093,9094 running (32m) 8m ago 46m 15.5M - 0.25.0 c8568f914cd2 c332090068e8 2024-04-12T21:31:46.046 INFO:teuthology.orchestra.run.smithi018.stdout:crash.smithi018 smithi018 running (28m) 8m ago 47m 6665k - 19.0.0-2879-g324baf8d 8717720dc5e5 b7f80d1f7982 2024-04-12T21:31:46.047 INFO:teuthology.orchestra.run.smithi018.stdout:crash.smithi186 smithi186 running (28m) 8m ago 45m 6660k - 19.0.0-2879-g324baf8d 8717720dc5e5 a4f55c81b623 2024-04-12T21:31:46.047 INFO:teuthology.orchestra.run.smithi018.stdout:grafana.smithi018 smithi018 *:3000 running (31m) 8m ago 46m 74.8M - 9.4.12 2bacad6d85d8 b08e3bf176dd 2024-04-12T21:31:46.047 INFO:teuthology.orchestra.run.smithi018.stdout:mds.cephfs.smithi018.acqbsc smithi018 running (24m) 8m ago 39m 16.7M - 19.0.0-2879-g324baf8d 8717720dc5e5 021f99ab24ae 2024-04-12T21:31:46.047 INFO:teuthology.orchestra.run.smithi018.stdout:mds.cephfs.smithi018.szjluo smithi018 running (24m) 8m ago 39m 16.5M - 19.0.0-2879-g324baf8d 8717720dc5e5 0003117479d4 2024-04-12T21:31:46.047 INFO:teuthology.orchestra.run.smithi018.stdout:mds.cephfs.smithi186.jdllyn smithi186 running (8m) 8m ago 39m 11.0M - 19.0.0-2879-g324baf8d 8717720dc5e5 21b1f46eef23 2024-04-12T21:31:46.047 INFO:teuthology.orchestra.run.smithi018.stdout:mds.cephfs.smithi186.wjnafs smithi186 running (39m) 8m ago 39m 19.5M - 17.2.7-827-gd3b5060a 21de9b7369ad 74b115d6d6fd 2024-04-12T21:31:46.047 INFO:teuthology.orchestra.run.smithi018.stdout:mgr.smithi018.xjzwga smithi018 *:8443,9283,8765 running (35m) 8m ago 48m 512M - 19.0.0-2879-g324baf8d 8717720dc5e5 56088305eab7 2024-04-12T21:31:46.047 INFO:teuthology.orchestra.run.smithi018.stdout:mgr.smithi186.kqcfxk smithi186 *:8443,9283,8765 running (33m) 8m ago 45m 459M - 19.0.0-2879-g324baf8d 8717720dc5e5 09cd766e5e0d 2024-04-12T21:31:46.047 INFO:teuthology.orchestra.run.smithi018.stdout:mon.smithi018 smithi018 running (30m) 8m ago 48m 73.1M 2048M 19.0.0-2879-g324baf8d 8717720dc5e5 df3e201dfed6 2024-04-12T21:31:46.047 INFO:teuthology.orchestra.run.smithi018.stdout:mon.smithi186 smithi186 running (29m) 8m ago 45m 47.2M 2048M 19.0.0-2879-g324baf8d 8717720dc5e5 c77498da9b2b 2024-04-12T21:31:46.047 INFO:teuthology.orchestra.run.smithi018.stdout:node-exporter.smithi018 smithi018 *:9100 running (33m) 8m ago 47m 8108k - 1.5.0 0da6a335fe13 326c3d9c6847 2024-04-12T21:31:46.047 INFO:teuthology.orchestra.run.smithi018.stdout:node-exporter.smithi186 smithi186 *:9100 running (33m) 8m ago 45m 8757k - 1.5.0 0da6a335fe13 ec2112baeecf 2024-04-12T21:31:46.047 INFO:teuthology.orchestra.run.smithi018.stdout:osd.0 smithi018 running (28m) 8m ago 44m 123M 1559M 19.0.0-2879-g324baf8d 8717720dc5e5 60b1273ca047 2024-04-12T21:31:46.047 INFO:teuthology.orchestra.run.smithi018.stdout:osd.1 smithi018 running (27m) 8m ago 43m 95.1M 1559M 19.0.0-2879-g324baf8d 8717720dc5e5 170ce1e5fb8d 2024-04-12T21:31:46.047 INFO:teuthology.orchestra.run.smithi018.stdout:osd.2 smithi018 running (26m) 8m ago 43m 94.3M 1559M 19.0.0-2879-g324baf8d 8717720dc5e5 c2cbc6aa2839 2024-04-12T21:31:46.047 INFO:teuthology.orchestra.run.smithi018.stdout:osd.3 smithi186 running (26m) 8m ago 42m 101M 2581M 19.0.0-2879-g324baf8d 8717720dc5e5 0138e8262e4b 2024-04-12T21:31:46.048 INFO:teuthology.orchestra.run.smithi018.stdout:osd.4 smithi186 running (25m) 8m ago 42m 91.0M 2581M 19.0.0-2879-g324baf8d 8717720dc5e5 b20cb63b25c9 2024-04-12T21:31:46.048 INFO:teuthology.orchestra.run.smithi018.stdout:osd.5 smithi186 running (25m) 8m ago 41m 92.0M 2581M 19.0.0-2879-g324baf8d 8717720dc5e5 2513bea2610e 2024-04-12T21:31:46.048 INFO:teuthology.orchestra.run.smithi018.stdout:prometheus.smithi018 smithi018 *:9095 running (32m) 8m ago 46m 67.9M - 2.43.0 a07b618ecd1d f58841127e21 2024-04-12T21:31:46.049 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:46.049+0000 7f43fac00700 1 -- 172.21.15.18:0/2242151551 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f43d806ec10 msgr2=0x7f43d80710c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:31:46.049 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:46.049+0000 7f43fac00700 1 --2- 172.21.15.18:0/2242151551 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f43d806ec10 0x7f43d80710c0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f43e400b9c0 tx=0x7f43e4009e30 comp rx=0 tx=0).stop 2024-04-12T21:31:46.049 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:46.049+0000 7f43fac00700 1 -- 172.21.15.18:0/2242151551 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f43f4071f90 msgr2=0x7f43f411a6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:31:46.049 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:46.049+0000 7f43fac00700 1 --2- 172.21.15.18:0/2242151551 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f43f4071f90 0x7f43f411a6e0 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7f43dc00c8c0 tx=0x7f43dc0044c0 comp rx=0 tx=0).stop 2024-04-12T21:31:46.049 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:46.050+0000 7f43fac00700 1 -- 172.21.15.18:0/2242151551 shutdown_connections 2024-04-12T21:31:46.049 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:46.050+0000 7f43fac00700 1 --2- 172.21.15.18:0/2242151551 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f43d806ec10 0x7f43d80710c0 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:31:46.049 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:46.050+0000 7f43fac00700 1 --2- 172.21.15.18:0/2242151551 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f43f40708a0 0x7f43f411a1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:31:46.050 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:46.050+0000 7f43fac00700 1 --2- 172.21.15.18:0/2242151551 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f43f4071f90 0x7f43f411a6e0 unknown :-1 s=CLOSED pgs=811 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:31:46.050 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:46.050+0000 7f43fac00700 1 -- 172.21.15.18:0/2242151551 >> 172.21.15.18:0/2242151551 conn(0x7f43f40f89b0 msgr2=0x7f43f40f9aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:31:46.050 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:46.050+0000 7f43fac00700 1 -- 172.21.15.18:0/2242151551 shutdown_connections 2024-04-12T21:31:46.050 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:46.050+0000 7f43fac00700 1 -- 172.21.15.18:0/2242151551 wait complete. 2024-04-12T21:31:46.205 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:46.205+0000 7fa5cd400700 1 -- 172.21.15.18:0/3552857526 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa5c80708a0 msgr2=0x7fa5c8070d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:31:46.205 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:46.205+0000 7fa5cd400700 1 --2- 172.21.15.18:0/3552857526 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa5c80708a0 0x7fa5c8070d10 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7fa5b4004970 tx=0x7fa5b4009e30 comp rx=0 tx=0).stop 2024-04-12T21:31:46.205 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:46.206+0000 7fa5cd400700 1 -- 172.21.15.18:0/3552857526 shutdown_connections 2024-04-12T21:31:46.205 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:46.206+0000 7fa5cd400700 1 --2- 172.21.15.18:0/3552857526 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa5c80708a0 0x7fa5c8070d10 unknown :-1 s=CLOSED pgs=812 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:31:46.205 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:46.206+0000 7fa5cd400700 1 --2- 172.21.15.18:0/3552857526 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa5c8071f90 0x7fa5c8070360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:31:46.205 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:46.206+0000 7fa5cd400700 1 -- 172.21.15.18:0/3552857526 >> 172.21.15.18:0/3552857526 conn(0x7fa5c80f8970 msgr2=0x7fa5c80fad80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:31:46.205 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:46.206+0000 7fa5cd400700 1 -- 172.21.15.18:0/3552857526 shutdown_connections 2024-04-12T21:31:46.205 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:46.206+0000 7fa5cd400700 1 -- 172.21.15.18:0/3552857526 wait complete. 2024-04-12T21:31:46.206 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:46.207+0000 7fa5cd400700 1 Processor -- start 2024-04-12T21:31:46.207 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:46.207+0000 7fa5cd400700 1 -- start start 2024-04-12T21:31:46.207 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:46.207+0000 7fa5cd400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa5c80708a0 0x7fa5c811a160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:31:46.207 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:46.207+0000 7fa5cd400700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa5c8071f90 0x7fa5c811a6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:31:46.207 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:46.207+0000 7fa5cd400700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5c811acf0 con 0x7fa5c80708a0 2024-04-12T21:31:46.207 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:46.207+0000 7fa5cd400700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5c81140c0 con 0x7fa5c8071f90 2024-04-12T21:31:46.207 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:46.207+0000 7fa5c6800700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa5c80708a0 0x7fa5c811a160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:31:46.207 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:46.207+0000 7fa5c6800700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa5c80708a0 0x7fa5c811a160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:40170/0 (socket says 172.21.15.18:40170) 2024-04-12T21:31:46.207 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:46.208+0000 7fa5c6800700 1 -- 172.21.15.18:0/1339723468 learned_addr learned my addr 172.21.15.18:0/1339723468 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:31:46.208 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:46.208+0000 7fa5c5e00700 1 --2- 172.21.15.18:0/1339723468 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa5c8071f90 0x7fa5c811a6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:31:46.208 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:46.208+0000 7fa5c6800700 1 -- 172.21.15.18:0/1339723468 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa5c8071f90 msgr2=0x7fa5c811a6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:31:46.208 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:46.208+0000 7fa5c6800700 1 --2- 172.21.15.18:0/1339723468 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa5c8071f90 0x7fa5c811a6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:31:46.208 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:46.208+0000 7fa5c6800700 1 -- 172.21.15.18:0/1339723468 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5b4005040 con 0x7fa5c80708a0 2024-04-12T21:31:46.208 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:46.208+0000 7fa5c6800700 1 --2- 172.21.15.18:0/1339723468 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa5c80708a0 0x7fa5c811a160 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7fa5b800c830 tx=0x7fa5b8004470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:31:46.208 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:46.209+0000 7fa5bea00700 1 -- 172.21.15.18:0/1339723468 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa5b800a760 con 0x7fa5c80708a0 2024-04-12T21:31:46.209 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:46.209+0000 7fa5cd400700 1 -- 172.21.15.18:0/1339723468 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa5c81143a0 con 0x7fa5c80708a0 2024-04-12T21:31:46.209 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:46.209+0000 7fa5bea00700 1 -- 172.21.15.18:0/1339723468 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa5b800ad90 con 0x7fa5c80708a0 2024-04-12T21:31:46.209 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:46.209+0000 7fa5cd400700 1 -- 172.21.15.18:0/1339723468 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa5c81148e0 con 0x7fa5c80708a0 2024-04-12T21:31:46.209 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:46.209+0000 7fa5bea00700 1 -- 172.21.15.18:0/1339723468 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa5b801b400 con 0x7fa5c80708a0 2024-04-12T21:31:46.209 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:46.210+0000 7fa5cd400700 1 -- 172.21.15.18:0/1339723468 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa5c8071aa0 con 0x7fa5c80708a0 2024-04-12T21:31:46.210 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:46.211+0000 7fa5bea00700 1 -- 172.21.15.18:0/1339723468 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7fa5b8012400 con 0x7fa5c80708a0 2024-04-12T21:31:46.211 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:46.212+0000 7fa5bea00700 1 --2- 172.21.15.18:0/1339723468 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fa5ac06ec00 0x7fa5ac0710b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:31:46.211 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:46.212+0000 7fa5bea00700 1 -- 172.21.15.18:0/1339723468 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 6757+0+0 (secure 0 0 0) 0x7fa5b808eac0 con 0x7fa5c80708a0 2024-04-12T21:31:46.211 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:46.212+0000 7fa5c5e00700 1 --2- 172.21.15.18:0/1339723468 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fa5ac06ec00 0x7fa5ac0710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:31:46.212 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:46.213+0000 7fa5c5e00700 1 --2- 172.21.15.18:0/1339723468 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fa5ac06ec00 0x7fa5ac0710b0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7fa5b400b370 tx=0x7fa5b4009e30 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:31:46.216 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:46.217+0000 7fa5bea00700 1 -- 172.21.15.18:0/1339723468 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7fa5b805b270 con 0x7fa5c80708a0 2024-04-12T21:31:46.559 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:46.559+0000 7fa5cd400700 1 -- 172.21.15.18:0/1339723468 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fa5c804bed0 con 0x7fa5c80708a0 2024-04-12T21:31:46.559 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:46.560+0000 7fa5bea00700 1 -- 172.21.15.18:0/1339723468 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+837 (secure 0 0 0) 0x7fa5b805ae00 con 0x7fa5c80708a0 2024-04-12T21:31:46.560 INFO:teuthology.orchestra.run.smithi018.stdout:{ 2024-04-12T21:31:46.560 INFO:teuthology.orchestra.run.smithi018.stdout: "mon": { 2024-04-12T21:31:46.560 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 19.0.0-2879-g324baf8d (324baf8ddb91dea46cac50cb535c306115791355) squid (dev)": 2 2024-04-12T21:31:46.560 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-12T21:31:46.560 INFO:teuthology.orchestra.run.smithi018.stdout: "mgr": { 2024-04-12T21:31:46.560 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 19.0.0-2879-g324baf8d (324baf8ddb91dea46cac50cb535c306115791355) squid (dev)": 2 2024-04-12T21:31:46.560 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-12T21:31:46.560 INFO:teuthology.orchestra.run.smithi018.stdout: "osd": { 2024-04-12T21:31:46.560 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 19.0.0-2879-g324baf8d (324baf8ddb91dea46cac50cb535c306115791355) squid (dev)": 6 2024-04-12T21:31:46.560 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-12T21:31:46.560 INFO:teuthology.orchestra.run.smithi018.stdout: "mds": { 2024-04-12T21:31:46.561 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 17.2.7-827-gd3b5060a (d3b5060a17449edfae9385cdf792afec109fca5d) quincy (stable)": 1, 2024-04-12T21:31:46.561 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 19.0.0-2879-g324baf8d (324baf8ddb91dea46cac50cb535c306115791355) squid (dev)": 3 2024-04-12T21:31:46.561 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-12T21:31:46.561 INFO:teuthology.orchestra.run.smithi018.stdout: "overall": { 2024-04-12T21:31:46.561 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 17.2.7-827-gd3b5060a (d3b5060a17449edfae9385cdf792afec109fca5d) quincy (stable)": 1, 2024-04-12T21:31:46.561 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 19.0.0-2879-g324baf8d (324baf8ddb91dea46cac50cb535c306115791355) squid (dev)": 13 2024-04-12T21:31:46.561 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-04-12T21:31:46.561 INFO:teuthology.orchestra.run.smithi018.stdout:} 2024-04-12T21:31:46.562 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:46.563+0000 7fa5cd400700 1 -- 172.21.15.18:0/1339723468 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fa5ac06ec00 msgr2=0x7fa5ac0710b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:31:46.562 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:46.563+0000 7fa5cd400700 1 --2- 172.21.15.18:0/1339723468 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fa5ac06ec00 0x7fa5ac0710b0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7fa5b400b370 tx=0x7fa5b4009e30 comp rx=0 tx=0).stop 2024-04-12T21:31:46.562 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:46.563+0000 7fa5cd400700 1 -- 172.21.15.18:0/1339723468 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa5c80708a0 msgr2=0x7fa5c811a160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:31:46.563 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:46.563+0000 7fa5cd400700 1 --2- 172.21.15.18:0/1339723468 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa5c80708a0 0x7fa5c811a160 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7fa5b800c830 tx=0x7fa5b8004470 comp rx=0 tx=0).stop 2024-04-12T21:31:46.563 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:46.563+0000 7fa5cd400700 1 -- 172.21.15.18:0/1339723468 shutdown_connections 2024-04-12T21:31:46.563 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:46.563+0000 7fa5cd400700 1 --2- 172.21.15.18:0/1339723468 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fa5ac06ec00 0x7fa5ac0710b0 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:31:46.563 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:46.563+0000 7fa5cd400700 1 --2- 172.21.15.18:0/1339723468 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa5c80708a0 0x7fa5c811a160 unknown :-1 s=CLOSED pgs=813 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:31:46.563 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:46.563+0000 7fa5cd400700 1 --2- 172.21.15.18:0/1339723468 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa5c8071f90 0x7fa5c811a6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:31:46.564 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:46.563+0000 7fa5cd400700 1 -- 172.21.15.18:0/1339723468 >> 172.21.15.18:0/1339723468 conn(0x7fa5c80f8970 msgr2=0x7fa5c80f99f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:31:46.564 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:46.564+0000 7fa5cd400700 1 -- 172.21.15.18:0/1339723468 shutdown_connections 2024-04-12T21:31:46.564 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:46.564+0000 7fa5cd400700 1 -- 172.21.15.18:0/1339723468 wait complete. 2024-04-12T21:31:46.719 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:46.718+0000 7f62cc800700 1 -- 172.21.15.18:0/1564762837 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f62c4109dd0 msgr2=0x7f62c4103440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:31:46.719 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:46.718+0000 7f62cc800700 1 --2- 172.21.15.18:0/1564762837 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f62c4109dd0 0x7f62c4103440 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7f62b4004940 tx=0x7f62b4009e30 comp rx=0 tx=0).stop 2024-04-12T21:31:46.719 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:46.718+0000 7f62cc800700 1 -- 172.21.15.18:0/1564762837 shutdown_connections 2024-04-12T21:31:46.719 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:46.718+0000 7f62cc800700 1 --2- 172.21.15.18:0/1564762837 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f62c4109dd0 0x7f62c4103440 unknown :-1 s=CLOSED pgs=814 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:31:46.719 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:46.718+0000 7f62cc800700 1 --2- 172.21.15.18:0/1564762837 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f62c4109430 0x7f62c4109800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:31:46.719 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:46.718+0000 7f62cc800700 1 -- 172.21.15.18:0/1564762837 >> 172.21.15.18:0/1564762837 conn(0x7f62c4072c00 msgr2=0x7f62c4075010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:31:46.719 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:46.718+0000 7f62cc800700 1 -- 172.21.15.18:0/1564762837 shutdown_connections 2024-04-12T21:31:46.719 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:46.718+0000 7f62cc800700 1 -- 172.21.15.18:0/1564762837 wait complete. 2024-04-12T21:31:46.719 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:46.719+0000 7f62cc800700 1 Processor -- start 2024-04-12T21:31:46.719 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:46.719+0000 7f62cc800700 1 -- start start 2024-04-12T21:31:46.720 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:46.720+0000 7f62cc800700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f62c4109430 0x7f62c4119f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:31:46.720 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:46.720+0000 7f62cc800700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f62c4109dd0 0x7f62c411a4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:31:46.720 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:46.720+0000 7f62cc800700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62c411ab10 con 0x7f62c4109dd0 2024-04-12T21:31:46.720 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:46.720+0000 7f62cc800700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62c41140f0 con 0x7f62c4109430 2024-04-12T21:31:46.721 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:46.720+0000 7f62c9400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f62c4109dd0 0x7f62c411a4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:31:46.721 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:46.720+0000 7f62c9400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f62c4109dd0 0x7f62c411a4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:40198/0 (socket says 172.21.15.18:40198) 2024-04-12T21:31:46.721 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:46.720+0000 7f62c9400700 1 -- 172.21.15.18:0/2215612033 learned_addr learned my addr 172.21.15.18:0/2215612033 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:31:46.721 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:46.720+0000 7f62c9e00700 1 --2- 172.21.15.18:0/2215612033 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f62c4109430 0x7f62c4119f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:31:46.722 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:46.721+0000 7f62c9400700 1 -- 172.21.15.18:0/2215612033 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f62c4109430 msgr2=0x7f62c4119f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:31:46.722 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:46.721+0000 7f62c9400700 1 --2- 172.21.15.18:0/2215612033 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f62c4109430 0x7f62c4119f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:31:46.722 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:46.721+0000 7f62c9400700 1 -- 172.21.15.18:0/2215612033 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62b4005040 con 0x7f62c4109dd0 2024-04-12T21:31:46.722 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:46.721+0000 7f62c9e00700 1 --2- 172.21.15.18:0/2215612033 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f62c4109430 0x7f62c4119f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-12T21:31:46.722 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:46.721+0000 7f62c9400700 1 --2- 172.21.15.18:0/2215612033 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f62c4109dd0 0x7f62c411a4c0 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7f62b4009100 tx=0x7f62b4009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:31:46.722 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:46.721+0000 7f62be000700 1 -- 172.21.15.18:0/2215612033 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f62b400e040 con 0x7f62c4109dd0 2024-04-12T21:31:46.722 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:46.721+0000 7f62be000700 1 -- 172.21.15.18:0/2215612033 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f62b4004290 con 0x7f62c4109dd0 2024-04-12T21:31:46.722 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:46.721+0000 7f62cc800700 1 -- 172.21.15.18:0/2215612033 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f62c4114370 con 0x7f62c4109dd0 2024-04-12T21:31:46.723 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:46.721+0000 7f62be000700 1 -- 172.21.15.18:0/2215612033 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f62b401c3f0 con 0x7f62c4109dd0 2024-04-12T21:31:46.723 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:46.721+0000 7f62cc800700 1 -- 172.21.15.18:0/2215612033 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f62c4114930 con 0x7f62c4109dd0 2024-04-12T21:31:46.723 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:46.723+0000 7f62cc800700 1 -- 172.21.15.18:0/2215612033 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f62c4104b80 con 0x7f62c4109dd0 2024-04-12T21:31:46.723 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:46.724+0000 7f62be000700 1 -- 172.21.15.18:0/2215612033 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7f62b4013790 con 0x7f62c4109dd0 2024-04-12T21:31:46.724 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:46.725+0000 7f62be000700 1 --2- 172.21.15.18:0/2215612033 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f62ac06ecd0 0x7f62ac071180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:31:46.725 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:46.725+0000 7f62be000700 1 -- 172.21.15.18:0/2215612033 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 6757+0+0 (secure 0 0 0) 0x7f62b408fc30 con 0x7f62c4109dd0 2024-04-12T21:31:46.725 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:46.725+0000 7f62c9e00700 1 --2- 172.21.15.18:0/2215612033 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f62ac06ecd0 0x7f62ac071180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:31:46.725 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:46.725+0000 7f62c9e00700 1 --2- 172.21.15.18:0/2215612033 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f62ac06ecd0 0x7f62ac071180 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f62b8009fd0 tx=0x7f62b800b040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:31:46.728 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:46.729+0000 7f62be000700 1 -- 172.21.15.18:0/2215612033 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f62b405c510 con 0x7f62c4109dd0 2024-04-12T21:31:47.003 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:47.004+0000 7f62cc800700 1 -- 172.21.15.18:0/2215612033 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f62c4110500 con 0x7f62c4109dd0 2024-04-12T21:31:47.004 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:47.004+0000 7f62be000700 1 -- 172.21.15.18:0/2215612033 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 31 v31) v1 ==== 76+0+1873 (secure 0 0 0) 0x7f62b4013420 con 0x7f62c4109dd0 2024-04-12T21:31:47.005 INFO:teuthology.orchestra.run.smithi018.stdout:e31 2024-04-12T21:31:47.005 INFO:teuthology.orchestra.run.smithi018.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-12T21:31:47.005 INFO:teuthology.orchestra.run.smithi018.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-12T21:31:47.005 INFO:teuthology.orchestra.run.smithi018.stdout:legacy client fscid: 1 2024-04-12T21:31:47.005 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-12T21:31:47.005 INFO:teuthology.orchestra.run.smithi018.stdout:Filesystem 'cephfs' (1) 2024-04-12T21:31:47.005 INFO:teuthology.orchestra.run.smithi018.stdout:fs_name cephfs 2024-04-12T21:31:47.005 INFO:teuthology.orchestra.run.smithi018.stdout:epoch 24 2024-04-12T21:31:47.005 INFO:teuthology.orchestra.run.smithi018.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-04-12T21:31:47.005 INFO:teuthology.orchestra.run.smithi018.stdout:created 2024-04-12T20:51:57.362850+0000 2024-04-12T21:31:47.006 INFO:teuthology.orchestra.run.smithi018.stdout:modified 2024-04-12T21:06:59.717190+0000 2024-04-12T21:31:47.006 INFO:teuthology.orchestra.run.smithi018.stdout:tableserver 0 2024-04-12T21:31:47.006 INFO:teuthology.orchestra.run.smithi018.stdout:root 0 2024-04-12T21:31:47.006 INFO:teuthology.orchestra.run.smithi018.stdout:session_timeout 60 2024-04-12T21:31:47.006 INFO:teuthology.orchestra.run.smithi018.stdout:session_autoclose 300 2024-04-12T21:31:47.006 INFO:teuthology.orchestra.run.smithi018.stdout:max_file_size 1099511627776 2024-04-12T21:31:47.006 INFO:teuthology.orchestra.run.smithi018.stdout:max_xattr_size 65536 2024-04-12T21:31:47.006 INFO:teuthology.orchestra.run.smithi018.stdout:required_client_features {} 2024-04-12T21:31:47.006 INFO:teuthology.orchestra.run.smithi018.stdout:last_failure 0 2024-04-12T21:31:47.006 INFO:teuthology.orchestra.run.smithi018.stdout:last_failure_osd_epoch 100 2024-04-12T21:31:47.006 INFO:teuthology.orchestra.run.smithi018.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-12T21:31:47.006 INFO:teuthology.orchestra.run.smithi018.stdout:max_mds 2 2024-04-12T21:31:47.006 INFO:teuthology.orchestra.run.smithi018.stdout:in 0,1 2024-04-12T21:31:47.007 INFO:teuthology.orchestra.run.smithi018.stdout:up {} 2024-04-12T21:31:47.007 INFO:teuthology.orchestra.run.smithi018.stdout:failed 0,1 2024-04-12T21:31:47.007 INFO:teuthology.orchestra.run.smithi018.stdout:damaged 2024-04-12T21:31:47.007 INFO:teuthology.orchestra.run.smithi018.stdout:stopped 2024-04-12T21:31:47.007 INFO:teuthology.orchestra.run.smithi018.stdout:data_pools [3] 2024-04-12T21:31:47.007 INFO:teuthology.orchestra.run.smithi018.stdout:metadata_pool 2 2024-04-12T21:31:47.007 INFO:teuthology.orchestra.run.smithi018.stdout:inline_data disabled 2024-04-12T21:31:47.007 INFO:teuthology.orchestra.run.smithi018.stdout:balancer 2024-04-12T21:31:47.007 INFO:teuthology.orchestra.run.smithi018.stdout:bal_rank_mask -1 2024-04-12T21:31:47.007 INFO:teuthology.orchestra.run.smithi018.stdout:standby_count_wanted 1 2024-04-12T21:31:47.007 INFO:teuthology.orchestra.run.smithi018.stdout:qdb_cluster leader: 0 members: 2024-04-12T21:31:47.007 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-12T21:31:47.007 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-12T21:31:47.007 INFO:teuthology.orchestra.run.smithi018.stdout:Standby daemons: 2024-04-12T21:31:47.008 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-12T21:31:47.008 INFO:teuthology.orchestra.run.smithi018.stdout:[mds.cephfs.smithi186.wjnafs{-1:24277} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.186:6826/438347207,v1:172.21.15.186:6827/438347207] compat {c=[1],r=[1],i=[7ff]}] 2024-04-12T21:31:47.008 INFO:teuthology.orchestra.run.smithi018.stdout:[mds.cephfs.smithi018.acqbsc{-1:34436} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.18:6828/755179764,v1:172.21.15.18:6829/755179764] compat {c=[1],r=[1],i=[1fff]}] 2024-04-12T21:31:47.008 INFO:teuthology.orchestra.run.smithi018.stdout:[mds.cephfs.smithi018.szjluo{-1:34468} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.18:6826/2407992653,v1:172.21.15.18:6827/2407992653] compat {c=[1],r=[1],i=[1fff]}] 2024-04-12T21:31:47.008 INFO:teuthology.orchestra.run.smithi018.stdout:[mds.cephfs.smithi186.jdllyn{-1:44595} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.186:6824/3523444202,v1:172.21.15.186:6825/3523444202] compat {c=[1],r=[1],i=[1fff]}] 2024-04-12T21:31:47.009 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:47.007+0000 7f62cc800700 1 -- 172.21.15.18:0/2215612033 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f62ac06ecd0 msgr2=0x7f62ac071180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:31:47.009 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:47.007+0000 7f62cc800700 1 --2- 172.21.15.18:0/2215612033 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f62ac06ecd0 0x7f62ac071180 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f62b8009fd0 tx=0x7f62b800b040 comp rx=0 tx=0).stop 2024-04-12T21:31:47.009 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:47.007+0000 7f62cc800700 1 -- 172.21.15.18:0/2215612033 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f62c4109dd0 msgr2=0x7f62c411a4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:31:47.009 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:47.007+0000 7f62cc800700 1 --2- 172.21.15.18:0/2215612033 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f62c4109dd0 0x7f62c411a4c0 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7f62b4009100 tx=0x7f62b4009e30 comp rx=0 tx=0).stop 2024-04-12T21:31:47.009 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:47.007+0000 7f62cc800700 1 -- 172.21.15.18:0/2215612033 shutdown_connections 2024-04-12T21:31:47.009 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:47.007+0000 7f62cc800700 1 --2- 172.21.15.18:0/2215612033 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f62ac06ecd0 0x7f62ac071180 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:31:47.009 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:47.007+0000 7f62cc800700 1 --2- 172.21.15.18:0/2215612033 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f62c4109430 0x7f62c4119f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:31:47.009 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:47.007+0000 7f62cc800700 1 --2- 172.21.15.18:0/2215612033 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f62c4109dd0 0x7f62c411a4c0 unknown :-1 s=CLOSED pgs=815 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:31:47.009 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:47.007+0000 7f62cc800700 1 -- 172.21.15.18:0/2215612033 >> 172.21.15.18:0/2215612033 conn(0x7f62c4072c00 msgr2=0x7f62c4074c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:31:47.009 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:47.008+0000 7f62cc800700 1 -- 172.21.15.18:0/2215612033 shutdown_connections 2024-04-12T21:31:47.009 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:47.008+0000 7f62cc800700 1 -- 172.21.15.18:0/2215612033 wait complete. 2024-04-12T21:31:47.010 INFO:teuthology.orchestra.run.smithi018.stderr:dumped fsmap epoch 31 2024-04-12T21:31:47.103 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:31:46 smithi018 ceph-mon[112590]: from='client.35680 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:31:47.103 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:31:46 smithi018 ceph-mon[112590]: from='client.35684 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:31:47.103 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:31:46 smithi018 ceph-mon[112590]: pgmap v960: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:31:47.103 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:31:46 smithi018 ceph-mon[112590]: from='client.? 172.21.15.18:0/1339723468' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T21:31:47.163 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:47.163+0000 7fe26d200700 1 -- 172.21.15.18:0/2350059945 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe268105a30 msgr2=0x7fe2680ff0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:31:47.163 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:47.163+0000 7fe26d200700 1 --2- 172.21.15.18:0/2350059945 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe268105a30 0x7fe2680ff0a0 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7fe250004940 tx=0x7fe250009e30 comp rx=0 tx=0).stop 2024-04-12T21:31:47.163 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:47.164+0000 7fe26d200700 1 -- 172.21.15.18:0/2350059945 shutdown_connections 2024-04-12T21:31:47.163 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:47.164+0000 7fe26d200700 1 --2- 172.21.15.18:0/2350059945 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe268105a30 0x7fe2680ff0a0 unknown :-1 s=CLOSED pgs=816 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:31:47.163 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:47.164+0000 7fe26d200700 1 --2- 172.21.15.18:0/2350059945 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe268105090 0x7fe268105460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:31:47.164 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:47.164+0000 7fe26d200700 1 -- 172.21.15.18:0/2350059945 >> 172.21.15.18:0/2350059945 conn(0x7fe2680fac00 msgr2=0x7fe2680fd010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:31:47.164 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:47.164+0000 7fe26d200700 1 -- 172.21.15.18:0/2350059945 shutdown_connections 2024-04-12T21:31:47.164 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:47.164+0000 7fe26d200700 1 -- 172.21.15.18:0/2350059945 wait complete. 2024-04-12T21:31:47.165 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:47.165+0000 7fe26d200700 1 Processor -- start 2024-04-12T21:31:47.165 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:47.165+0000 7fe26d200700 1 -- start start 2024-04-12T21:31:47.165 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:47.165+0000 7fe26d200700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe268105090 0x7fe268115b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:31:47.165 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:47.165+0000 7fe26d200700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe268105a30 0x7fe268116090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:31:47.165 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:47.165+0000 7fe26d200700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2681166e0 con 0x7fe268105a30 2024-04-12T21:31:47.165 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:47.165+0000 7fe26d200700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe26810fcc0 con 0x7fe268105090 2024-04-12T21:31:47.165 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:47.165+0000 7fe25d400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe268105a30 0x7fe268116090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:31:47.166 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:47.165+0000 7fe25d400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe268105a30 0x7fe268116090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:40204/0 (socket says 172.21.15.18:40204) 2024-04-12T21:31:47.166 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:47.165+0000 7fe25d400700 1 -- 172.21.15.18:0/3296251573 learned_addr learned my addr 172.21.15.18:0/3296251573 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:31:47.166 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:47.166+0000 7fe25d400700 1 -- 172.21.15.18:0/3296251573 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe268105090 msgr2=0x7fe268115b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:31:47.166 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:47.166+0000 7fe25d400700 1 --2- 172.21.15.18:0/3296251573 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe268105090 0x7fe268115b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:31:47.166 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:47.166+0000 7fe25d400700 1 -- 172.21.15.18:0/3296251573 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe250005040 con 0x7fe268105a30 2024-04-12T21:31:47.166 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:47.166+0000 7fe25d400700 1 --2- 172.21.15.18:0/3296251573 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe268105a30 0x7fe268116090 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7fe250009100 tx=0x7fe250009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:31:47.167 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:47.166+0000 7fe25ea00700 1 -- 172.21.15.18:0/3296251573 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe25000e040 con 0x7fe268105a30 2024-04-12T21:31:47.167 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:47.166+0000 7fe25ea00700 1 -- 172.21.15.18:0/3296251573 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe250004290 con 0x7fe268105a30 2024-04-12T21:31:47.167 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:47.166+0000 7fe25ea00700 1 -- 172.21.15.18:0/3296251573 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe25001c3f0 con 0x7fe268105a30 2024-04-12T21:31:47.167 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:47.166+0000 7fe26d200700 1 -- 172.21.15.18:0/3296251573 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe26810ff40 con 0x7fe268105a30 2024-04-12T21:31:47.167 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:47.167+0000 7fe26d200700 1 -- 172.21.15.18:0/3296251573 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe268110500 con 0x7fe268105a30 2024-04-12T21:31:47.167 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:47.168+0000 7fe26d200700 1 -- 172.21.15.18:0/3296251573 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe268107560 con 0x7fe268105a30 2024-04-12T21:31:47.168 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:47.169+0000 7fe25ea00700 1 -- 172.21.15.18:0/3296251573 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7fe250013790 con 0x7fe268105a30 2024-04-12T21:31:47.169 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:47.169+0000 7fe25ea00700 1 --2- 172.21.15.18:0/3296251573 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fe24c06ec00 0x7fe24c0710b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:31:47.169 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:47.170+0000 7fe25ea00700 1 -- 172.21.15.18:0/3296251573 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 6757+0+0 (secure 0 0 0) 0x7fe25008fc60 con 0x7fe268105a30 2024-04-12T21:31:47.169 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:47.170+0000 7fe265e00700 1 --2- 172.21.15.18:0/3296251573 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fe24c06ec00 0x7fe24c0710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:31:47.169 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:47.170+0000 7fe265e00700 1 --2- 172.21.15.18:0/3296251573 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fe24c06ec00 0x7fe24c0710b0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7fe258004d60 tx=0x7fe258007b30 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:31:47.173 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:47.174+0000 7fe25ea00700 1 -- 172.21.15.18:0/3296251573 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7fe25005c490 con 0x7fe268105a30 2024-04-12T21:31:47.281 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:31:46 smithi186 ceph-mon[86379]: from='client.35680 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:31:47.282 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:31:46 smithi186 ceph-mon[86379]: from='client.35684 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:31:47.282 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:31:46 smithi186 ceph-mon[86379]: pgmap v960: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:31:47.282 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:31:46 smithi186 ceph-mon[86379]: from='client.? 172.21.15.18:0/1339723468' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T21:31:47.413 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:47.414+0000 7fe26d200700 1 -- 172.21.15.18:0/3296251573 --> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fe2681111a0 con 0x7fe24c06ec00 2024-04-12T21:31:47.417 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:47.418+0000 7fe25ea00700 1 -- 172.21.15.18:0/3296251573 <== mgr.34102 v2:172.21.15.18:6800/3812891919 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7fe2681111a0 con 0x7fe24c06ec00 2024-04-12T21:31:47.417 INFO:teuthology.orchestra.run.smithi018.stdout:{ 2024-04-12T21:31:47.417 INFO:teuthology.orchestra.run.smithi018.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:f4452def3be4d350f4a8e5feaf7ebb1511be9298fe68478ba7a2d6fbd7d69a63", 2024-04-12T21:31:47.418 INFO:teuthology.orchestra.run.smithi018.stdout: "in_progress": true, 2024-04-12T21:31:47.418 INFO:teuthology.orchestra.run.smithi018.stdout: "which": "Upgrading all daemon types on all hosts", 2024-04-12T21:31:47.418 INFO:teuthology.orchestra.run.smithi018.stdout: "services_complete": [ 2024-04-12T21:31:47.418 INFO:teuthology.orchestra.run.smithi018.stdout: "crash", 2024-04-12T21:31:47.418 INFO:teuthology.orchestra.run.smithi018.stdout: "osd", 2024-04-12T21:31:47.418 INFO:teuthology.orchestra.run.smithi018.stdout: "mon", 2024-04-12T21:31:47.418 INFO:teuthology.orchestra.run.smithi018.stdout: "mgr" 2024-04-12T21:31:47.418 INFO:teuthology.orchestra.run.smithi018.stdout: ], 2024-04-12T21:31:47.418 INFO:teuthology.orchestra.run.smithi018.stdout: "progress": "15/21 daemons upgraded", 2024-04-12T21:31:47.418 INFO:teuthology.orchestra.run.smithi018.stdout: "message": "Currently upgrading mds daemons", 2024-04-12T21:31:47.418 INFO:teuthology.orchestra.run.smithi018.stdout: "is_paused": false 2024-04-12T21:31:47.418 INFO:teuthology.orchestra.run.smithi018.stdout:} 2024-04-12T21:31:47.420 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:47.420+0000 7fe26d200700 1 -- 172.21.15.18:0/3296251573 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fe24c06ec00 msgr2=0x7fe24c0710b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:31:47.420 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:47.420+0000 7fe26d200700 1 --2- 172.21.15.18:0/3296251573 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fe24c06ec00 0x7fe24c0710b0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7fe258004d60 tx=0x7fe258007b30 comp rx=0 tx=0).stop 2024-04-12T21:31:47.420 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:47.420+0000 7fe26d200700 1 -- 172.21.15.18:0/3296251573 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe268105a30 msgr2=0x7fe268116090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:31:47.420 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:47.420+0000 7fe26d200700 1 --2- 172.21.15.18:0/3296251573 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe268105a30 0x7fe268116090 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7fe250009100 tx=0x7fe250009e30 comp rx=0 tx=0).stop 2024-04-12T21:31:47.420 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:47.421+0000 7fe26d200700 1 -- 172.21.15.18:0/3296251573 shutdown_connections 2024-04-12T21:31:47.420 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:47.421+0000 7fe26d200700 1 --2- 172.21.15.18:0/3296251573 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fe24c06ec00 0x7fe24c0710b0 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:31:47.420 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:47.421+0000 7fe26d200700 1 --2- 172.21.15.18:0/3296251573 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe268105090 0x7fe268115b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:31:47.420 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:47.421+0000 7fe26d200700 1 --2- 172.21.15.18:0/3296251573 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe268105a30 0x7fe268116090 unknown :-1 s=CLOSED pgs=817 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:31:47.421 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:47.421+0000 7fe26d200700 1 -- 172.21.15.18:0/3296251573 >> 172.21.15.18:0/3296251573 conn(0x7fe2680fac00 msgr2=0x7fe2680fc9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:31:47.421 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:47.421+0000 7fe26d200700 1 -- 172.21.15.18:0/3296251573 shutdown_connections 2024-04-12T21:31:47.421 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:47.421+0000 7fe26d200700 1 -- 172.21.15.18:0/3296251573 wait complete. 2024-04-12T21:31:47.575 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:47.575+0000 7f87acc00700 1 -- 172.21.15.18:0/1476749406 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f87a8109c40 msgr2=0x7f87a81032b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:31:47.575 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:47.575+0000 7f87acc00700 1 --2- 172.21.15.18:0/1476749406 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f87a8109c40 0x7f87a81032b0 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7f8794004940 tx=0x7f8794009e30 comp rx=0 tx=0).stop 2024-04-12T21:31:47.575 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:47.575+0000 7f87acc00700 1 -- 172.21.15.18:0/1476749406 shutdown_connections 2024-04-12T21:31:47.575 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:47.575+0000 7f87acc00700 1 --2- 172.21.15.18:0/1476749406 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f87a8109c40 0x7f87a81032b0 unknown :-1 s=CLOSED pgs=818 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:31:47.575 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:47.575+0000 7f87acc00700 1 --2- 172.21.15.18:0/1476749406 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f87a81092a0 0x7f87a8109670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:31:47.575 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:47.575+0000 7f87acc00700 1 -- 172.21.15.18:0/1476749406 >> 172.21.15.18:0/1476749406 conn(0x7f87a8072a30 msgr2=0x7f87a8074e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:31:47.575 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:47.575+0000 7f87acc00700 1 -- 172.21.15.18:0/1476749406 shutdown_connections 2024-04-12T21:31:47.575 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:47.575+0000 7f87acc00700 1 -- 172.21.15.18:0/1476749406 wait complete. 2024-04-12T21:31:47.577 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:47.576+0000 7f87acc00700 1 Processor -- start 2024-04-12T21:31:47.577 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:47.576+0000 7f87acc00700 1 -- start start 2024-04-12T21:31:47.577 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:47.577+0000 7f87acc00700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f87a81092a0 0x7f87a8119fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:31:47.577 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:47.577+0000 7f87acc00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f87a8109c40 0x7f87a811a500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:31:47.577 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:47.577+0000 7f87acc00700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87a811ab50 con 0x7f87a8109c40 2024-04-12T21:31:47.577 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:47.577+0000 7f87acc00700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87a8113f20 con 0x7f87a81092a0 2024-04-12T21:31:47.577 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:47.577+0000 7f87a6a00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f87a8109c40 0x7f87a811a500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:31:47.577 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:47.577+0000 7f87a6a00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f87a8109c40 0x7f87a811a500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:40230/0 (socket says 172.21.15.18:40230) 2024-04-12T21:31:47.577 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:47.577+0000 7f87a6a00700 1 -- 172.21.15.18:0/3028586296 learned_addr learned my addr 172.21.15.18:0/3028586296 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:31:47.577 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:47.577+0000 7f87a7400700 1 --2- 172.21.15.18:0/3028586296 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f87a81092a0 0x7f87a8119fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:31:47.577 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:47.578+0000 7f87a6a00700 1 -- 172.21.15.18:0/3028586296 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f87a81092a0 msgr2=0x7f87a8119fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:31:47.577 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:47.578+0000 7f87a6a00700 1 --2- 172.21.15.18:0/3028586296 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f87a81092a0 0x7f87a8119fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:31:47.578 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:47.578+0000 7f87a6a00700 1 -- 172.21.15.18:0/3028586296 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8794005040 con 0x7f87a8109c40 2024-04-12T21:31:47.578 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:47.578+0000 7f87a6a00700 1 --2- 172.21.15.18:0/3028586296 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f87a8109c40 0x7f87a811a500 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7f8794009100 tx=0x7f8794009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:31:47.578 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:47.578+0000 7f879f400700 1 -- 172.21.15.18:0/3028586296 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f879400e040 con 0x7f87a8109c40 2024-04-12T21:31:47.578 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:47.578+0000 7f879f400700 1 -- 172.21.15.18:0/3028586296 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f8794013420 con 0x7f87a8109c40 2024-04-12T21:31:47.579 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:47.578+0000 7f87acc00700 1 -- 172.21.15.18:0/3028586296 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f87a81141a0 con 0x7f87a8109c40 2024-04-12T21:31:47.579 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:47.578+0000 7f879f400700 1 -- 172.21.15.18:0/3028586296 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f879401c630 con 0x7f87a8109c40 2024-04-12T21:31:47.579 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:47.579+0000 7f87acc00700 1 -- 172.21.15.18:0/3028586296 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f87a8114760 con 0x7f87a8109c40 2024-04-12T21:31:47.579 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:47.580+0000 7f87acc00700 1 -- 172.21.15.18:0/3028586296 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f87a81049f0 con 0x7f87a8109c40 2024-04-12T21:31:47.580 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:47.581+0000 7f879f400700 1 -- 172.21.15.18:0/3028586296 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7f8794013590 con 0x7f87a8109c40 2024-04-12T21:31:47.581 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:47.582+0000 7f879f400700 1 --2- 172.21.15.18:0/3028586296 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f878c06ec00 0x7f878c0710b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:31:47.581 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:47.582+0000 7f879f400700 1 -- 172.21.15.18:0/3028586296 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 6757+0+0 (secure 0 0 0) 0x7f879408fbc0 con 0x7f87a8109c40 2024-04-12T21:31:47.581 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:47.582+0000 7f87a7400700 1 --2- 172.21.15.18:0/3028586296 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f878c06ec00 0x7f878c0710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:31:47.582 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:47.583+0000 7f87a7400700 1 --2- 172.21.15.18:0/3028586296 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f878c06ec00 0x7f878c0710b0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f8798004990 tx=0x7f8798009ee0 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:31:47.586 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:47.587+0000 7f879f400700 1 -- 172.21.15.18:0/3028586296 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f879405c410 con 0x7f87a8109c40 2024-04-12T21:31:47.902 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:47.903+0000 7f87acc00700 1 -- 172.21.15.18:0/3028586296 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f87a8076560 con 0x7f87a8109c40 2024-04-12T21:31:47.903 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:47.904+0000 7f879f400700 1 -- 172.21.15.18:0/3028586296 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+4303 (secure 0 0 0) 0x7f879405bfa0 con 0x7f87a8109c40 2024-04-12T21:31:47.903 INFO:teuthology.orchestra.run.smithi018.stdout:HEALTH_ERR failed to probe daemons or devices; 1 filesystem is degraded; 1 filesystem is offline 2024-04-12T21:31:47.903 INFO:teuthology.orchestra.run.smithi018.stdout:[WRN] CEPHADM_REFRESH_FAILED: failed to probe daemons or devices 2024-04-12T21:31:47.903 INFO:teuthology.orchestra.run.smithi018.stdout: host smithi018 `cephadm ceph-volume` failed: cephadm exited with an error code: 1, stderr: Inferring config /var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/mon.smithi018/config 2024-04-12T21:31:47.904 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/podman: timeout after 895 seconds 2024-04-12T21:31:47.904 INFO:teuthology.orchestra.run.smithi018.stdout:Non-zero exit code 124 from /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:f4452def3be4d350f4a8e5feaf7ebb1511be9298fe68478ba7a2d6fbd7d69a63 -e NODE_NAME=smithi018 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968:/var/run/ceph:z -v /var/log/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968:/var/log/ceph:z -v /var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmpxsmwkkuh:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:f4452def3be4d350f4a8e5feaf7ebb1511be9298fe68478ba7a2d6fbd7d69a63 inventory --format=json-pretty --filter-for-batch 2024-04-12T21:31:47.904 INFO:teuthology.orchestra.run.smithi018.stdout:Traceback (most recent call last): 2024-04-12T21:31:47.904 INFO:teuthology.orchestra.run.smithi018.stdout: File "/usr/lib64/python3.9/runpy.py", line 197, in _run_module_as_main 2024-04-12T21:31:47.904 INFO:teuthology.orchestra.run.smithi018.stdout: return _run_code(code, main_globals, None, 2024-04-12T21:31:47.904 INFO:teuthology.orchestra.run.smithi018.stdout: File "/usr/lib64/python3.9/runpy.py", line 87, in _run_code 2024-04-12T21:31:47.904 INFO:teuthology.orchestra.run.smithi018.stdout: exec(code, run_globals) 2024-04-12T21:31:47.904 INFO:teuthology.orchestra.run.smithi018.stdout: File "/var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/cephadm.e44f3ed5e34e4ef52eedef233d1190bff004470d8e8005127b5312bff357fa7f/__main__.py", line 5618, in 2024-04-12T21:31:47.904 INFO:teuthology.orchestra.run.smithi018.stdout: File "/var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/cephadm.e44f3ed5e34e4ef52eedef233d1190bff004470d8e8005127b5312bff357fa7f/__main__.py", line 5606, in main 2024-04-12T21:31:47.904 INFO:teuthology.orchestra.run.smithi018.stdout: File "/var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/cephadm.e44f3ed5e34e4ef52eedef233d1190bff004470d8e8005127b5312bff357fa7f/__main__.py", line 408, in _infer_config 2024-04-12T21:31:47.904 INFO:teuthology.orchestra.run.smithi018.stdout: File "/var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/cephadm.e44f3ed5e34e4ef52eedef233d1190bff004470d8e8005127b5312bff357fa7f/__main__.py", line 323, in _infer_fsid 2024-04-12T21:31:47.904 INFO:teuthology.orchestra.run.smithi018.stdout: File "/var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/cephadm.e44f3ed5e34e4ef52eedef233d1190bff004470d8e8005127b5312bff357fa7f/__main__.py", line 436, in _infer_image 2024-04-12T21:31:47.904 INFO:teuthology.orchestra.run.smithi018.stdout: File "/var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/cephadm.e44f3ed5e34e4ef52eedef233d1190bff004470d8e8005127b5312bff357fa7f/__main__.py", line 310, in _validate_fsid 2024-04-12T21:31:47.904 INFO:teuthology.orchestra.run.smithi018.stdout: File "/var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/cephadm.e44f3ed5e34e4ef52eedef233d1190bff004470d8e8005127b5312bff357fa7f/__main__.py", line 3359, in command_ceph_volume 2024-04-12T21:31:47.905 INFO:teuthology.orchestra.run.smithi018.stdout: File "/var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/cephadm.e44f3ed5e34e4ef52eedef233d1190bff004470d8e8005127b5312bff357fa7f/cephadmlib/call_wrappers.py", line 310, in call_throws 2024-04-12T21:31:47.905 INFO:teuthology.orchestra.run.smithi018.stdout:RuntimeError: Failed command: /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:f4452def3be4d350f4a8e5feaf7ebb1511be9298fe68478ba7a2d6fbd7d69a63 -e NODE_NAME=smithi018 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968:/var/run/ceph:z -v /var/log/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968:/var/log/ceph:z -v /var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmpxsmwkkuh:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:f4452def3be4d350f4a8e5feaf7ebb1511be9298fe68478ba7a2d6fbd7d69a63 inventory --format=json-pretty --filter-for-batch 2024-04-12T21:31:47.905 INFO:teuthology.orchestra.run.smithi018.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-04-12T21:31:47.905 INFO:teuthology.orchestra.run.smithi018.stdout: fs cephfs is degraded 2024-04-12T21:31:47.905 INFO:teuthology.orchestra.run.smithi018.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-04-12T21:31:47.905 INFO:teuthology.orchestra.run.smithi018.stdout: fs cephfs is offline because no MDS is active for it. 2024-04-12T21:31:47.906 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:47.907+0000 7f87acc00700 1 -- 172.21.15.18:0/3028586296 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f878c06ec00 msgr2=0x7f878c0710b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:31:47.906 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:47.907+0000 7f87acc00700 1 --2- 172.21.15.18:0/3028586296 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f878c06ec00 0x7f878c0710b0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f8798004990 tx=0x7f8798009ee0 comp rx=0 tx=0).stop 2024-04-12T21:31:47.906 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:47.907+0000 7f87acc00700 1 -- 172.21.15.18:0/3028586296 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f87a8109c40 msgr2=0x7f87a811a500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:31:47.906 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:47.907+0000 7f87acc00700 1 --2- 172.21.15.18:0/3028586296 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f87a8109c40 0x7f87a811a500 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7f8794009100 tx=0x7f8794009e30 comp rx=0 tx=0).stop 2024-04-12T21:31:47.906 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:47.907+0000 7f87acc00700 1 -- 172.21.15.18:0/3028586296 shutdown_connections 2024-04-12T21:31:47.906 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:47.907+0000 7f87acc00700 1 --2- 172.21.15.18:0/3028586296 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f878c06ec00 0x7f878c0710b0 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:31:47.906 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:47.907+0000 7f87acc00700 1 --2- 172.21.15.18:0/3028586296 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f87a81092a0 0x7f87a8119fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:31:47.907 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:47.907+0000 7f87acc00700 1 --2- 172.21.15.18:0/3028586296 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f87a8109c40 0x7f87a811a500 unknown :-1 s=CLOSED pgs=819 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:31:47.907 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:47.907+0000 7f87acc00700 1 -- 172.21.15.18:0/3028586296 >> 172.21.15.18:0/3028586296 conn(0x7f87a8072a30 msgr2=0x7f87a8074910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:31:47.907 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:47.907+0000 7f87acc00700 1 -- 172.21.15.18:0/3028586296 shutdown_connections 2024-04-12T21:31:47.907 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:31:47.908+0000 7f87acc00700 1 -- 172.21.15.18:0/3028586296 wait complete. 2024-04-12T21:31:48.282 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:31:47 smithi186 ceph-mon[86379]: from='client.35688 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:31:48.282 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:31:47 smithi186 ceph-mon[86379]: from='client.? 172.21.15.18:0/2215612033' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-12T21:31:48.353 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:31:47 smithi018 ceph-mon[112590]: from='client.35688 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:31:48.353 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:31:47 smithi018 ceph-mon[112590]: from='client.? 172.21.15.18:0/2215612033' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-12T21:31:49.282 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:31:48 smithi186 ceph-mon[86379]: from='client.35700 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:31:49.282 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:31:48 smithi186 ceph-mon[86379]: pgmap v961: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:31:49.282 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:31:48 smithi186 ceph-mon[86379]: from='client.? 172.21.15.18:0/3028586296' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-12T21:31:49.352 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:31:48 smithi018 ceph-mon[112590]: from='client.35700 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:31:49.353 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:31:48 smithi018 ceph-mon[112590]: pgmap v961: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:31:49.353 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:31:48 smithi018 ceph-mon[112590]: from='client.? 172.21.15.18:0/3028586296' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-12T21:31:51.281 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:31:50 smithi186 ceph-mon[86379]: pgmap v962: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:31:51.352 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:31:50 smithi018 ceph-mon[112590]: pgmap v962: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:31:53.282 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:31:52 smithi186 ceph-mon[86379]: pgmap v963: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:31:53.352 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:31:52 smithi018 ceph-mon[112590]: pgmap v963: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:31:55.281 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:31:54 smithi186 ceph-mon[86379]: pgmap v964: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:31:55.352 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:31:54 smithi018 ceph-mon[112590]: pgmap v964: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:31:56.281 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:31:55 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T21:31:56.352 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:31:55 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T21:31:57.282 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:31:56 smithi186 ceph-mon[86379]: pgmap v965: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:31:57.353 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:31:56 smithi018 ceph-mon[112590]: pgmap v965: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:31:58.281 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:31:57 smithi186 ceph-mon[86379]: pgmap v966: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:31:58.352 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:31:57 smithi018 ceph-mon[112590]: pgmap v966: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:32:01.031 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:32:00 smithi186 ceph-mon[86379]: pgmap v967: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:32:01.102 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:32:00 smithi018 ceph-mon[112590]: pgmap v967: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:32:03.032 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:32:02 smithi186 ceph-mon[86379]: pgmap v968: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:32:03.102 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:32:02 smithi018 ceph-mon[112590]: pgmap v968: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:32:05.031 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:32:04 smithi186 ceph-mon[86379]: pgmap v969: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:32:05.103 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:32:04 smithi018 ceph-mon[112590]: pgmap v969: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:32:07.031 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:32:06 smithi186 ceph-mon[86379]: pgmap v970: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:32:07.102 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:32:06 smithi018 ceph-mon[112590]: pgmap v970: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:32:09.031 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:32:08 smithi186 ceph-mon[86379]: pgmap v971: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:32:09.102 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:32:08 smithi018 ceph-mon[112590]: pgmap v971: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:32:11.032 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:32:10 smithi186 ceph-mon[86379]: pgmap v972: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:32:11.032 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:32:10 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T21:32:11.102 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:32:10 smithi018 ceph-mon[112590]: pgmap v972: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:32:11.103 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:32:10 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T21:32:13.032 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:32:12 smithi186 ceph-mon[86379]: pgmap v973: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:32:13.102 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:32:12 smithi018 ceph-mon[112590]: pgmap v973: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:32:15.032 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:32:14 smithi186 ceph-mon[86379]: pgmap v974: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:32:15.102 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:32:14 smithi018 ceph-mon[112590]: pgmap v974: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:32:17.032 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:32:16 smithi186 ceph-mon[86379]: pgmap v975: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:32:17.103 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:32:16 smithi018 ceph-mon[112590]: pgmap v975: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:32:18.082 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:18.082+0000 7f314dc00700 1 -- 172.21.15.18:0/3699687845 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f31480fe7d0 msgr2=0x7f3148106e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:32:18.082 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:18.082+0000 7f314dc00700 1 --2- 172.21.15.18:0/3699687845 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f31480fe7d0 0x7f3148106e10 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7f3130004940 tx=0x7f3130009e30 comp rx=0 tx=0).stop 2024-04-12T21:32:18.082 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:18.082+0000 7f314dc00700 1 -- 172.21.15.18:0/3699687845 shutdown_connections 2024-04-12T21:32:18.082 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:18.082+0000 7f314dc00700 1 --2- 172.21.15.18:0/3699687845 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f31480fe7d0 0x7f3148106e10 unknown :-1 s=CLOSED pgs=820 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:32:18.082 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:18.082+0000 7f314dc00700 1 --2- 172.21.15.18:0/3699687845 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f31480fde30 0x7f31480fe200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:32:18.082 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:18.082+0000 7f314dc00700 1 -- 172.21.15.18:0/3699687845 >> 172.21.15.18:0/3699687845 conn(0x7f31480f7860 msgr2=0x7f31480f9c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:32:18.082 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:18.082+0000 7f314dc00700 1 -- 172.21.15.18:0/3699687845 shutdown_connections 2024-04-12T21:32:18.083 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:18.082+0000 7f314dc00700 1 -- 172.21.15.18:0/3699687845 wait complete. 2024-04-12T21:32:18.084 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:18.084+0000 7f314dc00700 1 Processor -- start 2024-04-12T21:32:18.084 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:18.084+0000 7f314dc00700 1 -- start start 2024-04-12T21:32:18.084 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:18.084+0000 7f314dc00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f31480fde30 0x7f3148115b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:32:18.084 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:18.084+0000 7f314dc00700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f31480fe7d0 0x7f31481160a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:32:18.084 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:18.084+0000 7f314dc00700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3148116780 con 0x7f31480fde30 2024-04-12T21:32:18.084 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:18.084+0000 7f314dc00700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f314810fcd0 con 0x7f31480fe7d0 2024-04-12T21:32:18.084 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:18.084+0000 7f3146a00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f31480fde30 0x7f3148115b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:32:18.084 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:18.085+0000 7f3146a00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f31480fde30 0x7f3148115b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:45544/0 (socket says 172.21.15.18:45544) 2024-04-12T21:32:18.085 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:18.085+0000 7f3146a00700 1 -- 172.21.15.18:0/112931860 learned_addr learned my addr 172.21.15.18:0/112931860 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:32:18.085 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:18.085+0000 7f313fe00700 1 --2- 172.21.15.18:0/112931860 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f31480fe7d0 0x7f31481160a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:32:18.086 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:18.085+0000 7f3146a00700 1 -- 172.21.15.18:0/112931860 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f31480fe7d0 msgr2=0x7f31481160a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:32:18.086 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:18.085+0000 7f3146a00700 1 --2- 172.21.15.18:0/112931860 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f31480fe7d0 0x7f31481160a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:32:18.086 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:18.085+0000 7f3146a00700 1 -- 172.21.15.18:0/112931860 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3130005040 con 0x7f31480fde30 2024-04-12T21:32:18.086 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:18.085+0000 7f313fe00700 1 --2- 172.21.15.18:0/112931860 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f31480fe7d0 0x7f31481160a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-12T21:32:18.086 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:18.085+0000 7f3146a00700 1 --2- 172.21.15.18:0/112931860 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f31480fde30 0x7f3148115b60 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7f313800b790 tx=0x7f3138009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:32:18.086 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:18.086+0000 7f313ea00700 1 -- 172.21.15.18:0/112931860 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f313800f770 con 0x7f31480fde30 2024-04-12T21:32:18.086 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:18.086+0000 7f313ea00700 1 -- 172.21.15.18:0/112931860 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f313800fda0 con 0x7f31480fde30 2024-04-12T21:32:18.086 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:18.086+0000 7f313ea00700 1 -- 172.21.15.18:0/112931860 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f313800d310 con 0x7f31480fde30 2024-04-12T21:32:18.086 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:18.086+0000 7f314dc00700 1 -- 172.21.15.18:0/112931860 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f314810ff50 con 0x7f31480fde30 2024-04-12T21:32:18.086 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:18.086+0000 7f314dc00700 1 -- 172.21.15.18:0/112931860 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3148110490 con 0x7f31480fde30 2024-04-12T21:32:18.087 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:18.087+0000 7f314dc00700 1 -- 172.21.15.18:0/112931860 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3148106880 con 0x7f31480fde30 2024-04-12T21:32:18.087 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:18.088+0000 7f313ea00700 1 -- 172.21.15.18:0/112931860 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7f3138010420 con 0x7f31480fde30 2024-04-12T21:32:18.089 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:18.090+0000 7f313ea00700 1 --2- 172.21.15.18:0/112931860 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f312c06ec00 0x7f312c0710b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:32:18.089 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:18.090+0000 7f313ea00700 1 -- 172.21.15.18:0/112931860 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 6757+0+0 (secure 0 0 0) 0x7f313808e9e0 con 0x7f31480fde30 2024-04-12T21:32:18.089 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:18.090+0000 7f313fe00700 1 --2- 172.21.15.18:0/112931860 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f312c06ec00 0x7f312c0710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:32:18.090 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:18.091+0000 7f313fe00700 1 --2- 172.21.15.18:0/112931860 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f312c06ec00 0x7f312c0710b0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f313000b3b0 tx=0x7f3130009e30 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:32:18.094 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:18.095+0000 7f313ea00700 1 -- 172.21.15.18:0/112931860 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f313805b230 con 0x7f31480fde30 2024-04-12T21:32:18.352 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:18.352+0000 7f314dc00700 1 -- 172.21.15.18:0/112931860 --> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f31481111b0 con 0x7f312c06ec00 2024-04-12T21:32:18.354 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:18.354+0000 7f313ea00700 1 -- 172.21.15.18:0/112931860 <== mgr.34102 v2:172.21.15.18:6800/3812891919 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f31481111b0 con 0x7f312c06ec00 2024-04-12T21:32:18.358 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:18.358+0000 7f314dc00700 1 -- 172.21.15.18:0/112931860 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f312c06ec00 msgr2=0x7f312c0710b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:32:18.358 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:18.358+0000 7f314dc00700 1 --2- 172.21.15.18:0/112931860 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f312c06ec00 0x7f312c0710b0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f313000b3b0 tx=0x7f3130009e30 comp rx=0 tx=0).stop 2024-04-12T21:32:18.358 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:18.358+0000 7f314dc00700 1 -- 172.21.15.18:0/112931860 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f31480fde30 msgr2=0x7f3148115b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:32:18.358 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:18.358+0000 7f314dc00700 1 --2- 172.21.15.18:0/112931860 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f31480fde30 0x7f3148115b60 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7f313800b790 tx=0x7f3138009250 comp rx=0 tx=0).stop 2024-04-12T21:32:18.358 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:18.358+0000 7f314dc00700 1 -- 172.21.15.18:0/112931860 shutdown_connections 2024-04-12T21:32:18.358 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:18.358+0000 7f314dc00700 1 --2- 172.21.15.18:0/112931860 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f312c06ec00 0x7f312c0710b0 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:32:18.358 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:18.358+0000 7f314dc00700 1 --2- 172.21.15.18:0/112931860 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f31480fde30 0x7f3148115b60 unknown :-1 s=CLOSED pgs=821 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:32:18.358 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:18.358+0000 7f314dc00700 1 --2- 172.21.15.18:0/112931860 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f31480fe7d0 0x7f31481160a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:32:18.358 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:18.358+0000 7f314dc00700 1 -- 172.21.15.18:0/112931860 >> 172.21.15.18:0/112931860 conn(0x7f31480f7860 msgr2=0x7f31480fd280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:32:18.358 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:18.358+0000 7f314dc00700 1 -- 172.21.15.18:0/112931860 shutdown_connections 2024-04-12T21:32:18.358 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:18.359+0000 7f314dc00700 1 -- 172.21.15.18:0/112931860 wait complete. 2024-04-12T21:32:18.373 INFO:teuthology.orchestra.run.smithi018.stdout:true 2024-04-12T21:32:18.512 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:18.513+0000 7f31a6e00700 1 -- 172.21.15.18:0/1129668291 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f31a01037a0 msgr2=0x7f31a00fce10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:32:18.513 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:18.513+0000 7f31a6e00700 1 --2- 172.21.15.18:0/1129668291 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f31a01037a0 0x7f31a00fce10 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7f3188004940 tx=0x7f3188009e30 comp rx=0 tx=0).stop 2024-04-12T21:32:18.513 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:18.513+0000 7f31a6e00700 1 -- 172.21.15.18:0/1129668291 shutdown_connections 2024-04-12T21:32:18.513 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:18.513+0000 7f31a6e00700 1 --2- 172.21.15.18:0/1129668291 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f31a01037a0 0x7f31a00fce10 unknown :-1 s=CLOSED pgs=822 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:32:18.513 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:18.513+0000 7f31a6e00700 1 --2- 172.21.15.18:0/1129668291 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f31a0102e00 0x7f31a01031d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:32:18.513 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:18.513+0000 7f31a6e00700 1 -- 172.21.15.18:0/1129668291 >> 172.21.15.18:0/1129668291 conn(0x7f31a00f89b0 msgr2=0x7f31a00fadc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:32:18.513 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:18.513+0000 7f31a6e00700 1 -- 172.21.15.18:0/1129668291 shutdown_connections 2024-04-12T21:32:18.513 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:18.513+0000 7f31a6e00700 1 -- 172.21.15.18:0/1129668291 wait complete. 2024-04-12T21:32:18.514 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:18.514+0000 7f31a6e00700 1 Processor -- start 2024-04-12T21:32:18.514 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:18.514+0000 7f31a6e00700 1 -- start start 2024-04-12T21:32:18.514 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:18.515+0000 7f31a6e00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f31a0102e00 0x7f31a0115b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:32:18.514 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:18.515+0000 7f31a6e00700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f31a01037a0 0x7f31a01160b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:32:18.514 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:18.515+0000 7f31a6e00700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31a0116790 con 0x7f31a0102e00 2024-04-12T21:32:18.514 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:18.515+0000 7f31a6e00700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31a010fd30 con 0x7f31a01037a0 2024-04-12T21:32:18.516 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:18.515+0000 7f319fe00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f31a0102e00 0x7f31a0115b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:32:18.516 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:18.515+0000 7f319fe00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f31a0102e00 0x7f31a0115b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:45570/0 (socket says 172.21.15.18:45570) 2024-04-12T21:32:18.516 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:18.515+0000 7f319fe00700 1 -- 172.21.15.18:0/3991798023 learned_addr learned my addr 172.21.15.18:0/3991798023 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:32:18.516 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:18.515+0000 7f319fe00700 1 -- 172.21.15.18:0/3991798023 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f31a01037a0 msgr2=0x7f31a01160b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:32:18.516 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:18.515+0000 7f3197e00700 1 --2- 172.21.15.18:0/3991798023 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f31a01037a0 0x7f31a01160b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:32:18.516 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:18.515+0000 7f319fe00700 1 --2- 172.21.15.18:0/3991798023 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f31a01037a0 0x7f31a01160b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:32:18.516 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:18.515+0000 7f319fe00700 1 -- 172.21.15.18:0/3991798023 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3188005040 con 0x7f31a0102e00 2024-04-12T21:32:18.516 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:18.515+0000 7f3197e00700 1 --2- 172.21.15.18:0/3991798023 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f31a01037a0 0x7f31a01160b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-12T21:32:18.516 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:18.516+0000 7f319fe00700 1 --2- 172.21.15.18:0/3991798023 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f31a0102e00 0x7f31a0115b70 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7f319000b790 tx=0x7f3190009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:32:18.516 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:18.516+0000 7f319cc00700 1 -- 172.21.15.18:0/3991798023 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f319000f770 con 0x7f31a0102e00 2024-04-12T21:32:18.516 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:18.516+0000 7f319cc00700 1 -- 172.21.15.18:0/3991798023 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f319000fda0 con 0x7f31a0102e00 2024-04-12T21:32:18.516 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:18.516+0000 7f31a6e00700 1 -- 172.21.15.18:0/3991798023 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f31a010ffb0 con 0x7f31a0102e00 2024-04-12T21:32:18.517 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:18.516+0000 7f319cc00700 1 -- 172.21.15.18:0/3991798023 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f319000d310 con 0x7f31a0102e00 2024-04-12T21:32:18.517 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:18.516+0000 7f31a6e00700 1 -- 172.21.15.18:0/3991798023 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f31a0110570 con 0x7f31a0102e00 2024-04-12T21:32:18.517 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:18.517+0000 7f31a6e00700 1 -- 172.21.15.18:0/3991798023 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f31a0107300 con 0x7f31a0102e00 2024-04-12T21:32:18.517 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:18.518+0000 7f319cc00700 1 -- 172.21.15.18:0/3991798023 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7f3190010420 con 0x7f31a0102e00 2024-04-12T21:32:18.519 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:18.519+0000 7f319cc00700 1 --2- 172.21.15.18:0/3991798023 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f318406ec00 0x7f31840710b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:32:18.519 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:18.519+0000 7f319cc00700 1 -- 172.21.15.18:0/3991798023 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 6757+0+0 (secure 0 0 0) 0x7f319008ea60 con 0x7f31a0102e00 2024-04-12T21:32:18.519 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:18.520+0000 7f3197e00700 1 --2- 172.21.15.18:0/3991798023 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f318406ec00 0x7f31840710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:32:18.519 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:18.520+0000 7f3197e00700 1 --2- 172.21.15.18:0/3991798023 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f318406ec00 0x7f31840710b0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f3188009fd0 tx=0x7f3188009e30 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:32:18.524 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:18.524+0000 7f319cc00700 1 -- 172.21.15.18:0/3991798023 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f319005b210 con 0x7f31a0102e00 2024-04-12T21:32:18.777 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:18.777+0000 7f31a6e00700 1 -- 172.21.15.18:0/3991798023 --> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f31a01111c0 con 0x7f318406ec00 2024-04-12T21:32:18.779 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:18.781+0000 7f319cc00700 1 -- 172.21.15.18:0/3991798023 <== mgr.34102 v2:172.21.15.18:6800/3812891919 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f31a01111c0 con 0x7f318406ec00 2024-04-12T21:32:18.783 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:18.783+0000 7f31a6e00700 1 -- 172.21.15.18:0/3991798023 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f318406ec00 msgr2=0x7f31840710b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:32:18.783 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:18.783+0000 7f31a6e00700 1 --2- 172.21.15.18:0/3991798023 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f318406ec00 0x7f31840710b0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f3188009fd0 tx=0x7f3188009e30 comp rx=0 tx=0).stop 2024-04-12T21:32:18.784 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:18.783+0000 7f31a6e00700 1 -- 172.21.15.18:0/3991798023 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f31a0102e00 msgr2=0x7f31a0115b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:32:18.784 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:18.783+0000 7f31a6e00700 1 --2- 172.21.15.18:0/3991798023 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f31a0102e00 0x7f31a0115b70 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7f319000b790 tx=0x7f3190009250 comp rx=0 tx=0).stop 2024-04-12T21:32:18.784 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:18.783+0000 7f31a6e00700 1 -- 172.21.15.18:0/3991798023 shutdown_connections 2024-04-12T21:32:18.784 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:18.783+0000 7f31a6e00700 1 --2- 172.21.15.18:0/3991798023 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f318406ec00 0x7f31840710b0 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:32:18.784 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:18.783+0000 7f31a6e00700 1 --2- 172.21.15.18:0/3991798023 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f31a0102e00 0x7f31a0115b70 unknown :-1 s=CLOSED pgs=823 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:32:18.784 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:18.783+0000 7f31a6e00700 1 --2- 172.21.15.18:0/3991798023 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f31a01037a0 0x7f31a01160b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:32:18.785 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:18.783+0000 7f31a6e00700 1 -- 172.21.15.18:0/3991798023 >> 172.21.15.18:0/3991798023 conn(0x7f31a00f89b0 msgr2=0x7f31a0070a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:32:18.785 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:18.783+0000 7f31a6e00700 1 -- 172.21.15.18:0/3991798023 shutdown_connections 2024-04-12T21:32:18.785 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:18.783+0000 7f31a6e00700 1 -- 172.21.15.18:0/3991798023 wait complete. 2024-04-12T21:32:18.941 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:18.940+0000 7f589d200700 1 -- 172.21.15.18:0/3009637441 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f58980fde90 msgr2=0x7f58980fe260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:32:18.941 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:18.940+0000 7f589d200700 1 --2- 172.21.15.18:0/3009637441 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f58980fde90 0x7f58980fe260 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7f5888004940 tx=0x7f5888009e30 comp rx=0 tx=0).stop 2024-04-12T21:32:18.941 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:18.941+0000 7f589d200700 1 -- 172.21.15.18:0/3009637441 shutdown_connections 2024-04-12T21:32:18.941 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:18.941+0000 7f589d200700 1 --2- 172.21.15.18:0/3009637441 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f58980fe830 0x7f5898106f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:32:18.941 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:18.941+0000 7f589d200700 1 --2- 172.21.15.18:0/3009637441 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f58980fde90 0x7f58980fe260 unknown :-1 s=CLOSED pgs=824 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:32:18.941 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:18.941+0000 7f589d200700 1 -- 172.21.15.18:0/3009637441 >> 172.21.15.18:0/3009637441 conn(0x7f5898072bc0 msgr2=0x7f5898072fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:32:18.941 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:18.941+0000 7f589d200700 1 -- 172.21.15.18:0/3009637441 shutdown_connections 2024-04-12T21:32:18.941 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:18.941+0000 7f589d200700 1 -- 172.21.15.18:0/3009637441 wait complete. 2024-04-12T21:32:18.942 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:18.942+0000 7f589d200700 1 Processor -- start 2024-04-12T21:32:18.942 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:18.943+0000 7f589d200700 1 -- start start 2024-04-12T21:32:18.942 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:18.943+0000 7f589d200700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f58980fde90 0x7f58980fbd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:32:18.942 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:18.943+0000 7f589d200700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f58980fe830 0x7f58980fc290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:32:18.942 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:18.943+0000 7f589d200700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58980fc820 con 0x7f58980fe830 2024-04-12T21:32:18.942 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:18.943+0000 7f589d200700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58980fc960 con 0x7f58980fde90 2024-04-12T21:32:18.944 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:18.943+0000 7f588fe00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f58980fe830 0x7f58980fc290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:32:18.944 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:18.943+0000 7f588fe00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f58980fe830 0x7f58980fc290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:45594/0 (socket says 172.21.15.18:45594) 2024-04-12T21:32:18.944 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:18.943+0000 7f588fe00700 1 -- 172.21.15.18:0/3690591949 learned_addr learned my addr 172.21.15.18:0/3690591949 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:32:18.944 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:18.944+0000 7f5895e00700 1 --2- 172.21.15.18:0/3690591949 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f58980fde90 0x7f58980fbd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:32:18.944 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:18.944+0000 7f588fe00700 1 -- 172.21.15.18:0/3690591949 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f58980fde90 msgr2=0x7f58980fbd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:32:18.944 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:18.944+0000 7f588fe00700 1 --2- 172.21.15.18:0/3690591949 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f58980fde90 0x7f58980fbd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:32:18.944 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:18.944+0000 7f588fe00700 1 -- 172.21.15.18:0/3690591949 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5888005040 con 0x7f58980fe830 2024-04-12T21:32:18.944 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:18.944+0000 7f588fe00700 1 --2- 172.21.15.18:0/3690591949 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f58980fe830 0x7f58980fc290 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7f5880002a50 tx=0x7f58800046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:32:18.944 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:18.945+0000 7f588e000700 1 -- 172.21.15.18:0/3690591949 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5880007cc0 con 0x7f58980fe830 2024-04-12T21:32:18.944 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:18.945+0000 7f588e000700 1 -- 172.21.15.18:0/3690591949 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f588000f040 con 0x7f58980fe830 2024-04-12T21:32:18.944 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:18.945+0000 7f589d200700 1 -- 172.21.15.18:0/3690591949 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5898119ed0 con 0x7f58980fe830 2024-04-12T21:32:18.945 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:18.945+0000 7f588e000700 1 -- 172.21.15.18:0/3690591949 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5880009640 con 0x7f58980fe830 2024-04-12T21:32:18.945 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:18.945+0000 7f589d200700 1 -- 172.21.15.18:0/3690591949 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f589811a2d0 con 0x7f58980fe830 2024-04-12T21:32:18.945 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:18.946+0000 7f589d200700 1 -- 172.21.15.18:0/3690591949 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5898065e30 con 0x7f58980fe830 2024-04-12T21:32:18.947 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:18.947+0000 7f588e000700 1 -- 172.21.15.18:0/3690591949 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7f58800097a0 con 0x7f58980fe830 2024-04-12T21:32:18.948 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:18.948+0000 7f588e000700 1 --2- 172.21.15.18:0/3690591949 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f587c06ec10 0x7f587c0710c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:32:18.948 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:18.948+0000 7f588e000700 1 -- 172.21.15.18:0/3690591949 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 6757+0+0 (secure 0 0 0) 0x7f588008e740 con 0x7f58980fe830 2024-04-12T21:32:18.948 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:18.949+0000 7f5895e00700 1 --2- 172.21.15.18:0/3690591949 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f587c06ec10 0x7f587c0710c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:32:18.948 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:18.949+0000 7f5895e00700 1 --2- 172.21.15.18:0/3690591949 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f587c06ec10 0x7f587c0710c0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f588800b3b0 tx=0x7f5888009e30 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:32:18.952 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:18.953+0000 7f588e000700 1 -- 172.21.15.18:0/3690591949 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f58800573c0 con 0x7f58980fe830 2024-04-12T21:32:19.032 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:32:18 smithi186 ceph-mon[86379]: pgmap v976: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:32:19.102 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:32:18 smithi018 ceph-mon[112590]: pgmap v976: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:32:19.191 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:19.192+0000 7f589d200700 1 -- 172.21.15.18:0/3690591949 --> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f589804bed0 con 0x7f587c06ec10 2024-04-12T21:32:19.206 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:19.207+0000 7f588e000700 1 -- 172.21.15.18:0/3690591949 <== mgr.34102 v2:172.21.15.18:6800/3812891919 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7f589804bed0 con 0x7f587c06ec10 2024-04-12T21:32:19.207 INFO:teuthology.orchestra.run.smithi018.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-12T21:32:19.207 INFO:teuthology.orchestra.run.smithi018.stdout:alertmanager.smithi018 smithi018 *:9093,9094 running (32m) 9m ago 47m 15.5M - 0.25.0 c8568f914cd2 c332090068e8 2024-04-12T21:32:19.207 INFO:teuthology.orchestra.run.smithi018.stdout:crash.smithi018 smithi018 running (29m) 9m ago 47m 6665k - 19.0.0-2879-g324baf8d 8717720dc5e5 b7f80d1f7982 2024-04-12T21:32:19.207 INFO:teuthology.orchestra.run.smithi018.stdout:crash.smithi186 smithi186 running (29m) 9m ago 46m 6660k - 19.0.0-2879-g324baf8d 8717720dc5e5 a4f55c81b623 2024-04-12T21:32:19.207 INFO:teuthology.orchestra.run.smithi018.stdout:grafana.smithi018 smithi018 *:3000 running (32m) 9m ago 47m 74.8M - 9.4.12 2bacad6d85d8 b08e3bf176dd 2024-04-12T21:32:19.207 INFO:teuthology.orchestra.run.smithi018.stdout:mds.cephfs.smithi018.acqbsc smithi018 running (25m) 9m ago 40m 16.7M - 19.0.0-2879-g324baf8d 8717720dc5e5 021f99ab24ae 2024-04-12T21:32:19.207 INFO:teuthology.orchestra.run.smithi018.stdout:mds.cephfs.smithi018.szjluo smithi018 running (24m) 9m ago 40m 16.5M - 19.0.0-2879-g324baf8d 8717720dc5e5 0003117479d4 2024-04-12T21:32:19.207 INFO:teuthology.orchestra.run.smithi018.stdout:mds.cephfs.smithi186.jdllyn smithi186 running (9m) 9m ago 40m 11.0M - 19.0.0-2879-g324baf8d 8717720dc5e5 21b1f46eef23 2024-04-12T21:32:19.207 INFO:teuthology.orchestra.run.smithi018.stdout:mds.cephfs.smithi186.wjnafs smithi186 running (40m) 9m ago 40m 19.5M - 17.2.7-827-gd3b5060a 21de9b7369ad 74b115d6d6fd 2024-04-12T21:32:19.207 INFO:teuthology.orchestra.run.smithi018.stdout:mgr.smithi018.xjzwga smithi018 *:8443,9283,8765 running (35m) 9m ago 49m 512M - 19.0.0-2879-g324baf8d 8717720dc5e5 56088305eab7 2024-04-12T21:32:19.207 INFO:teuthology.orchestra.run.smithi018.stdout:mgr.smithi186.kqcfxk smithi186 *:8443,9283,8765 running (34m) 9m ago 46m 459M - 19.0.0-2879-g324baf8d 8717720dc5e5 09cd766e5e0d 2024-04-12T21:32:19.208 INFO:teuthology.orchestra.run.smithi018.stdout:mon.smithi018 smithi018 running (30m) 9m ago 49m 73.1M 2048M 19.0.0-2879-g324baf8d 8717720dc5e5 df3e201dfed6 2024-04-12T21:32:19.208 INFO:teuthology.orchestra.run.smithi018.stdout:mon.smithi186 smithi186 running (30m) 9m ago 45m 47.2M 2048M 19.0.0-2879-g324baf8d 8717720dc5e5 c77498da9b2b 2024-04-12T21:32:19.208 INFO:teuthology.orchestra.run.smithi018.stdout:node-exporter.smithi018 smithi018 *:9100 running (33m) 9m ago 47m 8108k - 1.5.0 0da6a335fe13 326c3d9c6847 2024-04-12T21:32:19.208 INFO:teuthology.orchestra.run.smithi018.stdout:node-exporter.smithi186 smithi186 *:9100 running (33m) 9m ago 46m 8757k - 1.5.0 0da6a335fe13 ec2112baeecf 2024-04-12T21:32:19.208 INFO:teuthology.orchestra.run.smithi018.stdout:osd.0 smithi018 running (28m) 9m ago 45m 123M 1559M 19.0.0-2879-g324baf8d 8717720dc5e5 60b1273ca047 2024-04-12T21:32:19.208 INFO:teuthology.orchestra.run.smithi018.stdout:osd.1 smithi018 running (28m) 9m ago 44m 95.1M 1559M 19.0.0-2879-g324baf8d 8717720dc5e5 170ce1e5fb8d 2024-04-12T21:32:19.208 INFO:teuthology.orchestra.run.smithi018.stdout:osd.2 smithi018 running (27m) 9m ago 43m 94.3M 1559M 19.0.0-2879-g324baf8d 8717720dc5e5 c2cbc6aa2839 2024-04-12T21:32:19.208 INFO:teuthology.orchestra.run.smithi018.stdout:osd.3 smithi186 running (26m) 9m ago 43m 101M 2581M 19.0.0-2879-g324baf8d 8717720dc5e5 0138e8262e4b 2024-04-12T21:32:19.208 INFO:teuthology.orchestra.run.smithi018.stdout:osd.4 smithi186 running (26m) 9m ago 42m 91.0M 2581M 19.0.0-2879-g324baf8d 8717720dc5e5 b20cb63b25c9 2024-04-12T21:32:19.208 INFO:teuthology.orchestra.run.smithi018.stdout:osd.5 smithi186 running (25m) 9m ago 42m 92.0M 2581M 19.0.0-2879-g324baf8d 8717720dc5e5 2513bea2610e 2024-04-12T21:32:19.208 INFO:teuthology.orchestra.run.smithi018.stdout:prometheus.smithi018 smithi018 *:9095 running (33m) 9m ago 46m 67.9M - 2.43.0 a07b618ecd1d f58841127e21 2024-04-12T21:32:19.209 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:19.210+0000 7f589d200700 1 -- 172.21.15.18:0/3690591949 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f587c06ec10 msgr2=0x7f587c0710c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:32:19.210 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:19.210+0000 7f589d200700 1 --2- 172.21.15.18:0/3690591949 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f587c06ec10 0x7f587c0710c0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f588800b3b0 tx=0x7f5888009e30 comp rx=0 tx=0).stop 2024-04-12T21:32:19.210 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:19.210+0000 7f589d200700 1 -- 172.21.15.18:0/3690591949 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f58980fe830 msgr2=0x7f58980fc290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:32:19.210 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:19.210+0000 7f589d200700 1 --2- 172.21.15.18:0/3690591949 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f58980fe830 0x7f58980fc290 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7f5880002a50 tx=0x7f58800046b0 comp rx=0 tx=0).stop 2024-04-12T21:32:19.210 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:19.210+0000 7f589d200700 1 -- 172.21.15.18:0/3690591949 shutdown_connections 2024-04-12T21:32:19.210 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:19.210+0000 7f589d200700 1 --2- 172.21.15.18:0/3690591949 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f587c06ec10 0x7f587c0710c0 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:32:19.210 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:19.210+0000 7f589d200700 1 --2- 172.21.15.18:0/3690591949 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f58980fde90 0x7f58980fbd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:32:19.210 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:19.210+0000 7f589d200700 1 --2- 172.21.15.18:0/3690591949 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f58980fe830 0x7f58980fc290 unknown :-1 s=CLOSED pgs=825 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:32:19.210 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:19.210+0000 7f589d200700 1 -- 172.21.15.18:0/3690591949 >> 172.21.15.18:0/3690591949 conn(0x7f5898072bc0 msgr2=0x7f58980ff3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:32:19.210 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:19.210+0000 7f589d200700 1 -- 172.21.15.18:0/3690591949 shutdown_connections 2024-04-12T21:32:19.210 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:19.210+0000 7f589d200700 1 -- 172.21.15.18:0/3690591949 wait complete. 2024-04-12T21:32:19.352 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:19.352+0000 7fc6bf200700 1 -- 172.21.15.18:0/1599739797 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc6c0109c40 msgr2=0x7fc6c01032b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:32:19.352 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:19.352+0000 7fc6bf200700 1 --2- 172.21.15.18:0/1599739797 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc6c0109c40 0x7fc6c01032b0 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7fc6ac004940 tx=0x7fc6ac009e30 comp rx=0 tx=0).stop 2024-04-12T21:32:19.352 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:19.352+0000 7fc6bf200700 1 -- 172.21.15.18:0/1599739797 shutdown_connections 2024-04-12T21:32:19.352 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:19.352+0000 7fc6bf200700 1 --2- 172.21.15.18:0/1599739797 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc6c0109c40 0x7fc6c01032b0 unknown :-1 s=CLOSED pgs=826 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:32:19.352 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:19.352+0000 7fc6bf200700 1 --2- 172.21.15.18:0/1599739797 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc6c01092a0 0x7fc6c0109670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:32:19.352 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:19.352+0000 7fc6bf200700 1 -- 172.21.15.18:0/1599739797 >> 172.21.15.18:0/1599739797 conn(0x7fc6c0072a30 msgr2=0x7fc6c0074e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:32:19.352 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:19.352+0000 7fc6bf200700 1 -- 172.21.15.18:0/1599739797 shutdown_connections 2024-04-12T21:32:19.352 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:19.353+0000 7fc6bf200700 1 -- 172.21.15.18:0/1599739797 wait complete. 2024-04-12T21:32:19.353 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:19.353+0000 7fc6bf200700 1 Processor -- start 2024-04-12T21:32:19.353 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:19.354+0000 7fc6bf200700 1 -- start start 2024-04-12T21:32:19.353 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:19.354+0000 7fc6bf200700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc6c01092a0 0x7fc6c0119fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:32:19.353 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:19.354+0000 7fc6bf200700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc6c0109c40 0x7fc6c011a500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:32:19.353 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:19.354+0000 7fc6bf200700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6c011ab50 con 0x7fc6c01092a0 2024-04-12T21:32:19.353 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:19.354+0000 7fc6bf200700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6c0113f20 con 0x7fc6c0109c40 2024-04-12T21:32:19.355 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:19.354+0000 7fc6bde00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc6c01092a0 0x7fc6c0119fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:32:19.355 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:19.354+0000 7fc6bde00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc6c01092a0 0x7fc6c0119fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:45620/0 (socket says 172.21.15.18:45620) 2024-04-12T21:32:19.355 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:19.354+0000 7fc6bde00700 1 -- 172.21.15.18:0/3116352670 learned_addr learned my addr 172.21.15.18:0/3116352670 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:32:19.355 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:19.355+0000 7fc6bd400700 1 --2- 172.21.15.18:0/3116352670 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc6c0109c40 0x7fc6c011a500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:32:19.355 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:19.355+0000 7fc6bde00700 1 -- 172.21.15.18:0/3116352670 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc6c0109c40 msgr2=0x7fc6c011a500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:32:19.355 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:19.355+0000 7fc6bde00700 1 --2- 172.21.15.18:0/3116352670 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc6c0109c40 0x7fc6c011a500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:32:19.355 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:19.355+0000 7fc6bde00700 1 -- 172.21.15.18:0/3116352670 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6ac005040 con 0x7fc6c01092a0 2024-04-12T21:32:19.355 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:19.355+0000 7fc6bde00700 1 --2- 172.21.15.18:0/3116352670 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc6c01092a0 0x7fc6c0119fc0 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7fc6b000c830 tx=0x7fc6b0004470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:32:19.355 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:19.355+0000 7fc6b6000700 1 -- 172.21.15.18:0/3116352670 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc6b000a760 con 0x7fc6c01092a0 2024-04-12T21:32:19.355 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:19.355+0000 7fc6b6000700 1 -- 172.21.15.18:0/3116352670 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc6b000ad90 con 0x7fc6c01092a0 2024-04-12T21:32:19.355 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:19.355+0000 7fc6b6000700 1 -- 172.21.15.18:0/3116352670 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc6b00113a0 con 0x7fc6c01092a0 2024-04-12T21:32:19.355 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:19.355+0000 7fc6bf200700 1 -- 172.21.15.18:0/3116352670 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc6c0114200 con 0x7fc6c01092a0 2024-04-12T21:32:19.355 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:19.355+0000 7fc6bf200700 1 -- 172.21.15.18:0/3116352670 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc6c01147c0 con 0x7fc6c01092a0 2024-04-12T21:32:19.356 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:19.356+0000 7fc6bf200700 1 -- 172.21.15.18:0/3116352670 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc6c01049f0 con 0x7fc6c01092a0 2024-04-12T21:32:19.357 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:19.357+0000 7fc6b6000700 1 -- 172.21.15.18:0/3116352670 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7fc6b0012400 con 0x7fc6c01092a0 2024-04-12T21:32:19.358 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:19.358+0000 7fc6b6000700 1 --2- 172.21.15.18:0/3116352670 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fc6a406ec00 0x7fc6a40710b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:32:19.358 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:19.359+0000 7fc6b6000700 1 -- 172.21.15.18:0/3116352670 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 6757+0+0 (secure 0 0 0) 0x7fc6b008e9c0 con 0x7fc6c01092a0 2024-04-12T21:32:19.358 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:19.359+0000 7fc6bd400700 1 --2- 172.21.15.18:0/3116352670 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fc6a406ec00 0x7fc6a40710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:32:19.358 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:19.359+0000 7fc6bd400700 1 --2- 172.21.15.18:0/3116352670 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fc6a406ec00 0x7fc6a40710b0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7fc6ac00b3b0 tx=0x7fc6ac009e30 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:32:19.362 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:19.363+0000 7fc6b6000700 1 -- 172.21.15.18:0/3116352670 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7fc6b005b1f0 con 0x7fc6c01092a0 2024-04-12T21:32:19.677 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:19.678+0000 7fc6bf200700 1 -- 172.21.15.18:0/3116352670 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fc6c0076560 con 0x7fc6c01092a0 2024-04-12T21:32:19.678 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:19.679+0000 7fc6b6000700 1 -- 172.21.15.18:0/3116352670 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+837 (secure 0 0 0) 0x7fc6b005ad80 con 0x7fc6c01092a0 2024-04-12T21:32:19.678 INFO:teuthology.orchestra.run.smithi018.stdout:{ 2024-04-12T21:32:19.678 INFO:teuthology.orchestra.run.smithi018.stdout: "mon": { 2024-04-12T21:32:19.679 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 19.0.0-2879-g324baf8d (324baf8ddb91dea46cac50cb535c306115791355) squid (dev)": 2 2024-04-12T21:32:19.679 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-12T21:32:19.679 INFO:teuthology.orchestra.run.smithi018.stdout: "mgr": { 2024-04-12T21:32:19.679 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 19.0.0-2879-g324baf8d (324baf8ddb91dea46cac50cb535c306115791355) squid (dev)": 2 2024-04-12T21:32:19.679 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-12T21:32:19.679 INFO:teuthology.orchestra.run.smithi018.stdout: "osd": { 2024-04-12T21:32:19.679 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 19.0.0-2879-g324baf8d (324baf8ddb91dea46cac50cb535c306115791355) squid (dev)": 6 2024-04-12T21:32:19.680 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-12T21:32:19.680 INFO:teuthology.orchestra.run.smithi018.stdout: "mds": { 2024-04-12T21:32:19.680 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 17.2.7-827-gd3b5060a (d3b5060a17449edfae9385cdf792afec109fca5d) quincy (stable)": 1, 2024-04-12T21:32:19.680 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 19.0.0-2879-g324baf8d (324baf8ddb91dea46cac50cb535c306115791355) squid (dev)": 3 2024-04-12T21:32:19.680 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-12T21:32:19.680 INFO:teuthology.orchestra.run.smithi018.stdout: "overall": { 2024-04-12T21:32:19.680 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 17.2.7-827-gd3b5060a (d3b5060a17449edfae9385cdf792afec109fca5d) quincy (stable)": 1, 2024-04-12T21:32:19.680 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 19.0.0-2879-g324baf8d (324baf8ddb91dea46cac50cb535c306115791355) squid (dev)": 13 2024-04-12T21:32:19.680 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-04-12T21:32:19.680 INFO:teuthology.orchestra.run.smithi018.stdout:} 2024-04-12T21:32:19.682 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:19.682+0000 7fc6bf200700 1 -- 172.21.15.18:0/3116352670 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fc6a406ec00 msgr2=0x7fc6a40710b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:32:19.682 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:19.682+0000 7fc6bf200700 1 --2- 172.21.15.18:0/3116352670 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fc6a406ec00 0x7fc6a40710b0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7fc6ac00b3b0 tx=0x7fc6ac009e30 comp rx=0 tx=0).stop 2024-04-12T21:32:19.682 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:19.682+0000 7fc6bf200700 1 -- 172.21.15.18:0/3116352670 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc6c01092a0 msgr2=0x7fc6c0119fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:32:19.682 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:19.682+0000 7fc6bf200700 1 --2- 172.21.15.18:0/3116352670 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc6c01092a0 0x7fc6c0119fc0 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7fc6b000c830 tx=0x7fc6b0004470 comp rx=0 tx=0).stop 2024-04-12T21:32:19.682 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:19.682+0000 7fc6bf200700 1 -- 172.21.15.18:0/3116352670 shutdown_connections 2024-04-12T21:32:19.682 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:19.682+0000 7fc6bf200700 1 --2- 172.21.15.18:0/3116352670 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fc6a406ec00 0x7fc6a40710b0 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:32:19.682 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:19.682+0000 7fc6bf200700 1 --2- 172.21.15.18:0/3116352670 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc6c01092a0 0x7fc6c0119fc0 unknown :-1 s=CLOSED pgs=827 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:32:19.682 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:19.682+0000 7fc6bf200700 1 --2- 172.21.15.18:0/3116352670 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc6c0109c40 0x7fc6c011a500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:32:19.682 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:19.682+0000 7fc6bf200700 1 -- 172.21.15.18:0/3116352670 >> 172.21.15.18:0/3116352670 conn(0x7fc6c0072a30 msgr2=0x7fc6c0074910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:32:19.682 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:19.682+0000 7fc6bf200700 1 -- 172.21.15.18:0/3116352670 shutdown_connections 2024-04-12T21:32:19.682 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:19.682+0000 7fc6bf200700 1 -- 172.21.15.18:0/3116352670 wait complete. 2024-04-12T21:32:19.837 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:19.837+0000 7f6ca3200700 1 -- 172.21.15.18:0/313198310 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6ca4070bf0 msgr2=0x7f6ca4108ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:32:19.838 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:19.837+0000 7f6ca3200700 1 --2- 172.21.15.18:0/313198310 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6ca4070bf0 0x7f6ca4108ff0 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7f6c90004940 tx=0x7f6c90009e30 comp rx=0 tx=0).stop 2024-04-12T21:32:19.838 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:19.837+0000 7f6ca3200700 1 -- 172.21.15.18:0/313198310 shutdown_connections 2024-04-12T21:32:19.838 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:19.837+0000 7f6ca3200700 1 --2- 172.21.15.18:0/313198310 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6ca4070bf0 0x7f6ca4108ff0 unknown :-1 s=CLOSED pgs=828 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:32:19.838 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:19.837+0000 7f6ca3200700 1 --2- 172.21.15.18:0/313198310 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6ca40702e0 0x7f6ca40706b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:32:19.838 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:19.837+0000 7f6ca3200700 1 -- 172.21.15.18:0/313198310 >> 172.21.15.18:0/313198310 conn(0x7f6ca40f8830 msgr2=0x7f6ca40fac40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:32:19.838 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:19.837+0000 7f6ca3200700 1 -- 172.21.15.18:0/313198310 shutdown_connections 2024-04-12T21:32:19.838 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:19.837+0000 7f6ca3200700 1 -- 172.21.15.18:0/313198310 wait complete. 2024-04-12T21:32:19.838 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:19.838+0000 7f6ca3200700 1 Processor -- start 2024-04-12T21:32:19.838 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:19.838+0000 7f6ca3200700 1 -- start start 2024-04-12T21:32:19.840 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:19.839+0000 7f6ca3200700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6ca40702e0 0x7f6ca4115c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:32:19.840 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:19.839+0000 7f6ca3200700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6ca4070bf0 0x7f6ca41161a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:32:19.840 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:19.839+0000 7f6ca3200700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ca4116880 con 0x7f6ca40702e0 2024-04-12T21:32:19.841 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:19.839+0000 7f6ca3200700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ca410fbc0 con 0x7f6ca4070bf0 2024-04-12T21:32:19.841 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:19.839+0000 7f6ca1e00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6ca40702e0 0x7f6ca4115c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:32:19.841 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:19.839+0000 7f6ca1e00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6ca40702e0 0x7f6ca4115c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:45638/0 (socket says 172.21.15.18:45638) 2024-04-12T21:32:19.841 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:19.839+0000 7f6ca1e00700 1 -- 172.21.15.18:0/3445444388 learned_addr learned my addr 172.21.15.18:0/3445444388 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:32:19.841 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:19.839+0000 7f6ca1400700 1 --2- 172.21.15.18:0/3445444388 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6ca4070bf0 0x7f6ca41161a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:32:19.841 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:19.840+0000 7f6ca1e00700 1 -- 172.21.15.18:0/3445444388 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6ca4070bf0 msgr2=0x7f6ca41161a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:32:19.841 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:19.840+0000 7f6ca1e00700 1 --2- 172.21.15.18:0/3445444388 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6ca4070bf0 0x7f6ca41161a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:32:19.841 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:19.840+0000 7f6ca1e00700 1 -- 172.21.15.18:0/3445444388 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c90005040 con 0x7f6ca40702e0 2024-04-12T21:32:19.841 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:19.840+0000 7f6ca1e00700 1 --2- 172.21.15.18:0/3445444388 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6ca40702e0 0x7f6ca4115c60 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7f6c9400b790 tx=0x7f6c94009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:32:19.842 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:19.840+0000 7f6c9a000700 1 -- 172.21.15.18:0/3445444388 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6c9400f770 con 0x7f6ca40702e0 2024-04-12T21:32:19.842 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:19.840+0000 7f6c9a000700 1 -- 172.21.15.18:0/3445444388 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f6c9400fda0 con 0x7f6ca40702e0 2024-04-12T21:32:19.842 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:19.840+0000 7f6ca3200700 1 -- 172.21.15.18:0/3445444388 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6ca410fe40 con 0x7f6ca40702e0 2024-04-12T21:32:19.842 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:19.840+0000 7f6c9a000700 1 -- 172.21.15.18:0/3445444388 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6c9400d310 con 0x7f6ca40702e0 2024-04-12T21:32:19.842 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:19.840+0000 7f6ca3200700 1 -- 172.21.15.18:0/3445444388 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6ca4110380 con 0x7f6ca40702e0 2024-04-12T21:32:19.842 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:19.841+0000 7f6ca3200700 1 -- 172.21.15.18:0/3445444388 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6ca4071a20 con 0x7f6ca40702e0 2024-04-12T21:32:19.843 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:19.843+0000 7f6c9a000700 1 -- 172.21.15.18:0/3445444388 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7f6c94010420 con 0x7f6ca40702e0 2024-04-12T21:32:19.843 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:19.843+0000 7f6c9a000700 1 --2- 172.21.15.18:0/3445444388 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f6c8806ec10 0x7f6c880710c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:32:19.843 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:19.844+0000 7f6c9a000700 1 -- 172.21.15.18:0/3445444388 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 6757+0+0 (secure 0 0 0) 0x7f6c9408ea60 con 0x7f6ca40702e0 2024-04-12T21:32:19.843 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:19.844+0000 7f6ca1400700 1 --2- 172.21.15.18:0/3445444388 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f6c8806ec10 0x7f6c880710c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:32:19.844 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:19.845+0000 7f6ca1400700 1 --2- 172.21.15.18:0/3445444388 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f6c8806ec10 0x7f6c880710c0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f6c9000b020 tx=0x7f6c90009e30 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:32:19.848 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:19.850+0000 7f6c9a000700 1 -- 172.21.15.18:0/3445444388 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f6c9405b210 con 0x7f6ca40702e0 2024-04-12T21:32:20.103 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:32:19 smithi018 ceph-mon[112590]: from='client.35708 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:32:20.103 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:32:19 smithi018 ceph-mon[112590]: from='client.35712 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:32:20.103 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:32:19 smithi018 ceph-mon[112590]: from='client.? 172.21.15.18:0/3116352670' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T21:32:20.131 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:20.132+0000 7f6ca3200700 1 -- 172.21.15.18:0/3445444388 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f6ca405f5f0 con 0x7f6ca40702e0 2024-04-12T21:32:20.132 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:20.133+0000 7f6c9a000700 1 -- 172.21.15.18:0/3445444388 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 31 v31) v1 ==== 76+0+1873 (secure 0 0 0) 0x7f6c9405ada0 con 0x7f6ca40702e0 2024-04-12T21:32:20.133 INFO:teuthology.orchestra.run.smithi018.stdout:e31 2024-04-12T21:32:20.133 INFO:teuthology.orchestra.run.smithi018.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-12T21:32:20.133 INFO:teuthology.orchestra.run.smithi018.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-12T21:32:20.134 INFO:teuthology.orchestra.run.smithi018.stdout:legacy client fscid: 1 2024-04-12T21:32:20.134 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-12T21:32:20.134 INFO:teuthology.orchestra.run.smithi018.stdout:Filesystem 'cephfs' (1) 2024-04-12T21:32:20.134 INFO:teuthology.orchestra.run.smithi018.stdout:fs_name cephfs 2024-04-12T21:32:20.134 INFO:teuthology.orchestra.run.smithi018.stdout:epoch 24 2024-04-12T21:32:20.134 INFO:teuthology.orchestra.run.smithi018.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-04-12T21:32:20.134 INFO:teuthology.orchestra.run.smithi018.stdout:created 2024-04-12T20:51:57.362850+0000 2024-04-12T21:32:20.134 INFO:teuthology.orchestra.run.smithi018.stdout:modified 2024-04-12T21:06:59.717190+0000 2024-04-12T21:32:20.134 INFO:teuthology.orchestra.run.smithi018.stdout:tableserver 0 2024-04-12T21:32:20.134 INFO:teuthology.orchestra.run.smithi018.stdout:root 0 2024-04-12T21:32:20.134 INFO:teuthology.orchestra.run.smithi018.stdout:session_timeout 60 2024-04-12T21:32:20.134 INFO:teuthology.orchestra.run.smithi018.stdout:session_autoclose 300 2024-04-12T21:32:20.134 INFO:teuthology.orchestra.run.smithi018.stdout:max_file_size 1099511627776 2024-04-12T21:32:20.134 INFO:teuthology.orchestra.run.smithi018.stdout:max_xattr_size 65536 2024-04-12T21:32:20.135 INFO:teuthology.orchestra.run.smithi018.stdout:required_client_features {} 2024-04-12T21:32:20.135 INFO:teuthology.orchestra.run.smithi018.stdout:last_failure 0 2024-04-12T21:32:20.135 INFO:teuthology.orchestra.run.smithi018.stdout:last_failure_osd_epoch 100 2024-04-12T21:32:20.135 INFO:teuthology.orchestra.run.smithi018.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-12T21:32:20.135 INFO:teuthology.orchestra.run.smithi018.stdout:max_mds 2 2024-04-12T21:32:20.135 INFO:teuthology.orchestra.run.smithi018.stdout:in 0,1 2024-04-12T21:32:20.135 INFO:teuthology.orchestra.run.smithi018.stdout:up {} 2024-04-12T21:32:20.135 INFO:teuthology.orchestra.run.smithi018.stdout:failed 0,1 2024-04-12T21:32:20.135 INFO:teuthology.orchestra.run.smithi018.stdout:damaged 2024-04-12T21:32:20.135 INFO:teuthology.orchestra.run.smithi018.stdout:stopped 2024-04-12T21:32:20.135 INFO:teuthology.orchestra.run.smithi018.stdout:data_pools [3] 2024-04-12T21:32:20.135 INFO:teuthology.orchestra.run.smithi018.stdout:metadata_pool 2 2024-04-12T21:32:20.135 INFO:teuthology.orchestra.run.smithi018.stdout:inline_data disabled 2024-04-12T21:32:20.135 INFO:teuthology.orchestra.run.smithi018.stdout:balancer 2024-04-12T21:32:20.135 INFO:teuthology.orchestra.run.smithi018.stdout:bal_rank_mask -1 2024-04-12T21:32:20.136 INFO:teuthology.orchestra.run.smithi018.stdout:standby_count_wanted 1 2024-04-12T21:32:20.136 INFO:teuthology.orchestra.run.smithi018.stdout:qdb_cluster leader: 0 members: 2024-04-12T21:32:20.136 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-12T21:32:20.136 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-12T21:32:20.136 INFO:teuthology.orchestra.run.smithi018.stdout:Standby daemons: 2024-04-12T21:32:20.136 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-12T21:32:20.136 INFO:teuthology.orchestra.run.smithi018.stdout:[mds.cephfs.smithi186.wjnafs{-1:24277} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.186:6826/438347207,v1:172.21.15.186:6827/438347207] compat {c=[1],r=[1],i=[7ff]}] 2024-04-12T21:32:20.136 INFO:teuthology.orchestra.run.smithi018.stdout:[mds.cephfs.smithi018.acqbsc{-1:34436} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.18:6828/755179764,v1:172.21.15.18:6829/755179764] compat {c=[1],r=[1],i=[1fff]}] 2024-04-12T21:32:20.136 INFO:teuthology.orchestra.run.smithi018.stdout:[mds.cephfs.smithi018.szjluo{-1:34468} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.18:6826/2407992653,v1:172.21.15.18:6827/2407992653] compat {c=[1],r=[1],i=[1fff]}] 2024-04-12T21:32:20.136 INFO:teuthology.orchestra.run.smithi018.stdout:[mds.cephfs.smithi186.jdllyn{-1:44595} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.186:6824/3523444202,v1:172.21.15.186:6825/3523444202] compat {c=[1],r=[1],i=[1fff]}] 2024-04-12T21:32:20.137 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:20.136+0000 7f6ca3200700 1 -- 172.21.15.18:0/3445444388 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f6c8806ec10 msgr2=0x7f6c880710c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:32:20.137 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:20.136+0000 7f6ca3200700 1 --2- 172.21.15.18:0/3445444388 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f6c8806ec10 0x7f6c880710c0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f6c9000b020 tx=0x7f6c90009e30 comp rx=0 tx=0).stop 2024-04-12T21:32:20.137 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:20.137+0000 7f6ca3200700 1 -- 172.21.15.18:0/3445444388 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6ca40702e0 msgr2=0x7f6ca4115c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:32:20.137 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:20.137+0000 7f6ca3200700 1 --2- 172.21.15.18:0/3445444388 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6ca40702e0 0x7f6ca4115c60 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7f6c9400b790 tx=0x7f6c94009250 comp rx=0 tx=0).stop 2024-04-12T21:32:20.137 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:20.137+0000 7f6ca3200700 1 -- 172.21.15.18:0/3445444388 shutdown_connections 2024-04-12T21:32:20.137 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:20.137+0000 7f6ca3200700 1 --2- 172.21.15.18:0/3445444388 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f6c8806ec10 0x7f6c880710c0 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:32:20.137 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:20.137+0000 7f6ca3200700 1 --2- 172.21.15.18:0/3445444388 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6ca40702e0 0x7f6ca4115c60 unknown :-1 s=CLOSED pgs=829 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:32:20.137 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:20.137+0000 7f6ca3200700 1 --2- 172.21.15.18:0/3445444388 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6ca4070bf0 0x7f6ca41161a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:32:20.137 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:20.137+0000 7f6ca3200700 1 -- 172.21.15.18:0/3445444388 >> 172.21.15.18:0/3445444388 conn(0x7f6ca40f8830 msgr2=0x7f6ca40f9be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:32:20.137 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:20.137+0000 7f6ca3200700 1 -- 172.21.15.18:0/3445444388 shutdown_connections 2024-04-12T21:32:20.138 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:20.137+0000 7f6ca3200700 1 -- 172.21.15.18:0/3445444388 wait complete. 2024-04-12T21:32:20.138 INFO:teuthology.orchestra.run.smithi018.stderr:dumped fsmap epoch 31 2024-04-12T21:32:20.282 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:32:19 smithi186 ceph-mon[86379]: from='client.35708 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:32:20.282 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:32:19 smithi186 ceph-mon[86379]: from='client.35712 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:32:20.282 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:32:19 smithi186 ceph-mon[86379]: from='client.? 172.21.15.18:0/3116352670' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T21:32:20.294 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:20.295+0000 7ff685400700 1 -- 172.21.15.18:0/1127098430 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff680109dd0 msgr2=0x7ff680103440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:32:20.295 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:20.295+0000 7ff685400700 1 --2- 172.21.15.18:0/1127098430 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff680109dd0 0x7ff680103440 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7ff668009ce0 tx=0x7ff668004d10 comp rx=0 tx=0).stop 2024-04-12T21:32:20.295 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:20.295+0000 7ff685400700 1 -- 172.21.15.18:0/1127098430 shutdown_connections 2024-04-12T21:32:20.295 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:20.295+0000 7ff685400700 1 --2- 172.21.15.18:0/1127098430 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff680109dd0 0x7ff680103440 unknown :-1 s=CLOSED pgs=830 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:32:20.295 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:20.295+0000 7ff685400700 1 --2- 172.21.15.18:0/1127098430 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff680109430 0x7ff680109800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:32:20.295 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:20.295+0000 7ff685400700 1 -- 172.21.15.18:0/1127098430 >> 172.21.15.18:0/1127098430 conn(0x7ff680072c00 msgr2=0x7ff680075010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:32:20.295 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:20.295+0000 7ff685400700 1 -- 172.21.15.18:0/1127098430 shutdown_connections 2024-04-12T21:32:20.295 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:20.295+0000 7ff685400700 1 -- 172.21.15.18:0/1127098430 wait complete. 2024-04-12T21:32:20.296 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:20.296+0000 7ff685400700 1 Processor -- start 2024-04-12T21:32:20.296 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:20.296+0000 7ff685400700 1 -- start start 2024-04-12T21:32:20.296 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:20.296+0000 7ff685400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff680109430 0x7ff68011a180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:32:20.296 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:20.296+0000 7ff685400700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff680109dd0 0x7ff68011a6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:32:20.296 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:20.296+0000 7ff685400700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff68011ad10 con 0x7ff680109430 2024-04-12T21:32:20.296 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:20.296+0000 7ff685400700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6801142f0 con 0x7ff680109dd0 2024-04-12T21:32:20.296 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:20.297+0000 7ff67e800700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff680109430 0x7ff68011a180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:32:20.296 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:20.297+0000 7ff67e800700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff680109430 0x7ff68011a180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:45664/0 (socket says 172.21.15.18:45664) 2024-04-12T21:32:20.296 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:20.297+0000 7ff67e800700 1 -- 172.21.15.18:0/3795015052 learned_addr learned my addr 172.21.15.18:0/3795015052 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:32:20.297 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:20.297+0000 7ff675e00700 1 --2- 172.21.15.18:0/3795015052 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff680109dd0 0x7ff68011a6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:32:20.298 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:20.297+0000 7ff67e800700 1 -- 172.21.15.18:0/3795015052 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff680109dd0 msgr2=0x7ff68011a6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:32:20.298 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:20.297+0000 7ff67e800700 1 --2- 172.21.15.18:0/3795015052 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff680109dd0 0x7ff68011a6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:32:20.298 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:20.297+0000 7ff67e800700 1 -- 172.21.15.18:0/3795015052 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff668009940 con 0x7ff680109430 2024-04-12T21:32:20.298 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:20.297+0000 7ff67e800700 1 --2- 172.21.15.18:0/3795015052 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff680109430 0x7ff68011a180 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7ff670002a50 tx=0x7ff6700046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:32:20.298 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:20.298+0000 7ff677400700 1 -- 172.21.15.18:0/3795015052 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff670007cc0 con 0x7ff680109430 2024-04-12T21:32:20.298 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:20.298+0000 7ff677400700 1 -- 172.21.15.18:0/3795015052 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff67000f040 con 0x7ff680109430 2024-04-12T21:32:20.298 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:20.298+0000 7ff685400700 1 -- 172.21.15.18:0/3795015052 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff6801145d0 con 0x7ff680109430 2024-04-12T21:32:20.298 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:20.298+0000 7ff677400700 1 -- 172.21.15.18:0/3795015052 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff670009640 con 0x7ff680109430 2024-04-12T21:32:20.298 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:20.298+0000 7ff685400700 1 -- 172.21.15.18:0/3795015052 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff680114b90 con 0x7ff680109430 2024-04-12T21:32:20.299 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:20.299+0000 7ff685400700 1 -- 172.21.15.18:0/3795015052 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff680104b80 con 0x7ff680109430 2024-04-12T21:32:20.299 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:20.300+0000 7ff677400700 1 -- 172.21.15.18:0/3795015052 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7ff6700097c0 con 0x7ff680109430 2024-04-12T21:32:20.300 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:20.301+0000 7ff677400700 1 --2- 172.21.15.18:0/3795015052 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7ff66406ec00 0x7ff6640710b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:32:20.300 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:20.301+0000 7ff677400700 1 -- 172.21.15.18:0/3795015052 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 6757+0+0 (secure 0 0 0) 0x7ff67008e7d0 con 0x7ff680109430 2024-04-12T21:32:20.300 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:20.301+0000 7ff675e00700 1 --2- 172.21.15.18:0/3795015052 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7ff66406ec00 0x7ff6640710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:32:20.301 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:20.302+0000 7ff675e00700 1 --2- 172.21.15.18:0/3795015052 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7ff66406ec00 0x7ff6640710b0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7ff66800b740 tx=0x7ff668004d10 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:32:20.305 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:20.307+0000 7ff677400700 1 -- 172.21.15.18:0/3795015052 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7ff67005af00 con 0x7ff680109430 2024-04-12T21:32:20.552 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:20.552+0000 7ff685400700 1 -- 172.21.15.18:0/3795015052 --> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7ff680115840 con 0x7ff66406ec00 2024-04-12T21:32:20.555 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:20.556+0000 7ff677400700 1 -- 172.21.15.18:0/3795015052 <== mgr.34102 v2:172.21.15.18:6800/3812891919 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7ff680115840 con 0x7ff66406ec00 2024-04-12T21:32:20.556 INFO:teuthology.orchestra.run.smithi018.stdout:{ 2024-04-12T21:32:20.556 INFO:teuthology.orchestra.run.smithi018.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:f4452def3be4d350f4a8e5feaf7ebb1511be9298fe68478ba7a2d6fbd7d69a63", 2024-04-12T21:32:20.556 INFO:teuthology.orchestra.run.smithi018.stdout: "in_progress": true, 2024-04-12T21:32:20.556 INFO:teuthology.orchestra.run.smithi018.stdout: "which": "Upgrading all daemon types on all hosts", 2024-04-12T21:32:20.556 INFO:teuthology.orchestra.run.smithi018.stdout: "services_complete": [ 2024-04-12T21:32:20.556 INFO:teuthology.orchestra.run.smithi018.stdout: "crash", 2024-04-12T21:32:20.556 INFO:teuthology.orchestra.run.smithi018.stdout: "osd", 2024-04-12T21:32:20.556 INFO:teuthology.orchestra.run.smithi018.stdout: "mon", 2024-04-12T21:32:20.556 INFO:teuthology.orchestra.run.smithi018.stdout: "mgr" 2024-04-12T21:32:20.557 INFO:teuthology.orchestra.run.smithi018.stdout: ], 2024-04-12T21:32:20.557 INFO:teuthology.orchestra.run.smithi018.stdout: "progress": "15/21 daemons upgraded", 2024-04-12T21:32:20.557 INFO:teuthology.orchestra.run.smithi018.stdout: "message": "Currently upgrading mds daemons", 2024-04-12T21:32:20.557 INFO:teuthology.orchestra.run.smithi018.stdout: "is_paused": false 2024-04-12T21:32:20.557 INFO:teuthology.orchestra.run.smithi018.stdout:} 2024-04-12T21:32:20.559 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:20.559+0000 7ff685400700 1 -- 172.21.15.18:0/3795015052 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7ff66406ec00 msgr2=0x7ff6640710b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:32:20.559 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:20.559+0000 7ff685400700 1 --2- 172.21.15.18:0/3795015052 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7ff66406ec00 0x7ff6640710b0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7ff66800b740 tx=0x7ff668004d10 comp rx=0 tx=0).stop 2024-04-12T21:32:20.559 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:20.559+0000 7ff685400700 1 -- 172.21.15.18:0/3795015052 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff680109430 msgr2=0x7ff68011a180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:32:20.559 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:20.559+0000 7ff685400700 1 --2- 172.21.15.18:0/3795015052 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff680109430 0x7ff68011a180 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7ff670002a50 tx=0x7ff6700046b0 comp rx=0 tx=0).stop 2024-04-12T21:32:20.559 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:20.560+0000 7ff685400700 1 -- 172.21.15.18:0/3795015052 shutdown_connections 2024-04-12T21:32:20.559 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:20.560+0000 7ff685400700 1 --2- 172.21.15.18:0/3795015052 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7ff66406ec00 0x7ff6640710b0 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:32:20.559 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:20.560+0000 7ff685400700 1 --2- 172.21.15.18:0/3795015052 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff680109430 0x7ff68011a180 unknown :-1 s=CLOSED pgs=831 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:32:20.559 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:20.560+0000 7ff685400700 1 --2- 172.21.15.18:0/3795015052 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff680109dd0 0x7ff68011a6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:32:20.559 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:20.560+0000 7ff685400700 1 -- 172.21.15.18:0/3795015052 >> 172.21.15.18:0/3795015052 conn(0x7ff680072c00 msgr2=0x7ff680074c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:32:20.560 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:20.560+0000 7ff685400700 1 -- 172.21.15.18:0/3795015052 shutdown_connections 2024-04-12T21:32:20.560 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:20.560+0000 7ff685400700 1 -- 172.21.15.18:0/3795015052 wait complete. 2024-04-12T21:32:20.719 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:20.720+0000 7f211ac00700 1 -- 172.21.15.18:0/1667877453 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f21141037c0 msgr2=0x7f21140fce30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:32:20.720 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:20.720+0000 7f211ac00700 1 --2- 172.21.15.18:0/1667877453 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f21141037c0 0x7f21140fce30 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7f2100004940 tx=0x7f2100009e30 comp rx=0 tx=0).stop 2024-04-12T21:32:20.720 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:20.720+0000 7f211ac00700 1 -- 172.21.15.18:0/1667877453 shutdown_connections 2024-04-12T21:32:20.720 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:20.720+0000 7f211ac00700 1 --2- 172.21.15.18:0/1667877453 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f21141037c0 0x7f21140fce30 unknown :-1 s=CLOSED pgs=832 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:32:20.720 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:20.720+0000 7f211ac00700 1 --2- 172.21.15.18:0/1667877453 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2114102e20 0x7f21141031f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:32:20.720 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:20.720+0000 7f211ac00700 1 -- 172.21.15.18:0/1667877453 >> 172.21.15.18:0/1667877453 conn(0x7f21140f8970 msgr2=0x7f21140fad80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:32:20.720 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:20.720+0000 7f211ac00700 1 -- 172.21.15.18:0/1667877453 shutdown_connections 2024-04-12T21:32:20.720 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:20.720+0000 7f211ac00700 1 -- 172.21.15.18:0/1667877453 wait complete. 2024-04-12T21:32:20.721 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:20.721+0000 7f211ac00700 1 Processor -- start 2024-04-12T21:32:20.721 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:20.721+0000 7f211ac00700 1 -- start start 2024-04-12T21:32:20.721 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:20.721+0000 7f211ac00700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2114102e20 0x7f2114115e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:32:20.721 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:20.721+0000 7f211ac00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f21141037c0 0x7f21141163a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:32:20.722 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:20.721+0000 7f211ac00700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21141169f0 con 0x7f21141037c0 2024-04-12T21:32:20.722 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:20.721+0000 7f211ac00700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f211410fdc0 con 0x7f2114102e20 2024-04-12T21:32:20.722 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:20.723+0000 7f2113400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f21141037c0 0x7f21141163a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:32:20.722 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:20.723+0000 7f2113400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f21141037c0 0x7f21141163a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:45682/0 (socket says 172.21.15.18:45682) 2024-04-12T21:32:20.722 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:20.723+0000 7f2113400700 1 -- 172.21.15.18:0/3746684119 learned_addr learned my addr 172.21.15.18:0/3746684119 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:32:20.722 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:20.723+0000 7f2113400700 1 -- 172.21.15.18:0/3746684119 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2114102e20 msgr2=0x7f2114115e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:32:20.722 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:20.723+0000 7f2113400700 1 --2- 172.21.15.18:0/3746684119 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2114102e20 0x7f2114115e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:32:20.722 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:20.723+0000 7f2113400700 1 -- 172.21.15.18:0/3746684119 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2100005040 con 0x7f21141037c0 2024-04-12T21:32:20.723 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:20.723+0000 7f2113400700 1 --2- 172.21.15.18:0/3746684119 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f21141037c0 0x7f21141163a0 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7f2100009100 tx=0x7f2100009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:32:20.723 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:20.723+0000 7f210be00700 1 -- 172.21.15.18:0/3746684119 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f210000e040 con 0x7f21141037c0 2024-04-12T21:32:20.723 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:20.723+0000 7f210be00700 1 -- 172.21.15.18:0/3746684119 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f2100013420 con 0x7f21141037c0 2024-04-12T21:32:20.723 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:20.723+0000 7f210be00700 1 -- 172.21.15.18:0/3746684119 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f210001c630 con 0x7f21141037c0 2024-04-12T21:32:20.723 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:20.723+0000 7f211ac00700 1 -- 172.21.15.18:0/3746684119 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2114110040 con 0x7f21141037c0 2024-04-12T21:32:20.723 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:20.723+0000 7f211ac00700 1 -- 172.21.15.18:0/3746684119 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2114110580 con 0x7f21141037c0 2024-04-12T21:32:20.724 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:20.724+0000 7f211ac00700 1 -- 172.21.15.18:0/3746684119 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2114102140 con 0x7f21141037c0 2024-04-12T21:32:20.724 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:20.725+0000 7f210be00700 1 -- 172.21.15.18:0/3746684119 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7f2100013590 con 0x7f21141037c0 2024-04-12T21:32:20.726 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:20.727+0000 7f210be00700 1 --2- 172.21.15.18:0/3746684119 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f20f806ec10 0x7f20f80710c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:32:20.726 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:20.727+0000 7f210be00700 1 -- 172.21.15.18:0/3746684119 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 6757+0+0 (secure 0 0 0) 0x7f210008fba0 con 0x7f21141037c0 2024-04-12T21:32:20.726 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:20.727+0000 7f2113e00700 1 --2- 172.21.15.18:0/3746684119 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f20f806ec10 0x7f20f80710c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:32:20.727 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:20.728+0000 7f2113e00700 1 --2- 172.21.15.18:0/3746684119 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f20f806ec10 0x7f20f80710c0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f2104004d00 tx=0x7f2104009ce0 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:32:20.730 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:20.732+0000 7f210be00700 1 -- 172.21.15.18:0/3746684119 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f210005c3d0 con 0x7f21141037c0 2024-04-12T21:32:20.853 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:32:20 smithi018 ceph-mon[112590]: from='client.35716 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:32:20.853 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:32:20 smithi018 ceph-mon[112590]: pgmap v977: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:32:20.853 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:32:20 smithi018 ceph-mon[112590]: from='client.? 172.21.15.18:0/3445444388' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-12T21:32:21.058 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:21.060+0000 7f211ac00700 1 -- 172.21.15.18:0/3746684119 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f21140fe570 con 0x7f21141037c0 2024-04-12T21:32:21.059 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:21.061+0000 7f210be00700 1 -- 172.21.15.18:0/3746684119 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+4303 (secure 0 0 0) 0x7f2100058020 con 0x7f21141037c0 2024-04-12T21:32:21.060 INFO:teuthology.orchestra.run.smithi018.stdout:HEALTH_ERR failed to probe daemons or devices; 1 filesystem is degraded; 1 filesystem is offline 2024-04-12T21:32:21.060 INFO:teuthology.orchestra.run.smithi018.stdout:[WRN] CEPHADM_REFRESH_FAILED: failed to probe daemons or devices 2024-04-12T21:32:21.060 INFO:teuthology.orchestra.run.smithi018.stdout: host smithi018 `cephadm ceph-volume` failed: cephadm exited with an error code: 1, stderr: Inferring config /var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/mon.smithi018/config 2024-04-12T21:32:21.060 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/podman: timeout after 895 seconds 2024-04-12T21:32:21.060 INFO:teuthology.orchestra.run.smithi018.stdout:Non-zero exit code 124 from /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:f4452def3be4d350f4a8e5feaf7ebb1511be9298fe68478ba7a2d6fbd7d69a63 -e NODE_NAME=smithi018 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968:/var/run/ceph:z -v /var/log/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968:/var/log/ceph:z -v /var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmpxsmwkkuh:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:f4452def3be4d350f4a8e5feaf7ebb1511be9298fe68478ba7a2d6fbd7d69a63 inventory --format=json-pretty --filter-for-batch 2024-04-12T21:32:21.060 INFO:teuthology.orchestra.run.smithi018.stdout:Traceback (most recent call last): 2024-04-12T21:32:21.060 INFO:teuthology.orchestra.run.smithi018.stdout: File "/usr/lib64/python3.9/runpy.py", line 197, in _run_module_as_main 2024-04-12T21:32:21.060 INFO:teuthology.orchestra.run.smithi018.stdout: return _run_code(code, main_globals, None, 2024-04-12T21:32:21.060 INFO:teuthology.orchestra.run.smithi018.stdout: File "/usr/lib64/python3.9/runpy.py", line 87, in _run_code 2024-04-12T21:32:21.060 INFO:teuthology.orchestra.run.smithi018.stdout: exec(code, run_globals) 2024-04-12T21:32:21.060 INFO:teuthology.orchestra.run.smithi018.stdout: File "/var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/cephadm.e44f3ed5e34e4ef52eedef233d1190bff004470d8e8005127b5312bff357fa7f/__main__.py", line 5618, in 2024-04-12T21:32:21.061 INFO:teuthology.orchestra.run.smithi018.stdout: File "/var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/cephadm.e44f3ed5e34e4ef52eedef233d1190bff004470d8e8005127b5312bff357fa7f/__main__.py", line 5606, in main 2024-04-12T21:32:21.061 INFO:teuthology.orchestra.run.smithi018.stdout: File "/var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/cephadm.e44f3ed5e34e4ef52eedef233d1190bff004470d8e8005127b5312bff357fa7f/__main__.py", line 408, in _infer_config 2024-04-12T21:32:21.061 INFO:teuthology.orchestra.run.smithi018.stdout: File "/var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/cephadm.e44f3ed5e34e4ef52eedef233d1190bff004470d8e8005127b5312bff357fa7f/__main__.py", line 323, in _infer_fsid 2024-04-12T21:32:21.061 INFO:teuthology.orchestra.run.smithi018.stdout: File "/var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/cephadm.e44f3ed5e34e4ef52eedef233d1190bff004470d8e8005127b5312bff357fa7f/__main__.py", line 436, in _infer_image 2024-04-12T21:32:21.061 INFO:teuthology.orchestra.run.smithi018.stdout: File "/var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/cephadm.e44f3ed5e34e4ef52eedef233d1190bff004470d8e8005127b5312bff357fa7f/__main__.py", line 310, in _validate_fsid 2024-04-12T21:32:21.061 INFO:teuthology.orchestra.run.smithi018.stdout: File "/var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/cephadm.e44f3ed5e34e4ef52eedef233d1190bff004470d8e8005127b5312bff357fa7f/__main__.py", line 3359, in command_ceph_volume 2024-04-12T21:32:21.061 INFO:teuthology.orchestra.run.smithi018.stdout: File "/var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/cephadm.e44f3ed5e34e4ef52eedef233d1190bff004470d8e8005127b5312bff357fa7f/cephadmlib/call_wrappers.py", line 310, in call_throws 2024-04-12T21:32:21.061 INFO:teuthology.orchestra.run.smithi018.stdout:RuntimeError: Failed command: /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:f4452def3be4d350f4a8e5feaf7ebb1511be9298fe68478ba7a2d6fbd7d69a63 -e NODE_NAME=smithi018 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968:/var/run/ceph:z -v /var/log/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968:/var/log/ceph:z -v /var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmpxsmwkkuh:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:f4452def3be4d350f4a8e5feaf7ebb1511be9298fe68478ba7a2d6fbd7d69a63 inventory --format=json-pretty --filter-for-batch 2024-04-12T21:32:21.061 INFO:teuthology.orchestra.run.smithi018.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-04-12T21:32:21.061 INFO:teuthology.orchestra.run.smithi018.stdout: fs cephfs is degraded 2024-04-12T21:32:21.061 INFO:teuthology.orchestra.run.smithi018.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-04-12T21:32:21.061 INFO:teuthology.orchestra.run.smithi018.stdout: fs cephfs is offline because no MDS is active for it. 2024-04-12T21:32:21.062 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:21.063+0000 7f211ac00700 1 -- 172.21.15.18:0/3746684119 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f20f806ec10 msgr2=0x7f20f80710c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:32:21.062 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:21.063+0000 7f211ac00700 1 --2- 172.21.15.18:0/3746684119 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f20f806ec10 0x7f20f80710c0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f2104004d00 tx=0x7f2104009ce0 comp rx=0 tx=0).stop 2024-04-12T21:32:21.062 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:21.063+0000 7f211ac00700 1 -- 172.21.15.18:0/3746684119 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f21141037c0 msgr2=0x7f21141163a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:32:21.062 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:21.063+0000 7f211ac00700 1 --2- 172.21.15.18:0/3746684119 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f21141037c0 0x7f21141163a0 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7f2100009100 tx=0x7f2100009e30 comp rx=0 tx=0).stop 2024-04-12T21:32:21.062 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:21.063+0000 7f211ac00700 1 -- 172.21.15.18:0/3746684119 shutdown_connections 2024-04-12T21:32:21.062 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:21.064+0000 7f211ac00700 1 --2- 172.21.15.18:0/3746684119 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f20f806ec10 0x7f20f80710c0 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:32:21.063 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:21.064+0000 7f211ac00700 1 --2- 172.21.15.18:0/3746684119 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2114102e20 0x7f2114115e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:32:21.063 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:21.064+0000 7f211ac00700 1 --2- 172.21.15.18:0/3746684119 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f21141037c0 0x7f21141163a0 unknown :-1 s=CLOSED pgs=833 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:32:21.063 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:21.064+0000 7f211ac00700 1 -- 172.21.15.18:0/3746684119 >> 172.21.15.18:0/3746684119 conn(0x7f21140f8970 msgr2=0x7f2114070c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:32:21.063 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:21.064+0000 7f211ac00700 1 -- 172.21.15.18:0/3746684119 shutdown_connections 2024-04-12T21:32:21.063 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:21.064+0000 7f211ac00700 1 -- 172.21.15.18:0/3746684119 wait complete. 2024-04-12T21:32:21.282 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:32:20 smithi186 ceph-mon[86379]: from='client.35716 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:32:21.282 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:32:20 smithi186 ceph-mon[86379]: pgmap v977: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:32:21.282 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:32:20 smithi186 ceph-mon[86379]: from='client.? 172.21.15.18:0/3445444388' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-12T21:32:22.103 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:32:21 smithi018 ceph-mon[112590]: from='client.35728 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:32:22.103 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:32:21 smithi018 ceph-mon[112590]: from='client.? 172.21.15.18:0/3746684119' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-12T21:32:22.282 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:32:21 smithi186 ceph-mon[86379]: from='client.35728 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:32:22.282 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:32:21 smithi186 ceph-mon[86379]: from='client.? 172.21.15.18:0/3746684119' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-12T21:32:23.102 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:32:22 smithi018 ceph-mon[112590]: pgmap v978: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:32:23.282 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:32:22 smithi186 ceph-mon[86379]: pgmap v978: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:32:25.102 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:32:24 smithi018 ceph-mon[112590]: pgmap v979: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:32:25.282 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:32:24 smithi186 ceph-mon[86379]: pgmap v979: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:32:26.102 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:32:25 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T21:32:26.282 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:32:25 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T21:32:27.102 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:32:26 smithi018 ceph-mon[112590]: pgmap v980: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:32:27.282 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:32:26 smithi186 ceph-mon[86379]: pgmap v980: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:32:29.103 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:32:28 smithi018 ceph-mon[112590]: pgmap v981: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:32:29.282 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:32:28 smithi186 ceph-mon[86379]: pgmap v981: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:32:31.103 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:32:30 smithi018 ceph-mon[112590]: pgmap v982: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:32:31.282 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:32:30 smithi186 ceph-mon[86379]: pgmap v982: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:32:33.103 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:32:32 smithi018 ceph-mon[112590]: pgmap v983: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:32:33.282 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:32:32 smithi186 ceph-mon[86379]: pgmap v983: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:32:35.103 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:32:34 smithi018 ceph-mon[112590]: pgmap v984: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:32:35.282 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:32:34 smithi186 ceph-mon[86379]: pgmap v984: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:32:37.282 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:32:36 smithi186 ceph-mon[86379]: pgmap v985: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:32:37.353 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:32:36 smithi018 ceph-mon[112590]: pgmap v985: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:32:39.282 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:32:38 smithi186 ceph-mon[86379]: pgmap v986: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:32:39.353 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:32:38 smithi018 ceph-mon[112590]: pgmap v986: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:32:41.282 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:32:40 smithi186 ceph-mon[86379]: pgmap v987: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:32:41.282 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:32:40 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T21:32:41.353 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:32:40 smithi018 ceph-mon[112590]: pgmap v987: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:32:41.353 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:32:40 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T21:32:43.282 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:32:42 smithi186 ceph-mon[86379]: pgmap v988: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:32:43.353 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:32:42 smithi018 ceph-mon[112590]: pgmap v988: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:32:45.156 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:32:44 smithi018 ceph-mon[112590]: pgmap v989: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:32:45.282 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:32:44 smithi186 ceph-mon[86379]: pgmap v989: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:32:47.282 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:32:46 smithi186 ceph-mon[86379]: pgmap v990: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:32:47.353 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:32:46 smithi018 ceph-mon[112590]: pgmap v990: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:32:48.282 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:32:47 smithi186 ceph-mon[86379]: pgmap v991: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:32:48.353 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:32:47 smithi018 ceph-mon[112590]: pgmap v991: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:32:51.032 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:32:50 smithi186 ceph-mon[86379]: pgmap v992: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:32:51.103 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:32:50 smithi018 ceph-mon[112590]: pgmap v992: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:32:51.234 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:51.235+0000 7f934be00700 1 -- 172.21.15.18:0/2541885288 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f934c104e70 msgr2=0x7f934c105240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:32:51.235 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:51.235+0000 7f934be00700 1 --2- 172.21.15.18:0/2541885288 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f934c104e70 0x7f934c105240 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7f933c004940 tx=0x7f933c009e30 comp rx=0 tx=0).stop 2024-04-12T21:32:51.235 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:51.235+0000 7f934be00700 1 -- 172.21.15.18:0/2541885288 shutdown_connections 2024-04-12T21:32:51.235 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:51.235+0000 7f934be00700 1 --2- 172.21.15.18:0/2541885288 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f934c105810 0x7f934c0feed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:32:51.235 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:51.235+0000 7f934be00700 1 --2- 172.21.15.18:0/2541885288 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f934c104e70 0x7f934c105240 unknown :-1 s=CLOSED pgs=834 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:32:51.235 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:51.235+0000 7f934be00700 1 -- 172.21.15.18:0/2541885288 >> 172.21.15.18:0/2541885288 conn(0x7f934c0fab10 msgr2=0x7f934c0fcef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:32:51.235 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:51.235+0000 7f934be00700 1 -- 172.21.15.18:0/2541885288 shutdown_connections 2024-04-12T21:32:51.235 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:51.236+0000 7f934be00700 1 -- 172.21.15.18:0/2541885288 wait complete. 2024-04-12T21:32:51.235 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:51.236+0000 7f934be00700 1 Processor -- start 2024-04-12T21:32:51.236 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:51.236+0000 7f934be00700 1 -- start start 2024-04-12T21:32:51.236 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:51.237+0000 7f934be00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f934c104e70 0x7f934c115bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:32:51.237 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:51.237+0000 7f934be00700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f934c105810 0x7f934c1160f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:32:51.237 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:51.237+0000 7f934be00700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f934c1167d0 con 0x7f934c104e70 2024-04-12T21:32:51.237 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:51.237+0000 7f934be00700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f934c10fb30 con 0x7f934c105810 2024-04-12T21:32:51.237 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:51.237+0000 7f934aa00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f934c104e70 0x7f934c115bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:32:51.237 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:51.237+0000 7f934aa00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f934c104e70 0x7f934c115bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:46092/0 (socket says 172.21.15.18:46092) 2024-04-12T21:32:51.237 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:51.237+0000 7f934aa00700 1 -- 172.21.15.18:0/1048880183 learned_addr learned my addr 172.21.15.18:0/1048880183 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:32:51.237 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:51.238+0000 7f934a000700 1 --2- 172.21.15.18:0/1048880183 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f934c105810 0x7f934c1160f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:32:51.237 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:51.238+0000 7f934aa00700 1 -- 172.21.15.18:0/1048880183 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f934c105810 msgr2=0x7f934c1160f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:32:51.237 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:51.238+0000 7f934aa00700 1 --2- 172.21.15.18:0/1048880183 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f934c105810 0x7f934c1160f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:32:51.237 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:51.238+0000 7f934aa00700 1 -- 172.21.15.18:0/1048880183 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f933c005040 con 0x7f934c104e70 2024-04-12T21:32:51.237 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:51.238+0000 7f934aa00700 1 --2- 172.21.15.18:0/1048880183 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f934c104e70 0x7f934c115bb0 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7f933c009150 tx=0x7f933c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:32:51.238 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:51.238+0000 7f9342a00700 1 -- 172.21.15.18:0/1048880183 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f933c00e040 con 0x7f934c104e70 2024-04-12T21:32:51.238 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:51.238+0000 7f9342a00700 1 -- 172.21.15.18:0/1048880183 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f933c013420 con 0x7f934c104e70 2024-04-12T21:32:51.238 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:51.238+0000 7f934be00700 1 -- 172.21.15.18:0/1048880183 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f934c10fdb0 con 0x7f934c104e70 2024-04-12T21:32:51.238 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:51.239+0000 7f9342a00700 1 -- 172.21.15.18:0/1048880183 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f933c01c630 con 0x7f934c104e70 2024-04-12T21:32:51.238 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:51.239+0000 7f934be00700 1 -- 172.21.15.18:0/1048880183 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f934c110370 con 0x7f934c104e70 2024-04-12T21:32:51.239 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:51.240+0000 7f934be00700 1 -- 172.21.15.18:0/1048880183 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f934c1005c0 con 0x7f934c104e70 2024-04-12T21:32:51.239 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:51.241+0000 7f9342a00700 1 -- 172.21.15.18:0/1048880183 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7f933c013590 con 0x7f934c104e70 2024-04-12T21:32:51.241 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:51.242+0000 7f9342a00700 1 --2- 172.21.15.18:0/1048880183 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f933006ec00 0x7f93300710b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:32:51.241 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:51.243+0000 7f9342a00700 1 -- 172.21.15.18:0/1048880183 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 6757+0+0 (secure 0 0 0) 0x7f933c08fc00 con 0x7f934c104e70 2024-04-12T21:32:51.241 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:51.243+0000 7f934a000700 1 --2- 172.21.15.18:0/1048880183 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f933006ec00 0x7f93300710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:32:51.242 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:51.243+0000 7f934a000700 1 --2- 172.21.15.18:0/1048880183 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f933006ec00 0x7f93300710b0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f9338006fd0 tx=0x7f9338008040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:32:51.246 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:51.248+0000 7f9342a00700 1 -- 172.21.15.18:0/1048880183 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f933c05c430 con 0x7f934c104e70 2024-04-12T21:32:51.498 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:51.499+0000 7f934be00700 1 -- 172.21.15.18:0/1048880183 --> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f934c10bfe0 con 0x7f933006ec00 2024-04-12T21:32:51.500 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:51.501+0000 7f9342a00700 1 -- 172.21.15.18:0/1048880183 <== mgr.34102 v2:172.21.15.18:6800/3812891919 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f934c10bfe0 con 0x7f933006ec00 2024-04-12T21:32:51.503 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:51.505+0000 7f934be00700 1 -- 172.21.15.18:0/1048880183 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f933006ec00 msgr2=0x7f93300710b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:32:51.504 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:51.505+0000 7f934be00700 1 --2- 172.21.15.18:0/1048880183 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f933006ec00 0x7f93300710b0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f9338006fd0 tx=0x7f9338008040 comp rx=0 tx=0).stop 2024-04-12T21:32:51.504 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:51.505+0000 7f934be00700 1 -- 172.21.15.18:0/1048880183 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f934c104e70 msgr2=0x7f934c115bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:32:51.504 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:51.505+0000 7f934be00700 1 --2- 172.21.15.18:0/1048880183 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f934c104e70 0x7f934c115bb0 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7f933c009150 tx=0x7f933c009e30 comp rx=0 tx=0).stop 2024-04-12T21:32:51.504 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:51.505+0000 7f934be00700 1 -- 172.21.15.18:0/1048880183 shutdown_connections 2024-04-12T21:32:51.504 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:51.505+0000 7f934be00700 1 --2- 172.21.15.18:0/1048880183 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f933006ec00 0x7f93300710b0 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:32:51.504 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:51.505+0000 7f934be00700 1 --2- 172.21.15.18:0/1048880183 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f934c104e70 0x7f934c115bb0 unknown :-1 s=CLOSED pgs=835 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:32:51.504 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:51.505+0000 7f934be00700 1 --2- 172.21.15.18:0/1048880183 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f934c105810 0x7f934c1160f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:32:51.504 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:51.505+0000 7f934be00700 1 -- 172.21.15.18:0/1048880183 >> 172.21.15.18:0/1048880183 conn(0x7f934c0fab10 msgr2=0x7f934c0fca10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:32:51.504 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:51.505+0000 7f934be00700 1 -- 172.21.15.18:0/1048880183 shutdown_connections 2024-04-12T21:32:51.504 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:51.505+0000 7f934be00700 1 -- 172.21.15.18:0/1048880183 wait complete. 2024-04-12T21:32:51.520 INFO:teuthology.orchestra.run.smithi018.stdout:true 2024-04-12T21:32:51.667 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:51.668+0000 7fb8bd600700 1 -- 172.21.15.18:0/155431682 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb8b8109dd0 msgr2=0x7fb8b8103440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:32:51.667 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:51.668+0000 7fb8bd600700 1 --2- 172.21.15.18:0/155431682 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb8b8109dd0 0x7fb8b8103440 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7fb8a4004940 tx=0x7fb8a4009e30 comp rx=0 tx=0).stop 2024-04-12T21:32:51.667 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:51.668+0000 7fb8bd600700 1 -- 172.21.15.18:0/155431682 shutdown_connections 2024-04-12T21:32:51.668 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:51.668+0000 7fb8bd600700 1 --2- 172.21.15.18:0/155431682 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb8b8109dd0 0x7fb8b8103440 unknown :-1 s=CLOSED pgs=836 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:32:51.668 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:51.668+0000 7fb8bd600700 1 --2- 172.21.15.18:0/155431682 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb8b8109430 0x7fb8b8109800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:32:51.668 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:51.668+0000 7fb8bd600700 1 -- 172.21.15.18:0/155431682 >> 172.21.15.18:0/155431682 conn(0x7fb8b8072c00 msgr2=0x7fb8b8075010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:32:51.668 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:51.669+0000 7fb8bd600700 1 -- 172.21.15.18:0/155431682 shutdown_connections 2024-04-12T21:32:51.668 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:51.669+0000 7fb8bd600700 1 -- 172.21.15.18:0/155431682 wait complete. 2024-04-12T21:32:51.669 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:51.670+0000 7fb8bd600700 1 Processor -- start 2024-04-12T21:32:51.669 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:51.670+0000 7fb8bd600700 1 -- start start 2024-04-12T21:32:51.669 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:51.670+0000 7fb8bd600700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb8b8109430 0x7fb8b8119f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:32:51.670 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:51.670+0000 7fb8bd600700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb8b8109dd0 0x7fb8b811a450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:32:51.670 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:51.670+0000 7fb8bd600700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8b811aaa0 con 0x7fb8b8109430 2024-04-12T21:32:51.670 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:51.670+0000 7fb8bd600700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8b8114080 con 0x7fb8b8109dd0 2024-04-12T21:32:51.670 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:51.670+0000 7fb8b6a00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb8b8109430 0x7fb8b8119f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:32:51.670 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:51.671+0000 7fb8b6a00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb8b8109430 0x7fb8b8119f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:46118/0 (socket says 172.21.15.18:46118) 2024-04-12T21:32:51.670 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:51.671+0000 7fb8b6a00700 1 -- 172.21.15.18:0/1271394013 learned_addr learned my addr 172.21.15.18:0/1271394013 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:32:51.670 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:51.671+0000 7fb8b6000700 1 --2- 172.21.15.18:0/1271394013 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb8b8109dd0 0x7fb8b811a450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:32:51.670 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:51.671+0000 7fb8b6a00700 1 -- 172.21.15.18:0/1271394013 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb8b8109dd0 msgr2=0x7fb8b811a450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:32:51.670 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:51.671+0000 7fb8b6a00700 1 --2- 172.21.15.18:0/1271394013 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb8b8109dd0 0x7fb8b811a450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:32:51.670 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:51.671+0000 7fb8b6a00700 1 -- 172.21.15.18:0/1271394013 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8a4005040 con 0x7fb8b8109430 2024-04-12T21:32:51.670 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:51.671+0000 7fb8b6000700 1 --2- 172.21.15.18:0/1271394013 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb8b8109dd0 0x7fb8b811a450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-12T21:32:51.671 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:51.671+0000 7fb8b6a00700 1 --2- 172.21.15.18:0/1271394013 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb8b8109430 0x7fb8b8119f10 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7fb8a800eb40 tx=0x7fb8a800b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:32:51.671 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:51.672+0000 7fb8aea00700 1 -- 172.21.15.18:0/1271394013 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb8a8007cc0 con 0x7fb8b8109430 2024-04-12T21:32:51.671 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:51.672+0000 7fb8aea00700 1 -- 172.21.15.18:0/1271394013 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb8a800f040 con 0x7fb8b8109430 2024-04-12T21:32:51.671 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:51.672+0000 7fb8aea00700 1 -- 172.21.15.18:0/1271394013 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb8a8009620 con 0x7fb8b8109430 2024-04-12T21:32:51.671 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:51.672+0000 7fb8bd600700 1 -- 172.21.15.18:0/1271394013 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb8b8114300 con 0x7fb8b8109430 2024-04-12T21:32:51.671 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:51.672+0000 7fb8bd600700 1 -- 172.21.15.18:0/1271394013 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb8b81148c0 con 0x7fb8b8109430 2024-04-12T21:32:51.672 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:51.673+0000 7fb8bd600700 1 -- 172.21.15.18:0/1271394013 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb8b8104b80 con 0x7fb8b8109430 2024-04-12T21:32:51.672 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:51.674+0000 7fb8aea00700 1 -- 172.21.15.18:0/1271394013 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7fb8a8009780 con 0x7fb8b8109430 2024-04-12T21:32:51.674 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:51.675+0000 7fb8aea00700 1 --2- 172.21.15.18:0/1271394013 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fb89c06ec00 0x7fb89c0710b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:32:51.674 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:51.675+0000 7fb8aea00700 1 -- 172.21.15.18:0/1271394013 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 6757+0+0 (secure 0 0 0) 0x7fb8a808e650 con 0x7fb8b8109430 2024-04-12T21:32:51.674 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:51.675+0000 7fb8b6000700 1 --2- 172.21.15.18:0/1271394013 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fb89c06ec00 0x7fb89c0710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:32:51.675 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:51.676+0000 7fb8b6000700 1 --2- 172.21.15.18:0/1271394013 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fb89c06ec00 0x7fb89c0710b0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7fb8a4000c00 tx=0x7fb8a4009e30 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:32:51.679 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:51.680+0000 7fb8aea00700 1 -- 172.21.15.18:0/1271394013 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7fb8a805aed0 con 0x7fb8b8109430 2024-04-12T21:32:51.929 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:51.930+0000 7fb8bd600700 1 -- 172.21.15.18:0/1271394013 --> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fb8b8115600 con 0x7fb89c06ec00 2024-04-12T21:32:51.930 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:51.932+0000 7fb8aea00700 1 -- 172.21.15.18:0/1271394013 <== mgr.34102 v2:172.21.15.18:6800/3812891919 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7fb8b8115600 con 0x7fb89c06ec00 2024-04-12T21:32:51.934 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:51.935+0000 7fb8bd600700 1 -- 172.21.15.18:0/1271394013 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fb89c06ec00 msgr2=0x7fb89c0710b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:32:51.934 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:51.935+0000 7fb8bd600700 1 --2- 172.21.15.18:0/1271394013 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fb89c06ec00 0x7fb89c0710b0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7fb8a4000c00 tx=0x7fb8a4009e30 comp rx=0 tx=0).stop 2024-04-12T21:32:51.934 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:51.935+0000 7fb8bd600700 1 -- 172.21.15.18:0/1271394013 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb8b8109430 msgr2=0x7fb8b8119f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:32:51.934 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:51.935+0000 7fb8bd600700 1 --2- 172.21.15.18:0/1271394013 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb8b8109430 0x7fb8b8119f10 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7fb8a800eb40 tx=0x7fb8a800b040 comp rx=0 tx=0).stop 2024-04-12T21:32:51.934 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:51.935+0000 7fb8bd600700 1 -- 172.21.15.18:0/1271394013 shutdown_connections 2024-04-12T21:32:51.934 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:51.935+0000 7fb8bd600700 1 --2- 172.21.15.18:0/1271394013 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fb89c06ec00 0x7fb89c0710b0 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:32:51.934 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:51.935+0000 7fb8bd600700 1 --2- 172.21.15.18:0/1271394013 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb8b8109430 0x7fb8b8119f10 unknown :-1 s=CLOSED pgs=837 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:32:51.934 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:51.935+0000 7fb8bd600700 1 --2- 172.21.15.18:0/1271394013 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb8b8109dd0 0x7fb8b811a450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:32:51.934 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:51.935+0000 7fb8bd600700 1 -- 172.21.15.18:0/1271394013 >> 172.21.15.18:0/1271394013 conn(0x7fb8b8072c00 msgr2=0x7fb8b81073a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:32:51.935 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:51.936+0000 7fb8bd600700 1 -- 172.21.15.18:0/1271394013 shutdown_connections 2024-04-12T21:32:51.935 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:51.936+0000 7fb8bd600700 1 -- 172.21.15.18:0/1271394013 wait complete. 2024-04-12T21:32:52.094 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:52.095+0000 7f5ad0000700 1 -- 172.21.15.18:0/673378739 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5ac81059c0 msgr2=0x7f5ac80ff080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:32:52.094 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:52.095+0000 7f5ad0000700 1 --2- 172.21.15.18:0/673378739 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5ac81059c0 0x7f5ac80ff080 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7f5ab4004940 tx=0x7f5ab4009e30 comp rx=0 tx=0).stop 2024-04-12T21:32:52.094 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:52.095+0000 7f5ad0000700 1 -- 172.21.15.18:0/673378739 shutdown_connections 2024-04-12T21:32:52.094 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:52.095+0000 7f5ad0000700 1 --2- 172.21.15.18:0/673378739 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5ac81059c0 0x7f5ac80ff080 unknown :-1 s=CLOSED pgs=838 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:32:52.094 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:52.095+0000 7f5ad0000700 1 --2- 172.21.15.18:0/673378739 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5ac8105020 0x7f5ac81053f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:32:52.094 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:52.095+0000 7f5ad0000700 1 -- 172.21.15.18:0/673378739 >> 172.21.15.18:0/673378739 conn(0x7f5ac80faba0 msgr2=0x7f5ac80fcfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:32:52.094 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:52.095+0000 7f5ad0000700 1 -- 172.21.15.18:0/673378739 shutdown_connections 2024-04-12T21:32:52.094 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:52.095+0000 7f5ad0000700 1 -- 172.21.15.18:0/673378739 wait complete. 2024-04-12T21:32:52.095 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:52.096+0000 7f5ad0000700 1 Processor -- start 2024-04-12T21:32:52.095 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:52.096+0000 7f5ad0000700 1 -- start start 2024-04-12T21:32:52.095 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:52.096+0000 7f5ad0000700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5ac8105020 0x7f5ac8115b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:32:52.095 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:52.097+0000 7f5ad0000700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5ac81059c0 0x7f5ac8116040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:32:52.095 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:52.097+0000 7f5ad0000700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ac8116690 con 0x7f5ac81059c0 2024-04-12T21:32:52.096 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:52.097+0000 7f5ad0000700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ac810fc70 con 0x7f5ac8105020 2024-04-12T21:32:52.096 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:52.097+0000 7f5accc00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5ac81059c0 0x7f5ac8116040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:32:52.096 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:52.097+0000 7f5accc00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5ac81059c0 0x7f5ac8116040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:46138/0 (socket says 172.21.15.18:46138) 2024-04-12T21:32:52.096 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:52.097+0000 7f5accc00700 1 -- 172.21.15.18:0/2664881797 learned_addr learned my addr 172.21.15.18:0/2664881797 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:32:52.097 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:52.097+0000 7f5acd600700 1 --2- 172.21.15.18:0/2664881797 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5ac8105020 0x7f5ac8115b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:32:52.097 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:52.097+0000 7f5accc00700 1 -- 172.21.15.18:0/2664881797 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5ac8105020 msgr2=0x7f5ac8115b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:32:52.097 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:52.097+0000 7f5accc00700 1 --2- 172.21.15.18:0/2664881797 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5ac8105020 0x7f5ac8115b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:32:52.097 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:52.097+0000 7f5accc00700 1 -- 172.21.15.18:0/2664881797 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ab4005040 con 0x7f5ac81059c0 2024-04-12T21:32:52.097 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:52.098+0000 7f5accc00700 1 --2- 172.21.15.18:0/2664881797 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5ac81059c0 0x7f5ac8116040 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7f5ab4009100 tx=0x7f5ab4009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:32:52.097 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:52.098+0000 7f5ac5600700 1 -- 172.21.15.18:0/2664881797 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5ab400e040 con 0x7f5ac81059c0 2024-04-12T21:32:52.097 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:52.098+0000 7f5ac5600700 1 -- 172.21.15.18:0/2664881797 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f5ab4013420 con 0x7f5ac81059c0 2024-04-12T21:32:52.097 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:52.098+0000 7f5ac5600700 1 -- 172.21.15.18:0/2664881797 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5ab401c630 con 0x7f5ac81059c0 2024-04-12T21:32:52.097 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:52.098+0000 7f5ad0000700 1 -- 172.21.15.18:0/2664881797 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5ac810fef0 con 0x7f5ac81059c0 2024-04-12T21:32:52.097 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:52.098+0000 7f5ad0000700 1 -- 172.21.15.18:0/2664881797 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5ac81104b0 con 0x7f5ac81059c0 2024-04-12T21:32:52.098 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:52.099+0000 7f5ad0000700 1 -- 172.21.15.18:0/2664881797 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5ac8100770 con 0x7f5ac81059c0 2024-04-12T21:32:52.098 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:52.100+0000 7f5ac5600700 1 -- 172.21.15.18:0/2664881797 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7f5ab4013590 con 0x7f5ac81059c0 2024-04-12T21:32:52.100 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:52.102+0000 7f5ac5600700 1 --2- 172.21.15.18:0/2664881797 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f5ab006ec00 0x7f5ab00710b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:32:52.100 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:52.102+0000 7f5ac5600700 1 -- 172.21.15.18:0/2664881797 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 6757+0+0 (secure 0 0 0) 0x7f5ab408fc60 con 0x7f5ac81059c0 2024-04-12T21:32:52.101 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:52.102+0000 7f5acd600700 1 --2- 172.21.15.18:0/2664881797 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f5ab006ec00 0x7f5ab00710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:32:52.101 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:52.103+0000 7f5acd600700 1 --2- 172.21.15.18:0/2664881797 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f5ab006ec00 0x7f5ab00710b0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f5abc009b80 tx=0x7f5abc00b040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:32:52.105 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:52.107+0000 7f5ac5600700 1 -- 172.21.15.18:0/2664881797 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f5ab405c410 con 0x7f5ac81059c0 2024-04-12T21:32:52.337 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:52.338+0000 7f5ad0000700 1 -- 172.21.15.18:0/2664881797 --> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f5ac8100770 con 0x7f5ab006ec00 2024-04-12T21:32:52.356 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:52.358+0000 7f5ac5600700 1 -- 172.21.15.18:0/2664881797 <== mgr.34102 v2:172.21.15.18:6800/3812891919 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7f5ac8100770 con 0x7f5ab006ec00 2024-04-12T21:32:52.357 INFO:teuthology.orchestra.run.smithi018.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-12T21:32:52.357 INFO:teuthology.orchestra.run.smithi018.stdout:alertmanager.smithi018 smithi018 *:9093,9094 running (33m) 9m ago 47m 15.5M - 0.25.0 c8568f914cd2 c332090068e8 2024-04-12T21:32:52.357 INFO:teuthology.orchestra.run.smithi018.stdout:crash.smithi018 smithi018 running (29m) 9m ago 48m 6665k - 19.0.0-2879-g324baf8d 8717720dc5e5 b7f80d1f7982 2024-04-12T21:32:52.357 INFO:teuthology.orchestra.run.smithi018.stdout:crash.smithi186 smithi186 running (29m) 9m ago 46m 6660k - 19.0.0-2879-g324baf8d 8717720dc5e5 a4f55c81b623 2024-04-12T21:32:52.358 INFO:teuthology.orchestra.run.smithi018.stdout:grafana.smithi018 smithi018 *:3000 running (32m) 9m ago 47m 74.8M - 9.4.12 2bacad6d85d8 b08e3bf176dd 2024-04-12T21:32:52.358 INFO:teuthology.orchestra.run.smithi018.stdout:mds.cephfs.smithi018.acqbsc smithi018 running (25m) 9m ago 40m 16.7M - 19.0.0-2879-g324baf8d 8717720dc5e5 021f99ab24ae 2024-04-12T21:32:52.358 INFO:teuthology.orchestra.run.smithi018.stdout:mds.cephfs.smithi018.szjluo smithi018 running (25m) 9m ago 40m 16.5M - 19.0.0-2879-g324baf8d 8717720dc5e5 0003117479d4 2024-04-12T21:32:52.358 INFO:teuthology.orchestra.run.smithi018.stdout:mds.cephfs.smithi186.jdllyn smithi186 running (9m) 9m ago 40m 11.0M - 19.0.0-2879-g324baf8d 8717720dc5e5 21b1f46eef23 2024-04-12T21:32:52.358 INFO:teuthology.orchestra.run.smithi018.stdout:mds.cephfs.smithi186.wjnafs smithi186 running (40m) 9m ago 40m 19.5M - 17.2.7-827-gd3b5060a 21de9b7369ad 74b115d6d6fd 2024-04-12T21:32:52.358 INFO:teuthology.orchestra.run.smithi018.stdout:mgr.smithi018.xjzwga smithi018 *:8443,9283,8765 running (36m) 9m ago 49m 512M - 19.0.0-2879-g324baf8d 8717720dc5e5 56088305eab7 2024-04-12T21:32:52.358 INFO:teuthology.orchestra.run.smithi018.stdout:mgr.smithi186.kqcfxk smithi186 *:8443,9283,8765 running (34m) 9m ago 46m 459M - 19.0.0-2879-g324baf8d 8717720dc5e5 09cd766e5e0d 2024-04-12T21:32:52.358 INFO:teuthology.orchestra.run.smithi018.stdout:mon.smithi018 smithi018 running (31m) 9m ago 49m 73.1M 2048M 19.0.0-2879-g324baf8d 8717720dc5e5 df3e201dfed6 2024-04-12T21:32:52.358 INFO:teuthology.orchestra.run.smithi018.stdout:mon.smithi186 smithi186 running (30m) 9m ago 46m 47.2M 2048M 19.0.0-2879-g324baf8d 8717720dc5e5 c77498da9b2b 2024-04-12T21:32:52.358 INFO:teuthology.orchestra.run.smithi018.stdout:node-exporter.smithi018 smithi018 *:9100 running (34m) 9m ago 48m 8108k - 1.5.0 0da6a335fe13 326c3d9c6847 2024-04-12T21:32:52.358 INFO:teuthology.orchestra.run.smithi018.stdout:node-exporter.smithi186 smithi186 *:9100 running (34m) 9m ago 46m 8757k - 1.5.0 0da6a335fe13 ec2112baeecf 2024-04-12T21:32:52.358 INFO:teuthology.orchestra.run.smithi018.stdout:osd.0 smithi018 running (29m) 9m ago 45m 123M 1559M 19.0.0-2879-g324baf8d 8717720dc5e5 60b1273ca047 2024-04-12T21:32:52.358 INFO:teuthology.orchestra.run.smithi018.stdout:osd.1 smithi018 running (28m) 9m ago 44m 95.1M 1559M 19.0.0-2879-g324baf8d 8717720dc5e5 170ce1e5fb8d 2024-04-12T21:32:52.358 INFO:teuthology.orchestra.run.smithi018.stdout:osd.2 smithi018 running (28m) 9m ago 44m 94.3M 1559M 19.0.0-2879-g324baf8d 8717720dc5e5 c2cbc6aa2839 2024-04-12T21:32:52.358 INFO:teuthology.orchestra.run.smithi018.stdout:osd.3 smithi186 running (27m) 9m ago 43m 101M 2581M 19.0.0-2879-g324baf8d 8717720dc5e5 0138e8262e4b 2024-04-12T21:32:52.358 INFO:teuthology.orchestra.run.smithi018.stdout:osd.4 smithi186 running (26m) 9m ago 43m 91.0M 2581M 19.0.0-2879-g324baf8d 8717720dc5e5 b20cb63b25c9 2024-04-12T21:32:52.359 INFO:teuthology.orchestra.run.smithi018.stdout:osd.5 smithi186 running (26m) 9m ago 42m 92.0M 2581M 19.0.0-2879-g324baf8d 8717720dc5e5 2513bea2610e 2024-04-12T21:32:52.359 INFO:teuthology.orchestra.run.smithi018.stdout:prometheus.smithi018 smithi018 *:9095 running (33m) 9m ago 47m 67.9M - 2.43.0 a07b618ecd1d f58841127e21 2024-04-12T21:32:52.361 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:52.362+0000 7f5ad0000700 1 -- 172.21.15.18:0/2664881797 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f5ab006ec00 msgr2=0x7f5ab00710b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:32:52.361 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:52.362+0000 7f5ad0000700 1 --2- 172.21.15.18:0/2664881797 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f5ab006ec00 0x7f5ab00710b0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f5abc009b80 tx=0x7f5abc00b040 comp rx=0 tx=0).stop 2024-04-12T21:32:52.361 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:52.362+0000 7f5ad0000700 1 -- 172.21.15.18:0/2664881797 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5ac81059c0 msgr2=0x7f5ac8116040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:32:52.361 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:52.362+0000 7f5ad0000700 1 --2- 172.21.15.18:0/2664881797 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5ac81059c0 0x7f5ac8116040 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7f5ab4009100 tx=0x7f5ab4009e30 comp rx=0 tx=0).stop 2024-04-12T21:32:52.361 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:52.362+0000 7f5ad0000700 1 -- 172.21.15.18:0/2664881797 shutdown_connections 2024-04-12T21:32:52.361 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:52.362+0000 7f5ad0000700 1 --2- 172.21.15.18:0/2664881797 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f5ab006ec00 0x7f5ab00710b0 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:32:52.361 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:52.362+0000 7f5ad0000700 1 --2- 172.21.15.18:0/2664881797 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5ac8105020 0x7f5ac8115b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:32:52.361 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:52.362+0000 7f5ad0000700 1 --2- 172.21.15.18:0/2664881797 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5ac81059c0 0x7f5ac8116040 unknown :-1 s=CLOSED pgs=839 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:32:52.361 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:52.362+0000 7f5ad0000700 1 -- 172.21.15.18:0/2664881797 >> 172.21.15.18:0/2664881797 conn(0x7f5ac80faba0 msgr2=0x7f5ac80fba60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:32:52.361 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:52.362+0000 7f5ad0000700 1 -- 172.21.15.18:0/2664881797 shutdown_connections 2024-04-12T21:32:52.361 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:52.362+0000 7f5ad0000700 1 -- 172.21.15.18:0/2664881797 wait complete. 2024-04-12T21:32:52.510 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:52.511+0000 7fe11ec00700 1 -- 172.21.15.18:0/3235547646 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe118109dd0 msgr2=0x7fe118103440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:32:52.510 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:52.511+0000 7fe11ec00700 1 --2- 172.21.15.18:0/3235547646 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe118109dd0 0x7fe118103440 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7fe100004940 tx=0x7fe100009e30 comp rx=0 tx=0).stop 2024-04-12T21:32:52.511 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:52.512+0000 7fe11ec00700 1 -- 172.21.15.18:0/3235547646 shutdown_connections 2024-04-12T21:32:52.511 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:52.512+0000 7fe11ec00700 1 --2- 172.21.15.18:0/3235547646 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe118109dd0 0x7fe118103440 unknown :-1 s=CLOSED pgs=840 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:32:52.511 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:52.512+0000 7fe11ec00700 1 --2- 172.21.15.18:0/3235547646 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe118109430 0x7fe118109800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:32:52.511 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:52.512+0000 7fe11ec00700 1 -- 172.21.15.18:0/3235547646 >> 172.21.15.18:0/3235547646 conn(0x7fe118072c00 msgr2=0x7fe118075010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:32:52.511 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:52.512+0000 7fe11ec00700 1 -- 172.21.15.18:0/3235547646 shutdown_connections 2024-04-12T21:32:52.511 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:52.512+0000 7fe11ec00700 1 -- 172.21.15.18:0/3235547646 wait complete. 2024-04-12T21:32:52.513 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:52.512+0000 7fe11ec00700 1 Processor -- start 2024-04-12T21:32:52.513 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:52.513+0000 7fe11ec00700 1 -- start start 2024-04-12T21:32:52.513 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:52.513+0000 7fe11ec00700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe118109430 0x7fe118119fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:32:52.513 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:52.513+0000 7fe11ec00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe118109dd0 0x7fe11811a4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:32:52.513 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:52.513+0000 7fe11ec00700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe11811abc0 con 0x7fe118109dd0 2024-04-12T21:32:52.513 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:52.513+0000 7fe11ec00700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe118114110 con 0x7fe118109430 2024-04-12T21:32:52.513 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:52.513+0000 7fe10fe00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe118109dd0 0x7fe11811a4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:32:52.513 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:52.513+0000 7fe10fe00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe118109dd0 0x7fe11811a4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:46150/0 (socket says 172.21.15.18:46150) 2024-04-12T21:32:52.513 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:52.513+0000 7fe10fe00700 1 -- 172.21.15.18:0/3099948818 learned_addr learned my addr 172.21.15.18:0/3099948818 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:32:52.513 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:52.513+0000 7fe117e00700 1 --2- 172.21.15.18:0/3099948818 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe118109430 0x7fe118119fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:32:52.513 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:52.514+0000 7fe10fe00700 1 -- 172.21.15.18:0/3099948818 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe118109430 msgr2=0x7fe118119fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:32:52.514 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:52.514+0000 7fe10fe00700 1 --2- 172.21.15.18:0/3099948818 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe118109430 0x7fe118119fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:32:52.514 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:52.514+0000 7fe10fe00700 1 -- 172.21.15.18:0/3099948818 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe100005040 con 0x7fe118109dd0 2024-04-12T21:32:52.514 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:52.514+0000 7fe10fe00700 1 --2- 172.21.15.18:0/3099948818 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe118109dd0 0x7fe11811a4e0 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7fe100009100 tx=0x7fe100009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:32:52.514 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:52.514+0000 7fe114c00700 1 -- 172.21.15.18:0/3099948818 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe10000e040 con 0x7fe118109dd0 2024-04-12T21:32:52.514 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:52.514+0000 7fe114c00700 1 -- 172.21.15.18:0/3099948818 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe100013420 con 0x7fe118109dd0 2024-04-12T21:32:52.514 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:52.514+0000 7fe114c00700 1 -- 172.21.15.18:0/3099948818 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe10001c630 con 0x7fe118109dd0 2024-04-12T21:32:52.514 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:52.514+0000 7fe11ec00700 1 -- 172.21.15.18:0/3099948818 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe118114330 con 0x7fe118109dd0 2024-04-12T21:32:52.514 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:52.514+0000 7fe11ec00700 1 -- 172.21.15.18:0/3099948818 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe1181148f0 con 0x7fe118109dd0 2024-04-12T21:32:52.514 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:52.516+0000 7fe11ec00700 1 -- 172.21.15.18:0/3099948818 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe118104b80 con 0x7fe118109dd0 2024-04-12T21:32:52.515 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:52.517+0000 7fe114c00700 1 -- 172.21.15.18:0/3099948818 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7fe100013590 con 0x7fe118109dd0 2024-04-12T21:32:52.516 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:52.517+0000 7fe114c00700 1 --2- 172.21.15.18:0/3099948818 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fe0fc06ec00 0x7fe0fc0710b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:32:52.516 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:52.518+0000 7fe114c00700 1 -- 172.21.15.18:0/3099948818 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 6757+0+0 (secure 0 0 0) 0x7fe10008fba0 con 0x7fe118109dd0 2024-04-12T21:32:52.516 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:52.518+0000 7fe117e00700 1 --2- 172.21.15.18:0/3099948818 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fe0fc06ec00 0x7fe0fc0710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:32:52.517 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:52.518+0000 7fe117e00700 1 --2- 172.21.15.18:0/3099948818 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fe0fc06ec00 0x7fe0fc0710b0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7fe108006fd0 tx=0x7fe108008040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:32:52.520 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:52.522+0000 7fe114c00700 1 -- 172.21.15.18:0/3099948818 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7fe10005c3d0 con 0x7fe118109dd0 2024-04-12T21:32:52.831 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:52.833+0000 7fe11ec00700 1 -- 172.21.15.18:0/3099948818 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fe118110580 con 0x7fe118109dd0 2024-04-12T21:32:52.833 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:52.834+0000 7fe114c00700 1 -- 172.21.15.18:0/3099948818 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+837 (secure 0 0 0) 0x7fe100058030 con 0x7fe118109dd0 2024-04-12T21:32:52.833 INFO:teuthology.orchestra.run.smithi018.stdout:{ 2024-04-12T21:32:52.833 INFO:teuthology.orchestra.run.smithi018.stdout: "mon": { 2024-04-12T21:32:52.833 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 19.0.0-2879-g324baf8d (324baf8ddb91dea46cac50cb535c306115791355) squid (dev)": 2 2024-04-12T21:32:52.833 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-12T21:32:52.833 INFO:teuthology.orchestra.run.smithi018.stdout: "mgr": { 2024-04-12T21:32:52.833 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 19.0.0-2879-g324baf8d (324baf8ddb91dea46cac50cb535c306115791355) squid (dev)": 2 2024-04-12T21:32:52.834 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-12T21:32:52.834 INFO:teuthology.orchestra.run.smithi018.stdout: "osd": { 2024-04-12T21:32:52.834 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 19.0.0-2879-g324baf8d (324baf8ddb91dea46cac50cb535c306115791355) squid (dev)": 6 2024-04-12T21:32:52.834 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-12T21:32:52.834 INFO:teuthology.orchestra.run.smithi018.stdout: "mds": { 2024-04-12T21:32:52.834 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 17.2.7-827-gd3b5060a (d3b5060a17449edfae9385cdf792afec109fca5d) quincy (stable)": 1, 2024-04-12T21:32:52.834 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 19.0.0-2879-g324baf8d (324baf8ddb91dea46cac50cb535c306115791355) squid (dev)": 3 2024-04-12T21:32:52.834 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-12T21:32:52.834 INFO:teuthology.orchestra.run.smithi018.stdout: "overall": { 2024-04-12T21:32:52.834 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 17.2.7-827-gd3b5060a (d3b5060a17449edfae9385cdf792afec109fca5d) quincy (stable)": 1, 2024-04-12T21:32:52.834 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 19.0.0-2879-g324baf8d (324baf8ddb91dea46cac50cb535c306115791355) squid (dev)": 13 2024-04-12T21:32:52.834 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-04-12T21:32:52.835 INFO:teuthology.orchestra.run.smithi018.stdout:} 2024-04-12T21:32:52.837 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:52.838+0000 7fe11ec00700 1 -- 172.21.15.18:0/3099948818 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fe0fc06ec00 msgr2=0x7fe0fc0710b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:32:52.837 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:52.838+0000 7fe11ec00700 1 --2- 172.21.15.18:0/3099948818 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fe0fc06ec00 0x7fe0fc0710b0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7fe108006fd0 tx=0x7fe108008040 comp rx=0 tx=0).stop 2024-04-12T21:32:52.837 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:52.838+0000 7fe11ec00700 1 -- 172.21.15.18:0/3099948818 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe118109dd0 msgr2=0x7fe11811a4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:32:52.837 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:52.838+0000 7fe11ec00700 1 --2- 172.21.15.18:0/3099948818 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe118109dd0 0x7fe11811a4e0 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7fe100009100 tx=0x7fe100009e30 comp rx=0 tx=0).stop 2024-04-12T21:32:52.837 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:52.838+0000 7fe11ec00700 1 -- 172.21.15.18:0/3099948818 shutdown_connections 2024-04-12T21:32:52.837 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:52.838+0000 7fe11ec00700 1 --2- 172.21.15.18:0/3099948818 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fe0fc06ec00 0x7fe0fc0710b0 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:32:52.837 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:52.838+0000 7fe11ec00700 1 --2- 172.21.15.18:0/3099948818 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe118109430 0x7fe118119fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:32:52.837 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:52.838+0000 7fe11ec00700 1 --2- 172.21.15.18:0/3099948818 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe118109dd0 0x7fe11811a4e0 unknown :-1 s=CLOSED pgs=841 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:32:52.837 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:52.838+0000 7fe11ec00700 1 -- 172.21.15.18:0/3099948818 >> 172.21.15.18:0/3099948818 conn(0x7fe118072c00 msgr2=0x7fe118074b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:32:52.838 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:52.839+0000 7fe11ec00700 1 -- 172.21.15.18:0/3099948818 shutdown_connections 2024-04-12T21:32:52.838 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:52.839+0000 7fe11ec00700 1 -- 172.21.15.18:0/3099948818 wait complete. 2024-04-12T21:32:52.984 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:52.985+0000 7f3074800700 1 -- 172.21.15.18:0/1749676657 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f306c0fbc10 msgr2=0x7f306c0fbfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:32:52.984 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:52.985+0000 7f3074800700 1 --2- 172.21.15.18:0/1749676657 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f306c0fbc10 0x7f306c0fbfe0 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7f3060004940 tx=0x7f3060009e30 comp rx=0 tx=0).stop 2024-04-12T21:32:52.984 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:52.985+0000 7f3074800700 1 -- 172.21.15.18:0/1749676657 shutdown_connections 2024-04-12T21:32:52.984 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:52.985+0000 7f3074800700 1 --2- 172.21.15.18:0/1749676657 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f306c0fc5b0 0x7f306c100bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:32:52.984 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:52.985+0000 7f3074800700 1 --2- 172.21.15.18:0/1749676657 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f306c0fbc10 0x7f306c0fbfe0 unknown :-1 s=CLOSED pgs=842 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:32:52.984 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:52.985+0000 7f3074800700 1 -- 172.21.15.18:0/1749676657 >> 172.21.15.18:0/1749676657 conn(0x7f306c0f7870 msgr2=0x7f306c0f9c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:32:52.985 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:52.985+0000 7f3074800700 1 -- 172.21.15.18:0/1749676657 shutdown_connections 2024-04-12T21:32:52.985 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:52.985+0000 7f3074800700 1 -- 172.21.15.18:0/1749676657 wait complete. 2024-04-12T21:32:52.986 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:52.986+0000 7f3074800700 1 Processor -- start 2024-04-12T21:32:52.986 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:52.986+0000 7f3074800700 1 -- start start 2024-04-12T21:32:52.986 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:52.987+0000 7f3074800700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f306c0fbc10 0x7f306c11a040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:32:52.986 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:52.987+0000 7f3074800700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f306c0fc5b0 0x7f306c11a580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:32:52.986 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:52.987+0000 7f3074800700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f306c11abd0 con 0x7f306c0fc5b0 2024-04-12T21:32:52.987 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:52.987+0000 7f3074800700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f306c1141b0 con 0x7f306c0fbc10 2024-04-12T21:32:52.987 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:52.987+0000 7f3071400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f306c0fc5b0 0x7f306c11a580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:32:52.987 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:52.987+0000 7f3071400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f306c0fc5b0 0x7f306c11a580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:46166/0 (socket says 172.21.15.18:46166) 2024-04-12T21:32:52.987 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:52.987+0000 7f3071400700 1 -- 172.21.15.18:0/2897311373 learned_addr learned my addr 172.21.15.18:0/2897311373 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:32:52.987 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:52.987+0000 7f3071e00700 1 --2- 172.21.15.18:0/2897311373 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f306c0fbc10 0x7f306c11a040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:32:52.987 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:52.988+0000 7f3071400700 1 -- 172.21.15.18:0/2897311373 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f306c0fbc10 msgr2=0x7f306c11a040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:32:52.987 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:52.988+0000 7f3071400700 1 --2- 172.21.15.18:0/2897311373 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f306c0fbc10 0x7f306c11a040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:32:52.987 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:52.988+0000 7f3071400700 1 -- 172.21.15.18:0/2897311373 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3060005040 con 0x7f306c0fc5b0 2024-04-12T21:32:52.987 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:52.988+0000 7f3071400700 1 --2- 172.21.15.18:0/2897311373 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f306c0fc5b0 0x7f306c11a580 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7f305c002a50 tx=0x7f305c0046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:32:52.988 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:52.988+0000 7f3066000700 1 -- 172.21.15.18:0/2897311373 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f305c007cc0 con 0x7f306c0fc5b0 2024-04-12T21:32:52.988 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:52.988+0000 7f3066000700 1 -- 172.21.15.18:0/2897311373 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f305c00f040 con 0x7f306c0fc5b0 2024-04-12T21:32:52.988 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:52.988+0000 7f3066000700 1 -- 172.21.15.18:0/2897311373 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f305c00da30 con 0x7f306c0fc5b0 2024-04-12T21:32:52.988 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:52.988+0000 7f3074800700 1 -- 172.21.15.18:0/2897311373 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f306c114490 con 0x7f306c0fc5b0 2024-04-12T21:32:52.988 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:52.988+0000 7f3074800700 1 -- 172.21.15.18:0/2897311373 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f306c1149d0 con 0x7f306c0fc5b0 2024-04-12T21:32:52.988 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:52.989+0000 7f3074800700 1 -- 172.21.15.18:0/2897311373 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f306c066b50 con 0x7f306c0fc5b0 2024-04-12T21:32:52.989 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:52.990+0000 7f3066000700 1 -- 172.21.15.18:0/2897311373 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7f305c00db90 con 0x7f306c0fc5b0 2024-04-12T21:32:52.990 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:52.991+0000 7f3066000700 1 --2- 172.21.15.18:0/2897311373 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f305406ec00 0x7f30540710b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:32:52.990 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:52.992+0000 7f3066000700 1 -- 172.21.15.18:0/2897311373 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 6757+0+0 (secure 0 0 0) 0x7f305c08e700 con 0x7f306c0fc5b0 2024-04-12T21:32:52.990 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:52.992+0000 7f3071e00700 1 --2- 172.21.15.18:0/2897311373 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f305406ec00 0x7f30540710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:32:52.991 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:52.992+0000 7f3071e00700 1 --2- 172.21.15.18:0/2897311373 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f305406ec00 0x7f30540710b0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f306000b3b0 tx=0x7f3060009e30 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:32:52.995 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:52.996+0000 7f3066000700 1 -- 172.21.15.18:0/2897311373 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f305c05aeb0 con 0x7f306c0fc5b0 2024-04-12T21:32:52.995 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:32:52 smithi018 ceph-mon[112590]: from='client.35736 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:32:52.995 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:32:52 smithi018 ceph-mon[112590]: pgmap v993: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:32:53.032 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:32:52 smithi186 ceph-mon[86379]: from='client.35736 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:32:53.032 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:32:52 smithi186 ceph-mon[86379]: pgmap v993: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:32:53.263 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:53.264+0000 7f3074800700 1 -- 172.21.15.18:0/2897311373 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f306c110520 con 0x7f306c0fc5b0 2024-04-12T21:32:53.263 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:53.265+0000 7f3066000700 1 -- 172.21.15.18:0/2897311373 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 31 v31) v1 ==== 76+0+1873 (secure 0 0 0) 0x7f305c05aa40 con 0x7f306c0fc5b0 2024-04-12T21:32:53.264 INFO:teuthology.orchestra.run.smithi018.stdout:e31 2024-04-12T21:32:53.264 INFO:teuthology.orchestra.run.smithi018.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-12T21:32:53.264 INFO:teuthology.orchestra.run.smithi018.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-12T21:32:53.264 INFO:teuthology.orchestra.run.smithi018.stdout:legacy client fscid: 1 2024-04-12T21:32:53.264 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-12T21:32:53.264 INFO:teuthology.orchestra.run.smithi018.stdout:Filesystem 'cephfs' (1) 2024-04-12T21:32:53.264 INFO:teuthology.orchestra.run.smithi018.stdout:fs_name cephfs 2024-04-12T21:32:53.264 INFO:teuthology.orchestra.run.smithi018.stdout:epoch 24 2024-04-12T21:32:53.264 INFO:teuthology.orchestra.run.smithi018.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-04-12T21:32:53.264 INFO:teuthology.orchestra.run.smithi018.stdout:created 2024-04-12T20:51:57.362850+0000 2024-04-12T21:32:53.265 INFO:teuthology.orchestra.run.smithi018.stdout:modified 2024-04-12T21:06:59.717190+0000 2024-04-12T21:32:53.265 INFO:teuthology.orchestra.run.smithi018.stdout:tableserver 0 2024-04-12T21:32:53.265 INFO:teuthology.orchestra.run.smithi018.stdout:root 0 2024-04-12T21:32:53.265 INFO:teuthology.orchestra.run.smithi018.stdout:session_timeout 60 2024-04-12T21:32:53.265 INFO:teuthology.orchestra.run.smithi018.stdout:session_autoclose 300 2024-04-12T21:32:53.265 INFO:teuthology.orchestra.run.smithi018.stdout:max_file_size 1099511627776 2024-04-12T21:32:53.265 INFO:teuthology.orchestra.run.smithi018.stdout:max_xattr_size 65536 2024-04-12T21:32:53.265 INFO:teuthology.orchestra.run.smithi018.stdout:required_client_features {} 2024-04-12T21:32:53.265 INFO:teuthology.orchestra.run.smithi018.stdout:last_failure 0 2024-04-12T21:32:53.265 INFO:teuthology.orchestra.run.smithi018.stdout:last_failure_osd_epoch 100 2024-04-12T21:32:53.265 INFO:teuthology.orchestra.run.smithi018.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-12T21:32:53.265 INFO:teuthology.orchestra.run.smithi018.stdout:max_mds 2 2024-04-12T21:32:53.265 INFO:teuthology.orchestra.run.smithi018.stdout:in 0,1 2024-04-12T21:32:53.265 INFO:teuthology.orchestra.run.smithi018.stdout:up {} 2024-04-12T21:32:53.266 INFO:teuthology.orchestra.run.smithi018.stdout:failed 0,1 2024-04-12T21:32:53.266 INFO:teuthology.orchestra.run.smithi018.stdout:damaged 2024-04-12T21:32:53.266 INFO:teuthology.orchestra.run.smithi018.stdout:stopped 2024-04-12T21:32:53.266 INFO:teuthology.orchestra.run.smithi018.stdout:data_pools [3] 2024-04-12T21:32:53.266 INFO:teuthology.orchestra.run.smithi018.stdout:metadata_pool 2 2024-04-12T21:32:53.266 INFO:teuthology.orchestra.run.smithi018.stdout:inline_data disabled 2024-04-12T21:32:53.266 INFO:teuthology.orchestra.run.smithi018.stdout:balancer 2024-04-12T21:32:53.266 INFO:teuthology.orchestra.run.smithi018.stdout:bal_rank_mask -1 2024-04-12T21:32:53.266 INFO:teuthology.orchestra.run.smithi018.stdout:standby_count_wanted 1 2024-04-12T21:32:53.266 INFO:teuthology.orchestra.run.smithi018.stdout:qdb_cluster leader: 0 members: 2024-04-12T21:32:53.266 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-12T21:32:53.266 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-12T21:32:53.266 INFO:teuthology.orchestra.run.smithi018.stdout:Standby daemons: 2024-04-12T21:32:53.266 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-12T21:32:53.267 INFO:teuthology.orchestra.run.smithi018.stdout:[mds.cephfs.smithi186.wjnafs{-1:24277} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.186:6826/438347207,v1:172.21.15.186:6827/438347207] compat {c=[1],r=[1],i=[7ff]}] 2024-04-12T21:32:53.267 INFO:teuthology.orchestra.run.smithi018.stdout:[mds.cephfs.smithi018.acqbsc{-1:34436} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.18:6828/755179764,v1:172.21.15.18:6829/755179764] compat {c=[1],r=[1],i=[1fff]}] 2024-04-12T21:32:53.267 INFO:teuthology.orchestra.run.smithi018.stdout:[mds.cephfs.smithi018.szjluo{-1:34468} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.18:6826/2407992653,v1:172.21.15.18:6827/2407992653] compat {c=[1],r=[1],i=[1fff]}] 2024-04-12T21:32:53.267 INFO:teuthology.orchestra.run.smithi018.stdout:[mds.cephfs.smithi186.jdllyn{-1:44595} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.186:6824/3523444202,v1:172.21.15.186:6825/3523444202] compat {c=[1],r=[1],i=[1fff]}] 2024-04-12T21:32:53.268 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:53.268+0000 7f3074800700 1 -- 172.21.15.18:0/2897311373 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f305406ec00 msgr2=0x7f30540710b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:32:53.268 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:53.268+0000 7f3074800700 1 --2- 172.21.15.18:0/2897311373 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f305406ec00 0x7f30540710b0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f306000b3b0 tx=0x7f3060009e30 comp rx=0 tx=0).stop 2024-04-12T21:32:53.268 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:53.268+0000 7f3074800700 1 -- 172.21.15.18:0/2897311373 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f306c0fc5b0 msgr2=0x7f306c11a580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:32:53.268 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:53.268+0000 7f3074800700 1 --2- 172.21.15.18:0/2897311373 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f306c0fc5b0 0x7f306c11a580 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7f305c002a50 tx=0x7f305c0046b0 comp rx=0 tx=0).stop 2024-04-12T21:32:53.268 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:53.268+0000 7f3074800700 1 -- 172.21.15.18:0/2897311373 shutdown_connections 2024-04-12T21:32:53.268 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:53.268+0000 7f3074800700 1 --2- 172.21.15.18:0/2897311373 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f305406ec00 0x7f30540710b0 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:32:53.268 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:53.268+0000 7f3074800700 1 --2- 172.21.15.18:0/2897311373 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f306c0fbc10 0x7f306c11a040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:32:53.268 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:53.268+0000 7f3074800700 1 --2- 172.21.15.18:0/2897311373 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f306c0fc5b0 0x7f306c11a580 unknown :-1 s=CLOSED pgs=843 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:32:53.268 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:53.268+0000 7f3074800700 1 -- 172.21.15.18:0/2897311373 >> 172.21.15.18:0/2897311373 conn(0x7f306c0f7870 msgr2=0x7f306c0f8e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:32:53.268 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:53.268+0000 7f3074800700 1 -- 172.21.15.18:0/2897311373 shutdown_connections 2024-04-12T21:32:53.268 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:53.268+0000 7f3074800700 1 -- 172.21.15.18:0/2897311373 wait complete. 2024-04-12T21:32:53.269 INFO:teuthology.orchestra.run.smithi018.stderr:dumped fsmap epoch 31 2024-04-12T21:32:53.427 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:53.428+0000 7f6c53c00700 1 -- 172.21.15.18:0/3672066504 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6c4c070c70 msgr2=0x7f6c4c1090f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:32:53.427 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:53.428+0000 7f6c53c00700 1 --2- 172.21.15.18:0/3672066504 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6c4c070c70 0x7f6c4c1090f0 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7f6c38004970 tx=0x7f6c38009e30 comp rx=0 tx=0).stop 2024-04-12T21:32:53.428 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:53.429+0000 7f6c53c00700 1 -- 172.21.15.18:0/3672066504 shutdown_connections 2024-04-12T21:32:53.428 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:53.429+0000 7f6c53c00700 1 --2- 172.21.15.18:0/3672066504 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6c4c070c70 0x7f6c4c1090f0 unknown :-1 s=CLOSED pgs=844 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:32:53.428 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:53.429+0000 7f6c53c00700 1 --2- 172.21.15.18:0/3672066504 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6c4c070360 0x7f6c4c070730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:32:53.428 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:53.429+0000 7f6c53c00700 1 -- 172.21.15.18:0/3672066504 >> 172.21.15.18:0/3672066504 conn(0x7f6c4c0f8970 msgr2=0x7f6c4c0fad80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:32:53.428 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:53.429+0000 7f6c53c00700 1 -- 172.21.15.18:0/3672066504 shutdown_connections 2024-04-12T21:32:53.428 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:53.429+0000 7f6c53c00700 1 -- 172.21.15.18:0/3672066504 wait complete. 2024-04-12T21:32:53.429 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:53.430+0000 7f6c53c00700 1 Processor -- start 2024-04-12T21:32:53.429 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:53.430+0000 7f6c53c00700 1 -- start start 2024-04-12T21:32:53.429 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:53.430+0000 7f6c53c00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6c4c070360 0x7f6c4c115d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:32:53.429 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:53.430+0000 7f6c53c00700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6c4c070c70 0x7f6c4c116290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:32:53.429 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:53.430+0000 7f6c53c00700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c4c1168e0 con 0x7f6c4c070360 2024-04-12T21:32:53.429 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:53.430+0000 7f6c53c00700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c4c10fcb0 con 0x7f6c4c070c70 2024-04-12T21:32:53.429 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:53.431+0000 7f6c51200700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6c4c070360 0x7f6c4c115d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:32:53.429 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:53.431+0000 7f6c51200700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6c4c070360 0x7f6c4c115d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:46184/0 (socket says 172.21.15.18:46184) 2024-04-12T21:32:53.430 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:53.431+0000 7f6c51200700 1 -- 172.21.15.18:0/3543485436 learned_addr learned my addr 172.21.15.18:0/3543485436 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:32:53.431 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:53.431+0000 7f6c51200700 1 -- 172.21.15.18:0/3543485436 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6c4c070c70 msgr2=0x7f6c4c116290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:32:53.431 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:53.431+0000 7f6c4be00700 1 --2- 172.21.15.18:0/3543485436 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6c4c070c70 0x7f6c4c116290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:32:53.431 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:53.431+0000 7f6c51200700 1 --2- 172.21.15.18:0/3543485436 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6c4c070c70 0x7f6c4c116290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:32:53.431 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:53.431+0000 7f6c51200700 1 -- 172.21.15.18:0/3543485436 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c38005040 con 0x7f6c4c070360 2024-04-12T21:32:53.431 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:53.431+0000 7f6c51200700 1 --2- 172.21.15.18:0/3543485436 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6c4c070360 0x7f6c4c115d50 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7f6c40002a50 tx=0x7f6c400046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:32:53.431 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:53.432+0000 7f6c48c00700 1 -- 172.21.15.18:0/3543485436 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6c40007cc0 con 0x7f6c4c070360 2024-04-12T21:32:53.431 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:53.432+0000 7f6c48c00700 1 -- 172.21.15.18:0/3543485436 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f6c4000f040 con 0x7f6c4c070360 2024-04-12T21:32:53.431 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:53.432+0000 7f6c53c00700 1 -- 172.21.15.18:0/3543485436 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6c4c10ff90 con 0x7f6c4c070360 2024-04-12T21:32:53.431 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:53.432+0000 7f6c48c00700 1 -- 172.21.15.18:0/3543485436 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6c40009640 con 0x7f6c4c070360 2024-04-12T21:32:53.431 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:53.432+0000 7f6c53c00700 1 -- 172.21.15.18:0/3543485436 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6c4c1104d0 con 0x7f6c4c070360 2024-04-12T21:32:53.432 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:53.433+0000 7f6c53c00700 1 -- 172.21.15.18:0/3543485436 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6c4c071aa0 con 0x7f6c4c070360 2024-04-12T21:32:53.432 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:53.434+0000 7f6c48c00700 1 -- 172.21.15.18:0/3543485436 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7f6c400097a0 con 0x7f6c4c070360 2024-04-12T21:32:53.434 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:53.435+0000 7f6c48c00700 1 --2- 172.21.15.18:0/3543485436 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f6c3406ec10 0x7f6c340710c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:32:53.434 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:53.435+0000 7f6c48c00700 1 -- 172.21.15.18:0/3543485436 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 6757+0+0 (secure 0 0 0) 0x7f6c4008e6c0 con 0x7f6c4c070360 2024-04-12T21:32:53.434 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:53.435+0000 7f6c4be00700 1 --2- 172.21.15.18:0/3543485436 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f6c3406ec10 0x7f6c340710c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:32:53.434 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:53.436+0000 7f6c4be00700 1 --2- 172.21.15.18:0/3543485436 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f6c3406ec10 0x7f6c340710c0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f6c38000c00 tx=0x7f6c38009e30 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:32:53.438 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:53.440+0000 7f6c48c00700 1 -- 172.21.15.18:0/3543485436 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f6c4005af10 con 0x7f6c4c070360 2024-04-12T21:32:53.696 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:53.697+0000 7f6c53c00700 1 -- 172.21.15.18:0/3543485436 --> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f6c4c04bf70 con 0x7f6c3406ec10 2024-04-12T21:32:53.698 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:53.699+0000 7f6c48c00700 1 -- 172.21.15.18:0/3543485436 <== mgr.34102 v2:172.21.15.18:6800/3812891919 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f6c38009710 con 0x7f6c3406ec10 2024-04-12T21:32:53.699 INFO:teuthology.orchestra.run.smithi018.stdout:{ 2024-04-12T21:32:53.699 INFO:teuthology.orchestra.run.smithi018.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:f4452def3be4d350f4a8e5feaf7ebb1511be9298fe68478ba7a2d6fbd7d69a63", 2024-04-12T21:32:53.699 INFO:teuthology.orchestra.run.smithi018.stdout: "in_progress": true, 2024-04-12T21:32:53.700 INFO:teuthology.orchestra.run.smithi018.stdout: "which": "Upgrading all daemon types on all hosts", 2024-04-12T21:32:53.700 INFO:teuthology.orchestra.run.smithi018.stdout: "services_complete": [ 2024-04-12T21:32:53.700 INFO:teuthology.orchestra.run.smithi018.stdout: "crash", 2024-04-12T21:32:53.700 INFO:teuthology.orchestra.run.smithi018.stdout: "osd", 2024-04-12T21:32:53.701 INFO:teuthology.orchestra.run.smithi018.stdout: "mon", 2024-04-12T21:32:53.701 INFO:teuthology.orchestra.run.smithi018.stdout: "mgr" 2024-04-12T21:32:53.701 INFO:teuthology.orchestra.run.smithi018.stdout: ], 2024-04-12T21:32:53.701 INFO:teuthology.orchestra.run.smithi018.stdout: "progress": "15/21 daemons upgraded", 2024-04-12T21:32:53.701 INFO:teuthology.orchestra.run.smithi018.stdout: "message": "Currently upgrading mds daemons", 2024-04-12T21:32:53.701 INFO:teuthology.orchestra.run.smithi018.stdout: "is_paused": false 2024-04-12T21:32:53.701 INFO:teuthology.orchestra.run.smithi018.stdout:} 2024-04-12T21:32:53.703 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:53.702+0000 7f6c53c00700 1 -- 172.21.15.18:0/3543485436 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f6c3406ec10 msgr2=0x7f6c340710c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:32:53.703 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:53.702+0000 7f6c53c00700 1 --2- 172.21.15.18:0/3543485436 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f6c3406ec10 0x7f6c340710c0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f6c38000c00 tx=0x7f6c38009e30 comp rx=0 tx=0).stop 2024-04-12T21:32:53.703 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:53.702+0000 7f6c53c00700 1 -- 172.21.15.18:0/3543485436 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6c4c070360 msgr2=0x7f6c4c115d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:32:53.703 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:53.702+0000 7f6c53c00700 1 --2- 172.21.15.18:0/3543485436 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6c4c070360 0x7f6c4c115d50 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7f6c40002a50 tx=0x7f6c400046b0 comp rx=0 tx=0).stop 2024-04-12T21:32:53.703 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:53.702+0000 7f6c53c00700 1 -- 172.21.15.18:0/3543485436 shutdown_connections 2024-04-12T21:32:53.704 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:53.702+0000 7f6c53c00700 1 --2- 172.21.15.18:0/3543485436 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f6c3406ec10 0x7f6c340710c0 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:32:53.704 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:53.702+0000 7f6c53c00700 1 --2- 172.21.15.18:0/3543485436 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6c4c070360 0x7f6c4c115d50 unknown :-1 s=CLOSED pgs=845 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:32:53.704 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:53.702+0000 7f6c53c00700 1 --2- 172.21.15.18:0/3543485436 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6c4c070c70 0x7f6c4c116290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:32:53.704 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:53.702+0000 7f6c53c00700 1 -- 172.21.15.18:0/3543485436 >> 172.21.15.18:0/3543485436 conn(0x7f6c4c0f8970 msgr2=0x7f6c4c0f9d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:32:53.705 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:53.702+0000 7f6c53c00700 1 -- 172.21.15.18:0/3543485436 shutdown_connections 2024-04-12T21:32:53.705 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:53.703+0000 7f6c53c00700 1 -- 172.21.15.18:0/3543485436 wait complete. 2024-04-12T21:32:53.858 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:53.859+0000 7f0d11e00700 1 -- 172.21.15.18:0/3658558837 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0d0c102e10 msgr2=0x7f0d0c1031e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:32:53.858 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:53.859+0000 7f0d11e00700 1 --2- 172.21.15.18:0/3658558837 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0d0c102e10 0x7f0d0c1031e0 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7f0cfc004940 tx=0x7f0cfc009e30 comp rx=0 tx=0).stop 2024-04-12T21:32:53.859 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:53.859+0000 7f0d11e00700 1 -- 172.21.15.18:0/3658558837 shutdown_connections 2024-04-12T21:32:53.859 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:53.859+0000 7f0d11e00700 1 --2- 172.21.15.18:0/3658558837 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0d0c1037b0 0x7f0d0c0fce20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:32:53.859 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:53.859+0000 7f0d11e00700 1 --2- 172.21.15.18:0/3658558837 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0d0c102e10 0x7f0d0c1031e0 unknown :-1 s=CLOSED pgs=846 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:32:53.859 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:53.859+0000 7f0d11e00700 1 -- 172.21.15.18:0/3658558837 >> 172.21.15.18:0/3658558837 conn(0x7f0d0c0f89a0 msgr2=0x7f0d0c0fadb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:32:53.859 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:53.859+0000 7f0d11e00700 1 -- 172.21.15.18:0/3658558837 shutdown_connections 2024-04-12T21:32:53.859 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:53.859+0000 7f0d11e00700 1 -- 172.21.15.18:0/3658558837 wait complete. 2024-04-12T21:32:53.859 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:53.860+0000 7f0d11e00700 1 Processor -- start 2024-04-12T21:32:53.861 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:53.860+0000 7f0d11e00700 1 -- start start 2024-04-12T21:32:53.861 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:53.861+0000 7f0d11e00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0d0c102e10 0x7f0d0c115d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:32:53.861 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:53.861+0000 7f0d11e00700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0d0c1037b0 0x7f0d0c1162c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:32:53.861 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:53.861+0000 7f0d11e00700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d0c1169a0 con 0x7f0d0c102e10 2024-04-12T21:32:53.861 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:53.861+0000 7f0d11e00700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d0c10fce0 con 0x7f0d0c1037b0 2024-04-12T21:32:53.861 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:53.861+0000 7f0d0aa00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0d0c102e10 0x7f0d0c115d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:32:53.861 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:53.861+0000 7f0d0aa00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0d0c102e10 0x7f0d0c115d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:46208/0 (socket says 172.21.15.18:46208) 2024-04-12T21:32:53.861 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:53.861+0000 7f0d0aa00700 1 -- 172.21.15.18:0/3294358214 learned_addr learned my addr 172.21.15.18:0/3294358214 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:32:53.861 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:53.861+0000 7f0d0a000700 1 --2- 172.21.15.18:0/3294358214 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0d0c1037b0 0x7f0d0c1162c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:32:53.861 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:53.861+0000 7f0d0aa00700 1 -- 172.21.15.18:0/3294358214 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0d0c1037b0 msgr2=0x7f0d0c1162c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:32:53.861 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:53.861+0000 7f0d0aa00700 1 --2- 172.21.15.18:0/3294358214 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0d0c1037b0 0x7f0d0c1162c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:32:53.862 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:53.861+0000 7f0d0aa00700 1 -- 172.21.15.18:0/3294358214 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0cfc005040 con 0x7f0d0c102e10 2024-04-12T21:32:53.862 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:53.862+0000 7f0d0aa00700 1 --2- 172.21.15.18:0/3294358214 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0d0c102e10 0x7f0d0c115d80 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7f0cfc009180 tx=0x7f0cfc009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:32:53.862 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:53.862+0000 7f0d02a00700 1 -- 172.21.15.18:0/3294358214 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0cfc00e040 con 0x7f0d0c102e10 2024-04-12T21:32:53.862 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:53.862+0000 7f0d11e00700 1 -- 172.21.15.18:0/3294358214 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0d0c10ff00 con 0x7f0d0c102e10 2024-04-12T21:32:53.862 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:53.862+0000 7f0d02a00700 1 -- 172.21.15.18:0/3294358214 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f0cfc00bbc0 con 0x7f0d0c102e10 2024-04-12T21:32:53.862 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:53.862+0000 7f0d02a00700 1 -- 172.21.15.18:0/3294358214 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0cfc01c630 con 0x7f0d0c102e10 2024-04-12T21:32:53.862 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:53.862+0000 7f0d11e00700 1 -- 172.21.15.18:0/3294358214 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0d0c1104c0 con 0x7f0d0c102e10 2024-04-12T21:32:53.863 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:53.863+0000 7f0d11e00700 1 -- 172.21.15.18:0/3294358214 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0d0c0fe560 con 0x7f0d0c102e10 2024-04-12T21:32:53.863 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:53.864+0000 7f0d02a00700 1 -- 172.21.15.18:0/3294358214 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7f0cfc00bd30 con 0x7f0d0c102e10 2024-04-12T21:32:53.864 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:53.865+0000 7f0d02a00700 1 --2- 172.21.15.18:0/3294358214 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f0cf006ec10 0x7f0cf00710c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:32:53.864 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:53.865+0000 7f0d02a00700 1 -- 172.21.15.18:0/3294358214 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 6757+0+0 (secure 0 0 0) 0x7f0cfc08fe30 con 0x7f0d0c102e10 2024-04-12T21:32:53.864 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:53.866+0000 7f0d0a000700 1 --2- 172.21.15.18:0/3294358214 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f0cf006ec10 0x7f0cf00710c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:32:53.864 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:53.866+0000 7f0d0a000700 1 --2- 172.21.15.18:0/3294358214 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f0cf006ec10 0x7f0cf00710c0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f0cf8006fd0 tx=0x7f0cf8008040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:32:53.869 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:53.870+0000 7f0d02a00700 1 -- 172.21.15.18:0/3294358214 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f0cfc05c5e0 con 0x7f0d0c102e10 2024-04-12T21:32:54.032 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:32:53 smithi186 ceph-mon[86379]: from='client.35740 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:32:54.032 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:32:53 smithi186 ceph-mon[86379]: from='client.35744 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:32:54.032 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:32:53 smithi186 ceph-mon[86379]: from='client.? 172.21.15.18:0/3099948818' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T21:32:54.032 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:32:53 smithi186 ceph-mon[86379]: from='client.? 172.21.15.18:0/2897311373' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-12T21:32:54.103 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:32:53 smithi018 ceph-mon[112590]: from='client.35740 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:32:54.103 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:32:53 smithi018 ceph-mon[112590]: from='client.35744 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:32:54.103 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:32:53 smithi018 ceph-mon[112590]: from='client.? 172.21.15.18:0/3099948818' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T21:32:54.103 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:32:53 smithi018 ceph-mon[112590]: from='client.? 172.21.15.18:0/2897311373' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-12T21:32:54.175 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:54.176+0000 7f0d11e00700 1 -- 172.21.15.18:0/3294358214 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f0d0c04bf60 con 0x7f0d0c102e10 2024-04-12T21:32:54.176 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:54.177+0000 7f0d02a00700 1 -- 172.21.15.18:0/3294358214 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+4303 (secure 0 0 0) 0x7f0cfc05c170 con 0x7f0d0c102e10 2024-04-12T21:32:54.176 INFO:teuthology.orchestra.run.smithi018.stdout:HEALTH_ERR failed to probe daemons or devices; 1 filesystem is degraded; 1 filesystem is offline 2024-04-12T21:32:54.177 INFO:teuthology.orchestra.run.smithi018.stdout:[WRN] CEPHADM_REFRESH_FAILED: failed to probe daemons or devices 2024-04-12T21:32:54.177 INFO:teuthology.orchestra.run.smithi018.stdout: host smithi018 `cephadm ceph-volume` failed: cephadm exited with an error code: 1, stderr: Inferring config /var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/mon.smithi018/config 2024-04-12T21:32:54.177 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/podman: timeout after 895 seconds 2024-04-12T21:32:54.177 INFO:teuthology.orchestra.run.smithi018.stdout:Non-zero exit code 124 from /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:f4452def3be4d350f4a8e5feaf7ebb1511be9298fe68478ba7a2d6fbd7d69a63 -e NODE_NAME=smithi018 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968:/var/run/ceph:z -v /var/log/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968:/var/log/ceph:z -v /var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmpxsmwkkuh:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:f4452def3be4d350f4a8e5feaf7ebb1511be9298fe68478ba7a2d6fbd7d69a63 inventory --format=json-pretty --filter-for-batch 2024-04-12T21:32:54.177 INFO:teuthology.orchestra.run.smithi018.stdout:Traceback (most recent call last): 2024-04-12T21:32:54.177 INFO:teuthology.orchestra.run.smithi018.stdout: File "/usr/lib64/python3.9/runpy.py", line 197, in _run_module_as_main 2024-04-12T21:32:54.177 INFO:teuthology.orchestra.run.smithi018.stdout: return _run_code(code, main_globals, None, 2024-04-12T21:32:54.177 INFO:teuthology.orchestra.run.smithi018.stdout: File "/usr/lib64/python3.9/runpy.py", line 87, in _run_code 2024-04-12T21:32:54.177 INFO:teuthology.orchestra.run.smithi018.stdout: exec(code, run_globals) 2024-04-12T21:32:54.177 INFO:teuthology.orchestra.run.smithi018.stdout: File "/var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/cephadm.e44f3ed5e34e4ef52eedef233d1190bff004470d8e8005127b5312bff357fa7f/__main__.py", line 5618, in 2024-04-12T21:32:54.177 INFO:teuthology.orchestra.run.smithi018.stdout: File "/var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/cephadm.e44f3ed5e34e4ef52eedef233d1190bff004470d8e8005127b5312bff357fa7f/__main__.py", line 5606, in main 2024-04-12T21:32:54.177 INFO:teuthology.orchestra.run.smithi018.stdout: File "/var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/cephadm.e44f3ed5e34e4ef52eedef233d1190bff004470d8e8005127b5312bff357fa7f/__main__.py", line 408, in _infer_config 2024-04-12T21:32:54.177 INFO:teuthology.orchestra.run.smithi018.stdout: File "/var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/cephadm.e44f3ed5e34e4ef52eedef233d1190bff004470d8e8005127b5312bff357fa7f/__main__.py", line 323, in _infer_fsid 2024-04-12T21:32:54.177 INFO:teuthology.orchestra.run.smithi018.stdout: File "/var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/cephadm.e44f3ed5e34e4ef52eedef233d1190bff004470d8e8005127b5312bff357fa7f/__main__.py", line 436, in _infer_image 2024-04-12T21:32:54.178 INFO:teuthology.orchestra.run.smithi018.stdout: File "/var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/cephadm.e44f3ed5e34e4ef52eedef233d1190bff004470d8e8005127b5312bff357fa7f/__main__.py", line 310, in _validate_fsid 2024-04-12T21:32:54.178 INFO:teuthology.orchestra.run.smithi018.stdout: File "/var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/cephadm.e44f3ed5e34e4ef52eedef233d1190bff004470d8e8005127b5312bff357fa7f/__main__.py", line 3359, in command_ceph_volume 2024-04-12T21:32:54.178 INFO:teuthology.orchestra.run.smithi018.stdout: File "/var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/cephadm.e44f3ed5e34e4ef52eedef233d1190bff004470d8e8005127b5312bff357fa7f/cephadmlib/call_wrappers.py", line 310, in call_throws 2024-04-12T21:32:54.178 INFO:teuthology.orchestra.run.smithi018.stdout:RuntimeError: Failed command: /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:f4452def3be4d350f4a8e5feaf7ebb1511be9298fe68478ba7a2d6fbd7d69a63 -e NODE_NAME=smithi018 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968:/var/run/ceph:z -v /var/log/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968:/var/log/ceph:z -v /var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmpxsmwkkuh:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:f4452def3be4d350f4a8e5feaf7ebb1511be9298fe68478ba7a2d6fbd7d69a63 inventory --format=json-pretty --filter-for-batch 2024-04-12T21:32:54.178 INFO:teuthology.orchestra.run.smithi018.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-04-12T21:32:54.178 INFO:teuthology.orchestra.run.smithi018.stdout: fs cephfs is degraded 2024-04-12T21:32:54.178 INFO:teuthology.orchestra.run.smithi018.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-04-12T21:32:54.178 INFO:teuthology.orchestra.run.smithi018.stdout: fs cephfs is offline because no MDS is active for it. 2024-04-12T21:32:54.178 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:54.180+0000 7f0d11e00700 1 -- 172.21.15.18:0/3294358214 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f0cf006ec10 msgr2=0x7f0cf00710c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:32:54.179 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:54.180+0000 7f0d11e00700 1 --2- 172.21.15.18:0/3294358214 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f0cf006ec10 0x7f0cf00710c0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f0cf8006fd0 tx=0x7f0cf8008040 comp rx=0 tx=0).stop 2024-04-12T21:32:54.179 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:54.180+0000 7f0d11e00700 1 -- 172.21.15.18:0/3294358214 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0d0c102e10 msgr2=0x7f0d0c115d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:32:54.179 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:54.180+0000 7f0d11e00700 1 --2- 172.21.15.18:0/3294358214 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0d0c102e10 0x7f0d0c115d80 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7f0cfc009180 tx=0x7f0cfc009e30 comp rx=0 tx=0).stop 2024-04-12T21:32:54.179 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:54.180+0000 7f0d11e00700 1 -- 172.21.15.18:0/3294358214 shutdown_connections 2024-04-12T21:32:54.180 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:54.180+0000 7f0d11e00700 1 --2- 172.21.15.18:0/3294358214 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f0cf006ec10 0x7f0cf00710c0 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:32:54.180 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:54.180+0000 7f0d11e00700 1 --2- 172.21.15.18:0/3294358214 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0d0c102e10 0x7f0d0c115d80 unknown :-1 s=CLOSED pgs=847 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:32:54.180 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:54.180+0000 7f0d11e00700 1 --2- 172.21.15.18:0/3294358214 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0d0c1037b0 0x7f0d0c1162c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:32:54.180 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:54.180+0000 7f0d11e00700 1 -- 172.21.15.18:0/3294358214 >> 172.21.15.18:0/3294358214 conn(0x7f0d0c0f89a0 msgr2=0x7f0d0c071c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:32:54.180 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:54.181+0000 7f0d11e00700 1 -- 172.21.15.18:0/3294358214 shutdown_connections 2024-04-12T21:32:54.180 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:32:54.181+0000 7f0d11e00700 1 -- 172.21.15.18:0/3294358214 wait complete. 2024-04-12T21:32:55.032 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:32:54 smithi186 ceph-mon[86379]: pgmap v994: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:32:55.032 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:32:54 smithi186 ceph-mon[86379]: from='client.35756 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:32:55.032 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:32:54 smithi186 ceph-mon[86379]: from='client.? 172.21.15.18:0/3294358214' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-12T21:32:55.103 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:32:54 smithi018 ceph-mon[112590]: pgmap v994: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:32:55.103 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:32:54 smithi018 ceph-mon[112590]: from='client.35756 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:32:55.103 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:32:54 smithi018 ceph-mon[112590]: from='client.? 172.21.15.18:0/3294358214' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-12T21:32:56.032 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:32:55 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T21:32:56.103 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:32:55 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T21:32:57.032 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:32:56 smithi186 ceph-mon[86379]: pgmap v995: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:32:57.103 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:32:56 smithi018 ceph-mon[112590]: pgmap v995: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:32:59.032 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:32:58 smithi186 ceph-mon[86379]: pgmap v996: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:32:59.104 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:32:58 smithi018 ceph-mon[112590]: pgmap v996: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:33:01.103 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:33:00 smithi018 ceph-mon[112590]: pgmap v997: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:33:01.282 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:33:00 smithi186 ceph-mon[86379]: pgmap v997: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:33:03.103 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:33:02 smithi018 ceph-mon[112590]: pgmap v998: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:33:03.282 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:33:02 smithi186 ceph-mon[86379]: pgmap v998: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:33:05.103 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:33:04 smithi018 ceph-mon[112590]: pgmap v999: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:33:05.282 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:33:04 smithi186 ceph-mon[86379]: pgmap v999: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:33:07.103 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:33:06 smithi018 ceph-mon[112590]: pgmap v1000: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:33:07.282 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:33:06 smithi186 ceph-mon[86379]: pgmap v1000: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:33:09.103 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:33:08 smithi018 ceph-mon[112590]: pgmap v1001: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:33:09.282 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:33:08 smithi186 ceph-mon[86379]: pgmap v1001: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:33:11.103 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:33:10 smithi018 ceph-mon[112590]: pgmap v1002: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:33:11.103 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:33:10 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T21:33:11.282 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:33:10 smithi186 ceph-mon[86379]: pgmap v1002: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:33:11.282 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:33:10 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T21:33:13.103 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:33:12 smithi018 ceph-mon[112590]: pgmap v1003: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:33:13.282 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:33:12 smithi186 ceph-mon[86379]: pgmap v1003: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:33:15.085 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:33:14 smithi186 ceph-mon[86379]: pgmap v1004: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:33:15.103 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:33:14 smithi018 ceph-mon[112590]: pgmap v1004: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:33:17.103 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:33:16 smithi018 ceph-mon[112590]: pgmap v1005: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:33:17.282 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:33:16 smithi186 ceph-mon[86379]: pgmap v1005: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:33:19.103 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:33:18 smithi018 ceph-mon[112590]: pgmap v1006: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:33:19.282 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:33:18 smithi186 ceph-mon[86379]: pgmap v1006: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:33:21.282 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:33:20 smithi186 ceph-mon[86379]: pgmap v1007: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:33:21.353 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:33:20 smithi018 ceph-mon[112590]: pgmap v1007: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:33:23.282 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:33:22 smithi186 ceph-mon[86379]: pgmap v1008: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:33:23.353 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:33:22 smithi018 ceph-mon[112590]: pgmap v1008: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:33:24.351 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:24.351+0000 7ff166000700 1 -- 172.21.15.18:0/1911323304 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff160109430 msgr2=0x7ff160109800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:33:24.352 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:24.351+0000 7ff166000700 1 --2- 172.21.15.18:0/1911323304 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff160109430 0x7ff160109800 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7ff150004940 tx=0x7ff150009e30 comp rx=0 tx=0).stop 2024-04-12T21:33:24.352 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:24.351+0000 7ff166000700 1 -- 172.21.15.18:0/1911323304 shutdown_connections 2024-04-12T21:33:24.352 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:24.351+0000 7ff166000700 1 --2- 172.21.15.18:0/1911323304 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff160109dd0 0x7ff160103440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:33:24.352 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:24.351+0000 7ff166000700 1 --2- 172.21.15.18:0/1911323304 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff160109430 0x7ff160109800 unknown :-1 s=CLOSED pgs=848 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:33:24.352 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:24.351+0000 7ff166000700 1 -- 172.21.15.18:0/1911323304 >> 172.21.15.18:0/1911323304 conn(0x7ff160072c00 msgr2=0x7ff160075010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:33:24.352 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:24.351+0000 7ff166000700 1 -- 172.21.15.18:0/1911323304 shutdown_connections 2024-04-12T21:33:24.352 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:24.352+0000 7ff166000700 1 -- 172.21.15.18:0/1911323304 wait complete. 2024-04-12T21:33:24.353 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:24.352+0000 7ff166000700 1 Processor -- start 2024-04-12T21:33:24.354 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:24.353+0000 7ff166000700 1 -- start start 2024-04-12T21:33:24.354 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:24.353+0000 7ff166000700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff160109430 0x7ff160119f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:33:24.354 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:24.353+0000 7ff166000700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff160109dd0 0x7ff16011a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:33:24.354 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:24.353+0000 7ff166000700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff16011aad0 con 0x7ff160109dd0 2024-04-12T21:33:24.354 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:24.353+0000 7ff166000700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff160114100 con 0x7ff160109430 2024-04-12T21:33:24.354 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:24.353+0000 7ff15ea00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff160109dd0 0x7ff16011a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:33:24.354 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:24.353+0000 7ff15ea00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff160109dd0 0x7ff16011a480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:59060/0 (socket says 172.21.15.18:59060) 2024-04-12T21:33:24.354 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:24.353+0000 7ff15ea00700 1 -- 172.21.15.18:0/1440917452 learned_addr learned my addr 172.21.15.18:0/1440917452 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:33:24.354 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:24.354+0000 7ff15f400700 1 --2- 172.21.15.18:0/1440917452 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff160109430 0x7ff160119f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:33:24.354 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:24.354+0000 7ff15ea00700 1 -- 172.21.15.18:0/1440917452 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff160109430 msgr2=0x7ff160119f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:33:24.354 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:24.354+0000 7ff15ea00700 1 --2- 172.21.15.18:0/1440917452 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff160109430 0x7ff160119f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:33:24.354 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:24.354+0000 7ff15ea00700 1 -- 172.21.15.18:0/1440917452 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff150005040 con 0x7ff160109dd0 2024-04-12T21:33:24.354 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:24.354+0000 7ff15ea00700 1 --2- 172.21.15.18:0/1440917452 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff160109dd0 0x7ff16011a480 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7ff14c002a50 tx=0x7ff14c004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:33:24.355 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:24.354+0000 7ff157400700 1 -- 172.21.15.18:0/1440917452 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff14c007cc0 con 0x7ff160109dd0 2024-04-12T21:33:24.355 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:24.355+0000 7ff157400700 1 -- 172.21.15.18:0/1440917452 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff14c00f040 con 0x7ff160109dd0 2024-04-12T21:33:24.355 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:24.355+0000 7ff166000700 1 -- 172.21.15.18:0/1440917452 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff1601143e0 con 0x7ff160109dd0 2024-04-12T21:33:24.355 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:24.355+0000 7ff157400700 1 -- 172.21.15.18:0/1440917452 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff14c00daf0 con 0x7ff160109dd0 2024-04-12T21:33:24.355 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:24.355+0000 7ff166000700 1 -- 172.21.15.18:0/1440917452 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff1601149a0 con 0x7ff160109dd0 2024-04-12T21:33:24.356 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:24.356+0000 7ff166000700 1 -- 172.21.15.18:0/1440917452 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff160104b80 con 0x7ff160109dd0 2024-04-12T21:33:24.356 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:24.357+0000 7ff157400700 1 -- 172.21.15.18:0/1440917452 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7ff14c009810 con 0x7ff160109dd0 2024-04-12T21:33:24.357 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:24.358+0000 7ff157400700 1 --2- 172.21.15.18:0/1440917452 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7ff14406ec00 0x7ff1440710b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:33:24.358 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:24.358+0000 7ff157400700 1 -- 172.21.15.18:0/1440917452 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 6757+0+0 (secure 0 0 0) 0x7ff14c08e7d0 con 0x7ff160109dd0 2024-04-12T21:33:24.358 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:24.358+0000 7ff15f400700 1 --2- 172.21.15.18:0/1440917452 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7ff14406ec00 0x7ff1440710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:33:24.358 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:24.359+0000 7ff15f400700 1 --2- 172.21.15.18:0/1440917452 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7ff14406ec00 0x7ff1440710b0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7ff150004f90 tx=0x7ff150009e30 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:33:24.363 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:24.364+0000 7ff157400700 1 -- 172.21.15.18:0/1440917452 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7ff14c05af00 con 0x7ff160109dd0 2024-04-12T21:33:24.614 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:24.614+0000 7ff166000700 1 -- 172.21.15.18:0/1440917452 --> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7ff160110590 con 0x7ff14406ec00 2024-04-12T21:33:24.616 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:24.617+0000 7ff157400700 1 -- 172.21.15.18:0/1440917452 <== mgr.34102 v2:172.21.15.18:6800/3812891919 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7ff160110590 con 0x7ff14406ec00 2024-04-12T21:33:24.619 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:24.620+0000 7ff166000700 1 -- 172.21.15.18:0/1440917452 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7ff14406ec00 msgr2=0x7ff1440710b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:33:24.620 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:24.620+0000 7ff166000700 1 --2- 172.21.15.18:0/1440917452 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7ff14406ec00 0x7ff1440710b0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7ff150004f90 tx=0x7ff150009e30 comp rx=0 tx=0).stop 2024-04-12T21:33:24.620 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:24.620+0000 7ff166000700 1 -- 172.21.15.18:0/1440917452 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff160109dd0 msgr2=0x7ff16011a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:33:24.620 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:24.620+0000 7ff166000700 1 --2- 172.21.15.18:0/1440917452 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff160109dd0 0x7ff16011a480 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7ff14c002a50 tx=0x7ff14c004600 comp rx=0 tx=0).stop 2024-04-12T21:33:24.620 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:24.620+0000 7ff166000700 1 -- 172.21.15.18:0/1440917452 shutdown_connections 2024-04-12T21:33:24.620 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:24.620+0000 7ff166000700 1 --2- 172.21.15.18:0/1440917452 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7ff14406ec00 0x7ff1440710b0 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:33:24.620 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:24.620+0000 7ff166000700 1 --2- 172.21.15.18:0/1440917452 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff160109430 0x7ff160119f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:33:24.620 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:24.620+0000 7ff166000700 1 --2- 172.21.15.18:0/1440917452 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff160109dd0 0x7ff16011a480 unknown :-1 s=CLOSED pgs=849 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:33:24.621 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:24.620+0000 7ff166000700 1 -- 172.21.15.18:0/1440917452 >> 172.21.15.18:0/1440917452 conn(0x7ff160072c00 msgr2=0x7ff160074c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:33:24.621 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:24.620+0000 7ff166000700 1 -- 172.21.15.18:0/1440917452 shutdown_connections 2024-04-12T21:33:24.621 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:24.621+0000 7ff166000700 1 -- 172.21.15.18:0/1440917452 wait complete. 2024-04-12T21:33:24.637 INFO:teuthology.orchestra.run.smithi018.stdout:true 2024-04-12T21:33:24.784 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:24.783+0000 7fd00e000700 1 -- 172.21.15.18:0/4194207384 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd008109db0 msgr2=0x7fd008103420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:33:24.784 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:24.783+0000 7fd00e000700 1 --2- 172.21.15.18:0/4194207384 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd008109db0 0x7fd008103420 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7fcff4004940 tx=0x7fcff4009e30 comp rx=0 tx=0).stop 2024-04-12T21:33:24.784 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:24.784+0000 7fd00e000700 1 -- 172.21.15.18:0/4194207384 shutdown_connections 2024-04-12T21:33:24.784 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:24.784+0000 7fd00e000700 1 --2- 172.21.15.18:0/4194207384 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd008109db0 0x7fd008103420 unknown :-1 s=CLOSED pgs=850 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:33:24.785 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:24.784+0000 7fd00e000700 1 --2- 172.21.15.18:0/4194207384 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd008109410 0x7fd0081097e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:33:24.785 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:24.784+0000 7fd00e000700 1 -- 172.21.15.18:0/4194207384 >> 172.21.15.18:0/4194207384 conn(0x7fd008072c40 msgr2=0x7fd008075050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:33:24.785 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:24.784+0000 7fd00e000700 1 -- 172.21.15.18:0/4194207384 shutdown_connections 2024-04-12T21:33:24.785 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:24.784+0000 7fd00e000700 1 -- 172.21.15.18:0/4194207384 wait complete. 2024-04-12T21:33:24.785 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:24.785+0000 7fd00e000700 1 Processor -- start 2024-04-12T21:33:24.785 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:24.785+0000 7fd00e000700 1 -- start start 2024-04-12T21:33:24.787 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:24.785+0000 7fd00e000700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd008109410 0x7fd00811a1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:33:24.787 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:24.785+0000 7fd00e000700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd008109db0 0x7fd00811a6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:33:24.787 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:24.785+0000 7fd00e000700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd00811ad30 con 0x7fd008109db0 2024-04-12T21:33:24.787 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:24.785+0000 7fd00e000700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd008114100 con 0x7fd008109410 2024-04-12T21:33:24.787 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:24.785+0000 7fd006a00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd008109db0 0x7fd00811a6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:33:24.787 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:24.786+0000 7fd006a00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd008109db0 0x7fd00811a6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:59082/0 (socket says 172.21.15.18:59082) 2024-04-12T21:33:24.787 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:24.786+0000 7fd006a00700 1 -- 172.21.15.18:0/3544254896 learned_addr learned my addr 172.21.15.18:0/3544254896 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:33:24.787 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:24.786+0000 7fd006a00700 1 -- 172.21.15.18:0/3544254896 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd008109410 msgr2=0x7fd00811a1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:33:24.787 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:24.786+0000 7fd007400700 1 --2- 172.21.15.18:0/3544254896 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd008109410 0x7fd00811a1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:33:24.788 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:24.786+0000 7fd006a00700 1 --2- 172.21.15.18:0/3544254896 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd008109410 0x7fd00811a1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:33:24.788 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:24.786+0000 7fd006a00700 1 -- 172.21.15.18:0/3544254896 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcff4005040 con 0x7fd008109db0 2024-04-12T21:33:24.788 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:24.786+0000 7fd007400700 1 --2- 172.21.15.18:0/3544254896 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd008109410 0x7fd00811a1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-12T21:33:24.788 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:24.786+0000 7fd006a00700 1 --2- 172.21.15.18:0/3544254896 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd008109db0 0x7fd00811a6e0 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7fcff4009180 tx=0x7fcff4009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:33:24.788 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:24.787+0000 7fcfff400700 1 -- 172.21.15.18:0/3544254896 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcff400e040 con 0x7fd008109db0 2024-04-12T21:33:24.788 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:24.787+0000 7fcfff400700 1 -- 172.21.15.18:0/3544254896 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fcff400bbc0 con 0x7fd008109db0 2024-04-12T21:33:24.788 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:24.787+0000 7fd00e000700 1 -- 172.21.15.18:0/3544254896 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd008114380 con 0x7fd008109db0 2024-04-12T21:33:24.788 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:24.787+0000 7fcfff400700 1 -- 172.21.15.18:0/3544254896 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcff401c3f0 con 0x7fd008109db0 2024-04-12T21:33:24.788 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:24.787+0000 7fd00e000700 1 -- 172.21.15.18:0/3544254896 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd008114940 con 0x7fd008109db0 2024-04-12T21:33:24.789 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:24.788+0000 7fd00e000700 1 -- 172.21.15.18:0/3544254896 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd008104b60 con 0x7fd008109db0 2024-04-12T21:33:24.789 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:24.790+0000 7fcfff400700 1 -- 172.21.15.18:0/3544254896 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7fcff4013420 con 0x7fd008109db0 2024-04-12T21:33:24.790 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:24.791+0000 7fcfff400700 1 --2- 172.21.15.18:0/3544254896 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fcfec06ec00 0x7fcfec0710b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:33:24.790 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:24.791+0000 7fcfff400700 1 -- 172.21.15.18:0/3544254896 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 6757+0+0 (secure 0 0 0) 0x7fcff408f560 con 0x7fd008109db0 2024-04-12T21:33:24.790 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:24.791+0000 7fd007400700 1 --2- 172.21.15.18:0/3544254896 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fcfec06ec00 0x7fcfec0710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:33:24.791 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:24.792+0000 7fd007400700 1 --2- 172.21.15.18:0/3544254896 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fcfec06ec00 0x7fcfec0710b0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7fcff8007fd0 tx=0x7fcff8009040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:33:24.795 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:24.796+0000 7fcfff400700 1 -- 172.21.15.18:0/3544254896 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7fcff405bd90 con 0x7fd008109db0 2024-04-12T21:33:25.060 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:25.060+0000 7fd00e000700 1 -- 172.21.15.18:0/3544254896 --> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fd008110550 con 0x7fcfec06ec00 2024-04-12T21:33:25.062 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:25.062+0000 7fcfff400700 1 -- 172.21.15.18:0/3544254896 <== mgr.34102 v2:172.21.15.18:6800/3812891919 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7fd008110550 con 0x7fcfec06ec00 2024-04-12T21:33:25.065 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:25.065+0000 7fd00e000700 1 -- 172.21.15.18:0/3544254896 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fcfec06ec00 msgr2=0x7fcfec0710b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:33:25.065 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:25.065+0000 7fd00e000700 1 --2- 172.21.15.18:0/3544254896 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fcfec06ec00 0x7fcfec0710b0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7fcff8007fd0 tx=0x7fcff8009040 comp rx=0 tx=0).stop 2024-04-12T21:33:25.065 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:25.065+0000 7fd00e000700 1 -- 172.21.15.18:0/3544254896 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd008109db0 msgr2=0x7fd00811a6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:33:25.066 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:25.065+0000 7fd00e000700 1 --2- 172.21.15.18:0/3544254896 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd008109db0 0x7fd00811a6e0 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7fcff4009180 tx=0x7fcff4009e30 comp rx=0 tx=0).stop 2024-04-12T21:33:25.066 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:25.065+0000 7fd00e000700 1 -- 172.21.15.18:0/3544254896 shutdown_connections 2024-04-12T21:33:25.066 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:25.065+0000 7fd00e000700 1 --2- 172.21.15.18:0/3544254896 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fcfec06ec00 0x7fcfec0710b0 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:33:25.066 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:25.065+0000 7fd00e000700 1 --2- 172.21.15.18:0/3544254896 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd008109410 0x7fd00811a1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:33:25.066 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:25.065+0000 7fd00e000700 1 --2- 172.21.15.18:0/3544254896 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd008109db0 0x7fd00811a6e0 unknown :-1 s=CLOSED pgs=851 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:33:25.066 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:25.065+0000 7fd00e000700 1 -- 172.21.15.18:0/3544254896 >> 172.21.15.18:0/3544254896 conn(0x7fd008072c40 msgr2=0x7fd008074c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:33:25.066 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:25.066+0000 7fd00e000700 1 -- 172.21.15.18:0/3544254896 shutdown_connections 2024-04-12T21:33:25.066 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:25.066+0000 7fd00e000700 1 -- 172.21.15.18:0/3544254896 wait complete. 2024-04-12T21:33:25.230 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:25.230+0000 7f1e6f600700 1 -- 172.21.15.18:0/3854152537 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1e68102e20 msgr2=0x7f1e681031f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:33:25.231 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:25.230+0000 7f1e6f600700 1 --2- 172.21.15.18:0/3854152537 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1e68102e20 0x7f1e681031f0 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7f1e50004940 tx=0x7f1e50009e30 comp rx=0 tx=0).stop 2024-04-12T21:33:25.231 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:25.231+0000 7f1e6f600700 1 -- 172.21.15.18:0/3854152537 shutdown_connections 2024-04-12T21:33:25.231 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:25.231+0000 7f1e6f600700 1 --2- 172.21.15.18:0/3854152537 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1e681037c0 0x7f1e680fce30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:33:25.231 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:25.231+0000 7f1e6f600700 1 --2- 172.21.15.18:0/3854152537 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1e68102e20 0x7f1e681031f0 unknown :-1 s=CLOSED pgs=852 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:33:25.231 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:25.231+0000 7f1e6f600700 1 -- 172.21.15.18:0/3854152537 >> 172.21.15.18:0/3854152537 conn(0x7f1e680f8970 msgr2=0x7f1e680fad80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:33:25.231 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:33:24 smithi018 ceph-mon[112590]: pgmap v1009: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:33:25.232 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:25.231+0000 7f1e6f600700 1 -- 172.21.15.18:0/3854152537 shutdown_connections 2024-04-12T21:33:25.232 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:25.231+0000 7f1e6f600700 1 -- 172.21.15.18:0/3854152537 wait complete. 2024-04-12T21:33:25.232 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:25.232+0000 7f1e6f600700 1 Processor -- start 2024-04-12T21:33:25.232 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:25.232+0000 7f1e6f600700 1 -- start start 2024-04-12T21:33:25.232 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:25.233+0000 7f1e6f600700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1e68102e20 0x7f1e68113b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:33:25.232 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:25.233+0000 7f1e6f600700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1e681037c0 0x7f1e681140c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:33:25.233 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:25.233+0000 7f1e6f600700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e681147a0 con 0x7f1e681037c0 2024-04-12T21:33:25.233 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:25.233+0000 7f1e6f600700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e6810db20 con 0x7f1e68102e20 2024-04-12T21:33:25.233 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:25.233+0000 7f1e5fe00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1e681037c0 0x7f1e681140c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:33:25.233 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:25.233+0000 7f1e5fe00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1e681037c0 0x7f1e681140c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:59092/0 (socket says 172.21.15.18:59092) 2024-04-12T21:33:25.234 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:25.233+0000 7f1e5fe00700 1 -- 172.21.15.18:0/2618754243 learned_addr learned my addr 172.21.15.18:0/2618754243 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:33:25.234 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:25.234+0000 7f1e6cc00700 1 --2- 172.21.15.18:0/2618754243 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1e68102e20 0x7f1e68113b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:33:25.234 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:25.234+0000 7f1e5fe00700 1 -- 172.21.15.18:0/2618754243 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1e68102e20 msgr2=0x7f1e68113b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:33:25.234 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:25.234+0000 7f1e5fe00700 1 --2- 172.21.15.18:0/2618754243 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1e68102e20 0x7f1e68113b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:33:25.234 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:25.234+0000 7f1e5fe00700 1 -- 172.21.15.18:0/2618754243 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e50005040 con 0x7f1e681037c0 2024-04-12T21:33:25.234 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:25.234+0000 7f1e5fe00700 1 --2- 172.21.15.18:0/2618754243 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1e681037c0 0x7f1e681140c0 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7f1e5800b790 tx=0x7f1e58009220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:33:25.234 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:25.235+0000 7f1e65600700 1 -- 172.21.15.18:0/2618754243 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1e5800f6f0 con 0x7f1e681037c0 2024-04-12T21:33:25.235 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:25.235+0000 7f1e65600700 1 -- 172.21.15.18:0/2618754243 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f1e5800fd20 con 0x7f1e681037c0 2024-04-12T21:33:25.235 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:25.235+0000 7f1e6f600700 1 -- 172.21.15.18:0/2618754243 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1e6810de00 con 0x7f1e681037c0 2024-04-12T21:33:25.235 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:25.235+0000 7f1e65600700 1 -- 172.21.15.18:0/2618754243 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1e580103a0 con 0x7f1e681037c0 2024-04-12T21:33:25.235 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:25.235+0000 7f1e6f600700 1 -- 172.21.15.18:0/2618754243 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1e6810e3c0 con 0x7f1e681037c0 2024-04-12T21:33:25.236 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:25.236+0000 7f1e6f600700 1 -- 172.21.15.18:0/2618754243 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1e680fe570 con 0x7f1e681037c0 2024-04-12T21:33:25.237 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:25.237+0000 7f1e65600700 1 -- 172.21.15.18:0/2618754243 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7f1e58011400 con 0x7f1e681037c0 2024-04-12T21:33:25.238 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:25.238+0000 7f1e65600700 1 --2- 172.21.15.18:0/2618754243 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f1e4806ec00 0x7f1e480710b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:33:25.238 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:25.238+0000 7f1e65600700 1 -- 172.21.15.18:0/2618754243 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 6757+0+0 (secure 0 0 0) 0x7f1e5808e780 con 0x7f1e681037c0 2024-04-12T21:33:25.238 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:25.239+0000 7f1e6cc00700 1 --2- 172.21.15.18:0/2618754243 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f1e4806ec00 0x7f1e480710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:33:25.239 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:25.239+0000 7f1e6cc00700 1 --2- 172.21.15.18:0/2618754243 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f1e4806ec00 0x7f1e480710b0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f1e50004f90 tx=0x7f1e50009e30 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:33:25.243 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:25.243+0000 7f1e65600700 1 -- 172.21.15.18:0/2618754243 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f1e580574a0 con 0x7f1e681037c0 2024-04-12T21:33:25.282 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:33:24 smithi186 ceph-mon[86379]: pgmap v1009: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:33:25.491 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:25.491+0000 7f1e6f600700 1 -- 172.21.15.18:0/2618754243 --> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f1e6810a000 con 0x7f1e4806ec00 2024-04-12T21:33:25.521 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:25.521+0000 7f1e65600700 1 -- 172.21.15.18:0/2618754243 <== mgr.34102 v2:172.21.15.18:6800/3812891919 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7f1e6810a000 con 0x7f1e4806ec00 2024-04-12T21:33:25.521 INFO:teuthology.orchestra.run.smithi018.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-12T21:33:25.521 INFO:teuthology.orchestra.run.smithi018.stdout:alertmanager.smithi018 smithi018 *:9093,9094 running (34m) 10m ago 48m 15.5M - 0.25.0 c8568f914cd2 c332090068e8 2024-04-12T21:33:25.521 INFO:teuthology.orchestra.run.smithi018.stdout:crash.smithi018 smithi018 running (30m) 10m ago 48m 6665k - 19.0.0-2879-g324baf8d 8717720dc5e5 b7f80d1f7982 2024-04-12T21:33:25.522 INFO:teuthology.orchestra.run.smithi018.stdout:crash.smithi186 smithi186 running (30m) 10m ago 47m 6660k - 19.0.0-2879-g324baf8d 8717720dc5e5 a4f55c81b623 2024-04-12T21:33:25.522 INFO:teuthology.orchestra.run.smithi018.stdout:grafana.smithi018 smithi018 *:3000 running (33m) 10m ago 48m 74.8M - 9.4.12 2bacad6d85d8 b08e3bf176dd 2024-04-12T21:33:25.522 INFO:teuthology.orchestra.run.smithi018.stdout:mds.cephfs.smithi018.acqbsc smithi018 running (26m) 10m ago 41m 16.7M - 19.0.0-2879-g324baf8d 8717720dc5e5 021f99ab24ae 2024-04-12T21:33:25.522 INFO:teuthology.orchestra.run.smithi018.stdout:mds.cephfs.smithi018.szjluo smithi018 running (25m) 10m ago 41m 16.5M - 19.0.0-2879-g324baf8d 8717720dc5e5 0003117479d4 2024-04-12T21:33:25.522 INFO:teuthology.orchestra.run.smithi018.stdout:mds.cephfs.smithi186.jdllyn smithi186 running (10m) 10m ago 41m 11.0M - 19.0.0-2879-g324baf8d 8717720dc5e5 21b1f46eef23 2024-04-12T21:33:25.522 INFO:teuthology.orchestra.run.smithi018.stdout:mds.cephfs.smithi186.wjnafs smithi186 running (41m) 10m ago 41m 19.5M - 17.2.7-827-gd3b5060a 21de9b7369ad 74b115d6d6fd 2024-04-12T21:33:25.522 INFO:teuthology.orchestra.run.smithi018.stdout:mgr.smithi018.xjzwga smithi018 *:8443,9283,8765 running (36m) 10m ago 50m 512M - 19.0.0-2879-g324baf8d 8717720dc5e5 56088305eab7 2024-04-12T21:33:25.522 INFO:teuthology.orchestra.run.smithi018.stdout:mgr.smithi186.kqcfxk smithi186 *:8443,9283,8765 running (35m) 10m ago 47m 459M - 19.0.0-2879-g324baf8d 8717720dc5e5 09cd766e5e0d 2024-04-12T21:33:25.522 INFO:teuthology.orchestra.run.smithi018.stdout:mon.smithi018 smithi018 running (31m) 10m ago 50m 73.1M 2048M 19.0.0-2879-g324baf8d 8717720dc5e5 df3e201dfed6 2024-04-12T21:33:25.522 INFO:teuthology.orchestra.run.smithi018.stdout:mon.smithi186 smithi186 running (31m) 10m ago 47m 47.2M 2048M 19.0.0-2879-g324baf8d 8717720dc5e5 c77498da9b2b 2024-04-12T21:33:25.522 INFO:teuthology.orchestra.run.smithi018.stdout:node-exporter.smithi018 smithi018 *:9100 running (35m) 10m ago 48m 8108k - 1.5.0 0da6a335fe13 326c3d9c6847 2024-04-12T21:33:25.522 INFO:teuthology.orchestra.run.smithi018.stdout:node-exporter.smithi186 smithi186 *:9100 running (34m) 10m ago 47m 8757k - 1.5.0 0da6a335fe13 ec2112baeecf 2024-04-12T21:33:25.522 INFO:teuthology.orchestra.run.smithi018.stdout:osd.0 smithi018 running (29m) 10m ago 46m 123M 1559M 19.0.0-2879-g324baf8d 8717720dc5e5 60b1273ca047 2024-04-12T21:33:25.522 INFO:teuthology.orchestra.run.smithi018.stdout:osd.1 smithi018 running (29m) 10m ago 45m 95.1M 1559M 19.0.0-2879-g324baf8d 8717720dc5e5 170ce1e5fb8d 2024-04-12T21:33:25.523 INFO:teuthology.orchestra.run.smithi018.stdout:osd.2 smithi018 running (28m) 10m ago 44m 94.3M 1559M 19.0.0-2879-g324baf8d 8717720dc5e5 c2cbc6aa2839 2024-04-12T21:33:25.523 INFO:teuthology.orchestra.run.smithi018.stdout:osd.3 smithi186 running (28m) 10m ago 44m 101M 2581M 19.0.0-2879-g324baf8d 8717720dc5e5 0138e8262e4b 2024-04-12T21:33:25.523 INFO:teuthology.orchestra.run.smithi018.stdout:osd.4 smithi186 running (27m) 10m ago 43m 91.0M 2581M 19.0.0-2879-g324baf8d 8717720dc5e5 b20cb63b25c9 2024-04-12T21:33:25.523 INFO:teuthology.orchestra.run.smithi018.stdout:osd.5 smithi186 running (26m) 10m ago 43m 92.0M 2581M 19.0.0-2879-g324baf8d 8717720dc5e5 2513bea2610e 2024-04-12T21:33:25.523 INFO:teuthology.orchestra.run.smithi018.stdout:prometheus.smithi018 smithi018 *:9095 running (34m) 10m ago 47m 67.9M - 2.43.0 a07b618ecd1d f58841127e21 2024-04-12T21:33:25.525 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:25.524+0000 7f1e6f600700 1 -- 172.21.15.18:0/2618754243 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f1e4806ec00 msgr2=0x7f1e480710b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:33:25.525 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:25.524+0000 7f1e6f600700 1 --2- 172.21.15.18:0/2618754243 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f1e4806ec00 0x7f1e480710b0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f1e50004f90 tx=0x7f1e50009e30 comp rx=0 tx=0).stop 2024-04-12T21:33:25.525 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:25.524+0000 7f1e6f600700 1 -- 172.21.15.18:0/2618754243 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1e681037c0 msgr2=0x7f1e681140c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:33:25.525 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:25.524+0000 7f1e6f600700 1 --2- 172.21.15.18:0/2618754243 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1e681037c0 0x7f1e681140c0 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7f1e5800b790 tx=0x7f1e58009220 comp rx=0 tx=0).stop 2024-04-12T21:33:25.525 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:25.525+0000 7f1e6f600700 1 -- 172.21.15.18:0/2618754243 shutdown_connections 2024-04-12T21:33:25.525 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:25.525+0000 7f1e6f600700 1 --2- 172.21.15.18:0/2618754243 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f1e4806ec00 0x7f1e480710b0 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:33:25.525 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:25.525+0000 7f1e6f600700 1 --2- 172.21.15.18:0/2618754243 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1e68102e20 0x7f1e68113b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:33:25.525 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:25.525+0000 7f1e6f600700 1 --2- 172.21.15.18:0/2618754243 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1e681037c0 0x7f1e681140c0 unknown :-1 s=CLOSED pgs=853 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:33:25.525 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:25.525+0000 7f1e6f600700 1 -- 172.21.15.18:0/2618754243 >> 172.21.15.18:0/2618754243 conn(0x7f1e680f8970 msgr2=0x7f1e680f9960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:33:25.525 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:25.525+0000 7f1e6f600700 1 -- 172.21.15.18:0/2618754243 shutdown_connections 2024-04-12T21:33:25.525 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:25.525+0000 7f1e6f600700 1 -- 172.21.15.18:0/2618754243 wait complete. 2024-04-12T21:33:25.672 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:25.672+0000 7fb736400700 1 -- 172.21.15.18:0/2398754756 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb7301092a0 msgr2=0x7fb730109670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:33:25.672 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:25.672+0000 7fb736400700 1 --2- 172.21.15.18:0/2398754756 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb7301092a0 0x7fb730109670 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7fb724004940 tx=0x7fb724009e30 comp rx=0 tx=0).stop 2024-04-12T21:33:25.672 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:25.672+0000 7fb736400700 1 -- 172.21.15.18:0/2398754756 shutdown_connections 2024-04-12T21:33:25.672 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:25.672+0000 7fb736400700 1 --2- 172.21.15.18:0/2398754756 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb730109c40 0x7fb7301032b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:33:25.672 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:25.672+0000 7fb736400700 1 --2- 172.21.15.18:0/2398754756 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb7301092a0 0x7fb730109670 unknown :-1 s=CLOSED pgs=854 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:33:25.672 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:25.672+0000 7fb736400700 1 -- 172.21.15.18:0/2398754756 >> 172.21.15.18:0/2398754756 conn(0x7fb730072a30 msgr2=0x7fb730074e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:33:25.672 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:25.672+0000 7fb736400700 1 -- 172.21.15.18:0/2398754756 shutdown_connections 2024-04-12T21:33:25.673 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:25.672+0000 7fb736400700 1 -- 172.21.15.18:0/2398754756 wait complete. 2024-04-12T21:33:25.674 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:25.673+0000 7fb736400700 1 Processor -- start 2024-04-12T21:33:25.674 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:25.673+0000 7fb736400700 1 -- start start 2024-04-12T21:33:25.674 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:25.674+0000 7fb736400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb7301092a0 0x7fb730119ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:33:25.674 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:25.674+0000 7fb736400700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb730109c40 0x7fb73011a530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:33:25.674 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:25.674+0000 7fb736400700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb73011ab80 con 0x7fb7301092a0 2024-04-12T21:33:25.674 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:25.674+0000 7fb736400700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb730113f50 con 0x7fb730109c40 2024-04-12T21:33:25.674 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:25.674+0000 7fb735000700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb7301092a0 0x7fb730119ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:33:25.674 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:25.674+0000 7fb735000700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb7301092a0 0x7fb730119ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:59108/0 (socket says 172.21.15.18:59108) 2024-04-12T21:33:25.675 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:25.674+0000 7fb735000700 1 -- 172.21.15.18:0/1494162877 learned_addr learned my addr 172.21.15.18:0/1494162877 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:33:25.675 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:25.674+0000 7fb735000700 1 -- 172.21.15.18:0/1494162877 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb730109c40 msgr2=0x7fb73011a530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:33:25.675 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:25.674+0000 7fb72fe00700 1 --2- 172.21.15.18:0/1494162877 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb730109c40 0x7fb73011a530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:33:25.675 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:25.674+0000 7fb735000700 1 --2- 172.21.15.18:0/1494162877 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb730109c40 0x7fb73011a530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:33:25.675 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:25.674+0000 7fb735000700 1 -- 172.21.15.18:0/1494162877 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb724005040 con 0x7fb7301092a0 2024-04-12T21:33:25.675 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:25.675+0000 7fb735000700 1 --2- 172.21.15.18:0/1494162877 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb7301092a0 0x7fb730119ff0 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7fb724009180 tx=0x7fb724009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:33:25.676 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:25.675+0000 7fb72cc00700 1 -- 172.21.15.18:0/1494162877 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb72400e040 con 0x7fb7301092a0 2024-04-12T21:33:25.676 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:25.675+0000 7fb72cc00700 1 -- 172.21.15.18:0/1494162877 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb72400bbc0 con 0x7fb7301092a0 2024-04-12T21:33:25.676 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:25.675+0000 7fb72cc00700 1 -- 172.21.15.18:0/1494162877 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb72401c630 con 0x7fb7301092a0 2024-04-12T21:33:25.676 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:25.675+0000 7fb736400700 1 -- 172.21.15.18:0/1494162877 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb7301141d0 con 0x7fb7301092a0 2024-04-12T21:33:25.676 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:25.675+0000 7fb736400700 1 -- 172.21.15.18:0/1494162877 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb730114790 con 0x7fb7301092a0 2024-04-12T21:33:25.676 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:25.676+0000 7fb736400700 1 -- 172.21.15.18:0/1494162877 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb7301049f0 con 0x7fb7301092a0 2024-04-12T21:33:25.677 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:25.677+0000 7fb72cc00700 1 -- 172.21.15.18:0/1494162877 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7fb72400bd30 con 0x7fb7301092a0 2024-04-12T21:33:25.678 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:25.678+0000 7fb72cc00700 1 --2- 172.21.15.18:0/1494162877 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fb71806ec00 0x7fb7180710b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:33:25.678 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:25.678+0000 7fb72cc00700 1 -- 172.21.15.18:0/1494162877 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 6757+0+0 (secure 0 0 0) 0x7fb72408fc90 con 0x7fb7301092a0 2024-04-12T21:33:25.678 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:25.679+0000 7fb72fe00700 1 --2- 172.21.15.18:0/1494162877 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fb71806ec00 0x7fb7180710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:33:25.679 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:25.679+0000 7fb72fe00700 1 --2- 172.21.15.18:0/1494162877 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fb71806ec00 0x7fb7180710b0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7fb730115890 tx=0x7fb71c00b040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:33:25.683 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:25.684+0000 7fb72cc00700 1 -- 172.21.15.18:0/1494162877 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7fb72405c580 con 0x7fb7301092a0 2024-04-12T21:33:26.017 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:26.017+0000 7fb736400700 1 -- 172.21.15.18:0/1494162877 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fb7301049f0 con 0x7fb7301092a0 2024-04-12T21:33:26.018 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:26.018+0000 7fb72cc00700 1 -- 172.21.15.18:0/1494162877 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+837 (secure 0 0 0) 0x7fb72405c110 con 0x7fb7301092a0 2024-04-12T21:33:26.018 INFO:teuthology.orchestra.run.smithi018.stdout:{ 2024-04-12T21:33:26.018 INFO:teuthology.orchestra.run.smithi018.stdout: "mon": { 2024-04-12T21:33:26.018 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 19.0.0-2879-g324baf8d (324baf8ddb91dea46cac50cb535c306115791355) squid (dev)": 2 2024-04-12T21:33:26.018 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-12T21:33:26.018 INFO:teuthology.orchestra.run.smithi018.stdout: "mgr": { 2024-04-12T21:33:26.019 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 19.0.0-2879-g324baf8d (324baf8ddb91dea46cac50cb535c306115791355) squid (dev)": 2 2024-04-12T21:33:26.019 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-12T21:33:26.019 INFO:teuthology.orchestra.run.smithi018.stdout: "osd": { 2024-04-12T21:33:26.019 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 19.0.0-2879-g324baf8d (324baf8ddb91dea46cac50cb535c306115791355) squid (dev)": 6 2024-04-12T21:33:26.019 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-12T21:33:26.019 INFO:teuthology.orchestra.run.smithi018.stdout: "mds": { 2024-04-12T21:33:26.019 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 17.2.7-827-gd3b5060a (d3b5060a17449edfae9385cdf792afec109fca5d) quincy (stable)": 1, 2024-04-12T21:33:26.019 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 19.0.0-2879-g324baf8d (324baf8ddb91dea46cac50cb535c306115791355) squid (dev)": 3 2024-04-12T21:33:26.019 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-12T21:33:26.019 INFO:teuthology.orchestra.run.smithi018.stdout: "overall": { 2024-04-12T21:33:26.019 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 17.2.7-827-gd3b5060a (d3b5060a17449edfae9385cdf792afec109fca5d) quincy (stable)": 1, 2024-04-12T21:33:26.019 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 19.0.0-2879-g324baf8d (324baf8ddb91dea46cac50cb535c306115791355) squid (dev)": 13 2024-04-12T21:33:26.019 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-04-12T21:33:26.019 INFO:teuthology.orchestra.run.smithi018.stdout:} 2024-04-12T21:33:26.021 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:26.021+0000 7fb736400700 1 -- 172.21.15.18:0/1494162877 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fb71806ec00 msgr2=0x7fb7180710b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:33:26.021 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:26.021+0000 7fb736400700 1 --2- 172.21.15.18:0/1494162877 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fb71806ec00 0x7fb7180710b0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7fb730115890 tx=0x7fb71c00b040 comp rx=0 tx=0).stop 2024-04-12T21:33:26.021 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:26.021+0000 7fb736400700 1 -- 172.21.15.18:0/1494162877 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb7301092a0 msgr2=0x7fb730119ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:33:26.021 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:26.021+0000 7fb736400700 1 --2- 172.21.15.18:0/1494162877 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb7301092a0 0x7fb730119ff0 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7fb724009180 tx=0x7fb724009e30 comp rx=0 tx=0).stop 2024-04-12T21:33:26.022 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:26.021+0000 7fb736400700 1 -- 172.21.15.18:0/1494162877 shutdown_connections 2024-04-12T21:33:26.022 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:26.021+0000 7fb736400700 1 --2- 172.21.15.18:0/1494162877 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fb71806ec00 0x7fb7180710b0 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:33:26.022 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:26.021+0000 7fb736400700 1 --2- 172.21.15.18:0/1494162877 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb7301092a0 0x7fb730119ff0 unknown :-1 s=CLOSED pgs=855 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:33:26.022 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:26.021+0000 7fb736400700 1 --2- 172.21.15.18:0/1494162877 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb730109c40 0x7fb73011a530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:33:26.022 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:26.021+0000 7fb736400700 1 -- 172.21.15.18:0/1494162877 >> 172.21.15.18:0/1494162877 conn(0x7fb730072a30 msgr2=0x7fb7300748d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:33:26.022 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:26.022+0000 7fb736400700 1 -- 172.21.15.18:0/1494162877 shutdown_connections 2024-04-12T21:33:26.022 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:26.022+0000 7fb736400700 1 -- 172.21.15.18:0/1494162877 wait complete. 2024-04-12T21:33:26.166 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:26.166+0000 7fab23000700 1 -- 172.21.15.18:0/2660346351 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fab1c0fbbd0 msgr2=0x7fab1c0fbfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:33:26.166 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:26.166+0000 7fab23000700 1 --2- 172.21.15.18:0/2660346351 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fab1c0fbbd0 0x7fab1c0fbfa0 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7fab0c004940 tx=0x7fab0c009e30 comp rx=0 tx=0).stop 2024-04-12T21:33:26.167 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:26.166+0000 7fab23000700 1 -- 172.21.15.18:0/2660346351 shutdown_connections 2024-04-12T21:33:26.167 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:26.166+0000 7fab23000700 1 --2- 172.21.15.18:0/2660346351 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fab1c0fc4e0 0x7fab1c109240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:33:26.167 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:26.166+0000 7fab23000700 1 --2- 172.21.15.18:0/2660346351 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fab1c0fbbd0 0x7fab1c0fbfa0 unknown :-1 s=CLOSED pgs=856 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:33:26.167 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:33:25 smithi018 ceph-mon[112590]: from='client.35764 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:33:26.167 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:33:25 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T21:33:26.169 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:26.166+0000 7fab23000700 1 -- 172.21.15.18:0/2660346351 >> 172.21.15.18:0/2660346351 conn(0x7fab1c0f7860 msgr2=0x7fab1c0f9c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:33:26.169 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:26.167+0000 7fab23000700 1 -- 172.21.15.18:0/2660346351 shutdown_connections 2024-04-12T21:33:26.169 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:26.167+0000 7fab23000700 1 -- 172.21.15.18:0/2660346351 wait complete. 2024-04-12T21:33:26.169 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:26.168+0000 7fab23000700 1 Processor -- start 2024-04-12T21:33:26.169 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:26.168+0000 7fab23000700 1 -- start start 2024-04-12T21:33:26.169 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:26.168+0000 7fab23000700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fab1c0fbbd0 0x7fab1c115dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:33:26.169 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:26.168+0000 7fab23000700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fab1c0fc4e0 0x7fab1c116300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:33:26.169 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:26.168+0000 7fab23000700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab1c1169e0 con 0x7fab1c0fbbd0 2024-04-12T21:33:26.169 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:26.168+0000 7fab23000700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab1c10fd20 con 0x7fab1c0fc4e0 2024-04-12T21:33:26.170 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:26.169+0000 7fab1be00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fab1c0fbbd0 0x7fab1c115dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:33:26.170 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:26.169+0000 7fab1be00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fab1c0fbbd0 0x7fab1c115dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:59116/0 (socket says 172.21.15.18:59116) 2024-04-12T21:33:26.170 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:26.169+0000 7fab1be00700 1 -- 172.21.15.18:0/2118093978 learned_addr learned my addr 172.21.15.18:0/2118093978 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:33:26.170 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:26.169+0000 7fab1be00700 1 -- 172.21.15.18:0/2118093978 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fab1c0fc4e0 msgr2=0x7fab1c116300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:33:26.170 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:26.169+0000 7fab1b400700 1 --2- 172.21.15.18:0/2118093978 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fab1c0fc4e0 0x7fab1c116300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:33:26.170 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:26.169+0000 7fab1be00700 1 --2- 172.21.15.18:0/2118093978 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fab1c0fc4e0 0x7fab1c116300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:33:26.170 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:26.169+0000 7fab1be00700 1 -- 172.21.15.18:0/2118093978 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab0c005040 con 0x7fab1c0fbbd0 2024-04-12T21:33:26.170 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:26.169+0000 7fab1b400700 1 --2- 172.21.15.18:0/2118093978 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fab1c0fc4e0 0x7fab1c116300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-12T21:33:26.170 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:26.169+0000 7fab1be00700 1 --2- 172.21.15.18:0/2118093978 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fab1c0fbbd0 0x7fab1c115dc0 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7fab0c009380 tx=0x7fab0c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:33:26.171 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:26.170+0000 7fab13e00700 1 -- 172.21.15.18:0/2118093978 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fab0c00e040 con 0x7fab1c0fbbd0 2024-04-12T21:33:26.171 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:26.170+0000 7fab13e00700 1 -- 172.21.15.18:0/2118093978 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fab0c004290 con 0x7fab1c0fbbd0 2024-04-12T21:33:26.171 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:26.170+0000 7fab13e00700 1 -- 172.21.15.18:0/2118093978 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fab0c01c3f0 con 0x7fab1c0fbbd0 2024-04-12T21:33:26.171 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:26.170+0000 7fab23000700 1 -- 172.21.15.18:0/2118093978 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fab1c10ff40 con 0x7fab1c0fbbd0 2024-04-12T21:33:26.172 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:26.170+0000 7fab23000700 1 -- 172.21.15.18:0/2118093978 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fab1c110500 con 0x7fab1c0fbbd0 2024-04-12T21:33:26.172 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:26.171+0000 7fab23000700 1 -- 172.21.15.18:0/2118093978 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fab1c0666f0 con 0x7fab1c0fbbd0 2024-04-12T21:33:26.172 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:26.173+0000 7fab13e00700 1 -- 172.21.15.18:0/2118093978 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7fab0c013420 con 0x7fab1c0fbbd0 2024-04-12T21:33:26.173 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:26.174+0000 7fab13e00700 1 --2- 172.21.15.18:0/2118093978 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fab0006ec10 0x7fab000710c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:33:26.174 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:26.174+0000 7fab13e00700 1 -- 172.21.15.18:0/2118093978 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 6757+0+0 (secure 0 0 0) 0x7fab0c08fb80 con 0x7fab1c0fbbd0 2024-04-12T21:33:26.174 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:26.174+0000 7fab1b400700 1 --2- 172.21.15.18:0/2118093978 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fab0006ec10 0x7fab000710c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:33:26.174 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:26.175+0000 7fab1b400700 1 --2- 172.21.15.18:0/2118093978 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fab0006ec10 0x7fab000710c0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7fab08006fd0 tx=0x7fab08008040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:33:26.179 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:26.179+0000 7fab13e00700 1 -- 172.21.15.18:0/2118093978 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7fab0c05c470 con 0x7fab1c0fbbd0 2024-04-12T21:33:26.282 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:33:25 smithi186 ceph-mon[86379]: from='client.35764 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:33:26.282 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:33:25 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T21:33:26.456 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:26.456+0000 7fab23000700 1 -- 172.21.15.18:0/2118093978 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fab1c04bf00 con 0x7fab1c0fbbd0 2024-04-12T21:33:26.456 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:26.457+0000 7fab13e00700 1 -- 172.21.15.18:0/2118093978 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 31 v31) v1 ==== 76+0+1873 (secure 0 0 0) 0x7fab0c05c000 con 0x7fab1c0fbbd0 2024-04-12T21:33:26.457 INFO:teuthology.orchestra.run.smithi018.stdout:e31 2024-04-12T21:33:26.457 INFO:teuthology.orchestra.run.smithi018.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-12T21:33:26.457 INFO:teuthology.orchestra.run.smithi018.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-12T21:33:26.457 INFO:teuthology.orchestra.run.smithi018.stdout:legacy client fscid: 1 2024-04-12T21:33:26.457 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-12T21:33:26.457 INFO:teuthology.orchestra.run.smithi018.stdout:Filesystem 'cephfs' (1) 2024-04-12T21:33:26.458 INFO:teuthology.orchestra.run.smithi018.stdout:fs_name cephfs 2024-04-12T21:33:26.458 INFO:teuthology.orchestra.run.smithi018.stdout:epoch 24 2024-04-12T21:33:26.458 INFO:teuthology.orchestra.run.smithi018.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-04-12T21:33:26.458 INFO:teuthology.orchestra.run.smithi018.stdout:created 2024-04-12T20:51:57.362850+0000 2024-04-12T21:33:26.458 INFO:teuthology.orchestra.run.smithi018.stdout:modified 2024-04-12T21:06:59.717190+0000 2024-04-12T21:33:26.458 INFO:teuthology.orchestra.run.smithi018.stdout:tableserver 0 2024-04-12T21:33:26.458 INFO:teuthology.orchestra.run.smithi018.stdout:root 0 2024-04-12T21:33:26.458 INFO:teuthology.orchestra.run.smithi018.stdout:session_timeout 60 2024-04-12T21:33:26.458 INFO:teuthology.orchestra.run.smithi018.stdout:session_autoclose 300 2024-04-12T21:33:26.458 INFO:teuthology.orchestra.run.smithi018.stdout:max_file_size 1099511627776 2024-04-12T21:33:26.458 INFO:teuthology.orchestra.run.smithi018.stdout:max_xattr_size 65536 2024-04-12T21:33:26.458 INFO:teuthology.orchestra.run.smithi018.stdout:required_client_features {} 2024-04-12T21:33:26.458 INFO:teuthology.orchestra.run.smithi018.stdout:last_failure 0 2024-04-12T21:33:26.458 INFO:teuthology.orchestra.run.smithi018.stdout:last_failure_osd_epoch 100 2024-04-12T21:33:26.459 INFO:teuthology.orchestra.run.smithi018.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-12T21:33:26.459 INFO:teuthology.orchestra.run.smithi018.stdout:max_mds 2 2024-04-12T21:33:26.459 INFO:teuthology.orchestra.run.smithi018.stdout:in 0,1 2024-04-12T21:33:26.459 INFO:teuthology.orchestra.run.smithi018.stdout:up {} 2024-04-12T21:33:26.459 INFO:teuthology.orchestra.run.smithi018.stdout:failed 0,1 2024-04-12T21:33:26.459 INFO:teuthology.orchestra.run.smithi018.stdout:damaged 2024-04-12T21:33:26.459 INFO:teuthology.orchestra.run.smithi018.stdout:stopped 2024-04-12T21:33:26.459 INFO:teuthology.orchestra.run.smithi018.stdout:data_pools [3] 2024-04-12T21:33:26.459 INFO:teuthology.orchestra.run.smithi018.stdout:metadata_pool 2 2024-04-12T21:33:26.459 INFO:teuthology.orchestra.run.smithi018.stdout:inline_data disabled 2024-04-12T21:33:26.459 INFO:teuthology.orchestra.run.smithi018.stdout:balancer 2024-04-12T21:33:26.459 INFO:teuthology.orchestra.run.smithi018.stdout:bal_rank_mask -1 2024-04-12T21:33:26.459 INFO:teuthology.orchestra.run.smithi018.stdout:standby_count_wanted 1 2024-04-12T21:33:26.460 INFO:teuthology.orchestra.run.smithi018.stdout:qdb_cluster leader: 0 members: 2024-04-12T21:33:26.460 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-12T21:33:26.460 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-12T21:33:26.460 INFO:teuthology.orchestra.run.smithi018.stdout:Standby daemons: 2024-04-12T21:33:26.460 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-12T21:33:26.460 INFO:teuthology.orchestra.run.smithi018.stdout:[mds.cephfs.smithi186.wjnafs{-1:24277} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.186:6826/438347207,v1:172.21.15.186:6827/438347207] compat {c=[1],r=[1],i=[7ff]}] 2024-04-12T21:33:26.460 INFO:teuthology.orchestra.run.smithi018.stdout:[mds.cephfs.smithi018.acqbsc{-1:34436} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.18:6828/755179764,v1:172.21.15.18:6829/755179764] compat {c=[1],r=[1],i=[1fff]}] 2024-04-12T21:33:26.460 INFO:teuthology.orchestra.run.smithi018.stdout:[mds.cephfs.smithi018.szjluo{-1:34468} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.18:6826/2407992653,v1:172.21.15.18:6827/2407992653] compat {c=[1],r=[1],i=[1fff]}] 2024-04-12T21:33:26.460 INFO:teuthology.orchestra.run.smithi018.stdout:[mds.cephfs.smithi186.jdllyn{-1:44595} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.186:6824/3523444202,v1:172.21.15.186:6825/3523444202] compat {c=[1],r=[1],i=[1fff]}] 2024-04-12T21:33:26.461 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:26.460+0000 7fab23000700 1 -- 172.21.15.18:0/2118093978 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fab0006ec10 msgr2=0x7fab000710c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:33:26.461 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:26.460+0000 7fab23000700 1 --2- 172.21.15.18:0/2118093978 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fab0006ec10 0x7fab000710c0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7fab08006fd0 tx=0x7fab08008040 comp rx=0 tx=0).stop 2024-04-12T21:33:26.461 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:26.460+0000 7fab23000700 1 -- 172.21.15.18:0/2118093978 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fab1c0fbbd0 msgr2=0x7fab1c115dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:33:26.461 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:26.460+0000 7fab23000700 1 --2- 172.21.15.18:0/2118093978 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fab1c0fbbd0 0x7fab1c115dc0 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7fab0c009380 tx=0x7fab0c009e30 comp rx=0 tx=0).stop 2024-04-12T21:33:26.461 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:26.460+0000 7fab23000700 1 -- 172.21.15.18:0/2118093978 shutdown_connections 2024-04-12T21:33:26.461 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:26.460+0000 7fab23000700 1 --2- 172.21.15.18:0/2118093978 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fab0006ec10 0x7fab000710c0 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:33:26.461 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:26.460+0000 7fab23000700 1 --2- 172.21.15.18:0/2118093978 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fab1c0fbbd0 0x7fab1c115dc0 unknown :-1 s=CLOSED pgs=857 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:33:26.462 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:26.460+0000 7fab23000700 1 --2- 172.21.15.18:0/2118093978 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fab1c0fc4e0 0x7fab1c116300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:33:26.462 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:26.460+0000 7fab23000700 1 -- 172.21.15.18:0/2118093978 >> 172.21.15.18:0/2118093978 conn(0x7fab1c0f7860 msgr2=0x7fab1c0f7cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:33:26.462 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:26.460+0000 7fab23000700 1 -- 172.21.15.18:0/2118093978 shutdown_connections 2024-04-12T21:33:26.462 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:26.461+0000 7fab23000700 1 -- 172.21.15.18:0/2118093978 wait complete. 2024-04-12T21:33:26.462 INFO:teuthology.orchestra.run.smithi018.stderr:dumped fsmap epoch 31 2024-04-12T21:33:26.619 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:26.619+0000 7f4862200700 1 -- 172.21.15.18:0/589005332 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f485c102e00 msgr2=0x7f485c1031d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:33:26.619 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:26.619+0000 7f4862200700 1 --2- 172.21.15.18:0/589005332 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f485c102e00 0x7f485c1031d0 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7f484c004940 tx=0x7f484c009e30 comp rx=0 tx=0).stop 2024-04-12T21:33:26.619 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:26.619+0000 7f4862200700 1 -- 172.21.15.18:0/589005332 shutdown_connections 2024-04-12T21:33:26.619 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:26.619+0000 7f4862200700 1 --2- 172.21.15.18:0/589005332 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f485c1037a0 0x7f485c0fce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:33:26.619 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:26.619+0000 7f4862200700 1 --2- 172.21.15.18:0/589005332 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f485c102e00 0x7f485c1031d0 unknown :-1 s=CLOSED pgs=858 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:33:26.619 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:26.619+0000 7f4862200700 1 -- 172.21.15.18:0/589005332 >> 172.21.15.18:0/589005332 conn(0x7f485c0f89b0 msgr2=0x7f485c0fadc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:33:26.619 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:26.619+0000 7f4862200700 1 -- 172.21.15.18:0/589005332 shutdown_connections 2024-04-12T21:33:26.619 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:26.619+0000 7f4862200700 1 -- 172.21.15.18:0/589005332 wait complete. 2024-04-12T21:33:26.620 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:26.620+0000 7f4862200700 1 Processor -- start 2024-04-12T21:33:26.620 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:26.620+0000 7f4862200700 1 -- start start 2024-04-12T21:33:26.620 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:26.620+0000 7f4862200700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f485c102e00 0x7f485c0702c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:33:26.620 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:26.620+0000 7f4862200700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f485c1037a0 0x7f485c070800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:33:26.621 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:26.620+0000 7f4862200700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f485c0721c0 con 0x7f485c102e00 2024-04-12T21:33:26.621 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:26.620+0000 7f4862200700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f485c070d40 con 0x7f485c1037a0 2024-04-12T21:33:26.621 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:26.621+0000 7f485b400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f485c102e00 0x7f485c0702c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:33:26.621 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:26.621+0000 7f485b400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f485c102e00 0x7f485c0702c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:59130/0 (socket says 172.21.15.18:59130) 2024-04-12T21:33:26.621 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:26.621+0000 7f485b400700 1 -- 172.21.15.18:0/2371679428 learned_addr learned my addr 172.21.15.18:0/2371679428 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:33:26.622 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:26.621+0000 7f485b400700 1 -- 172.21.15.18:0/2371679428 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f485c1037a0 msgr2=0x7f485c070800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:33:26.622 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:26.621+0000 7f485b400700 1 --2- 172.21.15.18:0/2371679428 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f485c1037a0 0x7f485c070800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:33:26.622 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:26.621+0000 7f485b400700 1 -- 172.21.15.18:0/2371679428 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f484c005040 con 0x7f485c102e00 2024-04-12T21:33:26.622 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:26.622+0000 7f485b400700 1 --2- 172.21.15.18:0/2371679428 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f485c102e00 0x7f485c0702c0 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7f484c009150 tx=0x7f484c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:33:26.622 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:26.622+0000 7f4853400700 1 -- 172.21.15.18:0/2371679428 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f484c00e040 con 0x7f485c102e00 2024-04-12T21:33:26.622 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:26.622+0000 7f4853400700 1 -- 172.21.15.18:0/2371679428 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f484c013420 con 0x7f485c102e00 2024-04-12T21:33:26.622 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:26.622+0000 7f4862200700 1 -- 172.21.15.18:0/2371679428 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f485c070fc0 con 0x7f485c102e00 2024-04-12T21:33:26.622 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:26.622+0000 7f4853400700 1 -- 172.21.15.18:0/2371679428 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f484c01c630 con 0x7f485c102e00 2024-04-12T21:33:26.622 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:26.622+0000 7f4862200700 1 -- 172.21.15.18:0/2371679428 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f485c11a130 con 0x7f485c102e00 2024-04-12T21:33:26.623 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:26.623+0000 7f4862200700 1 -- 172.21.15.18:0/2371679428 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f485c100110 con 0x7f485c102e00 2024-04-12T21:33:26.623 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:26.624+0000 7f4853400700 1 -- 172.21.15.18:0/2371679428 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7f484c013590 con 0x7f485c102e00 2024-04-12T21:33:26.625 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:26.625+0000 7f4853400700 1 --2- 172.21.15.18:0/2371679428 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f484006ec00 0x7f48400710b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:33:26.625 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:26.625+0000 7f4853400700 1 -- 172.21.15.18:0/2371679428 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 6757+0+0 (secure 0 0 0) 0x7f484c08fcc0 con 0x7f485c102e00 2024-04-12T21:33:26.625 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:26.625+0000 7f485aa00700 1 --2- 172.21.15.18:0/2371679428 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f484006ec00 0x7f48400710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:33:26.625 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:26.626+0000 7f485aa00700 1 --2- 172.21.15.18:0/2371679428 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f484006ec00 0x7f48400710b0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f485c0719a0 tx=0x7f4848009ce0 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:33:26.629 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:26.630+0000 7f4853400700 1 -- 172.21.15.18:0/2371679428 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f484c095020 con 0x7f485c102e00 2024-04-12T21:33:26.875 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:26.876+0000 7f4862200700 1 -- 172.21.15.18:0/2371679428 --> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f485c109f50 con 0x7f484006ec00 2024-04-12T21:33:26.877 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:26.877+0000 7f4853400700 1 -- 172.21.15.18:0/2371679428 <== mgr.34102 v2:172.21.15.18:6800/3812891919 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f485c109f50 con 0x7f484006ec00 2024-04-12T21:33:26.877 INFO:teuthology.orchestra.run.smithi018.stdout:{ 2024-04-12T21:33:26.877 INFO:teuthology.orchestra.run.smithi018.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:f4452def3be4d350f4a8e5feaf7ebb1511be9298fe68478ba7a2d6fbd7d69a63", 2024-04-12T21:33:26.877 INFO:teuthology.orchestra.run.smithi018.stdout: "in_progress": true, 2024-04-12T21:33:26.877 INFO:teuthology.orchestra.run.smithi018.stdout: "which": "Upgrading all daemon types on all hosts", 2024-04-12T21:33:26.878 INFO:teuthology.orchestra.run.smithi018.stdout: "services_complete": [ 2024-04-12T21:33:26.878 INFO:teuthology.orchestra.run.smithi018.stdout: "crash", 2024-04-12T21:33:26.878 INFO:teuthology.orchestra.run.smithi018.stdout: "osd", 2024-04-12T21:33:26.878 INFO:teuthology.orchestra.run.smithi018.stdout: "mon", 2024-04-12T21:33:26.878 INFO:teuthology.orchestra.run.smithi018.stdout: "mgr" 2024-04-12T21:33:26.878 INFO:teuthology.orchestra.run.smithi018.stdout: ], 2024-04-12T21:33:26.878 INFO:teuthology.orchestra.run.smithi018.stdout: "progress": "15/21 daemons upgraded", 2024-04-12T21:33:26.878 INFO:teuthology.orchestra.run.smithi018.stdout: "message": "Currently upgrading mds daemons", 2024-04-12T21:33:26.878 INFO:teuthology.orchestra.run.smithi018.stdout: "is_paused": false 2024-04-12T21:33:26.878 INFO:teuthology.orchestra.run.smithi018.stdout:} 2024-04-12T21:33:26.880 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:26.880+0000 7f4862200700 1 -- 172.21.15.18:0/2371679428 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f484006ec00 msgr2=0x7f48400710b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:33:26.880 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:26.880+0000 7f4862200700 1 --2- 172.21.15.18:0/2371679428 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f484006ec00 0x7f48400710b0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f485c0719a0 tx=0x7f4848009ce0 comp rx=0 tx=0).stop 2024-04-12T21:33:26.880 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:26.880+0000 7f4862200700 1 -- 172.21.15.18:0/2371679428 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f485c102e00 msgr2=0x7f485c0702c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:33:26.880 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:26.880+0000 7f4862200700 1 --2- 172.21.15.18:0/2371679428 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f485c102e00 0x7f485c0702c0 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7f484c009150 tx=0x7f484c009e30 comp rx=0 tx=0).stop 2024-04-12T21:33:26.881 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:26.881+0000 7f4862200700 1 -- 172.21.15.18:0/2371679428 shutdown_connections 2024-04-12T21:33:26.881 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:26.881+0000 7f4862200700 1 --2- 172.21.15.18:0/2371679428 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f484006ec00 0x7f48400710b0 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:33:26.881 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:26.881+0000 7f4862200700 1 --2- 172.21.15.18:0/2371679428 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f485c102e00 0x7f485c0702c0 unknown :-1 s=CLOSED pgs=859 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:33:26.881 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:26.881+0000 7f4862200700 1 --2- 172.21.15.18:0/2371679428 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f485c1037a0 0x7f485c070800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:33:26.881 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:26.881+0000 7f4862200700 1 -- 172.21.15.18:0/2371679428 >> 172.21.15.18:0/2371679428 conn(0x7f485c0f89b0 msgr2=0x7f485c1073a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:33:26.881 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:26.881+0000 7f4862200700 1 -- 172.21.15.18:0/2371679428 shutdown_connections 2024-04-12T21:33:26.881 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:26.881+0000 7f4862200700 1 -- 172.21.15.18:0/2371679428 wait complete. 2024-04-12T21:33:27.034 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:27.033+0000 7f37c3e00700 1 -- 172.21.15.18:0/895511364 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f37c4072e00 msgr2=0x7f37c410d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:33:27.034 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:27.033+0000 7f37c3e00700 1 --2- 172.21.15.18:0/895511364 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f37c4072e00 0x7f37c410d5d0 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7f37b0004940 tx=0x7f37b0009e30 comp rx=0 tx=0).stop 2024-04-12T21:33:27.034 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:27.033+0000 7f37c3e00700 1 -- 172.21.15.18:0/895511364 shutdown_connections 2024-04-12T21:33:27.034 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:27.033+0000 7f37c3e00700 1 --2- 172.21.15.18:0/895511364 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f37c4072e00 0x7f37c410d5d0 unknown :-1 s=CLOSED pgs=860 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:33:27.034 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:27.033+0000 7f37c3e00700 1 --2- 172.21.15.18:0/895511364 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f37c40724f0 0x7f37c40728c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:33:27.034 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:27.033+0000 7f37c3e00700 1 -- 172.21.15.18:0/895511364 >> 172.21.15.18:0/895511364 conn(0x7f37c40fab10 msgr2=0x7f37c40fcef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:33:27.035 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:27.033+0000 7f37c3e00700 1 -- 172.21.15.18:0/895511364 shutdown_connections 2024-04-12T21:33:27.035 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:27.033+0000 7f37c3e00700 1 -- 172.21.15.18:0/895511364 wait complete. 2024-04-12T21:33:27.035 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:27.034+0000 7f37c3e00700 1 Processor -- start 2024-04-12T21:33:27.035 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:27.034+0000 7f37c3e00700 1 -- start start 2024-04-12T21:33:27.035 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:27.035+0000 7f37c3e00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f37c40724f0 0x7f37c411a020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:33:27.035 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:27.035+0000 7f37c3e00700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f37c4072e00 0x7f37c411a560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:33:27.035 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:27.035+0000 7f37c3e00700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37c411abb0 con 0x7f37c40724f0 2024-04-12T21:33:27.035 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:27.035+0000 7f37c3e00700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37c4113f80 con 0x7f37c4072e00 2024-04-12T21:33:27.036 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:27.035+0000 7f37c2a00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f37c40724f0 0x7f37c411a020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:33:27.037 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:27.035+0000 7f37c2a00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f37c40724f0 0x7f37c411a020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:59156/0 (socket says 172.21.15.18:59156) 2024-04-12T21:33:27.037 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:27.035+0000 7f37c2a00700 1 -- 172.21.15.18:0/3014957557 learned_addr learned my addr 172.21.15.18:0/3014957557 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:33:27.037 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:27.035+0000 7f37c2000700 1 --2- 172.21.15.18:0/3014957557 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f37c4072e00 0x7f37c411a560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:33:27.037 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:27.035+0000 7f37c2a00700 1 -- 172.21.15.18:0/3014957557 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f37c4072e00 msgr2=0x7f37c411a560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:33:27.037 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:27.035+0000 7f37c2a00700 1 --2- 172.21.15.18:0/3014957557 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f37c4072e00 0x7f37c411a560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:33:27.037 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:27.036+0000 7f37c2a00700 1 -- 172.21.15.18:0/3014957557 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37b0005040 con 0x7f37c40724f0 2024-04-12T21:33:27.037 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:27.036+0000 7f37c2a00700 1 --2- 172.21.15.18:0/3014957557 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f37c40724f0 0x7f37c411a020 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7f37b4002a50 tx=0x7f37b40046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:33:27.037 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:27.036+0000 7f37baa00700 1 -- 172.21.15.18:0/3014957557 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f37b4007cc0 con 0x7f37c40724f0 2024-04-12T21:33:27.037 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:27.036+0000 7f37baa00700 1 -- 172.21.15.18:0/3014957557 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f37b400f040 con 0x7f37c40724f0 2024-04-12T21:33:27.038 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:27.036+0000 7f37c3e00700 1 -- 172.21.15.18:0/3014957557 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f37c4114260 con 0x7f37c40724f0 2024-04-12T21:33:27.038 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:27.036+0000 7f37baa00700 1 -- 172.21.15.18:0/3014957557 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f37b4009640 con 0x7f37c40724f0 2024-04-12T21:33:27.038 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:27.036+0000 7f37c3e00700 1 -- 172.21.15.18:0/3014957557 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f37c41147a0 con 0x7f37c40724f0 2024-04-12T21:33:27.038 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:27.037+0000 7f37c3e00700 1 -- 172.21.15.18:0/3014957557 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f37c4073c30 con 0x7f37c40724f0 2024-04-12T21:33:27.039 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:27.039+0000 7f37baa00700 1 -- 172.21.15.18:0/3014957557 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7f37b40097a0 con 0x7f37c40724f0 2024-04-12T21:33:27.039 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:27.039+0000 7f37baa00700 1 --2- 172.21.15.18:0/3014957557 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f37a806ec10 0x7f37a80710c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:33:27.039 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:27.040+0000 7f37baa00700 1 -- 172.21.15.18:0/3014957557 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 6757+0+0 (secure 0 0 0) 0x7f37b408e6c0 con 0x7f37c40724f0 2024-04-12T21:33:27.040 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:27.040+0000 7f37c2000700 1 --2- 172.21.15.18:0/3014957557 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f37a806ec10 0x7f37a80710c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:33:27.040 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:27.040+0000 7f37c2000700 1 --2- 172.21.15.18:0/3014957557 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f37a806ec10 0x7f37a80710c0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f37c41158c0 tx=0x7f37b0009e30 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:33:27.044 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:27.044+0000 7f37baa00700 1 -- 172.21.15.18:0/3014957557 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f37b405af10 con 0x7f37c40724f0 2024-04-12T21:33:27.282 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:33:26 smithi186 ceph-mon[86379]: from='client.35768 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:33:27.282 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:33:26 smithi186 ceph-mon[86379]: from='client.35772 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:33:27.282 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:33:26 smithi186 ceph-mon[86379]: pgmap v1010: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:33:27.282 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:33:26 smithi186 ceph-mon[86379]: from='client.? 172.21.15.18:0/1494162877' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T21:33:27.282 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:33:26 smithi186 ceph-mon[86379]: from='client.? 172.21.15.18:0/2118093978' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-12T21:33:27.353 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:27.353+0000 7f37c3e00700 1 -- 172.21.15.18:0/3014957557 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f37c404bed0 con 0x7f37c40724f0 2024-04-12T21:33:27.353 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:33:26 smithi018 ceph-mon[112590]: from='client.35768 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:33:27.354 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:33:26 smithi018 ceph-mon[112590]: from='client.35772 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:33:27.354 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:33:26 smithi018 ceph-mon[112590]: pgmap v1010: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:33:27.354 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:33:26 smithi018 ceph-mon[112590]: from='client.? 172.21.15.18:0/1494162877' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T21:33:27.354 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:33:26 smithi018 ceph-mon[112590]: from='client.? 172.21.15.18:0/2118093978' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-12T21:33:27.354 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:27.354+0000 7f37baa00700 1 -- 172.21.15.18:0/3014957557 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+4303 (secure 0 0 0) 0x7f37b405aaa0 con 0x7f37c40724f0 2024-04-12T21:33:27.354 INFO:teuthology.orchestra.run.smithi018.stdout:HEALTH_ERR failed to probe daemons or devices; 1 filesystem is degraded; 1 filesystem is offline 2024-04-12T21:33:27.355 INFO:teuthology.orchestra.run.smithi018.stdout:[WRN] CEPHADM_REFRESH_FAILED: failed to probe daemons or devices 2024-04-12T21:33:27.355 INFO:teuthology.orchestra.run.smithi018.stdout: host smithi018 `cephadm ceph-volume` failed: cephadm exited with an error code: 1, stderr: Inferring config /var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/mon.smithi018/config 2024-04-12T21:33:27.355 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/podman: timeout after 895 seconds 2024-04-12T21:33:27.355 INFO:teuthology.orchestra.run.smithi018.stdout:Non-zero exit code 124 from /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:f4452def3be4d350f4a8e5feaf7ebb1511be9298fe68478ba7a2d6fbd7d69a63 -e NODE_NAME=smithi018 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968:/var/run/ceph:z -v /var/log/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968:/var/log/ceph:z -v /var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmpxsmwkkuh:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:f4452def3be4d350f4a8e5feaf7ebb1511be9298fe68478ba7a2d6fbd7d69a63 inventory --format=json-pretty --filter-for-batch 2024-04-12T21:33:27.355 INFO:teuthology.orchestra.run.smithi018.stdout:Traceback (most recent call last): 2024-04-12T21:33:27.355 INFO:teuthology.orchestra.run.smithi018.stdout: File "/usr/lib64/python3.9/runpy.py", line 197, in _run_module_as_main 2024-04-12T21:33:27.355 INFO:teuthology.orchestra.run.smithi018.stdout: return _run_code(code, main_globals, None, 2024-04-12T21:33:27.355 INFO:teuthology.orchestra.run.smithi018.stdout: File "/usr/lib64/python3.9/runpy.py", line 87, in _run_code 2024-04-12T21:33:27.355 INFO:teuthology.orchestra.run.smithi018.stdout: exec(code, run_globals) 2024-04-12T21:33:27.355 INFO:teuthology.orchestra.run.smithi018.stdout: File "/var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/cephadm.e44f3ed5e34e4ef52eedef233d1190bff004470d8e8005127b5312bff357fa7f/__main__.py", line 5618, in 2024-04-12T21:33:27.355 INFO:teuthology.orchestra.run.smithi018.stdout: File "/var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/cephadm.e44f3ed5e34e4ef52eedef233d1190bff004470d8e8005127b5312bff357fa7f/__main__.py", line 5606, in main 2024-04-12T21:33:27.356 INFO:teuthology.orchestra.run.smithi018.stdout: File "/var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/cephadm.e44f3ed5e34e4ef52eedef233d1190bff004470d8e8005127b5312bff357fa7f/__main__.py", line 408, in _infer_config 2024-04-12T21:33:27.356 INFO:teuthology.orchestra.run.smithi018.stdout: File "/var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/cephadm.e44f3ed5e34e4ef52eedef233d1190bff004470d8e8005127b5312bff357fa7f/__main__.py", line 323, in _infer_fsid 2024-04-12T21:33:27.356 INFO:teuthology.orchestra.run.smithi018.stdout: File "/var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/cephadm.e44f3ed5e34e4ef52eedef233d1190bff004470d8e8005127b5312bff357fa7f/__main__.py", line 436, in _infer_image 2024-04-12T21:33:27.356 INFO:teuthology.orchestra.run.smithi018.stdout: File "/var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/cephadm.e44f3ed5e34e4ef52eedef233d1190bff004470d8e8005127b5312bff357fa7f/__main__.py", line 310, in _validate_fsid 2024-04-12T21:33:27.356 INFO:teuthology.orchestra.run.smithi018.stdout: File "/var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/cephadm.e44f3ed5e34e4ef52eedef233d1190bff004470d8e8005127b5312bff357fa7f/__main__.py", line 3359, in command_ceph_volume 2024-04-12T21:33:27.356 INFO:teuthology.orchestra.run.smithi018.stdout: File "/var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/cephadm.e44f3ed5e34e4ef52eedef233d1190bff004470d8e8005127b5312bff357fa7f/cephadmlib/call_wrappers.py", line 310, in call_throws 2024-04-12T21:33:27.356 INFO:teuthology.orchestra.run.smithi018.stdout:RuntimeError: Failed command: /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:f4452def3be4d350f4a8e5feaf7ebb1511be9298fe68478ba7a2d6fbd7d69a63 -e NODE_NAME=smithi018 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968:/var/run/ceph:z -v /var/log/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968:/var/log/ceph:z -v /var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmpxsmwkkuh:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:f4452def3be4d350f4a8e5feaf7ebb1511be9298fe68478ba7a2d6fbd7d69a63 inventory --format=json-pretty --filter-for-batch 2024-04-12T21:33:27.356 INFO:teuthology.orchestra.run.smithi018.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-04-12T21:33:27.356 INFO:teuthology.orchestra.run.smithi018.stdout: fs cephfs is degraded 2024-04-12T21:33:27.356 INFO:teuthology.orchestra.run.smithi018.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-04-12T21:33:27.356 INFO:teuthology.orchestra.run.smithi018.stdout: fs cephfs is offline because no MDS is active for it. 2024-04-12T21:33:27.357 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:27.357+0000 7f37c3e00700 1 -- 172.21.15.18:0/3014957557 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f37a806ec10 msgr2=0x7f37a80710c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:33:27.357 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:27.357+0000 7f37c3e00700 1 --2- 172.21.15.18:0/3014957557 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f37a806ec10 0x7f37a80710c0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f37c41158c0 tx=0x7f37b0009e30 comp rx=0 tx=0).stop 2024-04-12T21:33:27.357 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:27.357+0000 7f37c3e00700 1 -- 172.21.15.18:0/3014957557 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f37c40724f0 msgr2=0x7f37c411a020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:33:27.357 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:27.357+0000 7f37c3e00700 1 --2- 172.21.15.18:0/3014957557 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f37c40724f0 0x7f37c411a020 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7f37b4002a50 tx=0x7f37b40046b0 comp rx=0 tx=0).stop 2024-04-12T21:33:27.358 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:27.357+0000 7f37c3e00700 1 -- 172.21.15.18:0/3014957557 shutdown_connections 2024-04-12T21:33:27.358 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:27.357+0000 7f37c3e00700 1 --2- 172.21.15.18:0/3014957557 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f37a806ec10 0x7f37a80710c0 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:33:27.358 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:27.357+0000 7f37c3e00700 1 --2- 172.21.15.18:0/3014957557 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f37c40724f0 0x7f37c411a020 unknown :-1 s=CLOSED pgs=861 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:33:27.358 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:27.357+0000 7f37c3e00700 1 --2- 172.21.15.18:0/3014957557 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f37c4072e00 0x7f37c411a560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:33:27.358 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:27.357+0000 7f37c3e00700 1 -- 172.21.15.18:0/3014957557 >> 172.21.15.18:0/3014957557 conn(0x7f37c40fab10 msgr2=0x7f37c40fafa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:33:27.358 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:27.357+0000 7f37c3e00700 1 -- 172.21.15.18:0/3014957557 shutdown_connections 2024-04-12T21:33:27.358 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:27.358+0000 7f37c3e00700 1 -- 172.21.15.18:0/3014957557 wait complete. 2024-04-12T21:33:28.282 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:33:27 smithi186 ceph-mon[86379]: from='client.35784 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:33:28.282 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:33:27 smithi186 ceph-mon[86379]: from='client.? 172.21.15.18:0/3014957557' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-12T21:33:28.353 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:33:27 smithi018 ceph-mon[112590]: from='client.35784 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:33:28.353 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:33:27 smithi018 ceph-mon[112590]: from='client.? 172.21.15.18:0/3014957557' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-12T21:33:29.282 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:33:28 smithi186 ceph-mon[86379]: pgmap v1011: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:33:29.353 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:33:28 smithi018 ceph-mon[112590]: pgmap v1011: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:33:31.282 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:33:30 smithi186 ceph-mon[86379]: pgmap v1012: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:33:31.353 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:33:30 smithi018 ceph-mon[112590]: pgmap v1012: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:33:32.282 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:33:31 smithi186 ceph-mon[86379]: pgmap v1013: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:33:32.353 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:33:31 smithi018 ceph-mon[112590]: pgmap v1013: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:33:35.032 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:33:34 smithi186 ceph-mon[86379]: pgmap v1014: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:33:35.103 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:33:34 smithi018 ceph-mon[112590]: pgmap v1014: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:33:37.032 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:33:36 smithi186 ceph-mon[86379]: pgmap v1015: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:33:37.103 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:33:36 smithi018 ceph-mon[112590]: pgmap v1015: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:33:39.032 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:33:38 smithi186 ceph-mon[86379]: pgmap v1016: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:33:39.103 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:33:38 smithi018 ceph-mon[112590]: pgmap v1016: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:33:41.032 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:33:40 smithi186 ceph-mon[86379]: pgmap v1017: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:33:41.032 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:33:40 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T21:33:41.103 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:33:40 smithi018 ceph-mon[112590]: pgmap v1017: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:33:41.103 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:33:40 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T21:33:43.103 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:33:42 smithi018 ceph-mon[112590]: pgmap v1018: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:33:43.282 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:33:42 smithi186 ceph-mon[86379]: pgmap v1018: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:33:45.085 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:33:44 smithi186 ceph-mon[86379]: pgmap v1019: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:33:45.103 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:33:44 smithi018 ceph-mon[112590]: pgmap v1019: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:33:47.103 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:33:46 smithi018 ceph-mon[112590]: pgmap v1020: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:33:47.282 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:33:46 smithi186 ceph-mon[86379]: pgmap v1020: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:33:49.103 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:33:48 smithi018 ceph-mon[112590]: pgmap v1021: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:33:49.282 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:33:48 smithi186 ceph-mon[86379]: pgmap v1021: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:33:51.103 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:33:50 smithi018 ceph-mon[112590]: pgmap v1022: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:33:51.282 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:33:50 smithi186 ceph-mon[86379]: pgmap v1022: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:33:53.103 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:33:52 smithi018 ceph-mon[112590]: pgmap v1023: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:33:53.282 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:33:52 smithi186 ceph-mon[86379]: pgmap v1023: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:33:55.103 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:33:54 smithi018 ceph-mon[112590]: pgmap v1024: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:33:55.282 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:33:54 smithi186 ceph-mon[86379]: pgmap v1024: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:33:56.103 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:33:55 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T21:33:56.282 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:33:55 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T21:33:57.103 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:33:56 smithi018 ceph-mon[112590]: pgmap v1025: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:33:57.282 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:33:56 smithi186 ceph-mon[86379]: pgmap v1025: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:33:57.530 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:57.531+0000 7f8149000700 1 -- 172.21.15.18:0/776158394 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8144071f90 msgr2=0x7f8144070360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:33:57.530 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:57.531+0000 7f8149000700 1 --2- 172.21.15.18:0/776158394 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8144071f90 0x7f8144070360 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7f8134004970 tx=0x7f8134009e30 comp rx=0 tx=0).stop 2024-04-12T21:33:57.530 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:57.531+0000 7f8149000700 1 -- 172.21.15.18:0/776158394 shutdown_connections 2024-04-12T21:33:57.530 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:57.531+0000 7f8149000700 1 --2- 172.21.15.18:0/776158394 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f81440708a0 0x7f8144070d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:33:57.531 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:57.531+0000 7f8149000700 1 --2- 172.21.15.18:0/776158394 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8144071f90 0x7f8144070360 unknown :-1 s=CLOSED pgs=862 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:33:57.531 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:57.531+0000 7f8149000700 1 -- 172.21.15.18:0/776158394 >> 172.21.15.18:0/776158394 conn(0x7f81440f8950 msgr2=0x7f81440fad60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:33:57.531 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:57.531+0000 7f8149000700 1 -- 172.21.15.18:0/776158394 shutdown_connections 2024-04-12T21:33:57.531 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:57.531+0000 7f8149000700 1 -- 172.21.15.18:0/776158394 wait complete. 2024-04-12T21:33:57.532 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:57.532+0000 7f8149000700 1 Processor -- start 2024-04-12T21:33:57.532 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:57.532+0000 7f8149000700 1 -- start start 2024-04-12T21:33:57.532 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:57.533+0000 7f8149000700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f81440708a0 0x7f8144119f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:33:57.532 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:57.533+0000 7f8149000700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8144071f90 0x7f814411a490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:33:57.532 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:57.533+0000 7f8149000700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f814411aae0 con 0x7f8144071f90 2024-04-12T21:33:57.532 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:57.533+0000 7f8149000700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81441140c0 con 0x7f81440708a0 2024-04-12T21:33:57.533 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:57.533+0000 7f8141400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8144071f90 0x7f814411a490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:33:57.533 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:57.533+0000 7f8141400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8144071f90 0x7f814411a490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:34930/0 (socket says 172.21.15.18:34930) 2024-04-12T21:33:57.533 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:57.533+0000 7f8141400700 1 -- 172.21.15.18:0/1753842008 learned_addr learned my addr 172.21.15.18:0/1753842008 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:33:57.533 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:57.533+0000 7f8141e00700 1 --2- 172.21.15.18:0/1753842008 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f81440708a0 0x7f8144119f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:33:57.533 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:57.533+0000 7f8141400700 1 -- 172.21.15.18:0/1753842008 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f81440708a0 msgr2=0x7f8144119f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:33:57.533 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:57.533+0000 7f8141400700 1 --2- 172.21.15.18:0/1753842008 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f81440708a0 0x7f8144119f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:33:57.533 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:57.534+0000 7f8141400700 1 -- 172.21.15.18:0/1753842008 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8134005040 con 0x7f8144071f90 2024-04-12T21:33:57.534 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:57.534+0000 7f8141400700 1 --2- 172.21.15.18:0/1753842008 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8144071f90 0x7f814411a490 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7f8130002a50 tx=0x7f81300046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:33:57.534 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:57.534+0000 7f813a000700 1 -- 172.21.15.18:0/1753842008 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8130007cc0 con 0x7f8144071f90 2024-04-12T21:33:57.534 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:57.534+0000 7f813a000700 1 -- 172.21.15.18:0/1753842008 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f813000f040 con 0x7f8144071f90 2024-04-12T21:33:57.534 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:57.534+0000 7f8149000700 1 -- 172.21.15.18:0/1753842008 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f81441143a0 con 0x7f8144071f90 2024-04-12T21:33:57.534 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:57.534+0000 7f813a000700 1 -- 172.21.15.18:0/1753842008 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8130009640 con 0x7f8144071f90 2024-04-12T21:33:57.534 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:57.534+0000 7f8149000700 1 -- 172.21.15.18:0/1753842008 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f81441148e0 con 0x7f8144071f90 2024-04-12T21:33:57.534 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:57.536+0000 7f8149000700 1 -- 172.21.15.18:0/1753842008 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8144071aa0 con 0x7f8144071f90 2024-04-12T21:33:57.535 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:57.537+0000 7f813a000700 1 -- 172.21.15.18:0/1753842008 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7f81300097a0 con 0x7f8144071f90 2024-04-12T21:33:57.536 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:57.538+0000 7f813a000700 1 --2- 172.21.15.18:0/1753842008 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f812806ec00 0x7f81280710b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:33:57.537 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:57.538+0000 7f813a000700 1 -- 172.21.15.18:0/1753842008 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 6757+0+0 (secure 0 0 0) 0x7f813008e6c0 con 0x7f8144071f90 2024-04-12T21:33:57.537 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:57.538+0000 7f8141e00700 1 --2- 172.21.15.18:0/1753842008 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f812806ec00 0x7f81280710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:33:57.537 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:57.539+0000 7f8141e00700 1 --2- 172.21.15.18:0/1753842008 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f812806ec00 0x7f81280710b0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f8134009fd0 tx=0x7f8134009e30 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:33:57.541 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:57.543+0000 7f813a000700 1 -- 172.21.15.18:0/1753842008 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f813005af10 con 0x7f8144071f90 2024-04-12T21:33:57.804 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:57.805+0000 7f8149000700 1 -- 172.21.15.18:0/1753842008 --> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f8144110570 con 0x7f812806ec00 2024-04-12T21:33:57.806 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:57.807+0000 7f813a000700 1 -- 172.21.15.18:0/1753842008 <== mgr.34102 v2:172.21.15.18:6800/3812891919 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f8144110570 con 0x7f812806ec00 2024-04-12T21:33:57.810 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:57.811+0000 7f8149000700 1 -- 172.21.15.18:0/1753842008 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f812806ec00 msgr2=0x7f81280710b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:33:57.810 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:57.811+0000 7f8149000700 1 --2- 172.21.15.18:0/1753842008 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f812806ec00 0x7f81280710b0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f8134009fd0 tx=0x7f8134009e30 comp rx=0 tx=0).stop 2024-04-12T21:33:57.810 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:57.811+0000 7f8149000700 1 -- 172.21.15.18:0/1753842008 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8144071f90 msgr2=0x7f814411a490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:33:57.810 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:57.811+0000 7f8149000700 1 --2- 172.21.15.18:0/1753842008 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8144071f90 0x7f814411a490 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7f8130002a50 tx=0x7f81300046b0 comp rx=0 tx=0).stop 2024-04-12T21:33:57.810 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:57.811+0000 7f8149000700 1 -- 172.21.15.18:0/1753842008 shutdown_connections 2024-04-12T21:33:57.810 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:57.811+0000 7f8149000700 1 --2- 172.21.15.18:0/1753842008 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f812806ec00 0x7f81280710b0 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:33:57.810 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:57.811+0000 7f8149000700 1 --2- 172.21.15.18:0/1753842008 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f81440708a0 0x7f8144119f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:33:57.810 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:57.811+0000 7f8149000700 1 --2- 172.21.15.18:0/1753842008 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8144071f90 0x7f814411a490 unknown :-1 s=CLOSED pgs=863 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:33:57.811 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:57.811+0000 7f8149000700 1 -- 172.21.15.18:0/1753842008 >> 172.21.15.18:0/1753842008 conn(0x7f81440f8950 msgr2=0x7f81440f9a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:33:57.811 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:57.812+0000 7f8149000700 1 -- 172.21.15.18:0/1753842008 shutdown_connections 2024-04-12T21:33:57.811 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:57.812+0000 7f8149000700 1 -- 172.21.15.18:0/1753842008 wait complete. 2024-04-12T21:33:57.827 INFO:teuthology.orchestra.run.smithi018.stdout:true 2024-04-12T21:33:57.975 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:57.976+0000 7f5558200700 1 -- 172.21.15.18:0/2627719331 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5550109dd0 msgr2=0x7f5550103440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:33:57.975 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:57.976+0000 7f5558200700 1 --2- 172.21.15.18:0/2627719331 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5550109dd0 0x7f5550103440 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7f553c004940 tx=0x7f553c009e30 comp rx=0 tx=0).stop 2024-04-12T21:33:57.975 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:57.976+0000 7f5558200700 1 -- 172.21.15.18:0/2627719331 shutdown_connections 2024-04-12T21:33:57.975 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:57.976+0000 7f5558200700 1 --2- 172.21.15.18:0/2627719331 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5550109dd0 0x7f5550103440 unknown :-1 s=CLOSED pgs=864 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:33:57.975 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:57.976+0000 7f5558200700 1 --2- 172.21.15.18:0/2627719331 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5550109430 0x7f5550109800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:33:57.975 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:57.976+0000 7f5558200700 1 -- 172.21.15.18:0/2627719331 >> 172.21.15.18:0/2627719331 conn(0x7f5550072c00 msgr2=0x7f5550075010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:33:57.975 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:57.976+0000 7f5558200700 1 -- 172.21.15.18:0/2627719331 shutdown_connections 2024-04-12T21:33:57.975 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:57.976+0000 7f5558200700 1 -- 172.21.15.18:0/2627719331 wait complete. 2024-04-12T21:33:57.976 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:57.977+0000 7f5558200700 1 Processor -- start 2024-04-12T21:33:57.976 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:57.977+0000 7f5558200700 1 -- start start 2024-04-12T21:33:57.976 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:57.978+0000 7f5558200700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5550109430 0x7f5550119f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:33:57.976 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:57.978+0000 7f5558200700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5550109dd0 0x7f555011a4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:33:57.976 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:57.978+0000 7f5558200700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f555011ab10 con 0x7f5550109dd0 2024-04-12T21:33:57.976 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:57.978+0000 7f5558200700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55501140f0 con 0x7f5550109430 2024-04-12T21:33:57.978 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:57.978+0000 7f5554e00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5550109dd0 0x7f555011a4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:33:57.978 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:57.978+0000 7f5554e00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5550109dd0 0x7f555011a4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:34946/0 (socket says 172.21.15.18:34946) 2024-04-12T21:33:57.978 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:57.978+0000 7f5554e00700 1 -- 172.21.15.18:0/391526947 learned_addr learned my addr 172.21.15.18:0/391526947 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:33:57.978 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:57.978+0000 7f5555800700 1 --2- 172.21.15.18:0/391526947 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5550109430 0x7f5550119f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:33:57.978 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:57.978+0000 7f5554e00700 1 -- 172.21.15.18:0/391526947 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5550109430 msgr2=0x7f5550119f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:33:57.978 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:57.978+0000 7f5554e00700 1 --2- 172.21.15.18:0/391526947 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5550109430 0x7f5550119f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:33:57.978 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:57.978+0000 7f5554e00700 1 -- 172.21.15.18:0/391526947 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f553c005040 con 0x7f5550109dd0 2024-04-12T21:33:57.978 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:57.978+0000 7f5555800700 1 --2- 172.21.15.18:0/391526947 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5550109430 0x7f5550119f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-12T21:33:57.978 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:57.979+0000 7f5554e00700 1 --2- 172.21.15.18:0/391526947 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5550109dd0 0x7f555011a4c0 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7f553c009100 tx=0x7f553c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:33:57.978 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:57.979+0000 7f554d600700 1 -- 172.21.15.18:0/391526947 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f553c00e040 con 0x7f5550109dd0 2024-04-12T21:33:57.979 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:57.979+0000 7f554d600700 1 -- 172.21.15.18:0/391526947 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f553c013420 con 0x7f5550109dd0 2024-04-12T21:33:57.979 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:57.979+0000 7f554d600700 1 -- 172.21.15.18:0/391526947 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f553c01c630 con 0x7f5550109dd0 2024-04-12T21:33:57.979 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:57.979+0000 7f5558200700 1 -- 172.21.15.18:0/391526947 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5550114370 con 0x7f5550109dd0 2024-04-12T21:33:57.979 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:57.979+0000 7f5558200700 1 -- 172.21.15.18:0/391526947 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5550114930 con 0x7f5550109dd0 2024-04-12T21:33:57.979 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:57.980+0000 7f5558200700 1 -- 172.21.15.18:0/391526947 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5550104b80 con 0x7f5550109dd0 2024-04-12T21:33:57.980 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:57.981+0000 7f554d600700 1 -- 172.21.15.18:0/391526947 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7f553c013590 con 0x7f5550109dd0 2024-04-12T21:33:57.981 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:57.982+0000 7f554d600700 1 --2- 172.21.15.18:0/391526947 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f553806ec00 0x7f55380710b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:33:57.981 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:57.982+0000 7f554d600700 1 -- 172.21.15.18:0/391526947 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 6757+0+0 (secure 0 0 0) 0x7f553c08fba0 con 0x7f5550109dd0 2024-04-12T21:33:57.981 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:57.983+0000 7f5555800700 1 --2- 172.21.15.18:0/391526947 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f553806ec00 0x7f55380710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:33:57.982 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:57.983+0000 7f5555800700 1 --2- 172.21.15.18:0/391526947 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f553806ec00 0x7f55380710b0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f5544007fd0 tx=0x7f5544009040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:33:57.986 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:57.987+0000 7f554d600700 1 -- 172.21.15.18:0/391526947 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f553c05c3d0 con 0x7f5550109dd0 2024-04-12T21:33:58.239 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:58.240+0000 7f5558200700 1 -- 172.21.15.18:0/391526947 --> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f5550110500 con 0x7f553806ec00 2024-04-12T21:33:58.241 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:58.243+0000 7f554d600700 1 -- 172.21.15.18:0/391526947 <== mgr.34102 v2:172.21.15.18:6800/3812891919 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f5550110500 con 0x7f553806ec00 2024-04-12T21:33:58.244 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:58.245+0000 7f5558200700 1 -- 172.21.15.18:0/391526947 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f553806ec00 msgr2=0x7f55380710b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:33:58.244 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:58.246+0000 7f5558200700 1 --2- 172.21.15.18:0/391526947 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f553806ec00 0x7f55380710b0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f5544007fd0 tx=0x7f5544009040 comp rx=0 tx=0).stop 2024-04-12T21:33:58.245 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:58.246+0000 7f5558200700 1 -- 172.21.15.18:0/391526947 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5550109dd0 msgr2=0x7f555011a4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:33:58.245 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:58.246+0000 7f5558200700 1 --2- 172.21.15.18:0/391526947 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5550109dd0 0x7f555011a4c0 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7f553c009100 tx=0x7f553c009e30 comp rx=0 tx=0).stop 2024-04-12T21:33:58.245 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:58.246+0000 7f5558200700 1 -- 172.21.15.18:0/391526947 shutdown_connections 2024-04-12T21:33:58.245 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:58.246+0000 7f5558200700 1 --2- 172.21.15.18:0/391526947 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f553806ec00 0x7f55380710b0 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:33:58.245 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:58.246+0000 7f5558200700 1 --2- 172.21.15.18:0/391526947 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5550109430 0x7f5550119f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:33:58.245 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:58.246+0000 7f5558200700 1 --2- 172.21.15.18:0/391526947 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5550109dd0 0x7f555011a4c0 unknown :-1 s=CLOSED pgs=865 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:33:58.246 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:58.246+0000 7f5558200700 1 -- 172.21.15.18:0/391526947 >> 172.21.15.18:0/391526947 conn(0x7f5550072c00 msgr2=0x7f5550074c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:33:58.246 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:58.246+0000 7f5558200700 1 -- 172.21.15.18:0/391526947 shutdown_connections 2024-04-12T21:33:58.246 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:58.246+0000 7f5558200700 1 -- 172.21.15.18:0/391526947 wait complete. 2024-04-12T21:33:58.408 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:58.409+0000 7f758f200700 1 -- 172.21.15.18:0/2165266966 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7588070360 msgr2=0x7f7588070730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:33:58.408 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:58.409+0000 7f758f200700 1 --2- 172.21.15.18:0/2165266966 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7588070360 0x7f7588070730 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7f7578004970 tx=0x7f7578009e30 comp rx=0 tx=0).stop 2024-04-12T21:33:58.409 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:58.409+0000 7f758f200700 1 -- 172.21.15.18:0/2165266966 shutdown_connections 2024-04-12T21:33:58.409 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:58.409+0000 7f758f200700 1 --2- 172.21.15.18:0/2165266966 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7588070c70 0x7f7588109110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:33:58.409 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:58.409+0000 7f758f200700 1 --2- 172.21.15.18:0/2165266966 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7588070360 0x7f7588070730 unknown :-1 s=CLOSED pgs=866 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:33:58.409 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:58.409+0000 7f758f200700 1 -- 172.21.15.18:0/2165266966 >> 172.21.15.18:0/2165266966 conn(0x7f75880f89b0 msgr2=0x7f75880fadc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:33:58.409 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:58.410+0000 7f758f200700 1 -- 172.21.15.18:0/2165266966 shutdown_connections 2024-04-12T21:33:58.409 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:58.410+0000 7f758f200700 1 -- 172.21.15.18:0/2165266966 wait complete. 2024-04-12T21:33:58.409 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:58.411+0000 7f758f200700 1 Processor -- start 2024-04-12T21:33:58.409 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:58.411+0000 7f758f200700 1 -- start start 2024-04-12T21:33:58.410 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:58.411+0000 7f758f200700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7588070360 0x7f7588115b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:33:58.410 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:58.411+0000 7f758f200700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7588070c70 0x7f75881160a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:33:58.410 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:58.411+0000 7f758f200700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75881166f0 con 0x7f7588070360 2024-04-12T21:33:58.411 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:58.411+0000 7f758f200700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f758810fcd0 con 0x7f7588070c70 2024-04-12T21:33:58.411 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:58.412+0000 7f7587e00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7588070360 0x7f7588115b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:33:58.411 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:58.412+0000 7f7587e00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7588070360 0x7f7588115b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:34960/0 (socket says 172.21.15.18:34960) 2024-04-12T21:33:58.411 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:58.412+0000 7f7587e00700 1 -- 172.21.15.18:0/367291681 learned_addr learned my addr 172.21.15.18:0/367291681 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:33:58.412 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:58.412+0000 7f7587400700 1 --2- 172.21.15.18:0/367291681 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7588070c70 0x7f75881160a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:33:58.412 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:58.412+0000 7f7587e00700 1 -- 172.21.15.18:0/367291681 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7588070c70 msgr2=0x7f75881160a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:33:58.412 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:58.412+0000 7f7587e00700 1 --2- 172.21.15.18:0/367291681 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7588070c70 0x7f75881160a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:33:58.412 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:58.412+0000 7f7587e00700 1 -- 172.21.15.18:0/367291681 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7578005040 con 0x7f7588070360 2024-04-12T21:33:58.412 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:58.412+0000 7f7587400700 1 --2- 172.21.15.18:0/367291681 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7588070c70 0x7f75881160a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-12T21:33:58.412 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:58.413+0000 7f7587e00700 1 --2- 172.21.15.18:0/367291681 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7588070360 0x7f7588115b60 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7f7578000c00 tx=0x7f7578009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:33:58.412 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:58.413+0000 7f757fe00700 1 -- 172.21.15.18:0/367291681 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f757800e040 con 0x7f7588070360 2024-04-12T21:33:58.412 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:58.413+0000 7f757fe00700 1 -- 172.21.15.18:0/367291681 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f757800bc10 con 0x7f7588070360 2024-04-12T21:33:58.413 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:58.413+0000 7f758f200700 1 -- 172.21.15.18:0/367291681 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f758810ff50 con 0x7f7588070360 2024-04-12T21:33:58.413 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:58.413+0000 7f757fe00700 1 -- 172.21.15.18:0/367291681 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f757801c3f0 con 0x7f7588070360 2024-04-12T21:33:58.413 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:58.413+0000 7f758f200700 1 -- 172.21.15.18:0/367291681 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7588110490 con 0x7f7588070360 2024-04-12T21:33:58.413 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:58.414+0000 7f758f200700 1 -- 172.21.15.18:0/367291681 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7588071aa0 con 0x7f7588070360 2024-04-12T21:33:58.414 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:58.415+0000 7f757fe00700 1 -- 172.21.15.18:0/367291681 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7f757800bd80 con 0x7f7588070360 2024-04-12T21:33:58.416 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:58.417+0000 7f757fe00700 1 --2- 172.21.15.18:0/367291681 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f756c06ec00 0x7f756c0710b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:33:58.416 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:58.417+0000 7f757fe00700 1 -- 172.21.15.18:0/367291681 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 6757+0+0 (secure 0 0 0) 0x7f757808fdb0 con 0x7f7588070360 2024-04-12T21:33:58.416 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:58.418+0000 7f7587400700 1 --2- 172.21.15.18:0/367291681 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f756c06ec00 0x7f756c0710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:33:58.417 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:58.418+0000 7f7587400700 1 --2- 172.21.15.18:0/367291681 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f756c06ec00 0x7f756c0710b0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f7574009fd0 tx=0x7f757400b040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:33:58.421 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:58.422+0000 7f757fe00700 1 -- 172.21.15.18:0/367291681 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f757805c560 con 0x7f7588070360 2024-04-12T21:33:58.662 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:58.663+0000 7f758f200700 1 -- 172.21.15.18:0/367291681 --> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f7588071aa0 con 0x7f756c06ec00 2024-04-12T21:33:58.692 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:58.693+0000 7f757fe00700 1 -- 172.21.15.18:0/367291681 <== mgr.34102 v2:172.21.15.18:6800/3812891919 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7f7588071aa0 con 0x7f756c06ec00 2024-04-12T21:33:58.692 INFO:teuthology.orchestra.run.smithi018.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-12T21:33:58.692 INFO:teuthology.orchestra.run.smithi018.stdout:alertmanager.smithi018 smithi018 *:9093,9094 running (34m) 10m ago 49m 15.5M - 0.25.0 c8568f914cd2 c332090068e8 2024-04-12T21:33:58.692 INFO:teuthology.orchestra.run.smithi018.stdout:crash.smithi018 smithi018 running (30m) 10m ago 49m 6665k - 19.0.0-2879-g324baf8d 8717720dc5e5 b7f80d1f7982 2024-04-12T21:33:58.693 INFO:teuthology.orchestra.run.smithi018.stdout:crash.smithi186 smithi186 running (30m) 10m ago 47m 6660k - 19.0.0-2879-g324baf8d 8717720dc5e5 a4f55c81b623 2024-04-12T21:33:58.693 INFO:teuthology.orchestra.run.smithi018.stdout:grafana.smithi018 smithi018 *:3000 running (33m) 10m ago 48m 74.8M - 9.4.12 2bacad6d85d8 b08e3bf176dd 2024-04-12T21:33:58.693 INFO:teuthology.orchestra.run.smithi018.stdout:mds.cephfs.smithi018.acqbsc smithi018 running (26m) 10m ago 41m 16.7M - 19.0.0-2879-g324baf8d 8717720dc5e5 021f99ab24ae 2024-04-12T21:33:58.693 INFO:teuthology.orchestra.run.smithi018.stdout:mds.cephfs.smithi018.szjluo smithi018 running (26m) 10m ago 41m 16.5M - 19.0.0-2879-g324baf8d 8717720dc5e5 0003117479d4 2024-04-12T21:33:58.693 INFO:teuthology.orchestra.run.smithi018.stdout:mds.cephfs.smithi186.jdllyn smithi186 running (11m) 10m ago 41m 11.0M - 19.0.0-2879-g324baf8d 8717720dc5e5 21b1f46eef23 2024-04-12T21:33:58.693 INFO:teuthology.orchestra.run.smithi018.stdout:mds.cephfs.smithi186.wjnafs smithi186 running (41m) 10m ago 41m 19.5M - 17.2.7-827-gd3b5060a 21de9b7369ad 74b115d6d6fd 2024-04-12T21:33:58.693 INFO:teuthology.orchestra.run.smithi018.stdout:mgr.smithi018.xjzwga smithi018 *:8443,9283,8765 running (37m) 10m ago 50m 512M - 19.0.0-2879-g324baf8d 8717720dc5e5 56088305eab7 2024-04-12T21:33:58.693 INFO:teuthology.orchestra.run.smithi018.stdout:mgr.smithi186.kqcfxk smithi186 *:8443,9283,8765 running (35m) 10m ago 47m 459M - 19.0.0-2879-g324baf8d 8717720dc5e5 09cd766e5e0d 2024-04-12T21:33:58.693 INFO:teuthology.orchestra.run.smithi018.stdout:mon.smithi018 smithi018 running (32m) 10m ago 50m 73.1M 2048M 19.0.0-2879-g324baf8d 8717720dc5e5 df3e201dfed6 2024-04-12T21:33:58.693 INFO:teuthology.orchestra.run.smithi018.stdout:mon.smithi186 smithi186 running (31m) 10m ago 47m 47.2M 2048M 19.0.0-2879-g324baf8d 8717720dc5e5 c77498da9b2b 2024-04-12T21:33:58.693 INFO:teuthology.orchestra.run.smithi018.stdout:node-exporter.smithi018 smithi018 *:9100 running (35m) 10m ago 49m 8108k - 1.5.0 0da6a335fe13 326c3d9c6847 2024-04-12T21:33:58.693 INFO:teuthology.orchestra.run.smithi018.stdout:node-exporter.smithi186 smithi186 *:9100 running (35m) 10m ago 47m 8757k - 1.5.0 0da6a335fe13 ec2112baeecf 2024-04-12T21:33:58.693 INFO:teuthology.orchestra.run.smithi018.stdout:osd.0 smithi018 running (30m) 10m ago 46m 123M 1559M 19.0.0-2879-g324baf8d 8717720dc5e5 60b1273ca047 2024-04-12T21:33:58.693 INFO:teuthology.orchestra.run.smithi018.stdout:osd.1 smithi018 running (29m) 10m ago 46m 95.1M 1559M 19.0.0-2879-g324baf8d 8717720dc5e5 170ce1e5fb8d 2024-04-12T21:33:58.693 INFO:teuthology.orchestra.run.smithi018.stdout:osd.2 smithi018 running (29m) 10m ago 45m 94.3M 1559M 19.0.0-2879-g324baf8d 8717720dc5e5 c2cbc6aa2839 2024-04-12T21:33:58.694 INFO:teuthology.orchestra.run.smithi018.stdout:osd.3 smithi186 running (28m) 10m ago 44m 101M 2581M 19.0.0-2879-g324baf8d 8717720dc5e5 0138e8262e4b 2024-04-12T21:33:58.694 INFO:teuthology.orchestra.run.smithi018.stdout:osd.4 smithi186 running (27m) 10m ago 44m 91.0M 2581M 19.0.0-2879-g324baf8d 8717720dc5e5 b20cb63b25c9 2024-04-12T21:33:58.694 INFO:teuthology.orchestra.run.smithi018.stdout:osd.5 smithi186 running (27m) 10m ago 43m 92.0M 2581M 19.0.0-2879-g324baf8d 8717720dc5e5 2513bea2610e 2024-04-12T21:33:58.694 INFO:teuthology.orchestra.run.smithi018.stdout:prometheus.smithi018 smithi018 *:9095 running (35m) 10m ago 48m 67.9M - 2.43.0 a07b618ecd1d f58841127e21 2024-04-12T21:33:58.695 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:58.696+0000 7f758f200700 1 -- 172.21.15.18:0/367291681 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f756c06ec00 msgr2=0x7f756c0710b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:33:58.695 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:58.696+0000 7f758f200700 1 --2- 172.21.15.18:0/367291681 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f756c06ec00 0x7f756c0710b0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f7574009fd0 tx=0x7f757400b040 comp rx=0 tx=0).stop 2024-04-12T21:33:58.695 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:58.696+0000 7f758f200700 1 -- 172.21.15.18:0/367291681 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7588070360 msgr2=0x7f7588115b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:33:58.695 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:58.696+0000 7f758f200700 1 --2- 172.21.15.18:0/367291681 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7588070360 0x7f7588115b60 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7f7578000c00 tx=0x7f7578009e30 comp rx=0 tx=0).stop 2024-04-12T21:33:58.695 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:58.696+0000 7f758f200700 1 -- 172.21.15.18:0/367291681 shutdown_connections 2024-04-12T21:33:58.695 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:58.696+0000 7f758f200700 1 --2- 172.21.15.18:0/367291681 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f756c06ec00 0x7f756c0710b0 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:33:58.695 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:58.696+0000 7f758f200700 1 --2- 172.21.15.18:0/367291681 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7588070360 0x7f7588115b60 unknown :-1 s=CLOSED pgs=867 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:33:58.696 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:58.696+0000 7f758f200700 1 --2- 172.21.15.18:0/367291681 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7588070c70 0x7f75881160a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:33:58.696 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:58.696+0000 7f758f200700 1 -- 172.21.15.18:0/367291681 >> 172.21.15.18:0/367291681 conn(0x7f75880f89b0 msgr2=0x7f75880f9de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:33:58.696 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:58.696+0000 7f758f200700 1 -- 172.21.15.18:0/367291681 shutdown_connections 2024-04-12T21:33:58.696 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:58.697+0000 7f758f200700 1 -- 172.21.15.18:0/367291681 wait complete. 2024-04-12T21:33:58.839 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:58.840+0000 7f6ef6e00700 1 -- 172.21.15.18:0/532574836 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6ef0070360 msgr2=0x7f6ef0070730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:33:58.839 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:58.840+0000 7f6ef6e00700 1 --2- 172.21.15.18:0/532574836 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6ef0070360 0x7f6ef0070730 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7f6ee0004970 tx=0x7f6ee0009e30 comp rx=0 tx=0).stop 2024-04-12T21:33:58.839 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:58.840+0000 7f6ef6e00700 1 -- 172.21.15.18:0/532574836 shutdown_connections 2024-04-12T21:33:58.839 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:58.840+0000 7f6ef6e00700 1 --2- 172.21.15.18:0/532574836 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6ef0070c70 0x7f6ef0109110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:33:58.839 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:58.840+0000 7f6ef6e00700 1 --2- 172.21.15.18:0/532574836 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6ef0070360 0x7f6ef0070730 unknown :-1 s=CLOSED pgs=868 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:33:58.839 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:58.840+0000 7f6ef6e00700 1 -- 172.21.15.18:0/532574836 >> 172.21.15.18:0/532574836 conn(0x7f6ef00f89b0 msgr2=0x7f6ef00fadc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:33:58.839 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:58.840+0000 7f6ef6e00700 1 -- 172.21.15.18:0/532574836 shutdown_connections 2024-04-12T21:33:58.839 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:58.840+0000 7f6ef6e00700 1 -- 172.21.15.18:0/532574836 wait complete. 2024-04-12T21:33:58.841 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:58.841+0000 7f6ef6e00700 1 Processor -- start 2024-04-12T21:33:58.841 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:58.841+0000 7f6ef6e00700 1 -- start start 2024-04-12T21:33:58.841 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:58.841+0000 7f6ef6e00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6ef0070360 0x7f6ef0115d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:33:58.841 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:58.841+0000 7f6ef6e00700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6ef0070c70 0x7f6ef0116290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:33:58.841 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:58.841+0000 7f6ef6e00700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ef01168e0 con 0x7f6ef0070360 2024-04-12T21:33:58.841 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:58.841+0000 7f6ef6e00700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ef010fcb0 con 0x7f6ef0070c70 2024-04-12T21:33:58.841 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:58.841+0000 7f6eefe00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6ef0070360 0x7f6ef0115d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:33:58.841 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:58.842+0000 7f6eefe00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6ef0070360 0x7f6ef0115d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:34968/0 (socket says 172.21.15.18:34968) 2024-04-12T21:33:58.841 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:58.842+0000 7f6eefe00700 1 -- 172.21.15.18:0/3850742393 learned_addr learned my addr 172.21.15.18:0/3850742393 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:33:58.841 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:58.842+0000 7f6eefe00700 1 -- 172.21.15.18:0/3850742393 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6ef0070c70 msgr2=0x7f6ef0116290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:33:58.841 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:58.842+0000 7f6eefe00700 1 --2- 172.21.15.18:0/3850742393 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6ef0070c70 0x7f6ef0116290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:33:58.842 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:58.842+0000 7f6eefe00700 1 -- 172.21.15.18:0/3850742393 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ee0005040 con 0x7f6ef0070360 2024-04-12T21:33:58.842 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:58.842+0000 7f6eefe00700 1 --2- 172.21.15.18:0/3850742393 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6ef0070360 0x7f6ef0115d50 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7f6ee0004f50 tx=0x7f6ee0009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:33:58.843 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:58.842+0000 7f6ee7e00700 1 -- 172.21.15.18:0/3850742393 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6ee000e040 con 0x7f6ef0070360 2024-04-12T21:33:58.843 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:58.843+0000 7f6ee7e00700 1 -- 172.21.15.18:0/3850742393 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f6ee0004290 con 0x7f6ef0070360 2024-04-12T21:33:58.843 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:58.843+0000 7f6ef6e00700 1 -- 172.21.15.18:0/3850742393 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6ef010ff30 con 0x7f6ef0070360 2024-04-12T21:33:58.843 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:58.843+0000 7f6ee7e00700 1 -- 172.21.15.18:0/3850742393 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6ee001c3f0 con 0x7f6ef0070360 2024-04-12T21:33:58.843 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:58.843+0000 7f6ef6e00700 1 -- 172.21.15.18:0/3850742393 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6ef0110470 con 0x7f6ef0070360 2024-04-12T21:33:58.843 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:58.844+0000 7f6ef6e00700 1 -- 172.21.15.18:0/3850742393 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6ef0071aa0 con 0x7f6ef0070360 2024-04-12T21:33:58.844 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:58.845+0000 7f6ee7e00700 1 -- 172.21.15.18:0/3850742393 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7f6ee000bc10 con 0x7f6ef0070360 2024-04-12T21:33:58.845 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:58.846+0000 7f6ee7e00700 1 --2- 172.21.15.18:0/3850742393 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f6ed406ec10 0x7f6ed40710c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:33:58.845 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:58.846+0000 7f6ee7e00700 1 -- 172.21.15.18:0/3850742393 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 6757+0+0 (secure 0 0 0) 0x7f6ee008fc00 con 0x7f6ef0070360 2024-04-12T21:33:58.845 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:58.846+0000 7f6eef400700 1 --2- 172.21.15.18:0/3850742393 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f6ed406ec10 0x7f6ed40710c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:33:58.845 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:58.847+0000 7f6eef400700 1 --2- 172.21.15.18:0/3850742393 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f6ed406ec10 0x7f6ed40710c0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f6ef01115f0 tx=0x7f6edc009ce0 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:33:58.850 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:58.851+0000 7f6ee7e00700 1 -- 172.21.15.18:0/3850742393 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f6ee005c4e0 con 0x7f6ef0070360 2024-04-12T21:33:59.103 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:33:58 smithi018 ceph-mon[112590]: pgmap v1026: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:33:59.103 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:33:58 smithi018 ceph-mon[112590]: from='client.35792 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:33:59.173 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:59.174+0000 7f6ef6e00700 1 -- 172.21.15.18:0/3850742393 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f6ef004bed0 con 0x7f6ef0070360 2024-04-12T21:33:59.174 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:59.175+0000 7f6ee7e00700 1 -- 172.21.15.18:0/3850742393 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+837 (secure 0 0 0) 0x7f6ee005c070 con 0x7f6ef0070360 2024-04-12T21:33:59.175 INFO:teuthology.orchestra.run.smithi018.stdout:{ 2024-04-12T21:33:59.175 INFO:teuthology.orchestra.run.smithi018.stdout: "mon": { 2024-04-12T21:33:59.175 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 19.0.0-2879-g324baf8d (324baf8ddb91dea46cac50cb535c306115791355) squid (dev)": 2 2024-04-12T21:33:59.175 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-12T21:33:59.175 INFO:teuthology.orchestra.run.smithi018.stdout: "mgr": { 2024-04-12T21:33:59.175 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 19.0.0-2879-g324baf8d (324baf8ddb91dea46cac50cb535c306115791355) squid (dev)": 2 2024-04-12T21:33:59.175 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-12T21:33:59.175 INFO:teuthology.orchestra.run.smithi018.stdout: "osd": { 2024-04-12T21:33:59.175 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 19.0.0-2879-g324baf8d (324baf8ddb91dea46cac50cb535c306115791355) squid (dev)": 6 2024-04-12T21:33:59.175 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-12T21:33:59.175 INFO:teuthology.orchestra.run.smithi018.stdout: "mds": { 2024-04-12T21:33:59.176 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 17.2.7-827-gd3b5060a (d3b5060a17449edfae9385cdf792afec109fca5d) quincy (stable)": 1, 2024-04-12T21:33:59.176 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 19.0.0-2879-g324baf8d (324baf8ddb91dea46cac50cb535c306115791355) squid (dev)": 3 2024-04-12T21:33:59.176 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-12T21:33:59.176 INFO:teuthology.orchestra.run.smithi018.stdout: "overall": { 2024-04-12T21:33:59.176 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 17.2.7-827-gd3b5060a (d3b5060a17449edfae9385cdf792afec109fca5d) quincy (stable)": 1, 2024-04-12T21:33:59.176 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 19.0.0-2879-g324baf8d (324baf8ddb91dea46cac50cb535c306115791355) squid (dev)": 13 2024-04-12T21:33:59.176 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-04-12T21:33:59.176 INFO:teuthology.orchestra.run.smithi018.stdout:} 2024-04-12T21:33:59.178 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:59.179+0000 7f6ef6e00700 1 -- 172.21.15.18:0/3850742393 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f6ed406ec10 msgr2=0x7f6ed40710c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:33:59.178 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:59.179+0000 7f6ef6e00700 1 --2- 172.21.15.18:0/3850742393 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f6ed406ec10 0x7f6ed40710c0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f6ef01115f0 tx=0x7f6edc009ce0 comp rx=0 tx=0).stop 2024-04-12T21:33:59.178 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:59.179+0000 7f6ef6e00700 1 -- 172.21.15.18:0/3850742393 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6ef0070360 msgr2=0x7f6ef0115d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:33:59.179 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:59.179+0000 7f6ef6e00700 1 --2- 172.21.15.18:0/3850742393 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6ef0070360 0x7f6ef0115d50 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7f6ee0004f50 tx=0x7f6ee0009e30 comp rx=0 tx=0).stop 2024-04-12T21:33:59.179 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:59.179+0000 7f6ef6e00700 1 -- 172.21.15.18:0/3850742393 shutdown_connections 2024-04-12T21:33:59.179 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:59.179+0000 7f6ef6e00700 1 --2- 172.21.15.18:0/3850742393 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f6ed406ec10 0x7f6ed40710c0 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:33:59.179 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:59.179+0000 7f6ef6e00700 1 --2- 172.21.15.18:0/3850742393 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6ef0070360 0x7f6ef0115d50 unknown :-1 s=CLOSED pgs=869 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:33:59.179 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:59.179+0000 7f6ef6e00700 1 --2- 172.21.15.18:0/3850742393 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6ef0070c70 0x7f6ef0116290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:33:59.179 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:59.179+0000 7f6ef6e00700 1 -- 172.21.15.18:0/3850742393 >> 172.21.15.18:0/3850742393 conn(0x7f6ef00f89b0 msgr2=0x7f6ef00f9e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:33:59.179 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:59.180+0000 7f6ef6e00700 1 -- 172.21.15.18:0/3850742393 shutdown_connections 2024-04-12T21:33:59.179 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:59.180+0000 7f6ef6e00700 1 -- 172.21.15.18:0/3850742393 wait complete. 2024-04-12T21:33:59.282 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:33:58 smithi186 ceph-mon[86379]: pgmap v1026: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:33:59.282 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:33:58 smithi186 ceph-mon[86379]: from='client.35792 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:33:59.341 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:59.341+0000 7f4e92400700 1 -- 172.21.15.18:0/274061715 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4e8c105990 msgr2=0x7f4e8c0ff000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:33:59.341 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:59.341+0000 7f4e92400700 1 --2- 172.21.15.18:0/274061715 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4e8c105990 0x7f4e8c0ff000 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7f4e78004940 tx=0x7f4e78009e30 comp rx=0 tx=0).stop 2024-04-12T21:33:59.341 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:59.342+0000 7f4e92400700 1 -- 172.21.15.18:0/274061715 shutdown_connections 2024-04-12T21:33:59.341 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:59.342+0000 7f4e92400700 1 --2- 172.21.15.18:0/274061715 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4e8c105990 0x7f4e8c0ff000 unknown :-1 s=CLOSED pgs=870 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:33:59.341 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:59.342+0000 7f4e92400700 1 --2- 172.21.15.18:0/274061715 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4e8c104ff0 0x7f4e8c1053c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:33:59.341 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:59.342+0000 7f4e92400700 1 -- 172.21.15.18:0/274061715 >> 172.21.15.18:0/274061715 conn(0x7f4e8c0faba0 msgr2=0x7f4e8c0fcfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:33:59.341 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:59.342+0000 7f4e92400700 1 -- 172.21.15.18:0/274061715 shutdown_connections 2024-04-12T21:33:59.341 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:59.342+0000 7f4e92400700 1 -- 172.21.15.18:0/274061715 wait complete. 2024-04-12T21:33:59.342 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:59.343+0000 7f4e92400700 1 Processor -- start 2024-04-12T21:33:59.342 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:59.343+0000 7f4e92400700 1 -- start start 2024-04-12T21:33:59.342 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:59.343+0000 7f4e92400700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4e8c104ff0 0x7f4e8c115b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:33:59.342 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:59.343+0000 7f4e92400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4e8c105990 0x7f4e8c1160d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:33:59.342 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:59.343+0000 7f4e92400700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e8c1167b0 con 0x7f4e8c105990 2024-04-12T21:33:59.343 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:59.343+0000 7f4e92400700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e8c10fd00 con 0x7f4e8c104ff0 2024-04-12T21:33:59.343 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:59.344+0000 7f4e8aa00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4e8c105990 0x7f4e8c1160d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:33:59.343 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:59.344+0000 7f4e8aa00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4e8c105990 0x7f4e8c1160d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:34990/0 (socket says 172.21.15.18:34990) 2024-04-12T21:33:59.343 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:59.344+0000 7f4e8aa00700 1 -- 172.21.15.18:0/230145070 learned_addr learned my addr 172.21.15.18:0/230145070 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:33:59.344 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:59.344+0000 7f4e8b400700 1 --2- 172.21.15.18:0/230145070 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4e8c104ff0 0x7f4e8c115b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:33:59.344 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:59.344+0000 7f4e8aa00700 1 -- 172.21.15.18:0/230145070 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4e8c104ff0 msgr2=0x7f4e8c115b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:33:59.344 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:59.344+0000 7f4e8aa00700 1 --2- 172.21.15.18:0/230145070 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4e8c104ff0 0x7f4e8c115b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:33:59.344 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:59.344+0000 7f4e8aa00700 1 -- 172.21.15.18:0/230145070 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e78005040 con 0x7f4e8c105990 2024-04-12T21:33:59.344 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:59.344+0000 7f4e8b400700 1 --2- 172.21.15.18:0/230145070 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4e8c104ff0 0x7f4e8c115b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-12T21:33:59.344 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:59.344+0000 7f4e8aa00700 1 --2- 172.21.15.18:0/230145070 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4e8c105990 0x7f4e8c1160d0 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7f4e78009100 tx=0x7f4e78009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:33:59.344 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:59.345+0000 7f4e83400700 1 -- 172.21.15.18:0/230145070 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4e7800e040 con 0x7f4e8c105990 2024-04-12T21:33:59.344 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:59.345+0000 7f4e83400700 1 -- 172.21.15.18:0/230145070 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f4e78004290 con 0x7f4e8c105990 2024-04-12T21:33:59.344 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:59.345+0000 7f4e83400700 1 -- 172.21.15.18:0/230145070 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4e7801c3f0 con 0x7f4e8c105990 2024-04-12T21:33:59.344 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:59.345+0000 7f4e92400700 1 -- 172.21.15.18:0/230145070 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4e8c10ff20 con 0x7f4e8c105990 2024-04-12T21:33:59.344 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:59.345+0000 7f4e92400700 1 -- 172.21.15.18:0/230145070 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4e8c1104e0 con 0x7f4e8c105990 2024-04-12T21:33:59.345 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:59.346+0000 7f4e92400700 1 -- 172.21.15.18:0/230145070 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4e8c100740 con 0x7f4e8c105990 2024-04-12T21:33:59.346 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:59.347+0000 7f4e83400700 1 -- 172.21.15.18:0/230145070 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7f4e78013790 con 0x7f4e8c105990 2024-04-12T21:33:59.347 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:59.348+0000 7f4e83400700 1 --2- 172.21.15.18:0/230145070 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f4e7006ec00 0x7f4e700710b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:33:59.347 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:59.348+0000 7f4e83400700 1 -- 172.21.15.18:0/230145070 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 6757+0+0 (secure 0 0 0) 0x7f4e7808fc30 con 0x7f4e8c105990 2024-04-12T21:33:59.347 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:59.348+0000 7f4e8b400700 1 --2- 172.21.15.18:0/230145070 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f4e7006ec00 0x7f4e700710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:33:59.348 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:59.349+0000 7f4e8b400700 1 --2- 172.21.15.18:0/230145070 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f4e7006ec00 0x7f4e700710b0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f4e7c006fd0 tx=0x7f4e7c008040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:33:59.352 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:59.353+0000 7f4e83400700 1 -- 172.21.15.18:0/230145070 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f4e7805c510 con 0x7f4e8c105990 2024-04-12T21:33:59.620 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:59.621+0000 7f4e92400700 1 -- 172.21.15.18:0/230145070 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f4e8c10c1a0 con 0x7f4e8c105990 2024-04-12T21:33:59.621 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:59.623+0000 7f4e83400700 1 -- 172.21.15.18:0/230145070 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 31 v31) v1 ==== 76+0+1873 (secure 0 0 0) 0x7f4e78013420 con 0x7f4e8c105990 2024-04-12T21:33:59.621 INFO:teuthology.orchestra.run.smithi018.stdout:e31 2024-04-12T21:33:59.622 INFO:teuthology.orchestra.run.smithi018.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-12T21:33:59.622 INFO:teuthology.orchestra.run.smithi018.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-12T21:33:59.622 INFO:teuthology.orchestra.run.smithi018.stdout:legacy client fscid: 1 2024-04-12T21:33:59.622 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-12T21:33:59.622 INFO:teuthology.orchestra.run.smithi018.stdout:Filesystem 'cephfs' (1) 2024-04-12T21:33:59.622 INFO:teuthology.orchestra.run.smithi018.stdout:fs_name cephfs 2024-04-12T21:33:59.622 INFO:teuthology.orchestra.run.smithi018.stdout:epoch 24 2024-04-12T21:33:59.622 INFO:teuthology.orchestra.run.smithi018.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-04-12T21:33:59.622 INFO:teuthology.orchestra.run.smithi018.stdout:created 2024-04-12T20:51:57.362850+0000 2024-04-12T21:33:59.622 INFO:teuthology.orchestra.run.smithi018.stdout:modified 2024-04-12T21:06:59.717190+0000 2024-04-12T21:33:59.622 INFO:teuthology.orchestra.run.smithi018.stdout:tableserver 0 2024-04-12T21:33:59.622 INFO:teuthology.orchestra.run.smithi018.stdout:root 0 2024-04-12T21:33:59.622 INFO:teuthology.orchestra.run.smithi018.stdout:session_timeout 60 2024-04-12T21:33:59.623 INFO:teuthology.orchestra.run.smithi018.stdout:session_autoclose 300 2024-04-12T21:33:59.623 INFO:teuthology.orchestra.run.smithi018.stdout:max_file_size 1099511627776 2024-04-12T21:33:59.623 INFO:teuthology.orchestra.run.smithi018.stdout:max_xattr_size 65536 2024-04-12T21:33:59.623 INFO:teuthology.orchestra.run.smithi018.stdout:required_client_features {} 2024-04-12T21:33:59.623 INFO:teuthology.orchestra.run.smithi018.stdout:last_failure 0 2024-04-12T21:33:59.623 INFO:teuthology.orchestra.run.smithi018.stdout:last_failure_osd_epoch 100 2024-04-12T21:33:59.623 INFO:teuthology.orchestra.run.smithi018.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-12T21:33:59.623 INFO:teuthology.orchestra.run.smithi018.stdout:max_mds 2 2024-04-12T21:33:59.623 INFO:teuthology.orchestra.run.smithi018.stdout:in 0,1 2024-04-12T21:33:59.623 INFO:teuthology.orchestra.run.smithi018.stdout:up {} 2024-04-12T21:33:59.623 INFO:teuthology.orchestra.run.smithi018.stdout:failed 0,1 2024-04-12T21:33:59.623 INFO:teuthology.orchestra.run.smithi018.stdout:damaged 2024-04-12T21:33:59.623 INFO:teuthology.orchestra.run.smithi018.stdout:stopped 2024-04-12T21:33:59.623 INFO:teuthology.orchestra.run.smithi018.stdout:data_pools [3] 2024-04-12T21:33:59.623 INFO:teuthology.orchestra.run.smithi018.stdout:metadata_pool 2 2024-04-12T21:33:59.624 INFO:teuthology.orchestra.run.smithi018.stdout:inline_data disabled 2024-04-12T21:33:59.624 INFO:teuthology.orchestra.run.smithi018.stdout:balancer 2024-04-12T21:33:59.624 INFO:teuthology.orchestra.run.smithi018.stdout:bal_rank_mask -1 2024-04-12T21:33:59.624 INFO:teuthology.orchestra.run.smithi018.stdout:standby_count_wanted 1 2024-04-12T21:33:59.624 INFO:teuthology.orchestra.run.smithi018.stdout:qdb_cluster leader: 0 members: 2024-04-12T21:33:59.624 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-12T21:33:59.624 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-12T21:33:59.624 INFO:teuthology.orchestra.run.smithi018.stdout:Standby daemons: 2024-04-12T21:33:59.624 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-12T21:33:59.624 INFO:teuthology.orchestra.run.smithi018.stdout:[mds.cephfs.smithi186.wjnafs{-1:24277} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.186:6826/438347207,v1:172.21.15.186:6827/438347207] compat {c=[1],r=[1],i=[7ff]}] 2024-04-12T21:33:59.624 INFO:teuthology.orchestra.run.smithi018.stdout:[mds.cephfs.smithi018.acqbsc{-1:34436} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.18:6828/755179764,v1:172.21.15.18:6829/755179764] compat {c=[1],r=[1],i=[1fff]}] 2024-04-12T21:33:59.624 INFO:teuthology.orchestra.run.smithi018.stdout:[mds.cephfs.smithi018.szjluo{-1:34468} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.18:6826/2407992653,v1:172.21.15.18:6827/2407992653] compat {c=[1],r=[1],i=[1fff]}] 2024-04-12T21:33:59.624 INFO:teuthology.orchestra.run.smithi018.stdout:[mds.cephfs.smithi186.jdllyn{-1:44595} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.186:6824/3523444202,v1:172.21.15.186:6825/3523444202] compat {c=[1],r=[1],i=[1fff]}] 2024-04-12T21:33:59.626 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:59.626+0000 7f4e92400700 1 -- 172.21.15.18:0/230145070 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f4e7006ec00 msgr2=0x7f4e700710b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:33:59.626 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:59.626+0000 7f4e92400700 1 --2- 172.21.15.18:0/230145070 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f4e7006ec00 0x7f4e700710b0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f4e7c006fd0 tx=0x7f4e7c008040 comp rx=0 tx=0).stop 2024-04-12T21:33:59.626 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:59.626+0000 7f4e92400700 1 -- 172.21.15.18:0/230145070 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4e8c105990 msgr2=0x7f4e8c1160d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:33:59.626 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:59.626+0000 7f4e92400700 1 --2- 172.21.15.18:0/230145070 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4e8c105990 0x7f4e8c1160d0 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7f4e78009100 tx=0x7f4e78009e30 comp rx=0 tx=0).stop 2024-04-12T21:33:59.626 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:59.627+0000 7f4e92400700 1 -- 172.21.15.18:0/230145070 shutdown_connections 2024-04-12T21:33:59.626 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:59.627+0000 7f4e92400700 1 --2- 172.21.15.18:0/230145070 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f4e7006ec00 0x7f4e700710b0 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:33:59.626 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:59.627+0000 7f4e92400700 1 --2- 172.21.15.18:0/230145070 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4e8c104ff0 0x7f4e8c115b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:33:59.626 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:59.627+0000 7f4e92400700 1 --2- 172.21.15.18:0/230145070 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4e8c105990 0x7f4e8c1160d0 unknown :-1 s=CLOSED pgs=871 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:33:59.626 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:59.627+0000 7f4e92400700 1 -- 172.21.15.18:0/230145070 >> 172.21.15.18:0/230145070 conn(0x7f4e8c0faba0 msgr2=0x7f4e8c0fbad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:33:59.626 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:59.627+0000 7f4e92400700 1 -- 172.21.15.18:0/230145070 shutdown_connections 2024-04-12T21:33:59.626 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:59.627+0000 7f4e92400700 1 -- 172.21.15.18:0/230145070 wait complete. 2024-04-12T21:33:59.628 INFO:teuthology.orchestra.run.smithi018.stderr:dumped fsmap epoch 31 2024-04-12T21:33:59.784 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:59.784+0000 7f84c0e00700 1 -- 172.21.15.18:0/3554685555 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f84bc109430 msgr2=0x7f84bc109800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:33:59.784 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:59.784+0000 7f84c0e00700 1 --2- 172.21.15.18:0/3554685555 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f84bc109430 0x7f84bc109800 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7f84ac004940 tx=0x7f84ac009e30 comp rx=0 tx=0).stop 2024-04-12T21:33:59.784 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:59.785+0000 7f84c0e00700 1 -- 172.21.15.18:0/3554685555 shutdown_connections 2024-04-12T21:33:59.784 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:59.785+0000 7f84c0e00700 1 --2- 172.21.15.18:0/3554685555 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f84bc109dd0 0x7f84bc103440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:33:59.784 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:59.785+0000 7f84c0e00700 1 --2- 172.21.15.18:0/3554685555 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f84bc109430 0x7f84bc109800 unknown :-1 s=CLOSED pgs=872 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:33:59.784 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:59.785+0000 7f84c0e00700 1 -- 172.21.15.18:0/3554685555 >> 172.21.15.18:0/3554685555 conn(0x7f84bc072c00 msgr2=0x7f84bc075010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:33:59.784 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:59.785+0000 7f84c0e00700 1 -- 172.21.15.18:0/3554685555 shutdown_connections 2024-04-12T21:33:59.784 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:59.785+0000 7f84c0e00700 1 -- 172.21.15.18:0/3554685555 wait complete. 2024-04-12T21:33:59.786 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:59.786+0000 7f84c0e00700 1 Processor -- start 2024-04-12T21:33:59.786 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:59.786+0000 7f84c0e00700 1 -- start start 2024-04-12T21:33:59.786 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:59.786+0000 7f84c0e00700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f84bc109430 0x7f84bc119f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:33:59.786 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:59.786+0000 7f84c0e00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f84bc109dd0 0x7f84bc11a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:33:59.786 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:59.786+0000 7f84c0e00700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84bc11aad0 con 0x7f84bc109dd0 2024-04-12T21:33:59.786 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:59.786+0000 7f84c0e00700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84bc114100 con 0x7f84bc109430 2024-04-12T21:33:59.786 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:59.786+0000 7f84b9400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f84bc109dd0 0x7f84bc11a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:33:59.786 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:59.787+0000 7f84b9400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f84bc109dd0 0x7f84bc11a480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:35016/0 (socket says 172.21.15.18:35016) 2024-04-12T21:33:59.787 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:59.787+0000 7f84b9400700 1 -- 172.21.15.18:0/2239745842 learned_addr learned my addr 172.21.15.18:0/2239745842 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:33:59.787 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:59.787+0000 7f84b9e00700 1 --2- 172.21.15.18:0/2239745842 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f84bc109430 0x7f84bc119f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:33:59.787 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:59.787+0000 7f84b9400700 1 -- 172.21.15.18:0/2239745842 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f84bc109430 msgr2=0x7f84bc119f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:33:59.787 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:59.787+0000 7f84b9400700 1 --2- 172.21.15.18:0/2239745842 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f84bc109430 0x7f84bc119f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:33:59.787 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:59.787+0000 7f84b9400700 1 -- 172.21.15.18:0/2239745842 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84ac005040 con 0x7f84bc109dd0 2024-04-12T21:33:59.787 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:59.787+0000 7f84b9400700 1 --2- 172.21.15.18:0/2239745842 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f84bc109dd0 0x7f84bc11a480 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7f84a800c800 tx=0x7f84a8004470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:33:59.788 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:59.788+0000 7f84b2000700 1 -- 172.21.15.18:0/2239745842 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f84a800a760 con 0x7f84bc109dd0 2024-04-12T21:33:59.788 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:59.788+0000 7f84b2000700 1 -- 172.21.15.18:0/2239745842 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f84a800ad90 con 0x7f84bc109dd0 2024-04-12T21:33:59.788 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:59.788+0000 7f84c0e00700 1 -- 172.21.15.18:0/2239745842 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f84bc1143e0 con 0x7f84bc109dd0 2024-04-12T21:33:59.788 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:59.788+0000 7f84b2000700 1 -- 172.21.15.18:0/2239745842 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f84a80113a0 con 0x7f84bc109dd0 2024-04-12T21:33:59.788 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:59.788+0000 7f84c0e00700 1 -- 172.21.15.18:0/2239745842 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f84bc1149a0 con 0x7f84bc109dd0 2024-04-12T21:33:59.788 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:59.789+0000 7f84c0e00700 1 -- 172.21.15.18:0/2239745842 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f84bc104b80 con 0x7f84bc109dd0 2024-04-12T21:33:59.789 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:59.790+0000 7f84b2000700 1 -- 172.21.15.18:0/2239745842 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7f84a8012400 con 0x7f84bc109dd0 2024-04-12T21:33:59.790 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:59.791+0000 7f84b2000700 1 --2- 172.21.15.18:0/2239745842 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f84a006ec00 0x7f84a00710b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:33:59.790 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:59.791+0000 7f84b2000700 1 -- 172.21.15.18:0/2239745842 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 6757+0+0 (secure 0 0 0) 0x7f84a808e9a0 con 0x7f84bc109dd0 2024-04-12T21:33:59.790 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:59.791+0000 7f84b9e00700 1 --2- 172.21.15.18:0/2239745842 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f84a006ec00 0x7f84a00710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:33:59.790 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:59.792+0000 7f84b9e00700 1 --2- 172.21.15.18:0/2239745842 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f84a006ec00 0x7f84a00710b0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f84ac00b3b0 tx=0x7f84ac009e30 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:33:59.795 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:33:59.796+0000 7f84b2000700 1 -- 172.21.15.18:0/2239745842 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f84a805b1f0 con 0x7f84bc109dd0 2024-04-12T21:34:00.048 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:34:00.049+0000 7f84c0e00700 1 -- 172.21.15.18:0/2239745842 --> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f84bc1105c0 con 0x7f84a006ec00 2024-04-12T21:34:00.048 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:33:59 smithi018 ceph-mon[112590]: from='client.35796 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:34:00.049 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:33:59 smithi018 ceph-mon[112590]: from='client.35800 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:34:00.049 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:33:59 smithi018 ceph-mon[112590]: from='client.? 172.21.15.18:0/3850742393' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T21:34:00.049 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:33:59 smithi018 ceph-mon[112590]: from='client.? 172.21.15.18:0/230145070' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-12T21:34:00.050 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:34:00.052+0000 7f84b2000700 1 -- 172.21.15.18:0/2239745842 <== mgr.34102 v2:172.21.15.18:6800/3812891919 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f84ac009710 con 0x7f84a006ec00 2024-04-12T21:34:00.051 INFO:teuthology.orchestra.run.smithi018.stdout:{ 2024-04-12T21:34:00.051 INFO:teuthology.orchestra.run.smithi018.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:f4452def3be4d350f4a8e5feaf7ebb1511be9298fe68478ba7a2d6fbd7d69a63", 2024-04-12T21:34:00.051 INFO:teuthology.orchestra.run.smithi018.stdout: "in_progress": true, 2024-04-12T21:34:00.051 INFO:teuthology.orchestra.run.smithi018.stdout: "which": "Upgrading all daemon types on all hosts", 2024-04-12T21:34:00.051 INFO:teuthology.orchestra.run.smithi018.stdout: "services_complete": [ 2024-04-12T21:34:00.051 INFO:teuthology.orchestra.run.smithi018.stdout: "crash", 2024-04-12T21:34:00.051 INFO:teuthology.orchestra.run.smithi018.stdout: "osd", 2024-04-12T21:34:00.051 INFO:teuthology.orchestra.run.smithi018.stdout: "mon", 2024-04-12T21:34:00.051 INFO:teuthology.orchestra.run.smithi018.stdout: "mgr" 2024-04-12T21:34:00.051 INFO:teuthology.orchestra.run.smithi018.stdout: ], 2024-04-12T21:34:00.051 INFO:teuthology.orchestra.run.smithi018.stdout: "progress": "15/21 daemons upgraded", 2024-04-12T21:34:00.051 INFO:teuthology.orchestra.run.smithi018.stdout: "message": "Currently upgrading mds daemons", 2024-04-12T21:34:00.051 INFO:teuthology.orchestra.run.smithi018.stdout: "is_paused": false 2024-04-12T21:34:00.052 INFO:teuthology.orchestra.run.smithi018.stdout:} 2024-04-12T21:34:00.055 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:34:00.056+0000 7f84c0e00700 1 -- 172.21.15.18:0/2239745842 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f84a006ec00 msgr2=0x7f84a00710b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:34:00.055 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:34:00.056+0000 7f84c0e00700 1 --2- 172.21.15.18:0/2239745842 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f84a006ec00 0x7f84a00710b0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f84ac00b3b0 tx=0x7f84ac009e30 comp rx=0 tx=0).stop 2024-04-12T21:34:00.055 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:34:00.056+0000 7f84c0e00700 1 -- 172.21.15.18:0/2239745842 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f84bc109dd0 msgr2=0x7f84bc11a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:34:00.055 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:34:00.056+0000 7f84c0e00700 1 --2- 172.21.15.18:0/2239745842 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f84bc109dd0 0x7f84bc11a480 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7f84a800c800 tx=0x7f84a8004470 comp rx=0 tx=0).stop 2024-04-12T21:34:00.055 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:34:00.056+0000 7f84c0e00700 1 -- 172.21.15.18:0/2239745842 shutdown_connections 2024-04-12T21:34:00.055 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:34:00.056+0000 7f84c0e00700 1 --2- 172.21.15.18:0/2239745842 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f84a006ec00 0x7f84a00710b0 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:34:00.055 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:34:00.056+0000 7f84c0e00700 1 --2- 172.21.15.18:0/2239745842 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f84bc109430 0x7f84bc119f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:34:00.056 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:34:00.056+0000 7f84c0e00700 1 --2- 172.21.15.18:0/2239745842 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f84bc109dd0 0x7f84bc11a480 unknown :-1 s=CLOSED pgs=873 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:34:00.056 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:34:00.056+0000 7f84c0e00700 1 -- 172.21.15.18:0/2239745842 >> 172.21.15.18:0/2239745842 conn(0x7f84bc072c00 msgr2=0x7f84bc074c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:34:00.056 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:34:00.056+0000 7f84c0e00700 1 -- 172.21.15.18:0/2239745842 shutdown_connections 2024-04-12T21:34:00.056 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:34:00.057+0000 7f84c0e00700 1 -- 172.21.15.18:0/2239745842 wait complete. 2024-04-12T21:34:00.201 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:34:00.202+0000 7fcf6ae00700 1 -- 172.21.15.18:0/2848827402 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fcf64105800 msgr2=0x7fcf640fee70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:34:00.201 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:34:00.202+0000 7fcf6ae00700 1 --2- 172.21.15.18:0/2848827402 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fcf64105800 0x7fcf640fee70 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7fcf54004940 tx=0x7fcf54009e30 comp rx=0 tx=0).stop 2024-04-12T21:34:00.201 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:34:00.202+0000 7fcf6ae00700 1 -- 172.21.15.18:0/2848827402 shutdown_connections 2024-04-12T21:34:00.201 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:34:00.202+0000 7fcf6ae00700 1 --2- 172.21.15.18:0/2848827402 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fcf64105800 0x7fcf640fee70 unknown :-1 s=CLOSED pgs=874 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:34:00.201 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:34:00.202+0000 7fcf6ae00700 1 --2- 172.21.15.18:0/2848827402 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcf64104e60 0x7fcf64105230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:34:00.201 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:34:00.202+0000 7fcf6ae00700 1 -- 172.21.15.18:0/2848827402 >> 172.21.15.18:0/2848827402 conn(0x7fcf640fab10 msgr2=0x7fcf640fcef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:34:00.201 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:34:00.202+0000 7fcf6ae00700 1 -- 172.21.15.18:0/2848827402 shutdown_connections 2024-04-12T21:34:00.201 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:34:00.202+0000 7fcf6ae00700 1 -- 172.21.15.18:0/2848827402 wait complete. 2024-04-12T21:34:00.203 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:34:00.203+0000 7fcf6ae00700 1 Processor -- start 2024-04-12T21:34:00.203 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:34:00.203+0000 7fcf6ae00700 1 -- start start 2024-04-12T21:34:00.203 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:34:00.203+0000 7fcf6ae00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fcf64104e60 0x7fcf64115bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:34:00.203 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:34:00.203+0000 7fcf6ae00700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcf64105800 0x7fcf641160f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:34:00.203 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:34:00.203+0000 7fcf6ae00700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf64116740 con 0x7fcf64104e60 2024-04-12T21:34:00.203 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:34:00.203+0000 7fcf6ae00700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf6410fb30 con 0x7fcf64105800 2024-04-12T21:34:00.203 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:34:00.204+0000 7fcf69a00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fcf64104e60 0x7fcf64115bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:34:00.203 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:34:00.204+0000 7fcf69a00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fcf64104e60 0x7fcf64115bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:35022/0 (socket says 172.21.15.18:35022) 2024-04-12T21:34:00.203 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:34:00.204+0000 7fcf69a00700 1 -- 172.21.15.18:0/3124672042 learned_addr learned my addr 172.21.15.18:0/3124672042 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:34:00.203 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:34:00.204+0000 7fcf69a00700 1 -- 172.21.15.18:0/3124672042 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcf64105800 msgr2=0x7fcf641160f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:34:00.203 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:34:00.204+0000 7fcf69a00700 1 --2- 172.21.15.18:0/3124672042 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcf64105800 0x7fcf641160f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:34:00.203 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:34:00.204+0000 7fcf69a00700 1 -- 172.21.15.18:0/3124672042 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf54005040 con 0x7fcf64104e60 2024-04-12T21:34:00.203 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:34:00.204+0000 7fcf69a00700 1 --2- 172.21.15.18:0/3124672042 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fcf64104e60 0x7fcf64115bb0 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7fcf58002a50 tx=0x7fcf580046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:34:00.204 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:34:00.204+0000 7fcf61600700 1 -- 172.21.15.18:0/3124672042 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcf58007cc0 con 0x7fcf64104e60 2024-04-12T21:34:00.204 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:34:00.205+0000 7fcf61600700 1 -- 172.21.15.18:0/3124672042 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fcf5800f040 con 0x7fcf64104e60 2024-04-12T21:34:00.204 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:34:00.205+0000 7fcf61600700 1 -- 172.21.15.18:0/3124672042 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcf58009640 con 0x7fcf64104e60 2024-04-12T21:34:00.204 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:34:00.205+0000 7fcf6ae00700 1 -- 172.21.15.18:0/3124672042 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcf6410fe10 con 0x7fcf64104e60 2024-04-12T21:34:00.204 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:34:00.205+0000 7fcf6ae00700 1 -- 172.21.15.18:0/3124672042 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcf641103d0 con 0x7fcf64104e60 2024-04-12T21:34:00.205 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:34:00.206+0000 7fcf6ae00700 1 -- 172.21.15.18:0/3124672042 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcf64107330 con 0x7fcf64104e60 2024-04-12T21:34:00.205 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:34:00.207+0000 7fcf61600700 1 -- 172.21.15.18:0/3124672042 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7fcf580097a0 con 0x7fcf64104e60 2024-04-12T21:34:00.206 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:34:00.208+0000 7fcf61600700 1 --2- 172.21.15.18:0/3124672042 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fcf4c06ec00 0x7fcf4c0710b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:34:00.207 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:34:00.208+0000 7fcf61600700 1 -- 172.21.15.18:0/3124672042 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 6757+0+0 (secure 0 0 0) 0x7fcf5808e760 con 0x7fcf64104e60 2024-04-12T21:34:00.207 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:34:00.208+0000 7fcf69000700 1 --2- 172.21.15.18:0/3124672042 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fcf4c06ec00 0x7fcf4c0710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:34:00.207 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:34:00.209+0000 7fcf69000700 1 --2- 172.21.15.18:0/3124672042 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fcf4c06ec00 0x7fcf4c0710b0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7fcf54004f90 tx=0x7fcf54009e30 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:34:00.211 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:34:00.212+0000 7fcf61600700 1 -- 172.21.15.18:0/3124672042 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7fcf5805af10 con 0x7fcf64104e60 2024-04-12T21:34:00.282 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:33:59 smithi186 ceph-mon[86379]: from='client.35796 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:34:00.282 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:33:59 smithi186 ceph-mon[86379]: from='client.35800 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:34:00.282 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:33:59 smithi186 ceph-mon[86379]: from='client.? 172.21.15.18:0/3850742393' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T21:34:00.282 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:33:59 smithi186 ceph-mon[86379]: from='client.? 172.21.15.18:0/230145070' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-12T21:34:00.522 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:34:00.523+0000 7fcf6ae00700 1 -- 172.21.15.18:0/3124672042 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7fcf64111200 con 0x7fcf64104e60 2024-04-12T21:34:00.523 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:34:00.524+0000 7fcf61600700 1 -- 172.21.15.18:0/3124672042 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+4303 (secure 0 0 0) 0x7fcf5805aaa0 con 0x7fcf64104e60 2024-04-12T21:34:00.523 INFO:teuthology.orchestra.run.smithi018.stdout:HEALTH_ERR failed to probe daemons or devices; 1 filesystem is degraded; 1 filesystem is offline 2024-04-12T21:34:00.523 INFO:teuthology.orchestra.run.smithi018.stdout:[WRN] CEPHADM_REFRESH_FAILED: failed to probe daemons or devices 2024-04-12T21:34:00.523 INFO:teuthology.orchestra.run.smithi018.stdout: host smithi018 `cephadm ceph-volume` failed: cephadm exited with an error code: 1, stderr: Inferring config /var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/mon.smithi018/config 2024-04-12T21:34:00.523 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/podman: timeout after 895 seconds 2024-04-12T21:34:00.524 INFO:teuthology.orchestra.run.smithi018.stdout:Non-zero exit code 124 from /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:f4452def3be4d350f4a8e5feaf7ebb1511be9298fe68478ba7a2d6fbd7d69a63 -e NODE_NAME=smithi018 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968:/var/run/ceph:z -v /var/log/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968:/var/log/ceph:z -v /var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmpxsmwkkuh:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:f4452def3be4d350f4a8e5feaf7ebb1511be9298fe68478ba7a2d6fbd7d69a63 inventory --format=json-pretty --filter-for-batch 2024-04-12T21:34:00.524 INFO:teuthology.orchestra.run.smithi018.stdout:Traceback (most recent call last): 2024-04-12T21:34:00.524 INFO:teuthology.orchestra.run.smithi018.stdout: File "/usr/lib64/python3.9/runpy.py", line 197, in _run_module_as_main 2024-04-12T21:34:00.524 INFO:teuthology.orchestra.run.smithi018.stdout: return _run_code(code, main_globals, None, 2024-04-12T21:34:00.524 INFO:teuthology.orchestra.run.smithi018.stdout: File "/usr/lib64/python3.9/runpy.py", line 87, in _run_code 2024-04-12T21:34:00.524 INFO:teuthology.orchestra.run.smithi018.stdout: exec(code, run_globals) 2024-04-12T21:34:00.524 INFO:teuthology.orchestra.run.smithi018.stdout: File "/var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/cephadm.e44f3ed5e34e4ef52eedef233d1190bff004470d8e8005127b5312bff357fa7f/__main__.py", line 5618, in 2024-04-12T21:34:00.524 INFO:teuthology.orchestra.run.smithi018.stdout: File "/var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/cephadm.e44f3ed5e34e4ef52eedef233d1190bff004470d8e8005127b5312bff357fa7f/__main__.py", line 5606, in main 2024-04-12T21:34:00.524 INFO:teuthology.orchestra.run.smithi018.stdout: File "/var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/cephadm.e44f3ed5e34e4ef52eedef233d1190bff004470d8e8005127b5312bff357fa7f/__main__.py", line 408, in _infer_config 2024-04-12T21:34:00.524 INFO:teuthology.orchestra.run.smithi018.stdout: File "/var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/cephadm.e44f3ed5e34e4ef52eedef233d1190bff004470d8e8005127b5312bff357fa7f/__main__.py", line 323, in _infer_fsid 2024-04-12T21:34:00.524 INFO:teuthology.orchestra.run.smithi018.stdout: File "/var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/cephadm.e44f3ed5e34e4ef52eedef233d1190bff004470d8e8005127b5312bff357fa7f/__main__.py", line 436, in _infer_image 2024-04-12T21:34:00.524 INFO:teuthology.orchestra.run.smithi018.stdout: File "/var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/cephadm.e44f3ed5e34e4ef52eedef233d1190bff004470d8e8005127b5312bff357fa7f/__main__.py", line 310, in _validate_fsid 2024-04-12T21:34:00.524 INFO:teuthology.orchestra.run.smithi018.stdout: File "/var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/cephadm.e44f3ed5e34e4ef52eedef233d1190bff004470d8e8005127b5312bff357fa7f/__main__.py", line 3359, in command_ceph_volume 2024-04-12T21:34:00.524 INFO:teuthology.orchestra.run.smithi018.stdout: File "/var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/cephadm.e44f3ed5e34e4ef52eedef233d1190bff004470d8e8005127b5312bff357fa7f/cephadmlib/call_wrappers.py", line 310, in call_throws 2024-04-12T21:34:00.525 INFO:teuthology.orchestra.run.smithi018.stdout:RuntimeError: Failed command: /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:f4452def3be4d350f4a8e5feaf7ebb1511be9298fe68478ba7a2d6fbd7d69a63 -e NODE_NAME=smithi018 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968:/var/run/ceph:z -v /var/log/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968:/var/log/ceph:z -v /var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmpxsmwkkuh:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:f4452def3be4d350f4a8e5feaf7ebb1511be9298fe68478ba7a2d6fbd7d69a63 inventory --format=json-pretty --filter-for-batch 2024-04-12T21:34:00.525 INFO:teuthology.orchestra.run.smithi018.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-04-12T21:34:00.525 INFO:teuthology.orchestra.run.smithi018.stdout: fs cephfs is degraded 2024-04-12T21:34:00.525 INFO:teuthology.orchestra.run.smithi018.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-04-12T21:34:00.525 INFO:teuthology.orchestra.run.smithi018.stdout: fs cephfs is offline because no MDS is active for it. 2024-04-12T21:34:00.526 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:34:00.527+0000 7fcf6ae00700 1 -- 172.21.15.18:0/3124672042 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fcf4c06ec00 msgr2=0x7fcf4c0710b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:34:00.526 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:34:00.527+0000 7fcf6ae00700 1 --2- 172.21.15.18:0/3124672042 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fcf4c06ec00 0x7fcf4c0710b0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7fcf54004f90 tx=0x7fcf54009e30 comp rx=0 tx=0).stop 2024-04-12T21:34:00.526 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:34:00.527+0000 7fcf6ae00700 1 -- 172.21.15.18:0/3124672042 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fcf64104e60 msgr2=0x7fcf64115bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:34:00.526 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:34:00.527+0000 7fcf6ae00700 1 --2- 172.21.15.18:0/3124672042 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fcf64104e60 0x7fcf64115bb0 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7fcf58002a50 tx=0x7fcf580046b0 comp rx=0 tx=0).stop 2024-04-12T21:34:00.527 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:34:00.527+0000 7fcf6ae00700 1 -- 172.21.15.18:0/3124672042 shutdown_connections 2024-04-12T21:34:00.527 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:34:00.527+0000 7fcf6ae00700 1 --2- 172.21.15.18:0/3124672042 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fcf4c06ec00 0x7fcf4c0710b0 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:34:00.527 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:34:00.527+0000 7fcf6ae00700 1 --2- 172.21.15.18:0/3124672042 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fcf64104e60 0x7fcf64115bb0 unknown :-1 s=CLOSED pgs=875 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:34:00.527 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:34:00.527+0000 7fcf6ae00700 1 --2- 172.21.15.18:0/3124672042 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcf64105800 0x7fcf641160f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:34:00.527 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:34:00.527+0000 7fcf6ae00700 1 -- 172.21.15.18:0/3124672042 >> 172.21.15.18:0/3124672042 conn(0x7fcf640fab10 msgr2=0x7fcf640fd940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:34:00.527 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:34:00.528+0000 7fcf6ae00700 1 -- 172.21.15.18:0/3124672042 shutdown_connections 2024-04-12T21:34:00.527 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:34:00.528+0000 7fcf6ae00700 1 -- 172.21.15.18:0/3124672042 wait complete. 2024-04-12T21:34:01.282 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:34:00 smithi186 ceph-mon[86379]: pgmap v1027: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:34:01.282 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:34:00 smithi186 ceph-mon[86379]: from='client.? 172.21.15.18:0/3124672042' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-12T21:34:01.353 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:34:00 smithi018 ceph-mon[112590]: pgmap v1027: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:34:01.353 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:34:00 smithi018 ceph-mon[112590]: from='client.? 172.21.15.18:0/3124672042' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-12T21:34:02.282 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:34:01 smithi186 ceph-mon[86379]: from='client.35812 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:34:02.353 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:34:01 smithi018 ceph-mon[112590]: from='client.35812 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:34:03.282 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:34:02 smithi186 ceph-mon[86379]: pgmap v1028: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:34:03.353 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:34:02 smithi018 ceph-mon[112590]: pgmap v1028: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:34:05.282 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:34:04 smithi186 ceph-mon[86379]: pgmap v1029: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:34:05.353 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:34:04 smithi018 ceph-mon[112590]: pgmap v1029: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:34:07.282 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:34:06 smithi186 ceph-mon[86379]: pgmap v1030: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:34:07.353 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:34:06 smithi018 ceph-mon[112590]: pgmap v1030: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:34:09.282 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:34:08 smithi186 ceph-mon[86379]: pgmap v1031: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:34:09.353 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:34:08 smithi018 ceph-mon[112590]: pgmap v1031: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:34:11.282 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:34:10 smithi186 ceph-mon[86379]: pgmap v1032: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:34:11.282 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:34:10 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T21:34:11.353 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:34:10 smithi018 ceph-mon[112590]: pgmap v1032: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:34:11.353 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:34:10 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T21:34:12.282 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:34:11 smithi186 ceph-mon[86379]: pgmap v1033: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:34:12.353 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:34:11 smithi018 ceph-mon[112590]: pgmap v1033: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:34:15.032 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:34:14 smithi186 ceph-mon[86379]: pgmap v1034: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:34:15.056 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:34:14 smithi018 ceph-mon[112590]: pgmap v1034: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:34:17.103 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:34:16 smithi018 ceph-mon[112590]: pgmap v1035: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:34:17.282 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:34:16 smithi186 ceph-mon[86379]: pgmap v1035: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:34:19.103 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:34:18 smithi018 ceph-mon[112590]: pgmap v1036: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:34:19.282 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:34:18 smithi186 ceph-mon[86379]: pgmap v1036: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:34:21.103 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:34:20 smithi018 ceph-mon[112590]: pgmap v1037: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:34:21.282 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:34:20 smithi186 ceph-mon[86379]: pgmap v1037: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:34:23.106 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:34:22 smithi018 ceph-mon[112590]: pgmap v1038: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:34:23.282 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:34:22 smithi186 ceph-mon[86379]: pgmap v1038: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:34:25.103 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:34:24 smithi018 ceph-mon[112590]: pgmap v1039: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:34:25.282 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:34:24 smithi186 ceph-mon[86379]: pgmap v1039: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:34:26.103 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:34:25 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T21:34:26.282 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:34:25 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T21:34:27.103 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:34:26 smithi018 ceph-mon[112590]: pgmap v1040: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:34:27.282 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:34:26 smithi186 ceph-mon[86379]: pgmap v1040: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:34:29.103 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:34:28 smithi018 ceph-mon[112590]: pgmap v1041: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:34:29.282 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:34:28 smithi186 ceph-mon[86379]: pgmap v1041: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:34:30.701 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:34:30.701+0000 7f0ea6c00700 1 -- 172.21.15.18:0/3730778908 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0ea0109dd0 msgr2=0x7f0ea0103440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:34:30.701 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:34:30.701+0000 7f0ea6c00700 1 --2- 172.21.15.18:0/3730778908 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0ea0109dd0 0x7f0ea0103440 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7f0e8c004940 tx=0x7f0e8c009e30 comp rx=0 tx=0).stop 2024-04-12T21:34:30.701 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:34:30.701+0000 7f0ea6c00700 1 -- 172.21.15.18:0/3730778908 shutdown_connections 2024-04-12T21:34:30.701 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:34:30.701+0000 7f0ea6c00700 1 --2- 172.21.15.18:0/3730778908 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0ea0109dd0 0x7f0ea0103440 unknown :-1 s=CLOSED pgs=876 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:34:30.702 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:34:30.701+0000 7f0ea6c00700 1 --2- 172.21.15.18:0/3730778908 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0ea0109430 0x7f0ea0109800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:34:30.702 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:34:30.701+0000 7f0ea6c00700 1 -- 172.21.15.18:0/3730778908 >> 172.21.15.18:0/3730778908 conn(0x7f0ea0072c00 msgr2=0x7f0ea0075010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:34:30.702 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:34:30.701+0000 7f0ea6c00700 1 -- 172.21.15.18:0/3730778908 shutdown_connections 2024-04-12T21:34:30.702 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:34:30.701+0000 7f0ea6c00700 1 -- 172.21.15.18:0/3730778908 wait complete. 2024-04-12T21:34:30.703 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:34:30.702+0000 7f0ea6c00700 1 Processor -- start 2024-04-12T21:34:30.703 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:34:30.702+0000 7f0ea6c00700 1 -- start start 2024-04-12T21:34:30.704 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:34:30.702+0000 7f0ea6c00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0ea0109430 0x7f0ea0119fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:34:30.704 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:34:30.702+0000 7f0ea6c00700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0ea0109dd0 0x7f0ea011a4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:34:30.704 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:34:30.702+0000 7f0ea6c00700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ea011abc0 con 0x7f0ea0109430 2024-04-12T21:34:30.704 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:34:30.702+0000 7f0ea6c00700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ea0114110 con 0x7f0ea0109dd0 2024-04-12T21:34:30.704 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:34:30.703+0000 7f0e9fe00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0ea0109430 0x7f0ea0119fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:34:30.704 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:34:30.703+0000 7f0e9fe00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0ea0109430 0x7f0ea0119fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:36846/0 (socket says 172.21.15.18:36846) 2024-04-12T21:34:30.704 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:34:30.703+0000 7f0e9fe00700 1 -- 172.21.15.18:0/1708007437 learned_addr learned my addr 172.21.15.18:0/1708007437 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:34:30.704 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:34:30.703+0000 7f0e9f400700 1 --2- 172.21.15.18:0/1708007437 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0ea0109dd0 0x7f0ea011a4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:34:30.704 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:34:30.703+0000 7f0e9fe00700 1 -- 172.21.15.18:0/1708007437 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0ea0109dd0 msgr2=0x7f0ea011a4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:34:30.704 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:34:30.703+0000 7f0e9fe00700 1 --2- 172.21.15.18:0/1708007437 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0ea0109dd0 0x7f0ea011a4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:34:30.704 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:34:30.703+0000 7f0e9fe00700 1 -- 172.21.15.18:0/1708007437 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e8c005040 con 0x7f0ea0109430 2024-04-12T21:34:30.704 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:34:30.704+0000 7f0e9fe00700 1 --2- 172.21.15.18:0/1708007437 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0ea0109430 0x7f0ea0119fa0 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7f0e9000b790 tx=0x7f0e90009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:34:30.705 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:34:30.704+0000 7f0e97e00700 1 -- 172.21.15.18:0/1708007437 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0e9000f770 con 0x7f0ea0109430 2024-04-12T21:34:30.705 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:34:30.704+0000 7f0e97e00700 1 -- 172.21.15.18:0/1708007437 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f0e9000fda0 con 0x7f0ea0109430 2024-04-12T21:34:30.705 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:34:30.704+0000 7f0ea6c00700 1 -- 172.21.15.18:0/1708007437 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0ea0114390 con 0x7f0ea0109430 2024-04-12T21:34:30.706 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:34:30.704+0000 7f0e97e00700 1 -- 172.21.15.18:0/1708007437 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0e9000d310 con 0x7f0ea0109430 2024-04-12T21:34:30.706 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:34:30.704+0000 7f0ea6c00700 1 -- 172.21.15.18:0/1708007437 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0ea0114950 con 0x7f0ea0109430 2024-04-12T21:34:30.706 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:34:30.705+0000 7f0ea6c00700 1 -- 172.21.15.18:0/1708007437 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0ea0104b80 con 0x7f0ea0109430 2024-04-12T21:34:30.706 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:34:30.706+0000 7f0e97e00700 1 -- 172.21.15.18:0/1708007437 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7f0e90010420 con 0x7f0ea0109430 2024-04-12T21:34:30.707 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:34:30.707+0000 7f0e97e00700 1 --2- 172.21.15.18:0/1708007437 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f0e8406ec00 0x7f0e840710b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:34:30.707 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:34:30.708+0000 7f0e97e00700 1 -- 172.21.15.18:0/1708007437 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 6757+0+0 (secure 0 0 0) 0x7f0e9008e9e0 con 0x7f0ea0109430 2024-04-12T21:34:30.707 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:34:30.708+0000 7f0e9f400700 1 --2- 172.21.15.18:0/1708007437 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f0e8406ec00 0x7f0e840710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:34:30.708 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:34:30.709+0000 7f0e9f400700 1 --2- 172.21.15.18:0/1708007437 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f0e8406ec00 0x7f0e840710b0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f0e8c00b020 tx=0x7f0e8c009e30 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:34:30.712 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:34:30.713+0000 7f0e97e00700 1 -- 172.21.15.18:0/1708007437 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f0e9005b230 con 0x7f0ea0109430 2024-04-12T21:34:30.973 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:34:30.973+0000 7f0ea6c00700 1 -- 172.21.15.18:0/1708007437 --> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f0ea0114bd0 con 0x7f0e8406ec00 2024-04-12T21:34:30.973 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:34:30 smithi018 ceph-mon[112590]: pgmap v1042: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:34:30.975 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:34:30.975+0000 7f0e97e00700 1 -- 172.21.15.18:0/1708007437 <== mgr.34102 v2:172.21.15.18:6800/3812891919 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f0ea0114bd0 con 0x7f0e8406ec00 2024-04-12T21:34:30.978 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:34:30.978+0000 7f0ea6c00700 1 -- 172.21.15.18:0/1708007437 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f0e8406ec00 msgr2=0x7f0e840710b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:34:30.979 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:34:30.978+0000 7f0ea6c00700 1 --2- 172.21.15.18:0/1708007437 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f0e8406ec00 0x7f0e840710b0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f0e8c00b020 tx=0x7f0e8c009e30 comp rx=0 tx=0).stop 2024-04-12T21:34:30.979 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:34:30.978+0000 7f0ea6c00700 1 -- 172.21.15.18:0/1708007437 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0ea0109430 msgr2=0x7f0ea0119fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:34:30.979 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:34:30.978+0000 7f0ea6c00700 1 --2- 172.21.15.18:0/1708007437 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0ea0109430 0x7f0ea0119fa0 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7f0e9000b790 tx=0x7f0e90009250 comp rx=0 tx=0).stop 2024-04-12T21:34:30.979 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:34:30.979+0000 7f0ea6c00700 1 -- 172.21.15.18:0/1708007437 shutdown_connections 2024-04-12T21:34:30.979 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:34:30.979+0000 7f0ea6c00700 1 --2- 172.21.15.18:0/1708007437 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f0e8406ec00 0x7f0e840710b0 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:34:30.979 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:34:30.979+0000 7f0ea6c00700 1 --2- 172.21.15.18:0/1708007437 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0ea0109430 0x7f0ea0119fa0 unknown :-1 s=CLOSED pgs=877 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:34:30.979 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:34:30.979+0000 7f0ea6c00700 1 --2- 172.21.15.18:0/1708007437 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0ea0109dd0 0x7f0ea011a4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:34:30.979 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:34:30.979+0000 7f0ea6c00700 1 -- 172.21.15.18:0/1708007437 >> 172.21.15.18:0/1708007437 conn(0x7f0ea0072c00 msgr2=0x7f0ea0074b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:34:30.980 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:34:30.979+0000 7f0ea6c00700 1 -- 172.21.15.18:0/1708007437 shutdown_connections 2024-04-12T21:34:30.980 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:34:30.979+0000 7f0ea6c00700 1 -- 172.21.15.18:0/1708007437 wait complete. 2024-04-12T21:34:30.995 INFO:teuthology.orchestra.run.smithi018.stdout:true 2024-04-12T21:34:31.135 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:34:31.135+0000 7f67a0c00700 1 -- 172.21.15.18:0/4186956612 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f679c109c40 msgr2=0x7f679c1032b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:34:31.136 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:34:31.135+0000 7f67a0c00700 1 --2- 172.21.15.18:0/4186956612 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f679c109c40 0x7f679c1032b0 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7f6788004940 tx=0x7f6788009e30 comp rx=0 tx=0).stop 2024-04-12T21:34:31.136 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:34:31.135+0000 7f67a0c00700 1 -- 172.21.15.18:0/4186956612 shutdown_connections 2024-04-12T21:34:31.136 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:34:31.135+0000 7f67a0c00700 1 --2- 172.21.15.18:0/4186956612 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f679c109c40 0x7f679c1032b0 unknown :-1 s=CLOSED pgs=878 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:34:31.136 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:34:31.135+0000 7f67a0c00700 1 --2- 172.21.15.18:0/4186956612 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f679c1092a0 0x7f679c109670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:34:31.136 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:34:31.135+0000 7f67a0c00700 1 -- 172.21.15.18:0/4186956612 >> 172.21.15.18:0/4186956612 conn(0x7f679c072a30 msgr2=0x7f679c074e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:34:31.136 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:34:31.136+0000 7f67a0c00700 1 -- 172.21.15.18:0/4186956612 shutdown_connections 2024-04-12T21:34:31.136 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:34:31.136+0000 7f67a0c00700 1 -- 172.21.15.18:0/4186956612 wait complete. 2024-04-12T21:34:31.137 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:34:31.136+0000 7f67a0c00700 1 Processor -- start 2024-04-12T21:34:31.137 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:34:31.136+0000 7f67a0c00700 1 -- start start 2024-04-12T21:34:31.137 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:34:31.137+0000 7f67a0c00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f679c1092a0 0x7f679c11a010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:34:31.137 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:34:31.137+0000 7f67a0c00700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f679c109c40 0x7f679c11a550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:34:31.137 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:34:31.137+0000 7f67a0c00700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f679c11aba0 con 0x7f679c1092a0 2024-04-12T21:34:31.138 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:34:31.137+0000 7f67a0c00700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f679c113f70 con 0x7f679c109c40 2024-04-12T21:34:31.138 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:34:31.137+0000 7f679b400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f679c1092a0 0x7f679c11a010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:34:31.138 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:34:31.137+0000 7f679b400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f679c1092a0 0x7f679c11a010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:34550/0 (socket says 172.21.15.18:34550) 2024-04-12T21:34:31.138 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:34:31.137+0000 7f679b400700 1 -- 172.21.15.18:0/1058290456 learned_addr learned my addr 172.21.15.18:0/1058290456 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:34:31.138 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:34:31.138+0000 7f679aa00700 1 --2- 172.21.15.18:0/1058290456 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f679c109c40 0x7f679c11a550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:34:31.139 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:34:31.138+0000 7f679b400700 1 -- 172.21.15.18:0/1058290456 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f679c109c40 msgr2=0x7f679c11a550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:34:31.139 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:34:31.138+0000 7f679b400700 1 --2- 172.21.15.18:0/1058290456 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f679c109c40 0x7f679c11a550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:34:31.139 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:34:31.138+0000 7f679b400700 1 -- 172.21.15.18:0/1058290456 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6788005040 con 0x7f679c1092a0 2024-04-12T21:34:31.139 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:34:31.138+0000 7f679b400700 1 --2- 172.21.15.18:0/1058290456 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f679c1092a0 0x7f679c11a010 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7f678c00c810 tx=0x7f678c004470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:34:31.139 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:34:31.138+0000 7f6793400700 1 -- 172.21.15.18:0/1058290456 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f678c012810 con 0x7f679c1092a0 2024-04-12T21:34:31.139 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:34:31.139+0000 7f6793400700 1 -- 172.21.15.18:0/1058290456 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f678c012e40 con 0x7f679c1092a0 2024-04-12T21:34:31.139 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:34:31.139+0000 7f67a0c00700 1 -- 172.21.15.18:0/1058290456 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f679c114250 con 0x7f679c1092a0 2024-04-12T21:34:31.139 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:34:31.139+0000 7f6793400700 1 -- 172.21.15.18:0/1058290456 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f678c00a370 con 0x7f679c1092a0 2024-04-12T21:34:31.139 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:34:31.139+0000 7f67a0c00700 1 -- 172.21.15.18:0/1058290456 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f679c114810 con 0x7f679c1092a0 2024-04-12T21:34:31.140 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:34:31.140+0000 7f67a0c00700 1 -- 172.21.15.18:0/1058290456 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f679c1049f0 con 0x7f679c1092a0 2024-04-12T21:34:31.140 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:34:31.141+0000 7f6793400700 1 -- 172.21.15.18:0/1058290456 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7f678c011810 con 0x7f679c1092a0 2024-04-12T21:34:31.142 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:34:31.142+0000 7f6793400700 1 --2- 172.21.15.18:0/1058290456 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f678006ec00 0x7f67800710b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:34:31.142 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:34:31.142+0000 7f6793400700 1 -- 172.21.15.18:0/1058290456 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 6757+0+0 (secure 0 0 0) 0x7f678c08ede0 con 0x7f679c1092a0 2024-04-12T21:34:31.142 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:34:31.142+0000 7f679aa00700 1 --2- 172.21.15.18:0/1058290456 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f678006ec00 0x7f67800710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:34:31.142 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:34:31.143+0000 7f679aa00700 1 --2- 172.21.15.18:0/1058290456 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f678006ec00 0x7f67800710b0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f678800b020 tx=0x7f6788009e30 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:34:31.147 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:34:31.147+0000 7f6793400700 1 -- 172.21.15.18:0/1058290456 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f678c05b590 con 0x7f679c1092a0 2024-04-12T21:34:31.282 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:34:30 smithi186 ceph-mon[86379]: pgmap v1042: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:34:31.398 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:34:31.398+0000 7f67a0c00700 1 -- 172.21.15.18:0/1058290456 --> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f679c04bed0 con 0x7f678006ec00 2024-04-12T21:34:31.401 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:34:31.401+0000 7f6793400700 1 -- 172.21.15.18:0/1058290456 <== mgr.34102 v2:172.21.15.18:6800/3812891919 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f679c04bed0 con 0x7f678006ec00 2024-04-12T21:34:31.404 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:34:31.403+0000 7f67a0c00700 1 -- 172.21.15.18:0/1058290456 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f678006ec00 msgr2=0x7f67800710b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:34:31.404 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:34:31.403+0000 7f67a0c00700 1 --2- 172.21.15.18:0/1058290456 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f678006ec00 0x7f67800710b0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f678800b020 tx=0x7f6788009e30 comp rx=0 tx=0).stop 2024-04-12T21:34:31.404 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:34:31.403+0000 7f67a0c00700 1 -- 172.21.15.18:0/1058290456 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f679c1092a0 msgr2=0x7f679c11a010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:34:31.404 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:34:31.403+0000 7f67a0c00700 1 --2- 172.21.15.18:0/1058290456 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f679c1092a0 0x7f679c11a010 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7f678c00c810 tx=0x7f678c004470 comp rx=0 tx=0).stop 2024-04-12T21:34:31.404 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:34:31.404+0000 7f67a0c00700 1 -- 172.21.15.18:0/1058290456 shutdown_connections 2024-04-12T21:34:31.404 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:34:31.404+0000 7f67a0c00700 1 --2- 172.21.15.18:0/1058290456 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f678006ec00 0x7f67800710b0 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:34:31.404 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:34:31.404+0000 7f67a0c00700 1 --2- 172.21.15.18:0/1058290456 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f679c1092a0 0x7f679c11a010 unknown :-1 s=CLOSED pgs=879 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:34:31.404 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:34:31.404+0000 7f67a0c00700 1 --2- 172.21.15.18:0/1058290456 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f679c109c40 0x7f679c11a550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:34:31.404 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:34:31.404+0000 7f67a0c00700 1 -- 172.21.15.18:0/1058290456 >> 172.21.15.18:0/1058290456 conn(0x7f679c072a30 msgr2=0x7f679c074990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:34:31.405 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:34:31.404+0000 7f67a0c00700 1 -- 172.21.15.18:0/1058290456 shutdown_connections 2024-04-12T21:34:31.405 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:34:31.404+0000 7f67a0c00700 1 -- 172.21.15.18:0/1058290456 wait complete. 2024-04-12T21:34:31.560 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:34:31.560+0000 7f7fe9800700 1 -- 172.21.15.18:0/3239048912 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7fe4070c70 msgr2=0x7f7fe4109110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:34:31.560 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:34:31.560+0000 7f7fe9800700 1 --2- 172.21.15.18:0/3239048912 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7fe4070c70 0x7f7fe4109110 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7f7fd0004970 tx=0x7f7fd0009e30 comp rx=0 tx=0).stop 2024-04-12T21:34:31.560 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:34:31.560+0000 7f7fe9800700 1 -- 172.21.15.18:0/3239048912 shutdown_connections 2024-04-12T21:34:31.560 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:34:31.560+0000 7f7fe9800700 1 --2- 172.21.15.18:0/3239048912 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7fe4070c70 0x7f7fe4109110 unknown :-1 s=CLOSED pgs=880 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:34:31.560 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:34:31.560+0000 7f7fe9800700 1 --2- 172.21.15.18:0/3239048912 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7fe4070360 0x7f7fe4070730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:34:31.561 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:34:31.560+0000 7f7fe9800700 1 -- 172.21.15.18:0/3239048912 >> 172.21.15.18:0/3239048912 conn(0x7f7fe40f89b0 msgr2=0x7f7fe40fadc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:34:31.561 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:34:31.560+0000 7f7fe9800700 1 -- 172.21.15.18:0/3239048912 shutdown_connections 2024-04-12T21:34:31.561 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:34:31.560+0000 7f7fe9800700 1 -- 172.21.15.18:0/3239048912 wait complete. 2024-04-12T21:34:31.562 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:34:31.561+0000 7f7fe9800700 1 Processor -- start 2024-04-12T21:34:31.562 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:34:31.561+0000 7f7fe9800700 1 -- start start 2024-04-12T21:34:31.562 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:34:31.561+0000 7f7fe9800700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7fe4070360 0x7f7fe4115d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:34:31.562 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:34:31.561+0000 7f7fe9800700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7fe4070c70 0x7f7fe41162b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:34:31.562 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:34:31.561+0000 7f7fe9800700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7fe4116900 con 0x7f7fe4070c70 2024-04-12T21:34:31.563 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:34:31.561+0000 7f7fe9800700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7fe410fcd0 con 0x7f7fe4070360 2024-04-12T21:34:31.563 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:34:31.562+0000 7f7fe2000700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7fe4070c70 0x7f7fe41162b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:34:31.563 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:34:31.562+0000 7f7fe2000700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7fe4070c70 0x7f7fe41162b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:34560/0 (socket says 172.21.15.18:34560) 2024-04-12T21:34:31.563 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:34:31.562+0000 7f7fe2000700 1 -- 172.21.15.18:0/383878425 learned_addr learned my addr 172.21.15.18:0/383878425 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:34:31.563 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:34:31.562+0000 7f7fe2a00700 1 --2- 172.21.15.18:0/383878425 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7fe4070360 0x7f7fe4115d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:34:31.563 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:34:31.562+0000 7f7fe2000700 1 -- 172.21.15.18:0/383878425 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7fe4070360 msgr2=0x7f7fe4115d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:34:31.563 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:34:31.562+0000 7f7fe2000700 1 --2- 172.21.15.18:0/383878425 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7fe4070360 0x7f7fe4115d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:34:31.563 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:34:31.562+0000 7f7fe2000700 1 -- 172.21.15.18:0/383878425 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7fd0005040 con 0x7f7fe4070c70 2024-04-12T21:34:31.563 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:34:31.563+0000 7f7fe2000700 1 --2- 172.21.15.18:0/383878425 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7fe4070c70 0x7f7fe41162b0 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7f7fd0004f50 tx=0x7f7fd0009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:34:31.564 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:34:31.563+0000 7f7fdaa00700 1 -- 172.21.15.18:0/383878425 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7fd000e040 con 0x7f7fe4070c70 2024-04-12T21:34:31.564 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:34:31.563+0000 7f7fdaa00700 1 -- 172.21.15.18:0/383878425 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f7fd000bc10 con 0x7f7fe4070c70 2024-04-12T21:34:31.564 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:34:31.563+0000 7f7fe9800700 1 -- 172.21.15.18:0/383878425 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7fe410ff50 con 0x7f7fe4070c70 2024-04-12T21:34:31.564 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:34:31.563+0000 7f7fe9800700 1 -- 172.21.15.18:0/383878425 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7fe4110490 con 0x7f7fe4070c70 2024-04-12T21:34:31.564 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:34:31.563+0000 7f7fdaa00700 1 -- 172.21.15.18:0/383878425 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7fd001c630 con 0x7f7fe4070c70 2024-04-12T21:34:31.564 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:34:31.565+0000 7f7fe9800700 1 -- 172.21.15.18:0/383878425 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7fe4071aa0 con 0x7f7fe4070c70 2024-04-12T21:34:31.566 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:34:31.566+0000 7f7fdaa00700 1 -- 172.21.15.18:0/383878425 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7f7fd0013420 con 0x7f7fe4070c70 2024-04-12T21:34:31.566 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:34:31.567+0000 7f7fdaa00700 1 --2- 172.21.15.18:0/383878425 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f7fc806ec10 0x7f7fc80710c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:34:31.567 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:34:31.567+0000 7f7fdaa00700 1 -- 172.21.15.18:0/383878425 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 6757+0+0 (secure 0 0 0) 0x7f7fd008fc90 con 0x7f7fe4070c70 2024-04-12T21:34:31.567 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:34:31.567+0000 7f7fe2a00700 1 --2- 172.21.15.18:0/383878425 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f7fc806ec10 0x7f7fc80710c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:34:31.567 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:34:31.568+0000 7f7fe2a00700 1 --2- 172.21.15.18:0/383878425 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f7fc806ec10 0x7f7fc80710c0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f7fd4004990 tx=0x7f7fd4009ee0 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:34:31.571 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:34:31.572+0000 7f7fdaa00700 1 -- 172.21.15.18:0/383878425 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f7fd005c4e0 con 0x7f7fe4070c70 2024-04-12T21:34:31.813 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:34:31.813+0000 7f7fe9800700 1 -- 172.21.15.18:0/383878425 --> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f7fe4071aa0 con 0x7f7fc806ec10 2024-04-12T21:34:31.843 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:34:31.843+0000 7f7fdaa00700 1 -- 172.21.15.18:0/383878425 <== mgr.34102 v2:172.21.15.18:6800/3812891919 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7f7fe4071aa0 con 0x7f7fc806ec10 2024-04-12T21:34:31.843 INFO:teuthology.orchestra.run.smithi018.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-12T21:34:31.844 INFO:teuthology.orchestra.run.smithi018.stdout:alertmanager.smithi018 smithi018 *:9093,9094 running (35m) 11m ago 49m 15.5M - 0.25.0 c8568f914cd2 c332090068e8 2024-04-12T21:34:31.844 INFO:teuthology.orchestra.run.smithi018.stdout:crash.smithi018 smithi018 running (31m) 11m ago 49m 6665k - 19.0.0-2879-g324baf8d 8717720dc5e5 b7f80d1f7982 2024-04-12T21:34:31.844 INFO:teuthology.orchestra.run.smithi018.stdout:crash.smithi186 smithi186 running (31m) 11m ago 48m 6660k - 19.0.0-2879-g324baf8d 8717720dc5e5 a4f55c81b623 2024-04-12T21:34:31.844 INFO:teuthology.orchestra.run.smithi018.stdout:grafana.smithi018 smithi018 *:3000 running (34m) 11m ago 49m 74.8M - 9.4.12 2bacad6d85d8 b08e3bf176dd 2024-04-12T21:34:31.844 INFO:teuthology.orchestra.run.smithi018.stdout:mds.cephfs.smithi018.acqbsc smithi018 running (27m) 11m ago 42m 16.7M - 19.0.0-2879-g324baf8d 8717720dc5e5 021f99ab24ae 2024-04-12T21:34:31.844 INFO:teuthology.orchestra.run.smithi018.stdout:mds.cephfs.smithi018.szjluo smithi018 running (26m) 11m ago 42m 16.5M - 19.0.0-2879-g324baf8d 8717720dc5e5 0003117479d4 2024-04-12T21:34:31.844 INFO:teuthology.orchestra.run.smithi018.stdout:mds.cephfs.smithi186.jdllyn smithi186 running (11m) 11m ago 42m 11.0M - 19.0.0-2879-g324baf8d 8717720dc5e5 21b1f46eef23 2024-04-12T21:34:31.844 INFO:teuthology.orchestra.run.smithi018.stdout:mds.cephfs.smithi186.wjnafs smithi186 running (42m) 11m ago 42m 19.5M - 17.2.7-827-gd3b5060a 21de9b7369ad 74b115d6d6fd 2024-04-12T21:34:31.844 INFO:teuthology.orchestra.run.smithi018.stdout:mgr.smithi018.xjzwga smithi018 *:8443,9283,8765 running (37m) 11m ago 51m 512M - 19.0.0-2879-g324baf8d 8717720dc5e5 56088305eab7 2024-04-12T21:34:31.844 INFO:teuthology.orchestra.run.smithi018.stdout:mgr.smithi186.kqcfxk smithi186 *:8443,9283,8765 running (36m) 11m ago 48m 459M - 19.0.0-2879-g324baf8d 8717720dc5e5 09cd766e5e0d 2024-04-12T21:34:31.844 INFO:teuthology.orchestra.run.smithi018.stdout:mon.smithi018 smithi018 running (33m) 11m ago 51m 73.1M 2048M 19.0.0-2879-g324baf8d 8717720dc5e5 df3e201dfed6 2024-04-12T21:34:31.844 INFO:teuthology.orchestra.run.smithi018.stdout:mon.smithi186 smithi186 running (32m) 11m ago 48m 47.2M 2048M 19.0.0-2879-g324baf8d 8717720dc5e5 c77498da9b2b 2024-04-12T21:34:31.844 INFO:teuthology.orchestra.run.smithi018.stdout:node-exporter.smithi018 smithi018 *:9100 running (36m) 11m ago 49m 8108k - 1.5.0 0da6a335fe13 326c3d9c6847 2024-04-12T21:34:31.845 INFO:teuthology.orchestra.run.smithi018.stdout:node-exporter.smithi186 smithi186 *:9100 running (36m) 11m ago 48m 8757k - 1.5.0 0da6a335fe13 ec2112baeecf 2024-04-12T21:34:31.845 INFO:teuthology.orchestra.run.smithi018.stdout:osd.0 smithi018 running (30m) 11m ago 47m 123M 1559M 19.0.0-2879-g324baf8d 8717720dc5e5 60b1273ca047 2024-04-12T21:34:31.845 INFO:teuthology.orchestra.run.smithi018.stdout:osd.1 smithi018 running (30m) 11m ago 46m 95.1M 1559M 19.0.0-2879-g324baf8d 8717720dc5e5 170ce1e5fb8d 2024-04-12T21:34:31.845 INFO:teuthology.orchestra.run.smithi018.stdout:osd.2 smithi018 running (29m) 11m ago 46m 94.3M 1559M 19.0.0-2879-g324baf8d 8717720dc5e5 c2cbc6aa2839 2024-04-12T21:34:31.845 INFO:teuthology.orchestra.run.smithi018.stdout:osd.3 smithi186 running (29m) 11m ago 45m 101M 2581M 19.0.0-2879-g324baf8d 8717720dc5e5 0138e8262e4b 2024-04-12T21:34:31.845 INFO:teuthology.orchestra.run.smithi018.stdout:osd.4 smithi186 running (28m) 11m ago 44m 91.0M 2581M 19.0.0-2879-g324baf8d 8717720dc5e5 b20cb63b25c9 2024-04-12T21:34:31.845 INFO:teuthology.orchestra.run.smithi018.stdout:osd.5 smithi186 running (27m) 11m ago 44m 92.0M 2581M 19.0.0-2879-g324baf8d 8717720dc5e5 2513bea2610e 2024-04-12T21:34:31.845 INFO:teuthology.orchestra.run.smithi018.stdout:prometheus.smithi018 smithi018 *:9095 running (35m) 11m ago 48m 67.9M - 2.43.0 a07b618ecd1d f58841127e21 2024-04-12T21:34:31.847 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:34:31.847+0000 7f7fe9800700 1 -- 172.21.15.18:0/383878425 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f7fc806ec10 msgr2=0x7f7fc80710c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:34:31.847 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:34:31.847+0000 7f7fe9800700 1 --2- 172.21.15.18:0/383878425 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f7fc806ec10 0x7f7fc80710c0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f7fd4004990 tx=0x7f7fd4009ee0 comp rx=0 tx=0).stop 2024-04-12T21:34:31.847 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:34:31.847+0000 7f7fe9800700 1 -- 172.21.15.18:0/383878425 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7fe4070c70 msgr2=0x7f7fe41162b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:34:31.847 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:34:31.847+0000 7f7fe9800700 1 --2- 172.21.15.18:0/383878425 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7fe4070c70 0x7f7fe41162b0 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7f7fd0004f50 tx=0x7f7fd0009e30 comp rx=0 tx=0).stop 2024-04-12T21:34:31.847 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:34:31.847+0000 7f7fe9800700 1 -- 172.21.15.18:0/383878425 shutdown_connections 2024-04-12T21:34:31.847 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:34:31.847+0000 7f7fe9800700 1 --2- 172.21.15.18:0/383878425 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f7fc806ec10 0x7f7fc80710c0 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:34:31.848 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:34:31.847+0000 7f7fe9800700 1 --2- 172.21.15.18:0/383878425 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7fe4070360 0x7f7fe4115d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:34:31.848 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:34:31.847+0000 7f7fe9800700 1 --2- 172.21.15.18:0/383878425 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7fe4070c70 0x7f7fe41162b0 unknown :-1 s=CLOSED pgs=881 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:34:31.848 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:34:31.847+0000 7f7fe9800700 1 -- 172.21.15.18:0/383878425 >> 172.21.15.18:0/383878425 conn(0x7f7fe40f89b0 msgr2=0x7f7fe40f9e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:34:31.848 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:34:31.847+0000 7f7fe9800700 1 -- 172.21.15.18:0/383878425 shutdown_connections 2024-04-12T21:34:31.848 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:34:31.847+0000 7f7fe9800700 1 -- 172.21.15.18:0/383878425 wait complete. 2024-04-12T21:34:32.001 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:34:32.001+0000 7f5675400700 1 -- 172.21.15.18:0/3279298937 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f56700724f0 msgr2=0x7f56700728c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:34:32.001 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:34:32.001+0000 7f5675400700 1 --2- 172.21.15.18:0/3279298937 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f56700724f0 0x7f56700728c0 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7f5660004940 tx=0x7f5660009e30 comp rx=0 tx=0).stop 2024-04-12T21:34:32.002 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:34:32.001+0000 7f5675400700 1 -- 172.21.15.18:0/3279298937 shutdown_connections 2024-04-12T21:34:32.002 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:34:32.001+0000 7f5675400700 1 --2- 172.21.15.18:0/3279298937 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5670072e00 0x7f567010d5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:34:32.002 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:34:32.001+0000 7f5675400700 1 --2- 172.21.15.18:0/3279298937 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f56700724f0 0x7f56700728c0 unknown :-1 s=CLOSED pgs=882 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:34:32.002 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:34:32.001+0000 7f5675400700 1 -- 172.21.15.18:0/3279298937 >> 172.21.15.18:0/3279298937 conn(0x7f56700fab10 msgr2=0x7f56700fcef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:34:32.002 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:34:32.001+0000 7f5675400700 1 -- 172.21.15.18:0/3279298937 shutdown_connections 2024-04-12T21:34:32.002 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:34:32.001+0000 7f5675400700 1 -- 172.21.15.18:0/3279298937 wait complete. 2024-04-12T21:34:32.004 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:34:32.002+0000 7f5675400700 1 Processor -- start 2024-04-12T21:34:32.004 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:34:32.002+0000 7f5675400700 1 -- start start 2024-04-12T21:34:32.004 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:34:32.003+0000 7f5675400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f56700724f0 0x7f5670119ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:34:32.004 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:34:32.003+0000 7f5675400700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5670072e00 0x7f567011a410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:34:32.004 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:34:32.003+0000 7f5675400700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f567011aaf0 con 0x7f56700724f0 2024-04-12T21:34:32.004 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:34:32.003+0000 7f5675400700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5670113e30 con 0x7f5670072e00 2024-04-12T21:34:32.004 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:34:32.003+0000 7f566fe00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f56700724f0 0x7f5670119ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:34:32.004 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:34:32.003+0000 7f566fe00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f56700724f0 0x7f5670119ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:34586/0 (socket says 172.21.15.18:34586) 2024-04-12T21:34:32.004 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:34:32.003+0000 7f566fe00700 1 -- 172.21.15.18:0/3958443037 learned_addr learned my addr 172.21.15.18:0/3958443037 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:34:32.004 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:34:32.004+0000 7f566f400700 1 --2- 172.21.15.18:0/3958443037 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5670072e00 0x7f567011a410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:34:32.004 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:34:32.004+0000 7f566fe00700 1 -- 172.21.15.18:0/3958443037 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5670072e00 msgr2=0x7f567011a410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:34:32.004 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:34:32.004+0000 7f566fe00700 1 --2- 172.21.15.18:0/3958443037 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5670072e00 0x7f567011a410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:34:32.005 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:34:32.004+0000 7f566fe00700 1 -- 172.21.15.18:0/3958443037 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5660005040 con 0x7f56700724f0 2024-04-12T21:34:32.005 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:34:32.004+0000 7f566fe00700 1 --2- 172.21.15.18:0/3958443037 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f56700724f0 0x7f5670119ed0 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7f566000b3b0 tx=0x7f5660009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:34:32.005 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:34:32.004+0000 7f5667e00700 1 -- 172.21.15.18:0/3958443037 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f566000e040 con 0x7f56700724f0 2024-04-12T21:34:32.005 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:34:32.004+0000 7f5667e00700 1 -- 172.21.15.18:0/3958443037 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f5660013420 con 0x7f56700724f0 2024-04-12T21:34:32.006 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:34:32.004+0000 7f5667e00700 1 -- 172.21.15.18:0/3958443037 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f566001c630 con 0x7f56700724f0 2024-04-12T21:34:32.006 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:34:32.004+0000 7f5675400700 1 -- 172.21.15.18:0/3958443037 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5670114050 con 0x7f56700724f0 2024-04-12T21:34:32.006 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:34:32.005+0000 7f5675400700 1 -- 172.21.15.18:0/3958443037 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5670114610 con 0x7f56700724f0 2024-04-12T21:34:32.006 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:34:32.006+0000 7f5675400700 1 -- 172.21.15.18:0/3958443037 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5670073c30 con 0x7f56700724f0 2024-04-12T21:34:32.007 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:34:32.007+0000 7f5667e00700 1 -- 172.21.15.18:0/3958443037 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7f566000baf0 con 0x7f56700724f0 2024-04-12T21:34:32.008 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:34:32.008+0000 7f5667e00700 1 --2- 172.21.15.18:0/3958443037 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f565406ec00 0x7f56540710b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:34:32.008 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:34:32.008+0000 7f5667e00700 1 -- 172.21.15.18:0/3958443037 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 6757+0+0 (secure 0 0 0) 0x7f566008fc30 con 0x7f56700724f0 2024-04-12T21:34:32.008 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:34:32.008+0000 7f566f400700 1 --2- 172.21.15.18:0/3958443037 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f565406ec00 0x7f56540710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:34:32.009 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:34:32.009+0000 7f566f400700 1 --2- 172.21.15.18:0/3958443037 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f565406ec00 0x7f56540710b0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f565c006fd0 tx=0x7f565c008040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:34:32.012 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:34:32.013+0000 7f5667e00700 1 -- 172.21.15.18:0/3958443037 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f566005c460 con 0x7f56700724f0 2024-04-12T21:34:32.325 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:34:32.325+0000 7f5675400700 1 -- 172.21.15.18:0/3958443037 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f567004bf00 con 0x7f56700724f0 2024-04-12T21:34:32.326 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:34:32.326+0000 7f5667e00700 1 -- 172.21.15.18:0/3958443037 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+837 (secure 0 0 0) 0x7f566005bff0 con 0x7f56700724f0 2024-04-12T21:34:32.326 INFO:teuthology.orchestra.run.smithi018.stdout:{ 2024-04-12T21:34:32.327 INFO:teuthology.orchestra.run.smithi018.stdout: "mon": { 2024-04-12T21:34:32.327 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 19.0.0-2879-g324baf8d (324baf8ddb91dea46cac50cb535c306115791355) squid (dev)": 2 2024-04-12T21:34:32.327 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-12T21:34:32.327 INFO:teuthology.orchestra.run.smithi018.stdout: "mgr": { 2024-04-12T21:34:32.327 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 19.0.0-2879-g324baf8d (324baf8ddb91dea46cac50cb535c306115791355) squid (dev)": 2 2024-04-12T21:34:32.327 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-12T21:34:32.327 INFO:teuthology.orchestra.run.smithi018.stdout: "osd": { 2024-04-12T21:34:32.327 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 19.0.0-2879-g324baf8d (324baf8ddb91dea46cac50cb535c306115791355) squid (dev)": 6 2024-04-12T21:34:32.327 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-12T21:34:32.327 INFO:teuthology.orchestra.run.smithi018.stdout: "mds": { 2024-04-12T21:34:32.327 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 17.2.7-827-gd3b5060a (d3b5060a17449edfae9385cdf792afec109fca5d) quincy (stable)": 1, 2024-04-12T21:34:32.327 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 19.0.0-2879-g324baf8d (324baf8ddb91dea46cac50cb535c306115791355) squid (dev)": 3 2024-04-12T21:34:32.327 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-12T21:34:32.327 INFO:teuthology.orchestra.run.smithi018.stdout: "overall": { 2024-04-12T21:34:32.328 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 17.2.7-827-gd3b5060a (d3b5060a17449edfae9385cdf792afec109fca5d) quincy (stable)": 1, 2024-04-12T21:34:32.328 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 19.0.0-2879-g324baf8d (324baf8ddb91dea46cac50cb535c306115791355) squid (dev)": 13 2024-04-12T21:34:32.328 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-04-12T21:34:32.328 INFO:teuthology.orchestra.run.smithi018.stdout:} 2024-04-12T21:34:32.329 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:34:32.329+0000 7f5675400700 1 -- 172.21.15.18:0/3958443037 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f565406ec00 msgr2=0x7f56540710b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:34:32.329 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:34:32.329+0000 7f5675400700 1 --2- 172.21.15.18:0/3958443037 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f565406ec00 0x7f56540710b0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f565c006fd0 tx=0x7f565c008040 comp rx=0 tx=0).stop 2024-04-12T21:34:32.329 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:34:32.329+0000 7f5675400700 1 -- 172.21.15.18:0/3958443037 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f56700724f0 msgr2=0x7f5670119ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:34:32.329 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:34:32.329+0000 7f5675400700 1 --2- 172.21.15.18:0/3958443037 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f56700724f0 0x7f5670119ed0 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7f566000b3b0 tx=0x7f5660009e30 comp rx=0 tx=0).stop 2024-04-12T21:34:32.330 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:34:32.329+0000 7f5675400700 1 -- 172.21.15.18:0/3958443037 shutdown_connections 2024-04-12T21:34:32.330 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:34:32.329+0000 7f5675400700 1 --2- 172.21.15.18:0/3958443037 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f565406ec00 0x7f56540710b0 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:34:32.330 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:34:32.329+0000 7f5675400700 1 --2- 172.21.15.18:0/3958443037 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f56700724f0 0x7f5670119ed0 unknown :-1 s=CLOSED pgs=883 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:34:32.330 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:34:32.329+0000 7f5675400700 1 --2- 172.21.15.18:0/3958443037 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5670072e00 0x7f567011a410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:34:32.330 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:34:32.329+0000 7f5675400700 1 -- 172.21.15.18:0/3958443037 >> 172.21.15.18:0/3958443037 conn(0x7f56700fab10 msgr2=0x7f56700faee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:34:32.330 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:34:32.329+0000 7f5675400700 1 -- 172.21.15.18:0/3958443037 shutdown_connections 2024-04-12T21:34:32.330 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:34:32.329+0000 7f5675400700 1 -- 172.21.15.18:0/3958443037 wait complete. 2024-04-12T21:34:32.485 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:34:32.485+0000 7fe415000700 1 -- 172.21.15.18:0/3571309894 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe4100fe830 msgr2=0x7fe410106f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:34:32.485 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:34:32.485+0000 7fe415000700 1 --2- 172.21.15.18:0/3571309894 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe4100fe830 0x7fe410106f00 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7fe3fc004940 tx=0x7fe3fc009e30 comp rx=0 tx=0).stop 2024-04-12T21:34:32.485 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:34:32.485+0000 7fe415000700 1 -- 172.21.15.18:0/3571309894 shutdown_connections 2024-04-12T21:34:32.485 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:34:32.485+0000 7fe415000700 1 --2- 172.21.15.18:0/3571309894 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe4100fe830 0x7fe410106f00 unknown :-1 s=CLOSED pgs=884 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:34:32.485 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:34:32.485+0000 7fe415000700 1 --2- 172.21.15.18:0/3571309894 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe4100fde90 0x7fe4100fe260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:34:32.486 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:34:32.485+0000 7fe415000700 1 -- 172.21.15.18:0/3571309894 >> 172.21.15.18:0/3571309894 conn(0x7fe410072bc0 msgr2=0x7fe410072fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:34:32.486 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:34:32.485+0000 7fe415000700 1 -- 172.21.15.18:0/3571309894 shutdown_connections 2024-04-12T21:34:32.486 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:34:32.485+0000 7fe415000700 1 -- 172.21.15.18:0/3571309894 wait complete. 2024-04-12T21:34:32.487 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:34:32.486+0000 7fe415000700 1 Processor -- start 2024-04-12T21:34:32.487 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:34:32.486+0000 7fe415000700 1 -- start start 2024-04-12T21:34:32.487 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:34:32.486+0000 7fe415000700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe4100fde90 0x7fe4100fbd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:34:32.487 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:34:32.486+0000 7fe415000700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe4100fe830 0x7fe4100fc290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:34:32.488 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:34:32.486+0000 7fe415000700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4100fc820 con 0x7fe4100fe830 2024-04-12T21:34:32.488 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:34:32.486+0000 7fe415000700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4100fc960 con 0x7fe4100fde90 2024-04-12T21:34:32.488 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:34:32.487+0000 7fe40d400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe4100fe830 0x7fe4100fc290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:34:32.488 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:34:32.487+0000 7fe40d400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe4100fe830 0x7fe4100fc290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:34602/0 (socket says 172.21.15.18:34602) 2024-04-12T21:34:32.488 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:34:32.487+0000 7fe40d400700 1 -- 172.21.15.18:0/3919034630 learned_addr learned my addr 172.21.15.18:0/3919034630 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:34:32.488 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:34:32.487+0000 7fe40de00700 1 --2- 172.21.15.18:0/3919034630 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe4100fde90 0x7fe4100fbd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:34:32.488 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:34:32.487+0000 7fe40d400700 1 -- 172.21.15.18:0/3919034630 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe4100fde90 msgr2=0x7fe4100fbd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:34:32.488 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:34:32.487+0000 7fe40d400700 1 --2- 172.21.15.18:0/3919034630 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe4100fde90 0x7fe4100fbd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:34:32.488 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:34:32.487+0000 7fe40d400700 1 -- 172.21.15.18:0/3919034630 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3fc005040 con 0x7fe4100fe830 2024-04-12T21:34:32.488 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:34:32.487+0000 7fe40de00700 1 --2- 172.21.15.18:0/3919034630 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe4100fde90 0x7fe4100fbd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-12T21:34:32.488 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:34:32.488+0000 7fe40d400700 1 --2- 172.21.15.18:0/3919034630 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe4100fe830 0x7fe4100fc290 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7fe3fc000c00 tx=0x7fe3fc009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:34:32.489 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:34:32.488+0000 7fe406000700 1 -- 172.21.15.18:0/3919034630 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe3fc00e040 con 0x7fe4100fe830 2024-04-12T21:34:32.489 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:34:32.488+0000 7fe406000700 1 -- 172.21.15.18:0/3919034630 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe3fc013420 con 0x7fe4100fe830 2024-04-12T21:34:32.489 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:34:32.488+0000 7fe415000700 1 -- 172.21.15.18:0/3919034630 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe4100fcbb0 con 0x7fe4100fe830 2024-04-12T21:34:32.489 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:34:32.488+0000 7fe406000700 1 -- 172.21.15.18:0/3919034630 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe3fc01c610 con 0x7fe4100fe830 2024-04-12T21:34:32.489 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:34:32.488+0000 7fe415000700 1 -- 172.21.15.18:0/3919034630 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe41011a2c0 con 0x7fe4100fe830 2024-04-12T21:34:32.489 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:34:32.489+0000 7fe415000700 1 -- 172.21.15.18:0/3919034630 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe410065e30 con 0x7fe4100fe830 2024-04-12T21:34:32.490 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:34:32.490+0000 7fe406000700 1 -- 172.21.15.18:0/3919034630 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7fe3fc013590 con 0x7fe4100fe830 2024-04-12T21:34:32.491 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:34:32.491+0000 7fe406000700 1 --2- 172.21.15.18:0/3919034630 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fe3f406ec10 0x7fe3f40710c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:34:32.491 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:34:32.491+0000 7fe406000700 1 -- 172.21.15.18:0/3919034630 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 6757+0+0 (secure 0 0 0) 0x7fe3fc08fcd0 con 0x7fe4100fe830 2024-04-12T21:34:32.491 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:34:32.492+0000 7fe40de00700 1 --2- 172.21.15.18:0/3919034630 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fe3f406ec10 0x7fe3f40710c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:34:32.492 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:34:32.492+0000 7fe40de00700 1 --2- 172.21.15.18:0/3919034630 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fe3f406ec10 0x7fe3f40710c0 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7fe400009fd0 tx=0x7fe40000b040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:34:32.496 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:34:32.497+0000 7fe406000700 1 -- 172.21.15.18:0/3919034630 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7fe3fc0588d0 con 0x7fe4100fe830 2024-04-12T21:34:32.773 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:34:32.773+0000 7fe415000700 1 -- 172.21.15.18:0/3919034630 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fe41004bed0 con 0x7fe4100fe830 2024-04-12T21:34:32.774 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:34:32.774+0000 7fe406000700 1 -- 172.21.15.18:0/3919034630 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 31 v31) v1 ==== 76+0+1873 (secure 0 0 0) 0x7fe3fc05bf90 con 0x7fe4100fe830 2024-04-12T21:34:32.775 INFO:teuthology.orchestra.run.smithi018.stdout:e31 2024-04-12T21:34:32.775 INFO:teuthology.orchestra.run.smithi018.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-12T21:34:32.775 INFO:teuthology.orchestra.run.smithi018.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-12T21:34:32.775 INFO:teuthology.orchestra.run.smithi018.stdout:legacy client fscid: 1 2024-04-12T21:34:32.775 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-12T21:34:32.775 INFO:teuthology.orchestra.run.smithi018.stdout:Filesystem 'cephfs' (1) 2024-04-12T21:34:32.775 INFO:teuthology.orchestra.run.smithi018.stdout:fs_name cephfs 2024-04-12T21:34:32.775 INFO:teuthology.orchestra.run.smithi018.stdout:epoch 24 2024-04-12T21:34:32.775 INFO:teuthology.orchestra.run.smithi018.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-04-12T21:34:32.775 INFO:teuthology.orchestra.run.smithi018.stdout:created 2024-04-12T20:51:57.362850+0000 2024-04-12T21:34:32.775 INFO:teuthology.orchestra.run.smithi018.stdout:modified 2024-04-12T21:06:59.717190+0000 2024-04-12T21:34:32.775 INFO:teuthology.orchestra.run.smithi018.stdout:tableserver 0 2024-04-12T21:34:32.775 INFO:teuthology.orchestra.run.smithi018.stdout:root 0 2024-04-12T21:34:32.776 INFO:teuthology.orchestra.run.smithi018.stdout:session_timeout 60 2024-04-12T21:34:32.776 INFO:teuthology.orchestra.run.smithi018.stdout:session_autoclose 300 2024-04-12T21:34:32.776 INFO:teuthology.orchestra.run.smithi018.stdout:max_file_size 1099511627776 2024-04-12T21:34:32.776 INFO:teuthology.orchestra.run.smithi018.stdout:max_xattr_size 65536 2024-04-12T21:34:32.776 INFO:teuthology.orchestra.run.smithi018.stdout:required_client_features {} 2024-04-12T21:34:32.776 INFO:teuthology.orchestra.run.smithi018.stdout:last_failure 0 2024-04-12T21:34:32.776 INFO:teuthology.orchestra.run.smithi018.stdout:last_failure_osd_epoch 100 2024-04-12T21:34:32.776 INFO:teuthology.orchestra.run.smithi018.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-12T21:34:32.776 INFO:teuthology.orchestra.run.smithi018.stdout:max_mds 2 2024-04-12T21:34:32.776 INFO:teuthology.orchestra.run.smithi018.stdout:in 0,1 2024-04-12T21:34:32.776 INFO:teuthology.orchestra.run.smithi018.stdout:up {} 2024-04-12T21:34:32.776 INFO:teuthology.orchestra.run.smithi018.stdout:failed 0,1 2024-04-12T21:34:32.776 INFO:teuthology.orchestra.run.smithi018.stdout:damaged 2024-04-12T21:34:32.776 INFO:teuthology.orchestra.run.smithi018.stdout:stopped 2024-04-12T21:34:32.777 INFO:teuthology.orchestra.run.smithi018.stdout:data_pools [3] 2024-04-12T21:34:32.777 INFO:teuthology.orchestra.run.smithi018.stdout:metadata_pool 2 2024-04-12T21:34:32.777 INFO:teuthology.orchestra.run.smithi018.stdout:inline_data disabled 2024-04-12T21:34:32.777 INFO:teuthology.orchestra.run.smithi018.stdout:balancer 2024-04-12T21:34:32.777 INFO:teuthology.orchestra.run.smithi018.stdout:bal_rank_mask -1 2024-04-12T21:34:32.777 INFO:teuthology.orchestra.run.smithi018.stdout:standby_count_wanted 1 2024-04-12T21:34:32.777 INFO:teuthology.orchestra.run.smithi018.stdout:qdb_cluster leader: 0 members: 2024-04-12T21:34:32.777 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-12T21:34:32.777 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-12T21:34:32.777 INFO:teuthology.orchestra.run.smithi018.stdout:Standby daemons: 2024-04-12T21:34:32.777 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-12T21:34:32.777 INFO:teuthology.orchestra.run.smithi018.stdout:[mds.cephfs.smithi186.wjnafs{-1:24277} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.186:6826/438347207,v1:172.21.15.186:6827/438347207] compat {c=[1],r=[1],i=[7ff]}] 2024-04-12T21:34:32.777 INFO:teuthology.orchestra.run.smithi018.stdout:[mds.cephfs.smithi018.acqbsc{-1:34436} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.18:6828/755179764,v1:172.21.15.18:6829/755179764] compat {c=[1],r=[1],i=[1fff]}] 2024-04-12T21:34:32.777 INFO:teuthology.orchestra.run.smithi018.stdout:[mds.cephfs.smithi018.szjluo{-1:34468} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.18:6826/2407992653,v1:172.21.15.18:6827/2407992653] compat {c=[1],r=[1],i=[1fff]}] 2024-04-12T21:34:32.778 INFO:teuthology.orchestra.run.smithi018.stdout:[mds.cephfs.smithi186.jdllyn{-1:44595} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.186:6824/3523444202,v1:172.21.15.186:6825/3523444202] compat {c=[1],r=[1],i=[1fff]}] 2024-04-12T21:34:32.778 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:34:32.777+0000 7fe415000700 1 -- 172.21.15.18:0/3919034630 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fe3f406ec10 msgr2=0x7fe3f40710c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:34:32.778 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:34:32.777+0000 7fe415000700 1 --2- 172.21.15.18:0/3919034630 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fe3f406ec10 0x7fe3f40710c0 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7fe400009fd0 tx=0x7fe40000b040 comp rx=0 tx=0).stop 2024-04-12T21:34:32.779 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:34:32.777+0000 7fe415000700 1 -- 172.21.15.18:0/3919034630 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe4100fe830 msgr2=0x7fe4100fc290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:34:32.779 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:34:32.777+0000 7fe415000700 1 --2- 172.21.15.18:0/3919034630 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe4100fe830 0x7fe4100fc290 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7fe3fc000c00 tx=0x7fe3fc009e30 comp rx=0 tx=0).stop 2024-04-12T21:34:32.779 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:34:32.777+0000 7fe415000700 1 -- 172.21.15.18:0/3919034630 shutdown_connections 2024-04-12T21:34:32.779 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:34:32.777+0000 7fe415000700 1 --2- 172.21.15.18:0/3919034630 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fe3f406ec10 0x7fe3f40710c0 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:34:32.779 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:34:32.777+0000 7fe415000700 1 --2- 172.21.15.18:0/3919034630 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe4100fde90 0x7fe4100fbd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:34:32.779 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:34:32.777+0000 7fe415000700 1 --2- 172.21.15.18:0/3919034630 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe4100fe830 0x7fe4100fc290 unknown :-1 s=CLOSED pgs=885 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:34:32.779 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:34:32.777+0000 7fe415000700 1 -- 172.21.15.18:0/3919034630 >> 172.21.15.18:0/3919034630 conn(0x7fe410072bc0 msgr2=0x7fe410103820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:34:32.779 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:34:32.777+0000 7fe415000700 1 -- 172.21.15.18:0/3919034630 shutdown_connections 2024-04-12T21:34:32.779 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:34:32.777+0000 7fe415000700 1 -- 172.21.15.18:0/3919034630 wait complete. 2024-04-12T21:34:32.779 INFO:teuthology.orchestra.run.smithi018.stderr:dumped fsmap epoch 31 2024-04-12T21:34:32.925 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:34:32.925+0000 7f9283200700 1 -- 172.21.15.18:0/1163225574 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9284102c40 msgr2=0x7f9284103010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:34:32.925 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:34:32.925+0000 7f9283200700 1 --2- 172.21.15.18:0/1163225574 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9284102c40 0x7f9284103010 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7f9274009ce0 tx=0x7f9274004d10 comp rx=0 tx=0).stop 2024-04-12T21:34:32.925 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:34:32.925+0000 7f9283200700 1 -- 172.21.15.18:0/1163225574 shutdown_connections 2024-04-12T21:34:32.925 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:34:32.925+0000 7f9283200700 1 --2- 172.21.15.18:0/1163225574 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f92841035e0 0x7f92840fcc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:34:32.926 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:34:32.925+0000 7f9283200700 1 --2- 172.21.15.18:0/1163225574 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9284102c40 0x7f9284103010 unknown :-1 s=CLOSED pgs=886 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:34:32.926 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:34:32.925+0000 7f9283200700 1 -- 172.21.15.18:0/1163225574 >> 172.21.15.18:0/1163225574 conn(0x7f92840f8830 msgr2=0x7f92840fac40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:34:32.926 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:34:32.925+0000 7f9283200700 1 -- 172.21.15.18:0/1163225574 shutdown_connections 2024-04-12T21:34:32.926 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:34:32.925+0000 7f9283200700 1 -- 172.21.15.18:0/1163225574 wait complete. 2024-04-12T21:34:32.927 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:34:32.926+0000 7f9283200700 1 Processor -- start 2024-04-12T21:34:32.927 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:34:32.926+0000 7f9283200700 1 -- start start 2024-04-12T21:34:32.927 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:34:32.926+0000 7f9283200700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9284102c40 0x7f9284115dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:34:32.927 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:34:32.927+0000 7f9283200700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f92841035e0 0x7f9284116310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:34:32.928 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:34:32.927+0000 7f9283200700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92841169f0 con 0x7f9284102c40 2024-04-12T21:34:32.928 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:34:32.927+0000 7f9283200700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f928410fd30 con 0x7f92841035e0 2024-04-12T21:34:32.928 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:34:32.927+0000 7f9281e00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9284102c40 0x7f9284115dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:34:32.928 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:34:32.927+0000 7f9281e00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9284102c40 0x7f9284115dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:34634/0 (socket says 172.21.15.18:34634) 2024-04-12T21:34:32.928 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:34:32.927+0000 7f9281e00700 1 -- 172.21.15.18:0/2397798263 learned_addr learned my addr 172.21.15.18:0/2397798263 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:34:32.928 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:34:32.927+0000 7f9279400700 1 --2- 172.21.15.18:0/2397798263 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f92841035e0 0x7f9284116310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:34:32.928 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:34:32.927+0000 7f9281e00700 1 -- 172.21.15.18:0/2397798263 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f92841035e0 msgr2=0x7f9284116310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:34:32.928 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:34:32.927+0000 7f9281e00700 1 --2- 172.21.15.18:0/2397798263 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f92841035e0 0x7f9284116310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:34:32.928 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:34:32.927+0000 7f9281e00700 1 -- 172.21.15.18:0/2397798263 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9274009940 con 0x7f9284102c40 2024-04-12T21:34:32.928 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:34:32.928+0000 7f9281e00700 1 --2- 172.21.15.18:0/2397798263 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9284102c40 0x7f9284115dd0 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7f9274005010 tx=0x7f9274004d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:34:32.929 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:34:32.928+0000 7f927aa00700 1 -- 172.21.15.18:0/2397798263 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f927400e040 con 0x7f9284102c40 2024-04-12T21:34:32.929 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:34:32.928+0000 7f927aa00700 1 -- 172.21.15.18:0/2397798263 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f9274008e20 con 0x7f9284102c40 2024-04-12T21:34:32.929 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:34:32.928+0000 7f927aa00700 1 -- 172.21.15.18:0/2397798263 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f927401c660 con 0x7f9284102c40 2024-04-12T21:34:32.929 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:34:32.928+0000 7f9283200700 1 -- 172.21.15.18:0/2397798263 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f928410ff50 con 0x7f9284102c40 2024-04-12T21:34:32.929 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:34:32.928+0000 7f9283200700 1 -- 172.21.15.18:0/2397798263 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9284110510 con 0x7f9284102c40 2024-04-12T21:34:32.929 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:34:32.929+0000 7f9283200700 1 -- 172.21.15.18:0/2397798263 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f92840fe390 con 0x7f9284102c40 2024-04-12T21:34:32.930 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:34:32.930+0000 7f927aa00700 1 -- 172.21.15.18:0/2397798263 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7f9274012a70 con 0x7f9284102c40 2024-04-12T21:34:32.931 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:34:32.931+0000 7f927aa00700 1 --2- 172.21.15.18:0/2397798263 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f926806ec10 0x7f92680710c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:34:32.932 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:34:32.931+0000 7f927aa00700 1 -- 172.21.15.18:0/2397798263 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 6757+0+0 (secure 0 0 0) 0x7f9274090530 con 0x7f9284102c40 2024-04-12T21:34:32.932 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:34:32.932+0000 7f9279400700 1 --2- 172.21.15.18:0/2397798263 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f926806ec10 0x7f92680710c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:34:32.932 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:34:32.932+0000 7f9279400700 1 --2- 172.21.15.18:0/2397798263 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f926806ec10 0x7f92680710c0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f926c006fd0 tx=0x7f926c008040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:34:32.935 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:34:32.936+0000 7f927aa00700 1 -- 172.21.15.18:0/2397798263 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f927405cce0 con 0x7f9284102c40 2024-04-12T21:34:33.103 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:34:32 smithi018 ceph-mon[112590]: from='client.35820 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:34:33.104 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:34:32 smithi018 ceph-mon[112590]: from='client.35824 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:34:33.104 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:34:32 smithi018 ceph-mon[112590]: pgmap v1043: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:34:33.104 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:34:32 smithi018 ceph-mon[112590]: from='client.35828 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:34:33.104 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:34:32 smithi018 ceph-mon[112590]: from='client.? 172.21.15.18:0/3958443037' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T21:34:33.104 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:34:32 smithi018 ceph-mon[112590]: from='client.? 172.21.15.18:0/3919034630' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-12T21:34:33.177 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:34:33.177+0000 7f9283200700 1 -- 172.21.15.18:0/2397798263 --> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f9284111420 con 0x7f926806ec10 2024-04-12T21:34:33.179 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:34:33.179+0000 7f927aa00700 1 -- 172.21.15.18:0/2397798263 <== mgr.34102 v2:172.21.15.18:6800/3812891919 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f9284111420 con 0x7f926806ec10 2024-04-12T21:34:33.180 INFO:teuthology.orchestra.run.smithi018.stdout:{ 2024-04-12T21:34:33.180 INFO:teuthology.orchestra.run.smithi018.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:f4452def3be4d350f4a8e5feaf7ebb1511be9298fe68478ba7a2d6fbd7d69a63", 2024-04-12T21:34:33.180 INFO:teuthology.orchestra.run.smithi018.stdout: "in_progress": true, 2024-04-12T21:34:33.180 INFO:teuthology.orchestra.run.smithi018.stdout: "which": "Upgrading all daemon types on all hosts", 2024-04-12T21:34:33.180 INFO:teuthology.orchestra.run.smithi018.stdout: "services_complete": [ 2024-04-12T21:34:33.180 INFO:teuthology.orchestra.run.smithi018.stdout: "crash", 2024-04-12T21:34:33.180 INFO:teuthology.orchestra.run.smithi018.stdout: "osd", 2024-04-12T21:34:33.180 INFO:teuthology.orchestra.run.smithi018.stdout: "mon", 2024-04-12T21:34:33.180 INFO:teuthology.orchestra.run.smithi018.stdout: "mgr" 2024-04-12T21:34:33.181 INFO:teuthology.orchestra.run.smithi018.stdout: ], 2024-04-12T21:34:33.181 INFO:teuthology.orchestra.run.smithi018.stdout: "progress": "15/21 daemons upgraded", 2024-04-12T21:34:33.181 INFO:teuthology.orchestra.run.smithi018.stdout: "message": "Currently upgrading mds daemons", 2024-04-12T21:34:33.181 INFO:teuthology.orchestra.run.smithi018.stdout: "is_paused": false 2024-04-12T21:34:33.181 INFO:teuthology.orchestra.run.smithi018.stdout:} 2024-04-12T21:34:33.182 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:34:33.182+0000 7f9283200700 1 -- 172.21.15.18:0/2397798263 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f926806ec10 msgr2=0x7f92680710c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:34:33.182 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:34:33.182+0000 7f9283200700 1 --2- 172.21.15.18:0/2397798263 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f926806ec10 0x7f92680710c0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f926c006fd0 tx=0x7f926c008040 comp rx=0 tx=0).stop 2024-04-12T21:34:33.182 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:34:33.182+0000 7f9283200700 1 -- 172.21.15.18:0/2397798263 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9284102c40 msgr2=0x7f9284115dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:34:33.182 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:34:33.182+0000 7f9283200700 1 --2- 172.21.15.18:0/2397798263 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9284102c40 0x7f9284115dd0 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7f9274005010 tx=0x7f9274004d10 comp rx=0 tx=0).stop 2024-04-12T21:34:33.183 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:34:33.183+0000 7f9283200700 1 -- 172.21.15.18:0/2397798263 shutdown_connections 2024-04-12T21:34:33.183 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:34:33.183+0000 7f9283200700 1 --2- 172.21.15.18:0/2397798263 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f926806ec10 0x7f92680710c0 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:34:33.183 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:34:33.183+0000 7f9283200700 1 --2- 172.21.15.18:0/2397798263 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9284102c40 0x7f9284115dd0 unknown :-1 s=CLOSED pgs=887 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:34:33.183 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:34:33.183+0000 7f9283200700 1 --2- 172.21.15.18:0/2397798263 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f92841035e0 0x7f9284116310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:34:33.183 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:34:33.183+0000 7f9283200700 1 -- 172.21.15.18:0/2397798263 >> 172.21.15.18:0/2397798263 conn(0x7f92840f8830 msgr2=0x7f9284071ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:34:33.183 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:34:33.183+0000 7f9283200700 1 -- 172.21.15.18:0/2397798263 shutdown_connections 2024-04-12T21:34:33.183 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:34:33.183+0000 7f9283200700 1 -- 172.21.15.18:0/2397798263 wait complete. 2024-04-12T21:34:33.282 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:34:32 smithi186 ceph-mon[86379]: from='client.35820 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:34:33.283 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:34:32 smithi186 ceph-mon[86379]: from='client.35824 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:34:33.283 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:34:32 smithi186 ceph-mon[86379]: pgmap v1043: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:34:33.283 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:34:32 smithi186 ceph-mon[86379]: from='client.35828 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:34:33.283 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:34:32 smithi186 ceph-mon[86379]: from='client.? 172.21.15.18:0/3958443037' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T21:34:33.283 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:34:32 smithi186 ceph-mon[86379]: from='client.? 172.21.15.18:0/3919034630' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-12T21:34:33.327 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:34:33.326+0000 7f5f93200700 1 -- 172.21.15.18:0/3482333896 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5f94104e60 msgr2=0x7f5f94105230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:34:33.327 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:34:33.326+0000 7f5f93200700 1 --2- 172.21.15.18:0/3482333896 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5f94104e60 0x7f5f94105230 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7f5f84004940 tx=0x7f5f84009e30 comp rx=0 tx=0).stop 2024-04-12T21:34:33.327 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:34:33.326+0000 7f5f93200700 1 -- 172.21.15.18:0/3482333896 shutdown_connections 2024-04-12T21:34:33.327 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:34:33.326+0000 7f5f93200700 1 --2- 172.21.15.18:0/3482333896 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5f94105800 0x7f5f940fee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:34:33.327 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:34:33.326+0000 7f5f93200700 1 --2- 172.21.15.18:0/3482333896 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5f94104e60 0x7f5f94105230 unknown :-1 s=CLOSED pgs=888 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:34:33.327 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:34:33.326+0000 7f5f93200700 1 -- 172.21.15.18:0/3482333896 >> 172.21.15.18:0/3482333896 conn(0x7f5f940fab10 msgr2=0x7f5f940fcef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:34:33.327 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:34:33.327+0000 7f5f93200700 1 -- 172.21.15.18:0/3482333896 shutdown_connections 2024-04-12T21:34:33.327 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:34:33.327+0000 7f5f93200700 1 -- 172.21.15.18:0/3482333896 wait complete. 2024-04-12T21:34:33.329 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:34:33.328+0000 7f5f93200700 1 Processor -- start 2024-04-12T21:34:33.329 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:34:33.328+0000 7f5f93200700 1 -- start start 2024-04-12T21:34:33.330 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:34:33.328+0000 7f5f93200700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5f94104e60 0x7f5f94115c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:34:33.330 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:34:33.328+0000 7f5f93200700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5f94105800 0x7f5f94116160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:34:33.330 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:34:33.328+0000 7f5f93200700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f941167b0 con 0x7f5f94105800 2024-04-12T21:34:33.330 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:34:33.328+0000 7f5f93200700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f9410fba0 con 0x7f5f94104e60 2024-04-12T21:34:33.330 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:34:33.328+0000 7f5f91400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5f94105800 0x7f5f94116160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:34:33.330 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:34:33.328+0000 7f5f91400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5f94105800 0x7f5f94116160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:34656/0 (socket says 172.21.15.18:34656) 2024-04-12T21:34:33.330 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:34:33.328+0000 7f5f91400700 1 -- 172.21.15.18:0/3029028834 learned_addr learned my addr 172.21.15.18:0/3029028834 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:34:33.330 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:34:33.329+0000 7f5f91e00700 1 --2- 172.21.15.18:0/3029028834 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5f94104e60 0x7f5f94115c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:34:33.330 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:34:33.329+0000 7f5f91400700 1 -- 172.21.15.18:0/3029028834 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5f94104e60 msgr2=0x7f5f94115c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:34:33.330 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:34:33.329+0000 7f5f91400700 1 --2- 172.21.15.18:0/3029028834 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5f94104e60 0x7f5f94115c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:34:33.330 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:34:33.329+0000 7f5f91400700 1 -- 172.21.15.18:0/3029028834 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f84005040 con 0x7f5f94105800 2024-04-12T21:34:33.330 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:34:33.329+0000 7f5f91400700 1 --2- 172.21.15.18:0/3029028834 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5f94105800 0x7f5f94116160 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7f5f8000c800 tx=0x7f5f80004470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:34:33.330 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:34:33.329+0000 7f5f8a000700 1 -- 172.21.15.18:0/3029028834 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5f8000a760 con 0x7f5f94105800 2024-04-12T21:34:33.331 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:34:33.329+0000 7f5f8a000700 1 -- 172.21.15.18:0/3029028834 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f5f8000ad90 con 0x7f5f94105800 2024-04-12T21:34:33.331 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:34:33.329+0000 7f5f93200700 1 -- 172.21.15.18:0/3029028834 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5f9410fe80 con 0x7f5f94105800 2024-04-12T21:34:33.331 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:34:33.329+0000 7f5f8a000700 1 -- 172.21.15.18:0/3029028834 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5f800113a0 con 0x7f5f94105800 2024-04-12T21:34:33.331 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:34:33.330+0000 7f5f93200700 1 -- 172.21.15.18:0/3029028834 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5f94110440 con 0x7f5f94105800 2024-04-12T21:34:33.331 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:34:33.331+0000 7f5f93200700 1 -- 172.21.15.18:0/3029028834 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5f941005b0 con 0x7f5f94105800 2024-04-12T21:34:33.331 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:34:33.332+0000 7f5f8a000700 1 -- 172.21.15.18:0/3029028834 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7f5f80012400 con 0x7f5f94105800 2024-04-12T21:34:33.333 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:34:33.333+0000 7f5f8a000700 1 --2- 172.21.15.18:0/3029028834 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f5f7806ec10 0x7f5f780710c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:34:33.333 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:34:33.333+0000 7f5f8a000700 1 -- 172.21.15.18:0/3029028834 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 6757+0+0 (secure 0 0 0) 0x7f5f8008ea40 con 0x7f5f94105800 2024-04-12T21:34:33.333 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:34:33.333+0000 7f5f91e00700 1 --2- 172.21.15.18:0/3029028834 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f5f7806ec10 0x7f5f780710c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:34:33.333 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:34:33.334+0000 7f5f91e00700 1 --2- 172.21.15.18:0/3029028834 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f5f7806ec10 0x7f5f780710c0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f5f8400b3b0 tx=0x7f5f84009e30 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:34:33.337 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:34:33.338+0000 7f5f8a000700 1 -- 172.21.15.18:0/3029028834 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f5f8005b1f0 con 0x7f5f94105800 2024-04-12T21:34:33.655 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:34:33.655+0000 7f5f93200700 1 -- 172.21.15.18:0/3029028834 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f5f9410bf60 con 0x7f5f94105800 2024-04-12T21:34:33.655 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:34:33.656+0000 7f5f8a000700 1 -- 172.21.15.18:0/3029028834 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+4303 (secure 0 0 0) 0x7f5f8005ad80 con 0x7f5f94105800 2024-04-12T21:34:33.656 INFO:teuthology.orchestra.run.smithi018.stdout:HEALTH_ERR failed to probe daemons or devices; 1 filesystem is degraded; 1 filesystem is offline 2024-04-12T21:34:33.656 INFO:teuthology.orchestra.run.smithi018.stdout:[WRN] CEPHADM_REFRESH_FAILED: failed to probe daemons or devices 2024-04-12T21:34:33.656 INFO:teuthology.orchestra.run.smithi018.stdout: host smithi018 `cephadm ceph-volume` failed: cephadm exited with an error code: 1, stderr: Inferring config /var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/mon.smithi018/config 2024-04-12T21:34:33.656 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/podman: timeout after 895 seconds 2024-04-12T21:34:33.656 INFO:teuthology.orchestra.run.smithi018.stdout:Non-zero exit code 124 from /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:f4452def3be4d350f4a8e5feaf7ebb1511be9298fe68478ba7a2d6fbd7d69a63 -e NODE_NAME=smithi018 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968:/var/run/ceph:z -v /var/log/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968:/var/log/ceph:z -v /var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmpxsmwkkuh:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:f4452def3be4d350f4a8e5feaf7ebb1511be9298fe68478ba7a2d6fbd7d69a63 inventory --format=json-pretty --filter-for-batch 2024-04-12T21:34:33.656 INFO:teuthology.orchestra.run.smithi018.stdout:Traceback (most recent call last): 2024-04-12T21:34:33.657 INFO:teuthology.orchestra.run.smithi018.stdout: File "/usr/lib64/python3.9/runpy.py", line 197, in _run_module_as_main 2024-04-12T21:34:33.657 INFO:teuthology.orchestra.run.smithi018.stdout: return _run_code(code, main_globals, None, 2024-04-12T21:34:33.657 INFO:teuthology.orchestra.run.smithi018.stdout: File "/usr/lib64/python3.9/runpy.py", line 87, in _run_code 2024-04-12T21:34:33.657 INFO:teuthology.orchestra.run.smithi018.stdout: exec(code, run_globals) 2024-04-12T21:34:33.657 INFO:teuthology.orchestra.run.smithi018.stdout: File "/var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/cephadm.e44f3ed5e34e4ef52eedef233d1190bff004470d8e8005127b5312bff357fa7f/__main__.py", line 5618, in 2024-04-12T21:34:33.657 INFO:teuthology.orchestra.run.smithi018.stdout: File "/var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/cephadm.e44f3ed5e34e4ef52eedef233d1190bff004470d8e8005127b5312bff357fa7f/__main__.py", line 5606, in main 2024-04-12T21:34:33.657 INFO:teuthology.orchestra.run.smithi018.stdout: File "/var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/cephadm.e44f3ed5e34e4ef52eedef233d1190bff004470d8e8005127b5312bff357fa7f/__main__.py", line 408, in _infer_config 2024-04-12T21:34:33.657 INFO:teuthology.orchestra.run.smithi018.stdout: File "/var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/cephadm.e44f3ed5e34e4ef52eedef233d1190bff004470d8e8005127b5312bff357fa7f/__main__.py", line 323, in _infer_fsid 2024-04-12T21:34:33.657 INFO:teuthology.orchestra.run.smithi018.stdout: File "/var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/cephadm.e44f3ed5e34e4ef52eedef233d1190bff004470d8e8005127b5312bff357fa7f/__main__.py", line 436, in _infer_image 2024-04-12T21:34:33.657 INFO:teuthology.orchestra.run.smithi018.stdout: File "/var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/cephadm.e44f3ed5e34e4ef52eedef233d1190bff004470d8e8005127b5312bff357fa7f/__main__.py", line 310, in _validate_fsid 2024-04-12T21:34:33.657 INFO:teuthology.orchestra.run.smithi018.stdout: File "/var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/cephadm.e44f3ed5e34e4ef52eedef233d1190bff004470d8e8005127b5312bff357fa7f/__main__.py", line 3359, in command_ceph_volume 2024-04-12T21:34:33.657 INFO:teuthology.orchestra.run.smithi018.stdout: File "/var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/cephadm.e44f3ed5e34e4ef52eedef233d1190bff004470d8e8005127b5312bff357fa7f/cephadmlib/call_wrappers.py", line 310, in call_throws 2024-04-12T21:34:33.657 INFO:teuthology.orchestra.run.smithi018.stdout:RuntimeError: Failed command: /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:f4452def3be4d350f4a8e5feaf7ebb1511be9298fe68478ba7a2d6fbd7d69a63 -e NODE_NAME=smithi018 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968:/var/run/ceph:z -v /var/log/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968:/var/log/ceph:z -v /var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmpxsmwkkuh:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:f4452def3be4d350f4a8e5feaf7ebb1511be9298fe68478ba7a2d6fbd7d69a63 inventory --format=json-pretty --filter-for-batch 2024-04-12T21:34:33.657 INFO:teuthology.orchestra.run.smithi018.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-04-12T21:34:33.658 INFO:teuthology.orchestra.run.smithi018.stdout: fs cephfs is degraded 2024-04-12T21:34:33.658 INFO:teuthology.orchestra.run.smithi018.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-04-12T21:34:33.658 INFO:teuthology.orchestra.run.smithi018.stdout: fs cephfs is offline because no MDS is active for it. 2024-04-12T21:34:33.658 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:34:33.658+0000 7f5f93200700 1 -- 172.21.15.18:0/3029028834 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f5f7806ec10 msgr2=0x7f5f780710c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:34:33.659 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:34:33.658+0000 7f5f93200700 1 --2- 172.21.15.18:0/3029028834 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f5f7806ec10 0x7f5f780710c0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f5f8400b3b0 tx=0x7f5f84009e30 comp rx=0 tx=0).stop 2024-04-12T21:34:33.659 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:34:33.658+0000 7f5f93200700 1 -- 172.21.15.18:0/3029028834 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5f94105800 msgr2=0x7f5f94116160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:34:33.659 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:34:33.658+0000 7f5f93200700 1 --2- 172.21.15.18:0/3029028834 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5f94105800 0x7f5f94116160 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7f5f8000c800 tx=0x7f5f80004470 comp rx=0 tx=0).stop 2024-04-12T21:34:33.659 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:34:33.658+0000 7f5f93200700 1 -- 172.21.15.18:0/3029028834 shutdown_connections 2024-04-12T21:34:33.659 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:34:33.658+0000 7f5f93200700 1 --2- 172.21.15.18:0/3029028834 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f5f7806ec10 0x7f5f780710c0 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:34:33.659 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:34:33.658+0000 7f5f93200700 1 --2- 172.21.15.18:0/3029028834 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5f94104e60 0x7f5f94115c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:34:33.659 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:34:33.658+0000 7f5f93200700 1 --2- 172.21.15.18:0/3029028834 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5f94105800 0x7f5f94116160 unknown :-1 s=CLOSED pgs=889 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:34:33.659 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:34:33.658+0000 7f5f93200700 1 -- 172.21.15.18:0/3029028834 >> 172.21.15.18:0/3029028834 conn(0x7f5f940fab10 msgr2=0x7f5f940fcb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:34:33.659 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:34:33.659+0000 7f5f93200700 1 -- 172.21.15.18:0/3029028834 shutdown_connections 2024-04-12T21:34:33.659 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:34:33.659+0000 7f5f93200700 1 -- 172.21.15.18:0/3029028834 wait complete. 2024-04-12T21:34:34.103 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:34:33 smithi018 ceph-mon[112590]: from='client.? 172.21.15.18:0/3029028834' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-12T21:34:34.282 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:34:33 smithi186 ceph-mon[86379]: from='client.? 172.21.15.18:0/3029028834' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-12T21:34:35.282 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:34:34 smithi186 ceph-mon[86379]: from='client.35840 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:34:35.282 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:34:34 smithi186 ceph-mon[86379]: pgmap v1044: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:34:35.353 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:34:34 smithi018 ceph-mon[112590]: from='client.35840 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:34:35.353 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:34:34 smithi018 ceph-mon[112590]: pgmap v1044: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:34:37.282 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:34:36 smithi186 ceph-mon[86379]: pgmap v1045: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:34:37.353 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:34:36 smithi018 ceph-mon[112590]: pgmap v1045: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:34:39.282 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:34:38 smithi186 ceph-mon[86379]: pgmap v1046: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:34:39.353 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:34:38 smithi018 ceph-mon[112590]: pgmap v1046: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:34:41.282 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:34:40 smithi186 ceph-mon[86379]: pgmap v1047: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:34:41.282 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:34:40 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T21:34:41.353 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:34:40 smithi018 ceph-mon[112590]: pgmap v1047: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:34:41.354 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:34:40 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T21:34:43.282 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:34:42 smithi186 ceph-mon[86379]: pgmap v1048: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:34:43.353 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:34:42 smithi018 ceph-mon[112590]: pgmap v1048: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:34:45.156 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:34:44 smithi018 ceph-mon[112590]: pgmap v1049: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:34:45.282 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:34:44 smithi186 ceph-mon[86379]: pgmap v1049: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:34:47.282 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:34:46 smithi186 ceph-mon[86379]: pgmap v1050: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:34:47.353 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:34:46 smithi018 ceph-mon[112590]: pgmap v1050: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:34:49.282 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:34:48 smithi186 ceph-mon[86379]: pgmap v1051: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:34:49.353 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:34:48 smithi018 ceph-mon[112590]: pgmap v1051: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:34:51.282 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:34:50 smithi186 ceph-mon[86379]: pgmap v1052: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:34:51.353 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:34:50 smithi018 ceph-mon[112590]: pgmap v1052: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:34:52.283 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:34:51 smithi186 ceph-mon[86379]: pgmap v1053: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:34:52.353 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:34:51 smithi018 ceph-mon[112590]: pgmap v1053: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:34:55.103 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:34:54 smithi018 ceph-mon[112590]: pgmap v1054: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:34:55.282 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:34:54 smithi186 ceph-mon[86379]: pgmap v1054: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:34:56.103 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:34:55 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T21:34:56.282 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:34:55 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T21:34:57.103 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:34:56 smithi018 ceph-mon[112590]: pgmap v1055: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:34:57.282 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:34:56 smithi186 ceph-mon[86379]: pgmap v1055: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:34:59.103 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:34:58 smithi018 ceph-mon[112590]: pgmap v1056: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:34:59.282 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:34:58 smithi186 ceph-mon[86379]: pgmap v1056: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:35:01.103 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:35:00 smithi018 ceph-mon[112590]: pgmap v1057: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:35:01.282 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:35:00 smithi186 ceph-mon[86379]: pgmap v1057: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:35:03.103 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:35:02 smithi018 ceph-mon[112590]: pgmap v1058: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:35:03.282 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:35:02 smithi186 ceph-mon[86379]: pgmap v1058: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:35:03.829 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:03.828+0000 7f7b2b200700 1 -- 172.21.15.18:0/2537690769 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7b24105090 msgr2=0x7f7b24105460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:35:03.829 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:03.828+0000 7f7b2b200700 1 --2- 172.21.15.18:0/2537690769 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7b24105090 0x7f7b24105460 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7f7b14004940 tx=0x7f7b14009e30 comp rx=0 tx=0).stop 2024-04-12T21:35:03.829 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:03.829+0000 7f7b2b200700 1 -- 172.21.15.18:0/2537690769 shutdown_connections 2024-04-12T21:35:03.829 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:03.829+0000 7f7b2b200700 1 --2- 172.21.15.18:0/2537690769 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7b24105a30 0x7f7b240ff0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:35:03.829 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:03.829+0000 7f7b2b200700 1 --2- 172.21.15.18:0/2537690769 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7b24105090 0x7f7b24105460 unknown :-1 s=CLOSED pgs=890 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:35:03.830 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:03.829+0000 7f7b2b200700 1 -- 172.21.15.18:0/2537690769 >> 172.21.15.18:0/2537690769 conn(0x7f7b240fac00 msgr2=0x7f7b240fd010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:35:03.830 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:03.829+0000 7f7b2b200700 1 -- 172.21.15.18:0/2537690769 shutdown_connections 2024-04-12T21:35:03.830 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:03.829+0000 7f7b2b200700 1 -- 172.21.15.18:0/2537690769 wait complete. 2024-04-12T21:35:03.831 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:03.830+0000 7f7b2b200700 1 Processor -- start 2024-04-12T21:35:03.831 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:03.830+0000 7f7b2b200700 1 -- start start 2024-04-12T21:35:03.831 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:03.830+0000 7f7b2b200700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7b24105090 0x7f7b24115b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:35:03.831 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:03.830+0000 7f7b2b200700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7b24105a30 0x7f7b24116090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:35:03.831 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:03.830+0000 7f7b2b200700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b24116770 con 0x7f7b24105090 2024-04-12T21:35:03.832 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:03.830+0000 7f7b2b200700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b2410fd10 con 0x7f7b24105a30 2024-04-12T21:35:03.832 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:03.831+0000 7f7b23e00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7b24105090 0x7f7b24115b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:35:03.832 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:03.831+0000 7f7b23e00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7b24105090 0x7f7b24115b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:49184/0 (socket says 172.21.15.18:49184) 2024-04-12T21:35:03.832 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:03.831+0000 7f7b23e00700 1 -- 172.21.15.18:0/450735032 learned_addr learned my addr 172.21.15.18:0/450735032 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:35:03.832 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:03.831+0000 7f7b23400700 1 --2- 172.21.15.18:0/450735032 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7b24105a30 0x7f7b24116090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:35:03.832 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:03.831+0000 7f7b23e00700 1 -- 172.21.15.18:0/450735032 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7b24105a30 msgr2=0x7f7b24116090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:35:03.832 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:03.831+0000 7f7b23e00700 1 --2- 172.21.15.18:0/450735032 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7b24105a30 0x7f7b24116090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:35:03.832 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:03.831+0000 7f7b23e00700 1 -- 172.21.15.18:0/450735032 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b14005040 con 0x7f7b24105090 2024-04-12T21:35:03.832 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:03.832+0000 7f7b23e00700 1 --2- 172.21.15.18:0/450735032 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7b24105090 0x7f7b24115b50 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7f7b14009180 tx=0x7f7b14009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:35:03.833 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:03.832+0000 7f7b1be00700 1 -- 172.21.15.18:0/450735032 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7b1400e040 con 0x7f7b24105090 2024-04-12T21:35:03.833 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:03.832+0000 7f7b1be00700 1 -- 172.21.15.18:0/450735032 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f7b1400bbc0 con 0x7f7b24105090 2024-04-12T21:35:03.833 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:03.832+0000 7f7b1be00700 1 -- 172.21.15.18:0/450735032 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7b1401c630 con 0x7f7b24105090 2024-04-12T21:35:03.833 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:03.832+0000 7f7b2b200700 1 -- 172.21.15.18:0/450735032 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7b2410ff30 con 0x7f7b24105090 2024-04-12T21:35:03.833 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:03.832+0000 7f7b2b200700 1 -- 172.21.15.18:0/450735032 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7b241104f0 con 0x7f7b24105090 2024-04-12T21:35:03.834 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:03.833+0000 7f7b2b200700 1 -- 172.21.15.18:0/450735032 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7b241023a0 con 0x7f7b24105090 2024-04-12T21:35:03.834 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:03.834+0000 7f7b1be00700 1 -- 172.21.15.18:0/450735032 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7f7b1400bd30 con 0x7f7b24105090 2024-04-12T21:35:03.835 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:03.835+0000 7f7b1be00700 1 --2- 172.21.15.18:0/450735032 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f7b0806ec00 0x7f7b080710b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:35:03.835 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:03.836+0000 7f7b1be00700 1 -- 172.21.15.18:0/450735032 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 6757+0+0 (secure 0 0 0) 0x7f7b1408fd70 con 0x7f7b24105090 2024-04-12T21:35:03.836 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:03.836+0000 7f7b23400700 1 --2- 172.21.15.18:0/450735032 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f7b0806ec00 0x7f7b080710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:35:03.836 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:03.836+0000 7f7b23400700 1 --2- 172.21.15.18:0/450735032 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f7b0806ec00 0x7f7b080710b0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f7b10006fd0 tx=0x7f7b10008040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:35:03.841 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:03.841+0000 7f7b1be00700 1 -- 172.21.15.18:0/450735032 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f7b1405c5a0 con 0x7f7b24105090 2024-04-12T21:35:04.093 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:04.093+0000 7f7b2b200700 1 -- 172.21.15.18:0/450735032 --> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f7b2410c270 con 0x7f7b0806ec00 2024-04-12T21:35:04.095 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:04.095+0000 7f7b1be00700 1 -- 172.21.15.18:0/450735032 <== mgr.34102 v2:172.21.15.18:6800/3812891919 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f7b2410c270 con 0x7f7b0806ec00 2024-04-12T21:35:04.098 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:04.098+0000 7f7b2b200700 1 -- 172.21.15.18:0/450735032 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f7b0806ec00 msgr2=0x7f7b080710b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:35:04.098 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:04.098+0000 7f7b2b200700 1 --2- 172.21.15.18:0/450735032 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f7b0806ec00 0x7f7b080710b0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f7b10006fd0 tx=0x7f7b10008040 comp rx=0 tx=0).stop 2024-04-12T21:35:04.098 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:04.098+0000 7f7b2b200700 1 -- 172.21.15.18:0/450735032 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7b24105090 msgr2=0x7f7b24115b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:35:04.098 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:04.098+0000 7f7b2b200700 1 --2- 172.21.15.18:0/450735032 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7b24105090 0x7f7b24115b50 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7f7b14009180 tx=0x7f7b14009e30 comp rx=0 tx=0).stop 2024-04-12T21:35:04.099 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:04.098+0000 7f7b2b200700 1 -- 172.21.15.18:0/450735032 shutdown_connections 2024-04-12T21:35:04.099 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:04.098+0000 7f7b2b200700 1 --2- 172.21.15.18:0/450735032 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f7b0806ec00 0x7f7b080710b0 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:35:04.099 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:04.098+0000 7f7b2b200700 1 --2- 172.21.15.18:0/450735032 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7b24105090 0x7f7b24115b50 unknown :-1 s=CLOSED pgs=891 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:35:04.099 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:04.098+0000 7f7b2b200700 1 --2- 172.21.15.18:0/450735032 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7b24105a30 0x7f7b24116090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:35:04.099 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:04.098+0000 7f7b2b200700 1 -- 172.21.15.18:0/450735032 >> 172.21.15.18:0/450735032 conn(0x7f7b240fac00 msgr2=0x7f7b240fc890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:35:04.099 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:04.099+0000 7f7b2b200700 1 -- 172.21.15.18:0/450735032 shutdown_connections 2024-04-12T21:35:04.099 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:04.099+0000 7f7b2b200700 1 -- 172.21.15.18:0/450735032 wait complete. 2024-04-12T21:35:04.116 INFO:teuthology.orchestra.run.smithi018.stdout:true 2024-04-12T21:35:04.258 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:04.258+0000 7f0240800700 1 -- 172.21.15.18:0/947876790 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0238102e00 msgr2=0x7f02381031d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:35:04.259 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:04.258+0000 7f0240800700 1 --2- 172.21.15.18:0/947876790 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0238102e00 0x7f02381031d0 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7f022c004940 tx=0x7f022c009e30 comp rx=0 tx=0).stop 2024-04-12T21:35:04.259 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:04.258+0000 7f0240800700 1 -- 172.21.15.18:0/947876790 shutdown_connections 2024-04-12T21:35:04.259 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:04.258+0000 7f0240800700 1 --2- 172.21.15.18:0/947876790 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f02381037a0 0x7f02380fce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:35:04.259 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:04.258+0000 7f0240800700 1 --2- 172.21.15.18:0/947876790 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0238102e00 0x7f02381031d0 unknown :-1 s=CLOSED pgs=892 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:35:04.259 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:04.258+0000 7f0240800700 1 -- 172.21.15.18:0/947876790 >> 172.21.15.18:0/947876790 conn(0x7f02380f89b0 msgr2=0x7f02380fadc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:35:04.259 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:04.258+0000 7f0240800700 1 -- 172.21.15.18:0/947876790 shutdown_connections 2024-04-12T21:35:04.259 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:04.258+0000 7f0240800700 1 -- 172.21.15.18:0/947876790 wait complete. 2024-04-12T21:35:04.260 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:04.259+0000 7f0240800700 1 Processor -- start 2024-04-12T21:35:04.260 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:04.260+0000 7f0240800700 1 -- start start 2024-04-12T21:35:04.260 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:04.260+0000 7f0240800700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0238102e00 0x7f0238115b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:35:04.260 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:04.260+0000 7f0240800700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f02381037a0 0x7f02381160d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:35:04.261 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:04.260+0000 7f0240800700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0238116720 con 0x7f0238102e00 2024-04-12T21:35:04.261 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:04.260+0000 7f0240800700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f023810fd00 con 0x7f02381037a0 2024-04-12T21:35:04.261 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:04.260+0000 7f023de00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0238102e00 0x7f0238115b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:35:04.261 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:04.260+0000 7f023de00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0238102e00 0x7f0238115b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:49208/0 (socket says 172.21.15.18:49208) 2024-04-12T21:35:04.261 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:04.260+0000 7f023de00700 1 -- 172.21.15.18:0/765950608 learned_addr learned my addr 172.21.15.18:0/765950608 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:35:04.262 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:04.261+0000 7f023d400700 1 --2- 172.21.15.18:0/765950608 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f02381037a0 0x7f02381160d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:35:04.262 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:04.261+0000 7f023de00700 1 -- 172.21.15.18:0/765950608 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f02381037a0 msgr2=0x7f02381160d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:35:04.262 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:04.261+0000 7f023de00700 1 --2- 172.21.15.18:0/765950608 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f02381037a0 0x7f02381160d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:35:04.262 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:04.261+0000 7f023de00700 1 -- 172.21.15.18:0/765950608 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f022c005040 con 0x7f0238102e00 2024-04-12T21:35:04.262 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:04.261+0000 7f023d400700 1 --2- 172.21.15.18:0/765950608 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f02381037a0 0x7f02381160d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-12T21:35:04.262 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:04.261+0000 7f023de00700 1 --2- 172.21.15.18:0/765950608 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0238102e00 0x7f0238115b90 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7f022c009180 tx=0x7f022c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:35:04.262 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:04.261+0000 7f0232000700 1 -- 172.21.15.18:0/765950608 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f022c00e040 con 0x7f0238102e00 2024-04-12T21:35:04.263 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:04.262+0000 7f0232000700 1 -- 172.21.15.18:0/765950608 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f022c00bbc0 con 0x7f0238102e00 2024-04-12T21:35:04.263 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:04.262+0000 7f0240800700 1 -- 172.21.15.18:0/765950608 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f023810ff80 con 0x7f0238102e00 2024-04-12T21:35:04.263 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:04.262+0000 7f0232000700 1 -- 172.21.15.18:0/765950608 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f022c01c630 con 0x7f0238102e00 2024-04-12T21:35:04.263 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:04.262+0000 7f0240800700 1 -- 172.21.15.18:0/765950608 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0238110540 con 0x7f0238102e00 2024-04-12T21:35:04.263 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:04.263+0000 7f0240800700 1 -- 172.21.15.18:0/765950608 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0238107300 con 0x7f0238102e00 2024-04-12T21:35:04.264 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:04.264+0000 7f0232000700 1 -- 172.21.15.18:0/765950608 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7f022c00bd30 con 0x7f0238102e00 2024-04-12T21:35:04.265 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:04.265+0000 7f0232000700 1 --2- 172.21.15.18:0/765950608 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f022006ec00 0x7f02200710b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:35:04.265 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:04.265+0000 7f0232000700 1 -- 172.21.15.18:0/765950608 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 6757+0+0 (secure 0 0 0) 0x7f022c08fd40 con 0x7f0238102e00 2024-04-12T21:35:04.265 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:04.265+0000 7f023d400700 1 --2- 172.21.15.18:0/765950608 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f022006ec00 0x7f02200710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:35:04.266 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:04.266+0000 7f023d400700 1 --2- 172.21.15.18:0/765950608 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f022006ec00 0x7f02200710b0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f0228009fd0 tx=0x7f022800b040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:35:04.270 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:04.270+0000 7f0232000700 1 -- 172.21.15.18:0/765950608 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f022c05c570 con 0x7f0238102e00 2024-04-12T21:35:04.513 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:04.513+0000 7f0240800700 1 -- 172.21.15.18:0/765950608 --> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f02381111e0 con 0x7f022006ec00 2024-04-12T21:35:04.518 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:04.516+0000 7f0232000700 1 -- 172.21.15.18:0/765950608 <== mgr.34102 v2:172.21.15.18:6800/3812891919 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f02381111e0 con 0x7f022006ec00 2024-04-12T21:35:04.519 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:04.519+0000 7f0240800700 1 -- 172.21.15.18:0/765950608 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f022006ec00 msgr2=0x7f02200710b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:35:04.519 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:04.519+0000 7f0240800700 1 --2- 172.21.15.18:0/765950608 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f022006ec00 0x7f02200710b0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f0228009fd0 tx=0x7f022800b040 comp rx=0 tx=0).stop 2024-04-12T21:35:04.519 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:04.519+0000 7f0240800700 1 -- 172.21.15.18:0/765950608 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0238102e00 msgr2=0x7f0238115b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:35:04.519 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:04.519+0000 7f0240800700 1 --2- 172.21.15.18:0/765950608 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0238102e00 0x7f0238115b90 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7f022c009180 tx=0x7f022c009e30 comp rx=0 tx=0).stop 2024-04-12T21:35:04.520 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:04.519+0000 7f0240800700 1 -- 172.21.15.18:0/765950608 shutdown_connections 2024-04-12T21:35:04.520 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:04.519+0000 7f0240800700 1 --2- 172.21.15.18:0/765950608 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f022006ec00 0x7f02200710b0 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:35:04.520 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:04.519+0000 7f0240800700 1 --2- 172.21.15.18:0/765950608 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0238102e00 0x7f0238115b90 unknown :-1 s=CLOSED pgs=893 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:35:04.520 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:04.519+0000 7f0240800700 1 --2- 172.21.15.18:0/765950608 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f02381037a0 0x7f02381160d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:35:04.520 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:04.519+0000 7f0240800700 1 -- 172.21.15.18:0/765950608 >> 172.21.15.18:0/765950608 conn(0x7f02380f89b0 msgr2=0x7f02380fa5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:35:04.521 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:04.519+0000 7f0240800700 1 -- 172.21.15.18:0/765950608 shutdown_connections 2024-04-12T21:35:04.521 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:04.519+0000 7f0240800700 1 -- 172.21.15.18:0/765950608 wait complete. 2024-04-12T21:35:04.679 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:04.678+0000 7fe7ed800700 1 -- 172.21.15.18:0/906601680 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe7e8109dd0 msgr2=0x7fe7e8103440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:35:04.679 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:04.678+0000 7fe7ed800700 1 --2- 172.21.15.18:0/906601680 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe7e8109dd0 0x7fe7e8103440 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7fe7d4004940 tx=0x7fe7d4009e30 comp rx=0 tx=0).stop 2024-04-12T21:35:04.679 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:04.678+0000 7fe7ed800700 1 -- 172.21.15.18:0/906601680 shutdown_connections 2024-04-12T21:35:04.679 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:04.678+0000 7fe7ed800700 1 --2- 172.21.15.18:0/906601680 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe7e8109dd0 0x7fe7e8103440 unknown :-1 s=CLOSED pgs=894 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:35:04.679 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:04.678+0000 7fe7ed800700 1 --2- 172.21.15.18:0/906601680 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe7e8109430 0x7fe7e8109800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:35:04.679 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:04.678+0000 7fe7ed800700 1 -- 172.21.15.18:0/906601680 >> 172.21.15.18:0/906601680 conn(0x7fe7e8072c00 msgr2=0x7fe7e8075010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:35:04.679 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:04.678+0000 7fe7ed800700 1 -- 172.21.15.18:0/906601680 shutdown_connections 2024-04-12T21:35:04.679 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:04.679+0000 7fe7ed800700 1 -- 172.21.15.18:0/906601680 wait complete. 2024-04-12T21:35:04.681 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:04.679+0000 7fe7ed800700 1 Processor -- start 2024-04-12T21:35:04.681 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:04.680+0000 7fe7ed800700 1 -- start start 2024-04-12T21:35:04.681 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:04.680+0000 7fe7ed800700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe7e8109430 0x7fe7e8119fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:35:04.682 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:04.680+0000 7fe7ed800700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe7e8109dd0 0x7fe7e811a4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:35:04.682 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:04.680+0000 7fe7ed800700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7e811abc0 con 0x7fe7e8109dd0 2024-04-12T21:35:04.682 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:04.680+0000 7fe7ed800700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7e8114110 con 0x7fe7e8109430 2024-04-12T21:35:04.682 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:04.680+0000 7fe7e6000700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe7e8109dd0 0x7fe7e811a4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:35:04.682 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:04.680+0000 7fe7e6000700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe7e8109dd0 0x7fe7e811a4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:49238/0 (socket says 172.21.15.18:49238) 2024-04-12T21:35:04.682 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:04.680+0000 7fe7e6000700 1 -- 172.21.15.18:0/16638217 learned_addr learned my addr 172.21.15.18:0/16638217 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:35:04.682 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:04.681+0000 7fe7e6a00700 1 --2- 172.21.15.18:0/16638217 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe7e8109430 0x7fe7e8119fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:35:04.682 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:04.681+0000 7fe7e6000700 1 -- 172.21.15.18:0/16638217 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe7e8109430 msgr2=0x7fe7e8119fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:35:04.682 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:04.681+0000 7fe7e6000700 1 --2- 172.21.15.18:0/16638217 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe7e8109430 0x7fe7e8119fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:35:04.682 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:04.681+0000 7fe7e6000700 1 -- 172.21.15.18:0/16638217 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7d4005040 con 0x7fe7e8109dd0 2024-04-12T21:35:04.682 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:04.681+0000 7fe7e6000700 1 --2- 172.21.15.18:0/16638217 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe7e8109dd0 0x7fe7e811a4e0 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7fe7d4009100 tx=0x7fe7d4009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:35:04.682 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:04.681+0000 7fe7dea00700 1 -- 172.21.15.18:0/16638217 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe7d400e040 con 0x7fe7e8109dd0 2024-04-12T21:35:04.683 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:04.681+0000 7fe7dea00700 1 -- 172.21.15.18:0/16638217 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe7d4004290 con 0x7fe7e8109dd0 2024-04-12T21:35:04.683 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:04.681+0000 7fe7dea00700 1 -- 172.21.15.18:0/16638217 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe7d401c3f0 con 0x7fe7e8109dd0 2024-04-12T21:35:04.683 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:04.681+0000 7fe7ed800700 1 -- 172.21.15.18:0/16638217 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe7e8114330 con 0x7fe7e8109dd0 2024-04-12T21:35:04.683 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:04.682+0000 7fe7ed800700 1 -- 172.21.15.18:0/16638217 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe7e81148f0 con 0x7fe7e8109dd0 2024-04-12T21:35:04.683 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:04.683+0000 7fe7ed800700 1 -- 172.21.15.18:0/16638217 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe7e8104b80 con 0x7fe7e8109dd0 2024-04-12T21:35:04.684 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:04.684+0000 7fe7dea00700 1 -- 172.21.15.18:0/16638217 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7fe7d4013790 con 0x7fe7e8109dd0 2024-04-12T21:35:04.685 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:04.685+0000 7fe7dea00700 1 --2- 172.21.15.18:0/16638217 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fe7cc06ec00 0x7fe7cc0710b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:35:04.685 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:04.685+0000 7fe7dea00700 1 -- 172.21.15.18:0/16638217 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 6757+0+0 (secure 0 0 0) 0x7fe7d408fc30 con 0x7fe7e8109dd0 2024-04-12T21:35:04.685 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:04.685+0000 7fe7e6a00700 1 --2- 172.21.15.18:0/16638217 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fe7cc06ec00 0x7fe7cc0710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:35:04.686 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:04.686+0000 7fe7e6a00700 1 --2- 172.21.15.18:0/16638217 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fe7cc06ec00 0x7fe7cc0710b0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7fe7d8006fd0 tx=0x7fe7d8008040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:35:04.690 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:04.690+0000 7fe7dea00700 1 -- 172.21.15.18:0/16638217 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7fe7d405c510 con 0x7fe7e8109dd0 2024-04-12T21:35:04.936 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:04.936+0000 7fe7ed800700 1 -- 172.21.15.18:0/16638217 --> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fe7e8114c20 con 0x7fe7cc06ec00 2024-04-12T21:35:04.952 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:04.952+0000 7fe7dea00700 1 -- 172.21.15.18:0/16638217 <== mgr.34102 v2:172.21.15.18:6800/3812891919 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7fe7e8114c20 con 0x7fe7cc06ec00 2024-04-12T21:35:04.952 INFO:teuthology.orchestra.run.smithi018.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-12T21:35:04.952 INFO:teuthology.orchestra.run.smithi018.stdout:alertmanager.smithi018 smithi018 *:9093,9094 running (35m) 11m ago 50m 15.5M - 0.25.0 c8568f914cd2 c332090068e8 2024-04-12T21:35:04.952 INFO:teuthology.orchestra.run.smithi018.stdout:crash.smithi018 smithi018 running (32m) 11m ago 50m 6665k - 19.0.0-2879-g324baf8d 8717720dc5e5 b7f80d1f7982 2024-04-12T21:35:04.952 INFO:teuthology.orchestra.run.smithi018.stdout:crash.smithi186 smithi186 running (31m) 12m ago 49m 6660k - 19.0.0-2879-g324baf8d 8717720dc5e5 a4f55c81b623 2024-04-12T21:35:04.952 INFO:teuthology.orchestra.run.smithi018.stdout:grafana.smithi018 smithi018 *:3000 running (35m) 11m ago 49m 74.8M - 9.4.12 2bacad6d85d8 b08e3bf176dd 2024-04-12T21:35:04.952 INFO:teuthology.orchestra.run.smithi018.stdout:mds.cephfs.smithi018.acqbsc smithi018 running (27m) 11m ago 43m 16.7M - 19.0.0-2879-g324baf8d 8717720dc5e5 021f99ab24ae 2024-04-12T21:35:04.952 INFO:teuthology.orchestra.run.smithi018.stdout:mds.cephfs.smithi018.szjluo smithi018 running (27m) 11m ago 42m 16.5M - 19.0.0-2879-g324baf8d 8717720dc5e5 0003117479d4 2024-04-12T21:35:04.952 INFO:teuthology.orchestra.run.smithi018.stdout:mds.cephfs.smithi186.jdllyn smithi186 running (12m) 12m ago 42m 11.0M - 19.0.0-2879-g324baf8d 8717720dc5e5 21b1f46eef23 2024-04-12T21:35:04.953 INFO:teuthology.orchestra.run.smithi018.stdout:mds.cephfs.smithi186.wjnafs smithi186 running (42m) 12m ago 42m 19.5M - 17.2.7-827-gd3b5060a 21de9b7369ad 74b115d6d6fd 2024-04-12T21:35:04.953 INFO:teuthology.orchestra.run.smithi018.stdout:mgr.smithi018.xjzwga smithi018 *:8443,9283,8765 running (38m) 11m ago 51m 512M - 19.0.0-2879-g324baf8d 8717720dc5e5 56088305eab7 2024-04-12T21:35:04.953 INFO:teuthology.orchestra.run.smithi018.stdout:mgr.smithi186.kqcfxk smithi186 *:8443,9283,8765 running (37m) 12m ago 48m 459M - 19.0.0-2879-g324baf8d 8717720dc5e5 09cd766e5e0d 2024-04-12T21:35:04.953 INFO:teuthology.orchestra.run.smithi018.stdout:mon.smithi018 smithi018 running (33m) 11m ago 52m 73.1M 2048M 19.0.0-2879-g324baf8d 8717720dc5e5 df3e201dfed6 2024-04-12T21:35:04.953 INFO:teuthology.orchestra.run.smithi018.stdout:mon.smithi186 smithi186 running (32m) 12m ago 48m 47.2M 2048M 19.0.0-2879-g324baf8d 8717720dc5e5 c77498da9b2b 2024-04-12T21:35:04.953 INFO:teuthology.orchestra.run.smithi018.stdout:node-exporter.smithi018 smithi018 *:9100 running (36m) 11m ago 50m 8108k - 1.5.0 0da6a335fe13 326c3d9c6847 2024-04-12T21:35:04.953 INFO:teuthology.orchestra.run.smithi018.stdout:node-exporter.smithi186 smithi186 *:9100 running (36m) 12m ago 48m 8757k - 1.5.0 0da6a335fe13 ec2112baeecf 2024-04-12T21:35:04.953 INFO:teuthology.orchestra.run.smithi018.stdout:osd.0 smithi018 running (31m) 11m ago 47m 123M 1559M 19.0.0-2879-g324baf8d 8717720dc5e5 60b1273ca047 2024-04-12T21:35:04.953 INFO:teuthology.orchestra.run.smithi018.stdout:osd.1 smithi018 running (30m) 11m ago 47m 95.1M 1559M 19.0.0-2879-g324baf8d 8717720dc5e5 170ce1e5fb8d 2024-04-12T21:35:04.953 INFO:teuthology.orchestra.run.smithi018.stdout:osd.2 smithi018 running (30m) 11m ago 46m 94.3M 1559M 19.0.0-2879-g324baf8d 8717720dc5e5 c2cbc6aa2839 2024-04-12T21:35:04.953 INFO:teuthology.orchestra.run.smithi018.stdout:osd.3 smithi186 running (29m) 12m ago 46m 101M 2581M 19.0.0-2879-g324baf8d 8717720dc5e5 0138e8262e4b 2024-04-12T21:35:04.953 INFO:teuthology.orchestra.run.smithi018.stdout:osd.4 smithi186 running (29m) 12m ago 45m 91.0M 2581M 19.0.0-2879-g324baf8d 8717720dc5e5 b20cb63b25c9 2024-04-12T21:35:04.953 INFO:teuthology.orchestra.run.smithi018.stdout:osd.5 smithi186 running (28m) 12m ago 44m 92.0M 2581M 19.0.0-2879-g324baf8d 8717720dc5e5 2513bea2610e 2024-04-12T21:35:04.953 INFO:teuthology.orchestra.run.smithi018.stdout:prometheus.smithi018 smithi018 *:9095 running (36m) 11m ago 49m 67.9M - 2.43.0 a07b618ecd1d f58841127e21 2024-04-12T21:35:04.955 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:04.954+0000 7fe7ed800700 1 -- 172.21.15.18:0/16638217 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fe7cc06ec00 msgr2=0x7fe7cc0710b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:35:04.955 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:04.954+0000 7fe7ed800700 1 --2- 172.21.15.18:0/16638217 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fe7cc06ec00 0x7fe7cc0710b0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7fe7d8006fd0 tx=0x7fe7d8008040 comp rx=0 tx=0).stop 2024-04-12T21:35:04.955 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:04.954+0000 7fe7ed800700 1 -- 172.21.15.18:0/16638217 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe7e8109dd0 msgr2=0x7fe7e811a4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:35:04.955 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:04.954+0000 7fe7ed800700 1 --2- 172.21.15.18:0/16638217 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe7e8109dd0 0x7fe7e811a4e0 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7fe7d4009100 tx=0x7fe7d4009e30 comp rx=0 tx=0).stop 2024-04-12T21:35:04.955 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:04.955+0000 7fe7ed800700 1 -- 172.21.15.18:0/16638217 shutdown_connections 2024-04-12T21:35:04.955 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:04.955+0000 7fe7ed800700 1 --2- 172.21.15.18:0/16638217 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fe7cc06ec00 0x7fe7cc0710b0 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:35:04.956 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:04.955+0000 7fe7ed800700 1 --2- 172.21.15.18:0/16638217 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe7e8109430 0x7fe7e8119fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:35:04.956 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:04.955+0000 7fe7ed800700 1 --2- 172.21.15.18:0/16638217 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe7e8109dd0 0x7fe7e811a4e0 unknown :-1 s=CLOSED pgs=895 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:35:04.956 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:04.955+0000 7fe7ed800700 1 -- 172.21.15.18:0/16638217 >> 172.21.15.18:0/16638217 conn(0x7fe7e8072c00 msgr2=0x7fe7e8074b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:35:04.956 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:04.955+0000 7fe7ed800700 1 -- 172.21.15.18:0/16638217 shutdown_connections 2024-04-12T21:35:04.956 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:04.955+0000 7fe7ed800700 1 -- 172.21.15.18:0/16638217 wait complete. 2024-04-12T21:35:05.103 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:35:04 smithi018 ceph-mon[112590]: pgmap v1059: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:35:05.115 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:05.115+0000 7f0d95000700 1 -- 172.21.15.18:0/4161196306 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0d90070360 msgr2=0x7f0d90070730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:35:05.116 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:05.115+0000 7f0d95000700 1 --2- 172.21.15.18:0/4161196306 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0d90070360 0x7f0d90070730 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7f0d80004970 tx=0x7f0d80009e30 comp rx=0 tx=0).stop 2024-04-12T21:35:05.116 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:05.116+0000 7f0d95000700 1 -- 172.21.15.18:0/4161196306 shutdown_connections 2024-04-12T21:35:05.116 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:05.116+0000 7f0d95000700 1 --2- 172.21.15.18:0/4161196306 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0d90070c70 0x7f0d90108f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:35:05.116 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:05.116+0000 7f0d95000700 1 --2- 172.21.15.18:0/4161196306 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0d90070360 0x7f0d90070730 unknown :-1 s=CLOSED pgs=896 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:35:05.116 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:05.116+0000 7f0d95000700 1 -- 172.21.15.18:0/4161196306 >> 172.21.15.18:0/4161196306 conn(0x7f0d900f87e0 msgr2=0x7f0d900fabf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:35:05.116 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:05.116+0000 7f0d95000700 1 -- 172.21.15.18:0/4161196306 shutdown_connections 2024-04-12T21:35:05.116 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:05.116+0000 7f0d95000700 1 -- 172.21.15.18:0/4161196306 wait complete. 2024-04-12T21:35:05.117 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:05.116+0000 7f0d95000700 1 Processor -- start 2024-04-12T21:35:05.117 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:05.117+0000 7f0d95000700 1 -- start start 2024-04-12T21:35:05.117 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:05.117+0000 7f0d95000700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0d90070360 0x7f0d90115b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:35:05.117 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:05.117+0000 7f0d95000700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0d90070c70 0x7f0d901160b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:35:05.117 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:05.117+0000 7f0d95000700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d90116700 con 0x7f0d90070360 2024-04-12T21:35:05.117 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:05.117+0000 7f0d95000700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d9010faf0 con 0x7f0d90070c70 2024-04-12T21:35:05.118 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:05.117+0000 7f0d8f400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0d90070360 0x7f0d90115b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:35:05.118 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:05.117+0000 7f0d8f400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0d90070360 0x7f0d90115b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:49258/0 (socket says 172.21.15.18:49258) 2024-04-12T21:35:05.118 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:05.117+0000 7f0d8f400700 1 -- 172.21.15.18:0/3102358418 learned_addr learned my addr 172.21.15.18:0/3102358418 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:35:05.119 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:05.118+0000 7f0d8ea00700 1 --2- 172.21.15.18:0/3102358418 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0d90070c70 0x7f0d901160b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:35:05.119 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:05.118+0000 7f0d8f400700 1 -- 172.21.15.18:0/3102358418 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0d90070c70 msgr2=0x7f0d901160b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:35:05.119 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:05.118+0000 7f0d8f400700 1 --2- 172.21.15.18:0/3102358418 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0d90070c70 0x7f0d901160b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:35:05.119 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:05.118+0000 7f0d8f400700 1 -- 172.21.15.18:0/3102358418 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d80005040 con 0x7f0d90070360 2024-04-12T21:35:05.119 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:05.118+0000 7f0d8f400700 1 --2- 172.21.15.18:0/3102358418 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0d90070360 0x7f0d90115b70 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7f0d80000c00 tx=0x7f0d80009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:35:05.119 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:05.118+0000 7f0d87400700 1 -- 172.21.15.18:0/3102358418 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0d8000e040 con 0x7f0d90070360 2024-04-12T21:35:05.119 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:05.118+0000 7f0d87400700 1 -- 172.21.15.18:0/3102358418 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f0d8000bc10 con 0x7f0d90070360 2024-04-12T21:35:05.119 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:05.118+0000 7f0d95000700 1 -- 172.21.15.18:0/3102358418 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0d9010fd70 con 0x7f0d90070360 2024-04-12T21:35:05.119 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:05.118+0000 7f0d87400700 1 -- 172.21.15.18:0/3102358418 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0d8001c630 con 0x7f0d90070360 2024-04-12T21:35:05.120 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:05.119+0000 7f0d95000700 1 -- 172.21.15.18:0/3102358418 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0d901102b0 con 0x7f0d90070360 2024-04-12T21:35:05.120 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:05.120+0000 7f0d95000700 1 -- 172.21.15.18:0/3102358418 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0d90071aa0 con 0x7f0d90070360 2024-04-12T21:35:05.121 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:05.121+0000 7f0d87400700 1 -- 172.21.15.18:0/3102358418 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7f0d8000bd80 con 0x7f0d90070360 2024-04-12T21:35:05.122 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:05.122+0000 7f0d87400700 1 --2- 172.21.15.18:0/3102358418 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f0d7406ec00 0x7f0d740710b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:35:05.122 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:05.123+0000 7f0d87400700 1 -- 172.21.15.18:0/3102358418 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 6757+0+0 (secure 0 0 0) 0x7f0d8008fbe0 con 0x7f0d90070360 2024-04-12T21:35:05.123 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:05.123+0000 7f0d8ea00700 1 --2- 172.21.15.18:0/3102358418 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f0d7406ec00 0x7f0d740710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:35:05.124 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:05.124+0000 7f0d8ea00700 1 --2- 172.21.15.18:0/3102358418 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f0d7406ec00 0x7f0d740710b0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f0d90110cb0 tx=0x7f0d7c00b040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:35:05.130 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:05.128+0000 7f0d87400700 1 -- 172.21.15.18:0/3102358418 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f0d8005c410 con 0x7f0d90070360 2024-04-12T21:35:05.282 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:35:04 smithi186 ceph-mon[86379]: pgmap v1059: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:35:05.464 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:05.463+0000 7f0d95000700 1 -- 172.21.15.18:0/3102358418 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f0d9010bf40 con 0x7f0d90070360 2024-04-12T21:35:05.464 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:05.464+0000 7f0d87400700 1 -- 172.21.15.18:0/3102358418 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+837 (secure 0 0 0) 0x7f0d80004de0 con 0x7f0d90070360 2024-04-12T21:35:05.465 INFO:teuthology.orchestra.run.smithi018.stdout:{ 2024-04-12T21:35:05.465 INFO:teuthology.orchestra.run.smithi018.stdout: "mon": { 2024-04-12T21:35:05.465 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 19.0.0-2879-g324baf8d (324baf8ddb91dea46cac50cb535c306115791355) squid (dev)": 2 2024-04-12T21:35:05.465 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-12T21:35:05.466 INFO:teuthology.orchestra.run.smithi018.stdout: "mgr": { 2024-04-12T21:35:05.466 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 19.0.0-2879-g324baf8d (324baf8ddb91dea46cac50cb535c306115791355) squid (dev)": 2 2024-04-12T21:35:05.466 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-12T21:35:05.466 INFO:teuthology.orchestra.run.smithi018.stdout: "osd": { 2024-04-12T21:35:05.466 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 19.0.0-2879-g324baf8d (324baf8ddb91dea46cac50cb535c306115791355) squid (dev)": 6 2024-04-12T21:35:05.466 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-12T21:35:05.466 INFO:teuthology.orchestra.run.smithi018.stdout: "mds": { 2024-04-12T21:35:05.466 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 17.2.7-827-gd3b5060a (d3b5060a17449edfae9385cdf792afec109fca5d) quincy (stable)": 1, 2024-04-12T21:35:05.467 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 19.0.0-2879-g324baf8d (324baf8ddb91dea46cac50cb535c306115791355) squid (dev)": 3 2024-04-12T21:35:05.467 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-12T21:35:05.467 INFO:teuthology.orchestra.run.smithi018.stdout: "overall": { 2024-04-12T21:35:05.467 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 17.2.7-827-gd3b5060a (d3b5060a17449edfae9385cdf792afec109fca5d) quincy (stable)": 1, 2024-04-12T21:35:05.467 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 19.0.0-2879-g324baf8d (324baf8ddb91dea46cac50cb535c306115791355) squid (dev)": 13 2024-04-12T21:35:05.467 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-04-12T21:35:05.467 INFO:teuthology.orchestra.run.smithi018.stdout:} 2024-04-12T21:35:05.468 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:05.467+0000 7f0d95000700 1 -- 172.21.15.18:0/3102358418 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f0d7406ec00 msgr2=0x7f0d740710b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:35:05.469 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:05.467+0000 7f0d95000700 1 --2- 172.21.15.18:0/3102358418 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f0d7406ec00 0x7f0d740710b0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f0d90110cb0 tx=0x7f0d7c00b040 comp rx=0 tx=0).stop 2024-04-12T21:35:05.469 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:05.467+0000 7f0d95000700 1 -- 172.21.15.18:0/3102358418 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0d90070360 msgr2=0x7f0d90115b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:35:05.469 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:05.467+0000 7f0d95000700 1 --2- 172.21.15.18:0/3102358418 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0d90070360 0x7f0d90115b70 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7f0d80000c00 tx=0x7f0d80009e30 comp rx=0 tx=0).stop 2024-04-12T21:35:05.469 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:05.467+0000 7f0d95000700 1 -- 172.21.15.18:0/3102358418 shutdown_connections 2024-04-12T21:35:05.469 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:05.467+0000 7f0d95000700 1 --2- 172.21.15.18:0/3102358418 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f0d7406ec00 0x7f0d740710b0 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:35:05.469 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:05.467+0000 7f0d95000700 1 --2- 172.21.15.18:0/3102358418 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0d90070360 0x7f0d90115b70 unknown :-1 s=CLOSED pgs=897 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:35:05.469 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:05.468+0000 7f0d95000700 1 --2- 172.21.15.18:0/3102358418 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0d90070c70 0x7f0d901160b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:35:05.470 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:05.468+0000 7f0d95000700 1 -- 172.21.15.18:0/3102358418 >> 172.21.15.18:0/3102358418 conn(0x7f0d900f87e0 msgr2=0x7f0d90105810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:35:05.470 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:05.468+0000 7f0d95000700 1 -- 172.21.15.18:0/3102358418 shutdown_connections 2024-04-12T21:35:05.470 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:05.468+0000 7f0d95000700 1 -- 172.21.15.18:0/3102358418 wait complete. 2024-04-12T21:35:05.628 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:05.627+0000 7f11ccc00700 1 -- 172.21.15.18:0/1282906645 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f11c8101ba0 msgr2=0x7f11c8101f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:35:05.628 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:05.627+0000 7f11ccc00700 1 --2- 172.21.15.18:0/1282906645 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f11c8101ba0 0x7f11c8101f70 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7f11b8004940 tx=0x7f11b8009e30 comp rx=0 tx=0).stop 2024-04-12T21:35:05.628 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:05.628+0000 7f11ccc00700 1 -- 172.21.15.18:0/1282906645 shutdown_connections 2024-04-12T21:35:05.628 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:05.628+0000 7f11ccc00700 1 --2- 172.21.15.18:0/1282906645 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f11c80fbb70 0x7f11c80fbfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:35:05.628 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:05.628+0000 7f11ccc00700 1 --2- 172.21.15.18:0/1282906645 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f11c8101ba0 0x7f11c8101f70 unknown :-1 s=CLOSED pgs=898 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:35:05.628 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:05.628+0000 7f11ccc00700 1 -- 172.21.15.18:0/1282906645 >> 172.21.15.18:0/1282906645 conn(0x7f11c8071b70 msgr2=0x7f11c8071f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:35:05.629 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:05.628+0000 7f11ccc00700 1 -- 172.21.15.18:0/1282906645 shutdown_connections 2024-04-12T21:35:05.629 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:05.628+0000 7f11ccc00700 1 -- 172.21.15.18:0/1282906645 wait complete. 2024-04-12T21:35:05.629 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:05.629+0000 7f11ccc00700 1 Processor -- start 2024-04-12T21:35:05.630 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:05.629+0000 7f11ccc00700 1 -- start start 2024-04-12T21:35:05.630 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:05.629+0000 7f11ccc00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f11c80fbb70 0x7f11c8102950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:35:05.630 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:05.629+0000 7f11ccc00700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f11c8101ba0 0x7f11c8102e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:35:05.630 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:05.629+0000 7f11ccc00700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11c8103420 con 0x7f11c80fbb70 2024-04-12T21:35:05.630 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:05.629+0000 7f11ccc00700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11c8103560 con 0x7f11c8101ba0 2024-04-12T21:35:05.630 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:05.630+0000 7f11c7400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f11c80fbb70 0x7f11c8102950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:35:05.630 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:05.630+0000 7f11c7400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f11c80fbb70 0x7f11c8102950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:49290/0 (socket says 172.21.15.18:49290) 2024-04-12T21:35:05.630 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:05.630+0000 7f11c7400700 1 -- 172.21.15.18:0/3381337886 learned_addr learned my addr 172.21.15.18:0/3381337886 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:35:05.631 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:05.630+0000 7f11bea00700 1 --2- 172.21.15.18:0/3381337886 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f11c8101ba0 0x7f11c8102e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:35:05.631 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:05.630+0000 7f11c7400700 1 -- 172.21.15.18:0/3381337886 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f11c8101ba0 msgr2=0x7f11c8102e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:35:05.631 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:05.630+0000 7f11c7400700 1 --2- 172.21.15.18:0/3381337886 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f11c8101ba0 0x7f11c8102e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:35:05.631 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:05.630+0000 7f11c7400700 1 -- 172.21.15.18:0/3381337886 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11b8005040 con 0x7f11c80fbb70 2024-04-12T21:35:05.631 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:05.631+0000 7f11c7400700 1 --2- 172.21.15.18:0/3381337886 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f11c80fbb70 0x7f11c8102950 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x7f11b800b3b0 tx=0x7f11b8009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:35:05.631 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:05.631+0000 7f11bfe00700 1 -- 172.21.15.18:0/3381337886 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f11b800e040 con 0x7f11c80fbb70 2024-04-12T21:35:05.632 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:05.631+0000 7f11bfe00700 1 -- 172.21.15.18:0/3381337886 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f11b800bbc0 con 0x7f11c80fbb70 2024-04-12T21:35:05.632 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:05.631+0000 7f11ccc00700 1 -- 172.21.15.18:0/3381337886 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f11c81037b0 con 0x7f11c80fbb70 2024-04-12T21:35:05.632 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:05.631+0000 7f11bfe00700 1 -- 172.21.15.18:0/3381337886 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f11b801c3f0 con 0x7f11c80fbb70 2024-04-12T21:35:05.632 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:05.631+0000 7f11ccc00700 1 -- 172.21.15.18:0/3381337886 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f11c811a0e0 con 0x7f11c80fbb70 2024-04-12T21:35:05.632 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:05.632+0000 7f11ccc00700 1 -- 172.21.15.18:0/3381337886 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f11c80fd200 con 0x7f11c80fbb70 2024-04-12T21:35:05.633 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:05.633+0000 7f11bfe00700 1 -- 172.21.15.18:0/3381337886 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7f11b800bd30 con 0x7f11c80fbb70 2024-04-12T21:35:05.634 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:05.635+0000 7f11bfe00700 1 --2- 172.21.15.18:0/3381337886 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f11ac06ec00 0x7f11ac0710b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:35:05.635 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:05.635+0000 7f11bfe00700 1 -- 172.21.15.18:0/3381337886 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 6757+0+0 (secure 0 0 0) 0x7f11b808fc50 con 0x7f11c80fbb70 2024-04-12T21:35:05.635 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:05.635+0000 7f11bea00700 1 --2- 172.21.15.18:0/3381337886 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f11ac06ec00 0x7f11ac0710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:35:05.636 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:05.636+0000 7f11bea00700 1 --2- 172.21.15.18:0/3381337886 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f11ac06ec00 0x7f11ac0710b0 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f11c8104290 tx=0x7f11b0009040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:35:05.640 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:05.640+0000 7f11bfe00700 1 -- 172.21.15.18:0/3381337886 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f11b80588f0 con 0x7f11c80fbb70 2024-04-12T21:35:05.922 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:05.922+0000 7f11ccc00700 1 -- 172.21.15.18:0/3381337886 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f11c804bed0 con 0x7f11c80fbb70 2024-04-12T21:35:05.922 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:35:05 smithi018 ceph-mon[112590]: from='client.35848 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:35:05.923 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:35:05 smithi018 ceph-mon[112590]: from='client.35852 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:35:05.923 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:35:05 smithi018 ceph-mon[112590]: from='client.? 172.21.15.18:0/3102358418' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T21:35:05.923 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:05.923+0000 7f11bfe00700 1 -- 172.21.15.18:0/3381337886 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 31 v31) v1 ==== 76+0+1873 (secure 0 0 0) 0x7f11b805bfb0 con 0x7f11c80fbb70 2024-04-12T21:35:05.923 INFO:teuthology.orchestra.run.smithi018.stdout:e31 2024-04-12T21:35:05.923 INFO:teuthology.orchestra.run.smithi018.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-12T21:35:05.923 INFO:teuthology.orchestra.run.smithi018.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-12T21:35:05.923 INFO:teuthology.orchestra.run.smithi018.stdout:legacy client fscid: 1 2024-04-12T21:35:05.923 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-12T21:35:05.923 INFO:teuthology.orchestra.run.smithi018.stdout:Filesystem 'cephfs' (1) 2024-04-12T21:35:05.924 INFO:teuthology.orchestra.run.smithi018.stdout:fs_name cephfs 2024-04-12T21:35:05.924 INFO:teuthology.orchestra.run.smithi018.stdout:epoch 24 2024-04-12T21:35:05.924 INFO:teuthology.orchestra.run.smithi018.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-04-12T21:35:05.924 INFO:teuthology.orchestra.run.smithi018.stdout:created 2024-04-12T20:51:57.362850+0000 2024-04-12T21:35:05.924 INFO:teuthology.orchestra.run.smithi018.stdout:modified 2024-04-12T21:06:59.717190+0000 2024-04-12T21:35:05.924 INFO:teuthology.orchestra.run.smithi018.stdout:tableserver 0 2024-04-12T21:35:05.924 INFO:teuthology.orchestra.run.smithi018.stdout:root 0 2024-04-12T21:35:05.924 INFO:teuthology.orchestra.run.smithi018.stdout:session_timeout 60 2024-04-12T21:35:05.924 INFO:teuthology.orchestra.run.smithi018.stdout:session_autoclose 300 2024-04-12T21:35:05.924 INFO:teuthology.orchestra.run.smithi018.stdout:max_file_size 1099511627776 2024-04-12T21:35:05.924 INFO:teuthology.orchestra.run.smithi018.stdout:max_xattr_size 65536 2024-04-12T21:35:05.924 INFO:teuthology.orchestra.run.smithi018.stdout:required_client_features {} 2024-04-12T21:35:05.924 INFO:teuthology.orchestra.run.smithi018.stdout:last_failure 0 2024-04-12T21:35:05.924 INFO:teuthology.orchestra.run.smithi018.stdout:last_failure_osd_epoch 100 2024-04-12T21:35:05.924 INFO:teuthology.orchestra.run.smithi018.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-12T21:35:05.925 INFO:teuthology.orchestra.run.smithi018.stdout:max_mds 2 2024-04-12T21:35:05.925 INFO:teuthology.orchestra.run.smithi018.stdout:in 0,1 2024-04-12T21:35:05.925 INFO:teuthology.orchestra.run.smithi018.stdout:up {} 2024-04-12T21:35:05.925 INFO:teuthology.orchestra.run.smithi018.stdout:failed 0,1 2024-04-12T21:35:05.925 INFO:teuthology.orchestra.run.smithi018.stdout:damaged 2024-04-12T21:35:05.925 INFO:teuthology.orchestra.run.smithi018.stdout:stopped 2024-04-12T21:35:05.925 INFO:teuthology.orchestra.run.smithi018.stdout:data_pools [3] 2024-04-12T21:35:05.925 INFO:teuthology.orchestra.run.smithi018.stdout:metadata_pool 2 2024-04-12T21:35:05.925 INFO:teuthology.orchestra.run.smithi018.stdout:inline_data disabled 2024-04-12T21:35:05.925 INFO:teuthology.orchestra.run.smithi018.stdout:balancer 2024-04-12T21:35:05.925 INFO:teuthology.orchestra.run.smithi018.stdout:bal_rank_mask -1 2024-04-12T21:35:05.925 INFO:teuthology.orchestra.run.smithi018.stdout:standby_count_wanted 1 2024-04-12T21:35:05.925 INFO:teuthology.orchestra.run.smithi018.stdout:qdb_cluster leader: 0 members: 2024-04-12T21:35:05.925 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-12T21:35:05.926 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-12T21:35:05.926 INFO:teuthology.orchestra.run.smithi018.stdout:Standby daemons: 2024-04-12T21:35:05.926 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-12T21:35:05.926 INFO:teuthology.orchestra.run.smithi018.stdout:[mds.cephfs.smithi186.wjnafs{-1:24277} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.186:6826/438347207,v1:172.21.15.186:6827/438347207] compat {c=[1],r=[1],i=[7ff]}] 2024-04-12T21:35:05.926 INFO:teuthology.orchestra.run.smithi018.stdout:[mds.cephfs.smithi018.acqbsc{-1:34436} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.18:6828/755179764,v1:172.21.15.18:6829/755179764] compat {c=[1],r=[1],i=[1fff]}] 2024-04-12T21:35:05.926 INFO:teuthology.orchestra.run.smithi018.stdout:[mds.cephfs.smithi018.szjluo{-1:34468} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.18:6826/2407992653,v1:172.21.15.18:6827/2407992653] compat {c=[1],r=[1],i=[1fff]}] 2024-04-12T21:35:05.926 INFO:teuthology.orchestra.run.smithi018.stdout:[mds.cephfs.smithi186.jdllyn{-1:44595} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.186:6824/3523444202,v1:172.21.15.186:6825/3523444202] compat {c=[1],r=[1],i=[1fff]}] 2024-04-12T21:35:05.927 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:05.925+0000 7f11ccc00700 1 -- 172.21.15.18:0/3381337886 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f11ac06ec00 msgr2=0x7f11ac0710b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:35:05.927 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:05.925+0000 7f11ccc00700 1 --2- 172.21.15.18:0/3381337886 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f11ac06ec00 0x7f11ac0710b0 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f11c8104290 tx=0x7f11b0009040 comp rx=0 tx=0).stop 2024-04-12T21:35:05.927 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:05.925+0000 7f11ccc00700 1 -- 172.21.15.18:0/3381337886 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f11c80fbb70 msgr2=0x7f11c8102950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:35:05.927 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:05.925+0000 7f11ccc00700 1 --2- 172.21.15.18:0/3381337886 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f11c80fbb70 0x7f11c8102950 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x7f11b800b3b0 tx=0x7f11b8009e30 comp rx=0 tx=0).stop 2024-04-12T21:35:05.927 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:05.926+0000 7f11ccc00700 1 -- 172.21.15.18:0/3381337886 shutdown_connections 2024-04-12T21:35:05.927 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:05.926+0000 7f11ccc00700 1 --2- 172.21.15.18:0/3381337886 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f11ac06ec00 0x7f11ac0710b0 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:35:05.927 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:05.926+0000 7f11ccc00700 1 --2- 172.21.15.18:0/3381337886 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f11c80fbb70 0x7f11c8102950 unknown :-1 s=CLOSED pgs=899 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:35:05.927 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:05.926+0000 7f11ccc00700 1 --2- 172.21.15.18:0/3381337886 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f11c8101ba0 0x7f11c8102e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:35:05.927 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:05.926+0000 7f11ccc00700 1 -- 172.21.15.18:0/3381337886 >> 172.21.15.18:0/3381337886 conn(0x7f11c8071b70 msgr2=0x7f11c80f9fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:35:05.927 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:05.926+0000 7f11ccc00700 1 -- 172.21.15.18:0/3381337886 shutdown_connections 2024-04-12T21:35:05.927 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:05.926+0000 7f11ccc00700 1 -- 172.21.15.18:0/3381337886 wait complete. 2024-04-12T21:35:05.928 INFO:teuthology.orchestra.run.smithi018.stderr:dumped fsmap epoch 31 2024-04-12T21:35:06.075 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:06.074+0000 7f6c02400700 1 -- 172.21.15.18:0/777935738 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6bfc102e20 msgr2=0x7f6bfc1031f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:35:06.075 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:06.074+0000 7f6c02400700 1 --2- 172.21.15.18:0/777935738 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6bfc102e20 0x7f6bfc1031f0 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7f6bec004940 tx=0x7f6bec009e30 comp rx=0 tx=0).stop 2024-04-12T21:35:06.075 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:06.074+0000 7f6c02400700 1 -- 172.21.15.18:0/777935738 shutdown_connections 2024-04-12T21:35:06.075 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:06.074+0000 7f6c02400700 1 --2- 172.21.15.18:0/777935738 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6bfc1037c0 0x7f6bfc0fce30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:35:06.075 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:06.074+0000 7f6c02400700 1 --2- 172.21.15.18:0/777935738 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6bfc102e20 0x7f6bfc1031f0 unknown :-1 s=CLOSED pgs=900 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:35:06.075 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:06.074+0000 7f6c02400700 1 -- 172.21.15.18:0/777935738 >> 172.21.15.18:0/777935738 conn(0x7f6bfc0f89d0 msgr2=0x7f6bfc0fade0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:35:06.075 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:06.075+0000 7f6c02400700 1 -- 172.21.15.18:0/777935738 shutdown_connections 2024-04-12T21:35:06.075 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:06.075+0000 7f6c02400700 1 -- 172.21.15.18:0/777935738 wait complete. 2024-04-12T21:35:06.076 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:06.075+0000 7f6c02400700 1 Processor -- start 2024-04-12T21:35:06.076 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:06.076+0000 7f6c02400700 1 -- start start 2024-04-12T21:35:06.076 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:06.076+0000 7f6c02400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6bfc102e20 0x7f6bfc115de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:35:06.076 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:06.076+0000 7f6c02400700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6bfc1037c0 0x7f6bfc116320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:35:06.076 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:06.076+0000 7f6c02400700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6bfc116970 con 0x7f6bfc102e20 2024-04-12T21:35:06.077 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:06.076+0000 7f6c02400700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6bfc10fd40 con 0x7f6bfc1037c0 2024-04-12T21:35:06.077 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:06.076+0000 7f6bfb400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6bfc102e20 0x7f6bfc115de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:35:06.077 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:06.077+0000 7f6bfb400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6bfc102e20 0x7f6bfc115de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:49300/0 (socket says 172.21.15.18:49300) 2024-04-12T21:35:06.077 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:06.077+0000 7f6bfb400700 1 -- 172.21.15.18:0/3488460745 learned_addr learned my addr 172.21.15.18:0/3488460745 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:35:06.078 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:06.077+0000 7f6bf3e00700 1 --2- 172.21.15.18:0/3488460745 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6bfc1037c0 0x7f6bfc116320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:35:06.078 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:06.077+0000 7f6bfb400700 1 -- 172.21.15.18:0/3488460745 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6bfc1037c0 msgr2=0x7f6bfc116320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:35:06.078 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:06.077+0000 7f6bfb400700 1 --2- 172.21.15.18:0/3488460745 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6bfc1037c0 0x7f6bfc116320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:35:06.078 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:06.077+0000 7f6bfb400700 1 -- 172.21.15.18:0/3488460745 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6bec005040 con 0x7f6bfc102e20 2024-04-12T21:35:06.078 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:06.077+0000 7f6bf3e00700 1 --2- 172.21.15.18:0/3488460745 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6bfc1037c0 0x7f6bfc116320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-12T21:35:06.078 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:06.077+0000 7f6bfb400700 1 --2- 172.21.15.18:0/3488460745 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6bfc102e20 0x7f6bfc115de0 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 crypto rx=0x7f6bec009180 tx=0x7f6bec009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:35:06.078 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:06.078+0000 7f6bf3400700 1 -- 172.21.15.18:0/3488460745 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6bec00e040 con 0x7f6bfc102e20 2024-04-12T21:35:06.079 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:06.078+0000 7f6bf3400700 1 -- 172.21.15.18:0/3488460745 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f6bec00bbc0 con 0x7f6bfc102e20 2024-04-12T21:35:06.079 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:06.078+0000 7f6bf3400700 1 -- 172.21.15.18:0/3488460745 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6bec01c630 con 0x7f6bfc102e20 2024-04-12T21:35:06.079 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:06.078+0000 7f6c02400700 1 -- 172.21.15.18:0/3488460745 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6bfc10ffc0 con 0x7f6bfc102e20 2024-04-12T21:35:06.079 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:06.078+0000 7f6c02400700 1 -- 172.21.15.18:0/3488460745 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6bfc110580 con 0x7f6bfc102e20 2024-04-12T21:35:06.079 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:06.079+0000 7f6c02400700 1 -- 172.21.15.18:0/3488460745 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6bfc107320 con 0x7f6bfc102e20 2024-04-12T21:35:06.080 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:06.080+0000 7f6bf3400700 1 -- 172.21.15.18:0/3488460745 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7f6bec00bd30 con 0x7f6bfc102e20 2024-04-12T21:35:06.081 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:06.081+0000 7f6bf3400700 1 --2- 172.21.15.18:0/3488460745 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f6be006ec10 0x7f6be00710c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:35:06.081 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:06.081+0000 7f6bf3400700 1 -- 172.21.15.18:0/3488460745 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 6757+0+0 (secure 0 0 0) 0x7f6bec08fcb0 con 0x7f6bfc102e20 2024-04-12T21:35:06.081 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:06.081+0000 7f6bf3e00700 1 --2- 172.21.15.18:0/3488460745 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f6be006ec10 0x7f6be00710c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:35:06.082 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:06.082+0000 7f6bf3e00700 1 --2- 172.21.15.18:0/3488460745 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f6be006ec10 0x7f6be00710c0 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f6be4009fd0 tx=0x7f6be400b040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:35:06.086 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:06.086+0000 7f6bf3400700 1 -- 172.21.15.18:0/3488460745 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f6bec05c5a0 con 0x7f6bfc102e20 2024-04-12T21:35:06.282 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:35:05 smithi186 ceph-mon[86379]: from='client.35848 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:35:06.283 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:35:05 smithi186 ceph-mon[86379]: from='client.35852 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:35:06.283 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:35:05 smithi186 ceph-mon[86379]: from='client.? 172.21.15.18:0/3102358418' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T21:35:06.333 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:06.334+0000 7f6c02400700 1 -- 172.21.15.18:0/3488460745 --> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f6bfc109f20 con 0x7f6be006ec10 2024-04-12T21:35:06.335 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:06.336+0000 7f6bf3400700 1 -- 172.21.15.18:0/3488460745 <== mgr.34102 v2:172.21.15.18:6800/3812891919 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f6bfc109f20 con 0x7f6be006ec10 2024-04-12T21:35:06.336 INFO:teuthology.orchestra.run.smithi018.stdout:{ 2024-04-12T21:35:06.336 INFO:teuthology.orchestra.run.smithi018.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:f4452def3be4d350f4a8e5feaf7ebb1511be9298fe68478ba7a2d6fbd7d69a63", 2024-04-12T21:35:06.336 INFO:teuthology.orchestra.run.smithi018.stdout: "in_progress": true, 2024-04-12T21:35:06.336 INFO:teuthology.orchestra.run.smithi018.stdout: "which": "Upgrading all daemon types on all hosts", 2024-04-12T21:35:06.336 INFO:teuthology.orchestra.run.smithi018.stdout: "services_complete": [ 2024-04-12T21:35:06.336 INFO:teuthology.orchestra.run.smithi018.stdout: "crash", 2024-04-12T21:35:06.336 INFO:teuthology.orchestra.run.smithi018.stdout: "osd", 2024-04-12T21:35:06.336 INFO:teuthology.orchestra.run.smithi018.stdout: "mon", 2024-04-12T21:35:06.336 INFO:teuthology.orchestra.run.smithi018.stdout: "mgr" 2024-04-12T21:35:06.336 INFO:teuthology.orchestra.run.smithi018.stdout: ], 2024-04-12T21:35:06.336 INFO:teuthology.orchestra.run.smithi018.stdout: "progress": "15/21 daemons upgraded", 2024-04-12T21:35:06.336 INFO:teuthology.orchestra.run.smithi018.stdout: "message": "Currently upgrading mds daemons", 2024-04-12T21:35:06.337 INFO:teuthology.orchestra.run.smithi018.stdout: "is_paused": false 2024-04-12T21:35:06.337 INFO:teuthology.orchestra.run.smithi018.stdout:} 2024-04-12T21:35:06.338 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:06.339+0000 7f6c02400700 1 -- 172.21.15.18:0/3488460745 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f6be006ec10 msgr2=0x7f6be00710c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:35:06.338 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:06.339+0000 7f6c02400700 1 --2- 172.21.15.18:0/3488460745 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f6be006ec10 0x7f6be00710c0 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f6be4009fd0 tx=0x7f6be400b040 comp rx=0 tx=0).stop 2024-04-12T21:35:06.339 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:06.339+0000 7f6c02400700 1 -- 172.21.15.18:0/3488460745 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6bfc102e20 msgr2=0x7f6bfc115de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:35:06.339 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:06.339+0000 7f6c02400700 1 --2- 172.21.15.18:0/3488460745 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6bfc102e20 0x7f6bfc115de0 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 crypto rx=0x7f6bec009180 tx=0x7f6bec009e30 comp rx=0 tx=0).stop 2024-04-12T21:35:06.339 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:06.339+0000 7f6c02400700 1 -- 172.21.15.18:0/3488460745 shutdown_connections 2024-04-12T21:35:06.339 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:06.339+0000 7f6c02400700 1 --2- 172.21.15.18:0/3488460745 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f6be006ec10 0x7f6be00710c0 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:35:06.339 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:06.339+0000 7f6c02400700 1 --2- 172.21.15.18:0/3488460745 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6bfc102e20 0x7f6bfc115de0 unknown :-1 s=CLOSED pgs=901 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:35:06.339 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:06.339+0000 7f6c02400700 1 --2- 172.21.15.18:0/3488460745 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6bfc1037c0 0x7f6bfc116320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:35:06.339 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:06.339+0000 7f6c02400700 1 -- 172.21.15.18:0/3488460745 >> 172.21.15.18:0/3488460745 conn(0x7f6bfc0f89d0 msgr2=0x7f6bfc0fa6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:35:06.339 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:06.340+0000 7f6c02400700 1 -- 172.21.15.18:0/3488460745 shutdown_connections 2024-04-12T21:35:06.339 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:06.340+0000 7f6c02400700 1 -- 172.21.15.18:0/3488460745 wait complete. 2024-04-12T21:35:06.497 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:06.497+0000 7ffb41e00700 1 -- 172.21.15.18:0/599021870 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ffb3c0fe830 msgr2=0x7ffb3c106f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:35:06.497 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:06.497+0000 7ffb41e00700 1 --2- 172.21.15.18:0/599021870 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ffb3c0fe830 0x7ffb3c106f00 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 crypto rx=0x7ffb28004940 tx=0x7ffb28009e30 comp rx=0 tx=0).stop 2024-04-12T21:35:06.497 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:06.497+0000 7ffb41e00700 1 -- 172.21.15.18:0/599021870 shutdown_connections 2024-04-12T21:35:06.497 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:06.497+0000 7ffb41e00700 1 --2- 172.21.15.18:0/599021870 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ffb3c0fe830 0x7ffb3c106f00 unknown :-1 s=CLOSED pgs=902 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:35:06.497 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:06.497+0000 7ffb41e00700 1 --2- 172.21.15.18:0/599021870 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ffb3c0fde90 0x7ffb3c0fe260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:35:06.497 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:06.497+0000 7ffb41e00700 1 -- 172.21.15.18:0/599021870 >> 172.21.15.18:0/599021870 conn(0x7ffb3c072bc0 msgr2=0x7ffb3c072fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:35:06.497 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:06.497+0000 7ffb41e00700 1 -- 172.21.15.18:0/599021870 shutdown_connections 2024-04-12T21:35:06.497 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:06.497+0000 7ffb41e00700 1 -- 172.21.15.18:0/599021870 wait complete. 2024-04-12T21:35:06.498 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:06.498+0000 7ffb41e00700 1 Processor -- start 2024-04-12T21:35:06.498 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:06.498+0000 7ffb41e00700 1 -- start start 2024-04-12T21:35:06.499 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:06.499+0000 7ffb41e00700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ffb3c0fde90 0x7ffb3c113ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:35:06.499 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:06.499+0000 7ffb41e00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ffb3c0fe830 0x7ffb3c1140e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:35:06.499 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:06.499+0000 7ffb41e00700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb3c114730 con 0x7ffb3c0fe830 2024-04-12T21:35:06.499 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:06.499+0000 7ffb41e00700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb3c10db20 con 0x7ffb3c0fde90 2024-04-12T21:35:06.499 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:06.499+0000 7ffb3a000700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ffb3c0fe830 0x7ffb3c1140e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:35:06.499 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:06.499+0000 7ffb3a000700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ffb3c0fe830 0x7ffb3c1140e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:49306/0 (socket says 172.21.15.18:49306) 2024-04-12T21:35:06.499 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:06.499+0000 7ffb3a000700 1 -- 172.21.15.18:0/1228383557 learned_addr learned my addr 172.21.15.18:0/1228383557 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:35:06.499 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:06.500+0000 7ffb3aa00700 1 --2- 172.21.15.18:0/1228383557 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ffb3c0fde90 0x7ffb3c113ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:35:06.500 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:06.500+0000 7ffb3a000700 1 -- 172.21.15.18:0/1228383557 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ffb3c0fde90 msgr2=0x7ffb3c113ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:35:06.500 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:06.500+0000 7ffb3a000700 1 --2- 172.21.15.18:0/1228383557 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ffb3c0fde90 0x7ffb3c113ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:35:06.500 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:06.500+0000 7ffb3a000700 1 -- 172.21.15.18:0/1228383557 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb28005040 con 0x7ffb3c0fe830 2024-04-12T21:35:06.501 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:06.500+0000 7ffb3a000700 1 --2- 172.21.15.18:0/1228383557 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ffb3c0fe830 0x7ffb3c1140e0 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x7ffb28000c00 tx=0x7ffb28009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:35:06.501 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:06.501+0000 7ffb32a00700 1 -- 172.21.15.18:0/1228383557 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ffb2800e040 con 0x7ffb3c0fe830 2024-04-12T21:35:06.501 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:06.501+0000 7ffb32a00700 1 -- 172.21.15.18:0/1228383557 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ffb28013420 con 0x7ffb3c0fe830 2024-04-12T21:35:06.501 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:06.501+0000 7ffb32a00700 1 -- 172.21.15.18:0/1228383557 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ffb2801c610 con 0x7ffb3c0fe830 2024-04-12T21:35:06.501 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:06.501+0000 7ffb41e00700 1 -- 172.21.15.18:0/1228383557 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffb3c10dda0 con 0x7ffb3c0fe830 2024-04-12T21:35:06.501 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:06.501+0000 7ffb41e00700 1 -- 172.21.15.18:0/1228383557 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffb3c10e2e0 con 0x7ffb3c0fe830 2024-04-12T21:35:06.501 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:06.502+0000 7ffb41e00700 1 -- 172.21.15.18:0/1228383557 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffb3c065e30 con 0x7ffb3c0fe830 2024-04-12T21:35:06.502 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:06.503+0000 7ffb32a00700 1 -- 172.21.15.18:0/1228383557 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7ffb28013590 con 0x7ffb3c0fe830 2024-04-12T21:35:06.503 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:06.504+0000 7ffb32a00700 1 --2- 172.21.15.18:0/1228383557 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7ffb2006ec10 0x7ffb200710c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:35:06.503 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:06.504+0000 7ffb32a00700 1 -- 172.21.15.18:0/1228383557 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 6757+0+0 (secure 0 0 0) 0x7ffb2808fbf0 con 0x7ffb3c0fe830 2024-04-12T21:35:06.503 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:06.504+0000 7ffb3aa00700 1 --2- 172.21.15.18:0/1228383557 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7ffb2006ec10 0x7ffb200710c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:35:06.504 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:06.505+0000 7ffb3aa00700 1 --2- 172.21.15.18:0/1228383557 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7ffb2006ec10 0x7ffb200710c0 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7ffb2c007fd0 tx=0x7ffb2c009040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:35:06.508 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:06.509+0000 7ffb32a00700 1 -- 172.21.15.18:0/1228383557 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7ffb2805c3a0 con 0x7ffb3c0fe830 2024-04-12T21:35:06.832 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:06.833+0000 7ffb41e00700 1 -- 172.21.15.18:0/1228383557 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7ffb3c109f10 con 0x7ffb3c0fe830 2024-04-12T21:35:06.833 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:06.834+0000 7ffb32a00700 1 -- 172.21.15.18:0/1228383557 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+4303 (secure 0 0 0) 0x7ffb2805bf30 con 0x7ffb3c0fe830 2024-04-12T21:35:06.834 INFO:teuthology.orchestra.run.smithi018.stdout:HEALTH_ERR failed to probe daemons or devices; 1 filesystem is degraded; 1 filesystem is offline 2024-04-12T21:35:06.834 INFO:teuthology.orchestra.run.smithi018.stdout:[WRN] CEPHADM_REFRESH_FAILED: failed to probe daemons or devices 2024-04-12T21:35:06.834 INFO:teuthology.orchestra.run.smithi018.stdout: host smithi018 `cephadm ceph-volume` failed: cephadm exited with an error code: 1, stderr: Inferring config /var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/mon.smithi018/config 2024-04-12T21:35:06.834 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/podman: timeout after 895 seconds 2024-04-12T21:35:06.834 INFO:teuthology.orchestra.run.smithi018.stdout:Non-zero exit code 124 from /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:f4452def3be4d350f4a8e5feaf7ebb1511be9298fe68478ba7a2d6fbd7d69a63 -e NODE_NAME=smithi018 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968:/var/run/ceph:z -v /var/log/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968:/var/log/ceph:z -v /var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmpxsmwkkuh:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:f4452def3be4d350f4a8e5feaf7ebb1511be9298fe68478ba7a2d6fbd7d69a63 inventory --format=json-pretty --filter-for-batch 2024-04-12T21:35:06.834 INFO:teuthology.orchestra.run.smithi018.stdout:Traceback (most recent call last): 2024-04-12T21:35:06.834 INFO:teuthology.orchestra.run.smithi018.stdout: File "/usr/lib64/python3.9/runpy.py", line 197, in _run_module_as_main 2024-04-12T21:35:06.834 INFO:teuthology.orchestra.run.smithi018.stdout: return _run_code(code, main_globals, None, 2024-04-12T21:35:06.834 INFO:teuthology.orchestra.run.smithi018.stdout: File "/usr/lib64/python3.9/runpy.py", line 87, in _run_code 2024-04-12T21:35:06.834 INFO:teuthology.orchestra.run.smithi018.stdout: exec(code, run_globals) 2024-04-12T21:35:06.834 INFO:teuthology.orchestra.run.smithi018.stdout: File "/var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/cephadm.e44f3ed5e34e4ef52eedef233d1190bff004470d8e8005127b5312bff357fa7f/__main__.py", line 5618, in 2024-04-12T21:35:06.834 INFO:teuthology.orchestra.run.smithi018.stdout: File "/var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/cephadm.e44f3ed5e34e4ef52eedef233d1190bff004470d8e8005127b5312bff357fa7f/__main__.py", line 5606, in main 2024-04-12T21:35:06.834 INFO:teuthology.orchestra.run.smithi018.stdout: File "/var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/cephadm.e44f3ed5e34e4ef52eedef233d1190bff004470d8e8005127b5312bff357fa7f/__main__.py", line 408, in _infer_config 2024-04-12T21:35:06.835 INFO:teuthology.orchestra.run.smithi018.stdout: File "/var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/cephadm.e44f3ed5e34e4ef52eedef233d1190bff004470d8e8005127b5312bff357fa7f/__main__.py", line 323, in _infer_fsid 2024-04-12T21:35:06.835 INFO:teuthology.orchestra.run.smithi018.stdout: File "/var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/cephadm.e44f3ed5e34e4ef52eedef233d1190bff004470d8e8005127b5312bff357fa7f/__main__.py", line 436, in _infer_image 2024-04-12T21:35:06.835 INFO:teuthology.orchestra.run.smithi018.stdout: File "/var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/cephadm.e44f3ed5e34e4ef52eedef233d1190bff004470d8e8005127b5312bff357fa7f/__main__.py", line 310, in _validate_fsid 2024-04-12T21:35:06.835 INFO:teuthology.orchestra.run.smithi018.stdout: File "/var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/cephadm.e44f3ed5e34e4ef52eedef233d1190bff004470d8e8005127b5312bff357fa7f/__main__.py", line 3359, in command_ceph_volume 2024-04-12T21:35:06.835 INFO:teuthology.orchestra.run.smithi018.stdout: File "/var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/cephadm.e44f3ed5e34e4ef52eedef233d1190bff004470d8e8005127b5312bff357fa7f/cephadmlib/call_wrappers.py", line 310, in call_throws 2024-04-12T21:35:06.835 INFO:teuthology.orchestra.run.smithi018.stdout:RuntimeError: Failed command: /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:f4452def3be4d350f4a8e5feaf7ebb1511be9298fe68478ba7a2d6fbd7d69a63 -e NODE_NAME=smithi018 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968:/var/run/ceph:z -v /var/log/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968:/var/log/ceph:z -v /var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmpxsmwkkuh:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:f4452def3be4d350f4a8e5feaf7ebb1511be9298fe68478ba7a2d6fbd7d69a63 inventory --format=json-pretty --filter-for-batch 2024-04-12T21:35:06.835 INFO:teuthology.orchestra.run.smithi018.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-04-12T21:35:06.835 INFO:teuthology.orchestra.run.smithi018.stdout: fs cephfs is degraded 2024-04-12T21:35:06.835 INFO:teuthology.orchestra.run.smithi018.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-04-12T21:35:06.835 INFO:teuthology.orchestra.run.smithi018.stdout: fs cephfs is offline because no MDS is active for it. 2024-04-12T21:35:06.836 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:06.837+0000 7ffb41e00700 1 -- 172.21.15.18:0/1228383557 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7ffb2006ec10 msgr2=0x7ffb200710c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:35:06.837 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:06.837+0000 7ffb41e00700 1 --2- 172.21.15.18:0/1228383557 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7ffb2006ec10 0x7ffb200710c0 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7ffb2c007fd0 tx=0x7ffb2c009040 comp rx=0 tx=0).stop 2024-04-12T21:35:06.837 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:06.837+0000 7ffb41e00700 1 -- 172.21.15.18:0/1228383557 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ffb3c0fe830 msgr2=0x7ffb3c1140e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:35:06.837 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:06.837+0000 7ffb41e00700 1 --2- 172.21.15.18:0/1228383557 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ffb3c0fe830 0x7ffb3c1140e0 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x7ffb28000c00 tx=0x7ffb28009e30 comp rx=0 tx=0).stop 2024-04-12T21:35:06.837 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:06.837+0000 7ffb41e00700 1 -- 172.21.15.18:0/1228383557 shutdown_connections 2024-04-12T21:35:06.837 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:06.837+0000 7ffb41e00700 1 --2- 172.21.15.18:0/1228383557 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7ffb2006ec10 0x7ffb200710c0 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:35:06.837 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:06.837+0000 7ffb41e00700 1 --2- 172.21.15.18:0/1228383557 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ffb3c0fde90 0x7ffb3c113ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:35:06.837 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:06.837+0000 7ffb41e00700 1 --2- 172.21.15.18:0/1228383557 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ffb3c0fe830 0x7ffb3c1140e0 unknown :-1 s=CLOSED pgs=903 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:35:06.837 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:06.837+0000 7ffb41e00700 1 -- 172.21.15.18:0/1228383557 >> 172.21.15.18:0/1228383557 conn(0x7ffb3c072bc0 msgr2=0x7ffb3c1037b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:35:06.837 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:06.837+0000 7ffb41e00700 1 -- 172.21.15.18:0/1228383557 shutdown_connections 2024-04-12T21:35:06.837 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:06.837+0000 7ffb41e00700 1 -- 172.21.15.18:0/1228383557 wait complete. 2024-04-12T21:35:07.103 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:35:06 smithi018 ceph-mon[112590]: from='client.35856 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:35:07.104 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:35:06 smithi018 ceph-mon[112590]: pgmap v1060: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:35:07.104 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:35:06 smithi018 ceph-mon[112590]: from='client.? 172.21.15.18:0/3381337886' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-12T21:35:07.104 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:35:06 smithi018 ceph-mon[112590]: from='client.? 172.21.15.18:0/1228383557' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-12T21:35:07.282 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:35:06 smithi186 ceph-mon[86379]: from='client.35856 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:35:07.283 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:35:06 smithi186 ceph-mon[86379]: pgmap v1060: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:35:07.283 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:35:06 smithi186 ceph-mon[86379]: from='client.? 172.21.15.18:0/3381337886' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-12T21:35:07.283 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:35:06 smithi186 ceph-mon[86379]: from='client.? 172.21.15.18:0/1228383557' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-12T21:35:08.282 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:35:07 smithi186 ceph-mon[86379]: from='client.35868 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:35:08.353 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:35:07 smithi018 ceph-mon[112590]: from='client.35868 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:35:09.282 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:35:08 smithi186 ceph-mon[86379]: pgmap v1061: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:35:09.353 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:35:08 smithi018 ceph-mon[112590]: pgmap v1061: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:35:11.282 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:35:10 smithi186 ceph-mon[86379]: pgmap v1062: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:35:11.283 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:35:10 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T21:35:11.353 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:35:10 smithi018 ceph-mon[112590]: pgmap v1062: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:35:11.354 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:35:10 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T21:35:13.282 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:35:12 smithi186 ceph-mon[86379]: pgmap v1063: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:35:13.353 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:35:12 smithi018 ceph-mon[112590]: pgmap v1063: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:35:15.156 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:35:14 smithi018 ceph-mon[112590]: pgmap v1064: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:35:15.282 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:35:14 smithi186 ceph-mon[86379]: pgmap v1064: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:35:16.782 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:35:16 smithi186 ceph-mon[86379]: pgmap v1065: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:35:16.853 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:35:16 smithi018 ceph-mon[112590]: pgmap v1065: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:35:19.103 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:35:18 smithi018 ceph-mon[112590]: pgmap v1066: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:35:19.282 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:35:18 smithi186 ceph-mon[86379]: pgmap v1066: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:35:21.103 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:35:20 smithi018 ceph-mon[112590]: pgmap v1067: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:35:21.282 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:35:20 smithi186 ceph-mon[86379]: pgmap v1067: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:35:23.103 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:35:22 smithi018 ceph-mon[112590]: pgmap v1068: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:35:23.282 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:35:22 smithi186 ceph-mon[86379]: pgmap v1068: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:35:25.103 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:35:24 smithi018 ceph-mon[112590]: pgmap v1069: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:35:25.282 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:35:24 smithi186 ceph-mon[86379]: pgmap v1069: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:35:26.103 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:35:25 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T21:35:26.282 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:35:25 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T21:35:27.103 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:35:26 smithi018 ceph-mon[112590]: pgmap v1070: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:35:27.282 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:35:26 smithi186 ceph-mon[86379]: pgmap v1070: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:35:29.282 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:35:28 smithi186 ceph-mon[86379]: pgmap v1071: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:35:29.353 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:35:28 smithi018 ceph-mon[112590]: pgmap v1071: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:35:31.282 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:35:30 smithi186 ceph-mon[86379]: pgmap v1072: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:35:31.353 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:35:30 smithi018 ceph-mon[112590]: pgmap v1072: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:35:33.282 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:35:32 smithi186 ceph-mon[86379]: pgmap v1073: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:35:33.353 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:35:32 smithi018 ceph-mon[112590]: pgmap v1073: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:35:35.282 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:35:34 smithi186 ceph-mon[86379]: pgmap v1074: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:35:35.353 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:35:34 smithi018 ceph-mon[112590]: pgmap v1074: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:35:37.016 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:37.017+0000 7f7d29600700 1 -- 172.21.15.18:0/686059742 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7d241059c0 msgr2=0x7f7d240ff080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:35:37.017 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:37.017+0000 7f7d29600700 1 --2- 172.21.15.18:0/686059742 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7d241059c0 0x7f7d240ff080 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x7f7d10004940 tx=0x7f7d10009e30 comp rx=0 tx=0).stop 2024-04-12T21:35:37.017 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:37.017+0000 7f7d29600700 1 -- 172.21.15.18:0/686059742 shutdown_connections 2024-04-12T21:35:37.017 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:37.017+0000 7f7d29600700 1 --2- 172.21.15.18:0/686059742 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7d241059c0 0x7f7d240ff080 unknown :-1 s=CLOSED pgs=904 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:35:37.017 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:37.017+0000 7f7d29600700 1 --2- 172.21.15.18:0/686059742 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7d24105020 0x7f7d241053f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:35:37.017 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:37.017+0000 7f7d29600700 1 -- 172.21.15.18:0/686059742 >> 172.21.15.18:0/686059742 conn(0x7f7d240faba0 msgr2=0x7f7d240fcfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:35:37.017 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:37.017+0000 7f7d29600700 1 -- 172.21.15.18:0/686059742 shutdown_connections 2024-04-12T21:35:37.017 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:37.017+0000 7f7d29600700 1 -- 172.21.15.18:0/686059742 wait complete. 2024-04-12T21:35:37.018 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:37.018+0000 7f7d29600700 1 Processor -- start 2024-04-12T21:35:37.019 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:37.018+0000 7f7d29600700 1 -- start start 2024-04-12T21:35:37.019 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:37.019+0000 7f7d29600700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7d24105020 0x7f7d24115b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:35:37.019 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:37.019+0000 7f7d29600700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7d241059c0 0x7f7d24116050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:35:37.019 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:37.019+0000 7f7d29600700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d24116730 con 0x7f7d24105020 2024-04-12T21:35:37.019 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:37.019+0000 7f7d29600700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d2410fcc0 con 0x7f7d241059c0 2024-04-12T21:35:37.019 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:37.019+0000 7f7d22a00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7d24105020 0x7f7d24115b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:35:37.019 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:37.019+0000 7f7d22a00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7d24105020 0x7f7d24115b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:33844/0 (socket says 172.21.15.18:33844) 2024-04-12T21:35:37.019 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:37.019+0000 7f7d22a00700 1 -- 172.21.15.18:0/3543649584 learned_addr learned my addr 172.21.15.18:0/3543649584 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:35:37.019 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:37.019+0000 7f7d22000700 1 --2- 172.21.15.18:0/3543649584 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7d241059c0 0x7f7d24116050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:35:37.019 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:37.020+0000 7f7d22a00700 1 -- 172.21.15.18:0/3543649584 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7d241059c0 msgr2=0x7f7d24116050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:35:37.019 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:37.020+0000 7f7d22a00700 1 --2- 172.21.15.18:0/3543649584 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7d241059c0 0x7f7d24116050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:35:37.019 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:37.020+0000 7f7d22a00700 1 -- 172.21.15.18:0/3543649584 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d10005040 con 0x7f7d24105020 2024-04-12T21:35:37.019 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:37.020+0000 7f7d22000700 1 --2- 172.21.15.18:0/3543649584 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7d241059c0 0x7f7d24116050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-12T21:35:37.020 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:37.020+0000 7f7d22a00700 1 --2- 172.21.15.18:0/3543649584 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7d24105020 0x7f7d24115b10 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7f7d1400b790 tx=0x7f7d14009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:35:37.020 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:37.020+0000 7f7d1aa00700 1 -- 172.21.15.18:0/3543649584 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7d1400f770 con 0x7f7d24105020 2024-04-12T21:35:37.020 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:37.020+0000 7f7d1aa00700 1 -- 172.21.15.18:0/3543649584 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f7d1400fda0 con 0x7f7d24105020 2024-04-12T21:35:37.021 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:37.020+0000 7f7d1aa00700 1 -- 172.21.15.18:0/3543649584 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7d1400d310 con 0x7f7d24105020 2024-04-12T21:35:37.021 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:37.020+0000 7f7d29600700 1 -- 172.21.15.18:0/3543649584 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7d2410ffa0 con 0x7f7d24105020 2024-04-12T21:35:37.021 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:37.021+0000 7f7d29600700 1 -- 172.21.15.18:0/3543649584 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7d24110560 con 0x7f7d24105020 2024-04-12T21:35:37.021 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:37.022+0000 7f7d29600700 1 -- 172.21.15.18:0/3543649584 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7d24100770 con 0x7f7d24105020 2024-04-12T21:35:37.022 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:37.023+0000 7f7d1aa00700 1 -- 172.21.15.18:0/3543649584 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7f7d14010420 con 0x7f7d24105020 2024-04-12T21:35:37.023 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:37.024+0000 7f7d1aa00700 1 --2- 172.21.15.18:0/3543649584 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f7d0806ec00 0x7f7d080710b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:35:37.024 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:37.024+0000 7f7d1aa00700 1 -- 172.21.15.18:0/3543649584 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 6757+0+0 (secure 0 0 0) 0x7f7d1408e8e0 con 0x7f7d24105020 2024-04-12T21:35:37.024 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:37.024+0000 7f7d22000700 1 --2- 172.21.15.18:0/3543649584 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f7d0806ec00 0x7f7d080710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:35:37.024 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:37.025+0000 7f7d22000700 1 --2- 172.21.15.18:0/3543649584 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f7d0806ec00 0x7f7d080710b0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f7d10000c00 tx=0x7f7d10009e30 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:35:37.028 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:37.029+0000 7f7d1aa00700 1 -- 172.21.15.18:0/3543649584 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f7d140576a0 con 0x7f7d24105020 2024-04-12T21:35:37.282 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:35:36 smithi186 ceph-mon[86379]: pgmap v1075: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:35:37.283 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:37.283+0000 7f7d29600700 1 -- 172.21.15.18:0/3543649584 --> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f7d24110890 con 0x7f7d0806ec00 2024-04-12T21:35:37.283 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:35:36 smithi018 ceph-mon[112590]: pgmap v1075: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:35:37.284 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:37.285+0000 7f7d1aa00700 1 -- 172.21.15.18:0/3543649584 <== mgr.34102 v2:172.21.15.18:6800/3812891919 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f7d24110890 con 0x7f7d0806ec00 2024-04-12T21:35:37.287 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:37.288+0000 7f7d29600700 1 -- 172.21.15.18:0/3543649584 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f7d0806ec00 msgr2=0x7f7d080710b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:35:37.287 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:37.288+0000 7f7d29600700 1 --2- 172.21.15.18:0/3543649584 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f7d0806ec00 0x7f7d080710b0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f7d10000c00 tx=0x7f7d10009e30 comp rx=0 tx=0).stop 2024-04-12T21:35:37.287 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:37.288+0000 7f7d29600700 1 -- 172.21.15.18:0/3543649584 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7d24105020 msgr2=0x7f7d24115b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:35:37.287 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:37.288+0000 7f7d29600700 1 --2- 172.21.15.18:0/3543649584 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7d24105020 0x7f7d24115b10 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7f7d1400b790 tx=0x7f7d14009250 comp rx=0 tx=0).stop 2024-04-12T21:35:37.288 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:37.288+0000 7f7d29600700 1 -- 172.21.15.18:0/3543649584 shutdown_connections 2024-04-12T21:35:37.288 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:37.288+0000 7f7d29600700 1 --2- 172.21.15.18:0/3543649584 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f7d0806ec00 0x7f7d080710b0 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:35:37.288 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:37.288+0000 7f7d29600700 1 --2- 172.21.15.18:0/3543649584 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7d24105020 0x7f7d24115b10 unknown :-1 s=CLOSED pgs=905 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:35:37.288 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:37.288+0000 7f7d29600700 1 --2- 172.21.15.18:0/3543649584 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7d241059c0 0x7f7d24116050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:35:37.288 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:37.288+0000 7f7d29600700 1 -- 172.21.15.18:0/3543649584 >> 172.21.15.18:0/3543649584 conn(0x7f7d240faba0 msgr2=0x7f7d240fbaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:35:37.288 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:37.289+0000 7f7d29600700 1 -- 172.21.15.18:0/3543649584 shutdown_connections 2024-04-12T21:35:37.289 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:37.289+0000 7f7d29600700 1 -- 172.21.15.18:0/3543649584 wait complete. 2024-04-12T21:35:37.304 INFO:teuthology.orchestra.run.smithi018.stdout:true 2024-04-12T21:35:37.454 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:37.454+0000 7fb608e00700 1 -- 172.21.15.18:0/4098087938 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb6040708a0 msgr2=0x7fb604070d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:35:37.454 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:37.454+0000 7fb608e00700 1 --2- 172.21.15.18:0/4098087938 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb6040708a0 0x7fb604070d10 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 crypto rx=0x7fb5ec004970 tx=0x7fb5ec009e30 comp rx=0 tx=0).stop 2024-04-12T21:35:37.454 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:37.454+0000 7fb608e00700 1 -- 172.21.15.18:0/4098087938 shutdown_connections 2024-04-12T21:35:37.454 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:37.454+0000 7fb608e00700 1 --2- 172.21.15.18:0/4098087938 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb6040708a0 0x7fb604070d10 unknown :-1 s=CLOSED pgs=906 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:35:37.454 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:37.454+0000 7fb608e00700 1 --2- 172.21.15.18:0/4098087938 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb604071f90 0x7fb604070360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:35:37.454 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:37.454+0000 7fb608e00700 1 -- 172.21.15.18:0/4098087938 >> 172.21.15.18:0/4098087938 conn(0x7fb6040f89b0 msgr2=0x7fb6040fadc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:35:37.454 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:37.454+0000 7fb608e00700 1 -- 172.21.15.18:0/4098087938 shutdown_connections 2024-04-12T21:35:37.454 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:37.455+0000 7fb608e00700 1 -- 172.21.15.18:0/4098087938 wait complete. 2024-04-12T21:35:37.456 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:37.455+0000 7fb608e00700 1 Processor -- start 2024-04-12T21:35:37.456 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:37.455+0000 7fb608e00700 1 -- start start 2024-04-12T21:35:37.456 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:37.456+0000 7fb608e00700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb6040708a0 0x7fb60411a0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:35:37.456 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:37.456+0000 7fb608e00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb604071f90 0x7fb60411a600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:35:37.456 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:37.456+0000 7fb608e00700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb60411ac50 con 0x7fb604071f90 2024-04-12T21:35:37.456 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:37.456+0000 7fb608e00700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6041140d0 con 0x7fb6040708a0 2024-04-12T21:35:37.456 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:37.456+0000 7fb5fbe00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb604071f90 0x7fb60411a600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:35:37.456 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:37.456+0000 7fb5fbe00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb604071f90 0x7fb60411a600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:33874/0 (socket says 172.21.15.18:33874) 2024-04-12T21:35:37.456 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:37.456+0000 7fb5fbe00700 1 -- 172.21.15.18:0/3717781202 learned_addr learned my addr 172.21.15.18:0/3717781202 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:35:37.456 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:37.456+0000 7fb601e00700 1 --2- 172.21.15.18:0/3717781202 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb6040708a0 0x7fb60411a0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:35:37.456 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:37.457+0000 7fb5fbe00700 1 -- 172.21.15.18:0/3717781202 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb6040708a0 msgr2=0x7fb60411a0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:35:37.456 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:37.457+0000 7fb5fbe00700 1 --2- 172.21.15.18:0/3717781202 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb6040708a0 0x7fb60411a0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:35:37.457 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:37.457+0000 7fb5fbe00700 1 -- 172.21.15.18:0/3717781202 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5ec005040 con 0x7fb604071f90 2024-04-12T21:35:37.457 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:37.457+0000 7fb601e00700 1 --2- 172.21.15.18:0/3717781202 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb6040708a0 0x7fb60411a0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-12T21:35:37.457 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:37.457+0000 7fb5fbe00700 1 --2- 172.21.15.18:0/3717781202 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb604071f90 0x7fb60411a600 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7fb5ec004f50 tx=0x7fb5ec009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:35:37.457 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:37.457+0000 7fb5fa000700 1 -- 172.21.15.18:0/3717781202 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb5ec00e040 con 0x7fb604071f90 2024-04-12T21:35:37.458 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:37.457+0000 7fb5fa000700 1 -- 172.21.15.18:0/3717781202 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb5ec00bc10 con 0x7fb604071f90 2024-04-12T21:35:37.458 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:37.457+0000 7fb5fa000700 1 -- 172.21.15.18:0/3717781202 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb5ec01c630 con 0x7fb604071f90 2024-04-12T21:35:37.458 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:37.457+0000 7fb608e00700 1 -- 172.21.15.18:0/3717781202 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb604114350 con 0x7fb604071f90 2024-04-12T21:35:37.458 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:37.458+0000 7fb608e00700 1 -- 172.21.15.18:0/3717781202 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb604114890 con 0x7fb604071f90 2024-04-12T21:35:37.458 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:37.459+0000 7fb608e00700 1 -- 172.21.15.18:0/3717781202 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb604071aa0 con 0x7fb604071f90 2024-04-12T21:35:37.459 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:37.460+0000 7fb5fa000700 1 -- 172.21.15.18:0/3717781202 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7fb5ec013420 con 0x7fb604071f90 2024-04-12T21:35:37.460 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:37.461+0000 7fb5fa000700 1 --2- 172.21.15.18:0/3717781202 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fb5e806ec10 0x7fb5e80710c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:35:37.460 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:37.461+0000 7fb5fa000700 1 -- 172.21.15.18:0/3717781202 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 6757+0+0 (secure 0 0 0) 0x7fb5ec08fd00 con 0x7fb604071f90 2024-04-12T21:35:37.460 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:37.462+0000 7fb601e00700 1 --2- 172.21.15.18:0/3717781202 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fb5e806ec10 0x7fb5e80710c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:35:37.461 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:37.462+0000 7fb601e00700 1 --2- 172.21.15.18:0/3717781202 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fb5e806ec10 0x7fb5e80710c0 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7fb5f4009fd0 tx=0x7fb5f400b040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:35:37.465 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:37.466+0000 7fb5fa000700 1 -- 172.21.15.18:0/3717781202 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7fb5ec058a00 con 0x7fb604071f90 2024-04-12T21:35:37.715 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:37.716+0000 7fb608e00700 1 -- 172.21.15.18:0/3717781202 --> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fb604110590 con 0x7fb5e806ec10 2024-04-12T21:35:37.717 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:37.718+0000 7fb5fa000700 1 -- 172.21.15.18:0/3717781202 <== mgr.34102 v2:172.21.15.18:6800/3812891919 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7fb604110590 con 0x7fb5e806ec10 2024-04-12T21:35:37.720 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:37.720+0000 7fb608e00700 1 -- 172.21.15.18:0/3717781202 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fb5e806ec10 msgr2=0x7fb5e80710c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:35:37.720 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:37.720+0000 7fb608e00700 1 --2- 172.21.15.18:0/3717781202 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fb5e806ec10 0x7fb5e80710c0 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7fb5f4009fd0 tx=0x7fb5f400b040 comp rx=0 tx=0).stop 2024-04-12T21:35:37.720 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:37.720+0000 7fb608e00700 1 -- 172.21.15.18:0/3717781202 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb604071f90 msgr2=0x7fb60411a600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:35:37.720 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:37.720+0000 7fb608e00700 1 --2- 172.21.15.18:0/3717781202 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb604071f90 0x7fb60411a600 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7fb5ec004f50 tx=0x7fb5ec009e30 comp rx=0 tx=0).stop 2024-04-12T21:35:37.720 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:37.721+0000 7fb608e00700 1 -- 172.21.15.18:0/3717781202 shutdown_connections 2024-04-12T21:35:37.720 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:37.721+0000 7fb608e00700 1 --2- 172.21.15.18:0/3717781202 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fb5e806ec10 0x7fb5e80710c0 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:35:37.720 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:37.721+0000 7fb608e00700 1 --2- 172.21.15.18:0/3717781202 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb6040708a0 0x7fb60411a0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:35:37.720 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:37.721+0000 7fb608e00700 1 --2- 172.21.15.18:0/3717781202 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb604071f90 0x7fb60411a600 unknown :-1 s=CLOSED pgs=907 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:35:37.720 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:37.721+0000 7fb608e00700 1 -- 172.21.15.18:0/3717781202 >> 172.21.15.18:0/3717781202 conn(0x7fb6040f89b0 msgr2=0x7fb6040f9a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:35:37.721 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:37.721+0000 7fb608e00700 1 -- 172.21.15.18:0/3717781202 shutdown_connections 2024-04-12T21:35:37.721 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:37.721+0000 7fb608e00700 1 -- 172.21.15.18:0/3717781202 wait complete. 2024-04-12T21:35:37.879 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:37.879+0000 7f80dcc00700 1 -- 172.21.15.18:0/382287403 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f80d8109430 msgr2=0x7f80d8109800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:35:37.879 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:37.879+0000 7f80dcc00700 1 --2- 172.21.15.18:0/382287403 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f80d8109430 0x7f80d8109800 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x7f80c8004940 tx=0x7f80c8009e30 comp rx=0 tx=0).stop 2024-04-12T21:35:37.879 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:37.879+0000 7f80dcc00700 1 -- 172.21.15.18:0/382287403 shutdown_connections 2024-04-12T21:35:37.879 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:37.879+0000 7f80dcc00700 1 --2- 172.21.15.18:0/382287403 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f80d8109dd0 0x7f80d8103440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:35:37.879 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:37.879+0000 7f80dcc00700 1 --2- 172.21.15.18:0/382287403 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f80d8109430 0x7f80d8109800 unknown :-1 s=CLOSED pgs=908 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:35:37.879 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:37.879+0000 7f80dcc00700 1 -- 172.21.15.18:0/382287403 >> 172.21.15.18:0/382287403 conn(0x7f80d8072c00 msgr2=0x7f80d8075010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:35:37.879 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:37.879+0000 7f80dcc00700 1 -- 172.21.15.18:0/382287403 shutdown_connections 2024-04-12T21:35:37.879 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:37.879+0000 7f80dcc00700 1 -- 172.21.15.18:0/382287403 wait complete. 2024-04-12T21:35:37.880 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:37.880+0000 7f80dcc00700 1 Processor -- start 2024-04-12T21:35:37.881 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:37.880+0000 7f80dcc00700 1 -- start start 2024-04-12T21:35:37.881 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:37.881+0000 7f80dcc00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f80d8109430 0x7f80d8119f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:35:37.881 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:37.881+0000 7f80dcc00700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f80d8109dd0 0x7f80d811a4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:35:37.881 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:37.881+0000 7f80dcc00700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80d811ab20 con 0x7f80d8109430 2024-04-12T21:35:37.881 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:37.881+0000 7f80dcc00700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80d8114100 con 0x7f80d8109dd0 2024-04-12T21:35:37.881 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:37.881+0000 7f80d5e00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f80d8109430 0x7f80d8119f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:35:37.881 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:37.881+0000 7f80d5e00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f80d8109430 0x7f80d8119f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:33882/0 (socket says 172.21.15.18:33882) 2024-04-12T21:35:37.881 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:37.881+0000 7f80d5e00700 1 -- 172.21.15.18:0/3623495182 learned_addr learned my addr 172.21.15.18:0/3623495182 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:35:37.881 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:37.881+0000 7f80d5400700 1 --2- 172.21.15.18:0/3623495182 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f80d8109dd0 0x7f80d811a4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:35:37.881 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:37.882+0000 7f80d5e00700 1 -- 172.21.15.18:0/3623495182 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f80d8109dd0 msgr2=0x7f80d811a4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:35:37.881 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:37.882+0000 7f80d5e00700 1 --2- 172.21.15.18:0/3623495182 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f80d8109dd0 0x7f80d811a4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:35:37.881 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:37.882+0000 7f80d5e00700 1 -- 172.21.15.18:0/3623495182 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80c8005040 con 0x7f80d8109430 2024-04-12T21:35:37.882 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:37.882+0000 7f80d5e00700 1 --2- 172.21.15.18:0/3623495182 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f80d8109430 0x7f80d8119f90 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 crypto rx=0x7f80c800b8b0 tx=0x7f80c8009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:35:37.882 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:37.882+0000 7f80ce000700 1 -- 172.21.15.18:0/3623495182 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f80c800e040 con 0x7f80d8109430 2024-04-12T21:35:37.882 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:37.882+0000 7f80ce000700 1 -- 172.21.15.18:0/3623495182 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f80c8004290 con 0x7f80d8109430 2024-04-12T21:35:37.882 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:37.882+0000 7f80dcc00700 1 -- 172.21.15.18:0/3623495182 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f80d8114380 con 0x7f80d8109430 2024-04-12T21:35:37.882 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:37.882+0000 7f80ce000700 1 -- 172.21.15.18:0/3623495182 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f80c801c3f0 con 0x7f80d8109430 2024-04-12T21:35:37.882 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:37.883+0000 7f80dcc00700 1 -- 172.21.15.18:0/3623495182 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f80d8114940 con 0x7f80d8109430 2024-04-12T21:35:37.883 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:37.884+0000 7f80dcc00700 1 -- 172.21.15.18:0/3623495182 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f80d8104b80 con 0x7f80d8109430 2024-04-12T21:35:37.883 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:37.885+0000 7f80ce000700 1 -- 172.21.15.18:0/3623495182 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7f80c80137b0 con 0x7f80d8109430 2024-04-12T21:35:37.885 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:37.886+0000 7f80ce000700 1 --2- 172.21.15.18:0/3623495182 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f80bc06ec00 0x7f80bc0710b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:35:37.885 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:37.886+0000 7f80ce000700 1 -- 172.21.15.18:0/3623495182 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 6757+0+0 (secure 0 0 0) 0x7f80c808fca0 con 0x7f80d8109430 2024-04-12T21:35:37.886 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:37.887+0000 7f80d5400700 1 --2- 172.21.15.18:0/3623495182 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f80bc06ec00 0x7f80bc0710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:35:37.886 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:37.887+0000 7f80d5400700 1 --2- 172.21.15.18:0/3623495182 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f80bc06ec00 0x7f80bc0710b0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f80c4009fd0 tx=0x7f80c4009f20 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:35:37.890 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:37.891+0000 7f80ce000700 1 -- 172.21.15.18:0/3623495182 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f80c805c4d0 con 0x7f80d8109430 2024-04-12T21:35:38.124 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:38.125+0000 7f80dcc00700 1 -- 172.21.15.18:0/3623495182 --> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f80d8115650 con 0x7f80bc06ec00 2024-04-12T21:35:38.154 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:38.155+0000 7f80ce000700 1 -- 172.21.15.18:0/3623495182 <== mgr.34102 v2:172.21.15.18:6800/3812891919 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7f80d8115650 con 0x7f80bc06ec00 2024-04-12T21:35:38.154 INFO:teuthology.orchestra.run.smithi018.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-12T21:35:38.155 INFO:teuthology.orchestra.run.smithi018.stdout:alertmanager.smithi018 smithi018 *:9093,9094 running (36m) 12m ago 50m 15.5M - 0.25.0 c8568f914cd2 c332090068e8 2024-04-12T21:35:38.155 INFO:teuthology.orchestra.run.smithi018.stdout:crash.smithi018 smithi018 running (32m) 12m ago 51m 6665k - 19.0.0-2879-g324baf8d 8717720dc5e5 b7f80d1f7982 2024-04-12T21:35:38.155 INFO:teuthology.orchestra.run.smithi018.stdout:crash.smithi186 smithi186 running (32m) 12m ago 49m 6660k - 19.0.0-2879-g324baf8d 8717720dc5e5 a4f55c81b623 2024-04-12T21:35:38.155 INFO:teuthology.orchestra.run.smithi018.stdout:grafana.smithi018 smithi018 *:3000 running (35m) 12m ago 50m 74.8M - 9.4.12 2bacad6d85d8 b08e3bf176dd 2024-04-12T21:35:38.155 INFO:teuthology.orchestra.run.smithi018.stdout:mds.cephfs.smithi018.acqbsc smithi018 running (28m) 12m ago 43m 16.7M - 19.0.0-2879-g324baf8d 8717720dc5e5 021f99ab24ae 2024-04-12T21:35:38.155 INFO:teuthology.orchestra.run.smithi018.stdout:mds.cephfs.smithi018.szjluo smithi018 running (28m) 12m ago 43m 16.5M - 19.0.0-2879-g324baf8d 8717720dc5e5 0003117479d4 2024-04-12T21:35:38.155 INFO:teuthology.orchestra.run.smithi018.stdout:mds.cephfs.smithi186.jdllyn smithi186 running (12m) 12m ago 43m 11.0M - 19.0.0-2879-g324baf8d 8717720dc5e5 21b1f46eef23 2024-04-12T21:35:38.155 INFO:teuthology.orchestra.run.smithi018.stdout:mds.cephfs.smithi186.wjnafs smithi186 running (43m) 12m ago 43m 19.5M - 17.2.7-827-gd3b5060a 21de9b7369ad 74b115d6d6fd 2024-04-12T21:35:38.155 INFO:teuthology.orchestra.run.smithi018.stdout:mgr.smithi018.xjzwga smithi018 *:8443,9283,8765 running (38m) 12m ago 52m 512M - 19.0.0-2879-g324baf8d 8717720dc5e5 56088305eab7 2024-04-12T21:35:38.155 INFO:teuthology.orchestra.run.smithi018.stdout:mgr.smithi186.kqcfxk smithi186 *:8443,9283,8765 running (37m) 12m ago 49m 459M - 19.0.0-2879-g324baf8d 8717720dc5e5 09cd766e5e0d 2024-04-12T21:35:38.155 INFO:teuthology.orchestra.run.smithi018.stdout:mon.smithi018 smithi018 running (34m) 12m ago 52m 73.1M 2048M 19.0.0-2879-g324baf8d 8717720dc5e5 df3e201dfed6 2024-04-12T21:35:38.155 INFO:teuthology.orchestra.run.smithi018.stdout:mon.smithi186 smithi186 running (33m) 12m ago 49m 47.2M 2048M 19.0.0-2879-g324baf8d 8717720dc5e5 c77498da9b2b 2024-04-12T21:35:38.155 INFO:teuthology.orchestra.run.smithi018.stdout:node-exporter.smithi018 smithi018 *:9100 running (37m) 12m ago 50m 8108k - 1.5.0 0da6a335fe13 326c3d9c6847 2024-04-12T21:35:38.156 INFO:teuthology.orchestra.run.smithi018.stdout:node-exporter.smithi186 smithi186 *:9100 running (37m) 12m ago 49m 8757k - 1.5.0 0da6a335fe13 ec2112baeecf 2024-04-12T21:35:38.156 INFO:teuthology.orchestra.run.smithi018.stdout:osd.0 smithi018 running (31m) 12m ago 48m 123M 1559M 19.0.0-2879-g324baf8d 8717720dc5e5 60b1273ca047 2024-04-12T21:35:38.156 INFO:teuthology.orchestra.run.smithi018.stdout:osd.1 smithi018 running (31m) 12m ago 47m 95.1M 1559M 19.0.0-2879-g324baf8d 8717720dc5e5 170ce1e5fb8d 2024-04-12T21:35:38.156 INFO:teuthology.orchestra.run.smithi018.stdout:osd.2 smithi018 running (30m) 12m ago 47m 94.3M 1559M 19.0.0-2879-g324baf8d 8717720dc5e5 c2cbc6aa2839 2024-04-12T21:35:38.156 INFO:teuthology.orchestra.run.smithi018.stdout:osd.3 smithi186 running (30m) 12m ago 46m 101M 2581M 19.0.0-2879-g324baf8d 8717720dc5e5 0138e8262e4b 2024-04-12T21:35:38.156 INFO:teuthology.orchestra.run.smithi018.stdout:osd.4 smithi186 running (29m) 12m ago 46m 91.0M 2581M 19.0.0-2879-g324baf8d 8717720dc5e5 b20cb63b25c9 2024-04-12T21:35:38.156 INFO:teuthology.orchestra.run.smithi018.stdout:osd.5 smithi186 running (28m) 12m ago 45m 92.0M 2581M 19.0.0-2879-g324baf8d 8717720dc5e5 2513bea2610e 2024-04-12T21:35:38.156 INFO:teuthology.orchestra.run.smithi018.stdout:prometheus.smithi018 smithi018 *:9095 running (36m) 12m ago 50m 67.9M - 2.43.0 a07b618ecd1d f58841127e21 2024-04-12T21:35:38.157 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:38.158+0000 7f80dcc00700 1 -- 172.21.15.18:0/3623495182 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f80bc06ec00 msgr2=0x7f80bc0710b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:35:38.157 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:38.158+0000 7f80dcc00700 1 --2- 172.21.15.18:0/3623495182 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f80bc06ec00 0x7f80bc0710b0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f80c4009fd0 tx=0x7f80c4009f20 comp rx=0 tx=0).stop 2024-04-12T21:35:38.157 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:38.158+0000 7f80dcc00700 1 -- 172.21.15.18:0/3623495182 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f80d8109430 msgr2=0x7f80d8119f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:35:38.157 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:38.158+0000 7f80dcc00700 1 --2- 172.21.15.18:0/3623495182 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f80d8109430 0x7f80d8119f90 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 crypto rx=0x7f80c800b8b0 tx=0x7f80c8009e30 comp rx=0 tx=0).stop 2024-04-12T21:35:38.158 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:38.158+0000 7f80dcc00700 1 -- 172.21.15.18:0/3623495182 shutdown_connections 2024-04-12T21:35:38.158 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:38.158+0000 7f80dcc00700 1 --2- 172.21.15.18:0/3623495182 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f80bc06ec00 0x7f80bc0710b0 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:35:38.158 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:38.158+0000 7f80dcc00700 1 --2- 172.21.15.18:0/3623495182 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f80d8109430 0x7f80d8119f90 unknown :-1 s=CLOSED pgs=909 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:35:38.158 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:38.158+0000 7f80dcc00700 1 --2- 172.21.15.18:0/3623495182 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f80d8109dd0 0x7f80d811a4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:35:38.158 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:38.158+0000 7f80dcc00700 1 -- 172.21.15.18:0/3623495182 >> 172.21.15.18:0/3623495182 conn(0x7f80d8072c00 msgr2=0x7f80d8074c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:35:38.158 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:38.158+0000 7f80dcc00700 1 -- 172.21.15.18:0/3623495182 shutdown_connections 2024-04-12T21:35:38.158 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:38.158+0000 7f80dcc00700 1 -- 172.21.15.18:0/3623495182 wait complete. 2024-04-12T21:35:38.316 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:38.316+0000 7fca2d800700 1 -- 172.21.15.18:0/2830131824 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fca28071f90 msgr2=0x7fca28070360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:35:38.316 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:38.316+0000 7fca2d800700 1 --2- 172.21.15.18:0/2830131824 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fca28071f90 0x7fca28070360 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 crypto rx=0x7fca18004970 tx=0x7fca18009e30 comp rx=0 tx=0).stop 2024-04-12T21:35:38.316 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:38.316+0000 7fca2d800700 1 -- 172.21.15.18:0/2830131824 shutdown_connections 2024-04-12T21:35:38.316 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:38.316+0000 7fca2d800700 1 --2- 172.21.15.18:0/2830131824 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fca280708a0 0x7fca28070d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:35:38.316 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:38.316+0000 7fca2d800700 1 --2- 172.21.15.18:0/2830131824 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fca28071f90 0x7fca28070360 unknown :-1 s=CLOSED pgs=910 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:35:38.316 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:38.316+0000 7fca2d800700 1 -- 172.21.15.18:0/2830131824 >> 172.21.15.18:0/2830131824 conn(0x7fca280f89b0 msgr2=0x7fca280fadc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:35:38.316 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:38.317+0000 7fca2d800700 1 -- 172.21.15.18:0/2830131824 shutdown_connections 2024-04-12T21:35:38.316 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:38.317+0000 7fca2d800700 1 -- 172.21.15.18:0/2830131824 wait complete. 2024-04-12T21:35:38.317 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:38.317+0000 7fca2d800700 1 Processor -- start 2024-04-12T21:35:38.317 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:38.318+0000 7fca2d800700 1 -- start start 2024-04-12T21:35:38.317 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:38.318+0000 7fca2d800700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fca280708a0 0x7fca2811a1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:35:38.318 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:38.318+0000 7fca2d800700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fca28071f90 0x7fca2811a6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:35:38.318 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:38.318+0000 7fca2d800700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca2811ad30 con 0x7fca280708a0 2024-04-12T21:35:38.318 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:38.318+0000 7fca2d800700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca28114100 con 0x7fca28071f90 2024-04-12T21:35:38.318 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:38.318+0000 7fca26a00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fca280708a0 0x7fca2811a1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:35:38.318 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:38.319+0000 7fca26a00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fca280708a0 0x7fca2811a1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:33898/0 (socket says 172.21.15.18:33898) 2024-04-12T21:35:38.318 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:38.319+0000 7fca26a00700 1 -- 172.21.15.18:0/990250032 learned_addr learned my addr 172.21.15.18:0/990250032 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:35:38.319 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:38.319+0000 7fca26a00700 1 -- 172.21.15.18:0/990250032 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fca28071f90 msgr2=0x7fca2811a6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:35:38.319 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:38.319+0000 7fca26a00700 1 --2- 172.21.15.18:0/990250032 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fca28071f90 0x7fca2811a6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:35:38.319 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:38.319+0000 7fca26a00700 1 -- 172.21.15.18:0/990250032 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca18005040 con 0x7fca280708a0 2024-04-12T21:35:38.319 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:38.319+0000 7fca26a00700 1 --2- 172.21.15.18:0/990250032 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fca280708a0 0x7fca2811a1a0 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 crypto rx=0x7fca18004f50 tx=0x7fca18009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:35:38.319 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:38.319+0000 7fca1ea00700 1 -- 172.21.15.18:0/990250032 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fca1800e040 con 0x7fca280708a0 2024-04-12T21:35:38.319 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:38.320+0000 7fca1ea00700 1 -- 172.21.15.18:0/990250032 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fca18004290 con 0x7fca280708a0 2024-04-12T21:35:38.319 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:38.320+0000 7fca1ea00700 1 -- 172.21.15.18:0/990250032 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fca1801c630 con 0x7fca280708a0 2024-04-12T21:35:38.319 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:38.320+0000 7fca2d800700 1 -- 172.21.15.18:0/990250032 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fca28114380 con 0x7fca280708a0 2024-04-12T21:35:38.319 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:38.320+0000 7fca2d800700 1 -- 172.21.15.18:0/990250032 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fca281148c0 con 0x7fca280708a0 2024-04-12T21:35:38.320 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:38.321+0000 7fca2d800700 1 -- 172.21.15.18:0/990250032 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fca28071aa0 con 0x7fca280708a0 2024-04-12T21:35:38.321 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:38.322+0000 7fca1ea00700 1 -- 172.21.15.18:0/990250032 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7fca1800bc10 con 0x7fca280708a0 2024-04-12T21:35:38.323 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:38.323+0000 7fca1ea00700 1 --2- 172.21.15.18:0/990250032 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fca0c06ec10 0x7fca0c0710c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:35:38.323 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:38.324+0000 7fca1ea00700 1 -- 172.21.15.18:0/990250032 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 6757+0+0 (secure 0 0 0) 0x7fca1808fca0 con 0x7fca280708a0 2024-04-12T21:35:38.323 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:38.324+0000 7fca26000700 1 --2- 172.21.15.18:0/990250032 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fca0c06ec10 0x7fca0c0710c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:35:38.324 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:38.325+0000 7fca26000700 1 --2- 172.21.15.18:0/990250032 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fca0c06ec10 0x7fca0c0710c0 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7fca28115a40 tx=0x7fca14009ce0 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:35:38.328 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:38.329+0000 7fca1ea00700 1 -- 172.21.15.18:0/990250032 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7fca1805c4d0 con 0x7fca280708a0 2024-04-12T21:35:38.663 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:38.664+0000 7fca2d800700 1 -- 172.21.15.18:0/990250032 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fca2804bed0 con 0x7fca280708a0 2024-04-12T21:35:38.665 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:38.666+0000 7fca1ea00700 1 -- 172.21.15.18:0/990250032 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+837 (secure 0 0 0) 0x7fca1805c060 con 0x7fca280708a0 2024-04-12T21:35:38.665 INFO:teuthology.orchestra.run.smithi018.stdout:{ 2024-04-12T21:35:38.665 INFO:teuthology.orchestra.run.smithi018.stdout: "mon": { 2024-04-12T21:35:38.665 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 19.0.0-2879-g324baf8d (324baf8ddb91dea46cac50cb535c306115791355) squid (dev)": 2 2024-04-12T21:35:38.665 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-12T21:35:38.665 INFO:teuthology.orchestra.run.smithi018.stdout: "mgr": { 2024-04-12T21:35:38.665 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 19.0.0-2879-g324baf8d (324baf8ddb91dea46cac50cb535c306115791355) squid (dev)": 2 2024-04-12T21:35:38.665 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-12T21:35:38.666 INFO:teuthology.orchestra.run.smithi018.stdout: "osd": { 2024-04-12T21:35:38.666 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 19.0.0-2879-g324baf8d (324baf8ddb91dea46cac50cb535c306115791355) squid (dev)": 6 2024-04-12T21:35:38.666 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-12T21:35:38.666 INFO:teuthology.orchestra.run.smithi018.stdout: "mds": { 2024-04-12T21:35:38.666 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 17.2.7-827-gd3b5060a (d3b5060a17449edfae9385cdf792afec109fca5d) quincy (stable)": 1, 2024-04-12T21:35:38.666 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 19.0.0-2879-g324baf8d (324baf8ddb91dea46cac50cb535c306115791355) squid (dev)": 3 2024-04-12T21:35:38.666 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-12T21:35:38.666 INFO:teuthology.orchestra.run.smithi018.stdout: "overall": { 2024-04-12T21:35:38.666 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 17.2.7-827-gd3b5060a (d3b5060a17449edfae9385cdf792afec109fca5d) quincy (stable)": 1, 2024-04-12T21:35:38.666 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 19.0.0-2879-g324baf8d (324baf8ddb91dea46cac50cb535c306115791355) squid (dev)": 13 2024-04-12T21:35:38.666 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-04-12T21:35:38.666 INFO:teuthology.orchestra.run.smithi018.stdout:} 2024-04-12T21:35:38.668 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:38.669+0000 7fca2d800700 1 -- 172.21.15.18:0/990250032 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fca0c06ec10 msgr2=0x7fca0c0710c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:35:38.668 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:38.669+0000 7fca2d800700 1 --2- 172.21.15.18:0/990250032 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fca0c06ec10 0x7fca0c0710c0 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7fca28115a40 tx=0x7fca14009ce0 comp rx=0 tx=0).stop 2024-04-12T21:35:38.668 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:38.669+0000 7fca2d800700 1 -- 172.21.15.18:0/990250032 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fca280708a0 msgr2=0x7fca2811a1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:35:38.669 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:38.669+0000 7fca2d800700 1 --2- 172.21.15.18:0/990250032 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fca280708a0 0x7fca2811a1a0 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 crypto rx=0x7fca18004f50 tx=0x7fca18009e30 comp rx=0 tx=0).stop 2024-04-12T21:35:38.669 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:38.669+0000 7fca2d800700 1 -- 172.21.15.18:0/990250032 shutdown_connections 2024-04-12T21:35:38.669 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:38.669+0000 7fca2d800700 1 --2- 172.21.15.18:0/990250032 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fca0c06ec10 0x7fca0c0710c0 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:35:38.669 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:38.669+0000 7fca2d800700 1 --2- 172.21.15.18:0/990250032 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fca280708a0 0x7fca2811a1a0 unknown :-1 s=CLOSED pgs=911 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:35:38.669 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:38.669+0000 7fca2d800700 1 --2- 172.21.15.18:0/990250032 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fca28071f90 0x7fca2811a6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:35:38.669 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:38.669+0000 7fca2d800700 1 -- 172.21.15.18:0/990250032 >> 172.21.15.18:0/990250032 conn(0x7fca280f89b0 msgr2=0x7fca280f9aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:35:38.669 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:38.669+0000 7fca2d800700 1 -- 172.21.15.18:0/990250032 shutdown_connections 2024-04-12T21:35:38.669 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:38.669+0000 7fca2d800700 1 -- 172.21.15.18:0/990250032 wait complete. 2024-04-12T21:35:38.826 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:38.827+0000 7fce40200700 1 -- 172.21.15.18:0/2300187796 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fce38109430 msgr2=0x7fce38109800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:35:38.827 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:38.827+0000 7fce40200700 1 --2- 172.21.15.18:0/2300187796 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fce38109430 0x7fce38109800 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 crypto rx=0x7fce2c004940 tx=0x7fce2c009e30 comp rx=0 tx=0).stop 2024-04-12T21:35:38.827 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:38.827+0000 7fce40200700 1 -- 172.21.15.18:0/2300187796 shutdown_connections 2024-04-12T21:35:38.827 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:38.827+0000 7fce40200700 1 --2- 172.21.15.18:0/2300187796 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fce38109dd0 0x7fce38103440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:35:38.827 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:38.827+0000 7fce40200700 1 --2- 172.21.15.18:0/2300187796 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fce38109430 0x7fce38109800 unknown :-1 s=CLOSED pgs=912 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:35:38.827 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:38.827+0000 7fce40200700 1 -- 172.21.15.18:0/2300187796 >> 172.21.15.18:0/2300187796 conn(0x7fce38072c00 msgr2=0x7fce38075010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:35:38.827 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:38.827+0000 7fce40200700 1 -- 172.21.15.18:0/2300187796 shutdown_connections 2024-04-12T21:35:38.827 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:38.827+0000 7fce40200700 1 -- 172.21.15.18:0/2300187796 wait complete. 2024-04-12T21:35:38.828 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:38.828+0000 7fce40200700 1 Processor -- start 2024-04-12T21:35:38.828 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:38.828+0000 7fce40200700 1 -- start start 2024-04-12T21:35:38.829 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:38.829+0000 7fce40200700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fce38109430 0x7fce38119f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:35:38.829 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:38.829+0000 7fce40200700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fce38109dd0 0x7fce3811a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:35:38.829 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:38.829+0000 7fce40200700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce3811aad0 con 0x7fce38109dd0 2024-04-12T21:35:38.829 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:38.829+0000 7fce40200700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce38114100 con 0x7fce38109430 2024-04-12T21:35:38.829 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:38.829+0000 7fce3ce00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fce38109dd0 0x7fce3811a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:35:38.829 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:38.829+0000 7fce3ce00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fce38109dd0 0x7fce3811a480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:33906/0 (socket says 172.21.15.18:33906) 2024-04-12T21:35:38.829 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:38.829+0000 7fce3ce00700 1 -- 172.21.15.18:0/3463027210 learned_addr learned my addr 172.21.15.18:0/3463027210 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:35:38.829 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:38.829+0000 7fce3d800700 1 --2- 172.21.15.18:0/3463027210 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fce38109430 0x7fce38119f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:35:38.829 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:38.830+0000 7fce3ce00700 1 -- 172.21.15.18:0/3463027210 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fce38109430 msgr2=0x7fce38119f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:35:38.829 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:38.830+0000 7fce3ce00700 1 --2- 172.21.15.18:0/3463027210 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fce38109430 0x7fce38119f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:35:38.829 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:38.830+0000 7fce3ce00700 1 -- 172.21.15.18:0/3463027210 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce2c005040 con 0x7fce38109dd0 2024-04-12T21:35:38.829 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:38.830+0000 7fce3d800700 1 --2- 172.21.15.18:0/3463027210 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fce38109430 0x7fce38119f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-12T21:35:38.830 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:38.830+0000 7fce3ce00700 1 --2- 172.21.15.18:0/3463027210 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fce38109dd0 0x7fce3811a480 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 crypto rx=0x7fce2400c890 tx=0x7fce240044c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:35:38.830 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:38.830+0000 7fce35600700 1 -- 172.21.15.18:0/3463027210 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fce2400a7d0 con 0x7fce38109dd0 2024-04-12T21:35:38.830 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:38.830+0000 7fce35600700 1 -- 172.21.15.18:0/3463027210 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fce2400ae00 con 0x7fce38109dd0 2024-04-12T21:35:38.830 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:38.830+0000 7fce40200700 1 -- 172.21.15.18:0/3463027210 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fce381143e0 con 0x7fce38109dd0 2024-04-12T21:35:38.830 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:38.830+0000 7fce35600700 1 -- 172.21.15.18:0/3463027210 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fce240113a0 con 0x7fce38109dd0 2024-04-12T21:35:38.830 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:38.830+0000 7fce40200700 1 -- 172.21.15.18:0/3463027210 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fce381149a0 con 0x7fce38109dd0 2024-04-12T21:35:38.831 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:38.831+0000 7fce40200700 1 -- 172.21.15.18:0/3463027210 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fce38104b80 con 0x7fce38109dd0 2024-04-12T21:35:38.831 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:38.833+0000 7fce35600700 1 -- 172.21.15.18:0/3463027210 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7fce24012400 con 0x7fce38109dd0 2024-04-12T21:35:38.833 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:38.834+0000 7fce35600700 1 --2- 172.21.15.18:0/3463027210 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fce2006ec00 0x7fce200710b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:35:38.833 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:38.834+0000 7fce35600700 1 -- 172.21.15.18:0/3463027210 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 6757+0+0 (secure 0 0 0) 0x7fce2408e980 con 0x7fce38109dd0 2024-04-12T21:35:38.833 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:38.834+0000 7fce3d800700 1 --2- 172.21.15.18:0/3463027210 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fce2006ec00 0x7fce200710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:35:38.834 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:38.835+0000 7fce3d800700 1 --2- 172.21.15.18:0/3463027210 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fce2006ec00 0x7fce200710b0 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7fce2c00b3b0 tx=0x7fce2c009e30 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:35:38.837 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:38.838+0000 7fce35600700 1 -- 172.21.15.18:0/3463027210 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7fce2405b270 con 0x7fce38109dd0 2024-04-12T21:35:39.103 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:35:38 smithi018 ceph-mon[112590]: from='client.35876 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:35:39.103 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:35:38 smithi018 ceph-mon[112590]: from='client.35880 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:35:39.103 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:35:38 smithi018 ceph-mon[112590]: pgmap v1076: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:35:39.104 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:35:38 smithi018 ceph-mon[112590]: from='client.? 172.21.15.18:0/990250032' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T21:35:39.105 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:39.106+0000 7fce40200700 1 -- 172.21.15.18:0/3463027210 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fce38110520 con 0x7fce38109dd0 2024-04-12T21:35:39.106 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:39.107+0000 7fce35600700 1 -- 172.21.15.18:0/3463027210 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 31 v31) v1 ==== 76+0+1873 (secure 0 0 0) 0x7fce2405ae00 con 0x7fce38109dd0 2024-04-12T21:35:39.107 INFO:teuthology.orchestra.run.smithi018.stdout:e31 2024-04-12T21:35:39.107 INFO:teuthology.orchestra.run.smithi018.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-12T21:35:39.107 INFO:teuthology.orchestra.run.smithi018.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-12T21:35:39.107 INFO:teuthology.orchestra.run.smithi018.stdout:legacy client fscid: 1 2024-04-12T21:35:39.107 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-12T21:35:39.107 INFO:teuthology.orchestra.run.smithi018.stdout:Filesystem 'cephfs' (1) 2024-04-12T21:35:39.107 INFO:teuthology.orchestra.run.smithi018.stdout:fs_name cephfs 2024-04-12T21:35:39.107 INFO:teuthology.orchestra.run.smithi018.stdout:epoch 24 2024-04-12T21:35:39.107 INFO:teuthology.orchestra.run.smithi018.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-04-12T21:35:39.107 INFO:teuthology.orchestra.run.smithi018.stdout:created 2024-04-12T20:51:57.362850+0000 2024-04-12T21:35:39.108 INFO:teuthology.orchestra.run.smithi018.stdout:modified 2024-04-12T21:06:59.717190+0000 2024-04-12T21:35:39.108 INFO:teuthology.orchestra.run.smithi018.stdout:tableserver 0 2024-04-12T21:35:39.108 INFO:teuthology.orchestra.run.smithi018.stdout:root 0 2024-04-12T21:35:39.108 INFO:teuthology.orchestra.run.smithi018.stdout:session_timeout 60 2024-04-12T21:35:39.108 INFO:teuthology.orchestra.run.smithi018.stdout:session_autoclose 300 2024-04-12T21:35:39.108 INFO:teuthology.orchestra.run.smithi018.stdout:max_file_size 1099511627776 2024-04-12T21:35:39.108 INFO:teuthology.orchestra.run.smithi018.stdout:max_xattr_size 65536 2024-04-12T21:35:39.108 INFO:teuthology.orchestra.run.smithi018.stdout:required_client_features {} 2024-04-12T21:35:39.108 INFO:teuthology.orchestra.run.smithi018.stdout:last_failure 0 2024-04-12T21:35:39.108 INFO:teuthology.orchestra.run.smithi018.stdout:last_failure_osd_epoch 100 2024-04-12T21:35:39.108 INFO:teuthology.orchestra.run.smithi018.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-12T21:35:39.108 INFO:teuthology.orchestra.run.smithi018.stdout:max_mds 2 2024-04-12T21:35:39.108 INFO:teuthology.orchestra.run.smithi018.stdout:in 0,1 2024-04-12T21:35:39.108 INFO:teuthology.orchestra.run.smithi018.stdout:up {} 2024-04-12T21:35:39.109 INFO:teuthology.orchestra.run.smithi018.stdout:failed 0,1 2024-04-12T21:35:39.109 INFO:teuthology.orchestra.run.smithi018.stdout:damaged 2024-04-12T21:35:39.109 INFO:teuthology.orchestra.run.smithi018.stdout:stopped 2024-04-12T21:35:39.109 INFO:teuthology.orchestra.run.smithi018.stdout:data_pools [3] 2024-04-12T21:35:39.109 INFO:teuthology.orchestra.run.smithi018.stdout:metadata_pool 2 2024-04-12T21:35:39.109 INFO:teuthology.orchestra.run.smithi018.stdout:inline_data disabled 2024-04-12T21:35:39.109 INFO:teuthology.orchestra.run.smithi018.stdout:balancer 2024-04-12T21:35:39.109 INFO:teuthology.orchestra.run.smithi018.stdout:bal_rank_mask -1 2024-04-12T21:35:39.109 INFO:teuthology.orchestra.run.smithi018.stdout:standby_count_wanted 1 2024-04-12T21:35:39.109 INFO:teuthology.orchestra.run.smithi018.stdout:qdb_cluster leader: 0 members: 2024-04-12T21:35:39.109 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-12T21:35:39.109 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-12T21:35:39.109 INFO:teuthology.orchestra.run.smithi018.stdout:Standby daemons: 2024-04-12T21:35:39.110 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-12T21:35:39.110 INFO:teuthology.orchestra.run.smithi018.stdout:[mds.cephfs.smithi186.wjnafs{-1:24277} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.186:6826/438347207,v1:172.21.15.186:6827/438347207] compat {c=[1],r=[1],i=[7ff]}] 2024-04-12T21:35:39.110 INFO:teuthology.orchestra.run.smithi018.stdout:[mds.cephfs.smithi018.acqbsc{-1:34436} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.18:6828/755179764,v1:172.21.15.18:6829/755179764] compat {c=[1],r=[1],i=[1fff]}] 2024-04-12T21:35:39.110 INFO:teuthology.orchestra.run.smithi018.stdout:[mds.cephfs.smithi018.szjluo{-1:34468} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.18:6826/2407992653,v1:172.21.15.18:6827/2407992653] compat {c=[1],r=[1],i=[1fff]}] 2024-04-12T21:35:39.110 INFO:teuthology.orchestra.run.smithi018.stdout:[mds.cephfs.smithi186.jdllyn{-1:44595} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.186:6824/3523444202,v1:172.21.15.186:6825/3523444202] compat {c=[1],r=[1],i=[1fff]}] 2024-04-12T21:35:39.111 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:39.110+0000 7fce40200700 1 -- 172.21.15.18:0/3463027210 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fce2006ec00 msgr2=0x7fce200710b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:35:39.111 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:39.110+0000 7fce40200700 1 --2- 172.21.15.18:0/3463027210 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fce2006ec00 0x7fce200710b0 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7fce2c00b3b0 tx=0x7fce2c009e30 comp rx=0 tx=0).stop 2024-04-12T21:35:39.111 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:39.110+0000 7fce40200700 1 -- 172.21.15.18:0/3463027210 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fce38109dd0 msgr2=0x7fce3811a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:35:39.111 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:39.110+0000 7fce40200700 1 --2- 172.21.15.18:0/3463027210 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fce38109dd0 0x7fce3811a480 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 crypto rx=0x7fce2400c890 tx=0x7fce240044c0 comp rx=0 tx=0).stop 2024-04-12T21:35:39.111 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:39.110+0000 7fce40200700 1 -- 172.21.15.18:0/3463027210 shutdown_connections 2024-04-12T21:35:39.111 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:39.110+0000 7fce40200700 1 --2- 172.21.15.18:0/3463027210 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fce2006ec00 0x7fce200710b0 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:35:39.111 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:39.110+0000 7fce40200700 1 --2- 172.21.15.18:0/3463027210 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fce38109430 0x7fce38119f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:35:39.111 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:39.110+0000 7fce40200700 1 --2- 172.21.15.18:0/3463027210 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fce38109dd0 0x7fce3811a480 unknown :-1 s=CLOSED pgs=913 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:35:39.111 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:39.110+0000 7fce40200700 1 -- 172.21.15.18:0/3463027210 >> 172.21.15.18:0/3463027210 conn(0x7fce38072c00 msgr2=0x7fce38074c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:35:39.111 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:39.111+0000 7fce40200700 1 -- 172.21.15.18:0/3463027210 shutdown_connections 2024-04-12T21:35:39.112 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:39.111+0000 7fce40200700 1 -- 172.21.15.18:0/3463027210 wait complete. 2024-04-12T21:35:39.112 INFO:teuthology.orchestra.run.smithi018.stderr:dumped fsmap epoch 31 2024-04-12T21:35:39.257 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:39.258+0000 7f8565800700 1 -- 172.21.15.18:0/3642117030 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8560109c40 msgr2=0x7f85601032b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:35:39.257 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:39.258+0000 7f8565800700 1 --2- 172.21.15.18:0/3642117030 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8560109c40 0x7f85601032b0 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 crypto rx=0x7f854c004940 tx=0x7f854c009e30 comp rx=0 tx=0).stop 2024-04-12T21:35:39.257 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:39.258+0000 7f8565800700 1 -- 172.21.15.18:0/3642117030 shutdown_connections 2024-04-12T21:35:39.257 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:39.258+0000 7f8565800700 1 --2- 172.21.15.18:0/3642117030 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8560109c40 0x7f85601032b0 unknown :-1 s=CLOSED pgs=914 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:35:39.257 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:39.258+0000 7f8565800700 1 --2- 172.21.15.18:0/3642117030 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f85601092a0 0x7f8560109670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:35:39.258 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:39.258+0000 7f8565800700 1 -- 172.21.15.18:0/3642117030 >> 172.21.15.18:0/3642117030 conn(0x7f8560072a30 msgr2=0x7f8560074e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:35:39.258 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:39.258+0000 7f8565800700 1 -- 172.21.15.18:0/3642117030 shutdown_connections 2024-04-12T21:35:39.258 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:39.258+0000 7f8565800700 1 -- 172.21.15.18:0/3642117030 wait complete. 2024-04-12T21:35:39.259 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:39.259+0000 7f8565800700 1 Processor -- start 2024-04-12T21:35:39.259 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:39.259+0000 7f8565800700 1 -- start start 2024-04-12T21:35:39.259 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:39.259+0000 7f8565800700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f85601092a0 0x7f8560119fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:35:39.259 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:39.260+0000 7f8565800700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8560109c40 0x7f856011a500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:35:39.259 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:39.260+0000 7f8565800700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f856011ab50 con 0x7f85601092a0 2024-04-12T21:35:39.259 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:39.260+0000 7f8565800700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8560113f20 con 0x7f8560109c40 2024-04-12T21:35:39.259 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:39.260+0000 7f855fe00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f85601092a0 0x7f8560119fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:35:39.259 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:39.260+0000 7f855fe00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f85601092a0 0x7f8560119fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:33916/0 (socket says 172.21.15.18:33916) 2024-04-12T21:35:39.259 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:39.260+0000 7f855fe00700 1 -- 172.21.15.18:0/3561282802 learned_addr learned my addr 172.21.15.18:0/3561282802 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:35:39.260 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:39.260+0000 7f855fe00700 1 -- 172.21.15.18:0/3561282802 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8560109c40 msgr2=0x7f856011a500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:35:39.260 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:39.260+0000 7f855fe00700 1 --2- 172.21.15.18:0/3561282802 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8560109c40 0x7f856011a500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:35:39.260 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:39.260+0000 7f855fe00700 1 -- 172.21.15.18:0/3561282802 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f854c005040 con 0x7f85601092a0 2024-04-12T21:35:39.260 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:39.261+0000 7f855fe00700 1 --2- 172.21.15.18:0/3561282802 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f85601092a0 0x7f8560119fc0 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 crypto rx=0x7f855000c830 tx=0x7f8550004470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:35:39.261 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:39.261+0000 7f8557e00700 1 -- 172.21.15.18:0/3561282802 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f855000a760 con 0x7f85601092a0 2024-04-12T21:35:39.261 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:39.261+0000 7f8557e00700 1 -- 172.21.15.18:0/3561282802 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f855000ad90 con 0x7f85601092a0 2024-04-12T21:35:39.261 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:39.261+0000 7f8565800700 1 -- 172.21.15.18:0/3561282802 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8560114200 con 0x7f85601092a0 2024-04-12T21:35:39.261 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:39.261+0000 7f8557e00700 1 -- 172.21.15.18:0/3561282802 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f85500113a0 con 0x7f85601092a0 2024-04-12T21:35:39.261 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:39.261+0000 7f8565800700 1 -- 172.21.15.18:0/3561282802 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f85601147c0 con 0x7f85601092a0 2024-04-12T21:35:39.261 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:39.262+0000 7f8565800700 1 -- 172.21.15.18:0/3561282802 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f85601049f0 con 0x7f85601092a0 2024-04-12T21:35:39.262 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:39.263+0000 7f8557e00700 1 -- 172.21.15.18:0/3561282802 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7f8550012400 con 0x7f85601092a0 2024-04-12T21:35:39.263 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:39.264+0000 7f8557e00700 1 --2- 172.21.15.18:0/3561282802 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f854406ec00 0x7f85440710b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:35:39.263 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:39.264+0000 7f8557e00700 1 -- 172.21.15.18:0/3561282802 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 6757+0+0 (secure 0 0 0) 0x7f855008e9c0 con 0x7f85601092a0 2024-04-12T21:35:39.263 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:39.264+0000 7f855f400700 1 --2- 172.21.15.18:0/3561282802 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f854406ec00 0x7f85440710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:35:39.264 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:39.265+0000 7f855f400700 1 --2- 172.21.15.18:0/3561282802 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f854406ec00 0x7f85440710b0 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f854c004f90 tx=0x7f854c009e30 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:35:39.268 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:39.269+0000 7f8557e00700 1 -- 172.21.15.18:0/3561282802 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f855005b1f0 con 0x7f85601092a0 2024-04-12T21:35:39.282 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:35:38 smithi186 ceph-mon[86379]: from='client.35876 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:35:39.282 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:35:38 smithi186 ceph-mon[86379]: from='client.35880 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:35:39.283 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:35:38 smithi186 ceph-mon[86379]: pgmap v1076: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:35:39.283 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:35:38 smithi186 ceph-mon[86379]: from='client.? 172.21.15.18:0/990250032' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T21:35:39.507 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:39.508+0000 7f8565800700 1 -- 172.21.15.18:0/3561282802 --> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f8560110370 con 0x7f854406ec00 2024-04-12T21:35:39.509 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:39.510+0000 7f8557e00700 1 -- 172.21.15.18:0/3561282802 <== mgr.34102 v2:172.21.15.18:6800/3812891919 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f8560110370 con 0x7f854406ec00 2024-04-12T21:35:39.509 INFO:teuthology.orchestra.run.smithi018.stdout:{ 2024-04-12T21:35:39.509 INFO:teuthology.orchestra.run.smithi018.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:f4452def3be4d350f4a8e5feaf7ebb1511be9298fe68478ba7a2d6fbd7d69a63", 2024-04-12T21:35:39.509 INFO:teuthology.orchestra.run.smithi018.stdout: "in_progress": true, 2024-04-12T21:35:39.509 INFO:teuthology.orchestra.run.smithi018.stdout: "which": "Upgrading all daemon types on all hosts", 2024-04-12T21:35:39.509 INFO:teuthology.orchestra.run.smithi018.stdout: "services_complete": [ 2024-04-12T21:35:39.510 INFO:teuthology.orchestra.run.smithi018.stdout: "crash", 2024-04-12T21:35:39.510 INFO:teuthology.orchestra.run.smithi018.stdout: "osd", 2024-04-12T21:35:39.510 INFO:teuthology.orchestra.run.smithi018.stdout: "mon", 2024-04-12T21:35:39.510 INFO:teuthology.orchestra.run.smithi018.stdout: "mgr" 2024-04-12T21:35:39.510 INFO:teuthology.orchestra.run.smithi018.stdout: ], 2024-04-12T21:35:39.510 INFO:teuthology.orchestra.run.smithi018.stdout: "progress": "15/21 daemons upgraded", 2024-04-12T21:35:39.510 INFO:teuthology.orchestra.run.smithi018.stdout: "message": "Currently upgrading mds daemons", 2024-04-12T21:35:39.510 INFO:teuthology.orchestra.run.smithi018.stdout: "is_paused": false 2024-04-12T21:35:39.510 INFO:teuthology.orchestra.run.smithi018.stdout:} 2024-04-12T21:35:39.512 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:39.512+0000 7f8565800700 1 -- 172.21.15.18:0/3561282802 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f854406ec00 msgr2=0x7f85440710b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:35:39.512 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:39.512+0000 7f8565800700 1 --2- 172.21.15.18:0/3561282802 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f854406ec00 0x7f85440710b0 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f854c004f90 tx=0x7f854c009e30 comp rx=0 tx=0).stop 2024-04-12T21:35:39.512 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:39.512+0000 7f8565800700 1 -- 172.21.15.18:0/3561282802 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f85601092a0 msgr2=0x7f8560119fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:35:39.512 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:39.512+0000 7f8565800700 1 --2- 172.21.15.18:0/3561282802 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f85601092a0 0x7f8560119fc0 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 crypto rx=0x7f855000c830 tx=0x7f8550004470 comp rx=0 tx=0).stop 2024-04-12T21:35:39.512 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:39.513+0000 7f8565800700 1 -- 172.21.15.18:0/3561282802 shutdown_connections 2024-04-12T21:35:39.512 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:39.513+0000 7f8565800700 1 --2- 172.21.15.18:0/3561282802 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f854406ec00 0x7f85440710b0 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:35:39.512 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:39.513+0000 7f8565800700 1 --2- 172.21.15.18:0/3561282802 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f85601092a0 0x7f8560119fc0 unknown :-1 s=CLOSED pgs=915 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:35:39.512 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:39.513+0000 7f8565800700 1 --2- 172.21.15.18:0/3561282802 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8560109c40 0x7f856011a500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:35:39.512 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:39.513+0000 7f8565800700 1 -- 172.21.15.18:0/3561282802 >> 172.21.15.18:0/3561282802 conn(0x7f8560072a30 msgr2=0x7f8560074910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:35:39.512 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:39.513+0000 7f8565800700 1 -- 172.21.15.18:0/3561282802 shutdown_connections 2024-04-12T21:35:39.513 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:39.513+0000 7f8565800700 1 -- 172.21.15.18:0/3561282802 wait complete. 2024-04-12T21:35:39.669 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:39.670+0000 7f3111600700 1 -- 172.21.15.18:0/3400554869 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f310c105020 msgr2=0x7f310c1053f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:35:39.670 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:39.670+0000 7f3111600700 1 --2- 172.21.15.18:0/3400554869 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f310c105020 0x7f310c1053f0 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 crypto rx=0x7f30fc004940 tx=0x7f30fc009e30 comp rx=0 tx=0).stop 2024-04-12T21:35:39.670 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:39.670+0000 7f3111600700 1 -- 172.21.15.18:0/3400554869 shutdown_connections 2024-04-12T21:35:39.670 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:39.670+0000 7f3111600700 1 --2- 172.21.15.18:0/3400554869 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f310c1059c0 0x7f310c0ff080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:35:39.670 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:39.670+0000 7f3111600700 1 --2- 172.21.15.18:0/3400554869 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f310c105020 0x7f310c1053f0 unknown :-1 s=CLOSED pgs=916 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:35:39.670 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:39.670+0000 7f3111600700 1 -- 172.21.15.18:0/3400554869 >> 172.21.15.18:0/3400554869 conn(0x7f310c0faba0 msgr2=0x7f310c0fcfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:35:39.670 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:39.670+0000 7f3111600700 1 -- 172.21.15.18:0/3400554869 shutdown_connections 2024-04-12T21:35:39.670 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:39.670+0000 7f3111600700 1 -- 172.21.15.18:0/3400554869 wait complete. 2024-04-12T21:35:39.671 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:39.671+0000 7f3111600700 1 Processor -- start 2024-04-12T21:35:39.671 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:39.671+0000 7f3111600700 1 -- start start 2024-04-12T21:35:39.671 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:39.672+0000 7f3111600700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f310c105020 0x7f310c115b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:35:39.671 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:39.672+0000 7f3111600700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f310c1059c0 0x7f310c1160d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:35:39.672 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:39.672+0000 7f3111600700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f310c116720 con 0x7f310c105020 2024-04-12T21:35:39.672 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:39.672+0000 7f3111600700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f310c10fd00 con 0x7f310c1059c0 2024-04-12T21:35:39.672 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:39.672+0000 7f310aa00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f310c105020 0x7f310c115b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:35:39.672 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:39.672+0000 7f310aa00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f310c105020 0x7f310c115b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:33940/0 (socket says 172.21.15.18:33940) 2024-04-12T21:35:39.672 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:39.672+0000 7f310aa00700 1 -- 172.21.15.18:0/4078075917 learned_addr learned my addr 172.21.15.18:0/4078075917 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:35:39.672 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:39.673+0000 7f310a000700 1 --2- 172.21.15.18:0/4078075917 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f310c1059c0 0x7f310c1160d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:35:39.673 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:39.673+0000 7f310aa00700 1 -- 172.21.15.18:0/4078075917 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f310c1059c0 msgr2=0x7f310c1160d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:35:39.673 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:39.673+0000 7f310aa00700 1 --2- 172.21.15.18:0/4078075917 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f310c1059c0 0x7f310c1160d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:35:39.673 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:39.673+0000 7f310aa00700 1 -- 172.21.15.18:0/4078075917 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30fc005040 con 0x7f310c105020 2024-04-12T21:35:39.673 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:39.673+0000 7f310aa00700 1 --2- 172.21.15.18:0/4078075917 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f310c105020 0x7f310c115b90 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 crypto rx=0x7f30fc009150 tx=0x7f30fc009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:35:39.673 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:39.673+0000 7f3102a00700 1 -- 172.21.15.18:0/4078075917 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f30fc00e040 con 0x7f310c105020 2024-04-12T21:35:39.673 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:39.673+0000 7f3102a00700 1 -- 172.21.15.18:0/4078075917 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f30fc013420 con 0x7f310c105020 2024-04-12T21:35:39.673 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:39.673+0000 7f3102a00700 1 -- 172.21.15.18:0/4078075917 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f30fc01c630 con 0x7f310c105020 2024-04-12T21:35:39.674 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:39.673+0000 7f3111600700 1 -- 172.21.15.18:0/4078075917 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f310c10ff80 con 0x7f310c105020 2024-04-12T21:35:39.674 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:39.674+0000 7f3111600700 1 -- 172.21.15.18:0/4078075917 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f310c110540 con 0x7f310c105020 2024-04-12T21:35:39.674 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:39.675+0000 7f3111600700 1 -- 172.21.15.18:0/4078075917 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f310c100770 con 0x7f310c105020 2024-04-12T21:35:39.675 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:39.676+0000 7f3102a00700 1 -- 172.21.15.18:0/4078075917 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7f30fc013590 con 0x7f310c105020 2024-04-12T21:35:39.676 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:39.677+0000 7f3102a00700 1 --2- 172.21.15.18:0/4078075917 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f30f006ec00 0x7f30f00710b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:35:39.676 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:39.677+0000 7f3102a00700 1 -- 172.21.15.18:0/4078075917 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 6757+0+0 (secure 0 0 0) 0x7f30fc08fc00 con 0x7f310c105020 2024-04-12T21:35:39.676 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:39.677+0000 7f310a000700 1 --2- 172.21.15.18:0/4078075917 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f30f006ec00 0x7f30f00710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:35:39.677 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:39.678+0000 7f310a000700 1 --2- 172.21.15.18:0/4078075917 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f30f006ec00 0x7f30f00710b0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f310c111430 tx=0x7f30f8009ee0 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:35:39.681 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:39.682+0000 7f3102a00700 1 -- 172.21.15.18:0/4078075917 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f30fc05c430 con 0x7f310c105020 2024-04-12T21:35:40.001 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:39.999+0000 7f3111600700 1 -- 172.21.15.18:0/4078075917 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f310c1111e0 con 0x7f310c105020 2024-04-12T21:35:40.002 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:40.000+0000 7f3102a00700 1 -- 172.21.15.18:0/4078075917 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+4303 (secure 0 0 0) 0x7f30fc05bfc0 con 0x7f310c105020 2024-04-12T21:35:40.002 INFO:teuthology.orchestra.run.smithi018.stdout:HEALTH_ERR failed to probe daemons or devices; 1 filesystem is degraded; 1 filesystem is offline 2024-04-12T21:35:40.002 INFO:teuthology.orchestra.run.smithi018.stdout:[WRN] CEPHADM_REFRESH_FAILED: failed to probe daemons or devices 2024-04-12T21:35:40.002 INFO:teuthology.orchestra.run.smithi018.stdout: host smithi018 `cephadm ceph-volume` failed: cephadm exited with an error code: 1, stderr: Inferring config /var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/mon.smithi018/config 2024-04-12T21:35:40.002 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/podman: timeout after 895 seconds 2024-04-12T21:35:40.002 INFO:teuthology.orchestra.run.smithi018.stdout:Non-zero exit code 124 from /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:f4452def3be4d350f4a8e5feaf7ebb1511be9298fe68478ba7a2d6fbd7d69a63 -e NODE_NAME=smithi018 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968:/var/run/ceph:z -v /var/log/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968:/var/log/ceph:z -v /var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmpxsmwkkuh:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:f4452def3be4d350f4a8e5feaf7ebb1511be9298fe68478ba7a2d6fbd7d69a63 inventory --format=json-pretty --filter-for-batch 2024-04-12T21:35:40.002 INFO:teuthology.orchestra.run.smithi018.stdout:Traceback (most recent call last): 2024-04-12T21:35:40.002 INFO:teuthology.orchestra.run.smithi018.stdout: File "/usr/lib64/python3.9/runpy.py", line 197, in _run_module_as_main 2024-04-12T21:35:40.003 INFO:teuthology.orchestra.run.smithi018.stdout: return _run_code(code, main_globals, None, 2024-04-12T21:35:40.003 INFO:teuthology.orchestra.run.smithi018.stdout: File "/usr/lib64/python3.9/runpy.py", line 87, in _run_code 2024-04-12T21:35:40.003 INFO:teuthology.orchestra.run.smithi018.stdout: exec(code, run_globals) 2024-04-12T21:35:40.003 INFO:teuthology.orchestra.run.smithi018.stdout: File "/var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/cephadm.e44f3ed5e34e4ef52eedef233d1190bff004470d8e8005127b5312bff357fa7f/__main__.py", line 5618, in 2024-04-12T21:35:40.003 INFO:teuthology.orchestra.run.smithi018.stdout: File "/var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/cephadm.e44f3ed5e34e4ef52eedef233d1190bff004470d8e8005127b5312bff357fa7f/__main__.py", line 5606, in main 2024-04-12T21:35:40.003 INFO:teuthology.orchestra.run.smithi018.stdout: File "/var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/cephadm.e44f3ed5e34e4ef52eedef233d1190bff004470d8e8005127b5312bff357fa7f/__main__.py", line 408, in _infer_config 2024-04-12T21:35:40.003 INFO:teuthology.orchestra.run.smithi018.stdout: File "/var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/cephadm.e44f3ed5e34e4ef52eedef233d1190bff004470d8e8005127b5312bff357fa7f/__main__.py", line 323, in _infer_fsid 2024-04-12T21:35:40.003 INFO:teuthology.orchestra.run.smithi018.stdout: File "/var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/cephadm.e44f3ed5e34e4ef52eedef233d1190bff004470d8e8005127b5312bff357fa7f/__main__.py", line 436, in _infer_image 2024-04-12T21:35:40.003 INFO:teuthology.orchestra.run.smithi018.stdout: File "/var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/cephadm.e44f3ed5e34e4ef52eedef233d1190bff004470d8e8005127b5312bff357fa7f/__main__.py", line 310, in _validate_fsid 2024-04-12T21:35:40.003 INFO:teuthology.orchestra.run.smithi018.stdout: File "/var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/cephadm.e44f3ed5e34e4ef52eedef233d1190bff004470d8e8005127b5312bff357fa7f/__main__.py", line 3359, in command_ceph_volume 2024-04-12T21:35:40.003 INFO:teuthology.orchestra.run.smithi018.stdout: File "/var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/cephadm.e44f3ed5e34e4ef52eedef233d1190bff004470d8e8005127b5312bff357fa7f/cephadmlib/call_wrappers.py", line 310, in call_throws 2024-04-12T21:35:40.003 INFO:teuthology.orchestra.run.smithi018.stdout:RuntimeError: Failed command: /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:f4452def3be4d350f4a8e5feaf7ebb1511be9298fe68478ba7a2d6fbd7d69a63 -e NODE_NAME=smithi018 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968:/var/run/ceph:z -v /var/log/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968:/var/log/ceph:z -v /var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmpxsmwkkuh:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:f4452def3be4d350f4a8e5feaf7ebb1511be9298fe68478ba7a2d6fbd7d69a63 inventory --format=json-pretty --filter-for-batch 2024-04-12T21:35:40.003 INFO:teuthology.orchestra.run.smithi018.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-04-12T21:35:40.003 INFO:teuthology.orchestra.run.smithi018.stdout: fs cephfs is degraded 2024-04-12T21:35:40.004 INFO:teuthology.orchestra.run.smithi018.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-04-12T21:35:40.004 INFO:teuthology.orchestra.run.smithi018.stdout: fs cephfs is offline because no MDS is active for it. 2024-04-12T21:35:40.004 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:40.003+0000 7f3111600700 1 -- 172.21.15.18:0/4078075917 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f30f006ec00 msgr2=0x7f30f00710b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:35:40.004 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:40.003+0000 7f3111600700 1 --2- 172.21.15.18:0/4078075917 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f30f006ec00 0x7f30f00710b0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f310c111430 tx=0x7f30f8009ee0 comp rx=0 tx=0).stop 2024-04-12T21:35:40.004 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:40.003+0000 7f3111600700 1 -- 172.21.15.18:0/4078075917 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f310c105020 msgr2=0x7f310c115b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:35:40.004 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:40.003+0000 7f3111600700 1 --2- 172.21.15.18:0/4078075917 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f310c105020 0x7f310c115b90 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 crypto rx=0x7f30fc009150 tx=0x7f30fc009e30 comp rx=0 tx=0).stop 2024-04-12T21:35:40.005 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:40.003+0000 7f3111600700 1 -- 172.21.15.18:0/4078075917 shutdown_connections 2024-04-12T21:35:40.005 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:40.003+0000 7f3111600700 1 --2- 172.21.15.18:0/4078075917 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f30f006ec00 0x7f30f00710b0 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:35:40.005 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:40.003+0000 7f3111600700 1 --2- 172.21.15.18:0/4078075917 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f310c105020 0x7f310c115b90 unknown :-1 s=CLOSED pgs=917 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:35:40.005 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:40.003+0000 7f3111600700 1 --2- 172.21.15.18:0/4078075917 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f310c1059c0 0x7f310c1160d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:35:40.005 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:40.003+0000 7f3111600700 1 -- 172.21.15.18:0/4078075917 >> 172.21.15.18:0/4078075917 conn(0x7f310c0faba0 msgr2=0x7f310c0fbb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:35:40.005 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:40.003+0000 7f3111600700 1 -- 172.21.15.18:0/4078075917 shutdown_connections 2024-04-12T21:35:40.005 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:35:40.003+0000 7f3111600700 1 -- 172.21.15.18:0/4078075917 wait complete. 2024-04-12T21:35:40.282 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:35:39 smithi186 ceph-mon[86379]: from='client.35884 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:35:40.283 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:35:39 smithi186 ceph-mon[86379]: from='client.? 172.21.15.18:0/3463027210' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-12T21:35:40.354 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:35:39 smithi018 ceph-mon[112590]: from='client.35884 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:35:40.354 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:35:39 smithi018 ceph-mon[112590]: from='client.? 172.21.15.18:0/3463027210' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-12T21:35:41.282 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:35:40 smithi186 ceph-mon[86379]: from='client.35896 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:35:41.283 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:35:40 smithi186 ceph-mon[86379]: pgmap v1077: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:35:41.283 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:35:40 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T21:35:41.283 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:35:40 smithi186 ceph-mon[86379]: from='client.? 172.21.15.18:0/4078075917' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-12T21:35:41.354 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:35:40 smithi018 ceph-mon[112590]: from='client.35896 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:35:41.354 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:35:40 smithi018 ceph-mon[112590]: pgmap v1077: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:35:41.354 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:35:40 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T21:35:41.354 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:35:40 smithi018 ceph-mon[112590]: from='client.? 172.21.15.18:0/4078075917' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-12T21:35:43.282 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:35:42 smithi186 ceph-mon[86379]: pgmap v1078: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:35:43.353 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:35:42 smithi018 ceph-mon[112590]: pgmap v1078: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:35:45.282 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:35:44 smithi186 ceph-mon[86379]: pgmap v1079: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:35:45.354 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:35:44 smithi018 ceph-mon[112590]: pgmap v1079: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:35:47.282 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:35:46 smithi186 ceph-mon[86379]: pgmap v1080: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:35:47.353 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:35:46 smithi018 ceph-mon[112590]: pgmap v1080: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:35:48.282 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:35:47 smithi186 ceph-mon[86379]: pgmap v1081: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:35:48.353 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:35:47 smithi018 ceph-mon[112590]: pgmap v1081: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:35:51.103 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:35:50 smithi018 ceph-mon[112590]: pgmap v1082: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:35:51.282 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:35:50 smithi186 ceph-mon[86379]: pgmap v1082: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:35:53.103 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:35:52 smithi018 ceph-mon[112590]: pgmap v1083: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:35:53.282 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:35:52 smithi186 ceph-mon[86379]: pgmap v1083: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:35:55.103 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:35:54 smithi018 ceph-mon[112590]: pgmap v1084: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:35:55.282 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:35:54 smithi186 ceph-mon[86379]: pgmap v1084: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:35:56.103 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:35:55 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T21:35:56.282 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:35:55 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T21:35:57.282 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:35:56 smithi186 ceph-mon[86379]: pgmap v1085: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:35:57.353 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:35:56 smithi018 ceph-mon[112590]: pgmap v1085: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:35:59.282 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:35:58 smithi186 ceph-mon[86379]: pgmap v1086: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:35:59.353 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:35:58 smithi018 ceph-mon[112590]: pgmap v1086: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:36:01.282 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:36:00 smithi186 ceph-mon[86379]: pgmap v1087: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:36:01.353 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:36:00 smithi018 ceph-mon[112590]: pgmap v1087: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:36:03.283 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:36:02 smithi186 ceph-mon[86379]: pgmap v1088: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:36:03.354 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:36:02 smithi018 ceph-mon[112590]: pgmap v1088: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:36:05.282 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:36:04 smithi186 ceph-mon[86379]: pgmap v1089: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:36:05.353 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:36:04 smithi018 ceph-mon[112590]: pgmap v1089: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:36:07.282 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:36:06 smithi186 ceph-mon[86379]: pgmap v1090: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:36:07.353 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:36:06 smithi018 ceph-mon[112590]: pgmap v1090: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:36:09.282 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:36:08 smithi186 ceph-mon[86379]: pgmap v1091: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:36:09.353 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:36:08 smithi018 ceph-mon[112590]: pgmap v1091: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:36:10.176 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:10.175+0000 7f4f05a00700 1 -- 172.21.15.18:0/471385302 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4f00109c40 msgr2=0x7f4f001032b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:36:10.176 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:10.175+0000 7f4f05a00700 1 --2- 172.21.15.18:0/471385302 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4f00109c40 0x7f4f001032b0 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 crypto rx=0x7f4ee8004940 tx=0x7f4ee8009e30 comp rx=0 tx=0).stop 2024-04-12T21:36:10.176 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:10.175+0000 7f4f05a00700 1 -- 172.21.15.18:0/471385302 shutdown_connections 2024-04-12T21:36:10.176 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:10.175+0000 7f4f05a00700 1 --2- 172.21.15.18:0/471385302 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4f00109c40 0x7f4f001032b0 unknown :-1 s=CLOSED pgs=918 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:36:10.176 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:10.175+0000 7f4f05a00700 1 --2- 172.21.15.18:0/471385302 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4f001092a0 0x7f4f00109670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:36:10.176 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:10.175+0000 7f4f05a00700 1 -- 172.21.15.18:0/471385302 >> 172.21.15.18:0/471385302 conn(0x7f4f00072a30 msgr2=0x7f4f00074e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:36:10.176 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:10.175+0000 7f4f05a00700 1 -- 172.21.15.18:0/471385302 shutdown_connections 2024-04-12T21:36:10.176 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:10.175+0000 7f4f05a00700 1 -- 172.21.15.18:0/471385302 wait complete. 2024-04-12T21:36:10.178 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:10.176+0000 7f4f05a00700 1 Processor -- start 2024-04-12T21:36:10.178 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:10.177+0000 7f4f05a00700 1 -- start start 2024-04-12T21:36:10.178 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:10.177+0000 7f4f05a00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4f001092a0 0x7f4f00119fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:36:10.178 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:10.177+0000 7f4f05a00700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4f00109c40 0x7f4f0011a500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:36:10.178 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:10.177+0000 7f4f05a00700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f0011ab50 con 0x7f4f001092a0 2024-04-12T21:36:10.178 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:10.177+0000 7f4f05a00700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f00113f20 con 0x7f4f00109c40 2024-04-12T21:36:10.178 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:10.177+0000 7f4effe00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4f001092a0 0x7f4f00119fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:36:10.178 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:10.178+0000 7f4effe00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4f001092a0 0x7f4f00119fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:54864/0 (socket says 172.21.15.18:54864) 2024-04-12T21:36:10.178 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:10.178+0000 7f4effe00700 1 -- 172.21.15.18:0/467478057 learned_addr learned my addr 172.21.15.18:0/467478057 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:36:10.179 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:10.178+0000 7f4ef7e00700 1 --2- 172.21.15.18:0/467478057 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4f00109c40 0x7f4f0011a500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:36:10.180 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:10.178+0000 7f4effe00700 1 -- 172.21.15.18:0/467478057 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4f00109c40 msgr2=0x7f4f0011a500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:36:10.180 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:10.178+0000 7f4effe00700 1 --2- 172.21.15.18:0/467478057 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4f00109c40 0x7f4f0011a500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:36:10.180 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:10.178+0000 7f4effe00700 1 -- 172.21.15.18:0/467478057 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ee8005040 con 0x7f4f001092a0 2024-04-12T21:36:10.180 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:10.178+0000 7f4effe00700 1 --2- 172.21.15.18:0/467478057 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4f001092a0 0x7f4f00119fc0 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 crypto rx=0x7f4ef000c8c0 tx=0x7f4ef00044c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:36:10.180 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:10.179+0000 7f4efcc00700 1 -- 172.21.15.18:0/467478057 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4ef000a980 con 0x7f4f001092a0 2024-04-12T21:36:10.180 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:10.179+0000 7f4efcc00700 1 -- 172.21.15.18:0/467478057 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f4ef000ce60 con 0x7f4f001092a0 2024-04-12T21:36:10.180 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:10.179+0000 7f4f05a00700 1 -- 172.21.15.18:0/467478057 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4f00114200 con 0x7f4f001092a0 2024-04-12T21:36:10.180 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:10.179+0000 7f4efcc00700 1 -- 172.21.15.18:0/467478057 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4ef00115e0 con 0x7f4f001092a0 2024-04-12T21:36:10.180 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:10.179+0000 7f4f05a00700 1 -- 172.21.15.18:0/467478057 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4f001147c0 con 0x7f4f001092a0 2024-04-12T21:36:10.181 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:10.180+0000 7f4f05a00700 1 -- 172.21.15.18:0/467478057 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4f001049f0 con 0x7f4f001092a0 2024-04-12T21:36:10.181 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:10.181+0000 7f4efcc00700 1 -- 172.21.15.18:0/467478057 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7f4ef0012400 con 0x7f4f001092a0 2024-04-12T21:36:10.182 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:10.182+0000 7f4efcc00700 1 --2- 172.21.15.18:0/467478057 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f4ee406ec00 0x7f4ee40710b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:36:10.183 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:10.182+0000 7f4efcc00700 1 -- 172.21.15.18:0/467478057 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 6757+0+0 (secure 0 0 0) 0x7f4ef008ea90 con 0x7f4f001092a0 2024-04-12T21:36:10.183 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:10.183+0000 7f4ef7e00700 1 --2- 172.21.15.18:0/467478057 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f4ee406ec00 0x7f4ee40710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:36:10.183 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:10.183+0000 7f4ef7e00700 1 --2- 172.21.15.18:0/467478057 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f4ee406ec00 0x7f4ee40710b0 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f4ee800b3b0 tx=0x7f4ee8009e30 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:36:10.188 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:10.188+0000 7f4efcc00700 1 -- 172.21.15.18:0/467478057 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f4ef005b2e0 con 0x7f4f001092a0 2024-04-12T21:36:10.447 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:10.447+0000 7f4f05a00700 1 -- 172.21.15.18:0/467478057 --> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f4f0004bed0 con 0x7f4ee406ec00 2024-04-12T21:36:10.448 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:10.448+0000 7f4efcc00700 1 -- 172.21.15.18:0/467478057 <== mgr.34102 v2:172.21.15.18:6800/3812891919 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f4f0004bed0 con 0x7f4ee406ec00 2024-04-12T21:36:10.452 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:10.451+0000 7f4f05a00700 1 -- 172.21.15.18:0/467478057 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f4ee406ec00 msgr2=0x7f4ee40710b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:36:10.452 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:10.451+0000 7f4f05a00700 1 --2- 172.21.15.18:0/467478057 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f4ee406ec00 0x7f4ee40710b0 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f4ee800b3b0 tx=0x7f4ee8009e30 comp rx=0 tx=0).stop 2024-04-12T21:36:10.452 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:10.451+0000 7f4f05a00700 1 -- 172.21.15.18:0/467478057 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4f001092a0 msgr2=0x7f4f00119fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:36:10.452 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:10.451+0000 7f4f05a00700 1 --2- 172.21.15.18:0/467478057 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4f001092a0 0x7f4f00119fc0 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 crypto rx=0x7f4ef000c8c0 tx=0x7f4ef00044c0 comp rx=0 tx=0).stop 2024-04-12T21:36:10.452 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:10.452+0000 7f4f05a00700 1 -- 172.21.15.18:0/467478057 shutdown_connections 2024-04-12T21:36:10.452 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:10.452+0000 7f4f05a00700 1 --2- 172.21.15.18:0/467478057 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f4ee406ec00 0x7f4ee40710b0 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:36:10.452 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:10.452+0000 7f4f05a00700 1 --2- 172.21.15.18:0/467478057 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4f001092a0 0x7f4f00119fc0 unknown :-1 s=CLOSED pgs=919 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:36:10.453 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:10.452+0000 7f4f05a00700 1 --2- 172.21.15.18:0/467478057 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4f00109c40 0x7f4f0011a500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:36:10.453 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:10.452+0000 7f4f05a00700 1 -- 172.21.15.18:0/467478057 >> 172.21.15.18:0/467478057 conn(0x7f4f00072a30 msgr2=0x7f4f00074910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:36:10.453 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:10.452+0000 7f4f05a00700 1 -- 172.21.15.18:0/467478057 shutdown_connections 2024-04-12T21:36:10.453 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:10.452+0000 7f4f05a00700 1 -- 172.21.15.18:0/467478057 wait complete. 2024-04-12T21:36:10.468 INFO:teuthology.orchestra.run.smithi018.stdout:true 2024-04-12T21:36:10.613 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:10.613+0000 7fa2be000700 1 -- 172.21.15.18:0/2237550397 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa2b81037a0 msgr2=0x7fa2b80fce10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:36:10.613 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:10.613+0000 7fa2be000700 1 --2- 172.21.15.18:0/2237550397 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa2b81037a0 0x7fa2b80fce10 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 crypto rx=0x7fa2a4004940 tx=0x7fa2a4009e30 comp rx=0 tx=0).stop 2024-04-12T21:36:10.613 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:10.613+0000 7fa2be000700 1 -- 172.21.15.18:0/2237550397 shutdown_connections 2024-04-12T21:36:10.614 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:10.613+0000 7fa2be000700 1 --2- 172.21.15.18:0/2237550397 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa2b81037a0 0x7fa2b80fce10 unknown :-1 s=CLOSED pgs=920 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:36:10.614 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:10.613+0000 7fa2be000700 1 --2- 172.21.15.18:0/2237550397 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa2b8102e00 0x7fa2b81031d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:36:10.614 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:10.613+0000 7fa2be000700 1 -- 172.21.15.18:0/2237550397 >> 172.21.15.18:0/2237550397 conn(0x7fa2b80f89b0 msgr2=0x7fa2b80fadc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:36:10.614 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:10.613+0000 7fa2be000700 1 -- 172.21.15.18:0/2237550397 shutdown_connections 2024-04-12T21:36:10.614 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:10.613+0000 7fa2be000700 1 -- 172.21.15.18:0/2237550397 wait complete. 2024-04-12T21:36:10.615 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:10.614+0000 7fa2be000700 1 Processor -- start 2024-04-12T21:36:10.615 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:10.614+0000 7fa2be000700 1 -- start start 2024-04-12T21:36:10.615 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:10.614+0000 7fa2be000700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa2b8102e00 0x7fa2b8115bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:36:10.615 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:10.614+0000 7fa2be000700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa2b81037a0 0x7fa2b8116100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:36:10.615 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:10.614+0000 7fa2be000700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2b81167e0 con 0x7fa2b81037a0 2024-04-12T21:36:10.616 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:10.614+0000 7fa2be000700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2b810fd30 con 0x7fa2b8102e00 2024-04-12T21:36:10.616 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:10.615+0000 7fa2b6a00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa2b81037a0 0x7fa2b8116100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:36:10.616 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:10.615+0000 7fa2b6a00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa2b81037a0 0x7fa2b8116100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:54882/0 (socket says 172.21.15.18:54882) 2024-04-12T21:36:10.616 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:10.615+0000 7fa2b6a00700 1 -- 172.21.15.18:0/2307116850 learned_addr learned my addr 172.21.15.18:0/2307116850 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:36:10.616 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:10.615+0000 7fa2b7400700 1 --2- 172.21.15.18:0/2307116850 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa2b8102e00 0x7fa2b8115bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:36:10.616 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:10.615+0000 7fa2b6a00700 1 -- 172.21.15.18:0/2307116850 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa2b8102e00 msgr2=0x7fa2b8115bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:36:10.616 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:10.615+0000 7fa2b6a00700 1 --2- 172.21.15.18:0/2307116850 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa2b8102e00 0x7fa2b8115bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:36:10.616 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:10.615+0000 7fa2b6a00700 1 -- 172.21.15.18:0/2307116850 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2a4005040 con 0x7fa2b81037a0 2024-04-12T21:36:10.616 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:10.615+0000 7fa2b7400700 1 --2- 172.21.15.18:0/2307116850 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa2b8102e00 0x7fa2b8115bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-12T21:36:10.617 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:10.616+0000 7fa2b6a00700 1 --2- 172.21.15.18:0/2307116850 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa2b81037a0 0x7fa2b8116100 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 crypto rx=0x7fa2a4009180 tx=0x7fa2a4009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:36:10.617 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:10.616+0000 7fa2af400700 1 -- 172.21.15.18:0/2307116850 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa2a400e040 con 0x7fa2b81037a0 2024-04-12T21:36:10.617 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:10.616+0000 7fa2af400700 1 -- 172.21.15.18:0/2307116850 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa2a400bbc0 con 0x7fa2b81037a0 2024-04-12T21:36:10.617 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:10.616+0000 7fa2af400700 1 -- 172.21.15.18:0/2307116850 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa2a401c3f0 con 0x7fa2b81037a0 2024-04-12T21:36:10.617 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:10.616+0000 7fa2be000700 1 -- 172.21.15.18:0/2307116850 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa2b810ff50 con 0x7fa2b81037a0 2024-04-12T21:36:10.617 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:10.616+0000 7fa2be000700 1 -- 172.21.15.18:0/2307116850 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa2b8110510 con 0x7fa2b81037a0 2024-04-12T21:36:10.617 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:10.617+0000 7fa2be000700 1 -- 172.21.15.18:0/2307116850 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa2b8106a00 con 0x7fa2b81037a0 2024-04-12T21:36:10.618 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:10.618+0000 7fa2af400700 1 -- 172.21.15.18:0/2307116850 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7fa2a400bd30 con 0x7fa2b81037a0 2024-04-12T21:36:10.619 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:10.619+0000 7fa2af400700 1 --2- 172.21.15.18:0/2307116850 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fa29c06ec00 0x7fa29c0710b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:36:10.620 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:10.619+0000 7fa2af400700 1 -- 172.21.15.18:0/2307116850 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 6757+0+0 (secure 0 0 0) 0x7fa2a408fc00 con 0x7fa2b81037a0 2024-04-12T21:36:10.620 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:10.620+0000 7fa2b7400700 1 --2- 172.21.15.18:0/2307116850 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fa29c06ec00 0x7fa29c0710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:36:10.620 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:10.620+0000 7fa2b7400700 1 --2- 172.21.15.18:0/2307116850 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fa29c06ec00 0x7fa29c0710b0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7fa2a8006fd0 tx=0x7fa2a8008040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:36:10.625 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:10.625+0000 7fa2af400700 1 -- 172.21.15.18:0/2307116850 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7fa2a4095020 con 0x7fa2b81037a0 2024-04-12T21:36:10.881 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:10.881+0000 7fa2be000700 1 -- 172.21.15.18:0/2307116850 --> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fa2b8109f50 con 0x7fa29c06ec00 2024-04-12T21:36:10.884 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:10.884+0000 7fa2af400700 1 -- 172.21.15.18:0/2307116850 <== mgr.34102 v2:172.21.15.18:6800/3812891919 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7fa2b8109f50 con 0x7fa29c06ec00 2024-04-12T21:36:10.887 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:10.886+0000 7fa2be000700 1 -- 172.21.15.18:0/2307116850 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fa29c06ec00 msgr2=0x7fa29c0710b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:36:10.887 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:10.886+0000 7fa2be000700 1 --2- 172.21.15.18:0/2307116850 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fa29c06ec00 0x7fa29c0710b0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7fa2a8006fd0 tx=0x7fa2a8008040 comp rx=0 tx=0).stop 2024-04-12T21:36:10.887 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:10.887+0000 7fa2be000700 1 -- 172.21.15.18:0/2307116850 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa2b81037a0 msgr2=0x7fa2b8116100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:36:10.887 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:10.887+0000 7fa2be000700 1 --2- 172.21.15.18:0/2307116850 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa2b81037a0 0x7fa2b8116100 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 crypto rx=0x7fa2a4009180 tx=0x7fa2a4009e30 comp rx=0 tx=0).stop 2024-04-12T21:36:10.887 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:10.887+0000 7fa2be000700 1 -- 172.21.15.18:0/2307116850 shutdown_connections 2024-04-12T21:36:10.887 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:10.887+0000 7fa2be000700 1 --2- 172.21.15.18:0/2307116850 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fa29c06ec00 0x7fa29c0710b0 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:36:10.888 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:10.887+0000 7fa2be000700 1 --2- 172.21.15.18:0/2307116850 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa2b8102e00 0x7fa2b8115bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:36:10.888 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:10.887+0000 7fa2be000700 1 --2- 172.21.15.18:0/2307116850 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa2b81037a0 0x7fa2b8116100 unknown :-1 s=CLOSED pgs=921 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:36:10.888 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:10.887+0000 7fa2be000700 1 -- 172.21.15.18:0/2307116850 >> 172.21.15.18:0/2307116850 conn(0x7fa2b80f89b0 msgr2=0x7fa2b80fa5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:36:10.888 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:10.887+0000 7fa2be000700 1 -- 172.21.15.18:0/2307116850 shutdown_connections 2024-04-12T21:36:10.888 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:10.887+0000 7fa2be000700 1 -- 172.21.15.18:0/2307116850 wait complete. 2024-04-12T21:36:11.041 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:11.040+0000 7f85a9800700 1 -- 172.21.15.18:0/3191374191 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f85a40fbbf0 msgr2=0x7f85a40fbfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:36:11.041 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:11.040+0000 7f85a9800700 1 --2- 172.21.15.18:0/3191374191 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f85a40fbbf0 0x7f85a40fbfc0 secure :-1 s=READY pgs=922 cs=0 l=1 rev1=1 crypto rx=0x7f8594004940 tx=0x7f8594009e30 comp rx=0 tx=0).stop 2024-04-12T21:36:11.041 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:11.040+0000 7f85a9800700 1 -- 172.21.15.18:0/3191374191 shutdown_connections 2024-04-12T21:36:11.041 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:11.040+0000 7f85a9800700 1 --2- 172.21.15.18:0/3191374191 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f85a40fc500 0x7f85a4109260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:36:11.041 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:11.040+0000 7f85a9800700 1 --2- 172.21.15.18:0/3191374191 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f85a40fbbf0 0x7f85a40fbfc0 unknown :-1 s=CLOSED pgs=922 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:36:11.041 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:11.040+0000 7f85a9800700 1 -- 172.21.15.18:0/3191374191 >> 172.21.15.18:0/3191374191 conn(0x7f85a40f7880 msgr2=0x7f85a40f9c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:36:11.041 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:11.040+0000 7f85a9800700 1 -- 172.21.15.18:0/3191374191 shutdown_connections 2024-04-12T21:36:11.041 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:11.040+0000 7f85a9800700 1 -- 172.21.15.18:0/3191374191 wait complete. 2024-04-12T21:36:11.042 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:11.041+0000 7f85a9800700 1 Processor -- start 2024-04-12T21:36:11.043 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:11.041+0000 7f85a9800700 1 -- start start 2024-04-12T21:36:11.043 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:11.041+0000 7f85a9800700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f85a40fbbf0 0x7f85a4115fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:36:11.043 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:11.042+0000 7f85a9800700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f85a40fc500 0x7f85a4116520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:36:11.043 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:11.042+0000 7f85a9800700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85a4116b70 con 0x7f85a40fbbf0 2024-04-12T21:36:11.043 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:11.042+0000 7f85a9800700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85a410ff90 con 0x7f85a40fc500 2024-04-12T21:36:11.043 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:11.042+0000 7f85a2a00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f85a40fbbf0 0x7f85a4115fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:36:11.043 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:11.042+0000 7f85a2a00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f85a40fbbf0 0x7f85a4115fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:34478/0 (socket says 172.21.15.18:34478) 2024-04-12T21:36:11.043 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:11.042+0000 7f85a2a00700 1 -- 172.21.15.18:0/3224207646 learned_addr learned my addr 172.21.15.18:0/3224207646 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:36:11.043 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:11.042+0000 7f85a2a00700 1 -- 172.21.15.18:0/3224207646 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f85a40fc500 msgr2=0x7f85a4116520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:36:11.043 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:11.042+0000 7f85a2a00700 1 --2- 172.21.15.18:0/3224207646 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f85a40fc500 0x7f85a4116520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:36:11.043 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:11.042+0000 7f85a2a00700 1 -- 172.21.15.18:0/3224207646 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8594005040 con 0x7f85a40fbbf0 2024-04-12T21:36:11.043 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:11.043+0000 7f85a2a00700 1 --2- 172.21.15.18:0/3224207646 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f85a40fbbf0 0x7f85a4115fe0 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 crypto rx=0x7f8594000c00 tx=0x7f8594009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:36:11.044 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:11.043+0000 7f859aa00700 1 -- 172.21.15.18:0/3224207646 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f859400e070 con 0x7f85a40fbbf0 2024-04-12T21:36:11.044 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:11.043+0000 7f859aa00700 1 -- 172.21.15.18:0/3224207646 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f8594013cd0 con 0x7f85a40fbbf0 2024-04-12T21:36:11.044 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:11.043+0000 7f85a9800700 1 -- 172.21.15.18:0/3224207646 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f85a4110210 con 0x7f85a40fbbf0 2024-04-12T21:36:11.044 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:11.043+0000 7f859aa00700 1 -- 172.21.15.18:0/3224207646 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8594012600 con 0x7f85a40fbbf0 2024-04-12T21:36:11.044 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:11.043+0000 7f85a9800700 1 -- 172.21.15.18:0/3224207646 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f85a41107d0 con 0x7f85a40fbbf0 2024-04-12T21:36:11.044 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:11.044+0000 7f85a9800700 1 -- 172.21.15.18:0/3224207646 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f85a4106990 con 0x7f85a40fbbf0 2024-04-12T21:36:11.045 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:11.045+0000 7f859aa00700 1 -- 172.21.15.18:0/3224207646 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7f859401c400 con 0x7f85a40fbbf0 2024-04-12T21:36:11.046 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:11.046+0000 7f859aa00700 1 --2- 172.21.15.18:0/3224207646 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f858806ec10 0x7f85880710c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:36:11.047 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:11.046+0000 7f859aa00700 1 -- 172.21.15.18:0/3224207646 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 6757+0+0 (secure 0 0 0) 0x7f8594090290 con 0x7f85a40fbbf0 2024-04-12T21:36:11.047 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:11.046+0000 7f85a2000700 1 --2- 172.21.15.18:0/3224207646 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f858806ec10 0x7f85880710c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:36:11.047 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:11.047+0000 7f85a2000700 1 --2- 172.21.15.18:0/3224207646 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f858806ec10 0x7f85880710c0 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f85a4111880 tx=0x7f859000b040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:36:11.051 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:11.051+0000 7f859aa00700 1 -- 172.21.15.18:0/3224207646 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f859405ca40 con 0x7f85a40fbbf0 2024-04-12T21:36:11.283 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:36:10 smithi186 ceph-mon[86379]: pgmap v1092: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:36:11.283 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:36:10 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T21:36:11.291 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:11.291+0000 7f85a9800700 1 -- 172.21.15.18:0/3224207646 --> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f85a410c0a0 con 0x7f858806ec10 2024-04-12T21:36:11.292 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:36:10 smithi018 ceph-mon[112590]: pgmap v1092: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:36:11.292 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:36:10 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T21:36:11.324 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:11.324+0000 7f859aa00700 1 -- 172.21.15.18:0/3224207646 <== mgr.34102 v2:172.21.15.18:6800/3812891919 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7f85a410c0a0 con 0x7f858806ec10 2024-04-12T21:36:11.325 INFO:teuthology.orchestra.run.smithi018.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-12T21:36:11.325 INFO:teuthology.orchestra.run.smithi018.stdout:alertmanager.smithi018 smithi018 *:9093,9094 running (36m) 13m ago 51m 15.5M - 0.25.0 c8568f914cd2 c332090068e8 2024-04-12T21:36:11.325 INFO:teuthology.orchestra.run.smithi018.stdout:crash.smithi018 smithi018 running (33m) 13m ago 51m 6665k - 19.0.0-2879-g324baf8d 8717720dc5e5 b7f80d1f7982 2024-04-12T21:36:11.325 INFO:teuthology.orchestra.run.smithi018.stdout:crash.smithi186 smithi186 running (32m) 13m ago 50m 6660k - 19.0.0-2879-g324baf8d 8717720dc5e5 a4f55c81b623 2024-04-12T21:36:11.325 INFO:teuthology.orchestra.run.smithi018.stdout:grafana.smithi018 smithi018 *:3000 running (36m) 13m ago 50m 74.8M - 9.4.12 2bacad6d85d8 b08e3bf176dd 2024-04-12T21:36:11.325 INFO:teuthology.orchestra.run.smithi018.stdout:mds.cephfs.smithi018.acqbsc smithi018 running (29m) 13m ago 44m 16.7M - 19.0.0-2879-g324baf8d 8717720dc5e5 021f99ab24ae 2024-04-12T21:36:11.325 INFO:teuthology.orchestra.run.smithi018.stdout:mds.cephfs.smithi018.szjluo smithi018 running (28m) 13m ago 44m 16.5M - 19.0.0-2879-g324baf8d 8717720dc5e5 0003117479d4 2024-04-12T21:36:11.325 INFO:teuthology.orchestra.run.smithi018.stdout:mds.cephfs.smithi186.jdllyn smithi186 running (13m) 13m ago 44m 11.0M - 19.0.0-2879-g324baf8d 8717720dc5e5 21b1f46eef23 2024-04-12T21:36:11.325 INFO:teuthology.orchestra.run.smithi018.stdout:mds.cephfs.smithi186.wjnafs smithi186 running (43m) 13m ago 43m 19.5M - 17.2.7-827-gd3b5060a 21de9b7369ad 74b115d6d6fd 2024-04-12T21:36:11.326 INFO:teuthology.orchestra.run.smithi018.stdout:mgr.smithi018.xjzwga smithi018 *:8443,9283,8765 running (39m) 13m ago 53m 512M - 19.0.0-2879-g324baf8d 8717720dc5e5 56088305eab7 2024-04-12T21:36:11.326 INFO:teuthology.orchestra.run.smithi018.stdout:mgr.smithi186.kqcfxk smithi186 *:8443,9283,8765 running (38m) 13m ago 49m 459M - 19.0.0-2879-g324baf8d 8717720dc5e5 09cd766e5e0d 2024-04-12T21:36:11.326 INFO:teuthology.orchestra.run.smithi018.stdout:mon.smithi018 smithi018 running (34m) 13m ago 53m 73.1M 2048M 19.0.0-2879-g324baf8d 8717720dc5e5 df3e201dfed6 2024-04-12T21:36:11.326 INFO:teuthology.orchestra.run.smithi018.stdout:mon.smithi186 smithi186 running (34m) 13m ago 49m 47.2M 2048M 19.0.0-2879-g324baf8d 8717720dc5e5 c77498da9b2b 2024-04-12T21:36:11.326 INFO:teuthology.orchestra.run.smithi018.stdout:node-exporter.smithi018 smithi018 *:9100 running (37m) 13m ago 51m 8108k - 1.5.0 0da6a335fe13 326c3d9c6847 2024-04-12T21:36:11.326 INFO:teuthology.orchestra.run.smithi018.stdout:node-exporter.smithi186 smithi186 *:9100 running (37m) 13m ago 50m 8757k - 1.5.0 0da6a335fe13 ec2112baeecf 2024-04-12T21:36:11.326 INFO:teuthology.orchestra.run.smithi018.stdout:osd.0 smithi018 running (32m) 13m ago 48m 123M 1559M 19.0.0-2879-g324baf8d 8717720dc5e5 60b1273ca047 2024-04-12T21:36:11.326 INFO:teuthology.orchestra.run.smithi018.stdout:osd.1 smithi018 running (31m) 13m ago 48m 95.1M 1559M 19.0.0-2879-g324baf8d 8717720dc5e5 170ce1e5fb8d 2024-04-12T21:36:11.326 INFO:teuthology.orchestra.run.smithi018.stdout:osd.2 smithi018 running (31m) 13m ago 47m 94.3M 1559M 19.0.0-2879-g324baf8d 8717720dc5e5 c2cbc6aa2839 2024-04-12T21:36:11.326 INFO:teuthology.orchestra.run.smithi018.stdout:osd.3 smithi186 running (30m) 13m ago 47m 101M 2581M 19.0.0-2879-g324baf8d 8717720dc5e5 0138e8262e4b 2024-04-12T21:36:11.326 INFO:teuthology.orchestra.run.smithi018.stdout:osd.4 smithi186 running (30m) 13m ago 46m 91.0M 2581M 19.0.0-2879-g324baf8d 8717720dc5e5 b20cb63b25c9 2024-04-12T21:36:11.326 INFO:teuthology.orchestra.run.smithi018.stdout:osd.5 smithi186 running (29m) 13m ago 46m 92.0M 2581M 19.0.0-2879-g324baf8d 8717720dc5e5 2513bea2610e 2024-04-12T21:36:11.326 INFO:teuthology.orchestra.run.smithi018.stdout:prometheus.smithi018 smithi018 *:9095 running (37m) 13m ago 50m 67.9M - 2.43.0 a07b618ecd1d f58841127e21 2024-04-12T21:36:11.327 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:11.327+0000 7f85a9800700 1 -- 172.21.15.18:0/3224207646 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f858806ec10 msgr2=0x7f85880710c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:36:11.327 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:11.327+0000 7f85a9800700 1 --2- 172.21.15.18:0/3224207646 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f858806ec10 0x7f85880710c0 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f85a4111880 tx=0x7f859000b040 comp rx=0 tx=0).stop 2024-04-12T21:36:11.328 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:11.327+0000 7f85a9800700 1 -- 172.21.15.18:0/3224207646 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f85a40fbbf0 msgr2=0x7f85a4115fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:36:11.328 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:11.327+0000 7f85a9800700 1 --2- 172.21.15.18:0/3224207646 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f85a40fbbf0 0x7f85a4115fe0 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 crypto rx=0x7f8594000c00 tx=0x7f8594009e30 comp rx=0 tx=0).stop 2024-04-12T21:36:11.328 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:11.327+0000 7f85a9800700 1 -- 172.21.15.18:0/3224207646 shutdown_connections 2024-04-12T21:36:11.328 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:11.327+0000 7f85a9800700 1 --2- 172.21.15.18:0/3224207646 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f858806ec10 0x7f85880710c0 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:36:11.328 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:11.327+0000 7f85a9800700 1 --2- 172.21.15.18:0/3224207646 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f85a40fbbf0 0x7f85a4115fe0 unknown :-1 s=CLOSED pgs=923 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:36:11.328 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:11.327+0000 7f85a9800700 1 --2- 172.21.15.18:0/3224207646 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f85a40fc500 0x7f85a4116520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:36:11.329 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:11.327+0000 7f85a9800700 1 -- 172.21.15.18:0/3224207646 >> 172.21.15.18:0/3224207646 conn(0x7f85a40f7880 msgr2=0x7f85a40f8040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:36:11.329 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:11.327+0000 7f85a9800700 1 -- 172.21.15.18:0/3224207646 shutdown_connections 2024-04-12T21:36:11.329 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:11.327+0000 7f85a9800700 1 -- 172.21.15.18:0/3224207646 wait complete. 2024-04-12T21:36:11.484 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:11.483+0000 7ff14d200700 1 -- 172.21.15.18:0/2790391022 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff1481035d0 msgr2=0x7ff1480fcc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:36:11.484 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:11.483+0000 7ff14d200700 1 --2- 172.21.15.18:0/2790391022 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff1481035d0 0x7ff1480fcc40 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 crypto rx=0x7ff134004940 tx=0x7ff134009e30 comp rx=0 tx=0).stop 2024-04-12T21:36:11.484 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:11.483+0000 7ff14d200700 1 -- 172.21.15.18:0/2790391022 shutdown_connections 2024-04-12T21:36:11.484 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:11.483+0000 7ff14d200700 1 --2- 172.21.15.18:0/2790391022 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff1481035d0 0x7ff1480fcc40 unknown :-1 s=CLOSED pgs=924 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:36:11.484 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:11.483+0000 7ff14d200700 1 --2- 172.21.15.18:0/2790391022 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff148102c30 0x7ff148103000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:36:11.485 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:11.483+0000 7ff14d200700 1 -- 172.21.15.18:0/2790391022 >> 172.21.15.18:0/2790391022 conn(0x7ff1480f8760 msgr2=0x7ff1480fab70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:36:11.485 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:11.484+0000 7ff14d200700 1 -- 172.21.15.18:0/2790391022 shutdown_connections 2024-04-12T21:36:11.485 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:11.484+0000 7ff14d200700 1 -- 172.21.15.18:0/2790391022 wait complete. 2024-04-12T21:36:11.486 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:11.485+0000 7ff14d200700 1 Processor -- start 2024-04-12T21:36:11.486 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:11.485+0000 7ff14d200700 1 -- start start 2024-04-12T21:36:11.486 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:11.485+0000 7ff14d200700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff148102c30 0x7ff1481139d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:36:11.486 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:11.485+0000 7ff14d200700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff1481035d0 0x7ff148113f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:36:11.486 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:11.485+0000 7ff14d200700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1481145f0 con 0x7ff1481035d0 2024-04-12T21:36:11.487 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:11.485+0000 7ff14d200700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff14810d950 con 0x7ff148102c30 2024-04-12T21:36:11.487 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:11.486+0000 7ff146a00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff1481035d0 0x7ff148113f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:36:11.487 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:11.486+0000 7ff146a00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff1481035d0 0x7ff148113f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:34502/0 (socket says 172.21.15.18:34502) 2024-04-12T21:36:11.487 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:11.486+0000 7ff146a00700 1 -- 172.21.15.18:0/297969574 learned_addr learned my addr 172.21.15.18:0/297969574 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:36:11.487 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:11.486+0000 7ff146a00700 1 -- 172.21.15.18:0/297969574 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff148102c30 msgr2=0x7ff1481139d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:36:11.487 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:11.486+0000 7ff146a00700 1 --2- 172.21.15.18:0/297969574 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff148102c30 0x7ff1481139d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:36:11.487 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:11.486+0000 7ff146a00700 1 -- 172.21.15.18:0/297969574 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff134005040 con 0x7ff1481035d0 2024-04-12T21:36:11.487 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:11.486+0000 7ff146a00700 1 --2- 172.21.15.18:0/297969574 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff1481035d0 0x7ff148113f10 secure :-1 s=READY pgs=925 cs=0 l=1 rev1=1 crypto rx=0x7ff134009100 tx=0x7ff134009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:36:11.488 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:11.487+0000 7ff13f400700 1 -- 172.21.15.18:0/297969574 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff13400e040 con 0x7ff1481035d0 2024-04-12T21:36:11.488 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:11.487+0000 7ff13f400700 1 -- 172.21.15.18:0/297969574 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff134013420 con 0x7ff1481035d0 2024-04-12T21:36:11.488 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:11.487+0000 7ff13f400700 1 -- 172.21.15.18:0/297969574 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff13401c630 con 0x7ff1481035d0 2024-04-12T21:36:11.488 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:11.487+0000 7ff14d200700 1 -- 172.21.15.18:0/297969574 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff14810dbd0 con 0x7ff1481035d0 2024-04-12T21:36:11.488 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:11.487+0000 7ff14d200700 1 -- 172.21.15.18:0/297969574 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff14810e190 con 0x7ff1481035d0 2024-04-12T21:36:11.489 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:11.488+0000 7ff14d200700 1 -- 172.21.15.18:0/297969574 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff1480fe380 con 0x7ff1481035d0 2024-04-12T21:36:11.489 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:11.489+0000 7ff13f400700 1 -- 172.21.15.18:0/297969574 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7ff134013590 con 0x7ff1481035d0 2024-04-12T21:36:11.490 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:11.490+0000 7ff13f400700 1 --2- 172.21.15.18:0/297969574 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7ff12c06ec00 0x7ff12c0710b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:36:11.490 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:11.490+0000 7ff13f400700 1 -- 172.21.15.18:0/297969574 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 6757+0+0 (secure 0 0 0) 0x7ff13408fc60 con 0x7ff1481035d0 2024-04-12T21:36:11.490 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:11.490+0000 7ff147400700 1 --2- 172.21.15.18:0/297969574 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7ff12c06ec00 0x7ff12c0710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:36:11.491 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:11.491+0000 7ff147400700 1 --2- 172.21.15.18:0/297969574 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7ff12c06ec00 0x7ff12c0710b0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7ff138006fd0 tx=0x7ff138006db0 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:36:11.494 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:11.495+0000 7ff13f400700 1 -- 172.21.15.18:0/297969574 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7ff13405c410 con 0x7ff1481035d0 2024-04-12T21:36:11.810 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:11.809+0000 7ff14d200700 1 -- 172.21.15.18:0/297969574 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7ff1480fe380 con 0x7ff1481035d0 2024-04-12T21:36:11.811 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:11.811+0000 7ff13f400700 1 -- 172.21.15.18:0/297969574 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+837 (secure 0 0 0) 0x7ff13405bfa0 con 0x7ff1481035d0 2024-04-12T21:36:11.811 INFO:teuthology.orchestra.run.smithi018.stdout:{ 2024-04-12T21:36:11.811 INFO:teuthology.orchestra.run.smithi018.stdout: "mon": { 2024-04-12T21:36:11.812 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 19.0.0-2879-g324baf8d (324baf8ddb91dea46cac50cb535c306115791355) squid (dev)": 2 2024-04-12T21:36:11.812 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-12T21:36:11.812 INFO:teuthology.orchestra.run.smithi018.stdout: "mgr": { 2024-04-12T21:36:11.812 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 19.0.0-2879-g324baf8d (324baf8ddb91dea46cac50cb535c306115791355) squid (dev)": 2 2024-04-12T21:36:11.812 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-12T21:36:11.812 INFO:teuthology.orchestra.run.smithi018.stdout: "osd": { 2024-04-12T21:36:11.812 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 19.0.0-2879-g324baf8d (324baf8ddb91dea46cac50cb535c306115791355) squid (dev)": 6 2024-04-12T21:36:11.812 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-12T21:36:11.812 INFO:teuthology.orchestra.run.smithi018.stdout: "mds": { 2024-04-12T21:36:11.812 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 17.2.7-827-gd3b5060a (d3b5060a17449edfae9385cdf792afec109fca5d) quincy (stable)": 1, 2024-04-12T21:36:11.812 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 19.0.0-2879-g324baf8d (324baf8ddb91dea46cac50cb535c306115791355) squid (dev)": 3 2024-04-12T21:36:11.812 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-12T21:36:11.812 INFO:teuthology.orchestra.run.smithi018.stdout: "overall": { 2024-04-12T21:36:11.812 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 17.2.7-827-gd3b5060a (d3b5060a17449edfae9385cdf792afec109fca5d) quincy (stable)": 1, 2024-04-12T21:36:11.813 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 19.0.0-2879-g324baf8d (324baf8ddb91dea46cac50cb535c306115791355) squid (dev)": 13 2024-04-12T21:36:11.813 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-04-12T21:36:11.813 INFO:teuthology.orchestra.run.smithi018.stdout:} 2024-04-12T21:36:11.814 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:11.813+0000 7ff14d200700 1 -- 172.21.15.18:0/297969574 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7ff12c06ec00 msgr2=0x7ff12c0710b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:36:11.814 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:11.813+0000 7ff14d200700 1 --2- 172.21.15.18:0/297969574 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7ff12c06ec00 0x7ff12c0710b0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7ff138006fd0 tx=0x7ff138006db0 comp rx=0 tx=0).stop 2024-04-12T21:36:11.814 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:11.813+0000 7ff14d200700 1 -- 172.21.15.18:0/297969574 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff1481035d0 msgr2=0x7ff148113f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:36:11.814 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:11.813+0000 7ff14d200700 1 --2- 172.21.15.18:0/297969574 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff1481035d0 0x7ff148113f10 secure :-1 s=READY pgs=925 cs=0 l=1 rev1=1 crypto rx=0x7ff134009100 tx=0x7ff134009e30 comp rx=0 tx=0).stop 2024-04-12T21:36:11.814 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:11.813+0000 7ff14d200700 1 -- 172.21.15.18:0/297969574 shutdown_connections 2024-04-12T21:36:11.814 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:11.813+0000 7ff14d200700 1 --2- 172.21.15.18:0/297969574 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7ff12c06ec00 0x7ff12c0710b0 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:36:11.814 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:11.813+0000 7ff14d200700 1 --2- 172.21.15.18:0/297969574 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff148102c30 0x7ff1481139d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:36:11.814 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:11.813+0000 7ff14d200700 1 --2- 172.21.15.18:0/297969574 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff1481035d0 0x7ff148113f10 unknown :-1 s=CLOSED pgs=925 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:36:11.814 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:11.813+0000 7ff14d200700 1 -- 172.21.15.18:0/297969574 >> 172.21.15.18:0/297969574 conn(0x7ff1480f8760 msgr2=0x7ff1480f9720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:36:11.814 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:11.814+0000 7ff14d200700 1 -- 172.21.15.18:0/297969574 shutdown_connections 2024-04-12T21:36:11.814 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:11.814+0000 7ff14d200700 1 -- 172.21.15.18:0/297969574 wait complete. 2024-04-12T21:36:11.970 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:11.969+0000 7f4729400700 1 -- 172.21.15.18:0/10153604 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f47241092a0 msgr2=0x7f4724109670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:36:11.970 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:11.969+0000 7f4729400700 1 --2- 172.21.15.18:0/10153604 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f47241092a0 0x7f4724109670 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 crypto rx=0x7f4714004940 tx=0x7f4714009e30 comp rx=0 tx=0).stop 2024-04-12T21:36:11.970 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:11.969+0000 7f4729400700 1 -- 172.21.15.18:0/10153604 shutdown_connections 2024-04-12T21:36:11.970 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:11.969+0000 7f4729400700 1 --2- 172.21.15.18:0/10153604 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4724109c40 0x7f47241032b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:36:11.970 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:11.969+0000 7f4729400700 1 --2- 172.21.15.18:0/10153604 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f47241092a0 0x7f4724109670 unknown :-1 s=CLOSED pgs=926 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:36:11.970 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:11.969+0000 7f4729400700 1 -- 172.21.15.18:0/10153604 >> 172.21.15.18:0/10153604 conn(0x7f4724072a30 msgr2=0x7f4724074e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:36:11.970 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:11.969+0000 7f4729400700 1 -- 172.21.15.18:0/10153604 shutdown_connections 2024-04-12T21:36:11.970 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:11.969+0000 7f4729400700 1 -- 172.21.15.18:0/10153604 wait complete. 2024-04-12T21:36:11.971 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:11.970+0000 7f4729400700 1 Processor -- start 2024-04-12T21:36:11.971 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:11.970+0000 7f4729400700 1 -- start start 2024-04-12T21:36:11.971 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:11.971+0000 7f4729400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f47241092a0 0x7f4724119f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:36:11.972 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:11.971+0000 7f4729400700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4724109c40 0x7f472411a4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:36:11.972 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:11.971+0000 7f4729400700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f472411aba0 con 0x7f47241092a0 2024-04-12T21:36:11.972 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:11.971+0000 7f4729400700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4724113ee0 con 0x7f4724109c40 2024-04-12T21:36:11.972 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:11.971+0000 7f4723e00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f47241092a0 0x7f4724119f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:36:11.972 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:11.971+0000 7f4723e00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f47241092a0 0x7f4724119f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:34518/0 (socket says 172.21.15.18:34518) 2024-04-12T21:36:11.972 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:11.971+0000 7f4723e00700 1 -- 172.21.15.18:0/953126627 learned_addr learned my addr 172.21.15.18:0/953126627 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:36:11.973 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:11.972+0000 7f4723400700 1 --2- 172.21.15.18:0/953126627 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4724109c40 0x7f472411a4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:36:11.973 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:11.972+0000 7f4723e00700 1 -- 172.21.15.18:0/953126627 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4724109c40 msgr2=0x7f472411a4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:36:11.973 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:11.972+0000 7f4723e00700 1 --2- 172.21.15.18:0/953126627 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4724109c40 0x7f472411a4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:36:11.973 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:11.972+0000 7f4723e00700 1 -- 172.21.15.18:0/953126627 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4714005040 con 0x7f47241092a0 2024-04-12T21:36:11.973 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:11.972+0000 7f4723400700 1 --2- 172.21.15.18:0/953126627 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4724109c40 0x7f472411a4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-12T21:36:11.973 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:11.972+0000 7f4723e00700 1 --2- 172.21.15.18:0/953126627 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f47241092a0 0x7f4724119f80 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 crypto rx=0x7f4714009150 tx=0x7f4714009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:36:11.973 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:11.972+0000 7f471be00700 1 -- 172.21.15.18:0/953126627 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f471400e040 con 0x7f47241092a0 2024-04-12T21:36:11.974 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:11.972+0000 7f471be00700 1 -- 172.21.15.18:0/953126627 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f4714013420 con 0x7f47241092a0 2024-04-12T21:36:11.974 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:11.972+0000 7f4729400700 1 -- 172.21.15.18:0/953126627 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4724114100 con 0x7f47241092a0 2024-04-12T21:36:11.974 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:11.972+0000 7f471be00700 1 -- 172.21.15.18:0/953126627 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f471401c630 con 0x7f47241092a0 2024-04-12T21:36:11.974 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:11.972+0000 7f4729400700 1 -- 172.21.15.18:0/953126627 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f47241146c0 con 0x7f47241092a0 2024-04-12T21:36:11.974 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:11.974+0000 7f4729400700 1 -- 172.21.15.18:0/953126627 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f47241049f0 con 0x7f47241092a0 2024-04-12T21:36:11.975 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:11.975+0000 7f471be00700 1 -- 172.21.15.18:0/953126627 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7f4714013590 con 0x7f47241092a0 2024-04-12T21:36:11.976 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:11.976+0000 7f471be00700 1 --2- 172.21.15.18:0/953126627 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f470806ec00 0x7f47080710b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:36:11.976 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:11.976+0000 7f471be00700 1 -- 172.21.15.18:0/953126627 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 6757+0+0 (secure 0 0 0) 0x7f471408fce0 con 0x7f47241092a0 2024-04-12T21:36:11.976 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:11.976+0000 7f4723400700 1 --2- 172.21.15.18:0/953126627 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f470806ec00 0x7f47080710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:36:11.977 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:11.977+0000 7f4723400700 1 --2- 172.21.15.18:0/953126627 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f470806ec00 0x7f47080710b0 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f4710006fd0 tx=0x7f4710008040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:36:11.981 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:11.981+0000 7f471be00700 1 -- 172.21.15.18:0/953126627 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f471405c490 con 0x7f47241092a0 2024-04-12T21:36:12.103 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:36:11 smithi018 ceph-mon[112590]: from='client.35904 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:36:12.104 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:36:11 smithi018 ceph-mon[112590]: from='client.35908 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:36:12.104 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:36:11 smithi018 ceph-mon[112590]: from='client.? 172.21.15.18:0/297969574' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T21:36:12.257 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:12.256+0000 7f4729400700 1 -- 172.21.15.18:0/953126627 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f472404bf60 con 0x7f47241092a0 2024-04-12T21:36:12.257 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:12.257+0000 7f471be00700 1 -- 172.21.15.18:0/953126627 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 31 v31) v1 ==== 76+0+1873 (secure 0 0 0) 0x7f471405c020 con 0x7f47241092a0 2024-04-12T21:36:12.258 INFO:teuthology.orchestra.run.smithi018.stdout:e31 2024-04-12T21:36:12.258 INFO:teuthology.orchestra.run.smithi018.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-12T21:36:12.258 INFO:teuthology.orchestra.run.smithi018.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-12T21:36:12.258 INFO:teuthology.orchestra.run.smithi018.stdout:legacy client fscid: 1 2024-04-12T21:36:12.259 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-12T21:36:12.259 INFO:teuthology.orchestra.run.smithi018.stdout:Filesystem 'cephfs' (1) 2024-04-12T21:36:12.259 INFO:teuthology.orchestra.run.smithi018.stdout:fs_name cephfs 2024-04-12T21:36:12.259 INFO:teuthology.orchestra.run.smithi018.stdout:epoch 24 2024-04-12T21:36:12.259 INFO:teuthology.orchestra.run.smithi018.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-04-12T21:36:12.259 INFO:teuthology.orchestra.run.smithi018.stdout:created 2024-04-12T20:51:57.362850+0000 2024-04-12T21:36:12.259 INFO:teuthology.orchestra.run.smithi018.stdout:modified 2024-04-12T21:06:59.717190+0000 2024-04-12T21:36:12.259 INFO:teuthology.orchestra.run.smithi018.stdout:tableserver 0 2024-04-12T21:36:12.259 INFO:teuthology.orchestra.run.smithi018.stdout:root 0 2024-04-12T21:36:12.259 INFO:teuthology.orchestra.run.smithi018.stdout:session_timeout 60 2024-04-12T21:36:12.259 INFO:teuthology.orchestra.run.smithi018.stdout:session_autoclose 300 2024-04-12T21:36:12.259 INFO:teuthology.orchestra.run.smithi018.stdout:max_file_size 1099511627776 2024-04-12T21:36:12.259 INFO:teuthology.orchestra.run.smithi018.stdout:max_xattr_size 65536 2024-04-12T21:36:12.259 INFO:teuthology.orchestra.run.smithi018.stdout:required_client_features {} 2024-04-12T21:36:12.259 INFO:teuthology.orchestra.run.smithi018.stdout:last_failure 0 2024-04-12T21:36:12.260 INFO:teuthology.orchestra.run.smithi018.stdout:last_failure_osd_epoch 100 2024-04-12T21:36:12.260 INFO:teuthology.orchestra.run.smithi018.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-12T21:36:12.260 INFO:teuthology.orchestra.run.smithi018.stdout:max_mds 2 2024-04-12T21:36:12.260 INFO:teuthology.orchestra.run.smithi018.stdout:in 0,1 2024-04-12T21:36:12.260 INFO:teuthology.orchestra.run.smithi018.stdout:up {} 2024-04-12T21:36:12.260 INFO:teuthology.orchestra.run.smithi018.stdout:failed 0,1 2024-04-12T21:36:12.260 INFO:teuthology.orchestra.run.smithi018.stdout:damaged 2024-04-12T21:36:12.260 INFO:teuthology.orchestra.run.smithi018.stdout:stopped 2024-04-12T21:36:12.260 INFO:teuthology.orchestra.run.smithi018.stdout:data_pools [3] 2024-04-12T21:36:12.260 INFO:teuthology.orchestra.run.smithi018.stdout:metadata_pool 2 2024-04-12T21:36:12.260 INFO:teuthology.orchestra.run.smithi018.stdout:inline_data disabled 2024-04-12T21:36:12.260 INFO:teuthology.orchestra.run.smithi018.stdout:balancer 2024-04-12T21:36:12.261 INFO:teuthology.orchestra.run.smithi018.stdout:bal_rank_mask -1 2024-04-12T21:36:12.261 INFO:teuthology.orchestra.run.smithi018.stdout:standby_count_wanted 1 2024-04-12T21:36:12.261 INFO:teuthology.orchestra.run.smithi018.stdout:qdb_cluster leader: 0 members: 2024-04-12T21:36:12.261 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-12T21:36:12.261 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-12T21:36:12.261 INFO:teuthology.orchestra.run.smithi018.stdout:Standby daemons: 2024-04-12T21:36:12.261 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-12T21:36:12.261 INFO:teuthology.orchestra.run.smithi018.stdout:[mds.cephfs.smithi186.wjnafs{-1:24277} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.186:6826/438347207,v1:172.21.15.186:6827/438347207] compat {c=[1],r=[1],i=[7ff]}] 2024-04-12T21:36:12.261 INFO:teuthology.orchestra.run.smithi018.stdout:[mds.cephfs.smithi018.acqbsc{-1:34436} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.18:6828/755179764,v1:172.21.15.18:6829/755179764] compat {c=[1],r=[1],i=[1fff]}] 2024-04-12T21:36:12.261 INFO:teuthology.orchestra.run.smithi018.stdout:[mds.cephfs.smithi018.szjluo{-1:34468} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.18:6826/2407992653,v1:172.21.15.18:6827/2407992653] compat {c=[1],r=[1],i=[1fff]}] 2024-04-12T21:36:12.261 INFO:teuthology.orchestra.run.smithi018.stdout:[mds.cephfs.smithi186.jdllyn{-1:44595} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.186:6824/3523444202,v1:172.21.15.186:6825/3523444202] compat {c=[1],r=[1],i=[1fff]}] 2024-04-12T21:36:12.262 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:12.260+0000 7f4729400700 1 -- 172.21.15.18:0/953126627 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f470806ec00 msgr2=0x7f47080710b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:36:12.262 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:12.260+0000 7f4729400700 1 --2- 172.21.15.18:0/953126627 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f470806ec00 0x7f47080710b0 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f4710006fd0 tx=0x7f4710008040 comp rx=0 tx=0).stop 2024-04-12T21:36:12.262 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:12.260+0000 7f4729400700 1 -- 172.21.15.18:0/953126627 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f47241092a0 msgr2=0x7f4724119f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:36:12.262 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:12.260+0000 7f4729400700 1 --2- 172.21.15.18:0/953126627 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f47241092a0 0x7f4724119f80 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 crypto rx=0x7f4714009150 tx=0x7f4714009e30 comp rx=0 tx=0).stop 2024-04-12T21:36:12.262 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:12.260+0000 7f4729400700 1 -- 172.21.15.18:0/953126627 shutdown_connections 2024-04-12T21:36:12.262 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:12.260+0000 7f4729400700 1 --2- 172.21.15.18:0/953126627 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f470806ec00 0x7f47080710b0 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:36:12.263 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:12.260+0000 7f4729400700 1 --2- 172.21.15.18:0/953126627 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f47241092a0 0x7f4724119f80 unknown :-1 s=CLOSED pgs=927 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:36:12.263 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:12.260+0000 7f4729400700 1 --2- 172.21.15.18:0/953126627 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4724109c40 0x7f472411a4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:36:12.263 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:12.260+0000 7f4729400700 1 -- 172.21.15.18:0/953126627 >> 172.21.15.18:0/953126627 conn(0x7f4724072a30 msgr2=0x7f47240748f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:36:12.263 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:12.260+0000 7f4729400700 1 -- 172.21.15.18:0/953126627 shutdown_connections 2024-04-12T21:36:12.263 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:12.261+0000 7f4729400700 1 -- 172.21.15.18:0/953126627 wait complete. 2024-04-12T21:36:12.263 INFO:teuthology.orchestra.run.smithi018.stderr:dumped fsmap epoch 31 2024-04-12T21:36:12.282 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:36:11 smithi186 ceph-mon[86379]: from='client.35904 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:36:12.282 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:36:11 smithi186 ceph-mon[86379]: from='client.35908 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:36:12.282 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:36:11 smithi186 ceph-mon[86379]: from='client.? 172.21.15.18:0/297969574' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T21:36:12.406 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:12.405+0000 7fcd08c00700 1 -- 172.21.15.18:0/3941993251 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fcd04070300 msgr2=0x7fcd040706d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:36:12.407 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:12.405+0000 7fcd08c00700 1 --2- 172.21.15.18:0/3941993251 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fcd04070300 0x7fcd040706d0 secure :-1 s=READY pgs=928 cs=0 l=1 rev1=1 crypto rx=0x7fccf4004970 tx=0x7fccf4009e30 comp rx=0 tx=0).stop 2024-04-12T21:36:12.407 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:12.406+0000 7fcd08c00700 1 -- 172.21.15.18:0/3941993251 shutdown_connections 2024-04-12T21:36:12.407 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:12.406+0000 7fcd08c00700 1 --2- 172.21.15.18:0/3941993251 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcd04070c10 0x7fcd04109170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:36:12.407 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:12.406+0000 7fcd08c00700 1 --2- 172.21.15.18:0/3941993251 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fcd04070300 0x7fcd040706d0 unknown :-1 s=CLOSED pgs=928 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:36:12.407 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:12.406+0000 7fcd08c00700 1 -- 172.21.15.18:0/3941993251 >> 172.21.15.18:0/3941993251 conn(0x7fcd040f89b0 msgr2=0x7fcd040fadc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:36:12.407 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:12.406+0000 7fcd08c00700 1 -- 172.21.15.18:0/3941993251 shutdown_connections 2024-04-12T21:36:12.407 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:12.406+0000 7fcd08c00700 1 -- 172.21.15.18:0/3941993251 wait complete. 2024-04-12T21:36:12.407 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:12.406+0000 7fcd08c00700 1 Processor -- start 2024-04-12T21:36:12.407 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:12.407+0000 7fcd08c00700 1 -- start start 2024-04-12T21:36:12.408 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:12.407+0000 7fcd08c00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fcd04070300 0x7fcd04115af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:36:12.409 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:12.407+0000 7fcd08c00700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcd04070c10 0x7fcd04116030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:36:12.409 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:12.407+0000 7fcd08c00700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd04116680 con 0x7fcd04070300 2024-04-12T21:36:12.409 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:12.407+0000 7fcd08c00700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd0410fca0 con 0x7fcd04070c10 2024-04-12T21:36:12.409 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:12.407+0000 7fcd01e00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fcd04070300 0x7fcd04115af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:36:12.409 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:12.407+0000 7fcd01e00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fcd04070300 0x7fcd04115af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:34530/0 (socket says 172.21.15.18:34530) 2024-04-12T21:36:12.409 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:12.407+0000 7fcd01e00700 1 -- 172.21.15.18:0/1080860977 learned_addr learned my addr 172.21.15.18:0/1080860977 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:36:12.409 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:12.408+0000 7fcd01e00700 1 -- 172.21.15.18:0/1080860977 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcd04070c10 msgr2=0x7fcd04116030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:36:12.409 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:12.408+0000 7fcd01e00700 1 --2- 172.21.15.18:0/1080860977 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcd04070c10 0x7fcd04116030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:36:12.409 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:12.408+0000 7fcd01e00700 1 -- 172.21.15.18:0/1080860977 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fccf4005040 con 0x7fcd04070300 2024-04-12T21:36:12.409 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:12.408+0000 7fcd01e00700 1 --2- 172.21.15.18:0/1080860977 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fcd04070300 0x7fcd04115af0 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 crypto rx=0x7fccf4004f50 tx=0x7fccf4009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:36:12.409 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:12.408+0000 7fccfa000700 1 -- 172.21.15.18:0/1080860977 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fccf400e040 con 0x7fcd04070300 2024-04-12T21:36:12.409 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:12.408+0000 7fccfa000700 1 -- 172.21.15.18:0/1080860977 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fccf400bc10 con 0x7fcd04070300 2024-04-12T21:36:12.410 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:12.408+0000 7fcd08c00700 1 -- 172.21.15.18:0/1080860977 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcd0410ff20 con 0x7fcd04070300 2024-04-12T21:36:12.410 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:12.408+0000 7fccfa000700 1 -- 172.21.15.18:0/1080860977 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fccf401c630 con 0x7fcd04070300 2024-04-12T21:36:12.410 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:12.408+0000 7fcd08c00700 1 -- 172.21.15.18:0/1080860977 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcd04110460 con 0x7fcd04070300 2024-04-12T21:36:12.410 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:12.409+0000 7fcd08c00700 1 -- 172.21.15.18:0/1080860977 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcd04071a40 con 0x7fcd04070300 2024-04-12T21:36:12.410 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:12.411+0000 7fccfa000700 1 -- 172.21.15.18:0/1080860977 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7fccf4013420 con 0x7fcd04070300 2024-04-12T21:36:12.412 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:12.411+0000 7fccfa000700 1 --2- 172.21.15.18:0/1080860977 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fcce806ec00 0x7fcce80710b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:36:12.412 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:12.412+0000 7fccfa000700 1 -- 172.21.15.18:0/1080860977 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 6757+0+0 (secure 0 0 0) 0x7fccf408fd30 con 0x7fcd04070300 2024-04-12T21:36:12.412 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:12.412+0000 7fcd01400700 1 --2- 172.21.15.18:0/1080860977 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fcce806ec00 0x7fcce80710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:36:12.412 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:12.412+0000 7fcd01400700 1 --2- 172.21.15.18:0/1080860977 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fcce806ec00 0x7fcce80710b0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7fcd041113d0 tx=0x7fccf0007d40 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:36:12.416 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:12.416+0000 7fccfa000700 1 -- 172.21.15.18:0/1080860977 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7fccf40589b0 con 0x7fcd04070300 2024-04-12T21:36:12.653 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:12.653+0000 7fcd08c00700 1 -- 172.21.15.18:0/1080860977 --> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fcd0410c160 con 0x7fcce806ec00 2024-04-12T21:36:12.655 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:12.655+0000 7fccfa000700 1 -- 172.21.15.18:0/1080860977 <== mgr.34102 v2:172.21.15.18:6800/3812891919 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7fcd0410c160 con 0x7fcce806ec00 2024-04-12T21:36:12.655 INFO:teuthology.orchestra.run.smithi018.stdout:{ 2024-04-12T21:36:12.655 INFO:teuthology.orchestra.run.smithi018.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:f4452def3be4d350f4a8e5feaf7ebb1511be9298fe68478ba7a2d6fbd7d69a63", 2024-04-12T21:36:12.655 INFO:teuthology.orchestra.run.smithi018.stdout: "in_progress": true, 2024-04-12T21:36:12.655 INFO:teuthology.orchestra.run.smithi018.stdout: "which": "Upgrading all daemon types on all hosts", 2024-04-12T21:36:12.655 INFO:teuthology.orchestra.run.smithi018.stdout: "services_complete": [ 2024-04-12T21:36:12.656 INFO:teuthology.orchestra.run.smithi018.stdout: "crash", 2024-04-12T21:36:12.656 INFO:teuthology.orchestra.run.smithi018.stdout: "osd", 2024-04-12T21:36:12.656 INFO:teuthology.orchestra.run.smithi018.stdout: "mon", 2024-04-12T21:36:12.656 INFO:teuthology.orchestra.run.smithi018.stdout: "mgr" 2024-04-12T21:36:12.656 INFO:teuthology.orchestra.run.smithi018.stdout: ], 2024-04-12T21:36:12.656 INFO:teuthology.orchestra.run.smithi018.stdout: "progress": "15/21 daemons upgraded", 2024-04-12T21:36:12.656 INFO:teuthology.orchestra.run.smithi018.stdout: "message": "Currently upgrading mds daemons", 2024-04-12T21:36:12.656 INFO:teuthology.orchestra.run.smithi018.stdout: "is_paused": false 2024-04-12T21:36:12.656 INFO:teuthology.orchestra.run.smithi018.stdout:} 2024-04-12T21:36:12.658 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:12.657+0000 7fcd08c00700 1 -- 172.21.15.18:0/1080860977 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fcce806ec00 msgr2=0x7fcce80710b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:36:12.658 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:12.657+0000 7fcd08c00700 1 --2- 172.21.15.18:0/1080860977 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fcce806ec00 0x7fcce80710b0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7fcd041113d0 tx=0x7fccf0007d40 comp rx=0 tx=0).stop 2024-04-12T21:36:12.658 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:12.657+0000 7fcd08c00700 1 -- 172.21.15.18:0/1080860977 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fcd04070300 msgr2=0x7fcd04115af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:36:12.658 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:12.657+0000 7fcd08c00700 1 --2- 172.21.15.18:0/1080860977 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fcd04070300 0x7fcd04115af0 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 crypto rx=0x7fccf4004f50 tx=0x7fccf4009e30 comp rx=0 tx=0).stop 2024-04-12T21:36:12.659 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:12.658+0000 7fcd08c00700 1 -- 172.21.15.18:0/1080860977 shutdown_connections 2024-04-12T21:36:12.659 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:12.658+0000 7fcd08c00700 1 --2- 172.21.15.18:0/1080860977 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fcce806ec00 0x7fcce80710b0 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:36:12.659 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:12.658+0000 7fcd08c00700 1 --2- 172.21.15.18:0/1080860977 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fcd04070300 0x7fcd04115af0 unknown :-1 s=CLOSED pgs=929 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:36:12.659 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:12.658+0000 7fcd08c00700 1 --2- 172.21.15.18:0/1080860977 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcd04070c10 0x7fcd04116030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:36:12.659 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:12.658+0000 7fcd08c00700 1 -- 172.21.15.18:0/1080860977 >> 172.21.15.18:0/1080860977 conn(0x7fcd040f89b0 msgr2=0x7fcd040f9de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:36:12.659 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:12.658+0000 7fcd08c00700 1 -- 172.21.15.18:0/1080860977 shutdown_connections 2024-04-12T21:36:12.659 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:12.658+0000 7fcd08c00700 1 -- 172.21.15.18:0/1080860977 wait complete. 2024-04-12T21:36:12.812 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:12.812+0000 7fcd14e00700 1 -- 172.21.15.18:0/755176740 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fcd10109c40 msgr2=0x7fcd101032b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:36:12.812 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:12.812+0000 7fcd14e00700 1 --2- 172.21.15.18:0/755176740 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fcd10109c40 0x7fcd101032b0 secure :-1 s=READY pgs=930 cs=0 l=1 rev1=1 crypto rx=0x7fccfc004940 tx=0x7fccfc009e30 comp rx=0 tx=0).stop 2024-04-12T21:36:12.813 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:12.812+0000 7fcd14e00700 1 -- 172.21.15.18:0/755176740 shutdown_connections 2024-04-12T21:36:12.813 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:12.812+0000 7fcd14e00700 1 --2- 172.21.15.18:0/755176740 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fcd10109c40 0x7fcd101032b0 unknown :-1 s=CLOSED pgs=930 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:36:12.813 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:12.812+0000 7fcd14e00700 1 --2- 172.21.15.18:0/755176740 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcd101092a0 0x7fcd10109670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:36:12.813 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:12.812+0000 7fcd14e00700 1 -- 172.21.15.18:0/755176740 >> 172.21.15.18:0/755176740 conn(0x7fcd10072a30 msgr2=0x7fcd10074e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:36:12.813 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:12.812+0000 7fcd14e00700 1 -- 172.21.15.18:0/755176740 shutdown_connections 2024-04-12T21:36:12.813 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:12.812+0000 7fcd14e00700 1 -- 172.21.15.18:0/755176740 wait complete. 2024-04-12T21:36:12.814 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:12.813+0000 7fcd14e00700 1 Processor -- start 2024-04-12T21:36:12.814 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:12.813+0000 7fcd14e00700 1 -- start start 2024-04-12T21:36:12.814 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:12.813+0000 7fcd14e00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fcd101092a0 0x7fcd10119fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:36:12.814 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:12.813+0000 7fcd14e00700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcd10109c40 0x7fcd1011a500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:36:12.814 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:12.813+0000 7fcd14e00700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd1011abe0 con 0x7fcd101092a0 2024-04-12T21:36:12.815 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:12.813+0000 7fcd14e00700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd10113f20 con 0x7fcd10109c40 2024-04-12T21:36:12.815 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:12.814+0000 7fcd0f400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fcd101092a0 0x7fcd10119fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:36:12.815 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:12.814+0000 7fcd0f400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fcd101092a0 0x7fcd10119fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:34554/0 (socket says 172.21.15.18:34554) 2024-04-12T21:36:12.815 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:12.814+0000 7fcd0f400700 1 -- 172.21.15.18:0/3024221559 learned_addr learned my addr 172.21.15.18:0/3024221559 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:36:12.815 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:12.814+0000 7fcd0f400700 1 -- 172.21.15.18:0/3024221559 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcd10109c40 msgr2=0x7fcd1011a500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:36:12.815 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:12.814+0000 7fcd0f400700 1 --2- 172.21.15.18:0/3024221559 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcd10109c40 0x7fcd1011a500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:36:12.815 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:12.814+0000 7fcd0f400700 1 -- 172.21.15.18:0/3024221559 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fccfc005040 con 0x7fcd101092a0 2024-04-12T21:36:12.815 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:12.814+0000 7fcd0f400700 1 --2- 172.21.15.18:0/3024221559 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fcd101092a0 0x7fcd10119fc0 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 crypto rx=0x7fcd0000b790 tx=0x7fcd00009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:36:12.816 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:12.814+0000 7fcd07400700 1 -- 172.21.15.18:0/3024221559 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcd0000f770 con 0x7fcd101092a0 2024-04-12T21:36:12.816 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:12.815+0000 7fcd07400700 1 -- 172.21.15.18:0/3024221559 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fcd0000fda0 con 0x7fcd101092a0 2024-04-12T21:36:12.816 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:12.815+0000 7fcd14e00700 1 -- 172.21.15.18:0/3024221559 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcd101141a0 con 0x7fcd101092a0 2024-04-12T21:36:12.816 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:12.815+0000 7fcd07400700 1 -- 172.21.15.18:0/3024221559 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcd0000d310 con 0x7fcd101092a0 2024-04-12T21:36:12.816 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:12.815+0000 7fcd14e00700 1 -- 172.21.15.18:0/3024221559 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcd10114760 con 0x7fcd101092a0 2024-04-12T21:36:12.816 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:12.816+0000 7fcd14e00700 1 -- 172.21.15.18:0/3024221559 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcd101049f0 con 0x7fcd101092a0 2024-04-12T21:36:12.817 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:12.817+0000 7fcd07400700 1 -- 172.21.15.18:0/3024221559 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7fcd00010420 con 0x7fcd101092a0 2024-04-12T21:36:12.818 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:12.818+0000 7fcd07400700 1 --2- 172.21.15.18:0/3024221559 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fccf406ec00 0x7fccf40710b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:36:12.818 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:12.818+0000 7fcd07400700 1 -- 172.21.15.18:0/3024221559 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 6757+0+0 (secure 0 0 0) 0x7fcd0008e9a0 con 0x7fcd101092a0 2024-04-12T21:36:12.818 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:12.818+0000 7fcd0ea00700 1 --2- 172.21.15.18:0/3024221559 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fccf406ec00 0x7fccf40710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:36:12.819 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:12.819+0000 7fcd0ea00700 1 --2- 172.21.15.18:0/3024221559 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fccf406ec00 0x7fccf40710b0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7fccfc004f90 tx=0x7fccfc009e30 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:36:12.823 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:12.823+0000 7fcd07400700 1 -- 172.21.15.18:0/3024221559 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7fcd0005b1d0 con 0x7fcd101092a0 2024-04-12T21:36:13.150 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:13.149+0000 7fcd14e00700 1 -- 172.21.15.18:0/3024221559 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7fcd10110420 con 0x7fcd101092a0 2024-04-12T21:36:13.150 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:13.150+0000 7fcd07400700 1 -- 172.21.15.18:0/3024221559 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+4303 (secure 0 0 0) 0x7fcd0005ad60 con 0x7fcd101092a0 2024-04-12T21:36:13.152 INFO:teuthology.orchestra.run.smithi018.stdout:HEALTH_ERR failed to probe daemons or devices; 1 filesystem is degraded; 1 filesystem is offline 2024-04-12T21:36:13.152 INFO:teuthology.orchestra.run.smithi018.stdout:[WRN] CEPHADM_REFRESH_FAILED: failed to probe daemons or devices 2024-04-12T21:36:13.152 INFO:teuthology.orchestra.run.smithi018.stdout: host smithi018 `cephadm ceph-volume` failed: cephadm exited with an error code: 1, stderr: Inferring config /var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/mon.smithi018/config 2024-04-12T21:36:13.152 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/podman: timeout after 895 seconds 2024-04-12T21:36:13.152 INFO:teuthology.orchestra.run.smithi018.stdout:Non-zero exit code 124 from /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:f4452def3be4d350f4a8e5feaf7ebb1511be9298fe68478ba7a2d6fbd7d69a63 -e NODE_NAME=smithi018 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968:/var/run/ceph:z -v /var/log/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968:/var/log/ceph:z -v /var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmpxsmwkkuh:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:f4452def3be4d350f4a8e5feaf7ebb1511be9298fe68478ba7a2d6fbd7d69a63 inventory --format=json-pretty --filter-for-batch 2024-04-12T21:36:13.152 INFO:teuthology.orchestra.run.smithi018.stdout:Traceback (most recent call last): 2024-04-12T21:36:13.152 INFO:teuthology.orchestra.run.smithi018.stdout: File "/usr/lib64/python3.9/runpy.py", line 197, in _run_module_as_main 2024-04-12T21:36:13.152 INFO:teuthology.orchestra.run.smithi018.stdout: return _run_code(code, main_globals, None, 2024-04-12T21:36:13.152 INFO:teuthology.orchestra.run.smithi018.stdout: File "/usr/lib64/python3.9/runpy.py", line 87, in _run_code 2024-04-12T21:36:13.152 INFO:teuthology.orchestra.run.smithi018.stdout: exec(code, run_globals) 2024-04-12T21:36:13.152 INFO:teuthology.orchestra.run.smithi018.stdout: File "/var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/cephadm.e44f3ed5e34e4ef52eedef233d1190bff004470d8e8005127b5312bff357fa7f/__main__.py", line 5618, in 2024-04-12T21:36:13.152 INFO:teuthology.orchestra.run.smithi018.stdout: File "/var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/cephadm.e44f3ed5e34e4ef52eedef233d1190bff004470d8e8005127b5312bff357fa7f/__main__.py", line 5606, in main 2024-04-12T21:36:13.153 INFO:teuthology.orchestra.run.smithi018.stdout: File "/var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/cephadm.e44f3ed5e34e4ef52eedef233d1190bff004470d8e8005127b5312bff357fa7f/__main__.py", line 408, in _infer_config 2024-04-12T21:36:13.153 INFO:teuthology.orchestra.run.smithi018.stdout: File "/var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/cephadm.e44f3ed5e34e4ef52eedef233d1190bff004470d8e8005127b5312bff357fa7f/__main__.py", line 323, in _infer_fsid 2024-04-12T21:36:13.153 INFO:teuthology.orchestra.run.smithi018.stdout: File "/var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/cephadm.e44f3ed5e34e4ef52eedef233d1190bff004470d8e8005127b5312bff357fa7f/__main__.py", line 436, in _infer_image 2024-04-12T21:36:13.153 INFO:teuthology.orchestra.run.smithi018.stdout: File "/var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/cephadm.e44f3ed5e34e4ef52eedef233d1190bff004470d8e8005127b5312bff357fa7f/__main__.py", line 310, in _validate_fsid 2024-04-12T21:36:13.153 INFO:teuthology.orchestra.run.smithi018.stdout: File "/var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/cephadm.e44f3ed5e34e4ef52eedef233d1190bff004470d8e8005127b5312bff357fa7f/__main__.py", line 3359, in command_ceph_volume 2024-04-12T21:36:13.153 INFO:teuthology.orchestra.run.smithi018.stdout: File "/var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/cephadm.e44f3ed5e34e4ef52eedef233d1190bff004470d8e8005127b5312bff357fa7f/cephadmlib/call_wrappers.py", line 310, in call_throws 2024-04-12T21:36:13.153 INFO:teuthology.orchestra.run.smithi018.stdout:RuntimeError: Failed command: /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:f4452def3be4d350f4a8e5feaf7ebb1511be9298fe68478ba7a2d6fbd7d69a63 -e NODE_NAME=smithi018 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968:/var/run/ceph:z -v /var/log/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968:/var/log/ceph:z -v /var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmpxsmwkkuh:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:f4452def3be4d350f4a8e5feaf7ebb1511be9298fe68478ba7a2d6fbd7d69a63 inventory --format=json-pretty --filter-for-batch 2024-04-12T21:36:13.153 INFO:teuthology.orchestra.run.smithi018.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-04-12T21:36:13.153 INFO:teuthology.orchestra.run.smithi018.stdout: fs cephfs is degraded 2024-04-12T21:36:13.153 INFO:teuthology.orchestra.run.smithi018.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-04-12T21:36:13.153 INFO:teuthology.orchestra.run.smithi018.stdout: fs cephfs is offline because no MDS is active for it. 2024-04-12T21:36:13.154 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:13.153+0000 7fcd14e00700 1 -- 172.21.15.18:0/3024221559 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fccf406ec00 msgr2=0x7fccf40710b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:36:13.154 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:13.153+0000 7fcd14e00700 1 --2- 172.21.15.18:0/3024221559 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fccf406ec00 0x7fccf40710b0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7fccfc004f90 tx=0x7fccfc009e30 comp rx=0 tx=0).stop 2024-04-12T21:36:13.155 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:13.153+0000 7fcd14e00700 1 -- 172.21.15.18:0/3024221559 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fcd101092a0 msgr2=0x7fcd10119fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:36:13.155 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:13.153+0000 7fcd14e00700 1 --2- 172.21.15.18:0/3024221559 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fcd101092a0 0x7fcd10119fc0 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 crypto rx=0x7fcd0000b790 tx=0x7fcd00009250 comp rx=0 tx=0).stop 2024-04-12T21:36:13.155 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:13.153+0000 7fcd14e00700 1 -- 172.21.15.18:0/3024221559 shutdown_connections 2024-04-12T21:36:13.155 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:13.153+0000 7fcd14e00700 1 --2- 172.21.15.18:0/3024221559 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fccf406ec00 0x7fccf40710b0 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:36:13.155 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:13.153+0000 7fcd14e00700 1 --2- 172.21.15.18:0/3024221559 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fcd101092a0 0x7fcd10119fc0 unknown :-1 s=CLOSED pgs=931 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:36:13.155 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:13.153+0000 7fcd14e00700 1 --2- 172.21.15.18:0/3024221559 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcd10109c40 0x7fcd1011a500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:36:13.155 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:13.153+0000 7fcd14e00700 1 -- 172.21.15.18:0/3024221559 >> 172.21.15.18:0/3024221559 conn(0x7fcd10072a30 msgr2=0x7fcd10074900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:36:13.155 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:13.153+0000 7fcd14e00700 1 -- 172.21.15.18:0/3024221559 shutdown_connections 2024-04-12T21:36:13.155 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:13.154+0000 7fcd14e00700 1 -- 172.21.15.18:0/3024221559 wait complete. 2024-04-12T21:36:13.283 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:36:12 smithi186 ceph-mon[86379]: from='client.35912 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:36:13.283 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:36:12 smithi186 ceph-mon[86379]: pgmap v1093: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:36:13.283 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:36:12 smithi186 ceph-mon[86379]: from='client.? 172.21.15.18:0/953126627' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-12T21:36:13.354 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:36:12 smithi018 ceph-mon[112590]: from='client.35912 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:36:13.354 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:36:12 smithi018 ceph-mon[112590]: pgmap v1093: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:36:13.354 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:36:12 smithi018 ceph-mon[112590]: from='client.? 172.21.15.18:0/953126627' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-12T21:36:14.283 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:36:13 smithi186 ceph-mon[86379]: from='client.35924 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:36:14.283 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:36:13 smithi186 ceph-mon[86379]: from='client.? 172.21.15.18:0/3024221559' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-12T21:36:14.354 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:36:13 smithi018 ceph-mon[112590]: from='client.35924 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:36:14.354 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:36:13 smithi018 ceph-mon[112590]: from='client.? 172.21.15.18:0/3024221559' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-12T21:36:15.255 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:36:14 smithi018 ceph-mon[112590]: pgmap v1094: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:36:15.282 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:36:14 smithi186 ceph-mon[86379]: pgmap v1094: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:36:17.282 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:36:16 smithi186 ceph-mon[86379]: pgmap v1095: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:36:17.353 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:36:16 smithi018 ceph-mon[112590]: pgmap v1095: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:36:18.282 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:36:17 smithi186 ceph-mon[86379]: pgmap v1096: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:36:18.353 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:36:17 smithi018 ceph-mon[112590]: pgmap v1096: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:36:21.103 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:36:20 smithi018 ceph-mon[112590]: pgmap v1097: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:36:21.283 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:36:20 smithi186 ceph-mon[86379]: pgmap v1097: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:36:23.103 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:36:22 smithi018 ceph-mon[112590]: pgmap v1098: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:36:23.282 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:36:22 smithi186 ceph-mon[86379]: pgmap v1098: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:36:25.282 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:36:24 smithi186 ceph-mon[86379]: pgmap v1099: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:36:25.353 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:36:24 smithi018 ceph-mon[112590]: pgmap v1099: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:36:26.282 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:36:25 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T21:36:26.354 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:36:25 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T21:36:27.282 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:36:26 smithi186 ceph-mon[86379]: pgmap v1100: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:36:27.353 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:36:26 smithi018 ceph-mon[112590]: pgmap v1100: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:36:29.282 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:36:28 smithi186 ceph-mon[86379]: pgmap v1101: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:36:29.353 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:36:28 smithi018 ceph-mon[112590]: pgmap v1101: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:36:31.282 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:36:30 smithi186 ceph-mon[86379]: pgmap v1102: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:36:31.354 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:36:30 smithi018 ceph-mon[112590]: pgmap v1102: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:36:33.282 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:36:32 smithi186 ceph-mon[86379]: pgmap v1103: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:36:33.353 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:36:32 smithi018 ceph-mon[112590]: pgmap v1103: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:36:35.282 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:36:34 smithi186 ceph-mon[86379]: pgmap v1104: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:36:35.353 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:36:34 smithi018 ceph-mon[112590]: pgmap v1104: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:36:37.282 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:36:36 smithi186 ceph-mon[86379]: pgmap v1105: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:36:37.354 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:36:36 smithi018 ceph-mon[112590]: pgmap v1105: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:36:39.283 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:36:38 smithi186 ceph-mon[86379]: pgmap v1106: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:36:39.353 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:36:38 smithi018 ceph-mon[112590]: pgmap v1106: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:36:40.282 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:36:39 smithi186 ceph-mon[86379]: pgmap v1107: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:36:40.283 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:36:39 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T21:36:40.353 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:36:39 smithi018 ceph-mon[112590]: pgmap v1107: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:36:40.353 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:36:39 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T21:36:43.282 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:36:42 smithi186 ceph-mon[86379]: pgmap v1108: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:36:43.327 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:43.327+0000 7fdd59a00700 1 -- 172.21.15.18:0/390690848 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fdd54070c70 msgr2=0x7fdd541090f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:36:43.327 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:43.327+0000 7fdd59a00700 1 --2- 172.21.15.18:0/390690848 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fdd54070c70 0x7fdd541090f0 secure :-1 s=READY pgs=932 cs=0 l=1 rev1=1 crypto rx=0x7fdd40004970 tx=0x7fdd40009e30 comp rx=0 tx=0).stop 2024-04-12T21:36:43.327 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:43.327+0000 7fdd59a00700 1 -- 172.21.15.18:0/390690848 shutdown_connections 2024-04-12T21:36:43.327 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:43.327+0000 7fdd59a00700 1 --2- 172.21.15.18:0/390690848 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fdd54070c70 0x7fdd541090f0 unknown :-1 s=CLOSED pgs=932 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:36:43.327 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:43.327+0000 7fdd59a00700 1 --2- 172.21.15.18:0/390690848 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdd54070360 0x7fdd54070730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:36:43.328 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:43.327+0000 7fdd59a00700 1 -- 172.21.15.18:0/390690848 >> 172.21.15.18:0/390690848 conn(0x7fdd540f8970 msgr2=0x7fdd540fad80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:36:43.328 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:36:42 smithi018 ceph-mon[112590]: pgmap v1108: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:36:43.328 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:43.328+0000 7fdd59a00700 1 -- 172.21.15.18:0/390690848 shutdown_connections 2024-04-12T21:36:43.328 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:43.328+0000 7fdd59a00700 1 -- 172.21.15.18:0/390690848 wait complete. 2024-04-12T21:36:43.329 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:43.329+0000 7fdd59a00700 1 Processor -- start 2024-04-12T21:36:43.329 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:43.330+0000 7fdd59a00700 1 -- start start 2024-04-12T21:36:43.329 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:43.330+0000 7fdd59a00700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdd54070360 0x7fdd54115d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:36:43.330 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:43.330+0000 7fdd59a00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fdd54070c70 0x7fdd54116290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:36:43.330 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:43.330+0000 7fdd59a00700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd541168e0 con 0x7fdd54070c70 2024-04-12T21:36:43.330 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:43.330+0000 7fdd59a00700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd5410fcb0 con 0x7fdd54070360 2024-04-12T21:36:43.331 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:43.331+0000 7fdd52000700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fdd54070c70 0x7fdd54116290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:36:43.331 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:43.331+0000 7fdd52000700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fdd54070c70 0x7fdd54116290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:40848/0 (socket says 172.21.15.18:40848) 2024-04-12T21:36:43.331 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:43.331+0000 7fdd52000700 1 -- 172.21.15.18:0/603964438 learned_addr learned my addr 172.21.15.18:0/603964438 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:36:43.331 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:43.331+0000 7fdd52000700 1 -- 172.21.15.18:0/603964438 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdd54070360 msgr2=0x7fdd54115d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:36:43.331 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:43.331+0000 7fdd52a00700 1 --2- 172.21.15.18:0/603964438 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdd54070360 0x7fdd54115d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:36:43.331 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:43.331+0000 7fdd52000700 1 --2- 172.21.15.18:0/603964438 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdd54070360 0x7fdd54115d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:36:43.331 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:43.331+0000 7fdd52000700 1 -- 172.21.15.18:0/603964438 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd40005040 con 0x7fdd54070c70 2024-04-12T21:36:43.331 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:43.331+0000 7fdd52000700 1 --2- 172.21.15.18:0/603964438 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fdd54070c70 0x7fdd54116290 secure :-1 s=READY pgs=933 cs=0 l=1 rev1=1 crypto rx=0x7fdd40004f50 tx=0x7fdd40009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:36:43.332 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:43.332+0000 7fdd4aa00700 1 -- 172.21.15.18:0/603964438 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdd4000e040 con 0x7fdd54070c70 2024-04-12T21:36:43.332 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:43.332+0000 7fdd4aa00700 1 -- 172.21.15.18:0/603964438 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fdd4000bc10 con 0x7fdd54070c70 2024-04-12T21:36:43.332 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:43.332+0000 7fdd4aa00700 1 -- 172.21.15.18:0/603964438 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdd4001c630 con 0x7fdd54070c70 2024-04-12T21:36:43.332 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:43.332+0000 7fdd59a00700 1 -- 172.21.15.18:0/603964438 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdd5410ff30 con 0x7fdd54070c70 2024-04-12T21:36:43.332 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:43.332+0000 7fdd59a00700 1 -- 172.21.15.18:0/603964438 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdd54110470 con 0x7fdd54070c70 2024-04-12T21:36:43.332 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:43.333+0000 7fdd59a00700 1 -- 172.21.15.18:0/603964438 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdd54071aa0 con 0x7fdd54070c70 2024-04-12T21:36:43.333 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:43.334+0000 7fdd4aa00700 1 -- 172.21.15.18:0/603964438 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7fdd40013420 con 0x7fdd54070c70 2024-04-12T21:36:43.335 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:43.335+0000 7fdd4aa00700 1 --2- 172.21.15.18:0/603964438 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fdd3806ec10 0x7fdd380710c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:36:43.335 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:43.336+0000 7fdd4aa00700 1 -- 172.21.15.18:0/603964438 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 6757+0+0 (secure 0 0 0) 0x7fdd4008fd30 con 0x7fdd54070c70 2024-04-12T21:36:43.335 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:43.336+0000 7fdd52a00700 1 --2- 172.21.15.18:0/603964438 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fdd3806ec10 0x7fdd380710c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:36:43.336 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:43.337+0000 7fdd52a00700 1 --2- 172.21.15.18:0/603964438 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fdd3806ec10 0x7fdd380710c0 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7fdd440041d0 tx=0x7fdd44009040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:36:43.340 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:43.341+0000 7fdd4aa00700 1 -- 172.21.15.18:0/603964438 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7fdd4005c4e0 con 0x7fdd54070c70 2024-04-12T21:36:43.593 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:43.594+0000 7fdd59a00700 1 -- 172.21.15.18:0/603964438 --> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fdd5410c0d0 con 0x7fdd3806ec10 2024-04-12T21:36:43.595 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:43.596+0000 7fdd4aa00700 1 -- 172.21.15.18:0/603964438 <== mgr.34102 v2:172.21.15.18:6800/3812891919 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7fdd5410c0d0 con 0x7fdd3806ec10 2024-04-12T21:36:43.599 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:43.599+0000 7fdd59a00700 1 -- 172.21.15.18:0/603964438 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fdd3806ec10 msgr2=0x7fdd380710c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:36:43.599 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:43.599+0000 7fdd59a00700 1 --2- 172.21.15.18:0/603964438 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fdd3806ec10 0x7fdd380710c0 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7fdd440041d0 tx=0x7fdd44009040 comp rx=0 tx=0).stop 2024-04-12T21:36:43.599 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:43.599+0000 7fdd59a00700 1 -- 172.21.15.18:0/603964438 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fdd54070c70 msgr2=0x7fdd54116290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:36:43.599 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:43.599+0000 7fdd59a00700 1 --2- 172.21.15.18:0/603964438 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fdd54070c70 0x7fdd54116290 secure :-1 s=READY pgs=933 cs=0 l=1 rev1=1 crypto rx=0x7fdd40004f50 tx=0x7fdd40009e30 comp rx=0 tx=0).stop 2024-04-12T21:36:43.599 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:43.599+0000 7fdd59a00700 1 -- 172.21.15.18:0/603964438 shutdown_connections 2024-04-12T21:36:43.599 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:43.599+0000 7fdd59a00700 1 --2- 172.21.15.18:0/603964438 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fdd3806ec10 0x7fdd380710c0 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:36:43.599 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:43.599+0000 7fdd59a00700 1 --2- 172.21.15.18:0/603964438 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdd54070360 0x7fdd54115d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:36:43.599 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:43.599+0000 7fdd59a00700 1 --2- 172.21.15.18:0/603964438 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fdd54070c70 0x7fdd54116290 unknown :-1 s=CLOSED pgs=933 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:36:43.599 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:43.599+0000 7fdd59a00700 1 -- 172.21.15.18:0/603964438 >> 172.21.15.18:0/603964438 conn(0x7fdd540f8970 msgr2=0x7fdd540f9d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:36:43.599 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:43.600+0000 7fdd59a00700 1 -- 172.21.15.18:0/603964438 shutdown_connections 2024-04-12T21:36:43.599 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:43.600+0000 7fdd59a00700 1 -- 172.21.15.18:0/603964438 wait complete. 2024-04-12T21:36:43.615 INFO:teuthology.orchestra.run.smithi018.stdout:true 2024-04-12T21:36:43.756 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:43.756+0000 7fa760a00700 1 -- 172.21.15.18:0/3477674539 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa75c109430 msgr2=0x7fa75c109800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:36:43.756 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:43.756+0000 7fa760a00700 1 --2- 172.21.15.18:0/3477674539 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa75c109430 0x7fa75c109800 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 crypto rx=0x7fa74c004940 tx=0x7fa74c009e30 comp rx=0 tx=0).stop 2024-04-12T21:36:43.756 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:43.756+0000 7fa760a00700 1 -- 172.21.15.18:0/3477674539 shutdown_connections 2024-04-12T21:36:43.756 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:43.756+0000 7fa760a00700 1 --2- 172.21.15.18:0/3477674539 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa75c109dd0 0x7fa75c103440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:36:43.756 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:43.756+0000 7fa760a00700 1 --2- 172.21.15.18:0/3477674539 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa75c109430 0x7fa75c109800 unknown :-1 s=CLOSED pgs=934 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:36:43.756 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:43.756+0000 7fa760a00700 1 -- 172.21.15.18:0/3477674539 >> 172.21.15.18:0/3477674539 conn(0x7fa75c072c00 msgr2=0x7fa75c075010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:36:43.756 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:43.756+0000 7fa760a00700 1 -- 172.21.15.18:0/3477674539 shutdown_connections 2024-04-12T21:36:43.756 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:43.756+0000 7fa760a00700 1 -- 172.21.15.18:0/3477674539 wait complete. 2024-04-12T21:36:43.757 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:43.757+0000 7fa760a00700 1 Processor -- start 2024-04-12T21:36:43.757 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:43.758+0000 7fa760a00700 1 -- start start 2024-04-12T21:36:43.757 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:43.758+0000 7fa760a00700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa75c109430 0x7fa75c119f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:36:43.757 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:43.758+0000 7fa760a00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa75c109dd0 0x7fa75c11a4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:36:43.758 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:43.758+0000 7fa760a00700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa75c11ab20 con 0x7fa75c109dd0 2024-04-12T21:36:43.758 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:43.758+0000 7fa760a00700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa75c114100 con 0x7fa75c109430 2024-04-12T21:36:43.759 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:43.759+0000 7fa759400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa75c109dd0 0x7fa75c11a4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:36:43.759 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:43.759+0000 7fa759400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa75c109dd0 0x7fa75c11a4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:40874/0 (socket says 172.21.15.18:40874) 2024-04-12T21:36:43.759 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:43.759+0000 7fa759400700 1 -- 172.21.15.18:0/946768626 learned_addr learned my addr 172.21.15.18:0/946768626 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:36:43.759 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:43.759+0000 7fa759e00700 1 --2- 172.21.15.18:0/946768626 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa75c109430 0x7fa75c119f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:36:43.759 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:43.759+0000 7fa759400700 1 -- 172.21.15.18:0/946768626 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa75c109430 msgr2=0x7fa75c119f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:36:43.759 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:43.759+0000 7fa759400700 1 --2- 172.21.15.18:0/946768626 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa75c109430 0x7fa75c119f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:36:43.759 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:43.759+0000 7fa759400700 1 -- 172.21.15.18:0/946768626 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa74c005040 con 0x7fa75c109dd0 2024-04-12T21:36:43.759 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:43.759+0000 7fa759e00700 1 --2- 172.21.15.18:0/946768626 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa75c109430 0x7fa75c119f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-12T21:36:43.759 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:43.760+0000 7fa759400700 1 --2- 172.21.15.18:0/946768626 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa75c109dd0 0x7fa75c11a4d0 secure :-1 s=READY pgs=935 cs=0 l=1 rev1=1 crypto rx=0x7fa748002a80 tx=0x7fa748004630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:36:43.760 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:43.760+0000 7fa752000700 1 -- 172.21.15.18:0/946768626 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa748007cc0 con 0x7fa75c109dd0 2024-04-12T21:36:43.760 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:43.760+0000 7fa752000700 1 -- 172.21.15.18:0/946768626 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa74800f040 con 0x7fa75c109dd0 2024-04-12T21:36:43.760 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:43.760+0000 7fa752000700 1 -- 172.21.15.18:0/946768626 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa74800daf0 con 0x7fa75c109dd0 2024-04-12T21:36:43.760 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:43.760+0000 7fa760a00700 1 -- 172.21.15.18:0/946768626 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa75c1143e0 con 0x7fa75c109dd0 2024-04-12T21:36:43.760 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:43.760+0000 7fa760a00700 1 -- 172.21.15.18:0/946768626 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa75c1149a0 con 0x7fa75c109dd0 2024-04-12T21:36:43.761 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:43.762+0000 7fa760a00700 1 -- 172.21.15.18:0/946768626 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa75c104b80 con 0x7fa75c109dd0 2024-04-12T21:36:43.762 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:43.763+0000 7fa752000700 1 -- 172.21.15.18:0/946768626 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7fa74800dc90 con 0x7fa75c109dd0 2024-04-12T21:36:43.763 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:43.764+0000 7fa752000700 1 --2- 172.21.15.18:0/946768626 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fa74006ec00 0x7fa7400710b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:36:43.763 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:43.764+0000 7fa752000700 1 -- 172.21.15.18:0/946768626 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 6757+0+0 (secure 0 0 0) 0x7fa74808dfd0 con 0x7fa75c109dd0 2024-04-12T21:36:43.763 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:43.764+0000 7fa759e00700 1 --2- 172.21.15.18:0/946768626 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fa74006ec00 0x7fa7400710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:36:43.764 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:43.765+0000 7fa759e00700 1 --2- 172.21.15.18:0/946768626 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fa74006ec00 0x7fa7400710b0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7fa74c00b3b0 tx=0x7fa74c009e30 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:36:43.768 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:43.769+0000 7fa752000700 1 -- 172.21.15.18:0/946768626 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7fa74805a820 con 0x7fa75c109dd0 2024-04-12T21:36:44.021 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:44.022+0000 7fa760a00700 1 -- 172.21.15.18:0/946768626 --> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fa75c1105b0 con 0x7fa74006ec00 2024-04-12T21:36:44.024 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:44.024+0000 7fa752000700 1 -- 172.21.15.18:0/946768626 <== mgr.34102 v2:172.21.15.18:6800/3812891919 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7fa75c1105b0 con 0x7fa74006ec00 2024-04-12T21:36:44.026 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:44.027+0000 7fa760a00700 1 -- 172.21.15.18:0/946768626 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fa74006ec00 msgr2=0x7fa7400710b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:36:44.027 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:44.027+0000 7fa760a00700 1 --2- 172.21.15.18:0/946768626 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fa74006ec00 0x7fa7400710b0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7fa74c00b3b0 tx=0x7fa74c009e30 comp rx=0 tx=0).stop 2024-04-12T21:36:44.027 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:44.027+0000 7fa760a00700 1 -- 172.21.15.18:0/946768626 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa75c109dd0 msgr2=0x7fa75c11a4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:36:44.027 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:44.027+0000 7fa760a00700 1 --2- 172.21.15.18:0/946768626 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa75c109dd0 0x7fa75c11a4d0 secure :-1 s=READY pgs=935 cs=0 l=1 rev1=1 crypto rx=0x7fa748002a80 tx=0x7fa748004630 comp rx=0 tx=0).stop 2024-04-12T21:36:44.027 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:44.027+0000 7fa760a00700 1 -- 172.21.15.18:0/946768626 shutdown_connections 2024-04-12T21:36:44.027 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:44.027+0000 7fa760a00700 1 --2- 172.21.15.18:0/946768626 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fa74006ec00 0x7fa7400710b0 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:36:44.027 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:44.027+0000 7fa760a00700 1 --2- 172.21.15.18:0/946768626 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa75c109430 0x7fa75c119f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:36:44.027 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:44.027+0000 7fa760a00700 1 --2- 172.21.15.18:0/946768626 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa75c109dd0 0x7fa75c11a4d0 unknown :-1 s=CLOSED pgs=935 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:36:44.027 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:44.027+0000 7fa760a00700 1 -- 172.21.15.18:0/946768626 >> 172.21.15.18:0/946768626 conn(0x7fa75c072c00 msgr2=0x7fa75c074c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:36:44.027 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:44.027+0000 7fa760a00700 1 -- 172.21.15.18:0/946768626 shutdown_connections 2024-04-12T21:36:44.027 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:44.028+0000 7fa760a00700 1 -- 172.21.15.18:0/946768626 wait complete. 2024-04-12T21:36:44.181 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:44.181+0000 7fc8eaa00700 1 -- 172.21.15.18:0/2291039750 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc8e40fe830 msgr2=0x7fc8e4106f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:36:44.181 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:44.181+0000 7fc8eaa00700 1 --2- 172.21.15.18:0/2291039750 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc8e40fe830 0x7fc8e4106f00 secure :-1 s=READY pgs=936 cs=0 l=1 rev1=1 crypto rx=0x7fc8d0004940 tx=0x7fc8d0009e30 comp rx=0 tx=0).stop 2024-04-12T21:36:44.181 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:44.182+0000 7fc8eaa00700 1 -- 172.21.15.18:0/2291039750 shutdown_connections 2024-04-12T21:36:44.181 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:44.182+0000 7fc8eaa00700 1 --2- 172.21.15.18:0/2291039750 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc8e40fe830 0x7fc8e4106f00 unknown :-1 s=CLOSED pgs=936 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:36:44.182 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:44.182+0000 7fc8eaa00700 1 --2- 172.21.15.18:0/2291039750 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc8e40fde90 0x7fc8e40fe260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:36:44.182 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:44.182+0000 7fc8eaa00700 1 -- 172.21.15.18:0/2291039750 >> 172.21.15.18:0/2291039750 conn(0x7fc8e4072bc0 msgr2=0x7fc8e4072fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:36:44.182 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:44.182+0000 7fc8eaa00700 1 -- 172.21.15.18:0/2291039750 shutdown_connections 2024-04-12T21:36:44.182 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:44.182+0000 7fc8eaa00700 1 -- 172.21.15.18:0/2291039750 wait complete. 2024-04-12T21:36:44.182 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:44.183+0000 7fc8eaa00700 1 Processor -- start 2024-04-12T21:36:44.182 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:44.183+0000 7fc8eaa00700 1 -- start start 2024-04-12T21:36:44.182 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:44.183+0000 7fc8eaa00700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc8e40fde90 0x7fc8e4115da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:36:44.183 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:44.183+0000 7fc8eaa00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc8e40fe830 0x7fc8e41162e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:36:44.183 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:44.183+0000 7fc8eaa00700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8e4116930 con 0x7fc8e40fe830 2024-04-12T21:36:44.183 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:44.183+0000 7fc8eaa00700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8e410fd00 con 0x7fc8e40fde90 2024-04-12T21:36:44.184 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:44.184+0000 7fc8e3400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc8e40fe830 0x7fc8e41162e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:36:44.184 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:44.184+0000 7fc8e3400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc8e40fe830 0x7fc8e41162e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:40890/0 (socket says 172.21.15.18:40890) 2024-04-12T21:36:44.184 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:44.184+0000 7fc8e3400700 1 -- 172.21.15.18:0/367522765 learned_addr learned my addr 172.21.15.18:0/367522765 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:36:44.184 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:44.184+0000 7fc8e3e00700 1 --2- 172.21.15.18:0/367522765 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc8e40fde90 0x7fc8e4115da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:36:44.184 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:44.184+0000 7fc8e3400700 1 -- 172.21.15.18:0/367522765 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc8e40fde90 msgr2=0x7fc8e4115da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:36:44.184 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:44.184+0000 7fc8e3400700 1 --2- 172.21.15.18:0/367522765 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc8e40fde90 0x7fc8e4115da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:36:44.184 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:44.184+0000 7fc8e3400700 1 -- 172.21.15.18:0/367522765 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8d0005040 con 0x7fc8e40fe830 2024-04-12T21:36:44.185 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:44.184+0000 7fc8e3400700 1 --2- 172.21.15.18:0/367522765 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc8e40fe830 0x7fc8e41162e0 secure :-1 s=READY pgs=937 cs=0 l=1 rev1=1 crypto rx=0x7fc8d0000c00 tx=0x7fc8d0009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:36:44.185 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:44.185+0000 7fc8dbe00700 1 -- 172.21.15.18:0/367522765 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc8d000e040 con 0x7fc8e40fe830 2024-04-12T21:36:44.185 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:44.185+0000 7fc8dbe00700 1 -- 172.21.15.18:0/367522765 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc8d0013420 con 0x7fc8e40fe830 2024-04-12T21:36:44.185 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:44.185+0000 7fc8dbe00700 1 -- 172.21.15.18:0/367522765 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc8d001c610 con 0x7fc8e40fe830 2024-04-12T21:36:44.185 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:44.185+0000 7fc8eaa00700 1 -- 172.21.15.18:0/367522765 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc8e410ff80 con 0x7fc8e40fe830 2024-04-12T21:36:44.185 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:44.185+0000 7fc8eaa00700 1 -- 172.21.15.18:0/367522765 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc8e41104c0 con 0x7fc8e40fe830 2024-04-12T21:36:44.185 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:44.186+0000 7fc8eaa00700 1 -- 172.21.15.18:0/367522765 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc8e4106a20 con 0x7fc8e40fe830 2024-04-12T21:36:44.186 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:44.187+0000 7fc8dbe00700 1 -- 172.21.15.18:0/367522765 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7fc8d0013590 con 0x7fc8e40fe830 2024-04-12T21:36:44.188 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:44.188+0000 7fc8dbe00700 1 --2- 172.21.15.18:0/367522765 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fc8c806ec10 0x7fc8c80710c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:36:44.188 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:44.188+0000 7fc8dbe00700 1 -- 172.21.15.18:0/367522765 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 6757+0+0 (secure 0 0 0) 0x7fc8d008fbf0 con 0x7fc8e40fe830 2024-04-12T21:36:44.188 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:44.188+0000 7fc8e3e00700 1 --2- 172.21.15.18:0/367522765 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fc8c806ec10 0x7fc8c80710c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:36:44.188 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:44.189+0000 7fc8e3e00700 1 --2- 172.21.15.18:0/367522765 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fc8c806ec10 0x7fc8c80710c0 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7fc8d40041d0 tx=0x7fc8d4009040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:36:44.192 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:44.193+0000 7fc8dbe00700 1 -- 172.21.15.18:0/367522765 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7fc8d005c3a0 con 0x7fc8e40fe830 2024-04-12T21:36:44.422 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:44.422+0000 7fc8eaa00700 1 -- 172.21.15.18:0/367522765 --> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fc8e41113f0 con 0x7fc8c806ec10 2024-04-12T21:36:44.453 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:44.453+0000 7fc8dbe00700 1 -- 172.21.15.18:0/367522765 <== mgr.34102 v2:172.21.15.18:6800/3812891919 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7fc8e41113f0 con 0x7fc8c806ec10 2024-04-12T21:36:44.454 INFO:teuthology.orchestra.run.smithi018.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-12T21:36:44.454 INFO:teuthology.orchestra.run.smithi018.stdout:alertmanager.smithi018 smithi018 *:9093,9094 running (37m) 13m ago 51m 15.5M - 0.25.0 c8568f914cd2 c332090068e8 2024-04-12T21:36:44.454 INFO:teuthology.orchestra.run.smithi018.stdout:crash.smithi018 smithi018 running (33m) 13m ago 52m 6665k - 19.0.0-2879-g324baf8d 8717720dc5e5 b7f80d1f7982 2024-04-12T21:36:44.454 INFO:teuthology.orchestra.run.smithi018.stdout:crash.smithi186 smithi186 running (33m) 13m ago 50m 6660k - 19.0.0-2879-g324baf8d 8717720dc5e5 a4f55c81b623 2024-04-12T21:36:44.454 INFO:teuthology.orchestra.run.smithi018.stdout:grafana.smithi018 smithi018 *:3000 running (36m) 13m ago 51m 74.8M - 9.4.12 2bacad6d85d8 b08e3bf176dd 2024-04-12T21:36:44.454 INFO:teuthology.orchestra.run.smithi018.stdout:mds.cephfs.smithi018.acqbsc smithi018 running (29m) 13m ago 44m 16.7M - 19.0.0-2879-g324baf8d 8717720dc5e5 021f99ab24ae 2024-04-12T21:36:44.454 INFO:teuthology.orchestra.run.smithi018.stdout:mds.cephfs.smithi018.szjluo smithi018 running (29m) 13m ago 44m 16.5M - 19.0.0-2879-g324baf8d 8717720dc5e5 0003117479d4 2024-04-12T21:36:44.454 INFO:teuthology.orchestra.run.smithi018.stdout:mds.cephfs.smithi186.jdllyn smithi186 running (13m) 13m ago 44m 11.0M - 19.0.0-2879-g324baf8d 8717720dc5e5 21b1f46eef23 2024-04-12T21:36:44.454 INFO:teuthology.orchestra.run.smithi018.stdout:mds.cephfs.smithi186.wjnafs smithi186 running (44m) 13m ago 44m 19.5M - 17.2.7-827-gd3b5060a 21de9b7369ad 74b115d6d6fd 2024-04-12T21:36:44.454 INFO:teuthology.orchestra.run.smithi018.stdout:mgr.smithi018.xjzwga smithi018 *:8443,9283,8765 running (40m) 13m ago 53m 512M - 19.0.0-2879-g324baf8d 8717720dc5e5 56088305eab7 2024-04-12T21:36:44.454 INFO:teuthology.orchestra.run.smithi018.stdout:mgr.smithi186.kqcfxk smithi186 *:8443,9283,8765 running (38m) 13m ago 50m 459M - 19.0.0-2879-g324baf8d 8717720dc5e5 09cd766e5e0d 2024-04-12T21:36:44.454 INFO:teuthology.orchestra.run.smithi018.stdout:mon.smithi018 smithi018 running (35m) 13m ago 53m 73.1M 2048M 19.0.0-2879-g324baf8d 8717720dc5e5 df3e201dfed6 2024-04-12T21:36:44.454 INFO:teuthology.orchestra.run.smithi018.stdout:mon.smithi186 smithi186 running (34m) 13m ago 50m 47.2M 2048M 19.0.0-2879-g324baf8d 8717720dc5e5 c77498da9b2b 2024-04-12T21:36:44.455 INFO:teuthology.orchestra.run.smithi018.stdout:node-exporter.smithi018 smithi018 *:9100 running (38m) 13m ago 52m 8108k - 1.5.0 0da6a335fe13 326c3d9c6847 2024-04-12T21:36:44.455 INFO:teuthology.orchestra.run.smithi018.stdout:node-exporter.smithi186 smithi186 *:9100 running (38m) 13m ago 50m 8757k - 1.5.0 0da6a335fe13 ec2112baeecf 2024-04-12T21:36:44.455 INFO:teuthology.orchestra.run.smithi018.stdout:osd.0 smithi018 running (33m) 13m ago 49m 123M 1559M 19.0.0-2879-g324baf8d 8717720dc5e5 60b1273ca047 2024-04-12T21:36:44.455 INFO:teuthology.orchestra.run.smithi018.stdout:osd.1 smithi018 running (32m) 13m ago 48m 95.1M 1559M 19.0.0-2879-g324baf8d 8717720dc5e5 170ce1e5fb8d 2024-04-12T21:36:44.455 INFO:teuthology.orchestra.run.smithi018.stdout:osd.2 smithi018 running (31m) 13m ago 48m 94.3M 1559M 19.0.0-2879-g324baf8d 8717720dc5e5 c2cbc6aa2839 2024-04-12T21:36:44.455 INFO:teuthology.orchestra.run.smithi018.stdout:osd.3 smithi186 running (31m) 13m ago 47m 101M 2581M 19.0.0-2879-g324baf8d 8717720dc5e5 0138e8262e4b 2024-04-12T21:36:44.455 INFO:teuthology.orchestra.run.smithi018.stdout:osd.4 smithi186 running (30m) 13m ago 47m 91.0M 2581M 19.0.0-2879-g324baf8d 8717720dc5e5 b20cb63b25c9 2024-04-12T21:36:44.455 INFO:teuthology.orchestra.run.smithi018.stdout:osd.5 smithi186 running (29m) 13m ago 46m 92.0M 2581M 19.0.0-2879-g324baf8d 8717720dc5e5 2513bea2610e 2024-04-12T21:36:44.455 INFO:teuthology.orchestra.run.smithi018.stdout:prometheus.smithi018 smithi018 *:9095 running (37m) 13m ago 51m 67.9M - 2.43.0 a07b618ecd1d f58841127e21 2024-04-12T21:36:44.456 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:44.456+0000 7fc8eaa00700 1 -- 172.21.15.18:0/367522765 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fc8c806ec10 msgr2=0x7fc8c80710c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:36:44.456 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:44.456+0000 7fc8eaa00700 1 --2- 172.21.15.18:0/367522765 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fc8c806ec10 0x7fc8c80710c0 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7fc8d40041d0 tx=0x7fc8d4009040 comp rx=0 tx=0).stop 2024-04-12T21:36:44.456 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:44.456+0000 7fc8eaa00700 1 -- 172.21.15.18:0/367522765 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc8e40fe830 msgr2=0x7fc8e41162e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:36:44.456 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:44.456+0000 7fc8eaa00700 1 --2- 172.21.15.18:0/367522765 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc8e40fe830 0x7fc8e41162e0 secure :-1 s=READY pgs=937 cs=0 l=1 rev1=1 crypto rx=0x7fc8d0000c00 tx=0x7fc8d0009e30 comp rx=0 tx=0).stop 2024-04-12T21:36:44.456 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:44.456+0000 7fc8eaa00700 1 -- 172.21.15.18:0/367522765 shutdown_connections 2024-04-12T21:36:44.456 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:44.456+0000 7fc8eaa00700 1 --2- 172.21.15.18:0/367522765 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fc8c806ec10 0x7fc8c80710c0 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:36:44.456 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:44.456+0000 7fc8eaa00700 1 --2- 172.21.15.18:0/367522765 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc8e40fde90 0x7fc8e4115da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:36:44.457 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:44.456+0000 7fc8eaa00700 1 --2- 172.21.15.18:0/367522765 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc8e40fe830 0x7fc8e41162e0 unknown :-1 s=CLOSED pgs=937 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:36:44.457 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:44.456+0000 7fc8eaa00700 1 -- 172.21.15.18:0/367522765 >> 172.21.15.18:0/367522765 conn(0x7fc8e4072bc0 msgr2=0x7fc8e40fb6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:36:44.457 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:44.457+0000 7fc8eaa00700 1 -- 172.21.15.18:0/367522765 shutdown_connections 2024-04-12T21:36:44.457 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:44.457+0000 7fc8eaa00700 1 -- 172.21.15.18:0/367522765 wait complete. 2024-04-12T21:36:44.614 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:44.614+0000 7fe645200700 1 -- 172.21.15.18:0/3355002627 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe640109dd0 msgr2=0x7fe640103440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:36:44.614 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:44.614+0000 7fe645200700 1 --2- 172.21.15.18:0/3355002627 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe640109dd0 0x7fe640103440 secure :-1 s=READY pgs=938 cs=0 l=1 rev1=1 crypto rx=0x7fe628004940 tx=0x7fe628009e30 comp rx=0 tx=0).stop 2024-04-12T21:36:44.614 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:44.614+0000 7fe645200700 1 -- 172.21.15.18:0/3355002627 shutdown_connections 2024-04-12T21:36:44.614 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:44.614+0000 7fe645200700 1 --2- 172.21.15.18:0/3355002627 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe640109dd0 0x7fe640103440 unknown :-1 s=CLOSED pgs=938 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:36:44.614 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:44.614+0000 7fe645200700 1 --2- 172.21.15.18:0/3355002627 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe640109430 0x7fe640109800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:36:44.614 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:44.614+0000 7fe645200700 1 -- 172.21.15.18:0/3355002627 >> 172.21.15.18:0/3355002627 conn(0x7fe640072c00 msgr2=0x7fe640075010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:36:44.614 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:44.614+0000 7fe645200700 1 -- 172.21.15.18:0/3355002627 shutdown_connections 2024-04-12T21:36:44.614 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:44.614+0000 7fe645200700 1 -- 172.21.15.18:0/3355002627 wait complete. 2024-04-12T21:36:44.616 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:44.615+0000 7fe645200700 1 Processor -- start 2024-04-12T21:36:44.616 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:44.615+0000 7fe645200700 1 -- start start 2024-04-12T21:36:44.616 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:44.616+0000 7fe645200700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe640109430 0x7fe640119f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:36:44.616 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:44.616+0000 7fe645200700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe640109dd0 0x7fe64011a4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:36:44.616 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:44.616+0000 7fe645200700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe64011ab10 con 0x7fe640109430 2024-04-12T21:36:44.616 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:44.616+0000 7fe645200700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6401140f0 con 0x7fe640109dd0 2024-04-12T21:36:44.616 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:44.616+0000 7fe63de00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe640109430 0x7fe640119f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:36:44.616 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:44.616+0000 7fe63de00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe640109430 0x7fe640119f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:40910/0 (socket says 172.21.15.18:40910) 2024-04-12T21:36:44.616 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:44.616+0000 7fe63de00700 1 -- 172.21.15.18:0/1652892496 learned_addr learned my addr 172.21.15.18:0/1652892496 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:36:44.616 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:44.616+0000 7fe637e00700 1 --2- 172.21.15.18:0/1652892496 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe640109dd0 0x7fe64011a4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:36:44.616 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:44.617+0000 7fe63de00700 1 -- 172.21.15.18:0/1652892496 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe640109dd0 msgr2=0x7fe64011a4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:36:44.616 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:44.617+0000 7fe63de00700 1 --2- 172.21.15.18:0/1652892496 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe640109dd0 0x7fe64011a4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:36:44.617 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:44.617+0000 7fe63de00700 1 -- 172.21.15.18:0/1652892496 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe628005040 con 0x7fe640109430 2024-04-12T21:36:44.617 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:44.617+0000 7fe637e00700 1 --2- 172.21.15.18:0/1652892496 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe640109dd0 0x7fe64011a4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-12T21:36:44.617 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:44.617+0000 7fe63de00700 1 --2- 172.21.15.18:0/1652892496 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe640109430 0x7fe640119f80 secure :-1 s=READY pgs=939 cs=0 l=1 rev1=1 crypto rx=0x7fe630002a50 tx=0x7fe6300046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:36:44.617 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:44.617+0000 7fe636000700 1 -- 172.21.15.18:0/1652892496 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe630007cc0 con 0x7fe640109430 2024-04-12T21:36:44.617 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:44.617+0000 7fe636000700 1 -- 172.21.15.18:0/1652892496 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe63000f040 con 0x7fe640109430 2024-04-12T21:36:44.618 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:44.617+0000 7fe636000700 1 -- 172.21.15.18:0/1652892496 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe630009640 con 0x7fe640109430 2024-04-12T21:36:44.618 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:44.617+0000 7fe645200700 1 -- 172.21.15.18:0/1652892496 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe6401143d0 con 0x7fe640109430 2024-04-12T21:36:44.618 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:44.617+0000 7fe645200700 1 -- 172.21.15.18:0/1652892496 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe640114990 con 0x7fe640109430 2024-04-12T21:36:44.618 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:44.619+0000 7fe645200700 1 -- 172.21.15.18:0/1652892496 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe640104b80 con 0x7fe640109430 2024-04-12T21:36:44.619 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:44.620+0000 7fe636000700 1 -- 172.21.15.18:0/1652892496 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7fe6300097a0 con 0x7fe640109430 2024-04-12T21:36:44.620 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:44.621+0000 7fe636000700 1 --2- 172.21.15.18:0/1652892496 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fe62406ec00 0x7fe6240710b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:36:44.620 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:44.621+0000 7fe636000700 1 -- 172.21.15.18:0/1652892496 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 6757+0+0 (secure 0 0 0) 0x7fe63008e760 con 0x7fe640109430 2024-04-12T21:36:44.620 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:44.621+0000 7fe637e00700 1 --2- 172.21.15.18:0/1652892496 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fe62406ec00 0x7fe6240710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:36:44.621 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:44.622+0000 7fe637e00700 1 --2- 172.21.15.18:0/1652892496 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fe62406ec00 0x7fe6240710b0 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7fe628009fd0 tx=0x7fe628009e30 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:36:44.624 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:44.625+0000 7fe636000700 1 -- 172.21.15.18:0/1652892496 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7fe63005af10 con 0x7fe640109430 2024-04-12T21:36:44.944 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:44.945+0000 7fe645200700 1 -- 172.21.15.18:0/1652892496 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fe640115600 con 0x7fe640109430 2024-04-12T21:36:44.945 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:44.946+0000 7fe636000700 1 -- 172.21.15.18:0/1652892496 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+837 (secure 0 0 0) 0x7fe63005aaa0 con 0x7fe640109430 2024-04-12T21:36:44.946 INFO:teuthology.orchestra.run.smithi018.stdout:{ 2024-04-12T21:36:44.946 INFO:teuthology.orchestra.run.smithi018.stdout: "mon": { 2024-04-12T21:36:44.946 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 19.0.0-2879-g324baf8d (324baf8ddb91dea46cac50cb535c306115791355) squid (dev)": 2 2024-04-12T21:36:44.946 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-12T21:36:44.946 INFO:teuthology.orchestra.run.smithi018.stdout: "mgr": { 2024-04-12T21:36:44.946 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 19.0.0-2879-g324baf8d (324baf8ddb91dea46cac50cb535c306115791355) squid (dev)": 2 2024-04-12T21:36:44.946 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-12T21:36:44.946 INFO:teuthology.orchestra.run.smithi018.stdout: "osd": { 2024-04-12T21:36:44.946 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 19.0.0-2879-g324baf8d (324baf8ddb91dea46cac50cb535c306115791355) squid (dev)": 6 2024-04-12T21:36:44.946 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-12T21:36:44.947 INFO:teuthology.orchestra.run.smithi018.stdout: "mds": { 2024-04-12T21:36:44.947 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 17.2.7-827-gd3b5060a (d3b5060a17449edfae9385cdf792afec109fca5d) quincy (stable)": 1, 2024-04-12T21:36:44.947 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 19.0.0-2879-g324baf8d (324baf8ddb91dea46cac50cb535c306115791355) squid (dev)": 3 2024-04-12T21:36:44.947 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-12T21:36:44.947 INFO:teuthology.orchestra.run.smithi018.stdout: "overall": { 2024-04-12T21:36:44.947 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 17.2.7-827-gd3b5060a (d3b5060a17449edfae9385cdf792afec109fca5d) quincy (stable)": 1, 2024-04-12T21:36:44.947 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 19.0.0-2879-g324baf8d (324baf8ddb91dea46cac50cb535c306115791355) squid (dev)": 13 2024-04-12T21:36:44.947 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-04-12T21:36:44.947 INFO:teuthology.orchestra.run.smithi018.stdout:} 2024-04-12T21:36:44.948 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:44.949+0000 7fe645200700 1 -- 172.21.15.18:0/1652892496 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fe62406ec00 msgr2=0x7fe6240710b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:36:44.948 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:44.949+0000 7fe645200700 1 --2- 172.21.15.18:0/1652892496 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fe62406ec00 0x7fe6240710b0 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7fe628009fd0 tx=0x7fe628009e30 comp rx=0 tx=0).stop 2024-04-12T21:36:44.949 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:44.949+0000 7fe645200700 1 -- 172.21.15.18:0/1652892496 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe640109430 msgr2=0x7fe640119f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:36:44.949 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:44.949+0000 7fe645200700 1 --2- 172.21.15.18:0/1652892496 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe640109430 0x7fe640119f80 secure :-1 s=READY pgs=939 cs=0 l=1 rev1=1 crypto rx=0x7fe630002a50 tx=0x7fe6300046b0 comp rx=0 tx=0).stop 2024-04-12T21:36:44.949 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:44.949+0000 7fe645200700 1 -- 172.21.15.18:0/1652892496 shutdown_connections 2024-04-12T21:36:44.949 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:44.949+0000 7fe645200700 1 --2- 172.21.15.18:0/1652892496 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fe62406ec00 0x7fe6240710b0 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:36:44.949 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:44.949+0000 7fe645200700 1 --2- 172.21.15.18:0/1652892496 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe640109430 0x7fe640119f80 unknown :-1 s=CLOSED pgs=939 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:36:44.949 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:44.949+0000 7fe645200700 1 --2- 172.21.15.18:0/1652892496 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe640109dd0 0x7fe64011a4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:36:44.949 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:44.949+0000 7fe645200700 1 -- 172.21.15.18:0/1652892496 >> 172.21.15.18:0/1652892496 conn(0x7fe640072c00 msgr2=0x7fe640074c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:36:44.949 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:44.949+0000 7fe645200700 1 -- 172.21.15.18:0/1652892496 shutdown_connections 2024-04-12T21:36:44.949 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:44.949+0000 7fe645200700 1 -- 172.21.15.18:0/1652892496 wait complete. 2024-04-12T21:36:45.095 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:45.096+0000 7f67fbe00700 1 -- 172.21.15.18:0/2295767269 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f67f4109dd0 msgr2=0x7f67f4103440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:36:45.095 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:45.096+0000 7f67fbe00700 1 --2- 172.21.15.18:0/2295767269 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f67f4109dd0 0x7f67f4103440 secure :-1 s=READY pgs=940 cs=0 l=1 rev1=1 crypto rx=0x7f67e0004940 tx=0x7f67e0009e30 comp rx=0 tx=0).stop 2024-04-12T21:36:45.096 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:45.096+0000 7f67fbe00700 1 -- 172.21.15.18:0/2295767269 shutdown_connections 2024-04-12T21:36:45.096 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:45.096+0000 7f67fbe00700 1 --2- 172.21.15.18:0/2295767269 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f67f4109dd0 0x7f67f4103440 unknown :-1 s=CLOSED pgs=940 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:36:45.096 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:45.096+0000 7f67fbe00700 1 --2- 172.21.15.18:0/2295767269 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f67f4109430 0x7f67f4109800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:36:45.096 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:45.096+0000 7f67fbe00700 1 -- 172.21.15.18:0/2295767269 >> 172.21.15.18:0/2295767269 conn(0x7f67f4072c00 msgr2=0x7f67f4075010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:36:45.096 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:45.096+0000 7f67fbe00700 1 -- 172.21.15.18:0/2295767269 shutdown_connections 2024-04-12T21:36:45.096 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:45.096+0000 7f67fbe00700 1 -- 172.21.15.18:0/2295767269 wait complete. 2024-04-12T21:36:45.097 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:45.097+0000 7f67fbe00700 1 Processor -- start 2024-04-12T21:36:45.098 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:45.097+0000 7f67fbe00700 1 -- start start 2024-04-12T21:36:45.098 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:45.097+0000 7f67fbe00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f67f4109430 0x7f67f4119fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:36:45.098 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:45.097+0000 7f67fbe00700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f67f4109dd0 0x7f67f411a4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:36:45.098 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:45.097+0000 7f67fbe00700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67f411abc0 con 0x7f67f4109430 2024-04-12T21:36:45.098 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:45.097+0000 7f67fbe00700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67f4114110 con 0x7f67f4109dd0 2024-04-12T21:36:45.098 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:45.098+0000 7f67f9400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f67f4109430 0x7f67f4119fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:36:45.098 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:45.098+0000 7f67f9400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f67f4109430 0x7f67f4119fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:40922/0 (socket says 172.21.15.18:40922) 2024-04-12T21:36:45.098 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:45.098+0000 7f67f9400700 1 -- 172.21.15.18:0/3100804443 learned_addr learned my addr 172.21.15.18:0/3100804443 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:36:45.098 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:45.098+0000 7f67f8a00700 1 --2- 172.21.15.18:0/3100804443 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f67f4109dd0 0x7f67f411a4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:36:45.098 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:45.098+0000 7f67f9400700 1 -- 172.21.15.18:0/3100804443 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f67f4109dd0 msgr2=0x7f67f411a4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:36:45.098 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:45.098+0000 7f67f9400700 1 --2- 172.21.15.18:0/3100804443 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f67f4109dd0 0x7f67f411a4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:36:45.098 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:45.098+0000 7f67f9400700 1 -- 172.21.15.18:0/3100804443 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67e0005040 con 0x7f67f4109430 2024-04-12T21:36:45.098 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:45.099+0000 7f67f9400700 1 --2- 172.21.15.18:0/3100804443 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f67f4109430 0x7f67f4119fa0 secure :-1 s=READY pgs=941 cs=0 l=1 rev1=1 crypto rx=0x7f67e800b790 tx=0x7f67e8009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:36:45.099 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:45.099+0000 7f67f1600700 1 -- 172.21.15.18:0/3100804443 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f67e800f770 con 0x7f67f4109430 2024-04-12T21:36:45.099 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:45.099+0000 7f67f1600700 1 -- 172.21.15.18:0/3100804443 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f67e800fda0 con 0x7f67f4109430 2024-04-12T21:36:45.099 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:45.099+0000 7f67f1600700 1 -- 172.21.15.18:0/3100804443 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f67e800d310 con 0x7f67f4109430 2024-04-12T21:36:45.099 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:45.099+0000 7f67fbe00700 1 -- 172.21.15.18:0/3100804443 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f67f4114390 con 0x7f67f4109430 2024-04-12T21:36:45.100 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:45.099+0000 7f67fbe00700 1 -- 172.21.15.18:0/3100804443 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f67f4114950 con 0x7f67f4109430 2024-04-12T21:36:45.100 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:45.100+0000 7f67fbe00700 1 -- 172.21.15.18:0/3100804443 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f67f4104b80 con 0x7f67f4109430 2024-04-12T21:36:45.100 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:45.101+0000 7f67f1600700 1 -- 172.21.15.18:0/3100804443 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7f67e8010420 con 0x7f67f4109430 2024-04-12T21:36:45.101 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:45.102+0000 7f67f1600700 1 --2- 172.21.15.18:0/3100804443 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f67dc06ec00 0x7f67dc0710b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:36:45.101 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:45.102+0000 7f67f1600700 1 -- 172.21.15.18:0/3100804443 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 6757+0+0 (secure 0 0 0) 0x7f67e808e9a0 con 0x7f67f4109430 2024-04-12T21:36:45.101 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:45.102+0000 7f67f8a00700 1 --2- 172.21.15.18:0/3100804443 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f67dc06ec00 0x7f67dc0710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:36:45.102 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:45.103+0000 7f67f8a00700 1 --2- 172.21.15.18:0/3100804443 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f67dc06ec00 0x7f67dc0710b0 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f67e000b3b0 tx=0x7f67e0009e30 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:36:45.106 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:45.107+0000 7f67f1600700 1 -- 172.21.15.18:0/3100804443 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f67e805b1d0 con 0x7f67f4109430 2024-04-12T21:36:45.282 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:36:44 smithi186 ceph-mon[86379]: from='client.35932 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:36:45.283 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:36:44 smithi186 ceph-mon[86379]: pgmap v1109: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:36:45.353 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:36:44 smithi018 ceph-mon[112590]: from='client.35932 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:36:45.354 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:36:44 smithi018 ceph-mon[112590]: pgmap v1109: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:36:45.378 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:45.378+0000 7f67fbe00700 1 -- 172.21.15.18:0/3100804443 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f67f4115620 con 0x7f67f4109430 2024-04-12T21:36:45.378 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:45.379+0000 7f67f1600700 1 -- 172.21.15.18:0/3100804443 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 31 v31) v1 ==== 76+0+1873 (secure 0 0 0) 0x7f67e805ad60 con 0x7f67f4109430 2024-04-12T21:36:45.380 INFO:teuthology.orchestra.run.smithi018.stdout:e31 2024-04-12T21:36:45.380 INFO:teuthology.orchestra.run.smithi018.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-12T21:36:45.380 INFO:teuthology.orchestra.run.smithi018.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-12T21:36:45.381 INFO:teuthology.orchestra.run.smithi018.stdout:legacy client fscid: 1 2024-04-12T21:36:45.381 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-12T21:36:45.381 INFO:teuthology.orchestra.run.smithi018.stdout:Filesystem 'cephfs' (1) 2024-04-12T21:36:45.381 INFO:teuthology.orchestra.run.smithi018.stdout:fs_name cephfs 2024-04-12T21:36:45.381 INFO:teuthology.orchestra.run.smithi018.stdout:epoch 24 2024-04-12T21:36:45.381 INFO:teuthology.orchestra.run.smithi018.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-04-12T21:36:45.381 INFO:teuthology.orchestra.run.smithi018.stdout:created 2024-04-12T20:51:57.362850+0000 2024-04-12T21:36:45.381 INFO:teuthology.orchestra.run.smithi018.stdout:modified 2024-04-12T21:06:59.717190+0000 2024-04-12T21:36:45.381 INFO:teuthology.orchestra.run.smithi018.stdout:tableserver 0 2024-04-12T21:36:45.381 INFO:teuthology.orchestra.run.smithi018.stdout:root 0 2024-04-12T21:36:45.381 INFO:teuthology.orchestra.run.smithi018.stdout:session_timeout 60 2024-04-12T21:36:45.381 INFO:teuthology.orchestra.run.smithi018.stdout:session_autoclose 300 2024-04-12T21:36:45.381 INFO:teuthology.orchestra.run.smithi018.stdout:max_file_size 1099511627776 2024-04-12T21:36:45.382 INFO:teuthology.orchestra.run.smithi018.stdout:max_xattr_size 65536 2024-04-12T21:36:45.382 INFO:teuthology.orchestra.run.smithi018.stdout:required_client_features {} 2024-04-12T21:36:45.382 INFO:teuthology.orchestra.run.smithi018.stdout:last_failure 0 2024-04-12T21:36:45.382 INFO:teuthology.orchestra.run.smithi018.stdout:last_failure_osd_epoch 100 2024-04-12T21:36:45.382 INFO:teuthology.orchestra.run.smithi018.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-12T21:36:45.382 INFO:teuthology.orchestra.run.smithi018.stdout:max_mds 2 2024-04-12T21:36:45.382 INFO:teuthology.orchestra.run.smithi018.stdout:in 0,1 2024-04-12T21:36:45.382 INFO:teuthology.orchestra.run.smithi018.stdout:up {} 2024-04-12T21:36:45.382 INFO:teuthology.orchestra.run.smithi018.stdout:failed 0,1 2024-04-12T21:36:45.382 INFO:teuthology.orchestra.run.smithi018.stdout:damaged 2024-04-12T21:36:45.382 INFO:teuthology.orchestra.run.smithi018.stdout:stopped 2024-04-12T21:36:45.382 INFO:teuthology.orchestra.run.smithi018.stdout:data_pools [3] 2024-04-12T21:36:45.382 INFO:teuthology.orchestra.run.smithi018.stdout:metadata_pool 2 2024-04-12T21:36:45.383 INFO:teuthology.orchestra.run.smithi018.stdout:inline_data disabled 2024-04-12T21:36:45.383 INFO:teuthology.orchestra.run.smithi018.stdout:balancer 2024-04-12T21:36:45.383 INFO:teuthology.orchestra.run.smithi018.stdout:bal_rank_mask -1 2024-04-12T21:36:45.383 INFO:teuthology.orchestra.run.smithi018.stdout:standby_count_wanted 1 2024-04-12T21:36:45.383 INFO:teuthology.orchestra.run.smithi018.stdout:qdb_cluster leader: 0 members: 2024-04-12T21:36:45.383 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-12T21:36:45.383 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-12T21:36:45.383 INFO:teuthology.orchestra.run.smithi018.stdout:Standby daemons: 2024-04-12T21:36:45.383 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-12T21:36:45.383 INFO:teuthology.orchestra.run.smithi018.stdout:[mds.cephfs.smithi186.wjnafs{-1:24277} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.186:6826/438347207,v1:172.21.15.186:6827/438347207] compat {c=[1],r=[1],i=[7ff]}] 2024-04-12T21:36:45.383 INFO:teuthology.orchestra.run.smithi018.stdout:[mds.cephfs.smithi018.acqbsc{-1:34436} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.18:6828/755179764,v1:172.21.15.18:6829/755179764] compat {c=[1],r=[1],i=[1fff]}] 2024-04-12T21:36:45.383 INFO:teuthology.orchestra.run.smithi018.stdout:[mds.cephfs.smithi018.szjluo{-1:34468} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.18:6826/2407992653,v1:172.21.15.18:6827/2407992653] compat {c=[1],r=[1],i=[1fff]}] 2024-04-12T21:36:45.383 INFO:teuthology.orchestra.run.smithi018.stdout:[mds.cephfs.smithi186.jdllyn{-1:44595} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.186:6824/3523444202,v1:172.21.15.186:6825/3523444202] compat {c=[1],r=[1],i=[1fff]}] 2024-04-12T21:36:45.384 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:45.382+0000 7f67fbe00700 1 -- 172.21.15.18:0/3100804443 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f67dc06ec00 msgr2=0x7f67dc0710b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:36:45.384 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:45.382+0000 7f67fbe00700 1 --2- 172.21.15.18:0/3100804443 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f67dc06ec00 0x7f67dc0710b0 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f67e000b3b0 tx=0x7f67e0009e30 comp rx=0 tx=0).stop 2024-04-12T21:36:45.385 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:45.382+0000 7f67fbe00700 1 -- 172.21.15.18:0/3100804443 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f67f4109430 msgr2=0x7f67f4119fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:36:45.385 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:45.382+0000 7f67fbe00700 1 --2- 172.21.15.18:0/3100804443 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f67f4109430 0x7f67f4119fa0 secure :-1 s=READY pgs=941 cs=0 l=1 rev1=1 crypto rx=0x7f67e800b790 tx=0x7f67e8009250 comp rx=0 tx=0).stop 2024-04-12T21:36:45.385 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:45.382+0000 7f67fbe00700 1 -- 172.21.15.18:0/3100804443 shutdown_connections 2024-04-12T21:36:45.385 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:45.382+0000 7f67fbe00700 1 --2- 172.21.15.18:0/3100804443 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f67dc06ec00 0x7f67dc0710b0 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:36:45.385 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:45.382+0000 7f67fbe00700 1 --2- 172.21.15.18:0/3100804443 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f67f4109430 0x7f67f4119fa0 unknown :-1 s=CLOSED pgs=941 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:36:45.385 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:45.382+0000 7f67fbe00700 1 --2- 172.21.15.18:0/3100804443 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f67f4109dd0 0x7f67f411a4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:36:45.385 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:45.382+0000 7f67fbe00700 1 -- 172.21.15.18:0/3100804443 >> 172.21.15.18:0/3100804443 conn(0x7f67f4072c00 msgr2=0x7f67f4074b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:36:45.385 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:45.382+0000 7f67fbe00700 1 -- 172.21.15.18:0/3100804443 shutdown_connections 2024-04-12T21:36:45.385 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:45.383+0000 7f67fbe00700 1 -- 172.21.15.18:0/3100804443 wait complete. 2024-04-12T21:36:45.385 INFO:teuthology.orchestra.run.smithi018.stderr:dumped fsmap epoch 31 2024-04-12T21:36:45.536 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:45.536+0000 7f4a40e00700 1 -- 172.21.15.18:0/4032557077 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4a3c109430 msgr2=0x7f4a3c109800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:36:45.536 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:45.536+0000 7f4a40e00700 1 --2- 172.21.15.18:0/4032557077 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4a3c109430 0x7f4a3c109800 secure :-1 s=READY pgs=942 cs=0 l=1 rev1=1 crypto rx=0x7f4a2c004940 tx=0x7f4a2c009e30 comp rx=0 tx=0).stop 2024-04-12T21:36:45.536 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:45.536+0000 7f4a40e00700 1 -- 172.21.15.18:0/4032557077 shutdown_connections 2024-04-12T21:36:45.536 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:45.536+0000 7f4a40e00700 1 --2- 172.21.15.18:0/4032557077 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4a3c109dd0 0x7f4a3c103440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:36:45.536 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:45.536+0000 7f4a40e00700 1 --2- 172.21.15.18:0/4032557077 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4a3c109430 0x7f4a3c109800 unknown :-1 s=CLOSED pgs=942 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:36:45.536 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:45.536+0000 7f4a40e00700 1 -- 172.21.15.18:0/4032557077 >> 172.21.15.18:0/4032557077 conn(0x7f4a3c072c00 msgr2=0x7f4a3c075010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:36:45.536 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:45.536+0000 7f4a40e00700 1 -- 172.21.15.18:0/4032557077 shutdown_connections 2024-04-12T21:36:45.536 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:45.536+0000 7f4a40e00700 1 -- 172.21.15.18:0/4032557077 wait complete. 2024-04-12T21:36:45.538 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:45.537+0000 7f4a40e00700 1 Processor -- start 2024-04-12T21:36:45.538 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:45.537+0000 7f4a40e00700 1 -- start start 2024-04-12T21:36:45.538 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:45.538+0000 7f4a40e00700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4a3c109430 0x7f4a3c119f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:36:45.538 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:45.538+0000 7f4a40e00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4a3c109dd0 0x7f4a3c11a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:36:45.538 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:45.538+0000 7f4a40e00700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a3c11aad0 con 0x7f4a3c109dd0 2024-04-12T21:36:45.538 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:45.538+0000 7f4a40e00700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a3c114100 con 0x7f4a3c109430 2024-04-12T21:36:45.538 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:45.538+0000 7f4a39400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4a3c109dd0 0x7f4a3c11a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:36:45.538 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:45.538+0000 7f4a39400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4a3c109dd0 0x7f4a3c11a480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:40936/0 (socket says 172.21.15.18:40936) 2024-04-12T21:36:45.538 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:45.538+0000 7f4a39400700 1 -- 172.21.15.18:0/2463612925 learned_addr learned my addr 172.21.15.18:0/2463612925 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:36:45.538 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:45.538+0000 7f4a39400700 1 -- 172.21.15.18:0/2463612925 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4a3c109430 msgr2=0x7f4a3c119f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:36:45.539 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:45.538+0000 7f4a39400700 1 --2- 172.21.15.18:0/2463612925 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4a3c109430 0x7f4a3c119f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:36:45.539 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:45.538+0000 7f4a39400700 1 -- 172.21.15.18:0/2463612925 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a2c005040 con 0x7f4a3c109dd0 2024-04-12T21:36:45.539 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:45.539+0000 7f4a39400700 1 --2- 172.21.15.18:0/2463612925 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4a3c109dd0 0x7f4a3c11a480 secure :-1 s=READY pgs=943 cs=0 l=1 rev1=1 crypto rx=0x7f4a2800c890 tx=0x7f4a280044c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:36:45.539 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:45.539+0000 7f4a32000700 1 -- 172.21.15.18:0/2463612925 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4a2800a7d0 con 0x7f4a3c109dd0 2024-04-12T21:36:45.539 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:45.539+0000 7f4a32000700 1 -- 172.21.15.18:0/2463612925 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f4a2800ae00 con 0x7f4a3c109dd0 2024-04-12T21:36:45.539 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:45.539+0000 7f4a32000700 1 -- 172.21.15.18:0/2463612925 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4a280113a0 con 0x7f4a3c109dd0 2024-04-12T21:36:45.539 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:45.539+0000 7f4a40e00700 1 -- 172.21.15.18:0/2463612925 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4a3c1143e0 con 0x7f4a3c109dd0 2024-04-12T21:36:45.540 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:45.539+0000 7f4a40e00700 1 -- 172.21.15.18:0/2463612925 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4a3c1149a0 con 0x7f4a3c109dd0 2024-04-12T21:36:45.540 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:45.540+0000 7f4a40e00700 1 -- 172.21.15.18:0/2463612925 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4a3c104b80 con 0x7f4a3c109dd0 2024-04-12T21:36:45.540 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:45.541+0000 7f4a32000700 1 -- 172.21.15.18:0/2463612925 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7f4a28012400 con 0x7f4a3c109dd0 2024-04-12T21:36:45.542 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:45.542+0000 7f4a32000700 1 --2- 172.21.15.18:0/2463612925 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f4a2006ec00 0x7f4a200710b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:36:45.542 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:45.542+0000 7f4a32000700 1 -- 172.21.15.18:0/2463612925 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 6757+0+0 (secure 0 0 0) 0x7f4a2808ea60 con 0x7f4a3c109dd0 2024-04-12T21:36:45.542 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:45.543+0000 7f4a39e00700 1 --2- 172.21.15.18:0/2463612925 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f4a2006ec00 0x7f4a200710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:36:45.542 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:45.543+0000 7f4a39e00700 1 --2- 172.21.15.18:0/2463612925 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f4a2006ec00 0x7f4a200710b0 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f4a2c004f90 tx=0x7f4a2c009e30 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:36:45.546 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:45.547+0000 7f4a32000700 1 -- 172.21.15.18:0/2463612925 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f4a2805b2b0 con 0x7f4a3c109dd0 2024-04-12T21:36:45.800 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:45.800+0000 7f4a40e00700 1 -- 172.21.15.18:0/2463612925 --> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f4a3c110590 con 0x7f4a2006ec00 2024-04-12T21:36:45.801 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:45.802+0000 7f4a32000700 1 -- 172.21.15.18:0/2463612925 <== mgr.34102 v2:172.21.15.18:6800/3812891919 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f4a3c110590 con 0x7f4a2006ec00 2024-04-12T21:36:45.802 INFO:teuthology.orchestra.run.smithi018.stdout:{ 2024-04-12T21:36:45.802 INFO:teuthology.orchestra.run.smithi018.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:f4452def3be4d350f4a8e5feaf7ebb1511be9298fe68478ba7a2d6fbd7d69a63", 2024-04-12T21:36:45.802 INFO:teuthology.orchestra.run.smithi018.stdout: "in_progress": true, 2024-04-12T21:36:45.802 INFO:teuthology.orchestra.run.smithi018.stdout: "which": "Upgrading all daemon types on all hosts", 2024-04-12T21:36:45.802 INFO:teuthology.orchestra.run.smithi018.stdout: "services_complete": [ 2024-04-12T21:36:45.802 INFO:teuthology.orchestra.run.smithi018.stdout: "crash", 2024-04-12T21:36:45.803 INFO:teuthology.orchestra.run.smithi018.stdout: "osd", 2024-04-12T21:36:45.803 INFO:teuthology.orchestra.run.smithi018.stdout: "mon", 2024-04-12T21:36:45.803 INFO:teuthology.orchestra.run.smithi018.stdout: "mgr" 2024-04-12T21:36:45.803 INFO:teuthology.orchestra.run.smithi018.stdout: ], 2024-04-12T21:36:45.803 INFO:teuthology.orchestra.run.smithi018.stdout: "progress": "15/21 daemons upgraded", 2024-04-12T21:36:45.803 INFO:teuthology.orchestra.run.smithi018.stdout: "message": "Currently upgrading mds daemons", 2024-04-12T21:36:45.803 INFO:teuthology.orchestra.run.smithi018.stdout: "is_paused": false 2024-04-12T21:36:45.803 INFO:teuthology.orchestra.run.smithi018.stdout:} 2024-04-12T21:36:45.805 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:45.805+0000 7f4a40e00700 1 -- 172.21.15.18:0/2463612925 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f4a2006ec00 msgr2=0x7f4a200710b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:36:45.805 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:45.805+0000 7f4a40e00700 1 --2- 172.21.15.18:0/2463612925 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f4a2006ec00 0x7f4a200710b0 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f4a2c004f90 tx=0x7f4a2c009e30 comp rx=0 tx=0).stop 2024-04-12T21:36:45.805 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:45.805+0000 7f4a40e00700 1 -- 172.21.15.18:0/2463612925 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4a3c109dd0 msgr2=0x7f4a3c11a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:36:45.805 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:45.805+0000 7f4a40e00700 1 --2- 172.21.15.18:0/2463612925 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4a3c109dd0 0x7f4a3c11a480 secure :-1 s=READY pgs=943 cs=0 l=1 rev1=1 crypto rx=0x7f4a2800c890 tx=0x7f4a280044c0 comp rx=0 tx=0).stop 2024-04-12T21:36:45.805 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:45.806+0000 7f4a40e00700 1 -- 172.21.15.18:0/2463612925 shutdown_connections 2024-04-12T21:36:45.806 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:45.806+0000 7f4a40e00700 1 --2- 172.21.15.18:0/2463612925 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f4a2006ec00 0x7f4a200710b0 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:36:45.806 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:45.806+0000 7f4a40e00700 1 --2- 172.21.15.18:0/2463612925 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4a3c109430 0x7f4a3c119f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:36:45.806 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:45.806+0000 7f4a40e00700 1 --2- 172.21.15.18:0/2463612925 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4a3c109dd0 0x7f4a3c11a480 unknown :-1 s=CLOSED pgs=943 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:36:45.806 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:45.806+0000 7f4a40e00700 1 -- 172.21.15.18:0/2463612925 >> 172.21.15.18:0/2463612925 conn(0x7f4a3c072c00 msgr2=0x7f4a3c074c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:36:45.806 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:45.806+0000 7f4a40e00700 1 -- 172.21.15.18:0/2463612925 shutdown_connections 2024-04-12T21:36:45.806 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:45.806+0000 7f4a40e00700 1 -- 172.21.15.18:0/2463612925 wait complete. 2024-04-12T21:36:45.962 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:45.962+0000 7f1811200700 1 -- 172.21.15.18:0/3571099563 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f180c105800 msgr2=0x7f180c0fee70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:36:45.962 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:45.962+0000 7f1811200700 1 --2- 172.21.15.18:0/3571099563 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f180c105800 0x7f180c0fee70 secure :-1 s=READY pgs=944 cs=0 l=1 rev1=1 crypto rx=0x7f17f8004940 tx=0x7f17f8009e30 comp rx=0 tx=0).stop 2024-04-12T21:36:45.963 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:45.962+0000 7f1811200700 1 -- 172.21.15.18:0/3571099563 shutdown_connections 2024-04-12T21:36:45.963 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:45.962+0000 7f1811200700 1 --2- 172.21.15.18:0/3571099563 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f180c105800 0x7f180c0fee70 unknown :-1 s=CLOSED pgs=944 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:36:45.963 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:45.962+0000 7f1811200700 1 --2- 172.21.15.18:0/3571099563 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f180c104e60 0x7f180c105230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:36:45.963 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:45.962+0000 7f1811200700 1 -- 172.21.15.18:0/3571099563 >> 172.21.15.18:0/3571099563 conn(0x7f180c0fab10 msgr2=0x7f180c0fcef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:36:45.963 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:45.963+0000 7f1811200700 1 -- 172.21.15.18:0/3571099563 shutdown_connections 2024-04-12T21:36:45.963 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:45.963+0000 7f1811200700 1 -- 172.21.15.18:0/3571099563 wait complete. 2024-04-12T21:36:45.964 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:45.964+0000 7f1811200700 1 Processor -- start 2024-04-12T21:36:45.964 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:45.964+0000 7f1811200700 1 -- start start 2024-04-12T21:36:45.964 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:45.964+0000 7f1811200700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f180c104e60 0x7f180c115bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:36:45.964 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:45.964+0000 7f1811200700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f180c105800 0x7f180c1160f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:36:45.964 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:45.964+0000 7f1811200700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f180c116740 con 0x7f180c104e60 2024-04-12T21:36:45.964 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:45.964+0000 7f1811200700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f180c10fb30 con 0x7f180c105800 2024-04-12T21:36:45.964 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:45.964+0000 7f180b400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f180c104e60 0x7f180c115bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:36:45.965 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:45.965+0000 7f180b400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f180c104e60 0x7f180c115bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:40966/0 (socket says 172.21.15.18:40966) 2024-04-12T21:36:45.965 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:45.965+0000 7f180b400700 1 -- 172.21.15.18:0/3488246582 learned_addr learned my addr 172.21.15.18:0/3488246582 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:36:45.965 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:45.965+0000 7f180b400700 1 -- 172.21.15.18:0/3488246582 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f180c105800 msgr2=0x7f180c1160f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:36:45.965 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:45.965+0000 7f180b400700 1 --2- 172.21.15.18:0/3488246582 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f180c105800 0x7f180c1160f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:36:45.965 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:45.965+0000 7f180b400700 1 -- 172.21.15.18:0/3488246582 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17f8005040 con 0x7f180c104e60 2024-04-12T21:36:45.966 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:45.965+0000 7f180b400700 1 --2- 172.21.15.18:0/3488246582 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f180c104e60 0x7f180c115bb0 secure :-1 s=READY pgs=945 cs=0 l=1 rev1=1 crypto rx=0x7f17fc002a50 tx=0x7f17fc0046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:36:45.966 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:45.966+0000 7f1803400700 1 -- 172.21.15.18:0/3488246582 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f17fc007cc0 con 0x7f180c104e60 2024-04-12T21:36:45.966 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:45.966+0000 7f1811200700 1 -- 172.21.15.18:0/3488246582 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f180c10fe10 con 0x7f180c104e60 2024-04-12T21:36:45.966 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:45.966+0000 7f1803400700 1 -- 172.21.15.18:0/3488246582 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f17fc00f040 con 0x7f180c104e60 2024-04-12T21:36:45.966 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:45.966+0000 7f1803400700 1 -- 172.21.15.18:0/3488246582 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f17fc009640 con 0x7f180c104e60 2024-04-12T21:36:45.966 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:45.966+0000 7f1811200700 1 -- 172.21.15.18:0/3488246582 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f180c1103d0 con 0x7f180c104e60 2024-04-12T21:36:45.966 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:45.967+0000 7f1811200700 1 -- 172.21.15.18:0/3488246582 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f180c107330 con 0x7f180c104e60 2024-04-12T21:36:45.967 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:45.968+0000 7f1803400700 1 -- 172.21.15.18:0/3488246582 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7f17fc0097a0 con 0x7f180c104e60 2024-04-12T21:36:45.968 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:45.969+0000 7f1803400700 1 --2- 172.21.15.18:0/3488246582 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f17f006ec00 0x7f17f00710b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:36:45.969 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:45.969+0000 7f1803400700 1 -- 172.21.15.18:0/3488246582 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 6757+0+0 (secure 0 0 0) 0x7f17fc08e760 con 0x7f180c104e60 2024-04-12T21:36:45.969 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:45.969+0000 7f180aa00700 1 --2- 172.21.15.18:0/3488246582 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f17f006ec00 0x7f17f00710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:36:45.969 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:45.970+0000 7f180aa00700 1 --2- 172.21.15.18:0/3488246582 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f17f006ec00 0x7f17f00710b0 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f17f8004f90 tx=0x7f17f8009e30 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:36:45.973 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:45.974+0000 7f1803400700 1 -- 172.21.15.18:0/3488246582 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f17fc05af10 con 0x7f180c104e60 2024-04-12T21:36:46.104 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:36:45 smithi018 ceph-mon[112590]: from='client.35936 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:36:46.104 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:36:45 smithi018 ceph-mon[112590]: from='client.35940 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:36:46.104 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:36:45 smithi018 ceph-mon[112590]: from='client.? 172.21.15.18:0/1652892496' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T21:36:46.104 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:36:45 smithi018 ceph-mon[112590]: from='client.? 172.21.15.18:0/3100804443' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-12T21:36:46.283 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:36:45 smithi186 ceph-mon[86379]: from='client.35936 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:36:46.283 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:36:45 smithi186 ceph-mon[86379]: from='client.35940 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:36:46.283 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:36:45 smithi186 ceph-mon[86379]: from='client.? 172.21.15.18:0/1652892496' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T21:36:46.283 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:36:45 smithi186 ceph-mon[86379]: from='client.? 172.21.15.18:0/3100804443' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-12T21:36:46.291 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:46.292+0000 7f1811200700 1 -- 172.21.15.18:0/3488246582 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f180c10bf40 con 0x7f180c104e60 2024-04-12T21:36:46.292 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:46.292+0000 7f1803400700 1 -- 172.21.15.18:0/3488246582 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+4303 (secure 0 0 0) 0x7f17fc05aaa0 con 0x7f180c104e60 2024-04-12T21:36:46.292 INFO:teuthology.orchestra.run.smithi018.stdout:HEALTH_ERR failed to probe daemons or devices; 1 filesystem is degraded; 1 filesystem is offline 2024-04-12T21:36:46.292 INFO:teuthology.orchestra.run.smithi018.stdout:[WRN] CEPHADM_REFRESH_FAILED: failed to probe daemons or devices 2024-04-12T21:36:46.292 INFO:teuthology.orchestra.run.smithi018.stdout: host smithi018 `cephadm ceph-volume` failed: cephadm exited with an error code: 1, stderr: Inferring config /var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/mon.smithi018/config 2024-04-12T21:36:46.293 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/podman: timeout after 895 seconds 2024-04-12T21:36:46.293 INFO:teuthology.orchestra.run.smithi018.stdout:Non-zero exit code 124 from /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:f4452def3be4d350f4a8e5feaf7ebb1511be9298fe68478ba7a2d6fbd7d69a63 -e NODE_NAME=smithi018 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968:/var/run/ceph:z -v /var/log/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968:/var/log/ceph:z -v /var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmpxsmwkkuh:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:f4452def3be4d350f4a8e5feaf7ebb1511be9298fe68478ba7a2d6fbd7d69a63 inventory --format=json-pretty --filter-for-batch 2024-04-12T21:36:46.293 INFO:teuthology.orchestra.run.smithi018.stdout:Traceback (most recent call last): 2024-04-12T21:36:46.293 INFO:teuthology.orchestra.run.smithi018.stdout: File "/usr/lib64/python3.9/runpy.py", line 197, in _run_module_as_main 2024-04-12T21:36:46.293 INFO:teuthology.orchestra.run.smithi018.stdout: return _run_code(code, main_globals, None, 2024-04-12T21:36:46.293 INFO:teuthology.orchestra.run.smithi018.stdout: File "/usr/lib64/python3.9/runpy.py", line 87, in _run_code 2024-04-12T21:36:46.293 INFO:teuthology.orchestra.run.smithi018.stdout: exec(code, run_globals) 2024-04-12T21:36:46.293 INFO:teuthology.orchestra.run.smithi018.stdout: File "/var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/cephadm.e44f3ed5e34e4ef52eedef233d1190bff004470d8e8005127b5312bff357fa7f/__main__.py", line 5618, in 2024-04-12T21:36:46.293 INFO:teuthology.orchestra.run.smithi018.stdout: File "/var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/cephadm.e44f3ed5e34e4ef52eedef233d1190bff004470d8e8005127b5312bff357fa7f/__main__.py", line 5606, in main 2024-04-12T21:36:46.293 INFO:teuthology.orchestra.run.smithi018.stdout: File "/var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/cephadm.e44f3ed5e34e4ef52eedef233d1190bff004470d8e8005127b5312bff357fa7f/__main__.py", line 408, in _infer_config 2024-04-12T21:36:46.293 INFO:teuthology.orchestra.run.smithi018.stdout: File "/var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/cephadm.e44f3ed5e34e4ef52eedef233d1190bff004470d8e8005127b5312bff357fa7f/__main__.py", line 323, in _infer_fsid 2024-04-12T21:36:46.293 INFO:teuthology.orchestra.run.smithi018.stdout: File "/var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/cephadm.e44f3ed5e34e4ef52eedef233d1190bff004470d8e8005127b5312bff357fa7f/__main__.py", line 436, in _infer_image 2024-04-12T21:36:46.293 INFO:teuthology.orchestra.run.smithi018.stdout: File "/var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/cephadm.e44f3ed5e34e4ef52eedef233d1190bff004470d8e8005127b5312bff357fa7f/__main__.py", line 310, in _validate_fsid 2024-04-12T21:36:46.294 INFO:teuthology.orchestra.run.smithi018.stdout: File "/var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/cephadm.e44f3ed5e34e4ef52eedef233d1190bff004470d8e8005127b5312bff357fa7f/__main__.py", line 3359, in command_ceph_volume 2024-04-12T21:36:46.294 INFO:teuthology.orchestra.run.smithi018.stdout: File "/var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/cephadm.e44f3ed5e34e4ef52eedef233d1190bff004470d8e8005127b5312bff357fa7f/cephadmlib/call_wrappers.py", line 310, in call_throws 2024-04-12T21:36:46.294 INFO:teuthology.orchestra.run.smithi018.stdout:RuntimeError: Failed command: /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:f4452def3be4d350f4a8e5feaf7ebb1511be9298fe68478ba7a2d6fbd7d69a63 -e NODE_NAME=smithi018 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968:/var/run/ceph:z -v /var/log/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968:/var/log/ceph:z -v /var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmpxsmwkkuh:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:f4452def3be4d350f4a8e5feaf7ebb1511be9298fe68478ba7a2d6fbd7d69a63 inventory --format=json-pretty --filter-for-batch 2024-04-12T21:36:46.294 INFO:teuthology.orchestra.run.smithi018.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-04-12T21:36:46.294 INFO:teuthology.orchestra.run.smithi018.stdout: fs cephfs is degraded 2024-04-12T21:36:46.294 INFO:teuthology.orchestra.run.smithi018.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-04-12T21:36:46.294 INFO:teuthology.orchestra.run.smithi018.stdout: fs cephfs is offline because no MDS is active for it. 2024-04-12T21:36:46.295 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:46.295+0000 7f1811200700 1 -- 172.21.15.18:0/3488246582 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f17f006ec00 msgr2=0x7f17f00710b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:36:46.295 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:46.295+0000 7f1811200700 1 --2- 172.21.15.18:0/3488246582 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f17f006ec00 0x7f17f00710b0 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f17f8004f90 tx=0x7f17f8009e30 comp rx=0 tx=0).stop 2024-04-12T21:36:46.295 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:46.295+0000 7f1811200700 1 -- 172.21.15.18:0/3488246582 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f180c104e60 msgr2=0x7f180c115bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:36:46.295 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:46.295+0000 7f1811200700 1 --2- 172.21.15.18:0/3488246582 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f180c104e60 0x7f180c115bb0 secure :-1 s=READY pgs=945 cs=0 l=1 rev1=1 crypto rx=0x7f17fc002a50 tx=0x7f17fc0046b0 comp rx=0 tx=0).stop 2024-04-12T21:36:46.295 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:46.295+0000 7f1811200700 1 -- 172.21.15.18:0/3488246582 shutdown_connections 2024-04-12T21:36:46.295 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:46.295+0000 7f1811200700 1 --2- 172.21.15.18:0/3488246582 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f17f006ec00 0x7f17f00710b0 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:36:46.295 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:46.295+0000 7f1811200700 1 --2- 172.21.15.18:0/3488246582 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f180c104e60 0x7f180c115bb0 unknown :-1 s=CLOSED pgs=945 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:36:46.295 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:46.295+0000 7f1811200700 1 --2- 172.21.15.18:0/3488246582 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f180c105800 0x7f180c1160f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:36:46.295 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:46.295+0000 7f1811200700 1 -- 172.21.15.18:0/3488246582 >> 172.21.15.18:0/3488246582 conn(0x7f180c0fab10 msgr2=0x7f180c0fd940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:36:46.295 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:46.295+0000 7f1811200700 1 -- 172.21.15.18:0/3488246582 shutdown_connections 2024-04-12T21:36:46.295 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:36:46.296+0000 7f1811200700 1 -- 172.21.15.18:0/3488246582 wait complete. 2024-04-12T21:36:47.283 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:36:46 smithi186 ceph-mon[86379]: pgmap v1110: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:36:47.283 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:36:46 smithi186 ceph-mon[86379]: from='client.35952 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:36:47.283 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:36:46 smithi186 ceph-mon[86379]: from='client.? 172.21.15.18:0/3488246582' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-12T21:36:47.354 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:36:46 smithi018 ceph-mon[112590]: pgmap v1110: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:36:47.354 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:36:46 smithi018 ceph-mon[112590]: from='client.35952 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:36:47.354 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:36:46 smithi018 ceph-mon[112590]: from='client.? 172.21.15.18:0/3488246582' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-12T21:36:49.283 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:36:48 smithi186 ceph-mon[86379]: pgmap v1111: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:36:49.354 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:36:48 smithi018 ceph-mon[112590]: pgmap v1111: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:36:51.282 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:36:50 smithi186 ceph-mon[86379]: pgmap v1112: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:36:51.353 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:36:50 smithi018 ceph-mon[112590]: pgmap v1112: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:36:53.283 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:36:52 smithi186 ceph-mon[86379]: pgmap v1113: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:36:53.354 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:36:52 smithi018 ceph-mon[112590]: pgmap v1113: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:36:55.283 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:36:54 smithi186 ceph-mon[86379]: pgmap v1114: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:36:55.354 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:36:54 smithi018 ceph-mon[112590]: pgmap v1114: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:36:56.282 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:36:55 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T21:36:56.354 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:36:55 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T21:36:57.283 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:36:56 smithi186 ceph-mon[86379]: pgmap v1115: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:36:57.353 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:36:56 smithi018 ceph-mon[112590]: pgmap v1115: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:36:59.283 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:36:58 smithi186 ceph-mon[86379]: pgmap v1116: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:36:59.354 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:36:58 smithi018 ceph-mon[112590]: pgmap v1116: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:37:01.285 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:37:00 smithi186 ceph-mon[86379]: pgmap v1117: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:37:01.353 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:37:00 smithi018 ceph-mon[112590]: pgmap v1117: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:37:03.283 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:37:02 smithi186 ceph-mon[86379]: pgmap v1118: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:37:03.354 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:37:02 smithi018 ceph-mon[112590]: pgmap v1118: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:37:05.282 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:37:04 smithi186 ceph-mon[86379]: pgmap v1119: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:37:05.354 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:37:04 smithi018 ceph-mon[112590]: pgmap v1119: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:37:06.283 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:37:06 smithi186 ceph-mon[86379]: pgmap v1120: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:37:06.354 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:37:05 smithi018 ceph-mon[112590]: pgmap v1120: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:37:09.283 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:37:08 smithi186 ceph-mon[86379]: pgmap v1121: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:37:09.354 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:37:08 smithi018 ceph-mon[112590]: pgmap v1121: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:37:11.283 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:37:10 smithi186 ceph-mon[86379]: pgmap v1122: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:37:11.283 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:37:10 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T21:37:11.354 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:37:10 smithi018 ceph-mon[112590]: pgmap v1122: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:37:11.354 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:37:10 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T21:37:13.283 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:37:12 smithi186 ceph-mon[86379]: pgmap v1123: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:37:13.354 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:37:12 smithi018 ceph-mon[112590]: pgmap v1123: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:37:15.157 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:37:14 smithi018 ceph-mon[112590]: pgmap v1124: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:37:15.282 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:37:14 smithi186 ceph-mon[86379]: pgmap v1124: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:37:16.467 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:16.466+0000 7faadb200700 1 -- 172.21.15.18:0/944519026 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7faadc104e60 msgr2=0x7faadc105230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:37:16.467 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:16.466+0000 7faadb200700 1 --2- 172.21.15.18:0/944519026 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7faadc104e60 0x7faadc105230 secure :-1 s=READY pgs=946 cs=0 l=1 rev1=1 crypto rx=0x7faacc004940 tx=0x7faacc009e30 comp rx=0 tx=0).stop 2024-04-12T21:37:16.468 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:16.467+0000 7faadb200700 1 -- 172.21.15.18:0/944519026 shutdown_connections 2024-04-12T21:37:16.468 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:16.467+0000 7faadb200700 1 --2- 172.21.15.18:0/944519026 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7faadc105800 0x7faadc0fee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:37:16.468 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:16.467+0000 7faadb200700 1 --2- 172.21.15.18:0/944519026 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7faadc104e60 0x7faadc105230 unknown :-1 s=CLOSED pgs=946 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:37:16.468 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:16.467+0000 7faadb200700 1 -- 172.21.15.18:0/944519026 >> 172.21.15.18:0/944519026 conn(0x7faadc0fab10 msgr2=0x7faadc0fcef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:37:16.468 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:16.467+0000 7faadb200700 1 -- 172.21.15.18:0/944519026 shutdown_connections 2024-04-12T21:37:16.468 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:16.467+0000 7faadb200700 1 -- 172.21.15.18:0/944519026 wait complete. 2024-04-12T21:37:16.470 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:16.468+0000 7faadb200700 1 Processor -- start 2024-04-12T21:37:16.470 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:16.468+0000 7faadb200700 1 -- start start 2024-04-12T21:37:16.470 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:16.468+0000 7faadb200700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7faadc105800 0x7faadc115d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:37:16.470 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:16.468+0000 7faadb200700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7faadc1162a0 0x7faadc10fcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:37:16.470 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:16.468+0000 7faadb200700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faadc116820 con 0x7faadc105800 2024-04-12T21:37:16.470 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:16.468+0000 7faadb200700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faadc116990 con 0x7faadc1162a0 2024-04-12T21:37:16.470 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:16.469+0000 7faad9e00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7faadc105800 0x7faadc115d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:37:16.470 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:16.469+0000 7faad9e00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7faadc105800 0x7faadc115d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:39460/0 (socket says 172.21.15.18:39460) 2024-04-12T21:37:16.470 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:16.469+0000 7faad9e00700 1 -- 172.21.15.18:0/1358252487 learned_addr learned my addr 172.21.15.18:0/1358252487 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:37:16.471 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:16.469+0000 7faad9e00700 1 -- 172.21.15.18:0/1358252487 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7faadc1162a0 msgr2=0x7faadc10fcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:37:16.471 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:16.469+0000 7faad9e00700 1 --2- 172.21.15.18:0/1358252487 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7faadc1162a0 0x7faadc10fcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:37:16.471 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:16.469+0000 7faad9e00700 1 -- 172.21.15.18:0/1358252487 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faacc005040 con 0x7faadc105800 2024-04-12T21:37:16.471 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:16.469+0000 7faad9e00700 1 --2- 172.21.15.18:0/1358252487 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7faadc105800 0x7faadc115d60 secure :-1 s=READY pgs=947 cs=0 l=1 rev1=1 crypto rx=0x7faacc00b3b0 tx=0x7faacc009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:37:16.471 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:16.470+0000 7faad2000700 1 -- 172.21.15.18:0/1358252487 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7faacc00e040 con 0x7faadc105800 2024-04-12T21:37:16.471 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:16.470+0000 7faadb200700 1 -- 172.21.15.18:0/1358252487 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faadc1102c0 con 0x7faadc105800 2024-04-12T21:37:16.471 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:16.470+0000 7faad2000700 1 -- 172.21.15.18:0/1358252487 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7faacc009430 con 0x7faadc105800 2024-04-12T21:37:16.472 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:16.470+0000 7faad2000700 1 -- 172.21.15.18:0/1358252487 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7faacc013e90 con 0x7faadc105800 2024-04-12T21:37:16.472 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:16.470+0000 7faadb200700 1 -- 172.21.15.18:0/1358252487 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faadc110880 con 0x7faadc105800 2024-04-12T21:37:16.472 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:16.471+0000 7faadb200700 1 -- 172.21.15.18:0/1358252487 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faadc1005b0 con 0x7faadc105800 2024-04-12T21:37:16.472 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:16.472+0000 7faad2000700 1 -- 172.21.15.18:0/1358252487 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7faacc013640 con 0x7faadc105800 2024-04-12T21:37:16.473 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:16.473+0000 7faad2000700 1 --2- 172.21.15.18:0/1358252487 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7faac006ec00 0x7faac00710b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:37:16.473 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:16.473+0000 7faad2000700 1 -- 172.21.15.18:0/1358252487 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 6757+0+0 (secure 0 0 0) 0x7faacc08fda0 con 0x7faadc105800 2024-04-12T21:37:16.474 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:16.473+0000 7faad9400700 1 --2- 172.21.15.18:0/1358252487 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7faac006ec00 0x7faac00710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:37:16.474 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:16.474+0000 7faad9400700 1 --2- 172.21.15.18:0/1358252487 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7faac006ec00 0x7faac00710b0 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7faadc067090 tx=0x7faac8007cc0 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:37:16.478 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:16.478+0000 7faad2000700 1 -- 172.21.15.18:0/1358252487 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7faacc05c550 con 0x7faadc105800 2024-04-12T21:37:16.732 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:16.732+0000 7faadb200700 1 -- 172.21.15.18:0/1358252487 --> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7faadc10bf60 con 0x7faac006ec00 2024-04-12T21:37:16.735 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:16.735+0000 7faad2000700 1 -- 172.21.15.18:0/1358252487 <== mgr.34102 v2:172.21.15.18:6800/3812891919 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7faadc10bf60 con 0x7faac006ec00 2024-04-12T21:37:16.739 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:16.738+0000 7faadb200700 1 -- 172.21.15.18:0/1358252487 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7faac006ec00 msgr2=0x7faac00710b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:37:16.740 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:16.738+0000 7faadb200700 1 --2- 172.21.15.18:0/1358252487 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7faac006ec00 0x7faac00710b0 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7faadc067090 tx=0x7faac8007cc0 comp rx=0 tx=0).stop 2024-04-12T21:37:16.740 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:16.738+0000 7faadb200700 1 -- 172.21.15.18:0/1358252487 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7faadc105800 msgr2=0x7faadc115d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:37:16.740 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:16.738+0000 7faadb200700 1 --2- 172.21.15.18:0/1358252487 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7faadc105800 0x7faadc115d60 secure :-1 s=READY pgs=947 cs=0 l=1 rev1=1 crypto rx=0x7faacc00b3b0 tx=0x7faacc009e30 comp rx=0 tx=0).stop 2024-04-12T21:37:16.740 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:16.739+0000 7faadb200700 1 -- 172.21.15.18:0/1358252487 shutdown_connections 2024-04-12T21:37:16.740 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:16.739+0000 7faadb200700 1 --2- 172.21.15.18:0/1358252487 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7faac006ec00 0x7faac00710b0 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:37:16.740 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:16.739+0000 7faadb200700 1 --2- 172.21.15.18:0/1358252487 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7faadc105800 0x7faadc115d60 unknown :-1 s=CLOSED pgs=947 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:37:16.740 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:16.739+0000 7faadb200700 1 --2- 172.21.15.18:0/1358252487 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7faadc1162a0 0x7faadc10fcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:37:16.740 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:16.739+0000 7faadb200700 1 -- 172.21.15.18:0/1358252487 >> 172.21.15.18:0/1358252487 conn(0x7faadc0fab10 msgr2=0x7faadc0fcb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:37:16.740 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:16.739+0000 7faadb200700 1 -- 172.21.15.18:0/1358252487 shutdown_connections 2024-04-12T21:37:16.740 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:16.739+0000 7faadb200700 1 -- 172.21.15.18:0/1358252487 wait complete. 2024-04-12T21:37:16.756 INFO:teuthology.orchestra.run.smithi018.stdout:true 2024-04-12T21:37:16.905 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:16.904+0000 7fd97f800700 1 -- 172.21.15.18:0/215204688 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd978070300 msgr2=0x7fd9780706d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:37:16.905 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:16.904+0000 7fd97f800700 1 --2- 172.21.15.18:0/215204688 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd978070300 0x7fd9780706d0 secure :-1 s=READY pgs=948 cs=0 l=1 rev1=1 crypto rx=0x7fd96c004970 tx=0x7fd96c009e30 comp rx=0 tx=0).stop 2024-04-12T21:37:16.905 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:16.905+0000 7fd97f800700 1 -- 172.21.15.18:0/215204688 shutdown_connections 2024-04-12T21:37:16.905 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:16.905+0000 7fd97f800700 1 --2- 172.21.15.18:0/215204688 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd978070c10 0x7fd978109180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:37:16.906 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:16.905+0000 7fd97f800700 1 --2- 172.21.15.18:0/215204688 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd978070300 0x7fd9780706d0 unknown :-1 s=CLOSED pgs=948 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:37:16.906 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:16.905+0000 7fd97f800700 1 -- 172.21.15.18:0/215204688 >> 172.21.15.18:0/215204688 conn(0x7fd9780f8980 msgr2=0x7fd9780fad90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:37:16.906 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:16.905+0000 7fd97f800700 1 -- 172.21.15.18:0/215204688 shutdown_connections 2024-04-12T21:37:16.906 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:16.905+0000 7fd97f800700 1 -- 172.21.15.18:0/215204688 wait complete. 2024-04-12T21:37:16.906 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:16.906+0000 7fd97f800700 1 Processor -- start 2024-04-12T21:37:16.907 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:16.906+0000 7fd97f800700 1 -- start start 2024-04-12T21:37:16.907 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:16.906+0000 7fd97f800700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd978070300 0x7fd978115d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:37:16.907 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:16.906+0000 7fd97f800700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd978070c10 0x7fd978116270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:37:16.907 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:16.906+0000 7fd97f800700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9781168c0 con 0x7fd978070c10 2024-04-12T21:37:16.907 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:16.906+0000 7fd97f800700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd97810fc90 con 0x7fd978070300 2024-04-12T21:37:16.908 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:16.907+0000 7fd977e00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd978070c10 0x7fd978116270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:37:16.908 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:16.907+0000 7fd977e00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd978070c10 0x7fd978116270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:39484/0 (socket says 172.21.15.18:39484) 2024-04-12T21:37:16.908 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:16.907+0000 7fd977e00700 1 -- 172.21.15.18:0/3846685278 learned_addr learned my addr 172.21.15.18:0/3846685278 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:37:16.908 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:16.907+0000 7fd97ce00700 1 --2- 172.21.15.18:0/3846685278 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd978070300 0x7fd978115d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:37:16.908 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:16.907+0000 7fd977e00700 1 -- 172.21.15.18:0/3846685278 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd978070300 msgr2=0x7fd978115d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:37:16.908 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:16.907+0000 7fd977e00700 1 --2- 172.21.15.18:0/3846685278 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd978070300 0x7fd978115d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:37:16.908 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:16.907+0000 7fd977e00700 1 -- 172.21.15.18:0/3846685278 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd96c005040 con 0x7fd978070c10 2024-04-12T21:37:16.908 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:16.907+0000 7fd97ce00700 1 --2- 172.21.15.18:0/3846685278 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd978070300 0x7fd978115d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-12T21:37:16.909 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:16.907+0000 7fd977e00700 1 --2- 172.21.15.18:0/3846685278 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd978070c10 0x7fd978116270 secure :-1 s=READY pgs=949 cs=0 l=1 rev1=1 crypto rx=0x7fd96400c830 tx=0x7fd964004470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:37:16.909 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:16.908+0000 7fd974c00700 1 -- 172.21.15.18:0/3846685278 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd96400a760 con 0x7fd978070c10 2024-04-12T21:37:16.909 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:16.908+0000 7fd974c00700 1 -- 172.21.15.18:0/3846685278 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd96400ad90 con 0x7fd978070c10 2024-04-12T21:37:16.909 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:16.908+0000 7fd97f800700 1 -- 172.21.15.18:0/3846685278 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd97810ff70 con 0x7fd978070c10 2024-04-12T21:37:16.909 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:16.908+0000 7fd974c00700 1 -- 172.21.15.18:0/3846685278 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd9640113a0 con 0x7fd978070c10 2024-04-12T21:37:16.909 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:16.908+0000 7fd97f800700 1 -- 172.21.15.18:0/3846685278 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd9781104b0 con 0x7fd978070c10 2024-04-12T21:37:16.909 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:16.909+0000 7fd97f800700 1 -- 172.21.15.18:0/3846685278 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd978071a40 con 0x7fd978070c10 2024-04-12T21:37:16.910 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:16.910+0000 7fd974c00700 1 -- 172.21.15.18:0/3846685278 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7fd964012400 con 0x7fd978070c10 2024-04-12T21:37:16.912 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:16.911+0000 7fd974c00700 1 --2- 172.21.15.18:0/3846685278 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fd96006ec10 0x7fd9600710c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:37:16.912 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:16.912+0000 7fd974c00700 1 -- 172.21.15.18:0/3846685278 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 6757+0+0 (secure 0 0 0) 0x7fd96408e900 con 0x7fd978070c10 2024-04-12T21:37:16.912 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:16.912+0000 7fd97ce00700 1 --2- 172.21.15.18:0/3846685278 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fd96006ec10 0x7fd9600710c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:37:16.913 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:16.913+0000 7fd97ce00700 1 --2- 172.21.15.18:0/3846685278 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fd96006ec10 0x7fd9600710c0 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7fd96c00b9c0 tx=0x7fd96c009e30 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:37:16.917 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:16.916+0000 7fd974c00700 1 -- 172.21.15.18:0/3846685278 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7fd96405b1f0 con 0x7fd978070c10 2024-04-12T21:37:17.104 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:37:16 smithi018 ceph-mon[112590]: pgmap v1125: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:37:17.170 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:17.169+0000 7fd97f800700 1 -- 172.21.15.18:0/3846685278 --> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fd97810c100 con 0x7fd96006ec10 2024-04-12T21:37:17.172 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:17.172+0000 7fd974c00700 1 -- 172.21.15.18:0/3846685278 <== mgr.34102 v2:172.21.15.18:6800/3812891919 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7fd97810c100 con 0x7fd96006ec10 2024-04-12T21:37:17.175 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:17.175+0000 7fd97f800700 1 -- 172.21.15.18:0/3846685278 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fd96006ec10 msgr2=0x7fd9600710c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:37:17.176 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:17.175+0000 7fd97f800700 1 --2- 172.21.15.18:0/3846685278 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fd96006ec10 0x7fd9600710c0 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7fd96c00b9c0 tx=0x7fd96c009e30 comp rx=0 tx=0).stop 2024-04-12T21:37:17.176 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:17.175+0000 7fd97f800700 1 -- 172.21.15.18:0/3846685278 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd978070c10 msgr2=0x7fd978116270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:37:17.176 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:17.175+0000 7fd97f800700 1 --2- 172.21.15.18:0/3846685278 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd978070c10 0x7fd978116270 secure :-1 s=READY pgs=949 cs=0 l=1 rev1=1 crypto rx=0x7fd96400c830 tx=0x7fd964004470 comp rx=0 tx=0).stop 2024-04-12T21:37:17.176 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:17.175+0000 7fd97f800700 1 -- 172.21.15.18:0/3846685278 shutdown_connections 2024-04-12T21:37:17.176 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:17.175+0000 7fd97f800700 1 --2- 172.21.15.18:0/3846685278 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fd96006ec10 0x7fd9600710c0 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:37:17.176 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:17.175+0000 7fd97f800700 1 --2- 172.21.15.18:0/3846685278 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd978070300 0x7fd978115d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:37:17.176 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:17.175+0000 7fd97f800700 1 --2- 172.21.15.18:0/3846685278 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd978070c10 0x7fd978116270 unknown :-1 s=CLOSED pgs=949 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:37:17.176 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:17.175+0000 7fd97f800700 1 -- 172.21.15.18:0/3846685278 >> 172.21.15.18:0/3846685278 conn(0x7fd9780f8980 msgr2=0x7fd9780f9db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:37:17.176 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:17.175+0000 7fd97f800700 1 -- 172.21.15.18:0/3846685278 shutdown_connections 2024-04-12T21:37:17.176 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:17.175+0000 7fd97f800700 1 -- 172.21.15.18:0/3846685278 wait complete. 2024-04-12T21:37:17.282 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:37:16 smithi186 ceph-mon[86379]: pgmap v1125: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:37:17.336 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:17.336+0000 7ff798a00700 1 -- 172.21.15.18:0/1179992928 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff7940fc620 msgr2=0x7ff794100cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:37:17.336 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:17.336+0000 7ff798a00700 1 --2- 172.21.15.18:0/1179992928 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff7940fc620 0x7ff794100cd0 secure :-1 s=READY pgs=950 cs=0 l=1 rev1=1 crypto rx=0x7ff77c004940 tx=0x7ff77c009e30 comp rx=0 tx=0).stop 2024-04-12T21:37:17.337 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:17.336+0000 7ff798a00700 1 -- 172.21.15.18:0/1179992928 shutdown_connections 2024-04-12T21:37:17.337 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:17.336+0000 7ff798a00700 1 --2- 172.21.15.18:0/1179992928 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff7940fc620 0x7ff794100cd0 unknown :-1 s=CLOSED pgs=950 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:37:17.337 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:17.336+0000 7ff798a00700 1 --2- 172.21.15.18:0/1179992928 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff7940fbc80 0x7ff7940fc050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:37:17.337 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:17.336+0000 7ff798a00700 1 -- 172.21.15.18:0/1179992928 >> 172.21.15.18:0/1179992928 conn(0x7ff794072bc0 msgr2=0x7ff794072fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:37:17.337 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:17.336+0000 7ff798a00700 1 -- 172.21.15.18:0/1179992928 shutdown_connections 2024-04-12T21:37:17.337 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:17.336+0000 7ff798a00700 1 -- 172.21.15.18:0/1179992928 wait complete. 2024-04-12T21:37:17.338 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:17.337+0000 7ff798a00700 1 Processor -- start 2024-04-12T21:37:17.338 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:17.337+0000 7ff798a00700 1 -- start start 2024-04-12T21:37:17.339 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:17.338+0000 7ff798a00700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff7940fbc80 0x7ff79411a1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:37:17.339 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:17.338+0000 7ff798a00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff7940fc620 0x7ff79411a710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:37:17.339 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:17.338+0000 7ff798a00700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff79411ad60 con 0x7ff7940fc620 2024-04-12T21:37:17.339 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:17.338+0000 7ff798a00700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff794114130 con 0x7ff7940fbc80 2024-04-12T21:37:17.339 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:17.338+0000 7ff78be00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff7940fc620 0x7ff79411a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:37:17.339 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:17.338+0000 7ff78be00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff7940fc620 0x7ff79411a710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:39506/0 (socket says 172.21.15.18:39506) 2024-04-12T21:37:17.339 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:17.338+0000 7ff78be00700 1 -- 172.21.15.18:0/2754114882 learned_addr learned my addr 172.21.15.18:0/2754114882 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:37:17.339 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:17.338+0000 7ff791e00700 1 --2- 172.21.15.18:0/2754114882 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff7940fbc80 0x7ff79411a1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:37:17.339 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:17.339+0000 7ff78be00700 1 -- 172.21.15.18:0/2754114882 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff7940fbc80 msgr2=0x7ff79411a1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:37:17.339 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:17.339+0000 7ff78be00700 1 --2- 172.21.15.18:0/2754114882 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff7940fbc80 0x7ff79411a1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:37:17.339 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:17.339+0000 7ff78be00700 1 -- 172.21.15.18:0/2754114882 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff77c005040 con 0x7ff7940fc620 2024-04-12T21:37:17.340 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:17.339+0000 7ff78be00700 1 --2- 172.21.15.18:0/2754114882 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff7940fc620 0x7ff79411a710 secure :-1 s=READY pgs=951 cs=0 l=1 rev1=1 crypto rx=0x7ff77c009150 tx=0x7ff77c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:37:17.340 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:17.339+0000 7ff78a000700 1 -- 172.21.15.18:0/2754114882 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff77c00e040 con 0x7ff7940fc620 2024-04-12T21:37:17.340 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:17.339+0000 7ff78a000700 1 -- 172.21.15.18:0/2754114882 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff77c013420 con 0x7ff7940fc620 2024-04-12T21:37:17.340 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:17.339+0000 7ff78a000700 1 -- 172.21.15.18:0/2754114882 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff77c01c630 con 0x7ff7940fc620 2024-04-12T21:37:17.340 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:17.339+0000 7ff798a00700 1 -- 172.21.15.18:0/2754114882 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff7941143b0 con 0x7ff7940fc620 2024-04-12T21:37:17.341 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:17.339+0000 7ff798a00700 1 -- 172.21.15.18:0/2754114882 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff7941148f0 con 0x7ff7940fc620 2024-04-12T21:37:17.341 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:17.340+0000 7ff798a00700 1 -- 172.21.15.18:0/2754114882 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff794065e30 con 0x7ff7940fc620 2024-04-12T21:37:17.342 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:17.342+0000 7ff78a000700 1 -- 172.21.15.18:0/2754114882 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7ff77c013590 con 0x7ff7940fc620 2024-04-12T21:37:17.343 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:17.343+0000 7ff78a000700 1 --2- 172.21.15.18:0/2754114882 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7ff77806ec10 0x7ff7780710c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:37:17.343 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:17.343+0000 7ff78a000700 1 -- 172.21.15.18:0/2754114882 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 6757+0+0 (secure 0 0 0) 0x7ff77c08fc10 con 0x7ff7940fc620 2024-04-12T21:37:17.343 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:17.343+0000 7ff791e00700 1 --2- 172.21.15.18:0/2754114882 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7ff77806ec10 0x7ff7780710c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:37:17.344 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:17.344+0000 7ff791e00700 1 --2- 172.21.15.18:0/2754114882 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7ff77806ec10 0x7ff7780710c0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7ff784004990 tx=0x7ff784009ee0 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:37:17.348 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:17.347+0000 7ff78a000700 1 -- 172.21.15.18:0/2754114882 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7ff77c05c440 con 0x7ff7940fc620 2024-04-12T21:37:17.589 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:17.589+0000 7ff798a00700 1 -- 172.21.15.18:0/2754114882 --> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7ff7941105d0 con 0x7ff77806ec10 2024-04-12T21:37:17.620 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:17.619+0000 7ff78a000700 1 -- 172.21.15.18:0/2754114882 <== mgr.34102 v2:172.21.15.18:6800/3812891919 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7ff7941105d0 con 0x7ff77806ec10 2024-04-12T21:37:17.620 INFO:teuthology.orchestra.run.smithi018.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-12T21:37:17.620 INFO:teuthology.orchestra.run.smithi018.stdout:alertmanager.smithi018 smithi018 *:9093,9094 running (37m) 14m ago 52m 15.5M - 0.25.0 c8568f914cd2 c332090068e8 2024-04-12T21:37:17.620 INFO:teuthology.orchestra.run.smithi018.stdout:crash.smithi018 smithi018 running (34m) 14m ago 52m 6665k - 19.0.0-2879-g324baf8d 8717720dc5e5 b7f80d1f7982 2024-04-12T21:37:17.620 INFO:teuthology.orchestra.run.smithi018.stdout:crash.smithi186 smithi186 running (34m) 14m ago 51m 6660k - 19.0.0-2879-g324baf8d 8717720dc5e5 a4f55c81b623 2024-04-12T21:37:17.620 INFO:teuthology.orchestra.run.smithi018.stdout:grafana.smithi018 smithi018 *:3000 running (37m) 14m ago 52m 74.8M - 9.4.12 2bacad6d85d8 b08e3bf176dd 2024-04-12T21:37:17.620 INFO:teuthology.orchestra.run.smithi018.stdout:mds.cephfs.smithi018.acqbsc smithi018 running (30m) 14m ago 45m 16.7M - 19.0.0-2879-g324baf8d 8717720dc5e5 021f99ab24ae 2024-04-12T21:37:17.620 INFO:teuthology.orchestra.run.smithi018.stdout:mds.cephfs.smithi018.szjluo smithi018 running (29m) 14m ago 45m 16.5M - 19.0.0-2879-g324baf8d 8717720dc5e5 0003117479d4 2024-04-12T21:37:17.620 INFO:teuthology.orchestra.run.smithi018.stdout:mds.cephfs.smithi186.jdllyn smithi186 running (14m) 14m ago 45m 11.0M - 19.0.0-2879-g324baf8d 8717720dc5e5 21b1f46eef23 2024-04-12T21:37:17.621 INFO:teuthology.orchestra.run.smithi018.stdout:mds.cephfs.smithi186.wjnafs smithi186 running (45m) 14m ago 45m 19.5M - 17.2.7-827-gd3b5060a 21de9b7369ad 74b115d6d6fd 2024-04-12T21:37:17.621 INFO:teuthology.orchestra.run.smithi018.stdout:mgr.smithi018.xjzwga smithi018 *:8443,9283,8765 running (40m) 14m ago 54m 512M - 19.0.0-2879-g324baf8d 8717720dc5e5 56088305eab7 2024-04-12T21:37:17.621 INFO:teuthology.orchestra.run.smithi018.stdout:mgr.smithi186.kqcfxk smithi186 *:8443,9283,8765 running (39m) 14m ago 51m 459M - 19.0.0-2879-g324baf8d 8717720dc5e5 09cd766e5e0d 2024-04-12T21:37:17.621 INFO:teuthology.orchestra.run.smithi018.stdout:mon.smithi018 smithi018 running (35m) 14m ago 54m 73.1M 2048M 19.0.0-2879-g324baf8d 8717720dc5e5 df3e201dfed6 2024-04-12T21:37:17.621 INFO:teuthology.orchestra.run.smithi018.stdout:mon.smithi186 smithi186 running (35m) 14m ago 50m 47.2M 2048M 19.0.0-2879-g324baf8d 8717720dc5e5 c77498da9b2b 2024-04-12T21:37:17.621 INFO:teuthology.orchestra.run.smithi018.stdout:node-exporter.smithi018 smithi018 *:9100 running (38m) 14m ago 52m 8108k - 1.5.0 0da6a335fe13 326c3d9c6847 2024-04-12T21:37:17.621 INFO:teuthology.orchestra.run.smithi018.stdout:node-exporter.smithi186 smithi186 *:9100 running (38m) 14m ago 51m 8757k - 1.5.0 0da6a335fe13 ec2112baeecf 2024-04-12T21:37:17.621 INFO:teuthology.orchestra.run.smithi018.stdout:osd.0 smithi018 running (33m) 14m ago 50m 123M 1559M 19.0.0-2879-g324baf8d 8717720dc5e5 60b1273ca047 2024-04-12T21:37:17.621 INFO:teuthology.orchestra.run.smithi018.stdout:osd.1 smithi018 running (33m) 14m ago 49m 95.1M 1559M 19.0.0-2879-g324baf8d 8717720dc5e5 170ce1e5fb8d 2024-04-12T21:37:17.621 INFO:teuthology.orchestra.run.smithi018.stdout:osd.2 smithi018 running (32m) 14m ago 48m 94.3M 1559M 19.0.0-2879-g324baf8d 8717720dc5e5 c2cbc6aa2839 2024-04-12T21:37:17.621 INFO:teuthology.orchestra.run.smithi018.stdout:osd.3 smithi186 running (31m) 14m ago 48m 101M 2581M 19.0.0-2879-g324baf8d 8717720dc5e5 0138e8262e4b 2024-04-12T21:37:17.621 INFO:teuthology.orchestra.run.smithi018.stdout:osd.4 smithi186 running (31m) 14m ago 47m 91.0M 2581M 19.0.0-2879-g324baf8d 8717720dc5e5 b20cb63b25c9 2024-04-12T21:37:17.621 INFO:teuthology.orchestra.run.smithi018.stdout:osd.5 smithi186 running (30m) 14m ago 47m 92.0M 2581M 19.0.0-2879-g324baf8d 8717720dc5e5 2513bea2610e 2024-04-12T21:37:17.621 INFO:teuthology.orchestra.run.smithi018.stdout:prometheus.smithi018 smithi018 *:9095 running (38m) 14m ago 51m 67.9M - 2.43.0 a07b618ecd1d f58841127e21 2024-04-12T21:37:17.623 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:17.622+0000 7ff798a00700 1 -- 172.21.15.18:0/2754114882 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7ff77806ec10 msgr2=0x7ff7780710c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:37:17.623 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:17.622+0000 7ff798a00700 1 --2- 172.21.15.18:0/2754114882 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7ff77806ec10 0x7ff7780710c0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7ff784004990 tx=0x7ff784009ee0 comp rx=0 tx=0).stop 2024-04-12T21:37:17.623 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:17.622+0000 7ff798a00700 1 -- 172.21.15.18:0/2754114882 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff7940fc620 msgr2=0x7ff79411a710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:37:17.623 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:17.622+0000 7ff798a00700 1 --2- 172.21.15.18:0/2754114882 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff7940fc620 0x7ff79411a710 secure :-1 s=READY pgs=951 cs=0 l=1 rev1=1 crypto rx=0x7ff77c009150 tx=0x7ff77c009e30 comp rx=0 tx=0).stop 2024-04-12T21:37:17.623 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:17.622+0000 7ff798a00700 1 -- 172.21.15.18:0/2754114882 shutdown_connections 2024-04-12T21:37:17.624 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:17.622+0000 7ff798a00700 1 --2- 172.21.15.18:0/2754114882 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7ff77806ec10 0x7ff7780710c0 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:37:17.624 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:17.622+0000 7ff798a00700 1 --2- 172.21.15.18:0/2754114882 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff7940fbc80 0x7ff79411a1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:37:17.624 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:17.622+0000 7ff798a00700 1 --2- 172.21.15.18:0/2754114882 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff7940fc620 0x7ff79411a710 unknown :-1 s=CLOSED pgs=951 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:37:17.624 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:17.623+0000 7ff798a00700 1 -- 172.21.15.18:0/2754114882 >> 172.21.15.18:0/2754114882 conn(0x7ff794072bc0 msgr2=0x7ff79410aab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:37:17.624 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:17.623+0000 7ff798a00700 1 -- 172.21.15.18:0/2754114882 shutdown_connections 2024-04-12T21:37:17.624 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:17.623+0000 7ff798a00700 1 -- 172.21.15.18:0/2754114882 wait complete. 2024-04-12T21:37:17.780 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:17.779+0000 7f11df800700 1 -- 172.21.15.18:0/3478109567 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f11d81059a0 msgr2=0x7f11d80ff010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:37:17.780 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:17.779+0000 7f11df800700 1 --2- 172.21.15.18:0/3478109567 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f11d81059a0 0x7f11d80ff010 secure :-1 s=READY pgs=952 cs=0 l=1 rev1=1 crypto rx=0x7f11c4004940 tx=0x7f11c4009e30 comp rx=0 tx=0).stop 2024-04-12T21:37:17.780 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:17.779+0000 7f11df800700 1 -- 172.21.15.18:0/3478109567 shutdown_connections 2024-04-12T21:37:17.780 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:17.779+0000 7f11df800700 1 --2- 172.21.15.18:0/3478109567 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f11d81059a0 0x7f11d80ff010 unknown :-1 s=CLOSED pgs=952 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:37:17.780 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:17.779+0000 7f11df800700 1 --2- 172.21.15.18:0/3478109567 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f11d8105000 0x7f11d81053d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:37:17.780 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:17.779+0000 7f11df800700 1 -- 172.21.15.18:0/3478109567 >> 172.21.15.18:0/3478109567 conn(0x7f11d80fab70 msgr2=0x7f11d80fcf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:37:17.780 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:17.779+0000 7f11df800700 1 -- 172.21.15.18:0/3478109567 shutdown_connections 2024-04-12T21:37:17.780 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:17.779+0000 7f11df800700 1 -- 172.21.15.18:0/3478109567 wait complete. 2024-04-12T21:37:17.781 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:17.780+0000 7f11df800700 1 Processor -- start 2024-04-12T21:37:17.782 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:17.780+0000 7f11df800700 1 -- start start 2024-04-12T21:37:17.782 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:17.781+0000 7f11df800700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f11d8105000 0x7f11d8115bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:37:17.782 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:17.781+0000 7f11df800700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f11d81059a0 0x7f11d8116110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:37:17.782 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:17.781+0000 7f11df800700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11d8116760 con 0x7f11d8105000 2024-04-12T21:37:17.782 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:17.781+0000 7f11df800700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11d810fd40 con 0x7f11d81059a0 2024-04-12T21:37:17.782 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:17.781+0000 7f11dce00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f11d8105000 0x7f11d8115bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:37:17.782 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:17.781+0000 7f11dce00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f11d8105000 0x7f11d8115bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:39522/0 (socket says 172.21.15.18:39522) 2024-04-12T21:37:17.782 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:17.781+0000 7f11dce00700 1 -- 172.21.15.18:0/2105051993 learned_addr learned my addr 172.21.15.18:0/2105051993 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:37:17.782 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:17.781+0000 7f11d7e00700 1 --2- 172.21.15.18:0/2105051993 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f11d81059a0 0x7f11d8116110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:37:17.783 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:17.782+0000 7f11dce00700 1 -- 172.21.15.18:0/2105051993 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f11d81059a0 msgr2=0x7f11d8116110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:37:17.783 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:17.782+0000 7f11dce00700 1 --2- 172.21.15.18:0/2105051993 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f11d81059a0 0x7f11d8116110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:37:17.783 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:17.782+0000 7f11dce00700 1 -- 172.21.15.18:0/2105051993 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11c4005040 con 0x7f11d8105000 2024-04-12T21:37:17.783 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:17.782+0000 7f11dce00700 1 --2- 172.21.15.18:0/2105051993 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f11d8105000 0x7f11d8115bd0 secure :-1 s=READY pgs=953 cs=0 l=1 rev1=1 crypto rx=0x7f11cc00cb10 tx=0x7f11cc007f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:37:17.783 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:17.782+0000 7f11d4c00700 1 -- 172.21.15.18:0/2105051993 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f11cc004200 con 0x7f11d8105000 2024-04-12T21:37:17.783 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:17.782+0000 7f11d4c00700 1 -- 172.21.15.18:0/2105051993 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f11cc00acc0 con 0x7f11d8105000 2024-04-12T21:37:17.783 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:17.782+0000 7f11df800700 1 -- 172.21.15.18:0/2105051993 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f11d8110020 con 0x7f11d8105000 2024-04-12T21:37:17.784 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:17.782+0000 7f11d4c00700 1 -- 172.21.15.18:0/2105051993 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f11cc003d90 con 0x7f11d8105000 2024-04-12T21:37:17.784 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:17.782+0000 7f11df800700 1 -- 172.21.15.18:0/2105051993 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f11d81105e0 con 0x7f11d8105000 2024-04-12T21:37:17.784 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:17.783+0000 7f11df800700 1 -- 172.21.15.18:0/2105051993 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f11d8100750 con 0x7f11d8105000 2024-04-12T21:37:17.785 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:17.785+0000 7f11d4c00700 1 -- 172.21.15.18:0/2105051993 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7f11cc003430 con 0x7f11d8105000 2024-04-12T21:37:17.786 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:17.786+0000 7f11d4c00700 1 --2- 172.21.15.18:0/2105051993 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f11c006ec00 0x7f11c00710b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:37:17.786 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:17.786+0000 7f11d4c00700 1 -- 172.21.15.18:0/2105051993 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 6757+0+0 (secure 0 0 0) 0x7f11cc08ef40 con 0x7f11d8105000 2024-04-12T21:37:17.786 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:17.786+0000 7f11d7e00700 1 --2- 172.21.15.18:0/2105051993 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f11c006ec00 0x7f11c00710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:37:17.787 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:17.786+0000 7f11d7e00700 1 --2- 172.21.15.18:0/2105051993 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f11c006ec00 0x7f11c00710b0 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f11d8111470 tx=0x7f11c4009e30 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:37:17.790 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:17.790+0000 7f11d4c00700 1 -- 172.21.15.18:0/2105051993 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f11cc05b6f0 con 0x7f11d8105000 2024-04-12T21:37:18.114 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:18.114+0000 7f11df800700 1 -- 172.21.15.18:0/2105051993 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f11d8111220 con 0x7f11d8105000 2024-04-12T21:37:18.115 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:18.115+0000 7f11d4c00700 1 -- 172.21.15.18:0/2105051993 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+837 (secure 0 0 0) 0x7f11cc060e60 con 0x7f11d8105000 2024-04-12T21:37:18.115 INFO:teuthology.orchestra.run.smithi018.stdout:{ 2024-04-12T21:37:18.116 INFO:teuthology.orchestra.run.smithi018.stdout: "mon": { 2024-04-12T21:37:18.116 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 19.0.0-2879-g324baf8d (324baf8ddb91dea46cac50cb535c306115791355) squid (dev)": 2 2024-04-12T21:37:18.116 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-12T21:37:18.116 INFO:teuthology.orchestra.run.smithi018.stdout: "mgr": { 2024-04-12T21:37:18.116 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 19.0.0-2879-g324baf8d (324baf8ddb91dea46cac50cb535c306115791355) squid (dev)": 2 2024-04-12T21:37:18.116 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-12T21:37:18.116 INFO:teuthology.orchestra.run.smithi018.stdout: "osd": { 2024-04-12T21:37:18.116 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 19.0.0-2879-g324baf8d (324baf8ddb91dea46cac50cb535c306115791355) squid (dev)": 6 2024-04-12T21:37:18.116 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-12T21:37:18.116 INFO:teuthology.orchestra.run.smithi018.stdout: "mds": { 2024-04-12T21:37:18.116 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 17.2.7-827-gd3b5060a (d3b5060a17449edfae9385cdf792afec109fca5d) quincy (stable)": 1, 2024-04-12T21:37:18.116 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 19.0.0-2879-g324baf8d (324baf8ddb91dea46cac50cb535c306115791355) squid (dev)": 3 2024-04-12T21:37:18.117 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-12T21:37:18.117 INFO:teuthology.orchestra.run.smithi018.stdout: "overall": { 2024-04-12T21:37:18.117 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 17.2.7-827-gd3b5060a (d3b5060a17449edfae9385cdf792afec109fca5d) quincy (stable)": 1, 2024-04-12T21:37:18.117 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 19.0.0-2879-g324baf8d (324baf8ddb91dea46cac50cb535c306115791355) squid (dev)": 13 2024-04-12T21:37:18.117 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-04-12T21:37:18.117 INFO:teuthology.orchestra.run.smithi018.stdout:} 2024-04-12T21:37:18.119 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:18.118+0000 7f11df800700 1 -- 172.21.15.18:0/2105051993 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f11c006ec00 msgr2=0x7f11c00710b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:37:18.119 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:18.118+0000 7f11df800700 1 --2- 172.21.15.18:0/2105051993 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f11c006ec00 0x7f11c00710b0 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f11d8111470 tx=0x7f11c4009e30 comp rx=0 tx=0).stop 2024-04-12T21:37:18.119 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:18.118+0000 7f11df800700 1 -- 172.21.15.18:0/2105051993 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f11d8105000 msgr2=0x7f11d8115bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:37:18.119 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:18.118+0000 7f11df800700 1 --2- 172.21.15.18:0/2105051993 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f11d8105000 0x7f11d8115bd0 secure :-1 s=READY pgs=953 cs=0 l=1 rev1=1 crypto rx=0x7f11cc00cb10 tx=0x7f11cc007f90 comp rx=0 tx=0).stop 2024-04-12T21:37:18.119 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:18.118+0000 7f11df800700 1 -- 172.21.15.18:0/2105051993 shutdown_connections 2024-04-12T21:37:18.119 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:18.118+0000 7f11df800700 1 --2- 172.21.15.18:0/2105051993 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f11c006ec00 0x7f11c00710b0 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:37:18.119 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:18.118+0000 7f11df800700 1 --2- 172.21.15.18:0/2105051993 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f11d8105000 0x7f11d8115bd0 unknown :-1 s=CLOSED pgs=953 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:37:18.119 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:18.118+0000 7f11df800700 1 --2- 172.21.15.18:0/2105051993 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f11d81059a0 0x7f11d8116110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:37:18.119 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:18.118+0000 7f11df800700 1 -- 172.21.15.18:0/2105051993 >> 172.21.15.18:0/2105051993 conn(0x7f11d80fab70 msgr2=0x7f11d80fbb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:37:18.119 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:18.118+0000 7f11df800700 1 -- 172.21.15.18:0/2105051993 shutdown_connections 2024-04-12T21:37:18.119 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:18.118+0000 7f11df800700 1 -- 172.21.15.18:0/2105051993 wait complete. 2024-04-12T21:37:18.280 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:18.280+0000 7f9150c00700 1 -- 172.21.15.18:0/65372905 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f914c104e60 msgr2=0x7f914c105230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:37:18.280 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:18.280+0000 7f9150c00700 1 --2- 172.21.15.18:0/65372905 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f914c104e60 0x7f914c105230 secure :-1 s=READY pgs=954 cs=0 l=1 rev1=1 crypto rx=0x7f913c004940 tx=0x7f913c009e30 comp rx=0 tx=0).stop 2024-04-12T21:37:18.281 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:37:17 smithi018 ceph-mon[112590]: from='client.35960 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:37:18.281 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:18.280+0000 7f9150c00700 1 -- 172.21.15.18:0/65372905 shutdown_connections 2024-04-12T21:37:18.281 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:18.280+0000 7f9150c00700 1 --2- 172.21.15.18:0/65372905 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f914c105800 0x7f914c0fee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:37:18.281 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:18.280+0000 7f9150c00700 1 --2- 172.21.15.18:0/65372905 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f914c104e60 0x7f914c105230 unknown :-1 s=CLOSED pgs=954 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:37:18.282 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:18.280+0000 7f9150c00700 1 -- 172.21.15.18:0/65372905 >> 172.21.15.18:0/65372905 conn(0x7f914c0fab10 msgr2=0x7f914c0fcef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:37:18.282 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:18.280+0000 7f9150c00700 1 -- 172.21.15.18:0/65372905 shutdown_connections 2024-04-12T21:37:18.282 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:18.280+0000 7f9150c00700 1 -- 172.21.15.18:0/65372905 wait complete. 2024-04-12T21:37:18.282 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:18.281+0000 7f9150c00700 1 Processor -- start 2024-04-12T21:37:18.282 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:18.281+0000 7f9150c00700 1 -- start start 2024-04-12T21:37:18.283 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:18.281+0000 7f9150c00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f914c104e60 0x7f914c115c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:37:18.283 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:18.281+0000 7f9150c00700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f914c105800 0x7f914c116160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:37:18.283 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:18.282+0000 7f9150c00700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f914c1167b0 con 0x7f914c104e60 2024-04-12T21:37:18.283 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:18.282+0000 7f9150c00700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f914c10fba0 con 0x7f914c105800 2024-04-12T21:37:18.283 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:18.282+0000 7f914b400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f914c104e60 0x7f914c115c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:37:18.283 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:18.282+0000 7f914b400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f914c104e60 0x7f914c115c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:39538/0 (socket says 172.21.15.18:39538) 2024-04-12T21:37:18.283 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:18.282+0000 7f914b400700 1 -- 172.21.15.18:0/359324888 learned_addr learned my addr 172.21.15.18:0/359324888 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:37:18.284 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:18.282+0000 7f914aa00700 1 --2- 172.21.15.18:0/359324888 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f914c105800 0x7f914c116160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:37:18.284 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:18.282+0000 7f914b400700 1 -- 172.21.15.18:0/359324888 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f914c105800 msgr2=0x7f914c116160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:37:18.284 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:18.282+0000 7f914b400700 1 --2- 172.21.15.18:0/359324888 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f914c105800 0x7f914c116160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:37:18.284 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:18.282+0000 7f914b400700 1 -- 172.21.15.18:0/359324888 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f913c005040 con 0x7f914c104e60 2024-04-12T21:37:18.284 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:18.283+0000 7f914b400700 1 --2- 172.21.15.18:0/359324888 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f914c104e60 0x7f914c115c20 secure :-1 s=READY pgs=955 cs=0 l=1 rev1=1 crypto rx=0x7f913c00b8b0 tx=0x7f913c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:37:18.284 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:37:17 smithi186 ceph-mon[86379]: from='client.35960 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:37:18.285 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:18.283+0000 7f9143400700 1 -- 172.21.15.18:0/359324888 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f913c00e040 con 0x7f914c104e60 2024-04-12T21:37:18.285 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:18.283+0000 7f9143400700 1 -- 172.21.15.18:0/359324888 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f913c004290 con 0x7f914c104e60 2024-04-12T21:37:18.285 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:18.283+0000 7f9150c00700 1 -- 172.21.15.18:0/359324888 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f914c10fe20 con 0x7f914c104e60 2024-04-12T21:37:18.285 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:18.283+0000 7f9143400700 1 -- 172.21.15.18:0/359324888 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f913c01c3f0 con 0x7f914c104e60 2024-04-12T21:37:18.285 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:18.284+0000 7f9150c00700 1 -- 172.21.15.18:0/359324888 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f914c1103e0 con 0x7f914c104e60 2024-04-12T21:37:18.285 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:18.285+0000 7f9150c00700 1 -- 172.21.15.18:0/359324888 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f914c1005b0 con 0x7f914c104e60 2024-04-12T21:37:18.286 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:18.286+0000 7f9143400700 1 -- 172.21.15.18:0/359324888 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7f913c0137b0 con 0x7f914c104e60 2024-04-12T21:37:18.288 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:18.287+0000 7f9143400700 1 --2- 172.21.15.18:0/359324888 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f913006ec00 0x7f91300710b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:37:18.288 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:18.288+0000 7f9143400700 1 -- 172.21.15.18:0/359324888 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 6757+0+0 (secure 0 0 0) 0x7f913c08fce0 con 0x7f914c104e60 2024-04-12T21:37:18.288 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:18.288+0000 7f914aa00700 1 --2- 172.21.15.18:0/359324888 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f913006ec00 0x7f91300710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:37:18.289 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:18.289+0000 7f914aa00700 1 --2- 172.21.15.18:0/359324888 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f913006ec00 0x7f91300710b0 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f914c111450 tx=0x7f9138009040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:37:18.293 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:18.292+0000 7f9143400700 1 -- 172.21.15.18:0/359324888 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f913c05c530 con 0x7f914c104e60 2024-04-12T21:37:18.566 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:18.565+0000 7f9150c00700 1 -- 172.21.15.18:0/359324888 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f914c04bed0 con 0x7f914c104e60 2024-04-12T21:37:18.566 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:18.566+0000 7f9143400700 1 -- 172.21.15.18:0/359324888 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 31 v31) v1 ==== 76+0+1873 (secure 0 0 0) 0x7f913c05c0c0 con 0x7f914c104e60 2024-04-12T21:37:18.567 INFO:teuthology.orchestra.run.smithi018.stdout:e31 2024-04-12T21:37:18.567 INFO:teuthology.orchestra.run.smithi018.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-12T21:37:18.567 INFO:teuthology.orchestra.run.smithi018.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-12T21:37:18.568 INFO:teuthology.orchestra.run.smithi018.stdout:legacy client fscid: 1 2024-04-12T21:37:18.568 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-12T21:37:18.568 INFO:teuthology.orchestra.run.smithi018.stdout:Filesystem 'cephfs' (1) 2024-04-12T21:37:18.568 INFO:teuthology.orchestra.run.smithi018.stdout:fs_name cephfs 2024-04-12T21:37:18.568 INFO:teuthology.orchestra.run.smithi018.stdout:epoch 24 2024-04-12T21:37:18.568 INFO:teuthology.orchestra.run.smithi018.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-04-12T21:37:18.568 INFO:teuthology.orchestra.run.smithi018.stdout:created 2024-04-12T20:51:57.362850+0000 2024-04-12T21:37:18.568 INFO:teuthology.orchestra.run.smithi018.stdout:modified 2024-04-12T21:06:59.717190+0000 2024-04-12T21:37:18.568 INFO:teuthology.orchestra.run.smithi018.stdout:tableserver 0 2024-04-12T21:37:18.568 INFO:teuthology.orchestra.run.smithi018.stdout:root 0 2024-04-12T21:37:18.568 INFO:teuthology.orchestra.run.smithi018.stdout:session_timeout 60 2024-04-12T21:37:18.568 INFO:teuthology.orchestra.run.smithi018.stdout:session_autoclose 300 2024-04-12T21:37:18.568 INFO:teuthology.orchestra.run.smithi018.stdout:max_file_size 1099511627776 2024-04-12T21:37:18.568 INFO:teuthology.orchestra.run.smithi018.stdout:max_xattr_size 65536 2024-04-12T21:37:18.568 INFO:teuthology.orchestra.run.smithi018.stdout:required_client_features {} 2024-04-12T21:37:18.568 INFO:teuthology.orchestra.run.smithi018.stdout:last_failure 0 2024-04-12T21:37:18.569 INFO:teuthology.orchestra.run.smithi018.stdout:last_failure_osd_epoch 100 2024-04-12T21:37:18.569 INFO:teuthology.orchestra.run.smithi018.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-12T21:37:18.569 INFO:teuthology.orchestra.run.smithi018.stdout:max_mds 2 2024-04-12T21:37:18.569 INFO:teuthology.orchestra.run.smithi018.stdout:in 0,1 2024-04-12T21:37:18.569 INFO:teuthology.orchestra.run.smithi018.stdout:up {} 2024-04-12T21:37:18.569 INFO:teuthology.orchestra.run.smithi018.stdout:failed 0,1 2024-04-12T21:37:18.569 INFO:teuthology.orchestra.run.smithi018.stdout:damaged 2024-04-12T21:37:18.569 INFO:teuthology.orchestra.run.smithi018.stdout:stopped 2024-04-12T21:37:18.569 INFO:teuthology.orchestra.run.smithi018.stdout:data_pools [3] 2024-04-12T21:37:18.569 INFO:teuthology.orchestra.run.smithi018.stdout:metadata_pool 2 2024-04-12T21:37:18.569 INFO:teuthology.orchestra.run.smithi018.stdout:inline_data disabled 2024-04-12T21:37:18.569 INFO:teuthology.orchestra.run.smithi018.stdout:balancer 2024-04-12T21:37:18.569 INFO:teuthology.orchestra.run.smithi018.stdout:bal_rank_mask -1 2024-04-12T21:37:18.569 INFO:teuthology.orchestra.run.smithi018.stdout:standby_count_wanted 1 2024-04-12T21:37:18.569 INFO:teuthology.orchestra.run.smithi018.stdout:qdb_cluster leader: 0 members: 2024-04-12T21:37:18.570 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-12T21:37:18.570 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-12T21:37:18.570 INFO:teuthology.orchestra.run.smithi018.stdout:Standby daemons: 2024-04-12T21:37:18.570 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-12T21:37:18.570 INFO:teuthology.orchestra.run.smithi018.stdout:[mds.cephfs.smithi186.wjnafs{-1:24277} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.186:6826/438347207,v1:172.21.15.186:6827/438347207] compat {c=[1],r=[1],i=[7ff]}] 2024-04-12T21:37:18.570 INFO:teuthology.orchestra.run.smithi018.stdout:[mds.cephfs.smithi018.acqbsc{-1:34436} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.18:6828/755179764,v1:172.21.15.18:6829/755179764] compat {c=[1],r=[1],i=[1fff]}] 2024-04-12T21:37:18.570 INFO:teuthology.orchestra.run.smithi018.stdout:[mds.cephfs.smithi018.szjluo{-1:34468} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.18:6826/2407992653,v1:172.21.15.18:6827/2407992653] compat {c=[1],r=[1],i=[1fff]}] 2024-04-12T21:37:18.570 INFO:teuthology.orchestra.run.smithi018.stdout:[mds.cephfs.smithi186.jdllyn{-1:44595} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.186:6824/3523444202,v1:172.21.15.186:6825/3523444202] compat {c=[1],r=[1],i=[1fff]}] 2024-04-12T21:37:18.571 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:18.569+0000 7f9150c00700 1 -- 172.21.15.18:0/359324888 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f913006ec00 msgr2=0x7f91300710b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:37:18.571 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:18.569+0000 7f9150c00700 1 --2- 172.21.15.18:0/359324888 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f913006ec00 0x7f91300710b0 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f914c111450 tx=0x7f9138009040 comp rx=0 tx=0).stop 2024-04-12T21:37:18.571 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:18.569+0000 7f9150c00700 1 -- 172.21.15.18:0/359324888 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f914c104e60 msgr2=0x7f914c115c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:37:18.571 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:18.569+0000 7f9150c00700 1 --2- 172.21.15.18:0/359324888 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f914c104e60 0x7f914c115c20 secure :-1 s=READY pgs=955 cs=0 l=1 rev1=1 crypto rx=0x7f913c00b8b0 tx=0x7f913c009e30 comp rx=0 tx=0).stop 2024-04-12T21:37:18.571 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:18.569+0000 7f9150c00700 1 -- 172.21.15.18:0/359324888 shutdown_connections 2024-04-12T21:37:18.571 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:18.569+0000 7f9150c00700 1 --2- 172.21.15.18:0/359324888 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f913006ec00 0x7f91300710b0 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:37:18.571 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:18.569+0000 7f9150c00700 1 --2- 172.21.15.18:0/359324888 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f914c104e60 0x7f914c115c20 unknown :-1 s=CLOSED pgs=955 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:37:18.571 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:18.569+0000 7f9150c00700 1 --2- 172.21.15.18:0/359324888 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f914c105800 0x7f914c116160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:37:18.571 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:18.569+0000 7f9150c00700 1 -- 172.21.15.18:0/359324888 >> 172.21.15.18:0/359324888 conn(0x7f914c0fab10 msgr2=0x7f914c0fcb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:37:18.571 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:18.570+0000 7f9150c00700 1 -- 172.21.15.18:0/359324888 shutdown_connections 2024-04-12T21:37:18.571 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:18.570+0000 7f9150c00700 1 -- 172.21.15.18:0/359324888 wait complete. 2024-04-12T21:37:18.572 INFO:teuthology.orchestra.run.smithi018.stderr:dumped fsmap epoch 31 2024-04-12T21:37:18.731 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:18.730+0000 7f2b36e00700 1 -- 172.21.15.18:0/1310050562 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2b300fc620 msgr2=0x7f2b30100cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:37:18.731 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:18.730+0000 7f2b36e00700 1 --2- 172.21.15.18:0/1310050562 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2b300fc620 0x7f2b30100cd0 secure :-1 s=READY pgs=956 cs=0 l=1 rev1=1 crypto rx=0x7f2b1c004940 tx=0x7f2b1c009e30 comp rx=0 tx=0).stop 2024-04-12T21:37:18.731 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:18.730+0000 7f2b36e00700 1 -- 172.21.15.18:0/1310050562 shutdown_connections 2024-04-12T21:37:18.731 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:18.730+0000 7f2b36e00700 1 --2- 172.21.15.18:0/1310050562 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2b300fc620 0x7f2b30100cd0 unknown :-1 s=CLOSED pgs=956 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:37:18.731 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:18.730+0000 7f2b36e00700 1 --2- 172.21.15.18:0/1310050562 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2b300fbc80 0x7f2b300fc050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:37:18.731 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:18.730+0000 7f2b36e00700 1 -- 172.21.15.18:0/1310050562 >> 172.21.15.18:0/1310050562 conn(0x7f2b30072bc0 msgr2=0x7f2b30072fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:37:18.731 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:18.730+0000 7f2b36e00700 1 -- 172.21.15.18:0/1310050562 shutdown_connections 2024-04-12T21:37:18.731 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:18.730+0000 7f2b36e00700 1 -- 172.21.15.18:0/1310050562 wait complete. 2024-04-12T21:37:18.731 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:18.731+0000 7f2b36e00700 1 Processor -- start 2024-04-12T21:37:18.733 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:18.731+0000 7f2b36e00700 1 -- start start 2024-04-12T21:37:18.733 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:18.731+0000 7f2b36e00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2b300fbc80 0x7f2b3011a1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:37:18.733 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:18.731+0000 7f2b36e00700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2b300fc620 0x7f2b3011a710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:37:18.733 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:18.731+0000 7f2b36e00700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b3011ad60 con 0x7f2b300fbc80 2024-04-12T21:37:18.733 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:18.731+0000 7f2b36e00700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b30114130 con 0x7f2b300fc620 2024-04-12T21:37:18.733 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:18.732+0000 7f2b2fe00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2b300fbc80 0x7f2b3011a1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:37:18.733 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:18.732+0000 7f2b2fe00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2b300fbc80 0x7f2b3011a1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:39560/0 (socket says 172.21.15.18:39560) 2024-04-12T21:37:18.733 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:18.732+0000 7f2b2fe00700 1 -- 172.21.15.18:0/983896309 learned_addr learned my addr 172.21.15.18:0/983896309 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:37:18.733 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:18.732+0000 7f2b2f400700 1 --2- 172.21.15.18:0/983896309 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2b300fc620 0x7f2b3011a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:37:18.733 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:18.732+0000 7f2b2fe00700 1 -- 172.21.15.18:0/983896309 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2b300fc620 msgr2=0x7f2b3011a710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:37:18.734 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:18.732+0000 7f2b2fe00700 1 --2- 172.21.15.18:0/983896309 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2b300fc620 0x7f2b3011a710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:37:18.734 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:18.732+0000 7f2b2fe00700 1 -- 172.21.15.18:0/983896309 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b1c005040 con 0x7f2b300fbc80 2024-04-12T21:37:18.734 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:18.733+0000 7f2b2fe00700 1 --2- 172.21.15.18:0/983896309 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2b300fbc80 0x7f2b3011a1d0 secure :-1 s=READY pgs=957 cs=0 l=1 rev1=1 crypto rx=0x7f2b20002a50 tx=0x7f2b200046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:37:18.734 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:18.733+0000 7f2b27e00700 1 -- 172.21.15.18:0/983896309 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2b20007cc0 con 0x7f2b300fbc80 2024-04-12T21:37:18.735 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:18.733+0000 7f2b27e00700 1 -- 172.21.15.18:0/983896309 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f2b2000f040 con 0x7f2b300fbc80 2024-04-12T21:37:18.735 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:18.733+0000 7f2b36e00700 1 -- 172.21.15.18:0/983896309 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2b30114410 con 0x7f2b300fbc80 2024-04-12T21:37:18.735 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:18.733+0000 7f2b27e00700 1 -- 172.21.15.18:0/983896309 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2b20009640 con 0x7f2b300fbc80 2024-04-12T21:37:18.735 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:18.733+0000 7f2b36e00700 1 -- 172.21.15.18:0/983896309 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2b30114950 con 0x7f2b300fbc80 2024-04-12T21:37:18.735 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:18.734+0000 7f2b36e00700 1 -- 172.21.15.18:0/983896309 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2b30065e30 con 0x7f2b300fbc80 2024-04-12T21:37:18.736 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:18.735+0000 7f2b27e00700 1 -- 172.21.15.18:0/983896309 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7f2b200097a0 con 0x7f2b300fbc80 2024-04-12T21:37:18.736 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:18.736+0000 7f2b27e00700 1 --2- 172.21.15.18:0/983896309 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f2b1406ec10 0x7f2b140710c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:37:18.737 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:18.736+0000 7f2b27e00700 1 -- 172.21.15.18:0/983896309 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 6757+0+0 (secure 0 0 0) 0x7f2b2008e760 con 0x7f2b300fbc80 2024-04-12T21:37:18.737 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:18.737+0000 7f2b2f400700 1 --2- 172.21.15.18:0/983896309 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f2b1406ec10 0x7f2b140710c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:37:18.738 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:18.738+0000 7f2b2f400700 1 --2- 172.21.15.18:0/983896309 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f2b1406ec10 0x7f2b140710c0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f2b1c00b020 tx=0x7f2b1c009e30 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:37:18.741 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:18.741+0000 7f2b27e00700 1 -- 172.21.15.18:0/983896309 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f2b2005af10 con 0x7f2b300fbc80 2024-04-12T21:37:18.992 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:18.992+0000 7f2b36e00700 1 -- 172.21.15.18:0/983896309 --> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f2b3004bf40 con 0x7f2b1406ec10 2024-04-12T21:37:18.994 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:18.994+0000 7f2b27e00700 1 -- 172.21.15.18:0/983896309 <== mgr.34102 v2:172.21.15.18:6800/3812891919 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f2b3004bf40 con 0x7f2b1406ec10 2024-04-12T21:37:18.995 INFO:teuthology.orchestra.run.smithi018.stdout:{ 2024-04-12T21:37:18.995 INFO:teuthology.orchestra.run.smithi018.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:f4452def3be4d350f4a8e5feaf7ebb1511be9298fe68478ba7a2d6fbd7d69a63", 2024-04-12T21:37:18.995 INFO:teuthology.orchestra.run.smithi018.stdout: "in_progress": true, 2024-04-12T21:37:18.995 INFO:teuthology.orchestra.run.smithi018.stdout: "which": "Upgrading all daemon types on all hosts", 2024-04-12T21:37:18.995 INFO:teuthology.orchestra.run.smithi018.stdout: "services_complete": [ 2024-04-12T21:37:18.995 INFO:teuthology.orchestra.run.smithi018.stdout: "crash", 2024-04-12T21:37:18.995 INFO:teuthology.orchestra.run.smithi018.stdout: "osd", 2024-04-12T21:37:18.995 INFO:teuthology.orchestra.run.smithi018.stdout: "mon", 2024-04-12T21:37:18.995 INFO:teuthology.orchestra.run.smithi018.stdout: "mgr" 2024-04-12T21:37:18.995 INFO:teuthology.orchestra.run.smithi018.stdout: ], 2024-04-12T21:37:18.995 INFO:teuthology.orchestra.run.smithi018.stdout: "progress": "15/21 daemons upgraded", 2024-04-12T21:37:18.995 INFO:teuthology.orchestra.run.smithi018.stdout: "message": "Currently upgrading mds daemons", 2024-04-12T21:37:18.995 INFO:teuthology.orchestra.run.smithi018.stdout: "is_paused": false 2024-04-12T21:37:18.995 INFO:teuthology.orchestra.run.smithi018.stdout:} 2024-04-12T21:37:18.997 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:18.997+0000 7f2b36e00700 1 -- 172.21.15.18:0/983896309 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f2b1406ec10 msgr2=0x7f2b140710c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:37:18.997 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:18.997+0000 7f2b36e00700 1 --2- 172.21.15.18:0/983896309 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f2b1406ec10 0x7f2b140710c0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f2b1c00b020 tx=0x7f2b1c009e30 comp rx=0 tx=0).stop 2024-04-12T21:37:18.998 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:18.997+0000 7f2b36e00700 1 -- 172.21.15.18:0/983896309 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2b300fbc80 msgr2=0x7f2b3011a1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:37:18.998 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:18.997+0000 7f2b36e00700 1 --2- 172.21.15.18:0/983896309 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2b300fbc80 0x7f2b3011a1d0 secure :-1 s=READY pgs=957 cs=0 l=1 rev1=1 crypto rx=0x7f2b20002a50 tx=0x7f2b200046b0 comp rx=0 tx=0).stop 2024-04-12T21:37:18.998 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:18.997+0000 7f2b36e00700 1 -- 172.21.15.18:0/983896309 shutdown_connections 2024-04-12T21:37:18.998 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:18.997+0000 7f2b36e00700 1 --2- 172.21.15.18:0/983896309 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f2b1406ec10 0x7f2b140710c0 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:37:18.998 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:18.997+0000 7f2b36e00700 1 --2- 172.21.15.18:0/983896309 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2b300fbc80 0x7f2b3011a1d0 unknown :-1 s=CLOSED pgs=957 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:37:18.998 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:18.997+0000 7f2b36e00700 1 --2- 172.21.15.18:0/983896309 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2b300fc620 0x7f2b3011a710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:37:18.998 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:18.997+0000 7f2b36e00700 1 -- 172.21.15.18:0/983896309 >> 172.21.15.18:0/983896309 conn(0x7f2b30072bc0 msgr2=0x7f2b3010aab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:37:18.998 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:18.997+0000 7f2b36e00700 1 -- 172.21.15.18:0/983896309 shutdown_connections 2024-04-12T21:37:18.998 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:18.997+0000 7f2b36e00700 1 -- 172.21.15.18:0/983896309 wait complete. 2024-04-12T21:37:19.148 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:19.147+0000 7fd2cf600700 1 -- 172.21.15.18:0/4050620668 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd2c80708a0 msgr2=0x7fd2c8070d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:37:19.148 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:19.147+0000 7fd2cf600700 1 --2- 172.21.15.18:0/4050620668 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd2c80708a0 0x7fd2c8070d10 secure :-1 s=READY pgs=958 cs=0 l=1 rev1=1 crypto rx=0x7fd2b4004970 tx=0x7fd2b4009e30 comp rx=0 tx=0).stop 2024-04-12T21:37:19.148 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:19.147+0000 7fd2cf600700 1 -- 172.21.15.18:0/4050620668 shutdown_connections 2024-04-12T21:37:19.148 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:19.147+0000 7fd2cf600700 1 --2- 172.21.15.18:0/4050620668 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd2c80708a0 0x7fd2c8070d10 unknown :-1 s=CLOSED pgs=958 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:37:19.148 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:19.147+0000 7fd2cf600700 1 --2- 172.21.15.18:0/4050620668 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd2c8071f90 0x7fd2c8070360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:37:19.148 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:19.147+0000 7fd2cf600700 1 -- 172.21.15.18:0/4050620668 >> 172.21.15.18:0/4050620668 conn(0x7fd2c80f89b0 msgr2=0x7fd2c80fadc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:37:19.148 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:19.147+0000 7fd2cf600700 1 -- 172.21.15.18:0/4050620668 shutdown_connections 2024-04-12T21:37:19.148 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:19.147+0000 7fd2cf600700 1 -- 172.21.15.18:0/4050620668 wait complete. 2024-04-12T21:37:19.149 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:19.148+0000 7fd2cf600700 1 Processor -- start 2024-04-12T21:37:19.149 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:19.149+0000 7fd2cf600700 1 -- start start 2024-04-12T21:37:19.149 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:19.149+0000 7fd2cf600700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd2c80708a0 0x7fd2c811a170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:37:19.149 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:19.149+0000 7fd2cf600700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd2c8071f90 0x7fd2c811a6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:37:19.149 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:19.149+0000 7fd2cf600700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2c811ad00 con 0x7fd2c8071f90 2024-04-12T21:37:19.149 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:19.149+0000 7fd2cf600700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2c81140d0 con 0x7fd2c80708a0 2024-04-12T21:37:19.151 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:19.149+0000 7fd2c7e00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd2c8071f90 0x7fd2c811a6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:37:19.151 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:19.149+0000 7fd2c7e00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd2c8071f90 0x7fd2c811a6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:39568/0 (socket says 172.21.15.18:39568) 2024-04-12T21:37:19.151 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:19.149+0000 7fd2c7e00700 1 -- 172.21.15.18:0/1288320290 learned_addr learned my addr 172.21.15.18:0/1288320290 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:37:19.151 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:19.150+0000 7fd2ccc00700 1 --2- 172.21.15.18:0/1288320290 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd2c80708a0 0x7fd2c811a170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:37:19.151 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:19.150+0000 7fd2c7e00700 1 -- 172.21.15.18:0/1288320290 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd2c80708a0 msgr2=0x7fd2c811a170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:37:19.151 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:19.150+0000 7fd2c7e00700 1 --2- 172.21.15.18:0/1288320290 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd2c80708a0 0x7fd2c811a170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:37:19.151 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:19.150+0000 7fd2c7e00700 1 -- 172.21.15.18:0/1288320290 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2b4005040 con 0x7fd2c8071f90 2024-04-12T21:37:19.151 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:19.150+0000 7fd2ccc00700 1 --2- 172.21.15.18:0/1288320290 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd2c80708a0 0x7fd2c811a170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-12T21:37:19.151 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:19.150+0000 7fd2c7e00700 1 --2- 172.21.15.18:0/1288320290 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd2c8071f90 0x7fd2c811a6b0 secure :-1 s=READY pgs=959 cs=0 l=1 rev1=1 crypto rx=0x7fd2b4004f50 tx=0x7fd2b4009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:37:19.151 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:19.150+0000 7fd2c4c00700 1 -- 172.21.15.18:0/1288320290 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd2b400e040 con 0x7fd2c8071f90 2024-04-12T21:37:19.152 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:19.150+0000 7fd2c4c00700 1 -- 172.21.15.18:0/1288320290 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd2b400bc10 con 0x7fd2c8071f90 2024-04-12T21:37:19.152 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:19.150+0000 7fd2cf600700 1 -- 172.21.15.18:0/1288320290 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd2c8114350 con 0x7fd2c8071f90 2024-04-12T21:37:19.152 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:19.150+0000 7fd2c4c00700 1 -- 172.21.15.18:0/1288320290 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd2b401c630 con 0x7fd2c8071f90 2024-04-12T21:37:19.152 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:19.151+0000 7fd2cf600700 1 -- 172.21.15.18:0/1288320290 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd2c8114890 con 0x7fd2c8071f90 2024-04-12T21:37:19.152 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:19.152+0000 7fd2cf600700 1 -- 172.21.15.18:0/1288320290 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd2c8071aa0 con 0x7fd2c8071f90 2024-04-12T21:37:19.153 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:19.153+0000 7fd2c4c00700 1 -- 172.21.15.18:0/1288320290 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7fd2b4013420 con 0x7fd2c8071f90 2024-04-12T21:37:19.154 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:19.154+0000 7fd2c4c00700 1 --2- 172.21.15.18:0/1288320290 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fd2b006ec10 0x7fd2b00710c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:37:19.154 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:19.154+0000 7fd2c4c00700 1 -- 172.21.15.18:0/1288320290 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 6757+0+0 (secure 0 0 0) 0x7fd2b408fd80 con 0x7fd2c8071f90 2024-04-12T21:37:19.154 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:19.154+0000 7fd2ccc00700 1 --2- 172.21.15.18:0/1288320290 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fd2b006ec10 0x7fd2b00710c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:37:19.155 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:19.155+0000 7fd2ccc00700 1 --2- 172.21.15.18:0/1288320290 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fd2b006ec10 0x7fd2b00710c0 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7fd2bc004550 tx=0x7fd2bc009040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:37:19.159 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:19.158+0000 7fd2c4c00700 1 -- 172.21.15.18:0/1288320290 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7fd2b405c530 con 0x7fd2c8071f90 2024-04-12T21:37:19.159 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:37:18 smithi018 ceph-mon[112590]: from='client.35964 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:37:19.159 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:37:18 smithi018 ceph-mon[112590]: from='client.35968 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:37:19.159 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:37:18 smithi018 ceph-mon[112590]: pgmap v1126: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:37:19.159 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:37:18 smithi018 ceph-mon[112590]: from='client.? 172.21.15.18:0/2105051993' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T21:37:19.159 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:37:18 smithi018 ceph-mon[112590]: from='client.? 172.21.15.18:0/359324888' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-12T21:37:19.283 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:37:18 smithi186 ceph-mon[86379]: from='client.35964 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:37:19.283 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:37:18 smithi186 ceph-mon[86379]: from='client.35968 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:37:19.283 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:37:18 smithi186 ceph-mon[86379]: pgmap v1126: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:37:19.283 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:37:18 smithi186 ceph-mon[86379]: from='client.? 172.21.15.18:0/2105051993' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T21:37:19.283 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:37:18 smithi186 ceph-mon[86379]: from='client.? 172.21.15.18:0/359324888' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-12T21:37:19.485 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:19.485+0000 7fd2cf600700 1 -- 172.21.15.18:0/1288320290 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7fd2c804bed0 con 0x7fd2c8071f90 2024-04-12T21:37:19.486 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:19.486+0000 7fd2c4c00700 1 -- 172.21.15.18:0/1288320290 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+4303 (secure 0 0 0) 0x7fd2b405c0c0 con 0x7fd2c8071f90 2024-04-12T21:37:19.486 INFO:teuthology.orchestra.run.smithi018.stdout:HEALTH_ERR failed to probe daemons or devices; 1 filesystem is degraded; 1 filesystem is offline 2024-04-12T21:37:19.486 INFO:teuthology.orchestra.run.smithi018.stdout:[WRN] CEPHADM_REFRESH_FAILED: failed to probe daemons or devices 2024-04-12T21:37:19.487 INFO:teuthology.orchestra.run.smithi018.stdout: host smithi018 `cephadm ceph-volume` failed: cephadm exited with an error code: 1, stderr: Inferring config /var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/mon.smithi018/config 2024-04-12T21:37:19.487 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/podman: timeout after 895 seconds 2024-04-12T21:37:19.487 INFO:teuthology.orchestra.run.smithi018.stdout:Non-zero exit code 124 from /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:f4452def3be4d350f4a8e5feaf7ebb1511be9298fe68478ba7a2d6fbd7d69a63 -e NODE_NAME=smithi018 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968:/var/run/ceph:z -v /var/log/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968:/var/log/ceph:z -v /var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmpxsmwkkuh:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:f4452def3be4d350f4a8e5feaf7ebb1511be9298fe68478ba7a2d6fbd7d69a63 inventory --format=json-pretty --filter-for-batch 2024-04-12T21:37:19.487 INFO:teuthology.orchestra.run.smithi018.stdout:Traceback (most recent call last): 2024-04-12T21:37:19.487 INFO:teuthology.orchestra.run.smithi018.stdout: File "/usr/lib64/python3.9/runpy.py", line 197, in _run_module_as_main 2024-04-12T21:37:19.487 INFO:teuthology.orchestra.run.smithi018.stdout: return _run_code(code, main_globals, None, 2024-04-12T21:37:19.487 INFO:teuthology.orchestra.run.smithi018.stdout: File "/usr/lib64/python3.9/runpy.py", line 87, in _run_code 2024-04-12T21:37:19.487 INFO:teuthology.orchestra.run.smithi018.stdout: exec(code, run_globals) 2024-04-12T21:37:19.487 INFO:teuthology.orchestra.run.smithi018.stdout: File "/var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/cephadm.e44f3ed5e34e4ef52eedef233d1190bff004470d8e8005127b5312bff357fa7f/__main__.py", line 5618, in 2024-04-12T21:37:19.487 INFO:teuthology.orchestra.run.smithi018.stdout: File "/var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/cephadm.e44f3ed5e34e4ef52eedef233d1190bff004470d8e8005127b5312bff357fa7f/__main__.py", line 5606, in main 2024-04-12T21:37:19.487 INFO:teuthology.orchestra.run.smithi018.stdout: File "/var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/cephadm.e44f3ed5e34e4ef52eedef233d1190bff004470d8e8005127b5312bff357fa7f/__main__.py", line 408, in _infer_config 2024-04-12T21:37:19.487 INFO:teuthology.orchestra.run.smithi018.stdout: File "/var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/cephadm.e44f3ed5e34e4ef52eedef233d1190bff004470d8e8005127b5312bff357fa7f/__main__.py", line 323, in _infer_fsid 2024-04-12T21:37:19.487 INFO:teuthology.orchestra.run.smithi018.stdout: File "/var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/cephadm.e44f3ed5e34e4ef52eedef233d1190bff004470d8e8005127b5312bff357fa7f/__main__.py", line 436, in _infer_image 2024-04-12T21:37:19.487 INFO:teuthology.orchestra.run.smithi018.stdout: File "/var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/cephadm.e44f3ed5e34e4ef52eedef233d1190bff004470d8e8005127b5312bff357fa7f/__main__.py", line 310, in _validate_fsid 2024-04-12T21:37:19.487 INFO:teuthology.orchestra.run.smithi018.stdout: File "/var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/cephadm.e44f3ed5e34e4ef52eedef233d1190bff004470d8e8005127b5312bff357fa7f/__main__.py", line 3359, in command_ceph_volume 2024-04-12T21:37:19.487 INFO:teuthology.orchestra.run.smithi018.stdout: File "/var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/cephadm.e44f3ed5e34e4ef52eedef233d1190bff004470d8e8005127b5312bff357fa7f/cephadmlib/call_wrappers.py", line 310, in call_throws 2024-04-12T21:37:19.488 INFO:teuthology.orchestra.run.smithi018.stdout:RuntimeError: Failed command: /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:f4452def3be4d350f4a8e5feaf7ebb1511be9298fe68478ba7a2d6fbd7d69a63 -e NODE_NAME=smithi018 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968:/var/run/ceph:z -v /var/log/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968:/var/log/ceph:z -v /var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmpxsmwkkuh:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:f4452def3be4d350f4a8e5feaf7ebb1511be9298fe68478ba7a2d6fbd7d69a63 inventory --format=json-pretty --filter-for-batch 2024-04-12T21:37:19.488 INFO:teuthology.orchestra.run.smithi018.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-04-12T21:37:19.488 INFO:teuthology.orchestra.run.smithi018.stdout: fs cephfs is degraded 2024-04-12T21:37:19.488 INFO:teuthology.orchestra.run.smithi018.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-04-12T21:37:19.488 INFO:teuthology.orchestra.run.smithi018.stdout: fs cephfs is offline because no MDS is active for it. 2024-04-12T21:37:19.489 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:19.489+0000 7fd2cf600700 1 -- 172.21.15.18:0/1288320290 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fd2b006ec10 msgr2=0x7fd2b00710c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:37:19.490 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:19.489+0000 7fd2cf600700 1 --2- 172.21.15.18:0/1288320290 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fd2b006ec10 0x7fd2b00710c0 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7fd2bc004550 tx=0x7fd2bc009040 comp rx=0 tx=0).stop 2024-04-12T21:37:19.490 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:19.489+0000 7fd2cf600700 1 -- 172.21.15.18:0/1288320290 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd2c8071f90 msgr2=0x7fd2c811a6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:37:19.490 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:19.489+0000 7fd2cf600700 1 --2- 172.21.15.18:0/1288320290 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd2c8071f90 0x7fd2c811a6b0 secure :-1 s=READY pgs=959 cs=0 l=1 rev1=1 crypto rx=0x7fd2b4004f50 tx=0x7fd2b4009e30 comp rx=0 tx=0).stop 2024-04-12T21:37:19.490 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:19.489+0000 7fd2cf600700 1 -- 172.21.15.18:0/1288320290 shutdown_connections 2024-04-12T21:37:19.490 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:19.489+0000 7fd2cf600700 1 --2- 172.21.15.18:0/1288320290 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fd2b006ec10 0x7fd2b00710c0 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:37:19.490 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:19.489+0000 7fd2cf600700 1 --2- 172.21.15.18:0/1288320290 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd2c80708a0 0x7fd2c811a170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:37:19.490 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:19.489+0000 7fd2cf600700 1 --2- 172.21.15.18:0/1288320290 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd2c8071f90 0x7fd2c811a6b0 unknown :-1 s=CLOSED pgs=959 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:37:19.490 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:19.489+0000 7fd2cf600700 1 -- 172.21.15.18:0/1288320290 >> 172.21.15.18:0/1288320290 conn(0x7fd2c80f89b0 msgr2=0x7fd2c80f9a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:37:19.490 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:19.489+0000 7fd2cf600700 1 -- 172.21.15.18:0/1288320290 shutdown_connections 2024-04-12T21:37:19.490 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:19.489+0000 7fd2cf600700 1 -- 172.21.15.18:0/1288320290 wait complete. 2024-04-12T21:37:20.283 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:37:19 smithi186 ceph-mon[86379]: from='client.? 172.21.15.18:0/1288320290' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-12T21:37:20.354 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:37:19 smithi018 ceph-mon[112590]: from='client.? 172.21.15.18:0/1288320290' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-12T21:37:21.184 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:37:20 smithi186 ceph-mon[86379]: from='client.35980 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:37:21.185 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:37:20 smithi186 ceph-mon[86379]: pgmap v1127: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:37:21.354 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:37:20 smithi018 ceph-mon[112590]: from='client.35980 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:37:21.354 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:37:20 smithi018 ceph-mon[112590]: pgmap v1127: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:37:23.283 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:37:22 smithi186 ceph-mon[86379]: pgmap v1128: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:37:23.354 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:37:22 smithi018 ceph-mon[112590]: pgmap v1128: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:37:25.283 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:37:24 smithi186 ceph-mon[86379]: pgmap v1129: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:37:25.283 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:37:24 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T21:37:25.354 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:37:24 smithi018 ceph-mon[112590]: pgmap v1129: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:37:25.354 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:37:24 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T21:37:27.283 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:37:26 smithi186 ceph-mon[86379]: pgmap v1130: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:37:27.354 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:37:26 smithi018 ceph-mon[112590]: pgmap v1130: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:37:29.283 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:37:28 smithi186 ceph-mon[86379]: pgmap v1131: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:37:29.354 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:37:28 smithi018 ceph-mon[112590]: pgmap v1131: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:37:30.283 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:37:30 smithi186 ceph-mon[86379]: pgmap v1132: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:37:30.354 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:37:30 smithi018 ceph-mon[112590]: pgmap v1132: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:37:33.283 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:37:32 smithi186 ceph-mon[86379]: pgmap v1133: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:37:33.354 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:37:32 smithi018 ceph-mon[112590]: pgmap v1133: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:37:35.283 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:37:34 smithi186 ceph-mon[86379]: pgmap v1134: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:37:35.354 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:37:34 smithi018 ceph-mon[112590]: pgmap v1134: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:37:37.283 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:37:36 smithi186 ceph-mon[86379]: pgmap v1135: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:37:37.354 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:37:36 smithi018 ceph-mon[112590]: pgmap v1135: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:37:39.283 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:37:38 smithi186 ceph-mon[86379]: pgmap v1136: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:37:39.354 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:37:38 smithi018 ceph-mon[112590]: pgmap v1136: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:37:41.283 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:37:40 smithi186 ceph-mon[86379]: pgmap v1137: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:37:41.283 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:37:40 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T21:37:41.354 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:37:40 smithi018 ceph-mon[112590]: pgmap v1137: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:37:41.354 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:37:40 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T21:37:43.283 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:37:42 smithi186 ceph-mon[86379]: pgmap v1138: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:37:43.354 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:37:42 smithi018 ceph-mon[112590]: pgmap v1138: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:37:45.283 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:37:44 smithi186 ceph-mon[86379]: pgmap v1139: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:37:45.354 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:37:44 smithi018 ceph-mon[112590]: pgmap v1139: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:37:47.283 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:37:46 smithi186 ceph-mon[86379]: pgmap v1140: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:37:47.354 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:37:46 smithi018 ceph-mon[112590]: pgmap v1140: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:37:49.283 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:37:48 smithi186 ceph-mon[86379]: pgmap v1141: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:37:49.354 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:37:48 smithi018 ceph-mon[112590]: pgmap v1141: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:37:49.665 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:49.664+0000 7ffa49a00700 1 -- 172.21.15.18:0/1111141980 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ffa441037c0 msgr2=0x7ffa440fce30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:37:49.665 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:49.664+0000 7ffa49a00700 1 --2- 172.21.15.18:0/1111141980 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ffa441037c0 0x7ffa440fce30 secure :-1 s=READY pgs=960 cs=0 l=1 rev1=1 crypto rx=0x7ffa2c004940 tx=0x7ffa2c009e30 comp rx=0 tx=0).stop 2024-04-12T21:37:49.665 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:49.664+0000 7ffa49a00700 1 -- 172.21.15.18:0/1111141980 shutdown_connections 2024-04-12T21:37:49.665 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:49.664+0000 7ffa49a00700 1 --2- 172.21.15.18:0/1111141980 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ffa441037c0 0x7ffa440fce30 unknown :-1 s=CLOSED pgs=960 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:37:49.665 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:49.664+0000 7ffa49a00700 1 --2- 172.21.15.18:0/1111141980 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ffa44102e20 0x7ffa441031f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:37:49.665 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:49.664+0000 7ffa49a00700 1 -- 172.21.15.18:0/1111141980 >> 172.21.15.18:0/1111141980 conn(0x7ffa440f8970 msgr2=0x7ffa440fad80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:37:49.665 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:49.664+0000 7ffa49a00700 1 -- 172.21.15.18:0/1111141980 shutdown_connections 2024-04-12T21:37:49.665 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:49.664+0000 7ffa49a00700 1 -- 172.21.15.18:0/1111141980 wait complete. 2024-04-12T21:37:49.666 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:49.665+0000 7ffa49a00700 1 Processor -- start 2024-04-12T21:37:49.666 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:49.665+0000 7ffa49a00700 1 -- start start 2024-04-12T21:37:49.666 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:49.666+0000 7ffa49a00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ffa44102e20 0x7ffa44115de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:37:49.666 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:49.666+0000 7ffa49a00700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ffa441037c0 0x7ffa44116320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:37:49.667 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:49.666+0000 7ffa49a00700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa44116970 con 0x7ffa44102e20 2024-04-12T21:37:49.667 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:49.666+0000 7ffa49a00700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa4410fd40 con 0x7ffa441037c0 2024-04-12T21:37:49.667 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:49.666+0000 7ffa42a00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ffa44102e20 0x7ffa44115de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:37:49.667 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:49.666+0000 7ffa42a00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ffa44102e20 0x7ffa44115de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:41154/0 (socket says 172.21.15.18:41154) 2024-04-12T21:37:49.667 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:49.666+0000 7ffa42a00700 1 -- 172.21.15.18:0/4250853649 learned_addr learned my addr 172.21.15.18:0/4250853649 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:37:49.668 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:49.666+0000 7ffa3a000700 1 --2- 172.21.15.18:0/4250853649 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ffa441037c0 0x7ffa44116320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:37:49.668 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:49.666+0000 7ffa42a00700 1 -- 172.21.15.18:0/4250853649 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ffa441037c0 msgr2=0x7ffa44116320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:37:49.668 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:49.666+0000 7ffa42a00700 1 --2- 172.21.15.18:0/4250853649 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ffa441037c0 0x7ffa44116320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:37:49.668 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:49.666+0000 7ffa42a00700 1 -- 172.21.15.18:0/4250853649 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa2c005040 con 0x7ffa44102e20 2024-04-12T21:37:49.668 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:49.667+0000 7ffa42a00700 1 --2- 172.21.15.18:0/4250853649 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ffa44102e20 0x7ffa44115de0 secure :-1 s=READY pgs=961 cs=0 l=1 rev1=1 crypto rx=0x7ffa3400cb10 tx=0x7ffa34007f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:37:49.668 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:49.667+0000 7ffa3b400700 1 -- 172.21.15.18:0/4250853649 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ffa34004200 con 0x7ffa44102e20 2024-04-12T21:37:49.669 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:49.667+0000 7ffa3b400700 1 -- 172.21.15.18:0/4250853649 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ffa3400acc0 con 0x7ffa44102e20 2024-04-12T21:37:49.669 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:49.667+0000 7ffa3b400700 1 -- 172.21.15.18:0/4250853649 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ffa34003d90 con 0x7ffa44102e20 2024-04-12T21:37:49.669 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:49.667+0000 7ffa49a00700 1 -- 172.21.15.18:0/4250853649 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffa44110020 con 0x7ffa44102e20 2024-04-12T21:37:49.669 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:49.667+0000 7ffa49a00700 1 -- 172.21.15.18:0/4250853649 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffa44110590 con 0x7ffa44102e20 2024-04-12T21:37:49.669 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:49.668+0000 7ffa49a00700 1 -- 172.21.15.18:0/4250853649 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffa44102140 con 0x7ffa44102e20 2024-04-12T21:37:49.670 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:49.670+0000 7ffa3b400700 1 -- 172.21.15.18:0/4250853649 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7ffa34003430 con 0x7ffa44102e20 2024-04-12T21:37:49.671 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:49.670+0000 7ffa3b400700 1 --2- 172.21.15.18:0/4250853649 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7ffa2806ec00 0x7ffa280710b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:37:49.671 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:49.671+0000 7ffa3b400700 1 -- 172.21.15.18:0/4250853649 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 6757+0+0 (secure 0 0 0) 0x7ffa3408ef80 con 0x7ffa44102e20 2024-04-12T21:37:49.671 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:49.671+0000 7ffa3a000700 1 --2- 172.21.15.18:0/4250853649 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7ffa2806ec00 0x7ffa280710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:37:49.672 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:49.672+0000 7ffa3a000700 1 --2- 172.21.15.18:0/4250853649 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7ffa2806ec00 0x7ffa280710b0 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7ffa2c000c00 tx=0x7ffa2c009e30 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:37:49.676 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:49.676+0000 7ffa3b400700 1 -- 172.21.15.18:0/4250853649 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7ffa3405b800 con 0x7ffa44102e20 2024-04-12T21:37:49.928 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:49.927+0000 7ffa49a00700 1 -- 172.21.15.18:0/4250853649 --> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7ffa44109f20 con 0x7ffa2806ec00 2024-04-12T21:37:49.931 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:49.931+0000 7ffa3b400700 1 -- 172.21.15.18:0/4250853649 <== mgr.34102 v2:172.21.15.18:6800/3812891919 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7ffa44109f20 con 0x7ffa2806ec00 2024-04-12T21:37:49.935 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:49.934+0000 7ffa49a00700 1 -- 172.21.15.18:0/4250853649 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7ffa2806ec00 msgr2=0x7ffa280710b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:37:49.935 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:49.934+0000 7ffa49a00700 1 --2- 172.21.15.18:0/4250853649 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7ffa2806ec00 0x7ffa280710b0 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7ffa2c000c00 tx=0x7ffa2c009e30 comp rx=0 tx=0).stop 2024-04-12T21:37:49.935 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:49.934+0000 7ffa49a00700 1 -- 172.21.15.18:0/4250853649 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ffa44102e20 msgr2=0x7ffa44115de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:37:49.935 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:49.934+0000 7ffa49a00700 1 --2- 172.21.15.18:0/4250853649 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ffa44102e20 0x7ffa44115de0 secure :-1 s=READY pgs=961 cs=0 l=1 rev1=1 crypto rx=0x7ffa3400cb10 tx=0x7ffa34007f70 comp rx=0 tx=0).stop 2024-04-12T21:37:49.936 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:49.934+0000 7ffa49a00700 1 -- 172.21.15.18:0/4250853649 shutdown_connections 2024-04-12T21:37:49.936 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:49.934+0000 7ffa49a00700 1 --2- 172.21.15.18:0/4250853649 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7ffa2806ec00 0x7ffa280710b0 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:37:49.936 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:49.934+0000 7ffa49a00700 1 --2- 172.21.15.18:0/4250853649 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ffa44102e20 0x7ffa44115de0 unknown :-1 s=CLOSED pgs=961 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:37:49.936 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:49.934+0000 7ffa49a00700 1 --2- 172.21.15.18:0/4250853649 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ffa441037c0 0x7ffa44116320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:37:49.936 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:49.934+0000 7ffa49a00700 1 -- 172.21.15.18:0/4250853649 >> 172.21.15.18:0/4250853649 conn(0x7ffa440f8970 msgr2=0x7ffa440fa640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:37:49.936 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:49.934+0000 7ffa49a00700 1 -- 172.21.15.18:0/4250853649 shutdown_connections 2024-04-12T21:37:49.936 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:49.934+0000 7ffa49a00700 1 -- 172.21.15.18:0/4250853649 wait complete. 2024-04-12T21:37:49.952 INFO:teuthology.orchestra.run.smithi018.stdout:true 2024-04-12T21:37:50.100 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:50.099+0000 7fc852600700 1 -- 172.21.15.18:0/1130117410 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc84c071f90 msgr2=0x7fc84c070360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:37:50.101 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:50.099+0000 7fc852600700 1 --2- 172.21.15.18:0/1130117410 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc84c071f90 0x7fc84c070360 secure :-1 s=READY pgs=962 cs=0 l=1 rev1=1 crypto rx=0x7fc83c009d10 tx=0x7fc83c004d10 comp rx=0 tx=0).stop 2024-04-12T21:37:50.101 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:50.099+0000 7fc852600700 1 -- 172.21.15.18:0/1130117410 shutdown_connections 2024-04-12T21:37:50.101 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:50.099+0000 7fc852600700 1 --2- 172.21.15.18:0/1130117410 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc84c0708a0 0x7fc84c070d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:37:50.101 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:50.099+0000 7fc852600700 1 --2- 172.21.15.18:0/1130117410 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc84c071f90 0x7fc84c070360 unknown :-1 s=CLOSED pgs=962 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:37:50.101 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:50.100+0000 7fc852600700 1 -- 172.21.15.18:0/1130117410 >> 172.21.15.18:0/1130117410 conn(0x7fc84c0f89b0 msgr2=0x7fc84c0fadc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:37:50.101 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:50.100+0000 7fc852600700 1 -- 172.21.15.18:0/1130117410 shutdown_connections 2024-04-12T21:37:50.101 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:50.100+0000 7fc852600700 1 -- 172.21.15.18:0/1130117410 wait complete. 2024-04-12T21:37:50.102 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:50.101+0000 7fc852600700 1 Processor -- start 2024-04-12T21:37:50.103 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:50.101+0000 7fc852600700 1 -- start start 2024-04-12T21:37:50.103 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:50.101+0000 7fc852600700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc84c0708a0 0x7fc84c11a3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:37:50.103 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:50.101+0000 7fc852600700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc84c071f90 0x7fc84c11a910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:37:50.103 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:50.101+0000 7fc852600700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc84c11af60 con 0x7fc84c071f90 2024-04-12T21:37:50.103 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:50.101+0000 7fc852600700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc84c114330 con 0x7fc84c0708a0 2024-04-12T21:37:50.103 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:50.101+0000 7fc84aa00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc84c071f90 0x7fc84c11a910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:37:50.103 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:50.102+0000 7fc84aa00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc84c071f90 0x7fc84c11a910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:41174/0 (socket says 172.21.15.18:41174) 2024-04-12T21:37:50.103 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:50.102+0000 7fc84aa00700 1 -- 172.21.15.18:0/932975356 learned_addr learned my addr 172.21.15.18:0/932975356 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:37:50.103 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:50.102+0000 7fc84b400700 1 --2- 172.21.15.18:0/932975356 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc84c0708a0 0x7fc84c11a3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:37:50.103 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:50.102+0000 7fc84aa00700 1 -- 172.21.15.18:0/932975356 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc84c0708a0 msgr2=0x7fc84c11a3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:37:50.103 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:50.102+0000 7fc84aa00700 1 --2- 172.21.15.18:0/932975356 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc84c0708a0 0x7fc84c11a3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:37:50.103 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:50.102+0000 7fc84aa00700 1 -- 172.21.15.18:0/932975356 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc83c009940 con 0x7fc84c071f90 2024-04-12T21:37:50.103 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:50.102+0000 7fc84aa00700 1 --2- 172.21.15.18:0/932975356 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc84c071f90 0x7fc84c11a910 secure :-1 s=READY pgs=963 cs=0 l=1 rev1=1 crypto rx=0x7fc83800c830 tx=0x7fc838004470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:37:50.104 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:50.103+0000 7fc843400700 1 -- 172.21.15.18:0/932975356 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc83800a760 con 0x7fc84c071f90 2024-04-12T21:37:50.104 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:50.103+0000 7fc843400700 1 -- 172.21.15.18:0/932975356 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc83800ad90 con 0x7fc84c071f90 2024-04-12T21:37:50.104 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:50.103+0000 7fc852600700 1 -- 172.21.15.18:0/932975356 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc84c114610 con 0x7fc84c071f90 2024-04-12T21:37:50.104 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:50.103+0000 7fc843400700 1 -- 172.21.15.18:0/932975356 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc8380113a0 con 0x7fc84c071f90 2024-04-12T21:37:50.104 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:50.103+0000 7fc852600700 1 -- 172.21.15.18:0/932975356 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc84c114b50 con 0x7fc84c071f90 2024-04-12T21:37:50.105 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:50.104+0000 7fc852600700 1 -- 172.21.15.18:0/932975356 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc84c071aa0 con 0x7fc84c071f90 2024-04-12T21:37:50.106 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:50.105+0000 7fc843400700 1 -- 172.21.15.18:0/932975356 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7fc838012400 con 0x7fc84c071f90 2024-04-12T21:37:50.107 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:50.106+0000 7fc843400700 1 --2- 172.21.15.18:0/932975356 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fc83006ec10 0x7fc8300710c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:37:50.107 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:50.106+0000 7fc843400700 1 -- 172.21.15.18:0/932975356 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 6757+0+0 (secure 0 0 0) 0x7fc83808ea80 con 0x7fc84c071f90 2024-04-12T21:37:50.107 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:50.107+0000 7fc84b400700 1 --2- 172.21.15.18:0/932975356 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fc83006ec10 0x7fc8300710c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:37:50.108 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:50.107+0000 7fc84b400700 1 --2- 172.21.15.18:0/932975356 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fc83006ec10 0x7fc8300710c0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7fc83c009ef0 tx=0x7fc83c004d10 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:37:50.112 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:50.112+0000 7fc843400700 1 -- 172.21.15.18:0/932975356 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7fc83805b230 con 0x7fc84c071f90 2024-04-12T21:37:50.366 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:50.366+0000 7fc852600700 1 -- 172.21.15.18:0/932975356 --> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fc84c1105c0 con 0x7fc83006ec10 2024-04-12T21:37:50.368 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:50.368+0000 7fc843400700 1 -- 172.21.15.18:0/932975356 <== mgr.34102 v2:172.21.15.18:6800/3812891919 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7fc83c009700 con 0x7fc83006ec10 2024-04-12T21:37:50.372 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:50.371+0000 7fc852600700 1 -- 172.21.15.18:0/932975356 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fc83006ec10 msgr2=0x7fc8300710c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:37:50.372 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:50.371+0000 7fc852600700 1 --2- 172.21.15.18:0/932975356 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fc83006ec10 0x7fc8300710c0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7fc83c009ef0 tx=0x7fc83c004d10 comp rx=0 tx=0).stop 2024-04-12T21:37:50.372 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:50.371+0000 7fc852600700 1 -- 172.21.15.18:0/932975356 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc84c071f90 msgr2=0x7fc84c11a910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:37:50.372 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:50.371+0000 7fc852600700 1 --2- 172.21.15.18:0/932975356 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc84c071f90 0x7fc84c11a910 secure :-1 s=READY pgs=963 cs=0 l=1 rev1=1 crypto rx=0x7fc83800c830 tx=0x7fc838004470 comp rx=0 tx=0).stop 2024-04-12T21:37:50.372 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:50.371+0000 7fc852600700 1 -- 172.21.15.18:0/932975356 shutdown_connections 2024-04-12T21:37:50.372 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:50.371+0000 7fc852600700 1 --2- 172.21.15.18:0/932975356 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fc83006ec10 0x7fc8300710c0 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:37:50.372 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:50.371+0000 7fc852600700 1 --2- 172.21.15.18:0/932975356 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc84c0708a0 0x7fc84c11a3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:37:50.372 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:50.371+0000 7fc852600700 1 --2- 172.21.15.18:0/932975356 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc84c071f90 0x7fc84c11a910 unknown :-1 s=CLOSED pgs=963 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:37:50.372 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:50.371+0000 7fc852600700 1 -- 172.21.15.18:0/932975356 >> 172.21.15.18:0/932975356 conn(0x7fc84c0f89b0 msgr2=0x7fc84c0f9bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:37:50.372 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:50.371+0000 7fc852600700 1 -- 172.21.15.18:0/932975356 shutdown_connections 2024-04-12T21:37:50.372 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:50.372+0000 7fc852600700 1 -- 172.21.15.18:0/932975356 wait complete. 2024-04-12T21:37:50.527 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:50.526+0000 7fbc4ca00700 1 -- 172.21.15.18:0/3302482254 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fbc48105020 msgr2=0x7fbc481053f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:37:50.527 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:50.526+0000 7fbc4ca00700 1 --2- 172.21.15.18:0/3302482254 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fbc48105020 0x7fbc481053f0 secure :-1 s=READY pgs=964 cs=0 l=1 rev1=1 crypto rx=0x7fbc38004940 tx=0x7fbc38009e30 comp rx=0 tx=0).stop 2024-04-12T21:37:50.527 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:50.526+0000 7fbc4ca00700 1 -- 172.21.15.18:0/3302482254 shutdown_connections 2024-04-12T21:37:50.528 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:50.526+0000 7fbc4ca00700 1 --2- 172.21.15.18:0/3302482254 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbc481059c0 0x7fbc480ff080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:37:50.528 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:50.526+0000 7fbc4ca00700 1 --2- 172.21.15.18:0/3302482254 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fbc48105020 0x7fbc481053f0 unknown :-1 s=CLOSED pgs=964 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:37:50.528 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:50.526+0000 7fbc4ca00700 1 -- 172.21.15.18:0/3302482254 >> 172.21.15.18:0/3302482254 conn(0x7fbc480faba0 msgr2=0x7fbc480fcfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:37:50.528 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:50.527+0000 7fbc4ca00700 1 -- 172.21.15.18:0/3302482254 shutdown_connections 2024-04-12T21:37:50.528 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:50.527+0000 7fbc4ca00700 1 -- 172.21.15.18:0/3302482254 wait complete. 2024-04-12T21:37:50.528 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:50.528+0000 7fbc4ca00700 1 Processor -- start 2024-04-12T21:37:50.528 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:50.528+0000 7fbc4ca00700 1 -- start start 2024-04-12T21:37:50.530 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:50.528+0000 7fbc4ca00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fbc48105020 0x7fbc48115bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:37:50.530 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:50.528+0000 7fbc4ca00700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbc481059c0 0x7fbc48116100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:37:50.530 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:50.529+0000 7fbc4ca00700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc48116750 con 0x7fbc48105020 2024-04-12T21:37:50.530 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:50.529+0000 7fbc4ca00700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc4810fd30 con 0x7fbc481059c0 2024-04-12T21:37:50.530 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:50.529+0000 7fbc45e00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fbc48105020 0x7fbc48115bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:37:50.530 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:50.529+0000 7fbc45e00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fbc48105020 0x7fbc48115bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:41196/0 (socket says 172.21.15.18:41196) 2024-04-12T21:37:50.530 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:50.529+0000 7fbc45e00700 1 -- 172.21.15.18:0/3338538796 learned_addr learned my addr 172.21.15.18:0/3338538796 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:37:50.530 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:50.529+0000 7fbc45400700 1 --2- 172.21.15.18:0/3338538796 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbc481059c0 0x7fbc48116100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:37:50.530 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:50.529+0000 7fbc45e00700 1 -- 172.21.15.18:0/3338538796 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbc481059c0 msgr2=0x7fbc48116100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:37:50.530 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:50.529+0000 7fbc45e00700 1 --2- 172.21.15.18:0/3338538796 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbc481059c0 0x7fbc48116100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:37:50.530 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:50.529+0000 7fbc45e00700 1 -- 172.21.15.18:0/3338538796 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc38005040 con 0x7fbc48105020 2024-04-12T21:37:50.531 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:50.530+0000 7fbc45e00700 1 --2- 172.21.15.18:0/3338538796 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fbc48105020 0x7fbc48115bc0 secure :-1 s=READY pgs=965 cs=0 l=1 rev1=1 crypto rx=0x7fbc38009150 tx=0x7fbc38009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:37:50.531 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:50.530+0000 7fbc3e000700 1 -- 172.21.15.18:0/3338538796 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbc3800e040 con 0x7fbc48105020 2024-04-12T21:37:50.531 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:50.530+0000 7fbc3e000700 1 -- 172.21.15.18:0/3338538796 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fbc38013420 con 0x7fbc48105020 2024-04-12T21:37:50.531 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:50.530+0000 7fbc3e000700 1 -- 172.21.15.18:0/3338538796 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbc3801c630 con 0x7fbc48105020 2024-04-12T21:37:50.531 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:50.530+0000 7fbc4ca00700 1 -- 172.21.15.18:0/3338538796 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbc4810ffb0 con 0x7fbc48105020 2024-04-12T21:37:50.531 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:50.531+0000 7fbc4ca00700 1 -- 172.21.15.18:0/3338538796 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbc48110570 con 0x7fbc48105020 2024-04-12T21:37:50.532 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:50.532+0000 7fbc4ca00700 1 -- 172.21.15.18:0/3338538796 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbc48100770 con 0x7fbc48105020 2024-04-12T21:37:50.534 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:50.533+0000 7fbc3e000700 1 -- 172.21.15.18:0/3338538796 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7fbc38013590 con 0x7fbc48105020 2024-04-12T21:37:50.535 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:50.534+0000 7fbc3e000700 1 --2- 172.21.15.18:0/3338538796 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fbc2c06ec00 0x7fbc2c0710b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:37:50.535 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:50.534+0000 7fbc3e000700 1 -- 172.21.15.18:0/3338538796 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 6757+0+0 (secure 0 0 0) 0x7fbc3808fc00 con 0x7fbc48105020 2024-04-12T21:37:50.535 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:50.535+0000 7fbc45400700 1 --2- 172.21.15.18:0/3338538796 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fbc2c06ec00 0x7fbc2c0710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:37:50.536 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:50.536+0000 7fbc45400700 1 --2- 172.21.15.18:0/3338538796 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fbc2c06ec00 0x7fbc2c0710b0 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7fbc34009fd0 tx=0x7fbc34009f20 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:37:50.540 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:50.541+0000 7fbc3e000700 1 -- 172.21.15.18:0/3338538796 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7fbc3805c430 con 0x7fbc48105020 2024-04-12T21:37:50.782 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:50.782+0000 7fbc4ca00700 1 -- 172.21.15.18:0/3338538796 --> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fbc48111280 con 0x7fbc2c06ec00 2024-04-12T21:37:50.793 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:50.793+0000 7fbc3e000700 1 -- 172.21.15.18:0/3338538796 <== mgr.34102 v2:172.21.15.18:6800/3812891919 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7fbc48111280 con 0x7fbc2c06ec00 2024-04-12T21:37:50.794 INFO:teuthology.orchestra.run.smithi018.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-12T21:37:50.794 INFO:teuthology.orchestra.run.smithi018.stdout:alertmanager.smithi018 smithi018 *:9093,9094 running (38m) 14m ago 52m 15.5M - 0.25.0 c8568f914cd2 c332090068e8 2024-04-12T21:37:50.794 INFO:teuthology.orchestra.run.smithi018.stdout:crash.smithi018 smithi018 running (34m) 14m ago 53m 6665k - 19.0.0-2879-g324baf8d 8717720dc5e5 b7f80d1f7982 2024-04-12T21:37:50.794 INFO:teuthology.orchestra.run.smithi018.stdout:crash.smithi186 smithi186 running (34m) 14m ago 51m 6660k - 19.0.0-2879-g324baf8d 8717720dc5e5 a4f55c81b623 2024-04-12T21:37:50.794 INFO:teuthology.orchestra.run.smithi018.stdout:grafana.smithi018 smithi018 *:3000 running (37m) 14m ago 52m 74.8M - 9.4.12 2bacad6d85d8 b08e3bf176dd 2024-04-12T21:37:50.794 INFO:teuthology.orchestra.run.smithi018.stdout:mds.cephfs.smithi018.acqbsc smithi018 running (30m) 14m ago 45m 16.7M - 19.0.0-2879-g324baf8d 8717720dc5e5 021f99ab24ae 2024-04-12T21:37:50.794 INFO:teuthology.orchestra.run.smithi018.stdout:mds.cephfs.smithi018.szjluo smithi018 running (30m) 14m ago 45m 16.5M - 19.0.0-2879-g324baf8d 8717720dc5e5 0003117479d4 2024-04-12T21:37:50.794 INFO:teuthology.orchestra.run.smithi018.stdout:mds.cephfs.smithi186.jdllyn smithi186 running (14m) 14m ago 45m 11.0M - 19.0.0-2879-g324baf8d 8717720dc5e5 21b1f46eef23 2024-04-12T21:37:50.794 INFO:teuthology.orchestra.run.smithi018.stdout:mds.cephfs.smithi186.wjnafs smithi186 running (45m) 14m ago 45m 19.5M - 17.2.7-827-gd3b5060a 21de9b7369ad 74b115d6d6fd 2024-04-12T21:37:50.794 INFO:teuthology.orchestra.run.smithi018.stdout:mgr.smithi018.xjzwga smithi018 *:8443,9283,8765 running (41m) 14m ago 54m 512M - 19.0.0-2879-g324baf8d 8717720dc5e5 56088305eab7 2024-04-12T21:37:50.794 INFO:teuthology.orchestra.run.smithi018.stdout:mgr.smithi186.kqcfxk smithi186 *:8443,9283,8765 running (39m) 14m ago 51m 459M - 19.0.0-2879-g324baf8d 8717720dc5e5 09cd766e5e0d 2024-04-12T21:37:50.795 INFO:teuthology.orchestra.run.smithi018.stdout:mon.smithi018 smithi018 running (36m) 14m ago 54m 73.1M 2048M 19.0.0-2879-g324baf8d 8717720dc5e5 df3e201dfed6 2024-04-12T21:37:50.795 INFO:teuthology.orchestra.run.smithi018.stdout:mon.smithi186 smithi186 running (35m) 14m ago 51m 47.2M 2048M 19.0.0-2879-g324baf8d 8717720dc5e5 c77498da9b2b 2024-04-12T21:37:50.795 INFO:teuthology.orchestra.run.smithi018.stdout:node-exporter.smithi018 smithi018 *:9100 running (39m) 14m ago 53m 8108k - 1.5.0 0da6a335fe13 326c3d9c6847 2024-04-12T21:37:50.795 INFO:teuthology.orchestra.run.smithi018.stdout:node-exporter.smithi186 smithi186 *:9100 running (39m) 14m ago 51m 8757k - 1.5.0 0da6a335fe13 ec2112baeecf 2024-04-12T21:37:50.795 INFO:teuthology.orchestra.run.smithi018.stdout:osd.0 smithi018 running (34m) 14m ago 50m 123M 1559M 19.0.0-2879-g324baf8d 8717720dc5e5 60b1273ca047 2024-04-12T21:37:50.795 INFO:teuthology.orchestra.run.smithi018.stdout:osd.1 smithi018 running (33m) 14m ago 49m 95.1M 1559M 19.0.0-2879-g324baf8d 8717720dc5e5 170ce1e5fb8d 2024-04-12T21:37:50.795 INFO:teuthology.orchestra.run.smithi018.stdout:osd.2 smithi018 running (33m) 14m ago 49m 94.3M 1559M 19.0.0-2879-g324baf8d 8717720dc5e5 c2cbc6aa2839 2024-04-12T21:37:50.795 INFO:teuthology.orchestra.run.smithi018.stdout:osd.3 smithi186 running (32m) 14m ago 48m 101M 2581M 19.0.0-2879-g324baf8d 8717720dc5e5 0138e8262e4b 2024-04-12T21:37:50.795 INFO:teuthology.orchestra.run.smithi018.stdout:osd.4 smithi186 running (31m) 14m ago 48m 91.0M 2581M 19.0.0-2879-g324baf8d 8717720dc5e5 b20cb63b25c9 2024-04-12T21:37:50.795 INFO:teuthology.orchestra.run.smithi018.stdout:osd.5 smithi186 running (31m) 14m ago 47m 92.0M 2581M 19.0.0-2879-g324baf8d 8717720dc5e5 2513bea2610e 2024-04-12T21:37:50.795 INFO:teuthology.orchestra.run.smithi018.stdout:prometheus.smithi018 smithi018 *:9095 running (38m) 14m ago 52m 67.9M - 2.43.0 a07b618ecd1d f58841127e21 2024-04-12T21:37:50.797 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:50.796+0000 7fbc4ca00700 1 -- 172.21.15.18:0/3338538796 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fbc2c06ec00 msgr2=0x7fbc2c0710b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:37:50.797 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:50.796+0000 7fbc4ca00700 1 --2- 172.21.15.18:0/3338538796 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fbc2c06ec00 0x7fbc2c0710b0 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7fbc34009fd0 tx=0x7fbc34009f20 comp rx=0 tx=0).stop 2024-04-12T21:37:50.797 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:50.796+0000 7fbc4ca00700 1 -- 172.21.15.18:0/3338538796 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fbc48105020 msgr2=0x7fbc48115bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:37:50.797 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:50.796+0000 7fbc4ca00700 1 --2- 172.21.15.18:0/3338538796 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fbc48105020 0x7fbc48115bc0 secure :-1 s=READY pgs=965 cs=0 l=1 rev1=1 crypto rx=0x7fbc38009150 tx=0x7fbc38009e30 comp rx=0 tx=0).stop 2024-04-12T21:37:50.797 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:50.796+0000 7fbc4ca00700 1 -- 172.21.15.18:0/3338538796 shutdown_connections 2024-04-12T21:37:50.797 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:50.796+0000 7fbc4ca00700 1 --2- 172.21.15.18:0/3338538796 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fbc2c06ec00 0x7fbc2c0710b0 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:37:50.797 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:50.796+0000 7fbc4ca00700 1 --2- 172.21.15.18:0/3338538796 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fbc48105020 0x7fbc48115bc0 unknown :-1 s=CLOSED pgs=965 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:37:50.797 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:50.796+0000 7fbc4ca00700 1 --2- 172.21.15.18:0/3338538796 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbc481059c0 0x7fbc48116100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:37:50.797 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:50.796+0000 7fbc4ca00700 1 -- 172.21.15.18:0/3338538796 >> 172.21.15.18:0/3338538796 conn(0x7fbc480faba0 msgr2=0x7fbc480fbba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:37:50.797 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:50.796+0000 7fbc4ca00700 1 -- 172.21.15.18:0/3338538796 shutdown_connections 2024-04-12T21:37:50.797 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:50.796+0000 7fbc4ca00700 1 -- 172.21.15.18:0/3338538796 wait complete. 2024-04-12T21:37:50.956 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:50.955+0000 7f56f3200700 1 -- 172.21.15.18:0/3329316072 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f56f4102c50 msgr2=0x7f56f4103020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:37:50.956 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:50.955+0000 7f56f3200700 1 --2- 172.21.15.18:0/3329316072 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f56f4102c50 0x7f56f4103020 secure :-1 s=READY pgs=966 cs=0 l=1 rev1=1 crypto rx=0x7f56e4004940 tx=0x7f56e4009e30 comp rx=0 tx=0).stop 2024-04-12T21:37:50.956 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:50.955+0000 7f56f3200700 1 -- 172.21.15.18:0/3329316072 shutdown_connections 2024-04-12T21:37:50.956 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:50.955+0000 7f56f3200700 1 --2- 172.21.15.18:0/3329316072 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f56f41035f0 0x7f56f40fcc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:37:50.956 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:50.955+0000 7f56f3200700 1 --2- 172.21.15.18:0/3329316072 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f56f4102c50 0x7f56f4103020 unknown :-1 s=CLOSED pgs=966 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:37:50.957 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:50.955+0000 7f56f3200700 1 -- 172.21.15.18:0/3329316072 >> 172.21.15.18:0/3329316072 conn(0x7f56f40f87e0 msgr2=0x7f56f40fabf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:37:50.957 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:50.956+0000 7f56f3200700 1 -- 172.21.15.18:0/3329316072 shutdown_connections 2024-04-12T21:37:50.957 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:50.956+0000 7f56f3200700 1 -- 172.21.15.18:0/3329316072 wait complete. 2024-04-12T21:37:50.958 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:50.956+0000 7f56f3200700 1 Processor -- start 2024-04-12T21:37:50.958 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:50.957+0000 7f56f3200700 1 -- start start 2024-04-12T21:37:50.958 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:50.957+0000 7f56f3200700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f56f4102c50 0x7f56f4115c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:37:50.958 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:50.957+0000 7f56f3200700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f56f41035f0 0x7f56f4116160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:37:50.958 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:50.957+0000 7f56f3200700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56f4116840 con 0x7f56f41035f0 2024-04-12T21:37:50.958 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:50.957+0000 7f56f3200700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56f410fba0 con 0x7f56f4102c50 2024-04-12T21:37:50.958 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:50.957+0000 7f56f1400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f56f41035f0 0x7f56f4116160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:37:50.959 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:50.957+0000 7f56f1400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f56f41035f0 0x7f56f4116160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:59342/0 (socket says 172.21.15.18:59342) 2024-04-12T21:37:50.959 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:50.957+0000 7f56f1400700 1 -- 172.21.15.18:0/537388797 learned_addr learned my addr 172.21.15.18:0/537388797 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:37:50.959 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:50.958+0000 7f56f1e00700 1 --2- 172.21.15.18:0/537388797 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f56f4102c50 0x7f56f4115c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:37:50.959 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:50.958+0000 7f56f1400700 1 -- 172.21.15.18:0/537388797 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f56f4102c50 msgr2=0x7f56f4115c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:37:50.959 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:50.958+0000 7f56f1400700 1 --2- 172.21.15.18:0/537388797 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f56f4102c50 0x7f56f4115c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:37:50.959 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:50.958+0000 7f56f1400700 1 -- 172.21.15.18:0/537388797 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56e4005040 con 0x7f56f41035f0 2024-04-12T21:37:50.960 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:50.958+0000 7f56f1400700 1 --2- 172.21.15.18:0/537388797 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f56f41035f0 0x7f56f4116160 secure :-1 s=READY pgs=967 cs=0 l=1 rev1=1 crypto rx=0x7f56e000b790 tx=0x7f56e0009220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:37:50.960 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:50.958+0000 7f56ea000700 1 -- 172.21.15.18:0/537388797 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f56e000f6f0 con 0x7f56f41035f0 2024-04-12T21:37:50.960 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:50.958+0000 7f56ea000700 1 -- 172.21.15.18:0/537388797 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f56e000fd20 con 0x7f56f41035f0 2024-04-12T21:37:50.960 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:50.958+0000 7f56ea000700 1 -- 172.21.15.18:0/537388797 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f56e00103a0 con 0x7f56f41035f0 2024-04-12T21:37:50.960 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:50.958+0000 7f56f3200700 1 -- 172.21.15.18:0/537388797 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f56f410fe80 con 0x7f56f41035f0 2024-04-12T21:37:50.960 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:50.959+0000 7f56f3200700 1 -- 172.21.15.18:0/537388797 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f56f4110440 con 0x7f56f41035f0 2024-04-12T21:37:50.960 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:50.960+0000 7f56f3200700 1 -- 172.21.15.18:0/537388797 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f56f40fe3a0 con 0x7f56f41035f0 2024-04-12T21:37:50.961 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:50.961+0000 7f56ea000700 1 -- 172.21.15.18:0/537388797 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7f56e0011400 con 0x7f56f41035f0 2024-04-12T21:37:50.963 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:50.962+0000 7f56ea000700 1 --2- 172.21.15.18:0/537388797 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f56d806ec10 0x7f56d80710c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:37:50.963 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:50.963+0000 7f56ea000700 1 -- 172.21.15.18:0/537388797 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 6757+0+0 (secure 0 0 0) 0x7f56e008e8d0 con 0x7f56f41035f0 2024-04-12T21:37:50.963 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:50.963+0000 7f56f1e00700 1 --2- 172.21.15.18:0/537388797 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f56d806ec10 0x7f56d80710c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:37:50.964 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:50.963+0000 7f56f1e00700 1 --2- 172.21.15.18:0/537388797 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f56d806ec10 0x7f56d80710c0 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f56e400b020 tx=0x7f56e4009e30 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:37:50.968 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:50.967+0000 7f56ea000700 1 -- 172.21.15.18:0/537388797 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f56e005b000 con 0x7f56f41035f0 2024-04-12T21:37:51.104 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:37:50 smithi018 ceph-mon[112590]: pgmap v1142: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:37:51.104 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:37:50 smithi018 ceph-mon[112590]: from='client.35988 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:37:51.283 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:37:50 smithi186 ceph-mon[86379]: pgmap v1142: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:37:51.283 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:37:50 smithi186 ceph-mon[86379]: from='client.35988 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:37:51.288 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:51.287+0000 7f56f3200700 1 -- 172.21.15.18:0/537388797 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f56f4110010 con 0x7f56f41035f0 2024-04-12T21:37:51.289 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:51.289+0000 7f56ea000700 1 -- 172.21.15.18:0/537388797 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+837 (secure 0 0 0) 0x7f56f4110010 con 0x7f56f41035f0 2024-04-12T21:37:51.290 INFO:teuthology.orchestra.run.smithi018.stdout:{ 2024-04-12T21:37:51.290 INFO:teuthology.orchestra.run.smithi018.stdout: "mon": { 2024-04-12T21:37:51.290 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 19.0.0-2879-g324baf8d (324baf8ddb91dea46cac50cb535c306115791355) squid (dev)": 2 2024-04-12T21:37:51.290 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-12T21:37:51.290 INFO:teuthology.orchestra.run.smithi018.stdout: "mgr": { 2024-04-12T21:37:51.290 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 19.0.0-2879-g324baf8d (324baf8ddb91dea46cac50cb535c306115791355) squid (dev)": 2 2024-04-12T21:37:51.290 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-12T21:37:51.290 INFO:teuthology.orchestra.run.smithi018.stdout: "osd": { 2024-04-12T21:37:51.290 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 19.0.0-2879-g324baf8d (324baf8ddb91dea46cac50cb535c306115791355) squid (dev)": 6 2024-04-12T21:37:51.290 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-12T21:37:51.290 INFO:teuthology.orchestra.run.smithi018.stdout: "mds": { 2024-04-12T21:37:51.291 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 17.2.7-827-gd3b5060a (d3b5060a17449edfae9385cdf792afec109fca5d) quincy (stable)": 1, 2024-04-12T21:37:51.291 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 19.0.0-2879-g324baf8d (324baf8ddb91dea46cac50cb535c306115791355) squid (dev)": 3 2024-04-12T21:37:51.291 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-12T21:37:51.291 INFO:teuthology.orchestra.run.smithi018.stdout: "overall": { 2024-04-12T21:37:51.291 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 17.2.7-827-gd3b5060a (d3b5060a17449edfae9385cdf792afec109fca5d) quincy (stable)": 1, 2024-04-12T21:37:51.291 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 19.0.0-2879-g324baf8d (324baf8ddb91dea46cac50cb535c306115791355) squid (dev)": 13 2024-04-12T21:37:51.291 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-04-12T21:37:51.291 INFO:teuthology.orchestra.run.smithi018.stdout:} 2024-04-12T21:37:51.292 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:51.292+0000 7f56f3200700 1 -- 172.21.15.18:0/537388797 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f56d806ec10 msgr2=0x7f56d80710c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:37:51.292 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:51.292+0000 7f56f3200700 1 --2- 172.21.15.18:0/537388797 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f56d806ec10 0x7f56d80710c0 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f56e400b020 tx=0x7f56e4009e30 comp rx=0 tx=0).stop 2024-04-12T21:37:51.293 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:51.292+0000 7f56f3200700 1 -- 172.21.15.18:0/537388797 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f56f41035f0 msgr2=0x7f56f4116160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:37:51.293 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:51.292+0000 7f56f3200700 1 --2- 172.21.15.18:0/537388797 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f56f41035f0 0x7f56f4116160 secure :-1 s=READY pgs=967 cs=0 l=1 rev1=1 crypto rx=0x7f56e000b790 tx=0x7f56e0009220 comp rx=0 tx=0).stop 2024-04-12T21:37:51.293 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:51.292+0000 7f56f3200700 1 -- 172.21.15.18:0/537388797 shutdown_connections 2024-04-12T21:37:51.293 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:51.292+0000 7f56f3200700 1 --2- 172.21.15.18:0/537388797 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f56d806ec10 0x7f56d80710c0 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:37:51.293 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:51.292+0000 7f56f3200700 1 --2- 172.21.15.18:0/537388797 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f56f4102c50 0x7f56f4115c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:37:51.293 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:51.292+0000 7f56f3200700 1 --2- 172.21.15.18:0/537388797 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f56f41035f0 0x7f56f4116160 unknown :-1 s=CLOSED pgs=967 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:37:51.293 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:51.292+0000 7f56f3200700 1 -- 172.21.15.18:0/537388797 >> 172.21.15.18:0/537388797 conn(0x7f56f40f87e0 msgr2=0x7f56f4071a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:37:51.293 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:51.292+0000 7f56f3200700 1 -- 172.21.15.18:0/537388797 shutdown_connections 2024-04-12T21:37:51.293 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:51.292+0000 7f56f3200700 1 -- 172.21.15.18:0/537388797 wait complete. 2024-04-12T21:37:51.442 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:51.442+0000 7f2299200700 1 -- 172.21.15.18:0/2272588139 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2294109dd0 msgr2=0x7f2294103440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:37:51.442 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:51.442+0000 7f2299200700 1 --2- 172.21.15.18:0/2272588139 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2294109dd0 0x7f2294103440 secure :-1 s=READY pgs=968 cs=0 l=1 rev1=1 crypto rx=0x7f2280004940 tx=0x7f2280009e30 comp rx=0 tx=0).stop 2024-04-12T21:37:51.442 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:51.442+0000 7f2299200700 1 -- 172.21.15.18:0/2272588139 shutdown_connections 2024-04-12T21:37:51.442 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:51.442+0000 7f2299200700 1 --2- 172.21.15.18:0/2272588139 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2294109dd0 0x7f2294103440 unknown :-1 s=CLOSED pgs=968 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:37:51.442 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:51.442+0000 7f2299200700 1 --2- 172.21.15.18:0/2272588139 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2294109430 0x7f2294109800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:37:51.442 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:51.442+0000 7f2299200700 1 -- 172.21.15.18:0/2272588139 >> 172.21.15.18:0/2272588139 conn(0x7f2294072c00 msgr2=0x7f2294075010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:37:51.442 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:51.442+0000 7f2299200700 1 -- 172.21.15.18:0/2272588139 shutdown_connections 2024-04-12T21:37:51.442 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:51.442+0000 7f2299200700 1 -- 172.21.15.18:0/2272588139 wait complete. 2024-04-12T21:37:51.443 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:51.443+0000 7f2299200700 1 Processor -- start 2024-04-12T21:37:51.444 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:51.443+0000 7f2299200700 1 -- start start 2024-04-12T21:37:51.444 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:51.443+0000 7f2299200700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2294109430 0x7f2294119f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:37:51.444 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:51.443+0000 7f2299200700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2294109dd0 0x7f229411a4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:37:51.444 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:51.443+0000 7f2299200700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f229411ab10 con 0x7f2294109dd0 2024-04-12T21:37:51.444 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:51.443+0000 7f2299200700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22941140f0 con 0x7f2294109430 2024-04-12T21:37:51.444 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:51.444+0000 7f2291400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2294109dd0 0x7f229411a4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:37:51.444 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:51.444+0000 7f2291400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2294109dd0 0x7f229411a4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:59366/0 (socket says 172.21.15.18:59366) 2024-04-12T21:37:51.444 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:51.444+0000 7f2291400700 1 -- 172.21.15.18:0/230165166 learned_addr learned my addr 172.21.15.18:0/230165166 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:37:51.444 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:51.444+0000 7f2291400700 1 -- 172.21.15.18:0/230165166 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2294109430 msgr2=0x7f2294119f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:37:51.444 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:51.444+0000 7f2291400700 1 --2- 172.21.15.18:0/230165166 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2294109430 0x7f2294119f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:37:51.444 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:51.444+0000 7f2291400700 1 -- 172.21.15.18:0/230165166 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2280005040 con 0x7f2294109dd0 2024-04-12T21:37:51.444 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:51.444+0000 7f2291400700 1 --2- 172.21.15.18:0/230165166 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2294109dd0 0x7f229411a4c0 secure :-1 s=READY pgs=969 cs=0 l=1 rev1=1 crypto rx=0x7f2280009100 tx=0x7f2280009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:37:51.445 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:51.445+0000 7f228a000700 1 -- 172.21.15.18:0/230165166 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f228000e040 con 0x7f2294109dd0 2024-04-12T21:37:51.445 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:51.445+0000 7f228a000700 1 -- 172.21.15.18:0/230165166 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f2280004290 con 0x7f2294109dd0 2024-04-12T21:37:51.445 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:51.445+0000 7f228a000700 1 -- 172.21.15.18:0/230165166 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f228001c3f0 con 0x7f2294109dd0 2024-04-12T21:37:51.445 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:51.445+0000 7f2299200700 1 -- 172.21.15.18:0/230165166 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2294114370 con 0x7f2294109dd0 2024-04-12T21:37:51.445 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:51.445+0000 7f2299200700 1 -- 172.21.15.18:0/230165166 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2294114930 con 0x7f2294109dd0 2024-04-12T21:37:51.445 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:51.446+0000 7f2299200700 1 -- 172.21.15.18:0/230165166 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2294104b80 con 0x7f2294109dd0 2024-04-12T21:37:51.446 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:51.447+0000 7f228a000700 1 -- 172.21.15.18:0/230165166 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7f2280013790 con 0x7f2294109dd0 2024-04-12T21:37:51.447 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:51.448+0000 7f228a000700 1 --2- 172.21.15.18:0/230165166 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f227806ec00 0x7f22780710b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:37:51.448 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:51.448+0000 7f228a000700 1 -- 172.21.15.18:0/230165166 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 6757+0+0 (secure 0 0 0) 0x7f228008fc30 con 0x7f2294109dd0 2024-04-12T21:37:51.448 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:51.448+0000 7f2291e00700 1 --2- 172.21.15.18:0/230165166 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f227806ec00 0x7f22780710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:37:51.448 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:51.449+0000 7f2291e00700 1 --2- 172.21.15.18:0/230165166 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f227806ec00 0x7f22780710b0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f2284004d00 tx=0x7f2284009ce0 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:37:51.452 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:51.453+0000 7f228a000700 1 -- 172.21.15.18:0/230165166 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f228005c510 con 0x7f2294109dd0 2024-04-12T21:37:51.725 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:51.725+0000 7f2299200700 1 -- 172.21.15.18:0/230165166 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f2294110500 con 0x7f2294109dd0 2024-04-12T21:37:51.726 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:51.727+0000 7f228a000700 1 -- 172.21.15.18:0/230165166 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 31 v31) v1 ==== 76+0+1873 (secure 0 0 0) 0x7f2280013420 con 0x7f2294109dd0 2024-04-12T21:37:51.727 INFO:teuthology.orchestra.run.smithi018.stdout:e31 2024-04-12T21:37:51.727 INFO:teuthology.orchestra.run.smithi018.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-12T21:37:51.727 INFO:teuthology.orchestra.run.smithi018.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-12T21:37:51.727 INFO:teuthology.orchestra.run.smithi018.stdout:legacy client fscid: 1 2024-04-12T21:37:51.727 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-12T21:37:51.728 INFO:teuthology.orchestra.run.smithi018.stdout:Filesystem 'cephfs' (1) 2024-04-12T21:37:51.728 INFO:teuthology.orchestra.run.smithi018.stdout:fs_name cephfs 2024-04-12T21:37:51.728 INFO:teuthology.orchestra.run.smithi018.stdout:epoch 24 2024-04-12T21:37:51.728 INFO:teuthology.orchestra.run.smithi018.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-04-12T21:37:51.728 INFO:teuthology.orchestra.run.smithi018.stdout:created 2024-04-12T20:51:57.362850+0000 2024-04-12T21:37:51.728 INFO:teuthology.orchestra.run.smithi018.stdout:modified 2024-04-12T21:06:59.717190+0000 2024-04-12T21:37:51.728 INFO:teuthology.orchestra.run.smithi018.stdout:tableserver 0 2024-04-12T21:37:51.728 INFO:teuthology.orchestra.run.smithi018.stdout:root 0 2024-04-12T21:37:51.728 INFO:teuthology.orchestra.run.smithi018.stdout:session_timeout 60 2024-04-12T21:37:51.728 INFO:teuthology.orchestra.run.smithi018.stdout:session_autoclose 300 2024-04-12T21:37:51.728 INFO:teuthology.orchestra.run.smithi018.stdout:max_file_size 1099511627776 2024-04-12T21:37:51.728 INFO:teuthology.orchestra.run.smithi018.stdout:max_xattr_size 65536 2024-04-12T21:37:51.728 INFO:teuthology.orchestra.run.smithi018.stdout:required_client_features {} 2024-04-12T21:37:51.728 INFO:teuthology.orchestra.run.smithi018.stdout:last_failure 0 2024-04-12T21:37:51.728 INFO:teuthology.orchestra.run.smithi018.stdout:last_failure_osd_epoch 100 2024-04-12T21:37:51.729 INFO:teuthology.orchestra.run.smithi018.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-12T21:37:51.729 INFO:teuthology.orchestra.run.smithi018.stdout:max_mds 2 2024-04-12T21:37:51.729 INFO:teuthology.orchestra.run.smithi018.stdout:in 0,1 2024-04-12T21:37:51.729 INFO:teuthology.orchestra.run.smithi018.stdout:up {} 2024-04-12T21:37:51.729 INFO:teuthology.orchestra.run.smithi018.stdout:failed 0,1 2024-04-12T21:37:51.729 INFO:teuthology.orchestra.run.smithi018.stdout:damaged 2024-04-12T21:37:51.729 INFO:teuthology.orchestra.run.smithi018.stdout:stopped 2024-04-12T21:37:51.729 INFO:teuthology.orchestra.run.smithi018.stdout:data_pools [3] 2024-04-12T21:37:51.729 INFO:teuthology.orchestra.run.smithi018.stdout:metadata_pool 2 2024-04-12T21:37:51.729 INFO:teuthology.orchestra.run.smithi018.stdout:inline_data disabled 2024-04-12T21:37:51.729 INFO:teuthology.orchestra.run.smithi018.stdout:balancer 2024-04-12T21:37:51.729 INFO:teuthology.orchestra.run.smithi018.stdout:bal_rank_mask -1 2024-04-12T21:37:51.729 INFO:teuthology.orchestra.run.smithi018.stdout:standby_count_wanted 1 2024-04-12T21:37:51.729 INFO:teuthology.orchestra.run.smithi018.stdout:qdb_cluster leader: 0 members: 2024-04-12T21:37:51.729 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-12T21:37:51.730 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-12T21:37:51.730 INFO:teuthology.orchestra.run.smithi018.stdout:Standby daemons: 2024-04-12T21:37:51.730 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-12T21:37:51.730 INFO:teuthology.orchestra.run.smithi018.stdout:[mds.cephfs.smithi186.wjnafs{-1:24277} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.186:6826/438347207,v1:172.21.15.186:6827/438347207] compat {c=[1],r=[1],i=[7ff]}] 2024-04-12T21:37:51.730 INFO:teuthology.orchestra.run.smithi018.stdout:[mds.cephfs.smithi018.acqbsc{-1:34436} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.18:6828/755179764,v1:172.21.15.18:6829/755179764] compat {c=[1],r=[1],i=[1fff]}] 2024-04-12T21:37:51.730 INFO:teuthology.orchestra.run.smithi018.stdout:[mds.cephfs.smithi018.szjluo{-1:34468} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.18:6826/2407992653,v1:172.21.15.18:6827/2407992653] compat {c=[1],r=[1],i=[1fff]}] 2024-04-12T21:37:51.730 INFO:teuthology.orchestra.run.smithi018.stdout:[mds.cephfs.smithi186.jdllyn{-1:44595} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.186:6824/3523444202,v1:172.21.15.186:6825/3523444202] compat {c=[1],r=[1],i=[1fff]}] 2024-04-12T21:37:51.731 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:51.729+0000 7f2299200700 1 -- 172.21.15.18:0/230165166 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f227806ec00 msgr2=0x7f22780710b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:37:51.731 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:51.729+0000 7f2299200700 1 --2- 172.21.15.18:0/230165166 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f227806ec00 0x7f22780710b0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f2284004d00 tx=0x7f2284009ce0 comp rx=0 tx=0).stop 2024-04-12T21:37:51.731 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:51.729+0000 7f2299200700 1 -- 172.21.15.18:0/230165166 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2294109dd0 msgr2=0x7f229411a4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:37:51.731 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:51.729+0000 7f2299200700 1 --2- 172.21.15.18:0/230165166 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2294109dd0 0x7f229411a4c0 secure :-1 s=READY pgs=969 cs=0 l=1 rev1=1 crypto rx=0x7f2280009100 tx=0x7f2280009e30 comp rx=0 tx=0).stop 2024-04-12T21:37:51.731 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:51.729+0000 7f2299200700 1 -- 172.21.15.18:0/230165166 shutdown_connections 2024-04-12T21:37:51.731 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:51.729+0000 7f2299200700 1 --2- 172.21.15.18:0/230165166 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f227806ec00 0x7f22780710b0 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:37:51.731 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:51.729+0000 7f2299200700 1 --2- 172.21.15.18:0/230165166 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2294109430 0x7f2294119f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:37:51.731 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:51.729+0000 7f2299200700 1 --2- 172.21.15.18:0/230165166 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2294109dd0 0x7f229411a4c0 unknown :-1 s=CLOSED pgs=969 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:37:51.731 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:51.729+0000 7f2299200700 1 -- 172.21.15.18:0/230165166 >> 172.21.15.18:0/230165166 conn(0x7f2294072c00 msgr2=0x7f2294074c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:37:51.732 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:51.730+0000 7f2299200700 1 -- 172.21.15.18:0/230165166 shutdown_connections 2024-04-12T21:37:51.732 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:51.730+0000 7f2299200700 1 -- 172.21.15.18:0/230165166 wait complete. 2024-04-12T21:37:51.732 INFO:teuthology.orchestra.run.smithi018.stderr:dumped fsmap epoch 31 2024-04-12T21:37:51.887 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:51.888+0000 7fa8d8600700 1 -- 172.21.15.18:0/720090382 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa8d00fc620 msgr2=0x7fa8d0100cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:37:51.888 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:51.888+0000 7fa8d8600700 1 --2- 172.21.15.18:0/720090382 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa8d00fc620 0x7fa8d0100cd0 secure :-1 s=READY pgs=970 cs=0 l=1 rev1=1 crypto rx=0x7fa8bc004940 tx=0x7fa8bc009e30 comp rx=0 tx=0).stop 2024-04-12T21:37:51.888 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:51.888+0000 7fa8d8600700 1 -- 172.21.15.18:0/720090382 shutdown_connections 2024-04-12T21:37:51.888 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:51.888+0000 7fa8d8600700 1 --2- 172.21.15.18:0/720090382 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa8d00fc620 0x7fa8d0100cd0 unknown :-1 s=CLOSED pgs=970 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:37:51.888 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:51.888+0000 7fa8d8600700 1 --2- 172.21.15.18:0/720090382 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa8d00fbc80 0x7fa8d00fc050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:37:51.888 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:51.888+0000 7fa8d8600700 1 -- 172.21.15.18:0/720090382 >> 172.21.15.18:0/720090382 conn(0x7fa8d0072bc0 msgr2=0x7fa8d0072fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:37:51.888 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:51.888+0000 7fa8d8600700 1 -- 172.21.15.18:0/720090382 shutdown_connections 2024-04-12T21:37:51.888 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:51.888+0000 7fa8d8600700 1 -- 172.21.15.18:0/720090382 wait complete. 2024-04-12T21:37:51.888 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:51.889+0000 7fa8d8600700 1 Processor -- start 2024-04-12T21:37:51.889 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:51.889+0000 7fa8d8600700 1 -- start start 2024-04-12T21:37:51.889 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:51.890+0000 7fa8d8600700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa8d00fbc80 0x7fa8d011a1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:37:51.889 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:51.890+0000 7fa8d8600700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa8d00fc620 0x7fa8d011a710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:37:51.889 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:51.890+0000 7fa8d8600700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8d011ad60 con 0x7fa8d00fbc80 2024-04-12T21:37:51.889 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:51.890+0000 7fa8d8600700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8d0114130 con 0x7fa8d00fc620 2024-04-12T21:37:51.890 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:51.890+0000 7fa8d5c00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa8d00fbc80 0x7fa8d011a1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:37:51.891 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:51.890+0000 7fa8d5c00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa8d00fbc80 0x7fa8d011a1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:59380/0 (socket says 172.21.15.18:59380) 2024-04-12T21:37:51.891 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:51.890+0000 7fa8d5c00700 1 -- 172.21.15.18:0/3317006118 learned_addr learned my addr 172.21.15.18:0/3317006118 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:37:51.891 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:51.890+0000 7fa8d5200700 1 --2- 172.21.15.18:0/3317006118 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa8d00fc620 0x7fa8d011a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:37:51.891 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:51.890+0000 7fa8d5c00700 1 -- 172.21.15.18:0/3317006118 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa8d00fc620 msgr2=0x7fa8d011a710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:37:51.891 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:51.891+0000 7fa8d5c00700 1 --2- 172.21.15.18:0/3317006118 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa8d00fc620 0x7fa8d011a710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:37:51.891 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:51.891+0000 7fa8d5c00700 1 -- 172.21.15.18:0/3317006118 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8bc005040 con 0x7fa8d00fbc80 2024-04-12T21:37:51.891 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:51.891+0000 7fa8d5c00700 1 --2- 172.21.15.18:0/3317006118 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa8d00fbc80 0x7fa8d011a1d0 secure :-1 s=READY pgs=971 cs=0 l=1 rev1=1 crypto rx=0x7fa8c400c830 tx=0x7fa8c4004470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:37:51.891 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:51.891+0000 7fa8cd600700 1 -- 172.21.15.18:0/3317006118 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa8c400a760 con 0x7fa8d00fbc80 2024-04-12T21:37:51.891 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:51.891+0000 7fa8cd600700 1 -- 172.21.15.18:0/3317006118 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa8c400ad90 con 0x7fa8d00fbc80 2024-04-12T21:37:51.891 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:51.891+0000 7fa8cd600700 1 -- 172.21.15.18:0/3317006118 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa8c40113a0 con 0x7fa8d00fbc80 2024-04-12T21:37:51.891 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:51.891+0000 7fa8d8600700 1 -- 172.21.15.18:0/3317006118 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa8d0114410 con 0x7fa8d00fbc80 2024-04-12T21:37:51.891 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:51.891+0000 7fa8d8600700 1 -- 172.21.15.18:0/3317006118 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa8d0114950 con 0x7fa8d00fbc80 2024-04-12T21:37:51.892 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:51.892+0000 7fa8d8600700 1 -- 172.21.15.18:0/3317006118 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa8d0065e30 con 0x7fa8d00fbc80 2024-04-12T21:37:51.893 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:51.893+0000 7fa8cd600700 1 -- 172.21.15.18:0/3317006118 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7fa8c4012400 con 0x7fa8d00fbc80 2024-04-12T21:37:51.894 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:51.894+0000 7fa8cd600700 1 --2- 172.21.15.18:0/3317006118 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fa8b806ec10 0x7fa8b80710c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:37:51.894 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:51.894+0000 7fa8cd600700 1 -- 172.21.15.18:0/3317006118 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 6757+0+0 (secure 0 0 0) 0x7fa8c408e9c0 con 0x7fa8d00fbc80 2024-04-12T21:37:51.894 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:51.895+0000 7fa8d5200700 1 --2- 172.21.15.18:0/3317006118 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fa8b806ec10 0x7fa8b80710c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:37:51.895 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:51.895+0000 7fa8d5200700 1 --2- 172.21.15.18:0/3317006118 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fa8b806ec10 0x7fa8b80710c0 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7fa8bc00b3b0 tx=0x7fa8bc009e30 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:37:51.899 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:51.899+0000 7fa8cd600700 1 -- 172.21.15.18:0/3317006118 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7fa8c405b1f0 con 0x7fa8d00fbc80 2024-04-12T21:37:52.104 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:37:51 smithi018 ceph-mon[112590]: from='client.35992 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:37:52.104 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:37:51 smithi018 ceph-mon[112590]: from='client.35996 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:37:52.104 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:37:51 smithi018 ceph-mon[112590]: from='client.? 172.21.15.18:0/537388797' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T21:37:52.104 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:37:51 smithi018 ceph-mon[112590]: from='client.? 172.21.15.18:0/230165166' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-12T21:37:52.153 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:52.153+0000 7fa8d8600700 1 -- 172.21.15.18:0/3317006118 --> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fa8d004bed0 con 0x7fa8b806ec10 2024-04-12T21:37:52.155 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:52.156+0000 7fa8cd600700 1 -- 172.21.15.18:0/3317006118 <== mgr.34102 v2:172.21.15.18:6800/3812891919 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7fa8d004bed0 con 0x7fa8b806ec10 2024-04-12T21:37:52.156 INFO:teuthology.orchestra.run.smithi018.stdout:{ 2024-04-12T21:37:52.156 INFO:teuthology.orchestra.run.smithi018.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:f4452def3be4d350f4a8e5feaf7ebb1511be9298fe68478ba7a2d6fbd7d69a63", 2024-04-12T21:37:52.156 INFO:teuthology.orchestra.run.smithi018.stdout: "in_progress": true, 2024-04-12T21:37:52.156 INFO:teuthology.orchestra.run.smithi018.stdout: "which": "Upgrading all daemon types on all hosts", 2024-04-12T21:37:52.156 INFO:teuthology.orchestra.run.smithi018.stdout: "services_complete": [ 2024-04-12T21:37:52.156 INFO:teuthology.orchestra.run.smithi018.stdout: "crash", 2024-04-12T21:37:52.156 INFO:teuthology.orchestra.run.smithi018.stdout: "osd", 2024-04-12T21:37:52.156 INFO:teuthology.orchestra.run.smithi018.stdout: "mon", 2024-04-12T21:37:52.156 INFO:teuthology.orchestra.run.smithi018.stdout: "mgr" 2024-04-12T21:37:52.156 INFO:teuthology.orchestra.run.smithi018.stdout: ], 2024-04-12T21:37:52.156 INFO:teuthology.orchestra.run.smithi018.stdout: "progress": "15/21 daemons upgraded", 2024-04-12T21:37:52.156 INFO:teuthology.orchestra.run.smithi018.stdout: "message": "Currently upgrading mds daemons", 2024-04-12T21:37:52.157 INFO:teuthology.orchestra.run.smithi018.stdout: "is_paused": false 2024-04-12T21:37:52.157 INFO:teuthology.orchestra.run.smithi018.stdout:} 2024-04-12T21:37:52.158 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:52.159+0000 7fa8d8600700 1 -- 172.21.15.18:0/3317006118 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fa8b806ec10 msgr2=0x7fa8b80710c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:37:52.158 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:52.159+0000 7fa8d8600700 1 --2- 172.21.15.18:0/3317006118 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fa8b806ec10 0x7fa8b80710c0 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7fa8bc00b3b0 tx=0x7fa8bc009e30 comp rx=0 tx=0).stop 2024-04-12T21:37:52.158 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:52.159+0000 7fa8d8600700 1 -- 172.21.15.18:0/3317006118 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa8d00fbc80 msgr2=0x7fa8d011a1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:37:52.158 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:52.159+0000 7fa8d8600700 1 --2- 172.21.15.18:0/3317006118 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa8d00fbc80 0x7fa8d011a1d0 secure :-1 s=READY pgs=971 cs=0 l=1 rev1=1 crypto rx=0x7fa8c400c830 tx=0x7fa8c4004470 comp rx=0 tx=0).stop 2024-04-12T21:37:52.159 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:52.159+0000 7fa8d8600700 1 -- 172.21.15.18:0/3317006118 shutdown_connections 2024-04-12T21:37:52.159 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:52.159+0000 7fa8d8600700 1 --2- 172.21.15.18:0/3317006118 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fa8b806ec10 0x7fa8b80710c0 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:37:52.159 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:52.159+0000 7fa8d8600700 1 --2- 172.21.15.18:0/3317006118 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa8d00fbc80 0x7fa8d011a1d0 unknown :-1 s=CLOSED pgs=971 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:37:52.159 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:52.159+0000 7fa8d8600700 1 --2- 172.21.15.18:0/3317006118 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa8d00fc620 0x7fa8d011a710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:37:52.159 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:52.159+0000 7fa8d8600700 1 -- 172.21.15.18:0/3317006118 >> 172.21.15.18:0/3317006118 conn(0x7fa8d0072bc0 msgr2=0x7fa8d010aab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:37:52.159 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:52.159+0000 7fa8d8600700 1 -- 172.21.15.18:0/3317006118 shutdown_connections 2024-04-12T21:37:52.159 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:52.159+0000 7fa8d8600700 1 -- 172.21.15.18:0/3317006118 wait complete. 2024-04-12T21:37:52.283 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:37:51 smithi186 ceph-mon[86379]: from='client.35992 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:37:52.283 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:37:51 smithi186 ceph-mon[86379]: from='client.35996 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:37:52.283 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:37:51 smithi186 ceph-mon[86379]: from='client.? 172.21.15.18:0/537388797' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T21:37:52.283 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:37:51 smithi186 ceph-mon[86379]: from='client.? 172.21.15.18:0/230165166' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-12T21:37:52.308 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:52.308+0000 7f11fee00700 1 -- 172.21.15.18:0/450242969 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f11f8070360 msgr2=0x7f11f8070730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:37:52.308 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:52.308+0000 7f11fee00700 1 --2- 172.21.15.18:0/450242969 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f11f8070360 0x7f11f8070730 secure :-1 s=READY pgs=972 cs=0 l=1 rev1=1 crypto rx=0x7f11ec009d10 tx=0x7f11ec004d10 comp rx=0 tx=0).stop 2024-04-12T21:37:52.308 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:52.308+0000 7f11fee00700 1 -- 172.21.15.18:0/450242969 shutdown_connections 2024-04-12T21:37:52.308 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:52.308+0000 7f11fee00700 1 --2- 172.21.15.18:0/450242969 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f11f8070c70 0x7f11f8108f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:37:52.308 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:52.308+0000 7f11fee00700 1 --2- 172.21.15.18:0/450242969 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f11f8070360 0x7f11f8070730 unknown :-1 s=CLOSED pgs=972 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:37:52.308 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:52.308+0000 7f11fee00700 1 -- 172.21.15.18:0/450242969 >> 172.21.15.18:0/450242969 conn(0x7f11f80f87e0 msgr2=0x7f11f80fabf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:37:52.308 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:52.308+0000 7f11fee00700 1 -- 172.21.15.18:0/450242969 shutdown_connections 2024-04-12T21:37:52.308 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:52.308+0000 7f11fee00700 1 -- 172.21.15.18:0/450242969 wait complete. 2024-04-12T21:37:52.309 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:52.309+0000 7f11fee00700 1 Processor -- start 2024-04-12T21:37:52.309 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:52.310+0000 7f11fee00700 1 -- start start 2024-04-12T21:37:52.310 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:52.310+0000 7f11fee00700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f11f8070360 0x7f11f8115eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:37:52.310 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:52.310+0000 7f11fee00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f11f8070c70 0x7f11f81163f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:37:52.310 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:52.310+0000 7f11fee00700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11f8116ad0 con 0x7f11f8070c70 2024-04-12T21:37:52.310 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:52.310+0000 7f11fee00700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11f810fe10 con 0x7f11f8070360 2024-04-12T21:37:52.310 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:52.311+0000 7f11fd000700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f11f8070c70 0x7f11f81163f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:37:52.310 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:52.311+0000 7f11fd000700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f11f8070c70 0x7f11f81163f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:59398/0 (socket says 172.21.15.18:59398) 2024-04-12T21:37:52.310 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:52.311+0000 7f11fd000700 1 -- 172.21.15.18:0/2316745834 learned_addr learned my addr 172.21.15.18:0/2316745834 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:37:52.311 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:52.311+0000 7f11fd000700 1 -- 172.21.15.18:0/2316745834 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f11f8070360 msgr2=0x7f11f8115eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:37:52.311 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:52.311+0000 7f11fd000700 1 --2- 172.21.15.18:0/2316745834 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f11f8070360 0x7f11f8115eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:37:52.311 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:52.311+0000 7f11fd000700 1 -- 172.21.15.18:0/2316745834 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11ec009940 con 0x7f11f8070c70 2024-04-12T21:37:52.311 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:52.311+0000 7f11fd000700 1 --2- 172.21.15.18:0/2316745834 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f11f8070c70 0x7f11f81163f0 secure :-1 s=READY pgs=973 cs=0 l=1 rev1=1 crypto rx=0x7f11e400b790 tx=0x7f11e4009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:37:52.312 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:52.312+0000 7f11f5600700 1 -- 172.21.15.18:0/2316745834 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f11e400f770 con 0x7f11f8070c70 2024-04-12T21:37:52.312 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:52.312+0000 7f11f5600700 1 -- 172.21.15.18:0/2316745834 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f11e400fda0 con 0x7f11f8070c70 2024-04-12T21:37:52.312 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:52.312+0000 7f11fee00700 1 -- 172.21.15.18:0/2316745834 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f11f8110090 con 0x7f11f8070c70 2024-04-12T21:37:52.312 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:52.312+0000 7f11f5600700 1 -- 172.21.15.18:0/2316745834 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f11e400d310 con 0x7f11f8070c70 2024-04-12T21:37:52.312 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:52.312+0000 7f11fee00700 1 -- 172.21.15.18:0/2316745834 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f11f81105d0 con 0x7f11f8070c70 2024-04-12T21:37:52.312 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:52.313+0000 7f11fee00700 1 -- 172.21.15.18:0/2316745834 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f11f8071aa0 con 0x7f11f8070c70 2024-04-12T21:37:52.313 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:52.314+0000 7f11f5600700 1 -- 172.21.15.18:0/2316745834 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7f11e4010420 con 0x7f11f8070c70 2024-04-12T21:37:52.316 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:52.316+0000 7f11f5600700 1 --2- 172.21.15.18:0/2316745834 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f11e006ec10 0x7f11e00710c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:37:52.316 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:52.317+0000 7f11f5600700 1 -- 172.21.15.18:0/2316745834 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 6757+0+0 (secure 0 0 0) 0x7f11e408e8e0 con 0x7f11f8070c70 2024-04-12T21:37:52.316 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:52.317+0000 7f11fda00700 1 --2- 172.21.15.18:0/2316745834 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f11e006ec10 0x7f11e00710c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:37:52.317 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:52.317+0000 7f11fda00700 1 --2- 172.21.15.18:0/2316745834 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f11e006ec10 0x7f11e00710c0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f11ec005010 tx=0x7f11ec004d10 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:37:52.320 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:52.321+0000 7f11f5600700 1 -- 172.21.15.18:0/2316745834 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f11e405b1d0 con 0x7f11f8070c70 2024-04-12T21:37:52.632 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:52.632+0000 7f11fee00700 1 -- 172.21.15.18:0/2316745834 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f11f805f5f0 con 0x7f11f8070c70 2024-04-12T21:37:52.632 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:52.633+0000 7f11f5600700 1 -- 172.21.15.18:0/2316745834 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+4303 (secure 0 0 0) 0x7f11e405ad60 con 0x7f11f8070c70 2024-04-12T21:37:52.633 INFO:teuthology.orchestra.run.smithi018.stdout:HEALTH_ERR failed to probe daemons or devices; 1 filesystem is degraded; 1 filesystem is offline 2024-04-12T21:37:52.633 INFO:teuthology.orchestra.run.smithi018.stdout:[WRN] CEPHADM_REFRESH_FAILED: failed to probe daemons or devices 2024-04-12T21:37:52.633 INFO:teuthology.orchestra.run.smithi018.stdout: host smithi018 `cephadm ceph-volume` failed: cephadm exited with an error code: 1, stderr: Inferring config /var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/mon.smithi018/config 2024-04-12T21:37:52.633 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/podman: timeout after 895 seconds 2024-04-12T21:37:52.633 INFO:teuthology.orchestra.run.smithi018.stdout:Non-zero exit code 124 from /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:f4452def3be4d350f4a8e5feaf7ebb1511be9298fe68478ba7a2d6fbd7d69a63 -e NODE_NAME=smithi018 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968:/var/run/ceph:z -v /var/log/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968:/var/log/ceph:z -v /var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmpxsmwkkuh:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:f4452def3be4d350f4a8e5feaf7ebb1511be9298fe68478ba7a2d6fbd7d69a63 inventory --format=json-pretty --filter-for-batch 2024-04-12T21:37:52.633 INFO:teuthology.orchestra.run.smithi018.stdout:Traceback (most recent call last): 2024-04-12T21:37:52.634 INFO:teuthology.orchestra.run.smithi018.stdout: File "/usr/lib64/python3.9/runpy.py", line 197, in _run_module_as_main 2024-04-12T21:37:52.634 INFO:teuthology.orchestra.run.smithi018.stdout: return _run_code(code, main_globals, None, 2024-04-12T21:37:52.634 INFO:teuthology.orchestra.run.smithi018.stdout: File "/usr/lib64/python3.9/runpy.py", line 87, in _run_code 2024-04-12T21:37:52.634 INFO:teuthology.orchestra.run.smithi018.stdout: exec(code, run_globals) 2024-04-12T21:37:52.634 INFO:teuthology.orchestra.run.smithi018.stdout: File "/var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/cephadm.e44f3ed5e34e4ef52eedef233d1190bff004470d8e8005127b5312bff357fa7f/__main__.py", line 5618, in 2024-04-12T21:37:52.634 INFO:teuthology.orchestra.run.smithi018.stdout: File "/var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/cephadm.e44f3ed5e34e4ef52eedef233d1190bff004470d8e8005127b5312bff357fa7f/__main__.py", line 5606, in main 2024-04-12T21:37:52.634 INFO:teuthology.orchestra.run.smithi018.stdout: File "/var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/cephadm.e44f3ed5e34e4ef52eedef233d1190bff004470d8e8005127b5312bff357fa7f/__main__.py", line 408, in _infer_config 2024-04-12T21:37:52.634 INFO:teuthology.orchestra.run.smithi018.stdout: File "/var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/cephadm.e44f3ed5e34e4ef52eedef233d1190bff004470d8e8005127b5312bff357fa7f/__main__.py", line 323, in _infer_fsid 2024-04-12T21:37:52.634 INFO:teuthology.orchestra.run.smithi018.stdout: File "/var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/cephadm.e44f3ed5e34e4ef52eedef233d1190bff004470d8e8005127b5312bff357fa7f/__main__.py", line 436, in _infer_image 2024-04-12T21:37:52.634 INFO:teuthology.orchestra.run.smithi018.stdout: File "/var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/cephadm.e44f3ed5e34e4ef52eedef233d1190bff004470d8e8005127b5312bff357fa7f/__main__.py", line 310, in _validate_fsid 2024-04-12T21:37:52.634 INFO:teuthology.orchestra.run.smithi018.stdout: File "/var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/cephadm.e44f3ed5e34e4ef52eedef233d1190bff004470d8e8005127b5312bff357fa7f/__main__.py", line 3359, in command_ceph_volume 2024-04-12T21:37:52.634 INFO:teuthology.orchestra.run.smithi018.stdout: File "/var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/cephadm.e44f3ed5e34e4ef52eedef233d1190bff004470d8e8005127b5312bff357fa7f/cephadmlib/call_wrappers.py", line 310, in call_throws 2024-04-12T21:37:52.634 INFO:teuthology.orchestra.run.smithi018.stdout:RuntimeError: Failed command: /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:f4452def3be4d350f4a8e5feaf7ebb1511be9298fe68478ba7a2d6fbd7d69a63 -e NODE_NAME=smithi018 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968:/var/run/ceph:z -v /var/log/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968:/var/log/ceph:z -v /var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmpxsmwkkuh:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:f4452def3be4d350f4a8e5feaf7ebb1511be9298fe68478ba7a2d6fbd7d69a63 inventory --format=json-pretty --filter-for-batch 2024-04-12T21:37:52.635 INFO:teuthology.orchestra.run.smithi018.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-04-12T21:37:52.635 INFO:teuthology.orchestra.run.smithi018.stdout: fs cephfs is degraded 2024-04-12T21:37:52.635 INFO:teuthology.orchestra.run.smithi018.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-04-12T21:37:52.635 INFO:teuthology.orchestra.run.smithi018.stdout: fs cephfs is offline because no MDS is active for it. 2024-04-12T21:37:52.636 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:52.636+0000 7f11fee00700 1 -- 172.21.15.18:0/2316745834 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f11e006ec10 msgr2=0x7f11e00710c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:37:52.636 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:52.636+0000 7f11fee00700 1 --2- 172.21.15.18:0/2316745834 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f11e006ec10 0x7f11e00710c0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f11ec005010 tx=0x7f11ec004d10 comp rx=0 tx=0).stop 2024-04-12T21:37:52.636 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:52.636+0000 7f11fee00700 1 -- 172.21.15.18:0/2316745834 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f11f8070c70 msgr2=0x7f11f81163f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:37:52.636 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:52.636+0000 7f11fee00700 1 --2- 172.21.15.18:0/2316745834 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f11f8070c70 0x7f11f81163f0 secure :-1 s=READY pgs=973 cs=0 l=1 rev1=1 crypto rx=0x7f11e400b790 tx=0x7f11e4009250 comp rx=0 tx=0).stop 2024-04-12T21:37:52.636 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:52.636+0000 7f11fee00700 1 -- 172.21.15.18:0/2316745834 shutdown_connections 2024-04-12T21:37:52.636 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:52.636+0000 7f11fee00700 1 --2- 172.21.15.18:0/2316745834 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f11e006ec10 0x7f11e00710c0 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:37:52.636 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:52.636+0000 7f11fee00700 1 --2- 172.21.15.18:0/2316745834 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f11f8070360 0x7f11f8115eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:37:52.636 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:52.636+0000 7f11fee00700 1 --2- 172.21.15.18:0/2316745834 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f11f8070c70 0x7f11f81163f0 unknown :-1 s=CLOSED pgs=973 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:37:52.636 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:52.636+0000 7f11fee00700 1 -- 172.21.15.18:0/2316745834 >> 172.21.15.18:0/2316745834 conn(0x7f11f80f87e0 msgr2=0x7f11f80f9ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:37:52.636 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:52.636+0000 7f11fee00700 1 -- 172.21.15.18:0/2316745834 shutdown_connections 2024-04-12T21:37:52.636 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:37:52.636+0000 7f11fee00700 1 -- 172.21.15.18:0/2316745834 wait complete. 2024-04-12T21:37:53.286 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:37:52 smithi186 ceph-mon[86379]: pgmap v1143: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:37:53.286 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:37:52 smithi186 ceph-mon[86379]: from='client.? 172.21.15.18:0/2316745834' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-12T21:37:53.354 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:37:52 smithi018 ceph-mon[112590]: pgmap v1143: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:37:53.354 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:37:52 smithi018 ceph-mon[112590]: from='client.? 172.21.15.18:0/2316745834' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-12T21:37:54.283 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:37:53 smithi186 ceph-mon[86379]: from='client.36008 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:37:54.354 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:37:53 smithi018 ceph-mon[112590]: from='client.36008 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:37:55.283 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:37:54 smithi186 ceph-mon[86379]: pgmap v1144: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:37:55.283 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:37:54 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T21:37:55.354 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:37:54 smithi018 ceph-mon[112590]: pgmap v1144: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:37:55.354 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:37:54 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T21:37:56.283 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:37:55 smithi186 ceph-mon[86379]: pgmap v1145: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:37:56.354 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:37:55 smithi018 ceph-mon[112590]: pgmap v1145: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:37:59.283 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:37:58 smithi186 ceph-mon[86379]: pgmap v1146: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:37:59.354 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:37:58 smithi018 ceph-mon[112590]: pgmap v1146: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:38:01.283 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:38:00 smithi186 ceph-mon[86379]: pgmap v1147: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:38:01.354 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:38:00 smithi018 ceph-mon[112590]: pgmap v1147: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:38:03.283 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:38:02 smithi186 ceph-mon[86379]: pgmap v1148: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:38:03.283 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:38:02 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-12T21:38:03.283 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:38:02 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-12T21:38:03.283 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:38:02 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' 2024-04-12T21:38:03.283 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:38:02 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-12T21:38:03.283 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:38:02 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T21:38:03.283 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:38:02 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T21:38:03.283 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:38:02 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T21:38:03.284 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:38:02 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T21:38:03.355 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:38:02 smithi018 ceph-mon[112590]: pgmap v1148: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:38:03.355 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:38:02 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-12T21:38:03.355 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:38:02 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-12T21:38:03.355 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:38:02 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' 2024-04-12T21:38:03.355 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:38:02 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-12T21:38:03.356 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:38:02 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T21:38:03.356 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:38:02 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T21:38:03.356 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:38:02 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T21:38:03.356 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:38:02 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T21:38:04.180 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:38:03 smithi186 ceph-mon[86379]: pgmap v1149: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:38:04.180 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:38:03 smithi186 ceph-mon[86379]: pgmap v1150: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:38:04.354 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:38:03 smithi018 ceph-mon[112590]: pgmap v1149: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:38:04.354 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:38:03 smithi018 ceph-mon[112590]: pgmap v1150: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:38:05.905 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:38:05 smithi186 ceph-mon[86379]: pgmap v1151: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:38:05.905 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:38:05 smithi186 ceph-mon[86379]: Upgrade: Updating mds.cephfs.smithi186.wjnafs 2024-04-12T21:38:05.905 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:38:05 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' 2024-04-12T21:38:05.905 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:38:05 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi186.wjnafs", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-12T21:38:05.905 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:38:05 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-12T21:38:05.905 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:38:05 smithi186 ceph-mon[86379]: Deploying daemon mds.cephfs.smithi186.wjnafs on smithi186 2024-04-12T21:38:06.104 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:38:05 smithi018 ceph-mon[112590]: pgmap v1151: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:38:06.104 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:38:05 smithi018 ceph-mon[112590]: Upgrade: Updating mds.cephfs.smithi186.wjnafs 2024-04-12T21:38:06.104 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:38:05 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' 2024-04-12T21:38:06.104 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:38:05 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi186.wjnafs", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-12T21:38:06.105 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:38:05 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-12T21:38:06.105 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:38:05 smithi018 ceph-mon[112590]: Deploying daemon mds.cephfs.smithi186.wjnafs on smithi186 2024-04-12T21:38:08.033 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:38:07 smithi186 ceph-mon[86379]: pgmap v1152: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:38:08.104 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:38:07 smithi018 ceph-mon[112590]: pgmap v1152: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:38:09.033 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:38:08 smithi186 ceph-mon[86379]: osdmap e104: 6 total, 6 up, 6 in 2024-04-12T21:38:09.033 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:38:08 smithi186 ceph-mon[86379]: fsmap cephfs:0/2 3 up:standby, 2 failed 2024-04-12T21:38:09.104 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:38:08 smithi018 ceph-mon[112590]: osdmap e104: 6 total, 6 up, 6 in 2024-04-12T21:38:09.104 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:38:08 smithi018 ceph-mon[112590]: fsmap cephfs:0/2 3 up:standby, 2 failed 2024-04-12T21:38:10.033 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:38:09 smithi186 ceph-mon[86379]: pgmap v1154: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:38:10.104 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:38:09 smithi018 ceph-mon[112590]: pgmap v1154: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:38:11.033 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:38:10 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T21:38:11.104 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:38:10 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T21:38:12.033 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:38:11 smithi186 ceph-mon[86379]: pgmap v1155: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:38:12.104 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:38:11 smithi018 ceph-mon[112590]: pgmap v1155: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:38:13.783 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:38:13 smithi186 ceph-mon[86379]: pgmap v1156: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:38:13.783 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:38:13 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' 2024-04-12T21:38:13.784 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:38:13 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' 2024-04-12T21:38:13.784 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:38:13 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-12T21:38:13.784 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:38:13 smithi186 ceph-mon[86379]: mds.? [v2:172.21.15.186:6826/3122443766,v1:172.21.15.186:6827/3122443766] up:boot 2024-04-12T21:38:13.784 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:38:13 smithi186 ceph-mon[86379]: fsmap cephfs:0/2 4 up:standby, 2 failed 2024-04-12T21:38:13.784 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:38:13 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi186.wjnafs"}]: dispatch 2024-04-12T21:38:13.784 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:38:13 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' 2024-04-12T21:38:13.784 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:38:13 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' 2024-04-12T21:38:13.784 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:38:13 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' 2024-04-12T21:38:13.854 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:38:13 smithi018 ceph-mon[112590]: pgmap v1156: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:38:13.854 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:38:13 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' 2024-04-12T21:38:13.855 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:38:13 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' 2024-04-12T21:38:13.855 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:38:13 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-12T21:38:13.855 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:38:13 smithi018 ceph-mon[112590]: mds.? [v2:172.21.15.186:6826/3122443766,v1:172.21.15.186:6827/3122443766] up:boot 2024-04-12T21:38:13.855 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:38:13 smithi018 ceph-mon[112590]: fsmap cephfs:0/2 4 up:standby, 2 failed 2024-04-12T21:38:13.855 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:38:13 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi186.wjnafs"}]: dispatch 2024-04-12T21:38:13.855 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:38:13 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' 2024-04-12T21:38:13.855 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:38:13 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' 2024-04-12T21:38:13.855 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:38:13 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' 2024-04-12T21:38:14.773 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:38:14 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' 2024-04-12T21:38:14.778 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:38:14 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' 2024-04-12T21:38:15.650 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:38:15 smithi186 ceph-mon[86379]: pgmap v1157: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:38:15.854 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:38:15 smithi018 ceph-mon[112590]: pgmap v1157: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:38:18.104 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:38:17 smithi018 ceph-mon[112590]: pgmap v1158: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:38:18.104 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:38:17 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' 2024-04-12T21:38:18.104 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:38:17 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' 2024-04-12T21:38:18.283 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:38:17 smithi186 ceph-mon[86379]: pgmap v1158: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:38:18.283 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:38:17 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' 2024-04-12T21:38:18.283 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:38:17 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' 2024-04-12T21:38:20.267 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:38:19 smithi186 ceph-mon[86379]: pgmap v1159: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:38:20.267 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:38:19 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' 2024-04-12T21:38:20.268 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:38:19 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' 2024-04-12T21:38:20.268 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:38:19 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' 2024-04-12T21:38:20.268 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:38:19 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' 2024-04-12T21:38:20.354 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:38:19 smithi018 ceph-mon[112590]: pgmap v1159: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:38:20.354 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:38:19 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' 2024-04-12T21:38:20.354 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:38:19 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' 2024-04-12T21:38:20.354 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:38:19 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' 2024-04-12T21:38:20.354 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:38:19 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' 2024-04-12T21:38:22.283 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:38:21 smithi186 ceph-mon[86379]: pgmap v1160: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:38:22.354 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:38:21 smithi018 ceph-mon[112590]: pgmap v1160: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:38:22.811 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:22.811+0000 7f0cb0600700 1 -- 172.21.15.18:0/694264533 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0ca80708a0 msgr2=0x7f0ca8070d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:38:22.811 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:22.811+0000 7f0cb0600700 1 --2- 172.21.15.18:0/694264533 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0ca80708a0 0x7f0ca8070d10 secure :-1 s=READY pgs=974 cs=0 l=1 rev1=1 crypto rx=0x7f0c94004970 tx=0x7f0c94009e30 comp rx=0 tx=0).stop 2024-04-12T21:38:22.811 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:22.811+0000 7f0cb0600700 1 -- 172.21.15.18:0/694264533 shutdown_connections 2024-04-12T21:38:22.812 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:22.811+0000 7f0cb0600700 1 --2- 172.21.15.18:0/694264533 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0ca80708a0 0x7f0ca8070d10 unknown :-1 s=CLOSED pgs=974 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:38:22.812 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:22.811+0000 7f0cb0600700 1 --2- 172.21.15.18:0/694264533 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0ca8071f90 0x7f0ca8070360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:38:22.812 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:22.811+0000 7f0cb0600700 1 -- 172.21.15.18:0/694264533 >> 172.21.15.18:0/694264533 conn(0x7f0ca80f89b0 msgr2=0x7f0ca80fadc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:38:22.812 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:22.812+0000 7f0cb0600700 1 -- 172.21.15.18:0/694264533 shutdown_connections 2024-04-12T21:38:22.812 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:22.812+0000 7f0cb0600700 1 -- 172.21.15.18:0/694264533 wait complete. 2024-04-12T21:38:22.812 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:22.812+0000 7f0cb0600700 1 Processor -- start 2024-04-12T21:38:22.813 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:22.813+0000 7f0cb0600700 1 -- start start 2024-04-12T21:38:22.813 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:22.813+0000 7f0cb0600700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0ca80708a0 0x7f0ca811a170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:38:22.813 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:22.813+0000 7f0cb0600700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0ca8071f90 0x7f0ca811a6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:38:22.813 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:22.813+0000 7f0cb0600700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ca811ad00 con 0x7f0ca80708a0 2024-04-12T21:38:22.813 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:22.813+0000 7f0cb0600700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ca81140d0 con 0x7f0ca8071f90 2024-04-12T21:38:22.814 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:22.813+0000 7f0cadc00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0ca80708a0 0x7f0ca811a170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:38:22.814 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:22.814+0000 7f0cadc00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0ca80708a0 0x7f0ca811a170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:37504/0 (socket says 172.21.15.18:37504) 2024-04-12T21:38:22.814 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:22.814+0000 7f0cadc00700 1 -- 172.21.15.18:0/1218857541 learned_addr learned my addr 172.21.15.18:0/1218857541 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:38:22.814 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:22.814+0000 7f0cad200700 1 --2- 172.21.15.18:0/1218857541 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0ca8071f90 0x7f0ca811a6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:38:22.814 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:22.814+0000 7f0cadc00700 1 -- 172.21.15.18:0/1218857541 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0ca8071f90 msgr2=0x7f0ca811a6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:38:22.814 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:22.814+0000 7f0cadc00700 1 --2- 172.21.15.18:0/1218857541 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0ca8071f90 0x7f0ca811a6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:38:22.814 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:22.814+0000 7f0cadc00700 1 -- 172.21.15.18:0/1218857541 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c94005040 con 0x7f0ca80708a0 2024-04-12T21:38:22.814 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:22.814+0000 7f0cad200700 1 --2- 172.21.15.18:0/1218857541 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0ca8071f90 0x7f0ca811a6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-12T21:38:22.815 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:22.815+0000 7f0cadc00700 1 --2- 172.21.15.18:0/1218857541 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0ca80708a0 0x7f0ca811a170 secure :-1 s=READY pgs=975 cs=0 l=1 rev1=1 crypto rx=0x7f0c9c002a80 tx=0x7f0c9c0046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:38:22.815 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:22.815+0000 7f0ca5600700 1 -- 172.21.15.18:0/1218857541 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0c9c007cc0 con 0x7f0ca80708a0 2024-04-12T21:38:22.815 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:22.815+0000 7f0ca5600700 1 -- 172.21.15.18:0/1218857541 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f0c9c00f040 con 0x7f0ca80708a0 2024-04-12T21:38:22.815 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:22.815+0000 7f0cb0600700 1 -- 172.21.15.18:0/1218857541 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0ca81143b0 con 0x7f0ca80708a0 2024-04-12T21:38:22.815 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:22.815+0000 7f0ca5600700 1 -- 172.21.15.18:0/1218857541 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0c9c0096f0 con 0x7f0ca80708a0 2024-04-12T21:38:22.815 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:22.815+0000 7f0cb0600700 1 -- 172.21.15.18:0/1218857541 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0ca81148f0 con 0x7f0ca80708a0 2024-04-12T21:38:22.816 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:22.816+0000 7f0cb0600700 1 -- 172.21.15.18:0/1218857541 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0ca8071aa0 con 0x7f0ca80708a0 2024-04-12T21:38:22.818 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:22.818+0000 7f0ca5600700 1 -- 172.21.15.18:0/1218857541 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7f0c9c00d690 con 0x7f0ca80708a0 2024-04-12T21:38:22.819 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:22.819+0000 7f0ca5600700 1 --2- 172.21.15.18:0/1218857541 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f0c9006ec10 0x7f0c900710c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:38:22.819 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:22.820+0000 7f0ca5600700 1 -- 172.21.15.18:0/1218857541 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 6757+0+0 (secure 0 0 0) 0x7f0c9c08e110 con 0x7f0ca80708a0 2024-04-12T21:38:22.819 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:22.820+0000 7f0cad200700 1 --2- 172.21.15.18:0/1218857541 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f0c9006ec10 0x7f0c900710c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:38:22.820 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:22.821+0000 7f0cad200700 1 --2- 172.21.15.18:0/1218857541 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f0c9006ec10 0x7f0c900710c0 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f0ca8115a10 tx=0x7f0c94009e30 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:38:22.825 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:22.825+0000 7f0ca5600700 1 -- 172.21.15.18:0/1218857541 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f0c9c05a8c0 con 0x7f0ca80708a0 2024-04-12T21:38:23.083 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:23.083+0000 7f0cb0600700 1 -- 172.21.15.18:0/1218857541 --> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f0ca804bf70 con 0x7f0c9006ec10 2024-04-12T21:38:23.083 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:38:22 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' 2024-04-12T21:38:23.084 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:38:22 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' 2024-04-12T21:38:23.086 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:23.086+0000 7f0ca5600700 1 -- 172.21.15.18:0/1218857541 <== mgr.34102 v2:172.21.15.18:6800/3812891919 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f0c94008d60 con 0x7f0c9006ec10 2024-04-12T21:38:23.089 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:23.089+0000 7f0cb0600700 1 -- 172.21.15.18:0/1218857541 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f0c9006ec10 msgr2=0x7f0c900710c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:38:23.089 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:23.089+0000 7f0cb0600700 1 --2- 172.21.15.18:0/1218857541 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f0c9006ec10 0x7f0c900710c0 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f0ca8115a10 tx=0x7f0c94009e30 comp rx=0 tx=0).stop 2024-04-12T21:38:23.089 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:23.089+0000 7f0cb0600700 1 -- 172.21.15.18:0/1218857541 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0ca80708a0 msgr2=0x7f0ca811a170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:38:23.089 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:23.089+0000 7f0cb0600700 1 --2- 172.21.15.18:0/1218857541 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0ca80708a0 0x7f0ca811a170 secure :-1 s=READY pgs=975 cs=0 l=1 rev1=1 crypto rx=0x7f0c9c002a80 tx=0x7f0c9c0046b0 comp rx=0 tx=0).stop 2024-04-12T21:38:23.090 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:23.089+0000 7f0cb0600700 1 -- 172.21.15.18:0/1218857541 shutdown_connections 2024-04-12T21:38:23.090 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:23.089+0000 7f0cb0600700 1 --2- 172.21.15.18:0/1218857541 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f0c9006ec10 0x7f0c900710c0 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:38:23.090 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:23.089+0000 7f0cb0600700 1 --2- 172.21.15.18:0/1218857541 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0ca80708a0 0x7f0ca811a170 unknown :-1 s=CLOSED pgs=975 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:38:23.090 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:23.089+0000 7f0cb0600700 1 --2- 172.21.15.18:0/1218857541 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0ca8071f90 0x7f0ca811a6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:38:23.090 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:23.089+0000 7f0cb0600700 1 -- 172.21.15.18:0/1218857541 >> 172.21.15.18:0/1218857541 conn(0x7f0ca80f89b0 msgr2=0x7f0ca80f9a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:38:23.090 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:23.090+0000 7f0cb0600700 1 -- 172.21.15.18:0/1218857541 shutdown_connections 2024-04-12T21:38:23.090 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:23.090+0000 7f0cb0600700 1 -- 172.21.15.18:0/1218857541 wait complete. 2024-04-12T21:38:23.105 INFO:teuthology.orchestra.run.smithi018.stdout:true 2024-04-12T21:38:23.186 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:38:22 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' 2024-04-12T21:38:23.186 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:38:22 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' 2024-04-12T21:38:23.258 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:23.258+0000 7f65be800700 1 -- 172.21.15.18:0/3361609351 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f65b80fc4f0 msgr2=0x7f65b8109250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:38:23.258 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:23.258+0000 7f65be800700 1 --2- 172.21.15.18:0/3361609351 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f65b80fc4f0 0x7f65b8109250 secure :-1 s=READY pgs=976 cs=0 l=1 rev1=1 crypto rx=0x7f65a4004940 tx=0x7f65a4009e30 comp rx=0 tx=0).stop 2024-04-12T21:38:23.258 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:23.258+0000 7f65be800700 1 -- 172.21.15.18:0/3361609351 shutdown_connections 2024-04-12T21:38:23.258 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:23.258+0000 7f65be800700 1 --2- 172.21.15.18:0/3361609351 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f65b80fc4f0 0x7f65b8109250 unknown :-1 s=CLOSED pgs=976 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:38:23.258 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:23.258+0000 7f65be800700 1 --2- 172.21.15.18:0/3361609351 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f65b80fbbe0 0x7f65b80fbfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:38:23.259 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:23.258+0000 7f65be800700 1 -- 172.21.15.18:0/3361609351 >> 172.21.15.18:0/3361609351 conn(0x7f65b80f78f0 msgr2=0x7f65b80f9d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:38:23.259 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:23.258+0000 7f65be800700 1 -- 172.21.15.18:0/3361609351 shutdown_connections 2024-04-12T21:38:23.259 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:23.258+0000 7f65be800700 1 -- 172.21.15.18:0/3361609351 wait complete. 2024-04-12T21:38:23.260 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:23.259+0000 7f65be800700 1 Processor -- start 2024-04-12T21:38:23.260 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:23.259+0000 7f65be800700 1 -- start start 2024-04-12T21:38:23.260 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:23.260+0000 7f65be800700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f65b80fbbe0 0x7f65b8115a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:38:23.260 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:23.260+0000 7f65be800700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f65b80fc4f0 0x7f65b8115fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:38:23.260 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:23.260+0000 7f65be800700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65b8116620 con 0x7f65b80fc4f0 2024-04-12T21:38:23.260 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:23.260+0000 7f65be800700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65b810fc00 con 0x7f65b80fbbe0 2024-04-12T21:38:23.260 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:23.260+0000 7f65b6a00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f65b80fc4f0 0x7f65b8115fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:38:23.260 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:23.260+0000 7f65b6a00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f65b80fc4f0 0x7f65b8115fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:37522/0 (socket says 172.21.15.18:37522) 2024-04-12T21:38:23.260 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:23.260+0000 7f65b6a00700 1 -- 172.21.15.18:0/2409902198 learned_addr learned my addr 172.21.15.18:0/2409902198 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:38:23.260 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:23.260+0000 7f65b7400700 1 --2- 172.21.15.18:0/2409902198 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f65b80fbbe0 0x7f65b8115a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:38:23.261 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:23.261+0000 7f65b6a00700 1 -- 172.21.15.18:0/2409902198 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f65b80fbbe0 msgr2=0x7f65b8115a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:38:23.261 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:23.261+0000 7f65b6a00700 1 --2- 172.21.15.18:0/2409902198 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f65b80fbbe0 0x7f65b8115a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:38:23.261 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:23.261+0000 7f65b6a00700 1 -- 172.21.15.18:0/2409902198 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65a4005040 con 0x7f65b80fc4f0 2024-04-12T21:38:23.261 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:23.261+0000 7f65b6a00700 1 --2- 172.21.15.18:0/2409902198 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f65b80fc4f0 0x7f65b8115fd0 secure :-1 s=READY pgs=977 cs=0 l=1 rev1=1 crypto rx=0x7f65a4000c00 tx=0x7f65a4009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:38:23.261 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:23.261+0000 7f65af400700 1 -- 172.21.15.18:0/2409902198 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f65a4021070 con 0x7f65b80fc4f0 2024-04-12T21:38:23.261 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:23.261+0000 7f65af400700 1 -- 172.21.15.18:0/2409902198 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f65a4013c00 con 0x7f65b80fc4f0 2024-04-12T21:38:23.261 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:23.261+0000 7f65af400700 1 -- 172.21.15.18:0/2409902198 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f65a400e070 con 0x7f65b80fc4f0 2024-04-12T21:38:23.262 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:23.261+0000 7f65be800700 1 -- 172.21.15.18:0/2409902198 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f65b810fe80 con 0x7f65b80fc4f0 2024-04-12T21:38:23.262 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:23.262+0000 7f65be800700 1 -- 172.21.15.18:0/2409902198 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f65b8110440 con 0x7f65b80fc4f0 2024-04-12T21:38:23.262 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:23.263+0000 7f65be800700 1 -- 172.21.15.18:0/2409902198 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f65b8106980 con 0x7f65b80fc4f0 2024-04-12T21:38:23.263 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:23.264+0000 7f65af400700 1 -- 172.21.15.18:0/2409902198 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7f65a401c400 con 0x7f65b80fc4f0 2024-04-12T21:38:23.264 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:23.265+0000 7f65af400700 1 --2- 172.21.15.18:0/2409902198 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f659c06ec00 0x7f659c0710b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:38:23.264 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:23.265+0000 7f65af400700 1 -- 172.21.15.18:0/2409902198 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 6757+0+0 (secure 0 0 0) 0x7f65a408fe60 con 0x7f65b80fc4f0 2024-04-12T21:38:23.265 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:23.265+0000 7f65b7400700 1 --2- 172.21.15.18:0/2409902198 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f659c06ec00 0x7f659c0710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:38:23.265 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:23.266+0000 7f65b7400700 1 --2- 172.21.15.18:0/2409902198 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f659c06ec00 0x7f659c0710b0 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f65a8009b80 tx=0x7f65a800b040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:38:23.269 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:23.270+0000 7f65af400700 1 -- 172.21.15.18:0/2409902198 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f65a405c690 con 0x7f65b80fc4f0 2024-04-12T21:38:23.532 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:23.532+0000 7f65be800700 1 -- 172.21.15.18:0/2409902198 --> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f65b8111150 con 0x7f659c06ec00 2024-04-12T21:38:23.533 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:23.534+0000 7f65af400700 1 -- 172.21.15.18:0/2409902198 <== mgr.34102 v2:172.21.15.18:6800/3812891919 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f65b8111150 con 0x7f659c06ec00 2024-04-12T21:38:23.536 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:23.536+0000 7f65be800700 1 -- 172.21.15.18:0/2409902198 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f659c06ec00 msgr2=0x7f659c0710b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:38:23.536 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:23.536+0000 7f65be800700 1 --2- 172.21.15.18:0/2409902198 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f659c06ec00 0x7f659c0710b0 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f65a8009b80 tx=0x7f65a800b040 comp rx=0 tx=0).stop 2024-04-12T21:38:23.536 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:23.536+0000 7f65be800700 1 -- 172.21.15.18:0/2409902198 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f65b80fc4f0 msgr2=0x7f65b8115fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:38:23.536 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:23.536+0000 7f65be800700 1 --2- 172.21.15.18:0/2409902198 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f65b80fc4f0 0x7f65b8115fd0 secure :-1 s=READY pgs=977 cs=0 l=1 rev1=1 crypto rx=0x7f65a4000c00 tx=0x7f65a4009e30 comp rx=0 tx=0).stop 2024-04-12T21:38:23.536 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:23.537+0000 7f65be800700 1 -- 172.21.15.18:0/2409902198 shutdown_connections 2024-04-12T21:38:23.537 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:23.537+0000 7f65be800700 1 --2- 172.21.15.18:0/2409902198 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f659c06ec00 0x7f659c0710b0 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:38:23.537 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:23.537+0000 7f65be800700 1 --2- 172.21.15.18:0/2409902198 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f65b80fbbe0 0x7f65b8115a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:38:23.537 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:23.537+0000 7f65be800700 1 --2- 172.21.15.18:0/2409902198 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f65b80fc4f0 0x7f65b8115fd0 unknown :-1 s=CLOSED pgs=977 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:38:23.537 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:23.537+0000 7f65be800700 1 -- 172.21.15.18:0/2409902198 >> 172.21.15.18:0/2409902198 conn(0x7f65b80f78f0 msgr2=0x7f65b80f8f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:38:23.537 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:23.537+0000 7f65be800700 1 -- 172.21.15.18:0/2409902198 shutdown_connections 2024-04-12T21:38:23.537 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:23.537+0000 7f65be800700 1 -- 172.21.15.18:0/2409902198 wait complete. 2024-04-12T21:38:23.687 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:23.687+0000 7fb119000700 1 -- 172.21.15.18:0/3741711700 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb1140724f0 msgr2=0x7fb1140728c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:38:23.687 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:23.687+0000 7fb119000700 1 --2- 172.21.15.18:0/3741711700 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb1140724f0 0x7fb1140728c0 secure :-1 s=READY pgs=978 cs=0 l=1 rev1=1 crypto rx=0x7fb104004940 tx=0x7fb104009e30 comp rx=0 tx=0).stop 2024-04-12T21:38:23.687 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:23.687+0000 7fb119000700 1 -- 172.21.15.18:0/3741711700 shutdown_connections 2024-04-12T21:38:23.687 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:23.687+0000 7fb119000700 1 --2- 172.21.15.18:0/3741711700 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb114072e00 0x7fb11410d760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:38:23.687 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:23.687+0000 7fb119000700 1 --2- 172.21.15.18:0/3741711700 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb1140724f0 0x7fb1140728c0 unknown :-1 s=CLOSED pgs=978 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:38:23.687 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:23.687+0000 7fb119000700 1 -- 172.21.15.18:0/3741711700 >> 172.21.15.18:0/3741711700 conn(0x7fb1140faba0 msgr2=0x7fb1140fcfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:38:23.687 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:23.687+0000 7fb119000700 1 -- 172.21.15.18:0/3741711700 shutdown_connections 2024-04-12T21:38:23.687 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:23.687+0000 7fb119000700 1 -- 172.21.15.18:0/3741711700 wait complete. 2024-04-12T21:38:23.688 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:23.688+0000 7fb119000700 1 Processor -- start 2024-04-12T21:38:23.688 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:23.688+0000 7fb119000700 1 -- start start 2024-04-12T21:38:23.688 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:23.689+0000 7fb119000700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb1140724f0 0x7fb114119e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:38:23.688 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:23.689+0000 7fb119000700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb114072e00 0x7fb11411a350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:38:23.688 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:23.689+0000 7fb119000700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb11411a9a0 con 0x7fb114072e00 2024-04-12T21:38:23.688 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:23.689+0000 7fb119000700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb114114080 con 0x7fb1140724f0 2024-04-12T21:38:23.690 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:23.689+0000 7fb10be00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb114072e00 0x7fb11411a350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:38:23.690 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:23.689+0000 7fb10be00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb114072e00 0x7fb11411a350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:37548/0 (socket says 172.21.15.18:37548) 2024-04-12T21:38:23.690 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:23.689+0000 7fb10be00700 1 -- 172.21.15.18:0/703874200 learned_addr learned my addr 172.21.15.18:0/703874200 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:38:23.690 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:23.689+0000 7fb111e00700 1 --2- 172.21.15.18:0/703874200 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb1140724f0 0x7fb114119e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:38:23.690 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:23.690+0000 7fb10be00700 1 -- 172.21.15.18:0/703874200 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb1140724f0 msgr2=0x7fb114119e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:38:23.690 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:23.690+0000 7fb10be00700 1 --2- 172.21.15.18:0/703874200 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb1140724f0 0x7fb114119e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:38:23.691 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:23.690+0000 7fb10be00700 1 -- 172.21.15.18:0/703874200 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb104005040 con 0x7fb114072e00 2024-04-12T21:38:23.691 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:23.690+0000 7fb10be00700 1 --2- 172.21.15.18:0/703874200 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb114072e00 0x7fb11411a350 secure :-1 s=READY pgs=979 cs=0 l=1 rev1=1 crypto rx=0x7fb0fc002a50 tx=0x7fb0fc0046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:38:23.691 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:23.690+0000 7fb10a000700 1 -- 172.21.15.18:0/703874200 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb0fc007cc0 con 0x7fb114072e00 2024-04-12T21:38:23.691 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:23.690+0000 7fb10a000700 1 -- 172.21.15.18:0/703874200 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb0fc00f040 con 0x7fb114072e00 2024-04-12T21:38:23.691 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:23.690+0000 7fb119000700 1 -- 172.21.15.18:0/703874200 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb114114360 con 0x7fb114072e00 2024-04-12T21:38:23.691 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:23.690+0000 7fb10a000700 1 -- 172.21.15.18:0/703874200 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb0fc00da30 con 0x7fb114072e00 2024-04-12T21:38:23.691 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:23.690+0000 7fb119000700 1 -- 172.21.15.18:0/703874200 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb114114920 con 0x7fb114072e00 2024-04-12T21:38:23.691 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:23.692+0000 7fb119000700 1 -- 172.21.15.18:0/703874200 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb11410a5a0 con 0x7fb114072e00 2024-04-12T21:38:23.692 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:23.693+0000 7fb10a000700 1 -- 172.21.15.18:0/703874200 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7fb0fc00db90 con 0x7fb114072e00 2024-04-12T21:38:23.693 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:23.694+0000 7fb10a000700 1 --2- 172.21.15.18:0/703874200 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fb0f806ec00 0x7fb0f80710b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:38:23.693 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:23.694+0000 7fb10a000700 1 -- 172.21.15.18:0/703874200 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 6757+0+0 (secure 0 0 0) 0x7fb0fc08e7d0 con 0x7fb114072e00 2024-04-12T21:38:23.693 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:23.694+0000 7fb111e00700 1 --2- 172.21.15.18:0/703874200 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fb0f806ec00 0x7fb0f80710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:38:23.694 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:23.695+0000 7fb111e00700 1 --2- 172.21.15.18:0/703874200 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fb0f806ec00 0x7fb0f80710b0 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7fb104009fd0 tx=0x7fb104009e30 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:38:23.698 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:23.699+0000 7fb10a000700 1 -- 172.21.15.18:0/703874200 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7fb0fc0573d0 con 0x7fb114072e00 2024-04-12T21:38:23.931 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:23.931+0000 7fb119000700 1 -- 172.21.15.18:0/703874200 --> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fb114115510 con 0x7fb0f806ec00 2024-04-12T21:38:23.944 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:23.944+0000 7fb10a000700 1 -- 172.21.15.18:0/703874200 <== mgr.34102 v2:172.21.15.18:6800/3812891919 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7fb114115510 con 0x7fb0f806ec00 2024-04-12T21:38:23.944 INFO:teuthology.orchestra.run.smithi018.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-12T21:38:23.944 INFO:teuthology.orchestra.run.smithi018.stdout:alertmanager.smithi018 smithi018 *:9093,9094 running (39m) 4s ago 53m 14.8M - 0.25.0 c8568f914cd2 c332090068e8 2024-04-12T21:38:23.944 INFO:teuthology.orchestra.run.smithi018.stdout:crash.smithi018 smithi018 running (35m) 4s ago 53m 6665k - 19.0.0-2879-g324baf8d 8717720dc5e5 b7f80d1f7982 2024-04-12T21:38:23.944 INFO:teuthology.orchestra.run.smithi018.stdout:crash.smithi186 smithi186 running (35m) 7s ago 52m 6660k - 19.0.0-2879-g324baf8d 8717720dc5e5 a4f55c81b623 2024-04-12T21:38:23.945 INFO:teuthology.orchestra.run.smithi018.stdout:grafana.smithi018 smithi018 *:3000 running (38m) 4s ago 53m 73.8M - 9.4.12 2bacad6d85d8 b08e3bf176dd 2024-04-12T21:38:23.945 INFO:teuthology.orchestra.run.smithi018.stdout:mds.cephfs.smithi018.acqbsc smithi018 running (31m) 4s ago 46m 21.9M - 19.0.0-2879-g324baf8d 8717720dc5e5 021f99ab24ae 2024-04-12T21:38:23.945 INFO:teuthology.orchestra.run.smithi018.stdout:mds.cephfs.smithi018.szjluo smithi018 running (30m) 4s ago 46m 21.7M - 19.0.0-2879-g324baf8d 8717720dc5e5 0003117479d4 2024-04-12T21:38:23.945 INFO:teuthology.orchestra.run.smithi018.stdout:mds.cephfs.smithi186.jdllyn smithi186 running (15m) 7s ago 46m 16.5M - 19.0.0-2879-g324baf8d 8717720dc5e5 21b1f46eef23 2024-04-12T21:38:23.945 INFO:teuthology.orchestra.run.smithi018.stdout:mds.cephfs.smithi186.wjnafs smithi186 running (12s) 7s ago 46m 11.0M - 19.0.0-2879-g324baf8d 8717720dc5e5 3cf1ef4148e2 2024-04-12T21:38:23.945 INFO:teuthology.orchestra.run.smithi018.stdout:mgr.smithi018.xjzwga smithi018 *:8443,9283,8765 running (41m) 4s ago 55m 517M - 19.0.0-2879-g324baf8d 8717720dc5e5 56088305eab7 2024-04-12T21:38:23.945 INFO:teuthology.orchestra.run.smithi018.stdout:mgr.smithi186.kqcfxk smithi186 *:8443,9283,8765 running (40m) 7s ago 52m 460M - 19.0.0-2879-g324baf8d 8717720dc5e5 09cd766e5e0d 2024-04-12T21:38:23.945 INFO:teuthology.orchestra.run.smithi018.stdout:mon.smithi018 smithi018 running (36m) 4s ago 55m 88.2M 2048M 19.0.0-2879-g324baf8d 8717720dc5e5 df3e201dfed6 2024-04-12T21:38:23.945 INFO:teuthology.orchestra.run.smithi018.stdout:mon.smithi186 smithi186 running (36m) 7s ago 52m 53.6M 2048M 19.0.0-2879-g324baf8d 8717720dc5e5 c77498da9b2b 2024-04-12T21:38:23.945 INFO:teuthology.orchestra.run.smithi018.stdout:node-exporter.smithi018 smithi018 *:9100 running (40m) 4s ago 53m 8468k - 1.5.0 0da6a335fe13 326c3d9c6847 2024-04-12T21:38:23.945 INFO:teuthology.orchestra.run.smithi018.stdout:node-exporter.smithi186 smithi186 *:9100 running (39m) 7s ago 52m 8166k - 1.5.0 0da6a335fe13 ec2112baeecf 2024-04-12T21:38:23.945 INFO:teuthology.orchestra.run.smithi018.stdout:osd.0 smithi018 running (34m) 4s ago 51m 123M 1559M 19.0.0-2879-g324baf8d 8717720dc5e5 60b1273ca047 2024-04-12T21:38:23.945 INFO:teuthology.orchestra.run.smithi018.stdout:osd.1 smithi018 running (34m) 4s ago 50m 95.4M 1559M 19.0.0-2879-g324baf8d 8717720dc5e5 170ce1e5fb8d 2024-04-12T21:38:23.946 INFO:teuthology.orchestra.run.smithi018.stdout:osd.2 smithi018 running (33m) 4s ago 49m 94.6M 1559M 19.0.0-2879-g324baf8d 8717720dc5e5 c2cbc6aa2839 2024-04-12T21:38:23.946 INFO:teuthology.orchestra.run.smithi018.stdout:osd.3 smithi186 running (33m) 7s ago 49m 101M 2581M 19.0.0-2879-g324baf8d 8717720dc5e5 0138e8262e4b 2024-04-12T21:38:23.946 INFO:teuthology.orchestra.run.smithi018.stdout:osd.4 smithi186 running (32m) 7s ago 48m 91.5M 2581M 19.0.0-2879-g324baf8d 8717720dc5e5 b20cb63b25c9 2024-04-12T21:38:23.946 INFO:teuthology.orchestra.run.smithi018.stdout:osd.5 smithi186 running (31m) 7s ago 48m 92.2M 2581M 19.0.0-2879-g324baf8d 8717720dc5e5 2513bea2610e 2024-04-12T21:38:23.946 INFO:teuthology.orchestra.run.smithi018.stdout:prometheus.smithi018 smithi018 *:9095 running (39m) 4s ago 52m 69.6M - 2.43.0 a07b618ecd1d f58841127e21 2024-04-12T21:38:23.947 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:23.947+0000 7fb119000700 1 -- 172.21.15.18:0/703874200 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fb0f806ec00 msgr2=0x7fb0f80710b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:38:23.947 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:23.947+0000 7fb119000700 1 --2- 172.21.15.18:0/703874200 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fb0f806ec00 0x7fb0f80710b0 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7fb104009fd0 tx=0x7fb104009e30 comp rx=0 tx=0).stop 2024-04-12T21:38:23.947 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:23.947+0000 7fb119000700 1 -- 172.21.15.18:0/703874200 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb114072e00 msgr2=0x7fb11411a350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:38:23.947 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:23.947+0000 7fb119000700 1 --2- 172.21.15.18:0/703874200 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb114072e00 0x7fb11411a350 secure :-1 s=READY pgs=979 cs=0 l=1 rev1=1 crypto rx=0x7fb0fc002a50 tx=0x7fb0fc0046b0 comp rx=0 tx=0).stop 2024-04-12T21:38:23.947 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:23.947+0000 7fb119000700 1 -- 172.21.15.18:0/703874200 shutdown_connections 2024-04-12T21:38:23.947 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:23.947+0000 7fb119000700 1 --2- 172.21.15.18:0/703874200 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fb0f806ec00 0x7fb0f80710b0 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:38:23.947 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:23.947+0000 7fb119000700 1 --2- 172.21.15.18:0/703874200 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb1140724f0 0x7fb114119e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:38:23.947 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:23.947+0000 7fb119000700 1 --2- 172.21.15.18:0/703874200 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb114072e00 0x7fb11411a350 unknown :-1 s=CLOSED pgs=979 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:38:23.947 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:23.947+0000 7fb119000700 1 -- 172.21.15.18:0/703874200 >> 172.21.15.18:0/703874200 conn(0x7fb1140faba0 msgr2=0x7fb114107720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:38:23.948 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:23.947+0000 7fb119000700 1 -- 172.21.15.18:0/703874200 shutdown_connections 2024-04-12T21:38:23.948 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:23.948+0000 7fb119000700 1 -- 172.21.15.18:0/703874200 wait complete. 2024-04-12T21:38:24.094 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:24.094+0000 7fdfc0400700 1 -- 172.21.15.18:0/1492804888 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fdfb8103780 msgr2=0x7fdfb80fcdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:38:24.094 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:24.094+0000 7fdfc0400700 1 --2- 172.21.15.18:0/1492804888 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fdfb8103780 0x7fdfb80fcdf0 secure :-1 s=READY pgs=980 cs=0 l=1 rev1=1 crypto rx=0x7fdfa4004940 tx=0x7fdfa4009e30 comp rx=0 tx=0).stop 2024-04-12T21:38:24.094 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:24.094+0000 7fdfc0400700 1 -- 172.21.15.18:0/1492804888 shutdown_connections 2024-04-12T21:38:24.094 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:24.094+0000 7fdfc0400700 1 --2- 172.21.15.18:0/1492804888 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fdfb8103780 0x7fdfb80fcdf0 unknown :-1 s=CLOSED pgs=980 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:38:24.095 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:24.094+0000 7fdfc0400700 1 --2- 172.21.15.18:0/1492804888 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdfb8102de0 0x7fdfb81031b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:38:24.095 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:24.094+0000 7fdfc0400700 1 -- 172.21.15.18:0/1492804888 >> 172.21.15.18:0/1492804888 conn(0x7fdfb80f8970 msgr2=0x7fdfb80fad80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:38:24.095 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:24.095+0000 7fdfc0400700 1 -- 172.21.15.18:0/1492804888 shutdown_connections 2024-04-12T21:38:24.095 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:24.095+0000 7fdfc0400700 1 -- 172.21.15.18:0/1492804888 wait complete. 2024-04-12T21:38:24.096 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:24.095+0000 7fdfc0400700 1 Processor -- start 2024-04-12T21:38:24.096 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:24.095+0000 7fdfc0400700 1 -- start start 2024-04-12T21:38:24.096 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:24.096+0000 7fdfc0400700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdfb8102de0 0x7fdfb8113b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:38:24.096 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:24.096+0000 7fdfc0400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fdfb8103780 0x7fdfb81140a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:38:24.096 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:24.096+0000 7fdfc0400700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdfb81146f0 con 0x7fdfb8103780 2024-04-12T21:38:24.097 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:24.096+0000 7fdfc0400700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdfb810dae0 con 0x7fdfb8102de0 2024-04-12T21:38:24.097 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:24.096+0000 7fdfbd000700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fdfb8103780 0x7fdfb81140a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:38:24.097 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:24.096+0000 7fdfbda00700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdfb8102de0 0x7fdfb8113b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:38:24.097 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:24.096+0000 7fdfbd000700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fdfb8103780 0x7fdfb81140a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:37564/0 (socket says 172.21.15.18:37564) 2024-04-12T21:38:24.097 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:24.096+0000 7fdfbd000700 1 -- 172.21.15.18:0/235381848 learned_addr learned my addr 172.21.15.18:0/235381848 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:38:24.097 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:24.096+0000 7fdfbd000700 1 -- 172.21.15.18:0/235381848 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdfb8102de0 msgr2=0x7fdfb8113b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:38:24.097 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:24.097+0000 7fdfbd000700 1 --2- 172.21.15.18:0/235381848 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdfb8102de0 0x7fdfb8113b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:38:24.097 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:24.097+0000 7fdfbd000700 1 -- 172.21.15.18:0/235381848 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdfa4005040 con 0x7fdfb8103780 2024-04-12T21:38:24.097 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:24.097+0000 7fdfbd000700 1 --2- 172.21.15.18:0/235381848 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fdfb8103780 0x7fdfb81140a0 secure :-1 s=READY pgs=981 cs=0 l=1 rev1=1 crypto rx=0x7fdfa4009100 tx=0x7fdfa4009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:38:24.098 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:24.097+0000 7fdfb5600700 1 -- 172.21.15.18:0/235381848 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdfa400e040 con 0x7fdfb8103780 2024-04-12T21:38:24.098 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:24.097+0000 7fdfb5600700 1 -- 172.21.15.18:0/235381848 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fdfa4004290 con 0x7fdfb8103780 2024-04-12T21:38:24.098 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:24.097+0000 7fdfb5600700 1 -- 172.21.15.18:0/235381848 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdfa401c3f0 con 0x7fdfb8103780 2024-04-12T21:38:24.098 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:24.097+0000 7fdfc0400700 1 -- 172.21.15.18:0/235381848 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdfb810dd60 con 0x7fdfb8103780 2024-04-12T21:38:24.099 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:24.097+0000 7fdfc0400700 1 -- 172.21.15.18:0/235381848 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdfb810e2d0 con 0x7fdfb8103780 2024-04-12T21:38:24.099 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:24.098+0000 7fdfc0400700 1 -- 172.21.15.18:0/235381848 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdfb80fe530 con 0x7fdfb8103780 2024-04-12T21:38:24.099 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:24.100+0000 7fdfb5600700 1 -- 172.21.15.18:0/235381848 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7fdfa4013690 con 0x7fdfb8103780 2024-04-12T21:38:24.100 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:24.101+0000 7fdfb5600700 1 --2- 172.21.15.18:0/235381848 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fdfa006ec10 0x7fdfa00710c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:38:24.101 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:24.101+0000 7fdfb5600700 1 -- 172.21.15.18:0/235381848 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 6757+0+0 (secure 0 0 0) 0x7fdfa408fbb0 con 0x7fdfb8103780 2024-04-12T21:38:24.101 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:24.101+0000 7fdfbda00700 1 --2- 172.21.15.18:0/235381848 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fdfa006ec10 0x7fdfa00710c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:38:24.101 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:24.102+0000 7fdfbda00700 1 --2- 172.21.15.18:0/235381848 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fdfa006ec10 0x7fdfa00710c0 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7fdfac004f40 tx=0x7fdfac00b040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:38:24.105 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:24.106+0000 7fdfb5600700 1 -- 172.21.15.18:0/235381848 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7fdfa405c360 con 0x7fdfb8103780 2024-04-12T21:38:24.283 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:38:23 smithi186 ceph-mon[86379]: pgmap v1161: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:38:24.354 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:38:23 smithi018 ceph-mon[112590]: pgmap v1161: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:38:24.426 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:24.427+0000 7fdfc0400700 1 -- 172.21.15.18:0/235381848 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fdfb8109ec0 con 0x7fdfb8103780 2024-04-12T21:38:24.427 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:24.428+0000 7fdfb5600700 1 -- 172.21.15.18:0/235381848 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+623 (secure 0 0 0) 0x7fdfa405bef0 con 0x7fdfb8103780 2024-04-12T21:38:24.427 INFO:teuthology.orchestra.run.smithi018.stdout:{ 2024-04-12T21:38:24.428 INFO:teuthology.orchestra.run.smithi018.stdout: "mon": { 2024-04-12T21:38:24.428 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 19.0.0-2879-g324baf8d (324baf8ddb91dea46cac50cb535c306115791355) squid (dev)": 2 2024-04-12T21:38:24.428 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-12T21:38:24.428 INFO:teuthology.orchestra.run.smithi018.stdout: "mgr": { 2024-04-12T21:38:24.428 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 19.0.0-2879-g324baf8d (324baf8ddb91dea46cac50cb535c306115791355) squid (dev)": 2 2024-04-12T21:38:24.428 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-12T21:38:24.428 INFO:teuthology.orchestra.run.smithi018.stdout: "osd": { 2024-04-12T21:38:24.428 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 19.0.0-2879-g324baf8d (324baf8ddb91dea46cac50cb535c306115791355) squid (dev)": 6 2024-04-12T21:38:24.428 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-12T21:38:24.428 INFO:teuthology.orchestra.run.smithi018.stdout: "mds": { 2024-04-12T21:38:24.428 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 19.0.0-2879-g324baf8d (324baf8ddb91dea46cac50cb535c306115791355) squid (dev)": 4 2024-04-12T21:38:24.428 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-12T21:38:24.428 INFO:teuthology.orchestra.run.smithi018.stdout: "overall": { 2024-04-12T21:38:24.429 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 19.0.0-2879-g324baf8d (324baf8ddb91dea46cac50cb535c306115791355) squid (dev)": 14 2024-04-12T21:38:24.429 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-04-12T21:38:24.429 INFO:teuthology.orchestra.run.smithi018.stdout:} 2024-04-12T21:38:24.430 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:24.430+0000 7fdfc0400700 1 -- 172.21.15.18:0/235381848 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fdfa006ec10 msgr2=0x7fdfa00710c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:38:24.430 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:24.430+0000 7fdfc0400700 1 --2- 172.21.15.18:0/235381848 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fdfa006ec10 0x7fdfa00710c0 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7fdfac004f40 tx=0x7fdfac00b040 comp rx=0 tx=0).stop 2024-04-12T21:38:24.430 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:24.430+0000 7fdfc0400700 1 -- 172.21.15.18:0/235381848 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fdfb8103780 msgr2=0x7fdfb81140a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:38:24.430 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:24.430+0000 7fdfc0400700 1 --2- 172.21.15.18:0/235381848 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fdfb8103780 0x7fdfb81140a0 secure :-1 s=READY pgs=981 cs=0 l=1 rev1=1 crypto rx=0x7fdfa4009100 tx=0x7fdfa4009e30 comp rx=0 tx=0).stop 2024-04-12T21:38:24.430 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:24.430+0000 7fdfc0400700 1 -- 172.21.15.18:0/235381848 shutdown_connections 2024-04-12T21:38:24.431 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:24.430+0000 7fdfc0400700 1 --2- 172.21.15.18:0/235381848 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fdfa006ec10 0x7fdfa00710c0 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:38:24.431 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:24.430+0000 7fdfc0400700 1 --2- 172.21.15.18:0/235381848 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdfb8102de0 0x7fdfb8113b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:38:24.431 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:24.430+0000 7fdfc0400700 1 --2- 172.21.15.18:0/235381848 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fdfb8103780 0x7fdfb81140a0 unknown :-1 s=CLOSED pgs=981 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:38:24.431 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:24.430+0000 7fdfc0400700 1 -- 172.21.15.18:0/235381848 >> 172.21.15.18:0/235381848 conn(0x7fdfb80f8970 msgr2=0x7fdfb80f9970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:38:24.431 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:24.431+0000 7fdfc0400700 1 -- 172.21.15.18:0/235381848 shutdown_connections 2024-04-12T21:38:24.431 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:24.431+0000 7fdfc0400700 1 -- 172.21.15.18:0/235381848 wait complete. 2024-04-12T21:38:24.582 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:24.582+0000 7f371f800700 1 -- 172.21.15.18:0/1570077092 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f37180fde20 msgr2=0x7f37180fe1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:38:24.582 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:24.582+0000 7f371f800700 1 --2- 172.21.15.18:0/1570077092 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f37180fde20 0x7f37180fe1f0 secure :-1 s=READY pgs=982 cs=0 l=1 rev1=1 crypto rx=0x7f370c004940 tx=0x7f370c009e30 comp rx=0 tx=0).stop 2024-04-12T21:38:24.583 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:24.582+0000 7f371f800700 1 -- 172.21.15.18:0/1570077092 shutdown_connections 2024-04-12T21:38:24.583 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:24.582+0000 7f371f800700 1 --2- 172.21.15.18:0/1570077092 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f37180fe730 0x7f3718106e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:38:24.583 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:24.582+0000 7f371f800700 1 --2- 172.21.15.18:0/1570077092 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f37180fde20 0x7f37180fe1f0 unknown :-1 s=CLOSED pgs=982 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:38:24.583 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:24.582+0000 7f371f800700 1 -- 172.21.15.18:0/1570077092 >> 172.21.15.18:0/1570077092 conn(0x7f37180f7860 msgr2=0x7f37180f9c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:38:24.583 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:24.583+0000 7f371f800700 1 -- 172.21.15.18:0/1570077092 shutdown_connections 2024-04-12T21:38:24.583 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:24.583+0000 7f371f800700 1 -- 172.21.15.18:0/1570077092 wait complete. 2024-04-12T21:38:24.584 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:24.584+0000 7f371f800700 1 Processor -- start 2024-04-12T21:38:24.584 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:24.584+0000 7f371f800700 1 -- start start 2024-04-12T21:38:24.584 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:24.584+0000 7f371f800700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f37180fde20 0x7f3718115dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:38:24.584 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:24.584+0000 7f371f800700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f37180fe730 0x7f3718116300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:38:24.584 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:24.584+0000 7f371f800700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37181169e0 con 0x7f37180fde20 2024-04-12T21:38:24.584 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:24.584+0000 7f371f800700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f371810fd20 con 0x7f37180fe730 2024-04-12T21:38:24.584 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:24.584+0000 7f371ce00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f37180fde20 0x7f3718115dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:38:24.584 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:24.585+0000 7f371ce00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f37180fde20 0x7f3718115dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:37582/0 (socket says 172.21.15.18:37582) 2024-04-12T21:38:24.584 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:24.585+0000 7f371ce00700 1 -- 172.21.15.18:0/3564913670 learned_addr learned my addr 172.21.15.18:0/3564913670 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:38:24.585 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:24.585+0000 7f3717e00700 1 --2- 172.21.15.18:0/3564913670 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f37180fe730 0x7f3718116300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:38:24.586 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:24.585+0000 7f371ce00700 1 -- 172.21.15.18:0/3564913670 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f37180fe730 msgr2=0x7f3718116300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:38:24.586 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:24.585+0000 7f371ce00700 1 --2- 172.21.15.18:0/3564913670 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f37180fe730 0x7f3718116300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:38:24.586 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:24.585+0000 7f371ce00700 1 -- 172.21.15.18:0/3564913670 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f370c005040 con 0x7f37180fde20 2024-04-12T21:38:24.586 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:24.585+0000 7f3717e00700 1 --2- 172.21.15.18:0/3564913670 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f37180fe730 0x7f3718116300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-12T21:38:24.586 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:24.585+0000 7f371ce00700 1 --2- 172.21.15.18:0/3564913670 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f37180fde20 0x7f3718115dc0 secure :-1 s=READY pgs=983 cs=0 l=1 rev1=1 crypto rx=0x7f370c009150 tx=0x7f370c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:38:24.586 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:24.586+0000 7f3714c00700 1 -- 172.21.15.18:0/3564913670 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f370c00e040 con 0x7f37180fde20 2024-04-12T21:38:24.586 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:24.586+0000 7f371f800700 1 -- 172.21.15.18:0/3564913670 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f371810ff40 con 0x7f37180fde20 2024-04-12T21:38:24.586 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:24.586+0000 7f3714c00700 1 -- 172.21.15.18:0/3564913670 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f370c013420 con 0x7f37180fde20 2024-04-12T21:38:24.586 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:24.586+0000 7f3714c00700 1 -- 172.21.15.18:0/3564913670 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f370c01c630 con 0x7f37180fde20 2024-04-12T21:38:24.586 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:24.586+0000 7f371f800700 1 -- 172.21.15.18:0/3564913670 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3718110480 con 0x7f37180fde20 2024-04-12T21:38:24.587 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:24.587+0000 7f371f800700 1 -- 172.21.15.18:0/3564913670 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f37180666b0 con 0x7f37180fde20 2024-04-12T21:38:24.589 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:24.590+0000 7f3714c00700 1 -- 172.21.15.18:0/3564913670 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7f370c013590 con 0x7f37180fde20 2024-04-12T21:38:24.590 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:24.590+0000 7f3714c00700 1 --2- 172.21.15.18:0/3564913670 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f370006ec10 0x7f37000710c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:38:24.590 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:24.591+0000 7f3714c00700 1 -- 172.21.15.18:0/3564913670 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 6757+0+0 (secure 0 0 0) 0x7f370c08fce0 con 0x7f37180fde20 2024-04-12T21:38:24.590 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:24.591+0000 7f3717e00700 1 --2- 172.21.15.18:0/3564913670 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f370006ec10 0x7f37000710c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:38:24.591 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:24.592+0000 7f3717e00700 1 --2- 172.21.15.18:0/3564913670 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f370006ec10 0x7f37000710c0 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f3704006fd0 tx=0x7f3704008040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:38:24.595 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:24.595+0000 7f3714c00700 1 -- 172.21.15.18:0/3564913670 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f370c05c490 con 0x7f37180fde20 2024-04-12T21:38:24.879 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:24.879+0000 7f371f800700 1 -- 172.21.15.18:0/3564913670 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f371804bf00 con 0x7f37180fde20 2024-04-12T21:38:24.879 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:24.880+0000 7f3714c00700 1 -- 172.21.15.18:0/3564913670 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 33 v33) v1 ==== 76+0+1876 (secure 0 0 0) 0x7f370c05c020 con 0x7f37180fde20 2024-04-12T21:38:24.880 INFO:teuthology.orchestra.run.smithi018.stdout:e33 2024-04-12T21:38:24.880 INFO:teuthology.orchestra.run.smithi018.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-12T21:38:24.880 INFO:teuthology.orchestra.run.smithi018.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-12T21:38:24.880 INFO:teuthology.orchestra.run.smithi018.stdout:legacy client fscid: 1 2024-04-12T21:38:24.880 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-12T21:38:24.880 INFO:teuthology.orchestra.run.smithi018.stdout:Filesystem 'cephfs' (1) 2024-04-12T21:38:24.880 INFO:teuthology.orchestra.run.smithi018.stdout:fs_name cephfs 2024-04-12T21:38:24.880 INFO:teuthology.orchestra.run.smithi018.stdout:epoch 24 2024-04-12T21:38:24.880 INFO:teuthology.orchestra.run.smithi018.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-04-12T21:38:24.880 INFO:teuthology.orchestra.run.smithi018.stdout:created 2024-04-12T20:51:57.362850+0000 2024-04-12T21:38:24.880 INFO:teuthology.orchestra.run.smithi018.stdout:modified 2024-04-12T21:06:59.717190+0000 2024-04-12T21:38:24.880 INFO:teuthology.orchestra.run.smithi018.stdout:tableserver 0 2024-04-12T21:38:24.881 INFO:teuthology.orchestra.run.smithi018.stdout:root 0 2024-04-12T21:38:24.881 INFO:teuthology.orchestra.run.smithi018.stdout:session_timeout 60 2024-04-12T21:38:24.881 INFO:teuthology.orchestra.run.smithi018.stdout:session_autoclose 300 2024-04-12T21:38:24.881 INFO:teuthology.orchestra.run.smithi018.stdout:max_file_size 1099511627776 2024-04-12T21:38:24.881 INFO:teuthology.orchestra.run.smithi018.stdout:max_xattr_size 65536 2024-04-12T21:38:24.881 INFO:teuthology.orchestra.run.smithi018.stdout:required_client_features {} 2024-04-12T21:38:24.881 INFO:teuthology.orchestra.run.smithi018.stdout:last_failure 0 2024-04-12T21:38:24.881 INFO:teuthology.orchestra.run.smithi018.stdout:last_failure_osd_epoch 100 2024-04-12T21:38:24.881 INFO:teuthology.orchestra.run.smithi018.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-12T21:38:24.881 INFO:teuthology.orchestra.run.smithi018.stdout:max_mds 2 2024-04-12T21:38:24.881 INFO:teuthology.orchestra.run.smithi018.stdout:in 0,1 2024-04-12T21:38:24.881 INFO:teuthology.orchestra.run.smithi018.stdout:up {} 2024-04-12T21:38:24.881 INFO:teuthology.orchestra.run.smithi018.stdout:failed 0,1 2024-04-12T21:38:24.881 INFO:teuthology.orchestra.run.smithi018.stdout:damaged 2024-04-12T21:38:24.881 INFO:teuthology.orchestra.run.smithi018.stdout:stopped 2024-04-12T21:38:24.881 INFO:teuthology.orchestra.run.smithi018.stdout:data_pools [3] 2024-04-12T21:38:24.882 INFO:teuthology.orchestra.run.smithi018.stdout:metadata_pool 2 2024-04-12T21:38:24.882 INFO:teuthology.orchestra.run.smithi018.stdout:inline_data disabled 2024-04-12T21:38:24.882 INFO:teuthology.orchestra.run.smithi018.stdout:balancer 2024-04-12T21:38:24.882 INFO:teuthology.orchestra.run.smithi018.stdout:bal_rank_mask -1 2024-04-12T21:38:24.882 INFO:teuthology.orchestra.run.smithi018.stdout:standby_count_wanted 1 2024-04-12T21:38:24.882 INFO:teuthology.orchestra.run.smithi018.stdout:qdb_cluster leader: 0 members: 2024-04-12T21:38:24.882 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-12T21:38:24.882 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-12T21:38:24.882 INFO:teuthology.orchestra.run.smithi018.stdout:Standby daemons: 2024-04-12T21:38:24.882 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-12T21:38:24.882 INFO:teuthology.orchestra.run.smithi018.stdout:[mds.cephfs.smithi018.acqbsc{-1:34436} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.18:6828/755179764,v1:172.21.15.18:6829/755179764] compat {c=[1],r=[1],i=[1fff]}] 2024-04-12T21:38:24.882 INFO:teuthology.orchestra.run.smithi018.stdout:[mds.cephfs.smithi018.szjluo{-1:34468} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.18:6826/2407992653,v1:172.21.15.18:6827/2407992653] compat {c=[1],r=[1],i=[1fff]}] 2024-04-12T21:38:24.882 INFO:teuthology.orchestra.run.smithi018.stdout:[mds.cephfs.smithi186.jdllyn{-1:44595} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.186:6824/3523444202,v1:172.21.15.186:6825/3523444202] compat {c=[1],r=[1],i=[1fff]}] 2024-04-12T21:38:24.882 INFO:teuthology.orchestra.run.smithi018.stdout:[mds.cephfs.smithi186.wjnafs{-1:44875} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.186:6826/3122443766,v1:172.21.15.186:6827/3122443766] compat {c=[1],r=[1],i=[1fff]}] 2024-04-12T21:38:24.883 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:24.882+0000 7f371f800700 1 -- 172.21.15.18:0/3564913670 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f370006ec10 msgr2=0x7f37000710c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:38:24.883 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:24.882+0000 7f371f800700 1 --2- 172.21.15.18:0/3564913670 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f370006ec10 0x7f37000710c0 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f3704006fd0 tx=0x7f3704008040 comp rx=0 tx=0).stop 2024-04-12T21:38:24.883 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:24.883+0000 7f371f800700 1 -- 172.21.15.18:0/3564913670 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f37180fde20 msgr2=0x7f3718115dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:38:24.883 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:24.883+0000 7f371f800700 1 --2- 172.21.15.18:0/3564913670 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f37180fde20 0x7f3718115dc0 secure :-1 s=READY pgs=983 cs=0 l=1 rev1=1 crypto rx=0x7f370c009150 tx=0x7f370c009e30 comp rx=0 tx=0).stop 2024-04-12T21:38:24.883 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:24.883+0000 7f371f800700 1 -- 172.21.15.18:0/3564913670 shutdown_connections 2024-04-12T21:38:24.884 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:24.883+0000 7f371f800700 1 --2- 172.21.15.18:0/3564913670 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f370006ec10 0x7f37000710c0 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:38:24.884 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:24.883+0000 7f371f800700 1 --2- 172.21.15.18:0/3564913670 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f37180fde20 0x7f3718115dc0 unknown :-1 s=CLOSED pgs=983 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:38:24.884 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:24.883+0000 7f371f800700 1 --2- 172.21.15.18:0/3564913670 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f37180fe730 0x7f3718116300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:38:24.884 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:24.883+0000 7f371f800700 1 -- 172.21.15.18:0/3564913670 >> 172.21.15.18:0/3564913670 conn(0x7f37180f7860 msgr2=0x7f37180f8cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:38:24.884 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:24.883+0000 7f371f800700 1 -- 172.21.15.18:0/3564913670 shutdown_connections 2024-04-12T21:38:24.884 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:24.883+0000 7f371f800700 1 -- 172.21.15.18:0/3564913670 wait complete. 2024-04-12T21:38:24.884 INFO:teuthology.orchestra.run.smithi018.stderr:dumped fsmap epoch 33 2024-04-12T21:38:24.988 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:38:24 smithi018 ceph-mon[112590]: from='client.36020 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:38:24.988 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:38:24 smithi018 ceph-mon[112590]: from='client.36024 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:38:24.988 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:38:24 smithi018 ceph-mon[112590]: from='client.36028 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:38:24.988 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:38:24 smithi018 ceph-mon[112590]: from='client.? 172.21.15.18:0/235381848' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T21:38:24.988 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:38:24 smithi018 ceph-mon[112590]: from='client.? 172.21.15.18:0/3564913670' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-12T21:38:24.988 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:38:24 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T21:38:25.035 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:25.035+0000 7f1616600700 1 -- 172.21.15.18:0/3667040413 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f16100fbc00 msgr2=0x7f16100fbfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:38:25.035 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:25.035+0000 7f1616600700 1 --2- 172.21.15.18:0/3667040413 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f16100fbc00 0x7f16100fbfd0 secure :-1 s=READY pgs=984 cs=0 l=1 rev1=1 crypto rx=0x7f1600004940 tx=0x7f1600009e30 comp rx=0 tx=0).stop 2024-04-12T21:38:25.035 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:25.035+0000 7f1616600700 1 -- 172.21.15.18:0/3667040413 shutdown_connections 2024-04-12T21:38:25.035 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:25.035+0000 7f1616600700 1 --2- 172.21.15.18:0/3667040413 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f16100fc5a0 0x7f1610100bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:38:25.036 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:25.035+0000 7f1616600700 1 --2- 172.21.15.18:0/3667040413 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f16100fbc00 0x7f16100fbfd0 unknown :-1 s=CLOSED pgs=984 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:38:25.036 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:25.035+0000 7f1616600700 1 -- 172.21.15.18:0/3667040413 >> 172.21.15.18:0/3667040413 conn(0x7f16100f7880 msgr2=0x7f16100f9c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:38:25.036 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:25.035+0000 7f1616600700 1 -- 172.21.15.18:0/3667040413 shutdown_connections 2024-04-12T21:38:25.036 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:25.035+0000 7f1616600700 1 -- 172.21.15.18:0/3667040413 wait complete. 2024-04-12T21:38:25.036 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:25.036+0000 7f1616600700 1 Processor -- start 2024-04-12T21:38:25.036 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:25.036+0000 7f1616600700 1 -- start start 2024-04-12T21:38:25.037 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:25.036+0000 7f1616600700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f16100fbc00 0x7f161011a520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:38:25.037 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:25.036+0000 7f1616600700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f16100fc5a0 0x7f161011aa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:38:25.037 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:25.036+0000 7f1616600700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f161011b0b0 con 0x7f16100fc5a0 2024-04-12T21:38:25.037 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:25.036+0000 7f1616600700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16101144a0 con 0x7f16100fbc00 2024-04-12T21:38:25.037 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:25.037+0000 7f160ea00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f16100fc5a0 0x7f161011aa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:38:25.037 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:25.037+0000 7f160ea00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f16100fc5a0 0x7f161011aa60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:37600/0 (socket says 172.21.15.18:37600) 2024-04-12T21:38:25.037 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:25.037+0000 7f160ea00700 1 -- 172.21.15.18:0/1454678714 learned_addr learned my addr 172.21.15.18:0/1454678714 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:38:25.037 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:25.037+0000 7f160f400700 1 --2- 172.21.15.18:0/1454678714 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f16100fbc00 0x7f161011a520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:38:25.037 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:25.037+0000 7f160ea00700 1 -- 172.21.15.18:0/1454678714 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f16100fbc00 msgr2=0x7f161011a520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:38:25.037 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:25.037+0000 7f160ea00700 1 --2- 172.21.15.18:0/1454678714 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f16100fbc00 0x7f161011a520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:38:25.037 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:25.037+0000 7f160ea00700 1 -- 172.21.15.18:0/1454678714 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1600005040 con 0x7f16100fc5a0 2024-04-12T21:38:25.038 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:25.038+0000 7f160ea00700 1 --2- 172.21.15.18:0/1454678714 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f16100fc5a0 0x7f161011aa60 secure :-1 s=READY pgs=985 cs=0 l=1 rev1=1 crypto rx=0x7f15fc00c830 tx=0x7f15fc004470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:38:25.038 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:25.038+0000 7f1607400700 1 -- 172.21.15.18:0/1454678714 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f15fc00a760 con 0x7f16100fc5a0 2024-04-12T21:38:25.038 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:25.038+0000 7f1607400700 1 -- 172.21.15.18:0/1454678714 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f15fc00ad90 con 0x7f16100fc5a0 2024-04-12T21:38:25.038 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:25.038+0000 7f1616600700 1 -- 172.21.15.18:0/1454678714 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1610114780 con 0x7f16100fc5a0 2024-04-12T21:38:25.038 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:25.038+0000 7f1607400700 1 -- 172.21.15.18:0/1454678714 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f15fc0113a0 con 0x7f16100fc5a0 2024-04-12T21:38:25.038 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:25.038+0000 7f1616600700 1 -- 172.21.15.18:0/1454678714 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1610114cc0 con 0x7f16100fc5a0 2024-04-12T21:38:25.039 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:25.039+0000 7f1616600700 1 -- 172.21.15.18:0/1454678714 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f16100fe2e0 con 0x7f16100fc5a0 2024-04-12T21:38:25.039 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:25.040+0000 7f1607400700 1 -- 172.21.15.18:0/1454678714 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7f15fc012400 con 0x7f16100fc5a0 2024-04-12T21:38:25.042 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:25.042+0000 7f1607400700 1 --2- 172.21.15.18:0/1454678714 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f15f406ec10 0x7f15f40710c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:38:25.042 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:25.042+0000 7f1607400700 1 -- 172.21.15.18:0/1454678714 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 6757+0+0 (secure 0 0 0) 0x7f15fc08e9c0 con 0x7f16100fc5a0 2024-04-12T21:38:25.042 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:25.042+0000 7f160f400700 1 --2- 172.21.15.18:0/1454678714 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f15f406ec10 0x7f15f40710c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:38:25.042 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:25.043+0000 7f160f400700 1 --2- 172.21.15.18:0/1454678714 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f15f406ec10 0x7f15f40710c0 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f160000b020 tx=0x7f1600009e30 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:38:25.046 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:25.047+0000 7f1607400700 1 -- 172.21.15.18:0/1454678714 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f15fc05b1f0 con 0x7f16100fc5a0 2024-04-12T21:38:25.283 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:38:24 smithi186 ceph-mon[86379]: from='client.36020 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:38:25.283 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:38:24 smithi186 ceph-mon[86379]: from='client.36024 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:38:25.283 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:38:24 smithi186 ceph-mon[86379]: from='client.36028 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:38:25.283 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:38:24 smithi186 ceph-mon[86379]: from='client.? 172.21.15.18:0/235381848' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T21:38:25.283 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:38:24 smithi186 ceph-mon[86379]: from='client.? 172.21.15.18:0/3564913670' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-12T21:38:25.283 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:38:24 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T21:38:25.294 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:25.295+0000 7f1616600700 1 -- 172.21.15.18:0/1454678714 --> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f1610115ba0 con 0x7f15f406ec10 2024-04-12T21:38:25.297 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:25.297+0000 7f1607400700 1 -- 172.21.15.18:0/1454678714 <== mgr.34102 v2:172.21.15.18:6800/3812891919 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f1610115ba0 con 0x7f15f406ec10 2024-04-12T21:38:25.297 INFO:teuthology.orchestra.run.smithi018.stdout:{ 2024-04-12T21:38:25.297 INFO:teuthology.orchestra.run.smithi018.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:f4452def3be4d350f4a8e5feaf7ebb1511be9298fe68478ba7a2d6fbd7d69a63", 2024-04-12T21:38:25.297 INFO:teuthology.orchestra.run.smithi018.stdout: "in_progress": true, 2024-04-12T21:38:25.297 INFO:teuthology.orchestra.run.smithi018.stdout: "which": "Upgrading all daemon types on all hosts", 2024-04-12T21:38:25.297 INFO:teuthology.orchestra.run.smithi018.stdout: "services_complete": [ 2024-04-12T21:38:25.297 INFO:teuthology.orchestra.run.smithi018.stdout: "mgr", 2024-04-12T21:38:25.297 INFO:teuthology.orchestra.run.smithi018.stdout: "mds", 2024-04-12T21:38:25.298 INFO:teuthology.orchestra.run.smithi018.stdout: "crash", 2024-04-12T21:38:25.298 INFO:teuthology.orchestra.run.smithi018.stdout: "osd", 2024-04-12T21:38:25.298 INFO:teuthology.orchestra.run.smithi018.stdout: "mon" 2024-04-12T21:38:25.298 INFO:teuthology.orchestra.run.smithi018.stdout: ], 2024-04-12T21:38:25.298 INFO:teuthology.orchestra.run.smithi018.stdout: "progress": "16/21 daemons upgraded", 2024-04-12T21:38:25.298 INFO:teuthology.orchestra.run.smithi018.stdout: "message": "Currently upgrading mds daemons", 2024-04-12T21:38:25.298 INFO:teuthology.orchestra.run.smithi018.stdout: "is_paused": false 2024-04-12T21:38:25.298 INFO:teuthology.orchestra.run.smithi018.stdout:} 2024-04-12T21:38:25.300 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:25.300+0000 7f1616600700 1 -- 172.21.15.18:0/1454678714 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f15f406ec10 msgr2=0x7f15f40710c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:38:25.300 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:25.300+0000 7f1616600700 1 --2- 172.21.15.18:0/1454678714 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f15f406ec10 0x7f15f40710c0 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f160000b020 tx=0x7f1600009e30 comp rx=0 tx=0).stop 2024-04-12T21:38:25.300 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:25.300+0000 7f1616600700 1 -- 172.21.15.18:0/1454678714 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f16100fc5a0 msgr2=0x7f161011aa60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:38:25.300 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:25.300+0000 7f1616600700 1 --2- 172.21.15.18:0/1454678714 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f16100fc5a0 0x7f161011aa60 secure :-1 s=READY pgs=985 cs=0 l=1 rev1=1 crypto rx=0x7f15fc00c830 tx=0x7f15fc004470 comp rx=0 tx=0).stop 2024-04-12T21:38:25.300 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:25.300+0000 7f1616600700 1 -- 172.21.15.18:0/1454678714 shutdown_connections 2024-04-12T21:38:25.300 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:25.300+0000 7f1616600700 1 --2- 172.21.15.18:0/1454678714 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f15f406ec10 0x7f15f40710c0 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:38:25.300 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:25.300+0000 7f1616600700 1 --2- 172.21.15.18:0/1454678714 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f16100fbc00 0x7f161011a520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:38:25.300 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:25.300+0000 7f1616600700 1 --2- 172.21.15.18:0/1454678714 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f16100fc5a0 0x7f161011aa60 unknown :-1 s=CLOSED pgs=985 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:38:25.300 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:25.300+0000 7f1616600700 1 -- 172.21.15.18:0/1454678714 >> 172.21.15.18:0/1454678714 conn(0x7f16100f7880 msgr2=0x7f16100f9160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:38:25.300 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:25.300+0000 7f1616600700 1 -- 172.21.15.18:0/1454678714 shutdown_connections 2024-04-12T21:38:25.300 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:25.300+0000 7f1616600700 1 -- 172.21.15.18:0/1454678714 wait complete. 2024-04-12T21:38:25.452 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:25.453+0000 7fe58d000700 1 -- 172.21.15.18:0/4265063583 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe5880fe830 msgr2=0x7fe588106f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:38:25.453 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:25.453+0000 7fe58d000700 1 --2- 172.21.15.18:0/4265063583 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe5880fe830 0x7fe588106f00 secure :-1 s=READY pgs=986 cs=0 l=1 rev1=1 crypto rx=0x7fe574004940 tx=0x7fe574009e30 comp rx=0 tx=0).stop 2024-04-12T21:38:25.453 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:25.453+0000 7fe58d000700 1 -- 172.21.15.18:0/4265063583 shutdown_connections 2024-04-12T21:38:25.453 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:25.453+0000 7fe58d000700 1 --2- 172.21.15.18:0/4265063583 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe5880fe830 0x7fe588106f00 unknown :-1 s=CLOSED pgs=986 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:38:25.453 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:25.453+0000 7fe58d000700 1 --2- 172.21.15.18:0/4265063583 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe5880fde90 0x7fe5880fe260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:38:25.453 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:25.453+0000 7fe58d000700 1 -- 172.21.15.18:0/4265063583 >> 172.21.15.18:0/4265063583 conn(0x7fe588072bc0 msgr2=0x7fe588072fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:38:25.453 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:25.453+0000 7fe58d000700 1 -- 172.21.15.18:0/4265063583 shutdown_connections 2024-04-12T21:38:25.454 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:25.454+0000 7fe58d000700 1 -- 172.21.15.18:0/4265063583 wait complete. 2024-04-12T21:38:25.455 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:25.454+0000 7fe58d000700 1 Processor -- start 2024-04-12T21:38:25.455 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:25.454+0000 7fe58d000700 1 -- start start 2024-04-12T21:38:25.455 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:25.455+0000 7fe58d000700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe5880fde90 0x7fe588113ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:38:25.455 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:25.455+0000 7fe58d000700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe5880fe830 0x7fe5881140e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:38:25.455 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:25.455+0000 7fe58d000700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe588114730 con 0x7fe5880fe830 2024-04-12T21:38:25.455 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:25.455+0000 7fe58d000700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe58810db20 con 0x7fe5880fde90 2024-04-12T21:38:25.455 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:25.455+0000 7fe585400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe5880fe830 0x7fe5881140e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:38:25.455 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:25.455+0000 7fe585400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe5880fe830 0x7fe5881140e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:37624/0 (socket says 172.21.15.18:37624) 2024-04-12T21:38:25.455 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:25.455+0000 7fe585400700 1 -- 172.21.15.18:0/4088500482 learned_addr learned my addr 172.21.15.18:0/4088500482 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:38:25.456 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:25.455+0000 7fe585e00700 1 --2- 172.21.15.18:0/4088500482 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe5880fde90 0x7fe588113ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:38:25.456 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:25.456+0000 7fe585400700 1 -- 172.21.15.18:0/4088500482 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe5880fde90 msgr2=0x7fe588113ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:38:25.456 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:25.456+0000 7fe585400700 1 --2- 172.21.15.18:0/4088500482 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe5880fde90 0x7fe588113ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:38:25.456 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:25.456+0000 7fe585400700 1 -- 172.21.15.18:0/4088500482 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe574005040 con 0x7fe5880fe830 2024-04-12T21:38:25.456 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:25.456+0000 7fe585400700 1 --2- 172.21.15.18:0/4088500482 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe5880fe830 0x7fe5881140e0 secure :-1 s=READY pgs=987 cs=0 l=1 rev1=1 crypto rx=0x7fe574000c00 tx=0x7fe574009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:38:25.456 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:25.456+0000 7fe57e000700 1 -- 172.21.15.18:0/4088500482 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe57400e040 con 0x7fe5880fe830 2024-04-12T21:38:25.456 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:25.456+0000 7fe57e000700 1 -- 172.21.15.18:0/4088500482 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe574013420 con 0x7fe5880fe830 2024-04-12T21:38:25.457 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:25.456+0000 7fe57e000700 1 -- 172.21.15.18:0/4088500482 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe57401c610 con 0x7fe5880fe830 2024-04-12T21:38:25.457 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:25.456+0000 7fe58d000700 1 -- 172.21.15.18:0/4088500482 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe58810dda0 con 0x7fe5880fe830 2024-04-12T21:38:25.457 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:25.456+0000 7fe58d000700 1 -- 172.21.15.18:0/4088500482 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe58810e2e0 con 0x7fe5880fe830 2024-04-12T21:38:25.457 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:25.458+0000 7fe58d000700 1 -- 172.21.15.18:0/4088500482 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe588065e30 con 0x7fe5880fe830 2024-04-12T21:38:25.458 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:25.459+0000 7fe57e000700 1 -- 172.21.15.18:0/4088500482 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7fe574013590 con 0x7fe5880fe830 2024-04-12T21:38:25.460 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:25.460+0000 7fe57e000700 1 --2- 172.21.15.18:0/4088500482 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fe56c06ec10 0x7fe56c0710c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:38:25.460 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:25.460+0000 7fe57e000700 1 -- 172.21.15.18:0/4088500482 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 6757+0+0 (secure 0 0 0) 0x7fe57408fbf0 con 0x7fe5880fe830 2024-04-12T21:38:25.460 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:25.461+0000 7fe585e00700 1 --2- 172.21.15.18:0/4088500482 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fe56c06ec10 0x7fe56c0710c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:38:25.461 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:25.461+0000 7fe585e00700 1 --2- 172.21.15.18:0/4088500482 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fe56c06ec10 0x7fe56c0710c0 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7fe578004990 tx=0x7fe578009ee0 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:38:25.465 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:25.466+0000 7fe57e000700 1 -- 172.21.15.18:0/4088500482 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7fe57405c3a0 con 0x7fe5880fe830 2024-04-12T21:38:25.774 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:25.774+0000 7fe58d000700 1 -- 172.21.15.18:0/4088500482 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7fe58805f5f0 con 0x7fe5880fe830 2024-04-12T21:38:25.774 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:25.775+0000 7fe57e000700 1 -- 172.21.15.18:0/4088500482 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+4303 (secure 0 0 0) 0x7fe57405bf30 con 0x7fe5880fe830 2024-04-12T21:38:25.775 INFO:teuthology.orchestra.run.smithi018.stdout:HEALTH_ERR failed to probe daemons or devices; 1 filesystem is degraded; 1 filesystem is offline 2024-04-12T21:38:25.775 INFO:teuthology.orchestra.run.smithi018.stdout:[WRN] CEPHADM_REFRESH_FAILED: failed to probe daemons or devices 2024-04-12T21:38:25.776 INFO:teuthology.orchestra.run.smithi018.stdout: host smithi186 `cephadm ceph-volume` failed: cephadm exited with an error code: 1, stderr: Inferring config /var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/mon.smithi186/config 2024-04-12T21:38:25.776 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/podman: timeout after 895 seconds 2024-04-12T21:38:25.776 INFO:teuthology.orchestra.run.smithi018.stdout:Non-zero exit code 124 from /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:f4452def3be4d350f4a8e5feaf7ebb1511be9298fe68478ba7a2d6fbd7d69a63 -e NODE_NAME=smithi186 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968:/var/run/ceph:z -v /var/log/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968:/var/log/ceph:z -v /var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmptgom53f9:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:f4452def3be4d350f4a8e5feaf7ebb1511be9298fe68478ba7a2d6fbd7d69a63 inventory --format=json-pretty --filter-for-batch 2024-04-12T21:38:25.776 INFO:teuthology.orchestra.run.smithi018.stdout:Traceback (most recent call last): 2024-04-12T21:38:25.776 INFO:teuthology.orchestra.run.smithi018.stdout: File "/usr/lib64/python3.9/runpy.py", line 197, in _run_module_as_main 2024-04-12T21:38:25.776 INFO:teuthology.orchestra.run.smithi018.stdout: return _run_code(code, main_globals, None, 2024-04-12T21:38:25.776 INFO:teuthology.orchestra.run.smithi018.stdout: File "/usr/lib64/python3.9/runpy.py", line 87, in _run_code 2024-04-12T21:38:25.776 INFO:teuthology.orchestra.run.smithi018.stdout: exec(code, run_globals) 2024-04-12T21:38:25.776 INFO:teuthology.orchestra.run.smithi018.stdout: File "/var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/cephadm.e44f3ed5e34e4ef52eedef233d1190bff004470d8e8005127b5312bff357fa7f/__main__.py", line 5618, in 2024-04-12T21:38:25.776 INFO:teuthology.orchestra.run.smithi018.stdout: File "/var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/cephadm.e44f3ed5e34e4ef52eedef233d1190bff004470d8e8005127b5312bff357fa7f/__main__.py", line 5606, in main 2024-04-12T21:38:25.776 INFO:teuthology.orchestra.run.smithi018.stdout: File "/var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/cephadm.e44f3ed5e34e4ef52eedef233d1190bff004470d8e8005127b5312bff357fa7f/__main__.py", line 408, in _infer_config 2024-04-12T21:38:25.776 INFO:teuthology.orchestra.run.smithi018.stdout: File "/var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/cephadm.e44f3ed5e34e4ef52eedef233d1190bff004470d8e8005127b5312bff357fa7f/__main__.py", line 323, in _infer_fsid 2024-04-12T21:38:25.776 INFO:teuthology.orchestra.run.smithi018.stdout: File "/var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/cephadm.e44f3ed5e34e4ef52eedef233d1190bff004470d8e8005127b5312bff357fa7f/__main__.py", line 436, in _infer_image 2024-04-12T21:38:25.777 INFO:teuthology.orchestra.run.smithi018.stdout: File "/var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/cephadm.e44f3ed5e34e4ef52eedef233d1190bff004470d8e8005127b5312bff357fa7f/__main__.py", line 310, in _validate_fsid 2024-04-12T21:38:25.777 INFO:teuthology.orchestra.run.smithi018.stdout: File "/var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/cephadm.e44f3ed5e34e4ef52eedef233d1190bff004470d8e8005127b5312bff357fa7f/__main__.py", line 3359, in command_ceph_volume 2024-04-12T21:38:25.777 INFO:teuthology.orchestra.run.smithi018.stdout: File "/var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/cephadm.e44f3ed5e34e4ef52eedef233d1190bff004470d8e8005127b5312bff357fa7f/cephadmlib/call_wrappers.py", line 310, in call_throws 2024-04-12T21:38:25.777 INFO:teuthology.orchestra.run.smithi018.stdout:RuntimeError: Failed command: /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:f4452def3be4d350f4a8e5feaf7ebb1511be9298fe68478ba7a2d6fbd7d69a63 -e NODE_NAME=smithi186 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968:/var/run/ceph:z -v /var/log/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968:/var/log/ceph:z -v /var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmptgom53f9:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:f4452def3be4d350f4a8e5feaf7ebb1511be9298fe68478ba7a2d6fbd7d69a63 inventory --format=json-pretty --filter-for-batch 2024-04-12T21:38:25.777 INFO:teuthology.orchestra.run.smithi018.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-04-12T21:38:25.777 INFO:teuthology.orchestra.run.smithi018.stdout: fs cephfs is degraded 2024-04-12T21:38:25.777 INFO:teuthology.orchestra.run.smithi018.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-04-12T21:38:25.777 INFO:teuthology.orchestra.run.smithi018.stdout: fs cephfs is offline because no MDS is active for it. 2024-04-12T21:38:25.778 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:25.778+0000 7fe58d000700 1 -- 172.21.15.18:0/4088500482 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fe56c06ec10 msgr2=0x7fe56c0710c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:38:25.778 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:25.778+0000 7fe58d000700 1 --2- 172.21.15.18:0/4088500482 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fe56c06ec10 0x7fe56c0710c0 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7fe578004990 tx=0x7fe578009ee0 comp rx=0 tx=0).stop 2024-04-12T21:38:25.778 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:25.778+0000 7fe58d000700 1 -- 172.21.15.18:0/4088500482 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe5880fe830 msgr2=0x7fe5881140e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:38:25.779 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:25.778+0000 7fe58d000700 1 --2- 172.21.15.18:0/4088500482 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe5880fe830 0x7fe5881140e0 secure :-1 s=READY pgs=987 cs=0 l=1 rev1=1 crypto rx=0x7fe574000c00 tx=0x7fe574009e30 comp rx=0 tx=0).stop 2024-04-12T21:38:25.779 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:25.778+0000 7fe58d000700 1 -- 172.21.15.18:0/4088500482 shutdown_connections 2024-04-12T21:38:25.779 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:25.778+0000 7fe58d000700 1 --2- 172.21.15.18:0/4088500482 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fe56c06ec10 0x7fe56c0710c0 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:38:25.779 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:25.778+0000 7fe58d000700 1 --2- 172.21.15.18:0/4088500482 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe5880fde90 0x7fe588113ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:38:25.779 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:25.778+0000 7fe58d000700 1 --2- 172.21.15.18:0/4088500482 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe5880fe830 0x7fe5881140e0 unknown :-1 s=CLOSED pgs=987 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:38:25.779 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:25.778+0000 7fe58d000700 1 -- 172.21.15.18:0/4088500482 >> 172.21.15.18:0/4088500482 conn(0x7fe588072bc0 msgr2=0x7fe5881037b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:38:25.779 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:25.779+0000 7fe58d000700 1 -- 172.21.15.18:0/4088500482 shutdown_connections 2024-04-12T21:38:25.779 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:25.779+0000 7fe58d000700 1 -- 172.21.15.18:0/4088500482 wait complete. 2024-04-12T21:38:26.104 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:38:25 smithi018 ceph-mon[112590]: pgmap v1162: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:38:26.104 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:38:25 smithi018 ceph-mon[112590]: from='client.? 172.21.15.18:0/4088500482' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-12T21:38:26.283 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:38:25 smithi186 ceph-mon[86379]: pgmap v1162: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:38:26.283 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:38:25 smithi186 ceph-mon[86379]: from='client.? 172.21.15.18:0/4088500482' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-12T21:38:27.283 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:38:26 smithi186 ceph-mon[86379]: from='client.36040 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:38:27.354 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:38:26 smithi018 ceph-mon[112590]: from='client.36040 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:38:28.283 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:38:27 smithi186 ceph-mon[86379]: pgmap v1163: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:38:28.354 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:38:27 smithi018 ceph-mon[112590]: pgmap v1163: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:38:29.283 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:38:28 smithi186 ceph-mon[86379]: pgmap v1164: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:38:29.354 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:38:28 smithi018 ceph-mon[112590]: pgmap v1164: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:38:31.533 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:38:31 smithi186 ceph-mon[86379]: pgmap v1165: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:38:31.604 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:38:31 smithi018 ceph-mon[112590]: pgmap v1165: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:38:33.533 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:38:33 smithi186 ceph-mon[86379]: pgmap v1166: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:38:33.604 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:38:33 smithi018 ceph-mon[112590]: pgmap v1166: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:38:35.385 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:38:35 smithi018 ceph-mon[112590]: pgmap v1167: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:38:35.533 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:38:35 smithi186 ceph-mon[86379]: pgmap v1167: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:38:37.354 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:38:37 smithi018 ceph-mon[112590]: pgmap v1168: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:38:37.533 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:38:37 smithi186 ceph-mon[86379]: pgmap v1168: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:38:39.533 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:38:39 smithi186 ceph-mon[86379]: pgmap v1169: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:38:39.604 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:38:39 smithi018 ceph-mon[112590]: pgmap v1169: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:38:40.533 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:38:40 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T21:38:40.604 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:38:40 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T21:38:41.533 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:38:41 smithi186 ceph-mon[86379]: pgmap v1170: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:38:41.604 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:38:41 smithi018 ceph-mon[112590]: pgmap v1170: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:38:43.533 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:38:43 smithi186 ceph-mon[86379]: pgmap v1171: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:38:43.604 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:38:43 smithi018 ceph-mon[112590]: pgmap v1171: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:38:45.533 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:38:45 smithi186 ceph-mon[86379]: pgmap v1172: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:38:45.604 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:38:45 smithi018 ceph-mon[112590]: pgmap v1172: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:38:47.533 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:38:47 smithi186 ceph-mon[86379]: pgmap v1173: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:38:47.604 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:38:47 smithi018 ceph-mon[112590]: pgmap v1173: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:38:49.533 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:38:49 smithi186 ceph-mon[86379]: pgmap v1174: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:38:49.604 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:38:49 smithi018 ceph-mon[112590]: pgmap v1174: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:38:51.533 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:38:51 smithi186 ceph-mon[86379]: pgmap v1175: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:38:51.604 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:38:51 smithi018 ceph-mon[112590]: pgmap v1175: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:38:53.533 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:38:53 smithi186 ceph-mon[86379]: pgmap v1176: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:38:53.604 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:38:53 smithi018 ceph-mon[112590]: pgmap v1176: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:38:55.533 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:38:55 smithi186 ceph-mon[86379]: pgmap v1177: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:38:55.533 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:38:55 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T21:38:55.604 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:38:55 smithi018 ceph-mon[112590]: pgmap v1177: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:38:55.604 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:38:55 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T21:38:55.952 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:55.951+0000 7faf14200700 1 -- 172.21.15.18:0/1261079458 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7faf0c0724f0 msgr2=0x7faf0c0728c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:38:55.953 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:55.951+0000 7faf14200700 1 --2- 172.21.15.18:0/1261079458 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7faf0c0724f0 0x7faf0c0728c0 secure :-1 s=READY pgs=988 cs=0 l=1 rev1=1 crypto rx=0x7faf00004940 tx=0x7faf00009e30 comp rx=0 tx=0).stop 2024-04-12T21:38:55.953 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:55.951+0000 7faf14200700 1 -- 172.21.15.18:0/1261079458 shutdown_connections 2024-04-12T21:38:55.953 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:55.951+0000 7faf14200700 1 --2- 172.21.15.18:0/1261079458 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7faf0c072e00 0x7faf0c10d780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:38:55.953 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:55.951+0000 7faf14200700 1 --2- 172.21.15.18:0/1261079458 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7faf0c0724f0 0x7faf0c0728c0 unknown :-1 s=CLOSED pgs=988 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:38:55.953 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:55.951+0000 7faf14200700 1 -- 172.21.15.18:0/1261079458 >> 172.21.15.18:0/1261079458 conn(0x7faf0c0faba0 msgr2=0x7faf0c0fcfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:38:55.953 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:55.951+0000 7faf14200700 1 -- 172.21.15.18:0/1261079458 shutdown_connections 2024-04-12T21:38:55.953 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:55.951+0000 7faf14200700 1 -- 172.21.15.18:0/1261079458 wait complete. 2024-04-12T21:38:55.953 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:55.952+0000 7faf14200700 1 Processor -- start 2024-04-12T21:38:55.953 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:55.952+0000 7faf14200700 1 -- start start 2024-04-12T21:38:55.955 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:55.952+0000 7faf14200700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7faf0c0724f0 0x7faf0c11a0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:38:55.955 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:55.952+0000 7faf14200700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7faf0c072e00 0x7faf0c11a630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:38:55.955 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:55.952+0000 7faf14200700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf0c11ac80 con 0x7faf0c0724f0 2024-04-12T21:38:55.955 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:55.952+0000 7faf14200700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf0c114090 con 0x7faf0c072e00 2024-04-12T21:38:55.955 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:55.953+0000 7faf11800700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7faf0c0724f0 0x7faf0c11a0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:38:55.955 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:55.953+0000 7faf11800700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7faf0c0724f0 0x7faf0c11a0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:60096/0 (socket says 172.21.15.18:60096) 2024-04-12T21:38:55.955 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:55.953+0000 7faf11800700 1 -- 172.21.15.18:0/3574308794 learned_addr learned my addr 172.21.15.18:0/3574308794 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:38:55.955 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:55.953+0000 7faf10e00700 1 --2- 172.21.15.18:0/3574308794 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7faf0c072e00 0x7faf0c11a630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:38:55.955 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:55.953+0000 7faf11800700 1 -- 172.21.15.18:0/3574308794 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7faf0c072e00 msgr2=0x7faf0c11a630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:38:55.955 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:55.953+0000 7faf11800700 1 --2- 172.21.15.18:0/3574308794 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7faf0c072e00 0x7faf0c11a630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:38:55.955 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:55.953+0000 7faf11800700 1 -- 172.21.15.18:0/3574308794 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf00005040 con 0x7faf0c0724f0 2024-04-12T21:38:55.955 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:55.954+0000 7faf11800700 1 --2- 172.21.15.18:0/3574308794 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7faf0c0724f0 0x7faf0c11a0f0 secure :-1 s=READY pgs=989 cs=0 l=1 rev1=1 crypto rx=0x7faf0000b3b0 tx=0x7faf00009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:38:55.955 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:55.954+0000 7faf09600700 1 -- 172.21.15.18:0/3574308794 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7faf0000e040 con 0x7faf0c0724f0 2024-04-12T21:38:55.956 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:55.954+0000 7faf09600700 1 -- 172.21.15.18:0/3574308794 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7faf00013420 con 0x7faf0c0724f0 2024-04-12T21:38:55.956 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:55.954+0000 7faf09600700 1 -- 172.21.15.18:0/3574308794 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7faf0001c630 con 0x7faf0c0724f0 2024-04-12T21:38:55.956 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:55.954+0000 7faf14200700 1 -- 172.21.15.18:0/3574308794 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faf0c114310 con 0x7faf0c0724f0 2024-04-12T21:38:55.956 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:55.954+0000 7faf14200700 1 -- 172.21.15.18:0/3574308794 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faf0c1148d0 con 0x7faf0c0724f0 2024-04-12T21:38:55.956 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:55.955+0000 7faf14200700 1 -- 172.21.15.18:0/3574308794 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faf0c073c30 con 0x7faf0c0724f0 2024-04-12T21:38:55.957 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:55.956+0000 7faf09600700 1 -- 172.21.15.18:0/3574308794 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7faf00013590 con 0x7faf0c0724f0 2024-04-12T21:38:55.958 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:55.957+0000 7faf09600700 1 --2- 172.21.15.18:0/3574308794 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7faef406ec10 0x7faef40710c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:38:55.958 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:55.958+0000 7faf09600700 1 -- 172.21.15.18:0/3574308794 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 6757+0+0 (secure 0 0 0) 0x7faf0008fc50 con 0x7faf0c0724f0 2024-04-12T21:38:55.958 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:55.958+0000 7faf10e00700 1 --2- 172.21.15.18:0/3574308794 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7faef406ec10 0x7faef40710c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:38:55.959 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:55.959+0000 7faf10e00700 1 --2- 172.21.15.18:0/3574308794 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7faef406ec10 0x7faef40710c0 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7faf0c1159d0 tx=0x7faef8009ee0 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:38:55.963 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:55.963+0000 7faf09600700 1 -- 172.21.15.18:0/3574308794 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7faf00058970 con 0x7faf0c0724f0 2024-04-12T21:38:56.215 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:56.214+0000 7faf14200700 1 -- 172.21.15.18:0/3574308794 --> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7faf0c04bed0 con 0x7faef406ec10 2024-04-12T21:38:56.217 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:56.216+0000 7faf09600700 1 -- 172.21.15.18:0/3574308794 <== mgr.34102 v2:172.21.15.18:6800/3812891919 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7faf0c04bed0 con 0x7faef406ec10 2024-04-12T21:38:56.221 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:56.219+0000 7faf14200700 1 -- 172.21.15.18:0/3574308794 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7faef406ec10 msgr2=0x7faef40710c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:38:56.221 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:56.219+0000 7faf14200700 1 --2- 172.21.15.18:0/3574308794 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7faef406ec10 0x7faef40710c0 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7faf0c1159d0 tx=0x7faef8009ee0 comp rx=0 tx=0).stop 2024-04-12T21:38:56.221 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:56.219+0000 7faf14200700 1 -- 172.21.15.18:0/3574308794 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7faf0c0724f0 msgr2=0x7faf0c11a0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:38:56.221 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:56.219+0000 7faf14200700 1 --2- 172.21.15.18:0/3574308794 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7faf0c0724f0 0x7faf0c11a0f0 secure :-1 s=READY pgs=989 cs=0 l=1 rev1=1 crypto rx=0x7faf0000b3b0 tx=0x7faf00009e30 comp rx=0 tx=0).stop 2024-04-12T21:38:56.221 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:56.219+0000 7faf14200700 1 -- 172.21.15.18:0/3574308794 shutdown_connections 2024-04-12T21:38:56.221 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:56.219+0000 7faf14200700 1 --2- 172.21.15.18:0/3574308794 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7faef406ec10 0x7faef40710c0 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:38:56.221 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:56.219+0000 7faf14200700 1 --2- 172.21.15.18:0/3574308794 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7faf0c0724f0 0x7faf0c11a0f0 unknown :-1 s=CLOSED pgs=989 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:38:56.221 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:56.219+0000 7faf14200700 1 --2- 172.21.15.18:0/3574308794 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7faf0c072e00 0x7faf0c11a630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:38:56.221 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:56.219+0000 7faf14200700 1 -- 172.21.15.18:0/3574308794 >> 172.21.15.18:0/3574308794 conn(0x7faf0c0faba0 msgr2=0x7faf0c0faf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:38:56.221 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:56.220+0000 7faf14200700 1 -- 172.21.15.18:0/3574308794 shutdown_connections 2024-04-12T21:38:56.221 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:56.220+0000 7faf14200700 1 -- 172.21.15.18:0/3574308794 wait complete. 2024-04-12T21:38:56.237 INFO:teuthology.orchestra.run.smithi018.stdout:true 2024-04-12T21:38:56.381 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:56.381+0000 7f19da000700 1 -- 172.21.15.18:0/2124736479 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f19d41059c0 msgr2=0x7f19d40ff080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:38:56.382 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:56.381+0000 7f19da000700 1 --2- 172.21.15.18:0/2124736479 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f19d41059c0 0x7f19d40ff080 secure :-1 s=READY pgs=990 cs=0 l=1 rev1=1 crypto rx=0x7f19c0004940 tx=0x7f19c0009e30 comp rx=0 tx=0).stop 2024-04-12T21:38:56.382 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:56.381+0000 7f19da000700 1 -- 172.21.15.18:0/2124736479 shutdown_connections 2024-04-12T21:38:56.382 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:56.381+0000 7f19da000700 1 --2- 172.21.15.18:0/2124736479 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f19d41059c0 0x7f19d40ff080 unknown :-1 s=CLOSED pgs=990 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:38:56.382 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:56.381+0000 7f19da000700 1 --2- 172.21.15.18:0/2124736479 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f19d4105020 0x7f19d41053f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:38:56.382 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:56.381+0000 7f19da000700 1 -- 172.21.15.18:0/2124736479 >> 172.21.15.18:0/2124736479 conn(0x7f19d40faba0 msgr2=0x7f19d40fcfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:38:56.382 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:56.381+0000 7f19da000700 1 -- 172.21.15.18:0/2124736479 shutdown_connections 2024-04-12T21:38:56.382 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:56.381+0000 7f19da000700 1 -- 172.21.15.18:0/2124736479 wait complete. 2024-04-12T21:38:56.383 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:56.382+0000 7f19da000700 1 Processor -- start 2024-04-12T21:38:56.383 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:56.382+0000 7f19da000700 1 -- start start 2024-04-12T21:38:56.383 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:56.383+0000 7f19da000700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f19d4105020 0x7f19d4115ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:38:56.383 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:56.383+0000 7f19da000700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f19d41059c0 0x7f19d41160e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:38:56.383 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:56.383+0000 7f19da000700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19d4116730 con 0x7f19d41059c0 2024-04-12T21:38:56.383 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:56.383+0000 7f19da000700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19d410fd10 con 0x7f19d4105020 2024-04-12T21:38:56.384 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:56.383+0000 7f19d2a00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f19d41059c0 0x7f19d41160e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:38:56.384 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:56.383+0000 7f19d2a00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f19d41059c0 0x7f19d41160e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:60114/0 (socket says 172.21.15.18:60114) 2024-04-12T21:38:56.384 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:56.383+0000 7f19d2a00700 1 -- 172.21.15.18:0/3041521550 learned_addr learned my addr 172.21.15.18:0/3041521550 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:38:56.384 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:56.383+0000 7f19d3400700 1 --2- 172.21.15.18:0/3041521550 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f19d4105020 0x7f19d4115ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:38:56.384 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:56.384+0000 7f19d2a00700 1 -- 172.21.15.18:0/3041521550 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f19d4105020 msgr2=0x7f19d4115ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:38:56.385 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:56.384+0000 7f19d2a00700 1 --2- 172.21.15.18:0/3041521550 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f19d4105020 0x7f19d4115ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:38:56.385 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:56.384+0000 7f19d2a00700 1 -- 172.21.15.18:0/3041521550 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19c0005040 con 0x7f19d41059c0 2024-04-12T21:38:56.385 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:56.384+0000 7f19d2a00700 1 --2- 172.21.15.18:0/3041521550 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f19d41059c0 0x7f19d41160e0 secure :-1 s=READY pgs=991 cs=0 l=1 rev1=1 crypto rx=0x7f19c0009100 tx=0x7f19c0009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:38:56.385 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:56.384+0000 7f19cb400700 1 -- 172.21.15.18:0/3041521550 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f19c000e040 con 0x7f19d41059c0 2024-04-12T21:38:56.385 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:56.384+0000 7f19cb400700 1 -- 172.21.15.18:0/3041521550 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f19c0004290 con 0x7f19d41059c0 2024-04-12T21:38:56.386 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:56.384+0000 7f19cb400700 1 -- 172.21.15.18:0/3041521550 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f19c001c3f0 con 0x7f19d41059c0 2024-04-12T21:38:56.386 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:56.384+0000 7f19da000700 1 -- 172.21.15.18:0/3041521550 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f19d410ff90 con 0x7f19d41059c0 2024-04-12T21:38:56.386 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:56.384+0000 7f19da000700 1 -- 172.21.15.18:0/3041521550 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f19d4110550 con 0x7f19d41059c0 2024-04-12T21:38:56.386 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:56.385+0000 7f19da000700 1 -- 172.21.15.18:0/3041521550 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f19d4100770 con 0x7f19d41059c0 2024-04-12T21:38:56.387 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:56.387+0000 7f19cb400700 1 -- 172.21.15.18:0/3041521550 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7f19c0013790 con 0x7f19d41059c0 2024-04-12T21:38:56.388 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:56.388+0000 7f19cb400700 1 --2- 172.21.15.18:0/3041521550 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f19b806ec00 0x7f19b80710b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:38:56.388 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:56.388+0000 7f19cb400700 1 -- 172.21.15.18:0/3041521550 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 6757+0+0 (secure 0 0 0) 0x7f19c008fc60 con 0x7f19d41059c0 2024-04-12T21:38:56.388 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:56.388+0000 7f19d3400700 1 --2- 172.21.15.18:0/3041521550 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f19b806ec00 0x7f19b80710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:38:56.389 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:56.389+0000 7f19d3400700 1 --2- 172.21.15.18:0/3041521550 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f19b806ec00 0x7f19b80710b0 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f19c4007fd0 tx=0x7f19c4009040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:38:56.393 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:56.393+0000 7f19cb400700 1 -- 172.21.15.18:0/3041521550 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f19c005c490 con 0x7f19d41059c0 2024-04-12T21:38:56.646 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:56.645+0000 7f19da000700 1 -- 172.21.15.18:0/3041521550 --> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f19d4100770 con 0x7f19b806ec00 2024-04-12T21:38:56.648 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:56.647+0000 7f19cb400700 1 -- 172.21.15.18:0/3041521550 <== mgr.34102 v2:172.21.15.18:6800/3812891919 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f19d4100770 con 0x7f19b806ec00 2024-04-12T21:38:56.651 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:56.650+0000 7f19da000700 1 -- 172.21.15.18:0/3041521550 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f19b806ec00 msgr2=0x7f19b80710b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:38:56.651 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:56.650+0000 7f19da000700 1 --2- 172.21.15.18:0/3041521550 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f19b806ec00 0x7f19b80710b0 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f19c4007fd0 tx=0x7f19c4009040 comp rx=0 tx=0).stop 2024-04-12T21:38:56.651 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:56.650+0000 7f19da000700 1 -- 172.21.15.18:0/3041521550 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f19d41059c0 msgr2=0x7f19d41160e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:38:56.651 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:56.650+0000 7f19da000700 1 --2- 172.21.15.18:0/3041521550 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f19d41059c0 0x7f19d41160e0 secure :-1 s=READY pgs=991 cs=0 l=1 rev1=1 crypto rx=0x7f19c0009100 tx=0x7f19c0009e30 comp rx=0 tx=0).stop 2024-04-12T21:38:56.652 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:56.651+0000 7f19da000700 1 -- 172.21.15.18:0/3041521550 shutdown_connections 2024-04-12T21:38:56.652 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:56.651+0000 7f19da000700 1 --2- 172.21.15.18:0/3041521550 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f19b806ec00 0x7f19b80710b0 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:38:56.652 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:56.651+0000 7f19da000700 1 --2- 172.21.15.18:0/3041521550 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f19d4105020 0x7f19d4115ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:38:56.652 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:56.651+0000 7f19da000700 1 --2- 172.21.15.18:0/3041521550 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f19d41059c0 0x7f19d41160e0 unknown :-1 s=CLOSED pgs=991 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:38:56.652 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:56.651+0000 7f19da000700 1 -- 172.21.15.18:0/3041521550 >> 172.21.15.18:0/3041521550 conn(0x7f19d40faba0 msgr2=0x7f19d40fbb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:38:56.652 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:56.651+0000 7f19da000700 1 -- 172.21.15.18:0/3041521550 shutdown_connections 2024-04-12T21:38:56.652 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:56.651+0000 7f19da000700 1 -- 172.21.15.18:0/3041521550 wait complete. 2024-04-12T21:38:56.795 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:56.794+0000 7fddd6800700 1 -- 172.21.15.18:0/628697373 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fddd0070300 msgr2=0x7fddd00706d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:38:56.796 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:56.794+0000 7fddd6800700 1 --2- 172.21.15.18:0/628697373 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fddd0070300 0x7fddd00706d0 secure :-1 s=READY pgs=992 cs=0 l=1 rev1=1 crypto rx=0x7fddc0004970 tx=0x7fddc0009e30 comp rx=0 tx=0).stop 2024-04-12T21:38:56.796 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:56.794+0000 7fddd6800700 1 -- 172.21.15.18:0/628697373 shutdown_connections 2024-04-12T21:38:56.796 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:56.794+0000 7fddd6800700 1 --2- 172.21.15.18:0/628697373 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fddd0070c10 0x7fddd0109190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:38:56.796 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:56.794+0000 7fddd6800700 1 --2- 172.21.15.18:0/628697373 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fddd0070300 0x7fddd00706d0 unknown :-1 s=CLOSED pgs=992 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:38:56.796 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:56.794+0000 7fddd6800700 1 -- 172.21.15.18:0/628697373 >> 172.21.15.18:0/628697373 conn(0x7fddd00f8970 msgr2=0x7fddd00fad80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:38:56.796 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:56.795+0000 7fddd6800700 1 -- 172.21.15.18:0/628697373 shutdown_connections 2024-04-12T21:38:56.796 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:56.795+0000 7fddd6800700 1 -- 172.21.15.18:0/628697373 wait complete. 2024-04-12T21:38:56.796 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:56.796+0000 7fddd6800700 1 Processor -- start 2024-04-12T21:38:56.797 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:56.796+0000 7fddd6800700 1 -- start start 2024-04-12T21:38:56.797 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:56.797+0000 7fddd6800700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fddd0070300 0x7fddd0115d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:38:56.797 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:56.797+0000 7fddd6800700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fddd0070c10 0x7fddd0116280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:38:56.797 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:56.797+0000 7fddd6800700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fddd01168d0 con 0x7fddd0070c10 2024-04-12T21:38:56.797 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:56.797+0000 7fddd6800700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fddd010fca0 con 0x7fddd0070300 2024-04-12T21:38:56.799 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:56.797+0000 7fddc7e00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fddd0070c10 0x7fddd0116280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:38:56.799 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:56.797+0000 7fddc7e00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fddd0070c10 0x7fddd0116280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:60128/0 (socket says 172.21.15.18:60128) 2024-04-12T21:38:56.799 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:56.797+0000 7fddc7e00700 1 -- 172.21.15.18:0/3243476567 learned_addr learned my addr 172.21.15.18:0/3243476567 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:38:56.799 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:56.797+0000 7fddcf400700 1 --2- 172.21.15.18:0/3243476567 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fddd0070300 0x7fddd0115d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:38:56.799 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:56.797+0000 7fddc7e00700 1 -- 172.21.15.18:0/3243476567 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fddd0070300 msgr2=0x7fddd0115d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:38:56.799 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:56.797+0000 7fddc7e00700 1 --2- 172.21.15.18:0/3243476567 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fddd0070300 0x7fddd0115d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:38:56.799 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:56.797+0000 7fddc7e00700 1 -- 172.21.15.18:0/3243476567 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fddc0005040 con 0x7fddd0070c10 2024-04-12T21:38:56.799 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:56.798+0000 7fddc7e00700 1 --2- 172.21.15.18:0/3243476567 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fddd0070c10 0x7fddd0116280 secure :-1 s=READY pgs=993 cs=0 l=1 rev1=1 crypto rx=0x7fddb800c830 tx=0x7fddb8004470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:38:56.799 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:56.798+0000 7fddc7400700 1 -- 172.21.15.18:0/3243476567 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fddb800a760 con 0x7fddd0070c10 2024-04-12T21:38:56.799 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:56.798+0000 7fddc7400700 1 -- 172.21.15.18:0/3243476567 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fddb800ad90 con 0x7fddd0070c10 2024-04-12T21:38:56.800 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:56.798+0000 7fddd6800700 1 -- 172.21.15.18:0/3243476567 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fddd010ff80 con 0x7fddd0070c10 2024-04-12T21:38:56.800 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:56.798+0000 7fddc7400700 1 -- 172.21.15.18:0/3243476567 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fddb80113a0 con 0x7fddd0070c10 2024-04-12T21:38:56.800 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:56.798+0000 7fddd6800700 1 -- 172.21.15.18:0/3243476567 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fddd01104c0 con 0x7fddd0070c10 2024-04-12T21:38:56.800 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:56.799+0000 7fddd6800700 1 -- 172.21.15.18:0/3243476567 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fddd0105fc0 con 0x7fddd0070c10 2024-04-12T21:38:56.801 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:56.800+0000 7fddc7400700 1 -- 172.21.15.18:0/3243476567 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7fddb8012400 con 0x7fddd0070c10 2024-04-12T21:38:56.802 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:56.802+0000 7fddc7400700 1 --2- 172.21.15.18:0/3243476567 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fddb406ec10 0x7fddb40710c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:38:56.802 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:56.802+0000 7fddc7400700 1 -- 172.21.15.18:0/3243476567 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 6757+0+0 (secure 0 0 0) 0x7fddb808e900 con 0x7fddd0070c10 2024-04-12T21:38:56.803 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:56.802+0000 7fddcf400700 1 --2- 172.21.15.18:0/3243476567 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fddb406ec10 0x7fddb40710c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:38:56.803 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:56.803+0000 7fddcf400700 1 --2- 172.21.15.18:0/3243476567 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fddb406ec10 0x7fddb40710c0 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7fddc0004ee0 tx=0x7fddc0009e30 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:38:56.807 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:56.806+0000 7fddc7400700 1 -- 172.21.15.18:0/3243476567 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7fddb805b1f0 con 0x7fddd0070c10 2024-04-12T21:38:57.036 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:57.035+0000 7fddd6800700 1 -- 172.21.15.18:0/3243476567 --> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fddd01113e0 con 0x7fddb406ec10 2024-04-12T21:38:57.066 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:57.065+0000 7fddc7400700 1 -- 172.21.15.18:0/3243476567 <== mgr.34102 v2:172.21.15.18:6800/3812891919 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7fddd01113e0 con 0x7fddb406ec10 2024-04-12T21:38:57.066 INFO:teuthology.orchestra.run.smithi018.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-12T21:38:57.067 INFO:teuthology.orchestra.run.smithi018.stdout:alertmanager.smithi018 smithi018 *:9093,9094 running (39m) 37s ago 54m 14.8M - 0.25.0 c8568f914cd2 c332090068e8 2024-04-12T21:38:57.067 INFO:teuthology.orchestra.run.smithi018.stdout:crash.smithi018 smithi018 running (35m) 37s ago 54m 6665k - 19.0.0-2879-g324baf8d 8717720dc5e5 b7f80d1f7982 2024-04-12T21:38:57.067 INFO:teuthology.orchestra.run.smithi018.stdout:crash.smithi186 smithi186 running (35m) 40s ago 52m 6660k - 19.0.0-2879-g324baf8d 8717720dc5e5 a4f55c81b623 2024-04-12T21:38:57.067 INFO:teuthology.orchestra.run.smithi018.stdout:grafana.smithi018 smithi018 *:3000 running (38m) 37s ago 53m 73.8M - 9.4.12 2bacad6d85d8 b08e3bf176dd 2024-04-12T21:38:57.067 INFO:teuthology.orchestra.run.smithi018.stdout:mds.cephfs.smithi018.acqbsc smithi018 running (31m) 37s ago 46m 21.9M - 19.0.0-2879-g324baf8d 8717720dc5e5 021f99ab24ae 2024-04-12T21:38:57.067 INFO:teuthology.orchestra.run.smithi018.stdout:mds.cephfs.smithi018.szjluo smithi018 running (31m) 37s ago 46m 21.7M - 19.0.0-2879-g324baf8d 8717720dc5e5 0003117479d4 2024-04-12T21:38:57.067 INFO:teuthology.orchestra.run.smithi018.stdout:mds.cephfs.smithi186.jdllyn smithi186 running (16m) 40s ago 46m 16.5M - 19.0.0-2879-g324baf8d 8717720dc5e5 21b1f46eef23 2024-04-12T21:38:57.067 INFO:teuthology.orchestra.run.smithi018.stdout:mds.cephfs.smithi186.wjnafs smithi186 running (45s) 40s ago 46m 11.0M - 19.0.0-2879-g324baf8d 8717720dc5e5 3cf1ef4148e2 2024-04-12T21:38:57.067 INFO:teuthology.orchestra.run.smithi018.stdout:mgr.smithi018.xjzwga smithi018 *:8443,9283,8765 running (42m) 37s ago 55m 517M - 19.0.0-2879-g324baf8d 8717720dc5e5 56088305eab7 2024-04-12T21:38:57.067 INFO:teuthology.orchestra.run.smithi018.stdout:mgr.smithi186.kqcfxk smithi186 *:8443,9283,8765 running (40m) 40s ago 52m 460M - 19.0.0-2879-g324baf8d 8717720dc5e5 09cd766e5e0d 2024-04-12T21:38:57.067 INFO:teuthology.orchestra.run.smithi018.stdout:mon.smithi018 smithi018 running (37m) 37s ago 55m 88.2M 2048M 19.0.0-2879-g324baf8d 8717720dc5e5 df3e201dfed6 2024-04-12T21:38:57.067 INFO:teuthology.orchestra.run.smithi018.stdout:mon.smithi186 smithi186 running (36m) 40s ago 52m 53.6M 2048M 19.0.0-2879-g324baf8d 8717720dc5e5 c77498da9b2b 2024-04-12T21:38:57.067 INFO:teuthology.orchestra.run.smithi018.stdout:node-exporter.smithi018 smithi018 *:9100 running (40m) 37s ago 54m 8468k - 1.5.0 0da6a335fe13 326c3d9c6847 2024-04-12T21:38:57.067 INFO:teuthology.orchestra.run.smithi018.stdout:node-exporter.smithi186 smithi186 *:9100 running (40m) 40s ago 52m 8166k - 1.5.0 0da6a335fe13 ec2112baeecf 2024-04-12T21:38:57.068 INFO:teuthology.orchestra.run.smithi018.stdout:osd.0 smithi018 running (35m) 37s ago 51m 123M 1559M 19.0.0-2879-g324baf8d 8717720dc5e5 60b1273ca047 2024-04-12T21:38:57.068 INFO:teuthology.orchestra.run.smithi018.stdout:osd.1 smithi018 running (34m) 37s ago 51m 95.4M 1559M 19.0.0-2879-g324baf8d 8717720dc5e5 170ce1e5fb8d 2024-04-12T21:38:57.068 INFO:teuthology.orchestra.run.smithi018.stdout:osd.2 smithi018 running (34m) 37s ago 50m 94.6M 1559M 19.0.0-2879-g324baf8d 8717720dc5e5 c2cbc6aa2839 2024-04-12T21:38:57.068 INFO:teuthology.orchestra.run.smithi018.stdout:osd.3 smithi186 running (33m) 40s ago 49m 101M 2581M 19.0.0-2879-g324baf8d 8717720dc5e5 0138e8262e4b 2024-04-12T21:38:57.068 INFO:teuthology.orchestra.run.smithi018.stdout:osd.4 smithi186 running (32m) 40s ago 49m 91.5M 2581M 19.0.0-2879-g324baf8d 8717720dc5e5 b20cb63b25c9 2024-04-12T21:38:57.068 INFO:teuthology.orchestra.run.smithi018.stdout:osd.5 smithi186 running (32m) 40s ago 48m 92.2M 2581M 19.0.0-2879-g324baf8d 8717720dc5e5 2513bea2610e 2024-04-12T21:38:57.068 INFO:teuthology.orchestra.run.smithi018.stdout:prometheus.smithi018 smithi018 *:9095 running (40m) 37s ago 53m 69.6M - 2.43.0 a07b618ecd1d f58841127e21 2024-04-12T21:38:57.070 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:57.069+0000 7fddd6800700 1 -- 172.21.15.18:0/3243476567 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fddb406ec10 msgr2=0x7fddb40710c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:38:57.070 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:57.069+0000 7fddd6800700 1 --2- 172.21.15.18:0/3243476567 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fddb406ec10 0x7fddb40710c0 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7fddc0004ee0 tx=0x7fddc0009e30 comp rx=0 tx=0).stop 2024-04-12T21:38:57.070 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:57.069+0000 7fddd6800700 1 -- 172.21.15.18:0/3243476567 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fddd0070c10 msgr2=0x7fddd0116280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:38:57.070 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:57.069+0000 7fddd6800700 1 --2- 172.21.15.18:0/3243476567 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fddd0070c10 0x7fddd0116280 secure :-1 s=READY pgs=993 cs=0 l=1 rev1=1 crypto rx=0x7fddb800c830 tx=0x7fddb8004470 comp rx=0 tx=0).stop 2024-04-12T21:38:57.070 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:57.069+0000 7fddd6800700 1 -- 172.21.15.18:0/3243476567 shutdown_connections 2024-04-12T21:38:57.070 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:57.069+0000 7fddd6800700 1 --2- 172.21.15.18:0/3243476567 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fddb406ec10 0x7fddb40710c0 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:38:57.070 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:57.069+0000 7fddd6800700 1 --2- 172.21.15.18:0/3243476567 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fddd0070300 0x7fddd0115d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:38:57.071 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:57.069+0000 7fddd6800700 1 --2- 172.21.15.18:0/3243476567 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fddd0070c10 0x7fddd0116280 unknown :-1 s=CLOSED pgs=993 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:38:57.071 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:57.069+0000 7fddd6800700 1 -- 172.21.15.18:0/3243476567 >> 172.21.15.18:0/3243476567 conn(0x7fddd00f8970 msgr2=0x7fddd0103140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:38:57.071 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:57.069+0000 7fddd6800700 1 -- 172.21.15.18:0/3243476567 shutdown_connections 2024-04-12T21:38:57.071 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:57.069+0000 7fddd6800700 1 -- 172.21.15.18:0/3243476567 wait complete. 2024-04-12T21:38:57.228 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:57.227+0000 7f1c31600700 1 -- 172.21.15.18:0/2143948447 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1c2c070c70 msgr2=0x7f1c2c108f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:38:57.228 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:57.227+0000 7f1c31600700 1 --2- 172.21.15.18:0/2143948447 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1c2c070c70 0x7f1c2c108f60 secure :-1 s=READY pgs=994 cs=0 l=1 rev1=1 crypto rx=0x7f1c18004970 tx=0x7f1c18009e30 comp rx=0 tx=0).stop 2024-04-12T21:38:57.228 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:57.227+0000 7f1c31600700 1 -- 172.21.15.18:0/2143948447 shutdown_connections 2024-04-12T21:38:57.228 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:57.227+0000 7f1c31600700 1 --2- 172.21.15.18:0/2143948447 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1c2c070c70 0x7f1c2c108f60 unknown :-1 s=CLOSED pgs=994 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:38:57.228 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:57.227+0000 7f1c31600700 1 --2- 172.21.15.18:0/2143948447 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1c2c070360 0x7f1c2c070730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:38:57.228 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:57.227+0000 7f1c31600700 1 -- 172.21.15.18:0/2143948447 >> 172.21.15.18:0/2143948447 conn(0x7f1c2c0f87e0 msgr2=0x7f1c2c0fabf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:38:57.228 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:57.227+0000 7f1c31600700 1 -- 172.21.15.18:0/2143948447 shutdown_connections 2024-04-12T21:38:57.228 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:57.227+0000 7f1c31600700 1 -- 172.21.15.18:0/2143948447 wait complete. 2024-04-12T21:38:57.229 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:57.228+0000 7f1c31600700 1 Processor -- start 2024-04-12T21:38:57.229 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:57.228+0000 7f1c31600700 1 -- start start 2024-04-12T21:38:57.229 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:57.229+0000 7f1c31600700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1c2c070360 0x7f1c2c0fcbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:38:57.229 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:57.229+0000 7f1c31600700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1c2c070c70 0x7f1c2c0fd100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:38:57.229 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:57.229+0000 7f1c31600700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c2c100d00 con 0x7f1c2c070c70 2024-04-12T21:38:57.229 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:57.229+0000 7f1c31600700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c2c0fd640 con 0x7f1c2c070360 2024-04-12T21:38:57.230 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:57.229+0000 7f1c2b400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1c2c070c70 0x7f1c2c0fd100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:38:57.230 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:57.229+0000 7f1c2b400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1c2c070c70 0x7f1c2c0fd100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:60146/0 (socket says 172.21.15.18:60146) 2024-04-12T21:38:57.231 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:57.229+0000 7f1c2b400700 1 -- 172.21.15.18:0/1092903600 learned_addr learned my addr 172.21.15.18:0/1092903600 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:38:57.231 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:57.229+0000 7f1c2be00700 1 --2- 172.21.15.18:0/1092903600 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1c2c070360 0x7f1c2c0fcbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:38:57.231 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:57.230+0000 7f1c2b400700 1 -- 172.21.15.18:0/1092903600 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1c2c070360 msgr2=0x7f1c2c0fcbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:38:57.231 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:57.230+0000 7f1c2b400700 1 --2- 172.21.15.18:0/1092903600 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1c2c070360 0x7f1c2c0fcbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:38:57.231 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:57.230+0000 7f1c2b400700 1 -- 172.21.15.18:0/1092903600 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c18005040 con 0x7f1c2c070c70 2024-04-12T21:38:57.232 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:57.230+0000 7f1c2b400700 1 --2- 172.21.15.18:0/1092903600 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1c2c070c70 0x7f1c2c0fd100 secure :-1 s=READY pgs=995 cs=0 l=1 rev1=1 crypto rx=0x7f1c18004f50 tx=0x7f1c18009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:38:57.232 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:57.230+0000 7f1c23e00700 1 -- 172.21.15.18:0/1092903600 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1c1800e040 con 0x7f1c2c070c70 2024-04-12T21:38:57.232 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:57.230+0000 7f1c23e00700 1 -- 172.21.15.18:0/1092903600 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f1c18004290 con 0x7f1c2c070c70 2024-04-12T21:38:57.232 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:57.230+0000 7f1c23e00700 1 -- 172.21.15.18:0/1092903600 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1c1801c3f0 con 0x7f1c2c070c70 2024-04-12T21:38:57.232 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:57.230+0000 7f1c31600700 1 -- 172.21.15.18:0/1092903600 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1c2c0fd8c0 con 0x7f1c2c070c70 2024-04-12T21:38:57.232 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:57.231+0000 7f1c31600700 1 -- 172.21.15.18:0/1092903600 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1c2c11e450 con 0x7f1c2c070c70 2024-04-12T21:38:57.232 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:57.232+0000 7f1c31600700 1 -- 172.21.15.18:0/1092903600 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1c2c071aa0 con 0x7f1c2c070c70 2024-04-12T21:38:57.233 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:57.233+0000 7f1c23e00700 1 -- 172.21.15.18:0/1092903600 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7f1c1800bc10 con 0x7f1c2c070c70 2024-04-12T21:38:57.235 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:57.234+0000 7f1c23e00700 1 --2- 172.21.15.18:0/1092903600 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f1c1006ec10 0x7f1c100710c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:38:57.235 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:57.234+0000 7f1c23e00700 1 -- 172.21.15.18:0/1092903600 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 6757+0+0 (secure 0 0 0) 0x7f1c1808fc10 con 0x7f1c2c070c70 2024-04-12T21:38:57.235 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:57.234+0000 7f1c2be00700 1 --2- 172.21.15.18:0/1092903600 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f1c1006ec10 0x7f1c100710c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:38:57.236 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:57.235+0000 7f1c2be00700 1 --2- 172.21.15.18:0/1092903600 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f1c1006ec10 0x7f1c100710c0 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f1c1c007fd0 tx=0x7f1c1c009040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:38:57.240 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:57.239+0000 7f1c23e00700 1 -- 172.21.15.18:0/1092903600 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f1c180589c0 con 0x7f1c2c070c70 2024-04-12T21:38:57.354 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:38:57 smithi018 ceph-mon[112590]: pgmap v1178: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:38:57.354 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:38:57 smithi018 ceph-mon[112590]: from='client.36048 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:38:57.354 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:38:57 smithi018 ceph-mon[112590]: from='client.36052 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:38:57.533 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:38:57 smithi186 ceph-mon[86379]: pgmap v1178: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:38:57.533 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:38:57 smithi186 ceph-mon[86379]: from='client.36048 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:38:57.533 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:38:57 smithi186 ceph-mon[86379]: from='client.36052 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:38:57.588 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:57.587+0000 7f1c31600700 1 -- 172.21.15.18:0/1092903600 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f1c2c05f5f0 con 0x7f1c2c070c70 2024-04-12T21:38:57.590 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:57.589+0000 7f1c23e00700 1 -- 172.21.15.18:0/1092903600 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+623 (secure 0 0 0) 0x7f1c1805c080 con 0x7f1c2c070c70 2024-04-12T21:38:57.590 INFO:teuthology.orchestra.run.smithi018.stdout:{ 2024-04-12T21:38:57.590 INFO:teuthology.orchestra.run.smithi018.stdout: "mon": { 2024-04-12T21:38:57.590 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 19.0.0-2879-g324baf8d (324baf8ddb91dea46cac50cb535c306115791355) squid (dev)": 2 2024-04-12T21:38:57.590 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-12T21:38:57.590 INFO:teuthology.orchestra.run.smithi018.stdout: "mgr": { 2024-04-12T21:38:57.590 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 19.0.0-2879-g324baf8d (324baf8ddb91dea46cac50cb535c306115791355) squid (dev)": 2 2024-04-12T21:38:57.590 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-12T21:38:57.591 INFO:teuthology.orchestra.run.smithi018.stdout: "osd": { 2024-04-12T21:38:57.591 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 19.0.0-2879-g324baf8d (324baf8ddb91dea46cac50cb535c306115791355) squid (dev)": 6 2024-04-12T21:38:57.591 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-12T21:38:57.591 INFO:teuthology.orchestra.run.smithi018.stdout: "mds": { 2024-04-12T21:38:57.591 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 19.0.0-2879-g324baf8d (324baf8ddb91dea46cac50cb535c306115791355) squid (dev)": 4 2024-04-12T21:38:57.591 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-12T21:38:57.591 INFO:teuthology.orchestra.run.smithi018.stdout: "overall": { 2024-04-12T21:38:57.591 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 19.0.0-2879-g324baf8d (324baf8ddb91dea46cac50cb535c306115791355) squid (dev)": 14 2024-04-12T21:38:57.591 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-04-12T21:38:57.591 INFO:teuthology.orchestra.run.smithi018.stdout:} 2024-04-12T21:38:57.593 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:57.591+0000 7f1c31600700 1 -- 172.21.15.18:0/1092903600 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f1c1006ec10 msgr2=0x7f1c100710c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:38:57.593 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:57.592+0000 7f1c31600700 1 --2- 172.21.15.18:0/1092903600 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f1c1006ec10 0x7f1c100710c0 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f1c1c007fd0 tx=0x7f1c1c009040 comp rx=0 tx=0).stop 2024-04-12T21:38:57.593 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:57.592+0000 7f1c31600700 1 -- 172.21.15.18:0/1092903600 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1c2c070c70 msgr2=0x7f1c2c0fd100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:38:57.593 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:57.592+0000 7f1c31600700 1 --2- 172.21.15.18:0/1092903600 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1c2c070c70 0x7f1c2c0fd100 secure :-1 s=READY pgs=995 cs=0 l=1 rev1=1 crypto rx=0x7f1c18004f50 tx=0x7f1c18009e30 comp rx=0 tx=0).stop 2024-04-12T21:38:57.593 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:57.592+0000 7f1c31600700 1 -- 172.21.15.18:0/1092903600 shutdown_connections 2024-04-12T21:38:57.593 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:57.592+0000 7f1c31600700 1 --2- 172.21.15.18:0/1092903600 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f1c1006ec10 0x7f1c100710c0 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:38:57.593 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:57.592+0000 7f1c31600700 1 --2- 172.21.15.18:0/1092903600 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1c2c070360 0x7f1c2c0fcbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:38:57.593 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:57.592+0000 7f1c31600700 1 --2- 172.21.15.18:0/1092903600 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1c2c070c70 0x7f1c2c0fd100 unknown :-1 s=CLOSED pgs=995 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:38:57.593 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:57.592+0000 7f1c31600700 1 -- 172.21.15.18:0/1092903600 >> 172.21.15.18:0/1092903600 conn(0x7f1c2c0f87e0 msgr2=0x7f1c2c105810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:38:57.593 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:57.592+0000 7f1c31600700 1 -- 172.21.15.18:0/1092903600 shutdown_connections 2024-04-12T21:38:57.594 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:57.592+0000 7f1c31600700 1 -- 172.21.15.18:0/1092903600 wait complete. 2024-04-12T21:38:57.736 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:57.735+0000 7fe686a00700 1 -- 172.21.15.18:0/658924393 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe680105800 msgr2=0x7fe6800fee70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:38:57.736 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:57.735+0000 7fe686a00700 1 --2- 172.21.15.18:0/658924393 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe680105800 0x7fe6800fee70 secure :-1 s=READY pgs=996 cs=0 l=1 rev1=1 crypto rx=0x7fe66c004940 tx=0x7fe66c009e30 comp rx=0 tx=0).stop 2024-04-12T21:38:57.736 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:57.735+0000 7fe686a00700 1 -- 172.21.15.18:0/658924393 shutdown_connections 2024-04-12T21:38:57.736 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:57.735+0000 7fe686a00700 1 --2- 172.21.15.18:0/658924393 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe680105800 0x7fe6800fee70 unknown :-1 s=CLOSED pgs=996 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:38:57.737 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:57.735+0000 7fe686a00700 1 --2- 172.21.15.18:0/658924393 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe680104e60 0x7fe680105230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:38:57.737 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:57.735+0000 7fe686a00700 1 -- 172.21.15.18:0/658924393 >> 172.21.15.18:0/658924393 conn(0x7fe6800fab10 msgr2=0x7fe6800fcef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:38:57.737 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:57.735+0000 7fe686a00700 1 -- 172.21.15.18:0/658924393 shutdown_connections 2024-04-12T21:38:57.737 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:57.735+0000 7fe686a00700 1 -- 172.21.15.18:0/658924393 wait complete. 2024-04-12T21:38:57.738 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:57.736+0000 7fe686a00700 1 Processor -- start 2024-04-12T21:38:57.739 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:57.736+0000 7fe686a00700 1 -- start start 2024-04-12T21:38:57.739 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:57.736+0000 7fe686a00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe680104e60 0x7fe680115bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:38:57.739 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:57.736+0000 7fe686a00700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe680105800 0x7fe6801160f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:38:57.739 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:57.736+0000 7fe686a00700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe680116740 con 0x7fe680104e60 2024-04-12T21:38:57.739 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:57.736+0000 7fe686a00700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe68010fb50 con 0x7fe680105800 2024-04-12T21:38:57.739 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:57.737+0000 7fe685600700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe680104e60 0x7fe680115bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:38:57.739 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:57.737+0000 7fe685600700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe680104e60 0x7fe680115bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:60174/0 (socket says 172.21.15.18:60174) 2024-04-12T21:38:57.739 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:57.737+0000 7fe685600700 1 -- 172.21.15.18:0/136062797 learned_addr learned my addr 172.21.15.18:0/136062797 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:38:57.739 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:57.737+0000 7fe684c00700 1 --2- 172.21.15.18:0/136062797 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe680105800 0x7fe6801160f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:38:57.739 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:57.737+0000 7fe685600700 1 -- 172.21.15.18:0/136062797 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe680105800 msgr2=0x7fe6801160f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:38:57.739 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:57.737+0000 7fe685600700 1 --2- 172.21.15.18:0/136062797 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe680105800 0x7fe6801160f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:38:57.739 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:57.737+0000 7fe685600700 1 -- 172.21.15.18:0/136062797 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe66c005040 con 0x7fe680104e60 2024-04-12T21:38:57.739 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:57.737+0000 7fe684c00700 1 --2- 172.21.15.18:0/136062797 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe680105800 0x7fe6801160f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-12T21:38:57.740 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:57.738+0000 7fe685600700 1 --2- 172.21.15.18:0/136062797 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe680104e60 0x7fe680115bb0 secure :-1 s=READY pgs=997 cs=0 l=1 rev1=1 crypto rx=0x7fe67400c8c0 tx=0x7fe6740044c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:38:57.740 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:57.738+0000 7fe67d600700 1 -- 172.21.15.18:0/136062797 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe67400a980 con 0x7fe680104e60 2024-04-12T21:38:57.740 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:57.738+0000 7fe686a00700 1 -- 172.21.15.18:0/136062797 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe68010fe30 con 0x7fe680104e60 2024-04-12T21:38:57.740 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:57.738+0000 7fe67d600700 1 -- 172.21.15.18:0/136062797 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe67400ce60 con 0x7fe680104e60 2024-04-12T21:38:57.740 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:57.738+0000 7fe67d600700 1 -- 172.21.15.18:0/136062797 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe6740115e0 con 0x7fe680104e60 2024-04-12T21:38:57.740 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:57.738+0000 7fe686a00700 1 -- 172.21.15.18:0/136062797 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe6801103f0 con 0x7fe680104e60 2024-04-12T21:38:57.740 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:57.739+0000 7fe686a00700 1 -- 172.21.15.18:0/136062797 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe680102170 con 0x7fe680104e60 2024-04-12T21:38:57.741 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:57.740+0000 7fe67d600700 1 -- 172.21.15.18:0/136062797 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7fe674012400 con 0x7fe680104e60 2024-04-12T21:38:57.742 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:57.741+0000 7fe67d600700 1 --2- 172.21.15.18:0/136062797 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fe66806ec00 0x7fe6680710b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:38:57.742 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:57.741+0000 7fe67d600700 1 -- 172.21.15.18:0/136062797 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 6757+0+0 (secure 0 0 0) 0x7fe67408e980 con 0x7fe680104e60 2024-04-12T21:38:57.742 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:57.741+0000 7fe684c00700 1 --2- 172.21.15.18:0/136062797 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fe66806ec00 0x7fe6680710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:38:57.743 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:57.742+0000 7fe684c00700 1 --2- 172.21.15.18:0/136062797 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fe66806ec00 0x7fe6680710b0 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7fe66c00b3b0 tx=0x7fe66c009e30 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:38:57.746 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:57.746+0000 7fe67d600700 1 -- 172.21.15.18:0/136062797 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7fe674057740 con 0x7fe680104e60 2024-04-12T21:38:58.017 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:58.017+0000 7fe686a00700 1 -- 172.21.15.18:0/136062797 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fe6801005b0 con 0x7fe680104e60 2024-04-12T21:38:58.018 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:58.018+0000 7fe67d600700 1 -- 172.21.15.18:0/136062797 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 33 v33) v1 ==== 76+0+1876 (secure 0 0 0) 0x7fe6740126b0 con 0x7fe680104e60 2024-04-12T21:38:58.019 INFO:teuthology.orchestra.run.smithi018.stdout:e33 2024-04-12T21:38:58.019 INFO:teuthology.orchestra.run.smithi018.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-12T21:38:58.019 INFO:teuthology.orchestra.run.smithi018.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-12T21:38:58.019 INFO:teuthology.orchestra.run.smithi018.stdout:legacy client fscid: 1 2024-04-12T21:38:58.019 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-12T21:38:58.019 INFO:teuthology.orchestra.run.smithi018.stdout:Filesystem 'cephfs' (1) 2024-04-12T21:38:58.019 INFO:teuthology.orchestra.run.smithi018.stdout:fs_name cephfs 2024-04-12T21:38:58.019 INFO:teuthology.orchestra.run.smithi018.stdout:epoch 24 2024-04-12T21:38:58.019 INFO:teuthology.orchestra.run.smithi018.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-04-12T21:38:58.019 INFO:teuthology.orchestra.run.smithi018.stdout:created 2024-04-12T20:51:57.362850+0000 2024-04-12T21:38:58.020 INFO:teuthology.orchestra.run.smithi018.stdout:modified 2024-04-12T21:06:59.717190+0000 2024-04-12T21:38:58.020 INFO:teuthology.orchestra.run.smithi018.stdout:tableserver 0 2024-04-12T21:38:58.020 INFO:teuthology.orchestra.run.smithi018.stdout:root 0 2024-04-12T21:38:58.020 INFO:teuthology.orchestra.run.smithi018.stdout:session_timeout 60 2024-04-12T21:38:58.020 INFO:teuthology.orchestra.run.smithi018.stdout:session_autoclose 300 2024-04-12T21:38:58.020 INFO:teuthology.orchestra.run.smithi018.stdout:max_file_size 1099511627776 2024-04-12T21:38:58.020 INFO:teuthology.orchestra.run.smithi018.stdout:max_xattr_size 65536 2024-04-12T21:38:58.020 INFO:teuthology.orchestra.run.smithi018.stdout:required_client_features {} 2024-04-12T21:38:58.020 INFO:teuthology.orchestra.run.smithi018.stdout:last_failure 0 2024-04-12T21:38:58.020 INFO:teuthology.orchestra.run.smithi018.stdout:last_failure_osd_epoch 100 2024-04-12T21:38:58.020 INFO:teuthology.orchestra.run.smithi018.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-12T21:38:58.020 INFO:teuthology.orchestra.run.smithi018.stdout:max_mds 2 2024-04-12T21:38:58.020 INFO:teuthology.orchestra.run.smithi018.stdout:in 0,1 2024-04-12T21:38:58.020 INFO:teuthology.orchestra.run.smithi018.stdout:up {} 2024-04-12T21:38:58.020 INFO:teuthology.orchestra.run.smithi018.stdout:failed 0,1 2024-04-12T21:38:58.020 INFO:teuthology.orchestra.run.smithi018.stdout:damaged 2024-04-12T21:38:58.021 INFO:teuthology.orchestra.run.smithi018.stdout:stopped 2024-04-12T21:38:58.021 INFO:teuthology.orchestra.run.smithi018.stdout:data_pools [3] 2024-04-12T21:38:58.021 INFO:teuthology.orchestra.run.smithi018.stdout:metadata_pool 2 2024-04-12T21:38:58.021 INFO:teuthology.orchestra.run.smithi018.stdout:inline_data disabled 2024-04-12T21:38:58.021 INFO:teuthology.orchestra.run.smithi018.stdout:balancer 2024-04-12T21:38:58.021 INFO:teuthology.orchestra.run.smithi018.stdout:bal_rank_mask -1 2024-04-12T21:38:58.021 INFO:teuthology.orchestra.run.smithi018.stdout:standby_count_wanted 1 2024-04-12T21:38:58.021 INFO:teuthology.orchestra.run.smithi018.stdout:qdb_cluster leader: 0 members: 2024-04-12T21:38:58.021 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-12T21:38:58.021 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-12T21:38:58.021 INFO:teuthology.orchestra.run.smithi018.stdout:Standby daemons: 2024-04-12T21:38:58.021 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-12T21:38:58.021 INFO:teuthology.orchestra.run.smithi018.stdout:[mds.cephfs.smithi018.acqbsc{-1:34436} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.18:6828/755179764,v1:172.21.15.18:6829/755179764] compat {c=[1],r=[1],i=[1fff]}] 2024-04-12T21:38:58.021 INFO:teuthology.orchestra.run.smithi018.stdout:[mds.cephfs.smithi018.szjluo{-1:34468} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.18:6826/2407992653,v1:172.21.15.18:6827/2407992653] compat {c=[1],r=[1],i=[1fff]}] 2024-04-12T21:38:58.022 INFO:teuthology.orchestra.run.smithi018.stdout:[mds.cephfs.smithi186.jdllyn{-1:44595} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.186:6824/3523444202,v1:172.21.15.186:6825/3523444202] compat {c=[1],r=[1],i=[1fff]}] 2024-04-12T21:38:58.022 INFO:teuthology.orchestra.run.smithi018.stdout:[mds.cephfs.smithi186.wjnafs{-1:44875} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.186:6826/3122443766,v1:172.21.15.186:6827/3122443766] compat {c=[1],r=[1],i=[1fff]}] 2024-04-12T21:38:58.022 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:58.020+0000 7fe686a00700 1 -- 172.21.15.18:0/136062797 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fe66806ec00 msgr2=0x7fe6680710b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:38:58.022 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:58.020+0000 7fe686a00700 1 --2- 172.21.15.18:0/136062797 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fe66806ec00 0x7fe6680710b0 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7fe66c00b3b0 tx=0x7fe66c009e30 comp rx=0 tx=0).stop 2024-04-12T21:38:58.022 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:58.020+0000 7fe686a00700 1 -- 172.21.15.18:0/136062797 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe680104e60 msgr2=0x7fe680115bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:38:58.022 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:58.020+0000 7fe686a00700 1 --2- 172.21.15.18:0/136062797 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe680104e60 0x7fe680115bb0 secure :-1 s=READY pgs=997 cs=0 l=1 rev1=1 crypto rx=0x7fe67400c8c0 tx=0x7fe6740044c0 comp rx=0 tx=0).stop 2024-04-12T21:38:58.023 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:58.020+0000 7fe686a00700 1 -- 172.21.15.18:0/136062797 shutdown_connections 2024-04-12T21:38:58.023 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:58.020+0000 7fe686a00700 1 --2- 172.21.15.18:0/136062797 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fe66806ec00 0x7fe6680710b0 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:38:58.023 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:58.020+0000 7fe686a00700 1 --2- 172.21.15.18:0/136062797 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe680104e60 0x7fe680115bb0 unknown :-1 s=CLOSED pgs=997 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:38:58.023 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:58.020+0000 7fe686a00700 1 --2- 172.21.15.18:0/136062797 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe680105800 0x7fe6801160f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:38:58.023 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:58.020+0000 7fe686a00700 1 -- 172.21.15.18:0/136062797 >> 172.21.15.18:0/136062797 conn(0x7fe6800fab10 msgr2=0x7fe6800fd8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:38:58.023 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:58.021+0000 7fe686a00700 1 -- 172.21.15.18:0/136062797 shutdown_connections 2024-04-12T21:38:58.023 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:58.021+0000 7fe686a00700 1 -- 172.21.15.18:0/136062797 wait complete. 2024-04-12T21:38:58.023 INFO:teuthology.orchestra.run.smithi018.stderr:dumped fsmap epoch 33 2024-04-12T21:38:58.183 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:58.182+0000 7f0801000700 1 -- 172.21.15.18:0/3765967303 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f07fc0fe7b0 msgr2=0x7f07fc106df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:38:58.183 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:58.182+0000 7f0801000700 1 --2- 172.21.15.18:0/3765967303 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f07fc0fe7b0 0x7f07fc106df0 secure :-1 s=READY pgs=998 cs=0 l=1 rev1=1 crypto rx=0x7f07e8004940 tx=0x7f07e8009e30 comp rx=0 tx=0).stop 2024-04-12T21:38:58.183 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:58.182+0000 7f0801000700 1 -- 172.21.15.18:0/3765967303 shutdown_connections 2024-04-12T21:38:58.183 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:58.182+0000 7f0801000700 1 --2- 172.21.15.18:0/3765967303 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f07fc0fe7b0 0x7f07fc106df0 unknown :-1 s=CLOSED pgs=998 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:38:58.183 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:58.182+0000 7f0801000700 1 --2- 172.21.15.18:0/3765967303 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f07fc0fde10 0x7f07fc0fe1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:38:58.183 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:58.182+0000 7f0801000700 1 -- 172.21.15.18:0/3765967303 >> 172.21.15.18:0/3765967303 conn(0x7f07fc0f7860 msgr2=0x7f07fc0f9c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:38:58.183 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:58.182+0000 7f0801000700 1 -- 172.21.15.18:0/3765967303 shutdown_connections 2024-04-12T21:38:58.183 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:58.182+0000 7f0801000700 1 -- 172.21.15.18:0/3765967303 wait complete. 2024-04-12T21:38:58.184 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:58.183+0000 7f0801000700 1 Processor -- start 2024-04-12T21:38:58.184 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:58.183+0000 7f0801000700 1 -- start start 2024-04-12T21:38:58.184 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:58.183+0000 7f0801000700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f07fc0fde10 0x7f07fc113a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:38:58.185 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:58.183+0000 7f0801000700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f07fc0fe7b0 0x7f07fc113f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:38:58.185 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:58.183+0000 7f0801000700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07fc114590 con 0x7f07fc0fde10 2024-04-12T21:38:58.185 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:58.184+0000 7f0801000700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07fc10db90 con 0x7f07fc0fe7b0 2024-04-12T21:38:58.185 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:58.184+0000 7f07f9e00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f07fc0fde10 0x7f07fc113a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:38:58.185 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:58.184+0000 7f07f9e00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f07fc0fde10 0x7f07fc113a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:60186/0 (socket says 172.21.15.18:60186) 2024-04-12T21:38:58.185 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:58.184+0000 7f07f9e00700 1 -- 172.21.15.18:0/2774573478 learned_addr learned my addr 172.21.15.18:0/2774573478 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:38:58.186 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:58.184+0000 7f07f9400700 1 --2- 172.21.15.18:0/2774573478 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f07fc0fe7b0 0x7f07fc113f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:38:58.186 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:58.184+0000 7f07f9e00700 1 -- 172.21.15.18:0/2774573478 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f07fc0fe7b0 msgr2=0x7f07fc113f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:38:58.186 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:58.184+0000 7f07f9e00700 1 --2- 172.21.15.18:0/2774573478 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f07fc0fe7b0 0x7f07fc113f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:38:58.186 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:58.184+0000 7f07f9e00700 1 -- 172.21.15.18:0/2774573478 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07e8005040 con 0x7f07fc0fde10 2024-04-12T21:38:58.186 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:58.184+0000 7f07f9400700 1 --2- 172.21.15.18:0/2774573478 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f07fc0fe7b0 0x7f07fc113f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-12T21:38:58.186 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:58.185+0000 7f07f9e00700 1 --2- 172.21.15.18:0/2774573478 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f07fc0fde10 0x7f07fc113a00 secure :-1 s=READY pgs=999 cs=0 l=1 rev1=1 crypto rx=0x7f07ec002a50 tx=0x7f07ec0046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:38:58.186 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:58.185+0000 7f07f2000700 1 -- 172.21.15.18:0/2774573478 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f07ec007cc0 con 0x7f07fc0fde10 2024-04-12T21:38:58.186 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:58.185+0000 7f07f2000700 1 -- 172.21.15.18:0/2774573478 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f07ec00f040 con 0x7f07fc0fde10 2024-04-12T21:38:58.186 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:58.185+0000 7f0801000700 1 -- 172.21.15.18:0/2774573478 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f07fc10de70 con 0x7f07fc0fde10 2024-04-12T21:38:58.187 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:58.185+0000 7f07f2000700 1 -- 172.21.15.18:0/2774573478 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f07ec009640 con 0x7f07fc0fde10 2024-04-12T21:38:58.187 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:58.185+0000 7f0801000700 1 -- 172.21.15.18:0/2774573478 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f07fc10e3b0 con 0x7f07fc0fde10 2024-04-12T21:38:58.187 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:58.186+0000 7f0801000700 1 -- 172.21.15.18:0/2774573478 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f07fc066af0 con 0x7f07fc0fde10 2024-04-12T21:38:58.188 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:58.187+0000 7f07f2000700 1 -- 172.21.15.18:0/2774573478 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7f07ec0097a0 con 0x7f07fc0fde10 2024-04-12T21:38:58.189 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:58.188+0000 7f07f2000700 1 --2- 172.21.15.18:0/2774573478 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f07e006ec00 0x7f07e00710b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:38:58.189 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:58.188+0000 7f07f2000700 1 -- 172.21.15.18:0/2774573478 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 6757+0+0 (secure 0 0 0) 0x7f07ec08e760 con 0x7f07fc0fde10 2024-04-12T21:38:58.189 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:58.189+0000 7f07f9400700 1 --2- 172.21.15.18:0/2774573478 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f07e006ec00 0x7f07e00710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:38:58.190 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:58.189+0000 7f07f9400700 1 --2- 172.21.15.18:0/2774573478 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f07e006ec00 0x7f07e00710b0 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f07fc10f230 tx=0x7f07e8009e30 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:38:58.194 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:58.193+0000 7f07f2000700 1 -- 172.21.15.18:0/2774573478 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f07ec05af10 con 0x7f07fc0fde10 2024-04-12T21:38:58.354 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:38:58 smithi018 ceph-mon[112590]: from='client.36056 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:38:58.354 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:38:58 smithi018 ceph-mon[112590]: from='client.? 172.21.15.18:0/1092903600' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T21:38:58.354 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:38:58 smithi018 ceph-mon[112590]: from='client.? 172.21.15.18:0/136062797' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-12T21:38:58.460 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:58.459+0000 7f0801000700 1 -- 172.21.15.18:0/2774573478 --> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f07fc10f030 con 0x7f07e006ec00 2024-04-12T21:38:58.462 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:58.461+0000 7f07f2000700 1 -- 172.21.15.18:0/2774573478 <== mgr.34102 v2:172.21.15.18:6800/3812891919 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f07fc10f030 con 0x7f07e006ec00 2024-04-12T21:38:58.462 INFO:teuthology.orchestra.run.smithi018.stdout:{ 2024-04-12T21:38:58.462 INFO:teuthology.orchestra.run.smithi018.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:f4452def3be4d350f4a8e5feaf7ebb1511be9298fe68478ba7a2d6fbd7d69a63", 2024-04-12T21:38:58.462 INFO:teuthology.orchestra.run.smithi018.stdout: "in_progress": true, 2024-04-12T21:38:58.462 INFO:teuthology.orchestra.run.smithi018.stdout: "which": "Upgrading all daemon types on all hosts", 2024-04-12T21:38:58.462 INFO:teuthology.orchestra.run.smithi018.stdout: "services_complete": [ 2024-04-12T21:38:58.462 INFO:teuthology.orchestra.run.smithi018.stdout: "mgr", 2024-04-12T21:38:58.462 INFO:teuthology.orchestra.run.smithi018.stdout: "mds", 2024-04-12T21:38:58.462 INFO:teuthology.orchestra.run.smithi018.stdout: "crash", 2024-04-12T21:38:58.463 INFO:teuthology.orchestra.run.smithi018.stdout: "osd", 2024-04-12T21:38:58.463 INFO:teuthology.orchestra.run.smithi018.stdout: "mon" 2024-04-12T21:38:58.463 INFO:teuthology.orchestra.run.smithi018.stdout: ], 2024-04-12T21:38:58.463 INFO:teuthology.orchestra.run.smithi018.stdout: "progress": "16/21 daemons upgraded", 2024-04-12T21:38:58.463 INFO:teuthology.orchestra.run.smithi018.stdout: "message": "Currently upgrading mds daemons", 2024-04-12T21:38:58.463 INFO:teuthology.orchestra.run.smithi018.stdout: "is_paused": false 2024-04-12T21:38:58.463 INFO:teuthology.orchestra.run.smithi018.stdout:} 2024-04-12T21:38:58.465 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:58.464+0000 7f0801000700 1 -- 172.21.15.18:0/2774573478 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f07e006ec00 msgr2=0x7f07e00710b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:38:58.465 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:58.464+0000 7f0801000700 1 --2- 172.21.15.18:0/2774573478 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f07e006ec00 0x7f07e00710b0 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f07fc10f230 tx=0x7f07e8009e30 comp rx=0 tx=0).stop 2024-04-12T21:38:58.465 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:58.464+0000 7f0801000700 1 -- 172.21.15.18:0/2774573478 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f07fc0fde10 msgr2=0x7f07fc113a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:38:58.465 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:58.464+0000 7f0801000700 1 --2- 172.21.15.18:0/2774573478 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f07fc0fde10 0x7f07fc113a00 secure :-1 s=READY pgs=999 cs=0 l=1 rev1=1 crypto rx=0x7f07ec002a50 tx=0x7f07ec0046b0 comp rx=0 tx=0).stop 2024-04-12T21:38:58.465 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:58.464+0000 7f0801000700 1 -- 172.21.15.18:0/2774573478 shutdown_connections 2024-04-12T21:38:58.465 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:58.464+0000 7f0801000700 1 --2- 172.21.15.18:0/2774573478 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f07e006ec00 0x7f07e00710b0 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:38:58.466 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:58.464+0000 7f0801000700 1 --2- 172.21.15.18:0/2774573478 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f07fc0fde10 0x7f07fc113a00 unknown :-1 s=CLOSED pgs=999 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:38:58.466 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:58.464+0000 7f0801000700 1 --2- 172.21.15.18:0/2774573478 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f07fc0fe7b0 0x7f07fc113f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:38:58.466 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:58.464+0000 7f0801000700 1 -- 172.21.15.18:0/2774573478 >> 172.21.15.18:0/2774573478 conn(0x7f07fc0f7860 msgr2=0x7f07fc0f8dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:38:58.466 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:58.464+0000 7f0801000700 1 -- 172.21.15.18:0/2774573478 shutdown_connections 2024-04-12T21:38:58.466 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:58.464+0000 7f0801000700 1 -- 172.21.15.18:0/2774573478 wait complete. 2024-04-12T21:38:58.533 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:38:58 smithi186 ceph-mon[86379]: from='client.36056 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:38:58.533 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:38:58 smithi186 ceph-mon[86379]: from='client.? 172.21.15.18:0/1092903600' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T21:38:58.533 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:38:58 smithi186 ceph-mon[86379]: from='client.? 172.21.15.18:0/136062797' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-12T21:38:58.618 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:58.618+0000 7fb805000700 1 -- 172.21.15.18:0/2553940461 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb8000fb980 msgr2=0x7fb8000fbd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:38:58.619 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:58.618+0000 7fb805000700 1 --2- 172.21.15.18:0/2553940461 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb8000fb980 0x7fb8000fbd50 secure :-1 s=READY pgs=1000 cs=0 l=1 rev1=1 crypto rx=0x7fb7f0004970 tx=0x7fb7f0009e30 comp rx=0 tx=0).stop 2024-04-12T21:38:58.619 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:58.618+0000 7fb805000700 1 -- 172.21.15.18:0/2553940461 shutdown_connections 2024-04-12T21:38:58.619 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:58.618+0000 7fb805000700 1 --2- 172.21.15.18:0/2553940461 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb8000fc290 0x7fb800108ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:38:58.619 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:58.618+0000 7fb805000700 1 --2- 172.21.15.18:0/2553940461 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb8000fb980 0x7fb8000fbd50 unknown :-1 s=CLOSED pgs=1000 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:38:58.619 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:58.618+0000 7fb805000700 1 -- 172.21.15.18:0/2553940461 >> 172.21.15.18:0/2553940461 conn(0x7fb8000f7650 msgr2=0x7fb8000f9a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:38:58.619 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:58.618+0000 7fb805000700 1 -- 172.21.15.18:0/2553940461 shutdown_connections 2024-04-12T21:38:58.619 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:58.618+0000 7fb805000700 1 -- 172.21.15.18:0/2553940461 wait complete. 2024-04-12T21:38:58.620 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:58.619+0000 7fb805000700 1 Processor -- start 2024-04-12T21:38:58.620 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:58.619+0000 7fb805000700 1 -- start start 2024-04-12T21:38:58.620 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:58.619+0000 7fb805000700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb8000fb980 0x7fb800115b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:38:58.620 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:58.619+0000 7fb805000700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb8000fc290 0x7fb800116060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:38:58.621 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:58.619+0000 7fb805000700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8001166b0 con 0x7fb8000fb980 2024-04-12T21:38:58.621 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:58.619+0000 7fb805000700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb80010faa0 con 0x7fb8000fc290 2024-04-12T21:38:58.621 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:58.620+0000 7fb7ff400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb8000fb980 0x7fb800115b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:38:58.621 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:58.620+0000 7fb7ff400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb8000fb980 0x7fb800115b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:60208/0 (socket says 172.21.15.18:60208) 2024-04-12T21:38:58.621 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:58.620+0000 7fb7ff400700 1 -- 172.21.15.18:0/3460883451 learned_addr learned my addr 172.21.15.18:0/3460883451 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:38:58.622 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:58.620+0000 7fb7ff400700 1 -- 172.21.15.18:0/3460883451 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb8000fc290 msgr2=0x7fb800116060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:38:58.622 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:58.620+0000 7fb7ff400700 1 --2- 172.21.15.18:0/3460883451 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb8000fc290 0x7fb800116060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:38:58.622 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:58.620+0000 7fb7ff400700 1 -- 172.21.15.18:0/3460883451 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7f0005040 con 0x7fb8000fb980 2024-04-12T21:38:58.622 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:58.620+0000 7fb7ff400700 1 --2- 172.21.15.18:0/3460883451 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb8000fb980 0x7fb800115b20 secure :-1 s=READY pgs=1001 cs=0 l=1 rev1=1 crypto rx=0x7fb7f0004f50 tx=0x7fb7f0009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:38:58.622 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:58.620+0000 7fb7f7400700 1 -- 172.21.15.18:0/3460883451 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb7f000e040 con 0x7fb8000fb980 2024-04-12T21:38:58.622 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:58.621+0000 7fb7f7400700 1 -- 172.21.15.18:0/3460883451 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb7f0004290 con 0x7fb8000fb980 2024-04-12T21:38:58.622 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:58.621+0000 7fb805000700 1 -- 172.21.15.18:0/3460883451 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb80010fd20 con 0x7fb8000fb980 2024-04-12T21:38:58.622 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:58.621+0000 7fb7f7400700 1 -- 172.21.15.18:0/3460883451 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb7f001c3f0 con 0x7fb8000fb980 2024-04-12T21:38:58.622 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:58.621+0000 7fb805000700 1 -- 172.21.15.18:0/3460883451 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb8001102e0 con 0x7fb8000fb980 2024-04-12T21:38:58.623 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:58.622+0000 7fb805000700 1 -- 172.21.15.18:0/3460883451 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb800106e30 con 0x7fb8000fb980 2024-04-12T21:38:58.623 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:58.623+0000 7fb7f7400700 1 -- 172.21.15.18:0/3460883451 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7fb7f000bc10 con 0x7fb8000fb980 2024-04-12T21:38:58.624 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:58.624+0000 7fb7f7400700 1 --2- 172.21.15.18:0/3460883451 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fb7e406ec00 0x7fb7e40710b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:38:58.624 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:58.624+0000 7fb7f7400700 1 -- 172.21.15.18:0/3460883451 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 6757+0+0 (secure 0 0 0) 0x7fb7f008fcb0 con 0x7fb8000fb980 2024-04-12T21:38:58.625 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:58.624+0000 7fb7fea00700 1 --2- 172.21.15.18:0/3460883451 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fb7e406ec00 0x7fb7e40710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:38:58.625 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:58.625+0000 7fb7fea00700 1 --2- 172.21.15.18:0/3460883451 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fb7e406ec00 0x7fb7e40710b0 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7fb800110c60 tx=0x7fb7ec009ce0 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:38:58.629 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:58.628+0000 7fb7f7400700 1 -- 172.21.15.18:0/3460883451 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7fb7f005c510 con 0x7fb8000fb980 2024-04-12T21:38:58.938 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:58.937+0000 7fb805000700 1 -- 172.21.15.18:0/3460883451 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7fb800106e30 con 0x7fb8000fb980 2024-04-12T21:38:58.939 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:58.938+0000 7fb7f7400700 1 -- 172.21.15.18:0/3460883451 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+4303 (secure 0 0 0) 0x7fb7f005c0a0 con 0x7fb8000fb980 2024-04-12T21:38:58.939 INFO:teuthology.orchestra.run.smithi018.stdout:HEALTH_ERR failed to probe daemons or devices; 1 filesystem is degraded; 1 filesystem is offline 2024-04-12T21:38:58.940 INFO:teuthology.orchestra.run.smithi018.stdout:[WRN] CEPHADM_REFRESH_FAILED: failed to probe daemons or devices 2024-04-12T21:38:58.940 INFO:teuthology.orchestra.run.smithi018.stdout: host smithi186 `cephadm ceph-volume` failed: cephadm exited with an error code: 1, stderr: Inferring config /var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/mon.smithi186/config 2024-04-12T21:38:58.940 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/podman: timeout after 895 seconds 2024-04-12T21:38:58.940 INFO:teuthology.orchestra.run.smithi018.stdout:Non-zero exit code 124 from /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:f4452def3be4d350f4a8e5feaf7ebb1511be9298fe68478ba7a2d6fbd7d69a63 -e NODE_NAME=smithi186 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968:/var/run/ceph:z -v /var/log/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968:/var/log/ceph:z -v /var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmptgom53f9:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:f4452def3be4d350f4a8e5feaf7ebb1511be9298fe68478ba7a2d6fbd7d69a63 inventory --format=json-pretty --filter-for-batch 2024-04-12T21:38:58.940 INFO:teuthology.orchestra.run.smithi018.stdout:Traceback (most recent call last): 2024-04-12T21:38:58.940 INFO:teuthology.orchestra.run.smithi018.stdout: File "/usr/lib64/python3.9/runpy.py", line 197, in _run_module_as_main 2024-04-12T21:38:58.940 INFO:teuthology.orchestra.run.smithi018.stdout: return _run_code(code, main_globals, None, 2024-04-12T21:38:58.940 INFO:teuthology.orchestra.run.smithi018.stdout: File "/usr/lib64/python3.9/runpy.py", line 87, in _run_code 2024-04-12T21:38:58.940 INFO:teuthology.orchestra.run.smithi018.stdout: exec(code, run_globals) 2024-04-12T21:38:58.940 INFO:teuthology.orchestra.run.smithi018.stdout: File "/var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/cephadm.e44f3ed5e34e4ef52eedef233d1190bff004470d8e8005127b5312bff357fa7f/__main__.py", line 5618, in 2024-04-12T21:38:58.940 INFO:teuthology.orchestra.run.smithi018.stdout: File "/var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/cephadm.e44f3ed5e34e4ef52eedef233d1190bff004470d8e8005127b5312bff357fa7f/__main__.py", line 5606, in main 2024-04-12T21:38:58.940 INFO:teuthology.orchestra.run.smithi018.stdout: File "/var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/cephadm.e44f3ed5e34e4ef52eedef233d1190bff004470d8e8005127b5312bff357fa7f/__main__.py", line 408, in _infer_config 2024-04-12T21:38:58.940 INFO:teuthology.orchestra.run.smithi018.stdout: File "/var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/cephadm.e44f3ed5e34e4ef52eedef233d1190bff004470d8e8005127b5312bff357fa7f/__main__.py", line 323, in _infer_fsid 2024-04-12T21:38:58.940 INFO:teuthology.orchestra.run.smithi018.stdout: File "/var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/cephadm.e44f3ed5e34e4ef52eedef233d1190bff004470d8e8005127b5312bff357fa7f/__main__.py", line 436, in _infer_image 2024-04-12T21:38:58.940 INFO:teuthology.orchestra.run.smithi018.stdout: File "/var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/cephadm.e44f3ed5e34e4ef52eedef233d1190bff004470d8e8005127b5312bff357fa7f/__main__.py", line 310, in _validate_fsid 2024-04-12T21:38:58.940 INFO:teuthology.orchestra.run.smithi018.stdout: File "/var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/cephadm.e44f3ed5e34e4ef52eedef233d1190bff004470d8e8005127b5312bff357fa7f/__main__.py", line 3359, in command_ceph_volume 2024-04-12T21:38:58.941 INFO:teuthology.orchestra.run.smithi018.stdout: File "/var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/cephadm.e44f3ed5e34e4ef52eedef233d1190bff004470d8e8005127b5312bff357fa7f/cephadmlib/call_wrappers.py", line 310, in call_throws 2024-04-12T21:38:58.941 INFO:teuthology.orchestra.run.smithi018.stdout:RuntimeError: Failed command: /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:f4452def3be4d350f4a8e5feaf7ebb1511be9298fe68478ba7a2d6fbd7d69a63 -e NODE_NAME=smithi186 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968:/var/run/ceph:z -v /var/log/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968:/var/log/ceph:z -v /var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmptgom53f9:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:f4452def3be4d350f4a8e5feaf7ebb1511be9298fe68478ba7a2d6fbd7d69a63 inventory --format=json-pretty --filter-for-batch 2024-04-12T21:38:58.941 INFO:teuthology.orchestra.run.smithi018.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-04-12T21:38:58.941 INFO:teuthology.orchestra.run.smithi018.stdout: fs cephfs is degraded 2024-04-12T21:38:58.941 INFO:teuthology.orchestra.run.smithi018.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-04-12T21:38:58.941 INFO:teuthology.orchestra.run.smithi018.stdout: fs cephfs is offline because no MDS is active for it. 2024-04-12T21:38:58.942 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:58.941+0000 7fb805000700 1 -- 172.21.15.18:0/3460883451 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fb7e406ec00 msgr2=0x7fb7e40710b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:38:58.942 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:58.941+0000 7fb805000700 1 --2- 172.21.15.18:0/3460883451 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fb7e406ec00 0x7fb7e40710b0 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7fb800110c60 tx=0x7fb7ec009ce0 comp rx=0 tx=0).stop 2024-04-12T21:38:58.942 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:58.941+0000 7fb805000700 1 -- 172.21.15.18:0/3460883451 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb8000fb980 msgr2=0x7fb800115b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:38:58.942 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:58.941+0000 7fb805000700 1 --2- 172.21.15.18:0/3460883451 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb8000fb980 0x7fb800115b20 secure :-1 s=READY pgs=1001 cs=0 l=1 rev1=1 crypto rx=0x7fb7f0004f50 tx=0x7fb7f0009e30 comp rx=0 tx=0).stop 2024-04-12T21:38:58.942 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:58.941+0000 7fb805000700 1 -- 172.21.15.18:0/3460883451 shutdown_connections 2024-04-12T21:38:58.942 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:58.941+0000 7fb805000700 1 --2- 172.21.15.18:0/3460883451 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fb7e406ec00 0x7fb7e40710b0 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:38:58.943 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:58.941+0000 7fb805000700 1 --2- 172.21.15.18:0/3460883451 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb8000fb980 0x7fb800115b20 unknown :-1 s=CLOSED pgs=1001 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:38:58.943 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:58.941+0000 7fb805000700 1 --2- 172.21.15.18:0/3460883451 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb8000fc290 0x7fb800116060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:38:58.943 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:58.941+0000 7fb805000700 1 -- 172.21.15.18:0/3460883451 >> 172.21.15.18:0/3460883451 conn(0x7fb8000f7650 msgr2=0x7fb800103fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:38:58.943 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:58.942+0000 7fb805000700 1 -- 172.21.15.18:0/3460883451 shutdown_connections 2024-04-12T21:38:58.943 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:38:58.942+0000 7fb805000700 1 -- 172.21.15.18:0/3460883451 wait complete. 2024-04-12T21:38:59.533 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:38:59 smithi186 ceph-mon[86379]: pgmap v1179: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:38:59.533 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:38:59 smithi186 ceph-mon[86379]: from='client.36068 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:38:59.533 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:38:59 smithi186 ceph-mon[86379]: from='client.? 172.21.15.18:0/3460883451' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-12T21:38:59.604 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:38:59 smithi018 ceph-mon[112590]: pgmap v1179: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:38:59.604 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:38:59 smithi018 ceph-mon[112590]: from='client.36068 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:38:59.604 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:38:59 smithi018 ceph-mon[112590]: from='client.? 172.21.15.18:0/3460883451' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-12T21:39:01.533 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:39:01 smithi186 ceph-mon[86379]: pgmap v1180: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:39:01.604 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:39:01 smithi018 ceph-mon[112590]: pgmap v1180: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:39:03.533 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:39:03 smithi186 ceph-mon[86379]: pgmap v1181: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:39:03.604 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:39:03 smithi018 ceph-mon[112590]: pgmap v1181: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:39:05.533 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:39:05 smithi186 ceph-mon[86379]: pgmap v1182: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:39:05.604 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:39:05 smithi018 ceph-mon[112590]: pgmap v1182: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:39:07.783 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:39:07 smithi186 ceph-mon[86379]: pgmap v1183: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:39:07.854 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:39:07 smithi018 ceph-mon[112590]: pgmap v1183: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:39:09.783 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:39:09 smithi186 ceph-mon[86379]: pgmap v1184: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:39:09.854 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:39:09 smithi018 ceph-mon[112590]: pgmap v1184: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:39:10.783 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:39:10 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T21:39:10.854 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:39:10 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T21:39:11.783 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:39:11 smithi186 ceph-mon[86379]: pgmap v1185: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:39:11.854 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:39:11 smithi018 ceph-mon[112590]: pgmap v1185: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:39:13.783 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:39:13 smithi186 ceph-mon[86379]: pgmap v1186: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:39:13.854 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:39:13 smithi018 ceph-mon[112590]: pgmap v1186: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:39:15.656 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:39:15 smithi018 ceph-mon[112590]: pgmap v1187: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:39:15.783 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:39:15 smithi186 ceph-mon[86379]: pgmap v1187: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:39:17.783 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:39:17 smithi186 ceph-mon[86379]: pgmap v1188: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:39:17.854 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:39:17 smithi018 ceph-mon[112590]: pgmap v1188: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:39:19.783 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:39:19 smithi186 ceph-mon[86379]: pgmap v1189: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:39:19.854 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:39:19 smithi018 ceph-mon[112590]: pgmap v1189: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:39:21.783 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:39:21 smithi186 ceph-mon[86379]: pgmap v1190: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:39:21.854 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:39:21 smithi018 ceph-mon[112590]: pgmap v1190: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:39:23.783 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:39:23 smithi186 ceph-mon[86379]: pgmap v1191: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:39:23.854 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:39:23 smithi018 ceph-mon[112590]: pgmap v1191: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:39:25.783 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:39:25 smithi186 ceph-mon[86379]: pgmap v1192: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:39:25.783 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:39:25 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T21:39:25.854 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:39:25 smithi018 ceph-mon[112590]: pgmap v1192: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:39:25.854 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:39:25 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T21:39:27.783 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:39:27 smithi186 ceph-mon[86379]: pgmap v1193: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:39:27.854 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:39:27 smithi018 ceph-mon[112590]: pgmap v1193: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:39:29.116 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:39:29.116+0000 7f517fc00700 1 -- 172.21.15.18:0/2903779001 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5178109dd0 msgr2=0x7f5178103440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:39:29.116 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:39:29.116+0000 7f517fc00700 1 --2- 172.21.15.18:0/2903779001 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5178109dd0 0x7f5178103440 secure :-1 s=READY pgs=1002 cs=0 l=1 rev1=1 crypto rx=0x7f5164004940 tx=0x7f5164009e30 comp rx=0 tx=0).stop 2024-04-12T21:39:29.116 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:39:29.116+0000 7f517fc00700 1 -- 172.21.15.18:0/2903779001 shutdown_connections 2024-04-12T21:39:29.116 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:39:29.116+0000 7f517fc00700 1 --2- 172.21.15.18:0/2903779001 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5178109dd0 0x7f5178103440 unknown :-1 s=CLOSED pgs=1002 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:39:29.116 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:39:29.116+0000 7f517fc00700 1 --2- 172.21.15.18:0/2903779001 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5178109430 0x7f5178109800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:39:29.116 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:39:29.116+0000 7f517fc00700 1 -- 172.21.15.18:0/2903779001 >> 172.21.15.18:0/2903779001 conn(0x7f5178072c00 msgr2=0x7f5178075010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:39:29.116 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:39:29.116+0000 7f517fc00700 1 -- 172.21.15.18:0/2903779001 shutdown_connections 2024-04-12T21:39:29.116 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:39:29.116+0000 7f517fc00700 1 -- 172.21.15.18:0/2903779001 wait complete. 2024-04-12T21:39:29.117 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:39:29.117+0000 7f517fc00700 1 Processor -- start 2024-04-12T21:39:29.117 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:39:29.117+0000 7f517fc00700 1 -- start start 2024-04-12T21:39:29.117 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:39:29.117+0000 7f517fc00700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5178109430 0x7f5178119f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:39:29.117 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:39:29.117+0000 7f517fc00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5178109dd0 0x7f517811a4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:39:29.118 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:39:29.117+0000 7f517fc00700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f517811ab10 con 0x7f5178109dd0 2024-04-12T21:39:29.118 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:39:29.117+0000 7f517fc00700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51781140f0 con 0x7f5178109430 2024-04-12T21:39:29.118 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:39:29.118+0000 7f5177e00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5178109dd0 0x7f517811a4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:39:29.118 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:39:29.118+0000 7f5177e00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5178109dd0 0x7f517811a4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:40140/0 (socket says 172.21.15.18:40140) 2024-04-12T21:39:29.118 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:39:29.118+0000 7f5177e00700 1 -- 172.21.15.18:0/118760581 learned_addr learned my addr 172.21.15.18:0/118760581 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:39:29.119 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:39:29.118+0000 7f517d200700 1 --2- 172.21.15.18:0/118760581 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5178109430 0x7f5178119f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:39:29.119 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:39:29.118+0000 7f5177e00700 1 -- 172.21.15.18:0/118760581 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5178109430 msgr2=0x7f5178119f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:39:29.119 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:39:29.118+0000 7f5177e00700 1 --2- 172.21.15.18:0/118760581 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5178109430 0x7f5178119f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:39:29.119 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:39:29.118+0000 7f5177e00700 1 -- 172.21.15.18:0/118760581 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5164005040 con 0x7f5178109dd0 2024-04-12T21:39:29.119 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:39:29.119+0000 7f5177e00700 1 --2- 172.21.15.18:0/118760581 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5178109dd0 0x7f517811a4c0 secure :-1 s=READY pgs=1003 cs=0 l=1 rev1=1 crypto rx=0x7f5164009100 tx=0x7f5164009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:39:29.119 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:39:29.119+0000 7f5174c00700 1 -- 172.21.15.18:0/118760581 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f516400e040 con 0x7f5178109dd0 2024-04-12T21:39:29.120 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:39:29.119+0000 7f5174c00700 1 -- 172.21.15.18:0/118760581 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f5164013420 con 0x7f5178109dd0 2024-04-12T21:39:29.120 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:39:29.119+0000 7f517fc00700 1 -- 172.21.15.18:0/118760581 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5178114370 con 0x7f5178109dd0 2024-04-12T21:39:29.120 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:39:29.119+0000 7f5174c00700 1 -- 172.21.15.18:0/118760581 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f516401c630 con 0x7f5178109dd0 2024-04-12T21:39:29.120 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:39:29.119+0000 7f517fc00700 1 -- 172.21.15.18:0/118760581 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5178114930 con 0x7f5178109dd0 2024-04-12T21:39:29.120 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:39:29.120+0000 7f517fc00700 1 -- 172.21.15.18:0/118760581 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5178104b80 con 0x7f5178109dd0 2024-04-12T21:39:29.121 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:39:29.122+0000 7f5174c00700 1 -- 172.21.15.18:0/118760581 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7f5164013590 con 0x7f5178109dd0 2024-04-12T21:39:29.123 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:39:29.123+0000 7f5174c00700 1 --2- 172.21.15.18:0/118760581 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f516006ec00 0x7f51600710b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:39:29.123 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:39:29.124+0000 7f5174c00700 1 -- 172.21.15.18:0/118760581 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 6757+0+0 (secure 0 0 0) 0x7f5164090710 con 0x7f5178109dd0 2024-04-12T21:39:29.123 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:39:29.124+0000 7f517d200700 1 --2- 172.21.15.18:0/118760581 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f516006ec00 0x7f51600710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:39:29.124 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:39:29.124+0000 7f517d200700 1 --2- 172.21.15.18:0/118760581 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f516006ec00 0x7f51600710b0 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f516c004c40 tx=0x7f516c009040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:39:29.128 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:39:29.128+0000 7f5174c00700 1 -- 172.21.15.18:0/118760581 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f516405cf40 con 0x7f5178109dd0 2024-04-12T21:39:29.377 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:39:29.377+0000 7f517fc00700 1 -- 172.21.15.18:0/118760581 --> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f5178110500 con 0x7f516006ec00 2024-04-12T21:39:29.379 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:39:29.380+0000 7f5174c00700 1 -- 172.21.15.18:0/118760581 <== mgr.34102 v2:172.21.15.18:6800/3812891919 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f5178110500 con 0x7f516006ec00 2024-04-12T21:39:29.384 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:39:29.384+0000 7f517fc00700 1 -- 172.21.15.18:0/118760581 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f516006ec00 msgr2=0x7f51600710b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:39:29.384 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:39:29.384+0000 7f517fc00700 1 --2- 172.21.15.18:0/118760581 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f516006ec00 0x7f51600710b0 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f516c004c40 tx=0x7f516c009040 comp rx=0 tx=0).stop 2024-04-12T21:39:29.384 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:39:29.384+0000 7f517fc00700 1 -- 172.21.15.18:0/118760581 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5178109dd0 msgr2=0x7f517811a4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:39:29.384 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:39:29.384+0000 7f517fc00700 1 --2- 172.21.15.18:0/118760581 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5178109dd0 0x7f517811a4c0 secure :-1 s=READY pgs=1003 cs=0 l=1 rev1=1 crypto rx=0x7f5164009100 tx=0x7f5164009e30 comp rx=0 tx=0).stop 2024-04-12T21:39:29.384 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:39:29.384+0000 7f517fc00700 1 -- 172.21.15.18:0/118760581 shutdown_connections 2024-04-12T21:39:29.384 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:39:29.384+0000 7f517fc00700 1 --2- 172.21.15.18:0/118760581 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f516006ec00 0x7f51600710b0 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:39:29.384 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:39:29.384+0000 7f517fc00700 1 --2- 172.21.15.18:0/118760581 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5178109430 0x7f5178119f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:39:29.384 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:39:29.384+0000 7f517fc00700 1 --2- 172.21.15.18:0/118760581 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5178109dd0 0x7f517811a4c0 unknown :-1 s=CLOSED pgs=1003 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:39:29.384 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:39:29.384+0000 7f517fc00700 1 -- 172.21.15.18:0/118760581 >> 172.21.15.18:0/118760581 conn(0x7f5178072c00 msgr2=0x7f5178074c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:39:29.384 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:39:29.384+0000 7f517fc00700 1 -- 172.21.15.18:0/118760581 shutdown_connections 2024-04-12T21:39:29.384 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:39:29.384+0000 7f517fc00700 1 -- 172.21.15.18:0/118760581 wait complete. 2024-04-12T21:39:29.400 INFO:teuthology.orchestra.run.smithi018.stdout:true 2024-04-12T21:39:29.547 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:39:29.547+0000 7fdae9e00700 1 -- 172.21.15.18:0/2324297588 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fdae4105020 msgr2=0x7fdae41053f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:39:29.548 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:39:29.547+0000 7fdae9e00700 1 --2- 172.21.15.18:0/2324297588 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fdae4105020 0x7fdae41053f0 secure :-1 s=READY pgs=1004 cs=0 l=1 rev1=1 crypto rx=0x7fdad4004940 tx=0x7fdad4009e30 comp rx=0 tx=0).stop 2024-04-12T21:39:29.548 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:39:29.548+0000 7fdae9e00700 1 -- 172.21.15.18:0/2324297588 shutdown_connections 2024-04-12T21:39:29.548 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:39:29.548+0000 7fdae9e00700 1 --2- 172.21.15.18:0/2324297588 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdae41059c0 0x7fdae40ff080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:39:29.548 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:39:29.548+0000 7fdae9e00700 1 --2- 172.21.15.18:0/2324297588 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fdae4105020 0x7fdae41053f0 unknown :-1 s=CLOSED pgs=1004 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:39:29.548 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:39:29.548+0000 7fdae9e00700 1 -- 172.21.15.18:0/2324297588 >> 172.21.15.18:0/2324297588 conn(0x7fdae40faba0 msgr2=0x7fdae40fcfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:39:29.548 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:39:29.548+0000 7fdae9e00700 1 -- 172.21.15.18:0/2324297588 shutdown_connections 2024-04-12T21:39:29.548 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:39:29.548+0000 7fdae9e00700 1 -- 172.21.15.18:0/2324297588 wait complete. 2024-04-12T21:39:29.549 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:39:29.549+0000 7fdae9e00700 1 Processor -- start 2024-04-12T21:39:29.549 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:39:29.549+0000 7fdae9e00700 1 -- start start 2024-04-12T21:39:29.549 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:39:29.549+0000 7fdae9e00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fdae4105020 0x7fdae4115b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:39:29.549 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:39:29.549+0000 7fdae9e00700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdae41059c0 0x7fdae41160d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:39:29.549 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:39:29.549+0000 7fdae9e00700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdae4116720 con 0x7fdae4105020 2024-04-12T21:39:29.550 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:39:29.549+0000 7fdae9e00700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdae410fd00 con 0x7fdae41059c0 2024-04-12T21:39:29.550 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:39:29.550+0000 7fdae2a00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fdae4105020 0x7fdae4115b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:39:29.550 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:39:29.550+0000 7fdae2a00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fdae4105020 0x7fdae4115b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:40162/0 (socket says 172.21.15.18:40162) 2024-04-12T21:39:29.550 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:39:29.550+0000 7fdae2a00700 1 -- 172.21.15.18:0/1269297345 learned_addr learned my addr 172.21.15.18:0/1269297345 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:39:29.550 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:39:29.550+0000 7fdae2a00700 1 -- 172.21.15.18:0/1269297345 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdae41059c0 msgr2=0x7fdae41160d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:39:29.551 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:39:29.550+0000 7fdae2a00700 1 --2- 172.21.15.18:0/1269297345 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdae41059c0 0x7fdae41160d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:39:29.551 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:39:29.550+0000 7fdae2a00700 1 -- 172.21.15.18:0/1269297345 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdad4005040 con 0x7fdae4105020 2024-04-12T21:39:29.551 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:39:29.550+0000 7fdae2a00700 1 --2- 172.21.15.18:0/1269297345 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fdae4105020 0x7fdae4115b90 secure :-1 s=READY pgs=1005 cs=0 l=1 rev1=1 crypto rx=0x7fdad4009180 tx=0x7fdad4009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:39:29.551 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:39:29.551+0000 7fdadaa00700 1 -- 172.21.15.18:0/1269297345 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdad400e040 con 0x7fdae4105020 2024-04-12T21:39:29.551 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:39:29.551+0000 7fdadaa00700 1 -- 172.21.15.18:0/1269297345 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fdad400bbc0 con 0x7fdae4105020 2024-04-12T21:39:29.551 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:39:29.551+0000 7fdadaa00700 1 -- 172.21.15.18:0/1269297345 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdad401c630 con 0x7fdae4105020 2024-04-12T21:39:29.551 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:39:29.551+0000 7fdae9e00700 1 -- 172.21.15.18:0/1269297345 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdae410ff80 con 0x7fdae4105020 2024-04-12T21:39:29.551 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:39:29.551+0000 7fdae9e00700 1 -- 172.21.15.18:0/1269297345 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdae4110540 con 0x7fdae4105020 2024-04-12T21:39:29.551 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:39:29.552+0000 7fdae9e00700 1 -- 172.21.15.18:0/1269297345 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdae4100770 con 0x7fdae4105020 2024-04-12T21:39:29.553 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:39:29.553+0000 7fdadaa00700 1 -- 172.21.15.18:0/1269297345 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7fdad400bd30 con 0x7fdae4105020 2024-04-12T21:39:29.554 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:39:29.554+0000 7fdadaa00700 1 --2- 172.21.15.18:0/1269297345 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fdac806ec00 0x7fdac80710b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:39:29.554 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:39:29.554+0000 7fdadaa00700 1 -- 172.21.15.18:0/1269297345 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 6757+0+0 (secure 0 0 0) 0x7fdad408fd90 con 0x7fdae4105020 2024-04-12T21:39:29.554 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:39:29.555+0000 7fdae2000700 1 --2- 172.21.15.18:0/1269297345 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fdac806ec00 0x7fdac80710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:39:29.555 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:39:29.555+0000 7fdae2000700 1 --2- 172.21.15.18:0/1269297345 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fdac806ec00 0x7fdac80710b0 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7fdae4111430 tx=0x7fdad0007d40 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:39:29.559 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:39:29.559+0000 7fdadaa00700 1 -- 172.21.15.18:0/1269297345 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7fdad405c5c0 con 0x7fdae4105020 2024-04-12T21:39:29.783 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:39:29 smithi186 ceph-mon[86379]: pgmap v1194: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:39:29.816 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:39:29.815+0000 7fdae9e00700 1 -- 172.21.15.18:0/1269297345 --> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fdae410c190 con 0x7fdac806ec00 2024-04-12T21:39:29.816 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:39:29 smithi018 ceph-mon[112590]: pgmap v1194: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:39:29.817 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:39:29.817+0000 7fdadaa00700 1 -- 172.21.15.18:0/1269297345 <== mgr.34102 v2:172.21.15.18:6800/3812891919 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7fdae410c190 con 0x7fdac806ec00 2024-04-12T21:39:29.821 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:39:29.821+0000 7fdae9e00700 1 -- 172.21.15.18:0/1269297345 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fdac806ec00 msgr2=0x7fdac80710b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:39:29.822 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:39:29.821+0000 7fdae9e00700 1 --2- 172.21.15.18:0/1269297345 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fdac806ec00 0x7fdac80710b0 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7fdae4111430 tx=0x7fdad0007d40 comp rx=0 tx=0).stop 2024-04-12T21:39:29.822 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:39:29.821+0000 7fdae9e00700 1 -- 172.21.15.18:0/1269297345 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fdae4105020 msgr2=0x7fdae4115b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:39:29.822 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:39:29.821+0000 7fdae9e00700 1 --2- 172.21.15.18:0/1269297345 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fdae4105020 0x7fdae4115b90 secure :-1 s=READY pgs=1005 cs=0 l=1 rev1=1 crypto rx=0x7fdad4009180 tx=0x7fdad4009e30 comp rx=0 tx=0).stop 2024-04-12T21:39:29.822 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:39:29.822+0000 7fdae9e00700 1 -- 172.21.15.18:0/1269297345 shutdown_connections 2024-04-12T21:39:29.822 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:39:29.822+0000 7fdae9e00700 1 --2- 172.21.15.18:0/1269297345 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fdac806ec00 0x7fdac80710b0 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:39:29.822 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:39:29.822+0000 7fdae9e00700 1 --2- 172.21.15.18:0/1269297345 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fdae4105020 0x7fdae4115b90 unknown :-1 s=CLOSED pgs=1005 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:39:29.822 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:39:29.822+0000 7fdae9e00700 1 --2- 172.21.15.18:0/1269297345 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdae41059c0 0x7fdae41160d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:39:29.822 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:39:29.822+0000 7fdae9e00700 1 -- 172.21.15.18:0/1269297345 >> 172.21.15.18:0/1269297345 conn(0x7fdae40faba0 msgr2=0x7fdae40fbb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:39:29.822 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:39:29.822+0000 7fdae9e00700 1 -- 172.21.15.18:0/1269297345 shutdown_connections 2024-04-12T21:39:29.822 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:39:29.822+0000 7fdae9e00700 1 -- 172.21.15.18:0/1269297345 wait complete. 2024-04-12T21:39:29.984 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:39:29.984+0000 7f39b6600700 1 -- 172.21.15.18:0/2667034698 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f39b00724f0 msgr2=0x7f39b00728c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:39:29.984 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:39:29.984+0000 7f39b6600700 1 --2- 172.21.15.18:0/2667034698 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f39b00724f0 0x7f39b00728c0 secure :-1 s=READY pgs=1006 cs=0 l=1 rev1=1 crypto rx=0x7f39a0009ce0 tx=0x7f39a0004d10 comp rx=0 tx=0).stop 2024-04-12T21:39:29.984 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:39:29.984+0000 7f39b6600700 1 -- 172.21.15.18:0/2667034698 shutdown_connections 2024-04-12T21:39:29.984 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:39:29.984+0000 7f39b6600700 1 --2- 172.21.15.18:0/2667034698 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f39b0072e00 0x7f39b010d760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:39:29.984 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:39:29.984+0000 7f39b6600700 1 --2- 172.21.15.18:0/2667034698 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f39b00724f0 0x7f39b00728c0 unknown :-1 s=CLOSED pgs=1006 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:39:29.984 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:39:29.984+0000 7f39b6600700 1 -- 172.21.15.18:0/2667034698 >> 172.21.15.18:0/2667034698 conn(0x7f39b00faba0 msgr2=0x7f39b00fcfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:39:29.984 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:39:29.984+0000 7f39b6600700 1 -- 172.21.15.18:0/2667034698 shutdown_connections 2024-04-12T21:39:29.984 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:39:29.984+0000 7f39b6600700 1 -- 172.21.15.18:0/2667034698 wait complete. 2024-04-12T21:39:29.986 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:39:29.985+0000 7f39b6600700 1 Processor -- start 2024-04-12T21:39:29.986 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:39:29.985+0000 7f39b6600700 1 -- start start 2024-04-12T21:39:29.986 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:39:29.985+0000 7f39b6600700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f39b00724f0 0x7f39b0119f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:39:29.986 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:39:29.985+0000 7f39b6600700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f39b0072e00 0x7f39b011a440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:39:29.986 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:39:29.985+0000 7f39b6600700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39b011aa90 con 0x7f39b0072e00 2024-04-12T21:39:29.986 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:39:29.985+0000 7f39b6600700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39b0114070 con 0x7f39b00724f0 2024-04-12T21:39:29.986 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:39:29.986+0000 7f39a6a00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f39b0072e00 0x7f39b011a440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:39:29.986 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:39:29.986+0000 7f39a6a00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f39b0072e00 0x7f39b011a440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:40188/0 (socket says 172.21.15.18:40188) 2024-04-12T21:39:29.987 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:39:29.986+0000 7f39a6a00700 1 -- 172.21.15.18:0/44683075 learned_addr learned my addr 172.21.15.18:0/44683075 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:39:29.987 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:39:29.986+0000 7f39af400700 1 --2- 172.21.15.18:0/44683075 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f39b00724f0 0x7f39b0119f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:39:29.987 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:39:29.986+0000 7f39a6a00700 1 -- 172.21.15.18:0/44683075 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f39b00724f0 msgr2=0x7f39b0119f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:39:29.987 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:39:29.986+0000 7f39a6a00700 1 --2- 172.21.15.18:0/44683075 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f39b00724f0 0x7f39b0119f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:39:29.987 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:39:29.986+0000 7f39a6a00700 1 -- 172.21.15.18:0/44683075 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39a0009940 con 0x7f39b0072e00 2024-04-12T21:39:29.987 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:39:29.986+0000 7f39af400700 1 --2- 172.21.15.18:0/44683075 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f39b00724f0 0x7f39b0119f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-12T21:39:29.988 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:39:29.987+0000 7f39a6a00700 1 --2- 172.21.15.18:0/44683075 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f39b0072e00 0x7f39b011a440 secure :-1 s=READY pgs=1007 cs=0 l=1 rev1=1 crypto rx=0x7f399800c8c0 tx=0x7f39980044c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:39:29.988 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:39:29.987+0000 7f39a7e00700 1 -- 172.21.15.18:0/44683075 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f399800a980 con 0x7f39b0072e00 2024-04-12T21:39:29.988 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:39:29.987+0000 7f39a7e00700 1 -- 172.21.15.18:0/44683075 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f399800ce60 con 0x7f39b0072e00 2024-04-12T21:39:29.988 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:39:29.987+0000 7f39b6600700 1 -- 172.21.15.18:0/44683075 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f39b0114350 con 0x7f39b0072e00 2024-04-12T21:39:29.988 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:39:29.987+0000 7f39a7e00700 1 -- 172.21.15.18:0/44683075 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f39980115e0 con 0x7f39b0072e00 2024-04-12T21:39:29.988 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:39:29.987+0000 7f39b6600700 1 -- 172.21.15.18:0/44683075 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f39b0114910 con 0x7f39b0072e00 2024-04-12T21:39:29.988 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:39:29.988+0000 7f39b6600700 1 -- 172.21.15.18:0/44683075 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f39b010a5a0 con 0x7f39b0072e00 2024-04-12T21:39:29.989 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:39:29.989+0000 7f39a7e00700 1 -- 172.21.15.18:0/44683075 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7f3998012400 con 0x7f39b0072e00 2024-04-12T21:39:29.990 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:39:29.990+0000 7f39a7e00700 1 --2- 172.21.15.18:0/44683075 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f399406ec00 0x7f39940710b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:39:29.990 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:39:29.990+0000 7f39a7e00700 1 -- 172.21.15.18:0/44683075 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 6757+0+0 (secure 0 0 0) 0x7f399808ea40 con 0x7f39b0072e00 2024-04-12T21:39:29.990 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:39:29.991+0000 7f39af400700 1 --2- 172.21.15.18:0/44683075 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f399406ec00 0x7f39940710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:39:29.991 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:39:29.991+0000 7f39af400700 1 --2- 172.21.15.18:0/44683075 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f399406ec00 0x7f39940710b0 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f39a000b740 tx=0x7f39a0004d10 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:39:29.995 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:39:29.995+0000 7f39a7e00700 1 -- 172.21.15.18:0/44683075 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f399805b270 con 0x7f39b0072e00 2024-04-12T21:39:30.239 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:39:30.239+0000 7f39b6600700 1 -- 172.21.15.18:0/44683075 --> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f39b01155c0 con 0x7f399406ec00 2024-04-12T21:39:30.255 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:39:30.255+0000 7f39a7e00700 1 -- 172.21.15.18:0/44683075 <== mgr.34102 v2:172.21.15.18:6800/3812891919 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7f39b01155c0 con 0x7f399406ec00 2024-04-12T21:39:30.255 INFO:teuthology.orchestra.run.smithi018.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-12T21:39:30.255 INFO:teuthology.orchestra.run.smithi018.stdout:alertmanager.smithi018 smithi018 *:9093,9094 running (40m) 70s ago 54m 14.8M - 0.25.0 c8568f914cd2 c332090068e8 2024-04-12T21:39:30.255 INFO:teuthology.orchestra.run.smithi018.stdout:crash.smithi018 smithi018 running (36m) 70s ago 54m 6665k - 19.0.0-2879-g324baf8d 8717720dc5e5 b7f80d1f7982 2024-04-12T21:39:30.255 INFO:teuthology.orchestra.run.smithi018.stdout:crash.smithi186 smithi186 running (36m) 73s ago 53m 6660k - 19.0.0-2879-g324baf8d 8717720dc5e5 a4f55c81b623 2024-04-12T21:39:30.255 INFO:teuthology.orchestra.run.smithi018.stdout:grafana.smithi018 smithi018 *:3000 running (39m) 70s ago 54m 73.8M - 9.4.12 2bacad6d85d8 b08e3bf176dd 2024-04-12T21:39:30.256 INFO:teuthology.orchestra.run.smithi018.stdout:mds.cephfs.smithi018.acqbsc smithi018 running (32m) 70s ago 47m 21.9M - 19.0.0-2879-g324baf8d 8717720dc5e5 021f99ab24ae 2024-04-12T21:39:30.256 INFO:teuthology.orchestra.run.smithi018.stdout:mds.cephfs.smithi018.szjluo smithi018 running (31m) 70s ago 47m 21.7M - 19.0.0-2879-g324baf8d 8717720dc5e5 0003117479d4 2024-04-12T21:39:30.256 INFO:teuthology.orchestra.run.smithi018.stdout:mds.cephfs.smithi186.jdllyn smithi186 running (16m) 73s ago 47m 16.5M - 19.0.0-2879-g324baf8d 8717720dc5e5 21b1f46eef23 2024-04-12T21:39:30.256 INFO:teuthology.orchestra.run.smithi018.stdout:mds.cephfs.smithi186.wjnafs smithi186 running (78s) 73s ago 47m 11.0M - 19.0.0-2879-g324baf8d 8717720dc5e5 3cf1ef4148e2 2024-04-12T21:39:30.256 INFO:teuthology.orchestra.run.smithi018.stdout:mgr.smithi018.xjzwga smithi018 *:8443,9283,8765 running (42m) 70s ago 56m 517M - 19.0.0-2879-g324baf8d 8717720dc5e5 56088305eab7 2024-04-12T21:39:30.256 INFO:teuthology.orchestra.run.smithi018.stdout:mgr.smithi186.kqcfxk smithi186 *:8443,9283,8765 running (41m) 73s ago 53m 460M - 19.0.0-2879-g324baf8d 8717720dc5e5 09cd766e5e0d 2024-04-12T21:39:30.257 INFO:teuthology.orchestra.run.smithi018.stdout:mon.smithi018 smithi018 running (38m) 70s ago 56m 88.2M 2048M 19.0.0-2879-g324baf8d 8717720dc5e5 df3e201dfed6 2024-04-12T21:39:30.257 INFO:teuthology.orchestra.run.smithi018.stdout:mon.smithi186 smithi186 running (37m) 73s ago 53m 53.6M 2048M 19.0.0-2879-g324baf8d 8717720dc5e5 c77498da9b2b 2024-04-12T21:39:30.257 INFO:teuthology.orchestra.run.smithi018.stdout:node-exporter.smithi018 smithi018 *:9100 running (41m) 70s ago 54m 8468k - 1.5.0 0da6a335fe13 326c3d9c6847 2024-04-12T21:39:30.257 INFO:teuthology.orchestra.run.smithi018.stdout:node-exporter.smithi186 smithi186 *:9100 running (41m) 73s ago 53m 8166k - 1.5.0 0da6a335fe13 ec2112baeecf 2024-04-12T21:39:30.257 INFO:teuthology.orchestra.run.smithi018.stdout:osd.0 smithi018 running (35m) 70s ago 52m 123M 1559M 19.0.0-2879-g324baf8d 8717720dc5e5 60b1273ca047 2024-04-12T21:39:30.257 INFO:teuthology.orchestra.run.smithi018.stdout:osd.1 smithi018 running (35m) 70s ago 51m 95.4M 1559M 19.0.0-2879-g324baf8d 8717720dc5e5 170ce1e5fb8d 2024-04-12T21:39:30.257 INFO:teuthology.orchestra.run.smithi018.stdout:osd.2 smithi018 running (34m) 70s ago 51m 94.6M 1559M 19.0.0-2879-g324baf8d 8717720dc5e5 c2cbc6aa2839 2024-04-12T21:39:30.257 INFO:teuthology.orchestra.run.smithi018.stdout:osd.3 smithi186 running (34m) 73s ago 50m 101M 2581M 19.0.0-2879-g324baf8d 8717720dc5e5 0138e8262e4b 2024-04-12T21:39:30.258 INFO:teuthology.orchestra.run.smithi018.stdout:osd.4 smithi186 running (33m) 73s ago 49m 91.5M 2581M 19.0.0-2879-g324baf8d 8717720dc5e5 b20cb63b25c9 2024-04-12T21:39:30.258 INFO:teuthology.orchestra.run.smithi018.stdout:osd.5 smithi186 running (32m) 73s ago 49m 92.2M 2581M 19.0.0-2879-g324baf8d 8717720dc5e5 2513bea2610e 2024-04-12T21:39:30.258 INFO:teuthology.orchestra.run.smithi018.stdout:prometheus.smithi018 smithi018 *:9095 running (40m) 70s ago 53m 69.6M - 2.43.0 a07b618ecd1d f58841127e21 2024-04-12T21:39:30.259 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:39:30.257+0000 7f39b6600700 1 -- 172.21.15.18:0/44683075 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f399406ec00 msgr2=0x7f39940710b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:39:30.259 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:39:30.257+0000 7f39b6600700 1 --2- 172.21.15.18:0/44683075 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f399406ec00 0x7f39940710b0 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f39a000b740 tx=0x7f39a0004d10 comp rx=0 tx=0).stop 2024-04-12T21:39:30.259 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:39:30.257+0000 7f39b6600700 1 -- 172.21.15.18:0/44683075 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f39b0072e00 msgr2=0x7f39b011a440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:39:30.259 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:39:30.257+0000 7f39b6600700 1 --2- 172.21.15.18:0/44683075 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f39b0072e00 0x7f39b011a440 secure :-1 s=READY pgs=1007 cs=0 l=1 rev1=1 crypto rx=0x7f399800c8c0 tx=0x7f39980044c0 comp rx=0 tx=0).stop 2024-04-12T21:39:30.260 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:39:30.258+0000 7f39b6600700 1 -- 172.21.15.18:0/44683075 shutdown_connections 2024-04-12T21:39:30.260 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:39:30.258+0000 7f39b6600700 1 --2- 172.21.15.18:0/44683075 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f399406ec00 0x7f39940710b0 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:39:30.260 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:39:30.258+0000 7f39b6600700 1 --2- 172.21.15.18:0/44683075 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f39b00724f0 0x7f39b0119f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:39:30.260 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:39:30.258+0000 7f39b6600700 1 --2- 172.21.15.18:0/44683075 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f39b0072e00 0x7f39b011a440 secure :-1 s=CLOSED pgs=1007 cs=0 l=1 rev1=1 crypto rx=0x7f399800c8c0 tx=0x7f39980044c0 comp rx=0 tx=0).stop 2024-04-12T21:39:30.260 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:39:30.258+0000 7f39b6600700 1 -- 172.21.15.18:0/44683075 >> 172.21.15.18:0/44683075 conn(0x7f39b00faba0 msgr2=0x7f39b0107720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:39:30.260 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:39:30.258+0000 7f39b6600700 1 -- 172.21.15.18:0/44683075 shutdown_connections 2024-04-12T21:39:30.260 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:39:30.258+0000 7f39b6600700 1 -- 172.21.15.18:0/44683075 wait complete. 2024-04-12T21:39:30.401 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:39:30.401+0000 7f17cf200700 1 -- 172.21.15.18:0/3992435978 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f17c8109dd0 msgr2=0x7f17c8103440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:39:30.401 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:39:30.401+0000 7f17cf200700 1 --2- 172.21.15.18:0/3992435978 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f17c8109dd0 0x7f17c8103440 secure :-1 s=READY pgs=1008 cs=0 l=1 rev1=1 crypto rx=0x7f17b4004940 tx=0x7f17b4009e30 comp rx=0 tx=0).stop 2024-04-12T21:39:30.401 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:39:30.401+0000 7f17cf200700 1 -- 172.21.15.18:0/3992435978 shutdown_connections 2024-04-12T21:39:30.401 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:39:30.401+0000 7f17cf200700 1 --2- 172.21.15.18:0/3992435978 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f17c8109dd0 0x7f17c8103440 unknown :-1 s=CLOSED pgs=1008 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:39:30.401 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:39:30.401+0000 7f17cf200700 1 --2- 172.21.15.18:0/3992435978 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f17c8109430 0x7f17c8109800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:39:30.402 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:39:30.401+0000 7f17cf200700 1 -- 172.21.15.18:0/3992435978 >> 172.21.15.18:0/3992435978 conn(0x7f17c8072c00 msgr2=0x7f17c8075010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:39:30.402 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:39:30.402+0000 7f17cf200700 1 -- 172.21.15.18:0/3992435978 shutdown_connections 2024-04-12T21:39:30.402 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:39:30.402+0000 7f17cf200700 1 -- 172.21.15.18:0/3992435978 wait complete. 2024-04-12T21:39:30.402 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:39:30.403+0000 7f17cf200700 1 Processor -- start 2024-04-12T21:39:30.403 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:39:30.403+0000 7f17cf200700 1 -- start start 2024-04-12T21:39:30.405 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:39:30.404+0000 7f17cf200700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f17c8109430 0x7f17c811a140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:39:30.405 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:39:30.404+0000 7f17cf200700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f17c811a680 0x7f17c81142b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:39:30.405 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:39:30.404+0000 7f17cf200700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17c811ac90 con 0x7f17c8109430 2024-04-12T21:39:30.405 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:39:30.404+0000 7f17cf200700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17c81147f0 con 0x7f17c811a680 2024-04-12T21:39:30.405 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:39:30.404+0000 7f17c7e00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f17c8109430 0x7f17c811a140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:39:30.405 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:39:30.404+0000 7f17c7e00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f17c8109430 0x7f17c811a140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:40206/0 (socket says 172.21.15.18:40206) 2024-04-12T21:39:30.405 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:39:30.404+0000 7f17c7e00700 1 -- 172.21.15.18:0/2599518936 learned_addr learned my addr 172.21.15.18:0/2599518936 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:39:30.405 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:39:30.404+0000 7f17c7400700 1 --2- 172.21.15.18:0/2599518936 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f17c811a680 0x7f17c81142b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:39:30.405 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:39:30.404+0000 7f17c7e00700 1 -- 172.21.15.18:0/2599518936 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f17c811a680 msgr2=0x7f17c81142b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:39:30.406 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:39:30.404+0000 7f17c7e00700 1 --2- 172.21.15.18:0/2599518936 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f17c811a680 0x7f17c81142b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:39:30.406 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:39:30.404+0000 7f17c7e00700 1 -- 172.21.15.18:0/2599518936 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17b4005040 con 0x7f17c8109430 2024-04-12T21:39:30.406 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:39:30.404+0000 7f17c7400700 1 --2- 172.21.15.18:0/2599518936 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f17c811a680 0x7f17c81142b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-12T21:39:30.406 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:39:30.405+0000 7f17c7e00700 1 --2- 172.21.15.18:0/2599518936 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f17c8109430 0x7f17c811a140 secure :-1 s=READY pgs=1009 cs=0 l=1 rev1=1 crypto rx=0x7f17b800b790 tx=0x7f17b8009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:39:30.406 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:39:30.405+0000 7f17bfe00700 1 -- 172.21.15.18:0/2599518936 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f17b800f770 con 0x7f17c8109430 2024-04-12T21:39:30.406 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:39:30.405+0000 7f17bfe00700 1 -- 172.21.15.18:0/2599518936 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f17b800fda0 con 0x7f17c8109430 2024-04-12T21:39:30.406 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:39:30.405+0000 7f17cf200700 1 -- 172.21.15.18:0/2599518936 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f17c8114ad0 con 0x7f17c8109430 2024-04-12T21:39:30.406 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:39:30.405+0000 7f17bfe00700 1 -- 172.21.15.18:0/2599518936 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f17b800d310 con 0x7f17c8109430 2024-04-12T21:39:30.406 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:39:30.405+0000 7f17cf200700 1 -- 172.21.15.18:0/2599518936 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f17c8115090 con 0x7f17c8109430 2024-04-12T21:39:30.407 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:39:30.406+0000 7f17cf200700 1 -- 172.21.15.18:0/2599518936 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f17c8104b80 con 0x7f17c8109430 2024-04-12T21:39:30.407 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:39:30.407+0000 7f17bfe00700 1 -- 172.21.15.18:0/2599518936 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7f17b8010420 con 0x7f17c8109430 2024-04-12T21:39:30.408 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:39:30.408+0000 7f17bfe00700 1 --2- 172.21.15.18:0/2599518936 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f17ac06ec00 0x7f17ac0710b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:39:30.408 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:39:30.408+0000 7f17bfe00700 1 -- 172.21.15.18:0/2599518936 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 6757+0+0 (secure 0 0 0) 0x7f17b808e9a0 con 0x7f17c8109430 2024-04-12T21:39:30.408 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:39:30.409+0000 7f17c7400700 1 --2- 172.21.15.18:0/2599518936 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f17ac06ec00 0x7f17ac0710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:39:30.409 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:39:30.409+0000 7f17c7400700 1 --2- 172.21.15.18:0/2599518936 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f17ac06ec00 0x7f17ac0710b0 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f17b4009040 tx=0x7f17b4009e30 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:39:30.413 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:39:30.413+0000 7f17bfe00700 1 -- 172.21.15.18:0/2599518936 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f17b805b1d0 con 0x7f17c8109430 2024-04-12T21:39:30.743 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:39:30.743+0000 7f17cf200700 1 -- 172.21.15.18:0/2599518936 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f17c8110580 con 0x7f17c8109430 2024-04-12T21:39:30.744 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:39:30.744+0000 7f17bfe00700 1 -- 172.21.15.18:0/2599518936 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+623 (secure 0 0 0) 0x7f17b805ad60 con 0x7f17c8109430 2024-04-12T21:39:30.744 INFO:teuthology.orchestra.run.smithi018.stdout:{ 2024-04-12T21:39:30.744 INFO:teuthology.orchestra.run.smithi018.stdout: "mon": { 2024-04-12T21:39:30.744 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 19.0.0-2879-g324baf8d (324baf8ddb91dea46cac50cb535c306115791355) squid (dev)": 2 2024-04-12T21:39:30.744 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-12T21:39:30.744 INFO:teuthology.orchestra.run.smithi018.stdout: "mgr": { 2024-04-12T21:39:30.744 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 19.0.0-2879-g324baf8d (324baf8ddb91dea46cac50cb535c306115791355) squid (dev)": 2 2024-04-12T21:39:30.745 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-12T21:39:30.745 INFO:teuthology.orchestra.run.smithi018.stdout: "osd": { 2024-04-12T21:39:30.745 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 19.0.0-2879-g324baf8d (324baf8ddb91dea46cac50cb535c306115791355) squid (dev)": 6 2024-04-12T21:39:30.745 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-12T21:39:30.745 INFO:teuthology.orchestra.run.smithi018.stdout: "mds": { 2024-04-12T21:39:30.745 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 19.0.0-2879-g324baf8d (324baf8ddb91dea46cac50cb535c306115791355) squid (dev)": 4 2024-04-12T21:39:30.745 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-12T21:39:30.745 INFO:teuthology.orchestra.run.smithi018.stdout: "overall": { 2024-04-12T21:39:30.745 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 19.0.0-2879-g324baf8d (324baf8ddb91dea46cac50cb535c306115791355) squid (dev)": 14 2024-04-12T21:39:30.745 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-04-12T21:39:30.745 INFO:teuthology.orchestra.run.smithi018.stdout:} 2024-04-12T21:39:30.747 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:39:30.747+0000 7f17cf200700 1 -- 172.21.15.18:0/2599518936 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f17ac06ec00 msgr2=0x7f17ac0710b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:39:30.747 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:39:30.747+0000 7f17cf200700 1 --2- 172.21.15.18:0/2599518936 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f17ac06ec00 0x7f17ac0710b0 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f17b4009040 tx=0x7f17b4009e30 comp rx=0 tx=0).stop 2024-04-12T21:39:30.747 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:39:30.747+0000 7f17cf200700 1 -- 172.21.15.18:0/2599518936 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f17c8109430 msgr2=0x7f17c811a140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:39:30.747 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:39:30.747+0000 7f17cf200700 1 --2- 172.21.15.18:0/2599518936 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f17c8109430 0x7f17c811a140 secure :-1 s=READY pgs=1009 cs=0 l=1 rev1=1 crypto rx=0x7f17b800b790 tx=0x7f17b8009250 comp rx=0 tx=0).stop 2024-04-12T21:39:30.747 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:39:30.747+0000 7f17cf200700 1 -- 172.21.15.18:0/2599518936 shutdown_connections 2024-04-12T21:39:30.747 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:39:30.747+0000 7f17cf200700 1 --2- 172.21.15.18:0/2599518936 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f17ac06ec00 0x7f17ac0710b0 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:39:30.747 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:39:30.747+0000 7f17cf200700 1 --2- 172.21.15.18:0/2599518936 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f17c8109430 0x7f17c811a140 unknown :-1 s=CLOSED pgs=1009 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:39:30.747 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:39:30.747+0000 7f17cf200700 1 --2- 172.21.15.18:0/2599518936 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f17c811a680 0x7f17c81142b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:39:30.747 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:39:30.747+0000 7f17cf200700 1 -- 172.21.15.18:0/2599518936 >> 172.21.15.18:0/2599518936 conn(0x7f17c8072c00 msgr2=0x7f17c8074d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:39:30.747 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:39:30.747+0000 7f17cf200700 1 -- 172.21.15.18:0/2599518936 shutdown_connections 2024-04-12T21:39:30.747 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:39:30.747+0000 7f17cf200700 1 -- 172.21.15.18:0/2599518936 wait complete. 2024-04-12T21:39:30.783 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:39:30 smithi186 ceph-mon[86379]: from='client.36076 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:39:30.783 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:39:30 smithi186 ceph-mon[86379]: from='client.36080 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:39:30.854 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:39:30 smithi018 ceph-mon[112590]: from='client.36076 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:39:30.854 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:39:30 smithi018 ceph-mon[112590]: from='client.36080 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:39:30.904 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:39:30.904+0000 7fc733e00700 1 -- 172.21.15.18:0/1120393100 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc734102c30 msgr2=0x7fc734103000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:39:30.904 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:39:30.904+0000 7fc733e00700 1 --2- 172.21.15.18:0/1120393100 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc734102c30 0x7fc734103000 secure :-1 s=READY pgs=1010 cs=0 l=1 rev1=1 crypto rx=0x7fc724004940 tx=0x7fc724009e30 comp rx=0 tx=0).stop 2024-04-12T21:39:30.904 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:39:30.904+0000 7fc733e00700 1 -- 172.21.15.18:0/1120393100 shutdown_connections 2024-04-12T21:39:30.904 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:39:30.904+0000 7fc733e00700 1 --2- 172.21.15.18:0/1120393100 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc7341035d0 0x7fc7340fcc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:39:30.904 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:39:30.904+0000 7fc733e00700 1 --2- 172.21.15.18:0/1120393100 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc734102c30 0x7fc734103000 unknown :-1 s=CLOSED pgs=1010 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:39:30.904 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:39:30.904+0000 7fc733e00700 1 -- 172.21.15.18:0/1120393100 >> 172.21.15.18:0/1120393100 conn(0x7fc7340f8760 msgr2=0x7fc7340fab70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:39:30.904 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:39:30.904+0000 7fc733e00700 1 -- 172.21.15.18:0/1120393100 shutdown_connections 2024-04-12T21:39:30.904 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:39:30.904+0000 7fc733e00700 1 -- 172.21.15.18:0/1120393100 wait complete. 2024-04-12T21:39:30.905 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:39:30.905+0000 7fc733e00700 1 Processor -- start 2024-04-12T21:39:30.905 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:39:30.905+0000 7fc733e00700 1 -- start start 2024-04-12T21:39:30.905 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:39:30.905+0000 7fc733e00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc734102c30 0x7fc734115c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:39:30.905 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:39:30.905+0000 7fc733e00700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc7341035d0 0x7fc734116140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:39:30.905 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:39:30.905+0000 7fc733e00700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc734116790 con 0x7fc734102c30 2024-04-12T21:39:30.905 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:39:30.905+0000 7fc733e00700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc73410fb80 con 0x7fc7341035d0 2024-04-12T21:39:30.907 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:39:30.906+0000 7fc732a00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc734102c30 0x7fc734115c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:39:30.907 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:39:30.906+0000 7fc732a00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc734102c30 0x7fc734115c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:33714/0 (socket says 172.21.15.18:33714) 2024-04-12T21:39:30.907 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:39:30.906+0000 7fc732a00700 1 -- 172.21.15.18:0/3864142227 learned_addr learned my addr 172.21.15.18:0/3864142227 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:39:30.907 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:39:30.906+0000 7fc72be00700 1 --2- 172.21.15.18:0/3864142227 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc7341035d0 0x7fc734116140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:39:30.907 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:39:30.906+0000 7fc732a00700 1 -- 172.21.15.18:0/3864142227 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc7341035d0 msgr2=0x7fc734116140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:39:30.907 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:39:30.906+0000 7fc732a00700 1 --2- 172.21.15.18:0/3864142227 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc7341035d0 0x7fc734116140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:39:30.907 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:39:30.906+0000 7fc732a00700 1 -- 172.21.15.18:0/3864142227 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc724005040 con 0x7fc734102c30 2024-04-12T21:39:30.907 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:39:30.907+0000 7fc732a00700 1 --2- 172.21.15.18:0/3864142227 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc734102c30 0x7fc734115c00 secure :-1 s=READY pgs=1011 cs=0 l=1 rev1=1 crypto rx=0x7fc724009150 tx=0x7fc724009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:39:30.907 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:39:30.907+0000 7fc72aa00700 1 -- 172.21.15.18:0/3864142227 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc72400e040 con 0x7fc734102c30 2024-04-12T21:39:30.907 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:39:30.907+0000 7fc733e00700 1 -- 172.21.15.18:0/3864142227 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc73410fe00 con 0x7fc734102c30 2024-04-12T21:39:30.907 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:39:30.907+0000 7fc72aa00700 1 -- 172.21.15.18:0/3864142227 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc724013420 con 0x7fc734102c30 2024-04-12T21:39:30.908 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:39:30.907+0000 7fc72aa00700 1 -- 172.21.15.18:0/3864142227 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc72401c630 con 0x7fc734102c30 2024-04-12T21:39:30.908 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:39:30.907+0000 7fc733e00700 1 -- 172.21.15.18:0/3864142227 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc7341103c0 con 0x7fc734102c30 2024-04-12T21:39:30.908 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:39:30.908+0000 7fc733e00700 1 -- 172.21.15.18:0/3864142227 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc7340fe380 con 0x7fc734102c30 2024-04-12T21:39:30.909 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:39:30.909+0000 7fc72aa00700 1 -- 172.21.15.18:0/3864142227 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7fc724013590 con 0x7fc734102c30 2024-04-12T21:39:30.910 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:39:30.910+0000 7fc72aa00700 1 --2- 172.21.15.18:0/3864142227 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fc71806ec00 0x7fc7180710b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:39:30.910 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:39:30.910+0000 7fc72aa00700 1 -- 172.21.15.18:0/3864142227 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 6757+0+0 (secure 0 0 0) 0x7fc72408fce0 con 0x7fc734102c30 2024-04-12T21:39:30.910 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:39:30.911+0000 7fc72be00700 1 --2- 172.21.15.18:0/3864142227 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fc71806ec00 0x7fc7180710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:39:30.911 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:39:30.911+0000 7fc72be00700 1 --2- 172.21.15.18:0/3864142227 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fc71806ec00 0x7fc7180710b0 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7fc71c009fd0 tx=0x7fc71c009f20 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:39:30.915 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:39:30.916+0000 7fc72aa00700 1 -- 172.21.15.18:0/3864142227 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7fc72405c490 con 0x7fc734102c30 2024-04-12T21:39:31.185 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:39:31.185+0000 7fc733e00700 1 -- 172.21.15.18:0/3864142227 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fc73404bed0 con 0x7fc734102c30 2024-04-12T21:39:31.185 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:39:31.186+0000 7fc72aa00700 1 -- 172.21.15.18:0/3864142227 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 33 v33) v1 ==== 76+0+1876 (secure 0 0 0) 0x7fc72405c020 con 0x7fc734102c30 2024-04-12T21:39:31.186 INFO:teuthology.orchestra.run.smithi018.stdout:e33 2024-04-12T21:39:31.186 INFO:teuthology.orchestra.run.smithi018.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-12T21:39:31.186 INFO:teuthology.orchestra.run.smithi018.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-12T21:39:31.186 INFO:teuthology.orchestra.run.smithi018.stdout:legacy client fscid: 1 2024-04-12T21:39:31.186 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-12T21:39:31.186 INFO:teuthology.orchestra.run.smithi018.stdout:Filesystem 'cephfs' (1) 2024-04-12T21:39:31.186 INFO:teuthology.orchestra.run.smithi018.stdout:fs_name cephfs 2024-04-12T21:39:31.187 INFO:teuthology.orchestra.run.smithi018.stdout:epoch 24 2024-04-12T21:39:31.187 INFO:teuthology.orchestra.run.smithi018.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-04-12T21:39:31.187 INFO:teuthology.orchestra.run.smithi018.stdout:created 2024-04-12T20:51:57.362850+0000 2024-04-12T21:39:31.187 INFO:teuthology.orchestra.run.smithi018.stdout:modified 2024-04-12T21:06:59.717190+0000 2024-04-12T21:39:31.187 INFO:teuthology.orchestra.run.smithi018.stdout:tableserver 0 2024-04-12T21:39:31.187 INFO:teuthology.orchestra.run.smithi018.stdout:root 0 2024-04-12T21:39:31.187 INFO:teuthology.orchestra.run.smithi018.stdout:session_timeout 60 2024-04-12T21:39:31.187 INFO:teuthology.orchestra.run.smithi018.stdout:session_autoclose 300 2024-04-12T21:39:31.187 INFO:teuthology.orchestra.run.smithi018.stdout:max_file_size 1099511627776 2024-04-12T21:39:31.187 INFO:teuthology.orchestra.run.smithi018.stdout:max_xattr_size 65536 2024-04-12T21:39:31.187 INFO:teuthology.orchestra.run.smithi018.stdout:required_client_features {} 2024-04-12T21:39:31.187 INFO:teuthology.orchestra.run.smithi018.stdout:last_failure 0 2024-04-12T21:39:31.187 INFO:teuthology.orchestra.run.smithi018.stdout:last_failure_osd_epoch 100 2024-04-12T21:39:31.187 INFO:teuthology.orchestra.run.smithi018.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-12T21:39:31.187 INFO:teuthology.orchestra.run.smithi018.stdout:max_mds 2 2024-04-12T21:39:31.188 INFO:teuthology.orchestra.run.smithi018.stdout:in 0,1 2024-04-12T21:39:31.188 INFO:teuthology.orchestra.run.smithi018.stdout:up {} 2024-04-12T21:39:31.188 INFO:teuthology.orchestra.run.smithi018.stdout:failed 0,1 2024-04-12T21:39:31.188 INFO:teuthology.orchestra.run.smithi018.stdout:damaged 2024-04-12T21:39:31.188 INFO:teuthology.orchestra.run.smithi018.stdout:stopped 2024-04-12T21:39:31.188 INFO:teuthology.orchestra.run.smithi018.stdout:data_pools [3] 2024-04-12T21:39:31.188 INFO:teuthology.orchestra.run.smithi018.stdout:metadata_pool 2 2024-04-12T21:39:31.188 INFO:teuthology.orchestra.run.smithi018.stdout:inline_data disabled 2024-04-12T21:39:31.188 INFO:teuthology.orchestra.run.smithi018.stdout:balancer 2024-04-12T21:39:31.188 INFO:teuthology.orchestra.run.smithi018.stdout:bal_rank_mask -1 2024-04-12T21:39:31.188 INFO:teuthology.orchestra.run.smithi018.stdout:standby_count_wanted 1 2024-04-12T21:39:31.188 INFO:teuthology.orchestra.run.smithi018.stdout:qdb_cluster leader: 0 members: 2024-04-12T21:39:31.188 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-12T21:39:31.188 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-12T21:39:31.188 INFO:teuthology.orchestra.run.smithi018.stdout:Standby daemons: 2024-04-12T21:39:31.188 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-12T21:39:31.189 INFO:teuthology.orchestra.run.smithi018.stdout:[mds.cephfs.smithi018.acqbsc{-1:34436} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.18:6828/755179764,v1:172.21.15.18:6829/755179764] compat {c=[1],r=[1],i=[1fff]}] 2024-04-12T21:39:31.189 INFO:teuthology.orchestra.run.smithi018.stdout:[mds.cephfs.smithi018.szjluo{-1:34468} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.18:6826/2407992653,v1:172.21.15.18:6827/2407992653] compat {c=[1],r=[1],i=[1fff]}] 2024-04-12T21:39:31.189 INFO:teuthology.orchestra.run.smithi018.stdout:[mds.cephfs.smithi186.jdllyn{-1:44595} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.186:6824/3523444202,v1:172.21.15.186:6825/3523444202] compat {c=[1],r=[1],i=[1fff]}] 2024-04-12T21:39:31.189 INFO:teuthology.orchestra.run.smithi018.stdout:[mds.cephfs.smithi186.wjnafs{-1:44875} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.186:6826/3122443766,v1:172.21.15.186:6827/3122443766] compat {c=[1],r=[1],i=[1fff]}] 2024-04-12T21:39:31.189 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:39:31.189+0000 7fc733e00700 1 -- 172.21.15.18:0/3864142227 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fc71806ec00 msgr2=0x7fc7180710b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:39:31.189 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:39:31.189+0000 7fc733e00700 1 --2- 172.21.15.18:0/3864142227 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fc71806ec00 0x7fc7180710b0 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7fc71c009fd0 tx=0x7fc71c009f20 comp rx=0 tx=0).stop 2024-04-12T21:39:31.190 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:39:31.189+0000 7fc733e00700 1 -- 172.21.15.18:0/3864142227 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc734102c30 msgr2=0x7fc734115c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:39:31.190 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:39:31.189+0000 7fc733e00700 1 --2- 172.21.15.18:0/3864142227 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc734102c30 0x7fc734115c00 secure :-1 s=READY pgs=1011 cs=0 l=1 rev1=1 crypto rx=0x7fc724009150 tx=0x7fc724009e30 comp rx=0 tx=0).stop 2024-04-12T21:39:31.190 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:39:31.189+0000 7fc733e00700 1 -- 172.21.15.18:0/3864142227 shutdown_connections 2024-04-12T21:39:31.190 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:39:31.189+0000 7fc733e00700 1 --2- 172.21.15.18:0/3864142227 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fc71806ec00 0x7fc7180710b0 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:39:31.190 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:39:31.189+0000 7fc733e00700 1 --2- 172.21.15.18:0/3864142227 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc734102c30 0x7fc734115c00 unknown :-1 s=CLOSED pgs=1011 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:39:31.190 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:39:31.189+0000 7fc733e00700 1 --2- 172.21.15.18:0/3864142227 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc7341035d0 0x7fc734116140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:39:31.190 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:39:31.189+0000 7fc733e00700 1 -- 172.21.15.18:0/3864142227 >> 172.21.15.18:0/3864142227 conn(0x7fc7340f8760 msgr2=0x7fc7340f9770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:39:31.190 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:39:31.189+0000 7fc733e00700 1 -- 172.21.15.18:0/3864142227 shutdown_connections 2024-04-12T21:39:31.190 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:39:31.189+0000 7fc733e00700 1 -- 172.21.15.18:0/3864142227 wait complete. 2024-04-12T21:39:31.190 INFO:teuthology.orchestra.run.smithi018.stderr:dumped fsmap epoch 33 2024-04-12T21:39:31.345 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:39:31.345+0000 7f8ceea00700 1 -- 172.21.15.18:0/134514910 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8ce8102e20 msgr2=0x7f8ce81031f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:39:31.345 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:39:31.345+0000 7f8ceea00700 1 --2- 172.21.15.18:0/134514910 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8ce8102e20 0x7f8ce81031f0 secure :-1 s=READY pgs=1012 cs=0 l=1 rev1=1 crypto rx=0x7f8cd8004940 tx=0x7f8cd8009e30 comp rx=0 tx=0).stop 2024-04-12T21:39:31.345 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:39:31.345+0000 7f8ceea00700 1 -- 172.21.15.18:0/134514910 shutdown_connections 2024-04-12T21:39:31.346 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:39:31.345+0000 7f8ceea00700 1 --2- 172.21.15.18:0/134514910 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8ce81037c0 0x7f8ce80fce30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:39:31.346 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:39:31.345+0000 7f8ceea00700 1 --2- 172.21.15.18:0/134514910 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8ce8102e20 0x7f8ce81031f0 unknown :-1 s=CLOSED pgs=1012 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:39:31.346 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:39:31.345+0000 7f8ceea00700 1 -- 172.21.15.18:0/134514910 >> 172.21.15.18:0/134514910 conn(0x7f8ce80f8970 msgr2=0x7f8ce80fad80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:39:31.346 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:39:31.345+0000 7f8ceea00700 1 -- 172.21.15.18:0/134514910 shutdown_connections 2024-04-12T21:39:31.346 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:39:31.346+0000 7f8ceea00700 1 -- 172.21.15.18:0/134514910 wait complete. 2024-04-12T21:39:31.346 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:39:31.346+0000 7f8ceea00700 1 Processor -- start 2024-04-12T21:39:31.346 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:39:31.347+0000 7f8ceea00700 1 -- start start 2024-04-12T21:39:31.347 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:39:31.347+0000 7f8ceea00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8ce8102e20 0x7f8ce8115db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:39:31.347 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:39:31.347+0000 7f8ceea00700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8ce81037c0 0x7f8ce81162f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:39:31.347 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:39:31.347+0000 7f8ceea00700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ce81169d0 con 0x7f8ce8102e20 2024-04-12T21:39:31.347 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:39:31.347+0000 7f8ceea00700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ce810fd10 con 0x7f8ce81037c0 2024-04-12T21:39:31.347 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:39:31.347+0000 7f8ce7e00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8ce8102e20 0x7f8ce8115db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:39:31.347 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:39:31.348+0000 7f8ce7e00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8ce8102e20 0x7f8ce8115db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:33724/0 (socket says 172.21.15.18:33724) 2024-04-12T21:39:31.348 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:39:31.348+0000 7f8ce7e00700 1 -- 172.21.15.18:0/577740816 learned_addr learned my addr 172.21.15.18:0/577740816 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:39:31.348 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:39:31.348+0000 7f8ce7e00700 1 -- 172.21.15.18:0/577740816 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8ce81037c0 msgr2=0x7f8ce81162f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:39:31.348 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:39:31.348+0000 7f8ce7e00700 1 --2- 172.21.15.18:0/577740816 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8ce81037c0 0x7f8ce81162f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:39:31.349 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:39:31.348+0000 7f8ce7e00700 1 -- 172.21.15.18:0/577740816 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8cd8005040 con 0x7f8ce8102e20 2024-04-12T21:39:31.349 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:39:31.348+0000 7f8ce7e00700 1 --2- 172.21.15.18:0/577740816 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8ce8102e20 0x7f8ce8115db0 secure :-1 s=READY pgs=1013 cs=0 l=1 rev1=1 crypto rx=0x7f8cd8009180 tx=0x7f8cd8009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:39:31.349 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:39:31.348+0000 7f8cdfe00700 1 -- 172.21.15.18:0/577740816 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8cd800e040 con 0x7f8ce8102e20 2024-04-12T21:39:31.349 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:39:31.349+0000 7f8cdfe00700 1 -- 172.21.15.18:0/577740816 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f8cd800bbc0 con 0x7f8ce8102e20 2024-04-12T21:39:31.349 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:39:31.349+0000 7f8ceea00700 1 -- 172.21.15.18:0/577740816 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8ce810ff30 con 0x7f8ce8102e20 2024-04-12T21:39:31.349 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:39:31.349+0000 7f8cdfe00700 1 -- 172.21.15.18:0/577740816 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8cd801c630 con 0x7f8ce8102e20 2024-04-12T21:39:31.349 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:39:31.349+0000 7f8ceea00700 1 -- 172.21.15.18:0/577740816 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8ce81104f0 con 0x7f8ce8102e20 2024-04-12T21:39:31.349 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:39:31.350+0000 7f8ceea00700 1 -- 172.21.15.18:0/577740816 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8ce80fe570 con 0x7f8ce8102e20 2024-04-12T21:39:31.350 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:39:31.351+0000 7f8cdfe00700 1 -- 172.21.15.18:0/577740816 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7f8cd800bd30 con 0x7f8ce8102e20 2024-04-12T21:39:31.352 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:39:31.352+0000 7f8cdfe00700 1 --2- 172.21.15.18:0/577740816 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f8ccc06ec10 0x7f8ccc0710c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:39:31.352 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:39:31.352+0000 7f8cdfe00700 1 -- 172.21.15.18:0/577740816 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 6757+0+0 (secure 0 0 0) 0x7f8cd808fe10 con 0x7f8ce8102e20 2024-04-12T21:39:31.352 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:39:31.352+0000 7f8ce7400700 1 --2- 172.21.15.18:0/577740816 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f8ccc06ec10 0x7f8ccc0710c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:39:31.352 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:39:31.353+0000 7f8ce7400700 1 --2- 172.21.15.18:0/577740816 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f8ccc06ec10 0x7f8ccc0710c0 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f8cd4006fd0 tx=0x7f8cd4006db0 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:39:31.357 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:39:31.357+0000 7f8cdfe00700 1 -- 172.21.15.18:0/577740816 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f8cd805c5c0 con 0x7f8ce8102e20 2024-04-12T21:39:31.612 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:39:31.612+0000 7f8ceea00700 1 -- 172.21.15.18:0/577740816 --> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f8ce810a030 con 0x7f8ccc06ec10 2024-04-12T21:39:31.614 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:39:31.615+0000 7f8cdfe00700 1 -- 172.21.15.18:0/577740816 <== mgr.34102 v2:172.21.15.18:6800/3812891919 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f8cd4009dd0 con 0x7f8ccc06ec10 2024-04-12T21:39:31.615 INFO:teuthology.orchestra.run.smithi018.stdout:{ 2024-04-12T21:39:31.615 INFO:teuthology.orchestra.run.smithi018.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:f4452def3be4d350f4a8e5feaf7ebb1511be9298fe68478ba7a2d6fbd7d69a63", 2024-04-12T21:39:31.615 INFO:teuthology.orchestra.run.smithi018.stdout: "in_progress": true, 2024-04-12T21:39:31.615 INFO:teuthology.orchestra.run.smithi018.stdout: "which": "Upgrading all daemon types on all hosts", 2024-04-12T21:39:31.615 INFO:teuthology.orchestra.run.smithi018.stdout: "services_complete": [ 2024-04-12T21:39:31.615 INFO:teuthology.orchestra.run.smithi018.stdout: "mgr", 2024-04-12T21:39:31.615 INFO:teuthology.orchestra.run.smithi018.stdout: "mds", 2024-04-12T21:39:31.615 INFO:teuthology.orchestra.run.smithi018.stdout: "crash", 2024-04-12T21:39:31.615 INFO:teuthology.orchestra.run.smithi018.stdout: "osd", 2024-04-12T21:39:31.615 INFO:teuthology.orchestra.run.smithi018.stdout: "mon" 2024-04-12T21:39:31.615 INFO:teuthology.orchestra.run.smithi018.stdout: ], 2024-04-12T21:39:31.615 INFO:teuthology.orchestra.run.smithi018.stdout: "progress": "16/21 daemons upgraded", 2024-04-12T21:39:31.615 INFO:teuthology.orchestra.run.smithi018.stdout: "message": "Currently upgrading mds daemons", 2024-04-12T21:39:31.615 INFO:teuthology.orchestra.run.smithi018.stdout: "is_paused": false 2024-04-12T21:39:31.615 INFO:teuthology.orchestra.run.smithi018.stdout:} 2024-04-12T21:39:31.617 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:39:31.617+0000 7f8ceea00700 1 -- 172.21.15.18:0/577740816 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f8ccc06ec10 msgr2=0x7f8ccc0710c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:39:31.617 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:39:31.617+0000 7f8ceea00700 1 --2- 172.21.15.18:0/577740816 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f8ccc06ec10 0x7f8ccc0710c0 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f8cd4006fd0 tx=0x7f8cd4006db0 comp rx=0 tx=0).stop 2024-04-12T21:39:31.617 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:39:31.617+0000 7f8ceea00700 1 -- 172.21.15.18:0/577740816 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8ce8102e20 msgr2=0x7f8ce8115db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:39:31.617 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:39:31.617+0000 7f8ceea00700 1 --2- 172.21.15.18:0/577740816 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8ce8102e20 0x7f8ce8115db0 secure :-1 s=READY pgs=1013 cs=0 l=1 rev1=1 crypto rx=0x7f8cd8009180 tx=0x7f8cd8009e30 comp rx=0 tx=0).stop 2024-04-12T21:39:31.618 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:39:31.617+0000 7f8ceea00700 1 -- 172.21.15.18:0/577740816 shutdown_connections 2024-04-12T21:39:31.618 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:39:31.618+0000 7f8ceea00700 1 --2- 172.21.15.18:0/577740816 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f8ccc06ec10 0x7f8ccc0710c0 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:39:31.618 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:39:31.618+0000 7f8ceea00700 1 --2- 172.21.15.18:0/577740816 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8ce8102e20 0x7f8ce8115db0 unknown :-1 s=CLOSED pgs=1013 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:39:31.618 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:39:31.618+0000 7f8ceea00700 1 --2- 172.21.15.18:0/577740816 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8ce81037c0 0x7f8ce81162f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:39:31.618 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:39:31.618+0000 7f8ceea00700 1 -- 172.21.15.18:0/577740816 >> 172.21.15.18:0/577740816 conn(0x7f8ce80f8970 msgr2=0x7f8ce80719f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:39:31.618 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:39:31.618+0000 7f8ceea00700 1 -- 172.21.15.18:0/577740816 shutdown_connections 2024-04-12T21:39:31.618 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:39:31.618+0000 7f8ceea00700 1 -- 172.21.15.18:0/577740816 wait complete. 2024-04-12T21:39:31.771 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:39:31.772+0000 7fe7bac00700 1 -- 172.21.15.18:0/944733036 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe7b4105020 msgr2=0x7fe7b41053f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:39:31.772 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:39:31.772+0000 7fe7bac00700 1 --2- 172.21.15.18:0/944733036 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe7b4105020 0x7fe7b41053f0 secure :-1 s=READY pgs=1014 cs=0 l=1 rev1=1 crypto rx=0x7fe7a4004940 tx=0x7fe7a4009e30 comp rx=0 tx=0).stop 2024-04-12T21:39:31.772 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:39:31 smithi018 ceph-mon[112590]: pgmap v1195: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:39:31.772 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:39:31 smithi018 ceph-mon[112590]: from='client.36084 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:39:31.772 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:39:31 smithi018 ceph-mon[112590]: from='client.? 172.21.15.18:0/2599518936' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T21:39:31.772 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:39:31 smithi018 ceph-mon[112590]: from='client.? 172.21.15.18:0/3864142227' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-12T21:39:31.773 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:39:31.772+0000 7fe7bac00700 1 -- 172.21.15.18:0/944733036 shutdown_connections 2024-04-12T21:39:31.773 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:39:31.772+0000 7fe7bac00700 1 --2- 172.21.15.18:0/944733036 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe7b41059c0 0x7fe7b40ff080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:39:31.773 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:39:31.772+0000 7fe7bac00700 1 --2- 172.21.15.18:0/944733036 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe7b4105020 0x7fe7b41053f0 unknown :-1 s=CLOSED pgs=1014 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:39:31.773 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:39:31.772+0000 7fe7bac00700 1 -- 172.21.15.18:0/944733036 >> 172.21.15.18:0/944733036 conn(0x7fe7b40faba0 msgr2=0x7fe7b40fcfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:39:31.773 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:39:31.772+0000 7fe7bac00700 1 -- 172.21.15.18:0/944733036 shutdown_connections 2024-04-12T21:39:31.773 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:39:31.772+0000 7fe7bac00700 1 -- 172.21.15.18:0/944733036 wait complete. 2024-04-12T21:39:31.773 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:39:31.773+0000 7fe7bac00700 1 Processor -- start 2024-04-12T21:39:31.773 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:39:31.773+0000 7fe7bac00700 1 -- start start 2024-04-12T21:39:31.774 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:39:31.774+0000 7fe7bac00700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe7b4105020 0x7fe7b4115bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:39:31.774 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:39:31.774+0000 7fe7bac00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe7b41059c0 0x7fe7b4116130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:39:31.774 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:39:31.774+0000 7fe7bac00700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7b4116810 con 0x7fe7b41059c0 2024-04-12T21:39:31.774 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:39:31.774+0000 7fe7bac00700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7b410fd60 con 0x7fe7b4105020 2024-04-12T21:39:31.774 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:39:31.774+0000 7fe7abe00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe7b41059c0 0x7fe7b4116130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:39:31.774 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:39:31.774+0000 7fe7abe00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe7b41059c0 0x7fe7b4116130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:33740/0 (socket says 172.21.15.18:33740) 2024-04-12T21:39:31.774 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:39:31.774+0000 7fe7abe00700 1 -- 172.21.15.18:0/643099633 learned_addr learned my addr 172.21.15.18:0/643099633 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:39:31.775 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:39:31.774+0000 7fe7b3e00700 1 --2- 172.21.15.18:0/643099633 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe7b4105020 0x7fe7b4115bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:39:31.775 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:39:31.775+0000 7fe7abe00700 1 -- 172.21.15.18:0/643099633 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe7b4105020 msgr2=0x7fe7b4115bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:39:31.775 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:39:31.775+0000 7fe7abe00700 1 --2- 172.21.15.18:0/643099633 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe7b4105020 0x7fe7b4115bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:39:31.776 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:39:31.775+0000 7fe7abe00700 1 -- 172.21.15.18:0/643099633 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7a4005040 con 0x7fe7b41059c0 2024-04-12T21:39:31.776 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:39:31.775+0000 7fe7abe00700 1 --2- 172.21.15.18:0/643099633 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe7b41059c0 0x7fe7b4116130 secure :-1 s=READY pgs=1015 cs=0 l=1 rev1=1 crypto rx=0x7fe79c00b790 tx=0x7fe79c009220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:39:31.776 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:39:31.775+0000 7fe7b0c00700 1 -- 172.21.15.18:0/643099633 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe79c00f6f0 con 0x7fe7b41059c0 2024-04-12T21:39:31.776 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:39:31.775+0000 7fe7b0c00700 1 -- 172.21.15.18:0/643099633 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe79c00fd20 con 0x7fe7b41059c0 2024-04-12T21:39:31.776 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:39:31.775+0000 7fe7b0c00700 1 -- 172.21.15.18:0/643099633 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe79c0103a0 con 0x7fe7b41059c0 2024-04-12T21:39:31.776 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:39:31.775+0000 7fe7bac00700 1 -- 172.21.15.18:0/643099633 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe7b410ffe0 con 0x7fe7b41059c0 2024-04-12T21:39:31.776 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:39:31.776+0000 7fe7bac00700 1 -- 172.21.15.18:0/643099633 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe7b4110520 con 0x7fe7b41059c0 2024-04-12T21:39:31.776 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:39:31.777+0000 7fe7bac00700 1 -- 172.21.15.18:0/643099633 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe7b4100770 con 0x7fe7b41059c0 2024-04-12T21:39:31.778 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:39:31.778+0000 7fe7b0c00700 1 -- 172.21.15.18:0/643099633 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7fe79c011400 con 0x7fe7b41059c0 2024-04-12T21:39:31.779 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:39:31.779+0000 7fe7b0c00700 1 --2- 172.21.15.18:0/643099633 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fe79806ec00 0x7fe7980710b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:39:31.779 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:39:31.779+0000 7fe7b0c00700 1 -- 172.21.15.18:0/643099633 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 6757+0+0 (secure 0 0 0) 0x7fe79c08e820 con 0x7fe7b41059c0 2024-04-12T21:39:31.779 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:39:31.779+0000 7fe7b3e00700 1 --2- 172.21.15.18:0/643099633 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fe79806ec00 0x7fe7980710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:39:31.779 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:39:31.780+0000 7fe7b3e00700 1 --2- 172.21.15.18:0/643099633 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fe79806ec00 0x7fe7980710b0 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7fe7a400b020 tx=0x7fe7a4009e30 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:39:31.783 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:39:31 smithi186 ceph-mon[86379]: pgmap v1195: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:39:31.783 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:39:31 smithi186 ceph-mon[86379]: from='client.36084 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:39:31.783 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:39:31 smithi186 ceph-mon[86379]: from='client.? 172.21.15.18:0/2599518936' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T21:39:31.783 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:39:31 smithi186 ceph-mon[86379]: from='client.? 172.21.15.18:0/3864142227' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-12T21:39:31.783 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:39:31.784+0000 7fe7b0c00700 1 -- 172.21.15.18:0/643099633 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7fe79c05afd0 con 0x7fe7b41059c0 2024-04-12T21:39:32.099 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:39:32.099+0000 7fe7bac00700 1 -- 172.21.15.18:0/643099633 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7fe7b410c190 con 0x7fe7b41059c0 2024-04-12T21:39:32.100 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:39:32.100+0000 7fe7b0c00700 1 -- 172.21.15.18:0/643099633 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+4303 (secure 0 0 0) 0x7fe79c05ab60 con 0x7fe7b41059c0 2024-04-12T21:39:32.101 INFO:teuthology.orchestra.run.smithi018.stdout:HEALTH_ERR failed to probe daemons or devices; 1 filesystem is degraded; 1 filesystem is offline 2024-04-12T21:39:32.101 INFO:teuthology.orchestra.run.smithi018.stdout:[WRN] CEPHADM_REFRESH_FAILED: failed to probe daemons or devices 2024-04-12T21:39:32.101 INFO:teuthology.orchestra.run.smithi018.stdout: host smithi186 `cephadm ceph-volume` failed: cephadm exited with an error code: 1, stderr: Inferring config /var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/mon.smithi186/config 2024-04-12T21:39:32.101 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/podman: timeout after 895 seconds 2024-04-12T21:39:32.101 INFO:teuthology.orchestra.run.smithi018.stdout:Non-zero exit code 124 from /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:f4452def3be4d350f4a8e5feaf7ebb1511be9298fe68478ba7a2d6fbd7d69a63 -e NODE_NAME=smithi186 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968:/var/run/ceph:z -v /var/log/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968:/var/log/ceph:z -v /var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmptgom53f9:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:f4452def3be4d350f4a8e5feaf7ebb1511be9298fe68478ba7a2d6fbd7d69a63 inventory --format=json-pretty --filter-for-batch 2024-04-12T21:39:32.101 INFO:teuthology.orchestra.run.smithi018.stdout:Traceback (most recent call last): 2024-04-12T21:39:32.101 INFO:teuthology.orchestra.run.smithi018.stdout: File "/usr/lib64/python3.9/runpy.py", line 197, in _run_module_as_main 2024-04-12T21:39:32.101 INFO:teuthology.orchestra.run.smithi018.stdout: return _run_code(code, main_globals, None, 2024-04-12T21:39:32.101 INFO:teuthology.orchestra.run.smithi018.stdout: File "/usr/lib64/python3.9/runpy.py", line 87, in _run_code 2024-04-12T21:39:32.101 INFO:teuthology.orchestra.run.smithi018.stdout: exec(code, run_globals) 2024-04-12T21:39:32.101 INFO:teuthology.orchestra.run.smithi018.stdout: File "/var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/cephadm.e44f3ed5e34e4ef52eedef233d1190bff004470d8e8005127b5312bff357fa7f/__main__.py", line 5618, in 2024-04-12T21:39:32.101 INFO:teuthology.orchestra.run.smithi018.stdout: File "/var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/cephadm.e44f3ed5e34e4ef52eedef233d1190bff004470d8e8005127b5312bff357fa7f/__main__.py", line 5606, in main 2024-04-12T21:39:32.102 INFO:teuthology.orchestra.run.smithi018.stdout: File "/var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/cephadm.e44f3ed5e34e4ef52eedef233d1190bff004470d8e8005127b5312bff357fa7f/__main__.py", line 408, in _infer_config 2024-04-12T21:39:32.102 INFO:teuthology.orchestra.run.smithi018.stdout: File "/var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/cephadm.e44f3ed5e34e4ef52eedef233d1190bff004470d8e8005127b5312bff357fa7f/__main__.py", line 323, in _infer_fsid 2024-04-12T21:39:32.102 INFO:teuthology.orchestra.run.smithi018.stdout: File "/var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/cephadm.e44f3ed5e34e4ef52eedef233d1190bff004470d8e8005127b5312bff357fa7f/__main__.py", line 436, in _infer_image 2024-04-12T21:39:32.102 INFO:teuthology.orchestra.run.smithi018.stdout: File "/var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/cephadm.e44f3ed5e34e4ef52eedef233d1190bff004470d8e8005127b5312bff357fa7f/__main__.py", line 310, in _validate_fsid 2024-04-12T21:39:32.102 INFO:teuthology.orchestra.run.smithi018.stdout: File "/var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/cephadm.e44f3ed5e34e4ef52eedef233d1190bff004470d8e8005127b5312bff357fa7f/__main__.py", line 3359, in command_ceph_volume 2024-04-12T21:39:32.102 INFO:teuthology.orchestra.run.smithi018.stdout: File "/var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/cephadm.e44f3ed5e34e4ef52eedef233d1190bff004470d8e8005127b5312bff357fa7f/cephadmlib/call_wrappers.py", line 310, in call_throws 2024-04-12T21:39:32.102 INFO:teuthology.orchestra.run.smithi018.stdout:RuntimeError: Failed command: /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:f4452def3be4d350f4a8e5feaf7ebb1511be9298fe68478ba7a2d6fbd7d69a63 -e NODE_NAME=smithi186 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968:/var/run/ceph:z -v /var/log/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968:/var/log/ceph:z -v /var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmptgom53f9:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:f4452def3be4d350f4a8e5feaf7ebb1511be9298fe68478ba7a2d6fbd7d69a63 inventory --format=json-pretty --filter-for-batch 2024-04-12T21:39:32.102 INFO:teuthology.orchestra.run.smithi018.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-04-12T21:39:32.102 INFO:teuthology.orchestra.run.smithi018.stdout: fs cephfs is degraded 2024-04-12T21:39:32.102 INFO:teuthology.orchestra.run.smithi018.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-04-12T21:39:32.102 INFO:teuthology.orchestra.run.smithi018.stdout: fs cephfs is offline because no MDS is active for it. 2024-04-12T21:39:32.104 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:39:32.104+0000 7fe7bac00700 1 -- 172.21.15.18:0/643099633 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fe79806ec00 msgr2=0x7fe7980710b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:39:32.104 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:39:32.104+0000 7fe7bac00700 1 --2- 172.21.15.18:0/643099633 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fe79806ec00 0x7fe7980710b0 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7fe7a400b020 tx=0x7fe7a4009e30 comp rx=0 tx=0).stop 2024-04-12T21:39:32.104 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:39:32.104+0000 7fe7bac00700 1 -- 172.21.15.18:0/643099633 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe7b41059c0 msgr2=0x7fe7b4116130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:39:32.104 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:39:32.104+0000 7fe7bac00700 1 --2- 172.21.15.18:0/643099633 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe7b41059c0 0x7fe7b4116130 secure :-1 s=READY pgs=1015 cs=0 l=1 rev1=1 crypto rx=0x7fe79c00b790 tx=0x7fe79c009220 comp rx=0 tx=0).stop 2024-04-12T21:39:32.104 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:39:32.104+0000 7fe7bac00700 1 -- 172.21.15.18:0/643099633 shutdown_connections 2024-04-12T21:39:32.105 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:39:32.104+0000 7fe7bac00700 1 --2- 172.21.15.18:0/643099633 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fe79806ec00 0x7fe7980710b0 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:39:32.105 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:39:32.104+0000 7fe7bac00700 1 --2- 172.21.15.18:0/643099633 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe7b4105020 0x7fe7b4115bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:39:32.105 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:39:32.104+0000 7fe7bac00700 1 --2- 172.21.15.18:0/643099633 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe7b41059c0 0x7fe7b4116130 unknown :-1 s=CLOSED pgs=1015 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:39:32.105 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:39:32.104+0000 7fe7bac00700 1 -- 172.21.15.18:0/643099633 >> 172.21.15.18:0/643099633 conn(0x7fe7b40faba0 msgr2=0x7fe7b40fbb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:39:32.105 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:39:32.104+0000 7fe7bac00700 1 -- 172.21.15.18:0/643099633 shutdown_connections 2024-04-12T21:39:32.105 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:39:32.105+0000 7fe7bac00700 1 -- 172.21.15.18:0/643099633 wait complete. 2024-04-12T21:39:32.783 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:39:32 smithi186 ceph-mon[86379]: from='client.36096 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:39:32.783 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:39:32 smithi186 ceph-mon[86379]: from='client.? 172.21.15.18:0/643099633' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-12T21:39:32.854 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:39:32 smithi018 ceph-mon[112590]: from='client.36096 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:39:32.854 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:39:32 smithi018 ceph-mon[112590]: from='client.? 172.21.15.18:0/643099633' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-12T21:39:33.854 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:39:33 smithi018 ceph-mon[112590]: pgmap v1196: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:39:34.033 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:39:33 smithi186 ceph-mon[86379]: pgmap v1196: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:39:35.854 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:39:35 smithi018 ceph-mon[112590]: pgmap v1197: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:39:36.033 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:39:35 smithi186 ceph-mon[86379]: pgmap v1197: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:39:37.854 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:39:37 smithi018 ceph-mon[112590]: pgmap v1198: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:39:38.033 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:39:37 smithi186 ceph-mon[86379]: pgmap v1198: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:39:39.854 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:39:39 smithi018 ceph-mon[112590]: pgmap v1199: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:39:40.033 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:39:39 smithi186 ceph-mon[86379]: pgmap v1199: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:39:40.854 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:39:40 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T21:39:41.033 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:39:40 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T21:39:41.854 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:39:41 smithi018 ceph-mon[112590]: pgmap v1200: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:39:42.033 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:39:41 smithi186 ceph-mon[86379]: pgmap v1200: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:39:43.854 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:39:43 smithi018 ceph-mon[112590]: pgmap v1201: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:39:44.033 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:39:43 smithi186 ceph-mon[86379]: pgmap v1201: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:39:45.657 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:39:45 smithi018 ceph-mon[112590]: pgmap v1202: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:39:46.033 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:39:45 smithi186 ceph-mon[86379]: pgmap v1202: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:39:47.854 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:39:47 smithi018 ceph-mon[112590]: pgmap v1203: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:39:48.033 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:39:47 smithi186 ceph-mon[86379]: pgmap v1203: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:39:49.854 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:39:49 smithi018 ceph-mon[112590]: pgmap v1204: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:39:50.033 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:39:49 smithi186 ceph-mon[86379]: pgmap v1204: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:39:51.854 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:39:51 smithi018 ceph-mon[112590]: pgmap v1205: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:39:52.033 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:39:51 smithi186 ceph-mon[86379]: pgmap v1205: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:39:54.033 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:39:53 smithi186 ceph-mon[86379]: pgmap v1206: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:39:54.104 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:39:53 smithi018 ceph-mon[112590]: pgmap v1206: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:39:55.854 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:39:55 smithi018 ceph-mon[112590]: pgmap v1207: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:39:55.854 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:39:55 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T21:39:56.033 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:39:55 smithi186 ceph-mon[86379]: pgmap v1207: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:39:56.033 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:39:55 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T21:39:58.033 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:39:57 smithi186 ceph-mon[86379]: pgmap v1208: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:39:58.104 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:39:57 smithi018 ceph-mon[112590]: pgmap v1208: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:40:00.000 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:39:59 smithi018 ceph-mon[112590]: pgmap v1209: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:40:00.033 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:39:59 smithi186 ceph-mon[86379]: pgmap v1209: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:40:00.355 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:40:00 smithi018 ceph-1cbe407a-f90d-11ee-bc8d-c7b262605968-mon-smithi018[112586]: 2024-04-12T21:39:59.998+0000 7fd45a600700 -1 log_channel(cluster) log [ERR] : overall HEALTH_ERR failed to probe daemons or devices; 1 filesystem is degraded; 1 filesystem is offline 2024-04-12T21:40:01.033 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:40:00 smithi186 ceph-mon[86379]: overall HEALTH_ERR failed to probe daemons or devices; 1 filesystem is degraded; 1 filesystem is offline 2024-04-12T21:40:01.104 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:40:00 smithi018 ceph-mon[112590]: overall HEALTH_ERR failed to probe daemons or devices; 1 filesystem is degraded; 1 filesystem is offline 2024-04-12T21:40:02.033 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:40:01 smithi186 ceph-mon[86379]: pgmap v1210: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:40:02.104 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:40:01 smithi018 ceph-mon[112590]: pgmap v1210: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:40:02.276 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:02.275+0000 7f19dec00700 1 -- 172.21.15.18:0/3471495963 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f19d8109c40 msgr2=0x7f19d81032b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:40:02.276 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:02.275+0000 7f19dec00700 1 --2- 172.21.15.18:0/3471495963 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f19d8109c40 0x7f19d81032b0 secure :-1 s=READY pgs=1016 cs=0 l=1 rev1=1 crypto rx=0x7f19c4004940 tx=0x7f19c4009e30 comp rx=0 tx=0).stop 2024-04-12T21:40:02.277 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:02.275+0000 7f19dec00700 1 -- 172.21.15.18:0/3471495963 shutdown_connections 2024-04-12T21:40:02.277 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:02.275+0000 7f19dec00700 1 --2- 172.21.15.18:0/3471495963 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f19d8109c40 0x7f19d81032b0 unknown :-1 s=CLOSED pgs=1016 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:40:02.277 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:02.275+0000 7f19dec00700 1 --2- 172.21.15.18:0/3471495963 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f19d81092a0 0x7f19d8109670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:40:02.277 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:02.275+0000 7f19dec00700 1 -- 172.21.15.18:0/3471495963 >> 172.21.15.18:0/3471495963 conn(0x7f19d8072a30 msgr2=0x7f19d8074e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:40:02.277 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:02.275+0000 7f19dec00700 1 -- 172.21.15.18:0/3471495963 shutdown_connections 2024-04-12T21:40:02.277 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:02.275+0000 7f19dec00700 1 -- 172.21.15.18:0/3471495963 wait complete. 2024-04-12T21:40:02.279 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:02.276+0000 7f19dec00700 1 Processor -- start 2024-04-12T21:40:02.279 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:02.276+0000 7f19dec00700 1 -- start start 2024-04-12T21:40:02.279 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:02.277+0000 7f19dec00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f19d81092a0 0x7f19d8119f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:40:02.279 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:02.277+0000 7f19dec00700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f19d8109c40 0x7f19d811a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:40:02.279 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:02.277+0000 7f19dec00700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19d811aad0 con 0x7f19d81092a0 2024-04-12T21:40:02.279 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:02.277+0000 7f19dec00700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19d8113ea0 con 0x7f19d8109c40 2024-04-12T21:40:02.279 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:02.277+0000 7f19dd800700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f19d81092a0 0x7f19d8119f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:40:02.279 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:02.277+0000 7f19dd800700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f19d81092a0 0x7f19d8119f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:47736/0 (socket says 172.21.15.18:47736) 2024-04-12T21:40:02.279 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:02.277+0000 7f19dd800700 1 -- 172.21.15.18:0/1781280877 learned_addr learned my addr 172.21.15.18:0/1781280877 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:40:02.279 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:02.277+0000 7f19dce00700 1 --2- 172.21.15.18:0/1781280877 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f19d8109c40 0x7f19d811a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:40:02.280 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:02.277+0000 7f19dd800700 1 -- 172.21.15.18:0/1781280877 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f19d8109c40 msgr2=0x7f19d811a480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:40:02.280 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:02.277+0000 7f19dd800700 1 --2- 172.21.15.18:0/1781280877 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f19d8109c40 0x7f19d811a480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:40:02.280 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:02.277+0000 7f19dd800700 1 -- 172.21.15.18:0/1781280877 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19c4005040 con 0x7f19d81092a0 2024-04-12T21:40:02.280 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:02.278+0000 7f19dd800700 1 --2- 172.21.15.18:0/1781280877 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f19d81092a0 0x7f19d8119f40 secure :-1 s=READY pgs=1017 cs=0 l=1 rev1=1 crypto rx=0x7f19cc00eb70 tx=0x7f19cc00b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:40:02.280 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:02.278+0000 7f19d5600700 1 -- 172.21.15.18:0/1781280877 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f19cc007cc0 con 0x7f19d81092a0 2024-04-12T21:40:02.280 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:02.278+0000 7f19d5600700 1 -- 172.21.15.18:0/1781280877 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f19cc00f040 con 0x7f19d81092a0 2024-04-12T21:40:02.280 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:02.278+0000 7f19dec00700 1 -- 172.21.15.18:0/1781280877 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f19d8114120 con 0x7f19d81092a0 2024-04-12T21:40:02.280 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:02.278+0000 7f19d5600700 1 -- 172.21.15.18:0/1781280877 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f19cc009640 con 0x7f19d81092a0 2024-04-12T21:40:02.280 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:02.278+0000 7f19dec00700 1 -- 172.21.15.18:0/1781280877 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f19d81146e0 con 0x7f19d81092a0 2024-04-12T21:40:02.281 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:02.279+0000 7f19dec00700 1 -- 172.21.15.18:0/1781280877 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f19d81049f0 con 0x7f19d81092a0 2024-04-12T21:40:02.281 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:02.280+0000 7f19d5600700 1 -- 172.21.15.18:0/1781280877 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7f19cc0097a0 con 0x7f19d81092a0 2024-04-12T21:40:02.282 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:02.281+0000 7f19d5600700 1 --2- 172.21.15.18:0/1781280877 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f19c006ec00 0x7f19c00710b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:40:02.282 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:02.281+0000 7f19d5600700 1 -- 172.21.15.18:0/1781280877 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 6757+0+0 (secure 0 0 0) 0x7f19cc08e520 con 0x7f19d81092a0 2024-04-12T21:40:02.282 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:02.282+0000 7f19dce00700 1 --2- 172.21.15.18:0/1781280877 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f19c006ec00 0x7f19c00710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:40:02.283 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:02.282+0000 7f19dce00700 1 --2- 172.21.15.18:0/1781280877 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f19c006ec00 0x7f19c00710b0 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f19c4000c00 tx=0x7f19c4009e30 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:40:02.287 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:02.287+0000 7f19d5600700 1 -- 172.21.15.18:0/1781280877 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f19cc05ad70 con 0x7f19d81092a0 2024-04-12T21:40:02.546 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:02.544+0000 7f19dec00700 1 -- 172.21.15.18:0/1781280877 --> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f19d804bf40 con 0x7f19c006ec00 2024-04-12T21:40:02.549 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:02.548+0000 7f19d5600700 1 -- 172.21.15.18:0/1781280877 <== mgr.34102 v2:172.21.15.18:6800/3812891919 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f19d804bf40 con 0x7f19c006ec00 2024-04-12T21:40:02.552 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:02.551+0000 7f19dec00700 1 -- 172.21.15.18:0/1781280877 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f19c006ec00 msgr2=0x7f19c00710b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:40:02.552 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:02.551+0000 7f19dec00700 1 --2- 172.21.15.18:0/1781280877 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f19c006ec00 0x7f19c00710b0 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f19c4000c00 tx=0x7f19c4009e30 comp rx=0 tx=0).stop 2024-04-12T21:40:02.552 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:02.551+0000 7f19dec00700 1 -- 172.21.15.18:0/1781280877 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f19d81092a0 msgr2=0x7f19d8119f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:40:02.552 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:02.551+0000 7f19dec00700 1 --2- 172.21.15.18:0/1781280877 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f19d81092a0 0x7f19d8119f40 secure :-1 s=READY pgs=1017 cs=0 l=1 rev1=1 crypto rx=0x7f19cc00eb70 tx=0x7f19cc00b040 comp rx=0 tx=0).stop 2024-04-12T21:40:02.552 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:02.551+0000 7f19dec00700 1 -- 172.21.15.18:0/1781280877 shutdown_connections 2024-04-12T21:40:02.552 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:02.551+0000 7f19dec00700 1 --2- 172.21.15.18:0/1781280877 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f19c006ec00 0x7f19c00710b0 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:40:02.553 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:02.551+0000 7f19dec00700 1 --2- 172.21.15.18:0/1781280877 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f19d81092a0 0x7f19d8119f40 unknown :-1 s=CLOSED pgs=1017 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:40:02.553 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:02.551+0000 7f19dec00700 1 --2- 172.21.15.18:0/1781280877 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f19d8109c40 0x7f19d811a480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:40:02.553 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:02.551+0000 7f19dec00700 1 -- 172.21.15.18:0/1781280877 >> 172.21.15.18:0/1781280877 conn(0x7f19d8072a30 msgr2=0x7f19d8074900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:40:02.553 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:02.552+0000 7f19dec00700 1 -- 172.21.15.18:0/1781280877 shutdown_connections 2024-04-12T21:40:02.553 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:02.552+0000 7f19dec00700 1 -- 172.21.15.18:0/1781280877 wait complete. 2024-04-12T21:40:02.570 INFO:teuthology.orchestra.run.smithi018.stdout:true 2024-04-12T21:40:02.709 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:02.708+0000 7f4ee2800700 1 -- 172.21.15.18:0/555962520 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4edc102e00 msgr2=0x7f4edc1031d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:40:02.709 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:02.708+0000 7f4ee2800700 1 --2- 172.21.15.18:0/555962520 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4edc102e00 0x7f4edc1031d0 secure :-1 s=READY pgs=1018 cs=0 l=1 rev1=1 crypto rx=0x7f4ecc004940 tx=0x7f4ecc009e30 comp rx=0 tx=0).stop 2024-04-12T21:40:02.709 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:02.708+0000 7f4ee2800700 1 -- 172.21.15.18:0/555962520 shutdown_connections 2024-04-12T21:40:02.709 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:02.708+0000 7f4ee2800700 1 --2- 172.21.15.18:0/555962520 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4edc1037a0 0x7f4edc0fce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:40:02.710 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:02.708+0000 7f4ee2800700 1 --2- 172.21.15.18:0/555962520 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4edc102e00 0x7f4edc1031d0 unknown :-1 s=CLOSED pgs=1018 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:40:02.710 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:02.708+0000 7f4ee2800700 1 -- 172.21.15.18:0/555962520 >> 172.21.15.18:0/555962520 conn(0x7f4edc0f89b0 msgr2=0x7f4edc0fadc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:40:02.710 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:02.708+0000 7f4ee2800700 1 -- 172.21.15.18:0/555962520 shutdown_connections 2024-04-12T21:40:02.710 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:02.708+0000 7f4ee2800700 1 -- 172.21.15.18:0/555962520 wait complete. 2024-04-12T21:40:02.710 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:02.710+0000 7f4ee2800700 1 Processor -- start 2024-04-12T21:40:02.710 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:02.710+0000 7f4ee2800700 1 -- start start 2024-04-12T21:40:02.711 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:02.710+0000 7f4ee2800700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4edc102e00 0x7f4edc115b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:40:02.711 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:02.710+0000 7f4ee2800700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4edc1037a0 0x7f4edc116040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:40:02.711 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:02.711+0000 7f4ee2800700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4edc116690 con 0x7f4edc1037a0 2024-04-12T21:40:02.711 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:02.711+0000 7f4ee2800700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4edc10fd20 con 0x7f4edc102e00 2024-04-12T21:40:02.712 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:02.711+0000 7f4edaa00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4edc1037a0 0x7f4edc116040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:40:02.712 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:02.711+0000 7f4edaa00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4edc1037a0 0x7f4edc116040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:47744/0 (socket says 172.21.15.18:47744) 2024-04-12T21:40:02.712 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:02.711+0000 7f4edaa00700 1 -- 172.21.15.18:0/3063098865 learned_addr learned my addr 172.21.15.18:0/3063098865 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:40:02.712 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:02.711+0000 7f4edb400700 1 --2- 172.21.15.18:0/3063098865 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4edc102e00 0x7f4edc115b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:40:02.713 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:02.712+0000 7f4edaa00700 1 -- 172.21.15.18:0/3063098865 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4edc102e00 msgr2=0x7f4edc115b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:40:02.713 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:02.712+0000 7f4edaa00700 1 --2- 172.21.15.18:0/3063098865 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4edc102e00 0x7f4edc115b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:40:02.713 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:02.712+0000 7f4edaa00700 1 -- 172.21.15.18:0/3063098865 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ecc005040 con 0x7f4edc1037a0 2024-04-12T21:40:02.713 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:02.712+0000 7f4edaa00700 1 --2- 172.21.15.18:0/3063098865 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4edc1037a0 0x7f4edc116040 secure :-1 s=READY pgs=1019 cs=0 l=1 rev1=1 crypto rx=0x7f4ec8002a50 tx=0x7f4ec8004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:40:02.713 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:02.712+0000 7f4ed3400700 1 -- 172.21.15.18:0/3063098865 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4ec8007cc0 con 0x7f4edc1037a0 2024-04-12T21:40:02.714 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:02.712+0000 7f4ed3400700 1 -- 172.21.15.18:0/3063098865 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f4ec800f040 con 0x7f4edc1037a0 2024-04-12T21:40:02.714 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:02.712+0000 7f4ed3400700 1 -- 172.21.15.18:0/3063098865 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4ec800d9c0 con 0x7f4edc1037a0 2024-04-12T21:40:02.714 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:02.712+0000 7f4ee2800700 1 -- 172.21.15.18:0/3063098865 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4edc110000 con 0x7f4edc1037a0 2024-04-12T21:40:02.714 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:02.713+0000 7f4ee2800700 1 -- 172.21.15.18:0/3063098865 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4edc1105c0 con 0x7f4edc1037a0 2024-04-12T21:40:02.714 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:02.714+0000 7f4ee2800700 1 -- 172.21.15.18:0/3063098865 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4edc106a50 con 0x7f4edc1037a0 2024-04-12T21:40:02.715 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:02.715+0000 7f4ed3400700 1 -- 172.21.15.18:0/3063098865 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7f4ec8009730 con 0x7f4edc1037a0 2024-04-12T21:40:02.717 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:02.716+0000 7f4ed3400700 1 --2- 172.21.15.18:0/3063098865 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f4ec006ec00 0x7f4ec00710b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:40:02.717 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:02.716+0000 7f4ed3400700 1 -- 172.21.15.18:0/3063098865 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 6757+0+0 (secure 0 0 0) 0x7f4ec808e630 con 0x7f4edc1037a0 2024-04-12T21:40:02.717 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:02.716+0000 7f4edb400700 1 --2- 172.21.15.18:0/3063098865 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f4ec006ec00 0x7f4ec00710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:40:02.717 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:02.717+0000 7f4edb400700 1 --2- 172.21.15.18:0/3063098865 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f4ec006ec00 0x7f4ec00710b0 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f4ecc00b3b0 tx=0x7f4ecc009e30 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:40:02.722 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:02.721+0000 7f4ed3400700 1 -- 172.21.15.18:0/3063098865 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f4ec80572b0 con 0x7f4edc1037a0 2024-04-12T21:40:02.967 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:02.966+0000 7f4ee2800700 1 -- 172.21.15.18:0/3063098865 --> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f4edc1112a0 con 0x7f4ec006ec00 2024-04-12T21:40:02.969 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:02.968+0000 7f4ed3400700 1 -- 172.21.15.18:0/3063098865 <== mgr.34102 v2:172.21.15.18:6800/3812891919 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f4edc1112a0 con 0x7f4ec006ec00 2024-04-12T21:40:02.972 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:02.971+0000 7f4ee2800700 1 -- 172.21.15.18:0/3063098865 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f4ec006ec00 msgr2=0x7f4ec00710b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:40:02.972 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:02.971+0000 7f4ee2800700 1 --2- 172.21.15.18:0/3063098865 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f4ec006ec00 0x7f4ec00710b0 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f4ecc00b3b0 tx=0x7f4ecc009e30 comp rx=0 tx=0).stop 2024-04-12T21:40:02.972 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:02.971+0000 7f4ee2800700 1 -- 172.21.15.18:0/3063098865 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4edc1037a0 msgr2=0x7f4edc116040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:40:02.972 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:02.971+0000 7f4ee2800700 1 --2- 172.21.15.18:0/3063098865 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4edc1037a0 0x7f4edc116040 secure :-1 s=READY pgs=1019 cs=0 l=1 rev1=1 crypto rx=0x7f4ec8002a50 tx=0x7f4ec8004600 comp rx=0 tx=0).stop 2024-04-12T21:40:02.972 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:02.971+0000 7f4ee2800700 1 -- 172.21.15.18:0/3063098865 shutdown_connections 2024-04-12T21:40:02.972 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:02.971+0000 7f4ee2800700 1 --2- 172.21.15.18:0/3063098865 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f4ec006ec00 0x7f4ec00710b0 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:40:02.972 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:02.971+0000 7f4ee2800700 1 --2- 172.21.15.18:0/3063098865 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4edc102e00 0x7f4edc115b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:40:02.973 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:02.971+0000 7f4ee2800700 1 --2- 172.21.15.18:0/3063098865 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4edc1037a0 0x7f4edc116040 unknown :-1 s=CLOSED pgs=1019 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:40:02.973 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:02.971+0000 7f4ee2800700 1 -- 172.21.15.18:0/3063098865 >> 172.21.15.18:0/3063098865 conn(0x7f4edc0f89b0 msgr2=0x7f4edc1073a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:40:02.973 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:02.971+0000 7f4ee2800700 1 -- 172.21.15.18:0/3063098865 shutdown_connections 2024-04-12T21:40:02.973 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:02.971+0000 7f4ee2800700 1 -- 172.21.15.18:0/3063098865 wait complete. 2024-04-12T21:40:03.131 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:03.129+0000 7fa314a00700 1 -- 172.21.15.18:0/2876586998 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa310105990 msgr2=0x7fa3100ff000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:40:03.131 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:03.129+0000 7fa314a00700 1 --2- 172.21.15.18:0/2876586998 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa310105990 0x7fa3100ff000 secure :-1 s=READY pgs=1020 cs=0 l=1 rev1=1 crypto rx=0x7fa2f8004940 tx=0x7fa2f8009e30 comp rx=0 tx=0).stop 2024-04-12T21:40:03.131 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:03.130+0000 7fa314a00700 1 -- 172.21.15.18:0/2876586998 shutdown_connections 2024-04-12T21:40:03.131 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:03.130+0000 7fa314a00700 1 --2- 172.21.15.18:0/2876586998 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa310105990 0x7fa3100ff000 unknown :-1 s=CLOSED pgs=1020 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:40:03.131 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:03.130+0000 7fa314a00700 1 --2- 172.21.15.18:0/2876586998 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa310104ff0 0x7fa3101053c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:40:03.131 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:03.130+0000 7fa314a00700 1 -- 172.21.15.18:0/2876586998 >> 172.21.15.18:0/2876586998 conn(0x7fa3100faba0 msgr2=0x7fa3100fcfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:40:03.131 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:03.130+0000 7fa314a00700 1 -- 172.21.15.18:0/2876586998 shutdown_connections 2024-04-12T21:40:03.131 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:03.130+0000 7fa314a00700 1 -- 172.21.15.18:0/2876586998 wait complete. 2024-04-12T21:40:03.132 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:03.131+0000 7fa314a00700 1 Processor -- start 2024-04-12T21:40:03.132 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:03.131+0000 7fa314a00700 1 -- start start 2024-04-12T21:40:03.132 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:03.131+0000 7fa314a00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa310104ff0 0x7fa310115db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:40:03.133 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:03.131+0000 7fa314a00700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa310105990 0x7fa3101162f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:40:03.133 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:03.131+0000 7fa314a00700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa310116940 con 0x7fa310104ff0 2024-04-12T21:40:03.133 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:03.131+0000 7fa314a00700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa31010fd10 con 0x7fa310105990 2024-04-12T21:40:03.133 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:03.132+0000 7fa30de00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa310104ff0 0x7fa310115db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:40:03.133 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:03.132+0000 7fa30de00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa310104ff0 0x7fa310115db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:47764/0 (socket says 172.21.15.18:47764) 2024-04-12T21:40:03.133 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:03.132+0000 7fa30de00700 1 -- 172.21.15.18:0/384437024 learned_addr learned my addr 172.21.15.18:0/384437024 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:40:03.134 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:03.132+0000 7fa30de00700 1 -- 172.21.15.18:0/384437024 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa310105990 msgr2=0x7fa3101162f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:40:03.134 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:03.132+0000 7fa30de00700 1 --2- 172.21.15.18:0/384437024 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa310105990 0x7fa3101162f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:40:03.134 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:03.132+0000 7fa30de00700 1 -- 172.21.15.18:0/384437024 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2f8005040 con 0x7fa310104ff0 2024-04-12T21:40:03.134 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:03.132+0000 7fa30de00700 1 --2- 172.21.15.18:0/384437024 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa310104ff0 0x7fa310115db0 secure :-1 s=READY pgs=1021 cs=0 l=1 rev1=1 crypto rx=0x7fa300002a50 tx=0x7fa3000046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:40:03.134 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:03.133+0000 7fa306000700 1 -- 172.21.15.18:0/384437024 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa300007cc0 con 0x7fa310104ff0 2024-04-12T21:40:03.134 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:03.133+0000 7fa306000700 1 -- 172.21.15.18:0/384437024 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa30000f040 con 0x7fa310104ff0 2024-04-12T21:40:03.134 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:03.133+0000 7fa306000700 1 -- 172.21.15.18:0/384437024 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa300009640 con 0x7fa310104ff0 2024-04-12T21:40:03.134 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:03.133+0000 7fa314a00700 1 -- 172.21.15.18:0/384437024 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa31010fff0 con 0x7fa310104ff0 2024-04-12T21:40:03.135 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:03.133+0000 7fa314a00700 1 -- 172.21.15.18:0/384437024 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa3101105b0 con 0x7fa310104ff0 2024-04-12T21:40:03.135 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:03.134+0000 7fa314a00700 1 -- 172.21.15.18:0/384437024 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa310100740 con 0x7fa310104ff0 2024-04-12T21:40:03.136 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:03.135+0000 7fa306000700 1 -- 172.21.15.18:0/384437024 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7fa3000097c0 con 0x7fa310104ff0 2024-04-12T21:40:03.137 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:03.136+0000 7fa306000700 1 --2- 172.21.15.18:0/384437024 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fa2f406ec10 0x7fa2f40710c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:40:03.137 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:03.136+0000 7fa306000700 1 -- 172.21.15.18:0/384437024 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 6757+0+0 (secure 0 0 0) 0x7fa30008e660 con 0x7fa310104ff0 2024-04-12T21:40:03.137 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:03.136+0000 7fa307e00700 1 --2- 172.21.15.18:0/384437024 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fa2f406ec10 0x7fa2f40710c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:40:03.138 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:03.137+0000 7fa307e00700 1 --2- 172.21.15.18:0/384437024 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fa2f406ec10 0x7fa2f40710c0 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7fa2f8004f90 tx=0x7fa2f8009e30 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:40:03.142 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:03.141+0000 7fa306000700 1 -- 172.21.15.18:0/384437024 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7fa30005aeb0 con 0x7fa310104ff0 2024-04-12T21:40:03.380 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:03.378+0000 7fa314a00700 1 -- 172.21.15.18:0/384437024 --> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fa31010c0d0 con 0x7fa2f406ec10 2024-04-12T21:40:03.391 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:03.390+0000 7fa306000700 1 -- 172.21.15.18:0/384437024 <== mgr.34102 v2:172.21.15.18:6800/3812891919 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3520 (secure 0 0 0) 0x7fa31010c0d0 con 0x7fa2f406ec10 2024-04-12T21:40:03.392 INFO:teuthology.orchestra.run.smithi018.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-12T21:40:03.392 INFO:teuthology.orchestra.run.smithi018.stdout:alertmanager.smithi018 smithi018 *:9093,9094 running (40m) 103s ago 55m 14.8M - 0.25.0 c8568f914cd2 c332090068e8 2024-04-12T21:40:03.392 INFO:teuthology.orchestra.run.smithi018.stdout:crash.smithi018 smithi018 running (36m) 103s ago 55m 6665k - 19.0.0-2879-g324baf8d 8717720dc5e5 b7f80d1f7982 2024-04-12T21:40:03.392 INFO:teuthology.orchestra.run.smithi018.stdout:crash.smithi186 smithi186 running (36m) 106s ago 54m 6660k - 19.0.0-2879-g324baf8d 8717720dc5e5 a4f55c81b623 2024-04-12T21:40:03.392 INFO:teuthology.orchestra.run.smithi018.stdout:grafana.smithi018 smithi018 *:3000 running (40m) 103s ago 54m 73.8M - 9.4.12 2bacad6d85d8 b08e3bf176dd 2024-04-12T21:40:03.392 INFO:teuthology.orchestra.run.smithi018.stdout:mds.cephfs.smithi018.acqbsc smithi018 running (32m) 103s ago 48m 21.9M - 19.0.0-2879-g324baf8d 8717720dc5e5 021f99ab24ae 2024-04-12T21:40:03.392 INFO:teuthology.orchestra.run.smithi018.stdout:mds.cephfs.smithi018.szjluo smithi018 running (32m) 103s ago 47m 21.7M - 19.0.0-2879-g324baf8d 8717720dc5e5 0003117479d4 2024-04-12T21:40:03.392 INFO:teuthology.orchestra.run.smithi018.stdout:mds.cephfs.smithi186.jdllyn smithi186 running (17m) 106s ago 47m 16.5M - 19.0.0-2879-g324baf8d 8717720dc5e5 21b1f46eef23 2024-04-12T21:40:03.392 INFO:teuthology.orchestra.run.smithi018.stdout:mds.cephfs.smithi186.wjnafs smithi186 running (111s) 106s ago 47m 11.0M - 19.0.0-2879-g324baf8d 8717720dc5e5 3cf1ef4148e2 2024-04-12T21:40:03.392 INFO:teuthology.orchestra.run.smithi018.stdout:mgr.smithi018.xjzwga smithi018 *:8443,9283,8765 running (43m) 103s ago 56m 517M - 19.0.0-2879-g324baf8d 8717720dc5e5 56088305eab7 2024-04-12T21:40:03.392 INFO:teuthology.orchestra.run.smithi018.stdout:mgr.smithi186.kqcfxk smithi186 *:8443,9283,8765 running (42m) 106s ago 53m 460M - 19.0.0-2879-g324baf8d 8717720dc5e5 09cd766e5e0d 2024-04-12T21:40:03.392 INFO:teuthology.orchestra.run.smithi018.stdout:mon.smithi018 smithi018 running (38m) 103s ago 57m 88.2M 2048M 19.0.0-2879-g324baf8d 8717720dc5e5 df3e201dfed6 2024-04-12T21:40:03.393 INFO:teuthology.orchestra.run.smithi018.stdout:mon.smithi186 smithi186 running (37m) 106s ago 53m 53.6M 2048M 19.0.0-2879-g324baf8d 8717720dc5e5 c77498da9b2b 2024-04-12T21:40:03.393 INFO:teuthology.orchestra.run.smithi018.stdout:node-exporter.smithi018 smithi018 *:9100 running (41m) 103s ago 55m 8468k - 1.5.0 0da6a335fe13 326c3d9c6847 2024-04-12T21:40:03.393 INFO:teuthology.orchestra.run.smithi018.stdout:node-exporter.smithi186 smithi186 *:9100 running (41m) 106s ago 53m 8166k - 1.5.0 0da6a335fe13 ec2112baeecf 2024-04-12T21:40:03.393 INFO:teuthology.orchestra.run.smithi018.stdout:osd.0 smithi018 running (36m) 103s ago 52m 123M 1559M 19.0.0-2879-g324baf8d 8717720dc5e5 60b1273ca047 2024-04-12T21:40:03.393 INFO:teuthology.orchestra.run.smithi018.stdout:osd.1 smithi018 running (35m) 103s ago 52m 95.4M 1559M 19.0.0-2879-g324baf8d 8717720dc5e5 170ce1e5fb8d 2024-04-12T21:40:03.393 INFO:teuthology.orchestra.run.smithi018.stdout:osd.2 smithi018 running (35m) 103s ago 51m 94.6M 1559M 19.0.0-2879-g324baf8d 8717720dc5e5 c2cbc6aa2839 2024-04-12T21:40:03.393 INFO:teuthology.orchestra.run.smithi018.stdout:osd.3 smithi186 running (34m) 106s ago 51m 101M 2581M 19.0.0-2879-g324baf8d 8717720dc5e5 0138e8262e4b 2024-04-12T21:40:03.393 INFO:teuthology.orchestra.run.smithi018.stdout:osd.4 smithi186 running (34m) 106s ago 50m 91.5M 2581M 19.0.0-2879-g324baf8d 8717720dc5e5 b20cb63b25c9 2024-04-12T21:40:03.393 INFO:teuthology.orchestra.run.smithi018.stdout:osd.5 smithi186 running (33m) 106s ago 49m 92.2M 2581M 19.0.0-2879-g324baf8d 8717720dc5e5 2513bea2610e 2024-04-12T21:40:03.393 INFO:teuthology.orchestra.run.smithi018.stdout:prometheus.smithi018 smithi018 *:9095 running (41m) 103s ago 54m 69.6M - 2.43.0 a07b618ecd1d f58841127e21 2024-04-12T21:40:03.395 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:03.394+0000 7fa314a00700 1 -- 172.21.15.18:0/384437024 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fa2f406ec10 msgr2=0x7fa2f40710c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:40:03.395 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:03.394+0000 7fa314a00700 1 --2- 172.21.15.18:0/384437024 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fa2f406ec10 0x7fa2f40710c0 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7fa2f8004f90 tx=0x7fa2f8009e30 comp rx=0 tx=0).stop 2024-04-12T21:40:03.395 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:03.394+0000 7fa314a00700 1 -- 172.21.15.18:0/384437024 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa310104ff0 msgr2=0x7fa310115db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:40:03.395 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:03.394+0000 7fa314a00700 1 --2- 172.21.15.18:0/384437024 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa310104ff0 0x7fa310115db0 secure :-1 s=READY pgs=1021 cs=0 l=1 rev1=1 crypto rx=0x7fa300002a50 tx=0x7fa3000046b0 comp rx=0 tx=0).stop 2024-04-12T21:40:03.396 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:03.394+0000 7fa314a00700 1 -- 172.21.15.18:0/384437024 shutdown_connections 2024-04-12T21:40:03.396 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:03.394+0000 7fa314a00700 1 --2- 172.21.15.18:0/384437024 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fa2f406ec10 0x7fa2f40710c0 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:40:03.396 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:03.394+0000 7fa314a00700 1 --2- 172.21.15.18:0/384437024 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa310104ff0 0x7fa310115db0 unknown :-1 s=CLOSED pgs=1021 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:40:03.396 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:03.394+0000 7fa314a00700 1 --2- 172.21.15.18:0/384437024 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa310105990 0x7fa3101162f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:40:03.396 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:03.394+0000 7fa314a00700 1 -- 172.21.15.18:0/384437024 >> 172.21.15.18:0/384437024 conn(0x7fa3100faba0 msgr2=0x7fa3100fbbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:40:03.396 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:03.394+0000 7fa314a00700 1 -- 172.21.15.18:0/384437024 shutdown_connections 2024-04-12T21:40:03.396 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:03.395+0000 7fa314a00700 1 -- 172.21.15.18:0/384437024 wait complete. 2024-04-12T21:40:03.552 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:03.551+0000 7f2374a00700 1 -- 172.21.15.18:0/3324549517 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f23700fddf0 msgr2=0x7f23700fe1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:40:03.552 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:03.551+0000 7f2374a00700 1 --2- 172.21.15.18:0/3324549517 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f23700fddf0 0x7f23700fe1c0 secure :-1 s=READY pgs=1022 cs=0 l=1 rev1=1 crypto rx=0x7f2360004940 tx=0x7f2360009e30 comp rx=0 tx=0).stop 2024-04-12T21:40:03.553 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:03.551+0000 7f2374a00700 1 -- 172.21.15.18:0/3324549517 shutdown_connections 2024-04-12T21:40:03.553 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:03.551+0000 7f2374a00700 1 --2- 172.21.15.18:0/3324549517 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f23700fe790 0x7f2370106dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:40:03.553 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:03.551+0000 7f2374a00700 1 --2- 172.21.15.18:0/3324549517 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f23700fddf0 0x7f23700fe1c0 unknown :-1 s=CLOSED pgs=1022 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:40:03.553 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:03.551+0000 7f2374a00700 1 -- 172.21.15.18:0/3324549517 >> 172.21.15.18:0/3324549517 conn(0x7f23700f7880 msgr2=0x7f23700f9c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:40:03.553 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:03.551+0000 7f2374a00700 1 -- 172.21.15.18:0/3324549517 shutdown_connections 2024-04-12T21:40:03.553 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:03.551+0000 7f2374a00700 1 -- 172.21.15.18:0/3324549517 wait complete. 2024-04-12T21:40:03.554 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:03.552+0000 7f2374a00700 1 Processor -- start 2024-04-12T21:40:03.554 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:03.553+0000 7f2374a00700 1 -- start start 2024-04-12T21:40:03.554 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:03.553+0000 7f2374a00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f23700fddf0 0x7f2370115b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:40:03.554 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:03.553+0000 7f2374a00700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f23700fe790 0x7f23701160b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:40:03.554 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:03.553+0000 7f2374a00700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2370116700 con 0x7f23700fddf0 2024-04-12T21:40:03.554 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:03.553+0000 7f2374a00700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f237010fce0 con 0x7f23700fe790 2024-04-12T21:40:03.556 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:03.553+0000 7f236de00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f23700fddf0 0x7f2370115b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:40:03.556 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:03.553+0000 7f236de00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f23700fddf0 0x7f2370115b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:47792/0 (socket says 172.21.15.18:47792) 2024-04-12T21:40:03.556 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:03.553+0000 7f236de00700 1 -- 172.21.15.18:0/3750616792 learned_addr learned my addr 172.21.15.18:0/3750616792 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:40:03.556 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:03.554+0000 7f2365400700 1 --2- 172.21.15.18:0/3750616792 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f23700fe790 0x7f23701160b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:40:03.556 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:03.554+0000 7f236de00700 1 -- 172.21.15.18:0/3750616792 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f23700fe790 msgr2=0x7f23701160b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:40:03.556 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:03.554+0000 7f236de00700 1 --2- 172.21.15.18:0/3750616792 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f23700fe790 0x7f23701160b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:40:03.556 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:03.554+0000 7f236de00700 1 -- 172.21.15.18:0/3750616792 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2360005040 con 0x7f23700fddf0 2024-04-12T21:40:03.556 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:03.554+0000 7f236de00700 1 --2- 172.21.15.18:0/3750616792 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f23700fddf0 0x7f2370115b70 secure :-1 s=READY pgs=1023 cs=0 l=1 rev1=1 crypto rx=0x7f236000b3b0 tx=0x7f2360009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:40:03.556 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:03.555+0000 7f2366a00700 1 -- 172.21.15.18:0/3750616792 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f236000e040 con 0x7f23700fddf0 2024-04-12T21:40:03.556 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:03.555+0000 7f2366a00700 1 -- 172.21.15.18:0/3750616792 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f2360013420 con 0x7f23700fddf0 2024-04-12T21:40:03.557 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:03.555+0000 7f2374a00700 1 -- 172.21.15.18:0/3750616792 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f237010ff60 con 0x7f23700fddf0 2024-04-12T21:40:03.557 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:03.555+0000 7f2366a00700 1 -- 172.21.15.18:0/3750616792 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f236001c630 con 0x7f23700fddf0 2024-04-12T21:40:03.557 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:03.555+0000 7f2374a00700 1 -- 172.21.15.18:0/3750616792 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f23701104a0 con 0x7f23700fddf0 2024-04-12T21:40:03.557 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:03.556+0000 7f2374a00700 1 -- 172.21.15.18:0/3750616792 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2370066bf0 con 0x7f23700fddf0 2024-04-12T21:40:03.558 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:03.557+0000 7f2366a00700 1 -- 172.21.15.18:0/3750616792 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7f2360013590 con 0x7f23700fddf0 2024-04-12T21:40:03.559 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:03.558+0000 7f2366a00700 1 --2- 172.21.15.18:0/3750616792 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f235406ec00 0x7f23540710b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:40:03.559 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:03.558+0000 7f2366a00700 1 -- 172.21.15.18:0/3750616792 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 6757+0+0 (secure 0 0 0) 0x7f236008fcf0 con 0x7f23700fddf0 2024-04-12T21:40:03.559 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:03.558+0000 7f2365400700 1 --2- 172.21.15.18:0/3750616792 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f235406ec00 0x7f23540710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:40:03.560 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:03.559+0000 7f2365400700 1 --2- 172.21.15.18:0/3750616792 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f235406ec00 0x7f23540710b0 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f2358009fd0 tx=0x7f2358009f20 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:40:03.564 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:03.563+0000 7f2366a00700 1 -- 172.21.15.18:0/3750616792 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f236005c4a0 con 0x7f23700fddf0 2024-04-12T21:40:03.890 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:03.888+0000 7f2374a00700 1 -- 172.21.15.18:0/3750616792 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f2370109f30 con 0x7f23700fddf0 2024-04-12T21:40:03.890 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:03.890+0000 7f2366a00700 1 -- 172.21.15.18:0/3750616792 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+623 (secure 0 0 0) 0x7f2360013840 con 0x7f23700fddf0 2024-04-12T21:40:03.891 INFO:teuthology.orchestra.run.smithi018.stdout:{ 2024-04-12T21:40:03.891 INFO:teuthology.orchestra.run.smithi018.stdout: "mon": { 2024-04-12T21:40:03.891 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 19.0.0-2879-g324baf8d (324baf8ddb91dea46cac50cb535c306115791355) squid (dev)": 2 2024-04-12T21:40:03.891 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-12T21:40:03.891 INFO:teuthology.orchestra.run.smithi018.stdout: "mgr": { 2024-04-12T21:40:03.891 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 19.0.0-2879-g324baf8d (324baf8ddb91dea46cac50cb535c306115791355) squid (dev)": 2 2024-04-12T21:40:03.891 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-12T21:40:03.891 INFO:teuthology.orchestra.run.smithi018.stdout: "osd": { 2024-04-12T21:40:03.891 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 19.0.0-2879-g324baf8d (324baf8ddb91dea46cac50cb535c306115791355) squid (dev)": 6 2024-04-12T21:40:03.892 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-12T21:40:03.892 INFO:teuthology.orchestra.run.smithi018.stdout: "mds": { 2024-04-12T21:40:03.892 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 19.0.0-2879-g324baf8d (324baf8ddb91dea46cac50cb535c306115791355) squid (dev)": 4 2024-04-12T21:40:03.892 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-12T21:40:03.892 INFO:teuthology.orchestra.run.smithi018.stdout: "overall": { 2024-04-12T21:40:03.892 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 19.0.0-2879-g324baf8d (324baf8ddb91dea46cac50cb535c306115791355) squid (dev)": 14 2024-04-12T21:40:03.892 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-04-12T21:40:03.892 INFO:teuthology.orchestra.run.smithi018.stdout:} 2024-04-12T21:40:03.894 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:03.893+0000 7f2374a00700 1 -- 172.21.15.18:0/3750616792 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f235406ec00 msgr2=0x7f23540710b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:40:03.894 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:03.893+0000 7f2374a00700 1 --2- 172.21.15.18:0/3750616792 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f235406ec00 0x7f23540710b0 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f2358009fd0 tx=0x7f2358009f20 comp rx=0 tx=0).stop 2024-04-12T21:40:03.894 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:03.893+0000 7f2374a00700 1 -- 172.21.15.18:0/3750616792 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f23700fddf0 msgr2=0x7f2370115b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:40:03.894 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:03.893+0000 7f2374a00700 1 --2- 172.21.15.18:0/3750616792 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f23700fddf0 0x7f2370115b70 secure :-1 s=READY pgs=1023 cs=0 l=1 rev1=1 crypto rx=0x7f236000b3b0 tx=0x7f2360009e30 comp rx=0 tx=0).stop 2024-04-12T21:40:03.895 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:03.893+0000 7f2374a00700 1 -- 172.21.15.18:0/3750616792 shutdown_connections 2024-04-12T21:40:03.895 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:03.893+0000 7f2374a00700 1 --2- 172.21.15.18:0/3750616792 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f235406ec00 0x7f23540710b0 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:40:03.895 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:03.893+0000 7f2374a00700 1 --2- 172.21.15.18:0/3750616792 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f23700fddf0 0x7f2370115b70 unknown :-1 s=CLOSED pgs=1023 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:40:03.895 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:03.893+0000 7f2374a00700 1 --2- 172.21.15.18:0/3750616792 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f23700fe790 0x7f23701160b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:40:03.895 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:03.893+0000 7f2374a00700 1 -- 172.21.15.18:0/3750616792 >> 172.21.15.18:0/3750616792 conn(0x7f23700f7880 msgr2=0x7f23700f8d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:40:03.895 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:03.893+0000 7f2374a00700 1 -- 172.21.15.18:0/3750616792 shutdown_connections 2024-04-12T21:40:03.895 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:03.893+0000 7f2374a00700 1 -- 172.21.15.18:0/3750616792 wait complete. 2024-04-12T21:40:04.034 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:40:03 smithi186 ceph-mon[86379]: pgmap v1211: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:40:04.034 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:40:03 smithi186 ceph-mon[86379]: from='client.36104 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:40:04.034 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:40:03 smithi186 ceph-mon[86379]: from='client.36108 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:40:04.054 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:04.053+0000 7f89fac00700 1 -- 172.21.15.18:0/3655174122 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f89f4070360 msgr2=0x7f89f4070730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:40:04.054 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:04.053+0000 7f89fac00700 1 --2- 172.21.15.18:0/3655174122 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f89f4070360 0x7f89f4070730 secure :-1 s=READY pgs=1024 cs=0 l=1 rev1=1 crypto rx=0x7f89e4004970 tx=0x7f89e4009e30 comp rx=0 tx=0).stop 2024-04-12T21:40:04.054 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:04.053+0000 7f89fac00700 1 -- 172.21.15.18:0/3655174122 shutdown_connections 2024-04-12T21:40:04.054 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:04.053+0000 7f89fac00700 1 --2- 172.21.15.18:0/3655174122 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f89f4070c70 0x7f89f4109110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:40:04.054 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:04.053+0000 7f89fac00700 1 --2- 172.21.15.18:0/3655174122 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f89f4070360 0x7f89f4070730 unknown :-1 s=CLOSED pgs=1024 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:40:04.054 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:04.053+0000 7f89fac00700 1 -- 172.21.15.18:0/3655174122 >> 172.21.15.18:0/3655174122 conn(0x7f89f40f89b0 msgr2=0x7f89f40fadc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:40:04.054 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:40:03 smithi018 ceph-mon[112590]: pgmap v1211: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:40:04.055 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:40:03 smithi018 ceph-mon[112590]: from='client.36104 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:40:04.055 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:40:03 smithi018 ceph-mon[112590]: from='client.36108 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:40:04.056 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:04.053+0000 7f89fac00700 1 -- 172.21.15.18:0/3655174122 shutdown_connections 2024-04-12T21:40:04.056 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:04.053+0000 7f89fac00700 1 -- 172.21.15.18:0/3655174122 wait complete. 2024-04-12T21:40:04.056 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:04.054+0000 7f89fac00700 1 Processor -- start 2024-04-12T21:40:04.056 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:04.054+0000 7f89fac00700 1 -- start start 2024-04-12T21:40:04.056 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:04.055+0000 7f89fac00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f89f4070360 0x7f89f4115b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:40:04.056 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:04.055+0000 7f89fac00700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f89f4070c70 0x7f89f41160a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:40:04.056 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:04.055+0000 7f89fac00700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89f41166f0 con 0x7f89f4070360 2024-04-12T21:40:04.056 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:04.055+0000 7f89fac00700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89f410fcd0 con 0x7f89f4070c70 2024-04-12T21:40:04.056 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:04.055+0000 7f89f3e00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f89f4070360 0x7f89f4115b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:40:04.056 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:04.055+0000 7f89f3e00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f89f4070360 0x7f89f4115b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:47806/0 (socket says 172.21.15.18:47806) 2024-04-12T21:40:04.056 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:04.055+0000 7f89f3e00700 1 -- 172.21.15.18:0/1715147650 learned_addr learned my addr 172.21.15.18:0/1715147650 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:40:04.057 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:04.055+0000 7f89f3400700 1 --2- 172.21.15.18:0/1715147650 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f89f4070c70 0x7f89f41160a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:40:04.057 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:04.055+0000 7f89f3e00700 1 -- 172.21.15.18:0/1715147650 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f89f4070c70 msgr2=0x7f89f41160a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:40:04.058 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:04.056+0000 7f89f3e00700 1 --2- 172.21.15.18:0/1715147650 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f89f4070c70 0x7f89f41160a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:40:04.058 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:04.056+0000 7f89f3e00700 1 -- 172.21.15.18:0/1715147650 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89e4005040 con 0x7f89f4070360 2024-04-12T21:40:04.058 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:04.056+0000 7f89f3e00700 1 --2- 172.21.15.18:0/1715147650 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f89f4070360 0x7f89f4115b60 secure :-1 s=READY pgs=1025 cs=0 l=1 rev1=1 crypto rx=0x7f89e4004f50 tx=0x7f89e4009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:40:04.058 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:04.056+0000 7f89ebe00700 1 -- 172.21.15.18:0/1715147650 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f89e400e040 con 0x7f89f4070360 2024-04-12T21:40:04.058 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:04.056+0000 7f89ebe00700 1 -- 172.21.15.18:0/1715147650 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f89e4004290 con 0x7f89f4070360 2024-04-12T21:40:04.058 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:04.056+0000 7f89fac00700 1 -- 172.21.15.18:0/1715147650 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f89f410ff50 con 0x7f89f4070360 2024-04-12T21:40:04.058 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:04.056+0000 7f89ebe00700 1 -- 172.21.15.18:0/1715147650 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f89e401c3f0 con 0x7f89f4070360 2024-04-12T21:40:04.058 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:04.056+0000 7f89fac00700 1 -- 172.21.15.18:0/1715147650 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f89f4110490 con 0x7f89f4070360 2024-04-12T21:40:04.058 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:04.058+0000 7f89fac00700 1 -- 172.21.15.18:0/1715147650 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f89f4071aa0 con 0x7f89f4070360 2024-04-12T21:40:04.059 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:04.059+0000 7f89ebe00700 1 -- 172.21.15.18:0/1715147650 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7f89e400bc10 con 0x7f89f4070360 2024-04-12T21:40:04.061 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:04.060+0000 7f89ebe00700 1 --2- 172.21.15.18:0/1715147650 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f89d806ec00 0x7f89d80710b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:40:04.061 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:04.060+0000 7f89ebe00700 1 -- 172.21.15.18:0/1715147650 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 6757+0+0 (secure 0 0 0) 0x7f89e408fcd0 con 0x7f89f4070360 2024-04-12T21:40:04.061 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:04.060+0000 7f89f3400700 1 --2- 172.21.15.18:0/1715147650 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f89d806ec00 0x7f89d80710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:40:04.061 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:04.061+0000 7f89f3400700 1 --2- 172.21.15.18:0/1715147650 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f89d806ec00 0x7f89d80710b0 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f89f4111400 tx=0x7f89e0009ee0 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:40:04.066 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:04.065+0000 7f89ebe00700 1 -- 172.21.15.18:0/1715147650 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f89e405c480 con 0x7f89f4070360 2024-04-12T21:40:04.343 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:04.342+0000 7f89fac00700 1 -- 172.21.15.18:0/1715147650 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f89f41111b0 con 0x7f89f4070360 2024-04-12T21:40:04.344 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:04.343+0000 7f89ebe00700 1 -- 172.21.15.18:0/1715147650 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 33 v33) v1 ==== 76+0+1876 (secure 0 0 0) 0x7f89e405c010 con 0x7f89f4070360 2024-04-12T21:40:04.344 INFO:teuthology.orchestra.run.smithi018.stdout:e33 2024-04-12T21:40:04.345 INFO:teuthology.orchestra.run.smithi018.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-12T21:40:04.345 INFO:teuthology.orchestra.run.smithi018.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-12T21:40:04.345 INFO:teuthology.orchestra.run.smithi018.stdout:legacy client fscid: 1 2024-04-12T21:40:04.345 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-12T21:40:04.345 INFO:teuthology.orchestra.run.smithi018.stdout:Filesystem 'cephfs' (1) 2024-04-12T21:40:04.345 INFO:teuthology.orchestra.run.smithi018.stdout:fs_name cephfs 2024-04-12T21:40:04.345 INFO:teuthology.orchestra.run.smithi018.stdout:epoch 24 2024-04-12T21:40:04.345 INFO:teuthology.orchestra.run.smithi018.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-04-12T21:40:04.345 INFO:teuthology.orchestra.run.smithi018.stdout:created 2024-04-12T20:51:57.362850+0000 2024-04-12T21:40:04.345 INFO:teuthology.orchestra.run.smithi018.stdout:modified 2024-04-12T21:06:59.717190+0000 2024-04-12T21:40:04.345 INFO:teuthology.orchestra.run.smithi018.stdout:tableserver 0 2024-04-12T21:40:04.345 INFO:teuthology.orchestra.run.smithi018.stdout:root 0 2024-04-12T21:40:04.346 INFO:teuthology.orchestra.run.smithi018.stdout:session_timeout 60 2024-04-12T21:40:04.346 INFO:teuthology.orchestra.run.smithi018.stdout:session_autoclose 300 2024-04-12T21:40:04.346 INFO:teuthology.orchestra.run.smithi018.stdout:max_file_size 1099511627776 2024-04-12T21:40:04.346 INFO:teuthology.orchestra.run.smithi018.stdout:max_xattr_size 65536 2024-04-12T21:40:04.346 INFO:teuthology.orchestra.run.smithi018.stdout:required_client_features {} 2024-04-12T21:40:04.346 INFO:teuthology.orchestra.run.smithi018.stdout:last_failure 0 2024-04-12T21:40:04.346 INFO:teuthology.orchestra.run.smithi018.stdout:last_failure_osd_epoch 100 2024-04-12T21:40:04.346 INFO:teuthology.orchestra.run.smithi018.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-12T21:40:04.346 INFO:teuthology.orchestra.run.smithi018.stdout:max_mds 2 2024-04-12T21:40:04.346 INFO:teuthology.orchestra.run.smithi018.stdout:in 0,1 2024-04-12T21:40:04.346 INFO:teuthology.orchestra.run.smithi018.stdout:up {} 2024-04-12T21:40:04.346 INFO:teuthology.orchestra.run.smithi018.stdout:failed 0,1 2024-04-12T21:40:04.346 INFO:teuthology.orchestra.run.smithi018.stdout:damaged 2024-04-12T21:40:04.347 INFO:teuthology.orchestra.run.smithi018.stdout:stopped 2024-04-12T21:40:04.347 INFO:teuthology.orchestra.run.smithi018.stdout:data_pools [3] 2024-04-12T21:40:04.347 INFO:teuthology.orchestra.run.smithi018.stdout:metadata_pool 2 2024-04-12T21:40:04.347 INFO:teuthology.orchestra.run.smithi018.stdout:inline_data disabled 2024-04-12T21:40:04.347 INFO:teuthology.orchestra.run.smithi018.stdout:balancer 2024-04-12T21:40:04.347 INFO:teuthology.orchestra.run.smithi018.stdout:bal_rank_mask -1 2024-04-12T21:40:04.347 INFO:teuthology.orchestra.run.smithi018.stdout:standby_count_wanted 1 2024-04-12T21:40:04.347 INFO:teuthology.orchestra.run.smithi018.stdout:qdb_cluster leader: 0 members: 2024-04-12T21:40:04.347 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-12T21:40:04.347 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-12T21:40:04.347 INFO:teuthology.orchestra.run.smithi018.stdout:Standby daemons: 2024-04-12T21:40:04.347 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-12T21:40:04.347 INFO:teuthology.orchestra.run.smithi018.stdout:[mds.cephfs.smithi018.acqbsc{-1:34436} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.18:6828/755179764,v1:172.21.15.18:6829/755179764] compat {c=[1],r=[1],i=[1fff]}] 2024-04-12T21:40:04.347 INFO:teuthology.orchestra.run.smithi018.stdout:[mds.cephfs.smithi018.szjluo{-1:34468} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.18:6826/2407992653,v1:172.21.15.18:6827/2407992653] compat {c=[1],r=[1],i=[1fff]}] 2024-04-12T21:40:04.347 INFO:teuthology.orchestra.run.smithi018.stdout:[mds.cephfs.smithi186.jdllyn{-1:44595} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.186:6824/3523444202,v1:172.21.15.186:6825/3523444202] compat {c=[1],r=[1],i=[1fff]}] 2024-04-12T21:40:04.348 INFO:teuthology.orchestra.run.smithi018.stdout:[mds.cephfs.smithi186.wjnafs{-1:44875} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.186:6826/3122443766,v1:172.21.15.186:6827/3122443766] compat {c=[1],r=[1],i=[1fff]}] 2024-04-12T21:40:04.348 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:04.346+0000 7f89fac00700 1 -- 172.21.15.18:0/1715147650 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f89d806ec00 msgr2=0x7f89d80710b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:40:04.348 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:04.346+0000 7f89fac00700 1 --2- 172.21.15.18:0/1715147650 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f89d806ec00 0x7f89d80710b0 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f89f4111400 tx=0x7f89e0009ee0 comp rx=0 tx=0).stop 2024-04-12T21:40:04.348 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:04.346+0000 7f89fac00700 1 -- 172.21.15.18:0/1715147650 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f89f4070360 msgr2=0x7f89f4115b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:40:04.348 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:04.346+0000 7f89fac00700 1 --2- 172.21.15.18:0/1715147650 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f89f4070360 0x7f89f4115b60 secure :-1 s=READY pgs=1025 cs=0 l=1 rev1=1 crypto rx=0x7f89e4004f50 tx=0x7f89e4009e30 comp rx=0 tx=0).stop 2024-04-12T21:40:04.348 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:04.346+0000 7f89fac00700 1 -- 172.21.15.18:0/1715147650 shutdown_connections 2024-04-12T21:40:04.349 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:04.346+0000 7f89fac00700 1 --2- 172.21.15.18:0/1715147650 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f89d806ec00 0x7f89d80710b0 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:40:04.349 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:04.346+0000 7f89fac00700 1 --2- 172.21.15.18:0/1715147650 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f89f4070360 0x7f89f4115b60 unknown :-1 s=CLOSED pgs=1025 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:40:04.349 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:04.346+0000 7f89fac00700 1 --2- 172.21.15.18:0/1715147650 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f89f4070c70 0x7f89f41160a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:40:04.349 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:04.346+0000 7f89fac00700 1 -- 172.21.15.18:0/1715147650 >> 172.21.15.18:0/1715147650 conn(0x7f89f40f89b0 msgr2=0x7f89f40f9de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:40:04.349 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:04.346+0000 7f89fac00700 1 -- 172.21.15.18:0/1715147650 shutdown_connections 2024-04-12T21:40:04.349 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:04.346+0000 7f89fac00700 1 -- 172.21.15.18:0/1715147650 wait complete. 2024-04-12T21:40:04.349 INFO:teuthology.orchestra.run.smithi018.stderr:dumped fsmap epoch 33 2024-04-12T21:40:04.504 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:04.503+0000 7fa5ccc00700 1 -- 172.21.15.18:0/3444456728 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa5c8074090 msgr2=0x7fa5c8074460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:40:04.504 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:04.503+0000 7fa5ccc00700 1 --2- 172.21.15.18:0/3444456728 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa5c8074090 0x7fa5c8074460 secure :-1 s=READY pgs=1026 cs=0 l=1 rev1=1 crypto rx=0x7fa5b8004970 tx=0x7fa5b8009e30 comp rx=0 tx=0).stop 2024-04-12T21:40:04.504 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:04.503+0000 7fa5ccc00700 1 -- 172.21.15.18:0/3444456728 shutdown_connections 2024-04-12T21:40:04.504 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:04.503+0000 7fa5ccc00700 1 --2- 172.21.15.18:0/3444456728 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa5c80749a0 0x7fa5c810d790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:40:04.504 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:04.503+0000 7fa5ccc00700 1 --2- 172.21.15.18:0/3444456728 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa5c8074090 0x7fa5c8074460 unknown :-1 s=CLOSED pgs=1026 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:40:04.505 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:04.503+0000 7fa5ccc00700 1 -- 172.21.15.18:0/3444456728 >> 172.21.15.18:0/3444456728 conn(0x7fa5c80fab70 msgr2=0x7fa5c80fcf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:40:04.505 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:04.503+0000 7fa5ccc00700 1 -- 172.21.15.18:0/3444456728 shutdown_connections 2024-04-12T21:40:04.505 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:04.503+0000 7fa5ccc00700 1 -- 172.21.15.18:0/3444456728 wait complete. 2024-04-12T21:40:04.506 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:04.504+0000 7fa5ccc00700 1 Processor -- start 2024-04-12T21:40:04.506 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:04.504+0000 7fa5ccc00700 1 -- start start 2024-04-12T21:40:04.506 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:04.505+0000 7fa5ccc00700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa5c8074090 0x7fa5c8119e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:40:04.506 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:04.505+0000 7fa5ccc00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa5c80749a0 0x7fa5c811a3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:40:04.506 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:04.505+0000 7fa5ccc00700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5c811aa00 con 0x7fa5c80749a0 2024-04-12T21:40:04.506 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:04.505+0000 7fa5ccc00700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5c8114090 con 0x7fa5c8074090 2024-04-12T21:40:04.506 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:04.505+0000 7fa5c5400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa5c80749a0 0x7fa5c811a3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:40:04.506 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:04.505+0000 7fa5c5400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa5c80749a0 0x7fa5c811a3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:47824/0 (socket says 172.21.15.18:47824) 2024-04-12T21:40:04.506 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:04.505+0000 7fa5c5400700 1 -- 172.21.15.18:0/3034845583 learned_addr learned my addr 172.21.15.18:0/3034845583 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:40:04.507 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:04.505+0000 7fa5c5e00700 1 --2- 172.21.15.18:0/3034845583 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa5c8074090 0x7fa5c8119e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:40:04.507 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:04.505+0000 7fa5c5400700 1 -- 172.21.15.18:0/3034845583 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa5c8074090 msgr2=0x7fa5c8119e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:40:04.507 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:04.505+0000 7fa5c5400700 1 --2- 172.21.15.18:0/3034845583 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa5c8074090 0x7fa5c8119e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:40:04.507 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:04.505+0000 7fa5c5400700 1 -- 172.21.15.18:0/3034845583 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5b8005040 con 0x7fa5c80749a0 2024-04-12T21:40:04.508 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:04.506+0000 7fa5c5400700 1 --2- 172.21.15.18:0/3034845583 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa5c80749a0 0x7fa5c811a3b0 secure :-1 s=READY pgs=1027 cs=0 l=1 rev1=1 crypto rx=0x7fa5b400c830 tx=0x7fa5b4004470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:40:04.508 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:04.506+0000 7fa5be000700 1 -- 172.21.15.18:0/3034845583 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa5b400a760 con 0x7fa5c80749a0 2024-04-12T21:40:04.508 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:04.506+0000 7fa5be000700 1 -- 172.21.15.18:0/3034845583 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa5b400ad90 con 0x7fa5c80749a0 2024-04-12T21:40:04.508 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:04.506+0000 7fa5be000700 1 -- 172.21.15.18:0/3034845583 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa5b40113a0 con 0x7fa5c80749a0 2024-04-12T21:40:04.508 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:04.506+0000 7fa5ccc00700 1 -- 172.21.15.18:0/3034845583 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa5c8114370 con 0x7fa5c80749a0 2024-04-12T21:40:04.508 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:04.506+0000 7fa5ccc00700 1 -- 172.21.15.18:0/3034845583 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa5c81148b0 con 0x7fa5c80749a0 2024-04-12T21:40:04.508 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:04.507+0000 7fa5ccc00700 1 -- 172.21.15.18:0/3034845583 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa5c8073c30 con 0x7fa5c80749a0 2024-04-12T21:40:04.509 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:04.508+0000 7fa5be000700 1 -- 172.21.15.18:0/3034845583 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7fa5b4012400 con 0x7fa5c80749a0 2024-04-12T21:40:04.511 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:04.510+0000 7fa5be000700 1 --2- 172.21.15.18:0/3034845583 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fa5ac06ec00 0x7fa5ac0710b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:40:04.511 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:04.510+0000 7fa5be000700 1 -- 172.21.15.18:0/3034845583 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 6757+0+0 (secure 0 0 0) 0x7fa5b408e9c0 con 0x7fa5c80749a0 2024-04-12T21:40:04.511 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:04.510+0000 7fa5c5e00700 1 --2- 172.21.15.18:0/3034845583 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fa5ac06ec00 0x7fa5ac0710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:40:04.511 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:04.511+0000 7fa5c5e00700 1 --2- 172.21.15.18:0/3034845583 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fa5ac06ec00 0x7fa5ac0710b0 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7fa5b8004f50 tx=0x7fa5b8009e30 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:40:04.515 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:04.515+0000 7fa5be000700 1 -- 172.21.15.18:0/3034845583 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7fa5b40576c0 con 0x7fa5c80749a0 2024-04-12T21:40:04.754 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:04.754+0000 7fa5ccc00700 1 -- 172.21.15.18:0/3034845583 --> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fa5c81155e0 con 0x7fa5ac06ec00 2024-04-12T21:40:04.756 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:04.756+0000 7fa5be000700 1 -- 172.21.15.18:0/3034845583 <== mgr.34102 v2:172.21.15.18:6800/3812891919 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7fa5c81155e0 con 0x7fa5ac06ec00 2024-04-12T21:40:04.757 INFO:teuthology.orchestra.run.smithi018.stdout:{ 2024-04-12T21:40:04.757 INFO:teuthology.orchestra.run.smithi018.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:f4452def3be4d350f4a8e5feaf7ebb1511be9298fe68478ba7a2d6fbd7d69a63", 2024-04-12T21:40:04.757 INFO:teuthology.orchestra.run.smithi018.stdout: "in_progress": true, 2024-04-12T21:40:04.757 INFO:teuthology.orchestra.run.smithi018.stdout: "which": "Upgrading all daemon types on all hosts", 2024-04-12T21:40:04.757 INFO:teuthology.orchestra.run.smithi018.stdout: "services_complete": [ 2024-04-12T21:40:04.757 INFO:teuthology.orchestra.run.smithi018.stdout: "mgr", 2024-04-12T21:40:04.757 INFO:teuthology.orchestra.run.smithi018.stdout: "mds", 2024-04-12T21:40:04.757 INFO:teuthology.orchestra.run.smithi018.stdout: "crash", 2024-04-12T21:40:04.757 INFO:teuthology.orchestra.run.smithi018.stdout: "osd", 2024-04-12T21:40:04.758 INFO:teuthology.orchestra.run.smithi018.stdout: "mon" 2024-04-12T21:40:04.758 INFO:teuthology.orchestra.run.smithi018.stdout: ], 2024-04-12T21:40:04.758 INFO:teuthology.orchestra.run.smithi018.stdout: "progress": "16/21 daemons upgraded", 2024-04-12T21:40:04.758 INFO:teuthology.orchestra.run.smithi018.stdout: "message": "Currently upgrading mds daemons", 2024-04-12T21:40:04.758 INFO:teuthology.orchestra.run.smithi018.stdout: "is_paused": false 2024-04-12T21:40:04.758 INFO:teuthology.orchestra.run.smithi018.stdout:} 2024-04-12T21:40:04.759 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:04.758+0000 7fa5ccc00700 1 -- 172.21.15.18:0/3034845583 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fa5ac06ec00 msgr2=0x7fa5ac0710b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:40:04.759 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:04.758+0000 7fa5ccc00700 1 --2- 172.21.15.18:0/3034845583 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fa5ac06ec00 0x7fa5ac0710b0 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7fa5b8004f50 tx=0x7fa5b8009e30 comp rx=0 tx=0).stop 2024-04-12T21:40:04.759 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:04.758+0000 7fa5ccc00700 1 -- 172.21.15.18:0/3034845583 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa5c80749a0 msgr2=0x7fa5c811a3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:40:04.759 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:04.758+0000 7fa5ccc00700 1 --2- 172.21.15.18:0/3034845583 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa5c80749a0 0x7fa5c811a3b0 secure :-1 s=READY pgs=1027 cs=0 l=1 rev1=1 crypto rx=0x7fa5b400c830 tx=0x7fa5b4004470 comp rx=0 tx=0).stop 2024-04-12T21:40:04.760 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:04.758+0000 7fa5ccc00700 1 -- 172.21.15.18:0/3034845583 shutdown_connections 2024-04-12T21:40:04.760 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:04.758+0000 7fa5ccc00700 1 --2- 172.21.15.18:0/3034845583 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fa5ac06ec00 0x7fa5ac0710b0 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:40:04.760 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:04.758+0000 7fa5ccc00700 1 --2- 172.21.15.18:0/3034845583 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa5c8074090 0x7fa5c8119e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:40:04.760 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:04.758+0000 7fa5ccc00700 1 --2- 172.21.15.18:0/3034845583 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa5c80749a0 0x7fa5c811a3b0 unknown :-1 s=CLOSED pgs=1027 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:40:04.760 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:04.758+0000 7fa5ccc00700 1 -- 172.21.15.18:0/3034845583 >> 172.21.15.18:0/3034845583 conn(0x7fa5c80fab70 msgr2=0x7fa5c80751a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:40:04.760 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:04.759+0000 7fa5ccc00700 1 -- 172.21.15.18:0/3034845583 shutdown_connections 2024-04-12T21:40:04.760 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:04.759+0000 7fa5ccc00700 1 -- 172.21.15.18:0/3034845583 wait complete. 2024-04-12T21:40:04.920 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:04.919+0000 7fdfbf200700 1 -- 172.21.15.18:0/2471199311 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fdfc01092a0 msgr2=0x7fdfc0109670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:40:04.920 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:04.919+0000 7fdfbf200700 1 --2- 172.21.15.18:0/2471199311 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fdfc01092a0 0x7fdfc0109670 secure :-1 s=READY pgs=1028 cs=0 l=1 rev1=1 crypto rx=0x7fdfb0004940 tx=0x7fdfb0009e30 comp rx=0 tx=0).stop 2024-04-12T21:40:04.920 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:04.919+0000 7fdfbf200700 1 -- 172.21.15.18:0/2471199311 shutdown_connections 2024-04-12T21:40:04.920 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:04.919+0000 7fdfbf200700 1 --2- 172.21.15.18:0/2471199311 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdfc0109c40 0x7fdfc01032b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:40:04.920 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:04.919+0000 7fdfbf200700 1 --2- 172.21.15.18:0/2471199311 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fdfc01092a0 0x7fdfc0109670 unknown :-1 s=CLOSED pgs=1028 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:40:04.921 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:04.919+0000 7fdfbf200700 1 -- 172.21.15.18:0/2471199311 >> 172.21.15.18:0/2471199311 conn(0x7fdfc0072a30 msgr2=0x7fdfc0074e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:40:04.921 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:40:04 smithi018 ceph-mon[112590]: from='client.36112 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:40:04.921 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:40:04 smithi018 ceph-mon[112590]: from='client.? 172.21.15.18:0/3750616792' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T21:40:04.921 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:40:04 smithi018 ceph-mon[112590]: from='client.? 172.21.15.18:0/1715147650' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-12T21:40:04.922 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:04.920+0000 7fdfbf200700 1 -- 172.21.15.18:0/2471199311 shutdown_connections 2024-04-12T21:40:04.922 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:04.920+0000 7fdfbf200700 1 -- 172.21.15.18:0/2471199311 wait complete. 2024-04-12T21:40:04.922 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:04.920+0000 7fdfbf200700 1 Processor -- start 2024-04-12T21:40:04.922 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:04.921+0000 7fdfbf200700 1 -- start start 2024-04-12T21:40:04.923 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:04.921+0000 7fdfbf200700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdfc01092a0 0x7fdfc0119fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:40:04.923 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:04.921+0000 7fdfbf200700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fdfc0109c40 0x7fdfc011a520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:40:04.923 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:04.921+0000 7fdfbf200700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdfc011ab70 con 0x7fdfc0109c40 2024-04-12T21:40:04.923 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:04.921+0000 7fdfbf200700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdfc0113f40 con 0x7fdfc01092a0 2024-04-12T21:40:04.923 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:04.921+0000 7fdfbd400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fdfc0109c40 0x7fdfc011a520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:40:04.923 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:04.921+0000 7fdfbd400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fdfc0109c40 0x7fdfc011a520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:47828/0 (socket says 172.21.15.18:47828) 2024-04-12T21:40:04.923 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:04.922+0000 7fdfbd400700 1 -- 172.21.15.18:0/2330209769 learned_addr learned my addr 172.21.15.18:0/2330209769 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:40:04.923 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:04.922+0000 7fdfbde00700 1 --2- 172.21.15.18:0/2330209769 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdfc01092a0 0x7fdfc0119fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:40:04.924 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:04.922+0000 7fdfbd400700 1 -- 172.21.15.18:0/2330209769 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdfc01092a0 msgr2=0x7fdfc0119fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:40:04.924 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:04.922+0000 7fdfbd400700 1 --2- 172.21.15.18:0/2330209769 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdfc01092a0 0x7fdfc0119fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:40:04.924 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:04.922+0000 7fdfbd400700 1 -- 172.21.15.18:0/2330209769 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdfb0005040 con 0x7fdfc0109c40 2024-04-12T21:40:04.924 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:04.922+0000 7fdfbd400700 1 --2- 172.21.15.18:0/2330209769 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fdfc0109c40 0x7fdfc011a520 secure :-1 s=READY pgs=1029 cs=0 l=1 rev1=1 crypto rx=0x7fdfac002a80 tx=0x7fdfac004630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:40:04.924 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:04.923+0000 7fdfb6000700 1 -- 172.21.15.18:0/2330209769 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdfac007cc0 con 0x7fdfc0109c40 2024-04-12T21:40:04.924 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:04.923+0000 7fdfb6000700 1 -- 172.21.15.18:0/2330209769 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fdfac00f040 con 0x7fdfc0109c40 2024-04-12T21:40:04.924 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:04.923+0000 7fdfb6000700 1 -- 172.21.15.18:0/2330209769 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdfac00daf0 con 0x7fdfc0109c40 2024-04-12T21:40:04.925 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:04.923+0000 7fdfbf200700 1 -- 172.21.15.18:0/2330209769 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdfc0114220 con 0x7fdfc0109c40 2024-04-12T21:40:04.925 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:04.923+0000 7fdfbf200700 1 -- 172.21.15.18:0/2330209769 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdfc01147e0 con 0x7fdfc0109c40 2024-04-12T21:40:04.925 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:04.924+0000 7fdfbf200700 1 -- 172.21.15.18:0/2330209769 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdfc01049f0 con 0x7fdfc0109c40 2024-04-12T21:40:04.926 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:04.925+0000 7fdfb6000700 1 -- 172.21.15.18:0/2330209769 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7fdfac00dc90 con 0x7fdfc0109c40 2024-04-12T21:40:04.927 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:04.926+0000 7fdfb6000700 1 --2- 172.21.15.18:0/2330209769 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fdfa406ec00 0x7fdfa40710b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:40:04.927 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:04.927+0000 7fdfb6000700 1 -- 172.21.15.18:0/2330209769 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 6757+0+0 (secure 0 0 0) 0x7fdfac08e070 con 0x7fdfc0109c40 2024-04-12T21:40:04.928 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:04.927+0000 7fdfbde00700 1 --2- 172.21.15.18:0/2330209769 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fdfa406ec00 0x7fdfa40710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:40:04.928 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:04.927+0000 7fdfbde00700 1 --2- 172.21.15.18:0/2330209769 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fdfa406ec00 0x7fdfa40710b0 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7fdfb000b3b0 tx=0x7fdfb0009e30 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:40:04.932 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:04.931+0000 7fdfb6000700 1 -- 172.21.15.18:0/2330209769 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7fdfac05a820 con 0x7fdfc0109c40 2024-04-12T21:40:05.033 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:40:04 smithi186 ceph-mon[86379]: from='client.36112 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:40:05.033 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:40:04 smithi186 ceph-mon[86379]: from='client.? 172.21.15.18:0/3750616792' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T21:40:05.033 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:40:04 smithi186 ceph-mon[86379]: from='client.? 172.21.15.18:0/1715147650' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-12T21:40:05.262 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:05.261+0000 7fdfbf200700 1 -- 172.21.15.18:0/2330209769 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7fdfc0076560 con 0x7fdfc0109c40 2024-04-12T21:40:05.262 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:05.262+0000 7fdfb6000700 1 -- 172.21.15.18:0/2330209769 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+4303 (secure 0 0 0) 0x7fdfac05a3b0 con 0x7fdfc0109c40 2024-04-12T21:40:05.263 INFO:teuthology.orchestra.run.smithi018.stdout:HEALTH_ERR failed to probe daemons or devices; 1 filesystem is degraded; 1 filesystem is offline 2024-04-12T21:40:05.263 INFO:teuthology.orchestra.run.smithi018.stdout:[WRN] CEPHADM_REFRESH_FAILED: failed to probe daemons or devices 2024-04-12T21:40:05.263 INFO:teuthology.orchestra.run.smithi018.stdout: host smithi186 `cephadm ceph-volume` failed: cephadm exited with an error code: 1, stderr: Inferring config /var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/mon.smithi186/config 2024-04-12T21:40:05.263 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/podman: timeout after 895 seconds 2024-04-12T21:40:05.263 INFO:teuthology.orchestra.run.smithi018.stdout:Non-zero exit code 124 from /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:f4452def3be4d350f4a8e5feaf7ebb1511be9298fe68478ba7a2d6fbd7d69a63 -e NODE_NAME=smithi186 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968:/var/run/ceph:z -v /var/log/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968:/var/log/ceph:z -v /var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmptgom53f9:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:f4452def3be4d350f4a8e5feaf7ebb1511be9298fe68478ba7a2d6fbd7d69a63 inventory --format=json-pretty --filter-for-batch 2024-04-12T21:40:05.263 INFO:teuthology.orchestra.run.smithi018.stdout:Traceback (most recent call last): 2024-04-12T21:40:05.263 INFO:teuthology.orchestra.run.smithi018.stdout: File "/usr/lib64/python3.9/runpy.py", line 197, in _run_module_as_main 2024-04-12T21:40:05.264 INFO:teuthology.orchestra.run.smithi018.stdout: return _run_code(code, main_globals, None, 2024-04-12T21:40:05.264 INFO:teuthology.orchestra.run.smithi018.stdout: File "/usr/lib64/python3.9/runpy.py", line 87, in _run_code 2024-04-12T21:40:05.264 INFO:teuthology.orchestra.run.smithi018.stdout: exec(code, run_globals) 2024-04-12T21:40:05.264 INFO:teuthology.orchestra.run.smithi018.stdout: File "/var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/cephadm.e44f3ed5e34e4ef52eedef233d1190bff004470d8e8005127b5312bff357fa7f/__main__.py", line 5618, in 2024-04-12T21:40:05.264 INFO:teuthology.orchestra.run.smithi018.stdout: File "/var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/cephadm.e44f3ed5e34e4ef52eedef233d1190bff004470d8e8005127b5312bff357fa7f/__main__.py", line 5606, in main 2024-04-12T21:40:05.264 INFO:teuthology.orchestra.run.smithi018.stdout: File "/var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/cephadm.e44f3ed5e34e4ef52eedef233d1190bff004470d8e8005127b5312bff357fa7f/__main__.py", line 408, in _infer_config 2024-04-12T21:40:05.264 INFO:teuthology.orchestra.run.smithi018.stdout: File "/var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/cephadm.e44f3ed5e34e4ef52eedef233d1190bff004470d8e8005127b5312bff357fa7f/__main__.py", line 323, in _infer_fsid 2024-04-12T21:40:05.264 INFO:teuthology.orchestra.run.smithi018.stdout: File "/var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/cephadm.e44f3ed5e34e4ef52eedef233d1190bff004470d8e8005127b5312bff357fa7f/__main__.py", line 436, in _infer_image 2024-04-12T21:40:05.264 INFO:teuthology.orchestra.run.smithi018.stdout: File "/var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/cephadm.e44f3ed5e34e4ef52eedef233d1190bff004470d8e8005127b5312bff357fa7f/__main__.py", line 310, in _validate_fsid 2024-04-12T21:40:05.264 INFO:teuthology.orchestra.run.smithi018.stdout: File "/var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/cephadm.e44f3ed5e34e4ef52eedef233d1190bff004470d8e8005127b5312bff357fa7f/__main__.py", line 3359, in command_ceph_volume 2024-04-12T21:40:05.264 INFO:teuthology.orchestra.run.smithi018.stdout: File "/var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/cephadm.e44f3ed5e34e4ef52eedef233d1190bff004470d8e8005127b5312bff357fa7f/cephadmlib/call_wrappers.py", line 310, in call_throws 2024-04-12T21:40:05.264 INFO:teuthology.orchestra.run.smithi018.stdout:RuntimeError: Failed command: /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:f4452def3be4d350f4a8e5feaf7ebb1511be9298fe68478ba7a2d6fbd7d69a63 -e NODE_NAME=smithi186 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968:/var/run/ceph:z -v /var/log/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968:/var/log/ceph:z -v /var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmptgom53f9:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:f4452def3be4d350f4a8e5feaf7ebb1511be9298fe68478ba7a2d6fbd7d69a63 inventory --format=json-pretty --filter-for-batch 2024-04-12T21:40:05.264 INFO:teuthology.orchestra.run.smithi018.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-04-12T21:40:05.265 INFO:teuthology.orchestra.run.smithi018.stdout: fs cephfs is degraded 2024-04-12T21:40:05.265 INFO:teuthology.orchestra.run.smithi018.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-04-12T21:40:05.265 INFO:teuthology.orchestra.run.smithi018.stdout: fs cephfs is offline because no MDS is active for it. 2024-04-12T21:40:05.266 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:05.265+0000 7fdfbf200700 1 -- 172.21.15.18:0/2330209769 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fdfa406ec00 msgr2=0x7fdfa40710b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:40:05.266 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:05.265+0000 7fdfbf200700 1 --2- 172.21.15.18:0/2330209769 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fdfa406ec00 0x7fdfa40710b0 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7fdfb000b3b0 tx=0x7fdfb0009e30 comp rx=0 tx=0).stop 2024-04-12T21:40:05.266 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:05.265+0000 7fdfbf200700 1 -- 172.21.15.18:0/2330209769 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fdfc0109c40 msgr2=0x7fdfc011a520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:40:05.266 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:05.265+0000 7fdfbf200700 1 --2- 172.21.15.18:0/2330209769 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fdfc0109c40 0x7fdfc011a520 secure :-1 s=READY pgs=1029 cs=0 l=1 rev1=1 crypto rx=0x7fdfac002a80 tx=0x7fdfac004630 comp rx=0 tx=0).stop 2024-04-12T21:40:05.266 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:05.265+0000 7fdfbf200700 1 -- 172.21.15.18:0/2330209769 shutdown_connections 2024-04-12T21:40:05.267 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:05.265+0000 7fdfbf200700 1 --2- 172.21.15.18:0/2330209769 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fdfa406ec00 0x7fdfa40710b0 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:40:05.267 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:05.265+0000 7fdfbf200700 1 --2- 172.21.15.18:0/2330209769 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdfc01092a0 0x7fdfc0119fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:40:05.267 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:05.265+0000 7fdfbf200700 1 --2- 172.21.15.18:0/2330209769 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fdfc0109c40 0x7fdfc011a520 unknown :-1 s=CLOSED pgs=1029 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:40:05.267 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:05.265+0000 7fdfbf200700 1 -- 172.21.15.18:0/2330209769 >> 172.21.15.18:0/2330209769 conn(0x7fdfc0072a30 msgr2=0x7fdfc0074960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:40:05.267 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:05.265+0000 7fdfbf200700 1 -- 172.21.15.18:0/2330209769 shutdown_connections 2024-04-12T21:40:05.267 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:05.265+0000 7fdfbf200700 1 -- 172.21.15.18:0/2330209769 wait complete. 2024-04-12T21:40:06.033 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:40:05 smithi186 ceph-mon[86379]: pgmap v1212: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:40:06.034 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:40:05 smithi186 ceph-mon[86379]: from='client.36124 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:40:06.034 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:40:05 smithi186 ceph-mon[86379]: from='client.? 172.21.15.18:0/2330209769' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-12T21:40:06.104 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:40:05 smithi018 ceph-mon[112590]: pgmap v1212: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:40:06.104 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:40:05 smithi018 ceph-mon[112590]: from='client.36124 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:40:06.104 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:40:05 smithi018 ceph-mon[112590]: from='client.? 172.21.15.18:0/2330209769' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-12T21:40:08.033 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:40:07 smithi186 ceph-mon[86379]: pgmap v1213: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:40:08.104 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:40:07 smithi018 ceph-mon[112590]: pgmap v1213: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:40:10.033 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:40:09 smithi186 ceph-mon[86379]: pgmap v1214: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:40:10.104 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:40:09 smithi018 ceph-mon[112590]: pgmap v1214: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:40:11.033 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:40:10 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T21:40:11.104 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:40:10 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T21:40:11.854 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:40:11 smithi018 ceph-mon[112590]: pgmap v1215: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:40:12.033 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:40:11 smithi186 ceph-mon[86379]: pgmap v1215: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:40:14.033 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:40:13 smithi186 ceph-mon[86379]: pgmap v1216: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:40:14.104 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:40:13 smithi018 ceph-mon[112590]: pgmap v1216: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:40:16.033 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:40:15 smithi186 ceph-mon[86379]: pgmap v1217: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:40:16.104 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:40:15 smithi018 ceph-mon[112590]: pgmap v1217: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:40:18.033 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:40:17 smithi186 ceph-mon[86379]: pgmap v1218: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:40:18.104 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:40:17 smithi018 ceph-mon[112590]: pgmap v1218: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:40:20.033 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:40:19 smithi186 ceph-mon[86379]: pgmap v1219: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:40:20.104 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:40:19 smithi018 ceph-mon[112590]: pgmap v1219: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:40:22.033 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:40:21 smithi186 ceph-mon[86379]: pgmap v1220: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:40:22.104 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:40:21 smithi018 ceph-mon[112590]: pgmap v1220: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:40:24.033 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:40:23 smithi186 ceph-mon[86379]: pgmap v1221: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:40:24.104 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:40:23 smithi018 ceph-mon[112590]: pgmap v1221: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:40:26.033 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:40:25 smithi186 ceph-mon[86379]: pgmap v1222: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:40:26.033 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:40:25 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T21:40:26.104 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:40:25 smithi018 ceph-mon[112590]: pgmap v1222: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:40:26.104 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:40:25 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T21:40:28.033 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:40:27 smithi186 ceph-mon[86379]: pgmap v1223: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:40:28.104 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:40:27 smithi018 ceph-mon[112590]: pgmap v1223: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:40:30.033 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:40:29 smithi186 ceph-mon[86379]: pgmap v1224: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:40:30.104 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:40:29 smithi018 ceph-mon[112590]: pgmap v1224: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:40:32.033 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:40:31 smithi186 ceph-mon[86379]: pgmap v1225: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:40:32.104 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:40:31 smithi018 ceph-mon[112590]: pgmap v1225: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:40:34.033 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:40:33 smithi186 ceph-mon[86379]: pgmap v1226: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:40:34.104 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:40:33 smithi018 ceph-mon[112590]: pgmap v1226: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:40:35.439 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:35.437+0000 7fac7f000700 1 -- 172.21.15.18:0/248877690 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fac78109430 msgr2=0x7fac78109800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:40:35.439 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:35.437+0000 7fac7f000700 1 --2- 172.21.15.18:0/248877690 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fac78109430 0x7fac78109800 secure :-1 s=READY pgs=1030 cs=0 l=1 rev1=1 crypto rx=0x7fac68004940 tx=0x7fac68009e30 comp rx=0 tx=0).stop 2024-04-12T21:40:35.439 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:35.437+0000 7fac7f000700 1 -- 172.21.15.18:0/248877690 shutdown_connections 2024-04-12T21:40:35.439 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:35.437+0000 7fac7f000700 1 --2- 172.21.15.18:0/248877690 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fac78109dd0 0x7fac78103440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:40:35.439 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:35.437+0000 7fac7f000700 1 --2- 172.21.15.18:0/248877690 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fac78109430 0x7fac78109800 unknown :-1 s=CLOSED pgs=1030 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:40:35.439 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:35.437+0000 7fac7f000700 1 -- 172.21.15.18:0/248877690 >> 172.21.15.18:0/248877690 conn(0x7fac78072c00 msgr2=0x7fac78075010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:40:35.439 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:35.437+0000 7fac7f000700 1 -- 172.21.15.18:0/248877690 shutdown_connections 2024-04-12T21:40:35.439 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:35.437+0000 7fac7f000700 1 -- 172.21.15.18:0/248877690 wait complete. 2024-04-12T21:40:35.441 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:35.438+0000 7fac7f000700 1 Processor -- start 2024-04-12T21:40:35.441 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:35.439+0000 7fac7f000700 1 -- start start 2024-04-12T21:40:35.441 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:35.439+0000 7fac7f000700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fac78109430 0x7fac78119f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:40:35.441 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:35.439+0000 7fac7f000700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fac78109dd0 0x7fac7811a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:40:35.441 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:35.439+0000 7fac7f000700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac7811aad0 con 0x7fac78109430 2024-04-12T21:40:35.441 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:35.439+0000 7fac7f000700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac78114100 con 0x7fac78109dd0 2024-04-12T21:40:35.441 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:35.439+0000 7fac77e00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fac78109430 0x7fac78119f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:40:35.442 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:35.439+0000 7fac77e00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fac78109430 0x7fac78119f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:56450/0 (socket says 172.21.15.18:56450) 2024-04-12T21:40:35.442 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:35.439+0000 7fac77e00700 1 -- 172.21.15.18:0/3448802193 learned_addr learned my addr 172.21.15.18:0/3448802193 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:40:35.442 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:35.440+0000 7fac77400700 1 --2- 172.21.15.18:0/3448802193 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fac78109dd0 0x7fac7811a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:40:35.442 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:35.440+0000 7fac77e00700 1 -- 172.21.15.18:0/3448802193 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fac78109dd0 msgr2=0x7fac7811a480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:40:35.442 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:35.440+0000 7fac77e00700 1 --2- 172.21.15.18:0/3448802193 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fac78109dd0 0x7fac7811a480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:40:35.442 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:35.440+0000 7fac77e00700 1 -- 172.21.15.18:0/3448802193 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac68005040 con 0x7fac78109430 2024-04-12T21:40:35.442 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:35.440+0000 7fac77400700 1 --2- 172.21.15.18:0/3448802193 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fac78109dd0 0x7fac7811a480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-12T21:40:35.442 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:35.440+0000 7fac77e00700 1 --2- 172.21.15.18:0/3448802193 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fac78109430 0x7fac78119f40 secure :-1 s=READY pgs=1031 cs=0 l=1 rev1=1 crypto rx=0x7fac68009150 tx=0x7fac68009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:40:35.443 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:35.441+0000 7fac6fe00700 1 -- 172.21.15.18:0/3448802193 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fac6800e040 con 0x7fac78109430 2024-04-12T21:40:35.443 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:35.441+0000 7fac6fe00700 1 -- 172.21.15.18:0/3448802193 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fac68013420 con 0x7fac78109430 2024-04-12T21:40:35.443 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:35.441+0000 7fac6fe00700 1 -- 172.21.15.18:0/3448802193 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fac6801c630 con 0x7fac78109430 2024-04-12T21:40:35.443 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:35.441+0000 7fac7f000700 1 -- 172.21.15.18:0/3448802193 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fac78114380 con 0x7fac78109430 2024-04-12T21:40:35.443 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:35.441+0000 7fac7f000700 1 -- 172.21.15.18:0/3448802193 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fac78114940 con 0x7fac78109430 2024-04-12T21:40:35.443 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:35.442+0000 7fac7f000700 1 -- 172.21.15.18:0/3448802193 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fac78104b80 con 0x7fac78109430 2024-04-12T21:40:35.444 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:35.443+0000 7fac6fe00700 1 -- 172.21.15.18:0/3448802193 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7fac68013590 con 0x7fac78109430 2024-04-12T21:40:35.445 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:35.445+0000 7fac6fe00700 1 --2- 172.21.15.18:0/3448802193 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fac5c06ec00 0x7fac5c0710b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:40:35.445 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:35.445+0000 7fac6fe00700 1 -- 172.21.15.18:0/3448802193 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 6757+0+0 (secure 0 0 0) 0x7fac6808fb40 con 0x7fac78109430 2024-04-12T21:40:35.446 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:35.445+0000 7fac77400700 1 --2- 172.21.15.18:0/3448802193 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fac5c06ec00 0x7fac5c0710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:40:35.446 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:35.446+0000 7fac77400700 1 --2- 172.21.15.18:0/3448802193 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fac5c06ec00 0x7fac5c0710b0 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7fac64007fd0 tx=0x7fac64009040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:40:35.450 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:35.450+0000 7fac6fe00700 1 -- 172.21.15.18:0/3448802193 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7fac6805c430 con 0x7fac78109430 2024-04-12T21:40:35.703 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:35.702+0000 7fac7f000700 1 -- 172.21.15.18:0/3448802193 --> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fac78115590 con 0x7fac5c06ec00 2024-04-12T21:40:35.705 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:35.704+0000 7fac6fe00700 1 -- 172.21.15.18:0/3448802193 <== mgr.34102 v2:172.21.15.18:6800/3812891919 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7fac78115590 con 0x7fac5c06ec00 2024-04-12T21:40:35.709 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:35.707+0000 7fac7f000700 1 -- 172.21.15.18:0/3448802193 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fac5c06ec00 msgr2=0x7fac5c0710b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:40:35.709 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:35.707+0000 7fac7f000700 1 --2- 172.21.15.18:0/3448802193 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fac5c06ec00 0x7fac5c0710b0 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7fac64007fd0 tx=0x7fac64009040 comp rx=0 tx=0).stop 2024-04-12T21:40:35.709 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:35.707+0000 7fac7f000700 1 -- 172.21.15.18:0/3448802193 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fac78109430 msgr2=0x7fac78119f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:40:35.709 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:35.707+0000 7fac7f000700 1 --2- 172.21.15.18:0/3448802193 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fac78109430 0x7fac78119f40 secure :-1 s=READY pgs=1031 cs=0 l=1 rev1=1 crypto rx=0x7fac68009150 tx=0x7fac68009e30 comp rx=0 tx=0).stop 2024-04-12T21:40:35.709 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:35.707+0000 7fac7f000700 1 -- 172.21.15.18:0/3448802193 shutdown_connections 2024-04-12T21:40:35.709 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:35.707+0000 7fac7f000700 1 --2- 172.21.15.18:0/3448802193 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fac5c06ec00 0x7fac5c0710b0 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:40:35.709 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:35.707+0000 7fac7f000700 1 --2- 172.21.15.18:0/3448802193 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fac78109430 0x7fac78119f40 unknown :-1 s=CLOSED pgs=1031 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:40:35.709 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:35.707+0000 7fac7f000700 1 --2- 172.21.15.18:0/3448802193 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fac78109dd0 0x7fac7811a480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:40:35.709 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:35.707+0000 7fac7f000700 1 -- 172.21.15.18:0/3448802193 >> 172.21.15.18:0/3448802193 conn(0x7fac78072c00 msgr2=0x7fac78074c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:40:35.709 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:35.708+0000 7fac7f000700 1 -- 172.21.15.18:0/3448802193 shutdown_connections 2024-04-12T21:40:35.709 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:35.708+0000 7fac7f000700 1 -- 172.21.15.18:0/3448802193 wait complete. 2024-04-12T21:40:35.725 INFO:teuthology.orchestra.run.smithi018.stdout:true 2024-04-12T21:40:35.875 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:35.873+0000 7ff2f7200700 1 -- 172.21.15.18:0/2488462156 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff2f0070c70 msgr2=0x7ff2f0108fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:40:35.875 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:35.873+0000 7ff2f7200700 1 --2- 172.21.15.18:0/2488462156 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff2f0070c70 0x7ff2f0108fc0 secure :-1 s=READY pgs=1032 cs=0 l=1 rev1=1 crypto rx=0x7ff2dc004970 tx=0x7ff2dc009e30 comp rx=0 tx=0).stop 2024-04-12T21:40:35.875 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:35.873+0000 7ff2f7200700 1 -- 172.21.15.18:0/2488462156 shutdown_connections 2024-04-12T21:40:35.875 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:35.873+0000 7ff2f7200700 1 --2- 172.21.15.18:0/2488462156 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff2f0070c70 0x7ff2f0108fc0 unknown :-1 s=CLOSED pgs=1032 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:40:35.875 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:35.873+0000 7ff2f7200700 1 --2- 172.21.15.18:0/2488462156 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff2f0070360 0x7ff2f0070730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:40:35.875 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:35.873+0000 7ff2f7200700 1 -- 172.21.15.18:0/2488462156 >> 172.21.15.18:0/2488462156 conn(0x7ff2f00f87e0 msgr2=0x7ff2f00fabf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:40:35.875 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:35.874+0000 7ff2f7200700 1 -- 172.21.15.18:0/2488462156 shutdown_connections 2024-04-12T21:40:35.875 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:35.874+0000 7ff2f7200700 1 -- 172.21.15.18:0/2488462156 wait complete. 2024-04-12T21:40:35.877 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:35.875+0000 7ff2f7200700 1 Processor -- start 2024-04-12T21:40:35.877 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:35.875+0000 7ff2f7200700 1 -- start start 2024-04-12T21:40:35.877 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:35.875+0000 7ff2f7200700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff2f0070360 0x7ff2f0115bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:40:35.877 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:35.875+0000 7ff2f7200700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff2f0070c70 0x7ff2f01160f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:40:35.877 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:35.875+0000 7ff2f7200700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2f0116740 con 0x7ff2f0070c70 2024-04-12T21:40:35.877 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:35.875+0000 7ff2f7200700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2f010fb30 con 0x7ff2f0070360 2024-04-12T21:40:35.877 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:35.876+0000 7ff2f5400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff2f0070c70 0x7ff2f01160f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:40:35.877 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:35.876+0000 7ff2f5400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff2f0070c70 0x7ff2f01160f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:56466/0 (socket says 172.21.15.18:56466) 2024-04-12T21:40:35.877 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:35.876+0000 7ff2f5400700 1 -- 172.21.15.18:0/2717301087 learned_addr learned my addr 172.21.15.18:0/2717301087 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:40:35.878 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:35.876+0000 7ff2f5e00700 1 --2- 172.21.15.18:0/2717301087 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff2f0070360 0x7ff2f0115bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:40:35.878 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:35.876+0000 7ff2f5400700 1 -- 172.21.15.18:0/2717301087 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff2f0070360 msgr2=0x7ff2f0115bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:40:35.878 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:35.876+0000 7ff2f5400700 1 --2- 172.21.15.18:0/2717301087 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff2f0070360 0x7ff2f0115bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:40:35.878 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:35.876+0000 7ff2f5400700 1 -- 172.21.15.18:0/2717301087 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2dc005040 con 0x7ff2f0070c70 2024-04-12T21:40:35.879 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:35.876+0000 7ff2f5400700 1 --2- 172.21.15.18:0/2717301087 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff2f0070c70 0x7ff2f01160f0 secure :-1 s=READY pgs=1033 cs=0 l=1 rev1=1 crypto rx=0x7ff2dc004f50 tx=0x7ff2dc009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:40:35.879 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:35.877+0000 7ff2ea000700 1 -- 172.21.15.18:0/2717301087 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff2dc00e040 con 0x7ff2f0070c70 2024-04-12T21:40:35.879 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:35.877+0000 7ff2ea000700 1 -- 172.21.15.18:0/2717301087 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff2dc00bc10 con 0x7ff2f0070c70 2024-04-12T21:40:35.879 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:35.877+0000 7ff2ea000700 1 -- 172.21.15.18:0/2717301087 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff2dc01c630 con 0x7ff2f0070c70 2024-04-12T21:40:35.879 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:35.877+0000 7ff2f7200700 1 -- 172.21.15.18:0/2717301087 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff2f010fdb0 con 0x7ff2f0070c70 2024-04-12T21:40:35.879 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:35.877+0000 7ff2f7200700 1 -- 172.21.15.18:0/2717301087 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff2f01102f0 con 0x7ff2f0070c70 2024-04-12T21:40:35.879 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:35.878+0000 7ff2f7200700 1 -- 172.21.15.18:0/2717301087 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff2f0071aa0 con 0x7ff2f0070c70 2024-04-12T21:40:35.880 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:35.879+0000 7ff2ea000700 1 -- 172.21.15.18:0/2717301087 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7ff2dc013420 con 0x7ff2f0070c70 2024-04-12T21:40:35.881 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:35.880+0000 7ff2ea000700 1 --2- 172.21.15.18:0/2717301087 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7ff2d806ec00 0x7ff2d80710b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:40:35.881 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:35.880+0000 7ff2ea000700 1 -- 172.21.15.18:0/2717301087 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 6757+0+0 (secure 0 0 0) 0x7ff2dc08fc70 con 0x7ff2f0070c70 2024-04-12T21:40:35.881 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:35.881+0000 7ff2f5e00700 1 --2- 172.21.15.18:0/2717301087 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7ff2d806ec00 0x7ff2d80710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:40:35.882 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:35.881+0000 7ff2f5e00700 1 --2- 172.21.15.18:0/2717301087 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7ff2d806ec00 0x7ff2d80710b0 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7ff2e4004990 tx=0x7ff2e4009ee0 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:40:35.886 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:35.886+0000 7ff2ea000700 1 -- 172.21.15.18:0/2717301087 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7ff2dc096080 con 0x7ff2f0070c70 2024-04-12T21:40:36.033 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:40:35 smithi186 ceph-mon[86379]: pgmap v1227: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:40:36.104 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:40:35 smithi018 ceph-mon[112590]: pgmap v1227: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:40:36.137 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:36.136+0000 7ff2f7200700 1 -- 172.21.15.18:0/2717301087 --> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7ff2f010bff0 con 0x7ff2d806ec00 2024-04-12T21:40:36.139 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:36.138+0000 7ff2ea000700 1 -- 172.21.15.18:0/2717301087 <== mgr.34102 v2:172.21.15.18:6800/3812891919 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7ff2f010bff0 con 0x7ff2d806ec00 2024-04-12T21:40:36.141 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:36.140+0000 7ff2f7200700 1 -- 172.21.15.18:0/2717301087 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7ff2d806ec00 msgr2=0x7ff2d80710b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:40:36.141 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:36.140+0000 7ff2f7200700 1 --2- 172.21.15.18:0/2717301087 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7ff2d806ec00 0x7ff2d80710b0 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7ff2e4004990 tx=0x7ff2e4009ee0 comp rx=0 tx=0).stop 2024-04-12T21:40:36.142 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:36.140+0000 7ff2f7200700 1 -- 172.21.15.18:0/2717301087 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff2f0070c70 msgr2=0x7ff2f01160f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:40:36.142 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:36.140+0000 7ff2f7200700 1 --2- 172.21.15.18:0/2717301087 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff2f0070c70 0x7ff2f01160f0 secure :-1 s=READY pgs=1033 cs=0 l=1 rev1=1 crypto rx=0x7ff2dc004f50 tx=0x7ff2dc009e30 comp rx=0 tx=0).stop 2024-04-12T21:40:36.142 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:36.141+0000 7ff2f7200700 1 -- 172.21.15.18:0/2717301087 shutdown_connections 2024-04-12T21:40:36.142 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:36.141+0000 7ff2f7200700 1 --2- 172.21.15.18:0/2717301087 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7ff2d806ec00 0x7ff2d80710b0 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:40:36.142 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:36.141+0000 7ff2f7200700 1 --2- 172.21.15.18:0/2717301087 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff2f0070360 0x7ff2f0115bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:40:36.142 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:36.141+0000 7ff2f7200700 1 --2- 172.21.15.18:0/2717301087 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff2f0070c70 0x7ff2f01160f0 unknown :-1 s=CLOSED pgs=1033 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:40:36.142 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:36.141+0000 7ff2f7200700 1 -- 172.21.15.18:0/2717301087 >> 172.21.15.18:0/2717301087 conn(0x7ff2f00f87e0 msgr2=0x7ff2f0101500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:40:36.143 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:36.141+0000 7ff2f7200700 1 -- 172.21.15.18:0/2717301087 shutdown_connections 2024-04-12T21:40:36.143 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:36.141+0000 7ff2f7200700 1 -- 172.21.15.18:0/2717301087 wait complete. 2024-04-12T21:40:36.287 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:36.286+0000 7fc187200700 1 -- 172.21.15.18:0/759492928 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc188072e00 msgr2=0x7fc18810d630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:40:36.287 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:36.286+0000 7fc187200700 1 --2- 172.21.15.18:0/759492928 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc188072e00 0x7fc18810d630 secure :-1 s=READY pgs=1034 cs=0 l=1 rev1=1 crypto rx=0x7fc170004940 tx=0x7fc170009e30 comp rx=0 tx=0).stop 2024-04-12T21:40:36.288 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:36.286+0000 7fc187200700 1 -- 172.21.15.18:0/759492928 shutdown_connections 2024-04-12T21:40:36.288 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:36.286+0000 7fc187200700 1 --2- 172.21.15.18:0/759492928 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc188072e00 0x7fc18810d630 unknown :-1 s=CLOSED pgs=1034 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:40:36.288 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:36.286+0000 7fc187200700 1 --2- 172.21.15.18:0/759492928 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc1880724f0 0x7fc1880728c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:40:36.288 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:36.286+0000 7fc187200700 1 -- 172.21.15.18:0/759492928 >> 172.21.15.18:0/759492928 conn(0x7fc1880fab10 msgr2=0x7fc1880fcef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:40:36.288 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:36.286+0000 7fc187200700 1 -- 172.21.15.18:0/759492928 shutdown_connections 2024-04-12T21:40:36.288 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:36.286+0000 7fc187200700 1 -- 172.21.15.18:0/759492928 wait complete. 2024-04-12T21:40:36.290 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:36.287+0000 7fc187200700 1 Processor -- start 2024-04-12T21:40:36.290 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:36.287+0000 7fc187200700 1 -- start start 2024-04-12T21:40:36.290 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:36.288+0000 7fc187200700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc1880724f0 0x7fc188119f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:40:36.290 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:36.288+0000 7fc187200700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc188072e00 0x7fc18811a4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:40:36.290 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:36.288+0000 7fc187200700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc18811ab10 con 0x7fc188072e00 2024-04-12T21:40:36.290 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:36.288+0000 7fc187200700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc188113ee0 con 0x7fc1880724f0 2024-04-12T21:40:36.290 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:36.288+0000 7fc17fe00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc188072e00 0x7fc18811a4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:40:36.290 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:36.288+0000 7fc17fe00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc188072e00 0x7fc18811a4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:56484/0 (socket says 172.21.15.18:56484) 2024-04-12T21:40:36.290 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:36.288+0000 7fc17fe00700 1 -- 172.21.15.18:0/3840264616 learned_addr learned my addr 172.21.15.18:0/3840264616 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:40:36.290 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:36.289+0000 7fc185e00700 1 --2- 172.21.15.18:0/3840264616 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc1880724f0 0x7fc188119f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:40:36.290 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:36.289+0000 7fc17fe00700 1 -- 172.21.15.18:0/3840264616 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc1880724f0 msgr2=0x7fc188119f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:40:36.291 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:36.289+0000 7fc17fe00700 1 --2- 172.21.15.18:0/3840264616 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc1880724f0 0x7fc188119f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:40:36.291 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:36.289+0000 7fc17fe00700 1 -- 172.21.15.18:0/3840264616 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc170005040 con 0x7fc188072e00 2024-04-12T21:40:36.291 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:36.289+0000 7fc17fe00700 1 --2- 172.21.15.18:0/3840264616 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc188072e00 0x7fc18811a4c0 secure :-1 s=READY pgs=1035 cs=0 l=1 rev1=1 crypto rx=0x7fc170009180 tx=0x7fc170009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:40:36.291 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:36.289+0000 7fc17e000700 1 -- 172.21.15.18:0/3840264616 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc17000e040 con 0x7fc188072e00 2024-04-12T21:40:36.291 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:36.289+0000 7fc17e000700 1 -- 172.21.15.18:0/3840264616 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc17000bbc0 con 0x7fc188072e00 2024-04-12T21:40:36.292 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:36.289+0000 7fc187200700 1 -- 172.21.15.18:0/3840264616 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc188114160 con 0x7fc188072e00 2024-04-12T21:40:36.292 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:36.289+0000 7fc17e000700 1 -- 172.21.15.18:0/3840264616 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc17001c630 con 0x7fc188072e00 2024-04-12T21:40:36.292 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:36.289+0000 7fc187200700 1 -- 172.21.15.18:0/3840264616 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc188114720 con 0x7fc188072e00 2024-04-12T21:40:36.292 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:36.290+0000 7fc187200700 1 -- 172.21.15.18:0/3840264616 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc188073be0 con 0x7fc188072e00 2024-04-12T21:40:36.292 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:36.291+0000 7fc17e000700 1 -- 172.21.15.18:0/3840264616 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7fc17000bd30 con 0x7fc188072e00 2024-04-12T21:40:36.293 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:36.292+0000 7fc17e000700 1 --2- 172.21.15.18:0/3840264616 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fc16c06ec00 0x7fc16c0710b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:40:36.293 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:36.292+0000 7fc17e000700 1 -- 172.21.15.18:0/3840264616 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 6757+0+0 (secure 0 0 0) 0x7fc17008fd70 con 0x7fc188072e00 2024-04-12T21:40:36.293 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:36.293+0000 7fc185e00700 1 --2- 172.21.15.18:0/3840264616 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fc16c06ec00 0x7fc16c0710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:40:36.294 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:36.293+0000 7fc185e00700 1 --2- 172.21.15.18:0/3840264616 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fc16c06ec00 0x7fc16c0710b0 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7fc178007fd0 tx=0x7fc178009040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:40:36.298 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:36.298+0000 7fc17e000700 1 -- 172.21.15.18:0/3840264616 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7fc17005c5a0 con 0x7fc188072e00 2024-04-12T21:40:36.529 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:36.529+0000 7fc187200700 1 -- 172.21.15.18:0/3840264616 --> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fc188110350 con 0x7fc16c06ec00 2024-04-12T21:40:36.563 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:36.563+0000 7fc17e000700 1 -- 172.21.15.18:0/3840264616 <== mgr.34102 v2:172.21.15.18:6800/3812891919 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7fc188110350 con 0x7fc16c06ec00 2024-04-12T21:40:36.564 INFO:teuthology.orchestra.run.smithi018.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-12T21:40:36.564 INFO:teuthology.orchestra.run.smithi018.stdout:alertmanager.smithi018 smithi018 *:9093,9094 running (41m) 2m ago 55m 14.8M - 0.25.0 c8568f914cd2 c332090068e8 2024-04-12T21:40:36.564 INFO:teuthology.orchestra.run.smithi018.stdout:crash.smithi018 smithi018 running (37m) 2m ago 55m 6665k - 19.0.0-2879-g324baf8d 8717720dc5e5 b7f80d1f7982 2024-04-12T21:40:36.564 INFO:teuthology.orchestra.run.smithi018.stdout:crash.smithi186 smithi186 running (37m) 2m ago 54m 6660k - 19.0.0-2879-g324baf8d 8717720dc5e5 a4f55c81b623 2024-04-12T21:40:36.564 INFO:teuthology.orchestra.run.smithi018.stdout:grafana.smithi018 smithi018 *:3000 running (40m) 2m ago 55m 73.8M - 9.4.12 2bacad6d85d8 b08e3bf176dd 2024-04-12T21:40:36.564 INFO:teuthology.orchestra.run.smithi018.stdout:mds.cephfs.smithi018.acqbsc smithi018 running (33m) 2m ago 48m 21.9M - 19.0.0-2879-g324baf8d 8717720dc5e5 021f99ab24ae 2024-04-12T21:40:36.564 INFO:teuthology.orchestra.run.smithi018.stdout:mds.cephfs.smithi018.szjluo smithi018 running (32m) 2m ago 48m 21.7M - 19.0.0-2879-g324baf8d 8717720dc5e5 0003117479d4 2024-04-12T21:40:36.564 INFO:teuthology.orchestra.run.smithi018.stdout:mds.cephfs.smithi186.jdllyn smithi186 running (17m) 2m ago 48m 16.5M - 19.0.0-2879-g324baf8d 8717720dc5e5 21b1f46eef23 2024-04-12T21:40:36.564 INFO:teuthology.orchestra.run.smithi018.stdout:mds.cephfs.smithi186.wjnafs smithi186 running (2m) 2m ago 48m 11.0M - 19.0.0-2879-g324baf8d 8717720dc5e5 3cf1ef4148e2 2024-04-12T21:40:36.565 INFO:teuthology.orchestra.run.smithi018.stdout:mgr.smithi018.xjzwga smithi018 *:8443,9283,8765 running (43m) 2m ago 57m 517M - 19.0.0-2879-g324baf8d 8717720dc5e5 56088305eab7 2024-04-12T21:40:36.565 INFO:teuthology.orchestra.run.smithi018.stdout:mgr.smithi186.kqcfxk smithi186 *:8443,9283,8765 running (42m) 2m ago 54m 460M - 19.0.0-2879-g324baf8d 8717720dc5e5 09cd766e5e0d 2024-04-12T21:40:36.565 INFO:teuthology.orchestra.run.smithi018.stdout:mon.smithi018 smithi018 running (39m) 2m ago 57m 88.2M 2048M 19.0.0-2879-g324baf8d 8717720dc5e5 df3e201dfed6 2024-04-12T21:40:36.565 INFO:teuthology.orchestra.run.smithi018.stdout:mon.smithi186 smithi186 running (38m) 2m ago 54m 53.6M 2048M 19.0.0-2879-g324baf8d 8717720dc5e5 c77498da9b2b 2024-04-12T21:40:36.565 INFO:teuthology.orchestra.run.smithi018.stdout:node-exporter.smithi018 smithi018 *:9100 running (42m) 2m ago 55m 8468k - 1.5.0 0da6a335fe13 326c3d9c6847 2024-04-12T21:40:36.565 INFO:teuthology.orchestra.run.smithi018.stdout:node-exporter.smithi186 smithi186 *:9100 running (42m) 2m ago 54m 8166k - 1.5.0 0da6a335fe13 ec2112baeecf 2024-04-12T21:40:36.565 INFO:teuthology.orchestra.run.smithi018.stdout:osd.0 smithi018 running (36m) 2m ago 53m 123M 1559M 19.0.0-2879-g324baf8d 8717720dc5e5 60b1273ca047 2024-04-12T21:40:36.565 INFO:teuthology.orchestra.run.smithi018.stdout:osd.1 smithi018 running (36m) 2m ago 52m 95.4M 1559M 19.0.0-2879-g324baf8d 8717720dc5e5 170ce1e5fb8d 2024-04-12T21:40:36.565 INFO:teuthology.orchestra.run.smithi018.stdout:osd.2 smithi018 running (35m) 2m ago 52m 94.6M 1559M 19.0.0-2879-g324baf8d 8717720dc5e5 c2cbc6aa2839 2024-04-12T21:40:36.565 INFO:teuthology.orchestra.run.smithi018.stdout:osd.3 smithi186 running (35m) 2m ago 51m 101M 2581M 19.0.0-2879-g324baf8d 8717720dc5e5 0138e8262e4b 2024-04-12T21:40:36.565 INFO:teuthology.orchestra.run.smithi018.stdout:osd.4 smithi186 running (34m) 2m ago 51m 91.5M 2581M 19.0.0-2879-g324baf8d 8717720dc5e5 b20cb63b25c9 2024-04-12T21:40:36.565 INFO:teuthology.orchestra.run.smithi018.stdout:osd.5 smithi186 running (33m) 2m ago 50m 92.2M 2581M 19.0.0-2879-g324baf8d 8717720dc5e5 2513bea2610e 2024-04-12T21:40:36.565 INFO:teuthology.orchestra.run.smithi018.stdout:prometheus.smithi018 smithi018 *:9095 running (41m) 2m ago 55m 69.6M - 2.43.0 a07b618ecd1d f58841127e21 2024-04-12T21:40:36.567 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:36.566+0000 7fc187200700 1 -- 172.21.15.18:0/3840264616 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fc16c06ec00 msgr2=0x7fc16c0710b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:40:36.567 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:36.566+0000 7fc187200700 1 --2- 172.21.15.18:0/3840264616 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fc16c06ec00 0x7fc16c0710b0 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7fc178007fd0 tx=0x7fc178009040 comp rx=0 tx=0).stop 2024-04-12T21:40:36.567 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:36.566+0000 7fc187200700 1 -- 172.21.15.18:0/3840264616 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc188072e00 msgr2=0x7fc18811a4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:40:36.567 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:36.566+0000 7fc187200700 1 --2- 172.21.15.18:0/3840264616 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc188072e00 0x7fc18811a4c0 secure :-1 s=READY pgs=1035 cs=0 l=1 rev1=1 crypto rx=0x7fc170009180 tx=0x7fc170009e30 comp rx=0 tx=0).stop 2024-04-12T21:40:36.567 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:36.567+0000 7fc187200700 1 -- 172.21.15.18:0/3840264616 shutdown_connections 2024-04-12T21:40:36.567 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:36.567+0000 7fc187200700 1 --2- 172.21.15.18:0/3840264616 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fc16c06ec00 0x7fc16c0710b0 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:40:36.567 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:36.567+0000 7fc187200700 1 --2- 172.21.15.18:0/3840264616 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc1880724f0 0x7fc188119f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:40:36.567 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:36.567+0000 7fc187200700 1 --2- 172.21.15.18:0/3840264616 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc188072e00 0x7fc18811a4c0 unknown :-1 s=CLOSED pgs=1035 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:40:36.567 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:36.567+0000 7fc187200700 1 -- 172.21.15.18:0/3840264616 >> 172.21.15.18:0/3840264616 conn(0x7fc1880fab10 msgr2=0x7fc1880faee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:40:36.567 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:36.567+0000 7fc187200700 1 -- 172.21.15.18:0/3840264616 shutdown_connections 2024-04-12T21:40:36.568 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:36.567+0000 7fc187200700 1 -- 172.21.15.18:0/3840264616 wait complete. 2024-04-12T21:40:36.719 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:36.719+0000 7f1324e00700 1 -- 172.21.15.18:0/2247298843 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f13200708a0 msgr2=0x7f1320070d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:40:36.720 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:36.719+0000 7f1324e00700 1 --2- 172.21.15.18:0/2247298843 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f13200708a0 0x7f1320070d10 secure :-1 s=READY pgs=1036 cs=0 l=1 rev1=1 crypto rx=0x7f130c004970 tx=0x7f130c009e30 comp rx=0 tx=0).stop 2024-04-12T21:40:36.720 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:36.719+0000 7f1324e00700 1 -- 172.21.15.18:0/2247298843 shutdown_connections 2024-04-12T21:40:36.720 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:36.719+0000 7f1324e00700 1 --2- 172.21.15.18:0/2247298843 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f13200708a0 0x7f1320070d10 unknown :-1 s=CLOSED pgs=1036 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:40:36.720 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:36.719+0000 7f1324e00700 1 --2- 172.21.15.18:0/2247298843 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1320071f90 0x7f1320070360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:40:36.720 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:36.719+0000 7f1324e00700 1 -- 172.21.15.18:0/2247298843 >> 172.21.15.18:0/2247298843 conn(0x7f13200f8760 msgr2=0x7f13200fab70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:40:36.720 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:36.719+0000 7f1324e00700 1 -- 172.21.15.18:0/2247298843 shutdown_connections 2024-04-12T21:40:36.720 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:36.719+0000 7f1324e00700 1 -- 172.21.15.18:0/2247298843 wait complete. 2024-04-12T21:40:36.721 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:36.720+0000 7f1324e00700 1 Processor -- start 2024-04-12T21:40:36.721 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:36.720+0000 7f1324e00700 1 -- start start 2024-04-12T21:40:36.721 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:36.721+0000 7f1324e00700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f13200708a0 0x7f1320119fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:40:36.721 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:36.721+0000 7f1324e00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1320071f90 0x7f132011a4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:40:36.721 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:36.721+0000 7f1324e00700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f132011ab30 con 0x7f1320071f90 2024-04-12T21:40:36.722 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:36.721+0000 7f1324e00700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1320113f00 con 0x7f13200708a0 2024-04-12T21:40:36.722 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:36.721+0000 7f131ea00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1320071f90 0x7f132011a4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:40:36.722 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:36.721+0000 7f131ea00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1320071f90 0x7f132011a4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:56506/0 (socket says 172.21.15.18:56506) 2024-04-12T21:40:36.722 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:36.721+0000 7f131ea00700 1 -- 172.21.15.18:0/3865273997 learned_addr learned my addr 172.21.15.18:0/3865273997 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:40:36.722 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:36.721+0000 7f131f400700 1 --2- 172.21.15.18:0/3865273997 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f13200708a0 0x7f1320119fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:40:36.722 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:36.721+0000 7f131ea00700 1 -- 172.21.15.18:0/3865273997 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f13200708a0 msgr2=0x7f1320119fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:40:36.722 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:36.721+0000 7f131ea00700 1 --2- 172.21.15.18:0/3865273997 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f13200708a0 0x7f1320119fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:40:36.722 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:36.721+0000 7f131ea00700 1 -- 172.21.15.18:0/3865273997 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f130c005040 con 0x7f1320071f90 2024-04-12T21:40:36.723 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:36.722+0000 7f131ea00700 1 --2- 172.21.15.18:0/3865273997 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1320071f90 0x7f132011a4e0 secure :-1 s=READY pgs=1037 cs=0 l=1 rev1=1 crypto rx=0x7f130c004f50 tx=0x7f130c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:40:36.723 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:36.722+0000 7f1317400700 1 -- 172.21.15.18:0/3865273997 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f130c00e040 con 0x7f1320071f90 2024-04-12T21:40:36.723 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:36.722+0000 7f1317400700 1 -- 172.21.15.18:0/3865273997 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f130c00bc10 con 0x7f1320071f90 2024-04-12T21:40:36.723 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:36.722+0000 7f1317400700 1 -- 172.21.15.18:0/3865273997 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f130c01c630 con 0x7f1320071f90 2024-04-12T21:40:36.723 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:36.722+0000 7f1324e00700 1 -- 172.21.15.18:0/3865273997 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1320114180 con 0x7f1320071f90 2024-04-12T21:40:36.723 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:36.722+0000 7f1324e00700 1 -- 172.21.15.18:0/3865273997 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f13201146c0 con 0x7f1320071f90 2024-04-12T21:40:36.723 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:36.723+0000 7f1324e00700 1 -- 172.21.15.18:0/3865273997 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1320071aa0 con 0x7f1320071f90 2024-04-12T21:40:36.724 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:36.724+0000 7f1317400700 1 -- 172.21.15.18:0/3865273997 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7f130c013420 con 0x7f1320071f90 2024-04-12T21:40:36.725 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:36.725+0000 7f1317400700 1 --2- 172.21.15.18:0/3865273997 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f130406ec00 0x7f13040710b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:40:36.725 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:36.725+0000 7f1317400700 1 -- 172.21.15.18:0/3865273997 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 6757+0+0 (secure 0 0 0) 0x7f130c08fc40 con 0x7f1320071f90 2024-04-12T21:40:36.725 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:36.726+0000 7f131f400700 1 --2- 172.21.15.18:0/3865273997 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f130406ec00 0x7f13040710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:40:36.726 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:36.726+0000 7f131f400700 1 --2- 172.21.15.18:0/3865273997 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f130406ec00 0x7f13040710b0 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f1310007fd0 tx=0x7f1310009040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:40:36.730 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:36.730+0000 7f1317400700 1 -- 172.21.15.18:0/3865273997 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f130c05c530 con 0x7f1320071f90 2024-04-12T21:40:36.854 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:40:36 smithi018 ceph-mon[112590]: from='client.36132 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:40:37.033 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:40:36 smithi186 ceph-mon[86379]: from='client.36132 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:40:37.052 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:37.052+0000 7f1324e00700 1 -- 172.21.15.18:0/3865273997 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f132004bed0 con 0x7f1320071f90 2024-04-12T21:40:37.053 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:37.053+0000 7f1317400700 1 -- 172.21.15.18:0/3865273997 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+623 (secure 0 0 0) 0x7f130c05c0c0 con 0x7f1320071f90 2024-04-12T21:40:37.053 INFO:teuthology.orchestra.run.smithi018.stdout:{ 2024-04-12T21:40:37.053 INFO:teuthology.orchestra.run.smithi018.stdout: "mon": { 2024-04-12T21:40:37.053 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 19.0.0-2879-g324baf8d (324baf8ddb91dea46cac50cb535c306115791355) squid (dev)": 2 2024-04-12T21:40:37.053 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-12T21:40:37.053 INFO:teuthology.orchestra.run.smithi018.stdout: "mgr": { 2024-04-12T21:40:37.054 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 19.0.0-2879-g324baf8d (324baf8ddb91dea46cac50cb535c306115791355) squid (dev)": 2 2024-04-12T21:40:37.054 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-12T21:40:37.054 INFO:teuthology.orchestra.run.smithi018.stdout: "osd": { 2024-04-12T21:40:37.054 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 19.0.0-2879-g324baf8d (324baf8ddb91dea46cac50cb535c306115791355) squid (dev)": 6 2024-04-12T21:40:37.054 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-12T21:40:37.054 INFO:teuthology.orchestra.run.smithi018.stdout: "mds": { 2024-04-12T21:40:37.054 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 19.0.0-2879-g324baf8d (324baf8ddb91dea46cac50cb535c306115791355) squid (dev)": 4 2024-04-12T21:40:37.054 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-12T21:40:37.054 INFO:teuthology.orchestra.run.smithi018.stdout: "overall": { 2024-04-12T21:40:37.054 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 19.0.0-2879-g324baf8d (324baf8ddb91dea46cac50cb535c306115791355) squid (dev)": 14 2024-04-12T21:40:37.054 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-04-12T21:40:37.054 INFO:teuthology.orchestra.run.smithi018.stdout:} 2024-04-12T21:40:37.056 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:37.056+0000 7f1324e00700 1 -- 172.21.15.18:0/3865273997 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f130406ec00 msgr2=0x7f13040710b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:40:37.056 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:37.056+0000 7f1324e00700 1 --2- 172.21.15.18:0/3865273997 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f130406ec00 0x7f13040710b0 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f1310007fd0 tx=0x7f1310009040 comp rx=0 tx=0).stop 2024-04-12T21:40:37.056 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:37.056+0000 7f1324e00700 1 -- 172.21.15.18:0/3865273997 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1320071f90 msgr2=0x7f132011a4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:40:37.056 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:37.056+0000 7f1324e00700 1 --2- 172.21.15.18:0/3865273997 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1320071f90 0x7f132011a4e0 secure :-1 s=READY pgs=1037 cs=0 l=1 rev1=1 crypto rx=0x7f130c004f50 tx=0x7f130c009e30 comp rx=0 tx=0).stop 2024-04-12T21:40:37.056 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:37.056+0000 7f1324e00700 1 -- 172.21.15.18:0/3865273997 shutdown_connections 2024-04-12T21:40:37.056 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:37.056+0000 7f1324e00700 1 --2- 172.21.15.18:0/3865273997 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f130406ec00 0x7f13040710b0 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:40:37.056 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:37.056+0000 7f1324e00700 1 --2- 172.21.15.18:0/3865273997 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f13200708a0 0x7f1320119fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:40:37.057 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:37.056+0000 7f1324e00700 1 --2- 172.21.15.18:0/3865273997 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1320071f90 0x7f132011a4e0 unknown :-1 s=CLOSED pgs=1037 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:40:37.057 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:37.056+0000 7f1324e00700 1 -- 172.21.15.18:0/3865273997 >> 172.21.15.18:0/3865273997 conn(0x7f13200f8760 msgr2=0x7f13200f97e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:40:37.057 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:37.056+0000 7f1324e00700 1 -- 172.21.15.18:0/3865273997 shutdown_connections 2024-04-12T21:40:37.057 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:37.056+0000 7f1324e00700 1 -- 172.21.15.18:0/3865273997 wait complete. 2024-04-12T21:40:37.213 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:37.212+0000 7f7895a00700 1 -- 172.21.15.18:0/3745673951 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f78900708a0 msgr2=0x7f7890070d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:40:37.213 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:37.212+0000 7f7895a00700 1 --2- 172.21.15.18:0/3745673951 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f78900708a0 0x7f7890070d10 secure :-1 s=READY pgs=1038 cs=0 l=1 rev1=1 crypto rx=0x7f787c004970 tx=0x7f787c009e30 comp rx=0 tx=0).stop 2024-04-12T21:40:37.213 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:37.213+0000 7f7895a00700 1 -- 172.21.15.18:0/3745673951 shutdown_connections 2024-04-12T21:40:37.213 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:37.213+0000 7f7895a00700 1 --2- 172.21.15.18:0/3745673951 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f78900708a0 0x7f7890070d10 unknown :-1 s=CLOSED pgs=1038 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:40:37.213 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:37.213+0000 7f7895a00700 1 --2- 172.21.15.18:0/3745673951 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7890071f90 0x7f7890070360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:40:37.214 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:37.213+0000 7f7895a00700 1 -- 172.21.15.18:0/3745673951 >> 172.21.15.18:0/3745673951 conn(0x7f78900f89b0 msgr2=0x7f78900fadc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:40:37.214 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:37.213+0000 7f7895a00700 1 -- 172.21.15.18:0/3745673951 shutdown_connections 2024-04-12T21:40:37.214 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:37.213+0000 7f7895a00700 1 -- 172.21.15.18:0/3745673951 wait complete. 2024-04-12T21:40:37.215 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:37.214+0000 7f7895a00700 1 Processor -- start 2024-04-12T21:40:37.215 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:37.214+0000 7f7895a00700 1 -- start start 2024-04-12T21:40:37.215 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:37.215+0000 7f7895a00700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f78900708a0 0x7f7890119f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:40:37.215 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:37.215+0000 7f7895a00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7890071f90 0x7f789011a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:40:37.215 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:37.215+0000 7f7895a00700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f789011ab00 con 0x7f7890071f90 2024-04-12T21:40:37.215 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:37.215+0000 7f7895a00700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78901140e0 con 0x7f78900708a0 2024-04-12T21:40:37.216 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:37.215+0000 7f788e000700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7890071f90 0x7f789011a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:40:37.216 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:37.215+0000 7f788e000700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7890071f90 0x7f789011a4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:56526/0 (socket says 172.21.15.18:56526) 2024-04-12T21:40:37.216 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:37.215+0000 7f788e000700 1 -- 172.21.15.18:0/1059166748 learned_addr learned my addr 172.21.15.18:0/1059166748 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:40:37.216 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:37.215+0000 7f788ea00700 1 --2- 172.21.15.18:0/1059166748 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f78900708a0 0x7f7890119f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:40:37.217 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:37.216+0000 7f788e000700 1 -- 172.21.15.18:0/1059166748 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f78900708a0 msgr2=0x7f7890119f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:40:37.217 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:37.216+0000 7f788e000700 1 --2- 172.21.15.18:0/1059166748 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f78900708a0 0x7f7890119f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:40:37.217 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:37.216+0000 7f788e000700 1 -- 172.21.15.18:0/1059166748 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f787c005040 con 0x7f7890071f90 2024-04-12T21:40:37.217 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:37.216+0000 7f788ea00700 1 --2- 172.21.15.18:0/1059166748 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f78900708a0 0x7f7890119f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-12T21:40:37.217 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:37.216+0000 7f788e000700 1 --2- 172.21.15.18:0/1059166748 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7890071f90 0x7f789011a4b0 secure :-1 s=READY pgs=1039 cs=0 l=1 rev1=1 crypto rx=0x7f787c004f50 tx=0x7f787c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:40:37.217 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:37.216+0000 7f7886a00700 1 -- 172.21.15.18:0/1059166748 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f787c00e040 con 0x7f7890071f90 2024-04-12T21:40:37.217 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:37.216+0000 7f7886a00700 1 -- 172.21.15.18:0/1059166748 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f787c004290 con 0x7f7890071f90 2024-04-12T21:40:37.217 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:37.216+0000 7f7886a00700 1 -- 172.21.15.18:0/1059166748 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f787c01c630 con 0x7f7890071f90 2024-04-12T21:40:37.217 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:37.216+0000 7f7895a00700 1 -- 172.21.15.18:0/1059166748 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7890114360 con 0x7f7890071f90 2024-04-12T21:40:37.218 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:37.217+0000 7f7895a00700 1 -- 172.21.15.18:0/1059166748 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f78901148a0 con 0x7f7890071f90 2024-04-12T21:40:37.218 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:37.218+0000 7f7895a00700 1 -- 172.21.15.18:0/1059166748 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7890071aa0 con 0x7f7890071f90 2024-04-12T21:40:37.218 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:37.219+0000 7f7886a00700 1 -- 172.21.15.18:0/1059166748 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7f787c00bc10 con 0x7f7890071f90 2024-04-12T21:40:37.220 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:37.220+0000 7f7886a00700 1 --2- 172.21.15.18:0/1059166748 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f787406ec00 0x7f78740710b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:40:37.220 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:37.220+0000 7f7886a00700 1 -- 172.21.15.18:0/1059166748 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 6757+0+0 (secure 0 0 0) 0x7f787c08fca0 con 0x7f7890071f90 2024-04-12T21:40:37.220 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:37.220+0000 7f788ea00700 1 --2- 172.21.15.18:0/1059166748 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f787406ec00 0x7f78740710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:40:37.220 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:37.221+0000 7f788ea00700 1 --2- 172.21.15.18:0/1059166748 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f787406ec00 0x7f78740710b0 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f78800041d0 tx=0x7f7880009040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:40:37.224 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:37.225+0000 7f7886a00700 1 -- 172.21.15.18:0/1059166748 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f787c05c4d0 con 0x7f7890071f90 2024-04-12T21:40:37.509 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:37.508+0000 7f7895a00700 1 -- 172.21.15.18:0/1059166748 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f789004c710 con 0x7f7890071f90 2024-04-12T21:40:37.509 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:37.509+0000 7f7886a00700 1 -- 172.21.15.18:0/1059166748 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 33 v33) v1 ==== 76+0+1876 (secure 0 0 0) 0x7f787c05c060 con 0x7f7890071f90 2024-04-12T21:40:37.510 INFO:teuthology.orchestra.run.smithi018.stdout:e33 2024-04-12T21:40:37.510 INFO:teuthology.orchestra.run.smithi018.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-12T21:40:37.510 INFO:teuthology.orchestra.run.smithi018.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-12T21:40:37.510 INFO:teuthology.orchestra.run.smithi018.stdout:legacy client fscid: 1 2024-04-12T21:40:37.510 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-12T21:40:37.510 INFO:teuthology.orchestra.run.smithi018.stdout:Filesystem 'cephfs' (1) 2024-04-12T21:40:37.510 INFO:teuthology.orchestra.run.smithi018.stdout:fs_name cephfs 2024-04-12T21:40:37.510 INFO:teuthology.orchestra.run.smithi018.stdout:epoch 24 2024-04-12T21:40:37.510 INFO:teuthology.orchestra.run.smithi018.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-04-12T21:40:37.510 INFO:teuthology.orchestra.run.smithi018.stdout:created 2024-04-12T20:51:57.362850+0000 2024-04-12T21:40:37.511 INFO:teuthology.orchestra.run.smithi018.stdout:modified 2024-04-12T21:06:59.717190+0000 2024-04-12T21:40:37.511 INFO:teuthology.orchestra.run.smithi018.stdout:tableserver 0 2024-04-12T21:40:37.511 INFO:teuthology.orchestra.run.smithi018.stdout:root 0 2024-04-12T21:40:37.511 INFO:teuthology.orchestra.run.smithi018.stdout:session_timeout 60 2024-04-12T21:40:37.511 INFO:teuthology.orchestra.run.smithi018.stdout:session_autoclose 300 2024-04-12T21:40:37.511 INFO:teuthology.orchestra.run.smithi018.stdout:max_file_size 1099511627776 2024-04-12T21:40:37.511 INFO:teuthology.orchestra.run.smithi018.stdout:max_xattr_size 65536 2024-04-12T21:40:37.511 INFO:teuthology.orchestra.run.smithi018.stdout:required_client_features {} 2024-04-12T21:40:37.511 INFO:teuthology.orchestra.run.smithi018.stdout:last_failure 0 2024-04-12T21:40:37.511 INFO:teuthology.orchestra.run.smithi018.stdout:last_failure_osd_epoch 100 2024-04-12T21:40:37.511 INFO:teuthology.orchestra.run.smithi018.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-12T21:40:37.511 INFO:teuthology.orchestra.run.smithi018.stdout:max_mds 2 2024-04-12T21:40:37.511 INFO:teuthology.orchestra.run.smithi018.stdout:in 0,1 2024-04-12T21:40:37.512 INFO:teuthology.orchestra.run.smithi018.stdout:up {} 2024-04-12T21:40:37.512 INFO:teuthology.orchestra.run.smithi018.stdout:failed 0,1 2024-04-12T21:40:37.512 INFO:teuthology.orchestra.run.smithi018.stdout:damaged 2024-04-12T21:40:37.512 INFO:teuthology.orchestra.run.smithi018.stdout:stopped 2024-04-12T21:40:37.512 INFO:teuthology.orchestra.run.smithi018.stdout:data_pools [3] 2024-04-12T21:40:37.512 INFO:teuthology.orchestra.run.smithi018.stdout:metadata_pool 2 2024-04-12T21:40:37.512 INFO:teuthology.orchestra.run.smithi018.stdout:inline_data disabled 2024-04-12T21:40:37.512 INFO:teuthology.orchestra.run.smithi018.stdout:balancer 2024-04-12T21:40:37.512 INFO:teuthology.orchestra.run.smithi018.stdout:bal_rank_mask -1 2024-04-12T21:40:37.512 INFO:teuthology.orchestra.run.smithi018.stdout:standby_count_wanted 1 2024-04-12T21:40:37.512 INFO:teuthology.orchestra.run.smithi018.stdout:qdb_cluster leader: 0 members: 2024-04-12T21:40:37.512 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-12T21:40:37.512 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-12T21:40:37.513 INFO:teuthology.orchestra.run.smithi018.stdout:Standby daemons: 2024-04-12T21:40:37.513 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-12T21:40:37.513 INFO:teuthology.orchestra.run.smithi018.stdout:[mds.cephfs.smithi018.acqbsc{-1:34436} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.18:6828/755179764,v1:172.21.15.18:6829/755179764] compat {c=[1],r=[1],i=[1fff]}] 2024-04-12T21:40:37.513 INFO:teuthology.orchestra.run.smithi018.stdout:[mds.cephfs.smithi018.szjluo{-1:34468} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.18:6826/2407992653,v1:172.21.15.18:6827/2407992653] compat {c=[1],r=[1],i=[1fff]}] 2024-04-12T21:40:37.513 INFO:teuthology.orchestra.run.smithi018.stdout:[mds.cephfs.smithi186.jdllyn{-1:44595} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.186:6824/3523444202,v1:172.21.15.186:6825/3523444202] compat {c=[1],r=[1],i=[1fff]}] 2024-04-12T21:40:37.513 INFO:teuthology.orchestra.run.smithi018.stdout:[mds.cephfs.smithi186.wjnafs{-1:44875} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.186:6826/3122443766,v1:172.21.15.186:6827/3122443766] compat {c=[1],r=[1],i=[1fff]}] 2024-04-12T21:40:37.514 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:37.512+0000 7f7895a00700 1 -- 172.21.15.18:0/1059166748 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f787406ec00 msgr2=0x7f78740710b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:40:37.514 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:37.512+0000 7f7895a00700 1 --2- 172.21.15.18:0/1059166748 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f787406ec00 0x7f78740710b0 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f78800041d0 tx=0x7f7880009040 comp rx=0 tx=0).stop 2024-04-12T21:40:37.514 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:37.512+0000 7f7895a00700 1 -- 172.21.15.18:0/1059166748 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7890071f90 msgr2=0x7f789011a4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:40:37.514 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:37.512+0000 7f7895a00700 1 --2- 172.21.15.18:0/1059166748 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7890071f90 0x7f789011a4b0 secure :-1 s=READY pgs=1039 cs=0 l=1 rev1=1 crypto rx=0x7f787c004f50 tx=0x7f787c009e30 comp rx=0 tx=0).stop 2024-04-12T21:40:37.514 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:37.512+0000 7f7895a00700 1 -- 172.21.15.18:0/1059166748 shutdown_connections 2024-04-12T21:40:37.514 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:37.512+0000 7f7895a00700 1 --2- 172.21.15.18:0/1059166748 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f787406ec00 0x7f78740710b0 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:40:37.514 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:37.512+0000 7f7895a00700 1 --2- 172.21.15.18:0/1059166748 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f78900708a0 0x7f7890119f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:40:37.514 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:37.512+0000 7f7895a00700 1 --2- 172.21.15.18:0/1059166748 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7890071f90 0x7f789011a4b0 unknown :-1 s=CLOSED pgs=1039 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:40:37.514 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:37.512+0000 7f7895a00700 1 -- 172.21.15.18:0/1059166748 >> 172.21.15.18:0/1059166748 conn(0x7f78900f89b0 msgr2=0x7f78900f9aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:40:37.515 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:37.512+0000 7f7895a00700 1 -- 172.21.15.18:0/1059166748 shutdown_connections 2024-04-12T21:40:37.515 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:37.512+0000 7f7895a00700 1 -- 172.21.15.18:0/1059166748 wait complete. 2024-04-12T21:40:37.515 INFO:teuthology.orchestra.run.smithi018.stderr:dumped fsmap epoch 33 2024-04-12T21:40:37.670 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:37.670+0000 7f420ae00700 1 -- 172.21.15.18:0/2464121784 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4204109dd0 msgr2=0x7f4204103440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:40:37.670 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:37.670+0000 7f420ae00700 1 --2- 172.21.15.18:0/2464121784 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4204109dd0 0x7f4204103440 secure :-1 s=READY pgs=1040 cs=0 l=1 rev1=1 crypto rx=0x7f41f0004940 tx=0x7f41f0009e30 comp rx=0 tx=0).stop 2024-04-12T21:40:37.671 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:37.670+0000 7f420ae00700 1 -- 172.21.15.18:0/2464121784 shutdown_connections 2024-04-12T21:40:37.671 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:37.670+0000 7f420ae00700 1 --2- 172.21.15.18:0/2464121784 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4204109dd0 0x7f4204103440 unknown :-1 s=CLOSED pgs=1040 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:40:37.671 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:37.670+0000 7f420ae00700 1 --2- 172.21.15.18:0/2464121784 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4204109430 0x7f4204109800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:40:37.671 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:37.670+0000 7f420ae00700 1 -- 172.21.15.18:0/2464121784 >> 172.21.15.18:0/2464121784 conn(0x7f4204072c00 msgr2=0x7f4204075010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:40:37.671 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:37.670+0000 7f420ae00700 1 -- 172.21.15.18:0/2464121784 shutdown_connections 2024-04-12T21:40:37.671 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:37.670+0000 7f420ae00700 1 -- 172.21.15.18:0/2464121784 wait complete. 2024-04-12T21:40:37.671 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:37.672+0000 7f420ae00700 1 Processor -- start 2024-04-12T21:40:37.671 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:37.672+0000 7f420ae00700 1 -- start start 2024-04-12T21:40:37.673 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:37.672+0000 7f420ae00700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4204109430 0x7f4204119f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:40:37.673 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:37.672+0000 7f420ae00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4204109dd0 0x7f420411a4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:40:37.673 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:37.672+0000 7f420ae00700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f420411ab10 con 0x7f4204109dd0 2024-04-12T21:40:37.673 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:37.672+0000 7f420ae00700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42041140f0 con 0x7f4204109430 2024-04-12T21:40:37.673 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:37.673+0000 7f4203400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4204109dd0 0x7f420411a4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:40:37.673 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:37.673+0000 7f4203400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4204109dd0 0x7f420411a4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:56548/0 (socket says 172.21.15.18:56548) 2024-04-12T21:40:37.673 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:37.673+0000 7f4203400700 1 -- 172.21.15.18:0/558998822 learned_addr learned my addr 172.21.15.18:0/558998822 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:40:37.673 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:37.673+0000 7f4203e00700 1 --2- 172.21.15.18:0/558998822 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4204109430 0x7f4204119f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:40:37.673 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:37.673+0000 7f4203400700 1 -- 172.21.15.18:0/558998822 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4204109430 msgr2=0x7f4204119f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:40:37.674 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:37.673+0000 7f4203400700 1 --2- 172.21.15.18:0/558998822 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4204109430 0x7f4204119f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:40:37.674 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:37.673+0000 7f4203400700 1 -- 172.21.15.18:0/558998822 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41f0005040 con 0x7f4204109dd0 2024-04-12T21:40:37.674 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:37.674+0000 7f4203400700 1 --2- 172.21.15.18:0/558998822 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4204109dd0 0x7f420411a4c0 secure :-1 s=READY pgs=1041 cs=0 l=1 rev1=1 crypto rx=0x7f41f0009180 tx=0x7f41f0009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:40:37.675 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:37.674+0000 7f41fbe00700 1 -- 172.21.15.18:0/558998822 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f41f000e040 con 0x7f4204109dd0 2024-04-12T21:40:37.675 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:37.674+0000 7f41fbe00700 1 -- 172.21.15.18:0/558998822 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f41f000bbc0 con 0x7f4204109dd0 2024-04-12T21:40:37.675 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:37.674+0000 7f420ae00700 1 -- 172.21.15.18:0/558998822 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4204114370 con 0x7f4204109dd0 2024-04-12T21:40:37.675 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:37.674+0000 7f41fbe00700 1 -- 172.21.15.18:0/558998822 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f41f001c3f0 con 0x7f4204109dd0 2024-04-12T21:40:37.675 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:37.674+0000 7f420ae00700 1 -- 172.21.15.18:0/558998822 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4204114930 con 0x7f4204109dd0 2024-04-12T21:40:37.675 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:37.675+0000 7f420ae00700 1 -- 172.21.15.18:0/558998822 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4204104b80 con 0x7f4204109dd0 2024-04-12T21:40:37.676 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:37.676+0000 7f41fbe00700 1 -- 172.21.15.18:0/558998822 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7f41f000bd30 con 0x7f4204109dd0 2024-04-12T21:40:37.677 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:37.677+0000 7f41fbe00700 1 --2- 172.21.15.18:0/558998822 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f41e806ec00 0x7f41e80710b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:40:37.677 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:37.677+0000 7f41fbe00700 1 -- 172.21.15.18:0/558998822 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 6757+0+0 (secure 0 0 0) 0x7f41f008fd00 con 0x7f4204109dd0 2024-04-12T21:40:37.677 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:37.677+0000 7f4203e00700 1 --2- 172.21.15.18:0/558998822 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f41e806ec00 0x7f41e80710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:40:37.677 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:37.678+0000 7f4203e00700 1 --2- 172.21.15.18:0/558998822 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f41e806ec00 0x7f41e80710b0 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f41f4004990 tx=0x7f41f4009ee0 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:40:37.681 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:37.682+0000 7f41fbe00700 1 -- 172.21.15.18:0/558998822 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f41f005c530 con 0x7f4204109dd0 2024-04-12T21:40:37.933 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:37.932+0000 7f420ae00700 1 -- 172.21.15.18:0/558998822 --> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f4204110500 con 0x7f41e806ec00 2024-04-12T21:40:37.935 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:37.935+0000 7f41fbe00700 1 -- 172.21.15.18:0/558998822 <== mgr.34102 v2:172.21.15.18:6800/3812891919 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f4204110500 con 0x7f41e806ec00 2024-04-12T21:40:37.935 INFO:teuthology.orchestra.run.smithi018.stdout:{ 2024-04-12T21:40:37.935 INFO:teuthology.orchestra.run.smithi018.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:f4452def3be4d350f4a8e5feaf7ebb1511be9298fe68478ba7a2d6fbd7d69a63", 2024-04-12T21:40:37.935 INFO:teuthology.orchestra.run.smithi018.stdout: "in_progress": true, 2024-04-12T21:40:37.935 INFO:teuthology.orchestra.run.smithi018.stdout: "which": "Upgrading all daemon types on all hosts", 2024-04-12T21:40:37.936 INFO:teuthology.orchestra.run.smithi018.stdout: "services_complete": [ 2024-04-12T21:40:37.936 INFO:teuthology.orchestra.run.smithi018.stdout: "mgr", 2024-04-12T21:40:37.936 INFO:teuthology.orchestra.run.smithi018.stdout: "mds", 2024-04-12T21:40:37.936 INFO:teuthology.orchestra.run.smithi018.stdout: "crash", 2024-04-12T21:40:37.936 INFO:teuthology.orchestra.run.smithi018.stdout: "osd", 2024-04-12T21:40:37.936 INFO:teuthology.orchestra.run.smithi018.stdout: "mon" 2024-04-12T21:40:37.936 INFO:teuthology.orchestra.run.smithi018.stdout: ], 2024-04-12T21:40:37.936 INFO:teuthology.orchestra.run.smithi018.stdout: "progress": "16/21 daemons upgraded", 2024-04-12T21:40:37.936 INFO:teuthology.orchestra.run.smithi018.stdout: "message": "Currently upgrading mds daemons", 2024-04-12T21:40:37.936 INFO:teuthology.orchestra.run.smithi018.stdout: "is_paused": false 2024-04-12T21:40:37.936 INFO:teuthology.orchestra.run.smithi018.stdout:} 2024-04-12T21:40:37.938 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:37.938+0000 7f420ae00700 1 -- 172.21.15.18:0/558998822 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f41e806ec00 msgr2=0x7f41e80710b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:40:37.939 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:37.938+0000 7f420ae00700 1 --2- 172.21.15.18:0/558998822 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f41e806ec00 0x7f41e80710b0 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f41f4004990 tx=0x7f41f4009ee0 comp rx=0 tx=0).stop 2024-04-12T21:40:37.939 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:37.938+0000 7f420ae00700 1 -- 172.21.15.18:0/558998822 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4204109dd0 msgr2=0x7f420411a4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:40:37.939 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:37.938+0000 7f420ae00700 1 --2- 172.21.15.18:0/558998822 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4204109dd0 0x7f420411a4c0 secure :-1 s=READY pgs=1041 cs=0 l=1 rev1=1 crypto rx=0x7f41f0009180 tx=0x7f41f0009e30 comp rx=0 tx=0).stop 2024-04-12T21:40:37.939 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:37.938+0000 7f420ae00700 1 -- 172.21.15.18:0/558998822 shutdown_connections 2024-04-12T21:40:37.939 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:37.938+0000 7f420ae00700 1 --2- 172.21.15.18:0/558998822 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f41e806ec00 0x7f41e80710b0 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:40:37.939 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:37.938+0000 7f420ae00700 1 --2- 172.21.15.18:0/558998822 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4204109430 0x7f4204119f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:40:37.939 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:37.939+0000 7f420ae00700 1 --2- 172.21.15.18:0/558998822 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4204109dd0 0x7f420411a4c0 unknown :-1 s=CLOSED pgs=1041 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:40:37.939 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:37.939+0000 7f420ae00700 1 -- 172.21.15.18:0/558998822 >> 172.21.15.18:0/558998822 conn(0x7f4204072c00 msgr2=0x7f4204074c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:40:37.939 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:37.939+0000 7f420ae00700 1 -- 172.21.15.18:0/558998822 shutdown_connections 2024-04-12T21:40:37.939 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:37.939+0000 7f420ae00700 1 -- 172.21.15.18:0/558998822 wait complete. 2024-04-12T21:40:38.033 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:40:37 smithi186 ceph-mon[86379]: pgmap v1228: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:40:38.034 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:40:37 smithi186 ceph-mon[86379]: from='client.36136 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:40:38.034 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:40:37 smithi186 ceph-mon[86379]: from='client.36140 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:40:38.034 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:40:37 smithi186 ceph-mon[86379]: from='client.? 172.21.15.18:0/3865273997' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T21:40:38.034 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:40:37 smithi186 ceph-mon[86379]: from='client.? 172.21.15.18:0/1059166748' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-12T21:40:38.093 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:38.093+0000 7fd892600700 1 -- 172.21.15.18:0/2642268799 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd88c109dd0 msgr2=0x7fd88c103440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:40:38.093 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:38.093+0000 7fd892600700 1 --2- 172.21.15.18:0/2642268799 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd88c109dd0 0x7fd88c103440 secure :-1 s=READY pgs=1042 cs=0 l=1 rev1=1 crypto rx=0x7fd878004940 tx=0x7fd878009e30 comp rx=0 tx=0).stop 2024-04-12T21:40:38.093 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:40:37 smithi018 ceph-mon[112590]: pgmap v1228: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:40:38.093 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:40:37 smithi018 ceph-mon[112590]: from='client.36136 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:40:38.093 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:40:37 smithi018 ceph-mon[112590]: from='client.36140 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:40:38.093 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:40:37 smithi018 ceph-mon[112590]: from='client.? 172.21.15.18:0/3865273997' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T21:40:38.094 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:40:37 smithi018 ceph-mon[112590]: from='client.? 172.21.15.18:0/1059166748' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-12T21:40:38.094 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:38.093+0000 7fd892600700 1 -- 172.21.15.18:0/2642268799 shutdown_connections 2024-04-12T21:40:38.094 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:38.093+0000 7fd892600700 1 --2- 172.21.15.18:0/2642268799 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd88c109dd0 0x7fd88c103440 unknown :-1 s=CLOSED pgs=1042 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:40:38.094 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:38.093+0000 7fd892600700 1 --2- 172.21.15.18:0/2642268799 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd88c109430 0x7fd88c109800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:40:38.094 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:38.093+0000 7fd892600700 1 -- 172.21.15.18:0/2642268799 >> 172.21.15.18:0/2642268799 conn(0x7fd88c072c00 msgr2=0x7fd88c075010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:40:38.094 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:38.093+0000 7fd892600700 1 -- 172.21.15.18:0/2642268799 shutdown_connections 2024-04-12T21:40:38.095 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:38.093+0000 7fd892600700 1 -- 172.21.15.18:0/2642268799 wait complete. 2024-04-12T21:40:38.095 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:38.094+0000 7fd892600700 1 Processor -- start 2024-04-12T21:40:38.095 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:38.094+0000 7fd892600700 1 -- start start 2024-04-12T21:40:38.095 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:38.094+0000 7fd892600700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd88c109430 0x7fd88c119fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:40:38.095 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:38.094+0000 7fd892600700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd88c109dd0 0x7fd88c11a4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:40:38.095 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:38.094+0000 7fd892600700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd88c11abc0 con 0x7fd88c109430 2024-04-12T21:40:38.095 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:38.094+0000 7fd892600700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd88c114110 con 0x7fd88c109dd0 2024-04-12T21:40:38.096 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:38.095+0000 7fd88b400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd88c109430 0x7fd88c119fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:40:38.096 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:38.095+0000 7fd88b400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd88c109430 0x7fd88c119fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:56560/0 (socket says 172.21.15.18:56560) 2024-04-12T21:40:38.096 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:38.095+0000 7fd88b400700 1 -- 172.21.15.18:0/320911186 learned_addr learned my addr 172.21.15.18:0/320911186 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:40:38.096 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:38.095+0000 7fd88aa00700 1 --2- 172.21.15.18:0/320911186 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd88c109dd0 0x7fd88c11a4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:40:38.097 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:38.095+0000 7fd88b400700 1 -- 172.21.15.18:0/320911186 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd88c109dd0 msgr2=0x7fd88c11a4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:40:38.097 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:38.095+0000 7fd88b400700 1 --2- 172.21.15.18:0/320911186 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd88c109dd0 0x7fd88c11a4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:40:38.097 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:38.095+0000 7fd88b400700 1 -- 172.21.15.18:0/320911186 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd878005040 con 0x7fd88c109430 2024-04-12T21:40:38.097 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:38.096+0000 7fd88b400700 1 --2- 172.21.15.18:0/320911186 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd88c109430 0x7fd88c119fa0 secure :-1 s=READY pgs=1043 cs=0 l=1 rev1=1 crypto rx=0x7fd87c00b790 tx=0x7fd87c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:40:38.097 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:38.096+0000 7fd883400700 1 -- 172.21.15.18:0/320911186 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd87c00f770 con 0x7fd88c109430 2024-04-12T21:40:38.097 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:38.096+0000 7fd883400700 1 -- 172.21.15.18:0/320911186 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd87c00fda0 con 0x7fd88c109430 2024-04-12T21:40:38.097 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:38.096+0000 7fd892600700 1 -- 172.21.15.18:0/320911186 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd88c114390 con 0x7fd88c109430 2024-04-12T21:40:38.097 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:38.096+0000 7fd892600700 1 -- 172.21.15.18:0/320911186 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd88c114950 con 0x7fd88c109430 2024-04-12T21:40:38.097 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:38.096+0000 7fd883400700 1 -- 172.21.15.18:0/320911186 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd87c00d310 con 0x7fd88c109430 2024-04-12T21:40:38.097 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:38.097+0000 7fd892600700 1 -- 172.21.15.18:0/320911186 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd86c0052b0 con 0x7fd88c109430 2024-04-12T21:40:38.098 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:38.099+0000 7fd883400700 1 -- 172.21.15.18:0/320911186 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7fd87c010420 con 0x7fd88c109430 2024-04-12T21:40:38.099 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:38.099+0000 7fd883400700 1 --2- 172.21.15.18:0/320911186 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fd87006ec00 0x7fd8700710b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:40:38.099 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:38.100+0000 7fd883400700 1 -- 172.21.15.18:0/320911186 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 6757+0+0 (secure 0 0 0) 0x7fd87c08ea30 con 0x7fd88c109430 2024-04-12T21:40:38.099 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:38.100+0000 7fd88aa00700 1 --2- 172.21.15.18:0/320911186 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fd87006ec00 0x7fd8700710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:40:38.100 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:38.100+0000 7fd88aa00700 1 --2- 172.21.15.18:0/320911186 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fd87006ec00 0x7fd8700710b0 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7fd88c115840 tx=0x7fd878009e30 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:40:38.104 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:38.105+0000 7fd883400700 1 -- 172.21.15.18:0/320911186 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7fd87c05b1e0 con 0x7fd88c109430 2024-04-12T21:40:38.413 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:38.413+0000 7fd892600700 1 -- 172.21.15.18:0/320911186 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7fd86c005560 con 0x7fd88c109430 2024-04-12T21:40:38.414 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:38.414+0000 7fd883400700 1 -- 172.21.15.18:0/320911186 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+4303 (secure 0 0 0) 0x7fd87c05ad70 con 0x7fd88c109430 2024-04-12T21:40:38.414 INFO:teuthology.orchestra.run.smithi018.stdout:HEALTH_ERR failed to probe daemons or devices; 1 filesystem is degraded; 1 filesystem is offline 2024-04-12T21:40:38.414 INFO:teuthology.orchestra.run.smithi018.stdout:[WRN] CEPHADM_REFRESH_FAILED: failed to probe daemons or devices 2024-04-12T21:40:38.414 INFO:teuthology.orchestra.run.smithi018.stdout: host smithi186 `cephadm ceph-volume` failed: cephadm exited with an error code: 1, stderr: Inferring config /var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/mon.smithi186/config 2024-04-12T21:40:38.414 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/podman: timeout after 895 seconds 2024-04-12T21:40:38.414 INFO:teuthology.orchestra.run.smithi018.stdout:Non-zero exit code 124 from /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:f4452def3be4d350f4a8e5feaf7ebb1511be9298fe68478ba7a2d6fbd7d69a63 -e NODE_NAME=smithi186 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968:/var/run/ceph:z -v /var/log/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968:/var/log/ceph:z -v /var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmptgom53f9:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:f4452def3be4d350f4a8e5feaf7ebb1511be9298fe68478ba7a2d6fbd7d69a63 inventory --format=json-pretty --filter-for-batch 2024-04-12T21:40:38.415 INFO:teuthology.orchestra.run.smithi018.stdout:Traceback (most recent call last): 2024-04-12T21:40:38.415 INFO:teuthology.orchestra.run.smithi018.stdout: File "/usr/lib64/python3.9/runpy.py", line 197, in _run_module_as_main 2024-04-12T21:40:38.415 INFO:teuthology.orchestra.run.smithi018.stdout: return _run_code(code, main_globals, None, 2024-04-12T21:40:38.415 INFO:teuthology.orchestra.run.smithi018.stdout: File "/usr/lib64/python3.9/runpy.py", line 87, in _run_code 2024-04-12T21:40:38.415 INFO:teuthology.orchestra.run.smithi018.stdout: exec(code, run_globals) 2024-04-12T21:40:38.415 INFO:teuthology.orchestra.run.smithi018.stdout: File "/var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/cephadm.e44f3ed5e34e4ef52eedef233d1190bff004470d8e8005127b5312bff357fa7f/__main__.py", line 5618, in 2024-04-12T21:40:38.415 INFO:teuthology.orchestra.run.smithi018.stdout: File "/var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/cephadm.e44f3ed5e34e4ef52eedef233d1190bff004470d8e8005127b5312bff357fa7f/__main__.py", line 5606, in main 2024-04-12T21:40:38.415 INFO:teuthology.orchestra.run.smithi018.stdout: File "/var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/cephadm.e44f3ed5e34e4ef52eedef233d1190bff004470d8e8005127b5312bff357fa7f/__main__.py", line 408, in _infer_config 2024-04-12T21:40:38.415 INFO:teuthology.orchestra.run.smithi018.stdout: File "/var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/cephadm.e44f3ed5e34e4ef52eedef233d1190bff004470d8e8005127b5312bff357fa7f/__main__.py", line 323, in _infer_fsid 2024-04-12T21:40:38.415 INFO:teuthology.orchestra.run.smithi018.stdout: File "/var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/cephadm.e44f3ed5e34e4ef52eedef233d1190bff004470d8e8005127b5312bff357fa7f/__main__.py", line 436, in _infer_image 2024-04-12T21:40:38.415 INFO:teuthology.orchestra.run.smithi018.stdout: File "/var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/cephadm.e44f3ed5e34e4ef52eedef233d1190bff004470d8e8005127b5312bff357fa7f/__main__.py", line 310, in _validate_fsid 2024-04-12T21:40:38.415 INFO:teuthology.orchestra.run.smithi018.stdout: File "/var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/cephadm.e44f3ed5e34e4ef52eedef233d1190bff004470d8e8005127b5312bff357fa7f/__main__.py", line 3359, in command_ceph_volume 2024-04-12T21:40:38.416 INFO:teuthology.orchestra.run.smithi018.stdout: File "/var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/cephadm.e44f3ed5e34e4ef52eedef233d1190bff004470d8e8005127b5312bff357fa7f/cephadmlib/call_wrappers.py", line 310, in call_throws 2024-04-12T21:40:38.416 INFO:teuthology.orchestra.run.smithi018.stdout:RuntimeError: Failed command: /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:f4452def3be4d350f4a8e5feaf7ebb1511be9298fe68478ba7a2d6fbd7d69a63 -e NODE_NAME=smithi186 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968:/var/run/ceph:z -v /var/log/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968:/var/log/ceph:z -v /var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmptgom53f9:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:f4452def3be4d350f4a8e5feaf7ebb1511be9298fe68478ba7a2d6fbd7d69a63 inventory --format=json-pretty --filter-for-batch 2024-04-12T21:40:38.416 INFO:teuthology.orchestra.run.smithi018.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-04-12T21:40:38.416 INFO:teuthology.orchestra.run.smithi018.stdout: fs cephfs is degraded 2024-04-12T21:40:38.416 INFO:teuthology.orchestra.run.smithi018.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-04-12T21:40:38.416 INFO:teuthology.orchestra.run.smithi018.stdout: fs cephfs is offline because no MDS is active for it. 2024-04-12T21:40:38.417 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:38.417+0000 7fd892600700 1 -- 172.21.15.18:0/320911186 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fd87006ec00 msgr2=0x7fd8700710b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:40:38.417 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:38.417+0000 7fd892600700 1 --2- 172.21.15.18:0/320911186 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fd87006ec00 0x7fd8700710b0 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7fd88c115840 tx=0x7fd878009e30 comp rx=0 tx=0).stop 2024-04-12T21:40:38.418 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:38.417+0000 7fd892600700 1 -- 172.21.15.18:0/320911186 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd88c109430 msgr2=0x7fd88c119fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:40:38.418 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:38.417+0000 7fd892600700 1 --2- 172.21.15.18:0/320911186 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd88c109430 0x7fd88c119fa0 secure :-1 s=READY pgs=1043 cs=0 l=1 rev1=1 crypto rx=0x7fd87c00b790 tx=0x7fd87c009250 comp rx=0 tx=0).stop 2024-04-12T21:40:38.418 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:38.417+0000 7fd892600700 1 -- 172.21.15.18:0/320911186 shutdown_connections 2024-04-12T21:40:38.418 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:38.417+0000 7fd892600700 1 --2- 172.21.15.18:0/320911186 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fd87006ec00 0x7fd8700710b0 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:40:38.418 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:38.417+0000 7fd892600700 1 --2- 172.21.15.18:0/320911186 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd88c109430 0x7fd88c119fa0 unknown :-1 s=CLOSED pgs=1043 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:40:38.418 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:38.417+0000 7fd892600700 1 --2- 172.21.15.18:0/320911186 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd88c109dd0 0x7fd88c11a4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:40:38.418 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:38.417+0000 7fd892600700 1 -- 172.21.15.18:0/320911186 >> 172.21.15.18:0/320911186 conn(0x7fd88c072c00 msgr2=0x7fd88c074b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:40:38.418 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:38.418+0000 7fd892600700 1 -- 172.21.15.18:0/320911186 shutdown_connections 2024-04-12T21:40:38.418 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:40:38.418+0000 7fd892600700 1 -- 172.21.15.18:0/320911186 wait complete. 2024-04-12T21:40:39.104 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:40:38 smithi018 ceph-mon[112590]: from='client.36152 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:40:39.104 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:40:38 smithi018 ceph-mon[112590]: from='client.? 172.21.15.18:0/320911186' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-12T21:40:39.283 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:40:38 smithi186 ceph-mon[86379]: from='client.36152 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:40:39.283 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:40:38 smithi186 ceph-mon[86379]: from='client.? 172.21.15.18:0/320911186' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-12T21:40:40.104 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:40:39 smithi018 ceph-mon[112590]: pgmap v1229: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:40:40.283 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:40:39 smithi186 ceph-mon[86379]: pgmap v1229: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:40:41.104 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:40:40 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T21:40:41.283 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:40:40 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T21:40:42.104 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:40:41 smithi018 ceph-mon[112590]: pgmap v1230: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:40:42.283 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:40:41 smithi186 ceph-mon[86379]: pgmap v1230: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:40:44.104 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:40:43 smithi018 ceph-mon[112590]: pgmap v1231: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:40:44.283 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:40:43 smithi186 ceph-mon[86379]: pgmap v1231: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:40:46.104 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:40:45 smithi018 ceph-mon[112590]: pgmap v1232: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:40:46.283 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:40:45 smithi186 ceph-mon[86379]: pgmap v1232: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:40:48.104 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:40:47 smithi018 ceph-mon[112590]: pgmap v1233: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:40:48.283 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:40:47 smithi186 ceph-mon[86379]: pgmap v1233: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:40:50.104 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:40:49 smithi018 ceph-mon[112590]: pgmap v1234: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:40:50.283 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:40:49 smithi186 ceph-mon[86379]: pgmap v1234: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:40:52.104 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:40:51 smithi018 ceph-mon[112590]: pgmap v1235: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:40:52.283 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:40:51 smithi186 ceph-mon[86379]: pgmap v1235: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:40:54.104 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:40:53 smithi018 ceph-mon[112590]: pgmap v1236: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:40:54.283 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:40:53 smithi186 ceph-mon[86379]: pgmap v1236: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:40:56.283 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:40:55 smithi186 ceph-mon[86379]: pgmap v1237: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:40:56.283 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:40:55 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T21:40:56.354 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:40:55 smithi018 ceph-mon[112590]: pgmap v1237: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:40:56.355 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:40:55 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T21:40:58.283 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:40:57 smithi186 ceph-mon[86379]: pgmap v1238: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:40:58.355 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:40:57 smithi018 ceph-mon[112590]: pgmap v1238: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:41:00.283 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:40:59 smithi186 ceph-mon[86379]: pgmap v1239: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:41:00.354 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:40:59 smithi018 ceph-mon[112590]: pgmap v1239: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:41:02.283 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:41:01 smithi186 ceph-mon[86379]: pgmap v1240: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:41:02.354 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:41:01 smithi018 ceph-mon[112590]: pgmap v1240: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:41:04.283 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:41:03 smithi186 ceph-mon[86379]: pgmap v1241: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:41:04.354 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:41:03 smithi018 ceph-mon[112590]: pgmap v1241: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:41:06.283 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:41:05 smithi186 ceph-mon[86379]: pgmap v1242: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:41:06.354 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:41:05 smithi018 ceph-mon[112590]: pgmap v1242: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:41:08.283 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:41:07 smithi186 ceph-mon[86379]: pgmap v1243: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:41:08.354 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:41:07 smithi018 ceph-mon[112590]: pgmap v1243: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:41:08.590 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:08.589+0000 7f8b90c00700 1 -- 172.21.15.18:0/63182836 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8b8c0fe830 msgr2=0x7f8b8c106f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:41:08.590 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:08.589+0000 7f8b90c00700 1 --2- 172.21.15.18:0/63182836 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8b8c0fe830 0x7f8b8c106f00 secure :-1 s=READY pgs=1044 cs=0 l=1 rev1=1 crypto rx=0x7f8b74004940 tx=0x7f8b74009e30 comp rx=0 tx=0).stop 2024-04-12T21:41:08.590 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:08.590+0000 7f8b90c00700 1 -- 172.21.15.18:0/63182836 shutdown_connections 2024-04-12T21:41:08.590 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:08.590+0000 7f8b90c00700 1 --2- 172.21.15.18:0/63182836 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8b8c0fe830 0x7f8b8c106f00 unknown :-1 s=CLOSED pgs=1044 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:41:08.590 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:08.590+0000 7f8b90c00700 1 --2- 172.21.15.18:0/63182836 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8b8c0fde90 0x7f8b8c0fe260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:41:08.590 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:08.590+0000 7f8b90c00700 1 -- 172.21.15.18:0/63182836 >> 172.21.15.18:0/63182836 conn(0x7f8b8c072bc0 msgr2=0x7f8b8c072fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:41:08.590 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:08.590+0000 7f8b90c00700 1 -- 172.21.15.18:0/63182836 shutdown_connections 2024-04-12T21:41:08.591 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:08.590+0000 7f8b90c00700 1 -- 172.21.15.18:0/63182836 wait complete. 2024-04-12T21:41:08.591 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:08.591+0000 7f8b90c00700 1 Processor -- start 2024-04-12T21:41:08.591 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:08.591+0000 7f8b90c00700 1 -- start start 2024-04-12T21:41:08.592 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:08.592+0000 7f8b90c00700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8b8c0fde90 0x7f8b8c0fbd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:41:08.592 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:08.592+0000 7f8b90c00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8b8c0fe830 0x7f8b8c0fc290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:41:08.592 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:08.592+0000 7f8b90c00700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b8c0fc820 con 0x7f8b8c0fe830 2024-04-12T21:41:08.592 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:08.592+0000 7f8b90c00700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b8c0fc960 con 0x7f8b8c0fde90 2024-04-12T21:41:08.592 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:08.592+0000 7f8b81400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8b8c0fe830 0x7f8b8c0fc290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:41:08.592 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:08.592+0000 7f8b81400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8b8c0fe830 0x7f8b8c0fc290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:40044/0 (socket says 172.21.15.18:40044) 2024-04-12T21:41:08.592 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:08.592+0000 7f8b81400700 1 -- 172.21.15.18:0/708898873 learned_addr learned my addr 172.21.15.18:0/708898873 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:41:08.592 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:08.592+0000 7f8b89e00700 1 --2- 172.21.15.18:0/708898873 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8b8c0fde90 0x7f8b8c0fbd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:41:08.593 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:08.593+0000 7f8b81400700 1 -- 172.21.15.18:0/708898873 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8b8c0fde90 msgr2=0x7f8b8c0fbd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:41:08.593 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:08.593+0000 7f8b81400700 1 --2- 172.21.15.18:0/708898873 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8b8c0fde90 0x7f8b8c0fbd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:41:08.593 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:08.593+0000 7f8b81400700 1 -- 172.21.15.18:0/708898873 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b74005040 con 0x7f8b8c0fe830 2024-04-12T21:41:08.594 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:08.593+0000 7f8b81400700 1 --2- 172.21.15.18:0/708898873 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8b8c0fe830 0x7f8b8c0fc290 secure :-1 s=READY pgs=1045 cs=0 l=1 rev1=1 crypto rx=0x7f8b74009360 tx=0x7f8b74009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:41:08.594 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:08.593+0000 7f8b82a00700 1 -- 172.21.15.18:0/708898873 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8b7400e040 con 0x7f8b8c0fe830 2024-04-12T21:41:08.594 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:08.593+0000 7f8b90c00700 1 -- 172.21.15.18:0/708898873 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8b8c0fcbb0 con 0x7f8b8c0fe830 2024-04-12T21:41:08.594 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:08.593+0000 7f8b82a00700 1 -- 172.21.15.18:0/708898873 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f8b74013420 con 0x7f8b8c0fe830 2024-04-12T21:41:08.594 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:08.593+0000 7f8b82a00700 1 -- 172.21.15.18:0/708898873 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8b740255e0 con 0x7f8b8c0fe830 2024-04-12T21:41:08.594 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:08.593+0000 7f8b90c00700 1 -- 172.21.15.18:0/708898873 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8b8c11a250 con 0x7f8b8c0fe830 2024-04-12T21:41:08.594 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:08.595+0000 7f8b90c00700 1 -- 172.21.15.18:0/708898873 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8b8c065e30 con 0x7f8b8c0fe830 2024-04-12T21:41:08.595 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:08.596+0000 7f8b82a00700 1 -- 172.21.15.18:0/708898873 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7f8b74013a90 con 0x7f8b8c0fe830 2024-04-12T21:41:08.597 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:08.597+0000 7f8b82a00700 1 --2- 172.21.15.18:0/708898873 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f8b7006ec00 0x7f8b700710b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:41:08.597 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:08.597+0000 7f8b82a00700 1 -- 172.21.15.18:0/708898873 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 6757+0+0 (secure 0 0 0) 0x7f8b7408fbe0 con 0x7f8b8c0fe830 2024-04-12T21:41:08.597 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:08.597+0000 7f8b89e00700 1 --2- 172.21.15.18:0/708898873 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f8b7006ec00 0x7f8b700710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:41:08.597 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:08.598+0000 7f8b89e00700 1 --2- 172.21.15.18:0/708898873 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f8b7006ec00 0x7f8b700710b0 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f8b7c004990 tx=0x7f8b7c009ee0 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:41:08.602 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:08.602+0000 7f8b82a00700 1 -- 172.21.15.18:0/708898873 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f8b74058860 con 0x7f8b8c0fe830 2024-04-12T21:41:08.849 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:08.848+0000 7f8b90c00700 1 -- 172.21.15.18:0/708898873 --> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f8b8c04bed0 con 0x7f8b7006ec00 2024-04-12T21:41:08.850 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:08.850+0000 7f8b82a00700 1 -- 172.21.15.18:0/708898873 <== mgr.34102 v2:172.21.15.18:6800/3812891919 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f8b8c04bed0 con 0x7f8b7006ec00 2024-04-12T21:41:08.853 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:08.853+0000 7f8b90c00700 1 -- 172.21.15.18:0/708898873 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f8b7006ec00 msgr2=0x7f8b700710b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:41:08.853 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:08.853+0000 7f8b90c00700 1 --2- 172.21.15.18:0/708898873 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f8b7006ec00 0x7f8b700710b0 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f8b7c004990 tx=0x7f8b7c009ee0 comp rx=0 tx=0).stop 2024-04-12T21:41:08.853 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:08.853+0000 7f8b90c00700 1 -- 172.21.15.18:0/708898873 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8b8c0fe830 msgr2=0x7f8b8c0fc290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:41:08.853 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:08.853+0000 7f8b90c00700 1 --2- 172.21.15.18:0/708898873 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8b8c0fe830 0x7f8b8c0fc290 secure :-1 s=READY pgs=1045 cs=0 l=1 rev1=1 crypto rx=0x7f8b74009360 tx=0x7f8b74009e30 comp rx=0 tx=0).stop 2024-04-12T21:41:08.854 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:08.853+0000 7f8b90c00700 1 -- 172.21.15.18:0/708898873 shutdown_connections 2024-04-12T21:41:08.854 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:08.853+0000 7f8b90c00700 1 --2- 172.21.15.18:0/708898873 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f8b7006ec00 0x7f8b700710b0 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:41:08.854 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:08.853+0000 7f8b90c00700 1 --2- 172.21.15.18:0/708898873 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8b8c0fde90 0x7f8b8c0fbd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:41:08.854 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:08.853+0000 7f8b90c00700 1 --2- 172.21.15.18:0/708898873 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8b8c0fe830 0x7f8b8c0fc290 unknown :-1 s=CLOSED pgs=1045 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:41:08.854 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:08.853+0000 7f8b90c00700 1 -- 172.21.15.18:0/708898873 >> 172.21.15.18:0/708898873 conn(0x7f8b8c072bc0 msgr2=0x7f8b8c1037b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:41:08.854 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:08.853+0000 7f8b90c00700 1 -- 172.21.15.18:0/708898873 shutdown_connections 2024-04-12T21:41:08.854 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:08.853+0000 7f8b90c00700 1 -- 172.21.15.18:0/708898873 wait complete. 2024-04-12T21:41:08.871 INFO:teuthology.orchestra.run.smithi018.stdout:true 2024-04-12T21:41:09.004 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:09.003+0000 7f2434400700 1 -- 172.21.15.18:0/3446972658 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f242c109430 msgr2=0x7f242c109800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:41:09.004 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:09.003+0000 7f2434400700 1 --2- 172.21.15.18:0/3446972658 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f242c109430 0x7f242c109800 secure :-1 s=READY pgs=1046 cs=0 l=1 rev1=1 crypto rx=0x7f2420004940 tx=0x7f2420009e30 comp rx=0 tx=0).stop 2024-04-12T21:41:09.004 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:09.003+0000 7f2434400700 1 -- 172.21.15.18:0/3446972658 shutdown_connections 2024-04-12T21:41:09.004 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:09.003+0000 7f2434400700 1 --2- 172.21.15.18:0/3446972658 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f242c109dd0 0x7f242c103440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:41:09.004 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:09.003+0000 7f2434400700 1 --2- 172.21.15.18:0/3446972658 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f242c109430 0x7f242c109800 unknown :-1 s=CLOSED pgs=1046 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:41:09.004 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:09.003+0000 7f2434400700 1 -- 172.21.15.18:0/3446972658 >> 172.21.15.18:0/3446972658 conn(0x7f242c072c00 msgr2=0x7f242c075010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:41:09.004 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:09.004+0000 7f2434400700 1 -- 172.21.15.18:0/3446972658 shutdown_connections 2024-04-12T21:41:09.004 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:09.004+0000 7f2434400700 1 -- 172.21.15.18:0/3446972658 wait complete. 2024-04-12T21:41:09.005 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:09.004+0000 7f2434400700 1 Processor -- start 2024-04-12T21:41:09.005 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:09.004+0000 7f2434400700 1 -- start start 2024-04-12T21:41:09.005 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:09.005+0000 7f2434400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f242c109430 0x7f242c119f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:41:09.006 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:09.005+0000 7f2434400700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f242c109dd0 0x7f242c11a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:41:09.006 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:09.005+0000 7f2434400700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f242c11ab60 con 0x7f242c109430 2024-04-12T21:41:09.006 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:09.005+0000 7f2434400700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f242c1140b0 con 0x7f242c109dd0 2024-04-12T21:41:09.006 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:09.005+0000 7f2431a00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f242c109430 0x7f242c119f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:41:09.006 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:09.005+0000 7f2431a00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f242c109430 0x7f242c119f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:40072/0 (socket says 172.21.15.18:40072) 2024-04-12T21:41:09.006 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:09.005+0000 7f2431a00700 1 -- 172.21.15.18:0/3106192129 learned_addr learned my addr 172.21.15.18:0/3106192129 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:41:09.006 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:09.005+0000 7f2431a00700 1 -- 172.21.15.18:0/3106192129 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f242c109dd0 msgr2=0x7f242c11a480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:41:09.006 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:09.005+0000 7f2431a00700 1 --2- 172.21.15.18:0/3106192129 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f242c109dd0 0x7f242c11a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:41:09.006 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:09.005+0000 7f2431a00700 1 -- 172.21.15.18:0/3106192129 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2420005040 con 0x7f242c109430 2024-04-12T21:41:09.006 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:09.006+0000 7f2431a00700 1 --2- 172.21.15.18:0/3106192129 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f242c109430 0x7f242c119f40 secure :-1 s=READY pgs=1047 cs=0 l=1 rev1=1 crypto rx=0x7f2420009150 tx=0x7f2420009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:41:09.007 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:09.006+0000 7f2429600700 1 -- 172.21.15.18:0/3106192129 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f242000e040 con 0x7f242c109430 2024-04-12T21:41:09.007 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:09.006+0000 7f2429600700 1 -- 172.21.15.18:0/3106192129 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f2420013420 con 0x7f242c109430 2024-04-12T21:41:09.007 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:09.006+0000 7f2434400700 1 -- 172.21.15.18:0/3106192129 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f242c1142d0 con 0x7f242c109430 2024-04-12T21:41:09.007 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:09.006+0000 7f2429600700 1 -- 172.21.15.18:0/3106192129 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f242001c630 con 0x7f242c109430 2024-04-12T21:41:09.007 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:09.006+0000 7f2434400700 1 -- 172.21.15.18:0/3106192129 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f242c114890 con 0x7f242c109430 2024-04-12T21:41:09.007 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:09.007+0000 7f2434400700 1 -- 172.21.15.18:0/3106192129 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f242c104b80 con 0x7f242c109430 2024-04-12T21:41:09.008 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:09.008+0000 7f2429600700 1 -- 172.21.15.18:0/3106192129 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7f2420013590 con 0x7f242c109430 2024-04-12T21:41:09.009 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:09.009+0000 7f2429600700 1 --2- 172.21.15.18:0/3106192129 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f241406ec00 0x7f24140710b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:41:09.009 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:09.009+0000 7f2429600700 1 -- 172.21.15.18:0/3106192129 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 6757+0+0 (secure 0 0 0) 0x7f242008fce0 con 0x7f242c109430 2024-04-12T21:41:09.009 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:09.009+0000 7f2431000700 1 --2- 172.21.15.18:0/3106192129 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f241406ec00 0x7f24140710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:41:09.010 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:09.010+0000 7f2431000700 1 --2- 172.21.15.18:0/3106192129 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f241406ec00 0x7f24140710b0 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f2418006fd0 tx=0x7f2418006db0 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:41:09.014 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:09.014+0000 7f2429600700 1 -- 172.21.15.18:0/3106192129 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f242005c490 con 0x7f242c109430 2024-04-12T21:41:09.267 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:09.267+0000 7f2434400700 1 -- 172.21.15.18:0/3106192129 --> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f242c110640 con 0x7f241406ec00 2024-04-12T21:41:09.269 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:09.269+0000 7f2429600700 1 -- 172.21.15.18:0/3106192129 <== mgr.34102 v2:172.21.15.18:6800/3812891919 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f2418009dd0 con 0x7f241406ec00 2024-04-12T21:41:09.273 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:09.272+0000 7f2434400700 1 -- 172.21.15.18:0/3106192129 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f241406ec00 msgr2=0x7f24140710b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:41:09.273 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:09.272+0000 7f2434400700 1 --2- 172.21.15.18:0/3106192129 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f241406ec00 0x7f24140710b0 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f2418006fd0 tx=0x7f2418006db0 comp rx=0 tx=0).stop 2024-04-12T21:41:09.273 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:09.272+0000 7f2434400700 1 -- 172.21.15.18:0/3106192129 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f242c109430 msgr2=0x7f242c119f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:41:09.273 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:09.272+0000 7f2434400700 1 --2- 172.21.15.18:0/3106192129 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f242c109430 0x7f242c119f40 secure :-1 s=READY pgs=1047 cs=0 l=1 rev1=1 crypto rx=0x7f2420009150 tx=0x7f2420009e30 comp rx=0 tx=0).stop 2024-04-12T21:41:09.273 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:09.273+0000 7f2434400700 1 -- 172.21.15.18:0/3106192129 shutdown_connections 2024-04-12T21:41:09.273 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:09.273+0000 7f2434400700 1 --2- 172.21.15.18:0/3106192129 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f241406ec00 0x7f24140710b0 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:41:09.273 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:09.273+0000 7f2434400700 1 --2- 172.21.15.18:0/3106192129 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f242c109430 0x7f242c119f40 unknown :-1 s=CLOSED pgs=1047 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:41:09.273 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:09.273+0000 7f2434400700 1 --2- 172.21.15.18:0/3106192129 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f242c109dd0 0x7f242c11a480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:41:09.273 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:09.273+0000 7f2434400700 1 -- 172.21.15.18:0/3106192129 >> 172.21.15.18:0/3106192129 conn(0x7f242c072c00 msgr2=0x7f242c074c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:41:09.273 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:09.273+0000 7f2434400700 1 -- 172.21.15.18:0/3106192129 shutdown_connections 2024-04-12T21:41:09.273 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:09.273+0000 7f2434400700 1 -- 172.21.15.18:0/3106192129 wait complete. 2024-04-12T21:41:09.428 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:09.428+0000 7f6d16600700 1 -- 172.21.15.18:0/1438744921 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6d10070300 msgr2=0x7f6d100706d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:41:09.429 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:09.428+0000 7f6d16600700 1 --2- 172.21.15.18:0/1438744921 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6d10070300 0x7f6d100706d0 secure :-1 s=READY pgs=1048 cs=0 l=1 rev1=1 crypto rx=0x7f6d00004970 tx=0x7f6d00009e30 comp rx=0 tx=0).stop 2024-04-12T21:41:09.429 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:09.428+0000 7f6d16600700 1 -- 172.21.15.18:0/1438744921 shutdown_connections 2024-04-12T21:41:09.429 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:09.428+0000 7f6d16600700 1 --2- 172.21.15.18:0/1438744921 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6d10070c10 0x7f6d10109180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:41:09.429 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:09.428+0000 7f6d16600700 1 --2- 172.21.15.18:0/1438744921 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6d10070300 0x7f6d100706d0 unknown :-1 s=CLOSED pgs=1048 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:41:09.429 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:09.428+0000 7f6d16600700 1 -- 172.21.15.18:0/1438744921 >> 172.21.15.18:0/1438744921 conn(0x7f6d100f8980 msgr2=0x7f6d100fad90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:41:09.429 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:09.428+0000 7f6d16600700 1 -- 172.21.15.18:0/1438744921 shutdown_connections 2024-04-12T21:41:09.429 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:09.429+0000 7f6d16600700 1 -- 172.21.15.18:0/1438744921 wait complete. 2024-04-12T21:41:09.430 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:09.429+0000 7f6d16600700 1 Processor -- start 2024-04-12T21:41:09.430 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:09.430+0000 7f6d16600700 1 -- start start 2024-04-12T21:41:09.430 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:09.430+0000 7f6d16600700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6d10070c10 0x7f6d10115ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:41:09.430 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:09.430+0000 7f6d16600700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6d10116430 0x7f6d1010fe50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:41:09.430 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:09.430+0000 7f6d16600700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d101169b0 con 0x7f6d10116430 2024-04-12T21:41:09.430 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:09.430+0000 7f6d16600700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d10116b20 con 0x7f6d10070c10 2024-04-12T21:41:09.431 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:09.430+0000 7f6d0ea00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6d10116430 0x7f6d1010fe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:41:09.432 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:09.430+0000 7f6d0ea00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6d10116430 0x7f6d1010fe50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:40090/0 (socket says 172.21.15.18:40090) 2024-04-12T21:41:09.432 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:09.430+0000 7f6d0ea00700 1 -- 172.21.15.18:0/21900301 learned_addr learned my addr 172.21.15.18:0/21900301 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:41:09.432 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:09.431+0000 7f6d0f400700 1 --2- 172.21.15.18:0/21900301 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6d10070c10 0x7f6d10115ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:41:09.432 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:09.431+0000 7f6d0ea00700 1 -- 172.21.15.18:0/21900301 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6d10070c10 msgr2=0x7f6d10115ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:41:09.432 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:09.431+0000 7f6d0ea00700 1 --2- 172.21.15.18:0/21900301 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6d10070c10 0x7f6d10115ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:41:09.432 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:09.431+0000 7f6d0ea00700 1 -- 172.21.15.18:0/21900301 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d00005040 con 0x7f6d10116430 2024-04-12T21:41:09.432 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:09.431+0000 7f6d0ea00700 1 --2- 172.21.15.18:0/21900301 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6d10116430 0x7f6d1010fe50 secure :-1 s=READY pgs=1049 cs=0 l=1 rev1=1 crypto rx=0x7f6cfc00c8c0 tx=0x7f6cfc0044c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:41:09.432 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:09.431+0000 7f6d07400700 1 -- 172.21.15.18:0/21900301 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6cfc00a980 con 0x7f6d10116430 2024-04-12T21:41:09.432 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:09.431+0000 7f6d07400700 1 -- 172.21.15.18:0/21900301 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f6cfc00ce60 con 0x7f6d10116430 2024-04-12T21:41:09.432 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:09.431+0000 7f6d07400700 1 -- 172.21.15.18:0/21900301 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6cfc0115e0 con 0x7f6d10116430 2024-04-12T21:41:09.432 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:09.431+0000 7f6d16600700 1 -- 172.21.15.18:0/21900301 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6d10110450 con 0x7f6d10116430 2024-04-12T21:41:09.432 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:09.432+0000 7f6d16600700 1 -- 172.21.15.18:0/21900301 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6d10110990 con 0x7f6d10116430 2024-04-12T21:41:09.433 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:09.433+0000 7f6d16600700 1 -- 172.21.15.18:0/21900301 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6d10110cc0 con 0x7f6d10116430 2024-04-12T21:41:09.433 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:09.434+0000 7f6d07400700 1 -- 172.21.15.18:0/21900301 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7f6cfc012400 con 0x7f6d10116430 2024-04-12T21:41:09.435 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:09.435+0000 7f6d07400700 1 --2- 172.21.15.18:0/21900301 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f6cf406ec00 0x7f6cf40710b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:41:09.435 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:09.435+0000 7f6d07400700 1 -- 172.21.15.18:0/21900301 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 6757+0+0 (secure 0 0 0) 0x7f6cfc08ea90 con 0x7f6d10116430 2024-04-12T21:41:09.435 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:09.435+0000 7f6d0f400700 1 --2- 172.21.15.18:0/21900301 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f6cf406ec00 0x7f6cf40710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:41:09.435 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:09.436+0000 7f6d0f400700 1 --2- 172.21.15.18:0/21900301 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f6cf406ec00 0x7f6cf40710b0 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f6d0000b370 tx=0x7f6d00009e30 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:41:09.439 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:09.439+0000 7f6d07400700 1 -- 172.21.15.18:0/21900301 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f6d10110cc0 con 0x7f6d10116430 2024-04-12T21:41:09.673 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:09.672+0000 7f6d16600700 1 -- 172.21.15.18:0/21900301 --> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f6d10110cc0 con 0x7f6cf406ec00 2024-04-12T21:41:09.702 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:09.702+0000 7f6d07400700 1 -- 172.21.15.18:0/21900301 <== mgr.34102 v2:172.21.15.18:6800/3812891919 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7f6d10110cc0 con 0x7f6cf406ec00 2024-04-12T21:41:09.702 INFO:teuthology.orchestra.run.smithi018.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-12T21:41:09.703 INFO:teuthology.orchestra.run.smithi018.stdout:alertmanager.smithi018 smithi018 *:9093,9094 running (41m) 2m ago 56m 14.8M - 0.25.0 c8568f914cd2 c332090068e8 2024-04-12T21:41:09.703 INFO:teuthology.orchestra.run.smithi018.stdout:crash.smithi018 smithi018 running (38m) 2m ago 56m 6665k - 19.0.0-2879-g324baf8d 8717720dc5e5 b7f80d1f7982 2024-04-12T21:41:09.703 INFO:teuthology.orchestra.run.smithi018.stdout:crash.smithi186 smithi186 running (37m) 2m ago 55m 6660k - 19.0.0-2879-g324baf8d 8717720dc5e5 a4f55c81b623 2024-04-12T21:41:09.703 INFO:teuthology.orchestra.run.smithi018.stdout:grafana.smithi018 smithi018 *:3000 running (41m) 2m ago 55m 73.8M - 9.4.12 2bacad6d85d8 b08e3bf176dd 2024-04-12T21:41:09.703 INFO:teuthology.orchestra.run.smithi018.stdout:mds.cephfs.smithi018.acqbsc smithi018 running (33m) 2m ago 49m 21.9M - 19.0.0-2879-g324baf8d 8717720dc5e5 021f99ab24ae 2024-04-12T21:41:09.703 INFO:teuthology.orchestra.run.smithi018.stdout:mds.cephfs.smithi018.szjluo smithi018 running (33m) 2m ago 48m 21.7M - 19.0.0-2879-g324baf8d 8717720dc5e5 0003117479d4 2024-04-12T21:41:09.703 INFO:teuthology.orchestra.run.smithi018.stdout:mds.cephfs.smithi186.jdllyn smithi186 running (18m) 2m ago 49m 16.5M - 19.0.0-2879-g324baf8d 8717720dc5e5 21b1f46eef23 2024-04-12T21:41:09.703 INFO:teuthology.orchestra.run.smithi018.stdout:mds.cephfs.smithi186.wjnafs smithi186 running (2m) 2m ago 48m 11.0M - 19.0.0-2879-g324baf8d 8717720dc5e5 3cf1ef4148e2 2024-04-12T21:41:09.703 INFO:teuthology.orchestra.run.smithi018.stdout:mgr.smithi018.xjzwga smithi018 *:8443,9283,8765 running (44m) 2m ago 57m 517M - 19.0.0-2879-g324baf8d 8717720dc5e5 56088305eab7 2024-04-12T21:41:09.703 INFO:teuthology.orchestra.run.smithi018.stdout:mgr.smithi186.kqcfxk smithi186 *:8443,9283,8765 running (43m) 2m ago 54m 460M - 19.0.0-2879-g324baf8d 8717720dc5e5 09cd766e5e0d 2024-04-12T21:41:09.703 INFO:teuthology.orchestra.run.smithi018.stdout:mon.smithi018 smithi018 running (39m) 2m ago 58m 88.2M 2048M 19.0.0-2879-g324baf8d 8717720dc5e5 df3e201dfed6 2024-04-12T21:41:09.703 INFO:teuthology.orchestra.run.smithi018.stdout:mon.smithi186 smithi186 running (39m) 2m ago 54m 53.6M 2048M 19.0.0-2879-g324baf8d 8717720dc5e5 c77498da9b2b 2024-04-12T21:41:09.703 INFO:teuthology.orchestra.run.smithi018.stdout:node-exporter.smithi018 smithi018 *:9100 running (42m) 2m ago 56m 8468k - 1.5.0 0da6a335fe13 326c3d9c6847 2024-04-12T21:41:09.703 INFO:teuthology.orchestra.run.smithi018.stdout:node-exporter.smithi186 smithi186 *:9100 running (42m) 2m ago 55m 8166k - 1.5.0 0da6a335fe13 ec2112baeecf 2024-04-12T21:41:09.704 INFO:teuthology.orchestra.run.smithi018.stdout:osd.0 smithi018 running (37m) 2m ago 53m 123M 1559M 19.0.0-2879-g324baf8d 8717720dc5e5 60b1273ca047 2024-04-12T21:41:09.704 INFO:teuthology.orchestra.run.smithi018.stdout:osd.1 smithi018 running (36m) 2m ago 53m 95.4M 1559M 19.0.0-2879-g324baf8d 8717720dc5e5 170ce1e5fb8d 2024-04-12T21:41:09.704 INFO:teuthology.orchestra.run.smithi018.stdout:osd.2 smithi018 running (36m) 2m ago 52m 94.6M 1559M 19.0.0-2879-g324baf8d 8717720dc5e5 c2cbc6aa2839 2024-04-12T21:41:09.704 INFO:teuthology.orchestra.run.smithi018.stdout:osd.3 smithi186 running (35m) 2m ago 52m 101M 2581M 19.0.0-2879-g324baf8d 8717720dc5e5 0138e8262e4b 2024-04-12T21:41:09.704 INFO:teuthology.orchestra.run.smithi018.stdout:osd.4 smithi186 running (35m) 2m ago 51m 91.5M 2581M 19.0.0-2879-g324baf8d 8717720dc5e5 b20cb63b25c9 2024-04-12T21:41:09.704 INFO:teuthology.orchestra.run.smithi018.stdout:osd.5 smithi186 running (34m) 2m ago 50m 92.2M 2581M 19.0.0-2879-g324baf8d 8717720dc5e5 2513bea2610e 2024-04-12T21:41:09.704 INFO:teuthology.orchestra.run.smithi018.stdout:prometheus.smithi018 smithi018 *:9095 running (42m) 2m ago 55m 69.6M - 2.43.0 a07b618ecd1d f58841127e21 2024-04-12T21:41:09.706 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:09.705+0000 7f6d16600700 1 -- 172.21.15.18:0/21900301 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f6cf406ec00 msgr2=0x7f6cf40710b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:41:09.706 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:09.705+0000 7f6d16600700 1 --2- 172.21.15.18:0/21900301 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f6cf406ec00 0x7f6cf40710b0 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f6d0000b370 tx=0x7f6d00009e30 comp rx=0 tx=0).stop 2024-04-12T21:41:09.706 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:09.705+0000 7f6d16600700 1 -- 172.21.15.18:0/21900301 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6d10116430 msgr2=0x7f6d1010fe50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:41:09.706 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:09.705+0000 7f6d16600700 1 --2- 172.21.15.18:0/21900301 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6d10116430 0x7f6d1010fe50 secure :-1 s=READY pgs=1049 cs=0 l=1 rev1=1 crypto rx=0x7f6cfc00c8c0 tx=0x7f6cfc0044c0 comp rx=0 tx=0).stop 2024-04-12T21:41:09.706 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:09.706+0000 7f6d16600700 1 -- 172.21.15.18:0/21900301 shutdown_connections 2024-04-12T21:41:09.706 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:09.706+0000 7f6d16600700 1 --2- 172.21.15.18:0/21900301 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f6cf406ec00 0x7f6cf40710b0 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:41:09.706 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:09.706+0000 7f6d16600700 1 --2- 172.21.15.18:0/21900301 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6d10070c10 0x7f6d10115ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:41:09.706 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:09.706+0000 7f6d16600700 1 --2- 172.21.15.18:0/21900301 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6d10116430 0x7f6d1010fe50 unknown :-1 s=CLOSED pgs=1049 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:41:09.707 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:09.706+0000 7f6d16600700 1 -- 172.21.15.18:0/21900301 >> 172.21.15.18:0/21900301 conn(0x7f6d100f8980 msgr2=0x7f6d101034a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:41:09.707 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:09.706+0000 7f6d16600700 1 -- 172.21.15.18:0/21900301 shutdown_connections 2024-04-12T21:41:09.707 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:09.706+0000 7f6d16600700 1 -- 172.21.15.18:0/21900301 wait complete. 2024-04-12T21:41:09.860 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:09.860+0000 7feb16a00700 1 -- 172.21.15.18:0/2374812139 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7feb10070c70 msgr2=0x7feb10109140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:41:09.860 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:09.860+0000 7feb16a00700 1 --2- 172.21.15.18:0/2374812139 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7feb10070c70 0x7feb10109140 secure :-1 s=READY pgs=1050 cs=0 l=1 rev1=1 crypto rx=0x7feaf8009d10 tx=0x7feaf8004d10 comp rx=0 tx=0).stop 2024-04-12T21:41:09.861 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:09.860+0000 7feb16a00700 1 -- 172.21.15.18:0/2374812139 shutdown_connections 2024-04-12T21:41:09.861 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:09.860+0000 7feb16a00700 1 --2- 172.21.15.18:0/2374812139 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7feb10070c70 0x7feb10109140 unknown :-1 s=CLOSED pgs=1050 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:41:09.861 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:09.860+0000 7feb16a00700 1 --2- 172.21.15.18:0/2374812139 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7feb10070360 0x7feb10070730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:41:09.861 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:09.860+0000 7feb16a00700 1 -- 172.21.15.18:0/2374812139 >> 172.21.15.18:0/2374812139 conn(0x7feb100f8a20 msgr2=0x7feb100fae30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:41:09.861 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:09.860+0000 7feb16a00700 1 -- 172.21.15.18:0/2374812139 shutdown_connections 2024-04-12T21:41:09.861 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:09.860+0000 7feb16a00700 1 -- 172.21.15.18:0/2374812139 wait complete. 2024-04-12T21:41:09.861 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:09.861+0000 7feb16a00700 1 Processor -- start 2024-04-12T21:41:09.861 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:09.862+0000 7feb16a00700 1 -- start start 2024-04-12T21:41:09.862 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:09.862+0000 7feb16a00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7feb10070360 0x7feb10115d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:41:09.862 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:09.862+0000 7feb16a00700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7feb10070c70 0x7feb101162c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:41:09.862 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:09.862+0000 7feb16a00700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb10116910 con 0x7feb10070360 2024-04-12T21:41:09.862 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:09.862+0000 7feb16a00700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb1010fef0 con 0x7feb10070c70 2024-04-12T21:41:09.863 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:09.862+0000 7feb0fe00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7feb10070360 0x7feb10115d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:41:09.863 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:09.863+0000 7feb0fe00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7feb10070360 0x7feb10115d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:40096/0 (socket says 172.21.15.18:40096) 2024-04-12T21:41:09.863 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:09.863+0000 7feb0fe00700 1 -- 172.21.15.18:0/279414448 learned_addr learned my addr 172.21.15.18:0/279414448 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:41:09.863 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:09.863+0000 7feb07400700 1 --2- 172.21.15.18:0/279414448 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7feb10070c70 0x7feb101162c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:41:09.863 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:09.863+0000 7feb0fe00700 1 -- 172.21.15.18:0/279414448 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7feb10070c70 msgr2=0x7feb101162c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:41:09.864 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:09.863+0000 7feb0fe00700 1 --2- 172.21.15.18:0/279414448 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7feb10070c70 0x7feb101162c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:41:09.864 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:09.863+0000 7feb0fe00700 1 -- 172.21.15.18:0/279414448 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feaf8009940 con 0x7feb10070360 2024-04-12T21:41:09.864 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:09.863+0000 7feb0fe00700 1 --2- 172.21.15.18:0/279414448 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7feb10070360 0x7feb10115d80 secure :-1 s=READY pgs=1051 cs=0 l=1 rev1=1 crypto rx=0x7feb00002a50 tx=0x7feb000046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:41:09.864 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:09.864+0000 7feb0cc00700 1 -- 172.21.15.18:0/279414448 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7feb00007cc0 con 0x7feb10070360 2024-04-12T21:41:09.864 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:09.864+0000 7feb0cc00700 1 -- 172.21.15.18:0/279414448 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7feb0000f040 con 0x7feb10070360 2024-04-12T21:41:09.865 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:09.864+0000 7feb16a00700 1 -- 172.21.15.18:0/279414448 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feb101101d0 con 0x7feb10070360 2024-04-12T21:41:09.865 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:09.864+0000 7feb0cc00700 1 -- 172.21.15.18:0/279414448 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7feb00009640 con 0x7feb10070360 2024-04-12T21:41:09.865 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:09.864+0000 7feb16a00700 1 -- 172.21.15.18:0/279414448 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feb10110710 con 0x7feb10070360 2024-04-12T21:41:09.865 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:09.865+0000 7feb16a00700 1 -- 172.21.15.18:0/279414448 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feb10071aa0 con 0x7feb10070360 2024-04-12T21:41:09.866 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:09.867+0000 7feb0cc00700 1 -- 172.21.15.18:0/279414448 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7feb000097a0 con 0x7feb10070360 2024-04-12T21:41:09.867 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:09.867+0000 7feb0cc00700 1 --2- 172.21.15.18:0/279414448 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7feaf406ec00 0x7feaf40710b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:41:09.867 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:09.868+0000 7feb0cc00700 1 -- 172.21.15.18:0/279414448 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 6757+0+0 (secure 0 0 0) 0x7feb0008e760 con 0x7feb10070360 2024-04-12T21:41:09.868 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:09.868+0000 7feb07400700 1 --2- 172.21.15.18:0/279414448 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7feaf406ec00 0x7feaf40710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:41:09.868 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:09.869+0000 7feb07400700 1 --2- 172.21.15.18:0/279414448 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7feaf406ec00 0x7feaf40710b0 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7feb10111620 tx=0x7feaf8004d10 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:41:09.873 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:09.873+0000 7feb0cc00700 1 -- 172.21.15.18:0/279414448 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7feb0005af10 con 0x7feb10070360 2024-04-12T21:41:10.104 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:41:09 smithi018 ceph-mon[112590]: pgmap v1244: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:41:10.104 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:41:09 smithi018 ceph-mon[112590]: from='client.36160 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:41:10.202 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:10.202+0000 7feb16a00700 1 -- 172.21.15.18:0/279414448 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7feb101113d0 con 0x7feb10070360 2024-04-12T21:41:10.203 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:10.203+0000 7feb0cc00700 1 -- 172.21.15.18:0/279414448 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+623 (secure 0 0 0) 0x7feb0005aaa0 con 0x7feb10070360 2024-04-12T21:41:10.204 INFO:teuthology.orchestra.run.smithi018.stdout:{ 2024-04-12T21:41:10.204 INFO:teuthology.orchestra.run.smithi018.stdout: "mon": { 2024-04-12T21:41:10.204 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 19.0.0-2879-g324baf8d (324baf8ddb91dea46cac50cb535c306115791355) squid (dev)": 2 2024-04-12T21:41:10.204 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-12T21:41:10.204 INFO:teuthology.orchestra.run.smithi018.stdout: "mgr": { 2024-04-12T21:41:10.204 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 19.0.0-2879-g324baf8d (324baf8ddb91dea46cac50cb535c306115791355) squid (dev)": 2 2024-04-12T21:41:10.204 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-12T21:41:10.204 INFO:teuthology.orchestra.run.smithi018.stdout: "osd": { 2024-04-12T21:41:10.204 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 19.0.0-2879-g324baf8d (324baf8ddb91dea46cac50cb535c306115791355) squid (dev)": 6 2024-04-12T21:41:10.204 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-12T21:41:10.204 INFO:teuthology.orchestra.run.smithi018.stdout: "mds": { 2024-04-12T21:41:10.205 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 19.0.0-2879-g324baf8d (324baf8ddb91dea46cac50cb535c306115791355) squid (dev)": 4 2024-04-12T21:41:10.205 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-12T21:41:10.205 INFO:teuthology.orchestra.run.smithi018.stdout: "overall": { 2024-04-12T21:41:10.205 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 19.0.0-2879-g324baf8d (324baf8ddb91dea46cac50cb535c306115791355) squid (dev)": 14 2024-04-12T21:41:10.205 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-04-12T21:41:10.205 INFO:teuthology.orchestra.run.smithi018.stdout:} 2024-04-12T21:41:10.206 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:10.206+0000 7feb16a00700 1 -- 172.21.15.18:0/279414448 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7feaf406ec00 msgr2=0x7feaf40710b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:41:10.207 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:10.206+0000 7feb16a00700 1 --2- 172.21.15.18:0/279414448 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7feaf406ec00 0x7feaf40710b0 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7feb10111620 tx=0x7feaf8004d10 comp rx=0 tx=0).stop 2024-04-12T21:41:10.207 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:10.206+0000 7feb16a00700 1 -- 172.21.15.18:0/279414448 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7feb10070360 msgr2=0x7feb10115d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:41:10.207 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:10.206+0000 7feb16a00700 1 --2- 172.21.15.18:0/279414448 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7feb10070360 0x7feb10115d80 secure :-1 s=READY pgs=1051 cs=0 l=1 rev1=1 crypto rx=0x7feb00002a50 tx=0x7feb000046b0 comp rx=0 tx=0).stop 2024-04-12T21:41:10.207 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:10.207+0000 7feb16a00700 1 -- 172.21.15.18:0/279414448 shutdown_connections 2024-04-12T21:41:10.207 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:10.207+0000 7feb16a00700 1 --2- 172.21.15.18:0/279414448 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7feaf406ec00 0x7feaf40710b0 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:41:10.207 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:10.207+0000 7feb16a00700 1 --2- 172.21.15.18:0/279414448 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7feb10070360 0x7feb10115d80 unknown :-1 s=CLOSED pgs=1051 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:41:10.207 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:10.207+0000 7feb16a00700 1 --2- 172.21.15.18:0/279414448 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7feb10070c70 0x7feb101162c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:41:10.207 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:10.207+0000 7feb16a00700 1 -- 172.21.15.18:0/279414448 >> 172.21.15.18:0/279414448 conn(0x7feb100f8a20 msgr2=0x7feb100f9f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:41:10.207 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:10.207+0000 7feb16a00700 1 -- 172.21.15.18:0/279414448 shutdown_connections 2024-04-12T21:41:10.208 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:10.207+0000 7feb16a00700 1 -- 172.21.15.18:0/279414448 wait complete. 2024-04-12T21:41:10.283 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:41:09 smithi186 ceph-mon[86379]: pgmap v1244: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:41:10.283 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:41:09 smithi186 ceph-mon[86379]: from='client.36160 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:41:10.363 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:10.362+0000 7f7fc1600700 1 -- 172.21.15.18:0/2411343478 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7fbc072e00 msgr2=0x7f7fbc10d5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:41:10.363 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:10.362+0000 7f7fc1600700 1 --2- 172.21.15.18:0/2411343478 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7fbc072e00 0x7f7fbc10d5a0 secure :-1 s=READY pgs=1052 cs=0 l=1 rev1=1 crypto rx=0x7f7fa8004940 tx=0x7f7fa8009e30 comp rx=0 tx=0).stop 2024-04-12T21:41:10.363 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:10.363+0000 7f7fc1600700 1 -- 172.21.15.18:0/2411343478 shutdown_connections 2024-04-12T21:41:10.363 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:10.363+0000 7f7fc1600700 1 --2- 172.21.15.18:0/2411343478 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7fbc072e00 0x7f7fbc10d5a0 unknown :-1 s=CLOSED pgs=1052 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:41:10.363 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:10.363+0000 7f7fc1600700 1 --2- 172.21.15.18:0/2411343478 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7fbc0724f0 0x7f7fbc0728c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:41:10.363 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:10.363+0000 7f7fc1600700 1 -- 172.21.15.18:0/2411343478 >> 172.21.15.18:0/2411343478 conn(0x7f7fbc0fab10 msgr2=0x7f7fbc0fcef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:41:10.363 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:10.363+0000 7f7fc1600700 1 -- 172.21.15.18:0/2411343478 shutdown_connections 2024-04-12T21:41:10.363 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:10.363+0000 7f7fc1600700 1 -- 172.21.15.18:0/2411343478 wait complete. 2024-04-12T21:41:10.364 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:10.364+0000 7f7fc1600700 1 Processor -- start 2024-04-12T21:41:10.364 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:10.364+0000 7f7fc1600700 1 -- start start 2024-04-12T21:41:10.365 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:10.364+0000 7f7fc1600700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7fbc0724f0 0x7f7fbc119f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:41:10.365 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:10.364+0000 7f7fc1600700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7fbc072e00 0x7f7fbc11a490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:41:10.365 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:10.364+0000 7f7fc1600700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7fbc11aae0 con 0x7f7fbc072e00 2024-04-12T21:41:10.365 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:10.364+0000 7f7fc1600700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7fbc113eb0 con 0x7f7fbc0724f0 2024-04-12T21:41:10.365 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:10.364+0000 7f7fbb400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7fbc072e00 0x7f7fbc11a490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:41:10.365 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:10.365+0000 7f7fbb400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7fbc072e00 0x7f7fbc11a490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:40108/0 (socket says 172.21.15.18:40108) 2024-04-12T21:41:10.365 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:10.365+0000 7f7fbb400700 1 -- 172.21.15.18:0/607742566 learned_addr learned my addr 172.21.15.18:0/607742566 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:41:10.365 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:10.365+0000 7f7fbb400700 1 -- 172.21.15.18:0/607742566 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7fbc0724f0 msgr2=0x7f7fbc119f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:41:10.366 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:10.365+0000 7f7fbb400700 1 --2- 172.21.15.18:0/607742566 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7fbc0724f0 0x7f7fbc119f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:41:10.366 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:10.365+0000 7f7fbb400700 1 -- 172.21.15.18:0/607742566 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7fa8005040 con 0x7f7fbc072e00 2024-04-12T21:41:10.366 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:10.365+0000 7f7fbb400700 1 --2- 172.21.15.18:0/607742566 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7fbc072e00 0x7f7fbc11a490 secure :-1 s=READY pgs=1053 cs=0 l=1 rev1=1 crypto rx=0x7f7fa8009180 tx=0x7f7fa8009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:41:10.366 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:10.365+0000 7f7fb3e00700 1 -- 172.21.15.18:0/607742566 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7fa800e040 con 0x7f7fbc072e00 2024-04-12T21:41:10.366 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:10.366+0000 7f7fb3e00700 1 -- 172.21.15.18:0/607742566 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f7fa8013420 con 0x7f7fbc072e00 2024-04-12T21:41:10.367 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:10.366+0000 7f7fb3e00700 1 -- 172.21.15.18:0/607742566 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7fa801c630 con 0x7f7fbc072e00 2024-04-12T21:41:10.367 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:10.366+0000 7f7fc1600700 1 -- 172.21.15.18:0/607742566 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7fbc114130 con 0x7f7fbc072e00 2024-04-12T21:41:10.367 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:10.366+0000 7f7fc1600700 1 -- 172.21.15.18:0/607742566 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7fbc1146f0 con 0x7f7fbc072e00 2024-04-12T21:41:10.367 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:10.367+0000 7f7fc1600700 1 -- 172.21.15.18:0/607742566 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7fbc073c30 con 0x7f7fbc072e00 2024-04-12T21:41:10.367 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:10.368+0000 7f7fb3e00700 1 -- 172.21.15.18:0/607742566 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7f7fa8013590 con 0x7f7fbc072e00 2024-04-12T21:41:10.368 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:10.369+0000 7f7fb3e00700 1 --2- 172.21.15.18:0/607742566 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f7fa006ec00 0x7f7fa00710b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:41:10.369 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:10.369+0000 7f7fb3e00700 1 -- 172.21.15.18:0/607742566 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 6757+0+0 (secure 0 0 0) 0x7f7fa808fd50 con 0x7f7fbc072e00 2024-04-12T21:41:10.369 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:10.369+0000 7f7fbbe00700 1 --2- 172.21.15.18:0/607742566 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f7fa006ec00 0x7f7fa00710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:41:10.369 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:10.370+0000 7f7fbbe00700 1 --2- 172.21.15.18:0/607742566 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f7fa006ec00 0x7f7fa00710b0 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f7fac004ad0 tx=0x7f7fac007d40 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:41:10.374 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:10.374+0000 7f7fb3e00700 1 -- 172.21.15.18:0/607742566 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f7fa805c580 con 0x7f7fbc072e00 2024-04-12T21:41:10.646 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:10.646+0000 7f7fc1600700 1 -- 172.21.15.18:0/607742566 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f7fbc073c30 con 0x7f7fbc072e00 2024-04-12T21:41:10.647 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:10.647+0000 7f7fb3e00700 1 -- 172.21.15.18:0/607742566 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 33 v33) v1 ==== 76+0+1876 (secure 0 0 0) 0x7f7fa805c110 con 0x7f7fbc072e00 2024-04-12T21:41:10.648 INFO:teuthology.orchestra.run.smithi018.stdout:e33 2024-04-12T21:41:10.648 INFO:teuthology.orchestra.run.smithi018.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-12T21:41:10.648 INFO:teuthology.orchestra.run.smithi018.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-12T21:41:10.648 INFO:teuthology.orchestra.run.smithi018.stdout:legacy client fscid: 1 2024-04-12T21:41:10.648 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-12T21:41:10.648 INFO:teuthology.orchestra.run.smithi018.stdout:Filesystem 'cephfs' (1) 2024-04-12T21:41:10.648 INFO:teuthology.orchestra.run.smithi018.stdout:fs_name cephfs 2024-04-12T21:41:10.648 INFO:teuthology.orchestra.run.smithi018.stdout:epoch 24 2024-04-12T21:41:10.648 INFO:teuthology.orchestra.run.smithi018.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-04-12T21:41:10.649 INFO:teuthology.orchestra.run.smithi018.stdout:created 2024-04-12T20:51:57.362850+0000 2024-04-12T21:41:10.649 INFO:teuthology.orchestra.run.smithi018.stdout:modified 2024-04-12T21:06:59.717190+0000 2024-04-12T21:41:10.649 INFO:teuthology.orchestra.run.smithi018.stdout:tableserver 0 2024-04-12T21:41:10.649 INFO:teuthology.orchestra.run.smithi018.stdout:root 0 2024-04-12T21:41:10.649 INFO:teuthology.orchestra.run.smithi018.stdout:session_timeout 60 2024-04-12T21:41:10.649 INFO:teuthology.orchestra.run.smithi018.stdout:session_autoclose 300 2024-04-12T21:41:10.649 INFO:teuthology.orchestra.run.smithi018.stdout:max_file_size 1099511627776 2024-04-12T21:41:10.649 INFO:teuthology.orchestra.run.smithi018.stdout:max_xattr_size 65536 2024-04-12T21:41:10.649 INFO:teuthology.orchestra.run.smithi018.stdout:required_client_features {} 2024-04-12T21:41:10.649 INFO:teuthology.orchestra.run.smithi018.stdout:last_failure 0 2024-04-12T21:41:10.649 INFO:teuthology.orchestra.run.smithi018.stdout:last_failure_osd_epoch 100 2024-04-12T21:41:10.649 INFO:teuthology.orchestra.run.smithi018.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-12T21:41:10.649 INFO:teuthology.orchestra.run.smithi018.stdout:max_mds 2 2024-04-12T21:41:10.649 INFO:teuthology.orchestra.run.smithi018.stdout:in 0,1 2024-04-12T21:41:10.649 INFO:teuthology.orchestra.run.smithi018.stdout:up {} 2024-04-12T21:41:10.650 INFO:teuthology.orchestra.run.smithi018.stdout:failed 0,1 2024-04-12T21:41:10.650 INFO:teuthology.orchestra.run.smithi018.stdout:damaged 2024-04-12T21:41:10.650 INFO:teuthology.orchestra.run.smithi018.stdout:stopped 2024-04-12T21:41:10.650 INFO:teuthology.orchestra.run.smithi018.stdout:data_pools [3] 2024-04-12T21:41:10.650 INFO:teuthology.orchestra.run.smithi018.stdout:metadata_pool 2 2024-04-12T21:41:10.650 INFO:teuthology.orchestra.run.smithi018.stdout:inline_data disabled 2024-04-12T21:41:10.650 INFO:teuthology.orchestra.run.smithi018.stdout:balancer 2024-04-12T21:41:10.650 INFO:teuthology.orchestra.run.smithi018.stdout:bal_rank_mask -1 2024-04-12T21:41:10.650 INFO:teuthology.orchestra.run.smithi018.stdout:standby_count_wanted 1 2024-04-12T21:41:10.650 INFO:teuthology.orchestra.run.smithi018.stdout:qdb_cluster leader: 0 members: 2024-04-12T21:41:10.650 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-12T21:41:10.650 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-12T21:41:10.650 INFO:teuthology.orchestra.run.smithi018.stdout:Standby daemons: 2024-04-12T21:41:10.650 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-12T21:41:10.650 INFO:teuthology.orchestra.run.smithi018.stdout:[mds.cephfs.smithi018.acqbsc{-1:34436} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.18:6828/755179764,v1:172.21.15.18:6829/755179764] compat {c=[1],r=[1],i=[1fff]}] 2024-04-12T21:41:10.651 INFO:teuthology.orchestra.run.smithi018.stdout:[mds.cephfs.smithi018.szjluo{-1:34468} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.18:6826/2407992653,v1:172.21.15.18:6827/2407992653] compat {c=[1],r=[1],i=[1fff]}] 2024-04-12T21:41:10.651 INFO:teuthology.orchestra.run.smithi018.stdout:[mds.cephfs.smithi186.jdllyn{-1:44595} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.186:6824/3523444202,v1:172.21.15.186:6825/3523444202] compat {c=[1],r=[1],i=[1fff]}] 2024-04-12T21:41:10.651 INFO:teuthology.orchestra.run.smithi018.stdout:[mds.cephfs.smithi186.wjnafs{-1:44875} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.186:6826/3122443766,v1:172.21.15.186:6827/3122443766] compat {c=[1],r=[1],i=[1fff]}] 2024-04-12T21:41:10.652 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:10.650+0000 7f7fc1600700 1 -- 172.21.15.18:0/607742566 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f7fa006ec00 msgr2=0x7f7fa00710b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:41:10.652 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:10.650+0000 7f7fc1600700 1 --2- 172.21.15.18:0/607742566 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f7fa006ec00 0x7f7fa00710b0 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f7fac004ad0 tx=0x7f7fac007d40 comp rx=0 tx=0).stop 2024-04-12T21:41:10.652 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:10.650+0000 7f7fc1600700 1 -- 172.21.15.18:0/607742566 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7fbc072e00 msgr2=0x7f7fbc11a490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:41:10.652 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:10.650+0000 7f7fc1600700 1 --2- 172.21.15.18:0/607742566 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7fbc072e00 0x7f7fbc11a490 secure :-1 s=READY pgs=1053 cs=0 l=1 rev1=1 crypto rx=0x7f7fa8009180 tx=0x7f7fa8009e30 comp rx=0 tx=0).stop 2024-04-12T21:41:10.652 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:10.650+0000 7f7fc1600700 1 -- 172.21.15.18:0/607742566 shutdown_connections 2024-04-12T21:41:10.652 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:10.650+0000 7f7fc1600700 1 --2- 172.21.15.18:0/607742566 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f7fa006ec00 0x7f7fa00710b0 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:41:10.652 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:10.650+0000 7f7fc1600700 1 --2- 172.21.15.18:0/607742566 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7fbc0724f0 0x7f7fbc119f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:41:10.652 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:10.650+0000 7f7fc1600700 1 --2- 172.21.15.18:0/607742566 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7fbc072e00 0x7f7fbc11a490 unknown :-1 s=CLOSED pgs=1053 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:41:10.652 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:10.650+0000 7f7fc1600700 1 -- 172.21.15.18:0/607742566 >> 172.21.15.18:0/607742566 conn(0x7f7fbc0fab10 msgr2=0x7f7fbc0faee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:41:10.652 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:10.650+0000 7f7fc1600700 1 -- 172.21.15.18:0/607742566 shutdown_connections 2024-04-12T21:41:10.652 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:10.651+0000 7f7fc1600700 1 -- 172.21.15.18:0/607742566 wait complete. 2024-04-12T21:41:10.652 INFO:teuthology.orchestra.run.smithi018.stderr:dumped fsmap epoch 33 2024-04-12T21:41:10.799 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:10.798+0000 7f316b200700 1 -- 172.21.15.18:0/3006253825 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f31640702e0 msgr2=0x7f31640706b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:41:10.799 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:10.798+0000 7f316b200700 1 --2- 172.21.15.18:0/3006253825 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f31640702e0 0x7f31640706b0 secure :-1 s=READY pgs=1054 cs=0 l=1 rev1=1 crypto rx=0x7f3154004940 tx=0x7f3154009e30 comp rx=0 tx=0).stop 2024-04-12T21:41:10.799 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:10.798+0000 7f316b200700 1 -- 172.21.15.18:0/3006253825 shutdown_connections 2024-04-12T21:41:10.799 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:10.798+0000 7f316b200700 1 --2- 172.21.15.18:0/3006253825 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3164070bf0 0x7f3164109170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:41:10.799 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:10.798+0000 7f316b200700 1 --2- 172.21.15.18:0/3006253825 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f31640702e0 0x7f31640706b0 unknown :-1 s=CLOSED pgs=1054 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:41:10.799 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:10.798+0000 7f316b200700 1 -- 172.21.15.18:0/3006253825 >> 172.21.15.18:0/3006253825 conn(0x7f31640f8970 msgr2=0x7f31640fad80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:41:10.799 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:10.799+0000 7f316b200700 1 -- 172.21.15.18:0/3006253825 shutdown_connections 2024-04-12T21:41:10.799 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:10.799+0000 7f316b200700 1 -- 172.21.15.18:0/3006253825 wait complete. 2024-04-12T21:41:10.800 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:10.799+0000 7f316b200700 1 Processor -- start 2024-04-12T21:41:10.800 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:10.800+0000 7f316b200700 1 -- start start 2024-04-12T21:41:10.800 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:10.800+0000 7f316b200700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f31640702e0 0x7f3164115db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:41:10.800 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:10.800+0000 7f316b200700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3164070bf0 0x7f31641162f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:41:10.800 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:10.800+0000 7f316b200700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3164116940 con 0x7f3164070bf0 2024-04-12T21:41:10.801 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:10.800+0000 7f316b200700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f316410fd10 con 0x7f31640702e0 2024-04-12T21:41:10.801 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:10.800+0000 7f3163400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3164070bf0 0x7f31641162f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:41:10.801 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:10.800+0000 7f3163400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3164070bf0 0x7f31641162f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:40126/0 (socket says 172.21.15.18:40126) 2024-04-12T21:41:10.801 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:10.800+0000 7f3163400700 1 -- 172.21.15.18:0/3806114275 learned_addr learned my addr 172.21.15.18:0/3806114275 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:41:10.802 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:10.801+0000 7f3163400700 1 -- 172.21.15.18:0/3806114275 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f31640702e0 msgr2=0x7f3164115db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:41:10.802 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:10.801+0000 7f3163400700 1 --2- 172.21.15.18:0/3806114275 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f31640702e0 0x7f3164115db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:41:10.802 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:10.801+0000 7f3163400700 1 -- 172.21.15.18:0/3806114275 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3154005040 con 0x7f3164070bf0 2024-04-12T21:41:10.802 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:10.801+0000 7f3163400700 1 --2- 172.21.15.18:0/3806114275 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3164070bf0 0x7f31641162f0 secure :-1 s=READY pgs=1055 cs=0 l=1 rev1=1 crypto rx=0x7f3150002a50 tx=0x7f31500046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:41:10.802 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:10.801+0000 7f315be00700 1 -- 172.21.15.18:0/3806114275 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3150007cc0 con 0x7f3164070bf0 2024-04-12T21:41:10.802 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:10.801+0000 7f316b200700 1 -- 172.21.15.18:0/3806114275 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f316410fff0 con 0x7f3164070bf0 2024-04-12T21:41:10.802 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:10.801+0000 7f315be00700 1 -- 172.21.15.18:0/3806114275 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f315000f040 con 0x7f3164070bf0 2024-04-12T21:41:10.802 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:10.801+0000 7f315be00700 1 -- 172.21.15.18:0/3806114275 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3150009640 con 0x7f3164070bf0 2024-04-12T21:41:10.802 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:10.801+0000 7f316b200700 1 -- 172.21.15.18:0/3806114275 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3164110530 con 0x7f3164070bf0 2024-04-12T21:41:10.803 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:10.802+0000 7f316b200700 1 -- 172.21.15.18:0/3806114275 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3164071a20 con 0x7f3164070bf0 2024-04-12T21:41:10.804 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:10.804+0000 7f315be00700 1 -- 172.21.15.18:0/3806114275 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7f31500097c0 con 0x7f3164070bf0 2024-04-12T21:41:10.805 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:10.805+0000 7f315be00700 1 --2- 172.21.15.18:0/3806114275 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f314806ec00 0x7f31480710b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:41:10.805 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:10.806+0000 7f315be00700 1 -- 172.21.15.18:0/3806114275 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 6757+0+0 (secure 0 0 0) 0x7f315008e3e0 con 0x7f3164070bf0 2024-04-12T21:41:10.806 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:10.806+0000 7f3163e00700 1 --2- 172.21.15.18:0/3806114275 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f314806ec00 0x7f31480710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:41:10.806 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:10.806+0000 7f3163e00700 1 --2- 172.21.15.18:0/3806114275 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f314806ec00 0x7f31480710b0 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f3154004f90 tx=0x7f3154009e30 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:41:10.810 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:10.811+0000 7f315be00700 1 -- 172.21.15.18:0/3806114275 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f315005ac30 con 0x7f3164070bf0 2024-04-12T21:41:11.069 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:11.069+0000 7f316b200700 1 -- 172.21.15.18:0/3806114275 --> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f316410c130 con 0x7f314806ec00 2024-04-12T21:41:11.072 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:11.072+0000 7f315be00700 1 -- 172.21.15.18:0/3806114275 <== mgr.34102 v2:172.21.15.18:6800/3812891919 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f316410c130 con 0x7f314806ec00 2024-04-12T21:41:11.072 INFO:teuthology.orchestra.run.smithi018.stdout:{ 2024-04-12T21:41:11.072 INFO:teuthology.orchestra.run.smithi018.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:f4452def3be4d350f4a8e5feaf7ebb1511be9298fe68478ba7a2d6fbd7d69a63", 2024-04-12T21:41:11.072 INFO:teuthology.orchestra.run.smithi018.stdout: "in_progress": true, 2024-04-12T21:41:11.072 INFO:teuthology.orchestra.run.smithi018.stdout: "which": "Upgrading all daemon types on all hosts", 2024-04-12T21:41:11.072 INFO:teuthology.orchestra.run.smithi018.stdout: "services_complete": [ 2024-04-12T21:41:11.072 INFO:teuthology.orchestra.run.smithi018.stdout: "mgr", 2024-04-12T21:41:11.072 INFO:teuthology.orchestra.run.smithi018.stdout: "mds", 2024-04-12T21:41:11.072 INFO:teuthology.orchestra.run.smithi018.stdout: "crash", 2024-04-12T21:41:11.072 INFO:teuthology.orchestra.run.smithi018.stdout: "osd", 2024-04-12T21:41:11.073 INFO:teuthology.orchestra.run.smithi018.stdout: "mon" 2024-04-12T21:41:11.073 INFO:teuthology.orchestra.run.smithi018.stdout: ], 2024-04-12T21:41:11.073 INFO:teuthology.orchestra.run.smithi018.stdout: "progress": "16/21 daemons upgraded", 2024-04-12T21:41:11.073 INFO:teuthology.orchestra.run.smithi018.stdout: "message": "Currently upgrading mds daemons", 2024-04-12T21:41:11.073 INFO:teuthology.orchestra.run.smithi018.stdout: "is_paused": false 2024-04-12T21:41:11.073 INFO:teuthology.orchestra.run.smithi018.stdout:} 2024-04-12T21:41:11.075 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:11.074+0000 7f316b200700 1 -- 172.21.15.18:0/3806114275 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f314806ec00 msgr2=0x7f31480710b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:41:11.075 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:11.074+0000 7f316b200700 1 --2- 172.21.15.18:0/3806114275 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f314806ec00 0x7f31480710b0 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f3154004f90 tx=0x7f3154009e30 comp rx=0 tx=0).stop 2024-04-12T21:41:11.075 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:11.074+0000 7f316b200700 1 -- 172.21.15.18:0/3806114275 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3164070bf0 msgr2=0x7f31641162f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:41:11.075 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:11.074+0000 7f316b200700 1 --2- 172.21.15.18:0/3806114275 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3164070bf0 0x7f31641162f0 secure :-1 s=READY pgs=1055 cs=0 l=1 rev1=1 crypto rx=0x7f3150002a50 tx=0x7f31500046b0 comp rx=0 tx=0).stop 2024-04-12T21:41:11.075 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:11.075+0000 7f316b200700 1 -- 172.21.15.18:0/3806114275 shutdown_connections 2024-04-12T21:41:11.075 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:11.075+0000 7f316b200700 1 --2- 172.21.15.18:0/3806114275 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f314806ec00 0x7f31480710b0 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:41:11.075 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:11.075+0000 7f316b200700 1 --2- 172.21.15.18:0/3806114275 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f31640702e0 0x7f3164115db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:41:11.075 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:11.075+0000 7f316b200700 1 --2- 172.21.15.18:0/3806114275 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3164070bf0 0x7f31641162f0 unknown :-1 s=CLOSED pgs=1055 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:41:11.075 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:11.075+0000 7f316b200700 1 -- 172.21.15.18:0/3806114275 >> 172.21.15.18:0/3806114275 conn(0x7f31640f8970 msgr2=0x7f31640f9eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:41:11.075 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:11.075+0000 7f316b200700 1 -- 172.21.15.18:0/3806114275 shutdown_connections 2024-04-12T21:41:11.076 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:11.075+0000 7f316b200700 1 -- 172.21.15.18:0/3806114275 wait complete. 2024-04-12T21:41:11.232 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:11.232+0000 7f3bc1200700 1 -- 172.21.15.18:0/645228694 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3bbc0fbb70 msgr2=0x7f3bbc0fbfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:41:11.232 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:11.232+0000 7f3bc1200700 1 --2- 172.21.15.18:0/645228694 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3bbc0fbb70 0x7f3bbc0fbfe0 secure :-1 s=READY pgs=1056 cs=0 l=1 rev1=1 crypto rx=0x7f3ba8004940 tx=0x7f3ba8009e30 comp rx=0 tx=0).stop 2024-04-12T21:41:11.232 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:41:10 smithi018 ceph-mon[112590]: from='client.36164 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:41:11.233 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:41:10 smithi018 ceph-mon[112590]: from='client.36168 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:41:11.233 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:41:10 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T21:41:11.233 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:41:10 smithi018 ceph-mon[112590]: from='client.? 172.21.15.18:0/279414448' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T21:41:11.233 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:41:10 smithi018 ceph-mon[112590]: from='client.? 172.21.15.18:0/607742566' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-12T21:41:11.233 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:11.232+0000 7f3bc1200700 1 -- 172.21.15.18:0/645228694 shutdown_connections 2024-04-12T21:41:11.233 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:11.232+0000 7f3bc1200700 1 --2- 172.21.15.18:0/645228694 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3bbc0fbb70 0x7f3bbc0fbfe0 unknown :-1 s=CLOSED pgs=1056 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:41:11.234 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:11.232+0000 7f3bc1200700 1 --2- 172.21.15.18:0/645228694 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3bbc101ba0 0x7f3bbc101f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:41:11.234 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:11.232+0000 7f3bc1200700 1 -- 172.21.15.18:0/645228694 >> 172.21.15.18:0/645228694 conn(0x7f3bbc071b70 msgr2=0x7f3bbc071f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:41:11.234 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:11.232+0000 7f3bc1200700 1 -- 172.21.15.18:0/645228694 shutdown_connections 2024-04-12T21:41:11.234 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:11.233+0000 7f3bc1200700 1 -- 172.21.15.18:0/645228694 wait complete. 2024-04-12T21:41:11.234 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:11.233+0000 7f3bc1200700 1 Processor -- start 2024-04-12T21:41:11.234 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:11.234+0000 7f3bc1200700 1 -- start start 2024-04-12T21:41:11.235 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:11.234+0000 7f3bc1200700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3bbc0fbb70 0x7f3bbc115c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:41:11.235 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:11.234+0000 7f3bc1200700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3bbc101ba0 0x7f3bbc116150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:41:11.235 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:11.234+0000 7f3bc1200700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3bbc1167a0 con 0x7f3bbc101ba0 2024-04-12T21:41:11.235 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:11.234+0000 7f3bc1200700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3bbc10fbb0 con 0x7f3bbc0fbb70 2024-04-12T21:41:11.235 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:11.234+0000 7f3bbaa00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3bbc101ba0 0x7f3bbc116150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:41:11.235 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:11.235+0000 7f3bbaa00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3bbc101ba0 0x7f3bbc116150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:56328/0 (socket says 172.21.15.18:56328) 2024-04-12T21:41:11.236 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:11.235+0000 7f3bbaa00700 1 -- 172.21.15.18:0/1424420069 learned_addr learned my addr 172.21.15.18:0/1424420069 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:41:11.236 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:11.235+0000 7f3bbb400700 1 --2- 172.21.15.18:0/1424420069 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3bbc0fbb70 0x7f3bbc115c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:41:11.236 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:11.235+0000 7f3bbaa00700 1 -- 172.21.15.18:0/1424420069 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3bbc0fbb70 msgr2=0x7f3bbc115c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:41:11.236 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:11.235+0000 7f3bbaa00700 1 --2- 172.21.15.18:0/1424420069 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3bbc0fbb70 0x7f3bbc115c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:41:11.236 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:11.235+0000 7f3bbaa00700 1 -- 172.21.15.18:0/1424420069 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ba8005040 con 0x7f3bbc101ba0 2024-04-12T21:41:11.236 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:11.235+0000 7f3bbaa00700 1 --2- 172.21.15.18:0/1424420069 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3bbc101ba0 0x7f3bbc116150 secure :-1 s=READY pgs=1057 cs=0 l=1 rev1=1 crypto rx=0x7f3ba8009150 tx=0x7f3ba8009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:41:11.236 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:11.236+0000 7f3bb3400700 1 -- 172.21.15.18:0/1424420069 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3ba800e040 con 0x7f3bbc101ba0 2024-04-12T21:41:11.236 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:11.236+0000 7f3bb3400700 1 -- 172.21.15.18:0/1424420069 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f3ba8013420 con 0x7f3bbc101ba0 2024-04-12T21:41:11.236 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:11.236+0000 7f3bb3400700 1 -- 172.21.15.18:0/1424420069 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3ba801c630 con 0x7f3bbc101ba0 2024-04-12T21:41:11.237 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:11.236+0000 7f3bc1200700 1 -- 172.21.15.18:0/1424420069 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3bbc10fe30 con 0x7f3bbc101ba0 2024-04-12T21:41:11.237 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:11.236+0000 7f3bc1200700 1 -- 172.21.15.18:0/1424420069 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3bbc110370 con 0x7f3bbc101ba0 2024-04-12T21:41:11.237 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:11.237+0000 7f3bc1200700 1 -- 172.21.15.18:0/1424420069 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3bbc0fd200 con 0x7f3bbc101ba0 2024-04-12T21:41:11.238 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:11.238+0000 7f3bb3400700 1 -- 172.21.15.18:0/1424420069 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7f3ba8013590 con 0x7f3bbc101ba0 2024-04-12T21:41:11.239 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:11.239+0000 7f3bb3400700 1 --2- 172.21.15.18:0/1424420069 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f3ba006ec10 0x7f3ba00710c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:41:11.239 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:11.240+0000 7f3bb3400700 1 -- 172.21.15.18:0/1424420069 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 6757+0+0 (secure 0 0 0) 0x7f3ba808fc30 con 0x7f3bbc101ba0 2024-04-12T21:41:11.239 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:11.240+0000 7f3bbb400700 1 --2- 172.21.15.18:0/1424420069 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f3ba006ec10 0x7f3ba00710c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:41:11.240 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:11.240+0000 7f3bbb400700 1 --2- 172.21.15.18:0/1424420069 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f3ba006ec10 0x7f3ba00710c0 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f3bac007fd0 tx=0x7f3bac009040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:41:11.244 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:11.244+0000 7f3bb3400700 1 -- 172.21.15.18:0/1424420069 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f3ba8058930 con 0x7f3bbc101ba0 2024-04-12T21:41:11.283 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:41:10 smithi186 ceph-mon[86379]: from='client.36164 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:41:11.283 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:41:10 smithi186 ceph-mon[86379]: from='client.36168 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:41:11.283 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:41:10 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T21:41:11.283 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:41:10 smithi186 ceph-mon[86379]: from='client.? 172.21.15.18:0/279414448' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T21:41:11.283 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:41:10 smithi186 ceph-mon[86379]: from='client.? 172.21.15.18:0/607742566' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-12T21:41:11.565 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:11.565+0000 7f3bc1200700 1 -- 172.21.15.18:0/1424420069 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f3bbc108c20 con 0x7f3bbc101ba0 2024-04-12T21:41:11.566 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:11.566+0000 7f3bb3400700 1 -- 172.21.15.18:0/1424420069 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+4303 (secure 0 0 0) 0x7f3ba805bff0 con 0x7f3bbc101ba0 2024-04-12T21:41:11.567 INFO:teuthology.orchestra.run.smithi018.stdout:HEALTH_ERR failed to probe daemons or devices; 1 filesystem is degraded; 1 filesystem is offline 2024-04-12T21:41:11.567 INFO:teuthology.orchestra.run.smithi018.stdout:[WRN] CEPHADM_REFRESH_FAILED: failed to probe daemons or devices 2024-04-12T21:41:11.567 INFO:teuthology.orchestra.run.smithi018.stdout: host smithi186 `cephadm ceph-volume` failed: cephadm exited with an error code: 1, stderr: Inferring config /var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/mon.smithi186/config 2024-04-12T21:41:11.567 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/podman: timeout after 895 seconds 2024-04-12T21:41:11.567 INFO:teuthology.orchestra.run.smithi018.stdout:Non-zero exit code 124 from /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:f4452def3be4d350f4a8e5feaf7ebb1511be9298fe68478ba7a2d6fbd7d69a63 -e NODE_NAME=smithi186 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968:/var/run/ceph:z -v /var/log/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968:/var/log/ceph:z -v /var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmptgom53f9:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:f4452def3be4d350f4a8e5feaf7ebb1511be9298fe68478ba7a2d6fbd7d69a63 inventory --format=json-pretty --filter-for-batch 2024-04-12T21:41:11.567 INFO:teuthology.orchestra.run.smithi018.stdout:Traceback (most recent call last): 2024-04-12T21:41:11.567 INFO:teuthology.orchestra.run.smithi018.stdout: File "/usr/lib64/python3.9/runpy.py", line 197, in _run_module_as_main 2024-04-12T21:41:11.567 INFO:teuthology.orchestra.run.smithi018.stdout: return _run_code(code, main_globals, None, 2024-04-12T21:41:11.567 INFO:teuthology.orchestra.run.smithi018.stdout: File "/usr/lib64/python3.9/runpy.py", line 87, in _run_code 2024-04-12T21:41:11.567 INFO:teuthology.orchestra.run.smithi018.stdout: exec(code, run_globals) 2024-04-12T21:41:11.567 INFO:teuthology.orchestra.run.smithi018.stdout: File "/var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/cephadm.e44f3ed5e34e4ef52eedef233d1190bff004470d8e8005127b5312bff357fa7f/__main__.py", line 5618, in 2024-04-12T21:41:11.568 INFO:teuthology.orchestra.run.smithi018.stdout: File "/var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/cephadm.e44f3ed5e34e4ef52eedef233d1190bff004470d8e8005127b5312bff357fa7f/__main__.py", line 5606, in main 2024-04-12T21:41:11.568 INFO:teuthology.orchestra.run.smithi018.stdout: File "/var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/cephadm.e44f3ed5e34e4ef52eedef233d1190bff004470d8e8005127b5312bff357fa7f/__main__.py", line 408, in _infer_config 2024-04-12T21:41:11.568 INFO:teuthology.orchestra.run.smithi018.stdout: File "/var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/cephadm.e44f3ed5e34e4ef52eedef233d1190bff004470d8e8005127b5312bff357fa7f/__main__.py", line 323, in _infer_fsid 2024-04-12T21:41:11.568 INFO:teuthology.orchestra.run.smithi018.stdout: File "/var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/cephadm.e44f3ed5e34e4ef52eedef233d1190bff004470d8e8005127b5312bff357fa7f/__main__.py", line 436, in _infer_image 2024-04-12T21:41:11.568 INFO:teuthology.orchestra.run.smithi018.stdout: File "/var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/cephadm.e44f3ed5e34e4ef52eedef233d1190bff004470d8e8005127b5312bff357fa7f/__main__.py", line 310, in _validate_fsid 2024-04-12T21:41:11.568 INFO:teuthology.orchestra.run.smithi018.stdout: File "/var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/cephadm.e44f3ed5e34e4ef52eedef233d1190bff004470d8e8005127b5312bff357fa7f/__main__.py", line 3359, in command_ceph_volume 2024-04-12T21:41:11.568 INFO:teuthology.orchestra.run.smithi018.stdout: File "/var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/cephadm.e44f3ed5e34e4ef52eedef233d1190bff004470d8e8005127b5312bff357fa7f/cephadmlib/call_wrappers.py", line 310, in call_throws 2024-04-12T21:41:11.568 INFO:teuthology.orchestra.run.smithi018.stdout:RuntimeError: Failed command: /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:f4452def3be4d350f4a8e5feaf7ebb1511be9298fe68478ba7a2d6fbd7d69a63 -e NODE_NAME=smithi186 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968:/var/run/ceph:z -v /var/log/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968:/var/log/ceph:z -v /var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmptgom53f9:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:f4452def3be4d350f4a8e5feaf7ebb1511be9298fe68478ba7a2d6fbd7d69a63 inventory --format=json-pretty --filter-for-batch 2024-04-12T21:41:11.568 INFO:teuthology.orchestra.run.smithi018.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-04-12T21:41:11.568 INFO:teuthology.orchestra.run.smithi018.stdout: fs cephfs is degraded 2024-04-12T21:41:11.568 INFO:teuthology.orchestra.run.smithi018.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-04-12T21:41:11.568 INFO:teuthology.orchestra.run.smithi018.stdout: fs cephfs is offline because no MDS is active for it. 2024-04-12T21:41:11.570 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:11.570+0000 7f3bc1200700 1 -- 172.21.15.18:0/1424420069 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f3ba006ec10 msgr2=0x7f3ba00710c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:41:11.570 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:11.570+0000 7f3bc1200700 1 --2- 172.21.15.18:0/1424420069 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f3ba006ec10 0x7f3ba00710c0 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f3bac007fd0 tx=0x7f3bac009040 comp rx=0 tx=0).stop 2024-04-12T21:41:11.570 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:11.570+0000 7f3bc1200700 1 -- 172.21.15.18:0/1424420069 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3bbc101ba0 msgr2=0x7f3bbc116150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:41:11.570 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:11.570+0000 7f3bc1200700 1 --2- 172.21.15.18:0/1424420069 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3bbc101ba0 0x7f3bbc116150 secure :-1 s=READY pgs=1057 cs=0 l=1 rev1=1 crypto rx=0x7f3ba8009150 tx=0x7f3ba8009e30 comp rx=0 tx=0).stop 2024-04-12T21:41:11.571 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:11.570+0000 7f3bc1200700 1 -- 172.21.15.18:0/1424420069 shutdown_connections 2024-04-12T21:41:11.571 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:11.570+0000 7f3bc1200700 1 --2- 172.21.15.18:0/1424420069 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f3ba006ec10 0x7f3ba00710c0 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:41:11.571 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:11.570+0000 7f3bc1200700 1 --2- 172.21.15.18:0/1424420069 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3bbc0fbb70 0x7f3bbc115c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:41:11.571 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:11.570+0000 7f3bc1200700 1 --2- 172.21.15.18:0/1424420069 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3bbc101ba0 0x7f3bbc116150 unknown :-1 s=CLOSED pgs=1057 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:41:11.571 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:11.570+0000 7f3bc1200700 1 -- 172.21.15.18:0/1424420069 >> 172.21.15.18:0/1424420069 conn(0x7f3bbc071b70 msgr2=0x7f3bbc065600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:41:11.571 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:11.571+0000 7f3bc1200700 1 -- 172.21.15.18:0/1424420069 shutdown_connections 2024-04-12T21:41:11.571 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:11.571+0000 7f3bc1200700 1 -- 172.21.15.18:0/1424420069 wait complete. 2024-04-12T21:41:12.283 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:41:11 smithi186 ceph-mon[86379]: pgmap v1245: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:41:12.284 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:41:11 smithi186 ceph-mon[86379]: from='client.? 172.21.15.18:0/1424420069' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-12T21:41:12.354 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:41:11 smithi018 ceph-mon[112590]: pgmap v1245: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:41:12.355 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:41:11 smithi018 ceph-mon[112590]: from='client.? 172.21.15.18:0/1424420069' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-12T21:41:13.283 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:41:12 smithi186 ceph-mon[86379]: from='client.36180 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:41:13.354 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:41:12 smithi018 ceph-mon[112590]: from='client.36180 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:41:14.283 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:41:13 smithi186 ceph-mon[86379]: pgmap v1246: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:41:14.354 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:41:13 smithi018 ceph-mon[112590]: pgmap v1246: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:41:16.283 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:41:15 smithi186 ceph-mon[86379]: pgmap v1247: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:41:16.354 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:41:15 smithi018 ceph-mon[112590]: pgmap v1247: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:41:18.283 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:41:17 smithi186 ceph-mon[86379]: pgmap v1248: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:41:18.354 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:41:17 smithi018 ceph-mon[112590]: pgmap v1248: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:41:20.283 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:41:19 smithi186 ceph-mon[86379]: pgmap v1249: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:41:20.354 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:41:19 smithi018 ceph-mon[112590]: pgmap v1249: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:41:22.283 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:41:21 smithi186 ceph-mon[86379]: pgmap v1250: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:41:22.354 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:41:21 smithi018 ceph-mon[112590]: pgmap v1250: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:41:24.283 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:41:23 smithi186 ceph-mon[86379]: pgmap v1251: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:41:24.355 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:41:23 smithi018 ceph-mon[112590]: pgmap v1251: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:41:25.283 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:41:24 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T21:41:25.354 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:41:24 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T21:41:26.283 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:41:25 smithi186 ceph-mon[86379]: pgmap v1252: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:41:26.354 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:41:25 smithi018 ceph-mon[112590]: pgmap v1252: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:41:28.283 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:41:27 smithi186 ceph-mon[86379]: pgmap v1253: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:41:28.354 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:41:27 smithi018 ceph-mon[112590]: pgmap v1253: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:41:30.283 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:41:29 smithi186 ceph-mon[86379]: pgmap v1254: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:41:30.354 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:41:29 smithi018 ceph-mon[112590]: pgmap v1254: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:41:32.283 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:41:31 smithi186 ceph-mon[86379]: pgmap v1255: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:41:32.354 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:41:31 smithi018 ceph-mon[112590]: pgmap v1255: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:41:34.283 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:41:34 smithi186 ceph-mon[86379]: pgmap v1256: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:41:34.354 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:41:34 smithi018 ceph-mon[112590]: pgmap v1256: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:41:35.283 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:41:35 smithi186 ceph-mon[86379]: pgmap v1257: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:41:35.354 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:41:35 smithi018 ceph-mon[112590]: pgmap v1257: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:41:37.533 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:41:37 smithi186 ceph-mon[86379]: pgmap v1258: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:41:37.604 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:41:37 smithi018 ceph-mon[112590]: pgmap v1258: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:41:39.534 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:41:39 smithi186 ceph-mon[86379]: pgmap v1259: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:41:39.604 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:41:39 smithi018 ceph-mon[112590]: pgmap v1259: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:41:40.533 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:41:40 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T21:41:40.604 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:41:40 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T21:41:41.533 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:41:41 smithi186 ceph-mon[86379]: pgmap v1260: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:41:41.604 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:41:41 smithi018 ceph-mon[112590]: pgmap v1260: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:41:41.747 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:41.745+0000 7f2e7be00700 1 -- 172.21.15.18:0/1999389708 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2e741059c0 msgr2=0x7f2e740ff080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:41:41.747 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:41.745+0000 7f2e7be00700 1 --2- 172.21.15.18:0/1999389708 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2e741059c0 0x7f2e740ff080 secure :-1 s=READY pgs=1058 cs=0 l=1 rev1=1 crypto rx=0x7f2e60004940 tx=0x7f2e60009e30 comp rx=0 tx=0).stop 2024-04-12T21:41:41.747 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:41.745+0000 7f2e7be00700 1 -- 172.21.15.18:0/1999389708 shutdown_connections 2024-04-12T21:41:41.747 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:41.745+0000 7f2e7be00700 1 --2- 172.21.15.18:0/1999389708 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2e741059c0 0x7f2e740ff080 unknown :-1 s=CLOSED pgs=1058 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:41:41.747 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:41.745+0000 7f2e7be00700 1 --2- 172.21.15.18:0/1999389708 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2e74105020 0x7f2e741053f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:41:41.747 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:41.745+0000 7f2e7be00700 1 -- 172.21.15.18:0/1999389708 >> 172.21.15.18:0/1999389708 conn(0x7f2e740faba0 msgr2=0x7f2e740fcfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:41:41.747 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:41.745+0000 7f2e7be00700 1 -- 172.21.15.18:0/1999389708 shutdown_connections 2024-04-12T21:41:41.747 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:41.745+0000 7f2e7be00700 1 -- 172.21.15.18:0/1999389708 wait complete. 2024-04-12T21:41:41.749 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:41.746+0000 7f2e7be00700 1 Processor -- start 2024-04-12T21:41:41.749 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:41.746+0000 7f2e7be00700 1 -- start start 2024-04-12T21:41:41.749 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:41.747+0000 7f2e7be00700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2e74105020 0x7f2e74115ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:41:41.749 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:41.747+0000 7f2e7be00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2e741059c0 0x7f2e741160e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:41:41.749 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:41.747+0000 7f2e7be00700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e74116730 con 0x7f2e741059c0 2024-04-12T21:41:41.749 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:41.747+0000 7f2e7be00700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e7410fd10 con 0x7f2e74105020 2024-04-12T21:41:41.749 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:41.747+0000 7f2e78a00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2e741059c0 0x7f2e741160e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:41:41.749 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:41.747+0000 7f2e78a00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2e741059c0 0x7f2e741160e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:58236/0 (socket says 172.21.15.18:58236) 2024-04-12T21:41:41.749 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:41.747+0000 7f2e78a00700 1 -- 172.21.15.18:0/2614638973 learned_addr learned my addr 172.21.15.18:0/2614638973 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:41:41.749 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:41.748+0000 7f2e79400700 1 --2- 172.21.15.18:0/2614638973 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2e74105020 0x7f2e74115ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:41:41.749 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:41.748+0000 7f2e78a00700 1 -- 172.21.15.18:0/2614638973 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2e74105020 msgr2=0x7f2e74115ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:41:41.749 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:41.748+0000 7f2e78a00700 1 --2- 172.21.15.18:0/2614638973 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2e74105020 0x7f2e74115ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:41:41.749 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:41.748+0000 7f2e78a00700 1 -- 172.21.15.18:0/2614638973 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e60005040 con 0x7f2e741059c0 2024-04-12T21:41:41.751 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:41.748+0000 7f2e78a00700 1 --2- 172.21.15.18:0/2614638973 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2e741059c0 0x7f2e741160e0 secure :-1 s=READY pgs=1059 cs=0 l=1 rev1=1 crypto rx=0x7f2e60009100 tx=0x7f2e60009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:41:41.751 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:41.748+0000 7f2e71600700 1 -- 172.21.15.18:0/2614638973 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2e6000e040 con 0x7f2e741059c0 2024-04-12T21:41:41.751 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:41.749+0000 7f2e71600700 1 -- 172.21.15.18:0/2614638973 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f2e60013420 con 0x7f2e741059c0 2024-04-12T21:41:41.751 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:41.749+0000 7f2e7be00700 1 -- 172.21.15.18:0/2614638973 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2e7410ff90 con 0x7f2e741059c0 2024-04-12T21:41:41.751 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:41.749+0000 7f2e71600700 1 -- 172.21.15.18:0/2614638973 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2e6001c630 con 0x7f2e741059c0 2024-04-12T21:41:41.751 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:41.749+0000 7f2e7be00700 1 -- 172.21.15.18:0/2614638973 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2e74110550 con 0x7f2e741059c0 2024-04-12T21:41:41.751 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:41.750+0000 7f2e7be00700 1 -- 172.21.15.18:0/2614638973 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2e74100770 con 0x7f2e741059c0 2024-04-12T21:41:41.752 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:41.751+0000 7f2e71600700 1 -- 172.21.15.18:0/2614638973 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7f2e60013590 con 0x7f2e741059c0 2024-04-12T21:41:41.753 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:41.752+0000 7f2e71600700 1 --2- 172.21.15.18:0/2614638973 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f2e5c06ec00 0x7f2e5c0710b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:41:41.753 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:41.752+0000 7f2e71600700 1 -- 172.21.15.18:0/2614638973 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 6757+0+0 (secure 0 0 0) 0x7f2e6008fba0 con 0x7f2e741059c0 2024-04-12T21:41:41.753 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:41.752+0000 7f2e79400700 1 --2- 172.21.15.18:0/2614638973 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f2e5c06ec00 0x7f2e5c0710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:41:41.754 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:41.753+0000 7f2e79400700 1 --2- 172.21.15.18:0/2614638973 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f2e5c06ec00 0x7f2e5c0710b0 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f2e68004990 tx=0x7f2e68009ee0 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:41:41.758 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:41.757+0000 7f2e71600700 1 -- 172.21.15.18:0/2614638973 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f2e6005c3d0 con 0x7f2e741059c0 2024-04-12T21:41:42.016 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:42.014+0000 7f2e7be00700 1 -- 172.21.15.18:0/2614638973 --> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f2e7410c100 con 0x7f2e5c06ec00 2024-04-12T21:41:42.018 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:42.017+0000 7f2e71600700 1 -- 172.21.15.18:0/2614638973 <== mgr.34102 v2:172.21.15.18:6800/3812891919 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f2e7410c100 con 0x7f2e5c06ec00 2024-04-12T21:41:42.022 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:42.020+0000 7f2e7be00700 1 -- 172.21.15.18:0/2614638973 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f2e5c06ec00 msgr2=0x7f2e5c0710b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:41:42.022 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:42.020+0000 7f2e7be00700 1 --2- 172.21.15.18:0/2614638973 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f2e5c06ec00 0x7f2e5c0710b0 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f2e68004990 tx=0x7f2e68009ee0 comp rx=0 tx=0).stop 2024-04-12T21:41:42.022 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:42.020+0000 7f2e7be00700 1 -- 172.21.15.18:0/2614638973 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2e741059c0 msgr2=0x7f2e741160e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:41:42.022 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:42.020+0000 7f2e7be00700 1 --2- 172.21.15.18:0/2614638973 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2e741059c0 0x7f2e741160e0 secure :-1 s=READY pgs=1059 cs=0 l=1 rev1=1 crypto rx=0x7f2e60009100 tx=0x7f2e60009e30 comp rx=0 tx=0).stop 2024-04-12T21:41:42.022 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:42.021+0000 7f2e7be00700 1 -- 172.21.15.18:0/2614638973 shutdown_connections 2024-04-12T21:41:42.022 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:42.021+0000 7f2e7be00700 1 --2- 172.21.15.18:0/2614638973 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f2e5c06ec00 0x7f2e5c0710b0 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:41:42.022 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:42.021+0000 7f2e7be00700 1 --2- 172.21.15.18:0/2614638973 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2e74105020 0x7f2e74115ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:41:42.022 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:42.021+0000 7f2e7be00700 1 --2- 172.21.15.18:0/2614638973 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2e741059c0 0x7f2e741160e0 unknown :-1 s=CLOSED pgs=1059 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:41:42.023 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:42.021+0000 7f2e7be00700 1 -- 172.21.15.18:0/2614638973 >> 172.21.15.18:0/2614638973 conn(0x7f2e740faba0 msgr2=0x7f2e740fbb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:41:42.023 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:42.021+0000 7f2e7be00700 1 -- 172.21.15.18:0/2614638973 shutdown_connections 2024-04-12T21:41:42.023 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:42.021+0000 7f2e7be00700 1 -- 172.21.15.18:0/2614638973 wait complete. 2024-04-12T21:41:42.043 INFO:teuthology.orchestra.run.smithi018.stdout:true 2024-04-12T21:41:42.192 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:42.190+0000 7f4c4b400700 1 -- 172.21.15.18:0/117554891 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4c440fe830 msgr2=0x7f4c44106f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:41:42.192 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:42.190+0000 7f4c4b400700 1 --2- 172.21.15.18:0/117554891 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4c440fe830 0x7f4c44106f00 secure :-1 s=READY pgs=1060 cs=0 l=1 rev1=1 crypto rx=0x7f4c2c004940 tx=0x7f4c2c009e30 comp rx=0 tx=0).stop 2024-04-12T21:41:42.192 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:42.191+0000 7f4c4b400700 1 -- 172.21.15.18:0/117554891 shutdown_connections 2024-04-12T21:41:42.193 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:42.191+0000 7f4c4b400700 1 --2- 172.21.15.18:0/117554891 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4c440fe830 0x7f4c44106f00 unknown :-1 s=CLOSED pgs=1060 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:41:42.193 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:42.191+0000 7f4c4b400700 1 --2- 172.21.15.18:0/117554891 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4c440fde90 0x7f4c440fe260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:41:42.193 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:42.191+0000 7f4c4b400700 1 -- 172.21.15.18:0/117554891 >> 172.21.15.18:0/117554891 conn(0x7f4c44072bc0 msgr2=0x7f4c44072fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:41:42.193 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:42.191+0000 7f4c4b400700 1 -- 172.21.15.18:0/117554891 shutdown_connections 2024-04-12T21:41:42.193 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:42.191+0000 7f4c4b400700 1 -- 172.21.15.18:0/117554891 wait complete. 2024-04-12T21:41:42.194 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:42.192+0000 7f4c4b400700 1 Processor -- start 2024-04-12T21:41:42.194 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:42.192+0000 7f4c4b400700 1 -- start start 2024-04-12T21:41:42.194 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:42.192+0000 7f4c4b400700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4c440fde90 0x7f4c44115d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:41:42.195 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:42.193+0000 7f4c4b400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4c440fe830 0x7f4c441162a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:41:42.195 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:42.193+0000 7f4c4b400700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c441168f0 con 0x7f4c440fe830 2024-04-12T21:41:42.195 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:42.193+0000 7f4c4b400700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c4410fd00 con 0x7f4c440fde90 2024-04-12T21:41:42.195 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:42.193+0000 7f4c3be00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4c440fe830 0x7f4c441162a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:41:42.195 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:42.193+0000 7f4c3be00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4c440fe830 0x7f4c441162a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:58252/0 (socket says 172.21.15.18:58252) 2024-04-12T21:41:42.195 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:42.193+0000 7f4c3be00700 1 -- 172.21.15.18:0/2125131435 learned_addr learned my addr 172.21.15.18:0/2125131435 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:41:42.195 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:42.193+0000 7f4c48a00700 1 --2- 172.21.15.18:0/2125131435 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4c440fde90 0x7f4c44115d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:41:42.195 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:42.193+0000 7f4c3be00700 1 -- 172.21.15.18:0/2125131435 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4c440fde90 msgr2=0x7f4c44115d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:41:42.195 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:42.193+0000 7f4c3be00700 1 --2- 172.21.15.18:0/2125131435 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4c440fde90 0x7f4c44115d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:41:42.195 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:42.193+0000 7f4c3be00700 1 -- 172.21.15.18:0/2125131435 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c2c005040 con 0x7f4c440fe830 2024-04-12T21:41:42.196 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:42.194+0000 7f4c3be00700 1 --2- 172.21.15.18:0/2125131435 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4c440fe830 0x7f4c441162a0 secure :-1 s=READY pgs=1061 cs=0 l=1 rev1=1 crypto rx=0x7f4c2c000c00 tx=0x7f4c2c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:41:42.196 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:42.194+0000 7f4c41600700 1 -- 172.21.15.18:0/2125131435 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4c2c00e040 con 0x7f4c440fe830 2024-04-12T21:41:42.196 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:42.194+0000 7f4c41600700 1 -- 172.21.15.18:0/2125131435 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f4c2c013420 con 0x7f4c440fe830 2024-04-12T21:41:42.196 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:42.194+0000 7f4c4b400700 1 -- 172.21.15.18:0/2125131435 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4c4410ff80 con 0x7f4c440fe830 2024-04-12T21:41:42.196 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:42.194+0000 7f4c41600700 1 -- 172.21.15.18:0/2125131435 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4c2c01c610 con 0x7f4c440fe830 2024-04-12T21:41:42.196 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:42.194+0000 7f4c4b400700 1 -- 172.21.15.18:0/2125131435 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4c441104c0 con 0x7f4c440fe830 2024-04-12T21:41:42.197 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:42.195+0000 7f4c4b400700 1 -- 172.21.15.18:0/2125131435 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4c44106a20 con 0x7f4c440fe830 2024-04-12T21:41:42.197 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:42.197+0000 7f4c41600700 1 -- 172.21.15.18:0/2125131435 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7f4c2c013590 con 0x7f4c440fe830 2024-04-12T21:41:42.198 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:42.197+0000 7f4c41600700 1 --2- 172.21.15.18:0/2125131435 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f4c2806ec10 0x7f4c280710c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:41:42.199 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:42.198+0000 7f4c41600700 1 -- 172.21.15.18:0/2125131435 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 6757+0+0 (secure 0 0 0) 0x7f4c2c08fcf0 con 0x7f4c440fe830 2024-04-12T21:41:42.199 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:42.198+0000 7f4c48a00700 1 --2- 172.21.15.18:0/2125131435 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f4c2806ec10 0x7f4c280710c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:41:42.199 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:42.199+0000 7f4c48a00700 1 --2- 172.21.15.18:0/2125131435 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f4c2806ec10 0x7f4c280710c0 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f4c34004990 tx=0x7f4c34009ee0 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:41:42.204 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:42.203+0000 7f4c41600700 1 -- 172.21.15.18:0/2125131435 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f4c2c0588f0 con 0x7f4c440fe830 2024-04-12T21:41:42.459 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:42.457+0000 7f4c4b400700 1 -- 172.21.15.18:0/2125131435 --> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f4c44111460 con 0x7f4c2806ec10 2024-04-12T21:41:42.461 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:42.460+0000 7f4c41600700 1 -- 172.21.15.18:0/2125131435 <== mgr.34102 v2:172.21.15.18:6800/3812891919 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f4c44111460 con 0x7f4c2806ec10 2024-04-12T21:41:42.464 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:42.463+0000 7f4c4b400700 1 -- 172.21.15.18:0/2125131435 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f4c2806ec10 msgr2=0x7f4c280710c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:41:42.464 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:42.463+0000 7f4c4b400700 1 --2- 172.21.15.18:0/2125131435 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f4c2806ec10 0x7f4c280710c0 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f4c34004990 tx=0x7f4c34009ee0 comp rx=0 tx=0).stop 2024-04-12T21:41:42.464 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:42.463+0000 7f4c4b400700 1 -- 172.21.15.18:0/2125131435 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4c440fe830 msgr2=0x7f4c441162a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:41:42.464 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:42.463+0000 7f4c4b400700 1 --2- 172.21.15.18:0/2125131435 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4c440fe830 0x7f4c441162a0 secure :-1 s=READY pgs=1061 cs=0 l=1 rev1=1 crypto rx=0x7f4c2c000c00 tx=0x7f4c2c009e30 comp rx=0 tx=0).stop 2024-04-12T21:41:42.465 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:42.463+0000 7f4c4b400700 1 -- 172.21.15.18:0/2125131435 shutdown_connections 2024-04-12T21:41:42.465 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:42.463+0000 7f4c4b400700 1 --2- 172.21.15.18:0/2125131435 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f4c2806ec10 0x7f4c280710c0 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:41:42.465 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:42.463+0000 7f4c4b400700 1 --2- 172.21.15.18:0/2125131435 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4c440fde90 0x7f4c44115d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:41:42.465 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:42.463+0000 7f4c4b400700 1 --2- 172.21.15.18:0/2125131435 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4c440fe830 0x7f4c441162a0 unknown :-1 s=CLOSED pgs=1061 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:41:42.465 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:42.463+0000 7f4c4b400700 1 -- 172.21.15.18:0/2125131435 >> 172.21.15.18:0/2125131435 conn(0x7f4c44072bc0 msgr2=0x7f4c440fd530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:41:42.465 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:42.463+0000 7f4c4b400700 1 -- 172.21.15.18:0/2125131435 shutdown_connections 2024-04-12T21:41:42.465 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:42.463+0000 7f4c4b400700 1 -- 172.21.15.18:0/2125131435 wait complete. 2024-04-12T21:41:42.615 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:42.614+0000 7f045fc00700 1 -- 172.21.15.18:0/2709757476 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f04581037a0 msgr2=0x7f04580fce10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:41:42.615 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:42.614+0000 7f045fc00700 1 --2- 172.21.15.18:0/2709757476 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f04581037a0 0x7f04580fce10 secure :-1 s=READY pgs=1062 cs=0 l=1 rev1=1 crypto rx=0x7f0444004940 tx=0x7f0444009e30 comp rx=0 tx=0).stop 2024-04-12T21:41:42.615 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:42.614+0000 7f045fc00700 1 -- 172.21.15.18:0/2709757476 shutdown_connections 2024-04-12T21:41:42.616 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:42.614+0000 7f045fc00700 1 --2- 172.21.15.18:0/2709757476 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f04581037a0 0x7f04580fce10 unknown :-1 s=CLOSED pgs=1062 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:41:42.616 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:42.614+0000 7f045fc00700 1 --2- 172.21.15.18:0/2709757476 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0458102e00 0x7f04581031d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:41:42.616 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:42.614+0000 7f045fc00700 1 -- 172.21.15.18:0/2709757476 >> 172.21.15.18:0/2709757476 conn(0x7f04580f89b0 msgr2=0x7f04580fadc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:41:42.616 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:42.614+0000 7f045fc00700 1 -- 172.21.15.18:0/2709757476 shutdown_connections 2024-04-12T21:41:42.616 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:42.614+0000 7f045fc00700 1 -- 172.21.15.18:0/2709757476 wait complete. 2024-04-12T21:41:42.617 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:42.615+0000 7f045fc00700 1 Processor -- start 2024-04-12T21:41:42.617 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:42.615+0000 7f045fc00700 1 -- start start 2024-04-12T21:41:42.617 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:42.616+0000 7f045fc00700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0458102e00 0x7f0458113980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:41:42.617 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:42.616+0000 7f045fc00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f04581037a0 0x7f0458113ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:41:42.617 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:42.616+0000 7f045fc00700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0458114510 con 0x7f04581037a0 2024-04-12T21:41:42.617 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:42.616+0000 7f045fc00700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f045810db30 con 0x7f0458102e00 2024-04-12T21:41:42.618 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:42.616+0000 7f0457e00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f04581037a0 0x7f0458113ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:41:42.618 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:42.616+0000 7f0457e00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f04581037a0 0x7f0458113ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:58276/0 (socket says 172.21.15.18:58276) 2024-04-12T21:41:42.618 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:42.616+0000 7f0457e00700 1 -- 172.21.15.18:0/932314493 learned_addr learned my addr 172.21.15.18:0/932314493 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:41:42.619 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:42.616+0000 7f045d200700 1 --2- 172.21.15.18:0/932314493 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0458102e00 0x7f0458113980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:41:42.619 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:42.616+0000 7f0457e00700 1 -- 172.21.15.18:0/932314493 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0458102e00 msgr2=0x7f0458113980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:41:42.619 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:42.616+0000 7f0457e00700 1 --2- 172.21.15.18:0/932314493 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0458102e00 0x7f0458113980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:41:42.619 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:42.617+0000 7f0457e00700 1 -- 172.21.15.18:0/932314493 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0444005040 con 0x7f04581037a0 2024-04-12T21:41:42.619 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:42.617+0000 7f0457e00700 1 --2- 172.21.15.18:0/932314493 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f04581037a0 0x7f0458113ec0 secure :-1 s=READY pgs=1063 cs=0 l=1 rev1=1 crypto rx=0x7f0444009100 tx=0x7f0444009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:41:42.619 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:42.617+0000 7f0454c00700 1 -- 172.21.15.18:0/932314493 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f044400e040 con 0x7f04581037a0 2024-04-12T21:41:42.619 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:42.617+0000 7f0454c00700 1 -- 172.21.15.18:0/932314493 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f0444013420 con 0x7f04581037a0 2024-04-12T21:41:42.619 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:42.617+0000 7f0454c00700 1 -- 172.21.15.18:0/932314493 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f044401c630 con 0x7f04581037a0 2024-04-12T21:41:42.619 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:42.617+0000 7f045fc00700 1 -- 172.21.15.18:0/932314493 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f045810ddb0 con 0x7f04581037a0 2024-04-12T21:41:42.619 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:42.617+0000 7f045fc00700 1 -- 172.21.15.18:0/932314493 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f045810e370 con 0x7f04581037a0 2024-04-12T21:41:42.620 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:42.619+0000 7f045fc00700 1 -- 172.21.15.18:0/932314493 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f04581052d0 con 0x7f04581037a0 2024-04-12T21:41:42.620 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:42.620+0000 7f0454c00700 1 -- 172.21.15.18:0/932314493 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7f0444013590 con 0x7f04581037a0 2024-04-12T21:41:42.621 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:42.620+0000 7f0454c00700 1 --2- 172.21.15.18:0/932314493 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f044006ec00 0x7f04400710b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:41:42.621 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:42.621+0000 7f0454c00700 1 -- 172.21.15.18:0/932314493 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 6757+0+0 (secure 0 0 0) 0x7f044408fae0 con 0x7f04581037a0 2024-04-12T21:41:42.622 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:42.621+0000 7f045d200700 1 --2- 172.21.15.18:0/932314493 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f044006ec00 0x7f04400710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:41:42.622 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:42.622+0000 7f045d200700 1 --2- 172.21.15.18:0/932314493 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f044006ec00 0x7f04400710b0 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f044c0041d0 tx=0x7f044c009040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:41:42.626 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:42.625+0000 7f0454c00700 1 -- 172.21.15.18:0/932314493 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f04440588a0 con 0x7f04581037a0 2024-04-12T21:41:42.868 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:42.867+0000 7f045fc00700 1 -- 172.21.15.18:0/932314493 --> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f045810efd0 con 0x7f044006ec00 2024-04-12T21:41:42.897 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:42.896+0000 7f0454c00700 1 -- 172.21.15.18:0/932314493 <== mgr.34102 v2:172.21.15.18:6800/3812891919 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7f045810efd0 con 0x7f044006ec00 2024-04-12T21:41:42.898 INFO:teuthology.orchestra.run.smithi018.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-12T21:41:42.898 INFO:teuthology.orchestra.run.smithi018.stdout:alertmanager.smithi018 smithi018 *:9093,9094 running (42m) 3m ago 56m 14.8M - 0.25.0 c8568f914cd2 c332090068e8 2024-04-12T21:41:42.898 INFO:teuthology.orchestra.run.smithi018.stdout:crash.smithi018 smithi018 running (38m) 3m ago 57m 6665k - 19.0.0-2879-g324baf8d 8717720dc5e5 b7f80d1f7982 2024-04-12T21:41:42.898 INFO:teuthology.orchestra.run.smithi018.stdout:crash.smithi186 smithi186 running (38m) 3m ago 55m 6660k - 19.0.0-2879-g324baf8d 8717720dc5e5 a4f55c81b623 2024-04-12T21:41:42.898 INFO:teuthology.orchestra.run.smithi018.stdout:grafana.smithi018 smithi018 *:3000 running (41m) 3m ago 56m 73.8M - 9.4.12 2bacad6d85d8 b08e3bf176dd 2024-04-12T21:41:42.898 INFO:teuthology.orchestra.run.smithi018.stdout:mds.cephfs.smithi018.acqbsc smithi018 running (34m) 3m ago 49m 21.9M - 19.0.0-2879-g324baf8d 8717720dc5e5 021f99ab24ae 2024-04-12T21:41:42.898 INFO:teuthology.orchestra.run.smithi018.stdout:mds.cephfs.smithi018.szjluo smithi018 running (34m) 3m ago 49m 21.7M - 19.0.0-2879-g324baf8d 8717720dc5e5 0003117479d4 2024-04-12T21:41:42.898 INFO:teuthology.orchestra.run.smithi018.stdout:mds.cephfs.smithi186.jdllyn smithi186 running (18m) 3m ago 49m 16.5M - 19.0.0-2879-g324baf8d 8717720dc5e5 21b1f46eef23 2024-04-12T21:41:42.898 INFO:teuthology.orchestra.run.smithi018.stdout:mds.cephfs.smithi186.wjnafs smithi186 running (3m) 3m ago 49m 11.0M - 19.0.0-2879-g324baf8d 8717720dc5e5 3cf1ef4148e2 2024-04-12T21:41:42.898 INFO:teuthology.orchestra.run.smithi018.stdout:mgr.smithi018.xjzwga smithi018 *:8443,9283,8765 running (44m) 3m ago 58m 517M - 19.0.0-2879-g324baf8d 8717720dc5e5 56088305eab7 2024-04-12T21:41:42.898 INFO:teuthology.orchestra.run.smithi018.stdout:mgr.smithi186.kqcfxk smithi186 *:8443,9283,8765 running (43m) 3m ago 55m 460M - 19.0.0-2879-g324baf8d 8717720dc5e5 09cd766e5e0d 2024-04-12T21:41:42.898 INFO:teuthology.orchestra.run.smithi018.stdout:mon.smithi018 smithi018 running (40m) 3m ago 58m 88.2M 2048M 19.0.0-2879-g324baf8d 8717720dc5e5 df3e201dfed6 2024-04-12T21:41:42.899 INFO:teuthology.orchestra.run.smithi018.stdout:mon.smithi186 smithi186 running (39m) 3m ago 55m 53.6M 2048M 19.0.0-2879-g324baf8d 8717720dc5e5 c77498da9b2b 2024-04-12T21:41:42.899 INFO:teuthology.orchestra.run.smithi018.stdout:node-exporter.smithi018 smithi018 *:9100 running (43m) 3m ago 56m 8468k - 1.5.0 0da6a335fe13 326c3d9c6847 2024-04-12T21:41:42.899 INFO:teuthology.orchestra.run.smithi018.stdout:node-exporter.smithi186 smithi186 *:9100 running (43m) 3m ago 55m 8166k - 1.5.0 0da6a335fe13 ec2112baeecf 2024-04-12T21:41:42.899 INFO:teuthology.orchestra.run.smithi018.stdout:osd.0 smithi018 running (37m) 3m ago 54m 123M 1559M 19.0.0-2879-g324baf8d 8717720dc5e5 60b1273ca047 2024-04-12T21:41:42.899 INFO:teuthology.orchestra.run.smithi018.stdout:osd.1 smithi018 running (37m) 3m ago 53m 95.4M 1559M 19.0.0-2879-g324baf8d 8717720dc5e5 170ce1e5fb8d 2024-04-12T21:41:42.899 INFO:teuthology.orchestra.run.smithi018.stdout:osd.2 smithi018 running (36m) 3m ago 53m 94.6M 1559M 19.0.0-2879-g324baf8d 8717720dc5e5 c2cbc6aa2839 2024-04-12T21:41:42.899 INFO:teuthology.orchestra.run.smithi018.stdout:osd.3 smithi186 running (36m) 3m ago 52m 101M 2581M 19.0.0-2879-g324baf8d 8717720dc5e5 0138e8262e4b 2024-04-12T21:41:42.899 INFO:teuthology.orchestra.run.smithi018.stdout:osd.4 smithi186 running (35m) 3m ago 52m 91.5M 2581M 19.0.0-2879-g324baf8d 8717720dc5e5 b20cb63b25c9 2024-04-12T21:41:42.899 INFO:teuthology.orchestra.run.smithi018.stdout:osd.5 smithi186 running (34m) 3m ago 51m 92.2M 2581M 19.0.0-2879-g324baf8d 8717720dc5e5 2513bea2610e 2024-04-12T21:41:42.899 INFO:teuthology.orchestra.run.smithi018.stdout:prometheus.smithi018 smithi018 *:9095 running (42m) 3m ago 56m 69.6M - 2.43.0 a07b618ecd1d f58841127e21 2024-04-12T21:41:42.900 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:42.899+0000 7f045fc00700 1 -- 172.21.15.18:0/932314493 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f044006ec00 msgr2=0x7f04400710b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:41:42.900 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:42.899+0000 7f045fc00700 1 --2- 172.21.15.18:0/932314493 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f044006ec00 0x7f04400710b0 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f044c0041d0 tx=0x7f044c009040 comp rx=0 tx=0).stop 2024-04-12T21:41:42.901 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:42.899+0000 7f045fc00700 1 -- 172.21.15.18:0/932314493 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f04581037a0 msgr2=0x7f0458113ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:41:42.901 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:42.899+0000 7f045fc00700 1 --2- 172.21.15.18:0/932314493 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f04581037a0 0x7f0458113ec0 secure :-1 s=READY pgs=1063 cs=0 l=1 rev1=1 crypto rx=0x7f0444009100 tx=0x7f0444009e30 comp rx=0 tx=0).stop 2024-04-12T21:41:42.901 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:42.899+0000 7f045fc00700 1 -- 172.21.15.18:0/932314493 shutdown_connections 2024-04-12T21:41:42.901 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:42.899+0000 7f045fc00700 1 --2- 172.21.15.18:0/932314493 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f044006ec00 0x7f04400710b0 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:41:42.901 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:42.899+0000 7f045fc00700 1 --2- 172.21.15.18:0/932314493 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0458102e00 0x7f0458113980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:41:42.902 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:42.899+0000 7f045fc00700 1 --2- 172.21.15.18:0/932314493 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f04581037a0 0x7f0458113ec0 unknown :-1 s=CLOSED pgs=1063 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:41:42.902 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:42.899+0000 7f045fc00700 1 -- 172.21.15.18:0/932314493 >> 172.21.15.18:0/932314493 conn(0x7f04580f89b0 msgr2=0x7f04581021c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:41:42.902 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:42.900+0000 7f045fc00700 1 -- 172.21.15.18:0/932314493 shutdown_connections 2024-04-12T21:41:42.902 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:42.900+0000 7f045fc00700 1 -- 172.21.15.18:0/932314493 wait complete. 2024-04-12T21:41:43.044 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:43.043+0000 7fe6e4600700 1 -- 172.21.15.18:0/1751713881 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe6dc0fe830 msgr2=0x7fe6dc106f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:41:43.045 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:43.043+0000 7fe6e4600700 1 --2- 172.21.15.18:0/1751713881 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe6dc0fe830 0x7fe6dc106f00 secure :-1 s=READY pgs=1064 cs=0 l=1 rev1=1 crypto rx=0x7fe6c8004940 tx=0x7fe6c8009e30 comp rx=0 tx=0).stop 2024-04-12T21:41:43.045 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:43.043+0000 7fe6e4600700 1 -- 172.21.15.18:0/1751713881 shutdown_connections 2024-04-12T21:41:43.045 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:43.043+0000 7fe6e4600700 1 --2- 172.21.15.18:0/1751713881 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe6dc0fe830 0x7fe6dc106f00 unknown :-1 s=CLOSED pgs=1064 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:41:43.045 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:43.043+0000 7fe6e4600700 1 --2- 172.21.15.18:0/1751713881 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe6dc0fde90 0x7fe6dc0fe260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:41:43.045 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:43.043+0000 7fe6e4600700 1 -- 172.21.15.18:0/1751713881 >> 172.21.15.18:0/1751713881 conn(0x7fe6dc072bc0 msgr2=0x7fe6dc072fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:41:43.045 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:43.043+0000 7fe6e4600700 1 -- 172.21.15.18:0/1751713881 shutdown_connections 2024-04-12T21:41:43.045 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:43.043+0000 7fe6e4600700 1 -- 172.21.15.18:0/1751713881 wait complete. 2024-04-12T21:41:43.047 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:43.044+0000 7fe6e4600700 1 Processor -- start 2024-04-12T21:41:43.047 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:43.044+0000 7fe6e4600700 1 -- start start 2024-04-12T21:41:43.047 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:43.045+0000 7fe6e4600700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe6dc0fde90 0x7fe6dc115da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:41:43.047 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:43.045+0000 7fe6e4600700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe6dc0fe830 0x7fe6dc1162e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:41:43.047 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:43.045+0000 7fe6e4600700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6dc116930 con 0x7fe6dc0fe830 2024-04-12T21:41:43.047 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:43.045+0000 7fe6e4600700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6dc10fd00 con 0x7fe6dc0fde90 2024-04-12T21:41:43.047 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:43.045+0000 7fe6e1200700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe6dc0fe830 0x7fe6dc1162e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:41:43.047 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:43.045+0000 7fe6e1200700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe6dc0fe830 0x7fe6dc1162e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:58280/0 (socket says 172.21.15.18:58280) 2024-04-12T21:41:43.048 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:43.045+0000 7fe6e1200700 1 -- 172.21.15.18:0/2533388885 learned_addr learned my addr 172.21.15.18:0/2533388885 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:41:43.048 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:43.045+0000 7fe6e1c00700 1 --2- 172.21.15.18:0/2533388885 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe6dc0fde90 0x7fe6dc115da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:41:43.048 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:43.046+0000 7fe6e1200700 1 -- 172.21.15.18:0/2533388885 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe6dc0fde90 msgr2=0x7fe6dc115da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:41:43.048 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:43.046+0000 7fe6e1200700 1 --2- 172.21.15.18:0/2533388885 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe6dc0fde90 0x7fe6dc115da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:41:43.048 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:43.046+0000 7fe6e1200700 1 -- 172.21.15.18:0/2533388885 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6c8005040 con 0x7fe6dc0fe830 2024-04-12T21:41:43.048 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:43.046+0000 7fe6e1c00700 1 --2- 172.21.15.18:0/2533388885 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe6dc0fde90 0x7fe6dc115da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-12T21:41:43.048 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:43.046+0000 7fe6e1200700 1 --2- 172.21.15.18:0/2533388885 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe6dc0fe830 0x7fe6dc1162e0 secure :-1 s=READY pgs=1065 cs=0 l=1 rev1=1 crypto rx=0x7fe6c80093b0 tx=0x7fe6c8009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:41:43.048 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:43.046+0000 7fe6d9600700 1 -- 172.21.15.18:0/2533388885 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe6c800e040 con 0x7fe6dc0fe830 2024-04-12T21:41:43.048 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:43.046+0000 7fe6d9600700 1 -- 172.21.15.18:0/2533388885 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe6c8004290 con 0x7fe6dc0fe830 2024-04-12T21:41:43.048 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:43.046+0000 7fe6d9600700 1 -- 172.21.15.18:0/2533388885 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe6c801c3f0 con 0x7fe6dc0fe830 2024-04-12T21:41:43.048 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:43.046+0000 7fe6e4600700 1 -- 172.21.15.18:0/2533388885 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe6dc10ff80 con 0x7fe6dc0fe830 2024-04-12T21:41:43.049 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:43.046+0000 7fe6e4600700 1 -- 172.21.15.18:0/2533388885 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe6dc1104c0 con 0x7fe6dc0fe830 2024-04-12T21:41:43.049 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:43.047+0000 7fe6e4600700 1 -- 172.21.15.18:0/2533388885 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe6dc106a20 con 0x7fe6dc0fe830 2024-04-12T21:41:43.049 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:43.048+0000 7fe6d9600700 1 -- 172.21.15.18:0/2533388885 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7fe6c8013890 con 0x7fe6dc0fe830 2024-04-12T21:41:43.050 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:43.049+0000 7fe6d9600700 1 --2- 172.21.15.18:0/2533388885 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fe6c406ec10 0x7fe6c40710c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:41:43.050 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:43.049+0000 7fe6d9600700 1 -- 172.21.15.18:0/2533388885 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 6757+0+0 (secure 0 0 0) 0x7fe6c808fc10 con 0x7fe6dc0fe830 2024-04-12T21:41:43.051 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:43.050+0000 7fe6e1c00700 1 --2- 172.21.15.18:0/2533388885 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fe6c406ec10 0x7fe6c40710c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:41:43.051 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:43.050+0000 7fe6e1c00700 1 --2- 172.21.15.18:0/2533388885 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fe6c406ec10 0x7fe6c40710c0 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7fe6d0009fd0 tx=0x7fe6d000b040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:41:43.055 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:43.054+0000 7fe6d9600700 1 -- 172.21.15.18:0/2533388885 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7fe6c805c440 con 0x7fe6dc0fe830 2024-04-12T21:41:43.368 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:43.367+0000 7fe6e4600700 1 -- 172.21.15.18:0/2533388885 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fe6dc110110 con 0x7fe6dc0fe830 2024-04-12T21:41:43.369 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:43.368+0000 7fe6d9600700 1 -- 172.21.15.18:0/2533388885 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+623 (secure 0 0 0) 0x7fe6dc110110 con 0x7fe6dc0fe830 2024-04-12T21:41:43.369 INFO:teuthology.orchestra.run.smithi018.stdout:{ 2024-04-12T21:41:43.369 INFO:teuthology.orchestra.run.smithi018.stdout: "mon": { 2024-04-12T21:41:43.369 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 19.0.0-2879-g324baf8d (324baf8ddb91dea46cac50cb535c306115791355) squid (dev)": 2 2024-04-12T21:41:43.370 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-12T21:41:43.370 INFO:teuthology.orchestra.run.smithi018.stdout: "mgr": { 2024-04-12T21:41:43.370 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 19.0.0-2879-g324baf8d (324baf8ddb91dea46cac50cb535c306115791355) squid (dev)": 2 2024-04-12T21:41:43.370 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-12T21:41:43.370 INFO:teuthology.orchestra.run.smithi018.stdout: "osd": { 2024-04-12T21:41:43.370 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 19.0.0-2879-g324baf8d (324baf8ddb91dea46cac50cb535c306115791355) squid (dev)": 6 2024-04-12T21:41:43.370 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-12T21:41:43.370 INFO:teuthology.orchestra.run.smithi018.stdout: "mds": { 2024-04-12T21:41:43.370 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 19.0.0-2879-g324baf8d (324baf8ddb91dea46cac50cb535c306115791355) squid (dev)": 4 2024-04-12T21:41:43.370 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-12T21:41:43.370 INFO:teuthology.orchestra.run.smithi018.stdout: "overall": { 2024-04-12T21:41:43.370 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 19.0.0-2879-g324baf8d (324baf8ddb91dea46cac50cb535c306115791355) squid (dev)": 14 2024-04-12T21:41:43.370 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-04-12T21:41:43.371 INFO:teuthology.orchestra.run.smithi018.stdout:} 2024-04-12T21:41:43.373 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:43.371+0000 7fe6e4600700 1 -- 172.21.15.18:0/2533388885 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fe6c406ec10 msgr2=0x7fe6c40710c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:41:43.373 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:43.371+0000 7fe6e4600700 1 --2- 172.21.15.18:0/2533388885 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fe6c406ec10 0x7fe6c40710c0 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7fe6d0009fd0 tx=0x7fe6d000b040 comp rx=0 tx=0).stop 2024-04-12T21:41:43.373 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:43.371+0000 7fe6e4600700 1 -- 172.21.15.18:0/2533388885 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe6dc0fe830 msgr2=0x7fe6dc1162e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:41:43.373 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:43.371+0000 7fe6e4600700 1 --2- 172.21.15.18:0/2533388885 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe6dc0fe830 0x7fe6dc1162e0 secure :-1 s=READY pgs=1065 cs=0 l=1 rev1=1 crypto rx=0x7fe6c80093b0 tx=0x7fe6c8009e30 comp rx=0 tx=0).stop 2024-04-12T21:41:43.373 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:43.371+0000 7fe6e4600700 1 -- 172.21.15.18:0/2533388885 shutdown_connections 2024-04-12T21:41:43.373 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:43.371+0000 7fe6e4600700 1 --2- 172.21.15.18:0/2533388885 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fe6c406ec10 0x7fe6c40710c0 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:41:43.373 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:43.371+0000 7fe6e4600700 1 --2- 172.21.15.18:0/2533388885 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe6dc0fde90 0x7fe6dc115da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:41:43.373 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:43.371+0000 7fe6e4600700 1 --2- 172.21.15.18:0/2533388885 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe6dc0fe830 0x7fe6dc1162e0 unknown :-1 s=CLOSED pgs=1065 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:41:43.373 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:43.371+0000 7fe6e4600700 1 -- 172.21.15.18:0/2533388885 >> 172.21.15.18:0/2533388885 conn(0x7fe6dc072bc0 msgr2=0x7fe6dc0fb6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:41:43.373 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:43.372+0000 7fe6e4600700 1 -- 172.21.15.18:0/2533388885 shutdown_connections 2024-04-12T21:41:43.373 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:43.372+0000 7fe6e4600700 1 -- 172.21.15.18:0/2533388885 wait complete. 2024-04-12T21:41:43.529 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:43.527+0000 7f4d07200700 1 -- 172.21.15.18:0/1557969010 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4d08109c40 msgr2=0x7f4d081032b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:41:43.529 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:43.527+0000 7f4d07200700 1 --2- 172.21.15.18:0/1557969010 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4d08109c40 0x7f4d081032b0 secure :-1 s=READY pgs=1066 cs=0 l=1 rev1=1 crypto rx=0x7f4cf4004940 tx=0x7f4cf4009e30 comp rx=0 tx=0).stop 2024-04-12T21:41:43.529 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:43.527+0000 7f4d07200700 1 -- 172.21.15.18:0/1557969010 shutdown_connections 2024-04-12T21:41:43.529 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:43.527+0000 7f4d07200700 1 --2- 172.21.15.18:0/1557969010 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4d08109c40 0x7f4d081032b0 unknown :-1 s=CLOSED pgs=1066 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:41:43.529 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:43.527+0000 7f4d07200700 1 --2- 172.21.15.18:0/1557969010 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4d081092a0 0x7f4d08109670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:41:43.529 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:43.527+0000 7f4d07200700 1 -- 172.21.15.18:0/1557969010 >> 172.21.15.18:0/1557969010 conn(0x7f4d08072a30 msgr2=0x7f4d08074e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:41:43.529 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:43.528+0000 7f4d07200700 1 -- 172.21.15.18:0/1557969010 shutdown_connections 2024-04-12T21:41:43.530 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:43.528+0000 7f4d07200700 1 -- 172.21.15.18:0/1557969010 wait complete. 2024-04-12T21:41:43.530 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:41:43 smithi018 ceph-mon[112590]: from='client.36188 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:41:43.530 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:41:43 smithi018 ceph-mon[112590]: pgmap v1261: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:41:43.530 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:41:43 smithi018 ceph-mon[112590]: from='client.36192 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:41:43.530 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:41:43 smithi018 ceph-mon[112590]: from='client.36196 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:41:43.532 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:43.528+0000 7f4d07200700 1 Processor -- start 2024-04-12T21:41:43.532 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:43.529+0000 7f4d07200700 1 -- start start 2024-04-12T21:41:43.532 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:43.529+0000 7f4d07200700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4d081092a0 0x7f4d08119fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:41:43.532 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:43.529+0000 7f4d07200700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4d08109c40 0x7f4d0811a500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:41:43.532 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:43.529+0000 7f4d07200700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d0811ab50 con 0x7f4d08109c40 2024-04-12T21:41:43.532 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:43.529+0000 7f4d07200700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d08113f20 con 0x7f4d081092a0 2024-04-12T21:41:43.532 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:43.529+0000 7f4d05400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4d08109c40 0x7f4d0811a500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:41:43.532 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:43.529+0000 7f4d05400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4d08109c40 0x7f4d0811a500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:58290/0 (socket says 172.21.15.18:58290) 2024-04-12T21:41:43.532 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:43.529+0000 7f4d05400700 1 -- 172.21.15.18:0/1034420739 learned_addr learned my addr 172.21.15.18:0/1034420739 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:41:43.533 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:43.530+0000 7f4d05e00700 1 --2- 172.21.15.18:0/1034420739 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4d081092a0 0x7f4d08119fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:41:43.533 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:43.530+0000 7f4d05400700 1 -- 172.21.15.18:0/1034420739 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4d081092a0 msgr2=0x7f4d08119fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:41:43.533 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:43.530+0000 7f4d05400700 1 --2- 172.21.15.18:0/1034420739 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4d081092a0 0x7f4d08119fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:41:43.533 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:43.530+0000 7f4d05400700 1 -- 172.21.15.18:0/1034420739 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4cf4005040 con 0x7f4d08109c40 2024-04-12T21:41:43.533 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:43.530+0000 7f4d05e00700 1 --2- 172.21.15.18:0/1034420739 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4d081092a0 0x7f4d08119fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-12T21:41:43.533 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:43.530+0000 7f4d05400700 1 --2- 172.21.15.18:0/1034420739 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4d08109c40 0x7f4d0811a500 secure :-1 s=READY pgs=1067 cs=0 l=1 rev1=1 crypto rx=0x7f4cf4009100 tx=0x7f4cf4009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:41:43.533 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:43.530+0000 7f4cfe000700 1 -- 172.21.15.18:0/1034420739 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4cf400e040 con 0x7f4d08109c40 2024-04-12T21:41:43.533 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:43.531+0000 7f4cfe000700 1 -- 172.21.15.18:0/1034420739 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f4cf4004290 con 0x7f4d08109c40 2024-04-12T21:41:43.533 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:43.531+0000 7f4cfe000700 1 -- 172.21.15.18:0/1034420739 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4cf401c3f0 con 0x7f4d08109c40 2024-04-12T21:41:43.533 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:43.531+0000 7f4d07200700 1 -- 172.21.15.18:0/1034420739 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4d081141a0 con 0x7f4d08109c40 2024-04-12T21:41:43.533 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:43.531+0000 7f4d07200700 1 -- 172.21.15.18:0/1034420739 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4d08114760 con 0x7f4d08109c40 2024-04-12T21:41:43.534 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:41:43 smithi186 ceph-mon[86379]: from='client.36188 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:41:43.534 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:41:43 smithi186 ceph-mon[86379]: pgmap v1261: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:41:43.534 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:41:43 smithi186 ceph-mon[86379]: from='client.36192 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:41:43.534 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:41:43 smithi186 ceph-mon[86379]: from='client.36196 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:41:43.535 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:43.532+0000 7f4d07200700 1 -- 172.21.15.18:0/1034420739 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4d081049f0 con 0x7f4d08109c40 2024-04-12T21:41:43.535 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:43.533+0000 7f4cfe000700 1 -- 172.21.15.18:0/1034420739 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7f4cf4013790 con 0x7f4d08109c40 2024-04-12T21:41:43.535 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:43.534+0000 7f4cfe000700 1 --2- 172.21.15.18:0/1034420739 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f4cec06ec00 0x7f4cec0710b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:41:43.535 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:43.534+0000 7f4cfe000700 1 -- 172.21.15.18:0/1034420739 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 6757+0+0 (secure 0 0 0) 0x7f4cf408f8c0 con 0x7f4d08109c40 2024-04-12T21:41:43.536 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:43.534+0000 7f4d05e00700 1 --2- 172.21.15.18:0/1034420739 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f4cec06ec00 0x7f4cec0710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:41:43.536 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:43.535+0000 7f4d05e00700 1 --2- 172.21.15.18:0/1034420739 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f4cec06ec00 0x7f4cec0710b0 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f4cf8007fd0 tx=0x7f4cf8009040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:41:43.539 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:43.539+0000 7f4cfe000700 1 -- 172.21.15.18:0/1034420739 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f4cf405c1b0 con 0x7f4d08109c40 2024-04-12T21:41:43.819 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:43.818+0000 7f4d07200700 1 -- 172.21.15.18:0/1034420739 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f4d081103c0 con 0x7f4d08109c40 2024-04-12T21:41:43.820 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:43.819+0000 7f4cfe000700 1 -- 172.21.15.18:0/1034420739 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 33 v33) v1 ==== 76+0+1876 (secure 0 0 0) 0x7f4cf4013420 con 0x7f4d08109c40 2024-04-12T21:41:43.821 INFO:teuthology.orchestra.run.smithi018.stdout:e33 2024-04-12T21:41:43.821 INFO:teuthology.orchestra.run.smithi018.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-12T21:41:43.821 INFO:teuthology.orchestra.run.smithi018.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-12T21:41:43.821 INFO:teuthology.orchestra.run.smithi018.stdout:legacy client fscid: 1 2024-04-12T21:41:43.821 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-12T21:41:43.821 INFO:teuthology.orchestra.run.smithi018.stdout:Filesystem 'cephfs' (1) 2024-04-12T21:41:43.821 INFO:teuthology.orchestra.run.smithi018.stdout:fs_name cephfs 2024-04-12T21:41:43.821 INFO:teuthology.orchestra.run.smithi018.stdout:epoch 24 2024-04-12T21:41:43.821 INFO:teuthology.orchestra.run.smithi018.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-04-12T21:41:43.822 INFO:teuthology.orchestra.run.smithi018.stdout:created 2024-04-12T20:51:57.362850+0000 2024-04-12T21:41:43.822 INFO:teuthology.orchestra.run.smithi018.stdout:modified 2024-04-12T21:06:59.717190+0000 2024-04-12T21:41:43.822 INFO:teuthology.orchestra.run.smithi018.stdout:tableserver 0 2024-04-12T21:41:43.822 INFO:teuthology.orchestra.run.smithi018.stdout:root 0 2024-04-12T21:41:43.822 INFO:teuthology.orchestra.run.smithi018.stdout:session_timeout 60 2024-04-12T21:41:43.822 INFO:teuthology.orchestra.run.smithi018.stdout:session_autoclose 300 2024-04-12T21:41:43.822 INFO:teuthology.orchestra.run.smithi018.stdout:max_file_size 1099511627776 2024-04-12T21:41:43.822 INFO:teuthology.orchestra.run.smithi018.stdout:max_xattr_size 65536 2024-04-12T21:41:43.822 INFO:teuthology.orchestra.run.smithi018.stdout:required_client_features {} 2024-04-12T21:41:43.822 INFO:teuthology.orchestra.run.smithi018.stdout:last_failure 0 2024-04-12T21:41:43.822 INFO:teuthology.orchestra.run.smithi018.stdout:last_failure_osd_epoch 100 2024-04-12T21:41:43.822 INFO:teuthology.orchestra.run.smithi018.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-12T21:41:43.822 INFO:teuthology.orchestra.run.smithi018.stdout:max_mds 2 2024-04-12T21:41:43.822 INFO:teuthology.orchestra.run.smithi018.stdout:in 0,1 2024-04-12T21:41:43.823 INFO:teuthology.orchestra.run.smithi018.stdout:up {} 2024-04-12T21:41:43.823 INFO:teuthology.orchestra.run.smithi018.stdout:failed 0,1 2024-04-12T21:41:43.823 INFO:teuthology.orchestra.run.smithi018.stdout:damaged 2024-04-12T21:41:43.823 INFO:teuthology.orchestra.run.smithi018.stdout:stopped 2024-04-12T21:41:43.823 INFO:teuthology.orchestra.run.smithi018.stdout:data_pools [3] 2024-04-12T21:41:43.823 INFO:teuthology.orchestra.run.smithi018.stdout:metadata_pool 2 2024-04-12T21:41:43.823 INFO:teuthology.orchestra.run.smithi018.stdout:inline_data disabled 2024-04-12T21:41:43.823 INFO:teuthology.orchestra.run.smithi018.stdout:balancer 2024-04-12T21:41:43.823 INFO:teuthology.orchestra.run.smithi018.stdout:bal_rank_mask -1 2024-04-12T21:41:43.823 INFO:teuthology.orchestra.run.smithi018.stdout:standby_count_wanted 1 2024-04-12T21:41:43.823 INFO:teuthology.orchestra.run.smithi018.stdout:qdb_cluster leader: 0 members: 2024-04-12T21:41:43.823 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-12T21:41:43.823 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-12T21:41:43.823 INFO:teuthology.orchestra.run.smithi018.stdout:Standby daemons: 2024-04-12T21:41:43.823 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-12T21:41:43.824 INFO:teuthology.orchestra.run.smithi018.stdout:[mds.cephfs.smithi018.acqbsc{-1:34436} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.18:6828/755179764,v1:172.21.15.18:6829/755179764] compat {c=[1],r=[1],i=[1fff]}] 2024-04-12T21:41:43.824 INFO:teuthology.orchestra.run.smithi018.stdout:[mds.cephfs.smithi018.szjluo{-1:34468} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.18:6826/2407992653,v1:172.21.15.18:6827/2407992653] compat {c=[1],r=[1],i=[1fff]}] 2024-04-12T21:41:43.824 INFO:teuthology.orchestra.run.smithi018.stdout:[mds.cephfs.smithi186.jdllyn{-1:44595} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.186:6824/3523444202,v1:172.21.15.186:6825/3523444202] compat {c=[1],r=[1],i=[1fff]}] 2024-04-12T21:41:43.824 INFO:teuthology.orchestra.run.smithi018.stdout:[mds.cephfs.smithi186.wjnafs{-1:44875} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.186:6826/3122443766,v1:172.21.15.186:6827/3122443766] compat {c=[1],r=[1],i=[1fff]}] 2024-04-12T21:41:43.824 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:43.823+0000 7f4d07200700 1 -- 172.21.15.18:0/1034420739 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f4cec06ec00 msgr2=0x7f4cec0710b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:41:43.825 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:43.823+0000 7f4d07200700 1 --2- 172.21.15.18:0/1034420739 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f4cec06ec00 0x7f4cec0710b0 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f4cf8007fd0 tx=0x7f4cf8009040 comp rx=0 tx=0).stop 2024-04-12T21:41:43.825 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:43.824+0000 7f4d07200700 1 -- 172.21.15.18:0/1034420739 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4d08109c40 msgr2=0x7f4d0811a500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:41:43.825 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:43.824+0000 7f4d07200700 1 --2- 172.21.15.18:0/1034420739 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4d08109c40 0x7f4d0811a500 secure :-1 s=READY pgs=1067 cs=0 l=1 rev1=1 crypto rx=0x7f4cf4009100 tx=0x7f4cf4009e30 comp rx=0 tx=0).stop 2024-04-12T21:41:43.825 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:43.824+0000 7f4d07200700 1 -- 172.21.15.18:0/1034420739 shutdown_connections 2024-04-12T21:41:43.825 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:43.824+0000 7f4d07200700 1 --2- 172.21.15.18:0/1034420739 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f4cec06ec00 0x7f4cec0710b0 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:41:43.826 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:43.824+0000 7f4d07200700 1 --2- 172.21.15.18:0/1034420739 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4d081092a0 0x7f4d08119fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:41:43.826 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:43.824+0000 7f4d07200700 1 --2- 172.21.15.18:0/1034420739 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4d08109c40 0x7f4d0811a500 unknown :-1 s=CLOSED pgs=1067 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:41:43.826 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:43.824+0000 7f4d07200700 1 -- 172.21.15.18:0/1034420739 >> 172.21.15.18:0/1034420739 conn(0x7f4d08072a30 msgr2=0x7f4d08074910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:41:43.826 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:43.824+0000 7f4d07200700 1 -- 172.21.15.18:0/1034420739 shutdown_connections 2024-04-12T21:41:43.826 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:43.824+0000 7f4d07200700 1 -- 172.21.15.18:0/1034420739 wait complete. 2024-04-12T21:41:43.827 INFO:teuthology.orchestra.run.smithi018.stderr:dumped fsmap epoch 33 2024-04-12T21:41:43.989 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:43.987+0000 7f908e000700 1 -- 172.21.15.18:0/399859457 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9088102ca0 msgr2=0x7f9088103070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:41:43.989 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:43.987+0000 7f908e000700 1 --2- 172.21.15.18:0/399859457 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9088102ca0 0x7f9088103070 secure :-1 s=READY pgs=1068 cs=0 l=1 rev1=1 crypto rx=0x7f907c004940 tx=0x7f907c009e30 comp rx=0 tx=0).stop 2024-04-12T21:41:43.989 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:43.988+0000 7f908e000700 1 -- 172.21.15.18:0/399859457 shutdown_connections 2024-04-12T21:41:43.989 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:43.988+0000 7f908e000700 1 --2- 172.21.15.18:0/399859457 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9088065630 0x7f9088065a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:41:43.989 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:43.988+0000 7f908e000700 1 --2- 172.21.15.18:0/399859457 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9088102ca0 0x7f9088103070 unknown :-1 s=CLOSED pgs=1068 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:41:43.990 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:43.988+0000 7f908e000700 1 -- 172.21.15.18:0/399859457 >> 172.21.15.18:0/399859457 conn(0x7f9088071b70 msgr2=0x7f9088071f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:41:43.990 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:43.988+0000 7f908e000700 1 -- 172.21.15.18:0/399859457 shutdown_connections 2024-04-12T21:41:43.990 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:43.988+0000 7f908e000700 1 -- 172.21.15.18:0/399859457 wait complete. 2024-04-12T21:41:43.991 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:43.989+0000 7f908e000700 1 Processor -- start 2024-04-12T21:41:43.991 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:43.989+0000 7f908e000700 1 -- start start 2024-04-12T21:41:43.991 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:43.989+0000 7f908e000700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9088065630 0x7f9088115c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:41:43.992 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:43.989+0000 7f908e000700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9088102ca0 0x7f9088116180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:41:43.992 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:43.989+0000 7f908e000700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9088116860 con 0x7f9088065630 2024-04-12T21:41:43.992 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:43.989+0000 7f908e000700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f908810fba0 con 0x7f9088102ca0 2024-04-12T21:41:43.992 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:43.990+0000 7f908cc00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9088065630 0x7f9088115c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:41:43.992 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:43.990+0000 7f908cc00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9088065630 0x7f9088115c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:58318/0 (socket says 172.21.15.18:58318) 2024-04-12T21:41:43.992 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:43.990+0000 7f908cc00700 1 -- 172.21.15.18:0/131403296 learned_addr learned my addr 172.21.15.18:0/131403296 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:41:43.992 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:43.990+0000 7f908cc00700 1 -- 172.21.15.18:0/131403296 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9088102ca0 msgr2=0x7f9088116180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:41:43.992 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:43.990+0000 7f9087e00700 1 --2- 172.21.15.18:0/131403296 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9088102ca0 0x7f9088116180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:41:43.992 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:43.990+0000 7f908cc00700 1 --2- 172.21.15.18:0/131403296 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9088102ca0 0x7f9088116180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:41:43.992 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:43.990+0000 7f908cc00700 1 -- 172.21.15.18:0/131403296 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f907c005040 con 0x7f9088065630 2024-04-12T21:41:43.992 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:43.990+0000 7f9087e00700 1 --2- 172.21.15.18:0/131403296 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9088102ca0 0x7f9088116180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-12T21:41:43.992 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:43.990+0000 7f908cc00700 1 --2- 172.21.15.18:0/131403296 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9088065630 0x7f9088115c40 secure :-1 s=READY pgs=1069 cs=0 l=1 rev1=1 crypto rx=0x7f907c00b8b0 tx=0x7f907c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:41:43.993 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:43.991+0000 7f9084c00700 1 -- 172.21.15.18:0/131403296 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f907c00e040 con 0x7f9088065630 2024-04-12T21:41:43.993 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:43.991+0000 7f9084c00700 1 -- 172.21.15.18:0/131403296 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f907c004290 con 0x7f9088065630 2024-04-12T21:41:43.994 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:43.991+0000 7f908e000700 1 -- 172.21.15.18:0/131403296 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f908810fdc0 con 0x7f9088065630 2024-04-12T21:41:43.994 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:43.991+0000 7f9084c00700 1 -- 172.21.15.18:0/131403296 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f907c01c3f0 con 0x7f9088065630 2024-04-12T21:41:43.994 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:43.991+0000 7f908e000700 1 -- 172.21.15.18:0/131403296 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9088110300 con 0x7f9088065630 2024-04-12T21:41:43.994 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:43.992+0000 7f908e000700 1 -- 172.21.15.18:0/131403296 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f90880fe380 con 0x7f9088065630 2024-04-12T21:41:43.994 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:43.993+0000 7f9084c00700 1 -- 172.21.15.18:0/131403296 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7f907c0137b0 con 0x7f9088065630 2024-04-12T21:41:43.995 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:43.994+0000 7f9084c00700 1 --2- 172.21.15.18:0/131403296 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f907006ec00 0x7f90700710b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:41:43.995 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:43.994+0000 7f9084c00700 1 -- 172.21.15.18:0/131403296 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 6757+0+0 (secure 0 0 0) 0x7f907c08fd80 con 0x7f9088065630 2024-04-12T21:41:43.995 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:43.994+0000 7f9087e00700 1 --2- 172.21.15.18:0/131403296 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f907006ec00 0x7f90700710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:41:43.996 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:43.995+0000 7f9087e00700 1 --2- 172.21.15.18:0/131403296 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f907006ec00 0x7f90700710b0 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f9074006fd0 tx=0x7f9074008040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:41:44.000 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:43.999+0000 7f9084c00700 1 -- 172.21.15.18:0/131403296 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f907c05c530 con 0x7f9088065630 2024-04-12T21:41:44.255 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:44.254+0000 7f908e000700 1 -- 172.21.15.18:0/131403296 --> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f908804bf40 con 0x7f907006ec00 2024-04-12T21:41:44.256 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:44.255+0000 7f9084c00700 1 -- 172.21.15.18:0/131403296 <== mgr.34102 v2:172.21.15.18:6800/3812891919 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f908804bf40 con 0x7f907006ec00 2024-04-12T21:41:44.257 INFO:teuthology.orchestra.run.smithi018.stdout:{ 2024-04-12T21:41:44.257 INFO:teuthology.orchestra.run.smithi018.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:f4452def3be4d350f4a8e5feaf7ebb1511be9298fe68478ba7a2d6fbd7d69a63", 2024-04-12T21:41:44.257 INFO:teuthology.orchestra.run.smithi018.stdout: "in_progress": true, 2024-04-12T21:41:44.257 INFO:teuthology.orchestra.run.smithi018.stdout: "which": "Upgrading all daemon types on all hosts", 2024-04-12T21:41:44.257 INFO:teuthology.orchestra.run.smithi018.stdout: "services_complete": [ 2024-04-12T21:41:44.257 INFO:teuthology.orchestra.run.smithi018.stdout: "mgr", 2024-04-12T21:41:44.257 INFO:teuthology.orchestra.run.smithi018.stdout: "mds", 2024-04-12T21:41:44.257 INFO:teuthology.orchestra.run.smithi018.stdout: "crash", 2024-04-12T21:41:44.257 INFO:teuthology.orchestra.run.smithi018.stdout: "osd", 2024-04-12T21:41:44.258 INFO:teuthology.orchestra.run.smithi018.stdout: "mon" 2024-04-12T21:41:44.258 INFO:teuthology.orchestra.run.smithi018.stdout: ], 2024-04-12T21:41:44.258 INFO:teuthology.orchestra.run.smithi018.stdout: "progress": "16/21 daemons upgraded", 2024-04-12T21:41:44.258 INFO:teuthology.orchestra.run.smithi018.stdout: "message": "Currently upgrading mds daemons", 2024-04-12T21:41:44.258 INFO:teuthology.orchestra.run.smithi018.stdout: "is_paused": false 2024-04-12T21:41:44.258 INFO:teuthology.orchestra.run.smithi018.stdout:} 2024-04-12T21:41:44.259 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:44.258+0000 7f908e000700 1 -- 172.21.15.18:0/131403296 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f907006ec00 msgr2=0x7f90700710b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:41:44.259 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:44.258+0000 7f908e000700 1 --2- 172.21.15.18:0/131403296 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f907006ec00 0x7f90700710b0 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f9074006fd0 tx=0x7f9074008040 comp rx=0 tx=0).stop 2024-04-12T21:41:44.259 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:44.258+0000 7f908e000700 1 -- 172.21.15.18:0/131403296 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9088065630 msgr2=0x7f9088115c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:41:44.259 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:44.258+0000 7f908e000700 1 --2- 172.21.15.18:0/131403296 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9088065630 0x7f9088115c40 secure :-1 s=READY pgs=1069 cs=0 l=1 rev1=1 crypto rx=0x7f907c00b8b0 tx=0x7f907c009e30 comp rx=0 tx=0).stop 2024-04-12T21:41:44.260 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:44.258+0000 7f908e000700 1 -- 172.21.15.18:0/131403296 shutdown_connections 2024-04-12T21:41:44.260 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:44.258+0000 7f908e000700 1 --2- 172.21.15.18:0/131403296 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f907006ec00 0x7f90700710b0 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:41:44.260 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:44.258+0000 7f908e000700 1 --2- 172.21.15.18:0/131403296 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9088065630 0x7f9088115c40 unknown :-1 s=CLOSED pgs=1069 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:41:44.260 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:44.258+0000 7f908e000700 1 --2- 172.21.15.18:0/131403296 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9088102ca0 0x7f9088116180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:41:44.260 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:44.258+0000 7f908e000700 1 -- 172.21.15.18:0/131403296 >> 172.21.15.18:0/131403296 conn(0x7f9088071b70 msgr2=0x7f90880fd0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:41:44.260 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:44.258+0000 7f908e000700 1 -- 172.21.15.18:0/131403296 shutdown_connections 2024-04-12T21:41:44.260 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:44.259+0000 7f908e000700 1 -- 172.21.15.18:0/131403296 wait complete. 2024-04-12T21:41:44.413 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:44.411+0000 7efda1600700 1 -- 172.21.15.18:0/1560113446 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7efd9c072e00 msgr2=0x7efd9c10d7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:41:44.413 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:44.411+0000 7efda1600700 1 --2- 172.21.15.18:0/1560113446 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7efd9c072e00 0x7efd9c10d7f0 secure :-1 s=READY pgs=1070 cs=0 l=1 rev1=1 crypto rx=0x7efd88004970 tx=0x7efd88009e30 comp rx=0 tx=0).stop 2024-04-12T21:41:44.413 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:44.411+0000 7efda1600700 1 -- 172.21.15.18:0/1560113446 shutdown_connections 2024-04-12T21:41:44.413 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:44.411+0000 7efda1600700 1 --2- 172.21.15.18:0/1560113446 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7efd9c072e00 0x7efd9c10d7f0 unknown :-1 s=CLOSED pgs=1070 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:41:44.413 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:44.411+0000 7efda1600700 1 --2- 172.21.15.18:0/1560113446 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7efd9c0724f0 0x7efd9c0728c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:41:44.413 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:44.411+0000 7efda1600700 1 -- 172.21.15.18:0/1560113446 >> 172.21.15.18:0/1560113446 conn(0x7efd9c0fabd0 msgr2=0x7efd9c0fcfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:41:44.413 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:44.411+0000 7efda1600700 1 -- 172.21.15.18:0/1560113446 shutdown_connections 2024-04-12T21:41:44.413 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:44.411+0000 7efda1600700 1 -- 172.21.15.18:0/1560113446 wait complete. 2024-04-12T21:41:44.415 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:44.412+0000 7efda1600700 1 Processor -- start 2024-04-12T21:41:44.415 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:44.412+0000 7efda1600700 1 -- start start 2024-04-12T21:41:44.415 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:44.413+0000 7efda1600700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7efd9c0724f0 0x7efd9c119fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:41:44.415 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:44.413+0000 7efda1600700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7efd9c072e00 0x7efd9c11a510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:41:44.416 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:44.413+0000 7efda1600700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd9c11ab60 con 0x7efd9c0724f0 2024-04-12T21:41:44.416 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:44.413+0000 7efda1600700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd9c114140 con 0x7efd9c072e00 2024-04-12T21:41:44.416 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:44.413+0000 7efd9aa00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7efd9c0724f0 0x7efd9c119fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:41:44.416 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:44.413+0000 7efd9aa00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7efd9c0724f0 0x7efd9c119fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:58328/0 (socket says 172.21.15.18:58328) 2024-04-12T21:41:44.416 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:44.413+0000 7efd9aa00700 1 -- 172.21.15.18:0/1104714828 learned_addr learned my addr 172.21.15.18:0/1104714828 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:41:44.416 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:44.413+0000 7efd9a000700 1 --2- 172.21.15.18:0/1104714828 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7efd9c072e00 0x7efd9c11a510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:41:44.416 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:44.413+0000 7efd9aa00700 1 -- 172.21.15.18:0/1104714828 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7efd9c072e00 msgr2=0x7efd9c11a510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:41:44.416 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:44.413+0000 7efd9aa00700 1 --2- 172.21.15.18:0/1104714828 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7efd9c072e00 0x7efd9c11a510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:41:44.416 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:44.413+0000 7efd9aa00700 1 -- 172.21.15.18:0/1104714828 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd88005040 con 0x7efd9c0724f0 2024-04-12T21:41:44.416 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:44.413+0000 7efd9a000700 1 --2- 172.21.15.18:0/1104714828 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7efd9c072e00 0x7efd9c11a510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-12T21:41:44.416 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:44.414+0000 7efd9aa00700 1 --2- 172.21.15.18:0/1104714828 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7efd9c0724f0 0x7efd9c119fd0 secure :-1 s=READY pgs=1071 cs=0 l=1 rev1=1 crypto rx=0x7efd8c00c8c0 tx=0x7efd8c0044c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:41:44.416 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:44.414+0000 7efd92a00700 1 -- 172.21.15.18:0/1104714828 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7efd8c00a980 con 0x7efd9c0724f0 2024-04-12T21:41:44.417 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:44.414+0000 7efd92a00700 1 -- 172.21.15.18:0/1104714828 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7efd8c00ce60 con 0x7efd9c0724f0 2024-04-12T21:41:44.417 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:44.414+0000 7efd92a00700 1 -- 172.21.15.18:0/1104714828 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7efd8c0115e0 con 0x7efd9c0724f0 2024-04-12T21:41:44.417 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:44.414+0000 7efda1600700 1 -- 172.21.15.18:0/1104714828 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efd9c114420 con 0x7efd9c0724f0 2024-04-12T21:41:44.417 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:44.414+0000 7efda1600700 1 -- 172.21.15.18:0/1104714828 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efd9c1149e0 con 0x7efd9c0724f0 2024-04-12T21:41:44.417 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:44.415+0000 7efda1600700 1 -- 172.21.15.18:0/1104714828 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efd9c04c710 con 0x7efd9c0724f0 2024-04-12T21:41:44.417 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:44.416+0000 7efd92a00700 1 -- 172.21.15.18:0/1104714828 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7efd8c012400 con 0x7efd9c0724f0 2024-04-12T21:41:44.418 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:44.417+0000 7efd92a00700 1 --2- 172.21.15.18:0/1104714828 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7efd8006ec00 0x7efd800710b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:41:44.418 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:44.417+0000 7efd92a00700 1 -- 172.21.15.18:0/1104714828 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 6757+0+0 (secure 0 0 0) 0x7efd8c08ea40 con 0x7efd9c0724f0 2024-04-12T21:41:44.418 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:44.417+0000 7efd9a000700 1 --2- 172.21.15.18:0/1104714828 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7efd8006ec00 0x7efd800710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:41:44.419 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:44.418+0000 7efd9a000700 1 --2- 172.21.15.18:0/1104714828 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7efd8006ec00 0x7efd800710b0 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7efd88000c00 tx=0x7efd88009e30 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:41:44.423 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:44.422+0000 7efd92a00700 1 -- 172.21.15.18:0/1104714828 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7efd8c05b270 con 0x7efd9c0724f0 2024-04-12T21:41:44.533 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:41:44 smithi186 ceph-mon[86379]: from='client.? 172.21.15.18:0/2533388885' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T21:41:44.533 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:41:44 smithi186 ceph-mon[86379]: from='client.? 172.21.15.18:0/1034420739' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-12T21:41:44.604 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:41:44 smithi018 ceph-mon[112590]: from='client.? 172.21.15.18:0/2533388885' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T21:41:44.604 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:41:44 smithi018 ceph-mon[112590]: from='client.? 172.21.15.18:0/1034420739' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-12T21:41:44.752 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:44.751+0000 7efda1600700 1 -- 172.21.15.18:0/1104714828 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7efd9c1105c0 con 0x7efd9c0724f0 2024-04-12T21:41:44.753 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:44.752+0000 7efd92a00700 1 -- 172.21.15.18:0/1104714828 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+4303 (secure 0 0 0) 0x7efd8c0126b0 con 0x7efd9c0724f0 2024-04-12T21:41:44.753 INFO:teuthology.orchestra.run.smithi018.stdout:HEALTH_ERR failed to probe daemons or devices; 1 filesystem is degraded; 1 filesystem is offline 2024-04-12T21:41:44.753 INFO:teuthology.orchestra.run.smithi018.stdout:[WRN] CEPHADM_REFRESH_FAILED: failed to probe daemons or devices 2024-04-12T21:41:44.753 INFO:teuthology.orchestra.run.smithi018.stdout: host smithi186 `cephadm ceph-volume` failed: cephadm exited with an error code: 1, stderr: Inferring config /var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/mon.smithi186/config 2024-04-12T21:41:44.754 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/podman: timeout after 895 seconds 2024-04-12T21:41:44.754 INFO:teuthology.orchestra.run.smithi018.stdout:Non-zero exit code 124 from /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:f4452def3be4d350f4a8e5feaf7ebb1511be9298fe68478ba7a2d6fbd7d69a63 -e NODE_NAME=smithi186 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968:/var/run/ceph:z -v /var/log/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968:/var/log/ceph:z -v /var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmptgom53f9:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:f4452def3be4d350f4a8e5feaf7ebb1511be9298fe68478ba7a2d6fbd7d69a63 inventory --format=json-pretty --filter-for-batch 2024-04-12T21:41:44.754 INFO:teuthology.orchestra.run.smithi018.stdout:Traceback (most recent call last): 2024-04-12T21:41:44.754 INFO:teuthology.orchestra.run.smithi018.stdout: File "/usr/lib64/python3.9/runpy.py", line 197, in _run_module_as_main 2024-04-12T21:41:44.754 INFO:teuthology.orchestra.run.smithi018.stdout: return _run_code(code, main_globals, None, 2024-04-12T21:41:44.754 INFO:teuthology.orchestra.run.smithi018.stdout: File "/usr/lib64/python3.9/runpy.py", line 87, in _run_code 2024-04-12T21:41:44.754 INFO:teuthology.orchestra.run.smithi018.stdout: exec(code, run_globals) 2024-04-12T21:41:44.754 INFO:teuthology.orchestra.run.smithi018.stdout: File "/var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/cephadm.e44f3ed5e34e4ef52eedef233d1190bff004470d8e8005127b5312bff357fa7f/__main__.py", line 5618, in 2024-04-12T21:41:44.754 INFO:teuthology.orchestra.run.smithi018.stdout: File "/var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/cephadm.e44f3ed5e34e4ef52eedef233d1190bff004470d8e8005127b5312bff357fa7f/__main__.py", line 5606, in main 2024-04-12T21:41:44.754 INFO:teuthology.orchestra.run.smithi018.stdout: File "/var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/cephadm.e44f3ed5e34e4ef52eedef233d1190bff004470d8e8005127b5312bff357fa7f/__main__.py", line 408, in _infer_config 2024-04-12T21:41:44.754 INFO:teuthology.orchestra.run.smithi018.stdout: File "/var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/cephadm.e44f3ed5e34e4ef52eedef233d1190bff004470d8e8005127b5312bff357fa7f/__main__.py", line 323, in _infer_fsid 2024-04-12T21:41:44.754 INFO:teuthology.orchestra.run.smithi018.stdout: File "/var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/cephadm.e44f3ed5e34e4ef52eedef233d1190bff004470d8e8005127b5312bff357fa7f/__main__.py", line 436, in _infer_image 2024-04-12T21:41:44.754 INFO:teuthology.orchestra.run.smithi018.stdout: File "/var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/cephadm.e44f3ed5e34e4ef52eedef233d1190bff004470d8e8005127b5312bff357fa7f/__main__.py", line 310, in _validate_fsid 2024-04-12T21:41:44.755 INFO:teuthology.orchestra.run.smithi018.stdout: File "/var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/cephadm.e44f3ed5e34e4ef52eedef233d1190bff004470d8e8005127b5312bff357fa7f/__main__.py", line 3359, in command_ceph_volume 2024-04-12T21:41:44.755 INFO:teuthology.orchestra.run.smithi018.stdout: File "/var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/cephadm.e44f3ed5e34e4ef52eedef233d1190bff004470d8e8005127b5312bff357fa7f/cephadmlib/call_wrappers.py", line 310, in call_throws 2024-04-12T21:41:44.755 INFO:teuthology.orchestra.run.smithi018.stdout:RuntimeError: Failed command: /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:f4452def3be4d350f4a8e5feaf7ebb1511be9298fe68478ba7a2d6fbd7d69a63 -e NODE_NAME=smithi186 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968:/var/run/ceph:z -v /var/log/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968:/var/log/ceph:z -v /var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmptgom53f9:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:f4452def3be4d350f4a8e5feaf7ebb1511be9298fe68478ba7a2d6fbd7d69a63 inventory --format=json-pretty --filter-for-batch 2024-04-12T21:41:44.755 INFO:teuthology.orchestra.run.smithi018.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-04-12T21:41:44.755 INFO:teuthology.orchestra.run.smithi018.stdout: fs cephfs is degraded 2024-04-12T21:41:44.755 INFO:teuthology.orchestra.run.smithi018.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-04-12T21:41:44.755 INFO:teuthology.orchestra.run.smithi018.stdout: fs cephfs is offline because no MDS is active for it. 2024-04-12T21:41:44.758 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:44.756+0000 7efda1600700 1 -- 172.21.15.18:0/1104714828 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7efd8006ec00 msgr2=0x7efd800710b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:41:44.758 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:44.756+0000 7efda1600700 1 --2- 172.21.15.18:0/1104714828 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7efd8006ec00 0x7efd800710b0 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7efd88000c00 tx=0x7efd88009e30 comp rx=0 tx=0).stop 2024-04-12T21:41:44.758 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:44.756+0000 7efda1600700 1 -- 172.21.15.18:0/1104714828 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7efd9c0724f0 msgr2=0x7efd9c119fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:41:44.758 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:44.756+0000 7efda1600700 1 --2- 172.21.15.18:0/1104714828 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7efd9c0724f0 0x7efd9c119fd0 secure :-1 s=READY pgs=1071 cs=0 l=1 rev1=1 crypto rx=0x7efd8c00c8c0 tx=0x7efd8c0044c0 comp rx=0 tx=0).stop 2024-04-12T21:41:44.758 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:44.756+0000 7efda1600700 1 -- 172.21.15.18:0/1104714828 shutdown_connections 2024-04-12T21:41:44.758 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:44.756+0000 7efda1600700 1 --2- 172.21.15.18:0/1104714828 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7efd8006ec00 0x7efd800710b0 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:41:44.758 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:44.756+0000 7efda1600700 1 --2- 172.21.15.18:0/1104714828 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7efd9c0724f0 0x7efd9c119fd0 unknown :-1 s=CLOSED pgs=1071 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:41:44.758 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:44.756+0000 7efda1600700 1 --2- 172.21.15.18:0/1104714828 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7efd9c072e00 0x7efd9c11a510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:41:44.758 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:44.756+0000 7efda1600700 1 -- 172.21.15.18:0/1104714828 >> 172.21.15.18:0/1104714828 conn(0x7efd9c0fabd0 msgr2=0x7efd9c0fb7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:41:44.758 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:44.757+0000 7efda1600700 1 -- 172.21.15.18:0/1104714828 shutdown_connections 2024-04-12T21:41:44.758 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:41:44.757+0000 7efda1600700 1 -- 172.21.15.18:0/1104714828 wait complete. 2024-04-12T21:41:45.385 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:41:45 smithi018 ceph-mon[112590]: pgmap v1262: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:41:45.385 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:41:45 smithi018 ceph-mon[112590]: from='client.36208 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:41:45.385 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:41:45 smithi018 ceph-mon[112590]: from='client.? 172.21.15.18:0/1104714828' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-12T21:41:45.533 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:41:45 smithi186 ceph-mon[86379]: pgmap v1262: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:41:45.534 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:41:45 smithi186 ceph-mon[86379]: from='client.36208 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:41:45.534 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:41:45 smithi186 ceph-mon[86379]: from='client.? 172.21.15.18:0/1104714828' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-12T21:41:47.533 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:41:47 smithi186 ceph-mon[86379]: pgmap v1263: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:41:47.604 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:41:47 smithi018 ceph-mon[112590]: pgmap v1263: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:41:49.533 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:41:49 smithi186 ceph-mon[86379]: pgmap v1264: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:41:49.604 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:41:49 smithi018 ceph-mon[112590]: pgmap v1264: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:41:51.307 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:41:51 smithi186 ceph-mon[86379]: pgmap v1265: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:41:51.604 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:41:51 smithi018 ceph-mon[112590]: pgmap v1265: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:41:53.533 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:41:53 smithi186 ceph-mon[86379]: pgmap v1266: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:41:53.604 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:41:53 smithi018 ceph-mon[112590]: pgmap v1266: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:41:55.604 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:41:55 smithi018 ceph-mon[112590]: pgmap v1267: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:41:55.605 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:41:55 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T21:41:55.783 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:41:55 smithi186 ceph-mon[86379]: pgmap v1267: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:41:55.783 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:41:55 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T21:41:57.604 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:41:57 smithi018 ceph-mon[112590]: pgmap v1268: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:41:57.783 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:41:57 smithi186 ceph-mon[86379]: pgmap v1268: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:41:59.604 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:41:59 smithi018 ceph-mon[112590]: pgmap v1269: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:41:59.783 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:41:59 smithi186 ceph-mon[86379]: pgmap v1269: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:42:01.604 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:42:01 smithi018 ceph-mon[112590]: pgmap v1270: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:42:01.783 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:42:01 smithi186 ceph-mon[86379]: pgmap v1270: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:42:03.604 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:42:03 smithi018 ceph-mon[112590]: pgmap v1271: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:42:03.783 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:42:03 smithi186 ceph-mon[86379]: pgmap v1271: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:42:05.604 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:42:05 smithi018 ceph-mon[112590]: pgmap v1272: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:42:05.783 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:42:05 smithi186 ceph-mon[86379]: pgmap v1272: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:42:07.604 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:42:07 smithi018 ceph-mon[112590]: pgmap v1273: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:42:07.783 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:42:07 smithi186 ceph-mon[86379]: pgmap v1273: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:42:09.604 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:42:09 smithi018 ceph-mon[112590]: pgmap v1274: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:42:09.783 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:42:09 smithi186 ceph-mon[86379]: pgmap v1274: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:42:10.604 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:42:10 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T21:42:10.783 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:42:10 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T21:42:11.604 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:42:11 smithi018 ceph-mon[112590]: pgmap v1275: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:42:11.783 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:42:11 smithi186 ceph-mon[86379]: pgmap v1275: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:42:13.783 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:42:13 smithi186 ceph-mon[86379]: pgmap v1276: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:42:13.854 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:42:13 smithi018 ceph-mon[112590]: pgmap v1276: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:42:14.930 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:14.929+0000 7f74d3000700 1 -- 172.21.15.18:0/3784949836 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f74cc0724f0 msgr2=0x7f74cc0728c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:42:14.930 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:14.929+0000 7f74d3000700 1 --2- 172.21.15.18:0/3784949836 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f74cc0724f0 0x7f74cc0728c0 secure :-1 s=READY pgs=1072 cs=0 l=1 rev1=1 crypto rx=0x7f74c0004940 tx=0x7f74c0009e30 comp rx=0 tx=0).stop 2024-04-12T21:42:14.930 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:14.929+0000 7f74d3000700 1 -- 172.21.15.18:0/3784949836 shutdown_connections 2024-04-12T21:42:14.930 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:14.929+0000 7f74d3000700 1 --2- 172.21.15.18:0/3784949836 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f74cc072e00 0x7f74cc10d630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:42:14.930 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:14.929+0000 7f74d3000700 1 --2- 172.21.15.18:0/3784949836 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f74cc0724f0 0x7f74cc0728c0 unknown :-1 s=CLOSED pgs=1072 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:42:14.930 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:14.929+0000 7f74d3000700 1 -- 172.21.15.18:0/3784949836 >> 172.21.15.18:0/3784949836 conn(0x7f74cc0fab10 msgr2=0x7f74cc0fcef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:42:14.930 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:14.929+0000 7f74d3000700 1 -- 172.21.15.18:0/3784949836 shutdown_connections 2024-04-12T21:42:14.930 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:14.929+0000 7f74d3000700 1 -- 172.21.15.18:0/3784949836 wait complete. 2024-04-12T21:42:14.931 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:14.930+0000 7f74d3000700 1 Processor -- start 2024-04-12T21:42:14.931 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:14.931+0000 7f74d3000700 1 -- start start 2024-04-12T21:42:14.931 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:14.931+0000 7f74d3000700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f74cc0724f0 0x7f74cc119f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:42:14.931 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:14.931+0000 7f74d3000700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f74cc072e00 0x7f74cc11a4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:42:14.931 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:14.931+0000 7f74d3000700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74cc11ab80 con 0x7f74cc072e00 2024-04-12T21:42:14.931 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:14.931+0000 7f74d3000700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74cc113ec0 con 0x7f74cc0724f0 2024-04-12T21:42:14.933 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:14.931+0000 7f74d1200700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f74cc072e00 0x7f74cc11a4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:42:14.933 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:14.932+0000 7f74d1200700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f74cc072e00 0x7f74cc11a4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:60630/0 (socket says 172.21.15.18:60630) 2024-04-12T21:42:14.933 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:14.932+0000 7f74d1200700 1 -- 172.21.15.18:0/72114482 learned_addr learned my addr 172.21.15.18:0/72114482 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:42:14.933 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:14.932+0000 7f74d1c00700 1 --2- 172.21.15.18:0/72114482 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f74cc0724f0 0x7f74cc119f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:42:14.933 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:14.932+0000 7f74d1200700 1 -- 172.21.15.18:0/72114482 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f74cc0724f0 msgr2=0x7f74cc119f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:42:14.933 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:14.932+0000 7f74d1200700 1 --2- 172.21.15.18:0/72114482 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f74cc0724f0 0x7f74cc119f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:42:14.933 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:14.932+0000 7f74d1200700 1 -- 172.21.15.18:0/72114482 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74c0005040 con 0x7f74cc072e00 2024-04-12T21:42:14.933 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:14.932+0000 7f74d1c00700 1 --2- 172.21.15.18:0/72114482 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f74cc0724f0 0x7f74cc119f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-12T21:42:14.933 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:14.932+0000 7f74d1200700 1 --2- 172.21.15.18:0/72114482 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f74cc072e00 0x7f74cc11a4a0 secure :-1 s=READY pgs=1073 cs=0 l=1 rev1=1 crypto rx=0x7f74b800b790 tx=0x7f74b8009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:42:14.933 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:14.933+0000 7f74c9600700 1 -- 172.21.15.18:0/72114482 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f74b800f770 con 0x7f74cc072e00 2024-04-12T21:42:14.933 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:14.933+0000 7f74c9600700 1 -- 172.21.15.18:0/72114482 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f74b800fda0 con 0x7f74cc072e00 2024-04-12T21:42:14.933 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:14.933+0000 7f74c9600700 1 -- 172.21.15.18:0/72114482 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f74b800d310 con 0x7f74cc072e00 2024-04-12T21:42:14.934 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:14.933+0000 7f74d3000700 1 -- 172.21.15.18:0/72114482 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f74cc114140 con 0x7f74cc072e00 2024-04-12T21:42:14.934 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:14.933+0000 7f74d3000700 1 -- 172.21.15.18:0/72114482 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f74cc114700 con 0x7f74cc072e00 2024-04-12T21:42:14.934 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:14.934+0000 7f74d3000700 1 -- 172.21.15.18:0/72114482 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f74cc073be0 con 0x7f74cc072e00 2024-04-12T21:42:14.935 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:14.935+0000 7f74c9600700 1 -- 172.21.15.18:0/72114482 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7f74b8010420 con 0x7f74cc072e00 2024-04-12T21:42:14.937 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:14.937+0000 7f74c9600700 1 --2- 172.21.15.18:0/72114482 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f74b406ec00 0x7f74b40710b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:42:14.938 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:14.938+0000 7f74c9600700 1 -- 172.21.15.18:0/72114482 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 6757+0+0 (secure 0 0 0) 0x7f74b808e9a0 con 0x7f74cc072e00 2024-04-12T21:42:14.938 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:14.938+0000 7f74d1c00700 1 --2- 172.21.15.18:0/72114482 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f74b406ec00 0x7f74b40710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:42:14.938 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:14.939+0000 7f74d1c00700 1 --2- 172.21.15.18:0/72114482 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f74b406ec00 0x7f74b40710b0 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f74c0000c00 tx=0x7f74c0009e30 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:42:14.943 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:14.943+0000 7f74c9600700 1 -- 172.21.15.18:0/72114482 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f74b805b1d0 con 0x7f74cc072e00 2024-04-12T21:42:15.206 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:15.206+0000 7f74d3000700 1 -- 172.21.15.18:0/72114482 --> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f74cc110440 con 0x7f74b406ec00 2024-04-12T21:42:15.208 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:15.208+0000 7f74c9600700 1 -- 172.21.15.18:0/72114482 <== mgr.34102 v2:172.21.15.18:6800/3812891919 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f74cc110440 con 0x7f74b406ec00 2024-04-12T21:42:15.211 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:15.211+0000 7f74d3000700 1 -- 172.21.15.18:0/72114482 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f74b406ec00 msgr2=0x7f74b40710b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:42:15.211 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:15.211+0000 7f74d3000700 1 --2- 172.21.15.18:0/72114482 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f74b406ec00 0x7f74b40710b0 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f74c0000c00 tx=0x7f74c0009e30 comp rx=0 tx=0).stop 2024-04-12T21:42:15.211 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:15.211+0000 7f74d3000700 1 -- 172.21.15.18:0/72114482 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f74cc072e00 msgr2=0x7f74cc11a4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:42:15.212 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:15.211+0000 7f74d3000700 1 --2- 172.21.15.18:0/72114482 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f74cc072e00 0x7f74cc11a4a0 secure :-1 s=READY pgs=1073 cs=0 l=1 rev1=1 crypto rx=0x7f74b800b790 tx=0x7f74b8009250 comp rx=0 tx=0).stop 2024-04-12T21:42:15.212 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:15.211+0000 7f74d3000700 1 -- 172.21.15.18:0/72114482 shutdown_connections 2024-04-12T21:42:15.212 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:15.211+0000 7f74d3000700 1 --2- 172.21.15.18:0/72114482 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f74b406ec00 0x7f74b40710b0 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:42:15.212 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:15.211+0000 7f74d3000700 1 --2- 172.21.15.18:0/72114482 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f74cc0724f0 0x7f74cc119f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:42:15.212 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:15.211+0000 7f74d3000700 1 --2- 172.21.15.18:0/72114482 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f74cc072e00 0x7f74cc11a4a0 unknown :-1 s=CLOSED pgs=1073 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:42:15.212 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:15.211+0000 7f74d3000700 1 -- 172.21.15.18:0/72114482 >> 172.21.15.18:0/72114482 conn(0x7f74cc0fab10 msgr2=0x7f74cc0faee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:42:15.212 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:15.211+0000 7f74d3000700 1 -- 172.21.15.18:0/72114482 shutdown_connections 2024-04-12T21:42:15.212 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:15.211+0000 7f74d3000700 1 -- 172.21.15.18:0/72114482 wait complete. 2024-04-12T21:42:15.227 INFO:teuthology.orchestra.run.smithi018.stdout:true 2024-04-12T21:42:15.366 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:15.365+0000 7f367a400700 1 -- 172.21.15.18:0/450291860 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3674102e10 msgr2=0x7f36741031e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:42:15.366 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:15.365+0000 7f367a400700 1 --2- 172.21.15.18:0/450291860 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3674102e10 0x7f36741031e0 secure :-1 s=READY pgs=1074 cs=0 l=1 rev1=1 crypto rx=0x7f3664004940 tx=0x7f3664009e30 comp rx=0 tx=0).stop 2024-04-12T21:42:15.366 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:15.366+0000 7f367a400700 1 -- 172.21.15.18:0/450291860 shutdown_connections 2024-04-12T21:42:15.366 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:15.366+0000 7f367a400700 1 --2- 172.21.15.18:0/450291860 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f36741037b0 0x7f36740fce20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:42:15.367 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:15.366+0000 7f367a400700 1 --2- 172.21.15.18:0/450291860 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3674102e10 0x7f36741031e0 unknown :-1 s=CLOSED pgs=1074 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:42:15.367 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:15.366+0000 7f367a400700 1 -- 172.21.15.18:0/450291860 >> 172.21.15.18:0/450291860 conn(0x7f36740f89a0 msgr2=0x7f36740fadb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:42:15.367 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:15.366+0000 7f367a400700 1 -- 172.21.15.18:0/450291860 shutdown_connections 2024-04-12T21:42:15.367 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:15.366+0000 7f367a400700 1 -- 172.21.15.18:0/450291860 wait complete. 2024-04-12T21:42:15.367 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:15.367+0000 7f367a400700 1 Processor -- start 2024-04-12T21:42:15.367 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:15.367+0000 7f367a400700 1 -- start start 2024-04-12T21:42:15.368 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:15.368+0000 7f367a400700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3674102e10 0x7f3674115dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:42:15.368 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:15.368+0000 7f367a400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f36741037b0 0x7f3674116300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:42:15.368 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:15.368+0000 7f367a400700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3674116950 con 0x7f36741037b0 2024-04-12T21:42:15.368 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:15.368+0000 7f367a400700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f367410fd70 con 0x7f3674102e10 2024-04-12T21:42:15.368 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:15.368+0000 7f366aa00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f36741037b0 0x7f3674116300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:42:15.368 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:15.368+0000 7f366aa00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f36741037b0 0x7f3674116300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:60644/0 (socket says 172.21.15.18:60644) 2024-04-12T21:42:15.369 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:15.368+0000 7f366aa00700 1 -- 172.21.15.18:0/3008726766 learned_addr learned my addr 172.21.15.18:0/3008726766 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:42:15.369 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:15.368+0000 7f3673400700 1 --2- 172.21.15.18:0/3008726766 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3674102e10 0x7f3674115dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:42:15.369 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:15.369+0000 7f366aa00700 1 -- 172.21.15.18:0/3008726766 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3674102e10 msgr2=0x7f3674115dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:42:15.370 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:15.369+0000 7f366aa00700 1 --2- 172.21.15.18:0/3008726766 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3674102e10 0x7f3674115dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:42:15.370 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:15.369+0000 7f366aa00700 1 -- 172.21.15.18:0/3008726766 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3664005040 con 0x7f36741037b0 2024-04-12T21:42:15.370 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:15.369+0000 7f366aa00700 1 --2- 172.21.15.18:0/3008726766 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f36741037b0 0x7f3674116300 secure :-1 s=READY pgs=1075 cs=0 l=1 rev1=1 crypto rx=0x7f365c001d00 tx=0x7f365c009e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:42:15.370 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:15.369+0000 7f366be00700 1 -- 172.21.15.18:0/3008726766 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f365c007d10 con 0x7f36741037b0 2024-04-12T21:42:15.370 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:15.369+0000 7f366be00700 1 -- 172.21.15.18:0/3008726766 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f365c007e70 con 0x7f36741037b0 2024-04-12T21:42:15.370 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:15.369+0000 7f367a400700 1 -- 172.21.15.18:0/3008726766 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3674110050 con 0x7f36741037b0 2024-04-12T21:42:15.370 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:15.369+0000 7f366be00700 1 -- 172.21.15.18:0/3008726766 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f365c014400 con 0x7f36741037b0 2024-04-12T21:42:15.370 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:15.370+0000 7f367a400700 1 -- 172.21.15.18:0/3008726766 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3674110610 con 0x7f36741037b0 2024-04-12T21:42:15.371 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:15.371+0000 7f367a400700 1 -- 172.21.15.18:0/3008726766 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3674106ba0 con 0x7f36741037b0 2024-04-12T21:42:15.372 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:15.372+0000 7f366be00700 1 -- 172.21.15.18:0/3008726766 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7f365c0091d0 con 0x7f36741037b0 2024-04-12T21:42:15.373 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:15.373+0000 7f366be00700 1 --2- 172.21.15.18:0/3008726766 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f365806ec10 0x7f36580710c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:42:15.373 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:15.373+0000 7f366be00700 1 -- 172.21.15.18:0/3008726766 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 6757+0+0 (secure 0 0 0) 0x7f365c08e790 con 0x7f36741037b0 2024-04-12T21:42:15.373 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:15.373+0000 7f3673400700 1 --2- 172.21.15.18:0/3008726766 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f365806ec10 0x7f36580710c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:42:15.374 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:15.374+0000 7f3673400700 1 --2- 172.21.15.18:0/3008726766 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f365806ec10 0x7f36580710c0 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f366400b3b0 tx=0x7f3664009e30 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:42:15.379 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:15.379+0000 7f366be00700 1 -- 172.21.15.18:0/3008726766 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f365c05af70 con 0x7f36741037b0 2024-04-12T21:42:15.604 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:42:15 smithi018 ceph-mon[112590]: pgmap v1277: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:42:15.634 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:15.633+0000 7f367a400700 1 -- 172.21.15.18:0/3008726766 --> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f3674111480 con 0x7f365806ec10 2024-04-12T21:42:15.636 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:15.636+0000 7f366be00700 1 -- 172.21.15.18:0/3008726766 <== mgr.34102 v2:172.21.15.18:6800/3812891919 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f3674111480 con 0x7f365806ec10 2024-04-12T21:42:15.639 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:15.639+0000 7f367a400700 1 -- 172.21.15.18:0/3008726766 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f365806ec10 msgr2=0x7f36580710c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:42:15.639 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:15.639+0000 7f367a400700 1 --2- 172.21.15.18:0/3008726766 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f365806ec10 0x7f36580710c0 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f366400b3b0 tx=0x7f3664009e30 comp rx=0 tx=0).stop 2024-04-12T21:42:15.639 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:15.639+0000 7f367a400700 1 -- 172.21.15.18:0/3008726766 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f36741037b0 msgr2=0x7f3674116300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:42:15.639 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:15.639+0000 7f367a400700 1 --2- 172.21.15.18:0/3008726766 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f36741037b0 0x7f3674116300 secure :-1 s=READY pgs=1075 cs=0 l=1 rev1=1 crypto rx=0x7f365c001d00 tx=0x7f365c009e40 comp rx=0 tx=0).stop 2024-04-12T21:42:15.639 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:15.639+0000 7f367a400700 1 -- 172.21.15.18:0/3008726766 shutdown_connections 2024-04-12T21:42:15.640 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:15.639+0000 7f367a400700 1 --2- 172.21.15.18:0/3008726766 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f365806ec10 0x7f36580710c0 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:42:15.640 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:15.639+0000 7f367a400700 1 --2- 172.21.15.18:0/3008726766 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3674102e10 0x7f3674115dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:42:15.640 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:15.639+0000 7f367a400700 1 --2- 172.21.15.18:0/3008726766 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f36741037b0 0x7f3674116300 unknown :-1 s=CLOSED pgs=1075 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:42:15.640 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:15.639+0000 7f367a400700 1 -- 172.21.15.18:0/3008726766 >> 172.21.15.18:0/3008726766 conn(0x7f36740f89a0 msgr2=0x7f36740fad40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:42:15.640 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:15.639+0000 7f367a400700 1 -- 172.21.15.18:0/3008726766 shutdown_connections 2024-04-12T21:42:15.640 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:15.639+0000 7f367a400700 1 -- 172.21.15.18:0/3008726766 wait complete. 2024-04-12T21:42:15.783 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:42:15 smithi186 ceph-mon[86379]: pgmap v1277: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:42:15.792 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:15.792+0000 7f8d92200700 1 -- 172.21.15.18:0/94234126 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8d8c109dd0 msgr2=0x7f8d8c103440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:42:15.792 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:15.792+0000 7f8d92200700 1 --2- 172.21.15.18:0/94234126 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8d8c109dd0 0x7f8d8c103440 secure :-1 s=READY pgs=1076 cs=0 l=1 rev1=1 crypto rx=0x7f8d74004940 tx=0x7f8d74009e30 comp rx=0 tx=0).stop 2024-04-12T21:42:15.792 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:15.792+0000 7f8d92200700 1 -- 172.21.15.18:0/94234126 shutdown_connections 2024-04-12T21:42:15.793 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:15.792+0000 7f8d92200700 1 --2- 172.21.15.18:0/94234126 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8d8c109dd0 0x7f8d8c103440 unknown :-1 s=CLOSED pgs=1076 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:42:15.793 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:15.792+0000 7f8d92200700 1 --2- 172.21.15.18:0/94234126 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8d8c109430 0x7f8d8c109800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:42:15.793 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:15.792+0000 7f8d92200700 1 -- 172.21.15.18:0/94234126 >> 172.21.15.18:0/94234126 conn(0x7f8d8c072c00 msgr2=0x7f8d8c075010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:42:15.793 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:15.792+0000 7f8d92200700 1 -- 172.21.15.18:0/94234126 shutdown_connections 2024-04-12T21:42:15.793 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:15.792+0000 7f8d92200700 1 -- 172.21.15.18:0/94234126 wait complete. 2024-04-12T21:42:15.793 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:15.793+0000 7f8d92200700 1 Processor -- start 2024-04-12T21:42:15.794 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:15.794+0000 7f8d92200700 1 -- start start 2024-04-12T21:42:15.794 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:15.794+0000 7f8d92200700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8d8c109430 0x7f8d8c119fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:42:15.794 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:15.794+0000 7f8d92200700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8d8c109dd0 0x7f8d8c11a4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:42:15.794 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:15.794+0000 7f8d92200700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d8c11abc0 con 0x7f8d8c109430 2024-04-12T21:42:15.794 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:15.794+0000 7f8d92200700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d8c114110 con 0x7f8d8c109dd0 2024-04-12T21:42:15.795 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:15.795+0000 7f8d8b400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8d8c109430 0x7f8d8c119fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:42:15.795 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:15.795+0000 7f8d8b400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8d8c109430 0x7f8d8c119fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:60676/0 (socket says 172.21.15.18:60676) 2024-04-12T21:42:15.795 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:15.795+0000 7f8d8b400700 1 -- 172.21.15.18:0/3656031692 learned_addr learned my addr 172.21.15.18:0/3656031692 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:42:15.795 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:15.795+0000 7f8d83e00700 1 --2- 172.21.15.18:0/3656031692 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8d8c109dd0 0x7f8d8c11a4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:42:15.796 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:15.795+0000 7f8d8b400700 1 -- 172.21.15.18:0/3656031692 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8d8c109dd0 msgr2=0x7f8d8c11a4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:42:15.796 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:15.795+0000 7f8d8b400700 1 --2- 172.21.15.18:0/3656031692 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8d8c109dd0 0x7f8d8c11a4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:42:15.796 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:15.795+0000 7f8d8b400700 1 -- 172.21.15.18:0/3656031692 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d74005040 con 0x7f8d8c109430 2024-04-12T21:42:15.796 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:15.796+0000 7f8d8b400700 1 --2- 172.21.15.18:0/3656031692 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8d8c109430 0x7f8d8c119fa0 secure :-1 s=READY pgs=1077 cs=0 l=1 rev1=1 crypto rx=0x7f8d7c00b790 tx=0x7f8d7c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:42:15.797 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:15.796+0000 7f8d83400700 1 -- 172.21.15.18:0/3656031692 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8d7c00f770 con 0x7f8d8c109430 2024-04-12T21:42:15.797 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:15.796+0000 7f8d83400700 1 -- 172.21.15.18:0/3656031692 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f8d7c00fda0 con 0x7f8d8c109430 2024-04-12T21:42:15.797 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:15.796+0000 7f8d83400700 1 -- 172.21.15.18:0/3656031692 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8d7c00d310 con 0x7f8d8c109430 2024-04-12T21:42:15.797 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:15.796+0000 7f8d92200700 1 -- 172.21.15.18:0/3656031692 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8d8c114390 con 0x7f8d8c109430 2024-04-12T21:42:15.797 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:15.796+0000 7f8d92200700 1 -- 172.21.15.18:0/3656031692 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8d8c114950 con 0x7f8d8c109430 2024-04-12T21:42:15.797 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:15.797+0000 7f8d92200700 1 -- 172.21.15.18:0/3656031692 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8d8c104b80 con 0x7f8d8c109430 2024-04-12T21:42:15.798 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:15.798+0000 7f8d83400700 1 -- 172.21.15.18:0/3656031692 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7f8d7c010420 con 0x7f8d8c109430 2024-04-12T21:42:15.799 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:15.799+0000 7f8d83400700 1 --2- 172.21.15.18:0/3656031692 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f8d7006ec00 0x7f8d700710b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:42:15.799 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:15.799+0000 7f8d83400700 1 -- 172.21.15.18:0/3656031692 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 6757+0+0 (secure 0 0 0) 0x7f8d7c08e9a0 con 0x7f8d8c109430 2024-04-12T21:42:15.800 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:15.800+0000 7f8d83e00700 1 --2- 172.21.15.18:0/3656031692 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f8d7006ec00 0x7f8d700710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:42:15.800 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:15.800+0000 7f8d83e00700 1 --2- 172.21.15.18:0/3656031692 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f8d7006ec00 0x7f8d700710b0 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f8d8c115840 tx=0x7f8d74009e30 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:42:15.805 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:15.805+0000 7f8d83400700 1 -- 172.21.15.18:0/3656031692 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f8d7c05b1d0 con 0x7f8d8c109430 2024-04-12T21:42:16.048 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:16.048+0000 7f8d92200700 1 -- 172.21.15.18:0/3656031692 --> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f8d8c115690 con 0x7f8d7006ec00 2024-04-12T21:42:16.086 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:16.086+0000 7f8d83400700 1 -- 172.21.15.18:0/3656031692 <== mgr.34102 v2:172.21.15.18:6800/3812891919 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7f8d8c115690 con 0x7f8d7006ec00 2024-04-12T21:42:16.087 INFO:teuthology.orchestra.run.smithi018.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-12T21:42:16.087 INFO:teuthology.orchestra.run.smithi018.stdout:alertmanager.smithi018 smithi018 *:9093,9094 running (42m) 3m ago 57m 14.8M - 0.25.0 c8568f914cd2 c332090068e8 2024-04-12T21:42:16.087 INFO:teuthology.orchestra.run.smithi018.stdout:crash.smithi018 smithi018 running (39m) 3m ago 57m 6665k - 19.0.0-2879-g324baf8d 8717720dc5e5 b7f80d1f7982 2024-04-12T21:42:16.087 INFO:teuthology.orchestra.run.smithi018.stdout:crash.smithi186 smithi186 running (39m) 3m ago 56m 6660k - 19.0.0-2879-g324baf8d 8717720dc5e5 a4f55c81b623 2024-04-12T21:42:16.087 INFO:teuthology.orchestra.run.smithi018.stdout:grafana.smithi018 smithi018 *:3000 running (42m) 3m ago 57m 73.8M - 9.4.12 2bacad6d85d8 b08e3bf176dd 2024-04-12T21:42:16.087 INFO:teuthology.orchestra.run.smithi018.stdout:mds.cephfs.smithi018.acqbsc smithi018 running (35m) 3m ago 50m 21.9M - 19.0.0-2879-g324baf8d 8717720dc5e5 021f99ab24ae 2024-04-12T21:42:16.087 INFO:teuthology.orchestra.run.smithi018.stdout:mds.cephfs.smithi018.szjluo smithi018 running (34m) 3m ago 50m 21.7M - 19.0.0-2879-g324baf8d 8717720dc5e5 0003117479d4 2024-04-12T21:42:16.087 INFO:teuthology.orchestra.run.smithi018.stdout:mds.cephfs.smithi186.jdllyn smithi186 running (19m) 3m ago 50m 16.5M - 19.0.0-2879-g324baf8d 8717720dc5e5 21b1f46eef23 2024-04-12T21:42:16.087 INFO:teuthology.orchestra.run.smithi018.stdout:mds.cephfs.smithi186.wjnafs smithi186 running (4m) 3m ago 50m 11.0M - 19.0.0-2879-g324baf8d 8717720dc5e5 3cf1ef4148e2 2024-04-12T21:42:16.087 INFO:teuthology.orchestra.run.smithi018.stdout:mgr.smithi018.xjzwga smithi018 *:8443,9283,8765 running (45m) 3m ago 59m 517M - 19.0.0-2879-g324baf8d 8717720dc5e5 56088305eab7 2024-04-12T21:42:16.088 INFO:teuthology.orchestra.run.smithi018.stdout:mgr.smithi186.kqcfxk smithi186 *:8443,9283,8765 running (44m) 3m ago 56m 460M - 19.0.0-2879-g324baf8d 8717720dc5e5 09cd766e5e0d 2024-04-12T21:42:16.088 INFO:teuthology.orchestra.run.smithi018.stdout:mon.smithi018 smithi018 running (40m) 3m ago 59m 88.2M 2048M 19.0.0-2879-g324baf8d 8717720dc5e5 df3e201dfed6 2024-04-12T21:42:16.088 INFO:teuthology.orchestra.run.smithi018.stdout:mon.smithi186 smithi186 running (40m) 3m ago 55m 53.6M 2048M 19.0.0-2879-g324baf8d 8717720dc5e5 c77498da9b2b 2024-04-12T21:42:16.088 INFO:teuthology.orchestra.run.smithi018.stdout:node-exporter.smithi018 smithi018 *:9100 running (43m) 3m ago 57m 8468k - 1.5.0 0da6a335fe13 326c3d9c6847 2024-04-12T21:42:16.088 INFO:teuthology.orchestra.run.smithi018.stdout:node-exporter.smithi186 smithi186 *:9100 running (43m) 3m ago 56m 8166k - 1.5.0 0da6a335fe13 ec2112baeecf 2024-04-12T21:42:16.088 INFO:teuthology.orchestra.run.smithi018.stdout:osd.0 smithi018 running (38m) 3m ago 54m 123M 1559M 19.0.0-2879-g324baf8d 8717720dc5e5 60b1273ca047 2024-04-12T21:42:16.088 INFO:teuthology.orchestra.run.smithi018.stdout:osd.1 smithi018 running (37m) 3m ago 54m 95.4M 1559M 19.0.0-2879-g324baf8d 8717720dc5e5 170ce1e5fb8d 2024-04-12T21:42:16.088 INFO:teuthology.orchestra.run.smithi018.stdout:osd.2 smithi018 running (37m) 3m ago 53m 94.6M 1559M 19.0.0-2879-g324baf8d 8717720dc5e5 c2cbc6aa2839 2024-04-12T21:42:16.088 INFO:teuthology.orchestra.run.smithi018.stdout:osd.3 smithi186 running (36m) 3m ago 53m 101M 2581M 19.0.0-2879-g324baf8d 8717720dc5e5 0138e8262e4b 2024-04-12T21:42:16.088 INFO:teuthology.orchestra.run.smithi018.stdout:osd.4 smithi186 running (36m) 3m ago 52m 91.5M 2581M 19.0.0-2879-g324baf8d 8717720dc5e5 b20cb63b25c9 2024-04-12T21:42:16.088 INFO:teuthology.orchestra.run.smithi018.stdout:osd.5 smithi186 running (35m) 3m ago 52m 92.2M 2581M 19.0.0-2879-g324baf8d 8717720dc5e5 2513bea2610e 2024-04-12T21:42:16.088 INFO:teuthology.orchestra.run.smithi018.stdout:prometheus.smithi018 smithi018 *:9095 running (43m) 3m ago 56m 69.6M - 2.43.0 a07b618ecd1d f58841127e21 2024-04-12T21:42:16.090 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:16.089+0000 7f8d92200700 1 -- 172.21.15.18:0/3656031692 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f8d7006ec00 msgr2=0x7f8d700710b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:42:16.090 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:16.089+0000 7f8d92200700 1 --2- 172.21.15.18:0/3656031692 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f8d7006ec00 0x7f8d700710b0 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f8d8c115840 tx=0x7f8d74009e30 comp rx=0 tx=0).stop 2024-04-12T21:42:16.090 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:16.089+0000 7f8d92200700 1 -- 172.21.15.18:0/3656031692 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8d8c109430 msgr2=0x7f8d8c119fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:42:16.090 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:16.089+0000 7f8d92200700 1 --2- 172.21.15.18:0/3656031692 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8d8c109430 0x7f8d8c119fa0 secure :-1 s=READY pgs=1077 cs=0 l=1 rev1=1 crypto rx=0x7f8d7c00b790 tx=0x7f8d7c009250 comp rx=0 tx=0).stop 2024-04-12T21:42:16.090 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:16.090+0000 7f8d92200700 1 -- 172.21.15.18:0/3656031692 shutdown_connections 2024-04-12T21:42:16.090 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:16.090+0000 7f8d92200700 1 --2- 172.21.15.18:0/3656031692 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f8d7006ec00 0x7f8d700710b0 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:42:16.090 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:16.090+0000 7f8d92200700 1 --2- 172.21.15.18:0/3656031692 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8d8c109430 0x7f8d8c119fa0 unknown :-1 s=CLOSED pgs=1077 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:42:16.090 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:16.090+0000 7f8d92200700 1 --2- 172.21.15.18:0/3656031692 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8d8c109dd0 0x7f8d8c11a4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:42:16.090 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:16.090+0000 7f8d92200700 1 -- 172.21.15.18:0/3656031692 >> 172.21.15.18:0/3656031692 conn(0x7f8d8c072c00 msgr2=0x7f8d8c074b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:42:16.091 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:16.090+0000 7f8d92200700 1 -- 172.21.15.18:0/3656031692 shutdown_connections 2024-04-12T21:42:16.091 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:16.090+0000 7f8d92200700 1 -- 172.21.15.18:0/3656031692 wait complete. 2024-04-12T21:42:16.238 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:16.237+0000 7fb0a2e00700 1 -- 172.21.15.18:0/28549871 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb09c0fe820 msgr2=0x7fb09c106e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:42:16.239 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:16.237+0000 7fb0a2e00700 1 --2- 172.21.15.18:0/28549871 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb09c0fe820 0x7fb09c106e60 secure :-1 s=READY pgs=1078 cs=0 l=1 rev1=1 crypto rx=0x7fb088004940 tx=0x7fb088009e30 comp rx=0 tx=0).stop 2024-04-12T21:42:16.239 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:16.238+0000 7fb0a2e00700 1 -- 172.21.15.18:0/28549871 shutdown_connections 2024-04-12T21:42:16.239 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:16.238+0000 7fb0a2e00700 1 --2- 172.21.15.18:0/28549871 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb09c0fe820 0x7fb09c106e60 unknown :-1 s=CLOSED pgs=1078 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:42:16.239 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:16.238+0000 7fb0a2e00700 1 --2- 172.21.15.18:0/28549871 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb09c0fde80 0x7fb09c0fe250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:42:16.239 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:16.238+0000 7fb0a2e00700 1 -- 172.21.15.18:0/28549871 >> 172.21.15.18:0/28549871 conn(0x7fb09c0f78f0 msgr2=0x7fb09c0f9d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:42:16.239 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:16.238+0000 7fb0a2e00700 1 -- 172.21.15.18:0/28549871 shutdown_connections 2024-04-12T21:42:16.239 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:16.238+0000 7fb0a2e00700 1 -- 172.21.15.18:0/28549871 wait complete. 2024-04-12T21:42:16.239 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:16.238+0000 7fb0a2e00700 1 Processor -- start 2024-04-12T21:42:16.239 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:16.239+0000 7fb0a2e00700 1 -- start start 2024-04-12T21:42:16.241 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:16.239+0000 7fb0a2e00700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb09c0fde80 0x7fb09c115af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:42:16.241 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:16.239+0000 7fb0a2e00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb09c0fe820 0x7fb09c116030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:42:16.241 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:16.239+0000 7fb0a2e00700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb09c116680 con 0x7fb09c0fe820 2024-04-12T21:42:16.241 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:16.239+0000 7fb0a2e00700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb09c10fc60 con 0x7fb09c0fde80 2024-04-12T21:42:16.241 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:16.239+0000 7fb09b400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb09c0fe820 0x7fb09c116030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:42:16.241 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:16.239+0000 7fb09b400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb09c0fe820 0x7fb09c116030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:60696/0 (socket says 172.21.15.18:60696) 2024-04-12T21:42:16.241 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:16.239+0000 7fb09b400700 1 -- 172.21.15.18:0/2070970936 learned_addr learned my addr 172.21.15.18:0/2070970936 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:42:16.241 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:16.240+0000 7fb09be00700 1 --2- 172.21.15.18:0/2070970936 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb09c0fde80 0x7fb09c115af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:42:16.241 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:16.240+0000 7fb09b400700 1 -- 172.21.15.18:0/2070970936 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb09c0fde80 msgr2=0x7fb09c115af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:42:16.241 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:16.240+0000 7fb09b400700 1 --2- 172.21.15.18:0/2070970936 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb09c0fde80 0x7fb09c115af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:42:16.241 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:16.240+0000 7fb09b400700 1 -- 172.21.15.18:0/2070970936 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb088005040 con 0x7fb09c0fe820 2024-04-12T21:42:16.242 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:16.240+0000 7fb09b400700 1 --2- 172.21.15.18:0/2070970936 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb09c0fe820 0x7fb09c116030 secure :-1 s=READY pgs=1079 cs=0 l=1 rev1=1 crypto rx=0x7fb088000c00 tx=0x7fb088009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:42:16.242 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:16.240+0000 7fb093e00700 1 -- 172.21.15.18:0/2070970936 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb088021070 con 0x7fb09c0fe820 2024-04-12T21:42:16.242 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:16.241+0000 7fb093e00700 1 -- 172.21.15.18:0/2070970936 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb088013cd0 con 0x7fb09c0fe820 2024-04-12T21:42:16.242 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:16.241+0000 7fb0a2e00700 1 -- 172.21.15.18:0/2070970936 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb09c10fee0 con 0x7fb09c0fe820 2024-04-12T21:42:16.242 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:16.241+0000 7fb093e00700 1 -- 172.21.15.18:0/2070970936 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb08800e070 con 0x7fb09c0fe820 2024-04-12T21:42:16.242 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:16.241+0000 7fb0a2e00700 1 -- 172.21.15.18:0/2070970936 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb09c110420 con 0x7fb09c0fe820 2024-04-12T21:42:16.242 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:16.242+0000 7fb0a2e00700 1 -- 172.21.15.18:0/2070970936 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb09c1068d0 con 0x7fb09c0fe820 2024-04-12T21:42:16.243 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:16.243+0000 7fb093e00700 1 -- 172.21.15.18:0/2070970936 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7fb08801ca60 con 0x7fb09c0fe820 2024-04-12T21:42:16.244 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:16.244+0000 7fb093e00700 1 --2- 172.21.15.18:0/2070970936 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fb08006ec00 0x7fb0800710b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:42:16.244 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:16.244+0000 7fb093e00700 1 -- 172.21.15.18:0/2070970936 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 6757+0+0 (secure 0 0 0) 0x7fb08808fdd0 con 0x7fb09c0fe820 2024-04-12T21:42:16.244 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:16.244+0000 7fb09be00700 1 --2- 172.21.15.18:0/2070970936 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fb08006ec00 0x7fb0800710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:42:16.245 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:16.245+0000 7fb09be00700 1 --2- 172.21.15.18:0/2070970936 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fb08006ec00 0x7fb0800710b0 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7fb08c004600 tx=0x7fb08c00b040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:42:16.249 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:16.249+0000 7fb093e00700 1 -- 172.21.15.18:0/2070970936 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7fb08805c600 con 0x7fb09c0fe820 2024-04-12T21:42:16.568 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:16.567+0000 7fb0a2e00700 1 -- 172.21.15.18:0/2070970936 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fb09c111140 con 0x7fb09c0fe820 2024-04-12T21:42:16.568 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:16.568+0000 7fb093e00700 1 -- 172.21.15.18:0/2070970936 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+623 (secure 0 0 0) 0x7fb08805c190 con 0x7fb09c0fe820 2024-04-12T21:42:16.569 INFO:teuthology.orchestra.run.smithi018.stdout:{ 2024-04-12T21:42:16.569 INFO:teuthology.orchestra.run.smithi018.stdout: "mon": { 2024-04-12T21:42:16.569 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 19.0.0-2879-g324baf8d (324baf8ddb91dea46cac50cb535c306115791355) squid (dev)": 2 2024-04-12T21:42:16.569 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-12T21:42:16.569 INFO:teuthology.orchestra.run.smithi018.stdout: "mgr": { 2024-04-12T21:42:16.569 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 19.0.0-2879-g324baf8d (324baf8ddb91dea46cac50cb535c306115791355) squid (dev)": 2 2024-04-12T21:42:16.569 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-12T21:42:16.569 INFO:teuthology.orchestra.run.smithi018.stdout: "osd": { 2024-04-12T21:42:16.569 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 19.0.0-2879-g324baf8d (324baf8ddb91dea46cac50cb535c306115791355) squid (dev)": 6 2024-04-12T21:42:16.569 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-12T21:42:16.569 INFO:teuthology.orchestra.run.smithi018.stdout: "mds": { 2024-04-12T21:42:16.569 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 19.0.0-2879-g324baf8d (324baf8ddb91dea46cac50cb535c306115791355) squid (dev)": 4 2024-04-12T21:42:16.569 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-12T21:42:16.569 INFO:teuthology.orchestra.run.smithi018.stdout: "overall": { 2024-04-12T21:42:16.570 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 19.0.0-2879-g324baf8d (324baf8ddb91dea46cac50cb535c306115791355) squid (dev)": 14 2024-04-12T21:42:16.570 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-04-12T21:42:16.570 INFO:teuthology.orchestra.run.smithi018.stdout:} 2024-04-12T21:42:16.571 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:16.571+0000 7fb0a2e00700 1 -- 172.21.15.18:0/2070970936 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fb08006ec00 msgr2=0x7fb0800710b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:42:16.571 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:16.571+0000 7fb0a2e00700 1 --2- 172.21.15.18:0/2070970936 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fb08006ec00 0x7fb0800710b0 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7fb08c004600 tx=0x7fb08c00b040 comp rx=0 tx=0).stop 2024-04-12T21:42:16.571 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:16.571+0000 7fb0a2e00700 1 -- 172.21.15.18:0/2070970936 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb09c0fe820 msgr2=0x7fb09c116030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:42:16.571 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:16.571+0000 7fb0a2e00700 1 --2- 172.21.15.18:0/2070970936 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb09c0fe820 0x7fb09c116030 secure :-1 s=READY pgs=1079 cs=0 l=1 rev1=1 crypto rx=0x7fb088000c00 tx=0x7fb088009e30 comp rx=0 tx=0).stop 2024-04-12T21:42:16.571 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:16.571+0000 7fb0a2e00700 1 -- 172.21.15.18:0/2070970936 shutdown_connections 2024-04-12T21:42:16.572 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:16.571+0000 7fb0a2e00700 1 --2- 172.21.15.18:0/2070970936 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fb08006ec00 0x7fb0800710b0 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:42:16.572 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:16.571+0000 7fb0a2e00700 1 --2- 172.21.15.18:0/2070970936 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb09c0fde80 0x7fb09c115af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:42:16.572 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:16.571+0000 7fb0a2e00700 1 --2- 172.21.15.18:0/2070970936 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb09c0fe820 0x7fb09c116030 unknown :-1 s=CLOSED pgs=1079 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:42:16.572 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:16.571+0000 7fb0a2e00700 1 -- 172.21.15.18:0/2070970936 >> 172.21.15.18:0/2070970936 conn(0x7fb09c0f78f0 msgr2=0x7fb09c0fd3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:42:16.572 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:16.571+0000 7fb0a2e00700 1 -- 172.21.15.18:0/2070970936 shutdown_connections 2024-04-12T21:42:16.572 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:16.571+0000 7fb0a2e00700 1 -- 172.21.15.18:0/2070970936 wait complete. 2024-04-12T21:42:16.724 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:16.724+0000 7fab54e00700 1 -- 172.21.15.18:0/1326599863 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fab500fe830 msgr2=0x7fab50106f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:42:16.724 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:16.724+0000 7fab54e00700 1 --2- 172.21.15.18:0/1326599863 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fab500fe830 0x7fab50106f00 secure :-1 s=READY pgs=1080 cs=0 l=1 rev1=1 crypto rx=0x7fab38004940 tx=0x7fab38009e30 comp rx=0 tx=0).stop 2024-04-12T21:42:16.724 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:16.724+0000 7fab54e00700 1 -- 172.21.15.18:0/1326599863 shutdown_connections 2024-04-12T21:42:16.724 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:16.724+0000 7fab54e00700 1 --2- 172.21.15.18:0/1326599863 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fab500fe830 0x7fab50106f00 unknown :-1 s=CLOSED pgs=1080 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:42:16.724 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:16.724+0000 7fab54e00700 1 --2- 172.21.15.18:0/1326599863 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fab500fde90 0x7fab500fe260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:42:16.724 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:16.724+0000 7fab54e00700 1 -- 172.21.15.18:0/1326599863 >> 172.21.15.18:0/1326599863 conn(0x7fab50072bc0 msgr2=0x7fab50072fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:42:16.725 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:42:16 smithi018 ceph-mon[112590]: from='client.36216 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:42:16.725 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:42:16 smithi018 ceph-mon[112590]: from='client.36220 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:42:16.726 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:16.724+0000 7fab54e00700 1 -- 172.21.15.18:0/1326599863 shutdown_connections 2024-04-12T21:42:16.726 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:16.724+0000 7fab54e00700 1 -- 172.21.15.18:0/1326599863 wait complete. 2024-04-12T21:42:16.726 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:16.725+0000 7fab54e00700 1 Processor -- start 2024-04-12T21:42:16.726 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:16.725+0000 7fab54e00700 1 -- start start 2024-04-12T21:42:16.726 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:16.726+0000 7fab54e00700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fab500fde90 0x7fab50113ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:42:16.726 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:16.726+0000 7fab54e00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fab500fe830 0x7fab501140e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:42:16.726 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:16.726+0000 7fab54e00700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab50114730 con 0x7fab500fe830 2024-04-12T21:42:16.726 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:16.726+0000 7fab54e00700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab5010db20 con 0x7fab500fde90 2024-04-12T21:42:16.726 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:16.726+0000 7fab45400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fab500fe830 0x7fab501140e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:42:16.727 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:16.726+0000 7fab45400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fab500fe830 0x7fab501140e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:60722/0 (socket says 172.21.15.18:60722) 2024-04-12T21:42:16.727 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:16.726+0000 7fab45400700 1 -- 172.21.15.18:0/2898992796 learned_addr learned my addr 172.21.15.18:0/2898992796 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:42:16.728 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:16.726+0000 7fab4de00700 1 --2- 172.21.15.18:0/2898992796 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fab500fde90 0x7fab50113ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:42:16.728 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:16.726+0000 7fab45400700 1 -- 172.21.15.18:0/2898992796 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fab500fde90 msgr2=0x7fab50113ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:42:16.728 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:16.726+0000 7fab45400700 1 --2- 172.21.15.18:0/2898992796 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fab500fde90 0x7fab50113ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:42:16.728 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:16.726+0000 7fab45400700 1 -- 172.21.15.18:0/2898992796 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab38005040 con 0x7fab500fe830 2024-04-12T21:42:16.728 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:16.726+0000 7fab4de00700 1 --2- 172.21.15.18:0/2898992796 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fab500fde90 0x7fab50113ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-12T21:42:16.728 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:16.727+0000 7fab45400700 1 --2- 172.21.15.18:0/2898992796 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fab500fe830 0x7fab501140e0 secure :-1 s=READY pgs=1081 cs=0 l=1 rev1=1 crypto rx=0x7fab380093b0 tx=0x7fab38009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:42:16.728 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:16.727+0000 7fab46a00700 1 -- 172.21.15.18:0/2898992796 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fab3800e040 con 0x7fab500fe830 2024-04-12T21:42:16.728 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:16.727+0000 7fab46a00700 1 -- 172.21.15.18:0/2898992796 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fab38004290 con 0x7fab500fe830 2024-04-12T21:42:16.728 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:16.727+0000 7fab54e00700 1 -- 172.21.15.18:0/2898992796 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fab5010dda0 con 0x7fab500fe830 2024-04-12T21:42:16.728 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:16.727+0000 7fab46a00700 1 -- 172.21.15.18:0/2898992796 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fab3801c3f0 con 0x7fab500fe830 2024-04-12T21:42:16.728 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:16.727+0000 7fab54e00700 1 -- 172.21.15.18:0/2898992796 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fab5010e2e0 con 0x7fab500fe830 2024-04-12T21:42:16.729 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:16.728+0000 7fab54e00700 1 -- 172.21.15.18:0/2898992796 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fab50065e30 con 0x7fab500fe830 2024-04-12T21:42:16.729 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:16.730+0000 7fab46a00700 1 -- 172.21.15.18:0/2898992796 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7fab38013890 con 0x7fab500fe830 2024-04-12T21:42:16.731 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:16.730+0000 7fab46a00700 1 --2- 172.21.15.18:0/2898992796 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fab3406ec10 0x7fab340710c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:42:16.731 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:16.731+0000 7fab46a00700 1 -- 172.21.15.18:0/2898992796 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 6757+0+0 (secure 0 0 0) 0x7fab3808fb60 con 0x7fab500fe830 2024-04-12T21:42:16.731 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:16.731+0000 7fab4de00700 1 --2- 172.21.15.18:0/2898992796 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fab3406ec10 0x7fab340710c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:42:16.731 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:16.731+0000 7fab4de00700 1 --2- 172.21.15.18:0/2898992796 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fab3406ec10 0x7fab340710c0 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7fab40007fd0 tx=0x7fab40009040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:42:16.735 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:16.735+0000 7fab46a00700 1 -- 172.21.15.18:0/2898992796 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7fab3805c450 con 0x7fab500fe830 2024-04-12T21:42:16.783 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:42:16 smithi186 ceph-mon[86379]: from='client.36216 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:42:16.783 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:42:16 smithi186 ceph-mon[86379]: from='client.36220 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:42:17.004 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:17.004+0000 7fab54e00700 1 -- 172.21.15.18:0/2898992796 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fab5010df30 con 0x7fab500fe830 2024-04-12T21:42:17.004 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:17.004+0000 7fab46a00700 1 -- 172.21.15.18:0/2898992796 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 33 v33) v1 ==== 76+0+1876 (secure 0 0 0) 0x7fab5010df30 con 0x7fab500fe830 2024-04-12T21:42:17.006 INFO:teuthology.orchestra.run.smithi018.stdout:e33 2024-04-12T21:42:17.006 INFO:teuthology.orchestra.run.smithi018.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-12T21:42:17.006 INFO:teuthology.orchestra.run.smithi018.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-12T21:42:17.006 INFO:teuthology.orchestra.run.smithi018.stdout:legacy client fscid: 1 2024-04-12T21:42:17.006 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-12T21:42:17.006 INFO:teuthology.orchestra.run.smithi018.stdout:Filesystem 'cephfs' (1) 2024-04-12T21:42:17.006 INFO:teuthology.orchestra.run.smithi018.stdout:fs_name cephfs 2024-04-12T21:42:17.006 INFO:teuthology.orchestra.run.smithi018.stdout:epoch 24 2024-04-12T21:42:17.007 INFO:teuthology.orchestra.run.smithi018.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-04-12T21:42:17.007 INFO:teuthology.orchestra.run.smithi018.stdout:created 2024-04-12T20:51:57.362850+0000 2024-04-12T21:42:17.007 INFO:teuthology.orchestra.run.smithi018.stdout:modified 2024-04-12T21:06:59.717190+0000 2024-04-12T21:42:17.007 INFO:teuthology.orchestra.run.smithi018.stdout:tableserver 0 2024-04-12T21:42:17.007 INFO:teuthology.orchestra.run.smithi018.stdout:root 0 2024-04-12T21:42:17.007 INFO:teuthology.orchestra.run.smithi018.stdout:session_timeout 60 2024-04-12T21:42:17.007 INFO:teuthology.orchestra.run.smithi018.stdout:session_autoclose 300 2024-04-12T21:42:17.007 INFO:teuthology.orchestra.run.smithi018.stdout:max_file_size 1099511627776 2024-04-12T21:42:17.007 INFO:teuthology.orchestra.run.smithi018.stdout:max_xattr_size 65536 2024-04-12T21:42:17.007 INFO:teuthology.orchestra.run.smithi018.stdout:required_client_features {} 2024-04-12T21:42:17.007 INFO:teuthology.orchestra.run.smithi018.stdout:last_failure 0 2024-04-12T21:42:17.007 INFO:teuthology.orchestra.run.smithi018.stdout:last_failure_osd_epoch 100 2024-04-12T21:42:17.007 INFO:teuthology.orchestra.run.smithi018.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-12T21:42:17.007 INFO:teuthology.orchestra.run.smithi018.stdout:max_mds 2 2024-04-12T21:42:17.007 INFO:teuthology.orchestra.run.smithi018.stdout:in 0,1 2024-04-12T21:42:17.007 INFO:teuthology.orchestra.run.smithi018.stdout:up {} 2024-04-12T21:42:17.007 INFO:teuthology.orchestra.run.smithi018.stdout:failed 0,1 2024-04-12T21:42:17.008 INFO:teuthology.orchestra.run.smithi018.stdout:damaged 2024-04-12T21:42:17.008 INFO:teuthology.orchestra.run.smithi018.stdout:stopped 2024-04-12T21:42:17.008 INFO:teuthology.orchestra.run.smithi018.stdout:data_pools [3] 2024-04-12T21:42:17.008 INFO:teuthology.orchestra.run.smithi018.stdout:metadata_pool 2 2024-04-12T21:42:17.008 INFO:teuthology.orchestra.run.smithi018.stdout:inline_data disabled 2024-04-12T21:42:17.008 INFO:teuthology.orchestra.run.smithi018.stdout:balancer 2024-04-12T21:42:17.008 INFO:teuthology.orchestra.run.smithi018.stdout:bal_rank_mask -1 2024-04-12T21:42:17.008 INFO:teuthology.orchestra.run.smithi018.stdout:standby_count_wanted 1 2024-04-12T21:42:17.008 INFO:teuthology.orchestra.run.smithi018.stdout:qdb_cluster leader: 0 members: 2024-04-12T21:42:17.008 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-12T21:42:17.008 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-12T21:42:17.008 INFO:teuthology.orchestra.run.smithi018.stdout:Standby daemons: 2024-04-12T21:42:17.008 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-12T21:42:17.008 INFO:teuthology.orchestra.run.smithi018.stdout:[mds.cephfs.smithi018.acqbsc{-1:34436} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.18:6828/755179764,v1:172.21.15.18:6829/755179764] compat {c=[1],r=[1],i=[1fff]}] 2024-04-12T21:42:17.008 INFO:teuthology.orchestra.run.smithi018.stdout:[mds.cephfs.smithi018.szjluo{-1:34468} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.18:6826/2407992653,v1:172.21.15.18:6827/2407992653] compat {c=[1],r=[1],i=[1fff]}] 2024-04-12T21:42:17.008 INFO:teuthology.orchestra.run.smithi018.stdout:[mds.cephfs.smithi186.jdllyn{-1:44595} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.186:6824/3523444202,v1:172.21.15.186:6825/3523444202] compat {c=[1],r=[1],i=[1fff]}] 2024-04-12T21:42:17.009 INFO:teuthology.orchestra.run.smithi018.stdout:[mds.cephfs.smithi186.wjnafs{-1:44875} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.186:6826/3122443766,v1:172.21.15.186:6827/3122443766] compat {c=[1],r=[1],i=[1fff]}] 2024-04-12T21:42:17.009 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:17.008+0000 7fab54e00700 1 -- 172.21.15.18:0/2898992796 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fab3406ec10 msgr2=0x7fab340710c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:42:17.009 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:17.008+0000 7fab54e00700 1 --2- 172.21.15.18:0/2898992796 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fab3406ec10 0x7fab340710c0 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7fab40007fd0 tx=0x7fab40009040 comp rx=0 tx=0).stop 2024-04-12T21:42:17.009 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:17.008+0000 7fab54e00700 1 -- 172.21.15.18:0/2898992796 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fab500fe830 msgr2=0x7fab501140e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:42:17.009 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:17.008+0000 7fab54e00700 1 --2- 172.21.15.18:0/2898992796 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fab500fe830 0x7fab501140e0 secure :-1 s=READY pgs=1081 cs=0 l=1 rev1=1 crypto rx=0x7fab380093b0 tx=0x7fab38009e30 comp rx=0 tx=0).stop 2024-04-12T21:42:17.009 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:17.008+0000 7fab54e00700 1 -- 172.21.15.18:0/2898992796 shutdown_connections 2024-04-12T21:42:17.009 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:17.008+0000 7fab54e00700 1 --2- 172.21.15.18:0/2898992796 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fab3406ec10 0x7fab340710c0 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:42:17.009 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:17.008+0000 7fab54e00700 1 --2- 172.21.15.18:0/2898992796 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fab500fde90 0x7fab50113ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:42:17.010 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:17.008+0000 7fab54e00700 1 --2- 172.21.15.18:0/2898992796 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fab500fe830 0x7fab501140e0 unknown :-1 s=CLOSED pgs=1081 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:42:17.010 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:17.008+0000 7fab54e00700 1 -- 172.21.15.18:0/2898992796 >> 172.21.15.18:0/2898992796 conn(0x7fab50072bc0 msgr2=0x7fab501037b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:42:17.010 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:17.008+0000 7fab54e00700 1 -- 172.21.15.18:0/2898992796 shutdown_connections 2024-04-12T21:42:17.010 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:17.009+0000 7fab54e00700 1 -- 172.21.15.18:0/2898992796 wait complete. 2024-04-12T21:42:17.010 INFO:teuthology.orchestra.run.smithi018.stderr:dumped fsmap epoch 33 2024-04-12T21:42:17.155 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:17.155+0000 7ff5fac00700 1 -- 172.21.15.18:0/4008107904 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff5f4102e10 msgr2=0x7ff5f41031e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:42:17.156 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:17.155+0000 7ff5fac00700 1 --2- 172.21.15.18:0/4008107904 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff5f4102e10 0x7ff5f41031e0 secure :-1 s=READY pgs=1082 cs=0 l=1 rev1=1 crypto rx=0x7ff5e4004940 tx=0x7ff5e4009e30 comp rx=0 tx=0).stop 2024-04-12T21:42:17.156 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:17.155+0000 7ff5fac00700 1 -- 172.21.15.18:0/4008107904 shutdown_connections 2024-04-12T21:42:17.156 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:17.155+0000 7ff5fac00700 1 --2- 172.21.15.18:0/4008107904 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff5f41037b0 0x7ff5f40fb3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:42:17.156 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:17.155+0000 7ff5fac00700 1 --2- 172.21.15.18:0/4008107904 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff5f4102e10 0x7ff5f41031e0 unknown :-1 s=CLOSED pgs=1082 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:42:17.156 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:17.155+0000 7ff5fac00700 1 -- 172.21.15.18:0/4008107904 >> 172.21.15.18:0/4008107904 conn(0x7ff5f4072310 msgr2=0x7ff5f4072710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:42:17.156 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:17.155+0000 7ff5fac00700 1 -- 172.21.15.18:0/4008107904 shutdown_connections 2024-04-12T21:42:17.156 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:17.155+0000 7ff5fac00700 1 -- 172.21.15.18:0/4008107904 wait complete. 2024-04-12T21:42:17.157 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:17.156+0000 7ff5fac00700 1 Processor -- start 2024-04-12T21:42:17.157 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:17.156+0000 7ff5fac00700 1 -- start start 2024-04-12T21:42:17.157 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:17.157+0000 7ff5fac00700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff5f4102e10 0x7ff5f4115be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:42:17.157 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:17.157+0000 7ff5fac00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff5f41037b0 0x7ff5f4116120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:42:17.157 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:17.157+0000 7ff5fac00700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5f4116800 con 0x7ff5f41037b0 2024-04-12T21:42:17.157 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:17.157+0000 7ff5fac00700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5f410fd90 con 0x7ff5f4102e10 2024-04-12T21:42:17.158 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:17.157+0000 7ff5ebe00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff5f41037b0 0x7ff5f4116120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:42:17.158 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:17.157+0000 7ff5ebe00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff5f41037b0 0x7ff5f4116120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:60742/0 (socket says 172.21.15.18:60742) 2024-04-12T21:42:17.158 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:17.157+0000 7ff5ebe00700 1 -- 172.21.15.18:0/1148021779 learned_addr learned my addr 172.21.15.18:0/1148021779 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:42:17.158 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:17.157+0000 7ff5f3e00700 1 --2- 172.21.15.18:0/1148021779 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff5f4102e10 0x7ff5f4115be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:42:17.158 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:17.157+0000 7ff5ebe00700 1 -- 172.21.15.18:0/1148021779 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff5f4102e10 msgr2=0x7ff5f4115be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:42:17.158 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:17.157+0000 7ff5ebe00700 1 --2- 172.21.15.18:0/1148021779 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff5f4102e10 0x7ff5f4115be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:42:17.158 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:17.157+0000 7ff5ebe00700 1 -- 172.21.15.18:0/1148021779 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5e4005040 con 0x7ff5f41037b0 2024-04-12T21:42:17.159 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:17.158+0000 7ff5ebe00700 1 --2- 172.21.15.18:0/1148021779 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff5f41037b0 0x7ff5f4116120 secure :-1 s=READY pgs=1083 cs=0 l=1 rev1=1 crypto rx=0x7ff5dc00b790 tx=0x7ff5dc009220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:42:17.159 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:17.158+0000 7ff5f0c00700 1 -- 172.21.15.18:0/1148021779 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff5dc00f6f0 con 0x7ff5f41037b0 2024-04-12T21:42:17.159 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:17.158+0000 7ff5f0c00700 1 -- 172.21.15.18:0/1148021779 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff5dc00fd20 con 0x7ff5f41037b0 2024-04-12T21:42:17.159 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:17.158+0000 7ff5fac00700 1 -- 172.21.15.18:0/1148021779 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff5f4110070 con 0x7ff5f41037b0 2024-04-12T21:42:17.159 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:17.158+0000 7ff5f0c00700 1 -- 172.21.15.18:0/1148021779 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff5dc0103a0 con 0x7ff5f41037b0 2024-04-12T21:42:17.159 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:17.158+0000 7ff5fac00700 1 -- 172.21.15.18:0/1148021779 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff5f41105b0 con 0x7ff5f41037b0 2024-04-12T21:42:17.159 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:17.159+0000 7ff5fac00700 1 -- 172.21.15.18:0/1148021779 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff5f40fe6d0 con 0x7ff5f41037b0 2024-04-12T21:42:17.160 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:17.160+0000 7ff5f0c00700 1 -- 172.21.15.18:0/1148021779 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7ff5dc011400 con 0x7ff5f41037b0 2024-04-12T21:42:17.161 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:17.161+0000 7ff5f0c00700 1 --2- 172.21.15.18:0/1148021779 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7ff5d806ec00 0x7ff5d80710b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:42:17.161 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:17.161+0000 7ff5f0c00700 1 -- 172.21.15.18:0/1148021779 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 6757+0+0 (secure 0 0 0) 0x7ff5dc08e8f0 con 0x7ff5f41037b0 2024-04-12T21:42:17.161 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:17.161+0000 7ff5f3e00700 1 --2- 172.21.15.18:0/1148021779 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7ff5d806ec00 0x7ff5d80710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:42:17.162 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:17.162+0000 7ff5f3e00700 1 --2- 172.21.15.18:0/1148021779 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7ff5d806ec00 0x7ff5d80710b0 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7ff5e4004f90 tx=0x7ff5e4009e30 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:42:17.174 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:17.174+0000 7ff5f0c00700 1 -- 172.21.15.18:0/1148021779 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7ff5dc0574f0 con 0x7ff5f41037b0 2024-04-12T21:42:17.412 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:17.412+0000 7ff5fac00700 1 -- 172.21.15.18:0/1148021779 --> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7ff5f410a020 con 0x7ff5d806ec00 2024-04-12T21:42:17.414 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:17.414+0000 7ff5f0c00700 1 -- 172.21.15.18:0/1148021779 <== mgr.34102 v2:172.21.15.18:6800/3812891919 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7ff5f410a020 con 0x7ff5d806ec00 2024-04-12T21:42:17.415 INFO:teuthology.orchestra.run.smithi018.stdout:{ 2024-04-12T21:42:17.415 INFO:teuthology.orchestra.run.smithi018.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:f4452def3be4d350f4a8e5feaf7ebb1511be9298fe68478ba7a2d6fbd7d69a63", 2024-04-12T21:42:17.415 INFO:teuthology.orchestra.run.smithi018.stdout: "in_progress": true, 2024-04-12T21:42:17.415 INFO:teuthology.orchestra.run.smithi018.stdout: "which": "Upgrading all daemon types on all hosts", 2024-04-12T21:42:17.415 INFO:teuthology.orchestra.run.smithi018.stdout: "services_complete": [ 2024-04-12T21:42:17.415 INFO:teuthology.orchestra.run.smithi018.stdout: "mgr", 2024-04-12T21:42:17.415 INFO:teuthology.orchestra.run.smithi018.stdout: "mds", 2024-04-12T21:42:17.415 INFO:teuthology.orchestra.run.smithi018.stdout: "crash", 2024-04-12T21:42:17.415 INFO:teuthology.orchestra.run.smithi018.stdout: "osd", 2024-04-12T21:42:17.415 INFO:teuthology.orchestra.run.smithi018.stdout: "mon" 2024-04-12T21:42:17.415 INFO:teuthology.orchestra.run.smithi018.stdout: ], 2024-04-12T21:42:17.415 INFO:teuthology.orchestra.run.smithi018.stdout: "progress": "16/21 daemons upgraded", 2024-04-12T21:42:17.415 INFO:teuthology.orchestra.run.smithi018.stdout: "message": "Currently upgrading mds daemons", 2024-04-12T21:42:17.415 INFO:teuthology.orchestra.run.smithi018.stdout: "is_paused": false 2024-04-12T21:42:17.416 INFO:teuthology.orchestra.run.smithi018.stdout:} 2024-04-12T21:42:17.417 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:17.417+0000 7ff5fac00700 1 -- 172.21.15.18:0/1148021779 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7ff5d806ec00 msgr2=0x7ff5d80710b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:42:17.417 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:17.417+0000 7ff5fac00700 1 --2- 172.21.15.18:0/1148021779 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7ff5d806ec00 0x7ff5d80710b0 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7ff5e4004f90 tx=0x7ff5e4009e30 comp rx=0 tx=0).stop 2024-04-12T21:42:17.418 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:17.417+0000 7ff5fac00700 1 -- 172.21.15.18:0/1148021779 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff5f41037b0 msgr2=0x7ff5f4116120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:42:17.418 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:17.417+0000 7ff5fac00700 1 --2- 172.21.15.18:0/1148021779 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff5f41037b0 0x7ff5f4116120 secure :-1 s=READY pgs=1083 cs=0 l=1 rev1=1 crypto rx=0x7ff5dc00b790 tx=0x7ff5dc009220 comp rx=0 tx=0).stop 2024-04-12T21:42:17.418 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:17.417+0000 7ff5fac00700 1 -- 172.21.15.18:0/1148021779 shutdown_connections 2024-04-12T21:42:17.418 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:17.417+0000 7ff5fac00700 1 --2- 172.21.15.18:0/1148021779 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7ff5d806ec00 0x7ff5d80710b0 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:42:17.418 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:17.417+0000 7ff5fac00700 1 --2- 172.21.15.18:0/1148021779 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff5f4102e10 0x7ff5f4115be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:42:17.418 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:17.417+0000 7ff5fac00700 1 --2- 172.21.15.18:0/1148021779 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff5f41037b0 0x7ff5f4116120 unknown :-1 s=CLOSED pgs=1083 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:42:17.418 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:17.417+0000 7ff5fac00700 1 -- 172.21.15.18:0/1148021779 >> 172.21.15.18:0/1148021779 conn(0x7ff5f4072310 msgr2=0x7ff5f40f9d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:42:17.418 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:17.417+0000 7ff5fac00700 1 -- 172.21.15.18:0/1148021779 shutdown_connections 2024-04-12T21:42:17.418 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:17.418+0000 7ff5fac00700 1 -- 172.21.15.18:0/1148021779 wait complete. 2024-04-12T21:42:17.576 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:17.576+0000 7f3974400700 1 -- 172.21.15.18:0/765643176 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f396c105020 msgr2=0x7f396c1053f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:42:17.577 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:17.576+0000 7f3974400700 1 --2- 172.21.15.18:0/765643176 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f396c105020 0x7f396c1053f0 secure :-1 s=READY pgs=1084 cs=0 l=1 rev1=1 crypto rx=0x7f3960004940 tx=0x7f3960009e30 comp rx=0 tx=0).stop 2024-04-12T21:42:17.577 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:17.576+0000 7f3974400700 1 -- 172.21.15.18:0/765643176 shutdown_connections 2024-04-12T21:42:17.577 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:17.576+0000 7f3974400700 1 --2- 172.21.15.18:0/765643176 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f396c1059c0 0x7f396c0ff080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:42:17.577 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:17.576+0000 7f3974400700 1 --2- 172.21.15.18:0/765643176 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f396c105020 0x7f396c1053f0 unknown :-1 s=CLOSED pgs=1084 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:42:17.577 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:17.576+0000 7f3974400700 1 -- 172.21.15.18:0/765643176 >> 172.21.15.18:0/765643176 conn(0x7f396c0faba0 msgr2=0x7f396c0fcfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:42:17.577 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:17.576+0000 7f3974400700 1 -- 172.21.15.18:0/765643176 shutdown_connections 2024-04-12T21:42:17.577 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:17.576+0000 7f3974400700 1 -- 172.21.15.18:0/765643176 wait complete. 2024-04-12T21:42:17.577 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:17.578+0000 7f3974400700 1 Processor -- start 2024-04-12T21:42:17.578 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:17.578+0000 7f3974400700 1 -- start start 2024-04-12T21:42:17.579 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:17.579+0000 7f3974400700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f396c105020 0x7f396c115b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:42:17.579 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:17.579+0000 7f3974400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f396c1059c0 0x7f396c1160d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:42:17.579 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:17.579+0000 7f3974400700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f396c116720 con 0x7f396c1059c0 2024-04-12T21:42:17.579 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:17.579+0000 7f3974400700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f396c10fd00 con 0x7f396c105020 2024-04-12T21:42:17.579 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:17.579+0000 7f3971000700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f396c1059c0 0x7f396c1160d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:42:17.579 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:17.579+0000 7f3971000700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f396c1059c0 0x7f396c1160d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:60762/0 (socket says 172.21.15.18:60762) 2024-04-12T21:42:17.579 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:17.579+0000 7f3971000700 1 -- 172.21.15.18:0/4149589550 learned_addr learned my addr 172.21.15.18:0/4149589550 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:42:17.580 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:17.579+0000 7f3971a00700 1 --2- 172.21.15.18:0/4149589550 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f396c105020 0x7f396c115b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:42:17.580 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:17.579+0000 7f3971000700 1 -- 172.21.15.18:0/4149589550 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f396c105020 msgr2=0x7f396c115b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:42:17.580 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:17.579+0000 7f3971000700 1 --2- 172.21.15.18:0/4149589550 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f396c105020 0x7f396c115b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:42:17.580 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:17.579+0000 7f3971000700 1 -- 172.21.15.18:0/4149589550 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3960005040 con 0x7f396c1059c0 2024-04-12T21:42:17.581 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:17.580+0000 7f3971000700 1 --2- 172.21.15.18:0/4149589550 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f396c1059c0 0x7f396c1160d0 secure :-1 s=READY pgs=1085 cs=0 l=1 rev1=1 crypto rx=0x7f395800c890 tx=0x7f39580044c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:42:17.581 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:17.580+0000 7f3969600700 1 -- 172.21.15.18:0/4149589550 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f395800a7d0 con 0x7f396c1059c0 2024-04-12T21:42:17.581 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:17.580+0000 7f3969600700 1 -- 172.21.15.18:0/4149589550 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f395800ae00 con 0x7f396c1059c0 2024-04-12T21:42:17.581 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:17.580+0000 7f3969600700 1 -- 172.21.15.18:0/4149589550 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f39580113a0 con 0x7f396c1059c0 2024-04-12T21:42:17.581 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:17.580+0000 7f3974400700 1 -- 172.21.15.18:0/4149589550 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f396c10ffe0 con 0x7f396c1059c0 2024-04-12T21:42:17.581 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:17.580+0000 7f3974400700 1 -- 172.21.15.18:0/4149589550 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f396c1105a0 con 0x7f396c1059c0 2024-04-12T21:42:17.581 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:17.581+0000 7f3974400700 1 -- 172.21.15.18:0/4149589550 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f396c100770 con 0x7f396c1059c0 2024-04-12T21:42:17.582 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:17.582+0000 7f3969600700 1 -- 172.21.15.18:0/4149589550 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7f3958012400 con 0x7f396c1059c0 2024-04-12T21:42:17.584 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:17.584+0000 7f3969600700 1 --2- 172.21.15.18:0/4149589550 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f395406ec00 0x7f39540710b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:42:17.584 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:17.584+0000 7f3969600700 1 -- 172.21.15.18:0/4149589550 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 6757+0+0 (secure 0 0 0) 0x7f395808eae0 con 0x7f396c1059c0 2024-04-12T21:42:17.584 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:17.584+0000 7f3971a00700 1 --2- 172.21.15.18:0/4149589550 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f395406ec00 0x7f39540710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:42:17.585 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:17.585+0000 7f3971a00700 1 --2- 172.21.15.18:0/4149589550 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f395406ec00 0x7f39540710b0 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f396000b020 tx=0x7f3960009e30 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:42:17.588 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:17.588+0000 7f3969600700 1 -- 172.21.15.18:0/4149589550 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f395805b290 con 0x7f396c1059c0 2024-04-12T21:42:17.783 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:42:17 smithi186 ceph-mon[86379]: from='client.36224 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:42:17.784 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:42:17 smithi186 ceph-mon[86379]: pgmap v1278: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:42:17.784 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:42:17 smithi186 ceph-mon[86379]: from='client.? 172.21.15.18:0/2070970936' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T21:42:17.784 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:42:17 smithi186 ceph-mon[86379]: from='client.? 172.21.15.18:0/2898992796' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-12T21:42:17.855 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:42:17 smithi018 ceph-mon[112590]: from='client.36224 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:42:17.855 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:42:17 smithi018 ceph-mon[112590]: pgmap v1278: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:42:17.855 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:42:17 smithi018 ceph-mon[112590]: from='client.? 172.21.15.18:0/2070970936' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T21:42:17.855 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:42:17 smithi018 ceph-mon[112590]: from='client.? 172.21.15.18:0/2898992796' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-12T21:42:17.906 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:17.905+0000 7f3974400700 1 -- 172.21.15.18:0/4149589550 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f396c10c120 con 0x7f396c1059c0 2024-04-12T21:42:17.906 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:17.906+0000 7f3969600700 1 -- 172.21.15.18:0/4149589550 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+4303 (secure 0 0 0) 0x7f395805ae20 con 0x7f396c1059c0 2024-04-12T21:42:17.907 INFO:teuthology.orchestra.run.smithi018.stdout:HEALTH_ERR failed to probe daemons or devices; 1 filesystem is degraded; 1 filesystem is offline 2024-04-12T21:42:17.907 INFO:teuthology.orchestra.run.smithi018.stdout:[WRN] CEPHADM_REFRESH_FAILED: failed to probe daemons or devices 2024-04-12T21:42:17.907 INFO:teuthology.orchestra.run.smithi018.stdout: host smithi186 `cephadm ceph-volume` failed: cephadm exited with an error code: 1, stderr: Inferring config /var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/mon.smithi186/config 2024-04-12T21:42:17.907 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/podman: timeout after 895 seconds 2024-04-12T21:42:17.907 INFO:teuthology.orchestra.run.smithi018.stdout:Non-zero exit code 124 from /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:f4452def3be4d350f4a8e5feaf7ebb1511be9298fe68478ba7a2d6fbd7d69a63 -e NODE_NAME=smithi186 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968:/var/run/ceph:z -v /var/log/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968:/var/log/ceph:z -v /var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmptgom53f9:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:f4452def3be4d350f4a8e5feaf7ebb1511be9298fe68478ba7a2d6fbd7d69a63 inventory --format=json-pretty --filter-for-batch 2024-04-12T21:42:17.907 INFO:teuthology.orchestra.run.smithi018.stdout:Traceback (most recent call last): 2024-04-12T21:42:17.907 INFO:teuthology.orchestra.run.smithi018.stdout: File "/usr/lib64/python3.9/runpy.py", line 197, in _run_module_as_main 2024-04-12T21:42:17.907 INFO:teuthology.orchestra.run.smithi018.stdout: return _run_code(code, main_globals, None, 2024-04-12T21:42:17.908 INFO:teuthology.orchestra.run.smithi018.stdout: File "/usr/lib64/python3.9/runpy.py", line 87, in _run_code 2024-04-12T21:42:17.908 INFO:teuthology.orchestra.run.smithi018.stdout: exec(code, run_globals) 2024-04-12T21:42:17.908 INFO:teuthology.orchestra.run.smithi018.stdout: File "/var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/cephadm.e44f3ed5e34e4ef52eedef233d1190bff004470d8e8005127b5312bff357fa7f/__main__.py", line 5618, in 2024-04-12T21:42:17.908 INFO:teuthology.orchestra.run.smithi018.stdout: File "/var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/cephadm.e44f3ed5e34e4ef52eedef233d1190bff004470d8e8005127b5312bff357fa7f/__main__.py", line 5606, in main 2024-04-12T21:42:17.908 INFO:teuthology.orchestra.run.smithi018.stdout: File "/var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/cephadm.e44f3ed5e34e4ef52eedef233d1190bff004470d8e8005127b5312bff357fa7f/__main__.py", line 408, in _infer_config 2024-04-12T21:42:17.908 INFO:teuthology.orchestra.run.smithi018.stdout: File "/var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/cephadm.e44f3ed5e34e4ef52eedef233d1190bff004470d8e8005127b5312bff357fa7f/__main__.py", line 323, in _infer_fsid 2024-04-12T21:42:17.908 INFO:teuthology.orchestra.run.smithi018.stdout: File "/var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/cephadm.e44f3ed5e34e4ef52eedef233d1190bff004470d8e8005127b5312bff357fa7f/__main__.py", line 436, in _infer_image 2024-04-12T21:42:17.908 INFO:teuthology.orchestra.run.smithi018.stdout: File "/var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/cephadm.e44f3ed5e34e4ef52eedef233d1190bff004470d8e8005127b5312bff357fa7f/__main__.py", line 310, in _validate_fsid 2024-04-12T21:42:17.908 INFO:teuthology.orchestra.run.smithi018.stdout: File "/var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/cephadm.e44f3ed5e34e4ef52eedef233d1190bff004470d8e8005127b5312bff357fa7f/__main__.py", line 3359, in command_ceph_volume 2024-04-12T21:42:17.908 INFO:teuthology.orchestra.run.smithi018.stdout: File "/var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/cephadm.e44f3ed5e34e4ef52eedef233d1190bff004470d8e8005127b5312bff357fa7f/cephadmlib/call_wrappers.py", line 310, in call_throws 2024-04-12T21:42:17.908 INFO:teuthology.orchestra.run.smithi018.stdout:RuntimeError: Failed command: /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:f4452def3be4d350f4a8e5feaf7ebb1511be9298fe68478ba7a2d6fbd7d69a63 -e NODE_NAME=smithi186 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968:/var/run/ceph:z -v /var/log/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968:/var/log/ceph:z -v /var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmptgom53f9:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:f4452def3be4d350f4a8e5feaf7ebb1511be9298fe68478ba7a2d6fbd7d69a63 inventory --format=json-pretty --filter-for-batch 2024-04-12T21:42:17.908 INFO:teuthology.orchestra.run.smithi018.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-04-12T21:42:17.908 INFO:teuthology.orchestra.run.smithi018.stdout: fs cephfs is degraded 2024-04-12T21:42:17.908 INFO:teuthology.orchestra.run.smithi018.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-04-12T21:42:17.908 INFO:teuthology.orchestra.run.smithi018.stdout: fs cephfs is offline because no MDS is active for it. 2024-04-12T21:42:17.909 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:17.909+0000 7f3974400700 1 -- 172.21.15.18:0/4149589550 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f395406ec00 msgr2=0x7f39540710b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:42:17.909 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:17.909+0000 7f3974400700 1 --2- 172.21.15.18:0/4149589550 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f395406ec00 0x7f39540710b0 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f396000b020 tx=0x7f3960009e30 comp rx=0 tx=0).stop 2024-04-12T21:42:17.909 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:17.909+0000 7f3974400700 1 -- 172.21.15.18:0/4149589550 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f396c1059c0 msgr2=0x7f396c1160d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:42:17.909 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:17.909+0000 7f3974400700 1 --2- 172.21.15.18:0/4149589550 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f396c1059c0 0x7f396c1160d0 secure :-1 s=READY pgs=1085 cs=0 l=1 rev1=1 crypto rx=0x7f395800c890 tx=0x7f39580044c0 comp rx=0 tx=0).stop 2024-04-12T21:42:17.910 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:17.909+0000 7f3974400700 1 -- 172.21.15.18:0/4149589550 shutdown_connections 2024-04-12T21:42:17.910 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:17.909+0000 7f3974400700 1 --2- 172.21.15.18:0/4149589550 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f395406ec00 0x7f39540710b0 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:42:17.910 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:17.909+0000 7f3974400700 1 --2- 172.21.15.18:0/4149589550 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f396c105020 0x7f396c115b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:42:17.910 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:17.909+0000 7f3974400700 1 --2- 172.21.15.18:0/4149589550 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f396c1059c0 0x7f396c1160d0 unknown :-1 s=CLOSED pgs=1085 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:42:17.910 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:17.909+0000 7f3974400700 1 -- 172.21.15.18:0/4149589550 >> 172.21.15.18:0/4149589550 conn(0x7f396c0faba0 msgr2=0x7f396c0fbb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:42:17.910 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:17.909+0000 7f3974400700 1 -- 172.21.15.18:0/4149589550 shutdown_connections 2024-04-12T21:42:17.910 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:17.909+0000 7f3974400700 1 -- 172.21.15.18:0/4149589550 wait complete. 2024-04-12T21:42:18.783 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:42:18 smithi186 ceph-mon[86379]: from='client.36236 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:42:18.784 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:42:18 smithi186 ceph-mon[86379]: from='client.? 172.21.15.18:0/4149589550' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-12T21:42:18.855 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:42:18 smithi018 ceph-mon[112590]: from='client.36236 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:42:18.855 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:42:18 smithi018 ceph-mon[112590]: from='client.? 172.21.15.18:0/4149589550' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-12T21:42:19.783 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:42:19 smithi186 ceph-mon[86379]: pgmap v1279: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:42:19.854 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:42:19 smithi018 ceph-mon[112590]: pgmap v1279: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:42:21.783 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:42:21 smithi186 ceph-mon[86379]: pgmap v1280: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:42:21.854 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:42:21 smithi018 ceph-mon[112590]: pgmap v1280: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:42:23.783 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:42:23 smithi186 ceph-mon[86379]: pgmap v1281: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:42:23.854 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:42:23 smithi018 ceph-mon[112590]: pgmap v1281: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:42:25.784 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:42:25 smithi186 ceph-mon[86379]: pgmap v1282: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:42:25.784 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:42:25 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T21:42:25.854 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:42:25 smithi018 ceph-mon[112590]: pgmap v1282: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:42:25.855 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:42:25 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T21:42:27.783 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:42:27 smithi186 ceph-mon[86379]: pgmap v1283: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:42:27.854 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:42:27 smithi018 ceph-mon[112590]: pgmap v1283: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:42:29.783 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:42:29 smithi186 ceph-mon[86379]: pgmap v1284: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:42:29.855 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:42:29 smithi018 ceph-mon[112590]: pgmap v1284: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:42:31.783 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:42:31 smithi186 ceph-mon[86379]: pgmap v1285: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:42:31.854 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:42:31 smithi018 ceph-mon[112590]: pgmap v1285: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:42:33.784 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:42:33 smithi186 ceph-mon[86379]: pgmap v1286: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:42:33.854 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:42:33 smithi018 ceph-mon[112590]: pgmap v1286: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:42:35.783 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:42:35 smithi186 ceph-mon[86379]: pgmap v1287: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:42:35.854 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:42:35 smithi018 ceph-mon[112590]: pgmap v1287: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:42:37.783 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:42:37 smithi186 ceph-mon[86379]: pgmap v1288: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:42:37.854 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:42:37 smithi018 ceph-mon[112590]: pgmap v1288: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:42:39.785 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:42:39 smithi186 ceph-mon[86379]: pgmap v1289: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:42:39.854 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:42:39 smithi018 ceph-mon[112590]: pgmap v1289: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:42:40.783 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:42:40 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T21:42:40.854 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:42:40 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T21:42:41.783 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:42:41 smithi186 ceph-mon[86379]: pgmap v1290: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:42:41.854 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:42:41 smithi018 ceph-mon[112590]: pgmap v1290: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:42:43.783 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:42:43 smithi186 ceph-mon[86379]: pgmap v1291: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:42:43.854 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:42:43 smithi018 ceph-mon[112590]: pgmap v1291: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:42:45.657 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:42:45 smithi018 ceph-mon[112590]: pgmap v1292: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:42:45.783 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:42:45 smithi186 ceph-mon[86379]: pgmap v1292: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:42:47.783 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:42:47 smithi186 ceph-mon[86379]: pgmap v1293: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:42:47.854 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:42:47 smithi018 ceph-mon[112590]: pgmap v1293: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:42:48.080 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:48.078+0000 7f3bdb200700 1 -- 172.21.15.18:0/1228654650 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3bdc1092a0 msgr2=0x7f3bdc109670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:42:48.080 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:48.078+0000 7f3bdb200700 1 --2- 172.21.15.18:0/1228654650 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3bdc1092a0 0x7f3bdc109670 secure :-1 s=READY pgs=1086 cs=0 l=1 rev1=1 crypto rx=0x7f3bcc004940 tx=0x7f3bcc009e30 comp rx=0 tx=0).stop 2024-04-12T21:42:48.080 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:48.078+0000 7f3bdb200700 1 -- 172.21.15.18:0/1228654650 shutdown_connections 2024-04-12T21:42:48.080 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:48.078+0000 7f3bdb200700 1 --2- 172.21.15.18:0/1228654650 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3bdc109c40 0x7f3bdc1032b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:42:48.080 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:48.078+0000 7f3bdb200700 1 --2- 172.21.15.18:0/1228654650 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3bdc1092a0 0x7f3bdc109670 unknown :-1 s=CLOSED pgs=1086 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:42:48.080 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:48.078+0000 7f3bdb200700 1 -- 172.21.15.18:0/1228654650 >> 172.21.15.18:0/1228654650 conn(0x7f3bdc072a30 msgr2=0x7f3bdc074e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:42:48.080 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:48.079+0000 7f3bdb200700 1 -- 172.21.15.18:0/1228654650 shutdown_connections 2024-04-12T21:42:48.080 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:48.079+0000 7f3bdb200700 1 -- 172.21.15.18:0/1228654650 wait complete. 2024-04-12T21:42:48.081 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:48.079+0000 7f3bdb200700 1 Processor -- start 2024-04-12T21:42:48.081 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:48.080+0000 7f3bdb200700 1 -- start start 2024-04-12T21:42:48.082 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:48.080+0000 7f3bdb200700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3bdc1092a0 0x7f3bdc119ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:42:48.082 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:48.080+0000 7f3bdb200700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3bdc109c40 0x7f3bdc11a530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:42:48.082 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:48.080+0000 7f3bdb200700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3bdc11ab80 con 0x7f3bdc109c40 2024-04-12T21:42:48.082 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:48.080+0000 7f3bdb200700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3bdc113f50 con 0x7f3bdc1092a0 2024-04-12T21:42:48.082 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:48.080+0000 7f3bd9400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3bdc109c40 0x7f3bdc11a530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:42:48.082 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:48.080+0000 7f3bd9400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3bdc109c40 0x7f3bdc11a530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:47938/0 (socket says 172.21.15.18:47938) 2024-04-12T21:42:48.082 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:48.080+0000 7f3bd9400700 1 -- 172.21.15.18:0/361776129 learned_addr learned my addr 172.21.15.18:0/361776129 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:42:48.083 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:48.081+0000 7f3bd9e00700 1 --2- 172.21.15.18:0/361776129 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3bdc1092a0 0x7f3bdc119ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:42:48.083 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:48.081+0000 7f3bd9400700 1 -- 172.21.15.18:0/361776129 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3bdc1092a0 msgr2=0x7f3bdc119ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:42:48.083 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:48.081+0000 7f3bd9400700 1 --2- 172.21.15.18:0/361776129 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3bdc1092a0 0x7f3bdc119ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:42:48.083 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:48.081+0000 7f3bd9400700 1 -- 172.21.15.18:0/361776129 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3bcc005040 con 0x7f3bdc109c40 2024-04-12T21:42:48.083 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:48.081+0000 7f3bd9400700 1 --2- 172.21.15.18:0/361776129 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3bdc109c40 0x7f3bdc11a530 secure :-1 s=READY pgs=1087 cs=0 l=1 rev1=1 crypto rx=0x7f3bc800c890 tx=0x7f3bc80044c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:42:48.083 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:48.082+0000 7f3bd2000700 1 -- 172.21.15.18:0/361776129 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3bc800a7d0 con 0x7f3bdc109c40 2024-04-12T21:42:48.083 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:48.082+0000 7f3bdb200700 1 -- 172.21.15.18:0/361776129 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3bdc114230 con 0x7f3bdc109c40 2024-04-12T21:42:48.084 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:48.082+0000 7f3bd2000700 1 -- 172.21.15.18:0/361776129 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f3bc800ae00 con 0x7f3bdc109c40 2024-04-12T21:42:48.084 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:48.082+0000 7f3bd2000700 1 -- 172.21.15.18:0/361776129 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3bc80113a0 con 0x7f3bdc109c40 2024-04-12T21:42:48.084 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:48.082+0000 7f3bdb200700 1 -- 172.21.15.18:0/361776129 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3bdc1147f0 con 0x7f3bdc109c40 2024-04-12T21:42:48.084 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:48.083+0000 7f3bdb200700 1 -- 172.21.15.18:0/361776129 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3bdc1049f0 con 0x7f3bdc109c40 2024-04-12T21:42:48.085 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:48.084+0000 7f3bd2000700 1 -- 172.21.15.18:0/361776129 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7f3bc8012400 con 0x7f3bdc109c40 2024-04-12T21:42:48.086 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:48.085+0000 7f3bd2000700 1 --2- 172.21.15.18:0/361776129 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f3bc006ec00 0x7f3bc00710b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:42:48.087 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:48.085+0000 7f3bd2000700 1 -- 172.21.15.18:0/361776129 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 6757+0+0 (secure 0 0 0) 0x7f3bc808ea40 con 0x7f3bdc109c40 2024-04-12T21:42:48.087 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:48.086+0000 7f3bd9e00700 1 --2- 172.21.15.18:0/361776129 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f3bc006ec00 0x7f3bc00710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:42:48.087 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:48.087+0000 7f3bd9e00700 1 --2- 172.21.15.18:0/361776129 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f3bc006ec00 0x7f3bc00710b0 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f3bcc00b3b0 tx=0x7f3bcc009e30 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:42:48.091 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:48.090+0000 7f3bd2000700 1 -- 172.21.15.18:0/361776129 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f3bc805b270 con 0x7f3bdc109c40 2024-04-12T21:42:48.341 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:48.339+0000 7f3bdb200700 1 -- 172.21.15.18:0/361776129 --> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f3bdc110400 con 0x7f3bc006ec00 2024-04-12T21:42:48.343 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:48.342+0000 7f3bd2000700 1 -- 172.21.15.18:0/361776129 <== mgr.34102 v2:172.21.15.18:6800/3812891919 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f3bdc110400 con 0x7f3bc006ec00 2024-04-12T21:42:48.346 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:48.345+0000 7f3bdb200700 1 -- 172.21.15.18:0/361776129 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f3bc006ec00 msgr2=0x7f3bc00710b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:42:48.346 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:48.345+0000 7f3bdb200700 1 --2- 172.21.15.18:0/361776129 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f3bc006ec00 0x7f3bc00710b0 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f3bcc00b3b0 tx=0x7f3bcc009e30 comp rx=0 tx=0).stop 2024-04-12T21:42:48.346 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:48.345+0000 7f3bdb200700 1 -- 172.21.15.18:0/361776129 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3bdc109c40 msgr2=0x7f3bdc11a530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:42:48.346 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:48.345+0000 7f3bdb200700 1 --2- 172.21.15.18:0/361776129 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3bdc109c40 0x7f3bdc11a530 secure :-1 s=READY pgs=1087 cs=0 l=1 rev1=1 crypto rx=0x7f3bc800c890 tx=0x7f3bc80044c0 comp rx=0 tx=0).stop 2024-04-12T21:42:48.347 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:48.345+0000 7f3bdb200700 1 -- 172.21.15.18:0/361776129 shutdown_connections 2024-04-12T21:42:48.347 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:48.345+0000 7f3bdb200700 1 --2- 172.21.15.18:0/361776129 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f3bc006ec00 0x7f3bc00710b0 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:42:48.347 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:48.345+0000 7f3bdb200700 1 --2- 172.21.15.18:0/361776129 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3bdc1092a0 0x7f3bdc119ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:42:48.347 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:48.345+0000 7f3bdb200700 1 --2- 172.21.15.18:0/361776129 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3bdc109c40 0x7f3bdc11a530 unknown :-1 s=CLOSED pgs=1087 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:42:48.347 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:48.345+0000 7f3bdb200700 1 -- 172.21.15.18:0/361776129 >> 172.21.15.18:0/361776129 conn(0x7f3bdc072a30 msgr2=0x7f3bdc0748d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:42:48.347 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:48.346+0000 7f3bdb200700 1 -- 172.21.15.18:0/361776129 shutdown_connections 2024-04-12T21:42:48.347 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:48.346+0000 7f3bdb200700 1 -- 172.21.15.18:0/361776129 wait complete. 2024-04-12T21:42:48.364 INFO:teuthology.orchestra.run.smithi018.stdout:true 2024-04-12T21:42:48.504 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:48.502+0000 7f3c4aa00700 1 -- 172.21.15.18:0/1317581163 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3c44105020 msgr2=0x7f3c441053f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:42:48.504 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:48.502+0000 7f3c4aa00700 1 --2- 172.21.15.18:0/1317581163 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3c44105020 0x7f3c441053f0 secure :-1 s=READY pgs=1088 cs=0 l=1 rev1=1 crypto rx=0x7f3c34004940 tx=0x7f3c34009e30 comp rx=0 tx=0).stop 2024-04-12T21:42:48.504 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:48.502+0000 7f3c4aa00700 1 -- 172.21.15.18:0/1317581163 shutdown_connections 2024-04-12T21:42:48.504 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:48.502+0000 7f3c4aa00700 1 --2- 172.21.15.18:0/1317581163 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3c441059c0 0x7f3c440ff080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:42:48.504 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:48.502+0000 7f3c4aa00700 1 --2- 172.21.15.18:0/1317581163 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3c44105020 0x7f3c441053f0 unknown :-1 s=CLOSED pgs=1088 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:42:48.504 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:48.502+0000 7f3c4aa00700 1 -- 172.21.15.18:0/1317581163 >> 172.21.15.18:0/1317581163 conn(0x7f3c440faba0 msgr2=0x7f3c440fcfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:42:48.504 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:48.502+0000 7f3c4aa00700 1 -- 172.21.15.18:0/1317581163 shutdown_connections 2024-04-12T21:42:48.504 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:48.503+0000 7f3c4aa00700 1 -- 172.21.15.18:0/1317581163 wait complete. 2024-04-12T21:42:48.505 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:48.503+0000 7f3c4aa00700 1 Processor -- start 2024-04-12T21:42:48.505 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:48.504+0000 7f3c4aa00700 1 -- start start 2024-04-12T21:42:48.505 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:48.504+0000 7f3c4aa00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3c44105020 0x7f3c44115b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:42:48.505 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:48.504+0000 7f3c4aa00700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3c441059c0 0x7f3c441160d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:42:48.505 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:48.504+0000 7f3c4aa00700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c44116720 con 0x7f3c44105020 2024-04-12T21:42:48.506 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:48.504+0000 7f3c4aa00700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c4410fd00 con 0x7f3c441059c0 2024-04-12T21:42:48.506 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:48.505+0000 7f3c43e00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3c44105020 0x7f3c44115b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:42:48.507 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:48.505+0000 7f3c43e00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3c44105020 0x7f3c44115b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:47954/0 (socket says 172.21.15.18:47954) 2024-04-12T21:42:48.507 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:48.505+0000 7f3c43e00700 1 -- 172.21.15.18:0/3549983022 learned_addr learned my addr 172.21.15.18:0/3549983022 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:42:48.507 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:48.505+0000 7f3c3be00700 1 --2- 172.21.15.18:0/3549983022 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3c441059c0 0x7f3c441160d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:42:48.507 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:48.505+0000 7f3c43e00700 1 -- 172.21.15.18:0/3549983022 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3c441059c0 msgr2=0x7f3c441160d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:42:48.507 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:48.505+0000 7f3c43e00700 1 --2- 172.21.15.18:0/3549983022 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3c441059c0 0x7f3c441160d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:42:48.507 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:48.505+0000 7f3c43e00700 1 -- 172.21.15.18:0/3549983022 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c34005040 con 0x7f3c44105020 2024-04-12T21:42:48.507 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:48.505+0000 7f3c3be00700 1 --2- 172.21.15.18:0/3549983022 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3c441059c0 0x7f3c441160d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-12T21:42:48.507 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:48.505+0000 7f3c43e00700 1 --2- 172.21.15.18:0/3549983022 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3c44105020 0x7f3c44115b90 secure :-1 s=READY pgs=1089 cs=0 l=1 rev1=1 crypto rx=0x7f3c34009180 tx=0x7f3c34009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:42:48.507 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:48.506+0000 7f3c40c00700 1 -- 172.21.15.18:0/3549983022 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3c3400e040 con 0x7f3c44105020 2024-04-12T21:42:48.507 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:48.506+0000 7f3c40c00700 1 -- 172.21.15.18:0/3549983022 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f3c3400bbc0 con 0x7f3c44105020 2024-04-12T21:42:48.508 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:48.506+0000 7f3c4aa00700 1 -- 172.21.15.18:0/3549983022 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3c4410ff80 con 0x7f3c44105020 2024-04-12T21:42:48.508 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:48.506+0000 7f3c40c00700 1 -- 172.21.15.18:0/3549983022 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3c3401c630 con 0x7f3c44105020 2024-04-12T21:42:48.508 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:48.506+0000 7f3c4aa00700 1 -- 172.21.15.18:0/3549983022 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3c44110540 con 0x7f3c44105020 2024-04-12T21:42:48.508 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:48.507+0000 7f3c4aa00700 1 -- 172.21.15.18:0/3549983022 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3c44100770 con 0x7f3c44105020 2024-04-12T21:42:48.509 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:48.508+0000 7f3c40c00700 1 -- 172.21.15.18:0/3549983022 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7f3c3400bd30 con 0x7f3c44105020 2024-04-12T21:42:48.510 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:48.509+0000 7f3c40c00700 1 --2- 172.21.15.18:0/3549983022 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f3c2806ec00 0x7f3c280710b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:42:48.510 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:48.509+0000 7f3c40c00700 1 -- 172.21.15.18:0/3549983022 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 6757+0+0 (secure 0 0 0) 0x7f3c3408fd90 con 0x7f3c44105020 2024-04-12T21:42:48.511 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:48.510+0000 7f3c3be00700 1 --2- 172.21.15.18:0/3549983022 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f3c2806ec00 0x7f3c280710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:42:48.511 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:48.510+0000 7f3c3be00700 1 --2- 172.21.15.18:0/3549983022 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f3c2806ec00 0x7f3c280710b0 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f3c2c007fd0 tx=0x7f3c2c009040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:42:48.515 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:48.514+0000 7f3c40c00700 1 -- 172.21.15.18:0/3549983022 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f3c3405c5c0 con 0x7f3c44105020 2024-04-12T21:42:48.766 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:48.764+0000 7f3c4aa00700 1 -- 172.21.15.18:0/3549983022 --> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f3c441111e0 con 0x7f3c2806ec00 2024-04-12T21:42:48.768 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:48.767+0000 7f3c40c00700 1 -- 172.21.15.18:0/3549983022 <== mgr.34102 v2:172.21.15.18:6800/3812891919 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f3c441111e0 con 0x7f3c2806ec00 2024-04-12T21:42:48.771 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:48.769+0000 7f3c4aa00700 1 -- 172.21.15.18:0/3549983022 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f3c2806ec00 msgr2=0x7f3c280710b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:42:48.771 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:48.769+0000 7f3c4aa00700 1 --2- 172.21.15.18:0/3549983022 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f3c2806ec00 0x7f3c280710b0 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f3c2c007fd0 tx=0x7f3c2c009040 comp rx=0 tx=0).stop 2024-04-12T21:42:48.771 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:48.770+0000 7f3c4aa00700 1 -- 172.21.15.18:0/3549983022 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3c44105020 msgr2=0x7f3c44115b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:42:48.771 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:48.770+0000 7f3c4aa00700 1 --2- 172.21.15.18:0/3549983022 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3c44105020 0x7f3c44115b90 secure :-1 s=READY pgs=1089 cs=0 l=1 rev1=1 crypto rx=0x7f3c34009180 tx=0x7f3c34009e30 comp rx=0 tx=0).stop 2024-04-12T21:42:48.771 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:48.770+0000 7f3c4aa00700 1 -- 172.21.15.18:0/3549983022 shutdown_connections 2024-04-12T21:42:48.772 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:48.770+0000 7f3c4aa00700 1 --2- 172.21.15.18:0/3549983022 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f3c2806ec00 0x7f3c280710b0 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:42:48.772 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:48.770+0000 7f3c4aa00700 1 --2- 172.21.15.18:0/3549983022 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3c44105020 0x7f3c44115b90 unknown :-1 s=CLOSED pgs=1089 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:42:48.772 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:48.770+0000 7f3c4aa00700 1 --2- 172.21.15.18:0/3549983022 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3c441059c0 0x7f3c441160d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:42:48.772 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:48.770+0000 7f3c4aa00700 1 -- 172.21.15.18:0/3549983022 >> 172.21.15.18:0/3549983022 conn(0x7f3c440faba0 msgr2=0x7f3c440fbb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:42:48.772 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:48.770+0000 7f3c4aa00700 1 -- 172.21.15.18:0/3549983022 shutdown_connections 2024-04-12T21:42:48.772 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:48.770+0000 7f3c4aa00700 1 -- 172.21.15.18:0/3549983022 wait complete. 2024-04-12T21:42:48.928 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:48.926+0000 7fddf1000700 1 -- 172.21.15.18:0/3603727216 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fddec1059b0 msgr2=0x7fddec0ff020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:42:48.928 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:48.926+0000 7fddf1000700 1 --2- 172.21.15.18:0/3603727216 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fddec1059b0 0x7fddec0ff020 secure :-1 s=READY pgs=1090 cs=0 l=1 rev1=1 crypto rx=0x7fddd8004940 tx=0x7fddd8009e30 comp rx=0 tx=0).stop 2024-04-12T21:42:48.929 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:48.927+0000 7fddf1000700 1 -- 172.21.15.18:0/3603727216 shutdown_connections 2024-04-12T21:42:48.929 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:48.927+0000 7fddf1000700 1 --2- 172.21.15.18:0/3603727216 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fddec1059b0 0x7fddec0ff020 unknown :-1 s=CLOSED pgs=1090 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:42:48.929 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:48.927+0000 7fddf1000700 1 --2- 172.21.15.18:0/3603727216 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fddec105010 0x7fddec1053e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:42:48.929 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:48.927+0000 7fddf1000700 1 -- 172.21.15.18:0/3603727216 >> 172.21.15.18:0/3603727216 conn(0x7fddec0faba0 msgr2=0x7fddec0fcfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:42:48.929 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:48.927+0000 7fddf1000700 1 -- 172.21.15.18:0/3603727216 shutdown_connections 2024-04-12T21:42:48.929 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:48.927+0000 7fddf1000700 1 -- 172.21.15.18:0/3603727216 wait complete. 2024-04-12T21:42:48.931 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:48.928+0000 7fddf1000700 1 Processor -- start 2024-04-12T21:42:48.931 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:48.928+0000 7fddf1000700 1 -- start start 2024-04-12T21:42:48.931 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:48.928+0000 7fddf1000700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fddec105010 0x7fddec115d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:42:48.931 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:48.928+0000 7fddf1000700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fddec1059b0 0x7fddec1162b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:42:48.931 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:48.928+0000 7fddf1000700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fddec116900 con 0x7fddec105010 2024-04-12T21:42:48.931 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:48.928+0000 7fddf1000700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fddec10fcd0 con 0x7fddec1059b0 2024-04-12T21:42:48.932 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:48.928+0000 7fdde9e00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fddec105010 0x7fddec115d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:42:48.932 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:48.929+0000 7fdde9e00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fddec105010 0x7fddec115d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:47974/0 (socket says 172.21.15.18:47974) 2024-04-12T21:42:48.932 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:48.929+0000 7fdde9e00700 1 -- 172.21.15.18:0/2274195810 learned_addr learned my addr 172.21.15.18:0/2274195810 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:42:48.932 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:48.929+0000 7fdde9400700 1 --2- 172.21.15.18:0/2274195810 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fddec1059b0 0x7fddec1162b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:42:48.932 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:48.929+0000 7fdde9e00700 1 -- 172.21.15.18:0/2274195810 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fddec1059b0 msgr2=0x7fddec1162b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:42:48.932 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:48.929+0000 7fdde9e00700 1 --2- 172.21.15.18:0/2274195810 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fddec1059b0 0x7fddec1162b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:42:48.932 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:48.929+0000 7fdde9e00700 1 -- 172.21.15.18:0/2274195810 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fddd8005040 con 0x7fddec105010 2024-04-12T21:42:48.932 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:48.929+0000 7fdde9400700 1 --2- 172.21.15.18:0/2274195810 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fddec1059b0 0x7fddec1162b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-12T21:42:48.932 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:48.929+0000 7fdde9e00700 1 --2- 172.21.15.18:0/2274195810 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fddec105010 0x7fddec115d70 secure :-1 s=READY pgs=1091 cs=0 l=1 rev1=1 crypto rx=0x7fdddc002a50 tx=0x7fdddc0046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:42:48.932 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:48.930+0000 7fdde2000700 1 -- 172.21.15.18:0/2274195810 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdddc007cc0 con 0x7fddec105010 2024-04-12T21:42:48.933 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:48.930+0000 7fdde2000700 1 -- 172.21.15.18:0/2274195810 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fdddc00f040 con 0x7fddec105010 2024-04-12T21:42:48.933 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:48.930+0000 7fddf1000700 1 -- 172.21.15.18:0/2274195810 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fddec10ffb0 con 0x7fddec105010 2024-04-12T21:42:48.933 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:48.930+0000 7fdde2000700 1 -- 172.21.15.18:0/2274195810 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdddc009640 con 0x7fddec105010 2024-04-12T21:42:48.933 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:48.930+0000 7fddf1000700 1 -- 172.21.15.18:0/2274195810 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fddec110570 con 0x7fddec105010 2024-04-12T21:42:48.933 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:48.931+0000 7fddf1000700 1 -- 172.21.15.18:0/2274195810 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fddec100760 con 0x7fddec105010 2024-04-12T21:42:48.933 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:48.932+0000 7fdde2000700 1 -- 172.21.15.18:0/2274195810 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7fdddc0097a0 con 0x7fddec105010 2024-04-12T21:42:48.934 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:48.933+0000 7fdde2000700 1 --2- 172.21.15.18:0/2274195810 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fddd006ec00 0x7fddd00710b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:42:48.934 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:48.933+0000 7fdde2000700 1 -- 172.21.15.18:0/2274195810 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 6757+0+0 (secure 0 0 0) 0x7fdddc08e6c0 con 0x7fddec105010 2024-04-12T21:42:48.934 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:48.933+0000 7fdde9400700 1 --2- 172.21.15.18:0/2274195810 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fddd006ec00 0x7fddd00710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:42:48.935 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:48.934+0000 7fdde9400700 1 --2- 172.21.15.18:0/2274195810 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fddd006ec00 0x7fddd00710b0 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7fddd8009fd0 tx=0x7fddd8009e30 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:42:48.939 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:48.938+0000 7fdde2000700 1 -- 172.21.15.18:0/2274195810 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7fdddc05af10 con 0x7fddec105010 2024-04-12T21:42:49.180 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:49.179+0000 7fddf1000700 1 -- 172.21.15.18:0/2274195810 --> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fddec04bf40 con 0x7fddd006ec00 2024-04-12T21:42:49.209 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:49.208+0000 7fdde2000700 1 -- 172.21.15.18:0/2274195810 <== mgr.34102 v2:172.21.15.18:6800/3812891919 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7fddec04bf40 con 0x7fddd006ec00 2024-04-12T21:42:49.210 INFO:teuthology.orchestra.run.smithi018.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-12T21:42:49.210 INFO:teuthology.orchestra.run.smithi018.stdout:alertmanager.smithi018 smithi018 *:9093,9094 running (43m) 4m ago 57m 14.8M - 0.25.0 c8568f914cd2 c332090068e8 2024-04-12T21:42:49.210 INFO:teuthology.orchestra.run.smithi018.stdout:crash.smithi018 smithi018 running (39m) 4m ago 58m 6665k - 19.0.0-2879-g324baf8d 8717720dc5e5 b7f80d1f7982 2024-04-12T21:42:49.210 INFO:teuthology.orchestra.run.smithi018.stdout:crash.smithi186 smithi186 running (39m) 4m ago 56m 6660k - 19.0.0-2879-g324baf8d 8717720dc5e5 a4f55c81b623 2024-04-12T21:42:49.210 INFO:teuthology.orchestra.run.smithi018.stdout:grafana.smithi018 smithi018 *:3000 running (42m) 4m ago 57m 73.8M - 9.4.12 2bacad6d85d8 b08e3bf176dd 2024-04-12T21:42:49.210 INFO:teuthology.orchestra.run.smithi018.stdout:mds.cephfs.smithi018.acqbsc smithi018 running (35m) 4m ago 50m 21.9M - 19.0.0-2879-g324baf8d 8717720dc5e5 021f99ab24ae 2024-04-12T21:42:49.210 INFO:teuthology.orchestra.run.smithi018.stdout:mds.cephfs.smithi018.szjluo smithi018 running (35m) 4m ago 50m 21.7M - 19.0.0-2879-g324baf8d 8717720dc5e5 0003117479d4 2024-04-12T21:42:49.210 INFO:teuthology.orchestra.run.smithi018.stdout:mds.cephfs.smithi186.jdllyn smithi186 running (19m) 4m ago 50m 16.5M - 19.0.0-2879-g324baf8d 8717720dc5e5 21b1f46eef23 2024-04-12T21:42:49.210 INFO:teuthology.orchestra.run.smithi018.stdout:mds.cephfs.smithi186.wjnafs smithi186 running (4m) 4m ago 50m 11.0M - 19.0.0-2879-g324baf8d 8717720dc5e5 3cf1ef4148e2 2024-04-12T21:42:49.211 INFO:teuthology.orchestra.run.smithi018.stdout:mgr.smithi018.xjzwga smithi018 *:8443,9283,8765 running (46m) 4m ago 59m 517M - 19.0.0-2879-g324baf8d 8717720dc5e5 56088305eab7 2024-04-12T21:42:49.211 INFO:teuthology.orchestra.run.smithi018.stdout:mgr.smithi186.kqcfxk smithi186 *:8443,9283,8765 running (44m) 4m ago 56m 460M - 19.0.0-2879-g324baf8d 8717720dc5e5 09cd766e5e0d 2024-04-12T21:42:49.211 INFO:teuthology.orchestra.run.smithi018.stdout:mon.smithi018 smithi018 running (41m) 4m ago 59m 88.2M 2048M 19.0.0-2879-g324baf8d 8717720dc5e5 df3e201dfed6 2024-04-12T21:42:49.211 INFO:teuthology.orchestra.run.smithi018.stdout:mon.smithi186 smithi186 running (40m) 4m ago 56m 53.6M 2048M 19.0.0-2879-g324baf8d 8717720dc5e5 c77498da9b2b 2024-04-12T21:42:49.211 INFO:teuthology.orchestra.run.smithi018.stdout:node-exporter.smithi018 smithi018 *:9100 running (44m) 4m ago 58m 8468k - 1.5.0 0da6a335fe13 326c3d9c6847 2024-04-12T21:42:49.211 INFO:teuthology.orchestra.run.smithi018.stdout:node-exporter.smithi186 smithi186 *:9100 running (44m) 4m ago 56m 8166k - 1.5.0 0da6a335fe13 ec2112baeecf 2024-04-12T21:42:49.211 INFO:teuthology.orchestra.run.smithi018.stdout:osd.0 smithi018 running (39m) 4m ago 55m 123M 1559M 19.0.0-2879-g324baf8d 8717720dc5e5 60b1273ca047 2024-04-12T21:42:49.211 INFO:teuthology.orchestra.run.smithi018.stdout:osd.1 smithi018 running (38m) 4m ago 54m 95.4M 1559M 19.0.0-2879-g324baf8d 8717720dc5e5 170ce1e5fb8d 2024-04-12T21:42:49.211 INFO:teuthology.orchestra.run.smithi018.stdout:osd.2 smithi018 running (38m) 4m ago 54m 94.6M 1559M 19.0.0-2879-g324baf8d 8717720dc5e5 c2cbc6aa2839 2024-04-12T21:42:49.211 INFO:teuthology.orchestra.run.smithi018.stdout:osd.3 smithi186 running (37m) 4m ago 53m 101M 2581M 19.0.0-2879-g324baf8d 8717720dc5e5 0138e8262e4b 2024-04-12T21:42:49.211 INFO:teuthology.orchestra.run.smithi018.stdout:osd.4 smithi186 running (36m) 4m ago 53m 91.5M 2581M 19.0.0-2879-g324baf8d 8717720dc5e5 b20cb63b25c9 2024-04-12T21:42:49.211 INFO:teuthology.orchestra.run.smithi018.stdout:osd.5 smithi186 running (36m) 4m ago 52m 92.2M 2581M 19.0.0-2879-g324baf8d 8717720dc5e5 2513bea2610e 2024-04-12T21:42:49.211 INFO:teuthology.orchestra.run.smithi018.stdout:prometheus.smithi018 smithi018 *:9095 running (43m) 4m ago 57m 69.6M - 2.43.0 a07b618ecd1d f58841127e21 2024-04-12T21:42:49.213 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:49.211+0000 7fddf1000700 1 -- 172.21.15.18:0/2274195810 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fddd006ec00 msgr2=0x7fddd00710b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:42:49.213 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:49.211+0000 7fddf1000700 1 --2- 172.21.15.18:0/2274195810 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fddd006ec00 0x7fddd00710b0 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7fddd8009fd0 tx=0x7fddd8009e30 comp rx=0 tx=0).stop 2024-04-12T21:42:49.213 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:49.211+0000 7fddf1000700 1 -- 172.21.15.18:0/2274195810 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fddec105010 msgr2=0x7fddec115d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:42:49.213 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:49.211+0000 7fddf1000700 1 --2- 172.21.15.18:0/2274195810 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fddec105010 0x7fddec115d70 secure :-1 s=READY pgs=1091 cs=0 l=1 rev1=1 crypto rx=0x7fdddc002a50 tx=0x7fdddc0046b0 comp rx=0 tx=0).stop 2024-04-12T21:42:49.213 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:49.211+0000 7fddf1000700 1 -- 172.21.15.18:0/2274195810 shutdown_connections 2024-04-12T21:42:49.213 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:49.211+0000 7fddf1000700 1 --2- 172.21.15.18:0/2274195810 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fddd006ec00 0x7fddd00710b0 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:42:49.213 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:49.211+0000 7fddf1000700 1 --2- 172.21.15.18:0/2274195810 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fddec105010 0x7fddec115d70 unknown :-1 s=CLOSED pgs=1091 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:42:49.213 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:49.211+0000 7fddf1000700 1 --2- 172.21.15.18:0/2274195810 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fddec1059b0 0x7fddec1162b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:42:49.213 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:49.211+0000 7fddf1000700 1 -- 172.21.15.18:0/2274195810 >> 172.21.15.18:0/2274195810 conn(0x7fddec0faba0 msgr2=0x7fddec0fbcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:42:49.213 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:49.211+0000 7fddf1000700 1 -- 172.21.15.18:0/2274195810 shutdown_connections 2024-04-12T21:42:49.213 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:49.212+0000 7fddf1000700 1 -- 172.21.15.18:0/2274195810 wait complete. 2024-04-12T21:42:49.373 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:49.371+0000 7f4247e00700 1 -- 172.21.15.18:0/2488009147 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f42481035d0 msgr2=0x7f42480fcc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:42:49.373 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:49.371+0000 7f4247e00700 1 --2- 172.21.15.18:0/2488009147 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f42481035d0 0x7f42480fcc40 secure :-1 s=READY pgs=1092 cs=0 l=1 rev1=1 crypto rx=0x7f4234004940 tx=0x7f4234009e30 comp rx=0 tx=0).stop 2024-04-12T21:42:49.373 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:49.371+0000 7f4247e00700 1 -- 172.21.15.18:0/2488009147 shutdown_connections 2024-04-12T21:42:49.373 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:49.371+0000 7f4247e00700 1 --2- 172.21.15.18:0/2488009147 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f42481035d0 0x7f42480fcc40 unknown :-1 s=CLOSED pgs=1092 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:42:49.373 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:49.371+0000 7f4247e00700 1 --2- 172.21.15.18:0/2488009147 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4248102c30 0x7f4248103000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:42:49.373 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:49.371+0000 7f4247e00700 1 -- 172.21.15.18:0/2488009147 >> 172.21.15.18:0/2488009147 conn(0x7f42480f8760 msgr2=0x7f42480fab70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:42:49.374 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:49.372+0000 7f4247e00700 1 -- 172.21.15.18:0/2488009147 shutdown_connections 2024-04-12T21:42:49.374 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:49.372+0000 7f4247e00700 1 -- 172.21.15.18:0/2488009147 wait complete. 2024-04-12T21:42:49.374 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:49.373+0000 7f4247e00700 1 Processor -- start 2024-04-12T21:42:49.375 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:49.373+0000 7f4247e00700 1 -- start start 2024-04-12T21:42:49.375 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:49.373+0000 7f4247e00700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4248102c30 0x7f4248115b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:42:49.375 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:49.373+0000 7f4247e00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f42481035d0 0x7f4248116080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:42:49.375 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:49.373+0000 7f4247e00700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42481166d0 con 0x7f42481035d0 2024-04-12T21:42:49.375 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:49.373+0000 7f4247e00700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f424810fac0 con 0x7f4248102c30 2024-04-12T21:42:49.376 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:49.374+0000 7f4246000700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f42481035d0 0x7f4248116080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:42:49.376 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:49.374+0000 7f4246000700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f42481035d0 0x7f4248116080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:48000/0 (socket says 172.21.15.18:48000) 2024-04-12T21:42:49.376 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:49.374+0000 7f4246000700 1 -- 172.21.15.18:0/1212577249 learned_addr learned my addr 172.21.15.18:0/1212577249 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:42:49.376 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:49.374+0000 7f4246a00700 1 --2- 172.21.15.18:0/1212577249 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4248102c30 0x7f4248115b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:42:49.376 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:49.374+0000 7f4246000700 1 -- 172.21.15.18:0/1212577249 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4248102c30 msgr2=0x7f4248115b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:42:49.376 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:49.374+0000 7f4246000700 1 --2- 172.21.15.18:0/1212577249 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4248102c30 0x7f4248115b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:42:49.376 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:49.374+0000 7f4246000700 1 -- 172.21.15.18:0/1212577249 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4234005040 con 0x7f42481035d0 2024-04-12T21:42:49.376 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:49.375+0000 7f4246000700 1 --2- 172.21.15.18:0/1212577249 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f42481035d0 0x7f4248116080 secure :-1 s=READY pgs=1093 cs=0 l=1 rev1=1 crypto rx=0x7f42340040d0 tx=0x7f4234009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:42:49.377 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:49.375+0000 7f423ea00700 1 -- 172.21.15.18:0/1212577249 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f423400e030 con 0x7f42481035d0 2024-04-12T21:42:49.377 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:49.375+0000 7f423ea00700 1 -- 172.21.15.18:0/1212577249 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f4234013c00 con 0x7f42481035d0 2024-04-12T21:42:49.377 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:49.375+0000 7f4247e00700 1 -- 172.21.15.18:0/1212577249 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f424810fd40 con 0x7f42481035d0 2024-04-12T21:42:49.377 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:49.375+0000 7f423ea00700 1 -- 172.21.15.18:0/1212577249 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f423400b650 con 0x7f42481035d0 2024-04-12T21:42:49.377 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:49.375+0000 7f4247e00700 1 -- 172.21.15.18:0/1212577249 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4248110300 con 0x7f42481035d0 2024-04-12T21:42:49.377 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:49.376+0000 7f4247e00700 1 -- 172.21.15.18:0/1212577249 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f42480fe380 con 0x7f42481035d0 2024-04-12T21:42:49.378 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:49.378+0000 7f423ea00700 1 -- 172.21.15.18:0/1212577249 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7f423400b8c0 con 0x7f42481035d0 2024-04-12T21:42:49.380 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:49.379+0000 7f423ea00700 1 --2- 172.21.15.18:0/1212577249 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f422c06ec00 0x7f422c0710b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:42:49.380 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:49.379+0000 7f423ea00700 1 -- 172.21.15.18:0/1212577249 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 6757+0+0 (secure 0 0 0) 0x7f4234090570 con 0x7f42481035d0 2024-04-12T21:42:49.380 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:49.379+0000 7f4246a00700 1 --2- 172.21.15.18:0/1212577249 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f422c06ec00 0x7f422c0710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:42:49.381 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:49.380+0000 7f4246a00700 1 --2- 172.21.15.18:0/1212577249 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f422c06ec00 0x7f422c0710b0 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f4238009fd0 tx=0x7f423800b040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:42:49.385 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:49.384+0000 7f423ea00700 1 -- 172.21.15.18:0/1212577249 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f423405cda0 con 0x7f42481035d0 2024-04-12T21:42:49.714 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:49.712+0000 7f4247e00700 1 -- 172.21.15.18:0/1212577249 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f42480fc3c0 con 0x7f42481035d0 2024-04-12T21:42:49.715 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:49.714+0000 7f423ea00700 1 -- 172.21.15.18:0/1212577249 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+623 (secure 0 0 0) 0x7f423405c930 con 0x7f42481035d0 2024-04-12T21:42:49.715 INFO:teuthology.orchestra.run.smithi018.stdout:{ 2024-04-12T21:42:49.715 INFO:teuthology.orchestra.run.smithi018.stdout: "mon": { 2024-04-12T21:42:49.715 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 19.0.0-2879-g324baf8d (324baf8ddb91dea46cac50cb535c306115791355) squid (dev)": 2 2024-04-12T21:42:49.715 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-12T21:42:49.716 INFO:teuthology.orchestra.run.smithi018.stdout: "mgr": { 2024-04-12T21:42:49.716 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 19.0.0-2879-g324baf8d (324baf8ddb91dea46cac50cb535c306115791355) squid (dev)": 2 2024-04-12T21:42:49.716 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-12T21:42:49.716 INFO:teuthology.orchestra.run.smithi018.stdout: "osd": { 2024-04-12T21:42:49.716 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 19.0.0-2879-g324baf8d (324baf8ddb91dea46cac50cb535c306115791355) squid (dev)": 6 2024-04-12T21:42:49.716 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-12T21:42:49.716 INFO:teuthology.orchestra.run.smithi018.stdout: "mds": { 2024-04-12T21:42:49.716 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 19.0.0-2879-g324baf8d (324baf8ddb91dea46cac50cb535c306115791355) squid (dev)": 4 2024-04-12T21:42:49.716 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-12T21:42:49.716 INFO:teuthology.orchestra.run.smithi018.stdout: "overall": { 2024-04-12T21:42:49.716 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 19.0.0-2879-g324baf8d (324baf8ddb91dea46cac50cb535c306115791355) squid (dev)": 14 2024-04-12T21:42:49.717 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-04-12T21:42:49.717 INFO:teuthology.orchestra.run.smithi018.stdout:} 2024-04-12T21:42:49.718 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:49.716+0000 7f4247e00700 1 -- 172.21.15.18:0/1212577249 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f422c06ec00 msgr2=0x7f422c0710b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:42:49.718 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:49.716+0000 7f4247e00700 1 --2- 172.21.15.18:0/1212577249 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f422c06ec00 0x7f422c0710b0 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f4238009fd0 tx=0x7f423800b040 comp rx=0 tx=0).stop 2024-04-12T21:42:49.718 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:49.716+0000 7f4247e00700 1 -- 172.21.15.18:0/1212577249 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f42481035d0 msgr2=0x7f4248116080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:42:49.718 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:49.716+0000 7f4247e00700 1 --2- 172.21.15.18:0/1212577249 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f42481035d0 0x7f4248116080 secure :-1 s=READY pgs=1093 cs=0 l=1 rev1=1 crypto rx=0x7f42340040d0 tx=0x7f4234009e30 comp rx=0 tx=0).stop 2024-04-12T21:42:49.718 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:49.716+0000 7f4247e00700 1 -- 172.21.15.18:0/1212577249 shutdown_connections 2024-04-12T21:42:49.718 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:49.716+0000 7f4247e00700 1 --2- 172.21.15.18:0/1212577249 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f422c06ec00 0x7f422c0710b0 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:42:49.718 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:49.716+0000 7f4247e00700 1 --2- 172.21.15.18:0/1212577249 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4248102c30 0x7f4248115b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:42:49.718 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:49.717+0000 7f4247e00700 1 --2- 172.21.15.18:0/1212577249 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f42481035d0 0x7f4248116080 unknown :-1 s=CLOSED pgs=1093 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:42:49.718 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:49.717+0000 7f4247e00700 1 -- 172.21.15.18:0/1212577249 >> 172.21.15.18:0/1212577249 conn(0x7f42480f8760 msgr2=0x7f4248071ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:42:49.719 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:49.717+0000 7f4247e00700 1 -- 172.21.15.18:0/1212577249 shutdown_connections 2024-04-12T21:42:49.719 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:49.717+0000 7f4247e00700 1 -- 172.21.15.18:0/1212577249 wait complete. 2024-04-12T21:42:49.784 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:42:49 smithi186 ceph-mon[86379]: pgmap v1294: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:42:49.784 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:42:49 smithi186 ceph-mon[86379]: from='client.36244 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:42:49.784 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:42:49 smithi186 ceph-mon[86379]: from='client.36248 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:42:49.854 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:42:49 smithi018 ceph-mon[112590]: pgmap v1294: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:42:49.854 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:42:49 smithi018 ceph-mon[112590]: from='client.36244 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:42:49.855 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:42:49 smithi018 ceph-mon[112590]: from='client.36248 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:42:49.873 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:49.872+0000 7fc62fe00700 1 -- 172.21.15.18:0/696330041 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc6281037b0 msgr2=0x7fc6280fce20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:42:49.873 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:49.872+0000 7fc62fe00700 1 --2- 172.21.15.18:0/696330041 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc6281037b0 0x7fc6280fce20 secure :-1 s=READY pgs=1094 cs=0 l=1 rev1=1 crypto rx=0x7fc614004940 tx=0x7fc614009e30 comp rx=0 tx=0).stop 2024-04-12T21:42:49.874 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:49.872+0000 7fc62fe00700 1 -- 172.21.15.18:0/696330041 shutdown_connections 2024-04-12T21:42:49.874 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:49.872+0000 7fc62fe00700 1 --2- 172.21.15.18:0/696330041 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc6281037b0 0x7fc6280fce20 unknown :-1 s=CLOSED pgs=1094 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:42:49.874 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:49.872+0000 7fc62fe00700 1 --2- 172.21.15.18:0/696330041 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc628102e10 0x7fc6281031e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:42:49.874 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:49.872+0000 7fc62fe00700 1 -- 172.21.15.18:0/696330041 >> 172.21.15.18:0/696330041 conn(0x7fc6280f89a0 msgr2=0x7fc6280fadb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:42:49.874 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:49.872+0000 7fc62fe00700 1 -- 172.21.15.18:0/696330041 shutdown_connections 2024-04-12T21:42:49.874 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:49.872+0000 7fc62fe00700 1 -- 172.21.15.18:0/696330041 wait complete. 2024-04-12T21:42:49.875 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:49.873+0000 7fc62fe00700 1 Processor -- start 2024-04-12T21:42:49.875 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:49.873+0000 7fc62fe00700 1 -- start start 2024-04-12T21:42:49.875 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:49.874+0000 7fc62fe00700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc628102e10 0x7fc6280702c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:42:49.875 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:49.874+0000 7fc62fe00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc6281037b0 0x7fc628070800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:42:49.875 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:49.874+0000 7fc62fe00700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc628070d90 con 0x7fc6281037b0 2024-04-12T21:42:49.875 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:49.874+0000 7fc62fe00700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc628070ed0 con 0x7fc628102e10 2024-04-12T21:42:49.876 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:49.874+0000 7fc62ca00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc6281037b0 0x7fc628070800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:42:49.876 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:49.874+0000 7fc62ca00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc6281037b0 0x7fc628070800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:48018/0 (socket says 172.21.15.18:48018) 2024-04-12T21:42:49.877 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:49.874+0000 7fc62ca00700 1 -- 172.21.15.18:0/599500054 learned_addr learned my addr 172.21.15.18:0/599500054 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:42:49.877 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:49.874+0000 7fc62d400700 1 --2- 172.21.15.18:0/599500054 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc628102e10 0x7fc6280702c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:42:49.877 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:49.875+0000 7fc62ca00700 1 -- 172.21.15.18:0/599500054 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc628102e10 msgr2=0x7fc6280702c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:42:49.877 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:49.875+0000 7fc62ca00700 1 --2- 172.21.15.18:0/599500054 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc628102e10 0x7fc6280702c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:42:49.877 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:49.875+0000 7fc62ca00700 1 -- 172.21.15.18:0/599500054 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc614005040 con 0x7fc6281037b0 2024-04-12T21:42:49.877 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:49.875+0000 7fc62ca00700 1 --2- 172.21.15.18:0/599500054 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc6281037b0 0x7fc628070800 secure :-1 s=READY pgs=1095 cs=0 l=1 rev1=1 crypto rx=0x7fc614009100 tx=0x7fc614009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:42:49.877 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:49.875+0000 7fc625600700 1 -- 172.21.15.18:0/599500054 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc61400e040 con 0x7fc6281037b0 2024-04-12T21:42:49.877 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:49.875+0000 7fc625600700 1 -- 172.21.15.18:0/599500054 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc614013420 con 0x7fc6281037b0 2024-04-12T21:42:49.877 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:49.875+0000 7fc625600700 1 -- 172.21.15.18:0/599500054 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc61401c630 con 0x7fc6281037b0 2024-04-12T21:42:49.877 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:49.875+0000 7fc62fe00700 1 -- 172.21.15.18:0/599500054 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc628071120 con 0x7fc6281037b0 2024-04-12T21:42:49.877 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:49.876+0000 7fc62fe00700 1 -- 172.21.15.18:0/599500054 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc62811a2d0 con 0x7fc6281037b0 2024-04-12T21:42:49.878 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:49.877+0000 7fc62fe00700 1 -- 172.21.15.18:0/599500054 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc6281052e0 con 0x7fc6281037b0 2024-04-12T21:42:49.879 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:49.878+0000 7fc625600700 1 -- 172.21.15.18:0/599500054 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7fc614013590 con 0x7fc6281037b0 2024-04-12T21:42:49.880 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:49.879+0000 7fc625600700 1 --2- 172.21.15.18:0/599500054 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fc61006ec10 0x7fc6100710c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:42:49.880 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:49.879+0000 7fc625600700 1 -- 172.21.15.18:0/599500054 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 6757+0+0 (secure 0 0 0) 0x7fc61408fba0 con 0x7fc6281037b0 2024-04-12T21:42:49.880 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:49.879+0000 7fc62d400700 1 --2- 172.21.15.18:0/599500054 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fc61006ec10 0x7fc6100710c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:42:49.880 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:49.880+0000 7fc62d400700 1 --2- 172.21.15.18:0/599500054 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fc61006ec10 0x7fc6100710c0 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7fc61c007fd0 tx=0x7fc61c009040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:42:49.884 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:49.883+0000 7fc625600700 1 -- 172.21.15.18:0/599500054 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7fc6140588a0 con 0x7fc6281037b0 2024-04-12T21:42:50.166 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:50.165+0000 7fc62fe00700 1 -- 172.21.15.18:0/599500054 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fc6280fe560 con 0x7fc6281037b0 2024-04-12T21:42:50.167 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:50.166+0000 7fc625600700 1 -- 172.21.15.18:0/599500054 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 33 v33) v1 ==== 76+0+1876 (secure 0 0 0) 0x7fc614058020 con 0x7fc6281037b0 2024-04-12T21:42:50.168 INFO:teuthology.orchestra.run.smithi018.stdout:e33 2024-04-12T21:42:50.169 INFO:teuthology.orchestra.run.smithi018.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-12T21:42:50.169 INFO:teuthology.orchestra.run.smithi018.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-12T21:42:50.169 INFO:teuthology.orchestra.run.smithi018.stdout:legacy client fscid: 1 2024-04-12T21:42:50.169 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-12T21:42:50.169 INFO:teuthology.orchestra.run.smithi018.stdout:Filesystem 'cephfs' (1) 2024-04-12T21:42:50.169 INFO:teuthology.orchestra.run.smithi018.stdout:fs_name cephfs 2024-04-12T21:42:50.170 INFO:teuthology.orchestra.run.smithi018.stdout:epoch 24 2024-04-12T21:42:50.170 INFO:teuthology.orchestra.run.smithi018.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-04-12T21:42:50.170 INFO:teuthology.orchestra.run.smithi018.stdout:created 2024-04-12T20:51:57.362850+0000 2024-04-12T21:42:50.170 INFO:teuthology.orchestra.run.smithi018.stdout:modified 2024-04-12T21:06:59.717190+0000 2024-04-12T21:42:50.170 INFO:teuthology.orchestra.run.smithi018.stdout:tableserver 0 2024-04-12T21:42:50.170 INFO:teuthology.orchestra.run.smithi018.stdout:root 0 2024-04-12T21:42:50.171 INFO:teuthology.orchestra.run.smithi018.stdout:session_timeout 60 2024-04-12T21:42:50.171 INFO:teuthology.orchestra.run.smithi018.stdout:session_autoclose 300 2024-04-12T21:42:50.171 INFO:teuthology.orchestra.run.smithi018.stdout:max_file_size 1099511627776 2024-04-12T21:42:50.171 INFO:teuthology.orchestra.run.smithi018.stdout:max_xattr_size 65536 2024-04-12T21:42:50.171 INFO:teuthology.orchestra.run.smithi018.stdout:required_client_features {} 2024-04-12T21:42:50.171 INFO:teuthology.orchestra.run.smithi018.stdout:last_failure 0 2024-04-12T21:42:50.172 INFO:teuthology.orchestra.run.smithi018.stdout:last_failure_osd_epoch 100 2024-04-12T21:42:50.172 INFO:teuthology.orchestra.run.smithi018.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-12T21:42:50.172 INFO:teuthology.orchestra.run.smithi018.stdout:max_mds 2 2024-04-12T21:42:50.172 INFO:teuthology.orchestra.run.smithi018.stdout:in 0,1 2024-04-12T21:42:50.172 INFO:teuthology.orchestra.run.smithi018.stdout:up {} 2024-04-12T21:42:50.172 INFO:teuthology.orchestra.run.smithi018.stdout:failed 0,1 2024-04-12T21:42:50.173 INFO:teuthology.orchestra.run.smithi018.stdout:damaged 2024-04-12T21:42:50.173 INFO:teuthology.orchestra.run.smithi018.stdout:stopped 2024-04-12T21:42:50.173 INFO:teuthology.orchestra.run.smithi018.stdout:data_pools [3] 2024-04-12T21:42:50.173 INFO:teuthology.orchestra.run.smithi018.stdout:metadata_pool 2 2024-04-12T21:42:50.173 INFO:teuthology.orchestra.run.smithi018.stdout:inline_data disabled 2024-04-12T21:42:50.173 INFO:teuthology.orchestra.run.smithi018.stdout:balancer 2024-04-12T21:42:50.174 INFO:teuthology.orchestra.run.smithi018.stdout:bal_rank_mask -1 2024-04-12T21:42:50.174 INFO:teuthology.orchestra.run.smithi018.stdout:standby_count_wanted 1 2024-04-12T21:42:50.174 INFO:teuthology.orchestra.run.smithi018.stdout:qdb_cluster leader: 0 members: 2024-04-12T21:42:50.174 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-12T21:42:50.174 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-12T21:42:50.174 INFO:teuthology.orchestra.run.smithi018.stdout:Standby daemons: 2024-04-12T21:42:50.175 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-12T21:42:50.175 INFO:teuthology.orchestra.run.smithi018.stdout:[mds.cephfs.smithi018.acqbsc{-1:34436} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.18:6828/755179764,v1:172.21.15.18:6829/755179764] compat {c=[1],r=[1],i=[1fff]}] 2024-04-12T21:42:50.175 INFO:teuthology.orchestra.run.smithi018.stdout:[mds.cephfs.smithi018.szjluo{-1:34468} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.18:6826/2407992653,v1:172.21.15.18:6827/2407992653] compat {c=[1],r=[1],i=[1fff]}] 2024-04-12T21:42:50.175 INFO:teuthology.orchestra.run.smithi018.stdout:[mds.cephfs.smithi186.jdllyn{-1:44595} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.186:6824/3523444202,v1:172.21.15.186:6825/3523444202] compat {c=[1],r=[1],i=[1fff]}] 2024-04-12T21:42:50.175 INFO:teuthology.orchestra.run.smithi018.stdout:[mds.cephfs.smithi186.wjnafs{-1:44875} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.186:6826/3122443766,v1:172.21.15.186:6827/3122443766] compat {c=[1],r=[1],i=[1fff]}] 2024-04-12T21:42:50.177 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:50.169+0000 7fc62fe00700 1 -- 172.21.15.18:0/599500054 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fc61006ec10 msgr2=0x7fc6100710c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:42:50.177 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:50.169+0000 7fc62fe00700 1 --2- 172.21.15.18:0/599500054 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fc61006ec10 0x7fc6100710c0 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7fc61c007fd0 tx=0x7fc61c009040 comp rx=0 tx=0).stop 2024-04-12T21:42:50.177 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:50.169+0000 7fc62fe00700 1 -- 172.21.15.18:0/599500054 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc6281037b0 msgr2=0x7fc628070800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:42:50.177 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:50.169+0000 7fc62fe00700 1 --2- 172.21.15.18:0/599500054 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc6281037b0 0x7fc628070800 secure :-1 s=READY pgs=1095 cs=0 l=1 rev1=1 crypto rx=0x7fc614009100 tx=0x7fc614009e30 comp rx=0 tx=0).stop 2024-04-12T21:42:50.177 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:50.169+0000 7fc62fe00700 1 -- 172.21.15.18:0/599500054 shutdown_connections 2024-04-12T21:42:50.178 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:50.169+0000 7fc62fe00700 1 --2- 172.21.15.18:0/599500054 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fc61006ec10 0x7fc6100710c0 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:42:50.178 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:50.169+0000 7fc62fe00700 1 --2- 172.21.15.18:0/599500054 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc628102e10 0x7fc6280702c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:42:50.178 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:50.169+0000 7fc62fe00700 1 --2- 172.21.15.18:0/599500054 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc6281037b0 0x7fc628070800 unknown :-1 s=CLOSED pgs=1095 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:42:50.178 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:50.169+0000 7fc62fe00700 1 -- 172.21.15.18:0/599500054 >> 172.21.15.18:0/599500054 conn(0x7fc6280f89a0 msgr2=0x7fc628102220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:42:50.178 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:50.169+0000 7fc62fe00700 1 -- 172.21.15.18:0/599500054 shutdown_connections 2024-04-12T21:42:50.178 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:50.170+0000 7fc62fe00700 1 -- 172.21.15.18:0/599500054 wait complete. 2024-04-12T21:42:50.179 INFO:teuthology.orchestra.run.smithi018.stderr:dumped fsmap epoch 33 2024-04-12T21:42:50.327 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:50.326+0000 7f659e200700 1 -- 172.21.15.18:0/319382681 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6598109dd0 msgr2=0x7f6598103440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:42:50.328 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:50.326+0000 7f659e200700 1 --2- 172.21.15.18:0/319382681 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6598109dd0 0x7f6598103440 secure :-1 s=READY pgs=1096 cs=0 l=1 rev1=1 crypto rx=0x7f6580004940 tx=0x7f6580009e30 comp rx=0 tx=0).stop 2024-04-12T21:42:50.328 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:50.326+0000 7f659e200700 1 -- 172.21.15.18:0/319382681 shutdown_connections 2024-04-12T21:42:50.328 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:50.326+0000 7f659e200700 1 --2- 172.21.15.18:0/319382681 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6598109dd0 0x7f6598103440 unknown :-1 s=CLOSED pgs=1096 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:42:50.328 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:50.326+0000 7f659e200700 1 --2- 172.21.15.18:0/319382681 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6598109430 0x7f6598109800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:42:50.328 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:50.326+0000 7f659e200700 1 -- 172.21.15.18:0/319382681 >> 172.21.15.18:0/319382681 conn(0x7f6598072c00 msgr2=0x7f6598075010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:42:50.328 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:50.326+0000 7f659e200700 1 -- 172.21.15.18:0/319382681 shutdown_connections 2024-04-12T21:42:50.328 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:50.326+0000 7f659e200700 1 -- 172.21.15.18:0/319382681 wait complete. 2024-04-12T21:42:50.329 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:50.327+0000 7f659e200700 1 Processor -- start 2024-04-12T21:42:50.329 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:50.327+0000 7f659e200700 1 -- start start 2024-04-12T21:42:50.330 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:50.328+0000 7f659e200700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6598109430 0x7f6598119f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:42:50.330 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:50.328+0000 7f659e200700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6598109dd0 0x7f659811a4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:42:50.330 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:50.328+0000 7f659e200700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f659811ab10 con 0x7f6598109430 2024-04-12T21:42:50.330 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:50.328+0000 7f659e200700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65981140f0 con 0x7f6598109dd0 2024-04-12T21:42:50.330 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:50.328+0000 7f6597400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6598109430 0x7f6598119f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:42:50.330 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:50.328+0000 7f6597400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6598109430 0x7f6598119f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:48040/0 (socket says 172.21.15.18:48040) 2024-04-12T21:42:50.330 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:50.328+0000 7f6597400700 1 -- 172.21.15.18:0/1220044807 learned_addr learned my addr 172.21.15.18:0/1220044807 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:42:50.330 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:50.328+0000 7f658fe00700 1 --2- 172.21.15.18:0/1220044807 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6598109dd0 0x7f659811a4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:42:50.330 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:50.328+0000 7f6597400700 1 -- 172.21.15.18:0/1220044807 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6598109dd0 msgr2=0x7f659811a4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:42:50.330 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:50.328+0000 7f6597400700 1 --2- 172.21.15.18:0/1220044807 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6598109dd0 0x7f659811a4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:42:50.330 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:50.328+0000 7f6597400700 1 -- 172.21.15.18:0/1220044807 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6580005040 con 0x7f6598109430 2024-04-12T21:42:50.331 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:50.329+0000 7f6597400700 1 --2- 172.21.15.18:0/1220044807 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6598109430 0x7f6598119f80 secure :-1 s=READY pgs=1097 cs=0 l=1 rev1=1 crypto rx=0x7f658800c8c0 tx=0x7f65880044c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:42:50.331 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:50.329+0000 7f658f400700 1 -- 172.21.15.18:0/1220044807 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f658800a980 con 0x7f6598109430 2024-04-12T21:42:50.331 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:50.329+0000 7f658f400700 1 -- 172.21.15.18:0/1220044807 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f658800ce60 con 0x7f6598109430 2024-04-12T21:42:50.331 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:50.329+0000 7f658f400700 1 -- 172.21.15.18:0/1220044807 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f65880115e0 con 0x7f6598109430 2024-04-12T21:42:50.331 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:50.329+0000 7f659e200700 1 -- 172.21.15.18:0/1220044807 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f65981143d0 con 0x7f6598109430 2024-04-12T21:42:50.331 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:50.329+0000 7f659e200700 1 -- 172.21.15.18:0/1220044807 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6598114990 con 0x7f6598109430 2024-04-12T21:42:50.332 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:50.331+0000 7f659e200700 1 -- 172.21.15.18:0/1220044807 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6598104b80 con 0x7f6598109430 2024-04-12T21:42:50.332 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:50.332+0000 7f658f400700 1 -- 172.21.15.18:0/1220044807 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7f6588012400 con 0x7f6598109430 2024-04-12T21:42:50.334 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:50.332+0000 7f658f400700 1 --2- 172.21.15.18:0/1220044807 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f657c06ec00 0x7f657c0710b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:42:50.334 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:50.333+0000 7f658f400700 1 -- 172.21.15.18:0/1220044807 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 6757+0+0 (secure 0 0 0) 0x7f658808e980 con 0x7f6598109430 2024-04-12T21:42:50.334 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:50.333+0000 7f658fe00700 1 --2- 172.21.15.18:0/1220044807 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f657c06ec00 0x7f657c0710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:42:50.334 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:50.334+0000 7f658fe00700 1 --2- 172.21.15.18:0/1220044807 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f657c06ec00 0x7f657c0710b0 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f658000b020 tx=0x7f6580009e30 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:42:50.339 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:50.338+0000 7f658f400700 1 -- 172.21.15.18:0/1220044807 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f658805b270 con 0x7f6598109430 2024-04-12T21:42:50.593 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:50.591+0000 7f659e200700 1 -- 172.21.15.18:0/1220044807 --> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f65981155d0 con 0x7f657c06ec00 2024-04-12T21:42:50.594 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:50.593+0000 7f658f400700 1 -- 172.21.15.18:0/1220044807 <== mgr.34102 v2:172.21.15.18:6800/3812891919 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f65981155d0 con 0x7f657c06ec00 2024-04-12T21:42:50.595 INFO:teuthology.orchestra.run.smithi018.stdout:{ 2024-04-12T21:42:50.595 INFO:teuthology.orchestra.run.smithi018.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:f4452def3be4d350f4a8e5feaf7ebb1511be9298fe68478ba7a2d6fbd7d69a63", 2024-04-12T21:42:50.595 INFO:teuthology.orchestra.run.smithi018.stdout: "in_progress": true, 2024-04-12T21:42:50.595 INFO:teuthology.orchestra.run.smithi018.stdout: "which": "Upgrading all daemon types on all hosts", 2024-04-12T21:42:50.595 INFO:teuthology.orchestra.run.smithi018.stdout: "services_complete": [ 2024-04-12T21:42:50.595 INFO:teuthology.orchestra.run.smithi018.stdout: "mgr", 2024-04-12T21:42:50.595 INFO:teuthology.orchestra.run.smithi018.stdout: "mds", 2024-04-12T21:42:50.595 INFO:teuthology.orchestra.run.smithi018.stdout: "crash", 2024-04-12T21:42:50.595 INFO:teuthology.orchestra.run.smithi018.stdout: "osd", 2024-04-12T21:42:50.595 INFO:teuthology.orchestra.run.smithi018.stdout: "mon" 2024-04-12T21:42:50.595 INFO:teuthology.orchestra.run.smithi018.stdout: ], 2024-04-12T21:42:50.595 INFO:teuthology.orchestra.run.smithi018.stdout: "progress": "16/21 daemons upgraded", 2024-04-12T21:42:50.596 INFO:teuthology.orchestra.run.smithi018.stdout: "message": "Currently upgrading mds daemons", 2024-04-12T21:42:50.596 INFO:teuthology.orchestra.run.smithi018.stdout: "is_paused": false 2024-04-12T21:42:50.596 INFO:teuthology.orchestra.run.smithi018.stdout:} 2024-04-12T21:42:50.598 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:50.596+0000 7f659e200700 1 -- 172.21.15.18:0/1220044807 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f657c06ec00 msgr2=0x7f657c0710b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:42:50.598 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:50.596+0000 7f659e200700 1 --2- 172.21.15.18:0/1220044807 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f657c06ec00 0x7f657c0710b0 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f658000b020 tx=0x7f6580009e30 comp rx=0 tx=0).stop 2024-04-12T21:42:50.598 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:50.596+0000 7f659e200700 1 -- 172.21.15.18:0/1220044807 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6598109430 msgr2=0x7f6598119f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:42:50.598 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:50.596+0000 7f659e200700 1 --2- 172.21.15.18:0/1220044807 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6598109430 0x7f6598119f80 secure :-1 s=READY pgs=1097 cs=0 l=1 rev1=1 crypto rx=0x7f658800c8c0 tx=0x7f65880044c0 comp rx=0 tx=0).stop 2024-04-12T21:42:50.598 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:50.596+0000 7f659e200700 1 -- 172.21.15.18:0/1220044807 shutdown_connections 2024-04-12T21:42:50.598 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:50.596+0000 7f659e200700 1 --2- 172.21.15.18:0/1220044807 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f657c06ec00 0x7f657c0710b0 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:42:50.598 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:50.596+0000 7f659e200700 1 --2- 172.21.15.18:0/1220044807 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6598109430 0x7f6598119f80 unknown :-1 s=CLOSED pgs=1097 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:42:50.598 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:50.596+0000 7f659e200700 1 --2- 172.21.15.18:0/1220044807 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6598109dd0 0x7f659811a4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:42:50.598 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:50.596+0000 7f659e200700 1 -- 172.21.15.18:0/1220044807 >> 172.21.15.18:0/1220044807 conn(0x7f6598072c00 msgr2=0x7f6598074c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:42:50.598 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:50.597+0000 7f659e200700 1 -- 172.21.15.18:0/1220044807 shutdown_connections 2024-04-12T21:42:50.598 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:50.597+0000 7f659e200700 1 -- 172.21.15.18:0/1220044807 wait complete. 2024-04-12T21:42:50.747 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:50.745+0000 7f0310000700 1 -- 172.21.15.18:0/630457222 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0308070c10 msgr2=0x7f0308109190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:42:50.747 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:50.745+0000 7f0310000700 1 --2- 172.21.15.18:0/630457222 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0308070c10 0x7f0308109190 secure :-1 s=READY pgs=1098 cs=0 l=1 rev1=1 crypto rx=0x7f02f4004970 tx=0x7f02f4009e30 comp rx=0 tx=0).stop 2024-04-12T21:42:50.747 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:50.745+0000 7f0310000700 1 -- 172.21.15.18:0/630457222 shutdown_connections 2024-04-12T21:42:50.747 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:50.745+0000 7f0310000700 1 --2- 172.21.15.18:0/630457222 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0308070c10 0x7f0308109190 unknown :-1 s=CLOSED pgs=1098 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:42:50.747 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:50.745+0000 7f0310000700 1 --2- 172.21.15.18:0/630457222 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0308070300 0x7f03080706d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:42:50.747 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:50.745+0000 7f0310000700 1 -- 172.21.15.18:0/630457222 >> 172.21.15.18:0/630457222 conn(0x7f03080f8970 msgr2=0x7f03080fad80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:42:50.747 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:50.745+0000 7f0310000700 1 -- 172.21.15.18:0/630457222 shutdown_connections 2024-04-12T21:42:50.747 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:50.746+0000 7f0310000700 1 -- 172.21.15.18:0/630457222 wait complete. 2024-04-12T21:42:50.749 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:50.746+0000 7f0310000700 1 Processor -- start 2024-04-12T21:42:50.749 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:50.746+0000 7f0310000700 1 -- start start 2024-04-12T21:42:50.749 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:50.747+0000 7f0310000700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0308070300 0x7f0308115e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:42:50.749 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:50.747+0000 7f0310000700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0308070c10 0x7f0308116350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:42:50.750 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:50.747+0000 7f0310000700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0308116a30 con 0x7f0308070c10 2024-04-12T21:42:50.750 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:50.747+0000 7f0310000700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f030810fd70 con 0x7f0308070300 2024-04-12T21:42:50.750 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:50.747+0000 7f030cc00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0308070c10 0x7f0308116350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:42:50.750 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:50.747+0000 7f030cc00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0308070c10 0x7f0308116350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:48064/0 (socket says 172.21.15.18:48064) 2024-04-12T21:42:50.750 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:50.747+0000 7f030cc00700 1 -- 172.21.15.18:0/3824713658 learned_addr learned my addr 172.21.15.18:0/3824713658 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:42:50.750 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:50.747+0000 7f030d600700 1 --2- 172.21.15.18:0/3824713658 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0308070300 0x7f0308115e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:42:50.750 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:50.747+0000 7f030cc00700 1 -- 172.21.15.18:0/3824713658 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0308070300 msgr2=0x7f0308115e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:42:50.750 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:50.748+0000 7f030cc00700 1 --2- 172.21.15.18:0/3824713658 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0308070300 0x7f0308115e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:42:50.750 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:50.748+0000 7f030cc00700 1 -- 172.21.15.18:0/3824713658 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02f4005040 con 0x7f0308070c10 2024-04-12T21:42:50.750 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:50.748+0000 7f030d600700 1 --2- 172.21.15.18:0/3824713658 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0308070300 0x7f0308115e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-12T21:42:50.750 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:50.748+0000 7f030cc00700 1 --2- 172.21.15.18:0/3824713658 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0308070c10 0x7f0308116350 secure :-1 s=READY pgs=1099 cs=0 l=1 rev1=1 crypto rx=0x7f02f4004f50 tx=0x7f02f4009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:42:50.750 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:50.748+0000 7f0305600700 1 -- 172.21.15.18:0/3824713658 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f02f400e040 con 0x7f0308070c10 2024-04-12T21:42:50.750 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:50.748+0000 7f0305600700 1 -- 172.21.15.18:0/3824713658 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f02f4004290 con 0x7f0308070c10 2024-04-12T21:42:50.751 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:50.748+0000 7f0305600700 1 -- 172.21.15.18:0/3824713658 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f02f401c3f0 con 0x7f0308070c10 2024-04-12T21:42:50.751 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:50.748+0000 7f0310000700 1 -- 172.21.15.18:0/3824713658 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f030810ff90 con 0x7f0308070c10 2024-04-12T21:42:50.751 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:50.748+0000 7f0310000700 1 -- 172.21.15.18:0/3824713658 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f03081104d0 con 0x7f0308070c10 2024-04-12T21:42:50.751 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:50.749+0000 7f0310000700 1 -- 172.21.15.18:0/3824713658 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0308071a40 con 0x7f0308070c10 2024-04-12T21:42:50.751 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:50.750+0000 7f0305600700 1 -- 172.21.15.18:0/3824713658 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7f02f400bc10 con 0x7f0308070c10 2024-04-12T21:42:50.752 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:50.751+0000 7f0305600700 1 --2- 172.21.15.18:0/3824713658 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f02ec06ec10 0x7f02ec0710c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:42:50.753 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:50.751+0000 7f0305600700 1 -- 172.21.15.18:0/3824713658 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 6757+0+0 (secure 0 0 0) 0x7f02f408fb40 con 0x7f0308070c10 2024-04-12T21:42:50.753 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:50.752+0000 7f030d600700 1 --2- 172.21.15.18:0/3824713658 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f02ec06ec10 0x7f02ec0710c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:42:50.753 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:50.752+0000 7f030d600700 1 --2- 172.21.15.18:0/3824713658 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f02ec06ec10 0x7f02ec0710c0 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f02f8006fd0 tx=0x7f02f8008040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:42:50.757 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:50.756+0000 7f0305600700 1 -- 172.21.15.18:0/3824713658 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f02f405c370 con 0x7f0308070c10 2024-04-12T21:42:50.783 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:42:50 smithi186 ceph-mon[86379]: from='client.36252 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:42:50.784 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:42:50 smithi186 ceph-mon[86379]: from='client.? 172.21.15.18:0/1212577249' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T21:42:50.784 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:42:50 smithi186 ceph-mon[86379]: from='client.? 172.21.15.18:0/599500054' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-12T21:42:50.855 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:42:50 smithi018 ceph-mon[112590]: from='client.36252 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:42:50.855 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:42:50 smithi018 ceph-mon[112590]: from='client.? 172.21.15.18:0/1212577249' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T21:42:50.855 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:42:50 smithi018 ceph-mon[112590]: from='client.? 172.21.15.18:0/599500054' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-12T21:42:51.076 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:51.074+0000 7f0310000700 1 -- 172.21.15.18:0/3824713658 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f030810c1c0 con 0x7f0308070c10 2024-04-12T21:42:51.076 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:51.075+0000 7f0305600700 1 -- 172.21.15.18:0/3824713658 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+4303 (secure 0 0 0) 0x7f02f405bf00 con 0x7f0308070c10 2024-04-12T21:42:51.077 INFO:teuthology.orchestra.run.smithi018.stdout:HEALTH_ERR failed to probe daemons or devices; 1 filesystem is degraded; 1 filesystem is offline 2024-04-12T21:42:51.077 INFO:teuthology.orchestra.run.smithi018.stdout:[WRN] CEPHADM_REFRESH_FAILED: failed to probe daemons or devices 2024-04-12T21:42:51.077 INFO:teuthology.orchestra.run.smithi018.stdout: host smithi186 `cephadm ceph-volume` failed: cephadm exited with an error code: 1, stderr: Inferring config /var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/mon.smithi186/config 2024-04-12T21:42:51.077 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/podman: timeout after 895 seconds 2024-04-12T21:42:51.077 INFO:teuthology.orchestra.run.smithi018.stdout:Non-zero exit code 124 from /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:f4452def3be4d350f4a8e5feaf7ebb1511be9298fe68478ba7a2d6fbd7d69a63 -e NODE_NAME=smithi186 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968:/var/run/ceph:z -v /var/log/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968:/var/log/ceph:z -v /var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmptgom53f9:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:f4452def3be4d350f4a8e5feaf7ebb1511be9298fe68478ba7a2d6fbd7d69a63 inventory --format=json-pretty --filter-for-batch 2024-04-12T21:42:51.077 INFO:teuthology.orchestra.run.smithi018.stdout:Traceback (most recent call last): 2024-04-12T21:42:51.077 INFO:teuthology.orchestra.run.smithi018.stdout: File "/usr/lib64/python3.9/runpy.py", line 197, in _run_module_as_main 2024-04-12T21:42:51.077 INFO:teuthology.orchestra.run.smithi018.stdout: return _run_code(code, main_globals, None, 2024-04-12T21:42:51.077 INFO:teuthology.orchestra.run.smithi018.stdout: File "/usr/lib64/python3.9/runpy.py", line 87, in _run_code 2024-04-12T21:42:51.077 INFO:teuthology.orchestra.run.smithi018.stdout: exec(code, run_globals) 2024-04-12T21:42:51.078 INFO:teuthology.orchestra.run.smithi018.stdout: File "/var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/cephadm.e44f3ed5e34e4ef52eedef233d1190bff004470d8e8005127b5312bff357fa7f/__main__.py", line 5618, in 2024-04-12T21:42:51.078 INFO:teuthology.orchestra.run.smithi018.stdout: File "/var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/cephadm.e44f3ed5e34e4ef52eedef233d1190bff004470d8e8005127b5312bff357fa7f/__main__.py", line 5606, in main 2024-04-12T21:42:51.078 INFO:teuthology.orchestra.run.smithi018.stdout: File "/var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/cephadm.e44f3ed5e34e4ef52eedef233d1190bff004470d8e8005127b5312bff357fa7f/__main__.py", line 408, in _infer_config 2024-04-12T21:42:51.078 INFO:teuthology.orchestra.run.smithi018.stdout: File "/var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/cephadm.e44f3ed5e34e4ef52eedef233d1190bff004470d8e8005127b5312bff357fa7f/__main__.py", line 323, in _infer_fsid 2024-04-12T21:42:51.078 INFO:teuthology.orchestra.run.smithi018.stdout: File "/var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/cephadm.e44f3ed5e34e4ef52eedef233d1190bff004470d8e8005127b5312bff357fa7f/__main__.py", line 436, in _infer_image 2024-04-12T21:42:51.078 INFO:teuthology.orchestra.run.smithi018.stdout: File "/var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/cephadm.e44f3ed5e34e4ef52eedef233d1190bff004470d8e8005127b5312bff357fa7f/__main__.py", line 310, in _validate_fsid 2024-04-12T21:42:51.078 INFO:teuthology.orchestra.run.smithi018.stdout: File "/var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/cephadm.e44f3ed5e34e4ef52eedef233d1190bff004470d8e8005127b5312bff357fa7f/__main__.py", line 3359, in command_ceph_volume 2024-04-12T21:42:51.078 INFO:teuthology.orchestra.run.smithi018.stdout: File "/var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/cephadm.e44f3ed5e34e4ef52eedef233d1190bff004470d8e8005127b5312bff357fa7f/cephadmlib/call_wrappers.py", line 310, in call_throws 2024-04-12T21:42:51.078 INFO:teuthology.orchestra.run.smithi018.stdout:RuntimeError: Failed command: /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:f4452def3be4d350f4a8e5feaf7ebb1511be9298fe68478ba7a2d6fbd7d69a63 -e NODE_NAME=smithi186 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968:/var/run/ceph:z -v /var/log/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968:/var/log/ceph:z -v /var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmptgom53f9:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:f4452def3be4d350f4a8e5feaf7ebb1511be9298fe68478ba7a2d6fbd7d69a63 inventory --format=json-pretty --filter-for-batch 2024-04-12T21:42:51.078 INFO:teuthology.orchestra.run.smithi018.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-04-12T21:42:51.078 INFO:teuthology.orchestra.run.smithi018.stdout: fs cephfs is degraded 2024-04-12T21:42:51.078 INFO:teuthology.orchestra.run.smithi018.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-04-12T21:42:51.078 INFO:teuthology.orchestra.run.smithi018.stdout: fs cephfs is offline because no MDS is active for it. 2024-04-12T21:42:51.079 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:51.077+0000 7f0310000700 1 -- 172.21.15.18:0/3824713658 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f02ec06ec10 msgr2=0x7f02ec0710c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:42:51.079 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:51.077+0000 7f0310000700 1 --2- 172.21.15.18:0/3824713658 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f02ec06ec10 0x7f02ec0710c0 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f02f8006fd0 tx=0x7f02f8008040 comp rx=0 tx=0).stop 2024-04-12T21:42:51.079 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:51.077+0000 7f0310000700 1 -- 172.21.15.18:0/3824713658 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0308070c10 msgr2=0x7f0308116350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:42:51.079 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:51.077+0000 7f0310000700 1 --2- 172.21.15.18:0/3824713658 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0308070c10 0x7f0308116350 secure :-1 s=READY pgs=1099 cs=0 l=1 rev1=1 crypto rx=0x7f02f4004f50 tx=0x7f02f4009e30 comp rx=0 tx=0).stop 2024-04-12T21:42:51.079 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:51.078+0000 7f0310000700 1 -- 172.21.15.18:0/3824713658 shutdown_connections 2024-04-12T21:42:51.079 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:51.078+0000 7f0310000700 1 --2- 172.21.15.18:0/3824713658 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f02ec06ec10 0x7f02ec0710c0 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:42:51.080 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:51.078+0000 7f0310000700 1 --2- 172.21.15.18:0/3824713658 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0308070300 0x7f0308115e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:42:51.080 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:51.078+0000 7f0310000700 1 --2- 172.21.15.18:0/3824713658 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0308070c10 0x7f0308116350 unknown :-1 s=CLOSED pgs=1099 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:42:51.080 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:51.078+0000 7f0310000700 1 -- 172.21.15.18:0/3824713658 >> 172.21.15.18:0/3824713658 conn(0x7f03080f8970 msgr2=0x7f03080f9d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:42:51.080 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:51.078+0000 7f0310000700 1 -- 172.21.15.18:0/3824713658 shutdown_connections 2024-04-12T21:42:51.080 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:42:51.078+0000 7f0310000700 1 -- 172.21.15.18:0/3824713658 wait complete. 2024-04-12T21:42:51.855 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:42:51 smithi018 ceph-mon[112590]: pgmap v1295: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:42:51.855 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:42:51 smithi018 ceph-mon[112590]: from='client.36264 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:42:51.855 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:42:51 smithi018 ceph-mon[112590]: from='client.? 172.21.15.18:0/3824713658' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-12T21:42:52.034 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:42:51 smithi186 ceph-mon[86379]: pgmap v1295: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:42:52.034 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:42:51 smithi186 ceph-mon[86379]: from='client.36264 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:42:52.034 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:42:51 smithi186 ceph-mon[86379]: from='client.? 172.21.15.18:0/3824713658' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-12T21:42:53.836 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:42:53 smithi186 ceph-mon[86379]: pgmap v1296: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:42:53.854 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:42:53 smithi018 ceph-mon[112590]: pgmap v1296: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:42:55.854 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:42:55 smithi018 ceph-mon[112590]: pgmap v1297: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:42:55.855 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:42:55 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T21:42:56.033 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:42:55 smithi186 ceph-mon[86379]: pgmap v1297: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:42:56.034 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:42:55 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T21:42:57.854 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:42:57 smithi018 ceph-mon[112590]: pgmap v1298: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:42:58.034 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:42:57 smithi186 ceph-mon[86379]: pgmap v1298: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:42:59.854 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:42:59 smithi018 ceph-mon[112590]: pgmap v1299: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:43:00.033 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:42:59 smithi186 ceph-mon[86379]: pgmap v1299: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:43:01.854 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:43:01 smithi018 ceph-mon[112590]: pgmap v1300: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:43:02.034 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:43:01 smithi186 ceph-mon[86379]: pgmap v1300: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:43:03.854 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:43:03 smithi018 ceph-mon[112590]: pgmap v1301: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:43:04.033 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:43:03 smithi186 ceph-mon[86379]: pgmap v1301: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:43:05.854 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:43:05 smithi018 ceph-mon[112590]: pgmap v1302: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:43:06.033 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:43:05 smithi186 ceph-mon[86379]: pgmap v1302: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:43:07.854 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:43:07 smithi018 ceph-mon[112590]: pgmap v1303: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:43:08.033 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:43:07 smithi186 ceph-mon[86379]: pgmap v1303: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:43:09.854 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:43:09 smithi018 ceph-mon[112590]: pgmap v1304: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:43:10.033 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:43:09 smithi186 ceph-mon[86379]: pgmap v1304: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:43:11.033 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:43:10 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T21:43:11.104 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:43:10 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T21:43:12.034 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:43:11 smithi186 ceph-mon[86379]: pgmap v1305: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:43:12.104 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:43:11 smithi018 ceph-mon[112590]: pgmap v1305: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:43:14.034 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:43:13 smithi186 ceph-mon[86379]: pgmap v1306: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:43:14.104 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:43:13 smithi018 ceph-mon[112590]: pgmap v1306: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:43:15.854 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:43:15 smithi018 ceph-mon[112590]: pgmap v1307: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:43:16.034 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:43:15 smithi186 ceph-mon[86379]: pgmap v1307: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:43:18.033 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:43:17 smithi186 ceph-mon[86379]: pgmap v1308: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:43:18.105 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:43:17 smithi018 ceph-mon[112590]: pgmap v1308: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:43:20.034 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:43:19 smithi186 ceph-mon[86379]: pgmap v1309: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:43:20.104 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:43:19 smithi018 ceph-mon[112590]: pgmap v1309: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:43:21.254 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:21.252+0000 7f8e53e00700 1 -- 172.21.15.18:0/151128145 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8e54104e60 msgr2=0x7f8e54105230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:43:21.254 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:21.252+0000 7f8e53e00700 1 --2- 172.21.15.18:0/151128145 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8e54104e60 0x7f8e54105230 secure :-1 s=READY pgs=1100 cs=0 l=1 rev1=1 crypto rx=0x7f8e44004940 tx=0x7f8e44009e30 comp rx=0 tx=0).stop 2024-04-12T21:43:21.254 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:21.252+0000 7f8e53e00700 1 -- 172.21.15.18:0/151128145 shutdown_connections 2024-04-12T21:43:21.254 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:21.252+0000 7f8e53e00700 1 --2- 172.21.15.18:0/151128145 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8e54105800 0x7f8e540fee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:43:21.254 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:21.252+0000 7f8e53e00700 1 --2- 172.21.15.18:0/151128145 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8e54104e60 0x7f8e54105230 unknown :-1 s=CLOSED pgs=1100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:43:21.254 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:21.252+0000 7f8e53e00700 1 -- 172.21.15.18:0/151128145 >> 172.21.15.18:0/151128145 conn(0x7f8e540fab10 msgr2=0x7f8e540fcef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:43:21.254 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:21.252+0000 7f8e53e00700 1 -- 172.21.15.18:0/151128145 shutdown_connections 2024-04-12T21:43:21.254 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:21.252+0000 7f8e53e00700 1 -- 172.21.15.18:0/151128145 wait complete. 2024-04-12T21:43:21.256 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:21.254+0000 7f8e53e00700 1 Processor -- start 2024-04-12T21:43:21.256 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:21.254+0000 7f8e53e00700 1 -- start start 2024-04-12T21:43:21.256 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:21.254+0000 7f8e53e00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8e54104e60 0x7f8e54115bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:43:21.256 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:21.254+0000 7f8e53e00700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8e54105800 0x7f8e541160f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:43:21.256 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:21.254+0000 7f8e53e00700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e54116740 con 0x7f8e54104e60 2024-04-12T21:43:21.256 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:21.254+0000 7f8e53e00700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e5410fb50 con 0x7f8e54105800 2024-04-12T21:43:21.256 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:21.254+0000 7f8e52a00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8e54104e60 0x7f8e54115bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:43:21.256 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:21.254+0000 7f8e52a00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8e54104e60 0x7f8e54115bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:37472/0 (socket says 172.21.15.18:37472) 2024-04-12T21:43:21.256 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:21.254+0000 7f8e52a00700 1 -- 172.21.15.18:0/2446863237 learned_addr learned my addr 172.21.15.18:0/2446863237 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:43:21.256 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:21.255+0000 7f8e52000700 1 --2- 172.21.15.18:0/2446863237 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8e54105800 0x7f8e541160f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:43:21.257 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:21.255+0000 7f8e52a00700 1 -- 172.21.15.18:0/2446863237 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8e54105800 msgr2=0x7f8e541160f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:43:21.257 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:21.255+0000 7f8e52a00700 1 --2- 172.21.15.18:0/2446863237 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8e54105800 0x7f8e541160f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:43:21.257 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:21.255+0000 7f8e52a00700 1 -- 172.21.15.18:0/2446863237 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e44005040 con 0x7f8e54104e60 2024-04-12T21:43:21.257 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:21.255+0000 7f8e52000700 1 --2- 172.21.15.18:0/2446863237 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8e54105800 0x7f8e541160f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-12T21:43:21.257 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:21.255+0000 7f8e52a00700 1 --2- 172.21.15.18:0/2446863237 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8e54104e60 0x7f8e54115bb0 secure :-1 s=READY pgs=1101 cs=0 l=1 rev1=1 crypto rx=0x7f8e44009180 tx=0x7f8e44009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:43:21.258 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:21.255+0000 7f8e4aa00700 1 -- 172.21.15.18:0/2446863237 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8e4400e040 con 0x7f8e54104e60 2024-04-12T21:43:21.258 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:21.255+0000 7f8e4aa00700 1 -- 172.21.15.18:0/2446863237 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f8e4400bbc0 con 0x7f8e54104e60 2024-04-12T21:43:21.258 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:21.255+0000 7f8e53e00700 1 -- 172.21.15.18:0/2446863237 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8e5410fdd0 con 0x7f8e54104e60 2024-04-12T21:43:21.258 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:21.256+0000 7f8e53e00700 1 -- 172.21.15.18:0/2446863237 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8e54110390 con 0x7f8e54104e60 2024-04-12T21:43:21.258 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:21.255+0000 7f8e4aa00700 1 -- 172.21.15.18:0/2446863237 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8e4401c630 con 0x7f8e54104e60 2024-04-12T21:43:21.258 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:21.257+0000 7f8e53e00700 1 -- 172.21.15.18:0/2446863237 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8e541005b0 con 0x7f8e54104e60 2024-04-12T21:43:21.259 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:21.258+0000 7f8e4aa00700 1 -- 172.21.15.18:0/2446863237 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7f8e4400bd30 con 0x7f8e54104e60 2024-04-12T21:43:21.260 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:21.259+0000 7f8e4aa00700 1 --2- 172.21.15.18:0/2446863237 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f8e3806ec00 0x7f8e380710b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:43:21.260 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:21.259+0000 7f8e4aa00700 1 -- 172.21.15.18:0/2446863237 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 6757+0+0 (secure 0 0 0) 0x7f8e4408fe40 con 0x7f8e54104e60 2024-04-12T21:43:21.260 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:21.259+0000 7f8e52000700 1 --2- 172.21.15.18:0/2446863237 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f8e3806ec00 0x7f8e380710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:43:21.261 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:21.260+0000 7f8e52000700 1 --2- 172.21.15.18:0/2446863237 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f8e3806ec00 0x7f8e380710b0 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f8e40007fd0 tx=0x7f8e40009040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:43:21.265 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:21.264+0000 7f8e4aa00700 1 -- 172.21.15.18:0/2446863237 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f8e44058ac0 con 0x7f8e54104e60 2024-04-12T21:43:21.522 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:21.522+0000 7f8e53e00700 1 -- 172.21.15.18:0/2446863237 --> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f8e5404bf40 con 0x7f8e3806ec00 2024-04-12T21:43:21.525 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:21.525+0000 7f8e4aa00700 1 -- 172.21.15.18:0/2446863237 <== mgr.34102 v2:172.21.15.18:6800/3812891919 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f8e5404bf40 con 0x7f8e3806ec00 2024-04-12T21:43:21.528 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:21.528+0000 7f8e53e00700 1 -- 172.21.15.18:0/2446863237 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f8e3806ec00 msgr2=0x7f8e380710b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:43:21.528 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:21.528+0000 7f8e53e00700 1 --2- 172.21.15.18:0/2446863237 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f8e3806ec00 0x7f8e380710b0 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f8e40007fd0 tx=0x7f8e40009040 comp rx=0 tx=0).stop 2024-04-12T21:43:21.528 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:21.528+0000 7f8e53e00700 1 -- 172.21.15.18:0/2446863237 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8e54104e60 msgr2=0x7f8e54115bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:43:21.529 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:21.528+0000 7f8e53e00700 1 --2- 172.21.15.18:0/2446863237 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8e54104e60 0x7f8e54115bb0 secure :-1 s=READY pgs=1101 cs=0 l=1 rev1=1 crypto rx=0x7f8e44009180 tx=0x7f8e44009e30 comp rx=0 tx=0).stop 2024-04-12T21:43:21.529 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:21.528+0000 7f8e53e00700 1 -- 172.21.15.18:0/2446863237 shutdown_connections 2024-04-12T21:43:21.529 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:21.528+0000 7f8e53e00700 1 --2- 172.21.15.18:0/2446863237 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f8e3806ec00 0x7f8e380710b0 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:43:21.529 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:21.528+0000 7f8e53e00700 1 --2- 172.21.15.18:0/2446863237 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8e54104e60 0x7f8e54115bb0 unknown :-1 s=CLOSED pgs=1101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:43:21.529 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:21.528+0000 7f8e53e00700 1 --2- 172.21.15.18:0/2446863237 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8e54105800 0x7f8e541160f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:43:21.529 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:21.528+0000 7f8e53e00700 1 -- 172.21.15.18:0/2446863237 >> 172.21.15.18:0/2446863237 conn(0x7f8e540fab10 msgr2=0x7f8e540fca40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:43:21.529 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:21.528+0000 7f8e53e00700 1 -- 172.21.15.18:0/2446863237 shutdown_connections 2024-04-12T21:43:21.529 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:21.528+0000 7f8e53e00700 1 -- 172.21.15.18:0/2446863237 wait complete. 2024-04-12T21:43:21.545 INFO:teuthology.orchestra.run.smithi018.stdout:true 2024-04-12T21:43:21.687 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:21.686+0000 7f3927400700 1 -- 172.21.15.18:0/3910795988 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f39201037a0 msgr2=0x7f39200fce10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:43:21.688 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:21.686+0000 7f3927400700 1 --2- 172.21.15.18:0/3910795988 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f39201037a0 0x7f39200fce10 secure :-1 s=READY pgs=1102 cs=0 l=1 rev1=1 crypto rx=0x7f3910004940 tx=0x7f3910009e30 comp rx=0 tx=0).stop 2024-04-12T21:43:21.688 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:21.687+0000 7f3927400700 1 -- 172.21.15.18:0/3910795988 shutdown_connections 2024-04-12T21:43:21.688 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:21.687+0000 7f3927400700 1 --2- 172.21.15.18:0/3910795988 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f39201037a0 0x7f39200fce10 unknown :-1 s=CLOSED pgs=1102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:43:21.688 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:21.687+0000 7f3927400700 1 --2- 172.21.15.18:0/3910795988 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3920102e00 0x7f39201031d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:43:21.688 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:21.687+0000 7f3927400700 1 -- 172.21.15.18:0/3910795988 >> 172.21.15.18:0/3910795988 conn(0x7f39200f89b0 msgr2=0x7f39200fadc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:43:21.688 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:21.687+0000 7f3927400700 1 -- 172.21.15.18:0/3910795988 shutdown_connections 2024-04-12T21:43:21.688 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:21.687+0000 7f3927400700 1 -- 172.21.15.18:0/3910795988 wait complete. 2024-04-12T21:43:21.689 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:21.688+0000 7f3927400700 1 Processor -- start 2024-04-12T21:43:21.689 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:21.688+0000 7f3927400700 1 -- start start 2024-04-12T21:43:21.689 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:21.688+0000 7f3927400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3920102e00 0x7f3920113990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:43:21.690 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:21.688+0000 7f3927400700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f39201037a0 0x7f3920113ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:43:21.690 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:21.688+0000 7f3927400700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39201145b0 con 0x7f3920102e00 2024-04-12T21:43:21.690 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:21.688+0000 7f3927400700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f392010db40 con 0x7f39201037a0 2024-04-12T21:43:21.690 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:21.689+0000 7f3924a00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3920102e00 0x7f3920113990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:43:21.690 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:21.689+0000 7f3924a00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3920102e00 0x7f3920113990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:37494/0 (socket says 172.21.15.18:37494) 2024-04-12T21:43:21.690 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:21.689+0000 7f3924a00700 1 -- 172.21.15.18:0/2614998282 learned_addr learned my addr 172.21.15.18:0/2614998282 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:43:21.690 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:21.689+0000 7f3917e00700 1 --2- 172.21.15.18:0/2614998282 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f39201037a0 0x7f3920113ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:43:21.690 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:21.689+0000 7f3924a00700 1 -- 172.21.15.18:0/2614998282 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f39201037a0 msgr2=0x7f3920113ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:43:21.690 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:21.689+0000 7f3924a00700 1 --2- 172.21.15.18:0/2614998282 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f39201037a0 0x7f3920113ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:43:21.690 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:21.689+0000 7f3924a00700 1 -- 172.21.15.18:0/2614998282 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3910005040 con 0x7f3920102e00 2024-04-12T21:43:21.691 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:21.690+0000 7f3924a00700 1 --2- 172.21.15.18:0/2614998282 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3920102e00 0x7f3920113990 secure :-1 s=READY pgs=1103 cs=0 l=1 rev1=1 crypto rx=0x7f390800b790 tx=0x7f3908009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:43:21.691 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:21.690+0000 7f391d600700 1 -- 172.21.15.18:0/2614998282 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f390800f770 con 0x7f3920102e00 2024-04-12T21:43:21.691 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:21.690+0000 7f391d600700 1 -- 172.21.15.18:0/2614998282 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f390800fda0 con 0x7f3920102e00 2024-04-12T21:43:21.691 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:21.690+0000 7f391d600700 1 -- 172.21.15.18:0/2614998282 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f390800d310 con 0x7f3920102e00 2024-04-12T21:43:21.691 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:21.690+0000 7f3927400700 1 -- 172.21.15.18:0/2614998282 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f392010de20 con 0x7f3920102e00 2024-04-12T21:43:21.691 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:21.690+0000 7f3927400700 1 -- 172.21.15.18:0/2614998282 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f392010e3e0 con 0x7f3920102e00 2024-04-12T21:43:21.692 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:21.691+0000 7f3927400700 1 -- 172.21.15.18:0/2614998282 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3920100110 con 0x7f3920102e00 2024-04-12T21:43:21.692 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:21.692+0000 7f391d600700 1 -- 172.21.15.18:0/2614998282 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7f3908010420 con 0x7f3920102e00 2024-04-12T21:43:21.694 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:21.694+0000 7f391d600700 1 --2- 172.21.15.18:0/2614998282 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f390006ec00 0x7f39000710b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:43:21.694 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:21.694+0000 7f391d600700 1 -- 172.21.15.18:0/2614998282 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 6757+0+0 (secure 0 0 0) 0x7f390808e9a0 con 0x7f3920102e00 2024-04-12T21:43:21.694 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:21.694+0000 7f3917e00700 1 --2- 172.21.15.18:0/2614998282 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f390006ec00 0x7f39000710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:43:21.695 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:21.695+0000 7f3917e00700 1 --2- 172.21.15.18:0/2614998282 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f390006ec00 0x7f39000710b0 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f391000b020 tx=0x7f3910009e30 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:43:21.699 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:21.699+0000 7f391d600700 1 -- 172.21.15.18:0/2614998282 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f39080576a0 con 0x7f3920102e00 2024-04-12T21:43:21.854 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:43:21 smithi018 ceph-mon[112590]: pgmap v1310: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:43:21.947 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:21.946+0000 7f3927400700 1 -- 172.21.15.18:0/2614998282 --> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f3920109fc0 con 0x7f390006ec00 2024-04-12T21:43:21.950 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:21.949+0000 7f391d600700 1 -- 172.21.15.18:0/2614998282 <== mgr.34102 v2:172.21.15.18:6800/3812891919 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f3920109fc0 con 0x7f390006ec00 2024-04-12T21:43:21.952 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:21.952+0000 7f3927400700 1 -- 172.21.15.18:0/2614998282 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f390006ec00 msgr2=0x7f39000710b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:43:21.952 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:21.952+0000 7f3927400700 1 --2- 172.21.15.18:0/2614998282 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f390006ec00 0x7f39000710b0 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f391000b020 tx=0x7f3910009e30 comp rx=0 tx=0).stop 2024-04-12T21:43:21.952 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:21.952+0000 7f3927400700 1 -- 172.21.15.18:0/2614998282 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3920102e00 msgr2=0x7f3920113990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:43:21.952 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:21.952+0000 7f3927400700 1 --2- 172.21.15.18:0/2614998282 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3920102e00 0x7f3920113990 secure :-1 s=READY pgs=1103 cs=0 l=1 rev1=1 crypto rx=0x7f390800b790 tx=0x7f3908009250 comp rx=0 tx=0).stop 2024-04-12T21:43:21.953 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:21.952+0000 7f3927400700 1 -- 172.21.15.18:0/2614998282 shutdown_connections 2024-04-12T21:43:21.953 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:21.952+0000 7f3927400700 1 --2- 172.21.15.18:0/2614998282 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f390006ec00 0x7f39000710b0 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:43:21.953 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:21.952+0000 7f3927400700 1 --2- 172.21.15.18:0/2614998282 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3920102e00 0x7f3920113990 unknown :-1 s=CLOSED pgs=1103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:43:21.953 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:21.952+0000 7f3927400700 1 --2- 172.21.15.18:0/2614998282 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f39201037a0 0x7f3920113ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:43:21.953 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:21.952+0000 7f3927400700 1 -- 172.21.15.18:0/2614998282 >> 172.21.15.18:0/2614998282 conn(0x7f39200f89b0 msgr2=0x7f39200fa770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:43:21.953 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:21.953+0000 7f3927400700 1 -- 172.21.15.18:0/2614998282 shutdown_connections 2024-04-12T21:43:21.953 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:21.953+0000 7f3927400700 1 -- 172.21.15.18:0/2614998282 wait complete. 2024-04-12T21:43:22.033 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:43:21 smithi186 ceph-mon[86379]: pgmap v1310: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:43:22.114 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:22.113+0000 7ffad5e00700 1 -- 172.21.15.18:0/1693496225 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ffad00fdbe0 msgr2=0x7ffad00fdfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:43:22.114 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:22.113+0000 7ffad5e00700 1 --2- 172.21.15.18:0/1693496225 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ffad00fdbe0 0x7ffad00fdfb0 secure :-1 s=READY pgs=1104 cs=0 l=1 rev1=1 crypto rx=0x7ffac0004940 tx=0x7ffac0009e30 comp rx=0 tx=0).stop 2024-04-12T21:43:22.114 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:22.114+0000 7ffad5e00700 1 -- 172.21.15.18:0/1693496225 shutdown_connections 2024-04-12T21:43:22.114 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:22.114+0000 7ffad5e00700 1 --2- 172.21.15.18:0/1693496225 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ffad00fb3b0 0x7ffad00fb820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:43:22.115 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:22.114+0000 7ffad5e00700 1 --2- 172.21.15.18:0/1693496225 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ffad00fdbe0 0x7ffad00fdfb0 unknown :-1 s=CLOSED pgs=1104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:43:22.115 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:22.114+0000 7ffad5e00700 1 -- 172.21.15.18:0/1693496225 >> 172.21.15.18:0/1693496225 conn(0x7ffad0072310 msgr2=0x7ffad0072710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:43:22.115 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:22.114+0000 7ffad5e00700 1 -- 172.21.15.18:0/1693496225 shutdown_connections 2024-04-12T21:43:22.115 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:22.114+0000 7ffad5e00700 1 -- 172.21.15.18:0/1693496225 wait complete. 2024-04-12T21:43:22.115 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:22.115+0000 7ffad5e00700 1 Processor -- start 2024-04-12T21:43:22.115 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:22.115+0000 7ffad5e00700 1 -- start start 2024-04-12T21:43:22.116 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:22.116+0000 7ffad5e00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ffad00fb3b0 0x7ffad0115b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:43:22.116 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:22.116+0000 7ffad5e00700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ffad00fdbe0 0x7ffad01160b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:43:22.116 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:22.116+0000 7ffad5e00700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffad0116700 con 0x7ffad00fb3b0 2024-04-12T21:43:22.116 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:22.116+0000 7ffad5e00700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffad010fce0 con 0x7ffad00fdbe0 2024-04-12T21:43:22.116 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:22.116+0000 7ffacea00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ffad00fb3b0 0x7ffad0115b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:43:22.116 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:22.116+0000 7ffacea00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ffad00fb3b0 0x7ffad0115b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:37512/0 (socket says 172.21.15.18:37512) 2024-04-12T21:43:22.117 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:22.116+0000 7ffacea00700 1 -- 172.21.15.18:0/2466863811 learned_addr learned my addr 172.21.15.18:0/2466863811 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:43:22.118 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:22.116+0000 7fface000700 1 --2- 172.21.15.18:0/2466863811 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ffad00fdbe0 0x7ffad01160b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:43:22.118 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:22.117+0000 7ffacea00700 1 -- 172.21.15.18:0/2466863811 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ffad00fdbe0 msgr2=0x7ffad01160b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:43:22.118 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:22.117+0000 7ffacea00700 1 --2- 172.21.15.18:0/2466863811 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ffad00fdbe0 0x7ffad01160b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:43:22.118 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:22.117+0000 7ffacea00700 1 -- 172.21.15.18:0/2466863811 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffac0005040 con 0x7ffad00fb3b0 2024-04-12T21:43:22.118 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:22.117+0000 7fface000700 1 --2- 172.21.15.18:0/2466863811 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ffad00fdbe0 0x7ffad01160b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-12T21:43:22.118 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:22.117+0000 7ffacea00700 1 --2- 172.21.15.18:0/2466863811 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ffad00fb3b0 0x7ffad0115b70 secure :-1 s=READY pgs=1105 cs=0 l=1 rev1=1 crypto rx=0x7ffac0009150 tx=0x7ffac0009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:43:22.118 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:22.118+0000 7ffac6a00700 1 -- 172.21.15.18:0/2466863811 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ffac000e040 con 0x7ffad00fb3b0 2024-04-12T21:43:22.118 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:22.118+0000 7ffac6a00700 1 -- 172.21.15.18:0/2466863811 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ffac0013420 con 0x7ffad00fb3b0 2024-04-12T21:43:22.118 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:22.118+0000 7ffad5e00700 1 -- 172.21.15.18:0/2466863811 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffad010ff60 con 0x7ffad00fb3b0 2024-04-12T21:43:22.118 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:22.118+0000 7ffac6a00700 1 -- 172.21.15.18:0/2466863811 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ffac001c630 con 0x7ffad00fb3b0 2024-04-12T21:43:22.118 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:22.118+0000 7ffad5e00700 1 -- 172.21.15.18:0/2466863811 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffad0110520 con 0x7ffad00fb3b0 2024-04-12T21:43:22.119 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:22.119+0000 7ffad5e00700 1 -- 172.21.15.18:0/2466863811 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffad00fc3c0 con 0x7ffad00fb3b0 2024-04-12T21:43:22.120 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:22.120+0000 7ffac6a00700 1 -- 172.21.15.18:0/2466863811 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7ffac0013590 con 0x7ffad00fb3b0 2024-04-12T21:43:22.121 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:22.121+0000 7ffac6a00700 1 --2- 172.21.15.18:0/2466863811 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7ffab406ec00 0x7ffab40710b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:43:22.121 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:22.121+0000 7ffac6a00700 1 -- 172.21.15.18:0/2466863811 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 6757+0+0 (secure 0 0 0) 0x7ffac008fce0 con 0x7ffad00fb3b0 2024-04-12T21:43:22.122 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:22.122+0000 7fface000700 1 --2- 172.21.15.18:0/2466863811 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7ffab406ec00 0x7ffab40710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:43:22.122 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:22.122+0000 7fface000700 1 --2- 172.21.15.18:0/2466863811 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7ffab406ec00 0x7ffab40710b0 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7ffabc009fd0 tx=0x7ffabc009ee0 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:43:22.126 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:22.126+0000 7ffac6a00700 1 -- 172.21.15.18:0/2466863811 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7ffac005c490 con 0x7ffad00fb3b0 2024-04-12T21:43:22.374 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:22.374+0000 7ffad5e00700 1 -- 172.21.15.18:0/2466863811 --> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7ffad01111c0 con 0x7ffab406ec00 2024-04-12T21:43:22.404 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:22.404+0000 7ffac6a00700 1 -- 172.21.15.18:0/2466863811 <== mgr.34102 v2:172.21.15.18:6800/3812891919 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7ffad01111c0 con 0x7ffab406ec00 2024-04-12T21:43:22.404 INFO:teuthology.orchestra.run.smithi018.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-12T21:43:22.405 INFO:teuthology.orchestra.run.smithi018.stdout:alertmanager.smithi018 smithi018 *:9093,9094 running (43m) 5m ago 58m 14.8M - 0.25.0 c8568f914cd2 c332090068e8 2024-04-12T21:43:22.405 INFO:teuthology.orchestra.run.smithi018.stdout:crash.smithi018 smithi018 running (40m) 5m ago 58m 6665k - 19.0.0-2879-g324baf8d 8717720dc5e5 b7f80d1f7982 2024-04-12T21:43:22.405 INFO:teuthology.orchestra.run.smithi018.stdout:crash.smithi186 smithi186 running (40m) 5m ago 57m 6660k - 19.0.0-2879-g324baf8d 8717720dc5e5 a4f55c81b623 2024-04-12T21:43:22.405 INFO:teuthology.orchestra.run.smithi018.stdout:grafana.smithi018 smithi018 *:3000 running (43m) 5m ago 58m 73.8M - 9.4.12 2bacad6d85d8 b08e3bf176dd 2024-04-12T21:43:22.405 INFO:teuthology.orchestra.run.smithi018.stdout:mds.cephfs.smithi018.acqbsc smithi018 running (36m) 5m ago 51m 21.9M - 19.0.0-2879-g324baf8d 8717720dc5e5 021f99ab24ae 2024-04-12T21:43:22.405 INFO:teuthology.orchestra.run.smithi018.stdout:mds.cephfs.smithi018.szjluo smithi018 running (35m) 5m ago 51m 21.7M - 19.0.0-2879-g324baf8d 8717720dc5e5 0003117479d4 2024-04-12T21:43:22.405 INFO:teuthology.orchestra.run.smithi018.stdout:mds.cephfs.smithi186.jdllyn smithi186 running (20m) 5m ago 51m 16.5M - 19.0.0-2879-g324baf8d 8717720dc5e5 21b1f46eef23 2024-04-12T21:43:22.405 INFO:teuthology.orchestra.run.smithi018.stdout:mds.cephfs.smithi186.wjnafs smithi186 running (5m) 5m ago 51m 11.0M - 19.0.0-2879-g324baf8d 8717720dc5e5 3cf1ef4148e2 2024-04-12T21:43:22.405 INFO:teuthology.orchestra.run.smithi018.stdout:mgr.smithi018.xjzwga smithi018 *:8443,9283,8765 running (46m) 5m ago 60m 517M - 19.0.0-2879-g324baf8d 8717720dc5e5 56088305eab7 2024-04-12T21:43:22.405 INFO:teuthology.orchestra.run.smithi018.stdout:mgr.smithi186.kqcfxk smithi186 *:8443,9283,8765 running (45m) 5m ago 57m 460M - 19.0.0-2879-g324baf8d 8717720dc5e5 09cd766e5e0d 2024-04-12T21:43:22.405 INFO:teuthology.orchestra.run.smithi018.stdout:mon.smithi018 smithi018 running (41m) 5m ago 60m 88.2M 2048M 19.0.0-2879-g324baf8d 8717720dc5e5 df3e201dfed6 2024-04-12T21:43:22.405 INFO:teuthology.orchestra.run.smithi018.stdout:mon.smithi186 smithi186 running (41m) 5m ago 57m 53.6M 2048M 19.0.0-2879-g324baf8d 8717720dc5e5 c77498da9b2b 2024-04-12T21:43:22.405 INFO:teuthology.orchestra.run.smithi018.stdout:node-exporter.smithi018 smithi018 *:9100 running (45m) 5m ago 58m 8468k - 1.5.0 0da6a335fe13 326c3d9c6847 2024-04-12T21:43:22.405 INFO:teuthology.orchestra.run.smithi018.stdout:node-exporter.smithi186 smithi186 *:9100 running (44m) 5m ago 57m 8166k - 1.5.0 0da6a335fe13 ec2112baeecf 2024-04-12T21:43:22.406 INFO:teuthology.orchestra.run.smithi018.stdout:osd.0 smithi018 running (39m) 5m ago 56m 123M 1559M 19.0.0-2879-g324baf8d 8717720dc5e5 60b1273ca047 2024-04-12T21:43:22.406 INFO:teuthology.orchestra.run.smithi018.stdout:osd.1 smithi018 running (39m) 5m ago 55m 95.4M 1559M 19.0.0-2879-g324baf8d 8717720dc5e5 170ce1e5fb8d 2024-04-12T21:43:22.406 INFO:teuthology.orchestra.run.smithi018.stdout:osd.2 smithi018 running (38m) 5m ago 54m 94.6M 1559M 19.0.0-2879-g324baf8d 8717720dc5e5 c2cbc6aa2839 2024-04-12T21:43:22.406 INFO:teuthology.orchestra.run.smithi018.stdout:osd.3 smithi186 running (38m) 5m ago 54m 101M 2581M 19.0.0-2879-g324baf8d 8717720dc5e5 0138e8262e4b 2024-04-12T21:43:22.406 INFO:teuthology.orchestra.run.smithi018.stdout:osd.4 smithi186 running (37m) 5m ago 53m 91.5M 2581M 19.0.0-2879-g324baf8d 8717720dc5e5 b20cb63b25c9 2024-04-12T21:43:22.406 INFO:teuthology.orchestra.run.smithi018.stdout:osd.5 smithi186 running (36m) 5m ago 53m 92.2M 2581M 19.0.0-2879-g324baf8d 8717720dc5e5 2513bea2610e 2024-04-12T21:43:22.406 INFO:teuthology.orchestra.run.smithi018.stdout:prometheus.smithi018 smithi018 *:9095 running (44m) 5m ago 57m 69.6M - 2.43.0 a07b618ecd1d f58841127e21 2024-04-12T21:43:22.407 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:22.407+0000 7ffad5e00700 1 -- 172.21.15.18:0/2466863811 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7ffab406ec00 msgr2=0x7ffab40710b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:43:22.407 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:22.407+0000 7ffad5e00700 1 --2- 172.21.15.18:0/2466863811 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7ffab406ec00 0x7ffab40710b0 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7ffabc009fd0 tx=0x7ffabc009ee0 comp rx=0 tx=0).stop 2024-04-12T21:43:22.408 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:22.407+0000 7ffad5e00700 1 -- 172.21.15.18:0/2466863811 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ffad00fb3b0 msgr2=0x7ffad0115b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:43:22.408 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:22.407+0000 7ffad5e00700 1 --2- 172.21.15.18:0/2466863811 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ffad00fb3b0 0x7ffad0115b70 secure :-1 s=READY pgs=1105 cs=0 l=1 rev1=1 crypto rx=0x7ffac0009150 tx=0x7ffac0009e30 comp rx=0 tx=0).stop 2024-04-12T21:43:22.408 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:22.407+0000 7ffad5e00700 1 -- 172.21.15.18:0/2466863811 shutdown_connections 2024-04-12T21:43:22.408 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:22.407+0000 7ffad5e00700 1 --2- 172.21.15.18:0/2466863811 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7ffab406ec00 0x7ffab40710b0 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:43:22.408 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:22.407+0000 7ffad5e00700 1 --2- 172.21.15.18:0/2466863811 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ffad00fb3b0 0x7ffad0115b70 unknown :-1 s=CLOSED pgs=1105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:43:22.408 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:22.407+0000 7ffad5e00700 1 --2- 172.21.15.18:0/2466863811 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ffad00fdbe0 0x7ffad01160b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:43:22.408 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:22.407+0000 7ffad5e00700 1 -- 172.21.15.18:0/2466863811 >> 172.21.15.18:0/2466863811 conn(0x7ffad0072310 msgr2=0x7ffad00f8820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:43:22.408 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:22.407+0000 7ffad5e00700 1 -- 172.21.15.18:0/2466863811 shutdown_connections 2024-04-12T21:43:22.408 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:22.408+0000 7ffad5e00700 1 -- 172.21.15.18:0/2466863811 wait complete. 2024-04-12T21:43:22.570 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:22.570+0000 7f9be1a00700 1 -- 172.21.15.18:0/3476762854 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9bdc0708a0 msgr2=0x7f9bdc070d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:43:22.571 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:22.570+0000 7f9be1a00700 1 --2- 172.21.15.18:0/3476762854 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9bdc0708a0 0x7f9bdc070d10 secure :-1 s=READY pgs=1106 cs=0 l=1 rev1=1 crypto rx=0x7f9bc4004970 tx=0x7f9bc4009e30 comp rx=0 tx=0).stop 2024-04-12T21:43:22.571 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:22.570+0000 7f9be1a00700 1 -- 172.21.15.18:0/3476762854 shutdown_connections 2024-04-12T21:43:22.571 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:22.570+0000 7f9be1a00700 1 --2- 172.21.15.18:0/3476762854 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9bdc0708a0 0x7f9bdc070d10 unknown :-1 s=CLOSED pgs=1106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:43:22.571 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:22.570+0000 7f9be1a00700 1 --2- 172.21.15.18:0/3476762854 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9bdc071f90 0x7f9bdc070360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:43:22.571 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:22.570+0000 7f9be1a00700 1 -- 172.21.15.18:0/3476762854 >> 172.21.15.18:0/3476762854 conn(0x7f9bdc0f8a20 msgr2=0x7f9bdc0fae30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:43:22.571 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:22.570+0000 7f9be1a00700 1 -- 172.21.15.18:0/3476762854 shutdown_connections 2024-04-12T21:43:22.571 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:22.570+0000 7f9be1a00700 1 -- 172.21.15.18:0/3476762854 wait complete. 2024-04-12T21:43:22.572 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:22.571+0000 7f9be1a00700 1 Processor -- start 2024-04-12T21:43:22.572 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:22.571+0000 7f9be1a00700 1 -- start start 2024-04-12T21:43:22.572 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:22.572+0000 7f9be1a00700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9bdc0708a0 0x7f9bdc119f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:43:22.572 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:22.572+0000 7f9be1a00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9bdc071f90 0x7f9bdc11a4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:43:22.572 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:22.572+0000 7f9be1a00700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9bdc11ab20 con 0x7f9bdc071f90 2024-04-12T21:43:22.572 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:22.572+0000 7f9be1a00700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9bdc114100 con 0x7f9bdc0708a0 2024-04-12T21:43:22.573 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:22.572+0000 7f9bd2000700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9bdc071f90 0x7f9bdc11a4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:43:22.573 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:22.572+0000 7f9bd2000700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9bdc071f90 0x7f9bdc11a4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:37524/0 (socket says 172.21.15.18:37524) 2024-04-12T21:43:22.573 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:22.572+0000 7f9bd2000700 1 -- 172.21.15.18:0/2434083329 learned_addr learned my addr 172.21.15.18:0/2434083329 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:43:22.574 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:22.572+0000 7f9bdaa00700 1 --2- 172.21.15.18:0/2434083329 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9bdc0708a0 0x7f9bdc119f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:43:22.574 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:22.573+0000 7f9bd2000700 1 -- 172.21.15.18:0/2434083329 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9bdc0708a0 msgr2=0x7f9bdc119f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:43:22.574 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:22.573+0000 7f9bd2000700 1 --2- 172.21.15.18:0/2434083329 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9bdc0708a0 0x7f9bdc119f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:43:22.574 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:22.573+0000 7f9bd2000700 1 -- 172.21.15.18:0/2434083329 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9bc4005040 con 0x7f9bdc071f90 2024-04-12T21:43:22.574 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:22.573+0000 7f9bd2000700 1 --2- 172.21.15.18:0/2434083329 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9bdc071f90 0x7f9bdc11a4d0 secure :-1 s=READY pgs=1107 cs=0 l=1 rev1=1 crypto rx=0x7f9bc4004f50 tx=0x7f9bc4009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:43:22.574 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:22.573+0000 7f9bd3400700 1 -- 172.21.15.18:0/2434083329 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9bc400e040 con 0x7f9bdc071f90 2024-04-12T21:43:22.574 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:22.573+0000 7f9bd3400700 1 -- 172.21.15.18:0/2434083329 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f9bc400bc10 con 0x7f9bdc071f90 2024-04-12T21:43:22.574 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:22.573+0000 7f9be1a00700 1 -- 172.21.15.18:0/2434083329 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9bdc114380 con 0x7f9bdc071f90 2024-04-12T21:43:22.574 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:22.573+0000 7f9bd3400700 1 -- 172.21.15.18:0/2434083329 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9bc401c630 con 0x7f9bdc071f90 2024-04-12T21:43:22.574 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:22.573+0000 7f9be1a00700 1 -- 172.21.15.18:0/2434083329 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9bdc1148c0 con 0x7f9bdc071f90 2024-04-12T21:43:22.575 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:22.575+0000 7f9be1a00700 1 -- 172.21.15.18:0/2434083329 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9bdc071aa0 con 0x7f9bdc071f90 2024-04-12T21:43:22.576 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:22.576+0000 7f9bd3400700 1 -- 172.21.15.18:0/2434083329 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7f9bc4013420 con 0x7f9bdc071f90 2024-04-12T21:43:22.577 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:22.577+0000 7f9bd3400700 1 --2- 172.21.15.18:0/2434083329 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f9bc006ec00 0x7f9bc00710b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:43:22.577 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:22.577+0000 7f9bd3400700 1 -- 172.21.15.18:0/2434083329 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 6757+0+0 (secure 0 0 0) 0x7f9bc408fd70 con 0x7f9bdc071f90 2024-04-12T21:43:22.577 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:22.577+0000 7f9bdaa00700 1 --2- 172.21.15.18:0/2434083329 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f9bc006ec00 0x7f9bc00710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:43:22.578 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:22.578+0000 7f9bdaa00700 1 --2- 172.21.15.18:0/2434083329 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f9bc006ec00 0x7f9bc00710b0 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f9bcc004990 tx=0x7f9bcc009ee0 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:43:22.582 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:22.582+0000 7f9bd3400700 1 -- 172.21.15.18:0/2434083329 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f9bc405c5d0 con 0x7f9bdc071f90 2024-04-12T21:43:22.907 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:22.906+0000 7f9be1a00700 1 -- 172.21.15.18:0/2434083329 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f9bdc114510 con 0x7f9bdc071f90 2024-04-12T21:43:22.907 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:43:22 smithi018 ceph-mon[112590]: from='client.36272 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:43:22.907 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:43:22 smithi018 ceph-mon[112590]: from='client.36276 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:43:22.908 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:22.907+0000 7f9bd3400700 1 -- 172.21.15.18:0/2434083329 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+623 (secure 0 0 0) 0x7f9bdc114510 con 0x7f9bdc071f90 2024-04-12T21:43:22.908 INFO:teuthology.orchestra.run.smithi018.stdout:{ 2024-04-12T21:43:22.908 INFO:teuthology.orchestra.run.smithi018.stdout: "mon": { 2024-04-12T21:43:22.908 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 19.0.0-2879-g324baf8d (324baf8ddb91dea46cac50cb535c306115791355) squid (dev)": 2 2024-04-12T21:43:22.908 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-12T21:43:22.908 INFO:teuthology.orchestra.run.smithi018.stdout: "mgr": { 2024-04-12T21:43:22.908 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 19.0.0-2879-g324baf8d (324baf8ddb91dea46cac50cb535c306115791355) squid (dev)": 2 2024-04-12T21:43:22.908 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-12T21:43:22.909 INFO:teuthology.orchestra.run.smithi018.stdout: "osd": { 2024-04-12T21:43:22.909 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 19.0.0-2879-g324baf8d (324baf8ddb91dea46cac50cb535c306115791355) squid (dev)": 6 2024-04-12T21:43:22.909 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-12T21:43:22.909 INFO:teuthology.orchestra.run.smithi018.stdout: "mds": { 2024-04-12T21:43:22.909 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 19.0.0-2879-g324baf8d (324baf8ddb91dea46cac50cb535c306115791355) squid (dev)": 4 2024-04-12T21:43:22.909 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-12T21:43:22.909 INFO:teuthology.orchestra.run.smithi018.stdout: "overall": { 2024-04-12T21:43:22.909 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 19.0.0-2879-g324baf8d (324baf8ddb91dea46cac50cb535c306115791355) squid (dev)": 14 2024-04-12T21:43:22.909 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-04-12T21:43:22.909 INFO:teuthology.orchestra.run.smithi018.stdout:} 2024-04-12T21:43:22.910 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:22.910+0000 7f9be1a00700 1 -- 172.21.15.18:0/2434083329 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f9bc006ec00 msgr2=0x7f9bc00710b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:43:22.910 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:22.910+0000 7f9be1a00700 1 --2- 172.21.15.18:0/2434083329 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f9bc006ec00 0x7f9bc00710b0 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f9bcc004990 tx=0x7f9bcc009ee0 comp rx=0 tx=0).stop 2024-04-12T21:43:22.910 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:22.910+0000 7f9be1a00700 1 -- 172.21.15.18:0/2434083329 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9bdc071f90 msgr2=0x7f9bdc11a4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:43:22.910 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:22.910+0000 7f9be1a00700 1 --2- 172.21.15.18:0/2434083329 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9bdc071f90 0x7f9bdc11a4d0 secure :-1 s=READY pgs=1107 cs=0 l=1 rev1=1 crypto rx=0x7f9bc4004f50 tx=0x7f9bc4009e30 comp rx=0 tx=0).stop 2024-04-12T21:43:22.911 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:22.910+0000 7f9be1a00700 1 -- 172.21.15.18:0/2434083329 shutdown_connections 2024-04-12T21:43:22.911 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:22.910+0000 7f9be1a00700 1 --2- 172.21.15.18:0/2434083329 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f9bc006ec00 0x7f9bc00710b0 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:43:22.911 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:22.910+0000 7f9be1a00700 1 --2- 172.21.15.18:0/2434083329 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9bdc0708a0 0x7f9bdc119f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:43:22.911 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:22.910+0000 7f9be1a00700 1 --2- 172.21.15.18:0/2434083329 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9bdc071f90 0x7f9bdc11a4d0 unknown :-1 s=CLOSED pgs=1107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:43:22.911 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:22.910+0000 7f9be1a00700 1 -- 172.21.15.18:0/2434083329 >> 172.21.15.18:0/2434083329 conn(0x7f9bdc0f8a20 msgr2=0x7f9bdc0f9a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:43:22.911 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:22.910+0000 7f9be1a00700 1 -- 172.21.15.18:0/2434083329 shutdown_connections 2024-04-12T21:43:22.911 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:22.910+0000 7f9be1a00700 1 -- 172.21.15.18:0/2434083329 wait complete. 2024-04-12T21:43:23.033 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:43:22 smithi186 ceph-mon[86379]: from='client.36272 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:43:23.034 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:43:22 smithi186 ceph-mon[86379]: from='client.36276 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:43:23.069 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:23.069+0000 7faf11400700 1 -- 172.21.15.18:0/1123141727 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7faf0c102710 msgr2=0x7faf0c0fbd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:43:23.069 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:23.069+0000 7faf11400700 1 --2- 172.21.15.18:0/1123141727 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7faf0c102710 0x7faf0c0fbd80 secure :-1 s=READY pgs=1108 cs=0 l=1 rev1=1 crypto rx=0x7faef8004940 tx=0x7faef8009e30 comp rx=0 tx=0).stop 2024-04-12T21:43:23.069 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:23.069+0000 7faf11400700 1 -- 172.21.15.18:0/1123141727 shutdown_connections 2024-04-12T21:43:23.069 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:23.069+0000 7faf11400700 1 --2- 172.21.15.18:0/1123141727 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7faf0c102710 0x7faf0c0fbd80 unknown :-1 s=CLOSED pgs=1108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:43:23.069 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:23.069+0000 7faf11400700 1 --2- 172.21.15.18:0/1123141727 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7faf0c101d70 0x7faf0c102140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:43:23.069 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:23.069+0000 7faf11400700 1 -- 172.21.15.18:0/1123141727 >> 172.21.15.18:0/1123141727 conn(0x7faf0c072bc0 msgr2=0x7faf0c072fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:43:23.069 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:23.069+0000 7faf11400700 1 -- 172.21.15.18:0/1123141727 shutdown_connections 2024-04-12T21:43:23.070 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:23.069+0000 7faf11400700 1 -- 172.21.15.18:0/1123141727 wait complete. 2024-04-12T21:43:23.070 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:23.070+0000 7faf11400700 1 Processor -- start 2024-04-12T21:43:23.070 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:23.070+0000 7faf11400700 1 -- start start 2024-04-12T21:43:23.071 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:23.071+0000 7faf11400700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7faf0c101d70 0x7faf0c0fe0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:43:23.071 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:23.071+0000 7faf11400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7faf0c102710 0x7faf0c0fe620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:43:23.071 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:23.071+0000 7faf11400700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf0c0febb0 con 0x7faf0c102710 2024-04-12T21:43:23.071 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:23.071+0000 7faf11400700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf0c0fecf0 con 0x7faf0c101d70 2024-04-12T21:43:23.071 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:23.071+0000 7faf09e00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7faf0c102710 0x7faf0c0fe620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:43:23.071 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:23.071+0000 7faf09e00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7faf0c102710 0x7faf0c0fe620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:37552/0 (socket says 172.21.15.18:37552) 2024-04-12T21:43:23.071 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:23.071+0000 7faf09e00700 1 -- 172.21.15.18:0/357429793 learned_addr learned my addr 172.21.15.18:0/357429793 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:43:23.073 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:23.071+0000 7faf0a800700 1 --2- 172.21.15.18:0/357429793 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7faf0c101d70 0x7faf0c0fe0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:43:23.073 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:23.072+0000 7faf09e00700 1 -- 172.21.15.18:0/357429793 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7faf0c101d70 msgr2=0x7faf0c0fe0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:43:23.073 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:23.072+0000 7faf09e00700 1 --2- 172.21.15.18:0/357429793 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7faf0c101d70 0x7faf0c0fe0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:43:23.073 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:23.072+0000 7faf09e00700 1 -- 172.21.15.18:0/357429793 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faef8005040 con 0x7faf0c102710 2024-04-12T21:43:23.073 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:23.072+0000 7faf0a800700 1 --2- 172.21.15.18:0/357429793 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7faf0c101d70 0x7faf0c0fe0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-12T21:43:23.073 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:23.072+0000 7faf09e00700 1 --2- 172.21.15.18:0/357429793 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7faf0c102710 0x7faf0c0fe620 secure :-1 s=READY pgs=1109 cs=0 l=1 rev1=1 crypto rx=0x7faef8009100 tx=0x7faef8009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:43:23.073 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:23.072+0000 7faf02a00700 1 -- 172.21.15.18:0/357429793 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7faef800e040 con 0x7faf0c102710 2024-04-12T21:43:23.073 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:23.072+0000 7faf02a00700 1 -- 172.21.15.18:0/357429793 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7faef8013420 con 0x7faf0c102710 2024-04-12T21:43:23.073 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:23.072+0000 7faf11400700 1 -- 172.21.15.18:0/357429793 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faf0c0fef40 con 0x7faf0c102710 2024-04-12T21:43:23.073 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:23.072+0000 7faf02a00700 1 -- 172.21.15.18:0/357429793 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7faef801c630 con 0x7faf0c102710 2024-04-12T21:43:23.073 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:23.073+0000 7faf11400700 1 -- 172.21.15.18:0/357429793 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faf0c11a2b0 con 0x7faf0c102710 2024-04-12T21:43:23.074 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:23.074+0000 7faf11400700 1 -- 172.21.15.18:0/357429793 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faf0c0fd4c0 con 0x7faf0c102710 2024-04-12T21:43:23.075 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:23.075+0000 7faf02a00700 1 -- 172.21.15.18:0/357429793 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7faef8013590 con 0x7faf0c102710 2024-04-12T21:43:23.076 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:23.076+0000 7faf02a00700 1 --2- 172.21.15.18:0/357429793 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7faef006ec60 0x7faef0071110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:43:23.077 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:23.076+0000 7faf02a00700 1 -- 172.21.15.18:0/357429793 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 6757+0+0 (secure 0 0 0) 0x7faef808fc60 con 0x7faf0c102710 2024-04-12T21:43:23.077 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:23.076+0000 7faf0a800700 1 --2- 172.21.15.18:0/357429793 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7faef006ec60 0x7faef0071110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:43:23.077 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:23.077+0000 7faf0a800700 1 --2- 172.21.15.18:0/357429793 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7faef006ec60 0x7faef0071110 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7faefc007fd0 tx=0x7faefc009040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:43:23.081 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:23.081+0000 7faf02a00700 1 -- 172.21.15.18:0/357429793 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7faef80588e0 con 0x7faf0c102710 2024-04-12T21:43:23.364 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:23.363+0000 7faf11400700 1 -- 172.21.15.18:0/357429793 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7faf0c04bed0 con 0x7faf0c102710 2024-04-12T21:43:23.365 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:23.365+0000 7faf02a00700 1 -- 172.21.15.18:0/357429793 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 33 v33) v1 ==== 76+0+1876 (secure 0 0 0) 0x7faef805bfa0 con 0x7faf0c102710 2024-04-12T21:43:23.365 INFO:teuthology.orchestra.run.smithi018.stdout:e33 2024-04-12T21:43:23.366 INFO:teuthology.orchestra.run.smithi018.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-12T21:43:23.366 INFO:teuthology.orchestra.run.smithi018.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-12T21:43:23.366 INFO:teuthology.orchestra.run.smithi018.stdout:legacy client fscid: 1 2024-04-12T21:43:23.366 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-12T21:43:23.366 INFO:teuthology.orchestra.run.smithi018.stdout:Filesystem 'cephfs' (1) 2024-04-12T21:43:23.366 INFO:teuthology.orchestra.run.smithi018.stdout:fs_name cephfs 2024-04-12T21:43:23.366 INFO:teuthology.orchestra.run.smithi018.stdout:epoch 24 2024-04-12T21:43:23.366 INFO:teuthology.orchestra.run.smithi018.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-04-12T21:43:23.366 INFO:teuthology.orchestra.run.smithi018.stdout:created 2024-04-12T20:51:57.362850+0000 2024-04-12T21:43:23.366 INFO:teuthology.orchestra.run.smithi018.stdout:modified 2024-04-12T21:06:59.717190+0000 2024-04-12T21:43:23.366 INFO:teuthology.orchestra.run.smithi018.stdout:tableserver 0 2024-04-12T21:43:23.366 INFO:teuthology.orchestra.run.smithi018.stdout:root 0 2024-04-12T21:43:23.367 INFO:teuthology.orchestra.run.smithi018.stdout:session_timeout 60 2024-04-12T21:43:23.367 INFO:teuthology.orchestra.run.smithi018.stdout:session_autoclose 300 2024-04-12T21:43:23.367 INFO:teuthology.orchestra.run.smithi018.stdout:max_file_size 1099511627776 2024-04-12T21:43:23.367 INFO:teuthology.orchestra.run.smithi018.stdout:max_xattr_size 65536 2024-04-12T21:43:23.367 INFO:teuthology.orchestra.run.smithi018.stdout:required_client_features {} 2024-04-12T21:43:23.367 INFO:teuthology.orchestra.run.smithi018.stdout:last_failure 0 2024-04-12T21:43:23.367 INFO:teuthology.orchestra.run.smithi018.stdout:last_failure_osd_epoch 100 2024-04-12T21:43:23.367 INFO:teuthology.orchestra.run.smithi018.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-12T21:43:23.367 INFO:teuthology.orchestra.run.smithi018.stdout:max_mds 2 2024-04-12T21:43:23.367 INFO:teuthology.orchestra.run.smithi018.stdout:in 0,1 2024-04-12T21:43:23.367 INFO:teuthology.orchestra.run.smithi018.stdout:up {} 2024-04-12T21:43:23.367 INFO:teuthology.orchestra.run.smithi018.stdout:failed 0,1 2024-04-12T21:43:23.368 INFO:teuthology.orchestra.run.smithi018.stdout:damaged 2024-04-12T21:43:23.368 INFO:teuthology.orchestra.run.smithi018.stdout:stopped 2024-04-12T21:43:23.368 INFO:teuthology.orchestra.run.smithi018.stdout:data_pools [3] 2024-04-12T21:43:23.368 INFO:teuthology.orchestra.run.smithi018.stdout:metadata_pool 2 2024-04-12T21:43:23.368 INFO:teuthology.orchestra.run.smithi018.stdout:inline_data disabled 2024-04-12T21:43:23.368 INFO:teuthology.orchestra.run.smithi018.stdout:balancer 2024-04-12T21:43:23.368 INFO:teuthology.orchestra.run.smithi018.stdout:bal_rank_mask -1 2024-04-12T21:43:23.368 INFO:teuthology.orchestra.run.smithi018.stdout:standby_count_wanted 1 2024-04-12T21:43:23.368 INFO:teuthology.orchestra.run.smithi018.stdout:qdb_cluster leader: 0 members: 2024-04-12T21:43:23.368 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-12T21:43:23.368 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-12T21:43:23.368 INFO:teuthology.orchestra.run.smithi018.stdout:Standby daemons: 2024-04-12T21:43:23.368 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-12T21:43:23.369 INFO:teuthology.orchestra.run.smithi018.stdout:[mds.cephfs.smithi018.acqbsc{-1:34436} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.18:6828/755179764,v1:172.21.15.18:6829/755179764] compat {c=[1],r=[1],i=[1fff]}] 2024-04-12T21:43:23.369 INFO:teuthology.orchestra.run.smithi018.stdout:[mds.cephfs.smithi018.szjluo{-1:34468} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.18:6826/2407992653,v1:172.21.15.18:6827/2407992653] compat {c=[1],r=[1],i=[1fff]}] 2024-04-12T21:43:23.369 INFO:teuthology.orchestra.run.smithi018.stdout:[mds.cephfs.smithi186.jdllyn{-1:44595} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.186:6824/3523444202,v1:172.21.15.186:6825/3523444202] compat {c=[1],r=[1],i=[1fff]}] 2024-04-12T21:43:23.369 INFO:teuthology.orchestra.run.smithi018.stdout:[mds.cephfs.smithi186.wjnafs{-1:44875} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.186:6826/3122443766,v1:172.21.15.186:6827/3122443766] compat {c=[1],r=[1],i=[1fff]}] 2024-04-12T21:43:23.369 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:23.368+0000 7faf11400700 1 -- 172.21.15.18:0/357429793 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7faef006ec60 msgr2=0x7faef0071110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:43:23.369 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:23.368+0000 7faf11400700 1 --2- 172.21.15.18:0/357429793 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7faef006ec60 0x7faef0071110 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7faefc007fd0 tx=0x7faefc009040 comp rx=0 tx=0).stop 2024-04-12T21:43:23.369 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:23.368+0000 7faf11400700 1 -- 172.21.15.18:0/357429793 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7faf0c102710 msgr2=0x7faf0c0fe620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:43:23.370 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:23.368+0000 7faf11400700 1 --2- 172.21.15.18:0/357429793 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7faf0c102710 0x7faf0c0fe620 secure :-1 s=READY pgs=1109 cs=0 l=1 rev1=1 crypto rx=0x7faef8009100 tx=0x7faef8009e30 comp rx=0 tx=0).stop 2024-04-12T21:43:23.370 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:23.368+0000 7faf11400700 1 -- 172.21.15.18:0/357429793 shutdown_connections 2024-04-12T21:43:23.370 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:23.368+0000 7faf11400700 1 --2- 172.21.15.18:0/357429793 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7faef006ec60 0x7faef0071110 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:43:23.370 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:23.368+0000 7faf11400700 1 --2- 172.21.15.18:0/357429793 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7faf0c101d70 0x7faf0c0fe0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:43:23.370 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:23.368+0000 7faf11400700 1 --2- 172.21.15.18:0/357429793 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7faf0c102710 0x7faf0c0fe620 unknown :-1 s=CLOSED pgs=1109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:43:23.370 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:23.368+0000 7faf11400700 1 -- 172.21.15.18:0/357429793 >> 172.21.15.18:0/357429793 conn(0x7faf0c072bc0 msgr2=0x7faf0c100600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:43:23.370 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:23.368+0000 7faf11400700 1 -- 172.21.15.18:0/357429793 shutdown_connections 2024-04-12T21:43:23.370 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:23.368+0000 7faf11400700 1 -- 172.21.15.18:0/357429793 wait complete. 2024-04-12T21:43:23.373 INFO:teuthology.orchestra.run.smithi018.stderr:dumped fsmap epoch 33 2024-04-12T21:43:23.530 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:23.528+0000 7f95e5e00700 1 -- 172.21.15.18:0/317719684 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f95e01057d0 msgr2=0x7f95e00fee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:43:23.530 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:23.528+0000 7f95e5e00700 1 --2- 172.21.15.18:0/317719684 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f95e01057d0 0x7f95e00fee40 secure :-1 s=READY pgs=1110 cs=0 l=1 rev1=1 crypto rx=0x7f95cc004940 tx=0x7f95cc009e30 comp rx=0 tx=0).stop 2024-04-12T21:43:23.530 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:23.529+0000 7f95e5e00700 1 -- 172.21.15.18:0/317719684 shutdown_connections 2024-04-12T21:43:23.530 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:23.529+0000 7f95e5e00700 1 --2- 172.21.15.18:0/317719684 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f95e01057d0 0x7f95e00fee40 unknown :-1 s=CLOSED pgs=1110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:43:23.530 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:23.529+0000 7f95e5e00700 1 --2- 172.21.15.18:0/317719684 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f95e0104e30 0x7f95e0105200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:43:23.530 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:23.529+0000 7f95e5e00700 1 -- 172.21.15.18:0/317719684 >> 172.21.15.18:0/317719684 conn(0x7f95e00fab10 msgr2=0x7f95e00fcef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:43:23.530 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:23.529+0000 7f95e5e00700 1 -- 172.21.15.18:0/317719684 shutdown_connections 2024-04-12T21:43:23.530 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:23.529+0000 7f95e5e00700 1 -- 172.21.15.18:0/317719684 wait complete. 2024-04-12T21:43:23.530 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:23.530+0000 7f95e5e00700 1 Processor -- start 2024-04-12T21:43:23.532 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:23.530+0000 7f95e5e00700 1 -- start start 2024-04-12T21:43:23.532 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:23.530+0000 7f95e5e00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f95e0104e30 0x7f95e0115ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:43:23.532 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:23.530+0000 7f95e5e00700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f95e01057d0 0x7f95e01160e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:43:23.532 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:23.530+0000 7f95e5e00700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95e01167c0 con 0x7f95e0104e30 2024-04-12T21:43:23.532 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:23.530+0000 7f95e5e00700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95e010fb40 con 0x7f95e01057d0 2024-04-12T21:43:23.532 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:23.531+0000 7f95e4a00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f95e0104e30 0x7f95e0115ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:43:23.532 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:23.531+0000 7f95e4a00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f95e0104e30 0x7f95e0115ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:37572/0 (socket says 172.21.15.18:37572) 2024-04-12T21:43:23.532 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:23.531+0000 7f95e4a00700 1 -- 172.21.15.18:0/1327251484 learned_addr learned my addr 172.21.15.18:0/1327251484 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:43:23.532 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:23.531+0000 7f95dfe00700 1 --2- 172.21.15.18:0/1327251484 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f95e01057d0 0x7f95e01160e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:43:23.532 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:23.531+0000 7f95e4a00700 1 -- 172.21.15.18:0/1327251484 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f95e01057d0 msgr2=0x7f95e01160e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:43:23.532 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:23.531+0000 7f95e4a00700 1 --2- 172.21.15.18:0/1327251484 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f95e01057d0 0x7f95e01160e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:43:23.532 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:23.531+0000 7f95e4a00700 1 -- 172.21.15.18:0/1327251484 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95cc005040 con 0x7f95e0104e30 2024-04-12T21:43:23.532 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:23.531+0000 7f95dfe00700 1 --2- 172.21.15.18:0/1327251484 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f95e01057d0 0x7f95e01160e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-12T21:43:23.533 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:23.532+0000 7f95e4a00700 1 --2- 172.21.15.18:0/1327251484 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f95e0104e30 0x7f95e0115ba0 secure :-1 s=READY pgs=1111 cs=0 l=1 rev1=1 crypto rx=0x7f95d400b790 tx=0x7f95d4009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:43:23.533 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:23.532+0000 7f95dcc00700 1 -- 172.21.15.18:0/1327251484 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f95d400f770 con 0x7f95e0104e30 2024-04-12T21:43:23.533 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:23.532+0000 7f95dcc00700 1 -- 172.21.15.18:0/1327251484 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f95d400fda0 con 0x7f95e0104e30 2024-04-12T21:43:23.534 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:23.532+0000 7f95dcc00700 1 -- 172.21.15.18:0/1327251484 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f95d400d310 con 0x7f95e0104e30 2024-04-12T21:43:23.534 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:23.532+0000 7f95e5e00700 1 -- 172.21.15.18:0/1327251484 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f95e010fe20 con 0x7f95e0104e30 2024-04-12T21:43:23.534 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:23.532+0000 7f95e5e00700 1 -- 172.21.15.18:0/1327251484 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f95e01103e0 con 0x7f95e0104e30 2024-04-12T21:43:23.534 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:23.533+0000 7f95e5e00700 1 -- 172.21.15.18:0/1327251484 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f95e0100580 con 0x7f95e0104e30 2024-04-12T21:43:23.534 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:23.534+0000 7f95dcc00700 1 -- 172.21.15.18:0/1327251484 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7f95d4010420 con 0x7f95e0104e30 2024-04-12T21:43:23.535 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:23.535+0000 7f95dcc00700 1 --2- 172.21.15.18:0/1327251484 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f95c806ec00 0x7f95c80710b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:43:23.535 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:23.535+0000 7f95dcc00700 1 -- 172.21.15.18:0/1327251484 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 6757+0+0 (secure 0 0 0) 0x7f95d408e9a0 con 0x7f95e0104e30 2024-04-12T21:43:23.536 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:23.536+0000 7f95dfe00700 1 --2- 172.21.15.18:0/1327251484 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f95c806ec00 0x7f95c80710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:43:23.536 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:23.536+0000 7f95dfe00700 1 --2- 172.21.15.18:0/1327251484 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f95c806ec00 0x7f95c80710b0 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f95cc00b3b0 tx=0x7f95cc009e30 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:43:23.540 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:23.540+0000 7f95dcc00700 1 -- 172.21.15.18:0/1327251484 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f95d40576a0 con 0x7f95e0104e30 2024-04-12T21:43:23.793 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:23.793+0000 7f95e5e00700 1 -- 172.21.15.18:0/1327251484 --> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f95e004bf40 con 0x7f95c806ec00 2024-04-12T21:43:23.795 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:23.794+0000 7f95dcc00700 1 -- 172.21.15.18:0/1327251484 <== mgr.34102 v2:172.21.15.18:6800/3812891919 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f95e004bf40 con 0x7f95c806ec00 2024-04-12T21:43:23.795 INFO:teuthology.orchestra.run.smithi018.stdout:{ 2024-04-12T21:43:23.795 INFO:teuthology.orchestra.run.smithi018.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:f4452def3be4d350f4a8e5feaf7ebb1511be9298fe68478ba7a2d6fbd7d69a63", 2024-04-12T21:43:23.795 INFO:teuthology.orchestra.run.smithi018.stdout: "in_progress": true, 2024-04-12T21:43:23.795 INFO:teuthology.orchestra.run.smithi018.stdout: "which": "Upgrading all daemon types on all hosts", 2024-04-12T21:43:23.795 INFO:teuthology.orchestra.run.smithi018.stdout: "services_complete": [ 2024-04-12T21:43:23.795 INFO:teuthology.orchestra.run.smithi018.stdout: "mgr", 2024-04-12T21:43:23.795 INFO:teuthology.orchestra.run.smithi018.stdout: "mds", 2024-04-12T21:43:23.795 INFO:teuthology.orchestra.run.smithi018.stdout: "crash", 2024-04-12T21:43:23.796 INFO:teuthology.orchestra.run.smithi018.stdout: "osd", 2024-04-12T21:43:23.796 INFO:teuthology.orchestra.run.smithi018.stdout: "mon" 2024-04-12T21:43:23.796 INFO:teuthology.orchestra.run.smithi018.stdout: ], 2024-04-12T21:43:23.796 INFO:teuthology.orchestra.run.smithi018.stdout: "progress": "16/21 daemons upgraded", 2024-04-12T21:43:23.796 INFO:teuthology.orchestra.run.smithi018.stdout: "message": "Currently upgrading mds daemons", 2024-04-12T21:43:23.796 INFO:teuthology.orchestra.run.smithi018.stdout: "is_paused": false 2024-04-12T21:43:23.796 INFO:teuthology.orchestra.run.smithi018.stdout:} 2024-04-12T21:43:23.798 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:23.797+0000 7f95e5e00700 1 -- 172.21.15.18:0/1327251484 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f95c806ec00 msgr2=0x7f95c80710b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:43:23.798 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:23.797+0000 7f95e5e00700 1 --2- 172.21.15.18:0/1327251484 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f95c806ec00 0x7f95c80710b0 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f95cc00b3b0 tx=0x7f95cc009e30 comp rx=0 tx=0).stop 2024-04-12T21:43:23.798 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:23.797+0000 7f95e5e00700 1 -- 172.21.15.18:0/1327251484 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f95e0104e30 msgr2=0x7f95e0115ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:43:23.798 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:23.797+0000 7f95e5e00700 1 --2- 172.21.15.18:0/1327251484 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f95e0104e30 0x7f95e0115ba0 secure :-1 s=READY pgs=1111 cs=0 l=1 rev1=1 crypto rx=0x7f95d400b790 tx=0x7f95d4009250 comp rx=0 tx=0).stop 2024-04-12T21:43:23.798 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:23.797+0000 7f95e5e00700 1 -- 172.21.15.18:0/1327251484 shutdown_connections 2024-04-12T21:43:23.798 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:23.797+0000 7f95e5e00700 1 --2- 172.21.15.18:0/1327251484 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f95c806ec00 0x7f95c80710b0 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:43:23.798 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:23.797+0000 7f95e5e00700 1 --2- 172.21.15.18:0/1327251484 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f95e0104e30 0x7f95e0115ba0 unknown :-1 s=CLOSED pgs=1111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:43:23.798 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:23.797+0000 7f95e5e00700 1 --2- 172.21.15.18:0/1327251484 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f95e01057d0 0x7f95e01160e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:43:23.798 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:23.797+0000 7f95e5e00700 1 -- 172.21.15.18:0/1327251484 >> 172.21.15.18:0/1327251484 conn(0x7f95e00fab10 msgr2=0x7f95e00fc990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:43:23.798 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:23.798+0000 7f95e5e00700 1 -- 172.21.15.18:0/1327251484 shutdown_connections 2024-04-12T21:43:23.799 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:23.798+0000 7f95e5e00700 1 -- 172.21.15.18:0/1327251484 wait complete. 2024-04-12T21:43:23.935 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:43:23 smithi186 ceph-mon[86379]: pgmap v1311: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:43:23.936 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:43:23 smithi186 ceph-mon[86379]: from='client.36280 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:43:23.936 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:43:23 smithi186 ceph-mon[86379]: from='client.? 172.21.15.18:0/2434083329' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T21:43:23.936 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:43:23 smithi186 ceph-mon[86379]: from='client.? 172.21.15.18:0/357429793' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-12T21:43:23.955 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:23.955+0000 7f4215e00700 1 -- 172.21.15.18:0/840116402 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4210070c10 msgr2=0x7f4210109190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:43:23.955 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:23.955+0000 7f4215e00700 1 --2- 172.21.15.18:0/840116402 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4210070c10 0x7f4210109190 secure :-1 s=READY pgs=1112 cs=0 l=1 rev1=1 crypto rx=0x7f41fc004970 tx=0x7f41fc009e30 comp rx=0 tx=0).stop 2024-04-12T21:43:23.956 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:43:23 smithi018 ceph-mon[112590]: pgmap v1311: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:43:23.956 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:43:23 smithi018 ceph-mon[112590]: from='client.36280 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:43:23.956 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:43:23 smithi018 ceph-mon[112590]: from='client.? 172.21.15.18:0/2434083329' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T21:43:23.956 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:43:23 smithi018 ceph-mon[112590]: from='client.? 172.21.15.18:0/357429793' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-12T21:43:23.956 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:23.955+0000 7f4215e00700 1 -- 172.21.15.18:0/840116402 shutdown_connections 2024-04-12T21:43:23.957 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:23.955+0000 7f4215e00700 1 --2- 172.21.15.18:0/840116402 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4210070c10 0x7f4210109190 unknown :-1 s=CLOSED pgs=1112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:43:23.957 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:23.955+0000 7f4215e00700 1 --2- 172.21.15.18:0/840116402 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4210070300 0x7f42100706d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:43:23.957 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:23.955+0000 7f4215e00700 1 -- 172.21.15.18:0/840116402 >> 172.21.15.18:0/840116402 conn(0x7f42100f8970 msgr2=0x7f42100fad80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:43:23.957 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:23.955+0000 7f4215e00700 1 -- 172.21.15.18:0/840116402 shutdown_connections 2024-04-12T21:43:23.957 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:23.955+0000 7f4215e00700 1 -- 172.21.15.18:0/840116402 wait complete. 2024-04-12T21:43:23.957 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:23.956+0000 7f4215e00700 1 Processor -- start 2024-04-12T21:43:23.957 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:23.957+0000 7f4215e00700 1 -- start start 2024-04-12T21:43:23.958 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:23.957+0000 7f4215e00700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4210070300 0x7f4210115e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:43:23.958 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:23.957+0000 7f4215e00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4210070c10 0x7f4210116350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:43:23.958 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:23.957+0000 7f4215e00700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4210116a30 con 0x7f4210070c10 2024-04-12T21:43:23.958 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:23.957+0000 7f4215e00700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f421010fd70 con 0x7f4210070300 2024-04-12T21:43:23.959 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:23.958+0000 7f420e000700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4210070c10 0x7f4210116350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:43:23.959 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:23.958+0000 7f420e000700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4210070c10 0x7f4210116350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:37594/0 (socket says 172.21.15.18:37594) 2024-04-12T21:43:23.959 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:23.958+0000 7f420e000700 1 -- 172.21.15.18:0/602288049 learned_addr learned my addr 172.21.15.18:0/602288049 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:43:23.959 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:23.958+0000 7f420ea00700 1 --2- 172.21.15.18:0/602288049 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4210070300 0x7f4210115e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:43:23.959 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:23.958+0000 7f420e000700 1 -- 172.21.15.18:0/602288049 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4210070300 msgr2=0x7f4210115e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:43:23.959 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:23.958+0000 7f420e000700 1 --2- 172.21.15.18:0/602288049 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4210070300 0x7f4210115e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:43:23.959 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:23.958+0000 7f420e000700 1 -- 172.21.15.18:0/602288049 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41fc005040 con 0x7f4210070c10 2024-04-12T21:43:23.959 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:23.958+0000 7f420ea00700 1 --2- 172.21.15.18:0/602288049 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4210070300 0x7f4210115e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-12T21:43:23.960 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:23.959+0000 7f420e000700 1 --2- 172.21.15.18:0/602288049 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4210070c10 0x7f4210116350 secure :-1 s=READY pgs=1113 cs=0 l=1 rev1=1 crypto rx=0x7f41fc004f50 tx=0x7f41fc009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:43:23.960 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:23.959+0000 7f4206a00700 1 -- 172.21.15.18:0/602288049 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f41fc00e040 con 0x7f4210070c10 2024-04-12T21:43:23.960 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:23.959+0000 7f4206a00700 1 -- 172.21.15.18:0/602288049 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f41fc00bc10 con 0x7f4210070c10 2024-04-12T21:43:23.960 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:23.959+0000 7f4215e00700 1 -- 172.21.15.18:0/602288049 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f421010ff90 con 0x7f4210070c10 2024-04-12T21:43:23.960 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:23.959+0000 7f4206a00700 1 -- 172.21.15.18:0/602288049 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f41fc01c630 con 0x7f4210070c10 2024-04-12T21:43:23.960 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:23.959+0000 7f4215e00700 1 -- 172.21.15.18:0/602288049 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f42101104d0 con 0x7f4210070c10 2024-04-12T21:43:23.961 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:23.960+0000 7f4215e00700 1 -- 172.21.15.18:0/602288049 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4210071a40 con 0x7f4210070c10 2024-04-12T21:43:23.961 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:23.961+0000 7f4206a00700 1 -- 172.21.15.18:0/602288049 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7f41fc013420 con 0x7f4210070c10 2024-04-12T21:43:23.963 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:23.963+0000 7f4206a00700 1 --2- 172.21.15.18:0/602288049 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f41f406ec10 0x7f41f40710c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:43:23.963 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:23.963+0000 7f4206a00700 1 -- 172.21.15.18:0/602288049 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 6757+0+0 (secure 0 0 0) 0x7f41fc08fc10 con 0x7f4210070c10 2024-04-12T21:43:23.963 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:23.963+0000 7f420ea00700 1 --2- 172.21.15.18:0/602288049 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f41f406ec10 0x7f41f40710c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:43:23.964 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:23.964+0000 7f420ea00700 1 --2- 172.21.15.18:0/602288049 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f41f406ec10 0x7f41f40710c0 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f4200006fd0 tx=0x7f4200008040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:43:23.968 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:23.968+0000 7f4206a00700 1 -- 172.21.15.18:0/602288049 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f41fc05c460 con 0x7f4210070c10 2024-04-12T21:43:24.292 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:24.291+0000 7f4215e00700 1 -- 172.21.15.18:0/602288049 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f421005f5f0 con 0x7f4210070c10 2024-04-12T21:43:24.292 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:24.292+0000 7f4206a00700 1 -- 172.21.15.18:0/602288049 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+4303 (secure 0 0 0) 0x7f41fc05bff0 con 0x7f4210070c10 2024-04-12T21:43:24.293 INFO:teuthology.orchestra.run.smithi018.stdout:HEALTH_ERR failed to probe daemons or devices; 1 filesystem is degraded; 1 filesystem is offline 2024-04-12T21:43:24.293 INFO:teuthology.orchestra.run.smithi018.stdout:[WRN] CEPHADM_REFRESH_FAILED: failed to probe daemons or devices 2024-04-12T21:43:24.293 INFO:teuthology.orchestra.run.smithi018.stdout: host smithi186 `cephadm ceph-volume` failed: cephadm exited with an error code: 1, stderr: Inferring config /var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/mon.smithi186/config 2024-04-12T21:43:24.293 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/podman: timeout after 895 seconds 2024-04-12T21:43:24.293 INFO:teuthology.orchestra.run.smithi018.stdout:Non-zero exit code 124 from /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:f4452def3be4d350f4a8e5feaf7ebb1511be9298fe68478ba7a2d6fbd7d69a63 -e NODE_NAME=smithi186 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968:/var/run/ceph:z -v /var/log/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968:/var/log/ceph:z -v /var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmptgom53f9:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:f4452def3be4d350f4a8e5feaf7ebb1511be9298fe68478ba7a2d6fbd7d69a63 inventory --format=json-pretty --filter-for-batch 2024-04-12T21:43:24.293 INFO:teuthology.orchestra.run.smithi018.stdout:Traceback (most recent call last): 2024-04-12T21:43:24.294 INFO:teuthology.orchestra.run.smithi018.stdout: File "/usr/lib64/python3.9/runpy.py", line 197, in _run_module_as_main 2024-04-12T21:43:24.294 INFO:teuthology.orchestra.run.smithi018.stdout: return _run_code(code, main_globals, None, 2024-04-12T21:43:24.294 INFO:teuthology.orchestra.run.smithi018.stdout: File "/usr/lib64/python3.9/runpy.py", line 87, in _run_code 2024-04-12T21:43:24.294 INFO:teuthology.orchestra.run.smithi018.stdout: exec(code, run_globals) 2024-04-12T21:43:24.294 INFO:teuthology.orchestra.run.smithi018.stdout: File "/var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/cephadm.e44f3ed5e34e4ef52eedef233d1190bff004470d8e8005127b5312bff357fa7f/__main__.py", line 5618, in 2024-04-12T21:43:24.294 INFO:teuthology.orchestra.run.smithi018.stdout: File "/var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/cephadm.e44f3ed5e34e4ef52eedef233d1190bff004470d8e8005127b5312bff357fa7f/__main__.py", line 5606, in main 2024-04-12T21:43:24.294 INFO:teuthology.orchestra.run.smithi018.stdout: File "/var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/cephadm.e44f3ed5e34e4ef52eedef233d1190bff004470d8e8005127b5312bff357fa7f/__main__.py", line 408, in _infer_config 2024-04-12T21:43:24.294 INFO:teuthology.orchestra.run.smithi018.stdout: File "/var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/cephadm.e44f3ed5e34e4ef52eedef233d1190bff004470d8e8005127b5312bff357fa7f/__main__.py", line 323, in _infer_fsid 2024-04-12T21:43:24.294 INFO:teuthology.orchestra.run.smithi018.stdout: File "/var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/cephadm.e44f3ed5e34e4ef52eedef233d1190bff004470d8e8005127b5312bff357fa7f/__main__.py", line 436, in _infer_image 2024-04-12T21:43:24.294 INFO:teuthology.orchestra.run.smithi018.stdout: File "/var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/cephadm.e44f3ed5e34e4ef52eedef233d1190bff004470d8e8005127b5312bff357fa7f/__main__.py", line 310, in _validate_fsid 2024-04-12T21:43:24.294 INFO:teuthology.orchestra.run.smithi018.stdout: File "/var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/cephadm.e44f3ed5e34e4ef52eedef233d1190bff004470d8e8005127b5312bff357fa7f/__main__.py", line 3359, in command_ceph_volume 2024-04-12T21:43:24.294 INFO:teuthology.orchestra.run.smithi018.stdout: File "/var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/cephadm.e44f3ed5e34e4ef52eedef233d1190bff004470d8e8005127b5312bff357fa7f/cephadmlib/call_wrappers.py", line 310, in call_throws 2024-04-12T21:43:24.294 INFO:teuthology.orchestra.run.smithi018.stdout:RuntimeError: Failed command: /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:f4452def3be4d350f4a8e5feaf7ebb1511be9298fe68478ba7a2d6fbd7d69a63 -e NODE_NAME=smithi186 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968:/var/run/ceph:z -v /var/log/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968:/var/log/ceph:z -v /var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmptgom53f9:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:f4452def3be4d350f4a8e5feaf7ebb1511be9298fe68478ba7a2d6fbd7d69a63 inventory --format=json-pretty --filter-for-batch 2024-04-12T21:43:24.295 INFO:teuthology.orchestra.run.smithi018.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-04-12T21:43:24.295 INFO:teuthology.orchestra.run.smithi018.stdout: fs cephfs is degraded 2024-04-12T21:43:24.295 INFO:teuthology.orchestra.run.smithi018.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-04-12T21:43:24.295 INFO:teuthology.orchestra.run.smithi018.stdout: fs cephfs is offline because no MDS is active for it. 2024-04-12T21:43:24.296 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:24.296+0000 7f4215e00700 1 -- 172.21.15.18:0/602288049 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f41f406ec10 msgr2=0x7f41f40710c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:43:24.296 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:24.296+0000 7f4215e00700 1 --2- 172.21.15.18:0/602288049 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f41f406ec10 0x7f41f40710c0 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f4200006fd0 tx=0x7f4200008040 comp rx=0 tx=0).stop 2024-04-12T21:43:24.296 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:24.296+0000 7f4215e00700 1 -- 172.21.15.18:0/602288049 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4210070c10 msgr2=0x7f4210116350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:43:24.296 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:24.296+0000 7f4215e00700 1 --2- 172.21.15.18:0/602288049 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4210070c10 0x7f4210116350 secure :-1 s=READY pgs=1113 cs=0 l=1 rev1=1 crypto rx=0x7f41fc004f50 tx=0x7f41fc009e30 comp rx=0 tx=0).stop 2024-04-12T21:43:24.297 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:24.296+0000 7f4215e00700 1 -- 172.21.15.18:0/602288049 shutdown_connections 2024-04-12T21:43:24.297 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:24.296+0000 7f4215e00700 1 --2- 172.21.15.18:0/602288049 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f41f406ec10 0x7f41f40710c0 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:43:24.297 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:24.296+0000 7f4215e00700 1 --2- 172.21.15.18:0/602288049 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4210070300 0x7f4210115e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:43:24.297 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:24.296+0000 7f4215e00700 1 --2- 172.21.15.18:0/602288049 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4210070c10 0x7f4210116350 unknown :-1 s=CLOSED pgs=1113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:43:24.297 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:24.296+0000 7f4215e00700 1 -- 172.21.15.18:0/602288049 >> 172.21.15.18:0/602288049 conn(0x7f42100f8970 msgr2=0x7f42100f9d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:43:24.297 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:24.296+0000 7f4215e00700 1 -- 172.21.15.18:0/602288049 shutdown_connections 2024-04-12T21:43:24.297 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:24.297+0000 7f4215e00700 1 -- 172.21.15.18:0/602288049 wait complete. 2024-04-12T21:43:25.034 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:43:24 smithi186 ceph-mon[86379]: from='client.36292 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:43:25.034 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:43:24 smithi186 ceph-mon[86379]: from='client.? 172.21.15.18:0/602288049' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-12T21:43:25.104 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:43:24 smithi018 ceph-mon[112590]: from='client.36292 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:43:25.105 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:43:24 smithi018 ceph-mon[112590]: from='client.? 172.21.15.18:0/602288049' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-12T21:43:26.034 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:43:25 smithi186 ceph-mon[86379]: pgmap v1312: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:43:26.034 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:43:25 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T21:43:26.105 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:43:25 smithi018 ceph-mon[112590]: pgmap v1312: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:43:26.105 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:43:25 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T21:43:28.033 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:43:27 smithi186 ceph-mon[86379]: pgmap v1313: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:43:28.104 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:43:27 smithi018 ceph-mon[112590]: pgmap v1313: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:43:30.033 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:43:29 smithi186 ceph-mon[86379]: pgmap v1314: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:43:30.104 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:43:29 smithi018 ceph-mon[112590]: pgmap v1314: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:43:32.033 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:43:31 smithi186 ceph-mon[86379]: pgmap v1315: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:43:32.104 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:43:31 smithi018 ceph-mon[112590]: pgmap v1315: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:43:34.033 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:43:33 smithi186 ceph-mon[86379]: pgmap v1316: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:43:34.104 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:43:33 smithi018 ceph-mon[112590]: pgmap v1316: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:43:36.034 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:43:35 smithi186 ceph-mon[86379]: pgmap v1317: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:43:36.104 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:43:35 smithi018 ceph-mon[112590]: pgmap v1317: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:43:38.034 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:43:37 smithi186 ceph-mon[86379]: pgmap v1318: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:43:38.104 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:43:37 smithi018 ceph-mon[112590]: pgmap v1318: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:43:40.033 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:43:39 smithi186 ceph-mon[86379]: pgmap v1319: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:43:40.105 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:43:39 smithi018 ceph-mon[112590]: pgmap v1319: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:43:41.034 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:43:40 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T21:43:41.105 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:43:40 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T21:43:42.034 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:43:41 smithi186 ceph-mon[86379]: pgmap v1320: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:43:42.104 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:43:41 smithi018 ceph-mon[112590]: pgmap v1320: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:43:44.034 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:43:43 smithi186 ceph-mon[86379]: pgmap v1321: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:43:44.105 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:43:43 smithi018 ceph-mon[112590]: pgmap v1321: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:43:46.033 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:43:45 smithi186 ceph-mon[86379]: pgmap v1322: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:43:46.105 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:43:45 smithi018 ceph-mon[112590]: pgmap v1322: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:43:48.034 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:43:47 smithi186 ceph-mon[86379]: pgmap v1323: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:43:48.104 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:43:47 smithi018 ceph-mon[112590]: pgmap v1323: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:43:50.034 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:43:49 smithi186 ceph-mon[86379]: pgmap v1324: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:43:50.104 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:43:49 smithi018 ceph-mon[112590]: pgmap v1324: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:43:52.034 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:43:51 smithi186 ceph-mon[86379]: pgmap v1325: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:43:52.105 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:43:51 smithi018 ceph-mon[112590]: pgmap v1325: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:43:54.033 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:43:53 smithi186 ceph-mon[86379]: pgmap v1326: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:43:54.104 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:43:53 smithi018 ceph-mon[112590]: pgmap v1326: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:43:54.469 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:54.469+0000 7f1126e00700 1 -- 172.21.15.18:0/4125728971 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1120071f90 msgr2=0x7f1120070360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:43:54.470 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:54.469+0000 7f1126e00700 1 --2- 172.21.15.18:0/4125728971 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1120071f90 0x7f1120070360 secure :-1 s=READY pgs=1114 cs=0 l=1 rev1=1 crypto rx=0x7f1114004970 tx=0x7f1114009e30 comp rx=0 tx=0).stop 2024-04-12T21:43:54.470 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:54.469+0000 7f1126e00700 1 -- 172.21.15.18:0/4125728971 shutdown_connections 2024-04-12T21:43:54.470 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:54.469+0000 7f1126e00700 1 --2- 172.21.15.18:0/4125728971 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f11200708a0 0x7f1120070d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:43:54.470 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:54.469+0000 7f1126e00700 1 --2- 172.21.15.18:0/4125728971 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1120071f90 0x7f1120070360 unknown :-1 s=CLOSED pgs=1114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:43:54.470 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:54.469+0000 7f1126e00700 1 -- 172.21.15.18:0/4125728971 >> 172.21.15.18:0/4125728971 conn(0x7f11200f87a0 msgr2=0x7f11200fabb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:43:54.470 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:54.469+0000 7f1126e00700 1 -- 172.21.15.18:0/4125728971 shutdown_connections 2024-04-12T21:43:54.470 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:54.469+0000 7f1126e00700 1 -- 172.21.15.18:0/4125728971 wait complete. 2024-04-12T21:43:54.471 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:54.470+0000 7f1126e00700 1 Processor -- start 2024-04-12T21:43:54.472 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:54.470+0000 7f1126e00700 1 -- start start 2024-04-12T21:43:54.472 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:54.471+0000 7f1126e00700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f11200708a0 0x7f1120119fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:43:54.472 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:54.471+0000 7f1126e00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1120071f90 0x7f112011a4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:43:54.472 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:54.471+0000 7f1126e00700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f112011ab40 con 0x7f1120071f90 2024-04-12T21:43:54.472 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:54.471+0000 7f1126e00700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1120113f10 con 0x7f11200708a0 2024-04-12T21:43:54.472 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:54.471+0000 7f1125000700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1120071f90 0x7f112011a4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:43:54.472 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:54.471+0000 7f1125000700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1120071f90 0x7f112011a4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:55280/0 (socket says 172.21.15.18:55280) 2024-04-12T21:43:54.472 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:54.471+0000 7f1125000700 1 -- 172.21.15.18:0/709919473 learned_addr learned my addr 172.21.15.18:0/709919473 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:43:54.472 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:54.471+0000 7f1125a00700 1 --2- 172.21.15.18:0/709919473 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f11200708a0 0x7f1120119fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:43:54.473 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:54.472+0000 7f1125000700 1 -- 172.21.15.18:0/709919473 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f11200708a0 msgr2=0x7f1120119fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:43:54.473 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:54.472+0000 7f1125000700 1 --2- 172.21.15.18:0/709919473 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f11200708a0 0x7f1120119fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:43:54.473 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:54.472+0000 7f1125000700 1 -- 172.21.15.18:0/709919473 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1114005040 con 0x7f1120071f90 2024-04-12T21:43:54.473 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:54.472+0000 7f1125000700 1 --2- 172.21.15.18:0/709919473 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1120071f90 0x7f112011a4f0 secure :-1 s=READY pgs=1115 cs=0 l=1 rev1=1 crypto rx=0x7f110c002a50 tx=0x7f110c0046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:43:54.473 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:54.472+0000 7f111d600700 1 -- 172.21.15.18:0/709919473 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f110c007cc0 con 0x7f1120071f90 2024-04-12T21:43:54.473 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:54.472+0000 7f111d600700 1 -- 172.21.15.18:0/709919473 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f110c00f040 con 0x7f1120071f90 2024-04-12T21:43:54.473 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:54.472+0000 7f111d600700 1 -- 172.21.15.18:0/709919473 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f110c00da30 con 0x7f1120071f90 2024-04-12T21:43:54.474 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:54.472+0000 7f1126e00700 1 -- 172.21.15.18:0/709919473 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f11201141f0 con 0x7f1120071f90 2024-04-12T21:43:54.474 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:54.473+0000 7f1126e00700 1 -- 172.21.15.18:0/709919473 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1120114730 con 0x7f1120071f90 2024-04-12T21:43:54.474 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:54.474+0000 7f1126e00700 1 -- 172.21.15.18:0/709919473 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1120071aa0 con 0x7f1120071f90 2024-04-12T21:43:54.475 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:54.475+0000 7f111d600700 1 -- 172.21.15.18:0/709919473 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7f110c00db90 con 0x7f1120071f90 2024-04-12T21:43:54.476 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:54.476+0000 7f111d600700 1 --2- 172.21.15.18:0/709919473 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f110806ec10 0x7f11080710c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:43:54.477 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:54.476+0000 7f111d600700 1 -- 172.21.15.18:0/709919473 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 6757+0+0 (secure 0 0 0) 0x7f110c08e700 con 0x7f1120071f90 2024-04-12T21:43:54.477 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:54.477+0000 7f1125a00700 1 --2- 172.21.15.18:0/709919473 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f110806ec10 0x7f11080710c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:43:54.477 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:54.477+0000 7f1125a00700 1 --2- 172.21.15.18:0/709919473 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f110806ec10 0x7f11080710c0 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f1114009fd0 tx=0x7f1114009e30 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:43:54.481 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:54.481+0000 7f111d600700 1 -- 172.21.15.18:0/709919473 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f110c05aeb0 con 0x7f1120071f90 2024-04-12T21:43:54.729 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:54.729+0000 7f1126e00700 1 -- 172.21.15.18:0/709919473 --> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f11201103c0 con 0x7f110806ec10 2024-04-12T21:43:54.731 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:54.731+0000 7f111d600700 1 -- 172.21.15.18:0/709919473 <== mgr.34102 v2:172.21.15.18:6800/3812891919 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f11201103c0 con 0x7f110806ec10 2024-04-12T21:43:54.734 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:54.734+0000 7f1126e00700 1 -- 172.21.15.18:0/709919473 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f110806ec10 msgr2=0x7f11080710c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:43:54.734 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:54.734+0000 7f1126e00700 1 --2- 172.21.15.18:0/709919473 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f110806ec10 0x7f11080710c0 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f1114009fd0 tx=0x7f1114009e30 comp rx=0 tx=0).stop 2024-04-12T21:43:54.734 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:54.734+0000 7f1126e00700 1 -- 172.21.15.18:0/709919473 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1120071f90 msgr2=0x7f112011a4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:43:54.735 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:54.734+0000 7f1126e00700 1 --2- 172.21.15.18:0/709919473 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1120071f90 0x7f112011a4f0 secure :-1 s=READY pgs=1115 cs=0 l=1 rev1=1 crypto rx=0x7f110c002a50 tx=0x7f110c0046b0 comp rx=0 tx=0).stop 2024-04-12T21:43:54.735 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:54.734+0000 7f1126e00700 1 -- 172.21.15.18:0/709919473 shutdown_connections 2024-04-12T21:43:54.735 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:54.734+0000 7f1126e00700 1 --2- 172.21.15.18:0/709919473 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f110806ec10 0x7f11080710c0 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:43:54.735 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:54.734+0000 7f1126e00700 1 --2- 172.21.15.18:0/709919473 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f11200708a0 0x7f1120119fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:43:54.735 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:54.734+0000 7f1126e00700 1 --2- 172.21.15.18:0/709919473 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1120071f90 0x7f112011a4f0 unknown :-1 s=CLOSED pgs=1115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:43:54.735 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:54.734+0000 7f1126e00700 1 -- 172.21.15.18:0/709919473 >> 172.21.15.18:0/709919473 conn(0x7f11200f87a0 msgr2=0x7f11200f9890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:43:54.735 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:54.734+0000 7f1126e00700 1 -- 172.21.15.18:0/709919473 shutdown_connections 2024-04-12T21:43:54.736 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:54.734+0000 7f1126e00700 1 -- 172.21.15.18:0/709919473 wait complete. 2024-04-12T21:43:54.750 INFO:teuthology.orchestra.run.smithi018.stdout:true 2024-04-12T21:43:54.896 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:54.895+0000 7f521aa00700 1 -- 172.21.15.18:0/181540870 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f52140708a0 msgr2=0x7f5214070d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:43:54.896 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:54.895+0000 7f521aa00700 1 --2- 172.21.15.18:0/181540870 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f52140708a0 0x7f5214070d10 secure :-1 s=READY pgs=1116 cs=0 l=1 rev1=1 crypto rx=0x7f51fc004970 tx=0x7f51fc009e30 comp rx=0 tx=0).stop 2024-04-12T21:43:54.896 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:54.895+0000 7f521aa00700 1 -- 172.21.15.18:0/181540870 shutdown_connections 2024-04-12T21:43:54.896 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:54.895+0000 7f521aa00700 1 --2- 172.21.15.18:0/181540870 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f52140708a0 0x7f5214070d10 unknown :-1 s=CLOSED pgs=1116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:43:54.896 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:54.895+0000 7f521aa00700 1 --2- 172.21.15.18:0/181540870 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5214071f90 0x7f5214070360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:43:54.896 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:54.895+0000 7f521aa00700 1 -- 172.21.15.18:0/181540870 >> 172.21.15.18:0/181540870 conn(0x7f52140f8970 msgr2=0x7f52140fad80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:43:54.896 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:54.895+0000 7f521aa00700 1 -- 172.21.15.18:0/181540870 shutdown_connections 2024-04-12T21:43:54.896 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:54.895+0000 7f521aa00700 1 -- 172.21.15.18:0/181540870 wait complete. 2024-04-12T21:43:54.897 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:54.896+0000 7f521aa00700 1 Processor -- start 2024-04-12T21:43:54.897 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:54.896+0000 7f521aa00700 1 -- start start 2024-04-12T21:43:54.897 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:54.897+0000 7f521aa00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f52140708a0 0x7f521411a170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:43:54.897 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:54.897+0000 7f521aa00700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5214071f90 0x7f521411a6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:43:54.898 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:54.897+0000 7f521aa00700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f521411ad00 con 0x7f52140708a0 2024-04-12T21:43:54.898 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:54.897+0000 7f521aa00700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52141140d0 con 0x7f5214071f90 2024-04-12T21:43:54.898 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:54.897+0000 7f5213e00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f52140708a0 0x7f521411a170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:43:54.898 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:54.897+0000 7f5213e00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f52140708a0 0x7f521411a170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:55308/0 (socket says 172.21.15.18:55308) 2024-04-12T21:43:54.898 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:54.897+0000 7f5213e00700 1 -- 172.21.15.18:0/25476501 learned_addr learned my addr 172.21.15.18:0/25476501 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:43:54.899 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:54.897+0000 7f520be00700 1 --2- 172.21.15.18:0/25476501 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5214071f90 0x7f521411a6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:43:54.899 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:54.898+0000 7f5213e00700 1 -- 172.21.15.18:0/25476501 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5214071f90 msgr2=0x7f521411a6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:43:54.899 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:54.898+0000 7f5213e00700 1 --2- 172.21.15.18:0/25476501 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5214071f90 0x7f521411a6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:43:54.899 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:54.898+0000 7f5213e00700 1 -- 172.21.15.18:0/25476501 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51fc005040 con 0x7f52140708a0 2024-04-12T21:43:54.899 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:54.898+0000 7f520be00700 1 --2- 172.21.15.18:0/25476501 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5214071f90 0x7f521411a6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-12T21:43:54.899 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:54.898+0000 7f5213e00700 1 --2- 172.21.15.18:0/25476501 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f52140708a0 0x7f521411a170 secure :-1 s=READY pgs=1117 cs=0 l=1 rev1=1 crypto rx=0x7f5204002a50 tx=0x7f52040046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:43:54.899 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:54.898+0000 7f5210c00700 1 -- 172.21.15.18:0/25476501 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5204007cc0 con 0x7f52140708a0 2024-04-12T21:43:54.900 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:54.898+0000 7f5210c00700 1 -- 172.21.15.18:0/25476501 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f520400f040 con 0x7f52140708a0 2024-04-12T21:43:54.900 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:54.898+0000 7f5210c00700 1 -- 172.21.15.18:0/25476501 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5204009640 con 0x7f52140708a0 2024-04-12T21:43:54.900 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:54.898+0000 7f521aa00700 1 -- 172.21.15.18:0/25476501 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f52141143b0 con 0x7f52140708a0 2024-04-12T21:43:54.900 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:54.898+0000 7f521aa00700 1 -- 172.21.15.18:0/25476501 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f52141148f0 con 0x7f52140708a0 2024-04-12T21:43:54.900 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:54.900+0000 7f521aa00700 1 -- 172.21.15.18:0/25476501 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5214071aa0 con 0x7f52140708a0 2024-04-12T21:43:54.901 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:54.901+0000 7f5210c00700 1 -- 172.21.15.18:0/25476501 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7f52040097a0 con 0x7f52140708a0 2024-04-12T21:43:54.902 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:54.902+0000 7f5210c00700 1 --2- 172.21.15.18:0/25476501 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f51f806ec10 0x7f51f80710c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:43:54.902 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:54.902+0000 7f5210c00700 1 -- 172.21.15.18:0/25476501 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 6757+0+0 (secure 0 0 0) 0x7f520408e760 con 0x7f52140708a0 2024-04-12T21:43:54.902 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:54.902+0000 7f520be00700 1 --2- 172.21.15.18:0/25476501 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f51f806ec10 0x7f51f80710c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:43:54.903 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:54.903+0000 7f520be00700 1 --2- 172.21.15.18:0/25476501 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f51f806ec10 0x7f51f80710c0 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f51fc00b9c0 tx=0x7f51fc009e30 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:43:54.907 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:54.907+0000 7f5210c00700 1 -- 172.21.15.18:0/25476501 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f520405af10 con 0x7f52140708a0 2024-04-12T21:43:55.154 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:55.153+0000 7f521aa00700 1 -- 172.21.15.18:0/25476501 --> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f521404bf40 con 0x7f51f806ec10 2024-04-12T21:43:55.156 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:55.156+0000 7f5210c00700 1 -- 172.21.15.18:0/25476501 <== mgr.34102 v2:172.21.15.18:6800/3812891919 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f521404bf40 con 0x7f51f806ec10 2024-04-12T21:43:55.159 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:55.159+0000 7f521aa00700 1 -- 172.21.15.18:0/25476501 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f51f806ec10 msgr2=0x7f51f80710c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:43:55.159 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:55.159+0000 7f521aa00700 1 --2- 172.21.15.18:0/25476501 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f51f806ec10 0x7f51f80710c0 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f51fc00b9c0 tx=0x7f51fc009e30 comp rx=0 tx=0).stop 2024-04-12T21:43:55.159 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:55.159+0000 7f521aa00700 1 -- 172.21.15.18:0/25476501 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f52140708a0 msgr2=0x7f521411a170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:43:55.159 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:55.159+0000 7f521aa00700 1 --2- 172.21.15.18:0/25476501 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f52140708a0 0x7f521411a170 secure :-1 s=READY pgs=1117 cs=0 l=1 rev1=1 crypto rx=0x7f5204002a50 tx=0x7f52040046b0 comp rx=0 tx=0).stop 2024-04-12T21:43:55.160 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:55.159+0000 7f521aa00700 1 -- 172.21.15.18:0/25476501 shutdown_connections 2024-04-12T21:43:55.160 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:55.159+0000 7f521aa00700 1 --2- 172.21.15.18:0/25476501 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f51f806ec10 0x7f51f80710c0 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:43:55.160 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:55.159+0000 7f521aa00700 1 --2- 172.21.15.18:0/25476501 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f52140708a0 0x7f521411a170 unknown :-1 s=CLOSED pgs=1117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:43:55.160 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:55.159+0000 7f521aa00700 1 --2- 172.21.15.18:0/25476501 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5214071f90 0x7f521411a6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:43:55.160 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:55.159+0000 7f521aa00700 1 -- 172.21.15.18:0/25476501 >> 172.21.15.18:0/25476501 conn(0x7f52140f8970 msgr2=0x7f52140f9990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:43:55.160 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:55.160+0000 7f521aa00700 1 -- 172.21.15.18:0/25476501 shutdown_connections 2024-04-12T21:43:55.161 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:55.160+0000 7f521aa00700 1 -- 172.21.15.18:0/25476501 wait complete. 2024-04-12T21:43:55.321 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:55.321+0000 7f0e31c00700 1 -- 172.21.15.18:0/2710366464 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0e2c0708a0 msgr2=0x7f0e2c070d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:43:55.322 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:55.321+0000 7f0e31c00700 1 --2- 172.21.15.18:0/2710366464 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0e2c0708a0 0x7f0e2c070d10 secure :-1 s=READY pgs=1118 cs=0 l=1 rev1=1 crypto rx=0x7f0e18004970 tx=0x7f0e18009e30 comp rx=0 tx=0).stop 2024-04-12T21:43:55.322 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:55.321+0000 7f0e31c00700 1 -- 172.21.15.18:0/2710366464 shutdown_connections 2024-04-12T21:43:55.322 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:55.321+0000 7f0e31c00700 1 --2- 172.21.15.18:0/2710366464 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0e2c0708a0 0x7f0e2c070d10 unknown :-1 s=CLOSED pgs=1118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:43:55.322 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:55.321+0000 7f0e31c00700 1 --2- 172.21.15.18:0/2710366464 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0e2c071f90 0x7f0e2c070360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:43:55.322 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:55.321+0000 7f0e31c00700 1 -- 172.21.15.18:0/2710366464 >> 172.21.15.18:0/2710366464 conn(0x7f0e2c0f8830 msgr2=0x7f0e2c0fac40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:43:55.322 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:55.321+0000 7f0e31c00700 1 -- 172.21.15.18:0/2710366464 shutdown_connections 2024-04-12T21:43:55.322 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:55.321+0000 7f0e31c00700 1 -- 172.21.15.18:0/2710366464 wait complete. 2024-04-12T21:43:55.323 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:55.322+0000 7f0e31c00700 1 Processor -- start 2024-04-12T21:43:55.323 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:55.322+0000 7f0e31c00700 1 -- start start 2024-04-12T21:43:55.323 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:55.322+0000 7f0e31c00700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0e2c0708a0 0x7f0e2c119fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:43:55.323 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:55.322+0000 7f0e31c00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0e2c071f90 0x7f0e2c11a510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:43:55.324 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:55.322+0000 7f0e31c00700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e2c11ab60 con 0x7f0e2c071f90 2024-04-12T21:43:55.324 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:55.322+0000 7f0e31c00700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e2c113f30 con 0x7f0e2c0708a0 2024-04-12T21:43:55.324 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:55.323+0000 7f0e2b400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0e2c071f90 0x7f0e2c11a510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:43:55.324 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:55.323+0000 7f0e2b400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0e2c071f90 0x7f0e2c11a510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:55332/0 (socket says 172.21.15.18:55332) 2024-04-12T21:43:55.324 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:55.323+0000 7f0e2b400700 1 -- 172.21.15.18:0/155042374 learned_addr learned my addr 172.21.15.18:0/155042374 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:43:55.324 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:55.323+0000 7f0e2be00700 1 --2- 172.21.15.18:0/155042374 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0e2c0708a0 0x7f0e2c119fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:43:55.324 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:55.323+0000 7f0e2b400700 1 -- 172.21.15.18:0/155042374 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0e2c0708a0 msgr2=0x7f0e2c119fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:43:55.324 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:55.323+0000 7f0e2b400700 1 --2- 172.21.15.18:0/155042374 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0e2c0708a0 0x7f0e2c119fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:43:55.324 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:55.323+0000 7f0e2b400700 1 -- 172.21.15.18:0/155042374 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e18005040 con 0x7f0e2c071f90 2024-04-12T21:43:55.325 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:55.324+0000 7f0e2b400700 1 --2- 172.21.15.18:0/155042374 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0e2c071f90 0x7f0e2c11a510 secure :-1 s=READY pgs=1119 cs=0 l=1 rev1=1 crypto rx=0x7f0e18004f50 tx=0x7f0e18009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:43:55.325 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:55.324+0000 7f0e23e00700 1 -- 172.21.15.18:0/155042374 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0e1800e040 con 0x7f0e2c071f90 2024-04-12T21:43:55.325 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:55.324+0000 7f0e23e00700 1 -- 172.21.15.18:0/155042374 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f0e1800bc10 con 0x7f0e2c071f90 2024-04-12T21:43:55.325 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:55.324+0000 7f0e23e00700 1 -- 172.21.15.18:0/155042374 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0e1801c630 con 0x7f0e2c071f90 2024-04-12T21:43:55.325 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:55.324+0000 7f0e31c00700 1 -- 172.21.15.18:0/155042374 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0e2c1141b0 con 0x7f0e2c071f90 2024-04-12T21:43:55.325 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:55.324+0000 7f0e31c00700 1 -- 172.21.15.18:0/155042374 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0e2c1146f0 con 0x7f0e2c071f90 2024-04-12T21:43:55.326 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:55.325+0000 7f0e31c00700 1 -- 172.21.15.18:0/155042374 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0e2c071aa0 con 0x7f0e2c071f90 2024-04-12T21:43:55.326 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:55.326+0000 7f0e23e00700 1 -- 172.21.15.18:0/155042374 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7f0e18013420 con 0x7f0e2c071f90 2024-04-12T21:43:55.328 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:55.327+0000 7f0e23e00700 1 --2- 172.21.15.18:0/155042374 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f0e1006ec10 0x7f0e100710c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:43:55.328 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:55.327+0000 7f0e23e00700 1 -- 172.21.15.18:0/155042374 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 6757+0+0 (secure 0 0 0) 0x7f0e1808fce0 con 0x7f0e2c071f90 2024-04-12T21:43:55.328 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:55.328+0000 7f0e2be00700 1 --2- 172.21.15.18:0/155042374 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f0e1006ec10 0x7f0e100710c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:43:55.328 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:55.328+0000 7f0e2be00700 1 --2- 172.21.15.18:0/155042374 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f0e1006ec10 0x7f0e100710c0 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f0e1c004990 tx=0x7f0e1c009ee0 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:43:55.332 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:55.332+0000 7f0e23e00700 1 -- 172.21.15.18:0/155042374 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f0e1805c510 con 0x7f0e2c071f90 2024-04-12T21:43:55.576 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:55.575+0000 7f0e31c00700 1 -- 172.21.15.18:0/155042374 --> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f0e2c1103d0 con 0x7f0e1006ec10 2024-04-12T21:43:55.612 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:55.612+0000 7f0e23e00700 1 -- 172.21.15.18:0/155042374 <== mgr.34102 v2:172.21.15.18:6800/3812891919 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7f0e2c1103d0 con 0x7f0e1006ec10 2024-04-12T21:43:55.613 INFO:teuthology.orchestra.run.smithi018.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-12T21:43:55.613 INFO:teuthology.orchestra.run.smithi018.stdout:alertmanager.smithi018 smithi018 *:9093,9094 running (44m) 5m ago 58m 14.8M - 0.25.0 c8568f914cd2 c332090068e8 2024-04-12T21:43:55.613 INFO:teuthology.orchestra.run.smithi018.stdout:crash.smithi018 smithi018 running (40m) 5m ago 59m 6665k - 19.0.0-2879-g324baf8d 8717720dc5e5 b7f80d1f7982 2024-04-12T21:43:55.613 INFO:teuthology.orchestra.run.smithi018.stdout:crash.smithi186 smithi186 running (40m) 5m ago 57m 6660k - 19.0.0-2879-g324baf8d 8717720dc5e5 a4f55c81b623 2024-04-12T21:43:55.613 INFO:teuthology.orchestra.run.smithi018.stdout:grafana.smithi018 smithi018 *:3000 running (43m) 5m ago 58m 73.8M - 9.4.12 2bacad6d85d8 b08e3bf176dd 2024-04-12T21:43:55.613 INFO:teuthology.orchestra.run.smithi018.stdout:mds.cephfs.smithi018.acqbsc smithi018 running (36m) 5m ago 51m 21.9M - 19.0.0-2879-g324baf8d 8717720dc5e5 021f99ab24ae 2024-04-12T21:43:55.613 INFO:teuthology.orchestra.run.smithi018.stdout:mds.cephfs.smithi018.szjluo smithi018 running (36m) 5m ago 51m 21.7M - 19.0.0-2879-g324baf8d 8717720dc5e5 0003117479d4 2024-04-12T21:43:55.613 INFO:teuthology.orchestra.run.smithi018.stdout:mds.cephfs.smithi186.jdllyn smithi186 running (20m) 5m ago 51m 16.5M - 19.0.0-2879-g324baf8d 8717720dc5e5 21b1f46eef23 2024-04-12T21:43:55.613 INFO:teuthology.orchestra.run.smithi018.stdout:mds.cephfs.smithi186.wjnafs smithi186 running (5m) 5m ago 51m 11.0M - 19.0.0-2879-g324baf8d 8717720dc5e5 3cf1ef4148e2 2024-04-12T21:43:55.613 INFO:teuthology.orchestra.run.smithi018.stdout:mgr.smithi018.xjzwga smithi018 *:8443,9283,8765 running (47m) 5m ago 60m 517M - 19.0.0-2879-g324baf8d 8717720dc5e5 56088305eab7 2024-04-12T21:43:55.613 INFO:teuthology.orchestra.run.smithi018.stdout:mgr.smithi186.kqcfxk smithi186 *:8443,9283,8765 running (45m) 5m ago 57m 460M - 19.0.0-2879-g324baf8d 8717720dc5e5 09cd766e5e0d 2024-04-12T21:43:55.614 INFO:teuthology.orchestra.run.smithi018.stdout:mon.smithi018 smithi018 running (42m) 5m ago 60m 88.2M 2048M 19.0.0-2879-g324baf8d 8717720dc5e5 df3e201dfed6 2024-04-12T21:43:55.614 INFO:teuthology.orchestra.run.smithi018.stdout:mon.smithi186 smithi186 running (41m) 5m ago 57m 53.6M 2048M 19.0.0-2879-g324baf8d 8717720dc5e5 c77498da9b2b 2024-04-12T21:43:55.614 INFO:teuthology.orchestra.run.smithi018.stdout:node-exporter.smithi018 smithi018 *:9100 running (45m) 5m ago 59m 8468k - 1.5.0 0da6a335fe13 326c3d9c6847 2024-04-12T21:43:55.614 INFO:teuthology.orchestra.run.smithi018.stdout:node-exporter.smithi186 smithi186 *:9100 running (45m) 5m ago 57m 8166k - 1.5.0 0da6a335fe13 ec2112baeecf 2024-04-12T21:43:55.614 INFO:teuthology.orchestra.run.smithi018.stdout:osd.0 smithi018 running (40m) 5m ago 56m 123M 1559M 19.0.0-2879-g324baf8d 8717720dc5e5 60b1273ca047 2024-04-12T21:43:55.614 INFO:teuthology.orchestra.run.smithi018.stdout:osd.1 smithi018 running (39m) 5m ago 56m 95.4M 1559M 19.0.0-2879-g324baf8d 8717720dc5e5 170ce1e5fb8d 2024-04-12T21:43:55.614 INFO:teuthology.orchestra.run.smithi018.stdout:osd.2 smithi018 running (39m) 5m ago 55m 94.6M 1559M 19.0.0-2879-g324baf8d 8717720dc5e5 c2cbc6aa2839 2024-04-12T21:43:55.614 INFO:teuthology.orchestra.run.smithi018.stdout:osd.3 smithi186 running (38m) 5m ago 54m 101M 2581M 19.0.0-2879-g324baf8d 8717720dc5e5 0138e8262e4b 2024-04-12T21:43:55.614 INFO:teuthology.orchestra.run.smithi018.stdout:osd.4 smithi186 running (37m) 5m ago 54m 91.5M 2581M 19.0.0-2879-g324baf8d 8717720dc5e5 b20cb63b25c9 2024-04-12T21:43:55.614 INFO:teuthology.orchestra.run.smithi018.stdout:osd.5 smithi186 running (37m) 5m ago 53m 92.2M 2581M 19.0.0-2879-g324baf8d 8717720dc5e5 2513bea2610e 2024-04-12T21:43:55.614 INFO:teuthology.orchestra.run.smithi018.stdout:prometheus.smithi018 smithi018 *:9095 running (44m) 5m ago 58m 69.6M - 2.43.0 a07b618ecd1d f58841127e21 2024-04-12T21:43:55.615 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:55.615+0000 7f0e31c00700 1 -- 172.21.15.18:0/155042374 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f0e1006ec10 msgr2=0x7f0e100710c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:43:55.615 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:55.615+0000 7f0e31c00700 1 --2- 172.21.15.18:0/155042374 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f0e1006ec10 0x7f0e100710c0 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f0e1c004990 tx=0x7f0e1c009ee0 comp rx=0 tx=0).stop 2024-04-12T21:43:55.616 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:55.615+0000 7f0e31c00700 1 -- 172.21.15.18:0/155042374 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0e2c071f90 msgr2=0x7f0e2c11a510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:43:55.616 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:55.615+0000 7f0e31c00700 1 --2- 172.21.15.18:0/155042374 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0e2c071f90 0x7f0e2c11a510 secure :-1 s=READY pgs=1119 cs=0 l=1 rev1=1 crypto rx=0x7f0e18004f50 tx=0x7f0e18009e30 comp rx=0 tx=0).stop 2024-04-12T21:43:55.616 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:55.615+0000 7f0e31c00700 1 -- 172.21.15.18:0/155042374 shutdown_connections 2024-04-12T21:43:55.616 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:55.615+0000 7f0e31c00700 1 --2- 172.21.15.18:0/155042374 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f0e1006ec10 0x7f0e100710c0 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:43:55.616 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:55.615+0000 7f0e31c00700 1 --2- 172.21.15.18:0/155042374 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0e2c0708a0 0x7f0e2c119fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:43:55.616 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:55.615+0000 7f0e31c00700 1 --2- 172.21.15.18:0/155042374 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0e2c071f90 0x7f0e2c11a510 unknown :-1 s=CLOSED pgs=1119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:43:55.616 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:55.615+0000 7f0e31c00700 1 -- 172.21.15.18:0/155042374 >> 172.21.15.18:0/155042374 conn(0x7f0e2c0f8830 msgr2=0x7f0e2c0f98e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:43:55.616 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:55.615+0000 7f0e31c00700 1 -- 172.21.15.18:0/155042374 shutdown_connections 2024-04-12T21:43:55.616 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:55.616+0000 7f0e31c00700 1 -- 172.21.15.18:0/155042374 wait complete. 2024-04-12T21:43:55.763 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:55.763+0000 7feb2aa00700 1 -- 172.21.15.18:0/1912538048 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7feb24109430 msgr2=0x7feb24109800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:43:55.763 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:55.763+0000 7feb2aa00700 1 --2- 172.21.15.18:0/1912538048 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7feb24109430 0x7feb24109800 secure :-1 s=READY pgs=1120 cs=0 l=1 rev1=1 crypto rx=0x7feb14004940 tx=0x7feb14009e30 comp rx=0 tx=0).stop 2024-04-12T21:43:55.763 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:55.763+0000 7feb2aa00700 1 -- 172.21.15.18:0/1912538048 shutdown_connections 2024-04-12T21:43:55.764 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:55.763+0000 7feb2aa00700 1 --2- 172.21.15.18:0/1912538048 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7feb24109dd0 0x7feb24103440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:43:55.764 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:55.763+0000 7feb2aa00700 1 --2- 172.21.15.18:0/1912538048 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7feb24109430 0x7feb24109800 unknown :-1 s=CLOSED pgs=1120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:43:55.764 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:55.763+0000 7feb2aa00700 1 -- 172.21.15.18:0/1912538048 >> 172.21.15.18:0/1912538048 conn(0x7feb24072c00 msgr2=0x7feb24075010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:43:55.764 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:55.763+0000 7feb2aa00700 1 -- 172.21.15.18:0/1912538048 shutdown_connections 2024-04-12T21:43:55.764 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:55.763+0000 7feb2aa00700 1 -- 172.21.15.18:0/1912538048 wait complete. 2024-04-12T21:43:55.765 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:55.764+0000 7feb2aa00700 1 Processor -- start 2024-04-12T21:43:55.765 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:55.764+0000 7feb2aa00700 1 -- start start 2024-04-12T21:43:55.765 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:55.764+0000 7feb2aa00700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7feb24109430 0x7feb24119f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:43:55.765 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:55.764+0000 7feb2aa00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7feb24109dd0 0x7feb2411a4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:43:55.765 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:55.764+0000 7feb2aa00700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb2411ab20 con 0x7feb24109dd0 2024-04-12T21:43:55.765 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:55.764+0000 7feb2aa00700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb24114100 con 0x7feb24109430 2024-04-12T21:43:55.765 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:55.765+0000 7feb23400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7feb24109dd0 0x7feb2411a4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:43:55.765 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:55.765+0000 7feb23400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7feb24109dd0 0x7feb2411a4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:55346/0 (socket says 172.21.15.18:55346) 2024-04-12T21:43:55.765 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:55.765+0000 7feb23400700 1 -- 172.21.15.18:0/3131222898 learned_addr learned my addr 172.21.15.18:0/3131222898 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:43:55.766 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:55.765+0000 7feb23400700 1 -- 172.21.15.18:0/3131222898 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7feb24109430 msgr2=0x7feb24119f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:43:55.766 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:55.765+0000 7feb23400700 1 --2- 172.21.15.18:0/3131222898 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7feb24109430 0x7feb24119f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:43:55.766 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:55.765+0000 7feb23400700 1 -- 172.21.15.18:0/3131222898 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb14005040 con 0x7feb24109dd0 2024-04-12T21:43:55.767 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:55.765+0000 7feb23400700 1 --2- 172.21.15.18:0/3131222898 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7feb24109dd0 0x7feb2411a4d0 secure :-1 s=READY pgs=1121 cs=0 l=1 rev1=1 crypto rx=0x7feb10002a80 tx=0x7feb10004630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:43:55.767 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:55.766+0000 7feb1be00700 1 -- 172.21.15.18:0/3131222898 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7feb10007cc0 con 0x7feb24109dd0 2024-04-12T21:43:55.767 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:55.766+0000 7feb1be00700 1 -- 172.21.15.18:0/3131222898 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7feb1000f040 con 0x7feb24109dd0 2024-04-12T21:43:55.767 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:55.766+0000 7feb1be00700 1 -- 172.21.15.18:0/3131222898 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7feb1000daf0 con 0x7feb24109dd0 2024-04-12T21:43:55.767 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:55.766+0000 7feb2aa00700 1 -- 172.21.15.18:0/3131222898 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feb241143e0 con 0x7feb24109dd0 2024-04-12T21:43:55.767 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:55.766+0000 7feb2aa00700 1 -- 172.21.15.18:0/3131222898 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feb24114950 con 0x7feb24109dd0 2024-04-12T21:43:55.767 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:55.767+0000 7feb2aa00700 1 -- 172.21.15.18:0/3131222898 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feb24104b80 con 0x7feb24109dd0 2024-04-12T21:43:55.768 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:55.768+0000 7feb1be00700 1 -- 172.21.15.18:0/3131222898 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7feb1000dc90 con 0x7feb24109dd0 2024-04-12T21:43:55.769 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:55.769+0000 7feb1be00700 1 --2- 172.21.15.18:0/3131222898 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7feb0806ec00 0x7feb080710b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:43:55.769 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:55.769+0000 7feb1be00700 1 -- 172.21.15.18:0/3131222898 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 6757+0+0 (secure 0 0 0) 0x7feb1008e090 con 0x7feb24109dd0 2024-04-12T21:43:55.770 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:55.770+0000 7feb23e00700 1 --2- 172.21.15.18:0/3131222898 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7feb0806ec00 0x7feb080710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:43:55.770 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:55.770+0000 7feb23e00700 1 --2- 172.21.15.18:0/3131222898 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7feb0806ec00 0x7feb080710b0 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7feb14004f90 tx=0x7feb14009e30 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:43:55.774 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:55.774+0000 7feb1be00700 1 -- 172.21.15.18:0/3131222898 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7feb1005a860 con 0x7feb24109dd0 2024-04-12T21:43:55.854 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:43:55 smithi018 ceph-mon[112590]: pgmap v1327: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:43:55.854 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:43:55 smithi018 ceph-mon[112590]: from='client.36300 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:43:55.855 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:43:55 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T21:43:56.034 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:43:55 smithi186 ceph-mon[86379]: pgmap v1327: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:43:56.034 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:43:55 smithi186 ceph-mon[86379]: from='client.36300 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:43:56.034 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:43:55 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T21:43:56.099 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:56.098+0000 7feb2aa00700 1 -- 172.21.15.18:0/3131222898 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7feb24110540 con 0x7feb24109dd0 2024-04-12T21:43:56.099 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:56.099+0000 7feb1be00700 1 -- 172.21.15.18:0/3131222898 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+623 (secure 0 0 0) 0x7feb1005a3f0 con 0x7feb24109dd0 2024-04-12T21:43:56.100 INFO:teuthology.orchestra.run.smithi018.stdout:{ 2024-04-12T21:43:56.100 INFO:teuthology.orchestra.run.smithi018.stdout: "mon": { 2024-04-12T21:43:56.100 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 19.0.0-2879-g324baf8d (324baf8ddb91dea46cac50cb535c306115791355) squid (dev)": 2 2024-04-12T21:43:56.100 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-12T21:43:56.100 INFO:teuthology.orchestra.run.smithi018.stdout: "mgr": { 2024-04-12T21:43:56.100 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 19.0.0-2879-g324baf8d (324baf8ddb91dea46cac50cb535c306115791355) squid (dev)": 2 2024-04-12T21:43:56.101 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-12T21:43:56.101 INFO:teuthology.orchestra.run.smithi018.stdout: "osd": { 2024-04-12T21:43:56.101 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 19.0.0-2879-g324baf8d (324baf8ddb91dea46cac50cb535c306115791355) squid (dev)": 6 2024-04-12T21:43:56.101 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-12T21:43:56.101 INFO:teuthology.orchestra.run.smithi018.stdout: "mds": { 2024-04-12T21:43:56.101 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 19.0.0-2879-g324baf8d (324baf8ddb91dea46cac50cb535c306115791355) squid (dev)": 4 2024-04-12T21:43:56.101 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-12T21:43:56.101 INFO:teuthology.orchestra.run.smithi018.stdout: "overall": { 2024-04-12T21:43:56.101 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 19.0.0-2879-g324baf8d (324baf8ddb91dea46cac50cb535c306115791355) squid (dev)": 14 2024-04-12T21:43:56.101 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-04-12T21:43:56.101 INFO:teuthology.orchestra.run.smithi018.stdout:} 2024-04-12T21:43:56.102 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:56.102+0000 7feb2aa00700 1 -- 172.21.15.18:0/3131222898 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7feb0806ec00 msgr2=0x7feb080710b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:43:56.102 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:56.102+0000 7feb2aa00700 1 --2- 172.21.15.18:0/3131222898 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7feb0806ec00 0x7feb080710b0 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7feb14004f90 tx=0x7feb14009e30 comp rx=0 tx=0).stop 2024-04-12T21:43:56.102 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:56.102+0000 7feb2aa00700 1 -- 172.21.15.18:0/3131222898 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7feb24109dd0 msgr2=0x7feb2411a4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:43:56.103 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:56.102+0000 7feb2aa00700 1 --2- 172.21.15.18:0/3131222898 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7feb24109dd0 0x7feb2411a4d0 secure :-1 s=READY pgs=1121 cs=0 l=1 rev1=1 crypto rx=0x7feb10002a80 tx=0x7feb10004630 comp rx=0 tx=0).stop 2024-04-12T21:43:56.103 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:56.102+0000 7feb2aa00700 1 -- 172.21.15.18:0/3131222898 shutdown_connections 2024-04-12T21:43:56.103 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:56.102+0000 7feb2aa00700 1 --2- 172.21.15.18:0/3131222898 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7feb0806ec00 0x7feb080710b0 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:43:56.103 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:56.102+0000 7feb2aa00700 1 --2- 172.21.15.18:0/3131222898 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7feb24109430 0x7feb24119f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:43:56.103 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:56.102+0000 7feb2aa00700 1 --2- 172.21.15.18:0/3131222898 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7feb24109dd0 0x7feb2411a4d0 unknown :-1 s=CLOSED pgs=1121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:43:56.103 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:56.102+0000 7feb2aa00700 1 -- 172.21.15.18:0/3131222898 >> 172.21.15.18:0/3131222898 conn(0x7feb24072c00 msgr2=0x7feb24074c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:43:56.103 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:56.103+0000 7feb2aa00700 1 -- 172.21.15.18:0/3131222898 shutdown_connections 2024-04-12T21:43:56.104 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:56.103+0000 7feb2aa00700 1 -- 172.21.15.18:0/3131222898 wait complete. 2024-04-12T21:43:56.256 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:56.255+0000 7f1cba600700 1 -- 172.21.15.18:0/556463468 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1cb4105000 msgr2=0x7f1cb41053d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:43:56.256 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:56.255+0000 7f1cba600700 1 --2- 172.21.15.18:0/556463468 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1cb4105000 0x7f1cb41053d0 secure :-1 s=READY pgs=1122 cs=0 l=1 rev1=1 crypto rx=0x7f1ca4004940 tx=0x7f1ca4009e30 comp rx=0 tx=0).stop 2024-04-12T21:43:56.256 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:56.255+0000 7f1cba600700 1 -- 172.21.15.18:0/556463468 shutdown_connections 2024-04-12T21:43:56.256 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:56.255+0000 7f1cba600700 1 --2- 172.21.15.18:0/556463468 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1cb41059a0 0x7f1cb40ff010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:43:56.257 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:56.255+0000 7f1cba600700 1 --2- 172.21.15.18:0/556463468 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1cb4105000 0x7f1cb41053d0 unknown :-1 s=CLOSED pgs=1122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:43:56.257 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:56.256+0000 7f1cba600700 1 -- 172.21.15.18:0/556463468 >> 172.21.15.18:0/556463468 conn(0x7f1cb40fab70 msgr2=0x7f1cb40fcf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:43:56.257 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:56.256+0000 7f1cba600700 1 -- 172.21.15.18:0/556463468 shutdown_connections 2024-04-12T21:43:56.257 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:56.256+0000 7f1cba600700 1 -- 172.21.15.18:0/556463468 wait complete. 2024-04-12T21:43:56.258 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:56.256+0000 7f1cba600700 1 Processor -- start 2024-04-12T21:43:56.258 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:56.257+0000 7f1cba600700 1 -- start start 2024-04-12T21:43:56.258 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:56.257+0000 7f1cba600700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1cb4105000 0x7f1cb4115b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:43:56.258 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:56.257+0000 7f1cba600700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1cb41059a0 0x7f1cb41160b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:43:56.259 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:56.257+0000 7f1cba600700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1cb4116700 con 0x7f1cb41059a0 2024-04-12T21:43:56.259 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:56.257+0000 7f1cba600700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1cb410fce0 con 0x7f1cb4105000 2024-04-12T21:43:56.259 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:56.257+0000 7f1cb2a00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1cb41059a0 0x7f1cb41160b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:43:56.259 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:56.258+0000 7f1cb2a00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1cb41059a0 0x7f1cb41160b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:55368/0 (socket says 172.21.15.18:55368) 2024-04-12T21:43:56.259 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:56.258+0000 7f1cb2a00700 1 -- 172.21.15.18:0/2635945263 learned_addr learned my addr 172.21.15.18:0/2635945263 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:43:56.259 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:56.258+0000 7f1cb3400700 1 --2- 172.21.15.18:0/2635945263 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1cb4105000 0x7f1cb4115b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:43:56.259 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:56.258+0000 7f1cb2a00700 1 -- 172.21.15.18:0/2635945263 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1cb4105000 msgr2=0x7f1cb4115b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:43:56.259 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:56.258+0000 7f1cb2a00700 1 --2- 172.21.15.18:0/2635945263 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1cb4105000 0x7f1cb4115b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:43:56.259 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:56.258+0000 7f1cb2a00700 1 -- 172.21.15.18:0/2635945263 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ca4005040 con 0x7f1cb41059a0 2024-04-12T21:43:56.259 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:56.258+0000 7f1cb2a00700 1 --2- 172.21.15.18:0/2635945263 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1cb41059a0 0x7f1cb41160b0 secure :-1 s=READY pgs=1123 cs=0 l=1 rev1=1 crypto rx=0x7f1ca0002a50 tx=0x7f1ca0004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:43:56.260 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:56.259+0000 7f1cab400700 1 -- 172.21.15.18:0/2635945263 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1ca0007cc0 con 0x7f1cb41059a0 2024-04-12T21:43:56.260 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:56.259+0000 7f1cab400700 1 -- 172.21.15.18:0/2635945263 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f1ca000f040 con 0x7f1cb41059a0 2024-04-12T21:43:56.260 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:56.259+0000 7f1cba600700 1 -- 172.21.15.18:0/2635945263 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1cb410ffc0 con 0x7f1cb41059a0 2024-04-12T21:43:56.260 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:56.259+0000 7f1cab400700 1 -- 172.21.15.18:0/2635945263 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1ca000daf0 con 0x7f1cb41059a0 2024-04-12T21:43:56.260 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:56.259+0000 7f1cba600700 1 -- 172.21.15.18:0/2635945263 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1cb4110580 con 0x7f1cb41059a0 2024-04-12T21:43:56.261 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:56.260+0000 7f1cba600700 1 -- 172.21.15.18:0/2635945263 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1cb4100750 con 0x7f1cb41059a0 2024-04-12T21:43:56.261 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:56.261+0000 7f1cab400700 1 -- 172.21.15.18:0/2635945263 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7f1ca0009810 con 0x7f1cb41059a0 2024-04-12T21:43:56.262 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:56.262+0000 7f1cab400700 1 --2- 172.21.15.18:0/2635945263 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f1c9806ec00 0x7f1c980710b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:43:56.262 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:56.262+0000 7f1cab400700 1 -- 172.21.15.18:0/2635945263 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 6757+0+0 (secure 0 0 0) 0x7f1ca008e700 con 0x7f1cb41059a0 2024-04-12T21:43:56.262 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:56.262+0000 7f1cb3400700 1 --2- 172.21.15.18:0/2635945263 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f1c9806ec00 0x7f1c980710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:43:56.263 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:56.263+0000 7f1cb3400700 1 --2- 172.21.15.18:0/2635945263 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f1c9806ec00 0x7f1c980710b0 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f1ca400b3b0 tx=0x7f1ca4009e30 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:43:56.267 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:56.267+0000 7f1cab400700 1 -- 172.21.15.18:0/2635945263 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f1ca005aeb0 con 0x7f1cb41059a0 2024-04-12T21:43:56.551 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:56.550+0000 7f1cba600700 1 -- 172.21.15.18:0/2635945263 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f1cb410c100 con 0x7f1cb41059a0 2024-04-12T21:43:56.552 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:56.551+0000 7f1cab400700 1 -- 172.21.15.18:0/2635945263 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 33 v33) v1 ==== 76+0+1876 (secure 0 0 0) 0x7f1ca005aa40 con 0x7f1cb41059a0 2024-04-12T21:43:56.553 INFO:teuthology.orchestra.run.smithi018.stdout:e33 2024-04-12T21:43:56.553 INFO:teuthology.orchestra.run.smithi018.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-12T21:43:56.553 INFO:teuthology.orchestra.run.smithi018.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-12T21:43:56.553 INFO:teuthology.orchestra.run.smithi018.stdout:legacy client fscid: 1 2024-04-12T21:43:56.553 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-12T21:43:56.553 INFO:teuthology.orchestra.run.smithi018.stdout:Filesystem 'cephfs' (1) 2024-04-12T21:43:56.553 INFO:teuthology.orchestra.run.smithi018.stdout:fs_name cephfs 2024-04-12T21:43:56.553 INFO:teuthology.orchestra.run.smithi018.stdout:epoch 24 2024-04-12T21:43:56.553 INFO:teuthology.orchestra.run.smithi018.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-04-12T21:43:56.553 INFO:teuthology.orchestra.run.smithi018.stdout:created 2024-04-12T20:51:57.362850+0000 2024-04-12T21:43:56.554 INFO:teuthology.orchestra.run.smithi018.stdout:modified 2024-04-12T21:06:59.717190+0000 2024-04-12T21:43:56.554 INFO:teuthology.orchestra.run.smithi018.stdout:tableserver 0 2024-04-12T21:43:56.554 INFO:teuthology.orchestra.run.smithi018.stdout:root 0 2024-04-12T21:43:56.554 INFO:teuthology.orchestra.run.smithi018.stdout:session_timeout 60 2024-04-12T21:43:56.554 INFO:teuthology.orchestra.run.smithi018.stdout:session_autoclose 300 2024-04-12T21:43:56.554 INFO:teuthology.orchestra.run.smithi018.stdout:max_file_size 1099511627776 2024-04-12T21:43:56.554 INFO:teuthology.orchestra.run.smithi018.stdout:max_xattr_size 65536 2024-04-12T21:43:56.554 INFO:teuthology.orchestra.run.smithi018.stdout:required_client_features {} 2024-04-12T21:43:56.554 INFO:teuthology.orchestra.run.smithi018.stdout:last_failure 0 2024-04-12T21:43:56.554 INFO:teuthology.orchestra.run.smithi018.stdout:last_failure_osd_epoch 100 2024-04-12T21:43:56.554 INFO:teuthology.orchestra.run.smithi018.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-12T21:43:56.554 INFO:teuthology.orchestra.run.smithi018.stdout:max_mds 2 2024-04-12T21:43:56.554 INFO:teuthology.orchestra.run.smithi018.stdout:in 0,1 2024-04-12T21:43:56.554 INFO:teuthology.orchestra.run.smithi018.stdout:up {} 2024-04-12T21:43:56.555 INFO:teuthology.orchestra.run.smithi018.stdout:failed 0,1 2024-04-12T21:43:56.555 INFO:teuthology.orchestra.run.smithi018.stdout:damaged 2024-04-12T21:43:56.555 INFO:teuthology.orchestra.run.smithi018.stdout:stopped 2024-04-12T21:43:56.555 INFO:teuthology.orchestra.run.smithi018.stdout:data_pools [3] 2024-04-12T21:43:56.555 INFO:teuthology.orchestra.run.smithi018.stdout:metadata_pool 2 2024-04-12T21:43:56.555 INFO:teuthology.orchestra.run.smithi018.stdout:inline_data disabled 2024-04-12T21:43:56.555 INFO:teuthology.orchestra.run.smithi018.stdout:balancer 2024-04-12T21:43:56.555 INFO:teuthology.orchestra.run.smithi018.stdout:bal_rank_mask -1 2024-04-12T21:43:56.555 INFO:teuthology.orchestra.run.smithi018.stdout:standby_count_wanted 1 2024-04-12T21:43:56.555 INFO:teuthology.orchestra.run.smithi018.stdout:qdb_cluster leader: 0 members: 2024-04-12T21:43:56.555 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-12T21:43:56.555 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-12T21:43:56.555 INFO:teuthology.orchestra.run.smithi018.stdout:Standby daemons: 2024-04-12T21:43:56.555 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-12T21:43:56.555 INFO:teuthology.orchestra.run.smithi018.stdout:[mds.cephfs.smithi018.acqbsc{-1:34436} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.18:6828/755179764,v1:172.21.15.18:6829/755179764] compat {c=[1],r=[1],i=[1fff]}] 2024-04-12T21:43:56.556 INFO:teuthology.orchestra.run.smithi018.stdout:[mds.cephfs.smithi018.szjluo{-1:34468} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.18:6826/2407992653,v1:172.21.15.18:6827/2407992653] compat {c=[1],r=[1],i=[1fff]}] 2024-04-12T21:43:56.556 INFO:teuthology.orchestra.run.smithi018.stdout:[mds.cephfs.smithi186.jdllyn{-1:44595} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.186:6824/3523444202,v1:172.21.15.186:6825/3523444202] compat {c=[1],r=[1],i=[1fff]}] 2024-04-12T21:43:56.556 INFO:teuthology.orchestra.run.smithi018.stdout:[mds.cephfs.smithi186.wjnafs{-1:44875} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.186:6826/3122443766,v1:172.21.15.186:6827/3122443766] compat {c=[1],r=[1],i=[1fff]}] 2024-04-12T21:43:56.556 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:56.554+0000 7f1cba600700 1 -- 172.21.15.18:0/2635945263 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f1c9806ec00 msgr2=0x7f1c980710b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:43:56.557 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:56.554+0000 7f1cba600700 1 --2- 172.21.15.18:0/2635945263 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f1c9806ec00 0x7f1c980710b0 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f1ca400b3b0 tx=0x7f1ca4009e30 comp rx=0 tx=0).stop 2024-04-12T21:43:56.557 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:56.554+0000 7f1cba600700 1 -- 172.21.15.18:0/2635945263 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1cb41059a0 msgr2=0x7f1cb41160b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:43:56.557 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:56.554+0000 7f1cba600700 1 --2- 172.21.15.18:0/2635945263 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1cb41059a0 0x7f1cb41160b0 secure :-1 s=READY pgs=1123 cs=0 l=1 rev1=1 crypto rx=0x7f1ca0002a50 tx=0x7f1ca0004600 comp rx=0 tx=0).stop 2024-04-12T21:43:56.557 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:56.555+0000 7f1cba600700 1 -- 172.21.15.18:0/2635945263 shutdown_connections 2024-04-12T21:43:56.557 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:56.555+0000 7f1cba600700 1 --2- 172.21.15.18:0/2635945263 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f1c9806ec00 0x7f1c980710b0 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:43:56.557 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:56.555+0000 7f1cba600700 1 --2- 172.21.15.18:0/2635945263 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1cb4105000 0x7f1cb4115b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:43:56.557 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:56.555+0000 7f1cba600700 1 --2- 172.21.15.18:0/2635945263 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1cb41059a0 0x7f1cb41160b0 unknown :-1 s=CLOSED pgs=1123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:43:56.557 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:56.555+0000 7f1cba600700 1 -- 172.21.15.18:0/2635945263 >> 172.21.15.18:0/2635945263 conn(0x7f1cb40fab70 msgr2=0x7f1cb40fbb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:43:56.557 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:56.555+0000 7f1cba600700 1 -- 172.21.15.18:0/2635945263 shutdown_connections 2024-04-12T21:43:56.557 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:56.555+0000 7f1cba600700 1 -- 172.21.15.18:0/2635945263 wait complete. 2024-04-12T21:43:56.557 INFO:teuthology.orchestra.run.smithi018.stderr:dumped fsmap epoch 33 2024-04-12T21:43:56.711 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:56.711+0000 7f3ed5c00700 1 -- 172.21.15.18:0/240363682 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3ed0103600 msgr2=0x7f3ed00fcc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:43:56.712 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:56.711+0000 7f3ed5c00700 1 --2- 172.21.15.18:0/240363682 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3ed0103600 0x7f3ed00fcc70 secure :-1 s=READY pgs=1124 cs=0 l=1 rev1=1 crypto rx=0x7f3ebc004940 tx=0x7f3ebc009e30 comp rx=0 tx=0).stop 2024-04-12T21:43:56.712 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:56.711+0000 7f3ed5c00700 1 -- 172.21.15.18:0/240363682 shutdown_connections 2024-04-12T21:43:56.712 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:56.711+0000 7f3ed5c00700 1 --2- 172.21.15.18:0/240363682 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3ed0103600 0x7f3ed00fcc70 unknown :-1 s=CLOSED pgs=1124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:43:56.712 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:56.711+0000 7f3ed5c00700 1 --2- 172.21.15.18:0/240363682 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3ed0102c60 0x7f3ed0103030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:43:56.712 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:56.711+0000 7f3ed5c00700 1 -- 172.21.15.18:0/240363682 >> 172.21.15.18:0/240363682 conn(0x7f3ed00f8770 msgr2=0x7f3ed00fab80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:43:56.712 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:56.711+0000 7f3ed5c00700 1 -- 172.21.15.18:0/240363682 shutdown_connections 2024-04-12T21:43:56.712 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:56.711+0000 7f3ed5c00700 1 -- 172.21.15.18:0/240363682 wait complete. 2024-04-12T21:43:56.713 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:56.712+0000 7f3ed5c00700 1 Processor -- start 2024-04-12T21:43:56.713 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:56.712+0000 7f3ed5c00700 1 -- start start 2024-04-12T21:43:56.713 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:56.712+0000 7f3ed5c00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3ed0102c60 0x7f3ed0115c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:43:56.713 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:56.712+0000 7f3ed5c00700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3ed0103600 0x7f3ed0116170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:43:56.714 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:56.712+0000 7f3ed5c00700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ed01167c0 con 0x7f3ed0102c60 2024-04-12T21:43:56.714 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:56.712+0000 7f3ed5c00700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ed010fb90 con 0x7f3ed0103600 2024-04-12T21:43:56.714 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:56.713+0000 7f3ecfe00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3ed0102c60 0x7f3ed0115c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:43:56.714 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:56.713+0000 7f3ecfe00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3ed0102c60 0x7f3ed0115c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:55378/0 (socket says 172.21.15.18:55378) 2024-04-12T21:43:56.714 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:56.713+0000 7f3ecfe00700 1 -- 172.21.15.18:0/1248002840 learned_addr learned my addr 172.21.15.18:0/1248002840 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:43:56.714 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:56.713+0000 7f3ecfe00700 1 -- 172.21.15.18:0/1248002840 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3ed0103600 msgr2=0x7f3ed0116170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:43:56.714 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:56.713+0000 7f3ecfe00700 1 --2- 172.21.15.18:0/1248002840 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3ed0103600 0x7f3ed0116170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:43:56.714 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:56.713+0000 7f3ecfe00700 1 -- 172.21.15.18:0/1248002840 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ebc005040 con 0x7f3ed0102c60 2024-04-12T21:43:56.714 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:56.713+0000 7f3ecfe00700 1 --2- 172.21.15.18:0/1248002840 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3ed0102c60 0x7f3ed0115c30 secure :-1 s=READY pgs=1125 cs=0 l=1 rev1=1 crypto rx=0x7f3ec000c810 tx=0x7f3ec0004470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:43:56.715 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:56.713+0000 7f3ec7e00700 1 -- 172.21.15.18:0/1248002840 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3ec0012810 con 0x7f3ed0102c60 2024-04-12T21:43:56.715 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:56.714+0000 7f3ec7e00700 1 -- 172.21.15.18:0/1248002840 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f3ec0012e40 con 0x7f3ed0102c60 2024-04-12T21:43:56.715 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:56.714+0000 7f3ec7e00700 1 -- 172.21.15.18:0/1248002840 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3ec000a370 con 0x7f3ed0102c60 2024-04-12T21:43:56.715 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:56.714+0000 7f3ed5c00700 1 -- 172.21.15.18:0/1248002840 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3ed010fe70 con 0x7f3ed0102c60 2024-04-12T21:43:56.715 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:56.714+0000 7f3ed5c00700 1 -- 172.21.15.18:0/1248002840 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3ed0110430 con 0x7f3ed0102c60 2024-04-12T21:43:56.715 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:56.715+0000 7f3ed5c00700 1 -- 172.21.15.18:0/1248002840 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3ed00fe3b0 con 0x7f3ed0102c60 2024-04-12T21:43:56.716 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:56.716+0000 7f3ec7e00700 1 -- 172.21.15.18:0/1248002840 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7f3ec0011810 con 0x7f3ed0102c60 2024-04-12T21:43:56.717 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:56.717+0000 7f3ec7e00700 1 --2- 172.21.15.18:0/1248002840 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f3eb406ec00 0x7f3eb40710b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:43:56.717 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:56.717+0000 7f3ec7e00700 1 -- 172.21.15.18:0/1248002840 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 6757+0+0 (secure 0 0 0) 0x7f3ec008ee20 con 0x7f3ed0102c60 2024-04-12T21:43:56.717 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:56.717+0000 7f3ecf400700 1 --2- 172.21.15.18:0/1248002840 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f3eb406ec00 0x7f3eb40710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:43:56.718 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:56.718+0000 7f3ecf400700 1 --2- 172.21.15.18:0/1248002840 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f3eb406ec00 0x7f3eb40710b0 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f3ebc004f90 tx=0x7f3ebc009e30 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:43:56.722 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:56.722+0000 7f3ec7e00700 1 -- 172.21.15.18:0/1248002840 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f3ec005b5f0 con 0x7f3ed0102c60 2024-04-12T21:43:56.855 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:43:56 smithi018 ceph-mon[112590]: from='client.36304 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:43:56.855 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:43:56 smithi018 ceph-mon[112590]: from='client.36308 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:43:56.855 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:43:56 smithi018 ceph-mon[112590]: from='client.? 172.21.15.18:0/3131222898' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T21:43:56.855 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:43:56 smithi018 ceph-mon[112590]: from='client.? 172.21.15.18:0/2635945263' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-12T21:43:56.968 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:56.967+0000 7f3ed5c00700 1 -- 172.21.15.18:0/1248002840 --> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f3ed0109dc0 con 0x7f3eb406ec00 2024-04-12T21:43:56.969 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:56.969+0000 7f3ec7e00700 1 -- 172.21.15.18:0/1248002840 <== mgr.34102 v2:172.21.15.18:6800/3812891919 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f3ed0109dc0 con 0x7f3eb406ec00 2024-04-12T21:43:56.969 INFO:teuthology.orchestra.run.smithi018.stdout:{ 2024-04-12T21:43:56.970 INFO:teuthology.orchestra.run.smithi018.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:f4452def3be4d350f4a8e5feaf7ebb1511be9298fe68478ba7a2d6fbd7d69a63", 2024-04-12T21:43:56.970 INFO:teuthology.orchestra.run.smithi018.stdout: "in_progress": true, 2024-04-12T21:43:56.970 INFO:teuthology.orchestra.run.smithi018.stdout: "which": "Upgrading all daemon types on all hosts", 2024-04-12T21:43:56.970 INFO:teuthology.orchestra.run.smithi018.stdout: "services_complete": [ 2024-04-12T21:43:56.970 INFO:teuthology.orchestra.run.smithi018.stdout: "mgr", 2024-04-12T21:43:56.970 INFO:teuthology.orchestra.run.smithi018.stdout: "mds", 2024-04-12T21:43:56.970 INFO:teuthology.orchestra.run.smithi018.stdout: "crash", 2024-04-12T21:43:56.970 INFO:teuthology.orchestra.run.smithi018.stdout: "osd", 2024-04-12T21:43:56.970 INFO:teuthology.orchestra.run.smithi018.stdout: "mon" 2024-04-12T21:43:56.970 INFO:teuthology.orchestra.run.smithi018.stdout: ], 2024-04-12T21:43:56.970 INFO:teuthology.orchestra.run.smithi018.stdout: "progress": "16/21 daemons upgraded", 2024-04-12T21:43:56.970 INFO:teuthology.orchestra.run.smithi018.stdout: "message": "Currently upgrading mds daemons", 2024-04-12T21:43:56.970 INFO:teuthology.orchestra.run.smithi018.stdout: "is_paused": false 2024-04-12T21:43:56.970 INFO:teuthology.orchestra.run.smithi018.stdout:} 2024-04-12T21:43:56.972 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:56.971+0000 7f3ed5c00700 1 -- 172.21.15.18:0/1248002840 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f3eb406ec00 msgr2=0x7f3eb40710b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:43:56.972 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:56.971+0000 7f3ed5c00700 1 --2- 172.21.15.18:0/1248002840 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f3eb406ec00 0x7f3eb40710b0 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f3ebc004f90 tx=0x7f3ebc009e30 comp rx=0 tx=0).stop 2024-04-12T21:43:56.972 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:56.971+0000 7f3ed5c00700 1 -- 172.21.15.18:0/1248002840 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3ed0102c60 msgr2=0x7f3ed0115c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:43:56.972 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:56.971+0000 7f3ed5c00700 1 --2- 172.21.15.18:0/1248002840 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3ed0102c60 0x7f3ed0115c30 secure :-1 s=READY pgs=1125 cs=0 l=1 rev1=1 crypto rx=0x7f3ec000c810 tx=0x7f3ec0004470 comp rx=0 tx=0).stop 2024-04-12T21:43:56.972 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:56.971+0000 7f3ed5c00700 1 -- 172.21.15.18:0/1248002840 shutdown_connections 2024-04-12T21:43:56.972 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:56.971+0000 7f3ed5c00700 1 --2- 172.21.15.18:0/1248002840 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f3eb406ec00 0x7f3eb40710b0 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:43:56.972 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:56.971+0000 7f3ed5c00700 1 --2- 172.21.15.18:0/1248002840 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3ed0102c60 0x7f3ed0115c30 unknown :-1 s=CLOSED pgs=1125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:43:56.973 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:56.971+0000 7f3ed5c00700 1 --2- 172.21.15.18:0/1248002840 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3ed0103600 0x7f3ed0116170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:43:56.973 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:56.971+0000 7f3ed5c00700 1 -- 172.21.15.18:0/1248002840 >> 172.21.15.18:0/1248002840 conn(0x7f3ed00f8770 msgr2=0x7f3ed0071ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:43:56.973 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:56.972+0000 7f3ed5c00700 1 -- 172.21.15.18:0/1248002840 shutdown_connections 2024-04-12T21:43:56.973 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:56.972+0000 7f3ed5c00700 1 -- 172.21.15.18:0/1248002840 wait complete. 2024-04-12T21:43:57.130 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:57.129+0000 7f5d0be00700 1 -- 172.21.15.18:0/546734249 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5d0c105800 msgr2=0x7f5d0c0fee70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:43:57.130 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:57.129+0000 7f5d0be00700 1 --2- 172.21.15.18:0/546734249 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5d0c105800 0x7f5d0c0fee70 secure :-1 s=READY pgs=1126 cs=0 l=1 rev1=1 crypto rx=0x7f5cf8004940 tx=0x7f5cf8009e30 comp rx=0 tx=0).stop 2024-04-12T21:43:57.130 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:57.129+0000 7f5d0be00700 1 -- 172.21.15.18:0/546734249 shutdown_connections 2024-04-12T21:43:57.130 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:57.129+0000 7f5d0be00700 1 --2- 172.21.15.18:0/546734249 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5d0c105800 0x7f5d0c0fee70 unknown :-1 s=CLOSED pgs=1126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:43:57.130 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:57.129+0000 7f5d0be00700 1 --2- 172.21.15.18:0/546734249 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5d0c104e60 0x7f5d0c105230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:43:57.130 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:57.129+0000 7f5d0be00700 1 -- 172.21.15.18:0/546734249 >> 172.21.15.18:0/546734249 conn(0x7f5d0c0fab10 msgr2=0x7f5d0c0fcef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:43:57.131 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:57.130+0000 7f5d0be00700 1 -- 172.21.15.18:0/546734249 shutdown_connections 2024-04-12T21:43:57.131 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:57.130+0000 7f5d0be00700 1 -- 172.21.15.18:0/546734249 wait complete. 2024-04-12T21:43:57.131 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:57.131+0000 7f5d0be00700 1 Processor -- start 2024-04-12T21:43:57.131 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:57.131+0000 7f5d0be00700 1 -- start start 2024-04-12T21:43:57.132 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:57.132+0000 7f5d0be00700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5d0c104e60 0x7f5d0c115bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:43:57.132 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:57.132+0000 7f5d0be00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5d0c105800 0x7f5d0c1160f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:43:57.132 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:57.132+0000 7f5d0be00700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d0c116740 con 0x7f5d0c105800 2024-04-12T21:43:57.132 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:57.132+0000 7f5d0be00700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d0c10fb30 con 0x7f5d0c104e60 2024-04-12T21:43:57.132 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:57.132+0000 7f5d0a000700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5d0c105800 0x7f5d0c1160f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:43:57.132 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:57.132+0000 7f5d0a000700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5d0c105800 0x7f5d0c1160f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:55400/0 (socket says 172.21.15.18:55400) 2024-04-12T21:43:57.133 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:57.132+0000 7f5d0a000700 1 -- 172.21.15.18:0/3323951084 learned_addr learned my addr 172.21.15.18:0/3323951084 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:43:57.134 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:57.132+0000 7f5d0aa00700 1 --2- 172.21.15.18:0/3323951084 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5d0c104e60 0x7f5d0c115bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:43:57.134 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:57.133+0000 7f5d0a000700 1 -- 172.21.15.18:0/3323951084 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5d0c104e60 msgr2=0x7f5d0c115bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:43:57.134 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:57.133+0000 7f5d0a000700 1 --2- 172.21.15.18:0/3323951084 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5d0c104e60 0x7f5d0c115bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:43:57.134 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:57.133+0000 7f5d0a000700 1 -- 172.21.15.18:0/3323951084 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5cf8005040 con 0x7f5d0c105800 2024-04-12T21:43:57.134 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:57.133+0000 7f5d0a000700 1 --2- 172.21.15.18:0/3323951084 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5d0c105800 0x7f5d0c1160f0 secure :-1 s=READY pgs=1127 cs=0 l=1 rev1=1 crypto rx=0x7f5cf8009100 tx=0x7f5cf8009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:43:57.134 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:57.133+0000 7f5d02a00700 1 -- 172.21.15.18:0/3323951084 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5cf800e040 con 0x7f5d0c105800 2024-04-12T21:43:57.134 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:57.133+0000 7f5d02a00700 1 -- 172.21.15.18:0/3323951084 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f5cf8013420 con 0x7f5d0c105800 2024-04-12T21:43:57.134 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:57.133+0000 7f5d02a00700 1 -- 172.21.15.18:0/3323951084 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5cf801c630 con 0x7f5d0c105800 2024-04-12T21:43:57.134 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:57.133+0000 7f5d0be00700 1 -- 172.21.15.18:0/3323951084 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5d0c10fdb0 con 0x7f5d0c105800 2024-04-12T21:43:57.134 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:57.133+0000 7f5d0be00700 1 -- 172.21.15.18:0/3323951084 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5d0c110370 con 0x7f5d0c105800 2024-04-12T21:43:57.135 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:57.134+0000 7f5d0be00700 1 -- 172.21.15.18:0/3323951084 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5d0c107330 con 0x7f5d0c105800 2024-04-12T21:43:57.136 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:57.136+0000 7f5d02a00700 1 -- 172.21.15.18:0/3323951084 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7f5cf8013590 con 0x7f5d0c105800 2024-04-12T21:43:57.137 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:57.137+0000 7f5d02a00700 1 --2- 172.21.15.18:0/3323951084 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f5cf006ec00 0x7f5cf00710b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:43:57.138 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:57.137+0000 7f5d02a00700 1 -- 172.21.15.18:0/3323951084 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 6757+0+0 (secure 0 0 0) 0x7f5cf8090730 con 0x7f5d0c105800 2024-04-12T21:43:57.138 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:57.138+0000 7f5d0aa00700 1 --2- 172.21.15.18:0/3323951084 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f5cf006ec00 0x7f5cf00710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:43:57.138 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:57.138+0000 7f5d0aa00700 1 --2- 172.21.15.18:0/3323951084 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f5cf006ec00 0x7f5cf00710b0 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f5cfc007fd0 tx=0x7f5cfc009040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:43:57.141 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:57.141+0000 7f5d02a00700 1 -- 172.21.15.18:0/3323951084 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f5cf805cf60 con 0x7f5d0c105800 2024-04-12T21:43:57.284 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:43:56 smithi186 ceph-mon[86379]: from='client.36304 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:43:57.284 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:43:56 smithi186 ceph-mon[86379]: from='client.36308 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:43:57.284 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:43:56 smithi186 ceph-mon[86379]: from='client.? 172.21.15.18:0/3131222898' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T21:43:57.284 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:43:56 smithi186 ceph-mon[86379]: from='client.? 172.21.15.18:0/2635945263' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-12T21:43:57.456 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:57.455+0000 7f5d0be00700 1 -- 172.21.15.18:0/3323951084 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f5d0c0fe680 con 0x7f5d0c105800 2024-04-12T21:43:57.456 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:57.456+0000 7f5d02a00700 1 -- 172.21.15.18:0/3323951084 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+4303 (secure 0 0 0) 0x7f5d0c0fe680 con 0x7f5d0c105800 2024-04-12T21:43:57.457 INFO:teuthology.orchestra.run.smithi018.stdout:HEALTH_ERR failed to probe daemons or devices; 1 filesystem is degraded; 1 filesystem is offline 2024-04-12T21:43:57.457 INFO:teuthology.orchestra.run.smithi018.stdout:[WRN] CEPHADM_REFRESH_FAILED: failed to probe daemons or devices 2024-04-12T21:43:57.457 INFO:teuthology.orchestra.run.smithi018.stdout: host smithi186 `cephadm ceph-volume` failed: cephadm exited with an error code: 1, stderr: Inferring config /var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/mon.smithi186/config 2024-04-12T21:43:57.457 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/podman: timeout after 895 seconds 2024-04-12T21:43:57.457 INFO:teuthology.orchestra.run.smithi018.stdout:Non-zero exit code 124 from /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:f4452def3be4d350f4a8e5feaf7ebb1511be9298fe68478ba7a2d6fbd7d69a63 -e NODE_NAME=smithi186 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968:/var/run/ceph:z -v /var/log/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968:/var/log/ceph:z -v /var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmptgom53f9:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:f4452def3be4d350f4a8e5feaf7ebb1511be9298fe68478ba7a2d6fbd7d69a63 inventory --format=json-pretty --filter-for-batch 2024-04-12T21:43:57.457 INFO:teuthology.orchestra.run.smithi018.stdout:Traceback (most recent call last): 2024-04-12T21:43:57.457 INFO:teuthology.orchestra.run.smithi018.stdout: File "/usr/lib64/python3.9/runpy.py", line 197, in _run_module_as_main 2024-04-12T21:43:57.457 INFO:teuthology.orchestra.run.smithi018.stdout: return _run_code(code, main_globals, None, 2024-04-12T21:43:57.457 INFO:teuthology.orchestra.run.smithi018.stdout: File "/usr/lib64/python3.9/runpy.py", line 87, in _run_code 2024-04-12T21:43:57.458 INFO:teuthology.orchestra.run.smithi018.stdout: exec(code, run_globals) 2024-04-12T21:43:57.458 INFO:teuthology.orchestra.run.smithi018.stdout: File "/var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/cephadm.e44f3ed5e34e4ef52eedef233d1190bff004470d8e8005127b5312bff357fa7f/__main__.py", line 5618, in 2024-04-12T21:43:57.458 INFO:teuthology.orchestra.run.smithi018.stdout: File "/var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/cephadm.e44f3ed5e34e4ef52eedef233d1190bff004470d8e8005127b5312bff357fa7f/__main__.py", line 5606, in main 2024-04-12T21:43:57.458 INFO:teuthology.orchestra.run.smithi018.stdout: File "/var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/cephadm.e44f3ed5e34e4ef52eedef233d1190bff004470d8e8005127b5312bff357fa7f/__main__.py", line 408, in _infer_config 2024-04-12T21:43:57.458 INFO:teuthology.orchestra.run.smithi018.stdout: File "/var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/cephadm.e44f3ed5e34e4ef52eedef233d1190bff004470d8e8005127b5312bff357fa7f/__main__.py", line 323, in _infer_fsid 2024-04-12T21:43:57.458 INFO:teuthology.orchestra.run.smithi018.stdout: File "/var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/cephadm.e44f3ed5e34e4ef52eedef233d1190bff004470d8e8005127b5312bff357fa7f/__main__.py", line 436, in _infer_image 2024-04-12T21:43:57.458 INFO:teuthology.orchestra.run.smithi018.stdout: File "/var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/cephadm.e44f3ed5e34e4ef52eedef233d1190bff004470d8e8005127b5312bff357fa7f/__main__.py", line 310, in _validate_fsid 2024-04-12T21:43:57.458 INFO:teuthology.orchestra.run.smithi018.stdout: File "/var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/cephadm.e44f3ed5e34e4ef52eedef233d1190bff004470d8e8005127b5312bff357fa7f/__main__.py", line 3359, in command_ceph_volume 2024-04-12T21:43:57.458 INFO:teuthology.orchestra.run.smithi018.stdout: File "/var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/cephadm.e44f3ed5e34e4ef52eedef233d1190bff004470d8e8005127b5312bff357fa7f/cephadmlib/call_wrappers.py", line 310, in call_throws 2024-04-12T21:43:57.458 INFO:teuthology.orchestra.run.smithi018.stdout:RuntimeError: Failed command: /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:f4452def3be4d350f4a8e5feaf7ebb1511be9298fe68478ba7a2d6fbd7d69a63 -e NODE_NAME=smithi186 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968:/var/run/ceph:z -v /var/log/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968:/var/log/ceph:z -v /var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmptgom53f9:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:f4452def3be4d350f4a8e5feaf7ebb1511be9298fe68478ba7a2d6fbd7d69a63 inventory --format=json-pretty --filter-for-batch 2024-04-12T21:43:57.458 INFO:teuthology.orchestra.run.smithi018.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-04-12T21:43:57.458 INFO:teuthology.orchestra.run.smithi018.stdout: fs cephfs is degraded 2024-04-12T21:43:57.458 INFO:teuthology.orchestra.run.smithi018.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-04-12T21:43:57.458 INFO:teuthology.orchestra.run.smithi018.stdout: fs cephfs is offline because no MDS is active for it. 2024-04-12T21:43:57.459 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:57.458+0000 7f5d0be00700 1 -- 172.21.15.18:0/3323951084 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f5cf006ec00 msgr2=0x7f5cf00710b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:43:57.459 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:57.459+0000 7f5d0be00700 1 --2- 172.21.15.18:0/3323951084 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f5cf006ec00 0x7f5cf00710b0 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f5cfc007fd0 tx=0x7f5cfc009040 comp rx=0 tx=0).stop 2024-04-12T21:43:57.459 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:57.459+0000 7f5d0be00700 1 -- 172.21.15.18:0/3323951084 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5d0c105800 msgr2=0x7f5d0c1160f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:43:57.459 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:57.459+0000 7f5d0be00700 1 --2- 172.21.15.18:0/3323951084 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5d0c105800 0x7f5d0c1160f0 secure :-1 s=READY pgs=1127 cs=0 l=1 rev1=1 crypto rx=0x7f5cf8009100 tx=0x7f5cf8009e30 comp rx=0 tx=0).stop 2024-04-12T21:43:57.460 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:57.459+0000 7f5d0be00700 1 -- 172.21.15.18:0/3323951084 shutdown_connections 2024-04-12T21:43:57.460 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:57.459+0000 7f5d0be00700 1 --2- 172.21.15.18:0/3323951084 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f5cf006ec00 0x7f5cf00710b0 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:43:57.460 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:57.459+0000 7f5d0be00700 1 --2- 172.21.15.18:0/3323951084 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5d0c104e60 0x7f5d0c115bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:43:57.460 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:57.459+0000 7f5d0be00700 1 --2- 172.21.15.18:0/3323951084 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5d0c105800 0x7f5d0c1160f0 unknown :-1 s=CLOSED pgs=1127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:43:57.460 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:57.459+0000 7f5d0be00700 1 -- 172.21.15.18:0/3323951084 >> 172.21.15.18:0/3323951084 conn(0x7f5d0c0fab10 msgr2=0x7f5d0c0fd940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:43:57.460 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:57.459+0000 7f5d0be00700 1 -- 172.21.15.18:0/3323951084 shutdown_connections 2024-04-12T21:43:57.460 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:43:57.459+0000 7f5d0be00700 1 -- 172.21.15.18:0/3323951084 wait complete. 2024-04-12T21:43:58.105 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:43:57 smithi018 ceph-mon[112590]: pgmap v1328: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:43:58.105 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:43:57 smithi018 ceph-mon[112590]: from='client.36320 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:43:58.105 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:43:57 smithi018 ceph-mon[112590]: from='client.? 172.21.15.18:0/3323951084' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-12T21:43:58.284 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:43:57 smithi186 ceph-mon[86379]: pgmap v1328: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:43:58.284 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:43:57 smithi186 ceph-mon[86379]: from='client.36320 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:43:58.284 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:43:57 smithi186 ceph-mon[86379]: from='client.? 172.21.15.18:0/3323951084' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-12T21:44:00.104 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:43:59 smithi018 ceph-mon[112590]: pgmap v1329: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:44:00.284 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:43:59 smithi186 ceph-mon[86379]: pgmap v1329: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:44:02.105 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:44:01 smithi018 ceph-mon[112590]: pgmap v1330: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:44:02.284 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:44:01 smithi186 ceph-mon[86379]: pgmap v1330: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:44:04.105 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:44:03 smithi018 ceph-mon[112590]: pgmap v1331: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:44:04.284 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:44:03 smithi186 ceph-mon[86379]: pgmap v1331: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:44:06.104 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:44:05 smithi018 ceph-mon[112590]: pgmap v1332: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:44:06.283 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:44:05 smithi186 ceph-mon[86379]: pgmap v1332: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:44:08.104 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:44:07 smithi018 ceph-mon[112590]: pgmap v1333: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:44:08.284 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:44:07 smithi186 ceph-mon[86379]: pgmap v1333: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:44:10.105 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:44:09 smithi018 ceph-mon[112590]: pgmap v1334: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:44:10.284 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:44:09 smithi186 ceph-mon[86379]: pgmap v1334: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:44:11.105 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:44:10 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T21:44:11.284 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:44:10 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T21:44:12.104 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:44:11 smithi018 ceph-mon[112590]: pgmap v1335: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:44:12.284 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:44:11 smithi186 ceph-mon[86379]: pgmap v1335: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:44:14.105 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:44:13 smithi018 ceph-mon[112590]: pgmap v1336: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:44:14.283 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:44:13 smithi186 ceph-mon[86379]: pgmap v1336: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:44:16.284 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:44:15 smithi186 ceph-mon[86379]: pgmap v1337: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:44:16.355 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:44:15 smithi018 ceph-mon[112590]: pgmap v1337: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:44:18.284 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:44:17 smithi186 ceph-mon[86379]: pgmap v1338: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:44:18.355 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:44:17 smithi018 ceph-mon[112590]: pgmap v1338: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:44:20.284 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:44:19 smithi186 ceph-mon[86379]: pgmap v1339: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:44:20.354 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:44:19 smithi018 ceph-mon[112590]: pgmap v1339: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:44:22.284 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:44:21 smithi186 ceph-mon[86379]: pgmap v1340: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:44:22.355 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:44:21 smithi018 ceph-mon[112590]: pgmap v1340: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:44:24.284 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:44:23 smithi186 ceph-mon[86379]: pgmap v1341: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:44:24.354 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:44:23 smithi018 ceph-mon[112590]: pgmap v1341: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:44:26.284 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:44:25 smithi186 ceph-mon[86379]: pgmap v1342: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:44:26.284 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:44:25 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T21:44:26.355 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:44:25 smithi018 ceph-mon[112590]: pgmap v1342: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:44:26.355 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:44:25 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T21:44:27.632 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:44:27.630+0000 7fdc14a00700 1 -- 172.21.15.18:0/2782243326 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fdc10071f90 msgr2=0x7fdc10070360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:44:27.632 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:44:27.630+0000 7fdc14a00700 1 --2- 172.21.15.18:0/2782243326 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fdc10071f90 0x7fdc10070360 secure :-1 s=READY pgs=1128 cs=0 l=1 rev1=1 crypto rx=0x7fdc00004970 tx=0x7fdc00009e30 comp rx=0 tx=0).stop 2024-04-12T21:44:27.632 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:44:27.630+0000 7fdc14a00700 1 -- 172.21.15.18:0/2782243326 shutdown_connections 2024-04-12T21:44:27.632 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:44:27.630+0000 7fdc14a00700 1 --2- 172.21.15.18:0/2782243326 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdc100708a0 0x7fdc10070d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:44:27.632 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:44:27.630+0000 7fdc14a00700 1 --2- 172.21.15.18:0/2782243326 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fdc10071f90 0x7fdc10070360 unknown :-1 s=CLOSED pgs=1128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:44:27.632 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:44:27.630+0000 7fdc14a00700 1 -- 172.21.15.18:0/2782243326 >> 172.21.15.18:0/2782243326 conn(0x7fdc100f89b0 msgr2=0x7fdc100fadc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:44:27.632 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:44:27.630+0000 7fdc14a00700 1 -- 172.21.15.18:0/2782243326 shutdown_connections 2024-04-12T21:44:27.633 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:44:27.630+0000 7fdc14a00700 1 -- 172.21.15.18:0/2782243326 wait complete. 2024-04-12T21:44:27.634 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:44:27.631+0000 7fdc14a00700 1 Processor -- start 2024-04-12T21:44:27.634 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:44:27.631+0000 7fdc14a00700 1 -- start start 2024-04-12T21:44:27.635 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:44:27.632+0000 7fdc14a00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fdc100708a0 0x7fdc1011a1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:44:27.635 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:44:27.632+0000 7fdc14a00700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdc10071f90 0x7fdc1011a6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:44:27.635 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:44:27.632+0000 7fdc14a00700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc1011ad30 con 0x7fdc100708a0 2024-04-12T21:44:27.635 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:44:27.632+0000 7fdc14a00700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc10114100 con 0x7fdc10071f90 2024-04-12T21:44:27.635 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:44:27.632+0000 7fdc0de00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fdc100708a0 0x7fdc1011a1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:44:27.635 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:44:27.632+0000 7fdc0de00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fdc100708a0 0x7fdc1011a1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:54058/0 (socket says 172.21.15.18:54058) 2024-04-12T21:44:27.635 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:44:27.632+0000 7fdc0de00700 1 -- 172.21.15.18:0/930141814 learned_addr learned my addr 172.21.15.18:0/930141814 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:44:27.635 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:44:27.632+0000 7fdc0d400700 1 --2- 172.21.15.18:0/930141814 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdc10071f90 0x7fdc1011a6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:44:27.635 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:44:27.632+0000 7fdc0de00700 1 -- 172.21.15.18:0/930141814 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdc10071f90 msgr2=0x7fdc1011a6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:44:27.635 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:44:27.632+0000 7fdc0de00700 1 --2- 172.21.15.18:0/930141814 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdc10071f90 0x7fdc1011a6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:44:27.636 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:44:27.632+0000 7fdc0de00700 1 -- 172.21.15.18:0/930141814 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc00005040 con 0x7fdc100708a0 2024-04-12T21:44:27.636 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:44:27.632+0000 7fdc0d400700 1 --2- 172.21.15.18:0/930141814 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdc10071f90 0x7fdc1011a6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-12T21:44:27.636 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:44:27.633+0000 7fdc0de00700 1 --2- 172.21.15.18:0/930141814 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fdc100708a0 0x7fdc1011a1a0 secure :-1 s=READY pgs=1129 cs=0 l=1 rev1=1 crypto rx=0x7fdc00004f50 tx=0x7fdc00009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:44:27.636 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:44:27.633+0000 7fdc06000700 1 -- 172.21.15.18:0/930141814 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdc0000e040 con 0x7fdc100708a0 2024-04-12T21:44:27.636 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:44:27.633+0000 7fdc14a00700 1 -- 172.21.15.18:0/930141814 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdc10114380 con 0x7fdc100708a0 2024-04-12T21:44:27.636 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:44:27.633+0000 7fdc06000700 1 -- 172.21.15.18:0/930141814 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fdc0000bc10 con 0x7fdc100708a0 2024-04-12T21:44:27.636 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:44:27.633+0000 7fdc14a00700 1 -- 172.21.15.18:0/930141814 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdc101148c0 con 0x7fdc100708a0 2024-04-12T21:44:27.636 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:44:27.633+0000 7fdc06000700 1 -- 172.21.15.18:0/930141814 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdc000255e0 con 0x7fdc100708a0 2024-04-12T21:44:27.637 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:44:27.634+0000 7fdc14a00700 1 -- 172.21.15.18:0/930141814 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdc10071aa0 con 0x7fdc100708a0 2024-04-12T21:44:27.637 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:44:27.635+0000 7fdc06000700 1 -- 172.21.15.18:0/930141814 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7fdc00013420 con 0x7fdc100708a0 2024-04-12T21:44:27.638 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:44:27.636+0000 7fdc06000700 1 --2- 172.21.15.18:0/930141814 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fdbf406ec10 0x7fdbf40710c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:44:27.638 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:44:27.637+0000 7fdc06000700 1 -- 172.21.15.18:0/930141814 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 6757+0+0 (secure 0 0 0) 0x7fdc0008fd60 con 0x7fdc100708a0 2024-04-12T21:44:27.638 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:44:27.637+0000 7fdc0d400700 1 --2- 172.21.15.18:0/930141814 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fdbf406ec10 0x7fdbf40710c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:44:27.639 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:44:27.637+0000 7fdc0d400700 1 --2- 172.21.15.18:0/930141814 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fdbf406ec10 0x7fdbf40710c0 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7fdbfc009fd0 tx=0x7fdbfc00b040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:44:27.645 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:44:27.644+0000 7fdc06000700 1 -- 172.21.15.18:0/930141814 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7fdc0005c5b0 con 0x7fdc100708a0 2024-04-12T21:44:27.903 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:44:27.901+0000 7fdc14a00700 1 -- 172.21.15.18:0/930141814 --> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fdc1004bf40 con 0x7fdbf406ec10 2024-04-12T21:44:27.905 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:44:27.904+0000 7fdc06000700 1 -- 172.21.15.18:0/930141814 <== mgr.34102 v2:172.21.15.18:6800/3812891919 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7fdc1004bf40 con 0x7fdbf406ec10 2024-04-12T21:44:27.909 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:44:27.907+0000 7fdc14a00700 1 -- 172.21.15.18:0/930141814 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fdbf406ec10 msgr2=0x7fdbf40710c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:44:27.909 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:44:27.907+0000 7fdc14a00700 1 --2- 172.21.15.18:0/930141814 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fdbf406ec10 0x7fdbf40710c0 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7fdbfc009fd0 tx=0x7fdbfc00b040 comp rx=0 tx=0).stop 2024-04-12T21:44:27.910 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:44:27.907+0000 7fdc14a00700 1 -- 172.21.15.18:0/930141814 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fdc100708a0 msgr2=0x7fdc1011a1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:44:27.910 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:44:27.907+0000 7fdc14a00700 1 --2- 172.21.15.18:0/930141814 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fdc100708a0 0x7fdc1011a1a0 secure :-1 s=READY pgs=1129 cs=0 l=1 rev1=1 crypto rx=0x7fdc00004f50 tx=0x7fdc00009e30 comp rx=0 tx=0).stop 2024-04-12T21:44:27.910 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:44:27.907+0000 7fdc14a00700 1 -- 172.21.15.18:0/930141814 shutdown_connections 2024-04-12T21:44:27.910 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:44:27.907+0000 7fdc14a00700 1 --2- 172.21.15.18:0/930141814 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fdbf406ec10 0x7fdbf40710c0 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:44:27.910 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:44:27.907+0000 7fdc14a00700 1 --2- 172.21.15.18:0/930141814 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fdc100708a0 0x7fdc1011a1a0 unknown :-1 s=CLOSED pgs=1129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:44:27.910 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:44:27.907+0000 7fdc14a00700 1 --2- 172.21.15.18:0/930141814 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdc10071f90 0x7fdc1011a6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:44:27.911 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:44:27.907+0000 7fdc14a00700 1 -- 172.21.15.18:0/930141814 >> 172.21.15.18:0/930141814 conn(0x7fdc100f89b0 msgr2=0x7fdc100f9aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:44:27.911 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:44:27.907+0000 7fdc14a00700 1 -- 172.21.15.18:0/930141814 shutdown_connections 2024-04-12T21:44:27.911 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:44:27.907+0000 7fdc14a00700 1 -- 172.21.15.18:0/930141814 wait complete. 2024-04-12T21:44:27.925 INFO:teuthology.orchestra.run.smithi018.stdout:true 2024-04-12T21:44:28.067 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:44:28.062+0000 7f8a16400700 1 -- 172.21.15.18:0/3309206248 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8a10070360 msgr2=0x7f8a10070730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:44:28.067 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:44:28.062+0000 7f8a16400700 1 --2- 172.21.15.18:0/3309206248 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8a10070360 0x7f8a10070730 secure :-1 s=READY pgs=1130 cs=0 l=1 rev1=1 crypto rx=0x7f8a00009d10 tx=0x7f8a00004d10 comp rx=0 tx=0).stop 2024-04-12T21:44:28.067 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:44:28.063+0000 7f8a16400700 1 -- 172.21.15.18:0/3309206248 shutdown_connections 2024-04-12T21:44:28.067 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:44:28.063+0000 7f8a16400700 1 --2- 172.21.15.18:0/3309206248 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8a10070c70 0x7f8a10109110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:44:28.067 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:44:28.063+0000 7f8a16400700 1 --2- 172.21.15.18:0/3309206248 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8a10070360 0x7f8a10070730 unknown :-1 s=CLOSED pgs=1130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:44:28.067 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:44:28.063+0000 7f8a16400700 1 -- 172.21.15.18:0/3309206248 >> 172.21.15.18:0/3309206248 conn(0x7f8a100f89b0 msgr2=0x7f8a100fadc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:44:28.068 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:44:28.063+0000 7f8a16400700 1 -- 172.21.15.18:0/3309206248 shutdown_connections 2024-04-12T21:44:28.068 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:44:28.063+0000 7f8a16400700 1 -- 172.21.15.18:0/3309206248 wait complete. 2024-04-12T21:44:28.068 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:44:28.063+0000 7f8a16400700 1 Processor -- start 2024-04-12T21:44:28.068 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:44:28.064+0000 7f8a16400700 1 -- start start 2024-04-12T21:44:28.068 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:44:28.064+0000 7f8a16400700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8a10070360 0x7f8a10115fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:44:28.068 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:44:28.064+0000 7f8a16400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8a10070c70 0x7f8a10116500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:44:28.068 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:44:28.064+0000 7f8a16400700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a10116be0 con 0x7f8a10070c70 2024-04-12T21:44:28.068 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:44:28.064+0000 7f8a16400700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a1010ff20 con 0x7f8a10070360 2024-04-12T21:44:28.068 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:44:28.064+0000 7f8a0ea00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8a10070c70 0x7f8a10116500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:44:28.068 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:44:28.065+0000 7f8a0ea00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8a10070c70 0x7f8a10116500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:54074/0 (socket says 172.21.15.18:54074) 2024-04-12T21:44:28.068 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:44:28.065+0000 7f8a0ea00700 1 -- 172.21.15.18:0/4119299892 learned_addr learned my addr 172.21.15.18:0/4119299892 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:44:28.068 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:44:28.065+0000 7f8a0f400700 1 --2- 172.21.15.18:0/4119299892 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8a10070360 0x7f8a10115fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:44:28.068 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:44:28.065+0000 7f8a0ea00700 1 -- 172.21.15.18:0/4119299892 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8a10070360 msgr2=0x7f8a10115fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:44:28.069 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:44:28.065+0000 7f8a0ea00700 1 --2- 172.21.15.18:0/4119299892 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8a10070360 0x7f8a10115fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:44:28.069 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:44:28.065+0000 7f8a0ea00700 1 -- 172.21.15.18:0/4119299892 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a00009940 con 0x7f8a10070c70 2024-04-12T21:44:28.069 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:44:28.065+0000 7f8a0ea00700 1 --2- 172.21.15.18:0/4119299892 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8a10070c70 0x7f8a10116500 secure :-1 s=READY pgs=1131 cs=0 l=1 rev1=1 crypto rx=0x7f89fc00b790 tx=0x7f89fc009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:44:28.069 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:44:28.066+0000 7f8a07400700 1 -- 172.21.15.18:0/4119299892 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f89fc00f770 con 0x7f8a10070c70 2024-04-12T21:44:28.069 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:44:28.066+0000 7f8a07400700 1 -- 172.21.15.18:0/4119299892 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f89fc00fda0 con 0x7f8a10070c70 2024-04-12T21:44:28.069 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:44:28.066+0000 7f8a16400700 1 -- 172.21.15.18:0/4119299892 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8a101101a0 con 0x7f8a10070c70 2024-04-12T21:44:28.069 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:44:28.066+0000 7f8a07400700 1 -- 172.21.15.18:0/4119299892 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f89fc00d310 con 0x7f8a10070c70 2024-04-12T21:44:28.069 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:44:28.066+0000 7f8a16400700 1 -- 172.21.15.18:0/4119299892 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8a101106e0 con 0x7f8a10070c70 2024-04-12T21:44:28.069 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:44:28.067+0000 7f8a16400700 1 -- 172.21.15.18:0/4119299892 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8a10071aa0 con 0x7f8a10070c70 2024-04-12T21:44:28.070 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:44:28.068+0000 7f8a07400700 1 -- 172.21.15.18:0/4119299892 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7f89fc010420 con 0x7f8a10070c70 2024-04-12T21:44:28.071 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:44:28.069+0000 7f8a07400700 1 --2- 172.21.15.18:0/4119299892 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f89f406ec10 0x7f89f40710c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:44:28.071 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:44:28.069+0000 7f8a07400700 1 -- 172.21.15.18:0/4119299892 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 6757+0+0 (secure 0 0 0) 0x7f89fc08e9e0 con 0x7f8a10070c70 2024-04-12T21:44:28.071 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:44:28.070+0000 7f8a0f400700 1 --2- 172.21.15.18:0/4119299892 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f89f406ec10 0x7f89f40710c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:44:28.071 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:44:28.070+0000 7f8a0f400700 1 --2- 172.21.15.18:0/4119299892 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f89f406ec10 0x7f89f40710c0 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f8a00015040 tx=0x7f8a00004d10 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:44:28.077 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:44:28.074+0000 7f8a07400700 1 -- 172.21.15.18:0/4119299892 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f89fc05b230 con 0x7f8a10070c70 2024-04-12T21:44:28.284 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:44:27 smithi186 ceph-mon[86379]: pgmap v1343: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:44:28.330 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:44:28.328+0000 7f8a16400700 1 -- 172.21.15.18:0/4119299892 --> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f8a1004bed0 con 0x7f89f406ec10 2024-04-12T21:44:28.330 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:44:27 smithi018 ceph-mon[112590]: pgmap v1343: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:44:28.334 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:44:28.333+0000 7f8a07400700 1 -- 172.21.15.18:0/4119299892 <== mgr.34102 v2:172.21.15.18:6800/3812891919 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f8a1004bed0 con 0x7f89f406ec10 2024-04-12T21:44:28.337 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:44:28.335+0000 7f8a16400700 1 -- 172.21.15.18:0/4119299892 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f89f406ec10 msgr2=0x7f89f40710c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:44:28.337 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:44:28.335+0000 7f8a16400700 1 --2- 172.21.15.18:0/4119299892 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f89f406ec10 0x7f89f40710c0 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f8a00015040 tx=0x7f8a00004d10 comp rx=0 tx=0).stop 2024-04-12T21:44:28.337 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:44:28.336+0000 7f8a16400700 1 -- 172.21.15.18:0/4119299892 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8a10070c70 msgr2=0x7f8a10116500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:44:28.337 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:44:28.336+0000 7f8a16400700 1 --2- 172.21.15.18:0/4119299892 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8a10070c70 0x7f8a10116500 secure :-1 s=READY pgs=1131 cs=0 l=1 rev1=1 crypto rx=0x7f89fc00b790 tx=0x7f89fc009250 comp rx=0 tx=0).stop 2024-04-12T21:44:28.338 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:44:28.336+0000 7f8a16400700 1 -- 172.21.15.18:0/4119299892 shutdown_connections 2024-04-12T21:44:28.338 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:44:28.336+0000 7f8a16400700 1 --2- 172.21.15.18:0/4119299892 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f89f406ec10 0x7f89f40710c0 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:44:28.338 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:44:28.336+0000 7f8a16400700 1 --2- 172.21.15.18:0/4119299892 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8a10070360 0x7f8a10115fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:44:28.338 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:44:28.336+0000 7f8a16400700 1 --2- 172.21.15.18:0/4119299892 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8a10070c70 0x7f8a10116500 unknown :-1 s=CLOSED pgs=1131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:44:28.338 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:44:28.336+0000 7f8a16400700 1 -- 172.21.15.18:0/4119299892 >> 172.21.15.18:0/4119299892 conn(0x7f8a100f89b0 msgr2=0x7f8a100f9ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:44:28.338 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:44:28.336+0000 7f8a16400700 1 -- 172.21.15.18:0/4119299892 shutdown_connections 2024-04-12T21:44:28.339 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:44:28.336+0000 7f8a16400700 1 -- 172.21.15.18:0/4119299892 wait complete. 2024-04-12T21:44:28.508 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:44:28.507+0000 7f80a7e00700 1 -- 172.21.15.18:0/3821188911 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f80a0109430 msgr2=0x7f80a0109800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:44:28.509 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:44:28.507+0000 7f80a7e00700 1 --2- 172.21.15.18:0/3821188911 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f80a0109430 0x7f80a0109800 secure :-1 s=READY pgs=1132 cs=0 l=1 rev1=1 crypto rx=0x7f8094004940 tx=0x7f8094009e30 comp rx=0 tx=0).stop 2024-04-12T21:44:28.509 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:44:28.507+0000 7f80a7e00700 1 -- 172.21.15.18:0/3821188911 shutdown_connections 2024-04-12T21:44:28.509 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:44:28.507+0000 7f80a7e00700 1 --2- 172.21.15.18:0/3821188911 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f80a0109dd0 0x7f80a0103440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:44:28.509 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:44:28.507+0000 7f80a7e00700 1 --2- 172.21.15.18:0/3821188911 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f80a0109430 0x7f80a0109800 unknown :-1 s=CLOSED pgs=1132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:44:28.509 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:44:28.507+0000 7f80a7e00700 1 -- 172.21.15.18:0/3821188911 >> 172.21.15.18:0/3821188911 conn(0x7f80a0072c00 msgr2=0x7f80a0075010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:44:28.509 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:44:28.507+0000 7f80a7e00700 1 -- 172.21.15.18:0/3821188911 shutdown_connections 2024-04-12T21:44:28.509 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:44:28.507+0000 7f80a7e00700 1 -- 172.21.15.18:0/3821188911 wait complete. 2024-04-12T21:44:28.510 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:44:28.508+0000 7f80a7e00700 1 Processor -- start 2024-04-12T21:44:28.510 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:44:28.508+0000 7f80a7e00700 1 -- start start 2024-04-12T21:44:28.510 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:44:28.509+0000 7f80a7e00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f80a0109430 0x7f80a0119f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:44:28.510 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:44:28.509+0000 7f80a7e00700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f80a0109dd0 0x7f80a011a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:44:28.510 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:44:28.509+0000 7f80a7e00700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80a011aad0 con 0x7f80a0109430 2024-04-12T21:44:28.510 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:44:28.509+0000 7f80a7e00700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80a0114100 con 0x7f80a0109dd0 2024-04-12T21:44:28.510 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:44:28.509+0000 7f80a5400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f80a0109430 0x7f80a0119f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:44:28.511 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:44:28.509+0000 7f80a5400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f80a0109430 0x7f80a0119f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:54100/0 (socket says 172.21.15.18:54100) 2024-04-12T21:44:28.511 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:44:28.509+0000 7f80a5400700 1 -- 172.21.15.18:0/2522344091 learned_addr learned my addr 172.21.15.18:0/2522344091 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:44:28.512 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:44:28.509+0000 7f80a4a00700 1 --2- 172.21.15.18:0/2522344091 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f80a0109dd0 0x7f80a011a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:44:28.512 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:44:28.509+0000 7f80a5400700 1 -- 172.21.15.18:0/2522344091 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f80a0109dd0 msgr2=0x7f80a011a480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:44:28.512 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:44:28.509+0000 7f80a5400700 1 --2- 172.21.15.18:0/2522344091 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f80a0109dd0 0x7f80a011a480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:44:28.512 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:44:28.509+0000 7f80a5400700 1 -- 172.21.15.18:0/2522344091 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8094005040 con 0x7f80a0109430 2024-04-12T21:44:28.512 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:44:28.510+0000 7f80a5400700 1 --2- 172.21.15.18:0/2522344091 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f80a0109430 0x7f80a0119f40 secure :-1 s=READY pgs=1133 cs=0 l=1 rev1=1 crypto rx=0x7f8094009150 tx=0x7f8094009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:44:28.512 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:44:28.510+0000 7f809d600700 1 -- 172.21.15.18:0/2522344091 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f809400e040 con 0x7f80a0109430 2024-04-12T21:44:28.512 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:44:28.510+0000 7f809d600700 1 -- 172.21.15.18:0/2522344091 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f8094013420 con 0x7f80a0109430 2024-04-12T21:44:28.512 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:44:28.510+0000 7f80a7e00700 1 -- 172.21.15.18:0/2522344091 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f80a0114380 con 0x7f80a0109430 2024-04-12T21:44:28.512 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:44:28.510+0000 7f809d600700 1 -- 172.21.15.18:0/2522344091 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f809401c630 con 0x7f80a0109430 2024-04-12T21:44:28.512 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:44:28.510+0000 7f80a7e00700 1 -- 172.21.15.18:0/2522344091 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f80a0114940 con 0x7f80a0109430 2024-04-12T21:44:28.513 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:44:28.512+0000 7f80a7e00700 1 -- 172.21.15.18:0/2522344091 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f80a0104b80 con 0x7f80a0109430 2024-04-12T21:44:28.514 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:44:28.513+0000 7f809d600700 1 -- 172.21.15.18:0/2522344091 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7f8094013590 con 0x7f80a0109430 2024-04-12T21:44:28.515 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:44:28.514+0000 7f809d600700 1 --2- 172.21.15.18:0/2522344091 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f808806ec00 0x7f80880710b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:44:28.515 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:44:28.514+0000 7f809d600700 1 -- 172.21.15.18:0/2522344091 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 6757+0+0 (secure 0 0 0) 0x7f809408fce0 con 0x7f80a0109430 2024-04-12T21:44:28.515 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:44:28.514+0000 7f80a4a00700 1 --2- 172.21.15.18:0/2522344091 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f808806ec00 0x7f80880710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:44:28.516 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:44:28.515+0000 7f80a4a00700 1 --2- 172.21.15.18:0/2522344091 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f808806ec00 0x7f80880710b0 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f80a01157e0 tx=0x7f808c009ee0 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:44:28.520 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:44:28.519+0000 7f809d600700 1 -- 172.21.15.18:0/2522344091 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f809405c490 con 0x7f80a0109430 2024-04-12T21:44:28.767 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:44:28.765+0000 7f80a7e00700 1 -- 172.21.15.18:0/2522344091 --> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f80a0115600 con 0x7f808806ec00 2024-04-12T21:44:28.779 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:44:28.777+0000 7f809d600700 1 -- 172.21.15.18:0/2522344091 <== mgr.34102 v2:172.21.15.18:6800/3812891919 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7f80a0115600 con 0x7f808806ec00 2024-04-12T21:44:28.779 INFO:teuthology.orchestra.run.smithi018.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-12T21:44:28.779 INFO:teuthology.orchestra.run.smithi018.stdout:alertmanager.smithi018 smithi018 *:9093,9094 running (45m) 6m ago 59m 14.8M - 0.25.0 c8568f914cd2 c332090068e8 2024-04-12T21:44:28.779 INFO:teuthology.orchestra.run.smithi018.stdout:crash.smithi018 smithi018 running (41m) 6m ago 59m 6665k - 19.0.0-2879-g324baf8d 8717720dc5e5 b7f80d1f7982 2024-04-12T21:44:28.779 INFO:teuthology.orchestra.run.smithi018.stdout:crash.smithi186 smithi186 running (41m) 6m ago 58m 6660k - 19.0.0-2879-g324baf8d 8717720dc5e5 a4f55c81b623 2024-04-12T21:44:28.779 INFO:teuthology.orchestra.run.smithi018.stdout:grafana.smithi018 smithi018 *:3000 running (44m) 6m ago 59m 73.8M - 9.4.12 2bacad6d85d8 b08e3bf176dd 2024-04-12T21:44:28.779 INFO:teuthology.orchestra.run.smithi018.stdout:mds.cephfs.smithi018.acqbsc smithi018 running (37m) 6m ago 52m 21.9M - 19.0.0-2879-g324baf8d 8717720dc5e5 021f99ab24ae 2024-04-12T21:44:28.780 INFO:teuthology.orchestra.run.smithi018.stdout:mds.cephfs.smithi018.szjluo smithi018 running (36m) 6m ago 52m 21.7M - 19.0.0-2879-g324baf8d 8717720dc5e5 0003117479d4 2024-04-12T21:44:28.780 INFO:teuthology.orchestra.run.smithi018.stdout:mds.cephfs.smithi186.jdllyn smithi186 running (21m) 6m ago 52m 16.5M - 19.0.0-2879-g324baf8d 8717720dc5e5 21b1f46eef23 2024-04-12T21:44:28.780 INFO:teuthology.orchestra.run.smithi018.stdout:mds.cephfs.smithi186.wjnafs smithi186 running (6m) 6m ago 52m 11.0M - 19.0.0-2879-g324baf8d 8717720dc5e5 3cf1ef4148e2 2024-04-12T21:44:28.780 INFO:teuthology.orchestra.run.smithi018.stdout:mgr.smithi018.xjzwga smithi018 *:8443,9283,8765 running (47m) 6m ago 61m 517M - 19.0.0-2879-g324baf8d 8717720dc5e5 56088305eab7 2024-04-12T21:44:28.780 INFO:teuthology.orchestra.run.smithi018.stdout:mgr.smithi186.kqcfxk smithi186 *:8443,9283,8765 running (46m) 6m ago 58m 460M - 19.0.0-2879-g324baf8d 8717720dc5e5 09cd766e5e0d 2024-04-12T21:44:28.780 INFO:teuthology.orchestra.run.smithi018.stdout:mon.smithi018 smithi018 running (42m) 6m ago 61m 88.2M 2048M 19.0.0-2879-g324baf8d 8717720dc5e5 df3e201dfed6 2024-04-12T21:44:28.780 INFO:teuthology.orchestra.run.smithi018.stdout:mon.smithi186 smithi186 running (42m) 6m ago 58m 53.6M 2048M 19.0.0-2879-g324baf8d 8717720dc5e5 c77498da9b2b 2024-04-12T21:44:28.780 INFO:teuthology.orchestra.run.smithi018.stdout:node-exporter.smithi018 smithi018 *:9100 running (46m) 6m ago 59m 8468k - 1.5.0 0da6a335fe13 326c3d9c6847 2024-04-12T21:44:28.780 INFO:teuthology.orchestra.run.smithi018.stdout:node-exporter.smithi186 smithi186 *:9100 running (46m) 6m ago 58m 8166k - 1.5.0 0da6a335fe13 ec2112baeecf 2024-04-12T21:44:28.780 INFO:teuthology.orchestra.run.smithi018.stdout:osd.0 smithi018 running (40m) 6m ago 57m 123M 1559M 19.0.0-2879-g324baf8d 8717720dc5e5 60b1273ca047 2024-04-12T21:44:28.780 INFO:teuthology.orchestra.run.smithi018.stdout:osd.1 smithi018 running (40m) 6m ago 56m 95.4M 1559M 19.0.0-2879-g324baf8d 8717720dc5e5 170ce1e5fb8d 2024-04-12T21:44:28.780 INFO:teuthology.orchestra.run.smithi018.stdout:osd.2 smithi018 running (39m) 6m ago 55m 94.6M 1559M 19.0.0-2879-g324baf8d 8717720dc5e5 c2cbc6aa2839 2024-04-12T21:44:28.780 INFO:teuthology.orchestra.run.smithi018.stdout:osd.3 smithi186 running (39m) 6m ago 55m 101M 2581M 19.0.0-2879-g324baf8d 8717720dc5e5 0138e8262e4b 2024-04-12T21:44:28.780 INFO:teuthology.orchestra.run.smithi018.stdout:osd.4 smithi186 running (38m) 6m ago 54m 91.5M 2581M 19.0.0-2879-g324baf8d 8717720dc5e5 b20cb63b25c9 2024-04-12T21:44:28.780 INFO:teuthology.orchestra.run.smithi018.stdout:osd.5 smithi186 running (37m) 6m ago 54m 92.2M 2581M 19.0.0-2879-g324baf8d 8717720dc5e5 2513bea2610e 2024-04-12T21:44:28.781 INFO:teuthology.orchestra.run.smithi018.stdout:prometheus.smithi018 smithi018 *:9095 running (45m) 6m ago 58m 69.6M - 2.43.0 a07b618ecd1d f58841127e21 2024-04-12T21:44:28.782 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:44:28.780+0000 7f80a7e00700 1 -- 172.21.15.18:0/2522344091 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f808806ec00 msgr2=0x7f80880710b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:44:28.782 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:44:28.780+0000 7f80a7e00700 1 --2- 172.21.15.18:0/2522344091 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f808806ec00 0x7f80880710b0 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f80a01157e0 tx=0x7f808c009ee0 comp rx=0 tx=0).stop 2024-04-12T21:44:28.782 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:44:28.780+0000 7f80a7e00700 1 -- 172.21.15.18:0/2522344091 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f80a0109430 msgr2=0x7f80a0119f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:44:28.782 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:44:28.780+0000 7f80a7e00700 1 --2- 172.21.15.18:0/2522344091 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f80a0109430 0x7f80a0119f40 secure :-1 s=READY pgs=1133 cs=0 l=1 rev1=1 crypto rx=0x7f8094009150 tx=0x7f8094009e30 comp rx=0 tx=0).stop 2024-04-12T21:44:28.782 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:44:28.780+0000 7f80a7e00700 1 -- 172.21.15.18:0/2522344091 shutdown_connections 2024-04-12T21:44:28.782 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:44:28.780+0000 7f80a7e00700 1 --2- 172.21.15.18:0/2522344091 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f808806ec00 0x7f80880710b0 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:44:28.782 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:44:28.780+0000 7f80a7e00700 1 --2- 172.21.15.18:0/2522344091 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f80a0109430 0x7f80a0119f40 unknown :-1 s=CLOSED pgs=1133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:44:28.782 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:44:28.780+0000 7f80a7e00700 1 --2- 172.21.15.18:0/2522344091 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f80a0109dd0 0x7f80a011a480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:44:28.782 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:44:28.780+0000 7f80a7e00700 1 -- 172.21.15.18:0/2522344091 >> 172.21.15.18:0/2522344091 conn(0x7f80a0072c00 msgr2=0x7f80a0074c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:44:28.782 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:44:28.781+0000 7f80a7e00700 1 -- 172.21.15.18:0/2522344091 shutdown_connections 2024-04-12T21:44:28.783 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:44:28.781+0000 7f80a7e00700 1 -- 172.21.15.18:0/2522344091 wait complete. 2024-04-12T21:44:28.943 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:44:28.941+0000 7f3ba7400700 1 -- 172.21.15.18:0/1834349588 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3ba0109430 msgr2=0x7f3ba0109800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:44:28.943 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:44:28.941+0000 7f3ba7400700 1 --2- 172.21.15.18:0/1834349588 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3ba0109430 0x7f3ba0109800 secure :-1 s=READY pgs=1134 cs=0 l=1 rev1=1 crypto rx=0x7f3b94004940 tx=0x7f3b94009e30 comp rx=0 tx=0).stop 2024-04-12T21:44:28.943 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:44:28.941+0000 7f3ba7400700 1 -- 172.21.15.18:0/1834349588 shutdown_connections 2024-04-12T21:44:28.943 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:44:28.941+0000 7f3ba7400700 1 --2- 172.21.15.18:0/1834349588 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3ba0109dd0 0x7f3ba0103440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:44:28.944 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:44:28.941+0000 7f3ba7400700 1 --2- 172.21.15.18:0/1834349588 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3ba0109430 0x7f3ba0109800 unknown :-1 s=CLOSED pgs=1134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:44:28.944 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:44:28.941+0000 7f3ba7400700 1 -- 172.21.15.18:0/1834349588 >> 172.21.15.18:0/1834349588 conn(0x7f3ba0072c00 msgr2=0x7f3ba0075010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:44:28.944 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:44:28.942+0000 7f3ba7400700 1 -- 172.21.15.18:0/1834349588 shutdown_connections 2024-04-12T21:44:28.944 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:44:28.942+0000 7f3ba7400700 1 -- 172.21.15.18:0/1834349588 wait complete. 2024-04-12T21:44:28.945 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:44:28.942+0000 7f3ba7400700 1 Processor -- start 2024-04-12T21:44:28.945 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:44:28.943+0000 7f3ba7400700 1 -- start start 2024-04-12T21:44:28.945 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:44:28.943+0000 7f3ba7400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3ba0109430 0x7f3ba0119f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:44:28.945 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:44:28.943+0000 7f3ba7400700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3ba0109dd0 0x7f3ba011a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:44:28.945 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:44:28.943+0000 7f3ba7400700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ba011ab60 con 0x7f3ba0109430 2024-04-12T21:44:28.945 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:44:28.943+0000 7f3ba7400700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ba01140b0 con 0x7f3ba0109dd0 2024-04-12T21:44:28.945 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:44:28.943+0000 7f3ba4a00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3ba0109430 0x7f3ba0119f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:44:28.945 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:44:28.944+0000 7f3ba4a00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3ba0109430 0x7f3ba0119f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:54110/0 (socket says 172.21.15.18:54110) 2024-04-12T21:44:28.945 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:44:28.944+0000 7f3ba4a00700 1 -- 172.21.15.18:0/2122077784 learned_addr learned my addr 172.21.15.18:0/2122077784 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:44:28.947 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:44:28.944+0000 7f3b9fe00700 1 --2- 172.21.15.18:0/2122077784 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3ba0109dd0 0x7f3ba011a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:44:28.947 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:44:28.944+0000 7f3ba4a00700 1 -- 172.21.15.18:0/2122077784 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3ba0109dd0 msgr2=0x7f3ba011a480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:44:28.947 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:44:28.944+0000 7f3ba4a00700 1 --2- 172.21.15.18:0/2122077784 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3ba0109dd0 0x7f3ba011a480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:44:28.947 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:44:28.944+0000 7f3ba4a00700 1 -- 172.21.15.18:0/2122077784 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b94005040 con 0x7f3ba0109430 2024-04-12T21:44:28.947 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:44:28.944+0000 7f3b9fe00700 1 --2- 172.21.15.18:0/2122077784 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3ba0109dd0 0x7f3ba011a480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-12T21:44:28.947 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:44:28.944+0000 7f3ba4a00700 1 --2- 172.21.15.18:0/2122077784 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3ba0109430 0x7f3ba0119f40 secure :-1 s=READY pgs=1135 cs=0 l=1 rev1=1 crypto rx=0x7f3b94009180 tx=0x7f3b94009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:44:28.947 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:44:28.945+0000 7f3b9cc00700 1 -- 172.21.15.18:0/2122077784 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3b9400e040 con 0x7f3ba0109430 2024-04-12T21:44:28.947 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:44:28.945+0000 7f3b9cc00700 1 -- 172.21.15.18:0/2122077784 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f3b9400bbc0 con 0x7f3ba0109430 2024-04-12T21:44:28.947 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:44:28.945+0000 7f3ba7400700 1 -- 172.21.15.18:0/2122077784 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3ba01142d0 con 0x7f3ba0109430 2024-04-12T21:44:28.947 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:44:28.945+0000 7f3b9cc00700 1 -- 172.21.15.18:0/2122077784 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3b9401c630 con 0x7f3ba0109430 2024-04-12T21:44:28.947 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:44:28.945+0000 7f3ba7400700 1 -- 172.21.15.18:0/2122077784 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3ba0114890 con 0x7f3ba0109430 2024-04-12T21:44:28.948 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:44:28.946+0000 7f3ba7400700 1 -- 172.21.15.18:0/2122077784 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3ba0104b80 con 0x7f3ba0109430 2024-04-12T21:44:28.948 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:44:28.947+0000 7f3b9cc00700 1 -- 172.21.15.18:0/2122077784 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7f3b9400bd30 con 0x7f3ba0109430 2024-04-12T21:44:28.950 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:44:28.948+0000 7f3b9cc00700 1 --2- 172.21.15.18:0/2122077784 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f3b8806ec00 0x7f3b880710b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:44:28.950 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:44:28.948+0000 7f3b9cc00700 1 -- 172.21.15.18:0/2122077784 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 6757+0+0 (secure 0 0 0) 0x7f3b9408fc80 con 0x7f3ba0109430 2024-04-12T21:44:28.950 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:44:28.948+0000 7f3b9fe00700 1 --2- 172.21.15.18:0/2122077784 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f3b8806ec00 0x7f3b880710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:44:28.950 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:44:28.949+0000 7f3b9fe00700 1 --2- 172.21.15.18:0/2122077784 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f3b8806ec00 0x7f3b880710b0 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f3b8c006fd0 tx=0x7f3b8c008040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:44:28.954 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:44:28.953+0000 7f3b9cc00700 1 -- 172.21.15.18:0/2122077784 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f3b9405c570 con 0x7f3ba0109430 2024-04-12T21:44:29.105 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:44:28 smithi018 ceph-mon[112590]: from='client.36328 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:44:29.274 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:44:29.272+0000 7f3ba7400700 1 -- 172.21.15.18:0/2122077784 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f3ba004c710 con 0x7f3ba0109430 2024-04-12T21:44:29.275 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:44:29.274+0000 7f3b9cc00700 1 -- 172.21.15.18:0/2122077784 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+623 (secure 0 0 0) 0x7f3b9405c100 con 0x7f3ba0109430 2024-04-12T21:44:29.276 INFO:teuthology.orchestra.run.smithi018.stdout:{ 2024-04-12T21:44:29.276 INFO:teuthology.orchestra.run.smithi018.stdout: "mon": { 2024-04-12T21:44:29.276 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 19.0.0-2879-g324baf8d (324baf8ddb91dea46cac50cb535c306115791355) squid (dev)": 2 2024-04-12T21:44:29.276 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-12T21:44:29.276 INFO:teuthology.orchestra.run.smithi018.stdout: "mgr": { 2024-04-12T21:44:29.276 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 19.0.0-2879-g324baf8d (324baf8ddb91dea46cac50cb535c306115791355) squid (dev)": 2 2024-04-12T21:44:29.276 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-12T21:44:29.276 INFO:teuthology.orchestra.run.smithi018.stdout: "osd": { 2024-04-12T21:44:29.276 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 19.0.0-2879-g324baf8d (324baf8ddb91dea46cac50cb535c306115791355) squid (dev)": 6 2024-04-12T21:44:29.276 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-12T21:44:29.276 INFO:teuthology.orchestra.run.smithi018.stdout: "mds": { 2024-04-12T21:44:29.276 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 19.0.0-2879-g324baf8d (324baf8ddb91dea46cac50cb535c306115791355) squid (dev)": 4 2024-04-12T21:44:29.276 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-12T21:44:29.276 INFO:teuthology.orchestra.run.smithi018.stdout: "overall": { 2024-04-12T21:44:29.277 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 19.0.0-2879-g324baf8d (324baf8ddb91dea46cac50cb535c306115791355) squid (dev)": 14 2024-04-12T21:44:29.277 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-04-12T21:44:29.277 INFO:teuthology.orchestra.run.smithi018.stdout:} 2024-04-12T21:44:29.278 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:44:29.277+0000 7f3ba7400700 1 -- 172.21.15.18:0/2122077784 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f3b8806ec00 msgr2=0x7f3b880710b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:44:29.279 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:44:29.277+0000 7f3ba7400700 1 --2- 172.21.15.18:0/2122077784 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f3b8806ec00 0x7f3b880710b0 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f3b8c006fd0 tx=0x7f3b8c008040 comp rx=0 tx=0).stop 2024-04-12T21:44:29.279 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:44:29.277+0000 7f3ba7400700 1 -- 172.21.15.18:0/2122077784 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3ba0109430 msgr2=0x7f3ba0119f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:44:29.279 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:44:29.277+0000 7f3ba7400700 1 --2- 172.21.15.18:0/2122077784 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3ba0109430 0x7f3ba0119f40 secure :-1 s=READY pgs=1135 cs=0 l=1 rev1=1 crypto rx=0x7f3b94009180 tx=0x7f3b94009e30 comp rx=0 tx=0).stop 2024-04-12T21:44:29.279 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:44:29.277+0000 7f3ba7400700 1 -- 172.21.15.18:0/2122077784 shutdown_connections 2024-04-12T21:44:29.279 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:44:29.277+0000 7f3ba7400700 1 --2- 172.21.15.18:0/2122077784 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f3b8806ec00 0x7f3b880710b0 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:44:29.279 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:44:29.277+0000 7f3ba7400700 1 --2- 172.21.15.18:0/2122077784 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3ba0109430 0x7f3ba0119f40 unknown :-1 s=CLOSED pgs=1135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:44:29.279 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:44:29.277+0000 7f3ba7400700 1 --2- 172.21.15.18:0/2122077784 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3ba0109dd0 0x7f3ba011a480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:44:29.279 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:44:29.277+0000 7f3ba7400700 1 -- 172.21.15.18:0/2122077784 >> 172.21.15.18:0/2122077784 conn(0x7f3ba0072c00 msgr2=0x7f3ba0074c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:44:29.279 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:44:29.277+0000 7f3ba7400700 1 -- 172.21.15.18:0/2122077784 shutdown_connections 2024-04-12T21:44:29.279 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:44:29.277+0000 7f3ba7400700 1 -- 172.21.15.18:0/2122077784 wait complete. 2024-04-12T21:44:29.284 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:44:28 smithi186 ceph-mon[86379]: from='client.36328 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:44:29.440 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:44:29.434+0000 7f0582600700 1 -- 172.21.15.18:0/1064636204 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f057c071f90 msgr2=0x7f057c070360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:44:29.440 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:44:29.434+0000 7f0582600700 1 --2- 172.21.15.18:0/1064636204 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f057c071f90 0x7f057c070360 secure :-1 s=READY pgs=1136 cs=0 l=1 rev1=1 crypto rx=0x7f056c004970 tx=0x7f056c009e30 comp rx=0 tx=0).stop 2024-04-12T21:44:29.441 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:44:29.434+0000 7f0582600700 1 -- 172.21.15.18:0/1064636204 shutdown_connections 2024-04-12T21:44:29.441 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:44:29.434+0000 7f0582600700 1 --2- 172.21.15.18:0/1064636204 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f057c0708a0 0x7f057c070d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:44:29.441 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:44:29.434+0000 7f0582600700 1 --2- 172.21.15.18:0/1064636204 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f057c071f90 0x7f057c070360 unknown :-1 s=CLOSED pgs=1136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:44:29.441 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:44:29.434+0000 7f0582600700 1 -- 172.21.15.18:0/1064636204 >> 172.21.15.18:0/1064636204 conn(0x7f057c0f8950 msgr2=0x7f057c0fad60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:44:29.441 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:44:29.434+0000 7f0582600700 1 -- 172.21.15.18:0/1064636204 shutdown_connections 2024-04-12T21:44:29.441 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:44:29.434+0000 7f0582600700 1 -- 172.21.15.18:0/1064636204 wait complete. 2024-04-12T21:44:29.441 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:44:29.435+0000 7f0582600700 1 Processor -- start 2024-04-12T21:44:29.441 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:44:29.435+0000 7f0582600700 1 -- start start 2024-04-12T21:44:29.441 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:44:29.436+0000 7f0582600700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f057c0708a0 0x7f057c119fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:44:29.441 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:44:29.436+0000 7f0582600700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f057c071f90 0x7f057c11a4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:44:29.441 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:44:29.436+0000 7f0582600700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f057c11ab40 con 0x7f057c071f90 2024-04-12T21:44:29.441 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:44:29.436+0000 7f0582600700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f057c114120 con 0x7f057c0708a0 2024-04-12T21:44:29.441 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:44:29.436+0000 7f057aa00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f057c071f90 0x7f057c11a4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:44:29.442 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:44:29.436+0000 7f057aa00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f057c071f90 0x7f057c11a4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:54142/0 (socket says 172.21.15.18:54142) 2024-04-12T21:44:29.442 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:44:29.436+0000 7f057aa00700 1 -- 172.21.15.18:0/1309615622 learned_addr learned my addr 172.21.15.18:0/1309615622 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:44:29.442 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:44:29.436+0000 7f057b400700 1 --2- 172.21.15.18:0/1309615622 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f057c0708a0 0x7f057c119fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:44:29.442 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:44:29.437+0000 7f057aa00700 1 -- 172.21.15.18:0/1309615622 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f057c0708a0 msgr2=0x7f057c119fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:44:29.442 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:44:29.437+0000 7f057aa00700 1 --2- 172.21.15.18:0/1309615622 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f057c0708a0 0x7f057c119fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:44:29.442 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:44:29.437+0000 7f057aa00700 1 -- 172.21.15.18:0/1309615622 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f056c005040 con 0x7f057c071f90 2024-04-12T21:44:29.442 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:44:29.437+0000 7f057aa00700 1 --2- 172.21.15.18:0/1309615622 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f057c071f90 0x7f057c11a4f0 secure :-1 s=READY pgs=1137 cs=0 l=1 rev1=1 crypto rx=0x7f0568002a50 tx=0x7f05680046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:44:29.442 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:44:29.437+0000 7f0573400700 1 -- 172.21.15.18:0/1309615622 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0568007cc0 con 0x7f057c071f90 2024-04-12T21:44:29.442 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:44:29.437+0000 7f0573400700 1 -- 172.21.15.18:0/1309615622 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f056800f040 con 0x7f057c071f90 2024-04-12T21:44:29.442 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:44:29.437+0000 7f0582600700 1 -- 172.21.15.18:0/1309615622 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f057c114400 con 0x7f057c071f90 2024-04-12T21:44:29.442 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:44:29.437+0000 7f0573400700 1 -- 172.21.15.18:0/1309615622 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f056800db70 con 0x7f057c071f90 2024-04-12T21:44:29.442 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:44:29.438+0000 7f0582600700 1 -- 172.21.15.18:0/1309615622 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f057c114940 con 0x7f057c071f90 2024-04-12T21:44:29.443 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:44:29.439+0000 7f0582600700 1 -- 172.21.15.18:0/1309615622 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f057c071aa0 con 0x7f057c071f90 2024-04-12T21:44:29.443 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:44:29.440+0000 7f0573400700 1 -- 172.21.15.18:0/1309615622 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7f056800dd10 con 0x7f057c071f90 2024-04-12T21:44:29.444 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:44:29.441+0000 7f0573400700 1 --2- 172.21.15.18:0/1309615622 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f056006ec00 0x7f05600710b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:44:29.444 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:44:29.441+0000 7f0573400700 1 -- 172.21.15.18:0/1309615622 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 6757+0+0 (secure 0 0 0) 0x7f056808e150 con 0x7f057c071f90 2024-04-12T21:44:29.444 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:44:29.441+0000 7f057b400700 1 --2- 172.21.15.18:0/1309615622 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f056006ec00 0x7f05600710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:44:29.444 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:44:29.442+0000 7f057b400700 1 --2- 172.21.15.18:0/1309615622 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f056006ec00 0x7f05600710b0 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f056c009fd0 tx=0x7f056c009e30 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:44:29.447 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:44:29.446+0000 7f0573400700 1 -- 172.21.15.18:0/1309615622 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f056805a880 con 0x7f057c071f90 2024-04-12T21:44:29.733 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:44:29.732+0000 7f0582600700 1 -- 172.21.15.18:0/1309615622 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f057c04c710 con 0x7f057c071f90 2024-04-12T21:44:29.734 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:44:29.733+0000 7f0573400700 1 -- 172.21.15.18:0/1309615622 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 33 v33) v1 ==== 76+0+1876 (secure 0 0 0) 0x7f056805a410 con 0x7f057c071f90 2024-04-12T21:44:29.735 INFO:teuthology.orchestra.run.smithi018.stdout:e33 2024-04-12T21:44:29.735 INFO:teuthology.orchestra.run.smithi018.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-12T21:44:29.735 INFO:teuthology.orchestra.run.smithi018.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-12T21:44:29.735 INFO:teuthology.orchestra.run.smithi018.stdout:legacy client fscid: 1 2024-04-12T21:44:29.735 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-12T21:44:29.735 INFO:teuthology.orchestra.run.smithi018.stdout:Filesystem 'cephfs' (1) 2024-04-12T21:44:29.735 INFO:teuthology.orchestra.run.smithi018.stdout:fs_name cephfs 2024-04-12T21:44:29.735 INFO:teuthology.orchestra.run.smithi018.stdout:epoch 24 2024-04-12T21:44:29.735 INFO:teuthology.orchestra.run.smithi018.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-04-12T21:44:29.735 INFO:teuthology.orchestra.run.smithi018.stdout:created 2024-04-12T20:51:57.362850+0000 2024-04-12T21:44:29.735 INFO:teuthology.orchestra.run.smithi018.stdout:modified 2024-04-12T21:06:59.717190+0000 2024-04-12T21:44:29.735 INFO:teuthology.orchestra.run.smithi018.stdout:tableserver 0 2024-04-12T21:44:29.735 INFO:teuthology.orchestra.run.smithi018.stdout:root 0 2024-04-12T21:44:29.735 INFO:teuthology.orchestra.run.smithi018.stdout:session_timeout 60 2024-04-12T21:44:29.736 INFO:teuthology.orchestra.run.smithi018.stdout:session_autoclose 300 2024-04-12T21:44:29.736 INFO:teuthology.orchestra.run.smithi018.stdout:max_file_size 1099511627776 2024-04-12T21:44:29.736 INFO:teuthology.orchestra.run.smithi018.stdout:max_xattr_size 65536 2024-04-12T21:44:29.736 INFO:teuthology.orchestra.run.smithi018.stdout:required_client_features {} 2024-04-12T21:44:29.736 INFO:teuthology.orchestra.run.smithi018.stdout:last_failure 0 2024-04-12T21:44:29.736 INFO:teuthology.orchestra.run.smithi018.stdout:last_failure_osd_epoch 100 2024-04-12T21:44:29.736 INFO:teuthology.orchestra.run.smithi018.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-12T21:44:29.736 INFO:teuthology.orchestra.run.smithi018.stdout:max_mds 2 2024-04-12T21:44:29.736 INFO:teuthology.orchestra.run.smithi018.stdout:in 0,1 2024-04-12T21:44:29.736 INFO:teuthology.orchestra.run.smithi018.stdout:up {} 2024-04-12T21:44:29.736 INFO:teuthology.orchestra.run.smithi018.stdout:failed 0,1 2024-04-12T21:44:29.736 INFO:teuthology.orchestra.run.smithi018.stdout:damaged 2024-04-12T21:44:29.736 INFO:teuthology.orchestra.run.smithi018.stdout:stopped 2024-04-12T21:44:29.736 INFO:teuthology.orchestra.run.smithi018.stdout:data_pools [3] 2024-04-12T21:44:29.736 INFO:teuthology.orchestra.run.smithi018.stdout:metadata_pool 2 2024-04-12T21:44:29.736 INFO:teuthology.orchestra.run.smithi018.stdout:inline_data disabled 2024-04-12T21:44:29.737 INFO:teuthology.orchestra.run.smithi018.stdout:balancer 2024-04-12T21:44:29.737 INFO:teuthology.orchestra.run.smithi018.stdout:bal_rank_mask -1 2024-04-12T21:44:29.737 INFO:teuthology.orchestra.run.smithi018.stdout:standby_count_wanted 1 2024-04-12T21:44:29.737 INFO:teuthology.orchestra.run.smithi018.stdout:qdb_cluster leader: 0 members: 2024-04-12T21:44:29.737 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-12T21:44:29.737 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-12T21:44:29.737 INFO:teuthology.orchestra.run.smithi018.stdout:Standby daemons: 2024-04-12T21:44:29.737 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-12T21:44:29.737 INFO:teuthology.orchestra.run.smithi018.stdout:[mds.cephfs.smithi018.acqbsc{-1:34436} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.18:6828/755179764,v1:172.21.15.18:6829/755179764] compat {c=[1],r=[1],i=[1fff]}] 2024-04-12T21:44:29.737 INFO:teuthology.orchestra.run.smithi018.stdout:[mds.cephfs.smithi018.szjluo{-1:34468} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.18:6826/2407992653,v1:172.21.15.18:6827/2407992653] compat {c=[1],r=[1],i=[1fff]}] 2024-04-12T21:44:29.737 INFO:teuthology.orchestra.run.smithi018.stdout:[mds.cephfs.smithi186.jdllyn{-1:44595} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.186:6824/3523444202,v1:172.21.15.186:6825/3523444202] compat {c=[1],r=[1],i=[1fff]}] 2024-04-12T21:44:29.737 INFO:teuthology.orchestra.run.smithi018.stdout:[mds.cephfs.smithi186.wjnafs{-1:44875} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.186:6826/3122443766,v1:172.21.15.186:6827/3122443766] compat {c=[1],r=[1],i=[1fff]}] 2024-04-12T21:44:29.738 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:44:29.736+0000 7f0582600700 1 -- 172.21.15.18:0/1309615622 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f056006ec00 msgr2=0x7f05600710b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:44:29.738 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:44:29.736+0000 7f0582600700 1 --2- 172.21.15.18:0/1309615622 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f056006ec00 0x7f05600710b0 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f056c009fd0 tx=0x7f056c009e30 comp rx=0 tx=0).stop 2024-04-12T21:44:29.738 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:44:29.736+0000 7f0582600700 1 -- 172.21.15.18:0/1309615622 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f057c071f90 msgr2=0x7f057c11a4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:44:29.738 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:44:29.736+0000 7f0582600700 1 --2- 172.21.15.18:0/1309615622 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f057c071f90 0x7f057c11a4f0 secure :-1 s=READY pgs=1137 cs=0 l=1 rev1=1 crypto rx=0x7f0568002a50 tx=0x7f05680046b0 comp rx=0 tx=0).stop 2024-04-12T21:44:29.738 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:44:29.736+0000 7f0582600700 1 -- 172.21.15.18:0/1309615622 shutdown_connections 2024-04-12T21:44:29.738 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:44:29.736+0000 7f0582600700 1 --2- 172.21.15.18:0/1309615622 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f056006ec00 0x7f05600710b0 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:44:29.738 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:44:29.736+0000 7f0582600700 1 --2- 172.21.15.18:0/1309615622 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f057c0708a0 0x7f057c119fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:44:29.738 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:44:29.736+0000 7f0582600700 1 --2- 172.21.15.18:0/1309615622 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f057c071f90 0x7f057c11a4f0 unknown :-1 s=CLOSED pgs=1137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:44:29.739 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:44:29.736+0000 7f0582600700 1 -- 172.21.15.18:0/1309615622 >> 172.21.15.18:0/1309615622 conn(0x7f057c0f8950 msgr2=0x7f057c0f9a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:44:29.739 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:44:29.736+0000 7f0582600700 1 -- 172.21.15.18:0/1309615622 shutdown_connections 2024-04-12T21:44:29.739 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:44:29.736+0000 7f0582600700 1 -- 172.21.15.18:0/1309615622 wait complete. 2024-04-12T21:44:29.739 INFO:teuthology.orchestra.run.smithi018.stderr:dumped fsmap epoch 33 2024-04-12T21:44:29.901 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:44:29.899+0000 7f7fcde00700 1 -- 172.21.15.18:0/1183142517 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7fc8105030 msgr2=0x7f7fc8105400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:44:29.901 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:44:29.899+0000 7f7fcde00700 1 --2- 172.21.15.18:0/1183142517 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7fc8105030 0x7f7fc8105400 secure :-1 s=READY pgs=1138 cs=0 l=1 rev1=1 crypto rx=0x7f7fb8004940 tx=0x7f7fb8009e30 comp rx=0 tx=0).stop 2024-04-12T21:44:29.901 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:44:29.899+0000 7f7fcde00700 1 -- 172.21.15.18:0/1183142517 shutdown_connections 2024-04-12T21:44:29.901 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:44:29.899+0000 7f7fcde00700 1 --2- 172.21.15.18:0/1183142517 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7fc81059d0 0x7f7fc80ff090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:44:29.901 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:44:29.899+0000 7f7fcde00700 1 --2- 172.21.15.18:0/1183142517 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7fc8105030 0x7f7fc8105400 unknown :-1 s=CLOSED pgs=1138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:44:29.901 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:44:29.899+0000 7f7fcde00700 1 -- 172.21.15.18:0/1183142517 >> 172.21.15.18:0/1183142517 conn(0x7f7fc80fabd0 msgr2=0x7f7fc80fcfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:44:29.901 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:44:29.899+0000 7f7fcde00700 1 -- 172.21.15.18:0/1183142517 shutdown_connections 2024-04-12T21:44:29.901 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:44:29.899+0000 7f7fcde00700 1 -- 172.21.15.18:0/1183142517 wait complete. 2024-04-12T21:44:29.902 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:44:29.900+0000 7f7fcde00700 1 Processor -- start 2024-04-12T21:44:29.902 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:44:29.900+0000 7f7fcde00700 1 -- start start 2024-04-12T21:44:29.902 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:44:29.901+0000 7f7fcde00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7fc8105030 0x7f7fc8115af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:44:29.902 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:44:29.901+0000 7f7fcde00700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7fc81059d0 0x7f7fc8116030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:44:29.903 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:44:29.901+0000 7f7fcde00700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7fc8116680 con 0x7f7fc8105030 2024-04-12T21:44:29.903 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:44:29.901+0000 7f7fcde00700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7fc810fd10 con 0x7f7fc81059d0 2024-04-12T21:44:29.903 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:44:29.901+0000 7f7fc6a00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7fc8105030 0x7f7fc8115af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:44:29.903 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:44:29.901+0000 7f7fc6a00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7fc8105030 0x7f7fc8115af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:54168/0 (socket says 172.21.15.18:54168) 2024-04-12T21:44:29.903 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:44:29.901+0000 7f7fc6a00700 1 -- 172.21.15.18:0/3207506512 learned_addr learned my addr 172.21.15.18:0/3207506512 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:44:29.904 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:44:29.901+0000 7f7fc6000700 1 --2- 172.21.15.18:0/3207506512 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7fc81059d0 0x7f7fc8116030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:44:29.904 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:44:29.901+0000 7f7fc6a00700 1 -- 172.21.15.18:0/3207506512 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7fc81059d0 msgr2=0x7f7fc8116030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:44:29.904 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:44:29.902+0000 7f7fc6a00700 1 --2- 172.21.15.18:0/3207506512 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7fc81059d0 0x7f7fc8116030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:44:29.904 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:44:29.902+0000 7f7fc6a00700 1 -- 172.21.15.18:0/3207506512 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7fb8005040 con 0x7f7fc8105030 2024-04-12T21:44:29.904 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:44:29.902+0000 7f7fc6a00700 1 --2- 172.21.15.18:0/3207506512 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7fc8105030 0x7f7fc8115af0 secure :-1 s=READY pgs=1139 cs=0 l=1 rev1=1 crypto rx=0x7f7fb8009150 tx=0x7f7fb8009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:44:29.904 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:44:29.902+0000 7f7fbea00700 1 -- 172.21.15.18:0/3207506512 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7fb800e040 con 0x7f7fc8105030 2024-04-12T21:44:29.904 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:44:29.902+0000 7f7fbea00700 1 -- 172.21.15.18:0/3207506512 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f7fb8013420 con 0x7f7fc8105030 2024-04-12T21:44:29.904 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:44:29.902+0000 7f7fcde00700 1 -- 172.21.15.18:0/3207506512 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7fc810ff90 con 0x7f7fc8105030 2024-04-12T21:44:29.904 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:44:29.902+0000 7f7fbea00700 1 -- 172.21.15.18:0/3207506512 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7fb801c630 con 0x7f7fc8105030 2024-04-12T21:44:29.904 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:44:29.902+0000 7f7fcde00700 1 -- 172.21.15.18:0/3207506512 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7fc8110550 con 0x7f7fc8105030 2024-04-12T21:44:29.905 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:44:29.904+0000 7f7fcde00700 1 -- 172.21.15.18:0/3207506512 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7fc8100780 con 0x7f7fc8105030 2024-04-12T21:44:29.906 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:44:29.905+0000 7f7fbea00700 1 -- 172.21.15.18:0/3207506512 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7f7fb8013590 con 0x7f7fc8105030 2024-04-12T21:44:29.907 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:44:29.906+0000 7f7fbea00700 1 --2- 172.21.15.18:0/3207506512 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f7fac06ec00 0x7f7fac0710b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:44:29.907 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:44:29.906+0000 7f7fbea00700 1 -- 172.21.15.18:0/3207506512 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 6757+0+0 (secure 0 0 0) 0x7f7fb808fc40 con 0x7f7fc8105030 2024-04-12T21:44:29.907 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:44:29.906+0000 7f7fc6000700 1 --2- 172.21.15.18:0/3207506512 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f7fac06ec00 0x7f7fac0710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:44:29.908 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:44:29.907+0000 7f7fc6000700 1 --2- 172.21.15.18:0/3207506512 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f7fac06ec00 0x7f7fac0710b0 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f7fc8109230 tx=0x7f7fb4009ee0 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:44:29.912 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:44:29.911+0000 7f7fbea00700 1 -- 172.21.15.18:0/3207506512 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f7fb8058960 con 0x7f7fc8105030 2024-04-12T21:44:30.105 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:44:29 smithi018 ceph-mon[112590]: pgmap v1344: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:44:30.105 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:44:29 smithi018 ceph-mon[112590]: from='client.36332 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:44:30.105 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:44:29 smithi018 ceph-mon[112590]: from='client.36336 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:44:30.105 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:44:29 smithi018 ceph-mon[112590]: from='client.? 172.21.15.18:0/2122077784' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T21:44:30.105 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:44:29 smithi018 ceph-mon[112590]: from='client.? 172.21.15.18:0/1309615622' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-12T21:44:30.165 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:44:30.163+0000 7f7fcde00700 1 -- 172.21.15.18:0/3207506512 --> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f7fc810c130 con 0x7f7fac06ec00 2024-04-12T21:44:30.166 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:44:30.165+0000 7f7fbea00700 1 -- 172.21.15.18:0/3207506512 <== mgr.34102 v2:172.21.15.18:6800/3812891919 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f7fc810c130 con 0x7f7fac06ec00 2024-04-12T21:44:30.167 INFO:teuthology.orchestra.run.smithi018.stdout:{ 2024-04-12T21:44:30.167 INFO:teuthology.orchestra.run.smithi018.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:f4452def3be4d350f4a8e5feaf7ebb1511be9298fe68478ba7a2d6fbd7d69a63", 2024-04-12T21:44:30.167 INFO:teuthology.orchestra.run.smithi018.stdout: "in_progress": true, 2024-04-12T21:44:30.167 INFO:teuthology.orchestra.run.smithi018.stdout: "which": "Upgrading all daemon types on all hosts", 2024-04-12T21:44:30.167 INFO:teuthology.orchestra.run.smithi018.stdout: "services_complete": [ 2024-04-12T21:44:30.167 INFO:teuthology.orchestra.run.smithi018.stdout: "mgr", 2024-04-12T21:44:30.167 INFO:teuthology.orchestra.run.smithi018.stdout: "mds", 2024-04-12T21:44:30.167 INFO:teuthology.orchestra.run.smithi018.stdout: "crash", 2024-04-12T21:44:30.167 INFO:teuthology.orchestra.run.smithi018.stdout: "osd", 2024-04-12T21:44:30.167 INFO:teuthology.orchestra.run.smithi018.stdout: "mon" 2024-04-12T21:44:30.167 INFO:teuthology.orchestra.run.smithi018.stdout: ], 2024-04-12T21:44:30.167 INFO:teuthology.orchestra.run.smithi018.stdout: "progress": "16/21 daemons upgraded", 2024-04-12T21:44:30.168 INFO:teuthology.orchestra.run.smithi018.stdout: "message": "Currently upgrading mds daemons", 2024-04-12T21:44:30.168 INFO:teuthology.orchestra.run.smithi018.stdout: "is_paused": false 2024-04-12T21:44:30.168 INFO:teuthology.orchestra.run.smithi018.stdout:} 2024-04-12T21:44:30.169 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:44:30.168+0000 7f7fcde00700 1 -- 172.21.15.18:0/3207506512 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f7fac06ec00 msgr2=0x7f7fac0710b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:44:30.169 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:44:30.168+0000 7f7fcde00700 1 --2- 172.21.15.18:0/3207506512 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f7fac06ec00 0x7f7fac0710b0 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f7fc8109230 tx=0x7f7fb4009ee0 comp rx=0 tx=0).stop 2024-04-12T21:44:30.169 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:44:30.168+0000 7f7fcde00700 1 -- 172.21.15.18:0/3207506512 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7fc8105030 msgr2=0x7f7fc8115af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:44:30.169 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:44:30.168+0000 7f7fcde00700 1 --2- 172.21.15.18:0/3207506512 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7fc8105030 0x7f7fc8115af0 secure :-1 s=READY pgs=1139 cs=0 l=1 rev1=1 crypto rx=0x7f7fb8009150 tx=0x7f7fb8009e30 comp rx=0 tx=0).stop 2024-04-12T21:44:30.170 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:44:30.168+0000 7f7fcde00700 1 -- 172.21.15.18:0/3207506512 shutdown_connections 2024-04-12T21:44:30.170 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:44:30.168+0000 7f7fcde00700 1 --2- 172.21.15.18:0/3207506512 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f7fac06ec00 0x7f7fac0710b0 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:44:30.170 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:44:30.168+0000 7f7fcde00700 1 --2- 172.21.15.18:0/3207506512 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7fc8105030 0x7f7fc8115af0 unknown :-1 s=CLOSED pgs=1139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:44:30.170 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:44:30.168+0000 7f7fcde00700 1 --2- 172.21.15.18:0/3207506512 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7fc81059d0 0x7f7fc8116030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:44:30.170 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:44:30.168+0000 7f7fcde00700 1 -- 172.21.15.18:0/3207506512 >> 172.21.15.18:0/3207506512 conn(0x7f7fc80fabd0 msgr2=0x7f7fc80fbbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:44:30.170 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:44:30.168+0000 7f7fcde00700 1 -- 172.21.15.18:0/3207506512 shutdown_connections 2024-04-12T21:44:30.171 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:44:30.168+0000 7f7fcde00700 1 -- 172.21.15.18:0/3207506512 wait complete. 2024-04-12T21:44:30.284 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:44:29 smithi186 ceph-mon[86379]: pgmap v1344: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:44:30.284 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:44:29 smithi186 ceph-mon[86379]: from='client.36332 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:44:30.284 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:44:29 smithi186 ceph-mon[86379]: from='client.36336 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:44:30.284 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:44:29 smithi186 ceph-mon[86379]: from='client.? 172.21.15.18:0/2122077784' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T21:44:30.284 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:44:29 smithi186 ceph-mon[86379]: from='client.? 172.21.15.18:0/1309615622' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-12T21:44:30.332 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:44:30.330+0000 7f7f2c200700 1 -- 172.21.15.18:0/1921535953 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7f241037a0 msgr2=0x7f7f240fce10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:44:30.332 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:44:30.330+0000 7f7f2c200700 1 --2- 172.21.15.18:0/1921535953 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7f241037a0 0x7f7f240fce10 secure :-1 s=READY pgs=1140 cs=0 l=1 rev1=1 crypto rx=0x7f7f10004940 tx=0x7f7f10009e30 comp rx=0 tx=0).stop 2024-04-12T21:44:30.333 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:44:30.331+0000 7f7f2c200700 1 -- 172.21.15.18:0/1921535953 shutdown_connections 2024-04-12T21:44:30.333 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:44:30.331+0000 7f7f2c200700 1 --2- 172.21.15.18:0/1921535953 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7f241037a0 0x7f7f240fce10 unknown :-1 s=CLOSED pgs=1140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:44:30.333 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:44:30.331+0000 7f7f2c200700 1 --2- 172.21.15.18:0/1921535953 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7f24102e00 0x7f7f241031d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:44:30.333 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:44:30.331+0000 7f7f2c200700 1 -- 172.21.15.18:0/1921535953 >> 172.21.15.18:0/1921535953 conn(0x7f7f240f89b0 msgr2=0x7f7f240fadc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:44:30.333 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:44:30.331+0000 7f7f2c200700 1 -- 172.21.15.18:0/1921535953 shutdown_connections 2024-04-12T21:44:30.333 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:44:30.331+0000 7f7f2c200700 1 -- 172.21.15.18:0/1921535953 wait complete. 2024-04-12T21:44:30.334 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:44:30.332+0000 7f7f2c200700 1 Processor -- start 2024-04-12T21:44:30.334 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:44:30.332+0000 7f7f2c200700 1 -- start start 2024-04-12T21:44:30.334 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:44:30.332+0000 7f7f2c200700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7f24102e00 0x7f7f24113bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:44:30.334 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:44:30.332+0000 7f7f2c200700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7f241037a0 0x7f7f241140f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:44:30.334 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:44:30.332+0000 7f7f2c200700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f24114740 con 0x7f7f24102e00 2024-04-12T21:44:30.334 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:44:30.332+0000 7f7f2c200700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f2410db30 con 0x7f7f241037a0 2024-04-12T21:44:30.335 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:44:30.333+0000 7f7f29800700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7f24102e00 0x7f7f24113bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:44:30.335 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:44:30.333+0000 7f7f29800700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7f24102e00 0x7f7f24113bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:54176/0 (socket says 172.21.15.18:54176) 2024-04-12T21:44:30.335 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:44:30.333+0000 7f7f29800700 1 -- 172.21.15.18:0/162043380 learned_addr learned my addr 172.21.15.18:0/162043380 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:44:30.335 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:44:30.333+0000 7f7f28e00700 1 --2- 172.21.15.18:0/162043380 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7f241037a0 0x7f7f241140f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:44:30.336 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:44:30.333+0000 7f7f29800700 1 -- 172.21.15.18:0/162043380 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7f241037a0 msgr2=0x7f7f241140f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:44:30.336 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:44:30.333+0000 7f7f29800700 1 --2- 172.21.15.18:0/162043380 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7f241037a0 0x7f7f241140f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:44:30.336 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:44:30.333+0000 7f7f29800700 1 -- 172.21.15.18:0/162043380 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f10005040 con 0x7f7f24102e00 2024-04-12T21:44:30.336 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:44:30.334+0000 7f7f29800700 1 --2- 172.21.15.18:0/162043380 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7f24102e00 0x7f7f24113bb0 secure :-1 s=READY pgs=1141 cs=0 l=1 rev1=1 crypto rx=0x7f7f18002a50 tx=0x7f7f180046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:44:30.336 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:44:30.334+0000 7f7f21600700 1 -- 172.21.15.18:0/162043380 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7f18007cc0 con 0x7f7f24102e00 2024-04-12T21:44:30.336 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:44:30.334+0000 7f7f21600700 1 -- 172.21.15.18:0/162043380 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f7f1800f040 con 0x7f7f24102e00 2024-04-12T21:44:30.336 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:44:30.334+0000 7f7f21600700 1 -- 172.21.15.18:0/162043380 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7f1800db70 con 0x7f7f24102e00 2024-04-12T21:44:30.336 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:44:30.334+0000 7f7f2c200700 1 -- 172.21.15.18:0/162043380 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7f2410de10 con 0x7f7f24102e00 2024-04-12T21:44:30.336 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:44:30.334+0000 7f7f2c200700 1 -- 172.21.15.18:0/162043380 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7f2410e3d0 con 0x7f7f24102e00 2024-04-12T21:44:30.337 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:44:30.335+0000 7f7f2c200700 1 -- 172.21.15.18:0/162043380 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7f241052d0 con 0x7f7f24102e00 2024-04-12T21:44:30.337 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:44:30.336+0000 7f7f21600700 1 -- 172.21.15.18:0/162043380 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7f7f1800dd10 con 0x7f7f24102e00 2024-04-12T21:44:30.339 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:44:30.337+0000 7f7f21600700 1 --2- 172.21.15.18:0/162043380 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f7f0c06ec10 0x7f7f0c0710c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:44:30.339 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:44:30.337+0000 7f7f21600700 1 -- 172.21.15.18:0/162043380 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 6757+0+0 (secure 0 0 0) 0x7f7f1808dff0 con 0x7f7f24102e00 2024-04-12T21:44:30.339 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:44:30.338+0000 7f7f28e00700 1 --2- 172.21.15.18:0/162043380 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f7f0c06ec10 0x7f7f0c0710c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:44:30.340 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:44:30.339+0000 7f7f28e00700 1 --2- 172.21.15.18:0/162043380 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f7f0c06ec10 0x7f7f0c0710c0 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f7f1000b3b0 tx=0x7f7f10009e30 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:44:30.344 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:44:30.343+0000 7f7f21600700 1 -- 172.21.15.18:0/162043380 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f7f1805a800 con 0x7f7f24102e00 2024-04-12T21:44:30.665 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:44:30.663+0000 7f7f2c200700 1 -- 172.21.15.18:0/162043380 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f7f2410f260 con 0x7f7f24102e00 2024-04-12T21:44:30.665 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:44:30.664+0000 7f7f21600700 1 -- 172.21.15.18:0/162043380 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+4303 (secure 0 0 0) 0x7f7f1805a390 con 0x7f7f24102e00 2024-04-12T21:44:30.665 INFO:teuthology.orchestra.run.smithi018.stdout:HEALTH_ERR failed to probe daemons or devices; 1 filesystem is degraded; 1 filesystem is offline 2024-04-12T21:44:30.666 INFO:teuthology.orchestra.run.smithi018.stdout:[WRN] CEPHADM_REFRESH_FAILED: failed to probe daemons or devices 2024-04-12T21:44:30.666 INFO:teuthology.orchestra.run.smithi018.stdout: host smithi186 `cephadm ceph-volume` failed: cephadm exited with an error code: 1, stderr: Inferring config /var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/mon.smithi186/config 2024-04-12T21:44:30.666 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/podman: timeout after 895 seconds 2024-04-12T21:44:30.666 INFO:teuthology.orchestra.run.smithi018.stdout:Non-zero exit code 124 from /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:f4452def3be4d350f4a8e5feaf7ebb1511be9298fe68478ba7a2d6fbd7d69a63 -e NODE_NAME=smithi186 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968:/var/run/ceph:z -v /var/log/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968:/var/log/ceph:z -v /var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmptgom53f9:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:f4452def3be4d350f4a8e5feaf7ebb1511be9298fe68478ba7a2d6fbd7d69a63 inventory --format=json-pretty --filter-for-batch 2024-04-12T21:44:30.666 INFO:teuthology.orchestra.run.smithi018.stdout:Traceback (most recent call last): 2024-04-12T21:44:30.666 INFO:teuthology.orchestra.run.smithi018.stdout: File "/usr/lib64/python3.9/runpy.py", line 197, in _run_module_as_main 2024-04-12T21:44:30.666 INFO:teuthology.orchestra.run.smithi018.stdout: return _run_code(code, main_globals, None, 2024-04-12T21:44:30.666 INFO:teuthology.orchestra.run.smithi018.stdout: File "/usr/lib64/python3.9/runpy.py", line 87, in _run_code 2024-04-12T21:44:30.666 INFO:teuthology.orchestra.run.smithi018.stdout: exec(code, run_globals) 2024-04-12T21:44:30.666 INFO:teuthology.orchestra.run.smithi018.stdout: File "/var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/cephadm.e44f3ed5e34e4ef52eedef233d1190bff004470d8e8005127b5312bff357fa7f/__main__.py", line 5618, in 2024-04-12T21:44:30.666 INFO:teuthology.orchestra.run.smithi018.stdout: File "/var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/cephadm.e44f3ed5e34e4ef52eedef233d1190bff004470d8e8005127b5312bff357fa7f/__main__.py", line 5606, in main 2024-04-12T21:44:30.666 INFO:teuthology.orchestra.run.smithi018.stdout: File "/var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/cephadm.e44f3ed5e34e4ef52eedef233d1190bff004470d8e8005127b5312bff357fa7f/__main__.py", line 408, in _infer_config 2024-04-12T21:44:30.666 INFO:teuthology.orchestra.run.smithi018.stdout: File "/var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/cephadm.e44f3ed5e34e4ef52eedef233d1190bff004470d8e8005127b5312bff357fa7f/__main__.py", line 323, in _infer_fsid 2024-04-12T21:44:30.667 INFO:teuthology.orchestra.run.smithi018.stdout: File "/var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/cephadm.e44f3ed5e34e4ef52eedef233d1190bff004470d8e8005127b5312bff357fa7f/__main__.py", line 436, in _infer_image 2024-04-12T21:44:30.667 INFO:teuthology.orchestra.run.smithi018.stdout: File "/var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/cephadm.e44f3ed5e34e4ef52eedef233d1190bff004470d8e8005127b5312bff357fa7f/__main__.py", line 310, in _validate_fsid 2024-04-12T21:44:30.667 INFO:teuthology.orchestra.run.smithi018.stdout: File "/var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/cephadm.e44f3ed5e34e4ef52eedef233d1190bff004470d8e8005127b5312bff357fa7f/__main__.py", line 3359, in command_ceph_volume 2024-04-12T21:44:30.667 INFO:teuthology.orchestra.run.smithi018.stdout: File "/var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/cephadm.e44f3ed5e34e4ef52eedef233d1190bff004470d8e8005127b5312bff357fa7f/cephadmlib/call_wrappers.py", line 310, in call_throws 2024-04-12T21:44:30.667 INFO:teuthology.orchestra.run.smithi018.stdout:RuntimeError: Failed command: /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:f4452def3be4d350f4a8e5feaf7ebb1511be9298fe68478ba7a2d6fbd7d69a63 -e NODE_NAME=smithi186 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968:/var/run/ceph:z -v /var/log/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968:/var/log/ceph:z -v /var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmptgom53f9:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:f4452def3be4d350f4a8e5feaf7ebb1511be9298fe68478ba7a2d6fbd7d69a63 inventory --format=json-pretty --filter-for-batch 2024-04-12T21:44:30.667 INFO:teuthology.orchestra.run.smithi018.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-04-12T21:44:30.667 INFO:teuthology.orchestra.run.smithi018.stdout: fs cephfs is degraded 2024-04-12T21:44:30.667 INFO:teuthology.orchestra.run.smithi018.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-04-12T21:44:30.667 INFO:teuthology.orchestra.run.smithi018.stdout: fs cephfs is offline because no MDS is active for it. 2024-04-12T21:44:30.668 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:44:30.667+0000 7f7f2c200700 1 -- 172.21.15.18:0/162043380 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f7f0c06ec10 msgr2=0x7f7f0c0710c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:44:30.668 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:44:30.667+0000 7f7f2c200700 1 --2- 172.21.15.18:0/162043380 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f7f0c06ec10 0x7f7f0c0710c0 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f7f1000b3b0 tx=0x7f7f10009e30 comp rx=0 tx=0).stop 2024-04-12T21:44:30.668 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:44:30.667+0000 7f7f2c200700 1 -- 172.21.15.18:0/162043380 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7f24102e00 msgr2=0x7f7f24113bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:44:30.669 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:44:30.667+0000 7f7f2c200700 1 --2- 172.21.15.18:0/162043380 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7f24102e00 0x7f7f24113bb0 secure :-1 s=READY pgs=1141 cs=0 l=1 rev1=1 crypto rx=0x7f7f18002a50 tx=0x7f7f180046b0 comp rx=0 tx=0).stop 2024-04-12T21:44:30.669 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:44:30.667+0000 7f7f2c200700 1 -- 172.21.15.18:0/162043380 shutdown_connections 2024-04-12T21:44:30.669 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:44:30.667+0000 7f7f2c200700 1 --2- 172.21.15.18:0/162043380 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f7f0c06ec10 0x7f7f0c0710c0 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:44:30.669 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:44:30.667+0000 7f7f2c200700 1 --2- 172.21.15.18:0/162043380 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7f24102e00 0x7f7f24113bb0 unknown :-1 s=CLOSED pgs=1141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:44:30.669 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:44:30.667+0000 7f7f2c200700 1 --2- 172.21.15.18:0/162043380 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7f241037a0 0x7f7f241140f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:44:30.669 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:44:30.667+0000 7f7f2c200700 1 -- 172.21.15.18:0/162043380 >> 172.21.15.18:0/162043380 conn(0x7f7f240f89b0 msgr2=0x7f7f241021c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:44:30.669 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:44:30.667+0000 7f7f2c200700 1 -- 172.21.15.18:0/162043380 shutdown_connections 2024-04-12T21:44:30.670 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:44:30.668+0000 7f7f2c200700 1 -- 172.21.15.18:0/162043380 wait complete. 2024-04-12T21:44:31.284 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:44:30 smithi186 ceph-mon[86379]: from='client.? 172.21.15.18:0/162043380' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-12T21:44:31.355 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:44:30 smithi018 ceph-mon[112590]: from='client.? 172.21.15.18:0/162043380' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-12T21:44:32.284 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:44:31 smithi186 ceph-mon[86379]: from='client.36348 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:44:32.284 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:44:31 smithi186 ceph-mon[86379]: pgmap v1345: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:44:32.355 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:44:31 smithi018 ceph-mon[112590]: from='client.36348 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:44:32.355 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:44:31 smithi018 ceph-mon[112590]: pgmap v1345: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:44:34.284 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:44:33 smithi186 ceph-mon[86379]: pgmap v1346: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:44:34.355 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:44:33 smithi018 ceph-mon[112590]: pgmap v1346: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:44:36.284 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:44:35 smithi186 ceph-mon[86379]: pgmap v1347: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:44:36.355 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:44:35 smithi018 ceph-mon[112590]: pgmap v1347: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:44:38.284 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:44:37 smithi186 ceph-mon[86379]: pgmap v1348: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:44:38.355 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:44:37 smithi018 ceph-mon[112590]: pgmap v1348: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:44:40.284 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:44:39 smithi186 ceph-mon[86379]: pgmap v1349: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:44:40.284 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:44:39 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T21:44:40.355 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:44:39 smithi018 ceph-mon[112590]: pgmap v1349: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:44:40.355 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:44:39 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T21:44:42.284 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:44:41 smithi186 ceph-mon[86379]: pgmap v1350: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:44:42.355 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:44:41 smithi018 ceph-mon[112590]: pgmap v1350: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:44:44.284 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:44:43 smithi186 ceph-mon[86379]: pgmap v1351: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:44:44.355 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:44:43 smithi018 ceph-mon[112590]: pgmap v1351: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:44:46.284 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:44:45 smithi186 ceph-mon[86379]: pgmap v1352: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:44:46.355 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:44:45 smithi018 ceph-mon[112590]: pgmap v1352: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:44:48.284 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:44:47 smithi186 ceph-mon[86379]: pgmap v1353: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:44:48.355 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:44:47 smithi018 ceph-mon[112590]: pgmap v1353: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:44:50.285 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:44:49 smithi186 ceph-mon[86379]: pgmap v1354: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:44:50.355 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:44:49 smithi018 ceph-mon[112590]: pgmap v1354: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:44:52.284 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:44:51 smithi186 ceph-mon[86379]: pgmap v1355: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:44:52.354 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:44:51 smithi018 ceph-mon[112590]: pgmap v1355: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:44:54.284 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:44:54 smithi186 ceph-mon[86379]: pgmap v1356: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:44:54.355 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:44:54 smithi018 ceph-mon[112590]: pgmap v1356: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:44:55.284 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:44:55 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T21:44:55.355 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:44:55 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T21:44:56.284 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:44:56 smithi186 ceph-mon[86379]: pgmap v1357: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:44:56.355 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:44:56 smithi018 ceph-mon[112590]: pgmap v1357: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:44:58.284 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:44:58 smithi186 ceph-mon[86379]: pgmap v1358: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:44:58.355 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:44:58 smithi018 ceph-mon[112590]: pgmap v1358: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:45:00.284 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:45:00 smithi186 ceph-mon[86379]: pgmap v1359: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:45:00.355 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:45:00 smithi018 ceph-mon[112590]: pgmap v1359: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:45:00.842 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:00.841+0000 7f2079c00700 1 -- 172.21.15.18:0/2386555983 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2074065620 msgr2=0x7f2074065a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:45:00.842 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:00.841+0000 7f2079c00700 1 --2- 172.21.15.18:0/2386555983 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2074065620 0x7f2074065a90 secure :-1 s=READY pgs=1142 cs=0 l=1 rev1=1 crypto rx=0x7f2060004940 tx=0x7f2060009e30 comp rx=0 tx=0).stop 2024-04-12T21:45:00.842 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:00.841+0000 7f2079c00700 1 -- 172.21.15.18:0/2386555983 shutdown_connections 2024-04-12T21:45:00.843 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:00.841+0000 7f2079c00700 1 --2- 172.21.15.18:0/2386555983 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2074065620 0x7f2074065a90 unknown :-1 s=CLOSED pgs=1142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:45:00.843 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:00.841+0000 7f2079c00700 1 --2- 172.21.15.18:0/2386555983 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2074102c40 0x7f2074103010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:45:00.843 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:00.841+0000 7f2079c00700 1 -- 172.21.15.18:0/2386555983 >> 172.21.15.18:0/2386555983 conn(0x7f2074071b70 msgr2=0x7f2074071f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:45:00.843 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:00.841+0000 7f2079c00700 1 -- 172.21.15.18:0/2386555983 shutdown_connections 2024-04-12T21:45:00.843 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:00.841+0000 7f2079c00700 1 -- 172.21.15.18:0/2386555983 wait complete. 2024-04-12T21:45:00.845 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:00.842+0000 7f2079c00700 1 Processor -- start 2024-04-12T21:45:00.845 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:00.842+0000 7f2079c00700 1 -- start start 2024-04-12T21:45:00.845 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:00.843+0000 7f2079c00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2074065620 0x7f2074115c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:45:00.845 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:00.843+0000 7f2079c00700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2074102c40 0x7f2074116170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:45:00.845 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:00.843+0000 7f2079c00700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2074116850 con 0x7f2074065620 2024-04-12T21:45:00.845 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:00.843+0000 7f2079c00700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f207410fb90 con 0x7f2074102c40 2024-04-12T21:45:00.845 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:00.843+0000 7f2073e00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2074065620 0x7f2074115c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:45:00.845 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:00.843+0000 7f2073e00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2074065620 0x7f2074115c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:58112/0 (socket says 172.21.15.18:58112) 2024-04-12T21:45:00.845 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:00.843+0000 7f2073e00700 1 -- 172.21.15.18:0/2538244038 learned_addr learned my addr 172.21.15.18:0/2538244038 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:45:00.846 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:00.844+0000 7f2073400700 1 --2- 172.21.15.18:0/2538244038 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2074102c40 0x7f2074116170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:45:00.846 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:00.844+0000 7f2073e00700 1 -- 172.21.15.18:0/2538244038 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2074102c40 msgr2=0x7f2074116170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:45:00.846 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:00.844+0000 7f2073e00700 1 --2- 172.21.15.18:0/2538244038 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2074102c40 0x7f2074116170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:45:00.846 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:00.844+0000 7f2073e00700 1 -- 172.21.15.18:0/2538244038 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2060005040 con 0x7f2074065620 2024-04-12T21:45:00.846 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:00.844+0000 7f2073400700 1 --2- 172.21.15.18:0/2538244038 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2074102c40 0x7f2074116170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-12T21:45:00.846 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:00.844+0000 7f2073e00700 1 --2- 172.21.15.18:0/2538244038 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2074065620 0x7f2074115c30 secure :-1 s=READY pgs=1143 cs=0 l=1 rev1=1 crypto rx=0x7f206400b790 tx=0x7f2064009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:45:00.846 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:00.844+0000 7f206be00700 1 -- 172.21.15.18:0/2538244038 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f206400f770 con 0x7f2074065620 2024-04-12T21:45:00.846 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:00.844+0000 7f206be00700 1 -- 172.21.15.18:0/2538244038 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f206400fda0 con 0x7f2074065620 2024-04-12T21:45:00.847 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:00.844+0000 7f2079c00700 1 -- 172.21.15.18:0/2538244038 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f207410fe10 con 0x7f2074065620 2024-04-12T21:45:00.847 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:00.844+0000 7f206be00700 1 -- 172.21.15.18:0/2538244038 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f206400d310 con 0x7f2074065620 2024-04-12T21:45:00.847 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:00.845+0000 7f2079c00700 1 -- 172.21.15.18:0/2538244038 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2074110350 con 0x7f2074065620 2024-04-12T21:45:00.847 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:00.846+0000 7f2079c00700 1 -- 172.21.15.18:0/2538244038 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f20740fe3a0 con 0x7f2074065620 2024-04-12T21:45:00.847 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:00.847+0000 7f206be00700 1 -- 172.21.15.18:0/2538244038 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7f2064010420 con 0x7f2074065620 2024-04-12T21:45:00.848 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:00.848+0000 7f206be00700 1 --2- 172.21.15.18:0/2538244038 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f205806ec10 0x7f20580710c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:45:00.848 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:00.848+0000 7f206be00700 1 -- 172.21.15.18:0/2538244038 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 6757+0+0 (secure 0 0 0) 0x7f206408e8e0 con 0x7f2074065620 2024-04-12T21:45:00.848 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:00.848+0000 7f2073400700 1 --2- 172.21.15.18:0/2538244038 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f205806ec10 0x7f20580710c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:45:00.849 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:00.849+0000 7f2073400700 1 --2- 172.21.15.18:0/2538244038 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f205806ec10 0x7f20580710c0 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f206000b3b0 tx=0x7f2060009e30 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:45:00.854 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:00.853+0000 7f206be00700 1 -- 172.21.15.18:0/2538244038 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f206405b1d0 con 0x7f2074065620 2024-04-12T21:45:01.110 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:01.109+0000 7f2079c00700 1 -- 172.21.15.18:0/2538244038 --> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f207404bf70 con 0x7f205806ec10 2024-04-12T21:45:01.112 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:01.111+0000 7f206be00700 1 -- 172.21.15.18:0/2538244038 <== mgr.34102 v2:172.21.15.18:6800/3812891919 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f2060008e30 con 0x7f205806ec10 2024-04-12T21:45:01.115 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:01.114+0000 7f2079c00700 1 -- 172.21.15.18:0/2538244038 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f205806ec10 msgr2=0x7f20580710c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:45:01.115 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:01.114+0000 7f2079c00700 1 --2- 172.21.15.18:0/2538244038 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f205806ec10 0x7f20580710c0 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f206000b3b0 tx=0x7f2060009e30 comp rx=0 tx=0).stop 2024-04-12T21:45:01.115 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:01.114+0000 7f2079c00700 1 -- 172.21.15.18:0/2538244038 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2074065620 msgr2=0x7f2074115c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:45:01.116 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:01.114+0000 7f2079c00700 1 --2- 172.21.15.18:0/2538244038 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2074065620 0x7f2074115c30 secure :-1 s=READY pgs=1143 cs=0 l=1 rev1=1 crypto rx=0x7f206400b790 tx=0x7f2064009250 comp rx=0 tx=0).stop 2024-04-12T21:45:01.116 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:01.115+0000 7f2079c00700 1 -- 172.21.15.18:0/2538244038 shutdown_connections 2024-04-12T21:45:01.116 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:01.115+0000 7f2079c00700 1 --2- 172.21.15.18:0/2538244038 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f205806ec10 0x7f20580710c0 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:45:01.116 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:01.115+0000 7f2079c00700 1 --2- 172.21.15.18:0/2538244038 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2074065620 0x7f2074115c30 unknown :-1 s=CLOSED pgs=1143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:45:01.116 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:01.115+0000 7f2079c00700 1 --2- 172.21.15.18:0/2538244038 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2074102c40 0x7f2074116170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:45:01.116 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:01.115+0000 7f2079c00700 1 -- 172.21.15.18:0/2538244038 >> 172.21.15.18:0/2538244038 conn(0x7f2074071b70 msgr2=0x7f20740fa010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:45:01.116 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:01.115+0000 7f2079c00700 1 -- 172.21.15.18:0/2538244038 shutdown_connections 2024-04-12T21:45:01.116 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:01.115+0000 7f2079c00700 1 -- 172.21.15.18:0/2538244038 wait complete. 2024-04-12T21:45:01.131 INFO:teuthology.orchestra.run.smithi018.stdout:true 2024-04-12T21:45:01.275 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:01.274+0000 7fb7e5c00700 1 -- 172.21.15.18:0/493566408 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb7e0109dd0 msgr2=0x7fb7e0103440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:45:01.275 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:01.274+0000 7fb7e5c00700 1 --2- 172.21.15.18:0/493566408 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb7e0109dd0 0x7fb7e0103440 secure :-1 s=READY pgs=1144 cs=0 l=1 rev1=1 crypto rx=0x7fb7cc004940 tx=0x7fb7cc009e30 comp rx=0 tx=0).stop 2024-04-12T21:45:01.275 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:01.274+0000 7fb7e5c00700 1 -- 172.21.15.18:0/493566408 shutdown_connections 2024-04-12T21:45:01.275 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:01.274+0000 7fb7e5c00700 1 --2- 172.21.15.18:0/493566408 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb7e0109dd0 0x7fb7e0103440 unknown :-1 s=CLOSED pgs=1144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:45:01.275 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:01.274+0000 7fb7e5c00700 1 --2- 172.21.15.18:0/493566408 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb7e0109430 0x7fb7e0109800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:45:01.275 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:01.274+0000 7fb7e5c00700 1 -- 172.21.15.18:0/493566408 >> 172.21.15.18:0/493566408 conn(0x7fb7e0072c00 msgr2=0x7fb7e0075010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:45:01.275 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:01.274+0000 7fb7e5c00700 1 -- 172.21.15.18:0/493566408 shutdown_connections 2024-04-12T21:45:01.275 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:01.274+0000 7fb7e5c00700 1 -- 172.21.15.18:0/493566408 wait complete. 2024-04-12T21:45:01.277 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:01.275+0000 7fb7e5c00700 1 Processor -- start 2024-04-12T21:45:01.277 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:01.275+0000 7fb7e5c00700 1 -- start start 2024-04-12T21:45:01.277 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:01.275+0000 7fb7e5c00700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb7e0109430 0x7fb7e0119fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:45:01.277 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:01.275+0000 7fb7e5c00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb7e0109dd0 0x7fb7e011a4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:45:01.277 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:01.275+0000 7fb7e5c00700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7e011abc0 con 0x7fb7e0109dd0 2024-04-12T21:45:01.277 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:01.275+0000 7fb7e5c00700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7e0114110 con 0x7fb7e0109430 2024-04-12T21:45:01.278 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:01.276+0000 7fb7de000700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb7e0109dd0 0x7fb7e011a4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:45:01.278 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:01.276+0000 7fb7de000700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb7e0109dd0 0x7fb7e011a4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:58130/0 (socket says 172.21.15.18:58130) 2024-04-12T21:45:01.278 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:01.276+0000 7fb7de000700 1 -- 172.21.15.18:0/3118678773 learned_addr learned my addr 172.21.15.18:0/3118678773 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:45:01.278 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:01.276+0000 7fb7dea00700 1 --2- 172.21.15.18:0/3118678773 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb7e0109430 0x7fb7e0119fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:45:01.278 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:01.276+0000 7fb7de000700 1 -- 172.21.15.18:0/3118678773 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb7e0109430 msgr2=0x7fb7e0119fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:45:01.278 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:01.276+0000 7fb7de000700 1 --2- 172.21.15.18:0/3118678773 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb7e0109430 0x7fb7e0119fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:45:01.278 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:01.276+0000 7fb7de000700 1 -- 172.21.15.18:0/3118678773 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7cc005040 con 0x7fb7e0109dd0 2024-04-12T21:45:01.278 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:01.276+0000 7fb7de000700 1 --2- 172.21.15.18:0/3118678773 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb7e0109dd0 0x7fb7e011a4e0 secure :-1 s=READY pgs=1145 cs=0 l=1 rev1=1 crypto rx=0x7fb7cc009fd0 tx=0x7fb7cc009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:45:01.278 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:01.277+0000 7fb7d6a00700 1 -- 172.21.15.18:0/3118678773 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb7cc00e030 con 0x7fb7e0109dd0 2024-04-12T21:45:01.279 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:01.277+0000 7fb7d6a00700 1 -- 172.21.15.18:0/3118678773 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb7cc00bde0 con 0x7fb7e0109dd0 2024-04-12T21:45:01.279 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:01.277+0000 7fb7e5c00700 1 -- 172.21.15.18:0/3118678773 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb7e0114330 con 0x7fb7e0109dd0 2024-04-12T21:45:01.279 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:01.277+0000 7fb7d6a00700 1 -- 172.21.15.18:0/3118678773 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb7cc013e10 con 0x7fb7e0109dd0 2024-04-12T21:45:01.279 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:01.277+0000 7fb7e5c00700 1 -- 172.21.15.18:0/3118678773 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb7e0114870 con 0x7fb7e0109dd0 2024-04-12T21:45:01.279 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:01.278+0000 7fb7e5c00700 1 -- 172.21.15.18:0/3118678773 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb7e0104b80 con 0x7fb7e0109dd0 2024-04-12T21:45:01.280 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:01.279+0000 7fb7d6a00700 1 -- 172.21.15.18:0/3118678773 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7fb7cc013640 con 0x7fb7e0109dd0 2024-04-12T21:45:01.281 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:01.280+0000 7fb7d6a00700 1 --2- 172.21.15.18:0/3118678773 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fb7c406eba0 0x7fb7c4071050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:45:01.281 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:01.280+0000 7fb7d6a00700 1 -- 172.21.15.18:0/3118678773 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 6757+0+0 (secure 0 0 0) 0x7fb7cc090410 con 0x7fb7e0109dd0 2024-04-12T21:45:01.281 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:01.281+0000 7fb7dea00700 1 --2- 172.21.15.18:0/3118678773 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fb7c406eba0 0x7fb7c4071050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:45:01.281 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:01.281+0000 7fb7dea00700 1 --2- 172.21.15.18:0/3118678773 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fb7c406eba0 0x7fb7c4071050 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7fb7d0006fd0 tx=0x7fb7d0008040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:45:01.285 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:01.285+0000 7fb7d6a00700 1 -- 172.21.15.18:0/3118678773 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7fb7cc05cb40 con 0x7fb7e0109dd0 2024-04-12T21:45:01.543 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:01.543+0000 7fb7e5c00700 1 -- 172.21.15.18:0/3118678773 --> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fb7e0110620 con 0x7fb7c406eba0 2024-04-12T21:45:01.545 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:01.545+0000 7fb7d6a00700 1 -- 172.21.15.18:0/3118678773 <== mgr.34102 v2:172.21.15.18:6800/3812891919 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7fb7d000d6e0 con 0x7fb7c406eba0 2024-04-12T21:45:01.550 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:01.547+0000 7fb7e5c00700 1 -- 172.21.15.18:0/3118678773 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fb7c406eba0 msgr2=0x7fb7c4071050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:45:01.550 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:01.547+0000 7fb7e5c00700 1 --2- 172.21.15.18:0/3118678773 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fb7c406eba0 0x7fb7c4071050 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7fb7d0006fd0 tx=0x7fb7d0008040 comp rx=0 tx=0).stop 2024-04-12T21:45:01.550 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:01.547+0000 7fb7e5c00700 1 -- 172.21.15.18:0/3118678773 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb7e0109dd0 msgr2=0x7fb7e011a4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:45:01.550 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:01.547+0000 7fb7e5c00700 1 --2- 172.21.15.18:0/3118678773 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb7e0109dd0 0x7fb7e011a4e0 secure :-1 s=READY pgs=1145 cs=0 l=1 rev1=1 crypto rx=0x7fb7cc009fd0 tx=0x7fb7cc009e30 comp rx=0 tx=0).stop 2024-04-12T21:45:01.550 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:01.547+0000 7fb7e5c00700 1 -- 172.21.15.18:0/3118678773 shutdown_connections 2024-04-12T21:45:01.550 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:01.547+0000 7fb7e5c00700 1 --2- 172.21.15.18:0/3118678773 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fb7c406eba0 0x7fb7c4071050 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:45:01.550 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:01.547+0000 7fb7e5c00700 1 --2- 172.21.15.18:0/3118678773 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb7e0109430 0x7fb7e0119fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:45:01.550 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:01.547+0000 7fb7e5c00700 1 --2- 172.21.15.18:0/3118678773 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb7e0109dd0 0x7fb7e011a4e0 unknown :-1 s=CLOSED pgs=1145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:45:01.550 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:01.547+0000 7fb7e5c00700 1 -- 172.21.15.18:0/3118678773 >> 172.21.15.18:0/3118678773 conn(0x7fb7e0072c00 msgr2=0x7fb7e0074b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:45:01.550 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:01.548+0000 7fb7e5c00700 1 -- 172.21.15.18:0/3118678773 shutdown_connections 2024-04-12T21:45:01.551 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:01.548+0000 7fb7e5c00700 1 -- 172.21.15.18:0/3118678773 wait complete. 2024-04-12T21:45:01.707 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:01.706+0000 7fbbc0000700 1 -- 172.21.15.18:0/1891555994 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fbbb81038e0 msgr2=0x7fbbb80fce40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:45:01.707 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:01.706+0000 7fbbc0000700 1 --2- 172.21.15.18:0/1891555994 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fbbb81038e0 0x7fbbb80fce40 secure :-1 s=READY pgs=1146 cs=0 l=1 rev1=1 crypto rx=0x7fbba4004940 tx=0x7fbba4009e30 comp rx=0 tx=0).stop 2024-04-12T21:45:01.707 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:01.706+0000 7fbbc0000700 1 -- 172.21.15.18:0/1891555994 shutdown_connections 2024-04-12T21:45:01.707 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:01.706+0000 7fbbc0000700 1 --2- 172.21.15.18:0/1891555994 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fbbb81038e0 0x7fbbb80fce40 unknown :-1 s=CLOSED pgs=1146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:45:01.707 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:01.706+0000 7fbbc0000700 1 --2- 172.21.15.18:0/1891555994 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbbb8102f40 0x7fbbb8103310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:45:01.707 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:01.706+0000 7fbbc0000700 1 -- 172.21.15.18:0/1891555994 >> 172.21.15.18:0/1891555994 conn(0x7fbbb80f8980 msgr2=0x7fbbb80fad90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:45:01.707 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:01.707+0000 7fbbc0000700 1 -- 172.21.15.18:0/1891555994 shutdown_connections 2024-04-12T21:45:01.708 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:01.707+0000 7fbbc0000700 1 -- 172.21.15.18:0/1891555994 wait complete. 2024-04-12T21:45:01.709 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:01.708+0000 7fbbc0000700 1 Processor -- start 2024-04-12T21:45:01.709 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:01.708+0000 7fbbc0000700 1 -- start start 2024-04-12T21:45:01.709 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:01.708+0000 7fbbc0000700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fbbb8102f40 0x7fbbb80702c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:45:01.709 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:01.708+0000 7fbbc0000700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbbb81038e0 0x7fbbb8070800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:45:01.709 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:01.708+0000 7fbbc0000700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbbb8072210 con 0x7fbbb8102f40 2024-04-12T21:45:01.709 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:01.708+0000 7fbbc0000700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbbb8070d40 con 0x7fbbb81038e0 2024-04-12T21:45:01.709 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:01.709+0000 7fbbbd600700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fbbb8102f40 0x7fbbb80702c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:45:01.709 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:01.709+0000 7fbbbd600700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fbbb8102f40 0x7fbbb80702c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:58154/0 (socket says 172.21.15.18:58154) 2024-04-12T21:45:01.709 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:01.709+0000 7fbbbd600700 1 -- 172.21.15.18:0/4026678768 learned_addr learned my addr 172.21.15.18:0/4026678768 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:45:01.711 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:01.709+0000 7fbbbcc00700 1 --2- 172.21.15.18:0/4026678768 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbbb81038e0 0x7fbbb8070800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:45:01.711 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:01.709+0000 7fbbbd600700 1 -- 172.21.15.18:0/4026678768 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbbb81038e0 msgr2=0x7fbbb8070800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:45:01.711 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:01.709+0000 7fbbbd600700 1 --2- 172.21.15.18:0/4026678768 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbbb81038e0 0x7fbbb8070800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:45:01.711 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:01.709+0000 7fbbbd600700 1 -- 172.21.15.18:0/4026678768 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbba4005040 con 0x7fbbb8102f40 2024-04-12T21:45:01.711 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:01.709+0000 7fbbbd600700 1 --2- 172.21.15.18:0/4026678768 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fbbb8102f40 0x7fbbb80702c0 secure :-1 s=READY pgs=1147 cs=0 l=1 rev1=1 crypto rx=0x7fbbac002a50 tx=0x7fbbac0046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:45:01.712 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:01.710+0000 7fbbb5600700 1 -- 172.21.15.18:0/4026678768 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbbac007cc0 con 0x7fbbb8102f40 2024-04-12T21:45:01.712 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:01.710+0000 7fbbb5600700 1 -- 172.21.15.18:0/4026678768 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fbbac00f040 con 0x7fbbb8102f40 2024-04-12T21:45:01.712 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:01.710+0000 7fbbb5600700 1 -- 172.21.15.18:0/4026678768 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbbac009640 con 0x7fbbb8102f40 2024-04-12T21:45:01.712 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:01.710+0000 7fbbc0000700 1 -- 172.21.15.18:0/4026678768 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbbb8071020 con 0x7fbbb8102f40 2024-04-12T21:45:01.712 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:01.710+0000 7fbbc0000700 1 -- 172.21.15.18:0/4026678768 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbbb811a260 con 0x7fbbb8102f40 2024-04-12T21:45:01.712 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:01.711+0000 7fbbc0000700 1 -- 172.21.15.18:0/4026678768 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbbb804c710 con 0x7fbbb8102f40 2024-04-12T21:45:01.712 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:01.712+0000 7fbbb5600700 1 -- 172.21.15.18:0/4026678768 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7fbbac0097a0 con 0x7fbbb8102f40 2024-04-12T21:45:01.713 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:01.713+0000 7fbbb5600700 1 --2- 172.21.15.18:0/4026678768 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fbba006ec00 0x7fbba00710b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:45:01.713 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:01.713+0000 7fbbb5600700 1 -- 172.21.15.18:0/4026678768 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 6757+0+0 (secure 0 0 0) 0x7fbbac08e740 con 0x7fbbb8102f40 2024-04-12T21:45:01.713 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:01.713+0000 7fbbbcc00700 1 --2- 172.21.15.18:0/4026678768 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fbba006ec00 0x7fbba00710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:45:01.714 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:01.714+0000 7fbbbcc00700 1 --2- 172.21.15.18:0/4026678768 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fbba006ec00 0x7fbba00710b0 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7fbba4000c00 tx=0x7fbba4009e30 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:45:01.718 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:01.717+0000 7fbbb5600700 1 -- 172.21.15.18:0/4026678768 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7fbbac0573c0 con 0x7fbbb8102f40 2024-04-12T21:45:01.956 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:01.955+0000 7fbbc0000700 1 -- 172.21.15.18:0/4026678768 --> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fbbb80fe700 con 0x7fbba006ec00 2024-04-12T21:45:01.973 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:01.973+0000 7fbbb5600700 1 -- 172.21.15.18:0/4026678768 <== mgr.34102 v2:172.21.15.18:6800/3812891919 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7fbbb80fe700 con 0x7fbba006ec00 2024-04-12T21:45:01.974 INFO:teuthology.orchestra.run.smithi018.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-12T21:45:01.974 INFO:teuthology.orchestra.run.smithi018.stdout:alertmanager.smithi018 smithi018 *:9093,9094 running (45m) 6m ago 60m 14.8M - 0.25.0 c8568f914cd2 c332090068e8 2024-04-12T21:45:01.974 INFO:teuthology.orchestra.run.smithi018.stdout:crash.smithi018 smithi018 running (41m) 6m ago 60m 6665k - 19.0.0-2879-g324baf8d 8717720dc5e5 b7f80d1f7982 2024-04-12T21:45:01.974 INFO:teuthology.orchestra.run.smithi018.stdout:crash.smithi186 smithi186 running (41m) 6m ago 58m 6660k - 19.0.0-2879-g324baf8d 8717720dc5e5 a4f55c81b623 2024-04-12T21:45:01.974 INFO:teuthology.orchestra.run.smithi018.stdout:grafana.smithi018 smithi018 *:3000 running (45m) 6m ago 59m 73.8M - 9.4.12 2bacad6d85d8 b08e3bf176dd 2024-04-12T21:45:01.974 INFO:teuthology.orchestra.run.smithi018.stdout:mds.cephfs.smithi018.acqbsc smithi018 running (37m) 6m ago 52m 21.9M - 19.0.0-2879-g324baf8d 8717720dc5e5 021f99ab24ae 2024-04-12T21:45:01.974 INFO:teuthology.orchestra.run.smithi018.stdout:mds.cephfs.smithi018.szjluo smithi018 running (37m) 6m ago 52m 21.7M - 19.0.0-2879-g324baf8d 8717720dc5e5 0003117479d4 2024-04-12T21:45:01.974 INFO:teuthology.orchestra.run.smithi018.stdout:mds.cephfs.smithi186.jdllyn smithi186 running (22m) 6m ago 52m 16.5M - 19.0.0-2879-g324baf8d 8717720dc5e5 21b1f46eef23 2024-04-12T21:45:01.975 INFO:teuthology.orchestra.run.smithi018.stdout:mds.cephfs.smithi186.wjnafs smithi186 running (6m) 6m ago 52m 11.0M - 19.0.0-2879-g324baf8d 8717720dc5e5 3cf1ef4148e2 2024-04-12T21:45:01.975 INFO:teuthology.orchestra.run.smithi018.stdout:mgr.smithi018.xjzwga smithi018 *:8443,9283,8765 running (48m) 6m ago 61m 517M - 19.0.0-2879-g324baf8d 8717720dc5e5 56088305eab7 2024-04-12T21:45:01.975 INFO:teuthology.orchestra.run.smithi018.stdout:mgr.smithi186.kqcfxk smithi186 *:8443,9283,8765 running (47m) 6m ago 58m 460M - 19.0.0-2879-g324baf8d 8717720dc5e5 09cd766e5e0d 2024-04-12T21:45:01.975 INFO:teuthology.orchestra.run.smithi018.stdout:mon.smithi018 smithi018 running (43m) 6m ago 62m 88.2M 2048M 19.0.0-2879-g324baf8d 8717720dc5e5 df3e201dfed6 2024-04-12T21:45:01.975 INFO:teuthology.orchestra.run.smithi018.stdout:mon.smithi186 smithi186 running (42m) 6m ago 58m 53.6M 2048M 19.0.0-2879-g324baf8d 8717720dc5e5 c77498da9b2b 2024-04-12T21:45:01.975 INFO:teuthology.orchestra.run.smithi018.stdout:node-exporter.smithi018 smithi018 *:9100 running (46m) 6m ago 60m 8468k - 1.5.0 0da6a335fe13 326c3d9c6847 2024-04-12T21:45:01.975 INFO:teuthology.orchestra.run.smithi018.stdout:node-exporter.smithi186 smithi186 *:9100 running (46m) 6m ago 58m 8166k - 1.5.0 0da6a335fe13 ec2112baeecf 2024-04-12T21:45:01.975 INFO:teuthology.orchestra.run.smithi018.stdout:osd.0 smithi018 running (41m) 6m ago 57m 123M 1559M 19.0.0-2879-g324baf8d 8717720dc5e5 60b1273ca047 2024-04-12T21:45:01.975 INFO:teuthology.orchestra.run.smithi018.stdout:osd.1 smithi018 running (40m) 6m ago 57m 95.4M 1559M 19.0.0-2879-g324baf8d 8717720dc5e5 170ce1e5fb8d 2024-04-12T21:45:01.975 INFO:teuthology.orchestra.run.smithi018.stdout:osd.2 smithi018 running (40m) 6m ago 56m 94.6M 1559M 19.0.0-2879-g324baf8d 8717720dc5e5 c2cbc6aa2839 2024-04-12T21:45:01.975 INFO:teuthology.orchestra.run.smithi018.stdout:osd.3 smithi186 running (39m) 6m ago 55m 101M 2581M 19.0.0-2879-g324baf8d 8717720dc5e5 0138e8262e4b 2024-04-12T21:45:01.975 INFO:teuthology.orchestra.run.smithi018.stdout:osd.4 smithi186 running (38m) 6m ago 55m 91.5M 2581M 19.0.0-2879-g324baf8d 8717720dc5e5 b20cb63b25c9 2024-04-12T21:45:01.976 INFO:teuthology.orchestra.run.smithi018.stdout:osd.5 smithi186 running (38m) 6m ago 54m 92.2M 2581M 19.0.0-2879-g324baf8d 8717720dc5e5 2513bea2610e 2024-04-12T21:45:01.976 INFO:teuthology.orchestra.run.smithi018.stdout:prometheus.smithi018 smithi018 *:9095 running (46m) 6m ago 59m 69.6M - 2.43.0 a07b618ecd1d f58841127e21 2024-04-12T21:45:01.976 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:01.976+0000 7fbbc0000700 1 -- 172.21.15.18:0/4026678768 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fbba006ec00 msgr2=0x7fbba00710b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:45:01.977 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:01.976+0000 7fbbc0000700 1 --2- 172.21.15.18:0/4026678768 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fbba006ec00 0x7fbba00710b0 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7fbba4000c00 tx=0x7fbba4009e30 comp rx=0 tx=0).stop 2024-04-12T21:45:01.977 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:01.976+0000 7fbbc0000700 1 -- 172.21.15.18:0/4026678768 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fbbb8102f40 msgr2=0x7fbbb80702c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:45:01.977 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:01.976+0000 7fbbc0000700 1 --2- 172.21.15.18:0/4026678768 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fbbb8102f40 0x7fbbb80702c0 secure :-1 s=READY pgs=1147 cs=0 l=1 rev1=1 crypto rx=0x7fbbac002a50 tx=0x7fbbac0046b0 comp rx=0 tx=0).stop 2024-04-12T21:45:01.977 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:01.976+0000 7fbbc0000700 1 -- 172.21.15.18:0/4026678768 shutdown_connections 2024-04-12T21:45:01.977 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:01.976+0000 7fbbc0000700 1 --2- 172.21.15.18:0/4026678768 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fbba006ec00 0x7fbba00710b0 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:45:01.977 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:01.976+0000 7fbbc0000700 1 --2- 172.21.15.18:0/4026678768 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fbbb8102f40 0x7fbbb80702c0 unknown :-1 s=CLOSED pgs=1147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:45:01.977 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:01.976+0000 7fbbc0000700 1 --2- 172.21.15.18:0/4026678768 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbbb81038e0 0x7fbbb8070800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:45:01.977 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:01.976+0000 7fbbc0000700 1 -- 172.21.15.18:0/4026678768 >> 172.21.15.18:0/4026678768 conn(0x7fbbb80f8980 msgr2=0x7fbbb80f99b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:45:01.978 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:01.977+0000 7fbbc0000700 1 -- 172.21.15.18:0/4026678768 shutdown_connections 2024-04-12T21:45:01.978 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:01.977+0000 7fbbc0000700 1 -- 172.21.15.18:0/4026678768 wait complete. 2024-04-12T21:45:02.134 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:02.133+0000 7f19cbe00700 1 -- 172.21.15.18:0/2172144042 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f19cc109c40 msgr2=0x7f19cc1032b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:45:02.134 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:02.133+0000 7f19cbe00700 1 --2- 172.21.15.18:0/2172144042 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f19cc109c40 0x7f19cc1032b0 secure :-1 s=READY pgs=1148 cs=0 l=1 rev1=1 crypto rx=0x7f19b8004940 tx=0x7f19b8009e30 comp rx=0 tx=0).stop 2024-04-12T21:45:02.134 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:02.133+0000 7f19cbe00700 1 -- 172.21.15.18:0/2172144042 shutdown_connections 2024-04-12T21:45:02.134 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:02.133+0000 7f19cbe00700 1 --2- 172.21.15.18:0/2172144042 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f19cc109c40 0x7f19cc1032b0 unknown :-1 s=CLOSED pgs=1148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:45:02.134 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:02.133+0000 7f19cbe00700 1 --2- 172.21.15.18:0/2172144042 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f19cc1092a0 0x7f19cc109670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:45:02.134 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:02.133+0000 7f19cbe00700 1 -- 172.21.15.18:0/2172144042 >> 172.21.15.18:0/2172144042 conn(0x7f19cc072a30 msgr2=0x7f19cc074e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:45:02.134 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:02.133+0000 7f19cbe00700 1 -- 172.21.15.18:0/2172144042 shutdown_connections 2024-04-12T21:45:02.134 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:02.133+0000 7f19cbe00700 1 -- 172.21.15.18:0/2172144042 wait complete. 2024-04-12T21:45:02.135 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:02.134+0000 7f19cbe00700 1 Processor -- start 2024-04-12T21:45:02.135 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:02.134+0000 7f19cbe00700 1 -- start start 2024-04-12T21:45:02.135 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:02.135+0000 7f19cbe00700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f19cc1092a0 0x7f19cc119fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:45:02.136 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:02.135+0000 7f19cbe00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f19cc109c40 0x7f19cc11a500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:45:02.136 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:02.135+0000 7f19cbe00700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19cc11abe0 con 0x7f19cc109c40 2024-04-12T21:45:02.136 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:02.135+0000 7f19cbe00700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19cc113f20 con 0x7f19cc1092a0 2024-04-12T21:45:02.136 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:02.135+0000 7f19ca000700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f19cc109c40 0x7f19cc11a500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:45:02.136 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:02.135+0000 7f19ca000700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f19cc109c40 0x7f19cc11a500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:58172/0 (socket says 172.21.15.18:58172) 2024-04-12T21:45:02.136 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:02.135+0000 7f19ca000700 1 -- 172.21.15.18:0/2525010972 learned_addr learned my addr 172.21.15.18:0/2525010972 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:45:02.137 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:02.136+0000 7f19caa00700 1 --2- 172.21.15.18:0/2525010972 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f19cc1092a0 0x7f19cc119fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:45:02.137 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:02.136+0000 7f19ca000700 1 -- 172.21.15.18:0/2525010972 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f19cc1092a0 msgr2=0x7f19cc119fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:45:02.137 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:02.136+0000 7f19ca000700 1 --2- 172.21.15.18:0/2525010972 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f19cc1092a0 0x7f19cc119fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:45:02.137 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:02.136+0000 7f19ca000700 1 -- 172.21.15.18:0/2525010972 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19b8005040 con 0x7f19cc109c40 2024-04-12T21:45:02.137 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:02.136+0000 7f19caa00700 1 --2- 172.21.15.18:0/2525010972 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f19cc1092a0 0x7f19cc119fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-12T21:45:02.137 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:02.136+0000 7f19ca000700 1 --2- 172.21.15.18:0/2525010972 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f19cc109c40 0x7f19cc11a500 secure :-1 s=READY pgs=1149 cs=0 l=1 rev1=1 crypto rx=0x7f19b8009100 tx=0x7f19b8009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:45:02.138 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:02.136+0000 7f19c2a00700 1 -- 172.21.15.18:0/2525010972 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f19b800e040 con 0x7f19cc109c40 2024-04-12T21:45:02.138 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:02.136+0000 7f19c2a00700 1 -- 172.21.15.18:0/2525010972 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f19b8013420 con 0x7f19cc109c40 2024-04-12T21:45:02.138 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:02.136+0000 7f19cbe00700 1 -- 172.21.15.18:0/2525010972 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f19cc114140 con 0x7f19cc109c40 2024-04-12T21:45:02.138 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:02.136+0000 7f19c2a00700 1 -- 172.21.15.18:0/2525010972 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f19b801c630 con 0x7f19cc109c40 2024-04-12T21:45:02.138 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:02.137+0000 7f19cbe00700 1 -- 172.21.15.18:0/2525010972 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f19cc114700 con 0x7f19cc109c40 2024-04-12T21:45:02.138 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:02.138+0000 7f19cbe00700 1 -- 172.21.15.18:0/2525010972 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f19cc1049f0 con 0x7f19cc109c40 2024-04-12T21:45:02.139 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:02.139+0000 7f19c2a00700 1 -- 172.21.15.18:0/2525010972 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7f19b8013590 con 0x7f19cc109c40 2024-04-12T21:45:02.140 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:02.140+0000 7f19c2a00700 1 --2- 172.21.15.18:0/2525010972 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f19b006ec00 0x7f19b00710b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:45:02.140 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:02.140+0000 7f19c2a00700 1 -- 172.21.15.18:0/2525010972 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 6757+0+0 (secure 0 0 0) 0x7f19b808fae0 con 0x7f19cc109c40 2024-04-12T21:45:02.140 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:02.140+0000 7f19caa00700 1 --2- 172.21.15.18:0/2525010972 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f19b006ec00 0x7f19b00710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:45:02.141 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:02.141+0000 7f19caa00700 1 --2- 172.21.15.18:0/2525010972 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f19b006ec00 0x7f19b00710b0 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f19bc006fd0 tx=0x7f19bc008040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:45:02.144 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:02.144+0000 7f19c2a00700 1 -- 172.21.15.18:0/2525010972 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f19b805c3d0 con 0x7f19cc109c40 2024-04-12T21:45:02.355 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:45:02 smithi018 ceph-mon[112590]: pgmap v1360: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:45:02.469 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:02.468+0000 7f19cbe00700 1 -- 172.21.15.18:0/2525010972 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f19cc04bed0 con 0x7f19cc109c40 2024-04-12T21:45:02.469 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:02.469+0000 7f19c2a00700 1 -- 172.21.15.18:0/2525010972 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+623 (secure 0 0 0) 0x7f19b8058070 con 0x7f19cc109c40 2024-04-12T21:45:02.470 INFO:teuthology.orchestra.run.smithi018.stdout:{ 2024-04-12T21:45:02.470 INFO:teuthology.orchestra.run.smithi018.stdout: "mon": { 2024-04-12T21:45:02.470 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 19.0.0-2879-g324baf8d (324baf8ddb91dea46cac50cb535c306115791355) squid (dev)": 2 2024-04-12T21:45:02.470 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-12T21:45:02.470 INFO:teuthology.orchestra.run.smithi018.stdout: "mgr": { 2024-04-12T21:45:02.470 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 19.0.0-2879-g324baf8d (324baf8ddb91dea46cac50cb535c306115791355) squid (dev)": 2 2024-04-12T21:45:02.470 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-12T21:45:02.471 INFO:teuthology.orchestra.run.smithi018.stdout: "osd": { 2024-04-12T21:45:02.471 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 19.0.0-2879-g324baf8d (324baf8ddb91dea46cac50cb535c306115791355) squid (dev)": 6 2024-04-12T21:45:02.471 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-12T21:45:02.471 INFO:teuthology.orchestra.run.smithi018.stdout: "mds": { 2024-04-12T21:45:02.471 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 19.0.0-2879-g324baf8d (324baf8ddb91dea46cac50cb535c306115791355) squid (dev)": 4 2024-04-12T21:45:02.471 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-12T21:45:02.471 INFO:teuthology.orchestra.run.smithi018.stdout: "overall": { 2024-04-12T21:45:02.471 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 19.0.0-2879-g324baf8d (324baf8ddb91dea46cac50cb535c306115791355) squid (dev)": 14 2024-04-12T21:45:02.471 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-04-12T21:45:02.471 INFO:teuthology.orchestra.run.smithi018.stdout:} 2024-04-12T21:45:02.473 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:02.472+0000 7f19cbe00700 1 -- 172.21.15.18:0/2525010972 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f19b006ec00 msgr2=0x7f19b00710b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:45:02.473 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:02.472+0000 7f19cbe00700 1 --2- 172.21.15.18:0/2525010972 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f19b006ec00 0x7f19b00710b0 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f19bc006fd0 tx=0x7f19bc008040 comp rx=0 tx=0).stop 2024-04-12T21:45:02.473 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:02.472+0000 7f19cbe00700 1 -- 172.21.15.18:0/2525010972 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f19cc109c40 msgr2=0x7f19cc11a500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:45:02.473 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:02.472+0000 7f19cbe00700 1 --2- 172.21.15.18:0/2525010972 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f19cc109c40 0x7f19cc11a500 secure :-1 s=READY pgs=1149 cs=0 l=1 rev1=1 crypto rx=0x7f19b8009100 tx=0x7f19b8009e30 comp rx=0 tx=0).stop 2024-04-12T21:45:02.473 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:02.472+0000 7f19cbe00700 1 -- 172.21.15.18:0/2525010972 shutdown_connections 2024-04-12T21:45:02.474 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:02.472+0000 7f19cbe00700 1 --2- 172.21.15.18:0/2525010972 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f19b006ec00 0x7f19b00710b0 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:45:02.474 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:02.472+0000 7f19cbe00700 1 --2- 172.21.15.18:0/2525010972 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f19cc1092a0 0x7f19cc119fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:45:02.474 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:02.472+0000 7f19cbe00700 1 --2- 172.21.15.18:0/2525010972 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f19cc109c40 0x7f19cc11a500 unknown :-1 s=CLOSED pgs=1149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:45:02.474 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:02.472+0000 7f19cbe00700 1 -- 172.21.15.18:0/2525010972 >> 172.21.15.18:0/2525010972 conn(0x7f19cc072a30 msgr2=0x7f19cc074900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:45:02.474 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:02.472+0000 7f19cbe00700 1 -- 172.21.15.18:0/2525010972 shutdown_connections 2024-04-12T21:45:02.474 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:02.473+0000 7f19cbe00700 1 -- 172.21.15.18:0/2525010972 wait complete. 2024-04-12T21:45:02.534 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:45:02 smithi186 ceph-mon[86379]: pgmap v1360: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:45:02.628 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:02.627+0000 7f23d1600700 1 -- 172.21.15.18:0/1344868177 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f23cc070300 msgr2=0x7f23cc0706d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:45:02.629 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:02.627+0000 7f23d1600700 1 --2- 172.21.15.18:0/1344868177 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f23cc070300 0x7f23cc0706d0 secure :-1 s=READY pgs=1150 cs=0 l=1 rev1=1 crypto rx=0x7f23bc004970 tx=0x7f23bc009e30 comp rx=0 tx=0).stop 2024-04-12T21:45:02.629 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:02.628+0000 7f23d1600700 1 -- 172.21.15.18:0/1344868177 shutdown_connections 2024-04-12T21:45:02.629 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:02.628+0000 7f23d1600700 1 --2- 172.21.15.18:0/1344868177 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f23cc070c10 0x7f23cc109190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:45:02.629 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:02.628+0000 7f23d1600700 1 --2- 172.21.15.18:0/1344868177 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f23cc070300 0x7f23cc0706d0 unknown :-1 s=CLOSED pgs=1150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:45:02.629 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:02.628+0000 7f23d1600700 1 -- 172.21.15.18:0/1344868177 >> 172.21.15.18:0/1344868177 conn(0x7f23cc0f8970 msgr2=0x7f23cc0fad80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:45:02.629 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:02.628+0000 7f23d1600700 1 -- 172.21.15.18:0/1344868177 shutdown_connections 2024-04-12T21:45:02.629 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:02.628+0000 7f23d1600700 1 -- 172.21.15.18:0/1344868177 wait complete. 2024-04-12T21:45:02.630 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:02.629+0000 7f23d1600700 1 Processor -- start 2024-04-12T21:45:02.630 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:02.629+0000 7f23d1600700 1 -- start start 2024-04-12T21:45:02.630 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:02.629+0000 7f23d1600700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f23cc070300 0x7f23cc115dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:45:02.630 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:02.630+0000 7f23d1600700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f23cc070c10 0x7f23cc116300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:45:02.630 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:02.630+0000 7f23d1600700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23cc1169e0 con 0x7f23cc070300 2024-04-12T21:45:02.630 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:02.630+0000 7f23d1600700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23cc10fd20 con 0x7f23cc070c10 2024-04-12T21:45:02.631 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:02.630+0000 7f23caa00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f23cc070300 0x7f23cc115dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:45:02.631 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:02.630+0000 7f23caa00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f23cc070300 0x7f23cc115dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:58194/0 (socket says 172.21.15.18:58194) 2024-04-12T21:45:02.631 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:02.630+0000 7f23caa00700 1 -- 172.21.15.18:0/1317389874 learned_addr learned my addr 172.21.15.18:0/1317389874 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:45:02.632 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:02.630+0000 7f23c3e00700 1 --2- 172.21.15.18:0/1317389874 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f23cc070c10 0x7f23cc116300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:45:02.632 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:02.630+0000 7f23caa00700 1 -- 172.21.15.18:0/1317389874 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f23cc070c10 msgr2=0x7f23cc116300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:45:02.632 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:02.630+0000 7f23caa00700 1 --2- 172.21.15.18:0/1317389874 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f23cc070c10 0x7f23cc116300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:45:02.632 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:02.630+0000 7f23caa00700 1 -- 172.21.15.18:0/1317389874 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23bc005040 con 0x7f23cc070300 2024-04-12T21:45:02.632 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:02.630+0000 7f23c3e00700 1 --2- 172.21.15.18:0/1317389874 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f23cc070c10 0x7f23cc116300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-12T21:45:02.633 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:02.631+0000 7f23caa00700 1 --2- 172.21.15.18:0/1317389874 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f23cc070300 0x7f23cc115dc0 secure :-1 s=READY pgs=1151 cs=0 l=1 rev1=1 crypto rx=0x7f23bc004f50 tx=0x7f23bc009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:45:02.633 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:02.631+0000 7f23c2a00700 1 -- 172.21.15.18:0/1317389874 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f23bc00e040 con 0x7f23cc070300 2024-04-12T21:45:02.633 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:02.631+0000 7f23c2a00700 1 -- 172.21.15.18:0/1317389874 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f23bc00bc10 con 0x7f23cc070300 2024-04-12T21:45:02.633 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:02.631+0000 7f23c2a00700 1 -- 172.21.15.18:0/1317389874 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f23bc01c630 con 0x7f23cc070300 2024-04-12T21:45:02.633 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:02.631+0000 7f23d1600700 1 -- 172.21.15.18:0/1317389874 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f23cc10ff40 con 0x7f23cc070300 2024-04-12T21:45:02.633 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:02.631+0000 7f23d1600700 1 -- 172.21.15.18:0/1317389874 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f23cc110480 con 0x7f23cc070300 2024-04-12T21:45:02.633 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:02.632+0000 7f23d1600700 1 -- 172.21.15.18:0/1317389874 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f23cc105fc0 con 0x7f23cc070300 2024-04-12T21:45:02.634 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:02.633+0000 7f23c2a00700 1 -- 172.21.15.18:0/1317389874 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7f23bc013420 con 0x7f23cc070300 2024-04-12T21:45:02.635 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:02.634+0000 7f23c2a00700 1 --2- 172.21.15.18:0/1317389874 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f23b006ec10 0x7f23b00710c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:45:02.635 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:02.635+0000 7f23c2a00700 1 -- 172.21.15.18:0/1317389874 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 6757+0+0 (secure 0 0 0) 0x7f23bc08fbd0 con 0x7f23cc070300 2024-04-12T21:45:02.635 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:02.635+0000 7f23c3e00700 1 --2- 172.21.15.18:0/1317389874 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f23b006ec10 0x7f23b00710c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:45:02.636 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:02.635+0000 7f23c3e00700 1 --2- 172.21.15.18:0/1317389874 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f23b006ec10 0x7f23b00710c0 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f23b4006fd0 tx=0x7f23b4008040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:45:02.640 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:02.639+0000 7f23c2a00700 1 -- 172.21.15.18:0/1317389874 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f23bc05c400 con 0x7f23cc070300 2024-04-12T21:45:02.919 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:02.918+0000 7f23d1600700 1 -- 172.21.15.18:0/1317389874 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f23cc04bed0 con 0x7f23cc070300 2024-04-12T21:45:02.919 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:02.919+0000 7f23c2a00700 1 -- 172.21.15.18:0/1317389874 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 33 v33) v1 ==== 76+0+1876 (secure 0 0 0) 0x7f23bc05bf90 con 0x7f23cc070300 2024-04-12T21:45:02.921 INFO:teuthology.orchestra.run.smithi018.stdout:e33 2024-04-12T21:45:02.921 INFO:teuthology.orchestra.run.smithi018.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-12T21:45:02.921 INFO:teuthology.orchestra.run.smithi018.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-12T21:45:02.921 INFO:teuthology.orchestra.run.smithi018.stdout:legacy client fscid: 1 2024-04-12T21:45:02.921 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-12T21:45:02.921 INFO:teuthology.orchestra.run.smithi018.stdout:Filesystem 'cephfs' (1) 2024-04-12T21:45:02.921 INFO:teuthology.orchestra.run.smithi018.stdout:fs_name cephfs 2024-04-12T21:45:02.921 INFO:teuthology.orchestra.run.smithi018.stdout:epoch 24 2024-04-12T21:45:02.921 INFO:teuthology.orchestra.run.smithi018.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-04-12T21:45:02.921 INFO:teuthology.orchestra.run.smithi018.stdout:created 2024-04-12T20:51:57.362850+0000 2024-04-12T21:45:02.921 INFO:teuthology.orchestra.run.smithi018.stdout:modified 2024-04-12T21:06:59.717190+0000 2024-04-12T21:45:02.922 INFO:teuthology.orchestra.run.smithi018.stdout:tableserver 0 2024-04-12T21:45:02.922 INFO:teuthology.orchestra.run.smithi018.stdout:root 0 2024-04-12T21:45:02.922 INFO:teuthology.orchestra.run.smithi018.stdout:session_timeout 60 2024-04-12T21:45:02.922 INFO:teuthology.orchestra.run.smithi018.stdout:session_autoclose 300 2024-04-12T21:45:02.922 INFO:teuthology.orchestra.run.smithi018.stdout:max_file_size 1099511627776 2024-04-12T21:45:02.922 INFO:teuthology.orchestra.run.smithi018.stdout:max_xattr_size 65536 2024-04-12T21:45:02.922 INFO:teuthology.orchestra.run.smithi018.stdout:required_client_features {} 2024-04-12T21:45:02.922 INFO:teuthology.orchestra.run.smithi018.stdout:last_failure 0 2024-04-12T21:45:02.922 INFO:teuthology.orchestra.run.smithi018.stdout:last_failure_osd_epoch 100 2024-04-12T21:45:02.922 INFO:teuthology.orchestra.run.smithi018.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-12T21:45:02.922 INFO:teuthology.orchestra.run.smithi018.stdout:max_mds 2 2024-04-12T21:45:02.922 INFO:teuthology.orchestra.run.smithi018.stdout:in 0,1 2024-04-12T21:45:02.922 INFO:teuthology.orchestra.run.smithi018.stdout:up {} 2024-04-12T21:45:02.922 INFO:teuthology.orchestra.run.smithi018.stdout:failed 0,1 2024-04-12T21:45:02.923 INFO:teuthology.orchestra.run.smithi018.stdout:damaged 2024-04-12T21:45:02.923 INFO:teuthology.orchestra.run.smithi018.stdout:stopped 2024-04-12T21:45:02.923 INFO:teuthology.orchestra.run.smithi018.stdout:data_pools [3] 2024-04-12T21:45:02.923 INFO:teuthology.orchestra.run.smithi018.stdout:metadata_pool 2 2024-04-12T21:45:02.923 INFO:teuthology.orchestra.run.smithi018.stdout:inline_data disabled 2024-04-12T21:45:02.923 INFO:teuthology.orchestra.run.smithi018.stdout:balancer 2024-04-12T21:45:02.923 INFO:teuthology.orchestra.run.smithi018.stdout:bal_rank_mask -1 2024-04-12T21:45:02.923 INFO:teuthology.orchestra.run.smithi018.stdout:standby_count_wanted 1 2024-04-12T21:45:02.923 INFO:teuthology.orchestra.run.smithi018.stdout:qdb_cluster leader: 0 members: 2024-04-12T21:45:02.923 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-12T21:45:02.923 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-12T21:45:02.923 INFO:teuthology.orchestra.run.smithi018.stdout:Standby daemons: 2024-04-12T21:45:02.923 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-12T21:45:02.923 INFO:teuthology.orchestra.run.smithi018.stdout:[mds.cephfs.smithi018.acqbsc{-1:34436} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.18:6828/755179764,v1:172.21.15.18:6829/755179764] compat {c=[1],r=[1],i=[1fff]}] 2024-04-12T21:45:02.923 INFO:teuthology.orchestra.run.smithi018.stdout:[mds.cephfs.smithi018.szjluo{-1:34468} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.18:6826/2407992653,v1:172.21.15.18:6827/2407992653] compat {c=[1],r=[1],i=[1fff]}] 2024-04-12T21:45:02.924 INFO:teuthology.orchestra.run.smithi018.stdout:[mds.cephfs.smithi186.jdllyn{-1:44595} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.186:6824/3523444202,v1:172.21.15.186:6825/3523444202] compat {c=[1],r=[1],i=[1fff]}] 2024-04-12T21:45:02.924 INFO:teuthology.orchestra.run.smithi018.stdout:[mds.cephfs.smithi186.wjnafs{-1:44875} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.186:6826/3122443766,v1:172.21.15.186:6827/3122443766] compat {c=[1],r=[1],i=[1fff]}] 2024-04-12T21:45:02.924 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:02.922+0000 7f23d1600700 1 -- 172.21.15.18:0/1317389874 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f23b006ec10 msgr2=0x7f23b00710c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:45:02.925 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:02.922+0000 7f23d1600700 1 --2- 172.21.15.18:0/1317389874 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f23b006ec10 0x7f23b00710c0 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f23b4006fd0 tx=0x7f23b4008040 comp rx=0 tx=0).stop 2024-04-12T21:45:02.925 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:02.922+0000 7f23d1600700 1 -- 172.21.15.18:0/1317389874 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f23cc070300 msgr2=0x7f23cc115dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:45:02.925 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:02.922+0000 7f23d1600700 1 --2- 172.21.15.18:0/1317389874 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f23cc070300 0x7f23cc115dc0 secure :-1 s=READY pgs=1151 cs=0 l=1 rev1=1 crypto rx=0x7f23bc004f50 tx=0x7f23bc009e30 comp rx=0 tx=0).stop 2024-04-12T21:45:02.925 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:02.922+0000 7f23d1600700 1 -- 172.21.15.18:0/1317389874 shutdown_connections 2024-04-12T21:45:02.925 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:02.922+0000 7f23d1600700 1 --2- 172.21.15.18:0/1317389874 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f23b006ec10 0x7f23b00710c0 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:45:02.925 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:02.922+0000 7f23d1600700 1 --2- 172.21.15.18:0/1317389874 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f23cc070300 0x7f23cc115dc0 unknown :-1 s=CLOSED pgs=1151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:45:02.925 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:02.922+0000 7f23d1600700 1 --2- 172.21.15.18:0/1317389874 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f23cc070c10 0x7f23cc116300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:45:02.925 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:02.922+0000 7f23d1600700 1 -- 172.21.15.18:0/1317389874 >> 172.21.15.18:0/1317389874 conn(0x7f23cc0f8970 msgr2=0x7f23cc103140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:45:02.925 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:02.922+0000 7f23d1600700 1 -- 172.21.15.18:0/1317389874 shutdown_connections 2024-04-12T21:45:02.925 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:02.922+0000 7f23d1600700 1 -- 172.21.15.18:0/1317389874 wait complete. 2024-04-12T21:45:02.925 INFO:teuthology.orchestra.run.smithi018.stderr:dumped fsmap epoch 33 2024-04-12T21:45:03.087 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:03.086+0000 7ff4de400700 1 -- 172.21.15.18:0/3909755314 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff4d80fe830 msgr2=0x7ff4d8106f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:45:03.087 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:03.086+0000 7ff4de400700 1 --2- 172.21.15.18:0/3909755314 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff4d80fe830 0x7ff4d8106f00 secure :-1 s=READY pgs=1152 cs=0 l=1 rev1=1 crypto rx=0x7ff4c0004940 tx=0x7ff4c0009e30 comp rx=0 tx=0).stop 2024-04-12T21:45:03.087 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:03.086+0000 7ff4de400700 1 -- 172.21.15.18:0/3909755314 shutdown_connections 2024-04-12T21:45:03.087 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:03.086+0000 7ff4de400700 1 --2- 172.21.15.18:0/3909755314 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff4d80fe830 0x7ff4d8106f00 unknown :-1 s=CLOSED pgs=1152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:45:03.087 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:03.086+0000 7ff4de400700 1 --2- 172.21.15.18:0/3909755314 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff4d80fde90 0x7ff4d80fe260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:45:03.087 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:03.086+0000 7ff4de400700 1 -- 172.21.15.18:0/3909755314 >> 172.21.15.18:0/3909755314 conn(0x7ff4d8072bc0 msgr2=0x7ff4d8072fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:45:03.087 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:03.087+0000 7ff4de400700 1 -- 172.21.15.18:0/3909755314 shutdown_connections 2024-04-12T21:45:03.088 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:03.087+0000 7ff4de400700 1 -- 172.21.15.18:0/3909755314 wait complete. 2024-04-12T21:45:03.088 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:03.088+0000 7ff4de400700 1 Processor -- start 2024-04-12T21:45:03.088 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:03.088+0000 7ff4de400700 1 -- start start 2024-04-12T21:45:03.089 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:03.088+0000 7ff4de400700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff4d80fde90 0x7ff4d8113b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:45:03.089 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:03.088+0000 7ff4de400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff4d80fe830 0x7ff4d8114080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:45:03.089 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:03.088+0000 7ff4de400700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4d81146d0 con 0x7ff4d80fe830 2024-04-12T21:45:03.089 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:03.088+0000 7ff4de400700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4d810dac0 con 0x7ff4d80fde90 2024-04-12T21:45:03.089 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:03.089+0000 7ff4cea00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff4d80fe830 0x7ff4d8114080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:45:03.089 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:03.089+0000 7ff4cea00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff4d80fe830 0x7ff4d8114080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:58210/0 (socket says 172.21.15.18:58210) 2024-04-12T21:45:03.089 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:03.089+0000 7ff4cea00700 1 -- 172.21.15.18:0/3112726120 learned_addr learned my addr 172.21.15.18:0/3112726120 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:45:03.090 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:03.089+0000 7ff4d7400700 1 --2- 172.21.15.18:0/3112726120 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff4d80fde90 0x7ff4d8113b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:45:03.091 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:03.089+0000 7ff4cea00700 1 -- 172.21.15.18:0/3112726120 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff4d80fde90 msgr2=0x7ff4d8113b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:45:03.091 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:03.089+0000 7ff4cea00700 1 --2- 172.21.15.18:0/3112726120 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff4d80fde90 0x7ff4d8113b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:45:03.091 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:03.089+0000 7ff4cea00700 1 -- 172.21.15.18:0/3112726120 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4c0005040 con 0x7ff4d80fe830 2024-04-12T21:45:03.091 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:03.090+0000 7ff4cea00700 1 --2- 172.21.15.18:0/3112726120 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff4d80fe830 0x7ff4d8114080 secure :-1 s=READY pgs=1153 cs=0 l=1 rev1=1 crypto rx=0x7ff4c0000c00 tx=0x7ff4c0009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:45:03.091 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:03.090+0000 7ff4cfe00700 1 -- 172.21.15.18:0/3112726120 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff4c0021070 con 0x7ff4d80fe830 2024-04-12T21:45:03.091 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:03.090+0000 7ff4cfe00700 1 -- 172.21.15.18:0/3112726120 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff4c0013cd0 con 0x7ff4d80fe830 2024-04-12T21:45:03.091 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:03.090+0000 7ff4de400700 1 -- 172.21.15.18:0/3112726120 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff4d810dd40 con 0x7ff4d80fe830 2024-04-12T21:45:03.091 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:03.090+0000 7ff4cfe00700 1 -- 172.21.15.18:0/3112726120 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff4c000e070 con 0x7ff4d80fe830 2024-04-12T21:45:03.091 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:03.090+0000 7ff4de400700 1 -- 172.21.15.18:0/3112726120 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff4d810e280 con 0x7ff4d80fe830 2024-04-12T21:45:03.092 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:03.091+0000 7ff4de400700 1 -- 172.21.15.18:0/3112726120 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff4d8101840 con 0x7ff4d80fe830 2024-04-12T21:45:03.093 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:03.092+0000 7ff4cfe00700 1 -- 172.21.15.18:0/3112726120 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7ff4c001ca60 con 0x7ff4d80fe830 2024-04-12T21:45:03.094 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:03.093+0000 7ff4cfe00700 1 --2- 172.21.15.18:0/3112726120 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7ff4bc06ec10 0x7ff4bc0710c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:45:03.094 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:03.094+0000 7ff4cfe00700 1 -- 172.21.15.18:0/3112726120 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 6757+0+0 (secure 0 0 0) 0x7ff4c008fd40 con 0x7ff4d80fe830 2024-04-12T21:45:03.094 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:03.094+0000 7ff4d7400700 1 --2- 172.21.15.18:0/3112726120 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7ff4bc06ec10 0x7ff4bc0710c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:45:03.095 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:03.095+0000 7ff4d7400700 1 --2- 172.21.15.18:0/3112726120 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7ff4bc06ec10 0x7ff4bc0710c0 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7ff4c8009f10 tx=0x7ff4c800b040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:45:03.099 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:03.098+0000 7ff4cfe00700 1 -- 172.21.15.18:0/3112726120 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7ff4c005c570 con 0x7ff4d80fe830 2024-04-12T21:45:03.347 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:03.346+0000 7ff4de400700 1 -- 172.21.15.18:0/3112726120 --> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7ff4d810f190 con 0x7ff4bc06ec10 2024-04-12T21:45:03.347 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:45:03 smithi018 ceph-mon[112590]: from='client.36356 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:45:03.347 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:45:03 smithi018 ceph-mon[112590]: from='client.36360 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:45:03.348 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:45:03 smithi018 ceph-mon[112590]: from='client.36364 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:45:03.348 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:45:03 smithi018 ceph-mon[112590]: from='client.? 172.21.15.18:0/2525010972' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T21:45:03.348 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:45:03 smithi018 ceph-mon[112590]: from='client.? 172.21.15.18:0/1317389874' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-12T21:45:03.348 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:03.348+0000 7ff4cfe00700 1 -- 172.21.15.18:0/3112726120 <== mgr.34102 v2:172.21.15.18:6800/3812891919 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7ff4d810f190 con 0x7ff4bc06ec10 2024-04-12T21:45:03.348 INFO:teuthology.orchestra.run.smithi018.stdout:{ 2024-04-12T21:45:03.349 INFO:teuthology.orchestra.run.smithi018.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:f4452def3be4d350f4a8e5feaf7ebb1511be9298fe68478ba7a2d6fbd7d69a63", 2024-04-12T21:45:03.349 INFO:teuthology.orchestra.run.smithi018.stdout: "in_progress": true, 2024-04-12T21:45:03.349 INFO:teuthology.orchestra.run.smithi018.stdout: "which": "Upgrading all daemon types on all hosts", 2024-04-12T21:45:03.349 INFO:teuthology.orchestra.run.smithi018.stdout: "services_complete": [ 2024-04-12T21:45:03.349 INFO:teuthology.orchestra.run.smithi018.stdout: "mgr", 2024-04-12T21:45:03.349 INFO:teuthology.orchestra.run.smithi018.stdout: "mds", 2024-04-12T21:45:03.349 INFO:teuthology.orchestra.run.smithi018.stdout: "crash", 2024-04-12T21:45:03.349 INFO:teuthology.orchestra.run.smithi018.stdout: "osd", 2024-04-12T21:45:03.349 INFO:teuthology.orchestra.run.smithi018.stdout: "mon" 2024-04-12T21:45:03.349 INFO:teuthology.orchestra.run.smithi018.stdout: ], 2024-04-12T21:45:03.349 INFO:teuthology.orchestra.run.smithi018.stdout: "progress": "16/21 daemons upgraded", 2024-04-12T21:45:03.349 INFO:teuthology.orchestra.run.smithi018.stdout: "message": "Currently upgrading mds daemons", 2024-04-12T21:45:03.349 INFO:teuthology.orchestra.run.smithi018.stdout: "is_paused": false 2024-04-12T21:45:03.349 INFO:teuthology.orchestra.run.smithi018.stdout:} 2024-04-12T21:45:03.351 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:03.350+0000 7ff4de400700 1 -- 172.21.15.18:0/3112726120 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7ff4bc06ec10 msgr2=0x7ff4bc0710c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:45:03.351 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:03.350+0000 7ff4de400700 1 --2- 172.21.15.18:0/3112726120 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7ff4bc06ec10 0x7ff4bc0710c0 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7ff4c8009f10 tx=0x7ff4c800b040 comp rx=0 tx=0).stop 2024-04-12T21:45:03.351 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:03.350+0000 7ff4de400700 1 -- 172.21.15.18:0/3112726120 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff4d80fe830 msgr2=0x7ff4d8114080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:45:03.351 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:03.350+0000 7ff4de400700 1 --2- 172.21.15.18:0/3112726120 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff4d80fe830 0x7ff4d8114080 secure :-1 s=READY pgs=1153 cs=0 l=1 rev1=1 crypto rx=0x7ff4c0000c00 tx=0x7ff4c0009e30 comp rx=0 tx=0).stop 2024-04-12T21:45:03.351 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:03.351+0000 7ff4de400700 1 -- 172.21.15.18:0/3112726120 shutdown_connections 2024-04-12T21:45:03.351 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:03.351+0000 7ff4de400700 1 --2- 172.21.15.18:0/3112726120 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7ff4bc06ec10 0x7ff4bc0710c0 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:45:03.351 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:03.351+0000 7ff4de400700 1 --2- 172.21.15.18:0/3112726120 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff4d80fde90 0x7ff4d8113b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:45:03.352 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:03.351+0000 7ff4de400700 1 --2- 172.21.15.18:0/3112726120 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff4d80fe830 0x7ff4d8114080 unknown :-1 s=CLOSED pgs=1153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:45:03.352 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:03.351+0000 7ff4de400700 1 -- 172.21.15.18:0/3112726120 >> 172.21.15.18:0/3112726120 conn(0x7ff4d8072bc0 msgr2=0x7ff4d80ff3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:45:03.352 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:03.351+0000 7ff4de400700 1 -- 172.21.15.18:0/3112726120 shutdown_connections 2024-04-12T21:45:03.352 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:03.351+0000 7ff4de400700 1 -- 172.21.15.18:0/3112726120 wait complete. 2024-04-12T21:45:03.507 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:03.506+0000 7f8bfa800700 1 -- 172.21.15.18:0/3014722430 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8bf4102de0 msgr2=0x7f8bf41031b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:45:03.507 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:03.506+0000 7f8bfa800700 1 --2- 172.21.15.18:0/3014722430 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8bf4102de0 0x7f8bf41031b0 secure :-1 s=READY pgs=1154 cs=0 l=1 rev1=1 crypto rx=0x7f8be4004940 tx=0x7f8be4009e30 comp rx=0 tx=0).stop 2024-04-12T21:45:03.507 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:03.506+0000 7f8bfa800700 1 -- 172.21.15.18:0/3014722430 shutdown_connections 2024-04-12T21:45:03.507 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:03.506+0000 7f8bfa800700 1 --2- 172.21.15.18:0/3014722430 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8bf4103780 0x7f8bf40fcdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:45:03.507 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:03.506+0000 7f8bfa800700 1 --2- 172.21.15.18:0/3014722430 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8bf4102de0 0x7f8bf41031b0 unknown :-1 s=CLOSED pgs=1154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:45:03.507 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:03.506+0000 7f8bfa800700 1 -- 172.21.15.18:0/3014722430 >> 172.21.15.18:0/3014722430 conn(0x7f8bf40f8970 msgr2=0x7f8bf40fad80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:45:03.507 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:03.506+0000 7f8bfa800700 1 -- 172.21.15.18:0/3014722430 shutdown_connections 2024-04-12T21:45:03.508 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:03.507+0000 7f8bfa800700 1 -- 172.21.15.18:0/3014722430 wait complete. 2024-04-12T21:45:03.508 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:03.508+0000 7f8bfa800700 1 Processor -- start 2024-04-12T21:45:03.508 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:03.508+0000 7f8bfa800700 1 -- start start 2024-04-12T21:45:03.509 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:03.509+0000 7f8bfa800700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8bf4102de0 0x7f8bf4115d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:45:03.509 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:03.509+0000 7f8bfa800700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8bf4103780 0x7f8bf4116240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:45:03.510 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:03.509+0000 7f8bfa800700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8bf4116890 con 0x7f8bf4102de0 2024-04-12T21:45:03.510 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:03.509+0000 7f8bfa800700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8bf410fc60 con 0x7f8bf4103780 2024-04-12T21:45:03.510 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:03.510+0000 7f8bf3400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8bf4102de0 0x7f8bf4115d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:45:03.510 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:03.510+0000 7f8bf3400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8bf4102de0 0x7f8bf4115d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:58230/0 (socket says 172.21.15.18:58230) 2024-04-12T21:45:03.510 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:03.510+0000 7f8bf3400700 1 -- 172.21.15.18:0/3726232229 learned_addr learned my addr 172.21.15.18:0/3726232229 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:45:03.511 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:03.510+0000 7f8bf3400700 1 -- 172.21.15.18:0/3726232229 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8bf4103780 msgr2=0x7f8bf4116240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:45:03.511 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:03.510+0000 7f8bf2a00700 1 --2- 172.21.15.18:0/3726232229 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8bf4103780 0x7f8bf4116240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:45:03.511 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:03.510+0000 7f8bf3400700 1 --2- 172.21.15.18:0/3726232229 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8bf4103780 0x7f8bf4116240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:45:03.511 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:03.510+0000 7f8bf3400700 1 -- 172.21.15.18:0/3726232229 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8be4005040 con 0x7f8bf4102de0 2024-04-12T21:45:03.511 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:03.510+0000 7f8bf3400700 1 --2- 172.21.15.18:0/3726232229 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8bf4102de0 0x7f8bf4115d00 secure :-1 s=READY pgs=1155 cs=0 l=1 rev1=1 crypto rx=0x7f8be4009150 tx=0x7f8be4009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:45:03.511 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:03.511+0000 7f8beb400700 1 -- 172.21.15.18:0/3726232229 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8be400e040 con 0x7f8bf4102de0 2024-04-12T21:45:03.511 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:03.511+0000 7f8beb400700 1 -- 172.21.15.18:0/3726232229 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f8be4013420 con 0x7f8bf4102de0 2024-04-12T21:45:03.512 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:03.511+0000 7f8beb400700 1 -- 172.21.15.18:0/3726232229 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8be401c630 con 0x7f8bf4102de0 2024-04-12T21:45:03.512 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:03.511+0000 7f8bfa800700 1 -- 172.21.15.18:0/3726232229 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8bf410fee0 con 0x7f8bf4102de0 2024-04-12T21:45:03.512 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:03.511+0000 7f8bfa800700 1 -- 172.21.15.18:0/3726232229 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8bf41104a0 con 0x7f8bf4102de0 2024-04-12T21:45:03.512 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:03.512+0000 7f8bfa800700 1 -- 172.21.15.18:0/3726232229 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8bf41072e0 con 0x7f8bf4102de0 2024-04-12T21:45:03.513 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:03.513+0000 7f8beb400700 1 -- 172.21.15.18:0/3726232229 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7f8be4013590 con 0x7f8bf4102de0 2024-04-12T21:45:03.514 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:03.514+0000 7f8beb400700 1 --2- 172.21.15.18:0/3726232229 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f8bd806ec10 0x7f8bd80710c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:45:03.514 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:03.514+0000 7f8beb400700 1 -- 172.21.15.18:0/3726232229 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 6757+0+0 (secure 0 0 0) 0x7f8be408fb40 con 0x7f8bf4102de0 2024-04-12T21:45:03.515 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:03.514+0000 7f8bf2a00700 1 --2- 172.21.15.18:0/3726232229 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f8bd806ec10 0x7f8bd80710c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:45:03.515 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:03.515+0000 7f8bf2a00700 1 --2- 172.21.15.18:0/3726232229 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f8bd806ec10 0x7f8bd80710c0 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f8be0009b80 tx=0x7f8be000b040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:45:03.519 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:03.519+0000 7f8beb400700 1 -- 172.21.15.18:0/3726232229 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f8be405c430 con 0x7f8bf4102de0 2024-04-12T21:45:03.534 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:45:03 smithi186 ceph-mon[86379]: from='client.36356 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:45:03.534 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:45:03 smithi186 ceph-mon[86379]: from='client.36360 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:45:03.534 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:45:03 smithi186 ceph-mon[86379]: from='client.36364 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:45:03.534 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:45:03 smithi186 ceph-mon[86379]: from='client.? 172.21.15.18:0/2525010972' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T21:45:03.534 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:45:03 smithi186 ceph-mon[86379]: from='client.? 172.21.15.18:0/1317389874' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-12T21:45:03.827 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:03.826+0000 7f8bfa800700 1 -- 172.21.15.18:0/3726232229 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f8bf404bed0 con 0x7f8bf4102de0 2024-04-12T21:45:03.828 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:03.827+0000 7f8beb400700 1 -- 172.21.15.18:0/3726232229 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+4303 (secure 0 0 0) 0x7f8be405bfc0 con 0x7f8bf4102de0 2024-04-12T21:45:03.828 INFO:teuthology.orchestra.run.smithi018.stdout:HEALTH_ERR failed to probe daemons or devices; 1 filesystem is degraded; 1 filesystem is offline 2024-04-12T21:45:03.828 INFO:teuthology.orchestra.run.smithi018.stdout:[WRN] CEPHADM_REFRESH_FAILED: failed to probe daemons or devices 2024-04-12T21:45:03.828 INFO:teuthology.orchestra.run.smithi018.stdout: host smithi186 `cephadm ceph-volume` failed: cephadm exited with an error code: 1, stderr: Inferring config /var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/mon.smithi186/config 2024-04-12T21:45:03.828 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/podman: timeout after 895 seconds 2024-04-12T21:45:03.828 INFO:teuthology.orchestra.run.smithi018.stdout:Non-zero exit code 124 from /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:f4452def3be4d350f4a8e5feaf7ebb1511be9298fe68478ba7a2d6fbd7d69a63 -e NODE_NAME=smithi186 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968:/var/run/ceph:z -v /var/log/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968:/var/log/ceph:z -v /var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmptgom53f9:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:f4452def3be4d350f4a8e5feaf7ebb1511be9298fe68478ba7a2d6fbd7d69a63 inventory --format=json-pretty --filter-for-batch 2024-04-12T21:45:03.828 INFO:teuthology.orchestra.run.smithi018.stdout:Traceback (most recent call last): 2024-04-12T21:45:03.828 INFO:teuthology.orchestra.run.smithi018.stdout: File "/usr/lib64/python3.9/runpy.py", line 197, in _run_module_as_main 2024-04-12T21:45:03.829 INFO:teuthology.orchestra.run.smithi018.stdout: return _run_code(code, main_globals, None, 2024-04-12T21:45:03.829 INFO:teuthology.orchestra.run.smithi018.stdout: File "/usr/lib64/python3.9/runpy.py", line 87, in _run_code 2024-04-12T21:45:03.829 INFO:teuthology.orchestra.run.smithi018.stdout: exec(code, run_globals) 2024-04-12T21:45:03.829 INFO:teuthology.orchestra.run.smithi018.stdout: File "/var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/cephadm.e44f3ed5e34e4ef52eedef233d1190bff004470d8e8005127b5312bff357fa7f/__main__.py", line 5618, in 2024-04-12T21:45:03.829 INFO:teuthology.orchestra.run.smithi018.stdout: File "/var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/cephadm.e44f3ed5e34e4ef52eedef233d1190bff004470d8e8005127b5312bff357fa7f/__main__.py", line 5606, in main 2024-04-12T21:45:03.829 INFO:teuthology.orchestra.run.smithi018.stdout: File "/var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/cephadm.e44f3ed5e34e4ef52eedef233d1190bff004470d8e8005127b5312bff357fa7f/__main__.py", line 408, in _infer_config 2024-04-12T21:45:03.829 INFO:teuthology.orchestra.run.smithi018.stdout: File "/var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/cephadm.e44f3ed5e34e4ef52eedef233d1190bff004470d8e8005127b5312bff357fa7f/__main__.py", line 323, in _infer_fsid 2024-04-12T21:45:03.829 INFO:teuthology.orchestra.run.smithi018.stdout: File "/var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/cephadm.e44f3ed5e34e4ef52eedef233d1190bff004470d8e8005127b5312bff357fa7f/__main__.py", line 436, in _infer_image 2024-04-12T21:45:03.829 INFO:teuthology.orchestra.run.smithi018.stdout: File "/var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/cephadm.e44f3ed5e34e4ef52eedef233d1190bff004470d8e8005127b5312bff357fa7f/__main__.py", line 310, in _validate_fsid 2024-04-12T21:45:03.829 INFO:teuthology.orchestra.run.smithi018.stdout: File "/var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/cephadm.e44f3ed5e34e4ef52eedef233d1190bff004470d8e8005127b5312bff357fa7f/__main__.py", line 3359, in command_ceph_volume 2024-04-12T21:45:03.829 INFO:teuthology.orchestra.run.smithi018.stdout: File "/var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/cephadm.e44f3ed5e34e4ef52eedef233d1190bff004470d8e8005127b5312bff357fa7f/cephadmlib/call_wrappers.py", line 310, in call_throws 2024-04-12T21:45:03.829 INFO:teuthology.orchestra.run.smithi018.stdout:RuntimeError: Failed command: /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:f4452def3be4d350f4a8e5feaf7ebb1511be9298fe68478ba7a2d6fbd7d69a63 -e NODE_NAME=smithi186 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968:/var/run/ceph:z -v /var/log/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968:/var/log/ceph:z -v /var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmptgom53f9:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:f4452def3be4d350f4a8e5feaf7ebb1511be9298fe68478ba7a2d6fbd7d69a63 inventory --format=json-pretty --filter-for-batch 2024-04-12T21:45:03.829 INFO:teuthology.orchestra.run.smithi018.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-04-12T21:45:03.829 INFO:teuthology.orchestra.run.smithi018.stdout: fs cephfs is degraded 2024-04-12T21:45:03.830 INFO:teuthology.orchestra.run.smithi018.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-04-12T21:45:03.830 INFO:teuthology.orchestra.run.smithi018.stdout: fs cephfs is offline because no MDS is active for it. 2024-04-12T21:45:03.831 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:03.830+0000 7f8bfa800700 1 -- 172.21.15.18:0/3726232229 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f8bd806ec10 msgr2=0x7f8bd80710c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:45:03.831 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:03.830+0000 7f8bfa800700 1 --2- 172.21.15.18:0/3726232229 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f8bd806ec10 0x7f8bd80710c0 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f8be0009b80 tx=0x7f8be000b040 comp rx=0 tx=0).stop 2024-04-12T21:45:03.831 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:03.830+0000 7f8bfa800700 1 -- 172.21.15.18:0/3726232229 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8bf4102de0 msgr2=0x7f8bf4115d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:45:03.831 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:03.830+0000 7f8bfa800700 1 --2- 172.21.15.18:0/3726232229 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8bf4102de0 0x7f8bf4115d00 secure :-1 s=READY pgs=1155 cs=0 l=1 rev1=1 crypto rx=0x7f8be4009150 tx=0x7f8be4009e30 comp rx=0 tx=0).stop 2024-04-12T21:45:03.831 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:03.830+0000 7f8bfa800700 1 -- 172.21.15.18:0/3726232229 shutdown_connections 2024-04-12T21:45:03.831 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:03.830+0000 7f8bfa800700 1 --2- 172.21.15.18:0/3726232229 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f8bd806ec10 0x7f8bd80710c0 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:45:03.831 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:03.830+0000 7f8bfa800700 1 --2- 172.21.15.18:0/3726232229 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8bf4102de0 0x7f8bf4115d00 unknown :-1 s=CLOSED pgs=1155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:45:03.831 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:03.830+0000 7f8bfa800700 1 --2- 172.21.15.18:0/3726232229 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8bf4103780 0x7f8bf4116240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:45:03.831 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:03.830+0000 7f8bfa800700 1 -- 172.21.15.18:0/3726232229 >> 172.21.15.18:0/3726232229 conn(0x7f8bf40f8970 msgr2=0x7f8bf40fa480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:45:03.831 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:03.830+0000 7f8bfa800700 1 -- 172.21.15.18:0/3726232229 shutdown_connections 2024-04-12T21:45:03.831 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:03.830+0000 7f8bfa800700 1 -- 172.21.15.18:0/3726232229 wait complete. 2024-04-12T21:45:04.105 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:45:04 smithi018 ceph-mon[112590]: pgmap v1361: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:45:04.105 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:45:04 smithi018 ceph-mon[112590]: from='client.? 172.21.15.18:0/3726232229' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-12T21:45:04.534 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:45:04 smithi186 ceph-mon[86379]: pgmap v1361: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:45:04.534 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:45:04 smithi186 ceph-mon[86379]: from='client.? 172.21.15.18:0/3726232229' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-12T21:45:05.355 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:45:05 smithi018 ceph-mon[112590]: from='client.36376 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:45:05.355 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:45:05 smithi018 ceph-mon[112590]: pgmap v1362: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:45:05.534 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:45:05 smithi186 ceph-mon[86379]: from='client.36376 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:45:05.534 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:45:05 smithi186 ceph-mon[86379]: pgmap v1362: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:45:07.605 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:45:07 smithi018 ceph-mon[112590]: pgmap v1363: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:45:07.784 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:45:07 smithi186 ceph-mon[86379]: pgmap v1363: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:45:09.605 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:45:09 smithi018 ceph-mon[112590]: pgmap v1364: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:45:09.784 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:45:09 smithi186 ceph-mon[86379]: pgmap v1364: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:45:10.605 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:45:10 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T21:45:10.784 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:45:10 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T21:45:11.605 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:45:11 smithi018 ceph-mon[112590]: pgmap v1365: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:45:11.784 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:45:11 smithi186 ceph-mon[86379]: pgmap v1365: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:45:13.605 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:45:13 smithi018 ceph-mon[112590]: pgmap v1366: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:45:13.784 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:45:13 smithi186 ceph-mon[86379]: pgmap v1366: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:45:15.605 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:45:15 smithi018 ceph-mon[112590]: pgmap v1367: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:45:15.784 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:45:15 smithi186 ceph-mon[86379]: pgmap v1367: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:45:17.784 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:45:17 smithi186 ceph-mon[86379]: pgmap v1368: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:45:17.855 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:45:17 smithi018 ceph-mon[112590]: pgmap v1368: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:45:19.784 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:45:19 smithi186 ceph-mon[86379]: pgmap v1369: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:45:19.855 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:45:19 smithi018 ceph-mon[112590]: pgmap v1369: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:45:21.784 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:45:21 smithi186 ceph-mon[86379]: pgmap v1370: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:45:21.855 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:45:21 smithi018 ceph-mon[112590]: pgmap v1370: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:45:23.784 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:45:23 smithi186 ceph-mon[86379]: pgmap v1371: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:45:23.855 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:45:23 smithi018 ceph-mon[112590]: pgmap v1371: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:45:25.784 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:45:25 smithi186 ceph-mon[86379]: pgmap v1372: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:45:25.784 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:45:25 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T21:45:25.855 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:45:25 smithi018 ceph-mon[112590]: pgmap v1372: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:45:25.855 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:45:25 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T21:45:27.784 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:45:27 smithi186 ceph-mon[86379]: pgmap v1373: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:45:27.855 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:45:27 smithi018 ceph-mon[112590]: pgmap v1373: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:45:29.784 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:45:29 smithi186 ceph-mon[86379]: pgmap v1374: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:45:29.855 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:45:29 smithi018 ceph-mon[112590]: pgmap v1374: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:45:31.784 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:45:31 smithi186 ceph-mon[86379]: pgmap v1375: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:45:31.855 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:45:31 smithi018 ceph-mon[112590]: pgmap v1375: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:45:33.784 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:45:33 smithi186 ceph-mon[86379]: pgmap v1376: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:45:33.855 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:45:33 smithi018 ceph-mon[112590]: pgmap v1376: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:45:34.003 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:34.001+0000 7f7ce4a00700 1 -- 172.21.15.18:0/2037160548 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7ce0109c40 msgr2=0x7f7ce01032b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:45:34.003 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:34.001+0000 7f7ce4a00700 1 --2- 172.21.15.18:0/2037160548 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7ce0109c40 0x7f7ce01032b0 secure :-1 s=READY pgs=1156 cs=0 l=1 rev1=1 crypto rx=0x7f7cc8004940 tx=0x7f7cc8009e30 comp rx=0 tx=0).stop 2024-04-12T21:45:34.003 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:34.001+0000 7f7ce4a00700 1 -- 172.21.15.18:0/2037160548 shutdown_connections 2024-04-12T21:45:34.003 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:34.001+0000 7f7ce4a00700 1 --2- 172.21.15.18:0/2037160548 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7ce0109c40 0x7f7ce01032b0 unknown :-1 s=CLOSED pgs=1156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:45:34.003 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:34.001+0000 7f7ce4a00700 1 --2- 172.21.15.18:0/2037160548 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7ce01092a0 0x7f7ce0109670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:45:34.003 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:34.001+0000 7f7ce4a00700 1 -- 172.21.15.18:0/2037160548 >> 172.21.15.18:0/2037160548 conn(0x7f7ce0072a30 msgr2=0x7f7ce0074e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:45:34.003 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:34.001+0000 7f7ce4a00700 1 -- 172.21.15.18:0/2037160548 shutdown_connections 2024-04-12T21:45:34.003 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:34.001+0000 7f7ce4a00700 1 -- 172.21.15.18:0/2037160548 wait complete. 2024-04-12T21:45:34.005 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:34.002+0000 7f7ce4a00700 1 Processor -- start 2024-04-12T21:45:34.005 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:34.002+0000 7f7ce4a00700 1 -- start start 2024-04-12T21:45:34.005 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:34.003+0000 7f7ce4a00700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7ce01092a0 0x7f7ce011a010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:45:34.005 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:34.003+0000 7f7ce4a00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7ce0109c40 0x7f7ce011a550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:45:34.005 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:34.003+0000 7f7ce4a00700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ce011aba0 con 0x7f7ce0109c40 2024-04-12T21:45:34.005 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:34.003+0000 7f7ce4a00700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ce0113f70 con 0x7f7ce01092a0 2024-04-12T21:45:34.005 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:34.003+0000 7f7cd5e00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7ce0109c40 0x7f7ce011a550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:45:34.005 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:34.003+0000 7f7cd5e00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7ce0109c40 0x7f7ce011a550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:44600/0 (socket says 172.21.15.18:44600) 2024-04-12T21:45:34.006 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:34.003+0000 7f7cd5e00700 1 -- 172.21.15.18:0/2078193996 learned_addr learned my addr 172.21.15.18:0/2078193996 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:45:34.006 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:34.003+0000 7f7cde800700 1 --2- 172.21.15.18:0/2078193996 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7ce01092a0 0x7f7ce011a010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:45:34.006 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:34.003+0000 7f7cd5e00700 1 -- 172.21.15.18:0/2078193996 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7ce01092a0 msgr2=0x7f7ce011a010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:45:34.006 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:34.003+0000 7f7cd5e00700 1 --2- 172.21.15.18:0/2078193996 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7ce01092a0 0x7f7ce011a010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:45:34.006 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:34.004+0000 7f7cd5e00700 1 -- 172.21.15.18:0/2078193996 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7cc8005040 con 0x7f7ce0109c40 2024-04-12T21:45:34.006 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:34.004+0000 7f7cd5e00700 1 --2- 172.21.15.18:0/2078193996 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7ce0109c40 0x7f7ce011a550 secure :-1 s=READY pgs=1157 cs=0 l=1 rev1=1 crypto rx=0x7f7cc8009100 tx=0x7f7cc8009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:45:34.007 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:34.004+0000 7f7cd7400700 1 -- 172.21.15.18:0/2078193996 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7cc800e040 con 0x7f7ce0109c40 2024-04-12T21:45:34.007 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:34.004+0000 7f7cd7400700 1 -- 172.21.15.18:0/2078193996 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f7cc8004290 con 0x7f7ce0109c40 2024-04-12T21:45:34.007 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:34.004+0000 7f7cd7400700 1 -- 172.21.15.18:0/2078193996 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7cc801c3f0 con 0x7f7ce0109c40 2024-04-12T21:45:34.007 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:34.004+0000 7f7ce4a00700 1 -- 172.21.15.18:0/2078193996 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7ce01141f0 con 0x7f7ce0109c40 2024-04-12T21:45:34.007 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:34.004+0000 7f7ce4a00700 1 -- 172.21.15.18:0/2078193996 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7ce01147b0 con 0x7f7ce0109c40 2024-04-12T21:45:34.007 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:34.006+0000 7f7ce4a00700 1 -- 172.21.15.18:0/2078193996 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7ce01049f0 con 0x7f7ce0109c40 2024-04-12T21:45:34.008 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:34.007+0000 7f7cd7400700 1 -- 172.21.15.18:0/2078193996 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7f7cc8013790 con 0x7f7ce0109c40 2024-04-12T21:45:34.009 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:34.008+0000 7f7cd7400700 1 --2- 172.21.15.18:0/2078193996 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f7cc406ec00 0x7f7cc40710b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:45:34.009 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:34.008+0000 7f7cd7400700 1 -- 172.21.15.18:0/2078193996 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 6757+0+0 (secure 0 0 0) 0x7f7cc808fc30 con 0x7f7ce0109c40 2024-04-12T21:45:34.009 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:34.008+0000 7f7cde800700 1 --2- 172.21.15.18:0/2078193996 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f7cc406ec00 0x7f7cc40710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:45:34.010 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:34.009+0000 7f7cde800700 1 --2- 172.21.15.18:0/2078193996 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f7cc406ec00 0x7f7cc40710b0 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f7cd0009fd0 tx=0x7f7cd0009f20 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:45:34.014 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:34.013+0000 7f7cd7400700 1 -- 172.21.15.18:0/2078193996 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f7cc805c510 con 0x7f7ce0109c40 2024-04-12T21:45:34.265 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:34.263+0000 7f7ce4a00700 1 -- 172.21.15.18:0/2078193996 --> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f7ce0110400 con 0x7f7cc406ec00 2024-04-12T21:45:34.266 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:34.265+0000 7f7cd7400700 1 -- 172.21.15.18:0/2078193996 <== mgr.34102 v2:172.21.15.18:6800/3812891919 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f7ce0110400 con 0x7f7cc406ec00 2024-04-12T21:45:34.269 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:34.267+0000 7f7ce4a00700 1 -- 172.21.15.18:0/2078193996 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f7cc406ec00 msgr2=0x7f7cc40710b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:45:34.269 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:34.267+0000 7f7ce4a00700 1 --2- 172.21.15.18:0/2078193996 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f7cc406ec00 0x7f7cc40710b0 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f7cd0009fd0 tx=0x7f7cd0009f20 comp rx=0 tx=0).stop 2024-04-12T21:45:34.269 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:34.267+0000 7f7ce4a00700 1 -- 172.21.15.18:0/2078193996 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7ce0109c40 msgr2=0x7f7ce011a550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:45:34.269 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:34.267+0000 7f7ce4a00700 1 --2- 172.21.15.18:0/2078193996 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7ce0109c40 0x7f7ce011a550 secure :-1 s=READY pgs=1157 cs=0 l=1 rev1=1 crypto rx=0x7f7cc8009100 tx=0x7f7cc8009e30 comp rx=0 tx=0).stop 2024-04-12T21:45:34.269 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:34.267+0000 7f7ce4a00700 1 -- 172.21.15.18:0/2078193996 shutdown_connections 2024-04-12T21:45:34.269 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:34.267+0000 7f7ce4a00700 1 --2- 172.21.15.18:0/2078193996 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f7cc406ec00 0x7f7cc40710b0 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:45:34.270 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:34.267+0000 7f7ce4a00700 1 --2- 172.21.15.18:0/2078193996 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7ce01092a0 0x7f7ce011a010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:45:34.270 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:34.267+0000 7f7ce4a00700 1 --2- 172.21.15.18:0/2078193996 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7ce0109c40 0x7f7ce011a550 unknown :-1 s=CLOSED pgs=1157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:45:34.270 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:34.267+0000 7f7ce4a00700 1 -- 172.21.15.18:0/2078193996 >> 172.21.15.18:0/2078193996 conn(0x7f7ce0072a30 msgr2=0x7f7ce0074990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:45:34.270 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:34.268+0000 7f7ce4a00700 1 -- 172.21.15.18:0/2078193996 shutdown_connections 2024-04-12T21:45:34.270 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:34.268+0000 7f7ce4a00700 1 -- 172.21.15.18:0/2078193996 wait complete. 2024-04-12T21:45:34.286 INFO:teuthology.orchestra.run.smithi018.stdout:true 2024-04-12T21:45:34.432 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:34.430+0000 7ff361200700 1 -- 172.21.15.18:0/2425567905 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff35c0fe830 msgr2=0x7ff35c106f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:45:34.432 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:34.430+0000 7ff361200700 1 --2- 172.21.15.18:0/2425567905 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff35c0fe830 0x7ff35c106f00 secure :-1 s=READY pgs=1158 cs=0 l=1 rev1=1 crypto rx=0x7ff348004940 tx=0x7ff348009e30 comp rx=0 tx=0).stop 2024-04-12T21:45:34.432 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:34.430+0000 7ff361200700 1 -- 172.21.15.18:0/2425567905 shutdown_connections 2024-04-12T21:45:34.432 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:34.430+0000 7ff361200700 1 --2- 172.21.15.18:0/2425567905 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff35c0fe830 0x7ff35c106f00 unknown :-1 s=CLOSED pgs=1158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:45:34.432 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:34.430+0000 7ff361200700 1 --2- 172.21.15.18:0/2425567905 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff35c0fde90 0x7ff35c0fe260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:45:34.432 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:34.430+0000 7ff361200700 1 -- 172.21.15.18:0/2425567905 >> 172.21.15.18:0/2425567905 conn(0x7ff35c072bc0 msgr2=0x7ff35c072fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:45:34.432 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:34.430+0000 7ff361200700 1 -- 172.21.15.18:0/2425567905 shutdown_connections 2024-04-12T21:45:34.432 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:34.430+0000 7ff361200700 1 -- 172.21.15.18:0/2425567905 wait complete. 2024-04-12T21:45:34.433 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:34.431+0000 7ff361200700 1 Processor -- start 2024-04-12T21:45:34.433 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:34.431+0000 7ff361200700 1 -- start start 2024-04-12T21:45:34.434 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:34.431+0000 7ff361200700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff35c0fde90 0x7ff35c115d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:45:34.434 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:34.431+0000 7ff361200700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff35c0fe830 0x7ff35c1162a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:45:34.434 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:34.431+0000 7ff361200700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff35c1168f0 con 0x7ff35c0fde90 2024-04-12T21:45:34.434 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:34.431+0000 7ff361200700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff35c10fd00 con 0x7ff35c0fe830 2024-04-12T21:45:34.434 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:34.432+0000 7ff359e00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff35c0fde90 0x7ff35c115d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:45:34.434 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:34.432+0000 7ff359e00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff35c0fde90 0x7ff35c115d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:44610/0 (socket says 172.21.15.18:44610) 2024-04-12T21:45:34.434 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:34.432+0000 7ff359e00700 1 -- 172.21.15.18:0/3929518879 learned_addr learned my addr 172.21.15.18:0/3929518879 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:45:34.434 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:34.432+0000 7ff359400700 1 --2- 172.21.15.18:0/3929518879 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff35c0fe830 0x7ff35c1162a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:45:34.434 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:34.432+0000 7ff359e00700 1 -- 172.21.15.18:0/3929518879 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff35c0fe830 msgr2=0x7ff35c1162a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:45:34.434 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:34.432+0000 7ff359e00700 1 --2- 172.21.15.18:0/3929518879 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff35c0fe830 0x7ff35c1162a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:45:34.434 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:34.432+0000 7ff359e00700 1 -- 172.21.15.18:0/3929518879 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff348005040 con 0x7ff35c0fde90 2024-04-12T21:45:34.434 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:34.432+0000 7ff359400700 1 --2- 172.21.15.18:0/3929518879 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff35c0fe830 0x7ff35c1162a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-12T21:45:34.435 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:34.433+0000 7ff359e00700 1 --2- 172.21.15.18:0/3929518879 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff35c0fde90 0x7ff35c115d60 secure :-1 s=READY pgs=1159 cs=0 l=1 rev1=1 crypto rx=0x7ff34c00c830 tx=0x7ff34c004470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:45:34.435 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:34.433+0000 7ff352000700 1 -- 172.21.15.18:0/3929518879 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff34c00a760 con 0x7ff35c0fde90 2024-04-12T21:45:34.435 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:34.433+0000 7ff352000700 1 -- 172.21.15.18:0/3929518879 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff34c00ad90 con 0x7ff35c0fde90 2024-04-12T21:45:34.436 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:34.433+0000 7ff361200700 1 -- 172.21.15.18:0/3929518879 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff35c10ffe0 con 0x7ff35c0fde90 2024-04-12T21:45:34.436 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:34.433+0000 7ff352000700 1 -- 172.21.15.18:0/3929518879 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff34c0113a0 con 0x7ff35c0fde90 2024-04-12T21:45:34.436 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:34.433+0000 7ff361200700 1 -- 172.21.15.18:0/3929518879 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff35c110520 con 0x7ff35c0fde90 2024-04-12T21:45:34.436 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:34.434+0000 7ff361200700 1 -- 172.21.15.18:0/3929518879 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff35c106a20 con 0x7ff35c0fde90 2024-04-12T21:45:34.436 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:34.435+0000 7ff352000700 1 -- 172.21.15.18:0/3929518879 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7ff34c012400 con 0x7ff35c0fde90 2024-04-12T21:45:34.438 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:34.437+0000 7ff352000700 1 --2- 172.21.15.18:0/3929518879 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7ff34006ec10 0x7ff3400710c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:45:34.438 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:34.437+0000 7ff352000700 1 -- 172.21.15.18:0/3929518879 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 6757+0+0 (secure 0 0 0) 0x7ff34c08e9c0 con 0x7ff35c0fde90 2024-04-12T21:45:34.438 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:34.437+0000 7ff359400700 1 --2- 172.21.15.18:0/3929518879 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7ff34006ec10 0x7ff3400710c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:45:34.439 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:34.438+0000 7ff359400700 1 --2- 172.21.15.18:0/3929518879 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7ff34006ec10 0x7ff3400710c0 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7ff348000c00 tx=0x7ff348009e30 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:45:34.443 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:34.441+0000 7ff352000700 1 -- 172.21.15.18:0/3929518879 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7ff34c0576c0 con 0x7ff35c0fde90 2024-04-12T21:45:34.696 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:34.694+0000 7ff361200700 1 -- 172.21.15.18:0/3929518879 --> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7ff35c109f80 con 0x7ff34006ec10 2024-04-12T21:45:34.699 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:34.697+0000 7ff352000700 1 -- 172.21.15.18:0/3929518879 <== mgr.34102 v2:172.21.15.18:6800/3812891919 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7ff35c109f80 con 0x7ff34006ec10 2024-04-12T21:45:34.702 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:34.700+0000 7ff361200700 1 -- 172.21.15.18:0/3929518879 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7ff34006ec10 msgr2=0x7ff3400710c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:45:34.702 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:34.700+0000 7ff361200700 1 --2- 172.21.15.18:0/3929518879 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7ff34006ec10 0x7ff3400710c0 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7ff348000c00 tx=0x7ff348009e30 comp rx=0 tx=0).stop 2024-04-12T21:45:34.702 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:34.700+0000 7ff361200700 1 -- 172.21.15.18:0/3929518879 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff35c0fde90 msgr2=0x7ff35c115d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:45:34.702 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:34.700+0000 7ff361200700 1 --2- 172.21.15.18:0/3929518879 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff35c0fde90 0x7ff35c115d60 secure :-1 s=READY pgs=1159 cs=0 l=1 rev1=1 crypto rx=0x7ff34c00c830 tx=0x7ff34c004470 comp rx=0 tx=0).stop 2024-04-12T21:45:34.702 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:34.700+0000 7ff361200700 1 -- 172.21.15.18:0/3929518879 shutdown_connections 2024-04-12T21:45:34.702 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:34.700+0000 7ff361200700 1 --2- 172.21.15.18:0/3929518879 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7ff34006ec10 0x7ff3400710c0 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:45:34.702 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:34.700+0000 7ff361200700 1 --2- 172.21.15.18:0/3929518879 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff35c0fde90 0x7ff35c115d60 unknown :-1 s=CLOSED pgs=1159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:45:34.702 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:34.700+0000 7ff361200700 1 --2- 172.21.15.18:0/3929518879 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff35c0fe830 0x7ff35c1162a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:45:34.702 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:34.700+0000 7ff361200700 1 -- 172.21.15.18:0/3929518879 >> 172.21.15.18:0/3929518879 conn(0x7ff35c072bc0 msgr2=0x7ff35c0fd530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:45:34.702 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:34.700+0000 7ff361200700 1 -- 172.21.15.18:0/3929518879 shutdown_connections 2024-04-12T21:45:34.702 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:34.700+0000 7ff361200700 1 -- 172.21.15.18:0/3929518879 wait complete. 2024-04-12T21:45:34.855 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:34.854+0000 7fbf8ee00700 1 -- 172.21.15.18:0/2273909214 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fbf88105810 msgr2=0x7fbf880feed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:45:34.856 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:34.854+0000 7fbf8ee00700 1 --2- 172.21.15.18:0/2273909214 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fbf88105810 0x7fbf880feed0 secure :-1 s=READY pgs=1160 cs=0 l=1 rev1=1 crypto rx=0x7fbf74004940 tx=0x7fbf74009e30 comp rx=0 tx=0).stop 2024-04-12T21:45:34.856 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:34.854+0000 7fbf8ee00700 1 -- 172.21.15.18:0/2273909214 shutdown_connections 2024-04-12T21:45:34.856 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:34.854+0000 7fbf8ee00700 1 --2- 172.21.15.18:0/2273909214 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fbf88105810 0x7fbf880feed0 unknown :-1 s=CLOSED pgs=1160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:45:34.856 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:34.854+0000 7fbf8ee00700 1 --2- 172.21.15.18:0/2273909214 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbf88104e70 0x7fbf88105240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:45:34.856 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:34.854+0000 7fbf8ee00700 1 -- 172.21.15.18:0/2273909214 >> 172.21.15.18:0/2273909214 conn(0x7fbf880fab10 msgr2=0x7fbf880fcef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:45:34.856 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:34.854+0000 7fbf8ee00700 1 -- 172.21.15.18:0/2273909214 shutdown_connections 2024-04-12T21:45:34.856 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:34.854+0000 7fbf8ee00700 1 -- 172.21.15.18:0/2273909214 wait complete. 2024-04-12T21:45:34.857 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:34.855+0000 7fbf8ee00700 1 Processor -- start 2024-04-12T21:45:34.857 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:34.855+0000 7fbf8ee00700 1 -- start start 2024-04-12T21:45:34.857 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:34.855+0000 7fbf8ee00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fbf88104e70 0x7fbf88115b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:45:34.857 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:34.855+0000 7fbf8ee00700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbf88105810 0x7fbf88116060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:45:34.857 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:34.855+0000 7fbf8ee00700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf881166b0 con 0x7fbf88104e70 2024-04-12T21:45:34.857 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:34.855+0000 7fbf8ee00700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf8810faa0 con 0x7fbf88105810 2024-04-12T21:45:34.857 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:34.856+0000 7fbf8da00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fbf88104e70 0x7fbf88115b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:45:34.857 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:34.856+0000 7fbf8da00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fbf88104e70 0x7fbf88115b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:44638/0 (socket says 172.21.15.18:44638) 2024-04-12T21:45:34.857 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:34.856+0000 7fbf8da00700 1 -- 172.21.15.18:0/4289571759 learned_addr learned my addr 172.21.15.18:0/4289571759 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:45:34.858 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:34.856+0000 7fbf8da00700 1 -- 172.21.15.18:0/4289571759 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbf88105810 msgr2=0x7fbf88116060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:45:34.858 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:34.856+0000 7fbf8da00700 1 --2- 172.21.15.18:0/4289571759 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbf88105810 0x7fbf88116060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:45:34.858 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:34.856+0000 7fbf8da00700 1 -- 172.21.15.18:0/4289571759 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf74005040 con 0x7fbf88104e70 2024-04-12T21:45:34.859 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:34.856+0000 7fbf8da00700 1 --2- 172.21.15.18:0/4289571759 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fbf88104e70 0x7fbf88115b20 secure :-1 s=READY pgs=1161 cs=0 l=1 rev1=1 crypto rx=0x7fbf7c002a80 tx=0x7fbf7c0046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:45:34.859 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:34.857+0000 7fbf85600700 1 -- 172.21.15.18:0/4289571759 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbf7c007cc0 con 0x7fbf88104e70 2024-04-12T21:45:34.859 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:34.857+0000 7fbf85600700 1 -- 172.21.15.18:0/4289571759 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fbf7c00f040 con 0x7fbf88104e70 2024-04-12T21:45:34.859 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:34.857+0000 7fbf8ee00700 1 -- 172.21.15.18:0/4289571759 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbf8810fd80 con 0x7fbf88104e70 2024-04-12T21:45:34.859 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:34.857+0000 7fbf85600700 1 -- 172.21.15.18:0/4289571759 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbf7c0096f0 con 0x7fbf88104e70 2024-04-12T21:45:34.859 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:34.857+0000 7fbf8ee00700 1 -- 172.21.15.18:0/4289571759 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbf88110340 con 0x7fbf88104e70 2024-04-12T21:45:34.859 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:34.858+0000 7fbf8ee00700 1 -- 172.21.15.18:0/4289571759 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbf88107340 con 0x7fbf88104e70 2024-04-12T21:45:34.860 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:34.859+0000 7fbf85600700 1 -- 172.21.15.18:0/4289571759 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7fbf7c00d690 con 0x7fbf88104e70 2024-04-12T21:45:34.862 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:34.860+0000 7fbf85600700 1 --2- 172.21.15.18:0/4289571759 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fbf7006ec00 0x7fbf700710b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:45:34.862 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:34.860+0000 7fbf85600700 1 -- 172.21.15.18:0/4289571759 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 6757+0+0 (secure 0 0 0) 0x7fbf7c08e200 con 0x7fbf88104e70 2024-04-12T21:45:34.862 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:34.861+0000 7fbf8d000700 1 --2- 172.21.15.18:0/4289571759 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fbf7006ec00 0x7fbf700710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:45:34.863 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:34.861+0000 7fbf8d000700 1 --2- 172.21.15.18:0/4289571759 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fbf7006ec00 0x7fbf700710b0 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7fbf74004f90 tx=0x7fbf74009e30 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:45:34.866 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:34.865+0000 7fbf85600700 1 -- 172.21.15.18:0/4289571759 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7fbf7c05a930 con 0x7fbf88104e70 2024-04-12T21:45:35.098 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:35.097+0000 7fbf8ee00700 1 -- 172.21.15.18:0/4289571759 --> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fbf8810bf50 con 0x7fbf7006ec00 2024-04-12T21:45:35.128 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:35.126+0000 7fbf85600700 1 -- 172.21.15.18:0/4289571759 <== mgr.34102 v2:172.21.15.18:6800/3812891919 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7fbf8810bf50 con 0x7fbf7006ec00 2024-04-12T21:45:35.128 INFO:teuthology.orchestra.run.smithi018.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-12T21:45:35.129 INFO:teuthology.orchestra.run.smithi018.stdout:alertmanager.smithi018 smithi018 *:9093,9094 running (46m) 7m ago 60m 14.8M - 0.25.0 c8568f914cd2 c332090068e8 2024-04-12T21:45:35.129 INFO:teuthology.orchestra.run.smithi018.stdout:crash.smithi018 smithi018 running (42m) 7m ago 60m 6665k - 19.0.0-2879-g324baf8d 8717720dc5e5 b7f80d1f7982 2024-04-12T21:45:35.129 INFO:teuthology.orchestra.run.smithi018.stdout:crash.smithi186 smithi186 running (42m) 7m ago 59m 6660k - 19.0.0-2879-g324baf8d 8717720dc5e5 a4f55c81b623 2024-04-12T21:45:35.129 INFO:teuthology.orchestra.run.smithi018.stdout:grafana.smithi018 smithi018 *:3000 running (45m) 7m ago 60m 73.8M - 9.4.12 2bacad6d85d8 b08e3bf176dd 2024-04-12T21:45:35.129 INFO:teuthology.orchestra.run.smithi018.stdout:mds.cephfs.smithi018.acqbsc smithi018 running (38m) 7m ago 53m 21.9M - 19.0.0-2879-g324baf8d 8717720dc5e5 021f99ab24ae 2024-04-12T21:45:35.129 INFO:teuthology.orchestra.run.smithi018.stdout:mds.cephfs.smithi018.szjluo smithi018 running (37m) 7m ago 53m 21.7M - 19.0.0-2879-g324baf8d 8717720dc5e5 0003117479d4 2024-04-12T21:45:35.129 INFO:teuthology.orchestra.run.smithi018.stdout:mds.cephfs.smithi186.jdllyn smithi186 running (22m) 7m ago 53m 16.5M - 19.0.0-2879-g324baf8d 8717720dc5e5 21b1f46eef23 2024-04-12T21:45:35.129 INFO:teuthology.orchestra.run.smithi018.stdout:mds.cephfs.smithi186.wjnafs smithi186 running (7m) 7m ago 53m 11.0M - 19.0.0-2879-g324baf8d 8717720dc5e5 3cf1ef4148e2 2024-04-12T21:45:35.129 INFO:teuthology.orchestra.run.smithi018.stdout:mgr.smithi018.xjzwga smithi018 *:8443,9283,8765 running (48m) 7m ago 62m 517M - 19.0.0-2879-g324baf8d 8717720dc5e5 56088305eab7 2024-04-12T21:45:35.129 INFO:teuthology.orchestra.run.smithi018.stdout:mgr.smithi186.kqcfxk smithi186 *:8443,9283,8765 running (47m) 7m ago 59m 460M - 19.0.0-2879-g324baf8d 8717720dc5e5 09cd766e5e0d 2024-04-12T21:45:35.129 INFO:teuthology.orchestra.run.smithi018.stdout:mon.smithi018 smithi018 running (44m) 7m ago 62m 88.2M 2048M 19.0.0-2879-g324baf8d 8717720dc5e5 df3e201dfed6 2024-04-12T21:45:35.129 INFO:teuthology.orchestra.run.smithi018.stdout:mon.smithi186 smithi186 running (43m) 7m ago 59m 53.6M 2048M 19.0.0-2879-g324baf8d 8717720dc5e5 c77498da9b2b 2024-04-12T21:45:35.129 INFO:teuthology.orchestra.run.smithi018.stdout:node-exporter.smithi018 smithi018 *:9100 running (47m) 7m ago 60m 8468k - 1.5.0 0da6a335fe13 326c3d9c6847 2024-04-12T21:45:35.130 INFO:teuthology.orchestra.run.smithi018.stdout:node-exporter.smithi186 smithi186 *:9100 running (47m) 7m ago 59m 8166k - 1.5.0 0da6a335fe13 ec2112baeecf 2024-04-12T21:45:35.130 INFO:teuthology.orchestra.run.smithi018.stdout:osd.0 smithi018 running (41m) 7m ago 58m 123M 1559M 19.0.0-2879-g324baf8d 8717720dc5e5 60b1273ca047 2024-04-12T21:45:35.130 INFO:teuthology.orchestra.run.smithi018.stdout:osd.1 smithi018 running (41m) 7m ago 57m 95.4M 1559M 19.0.0-2879-g324baf8d 8717720dc5e5 170ce1e5fb8d 2024-04-12T21:45:35.130 INFO:teuthology.orchestra.run.smithi018.stdout:osd.2 smithi018 running (40m) 7m ago 57m 94.6M 1559M 19.0.0-2879-g324baf8d 8717720dc5e5 c2cbc6aa2839 2024-04-12T21:45:35.130 INFO:teuthology.orchestra.run.smithi018.stdout:osd.3 smithi186 running (40m) 7m ago 56m 101M 2581M 19.0.0-2879-g324baf8d 8717720dc5e5 0138e8262e4b 2024-04-12T21:45:35.130 INFO:teuthology.orchestra.run.smithi018.stdout:osd.4 smithi186 running (39m) 7m ago 55m 91.5M 2581M 19.0.0-2879-g324baf8d 8717720dc5e5 b20cb63b25c9 2024-04-12T21:45:35.130 INFO:teuthology.orchestra.run.smithi018.stdout:osd.5 smithi186 running (38m) 7m ago 55m 92.2M 2581M 19.0.0-2879-g324baf8d 8717720dc5e5 2513bea2610e 2024-04-12T21:45:35.130 INFO:teuthology.orchestra.run.smithi018.stdout:prometheus.smithi018 smithi018 *:9095 running (46m) 7m ago 60m 69.6M - 2.43.0 a07b618ecd1d f58841127e21 2024-04-12T21:45:35.131 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:35.129+0000 7fbf8ee00700 1 -- 172.21.15.18:0/4289571759 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fbf7006ec00 msgr2=0x7fbf700710b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:45:35.131 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:35.129+0000 7fbf8ee00700 1 --2- 172.21.15.18:0/4289571759 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fbf7006ec00 0x7fbf700710b0 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7fbf74004f90 tx=0x7fbf74009e30 comp rx=0 tx=0).stop 2024-04-12T21:45:35.131 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:35.129+0000 7fbf8ee00700 1 -- 172.21.15.18:0/4289571759 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fbf88104e70 msgr2=0x7fbf88115b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:45:35.131 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:35.129+0000 7fbf8ee00700 1 --2- 172.21.15.18:0/4289571759 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fbf88104e70 0x7fbf88115b20 secure :-1 s=READY pgs=1161 cs=0 l=1 rev1=1 crypto rx=0x7fbf7c002a80 tx=0x7fbf7c0046b0 comp rx=0 tx=0).stop 2024-04-12T21:45:35.131 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:35.129+0000 7fbf8ee00700 1 -- 172.21.15.18:0/4289571759 shutdown_connections 2024-04-12T21:45:35.131 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:35.129+0000 7fbf8ee00700 1 --2- 172.21.15.18:0/4289571759 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fbf7006ec00 0x7fbf700710b0 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:45:35.131 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:35.129+0000 7fbf8ee00700 1 --2- 172.21.15.18:0/4289571759 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fbf88104e70 0x7fbf88115b20 unknown :-1 s=CLOSED pgs=1161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:45:35.131 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:35.129+0000 7fbf8ee00700 1 --2- 172.21.15.18:0/4289571759 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbf88105810 0x7fbf88116060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:45:35.132 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:35.129+0000 7fbf8ee00700 1 -- 172.21.15.18:0/4289571759 >> 172.21.15.18:0/4289571759 conn(0x7fbf880fab10 msgr2=0x7fbf880fd770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:45:35.132 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:35.130+0000 7fbf8ee00700 1 -- 172.21.15.18:0/4289571759 shutdown_connections 2024-04-12T21:45:35.132 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:35.130+0000 7fbf8ee00700 1 -- 172.21.15.18:0/4289571759 wait complete. 2024-04-12T21:45:35.290 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:35.288+0000 7f956ec00700 1 -- 172.21.15.18:0/3863719195 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9568104e60 msgr2=0x7f9568105230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:45:35.290 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:35.288+0000 7f956ec00700 1 --2- 172.21.15.18:0/3863719195 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9568104e60 0x7f9568105230 secure :-1 s=READY pgs=1162 cs=0 l=1 rev1=1 crypto rx=0x7f955c004940 tx=0x7f955c009e30 comp rx=0 tx=0).stop 2024-04-12T21:45:35.290 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:35.288+0000 7f956ec00700 1 -- 172.21.15.18:0/3863719195 shutdown_connections 2024-04-12T21:45:35.290 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:35.288+0000 7f956ec00700 1 --2- 172.21.15.18:0/3863719195 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9568105800 0x7f95680fee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:45:35.290 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:35.288+0000 7f956ec00700 1 --2- 172.21.15.18:0/3863719195 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9568104e60 0x7f9568105230 unknown :-1 s=CLOSED pgs=1162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:45:35.290 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:35.288+0000 7f956ec00700 1 -- 172.21.15.18:0/3863719195 >> 172.21.15.18:0/3863719195 conn(0x7f95680fab10 msgr2=0x7f95680fcef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:45:35.290 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:35.288+0000 7f956ec00700 1 -- 172.21.15.18:0/3863719195 shutdown_connections 2024-04-12T21:45:35.290 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:35.288+0000 7f956ec00700 1 -- 172.21.15.18:0/3863719195 wait complete. 2024-04-12T21:45:35.291 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:35.289+0000 7f956ec00700 1 Processor -- start 2024-04-12T21:45:35.291 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:35.289+0000 7f956ec00700 1 -- start start 2024-04-12T21:45:35.291 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:35.290+0000 7f956ec00700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9568104e60 0x7f9568115bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:45:35.291 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:35.290+0000 7f956ec00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9568105800 0x7f95681160f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:45:35.291 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:35.290+0000 7f956ec00700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9568116740 con 0x7f9568105800 2024-04-12T21:45:35.291 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:35.290+0000 7f956ec00700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f956810fb30 con 0x7f9568104e60 2024-04-12T21:45:35.291 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:35.290+0000 7f956ce00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9568105800 0x7f95681160f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:45:35.291 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:35.290+0000 7f956ce00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9568105800 0x7f95681160f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:44666/0 (socket says 172.21.15.18:44666) 2024-04-12T21:45:35.292 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:35.290+0000 7f956ce00700 1 -- 172.21.15.18:0/3501761803 learned_addr learned my addr 172.21.15.18:0/3501761803 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:45:35.292 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:35.290+0000 7f956d800700 1 --2- 172.21.15.18:0/3501761803 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9568104e60 0x7f9568115bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:45:35.293 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:35.290+0000 7f956ce00700 1 -- 172.21.15.18:0/3501761803 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9568104e60 msgr2=0x7f9568115bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:45:35.293 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:35.290+0000 7f956ce00700 1 --2- 172.21.15.18:0/3501761803 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9568104e60 0x7f9568115bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:45:35.293 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:35.290+0000 7f956ce00700 1 -- 172.21.15.18:0/3501761803 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f955c005040 con 0x7f9568105800 2024-04-12T21:45:35.293 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:35.290+0000 7f956d800700 1 --2- 172.21.15.18:0/3501761803 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9568104e60 0x7f9568115bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-12T21:45:35.293 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:35.291+0000 7f956ce00700 1 --2- 172.21.15.18:0/3501761803 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9568105800 0x7f95681160f0 secure :-1 s=READY pgs=1163 cs=0 l=1 rev1=1 crypto rx=0x7f955400c890 tx=0x7f95540044c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:45:35.293 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:35.291+0000 7f9565600700 1 -- 172.21.15.18:0/3501761803 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f955400a7d0 con 0x7f9568105800 2024-04-12T21:45:35.293 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:35.291+0000 7f9565600700 1 -- 172.21.15.18:0/3501761803 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f955400ae00 con 0x7f9568105800 2024-04-12T21:45:35.293 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:35.291+0000 7f9565600700 1 -- 172.21.15.18:0/3501761803 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f95540113a0 con 0x7f9568105800 2024-04-12T21:45:35.293 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:35.291+0000 7f956ec00700 1 -- 172.21.15.18:0/3501761803 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f956810fe10 con 0x7f9568105800 2024-04-12T21:45:35.293 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:35.291+0000 7f956ec00700 1 -- 172.21.15.18:0/3501761803 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f95681103d0 con 0x7f9568105800 2024-04-12T21:45:35.294 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:35.292+0000 7f956ec00700 1 -- 172.21.15.18:0/3501761803 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f95681005b0 con 0x7f9568105800 2024-04-12T21:45:35.295 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:35.294+0000 7f9565600700 1 -- 172.21.15.18:0/3501761803 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7f9554012400 con 0x7f9568105800 2024-04-12T21:45:35.296 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:35.295+0000 7f9565600700 1 --2- 172.21.15.18:0/3501761803 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f955006ec00 0x7f95500710b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:45:35.296 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:35.295+0000 7f9565600700 1 -- 172.21.15.18:0/3501761803 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 6757+0+0 (secure 0 0 0) 0x7f955408ea40 con 0x7f9568105800 2024-04-12T21:45:35.297 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:35.295+0000 7f956d800700 1 --2- 172.21.15.18:0/3501761803 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f955006ec00 0x7f95500710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:45:35.297 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:35.296+0000 7f956d800700 1 --2- 172.21.15.18:0/3501761803 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f955006ec00 0x7f95500710b0 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f955c00b3b0 tx=0x7f955c009e30 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:45:35.301 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:35.300+0000 7f9565600700 1 -- 172.21.15.18:0/3501761803 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f955405b270 con 0x7f9568105800 2024-04-12T21:45:35.630 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:35.628+0000 7f956ec00700 1 -- 172.21.15.18:0/3501761803 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f956810bf60 con 0x7f9568105800 2024-04-12T21:45:35.631 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:35.629+0000 7f9565600700 1 -- 172.21.15.18:0/3501761803 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+623 (secure 0 0 0) 0x7f955405ae00 con 0x7f9568105800 2024-04-12T21:45:35.631 INFO:teuthology.orchestra.run.smithi018.stdout:{ 2024-04-12T21:45:35.631 INFO:teuthology.orchestra.run.smithi018.stdout: "mon": { 2024-04-12T21:45:35.631 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 19.0.0-2879-g324baf8d (324baf8ddb91dea46cac50cb535c306115791355) squid (dev)": 2 2024-04-12T21:45:35.632 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-12T21:45:35.632 INFO:teuthology.orchestra.run.smithi018.stdout: "mgr": { 2024-04-12T21:45:35.632 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 19.0.0-2879-g324baf8d (324baf8ddb91dea46cac50cb535c306115791355) squid (dev)": 2 2024-04-12T21:45:35.632 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-12T21:45:35.632 INFO:teuthology.orchestra.run.smithi018.stdout: "osd": { 2024-04-12T21:45:35.632 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 19.0.0-2879-g324baf8d (324baf8ddb91dea46cac50cb535c306115791355) squid (dev)": 6 2024-04-12T21:45:35.632 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-12T21:45:35.632 INFO:teuthology.orchestra.run.smithi018.stdout: "mds": { 2024-04-12T21:45:35.632 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 19.0.0-2879-g324baf8d (324baf8ddb91dea46cac50cb535c306115791355) squid (dev)": 4 2024-04-12T21:45:35.632 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-12T21:45:35.632 INFO:teuthology.orchestra.run.smithi018.stdout: "overall": { 2024-04-12T21:45:35.632 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 19.0.0-2879-g324baf8d (324baf8ddb91dea46cac50cb535c306115791355) squid (dev)": 14 2024-04-12T21:45:35.632 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-04-12T21:45:35.632 INFO:teuthology.orchestra.run.smithi018.stdout:} 2024-04-12T21:45:35.634 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:45:35 smithi018 ceph-mon[112590]: from='client.36384 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:45:35.634 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:45:35 smithi018 ceph-mon[112590]: pgmap v1377: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:45:35.634 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:45:35 smithi018 ceph-mon[112590]: from='client.36388 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:45:35.635 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:35.632+0000 7f956ec00700 1 -- 172.21.15.18:0/3501761803 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f955006ec00 msgr2=0x7f95500710b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:45:35.635 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:35.632+0000 7f956ec00700 1 --2- 172.21.15.18:0/3501761803 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f955006ec00 0x7f95500710b0 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f955c00b3b0 tx=0x7f955c009e30 comp rx=0 tx=0).stop 2024-04-12T21:45:35.635 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:35.632+0000 7f956ec00700 1 -- 172.21.15.18:0/3501761803 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9568105800 msgr2=0x7f95681160f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:45:35.635 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:35.632+0000 7f956ec00700 1 --2- 172.21.15.18:0/3501761803 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9568105800 0x7f95681160f0 secure :-1 s=READY pgs=1163 cs=0 l=1 rev1=1 crypto rx=0x7f955400c890 tx=0x7f95540044c0 comp rx=0 tx=0).stop 2024-04-12T21:45:35.635 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:35.632+0000 7f956ec00700 1 -- 172.21.15.18:0/3501761803 shutdown_connections 2024-04-12T21:45:35.635 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:35.632+0000 7f956ec00700 1 --2- 172.21.15.18:0/3501761803 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f955006ec00 0x7f95500710b0 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:45:35.635 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:35.632+0000 7f956ec00700 1 --2- 172.21.15.18:0/3501761803 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9568104e60 0x7f9568115bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:45:35.635 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:35.632+0000 7f956ec00700 1 --2- 172.21.15.18:0/3501761803 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9568105800 0x7f95681160f0 unknown :-1 s=CLOSED pgs=1163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:45:35.635 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:35.632+0000 7f956ec00700 1 -- 172.21.15.18:0/3501761803 >> 172.21.15.18:0/3501761803 conn(0x7f95680fab10 msgr2=0x7f95680fca30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:45:35.635 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:35.633+0000 7f956ec00700 1 -- 172.21.15.18:0/3501761803 shutdown_connections 2024-04-12T21:45:35.635 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:35.633+0000 7f956ec00700 1 -- 172.21.15.18:0/3501761803 wait complete. 2024-04-12T21:45:35.784 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:35.778+0000 7f38f2a00700 1 -- 172.21.15.18:0/247202192 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f38ec105090 msgr2=0x7f38ec105460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:45:35.784 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:35.778+0000 7f38f2a00700 1 --2- 172.21.15.18:0/247202192 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f38ec105090 0x7f38ec105460 secure :-1 s=READY pgs=1164 cs=0 l=1 rev1=1 crypto rx=0x7f38dc004940 tx=0x7f38dc009e30 comp rx=0 tx=0).stop 2024-04-12T21:45:35.784 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:35.778+0000 7f38f2a00700 1 -- 172.21.15.18:0/247202192 shutdown_connections 2024-04-12T21:45:35.784 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:35.778+0000 7f38f2a00700 1 --2- 172.21.15.18:0/247202192 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f38ec105a30 0x7f38ec0ff0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:45:35.785 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:35.778+0000 7f38f2a00700 1 --2- 172.21.15.18:0/247202192 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f38ec105090 0x7f38ec105460 unknown :-1 s=CLOSED pgs=1164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:45:35.785 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:35.778+0000 7f38f2a00700 1 -- 172.21.15.18:0/247202192 >> 172.21.15.18:0/247202192 conn(0x7f38ec0fac00 msgr2=0x7f38ec0fd010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:45:35.785 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:35.778+0000 7f38f2a00700 1 -- 172.21.15.18:0/247202192 shutdown_connections 2024-04-12T21:45:35.785 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:35.778+0000 7f38f2a00700 1 -- 172.21.15.18:0/247202192 wait complete. 2024-04-12T21:45:35.785 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:35.779+0000 7f38f2a00700 1 Processor -- start 2024-04-12T21:45:35.785 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:35.779+0000 7f38f2a00700 1 -- start start 2024-04-12T21:45:35.786 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:35.779+0000 7f38f2a00700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f38ec105090 0x7f38ec115b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:45:35.786 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:35.779+0000 7f38f2a00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f38ec105a30 0x7f38ec1160b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:45:35.786 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:35.779+0000 7f38f2a00700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38ec116700 con 0x7f38ec105a30 2024-04-12T21:45:35.786 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:35.779+0000 7f38f2a00700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38ec10fce0 con 0x7f38ec105090 2024-04-12T21:45:35.786 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:35.780+0000 7f38eb400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f38ec105a30 0x7f38ec1160b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:45:35.787 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:35.780+0000 7f38eb400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f38ec105a30 0x7f38ec1160b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:44680/0 (socket says 172.21.15.18:44680) 2024-04-12T21:45:35.787 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:35.780+0000 7f38eb400700 1 -- 172.21.15.18:0/333830321 learned_addr learned my addr 172.21.15.18:0/333830321 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:45:35.787 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:35.780+0000 7f38ebe00700 1 --2- 172.21.15.18:0/333830321 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f38ec105090 0x7f38ec115b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:45:35.787 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:35.780+0000 7f38eb400700 1 -- 172.21.15.18:0/333830321 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f38ec105090 msgr2=0x7f38ec115b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:45:35.787 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:35.780+0000 7f38eb400700 1 --2- 172.21.15.18:0/333830321 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f38ec105090 0x7f38ec115b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:45:35.787 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:35.780+0000 7f38eb400700 1 -- 172.21.15.18:0/333830321 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38dc005040 con 0x7f38ec105a30 2024-04-12T21:45:35.788 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:35.780+0000 7f38ebe00700 1 --2- 172.21.15.18:0/333830321 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f38ec105090 0x7f38ec115b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-12T21:45:35.788 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:35.780+0000 7f38eb400700 1 --2- 172.21.15.18:0/333830321 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f38ec105a30 0x7f38ec1160b0 secure :-1 s=READY pgs=1165 cs=0 l=1 rev1=1 crypto rx=0x7f38d8002a80 tx=0x7f38d8004630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:45:35.788 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:35.781+0000 7f38e3e00700 1 -- 172.21.15.18:0/333830321 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f38d8007cc0 con 0x7f38ec105a30 2024-04-12T21:45:35.788 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:35.781+0000 7f38e3e00700 1 -- 172.21.15.18:0/333830321 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f38d800f040 con 0x7f38ec105a30 2024-04-12T21:45:35.788 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:35.781+0000 7f38f2a00700 1 -- 172.21.15.18:0/333830321 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f38ec10ffc0 con 0x7f38ec105a30 2024-04-12T21:45:35.789 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:35.781+0000 7f38e3e00700 1 -- 172.21.15.18:0/333830321 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f38d800daf0 con 0x7f38ec105a30 2024-04-12T21:45:35.789 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:35.781+0000 7f38f2a00700 1 -- 172.21.15.18:0/333830321 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f38ec110580 con 0x7f38ec105a30 2024-04-12T21:45:35.789 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:35.782+0000 7f38f2a00700 1 -- 172.21.15.18:0/333830321 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f38ec1023a0 con 0x7f38ec105a30 2024-04-12T21:45:35.790 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:45:35 smithi186 ceph-mon[86379]: from='client.36384 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:45:35.790 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:45:35 smithi186 ceph-mon[86379]: pgmap v1377: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:45:35.790 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:45:35 smithi186 ceph-mon[86379]: from='client.36388 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:45:35.792 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:35.783+0000 7f38e3e00700 1 -- 172.21.15.18:0/333830321 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7f38d800dd30 con 0x7f38ec105a30 2024-04-12T21:45:35.792 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:35.784+0000 7f38e3e00700 1 --2- 172.21.15.18:0/333830321 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f38d006ec00 0x7f38d00710b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:45:35.792 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:35.784+0000 7f38e3e00700 1 -- 172.21.15.18:0/333830321 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 6757+0+0 (secure 0 0 0) 0x7f38d808e070 con 0x7f38ec105a30 2024-04-12T21:45:35.792 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:35.785+0000 7f38ebe00700 1 --2- 172.21.15.18:0/333830321 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f38d006ec00 0x7f38d00710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:45:35.792 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:35.785+0000 7f38ebe00700 1 --2- 172.21.15.18:0/333830321 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f38d006ec00 0x7f38d00710b0 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f38dc004f90 tx=0x7f38dc009e30 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:45:35.793 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:35.790+0000 7f38e3e00700 1 -- 172.21.15.18:0/333830321 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f38d805a820 con 0x7f38ec105a30 2024-04-12T21:45:36.066 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:36.065+0000 7f38f2a00700 1 -- 172.21.15.18:0/333830321 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f38ec1111c0 con 0x7f38ec105a30 2024-04-12T21:45:36.067 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:36.066+0000 7f38e3e00700 1 -- 172.21.15.18:0/333830321 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 33 v33) v1 ==== 76+0+1876 (secure 0 0 0) 0x7f38d805a3b0 con 0x7f38ec105a30 2024-04-12T21:45:36.068 INFO:teuthology.orchestra.run.smithi018.stdout:e33 2024-04-12T21:45:36.068 INFO:teuthology.orchestra.run.smithi018.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-12T21:45:36.068 INFO:teuthology.orchestra.run.smithi018.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-12T21:45:36.068 INFO:teuthology.orchestra.run.smithi018.stdout:legacy client fscid: 1 2024-04-12T21:45:36.068 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-12T21:45:36.068 INFO:teuthology.orchestra.run.smithi018.stdout:Filesystem 'cephfs' (1) 2024-04-12T21:45:36.068 INFO:teuthology.orchestra.run.smithi018.stdout:fs_name cephfs 2024-04-12T21:45:36.068 INFO:teuthology.orchestra.run.smithi018.stdout:epoch 24 2024-04-12T21:45:36.068 INFO:teuthology.orchestra.run.smithi018.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-04-12T21:45:36.068 INFO:teuthology.orchestra.run.smithi018.stdout:created 2024-04-12T20:51:57.362850+0000 2024-04-12T21:45:36.069 INFO:teuthology.orchestra.run.smithi018.stdout:modified 2024-04-12T21:06:59.717190+0000 2024-04-12T21:45:36.069 INFO:teuthology.orchestra.run.smithi018.stdout:tableserver 0 2024-04-12T21:45:36.069 INFO:teuthology.orchestra.run.smithi018.stdout:root 0 2024-04-12T21:45:36.069 INFO:teuthology.orchestra.run.smithi018.stdout:session_timeout 60 2024-04-12T21:45:36.069 INFO:teuthology.orchestra.run.smithi018.stdout:session_autoclose 300 2024-04-12T21:45:36.069 INFO:teuthology.orchestra.run.smithi018.stdout:max_file_size 1099511627776 2024-04-12T21:45:36.069 INFO:teuthology.orchestra.run.smithi018.stdout:max_xattr_size 65536 2024-04-12T21:45:36.069 INFO:teuthology.orchestra.run.smithi018.stdout:required_client_features {} 2024-04-12T21:45:36.069 INFO:teuthology.orchestra.run.smithi018.stdout:last_failure 0 2024-04-12T21:45:36.069 INFO:teuthology.orchestra.run.smithi018.stdout:last_failure_osd_epoch 100 2024-04-12T21:45:36.069 INFO:teuthology.orchestra.run.smithi018.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-12T21:45:36.069 INFO:teuthology.orchestra.run.smithi018.stdout:max_mds 2 2024-04-12T21:45:36.069 INFO:teuthology.orchestra.run.smithi018.stdout:in 0,1 2024-04-12T21:45:36.069 INFO:teuthology.orchestra.run.smithi018.stdout:up {} 2024-04-12T21:45:36.070 INFO:teuthology.orchestra.run.smithi018.stdout:failed 0,1 2024-04-12T21:45:36.070 INFO:teuthology.orchestra.run.smithi018.stdout:damaged 2024-04-12T21:45:36.070 INFO:teuthology.orchestra.run.smithi018.stdout:stopped 2024-04-12T21:45:36.070 INFO:teuthology.orchestra.run.smithi018.stdout:data_pools [3] 2024-04-12T21:45:36.070 INFO:teuthology.orchestra.run.smithi018.stdout:metadata_pool 2 2024-04-12T21:45:36.070 INFO:teuthology.orchestra.run.smithi018.stdout:inline_data disabled 2024-04-12T21:45:36.070 INFO:teuthology.orchestra.run.smithi018.stdout:balancer 2024-04-12T21:45:36.070 INFO:teuthology.orchestra.run.smithi018.stdout:bal_rank_mask -1 2024-04-12T21:45:36.070 INFO:teuthology.orchestra.run.smithi018.stdout:standby_count_wanted 1 2024-04-12T21:45:36.070 INFO:teuthology.orchestra.run.smithi018.stdout:qdb_cluster leader: 0 members: 2024-04-12T21:45:36.070 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-12T21:45:36.070 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-12T21:45:36.070 INFO:teuthology.orchestra.run.smithi018.stdout:Standby daemons: 2024-04-12T21:45:36.070 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-12T21:45:36.070 INFO:teuthology.orchestra.run.smithi018.stdout:[mds.cephfs.smithi018.acqbsc{-1:34436} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.18:6828/755179764,v1:172.21.15.18:6829/755179764] compat {c=[1],r=[1],i=[1fff]}] 2024-04-12T21:45:36.071 INFO:teuthology.orchestra.run.smithi018.stdout:[mds.cephfs.smithi018.szjluo{-1:34468} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.18:6826/2407992653,v1:172.21.15.18:6827/2407992653] compat {c=[1],r=[1],i=[1fff]}] 2024-04-12T21:45:36.071 INFO:teuthology.orchestra.run.smithi018.stdout:[mds.cephfs.smithi186.jdllyn{-1:44595} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.186:6824/3523444202,v1:172.21.15.186:6825/3523444202] compat {c=[1],r=[1],i=[1fff]}] 2024-04-12T21:45:36.071 INFO:teuthology.orchestra.run.smithi018.stdout:[mds.cephfs.smithi186.wjnafs{-1:44875} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.186:6826/3122443766,v1:172.21.15.186:6827/3122443766] compat {c=[1],r=[1],i=[1fff]}] 2024-04-12T21:45:36.071 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:36.068+0000 7f38f2a00700 1 -- 172.21.15.18:0/333830321 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f38d006ec00 msgr2=0x7f38d00710b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:45:36.071 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:36.068+0000 7f38f2a00700 1 --2- 172.21.15.18:0/333830321 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f38d006ec00 0x7f38d00710b0 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f38dc004f90 tx=0x7f38dc009e30 comp rx=0 tx=0).stop 2024-04-12T21:45:36.072 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:36.068+0000 7f38f2a00700 1 -- 172.21.15.18:0/333830321 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f38ec105a30 msgr2=0x7f38ec1160b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:45:36.072 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:36.068+0000 7f38f2a00700 1 --2- 172.21.15.18:0/333830321 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f38ec105a30 0x7f38ec1160b0 secure :-1 s=READY pgs=1165 cs=0 l=1 rev1=1 crypto rx=0x7f38d8002a80 tx=0x7f38d8004630 comp rx=0 tx=0).stop 2024-04-12T21:45:36.072 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:36.068+0000 7f38f2a00700 1 -- 172.21.15.18:0/333830321 shutdown_connections 2024-04-12T21:45:36.072 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:36.068+0000 7f38f2a00700 1 --2- 172.21.15.18:0/333830321 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f38d006ec00 0x7f38d00710b0 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:45:36.072 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:36.068+0000 7f38f2a00700 1 --2- 172.21.15.18:0/333830321 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f38ec105090 0x7f38ec115b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:45:36.072 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:36.068+0000 7f38f2a00700 1 --2- 172.21.15.18:0/333830321 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f38ec105a30 0x7f38ec1160b0 unknown :-1 s=CLOSED pgs=1165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:45:36.072 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:36.068+0000 7f38f2a00700 1 -- 172.21.15.18:0/333830321 >> 172.21.15.18:0/333830321 conn(0x7f38ec0fac00 msgr2=0x7f38ec0fc940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:45:36.072 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:36.069+0000 7f38f2a00700 1 -- 172.21.15.18:0/333830321 shutdown_connections 2024-04-12T21:45:36.072 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:36.069+0000 7f38f2a00700 1 -- 172.21.15.18:0/333830321 wait complete. 2024-04-12T21:45:36.072 INFO:teuthology.orchestra.run.smithi018.stderr:dumped fsmap epoch 33 2024-04-12T21:45:36.232 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:36.231+0000 7f472b600700 1 -- 172.21.15.18:0/174607053 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4724105020 msgr2=0x7f47241053f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:45:36.233 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:36.231+0000 7f472b600700 1 --2- 172.21.15.18:0/174607053 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4724105020 0x7f47241053f0 secure :-1 s=READY pgs=1166 cs=0 l=1 rev1=1 crypto rx=0x7f4718004940 tx=0x7f4718009e30 comp rx=0 tx=0).stop 2024-04-12T21:45:36.233 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:36.231+0000 7f472b600700 1 -- 172.21.15.18:0/174607053 shutdown_connections 2024-04-12T21:45:36.233 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:36.231+0000 7f472b600700 1 --2- 172.21.15.18:0/174607053 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f47241059c0 0x7f47240ff080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:45:36.233 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:36.231+0000 7f472b600700 1 --2- 172.21.15.18:0/174607053 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4724105020 0x7f47241053f0 unknown :-1 s=CLOSED pgs=1166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:45:36.233 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:36.231+0000 7f472b600700 1 -- 172.21.15.18:0/174607053 >> 172.21.15.18:0/174607053 conn(0x7f47240faba0 msgr2=0x7f47240fcfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:45:36.233 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:36.231+0000 7f472b600700 1 -- 172.21.15.18:0/174607053 shutdown_connections 2024-04-12T21:45:36.233 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:36.231+0000 7f472b600700 1 -- 172.21.15.18:0/174607053 wait complete. 2024-04-12T21:45:36.234 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:36.232+0000 7f472b600700 1 Processor -- start 2024-04-12T21:45:36.234 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:36.232+0000 7f472b600700 1 -- start start 2024-04-12T21:45:36.234 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:36.232+0000 7f472b600700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4724105020 0x7f4724115b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:45:36.234 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:36.232+0000 7f472b600700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f47241059c0 0x7f47241160d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:45:36.234 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:36.232+0000 7f472b600700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4724116720 con 0x7f4724105020 2024-04-12T21:45:36.234 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:36.232+0000 7f472b600700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f472410fd00 con 0x7f47241059c0 2024-04-12T21:45:36.235 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:36.233+0000 7f4728c00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4724105020 0x7f4724115b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:45:36.235 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:36.233+0000 7f4728c00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4724105020 0x7f4724115b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:44702/0 (socket says 172.21.15.18:44702) 2024-04-12T21:45:36.235 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:36.233+0000 7f4728c00700 1 -- 172.21.15.18:0/1719595154 learned_addr learned my addr 172.21.15.18:0/1719595154 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:45:36.235 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:36.233+0000 7f4723e00700 1 --2- 172.21.15.18:0/1719595154 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f47241059c0 0x7f47241160d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:45:36.235 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:36.233+0000 7f4728c00700 1 -- 172.21.15.18:0/1719595154 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f47241059c0 msgr2=0x7f47241160d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:45:36.236 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:36.233+0000 7f4728c00700 1 --2- 172.21.15.18:0/1719595154 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f47241059c0 0x7f47241160d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:45:36.236 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:36.233+0000 7f4728c00700 1 -- 172.21.15.18:0/1719595154 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4718005040 con 0x7f4724105020 2024-04-12T21:45:36.236 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:36.234+0000 7f4728c00700 1 --2- 172.21.15.18:0/1719595154 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4724105020 0x7f4724115b90 secure :-1 s=READY pgs=1167 cs=0 l=1 rev1=1 crypto rx=0x7f471800b8b0 tx=0x7f4718009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:45:36.236 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:36.234+0000 7f4720c00700 1 -- 172.21.15.18:0/1719595154 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f471800e040 con 0x7f4724105020 2024-04-12T21:45:36.236 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:36.234+0000 7f4720c00700 1 -- 172.21.15.18:0/1719595154 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f4718004290 con 0x7f4724105020 2024-04-12T21:45:36.236 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:36.234+0000 7f472b600700 1 -- 172.21.15.18:0/1719595154 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f472410ff80 con 0x7f4724105020 2024-04-12T21:45:36.236 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:36.234+0000 7f4720c00700 1 -- 172.21.15.18:0/1719595154 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f471801c3f0 con 0x7f4724105020 2024-04-12T21:45:36.236 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:36.234+0000 7f472b600700 1 -- 172.21.15.18:0/1719595154 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4724110540 con 0x7f4724105020 2024-04-12T21:45:36.236 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:36.235+0000 7f472b600700 1 -- 172.21.15.18:0/1719595154 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4724100770 con 0x7f4724105020 2024-04-12T21:45:36.237 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:36.236+0000 7f4720c00700 1 -- 172.21.15.18:0/1719595154 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7f47180137b0 con 0x7f4724105020 2024-04-12T21:45:36.239 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:36.238+0000 7f4720c00700 1 --2- 172.21.15.18:0/1719595154 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f470c06ec00 0x7f470c0710b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:45:36.239 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:36.238+0000 7f4720c00700 1 -- 172.21.15.18:0/1719595154 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 6757+0+0 (secure 0 0 0) 0x7f471808fbe0 con 0x7f4724105020 2024-04-12T21:45:36.239 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:36.238+0000 7f4723e00700 1 --2- 172.21.15.18:0/1719595154 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f470c06ec00 0x7f470c0710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:45:36.240 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:36.239+0000 7f4723e00700 1 --2- 172.21.15.18:0/1719595154 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f470c06ec00 0x7f470c0710b0 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f4724111430 tx=0x7f471000b040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:45:36.244 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:36.242+0000 7f4720c00700 1 -- 172.21.15.18:0/1719595154 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f471805c4d0 con 0x7f4724105020 2024-04-12T21:45:36.498 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:36.496+0000 7f472b600700 1 -- 172.21.15.18:0/1719595154 --> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f4724111250 con 0x7f470c06ec00 2024-04-12T21:45:36.500 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:36.498+0000 7f4720c00700 1 -- 172.21.15.18:0/1719595154 <== mgr.34102 v2:172.21.15.18:6800/3812891919 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f4724111250 con 0x7f470c06ec00 2024-04-12T21:45:36.500 INFO:teuthology.orchestra.run.smithi018.stdout:{ 2024-04-12T21:45:36.500 INFO:teuthology.orchestra.run.smithi018.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:f4452def3be4d350f4a8e5feaf7ebb1511be9298fe68478ba7a2d6fbd7d69a63", 2024-04-12T21:45:36.500 INFO:teuthology.orchestra.run.smithi018.stdout: "in_progress": true, 2024-04-12T21:45:36.500 INFO:teuthology.orchestra.run.smithi018.stdout: "which": "Upgrading all daemon types on all hosts", 2024-04-12T21:45:36.500 INFO:teuthology.orchestra.run.smithi018.stdout: "services_complete": [ 2024-04-12T21:45:36.500 INFO:teuthology.orchestra.run.smithi018.stdout: "mgr", 2024-04-12T21:45:36.500 INFO:teuthology.orchestra.run.smithi018.stdout: "mds", 2024-04-12T21:45:36.500 INFO:teuthology.orchestra.run.smithi018.stdout: "crash", 2024-04-12T21:45:36.500 INFO:teuthology.orchestra.run.smithi018.stdout: "osd", 2024-04-12T21:45:36.500 INFO:teuthology.orchestra.run.smithi018.stdout: "mon" 2024-04-12T21:45:36.501 INFO:teuthology.orchestra.run.smithi018.stdout: ], 2024-04-12T21:45:36.501 INFO:teuthology.orchestra.run.smithi018.stdout: "progress": "16/21 daemons upgraded", 2024-04-12T21:45:36.501 INFO:teuthology.orchestra.run.smithi018.stdout: "message": "Currently upgrading mds daemons", 2024-04-12T21:45:36.501 INFO:teuthology.orchestra.run.smithi018.stdout: "is_paused": false 2024-04-12T21:45:36.501 INFO:teuthology.orchestra.run.smithi018.stdout:} 2024-04-12T21:45:36.502 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:36.501+0000 7f472b600700 1 -- 172.21.15.18:0/1719595154 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f470c06ec00 msgr2=0x7f470c0710b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:45:36.503 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:36.501+0000 7f472b600700 1 --2- 172.21.15.18:0/1719595154 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f470c06ec00 0x7f470c0710b0 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f4724111430 tx=0x7f471000b040 comp rx=0 tx=0).stop 2024-04-12T21:45:36.503 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:36.501+0000 7f472b600700 1 -- 172.21.15.18:0/1719595154 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4724105020 msgr2=0x7f4724115b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:45:36.503 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:36.501+0000 7f472b600700 1 --2- 172.21.15.18:0/1719595154 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4724105020 0x7f4724115b90 secure :-1 s=READY pgs=1167 cs=0 l=1 rev1=1 crypto rx=0x7f471800b8b0 tx=0x7f4718009e30 comp rx=0 tx=0).stop 2024-04-12T21:45:36.503 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:36.501+0000 7f472b600700 1 -- 172.21.15.18:0/1719595154 shutdown_connections 2024-04-12T21:45:36.504 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:36.501+0000 7f472b600700 1 --2- 172.21.15.18:0/1719595154 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f470c06ec00 0x7f470c0710b0 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:45:36.504 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:36.501+0000 7f472b600700 1 --2- 172.21.15.18:0/1719595154 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4724105020 0x7f4724115b90 unknown :-1 s=CLOSED pgs=1167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:45:36.504 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:36.501+0000 7f472b600700 1 --2- 172.21.15.18:0/1719595154 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f47241059c0 0x7f47241160d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:45:36.504 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:36.501+0000 7f472b600700 1 -- 172.21.15.18:0/1719595154 >> 172.21.15.18:0/1719595154 conn(0x7f47240faba0 msgr2=0x7f47240fbb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:45:36.504 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:36.502+0000 7f472b600700 1 -- 172.21.15.18:0/1719595154 shutdown_connections 2024-04-12T21:45:36.504 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:36.502+0000 7f472b600700 1 -- 172.21.15.18:0/1719595154 wait complete. 2024-04-12T21:45:36.657 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:36.655+0000 7fcfd6800700 1 -- 172.21.15.18:0/3660439378 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fcfd00fe600 msgr2=0x7fcfd0106c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:45:36.657 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:36.655+0000 7fcfd6800700 1 --2- 172.21.15.18:0/3660439378 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fcfd00fe600 0x7fcfd0106c40 secure :-1 s=READY pgs=1168 cs=0 l=1 rev1=1 crypto rx=0x7fcfbc004940 tx=0x7fcfbc009e30 comp rx=0 tx=0).stop 2024-04-12T21:45:36.657 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:36.656+0000 7fcfd6800700 1 -- 172.21.15.18:0/3660439378 shutdown_connections 2024-04-12T21:45:36.658 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:36.656+0000 7fcfd6800700 1 --2- 172.21.15.18:0/3660439378 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fcfd00fe600 0x7fcfd0106c40 unknown :-1 s=CLOSED pgs=1168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:45:36.658 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:36.656+0000 7fcfd6800700 1 --2- 172.21.15.18:0/3660439378 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcfd00fdc60 0x7fcfd00fe030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:45:36.658 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:36.656+0000 7fcfd6800700 1 -- 172.21.15.18:0/3660439378 >> 172.21.15.18:0/3660439378 conn(0x7fcfd00f76b0 msgr2=0x7fcfd00f9ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:45:36.658 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:36.656+0000 7fcfd6800700 1 -- 172.21.15.18:0/3660439378 shutdown_connections 2024-04-12T21:45:36.658 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:36.656+0000 7fcfd6800700 1 -- 172.21.15.18:0/3660439378 wait complete. 2024-04-12T21:45:36.659 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:36.657+0000 7fcfd6800700 1 Processor -- start 2024-04-12T21:45:36.660 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:36.657+0000 7fcfd6800700 1 -- start start 2024-04-12T21:45:36.660 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:36.657+0000 7fcfd6800700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fcfd00fdc60 0x7fcfd0115b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:45:36.660 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:36.657+0000 7fcfd6800700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcfd00fe600 0x7fcfd01160c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:45:36.660 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:36.657+0000 7fcfd6800700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcfd0116710 con 0x7fcfd00fdc60 2024-04-12T21:45:36.660 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:36.657+0000 7fcfd6800700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcfd010fb00 con 0x7fcfd00fe600 2024-04-12T21:45:36.660 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:36.658+0000 7fcfd5400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fcfd00fdc60 0x7fcfd0115b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:45:36.660 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:36.658+0000 7fcfd5400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fcfd00fdc60 0x7fcfd0115b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:44724/0 (socket says 172.21.15.18:44724) 2024-04-12T21:45:36.660 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:36.658+0000 7fcfd5400700 1 -- 172.21.15.18:0/692050346 learned_addr learned my addr 172.21.15.18:0/692050346 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:45:36.660 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:36.658+0000 7fcfd5400700 1 -- 172.21.15.18:0/692050346 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcfd00fe600 msgr2=0x7fcfd01160c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:45:36.660 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:36.658+0000 7fcfd4a00700 1 --2- 172.21.15.18:0/692050346 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcfd00fe600 0x7fcfd01160c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:45:36.660 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:36.658+0000 7fcfd5400700 1 --2- 172.21.15.18:0/692050346 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcfd00fe600 0x7fcfd01160c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:45:36.661 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:36.658+0000 7fcfd5400700 1 -- 172.21.15.18:0/692050346 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcfbc005040 con 0x7fcfd00fdc60 2024-04-12T21:45:36.661 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:36.658+0000 7fcfd5400700 1 --2- 172.21.15.18:0/692050346 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fcfd00fdc60 0x7fcfd0115b80 secure :-1 s=READY pgs=1169 cs=0 l=1 rev1=1 crypto rx=0x7fcfc4002a50 tx=0x7fcfc40046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:45:36.661 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:36.659+0000 7fcfcd600700 1 -- 172.21.15.18:0/692050346 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcfc4007cc0 con 0x7fcfd00fdc60 2024-04-12T21:45:36.662 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:36.659+0000 7fcfcd600700 1 -- 172.21.15.18:0/692050346 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fcfc400f040 con 0x7fcfd00fdc60 2024-04-12T21:45:36.662 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:36.659+0000 7fcfcd600700 1 -- 172.21.15.18:0/692050346 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcfc4009640 con 0x7fcfd00fdc60 2024-04-12T21:45:36.662 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:36.659+0000 7fcfd6800700 1 -- 172.21.15.18:0/692050346 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcfd010fde0 con 0x7fcfd00fdc60 2024-04-12T21:45:36.662 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:36.659+0000 7fcfd6800700 1 -- 172.21.15.18:0/692050346 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcfd01102d0 con 0x7fcfd00fdc60 2024-04-12T21:45:36.662 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:36.660+0000 7fcfd6800700 1 -- 172.21.15.18:0/692050346 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcfd00668d0 con 0x7fcfd00fdc60 2024-04-12T21:45:36.662 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:36.661+0000 7fcfcd600700 1 -- 172.21.15.18:0/692050346 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7fcfc40097c0 con 0x7fcfd00fdc60 2024-04-12T21:45:36.663 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:36.662+0000 7fcfcd600700 1 --2- 172.21.15.18:0/692050346 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fcfb806ec00 0x7fcfb80710b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:45:36.664 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:36.662+0000 7fcfcd600700 1 -- 172.21.15.18:0/692050346 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 6757+0+0 (secure 0 0 0) 0x7fcfc408e7d0 con 0x7fcfd00fdc60 2024-04-12T21:45:36.664 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:36.662+0000 7fcfd4a00700 1 --2- 172.21.15.18:0/692050346 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fcfb806ec00 0x7fcfb80710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:45:36.664 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:36.663+0000 7fcfd4a00700 1 --2- 172.21.15.18:0/692050346 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fcfb806ec00 0x7fcfb80710b0 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7fcfbc000c00 tx=0x7fcfbc009e30 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:45:36.668 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:36.667+0000 7fcfcd600700 1 -- 172.21.15.18:0/692050346 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7fcfc405af00 con 0x7fcfd00fdc60 2024-04-12T21:45:36.784 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:45:36 smithi186 ceph-mon[86379]: from='client.36392 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:45:36.784 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:45:36 smithi186 ceph-mon[86379]: from='client.? 172.21.15.18:0/3501761803' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T21:45:36.784 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:45:36 smithi186 ceph-mon[86379]: from='client.? 172.21.15.18:0/333830321' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-12T21:45:36.855 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:45:36 smithi018 ceph-mon[112590]: from='client.36392 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:45:36.855 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:45:36 smithi018 ceph-mon[112590]: from='client.? 172.21.15.18:0/3501761803' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T21:45:36.855 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:45:36 smithi018 ceph-mon[112590]: from='client.? 172.21.15.18:0/333830321' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-12T21:45:36.982 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:36.980+0000 7fcfd6800700 1 -- 172.21.15.18:0/692050346 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7fcfd0109cf0 con 0x7fcfd00fdc60 2024-04-12T21:45:36.983 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:36.982+0000 7fcfcd600700 1 -- 172.21.15.18:0/692050346 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+4303 (secure 0 0 0) 0x7fcfc405aa90 con 0x7fcfd00fdc60 2024-04-12T21:45:36.984 INFO:teuthology.orchestra.run.smithi018.stdout:HEALTH_ERR failed to probe daemons or devices; 1 filesystem is degraded; 1 filesystem is offline 2024-04-12T21:45:36.984 INFO:teuthology.orchestra.run.smithi018.stdout:[WRN] CEPHADM_REFRESH_FAILED: failed to probe daemons or devices 2024-04-12T21:45:36.984 INFO:teuthology.orchestra.run.smithi018.stdout: host smithi186 `cephadm ceph-volume` failed: cephadm exited with an error code: 1, stderr: Inferring config /var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/mon.smithi186/config 2024-04-12T21:45:36.984 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/podman: timeout after 895 seconds 2024-04-12T21:45:36.984 INFO:teuthology.orchestra.run.smithi018.stdout:Non-zero exit code 124 from /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:f4452def3be4d350f4a8e5feaf7ebb1511be9298fe68478ba7a2d6fbd7d69a63 -e NODE_NAME=smithi186 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968:/var/run/ceph:z -v /var/log/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968:/var/log/ceph:z -v /var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmptgom53f9:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:f4452def3be4d350f4a8e5feaf7ebb1511be9298fe68478ba7a2d6fbd7d69a63 inventory --format=json-pretty --filter-for-batch 2024-04-12T21:45:36.984 INFO:teuthology.orchestra.run.smithi018.stdout:Traceback (most recent call last): 2024-04-12T21:45:36.984 INFO:teuthology.orchestra.run.smithi018.stdout: File "/usr/lib64/python3.9/runpy.py", line 197, in _run_module_as_main 2024-04-12T21:45:36.984 INFO:teuthology.orchestra.run.smithi018.stdout: return _run_code(code, main_globals, None, 2024-04-12T21:45:36.984 INFO:teuthology.orchestra.run.smithi018.stdout: File "/usr/lib64/python3.9/runpy.py", line 87, in _run_code 2024-04-12T21:45:36.984 INFO:teuthology.orchestra.run.smithi018.stdout: exec(code, run_globals) 2024-04-12T21:45:36.984 INFO:teuthology.orchestra.run.smithi018.stdout: File "/var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/cephadm.e44f3ed5e34e4ef52eedef233d1190bff004470d8e8005127b5312bff357fa7f/__main__.py", line 5618, in 2024-04-12T21:45:36.984 INFO:teuthology.orchestra.run.smithi018.stdout: File "/var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/cephadm.e44f3ed5e34e4ef52eedef233d1190bff004470d8e8005127b5312bff357fa7f/__main__.py", line 5606, in main 2024-04-12T21:45:36.985 INFO:teuthology.orchestra.run.smithi018.stdout: File "/var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/cephadm.e44f3ed5e34e4ef52eedef233d1190bff004470d8e8005127b5312bff357fa7f/__main__.py", line 408, in _infer_config 2024-04-12T21:45:36.985 INFO:teuthology.orchestra.run.smithi018.stdout: File "/var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/cephadm.e44f3ed5e34e4ef52eedef233d1190bff004470d8e8005127b5312bff357fa7f/__main__.py", line 323, in _infer_fsid 2024-04-12T21:45:36.985 INFO:teuthology.orchestra.run.smithi018.stdout: File "/var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/cephadm.e44f3ed5e34e4ef52eedef233d1190bff004470d8e8005127b5312bff357fa7f/__main__.py", line 436, in _infer_image 2024-04-12T21:45:36.985 INFO:teuthology.orchestra.run.smithi018.stdout: File "/var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/cephadm.e44f3ed5e34e4ef52eedef233d1190bff004470d8e8005127b5312bff357fa7f/__main__.py", line 310, in _validate_fsid 2024-04-12T21:45:36.985 INFO:teuthology.orchestra.run.smithi018.stdout: File "/var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/cephadm.e44f3ed5e34e4ef52eedef233d1190bff004470d8e8005127b5312bff357fa7f/__main__.py", line 3359, in command_ceph_volume 2024-04-12T21:45:36.985 INFO:teuthology.orchestra.run.smithi018.stdout: File "/var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/cephadm.e44f3ed5e34e4ef52eedef233d1190bff004470d8e8005127b5312bff357fa7f/cephadmlib/call_wrappers.py", line 310, in call_throws 2024-04-12T21:45:36.985 INFO:teuthology.orchestra.run.smithi018.stdout:RuntimeError: Failed command: /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:f4452def3be4d350f4a8e5feaf7ebb1511be9298fe68478ba7a2d6fbd7d69a63 -e NODE_NAME=smithi186 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968:/var/run/ceph:z -v /var/log/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968:/var/log/ceph:z -v /var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmptgom53f9:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:f4452def3be4d350f4a8e5feaf7ebb1511be9298fe68478ba7a2d6fbd7d69a63 inventory --format=json-pretty --filter-for-batch 2024-04-12T21:45:36.985 INFO:teuthology.orchestra.run.smithi018.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-04-12T21:45:36.985 INFO:teuthology.orchestra.run.smithi018.stdout: fs cephfs is degraded 2024-04-12T21:45:36.985 INFO:teuthology.orchestra.run.smithi018.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-04-12T21:45:36.985 INFO:teuthology.orchestra.run.smithi018.stdout: fs cephfs is offline because no MDS is active for it. 2024-04-12T21:45:36.986 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:36.985+0000 7fcfd6800700 1 -- 172.21.15.18:0/692050346 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fcfb806ec00 msgr2=0x7fcfb80710b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:45:36.987 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:36.985+0000 7fcfd6800700 1 --2- 172.21.15.18:0/692050346 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fcfb806ec00 0x7fcfb80710b0 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7fcfbc000c00 tx=0x7fcfbc009e30 comp rx=0 tx=0).stop 2024-04-12T21:45:36.987 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:36.985+0000 7fcfd6800700 1 -- 172.21.15.18:0/692050346 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fcfd00fdc60 msgr2=0x7fcfd0115b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:45:36.987 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:36.985+0000 7fcfd6800700 1 --2- 172.21.15.18:0/692050346 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fcfd00fdc60 0x7fcfd0115b80 secure :-1 s=READY pgs=1169 cs=0 l=1 rev1=1 crypto rx=0x7fcfc4002a50 tx=0x7fcfc40046b0 comp rx=0 tx=0).stop 2024-04-12T21:45:36.987 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:36.985+0000 7fcfd6800700 1 -- 172.21.15.18:0/692050346 shutdown_connections 2024-04-12T21:45:36.987 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:36.985+0000 7fcfd6800700 1 --2- 172.21.15.18:0/692050346 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fcfb806ec00 0x7fcfb80710b0 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:45:36.988 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:36.985+0000 7fcfd6800700 1 --2- 172.21.15.18:0/692050346 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fcfd00fdc60 0x7fcfd0115b80 unknown :-1 s=CLOSED pgs=1169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:45:36.988 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:36.985+0000 7fcfd6800700 1 --2- 172.21.15.18:0/692050346 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcfd00fe600 0x7fcfd01160c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:45:36.988 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:36.985+0000 7fcfd6800700 1 -- 172.21.15.18:0/692050346 >> 172.21.15.18:0/692050346 conn(0x7fcfd00f76b0 msgr2=0x7fcfd00f8b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:45:36.988 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:36.986+0000 7fcfd6800700 1 -- 172.21.15.18:0/692050346 shutdown_connections 2024-04-12T21:45:36.988 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:45:36.986+0000 7fcfd6800700 1 -- 172.21.15.18:0/692050346 wait complete. 2024-04-12T21:45:37.784 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:45:37 smithi186 ceph-mon[86379]: pgmap v1378: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:45:37.784 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:45:37 smithi186 ceph-mon[86379]: from='client.36404 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:45:37.784 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:45:37 smithi186 ceph-mon[86379]: from='client.? 172.21.15.18:0/692050346' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-12T21:45:37.855 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:45:37 smithi018 ceph-mon[112590]: pgmap v1378: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:45:37.855 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:45:37 smithi018 ceph-mon[112590]: from='client.36404 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:45:37.855 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:45:37 smithi018 ceph-mon[112590]: from='client.? 172.21.15.18:0/692050346' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-12T21:45:39.784 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:45:39 smithi186 ceph-mon[86379]: pgmap v1379: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:45:39.855 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:45:39 smithi018 ceph-mon[112590]: pgmap v1379: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:45:40.784 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:45:40 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T21:45:40.855 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:45:40 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T21:45:41.784 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:45:41 smithi186 ceph-mon[86379]: pgmap v1380: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:45:41.855 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:45:41 smithi018 ceph-mon[112590]: pgmap v1380: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:45:43.784 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:45:43 smithi186 ceph-mon[86379]: pgmap v1381: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:45:43.855 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:45:43 smithi018 ceph-mon[112590]: pgmap v1381: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:45:45.658 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:45:45 smithi018 ceph-mon[112590]: pgmap v1382: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:45:45.784 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:45:45 smithi186 ceph-mon[86379]: pgmap v1382: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:45:47.784 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:45:47 smithi186 ceph-mon[86379]: pgmap v1383: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:45:47.854 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:45:47 smithi018 ceph-mon[112590]: pgmap v1383: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:45:49.784 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:45:49 smithi186 ceph-mon[86379]: pgmap v1384: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:45:49.855 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:45:49 smithi018 ceph-mon[112590]: pgmap v1384: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:45:51.784 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:45:51 smithi186 ceph-mon[86379]: pgmap v1385: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:45:51.854 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:45:51 smithi018 ceph-mon[112590]: pgmap v1385: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:45:53.784 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:45:53 smithi186 ceph-mon[86379]: pgmap v1386: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:45:53.855 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:45:53 smithi018 ceph-mon[112590]: pgmap v1386: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:45:55.784 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:45:55 smithi186 ceph-mon[86379]: pgmap v1387: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:45:55.784 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:45:55 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T21:45:55.855 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:45:55 smithi018 ceph-mon[112590]: pgmap v1387: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:45:55.855 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:45:55 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T21:45:57.784 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:45:57 smithi186 ceph-mon[86379]: pgmap v1388: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:45:57.855 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:45:57 smithi018 ceph-mon[112590]: pgmap v1388: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:45:59.784 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:45:59 smithi186 ceph-mon[86379]: pgmap v1389: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:45:59.855 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:45:59 smithi018 ceph-mon[112590]: pgmap v1389: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:46:01.784 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:46:01 smithi186 ceph-mon[86379]: pgmap v1390: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:46:01.855 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:46:01 smithi018 ceph-mon[112590]: pgmap v1390: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:46:03.784 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:46:03 smithi186 ceph-mon[86379]: pgmap v1391: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:46:03.855 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:46:03 smithi018 ceph-mon[112590]: pgmap v1391: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:46:05.855 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:46:05 smithi018 ceph-mon[112590]: pgmap v1392: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:46:06.034 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:46:05 smithi186 ceph-mon[86379]: pgmap v1392: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:46:07.161 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:07.159+0000 7f5bb3800700 1 -- 172.21.15.18:0/323040520 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5bac1059b0 msgr2=0x7f5bac0ff020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:46:07.161 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:07.159+0000 7f5bb3800700 1 --2- 172.21.15.18:0/323040520 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5bac1059b0 0x7f5bac0ff020 secure :-1 s=READY pgs=1170 cs=0 l=1 rev1=1 crypto rx=0x7f5b98004940 tx=0x7f5b98009e30 comp rx=0 tx=0).stop 2024-04-12T21:46:07.161 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:07.160+0000 7f5bb3800700 1 -- 172.21.15.18:0/323040520 shutdown_connections 2024-04-12T21:46:07.161 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:07.160+0000 7f5bb3800700 1 --2- 172.21.15.18:0/323040520 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5bac1059b0 0x7f5bac0ff020 unknown :-1 s=CLOSED pgs=1170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:46:07.161 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:07.160+0000 7f5bb3800700 1 --2- 172.21.15.18:0/323040520 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5bac105010 0x7f5bac1053e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:46:07.161 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:07.160+0000 7f5bb3800700 1 -- 172.21.15.18:0/323040520 >> 172.21.15.18:0/323040520 conn(0x7f5bac0faba0 msgr2=0x7f5bac0fcfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:46:07.161 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:07.160+0000 7f5bb3800700 1 -- 172.21.15.18:0/323040520 shutdown_connections 2024-04-12T21:46:07.162 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:07.160+0000 7f5bb3800700 1 -- 172.21.15.18:0/323040520 wait complete. 2024-04-12T21:46:07.163 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:07.161+0000 7f5bb3800700 1 Processor -- start 2024-04-12T21:46:07.163 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:07.161+0000 7f5bb3800700 1 -- start start 2024-04-12T21:46:07.163 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:07.161+0000 7f5bb3800700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5bac105010 0x7f5bac115d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:46:07.163 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:07.161+0000 7f5bb3800700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5bac1059b0 0x7f5bac1162b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:46:07.163 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:07.161+0000 7f5bb3800700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5bac116900 con 0x7f5bac1059b0 2024-04-12T21:46:07.163 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:07.161+0000 7f5bb3800700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5bac10fcd0 con 0x7f5bac105010 2024-04-12T21:46:07.163 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:07.162+0000 7f5babe00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5bac1059b0 0x7f5bac1162b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:46:07.164 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:07.162+0000 7f5babe00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5bac1059b0 0x7f5bac1162b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:60018/0 (socket says 172.21.15.18:60018) 2024-04-12T21:46:07.164 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:07.162+0000 7f5babe00700 1 -- 172.21.15.18:0/1948656345 learned_addr learned my addr 172.21.15.18:0/1948656345 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:46:07.164 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:07.162+0000 7f5bb0e00700 1 --2- 172.21.15.18:0/1948656345 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5bac105010 0x7f5bac115d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:46:07.164 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:07.162+0000 7f5babe00700 1 -- 172.21.15.18:0/1948656345 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5bac105010 msgr2=0x7f5bac115d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:46:07.164 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:07.162+0000 7f5babe00700 1 --2- 172.21.15.18:0/1948656345 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5bac105010 0x7f5bac115d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:46:07.164 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:07.162+0000 7f5babe00700 1 -- 172.21.15.18:0/1948656345 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b98005040 con 0x7f5bac1059b0 2024-04-12T21:46:07.164 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:07.163+0000 7f5babe00700 1 --2- 172.21.15.18:0/1948656345 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5bac1059b0 0x7f5bac1162b0 secure :-1 s=READY pgs=1171 cs=0 l=1 rev1=1 crypto rx=0x7f5b98009100 tx=0x7f5b98009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:46:07.165 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:07.163+0000 7f5ba8c00700 1 -- 172.21.15.18:0/1948656345 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5b9800e040 con 0x7f5bac1059b0 2024-04-12T21:46:07.165 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:07.163+0000 7f5ba8c00700 1 -- 172.21.15.18:0/1948656345 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f5b98013420 con 0x7f5bac1059b0 2024-04-12T21:46:07.165 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:07.163+0000 7f5ba8c00700 1 -- 172.21.15.18:0/1948656345 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5b9801c630 con 0x7f5bac1059b0 2024-04-12T21:46:07.165 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:07.163+0000 7f5bb3800700 1 -- 172.21.15.18:0/1948656345 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5bac10ff50 con 0x7f5bac1059b0 2024-04-12T21:46:07.165 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:07.163+0000 7f5bb3800700 1 -- 172.21.15.18:0/1948656345 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5bac110510 con 0x7f5bac1059b0 2024-04-12T21:46:07.165 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:07.164+0000 7f5bb3800700 1 -- 172.21.15.18:0/1948656345 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5bac100760 con 0x7f5bac1059b0 2024-04-12T21:46:07.166 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:07.166+0000 7f5ba8c00700 1 -- 172.21.15.18:0/1948656345 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7f5b98013590 con 0x7f5bac1059b0 2024-04-12T21:46:07.167 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:07.166+0000 7f5ba8c00700 1 --2- 172.21.15.18:0/1948656345 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f5b9406ec10 0x7f5b940710c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:46:07.167 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:07.167+0000 7f5ba8c00700 1 -- 172.21.15.18:0/1948656345 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 6757+0+0 (secure 0 0 0) 0x7f5b9808fc60 con 0x7f5bac1059b0 2024-04-12T21:46:07.167 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:07.167+0000 7f5bb0e00700 1 --2- 172.21.15.18:0/1948656345 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f5b9406ec10 0x7f5b940710c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:46:07.168 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:07.168+0000 7f5bb0e00700 1 --2- 172.21.15.18:0/1948656345 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f5b9406ec10 0x7f5b940710c0 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f5ba0004990 tx=0x7f5ba0009ee0 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:46:07.172 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:07.172+0000 7f5ba8c00700 1 -- 172.21.15.18:0/1948656345 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f5b9805c410 con 0x7f5bac1059b0 2024-04-12T21:46:07.430 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:07.430+0000 7f5bb3800700 1 -- 172.21.15.18:0/1948656345 --> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f5bac100760 con 0x7f5b9406ec10 2024-04-12T21:46:07.433 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:07.432+0000 7f5ba8c00700 1 -- 172.21.15.18:0/1948656345 <== mgr.34102 v2:172.21.15.18:6800/3812891919 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f5bac100760 con 0x7f5b9406ec10 2024-04-12T21:46:07.436 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:07.435+0000 7f5bb3800700 1 -- 172.21.15.18:0/1948656345 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f5b9406ec10 msgr2=0x7f5b940710c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:46:07.436 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:07.435+0000 7f5bb3800700 1 --2- 172.21.15.18:0/1948656345 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f5b9406ec10 0x7f5b940710c0 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f5ba0004990 tx=0x7f5ba0009ee0 comp rx=0 tx=0).stop 2024-04-12T21:46:07.436 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:07.435+0000 7f5bb3800700 1 -- 172.21.15.18:0/1948656345 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5bac1059b0 msgr2=0x7f5bac1162b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:46:07.436 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:07.435+0000 7f5bb3800700 1 --2- 172.21.15.18:0/1948656345 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5bac1059b0 0x7f5bac1162b0 secure :-1 s=READY pgs=1171 cs=0 l=1 rev1=1 crypto rx=0x7f5b98009100 tx=0x7f5b98009e30 comp rx=0 tx=0).stop 2024-04-12T21:46:07.437 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:07.435+0000 7f5bb3800700 1 -- 172.21.15.18:0/1948656345 shutdown_connections 2024-04-12T21:46:07.437 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:07.435+0000 7f5bb3800700 1 --2- 172.21.15.18:0/1948656345 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f5b9406ec10 0x7f5b940710c0 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:46:07.437 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:07.435+0000 7f5bb3800700 1 --2- 172.21.15.18:0/1948656345 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5bac105010 0x7f5bac115d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:46:07.437 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:07.435+0000 7f5bb3800700 1 --2- 172.21.15.18:0/1948656345 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5bac1059b0 0x7f5bac1162b0 unknown :-1 s=CLOSED pgs=1171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:46:07.437 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:07.435+0000 7f5bb3800700 1 -- 172.21.15.18:0/1948656345 >> 172.21.15.18:0/1948656345 conn(0x7f5bac0faba0 msgr2=0x7f5bac0fbcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:46:07.437 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:07.436+0000 7f5bb3800700 1 -- 172.21.15.18:0/1948656345 shutdown_connections 2024-04-12T21:46:07.437 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:07.436+0000 7f5bb3800700 1 -- 172.21.15.18:0/1948656345 wait complete. 2024-04-12T21:46:07.452 INFO:teuthology.orchestra.run.smithi018.stdout:true 2024-04-12T21:46:07.588 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:07.588+0000 7fb415000700 1 -- 172.21.15.18:0/2352449220 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb4101037a0 msgr2=0x7fb4100fce10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:46:07.589 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:07.588+0000 7fb415000700 1 --2- 172.21.15.18:0/2352449220 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb4101037a0 0x7fb4100fce10 secure :-1 s=READY pgs=1172 cs=0 l=1 rev1=1 crypto rx=0x7fb3fc004940 tx=0x7fb3fc009e30 comp rx=0 tx=0).stop 2024-04-12T21:46:07.589 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:07.588+0000 7fb415000700 1 -- 172.21.15.18:0/2352449220 shutdown_connections 2024-04-12T21:46:07.589 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:07.588+0000 7fb415000700 1 --2- 172.21.15.18:0/2352449220 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb4101037a0 0x7fb4100fce10 unknown :-1 s=CLOSED pgs=1172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:46:07.590 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:07.588+0000 7fb415000700 1 --2- 172.21.15.18:0/2352449220 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb410102e00 0x7fb4101031d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:46:07.590 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:07.588+0000 7fb415000700 1 -- 172.21.15.18:0/2352449220 >> 172.21.15.18:0/2352449220 conn(0x7fb4100f89b0 msgr2=0x7fb4100fadc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:46:07.590 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:07.588+0000 7fb415000700 1 -- 172.21.15.18:0/2352449220 shutdown_connections 2024-04-12T21:46:07.590 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:07.588+0000 7fb415000700 1 -- 172.21.15.18:0/2352449220 wait complete. 2024-04-12T21:46:07.592 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:07.589+0000 7fb415000700 1 Processor -- start 2024-04-12T21:46:07.592 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:07.589+0000 7fb415000700 1 -- start start 2024-04-12T21:46:07.592 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:07.589+0000 7fb415000700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb410102e00 0x7fb410115c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:46:07.592 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:07.589+0000 7fb415000700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb4101037a0 0x7fb410116150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:46:07.592 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:07.589+0000 7fb415000700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4101167a0 con 0x7fb410102e00 2024-04-12T21:46:07.592 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:07.589+0000 7fb415000700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb41010fd80 con 0x7fb4101037a0 2024-04-12T21:46:07.592 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:07.590+0000 7fb40de00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb410102e00 0x7fb410115c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:46:07.592 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:07.590+0000 7fb40de00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb410102e00 0x7fb410115c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:60038/0 (socket says 172.21.15.18:60038) 2024-04-12T21:46:07.592 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:07.590+0000 7fb40de00700 1 -- 172.21.15.18:0/3299216391 learned_addr learned my addr 172.21.15.18:0/3299216391 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:46:07.592 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:07.590+0000 7fb40d400700 1 --2- 172.21.15.18:0/3299216391 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb4101037a0 0x7fb410116150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:46:07.592 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:07.590+0000 7fb40de00700 1 -- 172.21.15.18:0/3299216391 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb4101037a0 msgr2=0x7fb410116150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:46:07.592 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:07.590+0000 7fb40de00700 1 --2- 172.21.15.18:0/3299216391 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb4101037a0 0x7fb410116150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:46:07.593 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:07.590+0000 7fb40de00700 1 -- 172.21.15.18:0/3299216391 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3fc005040 con 0x7fb410102e00 2024-04-12T21:46:07.593 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:07.590+0000 7fb40d400700 1 --2- 172.21.15.18:0/3299216391 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb4101037a0 0x7fb410116150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-12T21:46:07.593 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:07.591+0000 7fb40de00700 1 --2- 172.21.15.18:0/3299216391 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb410102e00 0x7fb410115c10 secure :-1 s=READY pgs=1173 cs=0 l=1 rev1=1 crypto rx=0x7fb400002a50 tx=0x7fb4000046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:46:07.593 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:07.591+0000 7fb406000700 1 -- 172.21.15.18:0/3299216391 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb400007cc0 con 0x7fb410102e00 2024-04-12T21:46:07.593 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:07.591+0000 7fb406000700 1 -- 172.21.15.18:0/3299216391 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb40000f040 con 0x7fb410102e00 2024-04-12T21:46:07.593 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:07.591+0000 7fb406000700 1 -- 172.21.15.18:0/3299216391 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb400009640 con 0x7fb410102e00 2024-04-12T21:46:07.593 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:07.591+0000 7fb415000700 1 -- 172.21.15.18:0/3299216391 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb410110060 con 0x7fb410102e00 2024-04-12T21:46:07.593 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:07.591+0000 7fb415000700 1 -- 172.21.15.18:0/3299216391 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb4101105a0 con 0x7fb410102e00 2024-04-12T21:46:07.594 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:07.592+0000 7fb415000700 1 -- 172.21.15.18:0/3299216391 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb410102120 con 0x7fb410102e00 2024-04-12T21:46:07.594 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:07.593+0000 7fb406000700 1 -- 172.21.15.18:0/3299216391 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7fb4000097a0 con 0x7fb410102e00 2024-04-12T21:46:07.594 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:07.594+0000 7fb406000700 1 --2- 172.21.15.18:0/3299216391 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fb3f406ec00 0x7fb3f40710b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:46:07.594 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:07.594+0000 7fb406000700 1 -- 172.21.15.18:0/3299216391 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 6757+0+0 (secure 0 0 0) 0x7fb40008e760 con 0x7fb410102e00 2024-04-12T21:46:07.595 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:07.594+0000 7fb40d400700 1 --2- 172.21.15.18:0/3299216391 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fb3f406ec00 0x7fb3f40710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:46:07.595 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:07.595+0000 7fb40d400700 1 --2- 172.21.15.18:0/3299216391 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fb3f406ec00 0x7fb3f40710b0 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7fb3fc009fd0 tx=0x7fb3fc009e30 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:46:07.600 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:07.599+0000 7fb406000700 1 -- 172.21.15.18:0/3299216391 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7fb40005af10 con 0x7fb410102e00 2024-04-12T21:46:07.850 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:07.849+0000 7fb415000700 1 -- 172.21.15.18:0/3299216391 --> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fb410109f50 con 0x7fb3f406ec00 2024-04-12T21:46:07.850 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:46:07 smithi018 ceph-mon[112590]: pgmap v1393: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:46:07.852 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:07.852+0000 7fb406000700 1 -- 172.21.15.18:0/3299216391 <== mgr.34102 v2:172.21.15.18:6800/3812891919 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7fb410109f50 con 0x7fb3f406ec00 2024-04-12T21:46:07.855 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:07.855+0000 7fb415000700 1 -- 172.21.15.18:0/3299216391 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fb3f406ec00 msgr2=0x7fb3f40710b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:46:07.855 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:07.855+0000 7fb415000700 1 --2- 172.21.15.18:0/3299216391 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fb3f406ec00 0x7fb3f40710b0 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7fb3fc009fd0 tx=0x7fb3fc009e30 comp rx=0 tx=0).stop 2024-04-12T21:46:07.855 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:07.855+0000 7fb415000700 1 -- 172.21.15.18:0/3299216391 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb410102e00 msgr2=0x7fb410115c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:46:07.855 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:07.855+0000 7fb415000700 1 --2- 172.21.15.18:0/3299216391 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb410102e00 0x7fb410115c10 secure :-1 s=READY pgs=1173 cs=0 l=1 rev1=1 crypto rx=0x7fb400002a50 tx=0x7fb4000046b0 comp rx=0 tx=0).stop 2024-04-12T21:46:07.856 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:07.855+0000 7fb415000700 1 -- 172.21.15.18:0/3299216391 shutdown_connections 2024-04-12T21:46:07.856 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:07.855+0000 7fb415000700 1 --2- 172.21.15.18:0/3299216391 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fb3f406ec00 0x7fb3f40710b0 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:46:07.856 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:07.855+0000 7fb415000700 1 --2- 172.21.15.18:0/3299216391 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb410102e00 0x7fb410115c10 unknown :-1 s=CLOSED pgs=1173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:46:07.856 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:07.855+0000 7fb415000700 1 --2- 172.21.15.18:0/3299216391 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb4101037a0 0x7fb410116150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:46:07.856 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:07.855+0000 7fb415000700 1 -- 172.21.15.18:0/3299216391 >> 172.21.15.18:0/3299216391 conn(0x7fb4100f89b0 msgr2=0x7fb410070be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:46:07.856 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:07.855+0000 7fb415000700 1 -- 172.21.15.18:0/3299216391 shutdown_connections 2024-04-12T21:46:07.856 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:07.855+0000 7fb415000700 1 -- 172.21.15.18:0/3299216391 wait complete. 2024-04-12T21:46:08.012 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:08.011+0000 7f69ae000700 1 -- 172.21.15.18:0/1578348674 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f69a80724f0 msgr2=0x7f69a80728c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:46:08.012 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:08.011+0000 7f69ae000700 1 --2- 172.21.15.18:0/1578348674 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f69a80724f0 0x7f69a80728c0 secure :-1 s=READY pgs=1174 cs=0 l=1 rev1=1 crypto rx=0x7f6998004940 tx=0x7f6998009e30 comp rx=0 tx=0).stop 2024-04-12T21:46:08.012 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:08.011+0000 7f69ae000700 1 -- 172.21.15.18:0/1578348674 shutdown_connections 2024-04-12T21:46:08.012 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:08.011+0000 7f69ae000700 1 --2- 172.21.15.18:0/1578348674 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f69a8072e00 0x7f69a810d780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:46:08.012 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:08.011+0000 7f69ae000700 1 --2- 172.21.15.18:0/1578348674 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f69a80724f0 0x7f69a80728c0 unknown :-1 s=CLOSED pgs=1174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:46:08.012 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:08.011+0000 7f69ae000700 1 -- 172.21.15.18:0/1578348674 >> 172.21.15.18:0/1578348674 conn(0x7f69a80faba0 msgr2=0x7f69a80fcfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:46:08.012 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:08.011+0000 7f69ae000700 1 -- 172.21.15.18:0/1578348674 shutdown_connections 2024-04-12T21:46:08.012 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:08.011+0000 7f69ae000700 1 -- 172.21.15.18:0/1578348674 wait complete. 2024-04-12T21:46:08.014 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:08.012+0000 7f69ae000700 1 Processor -- start 2024-04-12T21:46:08.014 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:08.012+0000 7f69ae000700 1 -- start start 2024-04-12T21:46:08.014 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:08.012+0000 7f69ae000700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f69a80724f0 0x7f69a811a120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:46:08.014 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:08.012+0000 7f69ae000700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f69a8072e00 0x7f69a811a660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:46:08.014 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:08.013+0000 7f69ae000700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69a811acb0 con 0x7f69a80724f0 2024-04-12T21:46:08.014 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:08.013+0000 7f69ae000700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69a81140c0 con 0x7f69a8072e00 2024-04-12T21:46:08.014 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:08.013+0000 7f69a7400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f69a80724f0 0x7f69a811a120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:46:08.014 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:08.013+0000 7f69a7400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f69a80724f0 0x7f69a811a120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:60056/0 (socket says 172.21.15.18:60056) 2024-04-12T21:46:08.014 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:08.013+0000 7f69a7400700 1 -- 172.21.15.18:0/1809604634 learned_addr learned my addr 172.21.15.18:0/1809604634 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:46:08.014 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:08.013+0000 7f69a6a00700 1 --2- 172.21.15.18:0/1809604634 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f69a8072e00 0x7f69a811a660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:46:08.014 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:08.013+0000 7f69a7400700 1 -- 172.21.15.18:0/1809604634 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f69a8072e00 msgr2=0x7f69a811a660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:46:08.014 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:08.013+0000 7f69a7400700 1 --2- 172.21.15.18:0/1809604634 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f69a8072e00 0x7f69a811a660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:46:08.015 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:08.013+0000 7f69a7400700 1 -- 172.21.15.18:0/1809604634 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6998005040 con 0x7f69a80724f0 2024-04-12T21:46:08.015 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:08.013+0000 7f69a6a00700 1 --2- 172.21.15.18:0/1809604634 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f69a8072e00 0x7f69a811a660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-12T21:46:08.015 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:08.014+0000 7f69a7400700 1 --2- 172.21.15.18:0/1809604634 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f69a80724f0 0x7f69a811a120 secure :-1 s=READY pgs=1175 cs=0 l=1 rev1=1 crypto rx=0x7f6998009180 tx=0x7f6998009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:46:08.015 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:08.014+0000 7f699f400700 1 -- 172.21.15.18:0/1809604634 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f699800e040 con 0x7f69a80724f0 2024-04-12T21:46:08.015 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:08.014+0000 7f699f400700 1 -- 172.21.15.18:0/1809604634 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f6998013420 con 0x7f69a80724f0 2024-04-12T21:46:08.016 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:08.014+0000 7f69ae000700 1 -- 172.21.15.18:0/1809604634 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f69a8114340 con 0x7f69a80724f0 2024-04-12T21:46:08.016 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:08.014+0000 7f699f400700 1 -- 172.21.15.18:0/1809604634 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f699801c630 con 0x7f69a80724f0 2024-04-12T21:46:08.016 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:08.014+0000 7f69ae000700 1 -- 172.21.15.18:0/1809604634 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f69a8114900 con 0x7f69a80724f0 2024-04-12T21:46:08.016 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:08.015+0000 7f69ae000700 1 -- 172.21.15.18:0/1809604634 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f69a8073c30 con 0x7f69a80724f0 2024-04-12T21:46:08.017 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:08.016+0000 7f699f400700 1 -- 172.21.15.18:0/1809604634 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7f6998013590 con 0x7f69a80724f0 2024-04-12T21:46:08.018 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:08.017+0000 7f699f400700 1 --2- 172.21.15.18:0/1809604634 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f698c06ec10 0x7f698c0710c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:46:08.018 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:08.017+0000 7f699f400700 1 -- 172.21.15.18:0/1809604634 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 6757+0+0 (secure 0 0 0) 0x7f699808fc60 con 0x7f69a80724f0 2024-04-12T21:46:08.018 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:08.018+0000 7f69a6a00700 1 --2- 172.21.15.18:0/1809604634 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f698c06ec10 0x7f698c0710c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:46:08.019 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:08.019+0000 7f69a6a00700 1 --2- 172.21.15.18:0/1809604634 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f698c06ec10 0x7f698c0710c0 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f6994009fd0 tx=0x7f699400b040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:46:08.022 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:08.022+0000 7f699f400700 1 -- 172.21.15.18:0/1809604634 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f6998058a20 con 0x7f69a80724f0 2024-04-12T21:46:08.033 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:46:07 smithi186 ceph-mon[86379]: pgmap v1393: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:46:08.258 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:08.257+0000 7f69ae000700 1 -- 172.21.15.18:0/1809604634 --> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f69a804bf40 con 0x7f698c06ec10 2024-04-12T21:46:08.274 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:08.274+0000 7f699f400700 1 -- 172.21.15.18:0/1809604634 <== mgr.34102 v2:172.21.15.18:6800/3812891919 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7f69a804bf40 con 0x7f698c06ec10 2024-04-12T21:46:08.275 INFO:teuthology.orchestra.run.smithi018.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-12T21:46:08.275 INFO:teuthology.orchestra.run.smithi018.stdout:alertmanager.smithi018 smithi018 *:9093,9094 running (46m) 7m ago 61m 14.8M - 0.25.0 c8568f914cd2 c332090068e8 2024-04-12T21:46:08.275 INFO:teuthology.orchestra.run.smithi018.stdout:crash.smithi018 smithi018 running (43m) 7m ago 61m 6665k - 19.0.0-2879-g324baf8d 8717720dc5e5 b7f80d1f7982 2024-04-12T21:46:08.275 INFO:teuthology.orchestra.run.smithi018.stdout:crash.smithi186 smithi186 running (42m) 7m ago 60m 6660k - 19.0.0-2879-g324baf8d 8717720dc5e5 a4f55c81b623 2024-04-12T21:46:08.275 INFO:teuthology.orchestra.run.smithi018.stdout:grafana.smithi018 smithi018 *:3000 running (46m) 7m ago 60m 73.8M - 9.4.12 2bacad6d85d8 b08e3bf176dd 2024-04-12T21:46:08.275 INFO:teuthology.orchestra.run.smithi018.stdout:mds.cephfs.smithi018.acqbsc smithi018 running (38m) 7m ago 54m 21.9M - 19.0.0-2879-g324baf8d 8717720dc5e5 021f99ab24ae 2024-04-12T21:46:08.275 INFO:teuthology.orchestra.run.smithi018.stdout:mds.cephfs.smithi018.szjluo smithi018 running (38m) 7m ago 53m 21.7M - 19.0.0-2879-g324baf8d 8717720dc5e5 0003117479d4 2024-04-12T21:46:08.275 INFO:teuthology.orchestra.run.smithi018.stdout:mds.cephfs.smithi186.jdllyn smithi186 running (23m) 7m ago 54m 16.5M - 19.0.0-2879-g324baf8d 8717720dc5e5 21b1f46eef23 2024-04-12T21:46:08.275 INFO:teuthology.orchestra.run.smithi018.stdout:mds.cephfs.smithi186.wjnafs smithi186 running (7m) 7m ago 53m 11.0M - 19.0.0-2879-g324baf8d 8717720dc5e5 3cf1ef4148e2 2024-04-12T21:46:08.275 INFO:teuthology.orchestra.run.smithi018.stdout:mgr.smithi018.xjzwga smithi018 *:8443,9283,8765 running (49m) 7m ago 62m 517M - 19.0.0-2879-g324baf8d 8717720dc5e5 56088305eab7 2024-04-12T21:46:08.275 INFO:teuthology.orchestra.run.smithi018.stdout:mgr.smithi186.kqcfxk smithi186 *:8443,9283,8765 running (48m) 7m ago 59m 460M - 19.0.0-2879-g324baf8d 8717720dc5e5 09cd766e5e0d 2024-04-12T21:46:08.275 INFO:teuthology.orchestra.run.smithi018.stdout:mon.smithi018 smithi018 running (44m) 7m ago 63m 88.2M 2048M 19.0.0-2879-g324baf8d 8717720dc5e5 df3e201dfed6 2024-04-12T21:46:08.276 INFO:teuthology.orchestra.run.smithi018.stdout:mon.smithi186 smithi186 running (44m) 7m ago 59m 53.6M 2048M 19.0.0-2879-g324baf8d 8717720dc5e5 c77498da9b2b 2024-04-12T21:46:08.276 INFO:teuthology.orchestra.run.smithi018.stdout:node-exporter.smithi018 smithi018 *:9100 running (47m) 7m ago 61m 8468k - 1.5.0 0da6a335fe13 326c3d9c6847 2024-04-12T21:46:08.276 INFO:teuthology.orchestra.run.smithi018.stdout:node-exporter.smithi186 smithi186 *:9100 running (47m) 7m ago 60m 8166k - 1.5.0 0da6a335fe13 ec2112baeecf 2024-04-12T21:46:08.276 INFO:teuthology.orchestra.run.smithi018.stdout:osd.0 smithi018 running (42m) 7m ago 58m 123M 1559M 19.0.0-2879-g324baf8d 8717720dc5e5 60b1273ca047 2024-04-12T21:46:08.276 INFO:teuthology.orchestra.run.smithi018.stdout:osd.1 smithi018 running (41m) 7m ago 58m 95.4M 1559M 19.0.0-2879-g324baf8d 8717720dc5e5 170ce1e5fb8d 2024-04-12T21:46:08.276 INFO:teuthology.orchestra.run.smithi018.stdout:osd.2 smithi018 running (41m) 7m ago 57m 94.6M 1559M 19.0.0-2879-g324baf8d 8717720dc5e5 c2cbc6aa2839 2024-04-12T21:46:08.276 INFO:teuthology.orchestra.run.smithi018.stdout:osd.3 smithi186 running (40m) 7m ago 57m 101M 2581M 19.0.0-2879-g324baf8d 8717720dc5e5 0138e8262e4b 2024-04-12T21:46:08.276 INFO:teuthology.orchestra.run.smithi018.stdout:osd.4 smithi186 running (40m) 7m ago 56m 91.5M 2581M 19.0.0-2879-g324baf8d 8717720dc5e5 b20cb63b25c9 2024-04-12T21:46:08.276 INFO:teuthology.orchestra.run.smithi018.stdout:osd.5 smithi186 running (39m) 7m ago 55m 92.2M 2581M 19.0.0-2879-g324baf8d 8717720dc5e5 2513bea2610e 2024-04-12T21:46:08.276 INFO:teuthology.orchestra.run.smithi018.stdout:prometheus.smithi018 smithi018 *:9095 running (47m) 7m ago 60m 69.6M - 2.43.0 a07b618ecd1d f58841127e21 2024-04-12T21:46:08.277 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:08.276+0000 7f69ae000700 1 -- 172.21.15.18:0/1809604634 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f698c06ec10 msgr2=0x7f698c0710c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:46:08.277 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:08.276+0000 7f69ae000700 1 --2- 172.21.15.18:0/1809604634 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f698c06ec10 0x7f698c0710c0 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f6994009fd0 tx=0x7f699400b040 comp rx=0 tx=0).stop 2024-04-12T21:46:08.277 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:08.276+0000 7f69ae000700 1 -- 172.21.15.18:0/1809604634 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f69a80724f0 msgr2=0x7f69a811a120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:46:08.277 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:08.276+0000 7f69ae000700 1 --2- 172.21.15.18:0/1809604634 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f69a80724f0 0x7f69a811a120 secure :-1 s=READY pgs=1175 cs=0 l=1 rev1=1 crypto rx=0x7f6998009180 tx=0x7f6998009e30 comp rx=0 tx=0).stop 2024-04-12T21:46:08.278 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:08.277+0000 7f69ae000700 1 -- 172.21.15.18:0/1809604634 shutdown_connections 2024-04-12T21:46:08.278 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:08.277+0000 7f69ae000700 1 --2- 172.21.15.18:0/1809604634 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f698c06ec10 0x7f698c0710c0 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:46:08.278 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:08.277+0000 7f69ae000700 1 --2- 172.21.15.18:0/1809604634 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f69a80724f0 0x7f69a811a120 unknown :-1 s=CLOSED pgs=1175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:46:08.278 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:08.277+0000 7f69ae000700 1 --2- 172.21.15.18:0/1809604634 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f69a8072e00 0x7f69a811a660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:46:08.278 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:08.277+0000 7f69ae000700 1 -- 172.21.15.18:0/1809604634 >> 172.21.15.18:0/1809604634 conn(0x7f69a80faba0 msgr2=0x7f69a80faf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:46:08.278 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:08.277+0000 7f69ae000700 1 -- 172.21.15.18:0/1809604634 shutdown_connections 2024-04-12T21:46:08.278 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:08.277+0000 7f69ae000700 1 -- 172.21.15.18:0/1809604634 wait complete. 2024-04-12T21:46:08.432 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:08.431+0000 7f64f9600700 1 -- 172.21.15.18:0/1848042391 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f64f40fe830 msgr2=0x7f64f4106f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:46:08.432 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:08.431+0000 7f64f9600700 1 --2- 172.21.15.18:0/1848042391 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f64f40fe830 0x7f64f4106f00 secure :-1 s=READY pgs=1176 cs=0 l=1 rev1=1 crypto rx=0x7f64dc004940 tx=0x7f64dc009e30 comp rx=0 tx=0).stop 2024-04-12T21:46:08.432 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:08.431+0000 7f64f9600700 1 -- 172.21.15.18:0/1848042391 shutdown_connections 2024-04-12T21:46:08.432 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:08.431+0000 7f64f9600700 1 --2- 172.21.15.18:0/1848042391 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f64f40fe830 0x7f64f4106f00 unknown :-1 s=CLOSED pgs=1176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:46:08.433 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:08.431+0000 7f64f9600700 1 --2- 172.21.15.18:0/1848042391 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f64f40fde90 0x7f64f40fe260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:46:08.433 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:08.431+0000 7f64f9600700 1 -- 172.21.15.18:0/1848042391 >> 172.21.15.18:0/1848042391 conn(0x7f64f4072bc0 msgr2=0x7f64f4072fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:46:08.433 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:08.431+0000 7f64f9600700 1 -- 172.21.15.18:0/1848042391 shutdown_connections 2024-04-12T21:46:08.433 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:08.432+0000 7f64f9600700 1 -- 172.21.15.18:0/1848042391 wait complete. 2024-04-12T21:46:08.434 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:08.432+0000 7f64f9600700 1 Processor -- start 2024-04-12T21:46:08.434 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:08.432+0000 7f64f9600700 1 -- start start 2024-04-12T21:46:08.434 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:08.433+0000 7f64f9600700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f64f40fde90 0x7f64f4115d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:46:08.434 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:08.433+0000 7f64f9600700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f64f40fe830 0x7f64f41162a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:46:08.434 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:08.433+0000 7f64f9600700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64f41168f0 con 0x7f64f40fde90 2024-04-12T21:46:08.434 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:08.433+0000 7f64f9600700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64f410fd00 con 0x7f64f40fe830 2024-04-12T21:46:08.434 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:08.433+0000 7f64f2a00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f64f40fde90 0x7f64f4115d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:46:08.435 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:08.433+0000 7f64f2a00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f64f40fde90 0x7f64f4115d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:60078/0 (socket says 172.21.15.18:60078) 2024-04-12T21:46:08.435 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:08.433+0000 7f64f2a00700 1 -- 172.21.15.18:0/1654472792 learned_addr learned my addr 172.21.15.18:0/1654472792 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:46:08.435 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:08.433+0000 7f64ebe00700 1 --2- 172.21.15.18:0/1654472792 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f64f40fe830 0x7f64f41162a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:46:08.435 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:08.434+0000 7f64f2a00700 1 -- 172.21.15.18:0/1654472792 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f64f40fe830 msgr2=0x7f64f41162a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:46:08.435 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:08.434+0000 7f64f2a00700 1 --2- 172.21.15.18:0/1654472792 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f64f40fe830 0x7f64f41162a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:46:08.435 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:08.434+0000 7f64f2a00700 1 -- 172.21.15.18:0/1654472792 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64dc005040 con 0x7f64f40fde90 2024-04-12T21:46:08.435 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:08.434+0000 7f64f2a00700 1 --2- 172.21.15.18:0/1654472792 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f64f40fde90 0x7f64f4115d60 secure :-1 s=READY pgs=1177 cs=0 l=1 rev1=1 crypto rx=0x7f64e400c830 tx=0x7f64e4004470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:46:08.436 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:08.434+0000 7f64eaa00700 1 -- 172.21.15.18:0/1654472792 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f64e400a760 con 0x7f64f40fde90 2024-04-12T21:46:08.436 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:08.434+0000 7f64eaa00700 1 -- 172.21.15.18:0/1654472792 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f64e400ad90 con 0x7f64f40fde90 2024-04-12T21:46:08.436 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:08.434+0000 7f64eaa00700 1 -- 172.21.15.18:0/1654472792 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f64e40113a0 con 0x7f64f40fde90 2024-04-12T21:46:08.436 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:08.434+0000 7f64f9600700 1 -- 172.21.15.18:0/1654472792 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f64f410ffe0 con 0x7f64f40fde90 2024-04-12T21:46:08.436 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:08.434+0000 7f64f9600700 1 -- 172.21.15.18:0/1654472792 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f64f4110520 con 0x7f64f40fde90 2024-04-12T21:46:08.436 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:08.435+0000 7f64f9600700 1 -- 172.21.15.18:0/1654472792 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f64f4106a20 con 0x7f64f40fde90 2024-04-12T21:46:08.437 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:08.437+0000 7f64eaa00700 1 -- 172.21.15.18:0/1654472792 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7f64e4012400 con 0x7f64f40fde90 2024-04-12T21:46:08.438 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:08.437+0000 7f64eaa00700 1 --2- 172.21.15.18:0/1654472792 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f64d806ec10 0x7f64d80710c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:46:08.438 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:08.438+0000 7f64eaa00700 1 -- 172.21.15.18:0/1654472792 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 6757+0+0 (secure 0 0 0) 0x7f64e408e9c0 con 0x7f64f40fde90 2024-04-12T21:46:08.438 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:08.438+0000 7f64ebe00700 1 --2- 172.21.15.18:0/1654472792 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f64d806ec10 0x7f64d80710c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:46:08.439 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:08.438+0000 7f64ebe00700 1 --2- 172.21.15.18:0/1654472792 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f64d806ec10 0x7f64d80710c0 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f64f4111640 tx=0x7f64dc009e30 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:46:08.442 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:08.442+0000 7f64eaa00700 1 -- 172.21.15.18:0/1654472792 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f64e40576c0 con 0x7f64f40fde90 2024-04-12T21:46:08.757 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:08.756+0000 7f64f9600700 1 -- 172.21.15.18:0/1654472792 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f64f4109f10 con 0x7f64f40fde90 2024-04-12T21:46:08.758 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:08.757+0000 7f64eaa00700 1 -- 172.21.15.18:0/1654472792 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+623 (secure 0 0 0) 0x7f64e405ad80 con 0x7f64f40fde90 2024-04-12T21:46:08.758 INFO:teuthology.orchestra.run.smithi018.stdout:{ 2024-04-12T21:46:08.759 INFO:teuthology.orchestra.run.smithi018.stdout: "mon": { 2024-04-12T21:46:08.759 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 19.0.0-2879-g324baf8d (324baf8ddb91dea46cac50cb535c306115791355) squid (dev)": 2 2024-04-12T21:46:08.759 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-12T21:46:08.759 INFO:teuthology.orchestra.run.smithi018.stdout: "mgr": { 2024-04-12T21:46:08.759 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 19.0.0-2879-g324baf8d (324baf8ddb91dea46cac50cb535c306115791355) squid (dev)": 2 2024-04-12T21:46:08.759 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-12T21:46:08.759 INFO:teuthology.orchestra.run.smithi018.stdout: "osd": { 2024-04-12T21:46:08.759 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 19.0.0-2879-g324baf8d (324baf8ddb91dea46cac50cb535c306115791355) squid (dev)": 6 2024-04-12T21:46:08.759 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-12T21:46:08.759 INFO:teuthology.orchestra.run.smithi018.stdout: "mds": { 2024-04-12T21:46:08.759 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 19.0.0-2879-g324baf8d (324baf8ddb91dea46cac50cb535c306115791355) squid (dev)": 4 2024-04-12T21:46:08.759 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-12T21:46:08.759 INFO:teuthology.orchestra.run.smithi018.stdout: "overall": { 2024-04-12T21:46:08.759 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 19.0.0-2879-g324baf8d (324baf8ddb91dea46cac50cb535c306115791355) squid (dev)": 14 2024-04-12T21:46:08.760 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-04-12T21:46:08.760 INFO:teuthology.orchestra.run.smithi018.stdout:} 2024-04-12T21:46:08.760 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:08.760+0000 7f64f9600700 1 -- 172.21.15.18:0/1654472792 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f64d806ec10 msgr2=0x7f64d80710c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:46:08.761 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:08.760+0000 7f64f9600700 1 --2- 172.21.15.18:0/1654472792 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f64d806ec10 0x7f64d80710c0 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f64f4111640 tx=0x7f64dc009e30 comp rx=0 tx=0).stop 2024-04-12T21:46:08.761 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:08.760+0000 7f64f9600700 1 -- 172.21.15.18:0/1654472792 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f64f40fde90 msgr2=0x7f64f4115d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:46:08.761 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:08.760+0000 7f64f9600700 1 --2- 172.21.15.18:0/1654472792 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f64f40fde90 0x7f64f4115d60 secure :-1 s=READY pgs=1177 cs=0 l=1 rev1=1 crypto rx=0x7f64e400c830 tx=0x7f64e4004470 comp rx=0 tx=0).stop 2024-04-12T21:46:08.761 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:08.760+0000 7f64f9600700 1 -- 172.21.15.18:0/1654472792 shutdown_connections 2024-04-12T21:46:08.761 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:08.760+0000 7f64f9600700 1 --2- 172.21.15.18:0/1654472792 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f64d806ec10 0x7f64d80710c0 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:46:08.762 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:08.760+0000 7f64f9600700 1 --2- 172.21.15.18:0/1654472792 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f64f40fde90 0x7f64f4115d60 unknown :-1 s=CLOSED pgs=1177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:46:08.762 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:08.760+0000 7f64f9600700 1 --2- 172.21.15.18:0/1654472792 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f64f40fe830 0x7f64f41162a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:46:08.762 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:08.760+0000 7f64f9600700 1 -- 172.21.15.18:0/1654472792 >> 172.21.15.18:0/1654472792 conn(0x7f64f4072bc0 msgr2=0x7f64f40fd530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:46:08.762 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:08.760+0000 7f64f9600700 1 -- 172.21.15.18:0/1654472792 shutdown_connections 2024-04-12T21:46:08.762 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:08.761+0000 7f64f9600700 1 -- 172.21.15.18:0/1654472792 wait complete. 2024-04-12T21:46:08.855 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:46:08 smithi018 ceph-mon[112590]: from='client.36412 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:46:08.855 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:46:08 smithi018 ceph-mon[112590]: from='client.36416 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:46:08.906 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:08.905+0000 7fc833e00700 1 -- 172.21.15.18:0/1586095971 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc8340708a0 msgr2=0x7fc834070d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:46:08.906 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:08.905+0000 7fc833e00700 1 --2- 172.21.15.18:0/1586095971 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc8340708a0 0x7fc834070d10 secure :-1 s=READY pgs=1178 cs=0 l=1 rev1=1 crypto rx=0x7fc820004970 tx=0x7fc820009e30 comp rx=0 tx=0).stop 2024-04-12T21:46:08.906 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:08.905+0000 7fc833e00700 1 -- 172.21.15.18:0/1586095971 shutdown_connections 2024-04-12T21:46:08.907 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:08.905+0000 7fc833e00700 1 --2- 172.21.15.18:0/1586095971 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc8340708a0 0x7fc834070d10 unknown :-1 s=CLOSED pgs=1178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:46:08.907 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:08.905+0000 7fc833e00700 1 --2- 172.21.15.18:0/1586095971 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc834071f90 0x7fc834070360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:46:08.907 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:08.905+0000 7fc833e00700 1 -- 172.21.15.18:0/1586095971 >> 172.21.15.18:0/1586095971 conn(0x7fc8340f8770 msgr2=0x7fc8340fab80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:46:08.907 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:08.906+0000 7fc833e00700 1 -- 172.21.15.18:0/1586095971 shutdown_connections 2024-04-12T21:46:08.907 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:08.906+0000 7fc833e00700 1 -- 172.21.15.18:0/1586095971 wait complete. 2024-04-12T21:46:08.909 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:08.906+0000 7fc833e00700 1 Processor -- start 2024-04-12T21:46:08.909 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:08.907+0000 7fc833e00700 1 -- start start 2024-04-12T21:46:08.909 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:08.907+0000 7fc833e00700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc8340708a0 0x7fc834119f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:46:08.909 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:08.907+0000 7fc833e00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc834071f90 0x7fc83411a4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:46:08.909 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:08.907+0000 7fc833e00700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc83411ab20 con 0x7fc834071f90 2024-04-12T21:46:08.909 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:08.907+0000 7fc833e00700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc834113ef0 con 0x7fc8340708a0 2024-04-12T21:46:08.909 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:08.907+0000 7fc832000700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc834071f90 0x7fc83411a4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:46:08.909 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:08.907+0000 7fc832000700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc834071f90 0x7fc83411a4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:60096/0 (socket says 172.21.15.18:60096) 2024-04-12T21:46:08.909 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:08.907+0000 7fc832000700 1 -- 172.21.15.18:0/809236031 learned_addr learned my addr 172.21.15.18:0/809236031 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:46:08.910 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:08.908+0000 7fc832a00700 1 --2- 172.21.15.18:0/809236031 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc8340708a0 0x7fc834119f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:46:08.910 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:08.908+0000 7fc832000700 1 -- 172.21.15.18:0/809236031 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc8340708a0 msgr2=0x7fc834119f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:46:08.910 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:08.908+0000 7fc832000700 1 --2- 172.21.15.18:0/809236031 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc8340708a0 0x7fc834119f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:46:08.910 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:08.908+0000 7fc832000700 1 -- 172.21.15.18:0/809236031 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc820005040 con 0x7fc834071f90 2024-04-12T21:46:08.910 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:08.908+0000 7fc832a00700 1 --2- 172.21.15.18:0/809236031 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc8340708a0 0x7fc834119f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-12T21:46:08.910 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:08.908+0000 7fc832000700 1 --2- 172.21.15.18:0/809236031 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc834071f90 0x7fc83411a4d0 secure :-1 s=READY pgs=1179 cs=0 l=1 rev1=1 crypto rx=0x7fc820004f50 tx=0x7fc820009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:46:08.910 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:08.908+0000 7fc82aa00700 1 -- 172.21.15.18:0/809236031 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc82000e040 con 0x7fc834071f90 2024-04-12T21:46:08.910 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:08.908+0000 7fc82aa00700 1 -- 172.21.15.18:0/809236031 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc820004290 con 0x7fc834071f90 2024-04-12T21:46:08.910 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:08.908+0000 7fc833e00700 1 -- 172.21.15.18:0/809236031 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc834114170 con 0x7fc834071f90 2024-04-12T21:46:08.910 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:08.908+0000 7fc82aa00700 1 -- 172.21.15.18:0/809236031 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc82001c630 con 0x7fc834071f90 2024-04-12T21:46:08.910 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:08.909+0000 7fc833e00700 1 -- 172.21.15.18:0/809236031 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc8341146b0 con 0x7fc834071f90 2024-04-12T21:46:08.911 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:08.910+0000 7fc833e00700 1 -- 172.21.15.18:0/809236031 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc834071aa0 con 0x7fc834071f90 2024-04-12T21:46:08.911 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:08.911+0000 7fc82aa00700 1 -- 172.21.15.18:0/809236031 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7fc82000bc10 con 0x7fc834071f90 2024-04-12T21:46:08.912 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:08.911+0000 7fc82aa00700 1 --2- 172.21.15.18:0/809236031 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fc81806ec00 0x7fc8180710b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:46:08.912 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:08.912+0000 7fc82aa00700 1 -- 172.21.15.18:0/809236031 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 6757+0+0 (secure 0 0 0) 0x7fc82008fd70 con 0x7fc834071f90 2024-04-12T21:46:08.912 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:08.912+0000 7fc832a00700 1 --2- 172.21.15.18:0/809236031 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fc81806ec00 0x7fc8180710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:46:08.913 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:08.912+0000 7fc832a00700 1 --2- 172.21.15.18:0/809236031 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fc81806ec00 0x7fc8180710b0 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7fc824007fd0 tx=0x7fc824009040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:46:08.916 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:08.916+0000 7fc82aa00700 1 -- 172.21.15.18:0/809236031 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7fc82005c520 con 0x7fc834071f90 2024-04-12T21:46:09.034 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:46:08 smithi186 ceph-mon[86379]: from='client.36412 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:46:09.034 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:46:08 smithi186 ceph-mon[86379]: from='client.36416 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:46:09.184 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:09.183+0000 7fc833e00700 1 -- 172.21.15.18:0/809236031 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fc83404bed0 con 0x7fc834071f90 2024-04-12T21:46:09.185 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:09.184+0000 7fc82aa00700 1 -- 172.21.15.18:0/809236031 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 33 v33) v1 ==== 76+0+1876 (secure 0 0 0) 0x7fc82005c0b0 con 0x7fc834071f90 2024-04-12T21:46:09.185 INFO:teuthology.orchestra.run.smithi018.stdout:e33 2024-04-12T21:46:09.186 INFO:teuthology.orchestra.run.smithi018.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-12T21:46:09.186 INFO:teuthology.orchestra.run.smithi018.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-12T21:46:09.186 INFO:teuthology.orchestra.run.smithi018.stdout:legacy client fscid: 1 2024-04-12T21:46:09.186 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-12T21:46:09.186 INFO:teuthology.orchestra.run.smithi018.stdout:Filesystem 'cephfs' (1) 2024-04-12T21:46:09.186 INFO:teuthology.orchestra.run.smithi018.stdout:fs_name cephfs 2024-04-12T21:46:09.186 INFO:teuthology.orchestra.run.smithi018.stdout:epoch 24 2024-04-12T21:46:09.186 INFO:teuthology.orchestra.run.smithi018.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-04-12T21:46:09.186 INFO:teuthology.orchestra.run.smithi018.stdout:created 2024-04-12T20:51:57.362850+0000 2024-04-12T21:46:09.186 INFO:teuthology.orchestra.run.smithi018.stdout:modified 2024-04-12T21:06:59.717190+0000 2024-04-12T21:46:09.186 INFO:teuthology.orchestra.run.smithi018.stdout:tableserver 0 2024-04-12T21:46:09.186 INFO:teuthology.orchestra.run.smithi018.stdout:root 0 2024-04-12T21:46:09.186 INFO:teuthology.orchestra.run.smithi018.stdout:session_timeout 60 2024-04-12T21:46:09.186 INFO:teuthology.orchestra.run.smithi018.stdout:session_autoclose 300 2024-04-12T21:46:09.187 INFO:teuthology.orchestra.run.smithi018.stdout:max_file_size 1099511627776 2024-04-12T21:46:09.187 INFO:teuthology.orchestra.run.smithi018.stdout:max_xattr_size 65536 2024-04-12T21:46:09.187 INFO:teuthology.orchestra.run.smithi018.stdout:required_client_features {} 2024-04-12T21:46:09.187 INFO:teuthology.orchestra.run.smithi018.stdout:last_failure 0 2024-04-12T21:46:09.187 INFO:teuthology.orchestra.run.smithi018.stdout:last_failure_osd_epoch 100 2024-04-12T21:46:09.187 INFO:teuthology.orchestra.run.smithi018.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-12T21:46:09.187 INFO:teuthology.orchestra.run.smithi018.stdout:max_mds 2 2024-04-12T21:46:09.187 INFO:teuthology.orchestra.run.smithi018.stdout:in 0,1 2024-04-12T21:46:09.187 INFO:teuthology.orchestra.run.smithi018.stdout:up {} 2024-04-12T21:46:09.187 INFO:teuthology.orchestra.run.smithi018.stdout:failed 0,1 2024-04-12T21:46:09.187 INFO:teuthology.orchestra.run.smithi018.stdout:damaged 2024-04-12T21:46:09.187 INFO:teuthology.orchestra.run.smithi018.stdout:stopped 2024-04-12T21:46:09.187 INFO:teuthology.orchestra.run.smithi018.stdout:data_pools [3] 2024-04-12T21:46:09.187 INFO:teuthology.orchestra.run.smithi018.stdout:metadata_pool 2 2024-04-12T21:46:09.187 INFO:teuthology.orchestra.run.smithi018.stdout:inline_data disabled 2024-04-12T21:46:09.187 INFO:teuthology.orchestra.run.smithi018.stdout:balancer 2024-04-12T21:46:09.188 INFO:teuthology.orchestra.run.smithi018.stdout:bal_rank_mask -1 2024-04-12T21:46:09.188 INFO:teuthology.orchestra.run.smithi018.stdout:standby_count_wanted 1 2024-04-12T21:46:09.188 INFO:teuthology.orchestra.run.smithi018.stdout:qdb_cluster leader: 0 members: 2024-04-12T21:46:09.188 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-12T21:46:09.188 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-12T21:46:09.188 INFO:teuthology.orchestra.run.smithi018.stdout:Standby daemons: 2024-04-12T21:46:09.188 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-12T21:46:09.188 INFO:teuthology.orchestra.run.smithi018.stdout:[mds.cephfs.smithi018.acqbsc{-1:34436} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.18:6828/755179764,v1:172.21.15.18:6829/755179764] compat {c=[1],r=[1],i=[1fff]}] 2024-04-12T21:46:09.188 INFO:teuthology.orchestra.run.smithi018.stdout:[mds.cephfs.smithi018.szjluo{-1:34468} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.18:6826/2407992653,v1:172.21.15.18:6827/2407992653] compat {c=[1],r=[1],i=[1fff]}] 2024-04-12T21:46:09.188 INFO:teuthology.orchestra.run.smithi018.stdout:[mds.cephfs.smithi186.jdllyn{-1:44595} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.186:6824/3523444202,v1:172.21.15.186:6825/3523444202] compat {c=[1],r=[1],i=[1fff]}] 2024-04-12T21:46:09.188 INFO:teuthology.orchestra.run.smithi018.stdout:[mds.cephfs.smithi186.wjnafs{-1:44875} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.186:6826/3122443766,v1:172.21.15.186:6827/3122443766] compat {c=[1],r=[1],i=[1fff]}] 2024-04-12T21:46:09.189 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:09.187+0000 7fc833e00700 1 -- 172.21.15.18:0/809236031 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fc81806ec00 msgr2=0x7fc8180710b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:46:09.189 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:09.187+0000 7fc833e00700 1 --2- 172.21.15.18:0/809236031 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fc81806ec00 0x7fc8180710b0 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7fc824007fd0 tx=0x7fc824009040 comp rx=0 tx=0).stop 2024-04-12T21:46:09.189 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:09.187+0000 7fc833e00700 1 -- 172.21.15.18:0/809236031 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc834071f90 msgr2=0x7fc83411a4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:46:09.189 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:09.187+0000 7fc833e00700 1 --2- 172.21.15.18:0/809236031 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc834071f90 0x7fc83411a4d0 secure :-1 s=READY pgs=1179 cs=0 l=1 rev1=1 crypto rx=0x7fc820004f50 tx=0x7fc820009e30 comp rx=0 tx=0).stop 2024-04-12T21:46:09.189 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:09.188+0000 7fc833e00700 1 -- 172.21.15.18:0/809236031 shutdown_connections 2024-04-12T21:46:09.189 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:09.188+0000 7fc833e00700 1 --2- 172.21.15.18:0/809236031 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fc81806ec00 0x7fc8180710b0 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:46:09.189 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:09.188+0000 7fc833e00700 1 --2- 172.21.15.18:0/809236031 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc8340708a0 0x7fc834119f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:46:09.190 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:09.188+0000 7fc833e00700 1 --2- 172.21.15.18:0/809236031 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc834071f90 0x7fc83411a4d0 unknown :-1 s=CLOSED pgs=1179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:46:09.190 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:09.188+0000 7fc833e00700 1 -- 172.21.15.18:0/809236031 >> 172.21.15.18:0/809236031 conn(0x7fc8340f8770 msgr2=0x7fc8340f9850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:46:09.190 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:09.188+0000 7fc833e00700 1 -- 172.21.15.18:0/809236031 shutdown_connections 2024-04-12T21:46:09.190 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:09.188+0000 7fc833e00700 1 -- 172.21.15.18:0/809236031 wait complete. 2024-04-12T21:46:09.190 INFO:teuthology.orchestra.run.smithi018.stderr:dumped fsmap epoch 33 2024-04-12T21:46:09.336 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:09.335+0000 7f01f2e00700 1 -- 172.21.15.18:0/2162131421 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f01ec0fc620 msgr2=0x7f01ec100cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:46:09.337 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:09.335+0000 7f01f2e00700 1 --2- 172.21.15.18:0/2162131421 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f01ec0fc620 0x7f01ec100cd0 secure :-1 s=READY pgs=1180 cs=0 l=1 rev1=1 crypto rx=0x7f01d8004940 tx=0x7f01d8009e30 comp rx=0 tx=0).stop 2024-04-12T21:46:09.337 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:09.335+0000 7f01f2e00700 1 -- 172.21.15.18:0/2162131421 shutdown_connections 2024-04-12T21:46:09.337 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:09.335+0000 7f01f2e00700 1 --2- 172.21.15.18:0/2162131421 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f01ec0fc620 0x7f01ec100cd0 unknown :-1 s=CLOSED pgs=1180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:46:09.337 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:09.335+0000 7f01f2e00700 1 --2- 172.21.15.18:0/2162131421 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f01ec0fbc80 0x7f01ec0fc050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:46:09.337 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:09.335+0000 7f01f2e00700 1 -- 172.21.15.18:0/2162131421 >> 172.21.15.18:0/2162131421 conn(0x7f01ec072bc0 msgr2=0x7f01ec072fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:46:09.337 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:09.336+0000 7f01f2e00700 1 -- 172.21.15.18:0/2162131421 shutdown_connections 2024-04-12T21:46:09.337 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:09.336+0000 7f01f2e00700 1 -- 172.21.15.18:0/2162131421 wait complete. 2024-04-12T21:46:09.338 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:09.336+0000 7f01f2e00700 1 Processor -- start 2024-04-12T21:46:09.338 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:09.337+0000 7f01f2e00700 1 -- start start 2024-04-12T21:46:09.338 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:09.337+0000 7f01f2e00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f01ec0fbc80 0x7f01ec11a1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:46:09.339 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:09.337+0000 7f01f2e00700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f01ec0fc620 0x7f01ec11a710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:46:09.339 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:09.337+0000 7f01f2e00700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01ec11ad60 con 0x7f01ec0fbc80 2024-04-12T21:46:09.339 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:09.337+0000 7f01f2e00700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01ec114130 con 0x7f01ec0fc620 2024-04-12T21:46:09.339 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:09.337+0000 7f01ebe00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f01ec0fbc80 0x7f01ec11a1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:46:09.339 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:09.337+0000 7f01ebe00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f01ec0fbc80 0x7f01ec11a1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:60128/0 (socket says 172.21.15.18:60128) 2024-04-12T21:46:09.339 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:09.337+0000 7f01ebe00700 1 -- 172.21.15.18:0/937713371 learned_addr learned my addr 172.21.15.18:0/937713371 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:46:09.339 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:09.338+0000 7f01eb400700 1 --2- 172.21.15.18:0/937713371 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f01ec0fc620 0x7f01ec11a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:46:09.339 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:09.338+0000 7f01ebe00700 1 -- 172.21.15.18:0/937713371 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f01ec0fc620 msgr2=0x7f01ec11a710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:46:09.339 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:09.338+0000 7f01ebe00700 1 --2- 172.21.15.18:0/937713371 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f01ec0fc620 0x7f01ec11a710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:46:09.339 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:09.338+0000 7f01ebe00700 1 -- 172.21.15.18:0/937713371 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01d8005040 con 0x7f01ec0fbc80 2024-04-12T21:46:09.339 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:09.338+0000 7f01eb400700 1 --2- 172.21.15.18:0/937713371 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f01ec0fc620 0x7f01ec11a710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-12T21:46:09.340 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:09.338+0000 7f01ebe00700 1 --2- 172.21.15.18:0/937713371 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f01ec0fbc80 0x7f01ec11a1d0 secure :-1 s=READY pgs=1181 cs=0 l=1 rev1=1 crypto rx=0x7f01dc00c830 tx=0x7f01dc004470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:46:09.340 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:09.339+0000 7f01e3e00700 1 -- 172.21.15.18:0/937713371 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f01dc00a760 con 0x7f01ec0fbc80 2024-04-12T21:46:09.340 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:09.339+0000 7f01e3e00700 1 -- 172.21.15.18:0/937713371 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f01dc00ad90 con 0x7f01ec0fbc80 2024-04-12T21:46:09.340 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:09.339+0000 7f01e3e00700 1 -- 172.21.15.18:0/937713371 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f01dc0113a0 con 0x7f01ec0fbc80 2024-04-12T21:46:09.340 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:09.339+0000 7f01f2e00700 1 -- 172.21.15.18:0/937713371 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f01ec114410 con 0x7f01ec0fbc80 2024-04-12T21:46:09.340 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:09.339+0000 7f01f2e00700 1 -- 172.21.15.18:0/937713371 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f01ec114950 con 0x7f01ec0fbc80 2024-04-12T21:46:09.341 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:09.340+0000 7f01f2e00700 1 -- 172.21.15.18:0/937713371 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f01ec065e30 con 0x7f01ec0fbc80 2024-04-12T21:46:09.341 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:09.341+0000 7f01e3e00700 1 -- 172.21.15.18:0/937713371 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7f01dc012400 con 0x7f01ec0fbc80 2024-04-12T21:46:09.343 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:09.342+0000 7f01e3e00700 1 --2- 172.21.15.18:0/937713371 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f01d006ec10 0x7f01d00710c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:46:09.343 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:09.342+0000 7f01e3e00700 1 -- 172.21.15.18:0/937713371 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 6757+0+0 (secure 0 0 0) 0x7f01dc08e9c0 con 0x7f01ec0fbc80 2024-04-12T21:46:09.343 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:09.342+0000 7f01eb400700 1 --2- 172.21.15.18:0/937713371 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f01d006ec10 0x7f01d00710c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:46:09.343 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:09.343+0000 7f01eb400700 1 --2- 172.21.15.18:0/937713371 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f01d006ec10 0x7f01d00710c0 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f01d8004f90 tx=0x7f01d8009e30 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:46:09.347 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:09.347+0000 7f01e3e00700 1 -- 172.21.15.18:0/937713371 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f01dc05b1f0 con 0x7f01ec0fbc80 2024-04-12T21:46:09.589 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:09.588+0000 7f01f2e00700 1 -- 172.21.15.18:0/937713371 --> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f01ec04bf70 con 0x7f01d006ec10 2024-04-12T21:46:09.591 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:09.590+0000 7f01e3e00700 1 -- 172.21.15.18:0/937713371 <== mgr.34102 v2:172.21.15.18:6800/3812891919 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f01d8008e30 con 0x7f01d006ec10 2024-04-12T21:46:09.591 INFO:teuthology.orchestra.run.smithi018.stdout:{ 2024-04-12T21:46:09.591 INFO:teuthology.orchestra.run.smithi018.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:f4452def3be4d350f4a8e5feaf7ebb1511be9298fe68478ba7a2d6fbd7d69a63", 2024-04-12T21:46:09.591 INFO:teuthology.orchestra.run.smithi018.stdout: "in_progress": true, 2024-04-12T21:46:09.591 INFO:teuthology.orchestra.run.smithi018.stdout: "which": "Upgrading all daemon types on all hosts", 2024-04-12T21:46:09.592 INFO:teuthology.orchestra.run.smithi018.stdout: "services_complete": [ 2024-04-12T21:46:09.592 INFO:teuthology.orchestra.run.smithi018.stdout: "mgr", 2024-04-12T21:46:09.592 INFO:teuthology.orchestra.run.smithi018.stdout: "mds", 2024-04-12T21:46:09.592 INFO:teuthology.orchestra.run.smithi018.stdout: "crash", 2024-04-12T21:46:09.592 INFO:teuthology.orchestra.run.smithi018.stdout: "osd", 2024-04-12T21:46:09.592 INFO:teuthology.orchestra.run.smithi018.stdout: "mon" 2024-04-12T21:46:09.592 INFO:teuthology.orchestra.run.smithi018.stdout: ], 2024-04-12T21:46:09.592 INFO:teuthology.orchestra.run.smithi018.stdout: "progress": "16/21 daemons upgraded", 2024-04-12T21:46:09.592 INFO:teuthology.orchestra.run.smithi018.stdout: "message": "Currently upgrading mds daemons", 2024-04-12T21:46:09.592 INFO:teuthology.orchestra.run.smithi018.stdout: "is_paused": false 2024-04-12T21:46:09.592 INFO:teuthology.orchestra.run.smithi018.stdout:} 2024-04-12T21:46:09.594 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:09.593+0000 7f01f2e00700 1 -- 172.21.15.18:0/937713371 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f01d006ec10 msgr2=0x7f01d00710c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:46:09.594 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:09.593+0000 7f01f2e00700 1 --2- 172.21.15.18:0/937713371 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f01d006ec10 0x7f01d00710c0 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f01d8004f90 tx=0x7f01d8009e30 comp rx=0 tx=0).stop 2024-04-12T21:46:09.594 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:09.593+0000 7f01f2e00700 1 -- 172.21.15.18:0/937713371 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f01ec0fbc80 msgr2=0x7f01ec11a1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:46:09.594 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:09.593+0000 7f01f2e00700 1 --2- 172.21.15.18:0/937713371 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f01ec0fbc80 0x7f01ec11a1d0 secure :-1 s=READY pgs=1181 cs=0 l=1 rev1=1 crypto rx=0x7f01dc00c830 tx=0x7f01dc004470 comp rx=0 tx=0).stop 2024-04-12T21:46:09.594 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:09.593+0000 7f01f2e00700 1 -- 172.21.15.18:0/937713371 shutdown_connections 2024-04-12T21:46:09.594 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:09.593+0000 7f01f2e00700 1 --2- 172.21.15.18:0/937713371 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f01d006ec10 0x7f01d00710c0 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:46:09.594 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:09.593+0000 7f01f2e00700 1 --2- 172.21.15.18:0/937713371 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f01ec0fbc80 0x7f01ec11a1d0 unknown :-1 s=CLOSED pgs=1181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:46:09.594 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:09.593+0000 7f01f2e00700 1 --2- 172.21.15.18:0/937713371 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f01ec0fc620 0x7f01ec11a710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:46:09.595 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:09.593+0000 7f01f2e00700 1 -- 172.21.15.18:0/937713371 >> 172.21.15.18:0/937713371 conn(0x7f01ec072bc0 msgr2=0x7f01ec10aab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:46:09.595 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:09.593+0000 7f01f2e00700 1 -- 172.21.15.18:0/937713371 shutdown_connections 2024-04-12T21:46:09.595 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:09.593+0000 7f01f2e00700 1 -- 172.21.15.18:0/937713371 wait complete. 2024-04-12T21:46:09.743 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:09.742+0000 7f83efc00700 1 -- 172.21.15.18:0/1895692676 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f83e8102e00 msgr2=0x7f83e81031d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:46:09.743 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:09.742+0000 7f83efc00700 1 --2- 172.21.15.18:0/1895692676 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f83e8102e00 0x7f83e81031d0 secure :-1 s=READY pgs=1182 cs=0 l=1 rev1=1 crypto rx=0x7f83dc009ce0 tx=0x7f83dc004d10 comp rx=0 tx=0).stop 2024-04-12T21:46:09.743 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:09.742+0000 7f83efc00700 1 -- 172.21.15.18:0/1895692676 shutdown_connections 2024-04-12T21:46:09.743 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:09.742+0000 7f83efc00700 1 --2- 172.21.15.18:0/1895692676 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f83e81037a0 0x7f83e80fce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:46:09.744 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:09.742+0000 7f83efc00700 1 --2- 172.21.15.18:0/1895692676 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f83e8102e00 0x7f83e81031d0 unknown :-1 s=CLOSED pgs=1182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:46:09.744 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:09.742+0000 7f83efc00700 1 -- 172.21.15.18:0/1895692676 >> 172.21.15.18:0/1895692676 conn(0x7f83e80f89b0 msgr2=0x7f83e80fadc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:46:09.744 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:09.743+0000 7f83efc00700 1 -- 172.21.15.18:0/1895692676 shutdown_connections 2024-04-12T21:46:09.744 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:09.743+0000 7f83efc00700 1 -- 172.21.15.18:0/1895692676 wait complete. 2024-04-12T21:46:09.745 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:09.743+0000 7f83efc00700 1 Processor -- start 2024-04-12T21:46:09.745 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:09.744+0000 7f83efc00700 1 -- start start 2024-04-12T21:46:09.745 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:09.744+0000 7f83efc00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f83e8102e00 0x7f83e8115d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:46:09.745 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:09.744+0000 7f83efc00700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f83e81037a0 0x7f83e8116250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:46:09.745 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:09.744+0000 7f83efc00700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83e8116930 con 0x7f83e8102e00 2024-04-12T21:46:09.746 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:09.744+0000 7f83efc00700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83e810fe80 con 0x7f83e81037a0 2024-04-12T21:46:09.746 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:09.744+0000 7f83ed200700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f83e8102e00 0x7f83e8115d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:46:09.746 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:09.744+0000 7f83ed200700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f83e8102e00 0x7f83e8115d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:60134/0 (socket says 172.21.15.18:60134) 2024-04-12T21:46:09.746 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:09.744+0000 7f83ed200700 1 -- 172.21.15.18:0/1975870057 learned_addr learned my addr 172.21.15.18:0/1975870057 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:46:09.746 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:09.745+0000 7f83ed200700 1 -- 172.21.15.18:0/1975870057 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f83e81037a0 msgr2=0x7f83e8116250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:46:09.746 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:09.745+0000 7f83ed200700 1 --2- 172.21.15.18:0/1975870057 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f83e81037a0 0x7f83e8116250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:46:09.746 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:09.745+0000 7f83ed200700 1 -- 172.21.15.18:0/1975870057 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83dc009940 con 0x7f83e8102e00 2024-04-12T21:46:09.746 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:09.745+0000 7f83ed200700 1 --2- 172.21.15.18:0/1975870057 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f83e8102e00 0x7f83e8115d10 secure :-1 s=READY pgs=1183 cs=0 l=1 rev1=1 crypto rx=0x7f83dc005010 tx=0x7f83dc004d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:46:09.747 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:09.745+0000 7f83e4c00700 1 -- 172.21.15.18:0/1975870057 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f83dc00e040 con 0x7f83e8102e00 2024-04-12T21:46:09.747 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:09.745+0000 7f83e4c00700 1 -- 172.21.15.18:0/1975870057 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f83dc008e20 con 0x7f83e8102e00 2024-04-12T21:46:09.747 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:09.745+0000 7f83efc00700 1 -- 172.21.15.18:0/1975870057 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f83e81100a0 con 0x7f83e8102e00 2024-04-12T21:46:09.747 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:09.745+0000 7f83e4c00700 1 -- 172.21.15.18:0/1975870057 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f83dc01c660 con 0x7f83e8102e00 2024-04-12T21:46:09.747 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:09.746+0000 7f83efc00700 1 -- 172.21.15.18:0/1975870057 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f83e8110660 con 0x7f83e8102e00 2024-04-12T21:46:09.747 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:09.747+0000 7f83efc00700 1 -- 172.21.15.18:0/1975870057 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f83e80fe550 con 0x7f83e8102e00 2024-04-12T21:46:09.748 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:09.748+0000 7f83e4c00700 1 -- 172.21.15.18:0/1975870057 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7f83dc012a70 con 0x7f83e8102e00 2024-04-12T21:46:09.749 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:09.749+0000 7f83e4c00700 1 --2- 172.21.15.18:0/1975870057 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f83d006ec00 0x7f83d00710b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:46:09.749 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:09.749+0000 7f83e4c00700 1 -- 172.21.15.18:0/1975870057 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 6757+0+0 (secure 0 0 0) 0x7f83dc0905f0 con 0x7f83e8102e00 2024-04-12T21:46:09.749 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:09.749+0000 7f83e7e00700 1 --2- 172.21.15.18:0/1975870057 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f83d006ec00 0x7f83d00710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:46:09.750 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:09.750+0000 7f83e7e00700 1 --2- 172.21.15.18:0/1975870057 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f83d006ec00 0x7f83d00710b0 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f83d4006fd0 tx=0x7f83d4006db0 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:46:09.754 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:09.753+0000 7f83e4c00700 1 -- 172.21.15.18:0/1975870057 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f83dc05cd20 con 0x7f83e8102e00 2024-04-12T21:46:09.855 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:46:09 smithi018 ceph-mon[112590]: from='client.36420 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:46:09.855 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:46:09 smithi018 ceph-mon[112590]: pgmap v1394: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:46:09.856 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:46:09 smithi018 ceph-mon[112590]: from='client.? 172.21.15.18:0/1654472792' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T21:46:09.856 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:46:09 smithi018 ceph-mon[112590]: from='client.? 172.21.15.18:0/809236031' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-12T21:46:10.034 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:46:09 smithi186 ceph-mon[86379]: from='client.36420 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:46:10.034 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:46:09 smithi186 ceph-mon[86379]: pgmap v1394: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:46:10.034 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:46:09 smithi186 ceph-mon[86379]: from='client.? 172.21.15.18:0/1654472792' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T21:46:10.034 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:46:09 smithi186 ceph-mon[86379]: from='client.? 172.21.15.18:0/809236031' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-12T21:46:10.076 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:10.075+0000 7f83efc00700 1 -- 172.21.15.18:0/1975870057 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f83e810a000 con 0x7f83e8102e00 2024-04-12T21:46:10.077 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:10.076+0000 7f83e4c00700 1 -- 172.21.15.18:0/1975870057 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+4303 (secure 0 0 0) 0x7f83dc05c8b0 con 0x7f83e8102e00 2024-04-12T21:46:10.077 INFO:teuthology.orchestra.run.smithi018.stdout:HEALTH_ERR failed to probe daemons or devices; 1 filesystem is degraded; 1 filesystem is offline 2024-04-12T21:46:10.077 INFO:teuthology.orchestra.run.smithi018.stdout:[WRN] CEPHADM_REFRESH_FAILED: failed to probe daemons or devices 2024-04-12T21:46:10.077 INFO:teuthology.orchestra.run.smithi018.stdout: host smithi186 `cephadm ceph-volume` failed: cephadm exited with an error code: 1, stderr: Inferring config /var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/mon.smithi186/config 2024-04-12T21:46:10.077 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/podman: timeout after 895 seconds 2024-04-12T21:46:10.077 INFO:teuthology.orchestra.run.smithi018.stdout:Non-zero exit code 124 from /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:f4452def3be4d350f4a8e5feaf7ebb1511be9298fe68478ba7a2d6fbd7d69a63 -e NODE_NAME=smithi186 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968:/var/run/ceph:z -v /var/log/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968:/var/log/ceph:z -v /var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmptgom53f9:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:f4452def3be4d350f4a8e5feaf7ebb1511be9298fe68478ba7a2d6fbd7d69a63 inventory --format=json-pretty --filter-for-batch 2024-04-12T21:46:10.077 INFO:teuthology.orchestra.run.smithi018.stdout:Traceback (most recent call last): 2024-04-12T21:46:10.077 INFO:teuthology.orchestra.run.smithi018.stdout: File "/usr/lib64/python3.9/runpy.py", line 197, in _run_module_as_main 2024-04-12T21:46:10.078 INFO:teuthology.orchestra.run.smithi018.stdout: return _run_code(code, main_globals, None, 2024-04-12T21:46:10.078 INFO:teuthology.orchestra.run.smithi018.stdout: File "/usr/lib64/python3.9/runpy.py", line 87, in _run_code 2024-04-12T21:46:10.078 INFO:teuthology.orchestra.run.smithi018.stdout: exec(code, run_globals) 2024-04-12T21:46:10.078 INFO:teuthology.orchestra.run.smithi018.stdout: File "/var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/cephadm.e44f3ed5e34e4ef52eedef233d1190bff004470d8e8005127b5312bff357fa7f/__main__.py", line 5618, in 2024-04-12T21:46:10.078 INFO:teuthology.orchestra.run.smithi018.stdout: File "/var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/cephadm.e44f3ed5e34e4ef52eedef233d1190bff004470d8e8005127b5312bff357fa7f/__main__.py", line 5606, in main 2024-04-12T21:46:10.078 INFO:teuthology.orchestra.run.smithi018.stdout: File "/var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/cephadm.e44f3ed5e34e4ef52eedef233d1190bff004470d8e8005127b5312bff357fa7f/__main__.py", line 408, in _infer_config 2024-04-12T21:46:10.078 INFO:teuthology.orchestra.run.smithi018.stdout: File "/var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/cephadm.e44f3ed5e34e4ef52eedef233d1190bff004470d8e8005127b5312bff357fa7f/__main__.py", line 323, in _infer_fsid 2024-04-12T21:46:10.078 INFO:teuthology.orchestra.run.smithi018.stdout: File "/var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/cephadm.e44f3ed5e34e4ef52eedef233d1190bff004470d8e8005127b5312bff357fa7f/__main__.py", line 436, in _infer_image 2024-04-12T21:46:10.078 INFO:teuthology.orchestra.run.smithi018.stdout: File "/var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/cephadm.e44f3ed5e34e4ef52eedef233d1190bff004470d8e8005127b5312bff357fa7f/__main__.py", line 310, in _validate_fsid 2024-04-12T21:46:10.078 INFO:teuthology.orchestra.run.smithi018.stdout: File "/var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/cephadm.e44f3ed5e34e4ef52eedef233d1190bff004470d8e8005127b5312bff357fa7f/__main__.py", line 3359, in command_ceph_volume 2024-04-12T21:46:10.078 INFO:teuthology.orchestra.run.smithi018.stdout: File "/var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/cephadm.e44f3ed5e34e4ef52eedef233d1190bff004470d8e8005127b5312bff357fa7f/cephadmlib/call_wrappers.py", line 310, in call_throws 2024-04-12T21:46:10.078 INFO:teuthology.orchestra.run.smithi018.stdout:RuntimeError: Failed command: /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:f4452def3be4d350f4a8e5feaf7ebb1511be9298fe68478ba7a2d6fbd7d69a63 -e NODE_NAME=smithi186 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968:/var/run/ceph:z -v /var/log/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968:/var/log/ceph:z -v /var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmptgom53f9:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:f4452def3be4d350f4a8e5feaf7ebb1511be9298fe68478ba7a2d6fbd7d69a63 inventory --format=json-pretty --filter-for-batch 2024-04-12T21:46:10.079 INFO:teuthology.orchestra.run.smithi018.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-04-12T21:46:10.079 INFO:teuthology.orchestra.run.smithi018.stdout: fs cephfs is degraded 2024-04-12T21:46:10.079 INFO:teuthology.orchestra.run.smithi018.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-04-12T21:46:10.079 INFO:teuthology.orchestra.run.smithi018.stdout: fs cephfs is offline because no MDS is active for it. 2024-04-12T21:46:10.080 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:10.079+0000 7f83efc00700 1 -- 172.21.15.18:0/1975870057 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f83d006ec00 msgr2=0x7f83d00710b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:46:10.080 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:10.079+0000 7f83efc00700 1 --2- 172.21.15.18:0/1975870057 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f83d006ec00 0x7f83d00710b0 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f83d4006fd0 tx=0x7f83d4006db0 comp rx=0 tx=0).stop 2024-04-12T21:46:10.081 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:10.079+0000 7f83efc00700 1 -- 172.21.15.18:0/1975870057 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f83e8102e00 msgr2=0x7f83e8115d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:46:10.081 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:10.080+0000 7f83efc00700 1 --2- 172.21.15.18:0/1975870057 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f83e8102e00 0x7f83e8115d10 secure :-1 s=READY pgs=1183 cs=0 l=1 rev1=1 crypto rx=0x7f83dc005010 tx=0x7f83dc004d10 comp rx=0 tx=0).stop 2024-04-12T21:46:10.081 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:10.080+0000 7f83efc00700 1 -- 172.21.15.18:0/1975870057 shutdown_connections 2024-04-12T21:46:10.081 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:10.080+0000 7f83efc00700 1 --2- 172.21.15.18:0/1975870057 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f83d006ec00 0x7f83d00710b0 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:46:10.081 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:10.080+0000 7f83efc00700 1 --2- 172.21.15.18:0/1975870057 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f83e8102e00 0x7f83e8115d10 unknown :-1 s=CLOSED pgs=1183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:46:10.081 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:10.080+0000 7f83efc00700 1 --2- 172.21.15.18:0/1975870057 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f83e81037a0 0x7f83e8116250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:46:10.081 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:10.080+0000 7f83efc00700 1 -- 172.21.15.18:0/1975870057 >> 172.21.15.18:0/1975870057 conn(0x7f83e80f89b0 msgr2=0x7f83e8071c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:46:10.081 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:10.080+0000 7f83efc00700 1 -- 172.21.15.18:0/1975870057 shutdown_connections 2024-04-12T21:46:10.081 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:10.080+0000 7f83efc00700 1 -- 172.21.15.18:0/1975870057 wait complete. 2024-04-12T21:46:10.855 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:46:10 smithi018 ceph-mon[112590]: from='client.36432 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:46:10.855 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:46:10 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T21:46:10.855 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:46:10 smithi018 ceph-mon[112590]: from='client.? 172.21.15.18:0/1975870057' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-12T21:46:11.034 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:46:10 smithi186 ceph-mon[86379]: from='client.36432 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:46:11.034 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:46:10 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T21:46:11.034 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:46:10 smithi186 ceph-mon[86379]: from='client.? 172.21.15.18:0/1975870057' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-12T21:46:11.855 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:46:11 smithi018 ceph-mon[112590]: pgmap v1395: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:46:12.034 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:46:11 smithi186 ceph-mon[86379]: pgmap v1395: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:46:13.855 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:46:13 smithi018 ceph-mon[112590]: pgmap v1396: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:46:14.034 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:46:13 smithi186 ceph-mon[86379]: pgmap v1396: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:46:15.658 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:46:15 smithi018 ceph-mon[112590]: pgmap v1397: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:46:16.034 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:46:15 smithi186 ceph-mon[86379]: pgmap v1397: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:46:17.855 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:46:17 smithi018 ceph-mon[112590]: pgmap v1398: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:46:18.034 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:46:17 smithi186 ceph-mon[86379]: pgmap v1398: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:46:19.855 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:46:19 smithi018 ceph-mon[112590]: pgmap v1399: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:46:20.034 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:46:19 smithi186 ceph-mon[86379]: pgmap v1399: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:46:21.855 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:46:21 smithi018 ceph-mon[112590]: pgmap v1400: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:46:22.034 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:46:21 smithi186 ceph-mon[86379]: pgmap v1400: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:46:24.034 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:46:23 smithi186 ceph-mon[86379]: pgmap v1401: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:46:24.105 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:46:23 smithi018 ceph-mon[112590]: pgmap v1401: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:46:25.855 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:46:25 smithi018 ceph-mon[112590]: pgmap v1402: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:46:25.855 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:46:25 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T21:46:26.034 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:46:25 smithi186 ceph-mon[86379]: pgmap v1402: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:46:26.034 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:46:25 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T21:46:28.034 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:46:27 smithi186 ceph-mon[86379]: pgmap v1403: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:46:28.105 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:46:27 smithi018 ceph-mon[112590]: pgmap v1403: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:46:30.034 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:46:29 smithi186 ceph-mon[86379]: pgmap v1404: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:46:30.105 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:46:29 smithi018 ceph-mon[112590]: pgmap v1404: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:46:32.034 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:46:31 smithi186 ceph-mon[86379]: pgmap v1405: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:46:32.105 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:46:31 smithi018 ceph-mon[112590]: pgmap v1405: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:46:34.034 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:46:33 smithi186 ceph-mon[86379]: pgmap v1406: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:46:34.105 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:46:33 smithi018 ceph-mon[112590]: pgmap v1406: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:46:36.034 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:46:35 smithi186 ceph-mon[86379]: pgmap v1407: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:46:36.105 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:46:35 smithi018 ceph-mon[112590]: pgmap v1407: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:46:38.034 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:46:37 smithi186 ceph-mon[86379]: pgmap v1408: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:46:38.105 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:46:37 smithi018 ceph-mon[112590]: pgmap v1408: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:46:40.034 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:46:39 smithi186 ceph-mon[86379]: pgmap v1409: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:46:40.105 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:46:39 smithi018 ceph-mon[112590]: pgmap v1409: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:46:40.254 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:40.252+0000 7f86e2400700 1 -- 172.21.15.18:0/1140778687 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f86dc070c10 msgr2=0x7f86dc108fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:46:40.254 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:40.252+0000 7f86e2400700 1 --2- 172.21.15.18:0/1140778687 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f86dc070c10 0x7f86dc108fa0 secure :-1 s=READY pgs=1184 cs=0 l=1 rev1=1 crypto rx=0x7f86c8004970 tx=0x7f86c8009e30 comp rx=0 tx=0).stop 2024-04-12T21:46:40.254 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:40.253+0000 7f86e2400700 1 -- 172.21.15.18:0/1140778687 shutdown_connections 2024-04-12T21:46:40.254 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:40.253+0000 7f86e2400700 1 --2- 172.21.15.18:0/1140778687 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f86dc070c10 0x7f86dc108fa0 unknown :-1 s=CLOSED pgs=1184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:46:40.254 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:40.253+0000 7f86e2400700 1 --2- 172.21.15.18:0/1140778687 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f86dc070300 0x7f86dc0706d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:46:40.254 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:40.253+0000 7f86e2400700 1 -- 172.21.15.18:0/1140778687 >> 172.21.15.18:0/1140778687 conn(0x7f86dc0f8760 msgr2=0x7f86dc0fab70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:46:40.254 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:40.253+0000 7f86e2400700 1 -- 172.21.15.18:0/1140778687 shutdown_connections 2024-04-12T21:46:40.254 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:40.253+0000 7f86e2400700 1 -- 172.21.15.18:0/1140778687 wait complete. 2024-04-12T21:46:40.255 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:40.254+0000 7f86e2400700 1 Processor -- start 2024-04-12T21:46:40.255 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:40.254+0000 7f86e2400700 1 -- start start 2024-04-12T21:46:40.255 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:40.255+0000 7f86e2400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f86dc070300 0x7f86dc115ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:46:40.255 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:40.255+0000 7f86e2400700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f86dc070c10 0x7f86dc1160e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:46:40.256 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:40.255+0000 7f86e2400700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86dc116730 con 0x7f86dc070300 2024-04-12T21:46:40.256 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:40.255+0000 7f86e2400700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86dc10fb20 con 0x7f86dc070c10 2024-04-12T21:46:40.256 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:40.255+0000 7f86e1000700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f86dc070300 0x7f86dc115ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:46:40.256 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:40.255+0000 7f86e1000700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f86dc070300 0x7f86dc115ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:40920/0 (socket says 172.21.15.18:40920) 2024-04-12T21:46:40.256 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:40.255+0000 7f86e1000700 1 -- 172.21.15.18:0/523984524 learned_addr learned my addr 172.21.15.18:0/523984524 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:46:40.257 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:40.255+0000 7f86dbe00700 1 --2- 172.21.15.18:0/523984524 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f86dc070c10 0x7f86dc1160e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:46:40.257 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:40.255+0000 7f86e1000700 1 -- 172.21.15.18:0/523984524 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f86dc070c10 msgr2=0x7f86dc1160e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:46:40.257 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:40.255+0000 7f86e1000700 1 --2- 172.21.15.18:0/523984524 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f86dc070c10 0x7f86dc1160e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:46:40.257 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:40.255+0000 7f86e1000700 1 -- 172.21.15.18:0/523984524 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86c8005040 con 0x7f86dc070300 2024-04-12T21:46:40.257 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:40.256+0000 7f86e1000700 1 --2- 172.21.15.18:0/523984524 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f86dc070300 0x7f86dc115ba0 secure :-1 s=READY pgs=1185 cs=0 l=1 rev1=1 crypto rx=0x7f86d0002a50 tx=0x7f86d00046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:46:40.257 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:40.256+0000 7f86d8c00700 1 -- 172.21.15.18:0/523984524 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f86d0007cc0 con 0x7f86dc070300 2024-04-12T21:46:40.257 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:40.256+0000 7f86d8c00700 1 -- 172.21.15.18:0/523984524 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f86d000f040 con 0x7f86dc070300 2024-04-12T21:46:40.257 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:40.256+0000 7f86d8c00700 1 -- 172.21.15.18:0/523984524 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f86d0009640 con 0x7f86dc070300 2024-04-12T21:46:40.257 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:40.256+0000 7f86e2400700 1 -- 172.21.15.18:0/523984524 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f86dc10fe00 con 0x7f86dc070300 2024-04-12T21:46:40.257 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:40.256+0000 7f86e2400700 1 -- 172.21.15.18:0/523984524 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f86dc110340 con 0x7f86dc070300 2024-04-12T21:46:40.258 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:40.258+0000 7f86e2400700 1 -- 172.21.15.18:0/523984524 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f86dc071a40 con 0x7f86dc070300 2024-04-12T21:46:40.259 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:40.259+0000 7f86d8c00700 1 -- 172.21.15.18:0/523984524 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7f86d00097a0 con 0x7f86dc070300 2024-04-12T21:46:40.260 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:40.260+0000 7f86d8c00700 1 --2- 172.21.15.18:0/523984524 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f86c406ec00 0x7f86c40710b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:46:40.261 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:40.260+0000 7f86d8c00700 1 -- 172.21.15.18:0/523984524 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 6757+0+0 (secure 0 0 0) 0x7f86d008e760 con 0x7f86dc070300 2024-04-12T21:46:40.261 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:40.260+0000 7f86dbe00700 1 --2- 172.21.15.18:0/523984524 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f86c406ec00 0x7f86c40710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:46:40.261 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:40.261+0000 7f86dbe00700 1 --2- 172.21.15.18:0/523984524 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f86c406ec00 0x7f86c40710b0 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f86c8009fd0 tx=0x7f86c8009e30 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:46:40.266 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:40.265+0000 7f86d8c00700 1 -- 172.21.15.18:0/523984524 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f86d005af10 con 0x7f86dc070300 2024-04-12T21:46:40.518 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:40.518+0000 7f86e2400700 1 -- 172.21.15.18:0/523984524 --> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f86dc04bed0 con 0x7f86c406ec00 2024-04-12T21:46:40.522 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:40.522+0000 7f86d8c00700 1 -- 172.21.15.18:0/523984524 <== mgr.34102 v2:172.21.15.18:6800/3812891919 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f86dc04bed0 con 0x7f86c406ec00 2024-04-12T21:46:40.525 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:40.525+0000 7f86e2400700 1 -- 172.21.15.18:0/523984524 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f86c406ec00 msgr2=0x7f86c40710b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:46:40.525 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:40.525+0000 7f86e2400700 1 --2- 172.21.15.18:0/523984524 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f86c406ec00 0x7f86c40710b0 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f86c8009fd0 tx=0x7f86c8009e30 comp rx=0 tx=0).stop 2024-04-12T21:46:40.525 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:40.525+0000 7f86e2400700 1 -- 172.21.15.18:0/523984524 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f86dc070300 msgr2=0x7f86dc115ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:46:40.526 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:40.525+0000 7f86e2400700 1 --2- 172.21.15.18:0/523984524 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f86dc070300 0x7f86dc115ba0 secure :-1 s=READY pgs=1185 cs=0 l=1 rev1=1 crypto rx=0x7f86d0002a50 tx=0x7f86d00046b0 comp rx=0 tx=0).stop 2024-04-12T21:46:40.526 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:40.525+0000 7f86e2400700 1 -- 172.21.15.18:0/523984524 shutdown_connections 2024-04-12T21:46:40.526 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:40.525+0000 7f86e2400700 1 --2- 172.21.15.18:0/523984524 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f86c406ec00 0x7f86c40710b0 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:46:40.526 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:40.525+0000 7f86e2400700 1 --2- 172.21.15.18:0/523984524 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f86dc070300 0x7f86dc115ba0 unknown :-1 s=CLOSED pgs=1185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:46:40.526 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:40.525+0000 7f86e2400700 1 --2- 172.21.15.18:0/523984524 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f86dc070c10 0x7f86dc1160e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:46:40.526 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:40.525+0000 7f86e2400700 1 -- 172.21.15.18:0/523984524 >> 172.21.15.18:0/523984524 conn(0x7f86dc0f8760 msgr2=0x7f86dc1057f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:46:40.526 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:40.525+0000 7f86e2400700 1 -- 172.21.15.18:0/523984524 shutdown_connections 2024-04-12T21:46:40.526 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:40.525+0000 7f86e2400700 1 -- 172.21.15.18:0/523984524 wait complete. 2024-04-12T21:46:40.541 INFO:teuthology.orchestra.run.smithi018.stdout:true 2024-04-12T21:46:40.680 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:40.679+0000 7fc2a1000700 1 -- 172.21.15.18:0/4238819229 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc29c070c10 msgr2=0x7fc29c109190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:46:40.680 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:40.679+0000 7fc2a1000700 1 --2- 172.21.15.18:0/4238819229 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc29c070c10 0x7fc29c109190 secure :-1 s=READY pgs=1186 cs=0 l=1 rev1=1 crypto rx=0x7fc284004970 tx=0x7fc284009e30 comp rx=0 tx=0).stop 2024-04-12T21:46:40.680 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:40.679+0000 7fc2a1000700 1 -- 172.21.15.18:0/4238819229 shutdown_connections 2024-04-12T21:46:40.680 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:40.679+0000 7fc2a1000700 1 --2- 172.21.15.18:0/4238819229 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc29c070c10 0x7fc29c109190 unknown :-1 s=CLOSED pgs=1186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:46:40.680 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:40.679+0000 7fc2a1000700 1 --2- 172.21.15.18:0/4238819229 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc29c070300 0x7fc29c0706d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:46:40.680 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:40.679+0000 7fc2a1000700 1 -- 172.21.15.18:0/4238819229 >> 172.21.15.18:0/4238819229 conn(0x7fc29c0f8970 msgr2=0x7fc29c0fad80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:46:40.680 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:40.679+0000 7fc2a1000700 1 -- 172.21.15.18:0/4238819229 shutdown_connections 2024-04-12T21:46:40.681 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:40.679+0000 7fc2a1000700 1 -- 172.21.15.18:0/4238819229 wait complete. 2024-04-12T21:46:40.681 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:40.680+0000 7fc2a1000700 1 Processor -- start 2024-04-12T21:46:40.681 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:40.681+0000 7fc2a1000700 1 -- start start 2024-04-12T21:46:40.682 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:40.681+0000 7fc2a1000700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc29c070300 0x7fc29c115d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:46:40.682 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:40.681+0000 7fc2a1000700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc29c070c10 0x7fc29c1162c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:46:40.682 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:40.681+0000 7fc2a1000700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc29c116910 con 0x7fc29c070300 2024-04-12T21:46:40.682 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:40.681+0000 7fc2a1000700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc29c10fd20 con 0x7fc29c070c10 2024-04-12T21:46:40.683 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:40.682+0000 7fc299e00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc29c070300 0x7fc29c115d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:46:40.683 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:40.682+0000 7fc299e00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc29c070300 0x7fc29c115d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:40942/0 (socket says 172.21.15.18:40942) 2024-04-12T21:46:40.683 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:40.682+0000 7fc299e00700 1 -- 172.21.15.18:0/499560629 learned_addr learned my addr 172.21.15.18:0/499560629 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:46:40.683 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:40.682+0000 7fc293e00700 1 --2- 172.21.15.18:0/499560629 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc29c070c10 0x7fc29c1162c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:46:40.683 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:40.682+0000 7fc299e00700 1 -- 172.21.15.18:0/499560629 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc29c070c10 msgr2=0x7fc29c1162c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:46:40.683 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:40.682+0000 7fc299e00700 1 --2- 172.21.15.18:0/499560629 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc29c070c10 0x7fc29c1162c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:46:40.683 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:40.682+0000 7fc299e00700 1 -- 172.21.15.18:0/499560629 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc284005040 con 0x7fc29c070300 2024-04-12T21:46:40.683 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:40.683+0000 7fc299e00700 1 --2- 172.21.15.18:0/499560629 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc29c070300 0x7fc29c115d80 secure :-1 s=READY pgs=1187 cs=0 l=1 rev1=1 crypto rx=0x7fc28c002a50 tx=0x7fc28c0046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:46:40.686 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:40.685+0000 7fc292000700 1 -- 172.21.15.18:0/499560629 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc28c007cc0 con 0x7fc29c070300 2024-04-12T21:46:40.686 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:40.685+0000 7fc292000700 1 -- 172.21.15.18:0/499560629 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc28c00f040 con 0x7fc29c070300 2024-04-12T21:46:40.686 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:40.685+0000 7fc292000700 1 -- 172.21.15.18:0/499560629 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc28c00db70 con 0x7fc29c070300 2024-04-12T21:46:40.686 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:40.685+0000 7fc2a1000700 1 -- 172.21.15.18:0/499560629 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc29c110000 con 0x7fc29c070300 2024-04-12T21:46:40.686 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:40.685+0000 7fc2a1000700 1 -- 172.21.15.18:0/499560629 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc29c110540 con 0x7fc29c070300 2024-04-12T21:46:40.687 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:40.687+0000 7fc2a1000700 1 -- 172.21.15.18:0/499560629 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc29c071a40 con 0x7fc29c070300 2024-04-12T21:46:40.688 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:40.688+0000 7fc292000700 1 -- 172.21.15.18:0/499560629 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7fc28c00dcd0 con 0x7fc29c070300 2024-04-12T21:46:40.689 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:40.689+0000 7fc292000700 1 --2- 172.21.15.18:0/499560629 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fc28006ecc0 0x7fc280071170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:46:40.689 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:40.689+0000 7fc292000700 1 -- 172.21.15.18:0/499560629 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 6757+0+0 (secure 0 0 0) 0x7fc28c08dfb0 con 0x7fc29c070300 2024-04-12T21:46:40.690 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:40.689+0000 7fc293e00700 1 --2- 172.21.15.18:0/499560629 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fc28006ecc0 0x7fc280071170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:46:40.690 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:40.690+0000 7fc293e00700 1 --2- 172.21.15.18:0/499560629 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fc28006ecc0 0x7fc280071170 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7fc29c111660 tx=0x7fc284009e30 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:46:40.695 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:40.694+0000 7fc292000700 1 -- 172.21.15.18:0/499560629 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7fc28c0c2ff0 con 0x7fc29c070300 2024-04-12T21:46:40.855 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:46:40 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T21:46:40.950 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:40.949+0000 7fc2a1000700 1 -- 172.21.15.18:0/499560629 --> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fc29c04bf20 con 0x7fc28006ecc0 2024-04-12T21:46:40.953 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:40.952+0000 7fc292000700 1 -- 172.21.15.18:0/499560629 <== mgr.34102 v2:172.21.15.18:6800/3812891919 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7fc29c04bf20 con 0x7fc28006ecc0 2024-04-12T21:46:40.956 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:40.955+0000 7fc2a1000700 1 -- 172.21.15.18:0/499560629 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fc28006ecc0 msgr2=0x7fc280071170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:46:40.956 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:40.955+0000 7fc2a1000700 1 --2- 172.21.15.18:0/499560629 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fc28006ecc0 0x7fc280071170 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7fc29c111660 tx=0x7fc284009e30 comp rx=0 tx=0).stop 2024-04-12T21:46:40.956 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:40.955+0000 7fc2a1000700 1 -- 172.21.15.18:0/499560629 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc29c070300 msgr2=0x7fc29c115d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:46:40.956 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:40.955+0000 7fc2a1000700 1 --2- 172.21.15.18:0/499560629 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc29c070300 0x7fc29c115d80 secure :-1 s=READY pgs=1187 cs=0 l=1 rev1=1 crypto rx=0x7fc28c002a50 tx=0x7fc28c0046b0 comp rx=0 tx=0).stop 2024-04-12T21:46:40.956 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:40.955+0000 7fc2a1000700 1 -- 172.21.15.18:0/499560629 shutdown_connections 2024-04-12T21:46:40.957 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:40.955+0000 7fc2a1000700 1 --2- 172.21.15.18:0/499560629 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fc28006ecc0 0x7fc280071170 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:46:40.957 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:40.955+0000 7fc2a1000700 1 --2- 172.21.15.18:0/499560629 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc29c070300 0x7fc29c115d80 unknown :-1 s=CLOSED pgs=1187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:46:40.957 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:40.955+0000 7fc2a1000700 1 --2- 172.21.15.18:0/499560629 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc29c070c10 0x7fc29c1162c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:46:40.957 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:40.955+0000 7fc2a1000700 1 -- 172.21.15.18:0/499560629 >> 172.21.15.18:0/499560629 conn(0x7fc29c0f8970 msgr2=0x7fc29c0f9dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:46:40.957 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:40.956+0000 7fc2a1000700 1 -- 172.21.15.18:0/499560629 shutdown_connections 2024-04-12T21:46:40.957 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:40.956+0000 7fc2a1000700 1 -- 172.21.15.18:0/499560629 wait complete. 2024-04-12T21:46:41.034 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:46:40 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T21:46:41.118 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:41.117+0000 7f1ba4000700 1 -- 172.21.15.18:0/1030885688 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1b9c109430 msgr2=0x7f1b9c109800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:46:41.118 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:41.117+0000 7f1ba4000700 1 --2- 172.21.15.18:0/1030885688 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1b9c109430 0x7f1b9c109800 secure :-1 s=READY pgs=1188 cs=0 l=1 rev1=1 crypto rx=0x7f1b90004940 tx=0x7f1b90009e30 comp rx=0 tx=0).stop 2024-04-12T21:46:41.118 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:41.117+0000 7f1ba4000700 1 -- 172.21.15.18:0/1030885688 shutdown_connections 2024-04-12T21:46:41.118 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:41.117+0000 7f1ba4000700 1 --2- 172.21.15.18:0/1030885688 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1b9c109dd0 0x7f1b9c103440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:46:41.119 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:41.117+0000 7f1ba4000700 1 --2- 172.21.15.18:0/1030885688 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1b9c109430 0x7f1b9c109800 unknown :-1 s=CLOSED pgs=1188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:46:41.119 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:41.117+0000 7f1ba4000700 1 -- 172.21.15.18:0/1030885688 >> 172.21.15.18:0/1030885688 conn(0x7f1b9c072c00 msgr2=0x7f1b9c075010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:46:41.119 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:41.117+0000 7f1ba4000700 1 -- 172.21.15.18:0/1030885688 shutdown_connections 2024-04-12T21:46:41.119 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:41.118+0000 7f1ba4000700 1 -- 172.21.15.18:0/1030885688 wait complete. 2024-04-12T21:46:41.120 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:41.118+0000 7f1ba4000700 1 Processor -- start 2024-04-12T21:46:41.120 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:41.119+0000 7f1ba4000700 1 -- start start 2024-04-12T21:46:41.120 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:41.119+0000 7f1ba4000700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1b9c109430 0x7f1b9c119f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:46:41.120 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:41.119+0000 7f1ba4000700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1b9c109dd0 0x7f1b9c11a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:46:41.120 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:41.119+0000 7f1ba4000700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b9c11aad0 con 0x7f1b9c109dd0 2024-04-12T21:46:41.120 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:41.119+0000 7f1ba4000700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b9c114100 con 0x7f1b9c109430 2024-04-12T21:46:41.121 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:41.119+0000 7f1ba0c00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1b9c109dd0 0x7f1b9c11a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:46:41.121 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:41.119+0000 7f1ba0c00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1b9c109dd0 0x7f1b9c11a480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:53386/0 (socket says 172.21.15.18:53386) 2024-04-12T21:46:41.121 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:41.119+0000 7f1ba0c00700 1 -- 172.21.15.18:0/1919542359 learned_addr learned my addr 172.21.15.18:0/1919542359 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:46:41.121 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:41.120+0000 7f1ba1600700 1 --2- 172.21.15.18:0/1919542359 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1b9c109430 0x7f1b9c119f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:46:41.121 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:41.120+0000 7f1ba0c00700 1 -- 172.21.15.18:0/1919542359 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1b9c109430 msgr2=0x7f1b9c119f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:46:41.121 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:41.120+0000 7f1ba0c00700 1 --2- 172.21.15.18:0/1919542359 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1b9c109430 0x7f1b9c119f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:46:41.121 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:41.120+0000 7f1ba0c00700 1 -- 172.21.15.18:0/1919542359 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b90005040 con 0x7f1b9c109dd0 2024-04-12T21:46:41.121 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:41.120+0000 7f1ba1600700 1 --2- 172.21.15.18:0/1919542359 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1b9c109430 0x7f1b9c119f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-12T21:46:41.122 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:41.120+0000 7f1ba0c00700 1 --2- 172.21.15.18:0/1919542359 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1b9c109dd0 0x7f1b9c11a480 secure :-1 s=READY pgs=1189 cs=0 l=1 rev1=1 crypto rx=0x7f1b88002a50 tx=0x7f1b88004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:46:41.122 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:41.120+0000 7f1b99600700 1 -- 172.21.15.18:0/1919542359 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1b88007cc0 con 0x7f1b9c109dd0 2024-04-12T21:46:41.122 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:41.120+0000 7f1b99600700 1 -- 172.21.15.18:0/1919542359 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f1b8800f040 con 0x7f1b9c109dd0 2024-04-12T21:46:41.122 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:41.120+0000 7f1ba4000700 1 -- 172.21.15.18:0/1919542359 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1b9c1143e0 con 0x7f1b9c109dd0 2024-04-12T21:46:41.122 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:41.121+0000 7f1b99600700 1 -- 172.21.15.18:0/1919542359 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1b8800daf0 con 0x7f1b9c109dd0 2024-04-12T21:46:41.122 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:41.121+0000 7f1ba4000700 1 -- 172.21.15.18:0/1919542359 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1b9c1149a0 con 0x7f1b9c109dd0 2024-04-12T21:46:41.123 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:41.122+0000 7f1ba4000700 1 -- 172.21.15.18:0/1919542359 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1b9c104b80 con 0x7f1b9c109dd0 2024-04-12T21:46:41.123 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:41.123+0000 7f1b99600700 1 -- 172.21.15.18:0/1919542359 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7f1b88009810 con 0x7f1b9c109dd0 2024-04-12T21:46:41.125 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:41.124+0000 7f1b99600700 1 --2- 172.21.15.18:0/1919542359 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f1b8406ec00 0x7f1b840710b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:46:41.125 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:41.124+0000 7f1b99600700 1 -- 172.21.15.18:0/1919542359 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 6757+0+0 (secure 0 0 0) 0x7f1b8808e700 con 0x7f1b9c109dd0 2024-04-12T21:46:41.125 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:41.125+0000 7f1ba1600700 1 --2- 172.21.15.18:0/1919542359 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f1b8406ec00 0x7f1b840710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:46:41.126 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:41.125+0000 7f1ba1600700 1 --2- 172.21.15.18:0/1919542359 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f1b8406ec00 0x7f1b840710b0 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f1b9000b3b0 tx=0x7f1b90009e30 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:46:41.129 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:41.129+0000 7f1b99600700 1 -- 172.21.15.18:0/1919542359 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f1b8805aeb0 con 0x7f1b9c109dd0 2024-04-12T21:46:41.370 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:41.369+0000 7f1ba4000700 1 -- 172.21.15.18:0/1919542359 --> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f1b9c110590 con 0x7f1b8406ec00 2024-04-12T21:46:41.383 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:41.382+0000 7f1b99600700 1 -- 172.21.15.18:0/1919542359 <== mgr.34102 v2:172.21.15.18:6800/3812891919 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7f1b9c110590 con 0x7f1b8406ec00 2024-04-12T21:46:41.383 INFO:teuthology.orchestra.run.smithi018.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-12T21:46:41.384 INFO:teuthology.orchestra.run.smithi018.stdout:alertmanager.smithi018 smithi018 *:9093,9094 running (47m) 8m ago 61m 14.8M - 0.25.0 c8568f914cd2 c332090068e8 2024-04-12T21:46:41.384 INFO:teuthology.orchestra.run.smithi018.stdout:crash.smithi018 smithi018 running (43m) 8m ago 62m 6665k - 19.0.0-2879-g324baf8d 8717720dc5e5 b7f80d1f7982 2024-04-12T21:46:41.384 INFO:teuthology.orchestra.run.smithi018.stdout:crash.smithi186 smithi186 running (43m) 8m ago 60m 6660k - 19.0.0-2879-g324baf8d 8717720dc5e5 a4f55c81b623 2024-04-12T21:46:41.384 INFO:teuthology.orchestra.run.smithi018.stdout:grafana.smithi018 smithi018 *:3000 running (46m) 8m ago 61m 73.8M - 9.4.12 2bacad6d85d8 b08e3bf176dd 2024-04-12T21:46:41.384 INFO:teuthology.orchestra.run.smithi018.stdout:mds.cephfs.smithi018.acqbsc smithi018 running (39m) 8m ago 54m 21.9M - 19.0.0-2879-g324baf8d 8717720dc5e5 021f99ab24ae 2024-04-12T21:46:41.384 INFO:teuthology.orchestra.run.smithi018.stdout:mds.cephfs.smithi018.szjluo smithi018 running (39m) 8m ago 54m 21.7M - 19.0.0-2879-g324baf8d 8717720dc5e5 0003117479d4 2024-04-12T21:46:41.384 INFO:teuthology.orchestra.run.smithi018.stdout:mds.cephfs.smithi186.jdllyn smithi186 running (23m) 8m ago 54m 16.5M - 19.0.0-2879-g324baf8d 8717720dc5e5 21b1f46eef23 2024-04-12T21:46:41.384 INFO:teuthology.orchestra.run.smithi018.stdout:mds.cephfs.smithi186.wjnafs smithi186 running (8m) 8m ago 54m 11.0M - 19.0.0-2879-g324baf8d 8717720dc5e5 3cf1ef4148e2 2024-04-12T21:46:41.384 INFO:teuthology.orchestra.run.smithi018.stdout:mgr.smithi018.xjzwga smithi018 *:8443,9283,8765 running (49m) 8m ago 63m 517M - 19.0.0-2879-g324baf8d 8717720dc5e5 56088305eab7 2024-04-12T21:46:41.384 INFO:teuthology.orchestra.run.smithi018.stdout:mgr.smithi186.kqcfxk smithi186 *:8443,9283,8765 running (48m) 8m ago 60m 460M - 19.0.0-2879-g324baf8d 8717720dc5e5 09cd766e5e0d 2024-04-12T21:46:41.384 INFO:teuthology.orchestra.run.smithi018.stdout:mon.smithi018 smithi018 running (45m) 8m ago 63m 88.2M 2048M 19.0.0-2879-g324baf8d 8717720dc5e5 df3e201dfed6 2024-04-12T21:46:41.384 INFO:teuthology.orchestra.run.smithi018.stdout:mon.smithi186 smithi186 running (44m) 8m ago 60m 53.6M 2048M 19.0.0-2879-g324baf8d 8717720dc5e5 c77498da9b2b 2024-04-12T21:46:41.384 INFO:teuthology.orchestra.run.smithi018.stdout:node-exporter.smithi018 smithi018 *:9100 running (48m) 8m ago 61m 8468k - 1.5.0 0da6a335fe13 326c3d9c6847 2024-04-12T21:46:41.385 INFO:teuthology.orchestra.run.smithi018.stdout:node-exporter.smithi186 smithi186 *:9100 running (48m) 8m ago 60m 8166k - 1.5.0 0da6a335fe13 ec2112baeecf 2024-04-12T21:46:41.385 INFO:teuthology.orchestra.run.smithi018.stdout:osd.0 smithi018 running (42m) 8m ago 59m 123M 1559M 19.0.0-2879-g324baf8d 8717720dc5e5 60b1273ca047 2024-04-12T21:46:41.385 INFO:teuthology.orchestra.run.smithi018.stdout:osd.1 smithi018 running (42m) 8m ago 58m 95.4M 1559M 19.0.0-2879-g324baf8d 8717720dc5e5 170ce1e5fb8d 2024-04-12T21:46:41.385 INFO:teuthology.orchestra.run.smithi018.stdout:osd.2 smithi018 running (41m) 8m ago 58m 94.6M 1559M 19.0.0-2879-g324baf8d 8717720dc5e5 c2cbc6aa2839 2024-04-12T21:46:41.385 INFO:teuthology.orchestra.run.smithi018.stdout:osd.3 smithi186 running (41m) 8m ago 57m 101M 2581M 19.0.0-2879-g324baf8d 8717720dc5e5 0138e8262e4b 2024-04-12T21:46:41.385 INFO:teuthology.orchestra.run.smithi018.stdout:osd.4 smithi186 running (40m) 8m ago 57m 91.5M 2581M 19.0.0-2879-g324baf8d 8717720dc5e5 b20cb63b25c9 2024-04-12T21:46:41.385 INFO:teuthology.orchestra.run.smithi018.stdout:osd.5 smithi186 running (39m) 8m ago 56m 92.2M 2581M 19.0.0-2879-g324baf8d 8717720dc5e5 2513bea2610e 2024-04-12T21:46:41.385 INFO:teuthology.orchestra.run.smithi018.stdout:prometheus.smithi018 smithi018 *:9095 running (47m) 8m ago 61m 69.6M - 2.43.0 a07b618ecd1d f58841127e21 2024-04-12T21:46:41.386 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:41.385+0000 7f1ba4000700 1 -- 172.21.15.18:0/1919542359 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f1b8406ec00 msgr2=0x7f1b840710b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:46:41.386 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:41.385+0000 7f1ba4000700 1 --2- 172.21.15.18:0/1919542359 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f1b8406ec00 0x7f1b840710b0 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f1b9000b3b0 tx=0x7f1b90009e30 comp rx=0 tx=0).stop 2024-04-12T21:46:41.386 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:41.385+0000 7f1ba4000700 1 -- 172.21.15.18:0/1919542359 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1b9c109dd0 msgr2=0x7f1b9c11a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:46:41.387 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:41.385+0000 7f1ba4000700 1 --2- 172.21.15.18:0/1919542359 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1b9c109dd0 0x7f1b9c11a480 secure :-1 s=READY pgs=1189 cs=0 l=1 rev1=1 crypto rx=0x7f1b88002a50 tx=0x7f1b88004600 comp rx=0 tx=0).stop 2024-04-12T21:46:41.387 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:41.386+0000 7f1ba4000700 1 -- 172.21.15.18:0/1919542359 shutdown_connections 2024-04-12T21:46:41.387 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:41.386+0000 7f1ba4000700 1 --2- 172.21.15.18:0/1919542359 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f1b8406ec00 0x7f1b840710b0 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:46:41.387 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:41.386+0000 7f1ba4000700 1 --2- 172.21.15.18:0/1919542359 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1b9c109430 0x7f1b9c119f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:46:41.387 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:41.386+0000 7f1ba4000700 1 --2- 172.21.15.18:0/1919542359 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1b9c109dd0 0x7f1b9c11a480 unknown :-1 s=CLOSED pgs=1189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:46:41.387 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:41.386+0000 7f1ba4000700 1 -- 172.21.15.18:0/1919542359 >> 172.21.15.18:0/1919542359 conn(0x7f1b9c072c00 msgr2=0x7f1b9c074c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:46:41.387 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:41.386+0000 7f1ba4000700 1 -- 172.21.15.18:0/1919542359 shutdown_connections 2024-04-12T21:46:41.387 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:41.386+0000 7f1ba4000700 1 -- 172.21.15.18:0/1919542359 wait complete. 2024-04-12T21:46:41.546 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:41.545+0000 7f7ecca00700 1 -- 172.21.15.18:0/3849670870 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7ec8101350 msgr2=0x7f7ec8101720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:46:41.546 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:41.545+0000 7f7ecca00700 1 --2- 172.21.15.18:0/3849670870 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7ec8101350 0x7f7ec8101720 secure :-1 s=READY pgs=1190 cs=0 l=1 rev1=1 crypto rx=0x7f7eb8004940 tx=0x7f7eb8009e30 comp rx=0 tx=0).stop 2024-04-12T21:46:41.546 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:41.545+0000 7f7ecca00700 1 -- 172.21.15.18:0/3849670870 shutdown_connections 2024-04-12T21:46:41.546 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:41.545+0000 7f7ecca00700 1 --2- 172.21.15.18:0/3849670870 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7ec8101cf0 0x7f7ec80fb3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:46:41.546 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:41.545+0000 7f7ecca00700 1 --2- 172.21.15.18:0/3849670870 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7ec8101350 0x7f7ec8101720 unknown :-1 s=CLOSED pgs=1190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:46:41.546 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:41.545+0000 7f7ecca00700 1 -- 172.21.15.18:0/3849670870 >> 172.21.15.18:0/3849670870 conn(0x7f7ec80721c0 msgr2=0x7f7ec80725c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:46:41.546 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:41.545+0000 7f7ecca00700 1 -- 172.21.15.18:0/3849670870 shutdown_connections 2024-04-12T21:46:41.546 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:41.545+0000 7f7ecca00700 1 -- 172.21.15.18:0/3849670870 wait complete. 2024-04-12T21:46:41.547 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:41.546+0000 7f7ecca00700 1 Processor -- start 2024-04-12T21:46:41.547 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:41.546+0000 7f7ecca00700 1 -- start start 2024-04-12T21:46:41.547 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:41.546+0000 7f7ecca00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7ec8101350 0x7f7ec8115bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:46:41.547 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:41.547+0000 7f7ecca00700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7ec8101cf0 0x7f7ec81160f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:46:41.547 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:41.547+0000 7f7ecca00700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ec8116740 con 0x7f7ec8101350 2024-04-12T21:46:41.548 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:41.547+0000 7f7ecca00700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ec810fb30 con 0x7f7ec8101cf0 2024-04-12T21:46:41.548 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:41.547+0000 7f7ec6800700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7ec8101350 0x7f7ec8115bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:46:41.548 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:41.547+0000 7f7ec6800700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7ec8101350 0x7f7ec8115bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:53404/0 (socket says 172.21.15.18:53404) 2024-04-12T21:46:41.548 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:41.547+0000 7f7ec6800700 1 -- 172.21.15.18:0/1679217578 learned_addr learned my addr 172.21.15.18:0/1679217578 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:46:41.548 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:41.547+0000 7f7ec6800700 1 -- 172.21.15.18:0/1679217578 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7ec8101cf0 msgr2=0x7f7ec81160f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:46:41.549 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:41.547+0000 7f7ec6800700 1 --2- 172.21.15.18:0/1679217578 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7ec8101cf0 0x7f7ec81160f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:46:41.549 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:41.547+0000 7f7ec6800700 1 -- 172.21.15.18:0/1679217578 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7eb8005040 con 0x7f7ec8101350 2024-04-12T21:46:41.549 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:41.548+0000 7f7ec6800700 1 --2- 172.21.15.18:0/1679217578 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7ec8101350 0x7f7ec8115bb0 secure :-1 s=READY pgs=1191 cs=0 l=1 rev1=1 crypto rx=0x7f7eb8009150 tx=0x7f7eb8009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:46:41.549 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:41.548+0000 7f7ebea00700 1 -- 172.21.15.18:0/1679217578 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7eb800e040 con 0x7f7ec8101350 2024-04-12T21:46:41.549 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:41.548+0000 7f7ebea00700 1 -- 172.21.15.18:0/1679217578 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f7eb8013420 con 0x7f7ec8101350 2024-04-12T21:46:41.549 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:41.548+0000 7f7ecca00700 1 -- 172.21.15.18:0/1679217578 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7ec810fdb0 con 0x7f7ec8101350 2024-04-12T21:46:41.549 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:41.548+0000 7f7ebea00700 1 -- 172.21.15.18:0/1679217578 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7eb801c630 con 0x7f7ec8101350 2024-04-12T21:46:41.549 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:41.548+0000 7f7ecca00700 1 -- 172.21.15.18:0/1679217578 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7ec8110370 con 0x7f7ec8101350 2024-04-12T21:46:41.549 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:41.549+0000 7f7ecca00700 1 -- 172.21.15.18:0/1679217578 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7ec8103840 con 0x7f7ec8101350 2024-04-12T21:46:41.551 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:41.550+0000 7f7ebea00700 1 -- 172.21.15.18:0/1679217578 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7f7eb8013590 con 0x7f7ec8101350 2024-04-12T21:46:41.552 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:41.551+0000 7f7ebea00700 1 --2- 172.21.15.18:0/1679217578 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f7eac06ec00 0x7f7eac0710b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:46:41.552 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:41.551+0000 7f7ebea00700 1 -- 172.21.15.18:0/1679217578 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 6757+0+0 (secure 0 0 0) 0x7f7eb808fce0 con 0x7f7ec8101350 2024-04-12T21:46:41.552 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:41.551+0000 7f7ec5e00700 1 --2- 172.21.15.18:0/1679217578 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f7eac06ec00 0x7f7eac0710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:46:41.552 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:41.552+0000 7f7ec5e00700 1 --2- 172.21.15.18:0/1679217578 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f7eac06ec00 0x7f7eac0710b0 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f7ec8110cf0 tx=0x7f7eb4009ce0 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:46:41.556 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:41.556+0000 7f7ebea00700 1 -- 172.21.15.18:0/1679217578 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f7eb805c490 con 0x7f7ec8101350 2024-04-12T21:46:41.870 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:41.869+0000 7f7ecca00700 1 -- 172.21.15.18:0/1679217578 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f7ec8111200 con 0x7f7ec8101350 2024-04-12T21:46:41.871 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:41.871+0000 7f7ebea00700 1 -- 172.21.15.18:0/1679217578 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+623 (secure 0 0 0) 0x7f7eb805c020 con 0x7f7ec8101350 2024-04-12T21:46:41.872 INFO:teuthology.orchestra.run.smithi018.stdout:{ 2024-04-12T21:46:41.872 INFO:teuthology.orchestra.run.smithi018.stdout: "mon": { 2024-04-12T21:46:41.873 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 19.0.0-2879-g324baf8d (324baf8ddb91dea46cac50cb535c306115791355) squid (dev)": 2 2024-04-12T21:46:41.873 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-12T21:46:41.873 INFO:teuthology.orchestra.run.smithi018.stdout: "mgr": { 2024-04-12T21:46:41.873 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 19.0.0-2879-g324baf8d (324baf8ddb91dea46cac50cb535c306115791355) squid (dev)": 2 2024-04-12T21:46:41.873 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-12T21:46:41.873 INFO:teuthology.orchestra.run.smithi018.stdout: "osd": { 2024-04-12T21:46:41.873 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 19.0.0-2879-g324baf8d (324baf8ddb91dea46cac50cb535c306115791355) squid (dev)": 6 2024-04-12T21:46:41.873 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-12T21:46:41.873 INFO:teuthology.orchestra.run.smithi018.stdout: "mds": { 2024-04-12T21:46:41.873 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 19.0.0-2879-g324baf8d (324baf8ddb91dea46cac50cb535c306115791355) squid (dev)": 4 2024-04-12T21:46:41.873 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-12T21:46:41.873 INFO:teuthology.orchestra.run.smithi018.stdout: "overall": { 2024-04-12T21:46:41.873 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 19.0.0-2879-g324baf8d (324baf8ddb91dea46cac50cb535c306115791355) squid (dev)": 14 2024-04-12T21:46:41.874 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-04-12T21:46:41.874 INFO:teuthology.orchestra.run.smithi018.stdout:} 2024-04-12T21:46:41.874 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:41.873+0000 7f7ecca00700 1 -- 172.21.15.18:0/1679217578 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f7eac06ec00 msgr2=0x7f7eac0710b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:46:41.874 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:41.873+0000 7f7ecca00700 1 --2- 172.21.15.18:0/1679217578 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f7eac06ec00 0x7f7eac0710b0 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f7ec8110cf0 tx=0x7f7eb4009ce0 comp rx=0 tx=0).stop 2024-04-12T21:46:41.875 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:41.873+0000 7f7ecca00700 1 -- 172.21.15.18:0/1679217578 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7ec8101350 msgr2=0x7f7ec8115bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:46:41.875 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:41.873+0000 7f7ecca00700 1 --2- 172.21.15.18:0/1679217578 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7ec8101350 0x7f7ec8115bb0 secure :-1 s=READY pgs=1191 cs=0 l=1 rev1=1 crypto rx=0x7f7eb8009150 tx=0x7f7eb8009e30 comp rx=0 tx=0).stop 2024-04-12T21:46:41.875 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:41.874+0000 7f7ecca00700 1 -- 172.21.15.18:0/1679217578 shutdown_connections 2024-04-12T21:46:41.875 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:41.874+0000 7f7ecca00700 1 --2- 172.21.15.18:0/1679217578 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f7eac06ec00 0x7f7eac0710b0 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:46:41.875 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:41.874+0000 7f7ecca00700 1 --2- 172.21.15.18:0/1679217578 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7ec8101350 0x7f7ec8115bb0 unknown :-1 s=CLOSED pgs=1191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:46:41.875 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:41.874+0000 7f7ecca00700 1 --2- 172.21.15.18:0/1679217578 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7ec8101cf0 0x7f7ec81160f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:46:41.875 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:41.874+0000 7f7ecca00700 1 -- 172.21.15.18:0/1679217578 >> 172.21.15.18:0/1679217578 conn(0x7f7ec80721c0 msgr2=0x7f7ec80fa4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:46:41.875 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:41.874+0000 7f7ecca00700 1 -- 172.21.15.18:0/1679217578 shutdown_connections 2024-04-12T21:46:41.875 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:41.874+0000 7f7ecca00700 1 -- 172.21.15.18:0/1679217578 wait complete. 2024-04-12T21:46:42.031 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:42.030+0000 7fb1d6000700 1 -- 172.21.15.18:0/3408911245 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb1d0109210 msgr2=0x7fb1d01095e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:46:42.031 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:42.030+0000 7fb1d6000700 1 --2- 172.21.15.18:0/3408911245 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb1d0109210 0x7fb1d01095e0 secure :-1 s=READY pgs=1192 cs=0 l=1 rev1=1 crypto rx=0x7fb1c4004940 tx=0x7fb1c4009e30 comp rx=0 tx=0).stop 2024-04-12T21:46:42.031 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:46:41 smithi018 ceph-mon[112590]: pgmap v1410: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:46:42.031 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:46:41 smithi018 ceph-mon[112590]: from='client.36440 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:46:42.031 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:46:41 smithi018 ceph-mon[112590]: from='client.36444 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:46:42.032 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:42.030+0000 7fb1d6000700 1 -- 172.21.15.18:0/3408911245 shutdown_connections 2024-04-12T21:46:42.032 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:42.030+0000 7fb1d6000700 1 --2- 172.21.15.18:0/3408911245 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb1d0109bb0 0x7fb1d0103220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:46:42.032 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:42.030+0000 7fb1d6000700 1 --2- 172.21.15.18:0/3408911245 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb1d0109210 0x7fb1d01095e0 unknown :-1 s=CLOSED pgs=1192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:46:42.032 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:42.030+0000 7fb1d6000700 1 -- 172.21.15.18:0/3408911245 >> 172.21.15.18:0/3408911245 conn(0x7fb1d0072a10 msgr2=0x7fb1d0074e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:46:42.032 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:42.030+0000 7fb1d6000700 1 -- 172.21.15.18:0/3408911245 shutdown_connections 2024-04-12T21:46:42.032 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:42.030+0000 7fb1d6000700 1 -- 172.21.15.18:0/3408911245 wait complete. 2024-04-12T21:46:42.032 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:42.031+0000 7fb1d6000700 1 Processor -- start 2024-04-12T21:46:42.032 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:42.031+0000 7fb1d6000700 1 -- start start 2024-04-12T21:46:42.033 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:42.032+0000 7fb1d6000700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb1d0109210 0x7fb1d0119f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:46:42.033 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:42.032+0000 7fb1d6000700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb1d0109bb0 0x7fb1d011a4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:46:42.033 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:42.032+0000 7fb1d6000700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1d011aaf0 con 0x7fb1d0109bb0 2024-04-12T21:46:42.034 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:42.032+0000 7fb1d6000700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1d0113ec0 con 0x7fb1d0109210 2024-04-12T21:46:42.034 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:42.032+0000 7fb1cfe00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb1d0109bb0 0x7fb1d011a4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:46:42.034 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:42.032+0000 7fb1cfe00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb1d0109bb0 0x7fb1d011a4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:53436/0 (socket says 172.21.15.18:53436) 2024-04-12T21:46:42.034 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:42.032+0000 7fb1cfe00700 1 -- 172.21.15.18:0/405276361 learned_addr learned my addr 172.21.15.18:0/405276361 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:46:42.034 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:42.033+0000 7fb1d4c00700 1 --2- 172.21.15.18:0/405276361 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb1d0109210 0x7fb1d0119f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:46:42.034 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:42.033+0000 7fb1cfe00700 1 -- 172.21.15.18:0/405276361 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb1d0109210 msgr2=0x7fb1d0119f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:46:42.034 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:42.033+0000 7fb1cfe00700 1 --2- 172.21.15.18:0/405276361 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb1d0109210 0x7fb1d0119f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:46:42.034 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:42.033+0000 7fb1cfe00700 1 -- 172.21.15.18:0/405276361 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1c4005040 con 0x7fb1d0109bb0 2024-04-12T21:46:42.034 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:42.033+0000 7fb1d4c00700 1 --2- 172.21.15.18:0/405276361 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb1d0109210 0x7fb1d0119f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-12T21:46:42.035 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:46:41 smithi186 ceph-mon[86379]: pgmap v1410: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:46:42.035 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:46:41 smithi186 ceph-mon[86379]: from='client.36440 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:46:42.035 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:46:41 smithi186 ceph-mon[86379]: from='client.36444 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:46:42.036 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:42.033+0000 7fb1cfe00700 1 --2- 172.21.15.18:0/405276361 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb1d0109bb0 0x7fb1d011a4a0 secure :-1 s=READY pgs=1193 cs=0 l=1 rev1=1 crypto rx=0x7fb1bc00c890 tx=0x7fb1bc0044c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:46:42.036 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:42.034+0000 7fb1ccc00700 1 -- 172.21.15.18:0/405276361 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb1bc00a7d0 con 0x7fb1d0109bb0 2024-04-12T21:46:42.036 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:42.034+0000 7fb1ccc00700 1 -- 172.21.15.18:0/405276361 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb1bc00ae00 con 0x7fb1d0109bb0 2024-04-12T21:46:42.036 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:42.034+0000 7fb1d6000700 1 -- 172.21.15.18:0/405276361 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb1d01141a0 con 0x7fb1d0109bb0 2024-04-12T21:46:42.036 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:42.034+0000 7fb1ccc00700 1 -- 172.21.15.18:0/405276361 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb1bc0113a0 con 0x7fb1d0109bb0 2024-04-12T21:46:42.036 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:42.034+0000 7fb1d6000700 1 -- 172.21.15.18:0/405276361 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb1d0114760 con 0x7fb1d0109bb0 2024-04-12T21:46:42.036 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:42.036+0000 7fb1d6000700 1 -- 172.21.15.18:0/405276361 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb1d0104960 con 0x7fb1d0109bb0 2024-04-12T21:46:42.038 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:42.037+0000 7fb1ccc00700 1 -- 172.21.15.18:0/405276361 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7fb1bc012400 con 0x7fb1d0109bb0 2024-04-12T21:46:42.039 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:42.039+0000 7fb1ccc00700 1 --2- 172.21.15.18:0/405276361 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fb1b806ec00 0x7fb1b80710b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:46:42.039 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:42.039+0000 7fb1ccc00700 1 -- 172.21.15.18:0/405276361 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 6757+0+0 (secure 0 0 0) 0x7fb1bc08ea40 con 0x7fb1d0109bb0 2024-04-12T21:46:42.040 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:42.039+0000 7fb1d4c00700 1 --2- 172.21.15.18:0/405276361 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fb1b806ec00 0x7fb1b80710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:46:42.040 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:42.040+0000 7fb1d4c00700 1 --2- 172.21.15.18:0/405276361 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fb1b806ec00 0x7fb1b80710b0 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7fb1c4000c00 tx=0x7fb1c4009e30 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:46:42.043 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:42.043+0000 7fb1ccc00700 1 -- 172.21.15.18:0/405276361 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7fb1bc05b270 con 0x7fb1d0109bb0 2024-04-12T21:46:42.327 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:42.326+0000 7fb1d6000700 1 -- 172.21.15.18:0/405276361 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fb1d004bed0 con 0x7fb1d0109bb0 2024-04-12T21:46:42.327 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:42.327+0000 7fb1ccc00700 1 -- 172.21.15.18:0/405276361 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 33 v33) v1 ==== 76+0+1876 (secure 0 0 0) 0x7fb1bc05ae00 con 0x7fb1d0109bb0 2024-04-12T21:46:42.329 INFO:teuthology.orchestra.run.smithi018.stdout:e33 2024-04-12T21:46:42.329 INFO:teuthology.orchestra.run.smithi018.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-12T21:46:42.329 INFO:teuthology.orchestra.run.smithi018.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-12T21:46:42.329 INFO:teuthology.orchestra.run.smithi018.stdout:legacy client fscid: 1 2024-04-12T21:46:42.329 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-12T21:46:42.329 INFO:teuthology.orchestra.run.smithi018.stdout:Filesystem 'cephfs' (1) 2024-04-12T21:46:42.329 INFO:teuthology.orchestra.run.smithi018.stdout:fs_name cephfs 2024-04-12T21:46:42.329 INFO:teuthology.orchestra.run.smithi018.stdout:epoch 24 2024-04-12T21:46:42.329 INFO:teuthology.orchestra.run.smithi018.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-04-12T21:46:42.329 INFO:teuthology.orchestra.run.smithi018.stdout:created 2024-04-12T20:51:57.362850+0000 2024-04-12T21:46:42.329 INFO:teuthology.orchestra.run.smithi018.stdout:modified 2024-04-12T21:06:59.717190+0000 2024-04-12T21:46:42.329 INFO:teuthology.orchestra.run.smithi018.stdout:tableserver 0 2024-04-12T21:46:42.329 INFO:teuthology.orchestra.run.smithi018.stdout:root 0 2024-04-12T21:46:42.330 INFO:teuthology.orchestra.run.smithi018.stdout:session_timeout 60 2024-04-12T21:46:42.330 INFO:teuthology.orchestra.run.smithi018.stdout:session_autoclose 300 2024-04-12T21:46:42.330 INFO:teuthology.orchestra.run.smithi018.stdout:max_file_size 1099511627776 2024-04-12T21:46:42.330 INFO:teuthology.orchestra.run.smithi018.stdout:max_xattr_size 65536 2024-04-12T21:46:42.330 INFO:teuthology.orchestra.run.smithi018.stdout:required_client_features {} 2024-04-12T21:46:42.330 INFO:teuthology.orchestra.run.smithi018.stdout:last_failure 0 2024-04-12T21:46:42.330 INFO:teuthology.orchestra.run.smithi018.stdout:last_failure_osd_epoch 100 2024-04-12T21:46:42.330 INFO:teuthology.orchestra.run.smithi018.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-12T21:46:42.330 INFO:teuthology.orchestra.run.smithi018.stdout:max_mds 2 2024-04-12T21:46:42.330 INFO:teuthology.orchestra.run.smithi018.stdout:in 0,1 2024-04-12T21:46:42.330 INFO:teuthology.orchestra.run.smithi018.stdout:up {} 2024-04-12T21:46:42.330 INFO:teuthology.orchestra.run.smithi018.stdout:failed 0,1 2024-04-12T21:46:42.330 INFO:teuthology.orchestra.run.smithi018.stdout:damaged 2024-04-12T21:46:42.331 INFO:teuthology.orchestra.run.smithi018.stdout:stopped 2024-04-12T21:46:42.331 INFO:teuthology.orchestra.run.smithi018.stdout:data_pools [3] 2024-04-12T21:46:42.331 INFO:teuthology.orchestra.run.smithi018.stdout:metadata_pool 2 2024-04-12T21:46:42.331 INFO:teuthology.orchestra.run.smithi018.stdout:inline_data disabled 2024-04-12T21:46:42.331 INFO:teuthology.orchestra.run.smithi018.stdout:balancer 2024-04-12T21:46:42.331 INFO:teuthology.orchestra.run.smithi018.stdout:bal_rank_mask -1 2024-04-12T21:46:42.331 INFO:teuthology.orchestra.run.smithi018.stdout:standby_count_wanted 1 2024-04-12T21:46:42.331 INFO:teuthology.orchestra.run.smithi018.stdout:qdb_cluster leader: 0 members: 2024-04-12T21:46:42.331 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-12T21:46:42.331 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-12T21:46:42.331 INFO:teuthology.orchestra.run.smithi018.stdout:Standby daemons: 2024-04-12T21:46:42.331 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-12T21:46:42.331 INFO:teuthology.orchestra.run.smithi018.stdout:[mds.cephfs.smithi018.acqbsc{-1:34436} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.18:6828/755179764,v1:172.21.15.18:6829/755179764] compat {c=[1],r=[1],i=[1fff]}] 2024-04-12T21:46:42.331 INFO:teuthology.orchestra.run.smithi018.stdout:[mds.cephfs.smithi018.szjluo{-1:34468} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.18:6826/2407992653,v1:172.21.15.18:6827/2407992653] compat {c=[1],r=[1],i=[1fff]}] 2024-04-12T21:46:42.331 INFO:teuthology.orchestra.run.smithi018.stdout:[mds.cephfs.smithi186.jdllyn{-1:44595} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.186:6824/3523444202,v1:172.21.15.186:6825/3523444202] compat {c=[1],r=[1],i=[1fff]}] 2024-04-12T21:46:42.332 INFO:teuthology.orchestra.run.smithi018.stdout:[mds.cephfs.smithi186.wjnafs{-1:44875} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.186:6826/3122443766,v1:172.21.15.186:6827/3122443766] compat {c=[1],r=[1],i=[1fff]}] 2024-04-12T21:46:42.332 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:42.331+0000 7fb1d6000700 1 -- 172.21.15.18:0/405276361 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fb1b806ec00 msgr2=0x7fb1b80710b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:46:42.332 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:42.331+0000 7fb1d6000700 1 --2- 172.21.15.18:0/405276361 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fb1b806ec00 0x7fb1b80710b0 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7fb1c4000c00 tx=0x7fb1c4009e30 comp rx=0 tx=0).stop 2024-04-12T21:46:42.332 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:42.331+0000 7fb1d6000700 1 -- 172.21.15.18:0/405276361 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb1d0109bb0 msgr2=0x7fb1d011a4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:46:42.332 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:42.331+0000 7fb1d6000700 1 --2- 172.21.15.18:0/405276361 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb1d0109bb0 0x7fb1d011a4a0 secure :-1 s=READY pgs=1193 cs=0 l=1 rev1=1 crypto rx=0x7fb1bc00c890 tx=0x7fb1bc0044c0 comp rx=0 tx=0).stop 2024-04-12T21:46:42.333 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:42.331+0000 7fb1d6000700 1 -- 172.21.15.18:0/405276361 shutdown_connections 2024-04-12T21:46:42.333 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:42.331+0000 7fb1d6000700 1 --2- 172.21.15.18:0/405276361 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fb1b806ec00 0x7fb1b80710b0 unknown :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:46:42.333 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:42.331+0000 7fb1d6000700 1 --2- 172.21.15.18:0/405276361 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb1d0109210 0x7fb1d0119f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:46:42.333 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:42.331+0000 7fb1d6000700 1 --2- 172.21.15.18:0/405276361 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb1d0109bb0 0x7fb1d011a4a0 unknown :-1 s=CLOSED pgs=1193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:46:42.333 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:42.331+0000 7fb1d6000700 1 -- 172.21.15.18:0/405276361 >> 172.21.15.18:0/405276361 conn(0x7fb1d0072a10 msgr2=0x7fb1d0074990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:46:42.333 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:42.332+0000 7fb1d6000700 1 -- 172.21.15.18:0/405276361 shutdown_connections 2024-04-12T21:46:42.333 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:42.332+0000 7fb1d6000700 1 -- 172.21.15.18:0/405276361 wait complete. 2024-04-12T21:46:42.334 INFO:teuthology.orchestra.run.smithi018.stderr:dumped fsmap epoch 33 2024-04-12T21:46:42.496 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:42.495+0000 7f58f8200700 1 -- 172.21.15.18:0/1670852870 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f58f0109430 msgr2=0x7f58f0109800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:46:42.496 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:42.495+0000 7f58f8200700 1 --2- 172.21.15.18:0/1670852870 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f58f0109430 0x7f58f0109800 secure :-1 s=READY pgs=1194 cs=0 l=1 rev1=1 crypto rx=0x7f58e4004940 tx=0x7f58e4009e30 comp rx=0 tx=0).stop 2024-04-12T21:46:42.496 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:42.495+0000 7f58f8200700 1 -- 172.21.15.18:0/1670852870 shutdown_connections 2024-04-12T21:46:42.496 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:42.495+0000 7f58f8200700 1 --2- 172.21.15.18:0/1670852870 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f58f0109dd0 0x7f58f0103440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:46:42.496 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:42.495+0000 7f58f8200700 1 --2- 172.21.15.18:0/1670852870 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f58f0109430 0x7f58f0109800 unknown :-1 s=CLOSED pgs=1194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:46:42.497 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:42.495+0000 7f58f8200700 1 -- 172.21.15.18:0/1670852870 >> 172.21.15.18:0/1670852870 conn(0x7f58f0072c00 msgr2=0x7f58f0075010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:46:42.497 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:42.495+0000 7f58f8200700 1 -- 172.21.15.18:0/1670852870 shutdown_connections 2024-04-12T21:46:42.497 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:42.495+0000 7f58f8200700 1 -- 172.21.15.18:0/1670852870 wait complete. 2024-04-12T21:46:42.498 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:42.496+0000 7f58f8200700 1 Processor -- start 2024-04-12T21:46:42.499 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:42.496+0000 7f58f8200700 1 -- start start 2024-04-12T21:46:42.499 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:42.497+0000 7f58f8200700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f58f0109430 0x7f58f0119f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:46:42.499 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:42.497+0000 7f58f8200700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f58f0109dd0 0x7f58f011a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:46:42.499 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:42.497+0000 7f58f8200700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58f011aad0 con 0x7f58f0109430 2024-04-12T21:46:42.499 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:42.497+0000 7f58f8200700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58f0114100 con 0x7f58f0109dd0 2024-04-12T21:46:42.499 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:42.497+0000 7f58f5800700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f58f0109430 0x7f58f0119f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:46:42.499 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:42.497+0000 7f58f5800700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f58f0109430 0x7f58f0119f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:53450/0 (socket says 172.21.15.18:53450) 2024-04-12T21:46:42.499 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:42.497+0000 7f58f5800700 1 -- 172.21.15.18:0/1429440976 learned_addr learned my addr 172.21.15.18:0/1429440976 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:46:42.499 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:42.497+0000 7f58f4e00700 1 --2- 172.21.15.18:0/1429440976 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f58f0109dd0 0x7f58f011a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:46:42.499 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:42.497+0000 7f58f5800700 1 -- 172.21.15.18:0/1429440976 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f58f0109dd0 msgr2=0x7f58f011a480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:46:42.499 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:42.497+0000 7f58f5800700 1 --2- 172.21.15.18:0/1429440976 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f58f0109dd0 0x7f58f011a480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:46:42.499 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:42.497+0000 7f58f5800700 1 -- 172.21.15.18:0/1429440976 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58e4005040 con 0x7f58f0109430 2024-04-12T21:46:42.499 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:42.497+0000 7f58f4e00700 1 --2- 172.21.15.18:0/1429440976 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f58f0109dd0 0x7f58f011a480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-12T21:46:42.499 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:42.498+0000 7f58f5800700 1 --2- 172.21.15.18:0/1429440976 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f58f0109430 0x7f58f0119f40 secure :-1 s=READY pgs=1195 cs=0 l=1 rev1=1 crypto rx=0x7f58e4009180 tx=0x7f58e4009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:46:42.500 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:42.498+0000 7f58ed600700 1 -- 172.21.15.18:0/1429440976 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f58e400e040 con 0x7f58f0109430 2024-04-12T21:46:42.500 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:42.498+0000 7f58ed600700 1 -- 172.21.15.18:0/1429440976 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f58e400bbc0 con 0x7f58f0109430 2024-04-12T21:46:42.500 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:42.498+0000 7f58ed600700 1 -- 172.21.15.18:0/1429440976 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f58e401c630 con 0x7f58f0109430 2024-04-12T21:46:42.500 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:42.498+0000 7f58f8200700 1 -- 172.21.15.18:0/1429440976 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f58f0114380 con 0x7f58f0109430 2024-04-12T21:46:42.500 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:42.498+0000 7f58f8200700 1 -- 172.21.15.18:0/1429440976 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f58f01148f0 con 0x7f58f0109430 2024-04-12T21:46:42.500 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:42.499+0000 7f58f8200700 1 -- 172.21.15.18:0/1429440976 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f58f0104b80 con 0x7f58f0109430 2024-04-12T21:46:42.501 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:42.501+0000 7f58ed600700 1 -- 172.21.15.18:0/1429440976 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7f58e400bd30 con 0x7f58f0109430 2024-04-12T21:46:42.502 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:42.502+0000 7f58ed600700 1 --2- 172.21.15.18:0/1429440976 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f58d806ec00 0x7f58d80710b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:46:42.502 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:42.502+0000 7f58ed600700 1 -- 172.21.15.18:0/1429440976 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 6757+0+0 (secure 0 0 0) 0x7f58e408fe30 con 0x7f58f0109430 2024-04-12T21:46:42.502 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:42.502+0000 7f58f4e00700 1 --2- 172.21.15.18:0/1429440976 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f58d806ec00 0x7f58d80710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:46:42.503 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:42.503+0000 7f58f4e00700 1 --2- 172.21.15.18:0/1429440976 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f58d806ec00 0x7f58d80710b0 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f58dc009fd0 tx=0x7f58dc00b040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:46:42.507 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:42.507+0000 7f58ed600700 1 -- 172.21.15.18:0/1429440976 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f58e405c5e0 con 0x7f58f0109430 2024-04-12T21:46:42.763 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:42.762+0000 7f58f8200700 1 -- 172.21.15.18:0/1429440976 --> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f58f0115600 con 0x7f58d806ec00 2024-04-12T21:46:42.764 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:42.764+0000 7f58ed600700 1 -- 172.21.15.18:0/1429440976 <== mgr.34102 v2:172.21.15.18:6800/3812891919 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f58f0115600 con 0x7f58d806ec00 2024-04-12T21:46:42.765 INFO:teuthology.orchestra.run.smithi018.stdout:{ 2024-04-12T21:46:42.765 INFO:teuthology.orchestra.run.smithi018.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:f4452def3be4d350f4a8e5feaf7ebb1511be9298fe68478ba7a2d6fbd7d69a63", 2024-04-12T21:46:42.765 INFO:teuthology.orchestra.run.smithi018.stdout: "in_progress": true, 2024-04-12T21:46:42.765 INFO:teuthology.orchestra.run.smithi018.stdout: "which": "Upgrading all daemon types on all hosts", 2024-04-12T21:46:42.765 INFO:teuthology.orchestra.run.smithi018.stdout: "services_complete": [ 2024-04-12T21:46:42.766 INFO:teuthology.orchestra.run.smithi018.stdout: "mgr", 2024-04-12T21:46:42.766 INFO:teuthology.orchestra.run.smithi018.stdout: "mds", 2024-04-12T21:46:42.766 INFO:teuthology.orchestra.run.smithi018.stdout: "crash", 2024-04-12T21:46:42.766 INFO:teuthology.orchestra.run.smithi018.stdout: "osd", 2024-04-12T21:46:42.766 INFO:teuthology.orchestra.run.smithi018.stdout: "mon" 2024-04-12T21:46:42.766 INFO:teuthology.orchestra.run.smithi018.stdout: ], 2024-04-12T21:46:42.766 INFO:teuthology.orchestra.run.smithi018.stdout: "progress": "16/21 daemons upgraded", 2024-04-12T21:46:42.766 INFO:teuthology.orchestra.run.smithi018.stdout: "message": "Currently upgrading mds daemons", 2024-04-12T21:46:42.766 INFO:teuthology.orchestra.run.smithi018.stdout: "is_paused": false 2024-04-12T21:46:42.766 INFO:teuthology.orchestra.run.smithi018.stdout:} 2024-04-12T21:46:42.768 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:42.767+0000 7f58f8200700 1 -- 172.21.15.18:0/1429440976 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f58d806ec00 msgr2=0x7f58d80710b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:46:42.768 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:42.767+0000 7f58f8200700 1 --2- 172.21.15.18:0/1429440976 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f58d806ec00 0x7f58d80710b0 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f58dc009fd0 tx=0x7f58dc00b040 comp rx=0 tx=0).stop 2024-04-12T21:46:42.768 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:42.767+0000 7f58f8200700 1 -- 172.21.15.18:0/1429440976 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f58f0109430 msgr2=0x7f58f0119f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:46:42.768 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:42.767+0000 7f58f8200700 1 --2- 172.21.15.18:0/1429440976 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f58f0109430 0x7f58f0119f40 secure :-1 s=READY pgs=1195 cs=0 l=1 rev1=1 crypto rx=0x7f58e4009180 tx=0x7f58e4009e30 comp rx=0 tx=0).stop 2024-04-12T21:46:42.768 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:42.767+0000 7f58f8200700 1 -- 172.21.15.18:0/1429440976 shutdown_connections 2024-04-12T21:46:42.768 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:42.767+0000 7f58f8200700 1 --2- 172.21.15.18:0/1429440976 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f58d806ec00 0x7f58d80710b0 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:46:42.768 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:42.767+0000 7f58f8200700 1 --2- 172.21.15.18:0/1429440976 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f58f0109430 0x7f58f0119f40 unknown :-1 s=CLOSED pgs=1195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:46:42.768 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:42.767+0000 7f58f8200700 1 --2- 172.21.15.18:0/1429440976 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f58f0109dd0 0x7f58f011a480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:46:42.768 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:42.767+0000 7f58f8200700 1 -- 172.21.15.18:0/1429440976 >> 172.21.15.18:0/1429440976 conn(0x7f58f0072c00 msgr2=0x7f58f0074c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:46:42.768 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:42.767+0000 7f58f8200700 1 -- 172.21.15.18:0/1429440976 shutdown_connections 2024-04-12T21:46:42.768 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:42.767+0000 7f58f8200700 1 -- 172.21.15.18:0/1429440976 wait complete. 2024-04-12T21:46:42.917 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:42.916+0000 7f6968a00700 1 -- 172.21.15.18:0/3135566206 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f69640fc430 msgr2=0x7f6964100a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:46:42.917 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:42.916+0000 7f6968a00700 1 --2- 172.21.15.18:0/3135566206 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f69640fc430 0x7f6964100a50 secure :-1 s=READY pgs=1196 cs=0 l=1 rev1=1 crypto rx=0x7f6950004940 tx=0x7f6950009e30 comp rx=0 tx=0).stop 2024-04-12T21:46:42.917 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:42.916+0000 7f6968a00700 1 -- 172.21.15.18:0/3135566206 shutdown_connections 2024-04-12T21:46:42.917 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:42.916+0000 7f6968a00700 1 --2- 172.21.15.18:0/3135566206 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f69640fc430 0x7f6964100a50 unknown :-1 s=CLOSED pgs=1196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:46:42.917 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:42.916+0000 7f6968a00700 1 --2- 172.21.15.18:0/3135566206 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f69640fba90 0x7f69640fbe60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:46:42.917 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:42.916+0000 7f6968a00700 1 -- 172.21.15.18:0/3135566206 >> 172.21.15.18:0/3135566206 conn(0x7f69640f7710 msgr2=0x7f69640f9b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:46:42.917 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:42.916+0000 7f6968a00700 1 -- 172.21.15.18:0/3135566206 shutdown_connections 2024-04-12T21:46:42.918 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:42.916+0000 7f6968a00700 1 -- 172.21.15.18:0/3135566206 wait complete. 2024-04-12T21:46:42.919 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:42.917+0000 7f6968a00700 1 Processor -- start 2024-04-12T21:46:42.919 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:42.917+0000 7f6968a00700 1 -- start start 2024-04-12T21:46:42.919 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:42.918+0000 7f6968a00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f69640fba90 0x7f696411a030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:46:42.919 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:42.918+0000 7f6968a00700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f69640fc430 0x7f696411a570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:46:42.919 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:42.918+0000 7f6968a00700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f696411abc0 con 0x7f69640fba90 2024-04-12T21:46:42.920 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:42.918+0000 7f6968a00700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6964113f90 con 0x7f69640fc430 2024-04-12T21:46:42.920 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:42.918+0000 7f6963400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f69640fba90 0x7f696411a030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:46:42.920 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:42.918+0000 7f6963400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f69640fba90 0x7f696411a030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:53464/0 (socket says 172.21.15.18:53464) 2024-04-12T21:46:42.920 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:42.918+0000 7f6963400700 1 -- 172.21.15.18:0/2322872851 learned_addr learned my addr 172.21.15.18:0/2322872851 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:46:42.920 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:42.919+0000 7f6962a00700 1 --2- 172.21.15.18:0/2322872851 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f69640fc430 0x7f696411a570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:46:42.920 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:42.919+0000 7f6963400700 1 -- 172.21.15.18:0/2322872851 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f69640fc430 msgr2=0x7f696411a570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:46:42.920 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:42.919+0000 7f6963400700 1 --2- 172.21.15.18:0/2322872851 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f69640fc430 0x7f696411a570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:46:42.920 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:42.919+0000 7f6963400700 1 -- 172.21.15.18:0/2322872851 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6950005040 con 0x7f69640fba90 2024-04-12T21:46:42.920 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:42.919+0000 7f6962a00700 1 --2- 172.21.15.18:0/2322872851 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f69640fc430 0x7f696411a570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-12T21:46:42.920 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:42.919+0000 7f6963400700 1 --2- 172.21.15.18:0/2322872851 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f69640fba90 0x7f696411a030 secure :-1 s=READY pgs=1197 cs=0 l=1 rev1=1 crypto rx=0x7f695400c8c0 tx=0x7f69540044c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:46:42.921 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:42.919+0000 7f695b400700 1 -- 172.21.15.18:0/2322872851 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f695400a980 con 0x7f69640fba90 2024-04-12T21:46:42.921 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:42.919+0000 7f695b400700 1 -- 172.21.15.18:0/2322872851 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f695400ce60 con 0x7f69640fba90 2024-04-12T21:46:42.921 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:42.919+0000 7f695b400700 1 -- 172.21.15.18:0/2322872851 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f69540115e0 con 0x7f69640fba90 2024-04-12T21:46:42.921 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:42.919+0000 7f6968a00700 1 -- 172.21.15.18:0/2322872851 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6964114270 con 0x7f69640fba90 2024-04-12T21:46:42.921 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:42.919+0000 7f6968a00700 1 -- 172.21.15.18:0/2322872851 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f69641147b0 con 0x7f69640fba90 2024-04-12T21:46:42.922 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:42.920+0000 7f6968a00700 1 -- 172.21.15.18:0/2322872851 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f69640669b0 con 0x7f69640fba90 2024-04-12T21:46:42.922 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:42.922+0000 7f695b400700 1 -- 172.21.15.18:0/2322872851 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7f6954012400 con 0x7f69640fba90 2024-04-12T21:46:42.923 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:42.923+0000 7f695b400700 1 --2- 172.21.15.18:0/2322872851 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f694806ec00 0x7f69480710b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:46:42.923 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:42.923+0000 7f695b400700 1 -- 172.21.15.18:0/2322872851 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 6757+0+0 (secure 0 0 0) 0x7f695408ea90 con 0x7f69640fba90 2024-04-12T21:46:42.923 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:42.923+0000 7f6962a00700 1 --2- 172.21.15.18:0/2322872851 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f694806ec00 0x7f69480710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:46:42.924 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:42.924+0000 7f6962a00700 1 --2- 172.21.15.18:0/2322872851 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f694806ec00 0x7f69480710b0 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f695000b3b0 tx=0x7f6950009e30 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:46:42.928 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:42.928+0000 7f695b400700 1 -- 172.21.15.18:0/2322872851 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f695405b2e0 con 0x7f69640fba90 2024-04-12T21:46:43.034 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:46:42 smithi186 ceph-mon[86379]: from='client.36448 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:46:43.034 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:46:42 smithi186 ceph-mon[86379]: from='client.? 172.21.15.18:0/1679217578' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T21:46:43.034 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:46:42 smithi186 ceph-mon[86379]: from='client.? 172.21.15.18:0/405276361' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-12T21:46:43.105 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:46:42 smithi018 ceph-mon[112590]: from='client.36448 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:46:43.105 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:46:42 smithi018 ceph-mon[112590]: from='client.? 172.21.15.18:0/1679217578' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T21:46:43.105 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:46:42 smithi018 ceph-mon[112590]: from='client.? 172.21.15.18:0/405276361' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-12T21:46:43.241 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:43.240+0000 7f6968a00700 1 -- 172.21.15.18:0/2322872851 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f69640fb330 con 0x7f69640fba90 2024-04-12T21:46:43.241 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:43.241+0000 7f695b400700 1 -- 172.21.15.18:0/2322872851 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+4303 (secure 0 0 0) 0x7f69540126b0 con 0x7f69640fba90 2024-04-12T21:46:43.241 INFO:teuthology.orchestra.run.smithi018.stdout:HEALTH_ERR failed to probe daemons or devices; 1 filesystem is degraded; 1 filesystem is offline 2024-04-12T21:46:43.242 INFO:teuthology.orchestra.run.smithi018.stdout:[WRN] CEPHADM_REFRESH_FAILED: failed to probe daemons or devices 2024-04-12T21:46:43.242 INFO:teuthology.orchestra.run.smithi018.stdout: host smithi186 `cephadm ceph-volume` failed: cephadm exited with an error code: 1, stderr: Inferring config /var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/mon.smithi186/config 2024-04-12T21:46:43.242 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/podman: timeout after 895 seconds 2024-04-12T21:46:43.242 INFO:teuthology.orchestra.run.smithi018.stdout:Non-zero exit code 124 from /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:f4452def3be4d350f4a8e5feaf7ebb1511be9298fe68478ba7a2d6fbd7d69a63 -e NODE_NAME=smithi186 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968:/var/run/ceph:z -v /var/log/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968:/var/log/ceph:z -v /var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmptgom53f9:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:f4452def3be4d350f4a8e5feaf7ebb1511be9298fe68478ba7a2d6fbd7d69a63 inventory --format=json-pretty --filter-for-batch 2024-04-12T21:46:43.242 INFO:teuthology.orchestra.run.smithi018.stdout:Traceback (most recent call last): 2024-04-12T21:46:43.242 INFO:teuthology.orchestra.run.smithi018.stdout: File "/usr/lib64/python3.9/runpy.py", line 197, in _run_module_as_main 2024-04-12T21:46:43.242 INFO:teuthology.orchestra.run.smithi018.stdout: return _run_code(code, main_globals, None, 2024-04-12T21:46:43.242 INFO:teuthology.orchestra.run.smithi018.stdout: File "/usr/lib64/python3.9/runpy.py", line 87, in _run_code 2024-04-12T21:46:43.242 INFO:teuthology.orchestra.run.smithi018.stdout: exec(code, run_globals) 2024-04-12T21:46:43.242 INFO:teuthology.orchestra.run.smithi018.stdout: File "/var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/cephadm.e44f3ed5e34e4ef52eedef233d1190bff004470d8e8005127b5312bff357fa7f/__main__.py", line 5618, in 2024-04-12T21:46:43.242 INFO:teuthology.orchestra.run.smithi018.stdout: File "/var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/cephadm.e44f3ed5e34e4ef52eedef233d1190bff004470d8e8005127b5312bff357fa7f/__main__.py", line 5606, in main 2024-04-12T21:46:43.243 INFO:teuthology.orchestra.run.smithi018.stdout: File "/var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/cephadm.e44f3ed5e34e4ef52eedef233d1190bff004470d8e8005127b5312bff357fa7f/__main__.py", line 408, in _infer_config 2024-04-12T21:46:43.243 INFO:teuthology.orchestra.run.smithi018.stdout: File "/var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/cephadm.e44f3ed5e34e4ef52eedef233d1190bff004470d8e8005127b5312bff357fa7f/__main__.py", line 323, in _infer_fsid 2024-04-12T21:46:43.243 INFO:teuthology.orchestra.run.smithi018.stdout: File "/var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/cephadm.e44f3ed5e34e4ef52eedef233d1190bff004470d8e8005127b5312bff357fa7f/__main__.py", line 436, in _infer_image 2024-04-12T21:46:43.243 INFO:teuthology.orchestra.run.smithi018.stdout: File "/var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/cephadm.e44f3ed5e34e4ef52eedef233d1190bff004470d8e8005127b5312bff357fa7f/__main__.py", line 310, in _validate_fsid 2024-04-12T21:46:43.243 INFO:teuthology.orchestra.run.smithi018.stdout: File "/var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/cephadm.e44f3ed5e34e4ef52eedef233d1190bff004470d8e8005127b5312bff357fa7f/__main__.py", line 3359, in command_ceph_volume 2024-04-12T21:46:43.243 INFO:teuthology.orchestra.run.smithi018.stdout: File "/var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/cephadm.e44f3ed5e34e4ef52eedef233d1190bff004470d8e8005127b5312bff357fa7f/cephadmlib/call_wrappers.py", line 310, in call_throws 2024-04-12T21:46:43.243 INFO:teuthology.orchestra.run.smithi018.stdout:RuntimeError: Failed command: /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:f4452def3be4d350f4a8e5feaf7ebb1511be9298fe68478ba7a2d6fbd7d69a63 -e NODE_NAME=smithi186 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968:/var/run/ceph:z -v /var/log/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968:/var/log/ceph:z -v /var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmptgom53f9:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:f4452def3be4d350f4a8e5feaf7ebb1511be9298fe68478ba7a2d6fbd7d69a63 inventory --format=json-pretty --filter-for-batch 2024-04-12T21:46:43.243 INFO:teuthology.orchestra.run.smithi018.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-04-12T21:46:43.243 INFO:teuthology.orchestra.run.smithi018.stdout: fs cephfs is degraded 2024-04-12T21:46:43.243 INFO:teuthology.orchestra.run.smithi018.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-04-12T21:46:43.243 INFO:teuthology.orchestra.run.smithi018.stdout: fs cephfs is offline because no MDS is active for it. 2024-04-12T21:46:43.244 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:43.243+0000 7f6968a00700 1 -- 172.21.15.18:0/2322872851 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f694806ec00 msgr2=0x7f69480710b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:46:43.244 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:43.243+0000 7f6968a00700 1 --2- 172.21.15.18:0/2322872851 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f694806ec00 0x7f69480710b0 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f695000b3b0 tx=0x7f6950009e30 comp rx=0 tx=0).stop 2024-04-12T21:46:43.244 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:43.243+0000 7f6968a00700 1 -- 172.21.15.18:0/2322872851 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f69640fba90 msgr2=0x7f696411a030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:46:43.244 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:43.243+0000 7f6968a00700 1 --2- 172.21.15.18:0/2322872851 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f69640fba90 0x7f696411a030 secure :-1 s=READY pgs=1197 cs=0 l=1 rev1=1 crypto rx=0x7f695400c8c0 tx=0x7f69540044c0 comp rx=0 tx=0).stop 2024-04-12T21:46:43.245 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:43.244+0000 7f6968a00700 1 -- 172.21.15.18:0/2322872851 shutdown_connections 2024-04-12T21:46:43.245 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:43.244+0000 7f6968a00700 1 --2- 172.21.15.18:0/2322872851 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f694806ec00 0x7f69480710b0 unknown :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:46:43.245 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:43.244+0000 7f6968a00700 1 --2- 172.21.15.18:0/2322872851 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f69640fba90 0x7f696411a030 unknown :-1 s=CLOSED pgs=1197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:46:43.245 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:43.244+0000 7f6968a00700 1 --2- 172.21.15.18:0/2322872851 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f69640fc430 0x7f696411a570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:46:43.245 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:43.244+0000 7f6968a00700 1 -- 172.21.15.18:0/2322872851 >> 172.21.15.18:0/2322872851 conn(0x7f69640f7710 msgr2=0x7f696410a830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:46:43.245 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:43.244+0000 7f6968a00700 1 -- 172.21.15.18:0/2322872851 shutdown_connections 2024-04-12T21:46:43.246 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:46:43.244+0000 7f6968a00700 1 -- 172.21.15.18:0/2322872851 wait complete. 2024-04-12T21:46:44.034 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:46:43 smithi186 ceph-mon[86379]: pgmap v1411: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:46:44.034 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:46:43 smithi186 ceph-mon[86379]: from='client.36460 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:46:44.034 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:46:43 smithi186 ceph-mon[86379]: from='client.? 172.21.15.18:0/2322872851' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-12T21:46:44.106 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:46:43 smithi018 ceph-mon[112590]: pgmap v1411: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:46:44.106 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:46:43 smithi018 ceph-mon[112590]: from='client.36460 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:46:44.106 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:46:43 smithi018 ceph-mon[112590]: from='client.? 172.21.15.18:0/2322872851' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-12T21:46:46.034 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:46:45 smithi186 ceph-mon[86379]: pgmap v1412: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:46:46.105 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:46:45 smithi018 ceph-mon[112590]: pgmap v1412: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:46:48.034 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:46:47 smithi186 ceph-mon[86379]: pgmap v1413: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:46:48.105 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:46:47 smithi018 ceph-mon[112590]: pgmap v1413: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:46:50.034 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:46:49 smithi186 ceph-mon[86379]: pgmap v1414: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:46:50.105 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:46:49 smithi018 ceph-mon[112590]: pgmap v1414: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:46:52.034 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:46:51 smithi186 ceph-mon[86379]: pgmap v1415: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:46:52.105 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:46:51 smithi018 ceph-mon[112590]: pgmap v1415: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:46:54.034 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:46:53 smithi186 ceph-mon[86379]: pgmap v1416: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:46:54.105 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:46:53 smithi018 ceph-mon[112590]: pgmap v1416: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:46:56.034 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:46:55 smithi186 ceph-mon[86379]: pgmap v1417: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:46:56.034 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:46:55 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T21:46:56.105 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:46:55 smithi018 ceph-mon[112590]: pgmap v1417: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:46:56.105 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:46:55 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T21:46:58.034 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:46:57 smithi186 ceph-mon[86379]: pgmap v1418: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:46:58.105 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:46:57 smithi018 ceph-mon[112590]: pgmap v1418: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:47:00.034 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:46:59 smithi186 ceph-mon[86379]: pgmap v1419: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:47:00.105 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:46:59 smithi018 ceph-mon[112590]: pgmap v1419: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:47:02.034 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:47:01 smithi186 ceph-mon[86379]: pgmap v1420: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:47:02.105 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:47:01 smithi018 ceph-mon[112590]: pgmap v1420: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:47:04.034 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:47:03 smithi186 ceph-mon[86379]: pgmap v1421: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:47:04.105 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:47:03 smithi018 ceph-mon[112590]: pgmap v1421: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:47:06.034 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:47:05 smithi186 ceph-mon[86379]: pgmap v1422: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:47:06.105 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:47:05 smithi018 ceph-mon[112590]: pgmap v1422: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:47:08.034 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:47:07 smithi186 ceph-mon[86379]: pgmap v1423: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:47:08.105 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:47:07 smithi018 ceph-mon[112590]: pgmap v1423: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:47:10.106 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:47:09 smithi018 ceph-mon[112590]: pgmap v1424: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:47:10.284 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:47:09 smithi186 ceph-mon[86379]: pgmap v1424: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:47:11.105 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:47:10 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T21:47:11.284 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:47:10 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T21:47:12.105 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:47:11 smithi018 ceph-mon[112590]: pgmap v1425: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:47:12.284 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:47:11 smithi186 ceph-mon[86379]: pgmap v1425: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:47:13.427 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:13.424+0000 7f2d35400700 1 -- 172.21.15.18:0/3251538475 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2d300fdc60 msgr2=0x7f2d300fe030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:47:13.427 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:13.424+0000 7f2d35400700 1 --2- 172.21.15.18:0/3251538475 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2d300fdc60 0x7f2d300fe030 secure :-1 s=READY pgs=1198 cs=0 l=1 rev1=1 crypto rx=0x7f2d20004940 tx=0x7f2d20009e30 comp rx=0 tx=0).stop 2024-04-12T21:47:13.427 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:13.425+0000 7f2d35400700 1 -- 172.21.15.18:0/3251538475 shutdown_connections 2024-04-12T21:47:13.427 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:13.425+0000 7f2d35400700 1 --2- 172.21.15.18:0/3251538475 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2d300fe600 0x7f2d30106c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:47:13.427 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:13.425+0000 7f2d35400700 1 --2- 172.21.15.18:0/3251538475 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2d300fdc60 0x7f2d300fe030 unknown :-1 s=CLOSED pgs=1198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:47:13.427 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:13.425+0000 7f2d35400700 1 -- 172.21.15.18:0/3251538475 >> 172.21.15.18:0/3251538475 conn(0x7f2d300f76b0 msgr2=0x7f2d300f9ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:47:13.427 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:13.425+0000 7f2d35400700 1 -- 172.21.15.18:0/3251538475 shutdown_connections 2024-04-12T21:47:13.428 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:13.425+0000 7f2d35400700 1 -- 172.21.15.18:0/3251538475 wait complete. 2024-04-12T21:47:13.429 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:13.426+0000 7f2d35400700 1 Processor -- start 2024-04-12T21:47:13.429 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:13.426+0000 7f2d35400700 1 -- start start 2024-04-12T21:47:13.429 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:13.426+0000 7f2d35400700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2d300fdc60 0x7f2d30113920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:47:13.429 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:13.426+0000 7f2d35400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2d300fe600 0x7f2d30113e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:47:13.429 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:13.426+0000 7f2d35400700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d301144b0 con 0x7f2d300fe600 2024-04-12T21:47:13.429 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:13.426+0000 7f2d35400700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d3010d8a0 con 0x7f2d300fdc60 2024-04-12T21:47:13.429 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:13.427+0000 7f2d27e00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2d300fe600 0x7f2d30113e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:47:13.429 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:13.427+0000 7f2d27e00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2d300fe600 0x7f2d30113e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:60758/0 (socket says 172.21.15.18:60758) 2024-04-12T21:47:13.429 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:13.427+0000 7f2d27e00700 1 -- 172.21.15.18:0/3265702753 learned_addr learned my addr 172.21.15.18:0/3265702753 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:47:13.430 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:13.427+0000 7f2d2fe00700 1 --2- 172.21.15.18:0/3265702753 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2d300fdc60 0x7f2d30113920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:47:13.430 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:13.427+0000 7f2d27e00700 1 -- 172.21.15.18:0/3265702753 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2d300fdc60 msgr2=0x7f2d30113920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:47:13.430 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:13.427+0000 7f2d27e00700 1 --2- 172.21.15.18:0/3265702753 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2d300fdc60 0x7f2d30113920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:47:13.430 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:13.427+0000 7f2d27e00700 1 -- 172.21.15.18:0/3265702753 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d20005040 con 0x7f2d300fe600 2024-04-12T21:47:13.431 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:13.428+0000 7f2d27e00700 1 --2- 172.21.15.18:0/3265702753 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2d300fe600 0x7f2d30113e60 secure :-1 s=READY pgs=1199 cs=0 l=1 rev1=1 crypto rx=0x7f2d1800eb40 tx=0x7f2d1800b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:47:13.431 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:13.428+0000 7f2d2cc00700 1 -- 172.21.15.18:0/3265702753 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2d18007cc0 con 0x7f2d300fe600 2024-04-12T21:47:13.431 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:13.428+0000 7f2d2cc00700 1 -- 172.21.15.18:0/3265702753 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f2d1800f040 con 0x7f2d300fe600 2024-04-12T21:47:13.431 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:13.428+0000 7f2d35400700 1 -- 172.21.15.18:0/3265702753 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2d3010db20 con 0x7f2d300fe600 2024-04-12T21:47:13.431 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:13.428+0000 7f2d2cc00700 1 -- 172.21.15.18:0/3265702753 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2d18009620 con 0x7f2d300fe600 2024-04-12T21:47:13.431 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:13.428+0000 7f2d35400700 1 -- 172.21.15.18:0/3265702753 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2d3010e060 con 0x7f2d300fe600 2024-04-12T21:47:13.431 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:13.429+0000 7f2d35400700 1 -- 172.21.15.18:0/3265702753 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2d300668d0 con 0x7f2d300fe600 2024-04-12T21:47:13.432 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:13.430+0000 7f2d2cc00700 1 -- 172.21.15.18:0/3265702753 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7f2d18009780 con 0x7f2d300fe600 2024-04-12T21:47:13.433 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:13.431+0000 7f2d2cc00700 1 --2- 172.21.15.18:0/3265702753 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f2d1406ec00 0x7f2d140710b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:47:13.433 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:13.431+0000 7f2d2cc00700 1 -- 172.21.15.18:0/3265702753 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 6757+0+0 (secure 0 0 0) 0x7f2d1808e650 con 0x7f2d300fe600 2024-04-12T21:47:13.433 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:13.432+0000 7f2d2fe00700 1 --2- 172.21.15.18:0/3265702753 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f2d1406ec00 0x7f2d140710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:47:13.434 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:13.432+0000 7f2d2fe00700 1 --2- 172.21.15.18:0/3265702753 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f2d1406ec00 0x7f2d140710b0 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f2d2000b0d0 tx=0x7f2d20009e30 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:47:13.438 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:13.437+0000 7f2d2cc00700 1 -- 172.21.15.18:0/3265702753 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f2d1805aeb0 con 0x7f2d300fe600 2024-04-12T21:47:13.691 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:13.689+0000 7f2d35400700 1 -- 172.21.15.18:0/3265702753 --> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f2d301135f0 con 0x7f2d1406ec00 2024-04-12T21:47:13.692 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:13.691+0000 7f2d2cc00700 1 -- 172.21.15.18:0/3265702753 <== mgr.34102 v2:172.21.15.18:6800/3812891919 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f2d301135f0 con 0x7f2d1406ec00 2024-04-12T21:47:13.695 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:13.693+0000 7f2d35400700 1 -- 172.21.15.18:0/3265702753 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f2d1406ec00 msgr2=0x7f2d140710b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:47:13.696 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:13.693+0000 7f2d35400700 1 --2- 172.21.15.18:0/3265702753 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f2d1406ec00 0x7f2d140710b0 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f2d2000b0d0 tx=0x7f2d20009e30 comp rx=0 tx=0).stop 2024-04-12T21:47:13.696 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:13.693+0000 7f2d35400700 1 -- 172.21.15.18:0/3265702753 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2d300fe600 msgr2=0x7f2d30113e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:47:13.696 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:13.693+0000 7f2d35400700 1 --2- 172.21.15.18:0/3265702753 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2d300fe600 0x7f2d30113e60 secure :-1 s=READY pgs=1199 cs=0 l=1 rev1=1 crypto rx=0x7f2d1800eb40 tx=0x7f2d1800b040 comp rx=0 tx=0).stop 2024-04-12T21:47:13.696 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:13.693+0000 7f2d35400700 1 -- 172.21.15.18:0/3265702753 shutdown_connections 2024-04-12T21:47:13.696 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:13.693+0000 7f2d35400700 1 --2- 172.21.15.18:0/3265702753 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f2d1406ec00 0x7f2d140710b0 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:47:13.696 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:13.693+0000 7f2d35400700 1 --2- 172.21.15.18:0/3265702753 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2d300fdc60 0x7f2d30113920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:47:13.696 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:13.693+0000 7f2d35400700 1 --2- 172.21.15.18:0/3265702753 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2d300fe600 0x7f2d30113e60 unknown :-1 s=CLOSED pgs=1199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:47:13.696 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:13.693+0000 7f2d35400700 1 -- 172.21.15.18:0/3265702753 >> 172.21.15.18:0/3265702753 conn(0x7f2d300f76b0 msgr2=0x7f2d300f8c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:47:13.696 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:13.694+0000 7f2d35400700 1 -- 172.21.15.18:0/3265702753 shutdown_connections 2024-04-12T21:47:13.696 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:13.694+0000 7f2d35400700 1 -- 172.21.15.18:0/3265702753 wait complete. 2024-04-12T21:47:13.711 INFO:teuthology.orchestra.run.smithi018.stdout:true 2024-04-12T21:47:13.858 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:13.856+0000 7f1b27800700 1 -- 172.21.15.18:0/627882514 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1b20100be0 msgr2=0x7f1b20101050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:47:13.859 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:13.856+0000 7f1b27800700 1 --2- 172.21.15.18:0/627882514 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1b20100be0 0x7f1b20101050 secure :-1 s=READY pgs=1200 cs=0 l=1 rev1=1 crypto rx=0x7f1b0c004940 tx=0x7f1b0c009e30 comp rx=0 tx=0).stop 2024-04-12T21:47:13.859 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:13.857+0000 7f1b27800700 1 -- 172.21.15.18:0/627882514 shutdown_connections 2024-04-12T21:47:13.859 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:13.857+0000 7f1b27800700 1 --2- 172.21.15.18:0/627882514 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1b20100be0 0x7f1b20101050 unknown :-1 s=CLOSED pgs=1200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:47:13.859 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:13.857+0000 7f1b27800700 1 --2- 172.21.15.18:0/627882514 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1b200ff3e0 0x7f1b200ff7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:47:13.859 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:13.857+0000 7f1b27800700 1 -- 172.21.15.18:0/627882514 >> 172.21.15.18:0/627882514 conn(0x7f1b200faba0 msgr2=0x7f1b200fcfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:47:13.859 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:13.857+0000 7f1b27800700 1 -- 172.21.15.18:0/627882514 shutdown_connections 2024-04-12T21:47:13.859 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:13.857+0000 7f1b27800700 1 -- 172.21.15.18:0/627882514 wait complete. 2024-04-12T21:47:13.860 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:13.857+0000 7f1b27800700 1 Processor -- start 2024-04-12T21:47:13.860 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:13.858+0000 7f1b27800700 1 -- start start 2024-04-12T21:47:13.860 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:13.858+0000 7f1b27800700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1b200ff3e0 0x7f1b20115ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:47:13.860 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:13.858+0000 7f1b27800700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1b20100be0 0x7f1b201160e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:47:13.860 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:13.858+0000 7f1b27800700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b20116730 con 0x7f1b20100be0 2024-04-12T21:47:13.860 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:13.858+0000 7f1b27800700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b2010fd10 con 0x7f1b200ff3e0 2024-04-12T21:47:13.861 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:13.858+0000 7f1b1fe00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1b20100be0 0x7f1b201160e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:47:13.861 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:13.858+0000 7f1b1fe00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1b20100be0 0x7f1b201160e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:60766/0 (socket says 172.21.15.18:60766) 2024-04-12T21:47:13.861 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:13.858+0000 7f1b1fe00700 1 -- 172.21.15.18:0/3250368665 learned_addr learned my addr 172.21.15.18:0/3250368665 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:47:13.861 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:13.859+0000 7f1b24e00700 1 --2- 172.21.15.18:0/3250368665 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1b200ff3e0 0x7f1b20115ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:47:13.861 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:13.859+0000 7f1b1fe00700 1 -- 172.21.15.18:0/3250368665 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1b200ff3e0 msgr2=0x7f1b20115ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:47:13.861 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:13.859+0000 7f1b1fe00700 1 --2- 172.21.15.18:0/3250368665 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1b200ff3e0 0x7f1b20115ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:47:13.861 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:13.859+0000 7f1b1fe00700 1 -- 172.21.15.18:0/3250368665 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b0c005040 con 0x7f1b20100be0 2024-04-12T21:47:13.862 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:13.859+0000 7f1b1fe00700 1 --2- 172.21.15.18:0/3250368665 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1b20100be0 0x7f1b201160e0 secure :-1 s=READY pgs=1201 cs=0 l=1 rev1=1 crypto rx=0x7f1b0c00b3b0 tx=0x7f1b0c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:47:13.862 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:13.859+0000 7f1b1cc00700 1 -- 172.21.15.18:0/3250368665 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1b0c00e040 con 0x7f1b20100be0 2024-04-12T21:47:13.862 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:13.859+0000 7f1b1cc00700 1 -- 172.21.15.18:0/3250368665 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f1b0c00bbc0 con 0x7f1b20100be0 2024-04-12T21:47:13.862 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:13.859+0000 7f1b1cc00700 1 -- 172.21.15.18:0/3250368665 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1b0c01c3f0 con 0x7f1b20100be0 2024-04-12T21:47:13.862 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:13.859+0000 7f1b27800700 1 -- 172.21.15.18:0/3250368665 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1b2010ff90 con 0x7f1b20100be0 2024-04-12T21:47:13.862 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:13.860+0000 7f1b27800700 1 -- 172.21.15.18:0/3250368665 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1b20110550 con 0x7f1b20100be0 2024-04-12T21:47:13.862 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:13.861+0000 7f1b27800700 1 -- 172.21.15.18:0/3250368665 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1b20100780 con 0x7f1b20100be0 2024-04-12T21:47:13.863 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:13.862+0000 7f1b1cc00700 1 -- 172.21.15.18:0/3250368665 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7f1b0c00bd30 con 0x7f1b20100be0 2024-04-12T21:47:13.864 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:13.863+0000 7f1b1cc00700 1 --2- 172.21.15.18:0/3250368665 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f1b0806ec00 0x7f1b080710b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:47:13.864 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:13.863+0000 7f1b1cc00700 1 -- 172.21.15.18:0/3250368665 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 6757+0+0 (secure 0 0 0) 0x7f1b0c08fcf0 con 0x7f1b20100be0 2024-04-12T21:47:13.865 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:13.863+0000 7f1b24e00700 1 --2- 172.21.15.18:0/3250368665 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f1b0806ec00 0x7f1b080710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:47:13.865 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:13.864+0000 7f1b24e00700 1 --2- 172.21.15.18:0/3250368665 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f1b0806ec00 0x7f1b080710b0 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f1b14004d60 tx=0x7f1b1400b040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:47:13.870 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:13.868+0000 7f1b1cc00700 1 -- 172.21.15.18:0/3250368665 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f1b0c05c420 con 0x7f1b20100be0 2024-04-12T21:47:14.105 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:47:13 smithi018 ceph-mon[112590]: pgmap v1426: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:47:14.128 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:14.127+0000 7f1b27800700 1 -- 172.21.15.18:0/3250368665 --> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f1b201111f0 con 0x7f1b0806ec00 2024-04-12T21:47:14.130 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:14.129+0000 7f1b1cc00700 1 -- 172.21.15.18:0/3250368665 <== mgr.34102 v2:172.21.15.18:6800/3812891919 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f1b201111f0 con 0x7f1b0806ec00 2024-04-12T21:47:14.134 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:14.132+0000 7f1b27800700 1 -- 172.21.15.18:0/3250368665 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f1b0806ec00 msgr2=0x7f1b080710b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:47:14.134 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:14.132+0000 7f1b27800700 1 --2- 172.21.15.18:0/3250368665 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f1b0806ec00 0x7f1b080710b0 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f1b14004d60 tx=0x7f1b1400b040 comp rx=0 tx=0).stop 2024-04-12T21:47:14.134 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:14.132+0000 7f1b27800700 1 -- 172.21.15.18:0/3250368665 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1b20100be0 msgr2=0x7f1b201160e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:47:14.134 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:14.132+0000 7f1b27800700 1 --2- 172.21.15.18:0/3250368665 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1b20100be0 0x7f1b201160e0 secure :-1 s=READY pgs=1201 cs=0 l=1 rev1=1 crypto rx=0x7f1b0c00b3b0 tx=0x7f1b0c009e30 comp rx=0 tx=0).stop 2024-04-12T21:47:14.134 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:14.132+0000 7f1b27800700 1 -- 172.21.15.18:0/3250368665 shutdown_connections 2024-04-12T21:47:14.134 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:14.132+0000 7f1b27800700 1 --2- 172.21.15.18:0/3250368665 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f1b0806ec00 0x7f1b080710b0 unknown :-1 s=CLOSED pgs=638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:47:14.134 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:14.132+0000 7f1b27800700 1 --2- 172.21.15.18:0/3250368665 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1b200ff3e0 0x7f1b20115ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:47:14.134 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:14.132+0000 7f1b27800700 1 --2- 172.21.15.18:0/3250368665 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1b20100be0 0x7f1b201160e0 unknown :-1 s=CLOSED pgs=1201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:47:14.134 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:14.132+0000 7f1b27800700 1 -- 172.21.15.18:0/3250368665 >> 172.21.15.18:0/3250368665 conn(0x7f1b200faba0 msgr2=0x7f1b200fc360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:47:14.134 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:14.132+0000 7f1b27800700 1 -- 172.21.15.18:0/3250368665 shutdown_connections 2024-04-12T21:47:14.134 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:14.132+0000 7f1b27800700 1 -- 172.21.15.18:0/3250368665 wait complete. 2024-04-12T21:47:14.284 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:47:13 smithi186 ceph-mon[86379]: pgmap v1426: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:47:14.296 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:14.294+0000 7fec4b400700 1 -- 172.21.15.18:0/1510158363 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fec440fbbd0 msgr2=0x7fec440fbfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:47:14.296 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:14.294+0000 7fec4b400700 1 --2- 172.21.15.18:0/1510158363 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fec440fbbd0 0x7fec440fbfa0 secure :-1 s=READY pgs=1202 cs=0 l=1 rev1=1 crypto rx=0x7fec38004970 tx=0x7fec38009e30 comp rx=0 tx=0).stop 2024-04-12T21:47:14.296 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:14.294+0000 7fec4b400700 1 -- 172.21.15.18:0/1510158363 shutdown_connections 2024-04-12T21:47:14.296 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:14.294+0000 7fec4b400700 1 --2- 172.21.15.18:0/1510158363 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fec440fc4e0 0x7fec44109240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:47:14.296 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:14.294+0000 7fec4b400700 1 --2- 172.21.15.18:0/1510158363 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fec440fbbd0 0x7fec440fbfa0 unknown :-1 s=CLOSED pgs=1202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:47:14.296 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:14.294+0000 7fec4b400700 1 -- 172.21.15.18:0/1510158363 >> 172.21.15.18:0/1510158363 conn(0x7fec440f7860 msgr2=0x7fec440f9c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:47:14.296 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:14.294+0000 7fec4b400700 1 -- 172.21.15.18:0/1510158363 shutdown_connections 2024-04-12T21:47:14.296 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:14.294+0000 7fec4b400700 1 -- 172.21.15.18:0/1510158363 wait complete. 2024-04-12T21:47:14.297 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:14.295+0000 7fec4b400700 1 Processor -- start 2024-04-12T21:47:14.297 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:14.295+0000 7fec4b400700 1 -- start start 2024-04-12T21:47:14.297 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:14.296+0000 7fec4b400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fec440fbbd0 0x7fec44115cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:47:14.298 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:14.296+0000 7fec4b400700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fec440fc4e0 0x7fec44116230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:47:14.298 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:14.296+0000 7fec4b400700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec44116880 con 0x7fec440fbbd0 2024-04-12T21:47:14.298 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:14.296+0000 7fec4b400700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec4410fc50 con 0x7fec440fc4e0 2024-04-12T21:47:14.298 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:14.296+0000 7fec48a00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fec440fbbd0 0x7fec44115cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:47:14.298 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:14.296+0000 7fec48a00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fec440fbbd0 0x7fec44115cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:60772/0 (socket says 172.21.15.18:60772) 2024-04-12T21:47:14.298 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:14.296+0000 7fec48a00700 1 -- 172.21.15.18:0/3997893944 learned_addr learned my addr 172.21.15.18:0/3997893944 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:47:14.299 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:14.296+0000 7fec43e00700 1 --2- 172.21.15.18:0/3997893944 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fec440fc4e0 0x7fec44116230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:47:14.299 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:14.296+0000 7fec48a00700 1 -- 172.21.15.18:0/3997893944 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fec440fc4e0 msgr2=0x7fec44116230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:47:14.299 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:14.296+0000 7fec48a00700 1 --2- 172.21.15.18:0/3997893944 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fec440fc4e0 0x7fec44116230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:47:14.299 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:14.296+0000 7fec48a00700 1 -- 172.21.15.18:0/3997893944 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec38005040 con 0x7fec440fbbd0 2024-04-12T21:47:14.299 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:14.297+0000 7fec48a00700 1 --2- 172.21.15.18:0/3997893944 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fec440fbbd0 0x7fec44115cf0 secure :-1 s=READY pgs=1203 cs=0 l=1 rev1=1 crypto rx=0x7fec38004f50 tx=0x7fec38009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:47:14.299 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:14.297+0000 7fec40c00700 1 -- 172.21.15.18:0/3997893944 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fec3800e040 con 0x7fec440fbbd0 2024-04-12T21:47:14.299 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:14.297+0000 7fec40c00700 1 -- 172.21.15.18:0/3997893944 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fec3800bc10 con 0x7fec440fbbd0 2024-04-12T21:47:14.299 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:14.297+0000 7fec4b400700 1 -- 172.21.15.18:0/3997893944 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fec4410fed0 con 0x7fec440fbbd0 2024-04-12T21:47:14.300 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:14.297+0000 7fec40c00700 1 -- 172.21.15.18:0/3997893944 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fec3801c630 con 0x7fec440fbbd0 2024-04-12T21:47:14.300 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:14.297+0000 7fec4b400700 1 -- 172.21.15.18:0/3997893944 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fec44110490 con 0x7fec440fbbd0 2024-04-12T21:47:14.300 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:14.298+0000 7fec4b400700 1 -- 172.21.15.18:0/3997893944 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fec44066710 con 0x7fec440fbbd0 2024-04-12T21:47:14.301 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:14.299+0000 7fec40c00700 1 -- 172.21.15.18:0/3997893944 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7fec38013420 con 0x7fec440fbbd0 2024-04-12T21:47:14.302 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:14.301+0000 7fec40c00700 1 --2- 172.21.15.18:0/3997893944 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fec2c06ec10 0x7fec2c0710c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:47:14.303 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:14.301+0000 7fec40c00700 1 -- 172.21.15.18:0/3997893944 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 6757+0+0 (secure 0 0 0) 0x7fec3808fc60 con 0x7fec440fbbd0 2024-04-12T21:47:14.303 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:14.301+0000 7fec43e00700 1 --2- 172.21.15.18:0/3997893944 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fec2c06ec10 0x7fec2c0710c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:47:14.303 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:14.302+0000 7fec43e00700 1 --2- 172.21.15.18:0/3997893944 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fec2c06ec10 0x7fec2c0710c0 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7fec44111590 tx=0x7fec3000b040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:47:14.307 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:14.306+0000 7fec40c00700 1 -- 172.21.15.18:0/3997893944 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7fec3805c490 con 0x7fec440fbbd0 2024-04-12T21:47:14.555 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:14.553+0000 7fec4b400700 1 -- 172.21.15.18:0/3997893944 --> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fec44066710 con 0x7fec2c06ec10 2024-04-12T21:47:14.569 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:14.567+0000 7fec40c00700 1 -- 172.21.15.18:0/3997893944 <== mgr.34102 v2:172.21.15.18:6800/3812891919 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7fec44066710 con 0x7fec2c06ec10 2024-04-12T21:47:14.569 INFO:teuthology.orchestra.run.smithi018.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-12T21:47:14.569 INFO:teuthology.orchestra.run.smithi018.stdout:alertmanager.smithi018 smithi018 *:9093,9094 running (47m) 8m ago 62m 14.8M - 0.25.0 c8568f914cd2 c332090068e8 2024-04-12T21:47:14.569 INFO:teuthology.orchestra.run.smithi018.stdout:crash.smithi018 smithi018 running (44m) 8m ago 62m 6665k - 19.0.0-2879-g324baf8d 8717720dc5e5 b7f80d1f7982 2024-04-12T21:47:14.570 INFO:teuthology.orchestra.run.smithi018.stdout:crash.smithi186 smithi186 running (44m) 8m ago 61m 6660k - 19.0.0-2879-g324baf8d 8717720dc5e5 a4f55c81b623 2024-04-12T21:47:14.570 INFO:teuthology.orchestra.run.smithi018.stdout:grafana.smithi018 smithi018 *:3000 running (47m) 8m ago 61m 73.8M - 9.4.12 2bacad6d85d8 b08e3bf176dd 2024-04-12T21:47:14.570 INFO:teuthology.orchestra.run.smithi018.stdout:mds.cephfs.smithi018.acqbsc smithi018 running (40m) 8m ago 55m 21.9M - 19.0.0-2879-g324baf8d 8717720dc5e5 021f99ab24ae 2024-04-12T21:47:14.570 INFO:teuthology.orchestra.run.smithi018.stdout:mds.cephfs.smithi018.szjluo smithi018 running (39m) 8m ago 55m 21.7M - 19.0.0-2879-g324baf8d 8717720dc5e5 0003117479d4 2024-04-12T21:47:14.570 INFO:teuthology.orchestra.run.smithi018.stdout:mds.cephfs.smithi186.jdllyn smithi186 running (24m) 8m ago 55m 16.5M - 19.0.0-2879-g324baf8d 8717720dc5e5 21b1f46eef23 2024-04-12T21:47:14.570 INFO:teuthology.orchestra.run.smithi018.stdout:mds.cephfs.smithi186.wjnafs smithi186 running (9m) 8m ago 55m 11.0M - 19.0.0-2879-g324baf8d 8717720dc5e5 3cf1ef4148e2 2024-04-12T21:47:14.570 INFO:teuthology.orchestra.run.smithi018.stdout:mgr.smithi018.xjzwga smithi018 *:8443,9283,8765 running (50m) 8m ago 64m 517M - 19.0.0-2879-g324baf8d 8717720dc5e5 56088305eab7 2024-04-12T21:47:14.570 INFO:teuthology.orchestra.run.smithi018.stdout:mgr.smithi186.kqcfxk smithi186 *:8443,9283,8765 running (49m) 8m ago 61m 460M - 19.0.0-2879-g324baf8d 8717720dc5e5 09cd766e5e0d 2024-04-12T21:47:14.570 INFO:teuthology.orchestra.run.smithi018.stdout:mon.smithi018 smithi018 running (45m) 8m ago 64m 88.2M 2048M 19.0.0-2879-g324baf8d 8717720dc5e5 df3e201dfed6 2024-04-12T21:47:14.570 INFO:teuthology.orchestra.run.smithi018.stdout:mon.smithi186 smithi186 running (45m) 8m ago 60m 53.6M 2048M 19.0.0-2879-g324baf8d 8717720dc5e5 c77498da9b2b 2024-04-12T21:47:14.570 INFO:teuthology.orchestra.run.smithi018.stdout:node-exporter.smithi018 smithi018 *:9100 running (48m) 8m ago 62m 8468k - 1.5.0 0da6a335fe13 326c3d9c6847 2024-04-12T21:47:14.570 INFO:teuthology.orchestra.run.smithi018.stdout:node-exporter.smithi186 smithi186 *:9100 running (48m) 8m ago 61m 8166k - 1.5.0 0da6a335fe13 ec2112baeecf 2024-04-12T21:47:14.570 INFO:teuthology.orchestra.run.smithi018.stdout:osd.0 smithi018 running (43m) 8m ago 59m 123M 1559M 19.0.0-2879-g324baf8d 8717720dc5e5 60b1273ca047 2024-04-12T21:47:14.570 INFO:teuthology.orchestra.run.smithi018.stdout:osd.1 smithi018 running (42m) 8m ago 59m 95.4M 1559M 19.0.0-2879-g324baf8d 8717720dc5e5 170ce1e5fb8d 2024-04-12T21:47:14.571 INFO:teuthology.orchestra.run.smithi018.stdout:osd.2 smithi018 running (42m) 8m ago 58m 94.6M 1559M 19.0.0-2879-g324baf8d 8717720dc5e5 c2cbc6aa2839 2024-04-12T21:47:14.571 INFO:teuthology.orchestra.run.smithi018.stdout:osd.3 smithi186 running (41m) 8m ago 58m 101M 2581M 19.0.0-2879-g324baf8d 8717720dc5e5 0138e8262e4b 2024-04-12T21:47:14.571 INFO:teuthology.orchestra.run.smithi018.stdout:osd.4 smithi186 running (41m) 8m ago 57m 91.5M 2581M 19.0.0-2879-g324baf8d 8717720dc5e5 b20cb63b25c9 2024-04-12T21:47:14.571 INFO:teuthology.orchestra.run.smithi018.stdout:osd.5 smithi186 running (40m) 8m ago 57m 92.2M 2581M 19.0.0-2879-g324baf8d 8717720dc5e5 2513bea2610e 2024-04-12T21:47:14.571 INFO:teuthology.orchestra.run.smithi018.stdout:prometheus.smithi018 smithi018 *:9095 running (48m) 8m ago 61m 69.6M - 2.43.0 a07b618ecd1d f58841127e21 2024-04-12T21:47:14.571 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:14.570+0000 7fec4b400700 1 -- 172.21.15.18:0/3997893944 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fec2c06ec10 msgr2=0x7fec2c0710c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:47:14.571 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:14.570+0000 7fec4b400700 1 --2- 172.21.15.18:0/3997893944 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fec2c06ec10 0x7fec2c0710c0 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7fec44111590 tx=0x7fec3000b040 comp rx=0 tx=0).stop 2024-04-12T21:47:14.572 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:14.570+0000 7fec4b400700 1 -- 172.21.15.18:0/3997893944 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fec440fbbd0 msgr2=0x7fec44115cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:47:14.572 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:14.570+0000 7fec4b400700 1 --2- 172.21.15.18:0/3997893944 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fec440fbbd0 0x7fec44115cf0 secure :-1 s=READY pgs=1203 cs=0 l=1 rev1=1 crypto rx=0x7fec38004f50 tx=0x7fec38009e30 comp rx=0 tx=0).stop 2024-04-12T21:47:14.572 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:14.570+0000 7fec4b400700 1 -- 172.21.15.18:0/3997893944 shutdown_connections 2024-04-12T21:47:14.572 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:14.570+0000 7fec4b400700 1 --2- 172.21.15.18:0/3997893944 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fec2c06ec10 0x7fec2c0710c0 unknown :-1 s=CLOSED pgs=639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:47:14.572 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:14.570+0000 7fec4b400700 1 --2- 172.21.15.18:0/3997893944 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fec440fbbd0 0x7fec44115cf0 unknown :-1 s=CLOSED pgs=1203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:47:14.572 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:14.570+0000 7fec4b400700 1 --2- 172.21.15.18:0/3997893944 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fec440fc4e0 0x7fec44116230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:47:14.572 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:14.570+0000 7fec4b400700 1 -- 172.21.15.18:0/3997893944 >> 172.21.15.18:0/3997893944 conn(0x7fec440f7860 msgr2=0x7fec440f7c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:47:14.573 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:14.571+0000 7fec4b400700 1 -- 172.21.15.18:0/3997893944 shutdown_connections 2024-04-12T21:47:14.573 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:14.571+0000 7fec4b400700 1 -- 172.21.15.18:0/3997893944 wait complete. 2024-04-12T21:47:14.731 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:14.729+0000 7fdd4da00700 1 -- 172.21.15.18:0/1900211704 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fdd48109dd0 msgr2=0x7fdd48103440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:47:14.732 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:14.729+0000 7fdd4da00700 1 --2- 172.21.15.18:0/1900211704 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fdd48109dd0 0x7fdd48103440 secure :-1 s=READY pgs=1204 cs=0 l=1 rev1=1 crypto rx=0x7fdd30004940 tx=0x7fdd30009e30 comp rx=0 tx=0).stop 2024-04-12T21:47:14.732 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:14.729+0000 7fdd4da00700 1 -- 172.21.15.18:0/1900211704 shutdown_connections 2024-04-12T21:47:14.732 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:14.729+0000 7fdd4da00700 1 --2- 172.21.15.18:0/1900211704 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fdd48109dd0 0x7fdd48103440 unknown :-1 s=CLOSED pgs=1204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:47:14.732 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:14.729+0000 7fdd4da00700 1 --2- 172.21.15.18:0/1900211704 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdd48109430 0x7fdd48109800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:47:14.732 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:14.729+0000 7fdd4da00700 1 -- 172.21.15.18:0/1900211704 >> 172.21.15.18:0/1900211704 conn(0x7fdd48072c00 msgr2=0x7fdd48075010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:47:14.732 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:14.730+0000 7fdd4da00700 1 -- 172.21.15.18:0/1900211704 shutdown_connections 2024-04-12T21:47:14.732 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:14.730+0000 7fdd4da00700 1 -- 172.21.15.18:0/1900211704 wait complete. 2024-04-12T21:47:14.733 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:14.731+0000 7fdd4da00700 1 Processor -- start 2024-04-12T21:47:14.733 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:14.731+0000 7fdd4da00700 1 -- start start 2024-04-12T21:47:14.733 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:14.731+0000 7fdd4da00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fdd48109430 0x7fdd48119f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:47:14.733 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:14.731+0000 7fdd4da00700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdd48109dd0 0x7fdd4811a4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:47:14.733 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:14.731+0000 7fdd4da00700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd4811ab10 con 0x7fdd48109430 2024-04-12T21:47:14.734 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:14.731+0000 7fdd4da00700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd481140f0 con 0x7fdd48109dd0 2024-04-12T21:47:14.734 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:14.732+0000 7fdd46a00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fdd48109430 0x7fdd48119f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:47:14.734 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:14.732+0000 7fdd46a00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fdd48109430 0x7fdd48119f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:60802/0 (socket says 172.21.15.18:60802) 2024-04-12T21:47:14.734 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:14.732+0000 7fdd46a00700 1 -- 172.21.15.18:0/609112026 learned_addr learned my addr 172.21.15.18:0/609112026 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:47:14.734 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:14.732+0000 7fdd3fe00700 1 --2- 172.21.15.18:0/609112026 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdd48109dd0 0x7fdd4811a4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:47:14.735 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:14.732+0000 7fdd46a00700 1 -- 172.21.15.18:0/609112026 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdd48109dd0 msgr2=0x7fdd4811a4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:47:14.735 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:14.732+0000 7fdd46a00700 1 --2- 172.21.15.18:0/609112026 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdd48109dd0 0x7fdd4811a4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:47:14.735 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:14.732+0000 7fdd46a00700 1 -- 172.21.15.18:0/609112026 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd30005040 con 0x7fdd48109430 2024-04-12T21:47:14.735 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:14.732+0000 7fdd46a00700 1 --2- 172.21.15.18:0/609112026 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fdd48109430 0x7fdd48119f80 secure :-1 s=READY pgs=1205 cs=0 l=1 rev1=1 crypto rx=0x7fdd38002a50 tx=0x7fdd380046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:47:14.735 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:14.733+0000 7fdd3ea00700 1 -- 172.21.15.18:0/609112026 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdd38007cc0 con 0x7fdd48109430 2024-04-12T21:47:14.735 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:14.733+0000 7fdd3ea00700 1 -- 172.21.15.18:0/609112026 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fdd3800f040 con 0x7fdd48109430 2024-04-12T21:47:14.735 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:14.733+0000 7fdd4da00700 1 -- 172.21.15.18:0/609112026 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdd481143d0 con 0x7fdd48109430 2024-04-12T21:47:14.735 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:14.733+0000 7fdd3ea00700 1 -- 172.21.15.18:0/609112026 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdd38009640 con 0x7fdd48109430 2024-04-12T21:47:14.735 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:14.733+0000 7fdd4da00700 1 -- 172.21.15.18:0/609112026 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdd48114990 con 0x7fdd48109430 2024-04-12T21:47:14.736 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:14.734+0000 7fdd4da00700 1 -- 172.21.15.18:0/609112026 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdd48104b80 con 0x7fdd48109430 2024-04-12T21:47:14.737 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:14.735+0000 7fdd3ea00700 1 -- 172.21.15.18:0/609112026 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7fdd380097a0 con 0x7fdd48109430 2024-04-12T21:47:14.738 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:14.736+0000 7fdd3ea00700 1 --2- 172.21.15.18:0/609112026 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fdd2c06ec00 0x7fdd2c0710b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:47:14.738 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:14.736+0000 7fdd3ea00700 1 -- 172.21.15.18:0/609112026 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 6757+0+0 (secure 0 0 0) 0x7fdd3808e760 con 0x7fdd48109430 2024-04-12T21:47:14.738 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:14.737+0000 7fdd3fe00700 1 --2- 172.21.15.18:0/609112026 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fdd2c06ec00 0x7fdd2c0710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:47:14.739 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:14.737+0000 7fdd3fe00700 1 --2- 172.21.15.18:0/609112026 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fdd2c06ec00 0x7fdd2c0710b0 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7fdd3000b3b0 tx=0x7fdd30009e30 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:47:14.742 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:14.741+0000 7fdd3ea00700 1 -- 172.21.15.18:0/609112026 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7fdd3805af10 con 0x7fdd48109430 2024-04-12T21:47:15.064 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:15.062+0000 7fdd4da00700 1 -- 172.21.15.18:0/609112026 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fdd481155d0 con 0x7fdd48109430 2024-04-12T21:47:15.064 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:15.063+0000 7fdd3ea00700 1 -- 172.21.15.18:0/609112026 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+623 (secure 0 0 0) 0x7fdd3805aaa0 con 0x7fdd48109430 2024-04-12T21:47:15.065 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:47:14 smithi018 ceph-mon[112590]: from='client.36468 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:47:15.065 INFO:teuthology.orchestra.run.smithi018.stdout:{ 2024-04-12T21:47:15.065 INFO:teuthology.orchestra.run.smithi018.stdout: "mon": { 2024-04-12T21:47:15.065 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 19.0.0-2879-g324baf8d (324baf8ddb91dea46cac50cb535c306115791355) squid (dev)": 2 2024-04-12T21:47:15.066 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-12T21:47:15.066 INFO:teuthology.orchestra.run.smithi018.stdout: "mgr": { 2024-04-12T21:47:15.066 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 19.0.0-2879-g324baf8d (324baf8ddb91dea46cac50cb535c306115791355) squid (dev)": 2 2024-04-12T21:47:15.066 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-12T21:47:15.066 INFO:teuthology.orchestra.run.smithi018.stdout: "osd": { 2024-04-12T21:47:15.066 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 19.0.0-2879-g324baf8d (324baf8ddb91dea46cac50cb535c306115791355) squid (dev)": 6 2024-04-12T21:47:15.066 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-12T21:47:15.066 INFO:teuthology.orchestra.run.smithi018.stdout: "mds": { 2024-04-12T21:47:15.066 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 19.0.0-2879-g324baf8d (324baf8ddb91dea46cac50cb535c306115791355) squid (dev)": 4 2024-04-12T21:47:15.066 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-12T21:47:15.066 INFO:teuthology.orchestra.run.smithi018.stdout: "overall": { 2024-04-12T21:47:15.066 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 19.0.0-2879-g324baf8d (324baf8ddb91dea46cac50cb535c306115791355) squid (dev)": 14 2024-04-12T21:47:15.066 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-04-12T21:47:15.066 INFO:teuthology.orchestra.run.smithi018.stdout:} 2024-04-12T21:47:15.067 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:15.066+0000 7fdd4da00700 1 -- 172.21.15.18:0/609112026 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fdd2c06ec00 msgr2=0x7fdd2c0710b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:47:15.068 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:15.066+0000 7fdd4da00700 1 --2- 172.21.15.18:0/609112026 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fdd2c06ec00 0x7fdd2c0710b0 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7fdd3000b3b0 tx=0x7fdd30009e30 comp rx=0 tx=0).stop 2024-04-12T21:47:15.068 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:15.066+0000 7fdd4da00700 1 -- 172.21.15.18:0/609112026 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fdd48109430 msgr2=0x7fdd48119f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:47:15.068 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:15.066+0000 7fdd4da00700 1 --2- 172.21.15.18:0/609112026 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fdd48109430 0x7fdd48119f80 secure :-1 s=READY pgs=1205 cs=0 l=1 rev1=1 crypto rx=0x7fdd38002a50 tx=0x7fdd380046b0 comp rx=0 tx=0).stop 2024-04-12T21:47:15.068 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:15.066+0000 7fdd4da00700 1 -- 172.21.15.18:0/609112026 shutdown_connections 2024-04-12T21:47:15.068 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:15.066+0000 7fdd4da00700 1 --2- 172.21.15.18:0/609112026 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fdd2c06ec00 0x7fdd2c0710b0 unknown :-1 s=CLOSED pgs=640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:47:15.069 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:15.066+0000 7fdd4da00700 1 --2- 172.21.15.18:0/609112026 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fdd48109430 0x7fdd48119f80 unknown :-1 s=CLOSED pgs=1205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:47:15.069 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:15.066+0000 7fdd4da00700 1 --2- 172.21.15.18:0/609112026 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdd48109dd0 0x7fdd4811a4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:47:15.069 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:15.066+0000 7fdd4da00700 1 -- 172.21.15.18:0/609112026 >> 172.21.15.18:0/609112026 conn(0x7fdd48072c00 msgr2=0x7fdd48074c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:47:15.069 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:15.066+0000 7fdd4da00700 1 -- 172.21.15.18:0/609112026 shutdown_connections 2024-04-12T21:47:15.069 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:15.066+0000 7fdd4da00700 1 -- 172.21.15.18:0/609112026 wait complete. 2024-04-12T21:47:15.226 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:15.224+0000 7f6e72000700 1 -- 172.21.15.18:0/728124808 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6e6c0fe830 msgr2=0x7f6e6c106f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:47:15.226 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:15.224+0000 7f6e72000700 1 --2- 172.21.15.18:0/728124808 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6e6c0fe830 0x7f6e6c106f00 secure :-1 s=READY pgs=1206 cs=0 l=1 rev1=1 crypto rx=0x7f6e54004940 tx=0x7f6e54009e30 comp rx=0 tx=0).stop 2024-04-12T21:47:15.226 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:15.224+0000 7f6e72000700 1 -- 172.21.15.18:0/728124808 shutdown_connections 2024-04-12T21:47:15.226 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:15.224+0000 7f6e72000700 1 --2- 172.21.15.18:0/728124808 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6e6c0fe830 0x7f6e6c106f00 unknown :-1 s=CLOSED pgs=1206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:47:15.226 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:15.224+0000 7f6e72000700 1 --2- 172.21.15.18:0/728124808 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6e6c0fde90 0x7f6e6c0fe260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:47:15.226 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:15.224+0000 7f6e72000700 1 -- 172.21.15.18:0/728124808 >> 172.21.15.18:0/728124808 conn(0x7f6e6c072bc0 msgr2=0x7f6e6c072fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:47:15.226 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:15.224+0000 7f6e72000700 1 -- 172.21.15.18:0/728124808 shutdown_connections 2024-04-12T21:47:15.226 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:15.224+0000 7f6e72000700 1 -- 172.21.15.18:0/728124808 wait complete. 2024-04-12T21:47:15.227 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:15.225+0000 7f6e72000700 1 Processor -- start 2024-04-12T21:47:15.228 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:15.225+0000 7f6e72000700 1 -- start start 2024-04-12T21:47:15.228 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:15.225+0000 7f6e72000700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6e6c0fde90 0x7f6e6c115d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:47:15.228 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:15.225+0000 7f6e72000700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6e6c0fe830 0x7f6e6c116240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:47:15.228 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:15.225+0000 7f6e72000700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e6c116890 con 0x7f6e6c0fe830 2024-04-12T21:47:15.228 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:15.225+0000 7f6e72000700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e6c10fc60 con 0x7f6e6c0fde90 2024-04-12T21:47:15.228 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:15.226+0000 7f6e63e00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6e6c0fe830 0x7f6e6c116240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:47:15.228 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:15.226+0000 7f6e63e00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6e6c0fe830 0x7f6e6c116240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:60818/0 (socket says 172.21.15.18:60818) 2024-04-12T21:47:15.228 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:15.226+0000 7f6e63e00700 1 -- 172.21.15.18:0/3512909730 learned_addr learned my addr 172.21.15.18:0/3512909730 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:47:15.228 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:15.226+0000 7f6e63e00700 1 -- 172.21.15.18:0/3512909730 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6e6c0fde90 msgr2=0x7f6e6c115d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:47:15.228 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:15.226+0000 7f6e63e00700 1 --2- 172.21.15.18:0/3512909730 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6e6c0fde90 0x7f6e6c115d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:47:15.228 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:15.226+0000 7f6e63e00700 1 -- 172.21.15.18:0/3512909730 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e54005040 con 0x7f6e6c0fe830 2024-04-12T21:47:15.229 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:15.226+0000 7f6e63e00700 1 --2- 172.21.15.18:0/3512909730 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6e6c0fe830 0x7f6e6c116240 secure :-1 s=READY pgs=1207 cs=0 l=1 rev1=1 crypto rx=0x7f6e54000c00 tx=0x7f6e54009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:47:15.229 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:15.227+0000 7f6e63400700 1 -- 172.21.15.18:0/3512909730 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6e54021070 con 0x7f6e6c0fe830 2024-04-12T21:47:15.229 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:15.227+0000 7f6e63400700 1 -- 172.21.15.18:0/3512909730 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f6e54013cd0 con 0x7f6e6c0fe830 2024-04-12T21:47:15.229 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:15.227+0000 7f6e72000700 1 -- 172.21.15.18:0/3512909730 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6e6c10fee0 con 0x7f6e6c0fe830 2024-04-12T21:47:15.229 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:15.227+0000 7f6e63400700 1 -- 172.21.15.18:0/3512909730 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6e5400e070 con 0x7f6e6c0fe830 2024-04-12T21:47:15.229 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:15.227+0000 7f6e72000700 1 -- 172.21.15.18:0/3512909730 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6e6c110420 con 0x7f6e6c0fe830 2024-04-12T21:47:15.230 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:15.228+0000 7f6e72000700 1 -- 172.21.15.18:0/3512909730 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6e6c106a20 con 0x7f6e6c0fe830 2024-04-12T21:47:15.230 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:15.229+0000 7f6e63400700 1 -- 172.21.15.18:0/3512909730 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7f6e5401ca60 con 0x7f6e6c0fe830 2024-04-12T21:47:15.234 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:15.230+0000 7f6e63400700 1 --2- 172.21.15.18:0/3512909730 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f6e5006ec10 0x7f6e500710c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:47:15.234 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:15.230+0000 7f6e63400700 1 -- 172.21.15.18:0/3512909730 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 6757+0+0 (secure 0 0 0) 0x7f6e5408fd40 con 0x7f6e6c0fe830 2024-04-12T21:47:15.234 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:15.230+0000 7f6e6b400700 1 --2- 172.21.15.18:0/3512909730 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f6e5006ec10 0x7f6e500710c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:47:15.234 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:15.231+0000 7f6e6b400700 1 --2- 172.21.15.18:0/3512909730 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f6e5006ec10 0x7f6e500710c0 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f6e5c009fd0 tx=0x7f6e5c00b040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:47:15.236 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:15.235+0000 7f6e63400700 1 -- 172.21.15.18:0/3512909730 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f6e5405c570 con 0x7f6e6c0fe830 2024-04-12T21:47:15.284 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:47:14 smithi186 ceph-mon[86379]: from='client.36468 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:47:15.529 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:15.527+0000 7f6e72000700 1 -- 172.21.15.18:0/3512909730 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f6e6c111350 con 0x7f6e6c0fe830 2024-04-12T21:47:15.530 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:15.528+0000 7f6e63400700 1 -- 172.21.15.18:0/3512909730 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 33 v33) v1 ==== 76+0+1876 (secure 0 0 0) 0x7f6e5405c100 con 0x7f6e6c0fe830 2024-04-12T21:47:15.531 INFO:teuthology.orchestra.run.smithi018.stdout:e33 2024-04-12T21:47:15.531 INFO:teuthology.orchestra.run.smithi018.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-12T21:47:15.531 INFO:teuthology.orchestra.run.smithi018.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-12T21:47:15.531 INFO:teuthology.orchestra.run.smithi018.stdout:legacy client fscid: 1 2024-04-12T21:47:15.531 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-12T21:47:15.531 INFO:teuthology.orchestra.run.smithi018.stdout:Filesystem 'cephfs' (1) 2024-04-12T21:47:15.531 INFO:teuthology.orchestra.run.smithi018.stdout:fs_name cephfs 2024-04-12T21:47:15.531 INFO:teuthology.orchestra.run.smithi018.stdout:epoch 24 2024-04-12T21:47:15.532 INFO:teuthology.orchestra.run.smithi018.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-04-12T21:47:15.532 INFO:teuthology.orchestra.run.smithi018.stdout:created 2024-04-12T20:51:57.362850+0000 2024-04-12T21:47:15.532 INFO:teuthology.orchestra.run.smithi018.stdout:modified 2024-04-12T21:06:59.717190+0000 2024-04-12T21:47:15.532 INFO:teuthology.orchestra.run.smithi018.stdout:tableserver 0 2024-04-12T21:47:15.532 INFO:teuthology.orchestra.run.smithi018.stdout:root 0 2024-04-12T21:47:15.532 INFO:teuthology.orchestra.run.smithi018.stdout:session_timeout 60 2024-04-12T21:47:15.532 INFO:teuthology.orchestra.run.smithi018.stdout:session_autoclose 300 2024-04-12T21:47:15.532 INFO:teuthology.orchestra.run.smithi018.stdout:max_file_size 1099511627776 2024-04-12T21:47:15.532 INFO:teuthology.orchestra.run.smithi018.stdout:max_xattr_size 65536 2024-04-12T21:47:15.532 INFO:teuthology.orchestra.run.smithi018.stdout:required_client_features {} 2024-04-12T21:47:15.532 INFO:teuthology.orchestra.run.smithi018.stdout:last_failure 0 2024-04-12T21:47:15.532 INFO:teuthology.orchestra.run.smithi018.stdout:last_failure_osd_epoch 100 2024-04-12T21:47:15.532 INFO:teuthology.orchestra.run.smithi018.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-12T21:47:15.532 INFO:teuthology.orchestra.run.smithi018.stdout:max_mds 2 2024-04-12T21:47:15.533 INFO:teuthology.orchestra.run.smithi018.stdout:in 0,1 2024-04-12T21:47:15.533 INFO:teuthology.orchestra.run.smithi018.stdout:up {} 2024-04-12T21:47:15.533 INFO:teuthology.orchestra.run.smithi018.stdout:failed 0,1 2024-04-12T21:47:15.533 INFO:teuthology.orchestra.run.smithi018.stdout:damaged 2024-04-12T21:47:15.533 INFO:teuthology.orchestra.run.smithi018.stdout:stopped 2024-04-12T21:47:15.533 INFO:teuthology.orchestra.run.smithi018.stdout:data_pools [3] 2024-04-12T21:47:15.533 INFO:teuthology.orchestra.run.smithi018.stdout:metadata_pool 2 2024-04-12T21:47:15.533 INFO:teuthology.orchestra.run.smithi018.stdout:inline_data disabled 2024-04-12T21:47:15.533 INFO:teuthology.orchestra.run.smithi018.stdout:balancer 2024-04-12T21:47:15.533 INFO:teuthology.orchestra.run.smithi018.stdout:bal_rank_mask -1 2024-04-12T21:47:15.533 INFO:teuthology.orchestra.run.smithi018.stdout:standby_count_wanted 1 2024-04-12T21:47:15.533 INFO:teuthology.orchestra.run.smithi018.stdout:qdb_cluster leader: 0 members: 2024-04-12T21:47:15.533 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-12T21:47:15.533 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-12T21:47:15.534 INFO:teuthology.orchestra.run.smithi018.stdout:Standby daemons: 2024-04-12T21:47:15.534 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-12T21:47:15.534 INFO:teuthology.orchestra.run.smithi018.stdout:[mds.cephfs.smithi018.acqbsc{-1:34436} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.18:6828/755179764,v1:172.21.15.18:6829/755179764] compat {c=[1],r=[1],i=[1fff]}] 2024-04-12T21:47:15.534 INFO:teuthology.orchestra.run.smithi018.stdout:[mds.cephfs.smithi018.szjluo{-1:34468} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.18:6826/2407992653,v1:172.21.15.18:6827/2407992653] compat {c=[1],r=[1],i=[1fff]}] 2024-04-12T21:47:15.534 INFO:teuthology.orchestra.run.smithi018.stdout:[mds.cephfs.smithi186.jdllyn{-1:44595} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.186:6824/3523444202,v1:172.21.15.186:6825/3523444202] compat {c=[1],r=[1],i=[1fff]}] 2024-04-12T21:47:15.534 INFO:teuthology.orchestra.run.smithi018.stdout:[mds.cephfs.smithi186.wjnafs{-1:44875} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.186:6826/3122443766,v1:172.21.15.186:6827/3122443766] compat {c=[1],r=[1],i=[1fff]}] 2024-04-12T21:47:15.535 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:15.531+0000 7f6e72000700 1 -- 172.21.15.18:0/3512909730 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f6e5006ec10 msgr2=0x7f6e500710c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:47:15.535 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:15.531+0000 7f6e72000700 1 --2- 172.21.15.18:0/3512909730 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f6e5006ec10 0x7f6e500710c0 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f6e5c009fd0 tx=0x7f6e5c00b040 comp rx=0 tx=0).stop 2024-04-12T21:47:15.535 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:15.531+0000 7f6e72000700 1 -- 172.21.15.18:0/3512909730 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6e6c0fe830 msgr2=0x7f6e6c116240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:47:15.535 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:15.531+0000 7f6e72000700 1 --2- 172.21.15.18:0/3512909730 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6e6c0fe830 0x7f6e6c116240 secure :-1 s=READY pgs=1207 cs=0 l=1 rev1=1 crypto rx=0x7f6e54000c00 tx=0x7f6e54009e30 comp rx=0 tx=0).stop 2024-04-12T21:47:15.535 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:15.531+0000 7f6e72000700 1 -- 172.21.15.18:0/3512909730 shutdown_connections 2024-04-12T21:47:15.535 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:15.531+0000 7f6e72000700 1 --2- 172.21.15.18:0/3512909730 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f6e5006ec10 0x7f6e500710c0 unknown :-1 s=CLOSED pgs=641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:47:15.535 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:15.531+0000 7f6e72000700 1 --2- 172.21.15.18:0/3512909730 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6e6c0fde90 0x7f6e6c115d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:47:15.536 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:15.531+0000 7f6e72000700 1 --2- 172.21.15.18:0/3512909730 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6e6c0fe830 0x7f6e6c116240 unknown :-1 s=CLOSED pgs=1207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:47:15.536 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:15.531+0000 7f6e72000700 1 -- 172.21.15.18:0/3512909730 >> 172.21.15.18:0/3512909730 conn(0x7f6e6c072bc0 msgr2=0x7f6e6c0fb530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:47:15.536 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:15.532+0000 7f6e72000700 1 -- 172.21.15.18:0/3512909730 shutdown_connections 2024-04-12T21:47:15.536 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:15.532+0000 7f6e72000700 1 -- 172.21.15.18:0/3512909730 wait complete. 2024-04-12T21:47:15.536 INFO:teuthology.orchestra.run.smithi018.stderr:dumped fsmap epoch 33 2024-04-12T21:47:15.689 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:15.687+0000 7fdbbea00700 1 -- 172.21.15.18:0/225151621 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fdbb8109dd0 msgr2=0x7fdbb8103440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:47:15.689 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:15.687+0000 7fdbbea00700 1 --2- 172.21.15.18:0/225151621 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fdbb8109dd0 0x7fdbb8103440 secure :-1 s=READY pgs=1208 cs=0 l=1 rev1=1 crypto rx=0x7fdba4004940 tx=0x7fdba4009e30 comp rx=0 tx=0).stop 2024-04-12T21:47:15.690 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:15.687+0000 7fdbbea00700 1 -- 172.21.15.18:0/225151621 shutdown_connections 2024-04-12T21:47:15.690 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:15.687+0000 7fdbbea00700 1 --2- 172.21.15.18:0/225151621 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fdbb8109dd0 0x7fdbb8103440 unknown :-1 s=CLOSED pgs=1208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:47:15.690 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:15.687+0000 7fdbbea00700 1 --2- 172.21.15.18:0/225151621 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdbb8109430 0x7fdbb8109800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:47:15.690 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:15.687+0000 7fdbbea00700 1 -- 172.21.15.18:0/225151621 >> 172.21.15.18:0/225151621 conn(0x7fdbb8072c00 msgr2=0x7fdbb8075010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:47:15.690 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:15.687+0000 7fdbbea00700 1 -- 172.21.15.18:0/225151621 shutdown_connections 2024-04-12T21:47:15.690 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:15.688+0000 7fdbbea00700 1 -- 172.21.15.18:0/225151621 wait complete. 2024-04-12T21:47:15.691 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:15.688+0000 7fdbbea00700 1 Processor -- start 2024-04-12T21:47:15.691 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:15.689+0000 7fdbbea00700 1 -- start start 2024-04-12T21:47:15.691 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:15.689+0000 7fdbbea00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fdbb8109430 0x7fdbb8119f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:47:15.691 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:15.689+0000 7fdbbea00700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdbb8109dd0 0x7fdbb811a4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:47:15.691 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:15.689+0000 7fdbbea00700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdbb811ab10 con 0x7fdbb8109430 2024-04-12T21:47:15.691 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:15.689+0000 7fdbbea00700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdbb81140f0 con 0x7fdbb8109dd0 2024-04-12T21:47:15.692 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:15.689+0000 7fdbb7e00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fdbb8109430 0x7fdbb8119f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:47:15.692 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:15.689+0000 7fdbb7e00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fdbb8109430 0x7fdbb8119f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:60826/0 (socket says 172.21.15.18:60826) 2024-04-12T21:47:15.692 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:15.689+0000 7fdbb7e00700 1 -- 172.21.15.18:0/1325487831 learned_addr learned my addr 172.21.15.18:0/1325487831 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:47:15.692 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:15.690+0000 7fdbb7400700 1 --2- 172.21.15.18:0/1325487831 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdbb8109dd0 0x7fdbb811a4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:47:15.693 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:15.690+0000 7fdbb7e00700 1 -- 172.21.15.18:0/1325487831 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdbb8109dd0 msgr2=0x7fdbb811a4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:47:15.693 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:15.690+0000 7fdbb7e00700 1 --2- 172.21.15.18:0/1325487831 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdbb8109dd0 0x7fdbb811a4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:47:15.693 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:15.690+0000 7fdbb7e00700 1 -- 172.21.15.18:0/1325487831 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdba4005040 con 0x7fdbb8109430 2024-04-12T21:47:15.693 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:15.690+0000 7fdbb7e00700 1 --2- 172.21.15.18:0/1325487831 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fdbb8109430 0x7fdbb8119f80 secure :-1 s=READY pgs=1209 cs=0 l=1 rev1=1 crypto rx=0x7fdba800c830 tx=0x7fdba8004470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:47:15.693 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:15.691+0000 7fdbafe00700 1 -- 172.21.15.18:0/1325487831 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdba800a760 con 0x7fdbb8109430 2024-04-12T21:47:15.693 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:15.691+0000 7fdbbea00700 1 -- 172.21.15.18:0/1325487831 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdbb81143d0 con 0x7fdbb8109430 2024-04-12T21:47:15.693 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:15.691+0000 7fdbafe00700 1 -- 172.21.15.18:0/1325487831 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fdba800ad90 con 0x7fdbb8109430 2024-04-12T21:47:15.693 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:15.691+0000 7fdbafe00700 1 -- 172.21.15.18:0/1325487831 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdba80113a0 con 0x7fdbb8109430 2024-04-12T21:47:15.693 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:15.691+0000 7fdbbea00700 1 -- 172.21.15.18:0/1325487831 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdbb8114990 con 0x7fdbb8109430 2024-04-12T21:47:15.694 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:15.692+0000 7fdbbea00700 1 -- 172.21.15.18:0/1325487831 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdbb8104b80 con 0x7fdbb8109430 2024-04-12T21:47:15.694 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:15.693+0000 7fdbafe00700 1 -- 172.21.15.18:0/1325487831 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7fdba8012400 con 0x7fdbb8109430 2024-04-12T21:47:15.696 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:15.694+0000 7fdbafe00700 1 --2- 172.21.15.18:0/1325487831 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fdb9c06ec00 0x7fdb9c0710b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:47:15.696 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:15.694+0000 7fdbafe00700 1 -- 172.21.15.18:0/1325487831 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 6757+0+0 (secure 0 0 0) 0x7fdba808e900 con 0x7fdbb8109430 2024-04-12T21:47:15.696 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:15.695+0000 7fdbb7400700 1 --2- 172.21.15.18:0/1325487831 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fdb9c06ec00 0x7fdb9c0710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:47:15.697 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:15.695+0000 7fdbb7400700 1 --2- 172.21.15.18:0/1325487831 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fdb9c06ec00 0x7fdb9c0710b0 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7fdba400b020 tx=0x7fdba4009e30 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:47:15.701 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:15.700+0000 7fdbafe00700 1 -- 172.21.15.18:0/1325487831 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7fdba805b1f0 con 0x7fdbb8109430 2024-04-12T21:47:15.945 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:15.943+0000 7fdbbea00700 1 -- 172.21.15.18:0/1325487831 --> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fdbb8115640 con 0x7fdb9c06ec00 2024-04-12T21:47:15.945 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:47:15 smithi018 ceph-mon[112590]: from='client.36472 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:47:15.945 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:47:15 smithi018 ceph-mon[112590]: pgmap v1427: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:47:15.946 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:47:15 smithi018 ceph-mon[112590]: from='client.36476 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:47:15.946 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:47:15 smithi018 ceph-mon[112590]: from='client.? 172.21.15.18:0/609112026' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T21:47:15.946 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:47:15 smithi018 ceph-mon[112590]: from='client.? 172.21.15.18:0/3512909730' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-12T21:47:15.946 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:15.945+0000 7fdbafe00700 1 -- 172.21.15.18:0/1325487831 <== mgr.34102 v2:172.21.15.18:6800/3812891919 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7fdbb8115640 con 0x7fdb9c06ec00 2024-04-12T21:47:15.947 INFO:teuthology.orchestra.run.smithi018.stdout:{ 2024-04-12T21:47:15.947 INFO:teuthology.orchestra.run.smithi018.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:f4452def3be4d350f4a8e5feaf7ebb1511be9298fe68478ba7a2d6fbd7d69a63", 2024-04-12T21:47:15.947 INFO:teuthology.orchestra.run.smithi018.stdout: "in_progress": true, 2024-04-12T21:47:15.947 INFO:teuthology.orchestra.run.smithi018.stdout: "which": "Upgrading all daemon types on all hosts", 2024-04-12T21:47:15.947 INFO:teuthology.orchestra.run.smithi018.stdout: "services_complete": [ 2024-04-12T21:47:15.947 INFO:teuthology.orchestra.run.smithi018.stdout: "mgr", 2024-04-12T21:47:15.947 INFO:teuthology.orchestra.run.smithi018.stdout: "mds", 2024-04-12T21:47:15.947 INFO:teuthology.orchestra.run.smithi018.stdout: "crash", 2024-04-12T21:47:15.947 INFO:teuthology.orchestra.run.smithi018.stdout: "osd", 2024-04-12T21:47:15.947 INFO:teuthology.orchestra.run.smithi018.stdout: "mon" 2024-04-12T21:47:15.947 INFO:teuthology.orchestra.run.smithi018.stdout: ], 2024-04-12T21:47:15.947 INFO:teuthology.orchestra.run.smithi018.stdout: "progress": "16/21 daemons upgraded", 2024-04-12T21:47:15.948 INFO:teuthology.orchestra.run.smithi018.stdout: "message": "Currently upgrading mds daemons", 2024-04-12T21:47:15.948 INFO:teuthology.orchestra.run.smithi018.stdout: "is_paused": false 2024-04-12T21:47:15.948 INFO:teuthology.orchestra.run.smithi018.stdout:} 2024-04-12T21:47:15.950 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:15.948+0000 7fdbbea00700 1 -- 172.21.15.18:0/1325487831 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fdb9c06ec00 msgr2=0x7fdb9c0710b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:47:15.950 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:15.948+0000 7fdbbea00700 1 --2- 172.21.15.18:0/1325487831 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fdb9c06ec00 0x7fdb9c0710b0 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7fdba400b020 tx=0x7fdba4009e30 comp rx=0 tx=0).stop 2024-04-12T21:47:15.950 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:15.948+0000 7fdbbea00700 1 -- 172.21.15.18:0/1325487831 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fdbb8109430 msgr2=0x7fdbb8119f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:47:15.950 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:15.948+0000 7fdbbea00700 1 --2- 172.21.15.18:0/1325487831 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fdbb8109430 0x7fdbb8119f80 secure :-1 s=READY pgs=1209 cs=0 l=1 rev1=1 crypto rx=0x7fdba800c830 tx=0x7fdba8004470 comp rx=0 tx=0).stop 2024-04-12T21:47:15.950 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:15.948+0000 7fdbbea00700 1 -- 172.21.15.18:0/1325487831 shutdown_connections 2024-04-12T21:47:15.950 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:15.948+0000 7fdbbea00700 1 --2- 172.21.15.18:0/1325487831 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fdb9c06ec00 0x7fdb9c0710b0 unknown :-1 s=CLOSED pgs=642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:47:15.950 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:15.948+0000 7fdbbea00700 1 --2- 172.21.15.18:0/1325487831 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fdbb8109430 0x7fdbb8119f80 unknown :-1 s=CLOSED pgs=1209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:47:15.950 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:15.948+0000 7fdbbea00700 1 --2- 172.21.15.18:0/1325487831 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdbb8109dd0 0x7fdbb811a4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:47:15.950 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:15.948+0000 7fdbbea00700 1 -- 172.21.15.18:0/1325487831 >> 172.21.15.18:0/1325487831 conn(0x7fdbb8072c00 msgr2=0x7fdbb8074c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:47:15.950 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:15.948+0000 7fdbbea00700 1 -- 172.21.15.18:0/1325487831 shutdown_connections 2024-04-12T21:47:15.951 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:15.948+0000 7fdbbea00700 1 -- 172.21.15.18:0/1325487831 wait complete. 2024-04-12T21:47:16.106 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:16.104+0000 7fc2d0000700 1 -- 172.21.15.18:0/740220683 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc2c80fbc80 msgr2=0x7fc2c80fc050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:47:16.107 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:16.104+0000 7fc2d0000700 1 --2- 172.21.15.18:0/740220683 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc2c80fbc80 0x7fc2c80fc050 secure :-1 s=READY pgs=1210 cs=0 l=1 rev1=1 crypto rx=0x7fc2bc004940 tx=0x7fc2bc009e30 comp rx=0 tx=0).stop 2024-04-12T21:47:16.107 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:16.105+0000 7fc2d0000700 1 -- 172.21.15.18:0/740220683 shutdown_connections 2024-04-12T21:47:16.107 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:16.105+0000 7fc2d0000700 1 --2- 172.21.15.18:0/740220683 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc2c80fc620 0x7fc2c8100cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:47:16.107 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:16.105+0000 7fc2d0000700 1 --2- 172.21.15.18:0/740220683 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc2c80fbc80 0x7fc2c80fc050 unknown :-1 s=CLOSED pgs=1210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:47:16.107 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:16.105+0000 7fc2d0000700 1 -- 172.21.15.18:0/740220683 >> 172.21.15.18:0/740220683 conn(0x7fc2c8072bc0 msgr2=0x7fc2c8072fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:47:16.107 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:16.105+0000 7fc2d0000700 1 -- 172.21.15.18:0/740220683 shutdown_connections 2024-04-12T21:47:16.107 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:16.105+0000 7fc2d0000700 1 -- 172.21.15.18:0/740220683 wait complete. 2024-04-12T21:47:16.108 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:16.106+0000 7fc2d0000700 1 Processor -- start 2024-04-12T21:47:16.108 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:16.106+0000 7fc2d0000700 1 -- start start 2024-04-12T21:47:16.108 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:16.106+0000 7fc2d0000700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc2c80fbc80 0x7fc2c811a1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:47:16.108 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:16.106+0000 7fc2d0000700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc2c80fc620 0x7fc2c811a710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:47:16.108 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:16.106+0000 7fc2d0000700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2c811ad60 con 0x7fc2c80fc620 2024-04-12T21:47:16.108 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:16.106+0000 7fc2d0000700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2c8114130 con 0x7fc2c80fbc80 2024-04-12T21:47:16.108 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:16.107+0000 7fc2ccc00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc2c80fc620 0x7fc2c811a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:47:16.109 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:16.107+0000 7fc2ccc00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc2c80fc620 0x7fc2c811a710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:60838/0 (socket says 172.21.15.18:60838) 2024-04-12T21:47:16.109 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:16.107+0000 7fc2ccc00700 1 -- 172.21.15.18:0/1827021894 learned_addr learned my addr 172.21.15.18:0/1827021894 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:47:16.110 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:16.107+0000 7fc2cd600700 1 --2- 172.21.15.18:0/1827021894 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc2c80fbc80 0x7fc2c811a1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:47:16.110 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:16.107+0000 7fc2ccc00700 1 -- 172.21.15.18:0/1827021894 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc2c80fbc80 msgr2=0x7fc2c811a1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:47:16.110 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:16.107+0000 7fc2ccc00700 1 --2- 172.21.15.18:0/1827021894 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc2c80fbc80 0x7fc2c811a1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:47:16.110 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:16.107+0000 7fc2ccc00700 1 -- 172.21.15.18:0/1827021894 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2bc005040 con 0x7fc2c80fc620 2024-04-12T21:47:16.110 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:16.108+0000 7fc2ccc00700 1 --2- 172.21.15.18:0/1827021894 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc2c80fc620 0x7fc2c811a710 secure :-1 s=READY pgs=1211 cs=0 l=1 rev1=1 crypto rx=0x7fc2b4002a50 tx=0x7fc2b40046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:47:16.110 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:16.108+0000 7fc2c5600700 1 -- 172.21.15.18:0/1827021894 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc2b4007cc0 con 0x7fc2c80fc620 2024-04-12T21:47:16.110 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:16.108+0000 7fc2c5600700 1 -- 172.21.15.18:0/1827021894 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc2b400f040 con 0x7fc2c80fc620 2024-04-12T21:47:16.110 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:16.108+0000 7fc2c5600700 1 -- 172.21.15.18:0/1827021894 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc2b4009640 con 0x7fc2c80fc620 2024-04-12T21:47:16.110 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:16.108+0000 7fc2d0000700 1 -- 172.21.15.18:0/1827021894 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc2c8114410 con 0x7fc2c80fc620 2024-04-12T21:47:16.110 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:16.108+0000 7fc2d0000700 1 -- 172.21.15.18:0/1827021894 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc2c8114950 con 0x7fc2c80fc620 2024-04-12T21:47:16.111 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:16.109+0000 7fc2d0000700 1 -- 172.21.15.18:0/1827021894 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc2c8065e30 con 0x7fc2c80fc620 2024-04-12T21:47:16.112 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:16.110+0000 7fc2c5600700 1 -- 172.21.15.18:0/1827021894 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7fc2b40097a0 con 0x7fc2c80fc620 2024-04-12T21:47:16.113 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:16.112+0000 7fc2c5600700 1 --2- 172.21.15.18:0/1827021894 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fc2b006ec00 0x7fc2b00710b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:47:16.113 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:16.112+0000 7fc2c5600700 1 -- 172.21.15.18:0/1827021894 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 6757+0+0 (secure 0 0 0) 0x7fc2b408e760 con 0x7fc2c80fc620 2024-04-12T21:47:16.113 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:16.112+0000 7fc2cd600700 1 --2- 172.21.15.18:0/1827021894 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fc2b006ec00 0x7fc2b00710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:47:16.114 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:16.113+0000 7fc2cd600700 1 --2- 172.21.15.18:0/1827021894 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fc2b006ec00 0x7fc2b00710b0 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7fc2bc004f90 tx=0x7fc2bc009e30 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:47:16.118 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:16.116+0000 7fc2c5600700 1 -- 172.21.15.18:0/1827021894 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7fc2b405af10 con 0x7fc2c80fc620 2024-04-12T21:47:16.284 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:47:15 smithi186 ceph-mon[86379]: from='client.36472 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:47:16.284 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:47:15 smithi186 ceph-mon[86379]: pgmap v1427: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:47:16.284 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:47:15 smithi186 ceph-mon[86379]: from='client.36476 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:47:16.284 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:47:15 smithi186 ceph-mon[86379]: from='client.? 172.21.15.18:0/609112026' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T21:47:16.284 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:47:15 smithi186 ceph-mon[86379]: from='client.? 172.21.15.18:0/3512909730' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-12T21:47:16.440 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:16.438+0000 7fc2d0000700 1 -- 172.21.15.18:0/1827021894 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7fc2c805f5f0 con 0x7fc2c80fc620 2024-04-12T21:47:16.440 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:16.439+0000 7fc2c5600700 1 -- 172.21.15.18:0/1827021894 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+4303 (secure 0 0 0) 0x7fc2b405aaa0 con 0x7fc2c80fc620 2024-04-12T21:47:16.441 INFO:teuthology.orchestra.run.smithi018.stdout:HEALTH_ERR failed to probe daemons or devices; 1 filesystem is degraded; 1 filesystem is offline 2024-04-12T21:47:16.441 INFO:teuthology.orchestra.run.smithi018.stdout:[WRN] CEPHADM_REFRESH_FAILED: failed to probe daemons or devices 2024-04-12T21:47:16.441 INFO:teuthology.orchestra.run.smithi018.stdout: host smithi186 `cephadm ceph-volume` failed: cephadm exited with an error code: 1, stderr: Inferring config /var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/mon.smithi186/config 2024-04-12T21:47:16.441 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/podman: timeout after 895 seconds 2024-04-12T21:47:16.442 INFO:teuthology.orchestra.run.smithi018.stdout:Non-zero exit code 124 from /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:f4452def3be4d350f4a8e5feaf7ebb1511be9298fe68478ba7a2d6fbd7d69a63 -e NODE_NAME=smithi186 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968:/var/run/ceph:z -v /var/log/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968:/var/log/ceph:z -v /var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmptgom53f9:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:f4452def3be4d350f4a8e5feaf7ebb1511be9298fe68478ba7a2d6fbd7d69a63 inventory --format=json-pretty --filter-for-batch 2024-04-12T21:47:16.442 INFO:teuthology.orchestra.run.smithi018.stdout:Traceback (most recent call last): 2024-04-12T21:47:16.442 INFO:teuthology.orchestra.run.smithi018.stdout: File "/usr/lib64/python3.9/runpy.py", line 197, in _run_module_as_main 2024-04-12T21:47:16.442 INFO:teuthology.orchestra.run.smithi018.stdout: return _run_code(code, main_globals, None, 2024-04-12T21:47:16.442 INFO:teuthology.orchestra.run.smithi018.stdout: File "/usr/lib64/python3.9/runpy.py", line 87, in _run_code 2024-04-12T21:47:16.442 INFO:teuthology.orchestra.run.smithi018.stdout: exec(code, run_globals) 2024-04-12T21:47:16.442 INFO:teuthology.orchestra.run.smithi018.stdout: File "/var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/cephadm.e44f3ed5e34e4ef52eedef233d1190bff004470d8e8005127b5312bff357fa7f/__main__.py", line 5618, in 2024-04-12T21:47:16.442 INFO:teuthology.orchestra.run.smithi018.stdout: File "/var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/cephadm.e44f3ed5e34e4ef52eedef233d1190bff004470d8e8005127b5312bff357fa7f/__main__.py", line 5606, in main 2024-04-12T21:47:16.442 INFO:teuthology.orchestra.run.smithi018.stdout: File "/var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/cephadm.e44f3ed5e34e4ef52eedef233d1190bff004470d8e8005127b5312bff357fa7f/__main__.py", line 408, in _infer_config 2024-04-12T21:47:16.442 INFO:teuthology.orchestra.run.smithi018.stdout: File "/var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/cephadm.e44f3ed5e34e4ef52eedef233d1190bff004470d8e8005127b5312bff357fa7f/__main__.py", line 323, in _infer_fsid 2024-04-12T21:47:16.442 INFO:teuthology.orchestra.run.smithi018.stdout: File "/var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/cephadm.e44f3ed5e34e4ef52eedef233d1190bff004470d8e8005127b5312bff357fa7f/__main__.py", line 436, in _infer_image 2024-04-12T21:47:16.442 INFO:teuthology.orchestra.run.smithi018.stdout: File "/var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/cephadm.e44f3ed5e34e4ef52eedef233d1190bff004470d8e8005127b5312bff357fa7f/__main__.py", line 310, in _validate_fsid 2024-04-12T21:47:16.442 INFO:teuthology.orchestra.run.smithi018.stdout: File "/var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/cephadm.e44f3ed5e34e4ef52eedef233d1190bff004470d8e8005127b5312bff357fa7f/__main__.py", line 3359, in command_ceph_volume 2024-04-12T21:47:16.442 INFO:teuthology.orchestra.run.smithi018.stdout: File "/var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/cephadm.e44f3ed5e34e4ef52eedef233d1190bff004470d8e8005127b5312bff357fa7f/cephadmlib/call_wrappers.py", line 310, in call_throws 2024-04-12T21:47:16.442 INFO:teuthology.orchestra.run.smithi018.stdout:RuntimeError: Failed command: /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:f4452def3be4d350f4a8e5feaf7ebb1511be9298fe68478ba7a2d6fbd7d69a63 -e NODE_NAME=smithi186 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968:/var/run/ceph:z -v /var/log/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968:/var/log/ceph:z -v /var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmptgom53f9:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:f4452def3be4d350f4a8e5feaf7ebb1511be9298fe68478ba7a2d6fbd7d69a63 inventory --format=json-pretty --filter-for-batch 2024-04-12T21:47:16.443 INFO:teuthology.orchestra.run.smithi018.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-04-12T21:47:16.443 INFO:teuthology.orchestra.run.smithi018.stdout: fs cephfs is degraded 2024-04-12T21:47:16.443 INFO:teuthology.orchestra.run.smithi018.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-04-12T21:47:16.443 INFO:teuthology.orchestra.run.smithi018.stdout: fs cephfs is offline because no MDS is active for it. 2024-04-12T21:47:16.444 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:16.441+0000 7fc2d0000700 1 -- 172.21.15.18:0/1827021894 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fc2b006ec00 msgr2=0x7fc2b00710b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:47:16.444 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:16.441+0000 7fc2d0000700 1 --2- 172.21.15.18:0/1827021894 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fc2b006ec00 0x7fc2b00710b0 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7fc2bc004f90 tx=0x7fc2bc009e30 comp rx=0 tx=0).stop 2024-04-12T21:47:16.444 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:16.441+0000 7fc2d0000700 1 -- 172.21.15.18:0/1827021894 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc2c80fc620 msgr2=0x7fc2c811a710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:47:16.444 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:16.441+0000 7fc2d0000700 1 --2- 172.21.15.18:0/1827021894 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc2c80fc620 0x7fc2c811a710 secure :-1 s=READY pgs=1211 cs=0 l=1 rev1=1 crypto rx=0x7fc2b4002a50 tx=0x7fc2b40046b0 comp rx=0 tx=0).stop 2024-04-12T21:47:16.444 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:16.441+0000 7fc2d0000700 1 -- 172.21.15.18:0/1827021894 shutdown_connections 2024-04-12T21:47:16.444 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:16.441+0000 7fc2d0000700 1 --2- 172.21.15.18:0/1827021894 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fc2b006ec00 0x7fc2b00710b0 unknown :-1 s=CLOSED pgs=643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:47:16.444 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:16.442+0000 7fc2d0000700 1 --2- 172.21.15.18:0/1827021894 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc2c80fbc80 0x7fc2c811a1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:47:16.444 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:16.442+0000 7fc2d0000700 1 --2- 172.21.15.18:0/1827021894 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc2c80fc620 0x7fc2c811a710 unknown :-1 s=CLOSED pgs=1211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:47:16.444 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:16.442+0000 7fc2d0000700 1 -- 172.21.15.18:0/1827021894 >> 172.21.15.18:0/1827021894 conn(0x7fc2c8072bc0 msgr2=0x7fc2c810aab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:47:16.444 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:16.442+0000 7fc2d0000700 1 -- 172.21.15.18:0/1827021894 shutdown_connections 2024-04-12T21:47:16.444 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:16.442+0000 7fc2d0000700 1 -- 172.21.15.18:0/1827021894 wait complete. 2024-04-12T21:47:17.105 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:47:16 smithi018 ceph-mon[112590]: from='client.36488 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:47:17.105 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:47:16 smithi018 ceph-mon[112590]: from='client.? 172.21.15.18:0/1827021894' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-12T21:47:17.286 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:47:16 smithi186 ceph-mon[86379]: from='client.36488 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:47:17.286 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:47:16 smithi186 ceph-mon[86379]: from='client.? 172.21.15.18:0/1827021894' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-12T21:47:18.105 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:47:17 smithi018 ceph-mon[112590]: pgmap v1428: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:47:18.284 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:47:17 smithi186 ceph-mon[86379]: pgmap v1428: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:47:20.105 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:47:19 smithi018 ceph-mon[112590]: pgmap v1429: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:47:20.284 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:47:19 smithi186 ceph-mon[86379]: pgmap v1429: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:47:22.105 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:47:21 smithi018 ceph-mon[112590]: pgmap v1430: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:47:22.284 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:47:21 smithi186 ceph-mon[86379]: pgmap v1430: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:47:24.105 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:47:23 smithi018 ceph-mon[112590]: pgmap v1431: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:47:24.284 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:47:23 smithi186 ceph-mon[86379]: pgmap v1431: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:47:26.286 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:47:25 smithi186 ceph-mon[86379]: pgmap v1432: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:47:26.286 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:47:25 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T21:47:26.355 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:47:25 smithi018 ceph-mon[112590]: pgmap v1432: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:47:26.355 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:47:25 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T21:47:28.284 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:47:27 smithi186 ceph-mon[86379]: pgmap v1433: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:47:28.355 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:47:27 smithi018 ceph-mon[112590]: pgmap v1433: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:47:30.284 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:47:29 smithi186 ceph-mon[86379]: pgmap v1434: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:47:30.355 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:47:29 smithi018 ceph-mon[112590]: pgmap v1434: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:47:32.284 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:47:31 smithi186 ceph-mon[86379]: pgmap v1435: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:47:32.355 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:47:31 smithi018 ceph-mon[112590]: pgmap v1435: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:47:34.284 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:47:33 smithi186 ceph-mon[86379]: pgmap v1436: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:47:34.355 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:47:33 smithi018 ceph-mon[112590]: pgmap v1436: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:47:36.284 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:47:35 smithi186 ceph-mon[86379]: pgmap v1437: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:47:36.355 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:47:35 smithi018 ceph-mon[112590]: pgmap v1437: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:47:38.284 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:47:37 smithi186 ceph-mon[86379]: pgmap v1438: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:47:38.355 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:47:37 smithi018 ceph-mon[112590]: pgmap v1438: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:47:40.284 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:47:39 smithi186 ceph-mon[86379]: pgmap v1439: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:47:40.355 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:47:39 smithi018 ceph-mon[112590]: pgmap v1439: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:47:41.284 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:47:40 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T21:47:41.355 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:47:40 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T21:47:42.284 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:47:41 smithi186 ceph-mon[86379]: pgmap v1440: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:47:42.355 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:47:41 smithi018 ceph-mon[112590]: pgmap v1440: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:47:44.284 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:47:43 smithi186 ceph-mon[86379]: pgmap v1441: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:47:44.355 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:47:43 smithi018 ceph-mon[112590]: pgmap v1441: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:47:46.284 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:47:45 smithi186 ceph-mon[86379]: pgmap v1442: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:47:46.355 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:47:45 smithi018 ceph-mon[112590]: pgmap v1442: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:47:46.619 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:46.618+0000 7f3ebf400700 1 -- 172.21.15.18:0/2490369202 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3eb8071f90 msgr2=0x7f3eb8070360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:47:46.620 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:46.618+0000 7f3ebf400700 1 --2- 172.21.15.18:0/2490369202 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3eb8071f90 0x7f3eb8070360 secure :-1 s=READY pgs=1212 cs=0 l=1 rev1=1 crypto rx=0x7f3eac004970 tx=0x7f3eac009e30 comp rx=0 tx=0).stop 2024-04-12T21:47:46.620 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:46.619+0000 7f3ebf400700 1 -- 172.21.15.18:0/2490369202 shutdown_connections 2024-04-12T21:47:46.620 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:46.619+0000 7f3ebf400700 1 --2- 172.21.15.18:0/2490369202 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3eb80708a0 0x7f3eb8070d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:47:46.620 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:46.619+0000 7f3ebf400700 1 --2- 172.21.15.18:0/2490369202 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3eb8071f90 0x7f3eb8070360 unknown :-1 s=CLOSED pgs=1212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:47:46.620 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:46.619+0000 7f3ebf400700 1 -- 172.21.15.18:0/2490369202 >> 172.21.15.18:0/2490369202 conn(0x7f3eb80f89b0 msgr2=0x7f3eb80fadc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:47:46.620 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:46.619+0000 7f3ebf400700 1 -- 172.21.15.18:0/2490369202 shutdown_connections 2024-04-12T21:47:46.620 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:46.619+0000 7f3ebf400700 1 -- 172.21.15.18:0/2490369202 wait complete. 2024-04-12T21:47:46.622 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:46.620+0000 7f3ebf400700 1 Processor -- start 2024-04-12T21:47:46.622 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:46.620+0000 7f3ebf400700 1 -- start start 2024-04-12T21:47:46.622 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:46.620+0000 7f3ebf400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3eb80708a0 0x7f3eb811a110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:47:46.622 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:46.620+0000 7f3ebf400700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3eb8071f90 0x7f3eb811a650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:47:46.622 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:46.620+0000 7f3ebf400700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3eb811aca0 con 0x7f3eb80708a0 2024-04-12T21:47:46.622 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:46.620+0000 7f3ebf400700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3eb8114120 con 0x7f3eb8071f90 2024-04-12T21:47:46.622 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:46.620+0000 7f3ebca00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3eb80708a0 0x7f3eb811a110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:47:46.622 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:46.621+0000 7f3ebca00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3eb80708a0 0x7f3eb811a110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:39734/0 (socket says 172.21.15.18:39734) 2024-04-12T21:47:46.622 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:46.621+0000 7f3ebca00700 1 -- 172.21.15.18:0/3614909221 learned_addr learned my addr 172.21.15.18:0/3614909221 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:47:46.622 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:46.621+0000 7f3ebca00700 1 -- 172.21.15.18:0/3614909221 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3eb8071f90 msgr2=0x7f3eb811a650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:47:46.622 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:46.621+0000 7f3ebca00700 1 --2- 172.21.15.18:0/3614909221 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3eb8071f90 0x7f3eb811a650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:47:46.622 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:46.621+0000 7f3ebca00700 1 -- 172.21.15.18:0/3614909221 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3eac005040 con 0x7f3eb80708a0 2024-04-12T21:47:46.623 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:46.621+0000 7f3ebca00700 1 --2- 172.21.15.18:0/3614909221 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3eb80708a0 0x7f3eb811a110 secure :-1 s=READY pgs=1213 cs=0 l=1 rev1=1 crypto rx=0x7f3eac004f50 tx=0x7f3eac009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:47:46.623 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:46.622+0000 7f3eb4c00700 1 -- 172.21.15.18:0/3614909221 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3eac00e040 con 0x7f3eb80708a0 2024-04-12T21:47:46.623 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:46.622+0000 7f3eb4c00700 1 -- 172.21.15.18:0/3614909221 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f3eac004290 con 0x7f3eb80708a0 2024-04-12T21:47:46.623 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:46.622+0000 7f3eb4c00700 1 -- 172.21.15.18:0/3614909221 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3eac01c630 con 0x7f3eb80708a0 2024-04-12T21:47:46.623 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:46.622+0000 7f3ebf400700 1 -- 172.21.15.18:0/3614909221 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3eb81143a0 con 0x7f3eb80708a0 2024-04-12T21:47:46.623 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:46.622+0000 7f3ebf400700 1 -- 172.21.15.18:0/3614909221 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3eb81148e0 con 0x7f3eb80708a0 2024-04-12T21:47:46.624 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:46.623+0000 7f3ebf400700 1 -- 172.21.15.18:0/3614909221 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3eb8071aa0 con 0x7f3eb80708a0 2024-04-12T21:47:46.625 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:46.624+0000 7f3eb4c00700 1 -- 172.21.15.18:0/3614909221 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7f3eac00bc10 con 0x7f3eb80708a0 2024-04-12T21:47:46.626 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:46.625+0000 7f3eb4c00700 1 --2- 172.21.15.18:0/3614909221 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f3ea006ec10 0x7f3ea00710c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:47:46.626 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:46.625+0000 7f3eb4c00700 1 -- 172.21.15.18:0/3614909221 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 6757+0+0 (secure 0 0 0) 0x7f3eac08fca0 con 0x7f3eb80708a0 2024-04-12T21:47:46.626 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:46.625+0000 7f3eb7e00700 1 --2- 172.21.15.18:0/3614909221 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f3ea006ec10 0x7f3ea00710c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:47:46.626 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:46.626+0000 7f3eb7e00700 1 --2- 172.21.15.18:0/3614909221 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f3ea006ec10 0x7f3ea00710c0 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f3eb810cf00 tx=0x7f3ea4009ce0 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:47:46.631 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:46.630+0000 7f3eb4c00700 1 -- 172.21.15.18:0/3614909221 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f3eac0589a0 con 0x7f3eb80708a0 2024-04-12T21:47:46.886 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:46.885+0000 7f3ebf400700 1 -- 172.21.15.18:0/3614909221 --> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f3eb8110540 con 0x7f3ea006ec10 2024-04-12T21:47:46.888 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:46.888+0000 7f3eb4c00700 1 -- 172.21.15.18:0/3614909221 <== mgr.34102 v2:172.21.15.18:6800/3812891919 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f3eb8110540 con 0x7f3ea006ec10 2024-04-12T21:47:46.891 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:46.891+0000 7f3ebf400700 1 -- 172.21.15.18:0/3614909221 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f3ea006ec10 msgr2=0x7f3ea00710c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:47:46.891 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:46.891+0000 7f3ebf400700 1 --2- 172.21.15.18:0/3614909221 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f3ea006ec10 0x7f3ea00710c0 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f3eb810cf00 tx=0x7f3ea4009ce0 comp rx=0 tx=0).stop 2024-04-12T21:47:46.891 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:46.891+0000 7f3ebf400700 1 -- 172.21.15.18:0/3614909221 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3eb80708a0 msgr2=0x7f3eb811a110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:47:46.891 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:46.891+0000 7f3ebf400700 1 --2- 172.21.15.18:0/3614909221 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3eb80708a0 0x7f3eb811a110 secure :-1 s=READY pgs=1213 cs=0 l=1 rev1=1 crypto rx=0x7f3eac004f50 tx=0x7f3eac009e30 comp rx=0 tx=0).stop 2024-04-12T21:47:46.892 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:46.891+0000 7f3ebf400700 1 -- 172.21.15.18:0/3614909221 shutdown_connections 2024-04-12T21:47:46.892 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:46.891+0000 7f3ebf400700 1 --2- 172.21.15.18:0/3614909221 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f3ea006ec10 0x7f3ea00710c0 unknown :-1 s=CLOSED pgs=644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:47:46.892 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:46.891+0000 7f3ebf400700 1 --2- 172.21.15.18:0/3614909221 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3eb80708a0 0x7f3eb811a110 unknown :-1 s=CLOSED pgs=1213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:47:46.892 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:46.891+0000 7f3ebf400700 1 --2- 172.21.15.18:0/3614909221 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3eb8071f90 0x7f3eb811a650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:47:46.892 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:46.891+0000 7f3ebf400700 1 -- 172.21.15.18:0/3614909221 >> 172.21.15.18:0/3614909221 conn(0x7f3eb80f89b0 msgr2=0x7f3eb80f9ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:47:46.892 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:46.891+0000 7f3ebf400700 1 -- 172.21.15.18:0/3614909221 shutdown_connections 2024-04-12T21:47:46.892 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:46.891+0000 7f3ebf400700 1 -- 172.21.15.18:0/3614909221 wait complete. 2024-04-12T21:47:46.908 INFO:teuthology.orchestra.run.smithi018.stdout:true 2024-04-12T21:47:47.049 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:47.048+0000 7fe40b200700 1 -- 172.21.15.18:0/1067907230 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe404109430 msgr2=0x7fe404109800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:47:47.049 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:47.048+0000 7fe40b200700 1 --2- 172.21.15.18:0/1067907230 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe404109430 0x7fe404109800 secure :-1 s=READY pgs=1214 cs=0 l=1 rev1=1 crypto rx=0x7fe3f4004940 tx=0x7fe3f4009e30 comp rx=0 tx=0).stop 2024-04-12T21:47:47.049 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:47.048+0000 7fe40b200700 1 -- 172.21.15.18:0/1067907230 shutdown_connections 2024-04-12T21:47:47.049 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:47.048+0000 7fe40b200700 1 --2- 172.21.15.18:0/1067907230 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe404109dd0 0x7fe404103440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:47:47.049 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:47.048+0000 7fe40b200700 1 --2- 172.21.15.18:0/1067907230 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe404109430 0x7fe404109800 unknown :-1 s=CLOSED pgs=1214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:47:47.050 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:47.048+0000 7fe40b200700 1 -- 172.21.15.18:0/1067907230 >> 172.21.15.18:0/1067907230 conn(0x7fe404072c00 msgr2=0x7fe404075010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:47:47.050 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:47.048+0000 7fe40b200700 1 -- 172.21.15.18:0/1067907230 shutdown_connections 2024-04-12T21:47:47.050 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:47.048+0000 7fe40b200700 1 -- 172.21.15.18:0/1067907230 wait complete. 2024-04-12T21:47:47.051 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:47.049+0000 7fe40b200700 1 Processor -- start 2024-04-12T21:47:47.051 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:47.049+0000 7fe40b200700 1 -- start start 2024-04-12T21:47:47.051 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:47.050+0000 7fe40b200700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe404109430 0x7fe404119f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:47:47.051 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:47.050+0000 7fe40b200700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe404109dd0 0x7fe40411a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:47:47.051 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:47.050+0000 7fe40b200700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe40411aad0 con 0x7fe404109430 2024-04-12T21:47:47.051 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:47.050+0000 7fe40b200700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe404114100 con 0x7fe404109dd0 2024-04-12T21:47:47.051 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:47.050+0000 7fe403e00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe404109430 0x7fe404119f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:47:47.052 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:47.050+0000 7fe403e00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe404109430 0x7fe404119f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:39760/0 (socket says 172.21.15.18:39760) 2024-04-12T21:47:47.052 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:47.050+0000 7fe403e00700 1 -- 172.21.15.18:0/91272860 learned_addr learned my addr 172.21.15.18:0/91272860 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:47:47.052 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:47.050+0000 7fe3fb400700 1 --2- 172.21.15.18:0/91272860 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe404109dd0 0x7fe40411a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:47:47.052 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:47.051+0000 7fe403e00700 1 -- 172.21.15.18:0/91272860 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe404109dd0 msgr2=0x7fe40411a480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:47:47.052 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:47.051+0000 7fe403e00700 1 --2- 172.21.15.18:0/91272860 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe404109dd0 0x7fe40411a480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:47:47.052 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:47.051+0000 7fe403e00700 1 -- 172.21.15.18:0/91272860 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3f4005040 con 0x7fe404109430 2024-04-12T21:47:47.053 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:47.051+0000 7fe403e00700 1 --2- 172.21.15.18:0/91272860 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe404109430 0x7fe404119f40 secure :-1 s=READY pgs=1215 cs=0 l=1 rev1=1 crypto rx=0x7fe3f4009150 tx=0x7fe3f4009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:47:47.053 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:47.051+0000 7fe400c00700 1 -- 172.21.15.18:0/91272860 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe3f400e040 con 0x7fe404109430 2024-04-12T21:47:47.053 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:47.051+0000 7fe400c00700 1 -- 172.21.15.18:0/91272860 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe3f4013420 con 0x7fe404109430 2024-04-12T21:47:47.053 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:47.051+0000 7fe400c00700 1 -- 172.21.15.18:0/91272860 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe3f401c630 con 0x7fe404109430 2024-04-12T21:47:47.053 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:47.051+0000 7fe40b200700 1 -- 172.21.15.18:0/91272860 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe404114380 con 0x7fe404109430 2024-04-12T21:47:47.053 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:47.052+0000 7fe40b200700 1 -- 172.21.15.18:0/91272860 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe404114940 con 0x7fe404109430 2024-04-12T21:47:47.053 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:47.053+0000 7fe40b200700 1 -- 172.21.15.18:0/91272860 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe404104b80 con 0x7fe404109430 2024-04-12T21:47:47.054 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:47.054+0000 7fe400c00700 1 -- 172.21.15.18:0/91272860 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7fe3f4013590 con 0x7fe404109430 2024-04-12T21:47:47.055 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:47.054+0000 7fe400c00700 1 --2- 172.21.15.18:0/91272860 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fe3e806ec00 0x7fe3e80710b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:47:47.055 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:47.055+0000 7fe400c00700 1 -- 172.21.15.18:0/91272860 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 6757+0+0 (secure 0 0 0) 0x7fe3f408fb40 con 0x7fe404109430 2024-04-12T21:47:47.055 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:47.055+0000 7fe3fb400700 1 --2- 172.21.15.18:0/91272860 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fe3e806ec00 0x7fe3e80710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:47:47.056 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:47.056+0000 7fe3fb400700 1 --2- 172.21.15.18:0/91272860 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fe3e806ec00 0x7fe3e80710b0 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7fe3ec007fd0 tx=0x7fe3ec009040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:47:47.060 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:47.059+0000 7fe400c00700 1 -- 172.21.15.18:0/91272860 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7fe3f405c430 con 0x7fe404109430 2024-04-12T21:47:47.312 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:47.311+0000 7fe40b200700 1 -- 172.21.15.18:0/91272860 --> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fe404115590 con 0x7fe3e806ec00 2024-04-12T21:47:47.314 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:47.313+0000 7fe400c00700 1 -- 172.21.15.18:0/91272860 <== mgr.34102 v2:172.21.15.18:6800/3812891919 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7fe404115590 con 0x7fe3e806ec00 2024-04-12T21:47:47.316 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:47.316+0000 7fe40b200700 1 -- 172.21.15.18:0/91272860 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fe3e806ec00 msgr2=0x7fe3e80710b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:47:47.316 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:47.316+0000 7fe40b200700 1 --2- 172.21.15.18:0/91272860 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fe3e806ec00 0x7fe3e80710b0 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7fe3ec007fd0 tx=0x7fe3ec009040 comp rx=0 tx=0).stop 2024-04-12T21:47:47.316 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:47.316+0000 7fe40b200700 1 -- 172.21.15.18:0/91272860 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe404109430 msgr2=0x7fe404119f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:47:47.317 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:47.316+0000 7fe40b200700 1 --2- 172.21.15.18:0/91272860 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe404109430 0x7fe404119f40 secure :-1 s=READY pgs=1215 cs=0 l=1 rev1=1 crypto rx=0x7fe3f4009150 tx=0x7fe3f4009e30 comp rx=0 tx=0).stop 2024-04-12T21:47:47.317 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:47.316+0000 7fe40b200700 1 -- 172.21.15.18:0/91272860 shutdown_connections 2024-04-12T21:47:47.317 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:47.316+0000 7fe40b200700 1 --2- 172.21.15.18:0/91272860 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fe3e806ec00 0x7fe3e80710b0 unknown :-1 s=CLOSED pgs=645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:47:47.317 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:47.316+0000 7fe40b200700 1 --2- 172.21.15.18:0/91272860 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe404109430 0x7fe404119f40 unknown :-1 s=CLOSED pgs=1215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:47:47.317 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:47.316+0000 7fe40b200700 1 --2- 172.21.15.18:0/91272860 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe404109dd0 0x7fe40411a480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:47:47.318 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:47.316+0000 7fe40b200700 1 -- 172.21.15.18:0/91272860 >> 172.21.15.18:0/91272860 conn(0x7fe404072c00 msgr2=0x7fe404074c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:47:47.318 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:47.317+0000 7fe40b200700 1 -- 172.21.15.18:0/91272860 shutdown_connections 2024-04-12T21:47:47.318 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:47.317+0000 7fe40b200700 1 -- 172.21.15.18:0/91272860 wait complete. 2024-04-12T21:47:47.468 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:47.466+0000 7f1ef2e00700 1 -- 172.21.15.18:0/2089089764 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1eec109dd0 msgr2=0x7f1eec103440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:47:47.468 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:47.466+0000 7f1ef2e00700 1 --2- 172.21.15.18:0/2089089764 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1eec109dd0 0x7f1eec103440 secure :-1 s=READY pgs=1216 cs=0 l=1 rev1=1 crypto rx=0x7f1ed8004940 tx=0x7f1ed8009e30 comp rx=0 tx=0).stop 2024-04-12T21:47:47.468 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:47.466+0000 7f1ef2e00700 1 -- 172.21.15.18:0/2089089764 shutdown_connections 2024-04-12T21:47:47.468 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:47.466+0000 7f1ef2e00700 1 --2- 172.21.15.18:0/2089089764 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1eec109dd0 0x7f1eec103440 unknown :-1 s=CLOSED pgs=1216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:47:47.468 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:47.466+0000 7f1ef2e00700 1 --2- 172.21.15.18:0/2089089764 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1eec109430 0x7f1eec109800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:47:47.468 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:47.466+0000 7f1ef2e00700 1 -- 172.21.15.18:0/2089089764 >> 172.21.15.18:0/2089089764 conn(0x7f1eec072c00 msgr2=0x7f1eec075010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:47:47.468 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:47.466+0000 7f1ef2e00700 1 -- 172.21.15.18:0/2089089764 shutdown_connections 2024-04-12T21:47:47.468 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:47.466+0000 7f1ef2e00700 1 -- 172.21.15.18:0/2089089764 wait complete. 2024-04-12T21:47:47.468 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:47.467+0000 7f1ef2e00700 1 Processor -- start 2024-04-12T21:47:47.468 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:47.467+0000 7f1ef2e00700 1 -- start start 2024-04-12T21:47:47.470 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:47.468+0000 7f1ef2e00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1eec109430 0x7f1eec11a160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:47:47.470 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:47.468+0000 7f1ef2e00700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1eec11a6a0 0x7f1eec1142d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:47:47.470 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:47.468+0000 7f1ef2e00700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1eec11ac20 con 0x7f1eec109430 2024-04-12T21:47:47.470 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:47.468+0000 7f1ef2e00700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1eec11ad90 con 0x7f1eec11a6a0 2024-04-12T21:47:47.470 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:47.468+0000 7f1eebe00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1eec109430 0x7f1eec11a160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:47:47.471 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:47.468+0000 7f1eebe00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1eec109430 0x7f1eec11a160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:39784/0 (socket says 172.21.15.18:39784) 2024-04-12T21:47:47.471 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:47.468+0000 7f1eebe00700 1 -- 172.21.15.18:0/3576661300 learned_addr learned my addr 172.21.15.18:0/3576661300 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:47:47.471 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:47.468+0000 7f1eebe00700 1 -- 172.21.15.18:0/3576661300 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1eec11a6a0 msgr2=0x7f1eec1142d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:47:47.471 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:47.468+0000 7f1eeb400700 1 --2- 172.21.15.18:0/3576661300 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1eec11a6a0 0x7f1eec1142d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:47:47.471 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:47.468+0000 7f1eebe00700 1 --2- 172.21.15.18:0/3576661300 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1eec11a6a0 0x7f1eec1142d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:47:47.471 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:47.468+0000 7f1eebe00700 1 -- 172.21.15.18:0/3576661300 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ed8005040 con 0x7f1eec109430 2024-04-12T21:47:47.471 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:47.468+0000 7f1eeb400700 1 --2- 172.21.15.18:0/3576661300 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1eec11a6a0 0x7f1eec1142d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-12T21:47:47.471 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:47.469+0000 7f1eebe00700 1 --2- 172.21.15.18:0/3576661300 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1eec109430 0x7f1eec11a160 secure :-1 s=READY pgs=1217 cs=0 l=1 rev1=1 crypto rx=0x7f1edc00c8c0 tx=0x7f1edc0044c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:47:47.471 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:47.469+0000 7f1ee3e00700 1 -- 172.21.15.18:0/3576661300 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1edc00a980 con 0x7f1eec109430 2024-04-12T21:47:47.471 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:47.469+0000 7f1ee3e00700 1 -- 172.21.15.18:0/3576661300 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f1edc00ce60 con 0x7f1eec109430 2024-04-12T21:47:47.471 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:47.469+0000 7f1ee3e00700 1 -- 172.21.15.18:0/3576661300 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1edc0115e0 con 0x7f1eec109430 2024-04-12T21:47:47.471 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:47.469+0000 7f1ef2e00700 1 -- 172.21.15.18:0/3576661300 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1eec1148d0 con 0x7f1eec109430 2024-04-12T21:47:47.472 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:47.469+0000 7f1ef2e00700 1 -- 172.21.15.18:0/3576661300 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1eec114e90 con 0x7f1eec109430 2024-04-12T21:47:47.472 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:47.470+0000 7f1ef2e00700 1 -- 172.21.15.18:0/3576661300 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1eec104b80 con 0x7f1eec109430 2024-04-12T21:47:47.472 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:47.471+0000 7f1ee3e00700 1 -- 172.21.15.18:0/3576661300 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7f1edc012400 con 0x7f1eec109430 2024-04-12T21:47:47.473 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:47.472+0000 7f1ee3e00700 1 --2- 172.21.15.18:0/3576661300 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f1ed006ec00 0x7f1ed00710b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:47:47.473 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:47.472+0000 7f1ee3e00700 1 -- 172.21.15.18:0/3576661300 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 6757+0+0 (secure 0 0 0) 0x7f1edc08ea90 con 0x7f1eec109430 2024-04-12T21:47:47.473 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:47.472+0000 7f1eeb400700 1 --2- 172.21.15.18:0/3576661300 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f1ed006ec00 0x7f1ed00710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:47:47.474 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:47.473+0000 7f1eeb400700 1 --2- 172.21.15.18:0/3576661300 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f1ed006ec00 0x7f1ed00710b0 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f1ed8009040 tx=0x7f1ed8009e30 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:47:47.478 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:47.477+0000 7f1ee3e00700 1 -- 172.21.15.18:0/3576661300 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f1edc05b2c0 con 0x7f1eec109430 2024-04-12T21:47:47.708 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:47.707+0000 7f1ef2e00700 1 -- 172.21.15.18:0/3576661300 --> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f1eec1157b0 con 0x7f1ed006ec00 2024-04-12T21:47:47.719 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:47.718+0000 7f1ee3e00700 1 -- 172.21.15.18:0/3576661300 <== mgr.34102 v2:172.21.15.18:6800/3812891919 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7f1eec1157b0 con 0x7f1ed006ec00 2024-04-12T21:47:47.719 INFO:teuthology.orchestra.run.smithi018.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-12T21:47:47.719 INFO:teuthology.orchestra.run.smithi018.stdout:alertmanager.smithi018 smithi018 *:9093,9094 running (48m) 9m ago 62m 14.8M - 0.25.0 c8568f914cd2 c332090068e8 2024-04-12T21:47:47.720 INFO:teuthology.orchestra.run.smithi018.stdout:crash.smithi018 smithi018 running (44m) 9m ago 63m 6665k - 19.0.0-2879-g324baf8d 8717720dc5e5 b7f80d1f7982 2024-04-12T21:47:47.720 INFO:teuthology.orchestra.run.smithi018.stdout:crash.smithi186 smithi186 running (44m) 9m ago 61m 6660k - 19.0.0-2879-g324baf8d 8717720dc5e5 a4f55c81b623 2024-04-12T21:47:47.720 INFO:teuthology.orchestra.run.smithi018.stdout:grafana.smithi018 smithi018 *:3000 running (47m) 9m ago 62m 73.8M - 9.4.12 2bacad6d85d8 b08e3bf176dd 2024-04-12T21:47:47.720 INFO:teuthology.orchestra.run.smithi018.stdout:mds.cephfs.smithi018.acqbsc smithi018 running (40m) 9m ago 55m 21.9M - 19.0.0-2879-g324baf8d 8717720dc5e5 021f99ab24ae 2024-04-12T21:47:47.720 INFO:teuthology.orchestra.run.smithi018.stdout:mds.cephfs.smithi018.szjluo smithi018 running (40m) 9m ago 55m 21.7M - 19.0.0-2879-g324baf8d 8717720dc5e5 0003117479d4 2024-04-12T21:47:47.720 INFO:teuthology.orchestra.run.smithi018.stdout:mds.cephfs.smithi186.jdllyn smithi186 running (24m) 9m ago 55m 16.5M - 19.0.0-2879-g324baf8d 8717720dc5e5 21b1f46eef23 2024-04-12T21:47:47.720 INFO:teuthology.orchestra.run.smithi018.stdout:mds.cephfs.smithi186.wjnafs smithi186 running (9m) 9m ago 55m 11.0M - 19.0.0-2879-g324baf8d 8717720dc5e5 3cf1ef4148e2 2024-04-12T21:47:47.720 INFO:teuthology.orchestra.run.smithi018.stdout:mgr.smithi018.xjzwga smithi018 *:8443,9283,8765 running (51m) 9m ago 64m 517M - 19.0.0-2879-g324baf8d 8717720dc5e5 56088305eab7 2024-04-12T21:47:47.720 INFO:teuthology.orchestra.run.smithi018.stdout:mgr.smithi186.kqcfxk smithi186 *:8443,9283,8765 running (49m) 9m ago 61m 460M - 19.0.0-2879-g324baf8d 8717720dc5e5 09cd766e5e0d 2024-04-12T21:47:47.720 INFO:teuthology.orchestra.run.smithi018.stdout:mon.smithi018 smithi018 running (46m) 9m ago 64m 88.2M 2048M 19.0.0-2879-g324baf8d 8717720dc5e5 df3e201dfed6 2024-04-12T21:47:47.720 INFO:teuthology.orchestra.run.smithi018.stdout:mon.smithi186 smithi186 running (45m) 9m ago 61m 53.6M 2048M 19.0.0-2879-g324baf8d 8717720dc5e5 c77498da9b2b 2024-04-12T21:47:47.720 INFO:teuthology.orchestra.run.smithi018.stdout:node-exporter.smithi018 smithi018 *:9100 running (49m) 9m ago 63m 8468k - 1.5.0 0da6a335fe13 326c3d9c6847 2024-04-12T21:47:47.720 INFO:teuthology.orchestra.run.smithi018.stdout:node-exporter.smithi186 smithi186 *:9100 running (49m) 9m ago 61m 8166k - 1.5.0 0da6a335fe13 ec2112baeecf 2024-04-12T21:47:47.720 INFO:teuthology.orchestra.run.smithi018.stdout:osd.0 smithi018 running (44m) 9m ago 60m 123M 1559M 19.0.0-2879-g324baf8d 8717720dc5e5 60b1273ca047 2024-04-12T21:47:47.721 INFO:teuthology.orchestra.run.smithi018.stdout:osd.1 smithi018 running (43m) 9m ago 59m 95.4M 1559M 19.0.0-2879-g324baf8d 8717720dc5e5 170ce1e5fb8d 2024-04-12T21:47:47.721 INFO:teuthology.orchestra.run.smithi018.stdout:osd.2 smithi018 running (42m) 9m ago 59m 94.6M 1559M 19.0.0-2879-g324baf8d 8717720dc5e5 c2cbc6aa2839 2024-04-12T21:47:47.721 INFO:teuthology.orchestra.run.smithi018.stdout:osd.3 smithi186 running (42m) 9m ago 58m 101M 2581M 19.0.0-2879-g324baf8d 8717720dc5e5 0138e8262e4b 2024-04-12T21:47:47.721 INFO:teuthology.orchestra.run.smithi018.stdout:osd.4 smithi186 running (41m) 9m ago 58m 91.5M 2581M 19.0.0-2879-g324baf8d 8717720dc5e5 b20cb63b25c9 2024-04-12T21:47:47.721 INFO:teuthology.orchestra.run.smithi018.stdout:osd.5 smithi186 running (41m) 9m ago 57m 92.2M 2581M 19.0.0-2879-g324baf8d 8717720dc5e5 2513bea2610e 2024-04-12T21:47:47.721 INFO:teuthology.orchestra.run.smithi018.stdout:prometheus.smithi018 smithi018 *:9095 running (48m) 9m ago 62m 69.6M - 2.43.0 a07b618ecd1d f58841127e21 2024-04-12T21:47:47.722 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:47.721+0000 7f1ef2e00700 1 -- 172.21.15.18:0/3576661300 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f1ed006ec00 msgr2=0x7f1ed00710b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:47:47.722 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:47.721+0000 7f1ef2e00700 1 --2- 172.21.15.18:0/3576661300 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f1ed006ec00 0x7f1ed00710b0 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f1ed8009040 tx=0x7f1ed8009e30 comp rx=0 tx=0).stop 2024-04-12T21:47:47.722 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:47.721+0000 7f1ef2e00700 1 -- 172.21.15.18:0/3576661300 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1eec109430 msgr2=0x7f1eec11a160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:47:47.722 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:47.721+0000 7f1ef2e00700 1 --2- 172.21.15.18:0/3576661300 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1eec109430 0x7f1eec11a160 secure :-1 s=READY pgs=1217 cs=0 l=1 rev1=1 crypto rx=0x7f1edc00c8c0 tx=0x7f1edc0044c0 comp rx=0 tx=0).stop 2024-04-12T21:47:47.722 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:47.721+0000 7f1ef2e00700 1 -- 172.21.15.18:0/3576661300 shutdown_connections 2024-04-12T21:47:47.723 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:47.721+0000 7f1ef2e00700 1 --2- 172.21.15.18:0/3576661300 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f1ed006ec00 0x7f1ed00710b0 unknown :-1 s=CLOSED pgs=646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:47:47.723 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:47.721+0000 7f1ef2e00700 1 --2- 172.21.15.18:0/3576661300 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1eec109430 0x7f1eec11a160 unknown :-1 s=CLOSED pgs=1217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:47:47.723 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:47.721+0000 7f1ef2e00700 1 --2- 172.21.15.18:0/3576661300 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1eec11a6a0 0x7f1eec1142d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:47:47.723 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:47.721+0000 7f1ef2e00700 1 -- 172.21.15.18:0/3576661300 >> 172.21.15.18:0/3576661300 conn(0x7f1eec072c00 msgr2=0x7f1eec10c1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:47:47.723 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:47.722+0000 7f1ef2e00700 1 -- 172.21.15.18:0/3576661300 shutdown_connections 2024-04-12T21:47:47.723 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:47.722+0000 7f1ef2e00700 1 -- 172.21.15.18:0/3576661300 wait complete. 2024-04-12T21:47:47.878 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:47.877+0000 7f7096000700 1 -- 172.21.15.18:0/4242247357 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f70901059b0 msgr2=0x7f70900ff020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:47:47.878 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:47.877+0000 7f7096000700 1 --2- 172.21.15.18:0/4242247357 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f70901059b0 0x7f70900ff020 secure :-1 s=READY pgs=1218 cs=0 l=1 rev1=1 crypto rx=0x7f707c004940 tx=0x7f707c009e30 comp rx=0 tx=0).stop 2024-04-12T21:47:47.879 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:47.877+0000 7f7096000700 1 -- 172.21.15.18:0/4242247357 shutdown_connections 2024-04-12T21:47:47.879 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:47.877+0000 7f7096000700 1 --2- 172.21.15.18:0/4242247357 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f70901059b0 0x7f70900ff020 unknown :-1 s=CLOSED pgs=1218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:47:47.879 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:47.877+0000 7f7096000700 1 --2- 172.21.15.18:0/4242247357 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7090105010 0x7f70901053e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:47:47.879 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:47.877+0000 7f7096000700 1 -- 172.21.15.18:0/4242247357 >> 172.21.15.18:0/4242247357 conn(0x7f70900faba0 msgr2=0x7f70900fcfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:47:47.879 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:47.878+0000 7f7096000700 1 -- 172.21.15.18:0/4242247357 shutdown_connections 2024-04-12T21:47:47.879 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:47.878+0000 7f7096000700 1 -- 172.21.15.18:0/4242247357 wait complete. 2024-04-12T21:47:47.879 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:47.879+0000 7f7096000700 1 Processor -- start 2024-04-12T21:47:47.880 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:47.879+0000 7f7096000700 1 -- start start 2024-04-12T21:47:47.880 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:47.879+0000 7f7096000700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7090105010 0x7f7090115e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:47:47.881 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:47.879+0000 7f7096000700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f70901059b0 0x7f7090116340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:47:47.881 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:47.879+0000 7f7096000700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7090116990 con 0x7f7090105010 2024-04-12T21:47:47.881 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:47.879+0000 7f7096000700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f709010fd60 con 0x7f70901059b0 2024-04-12T21:47:47.881 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:47.880+0000 7f708f400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7090105010 0x7f7090115e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:47:47.881 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:47.880+0000 7f708f400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7090105010 0x7f7090115e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:39804/0 (socket says 172.21.15.18:39804) 2024-04-12T21:47:47.881 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:47.880+0000 7f708f400700 1 -- 172.21.15.18:0/600205219 learned_addr learned my addr 172.21.15.18:0/600205219 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:47:47.881 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:47.880+0000 7f708ea00700 1 --2- 172.21.15.18:0/600205219 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f70901059b0 0x7f7090116340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:47:47.882 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:47.880+0000 7f708f400700 1 -- 172.21.15.18:0/600205219 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f70901059b0 msgr2=0x7f7090116340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:47:47.882 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:47.880+0000 7f708f400700 1 --2- 172.21.15.18:0/600205219 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f70901059b0 0x7f7090116340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:47:47.882 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:47.880+0000 7f708f400700 1 -- 172.21.15.18:0/600205219 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f707c005040 con 0x7f7090105010 2024-04-12T21:47:47.882 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:47.881+0000 7f708f400700 1 --2- 172.21.15.18:0/600205219 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7090105010 0x7f7090115e00 secure :-1 s=READY pgs=1219 cs=0 l=1 rev1=1 crypto rx=0x7f708000c8c0 tx=0x7f70800044c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:47:47.882 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:47.881+0000 7f7087400700 1 -- 172.21.15.18:0/600205219 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f708000a980 con 0x7f7090105010 2024-04-12T21:47:47.882 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:47.881+0000 7f7087400700 1 -- 172.21.15.18:0/600205219 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f708000ce60 con 0x7f7090105010 2024-04-12T21:47:47.883 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:47.881+0000 7f7096000700 1 -- 172.21.15.18:0/600205219 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7090110040 con 0x7f7090105010 2024-04-12T21:47:47.883 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:47.881+0000 7f7087400700 1 -- 172.21.15.18:0/600205219 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f70800115e0 con 0x7f7090105010 2024-04-12T21:47:47.883 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:47.881+0000 7f7096000700 1 -- 172.21.15.18:0/600205219 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7090110600 con 0x7f7090105010 2024-04-12T21:47:47.883 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:47.882+0000 7f7096000700 1 -- 172.21.15.18:0/600205219 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7090100760 con 0x7f7090105010 2024-04-12T21:47:47.884 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:47.883+0000 7f7087400700 1 -- 172.21.15.18:0/600205219 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7f7080012400 con 0x7f7090105010 2024-04-12T21:47:47.885 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:47.884+0000 7f7087400700 1 --2- 172.21.15.18:0/600205219 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f707406ec10 0x7f70740710c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:47:47.885 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:47.885+0000 7f7087400700 1 -- 172.21.15.18:0/600205219 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 6757+0+0 (secure 0 0 0) 0x7f708008ea40 con 0x7f7090105010 2024-04-12T21:47:47.885 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:47.885+0000 7f708ea00700 1 --2- 172.21.15.18:0/600205219 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f707406ec10 0x7f70740710c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:47:47.886 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:47.886+0000 7f708ea00700 1 --2- 172.21.15.18:0/600205219 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f707406ec10 0x7f70740710c0 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f707c00b020 tx=0x7f707c009e30 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:47:47.890 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:47.889+0000 7f7087400700 1 -- 172.21.15.18:0/600205219 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f708005b270 con 0x7f7090105010 2024-04-12T21:47:48.105 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:47:47 smithi018 ceph-mon[112590]: pgmap v1443: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:47:48.105 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:47:47 smithi018 ceph-mon[112590]: from='client.36496 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:47:48.216 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:48.215+0000 7f7096000700 1 -- 172.21.15.18:0/600205219 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f709004bed0 con 0x7f7090105010 2024-04-12T21:47:48.217 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:48.217+0000 7f7087400700 1 -- 172.21.15.18:0/600205219 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+623 (secure 0 0 0) 0x7f70800126b0 con 0x7f7090105010 2024-04-12T21:47:48.218 INFO:teuthology.orchestra.run.smithi018.stdout:{ 2024-04-12T21:47:48.218 INFO:teuthology.orchestra.run.smithi018.stdout: "mon": { 2024-04-12T21:47:48.218 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 19.0.0-2879-g324baf8d (324baf8ddb91dea46cac50cb535c306115791355) squid (dev)": 2 2024-04-12T21:47:48.218 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-12T21:47:48.218 INFO:teuthology.orchestra.run.smithi018.stdout: "mgr": { 2024-04-12T21:47:48.218 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 19.0.0-2879-g324baf8d (324baf8ddb91dea46cac50cb535c306115791355) squid (dev)": 2 2024-04-12T21:47:48.218 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-12T21:47:48.218 INFO:teuthology.orchestra.run.smithi018.stdout: "osd": { 2024-04-12T21:47:48.218 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 19.0.0-2879-g324baf8d (324baf8ddb91dea46cac50cb535c306115791355) squid (dev)": 6 2024-04-12T21:47:48.218 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-12T21:47:48.218 INFO:teuthology.orchestra.run.smithi018.stdout: "mds": { 2024-04-12T21:47:48.218 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 19.0.0-2879-g324baf8d (324baf8ddb91dea46cac50cb535c306115791355) squid (dev)": 4 2024-04-12T21:47:48.219 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-12T21:47:48.219 INFO:teuthology.orchestra.run.smithi018.stdout: "overall": { 2024-04-12T21:47:48.219 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 19.0.0-2879-g324baf8d (324baf8ddb91dea46cac50cb535c306115791355) squid (dev)": 14 2024-04-12T21:47:48.219 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-04-12T21:47:48.219 INFO:teuthology.orchestra.run.smithi018.stdout:} 2024-04-12T21:47:48.220 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:48.219+0000 7f7096000700 1 -- 172.21.15.18:0/600205219 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f707406ec10 msgr2=0x7f70740710c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:47:48.220 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:48.219+0000 7f7096000700 1 --2- 172.21.15.18:0/600205219 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f707406ec10 0x7f70740710c0 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f707c00b020 tx=0x7f707c009e30 comp rx=0 tx=0).stop 2024-04-12T21:47:48.220 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:48.219+0000 7f7096000700 1 -- 172.21.15.18:0/600205219 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7090105010 msgr2=0x7f7090115e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:47:48.221 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:48.219+0000 7f7096000700 1 --2- 172.21.15.18:0/600205219 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7090105010 0x7f7090115e00 secure :-1 s=READY pgs=1219 cs=0 l=1 rev1=1 crypto rx=0x7f708000c8c0 tx=0x7f70800044c0 comp rx=0 tx=0).stop 2024-04-12T21:47:48.221 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:48.219+0000 7f7096000700 1 -- 172.21.15.18:0/600205219 shutdown_connections 2024-04-12T21:47:48.221 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:48.219+0000 7f7096000700 1 --2- 172.21.15.18:0/600205219 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f707406ec10 0x7f70740710c0 unknown :-1 s=CLOSED pgs=647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:47:48.221 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:48.219+0000 7f7096000700 1 --2- 172.21.15.18:0/600205219 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7090105010 0x7f7090115e00 unknown :-1 s=CLOSED pgs=1219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:47:48.221 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:48.219+0000 7f7096000700 1 --2- 172.21.15.18:0/600205219 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f70901059b0 0x7f7090116340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:47:48.221 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:48.219+0000 7f7096000700 1 -- 172.21.15.18:0/600205219 >> 172.21.15.18:0/600205219 conn(0x7f70900faba0 msgr2=0x7f70900fbc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:47:48.221 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:48.220+0000 7f7096000700 1 -- 172.21.15.18:0/600205219 shutdown_connections 2024-04-12T21:47:48.221 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:48.220+0000 7f7096000700 1 -- 172.21.15.18:0/600205219 wait complete. 2024-04-12T21:47:48.284 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:47:47 smithi186 ceph-mon[86379]: pgmap v1443: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:47:48.284 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:47:47 smithi186 ceph-mon[86379]: from='client.36496 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:47:48.379 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:48.377+0000 7faf73800700 1 -- 172.21.15.18:0/3149598433 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7faf6c1059c0 msgr2=0x7faf6c0ff080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:47:48.379 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:48.377+0000 7faf73800700 1 --2- 172.21.15.18:0/3149598433 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7faf6c1059c0 0x7faf6c0ff080 secure :-1 s=READY pgs=1220 cs=0 l=1 rev1=1 crypto rx=0x7faf58004940 tx=0x7faf58009e30 comp rx=0 tx=0).stop 2024-04-12T21:47:48.379 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:48.378+0000 7faf73800700 1 -- 172.21.15.18:0/3149598433 shutdown_connections 2024-04-12T21:47:48.379 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:48.378+0000 7faf73800700 1 --2- 172.21.15.18:0/3149598433 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7faf6c1059c0 0x7faf6c0ff080 unknown :-1 s=CLOSED pgs=1220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:47:48.379 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:48.378+0000 7faf73800700 1 --2- 172.21.15.18:0/3149598433 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7faf6c105020 0x7faf6c1053f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:47:48.379 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:48.378+0000 7faf73800700 1 -- 172.21.15.18:0/3149598433 >> 172.21.15.18:0/3149598433 conn(0x7faf6c0faba0 msgr2=0x7faf6c0fcfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:47:48.379 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:48.378+0000 7faf73800700 1 -- 172.21.15.18:0/3149598433 shutdown_connections 2024-04-12T21:47:48.379 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:48.378+0000 7faf73800700 1 -- 172.21.15.18:0/3149598433 wait complete. 2024-04-12T21:47:48.381 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:48.379+0000 7faf73800700 1 Processor -- start 2024-04-12T21:47:48.381 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:48.379+0000 7faf73800700 1 -- start start 2024-04-12T21:47:48.381 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:48.379+0000 7faf73800700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7faf6c105020 0x7faf6c115ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:47:48.381 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:48.379+0000 7faf73800700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7faf6c1059c0 0x7faf6c1160e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:47:48.381 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:48.379+0000 7faf73800700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf6c116730 con 0x7faf6c105020 2024-04-12T21:47:48.381 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:48.379+0000 7faf73800700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf6c10fd10 con 0x7faf6c1059c0 2024-04-12T21:47:48.381 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:48.380+0000 7faf70e00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7faf6c105020 0x7faf6c115ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:47:48.381 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:48.380+0000 7faf70e00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7faf6c105020 0x7faf6c115ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:39820/0 (socket says 172.21.15.18:39820) 2024-04-12T21:47:48.381 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:48.380+0000 7faf70e00700 1 -- 172.21.15.18:0/1470323881 learned_addr learned my addr 172.21.15.18:0/1470323881 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:47:48.381 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:48.380+0000 7faf6be00700 1 --2- 172.21.15.18:0/1470323881 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7faf6c1059c0 0x7faf6c1160e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:47:48.381 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:48.380+0000 7faf70e00700 1 -- 172.21.15.18:0/1470323881 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7faf6c1059c0 msgr2=0x7faf6c1160e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:47:48.381 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:48.380+0000 7faf70e00700 1 --2- 172.21.15.18:0/1470323881 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7faf6c1059c0 0x7faf6c1160e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:47:48.381 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:48.380+0000 7faf70e00700 1 -- 172.21.15.18:0/1470323881 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf58005040 con 0x7faf6c105020 2024-04-12T21:47:48.382 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:48.380+0000 7faf70e00700 1 --2- 172.21.15.18:0/1470323881 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7faf6c105020 0x7faf6c115ba0 secure :-1 s=READY pgs=1221 cs=0 l=1 rev1=1 crypto rx=0x7faf60002a50 tx=0x7faf600046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:47:48.382 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:48.381+0000 7faf68c00700 1 -- 172.21.15.18:0/1470323881 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7faf60007cc0 con 0x7faf6c105020 2024-04-12T21:47:48.382 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:48.381+0000 7faf68c00700 1 -- 172.21.15.18:0/1470323881 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7faf6000f040 con 0x7faf6c105020 2024-04-12T21:47:48.383 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:48.381+0000 7faf68c00700 1 -- 172.21.15.18:0/1470323881 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7faf60009640 con 0x7faf6c105020 2024-04-12T21:47:48.383 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:48.381+0000 7faf73800700 1 -- 172.21.15.18:0/1470323881 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faf6c10fff0 con 0x7faf6c105020 2024-04-12T21:47:48.383 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:48.381+0000 7faf73800700 1 -- 172.21.15.18:0/1470323881 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faf6c1105b0 con 0x7faf6c105020 2024-04-12T21:47:48.383 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:48.382+0000 7faf73800700 1 -- 172.21.15.18:0/1470323881 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faf6c100770 con 0x7faf6c105020 2024-04-12T21:47:48.384 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:48.383+0000 7faf68c00700 1 -- 172.21.15.18:0/1470323881 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7faf600097a0 con 0x7faf6c105020 2024-04-12T21:47:48.385 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:48.384+0000 7faf68c00700 1 --2- 172.21.15.18:0/1470323881 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7faf5406ec00 0x7faf540710b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:47:48.385 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:48.384+0000 7faf68c00700 1 -- 172.21.15.18:0/1470323881 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 6757+0+0 (secure 0 0 0) 0x7faf6008e6c0 con 0x7faf6c105020 2024-04-12T21:47:48.385 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:48.384+0000 7faf6be00700 1 --2- 172.21.15.18:0/1470323881 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7faf5406ec00 0x7faf540710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:47:48.385 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:48.385+0000 7faf6be00700 1 --2- 172.21.15.18:0/1470323881 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7faf5406ec00 0x7faf540710b0 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7faf6c111440 tx=0x7faf58009e30 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:47:48.389 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:48.389+0000 7faf68c00700 1 -- 172.21.15.18:0/1470323881 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7faf6005af10 con 0x7faf6c105020 2024-04-12T21:47:48.662 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:48.661+0000 7faf73800700 1 -- 172.21.15.18:0/1470323881 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7faf6c1111f0 con 0x7faf6c105020 2024-04-12T21:47:48.663 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:48.662+0000 7faf68c00700 1 -- 172.21.15.18:0/1470323881 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 33 v33) v1 ==== 76+0+1876 (secure 0 0 0) 0x7faf6005aaa0 con 0x7faf6c105020 2024-04-12T21:47:48.664 INFO:teuthology.orchestra.run.smithi018.stdout:e33 2024-04-12T21:47:48.664 INFO:teuthology.orchestra.run.smithi018.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-12T21:47:48.664 INFO:teuthology.orchestra.run.smithi018.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-12T21:47:48.664 INFO:teuthology.orchestra.run.smithi018.stdout:legacy client fscid: 1 2024-04-12T21:47:48.664 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-12T21:47:48.664 INFO:teuthology.orchestra.run.smithi018.stdout:Filesystem 'cephfs' (1) 2024-04-12T21:47:48.664 INFO:teuthology.orchestra.run.smithi018.stdout:fs_name cephfs 2024-04-12T21:47:48.664 INFO:teuthology.orchestra.run.smithi018.stdout:epoch 24 2024-04-12T21:47:48.664 INFO:teuthology.orchestra.run.smithi018.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-04-12T21:47:48.664 INFO:teuthology.orchestra.run.smithi018.stdout:created 2024-04-12T20:51:57.362850+0000 2024-04-12T21:47:48.665 INFO:teuthology.orchestra.run.smithi018.stdout:modified 2024-04-12T21:06:59.717190+0000 2024-04-12T21:47:48.665 INFO:teuthology.orchestra.run.smithi018.stdout:tableserver 0 2024-04-12T21:47:48.665 INFO:teuthology.orchestra.run.smithi018.stdout:root 0 2024-04-12T21:47:48.665 INFO:teuthology.orchestra.run.smithi018.stdout:session_timeout 60 2024-04-12T21:47:48.665 INFO:teuthology.orchestra.run.smithi018.stdout:session_autoclose 300 2024-04-12T21:47:48.665 INFO:teuthology.orchestra.run.smithi018.stdout:max_file_size 1099511627776 2024-04-12T21:47:48.665 INFO:teuthology.orchestra.run.smithi018.stdout:max_xattr_size 65536 2024-04-12T21:47:48.665 INFO:teuthology.orchestra.run.smithi018.stdout:required_client_features {} 2024-04-12T21:47:48.665 INFO:teuthology.orchestra.run.smithi018.stdout:last_failure 0 2024-04-12T21:47:48.665 INFO:teuthology.orchestra.run.smithi018.stdout:last_failure_osd_epoch 100 2024-04-12T21:47:48.665 INFO:teuthology.orchestra.run.smithi018.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-12T21:47:48.665 INFO:teuthology.orchestra.run.smithi018.stdout:max_mds 2 2024-04-12T21:47:48.665 INFO:teuthology.orchestra.run.smithi018.stdout:in 0,1 2024-04-12T21:47:48.665 INFO:teuthology.orchestra.run.smithi018.stdout:up {} 2024-04-12T21:47:48.666 INFO:teuthology.orchestra.run.smithi018.stdout:failed 0,1 2024-04-12T21:47:48.666 INFO:teuthology.orchestra.run.smithi018.stdout:damaged 2024-04-12T21:47:48.666 INFO:teuthology.orchestra.run.smithi018.stdout:stopped 2024-04-12T21:47:48.666 INFO:teuthology.orchestra.run.smithi018.stdout:data_pools [3] 2024-04-12T21:47:48.666 INFO:teuthology.orchestra.run.smithi018.stdout:metadata_pool 2 2024-04-12T21:47:48.666 INFO:teuthology.orchestra.run.smithi018.stdout:inline_data disabled 2024-04-12T21:47:48.666 INFO:teuthology.orchestra.run.smithi018.stdout:balancer 2024-04-12T21:47:48.666 INFO:teuthology.orchestra.run.smithi018.stdout:bal_rank_mask -1 2024-04-12T21:47:48.666 INFO:teuthology.orchestra.run.smithi018.stdout:standby_count_wanted 1 2024-04-12T21:47:48.666 INFO:teuthology.orchestra.run.smithi018.stdout:qdb_cluster leader: 0 members: 2024-04-12T21:47:48.666 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-12T21:47:48.666 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-12T21:47:48.666 INFO:teuthology.orchestra.run.smithi018.stdout:Standby daemons: 2024-04-12T21:47:48.667 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-12T21:47:48.667 INFO:teuthology.orchestra.run.smithi018.stdout:[mds.cephfs.smithi018.acqbsc{-1:34436} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.18:6828/755179764,v1:172.21.15.18:6829/755179764] compat {c=[1],r=[1],i=[1fff]}] 2024-04-12T21:47:48.667 INFO:teuthology.orchestra.run.smithi018.stdout:[mds.cephfs.smithi018.szjluo{-1:34468} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.18:6826/2407992653,v1:172.21.15.18:6827/2407992653] compat {c=[1],r=[1],i=[1fff]}] 2024-04-12T21:47:48.667 INFO:teuthology.orchestra.run.smithi018.stdout:[mds.cephfs.smithi186.jdllyn{-1:44595} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.186:6824/3523444202,v1:172.21.15.186:6825/3523444202] compat {c=[1],r=[1],i=[1fff]}] 2024-04-12T21:47:48.667 INFO:teuthology.orchestra.run.smithi018.stdout:[mds.cephfs.smithi186.wjnafs{-1:44875} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.186:6826/3122443766,v1:172.21.15.186:6827/3122443766] compat {c=[1],r=[1],i=[1fff]}] 2024-04-12T21:47:48.668 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:48.665+0000 7faf73800700 1 -- 172.21.15.18:0/1470323881 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7faf5406ec00 msgr2=0x7faf540710b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:47:48.668 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:48.665+0000 7faf73800700 1 --2- 172.21.15.18:0/1470323881 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7faf5406ec00 0x7faf540710b0 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7faf6c111440 tx=0x7faf58009e30 comp rx=0 tx=0).stop 2024-04-12T21:47:48.668 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:48.665+0000 7faf73800700 1 -- 172.21.15.18:0/1470323881 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7faf6c105020 msgr2=0x7faf6c115ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:47:48.668 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:48.665+0000 7faf73800700 1 --2- 172.21.15.18:0/1470323881 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7faf6c105020 0x7faf6c115ba0 secure :-1 s=READY pgs=1221 cs=0 l=1 rev1=1 crypto rx=0x7faf60002a50 tx=0x7faf600046b0 comp rx=0 tx=0).stop 2024-04-12T21:47:48.668 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:48.665+0000 7faf73800700 1 -- 172.21.15.18:0/1470323881 shutdown_connections 2024-04-12T21:47:48.668 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:48.665+0000 7faf73800700 1 --2- 172.21.15.18:0/1470323881 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7faf5406ec00 0x7faf540710b0 unknown :-1 s=CLOSED pgs=648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:47:48.668 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:48.665+0000 7faf73800700 1 --2- 172.21.15.18:0/1470323881 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7faf6c105020 0x7faf6c115ba0 unknown :-1 s=CLOSED pgs=1221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:47:48.668 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:48.665+0000 7faf73800700 1 --2- 172.21.15.18:0/1470323881 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7faf6c1059c0 0x7faf6c1160e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:47:48.668 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:48.665+0000 7faf73800700 1 -- 172.21.15.18:0/1470323881 >> 172.21.15.18:0/1470323881 conn(0x7faf6c0faba0 msgr2=0x7faf6c0fbb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:47:48.668 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:48.665+0000 7faf73800700 1 -- 172.21.15.18:0/1470323881 shutdown_connections 2024-04-12T21:47:48.668 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:48.665+0000 7faf73800700 1 -- 172.21.15.18:0/1470323881 wait complete. 2024-04-12T21:47:48.669 INFO:teuthology.orchestra.run.smithi018.stderr:dumped fsmap epoch 33 2024-04-12T21:47:48.822 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:48.820+0000 7f3351800700 1 -- 172.21.15.18:0/76512964 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f334c109dd0 msgr2=0x7f334c103440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:47:48.823 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:48.820+0000 7f3351800700 1 --2- 172.21.15.18:0/76512964 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f334c109dd0 0x7f334c103440 secure :-1 s=READY pgs=1222 cs=0 l=1 rev1=1 crypto rx=0x7f3334004940 tx=0x7f3334009e30 comp rx=0 tx=0).stop 2024-04-12T21:47:48.823 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:48.820+0000 7f3351800700 1 -- 172.21.15.18:0/76512964 shutdown_connections 2024-04-12T21:47:48.823 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:48.820+0000 7f3351800700 1 --2- 172.21.15.18:0/76512964 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f334c109dd0 0x7f334c103440 unknown :-1 s=CLOSED pgs=1222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:47:48.823 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:48.820+0000 7f3351800700 1 --2- 172.21.15.18:0/76512964 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f334c109430 0x7f334c109800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:47:48.823 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:48.820+0000 7f3351800700 1 -- 172.21.15.18:0/76512964 >> 172.21.15.18:0/76512964 conn(0x7f334c072c00 msgr2=0x7f334c075010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:47:48.823 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:48.820+0000 7f3351800700 1 -- 172.21.15.18:0/76512964 shutdown_connections 2024-04-12T21:47:48.823 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:48.820+0000 7f3351800700 1 -- 172.21.15.18:0/76512964 wait complete. 2024-04-12T21:47:48.823 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:48.821+0000 7f3351800700 1 Processor -- start 2024-04-12T21:47:48.823 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:48.821+0000 7f3351800700 1 -- start start 2024-04-12T21:47:48.823 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:48.821+0000 7f3351800700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f334c109430 0x7f334c119f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:47:48.823 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:48.821+0000 7f3351800700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f334c109dd0 0x7f334c11a4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:47:48.823 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:48.821+0000 7f3351800700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f334c11ab20 con 0x7f334c109dd0 2024-04-12T21:47:48.823 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:48.821+0000 7f3351800700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f334c114100 con 0x7f334c109430 2024-04-12T21:47:48.824 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:48.822+0000 7f3343e00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f334c109dd0 0x7f334c11a4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:47:48.824 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:48.822+0000 7f3343e00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f334c109dd0 0x7f334c11a4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:39832/0 (socket says 172.21.15.18:39832) 2024-04-12T21:47:48.824 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:48.822+0000 7f3343e00700 1 -- 172.21.15.18:0/2752397685 learned_addr learned my addr 172.21.15.18:0/2752397685 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:47:48.824 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:48.822+0000 7f334aa00700 1 --2- 172.21.15.18:0/2752397685 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f334c109430 0x7f334c119f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:47:48.824 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:48.822+0000 7f3343e00700 1 -- 172.21.15.18:0/2752397685 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f334c109430 msgr2=0x7f334c119f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:47:48.824 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:48.822+0000 7f3343e00700 1 --2- 172.21.15.18:0/2752397685 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f334c109430 0x7f334c119f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:47:48.824 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:48.822+0000 7f3343e00700 1 -- 172.21.15.18:0/2752397685 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3334005040 con 0x7f334c109dd0 2024-04-12T21:47:48.825 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:48.822+0000 7f3343e00700 1 --2- 172.21.15.18:0/2752397685 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f334c109dd0 0x7f334c11a4d0 secure :-1 s=READY pgs=1223 cs=0 l=1 rev1=1 crypto rx=0x7f3334009100 tx=0x7f3334009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:47:48.825 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:48.823+0000 7f3342a00700 1 -- 172.21.15.18:0/2752397685 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f333400e040 con 0x7f334c109dd0 2024-04-12T21:47:48.825 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:48.823+0000 7f3342a00700 1 -- 172.21.15.18:0/2752397685 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f3334013420 con 0x7f334c109dd0 2024-04-12T21:47:48.825 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:48.823+0000 7f3351800700 1 -- 172.21.15.18:0/2752397685 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f334c114380 con 0x7f334c109dd0 2024-04-12T21:47:48.825 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:48.823+0000 7f3342a00700 1 -- 172.21.15.18:0/2752397685 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f333401c630 con 0x7f334c109dd0 2024-04-12T21:47:48.825 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:48.823+0000 7f3351800700 1 -- 172.21.15.18:0/2752397685 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f334c114940 con 0x7f334c109dd0 2024-04-12T21:47:48.826 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:48.824+0000 7f3351800700 1 -- 172.21.15.18:0/2752397685 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f334c104b80 con 0x7f334c109dd0 2024-04-12T21:47:48.826 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:48.825+0000 7f3342a00700 1 -- 172.21.15.18:0/2752397685 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7f3334013590 con 0x7f334c109dd0 2024-04-12T21:47:48.827 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:48.826+0000 7f3342a00700 1 --2- 172.21.15.18:0/2752397685 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f333006ec00 0x7f33300710b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:47:48.827 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:48.826+0000 7f3342a00700 1 -- 172.21.15.18:0/2752397685 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 6757+0+0 (secure 0 0 0) 0x7f333408fc60 con 0x7f334c109dd0 2024-04-12T21:47:48.827 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:48.827+0000 7f334aa00700 1 --2- 172.21.15.18:0/2752397685 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f333006ec00 0x7f33300710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:47:48.828 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:48.827+0000 7f334aa00700 1 --2- 172.21.15.18:0/2752397685 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f333006ec00 0x7f33300710b0 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f333c004c40 tx=0x7f333c009040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:47:48.831 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:48.831+0000 7f3342a00700 1 -- 172.21.15.18:0/2752397685 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f333405c410 con 0x7f334c109dd0 2024-04-12T21:47:49.086 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:49.085+0000 7f3351800700 1 -- 172.21.15.18:0/2752397685 --> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f334c104b80 con 0x7f333006ec00 2024-04-12T21:47:49.088 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:49.087+0000 7f3342a00700 1 -- 172.21.15.18:0/2752397685 <== mgr.34102 v2:172.21.15.18:6800/3812891919 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f334c104b80 con 0x7f333006ec00 2024-04-12T21:47:49.088 INFO:teuthology.orchestra.run.smithi018.stdout:{ 2024-04-12T21:47:49.088 INFO:teuthology.orchestra.run.smithi018.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:f4452def3be4d350f4a8e5feaf7ebb1511be9298fe68478ba7a2d6fbd7d69a63", 2024-04-12T21:47:49.088 INFO:teuthology.orchestra.run.smithi018.stdout: "in_progress": true, 2024-04-12T21:47:49.089 INFO:teuthology.orchestra.run.smithi018.stdout: "which": "Upgrading all daemon types on all hosts", 2024-04-12T21:47:49.089 INFO:teuthology.orchestra.run.smithi018.stdout: "services_complete": [ 2024-04-12T21:47:49.089 INFO:teuthology.orchestra.run.smithi018.stdout: "mgr", 2024-04-12T21:47:49.089 INFO:teuthology.orchestra.run.smithi018.stdout: "mds", 2024-04-12T21:47:49.089 INFO:teuthology.orchestra.run.smithi018.stdout: "crash", 2024-04-12T21:47:49.089 INFO:teuthology.orchestra.run.smithi018.stdout: "osd", 2024-04-12T21:47:49.089 INFO:teuthology.orchestra.run.smithi018.stdout: "mon" 2024-04-12T21:47:49.089 INFO:teuthology.orchestra.run.smithi018.stdout: ], 2024-04-12T21:47:49.089 INFO:teuthology.orchestra.run.smithi018.stdout: "progress": "16/21 daemons upgraded", 2024-04-12T21:47:49.089 INFO:teuthology.orchestra.run.smithi018.stdout: "message": "Currently upgrading mds daemons", 2024-04-12T21:47:49.089 INFO:teuthology.orchestra.run.smithi018.stdout: "is_paused": false 2024-04-12T21:47:49.089 INFO:teuthology.orchestra.run.smithi018.stdout:} 2024-04-12T21:47:49.092 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:49.091+0000 7f3351800700 1 -- 172.21.15.18:0/2752397685 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f333006ec00 msgr2=0x7f33300710b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:47:49.092 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:49.091+0000 7f3351800700 1 --2- 172.21.15.18:0/2752397685 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f333006ec00 0x7f33300710b0 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f333c004c40 tx=0x7f333c009040 comp rx=0 tx=0).stop 2024-04-12T21:47:49.092 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:49.091+0000 7f3351800700 1 -- 172.21.15.18:0/2752397685 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f334c109dd0 msgr2=0x7f334c11a4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:47:49.092 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:49.091+0000 7f3351800700 1 --2- 172.21.15.18:0/2752397685 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f334c109dd0 0x7f334c11a4d0 secure :-1 s=READY pgs=1223 cs=0 l=1 rev1=1 crypto rx=0x7f3334009100 tx=0x7f3334009e30 comp rx=0 tx=0).stop 2024-04-12T21:47:49.092 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:49.091+0000 7f3351800700 1 -- 172.21.15.18:0/2752397685 shutdown_connections 2024-04-12T21:47:49.092 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:49.091+0000 7f3351800700 1 --2- 172.21.15.18:0/2752397685 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f333006ec00 0x7f33300710b0 unknown :-1 s=CLOSED pgs=649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:47:49.092 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:49.091+0000 7f3351800700 1 --2- 172.21.15.18:0/2752397685 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f334c109430 0x7f334c119f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:47:49.092 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:49.091+0000 7f3351800700 1 --2- 172.21.15.18:0/2752397685 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f334c109dd0 0x7f334c11a4d0 unknown :-1 s=CLOSED pgs=1223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:47:49.092 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:49.091+0000 7f3351800700 1 -- 172.21.15.18:0/2752397685 >> 172.21.15.18:0/2752397685 conn(0x7f334c072c00 msgr2=0x7f334c074c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:47:49.092 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:49.091+0000 7f3351800700 1 -- 172.21.15.18:0/2752397685 shutdown_connections 2024-04-12T21:47:49.092 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:49.091+0000 7f3351800700 1 -- 172.21.15.18:0/2752397685 wait complete. 2024-04-12T21:47:49.238 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:49.237+0000 7f28f1000700 1 -- 172.21.15.18:0/1062385182 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f28ec0fe5c0 msgr2=0x7f28ec106c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:47:49.238 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:49.237+0000 7f28f1000700 1 --2- 172.21.15.18:0/1062385182 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f28ec0fe5c0 0x7f28ec106c00 secure :-1 s=READY pgs=1224 cs=0 l=1 rev1=1 crypto rx=0x7f28d8004940 tx=0x7f28d8009e30 comp rx=0 tx=0).stop 2024-04-12T21:47:49.238 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:49.237+0000 7f28f1000700 1 -- 172.21.15.18:0/1062385182 shutdown_connections 2024-04-12T21:47:49.238 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:49.237+0000 7f28f1000700 1 --2- 172.21.15.18:0/1062385182 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f28ec0fe5c0 0x7f28ec106c00 unknown :-1 s=CLOSED pgs=1224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:47:49.239 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:49.237+0000 7f28f1000700 1 --2- 172.21.15.18:0/1062385182 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f28ec0fdc20 0x7f28ec0fdff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:47:49.239 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:49.237+0000 7f28f1000700 1 -- 172.21.15.18:0/1062385182 >> 172.21.15.18:0/1062385182 conn(0x7f28ec0f7650 msgr2=0x7f28ec0f9a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:47:49.239 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:47:48 smithi018 ceph-mon[112590]: from='client.36500 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:47:49.239 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:47:48 smithi018 ceph-mon[112590]: from='client.36504 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:47:49.239 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:47:48 smithi018 ceph-mon[112590]: from='client.? 172.21.15.18:0/600205219' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T21:47:49.239 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:47:48 smithi018 ceph-mon[112590]: from='client.? 172.21.15.18:0/1470323881' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-12T21:47:49.240 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:49.238+0000 7f28f1000700 1 -- 172.21.15.18:0/1062385182 shutdown_connections 2024-04-12T21:47:49.240 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:49.238+0000 7f28f1000700 1 -- 172.21.15.18:0/1062385182 wait complete. 2024-04-12T21:47:49.240 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:49.239+0000 7f28f1000700 1 Processor -- start 2024-04-12T21:47:49.240 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:49.239+0000 7f28f1000700 1 -- start start 2024-04-12T21:47:49.241 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:49.239+0000 7f28f1000700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f28ec0fdc20 0x7f28ec0febc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:47:49.241 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:49.239+0000 7f28f1000700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f28ec0fe5c0 0x7f28ec0ff100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:47:49.241 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:49.239+0000 7f28f1000700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28ec102d50 con 0x7f28ec0fdc20 2024-04-12T21:47:49.241 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:49.239+0000 7f28f1000700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28ec0ff640 con 0x7f28ec0fe5c0 2024-04-12T21:47:49.241 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:49.240+0000 7f28eb400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f28ec0fdc20 0x7f28ec0febc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:47:49.241 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:49.240+0000 7f28eb400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f28ec0fdc20 0x7f28ec0febc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:39860/0 (socket says 172.21.15.18:39860) 2024-04-12T21:47:49.241 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:49.240+0000 7f28eb400700 1 -- 172.21.15.18:0/2066550157 learned_addr learned my addr 172.21.15.18:0/2066550157 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:47:49.242 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:49.240+0000 7f28eb400700 1 -- 172.21.15.18:0/2066550157 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f28ec0fe5c0 msgr2=0x7f28ec0ff100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:47:49.242 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:49.240+0000 7f28eb400700 1 --2- 172.21.15.18:0/2066550157 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f28ec0fe5c0 0x7f28ec0ff100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:47:49.242 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:49.240+0000 7f28eb400700 1 -- 172.21.15.18:0/2066550157 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28d8005040 con 0x7f28ec0fdc20 2024-04-12T21:47:49.242 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:49.241+0000 7f28eb400700 1 --2- 172.21.15.18:0/2066550157 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f28ec0fdc20 0x7f28ec0febc0 secure :-1 s=READY pgs=1225 cs=0 l=1 rev1=1 crypto rx=0x7f28dc00c8c0 tx=0x7f28dc0044c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:47:49.242 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:49.241+0000 7f28e3400700 1 -- 172.21.15.18:0/2066550157 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f28dc00a980 con 0x7f28ec0fdc20 2024-04-12T21:47:49.242 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:49.241+0000 7f28f1000700 1 -- 172.21.15.18:0/2066550157 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f28ec0ff920 con 0x7f28ec0fdc20 2024-04-12T21:47:49.242 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:49.241+0000 7f28e3400700 1 -- 172.21.15.18:0/2066550157 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f28dc00ce60 con 0x7f28ec0fdc20 2024-04-12T21:47:49.242 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:49.241+0000 7f28e3400700 1 -- 172.21.15.18:0/2066550157 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f28dc0115e0 con 0x7f28ec0fdc20 2024-04-12T21:47:49.242 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:49.241+0000 7f28f1000700 1 -- 172.21.15.18:0/2066550157 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f28ec11e520 con 0x7f28ec0fdc20 2024-04-12T21:47:49.243 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:49.242+0000 7f28f1000700 1 -- 172.21.15.18:0/2066550157 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f28ec066770 con 0x7f28ec0fdc20 2024-04-12T21:47:49.243 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:49.243+0000 7f28e3400700 1 -- 172.21.15.18:0/2066550157 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7f28dc012400 con 0x7f28ec0fdc20 2024-04-12T21:47:49.245 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:49.245+0000 7f28e3400700 1 --2- 172.21.15.18:0/2066550157 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f28d006ec00 0x7f28d00710b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:47:49.245 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:49.245+0000 7f28e3400700 1 -- 172.21.15.18:0/2066550157 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 6757+0+0 (secure 0 0 0) 0x7f28dc08eb30 con 0x7f28ec0fdc20 2024-04-12T21:47:49.246 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:49.245+0000 7f28eaa00700 1 --2- 172.21.15.18:0/2066550157 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f28d006ec00 0x7f28d00710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:47:49.246 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:49.246+0000 7f28eaa00700 1 --2- 172.21.15.18:0/2066550157 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f28d006ec00 0x7f28d00710b0 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f28d8004f90 tx=0x7f28d8009e30 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:47:49.250 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:49.250+0000 7f28e3400700 1 -- 172.21.15.18:0/2066550157 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f28dc0577b0 con 0x7f28ec0fdc20 2024-04-12T21:47:49.284 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:47:48 smithi186 ceph-mon[86379]: from='client.36500 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:47:49.284 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:47:48 smithi186 ceph-mon[86379]: from='client.36504 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:47:49.284 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:47:48 smithi186 ceph-mon[86379]: from='client.? 172.21.15.18:0/600205219' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T21:47:49.284 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:47:48 smithi186 ceph-mon[86379]: from='client.? 172.21.15.18:0/1470323881' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-12T21:47:49.567 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:49.566+0000 7f28f1000700 1 -- 172.21.15.18:0/2066550157 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f28ec0fb2b0 con 0x7f28ec0fdc20 2024-04-12T21:47:49.567 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:49.567+0000 7f28e3400700 1 -- 172.21.15.18:0/2066550157 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+4303 (secure 0 0 0) 0x7f28dc0126b0 con 0x7f28ec0fdc20 2024-04-12T21:47:49.568 INFO:teuthology.orchestra.run.smithi018.stdout:HEALTH_ERR failed to probe daemons or devices; 1 filesystem is degraded; 1 filesystem is offline 2024-04-12T21:47:49.568 INFO:teuthology.orchestra.run.smithi018.stdout:[WRN] CEPHADM_REFRESH_FAILED: failed to probe daemons or devices 2024-04-12T21:47:49.568 INFO:teuthology.orchestra.run.smithi018.stdout: host smithi186 `cephadm ceph-volume` failed: cephadm exited with an error code: 1, stderr: Inferring config /var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/mon.smithi186/config 2024-04-12T21:47:49.568 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/podman: timeout after 895 seconds 2024-04-12T21:47:49.568 INFO:teuthology.orchestra.run.smithi018.stdout:Non-zero exit code 124 from /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:f4452def3be4d350f4a8e5feaf7ebb1511be9298fe68478ba7a2d6fbd7d69a63 -e NODE_NAME=smithi186 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968:/var/run/ceph:z -v /var/log/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968:/var/log/ceph:z -v /var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmptgom53f9:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:f4452def3be4d350f4a8e5feaf7ebb1511be9298fe68478ba7a2d6fbd7d69a63 inventory --format=json-pretty --filter-for-batch 2024-04-12T21:47:49.568 INFO:teuthology.orchestra.run.smithi018.stdout:Traceback (most recent call last): 2024-04-12T21:47:49.568 INFO:teuthology.orchestra.run.smithi018.stdout: File "/usr/lib64/python3.9/runpy.py", line 197, in _run_module_as_main 2024-04-12T21:47:49.568 INFO:teuthology.orchestra.run.smithi018.stdout: return _run_code(code, main_globals, None, 2024-04-12T21:47:49.568 INFO:teuthology.orchestra.run.smithi018.stdout: File "/usr/lib64/python3.9/runpy.py", line 87, in _run_code 2024-04-12T21:47:49.569 INFO:teuthology.orchestra.run.smithi018.stdout: exec(code, run_globals) 2024-04-12T21:47:49.569 INFO:teuthology.orchestra.run.smithi018.stdout: File "/var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/cephadm.e44f3ed5e34e4ef52eedef233d1190bff004470d8e8005127b5312bff357fa7f/__main__.py", line 5618, in 2024-04-12T21:47:49.569 INFO:teuthology.orchestra.run.smithi018.stdout: File "/var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/cephadm.e44f3ed5e34e4ef52eedef233d1190bff004470d8e8005127b5312bff357fa7f/__main__.py", line 5606, in main 2024-04-12T21:47:49.569 INFO:teuthology.orchestra.run.smithi018.stdout: File "/var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/cephadm.e44f3ed5e34e4ef52eedef233d1190bff004470d8e8005127b5312bff357fa7f/__main__.py", line 408, in _infer_config 2024-04-12T21:47:49.569 INFO:teuthology.orchestra.run.smithi018.stdout: File "/var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/cephadm.e44f3ed5e34e4ef52eedef233d1190bff004470d8e8005127b5312bff357fa7f/__main__.py", line 323, in _infer_fsid 2024-04-12T21:47:49.569 INFO:teuthology.orchestra.run.smithi018.stdout: File "/var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/cephadm.e44f3ed5e34e4ef52eedef233d1190bff004470d8e8005127b5312bff357fa7f/__main__.py", line 436, in _infer_image 2024-04-12T21:47:49.569 INFO:teuthology.orchestra.run.smithi018.stdout: File "/var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/cephadm.e44f3ed5e34e4ef52eedef233d1190bff004470d8e8005127b5312bff357fa7f/__main__.py", line 310, in _validate_fsid 2024-04-12T21:47:49.569 INFO:teuthology.orchestra.run.smithi018.stdout: File "/var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/cephadm.e44f3ed5e34e4ef52eedef233d1190bff004470d8e8005127b5312bff357fa7f/__main__.py", line 3359, in command_ceph_volume 2024-04-12T21:47:49.569 INFO:teuthology.orchestra.run.smithi018.stdout: File "/var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/cephadm.e44f3ed5e34e4ef52eedef233d1190bff004470d8e8005127b5312bff357fa7f/cephadmlib/call_wrappers.py", line 310, in call_throws 2024-04-12T21:47:49.569 INFO:teuthology.orchestra.run.smithi018.stdout:RuntimeError: Failed command: /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:f4452def3be4d350f4a8e5feaf7ebb1511be9298fe68478ba7a2d6fbd7d69a63 -e NODE_NAME=smithi186 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968:/var/run/ceph:z -v /var/log/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968:/var/log/ceph:z -v /var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmptgom53f9:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:f4452def3be4d350f4a8e5feaf7ebb1511be9298fe68478ba7a2d6fbd7d69a63 inventory --format=json-pretty --filter-for-batch 2024-04-12T21:47:49.569 INFO:teuthology.orchestra.run.smithi018.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-04-12T21:47:49.569 INFO:teuthology.orchestra.run.smithi018.stdout: fs cephfs is degraded 2024-04-12T21:47:49.570 INFO:teuthology.orchestra.run.smithi018.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-04-12T21:47:49.570 INFO:teuthology.orchestra.run.smithi018.stdout: fs cephfs is offline because no MDS is active for it. 2024-04-12T21:47:49.570 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:49.569+0000 7f28f1000700 1 -- 172.21.15.18:0/2066550157 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f28d006ec00 msgr2=0x7f28d00710b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:47:49.570 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:49.569+0000 7f28f1000700 1 --2- 172.21.15.18:0/2066550157 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f28d006ec00 0x7f28d00710b0 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f28d8004f90 tx=0x7f28d8009e30 comp rx=0 tx=0).stop 2024-04-12T21:47:49.570 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:49.569+0000 7f28f1000700 1 -- 172.21.15.18:0/2066550157 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f28ec0fdc20 msgr2=0x7f28ec0febc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:47:49.570 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:49.569+0000 7f28f1000700 1 --2- 172.21.15.18:0/2066550157 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f28ec0fdc20 0x7f28ec0febc0 secure :-1 s=READY pgs=1225 cs=0 l=1 rev1=1 crypto rx=0x7f28dc00c8c0 tx=0x7f28dc0044c0 comp rx=0 tx=0).stop 2024-04-12T21:47:49.571 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:49.570+0000 7f28f1000700 1 -- 172.21.15.18:0/2066550157 shutdown_connections 2024-04-12T21:47:49.571 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:49.570+0000 7f28f1000700 1 --2- 172.21.15.18:0/2066550157 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f28d006ec00 0x7f28d00710b0 unknown :-1 s=CLOSED pgs=650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:47:49.571 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:49.570+0000 7f28f1000700 1 --2- 172.21.15.18:0/2066550157 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f28ec0fdc20 0x7f28ec0febc0 unknown :-1 s=CLOSED pgs=1225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:47:49.571 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:49.570+0000 7f28f1000700 1 --2- 172.21.15.18:0/2066550157 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f28ec0fe5c0 0x7f28ec0ff100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:47:49.571 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:49.570+0000 7f28f1000700 1 -- 172.21.15.18:0/2066550157 >> 172.21.15.18:0/2066550157 conn(0x7f28ec0f7650 msgr2=0x7f28ec0f8be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:47:49.571 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:49.570+0000 7f28f1000700 1 -- 172.21.15.18:0/2066550157 shutdown_connections 2024-04-12T21:47:49.572 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:47:49.570+0000 7f28f1000700 1 -- 172.21.15.18:0/2066550157 wait complete. 2024-04-12T21:47:50.284 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:47:49 smithi186 ceph-mon[86379]: pgmap v1444: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:47:50.284 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:47:49 smithi186 ceph-mon[86379]: from='client.? 172.21.15.18:0/2066550157' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-12T21:47:50.355 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:47:49 smithi018 ceph-mon[112590]: pgmap v1444: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:47:50.355 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:47:49 smithi018 ceph-mon[112590]: from='client.? 172.21.15.18:0/2066550157' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-12T21:47:51.284 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:47:50 smithi186 ceph-mon[86379]: from='client.36516 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:47:51.355 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:47:50 smithi018 ceph-mon[112590]: from='client.36516 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:47:52.284 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:47:51 smithi186 ceph-mon[86379]: pgmap v1445: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:47:52.355 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:47:51 smithi018 ceph-mon[112590]: pgmap v1445: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:47:54.284 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:47:53 smithi186 ceph-mon[86379]: pgmap v1446: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:47:54.355 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:47:53 smithi018 ceph-mon[112590]: pgmap v1446: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:47:55.284 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:47:54 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T21:47:55.355 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:47:54 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T21:47:56.284 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:47:55 smithi186 ceph-mon[86379]: pgmap v1447: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:47:56.355 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:47:55 smithi018 ceph-mon[112590]: pgmap v1447: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:47:58.285 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:47:57 smithi186 ceph-mon[86379]: pgmap v1448: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:47:58.355 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:47:57 smithi018 ceph-mon[112590]: pgmap v1448: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:48:00.284 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:48:00 smithi186 ceph-mon[86379]: pgmap v1449: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:48:00.355 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:48:00 smithi018 ceph-mon[112590]: pgmap v1449: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:48:02.284 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:48:02 smithi186 ceph-mon[86379]: pgmap v1450: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:48:02.355 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:48:02 smithi018 ceph-mon[112590]: pgmap v1450: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:48:04.284 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:48:04 smithi186 ceph-mon[86379]: pgmap v1451: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:48:04.355 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:48:04 smithi018 ceph-mon[112590]: pgmap v1451: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:48:06.284 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:48:06 smithi186 ceph-mon[86379]: pgmap v1452: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:48:06.357 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:48:06 smithi018 ceph-mon[112590]: pgmap v1452: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:48:08.284 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:48:08 smithi186 ceph-mon[86379]: pgmap v1453: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:48:08.355 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:48:08 smithi018 ceph-mon[112590]: pgmap v1453: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:48:10.355 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:48:10 smithi018 ceph-mon[112590]: pgmap v1454: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:48:10.355 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:48:10 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T21:48:10.534 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:48:10 smithi186 ceph-mon[86379]: pgmap v1454: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:48:10.534 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:48:10 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T21:48:12.355 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:48:12 smithi018 ceph-mon[112590]: pgmap v1455: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:48:12.534 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:48:12 smithi186 ceph-mon[86379]: pgmap v1455: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:48:14.355 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:48:14 smithi018 ceph-mon[112590]: pgmap v1456: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:48:14.534 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:48:14 smithi186 ceph-mon[86379]: pgmap v1456: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:48:16.355 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:48:16 smithi018 ceph-mon[112590]: pgmap v1457: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:48:16.534 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:48:16 smithi186 ceph-mon[86379]: pgmap v1457: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:48:18.355 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:48:18 smithi018 ceph-mon[112590]: pgmap v1458: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:48:18.534 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:48:18 smithi186 ceph-mon[86379]: pgmap v1458: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:48:19.744 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:19.742+0000 7fbc68a00700 1 -- 172.21.15.18:0/170709412 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fbc64104e60 msgr2=0x7fbc64105230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:48:19.745 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:19.742+0000 7fbc68a00700 1 --2- 172.21.15.18:0/170709412 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fbc64104e60 0x7fbc64105230 secure :-1 s=READY pgs=1226 cs=0 l=1 rev1=1 crypto rx=0x7fbc54004940 tx=0x7fbc54009e30 comp rx=0 tx=0).stop 2024-04-12T21:48:19.745 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:19.742+0000 7fbc68a00700 1 -- 172.21.15.18:0/170709412 shutdown_connections 2024-04-12T21:48:19.745 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:19.742+0000 7fbc68a00700 1 --2- 172.21.15.18:0/170709412 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbc64105800 0x7fbc640fee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:48:19.745 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:19.742+0000 7fbc68a00700 1 --2- 172.21.15.18:0/170709412 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fbc64104e60 0x7fbc64105230 unknown :-1 s=CLOSED pgs=1226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:48:19.745 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:19.742+0000 7fbc68a00700 1 -- 172.21.15.18:0/170709412 >> 172.21.15.18:0/170709412 conn(0x7fbc640fab10 msgr2=0x7fbc640fcef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:48:19.745 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:19.742+0000 7fbc68a00700 1 -- 172.21.15.18:0/170709412 shutdown_connections 2024-04-12T21:48:19.745 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:19.743+0000 7fbc68a00700 1 -- 172.21.15.18:0/170709412 wait complete. 2024-04-12T21:48:19.747 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:19.743+0000 7fbc68a00700 1 Processor -- start 2024-04-12T21:48:19.747 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:19.744+0000 7fbc68a00700 1 -- start start 2024-04-12T21:48:19.748 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:19.744+0000 7fbc68a00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fbc64104e60 0x7fbc64115bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:48:19.748 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:19.744+0000 7fbc68a00700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbc64105800 0x7fbc641160f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:48:19.748 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:19.744+0000 7fbc68a00700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc64116740 con 0x7fbc64104e60 2024-04-12T21:48:19.748 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:19.744+0000 7fbc68a00700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc6410fb50 con 0x7fbc64105800 2024-04-12T21:48:19.748 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:19.744+0000 7fbc63400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fbc64104e60 0x7fbc64115bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:48:19.748 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:19.744+0000 7fbc63400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fbc64104e60 0x7fbc64115bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:45356/0 (socket says 172.21.15.18:45356) 2024-04-12T21:48:19.748 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:19.744+0000 7fbc63400700 1 -- 172.21.15.18:0/1503091349 learned_addr learned my addr 172.21.15.18:0/1503091349 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:48:19.748 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:19.745+0000 7fbc62a00700 1 --2- 172.21.15.18:0/1503091349 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbc64105800 0x7fbc641160f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:48:19.748 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:19.745+0000 7fbc63400700 1 -- 172.21.15.18:0/1503091349 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbc64105800 msgr2=0x7fbc641160f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:48:19.748 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:19.745+0000 7fbc63400700 1 --2- 172.21.15.18:0/1503091349 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbc64105800 0x7fbc641160f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:48:19.748 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:19.745+0000 7fbc63400700 1 -- 172.21.15.18:0/1503091349 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc54005040 con 0x7fbc64104e60 2024-04-12T21:48:19.748 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:19.745+0000 7fbc62a00700 1 --2- 172.21.15.18:0/1503091349 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbc64105800 0x7fbc641160f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-12T21:48:19.749 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:19.745+0000 7fbc63400700 1 --2- 172.21.15.18:0/1503091349 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fbc64104e60 0x7fbc64115bb0 secure :-1 s=READY pgs=1227 cs=0 l=1 rev1=1 crypto rx=0x7fbc5400b8b0 tx=0x7fbc54009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:48:19.749 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:19.745+0000 7fbc5b400700 1 -- 172.21.15.18:0/1503091349 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbc5400e040 con 0x7fbc64104e60 2024-04-12T21:48:19.749 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:19.746+0000 7fbc68a00700 1 -- 172.21.15.18:0/1503091349 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbc6410fdd0 con 0x7fbc64104e60 2024-04-12T21:48:19.749 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:19.746+0000 7fbc5b400700 1 -- 172.21.15.18:0/1503091349 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fbc54004290 con 0x7fbc64104e60 2024-04-12T21:48:19.749 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:19.746+0000 7fbc5b400700 1 -- 172.21.15.18:0/1503091349 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbc5401c3f0 con 0x7fbc64104e60 2024-04-12T21:48:19.749 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:19.746+0000 7fbc68a00700 1 -- 172.21.15.18:0/1503091349 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbc64110390 con 0x7fbc64104e60 2024-04-12T21:48:19.749 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:19.747+0000 7fbc68a00700 1 -- 172.21.15.18:0/1503091349 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbc641005b0 con 0x7fbc64104e60 2024-04-12T21:48:19.750 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:19.748+0000 7fbc5b400700 1 -- 172.21.15.18:0/1503091349 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7fbc540137b0 con 0x7fbc64104e60 2024-04-12T21:48:19.751 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:19.749+0000 7fbc5b400700 1 --2- 172.21.15.18:0/1503091349 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fbc4806ec00 0x7fbc480710b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:48:19.751 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:19.749+0000 7fbc5b400700 1 -- 172.21.15.18:0/1503091349 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 6757+0+0 (secure 0 0 0) 0x7fbc5408fca0 con 0x7fbc64104e60 2024-04-12T21:48:19.751 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:19.749+0000 7fbc62a00700 1 --2- 172.21.15.18:0/1503091349 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fbc4806ec00 0x7fbc480710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:48:19.751 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:19.750+0000 7fbc62a00700 1 --2- 172.21.15.18:0/1503091349 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fbc4806ec00 0x7fbc480710b0 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7fbc50009fd0 tx=0x7fbc5000b040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:48:19.756 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:19.754+0000 7fbc5b400700 1 -- 172.21.15.18:0/1503091349 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7fbc540589a0 con 0x7fbc64104e60 2024-04-12T21:48:20.007 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:20.005+0000 7fbc68a00700 1 -- 172.21.15.18:0/1503091349 --> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fbc6404bf40 con 0x7fbc4806ec00 2024-04-12T21:48:20.009 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:20.007+0000 7fbc5b400700 1 -- 172.21.15.18:0/1503091349 <== mgr.34102 v2:172.21.15.18:6800/3812891919 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7fbc6404bf40 con 0x7fbc4806ec00 2024-04-12T21:48:20.012 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:20.010+0000 7fbc68a00700 1 -- 172.21.15.18:0/1503091349 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fbc4806ec00 msgr2=0x7fbc480710b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:48:20.012 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:20.010+0000 7fbc68a00700 1 --2- 172.21.15.18:0/1503091349 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fbc4806ec00 0x7fbc480710b0 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7fbc50009fd0 tx=0x7fbc5000b040 comp rx=0 tx=0).stop 2024-04-12T21:48:20.012 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:20.010+0000 7fbc68a00700 1 -- 172.21.15.18:0/1503091349 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fbc64104e60 msgr2=0x7fbc64115bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:48:20.012 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:20.010+0000 7fbc68a00700 1 --2- 172.21.15.18:0/1503091349 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fbc64104e60 0x7fbc64115bb0 secure :-1 s=READY pgs=1227 cs=0 l=1 rev1=1 crypto rx=0x7fbc5400b8b0 tx=0x7fbc54009e30 comp rx=0 tx=0).stop 2024-04-12T21:48:20.012 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:20.010+0000 7fbc68a00700 1 -- 172.21.15.18:0/1503091349 shutdown_connections 2024-04-12T21:48:20.012 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:20.010+0000 7fbc68a00700 1 --2- 172.21.15.18:0/1503091349 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fbc4806ec00 0x7fbc480710b0 unknown :-1 s=CLOSED pgs=651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:48:20.012 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:20.010+0000 7fbc68a00700 1 --2- 172.21.15.18:0/1503091349 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fbc64104e60 0x7fbc64115bb0 unknown :-1 s=CLOSED pgs=1227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:48:20.013 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:20.010+0000 7fbc68a00700 1 --2- 172.21.15.18:0/1503091349 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbc64105800 0x7fbc641160f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:48:20.013 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:20.010+0000 7fbc68a00700 1 -- 172.21.15.18:0/1503091349 >> 172.21.15.18:0/1503091349 conn(0x7fbc640fab10 msgr2=0x7fbc640fca40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:48:20.013 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:20.010+0000 7fbc68a00700 1 -- 172.21.15.18:0/1503091349 shutdown_connections 2024-04-12T21:48:20.013 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:20.011+0000 7fbc68a00700 1 -- 172.21.15.18:0/1503091349 wait complete. 2024-04-12T21:48:20.029 INFO:teuthology.orchestra.run.smithi018.stdout:true 2024-04-12T21:48:20.172 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:20.170+0000 7f2a79000700 1 -- 172.21.15.18:0/2205509228 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2a74072e00 msgr2=0x7f2a7410d5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:48:20.172 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:20.170+0000 7f2a79000700 1 --2- 172.21.15.18:0/2205509228 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2a74072e00 0x7f2a7410d5c0 secure :-1 s=READY pgs=1228 cs=0 l=1 rev1=1 crypto rx=0x7f2a60004940 tx=0x7f2a60009e30 comp rx=0 tx=0).stop 2024-04-12T21:48:20.172 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:20.170+0000 7f2a79000700 1 -- 172.21.15.18:0/2205509228 shutdown_connections 2024-04-12T21:48:20.172 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:20.170+0000 7f2a79000700 1 --2- 172.21.15.18:0/2205509228 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2a74072e00 0x7f2a7410d5c0 unknown :-1 s=CLOSED pgs=1228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:48:20.172 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:20.170+0000 7f2a79000700 1 --2- 172.21.15.18:0/2205509228 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2a740724f0 0x7f2a740728c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:48:20.172 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:20.170+0000 7f2a79000700 1 -- 172.21.15.18:0/2205509228 >> 172.21.15.18:0/2205509228 conn(0x7f2a740fab10 msgr2=0x7f2a740fcef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:48:20.172 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:20.170+0000 7f2a79000700 1 -- 172.21.15.18:0/2205509228 shutdown_connections 2024-04-12T21:48:20.173 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:20.170+0000 7f2a79000700 1 -- 172.21.15.18:0/2205509228 wait complete. 2024-04-12T21:48:20.173 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:20.171+0000 7f2a79000700 1 Processor -- start 2024-04-12T21:48:20.173 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:20.171+0000 7f2a79000700 1 -- start start 2024-04-12T21:48:20.174 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:20.172+0000 7f2a79000700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2a740724f0 0x7f2a74119fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:48:20.174 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:20.172+0000 7f2a79000700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2a74072e00 0x7f2a7411a4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:48:20.174 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:20.172+0000 7f2a79000700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a7411ab40 con 0x7f2a740724f0 2024-04-12T21:48:20.174 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:20.172+0000 7f2a79000700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a74113f10 con 0x7f2a74072e00 2024-04-12T21:48:20.175 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:20.172+0000 7f2a73400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2a740724f0 0x7f2a74119fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:48:20.175 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:20.172+0000 7f2a73400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2a740724f0 0x7f2a74119fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:45370/0 (socket says 172.21.15.18:45370) 2024-04-12T21:48:20.175 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:20.172+0000 7f2a73400700 1 -- 172.21.15.18:0/1106814054 learned_addr learned my addr 172.21.15.18:0/1106814054 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:48:20.175 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:20.173+0000 7f2a72a00700 1 --2- 172.21.15.18:0/1106814054 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2a74072e00 0x7f2a7411a4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:48:20.175 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:20.173+0000 7f2a73400700 1 -- 172.21.15.18:0/1106814054 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2a74072e00 msgr2=0x7f2a7411a4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:48:20.175 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:20.173+0000 7f2a73400700 1 --2- 172.21.15.18:0/1106814054 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2a74072e00 0x7f2a7411a4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:48:20.175 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:20.173+0000 7f2a73400700 1 -- 172.21.15.18:0/1106814054 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a60005040 con 0x7f2a740724f0 2024-04-12T21:48:20.176 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:20.173+0000 7f2a73400700 1 --2- 172.21.15.18:0/1106814054 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2a740724f0 0x7f2a74119fb0 secure :-1 s=READY pgs=1229 cs=0 l=1 rev1=1 crypto rx=0x7f2a64002a50 tx=0x7f2a640046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:48:20.176 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:20.173+0000 7f2a6b400700 1 -- 172.21.15.18:0/1106814054 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2a64007cc0 con 0x7f2a740724f0 2024-04-12T21:48:20.176 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:20.174+0000 7f2a6b400700 1 -- 172.21.15.18:0/1106814054 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f2a6400f040 con 0x7f2a740724f0 2024-04-12T21:48:20.176 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:20.174+0000 7f2a79000700 1 -- 172.21.15.18:0/1106814054 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2a741141f0 con 0x7f2a740724f0 2024-04-12T21:48:20.176 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:20.174+0000 7f2a6b400700 1 -- 172.21.15.18:0/1106814054 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2a64009640 con 0x7f2a740724f0 2024-04-12T21:48:20.176 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:20.174+0000 7f2a79000700 1 -- 172.21.15.18:0/1106814054 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2a741147b0 con 0x7f2a740724f0 2024-04-12T21:48:20.176 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:20.175+0000 7f2a79000700 1 -- 172.21.15.18:0/1106814054 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2a74073c30 con 0x7f2a740724f0 2024-04-12T21:48:20.178 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:20.176+0000 7f2a6b400700 1 -- 172.21.15.18:0/1106814054 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7f2a640097a0 con 0x7f2a740724f0 2024-04-12T21:48:20.179 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:20.177+0000 7f2a6b400700 1 --2- 172.21.15.18:0/1106814054 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f2a5806ec00 0x7f2a580710b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:48:20.179 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:20.177+0000 7f2a6b400700 1 -- 172.21.15.18:0/1106814054 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 6757+0+0 (secure 0 0 0) 0x7f2a6408e760 con 0x7f2a740724f0 2024-04-12T21:48:20.179 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:20.177+0000 7f2a72a00700 1 --2- 172.21.15.18:0/1106814054 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f2a5806ec00 0x7f2a580710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:48:20.180 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:20.178+0000 7f2a72a00700 1 --2- 172.21.15.18:0/1106814054 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f2a5806ec00 0x7f2a580710b0 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f2a6000b3b0 tx=0x7f2a60009e30 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:48:20.184 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:20.182+0000 7f2a6b400700 1 -- 172.21.15.18:0/1106814054 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f2a6405af10 con 0x7f2a740724f0 2024-04-12T21:48:20.355 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:48:20 smithi018 ceph-mon[112590]: pgmap v1459: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:48:20.435 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:20.433+0000 7f2a79000700 1 -- 172.21.15.18:0/1106814054 --> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f2a7404bf40 con 0x7f2a5806ec00 2024-04-12T21:48:20.437 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:20.436+0000 7f2a6b400700 1 -- 172.21.15.18:0/1106814054 <== mgr.34102 v2:172.21.15.18:6800/3812891919 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f2a7404bf40 con 0x7f2a5806ec00 2024-04-12T21:48:20.441 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:20.438+0000 7f2a79000700 1 -- 172.21.15.18:0/1106814054 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f2a5806ec00 msgr2=0x7f2a580710b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:48:20.441 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:20.438+0000 7f2a79000700 1 --2- 172.21.15.18:0/1106814054 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f2a5806ec00 0x7f2a580710b0 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f2a6000b3b0 tx=0x7f2a60009e30 comp rx=0 tx=0).stop 2024-04-12T21:48:20.441 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:20.438+0000 7f2a79000700 1 -- 172.21.15.18:0/1106814054 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2a740724f0 msgr2=0x7f2a74119fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:48:20.441 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:20.438+0000 7f2a79000700 1 --2- 172.21.15.18:0/1106814054 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2a740724f0 0x7f2a74119fb0 secure :-1 s=READY pgs=1229 cs=0 l=1 rev1=1 crypto rx=0x7f2a64002a50 tx=0x7f2a640046b0 comp rx=0 tx=0).stop 2024-04-12T21:48:20.441 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:20.439+0000 7f2a79000700 1 -- 172.21.15.18:0/1106814054 shutdown_connections 2024-04-12T21:48:20.441 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:20.439+0000 7f2a79000700 1 --2- 172.21.15.18:0/1106814054 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f2a5806ec00 0x7f2a580710b0 unknown :-1 s=CLOSED pgs=652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:48:20.441 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:20.439+0000 7f2a79000700 1 --2- 172.21.15.18:0/1106814054 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2a740724f0 0x7f2a74119fb0 unknown :-1 s=CLOSED pgs=1229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:48:20.441 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:20.439+0000 7f2a79000700 1 --2- 172.21.15.18:0/1106814054 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2a74072e00 0x7f2a7411a4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:48:20.441 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:20.439+0000 7f2a79000700 1 -- 172.21.15.18:0/1106814054 >> 172.21.15.18:0/1106814054 conn(0x7f2a740fab10 msgr2=0x7f2a740faf00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:48:20.442 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:20.439+0000 7f2a79000700 1 -- 172.21.15.18:0/1106814054 shutdown_connections 2024-04-12T21:48:20.442 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:20.439+0000 7f2a79000700 1 -- 172.21.15.18:0/1106814054 wait complete. 2024-04-12T21:48:20.534 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:48:20 smithi186 ceph-mon[86379]: pgmap v1459: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:48:20.604 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:20.601+0000 7f749d800700 1 -- 172.21.15.18:0/1455445717 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7498070c70 msgr2=0x7f7498109110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:48:20.604 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:20.601+0000 7f749d800700 1 --2- 172.21.15.18:0/1455445717 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7498070c70 0x7f7498109110 secure :-1 s=READY pgs=1230 cs=0 l=1 rev1=1 crypto rx=0x7f7484004970 tx=0x7f7484009e30 comp rx=0 tx=0).stop 2024-04-12T21:48:20.604 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:20.601+0000 7f749d800700 1 -- 172.21.15.18:0/1455445717 shutdown_connections 2024-04-12T21:48:20.604 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:20.602+0000 7f749d800700 1 --2- 172.21.15.18:0/1455445717 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7498070c70 0x7f7498109110 unknown :-1 s=CLOSED pgs=1230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:48:20.604 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:20.602+0000 7f749d800700 1 --2- 172.21.15.18:0/1455445717 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7498070360 0x7f7498070730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:48:20.604 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:20.602+0000 7f749d800700 1 -- 172.21.15.18:0/1455445717 >> 172.21.15.18:0/1455445717 conn(0x7f74980f89b0 msgr2=0x7f74980fadc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:48:20.604 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:20.602+0000 7f749d800700 1 -- 172.21.15.18:0/1455445717 shutdown_connections 2024-04-12T21:48:20.604 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:20.602+0000 7f749d800700 1 -- 172.21.15.18:0/1455445717 wait complete. 2024-04-12T21:48:20.606 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:20.603+0000 7f749d800700 1 Processor -- start 2024-04-12T21:48:20.606 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:20.603+0000 7f749d800700 1 -- start start 2024-04-12T21:48:20.606 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:20.603+0000 7f749d800700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7498070360 0x7f7498115b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:48:20.606 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:20.603+0000 7f749d800700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7498070c70 0x7f74981160a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:48:20.606 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:20.603+0000 7f749d800700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74981166f0 con 0x7f7498070c70 2024-04-12T21:48:20.606 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:20.603+0000 7f749d800700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f749810fcd0 con 0x7f7498070360 2024-04-12T21:48:20.606 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:20.604+0000 7f7496000700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7498070c70 0x7f74981160a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:48:20.606 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:20.604+0000 7f7496000700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7498070c70 0x7f74981160a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:45386/0 (socket says 172.21.15.18:45386) 2024-04-12T21:48:20.607 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:20.604+0000 7f7496000700 1 -- 172.21.15.18:0/3049331236 learned_addr learned my addr 172.21.15.18:0/3049331236 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:48:20.607 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:20.604+0000 7f7496a00700 1 --2- 172.21.15.18:0/3049331236 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7498070360 0x7f7498115b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:48:20.607 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:20.604+0000 7f7496000700 1 -- 172.21.15.18:0/3049331236 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7498070360 msgr2=0x7f7498115b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:48:20.607 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:20.604+0000 7f7496000700 1 --2- 172.21.15.18:0/3049331236 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7498070360 0x7f7498115b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:48:20.607 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:20.604+0000 7f7496000700 1 -- 172.21.15.18:0/3049331236 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7484005040 con 0x7f7498070c70 2024-04-12T21:48:20.607 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:20.604+0000 7f7496000700 1 --2- 172.21.15.18:0/3049331236 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7498070c70 0x7f74981160a0 secure :-1 s=READY pgs=1231 cs=0 l=1 rev1=1 crypto rx=0x7f7484004f50 tx=0x7f7484009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:48:20.608 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:20.605+0000 7f748ea00700 1 -- 172.21.15.18:0/3049331236 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f748400e040 con 0x7f7498070c70 2024-04-12T21:48:20.608 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:20.605+0000 7f748ea00700 1 -- 172.21.15.18:0/3049331236 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f748400bc10 con 0x7f7498070c70 2024-04-12T21:48:20.608 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:20.605+0000 7f749d800700 1 -- 172.21.15.18:0/3049331236 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f749810ff50 con 0x7f7498070c70 2024-04-12T21:48:20.608 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:20.605+0000 7f748ea00700 1 -- 172.21.15.18:0/3049331236 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f748401c630 con 0x7f7498070c70 2024-04-12T21:48:20.608 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:20.605+0000 7f749d800700 1 -- 172.21.15.18:0/3049331236 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7498110490 con 0x7f7498070c70 2024-04-12T21:48:20.609 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:20.606+0000 7f749d800700 1 -- 172.21.15.18:0/3049331236 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7498071aa0 con 0x7f7498070c70 2024-04-12T21:48:20.609 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:20.607+0000 7f748ea00700 1 -- 172.21.15.18:0/3049331236 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7f7484013420 con 0x7f7498070c70 2024-04-12T21:48:20.610 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:20.608+0000 7f748ea00700 1 --2- 172.21.15.18:0/3049331236 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f747c06ec00 0x7f747c0710b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:48:20.610 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:20.608+0000 7f748ea00700 1 -- 172.21.15.18:0/3049331236 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 6757+0+0 (secure 0 0 0) 0x7f748408fd30 con 0x7f7498070c70 2024-04-12T21:48:20.610 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:20.608+0000 7f7496a00700 1 --2- 172.21.15.18:0/3049331236 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f747c06ec00 0x7f747c0710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:48:20.610 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:20.609+0000 7f7496a00700 1 --2- 172.21.15.18:0/3049331236 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f747c06ec00 0x7f747c0710b0 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f7488004d60 tx=0x7f748800b040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:48:20.615 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:20.613+0000 7f748ea00700 1 -- 172.21.15.18:0/3049331236 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f748405c4e0 con 0x7f7498070c70 2024-04-12T21:48:20.854 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:20.852+0000 7f749d800700 1 -- 172.21.15.18:0/3049331236 --> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f749810c190 con 0x7f747c06ec00 2024-04-12T21:48:20.866 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:20.865+0000 7f748ea00700 1 -- 172.21.15.18:0/3049331236 <== mgr.34102 v2:172.21.15.18:6800/3812891919 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7f748800f040 con 0x7f747c06ec00 2024-04-12T21:48:20.867 INFO:teuthology.orchestra.run.smithi018.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-12T21:48:20.867 INFO:teuthology.orchestra.run.smithi018.stdout:alertmanager.smithi018 smithi018 *:9093,9094 running (48m) 10m ago 63m 14.8M - 0.25.0 c8568f914cd2 c332090068e8 2024-04-12T21:48:20.867 INFO:teuthology.orchestra.run.smithi018.stdout:crash.smithi018 smithi018 running (45m) 10m ago 63m 6665k - 19.0.0-2879-g324baf8d 8717720dc5e5 b7f80d1f7982 2024-04-12T21:48:20.867 INFO:teuthology.orchestra.run.smithi018.stdout:crash.smithi186 smithi186 running (45m) 10m ago 62m 6660k - 19.0.0-2879-g324baf8d 8717720dc5e5 a4f55c81b623 2024-04-12T21:48:20.867 INFO:teuthology.orchestra.run.smithi018.stdout:grafana.smithi018 smithi018 *:3000 running (48m) 10m ago 63m 73.8M - 9.4.12 2bacad6d85d8 b08e3bf176dd 2024-04-12T21:48:20.867 INFO:teuthology.orchestra.run.smithi018.stdout:mds.cephfs.smithi018.acqbsc smithi018 running (41m) 10m ago 56m 21.9M - 19.0.0-2879-g324baf8d 8717720dc5e5 021f99ab24ae 2024-04-12T21:48:20.867 INFO:teuthology.orchestra.run.smithi018.stdout:mds.cephfs.smithi018.szjluo smithi018 running (40m) 10m ago 56m 21.7M - 19.0.0-2879-g324baf8d 8717720dc5e5 0003117479d4 2024-04-12T21:48:20.867 INFO:teuthology.orchestra.run.smithi018.stdout:mds.cephfs.smithi186.jdllyn smithi186 running (25m) 10m ago 56m 16.5M - 19.0.0-2879-g324baf8d 8717720dc5e5 21b1f46eef23 2024-04-12T21:48:20.867 INFO:teuthology.orchestra.run.smithi018.stdout:mds.cephfs.smithi186.wjnafs smithi186 running (10m) 10m ago 56m 11.0M - 19.0.0-2879-g324baf8d 8717720dc5e5 3cf1ef4148e2 2024-04-12T21:48:20.868 INFO:teuthology.orchestra.run.smithi018.stdout:mgr.smithi018.xjzwga smithi018 *:8443,9283,8765 running (51m) 10m ago 65m 517M - 19.0.0-2879-g324baf8d 8717720dc5e5 56088305eab7 2024-04-12T21:48:20.868 INFO:teuthology.orchestra.run.smithi018.stdout:mgr.smithi186.kqcfxk smithi186 *:8443,9283,8765 running (50m) 10m ago 62m 460M - 19.0.0-2879-g324baf8d 8717720dc5e5 09cd766e5e0d 2024-04-12T21:48:20.868 INFO:teuthology.orchestra.run.smithi018.stdout:mon.smithi018 smithi018 running (46m) 10m ago 65m 88.2M 2048M 19.0.0-2879-g324baf8d 8717720dc5e5 df3e201dfed6 2024-04-12T21:48:20.868 INFO:teuthology.orchestra.run.smithi018.stdout:mon.smithi186 smithi186 running (46m) 10m ago 62m 53.6M 2048M 19.0.0-2879-g324baf8d 8717720dc5e5 c77498da9b2b 2024-04-12T21:48:20.868 INFO:teuthology.orchestra.run.smithi018.stdout:node-exporter.smithi018 smithi018 *:9100 running (50m) 10m ago 63m 8468k - 1.5.0 0da6a335fe13 326c3d9c6847 2024-04-12T21:48:20.868 INFO:teuthology.orchestra.run.smithi018.stdout:node-exporter.smithi186 smithi186 *:9100 running (49m) 10m ago 62m 8166k - 1.5.0 0da6a335fe13 ec2112baeecf 2024-04-12T21:48:20.868 INFO:teuthology.orchestra.run.smithi018.stdout:osd.0 smithi018 running (44m) 10m ago 61m 123M 1559M 19.0.0-2879-g324baf8d 8717720dc5e5 60b1273ca047 2024-04-12T21:48:20.868 INFO:teuthology.orchestra.run.smithi018.stdout:osd.1 smithi018 running (44m) 10m ago 60m 95.4M 1559M 19.0.0-2879-g324baf8d 8717720dc5e5 170ce1e5fb8d 2024-04-12T21:48:20.868 INFO:teuthology.orchestra.run.smithi018.stdout:osd.2 smithi018 running (43m) 10m ago 59m 94.6M 1559M 19.0.0-2879-g324baf8d 8717720dc5e5 c2cbc6aa2839 2024-04-12T21:48:20.868 INFO:teuthology.orchestra.run.smithi018.stdout:osd.3 smithi186 running (43m) 10m ago 59m 101M 2581M 19.0.0-2879-g324baf8d 8717720dc5e5 0138e8262e4b 2024-04-12T21:48:20.868 INFO:teuthology.orchestra.run.smithi018.stdout:osd.4 smithi186 running (42m) 10m ago 58m 91.5M 2581M 19.0.0-2879-g324baf8d 8717720dc5e5 b20cb63b25c9 2024-04-12T21:48:20.868 INFO:teuthology.orchestra.run.smithi018.stdout:osd.5 smithi186 running (41m) 10m ago 58m 92.2M 2581M 19.0.0-2879-g324baf8d 8717720dc5e5 2513bea2610e 2024-04-12T21:48:20.869 INFO:teuthology.orchestra.run.smithi018.stdout:prometheus.smithi018 smithi018 *:9095 running (49m) 10m ago 62m 69.6M - 2.43.0 a07b618ecd1d f58841127e21 2024-04-12T21:48:20.870 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:20.867+0000 7f749d800700 1 -- 172.21.15.18:0/3049331236 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f747c06ec00 msgr2=0x7f747c0710b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:48:20.870 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:20.867+0000 7f749d800700 1 --2- 172.21.15.18:0/3049331236 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f747c06ec00 0x7f747c0710b0 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f7488004d60 tx=0x7f748800b040 comp rx=0 tx=0).stop 2024-04-12T21:48:20.870 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:20.868+0000 7f749d800700 1 -- 172.21.15.18:0/3049331236 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7498070c70 msgr2=0x7f74981160a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:48:20.870 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:20.868+0000 7f749d800700 1 --2- 172.21.15.18:0/3049331236 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7498070c70 0x7f74981160a0 secure :-1 s=READY pgs=1231 cs=0 l=1 rev1=1 crypto rx=0x7f7484004f50 tx=0x7f7484009e30 comp rx=0 tx=0).stop 2024-04-12T21:48:20.870 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:20.868+0000 7f749d800700 1 -- 172.21.15.18:0/3049331236 shutdown_connections 2024-04-12T21:48:20.870 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:20.868+0000 7f749d800700 1 --2- 172.21.15.18:0/3049331236 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f747c06ec00 0x7f747c0710b0 unknown :-1 s=CLOSED pgs=653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:48:20.870 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:20.868+0000 7f749d800700 1 --2- 172.21.15.18:0/3049331236 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7498070360 0x7f7498115b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:48:20.870 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:20.868+0000 7f749d800700 1 --2- 172.21.15.18:0/3049331236 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7498070c70 0x7f74981160a0 unknown :-1 s=CLOSED pgs=1231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:48:20.870 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:20.868+0000 7f749d800700 1 -- 172.21.15.18:0/3049331236 >> 172.21.15.18:0/3049331236 conn(0x7f74980f89b0 msgr2=0x7f74980f9d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:48:20.870 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:20.868+0000 7f749d800700 1 -- 172.21.15.18:0/3049331236 shutdown_connections 2024-04-12T21:48:20.871 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:20.868+0000 7f749d800700 1 -- 172.21.15.18:0/3049331236 wait complete. 2024-04-12T21:48:21.021 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:21.019+0000 7f4f88200700 1 -- 172.21.15.18:0/2902441677 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4f80103710 msgr2=0x7f4f800fcd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:48:21.021 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:21.019+0000 7f4f88200700 1 --2- 172.21.15.18:0/2902441677 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4f80103710 0x7f4f800fcd80 secure :-1 s=READY pgs=1232 cs=0 l=1 rev1=1 crypto rx=0x7f4f6c004940 tx=0x7f4f6c009e30 comp rx=0 tx=0).stop 2024-04-12T21:48:21.021 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:21.019+0000 7f4f88200700 1 -- 172.21.15.18:0/2902441677 shutdown_connections 2024-04-12T21:48:21.021 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:21.019+0000 7f4f88200700 1 --2- 172.21.15.18:0/2902441677 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4f80103710 0x7f4f800fcd80 unknown :-1 s=CLOSED pgs=1232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:48:21.021 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:21.019+0000 7f4f88200700 1 --2- 172.21.15.18:0/2902441677 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4f80102d70 0x7f4f80103140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:48:21.021 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:21.019+0000 7f4f88200700 1 -- 172.21.15.18:0/2902441677 >> 172.21.15.18:0/2902441677 conn(0x7f4f800f8920 msgr2=0x7f4f800fad30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:48:21.021 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:21.019+0000 7f4f88200700 1 -- 172.21.15.18:0/2902441677 shutdown_connections 2024-04-12T21:48:21.021 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:21.019+0000 7f4f88200700 1 -- 172.21.15.18:0/2902441677 wait complete. 2024-04-12T21:48:21.022 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:21.020+0000 7f4f88200700 1 Processor -- start 2024-04-12T21:48:21.022 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:21.020+0000 7f4f88200700 1 -- start start 2024-04-12T21:48:21.023 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:21.021+0000 7f4f88200700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4f80102d70 0x7f4f80115d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:48:21.023 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:21.021+0000 7f4f88200700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4f80103710 0x7f4f80116270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:48:21.023 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:21.021+0000 7f4f88200700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f801168c0 con 0x7f4f80103710 2024-04-12T21:48:21.023 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:21.021+0000 7f4f88200700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f8010fc90 con 0x7f4f80102d70 2024-04-12T21:48:21.023 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:21.021+0000 7f4f84e00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4f80103710 0x7f4f80116270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:48:21.023 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:21.021+0000 7f4f84e00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4f80103710 0x7f4f80116270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:46324/0 (socket says 172.21.15.18:46324) 2024-04-12T21:48:21.023 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:21.021+0000 7f4f84e00700 1 -- 172.21.15.18:0/2127441013 learned_addr learned my addr 172.21.15.18:0/2127441013 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:48:21.023 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:21.022+0000 7f4f85800700 1 --2- 172.21.15.18:0/2127441013 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4f80102d70 0x7f4f80115d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:48:21.024 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:21.022+0000 7f4f84e00700 1 -- 172.21.15.18:0/2127441013 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4f80102d70 msgr2=0x7f4f80115d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:48:21.024 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:21.022+0000 7f4f84e00700 1 --2- 172.21.15.18:0/2127441013 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4f80102d70 0x7f4f80115d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:48:21.024 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:21.022+0000 7f4f84e00700 1 -- 172.21.15.18:0/2127441013 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f6c005040 con 0x7f4f80103710 2024-04-12T21:48:21.024 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:21.022+0000 7f4f84e00700 1 --2- 172.21.15.18:0/2127441013 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4f80103710 0x7f4f80116270 secure :-1 s=READY pgs=1233 cs=0 l=1 rev1=1 crypto rx=0x7f4f6c009100 tx=0x7f4f6c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:48:21.025 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:21.022+0000 7f4f7d600700 1 -- 172.21.15.18:0/2127441013 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4f6c00e040 con 0x7f4f80103710 2024-04-12T21:48:21.025 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:21.022+0000 7f4f7d600700 1 -- 172.21.15.18:0/2127441013 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f4f6c013420 con 0x7f4f80103710 2024-04-12T21:48:21.025 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:21.023+0000 7f4f7d600700 1 -- 172.21.15.18:0/2127441013 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4f6c01c610 con 0x7f4f80103710 2024-04-12T21:48:21.025 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:21.023+0000 7f4f88200700 1 -- 172.21.15.18:0/2127441013 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4f8010ff10 con 0x7f4f80103710 2024-04-12T21:48:21.025 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:21.023+0000 7f4f88200700 1 -- 172.21.15.18:0/2127441013 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4f801104d0 con 0x7f4f80103710 2024-04-12T21:48:21.025 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:21.024+0000 7f4f88200700 1 -- 172.21.15.18:0/2127441013 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4f800fe4c0 con 0x7f4f80103710 2024-04-12T21:48:21.026 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:21.025+0000 7f4f7d600700 1 -- 172.21.15.18:0/2127441013 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7f4f6c013590 con 0x7f4f80103710 2024-04-12T21:48:21.027 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:21.026+0000 7f4f7d600700 1 --2- 172.21.15.18:0/2127441013 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f4f6806ec00 0x7f4f680710b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:48:21.027 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:21.026+0000 7f4f7d600700 1 -- 172.21.15.18:0/2127441013 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 6757+0+0 (secure 0 0 0) 0x7f4f6c08fbb0 con 0x7f4f80103710 2024-04-12T21:48:21.028 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:21.026+0000 7f4f85800700 1 --2- 172.21.15.18:0/2127441013 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f4f6806ec00 0x7f4f680710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:48:21.028 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:21.027+0000 7f4f85800700 1 --2- 172.21.15.18:0/2127441013 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f4f6806ec00 0x7f4f680710b0 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f4f74004c40 tx=0x7f4f74009040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:48:21.032 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:21.031+0000 7f4f7d600700 1 -- 172.21.15.18:0/2127441013 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f4f6c05c3e0 con 0x7f4f80103710 2024-04-12T21:48:21.105 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:48:21 smithi018 ceph-mon[112590]: from='client.36524 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:48:21.357 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:21.355+0000 7f4f88200700 1 -- 172.21.15.18:0/2127441013 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f4f801113b0 con 0x7f4f80103710 2024-04-12T21:48:21.358 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:21.356+0000 7f4f7d600700 1 -- 172.21.15.18:0/2127441013 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+623 (secure 0 0 0) 0x7f4f6c05bf70 con 0x7f4f80103710 2024-04-12T21:48:21.358 INFO:teuthology.orchestra.run.smithi018.stdout:{ 2024-04-12T21:48:21.359 INFO:teuthology.orchestra.run.smithi018.stdout: "mon": { 2024-04-12T21:48:21.359 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 19.0.0-2879-g324baf8d (324baf8ddb91dea46cac50cb535c306115791355) squid (dev)": 2 2024-04-12T21:48:21.359 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-12T21:48:21.359 INFO:teuthology.orchestra.run.smithi018.stdout: "mgr": { 2024-04-12T21:48:21.359 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 19.0.0-2879-g324baf8d (324baf8ddb91dea46cac50cb535c306115791355) squid (dev)": 2 2024-04-12T21:48:21.359 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-12T21:48:21.359 INFO:teuthology.orchestra.run.smithi018.stdout: "osd": { 2024-04-12T21:48:21.359 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 19.0.0-2879-g324baf8d (324baf8ddb91dea46cac50cb535c306115791355) squid (dev)": 6 2024-04-12T21:48:21.359 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-12T21:48:21.359 INFO:teuthology.orchestra.run.smithi018.stdout: "mds": { 2024-04-12T21:48:21.359 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 19.0.0-2879-g324baf8d (324baf8ddb91dea46cac50cb535c306115791355) squid (dev)": 4 2024-04-12T21:48:21.359 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-12T21:48:21.359 INFO:teuthology.orchestra.run.smithi018.stdout: "overall": { 2024-04-12T21:48:21.359 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 19.0.0-2879-g324baf8d (324baf8ddb91dea46cac50cb535c306115791355) squid (dev)": 14 2024-04-12T21:48:21.360 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-04-12T21:48:21.360 INFO:teuthology.orchestra.run.smithi018.stdout:} 2024-04-12T21:48:21.361 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:21.359+0000 7f4f88200700 1 -- 172.21.15.18:0/2127441013 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f4f6806ec00 msgr2=0x7f4f680710b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:48:21.361 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:21.359+0000 7f4f88200700 1 --2- 172.21.15.18:0/2127441013 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f4f6806ec00 0x7f4f680710b0 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f4f74004c40 tx=0x7f4f74009040 comp rx=0 tx=0).stop 2024-04-12T21:48:21.361 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:21.359+0000 7f4f88200700 1 -- 172.21.15.18:0/2127441013 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4f80103710 msgr2=0x7f4f80116270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:48:21.361 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:21.359+0000 7f4f88200700 1 --2- 172.21.15.18:0/2127441013 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4f80103710 0x7f4f80116270 secure :-1 s=READY pgs=1233 cs=0 l=1 rev1=1 crypto rx=0x7f4f6c009100 tx=0x7f4f6c009e30 comp rx=0 tx=0).stop 2024-04-12T21:48:21.362 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:21.359+0000 7f4f88200700 1 -- 172.21.15.18:0/2127441013 shutdown_connections 2024-04-12T21:48:21.362 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:21.359+0000 7f4f88200700 1 --2- 172.21.15.18:0/2127441013 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f4f6806ec00 0x7f4f680710b0 unknown :-1 s=CLOSED pgs=654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:48:21.362 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:21.359+0000 7f4f88200700 1 --2- 172.21.15.18:0/2127441013 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4f80102d70 0x7f4f80115d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:48:21.362 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:21.359+0000 7f4f88200700 1 --2- 172.21.15.18:0/2127441013 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4f80103710 0x7f4f80116270 unknown :-1 s=CLOSED pgs=1233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:48:21.362 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:21.359+0000 7f4f88200700 1 -- 172.21.15.18:0/2127441013 >> 172.21.15.18:0/2127441013 conn(0x7f4f800f8920 msgr2=0x7f4f800f9940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:48:21.362 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:21.360+0000 7f4f88200700 1 -- 172.21.15.18:0/2127441013 shutdown_connections 2024-04-12T21:48:21.362 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:21.360+0000 7f4f88200700 1 -- 172.21.15.18:0/2127441013 wait complete. 2024-04-12T21:48:21.519 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:21.516+0000 7fa110c00700 1 -- 172.21.15.18:0/1514923623 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa10c109dd0 msgr2=0x7fa10c103440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:48:21.519 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:21.516+0000 7fa110c00700 1 --2- 172.21.15.18:0/1514923623 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa10c109dd0 0x7fa10c103440 secure :-1 s=READY pgs=1234 cs=0 l=1 rev1=1 crypto rx=0x7fa0f8004940 tx=0x7fa0f8009e30 comp rx=0 tx=0).stop 2024-04-12T21:48:21.519 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:21.516+0000 7fa110c00700 1 -- 172.21.15.18:0/1514923623 shutdown_connections 2024-04-12T21:48:21.519 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:21.516+0000 7fa110c00700 1 --2- 172.21.15.18:0/1514923623 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa10c109dd0 0x7fa10c103440 unknown :-1 s=CLOSED pgs=1234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:48:21.519 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:21.516+0000 7fa110c00700 1 --2- 172.21.15.18:0/1514923623 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa10c109430 0x7fa10c109800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:48:21.519 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:21.516+0000 7fa110c00700 1 -- 172.21.15.18:0/1514923623 >> 172.21.15.18:0/1514923623 conn(0x7fa10c072c00 msgr2=0x7fa10c075010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:48:21.519 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:21.516+0000 7fa110c00700 1 -- 172.21.15.18:0/1514923623 shutdown_connections 2024-04-12T21:48:21.519 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:21.516+0000 7fa110c00700 1 -- 172.21.15.18:0/1514923623 wait complete. 2024-04-12T21:48:21.520 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:21.517+0000 7fa110c00700 1 Processor -- start 2024-04-12T21:48:21.521 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:21.517+0000 7fa110c00700 1 -- start start 2024-04-12T21:48:21.521 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:21.518+0000 7fa110c00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa10c109430 0x7fa10c119f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:48:21.522 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:21.518+0000 7fa110c00700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa10c109dd0 0x7fa10c11a4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:48:21.522 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:21.518+0000 7fa110c00700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa10c11ab10 con 0x7fa10c109430 2024-04-12T21:48:21.522 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:21.518+0000 7fa110c00700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa10c1140f0 con 0x7fa10c109dd0 2024-04-12T21:48:21.522 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:21.518+0000 7fa109e00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa10c109430 0x7fa10c119f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:48:21.522 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:21.518+0000 7fa109e00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa10c109430 0x7fa10c119f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:46340/0 (socket says 172.21.15.18:46340) 2024-04-12T21:48:21.522 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:21.518+0000 7fa109e00700 1 -- 172.21.15.18:0/3750733533 learned_addr learned my addr 172.21.15.18:0/3750733533 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:48:21.522 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:21.519+0000 7fa109400700 1 --2- 172.21.15.18:0/3750733533 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa10c109dd0 0x7fa10c11a4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:48:21.522 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:21.519+0000 7fa109e00700 1 -- 172.21.15.18:0/3750733533 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa10c109dd0 msgr2=0x7fa10c11a4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:48:21.522 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:21.519+0000 7fa109e00700 1 --2- 172.21.15.18:0/3750733533 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa10c109dd0 0x7fa10c11a4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:48:21.523 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:21.519+0000 7fa109e00700 1 -- 172.21.15.18:0/3750733533 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0f8005040 con 0x7fa10c109430 2024-04-12T21:48:21.523 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:21.519+0000 7fa109e00700 1 --2- 172.21.15.18:0/3750733533 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa10c109430 0x7fa10c119f80 secure :-1 s=READY pgs=1235 cs=0 l=1 rev1=1 crypto rx=0x7fa0fc002a50 tx=0x7fa0fc0046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:48:21.523 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:21.519+0000 7fa102000700 1 -- 172.21.15.18:0/3750733533 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa0fc007cc0 con 0x7fa10c109430 2024-04-12T21:48:21.523 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:21.519+0000 7fa102000700 1 -- 172.21.15.18:0/3750733533 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa0fc00f040 con 0x7fa10c109430 2024-04-12T21:48:21.523 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:21.519+0000 7fa110c00700 1 -- 172.21.15.18:0/3750733533 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa10c1143d0 con 0x7fa10c109430 2024-04-12T21:48:21.523 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:21.520+0000 7fa102000700 1 -- 172.21.15.18:0/3750733533 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa0fc009640 con 0x7fa10c109430 2024-04-12T21:48:21.524 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:21.520+0000 7fa110c00700 1 -- 172.21.15.18:0/3750733533 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa10c114990 con 0x7fa10c109430 2024-04-12T21:48:21.524 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:21.521+0000 7fa110c00700 1 -- 172.21.15.18:0/3750733533 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa10c104b80 con 0x7fa10c109430 2024-04-12T21:48:21.524 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:21.522+0000 7fa102000700 1 -- 172.21.15.18:0/3750733533 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7fa0fc0097a0 con 0x7fa10c109430 2024-04-12T21:48:21.525 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:21.523+0000 7fa102000700 1 --2- 172.21.15.18:0/3750733533 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fa0f006ec00 0x7fa0f00710b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:48:21.525 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:21.523+0000 7fa102000700 1 -- 172.21.15.18:0/3750733533 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 6757+0+0 (secure 0 0 0) 0x7fa0fc08e760 con 0x7fa10c109430 2024-04-12T21:48:21.525 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:21.524+0000 7fa109400700 1 --2- 172.21.15.18:0/3750733533 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fa0f006ec00 0x7fa0f00710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:48:21.526 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:21.524+0000 7fa109400700 1 --2- 172.21.15.18:0/3750733533 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fa0f006ec00 0x7fa0f00710b0 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7fa0f800b020 tx=0x7fa0f8009e30 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:48:21.530 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:21.528+0000 7fa102000700 1 -- 172.21.15.18:0/3750733533 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7fa0fc05af10 con 0x7fa10c109430 2024-04-12T21:48:21.534 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:48:21 smithi186 ceph-mon[86379]: from='client.36524 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:48:21.801 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:21.799+0000 7fa110c00700 1 -- 172.21.15.18:0/3750733533 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fa10c1155d0 con 0x7fa10c109430 2024-04-12T21:48:21.801 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:21.800+0000 7fa102000700 1 -- 172.21.15.18:0/3750733533 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 33 v33) v1 ==== 76+0+1876 (secure 0 0 0) 0x7fa0fc05aaa0 con 0x7fa10c109430 2024-04-12T21:48:21.803 INFO:teuthology.orchestra.run.smithi018.stdout:e33 2024-04-12T21:48:21.803 INFO:teuthology.orchestra.run.smithi018.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-12T21:48:21.803 INFO:teuthology.orchestra.run.smithi018.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-12T21:48:21.803 INFO:teuthology.orchestra.run.smithi018.stdout:legacy client fscid: 1 2024-04-12T21:48:21.803 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-12T21:48:21.803 INFO:teuthology.orchestra.run.smithi018.stdout:Filesystem 'cephfs' (1) 2024-04-12T21:48:21.803 INFO:teuthology.orchestra.run.smithi018.stdout:fs_name cephfs 2024-04-12T21:48:21.803 INFO:teuthology.orchestra.run.smithi018.stdout:epoch 24 2024-04-12T21:48:21.803 INFO:teuthology.orchestra.run.smithi018.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-04-12T21:48:21.803 INFO:teuthology.orchestra.run.smithi018.stdout:created 2024-04-12T20:51:57.362850+0000 2024-04-12T21:48:21.804 INFO:teuthology.orchestra.run.smithi018.stdout:modified 2024-04-12T21:06:59.717190+0000 2024-04-12T21:48:21.804 INFO:teuthology.orchestra.run.smithi018.stdout:tableserver 0 2024-04-12T21:48:21.804 INFO:teuthology.orchestra.run.smithi018.stdout:root 0 2024-04-12T21:48:21.804 INFO:teuthology.orchestra.run.smithi018.stdout:session_timeout 60 2024-04-12T21:48:21.804 INFO:teuthology.orchestra.run.smithi018.stdout:session_autoclose 300 2024-04-12T21:48:21.804 INFO:teuthology.orchestra.run.smithi018.stdout:max_file_size 1099511627776 2024-04-12T21:48:21.804 INFO:teuthology.orchestra.run.smithi018.stdout:max_xattr_size 65536 2024-04-12T21:48:21.804 INFO:teuthology.orchestra.run.smithi018.stdout:required_client_features {} 2024-04-12T21:48:21.805 INFO:teuthology.orchestra.run.smithi018.stdout:last_failure 0 2024-04-12T21:48:21.805 INFO:teuthology.orchestra.run.smithi018.stdout:last_failure_osd_epoch 100 2024-04-12T21:48:21.805 INFO:teuthology.orchestra.run.smithi018.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-12T21:48:21.805 INFO:teuthology.orchestra.run.smithi018.stdout:max_mds 2 2024-04-12T21:48:21.805 INFO:teuthology.orchestra.run.smithi018.stdout:in 0,1 2024-04-12T21:48:21.805 INFO:teuthology.orchestra.run.smithi018.stdout:up {} 2024-04-12T21:48:21.805 INFO:teuthology.orchestra.run.smithi018.stdout:failed 0,1 2024-04-12T21:48:21.805 INFO:teuthology.orchestra.run.smithi018.stdout:damaged 2024-04-12T21:48:21.805 INFO:teuthology.orchestra.run.smithi018.stdout:stopped 2024-04-12T21:48:21.805 INFO:teuthology.orchestra.run.smithi018.stdout:data_pools [3] 2024-04-12T21:48:21.806 INFO:teuthology.orchestra.run.smithi018.stdout:metadata_pool 2 2024-04-12T21:48:21.806 INFO:teuthology.orchestra.run.smithi018.stdout:inline_data disabled 2024-04-12T21:48:21.806 INFO:teuthology.orchestra.run.smithi018.stdout:balancer 2024-04-12T21:48:21.806 INFO:teuthology.orchestra.run.smithi018.stdout:bal_rank_mask -1 2024-04-12T21:48:21.806 INFO:teuthology.orchestra.run.smithi018.stdout:standby_count_wanted 1 2024-04-12T21:48:21.806 INFO:teuthology.orchestra.run.smithi018.stdout:qdb_cluster leader: 0 members: 2024-04-12T21:48:21.806 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-12T21:48:21.806 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-12T21:48:21.806 INFO:teuthology.orchestra.run.smithi018.stdout:Standby daemons: 2024-04-12T21:48:21.806 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-12T21:48:21.806 INFO:teuthology.orchestra.run.smithi018.stdout:[mds.cephfs.smithi018.acqbsc{-1:34436} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.18:6828/755179764,v1:172.21.15.18:6829/755179764] compat {c=[1],r=[1],i=[1fff]}] 2024-04-12T21:48:21.806 INFO:teuthology.orchestra.run.smithi018.stdout:[mds.cephfs.smithi018.szjluo{-1:34468} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.18:6826/2407992653,v1:172.21.15.18:6827/2407992653] compat {c=[1],r=[1],i=[1fff]}] 2024-04-12T21:48:21.807 INFO:teuthology.orchestra.run.smithi018.stdout:[mds.cephfs.smithi186.jdllyn{-1:44595} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.186:6824/3523444202,v1:172.21.15.186:6825/3523444202] compat {c=[1],r=[1],i=[1fff]}] 2024-04-12T21:48:21.807 INFO:teuthology.orchestra.run.smithi018.stdout:[mds.cephfs.smithi186.wjnafs{-1:44875} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.186:6826/3122443766,v1:172.21.15.186:6827/3122443766] compat {c=[1],r=[1],i=[1fff]}] 2024-04-12T21:48:21.808 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:21.802+0000 7fa110c00700 1 -- 172.21.15.18:0/3750733533 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fa0f006ec00 msgr2=0x7fa0f00710b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:48:21.808 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:21.802+0000 7fa110c00700 1 --2- 172.21.15.18:0/3750733533 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fa0f006ec00 0x7fa0f00710b0 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7fa0f800b020 tx=0x7fa0f8009e30 comp rx=0 tx=0).stop 2024-04-12T21:48:21.808 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:21.802+0000 7fa110c00700 1 -- 172.21.15.18:0/3750733533 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa10c109430 msgr2=0x7fa10c119f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:48:21.808 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:21.802+0000 7fa110c00700 1 --2- 172.21.15.18:0/3750733533 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa10c109430 0x7fa10c119f80 secure :-1 s=READY pgs=1235 cs=0 l=1 rev1=1 crypto rx=0x7fa0fc002a50 tx=0x7fa0fc0046b0 comp rx=0 tx=0).stop 2024-04-12T21:48:21.808 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:21.803+0000 7fa110c00700 1 -- 172.21.15.18:0/3750733533 shutdown_connections 2024-04-12T21:48:21.808 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:21.803+0000 7fa110c00700 1 --2- 172.21.15.18:0/3750733533 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fa0f006ec00 0x7fa0f00710b0 unknown :-1 s=CLOSED pgs=655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:48:21.808 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:21.803+0000 7fa110c00700 1 --2- 172.21.15.18:0/3750733533 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa10c109430 0x7fa10c119f80 unknown :-1 s=CLOSED pgs=1235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:48:21.808 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:21.803+0000 7fa110c00700 1 --2- 172.21.15.18:0/3750733533 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa10c109dd0 0x7fa10c11a4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:48:21.809 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:21.803+0000 7fa110c00700 1 -- 172.21.15.18:0/3750733533 >> 172.21.15.18:0/3750733533 conn(0x7fa10c072c00 msgr2=0x7fa10c074c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:48:21.809 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:21.803+0000 7fa110c00700 1 -- 172.21.15.18:0/3750733533 shutdown_connections 2024-04-12T21:48:21.809 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:21.803+0000 7fa110c00700 1 -- 172.21.15.18:0/3750733533 wait complete. 2024-04-12T21:48:21.809 INFO:teuthology.orchestra.run.smithi018.stderr:dumped fsmap epoch 33 2024-04-12T21:48:21.957 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:21.954+0000 7ff364e00700 1 -- 172.21.15.18:0/1695473171 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff3600fe830 msgr2=0x7ff360106f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:48:21.957 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:21.954+0000 7ff364e00700 1 --2- 172.21.15.18:0/1695473171 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff3600fe830 0x7ff360106f00 secure :-1 s=READY pgs=1236 cs=0 l=1 rev1=1 crypto rx=0x7ff348004940 tx=0x7ff348009e30 comp rx=0 tx=0).stop 2024-04-12T21:48:21.957 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:21.954+0000 7ff364e00700 1 -- 172.21.15.18:0/1695473171 shutdown_connections 2024-04-12T21:48:21.957 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:21.954+0000 7ff364e00700 1 --2- 172.21.15.18:0/1695473171 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff3600fe830 0x7ff360106f00 unknown :-1 s=CLOSED pgs=1236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:48:21.957 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:21.954+0000 7ff364e00700 1 --2- 172.21.15.18:0/1695473171 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff3600fde90 0x7ff3600fe260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:48:21.957 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:21.954+0000 7ff364e00700 1 -- 172.21.15.18:0/1695473171 >> 172.21.15.18:0/1695473171 conn(0x7ff360072bc0 msgr2=0x7ff360072fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:48:21.957 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:21.955+0000 7ff364e00700 1 -- 172.21.15.18:0/1695473171 shutdown_connections 2024-04-12T21:48:21.957 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:21.955+0000 7ff364e00700 1 -- 172.21.15.18:0/1695473171 wait complete. 2024-04-12T21:48:21.959 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:21.955+0000 7ff364e00700 1 Processor -- start 2024-04-12T21:48:21.960 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:21.956+0000 7ff364e00700 1 -- start start 2024-04-12T21:48:21.960 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:21.956+0000 7ff364e00700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff3600fde90 0x7ff360115d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:48:21.960 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:21.956+0000 7ff364e00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff3600fe830 0x7ff3601162c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:48:21.960 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:21.956+0000 7ff364e00700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3601169a0 con 0x7ff3600fe830 2024-04-12T21:48:21.960 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:21.956+0000 7ff364e00700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff36010fce0 con 0x7ff3600fde90 2024-04-12T21:48:21.960 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:21.956+0000 7ff355400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff3600fe830 0x7ff3601162c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:48:21.960 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:21.956+0000 7ff355400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff3600fe830 0x7ff3601162c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:46362/0 (socket says 172.21.15.18:46362) 2024-04-12T21:48:21.960 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:21.956+0000 7ff355400700 1 -- 172.21.15.18:0/194210787 learned_addr learned my addr 172.21.15.18:0/194210787 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:48:21.960 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:21.957+0000 7ff35de00700 1 --2- 172.21.15.18:0/194210787 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff3600fde90 0x7ff360115d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:48:21.960 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:21.957+0000 7ff355400700 1 -- 172.21.15.18:0/194210787 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff3600fde90 msgr2=0x7ff360115d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:48:21.960 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:21.957+0000 7ff355400700 1 --2- 172.21.15.18:0/194210787 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff3600fde90 0x7ff360115d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:48:21.960 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:21.957+0000 7ff355400700 1 -- 172.21.15.18:0/194210787 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff348005040 con 0x7ff3600fe830 2024-04-12T21:48:21.960 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:21.957+0000 7ff355400700 1 --2- 172.21.15.18:0/194210787 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff3600fe830 0x7ff3601162c0 secure :-1 s=READY pgs=1237 cs=0 l=1 rev1=1 crypto rx=0x7ff348000c00 tx=0x7ff348009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:48:21.961 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:21.957+0000 7ff356a00700 1 -- 172.21.15.18:0/194210787 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff34800e040 con 0x7ff3600fe830 2024-04-12T21:48:21.961 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:21.957+0000 7ff356a00700 1 -- 172.21.15.18:0/194210787 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff348013420 con 0x7ff3600fe830 2024-04-12T21:48:21.961 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:21.957+0000 7ff364e00700 1 -- 172.21.15.18:0/194210787 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff36010ff00 con 0x7ff3600fe830 2024-04-12T21:48:21.961 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:21.957+0000 7ff356a00700 1 -- 172.21.15.18:0/194210787 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff34801c610 con 0x7ff3600fe830 2024-04-12T21:48:21.961 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:21.958+0000 7ff364e00700 1 -- 172.21.15.18:0/194210787 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff360110440 con 0x7ff3600fe830 2024-04-12T21:48:21.961 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:21.959+0000 7ff364e00700 1 -- 172.21.15.18:0/194210787 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff360106a20 con 0x7ff3600fe830 2024-04-12T21:48:21.962 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:21.960+0000 7ff356a00700 1 -- 172.21.15.18:0/194210787 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7ff348013590 con 0x7ff3600fe830 2024-04-12T21:48:21.963 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:21.961+0000 7ff356a00700 1 --2- 172.21.15.18:0/194210787 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7ff34406ec10 0x7ff3440710c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:48:21.963 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:21.961+0000 7ff356a00700 1 -- 172.21.15.18:0/194210787 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 6757+0+0 (secure 0 0 0) 0x7ff34808fbd0 con 0x7ff3600fe830 2024-04-12T21:48:21.963 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:21.961+0000 7ff35de00700 1 --2- 172.21.15.18:0/194210787 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7ff34406ec10 0x7ff3440710c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:48:21.963 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:21.962+0000 7ff35de00700 1 --2- 172.21.15.18:0/194210787 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7ff34406ec10 0x7ff3440710c0 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7ff350001250 tx=0x7ff350009620 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:48:21.967 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:21.966+0000 7ff356a00700 1 -- 172.21.15.18:0/194210787 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7ff34805c380 con 0x7ff3600fe830 2024-04-12T21:48:22.222 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:22.220+0000 7ff364e00700 1 -- 172.21.15.18:0/194210787 --> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7ff360109f80 con 0x7ff34406ec10 2024-04-12T21:48:22.223 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:22.221+0000 7ff356a00700 1 -- 172.21.15.18:0/194210787 <== mgr.34102 v2:172.21.15.18:6800/3812891919 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7ff360109f80 con 0x7ff34406ec10 2024-04-12T21:48:22.223 INFO:teuthology.orchestra.run.smithi018.stdout:{ 2024-04-12T21:48:22.224 INFO:teuthology.orchestra.run.smithi018.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:f4452def3be4d350f4a8e5feaf7ebb1511be9298fe68478ba7a2d6fbd7d69a63", 2024-04-12T21:48:22.224 INFO:teuthology.orchestra.run.smithi018.stdout: "in_progress": true, 2024-04-12T21:48:22.224 INFO:teuthology.orchestra.run.smithi018.stdout: "which": "Upgrading all daemon types on all hosts", 2024-04-12T21:48:22.224 INFO:teuthology.orchestra.run.smithi018.stdout: "services_complete": [ 2024-04-12T21:48:22.224 INFO:teuthology.orchestra.run.smithi018.stdout: "mgr", 2024-04-12T21:48:22.224 INFO:teuthology.orchestra.run.smithi018.stdout: "mds", 2024-04-12T21:48:22.224 INFO:teuthology.orchestra.run.smithi018.stdout: "crash", 2024-04-12T21:48:22.224 INFO:teuthology.orchestra.run.smithi018.stdout: "osd", 2024-04-12T21:48:22.224 INFO:teuthology.orchestra.run.smithi018.stdout: "mon" 2024-04-12T21:48:22.224 INFO:teuthology.orchestra.run.smithi018.stdout: ], 2024-04-12T21:48:22.224 INFO:teuthology.orchestra.run.smithi018.stdout: "progress": "16/21 daemons upgraded", 2024-04-12T21:48:22.224 INFO:teuthology.orchestra.run.smithi018.stdout: "message": "Currently upgrading mds daemons", 2024-04-12T21:48:22.224 INFO:teuthology.orchestra.run.smithi018.stdout: "is_paused": false 2024-04-12T21:48:22.224 INFO:teuthology.orchestra.run.smithi018.stdout:} 2024-04-12T21:48:22.226 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:22.224+0000 7ff364e00700 1 -- 172.21.15.18:0/194210787 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7ff34406ec10 msgr2=0x7ff3440710c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:48:22.226 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:22.224+0000 7ff364e00700 1 --2- 172.21.15.18:0/194210787 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7ff34406ec10 0x7ff3440710c0 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7ff350001250 tx=0x7ff350009620 comp rx=0 tx=0).stop 2024-04-12T21:48:22.226 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:22.224+0000 7ff364e00700 1 -- 172.21.15.18:0/194210787 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff3600fe830 msgr2=0x7ff3601162c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:48:22.226 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:22.224+0000 7ff364e00700 1 --2- 172.21.15.18:0/194210787 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff3600fe830 0x7ff3601162c0 secure :-1 s=READY pgs=1237 cs=0 l=1 rev1=1 crypto rx=0x7ff348000c00 tx=0x7ff348009e30 comp rx=0 tx=0).stop 2024-04-12T21:48:22.227 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:22.225+0000 7ff364e00700 1 -- 172.21.15.18:0/194210787 shutdown_connections 2024-04-12T21:48:22.227 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:22.225+0000 7ff364e00700 1 --2- 172.21.15.18:0/194210787 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7ff34406ec10 0x7ff3440710c0 unknown :-1 s=CLOSED pgs=656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:48:22.227 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:22.225+0000 7ff364e00700 1 --2- 172.21.15.18:0/194210787 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff3600fde90 0x7ff360115d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:48:22.227 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:22.225+0000 7ff364e00700 1 --2- 172.21.15.18:0/194210787 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff3600fe830 0x7ff3601162c0 unknown :-1 s=CLOSED pgs=1237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:48:22.228 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:22.225+0000 7ff364e00700 1 -- 172.21.15.18:0/194210787 >> 172.21.15.18:0/194210787 conn(0x7ff360072bc0 msgr2=0x7ff3600fb5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:48:22.228 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:22.225+0000 7ff364e00700 1 -- 172.21.15.18:0/194210787 shutdown_connections 2024-04-12T21:48:22.228 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:22.225+0000 7ff364e00700 1 -- 172.21.15.18:0/194210787 wait complete. 2024-04-12T21:48:22.355 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:48:22 smithi018 ceph-mon[112590]: pgmap v1460: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:48:22.355 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:48:22 smithi018 ceph-mon[112590]: from='client.36528 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:48:22.355 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:48:22 smithi018 ceph-mon[112590]: from='client.36532 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:48:22.355 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:48:22 smithi018 ceph-mon[112590]: from='client.? 172.21.15.18:0/2127441013' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T21:48:22.356 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:48:22 smithi018 ceph-mon[112590]: from='client.? 172.21.15.18:0/3750733533' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-12T21:48:22.389 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:22.386+0000 7fb826600700 1 -- 172.21.15.18:0/3694236207 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb820102710 msgr2=0x7fb8200fbd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:48:22.389 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:22.386+0000 7fb826600700 1 --2- 172.21.15.18:0/3694236207 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb820102710 0x7fb8200fbd80 secure :-1 s=READY pgs=1238 cs=0 l=1 rev1=1 crypto rx=0x7fb80c004940 tx=0x7fb80c009e30 comp rx=0 tx=0).stop 2024-04-12T21:48:22.389 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:22.387+0000 7fb826600700 1 -- 172.21.15.18:0/3694236207 shutdown_connections 2024-04-12T21:48:22.389 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:22.387+0000 7fb826600700 1 --2- 172.21.15.18:0/3694236207 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb820102710 0x7fb8200fbd80 unknown :-1 s=CLOSED pgs=1238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:48:22.389 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:22.387+0000 7fb826600700 1 --2- 172.21.15.18:0/3694236207 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb820101d70 0x7fb820102140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:48:22.389 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:22.387+0000 7fb826600700 1 -- 172.21.15.18:0/3694236207 >> 172.21.15.18:0/3694236207 conn(0x7fb820072bc0 msgr2=0x7fb820072fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:48:22.389 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:22.387+0000 7fb826600700 1 -- 172.21.15.18:0/3694236207 shutdown_connections 2024-04-12T21:48:22.389 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:22.387+0000 7fb826600700 1 -- 172.21.15.18:0/3694236207 wait complete. 2024-04-12T21:48:22.389 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:22.388+0000 7fb826600700 1 Processor -- start 2024-04-12T21:48:22.390 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:22.388+0000 7fb826600700 1 -- start start 2024-04-12T21:48:22.391 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:22.388+0000 7fb826600700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb820101d70 0x7fb820115dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:48:22.391 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:22.388+0000 7fb826600700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb820102710 0x7fb820116300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:48:22.391 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:22.388+0000 7fb826600700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb820116950 con 0x7fb820101d70 2024-04-12T21:48:22.391 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:22.388+0000 7fb826600700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb82010fd20 con 0x7fb820102710 2024-04-12T21:48:22.391 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:22.388+0000 7fb81f400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb820101d70 0x7fb820115dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:48:22.391 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:22.389+0000 7fb81f400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb820101d70 0x7fb820115dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:46378/0 (socket says 172.21.15.18:46378) 2024-04-12T21:48:22.391 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:22.389+0000 7fb81f400700 1 -- 172.21.15.18:0/403676454 learned_addr learned my addr 172.21.15.18:0/403676454 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:48:22.391 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:22.389+0000 7fb81f400700 1 -- 172.21.15.18:0/403676454 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb820102710 msgr2=0x7fb820116300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:48:22.391 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:22.389+0000 7fb81f400700 1 --2- 172.21.15.18:0/403676454 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb820102710 0x7fb820116300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:48:22.391 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:22.389+0000 7fb81f400700 1 -- 172.21.15.18:0/403676454 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb80c005040 con 0x7fb820101d70 2024-04-12T21:48:22.392 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:22.389+0000 7fb81f400700 1 --2- 172.21.15.18:0/403676454 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb820101d70 0x7fb820115dc0 secure :-1 s=READY pgs=1239 cs=0 l=1 rev1=1 crypto rx=0x7fb810002a50 tx=0x7fb8100046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:48:22.392 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:22.390+0000 7fb817400700 1 -- 172.21.15.18:0/403676454 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb810007cc0 con 0x7fb820101d70 2024-04-12T21:48:22.392 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:22.390+0000 7fb817400700 1 -- 172.21.15.18:0/403676454 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb81000f040 con 0x7fb820101d70 2024-04-12T21:48:22.392 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:22.390+0000 7fb817400700 1 -- 172.21.15.18:0/403676454 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb81000db70 con 0x7fb820101d70 2024-04-12T21:48:22.392 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:22.390+0000 7fb826600700 1 -- 172.21.15.18:0/403676454 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb820110000 con 0x7fb820101d70 2024-04-12T21:48:22.393 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:22.390+0000 7fb826600700 1 -- 172.21.15.18:0/403676454 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb8201105c0 con 0x7fb820101d70 2024-04-12T21:48:22.393 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:22.391+0000 7fb826600700 1 -- 172.21.15.18:0/403676454 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb8200fd4c0 con 0x7fb820101d70 2024-04-12T21:48:22.394 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:22.392+0000 7fb817400700 1 -- 172.21.15.18:0/403676454 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7fb81000dd10 con 0x7fb820101d70 2024-04-12T21:48:22.395 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:22.393+0000 7fb817400700 1 --2- 172.21.15.18:0/403676454 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fb80406ec10 0x7fb8040710c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:48:22.395 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:22.393+0000 7fb817400700 1 -- 172.21.15.18:0/403676454 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 6757+0+0 (secure 0 0 0) 0x7fb81008e0b0 con 0x7fb820101d70 2024-04-12T21:48:22.395 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:22.394+0000 7fb81ea00700 1 --2- 172.21.15.18:0/403676454 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fb80406ec10 0x7fb8040710c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:48:22.396 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:22.394+0000 7fb81ea00700 1 --2- 172.21.15.18:0/403676454 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fb80406ec10 0x7fb8040710c0 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7fb80c004f90 tx=0x7fb80c009e30 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:48:22.400 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:22.398+0000 7fb817400700 1 -- 172.21.15.18:0/403676454 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7fb81005a880 con 0x7fb820101d70 2024-04-12T21:48:22.534 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:48:22 smithi186 ceph-mon[86379]: pgmap v1460: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:48:22.535 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:48:22 smithi186 ceph-mon[86379]: from='client.36528 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:48:22.535 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:48:22 smithi186 ceph-mon[86379]: from='client.36532 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:48:22.535 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:48:22 smithi186 ceph-mon[86379]: from='client.? 172.21.15.18:0/2127441013' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T21:48:22.535 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:48:22 smithi186 ceph-mon[86379]: from='client.? 172.21.15.18:0/3750733533' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-12T21:48:22.720 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:22.718+0000 7fb826600700 1 -- 172.21.15.18:0/403676454 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7fb82005f5f0 con 0x7fb820101d70 2024-04-12T21:48:22.721 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:22.718+0000 7fb817400700 1 -- 172.21.15.18:0/403676454 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+4303 (secure 0 0 0) 0x7fb81005a410 con 0x7fb820101d70 2024-04-12T21:48:22.721 INFO:teuthology.orchestra.run.smithi018.stdout:HEALTH_ERR failed to probe daemons or devices; 1 filesystem is degraded; 1 filesystem is offline 2024-04-12T21:48:22.721 INFO:teuthology.orchestra.run.smithi018.stdout:[WRN] CEPHADM_REFRESH_FAILED: failed to probe daemons or devices 2024-04-12T21:48:22.721 INFO:teuthology.orchestra.run.smithi018.stdout: host smithi186 `cephadm ceph-volume` failed: cephadm exited with an error code: 1, stderr: Inferring config /var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/mon.smithi186/config 2024-04-12T21:48:22.722 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/podman: timeout after 895 seconds 2024-04-12T21:48:22.722 INFO:teuthology.orchestra.run.smithi018.stdout:Non-zero exit code 124 from /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:f4452def3be4d350f4a8e5feaf7ebb1511be9298fe68478ba7a2d6fbd7d69a63 -e NODE_NAME=smithi186 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968:/var/run/ceph:z -v /var/log/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968:/var/log/ceph:z -v /var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmptgom53f9:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:f4452def3be4d350f4a8e5feaf7ebb1511be9298fe68478ba7a2d6fbd7d69a63 inventory --format=json-pretty --filter-for-batch 2024-04-12T21:48:22.722 INFO:teuthology.orchestra.run.smithi018.stdout:Traceback (most recent call last): 2024-04-12T21:48:22.722 INFO:teuthology.orchestra.run.smithi018.stdout: File "/usr/lib64/python3.9/runpy.py", line 197, in _run_module_as_main 2024-04-12T21:48:22.722 INFO:teuthology.orchestra.run.smithi018.stdout: return _run_code(code, main_globals, None, 2024-04-12T21:48:22.722 INFO:teuthology.orchestra.run.smithi018.stdout: File "/usr/lib64/python3.9/runpy.py", line 87, in _run_code 2024-04-12T21:48:22.722 INFO:teuthology.orchestra.run.smithi018.stdout: exec(code, run_globals) 2024-04-12T21:48:22.722 INFO:teuthology.orchestra.run.smithi018.stdout: File "/var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/cephadm.e44f3ed5e34e4ef52eedef233d1190bff004470d8e8005127b5312bff357fa7f/__main__.py", line 5618, in 2024-04-12T21:48:22.722 INFO:teuthology.orchestra.run.smithi018.stdout: File "/var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/cephadm.e44f3ed5e34e4ef52eedef233d1190bff004470d8e8005127b5312bff357fa7f/__main__.py", line 5606, in main 2024-04-12T21:48:22.722 INFO:teuthology.orchestra.run.smithi018.stdout: File "/var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/cephadm.e44f3ed5e34e4ef52eedef233d1190bff004470d8e8005127b5312bff357fa7f/__main__.py", line 408, in _infer_config 2024-04-12T21:48:22.722 INFO:teuthology.orchestra.run.smithi018.stdout: File "/var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/cephadm.e44f3ed5e34e4ef52eedef233d1190bff004470d8e8005127b5312bff357fa7f/__main__.py", line 323, in _infer_fsid 2024-04-12T21:48:22.722 INFO:teuthology.orchestra.run.smithi018.stdout: File "/var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/cephadm.e44f3ed5e34e4ef52eedef233d1190bff004470d8e8005127b5312bff357fa7f/__main__.py", line 436, in _infer_image 2024-04-12T21:48:22.722 INFO:teuthology.orchestra.run.smithi018.stdout: File "/var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/cephadm.e44f3ed5e34e4ef52eedef233d1190bff004470d8e8005127b5312bff357fa7f/__main__.py", line 310, in _validate_fsid 2024-04-12T21:48:22.722 INFO:teuthology.orchestra.run.smithi018.stdout: File "/var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/cephadm.e44f3ed5e34e4ef52eedef233d1190bff004470d8e8005127b5312bff357fa7f/__main__.py", line 3359, in command_ceph_volume 2024-04-12T21:48:22.723 INFO:teuthology.orchestra.run.smithi018.stdout: File "/var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/cephadm.e44f3ed5e34e4ef52eedef233d1190bff004470d8e8005127b5312bff357fa7f/cephadmlib/call_wrappers.py", line 310, in call_throws 2024-04-12T21:48:22.723 INFO:teuthology.orchestra.run.smithi018.stdout:RuntimeError: Failed command: /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:f4452def3be4d350f4a8e5feaf7ebb1511be9298fe68478ba7a2d6fbd7d69a63 -e NODE_NAME=smithi186 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968:/var/run/ceph:z -v /var/log/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968:/var/log/ceph:z -v /var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmptgom53f9:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:f4452def3be4d350f4a8e5feaf7ebb1511be9298fe68478ba7a2d6fbd7d69a63 inventory --format=json-pretty --filter-for-batch 2024-04-12T21:48:22.723 INFO:teuthology.orchestra.run.smithi018.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-04-12T21:48:22.723 INFO:teuthology.orchestra.run.smithi018.stdout: fs cephfs is degraded 2024-04-12T21:48:22.723 INFO:teuthology.orchestra.run.smithi018.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-04-12T21:48:22.723 INFO:teuthology.orchestra.run.smithi018.stdout: fs cephfs is offline because no MDS is active for it. 2024-04-12T21:48:22.724 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:22.722+0000 7fb826600700 1 -- 172.21.15.18:0/403676454 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fb80406ec10 msgr2=0x7fb8040710c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:48:22.724 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:22.722+0000 7fb826600700 1 --2- 172.21.15.18:0/403676454 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fb80406ec10 0x7fb8040710c0 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7fb80c004f90 tx=0x7fb80c009e30 comp rx=0 tx=0).stop 2024-04-12T21:48:22.724 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:22.722+0000 7fb826600700 1 -- 172.21.15.18:0/403676454 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb820101d70 msgr2=0x7fb820115dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:48:22.724 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:22.722+0000 7fb826600700 1 --2- 172.21.15.18:0/403676454 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb820101d70 0x7fb820115dc0 secure :-1 s=READY pgs=1239 cs=0 l=1 rev1=1 crypto rx=0x7fb810002a50 tx=0x7fb8100046b0 comp rx=0 tx=0).stop 2024-04-12T21:48:22.725 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:22.723+0000 7fb826600700 1 -- 172.21.15.18:0/403676454 shutdown_connections 2024-04-12T21:48:22.725 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:22.723+0000 7fb826600700 1 --2- 172.21.15.18:0/403676454 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fb80406ec10 0x7fb8040710c0 unknown :-1 s=CLOSED pgs=657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:48:22.726 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:22.723+0000 7fb826600700 1 --2- 172.21.15.18:0/403676454 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb820101d70 0x7fb820115dc0 unknown :-1 s=CLOSED pgs=1239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:48:22.726 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:22.723+0000 7fb826600700 1 --2- 172.21.15.18:0/403676454 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb820102710 0x7fb820116300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:48:22.726 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:22.723+0000 7fb826600700 1 -- 172.21.15.18:0/403676454 >> 172.21.15.18:0/403676454 conn(0x7fb820072bc0 msgr2=0x7fb820065970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:48:22.726 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:22.723+0000 7fb826600700 1 -- 172.21.15.18:0/403676454 shutdown_connections 2024-04-12T21:48:22.726 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:22.723+0000 7fb826600700 1 -- 172.21.15.18:0/403676454 wait complete. 2024-04-12T21:48:23.355 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:48:23 smithi018 ceph-mon[112590]: from='client.? 172.21.15.18:0/403676454' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-12T21:48:23.534 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:48:23 smithi186 ceph-mon[86379]: from='client.? 172.21.15.18:0/403676454' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-12T21:48:24.355 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:48:24 smithi018 ceph-mon[112590]: from='client.36544 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:48:24.355 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:48:24 smithi018 ceph-mon[112590]: pgmap v1461: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:48:24.534 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:48:24 smithi186 ceph-mon[86379]: from='client.36544 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:48:24.534 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:48:24 smithi186 ceph-mon[86379]: pgmap v1461: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:48:25.385 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:48:25 smithi018 ceph-mon[112590]: pgmap v1462: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:48:25.385 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:48:25 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T21:48:25.534 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:48:25 smithi186 ceph-mon[86379]: pgmap v1462: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:48:25.534 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:48:25 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T21:48:27.784 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:48:27 smithi186 ceph-mon[86379]: pgmap v1463: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:48:27.855 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:48:27 smithi018 ceph-mon[112590]: pgmap v1463: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:48:29.784 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:48:29 smithi186 ceph-mon[86379]: pgmap v1464: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:48:29.855 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:48:29 smithi018 ceph-mon[112590]: pgmap v1464: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:48:31.784 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:48:31 smithi186 ceph-mon[86379]: pgmap v1465: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:48:31.855 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:48:31 smithi018 ceph-mon[112590]: pgmap v1465: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:48:33.784 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:48:33 smithi186 ceph-mon[86379]: pgmap v1466: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:48:33.855 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:48:33 smithi018 ceph-mon[112590]: pgmap v1466: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:48:35.784 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:48:35 smithi186 ceph-mon[86379]: pgmap v1467: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:48:35.855 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:48:35 smithi018 ceph-mon[112590]: pgmap v1467: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:48:37.784 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:48:37 smithi186 ceph-mon[86379]: pgmap v1468: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:48:37.855 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:48:37 smithi018 ceph-mon[112590]: pgmap v1468: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:48:39.784 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:48:39 smithi186 ceph-mon[86379]: pgmap v1469: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:48:39.855 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:48:39 smithi018 ceph-mon[112590]: pgmap v1469: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:48:40.784 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:48:40 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T21:48:40.855 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:48:40 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T21:48:41.784 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:48:41 smithi186 ceph-mon[86379]: pgmap v1470: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:48:41.855 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:48:41 smithi018 ceph-mon[112590]: pgmap v1470: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:48:43.784 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:48:43 smithi186 ceph-mon[86379]: pgmap v1471: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:48:43.855 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:48:43 smithi018 ceph-mon[112590]: pgmap v1471: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:48:45.657 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:48:45 smithi018 ceph-mon[112590]: pgmap v1472: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:48:45.784 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:48:45 smithi186 ceph-mon[86379]: pgmap v1472: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:48:47.784 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:48:47 smithi186 ceph-mon[86379]: pgmap v1473: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:48:47.855 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:48:47 smithi018 ceph-mon[112590]: pgmap v1473: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:48:49.784 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:48:49 smithi186 ceph-mon[86379]: pgmap v1474: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:48:49.855 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:48:49 smithi018 ceph-mon[112590]: pgmap v1474: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:48:51.784 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:48:51 smithi186 ceph-mon[86379]: pgmap v1475: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:48:51.855 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:48:51 smithi018 ceph-mon[112590]: pgmap v1475: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:48:52.903 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:52.899+0000 7fb21d800700 1 -- 172.21.15.18:0/3571868088 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb218102c50 msgr2=0x7fb218103020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:48:52.904 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:52.899+0000 7fb21d800700 1 --2- 172.21.15.18:0/3571868088 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb218102c50 0x7fb218103020 secure :-1 s=READY pgs=1240 cs=0 l=1 rev1=1 crypto rx=0x7fb208004940 tx=0x7fb208009e30 comp rx=0 tx=0).stop 2024-04-12T21:48:52.904 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:52.899+0000 7fb21d800700 1 -- 172.21.15.18:0/3571868088 shutdown_connections 2024-04-12T21:48:52.904 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:52.899+0000 7fb21d800700 1 --2- 172.21.15.18:0/3571868088 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb2181035f0 0x7fb2180fcc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:48:52.904 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:52.899+0000 7fb21d800700 1 --2- 172.21.15.18:0/3571868088 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb218102c50 0x7fb218103020 unknown :-1 s=CLOSED pgs=1240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:48:52.904 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:52.899+0000 7fb21d800700 1 -- 172.21.15.18:0/3571868088 >> 172.21.15.18:0/3571868088 conn(0x7fb2180f87e0 msgr2=0x7fb2180fabf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:48:52.904 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:52.899+0000 7fb21d800700 1 -- 172.21.15.18:0/3571868088 shutdown_connections 2024-04-12T21:48:52.904 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:52.899+0000 7fb21d800700 1 -- 172.21.15.18:0/3571868088 wait complete. 2024-04-12T21:48:52.904 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:52.900+0000 7fb21d800700 1 Processor -- start 2024-04-12T21:48:52.904 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:52.900+0000 7fb21d800700 1 -- start start 2024-04-12T21:48:52.904 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:52.900+0000 7fb21d800700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb218102c50 0x7fb218115c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:48:52.904 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:52.900+0000 7fb21d800700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb2181035f0 0x7fb218116150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:48:52.904 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:52.901+0000 7fb21d800700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb218116700 con 0x7fb218102c50 2024-04-12T21:48:52.904 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:52.901+0000 7fb21d800700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb21810fb40 con 0x7fb2181035f0 2024-04-12T21:48:52.905 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:52.901+0000 7fb217e00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb218102c50 0x7fb218115c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:48:52.905 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:52.901+0000 7fb217e00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb218102c50 0x7fb218115c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:38390/0 (socket says 172.21.15.18:38390) 2024-04-12T21:48:52.905 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:52.901+0000 7fb217e00700 1 -- 172.21.15.18:0/1402174506 learned_addr learned my addr 172.21.15.18:0/1402174506 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:48:52.905 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:52.901+0000 7fb20fe00700 1 --2- 172.21.15.18:0/1402174506 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb2181035f0 0x7fb218116150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:48:52.905 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:52.901+0000 7fb217e00700 1 -- 172.21.15.18:0/1402174506 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb2181035f0 msgr2=0x7fb218116150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:48:52.905 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:52.901+0000 7fb217e00700 1 --2- 172.21.15.18:0/1402174506 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb2181035f0 0x7fb218116150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:48:52.905 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:52.901+0000 7fb217e00700 1 -- 172.21.15.18:0/1402174506 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb208005040 con 0x7fb218102c50 2024-04-12T21:48:52.905 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:52.901+0000 7fb20fe00700 1 --2- 172.21.15.18:0/1402174506 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb2181035f0 0x7fb218116150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-12T21:48:52.905 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:52.902+0000 7fb217e00700 1 --2- 172.21.15.18:0/1402174506 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb218102c50 0x7fb218115c10 secure :-1 s=READY pgs=1241 cs=0 l=1 rev1=1 crypto rx=0x7fb208009180 tx=0x7fb208009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:48:52.905 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:52.902+0000 7fb214c00700 1 -- 172.21.15.18:0/1402174506 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb20800e040 con 0x7fb218102c50 2024-04-12T21:48:52.905 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:52.902+0000 7fb214c00700 1 -- 172.21.15.18:0/1402174506 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb20800bbc0 con 0x7fb218102c50 2024-04-12T21:48:52.906 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:52.902+0000 7fb21d800700 1 -- 172.21.15.18:0/1402174506 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb21810fdc0 con 0x7fb218102c50 2024-04-12T21:48:52.906 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:52.902+0000 7fb214c00700 1 -- 172.21.15.18:0/1402174506 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb20801c630 con 0x7fb218102c50 2024-04-12T21:48:52.906 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:52.902+0000 7fb21d800700 1 -- 172.21.15.18:0/1402174506 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb218110380 con 0x7fb218102c50 2024-04-12T21:48:52.907 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:52.904+0000 7fb21d800700 1 -- 172.21.15.18:0/1402174506 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb2180fe3a0 con 0x7fb218102c50 2024-04-12T21:48:52.907 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:52.905+0000 7fb214c00700 1 -- 172.21.15.18:0/1402174506 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7fb20800bd30 con 0x7fb218102c50 2024-04-12T21:48:52.907 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:52.906+0000 7fb214c00700 1 --2- 172.21.15.18:0/1402174506 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fb1fc06ec10 0x7fb1fc0710c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:48:52.907 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:52.906+0000 7fb214c00700 1 -- 172.21.15.18:0/1402174506 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 6757+0+0 (secure 0 0 0) 0x7fb20808fcb0 con 0x7fb218102c50 2024-04-12T21:48:52.907 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:52.906+0000 7fb20fe00700 1 --2- 172.21.15.18:0/1402174506 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fb1fc06ec10 0x7fb1fc0710c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:48:52.907 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:52.907+0000 7fb20fe00700 1 --2- 172.21.15.18:0/1402174506 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fb1fc06ec10 0x7fb1fc0710c0 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7fb200007fd0 tx=0x7fb200009040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:48:52.912 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:52.911+0000 7fb214c00700 1 -- 172.21.15.18:0/1402174506 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7fb208058a70 con 0x7fb218102c50 2024-04-12T21:48:53.160 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:53.159+0000 7fb21d800700 1 -- 172.21.15.18:0/1402174506 --> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fb21804bed0 con 0x7fb1fc06ec10 2024-04-12T21:48:53.167 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:53.162+0000 7fb214c00700 1 -- 172.21.15.18:0/1402174506 <== mgr.34102 v2:172.21.15.18:6800/3812891919 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7fb21804bed0 con 0x7fb1fc06ec10 2024-04-12T21:48:53.170 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:53.164+0000 7fb21d800700 1 -- 172.21.15.18:0/1402174506 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fb1fc06ec10 msgr2=0x7fb1fc0710c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:48:53.170 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:53.165+0000 7fb21d800700 1 --2- 172.21.15.18:0/1402174506 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fb1fc06ec10 0x7fb1fc0710c0 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7fb200007fd0 tx=0x7fb200009040 comp rx=0 tx=0).stop 2024-04-12T21:48:53.170 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:53.165+0000 7fb21d800700 1 -- 172.21.15.18:0/1402174506 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb218102c50 msgr2=0x7fb218115c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:48:53.170 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:53.165+0000 7fb21d800700 1 --2- 172.21.15.18:0/1402174506 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb218102c50 0x7fb218115c10 secure :-1 s=READY pgs=1241 cs=0 l=1 rev1=1 crypto rx=0x7fb208009180 tx=0x7fb208009e30 comp rx=0 tx=0).stop 2024-04-12T21:48:53.170 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:53.165+0000 7fb21d800700 1 -- 172.21.15.18:0/1402174506 shutdown_connections 2024-04-12T21:48:53.170 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:53.165+0000 7fb21d800700 1 --2- 172.21.15.18:0/1402174506 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fb1fc06ec10 0x7fb1fc0710c0 unknown :-1 s=CLOSED pgs=659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:48:53.171 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:53.165+0000 7fb21d800700 1 --2- 172.21.15.18:0/1402174506 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb218102c50 0x7fb218115c10 unknown :-1 s=CLOSED pgs=1241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:48:53.171 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:53.165+0000 7fb21d800700 1 --2- 172.21.15.18:0/1402174506 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb2181035f0 0x7fb218116150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:48:53.171 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:53.165+0000 7fb21d800700 1 -- 172.21.15.18:0/1402174506 >> 172.21.15.18:0/1402174506 conn(0x7fb2180f87e0 msgr2=0x7fb2180f9840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:48:53.171 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:53.165+0000 7fb21d800700 1 -- 172.21.15.18:0/1402174506 shutdown_connections 2024-04-12T21:48:53.171 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:53.165+0000 7fb21d800700 1 -- 172.21.15.18:0/1402174506 wait complete. 2024-04-12T21:48:53.181 INFO:teuthology.orchestra.run.smithi018.stdout:true 2024-04-12T21:48:53.324 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:53.322+0000 7fefe8800700 1 -- 172.21.15.18:0/1323263973 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fefe01037a0 msgr2=0x7fefe00fce10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:48:53.324 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:53.322+0000 7fefe8800700 1 --2- 172.21.15.18:0/1323263973 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fefe01037a0 0x7fefe00fce10 secure :-1 s=READY pgs=1242 cs=0 l=1 rev1=1 crypto rx=0x7fefd0004940 tx=0x7fefd0009e30 comp rx=0 tx=0).stop 2024-04-12T21:48:53.324 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:53.323+0000 7fefe8800700 1 -- 172.21.15.18:0/1323263973 shutdown_connections 2024-04-12T21:48:53.324 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:53.323+0000 7fefe8800700 1 --2- 172.21.15.18:0/1323263973 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fefe01037a0 0x7fefe00fce10 unknown :-1 s=CLOSED pgs=1242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:48:53.325 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:53.323+0000 7fefe8800700 1 --2- 172.21.15.18:0/1323263973 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fefe0102e00 0x7fefe01031d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:48:53.325 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:53.323+0000 7fefe8800700 1 -- 172.21.15.18:0/1323263973 >> 172.21.15.18:0/1323263973 conn(0x7fefe00f89b0 msgr2=0x7fefe00fadc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:48:53.325 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:53.323+0000 7fefe8800700 1 -- 172.21.15.18:0/1323263973 shutdown_connections 2024-04-12T21:48:53.325 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:53.323+0000 7fefe8800700 1 -- 172.21.15.18:0/1323263973 wait complete. 2024-04-12T21:48:53.325 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:53.324+0000 7fefe8800700 1 Processor -- start 2024-04-12T21:48:53.326 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:53.324+0000 7fefe8800700 1 -- start start 2024-04-12T21:48:53.326 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:53.324+0000 7fefe8800700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fefe0102e00 0x7fefe0115de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:48:53.326 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:53.324+0000 7fefe8800700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fefe01037a0 0x7fefe0116320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:48:53.326 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:53.324+0000 7fefe8800700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fefe0116970 con 0x7fefe0102e00 2024-04-12T21:48:53.326 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:53.324+0000 7fefe8800700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fefe010fd80 con 0x7fefe01037a0 2024-04-12T21:48:53.326 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:53.325+0000 7fefe5e00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fefe0102e00 0x7fefe0115de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:48:53.326 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:53.325+0000 7fefe5e00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fefe0102e00 0x7fefe0115de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:38408/0 (socket says 172.21.15.18:38408) 2024-04-12T21:48:53.327 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:53.325+0000 7fefe5e00700 1 -- 172.21.15.18:0/288005085 learned_addr learned my addr 172.21.15.18:0/288005085 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:48:53.327 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:53.325+0000 7fefe5e00700 1 -- 172.21.15.18:0/288005085 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fefe01037a0 msgr2=0x7fefe0116320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:48:53.327 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:53.325+0000 7fefe5e00700 1 --2- 172.21.15.18:0/288005085 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fefe01037a0 0x7fefe0116320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:48:53.327 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:53.325+0000 7fefe5e00700 1 -- 172.21.15.18:0/288005085 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fefd0005040 con 0x7fefe0102e00 2024-04-12T21:48:53.327 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:53.325+0000 7fefe5e00700 1 --2- 172.21.15.18:0/288005085 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fefe0102e00 0x7fefe0115de0 secure :-1 s=READY pgs=1243 cs=0 l=1 rev1=1 crypto rx=0x7fefd400c830 tx=0x7fefd4004470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:48:53.327 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:53.326+0000 7fefda000700 1 -- 172.21.15.18:0/288005085 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fefd400a760 con 0x7fefe0102e00 2024-04-12T21:48:53.328 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:53.326+0000 7fefda000700 1 -- 172.21.15.18:0/288005085 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fefd400ad90 con 0x7fefe0102e00 2024-04-12T21:48:53.328 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:53.326+0000 7fefe8800700 1 -- 172.21.15.18:0/288005085 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fefe0110060 con 0x7fefe0102e00 2024-04-12T21:48:53.328 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:53.326+0000 7fefda000700 1 -- 172.21.15.18:0/288005085 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fefd40113a0 con 0x7fefe0102e00 2024-04-12T21:48:53.328 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:53.326+0000 7fefe8800700 1 -- 172.21.15.18:0/288005085 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fefe01105a0 con 0x7fefe0102e00 2024-04-12T21:48:53.329 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:53.328+0000 7fefe8800700 1 -- 172.21.15.18:0/288005085 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fefe0102120 con 0x7fefe0102e00 2024-04-12T21:48:53.329 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:53.328+0000 7fefda000700 1 -- 172.21.15.18:0/288005085 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7fefd4012400 con 0x7fefe0102e00 2024-04-12T21:48:53.330 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:53.329+0000 7fefda000700 1 --2- 172.21.15.18:0/288005085 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fefc806ec10 0x7fefc80710c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:48:53.330 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:53.329+0000 7fefda000700 1 -- 172.21.15.18:0/288005085 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 6757+0+0 (secure 0 0 0) 0x7fefd408e900 con 0x7fefe0102e00 2024-04-12T21:48:53.330 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:53.330+0000 7fefe5400700 1 --2- 172.21.15.18:0/288005085 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fefc806ec10 0x7fefc80710c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:48:53.331 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:53.330+0000 7fefe5400700 1 --2- 172.21.15.18:0/288005085 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fefc806ec10 0x7fefc80710c0 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7fefd0004f90 tx=0x7fefd0009e30 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:48:53.335 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:53.335+0000 7fefda000700 1 -- 172.21.15.18:0/288005085 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7fefd40576c0 con 0x7fefe0102e00 2024-04-12T21:48:53.590 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:53.589+0000 7fefe8800700 1 -- 172.21.15.18:0/288005085 --> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fefe01114e0 con 0x7fefc806ec10 2024-04-12T21:48:53.591 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:53.591+0000 7fefda000700 1 -- 172.21.15.18:0/288005085 <== mgr.34102 v2:172.21.15.18:6800/3812891919 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7fefe01114e0 con 0x7fefc806ec10 2024-04-12T21:48:53.594 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:53.593+0000 7fefe8800700 1 -- 172.21.15.18:0/288005085 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fefc806ec10 msgr2=0x7fefc80710c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:48:53.595 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:53.593+0000 7fefe8800700 1 --2- 172.21.15.18:0/288005085 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fefc806ec10 0x7fefc80710c0 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7fefd0004f90 tx=0x7fefd0009e30 comp rx=0 tx=0).stop 2024-04-12T21:48:53.595 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:53.593+0000 7fefe8800700 1 -- 172.21.15.18:0/288005085 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fefe0102e00 msgr2=0x7fefe0115de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:48:53.595 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:53.593+0000 7fefe8800700 1 --2- 172.21.15.18:0/288005085 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fefe0102e00 0x7fefe0115de0 secure :-1 s=READY pgs=1243 cs=0 l=1 rev1=1 crypto rx=0x7fefd400c830 tx=0x7fefd4004470 comp rx=0 tx=0).stop 2024-04-12T21:48:53.595 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:53.594+0000 7fefe8800700 1 -- 172.21.15.18:0/288005085 shutdown_connections 2024-04-12T21:48:53.595 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:53.594+0000 7fefe8800700 1 --2- 172.21.15.18:0/288005085 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fefc806ec10 0x7fefc80710c0 unknown :-1 s=CLOSED pgs=660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:48:53.595 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:53.594+0000 7fefe8800700 1 --2- 172.21.15.18:0/288005085 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fefe0102e00 0x7fefe0115de0 unknown :-1 s=CLOSED pgs=1243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:48:53.596 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:53.594+0000 7fefe8800700 1 --2- 172.21.15.18:0/288005085 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fefe01037a0 0x7fefe0116320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:48:53.596 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:53.594+0000 7fefe8800700 1 -- 172.21.15.18:0/288005085 >> 172.21.15.18:0/288005085 conn(0x7fefe00f89b0 msgr2=0x7fefe0070bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:48:53.596 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:53.594+0000 7fefe8800700 1 -- 172.21.15.18:0/288005085 shutdown_connections 2024-04-12T21:48:53.596 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:53.594+0000 7fefe8800700 1 -- 172.21.15.18:0/288005085 wait complete. 2024-04-12T21:48:53.756 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:53.755+0000 7f216e600700 1 -- 172.21.15.18:0/475095269 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f21681059c0 msgr2=0x7f21680ff080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:48:53.757 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:53.755+0000 7f216e600700 1 --2- 172.21.15.18:0/475095269 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f21681059c0 0x7f21680ff080 secure :-1 s=READY pgs=1244 cs=0 l=1 rev1=1 crypto rx=0x7f2150004940 tx=0x7f2150009e30 comp rx=0 tx=0).stop 2024-04-12T21:48:53.757 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:53.756+0000 7f216e600700 1 -- 172.21.15.18:0/475095269 shutdown_connections 2024-04-12T21:48:53.757 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:53.756+0000 7f216e600700 1 --2- 172.21.15.18:0/475095269 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f21681059c0 0x7f21680ff080 unknown :-1 s=CLOSED pgs=1244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:48:53.757 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:53.756+0000 7f216e600700 1 --2- 172.21.15.18:0/475095269 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2168105020 0x7f21681053f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:48:53.758 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:53.756+0000 7f216e600700 1 -- 172.21.15.18:0/475095269 >> 172.21.15.18:0/475095269 conn(0x7f21680faba0 msgr2=0x7f21680fcfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:48:53.758 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:53.756+0000 7f216e600700 1 -- 172.21.15.18:0/475095269 shutdown_connections 2024-04-12T21:48:53.758 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:53.756+0000 7f216e600700 1 -- 172.21.15.18:0/475095269 wait complete. 2024-04-12T21:48:53.758 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:53.757+0000 7f216e600700 1 Processor -- start 2024-04-12T21:48:53.758 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:53.757+0000 7f216e600700 1 -- start start 2024-04-12T21:48:53.760 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:53.757+0000 7f216e600700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2168105020 0x7f2168115ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:48:53.760 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:53.757+0000 7f216e600700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f21681059c0 0x7f21681160e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:48:53.760 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:53.757+0000 7f216e600700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2168116730 con 0x7f2168105020 2024-04-12T21:48:53.760 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:53.757+0000 7f216e600700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f216810fd10 con 0x7f21681059c0 2024-04-12T21:48:53.760 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:53.757+0000 7f2167400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2168105020 0x7f2168115ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:48:53.760 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:53.757+0000 7f2167400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2168105020 0x7f2168115ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:38426/0 (socket says 172.21.15.18:38426) 2024-04-12T21:48:53.760 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:53.757+0000 7f2167400700 1 -- 172.21.15.18:0/1666700661 learned_addr learned my addr 172.21.15.18:0/1666700661 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:48:53.760 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:53.758+0000 7f2167400700 1 -- 172.21.15.18:0/1666700661 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f21681059c0 msgr2=0x7f21681160e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:48:53.760 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:53.758+0000 7f2167400700 1 --2- 172.21.15.18:0/1666700661 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f21681059c0 0x7f21681160e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:48:53.760 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:53.758+0000 7f2167400700 1 -- 172.21.15.18:0/1666700661 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2150005040 con 0x7f2168105020 2024-04-12T21:48:53.760 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:53.758+0000 7f2167400700 1 --2- 172.21.15.18:0/1666700661 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2168105020 0x7f2168115ba0 secure :-1 s=READY pgs=1245 cs=0 l=1 rev1=1 crypto rx=0x7f2158002a50 tx=0x7f21580046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:48:53.760 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:53.758+0000 7f215f400700 1 -- 172.21.15.18:0/1666700661 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2158007cc0 con 0x7f2168105020 2024-04-12T21:48:53.761 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:53.758+0000 7f215f400700 1 -- 172.21.15.18:0/1666700661 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f215800f040 con 0x7f2168105020 2024-04-12T21:48:53.761 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:53.758+0000 7f216e600700 1 -- 172.21.15.18:0/1666700661 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f216810fff0 con 0x7f2168105020 2024-04-12T21:48:53.761 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:53.758+0000 7f215f400700 1 -- 172.21.15.18:0/1666700661 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2158009640 con 0x7f2168105020 2024-04-12T21:48:53.761 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:53.759+0000 7f216e600700 1 -- 172.21.15.18:0/1666700661 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f21681105b0 con 0x7f2168105020 2024-04-12T21:48:53.761 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:53.760+0000 7f216e600700 1 -- 172.21.15.18:0/1666700661 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2168100770 con 0x7f2168105020 2024-04-12T21:48:53.761 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:53.761+0000 7f215f400700 1 -- 172.21.15.18:0/1666700661 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7f21580097a0 con 0x7f2168105020 2024-04-12T21:48:53.762 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:53.762+0000 7f215f400700 1 --2- 172.21.15.18:0/1666700661 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f214c06ec00 0x7f214c0710b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:48:53.762 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:53.762+0000 7f215f400700 1 -- 172.21.15.18:0/1666700661 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 6757+0+0 (secure 0 0 0) 0x7f215808e760 con 0x7f2168105020 2024-04-12T21:48:53.763 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:53.762+0000 7f215fe00700 1 --2- 172.21.15.18:0/1666700661 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f214c06ec00 0x7f214c0710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:48:53.763 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:53.763+0000 7f215fe00700 1 --2- 172.21.15.18:0/1666700661 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f214c06ec00 0x7f214c0710b0 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f2150004f90 tx=0x7f2150009e30 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:48:53.767 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:53.766+0000 7f215f400700 1 -- 172.21.15.18:0/1666700661 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f215805af10 con 0x7f2168105020 2024-04-12T21:48:53.767 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:48:53 smithi018 ceph-mon[112590]: pgmap v1476: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:48:53.784 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:48:53 smithi186 ceph-mon[86379]: pgmap v1476: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:48:54.002 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:54.001+0000 7f216e600700 1 -- 172.21.15.18:0/1666700661 --> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f216810c170 con 0x7f214c06ec00 2024-04-12T21:48:54.033 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:54.032+0000 7f215f400700 1 -- 172.21.15.18:0/1666700661 <== mgr.34102 v2:172.21.15.18:6800/3812891919 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7f216810c170 con 0x7f214c06ec00 2024-04-12T21:48:54.033 INFO:teuthology.orchestra.run.smithi018.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-12T21:48:54.033 INFO:teuthology.orchestra.run.smithi018.stdout:alertmanager.smithi018 smithi018 *:9093,9094 running (49m) 10m ago 63m 14.8M - 0.25.0 c8568f914cd2 c332090068e8 2024-04-12T21:48:54.034 INFO:teuthology.orchestra.run.smithi018.stdout:crash.smithi018 smithi018 running (45m) 10m ago 64m 6665k - 19.0.0-2879-g324baf8d 8717720dc5e5 b7f80d1f7982 2024-04-12T21:48:54.034 INFO:teuthology.orchestra.run.smithi018.stdout:crash.smithi186 smithi186 running (45m) 10m ago 62m 6660k - 19.0.0-2879-g324baf8d 8717720dc5e5 a4f55c81b623 2024-04-12T21:48:54.034 INFO:teuthology.orchestra.run.smithi018.stdout:grafana.smithi018 smithi018 *:3000 running (48m) 10m ago 63m 73.8M - 9.4.12 2bacad6d85d8 b08e3bf176dd 2024-04-12T21:48:54.034 INFO:teuthology.orchestra.run.smithi018.stdout:mds.cephfs.smithi018.acqbsc smithi018 running (41m) 10m ago 56m 21.9M - 19.0.0-2879-g324baf8d 8717720dc5e5 021f99ab24ae 2024-04-12T21:48:54.034 INFO:teuthology.orchestra.run.smithi018.stdout:mds.cephfs.smithi018.szjluo smithi018 running (41m) 10m ago 56m 21.7M - 19.0.0-2879-g324baf8d 8717720dc5e5 0003117479d4 2024-04-12T21:48:54.034 INFO:teuthology.orchestra.run.smithi018.stdout:mds.cephfs.smithi186.jdllyn smithi186 running (25m) 10m ago 56m 16.5M - 19.0.0-2879-g324baf8d 8717720dc5e5 21b1f46eef23 2024-04-12T21:48:54.034 INFO:teuthology.orchestra.run.smithi018.stdout:mds.cephfs.smithi186.wjnafs smithi186 running (10m) 10m ago 56m 11.0M - 19.0.0-2879-g324baf8d 8717720dc5e5 3cf1ef4148e2 2024-04-12T21:48:54.034 INFO:teuthology.orchestra.run.smithi018.stdout:mgr.smithi018.xjzwga smithi018 *:8443,9283,8765 running (52m) 10m ago 65m 517M - 19.0.0-2879-g324baf8d 8717720dc5e5 56088305eab7 2024-04-12T21:48:54.034 INFO:teuthology.orchestra.run.smithi018.stdout:mgr.smithi186.kqcfxk smithi186 *:8443,9283,8765 running (50m) 10m ago 62m 460M - 19.0.0-2879-g324baf8d 8717720dc5e5 09cd766e5e0d 2024-04-12T21:48:54.034 INFO:teuthology.orchestra.run.smithi018.stdout:mon.smithi018 smithi018 running (47m) 10m ago 65m 88.2M 2048M 19.0.0-2879-g324baf8d 8717720dc5e5 df3e201dfed6 2024-04-12T21:48:54.035 INFO:teuthology.orchestra.run.smithi018.stdout:mon.smithi186 smithi186 running (46m) 10m ago 62m 53.6M 2048M 19.0.0-2879-g324baf8d 8717720dc5e5 c77498da9b2b 2024-04-12T21:48:54.035 INFO:teuthology.orchestra.run.smithi018.stdout:node-exporter.smithi018 smithi018 *:9100 running (50m) 10m ago 64m 8468k - 1.5.0 0da6a335fe13 326c3d9c6847 2024-04-12T21:48:54.035 INFO:teuthology.orchestra.run.smithi018.stdout:node-exporter.smithi186 smithi186 *:9100 running (50m) 10m ago 62m 8166k - 1.5.0 0da6a335fe13 ec2112baeecf 2024-04-12T21:48:54.035 INFO:teuthology.orchestra.run.smithi018.stdout:osd.0 smithi018 running (45m) 10m ago 61m 123M 1559M 19.0.0-2879-g324baf8d 8717720dc5e5 60b1273ca047 2024-04-12T21:48:54.035 INFO:teuthology.orchestra.run.smithi018.stdout:osd.1 smithi018 running (44m) 10m ago 61m 95.4M 1559M 19.0.0-2879-g324baf8d 8717720dc5e5 170ce1e5fb8d 2024-04-12T21:48:54.035 INFO:teuthology.orchestra.run.smithi018.stdout:osd.2 smithi018 running (44m) 10m ago 60m 94.6M 1559M 19.0.0-2879-g324baf8d 8717720dc5e5 c2cbc6aa2839 2024-04-12T21:48:54.035 INFO:teuthology.orchestra.run.smithi018.stdout:osd.3 smithi186 running (43m) 10m ago 59m 101M 2581M 19.0.0-2879-g324baf8d 8717720dc5e5 0138e8262e4b 2024-04-12T21:48:54.035 INFO:teuthology.orchestra.run.smithi018.stdout:osd.4 smithi186 running (42m) 10m ago 59m 91.5M 2581M 19.0.0-2879-g324baf8d 8717720dc5e5 b20cb63b25c9 2024-04-12T21:48:54.035 INFO:teuthology.orchestra.run.smithi018.stdout:osd.5 smithi186 running (42m) 10m ago 58m 92.2M 2581M 19.0.0-2879-g324baf8d 8717720dc5e5 2513bea2610e 2024-04-12T21:48:54.035 INFO:teuthology.orchestra.run.smithi018.stdout:prometheus.smithi018 smithi018 *:9095 running (49m) 10m ago 63m 69.6M - 2.43.0 a07b618ecd1d f58841127e21 2024-04-12T21:48:54.036 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:54.035+0000 7f216e600700 1 -- 172.21.15.18:0/1666700661 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f214c06ec00 msgr2=0x7f214c0710b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:48:54.037 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:54.035+0000 7f216e600700 1 --2- 172.21.15.18:0/1666700661 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f214c06ec00 0x7f214c0710b0 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f2150004f90 tx=0x7f2150009e30 comp rx=0 tx=0).stop 2024-04-12T21:48:54.037 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:54.035+0000 7f216e600700 1 -- 172.21.15.18:0/1666700661 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2168105020 msgr2=0x7f2168115ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:48:54.037 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:54.035+0000 7f216e600700 1 --2- 172.21.15.18:0/1666700661 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2168105020 0x7f2168115ba0 secure :-1 s=READY pgs=1245 cs=0 l=1 rev1=1 crypto rx=0x7f2158002a50 tx=0x7f21580046b0 comp rx=0 tx=0).stop 2024-04-12T21:48:54.037 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:54.035+0000 7f216e600700 1 -- 172.21.15.18:0/1666700661 shutdown_connections 2024-04-12T21:48:54.037 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:54.035+0000 7f216e600700 1 --2- 172.21.15.18:0/1666700661 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f214c06ec00 0x7f214c0710b0 unknown :-1 s=CLOSED pgs=661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:48:54.037 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:54.035+0000 7f216e600700 1 --2- 172.21.15.18:0/1666700661 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2168105020 0x7f2168115ba0 unknown :-1 s=CLOSED pgs=1245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:48:54.037 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:54.035+0000 7f216e600700 1 --2- 172.21.15.18:0/1666700661 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f21681059c0 0x7f21681160e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:48:54.037 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:54.035+0000 7f216e600700 1 -- 172.21.15.18:0/1666700661 >> 172.21.15.18:0/1666700661 conn(0x7f21680faba0 msgr2=0x7f21680fbb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:48:54.037 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:54.035+0000 7f216e600700 1 -- 172.21.15.18:0/1666700661 shutdown_connections 2024-04-12T21:48:54.037 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:54.035+0000 7f216e600700 1 -- 172.21.15.18:0/1666700661 wait complete. 2024-04-12T21:48:54.179 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:54.178+0000 7fec39400700 1 -- 172.21.15.18:0/3054990736 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fec34109430 msgr2=0x7fec34109800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:48:54.179 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:54.178+0000 7fec39400700 1 --2- 172.21.15.18:0/3054990736 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fec34109430 0x7fec34109800 secure :-1 s=READY pgs=1246 cs=0 l=1 rev1=1 crypto rx=0x7fec24004940 tx=0x7fec24009e30 comp rx=0 tx=0).stop 2024-04-12T21:48:54.179 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:54.178+0000 7fec39400700 1 -- 172.21.15.18:0/3054990736 shutdown_connections 2024-04-12T21:48:54.179 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:54.178+0000 7fec39400700 1 --2- 172.21.15.18:0/3054990736 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fec34109dd0 0x7fec34103440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:48:54.180 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:54.178+0000 7fec39400700 1 --2- 172.21.15.18:0/3054990736 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fec34109430 0x7fec34109800 unknown :-1 s=CLOSED pgs=1246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:48:54.180 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:54.178+0000 7fec39400700 1 -- 172.21.15.18:0/3054990736 >> 172.21.15.18:0/3054990736 conn(0x7fec34072c00 msgr2=0x7fec34075010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:48:54.180 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:54.178+0000 7fec39400700 1 -- 172.21.15.18:0/3054990736 shutdown_connections 2024-04-12T21:48:54.180 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:54.178+0000 7fec39400700 1 -- 172.21.15.18:0/3054990736 wait complete. 2024-04-12T21:48:54.181 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:54.179+0000 7fec39400700 1 Processor -- start 2024-04-12T21:48:54.181 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:54.179+0000 7fec39400700 1 -- start start 2024-04-12T21:48:54.181 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:54.180+0000 7fec39400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fec34109430 0x7fec34119f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:48:54.181 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:54.180+0000 7fec39400700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fec34109dd0 0x7fec3411a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:48:54.181 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:54.180+0000 7fec39400700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec3411aad0 con 0x7fec34109430 2024-04-12T21:48:54.181 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:54.180+0000 7fec39400700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec34114100 con 0x7fec34109dd0 2024-04-12T21:48:54.181 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:54.180+0000 7fec32800700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fec34109430 0x7fec34119f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:48:54.181 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:54.180+0000 7fec32800700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fec34109430 0x7fec34119f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:38446/0 (socket says 172.21.15.18:38446) 2024-04-12T21:48:54.181 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:54.180+0000 7fec32800700 1 -- 172.21.15.18:0/2167066394 learned_addr learned my addr 172.21.15.18:0/2167066394 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:48:54.182 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:54.180+0000 7fec2be00700 1 --2- 172.21.15.18:0/2167066394 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fec34109dd0 0x7fec3411a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:48:54.182 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:54.180+0000 7fec32800700 1 -- 172.21.15.18:0/2167066394 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fec34109dd0 msgr2=0x7fec3411a480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:48:54.183 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:54.181+0000 7fec32800700 1 --2- 172.21.15.18:0/2167066394 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fec34109dd0 0x7fec3411a480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:48:54.183 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:54.181+0000 7fec32800700 1 -- 172.21.15.18:0/2167066394 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec24005040 con 0x7fec34109430 2024-04-12T21:48:54.183 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:54.181+0000 7fec32800700 1 --2- 172.21.15.18:0/2167066394 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fec34109430 0x7fec34119f40 secure :-1 s=READY pgs=1247 cs=0 l=1 rev1=1 crypto rx=0x7fec24009180 tx=0x7fec24009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:48:54.183 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:54.181+0000 7fec2aa00700 1 -- 172.21.15.18:0/2167066394 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fec2400e040 con 0x7fec34109430 2024-04-12T21:48:54.183 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:54.181+0000 7fec2aa00700 1 -- 172.21.15.18:0/2167066394 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fec2400bbc0 con 0x7fec34109430 2024-04-12T21:48:54.184 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:54.181+0000 7fec39400700 1 -- 172.21.15.18:0/2167066394 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fec34114380 con 0x7fec34109430 2024-04-12T21:48:54.184 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:54.181+0000 7fec2aa00700 1 -- 172.21.15.18:0/2167066394 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fec2401c630 con 0x7fec34109430 2024-04-12T21:48:54.184 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:54.181+0000 7fec39400700 1 -- 172.21.15.18:0/2167066394 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fec34114940 con 0x7fec34109430 2024-04-12T21:48:54.184 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:54.182+0000 7fec39400700 1 -- 172.21.15.18:0/2167066394 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fec34104b80 con 0x7fec34109430 2024-04-12T21:48:54.184 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:54.184+0000 7fec2aa00700 1 -- 172.21.15.18:0/2167066394 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7fec2400bd30 con 0x7fec34109430 2024-04-12T21:48:54.185 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:54.184+0000 7fec2aa00700 1 --2- 172.21.15.18:0/2167066394 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fec1806ec00 0x7fec180710b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:48:54.185 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:54.185+0000 7fec2aa00700 1 -- 172.21.15.18:0/2167066394 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 6757+0+0 (secure 0 0 0) 0x7fec2408fd70 con 0x7fec34109430 2024-04-12T21:48:54.186 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:54.185+0000 7fec2be00700 1 --2- 172.21.15.18:0/2167066394 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fec1806ec00 0x7fec180710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:48:54.186 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:54.186+0000 7fec2be00700 1 --2- 172.21.15.18:0/2167066394 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fec1806ec00 0x7fec180710b0 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7fec341157e0 tx=0x7fec1c00b040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:48:54.190 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:54.189+0000 7fec2aa00700 1 -- 172.21.15.18:0/2167066394 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7fec2405c5c0 con 0x7fec34109430 2024-04-12T21:48:54.503 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:54.502+0000 7fec39400700 1 -- 172.21.15.18:0/2167066394 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fec341155c0 con 0x7fec34109430 2024-04-12T21:48:54.503 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:54.503+0000 7fec2aa00700 1 -- 172.21.15.18:0/2167066394 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+623 (secure 0 0 0) 0x7fec2405c150 con 0x7fec34109430 2024-04-12T21:48:54.504 INFO:teuthology.orchestra.run.smithi018.stdout:{ 2024-04-12T21:48:54.504 INFO:teuthology.orchestra.run.smithi018.stdout: "mon": { 2024-04-12T21:48:54.504 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 19.0.0-2879-g324baf8d (324baf8ddb91dea46cac50cb535c306115791355) squid (dev)": 2 2024-04-12T21:48:54.504 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-12T21:48:54.504 INFO:teuthology.orchestra.run.smithi018.stdout: "mgr": { 2024-04-12T21:48:54.504 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 19.0.0-2879-g324baf8d (324baf8ddb91dea46cac50cb535c306115791355) squid (dev)": 2 2024-04-12T21:48:54.504 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-12T21:48:54.504 INFO:teuthology.orchestra.run.smithi018.stdout: "osd": { 2024-04-12T21:48:54.504 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 19.0.0-2879-g324baf8d (324baf8ddb91dea46cac50cb535c306115791355) squid (dev)": 6 2024-04-12T21:48:54.504 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-12T21:48:54.504 INFO:teuthology.orchestra.run.smithi018.stdout: "mds": { 2024-04-12T21:48:54.505 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 19.0.0-2879-g324baf8d (324baf8ddb91dea46cac50cb535c306115791355) squid (dev)": 4 2024-04-12T21:48:54.505 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-12T21:48:54.505 INFO:teuthology.orchestra.run.smithi018.stdout: "overall": { 2024-04-12T21:48:54.505 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 19.0.0-2879-g324baf8d (324baf8ddb91dea46cac50cb535c306115791355) squid (dev)": 14 2024-04-12T21:48:54.505 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-04-12T21:48:54.505 INFO:teuthology.orchestra.run.smithi018.stdout:} 2024-04-12T21:48:54.507 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:54.506+0000 7fec39400700 1 -- 172.21.15.18:0/2167066394 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fec1806ec00 msgr2=0x7fec180710b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:48:54.507 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:54.506+0000 7fec39400700 1 --2- 172.21.15.18:0/2167066394 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fec1806ec00 0x7fec180710b0 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7fec341157e0 tx=0x7fec1c00b040 comp rx=0 tx=0).stop 2024-04-12T21:48:54.507 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:54.506+0000 7fec39400700 1 -- 172.21.15.18:0/2167066394 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fec34109430 msgr2=0x7fec34119f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:48:54.507 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:54.506+0000 7fec39400700 1 --2- 172.21.15.18:0/2167066394 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fec34109430 0x7fec34119f40 secure :-1 s=READY pgs=1247 cs=0 l=1 rev1=1 crypto rx=0x7fec24009180 tx=0x7fec24009e30 comp rx=0 tx=0).stop 2024-04-12T21:48:54.507 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:54.506+0000 7fec39400700 1 -- 172.21.15.18:0/2167066394 shutdown_connections 2024-04-12T21:48:54.507 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:54.506+0000 7fec39400700 1 --2- 172.21.15.18:0/2167066394 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fec1806ec00 0x7fec180710b0 unknown :-1 s=CLOSED pgs=662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:48:54.507 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:54.506+0000 7fec39400700 1 --2- 172.21.15.18:0/2167066394 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fec34109430 0x7fec34119f40 unknown :-1 s=CLOSED pgs=1247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:48:54.507 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:54.506+0000 7fec39400700 1 --2- 172.21.15.18:0/2167066394 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fec34109dd0 0x7fec3411a480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:48:54.507 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:54.506+0000 7fec39400700 1 -- 172.21.15.18:0/2167066394 >> 172.21.15.18:0/2167066394 conn(0x7fec34072c00 msgr2=0x7fec34074c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:48:54.508 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:54.506+0000 7fec39400700 1 -- 172.21.15.18:0/2167066394 shutdown_connections 2024-04-12T21:48:54.508 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:54.506+0000 7fec39400700 1 -- 172.21.15.18:0/2167066394 wait complete. 2024-04-12T21:48:54.652 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:54.651+0000 7fb4c1c00700 1 -- 172.21.15.18:0/141056988 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb4bc0fc5c0 msgr2=0x7fb4bc100be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:48:54.652 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:54.651+0000 7fb4c1c00700 1 --2- 172.21.15.18:0/141056988 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb4bc0fc5c0 0x7fb4bc100be0 secure :-1 s=READY pgs=1248 cs=0 l=1 rev1=1 crypto rx=0x7fb4a8004940 tx=0x7fb4a8009e30 comp rx=0 tx=0).stop 2024-04-12T21:48:54.652 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:54.651+0000 7fb4c1c00700 1 -- 172.21.15.18:0/141056988 shutdown_connections 2024-04-12T21:48:54.652 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:54.651+0000 7fb4c1c00700 1 --2- 172.21.15.18:0/141056988 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb4bc0fc5c0 0x7fb4bc100be0 unknown :-1 s=CLOSED pgs=1248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:48:54.652 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:54.651+0000 7fb4c1c00700 1 --2- 172.21.15.18:0/141056988 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb4bc0fbc20 0x7fb4bc0fbff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:48:54.653 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:54.651+0000 7fb4c1c00700 1 -- 172.21.15.18:0/141056988 >> 172.21.15.18:0/141056988 conn(0x7fb4bc0f7860 msgr2=0x7fb4bc0f9c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:48:54.653 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:54.651+0000 7fb4c1c00700 1 -- 172.21.15.18:0/141056988 shutdown_connections 2024-04-12T21:48:54.653 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:54.651+0000 7fb4c1c00700 1 -- 172.21.15.18:0/141056988 wait complete. 2024-04-12T21:48:54.654 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:54.652+0000 7fb4c1c00700 1 Processor -- start 2024-04-12T21:48:54.654 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:54.652+0000 7fb4c1c00700 1 -- start start 2024-04-12T21:48:54.654 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:54.653+0000 7fb4c1c00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb4bc0fbc20 0x7fb4bc11a000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:48:54.654 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:54.653+0000 7fb4c1c00700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb4bc0fc5c0 0x7fb4bc11a540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:48:54.654 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:54.653+0000 7fb4c1c00700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4bc11ac20 con 0x7fb4bc0fbc20 2024-04-12T21:48:54.654 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:54.653+0000 7fb4c1c00700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4bc114170 con 0x7fb4bc0fc5c0 2024-04-12T21:48:54.654 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:54.653+0000 7fb4baa00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb4bc0fbc20 0x7fb4bc11a000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:48:54.654 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:54.653+0000 7fb4baa00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb4bc0fbc20 0x7fb4bc11a000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:38468/0 (socket says 172.21.15.18:38468) 2024-04-12T21:48:54.654 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:54.653+0000 7fb4baa00700 1 -- 172.21.15.18:0/3755749257 learned_addr learned my addr 172.21.15.18:0/3755749257 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:48:54.656 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:54.654+0000 7fb4ba000700 1 --2- 172.21.15.18:0/3755749257 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb4bc0fc5c0 0x7fb4bc11a540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:48:54.656 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:54.654+0000 7fb4baa00700 1 -- 172.21.15.18:0/3755749257 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb4bc0fc5c0 msgr2=0x7fb4bc11a540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:48:54.656 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:54.654+0000 7fb4baa00700 1 --2- 172.21.15.18:0/3755749257 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb4bc0fc5c0 0x7fb4bc11a540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:48:54.656 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:54.654+0000 7fb4baa00700 1 -- 172.21.15.18:0/3755749257 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4a8005040 con 0x7fb4bc0fbc20 2024-04-12T21:48:54.656 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:54.654+0000 7fb4ba000700 1 --2- 172.21.15.18:0/3755749257 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb4bc0fc5c0 0x7fb4bc11a540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-12T21:48:54.656 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:54.654+0000 7fb4baa00700 1 --2- 172.21.15.18:0/3755749257 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb4bc0fbc20 0x7fb4bc11a000 secure :-1 s=READY pgs=1249 cs=0 l=1 rev1=1 crypto rx=0x7fb4ac00b790 tx=0x7fb4ac009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:48:54.656 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:54.654+0000 7fb4b2a00700 1 -- 172.21.15.18:0/3755749257 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb4ac00f770 con 0x7fb4bc0fbc20 2024-04-12T21:48:54.656 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:54.654+0000 7fb4b2a00700 1 -- 172.21.15.18:0/3755749257 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb4ac00fda0 con 0x7fb4bc0fbc20 2024-04-12T21:48:54.656 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:54.654+0000 7fb4b2a00700 1 -- 172.21.15.18:0/3755749257 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb4ac00d310 con 0x7fb4bc0fbc20 2024-04-12T21:48:54.656 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:54.654+0000 7fb4c1c00700 1 -- 172.21.15.18:0/3755749257 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb4bc1143f0 con 0x7fb4bc0fbc20 2024-04-12T21:48:54.656 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:54.654+0000 7fb4c1c00700 1 -- 172.21.15.18:0/3755749257 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb4bc114930 con 0x7fb4bc0fbc20 2024-04-12T21:48:54.657 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:54.655+0000 7fb4c1c00700 1 -- 172.21.15.18:0/3755749257 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb4bc066ad0 con 0x7fb4bc0fbc20 2024-04-12T21:48:54.657 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:54.656+0000 7fb4b2a00700 1 -- 172.21.15.18:0/3755749257 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7fb4ac010420 con 0x7fb4bc0fbc20 2024-04-12T21:48:54.658 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:54.657+0000 7fb4b2a00700 1 --2- 172.21.15.18:0/3755749257 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fb4a006ec00 0x7fb4a00710b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:48:54.658 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:54.658+0000 7fb4b2a00700 1 -- 172.21.15.18:0/3755749257 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 6757+0+0 (secure 0 0 0) 0x7fb4ac08e9a0 con 0x7fb4bc0fbc20 2024-04-12T21:48:54.658 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:54.658+0000 7fb4ba000700 1 --2- 172.21.15.18:0/3755749257 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fb4a006ec00 0x7fb4a00710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:48:54.659 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:54.658+0000 7fb4ba000700 1 --2- 172.21.15.18:0/3755749257 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fb4a006ec00 0x7fb4a00710b0 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7fb4a800b3b0 tx=0x7fb4a8009e30 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:48:54.663 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:54.662+0000 7fb4b2a00700 1 -- 172.21.15.18:0/3755749257 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7fb4ac05b1d0 con 0x7fb4bc0fbc20 2024-04-12T21:48:54.784 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:48:54 smithi186 ceph-mon[86379]: from='client.36552 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:48:54.784 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:48:54 smithi186 ceph-mon[86379]: from='client.36556 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:48:54.784 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:48:54 smithi186 ceph-mon[86379]: from='client.36560 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:48:54.784 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:48:54 smithi186 ceph-mon[86379]: from='client.? 172.21.15.18:0/2167066394' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T21:48:54.855 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:48:54 smithi018 ceph-mon[112590]: from='client.36552 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:48:54.855 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:48:54 smithi018 ceph-mon[112590]: from='client.36556 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:48:54.855 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:48:54 smithi018 ceph-mon[112590]: from='client.36560 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:48:54.856 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:48:54 smithi018 ceph-mon[112590]: from='client.? 172.21.15.18:0/2167066394' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T21:48:54.942 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:54.941+0000 7fb4c1c00700 1 -- 172.21.15.18:0/3755749257 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fb4bc114c60 con 0x7fb4bc0fbc20 2024-04-12T21:48:54.943 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:54.942+0000 7fb4b2a00700 1 -- 172.21.15.18:0/3755749257 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 33 v33) v1 ==== 76+0+1876 (secure 0 0 0) 0x7fb4ac05ad60 con 0x7fb4bc0fbc20 2024-04-12T21:48:54.946 INFO:teuthology.orchestra.run.smithi018.stdout:e33 2024-04-12T21:48:54.946 INFO:teuthology.orchestra.run.smithi018.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-12T21:48:54.946 INFO:teuthology.orchestra.run.smithi018.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-12T21:48:54.946 INFO:teuthology.orchestra.run.smithi018.stdout:legacy client fscid: 1 2024-04-12T21:48:54.946 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-12T21:48:54.946 INFO:teuthology.orchestra.run.smithi018.stdout:Filesystem 'cephfs' (1) 2024-04-12T21:48:54.946 INFO:teuthology.orchestra.run.smithi018.stdout:fs_name cephfs 2024-04-12T21:48:54.946 INFO:teuthology.orchestra.run.smithi018.stdout:epoch 24 2024-04-12T21:48:54.947 INFO:teuthology.orchestra.run.smithi018.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-04-12T21:48:54.947 INFO:teuthology.orchestra.run.smithi018.stdout:created 2024-04-12T20:51:57.362850+0000 2024-04-12T21:48:54.947 INFO:teuthology.orchestra.run.smithi018.stdout:modified 2024-04-12T21:06:59.717190+0000 2024-04-12T21:48:54.947 INFO:teuthology.orchestra.run.smithi018.stdout:tableserver 0 2024-04-12T21:48:54.947 INFO:teuthology.orchestra.run.smithi018.stdout:root 0 2024-04-12T21:48:54.947 INFO:teuthology.orchestra.run.smithi018.stdout:session_timeout 60 2024-04-12T21:48:54.947 INFO:teuthology.orchestra.run.smithi018.stdout:session_autoclose 300 2024-04-12T21:48:54.947 INFO:teuthology.orchestra.run.smithi018.stdout:max_file_size 1099511627776 2024-04-12T21:48:54.947 INFO:teuthology.orchestra.run.smithi018.stdout:max_xattr_size 65536 2024-04-12T21:48:54.947 INFO:teuthology.orchestra.run.smithi018.stdout:required_client_features {} 2024-04-12T21:48:54.947 INFO:teuthology.orchestra.run.smithi018.stdout:last_failure 0 2024-04-12T21:48:54.947 INFO:teuthology.orchestra.run.smithi018.stdout:last_failure_osd_epoch 100 2024-04-12T21:48:54.947 INFO:teuthology.orchestra.run.smithi018.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-12T21:48:54.947 INFO:teuthology.orchestra.run.smithi018.stdout:max_mds 2 2024-04-12T21:48:54.947 INFO:teuthology.orchestra.run.smithi018.stdout:in 0,1 2024-04-12T21:48:54.948 INFO:teuthology.orchestra.run.smithi018.stdout:up {} 2024-04-12T21:48:54.948 INFO:teuthology.orchestra.run.smithi018.stdout:failed 0,1 2024-04-12T21:48:54.948 INFO:teuthology.orchestra.run.smithi018.stdout:damaged 2024-04-12T21:48:54.948 INFO:teuthology.orchestra.run.smithi018.stdout:stopped 2024-04-12T21:48:54.948 INFO:teuthology.orchestra.run.smithi018.stdout:data_pools [3] 2024-04-12T21:48:54.948 INFO:teuthology.orchestra.run.smithi018.stdout:metadata_pool 2 2024-04-12T21:48:54.948 INFO:teuthology.orchestra.run.smithi018.stdout:inline_data disabled 2024-04-12T21:48:54.948 INFO:teuthology.orchestra.run.smithi018.stdout:balancer 2024-04-12T21:48:54.948 INFO:teuthology.orchestra.run.smithi018.stdout:bal_rank_mask -1 2024-04-12T21:48:54.948 INFO:teuthology.orchestra.run.smithi018.stdout:standby_count_wanted 1 2024-04-12T21:48:54.948 INFO:teuthology.orchestra.run.smithi018.stdout:qdb_cluster leader: 0 members: 2024-04-12T21:48:54.948 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-12T21:48:54.948 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-12T21:48:54.948 INFO:teuthology.orchestra.run.smithi018.stdout:Standby daemons: 2024-04-12T21:48:54.948 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-12T21:48:54.949 INFO:teuthology.orchestra.run.smithi018.stdout:[mds.cephfs.smithi018.acqbsc{-1:34436} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.18:6828/755179764,v1:172.21.15.18:6829/755179764] compat {c=[1],r=[1],i=[1fff]}] 2024-04-12T21:48:54.949 INFO:teuthology.orchestra.run.smithi018.stdout:[mds.cephfs.smithi018.szjluo{-1:34468} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.18:6826/2407992653,v1:172.21.15.18:6827/2407992653] compat {c=[1],r=[1],i=[1fff]}] 2024-04-12T21:48:54.949 INFO:teuthology.orchestra.run.smithi018.stdout:[mds.cephfs.smithi186.jdllyn{-1:44595} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.186:6824/3523444202,v1:172.21.15.186:6825/3523444202] compat {c=[1],r=[1],i=[1fff]}] 2024-04-12T21:48:54.949 INFO:teuthology.orchestra.run.smithi018.stdout:[mds.cephfs.smithi186.wjnafs{-1:44875} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.186:6826/3122443766,v1:172.21.15.186:6827/3122443766] compat {c=[1],r=[1],i=[1fff]}] 2024-04-12T21:48:54.949 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:54.945+0000 7fb4c1c00700 1 -- 172.21.15.18:0/3755749257 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fb4a006ec00 msgr2=0x7fb4a00710b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:48:54.950 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:54.945+0000 7fb4c1c00700 1 --2- 172.21.15.18:0/3755749257 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fb4a006ec00 0x7fb4a00710b0 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7fb4a800b3b0 tx=0x7fb4a8009e30 comp rx=0 tx=0).stop 2024-04-12T21:48:54.950 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:54.945+0000 7fb4c1c00700 1 -- 172.21.15.18:0/3755749257 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb4bc0fbc20 msgr2=0x7fb4bc11a000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:48:54.950 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:54.945+0000 7fb4c1c00700 1 --2- 172.21.15.18:0/3755749257 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb4bc0fbc20 0x7fb4bc11a000 secure :-1 s=READY pgs=1249 cs=0 l=1 rev1=1 crypto rx=0x7fb4ac00b790 tx=0x7fb4ac009250 comp rx=0 tx=0).stop 2024-04-12T21:48:54.950 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:54.945+0000 7fb4c1c00700 1 -- 172.21.15.18:0/3755749257 shutdown_connections 2024-04-12T21:48:54.950 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:54.945+0000 7fb4c1c00700 1 --2- 172.21.15.18:0/3755749257 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fb4a006ec00 0x7fb4a00710b0 unknown :-1 s=CLOSED pgs=663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:48:54.950 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:54.945+0000 7fb4c1c00700 1 --2- 172.21.15.18:0/3755749257 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb4bc0fbc20 0x7fb4bc11a000 unknown :-1 s=CLOSED pgs=1249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:48:54.950 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:54.945+0000 7fb4c1c00700 1 --2- 172.21.15.18:0/3755749257 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb4bc0fc5c0 0x7fb4bc11a540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:48:54.950 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:54.945+0000 7fb4c1c00700 1 -- 172.21.15.18:0/3755749257 >> 172.21.15.18:0/3755749257 conn(0x7fb4bc0f7860 msgr2=0x7fb4bc0f8cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:48:54.950 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:54.945+0000 7fb4c1c00700 1 -- 172.21.15.18:0/3755749257 shutdown_connections 2024-04-12T21:48:54.950 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:54.946+0000 7fb4c1c00700 1 -- 172.21.15.18:0/3755749257 wait complete. 2024-04-12T21:48:54.950 INFO:teuthology.orchestra.run.smithi018.stderr:dumped fsmap epoch 33 2024-04-12T21:48:55.105 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:55.104+0000 7f801fc00700 1 -- 172.21.15.18:0/745296281 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8018105020 msgr2=0x7f80181053f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:48:55.106 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:55.104+0000 7f801fc00700 1 --2- 172.21.15.18:0/745296281 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8018105020 0x7f80181053f0 secure :-1 s=READY pgs=1250 cs=0 l=1 rev1=1 crypto rx=0x7f800c004940 tx=0x7f800c009e30 comp rx=0 tx=0).stop 2024-04-12T21:48:55.106 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:55.104+0000 7f801fc00700 1 -- 172.21.15.18:0/745296281 shutdown_connections 2024-04-12T21:48:55.106 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:55.104+0000 7f801fc00700 1 --2- 172.21.15.18:0/745296281 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f80181059c0 0x7f80180ff080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:48:55.106 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:55.104+0000 7f801fc00700 1 --2- 172.21.15.18:0/745296281 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8018105020 0x7f80181053f0 unknown :-1 s=CLOSED pgs=1250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:48:55.106 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:55.104+0000 7f801fc00700 1 -- 172.21.15.18:0/745296281 >> 172.21.15.18:0/745296281 conn(0x7f80180faba0 msgr2=0x7f80180fcfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:48:55.106 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:55.105+0000 7f801fc00700 1 -- 172.21.15.18:0/745296281 shutdown_connections 2024-04-12T21:48:55.106 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:55.105+0000 7f801fc00700 1 -- 172.21.15.18:0/745296281 wait complete. 2024-04-12T21:48:55.107 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:55.105+0000 7f801fc00700 1 Processor -- start 2024-04-12T21:48:55.108 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:55.105+0000 7f801fc00700 1 -- start start 2024-04-12T21:48:55.108 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:55.106+0000 7f801fc00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8018105020 0x7f8018115bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:48:55.108 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:55.106+0000 7f801fc00700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f80181059c0 0x7f8018116130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:48:55.108 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:55.106+0000 7f801fc00700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8018116810 con 0x7f8018105020 2024-04-12T21:48:55.108 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:55.106+0000 7f801fc00700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f801810fd60 con 0x7f80181059c0 2024-04-12T21:48:55.108 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:55.106+0000 7f801d200700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8018105020 0x7f8018115bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:48:55.108 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:55.106+0000 7f801d200700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8018105020 0x7f8018115bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:38484/0 (socket says 172.21.15.18:38484) 2024-04-12T21:48:55.108 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:55.106+0000 7f801d200700 1 -- 172.21.15.18:0/2152064429 learned_addr learned my addr 172.21.15.18:0/2152064429 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:48:55.108 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:55.106+0000 7f8017e00700 1 --2- 172.21.15.18:0/2152064429 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f80181059c0 0x7f8018116130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:48:55.108 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:55.107+0000 7f801d200700 1 -- 172.21.15.18:0/2152064429 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f80181059c0 msgr2=0x7f8018116130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:48:55.108 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:55.107+0000 7f801d200700 1 --2- 172.21.15.18:0/2152064429 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f80181059c0 0x7f8018116130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:48:55.108 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:55.107+0000 7f801d200700 1 -- 172.21.15.18:0/2152064429 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f800c005040 con 0x7f8018105020 2024-04-12T21:48:55.108 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:55.107+0000 7f8017e00700 1 --2- 172.21.15.18:0/2152064429 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f80181059c0 0x7f8018116130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-12T21:48:55.108 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:55.107+0000 7f801d200700 1 --2- 172.21.15.18:0/2152064429 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8018105020 0x7f8018115bf0 secure :-1 s=READY pgs=1251 cs=0 l=1 rev1=1 crypto rx=0x7f800c00b3b0 tx=0x7f800c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:48:55.109 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:55.107+0000 7f8014c00700 1 -- 172.21.15.18:0/2152064429 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f800c00e040 con 0x7f8018105020 2024-04-12T21:48:55.109 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:55.107+0000 7f8014c00700 1 -- 172.21.15.18:0/2152064429 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f800c00bbc0 con 0x7f8018105020 2024-04-12T21:48:55.109 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:55.107+0000 7f801fc00700 1 -- 172.21.15.18:0/2152064429 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f801810ff80 con 0x7f8018105020 2024-04-12T21:48:55.110 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:55.107+0000 7f8014c00700 1 -- 172.21.15.18:0/2152064429 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f800c01c3f0 con 0x7f8018105020 2024-04-12T21:48:55.110 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:55.107+0000 7f801fc00700 1 -- 172.21.15.18:0/2152064429 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f80181104c0 con 0x7f8018105020 2024-04-12T21:48:55.110 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:55.109+0000 7f801fc00700 1 -- 172.21.15.18:0/2152064429 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8018100770 con 0x7f8018105020 2024-04-12T21:48:55.110 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:55.110+0000 7f8014c00700 1 -- 172.21.15.18:0/2152064429 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7f800c00bd30 con 0x7f8018105020 2024-04-12T21:48:55.111 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:55.111+0000 7f8014c00700 1 --2- 172.21.15.18:0/2152064429 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f800006ec00 0x7f80000710b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:48:55.112 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:55.111+0000 7f8014c00700 1 -- 172.21.15.18:0/2152064429 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 6757+0+0 (secure 0 0 0) 0x7f800c08fb80 con 0x7f8018105020 2024-04-12T21:48:55.112 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:55.111+0000 7f8017e00700 1 --2- 172.21.15.18:0/2152064429 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f800006ec00 0x7f80000710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:48:55.112 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:55.112+0000 7f8017e00700 1 --2- 172.21.15.18:0/2152064429 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f800006ec00 0x7f80000710b0 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f8004006fd0 tx=0x7f8004008040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:48:55.117 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:55.116+0000 7f8014c00700 1 -- 172.21.15.18:0/2152064429 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f800c05c3d0 con 0x7f8018105020 2024-04-12T21:48:55.368 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:55.367+0000 7f801fc00700 1 -- 172.21.15.18:0/2152064429 --> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f80181107f0 con 0x7f800006ec00 2024-04-12T21:48:55.370 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:55.369+0000 7f8014c00700 1 -- 172.21.15.18:0/2152064429 <== mgr.34102 v2:172.21.15.18:6800/3812891919 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f80181107f0 con 0x7f800006ec00 2024-04-12T21:48:55.370 INFO:teuthology.orchestra.run.smithi018.stdout:{ 2024-04-12T21:48:55.370 INFO:teuthology.orchestra.run.smithi018.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:f4452def3be4d350f4a8e5feaf7ebb1511be9298fe68478ba7a2d6fbd7d69a63", 2024-04-12T21:48:55.370 INFO:teuthology.orchestra.run.smithi018.stdout: "in_progress": true, 2024-04-12T21:48:55.370 INFO:teuthology.orchestra.run.smithi018.stdout: "which": "Upgrading all daemon types on all hosts", 2024-04-12T21:48:55.370 INFO:teuthology.orchestra.run.smithi018.stdout: "services_complete": [ 2024-04-12T21:48:55.370 INFO:teuthology.orchestra.run.smithi018.stdout: "mgr", 2024-04-12T21:48:55.370 INFO:teuthology.orchestra.run.smithi018.stdout: "mds", 2024-04-12T21:48:55.370 INFO:teuthology.orchestra.run.smithi018.stdout: "crash", 2024-04-12T21:48:55.370 INFO:teuthology.orchestra.run.smithi018.stdout: "osd", 2024-04-12T21:48:55.371 INFO:teuthology.orchestra.run.smithi018.stdout: "mon" 2024-04-12T21:48:55.371 INFO:teuthology.orchestra.run.smithi018.stdout: ], 2024-04-12T21:48:55.371 INFO:teuthology.orchestra.run.smithi018.stdout: "progress": "16/21 daemons upgraded", 2024-04-12T21:48:55.371 INFO:teuthology.orchestra.run.smithi018.stdout: "message": "Currently upgrading mds daemons", 2024-04-12T21:48:55.371 INFO:teuthology.orchestra.run.smithi018.stdout: "is_paused": false 2024-04-12T21:48:55.371 INFO:teuthology.orchestra.run.smithi018.stdout:} 2024-04-12T21:48:55.372 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:55.372+0000 7f801fc00700 1 -- 172.21.15.18:0/2152064429 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f800006ec00 msgr2=0x7f80000710b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:48:55.373 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:55.372+0000 7f801fc00700 1 --2- 172.21.15.18:0/2152064429 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f800006ec00 0x7f80000710b0 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f8004006fd0 tx=0x7f8004008040 comp rx=0 tx=0).stop 2024-04-12T21:48:55.373 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:55.372+0000 7f801fc00700 1 -- 172.21.15.18:0/2152064429 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8018105020 msgr2=0x7f8018115bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:48:55.373 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:55.372+0000 7f801fc00700 1 --2- 172.21.15.18:0/2152064429 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8018105020 0x7f8018115bf0 secure :-1 s=READY pgs=1251 cs=0 l=1 rev1=1 crypto rx=0x7f800c00b3b0 tx=0x7f800c009e30 comp rx=0 tx=0).stop 2024-04-12T21:48:55.373 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:55.372+0000 7f801fc00700 1 -- 172.21.15.18:0/2152064429 shutdown_connections 2024-04-12T21:48:55.373 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:55.372+0000 7f801fc00700 1 --2- 172.21.15.18:0/2152064429 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f800006ec00 0x7f80000710b0 unknown :-1 s=CLOSED pgs=664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:48:55.373 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:55.372+0000 7f801fc00700 1 --2- 172.21.15.18:0/2152064429 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8018105020 0x7f8018115bf0 unknown :-1 s=CLOSED pgs=1251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:48:55.374 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:55.372+0000 7f801fc00700 1 --2- 172.21.15.18:0/2152064429 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f80181059c0 0x7f8018116130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:48:55.374 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:55.372+0000 7f801fc00700 1 -- 172.21.15.18:0/2152064429 >> 172.21.15.18:0/2152064429 conn(0x7f80180faba0 msgr2=0x7f80180fbb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:48:55.374 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:55.372+0000 7f801fc00700 1 -- 172.21.15.18:0/2152064429 shutdown_connections 2024-04-12T21:48:55.374 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:55.372+0000 7f801fc00700 1 -- 172.21.15.18:0/2152064429 wait complete. 2024-04-12T21:48:55.535 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:55.534+0000 7f6f7ca00700 1 -- 172.21.15.18:0/2051718835 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6f780fb4a0 msgr2=0x7f6f780fb910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:48:55.535 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:55.534+0000 7f6f7ca00700 1 --2- 172.21.15.18:0/2051718835 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6f780fb4a0 0x7f6f780fb910 secure :-1 s=READY pgs=1252 cs=0 l=1 rev1=1 crypto rx=0x7f6f64004940 tx=0x7f6f64009e30 comp rx=0 tx=0).stop 2024-04-12T21:48:55.535 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:55.534+0000 7f6f7ca00700 1 -- 172.21.15.18:0/2051718835 shutdown_connections 2024-04-12T21:48:55.535 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:55.534+0000 7f6f7ca00700 1 --2- 172.21.15.18:0/2051718835 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6f780fb4a0 0x7f6f780fb910 unknown :-1 s=CLOSED pgs=1252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:48:55.535 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:55.534+0000 7f6f7ca00700 1 --2- 172.21.15.18:0/2051718835 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6f78102e70 0x7f6f78103240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:48:55.536 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:55.534+0000 7f6f7ca00700 1 -- 172.21.15.18:0/2051718835 >> 172.21.15.18:0/2051718835 conn(0x7f6f780714c0 msgr2=0x7f6f780718c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:48:55.536 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:55.534+0000 7f6f7ca00700 1 -- 172.21.15.18:0/2051718835 shutdown_connections 2024-04-12T21:48:55.536 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:55.534+0000 7f6f7ca00700 1 -- 172.21.15.18:0/2051718835 wait complete. 2024-04-12T21:48:55.537 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:55.535+0000 7f6f7ca00700 1 Processor -- start 2024-04-12T21:48:55.537 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:55.535+0000 7f6f7ca00700 1 -- start start 2024-04-12T21:48:55.537 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:55.536+0000 7f6f7ca00700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6f780fb4a0 0x7f6f78115de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:48:55.537 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:55.536+0000 7f6f7ca00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6f78102e70 0x7f6f78116320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:48:55.537 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:55.536+0000 7f6f7ca00700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f78116970 con 0x7f6f78102e70 2024-04-12T21:48:55.537 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:55.536+0000 7f6f7ca00700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f7810fd40 con 0x7f6f780fb4a0 2024-04-12T21:48:55.537 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:55.536+0000 7f6f75400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6f78102e70 0x7f6f78116320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:48:55.537 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:55.536+0000 7f6f75400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6f78102e70 0x7f6f78116320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:38496/0 (socket says 172.21.15.18:38496) 2024-04-12T21:48:55.537 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:55.536+0000 7f6f75400700 1 -- 172.21.15.18:0/2232767494 learned_addr learned my addr 172.21.15.18:0/2232767494 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:48:55.538 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:55.536+0000 7f6f75e00700 1 --2- 172.21.15.18:0/2232767494 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6f780fb4a0 0x7f6f78115de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:48:55.538 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:55.537+0000 7f6f75400700 1 -- 172.21.15.18:0/2232767494 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6f780fb4a0 msgr2=0x7f6f78115de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:48:55.538 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:55.537+0000 7f6f75400700 1 --2- 172.21.15.18:0/2232767494 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6f780fb4a0 0x7f6f78115de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:48:55.539 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:55.537+0000 7f6f75400700 1 -- 172.21.15.18:0/2232767494 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f64005040 con 0x7f6f78102e70 2024-04-12T21:48:55.539 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:55.537+0000 7f6f75400700 1 --2- 172.21.15.18:0/2232767494 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6f78102e70 0x7f6f78116320 secure :-1 s=READY pgs=1253 cs=0 l=1 rev1=1 crypto rx=0x7f6f64009150 tx=0x7f6f64009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:48:55.539 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:55.537+0000 7f6f6e000700 1 -- 172.21.15.18:0/2232767494 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6f6400e040 con 0x7f6f78102e70 2024-04-12T21:48:55.539 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:55.537+0000 7f6f6e000700 1 -- 172.21.15.18:0/2232767494 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f6f64013420 con 0x7f6f78102e70 2024-04-12T21:48:55.539 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:55.537+0000 7f6f6e000700 1 -- 172.21.15.18:0/2232767494 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6f6401c630 con 0x7f6f78102e70 2024-04-12T21:48:55.539 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:55.537+0000 7f6f7ca00700 1 -- 172.21.15.18:0/2232767494 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6f7810ffc0 con 0x7f6f78102e70 2024-04-12T21:48:55.539 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:55.538+0000 7f6f7ca00700 1 -- 172.21.15.18:0/2232767494 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6f78110500 con 0x7f6f78102e70 2024-04-12T21:48:55.539 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:55.539+0000 7f6f7ca00700 1 -- 172.21.15.18:0/2232767494 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6f78107330 con 0x7f6f78102e70 2024-04-12T21:48:55.540 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:55.540+0000 7f6f6e000700 1 -- 172.21.15.18:0/2232767494 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7f6f64013590 con 0x7f6f78102e70 2024-04-12T21:48:55.541 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:55.541+0000 7f6f6e000700 1 --2- 172.21.15.18:0/2232767494 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f6f5c06ec10 0x7f6f5c0710c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:48:55.541 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:55.541+0000 7f6f6e000700 1 -- 172.21.15.18:0/2232767494 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 6757+0+0 (secure 0 0 0) 0x7f6f6408fb50 con 0x7f6f78102e70 2024-04-12T21:48:55.542 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:55.541+0000 7f6f75e00700 1 --2- 172.21.15.18:0/2232767494 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f6f5c06ec10 0x7f6f5c0710c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:48:55.542 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:55.542+0000 7f6f75e00700 1 --2- 172.21.15.18:0/2232767494 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f6f5c06ec10 0x7f6f5c0710c0 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f6f68007fd0 tx=0x7f6f68009040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:48:55.546 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:55.546+0000 7f6f6e000700 1 -- 172.21.15.18:0/2232767494 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f6f6405c440 con 0x7f6f78102e70 2024-04-12T21:48:55.784 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:48:55 smithi186 ceph-mon[86379]: pgmap v1477: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:48:55.784 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:48:55 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T21:48:55.784 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:48:55 smithi186 ceph-mon[86379]: from='client.? 172.21.15.18:0/3755749257' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-12T21:48:55.854 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:55.853+0000 7f6f7ca00700 1 -- 172.21.15.18:0/2232767494 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f6f780653a0 con 0x7f6f78102e70 2024-04-12T21:48:55.855 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:48:55 smithi018 ceph-mon[112590]: pgmap v1477: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:48:55.855 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:48:55 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T21:48:55.855 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:48:55 smithi018 ceph-mon[112590]: from='client.? 172.21.15.18:0/3755749257' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-12T21:48:55.855 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:55.854+0000 7f6f6e000700 1 -- 172.21.15.18:0/2232767494 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+4303 (secure 0 0 0) 0x7f6f6405bfd0 con 0x7f6f78102e70 2024-04-12T21:48:55.855 INFO:teuthology.orchestra.run.smithi018.stdout:HEALTH_ERR failed to probe daemons or devices; 1 filesystem is degraded; 1 filesystem is offline 2024-04-12T21:48:55.856 INFO:teuthology.orchestra.run.smithi018.stdout:[WRN] CEPHADM_REFRESH_FAILED: failed to probe daemons or devices 2024-04-12T21:48:55.856 INFO:teuthology.orchestra.run.smithi018.stdout: host smithi186 `cephadm ceph-volume` failed: cephadm exited with an error code: 1, stderr: Inferring config /var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/mon.smithi186/config 2024-04-12T21:48:55.856 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/podman: timeout after 895 seconds 2024-04-12T21:48:55.856 INFO:teuthology.orchestra.run.smithi018.stdout:Non-zero exit code 124 from /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:f4452def3be4d350f4a8e5feaf7ebb1511be9298fe68478ba7a2d6fbd7d69a63 -e NODE_NAME=smithi186 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968:/var/run/ceph:z -v /var/log/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968:/var/log/ceph:z -v /var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmptgom53f9:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:f4452def3be4d350f4a8e5feaf7ebb1511be9298fe68478ba7a2d6fbd7d69a63 inventory --format=json-pretty --filter-for-batch 2024-04-12T21:48:55.856 INFO:teuthology.orchestra.run.smithi018.stdout:Traceback (most recent call last): 2024-04-12T21:48:55.856 INFO:teuthology.orchestra.run.smithi018.stdout: File "/usr/lib64/python3.9/runpy.py", line 197, in _run_module_as_main 2024-04-12T21:48:55.856 INFO:teuthology.orchestra.run.smithi018.stdout: return _run_code(code, main_globals, None, 2024-04-12T21:48:55.856 INFO:teuthology.orchestra.run.smithi018.stdout: File "/usr/lib64/python3.9/runpy.py", line 87, in _run_code 2024-04-12T21:48:55.856 INFO:teuthology.orchestra.run.smithi018.stdout: exec(code, run_globals) 2024-04-12T21:48:55.856 INFO:teuthology.orchestra.run.smithi018.stdout: File "/var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/cephadm.e44f3ed5e34e4ef52eedef233d1190bff004470d8e8005127b5312bff357fa7f/__main__.py", line 5618, in 2024-04-12T21:48:55.856 INFO:teuthology.orchestra.run.smithi018.stdout: File "/var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/cephadm.e44f3ed5e34e4ef52eedef233d1190bff004470d8e8005127b5312bff357fa7f/__main__.py", line 5606, in main 2024-04-12T21:48:55.856 INFO:teuthology.orchestra.run.smithi018.stdout: File "/var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/cephadm.e44f3ed5e34e4ef52eedef233d1190bff004470d8e8005127b5312bff357fa7f/__main__.py", line 408, in _infer_config 2024-04-12T21:48:55.856 INFO:teuthology.orchestra.run.smithi018.stdout: File "/var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/cephadm.e44f3ed5e34e4ef52eedef233d1190bff004470d8e8005127b5312bff357fa7f/__main__.py", line 323, in _infer_fsid 2024-04-12T21:48:55.856 INFO:teuthology.orchestra.run.smithi018.stdout: File "/var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/cephadm.e44f3ed5e34e4ef52eedef233d1190bff004470d8e8005127b5312bff357fa7f/__main__.py", line 436, in _infer_image 2024-04-12T21:48:55.856 INFO:teuthology.orchestra.run.smithi018.stdout: File "/var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/cephadm.e44f3ed5e34e4ef52eedef233d1190bff004470d8e8005127b5312bff357fa7f/__main__.py", line 310, in _validate_fsid 2024-04-12T21:48:55.857 INFO:teuthology.orchestra.run.smithi018.stdout: File "/var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/cephadm.e44f3ed5e34e4ef52eedef233d1190bff004470d8e8005127b5312bff357fa7f/__main__.py", line 3359, in command_ceph_volume 2024-04-12T21:48:55.857 INFO:teuthology.orchestra.run.smithi018.stdout: File "/var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/cephadm.e44f3ed5e34e4ef52eedef233d1190bff004470d8e8005127b5312bff357fa7f/cephadmlib/call_wrappers.py", line 310, in call_throws 2024-04-12T21:48:55.857 INFO:teuthology.orchestra.run.smithi018.stdout:RuntimeError: Failed command: /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:f4452def3be4d350f4a8e5feaf7ebb1511be9298fe68478ba7a2d6fbd7d69a63 -e NODE_NAME=smithi186 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968:/var/run/ceph:z -v /var/log/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968:/var/log/ceph:z -v /var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmptgom53f9:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:f4452def3be4d350f4a8e5feaf7ebb1511be9298fe68478ba7a2d6fbd7d69a63 inventory --format=json-pretty --filter-for-batch 2024-04-12T21:48:55.857 INFO:teuthology.orchestra.run.smithi018.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-04-12T21:48:55.857 INFO:teuthology.orchestra.run.smithi018.stdout: fs cephfs is degraded 2024-04-12T21:48:55.857 INFO:teuthology.orchestra.run.smithi018.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-04-12T21:48:55.857 INFO:teuthology.orchestra.run.smithi018.stdout: fs cephfs is offline because no MDS is active for it. 2024-04-12T21:48:55.859 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:55.858+0000 7f6f7ca00700 1 -- 172.21.15.18:0/2232767494 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f6f5c06ec10 msgr2=0x7f6f5c0710c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:48:55.859 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:55.858+0000 7f6f7ca00700 1 --2- 172.21.15.18:0/2232767494 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f6f5c06ec10 0x7f6f5c0710c0 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f6f68007fd0 tx=0x7f6f68009040 comp rx=0 tx=0).stop 2024-04-12T21:48:55.859 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:55.858+0000 7f6f7ca00700 1 -- 172.21.15.18:0/2232767494 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6f78102e70 msgr2=0x7f6f78116320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:48:55.859 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:55.858+0000 7f6f7ca00700 1 --2- 172.21.15.18:0/2232767494 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6f78102e70 0x7f6f78116320 secure :-1 s=READY pgs=1253 cs=0 l=1 rev1=1 crypto rx=0x7f6f64009150 tx=0x7f6f64009e30 comp rx=0 tx=0).stop 2024-04-12T21:48:55.859 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:55.858+0000 7f6f7ca00700 1 -- 172.21.15.18:0/2232767494 shutdown_connections 2024-04-12T21:48:55.859 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:55.858+0000 7f6f7ca00700 1 --2- 172.21.15.18:0/2232767494 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f6f5c06ec10 0x7f6f5c0710c0 unknown :-1 s=CLOSED pgs=665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:48:55.859 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:55.858+0000 7f6f7ca00700 1 --2- 172.21.15.18:0/2232767494 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6f780fb4a0 0x7f6f78115de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:48:55.859 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:55.858+0000 7f6f7ca00700 1 --2- 172.21.15.18:0/2232767494 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6f78102e70 0x7f6f78116320 unknown :-1 s=CLOSED pgs=1253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:48:55.860 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:55.858+0000 7f6f7ca00700 1 -- 172.21.15.18:0/2232767494 >> 172.21.15.18:0/2232767494 conn(0x7f6f780714c0 msgr2=0x7f6f780fa590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:48:55.860 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:55.858+0000 7f6f7ca00700 1 -- 172.21.15.18:0/2232767494 shutdown_connections 2024-04-12T21:48:55.860 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:48:55.858+0000 7f6f7ca00700 1 -- 172.21.15.18:0/2232767494 wait complete. 2024-04-12T21:48:56.855 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:48:56 smithi018 ceph-mon[112590]: from='client.36572 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:48:56.855 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:48:56 smithi018 ceph-mon[112590]: from='client.? 172.21.15.18:0/2232767494' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-12T21:48:57.034 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:48:56 smithi186 ceph-mon[86379]: from='client.36572 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:48:57.034 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:48:56 smithi186 ceph-mon[86379]: from='client.? 172.21.15.18:0/2232767494' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-12T21:48:57.855 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:48:57 smithi018 ceph-mon[112590]: pgmap v1478: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:48:57.936 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:48:57 smithi186 ceph-mon[86379]: pgmap v1478: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:48:59.857 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:48:59 smithi018 ceph-mon[112590]: pgmap v1479: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:49:00.034 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:48:59 smithi186 ceph-mon[86379]: pgmap v1479: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:49:01.825 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:49:01 smithi186 ceph-mon[86379]: pgmap v1480: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:49:01.855 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:49:01 smithi018 ceph-mon[112590]: pgmap v1480: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:49:03.855 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:49:03 smithi018 ceph-mon[112590]: pgmap v1481: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:49:04.034 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:49:03 smithi186 ceph-mon[86379]: pgmap v1481: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:49:05.855 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:49:05 smithi018 ceph-mon[112590]: pgmap v1482: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:49:06.034 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:49:05 smithi186 ceph-mon[86379]: pgmap v1482: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:49:08.034 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:49:07 smithi186 ceph-mon[86379]: pgmap v1483: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:49:08.105 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:49:07 smithi018 ceph-mon[112590]: pgmap v1483: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:49:10.034 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:49:09 smithi186 ceph-mon[86379]: pgmap v1484: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:49:10.105 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:49:09 smithi018 ceph-mon[112590]: pgmap v1484: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:49:11.034 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:49:10 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T21:49:11.105 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:49:10 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T21:49:12.034 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:49:11 smithi186 ceph-mon[86379]: pgmap v1485: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:49:12.105 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:49:11 smithi018 ceph-mon[112590]: pgmap v1485: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:49:14.034 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:49:13 smithi186 ceph-mon[86379]: pgmap v1486: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:49:14.105 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:49:13 smithi018 ceph-mon[112590]: pgmap v1486: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:49:16.034 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:49:15 smithi186 ceph-mon[86379]: pgmap v1487: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:49:16.105 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:49:15 smithi018 ceph-mon[112590]: pgmap v1487: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:49:18.034 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:49:17 smithi186 ceph-mon[86379]: pgmap v1488: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:49:18.105 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:49:17 smithi018 ceph-mon[112590]: pgmap v1488: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:49:20.034 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:49:19 smithi186 ceph-mon[86379]: pgmap v1489: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:49:20.105 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:49:19 smithi018 ceph-mon[112590]: pgmap v1489: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:49:22.034 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:49:21 smithi186 ceph-mon[86379]: pgmap v1490: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:49:22.105 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:49:21 smithi018 ceph-mon[112590]: pgmap v1490: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:49:24.034 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:49:23 smithi186 ceph-mon[86379]: pgmap v1491: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:49:24.105 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:49:23 smithi018 ceph-mon[112590]: pgmap v1491: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:49:26.031 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:49:26.030+0000 7f45c1c00700 1 -- 172.21.15.18:0/1073960858 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f45bc104e60 msgr2=0x7f45bc105230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:49:26.031 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:49:26.030+0000 7f45c1c00700 1 --2- 172.21.15.18:0/1073960858 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f45bc104e60 0x7f45bc105230 secure :-1 s=READY pgs=1254 cs=0 l=1 rev1=1 crypto rx=0x7f45ac004940 tx=0x7f45ac009e30 comp rx=0 tx=0).stop 2024-04-12T21:49:26.032 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:49:25 smithi018 ceph-mon[112590]: pgmap v1492: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:49:26.032 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:49:25 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T21:49:26.034 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:49:26.030+0000 7f45c1c00700 1 -- 172.21.15.18:0/1073960858 shutdown_connections 2024-04-12T21:49:26.034 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:49:26.030+0000 7f45c1c00700 1 --2- 172.21.15.18:0/1073960858 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f45bc105800 0x7f45bc0fee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:49:26.034 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:49:26.030+0000 7f45c1c00700 1 --2- 172.21.15.18:0/1073960858 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f45bc104e60 0x7f45bc105230 unknown :-1 s=CLOSED pgs=1254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:49:26.034 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:49:26.030+0000 7f45c1c00700 1 -- 172.21.15.18:0/1073960858 >> 172.21.15.18:0/1073960858 conn(0x7f45bc0fab10 msgr2=0x7f45bc0fcef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:49:26.034 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:49:26.031+0000 7f45c1c00700 1 -- 172.21.15.18:0/1073960858 shutdown_connections 2024-04-12T21:49:26.034 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:49:26.031+0000 7f45c1c00700 1 -- 172.21.15.18:0/1073960858 wait complete. 2024-04-12T21:49:26.034 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:49:26.031+0000 7f45c1c00700 1 Processor -- start 2024-04-12T21:49:26.035 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:49:26.032+0000 7f45c1c00700 1 -- start start 2024-04-12T21:49:26.035 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:49:26.032+0000 7f45c1c00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f45bc104e60 0x7f45bc115ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:49:26.035 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:49:26.032+0000 7f45c1c00700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f45bc105800 0x7f45bc1160e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:49:26.035 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:49:26.032+0000 7f45c1c00700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45bc116730 con 0x7f45bc104e60 2024-04-12T21:49:26.035 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:49:26.032+0000 7f45c1c00700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45bc10fb40 con 0x7f45bc105800 2024-04-12T21:49:26.035 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:49:26.032+0000 7f45bbe00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f45bc104e60 0x7f45bc115ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:49:26.035 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:49:26.032+0000 7f45bbe00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f45bc104e60 0x7f45bc115ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:51654/0 (socket says 172.21.15.18:51654) 2024-04-12T21:49:26.035 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:49:26.032+0000 7f45bbe00700 1 -- 172.21.15.18:0/3484794065 learned_addr learned my addr 172.21.15.18:0/3484794065 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:49:26.035 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:49:26.033+0000 7f45bb400700 1 --2- 172.21.15.18:0/3484794065 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f45bc105800 0x7f45bc1160e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:49:26.035 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:49:26.033+0000 7f45bbe00700 1 -- 172.21.15.18:0/3484794065 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f45bc105800 msgr2=0x7f45bc1160e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:49:26.036 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:49:26.033+0000 7f45bbe00700 1 --2- 172.21.15.18:0/3484794065 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f45bc105800 0x7f45bc1160e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:49:26.036 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:49:26.033+0000 7f45bbe00700 1 -- 172.21.15.18:0/3484794065 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45ac005040 con 0x7f45bc104e60 2024-04-12T21:49:26.036 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:49:26.033+0000 7f45bbe00700 1 --2- 172.21.15.18:0/3484794065 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f45bc104e60 0x7f45bc115ba0 secure :-1 s=READY pgs=1255 cs=0 l=1 rev1=1 crypto rx=0x7f45ac00b3b0 tx=0x7f45ac009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:49:26.036 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:49:25 smithi186 ceph-mon[86379]: pgmap v1492: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:49:26.037 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:49:25 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T21:49:26.038 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:49:26.033+0000 7f45b3e00700 1 -- 172.21.15.18:0/3484794065 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f45ac00e040 con 0x7f45bc104e60 2024-04-12T21:49:26.038 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:49:26.033+0000 7f45b3e00700 1 -- 172.21.15.18:0/3484794065 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f45ac00bbc0 con 0x7f45bc104e60 2024-04-12T21:49:26.038 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:49:26.033+0000 7f45c1c00700 1 -- 172.21.15.18:0/3484794065 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f45bc10fdc0 con 0x7f45bc104e60 2024-04-12T21:49:26.038 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:49:26.033+0000 7f45b3e00700 1 -- 172.21.15.18:0/3484794065 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f45ac01c3f0 con 0x7f45bc104e60 2024-04-12T21:49:26.038 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:49:26.034+0000 7f45c1c00700 1 -- 172.21.15.18:0/3484794065 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f45bc110380 con 0x7f45bc104e60 2024-04-12T21:49:26.038 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:49:26.035+0000 7f45c1c00700 1 -- 172.21.15.18:0/3484794065 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f45bc1005b0 con 0x7f45bc104e60 2024-04-12T21:49:26.038 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:49:26.036+0000 7f45b3e00700 1 -- 172.21.15.18:0/3484794065 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7f45ac00bd30 con 0x7f45bc104e60 2024-04-12T21:49:26.038 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:49:26.037+0000 7f45b3e00700 1 --2- 172.21.15.18:0/3484794065 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f45a006ec00 0x7f45a00710b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:49:26.039 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:49:26.037+0000 7f45b3e00700 1 -- 172.21.15.18:0/3484794065 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 6757+0+0 (secure 0 0 0) 0x7f45ac08fb80 con 0x7f45bc104e60 2024-04-12T21:49:26.039 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:49:26.037+0000 7f45bb400700 1 --2- 172.21.15.18:0/3484794065 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f45a006ec00 0x7f45a00710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:49:26.039 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:49:26.038+0000 7f45bb400700 1 --2- 172.21.15.18:0/3484794065 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f45a006ec00 0x7f45a00710b0 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f45a8007fd0 tx=0x7f45a8009040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:49:26.043 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:49:26.042+0000 7f45b3e00700 1 -- 172.21.15.18:0/3484794065 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f45ac0588a0 con 0x7f45bc104e60 2024-04-12T21:49:26.298 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:49:26.297+0000 7f45c1c00700 1 -- 172.21.15.18:0/3484794065 --> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f45bc10bfd0 con 0x7f45a006ec00 2024-04-12T21:49:26.300 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:49:26.300+0000 7f45b3e00700 1 -- 172.21.15.18:0/3484794065 <== mgr.34102 v2:172.21.15.18:6800/3812891919 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f45bc10bfd0 con 0x7f45a006ec00 2024-04-12T21:49:26.304 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:49:26.303+0000 7f45c1c00700 1 -- 172.21.15.18:0/3484794065 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f45a006ec00 msgr2=0x7f45a00710b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:49:26.305 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:49:26.303+0000 7f45c1c00700 1 --2- 172.21.15.18:0/3484794065 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f45a006ec00 0x7f45a00710b0 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f45a8007fd0 tx=0x7f45a8009040 comp rx=0 tx=0).stop 2024-04-12T21:49:26.305 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:49:26.303+0000 7f45c1c00700 1 -- 172.21.15.18:0/3484794065 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f45bc104e60 msgr2=0x7f45bc115ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:49:26.305 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:49:26.303+0000 7f45c1c00700 1 --2- 172.21.15.18:0/3484794065 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f45bc104e60 0x7f45bc115ba0 secure :-1 s=READY pgs=1255 cs=0 l=1 rev1=1 crypto rx=0x7f45ac00b3b0 tx=0x7f45ac009e30 comp rx=0 tx=0).stop 2024-04-12T21:49:26.305 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:49:26.303+0000 7f45c1c00700 1 -- 172.21.15.18:0/3484794065 shutdown_connections 2024-04-12T21:49:26.305 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:49:26.303+0000 7f45c1c00700 1 --2- 172.21.15.18:0/3484794065 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f45a006ec00 0x7f45a00710b0 unknown :-1 s=CLOSED pgs=666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:49:26.305 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:49:26.303+0000 7f45c1c00700 1 --2- 172.21.15.18:0/3484794065 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f45bc104e60 0x7f45bc115ba0 unknown :-1 s=CLOSED pgs=1255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:49:26.305 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:49:26.303+0000 7f45c1c00700 1 --2- 172.21.15.18:0/3484794065 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f45bc105800 0x7f45bc1160e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:49:26.306 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:49:26.303+0000 7f45c1c00700 1 -- 172.21.15.18:0/3484794065 >> 172.21.15.18:0/3484794065 conn(0x7f45bc0fab10 msgr2=0x7f45bc0fca90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:49:26.306 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:49:26.304+0000 7f45c1c00700 1 -- 172.21.15.18:0/3484794065 shutdown_connections 2024-04-12T21:49:26.306 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:49:26.304+0000 7f45c1c00700 1 -- 172.21.15.18:0/3484794065 wait complete. 2024-04-12T21:49:26.321 INFO:teuthology.orchestra.run.smithi018.stdout:true 2024-04-12T21:49:26.472 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:49:26.467+0000 7f3ecf400700 1 -- 172.21.15.18:0/3989277943 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3ec8109430 msgr2=0x7f3ec8109800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:49:26.472 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:49:26.467+0000 7f3ecf400700 1 --2- 172.21.15.18:0/3989277943 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3ec8109430 0x7f3ec8109800 secure :-1 s=READY pgs=1256 cs=0 l=1 rev1=1 crypto rx=0x7f3eb8004940 tx=0x7f3eb8009e30 comp rx=0 tx=0).stop 2024-04-12T21:49:26.472 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:49:26.467+0000 7f3ecf400700 1 -- 172.21.15.18:0/3989277943 shutdown_connections 2024-04-12T21:49:26.472 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:49:26.467+0000 7f3ecf400700 1 --2- 172.21.15.18:0/3989277943 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3ec8109dd0 0x7f3ec8103440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:49:26.472 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:49:26.467+0000 7f3ecf400700 1 --2- 172.21.15.18:0/3989277943 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3ec8109430 0x7f3ec8109800 unknown :-1 s=CLOSED pgs=1256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:49:26.472 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:49:26.467+0000 7f3ecf400700 1 -- 172.21.15.18:0/3989277943 >> 172.21.15.18:0/3989277943 conn(0x7f3ec8072c00 msgr2=0x7f3ec8075010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:49:26.472 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:49:26.468+0000 7f3ecf400700 1 -- 172.21.15.18:0/3989277943 shutdown_connections 2024-04-12T21:49:26.472 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:49:26.468+0000 7f3ecf400700 1 -- 172.21.15.18:0/3989277943 wait complete. 2024-04-12T21:49:26.472 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:49:26.468+0000 7f3ecf400700 1 Processor -- start 2024-04-12T21:49:26.472 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:49:26.469+0000 7f3ecf400700 1 -- start start 2024-04-12T21:49:26.472 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:49:26.469+0000 7f3ecf400700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3ec8109430 0x7f3ec8119f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:49:26.473 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:49:26.469+0000 7f3ecf400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3ec8109dd0 0x7f3ec811a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:49:26.473 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:49:26.469+0000 7f3ecf400700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ec811aad0 con 0x7f3ec8109dd0 2024-04-12T21:49:26.473 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:49:26.469+0000 7f3ecf400700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ec8114100 con 0x7f3ec8109430 2024-04-12T21:49:26.473 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:49:26.469+0000 7f3ebfe00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3ec8109dd0 0x7f3ec811a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:49:26.473 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:49:26.469+0000 7f3ebfe00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3ec8109dd0 0x7f3ec811a480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:51660/0 (socket says 172.21.15.18:51660) 2024-04-12T21:49:26.473 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:49:26.469+0000 7f3ebfe00700 1 -- 172.21.15.18:0/3009079480 learned_addr learned my addr 172.21.15.18:0/3009079480 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:49:26.473 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:49:26.470+0000 7f3ecca00700 1 --2- 172.21.15.18:0/3009079480 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3ec8109430 0x7f3ec8119f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:49:26.473 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:49:26.470+0000 7f3ebfe00700 1 -- 172.21.15.18:0/3009079480 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3ec8109430 msgr2=0x7f3ec8119f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:49:26.473 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:49:26.470+0000 7f3ebfe00700 1 --2- 172.21.15.18:0/3009079480 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3ec8109430 0x7f3ec8119f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:49:26.473 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:49:26.470+0000 7f3ebfe00700 1 -- 172.21.15.18:0/3009079480 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3eb8005040 con 0x7f3ec8109dd0 2024-04-12T21:49:26.473 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:49:26.470+0000 7f3ebfe00700 1 --2- 172.21.15.18:0/3009079480 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3ec8109dd0 0x7f3ec811a480 secure :-1 s=READY pgs=1257 cs=0 l=1 rev1=1 crypto rx=0x7f3eb0002a50 tx=0x7f3eb0004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:49:26.473 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:49:26.470+0000 7f3ec5600700 1 -- 172.21.15.18:0/3009079480 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3eb0007cc0 con 0x7f3ec8109dd0 2024-04-12T21:49:26.473 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:49:26.470+0000 7f3ec5600700 1 -- 172.21.15.18:0/3009079480 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f3eb000f040 con 0x7f3ec8109dd0 2024-04-12T21:49:26.473 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:49:26.471+0000 7f3ec5600700 1 -- 172.21.15.18:0/3009079480 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3eb000daf0 con 0x7f3ec8109dd0 2024-04-12T21:49:26.473 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:49:26.471+0000 7f3ecf400700 1 -- 172.21.15.18:0/3009079480 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3ec81143e0 con 0x7f3ec8109dd0 2024-04-12T21:49:26.474 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:49:26.471+0000 7f3ecf400700 1 -- 172.21.15.18:0/3009079480 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3ec81149a0 con 0x7f3ec8109dd0 2024-04-12T21:49:26.474 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:49:26.472+0000 7f3ecf400700 1 -- 172.21.15.18:0/3009079480 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3ec8104b80 con 0x7f3ec8109dd0 2024-04-12T21:49:26.474 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:49:26.473+0000 7f3ec5600700 1 -- 172.21.15.18:0/3009079480 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7f3eb0009810 con 0x7f3ec8109dd0 2024-04-12T21:49:26.475 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:49:26.474+0000 7f3ec5600700 1 --2- 172.21.15.18:0/3009079480 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f3eac06ec00 0x7f3eac0710b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:49:26.475 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:49:26.474+0000 7f3ec5600700 1 -- 172.21.15.18:0/3009079480 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 6757+0+0 (secure 0 0 0) 0x7f3eb008e700 con 0x7f3ec8109dd0 2024-04-12T21:49:26.475 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:49:26.475+0000 7f3ecca00700 1 --2- 172.21.15.18:0/3009079480 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f3eac06ec00 0x7f3eac0710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:49:26.476 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:49:26.475+0000 7f3ecca00700 1 --2- 172.21.15.18:0/3009079480 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f3eac06ec00 0x7f3eac0710b0 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7f3eb800b3b0 tx=0x7f3eb8009e30 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:49:26.480 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:49:26.479+0000 7f3ec5600700 1 -- 172.21.15.18:0/3009079480 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f3eb005aeb0 con 0x7f3ec8109dd0 2024-04-12T21:49:26.733 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:49:26.732+0000 7f3ecf400700 1 -- 172.21.15.18:0/3009079480 --> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f3ec8110590 con 0x7f3eac06ec00 2024-04-12T21:49:26.735 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:49:26.735+0000 7f3ec5600700 1 -- 172.21.15.18:0/3009079480 <== mgr.34102 v2:172.21.15.18:6800/3812891919 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f3ec8110590 con 0x7f3eac06ec00 2024-04-12T21:49:26.739 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:49:26.738+0000 7f3ecf400700 1 -- 172.21.15.18:0/3009079480 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f3eac06ec00 msgr2=0x7f3eac0710b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:49:26.739 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:49:26.738+0000 7f3ecf400700 1 --2- 172.21.15.18:0/3009079480 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f3eac06ec00 0x7f3eac0710b0 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7f3eb800b3b0 tx=0x7f3eb8009e30 comp rx=0 tx=0).stop 2024-04-12T21:49:26.739 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:49:26.738+0000 7f3ecf400700 1 -- 172.21.15.18:0/3009079480 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3ec8109dd0 msgr2=0x7f3ec811a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:49:26.739 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:49:26.738+0000 7f3ecf400700 1 --2- 172.21.15.18:0/3009079480 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3ec8109dd0 0x7f3ec811a480 secure :-1 s=READY pgs=1257 cs=0 l=1 rev1=1 crypto rx=0x7f3eb0002a50 tx=0x7f3eb0004600 comp rx=0 tx=0).stop 2024-04-12T21:49:26.739 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:49:26.738+0000 7f3ecf400700 1 -- 172.21.15.18:0/3009079480 shutdown_connections 2024-04-12T21:49:26.739 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:49:26.738+0000 7f3ecf400700 1 --2- 172.21.15.18:0/3009079480 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f3eac06ec00 0x7f3eac0710b0 unknown :-1 s=CLOSED pgs=667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:49:26.739 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:49:26.738+0000 7f3ecf400700 1 --2- 172.21.15.18:0/3009079480 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3ec8109430 0x7f3ec8119f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:49:26.739 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:49:26.738+0000 7f3ecf400700 1 --2- 172.21.15.18:0/3009079480 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3ec8109dd0 0x7f3ec811a480 unknown :-1 s=CLOSED pgs=1257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:49:26.739 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:49:26.738+0000 7f3ecf400700 1 -- 172.21.15.18:0/3009079480 >> 172.21.15.18:0/3009079480 conn(0x7f3ec8072c00 msgr2=0x7f3ec8074c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:49:26.740 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:49:26.738+0000 7f3ecf400700 1 -- 172.21.15.18:0/3009079480 shutdown_connections 2024-04-12T21:49:26.740 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:49:26.738+0000 7f3ecf400700 1 -- 172.21.15.18:0/3009079480 wait complete. 2024-04-12T21:49:26.894 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:49:26.892+0000 7f4dd6a00700 1 -- 172.21.15.18:0/2440760863 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4dd0105800 msgr2=0x7f4dd00fee70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:49:26.894 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:49:26.892+0000 7f4dd6a00700 1 --2- 172.21.15.18:0/2440760863 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4dd0105800 0x7f4dd00fee70 secure :-1 s=READY pgs=1258 cs=0 l=1 rev1=1 crypto rx=0x7f4dbc004940 tx=0x7f4dbc009e30 comp rx=0 tx=0).stop 2024-04-12T21:49:26.894 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:49:26.893+0000 7f4dd6a00700 1 -- 172.21.15.18:0/2440760863 shutdown_connections 2024-04-12T21:49:26.894 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:49:26.893+0000 7f4dd6a00700 1 --2- 172.21.15.18:0/2440760863 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4dd0105800 0x7f4dd00fee70 unknown :-1 s=CLOSED pgs=1258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:49:26.894 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:49:26.893+0000 7f4dd6a00700 1 --2- 172.21.15.18:0/2440760863 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4dd0104e60 0x7f4dd0105230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:49:26.894 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:49:26.893+0000 7f4dd6a00700 1 -- 172.21.15.18:0/2440760863 >> 172.21.15.18:0/2440760863 conn(0x7f4dd00fab10 msgr2=0x7f4dd00fcef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:49:26.894 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:49:26.893+0000 7f4dd6a00700 1 -- 172.21.15.18:0/2440760863 shutdown_connections 2024-04-12T21:49:26.894 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:49:26.893+0000 7f4dd6a00700 1 -- 172.21.15.18:0/2440760863 wait complete. 2024-04-12T21:49:26.896 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:49:26.894+0000 7f4dd6a00700 1 Processor -- start 2024-04-12T21:49:26.896 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:49:26.894+0000 7f4dd6a00700 1 -- start start 2024-04-12T21:49:26.896 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:49:26.894+0000 7f4dd6a00700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4dd0104e60 0x7f4dd0115c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:49:26.896 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:49:26.894+0000 7f4dd6a00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4dd0105800 0x7f4dd0116150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:49:26.896 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:49:26.894+0000 7f4dd6a00700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4dd01167a0 con 0x7f4dd0105800 2024-04-12T21:49:26.896 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:49:26.894+0000 7f4dd6a00700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4dd010fb90 con 0x7f4dd0104e60 2024-04-12T21:49:26.896 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:49:26.895+0000 7f4dd4c00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4dd0105800 0x7f4dd0116150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:49:26.896 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:49:26.895+0000 7f4dd4c00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4dd0105800 0x7f4dd0116150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:51680/0 (socket says 172.21.15.18:51680) 2024-04-12T21:49:26.896 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:49:26.895+0000 7f4dd4c00700 1 -- 172.21.15.18:0/2048833669 learned_addr learned my addr 172.21.15.18:0/2048833669 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:49:26.896 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:49:26.895+0000 7f4dd5600700 1 --2- 172.21.15.18:0/2048833669 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4dd0104e60 0x7f4dd0115c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:49:26.897 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:49:26.895+0000 7f4dd4c00700 1 -- 172.21.15.18:0/2048833669 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4dd0104e60 msgr2=0x7f4dd0115c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:49:26.897 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:49:26.895+0000 7f4dd4c00700 1 --2- 172.21.15.18:0/2048833669 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4dd0104e60 0x7f4dd0115c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:49:26.897 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:49:26.895+0000 7f4dd4c00700 1 -- 172.21.15.18:0/2048833669 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4dbc005040 con 0x7f4dd0105800 2024-04-12T21:49:26.897 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:49:26.896+0000 7f4dd4c00700 1 --2- 172.21.15.18:0/2048833669 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4dd0105800 0x7f4dd0116150 secure :-1 s=READY pgs=1259 cs=0 l=1 rev1=1 crypto rx=0x7f4dbc009100 tx=0x7f4dbc009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:49:26.898 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:49:26.896+0000 7f4dcd600700 1 -- 172.21.15.18:0/2048833669 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4dbc00e040 con 0x7f4dd0105800 2024-04-12T21:49:26.898 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:49:26.896+0000 7f4dcd600700 1 -- 172.21.15.18:0/2048833669 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f4dbc013420 con 0x7f4dd0105800 2024-04-12T21:49:26.898 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:49:26.896+0000 7f4dcd600700 1 -- 172.21.15.18:0/2048833669 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4dbc01c630 con 0x7f4dd0105800 2024-04-12T21:49:26.898 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:49:26.896+0000 7f4dd6a00700 1 -- 172.21.15.18:0/2048833669 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4dd010fe10 con 0x7f4dd0105800 2024-04-12T21:49:26.898 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:49:26.896+0000 7f4dd6a00700 1 -- 172.21.15.18:0/2048833669 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4dd01103d0 con 0x7f4dd0105800 2024-04-12T21:49:26.898 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:49:26.897+0000 7f4dd6a00700 1 -- 172.21.15.18:0/2048833669 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4dd01005b0 con 0x7f4dd0105800 2024-04-12T21:49:26.899 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:49:26.898+0000 7f4dcd600700 1 -- 172.21.15.18:0/2048833669 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7f4dbc013590 con 0x7f4dd0105800 2024-04-12T21:49:26.901 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:49:26.901+0000 7f4dcd600700 1 --2- 172.21.15.18:0/2048833669 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f4db806ec00 0x7f4db80710b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:49:26.902 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:49:26.901+0000 7f4dcd600700 1 -- 172.21.15.18:0/2048833669 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 6757+0+0 (secure 0 0 0) 0x7f4dbc08fba0 con 0x7f4dd0105800 2024-04-12T21:49:26.902 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:49:26.901+0000 7f4dd5600700 1 --2- 172.21.15.18:0/2048833669 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f4db806ec00 0x7f4db80710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:49:26.902 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:49:26.902+0000 7f4dd5600700 1 --2- 172.21.15.18:0/2048833669 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f4db806ec00 0x7f4db80710b0 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7f4dc4009fd0 tx=0x7f4dc4009f20 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:49:26.906 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:49:26.905+0000 7f4dcd600700 1 -- 172.21.15.18:0/2048833669 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f4dbc05c3d0 con 0x7f4dd0105800 2024-04-12T21:49:27.148 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:49:27.147+0000 7f4dd6a00700 1 -- 172.21.15.18:0/2048833669 --> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f4dd010bf60 con 0x7f4db806ec00 2024-04-12T21:49:27.159 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:49:27.158+0000 7f4dcd600700 1 -- 172.21.15.18:0/2048833669 <== mgr.34102 v2:172.21.15.18:6800/3812891919 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7f4dd010bf60 con 0x7f4db806ec00 2024-04-12T21:49:27.160 INFO:teuthology.orchestra.run.smithi018.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-12T21:49:27.160 INFO:teuthology.orchestra.run.smithi018.stdout:alertmanager.smithi018 smithi018 *:9093,9094 running (50m) 11m ago 64m 14.8M - 0.25.0 c8568f914cd2 c332090068e8 2024-04-12T21:49:27.160 INFO:teuthology.orchestra.run.smithi018.stdout:crash.smithi018 smithi018 running (46m) 11m ago 64m 6665k - 19.0.0-2879-g324baf8d 8717720dc5e5 b7f80d1f7982 2024-04-12T21:49:27.161 INFO:teuthology.orchestra.run.smithi018.stdout:crash.smithi186 smithi186 running (46m) 11m ago 63m 6660k - 19.0.0-2879-g324baf8d 8717720dc5e5 a4f55c81b623 2024-04-12T21:49:27.161 INFO:teuthology.orchestra.run.smithi018.stdout:grafana.smithi018 smithi018 *:3000 running (49m) 11m ago 64m 73.8M - 9.4.12 2bacad6d85d8 b08e3bf176dd 2024-04-12T21:49:27.161 INFO:teuthology.orchestra.run.smithi018.stdout:mds.cephfs.smithi018.acqbsc smithi018 running (42m) 11m ago 57m 21.9M - 19.0.0-2879-g324baf8d 8717720dc5e5 021f99ab24ae 2024-04-12T21:49:27.161 INFO:teuthology.orchestra.run.smithi018.stdout:mds.cephfs.smithi018.szjluo smithi018 running (41m) 11m ago 57m 21.7M - 19.0.0-2879-g324baf8d 8717720dc5e5 0003117479d4 2024-04-12T21:49:27.161 INFO:teuthology.orchestra.run.smithi018.stdout:mds.cephfs.smithi186.jdllyn smithi186 running (26m) 11m ago 57m 16.5M - 19.0.0-2879-g324baf8d 8717720dc5e5 21b1f46eef23 2024-04-12T21:49:27.162 INFO:teuthology.orchestra.run.smithi018.stdout:mds.cephfs.smithi186.wjnafs smithi186 running (11m) 11m ago 57m 11.0M - 19.0.0-2879-g324baf8d 8717720dc5e5 3cf1ef4148e2 2024-04-12T21:49:27.162 INFO:teuthology.orchestra.run.smithi018.stdout:mgr.smithi018.xjzwga smithi018 *:8443,9283,8765 running (52m) 11m ago 66m 517M - 19.0.0-2879-g324baf8d 8717720dc5e5 56088305eab7 2024-04-12T21:49:27.162 INFO:teuthology.orchestra.run.smithi018.stdout:mgr.smithi186.kqcfxk smithi186 *:8443,9283,8765 running (51m) 11m ago 63m 460M - 19.0.0-2879-g324baf8d 8717720dc5e5 09cd766e5e0d 2024-04-12T21:49:27.162 INFO:teuthology.orchestra.run.smithi018.stdout:mon.smithi018 smithi018 running (47m) 11m ago 66m 88.2M 2048M 19.0.0-2879-g324baf8d 8717720dc5e5 df3e201dfed6 2024-04-12T21:49:27.162 INFO:teuthology.orchestra.run.smithi018.stdout:mon.smithi186 smithi186 running (47m) 11m ago 63m 53.6M 2048M 19.0.0-2879-g324baf8d 8717720dc5e5 c77498da9b2b 2024-04-12T21:49:27.162 INFO:teuthology.orchestra.run.smithi018.stdout:node-exporter.smithi018 smithi018 *:9100 running (51m) 11m ago 64m 8468k - 1.5.0 0da6a335fe13 326c3d9c6847 2024-04-12T21:49:27.162 INFO:teuthology.orchestra.run.smithi018.stdout:node-exporter.smithi186 smithi186 *:9100 running (50m) 11m ago 63m 8166k - 1.5.0 0da6a335fe13 ec2112baeecf 2024-04-12T21:49:27.163 INFO:teuthology.orchestra.run.smithi018.stdout:osd.0 smithi018 running (45m) 11m ago 62m 123M 1559M 19.0.0-2879-g324baf8d 8717720dc5e5 60b1273ca047 2024-04-12T21:49:27.163 INFO:teuthology.orchestra.run.smithi018.stdout:osd.1 smithi018 running (45m) 11m ago 61m 95.4M 1559M 19.0.0-2879-g324baf8d 8717720dc5e5 170ce1e5fb8d 2024-04-12T21:49:27.163 INFO:teuthology.orchestra.run.smithi018.stdout:osd.2 smithi018 running (44m) 11m ago 60m 94.6M 1559M 19.0.0-2879-g324baf8d 8717720dc5e5 c2cbc6aa2839 2024-04-12T21:49:27.163 INFO:teuthology.orchestra.run.smithi018.stdout:osd.3 smithi186 running (44m) 11m ago 60m 101M 2581M 19.0.0-2879-g324baf8d 8717720dc5e5 0138e8262e4b 2024-04-12T21:49:27.163 INFO:teuthology.orchestra.run.smithi018.stdout:osd.4 smithi186 running (43m) 11m ago 59m 91.5M 2581M 19.0.0-2879-g324baf8d 8717720dc5e5 b20cb63b25c9 2024-04-12T21:49:27.163 INFO:teuthology.orchestra.run.smithi018.stdout:osd.5 smithi186 running (42m) 11m ago 59m 92.2M 2581M 19.0.0-2879-g324baf8d 8717720dc5e5 2513bea2610e 2024-04-12T21:49:27.163 INFO:teuthology.orchestra.run.smithi018.stdout:prometheus.smithi018 smithi018 *:9095 running (50m) 11m ago 63m 69.6M - 2.43.0 a07b618ecd1d f58841127e21 2024-04-12T21:49:27.164 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:49:27.161+0000 7f4dd6a00700 1 -- 172.21.15.18:0/2048833669 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f4db806ec00 msgr2=0x7f4db80710b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:49:27.164 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:49:27.161+0000 7f4dd6a00700 1 --2- 172.21.15.18:0/2048833669 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f4db806ec00 0x7f4db80710b0 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7f4dc4009fd0 tx=0x7f4dc4009f20 comp rx=0 tx=0).stop 2024-04-12T21:49:27.164 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:49:27.161+0000 7f4dd6a00700 1 -- 172.21.15.18:0/2048833669 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4dd0105800 msgr2=0x7f4dd0116150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:49:27.164 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:49:27.161+0000 7f4dd6a00700 1 --2- 172.21.15.18:0/2048833669 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4dd0105800 0x7f4dd0116150 secure :-1 s=READY pgs=1259 cs=0 l=1 rev1=1 crypto rx=0x7f4dbc009100 tx=0x7f4dbc009e30 comp rx=0 tx=0).stop 2024-04-12T21:49:27.164 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:49:27.161+0000 7f4dd6a00700 1 -- 172.21.15.18:0/2048833669 shutdown_connections 2024-04-12T21:49:27.164 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:49:27.161+0000 7f4dd6a00700 1 --2- 172.21.15.18:0/2048833669 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f4db806ec00 0x7f4db80710b0 unknown :-1 s=CLOSED pgs=668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:49:27.164 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:49:27.161+0000 7f4dd6a00700 1 --2- 172.21.15.18:0/2048833669 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4dd0104e60 0x7f4dd0115c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:49:27.164 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:49:27.161+0000 7f4dd6a00700 1 --2- 172.21.15.18:0/2048833669 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4dd0105800 0x7f4dd0116150 unknown :-1 s=CLOSED pgs=1259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:49:27.165 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:49:27.161+0000 7f4dd6a00700 1 -- 172.21.15.18:0/2048833669 >> 172.21.15.18:0/2048833669 conn(0x7f4dd00fab10 msgr2=0x7f4dd00fcab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:49:27.165 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:49:27.161+0000 7f4dd6a00700 1 -- 172.21.15.18:0/2048833669 shutdown_connections 2024-04-12T21:49:27.165 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:49:27.161+0000 7f4dd6a00700 1 -- 172.21.15.18:0/2048833669 wait complete. 2024-04-12T21:49:27.305 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:49:27.304+0000 7ff9f8600700 1 -- 172.21.15.18:0/3636828712 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff9f0102de0 msgr2=0x7ff9f01031b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:49:27.305 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:49:27.304+0000 7ff9f8600700 1 --2- 172.21.15.18:0/3636828712 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff9f0102de0 0x7ff9f01031b0 secure :-1 s=READY pgs=1260 cs=0 l=1 rev1=1 crypto rx=0x7ff9e4004940 tx=0x7ff9e4009e30 comp rx=0 tx=0).stop 2024-04-12T21:49:27.305 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:49:27.304+0000 7ff9f8600700 1 -- 172.21.15.18:0/3636828712 shutdown_connections 2024-04-12T21:49:27.305 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:49:27.304+0000 7ff9f8600700 1 --2- 172.21.15.18:0/3636828712 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff9f0103780 0x7ff9f00fcdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:49:27.305 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:49:27.304+0000 7ff9f8600700 1 --2- 172.21.15.18:0/3636828712 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff9f0102de0 0x7ff9f01031b0 unknown :-1 s=CLOSED pgs=1260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:49:27.305 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:49:27.304+0000 7ff9f8600700 1 -- 172.21.15.18:0/3636828712 >> 172.21.15.18:0/3636828712 conn(0x7ff9f00f8970 msgr2=0x7ff9f00fad80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:49:27.305 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:49:27.304+0000 7ff9f8600700 1 -- 172.21.15.18:0/3636828712 shutdown_connections 2024-04-12T21:49:27.306 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:49:27.304+0000 7ff9f8600700 1 -- 172.21.15.18:0/3636828712 wait complete. 2024-04-12T21:49:27.307 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:49:27.305+0000 7ff9f8600700 1 Processor -- start 2024-04-12T21:49:27.307 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:49:27.305+0000 7ff9f8600700 1 -- start start 2024-04-12T21:49:27.307 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:49:27.306+0000 7ff9f8600700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff9f0102de0 0x7ff9f0115d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:49:27.308 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:49:27.306+0000 7ff9f8600700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff9f0103780 0x7ff9f01162b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:49:27.308 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:49:27.306+0000 7ff9f8600700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9f0116900 con 0x7ff9f0102de0 2024-04-12T21:49:27.308 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:49:27.306+0000 7ff9f8600700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9f010fd20 con 0x7ff9f0103780 2024-04-12T21:49:27.308 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:49:27.306+0000 7ff9f5c00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff9f0102de0 0x7ff9f0115d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:49:27.308 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:49:27.306+0000 7ff9f5c00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff9f0102de0 0x7ff9f0115d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:51704/0 (socket says 172.21.15.18:51704) 2024-04-12T21:49:27.308 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:49:27.306+0000 7ff9f5c00700 1 -- 172.21.15.18:0/403076369 learned_addr learned my addr 172.21.15.18:0/403076369 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:49:27.308 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:49:27.306+0000 7ff9f5200700 1 --2- 172.21.15.18:0/403076369 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff9f0103780 0x7ff9f01162b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:49:27.308 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:49:27.306+0000 7ff9f5c00700 1 -- 172.21.15.18:0/403076369 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff9f0103780 msgr2=0x7ff9f01162b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:49:27.309 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:49:27.306+0000 7ff9f5c00700 1 --2- 172.21.15.18:0/403076369 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff9f0103780 0x7ff9f01162b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:49:27.309 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:49:27.306+0000 7ff9f5c00700 1 -- 172.21.15.18:0/403076369 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9e4005040 con 0x7ff9f0102de0 2024-04-12T21:49:27.309 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:49:27.306+0000 7ff9f5200700 1 --2- 172.21.15.18:0/403076369 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff9f0103780 0x7ff9f01162b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-12T21:49:27.309 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:49:27.307+0000 7ff9f5c00700 1 --2- 172.21.15.18:0/403076369 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff9f0102de0 0x7ff9f0115d70 secure :-1 s=READY pgs=1261 cs=0 l=1 rev1=1 crypto rx=0x7ff9e400b8b0 tx=0x7ff9e4009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:49:27.310 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:49:27.307+0000 7ff9ed600700 1 -- 172.21.15.18:0/403076369 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff9e400e040 con 0x7ff9f0102de0 2024-04-12T21:49:27.310 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:49:27.307+0000 7ff9ed600700 1 -- 172.21.15.18:0/403076369 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff9e4004290 con 0x7ff9f0102de0 2024-04-12T21:49:27.310 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:49:27.307+0000 7ff9ed600700 1 -- 172.21.15.18:0/403076369 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff9e401c3f0 con 0x7ff9f0102de0 2024-04-12T21:49:27.310 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:49:27.307+0000 7ff9f8600700 1 -- 172.21.15.18:0/403076369 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff9f010ffa0 con 0x7ff9f0102de0 2024-04-12T21:49:27.310 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:49:27.307+0000 7ff9f8600700 1 -- 172.21.15.18:0/403076369 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff9f0110560 con 0x7ff9f0102de0 2024-04-12T21:49:27.310 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:49:27.308+0000 7ff9f8600700 1 -- 172.21.15.18:0/403076369 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff9f0106a80 con 0x7ff9f0102de0 2024-04-12T21:49:27.311 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:49:27.310+0000 7ff9ed600700 1 -- 172.21.15.18:0/403076369 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7ff9e40137b0 con 0x7ff9f0102de0 2024-04-12T21:49:27.311 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:49:27.310+0000 7ff9ed600700 1 --2- 172.21.15.18:0/403076369 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7ff9d806ec00 0x7ff9d80710b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:49:27.311 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:49:27.310+0000 7ff9ed600700 1 -- 172.21.15.18:0/403076369 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 6757+0+0 (secure 0 0 0) 0x7ff9e408fca0 con 0x7ff9f0102de0 2024-04-12T21:49:27.311 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:49:27.311+0000 7ff9f5200700 1 --2- 172.21.15.18:0/403076369 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7ff9d806ec00 0x7ff9d80710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:49:27.312 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:49:27.311+0000 7ff9f5200700 1 --2- 172.21.15.18:0/403076369 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7ff9d806ec00 0x7ff9d80710b0 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7ff9dc009fd0 tx=0x7ff9dc00b040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:49:27.316 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:49:27.315+0000 7ff9ed600700 1 -- 172.21.15.18:0/403076369 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7ff9e405c4d0 con 0x7ff9f0102de0 2024-04-12T21:49:27.633 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:49:27.631+0000 7ff9f8600700 1 -- 172.21.15.18:0/403076369 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7ff9f0109f70 con 0x7ff9f0102de0 2024-04-12T21:49:27.633 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:49:27.632+0000 7ff9ed600700 1 -- 172.21.15.18:0/403076369 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+623 (secure 0 0 0) 0x7ff9e405c060 con 0x7ff9f0102de0 2024-04-12T21:49:27.633 INFO:teuthology.orchestra.run.smithi018.stdout:{ 2024-04-12T21:49:27.634 INFO:teuthology.orchestra.run.smithi018.stdout: "mon": { 2024-04-12T21:49:27.634 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 19.0.0-2879-g324baf8d (324baf8ddb91dea46cac50cb535c306115791355) squid (dev)": 2 2024-04-12T21:49:27.634 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-12T21:49:27.634 INFO:teuthology.orchestra.run.smithi018.stdout: "mgr": { 2024-04-12T21:49:27.634 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 19.0.0-2879-g324baf8d (324baf8ddb91dea46cac50cb535c306115791355) squid (dev)": 2 2024-04-12T21:49:27.634 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-12T21:49:27.634 INFO:teuthology.orchestra.run.smithi018.stdout: "osd": { 2024-04-12T21:49:27.634 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 19.0.0-2879-g324baf8d (324baf8ddb91dea46cac50cb535c306115791355) squid (dev)": 6 2024-04-12T21:49:27.634 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-12T21:49:27.634 INFO:teuthology.orchestra.run.smithi018.stdout: "mds": { 2024-04-12T21:49:27.634 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 19.0.0-2879-g324baf8d (324baf8ddb91dea46cac50cb535c306115791355) squid (dev)": 4 2024-04-12T21:49:27.634 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-12T21:49:27.635 INFO:teuthology.orchestra.run.smithi018.stdout: "overall": { 2024-04-12T21:49:27.635 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 19.0.0-2879-g324baf8d (324baf8ddb91dea46cac50cb535c306115791355) squid (dev)": 14 2024-04-12T21:49:27.635 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-04-12T21:49:27.635 INFO:teuthology.orchestra.run.smithi018.stdout:} 2024-04-12T21:49:27.636 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:49:27.635+0000 7ff9f8600700 1 -- 172.21.15.18:0/403076369 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7ff9d806ec00 msgr2=0x7ff9d80710b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:49:27.636 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:49:27.635+0000 7ff9f8600700 1 --2- 172.21.15.18:0/403076369 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7ff9d806ec00 0x7ff9d80710b0 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7ff9dc009fd0 tx=0x7ff9dc00b040 comp rx=0 tx=0).stop 2024-04-12T21:49:27.636 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:49:27.635+0000 7ff9f8600700 1 -- 172.21.15.18:0/403076369 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff9f0102de0 msgr2=0x7ff9f0115d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:49:27.636 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:49:27.635+0000 7ff9f8600700 1 --2- 172.21.15.18:0/403076369 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff9f0102de0 0x7ff9f0115d70 secure :-1 s=READY pgs=1261 cs=0 l=1 rev1=1 crypto rx=0x7ff9e400b8b0 tx=0x7ff9e4009e30 comp rx=0 tx=0).stop 2024-04-12T21:49:27.637 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:49:27.636+0000 7ff9f8600700 1 -- 172.21.15.18:0/403076369 shutdown_connections 2024-04-12T21:49:27.637 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:49:27.636+0000 7ff9f8600700 1 --2- 172.21.15.18:0/403076369 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7ff9d806ec00 0x7ff9d80710b0 unknown :-1 s=CLOSED pgs=669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:49:27.637 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:49:27.636+0000 7ff9f8600700 1 --2- 172.21.15.18:0/403076369 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff9f0102de0 0x7ff9f0115d70 unknown :-1 s=CLOSED pgs=1261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:49:27.637 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:49:27.636+0000 7ff9f8600700 1 --2- 172.21.15.18:0/403076369 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff9f0103780 0x7ff9f01162b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:49:27.637 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:49:27.636+0000 7ff9f8600700 1 -- 172.21.15.18:0/403076369 >> 172.21.15.18:0/403076369 conn(0x7ff9f00f8970 msgr2=0x7ff9f01073d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:49:27.637 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:49:27.636+0000 7ff9f8600700 1 -- 172.21.15.18:0/403076369 shutdown_connections 2024-04-12T21:49:27.637 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:49:27.636+0000 7ff9f8600700 1 -- 172.21.15.18:0/403076369 wait complete. 2024-04-12T21:49:27.790 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:49:27.789+0000 7f9007c00700 1 -- 172.21.15.18:0/2282036745 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9000105020 msgr2=0x7f90001053f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:49:27.791 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:49:27.789+0000 7f9007c00700 1 --2- 172.21.15.18:0/2282036745 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9000105020 0x7f90001053f0 secure :-1 s=READY pgs=1262 cs=0 l=1 rev1=1 crypto rx=0x7f8ff0004940 tx=0x7f8ff0009e30 comp rx=0 tx=0).stop 2024-04-12T21:49:27.791 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:49:27.789+0000 7f9007c00700 1 -- 172.21.15.18:0/2282036745 shutdown_connections 2024-04-12T21:49:27.791 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:49:27.789+0000 7f9007c00700 1 --2- 172.21.15.18:0/2282036745 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f90001059c0 0x7f90000ff080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:49:27.791 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:49:27.789+0000 7f9007c00700 1 --2- 172.21.15.18:0/2282036745 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9000105020 0x7f90001053f0 unknown :-1 s=CLOSED pgs=1262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:49:27.791 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:49:27.789+0000 7f9007c00700 1 -- 172.21.15.18:0/2282036745 >> 172.21.15.18:0/2282036745 conn(0x7f90000faba0 msgr2=0x7f90000fcfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:49:27.791 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:49:27.789+0000 7f9007c00700 1 -- 172.21.15.18:0/2282036745 shutdown_connections 2024-04-12T21:49:27.791 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:49:27.789+0000 7f9007c00700 1 -- 172.21.15.18:0/2282036745 wait complete. 2024-04-12T21:49:27.791 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:49:27.790+0000 7f9007c00700 1 Processor -- start 2024-04-12T21:49:27.791 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:49:27.790+0000 7f9007c00700 1 -- start start 2024-04-12T21:49:27.793 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:49:27.790+0000 7f9007c00700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9000105020 0x7f9000115b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:49:27.793 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:49:27.790+0000 7f9007c00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f90001059c0 0x7f90001160d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:49:27.793 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:49:27.790+0000 7f9007c00700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9000116720 con 0x7f90001059c0 2024-04-12T21:49:27.793 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:49:27.790+0000 7f9007c00700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f900010fd00 con 0x7f9000105020 2024-04-12T21:49:27.794 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:49:27.791+0000 7f8ff7e00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f90001059c0 0x7f90001160d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:49:27.794 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:49:27.791+0000 7f8ff7e00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f90001059c0 0x7f90001160d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:51736/0 (socket says 172.21.15.18:51736) 2024-04-12T21:49:27.794 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:49:27.791+0000 7f8ff7e00700 1 -- 172.21.15.18:0/833557317 learned_addr learned my addr 172.21.15.18:0/833557317 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:49:27.794 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:49:27.791+0000 7f9005200700 1 --2- 172.21.15.18:0/833557317 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9000105020 0x7f9000115b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:49:27.794 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:49:27.791+0000 7f8ff7e00700 1 -- 172.21.15.18:0/833557317 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9000105020 msgr2=0x7f9000115b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:49:27.794 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:49:27.791+0000 7f8ff7e00700 1 --2- 172.21.15.18:0/833557317 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9000105020 0x7f9000115b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:49:27.794 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:49:27.791+0000 7f8ff7e00700 1 -- 172.21.15.18:0/833557317 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ff0005040 con 0x7f90001059c0 2024-04-12T21:49:27.794 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:49:27.791+0000 7f8ff7e00700 1 --2- 172.21.15.18:0/833557317 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f90001059c0 0x7f90001160d0 secure :-1 s=READY pgs=1263 cs=0 l=1 rev1=1 crypto rx=0x7f8fe800c800 tx=0x7f8fe8004470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:49:27.794 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:49:27.792+0000 7f8ffd600700 1 -- 172.21.15.18:0/833557317 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8fe800a760 con 0x7f90001059c0 2024-04-12T21:49:27.794 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:49:27.792+0000 7f8ffd600700 1 -- 172.21.15.18:0/833557317 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f8fe800ad90 con 0x7f90001059c0 2024-04-12T21:49:27.794 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:49:27.792+0000 7f8ffd600700 1 -- 172.21.15.18:0/833557317 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8fe80113a0 con 0x7f90001059c0 2024-04-12T21:49:27.794 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:49:27.792+0000 7f9007c00700 1 -- 172.21.15.18:0/833557317 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f900010ffe0 con 0x7f90001059c0 2024-04-12T21:49:27.795 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:49:27.792+0000 7f9007c00700 1 -- 172.21.15.18:0/833557317 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f90001105a0 con 0x7f90001059c0 2024-04-12T21:49:27.795 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:49:27.793+0000 7f9007c00700 1 -- 172.21.15.18:0/833557317 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9000100770 con 0x7f90001059c0 2024-04-12T21:49:27.795 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:49:27.794+0000 7f8ffd600700 1 -- 172.21.15.18:0/833557317 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7f8fe8012400 con 0x7f90001059c0 2024-04-12T21:49:27.797 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:49:27.796+0000 7f8ffd600700 1 --2- 172.21.15.18:0/833557317 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f8fe406ec00 0x7f8fe40710b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:49:27.797 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:49:27.796+0000 7f8ffd600700 1 -- 172.21.15.18:0/833557317 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 6757+0+0 (secure 0 0 0) 0x7f8fe808e960 con 0x7f90001059c0 2024-04-12T21:49:27.797 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:49:27.796+0000 7f9005200700 1 --2- 172.21.15.18:0/833557317 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f8fe406ec00 0x7f8fe40710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:49:27.797 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:49:27.797+0000 7f9005200700 1 --2- 172.21.15.18:0/833557317 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f8fe406ec00 0x7f8fe40710b0 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f8ff0004f90 tx=0x7f8ff0009e30 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:49:27.801 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:49:27.801+0000 7f8ffd600700 1 -- 172.21.15.18:0/833557317 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f8fe805b190 con 0x7f90001059c0 2024-04-12T21:49:28.034 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:49:27 smithi186 ceph-mon[86379]: from='client.36580 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:49:28.035 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:49:27 smithi186 ceph-mon[86379]: pgmap v1493: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:49:28.035 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:49:27 smithi186 ceph-mon[86379]: from='client.36584 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:49:28.035 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:49:27 smithi186 ceph-mon[86379]: from='client.? 172.21.15.18:0/403076369' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T21:49:28.071 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:49:28.070+0000 7f9007c00700 1 -- 172.21.15.18:0/833557317 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f900010c120 con 0x7f90001059c0 2024-04-12T21:49:28.072 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:49:27 smithi018 ceph-mon[112590]: from='client.36580 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:49:28.072 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:49:27 smithi018 ceph-mon[112590]: pgmap v1493: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:49:28.072 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:49:27 smithi018 ceph-mon[112590]: from='client.36584 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:49:28.072 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:49:27 smithi018 ceph-mon[112590]: from='client.? 172.21.15.18:0/403076369' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T21:49:28.072 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:49:28.071+0000 7f8ffd600700 1 -- 172.21.15.18:0/833557317 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 33 v33) v1 ==== 76+0+1876 (secure 0 0 0) 0x7f8fe805ad20 con 0x7f90001059c0 2024-04-12T21:49:28.074 INFO:teuthology.orchestra.run.smithi018.stdout:e33 2024-04-12T21:49:28.074 INFO:teuthology.orchestra.run.smithi018.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-12T21:49:28.074 INFO:teuthology.orchestra.run.smithi018.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-12T21:49:28.074 INFO:teuthology.orchestra.run.smithi018.stdout:legacy client fscid: 1 2024-04-12T21:49:28.074 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-12T21:49:28.074 INFO:teuthology.orchestra.run.smithi018.stdout:Filesystem 'cephfs' (1) 2024-04-12T21:49:28.074 INFO:teuthology.orchestra.run.smithi018.stdout:fs_name cephfs 2024-04-12T21:49:28.074 INFO:teuthology.orchestra.run.smithi018.stdout:epoch 24 2024-04-12T21:49:28.074 INFO:teuthology.orchestra.run.smithi018.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-04-12T21:49:28.074 INFO:teuthology.orchestra.run.smithi018.stdout:created 2024-04-12T20:51:57.362850+0000 2024-04-12T21:49:28.074 INFO:teuthology.orchestra.run.smithi018.stdout:modified 2024-04-12T21:06:59.717190+0000 2024-04-12T21:49:28.074 INFO:teuthology.orchestra.run.smithi018.stdout:tableserver 0 2024-04-12T21:49:28.075 INFO:teuthology.orchestra.run.smithi018.stdout:root 0 2024-04-12T21:49:28.075 INFO:teuthology.orchestra.run.smithi018.stdout:session_timeout 60 2024-04-12T21:49:28.075 INFO:teuthology.orchestra.run.smithi018.stdout:session_autoclose 300 2024-04-12T21:49:28.075 INFO:teuthology.orchestra.run.smithi018.stdout:max_file_size 1099511627776 2024-04-12T21:49:28.075 INFO:teuthology.orchestra.run.smithi018.stdout:max_xattr_size 65536 2024-04-12T21:49:28.075 INFO:teuthology.orchestra.run.smithi018.stdout:required_client_features {} 2024-04-12T21:49:28.075 INFO:teuthology.orchestra.run.smithi018.stdout:last_failure 0 2024-04-12T21:49:28.075 INFO:teuthology.orchestra.run.smithi018.stdout:last_failure_osd_epoch 100 2024-04-12T21:49:28.075 INFO:teuthology.orchestra.run.smithi018.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-12T21:49:28.075 INFO:teuthology.orchestra.run.smithi018.stdout:max_mds 2 2024-04-12T21:49:28.075 INFO:teuthology.orchestra.run.smithi018.stdout:in 0,1 2024-04-12T21:49:28.075 INFO:teuthology.orchestra.run.smithi018.stdout:up {} 2024-04-12T21:49:28.075 INFO:teuthology.orchestra.run.smithi018.stdout:failed 0,1 2024-04-12T21:49:28.075 INFO:teuthology.orchestra.run.smithi018.stdout:damaged 2024-04-12T21:49:28.076 INFO:teuthology.orchestra.run.smithi018.stdout:stopped 2024-04-12T21:49:28.076 INFO:teuthology.orchestra.run.smithi018.stdout:data_pools [3] 2024-04-12T21:49:28.076 INFO:teuthology.orchestra.run.smithi018.stdout:metadata_pool 2 2024-04-12T21:49:28.076 INFO:teuthology.orchestra.run.smithi018.stdout:inline_data disabled 2024-04-12T21:49:28.076 INFO:teuthology.orchestra.run.smithi018.stdout:balancer 2024-04-12T21:49:28.076 INFO:teuthology.orchestra.run.smithi018.stdout:bal_rank_mask -1 2024-04-12T21:49:28.076 INFO:teuthology.orchestra.run.smithi018.stdout:standby_count_wanted 1 2024-04-12T21:49:28.076 INFO:teuthology.orchestra.run.smithi018.stdout:qdb_cluster leader: 0 members: 2024-04-12T21:49:28.076 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-12T21:49:28.076 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-12T21:49:28.076 INFO:teuthology.orchestra.run.smithi018.stdout:Standby daemons: 2024-04-12T21:49:28.076 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-12T21:49:28.076 INFO:teuthology.orchestra.run.smithi018.stdout:[mds.cephfs.smithi018.acqbsc{-1:34436} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.18:6828/755179764,v1:172.21.15.18:6829/755179764] compat {c=[1],r=[1],i=[1fff]}] 2024-04-12T21:49:28.076 INFO:teuthology.orchestra.run.smithi018.stdout:[mds.cephfs.smithi018.szjluo{-1:34468} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.18:6826/2407992653,v1:172.21.15.18:6827/2407992653] compat {c=[1],r=[1],i=[1fff]}] 2024-04-12T21:49:28.076 INFO:teuthology.orchestra.run.smithi018.stdout:[mds.cephfs.smithi186.jdllyn{-1:44595} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.186:6824/3523444202,v1:172.21.15.186:6825/3523444202] compat {c=[1],r=[1],i=[1fff]}] 2024-04-12T21:49:28.077 INFO:teuthology.orchestra.run.smithi018.stdout:[mds.cephfs.smithi186.wjnafs{-1:44875} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.186:6826/3122443766,v1:172.21.15.186:6827/3122443766] compat {c=[1],r=[1],i=[1fff]}] 2024-04-12T21:49:28.077 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:49:28.074+0000 7f9007c00700 1 -- 172.21.15.18:0/833557317 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f8fe406ec00 msgr2=0x7f8fe40710b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:49:28.077 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:49:28.074+0000 7f9007c00700 1 --2- 172.21.15.18:0/833557317 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f8fe406ec00 0x7f8fe40710b0 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f8ff0004f90 tx=0x7f8ff0009e30 comp rx=0 tx=0).stop 2024-04-12T21:49:28.077 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:49:28.074+0000 7f9007c00700 1 -- 172.21.15.18:0/833557317 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f90001059c0 msgr2=0x7f90001160d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:49:28.078 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:49:28.074+0000 7f9007c00700 1 --2- 172.21.15.18:0/833557317 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f90001059c0 0x7f90001160d0 secure :-1 s=READY pgs=1263 cs=0 l=1 rev1=1 crypto rx=0x7f8fe800c800 tx=0x7f8fe8004470 comp rx=0 tx=0).stop 2024-04-12T21:49:28.078 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:49:28.074+0000 7f9007c00700 1 -- 172.21.15.18:0/833557317 shutdown_connections 2024-04-12T21:49:28.078 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:49:28.074+0000 7f9007c00700 1 --2- 172.21.15.18:0/833557317 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f8fe406ec00 0x7f8fe40710b0 unknown :-1 s=CLOSED pgs=670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:49:28.078 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:49:28.074+0000 7f9007c00700 1 --2- 172.21.15.18:0/833557317 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9000105020 0x7f9000115b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:49:28.078 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:49:28.074+0000 7f9007c00700 1 --2- 172.21.15.18:0/833557317 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f90001059c0 0x7f90001160d0 unknown :-1 s=CLOSED pgs=1263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:49:28.078 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:49:28.074+0000 7f9007c00700 1 -- 172.21.15.18:0/833557317 >> 172.21.15.18:0/833557317 conn(0x7f90000faba0 msgr2=0x7f90000fbb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:49:28.078 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:49:28.075+0000 7f9007c00700 1 -- 172.21.15.18:0/833557317 shutdown_connections 2024-04-12T21:49:28.078 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:49:28.075+0000 7f9007c00700 1 -- 172.21.15.18:0/833557317 wait complete. 2024-04-12T21:49:28.078 INFO:teuthology.orchestra.run.smithi018.stderr:dumped fsmap epoch 33 2024-04-12T21:49:28.221 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:49:28.220+0000 7f4331000700 1 -- 172.21.15.18:0/3151255014 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f432c104e60 msgr2=0x7f432c105230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:49:28.222 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:49:28.220+0000 7f4331000700 1 --2- 172.21.15.18:0/3151255014 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f432c104e60 0x7f432c105230 secure :-1 s=READY pgs=1264 cs=0 l=1 rev1=1 crypto rx=0x7f431c004940 tx=0x7f431c009e30 comp rx=0 tx=0).stop 2024-04-12T21:49:28.222 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:49:28.220+0000 7f4331000700 1 -- 172.21.15.18:0/3151255014 shutdown_connections 2024-04-12T21:49:28.222 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:49:28.220+0000 7f4331000700 1 --2- 172.21.15.18:0/3151255014 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f432c105800 0x7f432c0fee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:49:28.222 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:49:28.221+0000 7f4331000700 1 --2- 172.21.15.18:0/3151255014 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f432c104e60 0x7f432c105230 unknown :-1 s=CLOSED pgs=1264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:49:28.222 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:49:28.221+0000 7f4331000700 1 -- 172.21.15.18:0/3151255014 >> 172.21.15.18:0/3151255014 conn(0x7f432c0fab10 msgr2=0x7f432c0fcef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:49:28.222 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:49:28.221+0000 7f4331000700 1 -- 172.21.15.18:0/3151255014 shutdown_connections 2024-04-12T21:49:28.222 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:49:28.221+0000 7f4331000700 1 -- 172.21.15.18:0/3151255014 wait complete. 2024-04-12T21:49:28.224 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:49:28.221+0000 7f4331000700 1 Processor -- start 2024-04-12T21:49:28.224 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:49:28.222+0000 7f4331000700 1 -- start start 2024-04-12T21:49:28.224 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:49:28.222+0000 7f4331000700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f432c104e60 0x7f432c115bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:49:28.224 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:49:28.222+0000 7f4331000700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f432c105800 0x7f432c1160f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:49:28.224 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:49:28.222+0000 7f4331000700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f432c116740 con 0x7f432c104e60 2024-04-12T21:49:28.224 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:49:28.222+0000 7f4331000700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f432c10fb50 con 0x7f432c105800 2024-04-12T21:49:28.224 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:49:28.222+0000 7f432b400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f432c104e60 0x7f432c115bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:49:28.224 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:49:28.222+0000 7f432b400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f432c104e60 0x7f432c115bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:51748/0 (socket says 172.21.15.18:51748) 2024-04-12T21:49:28.224 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:49:28.222+0000 7f432b400700 1 -- 172.21.15.18:0/3422464948 learned_addr learned my addr 172.21.15.18:0/3422464948 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:49:28.224 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:49:28.223+0000 7f432aa00700 1 --2- 172.21.15.18:0/3422464948 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f432c105800 0x7f432c1160f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:49:28.224 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:49:28.223+0000 7f432b400700 1 -- 172.21.15.18:0/3422464948 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f432c105800 msgr2=0x7f432c1160f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:49:28.224 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:49:28.223+0000 7f432b400700 1 --2- 172.21.15.18:0/3422464948 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f432c105800 0x7f432c1160f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:49:28.224 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:49:28.223+0000 7f432b400700 1 -- 172.21.15.18:0/3422464948 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f431c005040 con 0x7f432c104e60 2024-04-12T21:49:28.225 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:49:28.223+0000 7f432aa00700 1 --2- 172.21.15.18:0/3422464948 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f432c105800 0x7f432c1160f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-12T21:49:28.225 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:49:28.223+0000 7f432b400700 1 --2- 172.21.15.18:0/3422464948 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f432c104e60 0x7f432c115bb0 secure :-1 s=READY pgs=1265 cs=0 l=1 rev1=1 crypto rx=0x7f431c009180 tx=0x7f431c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:49:28.225 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:49:28.223+0000 7f4323400700 1 -- 172.21.15.18:0/3422464948 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f431c00e040 con 0x7f432c104e60 2024-04-12T21:49:28.225 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:49:28.223+0000 7f4323400700 1 -- 172.21.15.18:0/3422464948 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f431c00bbc0 con 0x7f432c104e60 2024-04-12T21:49:28.225 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:49:28.223+0000 7f4323400700 1 -- 172.21.15.18:0/3422464948 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f431c01c630 con 0x7f432c104e60 2024-04-12T21:49:28.225 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:49:28.223+0000 7f4331000700 1 -- 172.21.15.18:0/3422464948 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f432c10fdd0 con 0x7f432c104e60 2024-04-12T21:49:28.226 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:49:28.223+0000 7f4331000700 1 -- 172.21.15.18:0/3422464948 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f432c110390 con 0x7f432c104e60 2024-04-12T21:49:28.226 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:49:28.224+0000 7f4331000700 1 -- 172.21.15.18:0/3422464948 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f432c1005b0 con 0x7f432c104e60 2024-04-12T21:49:28.226 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:49:28.226+0000 7f4323400700 1 -- 172.21.15.18:0/3422464948 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7f431c00bd30 con 0x7f432c104e60 2024-04-12T21:49:28.228 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:49:28.227+0000 7f4323400700 1 --2- 172.21.15.18:0/3422464948 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f431006ec00 0x7f43100710b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:49:28.228 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:49:28.227+0000 7f4323400700 1 -- 172.21.15.18:0/3422464948 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 6757+0+0 (secure 0 0 0) 0x7f431c08fcd0 con 0x7f432c104e60 2024-04-12T21:49:28.228 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:49:28.227+0000 7f432aa00700 1 --2- 172.21.15.18:0/3422464948 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f431006ec00 0x7f43100710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:49:28.228 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:49:28.228+0000 7f432aa00700 1 --2- 172.21.15.18:0/3422464948 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f431006ec00 0x7f43100710b0 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f4318007fd0 tx=0x7f4318009040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:49:28.232 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:49:28.232+0000 7f4323400700 1 -- 172.21.15.18:0/3422464948 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f431c058a90 con 0x7f432c104e60 2024-04-12T21:49:28.477 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:49:28.476+0000 7f4331000700 1 -- 172.21.15.18:0/3422464948 --> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f432c04bf40 con 0x7f431006ec00 2024-04-12T21:49:28.479 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:49:28.478+0000 7f4323400700 1 -- 172.21.15.18:0/3422464948 <== mgr.34102 v2:172.21.15.18:6800/3812891919 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f432c04bf40 con 0x7f431006ec00 2024-04-12T21:49:28.479 INFO:teuthology.orchestra.run.smithi018.stdout:{ 2024-04-12T21:49:28.479 INFO:teuthology.orchestra.run.smithi018.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:f4452def3be4d350f4a8e5feaf7ebb1511be9298fe68478ba7a2d6fbd7d69a63", 2024-04-12T21:49:28.479 INFO:teuthology.orchestra.run.smithi018.stdout: "in_progress": true, 2024-04-12T21:49:28.480 INFO:teuthology.orchestra.run.smithi018.stdout: "which": "Upgrading all daemon types on all hosts", 2024-04-12T21:49:28.480 INFO:teuthology.orchestra.run.smithi018.stdout: "services_complete": [ 2024-04-12T21:49:28.480 INFO:teuthology.orchestra.run.smithi018.stdout: "mgr", 2024-04-12T21:49:28.480 INFO:teuthology.orchestra.run.smithi018.stdout: "mds", 2024-04-12T21:49:28.480 INFO:teuthology.orchestra.run.smithi018.stdout: "crash", 2024-04-12T21:49:28.480 INFO:teuthology.orchestra.run.smithi018.stdout: "osd", 2024-04-12T21:49:28.480 INFO:teuthology.orchestra.run.smithi018.stdout: "mon" 2024-04-12T21:49:28.480 INFO:teuthology.orchestra.run.smithi018.stdout: ], 2024-04-12T21:49:28.480 INFO:teuthology.orchestra.run.smithi018.stdout: "progress": "16/21 daemons upgraded", 2024-04-12T21:49:28.480 INFO:teuthology.orchestra.run.smithi018.stdout: "message": "Currently upgrading mds daemons", 2024-04-12T21:49:28.480 INFO:teuthology.orchestra.run.smithi018.stdout: "is_paused": false 2024-04-12T21:49:28.480 INFO:teuthology.orchestra.run.smithi018.stdout:} 2024-04-12T21:49:28.481 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:49:28.480+0000 7f4331000700 1 -- 172.21.15.18:0/3422464948 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f431006ec00 msgr2=0x7f43100710b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:49:28.481 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:49:28.480+0000 7f4331000700 1 --2- 172.21.15.18:0/3422464948 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f431006ec00 0x7f43100710b0 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f4318007fd0 tx=0x7f4318009040 comp rx=0 tx=0).stop 2024-04-12T21:49:28.482 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:49:28.480+0000 7f4331000700 1 -- 172.21.15.18:0/3422464948 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f432c104e60 msgr2=0x7f432c115bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:49:28.482 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:49:28.480+0000 7f4331000700 1 --2- 172.21.15.18:0/3422464948 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f432c104e60 0x7f432c115bb0 secure :-1 s=READY pgs=1265 cs=0 l=1 rev1=1 crypto rx=0x7f431c009180 tx=0x7f431c009e30 comp rx=0 tx=0).stop 2024-04-12T21:49:28.482 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:49:28.481+0000 7f4331000700 1 -- 172.21.15.18:0/3422464948 shutdown_connections 2024-04-12T21:49:28.482 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:49:28.481+0000 7f4331000700 1 --2- 172.21.15.18:0/3422464948 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f431006ec00 0x7f43100710b0 unknown :-1 s=CLOSED pgs=671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:49:28.482 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:49:28.481+0000 7f4331000700 1 --2- 172.21.15.18:0/3422464948 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f432c104e60 0x7f432c115bb0 unknown :-1 s=CLOSED pgs=1265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:49:28.482 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:49:28.481+0000 7f4331000700 1 --2- 172.21.15.18:0/3422464948 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f432c105800 0x7f432c1160f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:49:28.482 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:49:28.481+0000 7f4331000700 1 -- 172.21.15.18:0/3422464948 >> 172.21.15.18:0/3422464948 conn(0x7f432c0fab10 msgr2=0x7f432c0fca40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:49:28.483 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:49:28.481+0000 7f4331000700 1 -- 172.21.15.18:0/3422464948 shutdown_connections 2024-04-12T21:49:28.483 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:49:28.481+0000 7f4331000700 1 -- 172.21.15.18:0/3422464948 wait complete. 2024-04-12T21:49:28.641 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:49:28.640+0000 7fd270600700 1 -- 172.21.15.18:0/3803411071 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd2681059d0 msgr2=0x7fd2680ff040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:49:28.641 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:49:28.640+0000 7fd270600700 1 --2- 172.21.15.18:0/3803411071 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd2681059d0 0x7fd2680ff040 secure :-1 s=READY pgs=1266 cs=0 l=1 rev1=1 crypto rx=0x7fd254004940 tx=0x7fd254009e30 comp rx=0 tx=0).stop 2024-04-12T21:49:28.641 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:49:28.640+0000 7fd270600700 1 -- 172.21.15.18:0/3803411071 shutdown_connections 2024-04-12T21:49:28.642 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:49:28.640+0000 7fd270600700 1 --2- 172.21.15.18:0/3803411071 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd2681059d0 0x7fd2680ff040 unknown :-1 s=CLOSED pgs=1266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:49:28.642 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:49:28.640+0000 7fd270600700 1 --2- 172.21.15.18:0/3803411071 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd268105030 0x7fd268105400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:49:28.642 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:49:28.640+0000 7fd270600700 1 -- 172.21.15.18:0/3803411071 >> 172.21.15.18:0/3803411071 conn(0x7fd2680faba0 msgr2=0x7fd2680fcfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:49:28.642 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:49:28.640+0000 7fd270600700 1 -- 172.21.15.18:0/3803411071 shutdown_connections 2024-04-12T21:49:28.642 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:49:28.640+0000 7fd270600700 1 -- 172.21.15.18:0/3803411071 wait complete. 2024-04-12T21:49:28.643 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:49:28.641+0000 7fd270600700 1 Processor -- start 2024-04-12T21:49:28.643 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:49:28.641+0000 7fd270600700 1 -- start start 2024-04-12T21:49:28.643 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:49:28.642+0000 7fd270600700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd268105030 0x7fd268115bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:49:28.643 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:49:28.642+0000 7fd270600700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd2681059d0 0x7fd268116100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:49:28.643 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:49:28.642+0000 7fd270600700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2681167e0 con 0x7fd268105030 2024-04-12T21:49:28.644 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:49:28.642+0000 7fd270600700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd26810fd30 con 0x7fd2681059d0 2024-04-12T21:49:28.644 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:49:28.642+0000 7fd26dc00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd268105030 0x7fd268115bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:49:28.644 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:49:28.642+0000 7fd26dc00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd268105030 0x7fd268115bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:51772/0 (socket says 172.21.15.18:51772) 2024-04-12T21:49:28.644 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:49:28.642+0000 7fd26dc00700 1 -- 172.21.15.18:0/926537962 learned_addr learned my addr 172.21.15.18:0/926537962 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:49:28.644 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:49:28.642+0000 7fd26d200700 1 --2- 172.21.15.18:0/926537962 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd2681059d0 0x7fd268116100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:49:28.644 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:49:28.642+0000 7fd26dc00700 1 -- 172.21.15.18:0/926537962 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd2681059d0 msgr2=0x7fd268116100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:49:28.644 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:49:28.642+0000 7fd26dc00700 1 --2- 172.21.15.18:0/926537962 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd2681059d0 0x7fd268116100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:49:28.644 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:49:28.643+0000 7fd26dc00700 1 -- 172.21.15.18:0/926537962 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd254005040 con 0x7fd268105030 2024-04-12T21:49:28.645 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:49:28.643+0000 7fd26dc00700 1 --2- 172.21.15.18:0/926537962 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd268105030 0x7fd268115bc0 secure :-1 s=READY pgs=1267 cs=0 l=1 rev1=1 crypto rx=0x7fd25c00b790 tx=0x7fd25c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:49:28.645 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:49:28.643+0000 7fd265600700 1 -- 172.21.15.18:0/926537962 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd25c00f770 con 0x7fd268105030 2024-04-12T21:49:28.645 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:49:28.643+0000 7fd265600700 1 -- 172.21.15.18:0/926537962 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd25c00fda0 con 0x7fd268105030 2024-04-12T21:49:28.645 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:49:28.643+0000 7fd265600700 1 -- 172.21.15.18:0/926537962 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd25c00d310 con 0x7fd268105030 2024-04-12T21:49:28.645 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:49:28.643+0000 7fd270600700 1 -- 172.21.15.18:0/926537962 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd26810ffb0 con 0x7fd268105030 2024-04-12T21:49:28.645 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:49:28.643+0000 7fd270600700 1 -- 172.21.15.18:0/926537962 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd268110570 con 0x7fd268105030 2024-04-12T21:49:28.646 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:49:28.644+0000 7fd270600700 1 -- 172.21.15.18:0/926537962 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd268100780 con 0x7fd268105030 2024-04-12T21:49:28.646 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:49:28.646+0000 7fd265600700 1 -- 172.21.15.18:0/926537962 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7fd25c010420 con 0x7fd268105030 2024-04-12T21:49:28.647 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:49:28.646+0000 7fd265600700 1 --2- 172.21.15.18:0/926537962 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fd25006ec00 0x7fd2500710b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:49:28.647 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:49:28.647+0000 7fd265600700 1 -- 172.21.15.18:0/926537962 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 6757+0+0 (secure 0 0 0) 0x7fd25c08e8e0 con 0x7fd268105030 2024-04-12T21:49:28.648 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:49:28.647+0000 7fd26d200700 1 --2- 172.21.15.18:0/926537962 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fd25006ec00 0x7fd2500710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:49:28.648 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:49:28.648+0000 7fd26d200700 1 --2- 172.21.15.18:0/926537962 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fd25006ec00 0x7fd2500710b0 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7fd25400b3b0 tx=0x7fd254009e30 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:49:28.653 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:49:28.652+0000 7fd265600700 1 -- 172.21.15.18:0/926537962 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7fd25c05b1d0 con 0x7fd268105030 2024-04-12T21:49:28.855 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:49:28 smithi018 ceph-mon[112590]: from='client.36588 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:49:28.855 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:49:28 smithi018 ceph-mon[112590]: from='client.? 172.21.15.18:0/833557317' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-12T21:49:28.970 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:49:28.969+0000 7fd270600700 1 -- 172.21.15.18:0/926537962 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7fd268111240 con 0x7fd268105030 2024-04-12T21:49:28.970 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:49:28.969+0000 7fd265600700 1 -- 172.21.15.18:0/926537962 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+4303 (secure 0 0 0) 0x7fd25c05ad60 con 0x7fd268105030 2024-04-12T21:49:28.971 INFO:teuthology.orchestra.run.smithi018.stdout:HEALTH_ERR failed to probe daemons or devices; 1 filesystem is degraded; 1 filesystem is offline 2024-04-12T21:49:28.971 INFO:teuthology.orchestra.run.smithi018.stdout:[WRN] CEPHADM_REFRESH_FAILED: failed to probe daemons or devices 2024-04-12T21:49:28.971 INFO:teuthology.orchestra.run.smithi018.stdout: host smithi186 `cephadm ceph-volume` failed: cephadm exited with an error code: 1, stderr: Inferring config /var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/mon.smithi186/config 2024-04-12T21:49:28.971 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/podman: timeout after 895 seconds 2024-04-12T21:49:28.971 INFO:teuthology.orchestra.run.smithi018.stdout:Non-zero exit code 124 from /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:f4452def3be4d350f4a8e5feaf7ebb1511be9298fe68478ba7a2d6fbd7d69a63 -e NODE_NAME=smithi186 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968:/var/run/ceph:z -v /var/log/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968:/var/log/ceph:z -v /var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmptgom53f9:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:f4452def3be4d350f4a8e5feaf7ebb1511be9298fe68478ba7a2d6fbd7d69a63 inventory --format=json-pretty --filter-for-batch 2024-04-12T21:49:28.971 INFO:teuthology.orchestra.run.smithi018.stdout:Traceback (most recent call last): 2024-04-12T21:49:28.971 INFO:teuthology.orchestra.run.smithi018.stdout: File "/usr/lib64/python3.9/runpy.py", line 197, in _run_module_as_main 2024-04-12T21:49:28.971 INFO:teuthology.orchestra.run.smithi018.stdout: return _run_code(code, main_globals, None, 2024-04-12T21:49:28.971 INFO:teuthology.orchestra.run.smithi018.stdout: File "/usr/lib64/python3.9/runpy.py", line 87, in _run_code 2024-04-12T21:49:28.971 INFO:teuthology.orchestra.run.smithi018.stdout: exec(code, run_globals) 2024-04-12T21:49:28.971 INFO:teuthology.orchestra.run.smithi018.stdout: File "/var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/cephadm.e44f3ed5e34e4ef52eedef233d1190bff004470d8e8005127b5312bff357fa7f/__main__.py", line 5618, in 2024-04-12T21:49:28.971 INFO:teuthology.orchestra.run.smithi018.stdout: File "/var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/cephadm.e44f3ed5e34e4ef52eedef233d1190bff004470d8e8005127b5312bff357fa7f/__main__.py", line 5606, in main 2024-04-12T21:49:28.972 INFO:teuthology.orchestra.run.smithi018.stdout: File "/var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/cephadm.e44f3ed5e34e4ef52eedef233d1190bff004470d8e8005127b5312bff357fa7f/__main__.py", line 408, in _infer_config 2024-04-12T21:49:28.972 INFO:teuthology.orchestra.run.smithi018.stdout: File "/var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/cephadm.e44f3ed5e34e4ef52eedef233d1190bff004470d8e8005127b5312bff357fa7f/__main__.py", line 323, in _infer_fsid 2024-04-12T21:49:28.972 INFO:teuthology.orchestra.run.smithi018.stdout: File "/var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/cephadm.e44f3ed5e34e4ef52eedef233d1190bff004470d8e8005127b5312bff357fa7f/__main__.py", line 436, in _infer_image 2024-04-12T21:49:28.972 INFO:teuthology.orchestra.run.smithi018.stdout: File "/var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/cephadm.e44f3ed5e34e4ef52eedef233d1190bff004470d8e8005127b5312bff357fa7f/__main__.py", line 310, in _validate_fsid 2024-04-12T21:49:28.972 INFO:teuthology.orchestra.run.smithi018.stdout: File "/var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/cephadm.e44f3ed5e34e4ef52eedef233d1190bff004470d8e8005127b5312bff357fa7f/__main__.py", line 3359, in command_ceph_volume 2024-04-12T21:49:28.972 INFO:teuthology.orchestra.run.smithi018.stdout: File "/var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/cephadm.e44f3ed5e34e4ef52eedef233d1190bff004470d8e8005127b5312bff357fa7f/cephadmlib/call_wrappers.py", line 310, in call_throws 2024-04-12T21:49:28.972 INFO:teuthology.orchestra.run.smithi018.stdout:RuntimeError: Failed command: /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:f4452def3be4d350f4a8e5feaf7ebb1511be9298fe68478ba7a2d6fbd7d69a63 -e NODE_NAME=smithi186 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968:/var/run/ceph:z -v /var/log/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968:/var/log/ceph:z -v /var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmptgom53f9:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:f4452def3be4d350f4a8e5feaf7ebb1511be9298fe68478ba7a2d6fbd7d69a63 inventory --format=json-pretty --filter-for-batch 2024-04-12T21:49:28.972 INFO:teuthology.orchestra.run.smithi018.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-04-12T21:49:28.972 INFO:teuthology.orchestra.run.smithi018.stdout: fs cephfs is degraded 2024-04-12T21:49:28.972 INFO:teuthology.orchestra.run.smithi018.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-04-12T21:49:28.972 INFO:teuthology.orchestra.run.smithi018.stdout: fs cephfs is offline because no MDS is active for it. 2024-04-12T21:49:28.973 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:49:28.972+0000 7fd270600700 1 -- 172.21.15.18:0/926537962 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fd25006ec00 msgr2=0x7fd2500710b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:49:28.973 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:49:28.972+0000 7fd270600700 1 --2- 172.21.15.18:0/926537962 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fd25006ec00 0x7fd2500710b0 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7fd25400b3b0 tx=0x7fd254009e30 comp rx=0 tx=0).stop 2024-04-12T21:49:28.973 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:49:28.972+0000 7fd270600700 1 -- 172.21.15.18:0/926537962 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd268105030 msgr2=0x7fd268115bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:49:28.973 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:49:28.972+0000 7fd270600700 1 --2- 172.21.15.18:0/926537962 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd268105030 0x7fd268115bc0 secure :-1 s=READY pgs=1267 cs=0 l=1 rev1=1 crypto rx=0x7fd25c00b790 tx=0x7fd25c009250 comp rx=0 tx=0).stop 2024-04-12T21:49:28.973 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:49:28.972+0000 7fd270600700 1 -- 172.21.15.18:0/926537962 shutdown_connections 2024-04-12T21:49:28.973 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:49:28.972+0000 7fd270600700 1 --2- 172.21.15.18:0/926537962 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fd25006ec00 0x7fd2500710b0 unknown :-1 s=CLOSED pgs=672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:49:28.973 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:49:28.972+0000 7fd270600700 1 --2- 172.21.15.18:0/926537962 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd268105030 0x7fd268115bc0 unknown :-1 s=CLOSED pgs=1267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:49:28.974 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:49:28.972+0000 7fd270600700 1 --2- 172.21.15.18:0/926537962 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd2681059d0 0x7fd268116100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:49:28.974 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:49:28.972+0000 7fd270600700 1 -- 172.21.15.18:0/926537962 >> 172.21.15.18:0/926537962 conn(0x7fd2680faba0 msgr2=0x7fd2680fba40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:49:28.974 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:49:28.972+0000 7fd270600700 1 -- 172.21.15.18:0/926537962 shutdown_connections 2024-04-12T21:49:28.974 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:49:28.972+0000 7fd270600700 1 -- 172.21.15.18:0/926537962 wait complete. 2024-04-12T21:49:29.034 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:49:28 smithi186 ceph-mon[86379]: from='client.36588 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:49:29.034 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:49:28 smithi186 ceph-mon[86379]: from='client.? 172.21.15.18:0/833557317' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-12T21:49:30.034 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:49:29 smithi186 ceph-mon[86379]: pgmap v1494: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:49:30.034 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:49:29 smithi186 ceph-mon[86379]: from='client.36600 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:49:30.035 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:49:29 smithi186 ceph-mon[86379]: from='client.? 172.21.15.18:0/926537962' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-12T21:49:30.105 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:49:29 smithi018 ceph-mon[112590]: pgmap v1494: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:49:30.105 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:49:29 smithi018 ceph-mon[112590]: from='client.36600 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:49:30.105 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:49:29 smithi018 ceph-mon[112590]: from='client.? 172.21.15.18:0/926537962' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-12T21:49:32.034 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:49:31 smithi186 ceph-mon[86379]: pgmap v1495: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:49:32.105 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:49:31 smithi018 ceph-mon[112590]: pgmap v1495: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:49:34.034 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:49:33 smithi186 ceph-mon[86379]: pgmap v1496: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:49:34.105 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:49:33 smithi018 ceph-mon[112590]: pgmap v1496: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:49:36.034 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:49:35 smithi186 ceph-mon[86379]: pgmap v1497: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:49:36.105 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:49:35 smithi018 ceph-mon[112590]: pgmap v1497: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:49:38.034 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:49:37 smithi186 ceph-mon[86379]: pgmap v1498: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:49:38.105 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:49:37 smithi018 ceph-mon[112590]: pgmap v1498: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:49:40.034 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:49:39 smithi186 ceph-mon[86379]: pgmap v1499: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:49:40.105 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:49:39 smithi018 ceph-mon[112590]: pgmap v1499: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:49:41.034 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:49:40 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T21:49:41.105 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:49:40 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T21:49:42.034 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:49:41 smithi186 ceph-mon[86379]: pgmap v1500: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:49:42.105 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:49:41 smithi018 ceph-mon[112590]: pgmap v1500: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:49:44.034 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:49:43 smithi186 ceph-mon[86379]: pgmap v1501: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:49:44.105 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:49:43 smithi018 ceph-mon[112590]: pgmap v1501: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:49:46.034 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:49:45 smithi186 ceph-mon[86379]: pgmap v1502: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:49:46.105 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:49:45 smithi018 ceph-mon[112590]: pgmap v1502: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:49:48.034 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:49:47 smithi186 ceph-mon[86379]: pgmap v1503: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:49:48.105 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:49:47 smithi018 ceph-mon[112590]: pgmap v1503: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:49:50.105 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:49:49 smithi018 ceph-mon[112590]: pgmap v1504: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:49:50.284 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:49:49 smithi186 ceph-mon[86379]: pgmap v1504: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:49:52.105 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:49:51 smithi018 ceph-mon[112590]: pgmap v1505: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:49:52.284 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:49:51 smithi186 ceph-mon[86379]: pgmap v1505: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:49:54.105 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:49:53 smithi018 ceph-mon[112590]: pgmap v1506: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:49:54.284 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:49:53 smithi186 ceph-mon[86379]: pgmap v1506: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:49:56.105 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:49:55 smithi018 ceph-mon[112590]: pgmap v1507: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:49:56.105 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:49:55 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T21:49:56.284 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:49:55 smithi186 ceph-mon[86379]: pgmap v1507: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:49:56.284 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:49:55 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T21:49:58.087 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:49:57 smithi186 ceph-mon[86379]: pgmap v1508: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:49:58.105 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:49:57 smithi018 ceph-mon[112590]: pgmap v1508: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:49:59.148 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:49:59.146+0000 7f0eddc00700 1 -- 172.21.15.18:0/1905904034 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0ed81035e0 msgr2=0x7f0ed80fcc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:49:59.148 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:49:59.146+0000 7f0eddc00700 1 --2- 172.21.15.18:0/1905904034 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0ed81035e0 0x7f0ed80fcc50 secure :-1 s=READY pgs=1268 cs=0 l=1 rev1=1 crypto rx=0x7f0ec4004940 tx=0x7f0ec4009e30 comp rx=0 tx=0).stop 2024-04-12T21:49:59.148 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:49:59.146+0000 7f0eddc00700 1 -- 172.21.15.18:0/1905904034 shutdown_connections 2024-04-12T21:49:59.148 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:49:59.146+0000 7f0eddc00700 1 --2- 172.21.15.18:0/1905904034 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0ed81035e0 0x7f0ed80fcc50 unknown :-1 s=CLOSED pgs=1268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:49:59.149 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:49:59.146+0000 7f0eddc00700 1 --2- 172.21.15.18:0/1905904034 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0ed8102c40 0x7f0ed8103010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:49:59.149 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:49:59.146+0000 7f0eddc00700 1 -- 172.21.15.18:0/1905904034 >> 172.21.15.18:0/1905904034 conn(0x7f0ed80f8770 msgr2=0x7f0ed80fab80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:49:59.149 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:49:59.146+0000 7f0eddc00700 1 -- 172.21.15.18:0/1905904034 shutdown_connections 2024-04-12T21:49:59.149 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:49:59.146+0000 7f0eddc00700 1 -- 172.21.15.18:0/1905904034 wait complete. 2024-04-12T21:49:59.150 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:49:59.147+0000 7f0eddc00700 1 Processor -- start 2024-04-12T21:49:59.150 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:49:59.147+0000 7f0eddc00700 1 -- start start 2024-04-12T21:49:59.151 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:49:59.147+0000 7f0eddc00700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0ed8102c40 0x7f0ed8115c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:49:59.151 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:49:59.147+0000 7f0eddc00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0ed81035e0 0x7f0ed8116150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:49:59.151 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:49:59.148+0000 7f0eddc00700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ed81167a0 con 0x7f0ed81035e0 2024-04-12T21:49:59.151 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:49:59.148+0000 7f0eddc00700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ed810fb90 con 0x7f0ed8102c40 2024-04-12T21:49:59.151 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:49:59.148+0000 7f0ed7400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0ed81035e0 0x7f0ed8116150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:49:59.151 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:49:59.148+0000 7f0ed7400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0ed81035e0 0x7f0ed8116150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:59072/0 (socket says 172.21.15.18:59072) 2024-04-12T21:49:59.151 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:49:59.148+0000 7f0ed7400700 1 -- 172.21.15.18:0/750124056 learned_addr learned my addr 172.21.15.18:0/750124056 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:49:59.151 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:49:59.148+0000 7f0ed7e00700 1 --2- 172.21.15.18:0/750124056 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0ed8102c40 0x7f0ed8115c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:49:59.151 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:49:59.148+0000 7f0ed7400700 1 -- 172.21.15.18:0/750124056 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0ed8102c40 msgr2=0x7f0ed8115c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:49:59.151 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:49:59.148+0000 7f0ed7400700 1 --2- 172.21.15.18:0/750124056 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0ed8102c40 0x7f0ed8115c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:49:59.151 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:49:59.148+0000 7f0ed7400700 1 -- 172.21.15.18:0/750124056 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ec4005040 con 0x7f0ed81035e0 2024-04-12T21:49:59.151 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:49:59.149+0000 7f0ed7400700 1 --2- 172.21.15.18:0/750124056 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0ed81035e0 0x7f0ed8116150 secure :-1 s=READY pgs=1269 cs=0 l=1 rev1=1 crypto rx=0x7f0ec4009100 tx=0x7f0ec4009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:49:59.152 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:49:59.149+0000 7f0ecfe00700 1 -- 172.21.15.18:0/750124056 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0ec400e040 con 0x7f0ed81035e0 2024-04-12T21:49:59.152 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:49:59.149+0000 7f0ecfe00700 1 -- 172.21.15.18:0/750124056 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f0ec4013420 con 0x7f0ed81035e0 2024-04-12T21:49:59.152 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:49:59.149+0000 7f0ecfe00700 1 -- 172.21.15.18:0/750124056 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0ec401c630 con 0x7f0ed81035e0 2024-04-12T21:49:59.152 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:49:59.149+0000 7f0eddc00700 1 -- 172.21.15.18:0/750124056 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0ed810fe10 con 0x7f0ed81035e0 2024-04-12T21:49:59.152 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:49:59.149+0000 7f0eddc00700 1 -- 172.21.15.18:0/750124056 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0ed81103d0 con 0x7f0ed81035e0 2024-04-12T21:49:59.153 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:49:59.150+0000 7f0eddc00700 1 -- 172.21.15.18:0/750124056 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0ed80fe390 con 0x7f0ed81035e0 2024-04-12T21:49:59.153 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:49:59.152+0000 7f0ecfe00700 1 -- 172.21.15.18:0/750124056 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7f0ec4013590 con 0x7f0ed81035e0 2024-04-12T21:49:59.154 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:49:59.153+0000 7f0ecfe00700 1 --2- 172.21.15.18:0/750124056 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f0ebc06ec00 0x7f0ebc0710b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:49:59.155 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:49:59.153+0000 7f0ecfe00700 1 -- 172.21.15.18:0/750124056 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 6757+0+0 (secure 0 0 0) 0x7f0ec408fae0 con 0x7f0ed81035e0 2024-04-12T21:49:59.155 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:49:59.153+0000 7f0ed7e00700 1 --2- 172.21.15.18:0/750124056 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f0ebc06ec00 0x7f0ebc0710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:49:59.155 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:49:59.154+0000 7f0ed7e00700 1 --2- 172.21.15.18:0/750124056 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f0ebc06ec00 0x7f0ebc0710b0 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f0ec8004fa0 tx=0x7f0ec800b040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:49:59.160 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:49:59.158+0000 7f0ecfe00700 1 -- 172.21.15.18:0/750124056 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f0ec405c3d0 con 0x7f0ed81035e0 2024-04-12T21:49:59.413 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:49:59.410+0000 7f0eddc00700 1 -- 172.21.15.18:0/750124056 --> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f0ed8109d10 con 0x7f0ebc06ec00 2024-04-12T21:49:59.416 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:49:59.414+0000 7f0ecfe00700 1 -- 172.21.15.18:0/750124056 <== mgr.34102 v2:172.21.15.18:6800/3812891919 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f0ed8109d10 con 0x7f0ebc06ec00 2024-04-12T21:49:59.419 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:49:59.417+0000 7f0eddc00700 1 -- 172.21.15.18:0/750124056 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f0ebc06ec00 msgr2=0x7f0ebc0710b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:49:59.419 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:49:59.417+0000 7f0eddc00700 1 --2- 172.21.15.18:0/750124056 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f0ebc06ec00 0x7f0ebc0710b0 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f0ec8004fa0 tx=0x7f0ec800b040 comp rx=0 tx=0).stop 2024-04-12T21:49:59.419 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:49:59.417+0000 7f0eddc00700 1 -- 172.21.15.18:0/750124056 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0ed81035e0 msgr2=0x7f0ed8116150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:49:59.419 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:49:59.417+0000 7f0eddc00700 1 --2- 172.21.15.18:0/750124056 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0ed81035e0 0x7f0ed8116150 secure :-1 s=READY pgs=1269 cs=0 l=1 rev1=1 crypto rx=0x7f0ec4009100 tx=0x7f0ec4009e30 comp rx=0 tx=0).stop 2024-04-12T21:49:59.419 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:49:59.417+0000 7f0eddc00700 1 -- 172.21.15.18:0/750124056 shutdown_connections 2024-04-12T21:49:59.419 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:49:59.417+0000 7f0eddc00700 1 --2- 172.21.15.18:0/750124056 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f0ebc06ec00 0x7f0ebc0710b0 unknown :-1 s=CLOSED pgs=673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:49:59.419 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:49:59.417+0000 7f0eddc00700 1 --2- 172.21.15.18:0/750124056 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0ed8102c40 0x7f0ed8115c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:49:59.419 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:49:59.417+0000 7f0eddc00700 1 --2- 172.21.15.18:0/750124056 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0ed81035e0 0x7f0ed8116150 unknown :-1 s=CLOSED pgs=1269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:49:59.419 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:49:59.417+0000 7f0eddc00700 1 -- 172.21.15.18:0/750124056 >> 172.21.15.18:0/750124056 conn(0x7f0ed80f8770 msgr2=0x7f0ed80f9720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:49:59.420 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:49:59.417+0000 7f0eddc00700 1 -- 172.21.15.18:0/750124056 shutdown_connections 2024-04-12T21:49:59.420 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:49:59.417+0000 7f0eddc00700 1 -- 172.21.15.18:0/750124056 wait complete. 2024-04-12T21:49:59.435 INFO:teuthology.orchestra.run.smithi018.stdout:true 2024-04-12T21:49:59.582 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:49:59.579+0000 7f68c5800700 1 -- 172.21.15.18:0/2097929529 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f68c0102de0 msgr2=0x7f68c01031b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:49:59.582 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:49:59.579+0000 7f68c5800700 1 --2- 172.21.15.18:0/2097929529 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f68c0102de0 0x7f68c01031b0 secure :-1 s=READY pgs=1270 cs=0 l=1 rev1=1 crypto rx=0x7f68b0004940 tx=0x7f68b0009e30 comp rx=0 tx=0).stop 2024-04-12T21:49:59.582 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:49:59.580+0000 7f68c5800700 1 -- 172.21.15.18:0/2097929529 shutdown_connections 2024-04-12T21:49:59.582 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:49:59.580+0000 7f68c5800700 1 --2- 172.21.15.18:0/2097929529 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f68c0103780 0x7f68c00fcdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:49:59.583 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:49:59.580+0000 7f68c5800700 1 --2- 172.21.15.18:0/2097929529 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f68c0102de0 0x7f68c01031b0 unknown :-1 s=CLOSED pgs=1270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:49:59.583 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:49:59.580+0000 7f68c5800700 1 -- 172.21.15.18:0/2097929529 >> 172.21.15.18:0/2097929529 conn(0x7f68c00f8950 msgr2=0x7f68c00fad60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:49:59.583 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:49:59.580+0000 7f68c5800700 1 -- 172.21.15.18:0/2097929529 shutdown_connections 2024-04-12T21:49:59.583 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:49:59.580+0000 7f68c5800700 1 -- 172.21.15.18:0/2097929529 wait complete. 2024-04-12T21:49:59.584 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:49:59.581+0000 7f68c5800700 1 Processor -- start 2024-04-12T21:49:59.584 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:49:59.581+0000 7f68c5800700 1 -- start start 2024-04-12T21:49:59.584 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:49:59.581+0000 7f68c5800700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f68c0102de0 0x7f68c010a6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:49:59.584 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:49:59.581+0000 7f68c5800700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f68c0103780 0x7f68c010abf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:49:59.584 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:49:59.581+0000 7f68c5800700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68c010e630 con 0x7f68c0102de0 2024-04-12T21:49:59.585 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:49:59.581+0000 7f68c5800700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68c010e770 con 0x7f68c0103780 2024-04-12T21:49:59.585 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:49:59.582+0000 7f68bea00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f68c0102de0 0x7f68c010a6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:49:59.585 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:49:59.582+0000 7f68bea00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f68c0102de0 0x7f68c010a6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:59092/0 (socket says 172.21.15.18:59092) 2024-04-12T21:49:59.585 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:49:59.582+0000 7f68bea00700 1 -- 172.21.15.18:0/545761340 learned_addr learned my addr 172.21.15.18:0/545761340 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:49:59.585 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:49:59.582+0000 7f68b7e00700 1 --2- 172.21.15.18:0/545761340 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f68c0103780 0x7f68c010abf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:49:59.585 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:49:59.582+0000 7f68bea00700 1 -- 172.21.15.18:0/545761340 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f68c0103780 msgr2=0x7f68c010abf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:49:59.585 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:49:59.582+0000 7f68bea00700 1 --2- 172.21.15.18:0/545761340 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f68c0103780 0x7f68c010abf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:49:59.585 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:49:59.582+0000 7f68bea00700 1 -- 172.21.15.18:0/545761340 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68b0005040 con 0x7f68c0102de0 2024-04-12T21:49:59.585 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:49:59.583+0000 7f68bea00700 1 --2- 172.21.15.18:0/545761340 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f68c0102de0 0x7f68c010a6b0 secure :-1 s=READY pgs=1271 cs=0 l=1 rev1=1 crypto rx=0x7f68b0009180 tx=0x7f68b0009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:49:59.586 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:49:59.583+0000 7f68b6a00700 1 -- 172.21.15.18:0/545761340 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f68b000e040 con 0x7f68c0102de0 2024-04-12T21:49:59.586 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:49:59.583+0000 7f68b6a00700 1 -- 172.21.15.18:0/545761340 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f68b000bbc0 con 0x7f68c0102de0 2024-04-12T21:49:59.586 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:49:59.583+0000 7f68b6a00700 1 -- 172.21.15.18:0/545761340 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f68b001c630 con 0x7f68c0102de0 2024-04-12T21:49:59.586 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:49:59.583+0000 7f68c5800700 1 -- 172.21.15.18:0/545761340 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f68c010b130 con 0x7f68c0102de0 2024-04-12T21:49:59.586 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:49:59.583+0000 7f68c5800700 1 -- 172.21.15.18:0/545761340 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f68c011e4e0 con 0x7f68c0102de0 2024-04-12T21:49:59.586 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:49:59.584+0000 7f68c5800700 1 -- 172.21.15.18:0/545761340 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f68c01072e0 con 0x7f68c0102de0 2024-04-12T21:49:59.587 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:49:59.586+0000 7f68b6a00700 1 -- 172.21.15.18:0/545761340 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7f68b000bd30 con 0x7f68c0102de0 2024-04-12T21:49:59.588 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:49:59.587+0000 7f68b6a00700 1 --2- 172.21.15.18:0/545761340 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f68a406ec00 0x7f68a40710b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:49:59.588 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:49:59.587+0000 7f68b6a00700 1 -- 172.21.15.18:0/545761340 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 6757+0+0 (secure 0 0 0) 0x7f68b008fd40 con 0x7f68c0102de0 2024-04-12T21:49:59.589 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:49:59.587+0000 7f68b7e00700 1 --2- 172.21.15.18:0/545761340 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f68a406ec00 0x7f68a40710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:49:59.589 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:49:59.588+0000 7f68b7e00700 1 --2- 172.21.15.18:0/545761340 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f68a406ec00 0x7f68a40710b0 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f68c010bde0 tx=0x7f68a800b040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:49:59.594 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:49:59.592+0000 7f68b6a00700 1 -- 172.21.15.18:0/545761340 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f68b0058a40 con 0x7f68c0102de0 2024-04-12T21:49:59.850 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:49:59.848+0000 7f68c5800700 1 -- 172.21.15.18:0/545761340 --> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f68c010bc00 con 0x7f68a406ec00 2024-04-12T21:49:59.852 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:49:59.851+0000 7f68b6a00700 1 -- 172.21.15.18:0/545761340 <== mgr.34102 v2:172.21.15.18:6800/3812891919 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f68c010bc00 con 0x7f68a406ec00 2024-04-12T21:49:59.855 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:49:59.853+0000 7f68c5800700 1 -- 172.21.15.18:0/545761340 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f68a406ec00 msgr2=0x7f68a40710b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:49:59.855 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:49:59.853+0000 7f68c5800700 1 --2- 172.21.15.18:0/545761340 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f68a406ec00 0x7f68a40710b0 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f68c010bde0 tx=0x7f68a800b040 comp rx=0 tx=0).stop 2024-04-12T21:49:59.855 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:49:59.853+0000 7f68c5800700 1 -- 172.21.15.18:0/545761340 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f68c0102de0 msgr2=0x7f68c010a6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:49:59.855 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:49:59.853+0000 7f68c5800700 1 --2- 172.21.15.18:0/545761340 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f68c0102de0 0x7f68c010a6b0 secure :-1 s=READY pgs=1271 cs=0 l=1 rev1=1 crypto rx=0x7f68b0009180 tx=0x7f68b0009e30 comp rx=0 tx=0).stop 2024-04-12T21:49:59.856 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:49:59.854+0000 7f68c5800700 1 -- 172.21.15.18:0/545761340 shutdown_connections 2024-04-12T21:49:59.856 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:49:59.854+0000 7f68c5800700 1 --2- 172.21.15.18:0/545761340 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f68a406ec00 0x7f68a40710b0 unknown :-1 s=CLOSED pgs=674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:49:59.856 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:49:59.854+0000 7f68c5800700 1 --2- 172.21.15.18:0/545761340 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f68c0102de0 0x7f68c010a6b0 unknown :-1 s=CLOSED pgs=1271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:49:59.856 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:49:59.854+0000 7f68c5800700 1 --2- 172.21.15.18:0/545761340 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f68c0103780 0x7f68c010abf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:49:59.856 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:49:59.854+0000 7f68c5800700 1 -- 172.21.15.18:0/545761340 >> 172.21.15.18:0/545761340 conn(0x7f68c00f8950 msgr2=0x7f68c00fa590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:49:59.856 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:49:59.854+0000 7f68c5800700 1 -- 172.21.15.18:0/545761340 shutdown_connections 2024-04-12T21:49:59.857 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:49:59.854+0000 7f68c5800700 1 -- 172.21.15.18:0/545761340 wait complete. 2024-04-12T21:50:00.012 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:50:00.009+0000 7f0e3d000700 1 -- 172.21.15.18:0/1666596551 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0e381024d0 msgr2=0x7f0e380fbb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:50:00.012 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:50:00.009+0000 7f0e3d000700 1 --2- 172.21.15.18:0/1666596551 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0e381024d0 0x7f0e380fbb40 secure :-1 s=READY pgs=1272 cs=0 l=1 rev1=1 crypto rx=0x7f0e24004940 tx=0x7f0e24009e30 comp rx=0 tx=0).stop 2024-04-12T21:50:00.012 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:50:00.010+0000 7f0e3d000700 1 -- 172.21.15.18:0/1666596551 shutdown_connections 2024-04-12T21:50:00.012 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:50:00.010+0000 7f0e3d000700 1 --2- 172.21.15.18:0/1666596551 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0e381024d0 0x7f0e380fbb40 unknown :-1 s=CLOSED pgs=1272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:50:00.012 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:50:00.010+0000 7f0e3d000700 1 --2- 172.21.15.18:0/1666596551 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0e38101b30 0x7f0e38101f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:50:00.012 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:50:00.010+0000 7f0e3d000700 1 -- 172.21.15.18:0/1666596551 >> 172.21.15.18:0/1666596551 conn(0x7f0e380f76a0 msgr2=0x7f0e380f9ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:50:00.013 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:50:00.010+0000 7f0e3d000700 1 -- 172.21.15.18:0/1666596551 shutdown_connections 2024-04-12T21:50:00.013 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:50:00.010+0000 7f0e3d000700 1 -- 172.21.15.18:0/1666596551 wait complete. 2024-04-12T21:50:00.014 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:50:00.011+0000 7f0e3d000700 1 Processor -- start 2024-04-12T21:50:00.014 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:50:00.011+0000 7f0e3d000700 1 -- start start 2024-04-12T21:50:00.014 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:50:00.011+0000 7f0e3d000700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0e38101b30 0x7f0e38113980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:50:00.014 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:50:00.011+0000 7f0e3d000700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0e381024d0 0x7f0e38113ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:50:00.015 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:50:00.011+0000 7f0e3d000700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e381145a0 con 0x7f0e38101b30 2024-04-12T21:50:00.015 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:50:00.011+0000 7f0e3d000700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e3810d900 con 0x7f0e381024d0 2024-04-12T21:50:00.015 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:50:00.012+0000 7f0e37400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0e38101b30 0x7f0e38113980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:50:00.015 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:50:00.012+0000 7f0e37400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0e38101b30 0x7f0e38113980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:59110/0 (socket says 172.21.15.18:59110) 2024-04-12T21:50:00.015 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:50:00.012+0000 7f0e37400700 1 -- 172.21.15.18:0/283211875 learned_addr learned my addr 172.21.15.18:0/283211875 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:50:00.015 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:50:00.012+0000 7f0e36a00700 1 --2- 172.21.15.18:0/283211875 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0e381024d0 0x7f0e38113ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:50:00.015 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:50:00.012+0000 7f0e37400700 1 -- 172.21.15.18:0/283211875 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0e381024d0 msgr2=0x7f0e38113ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:50:00.015 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:50:00.012+0000 7f0e37400700 1 --2- 172.21.15.18:0/283211875 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0e381024d0 0x7f0e38113ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:50:00.015 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:50:00.012+0000 7f0e37400700 1 -- 172.21.15.18:0/283211875 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e24005040 con 0x7f0e38101b30 2024-04-12T21:50:00.015 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:50:00.013+0000 7f0e37400700 1 --2- 172.21.15.18:0/283211875 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0e38101b30 0x7f0e38113980 secure :-1 s=READY pgs=1273 cs=0 l=1 rev1=1 crypto rx=0x7f0e2800b790 tx=0x7f0e28009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:50:00.016 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:50:00.013+0000 7f0e2f400700 1 -- 172.21.15.18:0/283211875 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0e2800f770 con 0x7f0e38101b30 2024-04-12T21:50:00.016 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:50:00.013+0000 7f0e2f400700 1 -- 172.21.15.18:0/283211875 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f0e2800fda0 con 0x7f0e38101b30 2024-04-12T21:50:00.016 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:50:00.013+0000 7f0e3d000700 1 -- 172.21.15.18:0/283211875 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0e3810dbe0 con 0x7f0e38101b30 2024-04-12T21:50:00.016 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:50:00.013+0000 7f0e2f400700 1 -- 172.21.15.18:0/283211875 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0e2800d310 con 0x7f0e38101b30 2024-04-12T21:50:00.016 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:50:00.013+0000 7f0e3d000700 1 -- 172.21.15.18:0/283211875 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0e3810e1a0 con 0x7f0e38101b30 2024-04-12T21:50:00.016 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:50:00.014+0000 7f0e3d000700 1 -- 172.21.15.18:0/283211875 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0e380fd280 con 0x7f0e38101b30 2024-04-12T21:50:00.017 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:50:00.015+0000 7f0e2f400700 1 -- 172.21.15.18:0/283211875 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7f0e28010420 con 0x7f0e38101b30 2024-04-12T21:50:00.018 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:50:00.016+0000 7f0e2f400700 1 --2- 172.21.15.18:0/283211875 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f0e1c06ec10 0x7f0e1c0710c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:50:00.018 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:50:00.016+0000 7f0e2f400700 1 -- 172.21.15.18:0/283211875 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 6757+0+0 (secure 0 0 0) 0x7f0e2808e940 con 0x7f0e38101b30 2024-04-12T21:50:00.018 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:50:00.017+0000 7f0e36a00700 1 --2- 172.21.15.18:0/283211875 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f0e1c06ec10 0x7f0e1c0710c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:50:00.019 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:50:00.017+0000 7f0e36a00700 1 --2- 172.21.15.18:0/283211875 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f0e1c06ec10 0x7f0e1c0710c0 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f0e3810eac0 tx=0x7f0e24009e30 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:50:00.023 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:50:00.021+0000 7f0e2f400700 1 -- 172.21.15.18:0/283211875 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f0e2805b230 con 0x7f0e38101b30 2024-04-12T21:50:00.105 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:49:59 smithi018 ceph-mon[112590]: pgmap v1509: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:50:00.105 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:50:00 smithi018 ceph-1cbe407a-f90d-11ee-bc8d-c7b262605968-mon-smithi018[112586]: 2024-04-12T21:49:59.998+0000 7fd45a600700 -1 log_channel(cluster) log [ERR] : overall HEALTH_ERR failed to probe daemons or devices; 1 filesystem is degraded; 1 filesystem is offline 2024-04-12T21:50:00.268 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:50:00.266+0000 7f0e3d000700 1 -- 172.21.15.18:0/283211875 --> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f0e38108cf0 con 0x7f0e1c06ec10 2024-04-12T21:50:00.284 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:49:59 smithi186 ceph-mon[86379]: pgmap v1509: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:50:00.296 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:50:00.294+0000 7f0e2f400700 1 -- 172.21.15.18:0/283211875 <== mgr.34102 v2:172.21.15.18:6800/3812891919 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7f0e38108cf0 con 0x7f0e1c06ec10 2024-04-12T21:50:00.297 INFO:teuthology.orchestra.run.smithi018.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-12T21:50:00.297 INFO:teuthology.orchestra.run.smithi018.stdout:alertmanager.smithi018 smithi018 *:9093,9094 running (50m) 11m ago 65m 14.8M - 0.25.0 c8568f914cd2 c332090068e8 2024-04-12T21:50:00.297 INFO:teuthology.orchestra.run.smithi018.stdout:crash.smithi018 smithi018 running (46m) 11m ago 65m 6665k - 19.0.0-2879-g324baf8d 8717720dc5e5 b7f80d1f7982 2024-04-12T21:50:00.297 INFO:teuthology.orchestra.run.smithi018.stdout:crash.smithi186 smithi186 running (46m) 11m ago 63m 6660k - 19.0.0-2879-g324baf8d 8717720dc5e5 a4f55c81b623 2024-04-12T21:50:00.297 INFO:teuthology.orchestra.run.smithi018.stdout:grafana.smithi018 smithi018 *:3000 running (50m) 11m ago 64m 73.8M - 9.4.12 2bacad6d85d8 b08e3bf176dd 2024-04-12T21:50:00.297 INFO:teuthology.orchestra.run.smithi018.stdout:mds.cephfs.smithi018.acqbsc smithi018 running (42m) 11m ago 57m 21.9M - 19.0.0-2879-g324baf8d 8717720dc5e5 021f99ab24ae 2024-04-12T21:50:00.297 INFO:teuthology.orchestra.run.smithi018.stdout:mds.cephfs.smithi018.szjluo smithi018 running (42m) 11m ago 57m 21.7M - 19.0.0-2879-g324baf8d 8717720dc5e5 0003117479d4 2024-04-12T21:50:00.297 INFO:teuthology.orchestra.run.smithi018.stdout:mds.cephfs.smithi186.jdllyn smithi186 running (27m) 11m ago 57m 16.5M - 19.0.0-2879-g324baf8d 8717720dc5e5 21b1f46eef23 2024-04-12T21:50:00.297 INFO:teuthology.orchestra.run.smithi018.stdout:mds.cephfs.smithi186.wjnafs smithi186 running (11m) 11m ago 57m 11.0M - 19.0.0-2879-g324baf8d 8717720dc5e5 3cf1ef4148e2 2024-04-12T21:50:00.297 INFO:teuthology.orchestra.run.smithi018.stdout:mgr.smithi018.xjzwga smithi018 *:8443,9283,8765 running (53m) 11m ago 66m 517M - 19.0.0-2879-g324baf8d 8717720dc5e5 56088305eab7 2024-04-12T21:50:00.298 INFO:teuthology.orchestra.run.smithi018.stdout:mgr.smithi186.kqcfxk smithi186 *:8443,9283,8765 running (51m) 11m ago 63m 460M - 19.0.0-2879-g324baf8d 8717720dc5e5 09cd766e5e0d 2024-04-12T21:50:00.298 INFO:teuthology.orchestra.run.smithi018.stdout:mon.smithi018 smithi018 running (48m) 11m ago 66m 88.2M 2048M 19.0.0-2879-g324baf8d 8717720dc5e5 df3e201dfed6 2024-04-12T21:50:00.298 INFO:teuthology.orchestra.run.smithi018.stdout:mon.smithi186 smithi186 running (47m) 11m ago 63m 53.6M 2048M 19.0.0-2879-g324baf8d 8717720dc5e5 c77498da9b2b 2024-04-12T21:50:00.298 INFO:teuthology.orchestra.run.smithi018.stdout:node-exporter.smithi018 smithi018 *:9100 running (51m) 11m ago 65m 8468k - 1.5.0 0da6a335fe13 326c3d9c6847 2024-04-12T21:50:00.298 INFO:teuthology.orchestra.run.smithi018.stdout:node-exporter.smithi186 smithi186 *:9100 running (51m) 11m ago 63m 8166k - 1.5.0 0da6a335fe13 ec2112baeecf 2024-04-12T21:50:00.298 INFO:teuthology.orchestra.run.smithi018.stdout:osd.0 smithi018 running (46m) 11m ago 62m 123M 1559M 19.0.0-2879-g324baf8d 8717720dc5e5 60b1273ca047 2024-04-12T21:50:00.298 INFO:teuthology.orchestra.run.smithi018.stdout:osd.1 smithi018 running (45m) 11m ago 62m 95.4M 1559M 19.0.0-2879-g324baf8d 8717720dc5e5 170ce1e5fb8d 2024-04-12T21:50:00.298 INFO:teuthology.orchestra.run.smithi018.stdout:osd.2 smithi018 running (45m) 11m ago 61m 94.6M 1559M 19.0.0-2879-g324baf8d 8717720dc5e5 c2cbc6aa2839 2024-04-12T21:50:00.298 INFO:teuthology.orchestra.run.smithi018.stdout:osd.3 smithi186 running (44m) 11m ago 60m 101M 2581M 19.0.0-2879-g324baf8d 8717720dc5e5 0138e8262e4b 2024-04-12T21:50:00.298 INFO:teuthology.orchestra.run.smithi018.stdout:osd.4 smithi186 running (43m) 11m ago 60m 91.5M 2581M 19.0.0-2879-g324baf8d 8717720dc5e5 b20cb63b25c9 2024-04-12T21:50:00.298 INFO:teuthology.orchestra.run.smithi018.stdout:osd.5 smithi186 running (43m) 11m ago 59m 92.2M 2581M 19.0.0-2879-g324baf8d 8717720dc5e5 2513bea2610e 2024-04-12T21:50:00.298 INFO:teuthology.orchestra.run.smithi018.stdout:prometheus.smithi018 smithi018 *:9095 running (51m) 11m ago 64m 69.6M - 2.43.0 a07b618ecd1d f58841127e21 2024-04-12T21:50:00.300 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:50:00.297+0000 7f0e3d000700 1 -- 172.21.15.18:0/283211875 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f0e1c06ec10 msgr2=0x7f0e1c0710c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:50:00.300 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:50:00.297+0000 7f0e3d000700 1 --2- 172.21.15.18:0/283211875 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f0e1c06ec10 0x7f0e1c0710c0 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f0e3810eac0 tx=0x7f0e24009e30 comp rx=0 tx=0).stop 2024-04-12T21:50:00.300 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:50:00.297+0000 7f0e3d000700 1 -- 172.21.15.18:0/283211875 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0e38101b30 msgr2=0x7f0e38113980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:50:00.300 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:50:00.297+0000 7f0e3d000700 1 --2- 172.21.15.18:0/283211875 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0e38101b30 0x7f0e38113980 secure :-1 s=READY pgs=1273 cs=0 l=1 rev1=1 crypto rx=0x7f0e2800b790 tx=0x7f0e28009250 comp rx=0 tx=0).stop 2024-04-12T21:50:00.300 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:50:00.298+0000 7f0e3d000700 1 -- 172.21.15.18:0/283211875 shutdown_connections 2024-04-12T21:50:00.300 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:50:00.298+0000 7f0e3d000700 1 --2- 172.21.15.18:0/283211875 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f0e1c06ec10 0x7f0e1c0710c0 unknown :-1 s=CLOSED pgs=675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:50:00.301 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:50:00.298+0000 7f0e3d000700 1 --2- 172.21.15.18:0/283211875 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0e38101b30 0x7f0e38113980 unknown :-1 s=CLOSED pgs=1273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:50:00.301 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:50:00.298+0000 7f0e3d000700 1 --2- 172.21.15.18:0/283211875 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0e381024d0 0x7f0e38113ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:50:00.301 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:50:00.298+0000 7f0e3d000700 1 -- 172.21.15.18:0/283211875 >> 172.21.15.18:0/283211875 conn(0x7f0e380f76a0 msgr2=0x7f0e38066790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:50:00.301 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:50:00.298+0000 7f0e3d000700 1 -- 172.21.15.18:0/283211875 shutdown_connections 2024-04-12T21:50:00.301 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:50:00.298+0000 7f0e3d000700 1 -- 172.21.15.18:0/283211875 wait complete. 2024-04-12T21:50:00.450 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:50:00.448+0000 7fc572600700 1 -- 172.21.15.18:0/2884316357 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc56c070c70 msgr2=0x7fc56c109110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:50:00.450 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:50:00.448+0000 7fc572600700 1 --2- 172.21.15.18:0/2884316357 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc56c070c70 0x7fc56c109110 secure :-1 s=READY pgs=1274 cs=0 l=1 rev1=1 crypto rx=0x7fc5540048b0 tx=0x7fc554009e30 comp rx=0 tx=0).stop 2024-04-12T21:50:00.451 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:50:00.448+0000 7fc572600700 1 -- 172.21.15.18:0/2884316357 shutdown_connections 2024-04-12T21:50:00.451 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:50:00.448+0000 7fc572600700 1 --2- 172.21.15.18:0/2884316357 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc56c070c70 0x7fc56c109110 unknown :-1 s=CLOSED pgs=1274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:50:00.451 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:50:00.448+0000 7fc572600700 1 --2- 172.21.15.18:0/2884316357 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc56c070360 0x7fc56c070730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:50:00.451 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:50:00.448+0000 7fc572600700 1 -- 172.21.15.18:0/2884316357 >> 172.21.15.18:0/2884316357 conn(0x7fc56c0f89b0 msgr2=0x7fc56c0fadc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:50:00.451 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:50:00.448+0000 7fc572600700 1 -- 172.21.15.18:0/2884316357 shutdown_connections 2024-04-12T21:50:00.451 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:50:00.448+0000 7fc572600700 1 -- 172.21.15.18:0/2884316357 wait complete. 2024-04-12T21:50:00.452 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:50:00.449+0000 7fc572600700 1 Processor -- start 2024-04-12T21:50:00.452 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:50:00.450+0000 7fc572600700 1 -- start start 2024-04-12T21:50:00.452 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:50:00.450+0000 7fc572600700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc56c070360 0x7fc56c115d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:50:00.452 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:50:00.450+0000 7fc572600700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc56c070c70 0x7fc56c1162b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:50:00.452 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:50:00.450+0000 7fc572600700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc56c116900 con 0x7fc56c070c70 2024-04-12T21:50:00.452 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:50:00.450+0000 7fc572600700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc56c10fcd0 con 0x7fc56c070360 2024-04-12T21:50:00.453 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:50:00.450+0000 7fc562a00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc56c070c70 0x7fc56c1162b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:50:00.453 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:50:00.450+0000 7fc562a00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc56c070c70 0x7fc56c1162b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:59122/0 (socket says 172.21.15.18:59122) 2024-04-12T21:50:00.453 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:50:00.450+0000 7fc562a00700 1 -- 172.21.15.18:0/2049026312 learned_addr learned my addr 172.21.15.18:0/2049026312 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:50:00.454 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:50:00.451+0000 7fc56b400700 1 --2- 172.21.15.18:0/2049026312 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc56c070360 0x7fc56c115d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:50:00.454 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:50:00.451+0000 7fc562a00700 1 -- 172.21.15.18:0/2049026312 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc56c070360 msgr2=0x7fc56c115d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:50:00.454 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:50:00.451+0000 7fc562a00700 1 --2- 172.21.15.18:0/2049026312 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc56c070360 0x7fc56c115d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:50:00.454 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:50:00.451+0000 7fc562a00700 1 -- 172.21.15.18:0/2049026312 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc554005040 con 0x7fc56c070c70 2024-04-12T21:50:00.454 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:50:00.451+0000 7fc562a00700 1 --2- 172.21.15.18:0/2049026312 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc56c070c70 0x7fc56c1162b0 secure :-1 s=READY pgs=1275 cs=0 l=1 rev1=1 crypto rx=0x7fc554000c00 tx=0x7fc554009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:50:00.454 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:50:00.451+0000 7fc563e00700 1 -- 172.21.15.18:0/2049026312 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc55400e040 con 0x7fc56c070c70 2024-04-12T21:50:00.454 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:50:00.451+0000 7fc563e00700 1 -- 172.21.15.18:0/2049026312 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc55401c900 con 0x7fc56c070c70 2024-04-12T21:50:00.454 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:50:00.452+0000 7fc572600700 1 -- 172.21.15.18:0/2049026312 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc56c10ff50 con 0x7fc56c070c70 2024-04-12T21:50:00.454 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:50:00.451+0000 7fc563e00700 1 -- 172.21.15.18:0/2049026312 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc554012810 con 0x7fc56c070c70 2024-04-12T21:50:00.454 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:50:00.452+0000 7fc572600700 1 -- 172.21.15.18:0/2049026312 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc56c110430 con 0x7fc56c070c70 2024-04-12T21:50:00.455 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:50:00.453+0000 7fc572600700 1 -- 172.21.15.18:0/2049026312 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc56c071aa0 con 0x7fc56c070c70 2024-04-12T21:50:00.455 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:50:00.454+0000 7fc563e00700 1 -- 172.21.15.18:0/2049026312 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7fc55401c430 con 0x7fc56c070c70 2024-04-12T21:50:00.457 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:50:00.455+0000 7fc563e00700 1 --2- 172.21.15.18:0/2049026312 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fc55006ec10 0x7fc5500710c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:50:00.457 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:50:00.455+0000 7fc563e00700 1 -- 172.21.15.18:0/2049026312 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 6757+0+0 (secure 0 0 0) 0x7fc55408ffe0 con 0x7fc56c070c70 2024-04-12T21:50:00.457 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:50:00.455+0000 7fc56b400700 1 --2- 172.21.15.18:0/2049026312 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fc55006ec10 0x7fc5500710c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:50:00.458 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:50:00.456+0000 7fc56b400700 1 --2- 172.21.15.18:0/2049026312 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fc55006ec10 0x7fc5500710c0 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7fc55c004750 tx=0x7fc55c00b040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:50:00.462 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:50:00.460+0000 7fc563e00700 1 -- 172.21.15.18:0/2049026312 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7fc5540c4ff0 con 0x7fc56c070c70 2024-04-12T21:50:00.783 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:50:00.781+0000 7fc572600700 1 -- 172.21.15.18:0/2049026312 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fc56c04bed0 con 0x7fc56c070c70 2024-04-12T21:50:00.784 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:50:00.782+0000 7fc563e00700 1 -- 172.21.15.18:0/2049026312 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+623 (secure 0 0 0) 0x7fc55405c830 con 0x7fc56c070c70 2024-04-12T21:50:00.785 INFO:teuthology.orchestra.run.smithi018.stdout:{ 2024-04-12T21:50:00.785 INFO:teuthology.orchestra.run.smithi018.stdout: "mon": { 2024-04-12T21:50:00.785 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 19.0.0-2879-g324baf8d (324baf8ddb91dea46cac50cb535c306115791355) squid (dev)": 2 2024-04-12T21:50:00.785 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-12T21:50:00.785 INFO:teuthology.orchestra.run.smithi018.stdout: "mgr": { 2024-04-12T21:50:00.785 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 19.0.0-2879-g324baf8d (324baf8ddb91dea46cac50cb535c306115791355) squid (dev)": 2 2024-04-12T21:50:00.785 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-12T21:50:00.785 INFO:teuthology.orchestra.run.smithi018.stdout: "osd": { 2024-04-12T21:50:00.785 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 19.0.0-2879-g324baf8d (324baf8ddb91dea46cac50cb535c306115791355) squid (dev)": 6 2024-04-12T21:50:00.785 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-12T21:50:00.785 INFO:teuthology.orchestra.run.smithi018.stdout: "mds": { 2024-04-12T21:50:00.786 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 19.0.0-2879-g324baf8d (324baf8ddb91dea46cac50cb535c306115791355) squid (dev)": 4 2024-04-12T21:50:00.786 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-12T21:50:00.786 INFO:teuthology.orchestra.run.smithi018.stdout: "overall": { 2024-04-12T21:50:00.786 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 19.0.0-2879-g324baf8d (324baf8ddb91dea46cac50cb535c306115791355) squid (dev)": 14 2024-04-12T21:50:00.786 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-04-12T21:50:00.786 INFO:teuthology.orchestra.run.smithi018.stdout:} 2024-04-12T21:50:00.788 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:50:00.786+0000 7fc572600700 1 -- 172.21.15.18:0/2049026312 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fc55006ec10 msgr2=0x7fc5500710c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:50:00.788 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:50:00.786+0000 7fc572600700 1 --2- 172.21.15.18:0/2049026312 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fc55006ec10 0x7fc5500710c0 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7fc55c004750 tx=0x7fc55c00b040 comp rx=0 tx=0).stop 2024-04-12T21:50:00.788 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:50:00.786+0000 7fc572600700 1 -- 172.21.15.18:0/2049026312 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc56c070c70 msgr2=0x7fc56c1162b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:50:00.788 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:50:00.786+0000 7fc572600700 1 --2- 172.21.15.18:0/2049026312 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc56c070c70 0x7fc56c1162b0 secure :-1 s=READY pgs=1275 cs=0 l=1 rev1=1 crypto rx=0x7fc554000c00 tx=0x7fc554009e30 comp rx=0 tx=0).stop 2024-04-12T21:50:00.789 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:50:00.786+0000 7fc572600700 1 -- 172.21.15.18:0/2049026312 shutdown_connections 2024-04-12T21:50:00.789 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:50:00.786+0000 7fc572600700 1 --2- 172.21.15.18:0/2049026312 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fc55006ec10 0x7fc5500710c0 unknown :-1 s=CLOSED pgs=676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:50:00.789 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:50:00.786+0000 7fc572600700 1 --2- 172.21.15.18:0/2049026312 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc56c070360 0x7fc56c115d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:50:00.789 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:50:00.786+0000 7fc572600700 1 --2- 172.21.15.18:0/2049026312 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc56c070c70 0x7fc56c1162b0 unknown :-1 s=CLOSED pgs=1275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:50:00.789 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:50:00.786+0000 7fc572600700 1 -- 172.21.15.18:0/2049026312 >> 172.21.15.18:0/2049026312 conn(0x7fc56c0f89b0 msgr2=0x7fc56c0f9e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:50:00.789 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:50:00.786+0000 7fc572600700 1 -- 172.21.15.18:0/2049026312 shutdown_connections 2024-04-12T21:50:00.789 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:50:00.786+0000 7fc572600700 1 -- 172.21.15.18:0/2049026312 wait complete. 2024-04-12T21:50:00.946 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:50:00.943+0000 7fb353400700 1 -- 172.21.15.18:0/327076794 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb34c109430 msgr2=0x7fb34c109800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:50:00.946 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:50:00.943+0000 7fb353400700 1 --2- 172.21.15.18:0/327076794 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb34c109430 0x7fb34c109800 secure :-1 s=READY pgs=1276 cs=0 l=1 rev1=1 crypto rx=0x7fb340004940 tx=0x7fb340009e30 comp rx=0 tx=0).stop 2024-04-12T21:50:00.946 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:50:00.944+0000 7fb353400700 1 -- 172.21.15.18:0/327076794 shutdown_connections 2024-04-12T21:50:00.946 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:50:00.944+0000 7fb353400700 1 --2- 172.21.15.18:0/327076794 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb34c109dd0 0x7fb34c103440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:50:00.946 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:50:00.944+0000 7fb353400700 1 --2- 172.21.15.18:0/327076794 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb34c109430 0x7fb34c109800 unknown :-1 s=CLOSED pgs=1276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:50:00.946 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:50:00.944+0000 7fb353400700 1 -- 172.21.15.18:0/327076794 >> 172.21.15.18:0/327076794 conn(0x7fb34c072c00 msgr2=0x7fb34c075010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:50:00.946 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:50:00.944+0000 7fb353400700 1 -- 172.21.15.18:0/327076794 shutdown_connections 2024-04-12T21:50:00.946 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:50:00.944+0000 7fb353400700 1 -- 172.21.15.18:0/327076794 wait complete. 2024-04-12T21:50:00.948 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:50:00.945+0000 7fb353400700 1 Processor -- start 2024-04-12T21:50:00.948 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:50:00.945+0000 7fb353400700 1 -- start start 2024-04-12T21:50:00.948 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:50:00.945+0000 7fb353400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb34c109430 0x7fb34c119f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:50:00.948 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:50:00.945+0000 7fb353400700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb34c109dd0 0x7fb34c11a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:50:00.948 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:50:00.945+0000 7fb353400700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb34c11ab60 con 0x7fb34c109430 2024-04-12T21:50:00.948 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:50:00.945+0000 7fb353400700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb34c1140b0 con 0x7fb34c109dd0 2024-04-12T21:50:00.948 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:50:00.946+0000 7fb350a00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb34c109430 0x7fb34c119f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:50:00.948 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:50:00.946+0000 7fb350a00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb34c109430 0x7fb34c119f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:57138/0 (socket says 172.21.15.18:57138) 2024-04-12T21:50:00.948 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:50:00.946+0000 7fb350a00700 1 -- 172.21.15.18:0/3820924420 learned_addr learned my addr 172.21.15.18:0/3820924420 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:50:00.948 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:50:00.946+0000 7fb350a00700 1 -- 172.21.15.18:0/3820924420 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb34c109dd0 msgr2=0x7fb34c11a480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:50:00.948 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:50:00.946+0000 7fb350a00700 1 --2- 172.21.15.18:0/3820924420 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb34c109dd0 0x7fb34c11a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:50:00.948 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:50:00.946+0000 7fb350a00700 1 -- 172.21.15.18:0/3820924420 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb340005040 con 0x7fb34c109430 2024-04-12T21:50:00.949 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:50:00.946+0000 7fb350a00700 1 --2- 172.21.15.18:0/3820924420 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb34c109430 0x7fb34c119f40 secure :-1 s=READY pgs=1277 cs=0 l=1 rev1=1 crypto rx=0x7fb340009180 tx=0x7fb340009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:50:00.949 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:50:00.947+0000 7fb348c00700 1 -- 172.21.15.18:0/3820924420 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb34000e040 con 0x7fb34c109430 2024-04-12T21:50:00.949 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:50:00.947+0000 7fb348c00700 1 -- 172.21.15.18:0/3820924420 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb34000bbc0 con 0x7fb34c109430 2024-04-12T21:50:00.949 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:50:00.947+0000 7fb348c00700 1 -- 172.21.15.18:0/3820924420 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb34001c630 con 0x7fb34c109430 2024-04-12T21:50:00.950 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:50:00.947+0000 7fb353400700 1 -- 172.21.15.18:0/3820924420 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb34c1142d0 con 0x7fb34c109430 2024-04-12T21:50:00.950 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:50:00.947+0000 7fb353400700 1 -- 172.21.15.18:0/3820924420 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb34c114890 con 0x7fb34c109430 2024-04-12T21:50:00.950 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:50:00.948+0000 7fb353400700 1 -- 172.21.15.18:0/3820924420 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb34c104b80 con 0x7fb34c109430 2024-04-12T21:50:00.951 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:50:00.949+0000 7fb348c00700 1 -- 172.21.15.18:0/3820924420 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7fb34000bd30 con 0x7fb34c109430 2024-04-12T21:50:00.952 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:50:00.950+0000 7fb348c00700 1 --2- 172.21.15.18:0/3820924420 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fb33406ec00 0x7fb3340710b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:50:00.952 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:50:00.950+0000 7fb348c00700 1 -- 172.21.15.18:0/3820924420 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 6757+0+0 (secure 0 0 0) 0x7fb34008fd40 con 0x7fb34c109430 2024-04-12T21:50:00.952 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:50:00.950+0000 7fb34be00700 1 --2- 172.21.15.18:0/3820924420 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fb33406ec00 0x7fb3340710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:50:00.952 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:50:00.951+0000 7fb34be00700 1 --2- 172.21.15.18:0/3820924420 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fb33406ec00 0x7fb3340710b0 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7fb338006fd0 tx=0x7fb338006db0 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:50:00.957 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:50:00.955+0000 7fb348c00700 1 -- 172.21.15.18:0/3820924420 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7fb34005c570 con 0x7fb34c109430 2024-04-12T21:50:01.106 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:50:00 smithi018 ceph-mon[112590]: from='client.36608 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:50:01.106 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:50:00 smithi018 ceph-mon[112590]: from='client.36612 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:50:01.106 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:50:00 smithi018 ceph-mon[112590]: overall HEALTH_ERR failed to probe daemons or devices; 1 filesystem is degraded; 1 filesystem is offline 2024-04-12T21:50:01.106 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:50:00 smithi018 ceph-mon[112590]: from='client.? 172.21.15.18:0/2049026312' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T21:50:01.234 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:50:01.232+0000 7fb353400700 1 -- 172.21.15.18:0/3820924420 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fb34c04c710 con 0x7fb34c109430 2024-04-12T21:50:01.234 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:50:01.232+0000 7fb348c00700 1 -- 172.21.15.18:0/3820924420 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 33 v33) v1 ==== 76+0+1876 (secure 0 0 0) 0x7fb34005c100 con 0x7fb34c109430 2024-04-12T21:50:01.235 INFO:teuthology.orchestra.run.smithi018.stdout:e33 2024-04-12T21:50:01.235 INFO:teuthology.orchestra.run.smithi018.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-12T21:50:01.235 INFO:teuthology.orchestra.run.smithi018.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-12T21:50:01.235 INFO:teuthology.orchestra.run.smithi018.stdout:legacy client fscid: 1 2024-04-12T21:50:01.235 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-12T21:50:01.235 INFO:teuthology.orchestra.run.smithi018.stdout:Filesystem 'cephfs' (1) 2024-04-12T21:50:01.236 INFO:teuthology.orchestra.run.smithi018.stdout:fs_name cephfs 2024-04-12T21:50:01.236 INFO:teuthology.orchestra.run.smithi018.stdout:epoch 24 2024-04-12T21:50:01.236 INFO:teuthology.orchestra.run.smithi018.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-04-12T21:50:01.236 INFO:teuthology.orchestra.run.smithi018.stdout:created 2024-04-12T20:51:57.362850+0000 2024-04-12T21:50:01.236 INFO:teuthology.orchestra.run.smithi018.stdout:modified 2024-04-12T21:06:59.717190+0000 2024-04-12T21:50:01.236 INFO:teuthology.orchestra.run.smithi018.stdout:tableserver 0 2024-04-12T21:50:01.236 INFO:teuthology.orchestra.run.smithi018.stdout:root 0 2024-04-12T21:50:01.236 INFO:teuthology.orchestra.run.smithi018.stdout:session_timeout 60 2024-04-12T21:50:01.236 INFO:teuthology.orchestra.run.smithi018.stdout:session_autoclose 300 2024-04-12T21:50:01.236 INFO:teuthology.orchestra.run.smithi018.stdout:max_file_size 1099511627776 2024-04-12T21:50:01.236 INFO:teuthology.orchestra.run.smithi018.stdout:max_xattr_size 65536 2024-04-12T21:50:01.236 INFO:teuthology.orchestra.run.smithi018.stdout:required_client_features {} 2024-04-12T21:50:01.236 INFO:teuthology.orchestra.run.smithi018.stdout:last_failure 0 2024-04-12T21:50:01.236 INFO:teuthology.orchestra.run.smithi018.stdout:last_failure_osd_epoch 100 2024-04-12T21:50:01.236 INFO:teuthology.orchestra.run.smithi018.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-12T21:50:01.237 INFO:teuthology.orchestra.run.smithi018.stdout:max_mds 2 2024-04-12T21:50:01.237 INFO:teuthology.orchestra.run.smithi018.stdout:in 0,1 2024-04-12T21:50:01.237 INFO:teuthology.orchestra.run.smithi018.stdout:up {} 2024-04-12T21:50:01.237 INFO:teuthology.orchestra.run.smithi018.stdout:failed 0,1 2024-04-12T21:50:01.237 INFO:teuthology.orchestra.run.smithi018.stdout:damaged 2024-04-12T21:50:01.237 INFO:teuthology.orchestra.run.smithi018.stdout:stopped 2024-04-12T21:50:01.237 INFO:teuthology.orchestra.run.smithi018.stdout:data_pools [3] 2024-04-12T21:50:01.237 INFO:teuthology.orchestra.run.smithi018.stdout:metadata_pool 2 2024-04-12T21:50:01.237 INFO:teuthology.orchestra.run.smithi018.stdout:inline_data disabled 2024-04-12T21:50:01.237 INFO:teuthology.orchestra.run.smithi018.stdout:balancer 2024-04-12T21:50:01.237 INFO:teuthology.orchestra.run.smithi018.stdout:bal_rank_mask -1 2024-04-12T21:50:01.237 INFO:teuthology.orchestra.run.smithi018.stdout:standby_count_wanted 1 2024-04-12T21:50:01.237 INFO:teuthology.orchestra.run.smithi018.stdout:qdb_cluster leader: 0 members: 2024-04-12T21:50:01.237 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-12T21:50:01.237 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-12T21:50:01.237 INFO:teuthology.orchestra.run.smithi018.stdout:Standby daemons: 2024-04-12T21:50:01.238 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-12T21:50:01.238 INFO:teuthology.orchestra.run.smithi018.stdout:[mds.cephfs.smithi018.acqbsc{-1:34436} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.18:6828/755179764,v1:172.21.15.18:6829/755179764] compat {c=[1],r=[1],i=[1fff]}] 2024-04-12T21:50:01.238 INFO:teuthology.orchestra.run.smithi018.stdout:[mds.cephfs.smithi018.szjluo{-1:34468} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.18:6826/2407992653,v1:172.21.15.18:6827/2407992653] compat {c=[1],r=[1],i=[1fff]}] 2024-04-12T21:50:01.238 INFO:teuthology.orchestra.run.smithi018.stdout:[mds.cephfs.smithi186.jdllyn{-1:44595} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.186:6824/3523444202,v1:172.21.15.186:6825/3523444202] compat {c=[1],r=[1],i=[1fff]}] 2024-04-12T21:50:01.238 INFO:teuthology.orchestra.run.smithi018.stdout:[mds.cephfs.smithi186.wjnafs{-1:44875} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.186:6826/3122443766,v1:172.21.15.186:6827/3122443766] compat {c=[1],r=[1],i=[1fff]}] 2024-04-12T21:50:01.239 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:50:01.235+0000 7fb353400700 1 -- 172.21.15.18:0/3820924420 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fb33406ec00 msgr2=0x7fb3340710b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:50:01.239 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:50:01.235+0000 7fb353400700 1 --2- 172.21.15.18:0/3820924420 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fb33406ec00 0x7fb3340710b0 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7fb338006fd0 tx=0x7fb338006db0 comp rx=0 tx=0).stop 2024-04-12T21:50:01.239 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:50:01.235+0000 7fb353400700 1 -- 172.21.15.18:0/3820924420 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb34c109430 msgr2=0x7fb34c119f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:50:01.239 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:50:01.235+0000 7fb353400700 1 --2- 172.21.15.18:0/3820924420 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb34c109430 0x7fb34c119f40 secure :-1 s=READY pgs=1277 cs=0 l=1 rev1=1 crypto rx=0x7fb340009180 tx=0x7fb340009e30 comp rx=0 tx=0).stop 2024-04-12T21:50:01.239 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:50:01.235+0000 7fb353400700 1 -- 172.21.15.18:0/3820924420 shutdown_connections 2024-04-12T21:50:01.239 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:50:01.235+0000 7fb353400700 1 --2- 172.21.15.18:0/3820924420 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fb33406ec00 0x7fb3340710b0 unknown :-1 s=CLOSED pgs=677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:50:01.239 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:50:01.235+0000 7fb353400700 1 --2- 172.21.15.18:0/3820924420 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb34c109430 0x7fb34c119f40 unknown :-1 s=CLOSED pgs=1277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:50:01.239 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:50:01.235+0000 7fb353400700 1 --2- 172.21.15.18:0/3820924420 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb34c109dd0 0x7fb34c11a480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:50:01.239 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:50:01.235+0000 7fb353400700 1 -- 172.21.15.18:0/3820924420 >> 172.21.15.18:0/3820924420 conn(0x7fb34c072c00 msgr2=0x7fb34c074c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:50:01.239 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:50:01.236+0000 7fb353400700 1 -- 172.21.15.18:0/3820924420 shutdown_connections 2024-04-12T21:50:01.239 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:50:01.236+0000 7fb353400700 1 -- 172.21.15.18:0/3820924420 wait complete. 2024-04-12T21:50:01.239 INFO:teuthology.orchestra.run.smithi018.stderr:dumped fsmap epoch 33 2024-04-12T21:50:01.284 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:50:00 smithi186 ceph-mon[86379]: from='client.36608 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:50:01.284 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:50:00 smithi186 ceph-mon[86379]: from='client.36612 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:50:01.284 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:50:00 smithi186 ceph-mon[86379]: overall HEALTH_ERR failed to probe daemons or devices; 1 filesystem is degraded; 1 filesystem is offline 2024-04-12T21:50:01.285 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:50:00 smithi186 ceph-mon[86379]: from='client.? 172.21.15.18:0/2049026312' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T21:50:01.382 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:50:01.380+0000 7fcb01000700 1 -- 172.21.15.18:0/3654394042 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fcafc109430 msgr2=0x7fcafc109800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:50:01.383 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:50:01.380+0000 7fcb01000700 1 --2- 172.21.15.18:0/3654394042 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fcafc109430 0x7fcafc109800 secure :-1 s=READY pgs=1278 cs=0 l=1 rev1=1 crypto rx=0x7fcaec004940 tx=0x7fcaec009e30 comp rx=0 tx=0).stop 2024-04-12T21:50:01.383 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:50:01.381+0000 7fcb01000700 1 -- 172.21.15.18:0/3654394042 shutdown_connections 2024-04-12T21:50:01.383 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:50:01.381+0000 7fcb01000700 1 --2- 172.21.15.18:0/3654394042 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcafc109dd0 0x7fcafc103440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:50:01.383 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:50:01.381+0000 7fcb01000700 1 --2- 172.21.15.18:0/3654394042 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fcafc109430 0x7fcafc109800 unknown :-1 s=CLOSED pgs=1278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:50:01.383 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:50:01.381+0000 7fcb01000700 1 -- 172.21.15.18:0/3654394042 >> 172.21.15.18:0/3654394042 conn(0x7fcafc072c00 msgr2=0x7fcafc075010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:50:01.387 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:50:01.381+0000 7fcb01000700 1 -- 172.21.15.18:0/3654394042 shutdown_connections 2024-04-12T21:50:01.387 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:50:01.385+0000 7fcb01000700 1 -- 172.21.15.18:0/3654394042 wait complete. 2024-04-12T21:50:01.387 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:50:01.386+0000 7fcb01000700 1 Processor -- start 2024-04-12T21:50:01.388 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:50:01.386+0000 7fcb01000700 1 -- start start 2024-04-12T21:50:01.388 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:50:01.386+0000 7fcb01000700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcafc109430 0x7fcafc119f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:50:01.388 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:50:01.386+0000 7fcb01000700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fcafc109dd0 0x7fcafc11a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:50:01.388 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:50:01.387+0000 7fcb01000700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcafc11aad0 con 0x7fcafc109dd0 2024-04-12T21:50:01.388 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:50:01.387+0000 7fcb01000700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcafc114100 con 0x7fcafc109430 2024-04-12T21:50:01.389 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:50:01.387+0000 7fcaf9400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fcafc109dd0 0x7fcafc11a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:50:01.389 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:50:01.387+0000 7fcaf9400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fcafc109dd0 0x7fcafc11a480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:57142/0 (socket says 172.21.15.18:57142) 2024-04-12T21:50:01.389 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:50:01.387+0000 7fcaf9400700 1 -- 172.21.15.18:0/571668362 learned_addr learned my addr 172.21.15.18:0/571668362 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:50:01.389 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:50:01.387+0000 7fcaf9e00700 1 --2- 172.21.15.18:0/571668362 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcafc109430 0x7fcafc119f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:50:01.389 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:50:01.388+0000 7fcaf9400700 1 -- 172.21.15.18:0/571668362 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcafc109430 msgr2=0x7fcafc119f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:50:01.390 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:50:01.388+0000 7fcaf9400700 1 --2- 172.21.15.18:0/571668362 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcafc109430 0x7fcafc119f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:50:01.390 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:50:01.388+0000 7fcaf9400700 1 -- 172.21.15.18:0/571668362 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcaec005040 con 0x7fcafc109dd0 2024-04-12T21:50:01.390 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:50:01.388+0000 7fcaf9400700 1 --2- 172.21.15.18:0/571668362 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fcafc109dd0 0x7fcafc11a480 secure :-1 s=READY pgs=1279 cs=0 l=1 rev1=1 crypto rx=0x7fcae8002a50 tx=0x7fcae8004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:50:01.390 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:50:01.388+0000 7fcaf2000700 1 -- 172.21.15.18:0/571668362 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcae8007cc0 con 0x7fcafc109dd0 2024-04-12T21:50:01.391 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:50:01.388+0000 7fcaf2000700 1 -- 172.21.15.18:0/571668362 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fcae800f040 con 0x7fcafc109dd0 2024-04-12T21:50:01.391 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:50:01.388+0000 7fcaf2000700 1 -- 172.21.15.18:0/571668362 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcae800d9c0 con 0x7fcafc109dd0 2024-04-12T21:50:01.391 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:50:01.388+0000 7fcb01000700 1 -- 172.21.15.18:0/571668362 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcafc1143e0 con 0x7fcafc109dd0 2024-04-12T21:50:01.391 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:50:01.388+0000 7fcb01000700 1 -- 172.21.15.18:0/571668362 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcafc1149a0 con 0x7fcafc109dd0 2024-04-12T21:50:01.391 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:50:01.390+0000 7fcb01000700 1 -- 172.21.15.18:0/571668362 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcafc104b80 con 0x7fcafc109dd0 2024-04-12T21:50:01.392 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:50:01.391+0000 7fcaf2000700 1 -- 172.21.15.18:0/571668362 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7fcae8009730 con 0x7fcafc109dd0 2024-04-12T21:50:01.394 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:50:01.392+0000 7fcaf2000700 1 --2- 172.21.15.18:0/571668362 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fcae006ec00 0x7fcae00710b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:50:01.394 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:50:01.392+0000 7fcaf2000700 1 -- 172.21.15.18:0/571668362 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 6757+0+0 (secure 0 0 0) 0x7fcae808e630 con 0x7fcafc109dd0 2024-04-12T21:50:01.394 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:50:01.392+0000 7fcaf9e00700 1 --2- 172.21.15.18:0/571668362 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fcae006ec00 0x7fcae00710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:50:01.394 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:50:01.393+0000 7fcaf9e00700 1 --2- 172.21.15.18:0/571668362 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fcae006ec00 0x7fcae00710b0 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7fcaec009fd0 tx=0x7fcaec009e30 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:50:01.398 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:50:01.397+0000 7fcaf2000700 1 -- 172.21.15.18:0/571668362 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7fcae805ade0 con 0x7fcafc109dd0 2024-04-12T21:50:01.644 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:50:01.642+0000 7fcb01000700 1 -- 172.21.15.18:0/571668362 --> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fcafc110590 con 0x7fcae006ec00 2024-04-12T21:50:01.645 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:50:01.643+0000 7fcaf2000700 1 -- 172.21.15.18:0/571668362 <== mgr.34102 v2:172.21.15.18:6800/3812891919 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7fcafc110590 con 0x7fcae006ec00 2024-04-12T21:50:01.646 INFO:teuthology.orchestra.run.smithi018.stdout:{ 2024-04-12T21:50:01.646 INFO:teuthology.orchestra.run.smithi018.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:f4452def3be4d350f4a8e5feaf7ebb1511be9298fe68478ba7a2d6fbd7d69a63", 2024-04-12T21:50:01.646 INFO:teuthology.orchestra.run.smithi018.stdout: "in_progress": true, 2024-04-12T21:50:01.646 INFO:teuthology.orchestra.run.smithi018.stdout: "which": "Upgrading all daemon types on all hosts", 2024-04-12T21:50:01.646 INFO:teuthology.orchestra.run.smithi018.stdout: "services_complete": [ 2024-04-12T21:50:01.646 INFO:teuthology.orchestra.run.smithi018.stdout: "mgr", 2024-04-12T21:50:01.646 INFO:teuthology.orchestra.run.smithi018.stdout: "mds", 2024-04-12T21:50:01.646 INFO:teuthology.orchestra.run.smithi018.stdout: "crash", 2024-04-12T21:50:01.646 INFO:teuthology.orchestra.run.smithi018.stdout: "osd", 2024-04-12T21:50:01.646 INFO:teuthology.orchestra.run.smithi018.stdout: "mon" 2024-04-12T21:50:01.646 INFO:teuthology.orchestra.run.smithi018.stdout: ], 2024-04-12T21:50:01.646 INFO:teuthology.orchestra.run.smithi018.stdout: "progress": "16/21 daemons upgraded", 2024-04-12T21:50:01.647 INFO:teuthology.orchestra.run.smithi018.stdout: "message": "Currently upgrading mds daemons", 2024-04-12T21:50:01.647 INFO:teuthology.orchestra.run.smithi018.stdout: "is_paused": false 2024-04-12T21:50:01.647 INFO:teuthology.orchestra.run.smithi018.stdout:} 2024-04-12T21:50:01.649 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:50:01.646+0000 7fcb01000700 1 -- 172.21.15.18:0/571668362 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fcae006ec00 msgr2=0x7fcae00710b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:50:01.649 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:50:01.646+0000 7fcb01000700 1 --2- 172.21.15.18:0/571668362 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fcae006ec00 0x7fcae00710b0 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7fcaec009fd0 tx=0x7fcaec009e30 comp rx=0 tx=0).stop 2024-04-12T21:50:01.649 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:50:01.646+0000 7fcb01000700 1 -- 172.21.15.18:0/571668362 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fcafc109dd0 msgr2=0x7fcafc11a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:50:01.649 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:50:01.646+0000 7fcb01000700 1 --2- 172.21.15.18:0/571668362 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fcafc109dd0 0x7fcafc11a480 secure :-1 s=READY pgs=1279 cs=0 l=1 rev1=1 crypto rx=0x7fcae8002a50 tx=0x7fcae8004600 comp rx=0 tx=0).stop 2024-04-12T21:50:01.649 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:50:01.647+0000 7fcb01000700 1 -- 172.21.15.18:0/571668362 shutdown_connections 2024-04-12T21:50:01.649 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:50:01.647+0000 7fcb01000700 1 --2- 172.21.15.18:0/571668362 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fcae006ec00 0x7fcae00710b0 unknown :-1 s=CLOSED pgs=678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:50:01.649 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:50:01.647+0000 7fcb01000700 1 --2- 172.21.15.18:0/571668362 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcafc109430 0x7fcafc119f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:50:01.649 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:50:01.647+0000 7fcb01000700 1 --2- 172.21.15.18:0/571668362 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fcafc109dd0 0x7fcafc11a480 unknown :-1 s=CLOSED pgs=1279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:50:01.649 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:50:01.647+0000 7fcb01000700 1 -- 172.21.15.18:0/571668362 >> 172.21.15.18:0/571668362 conn(0x7fcafc072c00 msgr2=0x7fcafc074c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:50:01.649 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:50:01.647+0000 7fcb01000700 1 -- 172.21.15.18:0/571668362 shutdown_connections 2024-04-12T21:50:01.650 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:50:01.647+0000 7fcb01000700 1 -- 172.21.15.18:0/571668362 wait complete. 2024-04-12T21:50:01.806 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:50:01.804+0000 7f55a7200700 1 -- 172.21.15.18:0/3053056669 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f55a0070c70 msgr2=0x7f55a0109110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:50:01.807 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:50:01.804+0000 7f55a7200700 1 --2- 172.21.15.18:0/3053056669 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f55a0070c70 0x7f55a0109110 secure :-1 s=READY pgs=1280 cs=0 l=1 rev1=1 crypto rx=0x7f5588004970 tx=0x7f5588009e30 comp rx=0 tx=0).stop 2024-04-12T21:50:01.807 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:50:01.804+0000 7f55a7200700 1 -- 172.21.15.18:0/3053056669 shutdown_connections 2024-04-12T21:50:01.807 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:50:01.804+0000 7f55a7200700 1 --2- 172.21.15.18:0/3053056669 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f55a0070c70 0x7f55a0109110 unknown :-1 s=CLOSED pgs=1280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:50:01.807 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:50:01.804+0000 7f55a7200700 1 --2- 172.21.15.18:0/3053056669 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f55a0070360 0x7f55a0070730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:50:01.807 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:50:01.804+0000 7f55a7200700 1 -- 172.21.15.18:0/3053056669 >> 172.21.15.18:0/3053056669 conn(0x7f55a00f89b0 msgr2=0x7f55a00fadc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:50:01.807 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:50:01.804+0000 7f55a7200700 1 -- 172.21.15.18:0/3053056669 shutdown_connections 2024-04-12T21:50:01.807 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:50:01.804+0000 7f55a7200700 1 -- 172.21.15.18:0/3053056669 wait complete. 2024-04-12T21:50:01.808 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:50:01.805+0000 7f55a7200700 1 Processor -- start 2024-04-12T21:50:01.808 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:50:01.805+0000 7f55a7200700 1 -- start start 2024-04-12T21:50:01.809 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:50:01.806+0000 7f55a7200700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f55a0070360 0x7f55a0115d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:50:01.809 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:50:01.806+0000 7f55a7200700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f55a0070c70 0x7f55a01162b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:50:01.809 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:50:01.806+0000 7f55a7200700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55a0116900 con 0x7f55a0070c70 2024-04-12T21:50:01.809 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:50:01.806+0000 7f55a7200700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55a010fcd0 con 0x7f55a0070360 2024-04-12T21:50:01.809 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:50:01.806+0000 7f5597e00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f55a0070c70 0x7f55a01162b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:50:01.809 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:50:01.806+0000 7f5597e00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f55a0070c70 0x7f55a01162b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:57158/0 (socket says 172.21.15.18:57158) 2024-04-12T21:50:01.809 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:50:01.806+0000 7f5597e00700 1 -- 172.21.15.18:0/1605848264 learned_addr learned my addr 172.21.15.18:0/1605848264 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:50:01.809 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:50:01.806+0000 7f559fe00700 1 --2- 172.21.15.18:0/1605848264 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f55a0070360 0x7f55a0115d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:50:01.809 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:50:01.806+0000 7f5597e00700 1 -- 172.21.15.18:0/1605848264 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f55a0070360 msgr2=0x7f55a0115d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:50:01.810 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:50:01.806+0000 7f5597e00700 1 --2- 172.21.15.18:0/1605848264 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f55a0070360 0x7f55a0115d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:50:01.810 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:50:01.806+0000 7f5597e00700 1 -- 172.21.15.18:0/1605848264 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5588005040 con 0x7f55a0070c70 2024-04-12T21:50:01.810 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:50:01.807+0000 7f5597e00700 1 --2- 172.21.15.18:0/1605848264 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f55a0070c70 0x7f55a01162b0 secure :-1 s=READY pgs=1281 cs=0 l=1 rev1=1 crypto rx=0x7f5588000c00 tx=0x7f5588009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:50:01.810 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:50:01.807+0000 7f559cc00700 1 -- 172.21.15.18:0/1605848264 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f558800e040 con 0x7f55a0070c70 2024-04-12T21:50:01.811 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:50:01.807+0000 7f559cc00700 1 -- 172.21.15.18:0/1605848264 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f558800bc10 con 0x7f55a0070c70 2024-04-12T21:50:01.811 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:50:01.807+0000 7f55a7200700 1 -- 172.21.15.18:0/1605848264 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f55a010ff50 con 0x7f55a0070c70 2024-04-12T21:50:01.811 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:50:01.807+0000 7f559cc00700 1 -- 172.21.15.18:0/1605848264 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f558801c630 con 0x7f55a0070c70 2024-04-12T21:50:01.811 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:50:01.807+0000 7f55a7200700 1 -- 172.21.15.18:0/1605848264 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f55a0110490 con 0x7f55a0070c70 2024-04-12T21:50:01.811 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:50:01.808+0000 7f55a7200700 1 -- 172.21.15.18:0/1605848264 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f55a0071aa0 con 0x7f55a0070c70 2024-04-12T21:50:01.811 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:50:01.809+0000 7f559cc00700 1 -- 172.21.15.18:0/1605848264 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7f558800bd80 con 0x7f55a0070c70 2024-04-12T21:50:01.812 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:50:01.810+0000 7f559cc00700 1 --2- 172.21.15.18:0/1605848264 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f558406ec00 0x7f55840710b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:50:01.812 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:50:01.810+0000 7f559cc00700 1 -- 172.21.15.18:0/1605848264 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 6757+0+0 (secure 0 0 0) 0x7f558808fc40 con 0x7f55a0070c70 2024-04-12T21:50:01.812 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:50:01.811+0000 7f559fe00700 1 --2- 172.21.15.18:0/1605848264 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f558406ec00 0x7f55840710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:50:01.813 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:50:01.811+0000 7f559fe00700 1 --2- 172.21.15.18:0/1605848264 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f558406ec00 0x7f55840710b0 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f5590004990 tx=0x7f5590009ee0 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:50:01.817 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:50:01.815+0000 7f559cc00700 1 -- 172.21.15.18:0/1605848264 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f558805c490 con 0x7f55a0070c70 2024-04-12T21:50:02.105 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:50:01 smithi018 ceph-mon[112590]: from='client.36616 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:50:02.105 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:50:01 smithi018 ceph-mon[112590]: pgmap v1510: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:50:02.106 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:50:01 smithi018 ceph-mon[112590]: from='client.? 172.21.15.18:0/3820924420' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-12T21:50:02.154 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:50:02.152+0000 7f55a7200700 1 -- 172.21.15.18:0/1605848264 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f55a005f5f0 con 0x7f55a0070c70 2024-04-12T21:50:02.154 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:50:02.153+0000 7f559cc00700 1 -- 172.21.15.18:0/1605848264 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+4303 (secure 0 0 0) 0x7f5588013e60 con 0x7f55a0070c70 2024-04-12T21:50:02.156 INFO:teuthology.orchestra.run.smithi018.stdout:HEALTH_ERR failed to probe daemons or devices; 1 filesystem is degraded; 1 filesystem is offline 2024-04-12T21:50:02.156 INFO:teuthology.orchestra.run.smithi018.stdout:[WRN] CEPHADM_REFRESH_FAILED: failed to probe daemons or devices 2024-04-12T21:50:02.156 INFO:teuthology.orchestra.run.smithi018.stdout: host smithi186 `cephadm ceph-volume` failed: cephadm exited with an error code: 1, stderr: Inferring config /var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/mon.smithi186/config 2024-04-12T21:50:02.156 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/podman: timeout after 895 seconds 2024-04-12T21:50:02.156 INFO:teuthology.orchestra.run.smithi018.stdout:Non-zero exit code 124 from /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:f4452def3be4d350f4a8e5feaf7ebb1511be9298fe68478ba7a2d6fbd7d69a63 -e NODE_NAME=smithi186 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968:/var/run/ceph:z -v /var/log/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968:/var/log/ceph:z -v /var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmptgom53f9:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:f4452def3be4d350f4a8e5feaf7ebb1511be9298fe68478ba7a2d6fbd7d69a63 inventory --format=json-pretty --filter-for-batch 2024-04-12T21:50:02.156 INFO:teuthology.orchestra.run.smithi018.stdout:Traceback (most recent call last): 2024-04-12T21:50:02.156 INFO:teuthology.orchestra.run.smithi018.stdout: File "/usr/lib64/python3.9/runpy.py", line 197, in _run_module_as_main 2024-04-12T21:50:02.156 INFO:teuthology.orchestra.run.smithi018.stdout: return _run_code(code, main_globals, None, 2024-04-12T21:50:02.156 INFO:teuthology.orchestra.run.smithi018.stdout: File "/usr/lib64/python3.9/runpy.py", line 87, in _run_code 2024-04-12T21:50:02.156 INFO:teuthology.orchestra.run.smithi018.stdout: exec(code, run_globals) 2024-04-12T21:50:02.156 INFO:teuthology.orchestra.run.smithi018.stdout: File "/var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/cephadm.e44f3ed5e34e4ef52eedef233d1190bff004470d8e8005127b5312bff357fa7f/__main__.py", line 5618, in 2024-04-12T21:50:02.157 INFO:teuthology.orchestra.run.smithi018.stdout: File "/var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/cephadm.e44f3ed5e34e4ef52eedef233d1190bff004470d8e8005127b5312bff357fa7f/__main__.py", line 5606, in main 2024-04-12T21:50:02.157 INFO:teuthology.orchestra.run.smithi018.stdout: File "/var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/cephadm.e44f3ed5e34e4ef52eedef233d1190bff004470d8e8005127b5312bff357fa7f/__main__.py", line 408, in _infer_config 2024-04-12T21:50:02.157 INFO:teuthology.orchestra.run.smithi018.stdout: File "/var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/cephadm.e44f3ed5e34e4ef52eedef233d1190bff004470d8e8005127b5312bff357fa7f/__main__.py", line 323, in _infer_fsid 2024-04-12T21:50:02.157 INFO:teuthology.orchestra.run.smithi018.stdout: File "/var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/cephadm.e44f3ed5e34e4ef52eedef233d1190bff004470d8e8005127b5312bff357fa7f/__main__.py", line 436, in _infer_image 2024-04-12T21:50:02.157 INFO:teuthology.orchestra.run.smithi018.stdout: File "/var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/cephadm.e44f3ed5e34e4ef52eedef233d1190bff004470d8e8005127b5312bff357fa7f/__main__.py", line 310, in _validate_fsid 2024-04-12T21:50:02.157 INFO:teuthology.orchestra.run.smithi018.stdout: File "/var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/cephadm.e44f3ed5e34e4ef52eedef233d1190bff004470d8e8005127b5312bff357fa7f/__main__.py", line 3359, in command_ceph_volume 2024-04-12T21:50:02.157 INFO:teuthology.orchestra.run.smithi018.stdout: File "/var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/cephadm.e44f3ed5e34e4ef52eedef233d1190bff004470d8e8005127b5312bff357fa7f/cephadmlib/call_wrappers.py", line 310, in call_throws 2024-04-12T21:50:02.157 INFO:teuthology.orchestra.run.smithi018.stdout:RuntimeError: Failed command: /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:f4452def3be4d350f4a8e5feaf7ebb1511be9298fe68478ba7a2d6fbd7d69a63 -e NODE_NAME=smithi186 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968:/var/run/ceph:z -v /var/log/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968:/var/log/ceph:z -v /var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmptgom53f9:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:f4452def3be4d350f4a8e5feaf7ebb1511be9298fe68478ba7a2d6fbd7d69a63 inventory --format=json-pretty --filter-for-batch 2024-04-12T21:50:02.157 INFO:teuthology.orchestra.run.smithi018.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-04-12T21:50:02.157 INFO:teuthology.orchestra.run.smithi018.stdout: fs cephfs is degraded 2024-04-12T21:50:02.157 INFO:teuthology.orchestra.run.smithi018.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-04-12T21:50:02.157 INFO:teuthology.orchestra.run.smithi018.stdout: fs cephfs is offline because no MDS is active for it. 2024-04-12T21:50:02.158 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:50:02.156+0000 7f55a7200700 1 -- 172.21.15.18:0/1605848264 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f558406ec00 msgr2=0x7f55840710b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:50:02.158 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:50:02.156+0000 7f55a7200700 1 --2- 172.21.15.18:0/1605848264 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f558406ec00 0x7f55840710b0 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f5590004990 tx=0x7f5590009ee0 comp rx=0 tx=0).stop 2024-04-12T21:50:02.158 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:50:02.156+0000 7f55a7200700 1 -- 172.21.15.18:0/1605848264 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f55a0070c70 msgr2=0x7f55a01162b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:50:02.158 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:50:02.156+0000 7f55a7200700 1 --2- 172.21.15.18:0/1605848264 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f55a0070c70 0x7f55a01162b0 secure :-1 s=READY pgs=1281 cs=0 l=1 rev1=1 crypto rx=0x7f5588000c00 tx=0x7f5588009e30 comp rx=0 tx=0).stop 2024-04-12T21:50:02.159 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:50:02.156+0000 7f55a7200700 1 -- 172.21.15.18:0/1605848264 shutdown_connections 2024-04-12T21:50:02.159 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:50:02.156+0000 7f55a7200700 1 --2- 172.21.15.18:0/1605848264 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f558406ec00 0x7f55840710b0 unknown :-1 s=CLOSED pgs=679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:50:02.159 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:50:02.156+0000 7f55a7200700 1 --2- 172.21.15.18:0/1605848264 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f55a0070360 0x7f55a0115d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:50:02.159 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:50:02.156+0000 7f55a7200700 1 --2- 172.21.15.18:0/1605848264 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f55a0070c70 0x7f55a01162b0 unknown :-1 s=CLOSED pgs=1281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:50:02.159 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:50:02.156+0000 7f55a7200700 1 -- 172.21.15.18:0/1605848264 >> 172.21.15.18:0/1605848264 conn(0x7f55a00f89b0 msgr2=0x7f55a00f9e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:50:02.159 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:50:02.156+0000 7f55a7200700 1 -- 172.21.15.18:0/1605848264 shutdown_connections 2024-04-12T21:50:02.159 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:50:02.156+0000 7f55a7200700 1 -- 172.21.15.18:0/1605848264 wait complete. 2024-04-12T21:50:02.284 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:50:01 smithi186 ceph-mon[86379]: from='client.36616 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:50:02.284 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:50:01 smithi186 ceph-mon[86379]: pgmap v1510: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:50:02.284 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:50:01 smithi186 ceph-mon[86379]: from='client.? 172.21.15.18:0/3820924420' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-12T21:50:03.105 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:50:02 smithi018 ceph-mon[112590]: from='client.36628 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:50:03.106 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:50:02 smithi018 ceph-mon[112590]: from='client.? 172.21.15.18:0/1605848264' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-12T21:50:03.284 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:50:02 smithi186 ceph-mon[86379]: from='client.36628 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:50:03.284 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:50:02 smithi186 ceph-mon[86379]: from='client.? 172.21.15.18:0/1605848264' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-12T21:50:04.105 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:50:03 smithi018 ceph-mon[112590]: pgmap v1511: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:50:04.284 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:50:03 smithi186 ceph-mon[86379]: pgmap v1511: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:50:06.284 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:50:05 smithi186 ceph-mon[86379]: pgmap v1512: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:50:06.355 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:50:05 smithi018 ceph-mon[112590]: pgmap v1512: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:50:08.284 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:50:07 smithi186 ceph-mon[86379]: pgmap v1513: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:50:08.355 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:50:07 smithi018 ceph-mon[112590]: pgmap v1513: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:50:10.284 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:50:09 smithi186 ceph-mon[86379]: pgmap v1514: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:50:10.355 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:50:09 smithi018 ceph-mon[112590]: pgmap v1514: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:50:11.284 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:50:10 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T21:50:11.355 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:50:10 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T21:50:12.284 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:50:11 smithi186 ceph-mon[86379]: pgmap v1515: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:50:12.355 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:50:11 smithi018 ceph-mon[112590]: pgmap v1515: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:50:14.284 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:50:13 smithi186 ceph-mon[86379]: pgmap v1516: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:50:14.355 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:50:13 smithi018 ceph-mon[112590]: pgmap v1516: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:50:16.284 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:50:15 smithi186 ceph-mon[86379]: pgmap v1517: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:50:16.355 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:50:15 smithi018 ceph-mon[112590]: pgmap v1517: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:50:18.284 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:50:17 smithi186 ceph-mon[86379]: pgmap v1518: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:50:18.355 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:50:17 smithi018 ceph-mon[112590]: pgmap v1518: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:50:20.284 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:50:19 smithi186 ceph-mon[86379]: pgmap v1519: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:50:20.355 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:50:19 smithi018 ceph-mon[112590]: pgmap v1519: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:50:22.284 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:50:21 smithi186 ceph-mon[86379]: pgmap v1520: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:50:22.355 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:50:21 smithi018 ceph-mon[112590]: pgmap v1520: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:50:24.284 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:50:23 smithi186 ceph-mon[86379]: pgmap v1521: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:50:24.355 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:50:23 smithi018 ceph-mon[112590]: pgmap v1521: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:50:26.284 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:50:25 smithi186 ceph-mon[86379]: pgmap v1522: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:50:26.285 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:50:25 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T21:50:26.355 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:50:25 smithi018 ceph-mon[112590]: pgmap v1522: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:50:26.356 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:50:25 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T21:50:28.284 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:50:27 smithi186 ceph-mon[86379]: pgmap v1523: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:50:28.355 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:50:27 smithi018 ceph-mon[112590]: pgmap v1523: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:50:30.284 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:50:29 smithi186 ceph-mon[86379]: pgmap v1524: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:50:30.355 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:50:29 smithi018 ceph-mon[112590]: pgmap v1524: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:50:32.284 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:50:31 smithi186 ceph-mon[86379]: pgmap v1525: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:50:32.332 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:50:32.330+0000 7f56c1a00700 1 -- 172.21.15.18:0/3457739886 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f56bc105020 msgr2=0x7f56bc1053f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:50:32.332 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:50:32.330+0000 7f56c1a00700 1 --2- 172.21.15.18:0/3457739886 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f56bc105020 0x7f56bc1053f0 secure :-1 s=READY pgs=1282 cs=0 l=1 rev1=1 crypto rx=0x7f56ac004940 tx=0x7f56ac009e30 comp rx=0 tx=0).stop 2024-04-12T21:50:32.332 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:50:32.331+0000 7f56c1a00700 1 -- 172.21.15.18:0/3457739886 shutdown_connections 2024-04-12T21:50:32.332 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:50:32.331+0000 7f56c1a00700 1 --2- 172.21.15.18:0/3457739886 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f56bc1059c0 0x7f56bc0ff080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:50:32.332 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:50:32.331+0000 7f56c1a00700 1 --2- 172.21.15.18:0/3457739886 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f56bc105020 0x7f56bc1053f0 unknown :-1 s=CLOSED pgs=1282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:50:32.332 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:50:32.331+0000 7f56c1a00700 1 -- 172.21.15.18:0/3457739886 >> 172.21.15.18:0/3457739886 conn(0x7f56bc0faba0 msgr2=0x7f56bc0fcfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:50:32.333 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:50:31 smithi018 ceph-mon[112590]: pgmap v1525: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:50:32.334 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:50:32.331+0000 7f56c1a00700 1 -- 172.21.15.18:0/3457739886 shutdown_connections 2024-04-12T21:50:32.334 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:50:32.331+0000 7f56c1a00700 1 -- 172.21.15.18:0/3457739886 wait complete. 2024-04-12T21:50:32.334 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:50:32.332+0000 7f56c1a00700 1 Processor -- start 2024-04-12T21:50:32.334 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:50:32.332+0000 7f56c1a00700 1 -- start start 2024-04-12T21:50:32.334 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:50:32.333+0000 7f56c1a00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f56bc105020 0x7f56bc115bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:50:32.334 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:50:32.333+0000 7f56c1a00700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f56bc1059c0 0x7f56bc116130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:50:32.335 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:50:32.333+0000 7f56c1a00700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56bc116810 con 0x7f56bc105020 2024-04-12T21:50:32.335 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:50:32.333+0000 7f56c1a00700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56bc10fd60 con 0x7f56bc1059c0 2024-04-12T21:50:32.335 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:50:32.333+0000 7f56baa00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f56bc105020 0x7f56bc115bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:50:32.335 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:50:32.333+0000 7f56baa00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f56bc105020 0x7f56bc115bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:45776/0 (socket says 172.21.15.18:45776) 2024-04-12T21:50:32.335 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:50:32.333+0000 7f56baa00700 1 -- 172.21.15.18:0/2922363387 learned_addr learned my addr 172.21.15.18:0/2922363387 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:50:32.336 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:50:32.333+0000 7f56ba000700 1 --2- 172.21.15.18:0/2922363387 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f56bc1059c0 0x7f56bc116130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:50:32.336 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:50:32.334+0000 7f56baa00700 1 -- 172.21.15.18:0/2922363387 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f56bc1059c0 msgr2=0x7f56bc116130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:50:32.336 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:50:32.334+0000 7f56baa00700 1 --2- 172.21.15.18:0/2922363387 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f56bc1059c0 0x7f56bc116130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:50:32.336 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:50:32.334+0000 7f56baa00700 1 -- 172.21.15.18:0/2922363387 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56ac005040 con 0x7f56bc105020 2024-04-12T21:50:32.336 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:50:32.334+0000 7f56ba000700 1 --2- 172.21.15.18:0/2922363387 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f56bc1059c0 0x7f56bc116130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-12T21:50:32.336 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:50:32.334+0000 7f56baa00700 1 --2- 172.21.15.18:0/2922363387 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f56bc105020 0x7f56bc115bf0 secure :-1 s=READY pgs=1283 cs=0 l=1 rev1=1 crypto rx=0x7f56ac009b30 tx=0x7f56ac009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:50:32.336 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:50:32.334+0000 7f56b2a00700 1 -- 172.21.15.18:0/2922363387 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f56ac00e040 con 0x7f56bc105020 2024-04-12T21:50:32.337 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:50:32.335+0000 7f56b2a00700 1 -- 172.21.15.18:0/2922363387 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f56ac00bbc0 con 0x7f56bc105020 2024-04-12T21:50:32.337 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:50:32.335+0000 7f56c1a00700 1 -- 172.21.15.18:0/2922363387 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f56bc10ff80 con 0x7f56bc105020 2024-04-12T21:50:32.337 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:50:32.335+0000 7f56b2a00700 1 -- 172.21.15.18:0/2922363387 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f56ac01c3f0 con 0x7f56bc105020 2024-04-12T21:50:32.337 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:50:32.335+0000 7f56c1a00700 1 -- 172.21.15.18:0/2922363387 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f56bc1104c0 con 0x7f56bc105020 2024-04-12T21:50:32.337 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:50:32.336+0000 7f56c1a00700 1 -- 172.21.15.18:0/2922363387 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f56bc100770 con 0x7f56bc105020 2024-04-12T21:50:32.338 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:50:32.337+0000 7f56b2a00700 1 -- 172.21.15.18:0/2922363387 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7f56ac00bd30 con 0x7f56bc105020 2024-04-12T21:50:32.339 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:50:32.338+0000 7f56b2a00700 1 --2- 172.21.15.18:0/2922363387 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f56a006ec00 0x7f56a00710b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:50:32.339 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:50:32.338+0000 7f56b2a00700 1 -- 172.21.15.18:0/2922363387 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 6757+0+0 (secure 0 0 0) 0x7f56ac08fda0 con 0x7f56bc105020 2024-04-12T21:50:32.339 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:50:32.339+0000 7f56ba000700 1 --2- 172.21.15.18:0/2922363387 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f56a006ec00 0x7f56a00710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:50:32.340 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:50:32.339+0000 7f56ba000700 1 --2- 172.21.15.18:0/2922363387 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f56a006ec00 0x7f56a00710b0 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f56bc111490 tx=0x7f56a8008040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:50:32.344 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:50:32.343+0000 7f56b2a00700 1 -- 172.21.15.18:0/2922363387 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f56ac05c4d0 con 0x7f56bc105020 2024-04-12T21:50:32.602 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:50:32.601+0000 7f56c1a00700 1 -- 172.21.15.18:0/2922363387 --> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f56bc1107a0 con 0x7f56a006ec00 2024-04-12T21:50:32.605 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:50:32.604+0000 7f56b2a00700 1 -- 172.21.15.18:0/2922363387 <== mgr.34102 v2:172.21.15.18:6800/3812891919 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f56bc1107a0 con 0x7f56a006ec00 2024-04-12T21:50:32.608 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:50:32.607+0000 7f56c1a00700 1 -- 172.21.15.18:0/2922363387 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f56a006ec00 msgr2=0x7f56a00710b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:50:32.608 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:50:32.607+0000 7f56c1a00700 1 --2- 172.21.15.18:0/2922363387 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f56a006ec00 0x7f56a00710b0 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f56bc111490 tx=0x7f56a8008040 comp rx=0 tx=0).stop 2024-04-12T21:50:32.609 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:50:32.607+0000 7f56c1a00700 1 -- 172.21.15.18:0/2922363387 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f56bc105020 msgr2=0x7f56bc115bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:50:32.609 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:50:32.607+0000 7f56c1a00700 1 --2- 172.21.15.18:0/2922363387 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f56bc105020 0x7f56bc115bf0 secure :-1 s=READY pgs=1283 cs=0 l=1 rev1=1 crypto rx=0x7f56ac009b30 tx=0x7f56ac009e30 comp rx=0 tx=0).stop 2024-04-12T21:50:32.609 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:50:32.607+0000 7f56c1a00700 1 -- 172.21.15.18:0/2922363387 shutdown_connections 2024-04-12T21:50:32.609 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:50:32.607+0000 7f56c1a00700 1 --2- 172.21.15.18:0/2922363387 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f56a006ec00 0x7f56a00710b0 unknown :-1 s=CLOSED pgs=680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:50:32.609 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:50:32.607+0000 7f56c1a00700 1 --2- 172.21.15.18:0/2922363387 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f56bc105020 0x7f56bc115bf0 unknown :-1 s=CLOSED pgs=1283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:50:32.609 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:50:32.607+0000 7f56c1a00700 1 --2- 172.21.15.18:0/2922363387 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f56bc1059c0 0x7f56bc116130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:50:32.609 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:50:32.607+0000 7f56c1a00700 1 -- 172.21.15.18:0/2922363387 >> 172.21.15.18:0/2922363387 conn(0x7f56bc0faba0 msgr2=0x7f56bc0fbb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:50:32.609 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:50:32.607+0000 7f56c1a00700 1 -- 172.21.15.18:0/2922363387 shutdown_connections 2024-04-12T21:50:32.609 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:50:32.607+0000 7f56c1a00700 1 -- 172.21.15.18:0/2922363387 wait complete. 2024-04-12T21:50:32.624 INFO:teuthology.orchestra.run.smithi018.stdout:true 2024-04-12T21:50:32.770 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:50:32.769+0000 7f63cc200700 1 -- 172.21.15.18:0/4086748530 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f63c40fe830 msgr2=0x7f63c4106f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:50:32.770 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:50:32.769+0000 7f63cc200700 1 --2- 172.21.15.18:0/4086748530 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f63c40fe830 0x7f63c4106f00 secure :-1 s=READY pgs=1284 cs=0 l=1 rev1=1 crypto rx=0x7f63b0004940 tx=0x7f63b0009e30 comp rx=0 tx=0).stop 2024-04-12T21:50:32.770 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:50:32.769+0000 7f63cc200700 1 -- 172.21.15.18:0/4086748530 shutdown_connections 2024-04-12T21:50:32.770 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:50:32.769+0000 7f63cc200700 1 --2- 172.21.15.18:0/4086748530 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f63c40fe830 0x7f63c4106f00 unknown :-1 s=CLOSED pgs=1284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:50:32.770 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:50:32.769+0000 7f63cc200700 1 --2- 172.21.15.18:0/4086748530 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f63c40fde90 0x7f63c40fe260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:50:32.770 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:50:32.769+0000 7f63cc200700 1 -- 172.21.15.18:0/4086748530 >> 172.21.15.18:0/4086748530 conn(0x7f63c4072bc0 msgr2=0x7f63c4072fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:50:32.770 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:50:32.769+0000 7f63cc200700 1 -- 172.21.15.18:0/4086748530 shutdown_connections 2024-04-12T21:50:32.771 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:50:32.769+0000 7f63cc200700 1 -- 172.21.15.18:0/4086748530 wait complete. 2024-04-12T21:50:32.772 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:50:32.770+0000 7f63cc200700 1 Processor -- start 2024-04-12T21:50:32.772 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:50:32.770+0000 7f63cc200700 1 -- start start 2024-04-12T21:50:32.772 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:50:32.770+0000 7f63cc200700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f63c40fde90 0x7f63c4115da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:50:32.772 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:50:32.770+0000 7f63cc200700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f63c40fe830 0x7f63c41162e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:50:32.772 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:50:32.770+0000 7f63cc200700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63c4116930 con 0x7f63c40fe830 2024-04-12T21:50:32.772 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:50:32.770+0000 7f63cc200700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63c410fd00 con 0x7f63c40fde90 2024-04-12T21:50:32.772 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:50:32.771+0000 7f63c8e00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f63c40fe830 0x7f63c41162e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:50:32.772 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:50:32.771+0000 7f63c8e00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f63c40fe830 0x7f63c41162e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:45786/0 (socket says 172.21.15.18:45786) 2024-04-12T21:50:32.772 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:50:32.771+0000 7f63c8e00700 1 -- 172.21.15.18:0/3593737758 learned_addr learned my addr 172.21.15.18:0/3593737758 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:50:32.773 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:50:32.771+0000 7f63c9800700 1 --2- 172.21.15.18:0/3593737758 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f63c40fde90 0x7f63c4115da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:50:32.773 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:50:32.771+0000 7f63c8e00700 1 -- 172.21.15.18:0/3593737758 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f63c40fde90 msgr2=0x7f63c4115da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:50:32.773 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:50:32.771+0000 7f63c8e00700 1 --2- 172.21.15.18:0/3593737758 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f63c40fde90 0x7f63c4115da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:50:32.773 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:50:32.771+0000 7f63c8e00700 1 -- 172.21.15.18:0/3593737758 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63b0005040 con 0x7f63c40fe830 2024-04-12T21:50:32.773 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:50:32.772+0000 7f63c8e00700 1 --2- 172.21.15.18:0/3593737758 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f63c40fe830 0x7f63c41162e0 secure :-1 s=READY pgs=1285 cs=0 l=1 rev1=1 crypto rx=0x7f63b0000c00 tx=0x7f63b0009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:50:32.774 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:50:32.772+0000 7f63c1600700 1 -- 172.21.15.18:0/3593737758 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f63b000e040 con 0x7f63c40fe830 2024-04-12T21:50:32.774 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:50:32.772+0000 7f63c1600700 1 -- 172.21.15.18:0/3593737758 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f63b0013420 con 0x7f63c40fe830 2024-04-12T21:50:32.774 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:50:32.772+0000 7f63c1600700 1 -- 172.21.15.18:0/3593737758 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f63b001c610 con 0x7f63c40fe830 2024-04-12T21:50:32.774 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:50:32.772+0000 7f63cc200700 1 -- 172.21.15.18:0/3593737758 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f63c410ff80 con 0x7f63c40fe830 2024-04-12T21:50:32.774 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:50:32.772+0000 7f63cc200700 1 -- 172.21.15.18:0/3593737758 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f63c41104c0 con 0x7f63c40fe830 2024-04-12T21:50:32.774 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:50:32.773+0000 7f63cc200700 1 -- 172.21.15.18:0/3593737758 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f63c4106a20 con 0x7f63c40fe830 2024-04-12T21:50:32.775 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:50:32.774+0000 7f63c1600700 1 -- 172.21.15.18:0/3593737758 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7f63b0013590 con 0x7f63c40fe830 2024-04-12T21:50:32.776 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:50:32.775+0000 7f63c1600700 1 --2- 172.21.15.18:0/3593737758 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f63ac06ec10 0x7f63ac0710c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:50:32.776 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:50:32.775+0000 7f63c1600700 1 -- 172.21.15.18:0/3593737758 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 6757+0+0 (secure 0 0 0) 0x7f63b008fb50 con 0x7f63c40fe830 2024-04-12T21:50:32.776 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:50:32.776+0000 7f63c9800700 1 --2- 172.21.15.18:0/3593737758 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f63ac06ec10 0x7f63ac0710c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:50:32.777 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:50:32.776+0000 7f63c9800700 1 --2- 172.21.15.18:0/3593737758 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f63ac06ec10 0x7f63ac0710c0 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f63b8004d60 tx=0x7f63b800b040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:50:32.781 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:50:32.780+0000 7f63c1600700 1 -- 172.21.15.18:0/3593737758 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f63b005c3a0 con 0x7f63c40fe830 2024-04-12T21:50:33.033 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:50:33.032+0000 7f63cc200700 1 -- 172.21.15.18:0/3593737758 --> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f63c4111460 con 0x7f63ac06ec10 2024-04-12T21:50:33.035 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:50:33.034+0000 7f63c1600700 1 -- 172.21.15.18:0/3593737758 <== mgr.34102 v2:172.21.15.18:6800/3812891919 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f63c4111460 con 0x7f63ac06ec10 2024-04-12T21:50:33.038 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:50:33.037+0000 7f63cc200700 1 -- 172.21.15.18:0/3593737758 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f63ac06ec10 msgr2=0x7f63ac0710c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:50:33.039 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:50:33.037+0000 7f63cc200700 1 --2- 172.21.15.18:0/3593737758 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f63ac06ec10 0x7f63ac0710c0 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f63b8004d60 tx=0x7f63b800b040 comp rx=0 tx=0).stop 2024-04-12T21:50:33.039 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:50:33.037+0000 7f63cc200700 1 -- 172.21.15.18:0/3593737758 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f63c40fe830 msgr2=0x7f63c41162e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:50:33.039 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:50:33.037+0000 7f63cc200700 1 --2- 172.21.15.18:0/3593737758 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f63c40fe830 0x7f63c41162e0 secure :-1 s=READY pgs=1285 cs=0 l=1 rev1=1 crypto rx=0x7f63b0000c00 tx=0x7f63b0009e30 comp rx=0 tx=0).stop 2024-04-12T21:50:33.039 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:50:33.037+0000 7f63cc200700 1 -- 172.21.15.18:0/3593737758 shutdown_connections 2024-04-12T21:50:33.039 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:50:33.037+0000 7f63cc200700 1 --2- 172.21.15.18:0/3593737758 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f63ac06ec10 0x7f63ac0710c0 unknown :-1 s=CLOSED pgs=681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:50:33.039 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:50:33.037+0000 7f63cc200700 1 --2- 172.21.15.18:0/3593737758 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f63c40fde90 0x7f63c4115da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:50:33.039 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:50:33.037+0000 7f63cc200700 1 --2- 172.21.15.18:0/3593737758 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f63c40fe830 0x7f63c41162e0 unknown :-1 s=CLOSED pgs=1285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:50:33.039 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:50:33.037+0000 7f63cc200700 1 -- 172.21.15.18:0/3593737758 >> 172.21.15.18:0/3593737758 conn(0x7f63c4072bc0 msgr2=0x7f63c40fb6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:50:33.039 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:50:33.038+0000 7f63cc200700 1 -- 172.21.15.18:0/3593737758 shutdown_connections 2024-04-12T21:50:33.039 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:50:33.038+0000 7f63cc200700 1 -- 172.21.15.18:0/3593737758 wait complete. 2024-04-12T21:50:33.203 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:50:33.200+0000 7f05bac00700 1 -- 172.21.15.18:0/3274658031 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f05b40708a0 msgr2=0x7f05b4070d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:50:33.203 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:50:33.200+0000 7f05bac00700 1 --2- 172.21.15.18:0/3274658031 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f05b40708a0 0x7f05b4070d10 secure :-1 s=READY pgs=1286 cs=0 l=1 rev1=1 crypto rx=0x7f05a0004970 tx=0x7f05a0009e30 comp rx=0 tx=0).stop 2024-04-12T21:50:33.203 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:50:33.201+0000 7f05bac00700 1 -- 172.21.15.18:0/3274658031 shutdown_connections 2024-04-12T21:50:33.203 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:50:33.201+0000 7f05bac00700 1 --2- 172.21.15.18:0/3274658031 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f05b40708a0 0x7f05b4070d10 unknown :-1 s=CLOSED pgs=1286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:50:33.204 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:50:33.201+0000 7f05bac00700 1 --2- 172.21.15.18:0/3274658031 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f05b4071f90 0x7f05b4070360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:50:33.204 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:50:33.201+0000 7f05bac00700 1 -- 172.21.15.18:0/3274658031 >> 172.21.15.18:0/3274658031 conn(0x7f05b40f87d0 msgr2=0x7f05b40fabe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:50:33.204 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:50:33.201+0000 7f05bac00700 1 -- 172.21.15.18:0/3274658031 shutdown_connections 2024-04-12T21:50:33.204 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:50:33.201+0000 7f05bac00700 1 -- 172.21.15.18:0/3274658031 wait complete. 2024-04-12T21:50:33.204 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:50:33.202+0000 7f05bac00700 1 Processor -- start 2024-04-12T21:50:33.204 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:50:33.202+0000 7f05bac00700 1 -- start start 2024-04-12T21:50:33.206 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:50:33.202+0000 7f05bac00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f05b40708a0 0x7f05b4119f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:50:33.206 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:50:33.202+0000 7f05bac00700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f05b4071f90 0x7f05b411a4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:50:33.206 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:50:33.202+0000 7f05bac00700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05b411aaf0 con 0x7f05b40708a0 2024-04-12T21:50:33.206 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:50:33.202+0000 7f05bac00700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05b4113f00 con 0x7f05b4071f90 2024-04-12T21:50:33.206 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:50:33.203+0000 7f05b9800700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f05b40708a0 0x7f05b4119f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:50:33.206 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:50:33.203+0000 7f05b9800700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f05b40708a0 0x7f05b4119f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:45808/0 (socket says 172.21.15.18:45808) 2024-04-12T21:50:33.206 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:50:33.203+0000 7f05b9800700 1 -- 172.21.15.18:0/2702024648 learned_addr learned my addr 172.21.15.18:0/2702024648 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:50:33.206 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:50:33.203+0000 7f05b8e00700 1 --2- 172.21.15.18:0/2702024648 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f05b4071f90 0x7f05b411a4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:50:33.207 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:50:33.203+0000 7f05b9800700 1 -- 172.21.15.18:0/2702024648 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f05b4071f90 msgr2=0x7f05b411a4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:50:33.207 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:50:33.203+0000 7f05b9800700 1 --2- 172.21.15.18:0/2702024648 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f05b4071f90 0x7f05b411a4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:50:33.207 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:50:33.203+0000 7f05b9800700 1 -- 172.21.15.18:0/2702024648 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05a0005040 con 0x7f05b40708a0 2024-04-12T21:50:33.207 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:50:33.203+0000 7f05b9800700 1 --2- 172.21.15.18:0/2702024648 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f05b40708a0 0x7f05b4119f60 secure :-1 s=READY pgs=1287 cs=0 l=1 rev1=1 crypto rx=0x7f05a8002a50 tx=0x7f05a80046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:50:33.207 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:50:33.204+0000 7f05b1600700 1 -- 172.21.15.18:0/2702024648 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f05a8007cc0 con 0x7f05b40708a0 2024-04-12T21:50:33.207 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:50:33.204+0000 7f05b1600700 1 -- 172.21.15.18:0/2702024648 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f05a800f040 con 0x7f05b40708a0 2024-04-12T21:50:33.207 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:50:33.204+0000 7f05bac00700 1 -- 172.21.15.18:0/2702024648 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f05b41141e0 con 0x7f05b40708a0 2024-04-12T21:50:33.207 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:50:33.204+0000 7f05b1600700 1 -- 172.21.15.18:0/2702024648 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f05a8009640 con 0x7f05b40708a0 2024-04-12T21:50:33.207 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:50:33.204+0000 7f05bac00700 1 -- 172.21.15.18:0/2702024648 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f05b4114720 con 0x7f05b40708a0 2024-04-12T21:50:33.207 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:50:33.205+0000 7f05bac00700 1 -- 172.21.15.18:0/2702024648 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f05b4071aa0 con 0x7f05b40708a0 2024-04-12T21:50:33.208 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:50:33.206+0000 7f05b1600700 1 -- 172.21.15.18:0/2702024648 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7f05a80097a0 con 0x7f05b40708a0 2024-04-12T21:50:33.208 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:50:33.207+0000 7f05b1600700 1 --2- 172.21.15.18:0/2702024648 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f059c06ec00 0x7f059c0710b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:50:33.208 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:50:33.207+0000 7f05b1600700 1 -- 172.21.15.18:0/2702024648 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 6757+0+0 (secure 0 0 0) 0x7f05a808e760 con 0x7f05b40708a0 2024-04-12T21:50:33.208 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:50:33.207+0000 7f05b8e00700 1 --2- 172.21.15.18:0/2702024648 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f059c06ec00 0x7f059c0710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:50:33.209 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:50:33.208+0000 7f05b8e00700 1 --2- 172.21.15.18:0/2702024648 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f059c06ec00 0x7f059c0710b0 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f05a0000c00 tx=0x7f05a0009e30 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:50:33.212 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:50:33.212+0000 7f05b1600700 1 -- 172.21.15.18:0/2702024648 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f05a80573e0 con 0x7f05b40708a0 2024-04-12T21:50:33.455 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:50:33.454+0000 7f05bac00700 1 -- 172.21.15.18:0/2702024648 --> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f05b404bed0 con 0x7f059c06ec00 2024-04-12T21:50:33.467 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:50:33.466+0000 7f05b1600700 1 -- 172.21.15.18:0/2702024648 <== mgr.34102 v2:172.21.15.18:6800/3812891919 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7f05b404bed0 con 0x7f059c06ec00 2024-04-12T21:50:33.468 INFO:teuthology.orchestra.run.smithi018.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-12T21:50:33.468 INFO:teuthology.orchestra.run.smithi018.stdout:alertmanager.smithi018 smithi018 *:9093,9094 running (51m) 12m ago 65m 14.8M - 0.25.0 c8568f914cd2 c332090068e8 2024-04-12T21:50:33.468 INFO:teuthology.orchestra.run.smithi018.stdout:crash.smithi018 smithi018 running (47m) 12m ago 65m 6665k - 19.0.0-2879-g324baf8d 8717720dc5e5 b7f80d1f7982 2024-04-12T21:50:33.468 INFO:teuthology.orchestra.run.smithi018.stdout:crash.smithi186 smithi186 running (47m) 12m ago 64m 6660k - 19.0.0-2879-g324baf8d 8717720dc5e5 a4f55c81b623 2024-04-12T21:50:33.468 INFO:teuthology.orchestra.run.smithi018.stdout:grafana.smithi018 smithi018 *:3000 running (50m) 12m ago 65m 73.8M - 9.4.12 2bacad6d85d8 b08e3bf176dd 2024-04-12T21:50:33.468 INFO:teuthology.orchestra.run.smithi018.stdout:mds.cephfs.smithi018.acqbsc smithi018 running (43m) 12m ago 58m 21.9M - 19.0.0-2879-g324baf8d 8717720dc5e5 021f99ab24ae 2024-04-12T21:50:33.468 INFO:teuthology.orchestra.run.smithi018.stdout:mds.cephfs.smithi018.szjluo smithi018 running (42m) 12m ago 58m 21.7M - 19.0.0-2879-g324baf8d 8717720dc5e5 0003117479d4 2024-04-12T21:50:33.468 INFO:teuthology.orchestra.run.smithi018.stdout:mds.cephfs.smithi186.jdllyn smithi186 running (27m) 12m ago 58m 16.5M - 19.0.0-2879-g324baf8d 8717720dc5e5 21b1f46eef23 2024-04-12T21:50:33.468 INFO:teuthology.orchestra.run.smithi018.stdout:mds.cephfs.smithi186.wjnafs smithi186 running (12m) 12m ago 58m 11.0M - 19.0.0-2879-g324baf8d 8717720dc5e5 3cf1ef4148e2 2024-04-12T21:50:33.469 INFO:teuthology.orchestra.run.smithi018.stdout:mgr.smithi018.xjzwga smithi018 *:8443,9283,8765 running (53m) 12m ago 67m 517M - 19.0.0-2879-g324baf8d 8717720dc5e5 56088305eab7 2024-04-12T21:50:33.469 INFO:teuthology.orchestra.run.smithi018.stdout:mgr.smithi186.kqcfxk smithi186 *:8443,9283,8765 running (52m) 12m ago 64m 460M - 19.0.0-2879-g324baf8d 8717720dc5e5 09cd766e5e0d 2024-04-12T21:50:33.469 INFO:teuthology.orchestra.run.smithi018.stdout:mon.smithi018 smithi018 running (49m) 12m ago 67m 88.2M 2048M 19.0.0-2879-g324baf8d 8717720dc5e5 df3e201dfed6 2024-04-12T21:50:33.469 INFO:teuthology.orchestra.run.smithi018.stdout:mon.smithi186 smithi186 running (48m) 12m ago 64m 53.6M 2048M 19.0.0-2879-g324baf8d 8717720dc5e5 c77498da9b2b 2024-04-12T21:50:33.469 INFO:teuthology.orchestra.run.smithi018.stdout:node-exporter.smithi018 smithi018 *:9100 running (52m) 12m ago 65m 8468k - 1.5.0 0da6a335fe13 326c3d9c6847 2024-04-12T21:50:33.469 INFO:teuthology.orchestra.run.smithi018.stdout:node-exporter.smithi186 smithi186 *:9100 running (52m) 12m ago 64m 8166k - 1.5.0 0da6a335fe13 ec2112baeecf 2024-04-12T21:50:33.469 INFO:teuthology.orchestra.run.smithi018.stdout:osd.0 smithi018 running (46m) 12m ago 63m 123M 1559M 19.0.0-2879-g324baf8d 8717720dc5e5 60b1273ca047 2024-04-12T21:50:33.469 INFO:teuthology.orchestra.run.smithi018.stdout:osd.1 smithi018 running (46m) 12m ago 62m 95.4M 1559M 19.0.0-2879-g324baf8d 8717720dc5e5 170ce1e5fb8d 2024-04-12T21:50:33.469 INFO:teuthology.orchestra.run.smithi018.stdout:osd.2 smithi018 running (45m) 12m ago 62m 94.6M 1559M 19.0.0-2879-g324baf8d 8717720dc5e5 c2cbc6aa2839 2024-04-12T21:50:33.469 INFO:teuthology.orchestra.run.smithi018.stdout:osd.3 smithi186 running (45m) 12m ago 61m 101M 2581M 19.0.0-2879-g324baf8d 8717720dc5e5 0138e8262e4b 2024-04-12T21:50:33.469 INFO:teuthology.orchestra.run.smithi018.stdout:osd.4 smithi186 running (44m) 12m ago 60m 91.5M 2581M 19.0.0-2879-g324baf8d 8717720dc5e5 b20cb63b25c9 2024-04-12T21:50:33.469 INFO:teuthology.orchestra.run.smithi018.stdout:osd.5 smithi186 running (43m) 12m ago 60m 92.2M 2581M 19.0.0-2879-g324baf8d 8717720dc5e5 2513bea2610e 2024-04-12T21:50:33.469 INFO:teuthology.orchestra.run.smithi018.stdout:prometheus.smithi018 smithi018 *:9095 running (51m) 12m ago 64m 69.6M - 2.43.0 a07b618ecd1d f58841127e21 2024-04-12T21:50:33.471 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:50:33.469+0000 7f05bac00700 1 -- 172.21.15.18:0/2702024648 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f059c06ec00 msgr2=0x7f059c0710b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:50:33.471 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:50:33.469+0000 7f05bac00700 1 --2- 172.21.15.18:0/2702024648 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f059c06ec00 0x7f059c0710b0 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f05a0000c00 tx=0x7f05a0009e30 comp rx=0 tx=0).stop 2024-04-12T21:50:33.471 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:50:33.469+0000 7f05bac00700 1 -- 172.21.15.18:0/2702024648 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f05b40708a0 msgr2=0x7f05b4119f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:50:33.471 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:50:33.469+0000 7f05bac00700 1 --2- 172.21.15.18:0/2702024648 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f05b40708a0 0x7f05b4119f60 secure :-1 s=READY pgs=1287 cs=0 l=1 rev1=1 crypto rx=0x7f05a8002a50 tx=0x7f05a80046b0 comp rx=0 tx=0).stop 2024-04-12T21:50:33.471 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:50:33.470+0000 7f05bac00700 1 -- 172.21.15.18:0/2702024648 shutdown_connections 2024-04-12T21:50:33.471 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:50:33.470+0000 7f05bac00700 1 --2- 172.21.15.18:0/2702024648 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f059c06ec00 0x7f059c0710b0 unknown :-1 s=CLOSED pgs=682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:50:33.471 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:50:33.470+0000 7f05bac00700 1 --2- 172.21.15.18:0/2702024648 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f05b40708a0 0x7f05b4119f60 unknown :-1 s=CLOSED pgs=1287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:50:33.471 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:50:33.470+0000 7f05bac00700 1 --2- 172.21.15.18:0/2702024648 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f05b4071f90 0x7f05b411a4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:50:33.471 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:50:33.470+0000 7f05bac00700 1 -- 172.21.15.18:0/2702024648 >> 172.21.15.18:0/2702024648 conn(0x7f05b40f87d0 msgr2=0x7f05b40f9760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:50:33.472 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:50:33.470+0000 7f05bac00700 1 -- 172.21.15.18:0/2702024648 shutdown_connections 2024-04-12T21:50:33.472 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:50:33.470+0000 7f05bac00700 1 -- 172.21.15.18:0/2702024648 wait complete. 2024-04-12T21:50:33.632 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:50:33.630+0000 7f80b7800700 1 -- 172.21.15.18:0/3454840567 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f80b00fe790 msgr2=0x7f80b0106dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:50:33.632 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:50:33.630+0000 7f80b7800700 1 --2- 172.21.15.18:0/3454840567 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f80b00fe790 0x7f80b0106dd0 secure :-1 s=READY pgs=1288 cs=0 l=1 rev1=1 crypto rx=0x7f809c004940 tx=0x7f809c009e30 comp rx=0 tx=0).stop 2024-04-12T21:50:33.632 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:50:33.631+0000 7f80b7800700 1 -- 172.21.15.18:0/3454840567 shutdown_connections 2024-04-12T21:50:33.632 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:50:33.631+0000 7f80b7800700 1 --2- 172.21.15.18:0/3454840567 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f80b00fe790 0x7f80b0106dd0 unknown :-1 s=CLOSED pgs=1288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:50:33.632 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:50:33.631+0000 7f80b7800700 1 --2- 172.21.15.18:0/3454840567 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f80b00fddf0 0x7f80b00fe1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:50:33.632 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:50:33.631+0000 7f80b7800700 1 -- 172.21.15.18:0/3454840567 >> 172.21.15.18:0/3454840567 conn(0x7f80b00f7860 msgr2=0x7f80b00f9c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:50:33.632 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:50:33.631+0000 7f80b7800700 1 -- 172.21.15.18:0/3454840567 shutdown_connections 2024-04-12T21:50:33.633 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:50:33.631+0000 7f80b7800700 1 -- 172.21.15.18:0/3454840567 wait complete. 2024-04-12T21:50:33.634 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:50:33.632+0000 7f80b7800700 1 Processor -- start 2024-04-12T21:50:33.634 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:50:33.632+0000 7f80b7800700 1 -- start start 2024-04-12T21:50:33.634 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:50:33.632+0000 7f80b7800700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f80b00fddf0 0x7f80b0115ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:50:33.634 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:50:33.632+0000 7f80b7800700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f80b00fe790 0x7f80b01161e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:50:33.634 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:50:33.632+0000 7f80b7800700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80b0116830 con 0x7f80b00fddf0 2024-04-12T21:50:33.634 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:50:33.632+0000 7f80b7800700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80b010fc40 con 0x7f80b00fe790 2024-04-12T21:50:33.634 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:50:33.632+0000 7f80b4e00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f80b00fddf0 0x7f80b0115ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:50:33.634 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:50:33.633+0000 7f80b4e00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f80b00fddf0 0x7f80b0115ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:45834/0 (socket says 172.21.15.18:45834) 2024-04-12T21:50:33.635 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:50:33.633+0000 7f80b4e00700 1 -- 172.21.15.18:0/915743825 learned_addr learned my addr 172.21.15.18:0/915743825 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:50:33.635 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:50:33.633+0000 7f80afe00700 1 --2- 172.21.15.18:0/915743825 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f80b00fe790 0x7f80b01161e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:50:33.635 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:50:33.633+0000 7f80b4e00700 1 -- 172.21.15.18:0/915743825 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f80b00fe790 msgr2=0x7f80b01161e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:50:33.635 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:50:33.633+0000 7f80b4e00700 1 --2- 172.21.15.18:0/915743825 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f80b00fe790 0x7f80b01161e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:50:33.635 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:50:33.633+0000 7f80b4e00700 1 -- 172.21.15.18:0/915743825 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f809c005040 con 0x7f80b00fddf0 2024-04-12T21:50:33.635 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:50:33.633+0000 7f80b4e00700 1 --2- 172.21.15.18:0/915743825 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f80b00fddf0 0x7f80b0115ca0 secure :-1 s=READY pgs=1289 cs=0 l=1 rev1=1 crypto rx=0x7f80a4002a80 tx=0x7f80a40046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:50:33.636 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:50:33.634+0000 7f80acc00700 1 -- 172.21.15.18:0/915743825 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f80a4007cc0 con 0x7f80b00fddf0 2024-04-12T21:50:33.636 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:50:33.634+0000 7f80acc00700 1 -- 172.21.15.18:0/915743825 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f80a400f040 con 0x7f80b00fddf0 2024-04-12T21:50:33.636 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:50:33.634+0000 7f80b7800700 1 -- 172.21.15.18:0/915743825 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f80b010ff20 con 0x7f80b00fddf0 2024-04-12T21:50:33.636 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:50:33.634+0000 7f80acc00700 1 -- 172.21.15.18:0/915743825 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f80a40096f0 con 0x7f80b00fddf0 2024-04-12T21:50:33.636 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:50:33.634+0000 7f80b7800700 1 -- 172.21.15.18:0/915743825 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f80b0110460 con 0x7f80b00fddf0 2024-04-12T21:50:33.636 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:50:33.635+0000 7f80b7800700 1 -- 172.21.15.18:0/915743825 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f80b0066770 con 0x7f80b00fddf0 2024-04-12T21:50:33.637 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:50:33.636+0000 7f80acc00700 1 -- 172.21.15.18:0/915743825 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7f80a400d690 con 0x7f80b00fddf0 2024-04-12T21:50:33.638 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:50:33.637+0000 7f80acc00700 1 --2- 172.21.15.18:0/915743825 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f809806ec10 0x7f80980710c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:50:33.638 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:50:33.637+0000 7f80acc00700 1 -- 172.21.15.18:0/915743825 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 6757+0+0 (secure 0 0 0) 0x7f80a408e110 con 0x7f80b00fddf0 2024-04-12T21:50:33.638 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:50:33.638+0000 7f80afe00700 1 --2- 172.21.15.18:0/915743825 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f809806ec10 0x7f80980710c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:50:33.639 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:50:33.638+0000 7f80afe00700 1 --2- 172.21.15.18:0/915743825 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f809806ec10 0x7f80980710c0 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f809c000c00 tx=0x7f809c009e30 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:50:33.643 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:50:33.642+0000 7f80acc00700 1 -- 172.21.15.18:0/915743825 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f80a4056d90 con 0x7f80b00fddf0 2024-04-12T21:50:33.972 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:50:33.971+0000 7f80b7800700 1 -- 172.21.15.18:0/915743825 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f80b004bed0 con 0x7f80b00fddf0 2024-04-12T21:50:33.973 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:50:33.972+0000 7f80acc00700 1 -- 172.21.15.18:0/915743825 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+623 (secure 0 0 0) 0x7f80a405a450 con 0x7f80b00fddf0 2024-04-12T21:50:33.973 INFO:teuthology.orchestra.run.smithi018.stdout:{ 2024-04-12T21:50:33.973 INFO:teuthology.orchestra.run.smithi018.stdout: "mon": { 2024-04-12T21:50:33.973 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 19.0.0-2879-g324baf8d (324baf8ddb91dea46cac50cb535c306115791355) squid (dev)": 2 2024-04-12T21:50:33.974 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-12T21:50:33.974 INFO:teuthology.orchestra.run.smithi018.stdout: "mgr": { 2024-04-12T21:50:33.974 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 19.0.0-2879-g324baf8d (324baf8ddb91dea46cac50cb535c306115791355) squid (dev)": 2 2024-04-12T21:50:33.974 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-12T21:50:33.974 INFO:teuthology.orchestra.run.smithi018.stdout: "osd": { 2024-04-12T21:50:33.974 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 19.0.0-2879-g324baf8d (324baf8ddb91dea46cac50cb535c306115791355) squid (dev)": 6 2024-04-12T21:50:33.974 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-12T21:50:33.974 INFO:teuthology.orchestra.run.smithi018.stdout: "mds": { 2024-04-12T21:50:33.974 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 19.0.0-2879-g324baf8d (324baf8ddb91dea46cac50cb535c306115791355) squid (dev)": 4 2024-04-12T21:50:33.974 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-12T21:50:33.974 INFO:teuthology.orchestra.run.smithi018.stdout: "overall": { 2024-04-12T21:50:33.974 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 19.0.0-2879-g324baf8d (324baf8ddb91dea46cac50cb535c306115791355) squid (dev)": 14 2024-04-12T21:50:33.974 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-04-12T21:50:33.974 INFO:teuthology.orchestra.run.smithi018.stdout:} 2024-04-12T21:50:33.976 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:50:33.975+0000 7f80b7800700 1 -- 172.21.15.18:0/915743825 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f809806ec10 msgr2=0x7f80980710c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:50:33.976 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:50:33.975+0000 7f80b7800700 1 --2- 172.21.15.18:0/915743825 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f809806ec10 0x7f80980710c0 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f809c000c00 tx=0x7f809c009e30 comp rx=0 tx=0).stop 2024-04-12T21:50:33.976 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:50:33.975+0000 7f80b7800700 1 -- 172.21.15.18:0/915743825 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f80b00fddf0 msgr2=0x7f80b0115ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:50:33.976 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:50:33.976+0000 7f80b7800700 1 --2- 172.21.15.18:0/915743825 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f80b00fddf0 0x7f80b0115ca0 secure :-1 s=READY pgs=1289 cs=0 l=1 rev1=1 crypto rx=0x7f80a4002a80 tx=0x7f80a40046b0 comp rx=0 tx=0).stop 2024-04-12T21:50:33.977 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:50:33.976+0000 7f80b7800700 1 -- 172.21.15.18:0/915743825 shutdown_connections 2024-04-12T21:50:33.977 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:50:33.976+0000 7f80b7800700 1 --2- 172.21.15.18:0/915743825 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f809806ec10 0x7f80980710c0 unknown :-1 s=CLOSED pgs=683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:50:33.977 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:50:33.976+0000 7f80b7800700 1 --2- 172.21.15.18:0/915743825 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f80b00fddf0 0x7f80b0115ca0 unknown :-1 s=CLOSED pgs=1289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:50:33.977 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:50:33.976+0000 7f80b7800700 1 --2- 172.21.15.18:0/915743825 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f80b00fe790 0x7f80b01161e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:50:33.977 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:50:33.976+0000 7f80b7800700 1 -- 172.21.15.18:0/915743825 >> 172.21.15.18:0/915743825 conn(0x7f80b00f7860 msgr2=0x7f80b00f8d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:50:33.977 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:50:33.976+0000 7f80b7800700 1 -- 172.21.15.18:0/915743825 shutdown_connections 2024-04-12T21:50:33.977 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:50:33.976+0000 7f80b7800700 1 -- 172.21.15.18:0/915743825 wait complete. 2024-04-12T21:50:34.132 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:50:34.131+0000 7fe8da600700 1 -- 172.21.15.18:0/378662208 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe8d4105020 msgr2=0x7fe8d41053f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:50:34.132 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:50:34.131+0000 7fe8da600700 1 --2- 172.21.15.18:0/378662208 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe8d4105020 0x7fe8d41053f0 secure :-1 s=READY pgs=1290 cs=0 l=1 rev1=1 crypto rx=0x7fe8c4004940 tx=0x7fe8c4009e30 comp rx=0 tx=0).stop 2024-04-12T21:50:34.132 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:50:34.131+0000 7fe8da600700 1 -- 172.21.15.18:0/378662208 shutdown_connections 2024-04-12T21:50:34.132 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:50:34.131+0000 7fe8da600700 1 --2- 172.21.15.18:0/378662208 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe8d41059c0 0x7fe8d40ff080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:50:34.132 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:50:34.131+0000 7fe8da600700 1 --2- 172.21.15.18:0/378662208 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe8d4105020 0x7fe8d41053f0 unknown :-1 s=CLOSED pgs=1290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:50:34.133 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:50:34.131+0000 7fe8da600700 1 -- 172.21.15.18:0/378662208 >> 172.21.15.18:0/378662208 conn(0x7fe8d40faba0 msgr2=0x7fe8d40fcfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:50:34.133 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:50:34.131+0000 7fe8da600700 1 -- 172.21.15.18:0/378662208 shutdown_connections 2024-04-12T21:50:34.133 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:50:34.131+0000 7fe8da600700 1 -- 172.21.15.18:0/378662208 wait complete. 2024-04-12T21:50:34.134 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:50:34.132+0000 7fe8da600700 1 Processor -- start 2024-04-12T21:50:34.134 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:50:34.132+0000 7fe8da600700 1 -- start start 2024-04-12T21:50:34.135 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:50:34.132+0000 7fe8da600700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe8d4105020 0x7fe8d4115bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:50:34.135 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:50:34.132+0000 7fe8da600700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe8d41059c0 0x7fe8d4116100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:50:34.135 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:50:34.133+0000 7fe8da600700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8d4116750 con 0x7fe8d4105020 2024-04-12T21:50:34.135 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:50:34.133+0000 7fe8da600700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8d410fd30 con 0x7fe8d41059c0 2024-04-12T21:50:34.135 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:50:34.133+0000 7fe8d3400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe8d4105020 0x7fe8d4115bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:50:34.135 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:50:34.133+0000 7fe8d3400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe8d4105020 0x7fe8d4115bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:45850/0 (socket says 172.21.15.18:45850) 2024-04-12T21:50:34.135 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:50:34.133+0000 7fe8d3400700 1 -- 172.21.15.18:0/2411343819 learned_addr learned my addr 172.21.15.18:0/2411343819 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:50:34.135 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:50:34.133+0000 7fe8d3400700 1 -- 172.21.15.18:0/2411343819 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe8d41059c0 msgr2=0x7fe8d4116100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:50:34.135 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:50:34.133+0000 7fe8d2a00700 1 --2- 172.21.15.18:0/2411343819 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe8d41059c0 0x7fe8d4116100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:50:34.135 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:50:34.133+0000 7fe8d3400700 1 --2- 172.21.15.18:0/2411343819 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe8d41059c0 0x7fe8d4116100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:50:34.135 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:50:34.133+0000 7fe8d3400700 1 -- 172.21.15.18:0/2411343819 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8c4005040 con 0x7fe8d4105020 2024-04-12T21:50:34.135 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:50:34.133+0000 7fe8d2a00700 1 --2- 172.21.15.18:0/2411343819 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe8d41059c0 0x7fe8d4116100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-12T21:50:34.135 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:50:34.134+0000 7fe8d3400700 1 --2- 172.21.15.18:0/2411343819 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe8d4105020 0x7fe8d4115bc0 secure :-1 s=READY pgs=1291 cs=0 l=1 rev1=1 crypto rx=0x7fe8c4009180 tx=0x7fe8c4009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:50:34.136 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:50:34.134+0000 7fe8cb400700 1 -- 172.21.15.18:0/2411343819 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe8c400e040 con 0x7fe8d4105020 2024-04-12T21:50:34.136 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:50:34.134+0000 7fe8cb400700 1 -- 172.21.15.18:0/2411343819 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe8c400bbc0 con 0x7fe8d4105020 2024-04-12T21:50:34.136 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:50:34.134+0000 7fe8cb400700 1 -- 172.21.15.18:0/2411343819 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe8c401c630 con 0x7fe8d4105020 2024-04-12T21:50:34.136 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:50:34.134+0000 7fe8da600700 1 -- 172.21.15.18:0/2411343819 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe8d410ffb0 con 0x7fe8d4105020 2024-04-12T21:50:34.137 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:50:34.134+0000 7fe8da600700 1 -- 172.21.15.18:0/2411343819 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe8d4110570 con 0x7fe8d4105020 2024-04-12T21:50:34.137 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:50:34.135+0000 7fe8da600700 1 -- 172.21.15.18:0/2411343819 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe8d4100770 con 0x7fe8d4105020 2024-04-12T21:50:34.137 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:50:34.136+0000 7fe8cb400700 1 -- 172.21.15.18:0/2411343819 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7fe8c400bd30 con 0x7fe8d4105020 2024-04-12T21:50:34.139 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:50:34.138+0000 7fe8cb400700 1 --2- 172.21.15.18:0/2411343819 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fe8b806ec00 0x7fe8b80710b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:50:34.139 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:50:34.138+0000 7fe8cb400700 1 -- 172.21.15.18:0/2411343819 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 6757+0+0 (secure 0 0 0) 0x7fe8c408fc90 con 0x7fe8d4105020 2024-04-12T21:50:34.139 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:50:34.138+0000 7fe8d2a00700 1 --2- 172.21.15.18:0/2411343819 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fe8b806ec00 0x7fe8b80710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:50:34.140 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:50:34.139+0000 7fe8d2a00700 1 --2- 172.21.15.18:0/2411343819 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fe8b806ec00 0x7fe8b80710b0 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7fe8c0009fd0 tx=0x7fe8c000b040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:50:34.144 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:50:34.143+0000 7fe8cb400700 1 -- 172.21.15.18:0/2411343819 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7fe8c405c580 con 0x7fe8d4105020 2024-04-12T21:50:34.285 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:50:33 smithi186 ceph-mon[86379]: pgmap v1526: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:50:34.285 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:50:33 smithi186 ceph-mon[86379]: from='client.36636 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:50:34.285 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:50:33 smithi186 ceph-mon[86379]: from='client.36640 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:50:34.355 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:50:33 smithi018 ceph-mon[112590]: pgmap v1526: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:50:34.356 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:50:33 smithi018 ceph-mon[112590]: from='client.36636 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:50:34.356 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:50:33 smithi018 ceph-mon[112590]: from='client.36640 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:50:34.422 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:50:34.421+0000 7fe8da600700 1 -- 172.21.15.18:0/2411343819 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fe8d4100770 con 0x7fe8d4105020 2024-04-12T21:50:34.423 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:50:34.422+0000 7fe8cb400700 1 -- 172.21.15.18:0/2411343819 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 33 v33) v1 ==== 76+0+1876 (secure 0 0 0) 0x7fe8c405c110 con 0x7fe8d4105020 2024-04-12T21:50:34.424 INFO:teuthology.orchestra.run.smithi018.stdout:e33 2024-04-12T21:50:34.424 INFO:teuthology.orchestra.run.smithi018.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-12T21:50:34.424 INFO:teuthology.orchestra.run.smithi018.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-12T21:50:34.425 INFO:teuthology.orchestra.run.smithi018.stdout:legacy client fscid: 1 2024-04-12T21:50:34.425 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-12T21:50:34.425 INFO:teuthology.orchestra.run.smithi018.stdout:Filesystem 'cephfs' (1) 2024-04-12T21:50:34.425 INFO:teuthology.orchestra.run.smithi018.stdout:fs_name cephfs 2024-04-12T21:50:34.425 INFO:teuthology.orchestra.run.smithi018.stdout:epoch 24 2024-04-12T21:50:34.425 INFO:teuthology.orchestra.run.smithi018.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-04-12T21:50:34.425 INFO:teuthology.orchestra.run.smithi018.stdout:created 2024-04-12T20:51:57.362850+0000 2024-04-12T21:50:34.426 INFO:teuthology.orchestra.run.smithi018.stdout:modified 2024-04-12T21:06:59.717190+0000 2024-04-12T21:50:34.426 INFO:teuthology.orchestra.run.smithi018.stdout:tableserver 0 2024-04-12T21:50:34.426 INFO:teuthology.orchestra.run.smithi018.stdout:root 0 2024-04-12T21:50:34.426 INFO:teuthology.orchestra.run.smithi018.stdout:session_timeout 60 2024-04-12T21:50:34.426 INFO:teuthology.orchestra.run.smithi018.stdout:session_autoclose 300 2024-04-12T21:50:34.426 INFO:teuthology.orchestra.run.smithi018.stdout:max_file_size 1099511627776 2024-04-12T21:50:34.426 INFO:teuthology.orchestra.run.smithi018.stdout:max_xattr_size 65536 2024-04-12T21:50:34.426 INFO:teuthology.orchestra.run.smithi018.stdout:required_client_features {} 2024-04-12T21:50:34.427 INFO:teuthology.orchestra.run.smithi018.stdout:last_failure 0 2024-04-12T21:50:34.427 INFO:teuthology.orchestra.run.smithi018.stdout:last_failure_osd_epoch 100 2024-04-12T21:50:34.427 INFO:teuthology.orchestra.run.smithi018.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-12T21:50:34.427 INFO:teuthology.orchestra.run.smithi018.stdout:max_mds 2 2024-04-12T21:50:34.427 INFO:teuthology.orchestra.run.smithi018.stdout:in 0,1 2024-04-12T21:50:34.427 INFO:teuthology.orchestra.run.smithi018.stdout:up {} 2024-04-12T21:50:34.428 INFO:teuthology.orchestra.run.smithi018.stdout:failed 0,1 2024-04-12T21:50:34.428 INFO:teuthology.orchestra.run.smithi018.stdout:damaged 2024-04-12T21:50:34.428 INFO:teuthology.orchestra.run.smithi018.stdout:stopped 2024-04-12T21:50:34.428 INFO:teuthology.orchestra.run.smithi018.stdout:data_pools [3] 2024-04-12T21:50:34.428 INFO:teuthology.orchestra.run.smithi018.stdout:metadata_pool 2 2024-04-12T21:50:34.428 INFO:teuthology.orchestra.run.smithi018.stdout:inline_data disabled 2024-04-12T21:50:34.428 INFO:teuthology.orchestra.run.smithi018.stdout:balancer 2024-04-12T21:50:34.428 INFO:teuthology.orchestra.run.smithi018.stdout:bal_rank_mask -1 2024-04-12T21:50:34.429 INFO:teuthology.orchestra.run.smithi018.stdout:standby_count_wanted 1 2024-04-12T21:50:34.429 INFO:teuthology.orchestra.run.smithi018.stdout:qdb_cluster leader: 0 members: 2024-04-12T21:50:34.429 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-12T21:50:34.429 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-12T21:50:34.429 INFO:teuthology.orchestra.run.smithi018.stdout:Standby daemons: 2024-04-12T21:50:34.429 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-12T21:50:34.429 INFO:teuthology.orchestra.run.smithi018.stdout:[mds.cephfs.smithi018.acqbsc{-1:34436} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.18:6828/755179764,v1:172.21.15.18:6829/755179764] compat {c=[1],r=[1],i=[1fff]}] 2024-04-12T21:50:34.429 INFO:teuthology.orchestra.run.smithi018.stdout:[mds.cephfs.smithi018.szjluo{-1:34468} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.18:6826/2407992653,v1:172.21.15.18:6827/2407992653] compat {c=[1],r=[1],i=[1fff]}] 2024-04-12T21:50:34.430 INFO:teuthology.orchestra.run.smithi018.stdout:[mds.cephfs.smithi186.jdllyn{-1:44595} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.186:6824/3523444202,v1:172.21.15.186:6825/3523444202] compat {c=[1],r=[1],i=[1fff]}] 2024-04-12T21:50:34.430 INFO:teuthology.orchestra.run.smithi018.stdout:[mds.cephfs.smithi186.wjnafs{-1:44875} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.186:6826/3122443766,v1:172.21.15.186:6827/3122443766] compat {c=[1],r=[1],i=[1fff]}] 2024-04-12T21:50:34.431 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:50:34.426+0000 7fe8da600700 1 -- 172.21.15.18:0/2411343819 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fe8b806ec00 msgr2=0x7fe8b80710b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:50:34.431 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:50:34.426+0000 7fe8da600700 1 --2- 172.21.15.18:0/2411343819 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fe8b806ec00 0x7fe8b80710b0 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7fe8c0009fd0 tx=0x7fe8c000b040 comp rx=0 tx=0).stop 2024-04-12T21:50:34.432 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:50:34.426+0000 7fe8da600700 1 -- 172.21.15.18:0/2411343819 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe8d4105020 msgr2=0x7fe8d4115bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:50:34.432 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:50:34.426+0000 7fe8da600700 1 --2- 172.21.15.18:0/2411343819 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe8d4105020 0x7fe8d4115bc0 secure :-1 s=READY pgs=1291 cs=0 l=1 rev1=1 crypto rx=0x7fe8c4009180 tx=0x7fe8c4009e30 comp rx=0 tx=0).stop 2024-04-12T21:50:34.432 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:50:34.426+0000 7fe8da600700 1 -- 172.21.15.18:0/2411343819 shutdown_connections 2024-04-12T21:50:34.432 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:50:34.426+0000 7fe8da600700 1 --2- 172.21.15.18:0/2411343819 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fe8b806ec00 0x7fe8b80710b0 unknown :-1 s=CLOSED pgs=684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:50:34.432 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:50:34.426+0000 7fe8da600700 1 --2- 172.21.15.18:0/2411343819 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe8d4105020 0x7fe8d4115bc0 unknown :-1 s=CLOSED pgs=1291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:50:34.433 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:50:34.426+0000 7fe8da600700 1 --2- 172.21.15.18:0/2411343819 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe8d41059c0 0x7fe8d4116100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:50:34.433 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:50:34.426+0000 7fe8da600700 1 -- 172.21.15.18:0/2411343819 >> 172.21.15.18:0/2411343819 conn(0x7fe8d40faba0 msgr2=0x7fe8d40fbba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:50:34.433 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:50:34.426+0000 7fe8da600700 1 -- 172.21.15.18:0/2411343819 shutdown_connections 2024-04-12T21:50:34.433 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:50:34.427+0000 7fe8da600700 1 -- 172.21.15.18:0/2411343819 wait complete. 2024-04-12T21:50:34.433 INFO:teuthology.orchestra.run.smithi018.stderr:dumped fsmap epoch 33 2024-04-12T21:50:34.584 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:50:34.583+0000 7f4076600700 1 -- 172.21.15.18:0/3974048595 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f40701059c0 msgr2=0x7f40700ff080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:50:34.585 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:50:34.583+0000 7f4076600700 1 --2- 172.21.15.18:0/3974048595 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f40701059c0 0x7f40700ff080 secure :-1 s=READY pgs=1292 cs=0 l=1 rev1=1 crypto rx=0x7f405c004940 tx=0x7f405c009e30 comp rx=0 tx=0).stop 2024-04-12T21:50:34.585 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:50:34.583+0000 7f4076600700 1 -- 172.21.15.18:0/3974048595 shutdown_connections 2024-04-12T21:50:34.585 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:50:34.583+0000 7f4076600700 1 --2- 172.21.15.18:0/3974048595 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f40701059c0 0x7f40700ff080 unknown :-1 s=CLOSED pgs=1292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:50:34.585 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:50:34.583+0000 7f4076600700 1 --2- 172.21.15.18:0/3974048595 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4070105020 0x7f40701053f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:50:34.585 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:50:34.583+0000 7f4076600700 1 -- 172.21.15.18:0/3974048595 >> 172.21.15.18:0/3974048595 conn(0x7f40700faba0 msgr2=0x7f40700fcfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:50:34.585 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:50:34.584+0000 7f4076600700 1 -- 172.21.15.18:0/3974048595 shutdown_connections 2024-04-12T21:50:34.585 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:50:34.584+0000 7f4076600700 1 -- 172.21.15.18:0/3974048595 wait complete. 2024-04-12T21:50:34.586 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:50:34.584+0000 7f4076600700 1 Processor -- start 2024-04-12T21:50:34.586 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:50:34.584+0000 7f4076600700 1 -- start start 2024-04-12T21:50:34.586 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:50:34.585+0000 7f4076600700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4070105020 0x7f4070115ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:50:34.587 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:50:34.585+0000 7f4076600700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f40701059c0 0x7f40701160e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:50:34.587 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:50:34.585+0000 7f4076600700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4070116730 con 0x7f4070105020 2024-04-12T21:50:34.587 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:50:34.585+0000 7f4076600700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f407010fd10 con 0x7f40701059c0 2024-04-12T21:50:34.587 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:50:34.585+0000 7f406f400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4070105020 0x7f4070115ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:50:34.587 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:50:34.585+0000 7f406f400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4070105020 0x7f4070115ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:45860/0 (socket says 172.21.15.18:45860) 2024-04-12T21:50:34.587 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:50:34.585+0000 7f406f400700 1 -- 172.21.15.18:0/1620183777 learned_addr learned my addr 172.21.15.18:0/1620183777 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:50:34.587 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:50:34.585+0000 7f406f400700 1 -- 172.21.15.18:0/1620183777 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f40701059c0 msgr2=0x7f40701160e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:50:34.587 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:50:34.585+0000 7f406f400700 1 --2- 172.21.15.18:0/1620183777 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f40701059c0 0x7f40701160e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:50:34.588 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:50:34.585+0000 7f406f400700 1 -- 172.21.15.18:0/1620183777 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f405c005040 con 0x7f4070105020 2024-04-12T21:50:34.588 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:50:34.586+0000 7f406f400700 1 --2- 172.21.15.18:0/1620183777 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4070105020 0x7f4070115ba0 secure :-1 s=READY pgs=1293 cs=0 l=1 rev1=1 crypto rx=0x7f4060002a50 tx=0x7f40600046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:50:34.588 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:50:34.586+0000 7f4067400700 1 -- 172.21.15.18:0/1620183777 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4060007cc0 con 0x7f4070105020 2024-04-12T21:50:34.589 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:50:34.586+0000 7f4067400700 1 -- 172.21.15.18:0/1620183777 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f406000f040 con 0x7f4070105020 2024-04-12T21:50:34.589 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:50:34.586+0000 7f4076600700 1 -- 172.21.15.18:0/1620183777 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f407010fff0 con 0x7f4070105020 2024-04-12T21:50:34.589 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:50:34.586+0000 7f4067400700 1 -- 172.21.15.18:0/1620183777 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f406000db70 con 0x7f4070105020 2024-04-12T21:50:34.589 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:50:34.586+0000 7f4076600700 1 -- 172.21.15.18:0/1620183777 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f40701105b0 con 0x7f4070105020 2024-04-12T21:50:34.589 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:50:34.587+0000 7f4076600700 1 -- 172.21.15.18:0/1620183777 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4070100770 con 0x7f4070105020 2024-04-12T21:50:34.589 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:50:34.588+0000 7f4067400700 1 -- 172.21.15.18:0/1620183777 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7f406000dd10 con 0x7f4070105020 2024-04-12T21:50:34.590 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:50:34.589+0000 7f4067400700 1 --2- 172.21.15.18:0/1620183777 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f405406ec00 0x7f40540710b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:50:34.591 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:50:34.589+0000 7f4067400700 1 -- 172.21.15.18:0/1620183777 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 6757+0+0 (secure 0 0 0) 0x7f406008dff0 con 0x7f4070105020 2024-04-12T21:50:34.591 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:50:34.590+0000 7f406ea00700 1 --2- 172.21.15.18:0/1620183777 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f405406ec00 0x7f40540710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:50:34.591 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:50:34.591+0000 7f406ea00700 1 --2- 172.21.15.18:0/1620183777 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f405406ec00 0x7f40540710b0 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7f405c004f90 tx=0x7f405c009e30 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:50:34.595 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:50:34.594+0000 7f4067400700 1 -- 172.21.15.18:0/1620183777 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f406005a800 con 0x7f4070105020 2024-04-12T21:50:34.837 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:50:34.836+0000 7f4076600700 1 -- 172.21.15.18:0/1620183777 --> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f407010c120 con 0x7f405406ec00 2024-04-12T21:50:34.839 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:50:34.838+0000 7f4067400700 1 -- 172.21.15.18:0/1620183777 <== mgr.34102 v2:172.21.15.18:6800/3812891919 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f407010c120 con 0x7f405406ec00 2024-04-12T21:50:34.840 INFO:teuthology.orchestra.run.smithi018.stdout:{ 2024-04-12T21:50:34.840 INFO:teuthology.orchestra.run.smithi018.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:f4452def3be4d350f4a8e5feaf7ebb1511be9298fe68478ba7a2d6fbd7d69a63", 2024-04-12T21:50:34.840 INFO:teuthology.orchestra.run.smithi018.stdout: "in_progress": true, 2024-04-12T21:50:34.840 INFO:teuthology.orchestra.run.smithi018.stdout: "which": "Upgrading all daemon types on all hosts", 2024-04-12T21:50:34.840 INFO:teuthology.orchestra.run.smithi018.stdout: "services_complete": [ 2024-04-12T21:50:34.840 INFO:teuthology.orchestra.run.smithi018.stdout: "mgr", 2024-04-12T21:50:34.840 INFO:teuthology.orchestra.run.smithi018.stdout: "mds", 2024-04-12T21:50:34.840 INFO:teuthology.orchestra.run.smithi018.stdout: "crash", 2024-04-12T21:50:34.840 INFO:teuthology.orchestra.run.smithi018.stdout: "osd", 2024-04-12T21:50:34.840 INFO:teuthology.orchestra.run.smithi018.stdout: "mon" 2024-04-12T21:50:34.840 INFO:teuthology.orchestra.run.smithi018.stdout: ], 2024-04-12T21:50:34.841 INFO:teuthology.orchestra.run.smithi018.stdout: "progress": "16/21 daemons upgraded", 2024-04-12T21:50:34.841 INFO:teuthology.orchestra.run.smithi018.stdout: "message": "Currently upgrading mds daemons", 2024-04-12T21:50:34.841 INFO:teuthology.orchestra.run.smithi018.stdout: "is_paused": false 2024-04-12T21:50:34.841 INFO:teuthology.orchestra.run.smithi018.stdout:} 2024-04-12T21:50:34.843 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:50:34.841+0000 7f4076600700 1 -- 172.21.15.18:0/1620183777 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f405406ec00 msgr2=0x7f40540710b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:50:34.843 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:50:34.841+0000 7f4076600700 1 --2- 172.21.15.18:0/1620183777 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f405406ec00 0x7f40540710b0 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7f405c004f90 tx=0x7f405c009e30 comp rx=0 tx=0).stop 2024-04-12T21:50:34.843 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:50:34.841+0000 7f4076600700 1 -- 172.21.15.18:0/1620183777 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4070105020 msgr2=0x7f4070115ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:50:34.843 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:50:34.841+0000 7f4076600700 1 --2- 172.21.15.18:0/1620183777 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4070105020 0x7f4070115ba0 secure :-1 s=READY pgs=1293 cs=0 l=1 rev1=1 crypto rx=0x7f4060002a50 tx=0x7f40600046b0 comp rx=0 tx=0).stop 2024-04-12T21:50:34.843 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:50:34.841+0000 7f4076600700 1 -- 172.21.15.18:0/1620183777 shutdown_connections 2024-04-12T21:50:34.843 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:50:34.841+0000 7f4076600700 1 --2- 172.21.15.18:0/1620183777 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f405406ec00 0x7f40540710b0 unknown :-1 s=CLOSED pgs=685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:50:34.843 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:50:34.841+0000 7f4076600700 1 --2- 172.21.15.18:0/1620183777 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4070105020 0x7f4070115ba0 unknown :-1 s=CLOSED pgs=1293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:50:34.843 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:50:34.841+0000 7f4076600700 1 --2- 172.21.15.18:0/1620183777 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f40701059c0 0x7f40701160e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:50:34.843 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:50:34.841+0000 7f4076600700 1 -- 172.21.15.18:0/1620183777 >> 172.21.15.18:0/1620183777 conn(0x7f40700faba0 msgr2=0x7f40700fbac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:50:34.843 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:50:34.842+0000 7f4076600700 1 -- 172.21.15.18:0/1620183777 shutdown_connections 2024-04-12T21:50:34.843 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:50:34.842+0000 7f4076600700 1 -- 172.21.15.18:0/1620183777 wait complete. 2024-04-12T21:50:34.998 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:50:34.997+0000 7fb889a00700 1 -- 172.21.15.18:0/3878471075 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb884109dd0 msgr2=0x7fb884103440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:50:34.998 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:50:34.997+0000 7fb889a00700 1 --2- 172.21.15.18:0/3878471075 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb884109dd0 0x7fb884103440 secure :-1 s=READY pgs=1294 cs=0 l=1 rev1=1 crypto rx=0x7fb870004940 tx=0x7fb870009e30 comp rx=0 tx=0).stop 2024-04-12T21:50:34.999 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:50:34.997+0000 7fb889a00700 1 -- 172.21.15.18:0/3878471075 shutdown_connections 2024-04-12T21:50:34.999 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:50:34.997+0000 7fb889a00700 1 --2- 172.21.15.18:0/3878471075 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb884109dd0 0x7fb884103440 unknown :-1 s=CLOSED pgs=1294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:50:34.999 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:50:34.997+0000 7fb889a00700 1 --2- 172.21.15.18:0/3878471075 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb884109430 0x7fb884109800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:50:34.999 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:50:34.997+0000 7fb889a00700 1 -- 172.21.15.18:0/3878471075 >> 172.21.15.18:0/3878471075 conn(0x7fb884072c00 msgr2=0x7fb884075010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:50:34.999 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:50:34.997+0000 7fb889a00700 1 -- 172.21.15.18:0/3878471075 shutdown_connections 2024-04-12T21:50:34.999 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:50:34.998+0000 7fb889a00700 1 -- 172.21.15.18:0/3878471075 wait complete. 2024-04-12T21:50:35.000 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:50:34.998+0000 7fb889a00700 1 Processor -- start 2024-04-12T21:50:35.000 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:50:34.998+0000 7fb889a00700 1 -- start start 2024-04-12T21:50:35.000 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:50:34.999+0000 7fb889a00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb884109430 0x7fb884119fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:50:35.000 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:50:34.999+0000 7fb889a00700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb884109dd0 0x7fb88411a4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:50:35.000 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:50:34.999+0000 7fb889a00700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb88411abc0 con 0x7fb884109430 2024-04-12T21:50:35.000 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:50:34.999+0000 7fb889a00700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb884114110 con 0x7fb884109dd0 2024-04-12T21:50:35.000 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:50:34.999+0000 7fb882a00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb884109430 0x7fb884119fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:50:35.000 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:50:34.999+0000 7fb882a00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb884109430 0x7fb884119fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:45868/0 (socket says 172.21.15.18:45868) 2024-04-12T21:50:35.000 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:50:34.999+0000 7fb882a00700 1 -- 172.21.15.18:0/2062131110 learned_addr learned my addr 172.21.15.18:0/2062131110 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:50:35.001 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:50:35.000+0000 7fb882000700 1 --2- 172.21.15.18:0/2062131110 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb884109dd0 0x7fb88411a4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:50:35.002 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:50:35.000+0000 7fb882a00700 1 -- 172.21.15.18:0/2062131110 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb884109dd0 msgr2=0x7fb88411a4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:50:35.002 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:50:35.000+0000 7fb882a00700 1 --2- 172.21.15.18:0/2062131110 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb884109dd0 0x7fb88411a4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:50:35.002 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:50:35.000+0000 7fb882a00700 1 -- 172.21.15.18:0/2062131110 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb870005040 con 0x7fb884109430 2024-04-12T21:50:35.002 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:50:35.000+0000 7fb882a00700 1 --2- 172.21.15.18:0/2062131110 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb884109430 0x7fb884119fa0 secure :-1 s=READY pgs=1295 cs=0 l=1 rev1=1 crypto rx=0x7fb87400b790 tx=0x7fb874009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:50:35.002 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:50:35.000+0000 7fb87aa00700 1 -- 172.21.15.18:0/2062131110 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb87400f770 con 0x7fb884109430 2024-04-12T21:50:35.002 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:50:35.000+0000 7fb87aa00700 1 -- 172.21.15.18:0/2062131110 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb87400fda0 con 0x7fb884109430 2024-04-12T21:50:35.002 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:50:35.000+0000 7fb889a00700 1 -- 172.21.15.18:0/2062131110 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb884114390 con 0x7fb884109430 2024-04-12T21:50:35.002 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:50:35.000+0000 7fb87aa00700 1 -- 172.21.15.18:0/2062131110 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb87400d310 con 0x7fb884109430 2024-04-12T21:50:35.002 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:50:35.001+0000 7fb889a00700 1 -- 172.21.15.18:0/2062131110 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb884114950 con 0x7fb884109430 2024-04-12T21:50:35.002 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:50:35.002+0000 7fb889a00700 1 -- 172.21.15.18:0/2062131110 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb884104b80 con 0x7fb884109430 2024-04-12T21:50:35.003 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:50:35.003+0000 7fb87aa00700 1 -- 172.21.15.18:0/2062131110 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7fb874010420 con 0x7fb884109430 2024-04-12T21:50:35.005 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:50:35.004+0000 7fb87aa00700 1 --2- 172.21.15.18:0/2062131110 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fb86806ec00 0x7fb8680710b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:50:35.005 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:50:35.004+0000 7fb87aa00700 1 -- 172.21.15.18:0/2062131110 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 6757+0+0 (secure 0 0 0) 0x7fb87408e9a0 con 0x7fb884109430 2024-04-12T21:50:35.005 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:50:35.005+0000 7fb882000700 1 --2- 172.21.15.18:0/2062131110 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fb86806ec00 0x7fb8680710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:50:35.006 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:50:35.005+0000 7fb882000700 1 --2- 172.21.15.18:0/2062131110 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fb86806ec00 0x7fb8680710b0 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7fb87000b020 tx=0x7fb870009e30 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:50:35.010 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:50:35.009+0000 7fb87aa00700 1 -- 172.21.15.18:0/2062131110 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7fb87405b1d0 con 0x7fb884109430 2024-04-12T21:50:35.105 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:50:34 smithi018 ceph-mon[112590]: from='client.36644 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:50:35.105 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:50:34 smithi018 ceph-mon[112590]: from='client.? 172.21.15.18:0/915743825' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T21:50:35.105 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:50:34 smithi018 ceph-mon[112590]: from='client.? 172.21.15.18:0/2411343819' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-12T21:50:35.284 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:50:34 smithi186 ceph-mon[86379]: from='client.36644 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:50:35.284 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:50:34 smithi186 ceph-mon[86379]: from='client.? 172.21.15.18:0/915743825' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T21:50:35.284 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:50:34 smithi186 ceph-mon[86379]: from='client.? 172.21.15.18:0/2411343819' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-12T21:50:35.328 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:50:35.327+0000 7fb889a00700 1 -- 172.21.15.18:0/2062131110 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7fb884115620 con 0x7fb884109430 2024-04-12T21:50:35.329 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:50:35.328+0000 7fb87aa00700 1 -- 172.21.15.18:0/2062131110 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+4303 (secure 0 0 0) 0x7fb87405ad60 con 0x7fb884109430 2024-04-12T21:50:35.330 INFO:teuthology.orchestra.run.smithi018.stdout:HEALTH_ERR failed to probe daemons or devices; 1 filesystem is degraded; 1 filesystem is offline 2024-04-12T21:50:35.330 INFO:teuthology.orchestra.run.smithi018.stdout:[WRN] CEPHADM_REFRESH_FAILED: failed to probe daemons or devices 2024-04-12T21:50:35.330 INFO:teuthology.orchestra.run.smithi018.stdout: host smithi186 `cephadm ceph-volume` failed: cephadm exited with an error code: 1, stderr: Inferring config /var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/mon.smithi186/config 2024-04-12T21:50:35.330 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/podman: timeout after 895 seconds 2024-04-12T21:50:35.330 INFO:teuthology.orchestra.run.smithi018.stdout:Non-zero exit code 124 from /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:f4452def3be4d350f4a8e5feaf7ebb1511be9298fe68478ba7a2d6fbd7d69a63 -e NODE_NAME=smithi186 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968:/var/run/ceph:z -v /var/log/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968:/var/log/ceph:z -v /var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmptgom53f9:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:f4452def3be4d350f4a8e5feaf7ebb1511be9298fe68478ba7a2d6fbd7d69a63 inventory --format=json-pretty --filter-for-batch 2024-04-12T21:50:35.330 INFO:teuthology.orchestra.run.smithi018.stdout:Traceback (most recent call last): 2024-04-12T21:50:35.330 INFO:teuthology.orchestra.run.smithi018.stdout: File "/usr/lib64/python3.9/runpy.py", line 197, in _run_module_as_main 2024-04-12T21:50:35.330 INFO:teuthology.orchestra.run.smithi018.stdout: return _run_code(code, main_globals, None, 2024-04-12T21:50:35.330 INFO:teuthology.orchestra.run.smithi018.stdout: File "/usr/lib64/python3.9/runpy.py", line 87, in _run_code 2024-04-12T21:50:35.330 INFO:teuthology.orchestra.run.smithi018.stdout: exec(code, run_globals) 2024-04-12T21:50:35.331 INFO:teuthology.orchestra.run.smithi018.stdout: File "/var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/cephadm.e44f3ed5e34e4ef52eedef233d1190bff004470d8e8005127b5312bff357fa7f/__main__.py", line 5618, in 2024-04-12T21:50:35.331 INFO:teuthology.orchestra.run.smithi018.stdout: File "/var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/cephadm.e44f3ed5e34e4ef52eedef233d1190bff004470d8e8005127b5312bff357fa7f/__main__.py", line 5606, in main 2024-04-12T21:50:35.331 INFO:teuthology.orchestra.run.smithi018.stdout: File "/var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/cephadm.e44f3ed5e34e4ef52eedef233d1190bff004470d8e8005127b5312bff357fa7f/__main__.py", line 408, in _infer_config 2024-04-12T21:50:35.331 INFO:teuthology.orchestra.run.smithi018.stdout: File "/var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/cephadm.e44f3ed5e34e4ef52eedef233d1190bff004470d8e8005127b5312bff357fa7f/__main__.py", line 323, in _infer_fsid 2024-04-12T21:50:35.331 INFO:teuthology.orchestra.run.smithi018.stdout: File "/var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/cephadm.e44f3ed5e34e4ef52eedef233d1190bff004470d8e8005127b5312bff357fa7f/__main__.py", line 436, in _infer_image 2024-04-12T21:50:35.331 INFO:teuthology.orchestra.run.smithi018.stdout: File "/var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/cephadm.e44f3ed5e34e4ef52eedef233d1190bff004470d8e8005127b5312bff357fa7f/__main__.py", line 310, in _validate_fsid 2024-04-12T21:50:35.331 INFO:teuthology.orchestra.run.smithi018.stdout: File "/var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/cephadm.e44f3ed5e34e4ef52eedef233d1190bff004470d8e8005127b5312bff357fa7f/__main__.py", line 3359, in command_ceph_volume 2024-04-12T21:50:35.331 INFO:teuthology.orchestra.run.smithi018.stdout: File "/var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/cephadm.e44f3ed5e34e4ef52eedef233d1190bff004470d8e8005127b5312bff357fa7f/cephadmlib/call_wrappers.py", line 310, in call_throws 2024-04-12T21:50:35.331 INFO:teuthology.orchestra.run.smithi018.stdout:RuntimeError: Failed command: /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:f4452def3be4d350f4a8e5feaf7ebb1511be9298fe68478ba7a2d6fbd7d69a63 -e NODE_NAME=smithi186 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968:/var/run/ceph:z -v /var/log/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968:/var/log/ceph:z -v /var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmptgom53f9:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:f4452def3be4d350f4a8e5feaf7ebb1511be9298fe68478ba7a2d6fbd7d69a63 inventory --format=json-pretty --filter-for-batch 2024-04-12T21:50:35.331 INFO:teuthology.orchestra.run.smithi018.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-04-12T21:50:35.331 INFO:teuthology.orchestra.run.smithi018.stdout: fs cephfs is degraded 2024-04-12T21:50:35.331 INFO:teuthology.orchestra.run.smithi018.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-04-12T21:50:35.331 INFO:teuthology.orchestra.run.smithi018.stdout: fs cephfs is offline because no MDS is active for it. 2024-04-12T21:50:35.332 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:50:35.331+0000 7fb889a00700 1 -- 172.21.15.18:0/2062131110 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fb86806ec00 msgr2=0x7fb8680710b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:50:35.332 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:50:35.331+0000 7fb889a00700 1 --2- 172.21.15.18:0/2062131110 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fb86806ec00 0x7fb8680710b0 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7fb87000b020 tx=0x7fb870009e30 comp rx=0 tx=0).stop 2024-04-12T21:50:35.332 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:50:35.331+0000 7fb889a00700 1 -- 172.21.15.18:0/2062131110 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb884109430 msgr2=0x7fb884119fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:50:35.332 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:50:35.331+0000 7fb889a00700 1 --2- 172.21.15.18:0/2062131110 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb884109430 0x7fb884119fa0 secure :-1 s=READY pgs=1295 cs=0 l=1 rev1=1 crypto rx=0x7fb87400b790 tx=0x7fb874009250 comp rx=0 tx=0).stop 2024-04-12T21:50:35.332 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:50:35.331+0000 7fb889a00700 1 -- 172.21.15.18:0/2062131110 shutdown_connections 2024-04-12T21:50:35.332 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:50:35.331+0000 7fb889a00700 1 --2- 172.21.15.18:0/2062131110 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fb86806ec00 0x7fb8680710b0 unknown :-1 s=CLOSED pgs=686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:50:35.332 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:50:35.331+0000 7fb889a00700 1 --2- 172.21.15.18:0/2062131110 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb884109430 0x7fb884119fa0 unknown :-1 s=CLOSED pgs=1295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:50:35.332 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:50:35.331+0000 7fb889a00700 1 --2- 172.21.15.18:0/2062131110 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb884109dd0 0x7fb88411a4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:50:35.333 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:50:35.331+0000 7fb889a00700 1 -- 172.21.15.18:0/2062131110 >> 172.21.15.18:0/2062131110 conn(0x7fb884072c00 msgr2=0x7fb884074b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:50:35.333 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:50:35.332+0000 7fb889a00700 1 -- 172.21.15.18:0/2062131110 shutdown_connections 2024-04-12T21:50:35.333 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:50:35.332+0000 7fb889a00700 1 -- 172.21.15.18:0/2062131110 wait complete. 2024-04-12T21:50:36.284 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:50:35 smithi186 ceph-mon[86379]: pgmap v1527: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:50:36.284 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:50:35 smithi186 ceph-mon[86379]: from='client.36656 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:50:36.285 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:50:35 smithi186 ceph-mon[86379]: from='client.? 172.21.15.18:0/2062131110' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-12T21:50:36.355 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:50:35 smithi018 ceph-mon[112590]: pgmap v1527: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:50:36.355 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:50:35 smithi018 ceph-mon[112590]: from='client.36656 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:50:36.355 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:50:35 smithi018 ceph-mon[112590]: from='client.? 172.21.15.18:0/2062131110' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-12T21:50:38.284 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:50:37 smithi186 ceph-mon[86379]: pgmap v1528: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:50:38.355 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:50:37 smithi018 ceph-mon[112590]: pgmap v1528: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:50:40.284 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:50:39 smithi186 ceph-mon[86379]: pgmap v1529: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:50:40.284 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:50:39 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T21:50:40.355 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:50:39 smithi018 ceph-mon[112590]: pgmap v1529: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:50:40.355 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:50:39 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T21:50:42.284 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:50:42 smithi186 ceph-mon[86379]: pgmap v1530: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:50:42.355 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:50:41 smithi018 ceph-mon[112590]: pgmap v1530: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:50:44.284 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:50:44 smithi186 ceph-mon[86379]: pgmap v1531: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:50:44.355 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:50:44 smithi018 ceph-mon[112590]: pgmap v1531: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:50:46.284 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:50:46 smithi186 ceph-mon[86379]: pgmap v1532: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:50:46.355 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:50:46 smithi018 ceph-mon[112590]: pgmap v1532: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:50:48.284 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:50:48 smithi186 ceph-mon[86379]: pgmap v1533: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:50:48.355 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:50:48 smithi018 ceph-mon[112590]: pgmap v1533: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:50:50.284 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:50:50 smithi186 ceph-mon[86379]: pgmap v1534: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:50:50.355 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:50:50 smithi018 ceph-mon[112590]: pgmap v1534: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:50:52.284 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:50:52 smithi186 ceph-mon[86379]: pgmap v1535: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:50:52.355 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:50:52 smithi018 ceph-mon[112590]: pgmap v1535: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:50:54.358 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:50:54 smithi018 ceph-mon[112590]: pgmap v1536: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:50:54.534 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:50:54 smithi186 ceph-mon[86379]: pgmap v1536: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:50:55.355 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:50:55 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T21:50:55.534 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:50:55 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T21:50:56.355 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:50:56 smithi018 ceph-mon[112590]: pgmap v1537: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:50:56.534 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:50:56 smithi186 ceph-mon[86379]: pgmap v1537: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:50:58.337 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:50:58 smithi186 ceph-mon[86379]: pgmap v1538: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:50:58.355 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:50:58 smithi018 ceph-mon[112590]: pgmap v1538: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:51:00.355 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:51:00 smithi018 ceph-mon[112590]: pgmap v1539: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:51:00.534 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:51:00 smithi186 ceph-mon[86379]: pgmap v1539: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:51:02.358 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:51:02 smithi018 ceph-mon[112590]: pgmap v1540: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:51:02.534 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:51:02 smithi186 ceph-mon[86379]: pgmap v1540: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:51:04.355 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:51:04 smithi018 ceph-mon[112590]: pgmap v1541: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:51:04.534 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:51:04 smithi186 ceph-mon[86379]: pgmap v1541: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:51:05.505 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:05.503+0000 7f01d4800700 1 -- 172.21.15.18:0/2505742431 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f01cc1037a0 msgr2=0x7f01cc0fce10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:51:05.506 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:05.503+0000 7f01d4800700 1 --2- 172.21.15.18:0/2505742431 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f01cc1037a0 0x7f01cc0fce10 secure :-1 s=READY pgs=1296 cs=0 l=1 rev1=1 crypto rx=0x7f01bc004940 tx=0x7f01bc009e30 comp rx=0 tx=0).stop 2024-04-12T21:51:05.506 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:05.503+0000 7f01d4800700 1 -- 172.21.15.18:0/2505742431 shutdown_connections 2024-04-12T21:51:05.506 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:05.503+0000 7f01d4800700 1 --2- 172.21.15.18:0/2505742431 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f01cc1037a0 0x7f01cc0fce10 unknown :-1 s=CLOSED pgs=1296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:51:05.506 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:05.503+0000 7f01d4800700 1 --2- 172.21.15.18:0/2505742431 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f01cc102e00 0x7f01cc1031d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:51:05.506 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:05.503+0000 7f01d4800700 1 -- 172.21.15.18:0/2505742431 >> 172.21.15.18:0/2505742431 conn(0x7f01cc0f8970 msgr2=0x7f01cc0fad80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:51:05.506 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:05.503+0000 7f01d4800700 1 -- 172.21.15.18:0/2505742431 shutdown_connections 2024-04-12T21:51:05.506 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:05.503+0000 7f01d4800700 1 -- 172.21.15.18:0/2505742431 wait complete. 2024-04-12T21:51:05.507 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:05.504+0000 7f01d4800700 1 Processor -- start 2024-04-12T21:51:05.507 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:05.504+0000 7f01d4800700 1 -- start start 2024-04-12T21:51:05.508 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:05.505+0000 7f01d4800700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f01cc102e00 0x7f01cc115da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:51:05.508 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:05.505+0000 7f01d4800700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f01cc1037a0 0x7f01cc1162e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:51:05.508 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:05.505+0000 7f01d4800700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01cc116930 con 0x7f01cc1037a0 2024-04-12T21:51:05.508 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:05.505+0000 7f01d4800700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01cc10fd00 con 0x7f01cc102e00 2024-04-12T21:51:05.508 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:05.505+0000 7f01d1400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f01cc1037a0 0x7f01cc1162e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:51:05.508 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:05.505+0000 7f01d1400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f01cc1037a0 0x7f01cc1162e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:57158/0 (socket says 172.21.15.18:57158) 2024-04-12T21:51:05.508 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:05.505+0000 7f01d1400700 1 -- 172.21.15.18:0/1371843675 learned_addr learned my addr 172.21.15.18:0/1371843675 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:51:05.508 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:05.506+0000 7f01d1e00700 1 --2- 172.21.15.18:0/1371843675 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f01cc102e00 0x7f01cc115da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:51:05.508 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:05.506+0000 7f01d1400700 1 -- 172.21.15.18:0/1371843675 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f01cc102e00 msgr2=0x7f01cc115da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:51:05.508 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:05.506+0000 7f01d1400700 1 --2- 172.21.15.18:0/1371843675 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f01cc102e00 0x7f01cc115da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:51:05.508 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:05.506+0000 7f01d1400700 1 -- 172.21.15.18:0/1371843675 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01bc005040 con 0x7f01cc1037a0 2024-04-12T21:51:05.509 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:05.506+0000 7f01d1400700 1 --2- 172.21.15.18:0/1371843675 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f01cc1037a0 0x7f01cc1162e0 secure :-1 s=READY pgs=1297 cs=0 l=1 rev1=1 crypto rx=0x7f01bc009100 tx=0x7f01bc009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:51:05.509 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:05.506+0000 7f01c6000700 1 -- 172.21.15.18:0/1371843675 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f01bc00e040 con 0x7f01cc1037a0 2024-04-12T21:51:05.509 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:05.507+0000 7f01c6000700 1 -- 172.21.15.18:0/1371843675 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f01bc013420 con 0x7f01cc1037a0 2024-04-12T21:51:05.509 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:05.507+0000 7f01d4800700 1 -- 172.21.15.18:0/1371843675 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f01cc10ff80 con 0x7f01cc1037a0 2024-04-12T21:51:05.510 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:05.507+0000 7f01c6000700 1 -- 172.21.15.18:0/1371843675 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f01bc01c630 con 0x7f01cc1037a0 2024-04-12T21:51:05.510 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:05.507+0000 7f01d4800700 1 -- 172.21.15.18:0/1371843675 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f01cc110540 con 0x7f01cc1037a0 2024-04-12T21:51:05.510 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:05.508+0000 7f01d4800700 1 -- 172.21.15.18:0/1371843675 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f01cc102120 con 0x7f01cc1037a0 2024-04-12T21:51:05.511 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:05.509+0000 7f01c6000700 1 -- 172.21.15.18:0/1371843675 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7f01bc013590 con 0x7f01cc1037a0 2024-04-12T21:51:05.512 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:05.510+0000 7f01c6000700 1 --2- 172.21.15.18:0/1371843675 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f01b406ec10 0x7f01b40710c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:51:05.512 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:05.510+0000 7f01c6000700 1 -- 172.21.15.18:0/1371843675 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 6757+0+0 (secure 0 0 0) 0x7f01bc08fc70 con 0x7f01cc1037a0 2024-04-12T21:51:05.512 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:05.510+0000 7f01d1e00700 1 --2- 172.21.15.18:0/1371843675 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f01b406ec10 0x7f01b40710c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:51:05.513 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:05.511+0000 7f01d1e00700 1 --2- 172.21.15.18:0/1371843675 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f01b406ec10 0x7f01b40710c0 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f01c0007fd0 tx=0x7f01c0009040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:51:05.517 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:05.515+0000 7f01c6000700 1 -- 172.21.15.18:0/1371843675 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f01bc096080 con 0x7f01cc1037a0 2024-04-12T21:51:05.769 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:05.767+0000 7f01d4800700 1 -- 172.21.15.18:0/1371843675 --> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f01cc111460 con 0x7f01b406ec10 2024-04-12T21:51:05.772 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:05.770+0000 7f01c6000700 1 -- 172.21.15.18:0/1371843675 <== mgr.34102 v2:172.21.15.18:6800/3812891919 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f01cc111460 con 0x7f01b406ec10 2024-04-12T21:51:05.775 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:05.772+0000 7f01d4800700 1 -- 172.21.15.18:0/1371843675 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f01b406ec10 msgr2=0x7f01b40710c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:51:05.775 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:05.772+0000 7f01d4800700 1 --2- 172.21.15.18:0/1371843675 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f01b406ec10 0x7f01b40710c0 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f01c0007fd0 tx=0x7f01c0009040 comp rx=0 tx=0).stop 2024-04-12T21:51:05.775 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:05.772+0000 7f01d4800700 1 -- 172.21.15.18:0/1371843675 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f01cc1037a0 msgr2=0x7f01cc1162e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:51:05.775 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:05.772+0000 7f01d4800700 1 --2- 172.21.15.18:0/1371843675 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f01cc1037a0 0x7f01cc1162e0 secure :-1 s=READY pgs=1297 cs=0 l=1 rev1=1 crypto rx=0x7f01bc009100 tx=0x7f01bc009e30 comp rx=0 tx=0).stop 2024-04-12T21:51:05.775 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:05.772+0000 7f01d4800700 1 -- 172.21.15.18:0/1371843675 shutdown_connections 2024-04-12T21:51:05.775 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:05.773+0000 7f01d4800700 1 --2- 172.21.15.18:0/1371843675 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f01b406ec10 0x7f01b40710c0 unknown :-1 s=CLOSED pgs=687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:51:05.775 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:05.773+0000 7f01d4800700 1 --2- 172.21.15.18:0/1371843675 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f01cc102e00 0x7f01cc115da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:51:05.775 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:05.773+0000 7f01d4800700 1 --2- 172.21.15.18:0/1371843675 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f01cc1037a0 0x7f01cc1162e0 unknown :-1 s=CLOSED pgs=1297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:51:05.775 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:05.773+0000 7f01d4800700 1 -- 172.21.15.18:0/1371843675 >> 172.21.15.18:0/1371843675 conn(0x7f01cc0f8970 msgr2=0x7f01cc070b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:51:05.775 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:05.773+0000 7f01d4800700 1 -- 172.21.15.18:0/1371843675 shutdown_connections 2024-04-12T21:51:05.775 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:05.773+0000 7f01d4800700 1 -- 172.21.15.18:0/1371843675 wait complete. 2024-04-12T21:51:05.791 INFO:teuthology.orchestra.run.smithi018.stdout:true 2024-04-12T21:51:05.930 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:05.928+0000 7f803de00700 1 -- 172.21.15.18:0/1332073208 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f80381059c0 msgr2=0x7f80380ff080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:51:05.931 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:05.928+0000 7f803de00700 1 --2- 172.21.15.18:0/1332073208 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f80381059c0 0x7f80380ff080 secure :-1 s=READY pgs=1298 cs=0 l=1 rev1=1 crypto rx=0x7f8024004940 tx=0x7f8024009e30 comp rx=0 tx=0).stop 2024-04-12T21:51:05.931 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:05.928+0000 7f803de00700 1 -- 172.21.15.18:0/1332073208 shutdown_connections 2024-04-12T21:51:05.931 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:05.928+0000 7f803de00700 1 --2- 172.21.15.18:0/1332073208 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f80381059c0 0x7f80380ff080 unknown :-1 s=CLOSED pgs=1298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:51:05.931 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:05.928+0000 7f803de00700 1 --2- 172.21.15.18:0/1332073208 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8038105020 0x7f80381053f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:51:05.931 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:05.928+0000 7f803de00700 1 -- 172.21.15.18:0/1332073208 >> 172.21.15.18:0/1332073208 conn(0x7f80380faba0 msgr2=0x7f80380fcfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:51:05.931 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:05.928+0000 7f803de00700 1 -- 172.21.15.18:0/1332073208 shutdown_connections 2024-04-12T21:51:05.931 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:05.928+0000 7f803de00700 1 -- 172.21.15.18:0/1332073208 wait complete. 2024-04-12T21:51:05.932 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:05.929+0000 7f803de00700 1 Processor -- start 2024-04-12T21:51:05.932 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:05.929+0000 7f803de00700 1 -- start start 2024-04-12T21:51:05.932 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:05.930+0000 7f803de00700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8038105020 0x7f8038115ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:51:05.932 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:05.930+0000 7f803de00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f80381059c0 0x7f80381160e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:51:05.932 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:05.930+0000 7f803de00700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8038116730 con 0x7f80381059c0 2024-04-12T21:51:05.932 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:05.930+0000 7f803de00700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f803810fd10 con 0x7f8038105020 2024-04-12T21:51:05.932 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:05.930+0000 7f8036000700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f80381059c0 0x7f80381160e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:51:05.933 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:05.930+0000 7f8036000700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f80381059c0 0x7f80381160e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:57168/0 (socket says 172.21.15.18:57168) 2024-04-12T21:51:05.933 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:05.930+0000 7f8036000700 1 -- 172.21.15.18:0/2128996816 learned_addr learned my addr 172.21.15.18:0/2128996816 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:51:05.934 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:05.930+0000 7f8036a00700 1 --2- 172.21.15.18:0/2128996816 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8038105020 0x7f8038115ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:51:05.934 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:05.931+0000 7f8036000700 1 -- 172.21.15.18:0/2128996816 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8038105020 msgr2=0x7f8038115ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:51:05.934 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:05.931+0000 7f8036000700 1 --2- 172.21.15.18:0/2128996816 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8038105020 0x7f8038115ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:51:05.934 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:05.931+0000 7f8036000700 1 -- 172.21.15.18:0/2128996816 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8024005040 con 0x7f80381059c0 2024-04-12T21:51:05.934 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:05.931+0000 7f8036a00700 1 --2- 172.21.15.18:0/2128996816 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8038105020 0x7f8038115ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-12T21:51:05.934 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:05.931+0000 7f8036000700 1 --2- 172.21.15.18:0/2128996816 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f80381059c0 0x7f80381160e0 secure :-1 s=READY pgs=1299 cs=0 l=1 rev1=1 crypto rx=0x7f8024009100 tx=0x7f8024009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:51:05.934 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:05.931+0000 7f802ea00700 1 -- 172.21.15.18:0/2128996816 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f802400e040 con 0x7f80381059c0 2024-04-12T21:51:05.934 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:05.931+0000 7f802ea00700 1 -- 172.21.15.18:0/2128996816 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f8024004290 con 0x7f80381059c0 2024-04-12T21:51:05.934 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:05.931+0000 7f803de00700 1 -- 172.21.15.18:0/2128996816 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f803810ff90 con 0x7f80381059c0 2024-04-12T21:51:05.934 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:05.931+0000 7f802ea00700 1 -- 172.21.15.18:0/2128996816 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f802401c3f0 con 0x7f80381059c0 2024-04-12T21:51:05.934 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:05.931+0000 7f803de00700 1 -- 172.21.15.18:0/2128996816 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8038110550 con 0x7f80381059c0 2024-04-12T21:51:05.935 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:05.932+0000 7f803de00700 1 -- 172.21.15.18:0/2128996816 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8038100770 con 0x7f80381059c0 2024-04-12T21:51:05.935 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:05.934+0000 7f802ea00700 1 -- 172.21.15.18:0/2128996816 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7f8024013790 con 0x7f80381059c0 2024-04-12T21:51:05.937 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:05.935+0000 7f802ea00700 1 --2- 172.21.15.18:0/2128996816 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f801c06ec00 0x7f801c0710b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:51:05.937 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:05.935+0000 7f802ea00700 1 -- 172.21.15.18:0/2128996816 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 6757+0+0 (secure 0 0 0) 0x7f802408fd10 con 0x7f80381059c0 2024-04-12T21:51:05.937 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:05.935+0000 7f8036a00700 1 --2- 172.21.15.18:0/2128996816 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f801c06ec00 0x7f801c0710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:51:05.938 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:05.936+0000 7f8036a00700 1 --2- 172.21.15.18:0/2128996816 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f801c06ec00 0x7f801c0710b0 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7f8028009fd0 tx=0x7f802800b040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:51:05.942 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:05.940+0000 7f802ea00700 1 -- 172.21.15.18:0/2128996816 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f802405c4c0 con 0x7f80381059c0 2024-04-12T21:51:06.187 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:06.185+0000 7f803de00700 1 -- 172.21.15.18:0/2128996816 --> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f803810c170 con 0x7f801c06ec00 2024-04-12T21:51:06.189 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:06.187+0000 7f802ea00700 1 -- 172.21.15.18:0/2128996816 <== mgr.34102 v2:172.21.15.18:6800/3812891919 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f803810c170 con 0x7f801c06ec00 2024-04-12T21:51:06.194 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:06.192+0000 7f803de00700 1 -- 172.21.15.18:0/2128996816 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f801c06ec00 msgr2=0x7f801c0710b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:51:06.194 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:06.192+0000 7f803de00700 1 --2- 172.21.15.18:0/2128996816 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f801c06ec00 0x7f801c0710b0 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7f8028009fd0 tx=0x7f802800b040 comp rx=0 tx=0).stop 2024-04-12T21:51:06.194 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:06.192+0000 7f803de00700 1 -- 172.21.15.18:0/2128996816 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f80381059c0 msgr2=0x7f80381160e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:51:06.194 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:06.192+0000 7f803de00700 1 --2- 172.21.15.18:0/2128996816 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f80381059c0 0x7f80381160e0 secure :-1 s=READY pgs=1299 cs=0 l=1 rev1=1 crypto rx=0x7f8024009100 tx=0x7f8024009e30 comp rx=0 tx=0).stop 2024-04-12T21:51:06.194 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:06.192+0000 7f803de00700 1 -- 172.21.15.18:0/2128996816 shutdown_connections 2024-04-12T21:51:06.195 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:06.192+0000 7f803de00700 1 --2- 172.21.15.18:0/2128996816 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f801c06ec00 0x7f801c0710b0 unknown :-1 s=CLOSED pgs=688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:51:06.195 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:06.192+0000 7f803de00700 1 --2- 172.21.15.18:0/2128996816 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8038105020 0x7f8038115ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:51:06.195 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:06.192+0000 7f803de00700 1 --2- 172.21.15.18:0/2128996816 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f80381059c0 0x7f80381160e0 unknown :-1 s=CLOSED pgs=1299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:51:06.195 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:06.192+0000 7f803de00700 1 -- 172.21.15.18:0/2128996816 >> 172.21.15.18:0/2128996816 conn(0x7f80380faba0 msgr2=0x7f80380fbb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:51:06.195 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:06.192+0000 7f803de00700 1 -- 172.21.15.18:0/2128996816 shutdown_connections 2024-04-12T21:51:06.195 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:06.192+0000 7f803de00700 1 -- 172.21.15.18:0/2128996816 wait complete. 2024-04-12T21:51:06.348 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:06.346+0000 7f82ec000700 1 -- 172.21.15.18:0/923281471 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f82e4104fb0 msgr2=0x7f82e4105380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:51:06.348 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:06.346+0000 7f82ec000700 1 --2- 172.21.15.18:0/923281471 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f82e4104fb0 0x7f82e4105380 secure :-1 s=READY pgs=1300 cs=0 l=1 rev1=1 crypto rx=0x7f82d8004940 tx=0x7f82d8009e30 comp rx=0 tx=0).stop 2024-04-12T21:51:06.349 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:51:06 smithi018 ceph-mon[112590]: pgmap v1542: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:51:06.349 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:06.346+0000 7f82ec000700 1 -- 172.21.15.18:0/923281471 shutdown_connections 2024-04-12T21:51:06.349 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:06.346+0000 7f82ec000700 1 --2- 172.21.15.18:0/923281471 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f82e4105950 0x7f82e40fefc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:51:06.349 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:06.346+0000 7f82ec000700 1 --2- 172.21.15.18:0/923281471 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f82e4104fb0 0x7f82e4105380 unknown :-1 s=CLOSED pgs=1300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:51:06.349 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:06.346+0000 7f82ec000700 1 -- 172.21.15.18:0/923281471 >> 172.21.15.18:0/923281471 conn(0x7f82e40fab40 msgr2=0x7f82e40fcf50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:51:06.349 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:06.346+0000 7f82ec000700 1 -- 172.21.15.18:0/923281471 shutdown_connections 2024-04-12T21:51:06.350 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:06.346+0000 7f82ec000700 1 -- 172.21.15.18:0/923281471 wait complete. 2024-04-12T21:51:06.350 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:06.347+0000 7f82ec000700 1 Processor -- start 2024-04-12T21:51:06.350 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:06.348+0000 7f82ec000700 1 -- start start 2024-04-12T21:51:06.350 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:06.348+0000 7f82ec000700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f82e4104fb0 0x7f82e4115d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:51:06.351 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:06.348+0000 7f82ec000700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f82e4105950 0x7f82e4116270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:51:06.351 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:06.348+0000 7f82ec000700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82e41168c0 con 0x7f82e4105950 2024-04-12T21:51:06.351 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:06.348+0000 7f82ec000700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82e410fc90 con 0x7f82e4104fb0 2024-04-12T21:51:06.351 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:06.349+0000 7f82e8c00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f82e4105950 0x7f82e4116270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:51:06.352 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:06.349+0000 7f82e8c00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f82e4105950 0x7f82e4116270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:57188/0 (socket says 172.21.15.18:57188) 2024-04-12T21:51:06.352 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:06.349+0000 7f82e8c00700 1 -- 172.21.15.18:0/128850486 learned_addr learned my addr 172.21.15.18:0/128850486 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:51:06.352 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:06.349+0000 7f82e9600700 1 --2- 172.21.15.18:0/128850486 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f82e4104fb0 0x7f82e4115d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:51:06.352 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:06.349+0000 7f82e8c00700 1 -- 172.21.15.18:0/128850486 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f82e4104fb0 msgr2=0x7f82e4115d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:51:06.352 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:06.349+0000 7f82e8c00700 1 --2- 172.21.15.18:0/128850486 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f82e4104fb0 0x7f82e4115d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:51:06.352 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:06.349+0000 7f82e8c00700 1 -- 172.21.15.18:0/128850486 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82d8005040 con 0x7f82e4105950 2024-04-12T21:51:06.352 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:06.350+0000 7f82e8c00700 1 --2- 172.21.15.18:0/128850486 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f82e4105950 0x7f82e4116270 secure :-1 s=READY pgs=1301 cs=0 l=1 rev1=1 crypto rx=0x7f82d0002a50 tx=0x7f82d0004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:51:06.352 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:06.350+0000 7f82e1600700 1 -- 172.21.15.18:0/128850486 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f82d0007cc0 con 0x7f82e4105950 2024-04-12T21:51:06.353 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:06.350+0000 7f82ec000700 1 -- 172.21.15.18:0/128850486 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f82e410ff70 con 0x7f82e4105950 2024-04-12T21:51:06.353 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:06.350+0000 7f82e1600700 1 -- 172.21.15.18:0/128850486 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f82d000f040 con 0x7f82e4105950 2024-04-12T21:51:06.353 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:06.350+0000 7f82e1600700 1 -- 172.21.15.18:0/128850486 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f82d000daf0 con 0x7f82e4105950 2024-04-12T21:51:06.353 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:06.350+0000 7f82ec000700 1 -- 172.21.15.18:0/128850486 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f82e4110530 con 0x7f82e4105950 2024-04-12T21:51:06.354 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:06.352+0000 7f82ec000700 1 -- 172.21.15.18:0/128850486 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f82e4100700 con 0x7f82e4105950 2024-04-12T21:51:06.355 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:06.353+0000 7f82e1600700 1 -- 172.21.15.18:0/128850486 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7f82d0009810 con 0x7f82e4105950 2024-04-12T21:51:06.357 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:06.354+0000 7f82e1600700 1 --2- 172.21.15.18:0/128850486 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f82cc06ec00 0x7f82cc0710b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:51:06.357 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:06.355+0000 7f82e1600700 1 -- 172.21.15.18:0/128850486 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 6757+0+0 (secure 0 0 0) 0x7f82d008e7d0 con 0x7f82e4105950 2024-04-12T21:51:06.357 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:06.355+0000 7f82e9600700 1 --2- 172.21.15.18:0/128850486 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f82cc06ec00 0x7f82cc0710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:51:06.357 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:06.356+0000 7f82e9600700 1 --2- 172.21.15.18:0/128850486 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f82cc06ec00 0x7f82cc0710b0 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f82d800b3b0 tx=0x7f82d8009e30 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:51:06.361 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:06.360+0000 7f82e1600700 1 -- 172.21.15.18:0/128850486 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f82d005af00 con 0x7f82e4105950 2024-04-12T21:51:06.534 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:51:06 smithi186 ceph-mon[86379]: pgmap v1542: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:51:06.606 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:06.604+0000 7f82ec000700 1 -- 172.21.15.18:0/128850486 --> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f82e410c120 con 0x7f82cc06ec00 2024-04-12T21:51:06.619 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:06.617+0000 7f82e1600700 1 -- 172.21.15.18:0/128850486 <== mgr.34102 v2:172.21.15.18:6800/3812891919 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7f82e410c120 con 0x7f82cc06ec00 2024-04-12T21:51:06.620 INFO:teuthology.orchestra.run.smithi018.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-12T21:51:06.620 INFO:teuthology.orchestra.run.smithi018.stdout:alertmanager.smithi018 smithi018 *:9093,9094 running (51m) 12m ago 66m 14.8M - 0.25.0 c8568f914cd2 c332090068e8 2024-04-12T21:51:06.620 INFO:teuthology.orchestra.run.smithi018.stdout:crash.smithi018 smithi018 running (48m) 12m ago 66m 6665k - 19.0.0-2879-g324baf8d 8717720dc5e5 b7f80d1f7982 2024-04-12T21:51:06.620 INFO:teuthology.orchestra.run.smithi018.stdout:crash.smithi186 smithi186 running (47m) 12m ago 65m 6660k - 19.0.0-2879-g324baf8d 8717720dc5e5 a4f55c81b623 2024-04-12T21:51:06.620 INFO:teuthology.orchestra.run.smithi018.stdout:grafana.smithi018 smithi018 *:3000 running (51m) 12m ago 65m 73.8M - 9.4.12 2bacad6d85d8 b08e3bf176dd 2024-04-12T21:51:06.620 INFO:teuthology.orchestra.run.smithi018.stdout:mds.cephfs.smithi018.acqbsc smithi018 running (43m) 12m ago 59m 21.9M - 19.0.0-2879-g324baf8d 8717720dc5e5 021f99ab24ae 2024-04-12T21:51:06.620 INFO:teuthology.orchestra.run.smithi018.stdout:mds.cephfs.smithi018.szjluo smithi018 running (43m) 12m ago 58m 21.7M - 19.0.0-2879-g324baf8d 8717720dc5e5 0003117479d4 2024-04-12T21:51:06.620 INFO:teuthology.orchestra.run.smithi018.stdout:mds.cephfs.smithi186.jdllyn smithi186 running (28m) 12m ago 59m 16.5M - 19.0.0-2879-g324baf8d 8717720dc5e5 21b1f46eef23 2024-04-12T21:51:06.620 INFO:teuthology.orchestra.run.smithi018.stdout:mds.cephfs.smithi186.wjnafs smithi186 running (12m) 12m ago 58m 11.0M - 19.0.0-2879-g324baf8d 8717720dc5e5 3cf1ef4148e2 2024-04-12T21:51:06.620 INFO:teuthology.orchestra.run.smithi018.stdout:mgr.smithi018.xjzwga smithi018 *:8443,9283,8765 running (54m) 12m ago 67m 517M - 19.0.0-2879-g324baf8d 8717720dc5e5 56088305eab7 2024-04-12T21:51:06.620 INFO:teuthology.orchestra.run.smithi018.stdout:mgr.smithi186.kqcfxk smithi186 *:8443,9283,8765 running (53m) 12m ago 64m 460M - 19.0.0-2879-g324baf8d 8717720dc5e5 09cd766e5e0d 2024-04-12T21:51:06.620 INFO:teuthology.orchestra.run.smithi018.stdout:mon.smithi018 smithi018 running (49m) 12m ago 68m 88.2M 2048M 19.0.0-2879-g324baf8d 8717720dc5e5 df3e201dfed6 2024-04-12T21:51:06.620 INFO:teuthology.orchestra.run.smithi018.stdout:mon.smithi186 smithi186 running (49m) 12m ago 64m 53.6M 2048M 19.0.0-2879-g324baf8d 8717720dc5e5 c77498da9b2b 2024-04-12T21:51:06.621 INFO:teuthology.orchestra.run.smithi018.stdout:node-exporter.smithi018 smithi018 *:9100 running (52m) 12m ago 66m 8468k - 1.5.0 0da6a335fe13 326c3d9c6847 2024-04-12T21:51:06.621 INFO:teuthology.orchestra.run.smithi018.stdout:node-exporter.smithi186 smithi186 *:9100 running (52m) 12m ago 64m 8166k - 1.5.0 0da6a335fe13 ec2112baeecf 2024-04-12T21:51:06.621 INFO:teuthology.orchestra.run.smithi018.stdout:osd.0 smithi018 running (47m) 12m ago 63m 123M 1559M 19.0.0-2879-g324baf8d 8717720dc5e5 60b1273ca047 2024-04-12T21:51:06.621 INFO:teuthology.orchestra.run.smithi018.stdout:osd.1 smithi018 running (46m) 12m ago 63m 95.4M 1559M 19.0.0-2879-g324baf8d 8717720dc5e5 170ce1e5fb8d 2024-04-12T21:51:06.621 INFO:teuthology.orchestra.run.smithi018.stdout:osd.2 smithi018 running (46m) 12m ago 62m 94.6M 1559M 19.0.0-2879-g324baf8d 8717720dc5e5 c2cbc6aa2839 2024-04-12T21:51:06.621 INFO:teuthology.orchestra.run.smithi018.stdout:osd.3 smithi186 running (45m) 12m ago 62m 101M 2581M 19.0.0-2879-g324baf8d 8717720dc5e5 0138e8262e4b 2024-04-12T21:51:06.621 INFO:teuthology.orchestra.run.smithi018.stdout:osd.4 smithi186 running (45m) 12m ago 61m 91.5M 2581M 19.0.0-2879-g324baf8d 8717720dc5e5 b20cb63b25c9 2024-04-12T21:51:06.621 INFO:teuthology.orchestra.run.smithi018.stdout:osd.5 smithi186 running (44m) 12m ago 60m 92.2M 2581M 19.0.0-2879-g324baf8d 8717720dc5e5 2513bea2610e 2024-04-12T21:51:06.621 INFO:teuthology.orchestra.run.smithi018.stdout:prometheus.smithi018 smithi018 *:9095 running (52m) 12m ago 65m 69.6M - 2.43.0 a07b618ecd1d f58841127e21 2024-04-12T21:51:06.623 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:06.621+0000 7f82ec000700 1 -- 172.21.15.18:0/128850486 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f82cc06ec00 msgr2=0x7f82cc0710b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:51:06.623 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:06.621+0000 7f82ec000700 1 --2- 172.21.15.18:0/128850486 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f82cc06ec00 0x7f82cc0710b0 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f82d800b3b0 tx=0x7f82d8009e30 comp rx=0 tx=0).stop 2024-04-12T21:51:06.623 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:06.621+0000 7f82ec000700 1 -- 172.21.15.18:0/128850486 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f82e4105950 msgr2=0x7f82e4116270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:51:06.623 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:06.621+0000 7f82ec000700 1 --2- 172.21.15.18:0/128850486 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f82e4105950 0x7f82e4116270 secure :-1 s=READY pgs=1301 cs=0 l=1 rev1=1 crypto rx=0x7f82d0002a50 tx=0x7f82d0004600 comp rx=0 tx=0).stop 2024-04-12T21:51:06.623 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:06.621+0000 7f82ec000700 1 -- 172.21.15.18:0/128850486 shutdown_connections 2024-04-12T21:51:06.624 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:06.621+0000 7f82ec000700 1 --2- 172.21.15.18:0/128850486 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f82cc06ec00 0x7f82cc0710b0 unknown :-1 s=CLOSED pgs=689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:51:06.624 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:06.621+0000 7f82ec000700 1 --2- 172.21.15.18:0/128850486 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f82e4104fb0 0x7f82e4115d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:51:06.624 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:06.621+0000 7f82ec000700 1 --2- 172.21.15.18:0/128850486 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f82e4105950 0x7f82e4116270 unknown :-1 s=CLOSED pgs=1301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:51:06.624 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:06.621+0000 7f82ec000700 1 -- 172.21.15.18:0/128850486 >> 172.21.15.18:0/128850486 conn(0x7f82e40fab40 msgr2=0x7f82e40fbb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:51:06.624 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:06.621+0000 7f82ec000700 1 -- 172.21.15.18:0/128850486 shutdown_connections 2024-04-12T21:51:06.624 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:06.621+0000 7f82ec000700 1 -- 172.21.15.18:0/128850486 wait complete. 2024-04-12T21:51:06.785 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:06.782+0000 7f7cbe800700 1 -- 172.21.15.18:0/1070384851 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7cb8109410 msgr2=0x7f7cb81097e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:51:06.785 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:06.782+0000 7f7cbe800700 1 --2- 172.21.15.18:0/1070384851 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7cb8109410 0x7f7cb81097e0 secure :-1 s=READY pgs=1302 cs=0 l=1 rev1=1 crypto rx=0x7f7ca8004940 tx=0x7f7ca8009e30 comp rx=0 tx=0).stop 2024-04-12T21:51:06.785 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:06.783+0000 7f7cbe800700 1 -- 172.21.15.18:0/1070384851 shutdown_connections 2024-04-12T21:51:06.785 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:06.783+0000 7f7cbe800700 1 --2- 172.21.15.18:0/1070384851 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7cb8109db0 0x7f7cb8103470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:51:06.785 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:06.783+0000 7f7cbe800700 1 --2- 172.21.15.18:0/1070384851 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7cb8109410 0x7f7cb81097e0 unknown :-1 s=CLOSED pgs=1302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:51:06.785 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:06.783+0000 7f7cbe800700 1 -- 172.21.15.18:0/1070384851 >> 172.21.15.18:0/1070384851 conn(0x7f7cb8072c50 msgr2=0x7f7cb8075060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:51:06.785 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:06.783+0000 7f7cbe800700 1 -- 172.21.15.18:0/1070384851 shutdown_connections 2024-04-12T21:51:06.785 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:06.783+0000 7f7cbe800700 1 -- 172.21.15.18:0/1070384851 wait complete. 2024-04-12T21:51:06.787 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:06.783+0000 7f7cbe800700 1 Processor -- start 2024-04-12T21:51:06.787 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:06.784+0000 7f7cbe800700 1 -- start start 2024-04-12T21:51:06.787 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:06.784+0000 7f7cbe800700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7cb8109410 0x7f7cb811a180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:51:06.787 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:06.784+0000 7f7cbe800700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7cb8109db0 0x7f7cb811a6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:51:06.787 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:06.784+0000 7f7cbe800700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7cb811ad10 con 0x7f7cb8109db0 2024-04-12T21:51:06.787 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:06.784+0000 7f7cbe800700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7cb81140e0 con 0x7f7cb8109410 2024-04-12T21:51:06.787 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:06.784+0000 7f7cb6a00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7cb8109db0 0x7f7cb811a6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:51:06.787 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:06.784+0000 7f7cb6a00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7cb8109db0 0x7f7cb811a6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:57204/0 (socket says 172.21.15.18:57204) 2024-04-12T21:51:06.787 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:06.784+0000 7f7cb6a00700 1 -- 172.21.15.18:0/1237734531 learned_addr learned my addr 172.21.15.18:0/1237734531 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:51:06.787 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:06.785+0000 7f7cb7400700 1 --2- 172.21.15.18:0/1237734531 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7cb8109410 0x7f7cb811a180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:51:06.787 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:06.785+0000 7f7cb6a00700 1 -- 172.21.15.18:0/1237734531 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7cb8109410 msgr2=0x7f7cb811a180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:51:06.787 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:06.785+0000 7f7cb6a00700 1 --2- 172.21.15.18:0/1237734531 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7cb8109410 0x7f7cb811a180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:51:06.788 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:06.785+0000 7f7cb6a00700 1 -- 172.21.15.18:0/1237734531 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ca8005040 con 0x7f7cb8109db0 2024-04-12T21:51:06.788 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:06.785+0000 7f7cb6a00700 1 --2- 172.21.15.18:0/1237734531 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7cb8109db0 0x7f7cb811a6c0 secure :-1 s=READY pgs=1303 cs=0 l=1 rev1=1 crypto rx=0x7f7ca4002a50 tx=0x7f7ca4004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:51:06.788 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:06.785+0000 7f7caf400700 1 -- 172.21.15.18:0/1237734531 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7ca4007cc0 con 0x7f7cb8109db0 2024-04-12T21:51:06.788 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:06.786+0000 7f7caf400700 1 -- 172.21.15.18:0/1237734531 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f7ca400f040 con 0x7f7cb8109db0 2024-04-12T21:51:06.788 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:06.786+0000 7f7cbe800700 1 -- 172.21.15.18:0/1237734531 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7cb81143c0 con 0x7f7cb8109db0 2024-04-12T21:51:06.789 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:06.786+0000 7f7caf400700 1 -- 172.21.15.18:0/1237734531 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7ca400daf0 con 0x7f7cb8109db0 2024-04-12T21:51:06.789 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:06.786+0000 7f7cbe800700 1 -- 172.21.15.18:0/1237734531 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7cb8114980 con 0x7f7cb8109db0 2024-04-12T21:51:06.789 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:06.787+0000 7f7cbe800700 1 -- 172.21.15.18:0/1237734531 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7cb8104b60 con 0x7f7cb8109db0 2024-04-12T21:51:06.790 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:06.788+0000 7f7caf400700 1 -- 172.21.15.18:0/1237734531 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7f7ca4009810 con 0x7f7cb8109db0 2024-04-12T21:51:06.791 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:06.789+0000 7f7caf400700 1 --2- 172.21.15.18:0/1237734531 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f7c9c06ec00 0x7f7c9c0710b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:51:06.791 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:06.789+0000 7f7caf400700 1 -- 172.21.15.18:0/1237734531 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 6757+0+0 (secure 0 0 0) 0x7f7ca408e7d0 con 0x7f7cb8109db0 2024-04-12T21:51:06.791 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:06.789+0000 7f7cb7400700 1 --2- 172.21.15.18:0/1237734531 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f7c9c06ec00 0x7f7c9c0710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:51:06.792 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:06.790+0000 7f7cb7400700 1 --2- 172.21.15.18:0/1237734531 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f7c9c06ec00 0x7f7c9c0710b0 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f7ca8004f90 tx=0x7f7ca8009e30 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:51:06.796 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:06.794+0000 7f7caf400700 1 -- 172.21.15.18:0/1237734531 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f7ca405af00 con 0x7f7cb8109db0 2024-04-12T21:51:07.127 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:07.125+0000 7f7cbe800700 1 -- 172.21.15.18:0/1237734531 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f7cb804bed0 con 0x7f7cb8109db0 2024-04-12T21:51:07.128 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:07.126+0000 7f7caf400700 1 -- 172.21.15.18:0/1237734531 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+623 (secure 0 0 0) 0x7f7ca405aa90 con 0x7f7cb8109db0 2024-04-12T21:51:07.128 INFO:teuthology.orchestra.run.smithi018.stdout:{ 2024-04-12T21:51:07.128 INFO:teuthology.orchestra.run.smithi018.stdout: "mon": { 2024-04-12T21:51:07.128 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 19.0.0-2879-g324baf8d (324baf8ddb91dea46cac50cb535c306115791355) squid (dev)": 2 2024-04-12T21:51:07.128 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-12T21:51:07.128 INFO:teuthology.orchestra.run.smithi018.stdout: "mgr": { 2024-04-12T21:51:07.129 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 19.0.0-2879-g324baf8d (324baf8ddb91dea46cac50cb535c306115791355) squid (dev)": 2 2024-04-12T21:51:07.129 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-12T21:51:07.129 INFO:teuthology.orchestra.run.smithi018.stdout: "osd": { 2024-04-12T21:51:07.129 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 19.0.0-2879-g324baf8d (324baf8ddb91dea46cac50cb535c306115791355) squid (dev)": 6 2024-04-12T21:51:07.129 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-12T21:51:07.129 INFO:teuthology.orchestra.run.smithi018.stdout: "mds": { 2024-04-12T21:51:07.129 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 19.0.0-2879-g324baf8d (324baf8ddb91dea46cac50cb535c306115791355) squid (dev)": 4 2024-04-12T21:51:07.129 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-12T21:51:07.129 INFO:teuthology.orchestra.run.smithi018.stdout: "overall": { 2024-04-12T21:51:07.129 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 19.0.0-2879-g324baf8d (324baf8ddb91dea46cac50cb535c306115791355) squid (dev)": 14 2024-04-12T21:51:07.129 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-04-12T21:51:07.129 INFO:teuthology.orchestra.run.smithi018.stdout:} 2024-04-12T21:51:07.131 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:07.129+0000 7f7cbe800700 1 -- 172.21.15.18:0/1237734531 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f7c9c06ec00 msgr2=0x7f7c9c0710b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:51:07.131 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:07.129+0000 7f7cbe800700 1 --2- 172.21.15.18:0/1237734531 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f7c9c06ec00 0x7f7c9c0710b0 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f7ca8004f90 tx=0x7f7ca8009e30 comp rx=0 tx=0).stop 2024-04-12T21:51:07.131 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:07.129+0000 7f7cbe800700 1 -- 172.21.15.18:0/1237734531 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7cb8109db0 msgr2=0x7f7cb811a6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:51:07.131 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:07.129+0000 7f7cbe800700 1 --2- 172.21.15.18:0/1237734531 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7cb8109db0 0x7f7cb811a6c0 secure :-1 s=READY pgs=1303 cs=0 l=1 rev1=1 crypto rx=0x7f7ca4002a50 tx=0x7f7ca4004600 comp rx=0 tx=0).stop 2024-04-12T21:51:07.132 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:07.129+0000 7f7cbe800700 1 -- 172.21.15.18:0/1237734531 shutdown_connections 2024-04-12T21:51:07.132 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:07.129+0000 7f7cbe800700 1 --2- 172.21.15.18:0/1237734531 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f7c9c06ec00 0x7f7c9c0710b0 unknown :-1 s=CLOSED pgs=690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:51:07.132 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:07.129+0000 7f7cbe800700 1 --2- 172.21.15.18:0/1237734531 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7cb8109410 0x7f7cb811a180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:51:07.132 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:07.129+0000 7f7cbe800700 1 --2- 172.21.15.18:0/1237734531 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7cb8109db0 0x7f7cb811a6c0 unknown :-1 s=CLOSED pgs=1303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:51:07.132 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:07.129+0000 7f7cbe800700 1 -- 172.21.15.18:0/1237734531 >> 172.21.15.18:0/1237734531 conn(0x7f7cb8072c50 msgr2=0x7f7cb8074c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:51:07.132 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:07.129+0000 7f7cbe800700 1 -- 172.21.15.18:0/1237734531 shutdown_connections 2024-04-12T21:51:07.132 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:07.129+0000 7f7cbe800700 1 -- 172.21.15.18:0/1237734531 wait complete. 2024-04-12T21:51:07.285 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:07.283+0000 7f262a000700 1 -- 172.21.15.18:0/973920561 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2624109dd0 msgr2=0x7f2624103440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:51:07.285 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:07.283+0000 7f262a000700 1 --2- 172.21.15.18:0/973920561 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2624109dd0 0x7f2624103440 secure :-1 s=READY pgs=1304 cs=0 l=1 rev1=1 crypto rx=0x7f2610004940 tx=0x7f2610009e30 comp rx=0 tx=0).stop 2024-04-12T21:51:07.286 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:07.283+0000 7f262a000700 1 -- 172.21.15.18:0/973920561 shutdown_connections 2024-04-12T21:51:07.286 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:07.283+0000 7f262a000700 1 --2- 172.21.15.18:0/973920561 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2624109dd0 0x7f2624103440 unknown :-1 s=CLOSED pgs=1304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:51:07.286 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:07.283+0000 7f262a000700 1 --2- 172.21.15.18:0/973920561 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2624109430 0x7f2624109800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:51:07.286 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:07.283+0000 7f262a000700 1 -- 172.21.15.18:0/973920561 >> 172.21.15.18:0/973920561 conn(0x7f2624072c00 msgr2=0x7f2624075010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:51:07.286 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:07.283+0000 7f262a000700 1 -- 172.21.15.18:0/973920561 shutdown_connections 2024-04-12T21:51:07.286 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:07.283+0000 7f262a000700 1 -- 172.21.15.18:0/973920561 wait complete. 2024-04-12T21:51:07.287 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:07.284+0000 7f262a000700 1 Processor -- start 2024-04-12T21:51:07.287 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:07.284+0000 7f262a000700 1 -- start start 2024-04-12T21:51:07.287 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:07.285+0000 7f262a000700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2624109430 0x7f2624119fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:51:07.287 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:07.285+0000 7f262a000700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2624109dd0 0x7f262411a4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:51:07.287 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:07.285+0000 7f262a000700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f262411abc0 con 0x7f2624109430 2024-04-12T21:51:07.287 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:07.285+0000 7f262a000700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2624114110 con 0x7f2624109dd0 2024-04-12T21:51:07.287 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:07.285+0000 7f2623400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2624109430 0x7f2624119fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:51:07.287 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:07.285+0000 7f2623400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2624109430 0x7f2624119fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:57226/0 (socket says 172.21.15.18:57226) 2024-04-12T21:51:07.288 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:07.285+0000 7f2623400700 1 -- 172.21.15.18:0/3143708462 learned_addr learned my addr 172.21.15.18:0/3143708462 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:51:07.288 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:07.285+0000 7f2623400700 1 -- 172.21.15.18:0/3143708462 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2624109dd0 msgr2=0x7f262411a4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:51:07.288 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:07.285+0000 7f2623400700 1 --2- 172.21.15.18:0/3143708462 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2624109dd0 0x7f262411a4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:51:07.288 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:07.285+0000 7f2623400700 1 -- 172.21.15.18:0/3143708462 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2610005040 con 0x7f2624109430 2024-04-12T21:51:07.288 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:07.286+0000 7f2623400700 1 --2- 172.21.15.18:0/3143708462 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2624109430 0x7f2624119fa0 secure :-1 s=READY pgs=1305 cs=0 l=1 rev1=1 crypto rx=0x7f261400b790 tx=0x7f2614009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:51:07.289 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:07.286+0000 7f261b400700 1 -- 172.21.15.18:0/3143708462 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f261400f770 con 0x7f2624109430 2024-04-12T21:51:07.289 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:07.286+0000 7f261b400700 1 -- 172.21.15.18:0/3143708462 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f261400fda0 con 0x7f2624109430 2024-04-12T21:51:07.289 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:07.286+0000 7f261b400700 1 -- 172.21.15.18:0/3143708462 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f261400d310 con 0x7f2624109430 2024-04-12T21:51:07.289 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:07.286+0000 7f262a000700 1 -- 172.21.15.18:0/3143708462 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2624114390 con 0x7f2624109430 2024-04-12T21:51:07.289 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:07.286+0000 7f262a000700 1 -- 172.21.15.18:0/3143708462 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2624114950 con 0x7f2624109430 2024-04-12T21:51:07.289 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:07.287+0000 7f262a000700 1 -- 172.21.15.18:0/3143708462 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2624104b80 con 0x7f2624109430 2024-04-12T21:51:07.290 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:07.289+0000 7f261b400700 1 -- 172.21.15.18:0/3143708462 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7f2614010420 con 0x7f2624109430 2024-04-12T21:51:07.291 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:07.289+0000 7f261b400700 1 --2- 172.21.15.18:0/3143708462 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f260806ec00 0x7f26080710b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:51:07.291 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:07.290+0000 7f261b400700 1 -- 172.21.15.18:0/3143708462 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 6757+0+0 (secure 0 0 0) 0x7f261408e8e0 con 0x7f2624109430 2024-04-12T21:51:07.292 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:07.290+0000 7f2622a00700 1 --2- 172.21.15.18:0/3143708462 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f260806ec00 0x7f26080710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:51:07.292 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:07.291+0000 7f2622a00700 1 --2- 172.21.15.18:0/3143708462 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f260806ec00 0x7f26080710b0 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7f2610004f90 tx=0x7f2610009e30 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:51:07.296 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:07.294+0000 7f261b400700 1 -- 172.21.15.18:0/3143708462 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f261405b1d0 con 0x7f2624109430 2024-04-12T21:51:07.355 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:51:07 smithi018 ceph-mon[112590]: from='client.36664 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:51:07.534 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:51:07 smithi186 ceph-mon[86379]: from='client.36664 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:51:07.567 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:07.565+0000 7f262a000700 1 -- 172.21.15.18:0/3143708462 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f26241105b0 con 0x7f2624109430 2024-04-12T21:51:07.568 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:07.566+0000 7f261b400700 1 -- 172.21.15.18:0/3143708462 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 33 v33) v1 ==== 76+0+1876 (secure 0 0 0) 0x7f261405ad60 con 0x7f2624109430 2024-04-12T21:51:07.568 INFO:teuthology.orchestra.run.smithi018.stdout:e33 2024-04-12T21:51:07.569 INFO:teuthology.orchestra.run.smithi018.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-12T21:51:07.569 INFO:teuthology.orchestra.run.smithi018.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-12T21:51:07.569 INFO:teuthology.orchestra.run.smithi018.stdout:legacy client fscid: 1 2024-04-12T21:51:07.569 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-12T21:51:07.569 INFO:teuthology.orchestra.run.smithi018.stdout:Filesystem 'cephfs' (1) 2024-04-12T21:51:07.569 INFO:teuthology.orchestra.run.smithi018.stdout:fs_name cephfs 2024-04-12T21:51:07.569 INFO:teuthology.orchestra.run.smithi018.stdout:epoch 24 2024-04-12T21:51:07.569 INFO:teuthology.orchestra.run.smithi018.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-04-12T21:51:07.569 INFO:teuthology.orchestra.run.smithi018.stdout:created 2024-04-12T20:51:57.362850+0000 2024-04-12T21:51:07.569 INFO:teuthology.orchestra.run.smithi018.stdout:modified 2024-04-12T21:06:59.717190+0000 2024-04-12T21:51:07.569 INFO:teuthology.orchestra.run.smithi018.stdout:tableserver 0 2024-04-12T21:51:07.569 INFO:teuthology.orchestra.run.smithi018.stdout:root 0 2024-04-12T21:51:07.569 INFO:teuthology.orchestra.run.smithi018.stdout:session_timeout 60 2024-04-12T21:51:07.569 INFO:teuthology.orchestra.run.smithi018.stdout:session_autoclose 300 2024-04-12T21:51:07.570 INFO:teuthology.orchestra.run.smithi018.stdout:max_file_size 1099511627776 2024-04-12T21:51:07.570 INFO:teuthology.orchestra.run.smithi018.stdout:max_xattr_size 65536 2024-04-12T21:51:07.570 INFO:teuthology.orchestra.run.smithi018.stdout:required_client_features {} 2024-04-12T21:51:07.570 INFO:teuthology.orchestra.run.smithi018.stdout:last_failure 0 2024-04-12T21:51:07.570 INFO:teuthology.orchestra.run.smithi018.stdout:last_failure_osd_epoch 100 2024-04-12T21:51:07.570 INFO:teuthology.orchestra.run.smithi018.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-12T21:51:07.570 INFO:teuthology.orchestra.run.smithi018.stdout:max_mds 2 2024-04-12T21:51:07.570 INFO:teuthology.orchestra.run.smithi018.stdout:in 0,1 2024-04-12T21:51:07.570 INFO:teuthology.orchestra.run.smithi018.stdout:up {} 2024-04-12T21:51:07.570 INFO:teuthology.orchestra.run.smithi018.stdout:failed 0,1 2024-04-12T21:51:07.570 INFO:teuthology.orchestra.run.smithi018.stdout:damaged 2024-04-12T21:51:07.570 INFO:teuthology.orchestra.run.smithi018.stdout:stopped 2024-04-12T21:51:07.571 INFO:teuthology.orchestra.run.smithi018.stdout:data_pools [3] 2024-04-12T21:51:07.571 INFO:teuthology.orchestra.run.smithi018.stdout:metadata_pool 2 2024-04-12T21:51:07.571 INFO:teuthology.orchestra.run.smithi018.stdout:inline_data disabled 2024-04-12T21:51:07.571 INFO:teuthology.orchestra.run.smithi018.stdout:balancer 2024-04-12T21:51:07.571 INFO:teuthology.orchestra.run.smithi018.stdout:bal_rank_mask -1 2024-04-12T21:51:07.571 INFO:teuthology.orchestra.run.smithi018.stdout:standby_count_wanted 1 2024-04-12T21:51:07.571 INFO:teuthology.orchestra.run.smithi018.stdout:qdb_cluster leader: 0 members: 2024-04-12T21:51:07.571 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-12T21:51:07.571 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-12T21:51:07.571 INFO:teuthology.orchestra.run.smithi018.stdout:Standby daemons: 2024-04-12T21:51:07.571 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-12T21:51:07.571 INFO:teuthology.orchestra.run.smithi018.stdout:[mds.cephfs.smithi018.acqbsc{-1:34436} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.18:6828/755179764,v1:172.21.15.18:6829/755179764] compat {c=[1],r=[1],i=[1fff]}] 2024-04-12T21:51:07.572 INFO:teuthology.orchestra.run.smithi018.stdout:[mds.cephfs.smithi018.szjluo{-1:34468} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.18:6826/2407992653,v1:172.21.15.18:6827/2407992653] compat {c=[1],r=[1],i=[1fff]}] 2024-04-12T21:51:07.572 INFO:teuthology.orchestra.run.smithi018.stdout:[mds.cephfs.smithi186.jdllyn{-1:44595} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.186:6824/3523444202,v1:172.21.15.186:6825/3523444202] compat {c=[1],r=[1],i=[1fff]}] 2024-04-12T21:51:07.572 INFO:teuthology.orchestra.run.smithi018.stdout:[mds.cephfs.smithi186.wjnafs{-1:44875} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.186:6826/3122443766,v1:172.21.15.186:6827/3122443766] compat {c=[1],r=[1],i=[1fff]}] 2024-04-12T21:51:07.572 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:07.568+0000 7f262a000700 1 -- 172.21.15.18:0/3143708462 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f260806ec00 msgr2=0x7f26080710b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:51:07.572 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:07.569+0000 7f262a000700 1 --2- 172.21.15.18:0/3143708462 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f260806ec00 0x7f26080710b0 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7f2610004f90 tx=0x7f2610009e30 comp rx=0 tx=0).stop 2024-04-12T21:51:07.573 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:07.569+0000 7f262a000700 1 -- 172.21.15.18:0/3143708462 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2624109430 msgr2=0x7f2624119fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:51:07.573 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:07.569+0000 7f262a000700 1 --2- 172.21.15.18:0/3143708462 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2624109430 0x7f2624119fa0 secure :-1 s=READY pgs=1305 cs=0 l=1 rev1=1 crypto rx=0x7f261400b790 tx=0x7f2614009250 comp rx=0 tx=0).stop 2024-04-12T21:51:07.573 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:07.569+0000 7f262a000700 1 -- 172.21.15.18:0/3143708462 shutdown_connections 2024-04-12T21:51:07.573 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:07.569+0000 7f262a000700 1 --2- 172.21.15.18:0/3143708462 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f260806ec00 0x7f26080710b0 unknown :-1 s=CLOSED pgs=691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:51:07.573 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:07.569+0000 7f262a000700 1 --2- 172.21.15.18:0/3143708462 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2624109430 0x7f2624119fa0 unknown :-1 s=CLOSED pgs=1305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:51:07.573 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:07.569+0000 7f262a000700 1 --2- 172.21.15.18:0/3143708462 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2624109dd0 0x7f262411a4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:51:07.573 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:07.569+0000 7f262a000700 1 -- 172.21.15.18:0/3143708462 >> 172.21.15.18:0/3143708462 conn(0x7f2624072c00 msgr2=0x7f2624074b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:51:07.573 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:07.569+0000 7f262a000700 1 -- 172.21.15.18:0/3143708462 shutdown_connections 2024-04-12T21:51:07.573 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:07.569+0000 7f262a000700 1 -- 172.21.15.18:0/3143708462 wait complete. 2024-04-12T21:51:07.573 INFO:teuthology.orchestra.run.smithi018.stderr:dumped fsmap epoch 33 2024-04-12T21:51:07.730 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:07.727+0000 7f825c200700 1 -- 172.21.15.18:0/1963736714 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8254109430 msgr2=0x7f8254109800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:51:07.730 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:07.727+0000 7f825c200700 1 --2- 172.21.15.18:0/1963736714 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8254109430 0x7f8254109800 secure :-1 s=READY pgs=1306 cs=0 l=1 rev1=1 crypto rx=0x7f8248004940 tx=0x7f8248009e30 comp rx=0 tx=0).stop 2024-04-12T21:51:07.730 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:07.728+0000 7f825c200700 1 -- 172.21.15.18:0/1963736714 shutdown_connections 2024-04-12T21:51:07.730 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:07.728+0000 7f825c200700 1 --2- 172.21.15.18:0/1963736714 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8254109dd0 0x7f8254103440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:51:07.730 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:07.728+0000 7f825c200700 1 --2- 172.21.15.18:0/1963736714 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8254109430 0x7f8254109800 unknown :-1 s=CLOSED pgs=1306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:51:07.730 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:07.728+0000 7f825c200700 1 -- 172.21.15.18:0/1963736714 >> 172.21.15.18:0/1963736714 conn(0x7f8254072c00 msgr2=0x7f8254075010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:51:07.730 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:07.728+0000 7f825c200700 1 -- 172.21.15.18:0/1963736714 shutdown_connections 2024-04-12T21:51:07.730 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:07.728+0000 7f825c200700 1 -- 172.21.15.18:0/1963736714 wait complete. 2024-04-12T21:51:07.731 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:07.729+0000 7f825c200700 1 Processor -- start 2024-04-12T21:51:07.731 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:07.729+0000 7f825c200700 1 -- start start 2024-04-12T21:51:07.732 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:07.730+0000 7f825c200700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8254109430 0x7f8254119f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:51:07.732 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:07.730+0000 7f825c200700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8254109dd0 0x7f825411a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:51:07.732 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:07.730+0000 7f825c200700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f825411ab60 con 0x7f8254109430 2024-04-12T21:51:07.732 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:07.730+0000 7f825c200700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82541140b0 con 0x7f8254109dd0 2024-04-12T21:51:07.732 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:07.730+0000 7f8259800700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8254109430 0x7f8254119f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:51:07.732 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:07.730+0000 7f8259800700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8254109430 0x7f8254119f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:57250/0 (socket says 172.21.15.18:57250) 2024-04-12T21:51:07.732 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:07.730+0000 7f8259800700 1 -- 172.21.15.18:0/4149223155 learned_addr learned my addr 172.21.15.18:0/4149223155 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:51:07.733 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:07.730+0000 7f8259800700 1 -- 172.21.15.18:0/4149223155 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8254109dd0 msgr2=0x7f825411a480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:51:07.733 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:07.730+0000 7f8258e00700 1 --2- 172.21.15.18:0/4149223155 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8254109dd0 0x7f825411a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:51:07.733 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:07.731+0000 7f8259800700 1 --2- 172.21.15.18:0/4149223155 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8254109dd0 0x7f825411a480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:51:07.734 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:07.731+0000 7f8259800700 1 -- 172.21.15.18:0/4149223155 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8248005040 con 0x7f8254109430 2024-04-12T21:51:07.734 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:07.731+0000 7f8258e00700 1 --2- 172.21.15.18:0/4149223155 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8254109dd0 0x7f825411a480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-12T21:51:07.734 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:07.731+0000 7f8259800700 1 --2- 172.21.15.18:0/4149223155 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8254109430 0x7f8254119f40 secure :-1 s=READY pgs=1307 cs=0 l=1 rev1=1 crypto rx=0x7f8248009180 tx=0x7f8248009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:51:07.734 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:07.731+0000 7f8251600700 1 -- 172.21.15.18:0/4149223155 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f824800e040 con 0x7f8254109430 2024-04-12T21:51:07.734 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:07.731+0000 7f8251600700 1 -- 172.21.15.18:0/4149223155 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f824800bbc0 con 0x7f8254109430 2024-04-12T21:51:07.734 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:07.731+0000 7f8251600700 1 -- 172.21.15.18:0/4149223155 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f824801c630 con 0x7f8254109430 2024-04-12T21:51:07.734 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:07.731+0000 7f825c200700 1 -- 172.21.15.18:0/4149223155 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f82541142d0 con 0x7f8254109430 2024-04-12T21:51:07.734 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:07.731+0000 7f825c200700 1 -- 172.21.15.18:0/4149223155 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8254114890 con 0x7f8254109430 2024-04-12T21:51:07.734 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:07.732+0000 7f825c200700 1 -- 172.21.15.18:0/4149223155 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8254104b80 con 0x7f8254109430 2024-04-12T21:51:07.735 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:07.734+0000 7f8251600700 1 -- 172.21.15.18:0/4149223155 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7f824800bd30 con 0x7f8254109430 2024-04-12T21:51:07.737 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:07.735+0000 7f8251600700 1 --2- 172.21.15.18:0/4149223155 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f823c06ec00 0x7f823c0710b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:51:07.737 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:07.735+0000 7f8251600700 1 -- 172.21.15.18:0/4149223155 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 6757+0+0 (secure 0 0 0) 0x7f824808fd90 con 0x7f8254109430 2024-04-12T21:51:07.737 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:07.735+0000 7f8258e00700 1 --2- 172.21.15.18:0/4149223155 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f823c06ec00 0x7f823c0710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:51:07.737 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:07.736+0000 7f8258e00700 1 --2- 172.21.15.18:0/4149223155 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f823c06ec00 0x7f823c0710b0 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7f8240006fd0 tx=0x7f8240008040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:51:07.742 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:07.740+0000 7f8251600700 1 -- 172.21.15.18:0/4149223155 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f824805c5c0 con 0x7f8254109430 2024-04-12T21:51:07.995 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:07.993+0000 7f825c200700 1 -- 172.21.15.18:0/4149223155 --> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f8254114bc0 con 0x7f823c06ec00 2024-04-12T21:51:07.998 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:07.996+0000 7f8251600700 1 -- 172.21.15.18:0/4149223155 <== mgr.34102 v2:172.21.15.18:6800/3812891919 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f8254114bc0 con 0x7f823c06ec00 2024-04-12T21:51:07.998 INFO:teuthology.orchestra.run.smithi018.stdout:{ 2024-04-12T21:51:07.998 INFO:teuthology.orchestra.run.smithi018.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:f4452def3be4d350f4a8e5feaf7ebb1511be9298fe68478ba7a2d6fbd7d69a63", 2024-04-12T21:51:07.998 INFO:teuthology.orchestra.run.smithi018.stdout: "in_progress": true, 2024-04-12T21:51:07.999 INFO:teuthology.orchestra.run.smithi018.stdout: "which": "Upgrading all daemon types on all hosts", 2024-04-12T21:51:07.999 INFO:teuthology.orchestra.run.smithi018.stdout: "services_complete": [ 2024-04-12T21:51:07.999 INFO:teuthology.orchestra.run.smithi018.stdout: "mgr", 2024-04-12T21:51:07.999 INFO:teuthology.orchestra.run.smithi018.stdout: "mds", 2024-04-12T21:51:07.999 INFO:teuthology.orchestra.run.smithi018.stdout: "crash", 2024-04-12T21:51:07.999 INFO:teuthology.orchestra.run.smithi018.stdout: "osd", 2024-04-12T21:51:07.999 INFO:teuthology.orchestra.run.smithi018.stdout: "mon" 2024-04-12T21:51:07.999 INFO:teuthology.orchestra.run.smithi018.stdout: ], 2024-04-12T21:51:07.999 INFO:teuthology.orchestra.run.smithi018.stdout: "progress": "16/21 daemons upgraded", 2024-04-12T21:51:07.999 INFO:teuthology.orchestra.run.smithi018.stdout: "message": "Currently upgrading mds daemons", 2024-04-12T21:51:07.999 INFO:teuthology.orchestra.run.smithi018.stdout: "is_paused": false 2024-04-12T21:51:07.999 INFO:teuthology.orchestra.run.smithi018.stdout:} 2024-04-12T21:51:08.001 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:07.999+0000 7f825c200700 1 -- 172.21.15.18:0/4149223155 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f823c06ec00 msgr2=0x7f823c0710b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:51:08.002 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:07.999+0000 7f825c200700 1 --2- 172.21.15.18:0/4149223155 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f823c06ec00 0x7f823c0710b0 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7f8240006fd0 tx=0x7f8240008040 comp rx=0 tx=0).stop 2024-04-12T21:51:08.002 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:07.999+0000 7f825c200700 1 -- 172.21.15.18:0/4149223155 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8254109430 msgr2=0x7f8254119f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:51:08.002 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:07.999+0000 7f825c200700 1 --2- 172.21.15.18:0/4149223155 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8254109430 0x7f8254119f40 secure :-1 s=READY pgs=1307 cs=0 l=1 rev1=1 crypto rx=0x7f8248009180 tx=0x7f8248009e30 comp rx=0 tx=0).stop 2024-04-12T21:51:08.002 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:07.999+0000 7f825c200700 1 -- 172.21.15.18:0/4149223155 shutdown_connections 2024-04-12T21:51:08.002 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:07.999+0000 7f825c200700 1 --2- 172.21.15.18:0/4149223155 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f823c06ec00 0x7f823c0710b0 unknown :-1 s=CLOSED pgs=692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:51:08.002 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:07.999+0000 7f825c200700 1 --2- 172.21.15.18:0/4149223155 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8254109430 0x7f8254119f40 unknown :-1 s=CLOSED pgs=1307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:51:08.002 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:07.999+0000 7f825c200700 1 --2- 172.21.15.18:0/4149223155 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8254109dd0 0x7f825411a480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:51:08.002 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:07.999+0000 7f825c200700 1 -- 172.21.15.18:0/4149223155 >> 172.21.15.18:0/4149223155 conn(0x7f8254072c00 msgr2=0x7f8254074c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:51:08.002 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:08.000+0000 7f825c200700 1 -- 172.21.15.18:0/4149223155 shutdown_connections 2024-04-12T21:51:08.002 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:08.000+0000 7f825c200700 1 -- 172.21.15.18:0/4149223155 wait complete. 2024-04-12T21:51:08.164 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:08.161+0000 7f131a800700 1 -- 172.21.15.18:0/1662557675 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f13140fc620 msgr2=0x7f1314100cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:51:08.164 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:08.161+0000 7f131a800700 1 --2- 172.21.15.18:0/1662557675 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f13140fc620 0x7f1314100cd0 secure :-1 s=READY pgs=1308 cs=0 l=1 rev1=1 crypto rx=0x7f12fc004940 tx=0x7f12fc009e30 comp rx=0 tx=0).stop 2024-04-12T21:51:08.164 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:08.162+0000 7f131a800700 1 -- 172.21.15.18:0/1662557675 shutdown_connections 2024-04-12T21:51:08.164 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:08.162+0000 7f131a800700 1 --2- 172.21.15.18:0/1662557675 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f13140fc620 0x7f1314100cd0 unknown :-1 s=CLOSED pgs=1308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:51:08.164 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:08.162+0000 7f131a800700 1 --2- 172.21.15.18:0/1662557675 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f13140fbc80 0x7f13140fc050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:51:08.164 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:08.162+0000 7f131a800700 1 -- 172.21.15.18:0/1662557675 >> 172.21.15.18:0/1662557675 conn(0x7f1314072bc0 msgr2=0x7f1314072fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:51:08.164 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:08.162+0000 7f131a800700 1 -- 172.21.15.18:0/1662557675 shutdown_connections 2024-04-12T21:51:08.164 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:08.162+0000 7f131a800700 1 -- 172.21.15.18:0/1662557675 wait complete. 2024-04-12T21:51:08.165 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:08.163+0000 7f131a800700 1 Processor -- start 2024-04-12T21:51:08.165 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:08.163+0000 7f131a800700 1 -- start start 2024-04-12T21:51:08.165 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:08.163+0000 7f131a800700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f13140fbc80 0x7f131411a1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:51:08.166 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:08.163+0000 7f131a800700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f13140fc620 0x7f131411a710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:51:08.166 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:08.163+0000 7f131a800700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f131411ad60 con 0x7f13140fc620 2024-04-12T21:51:08.166 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:08.163+0000 7f131a800700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1314114130 con 0x7f13140fbc80 2024-04-12T21:51:08.166 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:08.163+0000 7f130be00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f13140fc620 0x7f131411a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:51:08.166 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:08.164+0000 7f130be00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f13140fc620 0x7f131411a710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:57276/0 (socket says 172.21.15.18:57276) 2024-04-12T21:51:08.166 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:08.164+0000 7f130be00700 1 -- 172.21.15.18:0/900621082 learned_addr learned my addr 172.21.15.18:0/900621082 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:51:08.167 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:08.164+0000 7f1313400700 1 --2- 172.21.15.18:0/900621082 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f13140fbc80 0x7f131411a1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:51:08.167 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:08.164+0000 7f130be00700 1 -- 172.21.15.18:0/900621082 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f13140fbc80 msgr2=0x7f131411a1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:51:08.167 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:08.164+0000 7f130be00700 1 --2- 172.21.15.18:0/900621082 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f13140fbc80 0x7f131411a1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:51:08.167 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:08.164+0000 7f130be00700 1 -- 172.21.15.18:0/900621082 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12fc005040 con 0x7f13140fc620 2024-04-12T21:51:08.167 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:08.164+0000 7f130be00700 1 --2- 172.21.15.18:0/900621082 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f13140fc620 0x7f131411a710 secure :-1 s=READY pgs=1309 cs=0 l=1 rev1=1 crypto rx=0x7f12fc009180 tx=0x7f12fc009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:51:08.167 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:08.165+0000 7f130b400700 1 -- 172.21.15.18:0/900621082 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f12fc00e040 con 0x7f13140fc620 2024-04-12T21:51:08.167 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:08.165+0000 7f131a800700 1 -- 172.21.15.18:0/900621082 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f13141143b0 con 0x7f13140fc620 2024-04-12T21:51:08.167 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:08.165+0000 7f130b400700 1 -- 172.21.15.18:0/900621082 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f12fc00bbc0 con 0x7f13140fc620 2024-04-12T21:51:08.167 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:08.165+0000 7f130b400700 1 -- 172.21.15.18:0/900621082 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f12fc01c630 con 0x7f13140fc620 2024-04-12T21:51:08.168 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:08.165+0000 7f131a800700 1 -- 172.21.15.18:0/900621082 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f13141148f0 con 0x7f13140fc620 2024-04-12T21:51:08.168 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:08.166+0000 7f131a800700 1 -- 172.21.15.18:0/900621082 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1314065e30 con 0x7f13140fc620 2024-04-12T21:51:08.169 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:08.167+0000 7f130b400700 1 -- 172.21.15.18:0/900621082 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7f12fc00bd30 con 0x7f13140fc620 2024-04-12T21:51:08.170 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:08.168+0000 7f130b400700 1 --2- 172.21.15.18:0/900621082 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f12f806ec10 0x7f12f80710c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:51:08.170 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:08.168+0000 7f130b400700 1 -- 172.21.15.18:0/900621082 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 6757+0+0 (secure 0 0 0) 0x7f12fc08fe10 con 0x7f13140fc620 2024-04-12T21:51:08.170 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:08.168+0000 7f1313400700 1 --2- 172.21.15.18:0/900621082 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f12f806ec10 0x7f12f80710c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:51:08.170 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:08.169+0000 7f1313400700 1 --2- 172.21.15.18:0/900621082 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f12f806ec10 0x7f12f80710c0 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7f13040041d0 tx=0x7f1304009040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:51:08.174 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:08.173+0000 7f130b400700 1 -- 172.21.15.18:0/900621082 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f12fc05c5c0 con 0x7f13140fc620 2024-04-12T21:51:08.355 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:51:08 smithi018 ceph-mon[112590]: from='client.36668 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:51:08.356 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:51:08 smithi018 ceph-mon[112590]: pgmap v1543: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:51:08.356 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:51:08 smithi018 ceph-mon[112590]: from='client.36672 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:51:08.356 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:51:08 smithi018 ceph-mon[112590]: from='client.? 172.21.15.18:0/1237734531' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T21:51:08.356 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:51:08 smithi018 ceph-mon[112590]: from='client.? 172.21.15.18:0/3143708462' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-12T21:51:08.489 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:08.487+0000 7f131a800700 1 -- 172.21.15.18:0/900621082 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f131405f5f0 con 0x7f13140fc620 2024-04-12T21:51:08.489 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:08.487+0000 7f130b400700 1 -- 172.21.15.18:0/900621082 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+4303 (secure 0 0 0) 0x7f12fc05c150 con 0x7f13140fc620 2024-04-12T21:51:08.490 INFO:teuthology.orchestra.run.smithi018.stdout:HEALTH_ERR failed to probe daemons or devices; 1 filesystem is degraded; 1 filesystem is offline 2024-04-12T21:51:08.490 INFO:teuthology.orchestra.run.smithi018.stdout:[WRN] CEPHADM_REFRESH_FAILED: failed to probe daemons or devices 2024-04-12T21:51:08.490 INFO:teuthology.orchestra.run.smithi018.stdout: host smithi186 `cephadm ceph-volume` failed: cephadm exited with an error code: 1, stderr: Inferring config /var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/mon.smithi186/config 2024-04-12T21:51:08.490 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/podman: timeout after 895 seconds 2024-04-12T21:51:08.490 INFO:teuthology.orchestra.run.smithi018.stdout:Non-zero exit code 124 from /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:f4452def3be4d350f4a8e5feaf7ebb1511be9298fe68478ba7a2d6fbd7d69a63 -e NODE_NAME=smithi186 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968:/var/run/ceph:z -v /var/log/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968:/var/log/ceph:z -v /var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmptgom53f9:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:f4452def3be4d350f4a8e5feaf7ebb1511be9298fe68478ba7a2d6fbd7d69a63 inventory --format=json-pretty --filter-for-batch 2024-04-12T21:51:08.490 INFO:teuthology.orchestra.run.smithi018.stdout:Traceback (most recent call last): 2024-04-12T21:51:08.490 INFO:teuthology.orchestra.run.smithi018.stdout: File "/usr/lib64/python3.9/runpy.py", line 197, in _run_module_as_main 2024-04-12T21:51:08.491 INFO:teuthology.orchestra.run.smithi018.stdout: return _run_code(code, main_globals, None, 2024-04-12T21:51:08.491 INFO:teuthology.orchestra.run.smithi018.stdout: File "/usr/lib64/python3.9/runpy.py", line 87, in _run_code 2024-04-12T21:51:08.491 INFO:teuthology.orchestra.run.smithi018.stdout: exec(code, run_globals) 2024-04-12T21:51:08.491 INFO:teuthology.orchestra.run.smithi018.stdout: File "/var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/cephadm.e44f3ed5e34e4ef52eedef233d1190bff004470d8e8005127b5312bff357fa7f/__main__.py", line 5618, in 2024-04-12T21:51:08.491 INFO:teuthology.orchestra.run.smithi018.stdout: File "/var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/cephadm.e44f3ed5e34e4ef52eedef233d1190bff004470d8e8005127b5312bff357fa7f/__main__.py", line 5606, in main 2024-04-12T21:51:08.491 INFO:teuthology.orchestra.run.smithi018.stdout: File "/var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/cephadm.e44f3ed5e34e4ef52eedef233d1190bff004470d8e8005127b5312bff357fa7f/__main__.py", line 408, in _infer_config 2024-04-12T21:51:08.491 INFO:teuthology.orchestra.run.smithi018.stdout: File "/var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/cephadm.e44f3ed5e34e4ef52eedef233d1190bff004470d8e8005127b5312bff357fa7f/__main__.py", line 323, in _infer_fsid 2024-04-12T21:51:08.491 INFO:teuthology.orchestra.run.smithi018.stdout: File "/var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/cephadm.e44f3ed5e34e4ef52eedef233d1190bff004470d8e8005127b5312bff357fa7f/__main__.py", line 436, in _infer_image 2024-04-12T21:51:08.491 INFO:teuthology.orchestra.run.smithi018.stdout: File "/var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/cephadm.e44f3ed5e34e4ef52eedef233d1190bff004470d8e8005127b5312bff357fa7f/__main__.py", line 310, in _validate_fsid 2024-04-12T21:51:08.491 INFO:teuthology.orchestra.run.smithi018.stdout: File "/var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/cephadm.e44f3ed5e34e4ef52eedef233d1190bff004470d8e8005127b5312bff357fa7f/__main__.py", line 3359, in command_ceph_volume 2024-04-12T21:51:08.491 INFO:teuthology.orchestra.run.smithi018.stdout: File "/var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/cephadm.e44f3ed5e34e4ef52eedef233d1190bff004470d8e8005127b5312bff357fa7f/cephadmlib/call_wrappers.py", line 310, in call_throws 2024-04-12T21:51:08.491 INFO:teuthology.orchestra.run.smithi018.stdout:RuntimeError: Failed command: /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:f4452def3be4d350f4a8e5feaf7ebb1511be9298fe68478ba7a2d6fbd7d69a63 -e NODE_NAME=smithi186 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968:/var/run/ceph:z -v /var/log/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968:/var/log/ceph:z -v /var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmptgom53f9:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:f4452def3be4d350f4a8e5feaf7ebb1511be9298fe68478ba7a2d6fbd7d69a63 inventory --format=json-pretty --filter-for-batch 2024-04-12T21:51:08.491 INFO:teuthology.orchestra.run.smithi018.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-04-12T21:51:08.491 INFO:teuthology.orchestra.run.smithi018.stdout: fs cephfs is degraded 2024-04-12T21:51:08.491 INFO:teuthology.orchestra.run.smithi018.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-04-12T21:51:08.492 INFO:teuthology.orchestra.run.smithi018.stdout: fs cephfs is offline because no MDS is active for it. 2024-04-12T21:51:08.492 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:08.490+0000 7f131a800700 1 -- 172.21.15.18:0/900621082 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f12f806ec10 msgr2=0x7f12f80710c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:51:08.493 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:08.490+0000 7f131a800700 1 --2- 172.21.15.18:0/900621082 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f12f806ec10 0x7f12f80710c0 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7f13040041d0 tx=0x7f1304009040 comp rx=0 tx=0).stop 2024-04-12T21:51:08.493 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:08.490+0000 7f131a800700 1 -- 172.21.15.18:0/900621082 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f13140fc620 msgr2=0x7f131411a710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:51:08.493 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:08.490+0000 7f131a800700 1 --2- 172.21.15.18:0/900621082 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f13140fc620 0x7f131411a710 secure :-1 s=READY pgs=1309 cs=0 l=1 rev1=1 crypto rx=0x7f12fc009180 tx=0x7f12fc009e30 comp rx=0 tx=0).stop 2024-04-12T21:51:08.493 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:08.491+0000 7f131a800700 1 -- 172.21.15.18:0/900621082 shutdown_connections 2024-04-12T21:51:08.493 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:08.491+0000 7f131a800700 1 --2- 172.21.15.18:0/900621082 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f12f806ec10 0x7f12f80710c0 unknown :-1 s=CLOSED pgs=693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:51:08.493 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:08.491+0000 7f131a800700 1 --2- 172.21.15.18:0/900621082 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f13140fbc80 0x7f131411a1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:51:08.493 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:08.491+0000 7f131a800700 1 --2- 172.21.15.18:0/900621082 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f13140fc620 0x7f131411a710 unknown :-1 s=CLOSED pgs=1309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:51:08.493 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:08.491+0000 7f131a800700 1 -- 172.21.15.18:0/900621082 >> 172.21.15.18:0/900621082 conn(0x7f1314072bc0 msgr2=0x7f131410aab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:51:08.493 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:08.491+0000 7f131a800700 1 -- 172.21.15.18:0/900621082 shutdown_connections 2024-04-12T21:51:08.493 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:08.491+0000 7f131a800700 1 -- 172.21.15.18:0/900621082 wait complete. 2024-04-12T21:51:08.534 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:51:08 smithi186 ceph-mon[86379]: from='client.36668 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:51:08.535 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:51:08 smithi186 ceph-mon[86379]: pgmap v1543: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:51:08.535 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:51:08 smithi186 ceph-mon[86379]: from='client.36672 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:51:08.535 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:51:08 smithi186 ceph-mon[86379]: from='client.? 172.21.15.18:0/1237734531' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T21:51:08.535 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:51:08 smithi186 ceph-mon[86379]: from='client.? 172.21.15.18:0/3143708462' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-12T21:51:09.535 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:51:09 smithi186 ceph-mon[86379]: from='client.36684 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:51:09.535 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:51:09 smithi186 ceph-mon[86379]: from='client.? 172.21.15.18:0/900621082' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-12T21:51:09.605 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:51:09 smithi018 ceph-mon[112590]: from='client.36684 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:51:09.606 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:51:09 smithi018 ceph-mon[112590]: from='client.? 172.21.15.18:0/900621082' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-12T21:51:10.537 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:51:10 smithi186 ceph-mon[86379]: pgmap v1544: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:51:10.538 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:51:10 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T21:51:10.605 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:51:10 smithi018 ceph-mon[112590]: pgmap v1544: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:51:10.605 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:51:10 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T21:51:12.534 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:51:12 smithi186 ceph-mon[86379]: pgmap v1545: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:51:12.605 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:51:12 smithi018 ceph-mon[112590]: pgmap v1545: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:51:14.534 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:51:14 smithi186 ceph-mon[86379]: pgmap v1546: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:51:14.605 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:51:14 smithi018 ceph-mon[112590]: pgmap v1546: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:51:15.507 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:51:15 smithi018 ceph-mon[112590]: pgmap v1547: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:51:15.534 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:51:15 smithi186 ceph-mon[86379]: pgmap v1547: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:51:17.784 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:51:17 smithi186 ceph-mon[86379]: pgmap v1548: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:51:17.855 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:51:17 smithi018 ceph-mon[112590]: pgmap v1548: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:51:19.784 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:51:19 smithi186 ceph-mon[86379]: pgmap v1549: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:51:19.855 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:51:19 smithi018 ceph-mon[112590]: pgmap v1549: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:51:21.784 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:51:21 smithi186 ceph-mon[86379]: pgmap v1550: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:51:21.856 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:51:21 smithi018 ceph-mon[112590]: pgmap v1550: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:51:23.855 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:51:23 smithi018 ceph-mon[112590]: pgmap v1551: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:51:24.034 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:51:23 smithi186 ceph-mon[86379]: pgmap v1551: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:51:25.855 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:51:25 smithi018 ceph-mon[112590]: pgmap v1552: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:51:25.856 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:51:25 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T21:51:26.034 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:51:25 smithi186 ceph-mon[86379]: pgmap v1552: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:51:26.035 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:51:25 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T21:51:27.855 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:51:27 smithi018 ceph-mon[112590]: pgmap v1553: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:51:28.034 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:51:27 smithi186 ceph-mon[86379]: pgmap v1553: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:51:29.837 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:51:29 smithi186 ceph-mon[86379]: pgmap v1554: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:51:29.855 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:51:29 smithi018 ceph-mon[112590]: pgmap v1554: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:51:32.034 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:51:31 smithi186 ceph-mon[86379]: pgmap v1555: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:51:32.105 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:51:31 smithi018 ceph-mon[112590]: pgmap v1555: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:51:34.034 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:51:33 smithi186 ceph-mon[86379]: pgmap v1556: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:51:34.105 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:51:33 smithi018 ceph-mon[112590]: pgmap v1556: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:51:36.034 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:51:35 smithi186 ceph-mon[86379]: pgmap v1557: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:51:36.105 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:51:35 smithi018 ceph-mon[112590]: pgmap v1557: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:51:38.035 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:51:37 smithi186 ceph-mon[86379]: pgmap v1558: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:51:38.105 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:51:37 smithi018 ceph-mon[112590]: pgmap v1558: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:51:38.669 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:38.667+0000 7f280f200700 1 -- 172.21.15.18:0/175178386 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f28100708a0 msgr2=0x7f2810070d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:51:38.669 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:38.667+0000 7f280f200700 1 --2- 172.21.15.18:0/175178386 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f28100708a0 0x7f2810070d10 secure :-1 s=READY pgs=1310 cs=0 l=1 rev1=1 crypto rx=0x7f27fc004970 tx=0x7f27fc009e30 comp rx=0 tx=0).stop 2024-04-12T21:51:38.669 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:38.667+0000 7f280f200700 1 -- 172.21.15.18:0/175178386 shutdown_connections 2024-04-12T21:51:38.669 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:38.667+0000 7f280f200700 1 --2- 172.21.15.18:0/175178386 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f28100708a0 0x7f2810070d10 unknown :-1 s=CLOSED pgs=1310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:51:38.669 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:38.667+0000 7f280f200700 1 --2- 172.21.15.18:0/175178386 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2810071f90 0x7f2810070360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:51:38.669 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:38.667+0000 7f280f200700 1 -- 172.21.15.18:0/175178386 >> 172.21.15.18:0/175178386 conn(0x7f28100f8760 msgr2=0x7f28100fab70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:51:38.669 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:38.668+0000 7f280f200700 1 -- 172.21.15.18:0/175178386 shutdown_connections 2024-04-12T21:51:38.669 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:38.668+0000 7f280f200700 1 -- 172.21.15.18:0/175178386 wait complete. 2024-04-12T21:51:38.670 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:38.669+0000 7f280f200700 1 Processor -- start 2024-04-12T21:51:38.670 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:38.669+0000 7f280f200700 1 -- start start 2024-04-12T21:51:38.672 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:38.669+0000 7f280f200700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f28100708a0 0x7f2810119fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:51:38.672 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:38.669+0000 7f280f200700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2810071f90 0x7f281011a4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:51:38.672 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:38.669+0000 7f280f200700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f281011ab30 con 0x7f28100708a0 2024-04-12T21:51:38.672 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:38.669+0000 7f280f200700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2810113f00 con 0x7f2810071f90 2024-04-12T21:51:38.673 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:38.670+0000 7f280de00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f28100708a0 0x7f2810119fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:51:38.673 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:38.670+0000 7f280de00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f28100708a0 0x7f2810119fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:57056/0 (socket says 172.21.15.18:57056) 2024-04-12T21:51:38.673 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:38.670+0000 7f280de00700 1 -- 172.21.15.18:0/2709059142 learned_addr learned my addr 172.21.15.18:0/2709059142 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:51:38.673 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:38.670+0000 7f280d400700 1 --2- 172.21.15.18:0/2709059142 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2810071f90 0x7f281011a4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:51:38.673 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:38.670+0000 7f280de00700 1 -- 172.21.15.18:0/2709059142 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2810071f90 msgr2=0x7f281011a4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:51:38.673 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:38.670+0000 7f280de00700 1 --2- 172.21.15.18:0/2709059142 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2810071f90 0x7f281011a4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:51:38.673 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:38.670+0000 7f280de00700 1 -- 172.21.15.18:0/2709059142 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27fc005040 con 0x7f28100708a0 2024-04-12T21:51:38.673 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:38.670+0000 7f280d400700 1 --2- 172.21.15.18:0/2709059142 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2810071f90 0x7f281011a4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-12T21:51:38.673 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:38.671+0000 7f280de00700 1 --2- 172.21.15.18:0/2709059142 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f28100708a0 0x7f2810119fa0 secure :-1 s=READY pgs=1311 cs=0 l=1 rev1=1 crypto rx=0x7f280000c830 tx=0x7f2800004470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:51:38.673 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:38.671+0000 7f2806000700 1 -- 172.21.15.18:0/2709059142 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f280000a760 con 0x7f28100708a0 2024-04-12T21:51:38.673 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:38.671+0000 7f2806000700 1 -- 172.21.15.18:0/2709059142 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f280000ad90 con 0x7f28100708a0 2024-04-12T21:51:38.673 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:38.671+0000 7f280f200700 1 -- 172.21.15.18:0/2709059142 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f28101141e0 con 0x7f28100708a0 2024-04-12T21:51:38.673 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:38.671+0000 7f2806000700 1 -- 172.21.15.18:0/2709059142 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f28000113a0 con 0x7f28100708a0 2024-04-12T21:51:38.674 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:38.671+0000 7f280f200700 1 -- 172.21.15.18:0/2709059142 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2810114720 con 0x7f28100708a0 2024-04-12T21:51:38.674 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:38.672+0000 7f280f200700 1 -- 172.21.15.18:0/2709059142 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2810071aa0 con 0x7f28100708a0 2024-04-12T21:51:38.674 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:38.673+0000 7f2806000700 1 -- 172.21.15.18:0/2709059142 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7f2800012400 con 0x7f28100708a0 2024-04-12T21:51:38.676 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:38.674+0000 7f2806000700 1 --2- 172.21.15.18:0/2709059142 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f27f406ec00 0x7f27f40710b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:51:38.676 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:38.674+0000 7f2806000700 1 -- 172.21.15.18:0/2709059142 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 6757+0+0 (secure 0 0 0) 0x7f280008e900 con 0x7f28100708a0 2024-04-12T21:51:38.676 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:38.675+0000 7f280d400700 1 --2- 172.21.15.18:0/2709059142 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f27f406ec00 0x7f27f40710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:51:38.677 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:38.675+0000 7f280d400700 1 --2- 172.21.15.18:0/2709059142 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f27f406ec00 0x7f27f40710b0 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f27fc00b9c0 tx=0x7f27fc009e30 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:51:38.681 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:38.680+0000 7f2806000700 1 -- 172.21.15.18:0/2709059142 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f280005b1f0 con 0x7f28100708a0 2024-04-12T21:51:38.933 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:38.932+0000 7f280f200700 1 -- 172.21.15.18:0/2709059142 --> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f281004bed0 con 0x7f27f406ec00 2024-04-12T21:51:38.936 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:38.935+0000 7f2806000700 1 -- 172.21.15.18:0/2709059142 <== mgr.34102 v2:172.21.15.18:6800/3812891919 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f281004bed0 con 0x7f27f406ec00 2024-04-12T21:51:38.940 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:38.938+0000 7f280f200700 1 -- 172.21.15.18:0/2709059142 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f27f406ec00 msgr2=0x7f27f40710b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:51:38.940 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:38.938+0000 7f280f200700 1 --2- 172.21.15.18:0/2709059142 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f27f406ec00 0x7f27f40710b0 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f27fc00b9c0 tx=0x7f27fc009e30 comp rx=0 tx=0).stop 2024-04-12T21:51:38.940 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:38.939+0000 7f280f200700 1 -- 172.21.15.18:0/2709059142 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f28100708a0 msgr2=0x7f2810119fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:51:38.940 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:38.939+0000 7f280f200700 1 --2- 172.21.15.18:0/2709059142 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f28100708a0 0x7f2810119fa0 secure :-1 s=READY pgs=1311 cs=0 l=1 rev1=1 crypto rx=0x7f280000c830 tx=0x7f2800004470 comp rx=0 tx=0).stop 2024-04-12T21:51:38.940 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:38.939+0000 7f280f200700 1 -- 172.21.15.18:0/2709059142 shutdown_connections 2024-04-12T21:51:38.941 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:38.939+0000 7f280f200700 1 --2- 172.21.15.18:0/2709059142 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f27f406ec00 0x7f27f40710b0 unknown :-1 s=CLOSED pgs=694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:51:38.941 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:38.939+0000 7f280f200700 1 --2- 172.21.15.18:0/2709059142 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f28100708a0 0x7f2810119fa0 unknown :-1 s=CLOSED pgs=1311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:51:38.941 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:38.939+0000 7f280f200700 1 --2- 172.21.15.18:0/2709059142 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2810071f90 0x7f281011a4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:51:38.941 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:38.939+0000 7f280f200700 1 -- 172.21.15.18:0/2709059142 >> 172.21.15.18:0/2709059142 conn(0x7f28100f8760 msgr2=0x7f28100f9850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:51:38.941 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:38.939+0000 7f280f200700 1 -- 172.21.15.18:0/2709059142 shutdown_connections 2024-04-12T21:51:38.941 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:38.939+0000 7f280f200700 1 -- 172.21.15.18:0/2709059142 wait complete. 2024-04-12T21:51:38.956 INFO:teuthology.orchestra.run.smithi018.stdout:true 2024-04-12T21:51:39.105 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:39.104+0000 7f0a8e000700 1 -- 172.21.15.18:0/1110603250 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0a88072e00 msgr2=0x7f0a8810d760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:51:39.106 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:39.104+0000 7f0a8e000700 1 --2- 172.21.15.18:0/1110603250 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0a88072e00 0x7f0a8810d760 secure :-1 s=READY pgs=1312 cs=0 l=1 rev1=1 crypto rx=0x7f0a74004940 tx=0x7f0a74009e30 comp rx=0 tx=0).stop 2024-04-12T21:51:39.106 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:39.104+0000 7f0a8e000700 1 -- 172.21.15.18:0/1110603250 shutdown_connections 2024-04-12T21:51:39.106 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:39.104+0000 7f0a8e000700 1 --2- 172.21.15.18:0/1110603250 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0a88072e00 0x7f0a8810d760 unknown :-1 s=CLOSED pgs=1312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:51:39.106 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:39.104+0000 7f0a8e000700 1 --2- 172.21.15.18:0/1110603250 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0a880724f0 0x7f0a880728c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:51:39.106 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:39.104+0000 7f0a8e000700 1 -- 172.21.15.18:0/1110603250 >> 172.21.15.18:0/1110603250 conn(0x7f0a880faba0 msgr2=0x7f0a880fcfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:51:39.106 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:39.104+0000 7f0a8e000700 1 -- 172.21.15.18:0/1110603250 shutdown_connections 2024-04-12T21:51:39.106 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:39.104+0000 7f0a8e000700 1 -- 172.21.15.18:0/1110603250 wait complete. 2024-04-12T21:51:39.108 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:39.105+0000 7f0a8e000700 1 Processor -- start 2024-04-12T21:51:39.108 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:39.105+0000 7f0a8e000700 1 -- start start 2024-04-12T21:51:39.108 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:39.106+0000 7f0a8e000700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0a880724f0 0x7f0a88119ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:51:39.108 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:39.106+0000 7f0a8e000700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0a88072e00 0x7f0a8811a410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:51:39.108 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:39.106+0000 7f0a8e000700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a8811aaf0 con 0x7f0a88072e00 2024-04-12T21:51:39.108 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:39.106+0000 7f0a8e000700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a88114040 con 0x7f0a880724f0 2024-04-12T21:51:39.108 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:39.106+0000 7f0a86a00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0a88072e00 0x7f0a8811a410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:51:39.108 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:39.106+0000 7f0a86a00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0a88072e00 0x7f0a8811a410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:57066/0 (socket says 172.21.15.18:57066) 2024-04-12T21:51:39.109 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:39.106+0000 7f0a86a00700 1 -- 172.21.15.18:0/4275742633 learned_addr learned my addr 172.21.15.18:0/4275742633 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:51:39.109 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:39.106+0000 7f0a87400700 1 --2- 172.21.15.18:0/4275742633 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0a880724f0 0x7f0a88119ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:51:39.109 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:39.106+0000 7f0a86a00700 1 -- 172.21.15.18:0/4275742633 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0a880724f0 msgr2=0x7f0a88119ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:51:39.109 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:39.106+0000 7f0a86a00700 1 --2- 172.21.15.18:0/4275742633 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0a880724f0 0x7f0a88119ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:51:39.109 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:39.106+0000 7f0a86a00700 1 -- 172.21.15.18:0/4275742633 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a74005040 con 0x7f0a88072e00 2024-04-12T21:51:39.109 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:39.107+0000 7f0a87400700 1 --2- 172.21.15.18:0/4275742633 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0a880724f0 0x7f0a88119ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-12T21:51:39.109 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:39.107+0000 7f0a86a00700 1 --2- 172.21.15.18:0/4275742633 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0a88072e00 0x7f0a8811a410 secure :-1 s=READY pgs=1313 cs=0 l=1 rev1=1 crypto rx=0x7f0a74009180 tx=0x7f0a74009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:51:39.109 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:39.107+0000 7f0a7f400700 1 -- 172.21.15.18:0/4275742633 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0a7400e040 con 0x7f0a88072e00 2024-04-12T21:51:39.109 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:39.107+0000 7f0a7f400700 1 -- 172.21.15.18:0/4275742633 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f0a74013420 con 0x7f0a88072e00 2024-04-12T21:51:39.109 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:39.107+0000 7f0a8e000700 1 -- 172.21.15.18:0/4275742633 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0a88114260 con 0x7f0a88072e00 2024-04-12T21:51:39.110 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:39.107+0000 7f0a7f400700 1 -- 172.21.15.18:0/4275742633 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0a7401c630 con 0x7f0a88072e00 2024-04-12T21:51:39.110 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:39.107+0000 7f0a8e000700 1 -- 172.21.15.18:0/4275742633 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0a88114820 con 0x7f0a88072e00 2024-04-12T21:51:39.110 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:39.108+0000 7f0a8e000700 1 -- 172.21.15.18:0/4275742633 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0a88073c30 con 0x7f0a88072e00 2024-04-12T21:51:39.111 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:39.110+0000 7f0a7f400700 1 -- 172.21.15.18:0/4275742633 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7f0a74013590 con 0x7f0a88072e00 2024-04-12T21:51:39.112 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:39.111+0000 7f0a7f400700 1 --2- 172.21.15.18:0/4275742633 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f0a6c06ec00 0x7f0a6c0710b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:51:39.112 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:39.111+0000 7f0a7f400700 1 -- 172.21.15.18:0/4275742633 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 6757+0+0 (secure 0 0 0) 0x7f0a7408fd20 con 0x7f0a88072e00 2024-04-12T21:51:39.112 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:39.111+0000 7f0a87400700 1 --2- 172.21.15.18:0/4275742633 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f0a6c06ec00 0x7f0a6c0710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:51:39.113 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:39.112+0000 7f0a87400700 1 --2- 172.21.15.18:0/4275742633 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f0a6c06ec00 0x7f0a6c0710b0 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7f0a78006fd0 tx=0x7f0a78008040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:51:39.117 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:39.116+0000 7f0a7f400700 1 -- 172.21.15.18:0/4275742633 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f0a7405c550 con 0x7f0a88072e00 2024-04-12T21:51:39.371 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:39.370+0000 7f0a8e000700 1 -- 172.21.15.18:0/4275742633 --> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f0a88114b50 con 0x7f0a6c06ec00 2024-04-12T21:51:39.374 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:39.373+0000 7f0a7f400700 1 -- 172.21.15.18:0/4275742633 <== mgr.34102 v2:172.21.15.18:6800/3812891919 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f0a88114b50 con 0x7f0a6c06ec00 2024-04-12T21:51:39.377 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:39.375+0000 7f0a8e000700 1 -- 172.21.15.18:0/4275742633 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f0a6c06ec00 msgr2=0x7f0a6c0710b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:51:39.377 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:39.375+0000 7f0a8e000700 1 --2- 172.21.15.18:0/4275742633 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f0a6c06ec00 0x7f0a6c0710b0 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7f0a78006fd0 tx=0x7f0a78008040 comp rx=0 tx=0).stop 2024-04-12T21:51:39.377 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:39.375+0000 7f0a8e000700 1 -- 172.21.15.18:0/4275742633 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0a88072e00 msgr2=0x7f0a8811a410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:51:39.377 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:39.375+0000 7f0a8e000700 1 --2- 172.21.15.18:0/4275742633 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0a88072e00 0x7f0a8811a410 secure :-1 s=READY pgs=1313 cs=0 l=1 rev1=1 crypto rx=0x7f0a74009180 tx=0x7f0a74009e30 comp rx=0 tx=0).stop 2024-04-12T21:51:39.377 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:39.376+0000 7f0a8e000700 1 -- 172.21.15.18:0/4275742633 shutdown_connections 2024-04-12T21:51:39.377 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:39.376+0000 7f0a8e000700 1 --2- 172.21.15.18:0/4275742633 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f0a6c06ec00 0x7f0a6c0710b0 unknown :-1 s=CLOSED pgs=695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:51:39.377 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:39.376+0000 7f0a8e000700 1 --2- 172.21.15.18:0/4275742633 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0a880724f0 0x7f0a88119ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:51:39.377 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:39.376+0000 7f0a8e000700 1 --2- 172.21.15.18:0/4275742633 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0a88072e00 0x7f0a8811a410 unknown :-1 s=CLOSED pgs=1313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:51:39.377 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:39.376+0000 7f0a8e000700 1 -- 172.21.15.18:0/4275742633 >> 172.21.15.18:0/4275742633 conn(0x7f0a880faba0 msgr2=0x7f0a88075040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:51:39.377 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:39.376+0000 7f0a8e000700 1 -- 172.21.15.18:0/4275742633 shutdown_connections 2024-04-12T21:51:39.378 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:39.376+0000 7f0a8e000700 1 -- 172.21.15.18:0/4275742633 wait complete. 2024-04-12T21:51:39.535 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:39.533+0000 7f800e000700 1 -- 172.21.15.18:0/2545698221 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f80081059c0 msgr2=0x7f80080ff080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:51:39.535 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:39.533+0000 7f800e000700 1 --2- 172.21.15.18:0/2545698221 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f80081059c0 0x7f80080ff080 secure :-1 s=READY pgs=1314 cs=0 l=1 rev1=1 crypto rx=0x7f7ff4004940 tx=0x7f7ff4009e30 comp rx=0 tx=0).stop 2024-04-12T21:51:39.535 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:39.533+0000 7f800e000700 1 -- 172.21.15.18:0/2545698221 shutdown_connections 2024-04-12T21:51:39.535 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:39.533+0000 7f800e000700 1 --2- 172.21.15.18:0/2545698221 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f80081059c0 0x7f80080ff080 unknown :-1 s=CLOSED pgs=1314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:51:39.535 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:39.533+0000 7f800e000700 1 --2- 172.21.15.18:0/2545698221 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8008105020 0x7f80081053f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:51:39.535 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:39.533+0000 7f800e000700 1 -- 172.21.15.18:0/2545698221 >> 172.21.15.18:0/2545698221 conn(0x7f80080faba0 msgr2=0x7f80080fcfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:51:39.535 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:39.533+0000 7f800e000700 1 -- 172.21.15.18:0/2545698221 shutdown_connections 2024-04-12T21:51:39.535 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:39.533+0000 7f800e000700 1 -- 172.21.15.18:0/2545698221 wait complete. 2024-04-12T21:51:39.537 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:39.534+0000 7f800e000700 1 Processor -- start 2024-04-12T21:51:39.537 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:39.535+0000 7f800e000700 1 -- start start 2024-04-12T21:51:39.537 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:39.535+0000 7f800e000700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8008105020 0x7f8008115b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:51:39.537 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:39.535+0000 7f800e000700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f80081059c0 0x7f8008116050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:51:39.537 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:39.535+0000 7f800e000700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8008116730 con 0x7f8008105020 2024-04-12T21:51:39.537 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:39.535+0000 7f800e000700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f800810fcc0 con 0x7f80081059c0 2024-04-12T21:51:39.537 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:39.535+0000 7f8007400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8008105020 0x7f8008115b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:51:39.537 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:39.535+0000 7f8007400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8008105020 0x7f8008115b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:57084/0 (socket says 172.21.15.18:57084) 2024-04-12T21:51:39.537 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:39.535+0000 7f8007400700 1 -- 172.21.15.18:0/431194638 learned_addr learned my addr 172.21.15.18:0/431194638 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:51:39.537 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:39.536+0000 7f8006a00700 1 --2- 172.21.15.18:0/431194638 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f80081059c0 0x7f8008116050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:51:39.537 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:39.536+0000 7f8007400700 1 -- 172.21.15.18:0/431194638 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f80081059c0 msgr2=0x7f8008116050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:51:39.538 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:39.536+0000 7f8007400700 1 --2- 172.21.15.18:0/431194638 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f80081059c0 0x7f8008116050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:51:39.538 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:39.536+0000 7f8007400700 1 -- 172.21.15.18:0/431194638 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ff4005040 con 0x7f8008105020 2024-04-12T21:51:39.538 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:39.536+0000 7f8007400700 1 --2- 172.21.15.18:0/431194638 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8008105020 0x7f8008115b10 secure :-1 s=READY pgs=1315 cs=0 l=1 rev1=1 crypto rx=0x7f7ff800b790 tx=0x7f7ff8009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:51:39.538 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:39.536+0000 7f7fff400700 1 -- 172.21.15.18:0/431194638 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7ff800f770 con 0x7f8008105020 2024-04-12T21:51:39.539 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:39.536+0000 7f7fff400700 1 -- 172.21.15.18:0/431194638 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f7ff800fda0 con 0x7f8008105020 2024-04-12T21:51:39.539 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:39.536+0000 7f7fff400700 1 -- 172.21.15.18:0/431194638 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7ff800d310 con 0x7f8008105020 2024-04-12T21:51:39.539 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:39.536+0000 7f800e000700 1 -- 172.21.15.18:0/431194638 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f800810ffa0 con 0x7f8008105020 2024-04-12T21:51:39.539 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:39.537+0000 7f800e000700 1 -- 172.21.15.18:0/431194638 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8008110560 con 0x7f8008105020 2024-04-12T21:51:39.539 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:39.538+0000 7f800e000700 1 -- 172.21.15.18:0/431194638 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8008100770 con 0x7f8008105020 2024-04-12T21:51:39.539 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:39.539+0000 7f7fff400700 1 -- 172.21.15.18:0/431194638 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7f7ff8010420 con 0x7f8008105020 2024-04-12T21:51:39.541 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:39.540+0000 7f7fff400700 1 --2- 172.21.15.18:0/431194638 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f7fec06ec00 0x7f7fec0710b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:51:39.541 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:39.540+0000 7f7fff400700 1 -- 172.21.15.18:0/431194638 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 6757+0+0 (secure 0 0 0) 0x7f7ff808e9a0 con 0x7f8008105020 2024-04-12T21:51:39.541 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:39.540+0000 7f8006a00700 1 --2- 172.21.15.18:0/431194638 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f7fec06ec00 0x7f7fec0710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:51:39.542 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:39.541+0000 7f8006a00700 1 --2- 172.21.15.18:0/431194638 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f7fec06ec00 0x7f7fec0710b0 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7f7ff4000c00 tx=0x7f7ff4009e30 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:51:39.546 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:39.545+0000 7f7fff400700 1 -- 172.21.15.18:0/431194638 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f7ff80576a0 con 0x7f8008105020 2024-04-12T21:51:39.784 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:39.783+0000 7f800e000700 1 -- 172.21.15.18:0/431194638 --> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f8008110830 con 0x7f7fec06ec00 2024-04-12T21:51:39.795 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:39.795+0000 7f7fff400700 1 -- 172.21.15.18:0/431194638 <== mgr.34102 v2:172.21.15.18:6800/3812891919 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7f8008110830 con 0x7f7fec06ec00 2024-04-12T21:51:39.797 INFO:teuthology.orchestra.run.smithi018.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-12T21:51:39.797 INFO:teuthology.orchestra.run.smithi018.stdout:alertmanager.smithi018 smithi018 *:9093,9094 running (52m) 13m ago 66m 14.8M - 0.25.0 c8568f914cd2 c332090068e8 2024-04-12T21:51:39.797 INFO:teuthology.orchestra.run.smithi018.stdout:crash.smithi018 smithi018 running (48m) 13m ago 67m 6665k - 19.0.0-2879-g324baf8d 8717720dc5e5 b7f80d1f7982 2024-04-12T21:51:39.798 INFO:teuthology.orchestra.run.smithi018.stdout:crash.smithi186 smithi186 running (48m) 13m ago 65m 6660k - 19.0.0-2879-g324baf8d 8717720dc5e5 a4f55c81b623 2024-04-12T21:51:39.798 INFO:teuthology.orchestra.run.smithi018.stdout:grafana.smithi018 smithi018 *:3000 running (51m) 13m ago 66m 73.8M - 9.4.12 2bacad6d85d8 b08e3bf176dd 2024-04-12T21:51:39.798 INFO:teuthology.orchestra.run.smithi018.stdout:mds.cephfs.smithi018.acqbsc smithi018 running (44m) 13m ago 59m 21.9M - 19.0.0-2879-g324baf8d 8717720dc5e5 021f99ab24ae 2024-04-12T21:51:39.798 INFO:teuthology.orchestra.run.smithi018.stdout:mds.cephfs.smithi018.szjluo smithi018 running (44m) 13m ago 59m 21.7M - 19.0.0-2879-g324baf8d 8717720dc5e5 0003117479d4 2024-04-12T21:51:39.798 INFO:teuthology.orchestra.run.smithi018.stdout:mds.cephfs.smithi186.jdllyn smithi186 running (28m) 13m ago 59m 16.5M - 19.0.0-2879-g324baf8d 8717720dc5e5 21b1f46eef23 2024-04-12T21:51:39.798 INFO:teuthology.orchestra.run.smithi018.stdout:mds.cephfs.smithi186.wjnafs smithi186 running (13m) 13m ago 59m 11.0M - 19.0.0-2879-g324baf8d 8717720dc5e5 3cf1ef4148e2 2024-04-12T21:51:39.798 INFO:teuthology.orchestra.run.smithi018.stdout:mgr.smithi018.xjzwga smithi018 *:8443,9283,8765 running (54m) 13m ago 68m 517M - 19.0.0-2879-g324baf8d 8717720dc5e5 56088305eab7 2024-04-12T21:51:39.798 INFO:teuthology.orchestra.run.smithi018.stdout:mgr.smithi186.kqcfxk smithi186 *:8443,9283,8765 running (53m) 13m ago 65m 460M - 19.0.0-2879-g324baf8d 8717720dc5e5 09cd766e5e0d 2024-04-12T21:51:39.798 INFO:teuthology.orchestra.run.smithi018.stdout:mon.smithi018 smithi018 running (50m) 13m ago 68m 88.2M 2048M 19.0.0-2879-g324baf8d 8717720dc5e5 df3e201dfed6 2024-04-12T21:51:39.798 INFO:teuthology.orchestra.run.smithi018.stdout:mon.smithi186 smithi186 running (49m) 13m ago 65m 53.6M 2048M 19.0.0-2879-g324baf8d 8717720dc5e5 c77498da9b2b 2024-04-12T21:51:39.798 INFO:teuthology.orchestra.run.smithi018.stdout:node-exporter.smithi018 smithi018 *:9100 running (53m) 13m ago 66m 8468k - 1.5.0 0da6a335fe13 326c3d9c6847 2024-04-12T21:51:39.798 INFO:teuthology.orchestra.run.smithi018.stdout:node-exporter.smithi186 smithi186 *:9100 running (53m) 13m ago 65m 8166k - 1.5.0 0da6a335fe13 ec2112baeecf 2024-04-12T21:51:39.798 INFO:teuthology.orchestra.run.smithi018.stdout:osd.0 smithi018 running (47m) 13m ago 64m 123M 1559M 19.0.0-2879-g324baf8d 8717720dc5e5 60b1273ca047 2024-04-12T21:51:39.798 INFO:teuthology.orchestra.run.smithi018.stdout:osd.1 smithi018 running (47m) 13m ago 63m 95.4M 1559M 19.0.0-2879-g324baf8d 8717720dc5e5 170ce1e5fb8d 2024-04-12T21:51:39.799 INFO:teuthology.orchestra.run.smithi018.stdout:osd.2 smithi018 running (46m) 13m ago 63m 94.6M 1559M 19.0.0-2879-g324baf8d 8717720dc5e5 c2cbc6aa2839 2024-04-12T21:51:39.799 INFO:teuthology.orchestra.run.smithi018.stdout:osd.3 smithi186 running (46m) 13m ago 62m 101M 2581M 19.0.0-2879-g324baf8d 8717720dc5e5 0138e8262e4b 2024-04-12T21:51:39.799 INFO:teuthology.orchestra.run.smithi018.stdout:osd.4 smithi186 running (45m) 13m ago 62m 91.5M 2581M 19.0.0-2879-g324baf8d 8717720dc5e5 b20cb63b25c9 2024-04-12T21:51:39.799 INFO:teuthology.orchestra.run.smithi018.stdout:osd.5 smithi186 running (44m) 13m ago 61m 92.2M 2581M 19.0.0-2879-g324baf8d 8717720dc5e5 2513bea2610e 2024-04-12T21:51:39.799 INFO:teuthology.orchestra.run.smithi018.stdout:prometheus.smithi018 smithi018 *:9095 running (52m) 13m ago 66m 69.6M - 2.43.0 a07b618ecd1d f58841127e21 2024-04-12T21:51:39.800 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:39.797+0000 7f800e000700 1 -- 172.21.15.18:0/431194638 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f7fec06ec00 msgr2=0x7f7fec0710b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:51:39.800 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:39.797+0000 7f800e000700 1 --2- 172.21.15.18:0/431194638 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f7fec06ec00 0x7f7fec0710b0 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7f7ff4000c00 tx=0x7f7ff4009e30 comp rx=0 tx=0).stop 2024-04-12T21:51:39.800 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:39.797+0000 7f800e000700 1 -- 172.21.15.18:0/431194638 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8008105020 msgr2=0x7f8008115b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:51:39.800 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:39.797+0000 7f800e000700 1 --2- 172.21.15.18:0/431194638 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8008105020 0x7f8008115b10 secure :-1 s=READY pgs=1315 cs=0 l=1 rev1=1 crypto rx=0x7f7ff800b790 tx=0x7f7ff8009250 comp rx=0 tx=0).stop 2024-04-12T21:51:39.800 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:39.797+0000 7f800e000700 1 -- 172.21.15.18:0/431194638 shutdown_connections 2024-04-12T21:51:39.800 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:39.797+0000 7f800e000700 1 --2- 172.21.15.18:0/431194638 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f7fec06ec00 0x7f7fec0710b0 unknown :-1 s=CLOSED pgs=696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:51:39.800 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:39.797+0000 7f800e000700 1 --2- 172.21.15.18:0/431194638 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8008105020 0x7f8008115b10 unknown :-1 s=CLOSED pgs=1315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:51:39.800 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:39.797+0000 7f800e000700 1 --2- 172.21.15.18:0/431194638 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f80081059c0 0x7f8008116050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:51:39.800 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:39.797+0000 7f800e000700 1 -- 172.21.15.18:0/431194638 >> 172.21.15.18:0/431194638 conn(0x7f80080faba0 msgr2=0x7f80080fbaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:51:39.800 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:39.798+0000 7f800e000700 1 -- 172.21.15.18:0/431194638 shutdown_connections 2024-04-12T21:51:39.801 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:39.798+0000 7f800e000700 1 -- 172.21.15.18:0/431194638 wait complete. 2024-04-12T21:51:39.956 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:39.955+0000 7f6a3b200700 1 -- 172.21.15.18:0/3734248947 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6a3c101ae0 msgr2=0x7f6a3c065630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:51:39.956 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:39.955+0000 7f6a3b200700 1 --2- 172.21.15.18:0/3734248947 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6a3c101ae0 0x7f6a3c065630 secure :-1 s=READY pgs=1316 cs=0 l=1 rev1=1 crypto rx=0x7f6a2c004940 tx=0x7f6a2c009e30 comp rx=0 tx=0).stop 2024-04-12T21:51:39.956 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:39.955+0000 7f6a3b200700 1 -- 172.21.15.18:0/3734248947 shutdown_connections 2024-04-12T21:51:39.956 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:39.955+0000 7f6a3b200700 1 --2- 172.21.15.18:0/3734248947 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6a3c065b70 0x7f6a3c065fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:51:39.956 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:39.955+0000 7f6a3b200700 1 --2- 172.21.15.18:0/3734248947 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6a3c101ae0 0x7f6a3c065630 unknown :-1 s=CLOSED pgs=1316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:51:39.957 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:39.955+0000 7f6a3b200700 1 -- 172.21.15.18:0/3734248947 >> 172.21.15.18:0/3734248947 conn(0x7f6a3c071b70 msgr2=0x7f6a3c071f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:51:39.957 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:39.955+0000 7f6a3b200700 1 -- 172.21.15.18:0/3734248947 shutdown_connections 2024-04-12T21:51:39.957 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:39.955+0000 7f6a3b200700 1 -- 172.21.15.18:0/3734248947 wait complete. 2024-04-12T21:51:39.957 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:51:39 smithi018 ceph-mon[112590]: pgmap v1559: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:51:39.957 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:51:39 smithi018 ceph-mon[112590]: from='client.36692 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:51:39.958 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:39.956+0000 7f6a3b200700 1 Processor -- start 2024-04-12T21:51:39.958 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:39.956+0000 7f6a3b200700 1 -- start start 2024-04-12T21:51:39.958 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:39.957+0000 7f6a3b200700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6a3c065b70 0x7f6a3c119ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:51:39.958 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:39.957+0000 7f6a3b200700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6a3c101ae0 0x7f6a3c11a530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:51:39.958 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:39.957+0000 7f6a3b200700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a3c11ac10 con 0x7f6a3c065b70 2024-04-12T21:51:39.958 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:39.957+0000 7f6a3b200700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a3c113f50 con 0x7f6a3c101ae0 2024-04-12T21:51:39.958 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:39.957+0000 7f6a39e00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6a3c065b70 0x7f6a3c119ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:51:39.958 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:39.957+0000 7f6a39e00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6a3c065b70 0x7f6a3c119ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:57100/0 (socket says 172.21.15.18:57100) 2024-04-12T21:51:39.958 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:39.957+0000 7f6a39e00700 1 -- 172.21.15.18:0/667831685 learned_addr learned my addr 172.21.15.18:0/667831685 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:51:39.959 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:39.957+0000 7f6a39e00700 1 -- 172.21.15.18:0/667831685 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6a3c101ae0 msgr2=0x7f6a3c11a530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:51:39.959 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:39.957+0000 7f6a39e00700 1 --2- 172.21.15.18:0/667831685 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6a3c101ae0 0x7f6a3c11a530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:51:39.959 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:39.957+0000 7f6a39e00700 1 -- 172.21.15.18:0/667831685 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a2c005040 con 0x7f6a3c065b70 2024-04-12T21:51:39.959 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:39.958+0000 7f6a39e00700 1 --2- 172.21.15.18:0/667831685 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6a3c065b70 0x7f6a3c119ff0 secure :-1 s=READY pgs=1317 cs=0 l=1 rev1=1 crypto rx=0x7f6a2c009180 tx=0x7f6a2c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:51:39.959 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:39.958+0000 7f6a32000700 1 -- 172.21.15.18:0/667831685 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6a2c00e040 con 0x7f6a3c065b70 2024-04-12T21:51:39.959 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:39.958+0000 7f6a32000700 1 -- 172.21.15.18:0/667831685 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f6a2c00bbc0 con 0x7f6a3c065b70 2024-04-12T21:51:39.960 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:39.958+0000 7f6a32000700 1 -- 172.21.15.18:0/667831685 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6a2c01c630 con 0x7f6a3c065b70 2024-04-12T21:51:39.960 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:39.958+0000 7f6a3b200700 1 -- 172.21.15.18:0/667831685 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6a3c114170 con 0x7f6a3c065b70 2024-04-12T21:51:39.960 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:39.958+0000 7f6a3b200700 1 -- 172.21.15.18:0/667831685 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6a3c1146b0 con 0x7f6a3c065b70 2024-04-12T21:51:39.961 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:39.960+0000 7f6a3b200700 1 -- 172.21.15.18:0/667831685 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6a3c0fc170 con 0x7f6a3c065b70 2024-04-12T21:51:39.963 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:39.962+0000 7f6a32000700 1 -- 172.21.15.18:0/667831685 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7f6a2c00bd30 con 0x7f6a3c065b70 2024-04-12T21:51:39.964 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:39.963+0000 7f6a32000700 1 --2- 172.21.15.18:0/667831685 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f6a2006ec10 0x7f6a200710c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:51:39.964 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:39.963+0000 7f6a32000700 1 -- 172.21.15.18:0/667831685 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 6757+0+0 (secure 0 0 0) 0x7f6a2c08fd10 con 0x7f6a3c065b70 2024-04-12T21:51:39.964 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:39.963+0000 7f6a39400700 1 --2- 172.21.15.18:0/667831685 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f6a2006ec10 0x7f6a200710c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:51:39.965 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:39.964+0000 7f6a39400700 1 --2- 172.21.15.18:0/667831685 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f6a2006ec10 0x7f6a200710c0 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7f6a28006fd0 tx=0x7f6a28006db0 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:51:39.973 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:39.972+0000 7f6a32000700 1 -- 172.21.15.18:0/667831685 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f6a2c05c580 con 0x7f6a3c065b70 2024-04-12T21:51:40.034 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:51:39 smithi186 ceph-mon[86379]: pgmap v1559: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:51:40.034 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:51:39 smithi186 ceph-mon[86379]: from='client.36692 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:51:40.291 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:40.290+0000 7f6a3b200700 1 -- 172.21.15.18:0/667831685 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f6a3c0fc170 con 0x7f6a3c065b70 2024-04-12T21:51:40.292 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:40.291+0000 7f6a32000700 1 -- 172.21.15.18:0/667831685 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+623 (secure 0 0 0) 0x7f6a2c05c110 con 0x7f6a3c065b70 2024-04-12T21:51:40.293 INFO:teuthology.orchestra.run.smithi018.stdout:{ 2024-04-12T21:51:40.293 INFO:teuthology.orchestra.run.smithi018.stdout: "mon": { 2024-04-12T21:51:40.293 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 19.0.0-2879-g324baf8d (324baf8ddb91dea46cac50cb535c306115791355) squid (dev)": 2 2024-04-12T21:51:40.293 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-12T21:51:40.293 INFO:teuthology.orchestra.run.smithi018.stdout: "mgr": { 2024-04-12T21:51:40.293 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 19.0.0-2879-g324baf8d (324baf8ddb91dea46cac50cb535c306115791355) squid (dev)": 2 2024-04-12T21:51:40.293 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-12T21:51:40.293 INFO:teuthology.orchestra.run.smithi018.stdout: "osd": { 2024-04-12T21:51:40.293 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 19.0.0-2879-g324baf8d (324baf8ddb91dea46cac50cb535c306115791355) squid (dev)": 6 2024-04-12T21:51:40.293 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-12T21:51:40.293 INFO:teuthology.orchestra.run.smithi018.stdout: "mds": { 2024-04-12T21:51:40.293 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 19.0.0-2879-g324baf8d (324baf8ddb91dea46cac50cb535c306115791355) squid (dev)": 4 2024-04-12T21:51:40.293 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-12T21:51:40.294 INFO:teuthology.orchestra.run.smithi018.stdout: "overall": { 2024-04-12T21:51:40.294 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 19.0.0-2879-g324baf8d (324baf8ddb91dea46cac50cb535c306115791355) squid (dev)": 14 2024-04-12T21:51:40.294 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-04-12T21:51:40.294 INFO:teuthology.orchestra.run.smithi018.stdout:} 2024-04-12T21:51:40.295 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:40.294+0000 7f6a3b200700 1 -- 172.21.15.18:0/667831685 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f6a2006ec10 msgr2=0x7f6a200710c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:51:40.295 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:40.294+0000 7f6a3b200700 1 --2- 172.21.15.18:0/667831685 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f6a2006ec10 0x7f6a200710c0 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7f6a28006fd0 tx=0x7f6a28006db0 comp rx=0 tx=0).stop 2024-04-12T21:51:40.295 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:40.294+0000 7f6a3b200700 1 -- 172.21.15.18:0/667831685 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6a3c065b70 msgr2=0x7f6a3c119ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:51:40.296 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:40.294+0000 7f6a3b200700 1 --2- 172.21.15.18:0/667831685 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6a3c065b70 0x7f6a3c119ff0 secure :-1 s=READY pgs=1317 cs=0 l=1 rev1=1 crypto rx=0x7f6a2c009180 tx=0x7f6a2c009e30 comp rx=0 tx=0).stop 2024-04-12T21:51:40.296 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:40.294+0000 7f6a3b200700 1 -- 172.21.15.18:0/667831685 shutdown_connections 2024-04-12T21:51:40.296 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:40.294+0000 7f6a3b200700 1 --2- 172.21.15.18:0/667831685 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f6a2006ec10 0x7f6a200710c0 unknown :-1 s=CLOSED pgs=697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:51:40.296 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:40.294+0000 7f6a3b200700 1 --2- 172.21.15.18:0/667831685 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6a3c065b70 0x7f6a3c119ff0 unknown :-1 s=CLOSED pgs=1317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:51:40.296 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:40.294+0000 7f6a3b200700 1 --2- 172.21.15.18:0/667831685 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6a3c101ae0 0x7f6a3c11a530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:51:40.296 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:40.294+0000 7f6a3b200700 1 -- 172.21.15.18:0/667831685 >> 172.21.15.18:0/667831685 conn(0x7f6a3c071b70 msgr2=0x7f6a3c0f9b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:51:40.296 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:40.294+0000 7f6a3b200700 1 -- 172.21.15.18:0/667831685 shutdown_connections 2024-04-12T21:51:40.296 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:40.294+0000 7f6a3b200700 1 -- 172.21.15.18:0/667831685 wait complete. 2024-04-12T21:51:40.452 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:40.451+0000 7f9d3f400700 1 -- 172.21.15.18:0/2740639158 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9d38101d20 msgr2=0x7f9d381020f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:51:40.452 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:40.451+0000 7f9d3f400700 1 --2- 172.21.15.18:0/2740639158 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9d38101d20 0x7f9d381020f0 secure :-1 s=READY pgs=1318 cs=0 l=1 rev1=1 crypto rx=0x7f9d2c004940 tx=0x7f9d2c009e30 comp rx=0 tx=0).stop 2024-04-12T21:51:40.452 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:40.451+0000 7f9d3f400700 1 -- 172.21.15.18:0/2740639158 shutdown_connections 2024-04-12T21:51:40.452 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:40.451+0000 7f9d3f400700 1 --2- 172.21.15.18:0/2740639158 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9d381026c0 0x7f9d380fbd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:51:40.452 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:40.451+0000 7f9d3f400700 1 --2- 172.21.15.18:0/2740639158 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9d38101d20 0x7f9d381020f0 unknown :-1 s=CLOSED pgs=1318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:51:40.452 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:40.451+0000 7f9d3f400700 1 -- 172.21.15.18:0/2740639158 >> 172.21.15.18:0/2740639158 conn(0x7f9d380f7870 msgr2=0x7f9d380f9c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:51:40.453 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:40.451+0000 7f9d3f400700 1 -- 172.21.15.18:0/2740639158 shutdown_connections 2024-04-12T21:51:40.453 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:40.451+0000 7f9d3f400700 1 -- 172.21.15.18:0/2740639158 wait complete. 2024-04-12T21:51:40.454 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:40.452+0000 7f9d3f400700 1 Processor -- start 2024-04-12T21:51:40.454 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:40.452+0000 7f9d3f400700 1 -- start start 2024-04-12T21:51:40.454 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:40.452+0000 7f9d3f400700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9d38101d20 0x7f9d38115b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:51:40.454 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:40.452+0000 7f9d3f400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9d381026c0 0x7f9d38116090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:51:40.454 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:40.452+0000 7f9d3f400700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d38116770 con 0x7f9d381026c0 2024-04-12T21:51:40.454 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:40.452+0000 7f9d3f400700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d3810fcc0 con 0x7f9d38101d20 2024-04-12T21:51:40.454 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:40.452+0000 7f9d37e00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9d381026c0 0x7f9d38116090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:51:40.454 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:40.452+0000 7f9d37e00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9d381026c0 0x7f9d38116090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:57118/0 (socket says 172.21.15.18:57118) 2024-04-12T21:51:40.454 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:40.453+0000 7f9d37e00700 1 -- 172.21.15.18:0/1576464768 learned_addr learned my addr 172.21.15.18:0/1576464768 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:51:40.455 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:40.453+0000 7f9d37e00700 1 -- 172.21.15.18:0/1576464768 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9d38101d20 msgr2=0x7f9d38115b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:51:40.455 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:40.453+0000 7f9d37e00700 1 --2- 172.21.15.18:0/1576464768 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9d38101d20 0x7f9d38115b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:51:40.455 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:40.453+0000 7f9d37e00700 1 -- 172.21.15.18:0/1576464768 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d2c005040 con 0x7f9d381026c0 2024-04-12T21:51:40.455 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:40.453+0000 7f9d37e00700 1 --2- 172.21.15.18:0/1576464768 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9d381026c0 0x7f9d38116090 secure :-1 s=READY pgs=1319 cs=0 l=1 rev1=1 crypto rx=0x7f9d2400b790 tx=0x7f9d24009220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:51:40.455 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:40.453+0000 7f9d34c00700 1 -- 172.21.15.18:0/1576464768 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9d2400f6f0 con 0x7f9d381026c0 2024-04-12T21:51:40.455 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:40.454+0000 7f9d34c00700 1 -- 172.21.15.18:0/1576464768 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f9d2400fd20 con 0x7f9d381026c0 2024-04-12T21:51:40.455 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:40.454+0000 7f9d3f400700 1 -- 172.21.15.18:0/1576464768 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9d3810ff40 con 0x7f9d381026c0 2024-04-12T21:51:40.456 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:40.454+0000 7f9d34c00700 1 -- 172.21.15.18:0/1576464768 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9d240103a0 con 0x7f9d381026c0 2024-04-12T21:51:40.456 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:40.454+0000 7f9d3f400700 1 -- 172.21.15.18:0/1576464768 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9d38110500 con 0x7f9d381026c0 2024-04-12T21:51:40.456 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:40.455+0000 7f9d3f400700 1 -- 172.21.15.18:0/1576464768 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9d38106220 con 0x7f9d381026c0 2024-04-12T21:51:40.457 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:40.456+0000 7f9d34c00700 1 -- 172.21.15.18:0/1576464768 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7f9d24011400 con 0x7f9d381026c0 2024-04-12T21:51:40.459 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:40.458+0000 7f9d34c00700 1 --2- 172.21.15.18:0/1576464768 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f9d2006ec00 0x7f9d200710b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:51:40.459 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:40.458+0000 7f9d34c00700 1 -- 172.21.15.18:0/1576464768 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 6757+0+0 (secure 0 0 0) 0x7f9d2408e8f0 con 0x7f9d381026c0 2024-04-12T21:51:40.459 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:40.458+0000 7f9d3ca00700 1 --2- 172.21.15.18:0/1576464768 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f9d2006ec00 0x7f9d200710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:51:40.460 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:40.459+0000 7f9d3ca00700 1 --2- 172.21.15.18:0/1576464768 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f9d2006ec00 0x7f9d200710b0 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7f9d2c004f90 tx=0x7f9d2c009e30 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:51:40.463 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:40.463+0000 7f9d34c00700 1 -- 172.21.15.18:0/1576464768 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f9d2405b020 con 0x7f9d381026c0 2024-04-12T21:51:40.752 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:40.751+0000 7f9d3f400700 1 -- 172.21.15.18:0/1576464768 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f9d38108e90 con 0x7f9d381026c0 2024-04-12T21:51:40.753 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:40.752+0000 7f9d34c00700 1 -- 172.21.15.18:0/1576464768 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 33 v33) v1 ==== 76+0+1876 (secure 0 0 0) 0x7f9d2405abb0 con 0x7f9d381026c0 2024-04-12T21:51:40.753 INFO:teuthology.orchestra.run.smithi018.stdout:e33 2024-04-12T21:51:40.753 INFO:teuthology.orchestra.run.smithi018.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-12T21:51:40.754 INFO:teuthology.orchestra.run.smithi018.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-12T21:51:40.754 INFO:teuthology.orchestra.run.smithi018.stdout:legacy client fscid: 1 2024-04-12T21:51:40.754 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-12T21:51:40.754 INFO:teuthology.orchestra.run.smithi018.stdout:Filesystem 'cephfs' (1) 2024-04-12T21:51:40.754 INFO:teuthology.orchestra.run.smithi018.stdout:fs_name cephfs 2024-04-12T21:51:40.754 INFO:teuthology.orchestra.run.smithi018.stdout:epoch 24 2024-04-12T21:51:40.754 INFO:teuthology.orchestra.run.smithi018.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-04-12T21:51:40.754 INFO:teuthology.orchestra.run.smithi018.stdout:created 2024-04-12T20:51:57.362850+0000 2024-04-12T21:51:40.754 INFO:teuthology.orchestra.run.smithi018.stdout:modified 2024-04-12T21:06:59.717190+0000 2024-04-12T21:51:40.754 INFO:teuthology.orchestra.run.smithi018.stdout:tableserver 0 2024-04-12T21:51:40.754 INFO:teuthology.orchestra.run.smithi018.stdout:root 0 2024-04-12T21:51:40.754 INFO:teuthology.orchestra.run.smithi018.stdout:session_timeout 60 2024-04-12T21:51:40.754 INFO:teuthology.orchestra.run.smithi018.stdout:session_autoclose 300 2024-04-12T21:51:40.754 INFO:teuthology.orchestra.run.smithi018.stdout:max_file_size 1099511627776 2024-04-12T21:51:40.755 INFO:teuthology.orchestra.run.smithi018.stdout:max_xattr_size 65536 2024-04-12T21:51:40.755 INFO:teuthology.orchestra.run.smithi018.stdout:required_client_features {} 2024-04-12T21:51:40.755 INFO:teuthology.orchestra.run.smithi018.stdout:last_failure 0 2024-04-12T21:51:40.755 INFO:teuthology.orchestra.run.smithi018.stdout:last_failure_osd_epoch 100 2024-04-12T21:51:40.755 INFO:teuthology.orchestra.run.smithi018.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-12T21:51:40.755 INFO:teuthology.orchestra.run.smithi018.stdout:max_mds 2 2024-04-12T21:51:40.755 INFO:teuthology.orchestra.run.smithi018.stdout:in 0,1 2024-04-12T21:51:40.755 INFO:teuthology.orchestra.run.smithi018.stdout:up {} 2024-04-12T21:51:40.755 INFO:teuthology.orchestra.run.smithi018.stdout:failed 0,1 2024-04-12T21:51:40.755 INFO:teuthology.orchestra.run.smithi018.stdout:damaged 2024-04-12T21:51:40.755 INFO:teuthology.orchestra.run.smithi018.stdout:stopped 2024-04-12T21:51:40.755 INFO:teuthology.orchestra.run.smithi018.stdout:data_pools [3] 2024-04-12T21:51:40.755 INFO:teuthology.orchestra.run.smithi018.stdout:metadata_pool 2 2024-04-12T21:51:40.755 INFO:teuthology.orchestra.run.smithi018.stdout:inline_data disabled 2024-04-12T21:51:40.755 INFO:teuthology.orchestra.run.smithi018.stdout:balancer 2024-04-12T21:51:40.756 INFO:teuthology.orchestra.run.smithi018.stdout:bal_rank_mask -1 2024-04-12T21:51:40.756 INFO:teuthology.orchestra.run.smithi018.stdout:standby_count_wanted 1 2024-04-12T21:51:40.756 INFO:teuthology.orchestra.run.smithi018.stdout:qdb_cluster leader: 0 members: 2024-04-12T21:51:40.756 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-12T21:51:40.756 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-12T21:51:40.756 INFO:teuthology.orchestra.run.smithi018.stdout:Standby daemons: 2024-04-12T21:51:40.756 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-12T21:51:40.756 INFO:teuthology.orchestra.run.smithi018.stdout:[mds.cephfs.smithi018.acqbsc{-1:34436} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.18:6828/755179764,v1:172.21.15.18:6829/755179764] compat {c=[1],r=[1],i=[1fff]}] 2024-04-12T21:51:40.756 INFO:teuthology.orchestra.run.smithi018.stdout:[mds.cephfs.smithi018.szjluo{-1:34468} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.18:6826/2407992653,v1:172.21.15.18:6827/2407992653] compat {c=[1],r=[1],i=[1fff]}] 2024-04-12T21:51:40.756 INFO:teuthology.orchestra.run.smithi018.stdout:[mds.cephfs.smithi186.jdllyn{-1:44595} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.186:6824/3523444202,v1:172.21.15.186:6825/3523444202] compat {c=[1],r=[1],i=[1fff]}] 2024-04-12T21:51:40.756 INFO:teuthology.orchestra.run.smithi018.stdout:[mds.cephfs.smithi186.wjnafs{-1:44875} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.186:6826/3122443766,v1:172.21.15.186:6827/3122443766] compat {c=[1],r=[1],i=[1fff]}] 2024-04-12T21:51:40.757 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:40.755+0000 7f9d3f400700 1 -- 172.21.15.18:0/1576464768 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f9d2006ec00 msgr2=0x7f9d200710b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:51:40.757 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:40.755+0000 7f9d3f400700 1 --2- 172.21.15.18:0/1576464768 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f9d2006ec00 0x7f9d200710b0 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7f9d2c004f90 tx=0x7f9d2c009e30 comp rx=0 tx=0).stop 2024-04-12T21:51:40.757 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:40.755+0000 7f9d3f400700 1 -- 172.21.15.18:0/1576464768 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9d381026c0 msgr2=0x7f9d38116090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:51:40.757 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:40.755+0000 7f9d3f400700 1 --2- 172.21.15.18:0/1576464768 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9d381026c0 0x7f9d38116090 secure :-1 s=READY pgs=1319 cs=0 l=1 rev1=1 crypto rx=0x7f9d2400b790 tx=0x7f9d24009220 comp rx=0 tx=0).stop 2024-04-12T21:51:40.757 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:40.755+0000 7f9d3f400700 1 -- 172.21.15.18:0/1576464768 shutdown_connections 2024-04-12T21:51:40.758 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:40.755+0000 7f9d3f400700 1 --2- 172.21.15.18:0/1576464768 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f9d2006ec00 0x7f9d200710b0 unknown :-1 s=CLOSED pgs=698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:51:40.758 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:40.755+0000 7f9d3f400700 1 --2- 172.21.15.18:0/1576464768 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9d38101d20 0x7f9d38115b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:51:40.758 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:40.755+0000 7f9d3f400700 1 --2- 172.21.15.18:0/1576464768 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9d381026c0 0x7f9d38116090 unknown :-1 s=CLOSED pgs=1319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:51:40.758 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:40.755+0000 7f9d3f400700 1 -- 172.21.15.18:0/1576464768 >> 172.21.15.18:0/1576464768 conn(0x7f9d380f7870 msgr2=0x7f9d38066af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:51:40.758 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:40.755+0000 7f9d3f400700 1 -- 172.21.15.18:0/1576464768 shutdown_connections 2024-04-12T21:51:40.758 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:40.756+0000 7f9d3f400700 1 -- 172.21.15.18:0/1576464768 wait complete. 2024-04-12T21:51:40.758 INFO:teuthology.orchestra.run.smithi018.stderr:dumped fsmap epoch 33 2024-04-12T21:51:40.922 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:40.920+0000 7f5cb9600700 1 -- 172.21.15.18:0/3198960306 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5cb4109dd0 msgr2=0x7f5cb4103440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:51:40.922 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:40.920+0000 7f5cb9600700 1 --2- 172.21.15.18:0/3198960306 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5cb4109dd0 0x7f5cb4103440 secure :-1 s=READY pgs=1320 cs=0 l=1 rev1=1 crypto rx=0x7f5ca0004940 tx=0x7f5ca0009e30 comp rx=0 tx=0).stop 2024-04-12T21:51:40.922 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:40.921+0000 7f5cb9600700 1 -- 172.21.15.18:0/3198960306 shutdown_connections 2024-04-12T21:51:40.922 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:40.921+0000 7f5cb9600700 1 --2- 172.21.15.18:0/3198960306 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5cb4109dd0 0x7f5cb4103440 unknown :-1 s=CLOSED pgs=1320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:51:40.922 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:40.921+0000 7f5cb9600700 1 --2- 172.21.15.18:0/3198960306 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5cb4109430 0x7f5cb4109800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:51:40.922 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:40.921+0000 7f5cb9600700 1 -- 172.21.15.18:0/3198960306 >> 172.21.15.18:0/3198960306 conn(0x7f5cb4072c00 msgr2=0x7f5cb4075010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:51:40.922 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:40.921+0000 7f5cb9600700 1 -- 172.21.15.18:0/3198960306 shutdown_connections 2024-04-12T21:51:40.923 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:40.921+0000 7f5cb9600700 1 -- 172.21.15.18:0/3198960306 wait complete. 2024-04-12T21:51:40.923 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:40.922+0000 7f5cb9600700 1 Processor -- start 2024-04-12T21:51:40.923 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:40.922+0000 7f5cb9600700 1 -- start start 2024-04-12T21:51:40.923 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:40.923+0000 7f5cb9600700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5cb4109430 0x7f5cb4119f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:51:40.923 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:40.923+0000 7f5cb9600700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5cb4109dd0 0x7f5cb411a4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:51:40.924 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:40.923+0000 7f5cb9600700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5cb411ab10 con 0x7f5cb4109430 2024-04-12T21:51:40.924 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:40.923+0000 7f5cb9600700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5cb41140f0 con 0x7f5cb4109dd0 2024-04-12T21:51:40.925 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:40.923+0000 7f5cb2a00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5cb4109430 0x7f5cb4119f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:51:40.925 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:40.923+0000 7f5cb2a00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5cb4109430 0x7f5cb4119f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:40846/0 (socket says 172.21.15.18:40846) 2024-04-12T21:51:40.925 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:40.923+0000 7f5cb2a00700 1 -- 172.21.15.18:0/3086870572 learned_addr learned my addr 172.21.15.18:0/3086870572 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:51:40.925 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:40.923+0000 7f5cb2000700 1 --2- 172.21.15.18:0/3086870572 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5cb4109dd0 0x7f5cb411a4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:51:40.925 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:40.923+0000 7f5cb2a00700 1 -- 172.21.15.18:0/3086870572 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5cb4109dd0 msgr2=0x7f5cb411a4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:51:40.925 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:40.923+0000 7f5cb2a00700 1 --2- 172.21.15.18:0/3086870572 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5cb4109dd0 0x7f5cb411a4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:51:40.925 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:40.923+0000 7f5cb2a00700 1 -- 172.21.15.18:0/3086870572 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ca0005040 con 0x7f5cb4109430 2024-04-12T21:51:40.925 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:40.923+0000 7f5cb2000700 1 --2- 172.21.15.18:0/3086870572 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5cb4109dd0 0x7f5cb411a4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-12T21:51:40.926 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:40.924+0000 7f5cb2a00700 1 --2- 172.21.15.18:0/3086870572 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5cb4109430 0x7f5cb4119f80 secure :-1 s=READY pgs=1321 cs=0 l=1 rev1=1 crypto rx=0x7f5ca4002a50 tx=0x7f5ca40046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:51:40.926 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:40.924+0000 7f5caaa00700 1 -- 172.21.15.18:0/3086870572 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5ca4007cc0 con 0x7f5cb4109430 2024-04-12T21:51:40.926 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:40.924+0000 7f5caaa00700 1 -- 172.21.15.18:0/3086870572 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f5ca400f040 con 0x7f5cb4109430 2024-04-12T21:51:40.926 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:40.924+0000 7f5cb9600700 1 -- 172.21.15.18:0/3086870572 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5cb41143d0 con 0x7f5cb4109430 2024-04-12T21:51:40.926 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:40.924+0000 7f5caaa00700 1 -- 172.21.15.18:0/3086870572 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5ca4009640 con 0x7f5cb4109430 2024-04-12T21:51:40.926 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:40.924+0000 7f5cb9600700 1 -- 172.21.15.18:0/3086870572 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5cb4114990 con 0x7f5cb4109430 2024-04-12T21:51:40.926 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:40.925+0000 7f5cb9600700 1 -- 172.21.15.18:0/3086870572 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5cb4104b80 con 0x7f5cb4109430 2024-04-12T21:51:40.927 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:40.926+0000 7f5caaa00700 1 -- 172.21.15.18:0/3086870572 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7f5ca40097a0 con 0x7f5cb4109430 2024-04-12T21:51:40.928 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:40.927+0000 7f5caaa00700 1 --2- 172.21.15.18:0/3086870572 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f5c9806ec00 0x7f5c980710b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:51:40.928 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:40.927+0000 7f5caaa00700 1 -- 172.21.15.18:0/3086870572 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 6757+0+0 (secure 0 0 0) 0x7f5ca408e6c0 con 0x7f5cb4109430 2024-04-12T21:51:40.929 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:40.928+0000 7f5cb2000700 1 --2- 172.21.15.18:0/3086870572 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f5c9806ec00 0x7f5c980710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:51:40.929 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:40.928+0000 7f5cb2000700 1 --2- 172.21.15.18:0/3086870572 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f5c9806ec00 0x7f5c980710b0 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7f5ca0009fd0 tx=0x7f5ca0009e30 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:51:40.934 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:40.933+0000 7f5caaa00700 1 -- 172.21.15.18:0/3086870572 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f5ca405af10 con 0x7f5cb4109430 2024-04-12T21:51:41.034 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:51:40 smithi186 ceph-mon[86379]: from='client.36696 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:51:41.035 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:51:40 smithi186 ceph-mon[86379]: from='client.36700 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:51:41.035 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:51:40 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T21:51:41.035 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:51:40 smithi186 ceph-mon[86379]: from='client.? 172.21.15.18:0/667831685' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T21:51:41.105 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:51:40 smithi018 ceph-mon[112590]: from='client.36696 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:51:41.106 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:51:40 smithi018 ceph-mon[112590]: from='client.36700 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:51:41.106 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:51:40 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T21:51:41.106 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:51:40 smithi018 ceph-mon[112590]: from='client.? 172.21.15.18:0/667831685' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T21:51:41.192 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:41.190+0000 7f5cb9600700 1 -- 172.21.15.18:0/3086870572 --> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f5cb4115670 con 0x7f5c9806ec00 2024-04-12T21:51:41.194 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:41.193+0000 7f5caaa00700 1 -- 172.21.15.18:0/3086870572 <== mgr.34102 v2:172.21.15.18:6800/3812891919 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f5cb4115670 con 0x7f5c9806ec00 2024-04-12T21:51:41.194 INFO:teuthology.orchestra.run.smithi018.stdout:{ 2024-04-12T21:51:41.194 INFO:teuthology.orchestra.run.smithi018.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:f4452def3be4d350f4a8e5feaf7ebb1511be9298fe68478ba7a2d6fbd7d69a63", 2024-04-12T21:51:41.194 INFO:teuthology.orchestra.run.smithi018.stdout: "in_progress": true, 2024-04-12T21:51:41.194 INFO:teuthology.orchestra.run.smithi018.stdout: "which": "Upgrading all daemon types on all hosts", 2024-04-12T21:51:41.194 INFO:teuthology.orchestra.run.smithi018.stdout: "services_complete": [ 2024-04-12T21:51:41.194 INFO:teuthology.orchestra.run.smithi018.stdout: "mgr", 2024-04-12T21:51:41.194 INFO:teuthology.orchestra.run.smithi018.stdout: "mds", 2024-04-12T21:51:41.194 INFO:teuthology.orchestra.run.smithi018.stdout: "crash", 2024-04-12T21:51:41.195 INFO:teuthology.orchestra.run.smithi018.stdout: "osd", 2024-04-12T21:51:41.195 INFO:teuthology.orchestra.run.smithi018.stdout: "mon" 2024-04-12T21:51:41.195 INFO:teuthology.orchestra.run.smithi018.stdout: ], 2024-04-12T21:51:41.195 INFO:teuthology.orchestra.run.smithi018.stdout: "progress": "16/21 daemons upgraded", 2024-04-12T21:51:41.195 INFO:teuthology.orchestra.run.smithi018.stdout: "message": "Currently upgrading mds daemons", 2024-04-12T21:51:41.195 INFO:teuthology.orchestra.run.smithi018.stdout: "is_paused": false 2024-04-12T21:51:41.195 INFO:teuthology.orchestra.run.smithi018.stdout:} 2024-04-12T21:51:41.197 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:41.196+0000 7f5cb9600700 1 -- 172.21.15.18:0/3086870572 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f5c9806ec00 msgr2=0x7f5c980710b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:51:41.198 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:41.196+0000 7f5cb9600700 1 --2- 172.21.15.18:0/3086870572 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f5c9806ec00 0x7f5c980710b0 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7f5ca0009fd0 tx=0x7f5ca0009e30 comp rx=0 tx=0).stop 2024-04-12T21:51:41.198 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:41.196+0000 7f5cb9600700 1 -- 172.21.15.18:0/3086870572 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5cb4109430 msgr2=0x7f5cb4119f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:51:41.198 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:41.196+0000 7f5cb9600700 1 --2- 172.21.15.18:0/3086870572 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5cb4109430 0x7f5cb4119f80 secure :-1 s=READY pgs=1321 cs=0 l=1 rev1=1 crypto rx=0x7f5ca4002a50 tx=0x7f5ca40046b0 comp rx=0 tx=0).stop 2024-04-12T21:51:41.198 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:41.196+0000 7f5cb9600700 1 -- 172.21.15.18:0/3086870572 shutdown_connections 2024-04-12T21:51:41.198 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:41.196+0000 7f5cb9600700 1 --2- 172.21.15.18:0/3086870572 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f5c9806ec00 0x7f5c980710b0 unknown :-1 s=CLOSED pgs=699 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:51:41.198 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:41.196+0000 7f5cb9600700 1 --2- 172.21.15.18:0/3086870572 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5cb4109430 0x7f5cb4119f80 unknown :-1 s=CLOSED pgs=1321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:51:41.198 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:41.196+0000 7f5cb9600700 1 --2- 172.21.15.18:0/3086870572 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5cb4109dd0 0x7f5cb411a4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:51:41.198 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:41.196+0000 7f5cb9600700 1 -- 172.21.15.18:0/3086870572 >> 172.21.15.18:0/3086870572 conn(0x7f5cb4072c00 msgr2=0x7f5cb4074c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:51:41.198 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:41.196+0000 7f5cb9600700 1 -- 172.21.15.18:0/3086870572 shutdown_connections 2024-04-12T21:51:41.198 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:41.197+0000 7f5cb9600700 1 -- 172.21.15.18:0/3086870572 wait complete. 2024-04-12T21:51:41.362 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:41.361+0000 7fccfe000700 1 -- 172.21.15.18:0/2811268396 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fccf8102c30 msgr2=0x7fccf8103000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:51:41.363 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:41.361+0000 7fccfe000700 1 --2- 172.21.15.18:0/2811268396 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fccf8102c30 0x7fccf8103000 secure :-1 s=READY pgs=1322 cs=0 l=1 rev1=1 crypto rx=0x7fccec004940 tx=0x7fccec009e30 comp rx=0 tx=0).stop 2024-04-12T21:51:41.363 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:41.361+0000 7fccfe000700 1 -- 172.21.15.18:0/2811268396 shutdown_connections 2024-04-12T21:51:41.363 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:41.361+0000 7fccfe000700 1 --2- 172.21.15.18:0/2811268396 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fccf81035d0 0x7fccf80fcc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:51:41.363 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:41.361+0000 7fccfe000700 1 --2- 172.21.15.18:0/2811268396 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fccf8102c30 0x7fccf8103000 unknown :-1 s=CLOSED pgs=1322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:51:41.363 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:41.361+0000 7fccfe000700 1 -- 172.21.15.18:0/2811268396 >> 172.21.15.18:0/2811268396 conn(0x7fccf80f87a0 msgr2=0x7fccf80fabb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:51:41.363 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:41.361+0000 7fccfe000700 1 -- 172.21.15.18:0/2811268396 shutdown_connections 2024-04-12T21:51:41.363 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:41.361+0000 7fccfe000700 1 -- 172.21.15.18:0/2811268396 wait complete. 2024-04-12T21:51:41.364 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:41.362+0000 7fccfe000700 1 Processor -- start 2024-04-12T21:51:41.364 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:41.362+0000 7fccfe000700 1 -- start start 2024-04-12T21:51:41.364 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:41.363+0000 7fccfe000700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fccf8102c30 0x7fccf8115b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:51:41.364 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:41.363+0000 7fccfe000700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fccf81035d0 0x7fccf81160d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:51:41.364 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:41.363+0000 7fccfe000700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fccf8116720 con 0x7fccf81035d0 2024-04-12T21:51:41.365 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:41.363+0000 7fccfe000700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fccf810fb30 con 0x7fccf8102c30 2024-04-12T21:51:41.365 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:41.363+0000 7fccf7e00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fccf81035d0 0x7fccf81160d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:51:41.365 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:41.363+0000 7fccf7e00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fccf81035d0 0x7fccf81160d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:40868/0 (socket says 172.21.15.18:40868) 2024-04-12T21:51:41.365 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:41.363+0000 7fccf7e00700 1 -- 172.21.15.18:0/1867190426 learned_addr learned my addr 172.21.15.18:0/1867190426 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:51:41.365 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:41.363+0000 7fccfcc00700 1 --2- 172.21.15.18:0/1867190426 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fccf8102c30 0x7fccf8115b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:51:41.366 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:41.364+0000 7fccf7e00700 1 -- 172.21.15.18:0/1867190426 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fccf8102c30 msgr2=0x7fccf8115b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:51:41.366 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:41.364+0000 7fccf7e00700 1 --2- 172.21.15.18:0/1867190426 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fccf8102c30 0x7fccf8115b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:51:41.366 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:41.364+0000 7fccf7e00700 1 -- 172.21.15.18:0/1867190426 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fccec005040 con 0x7fccf81035d0 2024-04-12T21:51:41.366 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:41.364+0000 7fccfcc00700 1 --2- 172.21.15.18:0/1867190426 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fccf8102c30 0x7fccf8115b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-12T21:51:41.366 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:41.364+0000 7fccf7e00700 1 --2- 172.21.15.18:0/1867190426 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fccf81035d0 0x7fccf81160d0 secure :-1 s=READY pgs=1323 cs=0 l=1 rev1=1 crypto rx=0x7fcce400c890 tx=0x7fcce40044c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:51:41.366 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:41.364+0000 7fccf4c00700 1 -- 172.21.15.18:0/1867190426 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcce400a7d0 con 0x7fccf81035d0 2024-04-12T21:51:41.366 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:41.365+0000 7fccf4c00700 1 -- 172.21.15.18:0/1867190426 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fcce400ae00 con 0x7fccf81035d0 2024-04-12T21:51:41.366 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:41.365+0000 7fccfe000700 1 -- 172.21.15.18:0/1867190426 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fccf810fe10 con 0x7fccf81035d0 2024-04-12T21:51:41.367 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:41.365+0000 7fccf4c00700 1 -- 172.21.15.18:0/1867190426 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcce40113a0 con 0x7fccf81035d0 2024-04-12T21:51:41.367 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:41.365+0000 7fccfe000700 1 -- 172.21.15.18:0/1867190426 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fccf81103d0 con 0x7fccf81035d0 2024-04-12T21:51:41.367 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:41.366+0000 7fccfe000700 1 -- 172.21.15.18:0/1867190426 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fccf80fe380 con 0x7fccf81035d0 2024-04-12T21:51:41.368 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:41.367+0000 7fccf4c00700 1 -- 172.21.15.18:0/1867190426 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7fcce4012400 con 0x7fccf81035d0 2024-04-12T21:51:41.369 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:41.368+0000 7fccf4c00700 1 --2- 172.21.15.18:0/1867190426 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fcce006ec10 0x7fcce00710c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:51:41.369 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:41.368+0000 7fccf4c00700 1 -- 172.21.15.18:0/1867190426 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 6757+0+0 (secure 0 0 0) 0x7fcce408eae0 con 0x7fccf81035d0 2024-04-12T21:51:41.370 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:41.369+0000 7fccfcc00700 1 --2- 172.21.15.18:0/1867190426 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fcce006ec10 0x7fcce00710c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:51:41.370 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:41.369+0000 7fccfcc00700 1 --2- 172.21.15.18:0/1867190426 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fcce006ec10 0x7fcce00710c0 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7fccec00b3b0 tx=0x7fccec009e30 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:51:41.374 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:41.373+0000 7fccf4c00700 1 -- 172.21.15.18:0/1867190426 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7fcce4057760 con 0x7fccf81035d0 2024-04-12T21:51:41.704 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:41.703+0000 7fccfe000700 1 -- 172.21.15.18:0/1867190426 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7fccf8109d20 con 0x7fccf81035d0 2024-04-12T21:51:41.705 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:41.704+0000 7fccf4c00700 1 -- 172.21.15.18:0/1867190426 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+4303 (secure 0 0 0) 0x7fcce405ae20 con 0x7fccf81035d0 2024-04-12T21:51:41.705 INFO:teuthology.orchestra.run.smithi018.stdout:HEALTH_ERR failed to probe daemons or devices; 1 filesystem is degraded; 1 filesystem is offline 2024-04-12T21:51:41.705 INFO:teuthology.orchestra.run.smithi018.stdout:[WRN] CEPHADM_REFRESH_FAILED: failed to probe daemons or devices 2024-04-12T21:51:41.706 INFO:teuthology.orchestra.run.smithi018.stdout: host smithi186 `cephadm ceph-volume` failed: cephadm exited with an error code: 1, stderr: Inferring config /var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/mon.smithi186/config 2024-04-12T21:51:41.706 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/podman: timeout after 895 seconds 2024-04-12T21:51:41.706 INFO:teuthology.orchestra.run.smithi018.stdout:Non-zero exit code 124 from /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:f4452def3be4d350f4a8e5feaf7ebb1511be9298fe68478ba7a2d6fbd7d69a63 -e NODE_NAME=smithi186 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968:/var/run/ceph:z -v /var/log/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968:/var/log/ceph:z -v /var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmptgom53f9:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:f4452def3be4d350f4a8e5feaf7ebb1511be9298fe68478ba7a2d6fbd7d69a63 inventory --format=json-pretty --filter-for-batch 2024-04-12T21:51:41.706 INFO:teuthology.orchestra.run.smithi018.stdout:Traceback (most recent call last): 2024-04-12T21:51:41.706 INFO:teuthology.orchestra.run.smithi018.stdout: File "/usr/lib64/python3.9/runpy.py", line 197, in _run_module_as_main 2024-04-12T21:51:41.706 INFO:teuthology.orchestra.run.smithi018.stdout: return _run_code(code, main_globals, None, 2024-04-12T21:51:41.706 INFO:teuthology.orchestra.run.smithi018.stdout: File "/usr/lib64/python3.9/runpy.py", line 87, in _run_code 2024-04-12T21:51:41.706 INFO:teuthology.orchestra.run.smithi018.stdout: exec(code, run_globals) 2024-04-12T21:51:41.706 INFO:teuthology.orchestra.run.smithi018.stdout: File "/var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/cephadm.e44f3ed5e34e4ef52eedef233d1190bff004470d8e8005127b5312bff357fa7f/__main__.py", line 5618, in 2024-04-12T21:51:41.706 INFO:teuthology.orchestra.run.smithi018.stdout: File "/var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/cephadm.e44f3ed5e34e4ef52eedef233d1190bff004470d8e8005127b5312bff357fa7f/__main__.py", line 5606, in main 2024-04-12T21:51:41.706 INFO:teuthology.orchestra.run.smithi018.stdout: File "/var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/cephadm.e44f3ed5e34e4ef52eedef233d1190bff004470d8e8005127b5312bff357fa7f/__main__.py", line 408, in _infer_config 2024-04-12T21:51:41.706 INFO:teuthology.orchestra.run.smithi018.stdout: File "/var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/cephadm.e44f3ed5e34e4ef52eedef233d1190bff004470d8e8005127b5312bff357fa7f/__main__.py", line 323, in _infer_fsid 2024-04-12T21:51:41.706 INFO:teuthology.orchestra.run.smithi018.stdout: File "/var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/cephadm.e44f3ed5e34e4ef52eedef233d1190bff004470d8e8005127b5312bff357fa7f/__main__.py", line 436, in _infer_image 2024-04-12T21:51:41.706 INFO:teuthology.orchestra.run.smithi018.stdout: File "/var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/cephadm.e44f3ed5e34e4ef52eedef233d1190bff004470d8e8005127b5312bff357fa7f/__main__.py", line 310, in _validate_fsid 2024-04-12T21:51:41.707 INFO:teuthology.orchestra.run.smithi018.stdout: File "/var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/cephadm.e44f3ed5e34e4ef52eedef233d1190bff004470d8e8005127b5312bff357fa7f/__main__.py", line 3359, in command_ceph_volume 2024-04-12T21:51:41.707 INFO:teuthology.orchestra.run.smithi018.stdout: File "/var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/cephadm.e44f3ed5e34e4ef52eedef233d1190bff004470d8e8005127b5312bff357fa7f/cephadmlib/call_wrappers.py", line 310, in call_throws 2024-04-12T21:51:41.707 INFO:teuthology.orchestra.run.smithi018.stdout:RuntimeError: Failed command: /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:f4452def3be4d350f4a8e5feaf7ebb1511be9298fe68478ba7a2d6fbd7d69a63 -e NODE_NAME=smithi186 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968:/var/run/ceph:z -v /var/log/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968:/var/log/ceph:z -v /var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmptgom53f9:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:f4452def3be4d350f4a8e5feaf7ebb1511be9298fe68478ba7a2d6fbd7d69a63 inventory --format=json-pretty --filter-for-batch 2024-04-12T21:51:41.707 INFO:teuthology.orchestra.run.smithi018.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-04-12T21:51:41.707 INFO:teuthology.orchestra.run.smithi018.stdout: fs cephfs is degraded 2024-04-12T21:51:41.707 INFO:teuthology.orchestra.run.smithi018.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-04-12T21:51:41.707 INFO:teuthology.orchestra.run.smithi018.stdout: fs cephfs is offline because no MDS is active for it. 2024-04-12T21:51:41.708 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:41.707+0000 7fccfe000700 1 -- 172.21.15.18:0/1867190426 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fcce006ec10 msgr2=0x7fcce00710c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:51:41.708 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:41.707+0000 7fccfe000700 1 --2- 172.21.15.18:0/1867190426 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fcce006ec10 0x7fcce00710c0 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7fccec00b3b0 tx=0x7fccec009e30 comp rx=0 tx=0).stop 2024-04-12T21:51:41.709 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:41.707+0000 7fccfe000700 1 -- 172.21.15.18:0/1867190426 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fccf81035d0 msgr2=0x7fccf81160d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:51:41.709 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:41.707+0000 7fccfe000700 1 --2- 172.21.15.18:0/1867190426 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fccf81035d0 0x7fccf81160d0 secure :-1 s=READY pgs=1323 cs=0 l=1 rev1=1 crypto rx=0x7fcce400c890 tx=0x7fcce40044c0 comp rx=0 tx=0).stop 2024-04-12T21:51:41.709 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:41.707+0000 7fccfe000700 1 -- 172.21.15.18:0/1867190426 shutdown_connections 2024-04-12T21:51:41.709 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:41.707+0000 7fccfe000700 1 --2- 172.21.15.18:0/1867190426 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fcce006ec10 0x7fcce00710c0 unknown :-1 s=CLOSED pgs=700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:51:41.709 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:41.707+0000 7fccfe000700 1 --2- 172.21.15.18:0/1867190426 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fccf8102c30 0x7fccf8115b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:51:41.709 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:41.707+0000 7fccfe000700 1 --2- 172.21.15.18:0/1867190426 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fccf81035d0 0x7fccf81160d0 unknown :-1 s=CLOSED pgs=1323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:51:41.709 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:41.707+0000 7fccfe000700 1 -- 172.21.15.18:0/1867190426 >> 172.21.15.18:0/1867190426 conn(0x7fccf80f87a0 msgr2=0x7fccf80f9690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:51:41.709 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:41.708+0000 7fccfe000700 1 -- 172.21.15.18:0/1867190426 shutdown_connections 2024-04-12T21:51:41.709 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:51:41.708+0000 7fccfe000700 1 -- 172.21.15.18:0/1867190426 wait complete. 2024-04-12T21:51:42.034 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:51:41 smithi186 ceph-mon[86379]: pgmap v1560: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:51:42.034 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:51:41 smithi186 ceph-mon[86379]: from='client.? 172.21.15.18:0/1576464768' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-12T21:51:42.035 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:51:41 smithi186 ceph-mon[86379]: from='client.? 172.21.15.18:0/1867190426' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-12T21:51:42.105 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:51:41 smithi018 ceph-mon[112590]: pgmap v1560: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:51:42.106 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:51:41 smithi018 ceph-mon[112590]: from='client.? 172.21.15.18:0/1576464768' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-12T21:51:42.106 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:51:41 smithi018 ceph-mon[112590]: from='client.? 172.21.15.18:0/1867190426' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-12T21:51:43.034 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:51:42 smithi186 ceph-mon[86379]: from='client.36712 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:51:43.105 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:51:42 smithi018 ceph-mon[112590]: from='client.36712 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:51:44.034 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:51:43 smithi186 ceph-mon[86379]: pgmap v1561: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:51:44.105 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:51:43 smithi018 ceph-mon[112590]: pgmap v1561: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:51:46.034 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:51:45 smithi186 ceph-mon[86379]: pgmap v1562: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:51:46.105 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:51:45 smithi018 ceph-mon[112590]: pgmap v1562: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:51:48.034 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:51:47 smithi186 ceph-mon[86379]: pgmap v1563: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:51:48.105 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:51:47 smithi018 ceph-mon[112590]: pgmap v1563: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:51:50.034 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:51:49 smithi186 ceph-mon[86379]: pgmap v1564: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:51:50.105 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:51:49 smithi018 ceph-mon[112590]: pgmap v1564: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:51:52.034 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:51:51 smithi186 ceph-mon[86379]: pgmap v1565: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:51:52.105 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:51:51 smithi018 ceph-mon[112590]: pgmap v1565: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:51:54.034 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:51:53 smithi186 ceph-mon[86379]: pgmap v1566: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:51:54.105 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:51:53 smithi018 ceph-mon[112590]: pgmap v1566: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:51:56.034 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:51:55 smithi186 ceph-mon[86379]: pgmap v1567: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:51:56.035 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:51:55 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T21:51:56.105 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:51:55 smithi018 ceph-mon[112590]: pgmap v1567: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:51:56.106 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:51:55 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T21:51:58.034 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:51:57 smithi186 ceph-mon[86379]: pgmap v1568: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:51:58.105 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:51:57 smithi018 ceph-mon[112590]: pgmap v1568: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:52:00.034 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:51:59 smithi186 ceph-mon[86379]: pgmap v1569: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:52:00.105 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:51:59 smithi018 ceph-mon[112590]: pgmap v1569: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:52:02.035 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:52:01 smithi186 ceph-mon[86379]: pgmap v1570: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:52:02.105 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:52:01 smithi018 ceph-mon[112590]: pgmap v1570: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:52:04.105 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:52:03 smithi018 ceph-mon[112590]: pgmap v1571: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:52:04.284 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:52:03 smithi186 ceph-mon[86379]: pgmap v1571: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:52:06.105 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:52:05 smithi018 ceph-mon[112590]: pgmap v1572: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:52:06.284 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:52:05 smithi186 ceph-mon[86379]: pgmap v1572: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:52:08.105 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:52:07 smithi018 ceph-mon[112590]: pgmap v1573: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:52:08.284 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:52:07 smithi186 ceph-mon[86379]: pgmap v1573: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:52:10.105 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:52:09 smithi018 ceph-mon[112590]: pgmap v1574: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:52:10.284 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:52:09 smithi186 ceph-mon[86379]: pgmap v1574: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:52:11.105 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:52:10 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T21:52:11.284 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:52:10 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T21:52:11.884 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:11.882+0000 7fa6d9e00700 1 -- 172.21.15.18:0/628768649 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa6d41037d0 msgr2=0x7fa6d40fb620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:52:11.884 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:11.882+0000 7fa6d9e00700 1 --2- 172.21.15.18:0/628768649 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa6d41037d0 0x7fa6d40fb620 secure :-1 s=READY pgs=1324 cs=0 l=1 rev1=1 crypto rx=0x7fa6c0004940 tx=0x7fa6c0009e30 comp rx=0 tx=0).stop 2024-04-12T21:52:11.885 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:11.883+0000 7fa6d9e00700 1 -- 172.21.15.18:0/628768649 shutdown_connections 2024-04-12T21:52:11.885 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:11.883+0000 7fa6d9e00700 1 --2- 172.21.15.18:0/628768649 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa6d41037d0 0x7fa6d40fb620 unknown :-1 s=CLOSED pgs=1324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:52:11.885 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:11.883+0000 7fa6d9e00700 1 --2- 172.21.15.18:0/628768649 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa6d4102e30 0x7fa6d4103200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:52:11.885 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:11.883+0000 7fa6d9e00700 1 -- 172.21.15.18:0/628768649 >> 172.21.15.18:0/628768649 conn(0x7fa6d40714c0 msgr2=0x7fa6d40718c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:52:11.885 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:11.883+0000 7fa6d9e00700 1 -- 172.21.15.18:0/628768649 shutdown_connections 2024-04-12T21:52:11.885 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:11.883+0000 7fa6d9e00700 1 -- 172.21.15.18:0/628768649 wait complete. 2024-04-12T21:52:11.886 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:11.884+0000 7fa6d9e00700 1 Processor -- start 2024-04-12T21:52:11.887 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:11.884+0000 7fa6d9e00700 1 -- start start 2024-04-12T21:52:11.887 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:11.885+0000 7fa6d9e00700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa6d4102e30 0x7fa6d4115b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:52:11.887 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:11.885+0000 7fa6d9e00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa6d41037d0 0x7fa6d41160c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:52:11.887 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:11.885+0000 7fa6d9e00700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6d4116710 con 0x7fa6d41037d0 2024-04-12T21:52:11.887 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:11.885+0000 7fa6d9e00700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6d410fcf0 con 0x7fa6d4102e30 2024-04-12T21:52:11.887 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:11.885+0000 7fa6d2000700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa6d41037d0 0x7fa6d41160c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:52:11.887 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:11.885+0000 7fa6d2000700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa6d41037d0 0x7fa6d41160c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:52518/0 (socket says 172.21.15.18:52518) 2024-04-12T21:52:11.887 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:11.885+0000 7fa6d2000700 1 -- 172.21.15.18:0/2367751301 learned_addr learned my addr 172.21.15.18:0/2367751301 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:52:11.887 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:11.885+0000 7fa6d2a00700 1 --2- 172.21.15.18:0/2367751301 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa6d4102e30 0x7fa6d4115b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:52:11.887 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:11.885+0000 7fa6d2000700 1 -- 172.21.15.18:0/2367751301 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa6d4102e30 msgr2=0x7fa6d4115b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:52:11.887 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:11.885+0000 7fa6d2000700 1 --2- 172.21.15.18:0/2367751301 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa6d4102e30 0x7fa6d4115b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:52:11.887 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:11.885+0000 7fa6d2000700 1 -- 172.21.15.18:0/2367751301 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6c0005040 con 0x7fa6d41037d0 2024-04-12T21:52:11.887 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:11.885+0000 7fa6d2a00700 1 --2- 172.21.15.18:0/2367751301 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa6d4102e30 0x7fa6d4115b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-12T21:52:11.888 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:11.886+0000 7fa6d2000700 1 --2- 172.21.15.18:0/2367751301 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa6d41037d0 0x7fa6d41160c0 secure :-1 s=READY pgs=1325 cs=0 l=1 rev1=1 crypto rx=0x7fa6c0009100 tx=0x7fa6c0009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:52:11.888 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:11.886+0000 7fa6caa00700 1 -- 172.21.15.18:0/2367751301 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa6c000e040 con 0x7fa6d41037d0 2024-04-12T21:52:11.888 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:11.886+0000 7fa6caa00700 1 -- 172.21.15.18:0/2367751301 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa6c0013420 con 0x7fa6d41037d0 2024-04-12T21:52:11.889 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:11.886+0000 7fa6d9e00700 1 -- 172.21.15.18:0/2367751301 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa6d410ff70 con 0x7fa6d41037d0 2024-04-12T21:52:11.889 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:11.886+0000 7fa6caa00700 1 -- 172.21.15.18:0/2367751301 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa6c001c630 con 0x7fa6d41037d0 2024-04-12T21:52:11.889 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:11.886+0000 7fa6d9e00700 1 -- 172.21.15.18:0/2367751301 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa6d4110530 con 0x7fa6d41037d0 2024-04-12T21:52:11.889 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:11.888+0000 7fa6d9e00700 1 -- 172.21.15.18:0/2367751301 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa6d4102150 con 0x7fa6d41037d0 2024-04-12T21:52:11.889 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:11.889+0000 7fa6caa00700 1 -- 172.21.15.18:0/2367751301 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7fa6c0013590 con 0x7fa6d41037d0 2024-04-12T21:52:11.890 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:11.890+0000 7fa6caa00700 1 --2- 172.21.15.18:0/2367751301 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fa6b806ec00 0x7fa6b80710b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:52:11.891 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:11.890+0000 7fa6caa00700 1 -- 172.21.15.18:0/2367751301 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 6757+0+0 (secure 0 0 0) 0x7fa6c008fbc0 con 0x7fa6d41037d0 2024-04-12T21:52:11.891 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:11.890+0000 7fa6d2a00700 1 --2- 172.21.15.18:0/2367751301 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fa6b806ec00 0x7fa6b80710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:52:11.891 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:11.891+0000 7fa6d2a00700 1 --2- 172.21.15.18:0/2367751301 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fa6b806ec00 0x7fa6b80710b0 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7fa6c4009fd0 tx=0x7fa6c400b040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:52:11.896 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:11.895+0000 7fa6caa00700 1 -- 172.21.15.18:0/2367751301 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7fa6c005c410 con 0x7fa6d41037d0 2024-04-12T21:52:12.105 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:52:11 smithi018 ceph-mon[112590]: pgmap v1575: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:52:12.147 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:12.145+0000 7fa6d9e00700 1 -- 172.21.15.18:0/2367751301 --> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fa6d4111270 con 0x7fa6b806ec00 2024-04-12T21:52:12.148 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:12.147+0000 7fa6caa00700 1 -- 172.21.15.18:0/2367751301 <== mgr.34102 v2:172.21.15.18:6800/3812891919 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7fa6d4111270 con 0x7fa6b806ec00 2024-04-12T21:52:12.152 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:12.150+0000 7fa6d9e00700 1 -- 172.21.15.18:0/2367751301 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fa6b806ec00 msgr2=0x7fa6b80710b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:52:12.152 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:12.150+0000 7fa6d9e00700 1 --2- 172.21.15.18:0/2367751301 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fa6b806ec00 0x7fa6b80710b0 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7fa6c4009fd0 tx=0x7fa6c400b040 comp rx=0 tx=0).stop 2024-04-12T21:52:12.152 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:12.150+0000 7fa6d9e00700 1 -- 172.21.15.18:0/2367751301 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa6d41037d0 msgr2=0x7fa6d41160c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:52:12.152 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:12.150+0000 7fa6d9e00700 1 --2- 172.21.15.18:0/2367751301 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa6d41037d0 0x7fa6d41160c0 secure :-1 s=READY pgs=1325 cs=0 l=1 rev1=1 crypto rx=0x7fa6c0009100 tx=0x7fa6c0009e30 comp rx=0 tx=0).stop 2024-04-12T21:52:12.152 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:12.150+0000 7fa6d9e00700 1 -- 172.21.15.18:0/2367751301 shutdown_connections 2024-04-12T21:52:12.152 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:12.150+0000 7fa6d9e00700 1 --2- 172.21.15.18:0/2367751301 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fa6b806ec00 0x7fa6b80710b0 unknown :-1 s=CLOSED pgs=702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:52:12.152 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:12.150+0000 7fa6d9e00700 1 --2- 172.21.15.18:0/2367751301 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa6d4102e30 0x7fa6d4115b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:52:12.152 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:12.150+0000 7fa6d9e00700 1 --2- 172.21.15.18:0/2367751301 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa6d41037d0 0x7fa6d41160c0 unknown :-1 s=CLOSED pgs=1325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:52:12.152 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:12.150+0000 7fa6d9e00700 1 -- 172.21.15.18:0/2367751301 >> 172.21.15.18:0/2367751301 conn(0x7fa6d40714c0 msgr2=0x7fa6d40fa4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:52:12.152 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:12.151+0000 7fa6d9e00700 1 -- 172.21.15.18:0/2367751301 shutdown_connections 2024-04-12T21:52:12.152 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:12.151+0000 7fa6d9e00700 1 -- 172.21.15.18:0/2367751301 wait complete. 2024-04-12T21:52:12.168 INFO:teuthology.orchestra.run.smithi018.stdout:true 2024-04-12T21:52:12.284 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:52:11 smithi186 ceph-mon[86379]: pgmap v1575: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:52:12.308 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:12.307+0000 7f430e000700 1 -- 172.21.15.18:0/1691534448 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4308105020 msgr2=0x7f43081053f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:52:12.308 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:12.307+0000 7f430e000700 1 --2- 172.21.15.18:0/1691534448 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4308105020 0x7f43081053f0 secure :-1 s=READY pgs=1326 cs=0 l=1 rev1=1 crypto rx=0x7f42f8004940 tx=0x7f42f8009e30 comp rx=0 tx=0).stop 2024-04-12T21:52:12.308 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:12.307+0000 7f430e000700 1 -- 172.21.15.18:0/1691534448 shutdown_connections 2024-04-12T21:52:12.309 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:12.307+0000 7f430e000700 1 --2- 172.21.15.18:0/1691534448 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f43081059c0 0x7f43080ff080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:52:12.309 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:12.307+0000 7f430e000700 1 --2- 172.21.15.18:0/1691534448 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4308105020 0x7f43081053f0 unknown :-1 s=CLOSED pgs=1326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:52:12.309 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:12.307+0000 7f430e000700 1 -- 172.21.15.18:0/1691534448 >> 172.21.15.18:0/1691534448 conn(0x7f43080faba0 msgr2=0x7f43080fcfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:52:12.309 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:12.307+0000 7f430e000700 1 -- 172.21.15.18:0/1691534448 shutdown_connections 2024-04-12T21:52:12.309 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:12.307+0000 7f430e000700 1 -- 172.21.15.18:0/1691534448 wait complete. 2024-04-12T21:52:12.310 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:12.308+0000 7f430e000700 1 Processor -- start 2024-04-12T21:52:12.310 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:12.308+0000 7f430e000700 1 -- start start 2024-04-12T21:52:12.310 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:12.309+0000 7f430e000700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4308105020 0x7f4308115b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:52:12.310 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:12.309+0000 7f430e000700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f43081059c0 0x7f43081160d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:52:12.310 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:12.309+0000 7f430e000700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4308116720 con 0x7f4308105020 2024-04-12T21:52:12.310 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:12.309+0000 7f430e000700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f430810fd00 con 0x7f43081059c0 2024-04-12T21:52:12.311 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:12.309+0000 7f4307400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4308105020 0x7f4308115b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:52:12.311 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:12.309+0000 7f4307400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4308105020 0x7f4308115b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:52530/0 (socket says 172.21.15.18:52530) 2024-04-12T21:52:12.311 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:12.309+0000 7f4307400700 1 -- 172.21.15.18:0/538927883 learned_addr learned my addr 172.21.15.18:0/538927883 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:52:12.311 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:12.309+0000 7f4306a00700 1 --2- 172.21.15.18:0/538927883 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f43081059c0 0x7f43081160d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:52:12.311 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:12.310+0000 7f4307400700 1 -- 172.21.15.18:0/538927883 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f43081059c0 msgr2=0x7f43081160d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:52:12.311 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:12.310+0000 7f4307400700 1 --2- 172.21.15.18:0/538927883 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f43081059c0 0x7f43081160d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:52:12.312 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:12.310+0000 7f4307400700 1 -- 172.21.15.18:0/538927883 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42f8005040 con 0x7f4308105020 2024-04-12T21:52:12.312 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:12.310+0000 7f4307400700 1 --2- 172.21.15.18:0/538927883 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4308105020 0x7f4308115b90 secure :-1 s=READY pgs=1327 cs=0 l=1 rev1=1 crypto rx=0x7f42f8009150 tx=0x7f42f8009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:52:12.312 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:12.310+0000 7f42ff400700 1 -- 172.21.15.18:0/538927883 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f42f800e040 con 0x7f4308105020 2024-04-12T21:52:12.312 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:12.310+0000 7f42ff400700 1 -- 172.21.15.18:0/538927883 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f42f8013420 con 0x7f4308105020 2024-04-12T21:52:12.312 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:12.310+0000 7f42ff400700 1 -- 172.21.15.18:0/538927883 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f42f801c630 con 0x7f4308105020 2024-04-12T21:52:12.312 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:12.310+0000 7f430e000700 1 -- 172.21.15.18:0/538927883 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f430810ff80 con 0x7f4308105020 2024-04-12T21:52:12.312 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:12.310+0000 7f430e000700 1 -- 172.21.15.18:0/538927883 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4308110540 con 0x7f4308105020 2024-04-12T21:52:12.312 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:12.312+0000 7f430e000700 1 -- 172.21.15.18:0/538927883 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4308100770 con 0x7f4308105020 2024-04-12T21:52:12.314 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:12.313+0000 7f42ff400700 1 -- 172.21.15.18:0/538927883 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7f42f8013590 con 0x7f4308105020 2024-04-12T21:52:12.315 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:12.314+0000 7f42ff400700 1 --2- 172.21.15.18:0/538927883 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f42ec06ec00 0x7f42ec0710b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:52:12.315 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:12.314+0000 7f42ff400700 1 -- 172.21.15.18:0/538927883 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 6757+0+0 (secure 0 0 0) 0x7f42f808fc00 con 0x7f4308105020 2024-04-12T21:52:12.315 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:12.314+0000 7f4306a00700 1 --2- 172.21.15.18:0/538927883 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f42ec06ec00 0x7f42ec0710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:52:12.316 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:12.315+0000 7f4306a00700 1 --2- 172.21.15.18:0/538927883 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f42ec06ec00 0x7f42ec0710b0 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7f42f4007fd0 tx=0x7f42f4009040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:52:12.320 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:12.319+0000 7f42ff400700 1 -- 172.21.15.18:0/538927883 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f42f805c430 con 0x7f4308105020 2024-04-12T21:52:12.571 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:12.569+0000 7f430e000700 1 -- 172.21.15.18:0/538927883 --> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f43081111e0 con 0x7f42ec06ec00 2024-04-12T21:52:12.572 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:12.571+0000 7f42ff400700 1 -- 172.21.15.18:0/538927883 <== mgr.34102 v2:172.21.15.18:6800/3812891919 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f43081111e0 con 0x7f42ec06ec00 2024-04-12T21:52:12.576 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:12.574+0000 7f430e000700 1 -- 172.21.15.18:0/538927883 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f42ec06ec00 msgr2=0x7f42ec0710b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:52:12.576 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:12.574+0000 7f430e000700 1 --2- 172.21.15.18:0/538927883 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f42ec06ec00 0x7f42ec0710b0 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7f42f4007fd0 tx=0x7f42f4009040 comp rx=0 tx=0).stop 2024-04-12T21:52:12.576 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:12.574+0000 7f430e000700 1 -- 172.21.15.18:0/538927883 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4308105020 msgr2=0x7f4308115b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:52:12.576 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:12.574+0000 7f430e000700 1 --2- 172.21.15.18:0/538927883 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4308105020 0x7f4308115b90 secure :-1 s=READY pgs=1327 cs=0 l=1 rev1=1 crypto rx=0x7f42f8009150 tx=0x7f42f8009e30 comp rx=0 tx=0).stop 2024-04-12T21:52:12.576 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:12.574+0000 7f430e000700 1 -- 172.21.15.18:0/538927883 shutdown_connections 2024-04-12T21:52:12.576 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:12.574+0000 7f430e000700 1 --2- 172.21.15.18:0/538927883 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f42ec06ec00 0x7f42ec0710b0 unknown :-1 s=CLOSED pgs=703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:52:12.576 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:12.574+0000 7f430e000700 1 --2- 172.21.15.18:0/538927883 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4308105020 0x7f4308115b90 unknown :-1 s=CLOSED pgs=1327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:52:12.576 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:12.574+0000 7f430e000700 1 --2- 172.21.15.18:0/538927883 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f43081059c0 0x7f43081160d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:52:12.576 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:12.575+0000 7f430e000700 1 -- 172.21.15.18:0/538927883 >> 172.21.15.18:0/538927883 conn(0x7f43080faba0 msgr2=0x7f43080fbb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:52:12.577 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:12.575+0000 7f430e000700 1 -- 172.21.15.18:0/538927883 shutdown_connections 2024-04-12T21:52:12.577 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:12.575+0000 7f430e000700 1 -- 172.21.15.18:0/538927883 wait complete. 2024-04-12T21:52:12.733 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:12.731+0000 7f9471000700 1 -- 172.21.15.18:0/1585680153 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f946c1037c0 msgr2=0x7f946c0fce30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:52:12.733 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:12.731+0000 7f9471000700 1 --2- 172.21.15.18:0/1585680153 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f946c1037c0 0x7f946c0fce30 secure :-1 s=READY pgs=1328 cs=0 l=1 rev1=1 crypto rx=0x7f9458004940 tx=0x7f9458009e30 comp rx=0 tx=0).stop 2024-04-12T21:52:12.733 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:12.732+0000 7f9471000700 1 -- 172.21.15.18:0/1585680153 shutdown_connections 2024-04-12T21:52:12.733 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:12.732+0000 7f9471000700 1 --2- 172.21.15.18:0/1585680153 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f946c1037c0 0x7f946c0fce30 unknown :-1 s=CLOSED pgs=1328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:52:12.733 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:12.732+0000 7f9471000700 1 --2- 172.21.15.18:0/1585680153 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f946c102e20 0x7f946c1031f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:52:12.733 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:12.732+0000 7f9471000700 1 -- 172.21.15.18:0/1585680153 >> 172.21.15.18:0/1585680153 conn(0x7f946c0f8970 msgr2=0x7f946c0fad80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:52:12.733 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:12.732+0000 7f9471000700 1 -- 172.21.15.18:0/1585680153 shutdown_connections 2024-04-12T21:52:12.733 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:12.732+0000 7f9471000700 1 -- 172.21.15.18:0/1585680153 wait complete. 2024-04-12T21:52:12.734 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:12.733+0000 7f9471000700 1 Processor -- start 2024-04-12T21:52:12.735 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:12.733+0000 7f9471000700 1 -- start start 2024-04-12T21:52:12.735 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:12.733+0000 7f9471000700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f946c102e20 0x7f946c113c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:52:12.735 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:12.733+0000 7f9471000700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f946c1037c0 0x7f946c114170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:52:12.735 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:12.733+0000 7f9471000700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f946c1147c0 con 0x7f946c102e20 2024-04-12T21:52:12.735 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:12.733+0000 7f9471000700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f946c10db90 con 0x7f946c1037c0 2024-04-12T21:52:12.735 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:12.734+0000 7f9469e00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f946c102e20 0x7f946c113c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:52:12.735 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:12.734+0000 7f9469e00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f946c102e20 0x7f946c113c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:52552/0 (socket says 172.21.15.18:52552) 2024-04-12T21:52:12.735 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:12.734+0000 7f9469e00700 1 -- 172.21.15.18:0/1004123294 learned_addr learned my addr 172.21.15.18:0/1004123294 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:52:12.736 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:12.734+0000 7f9469400700 1 --2- 172.21.15.18:0/1004123294 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f946c1037c0 0x7f946c114170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:52:12.736 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:12.734+0000 7f9469e00700 1 -- 172.21.15.18:0/1004123294 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f946c1037c0 msgr2=0x7f946c114170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:52:12.736 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:12.734+0000 7f9469e00700 1 --2- 172.21.15.18:0/1004123294 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f946c1037c0 0x7f946c114170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:52:12.736 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:12.734+0000 7f9469e00700 1 -- 172.21.15.18:0/1004123294 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9458005040 con 0x7f946c102e20 2024-04-12T21:52:12.736 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:12.734+0000 7f9469400700 1 --2- 172.21.15.18:0/1004123294 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f946c1037c0 0x7f946c114170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-12T21:52:12.736 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:12.735+0000 7f9469e00700 1 --2- 172.21.15.18:0/1004123294 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f946c102e20 0x7f946c113c30 secure :-1 s=READY pgs=1329 cs=0 l=1 rev1=1 crypto rx=0x7f945c00c8c0 tx=0x7f945c0044c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:52:12.736 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:12.735+0000 7f9462000700 1 -- 172.21.15.18:0/1004123294 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f945c00a980 con 0x7f946c102e20 2024-04-12T21:52:12.737 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:12.735+0000 7f9462000700 1 -- 172.21.15.18:0/1004123294 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f945c00ce60 con 0x7f946c102e20 2024-04-12T21:52:12.737 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:12.735+0000 7f9471000700 1 -- 172.21.15.18:0/1004123294 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f946c10de70 con 0x7f946c102e20 2024-04-12T21:52:12.737 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:12.735+0000 7f9462000700 1 -- 172.21.15.18:0/1004123294 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f945c0115e0 con 0x7f946c102e20 2024-04-12T21:52:12.737 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:12.735+0000 7f9471000700 1 -- 172.21.15.18:0/1004123294 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f946c10e430 con 0x7f946c102e20 2024-04-12T21:52:12.737 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:12.736+0000 7f9471000700 1 -- 172.21.15.18:0/1004123294 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f946c1052f0 con 0x7f946c102e20 2024-04-12T21:52:12.738 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:12.737+0000 7f9462000700 1 -- 172.21.15.18:0/1004123294 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7f945c012400 con 0x7f946c102e20 2024-04-12T21:52:12.739 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:12.738+0000 7f9462000700 1 --2- 172.21.15.18:0/1004123294 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f945006ec10 0x7f94500710c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:52:12.739 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:12.738+0000 7f9462000700 1 -- 172.21.15.18:0/1004123294 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 6757+0+0 (secure 0 0 0) 0x7f945c08ea40 con 0x7f946c102e20 2024-04-12T21:52:12.739 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:12.739+0000 7f9469400700 1 --2- 172.21.15.18:0/1004123294 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f945006ec10 0x7f94500710c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:52:12.740 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:12.739+0000 7f9469400700 1 --2- 172.21.15.18:0/1004123294 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f945006ec10 0x7f94500710c0 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7f9458009fd0 tx=0x7f9458009e30 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:52:12.744 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:12.743+0000 7f9462000700 1 -- 172.21.15.18:0/1004123294 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f945c05b270 con 0x7f946c102e20 2024-04-12T21:52:12.988 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:12.987+0000 7f9471000700 1 -- 172.21.15.18:0/1004123294 --> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f946c109f70 con 0x7f945006ec10 2024-04-12T21:52:13.011 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:13.009+0000 7f9462000700 1 -- 172.21.15.18:0/1004123294 <== mgr.34102 v2:172.21.15.18:6800/3812891919 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7f946c109f70 con 0x7f945006ec10 2024-04-12T21:52:13.011 INFO:teuthology.orchestra.run.smithi018.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-12T21:52:13.011 INFO:teuthology.orchestra.run.smithi018.stdout:alertmanager.smithi018 smithi018 *:9093,9094 running (52m) 13m ago 67m 14.8M - 0.25.0 c8568f914cd2 c332090068e8 2024-04-12T21:52:13.011 INFO:teuthology.orchestra.run.smithi018.stdout:crash.smithi018 smithi018 running (49m) 13m ago 67m 6665k - 19.0.0-2879-g324baf8d 8717720dc5e5 b7f80d1f7982 2024-04-12T21:52:13.011 INFO:teuthology.orchestra.run.smithi018.stdout:crash.smithi186 smithi186 running (49m) 13m ago 66m 6660k - 19.0.0-2879-g324baf8d 8717720dc5e5 a4f55c81b623 2024-04-12T21:52:13.012 INFO:teuthology.orchestra.run.smithi018.stdout:grafana.smithi018 smithi018 *:3000 running (52m) 13m ago 66m 73.8M - 9.4.12 2bacad6d85d8 b08e3bf176dd 2024-04-12T21:52:13.012 INFO:teuthology.orchestra.run.smithi018.stdout:mds.cephfs.smithi018.acqbsc smithi018 running (45m) 13m ago 60m 21.9M - 19.0.0-2879-g324baf8d 8717720dc5e5 021f99ab24ae 2024-04-12T21:52:13.012 INFO:teuthology.orchestra.run.smithi018.stdout:mds.cephfs.smithi018.szjluo smithi018 running (44m) 13m ago 60m 21.7M - 19.0.0-2879-g324baf8d 8717720dc5e5 0003117479d4 2024-04-12T21:52:13.012 INFO:teuthology.orchestra.run.smithi018.stdout:mds.cephfs.smithi186.jdllyn smithi186 running (29m) 13m ago 60m 16.5M - 19.0.0-2879-g324baf8d 8717720dc5e5 21b1f46eef23 2024-04-12T21:52:13.012 INFO:teuthology.orchestra.run.smithi018.stdout:mds.cephfs.smithi186.wjnafs smithi186 running (14m) 13m ago 59m 11.0M - 19.0.0-2879-g324baf8d 8717720dc5e5 3cf1ef4148e2 2024-04-12T21:52:13.012 INFO:teuthology.orchestra.run.smithi018.stdout:mgr.smithi018.xjzwga smithi018 *:8443,9283,8765 running (55m) 13m ago 69m 517M - 19.0.0-2879-g324baf8d 8717720dc5e5 56088305eab7 2024-04-12T21:52:13.012 INFO:teuthology.orchestra.run.smithi018.stdout:mgr.smithi186.kqcfxk smithi186 *:8443,9283,8765 running (54m) 13m ago 66m 460M - 19.0.0-2879-g324baf8d 8717720dc5e5 09cd766e5e0d 2024-04-12T21:52:13.012 INFO:teuthology.orchestra.run.smithi018.stdout:mon.smithi018 smithi018 running (50m) 13m ago 69m 88.2M 2048M 19.0.0-2879-g324baf8d 8717720dc5e5 df3e201dfed6 2024-04-12T21:52:13.012 INFO:teuthology.orchestra.run.smithi018.stdout:mon.smithi186 smithi186 running (50m) 13m ago 65m 53.6M 2048M 19.0.0-2879-g324baf8d 8717720dc5e5 c77498da9b2b 2024-04-12T21:52:13.012 INFO:teuthology.orchestra.run.smithi018.stdout:node-exporter.smithi018 smithi018 *:9100 running (53m) 13m ago 67m 8468k - 1.5.0 0da6a335fe13 326c3d9c6847 2024-04-12T21:52:13.012 INFO:teuthology.orchestra.run.smithi018.stdout:node-exporter.smithi186 smithi186 *:9100 running (53m) 13m ago 66m 8166k - 1.5.0 0da6a335fe13 ec2112baeecf 2024-04-12T21:52:13.012 INFO:teuthology.orchestra.run.smithi018.stdout:osd.0 smithi018 running (48m) 13m ago 64m 123M 1559M 19.0.0-2879-g324baf8d 8717720dc5e5 60b1273ca047 2024-04-12T21:52:13.012 INFO:teuthology.orchestra.run.smithi018.stdout:osd.1 smithi018 running (47m) 13m ago 64m 95.4M 1559M 19.0.0-2879-g324baf8d 8717720dc5e5 170ce1e5fb8d 2024-04-12T21:52:13.012 INFO:teuthology.orchestra.run.smithi018.stdout:osd.2 smithi018 running (47m) 13m ago 63m 94.6M 1559M 19.0.0-2879-g324baf8d 8717720dc5e5 c2cbc6aa2839 2024-04-12T21:52:13.012 INFO:teuthology.orchestra.run.smithi018.stdout:osd.3 smithi186 running (46m) 13m ago 63m 101M 2581M 19.0.0-2879-g324baf8d 8717720dc5e5 0138e8262e4b 2024-04-12T21:52:13.013 INFO:teuthology.orchestra.run.smithi018.stdout:osd.4 smithi186 running (46m) 13m ago 62m 91.5M 2581M 19.0.0-2879-g324baf8d 8717720dc5e5 b20cb63b25c9 2024-04-12T21:52:13.013 INFO:teuthology.orchestra.run.smithi018.stdout:osd.5 smithi186 running (45m) 13m ago 62m 92.2M 2581M 19.0.0-2879-g324baf8d 8717720dc5e5 2513bea2610e 2024-04-12T21:52:13.013 INFO:teuthology.orchestra.run.smithi018.stdout:prometheus.smithi018 smithi018 *:9095 running (53m) 13m ago 66m 69.6M - 2.43.0 a07b618ecd1d f58841127e21 2024-04-12T21:52:13.015 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:13.013+0000 7f9471000700 1 -- 172.21.15.18:0/1004123294 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f945006ec10 msgr2=0x7f94500710c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:52:13.015 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:13.013+0000 7f9471000700 1 --2- 172.21.15.18:0/1004123294 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f945006ec10 0x7f94500710c0 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7f9458009fd0 tx=0x7f9458009e30 comp rx=0 tx=0).stop 2024-04-12T21:52:13.015 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:13.014+0000 7f9471000700 1 -- 172.21.15.18:0/1004123294 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f946c102e20 msgr2=0x7f946c113c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:52:13.015 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:13.014+0000 7f9471000700 1 --2- 172.21.15.18:0/1004123294 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f946c102e20 0x7f946c113c30 secure :-1 s=READY pgs=1329 cs=0 l=1 rev1=1 crypto rx=0x7f945c00c8c0 tx=0x7f945c0044c0 comp rx=0 tx=0).stop 2024-04-12T21:52:13.015 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:13.014+0000 7f9471000700 1 -- 172.21.15.18:0/1004123294 shutdown_connections 2024-04-12T21:52:13.015 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:13.014+0000 7f9471000700 1 --2- 172.21.15.18:0/1004123294 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f945006ec10 0x7f94500710c0 unknown :-1 s=CLOSED pgs=704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:52:13.015 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:13.014+0000 7f9471000700 1 --2- 172.21.15.18:0/1004123294 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f946c102e20 0x7f946c113c30 unknown :-1 s=CLOSED pgs=1329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:52:13.015 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:13.014+0000 7f9471000700 1 --2- 172.21.15.18:0/1004123294 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f946c1037c0 0x7f946c114170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:52:13.015 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:13.014+0000 7f9471000700 1 -- 172.21.15.18:0/1004123294 >> 172.21.15.18:0/1004123294 conn(0x7f946c0f8970 msgr2=0x7f946c102230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:52:13.015 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:13.014+0000 7f9471000700 1 -- 172.21.15.18:0/1004123294 shutdown_connections 2024-04-12T21:52:13.016 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:13.014+0000 7f9471000700 1 -- 172.21.15.18:0/1004123294 wait complete. 2024-04-12T21:52:13.160 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:13.158+0000 7fd060e00700 1 -- 172.21.15.18:0/646303343 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd05c1037a0 msgr2=0x7fd05c0fce10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:52:13.160 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:13.158+0000 7fd060e00700 1 --2- 172.21.15.18:0/646303343 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd05c1037a0 0x7fd05c0fce10 secure :-1 s=READY pgs=1330 cs=0 l=1 rev1=1 crypto rx=0x7fd048004940 tx=0x7fd048009e30 comp rx=0 tx=0).stop 2024-04-12T21:52:13.160 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:13.158+0000 7fd060e00700 1 -- 172.21.15.18:0/646303343 shutdown_connections 2024-04-12T21:52:13.160 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:13.158+0000 7fd060e00700 1 --2- 172.21.15.18:0/646303343 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd05c1037a0 0x7fd05c0fce10 unknown :-1 s=CLOSED pgs=1330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:52:13.160 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:13.158+0000 7fd060e00700 1 --2- 172.21.15.18:0/646303343 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd05c102e00 0x7fd05c1031d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:52:13.160 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:13.158+0000 7fd060e00700 1 -- 172.21.15.18:0/646303343 >> 172.21.15.18:0/646303343 conn(0x7fd05c0f89b0 msgr2=0x7fd05c0fadc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:52:13.160 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:13.158+0000 7fd060e00700 1 -- 172.21.15.18:0/646303343 shutdown_connections 2024-04-12T21:52:13.160 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:13.158+0000 7fd060e00700 1 -- 172.21.15.18:0/646303343 wait complete. 2024-04-12T21:52:13.161 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:13.159+0000 7fd060e00700 1 Processor -- start 2024-04-12T21:52:13.162 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:13.160+0000 7fd060e00700 1 -- start start 2024-04-12T21:52:13.162 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:13.160+0000 7fd060e00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd05c102e00 0x7fd05c1139a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:52:13.162 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:13.160+0000 7fd060e00700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd05c1037a0 0x7fd05c113ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:52:13.162 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:13.160+0000 7fd060e00700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd05c114530 con 0x7fd05c102e00 2024-04-12T21:52:13.162 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:13.160+0000 7fd060e00700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd05c10db30 con 0x7fd05c1037a0 2024-04-12T21:52:13.162 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:13.160+0000 7fd059e00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd05c102e00 0x7fd05c1139a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:52:13.162 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:13.160+0000 7fd059e00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd05c102e00 0x7fd05c1139a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:52574/0 (socket says 172.21.15.18:52574) 2024-04-12T21:52:13.162 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:13.160+0000 7fd059e00700 1 -- 172.21.15.18:0/2554668329 learned_addr learned my addr 172.21.15.18:0/2554668329 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:52:13.162 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:13.161+0000 7fd059400700 1 --2- 172.21.15.18:0/2554668329 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd05c1037a0 0x7fd05c113ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:52:13.162 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:13.161+0000 7fd059e00700 1 -- 172.21.15.18:0/2554668329 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd05c1037a0 msgr2=0x7fd05c113ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:52:13.162 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:13.161+0000 7fd059e00700 1 --2- 172.21.15.18:0/2554668329 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd05c1037a0 0x7fd05c113ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:52:13.162 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:13.161+0000 7fd059e00700 1 -- 172.21.15.18:0/2554668329 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd048005040 con 0x7fd05c102e00 2024-04-12T21:52:13.162 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:13.161+0000 7fd059400700 1 --2- 172.21.15.18:0/2554668329 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd05c1037a0 0x7fd05c113ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-12T21:52:13.163 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:13.161+0000 7fd059e00700 1 --2- 172.21.15.18:0/2554668329 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd05c102e00 0x7fd05c1139a0 secure :-1 s=READY pgs=1331 cs=0 l=1 rev1=1 crypto rx=0x7fd04c00c8c0 tx=0x7fd04c0044c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:52:13.163 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:13.161+0000 7fd052000700 1 -- 172.21.15.18:0/2554668329 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd04c00a980 con 0x7fd05c102e00 2024-04-12T21:52:13.164 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:13.161+0000 7fd052000700 1 -- 172.21.15.18:0/2554668329 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd04c00ce60 con 0x7fd05c102e00 2024-04-12T21:52:13.164 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:13.161+0000 7fd052000700 1 -- 172.21.15.18:0/2554668329 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd04c0115e0 con 0x7fd05c102e00 2024-04-12T21:52:13.164 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:13.161+0000 7fd060e00700 1 -- 172.21.15.18:0/2554668329 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd05c10de10 con 0x7fd05c102e00 2024-04-12T21:52:13.164 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:13.161+0000 7fd060e00700 1 -- 172.21.15.18:0/2554668329 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd05c10e3d0 con 0x7fd05c102e00 2024-04-12T21:52:13.164 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:13.162+0000 7fd060e00700 1 -- 172.21.15.18:0/2554668329 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd05c100110 con 0x7fd05c102e00 2024-04-12T21:52:13.165 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:13.164+0000 7fd052000700 1 -- 172.21.15.18:0/2554668329 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7fd04c012400 con 0x7fd05c102e00 2024-04-12T21:52:13.166 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:13.165+0000 7fd052000700 1 --2- 172.21.15.18:0/2554668329 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fd04006ec00 0x7fd0400710b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:52:13.166 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:13.165+0000 7fd052000700 1 -- 172.21.15.18:0/2554668329 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 6757+0+0 (secure 0 0 0) 0x7fd04c08f6f0 con 0x7fd05c102e00 2024-04-12T21:52:13.166 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:13.165+0000 7fd059400700 1 --2- 172.21.15.18:0/2554668329 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fd04006ec00 0x7fd0400710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:52:13.167 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:13.166+0000 7fd059400700 1 --2- 172.21.15.18:0/2554668329 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fd04006ec00 0x7fd0400710b0 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7fd048009fd0 tx=0x7fd048009e30 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:52:13.170 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:13.169+0000 7fd052000700 1 -- 172.21.15.18:0/2554668329 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7fd04c05bf20 con 0x7fd05c102e00 2024-04-12T21:52:13.494 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:13.493+0000 7fd060e00700 1 -- 172.21.15.18:0/2554668329 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fd05c10eff0 con 0x7fd05c102e00 2024-04-12T21:52:13.495 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:13.495+0000 7fd052000700 1 -- 172.21.15.18:0/2554668329 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+623 (secure 0 0 0) 0x7fd04c0126b0 con 0x7fd05c102e00 2024-04-12T21:52:13.496 INFO:teuthology.orchestra.run.smithi018.stdout:{ 2024-04-12T21:52:13.496 INFO:teuthology.orchestra.run.smithi018.stdout: "mon": { 2024-04-12T21:52:13.496 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 19.0.0-2879-g324baf8d (324baf8ddb91dea46cac50cb535c306115791355) squid (dev)": 2 2024-04-12T21:52:13.497 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-12T21:52:13.497 INFO:teuthology.orchestra.run.smithi018.stdout: "mgr": { 2024-04-12T21:52:13.497 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 19.0.0-2879-g324baf8d (324baf8ddb91dea46cac50cb535c306115791355) squid (dev)": 2 2024-04-12T21:52:13.497 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-12T21:52:13.497 INFO:teuthology.orchestra.run.smithi018.stdout: "osd": { 2024-04-12T21:52:13.497 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 19.0.0-2879-g324baf8d (324baf8ddb91dea46cac50cb535c306115791355) squid (dev)": 6 2024-04-12T21:52:13.497 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-12T21:52:13.497 INFO:teuthology.orchestra.run.smithi018.stdout: "mds": { 2024-04-12T21:52:13.497 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 19.0.0-2879-g324baf8d (324baf8ddb91dea46cac50cb535c306115791355) squid (dev)": 4 2024-04-12T21:52:13.497 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-12T21:52:13.497 INFO:teuthology.orchestra.run.smithi018.stdout: "overall": { 2024-04-12T21:52:13.497 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 19.0.0-2879-g324baf8d (324baf8ddb91dea46cac50cb535c306115791355) squid (dev)": 14 2024-04-12T21:52:13.497 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-04-12T21:52:13.497 INFO:teuthology.orchestra.run.smithi018.stdout:} 2024-04-12T21:52:13.499 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:13.497+0000 7fd060e00700 1 -- 172.21.15.18:0/2554668329 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fd04006ec00 msgr2=0x7fd0400710b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:52:13.499 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:13.497+0000 7fd060e00700 1 --2- 172.21.15.18:0/2554668329 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fd04006ec00 0x7fd0400710b0 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7fd048009fd0 tx=0x7fd048009e30 comp rx=0 tx=0).stop 2024-04-12T21:52:13.499 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:13.497+0000 7fd060e00700 1 -- 172.21.15.18:0/2554668329 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd05c102e00 msgr2=0x7fd05c1139a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:52:13.499 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:13.497+0000 7fd060e00700 1 --2- 172.21.15.18:0/2554668329 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd05c102e00 0x7fd05c1139a0 secure :-1 s=READY pgs=1331 cs=0 l=1 rev1=1 crypto rx=0x7fd04c00c8c0 tx=0x7fd04c0044c0 comp rx=0 tx=0).stop 2024-04-12T21:52:13.499 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:13.497+0000 7fd060e00700 1 -- 172.21.15.18:0/2554668329 shutdown_connections 2024-04-12T21:52:13.499 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:13.497+0000 7fd060e00700 1 --2- 172.21.15.18:0/2554668329 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fd04006ec00 0x7fd0400710b0 unknown :-1 s=CLOSED pgs=705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:52:13.499 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:13.497+0000 7fd060e00700 1 --2- 172.21.15.18:0/2554668329 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd05c102e00 0x7fd05c1139a0 unknown :-1 s=CLOSED pgs=1331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:52:13.499 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:13.497+0000 7fd060e00700 1 --2- 172.21.15.18:0/2554668329 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd05c1037a0 0x7fd05c113ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:52:13.499 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:13.497+0000 7fd060e00700 1 -- 172.21.15.18:0/2554668329 >> 172.21.15.18:0/2554668329 conn(0x7fd05c0f89b0 msgr2=0x7fd05c107380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:52:13.499 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:13.498+0000 7fd060e00700 1 -- 172.21.15.18:0/2554668329 shutdown_connections 2024-04-12T21:52:13.499 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:13.498+0000 7fd060e00700 1 -- 172.21.15.18:0/2554668329 wait complete. 2024-04-12T21:52:13.643 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:13.642+0000 7fb50fe00700 1 -- 172.21.15.18:0/2323220824 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb510104e30 msgr2=0x7fb510105200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:52:13.644 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:13.642+0000 7fb50fe00700 1 --2- 172.21.15.18:0/2323220824 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb510104e30 0x7fb510105200 secure :-1 s=READY pgs=1332 cs=0 l=1 rev1=1 crypto rx=0x7fb500004940 tx=0x7fb500009e30 comp rx=0 tx=0).stop 2024-04-12T21:52:13.644 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:13.642+0000 7fb50fe00700 1 -- 172.21.15.18:0/2323220824 shutdown_connections 2024-04-12T21:52:13.644 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:13.642+0000 7fb50fe00700 1 --2- 172.21.15.18:0/2323220824 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb5101057d0 0x7fb5100fee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:52:13.644 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:13.642+0000 7fb50fe00700 1 --2- 172.21.15.18:0/2323220824 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb510104e30 0x7fb510105200 unknown :-1 s=CLOSED pgs=1332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:52:13.644 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:13.642+0000 7fb50fe00700 1 -- 172.21.15.18:0/2323220824 >> 172.21.15.18:0/2323220824 conn(0x7fb5100fab10 msgr2=0x7fb5100fcef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:52:13.644 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:13.642+0000 7fb50fe00700 1 -- 172.21.15.18:0/2323220824 shutdown_connections 2024-04-12T21:52:13.644 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:13.642+0000 7fb50fe00700 1 -- 172.21.15.18:0/2323220824 wait complete. 2024-04-12T21:52:13.645 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:13.643+0000 7fb50fe00700 1 Processor -- start 2024-04-12T21:52:13.645 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:13.643+0000 7fb50fe00700 1 -- start start 2024-04-12T21:52:13.645 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:13.644+0000 7fb50fe00700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb510104e30 0x7fb510115ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:52:13.645 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:13.644+0000 7fb50fe00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb5101057d0 0x7fb5101160e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:52:13.645 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:13.644+0000 7fb50fe00700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb510116730 con 0x7fb5101057d0 2024-04-12T21:52:13.645 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:13.644+0000 7fb50fe00700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb51010fb20 con 0x7fb510104e30 2024-04-12T21:52:13.645 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:13.644+0000 7fb50e000700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb5101057d0 0x7fb5101160e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:52:13.646 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:13.644+0000 7fb50e000700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb5101057d0 0x7fb5101160e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:52596/0 (socket says 172.21.15.18:52596) 2024-04-12T21:52:13.646 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:13.644+0000 7fb50e000700 1 -- 172.21.15.18:0/4207979881 learned_addr learned my addr 172.21.15.18:0/4207979881 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:52:13.647 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:13.645+0000 7fb50e000700 1 -- 172.21.15.18:0/4207979881 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb510104e30 msgr2=0x7fb510115ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:52:13.647 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:13.645+0000 7fb50ea00700 1 --2- 172.21.15.18:0/4207979881 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb510104e30 0x7fb510115ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:52:13.647 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:13.645+0000 7fb50e000700 1 --2- 172.21.15.18:0/4207979881 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb510104e30 0x7fb510115ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:52:13.647 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:13.645+0000 7fb50e000700 1 -- 172.21.15.18:0/4207979881 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb500005040 con 0x7fb5101057d0 2024-04-12T21:52:13.647 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:13.645+0000 7fb50e000700 1 --2- 172.21.15.18:0/4207979881 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb5101057d0 0x7fb5101160e0 secure :-1 s=READY pgs=1333 cs=0 l=1 rev1=1 crypto rx=0x7fb4fc002a50 tx=0x7fb4fc004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:52:13.647 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:13.645+0000 7fb506a00700 1 -- 172.21.15.18:0/4207979881 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb4fc007cc0 con 0x7fb5101057d0 2024-04-12T21:52:13.647 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:13.645+0000 7fb506a00700 1 -- 172.21.15.18:0/4207979881 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb4fc00f040 con 0x7fb5101057d0 2024-04-12T21:52:13.647 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:13.645+0000 7fb506a00700 1 -- 172.21.15.18:0/4207979881 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb4fc00daf0 con 0x7fb5101057d0 2024-04-12T21:52:13.647 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:13.645+0000 7fb50fe00700 1 -- 172.21.15.18:0/4207979881 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb51010fe00 con 0x7fb5101057d0 2024-04-12T21:52:13.647 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:13.646+0000 7fb50fe00700 1 -- 172.21.15.18:0/4207979881 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb5101103c0 con 0x7fb5101057d0 2024-04-12T21:52:13.648 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:13.647+0000 7fb50fe00700 1 -- 172.21.15.18:0/4207979881 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb510102140 con 0x7fb5101057d0 2024-04-12T21:52:13.649 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:13.648+0000 7fb506a00700 1 -- 172.21.15.18:0/4207979881 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7fb4fc009810 con 0x7fb5101057d0 2024-04-12T21:52:13.650 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:13.649+0000 7fb506a00700 1 --2- 172.21.15.18:0/4207979881 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fb4f406ec00 0x7fb4f40710b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:52:13.650 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:13.649+0000 7fb506a00700 1 -- 172.21.15.18:0/4207979881 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 6757+0+0 (secure 0 0 0) 0x7fb4fc08e660 con 0x7fb5101057d0 2024-04-12T21:52:13.650 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:13.649+0000 7fb50ea00700 1 --2- 172.21.15.18:0/4207979881 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fb4f406ec00 0x7fb4f40710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:52:13.651 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:13.650+0000 7fb50ea00700 1 --2- 172.21.15.18:0/4207979881 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fb4f406ec00 0x7fb4f40710b0 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7fb50000b3b0 tx=0x7fb500009e30 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:52:13.656 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:13.655+0000 7fb506a00700 1 -- 172.21.15.18:0/4207979881 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7fb4fc05aeb0 con 0x7fb5101057d0 2024-04-12T21:52:13.930 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:13.928+0000 7fb50fe00700 1 -- 172.21.15.18:0/4207979881 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fb5101111f0 con 0x7fb5101057d0 2024-04-12T21:52:13.930 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:13.929+0000 7fb506a00700 1 -- 172.21.15.18:0/4207979881 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 33 v33) v1 ==== 76+0+1876 (secure 0 0 0) 0x7fb4fc05aa40 con 0x7fb5101057d0 2024-04-12T21:52:13.931 INFO:teuthology.orchestra.run.smithi018.stdout:e33 2024-04-12T21:52:13.931 INFO:teuthology.orchestra.run.smithi018.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-12T21:52:13.931 INFO:teuthology.orchestra.run.smithi018.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-12T21:52:13.932 INFO:teuthology.orchestra.run.smithi018.stdout:legacy client fscid: 1 2024-04-12T21:52:13.932 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-12T21:52:13.932 INFO:teuthology.orchestra.run.smithi018.stdout:Filesystem 'cephfs' (1) 2024-04-12T21:52:13.932 INFO:teuthology.orchestra.run.smithi018.stdout:fs_name cephfs 2024-04-12T21:52:13.932 INFO:teuthology.orchestra.run.smithi018.stdout:epoch 24 2024-04-12T21:52:13.932 INFO:teuthology.orchestra.run.smithi018.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-04-12T21:52:13.932 INFO:teuthology.orchestra.run.smithi018.stdout:created 2024-04-12T20:51:57.362850+0000 2024-04-12T21:52:13.932 INFO:teuthology.orchestra.run.smithi018.stdout:modified 2024-04-12T21:06:59.717190+0000 2024-04-12T21:52:13.932 INFO:teuthology.orchestra.run.smithi018.stdout:tableserver 0 2024-04-12T21:52:13.932 INFO:teuthology.orchestra.run.smithi018.stdout:root 0 2024-04-12T21:52:13.932 INFO:teuthology.orchestra.run.smithi018.stdout:session_timeout 60 2024-04-12T21:52:13.932 INFO:teuthology.orchestra.run.smithi018.stdout:session_autoclose 300 2024-04-12T21:52:13.932 INFO:teuthology.orchestra.run.smithi018.stdout:max_file_size 1099511627776 2024-04-12T21:52:13.932 INFO:teuthology.orchestra.run.smithi018.stdout:max_xattr_size 65536 2024-04-12T21:52:13.933 INFO:teuthology.orchestra.run.smithi018.stdout:required_client_features {} 2024-04-12T21:52:13.933 INFO:teuthology.orchestra.run.smithi018.stdout:last_failure 0 2024-04-12T21:52:13.933 INFO:teuthology.orchestra.run.smithi018.stdout:last_failure_osd_epoch 100 2024-04-12T21:52:13.933 INFO:teuthology.orchestra.run.smithi018.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-12T21:52:13.933 INFO:teuthology.orchestra.run.smithi018.stdout:max_mds 2 2024-04-12T21:52:13.933 INFO:teuthology.orchestra.run.smithi018.stdout:in 0,1 2024-04-12T21:52:13.933 INFO:teuthology.orchestra.run.smithi018.stdout:up {} 2024-04-12T21:52:13.933 INFO:teuthology.orchestra.run.smithi018.stdout:failed 0,1 2024-04-12T21:52:13.933 INFO:teuthology.orchestra.run.smithi018.stdout:damaged 2024-04-12T21:52:13.933 INFO:teuthology.orchestra.run.smithi018.stdout:stopped 2024-04-12T21:52:13.933 INFO:teuthology.orchestra.run.smithi018.stdout:data_pools [3] 2024-04-12T21:52:13.933 INFO:teuthology.orchestra.run.smithi018.stdout:metadata_pool 2 2024-04-12T21:52:13.933 INFO:teuthology.orchestra.run.smithi018.stdout:inline_data disabled 2024-04-12T21:52:13.933 INFO:teuthology.orchestra.run.smithi018.stdout:balancer 2024-04-12T21:52:13.933 INFO:teuthology.orchestra.run.smithi018.stdout:bal_rank_mask -1 2024-04-12T21:52:13.934 INFO:teuthology.orchestra.run.smithi018.stdout:standby_count_wanted 1 2024-04-12T21:52:13.934 INFO:teuthology.orchestra.run.smithi018.stdout:qdb_cluster leader: 0 members: 2024-04-12T21:52:13.934 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-12T21:52:13.934 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-12T21:52:13.934 INFO:teuthology.orchestra.run.smithi018.stdout:Standby daemons: 2024-04-12T21:52:13.934 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-12T21:52:13.934 INFO:teuthology.orchestra.run.smithi018.stdout:[mds.cephfs.smithi018.acqbsc{-1:34436} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.18:6828/755179764,v1:172.21.15.18:6829/755179764] compat {c=[1],r=[1],i=[1fff]}] 2024-04-12T21:52:13.934 INFO:teuthology.orchestra.run.smithi018.stdout:[mds.cephfs.smithi018.szjluo{-1:34468} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.18:6826/2407992653,v1:172.21.15.18:6827/2407992653] compat {c=[1],r=[1],i=[1fff]}] 2024-04-12T21:52:13.934 INFO:teuthology.orchestra.run.smithi018.stdout:[mds.cephfs.smithi186.jdllyn{-1:44595} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.186:6824/3523444202,v1:172.21.15.186:6825/3523444202] compat {c=[1],r=[1],i=[1fff]}] 2024-04-12T21:52:13.934 INFO:teuthology.orchestra.run.smithi018.stdout:[mds.cephfs.smithi186.wjnafs{-1:44875} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.186:6826/3122443766,v1:172.21.15.186:6827/3122443766] compat {c=[1],r=[1],i=[1fff]}] 2024-04-12T21:52:13.935 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:13.931+0000 7fb50fe00700 1 -- 172.21.15.18:0/4207979881 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fb4f406ec00 msgr2=0x7fb4f40710b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:52:13.935 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:13.931+0000 7fb50fe00700 1 --2- 172.21.15.18:0/4207979881 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fb4f406ec00 0x7fb4f40710b0 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7fb50000b3b0 tx=0x7fb500009e30 comp rx=0 tx=0).stop 2024-04-12T21:52:13.935 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:13.932+0000 7fb50fe00700 1 -- 172.21.15.18:0/4207979881 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb5101057d0 msgr2=0x7fb5101160e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:52:13.935 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:13.932+0000 7fb50fe00700 1 --2- 172.21.15.18:0/4207979881 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb5101057d0 0x7fb5101160e0 secure :-1 s=READY pgs=1333 cs=0 l=1 rev1=1 crypto rx=0x7fb4fc002a50 tx=0x7fb4fc004600 comp rx=0 tx=0).stop 2024-04-12T21:52:13.935 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:13.932+0000 7fb50fe00700 1 -- 172.21.15.18:0/4207979881 shutdown_connections 2024-04-12T21:52:13.935 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:13.932+0000 7fb50fe00700 1 --2- 172.21.15.18:0/4207979881 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fb4f406ec00 0x7fb4f40710b0 unknown :-1 s=CLOSED pgs=706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:52:13.936 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:13.932+0000 7fb50fe00700 1 --2- 172.21.15.18:0/4207979881 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb510104e30 0x7fb510115ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:52:13.936 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:13.932+0000 7fb50fe00700 1 --2- 172.21.15.18:0/4207979881 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb5101057d0 0x7fb5101160e0 unknown :-1 s=CLOSED pgs=1333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:52:13.936 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:13.932+0000 7fb50fe00700 1 -- 172.21.15.18:0/4207979881 >> 172.21.15.18:0/4207979881 conn(0x7fb5100fab10 msgr2=0x7fb5100fd930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:52:13.936 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:13.932+0000 7fb50fe00700 1 -- 172.21.15.18:0/4207979881 shutdown_connections 2024-04-12T21:52:13.936 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:13.932+0000 7fb50fe00700 1 -- 172.21.15.18:0/4207979881 wait complete. 2024-04-12T21:52:13.936 INFO:teuthology.orchestra.run.smithi018.stderr:dumped fsmap epoch 33 2024-04-12T21:52:14.087 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:14.086+0000 7fae13200700 1 -- 172.21.15.18:0/2657293948 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fae14109c40 msgr2=0x7fae141032b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:52:14.087 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:14.086+0000 7fae13200700 1 --2- 172.21.15.18:0/2657293948 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fae14109c40 0x7fae141032b0 secure :-1 s=READY pgs=1334 cs=0 l=1 rev1=1 crypto rx=0x7fae00004940 tx=0x7fae00009e30 comp rx=0 tx=0).stop 2024-04-12T21:52:14.087 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:14.086+0000 7fae13200700 1 -- 172.21.15.18:0/2657293948 shutdown_connections 2024-04-12T21:52:14.087 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:14.086+0000 7fae13200700 1 --2- 172.21.15.18:0/2657293948 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fae14109c40 0x7fae141032b0 unknown :-1 s=CLOSED pgs=1334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:52:14.087 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:14.086+0000 7fae13200700 1 --2- 172.21.15.18:0/2657293948 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fae141092a0 0x7fae14109670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:52:14.087 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:14.086+0000 7fae13200700 1 -- 172.21.15.18:0/2657293948 >> 172.21.15.18:0/2657293948 conn(0x7fae14072a30 msgr2=0x7fae14074e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:52:14.088 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:52:13 smithi018 ceph-mon[112590]: from='client.36720 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:52:14.088 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:52:13 smithi018 ceph-mon[112590]: pgmap v1576: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:52:14.088 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:52:13 smithi018 ceph-mon[112590]: from='client.36724 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:52:14.088 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:52:13 smithi018 ceph-mon[112590]: from='client.36728 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:52:14.088 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:52:13 smithi018 ceph-mon[112590]: from='client.? 172.21.15.18:0/2554668329' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T21:52:14.089 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:14.086+0000 7fae13200700 1 -- 172.21.15.18:0/2657293948 shutdown_connections 2024-04-12T21:52:14.090 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:14.086+0000 7fae13200700 1 -- 172.21.15.18:0/2657293948 wait complete. 2024-04-12T21:52:14.090 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:14.087+0000 7fae13200700 1 Processor -- start 2024-04-12T21:52:14.090 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:14.087+0000 7fae13200700 1 -- start start 2024-04-12T21:52:14.090 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:14.087+0000 7fae13200700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fae141092a0 0x7fae14119f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:52:14.090 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:14.087+0000 7fae13200700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fae14109c40 0x7fae1411a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:52:14.090 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:14.087+0000 7fae13200700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae1411aad0 con 0x7fae14109c40 2024-04-12T21:52:14.090 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:14.087+0000 7fae13200700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae14113ea0 con 0x7fae141092a0 2024-04-12T21:52:14.090 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:14.088+0000 7fae11400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fae14109c40 0x7fae1411a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:52:14.090 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:14.088+0000 7fae11400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fae14109c40 0x7fae1411a480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:52606/0 (socket says 172.21.15.18:52606) 2024-04-12T21:52:14.090 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:14.088+0000 7fae11400700 1 -- 172.21.15.18:0/1027862777 learned_addr learned my addr 172.21.15.18:0/1027862777 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:52:14.090 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:14.088+0000 7fae11e00700 1 --2- 172.21.15.18:0/1027862777 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fae141092a0 0x7fae14119f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:52:14.090 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:14.088+0000 7fae11400700 1 -- 172.21.15.18:0/1027862777 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fae141092a0 msgr2=0x7fae14119f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:52:14.091 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:14.088+0000 7fae11400700 1 --2- 172.21.15.18:0/1027862777 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fae141092a0 0x7fae14119f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:52:14.091 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:14.088+0000 7fae11400700 1 -- 172.21.15.18:0/1027862777 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae00005040 con 0x7fae14109c40 2024-04-12T21:52:14.091 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:14.089+0000 7fae11400700 1 --2- 172.21.15.18:0/1027862777 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fae14109c40 0x7fae1411a480 secure :-1 s=READY pgs=1335 cs=0 l=1 rev1=1 crypto rx=0x7fae000040d0 tx=0x7fae00009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:52:14.091 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:14.089+0000 7fae0a000700 1 -- 172.21.15.18:0/1027862777 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fae0000e030 con 0x7fae14109c40 2024-04-12T21:52:14.091 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:14.089+0000 7fae0a000700 1 -- 172.21.15.18:0/1027862777 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fae00013c00 con 0x7fae14109c40 2024-04-12T21:52:14.092 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:14.089+0000 7fae13200700 1 -- 172.21.15.18:0/1027862777 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fae14114120 con 0x7fae14109c40 2024-04-12T21:52:14.092 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:14.089+0000 7fae0a000700 1 -- 172.21.15.18:0/1027862777 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fae0000b650 con 0x7fae14109c40 2024-04-12T21:52:14.092 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:14.089+0000 7fae13200700 1 -- 172.21.15.18:0/1027862777 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fae141146e0 con 0x7fae14109c40 2024-04-12T21:52:14.092 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:14.090+0000 7fae13200700 1 -- 172.21.15.18:0/1027862777 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fae141049f0 con 0x7fae14109c40 2024-04-12T21:52:14.093 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:14.091+0000 7fae0a000700 1 -- 172.21.15.18:0/1027862777 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7fae0000b8c0 con 0x7fae14109c40 2024-04-12T21:52:14.093 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:14.092+0000 7fae0a000700 1 --2- 172.21.15.18:0/1027862777 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fadf806ec00 0x7fadf80710b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:52:14.094 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:14.093+0000 7fae0a000700 1 -- 172.21.15.18:0/1027862777 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 6757+0+0 (secure 0 0 0) 0x7fae00090590 con 0x7fae14109c40 2024-04-12T21:52:14.094 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:14.093+0000 7fae11e00700 1 --2- 172.21.15.18:0/1027862777 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fadf806ec00 0x7fadf80710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:52:14.094 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:14.093+0000 7fae11e00700 1 --2- 172.21.15.18:0/1027862777 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fadf806ec00 0x7fadf80710b0 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7fae04009fd0 tx=0x7fae0400b040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:52:14.098 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:14.097+0000 7fae0a000700 1 -- 172.21.15.18:0/1027862777 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7fae0005cdc0 con 0x7fae14109c40 2024-04-12T21:52:14.285 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:52:13 smithi186 ceph-mon[86379]: from='client.36720 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:52:14.285 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:52:13 smithi186 ceph-mon[86379]: pgmap v1576: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:52:14.285 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:52:13 smithi186 ceph-mon[86379]: from='client.36724 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:52:14.285 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:52:13 smithi186 ceph-mon[86379]: from='client.36728 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:52:14.285 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:52:13 smithi186 ceph-mon[86379]: from='client.? 172.21.15.18:0/2554668329' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T21:52:14.345 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:14.344+0000 7fae13200700 1 -- 172.21.15.18:0/1027862777 --> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fae141103e0 con 0x7fadf806ec00 2024-04-12T21:52:14.347 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:14.346+0000 7fae0a000700 1 -- 172.21.15.18:0/1027862777 <== mgr.34102 v2:172.21.15.18:6800/3812891919 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7fae141103e0 con 0x7fadf806ec00 2024-04-12T21:52:14.347 INFO:teuthology.orchestra.run.smithi018.stdout:{ 2024-04-12T21:52:14.347 INFO:teuthology.orchestra.run.smithi018.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:f4452def3be4d350f4a8e5feaf7ebb1511be9298fe68478ba7a2d6fbd7d69a63", 2024-04-12T21:52:14.347 INFO:teuthology.orchestra.run.smithi018.stdout: "in_progress": true, 2024-04-12T21:52:14.347 INFO:teuthology.orchestra.run.smithi018.stdout: "which": "Upgrading all daemon types on all hosts", 2024-04-12T21:52:14.347 INFO:teuthology.orchestra.run.smithi018.stdout: "services_complete": [ 2024-04-12T21:52:14.347 INFO:teuthology.orchestra.run.smithi018.stdout: "mgr", 2024-04-12T21:52:14.348 INFO:teuthology.orchestra.run.smithi018.stdout: "mds", 2024-04-12T21:52:14.348 INFO:teuthology.orchestra.run.smithi018.stdout: "crash", 2024-04-12T21:52:14.348 INFO:teuthology.orchestra.run.smithi018.stdout: "osd", 2024-04-12T21:52:14.348 INFO:teuthology.orchestra.run.smithi018.stdout: "mon" 2024-04-12T21:52:14.348 INFO:teuthology.orchestra.run.smithi018.stdout: ], 2024-04-12T21:52:14.348 INFO:teuthology.orchestra.run.smithi018.stdout: "progress": "16/21 daemons upgraded", 2024-04-12T21:52:14.348 INFO:teuthology.orchestra.run.smithi018.stdout: "message": "Currently upgrading mds daemons", 2024-04-12T21:52:14.348 INFO:teuthology.orchestra.run.smithi018.stdout: "is_paused": false 2024-04-12T21:52:14.348 INFO:teuthology.orchestra.run.smithi018.stdout:} 2024-04-12T21:52:14.349 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:14.348+0000 7fae13200700 1 -- 172.21.15.18:0/1027862777 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fadf806ec00 msgr2=0x7fadf80710b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:52:14.349 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:14.348+0000 7fae13200700 1 --2- 172.21.15.18:0/1027862777 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fadf806ec00 0x7fadf80710b0 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7fae04009fd0 tx=0x7fae0400b040 comp rx=0 tx=0).stop 2024-04-12T21:52:14.349 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:14.348+0000 7fae13200700 1 -- 172.21.15.18:0/1027862777 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fae14109c40 msgr2=0x7fae1411a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:52:14.349 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:14.348+0000 7fae13200700 1 --2- 172.21.15.18:0/1027862777 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fae14109c40 0x7fae1411a480 secure :-1 s=READY pgs=1335 cs=0 l=1 rev1=1 crypto rx=0x7fae000040d0 tx=0x7fae00009e30 comp rx=0 tx=0).stop 2024-04-12T21:52:14.350 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:14.348+0000 7fae13200700 1 -- 172.21.15.18:0/1027862777 shutdown_connections 2024-04-12T21:52:14.350 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:14.348+0000 7fae13200700 1 --2- 172.21.15.18:0/1027862777 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fadf806ec00 0x7fadf80710b0 unknown :-1 s=CLOSED pgs=707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:52:14.350 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:14.349+0000 7fae13200700 1 --2- 172.21.15.18:0/1027862777 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fae141092a0 0x7fae14119f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:52:14.350 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:14.349+0000 7fae13200700 1 --2- 172.21.15.18:0/1027862777 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fae14109c40 0x7fae1411a480 unknown :-1 s=CLOSED pgs=1335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:52:14.350 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:14.349+0000 7fae13200700 1 -- 172.21.15.18:0/1027862777 >> 172.21.15.18:0/1027862777 conn(0x7fae14072a30 msgr2=0x7fae14074870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:52:14.350 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:14.349+0000 7fae13200700 1 -- 172.21.15.18:0/1027862777 shutdown_connections 2024-04-12T21:52:14.350 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:14.349+0000 7fae13200700 1 -- 172.21.15.18:0/1027862777 wait complete. 2024-04-12T21:52:14.505 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:14.503+0000 7f366c800700 1 -- 172.21.15.18:0/2025187018 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3664070c70 msgr2=0x7f3664109110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:52:14.505 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:14.503+0000 7f366c800700 1 --2- 172.21.15.18:0/2025187018 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3664070c70 0x7f3664109110 secure :-1 s=READY pgs=1336 cs=0 l=1 rev1=1 crypto rx=0x7f3654004970 tx=0x7f3654009e30 comp rx=0 tx=0).stop 2024-04-12T21:52:14.505 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:14.504+0000 7f366c800700 1 -- 172.21.15.18:0/2025187018 shutdown_connections 2024-04-12T21:52:14.505 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:14.504+0000 7f366c800700 1 --2- 172.21.15.18:0/2025187018 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3664070c70 0x7f3664109110 unknown :-1 s=CLOSED pgs=1336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:52:14.505 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:14.504+0000 7f366c800700 1 --2- 172.21.15.18:0/2025187018 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3664070360 0x7f3664070730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:52:14.505 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:14.504+0000 7f366c800700 1 -- 172.21.15.18:0/2025187018 >> 172.21.15.18:0/2025187018 conn(0x7f36640f89b0 msgr2=0x7f36640fadc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:52:14.505 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:14.504+0000 7f366c800700 1 -- 172.21.15.18:0/2025187018 shutdown_connections 2024-04-12T21:52:14.505 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:14.504+0000 7f366c800700 1 -- 172.21.15.18:0/2025187018 wait complete. 2024-04-12T21:52:14.507 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:14.505+0000 7f366c800700 1 Processor -- start 2024-04-12T21:52:14.507 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:14.505+0000 7f366c800700 1 -- start start 2024-04-12T21:52:14.507 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:14.505+0000 7f366c800700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3664070360 0x7f3664115bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:52:14.507 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:14.505+0000 7f366c800700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3664070c70 0x7f3664116100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:52:14.507 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:14.505+0000 7f366c800700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36641167e0 con 0x7f3664070360 2024-04-12T21:52:14.507 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:14.505+0000 7f366c800700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f366410fd30 con 0x7f3664070c70 2024-04-12T21:52:14.507 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:14.505+0000 7f3669e00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3664070360 0x7f3664115bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:52:14.507 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:14.506+0000 7f3669e00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3664070360 0x7f3664115bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:52620/0 (socket says 172.21.15.18:52620) 2024-04-12T21:52:14.507 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:14.506+0000 7f3669e00700 1 -- 172.21.15.18:0/3932773701 learned_addr learned my addr 172.21.15.18:0/3932773701 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:52:14.507 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:14.506+0000 7f3669400700 1 --2- 172.21.15.18:0/3932773701 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3664070c70 0x7f3664116100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:52:14.507 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:14.506+0000 7f3669e00700 1 -- 172.21.15.18:0/3932773701 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3664070c70 msgr2=0x7f3664116100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:52:14.508 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:14.506+0000 7f3669e00700 1 --2- 172.21.15.18:0/3932773701 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3664070c70 0x7f3664116100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:52:14.508 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:14.506+0000 7f3669e00700 1 -- 172.21.15.18:0/3932773701 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3654005040 con 0x7f3664070360 2024-04-12T21:52:14.509 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:14.506+0000 7f3669e00700 1 --2- 172.21.15.18:0/3932773701 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3664070360 0x7f3664115bc0 secure :-1 s=READY pgs=1337 cs=0 l=1 rev1=1 crypto rx=0x7f365800b790 tx=0x7f3658009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:52:14.509 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:14.507+0000 7f365e000700 1 -- 172.21.15.18:0/3932773701 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f365800f770 con 0x7f3664070360 2024-04-12T21:52:14.509 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:14.507+0000 7f365e000700 1 -- 172.21.15.18:0/3932773701 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f365800fda0 con 0x7f3664070360 2024-04-12T21:52:14.509 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:14.507+0000 7f365e000700 1 -- 172.21.15.18:0/3932773701 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f365800d310 con 0x7f3664070360 2024-04-12T21:52:14.509 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:14.507+0000 7f366c800700 1 -- 172.21.15.18:0/3932773701 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f366410ffb0 con 0x7f3664070360 2024-04-12T21:52:14.509 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:14.507+0000 7f366c800700 1 -- 172.21.15.18:0/3932773701 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f36641104c0 con 0x7f3664070360 2024-04-12T21:52:14.509 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:14.508+0000 7f366c800700 1 -- 172.21.15.18:0/3932773701 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3664071aa0 con 0x7f3664070360 2024-04-12T21:52:14.510 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:14.509+0000 7f365e000700 1 -- 172.21.15.18:0/3932773701 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7f3658010420 con 0x7f3664070360 2024-04-12T21:52:14.511 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:14.510+0000 7f365e000700 1 --2- 172.21.15.18:0/3932773701 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f364c06ec00 0x7f364c0710b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:52:14.511 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:14.510+0000 7f365e000700 1 -- 172.21.15.18:0/3932773701 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 6757+0+0 (secure 0 0 0) 0x7f365808e9a0 con 0x7f3664070360 2024-04-12T21:52:14.511 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:14.510+0000 7f3669400700 1 --2- 172.21.15.18:0/3932773701 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f364c06ec00 0x7f364c0710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:52:14.511 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:14.511+0000 7f3669400700 1 --2- 172.21.15.18:0/3932773701 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f364c06ec00 0x7f364c0710b0 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7f3654004f50 tx=0x7f3654009e30 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:52:14.516 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:14.515+0000 7f365e000700 1 -- 172.21.15.18:0/3932773701 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f365805b1d0 con 0x7f3664070360 2024-04-12T21:52:14.828 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:14.826+0000 7f366c800700 1 -- 172.21.15.18:0/3932773701 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f366404c710 con 0x7f3664070360 2024-04-12T21:52:14.832 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:14.830+0000 7f365e000700 1 -- 172.21.15.18:0/3932773701 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+4303 (secure 0 0 0) 0x7f365805ad60 con 0x7f3664070360 2024-04-12T21:52:14.832 INFO:teuthology.orchestra.run.smithi018.stdout:HEALTH_ERR failed to probe daemons or devices; 1 filesystem is degraded; 1 filesystem is offline 2024-04-12T21:52:14.832 INFO:teuthology.orchestra.run.smithi018.stdout:[WRN] CEPHADM_REFRESH_FAILED: failed to probe daemons or devices 2024-04-12T21:52:14.832 INFO:teuthology.orchestra.run.smithi018.stdout: host smithi186 `cephadm ceph-volume` failed: cephadm exited with an error code: 1, stderr: Inferring config /var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/mon.smithi186/config 2024-04-12T21:52:14.832 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/podman: timeout after 895 seconds 2024-04-12T21:52:14.833 INFO:teuthology.orchestra.run.smithi018.stdout:Non-zero exit code 124 from /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:f4452def3be4d350f4a8e5feaf7ebb1511be9298fe68478ba7a2d6fbd7d69a63 -e NODE_NAME=smithi186 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968:/var/run/ceph:z -v /var/log/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968:/var/log/ceph:z -v /var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmptgom53f9:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:f4452def3be4d350f4a8e5feaf7ebb1511be9298fe68478ba7a2d6fbd7d69a63 inventory --format=json-pretty --filter-for-batch 2024-04-12T21:52:14.833 INFO:teuthology.orchestra.run.smithi018.stdout:Traceback (most recent call last): 2024-04-12T21:52:14.833 INFO:teuthology.orchestra.run.smithi018.stdout: File "/usr/lib64/python3.9/runpy.py", line 197, in _run_module_as_main 2024-04-12T21:52:14.833 INFO:teuthology.orchestra.run.smithi018.stdout: return _run_code(code, main_globals, None, 2024-04-12T21:52:14.833 INFO:teuthology.orchestra.run.smithi018.stdout: File "/usr/lib64/python3.9/runpy.py", line 87, in _run_code 2024-04-12T21:52:14.833 INFO:teuthology.orchestra.run.smithi018.stdout: exec(code, run_globals) 2024-04-12T21:52:14.833 INFO:teuthology.orchestra.run.smithi018.stdout: File "/var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/cephadm.e44f3ed5e34e4ef52eedef233d1190bff004470d8e8005127b5312bff357fa7f/__main__.py", line 5618, in 2024-04-12T21:52:14.833 INFO:teuthology.orchestra.run.smithi018.stdout: File "/var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/cephadm.e44f3ed5e34e4ef52eedef233d1190bff004470d8e8005127b5312bff357fa7f/__main__.py", line 5606, in main 2024-04-12T21:52:14.833 INFO:teuthology.orchestra.run.smithi018.stdout: File "/var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/cephadm.e44f3ed5e34e4ef52eedef233d1190bff004470d8e8005127b5312bff357fa7f/__main__.py", line 408, in _infer_config 2024-04-12T21:52:14.833 INFO:teuthology.orchestra.run.smithi018.stdout: File "/var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/cephadm.e44f3ed5e34e4ef52eedef233d1190bff004470d8e8005127b5312bff357fa7f/__main__.py", line 323, in _infer_fsid 2024-04-12T21:52:14.833 INFO:teuthology.orchestra.run.smithi018.stdout: File "/var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/cephadm.e44f3ed5e34e4ef52eedef233d1190bff004470d8e8005127b5312bff357fa7f/__main__.py", line 436, in _infer_image 2024-04-12T21:52:14.833 INFO:teuthology.orchestra.run.smithi018.stdout: File "/var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/cephadm.e44f3ed5e34e4ef52eedef233d1190bff004470d8e8005127b5312bff357fa7f/__main__.py", line 310, in _validate_fsid 2024-04-12T21:52:14.834 INFO:teuthology.orchestra.run.smithi018.stdout: File "/var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/cephadm.e44f3ed5e34e4ef52eedef233d1190bff004470d8e8005127b5312bff357fa7f/__main__.py", line 3359, in command_ceph_volume 2024-04-12T21:52:14.834 INFO:teuthology.orchestra.run.smithi018.stdout: File "/var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/cephadm.e44f3ed5e34e4ef52eedef233d1190bff004470d8e8005127b5312bff357fa7f/cephadmlib/call_wrappers.py", line 310, in call_throws 2024-04-12T21:52:14.834 INFO:teuthology.orchestra.run.smithi018.stdout:RuntimeError: Failed command: /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:f4452def3be4d350f4a8e5feaf7ebb1511be9298fe68478ba7a2d6fbd7d69a63 -e NODE_NAME=smithi186 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968:/var/run/ceph:z -v /var/log/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968:/var/log/ceph:z -v /var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmptgom53f9:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:f4452def3be4d350f4a8e5feaf7ebb1511be9298fe68478ba7a2d6fbd7d69a63 inventory --format=json-pretty --filter-for-batch 2024-04-12T21:52:14.834 INFO:teuthology.orchestra.run.smithi018.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-04-12T21:52:14.834 INFO:teuthology.orchestra.run.smithi018.stdout: fs cephfs is degraded 2024-04-12T21:52:14.834 INFO:teuthology.orchestra.run.smithi018.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-04-12T21:52:14.834 INFO:teuthology.orchestra.run.smithi018.stdout: fs cephfs is offline because no MDS is active for it. 2024-04-12T21:52:14.835 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:14.832+0000 7f366c800700 1 -- 172.21.15.18:0/3932773701 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f364c06ec00 msgr2=0x7f364c0710b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:52:14.835 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:14.832+0000 7f366c800700 1 --2- 172.21.15.18:0/3932773701 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f364c06ec00 0x7f364c0710b0 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7f3654004f50 tx=0x7f3654009e30 comp rx=0 tx=0).stop 2024-04-12T21:52:14.835 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:14.832+0000 7f366c800700 1 -- 172.21.15.18:0/3932773701 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3664070360 msgr2=0x7f3664115bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:52:14.835 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:14.832+0000 7f366c800700 1 --2- 172.21.15.18:0/3932773701 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3664070360 0x7f3664115bc0 secure :-1 s=READY pgs=1337 cs=0 l=1 rev1=1 crypto rx=0x7f365800b790 tx=0x7f3658009250 comp rx=0 tx=0).stop 2024-04-12T21:52:14.835 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:14.832+0000 7f366c800700 1 -- 172.21.15.18:0/3932773701 shutdown_connections 2024-04-12T21:52:14.835 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:14.832+0000 7f366c800700 1 --2- 172.21.15.18:0/3932773701 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f364c06ec00 0x7f364c0710b0 unknown :-1 s=CLOSED pgs=708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:52:14.836 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:14.832+0000 7f366c800700 1 --2- 172.21.15.18:0/3932773701 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3664070360 0x7f3664115bc0 unknown :-1 s=CLOSED pgs=1337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:52:14.836 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:14.833+0000 7f366c800700 1 --2- 172.21.15.18:0/3932773701 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3664070c70 0x7f3664116100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:52:14.836 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:14.833+0000 7f366c800700 1 -- 172.21.15.18:0/3932773701 >> 172.21.15.18:0/3932773701 conn(0x7f36640f89b0 msgr2=0x7f36640f9e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:52:14.836 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:14.833+0000 7f366c800700 1 -- 172.21.15.18:0/3932773701 shutdown_connections 2024-04-12T21:52:14.836 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:14.833+0000 7f366c800700 1 -- 172.21.15.18:0/3932773701 wait complete. 2024-04-12T21:52:15.105 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:52:14 smithi018 ceph-mon[112590]: from='client.? 172.21.15.18:0/4207979881' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-12T21:52:15.285 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:52:14 smithi186 ceph-mon[86379]: from='client.? 172.21.15.18:0/4207979881' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-12T21:52:16.105 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:52:15 smithi018 ceph-mon[112590]: from='client.36740 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:52:16.106 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:52:15 smithi018 ceph-mon[112590]: pgmap v1577: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:52:16.106 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:52:15 smithi018 ceph-mon[112590]: from='client.? 172.21.15.18:0/3932773701' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-12T21:52:16.284 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:52:15 smithi186 ceph-mon[86379]: from='client.36740 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:52:16.285 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:52:15 smithi186 ceph-mon[86379]: pgmap v1577: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:52:16.285 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:52:15 smithi186 ceph-mon[86379]: from='client.? 172.21.15.18:0/3932773701' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-12T21:52:18.105 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:52:17 smithi018 ceph-mon[112590]: pgmap v1578: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:52:18.285 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:52:17 smithi186 ceph-mon[86379]: pgmap v1578: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:52:20.105 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:52:19 smithi018 ceph-mon[112590]: pgmap v1579: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:52:20.284 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:52:19 smithi186 ceph-mon[86379]: pgmap v1579: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:52:22.284 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:52:21 smithi186 ceph-mon[86379]: pgmap v1580: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:52:22.355 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:52:21 smithi018 ceph-mon[112590]: pgmap v1580: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:52:24.284 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:52:23 smithi186 ceph-mon[86379]: pgmap v1581: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:52:24.355 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:52:23 smithi018 ceph-mon[112590]: pgmap v1581: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:52:26.284 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:52:25 smithi186 ceph-mon[86379]: pgmap v1582: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:52:26.285 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:52:25 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T21:52:26.355 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:52:25 smithi018 ceph-mon[112590]: pgmap v1582: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:52:26.356 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:52:25 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T21:52:28.284 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:52:27 smithi186 ceph-mon[86379]: pgmap v1583: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:52:28.355 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:52:27 smithi018 ceph-mon[112590]: pgmap v1583: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:52:30.284 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:52:29 smithi186 ceph-mon[86379]: pgmap v1584: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:52:30.355 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:52:29 smithi018 ceph-mon[112590]: pgmap v1584: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:52:32.284 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:52:31 smithi186 ceph-mon[86379]: pgmap v1585: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:52:32.355 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:52:31 smithi018 ceph-mon[112590]: pgmap v1585: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:52:34.284 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:52:33 smithi186 ceph-mon[86379]: pgmap v1586: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:52:34.355 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:52:33 smithi018 ceph-mon[112590]: pgmap v1586: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:52:36.286 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:52:35 smithi186 ceph-mon[86379]: pgmap v1587: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:52:36.355 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:52:35 smithi018 ceph-mon[112590]: pgmap v1587: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:52:38.284 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:52:37 smithi186 ceph-mon[86379]: pgmap v1588: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:52:38.355 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:52:37 smithi018 ceph-mon[112590]: pgmap v1588: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:52:40.284 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:52:39 smithi186 ceph-mon[86379]: pgmap v1589: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:52:40.355 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:52:39 smithi018 ceph-mon[112590]: pgmap v1589: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:52:41.284 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:52:40 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T21:52:41.355 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:52:40 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T21:52:42.284 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:52:41 smithi186 ceph-mon[86379]: pgmap v1590: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:52:42.355 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:52:41 smithi018 ceph-mon[112590]: pgmap v1590: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:52:44.284 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:52:43 smithi186 ceph-mon[86379]: pgmap v1591: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:52:44.355 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:52:43 smithi018 ceph-mon[112590]: pgmap v1591: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:52:45.012 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:45.009+0000 7f3e31400700 1 -- 172.21.15.18:0/2039298006 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3e2c104e60 msgr2=0x7f3e2c105230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:52:45.012 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:45.009+0000 7f3e31400700 1 --2- 172.21.15.18:0/2039298006 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3e2c104e60 0x7f3e2c105230 secure :-1 s=READY pgs=1338 cs=0 l=1 rev1=1 crypto rx=0x7f3e1c004940 tx=0x7f3e1c009e30 comp rx=0 tx=0).stop 2024-04-12T21:52:45.012 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:45.009+0000 7f3e31400700 1 -- 172.21.15.18:0/2039298006 shutdown_connections 2024-04-12T21:52:45.012 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:45.009+0000 7f3e31400700 1 --2- 172.21.15.18:0/2039298006 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3e2c105800 0x7f3e2c0fee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:52:45.012 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:45.009+0000 7f3e31400700 1 --2- 172.21.15.18:0/2039298006 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3e2c104e60 0x7f3e2c105230 unknown :-1 s=CLOSED pgs=1338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:52:45.012 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:45.009+0000 7f3e31400700 1 -- 172.21.15.18:0/2039298006 >> 172.21.15.18:0/2039298006 conn(0x7f3e2c0fab10 msgr2=0x7f3e2c0fcef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:52:45.012 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:45.009+0000 7f3e31400700 1 -- 172.21.15.18:0/2039298006 shutdown_connections 2024-04-12T21:52:45.012 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:45.009+0000 7f3e31400700 1 -- 172.21.15.18:0/2039298006 wait complete. 2024-04-12T21:52:45.013 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:45.010+0000 7f3e31400700 1 Processor -- start 2024-04-12T21:52:45.013 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:45.011+0000 7f3e31400700 1 -- start start 2024-04-12T21:52:45.014 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:45.011+0000 7f3e31400700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3e2c104e60 0x7f3e2c115bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:52:45.014 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:45.011+0000 7f3e31400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3e2c105800 0x7f3e2c1160f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:52:45.014 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:45.011+0000 7f3e31400700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e2c116740 con 0x7f3e2c105800 2024-04-12T21:52:45.014 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:45.011+0000 7f3e31400700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e2c10fb30 con 0x7f3e2c104e60 2024-04-12T21:52:45.014 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:45.011+0000 7f3e2b400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3e2c105800 0x7f3e2c1160f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:52:45.014 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:45.011+0000 7f3e2b400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3e2c105800 0x7f3e2c1160f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:42170/0 (socket says 172.21.15.18:42170) 2024-04-12T21:52:45.014 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:45.011+0000 7f3e2b400700 1 -- 172.21.15.18:0/637468278 learned_addr learned my addr 172.21.15.18:0/637468278 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:52:45.014 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:45.012+0000 7f3e2be00700 1 --2- 172.21.15.18:0/637468278 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3e2c104e60 0x7f3e2c115bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:52:45.015 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:45.012+0000 7f3e2b400700 1 -- 172.21.15.18:0/637468278 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3e2c104e60 msgr2=0x7f3e2c115bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:52:45.015 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:45.012+0000 7f3e2b400700 1 --2- 172.21.15.18:0/637468278 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3e2c104e60 0x7f3e2c115bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:52:45.015 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:45.012+0000 7f3e2b400700 1 -- 172.21.15.18:0/637468278 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e1c005040 con 0x7f3e2c105800 2024-04-12T21:52:45.015 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:45.012+0000 7f3e2b400700 1 --2- 172.21.15.18:0/637468278 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3e2c105800 0x7f3e2c1160f0 secure :-1 s=READY pgs=1339 cs=0 l=1 rev1=1 crypto rx=0x7f3e18002a50 tx=0x7f3e18004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:52:45.015 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:45.013+0000 7f3e23e00700 1 -- 172.21.15.18:0/637468278 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3e18007cc0 con 0x7f3e2c105800 2024-04-12T21:52:45.015 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:45.013+0000 7f3e23e00700 1 -- 172.21.15.18:0/637468278 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f3e1800f040 con 0x7f3e2c105800 2024-04-12T21:52:45.015 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:45.013+0000 7f3e23e00700 1 -- 172.21.15.18:0/637468278 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3e1800daf0 con 0x7f3e2c105800 2024-04-12T21:52:45.015 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:45.013+0000 7f3e31400700 1 -- 172.21.15.18:0/637468278 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3e2c10fe10 con 0x7f3e2c105800 2024-04-12T21:52:45.016 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:45.013+0000 7f3e31400700 1 -- 172.21.15.18:0/637468278 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3e2c1103d0 con 0x7f3e2c105800 2024-04-12T21:52:45.016 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:45.014+0000 7f3e31400700 1 -- 172.21.15.18:0/637468278 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3e2c1005b0 con 0x7f3e2c105800 2024-04-12T21:52:45.017 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:45.016+0000 7f3e23e00700 1 -- 172.21.15.18:0/637468278 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7f3e18009810 con 0x7f3e2c105800 2024-04-12T21:52:45.019 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:45.017+0000 7f3e23e00700 1 --2- 172.21.15.18:0/637468278 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f3e1006ec00 0x7f3e100710b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:52:45.019 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:45.017+0000 7f3e23e00700 1 -- 172.21.15.18:0/637468278 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 6757+0+0 (secure 0 0 0) 0x7f3e1808e700 con 0x7f3e2c105800 2024-04-12T21:52:45.019 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:45.017+0000 7f3e2be00700 1 --2- 172.21.15.18:0/637468278 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f3e1006ec00 0x7f3e100710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:52:45.020 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:45.018+0000 7f3e2be00700 1 --2- 172.21.15.18:0/637468278 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f3e1006ec00 0x7f3e100710b0 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7f3e1c004f90 tx=0x7f3e1c009e30 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:52:45.024 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:45.022+0000 7f3e23e00700 1 -- 172.21.15.18:0/637468278 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f3e1805aeb0 con 0x7f3e2c105800 2024-04-12T21:52:45.302 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:45.300+0000 7f3e31400700 1 -- 172.21.15.18:0/637468278 --> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f3e2c10c000 con 0x7f3e1006ec00 2024-04-12T21:52:45.304 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:45.302+0000 7f3e23e00700 1 -- 172.21.15.18:0/637468278 <== mgr.34102 v2:172.21.15.18:6800/3812891919 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f3e1c00b5c0 con 0x7f3e1006ec00 2024-04-12T21:52:45.307 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:45.305+0000 7f3e31400700 1 -- 172.21.15.18:0/637468278 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f3e1006ec00 msgr2=0x7f3e100710b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:52:45.307 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:45.305+0000 7f3e31400700 1 --2- 172.21.15.18:0/637468278 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f3e1006ec00 0x7f3e100710b0 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7f3e1c004f90 tx=0x7f3e1c009e30 comp rx=0 tx=0).stop 2024-04-12T21:52:45.307 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:45.305+0000 7f3e31400700 1 -- 172.21.15.18:0/637468278 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3e2c105800 msgr2=0x7f3e2c1160f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:52:45.307 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:45.305+0000 7f3e31400700 1 --2- 172.21.15.18:0/637468278 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3e2c105800 0x7f3e2c1160f0 secure :-1 s=READY pgs=1339 cs=0 l=1 rev1=1 crypto rx=0x7f3e18002a50 tx=0x7f3e18004600 comp rx=0 tx=0).stop 2024-04-12T21:52:45.307 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:45.305+0000 7f3e31400700 1 -- 172.21.15.18:0/637468278 shutdown_connections 2024-04-12T21:52:45.308 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:45.305+0000 7f3e31400700 1 --2- 172.21.15.18:0/637468278 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f3e1006ec00 0x7f3e100710b0 unknown :-1 s=CLOSED pgs=710 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:52:45.308 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:45.305+0000 7f3e31400700 1 --2- 172.21.15.18:0/637468278 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3e2c104e60 0x7f3e2c115bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:52:45.308 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:45.305+0000 7f3e31400700 1 --2- 172.21.15.18:0/637468278 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3e2c105800 0x7f3e2c1160f0 unknown :-1 s=CLOSED pgs=1339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:52:45.308 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:45.305+0000 7f3e31400700 1 -- 172.21.15.18:0/637468278 >> 172.21.15.18:0/637468278 conn(0x7f3e2c0fab10 msgr2=0x7f3e2c0fca30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:52:45.308 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:45.305+0000 7f3e31400700 1 -- 172.21.15.18:0/637468278 shutdown_connections 2024-04-12T21:52:45.308 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:45.305+0000 7f3e31400700 1 -- 172.21.15.18:0/637468278 wait complete. 2024-04-12T21:52:45.323 INFO:teuthology.orchestra.run.smithi018.stdout:true 2024-04-12T21:52:45.469 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:45.467+0000 7f21c8e00700 1 -- 172.21.15.18:0/2718863062 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f21c4102710 msgr2=0x7f21c40fbd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:52:45.470 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:45.467+0000 7f21c8e00700 1 --2- 172.21.15.18:0/2718863062 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f21c4102710 0x7f21c40fbd80 secure :-1 s=READY pgs=1340 cs=0 l=1 rev1=1 crypto rx=0x7f21b0004940 tx=0x7f21b0009e30 comp rx=0 tx=0).stop 2024-04-12T21:52:45.470 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:45.467+0000 7f21c8e00700 1 -- 172.21.15.18:0/2718863062 shutdown_connections 2024-04-12T21:52:45.470 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:45.467+0000 7f21c8e00700 1 --2- 172.21.15.18:0/2718863062 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f21c4102710 0x7f21c40fbd80 unknown :-1 s=CLOSED pgs=1340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:52:45.470 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:45.467+0000 7f21c8e00700 1 --2- 172.21.15.18:0/2718863062 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f21c4101d70 0x7f21c4102140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:52:45.470 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:45.467+0000 7f21c8e00700 1 -- 172.21.15.18:0/2718863062 >> 172.21.15.18:0/2718863062 conn(0x7f21c4072bc0 msgr2=0x7f21c4072fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:52:45.470 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:45.467+0000 7f21c8e00700 1 -- 172.21.15.18:0/2718863062 shutdown_connections 2024-04-12T21:52:45.470 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:45.467+0000 7f21c8e00700 1 -- 172.21.15.18:0/2718863062 wait complete. 2024-04-12T21:52:45.472 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:45.468+0000 7f21c8e00700 1 Processor -- start 2024-04-12T21:52:45.472 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:45.468+0000 7f21c8e00700 1 -- start start 2024-04-12T21:52:45.472 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:45.469+0000 7f21c8e00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f21c4101d70 0x7f21c4115d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:52:45.472 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:45.469+0000 7f21c8e00700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f21c4102710 0x7f21c4116280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:52:45.472 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:45.469+0000 7f21c8e00700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21c41168d0 con 0x7f21c4101d70 2024-04-12T21:52:45.472 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:45.469+0000 7f21c8e00700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21c410fca0 con 0x7f21c4102710 2024-04-12T21:52:45.472 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:45.469+0000 7f21c1e00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f21c4101d70 0x7f21c4115d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:52:45.472 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:45.469+0000 7f21c1e00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f21c4101d70 0x7f21c4115d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:42180/0 (socket says 172.21.15.18:42180) 2024-04-12T21:52:45.473 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:45.469+0000 7f21c1e00700 1 -- 172.21.15.18:0/1979640676 learned_addr learned my addr 172.21.15.18:0/1979640676 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:52:45.473 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:45.469+0000 7f21c1400700 1 --2- 172.21.15.18:0/1979640676 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f21c4102710 0x7f21c4116280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:52:45.473 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:45.469+0000 7f21c1e00700 1 -- 172.21.15.18:0/1979640676 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f21c4102710 msgr2=0x7f21c4116280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:52:45.473 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:45.470+0000 7f21c1e00700 1 --2- 172.21.15.18:0/1979640676 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f21c4102710 0x7f21c4116280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:52:45.473 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:45.470+0000 7f21c1e00700 1 -- 172.21.15.18:0/1979640676 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21b0005040 con 0x7f21c4101d70 2024-04-12T21:52:45.473 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:45.470+0000 7f21c1400700 1 --2- 172.21.15.18:0/1979640676 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f21c4102710 0x7f21c4116280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-12T21:52:45.473 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:45.470+0000 7f21c1e00700 1 --2- 172.21.15.18:0/1979640676 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f21c4101d70 0x7f21c4115d40 secure :-1 s=READY pgs=1341 cs=0 l=1 rev1=1 crypto rx=0x7f21b4002a50 tx=0x7f21b40046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:52:45.474 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:45.470+0000 7f21ba000700 1 -- 172.21.15.18:0/1979640676 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f21b4007cc0 con 0x7f21c4101d70 2024-04-12T21:52:45.474 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:45.470+0000 7f21ba000700 1 -- 172.21.15.18:0/1979640676 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f21b400f040 con 0x7f21c4101d70 2024-04-12T21:52:45.474 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:45.470+0000 7f21c8e00700 1 -- 172.21.15.18:0/1979640676 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f21c410ff80 con 0x7f21c4101d70 2024-04-12T21:52:45.474 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:45.470+0000 7f21ba000700 1 -- 172.21.15.18:0/1979640676 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f21b4009640 con 0x7f21c4101d70 2024-04-12T21:52:45.474 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:45.470+0000 7f21c8e00700 1 -- 172.21.15.18:0/1979640676 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f21c4110540 con 0x7f21c4101d70 2024-04-12T21:52:45.474 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:45.471+0000 7f21c8e00700 1 -- 172.21.15.18:0/1979640676 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f21c40fd4c0 con 0x7f21c4101d70 2024-04-12T21:52:45.475 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:45.472+0000 7f21ba000700 1 -- 172.21.15.18:0/1979640676 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7f21b40097a0 con 0x7f21c4101d70 2024-04-12T21:52:45.475 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:45.473+0000 7f21ba000700 1 --2- 172.21.15.18:0/1979640676 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f21a806ec10 0x7f21a80710c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:52:45.476 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:45.474+0000 7f21ba000700 1 -- 172.21.15.18:0/1979640676 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 6757+0+0 (secure 0 0 0) 0x7f21b408e830 con 0x7f21c4101d70 2024-04-12T21:52:45.476 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:45.474+0000 7f21c1400700 1 --2- 172.21.15.18:0/1979640676 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f21a806ec10 0x7f21a80710c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:52:45.476 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:45.475+0000 7f21c1400700 1 --2- 172.21.15.18:0/1979640676 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f21a806ec10 0x7f21a80710c0 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7f21b000b3b0 tx=0x7f21b0009e30 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:52:45.480 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:45.479+0000 7f21ba000700 1 -- 172.21.15.18:0/1979640676 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f21b405af60 con 0x7f21c4101d70 2024-04-12T21:52:45.726 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:45.723+0000 7f21c8e00700 1 -- 172.21.15.18:0/1979640676 --> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f21c404bf40 con 0x7f21a806ec10 2024-04-12T21:52:45.728 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:45.726+0000 7f21ba000700 1 -- 172.21.15.18:0/1979640676 <== mgr.34102 v2:172.21.15.18:6800/3812891919 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f21c404bf40 con 0x7f21a806ec10 2024-04-12T21:52:45.731 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:45.728+0000 7f21c8e00700 1 -- 172.21.15.18:0/1979640676 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f21a806ec10 msgr2=0x7f21a80710c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:52:45.731 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:45.728+0000 7f21c8e00700 1 --2- 172.21.15.18:0/1979640676 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f21a806ec10 0x7f21a80710c0 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7f21b000b3b0 tx=0x7f21b0009e30 comp rx=0 tx=0).stop 2024-04-12T21:52:45.731 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:45.728+0000 7f21c8e00700 1 -- 172.21.15.18:0/1979640676 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f21c4101d70 msgr2=0x7f21c4115d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:52:45.731 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:45.728+0000 7f21c8e00700 1 --2- 172.21.15.18:0/1979640676 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f21c4101d70 0x7f21c4115d40 secure :-1 s=READY pgs=1341 cs=0 l=1 rev1=1 crypto rx=0x7f21b4002a50 tx=0x7f21b40046b0 comp rx=0 tx=0).stop 2024-04-12T21:52:45.731 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:45.728+0000 7f21c8e00700 1 -- 172.21.15.18:0/1979640676 shutdown_connections 2024-04-12T21:52:45.731 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:45.728+0000 7f21c8e00700 1 --2- 172.21.15.18:0/1979640676 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f21a806ec10 0x7f21a80710c0 unknown :-1 s=CLOSED pgs=711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:52:45.731 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:45.728+0000 7f21c8e00700 1 --2- 172.21.15.18:0/1979640676 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f21c4101d70 0x7f21c4115d40 unknown :-1 s=CLOSED pgs=1341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:52:45.731 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:45.728+0000 7f21c8e00700 1 --2- 172.21.15.18:0/1979640676 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f21c4102710 0x7f21c4116280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:52:45.731 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:45.728+0000 7f21c8e00700 1 -- 172.21.15.18:0/1979640676 >> 172.21.15.18:0/1979640676 conn(0x7f21c4072bc0 msgr2=0x7f21c40659a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:52:45.731 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:45.729+0000 7f21c8e00700 1 -- 172.21.15.18:0/1979640676 shutdown_connections 2024-04-12T21:52:45.731 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:45.729+0000 7f21c8e00700 1 -- 172.21.15.18:0/1979640676 wait complete. 2024-04-12T21:52:45.889 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:45.886+0000 7fdbe1c00700 1 -- 172.21.15.18:0/259356331 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fdbdc103780 msgr2=0x7fdbdc0fcdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:52:45.889 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:45.886+0000 7fdbe1c00700 1 --2- 172.21.15.18:0/259356331 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fdbdc103780 0x7fdbdc0fcdf0 secure :-1 s=READY pgs=1342 cs=0 l=1 rev1=1 crypto rx=0x7fdbc8004940 tx=0x7fdbc8009e30 comp rx=0 tx=0).stop 2024-04-12T21:52:45.889 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:45.887+0000 7fdbe1c00700 1 -- 172.21.15.18:0/259356331 shutdown_connections 2024-04-12T21:52:45.889 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:45.887+0000 7fdbe1c00700 1 --2- 172.21.15.18:0/259356331 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fdbdc103780 0x7fdbdc0fcdf0 unknown :-1 s=CLOSED pgs=1342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:52:45.889 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:45.887+0000 7fdbe1c00700 1 --2- 172.21.15.18:0/259356331 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdbdc102de0 0x7fdbdc1031b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:52:45.889 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:45.887+0000 7fdbe1c00700 1 -- 172.21.15.18:0/259356331 >> 172.21.15.18:0/259356331 conn(0x7fdbdc0f8970 msgr2=0x7fdbdc0fad80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:52:45.890 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:45.887+0000 7fdbe1c00700 1 -- 172.21.15.18:0/259356331 shutdown_connections 2024-04-12T21:52:45.890 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:45.887+0000 7fdbe1c00700 1 -- 172.21.15.18:0/259356331 wait complete. 2024-04-12T21:52:45.891 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:45.888+0000 7fdbe1c00700 1 Processor -- start 2024-04-12T21:52:45.891 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:45.888+0000 7fdbe1c00700 1 -- start start 2024-04-12T21:52:45.891 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:45.888+0000 7fdbe1c00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fdbdc102de0 0x7fdbdc115d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:52:45.891 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:45.888+0000 7fdbe1c00700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdbdc103780 0x7fdbdc116290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:52:45.891 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:45.888+0000 7fdbe1c00700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdbdc1168e0 con 0x7fdbdc102de0 2024-04-12T21:52:45.891 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:45.888+0000 7fdbe1c00700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdbdc10fcf0 con 0x7fdbdc103780 2024-04-12T21:52:45.891 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:45.888+0000 7fdbdaa00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fdbdc102de0 0x7fdbdc115d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:52:45.891 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:45.888+0000 7fdbdaa00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fdbdc102de0 0x7fdbdc115d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:42188/0 (socket says 172.21.15.18:42188) 2024-04-12T21:52:45.891 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:45.888+0000 7fdbdaa00700 1 -- 172.21.15.18:0/1102700265 learned_addr learned my addr 172.21.15.18:0/1102700265 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:52:45.891 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:45.889+0000 7fdbda000700 1 --2- 172.21.15.18:0/1102700265 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdbdc103780 0x7fdbdc116290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:52:45.892 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:45.889+0000 7fdbdaa00700 1 -- 172.21.15.18:0/1102700265 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdbdc103780 msgr2=0x7fdbdc116290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:52:45.892 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:45.889+0000 7fdbdaa00700 1 --2- 172.21.15.18:0/1102700265 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdbdc103780 0x7fdbdc116290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:52:45.892 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:45.889+0000 7fdbdaa00700 1 -- 172.21.15.18:0/1102700265 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdbc8005040 con 0x7fdbdc102de0 2024-04-12T21:52:45.892 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:45.889+0000 7fdbda000700 1 --2- 172.21.15.18:0/1102700265 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdbdc103780 0x7fdbdc116290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-12T21:52:45.892 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:45.889+0000 7fdbdaa00700 1 --2- 172.21.15.18:0/1102700265 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fdbdc102de0 0x7fdbdc115d50 secure :-1 s=READY pgs=1343 cs=0 l=1 rev1=1 crypto rx=0x7fdbcc002a50 tx=0x7fdbcc0046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:52:45.893 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:45.890+0000 7fdbd2a00700 1 -- 172.21.15.18:0/1102700265 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdbcc007cc0 con 0x7fdbdc102de0 2024-04-12T21:52:45.893 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:45.890+0000 7fdbd2a00700 1 -- 172.21.15.18:0/1102700265 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fdbcc00f040 con 0x7fdbdc102de0 2024-04-12T21:52:45.893 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:45.890+0000 7fdbd2a00700 1 -- 172.21.15.18:0/1102700265 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdbcc009640 con 0x7fdbdc102de0 2024-04-12T21:52:45.893 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:45.890+0000 7fdbe1c00700 1 -- 172.21.15.18:0/1102700265 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdbdc10ffd0 con 0x7fdbdc102de0 2024-04-12T21:52:45.893 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:45.890+0000 7fdbe1c00700 1 -- 172.21.15.18:0/1102700265 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdbdc110590 con 0x7fdbdc102de0 2024-04-12T21:52:45.893 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:45.891+0000 7fdbe1c00700 1 -- 172.21.15.18:0/1102700265 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdbdc0fe530 con 0x7fdbdc102de0 2024-04-12T21:52:45.894 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:45.892+0000 7fdbd2a00700 1 -- 172.21.15.18:0/1102700265 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7fdbcc0097a0 con 0x7fdbdc102de0 2024-04-12T21:52:45.895 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:45.893+0000 7fdbd2a00700 1 --2- 172.21.15.18:0/1102700265 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fdbc006ec10 0x7fdbc00710c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:52:45.895 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:45.893+0000 7fdbd2a00700 1 -- 172.21.15.18:0/1102700265 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 6757+0+0 (secure 0 0 0) 0x7fdbcc08e6c0 con 0x7fdbdc102de0 2024-04-12T21:52:45.895 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:45.894+0000 7fdbda000700 1 --2- 172.21.15.18:0/1102700265 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fdbc006ec10 0x7fdbc00710c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:52:45.896 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:45.894+0000 7fdbda000700 1 --2- 172.21.15.18:0/1102700265 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fdbc006ec10 0x7fdbc00710c0 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7fdbc800b3b0 tx=0x7fdbc8009e30 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:52:45.900 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:45.899+0000 7fdbd2a00700 1 -- 172.21.15.18:0/1102700265 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7fdbcc0573e0 con 0x7fdbdc102de0 2024-04-12T21:52:46.105 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:52:45 smithi018 ceph-mon[112590]: pgmap v1592: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:52:46.143 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:46.141+0000 7fdbe1c00700 1 -- 172.21.15.18:0/1102700265 --> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fdbdc04bf40 con 0x7fdbc006ec10 2024-04-12T21:52:46.172 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:46.170+0000 7fdbd2a00700 1 -- 172.21.15.18:0/1102700265 <== mgr.34102 v2:172.21.15.18:6800/3812891919 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7fdbdc04bf40 con 0x7fdbc006ec10 2024-04-12T21:52:46.173 INFO:teuthology.orchestra.run.smithi018.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-12T21:52:46.173 INFO:teuthology.orchestra.run.smithi018.stdout:alertmanager.smithi018 smithi018 *:9093,9094 running (53m) 14m ago 67m 14.8M - 0.25.0 c8568f914cd2 c332090068e8 2024-04-12T21:52:46.173 INFO:teuthology.orchestra.run.smithi018.stdout:crash.smithi018 smithi018 running (49m) 14m ago 68m 6665k - 19.0.0-2879-g324baf8d 8717720dc5e5 b7f80d1f7982 2024-04-12T21:52:46.173 INFO:teuthology.orchestra.run.smithi018.stdout:crash.smithi186 smithi186 running (49m) 14m ago 66m 6660k - 19.0.0-2879-g324baf8d 8717720dc5e5 a4f55c81b623 2024-04-12T21:52:46.173 INFO:teuthology.orchestra.run.smithi018.stdout:grafana.smithi018 smithi018 *:3000 running (52m) 14m ago 67m 73.8M - 9.4.12 2bacad6d85d8 b08e3bf176dd 2024-04-12T21:52:46.173 INFO:teuthology.orchestra.run.smithi018.stdout:mds.cephfs.smithi018.acqbsc smithi018 running (45m) 14m ago 60m 21.9M - 19.0.0-2879-g324baf8d 8717720dc5e5 021f99ab24ae 2024-04-12T21:52:46.173 INFO:teuthology.orchestra.run.smithi018.stdout:mds.cephfs.smithi018.szjluo smithi018 running (45m) 14m ago 60m 21.7M - 19.0.0-2879-g324baf8d 8717720dc5e5 0003117479d4 2024-04-12T21:52:46.174 INFO:teuthology.orchestra.run.smithi018.stdout:mds.cephfs.smithi186.jdllyn smithi186 running (29m) 14m ago 60m 16.5M - 19.0.0-2879-g324baf8d 8717720dc5e5 21b1f46eef23 2024-04-12T21:52:46.174 INFO:teuthology.orchestra.run.smithi018.stdout:mds.cephfs.smithi186.wjnafs smithi186 running (14m) 14m ago 60m 11.0M - 19.0.0-2879-g324baf8d 8717720dc5e5 3cf1ef4148e2 2024-04-12T21:52:46.174 INFO:teuthology.orchestra.run.smithi018.stdout:mgr.smithi018.xjzwga smithi018 *:8443,9283,8765 running (56m) 14m ago 69m 517M - 19.0.0-2879-g324baf8d 8717720dc5e5 56088305eab7 2024-04-12T21:52:46.174 INFO:teuthology.orchestra.run.smithi018.stdout:mgr.smithi186.kqcfxk smithi186 *:8443,9283,8765 running (54m) 14m ago 66m 460M - 19.0.0-2879-g324baf8d 8717720dc5e5 09cd766e5e0d 2024-04-12T21:52:46.174 INFO:teuthology.orchestra.run.smithi018.stdout:mon.smithi018 smithi018 running (51m) 14m ago 69m 88.2M 2048M 19.0.0-2879-g324baf8d 8717720dc5e5 df3e201dfed6 2024-04-12T21:52:46.174 INFO:teuthology.orchestra.run.smithi018.stdout:mon.smithi186 smithi186 running (50m) 14m ago 66m 53.6M 2048M 19.0.0-2879-g324baf8d 8717720dc5e5 c77498da9b2b 2024-04-12T21:52:46.174 INFO:teuthology.orchestra.run.smithi018.stdout:node-exporter.smithi018 smithi018 *:9100 running (54m) 14m ago 68m 8468k - 1.5.0 0da6a335fe13 326c3d9c6847 2024-04-12T21:52:46.174 INFO:teuthology.orchestra.run.smithi018.stdout:node-exporter.smithi186 smithi186 *:9100 running (54m) 14m ago 66m 8166k - 1.5.0 0da6a335fe13 ec2112baeecf 2024-04-12T21:52:46.174 INFO:teuthology.orchestra.run.smithi018.stdout:osd.0 smithi018 running (49m) 14m ago 65m 123M 1559M 19.0.0-2879-g324baf8d 8717720dc5e5 60b1273ca047 2024-04-12T21:52:46.174 INFO:teuthology.orchestra.run.smithi018.stdout:osd.1 smithi018 running (48m) 14m ago 64m 95.4M 1559M 19.0.0-2879-g324baf8d 8717720dc5e5 170ce1e5fb8d 2024-04-12T21:52:46.174 INFO:teuthology.orchestra.run.smithi018.stdout:osd.2 smithi018 running (47m) 14m ago 64m 94.6M 1559M 19.0.0-2879-g324baf8d 8717720dc5e5 c2cbc6aa2839 2024-04-12T21:52:46.174 INFO:teuthology.orchestra.run.smithi018.stdout:osd.3 smithi186 running (47m) 14m ago 63m 101M 2581M 19.0.0-2879-g324baf8d 8717720dc5e5 0138e8262e4b 2024-04-12T21:52:46.174 INFO:teuthology.orchestra.run.smithi018.stdout:osd.4 smithi186 running (46m) 14m ago 63m 91.5M 2581M 19.0.0-2879-g324baf8d 8717720dc5e5 b20cb63b25c9 2024-04-12T21:52:46.174 INFO:teuthology.orchestra.run.smithi018.stdout:osd.5 smithi186 running (46m) 14m ago 62m 92.2M 2581M 19.0.0-2879-g324baf8d 8717720dc5e5 2513bea2610e 2024-04-12T21:52:46.175 INFO:teuthology.orchestra.run.smithi018.stdout:prometheus.smithi018 smithi018 *:9095 running (53m) 14m ago 67m 69.6M - 2.43.0 a07b618ecd1d f58841127e21 2024-04-12T21:52:46.178 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:46.175+0000 7fdbe1c00700 1 -- 172.21.15.18:0/1102700265 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fdbc006ec10 msgr2=0x7fdbc00710c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:52:46.178 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:46.175+0000 7fdbe1c00700 1 --2- 172.21.15.18:0/1102700265 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fdbc006ec10 0x7fdbc00710c0 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7fdbc800b3b0 tx=0x7fdbc8009e30 comp rx=0 tx=0).stop 2024-04-12T21:52:46.178 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:46.176+0000 7fdbe1c00700 1 -- 172.21.15.18:0/1102700265 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fdbdc102de0 msgr2=0x7fdbdc115d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:52:46.178 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:46.176+0000 7fdbe1c00700 1 --2- 172.21.15.18:0/1102700265 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fdbdc102de0 0x7fdbdc115d50 secure :-1 s=READY pgs=1343 cs=0 l=1 rev1=1 crypto rx=0x7fdbcc002a50 tx=0x7fdbcc0046b0 comp rx=0 tx=0).stop 2024-04-12T21:52:46.178 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:46.176+0000 7fdbe1c00700 1 -- 172.21.15.18:0/1102700265 shutdown_connections 2024-04-12T21:52:46.179 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:46.176+0000 7fdbe1c00700 1 --2- 172.21.15.18:0/1102700265 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fdbc006ec10 0x7fdbc00710c0 unknown :-1 s=CLOSED pgs=712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:52:46.179 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:46.176+0000 7fdbe1c00700 1 --2- 172.21.15.18:0/1102700265 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fdbdc102de0 0x7fdbdc115d50 unknown :-1 s=CLOSED pgs=1343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:52:46.179 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:46.176+0000 7fdbe1c00700 1 --2- 172.21.15.18:0/1102700265 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdbdc103780 0x7fdbdc116290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:52:46.179 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:46.176+0000 7fdbe1c00700 1 -- 172.21.15.18:0/1102700265 >> 172.21.15.18:0/1102700265 conn(0x7fdbdc0f8970 msgr2=0x7fdbdc0f98e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:52:46.179 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:46.176+0000 7fdbe1c00700 1 -- 172.21.15.18:0/1102700265 shutdown_connections 2024-04-12T21:52:46.179 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:46.176+0000 7fdbe1c00700 1 -- 172.21.15.18:0/1102700265 wait complete. 2024-04-12T21:52:46.284 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:52:45 smithi186 ceph-mon[86379]: pgmap v1592: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:52:46.336 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:46.333+0000 7fd799200700 1 -- 172.21.15.18:0/2197023168 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd7940724f0 msgr2=0x7fd7940728c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:52:46.336 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:46.333+0000 7fd799200700 1 --2- 172.21.15.18:0/2197023168 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd7940724f0 0x7fd7940728c0 secure :-1 s=READY pgs=1344 cs=0 l=1 rev1=1 crypto rx=0x7fd784004940 tx=0x7fd784009e30 comp rx=0 tx=0).stop 2024-04-12T21:52:46.336 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:46.333+0000 7fd799200700 1 -- 172.21.15.18:0/2197023168 shutdown_connections 2024-04-12T21:52:46.336 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:46.333+0000 7fd799200700 1 --2- 172.21.15.18:0/2197023168 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd794072e00 0x7fd79410d5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:52:46.336 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:46.333+0000 7fd799200700 1 --2- 172.21.15.18:0/2197023168 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd7940724f0 0x7fd7940728c0 unknown :-1 s=CLOSED pgs=1344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:52:46.336 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:46.333+0000 7fd799200700 1 -- 172.21.15.18:0/2197023168 >> 172.21.15.18:0/2197023168 conn(0x7fd7940fab10 msgr2=0x7fd7940fcef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:52:46.336 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:46.333+0000 7fd799200700 1 -- 172.21.15.18:0/2197023168 shutdown_connections 2024-04-12T21:52:46.336 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:46.333+0000 7fd799200700 1 -- 172.21.15.18:0/2197023168 wait complete. 2024-04-12T21:52:46.337 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:46.334+0000 7fd799200700 1 Processor -- start 2024-04-12T21:52:46.337 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:46.335+0000 7fd799200700 1 -- start start 2024-04-12T21:52:46.337 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:46.335+0000 7fd799200700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd7940724f0 0x7fd794119f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:52:46.337 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:46.335+0000 7fd799200700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd794072e00 0x7fd79411a450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:52:46.338 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:46.335+0000 7fd799200700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd79411aaa0 con 0x7fd7940724f0 2024-04-12T21:52:46.338 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:46.335+0000 7fd799200700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd794113e70 con 0x7fd794072e00 2024-04-12T21:52:46.338 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:46.335+0000 7fd793400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd7940724f0 0x7fd794119f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:52:46.338 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:46.336+0000 7fd793400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd7940724f0 0x7fd794119f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:42210/0 (socket says 172.21.15.18:42210) 2024-04-12T21:52:46.338 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:46.336+0000 7fd793400700 1 -- 172.21.15.18:0/2513714215 learned_addr learned my addr 172.21.15.18:0/2513714215 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:52:46.339 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:46.336+0000 7fd792a00700 1 --2- 172.21.15.18:0/2513714215 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd794072e00 0x7fd79411a450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:52:46.339 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:46.336+0000 7fd793400700 1 -- 172.21.15.18:0/2513714215 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd794072e00 msgr2=0x7fd79411a450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:52:46.339 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:46.336+0000 7fd793400700 1 --2- 172.21.15.18:0/2513714215 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd794072e00 0x7fd79411a450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:52:46.339 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:46.336+0000 7fd793400700 1 -- 172.21.15.18:0/2513714215 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd784005040 con 0x7fd7940724f0 2024-04-12T21:52:46.339 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:46.336+0000 7fd792a00700 1 --2- 172.21.15.18:0/2513714215 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd794072e00 0x7fd79411a450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-12T21:52:46.339 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:46.336+0000 7fd793400700 1 --2- 172.21.15.18:0/2513714215 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd7940724f0 0x7fd794119f10 secure :-1 s=READY pgs=1345 cs=0 l=1 rev1=1 crypto rx=0x7fd784000c00 tx=0x7fd784009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:52:46.339 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:46.337+0000 7fd78b400700 1 -- 172.21.15.18:0/2513714215 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd784021070 con 0x7fd7940724f0 2024-04-12T21:52:46.339 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:46.337+0000 7fd78b400700 1 -- 172.21.15.18:0/2513714215 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd784013cd0 con 0x7fd7940724f0 2024-04-12T21:52:46.339 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:46.337+0000 7fd799200700 1 -- 172.21.15.18:0/2513714215 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd7941140f0 con 0x7fd7940724f0 2024-04-12T21:52:46.339 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:46.337+0000 7fd78b400700 1 -- 172.21.15.18:0/2513714215 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd78400e070 con 0x7fd7940724f0 2024-04-12T21:52:46.340 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:46.337+0000 7fd799200700 1 -- 172.21.15.18:0/2513714215 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd7941146b0 con 0x7fd7940724f0 2024-04-12T21:52:46.340 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:46.338+0000 7fd799200700 1 -- 172.21.15.18:0/2513714215 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd794073c30 con 0x7fd7940724f0 2024-04-12T21:52:46.341 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:46.339+0000 7fd78b400700 1 -- 172.21.15.18:0/2513714215 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7fd78401ca60 con 0x7fd7940724f0 2024-04-12T21:52:46.342 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:46.340+0000 7fd78b400700 1 --2- 172.21.15.18:0/2513714215 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fd77806ec10 0x7fd7780710c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:52:46.342 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:46.340+0000 7fd78b400700 1 -- 172.21.15.18:0/2513714215 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 6757+0+0 (secure 0 0 0) 0x7fd78408fd10 con 0x7fd7940724f0 2024-04-12T21:52:46.342 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:46.340+0000 7fd792a00700 1 --2- 172.21.15.18:0/2513714215 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fd77806ec10 0x7fd7780710c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:52:46.343 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:46.341+0000 7fd792a00700 1 --2- 172.21.15.18:0/2513714215 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fd77806ec10 0x7fd7780710c0 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7fd780004600 tx=0x7fd78000b040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:52:46.347 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:46.345+0000 7fd78b400700 1 -- 172.21.15.18:0/2513714215 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7fd78405c600 con 0x7fd7940724f0 2024-04-12T21:52:46.674 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:46.672+0000 7fd799200700 1 -- 172.21.15.18:0/2513714215 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fd79405f5f0 con 0x7fd7940724f0 2024-04-12T21:52:46.675 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:46.673+0000 7fd78b400700 1 -- 172.21.15.18:0/2513714215 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+623 (secure 0 0 0) 0x7fd78405c190 con 0x7fd7940724f0 2024-04-12T21:52:46.676 INFO:teuthology.orchestra.run.smithi018.stdout:{ 2024-04-12T21:52:46.676 INFO:teuthology.orchestra.run.smithi018.stdout: "mon": { 2024-04-12T21:52:46.676 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 19.0.0-2879-g324baf8d (324baf8ddb91dea46cac50cb535c306115791355) squid (dev)": 2 2024-04-12T21:52:46.676 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-12T21:52:46.676 INFO:teuthology.orchestra.run.smithi018.stdout: "mgr": { 2024-04-12T21:52:46.676 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 19.0.0-2879-g324baf8d (324baf8ddb91dea46cac50cb535c306115791355) squid (dev)": 2 2024-04-12T21:52:46.676 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-12T21:52:46.676 INFO:teuthology.orchestra.run.smithi018.stdout: "osd": { 2024-04-12T21:52:46.676 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 19.0.0-2879-g324baf8d (324baf8ddb91dea46cac50cb535c306115791355) squid (dev)": 6 2024-04-12T21:52:46.676 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-12T21:52:46.676 INFO:teuthology.orchestra.run.smithi018.stdout: "mds": { 2024-04-12T21:52:46.676 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 19.0.0-2879-g324baf8d (324baf8ddb91dea46cac50cb535c306115791355) squid (dev)": 4 2024-04-12T21:52:46.677 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-12T21:52:46.677 INFO:teuthology.orchestra.run.smithi018.stdout: "overall": { 2024-04-12T21:52:46.677 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 19.0.0-2879-g324baf8d (324baf8ddb91dea46cac50cb535c306115791355) squid (dev)": 14 2024-04-12T21:52:46.677 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-04-12T21:52:46.677 INFO:teuthology.orchestra.run.smithi018.stdout:} 2024-04-12T21:52:46.678 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:46.676+0000 7fd799200700 1 -- 172.21.15.18:0/2513714215 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fd77806ec10 msgr2=0x7fd7780710c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:52:46.678 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:46.676+0000 7fd799200700 1 --2- 172.21.15.18:0/2513714215 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fd77806ec10 0x7fd7780710c0 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7fd780004600 tx=0x7fd78000b040 comp rx=0 tx=0).stop 2024-04-12T21:52:46.678 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:46.676+0000 7fd799200700 1 -- 172.21.15.18:0/2513714215 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd7940724f0 msgr2=0x7fd794119f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:52:46.678 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:46.676+0000 7fd799200700 1 --2- 172.21.15.18:0/2513714215 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd7940724f0 0x7fd794119f10 secure :-1 s=READY pgs=1345 cs=0 l=1 rev1=1 crypto rx=0x7fd784000c00 tx=0x7fd784009e30 comp rx=0 tx=0).stop 2024-04-12T21:52:46.679 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:46.676+0000 7fd799200700 1 -- 172.21.15.18:0/2513714215 shutdown_connections 2024-04-12T21:52:46.679 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:46.676+0000 7fd799200700 1 --2- 172.21.15.18:0/2513714215 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fd77806ec10 0x7fd7780710c0 unknown :-1 s=CLOSED pgs=713 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:52:46.679 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:46.676+0000 7fd799200700 1 --2- 172.21.15.18:0/2513714215 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd7940724f0 0x7fd794119f10 unknown :-1 s=CLOSED pgs=1345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:52:46.679 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:46.676+0000 7fd799200700 1 --2- 172.21.15.18:0/2513714215 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd794072e00 0x7fd79411a450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:52:46.679 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:46.676+0000 7fd799200700 1 -- 172.21.15.18:0/2513714215 >> 172.21.15.18:0/2513714215 conn(0x7fd7940fab10 msgr2=0x7fd7940faf00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:52:46.679 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:46.676+0000 7fd799200700 1 -- 172.21.15.18:0/2513714215 shutdown_connections 2024-04-12T21:52:46.679 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:46.676+0000 7fd799200700 1 -- 172.21.15.18:0/2513714215 wait complete. 2024-04-12T21:52:46.825 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:46.822+0000 7fc25d600700 1 -- 172.21.15.18:0/6296457 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc258104e60 msgr2=0x7fc258105230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:52:46.825 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:46.822+0000 7fc25d600700 1 --2- 172.21.15.18:0/6296457 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc258104e60 0x7fc258105230 secure :-1 s=READY pgs=1346 cs=0 l=1 rev1=1 crypto rx=0x7fc248004940 tx=0x7fc248009e30 comp rx=0 tx=0).stop 2024-04-12T21:52:46.825 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:46.822+0000 7fc25d600700 1 -- 172.21.15.18:0/6296457 shutdown_connections 2024-04-12T21:52:46.825 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:46.822+0000 7fc25d600700 1 --2- 172.21.15.18:0/6296457 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc258105800 0x7fc2580fee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:52:46.825 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:46.822+0000 7fc25d600700 1 --2- 172.21.15.18:0/6296457 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc258104e60 0x7fc258105230 unknown :-1 s=CLOSED pgs=1346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:52:46.825 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:46.822+0000 7fc25d600700 1 -- 172.21.15.18:0/6296457 >> 172.21.15.18:0/6296457 conn(0x7fc2580fab10 msgr2=0x7fc2580fcef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:52:46.825 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:46.823+0000 7fc25d600700 1 -- 172.21.15.18:0/6296457 shutdown_connections 2024-04-12T21:52:46.825 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:46.823+0000 7fc25d600700 1 -- 172.21.15.18:0/6296457 wait complete. 2024-04-12T21:52:46.827 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:46.823+0000 7fc25d600700 1 Processor -- start 2024-04-12T21:52:46.827 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:46.824+0000 7fc25d600700 1 -- start start 2024-04-12T21:52:46.827 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:46.824+0000 7fc25d600700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc258104e60 0x7fc258115b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:52:46.827 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:46.824+0000 7fc25d600700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc258105800 0x7fc258116070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:52:46.827 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:46.824+0000 7fc25d600700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2581166c0 con 0x7fc258105800 2024-04-12T21:52:46.827 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:46.824+0000 7fc25d600700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc25810fab0 con 0x7fc258104e60 2024-04-12T21:52:46.827 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:46.824+0000 7fc257400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc258105800 0x7fc258116070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:52:46.827 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:46.824+0000 7fc257400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc258105800 0x7fc258116070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:42222/0 (socket says 172.21.15.18:42222) 2024-04-12T21:52:46.827 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:46.824+0000 7fc257400700 1 -- 172.21.15.18:0/1951980028 learned_addr learned my addr 172.21.15.18:0/1951980028 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:52:46.827 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:46.825+0000 7fc257e00700 1 --2- 172.21.15.18:0/1951980028 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc258104e60 0x7fc258115b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:52:46.827 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:46.825+0000 7fc257400700 1 -- 172.21.15.18:0/1951980028 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc258104e60 msgr2=0x7fc258115b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:52:46.827 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:46.825+0000 7fc257400700 1 --2- 172.21.15.18:0/1951980028 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc258104e60 0x7fc258115b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:52:46.828 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:46.825+0000 7fc257400700 1 -- 172.21.15.18:0/1951980028 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc248005040 con 0x7fc258105800 2024-04-12T21:52:46.828 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:46.825+0000 7fc257400700 1 --2- 172.21.15.18:0/1951980028 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc258105800 0x7fc258116070 secure :-1 s=READY pgs=1347 cs=0 l=1 rev1=1 crypto rx=0x7fc24400eb40 tx=0x7fc24400b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:52:46.828 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:46.825+0000 7fc24fe00700 1 -- 172.21.15.18:0/1951980028 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc244007cc0 con 0x7fc258105800 2024-04-12T21:52:46.829 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:46.825+0000 7fc24fe00700 1 -- 172.21.15.18:0/1951980028 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc24400f040 con 0x7fc258105800 2024-04-12T21:52:46.829 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:46.825+0000 7fc24fe00700 1 -- 172.21.15.18:0/1951980028 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc2440095e0 con 0x7fc258105800 2024-04-12T21:52:46.829 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:46.825+0000 7fc25d600700 1 -- 172.21.15.18:0/1951980028 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc25810fd30 con 0x7fc258105800 2024-04-12T21:52:46.829 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:46.825+0000 7fc25d600700 1 -- 172.21.15.18:0/1951980028 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc258110270 con 0x7fc258105800 2024-04-12T21:52:46.829 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:46.826+0000 7fc25d600700 1 -- 172.21.15.18:0/1951980028 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc2581005b0 con 0x7fc258105800 2024-04-12T21:52:46.830 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:46.828+0000 7fc24fe00700 1 -- 172.21.15.18:0/1951980028 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7fc244009740 con 0x7fc258105800 2024-04-12T21:52:46.831 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:46.829+0000 7fc24fe00700 1 --2- 172.21.15.18:0/1951980028 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fc23c06ec00 0x7fc23c0710b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:52:46.831 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:46.829+0000 7fc24fe00700 1 -- 172.21.15.18:0/1951980028 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 6757+0+0 (secure 0 0 0) 0x7fc24408e530 con 0x7fc258105800 2024-04-12T21:52:46.831 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:46.829+0000 7fc257e00700 1 --2- 172.21.15.18:0/1951980028 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fc23c06ec00 0x7fc23c0710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:52:46.832 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:46.830+0000 7fc257e00700 1 --2- 172.21.15.18:0/1951980028 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fc23c06ec00 0x7fc23c0710b0 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7fc248007f50 tx=0x7fc248009e30 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:52:46.836 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:46.834+0000 7fc24fe00700 1 -- 172.21.15.18:0/1951980028 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7fc24405ad10 con 0x7fc258105800 2024-04-12T21:52:47.107 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:47.105+0000 7fc25d600700 1 -- 172.21.15.18:0/1951980028 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fc25804bed0 con 0x7fc258105800 2024-04-12T21:52:47.108 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:47.106+0000 7fc24fe00700 1 -- 172.21.15.18:0/1951980028 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 33 v33) v1 ==== 76+0+1876 (secure 0 0 0) 0x7fc24405a8a0 con 0x7fc258105800 2024-04-12T21:52:47.109 INFO:teuthology.orchestra.run.smithi018.stdout:e33 2024-04-12T21:52:47.109 INFO:teuthology.orchestra.run.smithi018.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-12T21:52:47.109 INFO:teuthology.orchestra.run.smithi018.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-12T21:52:47.109 INFO:teuthology.orchestra.run.smithi018.stdout:legacy client fscid: 1 2024-04-12T21:52:47.109 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-12T21:52:47.109 INFO:teuthology.orchestra.run.smithi018.stdout:Filesystem 'cephfs' (1) 2024-04-12T21:52:47.109 INFO:teuthology.orchestra.run.smithi018.stdout:fs_name cephfs 2024-04-12T21:52:47.109 INFO:teuthology.orchestra.run.smithi018.stdout:epoch 24 2024-04-12T21:52:47.110 INFO:teuthology.orchestra.run.smithi018.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-04-12T21:52:47.110 INFO:teuthology.orchestra.run.smithi018.stdout:created 2024-04-12T20:51:57.362850+0000 2024-04-12T21:52:47.110 INFO:teuthology.orchestra.run.smithi018.stdout:modified 2024-04-12T21:06:59.717190+0000 2024-04-12T21:52:47.110 INFO:teuthology.orchestra.run.smithi018.stdout:tableserver 0 2024-04-12T21:52:47.110 INFO:teuthology.orchestra.run.smithi018.stdout:root 0 2024-04-12T21:52:47.110 INFO:teuthology.orchestra.run.smithi018.stdout:session_timeout 60 2024-04-12T21:52:47.110 INFO:teuthology.orchestra.run.smithi018.stdout:session_autoclose 300 2024-04-12T21:52:47.110 INFO:teuthology.orchestra.run.smithi018.stdout:max_file_size 1099511627776 2024-04-12T21:52:47.110 INFO:teuthology.orchestra.run.smithi018.stdout:max_xattr_size 65536 2024-04-12T21:52:47.110 INFO:teuthology.orchestra.run.smithi018.stdout:required_client_features {} 2024-04-12T21:52:47.110 INFO:teuthology.orchestra.run.smithi018.stdout:last_failure 0 2024-04-12T21:52:47.110 INFO:teuthology.orchestra.run.smithi018.stdout:last_failure_osd_epoch 100 2024-04-12T21:52:47.110 INFO:teuthology.orchestra.run.smithi018.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-12T21:52:47.110 INFO:teuthology.orchestra.run.smithi018.stdout:max_mds 2 2024-04-12T21:52:47.111 INFO:teuthology.orchestra.run.smithi018.stdout:in 0,1 2024-04-12T21:52:47.111 INFO:teuthology.orchestra.run.smithi018.stdout:up {} 2024-04-12T21:52:47.111 INFO:teuthology.orchestra.run.smithi018.stdout:failed 0,1 2024-04-12T21:52:47.111 INFO:teuthology.orchestra.run.smithi018.stdout:damaged 2024-04-12T21:52:47.111 INFO:teuthology.orchestra.run.smithi018.stdout:stopped 2024-04-12T21:52:47.111 INFO:teuthology.orchestra.run.smithi018.stdout:data_pools [3] 2024-04-12T21:52:47.111 INFO:teuthology.orchestra.run.smithi018.stdout:metadata_pool 2 2024-04-12T21:52:47.111 INFO:teuthology.orchestra.run.smithi018.stdout:inline_data disabled 2024-04-12T21:52:47.111 INFO:teuthology.orchestra.run.smithi018.stdout:balancer 2024-04-12T21:52:47.111 INFO:teuthology.orchestra.run.smithi018.stdout:bal_rank_mask -1 2024-04-12T21:52:47.111 INFO:teuthology.orchestra.run.smithi018.stdout:standby_count_wanted 1 2024-04-12T21:52:47.111 INFO:teuthology.orchestra.run.smithi018.stdout:qdb_cluster leader: 0 members: 2024-04-12T21:52:47.111 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-12T21:52:47.111 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-12T21:52:47.111 INFO:teuthology.orchestra.run.smithi018.stdout:Standby daemons: 2024-04-12T21:52:47.112 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-12T21:52:47.112 INFO:teuthology.orchestra.run.smithi018.stdout:[mds.cephfs.smithi018.acqbsc{-1:34436} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.18:6828/755179764,v1:172.21.15.18:6829/755179764] compat {c=[1],r=[1],i=[1fff]}] 2024-04-12T21:52:47.112 INFO:teuthology.orchestra.run.smithi018.stdout:[mds.cephfs.smithi018.szjluo{-1:34468} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.18:6826/2407992653,v1:172.21.15.18:6827/2407992653] compat {c=[1],r=[1],i=[1fff]}] 2024-04-12T21:52:47.112 INFO:teuthology.orchestra.run.smithi018.stdout:[mds.cephfs.smithi186.jdllyn{-1:44595} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.186:6824/3523444202,v1:172.21.15.186:6825/3523444202] compat {c=[1],r=[1],i=[1fff]}] 2024-04-12T21:52:47.112 INFO:teuthology.orchestra.run.smithi018.stdout:[mds.cephfs.smithi186.wjnafs{-1:44875} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.186:6826/3122443766,v1:172.21.15.186:6827/3122443766] compat {c=[1],r=[1],i=[1fff]}] 2024-04-12T21:52:47.113 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:47.109+0000 7fc25d600700 1 -- 172.21.15.18:0/1951980028 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fc23c06ec00 msgr2=0x7fc23c0710b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:52:47.113 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:47.109+0000 7fc25d600700 1 --2- 172.21.15.18:0/1951980028 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fc23c06ec00 0x7fc23c0710b0 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7fc248007f50 tx=0x7fc248009e30 comp rx=0 tx=0).stop 2024-04-12T21:52:47.113 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:47.109+0000 7fc25d600700 1 -- 172.21.15.18:0/1951980028 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc258105800 msgr2=0x7fc258116070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:52:47.113 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:47.109+0000 7fc25d600700 1 --2- 172.21.15.18:0/1951980028 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc258105800 0x7fc258116070 secure :-1 s=READY pgs=1347 cs=0 l=1 rev1=1 crypto rx=0x7fc24400eb40 tx=0x7fc24400b040 comp rx=0 tx=0).stop 2024-04-12T21:52:47.113 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:47.109+0000 7fc25d600700 1 -- 172.21.15.18:0/1951980028 shutdown_connections 2024-04-12T21:52:47.113 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:47.109+0000 7fc25d600700 1 --2- 172.21.15.18:0/1951980028 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fc23c06ec00 0x7fc23c0710b0 unknown :-1 s=CLOSED pgs=714 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:52:47.113 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:47.109+0000 7fc25d600700 1 --2- 172.21.15.18:0/1951980028 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc258104e60 0x7fc258115b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:52:47.113 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:47.109+0000 7fc25d600700 1 --2- 172.21.15.18:0/1951980028 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc258105800 0x7fc258116070 unknown :-1 s=CLOSED pgs=1347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:52:47.113 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:47.109+0000 7fc25d600700 1 -- 172.21.15.18:0/1951980028 >> 172.21.15.18:0/1951980028 conn(0x7fc2580fab10 msgr2=0x7fc2580fc9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:52:47.113 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:47.109+0000 7fc25d600700 1 -- 172.21.15.18:0/1951980028 shutdown_connections 2024-04-12T21:52:47.113 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:47.109+0000 7fc25d600700 1 -- 172.21.15.18:0/1951980028 wait complete. 2024-04-12T21:52:47.113 INFO:teuthology.orchestra.run.smithi018.stderr:dumped fsmap epoch 33 2024-04-12T21:52:47.259 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:47.257+0000 7feefea00700 1 -- 172.21.15.18:0/435780140 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7feef8102e00 msgr2=0x7feef81031d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:52:47.259 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:47.257+0000 7feefea00700 1 --2- 172.21.15.18:0/435780140 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7feef8102e00 0x7feef81031d0 secure :-1 s=READY pgs=1348 cs=0 l=1 rev1=1 crypto rx=0x7feee8004940 tx=0x7feee8009e30 comp rx=0 tx=0).stop 2024-04-12T21:52:47.259 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:47.257+0000 7feefea00700 1 -- 172.21.15.18:0/435780140 shutdown_connections 2024-04-12T21:52:47.259 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:47.257+0000 7feefea00700 1 --2- 172.21.15.18:0/435780140 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7feef81037a0 0x7feef80fce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:52:47.259 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:47.257+0000 7feefea00700 1 --2- 172.21.15.18:0/435780140 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7feef8102e00 0x7feef81031d0 unknown :-1 s=CLOSED pgs=1348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:52:47.260 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:47.257+0000 7feefea00700 1 -- 172.21.15.18:0/435780140 >> 172.21.15.18:0/435780140 conn(0x7feef80f89b0 msgr2=0x7feef80fadc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:52:47.260 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:52:46 smithi018 ceph-mon[112590]: from='client.36748 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:52:47.260 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:52:46 smithi018 ceph-mon[112590]: from='client.36752 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:52:47.260 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:52:46 smithi018 ceph-mon[112590]: from='client.36756 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:52:47.260 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:52:46 smithi018 ceph-mon[112590]: from='client.? 172.21.15.18:0/2513714215' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T21:52:47.262 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:47.257+0000 7feefea00700 1 -- 172.21.15.18:0/435780140 shutdown_connections 2024-04-12T21:52:47.262 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:47.257+0000 7feefea00700 1 -- 172.21.15.18:0/435780140 wait complete. 2024-04-12T21:52:47.262 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:47.258+0000 7feefea00700 1 Processor -- start 2024-04-12T21:52:47.262 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:47.258+0000 7feefea00700 1 -- start start 2024-04-12T21:52:47.262 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:47.258+0000 7feefea00700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7feef8102e00 0x7feef8115b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:52:47.262 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:47.258+0000 7feefea00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7feef81037a0 0x7feef8116040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:52:47.262 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:47.258+0000 7feefea00700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feef8116690 con 0x7feef81037a0 2024-04-12T21:52:47.263 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:47.258+0000 7feefea00700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feef810fcc0 con 0x7feef8102e00 2024-04-12T21:52:47.263 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:47.259+0000 7feef7400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7feef81037a0 0x7feef8116040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:52:47.263 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:47.259+0000 7feef7400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7feef81037a0 0x7feef8116040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:42240/0 (socket says 172.21.15.18:42240) 2024-04-12T21:52:47.263 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:47.259+0000 7feef7400700 1 -- 172.21.15.18:0/3344200600 learned_addr learned my addr 172.21.15.18:0/3344200600 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:52:47.263 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:47.259+0000 7feef7e00700 1 --2- 172.21.15.18:0/3344200600 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7feef8102e00 0x7feef8115b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:52:47.263 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:47.259+0000 7feef7400700 1 -- 172.21.15.18:0/3344200600 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7feef8102e00 msgr2=0x7feef8115b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:52:47.263 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:47.259+0000 7feef7400700 1 --2- 172.21.15.18:0/3344200600 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7feef8102e00 0x7feef8115b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:52:47.263 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:47.259+0000 7feef7400700 1 -- 172.21.15.18:0/3344200600 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feee8005040 con 0x7feef81037a0 2024-04-12T21:52:47.263 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:47.260+0000 7feef7400700 1 --2- 172.21.15.18:0/3344200600 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7feef81037a0 0x7feef8116040 secure :-1 s=READY pgs=1349 cs=0 l=1 rev1=1 crypto rx=0x7feee400eb40 tx=0x7feee400b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:52:47.263 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:47.260+0000 7feeefe00700 1 -- 172.21.15.18:0/3344200600 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7feee4007cc0 con 0x7feef81037a0 2024-04-12T21:52:47.263 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:47.260+0000 7feeefe00700 1 -- 172.21.15.18:0/3344200600 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7feee400f040 con 0x7feef81037a0 2024-04-12T21:52:47.263 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:47.260+0000 7feefea00700 1 -- 172.21.15.18:0/3344200600 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feef810ff40 con 0x7feef81037a0 2024-04-12T21:52:47.263 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:47.260+0000 7feeefe00700 1 -- 172.21.15.18:0/3344200600 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7feee40095e0 con 0x7feef81037a0 2024-04-12T21:52:47.263 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:47.260+0000 7feefea00700 1 -- 172.21.15.18:0/3344200600 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feef8110480 con 0x7feef81037a0 2024-04-12T21:52:47.264 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:47.262+0000 7feefea00700 1 -- 172.21.15.18:0/3344200600 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feef8106960 con 0x7feef81037a0 2024-04-12T21:52:47.265 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:47.263+0000 7feeefe00700 1 -- 172.21.15.18:0/3344200600 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7feee4009740 con 0x7feef81037a0 2024-04-12T21:52:47.267 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:47.264+0000 7feeefe00700 1 --2- 172.21.15.18:0/3344200600 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7feedc06ec00 0x7feedc0710b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:52:47.267 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:47.264+0000 7feeefe00700 1 -- 172.21.15.18:0/3344200600 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 6757+0+0 (secure 0 0 0) 0x7feee408e3d0 con 0x7feef81037a0 2024-04-12T21:52:47.268 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:47.264+0000 7feef7e00700 1 --2- 172.21.15.18:0/3344200600 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7feedc06ec00 0x7feedc0710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:52:47.268 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:47.265+0000 7feef7e00700 1 --2- 172.21.15.18:0/3344200600 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7feedc06ec00 0x7feedc0710b0 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7feee8007f50 tx=0x7feee8009e30 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:52:47.271 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:47.269+0000 7feeefe00700 1 -- 172.21.15.18:0/3344200600 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7feee405acd0 con 0x7feef81037a0 2024-04-12T21:52:47.284 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:52:46 smithi186 ceph-mon[86379]: from='client.36748 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:52:47.285 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:52:46 smithi186 ceph-mon[86379]: from='client.36752 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:52:47.285 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:52:46 smithi186 ceph-mon[86379]: from='client.36756 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:52:47.285 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:52:46 smithi186 ceph-mon[86379]: from='client.? 172.21.15.18:0/2513714215' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T21:52:47.519 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:47.516+0000 7feefea00700 1 -- 172.21.15.18:0/3344200600 --> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7feef81111c0 con 0x7feedc06ec00 2024-04-12T21:52:47.521 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:47.519+0000 7feeefe00700 1 -- 172.21.15.18:0/3344200600 <== mgr.34102 v2:172.21.15.18:6800/3812891919 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7feef81111c0 con 0x7feedc06ec00 2024-04-12T21:52:47.521 INFO:teuthology.orchestra.run.smithi018.stdout:{ 2024-04-12T21:52:47.521 INFO:teuthology.orchestra.run.smithi018.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:f4452def3be4d350f4a8e5feaf7ebb1511be9298fe68478ba7a2d6fbd7d69a63", 2024-04-12T21:52:47.521 INFO:teuthology.orchestra.run.smithi018.stdout: "in_progress": true, 2024-04-12T21:52:47.521 INFO:teuthology.orchestra.run.smithi018.stdout: "which": "Upgrading all daemon types on all hosts", 2024-04-12T21:52:47.521 INFO:teuthology.orchestra.run.smithi018.stdout: "services_complete": [ 2024-04-12T21:52:47.522 INFO:teuthology.orchestra.run.smithi018.stdout: "mgr", 2024-04-12T21:52:47.522 INFO:teuthology.orchestra.run.smithi018.stdout: "mds", 2024-04-12T21:52:47.522 INFO:teuthology.orchestra.run.smithi018.stdout: "crash", 2024-04-12T21:52:47.522 INFO:teuthology.orchestra.run.smithi018.stdout: "osd", 2024-04-12T21:52:47.522 INFO:teuthology.orchestra.run.smithi018.stdout: "mon" 2024-04-12T21:52:47.522 INFO:teuthology.orchestra.run.smithi018.stdout: ], 2024-04-12T21:52:47.522 INFO:teuthology.orchestra.run.smithi018.stdout: "progress": "16/21 daemons upgraded", 2024-04-12T21:52:47.522 INFO:teuthology.orchestra.run.smithi018.stdout: "message": "Currently upgrading mds daemons", 2024-04-12T21:52:47.522 INFO:teuthology.orchestra.run.smithi018.stdout: "is_paused": false 2024-04-12T21:52:47.522 INFO:teuthology.orchestra.run.smithi018.stdout:} 2024-04-12T21:52:47.524 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:47.522+0000 7feefea00700 1 -- 172.21.15.18:0/3344200600 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7feedc06ec00 msgr2=0x7feedc0710b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:52:47.524 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:47.522+0000 7feefea00700 1 --2- 172.21.15.18:0/3344200600 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7feedc06ec00 0x7feedc0710b0 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7feee8007f50 tx=0x7feee8009e30 comp rx=0 tx=0).stop 2024-04-12T21:52:47.524 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:47.522+0000 7feefea00700 1 -- 172.21.15.18:0/3344200600 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7feef81037a0 msgr2=0x7feef8116040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:52:47.524 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:47.522+0000 7feefea00700 1 --2- 172.21.15.18:0/3344200600 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7feef81037a0 0x7feef8116040 secure :-1 s=READY pgs=1349 cs=0 l=1 rev1=1 crypto rx=0x7feee400eb40 tx=0x7feee400b040 comp rx=0 tx=0).stop 2024-04-12T21:52:47.524 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:47.522+0000 7feefea00700 1 -- 172.21.15.18:0/3344200600 shutdown_connections 2024-04-12T21:52:47.524 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:47.522+0000 7feefea00700 1 --2- 172.21.15.18:0/3344200600 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7feedc06ec00 0x7feedc0710b0 unknown :-1 s=CLOSED pgs=715 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:52:47.524 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:47.522+0000 7feefea00700 1 --2- 172.21.15.18:0/3344200600 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7feef8102e00 0x7feef8115b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:52:47.524 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:47.522+0000 7feefea00700 1 --2- 172.21.15.18:0/3344200600 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7feef81037a0 0x7feef8116040 unknown :-1 s=CLOSED pgs=1349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:52:47.524 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:47.522+0000 7feefea00700 1 -- 172.21.15.18:0/3344200600 >> 172.21.15.18:0/3344200600 conn(0x7feef80f89b0 msgr2=0x7feef80fa4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:52:47.525 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:47.522+0000 7feefea00700 1 -- 172.21.15.18:0/3344200600 shutdown_connections 2024-04-12T21:52:47.525 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:47.522+0000 7feefea00700 1 -- 172.21.15.18:0/3344200600 wait complete. 2024-04-12T21:52:47.680 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:47.677+0000 7fb28d800700 1 -- 172.21.15.18:0/2408985897 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb288104e70 msgr2=0x7fb288105240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:52:47.680 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:47.677+0000 7fb28d800700 1 --2- 172.21.15.18:0/2408985897 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb288104e70 0x7fb288105240 secure :-1 s=READY pgs=1350 cs=0 l=1 rev1=1 crypto rx=0x7fb278004940 tx=0x7fb278009e30 comp rx=0 tx=0).stop 2024-04-12T21:52:47.680 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:47.677+0000 7fb28d800700 1 -- 172.21.15.18:0/2408985897 shutdown_connections 2024-04-12T21:52:47.680 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:47.677+0000 7fb28d800700 1 --2- 172.21.15.18:0/2408985897 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb288105810 0x7fb2880feed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:52:47.680 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:47.677+0000 7fb28d800700 1 --2- 172.21.15.18:0/2408985897 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb288104e70 0x7fb288105240 unknown :-1 s=CLOSED pgs=1350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:52:47.680 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:47.677+0000 7fb28d800700 1 -- 172.21.15.18:0/2408985897 >> 172.21.15.18:0/2408985897 conn(0x7fb2880fab10 msgr2=0x7fb2880fcef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:52:47.680 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:47.677+0000 7fb28d800700 1 -- 172.21.15.18:0/2408985897 shutdown_connections 2024-04-12T21:52:47.680 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:47.678+0000 7fb28d800700 1 -- 172.21.15.18:0/2408985897 wait complete. 2024-04-12T21:52:47.681 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:47.678+0000 7fb28d800700 1 Processor -- start 2024-04-12T21:52:47.681 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:47.679+0000 7fb28d800700 1 -- start start 2024-04-12T21:52:47.681 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:47.679+0000 7fb28d800700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb288104e70 0x7fb288115bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:52:47.681 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:47.679+0000 7fb28d800700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb288105810 0x7fb2881160f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:52:47.681 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:47.679+0000 7fb28d800700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb288116740 con 0x7fb288104e70 2024-04-12T21:52:47.682 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:47.679+0000 7fb28d800700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb28810fb50 con 0x7fb288105810 2024-04-12T21:52:47.682 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:47.679+0000 7fb287e00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb288104e70 0x7fb288115bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:52:47.682 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:47.679+0000 7fb287e00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb288104e70 0x7fb288115bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:42258/0 (socket says 172.21.15.18:42258) 2024-04-12T21:52:47.682 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:47.679+0000 7fb287e00700 1 -- 172.21.15.18:0/1190387960 learned_addr learned my addr 172.21.15.18:0/1190387960 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:52:47.683 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:47.680+0000 7fb27fe00700 1 --2- 172.21.15.18:0/1190387960 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb288105810 0x7fb2881160f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:52:47.683 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:47.680+0000 7fb287e00700 1 -- 172.21.15.18:0/1190387960 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb288105810 msgr2=0x7fb2881160f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:52:47.683 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:47.680+0000 7fb287e00700 1 --2- 172.21.15.18:0/1190387960 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb288105810 0x7fb2881160f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:52:47.683 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:47.680+0000 7fb287e00700 1 -- 172.21.15.18:0/1190387960 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb278005040 con 0x7fb288104e70 2024-04-12T21:52:47.683 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:47.680+0000 7fb287e00700 1 --2- 172.21.15.18:0/1190387960 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb288104e70 0x7fb288115bb0 secure :-1 s=READY pgs=1351 cs=0 l=1 rev1=1 crypto rx=0x7fb278009150 tx=0x7fb278009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:52:47.683 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:47.680+0000 7fb284c00700 1 -- 172.21.15.18:0/1190387960 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb27800e040 con 0x7fb288104e70 2024-04-12T21:52:47.683 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:47.680+0000 7fb284c00700 1 -- 172.21.15.18:0/1190387960 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb278013420 con 0x7fb288104e70 2024-04-12T21:52:47.683 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:47.680+0000 7fb28d800700 1 -- 172.21.15.18:0/1190387960 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb28810fdd0 con 0x7fb288104e70 2024-04-12T21:52:47.683 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:47.680+0000 7fb284c00700 1 -- 172.21.15.18:0/1190387960 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb27801c630 con 0x7fb288104e70 2024-04-12T21:52:47.684 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:47.681+0000 7fb28d800700 1 -- 172.21.15.18:0/1190387960 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb288110390 con 0x7fb288104e70 2024-04-12T21:52:47.684 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:47.682+0000 7fb28d800700 1 -- 172.21.15.18:0/1190387960 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb2881005c0 con 0x7fb288104e70 2024-04-12T21:52:47.685 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:47.683+0000 7fb284c00700 1 -- 172.21.15.18:0/1190387960 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7fb278013590 con 0x7fb288104e70 2024-04-12T21:52:47.686 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:47.684+0000 7fb284c00700 1 --2- 172.21.15.18:0/1190387960 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fb26c06ec00 0x7fb26c0710b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:52:47.686 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:47.684+0000 7fb284c00700 1 -- 172.21.15.18:0/1190387960 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 6757+0+0 (secure 0 0 0) 0x7fb27808fce0 con 0x7fb288104e70 2024-04-12T21:52:47.686 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:47.684+0000 7fb27fe00700 1 --2- 172.21.15.18:0/1190387960 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fb26c06ec00 0x7fb26c0710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:52:47.687 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:47.685+0000 7fb27fe00700 1 --2- 172.21.15.18:0/1190387960 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fb26c06ec00 0x7fb26c0710b0 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7fb288111450 tx=0x7fb27000b040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:52:47.691 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:47.689+0000 7fb284c00700 1 -- 172.21.15.18:0/1190387960 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7fb278058960 con 0x7fb288104e70 2024-04-12T21:52:48.004 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:48.002+0000 7fb28d800700 1 -- 172.21.15.18:0/1190387960 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7fb28804bed0 con 0x7fb288104e70 2024-04-12T21:52:48.005 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:48.003+0000 7fb284c00700 1 -- 172.21.15.18:0/1190387960 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+4303 (secure 0 0 0) 0x7fb27805c020 con 0x7fb288104e70 2024-04-12T21:52:48.005 INFO:teuthology.orchestra.run.smithi018.stdout:HEALTH_ERR failed to probe daemons or devices; 1 filesystem is degraded; 1 filesystem is offline 2024-04-12T21:52:48.005 INFO:teuthology.orchestra.run.smithi018.stdout:[WRN] CEPHADM_REFRESH_FAILED: failed to probe daemons or devices 2024-04-12T21:52:48.005 INFO:teuthology.orchestra.run.smithi018.stdout: host smithi186 `cephadm ceph-volume` failed: cephadm exited with an error code: 1, stderr: Inferring config /var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/mon.smithi186/config 2024-04-12T21:52:48.006 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/podman: timeout after 895 seconds 2024-04-12T21:52:48.006 INFO:teuthology.orchestra.run.smithi018.stdout:Non-zero exit code 124 from /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:f4452def3be4d350f4a8e5feaf7ebb1511be9298fe68478ba7a2d6fbd7d69a63 -e NODE_NAME=smithi186 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968:/var/run/ceph:z -v /var/log/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968:/var/log/ceph:z -v /var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmptgom53f9:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:f4452def3be4d350f4a8e5feaf7ebb1511be9298fe68478ba7a2d6fbd7d69a63 inventory --format=json-pretty --filter-for-batch 2024-04-12T21:52:48.006 INFO:teuthology.orchestra.run.smithi018.stdout:Traceback (most recent call last): 2024-04-12T21:52:48.006 INFO:teuthology.orchestra.run.smithi018.stdout: File "/usr/lib64/python3.9/runpy.py", line 197, in _run_module_as_main 2024-04-12T21:52:48.006 INFO:teuthology.orchestra.run.smithi018.stdout: return _run_code(code, main_globals, None, 2024-04-12T21:52:48.006 INFO:teuthology.orchestra.run.smithi018.stdout: File "/usr/lib64/python3.9/runpy.py", line 87, in _run_code 2024-04-12T21:52:48.006 INFO:teuthology.orchestra.run.smithi018.stdout: exec(code, run_globals) 2024-04-12T21:52:48.006 INFO:teuthology.orchestra.run.smithi018.stdout: File "/var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/cephadm.e44f3ed5e34e4ef52eedef233d1190bff004470d8e8005127b5312bff357fa7f/__main__.py", line 5618, in 2024-04-12T21:52:48.006 INFO:teuthology.orchestra.run.smithi018.stdout: File "/var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/cephadm.e44f3ed5e34e4ef52eedef233d1190bff004470d8e8005127b5312bff357fa7f/__main__.py", line 5606, in main 2024-04-12T21:52:48.006 INFO:teuthology.orchestra.run.smithi018.stdout: File "/var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/cephadm.e44f3ed5e34e4ef52eedef233d1190bff004470d8e8005127b5312bff357fa7f/__main__.py", line 408, in _infer_config 2024-04-12T21:52:48.006 INFO:teuthology.orchestra.run.smithi018.stdout: File "/var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/cephadm.e44f3ed5e34e4ef52eedef233d1190bff004470d8e8005127b5312bff357fa7f/__main__.py", line 323, in _infer_fsid 2024-04-12T21:52:48.006 INFO:teuthology.orchestra.run.smithi018.stdout: File "/var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/cephadm.e44f3ed5e34e4ef52eedef233d1190bff004470d8e8005127b5312bff357fa7f/__main__.py", line 436, in _infer_image 2024-04-12T21:52:48.006 INFO:teuthology.orchestra.run.smithi018.stdout: File "/var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/cephadm.e44f3ed5e34e4ef52eedef233d1190bff004470d8e8005127b5312bff357fa7f/__main__.py", line 310, in _validate_fsid 2024-04-12T21:52:48.007 INFO:teuthology.orchestra.run.smithi018.stdout: File "/var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/cephadm.e44f3ed5e34e4ef52eedef233d1190bff004470d8e8005127b5312bff357fa7f/__main__.py", line 3359, in command_ceph_volume 2024-04-12T21:52:48.007 INFO:teuthology.orchestra.run.smithi018.stdout: File "/var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/cephadm.e44f3ed5e34e4ef52eedef233d1190bff004470d8e8005127b5312bff357fa7f/cephadmlib/call_wrappers.py", line 310, in call_throws 2024-04-12T21:52:48.007 INFO:teuthology.orchestra.run.smithi018.stdout:RuntimeError: Failed command: /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:f4452def3be4d350f4a8e5feaf7ebb1511be9298fe68478ba7a2d6fbd7d69a63 -e NODE_NAME=smithi186 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968:/var/run/ceph:z -v /var/log/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968:/var/log/ceph:z -v /var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmptgom53f9:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:f4452def3be4d350f4a8e5feaf7ebb1511be9298fe68478ba7a2d6fbd7d69a63 inventory --format=json-pretty --filter-for-batch 2024-04-12T21:52:48.007 INFO:teuthology.orchestra.run.smithi018.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-04-12T21:52:48.007 INFO:teuthology.orchestra.run.smithi018.stdout: fs cephfs is degraded 2024-04-12T21:52:48.007 INFO:teuthology.orchestra.run.smithi018.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-04-12T21:52:48.007 INFO:teuthology.orchestra.run.smithi018.stdout: fs cephfs is offline because no MDS is active for it. 2024-04-12T21:52:48.008 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:48.005+0000 7fb28d800700 1 -- 172.21.15.18:0/1190387960 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fb26c06ec00 msgr2=0x7fb26c0710b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:52:48.008 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:48.005+0000 7fb28d800700 1 --2- 172.21.15.18:0/1190387960 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fb26c06ec00 0x7fb26c0710b0 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7fb288111450 tx=0x7fb27000b040 comp rx=0 tx=0).stop 2024-04-12T21:52:48.008 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:48.005+0000 7fb28d800700 1 -- 172.21.15.18:0/1190387960 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb288104e70 msgr2=0x7fb288115bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:52:48.008 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:48.005+0000 7fb28d800700 1 --2- 172.21.15.18:0/1190387960 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb288104e70 0x7fb288115bb0 secure :-1 s=READY pgs=1351 cs=0 l=1 rev1=1 crypto rx=0x7fb278009150 tx=0x7fb278009e30 comp rx=0 tx=0).stop 2024-04-12T21:52:48.008 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:48.006+0000 7fb28d800700 1 -- 172.21.15.18:0/1190387960 shutdown_connections 2024-04-12T21:52:48.008 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:48.006+0000 7fb28d800700 1 --2- 172.21.15.18:0/1190387960 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fb26c06ec00 0x7fb26c0710b0 unknown :-1 s=CLOSED pgs=716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:52:48.008 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:48.006+0000 7fb28d800700 1 --2- 172.21.15.18:0/1190387960 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb288104e70 0x7fb288115bb0 unknown :-1 s=CLOSED pgs=1351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:52:48.008 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:48.006+0000 7fb28d800700 1 --2- 172.21.15.18:0/1190387960 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb288105810 0x7fb2881160f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:52:48.009 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:48.006+0000 7fb28d800700 1 -- 172.21.15.18:0/1190387960 >> 172.21.15.18:0/1190387960 conn(0x7fb2880fab10 msgr2=0x7fb2880fca70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:52:48.009 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:48.006+0000 7fb28d800700 1 -- 172.21.15.18:0/1190387960 shutdown_connections 2024-04-12T21:52:48.009 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:52:48.006+0000 7fb28d800700 1 -- 172.21.15.18:0/1190387960 wait complete. 2024-04-12T21:52:48.284 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:52:47 smithi186 ceph-mon[86379]: pgmap v1593: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:52:48.285 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:52:47 smithi186 ceph-mon[86379]: from='client.? 172.21.15.18:0/1951980028' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-12T21:52:48.355 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:52:47 smithi018 ceph-mon[112590]: pgmap v1593: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:52:48.356 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:52:47 smithi018 ceph-mon[112590]: from='client.? 172.21.15.18:0/1951980028' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-12T21:52:49.284 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:52:48 smithi186 ceph-mon[86379]: from='client.36768 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:52:49.285 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:52:48 smithi186 ceph-mon[86379]: from='client.? 172.21.15.18:0/1190387960' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-12T21:52:49.355 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:52:48 smithi018 ceph-mon[112590]: from='client.36768 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:52:49.355 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:52:48 smithi018 ceph-mon[112590]: from='client.? 172.21.15.18:0/1190387960' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-12T21:52:50.284 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:52:49 smithi186 ceph-mon[86379]: pgmap v1594: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:52:50.355 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:52:49 smithi018 ceph-mon[112590]: pgmap v1594: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:52:52.284 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:52:51 smithi186 ceph-mon[86379]: pgmap v1595: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:52:52.355 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:52:51 smithi018 ceph-mon[112590]: pgmap v1595: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:52:54.284 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:52:53 smithi186 ceph-mon[86379]: pgmap v1596: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:52:54.356 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:52:53 smithi018 ceph-mon[112590]: pgmap v1596: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:52:55.489 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:52:55 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T21:52:55.489 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:52:55 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T21:52:56.284 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:52:56 smithi186 ceph-mon[86379]: pgmap v1597: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:52:56.355 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:52:56 smithi018 ceph-mon[112590]: pgmap v1597: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:52:58.284 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:52:58 smithi186 ceph-mon[86379]: pgmap v1598: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:52:58.355 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:52:58 smithi018 ceph-mon[112590]: pgmap v1598: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:53:00.285 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:53:00 smithi186 ceph-mon[86379]: pgmap v1599: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:53:00.355 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:53:00 smithi018 ceph-mon[112590]: pgmap v1599: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:53:02.284 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:53:02 smithi186 ceph-mon[86379]: pgmap v1600: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:53:02.355 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:53:02 smithi018 ceph-mon[112590]: pgmap v1600: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:53:04.356 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:53:04 smithi018 ceph-mon[112590]: pgmap v1601: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:53:04.534 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:53:04 smithi186 ceph-mon[86379]: pgmap v1601: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:53:06.355 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:53:06 smithi018 ceph-mon[112590]: pgmap v1602: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:53:06.535 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:53:06 smithi186 ceph-mon[86379]: pgmap v1602: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:53:08.355 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:53:08 smithi018 ceph-mon[112590]: pgmap v1603: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:53:08.534 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:53:08 smithi186 ceph-mon[86379]: pgmap v1603: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:53:10.355 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:53:10 smithi018 ceph-mon[112590]: pgmap v1604: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:53:10.355 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:53:10 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T21:53:10.535 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:53:10 smithi186 ceph-mon[86379]: pgmap v1604: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:53:10.535 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:53:10 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T21:53:12.355 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:53:12 smithi018 ceph-mon[112590]: pgmap v1605: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:53:12.534 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:53:12 smithi186 ceph-mon[86379]: pgmap v1605: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:53:14.355 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:53:14 smithi018 ceph-mon[112590]: pgmap v1606: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:53:14.535 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:53:14 smithi186 ceph-mon[86379]: pgmap v1606: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:53:16.355 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:53:16 smithi018 ceph-mon[112590]: pgmap v1607: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:53:16.534 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:53:16 smithi186 ceph-mon[86379]: pgmap v1607: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:53:18.179 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:53:18.177+0000 7f0c09600700 1 -- 172.21.15.18:0/2826470898 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0c04105800 msgr2=0x7f0c040fee70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:53:18.179 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:53:18.177+0000 7f0c09600700 1 --2- 172.21.15.18:0/2826470898 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0c04105800 0x7f0c040fee70 secure :-1 s=READY pgs=1352 cs=0 l=1 rev1=1 crypto rx=0x7f0bf0004940 tx=0x7f0bf0009e30 comp rx=0 tx=0).stop 2024-04-12T21:53:18.179 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:53:18.178+0000 7f0c09600700 1 -- 172.21.15.18:0/2826470898 shutdown_connections 2024-04-12T21:53:18.179 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:53:18.178+0000 7f0c09600700 1 --2- 172.21.15.18:0/2826470898 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0c04105800 0x7f0c040fee70 unknown :-1 s=CLOSED pgs=1352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:53:18.180 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:53:18.178+0000 7f0c09600700 1 --2- 172.21.15.18:0/2826470898 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0c04104e60 0x7f0c04105230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:53:18.180 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:53:18.178+0000 7f0c09600700 1 -- 172.21.15.18:0/2826470898 >> 172.21.15.18:0/2826470898 conn(0x7f0c040fab10 msgr2=0x7f0c040fcef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:53:18.180 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:53:18.178+0000 7f0c09600700 1 -- 172.21.15.18:0/2826470898 shutdown_connections 2024-04-12T21:53:18.180 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:53:18.178+0000 7f0c09600700 1 -- 172.21.15.18:0/2826470898 wait complete. 2024-04-12T21:53:18.180 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:53:18.179+0000 7f0c09600700 1 Processor -- start 2024-04-12T21:53:18.180 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:53:18.179+0000 7f0c09600700 1 -- start start 2024-04-12T21:53:18.181 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:53:18.180+0000 7f0c09600700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0c04104e60 0x7f0c04115bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:53:18.181 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:53:18.180+0000 7f0c09600700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0c04105800 0x7f0c041160f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:53:18.181 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:53:18.180+0000 7f0c09600700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c04116740 con 0x7f0c04105800 2024-04-12T21:53:18.181 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:53:18.180+0000 7f0c09600700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c0410fb50 con 0x7f0c04104e60 2024-04-12T21:53:18.181 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:53:18.180+0000 7f0c03400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0c04105800 0x7f0c041160f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:53:18.181 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:53:18.180+0000 7f0c03400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0c04105800 0x7f0c041160f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:52610/0 (socket says 172.21.15.18:52610) 2024-04-12T21:53:18.181 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:53:18.180+0000 7f0c03400700 1 -- 172.21.15.18:0/2725022849 learned_addr learned my addr 172.21.15.18:0/2725022849 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:53:18.182 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:53:18.180+0000 7f0c03e00700 1 --2- 172.21.15.18:0/2725022849 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0c04104e60 0x7f0c04115bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:53:18.183 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:53:18.181+0000 7f0c03400700 1 -- 172.21.15.18:0/2725022849 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0c04104e60 msgr2=0x7f0c04115bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:53:18.183 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:53:18.181+0000 7f0c03400700 1 --2- 172.21.15.18:0/2725022849 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0c04104e60 0x7f0c04115bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:53:18.183 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:53:18.181+0000 7f0c03400700 1 -- 172.21.15.18:0/2725022849 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0bf0005040 con 0x7f0c04105800 2024-04-12T21:53:18.183 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:53:18.181+0000 7f0c03400700 1 --2- 172.21.15.18:0/2725022849 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0c04105800 0x7f0c041160f0 secure :-1 s=READY pgs=1353 cs=0 l=1 rev1=1 crypto rx=0x7f0bf0009180 tx=0x7f0bf0009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:53:18.183 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:53:18.181+0000 7f0bfbe00700 1 -- 172.21.15.18:0/2725022849 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0bf000e040 con 0x7f0c04105800 2024-04-12T21:53:18.183 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:53:18.181+0000 7f0bfbe00700 1 -- 172.21.15.18:0/2725022849 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f0bf000bbc0 con 0x7f0c04105800 2024-04-12T21:53:18.183 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:53:18.181+0000 7f0bfbe00700 1 -- 172.21.15.18:0/2725022849 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0bf001c3f0 con 0x7f0c04105800 2024-04-12T21:53:18.183 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:53:18.181+0000 7f0c09600700 1 -- 172.21.15.18:0/2725022849 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0c0410fdd0 con 0x7f0c04105800 2024-04-12T21:53:18.183 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:53:18.181+0000 7f0c09600700 1 -- 172.21.15.18:0/2725022849 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0c04110390 con 0x7f0c04105800 2024-04-12T21:53:18.183 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:53:18.183+0000 7f0c09600700 1 -- 172.21.15.18:0/2725022849 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0c04102170 con 0x7f0c04105800 2024-04-12T21:53:18.185 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:53:18.184+0000 7f0bfbe00700 1 -- 172.21.15.18:0/2725022849 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7f0bf000bd30 con 0x7f0c04105800 2024-04-12T21:53:18.186 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:53:18.185+0000 7f0bfbe00700 1 --2- 172.21.15.18:0/2725022849 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f0be806ec00 0x7f0be80710b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:53:18.186 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:53:18.185+0000 7f0bfbe00700 1 -- 172.21.15.18:0/2725022849 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 6757+0+0 (secure 0 0 0) 0x7f0bf008fd00 con 0x7f0c04105800 2024-04-12T21:53:18.186 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:53:18.185+0000 7f0c03e00700 1 --2- 172.21.15.18:0/2725022849 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f0be806ec00 0x7f0be80710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:53:18.187 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:53:18.186+0000 7f0c03e00700 1 --2- 172.21.15.18:0/2725022849 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f0be806ec00 0x7f0be80710b0 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7f0bf4007fd0 tx=0x7f0bf4009040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:53:18.191 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:53:18.190+0000 7f0bfbe00700 1 -- 172.21.15.18:0/2725022849 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f0bf0058a00 con 0x7f0c04105800 2024-04-12T21:53:18.355 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:53:18 smithi018 ceph-mon[112590]: pgmap v1608: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:53:18.439 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:53:18.437+0000 7f0c09600700 1 -- 172.21.15.18:0/2725022849 --> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f0c04111200 con 0x7f0be806ec00 2024-04-12T21:53:18.441 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:53:18.440+0000 7f0bfbe00700 1 -- 172.21.15.18:0/2725022849 <== mgr.34102 v2:172.21.15.18:6800/3812891919 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f0c04111200 con 0x7f0be806ec00 2024-04-12T21:53:18.444 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:53:18.442+0000 7f0c09600700 1 -- 172.21.15.18:0/2725022849 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f0be806ec00 msgr2=0x7f0be80710b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:53:18.444 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:53:18.443+0000 7f0c09600700 1 --2- 172.21.15.18:0/2725022849 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f0be806ec00 0x7f0be80710b0 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7f0bf4007fd0 tx=0x7f0bf4009040 comp rx=0 tx=0).stop 2024-04-12T21:53:18.444 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:53:18.443+0000 7f0c09600700 1 -- 172.21.15.18:0/2725022849 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0c04105800 msgr2=0x7f0c041160f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:53:18.444 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:53:18.443+0000 7f0c09600700 1 --2- 172.21.15.18:0/2725022849 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0c04105800 0x7f0c041160f0 secure :-1 s=READY pgs=1353 cs=0 l=1 rev1=1 crypto rx=0x7f0bf0009180 tx=0x7f0bf0009e30 comp rx=0 tx=0).stop 2024-04-12T21:53:18.445 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:53:18.443+0000 7f0c09600700 1 -- 172.21.15.18:0/2725022849 shutdown_connections 2024-04-12T21:53:18.445 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:53:18.443+0000 7f0c09600700 1 --2- 172.21.15.18:0/2725022849 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f0be806ec00 0x7f0be80710b0 unknown :-1 s=CLOSED pgs=718 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:53:18.445 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:53:18.443+0000 7f0c09600700 1 --2- 172.21.15.18:0/2725022849 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0c04104e60 0x7f0c04115bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:53:18.445 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:53:18.443+0000 7f0c09600700 1 --2- 172.21.15.18:0/2725022849 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0c04105800 0x7f0c041160f0 unknown :-1 s=CLOSED pgs=1353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:53:18.445 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:53:18.443+0000 7f0c09600700 1 -- 172.21.15.18:0/2725022849 >> 172.21.15.18:0/2725022849 conn(0x7f0c040fab10 msgr2=0x7f0c040fd8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:53:18.445 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:53:18.443+0000 7f0c09600700 1 -- 172.21.15.18:0/2725022849 shutdown_connections 2024-04-12T21:53:18.445 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:53:18.443+0000 7f0c09600700 1 -- 172.21.15.18:0/2725022849 wait complete. 2024-04-12T21:53:18.460 INFO:teuthology.orchestra.run.smithi018.stdout:true 2024-04-12T21:53:18.534 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:53:18 smithi186 ceph-mon[86379]: pgmap v1608: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:53:18.592 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:53:18.590+0000 7f8bf2200700 1 -- 172.21.15.18:0/2379138883 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8bec104ff0 msgr2=0x7f8bec1053c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:53:18.592 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:53:18.591+0000 7f8bf2200700 1 --2- 172.21.15.18:0/2379138883 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8bec104ff0 0x7f8bec1053c0 secure :-1 s=READY pgs=1354 cs=0 l=1 rev1=1 crypto rx=0x7f8bdc004940 tx=0x7f8bdc009e30 comp rx=0 tx=0).stop 2024-04-12T21:53:18.592 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:53:18.591+0000 7f8bf2200700 1 -- 172.21.15.18:0/2379138883 shutdown_connections 2024-04-12T21:53:18.592 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:53:18.591+0000 7f8bf2200700 1 --2- 172.21.15.18:0/2379138883 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8bec105990 0x7f8bec0ff000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:53:18.593 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:53:18.591+0000 7f8bf2200700 1 --2- 172.21.15.18:0/2379138883 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8bec104ff0 0x7f8bec1053c0 unknown :-1 s=CLOSED pgs=1354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:53:18.593 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:53:18.591+0000 7f8bf2200700 1 -- 172.21.15.18:0/2379138883 >> 172.21.15.18:0/2379138883 conn(0x7f8bec0faba0 msgr2=0x7f8bec0fcfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:53:18.593 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:53:18.591+0000 7f8bf2200700 1 -- 172.21.15.18:0/2379138883 shutdown_connections 2024-04-12T21:53:18.593 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:53:18.591+0000 7f8bf2200700 1 -- 172.21.15.18:0/2379138883 wait complete. 2024-04-12T21:53:18.594 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:53:18.592+0000 7f8bf2200700 1 Processor -- start 2024-04-12T21:53:18.594 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:53:18.592+0000 7f8bf2200700 1 -- start start 2024-04-12T21:53:18.594 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:53:18.592+0000 7f8bf2200700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8bec104ff0 0x7f8bec115b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:53:18.594 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:53:18.592+0000 7f8bf2200700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8bec105990 0x7f8bec1160a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:53:18.594 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:53:18.592+0000 7f8bf2200700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8bec1166f0 con 0x7f8bec105990 2024-04-12T21:53:18.595 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:53:18.592+0000 7f8bf2200700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8bec10fcd0 con 0x7f8bec104ff0 2024-04-12T21:53:18.595 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:53:18.592+0000 7f8be3e00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8bec105990 0x7f8bec1160a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:53:18.595 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:53:18.593+0000 7f8be3e00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8bec105990 0x7f8bec1160a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:52620/0 (socket says 172.21.15.18:52620) 2024-04-12T21:53:18.595 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:53:18.593+0000 7f8be3e00700 1 -- 172.21.15.18:0/2255393884 learned_addr learned my addr 172.21.15.18:0/2255393884 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:53:18.595 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:53:18.593+0000 7f8beb400700 1 --2- 172.21.15.18:0/2255393884 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8bec104ff0 0x7f8bec115b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:53:18.595 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:53:18.593+0000 7f8be3e00700 1 -- 172.21.15.18:0/2255393884 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8bec104ff0 msgr2=0x7f8bec115b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:53:18.595 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:53:18.593+0000 7f8be3e00700 1 --2- 172.21.15.18:0/2255393884 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8bec104ff0 0x7f8bec115b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:53:18.595 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:53:18.593+0000 7f8be3e00700 1 -- 172.21.15.18:0/2255393884 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8bdc005040 con 0x7f8bec105990 2024-04-12T21:53:18.595 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:53:18.593+0000 7f8be3e00700 1 --2- 172.21.15.18:0/2255393884 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8bec105990 0x7f8bec1160a0 secure :-1 s=READY pgs=1355 cs=0 l=1 rev1=1 crypto rx=0x7f8bd400c800 tx=0x7f8bd4004470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:53:18.596 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:53:18.594+0000 7f8be3400700 1 -- 172.21.15.18:0/2255393884 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8bd400a760 con 0x7f8bec105990 2024-04-12T21:53:18.596 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:53:18.594+0000 7f8be3400700 1 -- 172.21.15.18:0/2255393884 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f8bd400ad90 con 0x7f8bec105990 2024-04-12T21:53:18.596 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:53:18.594+0000 7f8be3400700 1 -- 172.21.15.18:0/2255393884 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8bd40113a0 con 0x7f8bec105990 2024-04-12T21:53:18.596 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:53:18.594+0000 7f8bf2200700 1 -- 172.21.15.18:0/2255393884 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8bec10ffb0 con 0x7f8bec105990 2024-04-12T21:53:18.596 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:53:18.594+0000 7f8bf2200700 1 -- 172.21.15.18:0/2255393884 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8bec110570 con 0x7f8bec105990 2024-04-12T21:53:18.596 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:53:18.595+0000 7f8bf2200700 1 -- 172.21.15.18:0/2255393884 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8bec100740 con 0x7f8bec105990 2024-04-12T21:53:18.597 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:53:18.596+0000 7f8be3400700 1 -- 172.21.15.18:0/2255393884 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7f8bd4012400 con 0x7f8bec105990 2024-04-12T21:53:18.598 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:53:18.597+0000 7f8be3400700 1 --2- 172.21.15.18:0/2255393884 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f8bd006ec00 0x7f8bd00710b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:53:18.598 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:53:18.597+0000 7f8be3400700 1 -- 172.21.15.18:0/2255393884 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 6757+0+0 (secure 0 0 0) 0x7f8bd408ea40 con 0x7f8bec105990 2024-04-12T21:53:18.598 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:53:18.597+0000 7f8beb400700 1 --2- 172.21.15.18:0/2255393884 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f8bd006ec00 0x7f8bd00710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:53:18.599 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:53:18.598+0000 7f8beb400700 1 --2- 172.21.15.18:0/2255393884 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f8bd006ec00 0x7f8bd00710b0 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7f8bdc004f90 tx=0x7f8bdc009e30 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:53:18.603 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:53:18.602+0000 7f8be3400700 1 -- 172.21.15.18:0/2255393884 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f8bd405b1f0 con 0x7f8bec105990 2024-04-12T21:53:18.844 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:53:18.842+0000 7f8bf2200700 1 -- 172.21.15.18:0/2255393884 --> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f8bec10c160 con 0x7f8bd006ec00 2024-04-12T21:53:18.847 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:53:18.846+0000 7f8be3400700 1 -- 172.21.15.18:0/2255393884 <== mgr.34102 v2:172.21.15.18:6800/3812891919 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f8bec10c160 con 0x7f8bd006ec00 2024-04-12T21:53:18.850 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:53:18.849+0000 7f8bf2200700 1 -- 172.21.15.18:0/2255393884 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f8bd006ec00 msgr2=0x7f8bd00710b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:53:18.851 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:53:18.849+0000 7f8bf2200700 1 --2- 172.21.15.18:0/2255393884 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f8bd006ec00 0x7f8bd00710b0 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7f8bdc004f90 tx=0x7f8bdc009e30 comp rx=0 tx=0).stop 2024-04-12T21:53:18.851 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:53:18.849+0000 7f8bf2200700 1 -- 172.21.15.18:0/2255393884 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8bec105990 msgr2=0x7f8bec1160a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:53:18.851 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:53:18.849+0000 7f8bf2200700 1 --2- 172.21.15.18:0/2255393884 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8bec105990 0x7f8bec1160a0 secure :-1 s=READY pgs=1355 cs=0 l=1 rev1=1 crypto rx=0x7f8bd400c800 tx=0x7f8bd4004470 comp rx=0 tx=0).stop 2024-04-12T21:53:18.851 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:53:18.849+0000 7f8bf2200700 1 -- 172.21.15.18:0/2255393884 shutdown_connections 2024-04-12T21:53:18.851 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:53:18.849+0000 7f8bf2200700 1 --2- 172.21.15.18:0/2255393884 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f8bd006ec00 0x7f8bd00710b0 unknown :-1 s=CLOSED pgs=719 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:53:18.851 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:53:18.849+0000 7f8bf2200700 1 --2- 172.21.15.18:0/2255393884 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8bec104ff0 0x7f8bec115b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:53:18.851 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:53:18.849+0000 7f8bf2200700 1 --2- 172.21.15.18:0/2255393884 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8bec105990 0x7f8bec1160a0 unknown :-1 s=CLOSED pgs=1355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:53:18.851 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:53:18.849+0000 7f8bf2200700 1 -- 172.21.15.18:0/2255393884 >> 172.21.15.18:0/2255393884 conn(0x7f8bec0faba0 msgr2=0x7f8bec0fbb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:53:18.851 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:53:18.849+0000 7f8bf2200700 1 -- 172.21.15.18:0/2255393884 shutdown_connections 2024-04-12T21:53:18.851 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:53:18.849+0000 7f8bf2200700 1 -- 172.21.15.18:0/2255393884 wait complete. 2024-04-12T21:53:18.998 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:53:18.996+0000 7fd287e00700 1 -- 172.21.15.18:0/1445022464 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd2881035d0 msgr2=0x7fd2880fcc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:53:18.998 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:53:18.996+0000 7fd287e00700 1 --2- 172.21.15.18:0/1445022464 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd2881035d0 0x7fd2880fcc40 secure :-1 s=READY pgs=1356 cs=0 l=1 rev1=1 crypto rx=0x7fd274004940 tx=0x7fd274009e30 comp rx=0 tx=0).stop 2024-04-12T21:53:18.998 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:53:18.996+0000 7fd287e00700 1 -- 172.21.15.18:0/1445022464 shutdown_connections 2024-04-12T21:53:18.998 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:53:18.996+0000 7fd287e00700 1 --2- 172.21.15.18:0/1445022464 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd2881035d0 0x7fd2880fcc40 unknown :-1 s=CLOSED pgs=1356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:53:18.998 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:53:18.996+0000 7fd287e00700 1 --2- 172.21.15.18:0/1445022464 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd288102c30 0x7fd288103000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:53:18.998 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:53:18.996+0000 7fd287e00700 1 -- 172.21.15.18:0/1445022464 >> 172.21.15.18:0/1445022464 conn(0x7fd2880f8760 msgr2=0x7fd2880fab70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:53:18.998 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:53:18.997+0000 7fd287e00700 1 -- 172.21.15.18:0/1445022464 shutdown_connections 2024-04-12T21:53:18.999 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:53:18.997+0000 7fd287e00700 1 -- 172.21.15.18:0/1445022464 wait complete. 2024-04-12T21:53:19.001 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:53:18.997+0000 7fd287e00700 1 Processor -- start 2024-04-12T21:53:19.001 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:53:18.998+0000 7fd287e00700 1 -- start start 2024-04-12T21:53:19.001 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:53:18.998+0000 7fd287e00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd288102c30 0x7fd288115ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:53:19.001 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:53:18.998+0000 7fd287e00700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd2881035d0 0x7fd2881160e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:53:19.001 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:53:18.998+0000 7fd287e00700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd288116730 con 0x7fd288102c30 2024-04-12T21:53:19.001 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:53:18.998+0000 7fd287e00700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd28810fb40 con 0x7fd2881035d0 2024-04-12T21:53:19.001 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:53:18.998+0000 7fd286a00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd288102c30 0x7fd288115ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:53:19.001 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:53:18.998+0000 7fd286a00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd288102c30 0x7fd288115ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:52630/0 (socket says 172.21.15.18:52630) 2024-04-12T21:53:19.001 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:53:18.998+0000 7fd286a00700 1 -- 172.21.15.18:0/4248947067 learned_addr learned my addr 172.21.15.18:0/4248947067 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:53:19.001 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:53:18.999+0000 7fd286000700 1 --2- 172.21.15.18:0/4248947067 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd2881035d0 0x7fd2881160e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:53:19.002 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:53:18.999+0000 7fd286a00700 1 -- 172.21.15.18:0/4248947067 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd2881035d0 msgr2=0x7fd2881160e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:53:19.002 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:53:18.999+0000 7fd286a00700 1 --2- 172.21.15.18:0/4248947067 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd2881035d0 0x7fd2881160e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:53:19.002 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:53:18.999+0000 7fd286a00700 1 -- 172.21.15.18:0/4248947067 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd274005040 con 0x7fd288102c30 2024-04-12T21:53:19.002 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:53:18.999+0000 7fd286000700 1 --2- 172.21.15.18:0/4248947067 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd2881035d0 0x7fd2881160e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-12T21:53:19.002 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:53:18.999+0000 7fd286a00700 1 --2- 172.21.15.18:0/4248947067 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd288102c30 0x7fd288115ba0 secure :-1 s=READY pgs=1357 cs=0 l=1 rev1=1 crypto rx=0x7fd278002a50 tx=0x7fd2780046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:53:19.002 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:53:18.999+0000 7fd27ea00700 1 -- 172.21.15.18:0/4248947067 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd278007cc0 con 0x7fd288102c30 2024-04-12T21:53:19.002 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:53:18.999+0000 7fd27ea00700 1 -- 172.21.15.18:0/4248947067 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd27800f040 con 0x7fd288102c30 2024-04-12T21:53:19.002 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:53:18.999+0000 7fd27ea00700 1 -- 172.21.15.18:0/4248947067 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd278009640 con 0x7fd288102c30 2024-04-12T21:53:19.002 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:53:18.999+0000 7fd287e00700 1 -- 172.21.15.18:0/4248947067 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd28810fe20 con 0x7fd288102c30 2024-04-12T21:53:19.002 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:53:19.000+0000 7fd287e00700 1 -- 172.21.15.18:0/4248947067 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd2881103e0 con 0x7fd288102c30 2024-04-12T21:53:19.003 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:53:19.001+0000 7fd287e00700 1 -- 172.21.15.18:0/4248947067 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd2880fe380 con 0x7fd288102c30 2024-04-12T21:53:19.003 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:53:19.002+0000 7fd27ea00700 1 -- 172.21.15.18:0/4248947067 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7fd2780097a0 con 0x7fd288102c30 2024-04-12T21:53:19.004 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:53:19.003+0000 7fd27ea00700 1 --2- 172.21.15.18:0/4248947067 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fd26c06ec00 0x7fd26c0710b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:53:19.004 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:53:19.003+0000 7fd27ea00700 1 -- 172.21.15.18:0/4248947067 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 6757+0+0 (secure 0 0 0) 0x7fd27808e6c0 con 0x7fd288102c30 2024-04-12T21:53:19.004 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:53:19.003+0000 7fd286000700 1 --2- 172.21.15.18:0/4248947067 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fd26c06ec00 0x7fd26c0710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:53:19.005 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:53:19.004+0000 7fd286000700 1 --2- 172.21.15.18:0/4248947067 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fd26c06ec00 0x7fd26c0710b0 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7fd27400b3b0 tx=0x7fd274009e30 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:53:19.009 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:53:19.008+0000 7fd27ea00700 1 -- 172.21.15.18:0/4248947067 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7fd2780573e0 con 0x7fd288102c30 2024-04-12T21:53:19.241 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:53:19.240+0000 7fd287e00700 1 -- 172.21.15.18:0/4248947067 --> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fd28804bed0 con 0x7fd26c06ec00 2024-04-12T21:53:19.271 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:53:19.270+0000 7fd27ea00700 1 -- 172.21.15.18:0/4248947067 <== mgr.34102 v2:172.21.15.18:6800/3812891919 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7fd28804bed0 con 0x7fd26c06ec00 2024-04-12T21:53:19.271 INFO:teuthology.orchestra.run.smithi018.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-12T21:53:19.272 INFO:teuthology.orchestra.run.smithi018.stdout:alertmanager.smithi018 smithi018 *:9093,9094 running (53m) 14m ago 68m 14.8M - 0.25.0 c8568f914cd2 c332090068e8 2024-04-12T21:53:19.272 INFO:teuthology.orchestra.run.smithi018.stdout:crash.smithi018 smithi018 running (50m) 14m ago 68m 6665k - 19.0.0-2879-g324baf8d 8717720dc5e5 b7f80d1f7982 2024-04-12T21:53:19.272 INFO:teuthology.orchestra.run.smithi018.stdout:crash.smithi186 smithi186 running (50m) 15m ago 67m 6660k - 19.0.0-2879-g324baf8d 8717720dc5e5 a4f55c81b623 2024-04-12T21:53:19.272 INFO:teuthology.orchestra.run.smithi018.stdout:grafana.smithi018 smithi018 *:3000 running (53m) 14m ago 68m 73.8M - 9.4.12 2bacad6d85d8 b08e3bf176dd 2024-04-12T21:53:19.272 INFO:teuthology.orchestra.run.smithi018.stdout:mds.cephfs.smithi018.acqbsc smithi018 running (46m) 14m ago 61m 21.9M - 19.0.0-2879-g324baf8d 8717720dc5e5 021f99ab24ae 2024-04-12T21:53:19.272 INFO:teuthology.orchestra.run.smithi018.stdout:mds.cephfs.smithi018.szjluo smithi018 running (45m) 14m ago 61m 21.7M - 19.0.0-2879-g324baf8d 8717720dc5e5 0003117479d4 2024-04-12T21:53:19.272 INFO:teuthology.orchestra.run.smithi018.stdout:mds.cephfs.smithi186.jdllyn smithi186 running (30m) 15m ago 61m 16.5M - 19.0.0-2879-g324baf8d 8717720dc5e5 21b1f46eef23 2024-04-12T21:53:19.272 INFO:teuthology.orchestra.run.smithi018.stdout:mds.cephfs.smithi186.wjnafs smithi186 running (15m) 15m ago 61m 11.0M - 19.0.0-2879-g324baf8d 8717720dc5e5 3cf1ef4148e2 2024-04-12T21:53:19.272 INFO:teuthology.orchestra.run.smithi018.stdout:mgr.smithi018.xjzwga smithi018 *:8443,9283,8765 running (56m) 14m ago 70m 517M - 19.0.0-2879-g324baf8d 8717720dc5e5 56088305eab7 2024-04-12T21:53:19.272 INFO:teuthology.orchestra.run.smithi018.stdout:mgr.smithi186.kqcfxk smithi186 *:8443,9283,8765 running (55m) 15m ago 67m 460M - 19.0.0-2879-g324baf8d 8717720dc5e5 09cd766e5e0d 2024-04-12T21:53:19.272 INFO:teuthology.orchestra.run.smithi018.stdout:mon.smithi018 smithi018 running (51m) 14m ago 70m 88.2M 2048M 19.0.0-2879-g324baf8d 8717720dc5e5 df3e201dfed6 2024-04-12T21:53:19.272 INFO:teuthology.orchestra.run.smithi018.stdout:mon.smithi186 smithi186 running (51m) 15m ago 66m 53.6M 2048M 19.0.0-2879-g324baf8d 8717720dc5e5 c77498da9b2b 2024-04-12T21:53:19.272 INFO:teuthology.orchestra.run.smithi018.stdout:node-exporter.smithi018 smithi018 *:9100 running (54m) 14m ago 68m 8468k - 1.5.0 0da6a335fe13 326c3d9c6847 2024-04-12T21:53:19.272 INFO:teuthology.orchestra.run.smithi018.stdout:node-exporter.smithi186 smithi186 *:9100 running (54m) 15m ago 67m 8166k - 1.5.0 0da6a335fe13 ec2112baeecf 2024-04-12T21:53:19.273 INFO:teuthology.orchestra.run.smithi018.stdout:osd.0 smithi018 running (49m) 14m ago 66m 123M 1559M 19.0.0-2879-g324baf8d 8717720dc5e5 60b1273ca047 2024-04-12T21:53:19.273 INFO:teuthology.orchestra.run.smithi018.stdout:osd.1 smithi018 running (49m) 14m ago 65m 95.4M 1559M 19.0.0-2879-g324baf8d 8717720dc5e5 170ce1e5fb8d 2024-04-12T21:53:19.273 INFO:teuthology.orchestra.run.smithi018.stdout:osd.2 smithi018 running (48m) 14m ago 64m 94.6M 1559M 19.0.0-2879-g324baf8d 8717720dc5e5 c2cbc6aa2839 2024-04-12T21:53:19.273 INFO:teuthology.orchestra.run.smithi018.stdout:osd.3 smithi186 running (47m) 15m ago 64m 101M 2581M 19.0.0-2879-g324baf8d 8717720dc5e5 0138e8262e4b 2024-04-12T21:53:19.273 INFO:teuthology.orchestra.run.smithi018.stdout:osd.4 smithi186 running (47m) 15m ago 63m 91.5M 2581M 19.0.0-2879-g324baf8d 8717720dc5e5 b20cb63b25c9 2024-04-12T21:53:19.273 INFO:teuthology.orchestra.run.smithi018.stdout:osd.5 smithi186 running (46m) 15m ago 63m 92.2M 2581M 19.0.0-2879-g324baf8d 8717720dc5e5 2513bea2610e 2024-04-12T21:53:19.273 INFO:teuthology.orchestra.run.smithi018.stdout:prometheus.smithi018 smithi018 *:9095 running (54m) 14m ago 67m 69.6M - 2.43.0 a07b618ecd1d f58841127e21 2024-04-12T21:53:19.273 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:53:19.272+0000 7fd287e00700 1 -- 172.21.15.18:0/4248947067 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fd26c06ec00 msgr2=0x7fd26c0710b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:53:19.274 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:53:19.272+0000 7fd287e00700 1 --2- 172.21.15.18:0/4248947067 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fd26c06ec00 0x7fd26c0710b0 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7fd27400b3b0 tx=0x7fd274009e30 comp rx=0 tx=0).stop 2024-04-12T21:53:19.274 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:53:19.272+0000 7fd287e00700 1 -- 172.21.15.18:0/4248947067 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd288102c30 msgr2=0x7fd288115ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:53:19.274 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:53:19.272+0000 7fd287e00700 1 --2- 172.21.15.18:0/4248947067 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd288102c30 0x7fd288115ba0 secure :-1 s=READY pgs=1357 cs=0 l=1 rev1=1 crypto rx=0x7fd278002a50 tx=0x7fd2780046b0 comp rx=0 tx=0).stop 2024-04-12T21:53:19.274 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:53:19.273+0000 7fd287e00700 1 -- 172.21.15.18:0/4248947067 shutdown_connections 2024-04-12T21:53:19.274 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:53:19.273+0000 7fd287e00700 1 --2- 172.21.15.18:0/4248947067 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fd26c06ec00 0x7fd26c0710b0 unknown :-1 s=CLOSED pgs=720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:53:19.274 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:53:19.273+0000 7fd287e00700 1 --2- 172.21.15.18:0/4248947067 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd288102c30 0x7fd288115ba0 unknown :-1 s=CLOSED pgs=1357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:53:19.274 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:53:19.273+0000 7fd287e00700 1 --2- 172.21.15.18:0/4248947067 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd2881035d0 0x7fd2881160e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:53:19.274 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:53:19.273+0000 7fd287e00700 1 -- 172.21.15.18:0/4248947067 >> 172.21.15.18:0/4248947067 conn(0x7fd2880f8760 msgr2=0x7fd2880f9780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:53:19.275 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:53:19.273+0000 7fd287e00700 1 -- 172.21.15.18:0/4248947067 shutdown_connections 2024-04-12T21:53:19.275 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:53:19.273+0000 7fd287e00700 1 -- 172.21.15.18:0/4248947067 wait complete. 2024-04-12T21:53:19.421 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:53:19.419+0000 7efe14e00700 1 -- 172.21.15.18:0/3783134372 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7efe100fe800 msgr2=0x7efe10106e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:53:19.421 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:53:19.419+0000 7efe14e00700 1 --2- 172.21.15.18:0/3783134372 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7efe100fe800 0x7efe10106e40 secure :-1 s=READY pgs=1358 cs=0 l=1 rev1=1 crypto rx=0x7efdfc004940 tx=0x7efdfc009e30 comp rx=0 tx=0).stop 2024-04-12T21:53:19.421 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:53:19.419+0000 7efe14e00700 1 -- 172.21.15.18:0/3783134372 shutdown_connections 2024-04-12T21:53:19.421 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:53:19.419+0000 7efe14e00700 1 --2- 172.21.15.18:0/3783134372 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7efe100fe800 0x7efe10106e40 unknown :-1 s=CLOSED pgs=1358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:53:19.421 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:53:19.419+0000 7efe14e00700 1 --2- 172.21.15.18:0/3783134372 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7efe100fde60 0x7efe100fe230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:53:19.421 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:53:19.420+0000 7efe14e00700 1 -- 172.21.15.18:0/3783134372 >> 172.21.15.18:0/3783134372 conn(0x7efe100f78d0 msgr2=0x7efe100f9ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:53:19.422 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:53:19.420+0000 7efe14e00700 1 -- 172.21.15.18:0/3783134372 shutdown_connections 2024-04-12T21:53:19.422 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:53:19.420+0000 7efe14e00700 1 -- 172.21.15.18:0/3783134372 wait complete. 2024-04-12T21:53:19.423 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:53:19.421+0000 7efe14e00700 1 Processor -- start 2024-04-12T21:53:19.423 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:53:19.421+0000 7efe14e00700 1 -- start start 2024-04-12T21:53:19.423 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:53:19.421+0000 7efe14e00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7efe100fde60 0x7efe100fbc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:53:19.423 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:53:19.422+0000 7efe14e00700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7efe100fe800 0x7efe100fc1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:53:19.423 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:53:19.422+0000 7efe14e00700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe100fc7f0 con 0x7efe100fde60 2024-04-12T21:53:19.423 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:53:19.422+0000 7efe14e00700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe100fc930 con 0x7efe100fe800 2024-04-12T21:53:19.423 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:53:19.422+0000 7efe0de00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7efe100fde60 0x7efe100fbc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:53:19.424 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:53:19.422+0000 7efe0de00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7efe100fde60 0x7efe100fbc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:52648/0 (socket says 172.21.15.18:52648) 2024-04-12T21:53:19.424 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:53:19.422+0000 7efe0de00700 1 -- 172.21.15.18:0/37198954 learned_addr learned my addr 172.21.15.18:0/37198954 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:53:19.425 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:53:19.422+0000 7efe0d400700 1 --2- 172.21.15.18:0/37198954 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7efe100fe800 0x7efe100fc1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:53:19.425 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:53:19.422+0000 7efe0de00700 1 -- 172.21.15.18:0/37198954 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7efe100fe800 msgr2=0x7efe100fc1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:53:19.425 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:53:19.422+0000 7efe0de00700 1 --2- 172.21.15.18:0/37198954 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7efe100fe800 0x7efe100fc1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:53:19.425 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:53:19.422+0000 7efe0de00700 1 -- 172.21.15.18:0/37198954 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efdfc005040 con 0x7efe100fde60 2024-04-12T21:53:19.425 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:53:19.423+0000 7efe0de00700 1 --2- 172.21.15.18:0/37198954 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7efe100fde60 0x7efe100fbc90 secure :-1 s=READY pgs=1359 cs=0 l=1 rev1=1 crypto rx=0x7efe0000b790 tx=0x7efe00009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:53:19.425 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:53:19.423+0000 7efe06000700 1 -- 172.21.15.18:0/37198954 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7efe0000f770 con 0x7efe100fde60 2024-04-12T21:53:19.425 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:53:19.423+0000 7efe14e00700 1 -- 172.21.15.18:0/37198954 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efe10119e90 con 0x7efe100fde60 2024-04-12T21:53:19.425 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:53:19.423+0000 7efe06000700 1 -- 172.21.15.18:0/37198954 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7efe0000fda0 con 0x7efe100fde60 2024-04-12T21:53:19.425 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:53:19.423+0000 7efe06000700 1 -- 172.21.15.18:0/37198954 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7efe0000d310 con 0x7efe100fde60 2024-04-12T21:53:19.425 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:53:19.423+0000 7efe14e00700 1 -- 172.21.15.18:0/37198954 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efe1011a2f0 con 0x7efe100fde60 2024-04-12T21:53:19.426 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:53:19.424+0000 7efe14e00700 1 -- 172.21.15.18:0/37198954 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efe10066710 con 0x7efe100fde60 2024-04-12T21:53:19.426 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:53:19.425+0000 7efe06000700 1 -- 172.21.15.18:0/37198954 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7efe00010420 con 0x7efe100fde60 2024-04-12T21:53:19.427 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:53:19.426+0000 7efe06000700 1 --2- 172.21.15.18:0/37198954 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7efdf406ec10 0x7efdf40710c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:53:19.427 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:53:19.426+0000 7efe06000700 1 -- 172.21.15.18:0/37198954 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 6757+0+0 (secure 0 0 0) 0x7efe0008e9b0 con 0x7efe100fde60 2024-04-12T21:53:19.428 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:53:19.426+0000 7efe0d400700 1 --2- 172.21.15.18:0/37198954 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7efdf406ec10 0x7efdf40710c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:53:19.428 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:53:19.427+0000 7efe0d400700 1 --2- 172.21.15.18:0/37198954 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7efdf406ec10 0x7efdf40710c0 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7efdfc00b3b0 tx=0x7efdfc009e30 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:53:19.432 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:53:19.431+0000 7efe06000700 1 -- 172.21.15.18:0/37198954 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7efe000576d0 con 0x7efe100fde60 2024-04-12T21:53:19.779 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:53:19.777+0000 7efe14e00700 1 -- 172.21.15.18:0/37198954 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7efe1004bed0 con 0x7efe100fde60 2024-04-12T21:53:19.780 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:53:19.779+0000 7efe06000700 1 -- 172.21.15.18:0/37198954 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+623 (secure 0 0 0) 0x7efe0005ad90 con 0x7efe100fde60 2024-04-12T21:53:19.781 INFO:teuthology.orchestra.run.smithi018.stdout:{ 2024-04-12T21:53:19.781 INFO:teuthology.orchestra.run.smithi018.stdout: "mon": { 2024-04-12T21:53:19.781 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 19.0.0-2879-g324baf8d (324baf8ddb91dea46cac50cb535c306115791355) squid (dev)": 2 2024-04-12T21:53:19.781 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-12T21:53:19.781 INFO:teuthology.orchestra.run.smithi018.stdout: "mgr": { 2024-04-12T21:53:19.781 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 19.0.0-2879-g324baf8d (324baf8ddb91dea46cac50cb535c306115791355) squid (dev)": 2 2024-04-12T21:53:19.782 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-12T21:53:19.782 INFO:teuthology.orchestra.run.smithi018.stdout: "osd": { 2024-04-12T21:53:19.782 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 19.0.0-2879-g324baf8d (324baf8ddb91dea46cac50cb535c306115791355) squid (dev)": 6 2024-04-12T21:53:19.782 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-12T21:53:19.782 INFO:teuthology.orchestra.run.smithi018.stdout: "mds": { 2024-04-12T21:53:19.783 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 19.0.0-2879-g324baf8d (324baf8ddb91dea46cac50cb535c306115791355) squid (dev)": 4 2024-04-12T21:53:19.783 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-12T21:53:19.783 INFO:teuthology.orchestra.run.smithi018.stdout: "overall": { 2024-04-12T21:53:19.783 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 19.0.0-2879-g324baf8d (324baf8ddb91dea46cac50cb535c306115791355) squid (dev)": 14 2024-04-12T21:53:19.783 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-04-12T21:53:19.783 INFO:teuthology.orchestra.run.smithi018.stdout:} 2024-04-12T21:53:19.784 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:53:19.782+0000 7efe14e00700 1 -- 172.21.15.18:0/37198954 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7efdf406ec10 msgr2=0x7efdf40710c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:53:19.784 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:53:19.782+0000 7efe14e00700 1 --2- 172.21.15.18:0/37198954 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7efdf406ec10 0x7efdf40710c0 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7efdfc00b3b0 tx=0x7efdfc009e30 comp rx=0 tx=0).stop 2024-04-12T21:53:19.785 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:53:19.782+0000 7efe14e00700 1 -- 172.21.15.18:0/37198954 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7efe100fde60 msgr2=0x7efe100fbc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:53:19.785 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:53:19.782+0000 7efe14e00700 1 --2- 172.21.15.18:0/37198954 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7efe100fde60 0x7efe100fbc90 secure :-1 s=READY pgs=1359 cs=0 l=1 rev1=1 crypto rx=0x7efe0000b790 tx=0x7efe00009250 comp rx=0 tx=0).stop 2024-04-12T21:53:19.785 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:53:19.782+0000 7efe14e00700 1 -- 172.21.15.18:0/37198954 shutdown_connections 2024-04-12T21:53:19.785 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:53:19.782+0000 7efe14e00700 1 --2- 172.21.15.18:0/37198954 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7efdf406ec10 0x7efdf40710c0 unknown :-1 s=CLOSED pgs=721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:53:19.785 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:53:19.782+0000 7efe14e00700 1 --2- 172.21.15.18:0/37198954 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7efe100fde60 0x7efe100fbc90 unknown :-1 s=CLOSED pgs=1359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:53:19.785 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:53:19.782+0000 7efe14e00700 1 --2- 172.21.15.18:0/37198954 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7efe100fe800 0x7efe100fc1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:53:19.785 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:53:19.782+0000 7efe14e00700 1 -- 172.21.15.18:0/37198954 >> 172.21.15.18:0/37198954 conn(0x7efe100f78d0 msgr2=0x7efe10102fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:53:19.785 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:53:19.782+0000 7efe14e00700 1 -- 172.21.15.18:0/37198954 shutdown_connections 2024-04-12T21:53:19.786 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:53:19.782+0000 7efe14e00700 1 -- 172.21.15.18:0/37198954 wait complete. 2024-04-12T21:53:19.948 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:53:19.946+0000 7f4ba7000700 1 -- 172.21.15.18:0/1390615519 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4ba0072e00 msgr2=0x7f4ba010d760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:53:19.948 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:53:19.946+0000 7f4ba7000700 1 --2- 172.21.15.18:0/1390615519 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4ba0072e00 0x7f4ba010d760 secure :-1 s=READY pgs=1360 cs=0 l=1 rev1=1 crypto rx=0x7f4b8c004940 tx=0x7f4b8c009e30 comp rx=0 tx=0).stop 2024-04-12T21:53:19.948 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:53:19.946+0000 7f4ba7000700 1 -- 172.21.15.18:0/1390615519 shutdown_connections 2024-04-12T21:53:19.948 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:53:19.946+0000 7f4ba7000700 1 --2- 172.21.15.18:0/1390615519 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4ba0072e00 0x7f4ba010d760 unknown :-1 s=CLOSED pgs=1360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:53:19.948 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:53:19.946+0000 7f4ba7000700 1 --2- 172.21.15.18:0/1390615519 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4ba00724f0 0x7f4ba00728c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:53:19.948 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:53:19.946+0000 7f4ba7000700 1 -- 172.21.15.18:0/1390615519 >> 172.21.15.18:0/1390615519 conn(0x7f4ba00faba0 msgr2=0x7f4ba00fcfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:53:19.948 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:53:19.946+0000 7f4ba7000700 1 -- 172.21.15.18:0/1390615519 shutdown_connections 2024-04-12T21:53:19.948 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:53:19.946+0000 7f4ba7000700 1 -- 172.21.15.18:0/1390615519 wait complete. 2024-04-12T21:53:19.950 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:53:19.947+0000 7f4ba7000700 1 Processor -- start 2024-04-12T21:53:19.950 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:53:19.947+0000 7f4ba7000700 1 -- start start 2024-04-12T21:53:19.951 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:53:19.948+0000 7f4ba7000700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4ba00724f0 0x7f4ba0119f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:53:19.951 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:53:19.948+0000 7f4ba7000700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4ba0072e00 0x7f4ba011a450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:53:19.951 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:53:19.948+0000 7f4ba7000700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ba011aaa0 con 0x7f4ba00724f0 2024-04-12T21:53:19.951 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:53:19.948+0000 7f4ba7000700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ba0114080 con 0x7f4ba0072e00 2024-04-12T21:53:19.951 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:53:19.948+0000 7f4b9fe00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4ba00724f0 0x7f4ba0119f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:53:19.951 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:53:19.948+0000 7f4b9fe00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4ba00724f0 0x7f4ba0119f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:52664/0 (socket says 172.21.15.18:52664) 2024-04-12T21:53:19.951 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:53:19.948+0000 7f4b9fe00700 1 -- 172.21.15.18:0/4186151126 learned_addr learned my addr 172.21.15.18:0/4186151126 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:53:19.951 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:53:19.948+0000 7f4b9f400700 1 --2- 172.21.15.18:0/4186151126 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4ba0072e00 0x7f4ba011a450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:53:19.951 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:53:19.948+0000 7f4b9fe00700 1 -- 172.21.15.18:0/4186151126 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4ba0072e00 msgr2=0x7f4ba011a450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:53:19.951 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:53:19.948+0000 7f4b9fe00700 1 --2- 172.21.15.18:0/4186151126 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4ba0072e00 0x7f4ba011a450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:53:19.951 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:53:19.948+0000 7f4b9fe00700 1 -- 172.21.15.18:0/4186151126 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b8c005040 con 0x7f4ba00724f0 2024-04-12T21:53:19.952 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:53:19.949+0000 7f4b9fe00700 1 --2- 172.21.15.18:0/4186151126 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4ba00724f0 0x7f4ba0119f10 secure :-1 s=READY pgs=1361 cs=0 l=1 rev1=1 crypto rx=0x7f4b90002a50 tx=0x7f4b900046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:53:19.952 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:53:19.949+0000 7f4b97e00700 1 -- 172.21.15.18:0/4186151126 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4b90007cc0 con 0x7f4ba00724f0 2024-04-12T21:53:19.952 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:53:19.949+0000 7f4b97e00700 1 -- 172.21.15.18:0/4186151126 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f4b9000f040 con 0x7f4ba00724f0 2024-04-12T21:53:19.952 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:53:19.949+0000 7f4b97e00700 1 -- 172.21.15.18:0/4186151126 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4b90009640 con 0x7f4ba00724f0 2024-04-12T21:53:19.952 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:53:19.949+0000 7f4ba7000700 1 -- 172.21.15.18:0/4186151126 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4ba0114360 con 0x7f4ba00724f0 2024-04-12T21:53:19.952 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:53:19.949+0000 7f4ba7000700 1 -- 172.21.15.18:0/4186151126 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4ba0114920 con 0x7f4ba00724f0 2024-04-12T21:53:19.952 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:53:19.950+0000 7f4ba7000700 1 -- 172.21.15.18:0/4186151126 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4ba010a5a0 con 0x7f4ba00724f0 2024-04-12T21:53:19.952 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:53:19.951+0000 7f4b97e00700 1 -- 172.21.15.18:0/4186151126 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7f4b900097a0 con 0x7f4ba00724f0 2024-04-12T21:53:19.954 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:53:19.952+0000 7f4b97e00700 1 --2- 172.21.15.18:0/4186151126 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f4b8406ec00 0x7f4b840710b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:53:19.954 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:53:19.953+0000 7f4b97e00700 1 -- 172.21.15.18:0/4186151126 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 6757+0+0 (secure 0 0 0) 0x7f4b9008e510 con 0x7f4ba00724f0 2024-04-12T21:53:19.954 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:53:19.953+0000 7f4b9f400700 1 --2- 172.21.15.18:0/4186151126 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f4b8406ec00 0x7f4b840710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:53:19.954 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:53:19.953+0000 7f4b9f400700 1 --2- 172.21.15.18:0/4186151126 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f4b8406ec00 0x7f4b840710b0 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7f4b8c000c00 tx=0x7f4b8c009e30 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:53:19.958 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:53:19.957+0000 7f4b97e00700 1 -- 172.21.15.18:0/4186151126 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f4b9005ace0 con 0x7f4ba00724f0 2024-04-12T21:53:20.255 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:53:20.254+0000 7f4ba7000700 1 -- 172.21.15.18:0/4186151126 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f4ba0110470 con 0x7f4ba00724f0 2024-04-12T21:53:20.256 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:53:20.255+0000 7f4b97e00700 1 -- 172.21.15.18:0/4186151126 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 33 v33) v1 ==== 76+0+1876 (secure 0 0 0) 0x7f4b9005a870 con 0x7f4ba00724f0 2024-04-12T21:53:20.258 INFO:teuthology.orchestra.run.smithi018.stdout:e33 2024-04-12T21:53:20.258 INFO:teuthology.orchestra.run.smithi018.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-12T21:53:20.258 INFO:teuthology.orchestra.run.smithi018.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-12T21:53:20.258 INFO:teuthology.orchestra.run.smithi018.stdout:legacy client fscid: 1 2024-04-12T21:53:20.259 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-12T21:53:20.259 INFO:teuthology.orchestra.run.smithi018.stdout:Filesystem 'cephfs' (1) 2024-04-12T21:53:20.259 INFO:teuthology.orchestra.run.smithi018.stdout:fs_name cephfs 2024-04-12T21:53:20.259 INFO:teuthology.orchestra.run.smithi018.stdout:epoch 24 2024-04-12T21:53:20.259 INFO:teuthology.orchestra.run.smithi018.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-04-12T21:53:20.259 INFO:teuthology.orchestra.run.smithi018.stdout:created 2024-04-12T20:51:57.362850+0000 2024-04-12T21:53:20.260 INFO:teuthology.orchestra.run.smithi018.stdout:modified 2024-04-12T21:06:59.717190+0000 2024-04-12T21:53:20.260 INFO:teuthology.orchestra.run.smithi018.stdout:tableserver 0 2024-04-12T21:53:20.260 INFO:teuthology.orchestra.run.smithi018.stdout:root 0 2024-04-12T21:53:20.260 INFO:teuthology.orchestra.run.smithi018.stdout:session_timeout 60 2024-04-12T21:53:20.261 INFO:teuthology.orchestra.run.smithi018.stdout:session_autoclose 300 2024-04-12T21:53:20.261 INFO:teuthology.orchestra.run.smithi018.stdout:max_file_size 1099511627776 2024-04-12T21:53:20.261 INFO:teuthology.orchestra.run.smithi018.stdout:max_xattr_size 65536 2024-04-12T21:53:20.261 INFO:teuthology.orchestra.run.smithi018.stdout:required_client_features {} 2024-04-12T21:53:20.261 INFO:teuthology.orchestra.run.smithi018.stdout:last_failure 0 2024-04-12T21:53:20.261 INFO:teuthology.orchestra.run.smithi018.stdout:last_failure_osd_epoch 100 2024-04-12T21:53:20.262 INFO:teuthology.orchestra.run.smithi018.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-12T21:53:20.262 INFO:teuthology.orchestra.run.smithi018.stdout:max_mds 2 2024-04-12T21:53:20.262 INFO:teuthology.orchestra.run.smithi018.stdout:in 0,1 2024-04-12T21:53:20.262 INFO:teuthology.orchestra.run.smithi018.stdout:up {} 2024-04-12T21:53:20.262 INFO:teuthology.orchestra.run.smithi018.stdout:failed 0,1 2024-04-12T21:53:20.262 INFO:teuthology.orchestra.run.smithi018.stdout:damaged 2024-04-12T21:53:20.262 INFO:teuthology.orchestra.run.smithi018.stdout:stopped 2024-04-12T21:53:20.263 INFO:teuthology.orchestra.run.smithi018.stdout:data_pools [3] 2024-04-12T21:53:20.263 INFO:teuthology.orchestra.run.smithi018.stdout:metadata_pool 2 2024-04-12T21:53:20.263 INFO:teuthology.orchestra.run.smithi018.stdout:inline_data disabled 2024-04-12T21:53:20.263 INFO:teuthology.orchestra.run.smithi018.stdout:balancer 2024-04-12T21:53:20.263 INFO:teuthology.orchestra.run.smithi018.stdout:bal_rank_mask -1 2024-04-12T21:53:20.263 INFO:teuthology.orchestra.run.smithi018.stdout:standby_count_wanted 1 2024-04-12T21:53:20.263 INFO:teuthology.orchestra.run.smithi018.stdout:qdb_cluster leader: 0 members: 2024-04-12T21:53:20.264 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-12T21:53:20.264 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-12T21:53:20.264 INFO:teuthology.orchestra.run.smithi018.stdout:Standby daemons: 2024-04-12T21:53:20.264 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-12T21:53:20.264 INFO:teuthology.orchestra.run.smithi018.stdout:[mds.cephfs.smithi018.acqbsc{-1:34436} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.18:6828/755179764,v1:172.21.15.18:6829/755179764] compat {c=[1],r=[1],i=[1fff]}] 2024-04-12T21:53:20.264 INFO:teuthology.orchestra.run.smithi018.stdout:[mds.cephfs.smithi018.szjluo{-1:34468} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.18:6826/2407992653,v1:172.21.15.18:6827/2407992653] compat {c=[1],r=[1],i=[1fff]}] 2024-04-12T21:53:20.264 INFO:teuthology.orchestra.run.smithi018.stdout:[mds.cephfs.smithi186.jdllyn{-1:44595} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.186:6824/3523444202,v1:172.21.15.186:6825/3523444202] compat {c=[1],r=[1],i=[1fff]}] 2024-04-12T21:53:20.264 INFO:teuthology.orchestra.run.smithi018.stdout:[mds.cephfs.smithi186.wjnafs{-1:44875} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.186:6826/3122443766,v1:172.21.15.186:6827/3122443766] compat {c=[1],r=[1],i=[1fff]}] 2024-04-12T21:53:20.266 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:53:20.259+0000 7f4ba7000700 1 -- 172.21.15.18:0/4186151126 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f4b8406ec00 msgr2=0x7f4b840710b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:53:20.266 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:53:20.259+0000 7f4ba7000700 1 --2- 172.21.15.18:0/4186151126 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f4b8406ec00 0x7f4b840710b0 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7f4b8c000c00 tx=0x7f4b8c009e30 comp rx=0 tx=0).stop 2024-04-12T21:53:20.266 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:53:20.259+0000 7f4ba7000700 1 -- 172.21.15.18:0/4186151126 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4ba00724f0 msgr2=0x7f4ba0119f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:53:20.266 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:53:20.259+0000 7f4ba7000700 1 --2- 172.21.15.18:0/4186151126 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4ba00724f0 0x7f4ba0119f10 secure :-1 s=READY pgs=1361 cs=0 l=1 rev1=1 crypto rx=0x7f4b90002a50 tx=0x7f4b900046b0 comp rx=0 tx=0).stop 2024-04-12T21:53:20.266 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:53:20.259+0000 7f4ba7000700 1 -- 172.21.15.18:0/4186151126 shutdown_connections 2024-04-12T21:53:20.267 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:53:20.259+0000 7f4ba7000700 1 --2- 172.21.15.18:0/4186151126 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f4b8406ec00 0x7f4b840710b0 unknown :-1 s=CLOSED pgs=722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:53:20.267 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:53:20.259+0000 7f4ba7000700 1 --2- 172.21.15.18:0/4186151126 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4ba00724f0 0x7f4ba0119f10 unknown :-1 s=CLOSED pgs=1361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:53:20.267 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:53:20.259+0000 7f4ba7000700 1 --2- 172.21.15.18:0/4186151126 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4ba0072e00 0x7f4ba011a450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:53:20.267 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:53:20.259+0000 7f4ba7000700 1 -- 172.21.15.18:0/4186151126 >> 172.21.15.18:0/4186151126 conn(0x7f4ba00faba0 msgr2=0x7f4ba0107720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:53:20.267 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:53:20.259+0000 7f4ba7000700 1 -- 172.21.15.18:0/4186151126 shutdown_connections 2024-04-12T21:53:20.267 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:53:20.259+0000 7f4ba7000700 1 -- 172.21.15.18:0/4186151126 wait complete. 2024-04-12T21:53:20.267 INFO:teuthology.orchestra.run.smithi018.stderr:dumped fsmap epoch 33 2024-04-12T21:53:20.356 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:53:20 smithi018 ceph-mon[112590]: from='client.36776 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:53:20.357 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:53:20 smithi018 ceph-mon[112590]: pgmap v1609: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:53:20.357 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:53:20 smithi018 ceph-mon[112590]: from='client.36780 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:53:20.357 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:53:20 smithi018 ceph-mon[112590]: from='client.? 172.21.15.18:0/37198954' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T21:53:20.357 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:53:20 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-12T21:53:20.357 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:53:20 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-12T21:53:20.357 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:53:20 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' 2024-04-12T21:53:20.422 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:53:20.416+0000 7f1d2ce00700 1 -- 172.21.15.18:0/500893930 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1d281037c0 msgr2=0x7f1d280fce30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:53:20.423 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:53:20.416+0000 7f1d2ce00700 1 --2- 172.21.15.18:0/500893930 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1d281037c0 0x7f1d280fce30 secure :-1 s=READY pgs=1362 cs=0 l=1 rev1=1 crypto rx=0x7f1d14004940 tx=0x7f1d14009e30 comp rx=0 tx=0).stop 2024-04-12T21:53:20.423 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:53:20.416+0000 7f1d2ce00700 1 -- 172.21.15.18:0/500893930 shutdown_connections 2024-04-12T21:53:20.423 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:53:20.416+0000 7f1d2ce00700 1 --2- 172.21.15.18:0/500893930 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1d281037c0 0x7f1d280fce30 unknown :-1 s=CLOSED pgs=1362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:53:20.423 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:53:20.416+0000 7f1d2ce00700 1 --2- 172.21.15.18:0/500893930 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1d28102e20 0x7f1d281031f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:53:20.424 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:53:20.417+0000 7f1d2ce00700 1 -- 172.21.15.18:0/500893930 >> 172.21.15.18:0/500893930 conn(0x7f1d280f8970 msgr2=0x7f1d280fad80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:53:20.424 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:53:20.417+0000 7f1d2ce00700 1 -- 172.21.15.18:0/500893930 shutdown_connections 2024-04-12T21:53:20.424 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:53:20.417+0000 7f1d2ce00700 1 -- 172.21.15.18:0/500893930 wait complete. 2024-04-12T21:53:20.424 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:53:20.418+0000 7f1d2ce00700 1 Processor -- start 2024-04-12T21:53:20.424 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:53:20.418+0000 7f1d2ce00700 1 -- start start 2024-04-12T21:53:20.424 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:53:20.418+0000 7f1d2ce00700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1d28102e20 0x7f1d280702c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:53:20.424 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:53:20.418+0000 7f1d2ce00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1d281037c0 0x7f1d28070800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:53:20.425 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:53:20.418+0000 7f1d2ce00700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d28070e20 con 0x7f1d281037c0 2024-04-12T21:53:20.425 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:53:20.418+0000 7f1d2ce00700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d28070f60 con 0x7f1d28102e20 2024-04-12T21:53:20.425 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:53:20.418+0000 7f1d25400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1d281037c0 0x7f1d28070800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:53:20.425 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:53:20.418+0000 7f1d25400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1d281037c0 0x7f1d28070800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:52680/0 (socket says 172.21.15.18:52680) 2024-04-12T21:53:20.425 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:53:20.418+0000 7f1d25400700 1 -- 172.21.15.18:0/1256621126 learned_addr learned my addr 172.21.15.18:0/1256621126 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:53:20.425 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:53:20.419+0000 7f1d25400700 1 -- 172.21.15.18:0/1256621126 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1d28102e20 msgr2=0x7f1d280702c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:53:20.425 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:53:20.419+0000 7f1d25400700 1 --2- 172.21.15.18:0/1256621126 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1d28102e20 0x7f1d280702c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:53:20.426 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:53:20.419+0000 7f1d25400700 1 -- 172.21.15.18:0/1256621126 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d14005040 con 0x7f1d281037c0 2024-04-12T21:53:20.426 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:53:20.419+0000 7f1d25400700 1 --2- 172.21.15.18:0/1256621126 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1d281037c0 0x7f1d28070800 secure :-1 s=READY pgs=1363 cs=0 l=1 rev1=1 crypto rx=0x7f1d14009100 tx=0x7f1d14009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:53:20.426 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:53:20.419+0000 7f1d1e000700 1 -- 172.21.15.18:0/1256621126 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1d1400e040 con 0x7f1d281037c0 2024-04-12T21:53:20.426 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:53:20.420+0000 7f1d1e000700 1 -- 172.21.15.18:0/1256621126 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f1d14013420 con 0x7f1d281037c0 2024-04-12T21:53:20.426 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:53:20.420+0000 7f1d1e000700 1 -- 172.21.15.18:0/1256621126 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1d1401c630 con 0x7f1d281037c0 2024-04-12T21:53:20.426 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:53:20.420+0000 7f1d2ce00700 1 -- 172.21.15.18:0/1256621126 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1d28119ef0 con 0x7f1d281037c0 2024-04-12T21:53:20.427 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:53:20.420+0000 7f1d2ce00700 1 -- 172.21.15.18:0/1256621126 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1d2811a370 con 0x7f1d281037c0 2024-04-12T21:53:20.427 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:53:20.421+0000 7f1d2ce00700 1 -- 172.21.15.18:0/1256621126 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1d280fe570 con 0x7f1d281037c0 2024-04-12T21:53:20.428 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:53:20.422+0000 7f1d1e000700 1 -- 172.21.15.18:0/1256621126 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7f1d14013590 con 0x7f1d281037c0 2024-04-12T21:53:20.428 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:53:20.423+0000 7f1d1e000700 1 --2- 172.21.15.18:0/1256621126 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f1d0c06ec10 0x7f1d0c0710c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:53:20.429 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:53:20.423+0000 7f1d1e000700 1 -- 172.21.15.18:0/1256621126 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 6757+0+0 (secure 0 0 0) 0x7f1d1408fc60 con 0x7f1d281037c0 2024-04-12T21:53:20.429 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:53:20.423+0000 7f1d25e00700 1 --2- 172.21.15.18:0/1256621126 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f1d0c06ec10 0x7f1d0c0710c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:53:20.429 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:53:20.424+0000 7f1d25e00700 1 --2- 172.21.15.18:0/1256621126 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f1d0c06ec10 0x7f1d0c0710c0 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f1d18006fd0 tx=0x7f1d18006db0 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:53:20.430 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:53:20.428+0000 7f1d1e000700 1 -- 172.21.15.18:0/1256621126 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f1d140588e0 con 0x7f1d281037c0 2024-04-12T21:53:20.535 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:53:20 smithi186 ceph-mon[86379]: from='client.36776 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:53:20.536 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:53:20 smithi186 ceph-mon[86379]: pgmap v1609: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:53:20.536 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:53:20 smithi186 ceph-mon[86379]: from='client.36780 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:53:20.536 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:53:20 smithi186 ceph-mon[86379]: from='client.? 172.21.15.18:0/37198954' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T21:53:20.536 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:53:20 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-12T21:53:20.536 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:53:20 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-12T21:53:20.536 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:53:20 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' 2024-04-12T21:53:20.686 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:53:20.684+0000 7f1d2ce00700 1 -- 172.21.15.18:0/1256621126 --> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f1d280fe570 con 0x7f1d0c06ec10 2024-04-12T21:53:20.687 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:53:20.686+0000 7f1d1e000700 1 -- 172.21.15.18:0/1256621126 <== mgr.34102 v2:172.21.15.18:6800/3812891919 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f1d280fe570 con 0x7f1d0c06ec10 2024-04-12T21:53:20.688 INFO:teuthology.orchestra.run.smithi018.stdout:{ 2024-04-12T21:53:20.688 INFO:teuthology.orchestra.run.smithi018.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:f4452def3be4d350f4a8e5feaf7ebb1511be9298fe68478ba7a2d6fbd7d69a63", 2024-04-12T21:53:20.689 INFO:teuthology.orchestra.run.smithi018.stdout: "in_progress": true, 2024-04-12T21:53:20.689 INFO:teuthology.orchestra.run.smithi018.stdout: "which": "Upgrading all daemon types on all hosts", 2024-04-12T21:53:20.689 INFO:teuthology.orchestra.run.smithi018.stdout: "services_complete": [ 2024-04-12T21:53:20.689 INFO:teuthology.orchestra.run.smithi018.stdout: "mgr", 2024-04-12T21:53:20.689 INFO:teuthology.orchestra.run.smithi018.stdout: "mds", 2024-04-12T21:53:20.689 INFO:teuthology.orchestra.run.smithi018.stdout: "crash", 2024-04-12T21:53:20.689 INFO:teuthology.orchestra.run.smithi018.stdout: "osd", 2024-04-12T21:53:20.689 INFO:teuthology.orchestra.run.smithi018.stdout: "mon" 2024-04-12T21:53:20.689 INFO:teuthology.orchestra.run.smithi018.stdout: ], 2024-04-12T21:53:20.689 INFO:teuthology.orchestra.run.smithi018.stdout: "progress": "16/21 daemons upgraded", 2024-04-12T21:53:20.690 INFO:teuthology.orchestra.run.smithi018.stdout: "message": "Currently upgrading mds daemons", 2024-04-12T21:53:20.690 INFO:teuthology.orchestra.run.smithi018.stdout: "is_paused": false 2024-04-12T21:53:20.690 INFO:teuthology.orchestra.run.smithi018.stdout:} 2024-04-12T21:53:20.692 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:53:20.690+0000 7f1d2ce00700 1 -- 172.21.15.18:0/1256621126 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f1d0c06ec10 msgr2=0x7f1d0c0710c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:53:20.692 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:53:20.690+0000 7f1d2ce00700 1 --2- 172.21.15.18:0/1256621126 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f1d0c06ec10 0x7f1d0c0710c0 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f1d18006fd0 tx=0x7f1d18006db0 comp rx=0 tx=0).stop 2024-04-12T21:53:20.692 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:53:20.690+0000 7f1d2ce00700 1 -- 172.21.15.18:0/1256621126 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1d281037c0 msgr2=0x7f1d28070800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:53:20.692 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:53:20.690+0000 7f1d2ce00700 1 --2- 172.21.15.18:0/1256621126 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1d281037c0 0x7f1d28070800 secure :-1 s=READY pgs=1363 cs=0 l=1 rev1=1 crypto rx=0x7f1d14009100 tx=0x7f1d14009e30 comp rx=0 tx=0).stop 2024-04-12T21:53:20.693 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:53:20.691+0000 7f1d2ce00700 1 -- 172.21.15.18:0/1256621126 shutdown_connections 2024-04-12T21:53:20.693 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:53:20.691+0000 7f1d2ce00700 1 --2- 172.21.15.18:0/1256621126 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f1d0c06ec10 0x7f1d0c0710c0 unknown :-1 s=CLOSED pgs=723 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:53:20.693 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:53:20.691+0000 7f1d2ce00700 1 --2- 172.21.15.18:0/1256621126 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1d28102e20 0x7f1d280702c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:53:20.693 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:53:20.691+0000 7f1d2ce00700 1 --2- 172.21.15.18:0/1256621126 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1d281037c0 0x7f1d28070800 unknown :-1 s=CLOSED pgs=1363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:53:20.693 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:53:20.691+0000 7f1d2ce00700 1 -- 172.21.15.18:0/1256621126 >> 172.21.15.18:0/1256621126 conn(0x7f1d280f8970 msgr2=0x7f1d280f9900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:53:20.693 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:53:20.691+0000 7f1d2ce00700 1 -- 172.21.15.18:0/1256621126 shutdown_connections 2024-04-12T21:53:20.693 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:53:20.691+0000 7f1d2ce00700 1 -- 172.21.15.18:0/1256621126 wait complete. 2024-04-12T21:53:20.922 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:53:20.920+0000 7fb0ea600700 1 -- 172.21.15.18:0/4227696078 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb0e4102c40 msgr2=0x7fb0e4103010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:53:20.922 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:53:20.920+0000 7fb0ea600700 1 --2- 172.21.15.18:0/4227696078 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb0e4102c40 0x7fb0e4103010 secure :-1 s=READY pgs=1364 cs=0 l=1 rev1=1 crypto rx=0x7fb0d8004940 tx=0x7fb0d8009e30 comp rx=0 tx=0).stop 2024-04-12T21:53:20.922 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:53:20.920+0000 7fb0ea600700 1 -- 172.21.15.18:0/4227696078 shutdown_connections 2024-04-12T21:53:20.923 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:53:20.920+0000 7fb0ea600700 1 --2- 172.21.15.18:0/4227696078 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb0e4065620 0x7fb0e4065a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:53:20.923 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:53:20.920+0000 7fb0ea600700 1 --2- 172.21.15.18:0/4227696078 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb0e4102c40 0x7fb0e4103010 unknown :-1 s=CLOSED pgs=1364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:53:20.923 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:53:20.920+0000 7fb0ea600700 1 -- 172.21.15.18:0/4227696078 >> 172.21.15.18:0/4227696078 conn(0x7fb0e4071b70 msgr2=0x7fb0e4071f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:53:20.923 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:53:20.921+0000 7fb0ea600700 1 -- 172.21.15.18:0/4227696078 shutdown_connections 2024-04-12T21:53:20.923 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:53:20.921+0000 7fb0ea600700 1 -- 172.21.15.18:0/4227696078 wait complete. 2024-04-12T21:53:20.924 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:53:20.921+0000 7fb0ea600700 1 Processor -- start 2024-04-12T21:53:20.924 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:53:20.922+0000 7fb0ea600700 1 -- start start 2024-04-12T21:53:20.924 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:53:20.922+0000 7fb0ea600700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb0e4065620 0x7fb0e4115b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:53:20.924 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:53:20.922+0000 7fb0ea600700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb0e4102c40 0x7fb0e41160c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:53:20.924 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:53:20.922+0000 7fb0ea600700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0e4116710 con 0x7fb0e4102c40 2024-04-12T21:53:20.924 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:53:20.922+0000 7fb0ea600700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0e410fb70 con 0x7fb0e4065620 2024-04-12T21:53:20.925 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:53:20.923+0000 7fb0e3e00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb0e4102c40 0x7fb0e41160c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:53:20.925 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:53:20.923+0000 7fb0e3e00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb0e4102c40 0x7fb0e41160c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:52478/0 (socket says 172.21.15.18:52478) 2024-04-12T21:53:20.925 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:53:20.923+0000 7fb0e3e00700 1 -- 172.21.15.18:0/2346350212 learned_addr learned my addr 172.21.15.18:0/2346350212 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:53:20.925 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:53:20.923+0000 7fb0e9200700 1 --2- 172.21.15.18:0/2346350212 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb0e4065620 0x7fb0e4115b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:53:20.925 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:53:20.924+0000 7fb0e3e00700 1 -- 172.21.15.18:0/2346350212 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb0e4065620 msgr2=0x7fb0e4115b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:53:20.925 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:53:20.924+0000 7fb0e3e00700 1 --2- 172.21.15.18:0/2346350212 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb0e4065620 0x7fb0e4115b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:53:20.925 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:53:20.924+0000 7fb0e3e00700 1 -- 172.21.15.18:0/2346350212 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0d8005040 con 0x7fb0e4102c40 2024-04-12T21:53:20.926 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:53:20.924+0000 7fb0e3e00700 1 --2- 172.21.15.18:0/2346350212 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb0e4102c40 0x7fb0e41160c0 secure :-1 s=READY pgs=1365 cs=0 l=1 rev1=1 crypto rx=0x7fb0d0002a50 tx=0x7fb0d00046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:53:20.929 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:53:20.927+0000 7fb0e0c00700 1 -- 172.21.15.18:0/2346350212 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb0d0007cc0 con 0x7fb0e4102c40 2024-04-12T21:53:20.929 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:53:20.927+0000 7fb0e0c00700 1 -- 172.21.15.18:0/2346350212 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb0d000f040 con 0x7fb0e4102c40 2024-04-12T21:53:20.930 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:53:20.927+0000 7fb0e0c00700 1 -- 172.21.15.18:0/2346350212 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb0d000da30 con 0x7fb0e4102c40 2024-04-12T21:53:20.930 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:53:20.927+0000 7fb0ea600700 1 -- 172.21.15.18:0/2346350212 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb0e410fe50 con 0x7fb0e4102c40 2024-04-12T21:53:20.930 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:53:20.928+0000 7fb0ea600700 1 -- 172.21.15.18:0/2346350212 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb0e4110390 con 0x7fb0e4102c40 2024-04-12T21:53:20.931 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:53:20.929+0000 7fb0ea600700 1 -- 172.21.15.18:0/2346350212 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb0e4106810 con 0x7fb0e4102c40 2024-04-12T21:53:20.933 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:53:20.932+0000 7fb0e0c00700 1 -- 172.21.15.18:0/2346350212 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7fb0d000dbb0 con 0x7fb0e4102c40 2024-04-12T21:53:20.936 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:53:20.935+0000 7fb0e0c00700 1 --2- 172.21.15.18:0/2346350212 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fb0cc06ec10 0x7fb0cc0710c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:53:20.937 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:53:20.935+0000 7fb0e0c00700 1 -- 172.21.15.18:0/2346350212 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 6757+0+0 (secure 0 0 0) 0x7fb0d008ed20 con 0x7fb0e4102c40 2024-04-12T21:53:20.937 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:53:20.936+0000 7fb0e9200700 1 --2- 172.21.15.18:0/2346350212 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fb0cc06ec10 0x7fb0cc0710c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:53:20.938 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:53:20.937+0000 7fb0e9200700 1 --2- 172.21.15.18:0/2346350212 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fb0cc06ec10 0x7fb0cc0710c0 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7fb0d8009fd0 tx=0x7fb0d8009e30 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:53:20.941 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:53:20.940+0000 7fb0e0c00700 1 -- 172.21.15.18:0/2346350212 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7fb0d0057ae0 con 0x7fb0e4102c40 2024-04-12T21:53:21.305 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:53:21.304+0000 7fb0ea600700 1 -- 172.21.15.18:0/2346350212 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7fb0e41111d0 con 0x7fb0e4102c40 2024-04-12T21:53:21.306 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:53:21 smithi018 ceph-mon[112590]: from='client.36784 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:53:21.306 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:53:21 smithi018 ceph-mon[112590]: pgmap v1610: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:53:21.307 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:53:21 smithi018 ceph-mon[112590]: pgmap v1611: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:53:21.307 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:53:21 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-12T21:53:21.307 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:53:21 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T21:53:21.307 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:53:21 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T21:53:21.307 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:53:21 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T21:53:21.307 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:53:21 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T21:53:21.308 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:53:21 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T21:53:21.308 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:53:21 smithi018 ceph-mon[112590]: Upgrade: Setting container_image for all mds 2024-04-12T21:53:21.308 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:53:21 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' 2024-04-12T21:53:21.308 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:53:21 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi018.acqbsc"}]: dispatch 2024-04-12T21:53:21.308 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:53:21 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi018.acqbsc"}]': finished 2024-04-12T21:53:21.309 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:53:21 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi018.szjluo"}]: dispatch 2024-04-12T21:53:21.309 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:53:21 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi018.szjluo"}]': finished 2024-04-12T21:53:21.309 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:53:21 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi186.jdllyn"}]: dispatch 2024-04-12T21:53:21.309 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:53:21 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi186.jdllyn"}]': finished 2024-04-12T21:53:21.309 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:53:21 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi186.wjnafs"}]: dispatch 2024-04-12T21:53:21.309 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:53:21 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi186.wjnafs"}]': finished 2024-04-12T21:53:21.309 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:53:21 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]: dispatch 2024-04-12T21:53:21.310 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:53:21 smithi018 ceph-mon[112590]: from='client.? 172.21.15.18:0/4186151126' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-12T21:53:21.310 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:53:21 smithi018 ceph-mon[112590]: Standby daemon mds.cephfs.smithi018.acqbsc assigned to filesystem cephfs as rank 0 2024-04-12T21:53:21.310 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:53:21 smithi018 ceph-mon[112590]: Standby daemon mds.cephfs.smithi018.szjluo assigned to filesystem cephfs as rank 1 2024-04-12T21:53:21.310 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:53:21 smithi018 ceph-mon[112590]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-04-12T21:53:21.311 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:53:21 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]': finished 2024-04-12T21:53:21.311 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:53:21 smithi018 ceph-mon[112590]: fsmap cephfs:2/2 {0=cephfs.smithi018.acqbsc=up:replay,1=cephfs.smithi018.szjluo=up:replay} 2 up:standby 2024-04-12T21:53:21.311 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:53:21 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T21:53:21.312 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:53:21 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' 2024-04-12T21:53:21.312 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:53:21 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T21:53:21.312 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:53:21 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' 2024-04-12T21:53:21.312 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:53:21 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T21:53:21.312 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:53:21 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T21:53:21.313 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:53:21 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' 2024-04-12T21:53:21.313 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:53:21 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T21:53:21.313 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:53:21 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' 2024-04-12T21:53:21.313 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:53:21 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T21:53:21.313 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:53:21 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' 2024-04-12T21:53:21.313 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:53:21 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T21:53:21.313 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:53:21 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' 2024-04-12T21:53:21.314 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:53:21 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T21:53:21.314 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:53:21 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T21:53:21.314 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:53:21 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T21:53:21.314 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:53:21 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T21:53:21.314 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:53:21 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T21:53:21.314 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:53:21 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T21:53:21.314 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:53:21 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr"}]: dispatch 2024-04-12T21:53:21.315 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:53:21 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mgr"}]': finished 2024-04-12T21:53:21.315 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:53:21 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-04-12T21:53:21.315 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:53:21 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mon"}]': finished 2024-04-12T21:53:21.315 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:53:21 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.crash"}]: dispatch 2024-04-12T21:53:21.315 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:53:21 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.crash"}]': finished 2024-04-12T21:53:21.315 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:53:21 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd"}]: dispatch 2024-04-12T21:53:21.316 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:53:21 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd"}]': finished 2024-04-12T21:53:21.316 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:53:21 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds"}]: dispatch 2024-04-12T21:53:21.316 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:53:21 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds"}]': finished 2024-04-12T21:53:21.316 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:53:21 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.rgw"}]: dispatch 2024-04-12T21:53:21.316 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:53:21 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.rgw"}]': finished 2024-04-12T21:53:21.316 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:53:21 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.rbd-mirror"}]: dispatch 2024-04-12T21:53:21.317 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:53:21 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.rbd-mirror"}]': finished 2024-04-12T21:53:21.317 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:53:21 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-04-12T21:53:21.317 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:53:21 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter"}]: dispatch 2024-04-12T21:53:21.317 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:53:21 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter"}]': finished 2024-04-12T21:53:21.317 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:53:21 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.iscsi"}]: dispatch 2024-04-12T21:53:21.317 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:53:21 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.iscsi"}]': finished 2024-04-12T21:53:21.317 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:53:21 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.nfs"}]: dispatch 2024-04-12T21:53:21.318 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:53:21 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.nfs"}]': finished 2024-04-12T21:53:21.318 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:53:21 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.nvmeof"}]: dispatch 2024-04-12T21:53:21.318 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:53:21 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.nvmeof"}]': finished 2024-04-12T21:53:21.318 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:53:21 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-04-12T21:53:21.318 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:53:21 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-04-12T21:53:21.318 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:53:21 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-04-12T21:53:21.318 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:53:21 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-04-12T21:53:21.318 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:53:21 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-04-12T21:53:21.318 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:53:21 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-04-12T21:53:21.319 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:53:21 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix":"config-key del","key":"mgr/cephadm/upgrade_state"}]: dispatch 2024-04-12T21:53:21.319 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:53:21 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd='[{"prefix":"config-key del","key":"mgr/cephadm/upgrade_state"}]': finished 2024-04-12T21:53:21.319 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:53:21 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-12T21:53:21.322 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:53:21.307+0000 7fb0e0c00700 1 -- 172.21.15.18:0/2346350212 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+8146 (secure 0 0 0) 0x7fb0d005b1a0 con 0x7fb0e4102c40 2024-04-12T21:53:21.323 INFO:teuthology.orchestra.run.smithi018.stdout:HEALTH_WARN failed to probe daemons or devices; 1 filesystem is degraded 2024-04-12T21:53:21.323 INFO:teuthology.orchestra.run.smithi018.stdout:[WRN] CEPHADM_REFRESH_FAILED: failed to probe daemons or devices 2024-04-12T21:53:21.323 INFO:teuthology.orchestra.run.smithi018.stdout: host smithi186 `cephadm ceph-volume` failed: cephadm exited with an error code: 1, stderr: Inferring config /var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/mon.smithi186/config 2024-04-12T21:53:21.324 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/podman: timeout after 895 seconds 2024-04-12T21:53:21.324 INFO:teuthology.orchestra.run.smithi018.stdout:Non-zero exit code 124 from /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:f4452def3be4d350f4a8e5feaf7ebb1511be9298fe68478ba7a2d6fbd7d69a63 -e NODE_NAME=smithi186 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968:/var/run/ceph:z -v /var/log/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968:/var/log/ceph:z -v /var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmp3wdza92s:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:f4452def3be4d350f4a8e5feaf7ebb1511be9298fe68478ba7a2d6fbd7d69a63 inventory --format=json-pretty --filter-for-batch 2024-04-12T21:53:21.324 INFO:teuthology.orchestra.run.smithi018.stdout:Traceback (most recent call last): 2024-04-12T21:53:21.324 INFO:teuthology.orchestra.run.smithi018.stdout: File "/usr/lib64/python3.9/runpy.py", line 197, in _run_module_as_main 2024-04-12T21:53:21.324 INFO:teuthology.orchestra.run.smithi018.stdout: return _run_code(code, main_globals, None, 2024-04-12T21:53:21.324 INFO:teuthology.orchestra.run.smithi018.stdout: File "/usr/lib64/python3.9/runpy.py", line 87, in _run_code 2024-04-12T21:53:21.324 INFO:teuthology.orchestra.run.smithi018.stdout: exec(code, run_globals) 2024-04-12T21:53:21.325 INFO:teuthology.orchestra.run.smithi018.stdout: File "/var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/cephadm.e44f3ed5e34e4ef52eedef233d1190bff004470d8e8005127b5312bff357fa7f/__main__.py", line 5618, in 2024-04-12T21:53:21.325 INFO:teuthology.orchestra.run.smithi018.stdout: File "/var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/cephadm.e44f3ed5e34e4ef52eedef233d1190bff004470d8e8005127b5312bff357fa7f/__main__.py", line 5606, in main 2024-04-12T21:53:21.325 INFO:teuthology.orchestra.run.smithi018.stdout: File "/var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/cephadm.e44f3ed5e34e4ef52eedef233d1190bff004470d8e8005127b5312bff357fa7f/__main__.py", line 408, in _infer_config 2024-04-12T21:53:21.325 INFO:teuthology.orchestra.run.smithi018.stdout: File "/var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/cephadm.e44f3ed5e34e4ef52eedef233d1190bff004470d8e8005127b5312bff357fa7f/__main__.py", line 323, in _infer_fsid 2024-04-12T21:53:21.325 INFO:teuthology.orchestra.run.smithi018.stdout: File "/var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/cephadm.e44f3ed5e34e4ef52eedef233d1190bff004470d8e8005127b5312bff357fa7f/__main__.py", line 436, in _infer_image 2024-04-12T21:53:21.325 INFO:teuthology.orchestra.run.smithi018.stdout: File "/var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/cephadm.e44f3ed5e34e4ef52eedef233d1190bff004470d8e8005127b5312bff357fa7f/__main__.py", line 310, in _validate_fsid 2024-04-12T21:53:21.325 INFO:teuthology.orchestra.run.smithi018.stdout: File "/var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/cephadm.e44f3ed5e34e4ef52eedef233d1190bff004470d8e8005127b5312bff357fa7f/__main__.py", line 3359, in command_ceph_volume 2024-04-12T21:53:21.326 INFO:teuthology.orchestra.run.smithi018.stdout: File "/var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/cephadm.e44f3ed5e34e4ef52eedef233d1190bff004470d8e8005127b5312bff357fa7f/cephadmlib/call_wrappers.py", line 310, in call_throws 2024-04-12T21:53:21.326 INFO:teuthology.orchestra.run.smithi018.stdout:RuntimeError: Failed command: /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:f4452def3be4d350f4a8e5feaf7ebb1511be9298fe68478ba7a2d6fbd7d69a63 -e NODE_NAME=smithi186 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968:/var/run/ceph:z -v /var/log/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968:/var/log/ceph:z -v /var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmp3wdza92s:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:f4452def3be4d350f4a8e5feaf7ebb1511be9298fe68478ba7a2d6fbd7d69a63 inventory --format=json-pretty --filter-for-batch 2024-04-12T21:53:21.326 INFO:teuthology.orchestra.run.smithi018.stdout: host smithi018 `cephadm ceph-volume` failed: cephadm exited with an error code: 1, stderr: Inferring config /var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/mon.smithi018/config 2024-04-12T21:53:21.326 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/podman: timeout after 895 seconds 2024-04-12T21:53:21.326 INFO:teuthology.orchestra.run.smithi018.stdout:Non-zero exit code 124 from /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:f4452def3be4d350f4a8e5feaf7ebb1511be9298fe68478ba7a2d6fbd7d69a63 -e NODE_NAME=smithi018 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968:/var/run/ceph:z -v /var/log/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968:/var/log/ceph:z -v /var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmp2vf0v4bi:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:f4452def3be4d350f4a8e5feaf7ebb1511be9298fe68478ba7a2d6fbd7d69a63 inventory --format=json-pretty --filter-for-batch 2024-04-12T21:53:21.326 INFO:teuthology.orchestra.run.smithi018.stdout:Traceback (most recent call last): 2024-04-12T21:53:21.326 INFO:teuthology.orchestra.run.smithi018.stdout: File "/usr/lib64/python3.9/runpy.py", line 197, in _run_module_as_main 2024-04-12T21:53:21.327 INFO:teuthology.orchestra.run.smithi018.stdout: return _run_code(code, main_globals, None, 2024-04-12T21:53:21.327 INFO:teuthology.orchestra.run.smithi018.stdout: File "/usr/lib64/python3.9/runpy.py", line 87, in _run_code 2024-04-12T21:53:21.327 INFO:teuthology.orchestra.run.smithi018.stdout: exec(code, run_globals) 2024-04-12T21:53:21.327 INFO:teuthology.orchestra.run.smithi018.stdout: File "/var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/cephadm.e44f3ed5e34e4ef52eedef233d1190bff004470d8e8005127b5312bff357fa7f/__main__.py", line 5618, in 2024-04-12T21:53:21.327 INFO:teuthology.orchestra.run.smithi018.stdout: File "/var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/cephadm.e44f3ed5e34e4ef52eedef233d1190bff004470d8e8005127b5312bff357fa7f/__main__.py", line 5606, in main 2024-04-12T21:53:21.327 INFO:teuthology.orchestra.run.smithi018.stdout: File "/var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/cephadm.e44f3ed5e34e4ef52eedef233d1190bff004470d8e8005127b5312bff357fa7f/__main__.py", line 408, in _infer_config 2024-04-12T21:53:21.327 INFO:teuthology.orchestra.run.smithi018.stdout: File "/var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/cephadm.e44f3ed5e34e4ef52eedef233d1190bff004470d8e8005127b5312bff357fa7f/__main__.py", line 323, in _infer_fsid 2024-04-12T21:53:21.327 INFO:teuthology.orchestra.run.smithi018.stdout: File "/var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/cephadm.e44f3ed5e34e4ef52eedef233d1190bff004470d8e8005127b5312bff357fa7f/__main__.py", line 436, in _infer_image 2024-04-12T21:53:21.328 INFO:teuthology.orchestra.run.smithi018.stdout: File "/var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/cephadm.e44f3ed5e34e4ef52eedef233d1190bff004470d8e8005127b5312bff357fa7f/__main__.py", line 310, in _validate_fsid 2024-04-12T21:53:21.328 INFO:teuthology.orchestra.run.smithi018.stdout: File "/var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/cephadm.e44f3ed5e34e4ef52eedef233d1190bff004470d8e8005127b5312bff357fa7f/__main__.py", line 3359, in command_ceph_volume 2024-04-12T21:53:21.328 INFO:teuthology.orchestra.run.smithi018.stdout: File "/var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/cephadm.e44f3ed5e34e4ef52eedef233d1190bff004470d8e8005127b5312bff357fa7f/cephadmlib/call_wrappers.py", line 310, in call_throws 2024-04-12T21:53:21.328 INFO:teuthology.orchestra.run.smithi018.stdout:RuntimeError: Failed command: /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:f4452def3be4d350f4a8e5feaf7ebb1511be9298fe68478ba7a2d6fbd7d69a63 -e NODE_NAME=smithi018 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968:/var/run/ceph:z -v /var/log/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968:/var/log/ceph:z -v /var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmp2vf0v4bi:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:f4452def3be4d350f4a8e5feaf7ebb1511be9298fe68478ba7a2d6fbd7d69a63 inventory --format=json-pretty --filter-for-batch 2024-04-12T21:53:21.328 INFO:teuthology.orchestra.run.smithi018.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-04-12T21:53:21.328 INFO:teuthology.orchestra.run.smithi018.stdout: fs cephfs is degraded 2024-04-12T21:53:21.329 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:53:21.313+0000 7fb0ea600700 1 -- 172.21.15.18:0/2346350212 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fb0cc06ec10 msgr2=0x7fb0cc0710c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:53:21.330 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:53:21.313+0000 7fb0ea600700 1 --2- 172.21.15.18:0/2346350212 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fb0cc06ec10 0x7fb0cc0710c0 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7fb0d8009fd0 tx=0x7fb0d8009e30 comp rx=0 tx=0).stop 2024-04-12T21:53:21.330 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:53:21.313+0000 7fb0ea600700 1 -- 172.21.15.18:0/2346350212 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb0e4102c40 msgr2=0x7fb0e41160c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:53:21.330 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:53:21.313+0000 7fb0ea600700 1 --2- 172.21.15.18:0/2346350212 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb0e4102c40 0x7fb0e41160c0 secure :-1 s=READY pgs=1365 cs=0 l=1 rev1=1 crypto rx=0x7fb0d0002a50 tx=0x7fb0d00046b0 comp rx=0 tx=0).stop 2024-04-12T21:53:21.330 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:53:21.314+0000 7fb0ea600700 1 -- 172.21.15.18:0/2346350212 shutdown_connections 2024-04-12T21:53:21.330 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:53:21.314+0000 7fb0ea600700 1 --2- 172.21.15.18:0/2346350212 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fb0cc06ec10 0x7fb0cc0710c0 unknown :-1 s=CLOSED pgs=724 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:53:21.330 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:53:21.314+0000 7fb0ea600700 1 --2- 172.21.15.18:0/2346350212 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb0e4065620 0x7fb0e4115b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:53:21.330 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:53:21.314+0000 7fb0ea600700 1 --2- 172.21.15.18:0/2346350212 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb0e4102c40 0x7fb0e41160c0 unknown :-1 s=CLOSED pgs=1365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:53:21.330 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:53:21.314+0000 7fb0ea600700 1 -- 172.21.15.18:0/2346350212 >> 172.21.15.18:0/2346350212 conn(0x7fb0e4071b70 msgr2=0x7fb0e4103990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:53:21.330 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:53:21.315+0000 7fb0ea600700 1 -- 172.21.15.18:0/2346350212 shutdown_connections 2024-04-12T21:53:21.330 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:53:21.315+0000 7fb0ea600700 1 -- 172.21.15.18:0/2346350212 wait complete. 2024-04-12T21:53:21.535 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:53:21 smithi186 ceph-mon[86379]: from='client.36784 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:53:21.536 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:53:21 smithi186 ceph-mon[86379]: pgmap v1610: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:53:21.536 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:53:21 smithi186 ceph-mon[86379]: pgmap v1611: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:53:21.536 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:53:21 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-12T21:53:21.536 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:53:21 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T21:53:21.536 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:53:21 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T21:53:21.536 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:53:21 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T21:53:21.536 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:53:21 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T21:53:21.536 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:53:21 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T21:53:21.536 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:53:21 smithi186 ceph-mon[86379]: Upgrade: Setting container_image for all mds 2024-04-12T21:53:21.536 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:53:21 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' 2024-04-12T21:53:21.536 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:53:21 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi018.acqbsc"}]: dispatch 2024-04-12T21:53:21.537 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:53:21 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi018.acqbsc"}]': finished 2024-04-12T21:53:21.537 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:53:21 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi018.szjluo"}]: dispatch 2024-04-12T21:53:21.537 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:53:21 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi018.szjluo"}]': finished 2024-04-12T21:53:21.537 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:53:21 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi186.jdllyn"}]: dispatch 2024-04-12T21:53:21.537 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:53:21 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi186.jdllyn"}]': finished 2024-04-12T21:53:21.537 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:53:21 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi186.wjnafs"}]: dispatch 2024-04-12T21:53:21.537 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:53:21 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi186.wjnafs"}]': finished 2024-04-12T21:53:21.537 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:53:21 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]: dispatch 2024-04-12T21:53:21.537 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:53:21 smithi186 ceph-mon[86379]: from='client.? 172.21.15.18:0/4186151126' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-12T21:53:21.537 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:53:21 smithi186 ceph-mon[86379]: Standby daemon mds.cephfs.smithi018.acqbsc assigned to filesystem cephfs as rank 0 2024-04-12T21:53:21.537 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:53:21 smithi186 ceph-mon[86379]: Standby daemon mds.cephfs.smithi018.szjluo assigned to filesystem cephfs as rank 1 2024-04-12T21:53:21.538 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:53:21 smithi186 ceph-mon[86379]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-04-12T21:53:21.538 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:53:21 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]': finished 2024-04-12T21:53:21.539 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:53:21 smithi186 ceph-mon[86379]: fsmap cephfs:2/2 {0=cephfs.smithi018.acqbsc=up:replay,1=cephfs.smithi018.szjluo=up:replay} 2 up:standby 2024-04-12T21:53:21.539 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:53:21 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T21:53:21.539 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:53:21 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' 2024-04-12T21:53:21.539 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:53:21 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T21:53:21.539 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:53:21 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' 2024-04-12T21:53:21.539 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:53:21 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T21:53:21.539 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:53:21 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T21:53:21.539 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:53:21 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' 2024-04-12T21:53:21.539 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:53:21 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T21:53:21.539 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:53:21 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' 2024-04-12T21:53:21.539 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:53:21 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T21:53:21.540 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:53:21 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' 2024-04-12T21:53:21.540 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:53:21 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T21:53:21.540 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:53:21 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' 2024-04-12T21:53:21.540 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:53:21 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T21:53:21.540 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:53:21 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T21:53:21.540 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:53:21 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T21:53:21.540 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:53:21 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T21:53:21.540 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:53:21 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T21:53:21.540 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:53:21 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T21:53:21.541 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:53:21 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr"}]: dispatch 2024-04-12T21:53:21.541 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:53:21 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mgr"}]': finished 2024-04-12T21:53:21.541 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:53:21 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-04-12T21:53:21.541 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:53:21 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mon"}]': finished 2024-04-12T21:53:21.541 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:53:21 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.crash"}]: dispatch 2024-04-12T21:53:21.541 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:53:21 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.crash"}]': finished 2024-04-12T21:53:21.541 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:53:21 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd"}]: dispatch 2024-04-12T21:53:21.541 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:53:21 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd"}]': finished 2024-04-12T21:53:21.541 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:53:21 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds"}]: dispatch 2024-04-12T21:53:21.541 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:53:21 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds"}]': finished 2024-04-12T21:53:21.541 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:53:21 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.rgw"}]: dispatch 2024-04-12T21:53:21.541 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:53:21 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.rgw"}]': finished 2024-04-12T21:53:21.542 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:53:21 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.rbd-mirror"}]: dispatch 2024-04-12T21:53:21.542 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:53:21 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.rbd-mirror"}]': finished 2024-04-12T21:53:21.542 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:53:21 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-04-12T21:53:21.542 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:53:21 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter"}]: dispatch 2024-04-12T21:53:21.542 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:53:21 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter"}]': finished 2024-04-12T21:53:21.542 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:53:21 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.iscsi"}]: dispatch 2024-04-12T21:53:21.542 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:53:21 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.iscsi"}]': finished 2024-04-12T21:53:21.542 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:53:21 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.nfs"}]: dispatch 2024-04-12T21:53:21.542 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:53:21 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.nfs"}]': finished 2024-04-12T21:53:21.542 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:53:21 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.nvmeof"}]: dispatch 2024-04-12T21:53:21.542 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:53:21 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.nvmeof"}]': finished 2024-04-12T21:53:21.542 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:53:21 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-04-12T21:53:21.543 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:53:21 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-04-12T21:53:21.543 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:53:21 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-04-12T21:53:21.543 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:53:21 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-04-12T21:53:21.543 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:53:21 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-04-12T21:53:21.543 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:53:21 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-04-12T21:53:21.543 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:53:21 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix":"config-key del","key":"mgr/cephadm/upgrade_state"}]: dispatch 2024-04-12T21:53:21.543 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:53:21 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd='[{"prefix":"config-key del","key":"mgr/cephadm/upgrade_state"}]': finished 2024-04-12T21:53:21.543 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:53:21 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-12T21:53:22.534 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:53:22 smithi186 ceph-mon[86379]: Upgrade: Setting filesystem cephfs Joinable 2024-04-12T21:53:22.535 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:53:22 smithi186 ceph-mon[86379]: from='client.36796 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:53:22.535 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:53:22 smithi186 ceph-mon[86379]: Upgrade: Setting container_image for all rgw 2024-04-12T21:53:22.535 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:53:22 smithi186 ceph-mon[86379]: Upgrade: Setting container_image for all rbd-mirror 2024-04-12T21:53:22.535 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:53:22 smithi186 ceph-mon[86379]: Upgrade: Setting container_image for all ceph-exporter 2024-04-12T21:53:22.535 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:53:22 smithi186 ceph-mon[86379]: Upgrade: Setting container_image for all iscsi 2024-04-12T21:53:22.535 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:53:22 smithi186 ceph-mon[86379]: Upgrade: Setting container_image for all nfs 2024-04-12T21:53:22.535 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:53:22 smithi186 ceph-mon[86379]: Upgrade: Setting container_image for all nvmeof 2024-04-12T21:53:22.535 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:53:22 smithi186 ceph-mon[86379]: Upgrade: Finalizing container_image settings 2024-04-12T21:53:22.535 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:53:22 smithi186 ceph-mon[86379]: Upgrade: Complete! 2024-04-12T21:53:22.535 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:53:22 smithi186 ceph-mon[86379]: from='client.? 172.21.15.18:0/2346350212' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-12T21:53:22.536 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:53:22 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' 2024-04-12T21:53:22.536 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:53:22 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' 2024-04-12T21:53:22.536 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:53:22 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' 2024-04-12T21:53:22.536 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:53:22 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' 2024-04-12T21:53:22.605 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:53:22 smithi018 ceph-mon[112590]: Upgrade: Setting filesystem cephfs Joinable 2024-04-12T21:53:22.606 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:53:22 smithi018 ceph-mon[112590]: from='client.36796 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:53:22.606 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:53:22 smithi018 ceph-mon[112590]: Upgrade: Setting container_image for all rgw 2024-04-12T21:53:22.606 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:53:22 smithi018 ceph-mon[112590]: Upgrade: Setting container_image for all rbd-mirror 2024-04-12T21:53:22.606 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:53:22 smithi018 ceph-mon[112590]: Upgrade: Setting container_image for all ceph-exporter 2024-04-12T21:53:22.606 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:53:22 smithi018 ceph-mon[112590]: Upgrade: Setting container_image for all iscsi 2024-04-12T21:53:22.606 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:53:22 smithi018 ceph-mon[112590]: Upgrade: Setting container_image for all nfs 2024-04-12T21:53:22.606 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:53:22 smithi018 ceph-mon[112590]: Upgrade: Setting container_image for all nvmeof 2024-04-12T21:53:22.606 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:53:22 smithi018 ceph-mon[112590]: Upgrade: Finalizing container_image settings 2024-04-12T21:53:22.606 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:53:22 smithi018 ceph-mon[112590]: Upgrade: Complete! 2024-04-12T21:53:22.606 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:53:22 smithi018 ceph-mon[112590]: from='client.? 172.21.15.18:0/2346350212' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-12T21:53:22.606 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:53:22 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' 2024-04-12T21:53:22.606 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:53:22 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' 2024-04-12T21:53:22.607 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:53:22 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' 2024-04-12T21:53:22.607 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:53:22 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' 2024-04-12T21:53:23.403 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:53:23 smithi018 ceph-mon[112590]: pgmap v1612: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail; 5.6 MiB/s rd, 2 op/s 2024-04-12T21:53:23.534 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:53:23 smithi186 ceph-mon[86379]: pgmap v1612: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail; 5.6 MiB/s rd, 2 op/s 2024-04-12T21:53:25.397 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:53:25 smithi018 ceph-mon[112590]: pgmap v1613: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail; 9.5 MiB/s rd, 3 op/s 2024-04-12T21:53:25.397 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:53:25 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' 2024-04-12T21:53:25.397 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:53:25 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T21:53:25.534 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:53:25 smithi186 ceph-mon[86379]: pgmap v1613: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail; 9.5 MiB/s rd, 3 op/s 2024-04-12T21:53:25.535 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:53:25 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' 2024-04-12T21:53:25.535 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:53:25 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T21:53:26.534 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:53:26 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' 2024-04-12T21:53:26.534 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:53:26 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' 2024-04-12T21:53:26.534 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:53:26 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' 2024-04-12T21:53:26.534 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:53:26 smithi186 ceph-mon[86379]: pgmap v1614: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail; 14 MiB/s rd, 6 op/s 2024-04-12T21:53:26.605 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:53:26 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' 2024-04-12T21:53:26.605 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:53:26 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' 2024-04-12T21:53:26.605 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:53:26 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' 2024-04-12T21:53:26.606 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:53:26 smithi018 ceph-mon[112590]: pgmap v1614: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail; 14 MiB/s rd, 6 op/s 2024-04-12T21:53:28.785 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:53:28 smithi186 ceph-mon[86379]: mds.? [v2:172.21.15.18:6826/2407992653,v1:172.21.15.18:6827/2407992653] up:resolve 2024-04-12T21:53:28.785 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:53:28 smithi186 ceph-mon[86379]: mds.? [v2:172.21.15.18:6828/755179764,v1:172.21.15.18:6829/755179764] up:resolve 2024-04-12T21:53:28.785 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:53:28 smithi186 ceph-mon[86379]: fsmap cephfs:2/2 {0=cephfs.smithi018.acqbsc=up:resolve,1=cephfs.smithi018.szjluo=up:resolve} 2 up:standby 2024-04-12T21:53:28.785 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:53:28 smithi186 ceph-mon[86379]: pgmap v1615: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail; 27 MiB/s rd, 9 op/s 2024-04-12T21:53:29.105 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:53:28 smithi018 ceph-mon[112590]: mds.? [v2:172.21.15.18:6826/2407992653,v1:172.21.15.18:6827/2407992653] up:resolve 2024-04-12T21:53:29.105 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:53:28 smithi018 ceph-mon[112590]: mds.? [v2:172.21.15.18:6828/755179764,v1:172.21.15.18:6829/755179764] up:resolve 2024-04-12T21:53:29.105 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:53:28 smithi018 ceph-mon[112590]: fsmap cephfs:2/2 {0=cephfs.smithi018.acqbsc=up:resolve,1=cephfs.smithi018.szjluo=up:resolve} 2 up:standby 2024-04-12T21:53:29.106 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:53:28 smithi018 ceph-mon[112590]: pgmap v1615: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail; 27 MiB/s rd, 9 op/s 2024-04-12T21:53:29.754 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:53:29 smithi186 ceph-mon[86379]: reconnect by client.14516 192.168.0.1:0/3927222862 after 0.00199998 2024-04-12T21:53:29.755 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:53:29 smithi186 ceph-mon[86379]: reconnect by client.14516 192.168.0.1:0/3927222862 after 0.00199998 2024-04-12T21:53:29.755 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:53:29 smithi186 ceph-mon[86379]: reconnect by client.24313 192.168.0.1:0/2102418092 after 0.00299996 2024-04-12T21:53:29.755 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:53:29 smithi186 ceph-mon[86379]: reconnect by client.24313 192.168.0.1:0/2102418092 after 0.00299996 2024-04-12T21:53:29.755 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:53:29 smithi186 ceph-mon[86379]: mds.? [v2:172.21.15.18:6828/755179764,v1:172.21.15.18:6829/755179764] up:reconnect 2024-04-12T21:53:29.755 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:53:29 smithi186 ceph-mon[86379]: mds.? [v2:172.21.15.18:6826/2407992653,v1:172.21.15.18:6827/2407992653] up:reconnect 2024-04-12T21:53:29.756 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:53:29 smithi186 ceph-mon[86379]: fsmap cephfs:2/2 {0=cephfs.smithi018.acqbsc=up:reconnect,1=cephfs.smithi018.szjluo=up:reconnect} 2 up:standby 2024-04-12T21:53:29.756 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:53:29 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' 2024-04-12T21:53:29.756 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:53:29 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' 2024-04-12T21:53:30.106 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:53:29 smithi018 ceph-mon[112590]: reconnect by client.14516 192.168.0.1:0/3927222862 after 0.00199998 2024-04-12T21:53:30.107 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:53:29 smithi018 ceph-mon[112590]: reconnect by client.14516 192.168.0.1:0/3927222862 after 0.00199998 2024-04-12T21:53:30.107 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:53:29 smithi018 ceph-mon[112590]: reconnect by client.24313 192.168.0.1:0/2102418092 after 0.00299996 2024-04-12T21:53:30.107 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:53:29 smithi018 ceph-mon[112590]: reconnect by client.24313 192.168.0.1:0/2102418092 after 0.00299996 2024-04-12T21:53:30.107 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:53:29 smithi018 ceph-mon[112590]: mds.? [v2:172.21.15.18:6828/755179764,v1:172.21.15.18:6829/755179764] up:reconnect 2024-04-12T21:53:30.107 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:53:29 smithi018 ceph-mon[112590]: mds.? [v2:172.21.15.18:6826/2407992653,v1:172.21.15.18:6827/2407992653] up:reconnect 2024-04-12T21:53:30.107 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:53:29 smithi018 ceph-mon[112590]: fsmap cephfs:2/2 {0=cephfs.smithi018.acqbsc=up:reconnect,1=cephfs.smithi018.szjluo=up:reconnect} 2 up:standby 2024-04-12T21:53:30.107 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:53:29 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' 2024-04-12T21:53:30.107 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:53:29 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' 2024-04-12T21:53:30.935 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:53:30 smithi186 ceph-mon[86379]: mds.? [v2:172.21.15.18:6826/2407992653,v1:172.21.15.18:6827/2407992653] up:rejoin 2024-04-12T21:53:30.936 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:53:30 smithi186 ceph-mon[86379]: mds.? [v2:172.21.15.18:6828/755179764,v1:172.21.15.18:6829/755179764] up:rejoin 2024-04-12T21:53:30.936 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:53:30 smithi186 ceph-mon[86379]: fsmap cephfs:2/2 {0=cephfs.smithi018.acqbsc=up:rejoin,1=cephfs.smithi018.szjluo=up:rejoin} 2 up:standby 2024-04-12T21:53:30.936 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:53:30 smithi186 ceph-mon[86379]: daemon mds.cephfs.smithi018.szjluo is now active in filesystem cephfs as rank 1 2024-04-12T21:53:30.936 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:53:30 smithi186 ceph-mon[86379]: daemon mds.cephfs.smithi018.acqbsc is now active in filesystem cephfs as rank 0 2024-04-12T21:53:30.936 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:53:30 smithi186 ceph-mon[86379]: pgmap v1616: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail; 25 MiB/s rd, 8 op/s 2024-04-12T21:53:31.105 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:53:30 smithi018 ceph-mon[112590]: mds.? [v2:172.21.15.18:6826/2407992653,v1:172.21.15.18:6827/2407992653] up:rejoin 2024-04-12T21:53:31.105 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:53:30 smithi018 ceph-mon[112590]: mds.? [v2:172.21.15.18:6828/755179764,v1:172.21.15.18:6829/755179764] up:rejoin 2024-04-12T21:53:31.105 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:53:30 smithi018 ceph-mon[112590]: fsmap cephfs:2/2 {0=cephfs.smithi018.acqbsc=up:rejoin,1=cephfs.smithi018.szjluo=up:rejoin} 2 up:standby 2024-04-12T21:53:31.105 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:53:30 smithi018 ceph-mon[112590]: daemon mds.cephfs.smithi018.szjluo is now active in filesystem cephfs as rank 1 2024-04-12T21:53:31.105 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:53:30 smithi018 ceph-mon[112590]: daemon mds.cephfs.smithi018.acqbsc is now active in filesystem cephfs as rank 0 2024-04-12T21:53:31.105 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:53:30 smithi018 ceph-mon[112590]: pgmap v1616: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail; 25 MiB/s rd, 8 op/s 2024-04-12T21:53:31.947 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:53:31 smithi186 ceph-mon[86379]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-04-12T21:53:31.947 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:53:31 smithi186 ceph-mon[86379]: mds.? [v2:172.21.15.18:6826/2407992653,v1:172.21.15.18:6827/2407992653] up:active 2024-04-12T21:53:31.948 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:53:31 smithi186 ceph-mon[86379]: mds.? [v2:172.21.15.18:6828/755179764,v1:172.21.15.18:6829/755179764] up:active 2024-04-12T21:53:31.948 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:53:31 smithi186 ceph-mon[86379]: fsmap cephfs:2 {0=cephfs.smithi018.acqbsc=up:active,1=cephfs.smithi018.szjluo=up:active} 2 up:standby 2024-04-12T21:53:32.106 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:53:31 smithi018 ceph-mon[112590]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-04-12T21:53:32.106 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:53:31 smithi018 ceph-mon[112590]: mds.? [v2:172.21.15.18:6826/2407992653,v1:172.21.15.18:6827/2407992653] up:active 2024-04-12T21:53:32.106 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:53:31 smithi018 ceph-mon[112590]: mds.? [v2:172.21.15.18:6828/755179764,v1:172.21.15.18:6829/755179764] up:active 2024-04-12T21:53:32.106 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:53:31 smithi018 ceph-mon[112590]: fsmap cephfs:2 {0=cephfs.smithi018.acqbsc=up:active,1=cephfs.smithi018.szjluo=up:active} 2 up:standby 2024-04-12T21:53:33.035 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:53:32 smithi186 ceph-mon[86379]: pgmap v1617: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail; 21 MiB/s rd, 341 B/s wr, 11 op/s 2024-04-12T21:53:33.035 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:53:32 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' 2024-04-12T21:53:33.035 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:53:32 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' 2024-04-12T21:53:33.097 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:53:32 smithi018 ceph-mon[112590]: pgmap v1617: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail; 21 MiB/s rd, 341 B/s wr, 11 op/s 2024-04-12T21:53:33.097 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:53:32 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' 2024-04-12T21:53:33.097 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:53:32 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' 2024-04-12T21:53:35.284 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:53:34 smithi186 ceph-mon[86379]: pgmap v1618: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail; 17 MiB/s rd, 426 B/s wr, 11 op/s 2024-04-12T21:53:35.356 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:53:34 smithi018 ceph-mon[112590]: pgmap v1618: 65 pgs: 65 active+clean; 257 MiB data, 971 MiB used, 535 GiB / 536 GiB avail; 17 MiB/s rd, 426 B/s wr, 11 op/s 2024-04-12T21:53:37.267 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:53:36 smithi018 ceph-mon[112590]: pgmap v1619: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail; 14 MiB/s rd, 767 B/s wr, 11 op/s 2024-04-12T21:53:37.284 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:53:36 smithi186 ceph-mon[86379]: pgmap v1619: 65 pgs: 65 active+clean; 257 MiB data, 972 MiB used, 535 GiB / 536 GiB avail; 14 MiB/s rd, 767 B/s wr, 11 op/s 2024-04-12T21:53:39.071 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:53:38 smithi018 ceph-mon[112590]: pgmap v1620: 65 pgs: 65 active+clean; 257 MiB data, 976 MiB used, 535 GiB / 536 GiB avail; 10 MiB/s rd, 4.7 KiB/s wr, 11 op/s 2024-04-12T21:53:39.284 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:53:38 smithi186 ceph-mon[86379]: pgmap v1620: 65 pgs: 65 active+clean; 257 MiB data, 976 MiB used, 535 GiB / 536 GiB avail; 10 MiB/s rd, 4.7 KiB/s wr, 11 op/s 2024-04-12T21:53:41.230 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:53:40 smithi018 ceph-mon[112590]: pgmap v1621: 65 pgs: 65 active+clean; 257 MiB data, 976 MiB used, 535 GiB / 536 GiB avail; 119 KiB/s rd, 9.1 KiB/s wr, 10 op/s 2024-04-12T21:53:41.230 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:53:40 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' 2024-04-12T21:53:41.230 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:53:40 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T21:53:41.284 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:53:40 smithi186 ceph-mon[86379]: pgmap v1621: 65 pgs: 65 active+clean; 257 MiB data, 976 MiB used, 535 GiB / 536 GiB avail; 119 KiB/s rd, 9.1 KiB/s wr, 10 op/s 2024-04-12T21:53:41.285 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:53:40 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' 2024-04-12T21:53:41.285 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:53:40 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T21:53:43.356 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:53:43 smithi018 ceph-mon[112590]: pgmap v1622: 65 pgs: 65 active+clean; 257 MiB data, 976 MiB used, 535 GiB / 536 GiB avail; 6.4 KiB/s rd, 9.1 KiB/s wr, 10 op/s 2024-04-12T21:53:43.356 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:53:43 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' 2024-04-12T21:53:43.356 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:53:43 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' 2024-04-12T21:53:43.356 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:53:43 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-12T21:53:43.356 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:53:43 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-12T21:53:43.356 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:53:43 smithi018 ceph-mon[112590]: pgmap v1623: 65 pgs: 65 active+clean; 257 MiB data, 976 MiB used, 535 GiB / 536 GiB avail; 2.3 KiB/s rd, 10 KiB/s wr, 7 op/s 2024-04-12T21:53:43.356 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:53:43 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' 2024-04-12T21:53:43.357 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:53:43 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-12T21:53:43.357 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:53:43 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-12T21:53:43.357 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:53:43 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-12T21:53:43.357 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:53:43 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' 2024-04-12T21:53:43.535 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:53:43 smithi186 ceph-mon[86379]: pgmap v1622: 65 pgs: 65 active+clean; 257 MiB data, 976 MiB used, 535 GiB / 536 GiB avail; 6.4 KiB/s rd, 9.1 KiB/s wr, 10 op/s 2024-04-12T21:53:43.535 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:53:43 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' 2024-04-12T21:53:43.535 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:53:43 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' 2024-04-12T21:53:43.535 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:53:43 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-12T21:53:43.535 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:53:43 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-12T21:53:43.535 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:53:43 smithi186 ceph-mon[86379]: pgmap v1623: 65 pgs: 65 active+clean; 257 MiB data, 976 MiB used, 535 GiB / 536 GiB avail; 2.3 KiB/s rd, 10 KiB/s wr, 7 op/s 2024-04-12T21:53:43.535 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:53:43 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' 2024-04-12T21:53:43.535 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:53:43 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-12T21:53:43.536 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:53:43 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-12T21:53:43.536 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:53:43 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-12T21:53:43.536 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:53:43 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' 2024-04-12T21:53:44.355 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:53:44 smithi018 ceph-mon[112590]: Health check cleared: CEPHADM_REFRESH_FAILED (was: failed to probe daemons or devices) 2024-04-12T21:53:44.356 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:53:44 smithi018 ceph-mon[112590]: Cluster is now healthy 2024-04-12T21:53:44.534 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:53:44 smithi186 ceph-mon[86379]: Health check cleared: CEPHADM_REFRESH_FAILED (was: failed to probe daemons or devices) 2024-04-12T21:53:44.535 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:53:44 smithi186 ceph-mon[86379]: Cluster is now healthy 2024-04-12T21:53:45.386 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:53:45 smithi018 ceph-mon[112590]: pgmap v1624: 65 pgs: 65 active+clean; 257 MiB data, 976 MiB used, 535 GiB / 536 GiB avail; 596 B/s rd, 10 KiB/s wr, 5 op/s 2024-04-12T21:53:45.534 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:53:45 smithi186 ceph-mon[86379]: pgmap v1624: 65 pgs: 65 active+clean; 257 MiB data, 976 MiB used, 535 GiB / 536 GiB avail; 596 B/s rd, 10 KiB/s wr, 5 op/s 2024-04-12T21:53:46.356 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:53:46 smithi018 ceph-mon[112590]: pgmap v1625: 65 pgs: 65 active+clean; 257 MiB data, 976 MiB used, 535 GiB / 536 GiB avail; 9.7 KiB/s wr, 4 op/s 2024-04-12T21:53:46.534 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:53:46 smithi186 ceph-mon[86379]: pgmap v1625: 65 pgs: 65 active+clean; 257 MiB data, 976 MiB used, 535 GiB / 536 GiB avail; 9.7 KiB/s wr, 4 op/s 2024-04-12T21:53:48.535 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:53:48 smithi186 ceph-mon[86379]: pgmap v1626: 65 pgs: 65 active+clean; 257 MiB data, 976 MiB used, 535 GiB / 536 GiB avail; 5.1 KiB/s wr, 2 op/s 2024-04-12T21:53:48.605 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:53:48 smithi018 ceph-mon[112590]: pgmap v1626: 65 pgs: 65 active+clean; 257 MiB data, 976 MiB used, 535 GiB / 536 GiB avail; 5.1 KiB/s wr, 2 op/s 2024-04-12T21:53:50.534 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:53:50 smithi186 ceph-mon[86379]: pgmap v1627: 65 pgs: 65 active+clean; 257 MiB data, 976 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:53:50.605 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:53:50 smithi018 ceph-mon[112590]: pgmap v1627: 65 pgs: 65 active+clean; 257 MiB data, 976 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:53:51.504 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:53:51.498+0000 7ff835c00700 1 -- 172.21.15.18:0/1041440088 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff8301059c0 msgr2=0x7ff8300ff080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:53:51.505 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:53:51.498+0000 7ff835c00700 1 --2- 172.21.15.18:0/1041440088 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff8301059c0 0x7ff8300ff080 secure :-1 s=READY pgs=1366 cs=0 l=1 rev1=1 crypto rx=0x7ff818004940 tx=0x7ff818009e30 comp rx=0 tx=0).stop 2024-04-12T21:53:51.505 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:53:51.499+0000 7ff835c00700 1 -- 172.21.15.18:0/1041440088 shutdown_connections 2024-04-12T21:53:51.505 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:53:51.499+0000 7ff835c00700 1 --2- 172.21.15.18:0/1041440088 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff8301059c0 0x7ff8300ff080 unknown :-1 s=CLOSED pgs=1366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:53:51.505 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:53:51.499+0000 7ff835c00700 1 --2- 172.21.15.18:0/1041440088 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff830105020 0x7ff8301053f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:53:51.505 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:53:51.499+0000 7ff835c00700 1 -- 172.21.15.18:0/1041440088 >> 172.21.15.18:0/1041440088 conn(0x7ff8300faba0 msgr2=0x7ff8300fcfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:53:51.505 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:53:51.499+0000 7ff835c00700 1 -- 172.21.15.18:0/1041440088 shutdown_connections 2024-04-12T21:53:51.505 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:53:51.499+0000 7ff835c00700 1 -- 172.21.15.18:0/1041440088 wait complete. 2024-04-12T21:53:51.505 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:53:51.500+0000 7ff835c00700 1 Processor -- start 2024-04-12T21:53:51.505 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:53:51.500+0000 7ff835c00700 1 -- start start 2024-04-12T21:53:51.505 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:53:51.500+0000 7ff835c00700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff830105020 0x7ff830115ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:53:51.506 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:53:51.500+0000 7ff835c00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff8301059c0 0x7ff8301160e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:53:51.506 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:53:51.500+0000 7ff835c00700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff830116730 con 0x7ff8301059c0 2024-04-12T21:53:51.506 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:53:51.500+0000 7ff835c00700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff83010fd10 con 0x7ff830105020 2024-04-12T21:53:51.506 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:53:51.501+0000 7ff827e00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff8301059c0 0x7ff8301160e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:53:51.506 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:53:51.501+0000 7ff827e00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff8301059c0 0x7ff8301160e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:45298/0 (socket says 172.21.15.18:45298) 2024-04-12T21:53:51.506 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:53:51.501+0000 7ff827e00700 1 -- 172.21.15.18:0/1468901444 learned_addr learned my addr 172.21.15.18:0/1468901444 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:53:51.506 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:53:51.501+0000 7ff82ea00700 1 --2- 172.21.15.18:0/1468901444 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff830105020 0x7ff830115ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:53:51.506 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:53:51.501+0000 7ff827e00700 1 -- 172.21.15.18:0/1468901444 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff830105020 msgr2=0x7ff830115ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:53:51.506 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:53:51.501+0000 7ff827e00700 1 --2- 172.21.15.18:0/1468901444 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff830105020 0x7ff830115ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:53:51.506 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:53:51.501+0000 7ff827e00700 1 -- 172.21.15.18:0/1468901444 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff818005040 con 0x7ff8301059c0 2024-04-12T21:53:51.506 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:53:51.501+0000 7ff82ea00700 1 --2- 172.21.15.18:0/1468901444 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff830105020 0x7ff830115ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-12T21:53:51.507 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:53:51.501+0000 7ff827e00700 1 --2- 172.21.15.18:0/1468901444 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff8301059c0 0x7ff8301160e0 secure :-1 s=READY pgs=1367 cs=0 l=1 rev1=1 crypto rx=0x7ff818009100 tx=0x7ff818009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:53:51.507 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:53:51.502+0000 7ff826a00700 1 -- 172.21.15.18:0/1468901444 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff81800e040 con 0x7ff8301059c0 2024-04-12T21:53:51.507 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:53:51.502+0000 7ff835c00700 1 -- 172.21.15.18:0/1468901444 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff83010ff90 con 0x7ff8301059c0 2024-04-12T21:53:51.507 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:53:51.502+0000 7ff826a00700 1 -- 172.21.15.18:0/1468901444 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff818013420 con 0x7ff8301059c0 2024-04-12T21:53:51.507 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:53:51.502+0000 7ff826a00700 1 -- 172.21.15.18:0/1468901444 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff81801c630 con 0x7ff8301059c0 2024-04-12T21:53:51.507 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:53:51.502+0000 7ff835c00700 1 -- 172.21.15.18:0/1468901444 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff830110550 con 0x7ff8301059c0 2024-04-12T21:53:51.508 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:53:51.503+0000 7ff835c00700 1 -- 172.21.15.18:0/1468901444 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff830100770 con 0x7ff8301059c0 2024-04-12T21:53:51.508 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:53:51.504+0000 7ff826a00700 1 -- 172.21.15.18:0/1468901444 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7ff818013590 con 0x7ff8301059c0 2024-04-12T21:53:51.508 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:53:51.505+0000 7ff826a00700 1 --2- 172.21.15.18:0/1468901444 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7ff81006ec00 0x7ff8100710b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:53:51.508 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:53:51.505+0000 7ff826a00700 1 -- 172.21.15.18:0/1468901444 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 6757+0+0 (secure 0 0 0) 0x7ff81808fba0 con 0x7ff8301059c0 2024-04-12T21:53:51.508 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:53:51.506+0000 7ff82ea00700 1 --2- 172.21.15.18:0/1468901444 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7ff81006ec00 0x7ff8100710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:53:51.509 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:53:51.506+0000 7ff82ea00700 1 --2- 172.21.15.18:0/1468901444 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7ff81006ec00 0x7ff8100710b0 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7ff820009fd0 tx=0x7ff82000b040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:53:51.513 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:53:51.511+0000 7ff826a00700 1 -- 172.21.15.18:0/1468901444 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7ff81805c3d0 con 0x7ff8301059c0 2024-04-12T21:53:51.766 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:53:51.763+0000 7ff835c00700 1 -- 172.21.15.18:0/1468901444 --> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7ff83010c170 con 0x7ff81006ec00 2024-04-12T21:53:51.769 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:53:51.766+0000 7ff826a00700 1 -- 172.21.15.18:0/1468901444 <== mgr.34102 v2:172.21.15.18:6800/3812891919 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+175 (secure 0 0 0) 0x7ff83010c170 con 0x7ff81006ec00 2024-04-12T21:53:51.772 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:53:51.769+0000 7ff835c00700 1 -- 172.21.15.18:0/1468901444 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7ff81006ec00 msgr2=0x7ff8100710b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:53:51.772 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:53:51.769+0000 7ff835c00700 1 --2- 172.21.15.18:0/1468901444 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7ff81006ec00 0x7ff8100710b0 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7ff820009fd0 tx=0x7ff82000b040 comp rx=0 tx=0).stop 2024-04-12T21:53:51.772 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:53:51.769+0000 7ff835c00700 1 -- 172.21.15.18:0/1468901444 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff8301059c0 msgr2=0x7ff8301160e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:53:51.772 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:53:51.769+0000 7ff835c00700 1 --2- 172.21.15.18:0/1468901444 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff8301059c0 0x7ff8301160e0 secure :-1 s=READY pgs=1367 cs=0 l=1 rev1=1 crypto rx=0x7ff818009100 tx=0x7ff818009e30 comp rx=0 tx=0).stop 2024-04-12T21:53:51.772 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:53:51.769+0000 7ff835c00700 1 -- 172.21.15.18:0/1468901444 shutdown_connections 2024-04-12T21:53:51.773 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:53:51.769+0000 7ff835c00700 1 --2- 172.21.15.18:0/1468901444 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7ff81006ec00 0x7ff8100710b0 unknown :-1 s=CLOSED pgs=725 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:53:51.773 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:53:51.769+0000 7ff835c00700 1 --2- 172.21.15.18:0/1468901444 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff830105020 0x7ff830115ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:53:51.773 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:53:51.769+0000 7ff835c00700 1 --2- 172.21.15.18:0/1468901444 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff8301059c0 0x7ff8301160e0 unknown :-1 s=CLOSED pgs=1367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:53:51.773 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:53:51.769+0000 7ff835c00700 1 -- 172.21.15.18:0/1468901444 >> 172.21.15.18:0/1468901444 conn(0x7ff8300faba0 msgr2=0x7ff8300fbb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:53:51.773 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:53:51.769+0000 7ff835c00700 1 -- 172.21.15.18:0/1468901444 shutdown_connections 2024-04-12T21:53:51.773 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:53:51.769+0000 7ff835c00700 1 -- 172.21.15.18:0/1468901444 wait complete. 2024-04-12T21:53:52.356 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:53:52 smithi018 ceph-mon[112590]: from='client.36804 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:53:52.356 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:53:52 smithi018 ceph-mon[112590]: pgmap v1628: 65 pgs: 65 active+clean; 257 MiB data, 976 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:53:52.521 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1cbe407a-f90d-11ee-bc8d-c7b262605968 -e sha1=324baf8ddb91dea46cac50cb535c306115791355 -- bash -c 'ceph orch ps' 2024-04-12T21:53:52.534 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:53:52 smithi186 ceph-mon[86379]: from='client.36804 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:53:52.535 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:53:52 smithi186 ceph-mon[86379]: pgmap v1628: 65 pgs: 65 active+clean; 257 MiB data, 976 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:53:53.067 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/mon.smithi018/config 2024-04-12T21:53:54.356 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:53:54 smithi018 ceph-mon[112590]: pgmap v1629: 65 pgs: 65 active+clean; 257 MiB data, 976 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:53:54.534 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:53:54 smithi186 ceph-mon[86379]: pgmap v1629: 65 pgs: 65 active+clean; 257 MiB data, 976 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:53:55.307 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:53:55.305+0000 7fb551400700 1 -- 172.21.15.18:0/1903932777 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb54c0fddf0 msgr2=0x7fb54c0fe1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:53:55.307 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:53:55.305+0000 7fb551400700 1 --2- 172.21.15.18:0/1903932777 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb54c0fddf0 0x7fb54c0fe1c0 secure :-1 s=READY pgs=1368 cs=0 l=1 rev1=1 crypto rx=0x7fb53c004940 tx=0x7fb53c009e30 comp rx=0 tx=0).stop 2024-04-12T21:53:55.308 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:53:55.305+0000 7fb551400700 1 -- 172.21.15.18:0/1903932777 shutdown_connections 2024-04-12T21:53:55.308 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:53:55.305+0000 7fb551400700 1 --2- 172.21.15.18:0/1903932777 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb54c0fe790 0x7fb54c106dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:53:55.308 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:53:55.305+0000 7fb551400700 1 --2- 172.21.15.18:0/1903932777 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb54c0fddf0 0x7fb54c0fe1c0 unknown :-1 s=CLOSED pgs=1368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:53:55.308 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:53:55.305+0000 7fb551400700 1 -- 172.21.15.18:0/1903932777 >> 172.21.15.18:0/1903932777 conn(0x7fb54c0f7860 msgr2=0x7fb54c0f9c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:53:55.308 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:53:55.305+0000 7fb551400700 1 -- 172.21.15.18:0/1903932777 shutdown_connections 2024-04-12T21:53:55.308 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:53:55.305+0000 7fb551400700 1 -- 172.21.15.18:0/1903932777 wait complete. 2024-04-12T21:53:55.309 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:53:55.306+0000 7fb551400700 1 Processor -- start 2024-04-12T21:53:55.309 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:53:55.306+0000 7fb551400700 1 -- start start 2024-04-12T21:53:55.309 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:53:55.306+0000 7fb551400700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb54c0fddf0 0x7fb54c115d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:53:55.309 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:53:55.306+0000 7fb551400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb54c0fe790 0x7fb54c1162b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:53:55.310 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:53:55.306+0000 7fb551400700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb54c116900 con 0x7fb54c0fe790 2024-04-12T21:53:55.310 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:53:55.306+0000 7fb551400700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb54c10fcd0 con 0x7fb54c0fddf0 2024-04-12T21:53:55.310 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:53:55.307+0000 7fb549e00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb54c0fe790 0x7fb54c1162b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:53:55.310 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:53:55.307+0000 7fb549e00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb54c0fe790 0x7fb54c1162b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:45324/0 (socket says 172.21.15.18:45324) 2024-04-12T21:53:55.310 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:53:55.307+0000 7fb549e00700 1 -- 172.21.15.18:0/195658962 learned_addr learned my addr 172.21.15.18:0/195658962 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:53:55.310 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:53:55.307+0000 7fb54a800700 1 --2- 172.21.15.18:0/195658962 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb54c0fddf0 0x7fb54c115d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:53:55.310 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:53:55.307+0000 7fb549e00700 1 -- 172.21.15.18:0/195658962 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb54c0fddf0 msgr2=0x7fb54c115d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:53:55.310 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:53:55.307+0000 7fb549e00700 1 --2- 172.21.15.18:0/195658962 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb54c0fddf0 0x7fb54c115d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:53:55.310 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:53:55.307+0000 7fb549e00700 1 -- 172.21.15.18:0/195658962 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb53c005040 con 0x7fb54c0fe790 2024-04-12T21:53:55.310 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:53:55.307+0000 7fb54a800700 1 --2- 172.21.15.18:0/195658962 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb54c0fddf0 0x7fb54c115d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-12T21:53:55.311 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:53:55.308+0000 7fb549e00700 1 --2- 172.21.15.18:0/195658962 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb54c0fe790 0x7fb54c1162b0 secure :-1 s=READY pgs=1369 cs=0 l=1 rev1=1 crypto rx=0x7fb53800c8c0 tx=0x7fb5380044c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:53:55.311 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:53:55.308+0000 7fb542a00700 1 -- 172.21.15.18:0/195658962 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb53800a980 con 0x7fb54c0fe790 2024-04-12T21:53:55.311 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:53:55.308+0000 7fb542a00700 1 -- 172.21.15.18:0/195658962 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb53800ce60 con 0x7fb54c0fe790 2024-04-12T21:53:55.311 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:53:55.308+0000 7fb551400700 1 -- 172.21.15.18:0/195658962 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb54c10ffb0 con 0x7fb54c0fe790 2024-04-12T21:53:55.311 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:53:55.308+0000 7fb542a00700 1 -- 172.21.15.18:0/195658962 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb5380115e0 con 0x7fb54c0fe790 2024-04-12T21:53:55.311 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:53:55.308+0000 7fb551400700 1 -- 172.21.15.18:0/195658962 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb54c1104f0 con 0x7fb54c0fe790 2024-04-12T21:53:55.311 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:53:55.309+0000 7fb551400700 1 -- 172.21.15.18:0/195658962 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb54c066770 con 0x7fb54c0fe790 2024-04-12T21:53:55.313 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:53:55.311+0000 7fb542a00700 1 -- 172.21.15.18:0/195658962 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7fb538012400 con 0x7fb54c0fe790 2024-04-12T21:53:55.314 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:53:55.312+0000 7fb542a00700 1 --2- 172.21.15.18:0/195658962 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fb53006ec10 0x7fb5300710c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:53:55.314 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:53:55.312+0000 7fb542a00700 1 -- 172.21.15.18:0/195658962 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 6757+0+0 (secure 0 0 0) 0x7fb53808ea40 con 0x7fb54c0fe790 2024-04-12T21:53:55.314 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:53:55.312+0000 7fb54a800700 1 --2- 172.21.15.18:0/195658962 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fb53006ec10 0x7fb5300710c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:53:55.315 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:53:55.313+0000 7fb54a800700 1 --2- 172.21.15.18:0/195658962 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fb53006ec10 0x7fb5300710c0 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7fb53c009fd0 tx=0x7fb53c009e30 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:53:55.319 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:53:55.317+0000 7fb542a00700 1 -- 172.21.15.18:0/195658962 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7fb53805b270 con 0x7fb54c0fe790 2024-04-12T21:53:55.563 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:53:55.561+0000 7fb551400700 1 -- 172.21.15.18:0/195658962 --> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fb54c04bed0 con 0x7fb53006ec10 2024-04-12T21:53:55.594 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:53:55.592+0000 7fb542a00700 1 -- 172.21.15.18:0/195658962 <== mgr.34102 v2:172.21.15.18:6800/3812891919 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7fb54c04bed0 con 0x7fb53006ec10 2024-04-12T21:53:55.595 INFO:teuthology.orchestra.run.smithi018.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-12T21:53:55.595 INFO:teuthology.orchestra.run.smithi018.stdout:alertmanager.smithi018 smithi018 *:9093,9094 running (54m) 26s ago 68m 14.6M - 0.25.0 c8568f914cd2 c332090068e8 2024-04-12T21:53:55.595 INFO:teuthology.orchestra.run.smithi018.stdout:crash.smithi018 smithi018 running (50m) 26s ago 69m 6665k - 19.0.0-2879-g324baf8d 8717720dc5e5 b7f80d1f7982 2024-04-12T21:53:55.595 INFO:teuthology.orchestra.run.smithi018.stdout:crash.smithi186 smithi186 running (50m) 30s ago 67m 6660k - 19.0.0-2879-g324baf8d 8717720dc5e5 a4f55c81b623 2024-04-12T21:53:55.595 INFO:teuthology.orchestra.run.smithi018.stdout:grafana.smithi018 smithi018 *:3000 running (53m) 26s ago 68m 75.1M - 9.4.12 2bacad6d85d8 b08e3bf176dd 2024-04-12T21:53:55.595 INFO:teuthology.orchestra.run.smithi018.stdout:mds.cephfs.smithi018.acqbsc smithi018 running (46m) 26s ago 61m 102M - 19.0.0-2879-g324baf8d 8717720dc5e5 021f99ab24ae 2024-04-12T21:53:55.595 INFO:teuthology.orchestra.run.smithi018.stdout:mds.cephfs.smithi018.szjluo smithi018 running (46m) 26s ago 61m 106M - 19.0.0-2879-g324baf8d 8717720dc5e5 0003117479d4 2024-04-12T21:53:55.595 INFO:teuthology.orchestra.run.smithi018.stdout:mds.cephfs.smithi186.jdllyn smithi186 running (30m) 30s ago 61m 21.7M - 19.0.0-2879-g324baf8d 8717720dc5e5 21b1f46eef23 2024-04-12T21:53:55.596 INFO:teuthology.orchestra.run.smithi018.stdout:mds.cephfs.smithi186.wjnafs smithi186 running (15m) 30s ago 61m 16.5M - 19.0.0-2879-g324baf8d 8717720dc5e5 3cf1ef4148e2 2024-04-12T21:53:55.596 INFO:teuthology.orchestra.run.smithi018.stdout:mgr.smithi018.xjzwga smithi018 *:8443,9283,8765 running (57m) 26s ago 70m 521M - 19.0.0-2879-g324baf8d 8717720dc5e5 56088305eab7 2024-04-12T21:53:55.596 INFO:teuthology.orchestra.run.smithi018.stdout:mgr.smithi186.kqcfxk smithi186 *:8443,9283,8765 running (55m) 30s ago 67m 460M - 19.0.0-2879-g324baf8d 8717720dc5e5 09cd766e5e0d 2024-04-12T21:53:55.596 INFO:teuthology.orchestra.run.smithi018.stdout:mon.smithi018 smithi018 running (52m) 26s ago 70m 104M 2048M 19.0.0-2879-g324baf8d 8717720dc5e5 df3e201dfed6 2024-04-12T21:53:55.596 INFO:teuthology.orchestra.run.smithi018.stdout:mon.smithi186 smithi186 running (51m) 30s ago 67m 62.2M 2048M 19.0.0-2879-g324baf8d 8717720dc5e5 c77498da9b2b 2024-04-12T21:53:55.596 INFO:teuthology.orchestra.run.smithi018.stdout:node-exporter.smithi018 smithi018 *:9100 running (55m) 26s ago 69m 8812k - 1.5.0 0da6a335fe13 326c3d9c6847 2024-04-12T21:53:55.596 INFO:teuthology.orchestra.run.smithi018.stdout:node-exporter.smithi186 smithi186 *:9100 running (55m) 30s ago 67m 8786k - 1.5.0 0da6a335fe13 ec2112baeecf 2024-04-12T21:53:55.596 INFO:teuthology.orchestra.run.smithi018.stdout:osd.0 smithi018 running (50m) 26s ago 66m 117M 1559M 19.0.0-2879-g324baf8d 8717720dc5e5 60b1273ca047 2024-04-12T21:53:55.596 INFO:teuthology.orchestra.run.smithi018.stdout:osd.1 smithi018 running (49m) 26s ago 66m 102M 1559M 19.0.0-2879-g324baf8d 8717720dc5e5 170ce1e5fb8d 2024-04-12T21:53:55.596 INFO:teuthology.orchestra.run.smithi018.stdout:osd.2 smithi018 running (49m) 26s ago 65m 99.8M 1559M 19.0.0-2879-g324baf8d 8717720dc5e5 c2cbc6aa2839 2024-04-12T21:53:55.596 INFO:teuthology.orchestra.run.smithi018.stdout:osd.3 smithi186 running (48m) 30s ago 64m 107M 2581M 19.0.0-2879-g324baf8d 8717720dc5e5 0138e8262e4b 2024-04-12T21:53:55.596 INFO:teuthology.orchestra.run.smithi018.stdout:osd.4 smithi186 running (47m) 30s ago 64m 94.8M 2581M 19.0.0-2879-g324baf8d 8717720dc5e5 b20cb63b25c9 2024-04-12T21:53:55.596 INFO:teuthology.orchestra.run.smithi018.stdout:osd.5 smithi186 running (47m) 30s ago 63m 98.7M 2581M 19.0.0-2879-g324baf8d 8717720dc5e5 2513bea2610e 2024-04-12T21:53:55.596 INFO:teuthology.orchestra.run.smithi018.stdout:prometheus.smithi018 smithi018 *:9095 running (54m) 26s ago 68m 74.2M - 2.43.0 a07b618ecd1d f58841127e21 2024-04-12T21:53:55.598 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:53:55.595+0000 7fb551400700 1 -- 172.21.15.18:0/195658962 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fb53006ec10 msgr2=0x7fb5300710c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:53:55.598 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:53:55.595+0000 7fb551400700 1 --2- 172.21.15.18:0/195658962 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fb53006ec10 0x7fb5300710c0 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7fb53c009fd0 tx=0x7fb53c009e30 comp rx=0 tx=0).stop 2024-04-12T21:53:55.598 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:53:55.595+0000 7fb551400700 1 -- 172.21.15.18:0/195658962 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb54c0fe790 msgr2=0x7fb54c1162b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:53:55.598 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:53:55.595+0000 7fb551400700 1 --2- 172.21.15.18:0/195658962 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb54c0fe790 0x7fb54c1162b0 secure :-1 s=READY pgs=1369 cs=0 l=1 rev1=1 crypto rx=0x7fb53800c8c0 tx=0x7fb5380044c0 comp rx=0 tx=0).stop 2024-04-12T21:53:55.598 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:53:55.595+0000 7fb551400700 1 -- 172.21.15.18:0/195658962 shutdown_connections 2024-04-12T21:53:55.598 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:53:55.595+0000 7fb551400700 1 --2- 172.21.15.18:0/195658962 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fb53006ec10 0x7fb5300710c0 unknown :-1 s=CLOSED pgs=726 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:53:55.598 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:53:55.595+0000 7fb551400700 1 --2- 172.21.15.18:0/195658962 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb54c0fddf0 0x7fb54c115d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:53:55.598 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:53:55.595+0000 7fb551400700 1 --2- 172.21.15.18:0/195658962 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb54c0fe790 0x7fb54c1162b0 unknown :-1 s=CLOSED pgs=1369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:53:55.598 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:53:55.595+0000 7fb551400700 1 -- 172.21.15.18:0/195658962 >> 172.21.15.18:0/195658962 conn(0x7fb54c0f7860 msgr2=0x7fb54c0f8e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:53:55.599 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:53:55.596+0000 7fb551400700 1 -- 172.21.15.18:0/195658962 shutdown_connections 2024-04-12T21:53:55.599 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:53:55.596+0000 7fb551400700 1 -- 172.21.15.18:0/195658962 wait complete. 2024-04-12T21:53:56.284 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:53:55 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' 2024-04-12T21:53:56.285 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:53:55 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T21:53:56.303 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:53:55 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' 2024-04-12T21:53:56.303 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:53:55 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T21:53:56.366 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1cbe407a-f90d-11ee-bc8d-c7b262605968 -e sha1=324baf8ddb91dea46cac50cb535c306115791355 -- bash -c 'ceph orch upgrade status' 2024-04-12T21:53:56.881 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/mon.smithi018/config 2024-04-12T21:53:57.224 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:53:56 smithi018 ceph-mon[112590]: from='client.36808 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:53:57.224 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:53:56 smithi018 ceph-mon[112590]: pgmap v1630: 65 pgs: 65 active+clean; 257 MiB data, 976 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:53:57.284 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:53:56 smithi186 ceph-mon[86379]: from='client.36808 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:53:57.285 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:53:56 smithi186 ceph-mon[86379]: pgmap v1630: 65 pgs: 65 active+clean; 257 MiB data, 976 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:53:58.534 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:53:58 smithi186 ceph-mon[86379]: pgmap v1631: 65 pgs: 65 active+clean; 257 MiB data, 976 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:53:58.605 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:53:58 smithi018 ceph-mon[112590]: pgmap v1631: 65 pgs: 65 active+clean; 257 MiB data, 976 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:53:59.207 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:53:59.204+0000 7f35f7200700 1 -- 172.21.15.18:0/471334504 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f35f0109dd0 msgr2=0x7f35f0103440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:53:59.208 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:53:59.204+0000 7f35f7200700 1 --2- 172.21.15.18:0/471334504 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f35f0109dd0 0x7f35f0103440 secure :-1 s=READY pgs=1370 cs=0 l=1 rev1=1 crypto rx=0x7f35d8004940 tx=0x7f35d8009e30 comp rx=0 tx=0).stop 2024-04-12T21:53:59.208 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:53:59.204+0000 7f35f7200700 1 -- 172.21.15.18:0/471334504 shutdown_connections 2024-04-12T21:53:59.208 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:53:59.204+0000 7f35f7200700 1 --2- 172.21.15.18:0/471334504 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f35f0109dd0 0x7f35f0103440 unknown :-1 s=CLOSED pgs=1370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:53:59.208 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:53:59.204+0000 7f35f7200700 1 --2- 172.21.15.18:0/471334504 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f35f0109430 0x7f35f0109800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:53:59.208 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:53:59.204+0000 7f35f7200700 1 -- 172.21.15.18:0/471334504 >> 172.21.15.18:0/471334504 conn(0x7f35f0072c00 msgr2=0x7f35f0075010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:53:59.208 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:53:59.204+0000 7f35f7200700 1 -- 172.21.15.18:0/471334504 shutdown_connections 2024-04-12T21:53:59.208 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:53:59.204+0000 7f35f7200700 1 -- 172.21.15.18:0/471334504 wait complete. 2024-04-12T21:53:59.208 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:53:59.205+0000 7f35f7200700 1 Processor -- start 2024-04-12T21:53:59.208 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:53:59.205+0000 7f35f7200700 1 -- start start 2024-04-12T21:53:59.209 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:53:59.206+0000 7f35f7200700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f35f0109430 0x7f35f0119f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:53:59.210 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:53:59.206+0000 7f35f7200700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f35f0109dd0 0x7f35f011a4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:53:59.210 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:53:59.206+0000 7f35f7200700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35f011ab10 con 0x7f35f0109430 2024-04-12T21:53:59.210 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:53:59.206+0000 7f35f7200700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35f01140f0 con 0x7f35f0109dd0 2024-04-12T21:53:59.210 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:53:59.206+0000 7f35efe00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f35f0109430 0x7f35f0119f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:53:59.210 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:53:59.206+0000 7f35efe00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f35f0109430 0x7f35f0119f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:45342/0 (socket says 172.21.15.18:45342) 2024-04-12T21:53:59.210 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:53:59.206+0000 7f35efe00700 1 -- 172.21.15.18:0/2101376257 learned_addr learned my addr 172.21.15.18:0/2101376257 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:53:59.210 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:53:59.207+0000 7f35e7e00700 1 --2- 172.21.15.18:0/2101376257 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f35f0109dd0 0x7f35f011a4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:53:59.210 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:53:59.207+0000 7f35efe00700 1 -- 172.21.15.18:0/2101376257 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f35f0109dd0 msgr2=0x7f35f011a4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:53:59.210 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:53:59.207+0000 7f35efe00700 1 --2- 172.21.15.18:0/2101376257 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f35f0109dd0 0x7f35f011a4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:53:59.210 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:53:59.207+0000 7f35efe00700 1 -- 172.21.15.18:0/2101376257 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35d8005040 con 0x7f35f0109430 2024-04-12T21:53:59.210 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:53:59.207+0000 7f35efe00700 1 --2- 172.21.15.18:0/2101376257 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f35f0109430 0x7f35f0119f80 secure :-1 s=READY pgs=1371 cs=0 l=1 rev1=1 crypto rx=0x7f35e0002a50 tx=0x7f35e00046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:53:59.210 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:53:59.207+0000 7f35ecc00700 1 -- 172.21.15.18:0/2101376257 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f35e0007cc0 con 0x7f35f0109430 2024-04-12T21:53:59.210 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:53:59.207+0000 7f35ecc00700 1 -- 172.21.15.18:0/2101376257 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f35e000f040 con 0x7f35f0109430 2024-04-12T21:53:59.211 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:53:59.207+0000 7f35f7200700 1 -- 172.21.15.18:0/2101376257 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f35f01143d0 con 0x7f35f0109430 2024-04-12T21:53:59.211 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:53:59.208+0000 7f35ecc00700 1 -- 172.21.15.18:0/2101376257 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f35e0009640 con 0x7f35f0109430 2024-04-12T21:53:59.211 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:53:59.208+0000 7f35f7200700 1 -- 172.21.15.18:0/2101376257 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f35f0114990 con 0x7f35f0109430 2024-04-12T21:53:59.211 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:53:59.209+0000 7f35f7200700 1 -- 172.21.15.18:0/2101376257 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f35f0104b80 con 0x7f35f0109430 2024-04-12T21:53:59.212 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:53:59.210+0000 7f35ecc00700 1 -- 172.21.15.18:0/2101376257 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7f35e00097a0 con 0x7f35f0109430 2024-04-12T21:53:59.213 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:53:59.211+0000 7f35ecc00700 1 --2- 172.21.15.18:0/2101376257 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f35d406ec00 0x7f35d40710b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:53:59.213 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:53:59.211+0000 7f35ecc00700 1 -- 172.21.15.18:0/2101376257 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 6757+0+0 (secure 0 0 0) 0x7f35e008e760 con 0x7f35f0109430 2024-04-12T21:53:59.213 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:53:59.211+0000 7f35e7e00700 1 --2- 172.21.15.18:0/2101376257 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f35d406ec00 0x7f35d40710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:53:59.214 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:53:59.212+0000 7f35e7e00700 1 --2- 172.21.15.18:0/2101376257 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f35d406ec00 0x7f35d40710b0 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7f35d800b020 tx=0x7f35d8009e30 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:53:59.218 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:53:59.216+0000 7f35ecc00700 1 -- 172.21.15.18:0/2101376257 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f35e005af10 con 0x7f35f0109430 2024-04-12T21:53:59.469 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:53:59.467+0000 7f35f7200700 1 -- 172.21.15.18:0/2101376257 --> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f35f0115640 con 0x7f35d406ec00 2024-04-12T21:53:59.471 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:53:59.469+0000 7f35ecc00700 1 -- 172.21.15.18:0/2101376257 <== mgr.34102 v2:172.21.15.18:6800/3812891919 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+175 (secure 0 0 0) 0x7f35f0115640 con 0x7f35d406ec00 2024-04-12T21:53:59.472 INFO:teuthology.orchestra.run.smithi018.stdout:{ 2024-04-12T21:53:59.472 INFO:teuthology.orchestra.run.smithi018.stdout: "target_image": null, 2024-04-12T21:53:59.472 INFO:teuthology.orchestra.run.smithi018.stdout: "in_progress": false, 2024-04-12T21:53:59.472 INFO:teuthology.orchestra.run.smithi018.stdout: "which": "", 2024-04-12T21:53:59.472 INFO:teuthology.orchestra.run.smithi018.stdout: "services_complete": [], 2024-04-12T21:53:59.472 INFO:teuthology.orchestra.run.smithi018.stdout: "progress": null, 2024-04-12T21:53:59.472 INFO:teuthology.orchestra.run.smithi018.stdout: "message": "", 2024-04-12T21:53:59.472 INFO:teuthology.orchestra.run.smithi018.stdout: "is_paused": false 2024-04-12T21:53:59.472 INFO:teuthology.orchestra.run.smithi018.stdout:} 2024-04-12T21:53:59.475 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:53:59.472+0000 7f35f7200700 1 -- 172.21.15.18:0/2101376257 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f35d406ec00 msgr2=0x7f35d40710b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:53:59.476 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:53:59.472+0000 7f35f7200700 1 --2- 172.21.15.18:0/2101376257 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f35d406ec00 0x7f35d40710b0 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7f35d800b020 tx=0x7f35d8009e30 comp rx=0 tx=0).stop 2024-04-12T21:53:59.476 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:53:59.472+0000 7f35f7200700 1 -- 172.21.15.18:0/2101376257 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f35f0109430 msgr2=0x7f35f0119f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:53:59.476 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:53:59.472+0000 7f35f7200700 1 --2- 172.21.15.18:0/2101376257 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f35f0109430 0x7f35f0119f80 secure :-1 s=READY pgs=1371 cs=0 l=1 rev1=1 crypto rx=0x7f35e0002a50 tx=0x7f35e00046b0 comp rx=0 tx=0).stop 2024-04-12T21:53:59.476 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:53:59.472+0000 7f35f7200700 1 -- 172.21.15.18:0/2101376257 shutdown_connections 2024-04-12T21:53:59.476 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:53:59.472+0000 7f35f7200700 1 --2- 172.21.15.18:0/2101376257 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f35d406ec00 0x7f35d40710b0 unknown :-1 s=CLOSED pgs=727 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:53:59.476 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:53:59.472+0000 7f35f7200700 1 --2- 172.21.15.18:0/2101376257 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f35f0109430 0x7f35f0119f80 unknown :-1 s=CLOSED pgs=1371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:53:59.476 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:53:59.472+0000 7f35f7200700 1 --2- 172.21.15.18:0/2101376257 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f35f0109dd0 0x7f35f011a4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:53:59.476 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:53:59.473+0000 7f35f7200700 1 -- 172.21.15.18:0/2101376257 >> 172.21.15.18:0/2101376257 conn(0x7f35f0072c00 msgr2=0x7f35f0074c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:53:59.476 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:53:59.473+0000 7f35f7200700 1 -- 172.21.15.18:0/2101376257 shutdown_connections 2024-04-12T21:53:59.476 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:53:59.473+0000 7f35f7200700 1 -- 172.21.15.18:0/2101376257 wait complete. 2024-04-12T21:54:00.309 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1cbe407a-f90d-11ee-bc8d-c7b262605968 -e sha1=324baf8ddb91dea46cac50cb535c306115791355 -- bash -c 'ceph health detail' 2024-04-12T21:54:00.534 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:54:00 smithi186 ceph-mon[86379]: from='client.36812 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:54:00.534 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:54:00 smithi186 ceph-mon[86379]: pgmap v1632: 65 pgs: 65 active+clean; 257 MiB data, 976 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:54:00.606 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:54:00 smithi018 ceph-mon[112590]: from='client.36812 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:54:00.606 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:54:00 smithi018 ceph-mon[112590]: pgmap v1632: 65 pgs: 65 active+clean; 257 MiB data, 976 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:54:00.825 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/mon.smithi018/config 2024-04-12T21:54:02.355 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:54:02 smithi018 ceph-mon[112590]: pgmap v1633: 65 pgs: 65 active+clean; 257 MiB data, 976 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:54:02.534 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:54:02 smithi186 ceph-mon[86379]: pgmap v1633: 65 pgs: 65 active+clean; 257 MiB data, 976 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:54:02.977 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:54:02.974+0000 7fb1c3000700 1 -- 172.21.15.18:0/1194123122 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb1bc102e20 msgr2=0x7fb1bc1031f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:54:02.977 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:54:02.974+0000 7fb1c3000700 1 --2- 172.21.15.18:0/1194123122 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb1bc102e20 0x7fb1bc1031f0 secure :-1 s=READY pgs=1372 cs=0 l=1 rev1=1 crypto rx=0x7fb1ac009ce0 tx=0x7fb1ac004d10 comp rx=0 tx=0).stop 2024-04-12T21:54:02.977 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:54:02.975+0000 7fb1c3000700 1 -- 172.21.15.18:0/1194123122 shutdown_connections 2024-04-12T21:54:02.977 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:54:02.975+0000 7fb1c3000700 1 --2- 172.21.15.18:0/1194123122 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb1bc1037c0 0x7fb1bc0fce30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:54:02.977 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:54:02.975+0000 7fb1c3000700 1 --2- 172.21.15.18:0/1194123122 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb1bc102e20 0x7fb1bc1031f0 unknown :-1 s=CLOSED pgs=1372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:54:02.978 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:54:02.975+0000 7fb1c3000700 1 -- 172.21.15.18:0/1194123122 >> 172.21.15.18:0/1194123122 conn(0x7fb1bc0f89d0 msgr2=0x7fb1bc0fade0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:54:02.978 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:54:02.975+0000 7fb1c3000700 1 -- 172.21.15.18:0/1194123122 shutdown_connections 2024-04-12T21:54:02.978 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:54:02.975+0000 7fb1c3000700 1 -- 172.21.15.18:0/1194123122 wait complete. 2024-04-12T21:54:02.979 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:54:02.976+0000 7fb1c3000700 1 Processor -- start 2024-04-12T21:54:02.979 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:54:02.976+0000 7fb1c3000700 1 -- start start 2024-04-12T21:54:02.979 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:54:02.976+0000 7fb1c3000700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb1bc102e20 0x7fb1bc0702c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:54:02.979 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:54:02.976+0000 7fb1c3000700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb1bc1037c0 0x7fb1bc070800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:54:02.979 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:54:02.976+0000 7fb1c3000700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1bc070d40 con 0x7fb1bc102e20 2024-04-12T21:54:02.979 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:54:02.976+0000 7fb1c3000700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1bc070e80 con 0x7fb1bc1037c0 2024-04-12T21:54:02.979 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:54:02.977+0000 7fb1bbe00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb1bc102e20 0x7fb1bc0702c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:54:02.979 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:54:02.977+0000 7fb1bbe00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb1bc102e20 0x7fb1bc0702c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:41852/0 (socket says 172.21.15.18:41852) 2024-04-12T21:54:02.979 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:54:02.977+0000 7fb1bbe00700 1 -- 172.21.15.18:0/2303247093 learned_addr learned my addr 172.21.15.18:0/2303247093 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:54:02.980 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:54:02.977+0000 7fb1bb400700 1 --2- 172.21.15.18:0/2303247093 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb1bc1037c0 0x7fb1bc070800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:54:02.980 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:54:02.977+0000 7fb1bbe00700 1 -- 172.21.15.18:0/2303247093 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb1bc1037c0 msgr2=0x7fb1bc070800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:54:02.980 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:54:02.977+0000 7fb1bbe00700 1 --2- 172.21.15.18:0/2303247093 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb1bc1037c0 0x7fb1bc070800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:54:02.981 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:54:02.977+0000 7fb1bbe00700 1 -- 172.21.15.18:0/2303247093 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1ac009940 con 0x7fb1bc102e20 2024-04-12T21:54:02.981 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:54:02.977+0000 7fb1bb400700 1 --2- 172.21.15.18:0/2303247093 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb1bc1037c0 0x7fb1bc070800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-12T21:54:02.981 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:54:02.978+0000 7fb1bbe00700 1 --2- 172.21.15.18:0/2303247093 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb1bc102e20 0x7fb1bc0702c0 secure :-1 s=READY pgs=1373 cs=0 l=1 rev1=1 crypto rx=0x7fb1ac003400 tx=0x7fb1ac004d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:54:02.981 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:54:02.978+0000 7fb1b3e00700 1 -- 172.21.15.18:0/2303247093 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb1ac00e040 con 0x7fb1bc102e20 2024-04-12T21:54:02.981 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:54:02.978+0000 7fb1b3e00700 1 -- 172.21.15.18:0/2303247093 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb1ac004610 con 0x7fb1bc102e20 2024-04-12T21:54:02.981 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:54:02.978+0000 7fb1c3000700 1 -- 172.21.15.18:0/2303247093 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb1bc0710d0 con 0x7fb1bc102e20 2024-04-12T21:54:02.981 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:54:02.978+0000 7fb1b3e00700 1 -- 172.21.15.18:0/2303247093 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb1ac01c6d0 con 0x7fb1bc102e20 2024-04-12T21:54:02.981 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:54:02.978+0000 7fb1c3000700 1 -- 172.21.15.18:0/2303247093 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb1bc11a360 con 0x7fb1bc102e20 2024-04-12T21:54:02.981 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:54:02.979+0000 7fb1c3000700 1 -- 172.21.15.18:0/2303247093 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb1bc100130 con 0x7fb1bc102e20 2024-04-12T21:54:02.982 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:54:02.980+0000 7fb1b3e00700 1 -- 172.21.15.18:0/2303247093 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7fb1ac012b00 con 0x7fb1bc102e20 2024-04-12T21:54:02.983 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:54:02.981+0000 7fb1b3e00700 1 --2- 172.21.15.18:0/2303247093 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fb1a006ec00 0x7fb1a00710b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:54:02.983 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:54:02.981+0000 7fb1b3e00700 1 -- 172.21.15.18:0/2303247093 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 6757+0+0 (secure 0 0 0) 0x7fb1ac0904a0 con 0x7fb1bc102e20 2024-04-12T21:54:02.984 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:54:02.982+0000 7fb1bb400700 1 --2- 172.21.15.18:0/2303247093 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fb1a006ec00 0x7fb1a00710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:54:02.984 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:54:02.983+0000 7fb1bb400700 1 --2- 172.21.15.18:0/2303247093 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fb1a006ec00 0x7fb1a00710b0 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7fb1a8009b80 tx=0x7fb1a800b040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:54:02.989 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:54:02.987+0000 7fb1b3e00700 1 -- 172.21.15.18:0/2303247093 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7fb1ac0590a0 con 0x7fb1bc102e20 2024-04-12T21:54:03.317 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:54:03.314+0000 7fb1c3000700 1 -- 172.21.15.18:0/2303247093 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7fb1bc0fe570 con 0x7fb1bc102e20 2024-04-12T21:54:03.317 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:54:03.315+0000 7fb1b3e00700 1 -- 172.21.15.18:0/2303247093 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+10 (secure 0 0 0) 0x7fb1bc0fe570 con 0x7fb1bc102e20 2024-04-12T21:54:03.318 INFO:teuthology.orchestra.run.smithi018.stdout:HEALTH_OK 2024-04-12T21:54:03.321 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:54:03.319+0000 7fb1c3000700 1 -- 172.21.15.18:0/2303247093 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fb1a006ec00 msgr2=0x7fb1a00710b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:54:03.321 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:54:03.319+0000 7fb1c3000700 1 --2- 172.21.15.18:0/2303247093 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fb1a006ec00 0x7fb1a00710b0 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7fb1a8009b80 tx=0x7fb1a800b040 comp rx=0 tx=0).stop 2024-04-12T21:54:03.321 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:54:03.319+0000 7fb1c3000700 1 -- 172.21.15.18:0/2303247093 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb1bc102e20 msgr2=0x7fb1bc0702c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:54:03.321 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:54:03.319+0000 7fb1c3000700 1 --2- 172.21.15.18:0/2303247093 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb1bc102e20 0x7fb1bc0702c0 secure :-1 s=READY pgs=1373 cs=0 l=1 rev1=1 crypto rx=0x7fb1ac003400 tx=0x7fb1ac004d10 comp rx=0 tx=0).stop 2024-04-12T21:54:03.322 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:54:03.319+0000 7fb1c3000700 1 -- 172.21.15.18:0/2303247093 shutdown_connections 2024-04-12T21:54:03.322 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:54:03.319+0000 7fb1c3000700 1 --2- 172.21.15.18:0/2303247093 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fb1a006ec00 0x7fb1a00710b0 unknown :-1 s=CLOSED pgs=728 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:54:03.322 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:54:03.319+0000 7fb1c3000700 1 --2- 172.21.15.18:0/2303247093 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb1bc102e20 0x7fb1bc0702c0 unknown :-1 s=CLOSED pgs=1373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:54:03.322 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:54:03.319+0000 7fb1c3000700 1 --2- 172.21.15.18:0/2303247093 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb1bc1037c0 0x7fb1bc070800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:54:03.322 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:54:03.319+0000 7fb1c3000700 1 -- 172.21.15.18:0/2303247093 >> 172.21.15.18:0/2303247093 conn(0x7fb1bc0f89d0 msgr2=0x7fb1bc1073c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:54:03.322 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:54:03.320+0000 7fb1c3000700 1 -- 172.21.15.18:0/2303247093 shutdown_connections 2024-04-12T21:54:03.323 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:54:03.320+0000 7fb1c3000700 1 -- 172.21.15.18:0/2303247093 wait complete. 2024-04-12T21:54:03.605 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:54:03 smithi018 ceph-mon[112590]: from='client.? 172.21.15.18:0/2303247093' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-12T21:54:03.784 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:54:03 smithi186 ceph-mon[86379]: from='client.? 172.21.15.18:0/2303247093' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-12T21:54:04.067 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1cbe407a-f90d-11ee-bc8d-c7b262605968 -e sha1=324baf8ddb91dea46cac50cb535c306115791355 -- bash -c 'ceph versions' 2024-04-12T21:54:04.559 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/mon.smithi018/config 2024-04-12T21:54:04.704 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:54:04 smithi018 ceph-mon[112590]: pgmap v1634: 65 pgs: 65 active+clean; 257 MiB data, 976 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:54:04.784 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:54:04 smithi186 ceph-mon[86379]: pgmap v1634: 65 pgs: 65 active+clean; 257 MiB data, 976 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:54:06.477 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:54:06 smithi018 ceph-mon[112590]: pgmap v1635: 65 pgs: 65 active+clean; 257 MiB data, 976 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:54:06.535 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:54:06 smithi186 ceph-mon[86379]: pgmap v1635: 65 pgs: 65 active+clean; 257 MiB data, 976 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:54:06.648 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:54:06.645+0000 7fd134a00700 1 -- 172.21.15.18:0/1623345490 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd1300fde90 msgr2=0x7fd1300fe260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:54:06.648 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:54:06.645+0000 7fd134a00700 1 --2- 172.21.15.18:0/1623345490 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd1300fde90 0x7fd1300fe260 secure :-1 s=READY pgs=1374 cs=0 l=1 rev1=1 crypto rx=0x7fd120004940 tx=0x7fd120009e30 comp rx=0 tx=0).stop 2024-04-12T21:54:06.648 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:54:06.645+0000 7fd134a00700 1 -- 172.21.15.18:0/1623345490 shutdown_connections 2024-04-12T21:54:06.648 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:54:06.645+0000 7fd134a00700 1 --2- 172.21.15.18:0/1623345490 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd1300fe830 0x7fd130106f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:54:06.648 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:54:06.645+0000 7fd134a00700 1 --2- 172.21.15.18:0/1623345490 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd1300fde90 0x7fd1300fe260 unknown :-1 s=CLOSED pgs=1374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:54:06.648 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:54:06.645+0000 7fd134a00700 1 -- 172.21.15.18:0/1623345490 >> 172.21.15.18:0/1623345490 conn(0x7fd130072bc0 msgr2=0x7fd130072fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:54:06.649 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:54:06.645+0000 7fd134a00700 1 -- 172.21.15.18:0/1623345490 shutdown_connections 2024-04-12T21:54:06.649 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:54:06.645+0000 7fd134a00700 1 -- 172.21.15.18:0/1623345490 wait complete. 2024-04-12T21:54:06.650 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:54:06.646+0000 7fd134a00700 1 Processor -- start 2024-04-12T21:54:06.650 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:54:06.646+0000 7fd134a00700 1 -- start start 2024-04-12T21:54:06.650 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:54:06.647+0000 7fd134a00700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd1300fde90 0x7fd130115de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:54:06.650 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:54:06.647+0000 7fd134a00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd1300fe830 0x7fd130116320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:54:06.650 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:54:06.647+0000 7fd134a00700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd130116970 con 0x7fd1300fe830 2024-04-12T21:54:06.651 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:54:06.647+0000 7fd134a00700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd13010fd40 con 0x7fd1300fde90 2024-04-12T21:54:06.651 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:54:06.647+0000 7fd12d400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd1300fe830 0x7fd130116320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:54:06.651 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:54:06.647+0000 7fd12d400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd1300fe830 0x7fd130116320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:41878/0 (socket says 172.21.15.18:41878) 2024-04-12T21:54:06.651 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:54:06.647+0000 7fd12d400700 1 -- 172.21.15.18:0/1012908245 learned_addr learned my addr 172.21.15.18:0/1012908245 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:54:06.651 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:54:06.648+0000 7fd12de00700 1 --2- 172.21.15.18:0/1012908245 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd1300fde90 0x7fd130115de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:54:06.651 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:54:06.648+0000 7fd12d400700 1 -- 172.21.15.18:0/1012908245 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd1300fde90 msgr2=0x7fd130115de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:54:06.651 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:54:06.648+0000 7fd12d400700 1 --2- 172.21.15.18:0/1012908245 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd1300fde90 0x7fd130115de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:54:06.651 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:54:06.648+0000 7fd12d400700 1 -- 172.21.15.18:0/1012908245 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd120005040 con 0x7fd1300fe830 2024-04-12T21:54:06.651 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:54:06.648+0000 7fd12d400700 1 --2- 172.21.15.18:0/1012908245 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd1300fe830 0x7fd130116320 secure :-1 s=READY pgs=1375 cs=0 l=1 rev1=1 crypto rx=0x7fd11c002a50 tx=0x7fd11c0046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:54:06.652 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:54:06.648+0000 7fd126000700 1 -- 172.21.15.18:0/1012908245 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd11c007cc0 con 0x7fd1300fe830 2024-04-12T21:54:06.652 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:54:06.649+0000 7fd126000700 1 -- 172.21.15.18:0/1012908245 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd11c00f040 con 0x7fd1300fe830 2024-04-12T21:54:06.652 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:54:06.649+0000 7fd134a00700 1 -- 172.21.15.18:0/1012908245 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd130110020 con 0x7fd1300fe830 2024-04-12T21:54:06.652 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:54:06.649+0000 7fd126000700 1 -- 172.21.15.18:0/1012908245 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd11c00db70 con 0x7fd1300fe830 2024-04-12T21:54:06.652 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:54:06.649+0000 7fd134a00700 1 -- 172.21.15.18:0/1012908245 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd130110560 con 0x7fd1300fe830 2024-04-12T21:54:06.652 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:54:06.650+0000 7fd134a00700 1 -- 172.21.15.18:0/1012908245 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd130106a20 con 0x7fd1300fe830 2024-04-12T21:54:06.653 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:54:06.651+0000 7fd126000700 1 -- 172.21.15.18:0/1012908245 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7fd11c00dd10 con 0x7fd1300fe830 2024-04-12T21:54:06.654 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:54:06.652+0000 7fd126000700 1 --2- 172.21.15.18:0/1012908245 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fd11406ec10 0x7fd1140710c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:54:06.654 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:54:06.652+0000 7fd126000700 1 -- 172.21.15.18:0/1012908245 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 6757+0+0 (secure 0 0 0) 0x7fd11c08e150 con 0x7fd1300fe830 2024-04-12T21:54:06.654 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:54:06.652+0000 7fd12de00700 1 --2- 172.21.15.18:0/1012908245 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fd11406ec10 0x7fd1140710c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:54:06.655 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:54:06.653+0000 7fd12de00700 1 --2- 172.21.15.18:0/1012908245 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fd11406ec10 0x7fd1140710c0 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7fd12000b3b0 tx=0x7fd120009e30 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:54:06.660 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:54:06.658+0000 7fd126000700 1 -- 172.21.15.18:0/1012908245 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7fd11c05a880 con 0x7fd1300fe830 2024-04-12T21:54:06.986 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:54:06.983+0000 7fd134a00700 1 -- 172.21.15.18:0/1012908245 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fd130065e30 con 0x7fd1300fe830 2024-04-12T21:54:06.987 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:54:06.984+0000 7fd126000700 1 -- 172.21.15.18:0/1012908245 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+623 (secure 0 0 0) 0x7fd11c05a410 con 0x7fd1300fe830 2024-04-12T21:54:06.987 INFO:teuthology.orchestra.run.smithi018.stdout:{ 2024-04-12T21:54:06.987 INFO:teuthology.orchestra.run.smithi018.stdout: "mon": { 2024-04-12T21:54:06.987 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 19.0.0-2879-g324baf8d (324baf8ddb91dea46cac50cb535c306115791355) squid (dev)": 2 2024-04-12T21:54:06.987 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-12T21:54:06.988 INFO:teuthology.orchestra.run.smithi018.stdout: "mgr": { 2024-04-12T21:54:06.988 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 19.0.0-2879-g324baf8d (324baf8ddb91dea46cac50cb535c306115791355) squid (dev)": 2 2024-04-12T21:54:06.988 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-12T21:54:06.988 INFO:teuthology.orchestra.run.smithi018.stdout: "osd": { 2024-04-12T21:54:06.988 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 19.0.0-2879-g324baf8d (324baf8ddb91dea46cac50cb535c306115791355) squid (dev)": 6 2024-04-12T21:54:06.988 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-12T21:54:06.988 INFO:teuthology.orchestra.run.smithi018.stdout: "mds": { 2024-04-12T21:54:06.988 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 19.0.0-2879-g324baf8d (324baf8ddb91dea46cac50cb535c306115791355) squid (dev)": 4 2024-04-12T21:54:06.988 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-12T21:54:06.988 INFO:teuthology.orchestra.run.smithi018.stdout: "overall": { 2024-04-12T21:54:06.988 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 19.0.0-2879-g324baf8d (324baf8ddb91dea46cac50cb535c306115791355) squid (dev)": 14 2024-04-12T21:54:06.988 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-04-12T21:54:06.988 INFO:teuthology.orchestra.run.smithi018.stdout:} 2024-04-12T21:54:06.990 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:54:06.988+0000 7fd134a00700 1 -- 172.21.15.18:0/1012908245 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fd11406ec10 msgr2=0x7fd1140710c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:54:06.991 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:54:06.988+0000 7fd134a00700 1 --2- 172.21.15.18:0/1012908245 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fd11406ec10 0x7fd1140710c0 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7fd12000b3b0 tx=0x7fd120009e30 comp rx=0 tx=0).stop 2024-04-12T21:54:06.991 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:54:06.988+0000 7fd134a00700 1 -- 172.21.15.18:0/1012908245 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd1300fe830 msgr2=0x7fd130116320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:54:06.991 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:54:06.988+0000 7fd134a00700 1 --2- 172.21.15.18:0/1012908245 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd1300fe830 0x7fd130116320 secure :-1 s=READY pgs=1375 cs=0 l=1 rev1=1 crypto rx=0x7fd11c002a50 tx=0x7fd11c0046b0 comp rx=0 tx=0).stop 2024-04-12T21:54:06.991 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:54:06.988+0000 7fd134a00700 1 -- 172.21.15.18:0/1012908245 shutdown_connections 2024-04-12T21:54:06.991 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:54:06.988+0000 7fd134a00700 1 --2- 172.21.15.18:0/1012908245 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fd11406ec10 0x7fd1140710c0 unknown :-1 s=CLOSED pgs=729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:54:06.991 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:54:06.988+0000 7fd134a00700 1 --2- 172.21.15.18:0/1012908245 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd1300fde90 0x7fd130115de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:54:06.991 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:54:06.988+0000 7fd134a00700 1 --2- 172.21.15.18:0/1012908245 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd1300fe830 0x7fd130116320 unknown :-1 s=CLOSED pgs=1375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:54:06.991 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:54:06.988+0000 7fd134a00700 1 -- 172.21.15.18:0/1012908245 >> 172.21.15.18:0/1012908245 conn(0x7fd130072bc0 msgr2=0x7fd1300fb820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:54:06.991 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:54:06.988+0000 7fd134a00700 1 -- 172.21.15.18:0/1012908245 shutdown_connections 2024-04-12T21:54:06.991 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:54:06.988+0000 7fd134a00700 1 -- 172.21.15.18:0/1012908245 wait complete. 2024-04-12T21:54:07.328 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:54:07 smithi018 ceph-mon[112590]: from='client.? 172.21.15.18:0/1012908245' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T21:54:07.534 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:54:07 smithi186 ceph-mon[86379]: from='client.? 172.21.15.18:0/1012908245' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T21:54:07.752 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1cbe407a-f90d-11ee-bc8d-c7b262605968 -e sha1=324baf8ddb91dea46cac50cb535c306115791355 -- bash -c 'echo "wait for servicemap items w/ changing names to refresh"' 2024-04-12T21:54:08.273 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/mon.smithi018/config 2024-04-12T21:54:08.534 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:54:08 smithi186 ceph-mon[86379]: pgmap v1636: 65 pgs: 65 active+clean; 257 MiB data, 976 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:54:08.587 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:54:08 smithi018 ceph-mon[112590]: pgmap v1636: 65 pgs: 65 active+clean; 257 MiB data, 976 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:54:10.264 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:54:10 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T21:54:10.284 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:54:10 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T21:54:10.489 INFO:teuthology.orchestra.run.smithi018.stdout:wait for servicemap items w/ changing names to refresh 2024-04-12T21:54:11.277 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:54:11 smithi018 ceph-mon[112590]: pgmap v1637: 65 pgs: 65 active+clean; 257 MiB data, 976 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:54:11.284 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:54:11 smithi186 ceph-mon[86379]: pgmap v1637: 65 pgs: 65 active+clean; 257 MiB data, 976 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:54:11.333 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1cbe407a-f90d-11ee-bc8d-c7b262605968 -e sha1=324baf8ddb91dea46cac50cb535c306115791355 -- bash -c 'sleep 60' 2024-04-12T21:54:11.841 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/mon.smithi018/config 2024-04-12T21:54:12.315 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:54:12 smithi018 ceph-mon[112590]: pgmap v1638: 65 pgs: 65 active+clean; 257 MiB data, 976 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:54:12.534 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:54:12 smithi186 ceph-mon[86379]: pgmap v1638: 65 pgs: 65 active+clean; 257 MiB data, 976 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:54:14.534 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:54:14 smithi186 ceph-mon[86379]: pgmap v1639: 65 pgs: 65 active+clean; 257 MiB data, 976 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:54:14.606 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:54:14 smithi018 ceph-mon[112590]: pgmap v1639: 65 pgs: 65 active+clean; 257 MiB data, 976 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:54:16.534 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:54:16 smithi186 ceph-mon[86379]: pgmap v1640: 65 pgs: 65 active+clean; 257 MiB data, 976 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:54:16.606 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:54:16 smithi018 ceph-mon[112590]: pgmap v1640: 65 pgs: 65 active+clean; 257 MiB data, 976 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:54:18.356 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:54:18 smithi018 ceph-mon[112590]: pgmap v1641: 65 pgs: 65 active+clean; 257 MiB data, 976 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:54:18.535 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:54:18 smithi186 ceph-mon[86379]: pgmap v1641: 65 pgs: 65 active+clean; 257 MiB data, 976 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:54:20.534 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:54:20 smithi186 ceph-mon[86379]: pgmap v1642: 65 pgs: 65 active+clean; 257 MiB data, 976 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:54:20.605 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:54:20 smithi018 ceph-mon[112590]: pgmap v1642: 65 pgs: 65 active+clean; 257 MiB data, 976 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:54:22.534 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:54:22 smithi186 ceph-mon[86379]: pgmap v1643: 65 pgs: 65 active+clean; 257 MiB data, 976 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:54:22.606 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:54:22 smithi018 ceph-mon[112590]: pgmap v1643: 65 pgs: 65 active+clean; 257 MiB data, 976 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:54:24.534 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:54:24 smithi186 ceph-mon[86379]: pgmap v1644: 65 pgs: 65 active+clean; 257 MiB data, 976 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:54:24.605 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:54:24 smithi018 ceph-mon[112590]: pgmap v1644: 65 pgs: 65 active+clean; 257 MiB data, 976 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:54:25.535 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:54:25 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T21:54:25.605 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:54:25 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T21:54:26.534 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:54:26 smithi186 ceph-mon[86379]: pgmap v1645: 65 pgs: 65 active+clean; 257 MiB data, 976 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:54:26.606 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:54:26 smithi018 ceph-mon[112590]: pgmap v1645: 65 pgs: 65 active+clean; 257 MiB data, 976 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:54:28.535 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:54:28 smithi186 ceph-mon[86379]: pgmap v1646: 65 pgs: 65 active+clean; 257 MiB data, 976 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:54:28.606 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:54:28 smithi018 ceph-mon[112590]: pgmap v1646: 65 pgs: 65 active+clean; 257 MiB data, 976 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:54:30.534 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:54:30 smithi186 ceph-mon[86379]: pgmap v1647: 65 pgs: 65 active+clean; 257 MiB data, 976 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:54:30.605 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:54:30 smithi018 ceph-mon[112590]: pgmap v1647: 65 pgs: 65 active+clean; 257 MiB data, 976 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:54:32.534 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:54:32 smithi186 ceph-mon[86379]: pgmap v1648: 65 pgs: 65 active+clean; 257 MiB data, 976 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:54:32.605 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:54:32 smithi018 ceph-mon[112590]: pgmap v1648: 65 pgs: 65 active+clean; 257 MiB data, 976 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:54:34.535 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:54:34 smithi186 ceph-mon[86379]: pgmap v1649: 65 pgs: 65 active+clean; 257 MiB data, 976 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:54:34.606 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:54:34 smithi018 ceph-mon[112590]: pgmap v1649: 65 pgs: 65 active+clean; 257 MiB data, 976 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:54:36.534 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:54:36 smithi186 ceph-mon[86379]: pgmap v1650: 65 pgs: 65 active+clean; 257 MiB data, 976 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:54:36.606 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:54:36 smithi018 ceph-mon[112590]: pgmap v1650: 65 pgs: 65 active+clean; 257 MiB data, 976 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:54:38.535 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:54:38 smithi186 ceph-mon[86379]: pgmap v1651: 65 pgs: 65 active+clean; 257 MiB data, 976 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:54:38.606 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:54:38 smithi018 ceph-mon[112590]: pgmap v1651: 65 pgs: 65 active+clean; 257 MiB data, 976 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:54:40.285 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:54:40 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T21:54:40.355 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:54:40 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T21:54:41.285 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:54:41 smithi186 ceph-mon[86379]: pgmap v1652: 65 pgs: 65 active+clean; 257 MiB data, 976 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:54:41.355 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:54:41 smithi018 ceph-mon[112590]: pgmap v1652: 65 pgs: 65 active+clean; 257 MiB data, 976 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:54:42.534 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:54:42 smithi186 ceph-mon[86379]: pgmap v1653: 65 pgs: 65 active+clean; 257 MiB data, 976 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:54:42.606 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:54:42 smithi018 ceph-mon[112590]: pgmap v1653: 65 pgs: 65 active+clean; 257 MiB data, 976 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:54:43.535 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:54:43 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-12T21:54:43.606 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:54:43 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-12T21:54:44.534 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:54:44 smithi186 ceph-mon[86379]: pgmap v1654: 65 pgs: 65 active+clean; 257 MiB data, 976 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:54:44.535 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:54:44 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-12T21:54:44.535 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:54:44 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-12T21:54:44.535 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:54:44 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' 2024-04-12T21:54:44.606 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:54:44 smithi018 ceph-mon[112590]: pgmap v1654: 65 pgs: 65 active+clean; 257 MiB data, 976 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:54:44.606 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:54:44 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-12T21:54:44.606 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:54:44 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-12T21:54:44.606 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:54:44 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' 2024-04-12T21:54:46.535 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:54:46 smithi186 ceph-mon[86379]: pgmap v1655: 65 pgs: 65 active+clean; 257 MiB data, 976 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:54:46.606 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:54:46 smithi018 ceph-mon[112590]: pgmap v1655: 65 pgs: 65 active+clean; 257 MiB data, 976 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:54:48.535 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:54:48 smithi186 ceph-mon[86379]: pgmap v1656: 65 pgs: 65 active+clean; 257 MiB data, 976 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:54:48.606 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:54:48 smithi018 ceph-mon[112590]: pgmap v1656: 65 pgs: 65 active+clean; 257 MiB data, 976 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:54:50.534 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:54:50 smithi186 ceph-mon[86379]: pgmap v1657: 65 pgs: 65 active+clean; 257 MiB data, 976 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:54:50.605 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:54:50 smithi018 ceph-mon[112590]: pgmap v1657: 65 pgs: 65 active+clean; 257 MiB data, 976 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:54:52.534 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:54:52 smithi186 ceph-mon[86379]: pgmap v1658: 65 pgs: 65 active+clean; 257 MiB data, 976 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:54:52.606 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:54:52 smithi018 ceph-mon[112590]: pgmap v1658: 65 pgs: 65 active+clean; 257 MiB data, 976 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:54:54.534 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:54:54 smithi186 ceph-mon[86379]: pgmap v1659: 65 pgs: 65 active+clean; 257 MiB data, 976 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:54:54.605 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:54:54 smithi018 ceph-mon[112590]: pgmap v1659: 65 pgs: 65 active+clean; 257 MiB data, 976 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:54:55.534 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:54:55 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T21:54:55.605 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:54:55 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T21:54:56.534 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:54:56 smithi186 ceph-mon[86379]: pgmap v1660: 65 pgs: 65 active+clean; 257 MiB data, 976 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:54:56.605 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:54:56 smithi018 ceph-mon[112590]: pgmap v1660: 65 pgs: 65 active+clean; 257 MiB data, 976 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:54:58.534 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:54:58 smithi186 ceph-mon[86379]: pgmap v1661: 65 pgs: 65 active+clean; 257 MiB data, 976 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:54:58.605 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:54:58 smithi018 ceph-mon[112590]: pgmap v1661: 65 pgs: 65 active+clean; 257 MiB data, 976 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:55:00.535 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:55:00 smithi186 ceph-mon[86379]: pgmap v1662: 65 pgs: 65 active+clean; 257 MiB data, 976 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:55:00.606 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:55:00 smithi018 ceph-mon[112590]: pgmap v1662: 65 pgs: 65 active+clean; 257 MiB data, 976 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:55:02.534 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:55:02 smithi186 ceph-mon[86379]: pgmap v1663: 65 pgs: 65 active+clean; 257 MiB data, 976 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:55:02.605 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:55:02 smithi018 ceph-mon[112590]: pgmap v1663: 65 pgs: 65 active+clean; 257 MiB data, 976 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:55:04.534 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:55:04 smithi186 ceph-mon[86379]: pgmap v1664: 65 pgs: 65 active+clean; 257 MiB data, 976 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:55:04.605 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:55:04 smithi018 ceph-mon[112590]: pgmap v1664: 65 pgs: 65 active+clean; 257 MiB data, 976 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:55:07.034 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:55:06 smithi186 ceph-mon[86379]: pgmap v1665: 65 pgs: 65 active+clean; 257 MiB data, 976 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:55:07.106 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:55:06 smithi018 ceph-mon[112590]: pgmap v1665: 65 pgs: 65 active+clean; 257 MiB data, 976 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:55:08.535 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:55:08 smithi186 ceph-mon[86379]: pgmap v1666: 65 pgs: 65 active+clean; 257 MiB data, 976 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:55:08.606 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:55:08 smithi018 ceph-mon[112590]: pgmap v1666: 65 pgs: 65 active+clean; 257 MiB data, 976 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:55:10.285 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:55:10 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T21:55:10.356 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:55:10 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T21:55:11.285 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:55:11 smithi186 ceph-mon[86379]: pgmap v1667: 65 pgs: 65 active+clean; 257 MiB data, 976 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:55:11.356 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:55:11 smithi018 ceph-mon[112590]: pgmap v1667: 65 pgs: 65 active+clean; 257 MiB data, 976 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:55:12.535 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:55:12 smithi186 ceph-mon[86379]: pgmap v1668: 65 pgs: 65 active+clean; 257 MiB data, 976 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:55:12.606 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:55:12 smithi018 ceph-mon[112590]: pgmap v1668: 65 pgs: 65 active+clean; 257 MiB data, 976 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:55:14.534 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:55:14 smithi186 ceph-mon[86379]: pgmap v1669: 65 pgs: 65 active+clean; 257 MiB data, 976 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:55:14.556 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:55:14 smithi018 ceph-mon[112590]: pgmap v1669: 65 pgs: 65 active+clean; 257 MiB data, 976 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:55:14.637 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1cbe407a-f90d-11ee-bc8d-c7b262605968 -e sha1=324baf8ddb91dea46cac50cb535c306115791355 -- bash -c 'ceph orch ps' 2024-04-12T21:55:15.167 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/mon.smithi018/config 2024-04-12T21:55:16.333 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:55:16 smithi018 ceph-mon[112590]: pgmap v1670: 65 pgs: 65 active+clean; 257 MiB data, 976 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:55:16.535 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:55:16 smithi186 ceph-mon[86379]: pgmap v1670: 65 pgs: 65 active+clean; 257 MiB data, 976 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:55:17.284 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:17.281+0000 7f2ec6800700 1 -- 172.21.15.18:0/3990791854 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2ec00724f0 msgr2=0x7f2ec00728c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:55:17.284 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:17.281+0000 7f2ec6800700 1 --2- 172.21.15.18:0/3990791854 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2ec00724f0 0x7f2ec00728c0 secure :-1 s=READY pgs=1376 cs=0 l=1 rev1=1 crypto rx=0x7f2eb0004940 tx=0x7f2eb0009e30 comp rx=0 tx=0).stop 2024-04-12T21:55:17.284 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:17.281+0000 7f2ec6800700 1 -- 172.21.15.18:0/3990791854 shutdown_connections 2024-04-12T21:55:17.285 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:17.281+0000 7f2ec6800700 1 --2- 172.21.15.18:0/3990791854 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2ec0072e00 0x7f2ec010d770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:55:17.285 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:17.281+0000 7f2ec6800700 1 --2- 172.21.15.18:0/3990791854 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2ec00724f0 0x7f2ec00728c0 unknown :-1 s=CLOSED pgs=1376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:55:17.285 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:17.281+0000 7f2ec6800700 1 -- 172.21.15.18:0/3990791854 >> 172.21.15.18:0/3990791854 conn(0x7f2ec00fabd0 msgr2=0x7f2ec00fcfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:55:17.285 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:17.281+0000 7f2ec6800700 1 -- 172.21.15.18:0/3990791854 shutdown_connections 2024-04-12T21:55:17.285 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:17.281+0000 7f2ec6800700 1 -- 172.21.15.18:0/3990791854 wait complete. 2024-04-12T21:55:17.285 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:17.282+0000 7f2ec6800700 1 Processor -- start 2024-04-12T21:55:17.285 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:17.282+0000 7f2ec6800700 1 -- start start 2024-04-12T21:55:17.286 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:17.282+0000 7f2ec6800700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2ec00724f0 0x7f2ec011a120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:55:17.286 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:17.282+0000 7f2ec6800700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2ec0072e00 0x7f2ec011a660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:55:17.286 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:17.283+0000 7f2ec6800700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ec011ad40 con 0x7f2ec00724f0 2024-04-12T21:55:17.286 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:17.283+0000 7f2ec6800700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ec0114080 con 0x7f2ec0072e00 2024-04-12T21:55:17.286 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:17.283+0000 7f2ebf400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2ec00724f0 0x7f2ec011a120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:55:17.286 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:17.283+0000 7f2ebf400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2ec00724f0 0x7f2ec011a120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:55522/0 (socket says 172.21.15.18:55522) 2024-04-12T21:55:17.286 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:17.283+0000 7f2ebf400700 1 -- 172.21.15.18:0/2981068171 learned_addr learned my addr 172.21.15.18:0/2981068171 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:55:17.286 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:17.283+0000 7f2ebea00700 1 --2- 172.21.15.18:0/2981068171 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2ec0072e00 0x7f2ec011a660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:55:17.287 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:17.283+0000 7f2ebf400700 1 -- 172.21.15.18:0/2981068171 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2ec0072e00 msgr2=0x7f2ec011a660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:55:17.287 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:17.283+0000 7f2ebf400700 1 --2- 172.21.15.18:0/2981068171 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2ec0072e00 0x7f2ec011a660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:55:17.287 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:17.283+0000 7f2ebf400700 1 -- 172.21.15.18:0/2981068171 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2eb0005040 con 0x7f2ec00724f0 2024-04-12T21:55:17.287 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:17.283+0000 7f2ebea00700 1 --2- 172.21.15.18:0/2981068171 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2ec0072e00 0x7f2ec011a660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-12T21:55:17.287 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:17.284+0000 7f2ebf400700 1 --2- 172.21.15.18:0/2981068171 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2ec00724f0 0x7f2ec011a120 secure :-1 s=READY pgs=1377 cs=0 l=1 rev1=1 crypto rx=0x7f2eb0004890 tx=0x7f2eb0009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:55:17.287 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:17.284+0000 7f2eb7400700 1 -- 172.21.15.18:0/2981068171 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2eb000e040 con 0x7f2ec00724f0 2024-04-12T21:55:17.287 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:17.284+0000 7f2eb7400700 1 -- 172.21.15.18:0/2981068171 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f2eb0009af0 con 0x7f2ec00724f0 2024-04-12T21:55:17.287 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:17.284+0000 7f2ec6800700 1 -- 172.21.15.18:0/2981068171 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2ec01142a0 con 0x7f2ec00724f0 2024-04-12T21:55:17.288 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:17.285+0000 7f2eb7400700 1 -- 172.21.15.18:0/2981068171 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2eb001c630 con 0x7f2ec00724f0 2024-04-12T21:55:17.288 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:17.285+0000 7f2ec6800700 1 -- 172.21.15.18:0/2981068171 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2ec0114860 con 0x7f2ec00724f0 2024-04-12T21:55:17.288 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:17.286+0000 7f2ec6800700 1 -- 172.21.15.18:0/2981068171 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2ec010a5b0 con 0x7f2ec00724f0 2024-04-12T21:55:17.288 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:17.287+0000 7f2eb7400700 1 -- 172.21.15.18:0/2981068171 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7f2eb0013420 con 0x7f2ec00724f0 2024-04-12T21:55:17.290 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:17.287+0000 7f2eb7400700 1 --2- 172.21.15.18:0/2981068171 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f2ea406ec00 0x7f2ea40710b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:55:17.290 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:17.288+0000 7f2eb7400700 1 -- 172.21.15.18:0/2981068171 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 6757+0+0 (secure 0 0 0) 0x7f2eb008fd70 con 0x7f2ec00724f0 2024-04-12T21:55:17.290 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:17.288+0000 7f2ebea00700 1 --2- 172.21.15.18:0/2981068171 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f2ea406ec00 0x7f2ea40710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:55:17.290 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:17.289+0000 7f2ebea00700 1 --2- 172.21.15.18:0/2981068171 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f2ea406ec00 0x7f2ea40710b0 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7f2eac006fd0 tx=0x7f2eac008040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:55:17.295 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:17.293+0000 7f2eb7400700 1 -- 172.21.15.18:0/2981068171 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f2eb005c520 con 0x7f2ec00724f0 2024-04-12T21:55:17.536 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:17.534+0000 7f2ec6800700 1 -- 172.21.15.18:0/2981068171 --> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f2ec0110560 con 0x7f2ea406ec00 2024-04-12T21:55:17.567 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:17.564+0000 7f2eb7400700 1 -- 172.21.15.18:0/2981068171 <== mgr.34102 v2:172.21.15.18:6800/3812891919 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7f2ec0110560 con 0x7f2ea406ec00 2024-04-12T21:55:17.570 INFO:teuthology.orchestra.run.smithi018.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-12T21:55:17.570 INFO:teuthology.orchestra.run.smithi018.stdout:alertmanager.smithi018 smithi018 *:9093,9094 running (55m) 108s ago 70m 14.6M - 0.25.0 c8568f914cd2 c332090068e8 2024-04-12T21:55:17.570 INFO:teuthology.orchestra.run.smithi018.stdout:crash.smithi018 smithi018 running (52m) 108s ago 70m 6665k - 19.0.0-2879-g324baf8d 8717720dc5e5 b7f80d1f7982 2024-04-12T21:55:17.570 INFO:teuthology.orchestra.run.smithi018.stdout:crash.smithi186 smithi186 running (52m) 112s ago 69m 6660k - 19.0.0-2879-g324baf8d 8717720dc5e5 a4f55c81b623 2024-04-12T21:55:17.570 INFO:teuthology.orchestra.run.smithi018.stdout:grafana.smithi018 smithi018 *:3000 running (55m) 108s ago 70m 75.1M - 9.4.12 2bacad6d85d8 b08e3bf176dd 2024-04-12T21:55:17.570 INFO:teuthology.orchestra.run.smithi018.stdout:mds.cephfs.smithi018.acqbsc smithi018 running (48m) 108s ago 63m 102M - 19.0.0-2879-g324baf8d 8717720dc5e5 021f99ab24ae 2024-04-12T21:55:17.570 INFO:teuthology.orchestra.run.smithi018.stdout:mds.cephfs.smithi018.szjluo smithi018 running (47m) 108s ago 63m 106M - 19.0.0-2879-g324baf8d 8717720dc5e5 0003117479d4 2024-04-12T21:55:17.570 INFO:teuthology.orchestra.run.smithi018.stdout:mds.cephfs.smithi186.jdllyn smithi186 running (32m) 112s ago 63m 21.7M - 19.0.0-2879-g324baf8d 8717720dc5e5 21b1f46eef23 2024-04-12T21:55:17.570 INFO:teuthology.orchestra.run.smithi018.stdout:mds.cephfs.smithi186.wjnafs smithi186 running (17m) 112s ago 63m 16.5M - 19.0.0-2879-g324baf8d 8717720dc5e5 3cf1ef4148e2 2024-04-12T21:55:17.571 INFO:teuthology.orchestra.run.smithi018.stdout:mgr.smithi018.xjzwga smithi018 *:8443,9283,8765 running (58m) 108s ago 72m 521M - 19.0.0-2879-g324baf8d 8717720dc5e5 56088305eab7 2024-04-12T21:55:17.571 INFO:teuthology.orchestra.run.smithi018.stdout:mgr.smithi186.kqcfxk smithi186 *:8443,9283,8765 running (57m) 112s ago 69m 460M - 19.0.0-2879-g324baf8d 8717720dc5e5 09cd766e5e0d 2024-04-12T21:55:17.571 INFO:teuthology.orchestra.run.smithi018.stdout:mon.smithi018 smithi018 running (53m) 108s ago 72m 104M 2048M 19.0.0-2879-g324baf8d 8717720dc5e5 df3e201dfed6 2024-04-12T21:55:17.571 INFO:teuthology.orchestra.run.smithi018.stdout:mon.smithi186 smithi186 running (53m) 112s ago 68m 62.2M 2048M 19.0.0-2879-g324baf8d 8717720dc5e5 c77498da9b2b 2024-04-12T21:55:17.571 INFO:teuthology.orchestra.run.smithi018.stdout:node-exporter.smithi018 smithi018 *:9100 running (56m) 108s ago 70m 8812k - 1.5.0 0da6a335fe13 326c3d9c6847 2024-04-12T21:55:17.571 INFO:teuthology.orchestra.run.smithi018.stdout:node-exporter.smithi186 smithi186 *:9100 running (56m) 112s ago 69m 8786k - 1.5.0 0da6a335fe13 ec2112baeecf 2024-04-12T21:55:17.571 INFO:teuthology.orchestra.run.smithi018.stdout:osd.0 smithi018 running (51m) 108s ago 68m 117M 1559M 19.0.0-2879-g324baf8d 8717720dc5e5 60b1273ca047 2024-04-12T21:55:17.571 INFO:teuthology.orchestra.run.smithi018.stdout:osd.1 smithi018 running (51m) 108s ago 67m 102M 1559M 19.0.0-2879-g324baf8d 8717720dc5e5 170ce1e5fb8d 2024-04-12T21:55:17.571 INFO:teuthology.orchestra.run.smithi018.stdout:osd.2 smithi018 running (50m) 108s ago 66m 99.8M 1559M 19.0.0-2879-g324baf8d 8717720dc5e5 c2cbc6aa2839 2024-04-12T21:55:17.571 INFO:teuthology.orchestra.run.smithi018.stdout:osd.3 smithi186 running (49m) 112s ago 66m 107M 2581M 19.0.0-2879-g324baf8d 8717720dc5e5 0138e8262e4b 2024-04-12T21:55:17.571 INFO:teuthology.orchestra.run.smithi018.stdout:osd.4 smithi186 running (49m) 112s ago 65m 94.8M 2581M 19.0.0-2879-g324baf8d 8717720dc5e5 b20cb63b25c9 2024-04-12T21:55:17.571 INFO:teuthology.orchestra.run.smithi018.stdout:osd.5 smithi186 running (48m) 112s ago 65m 98.7M 2581M 19.0.0-2879-g324baf8d 8717720dc5e5 2513bea2610e 2024-04-12T21:55:17.571 INFO:teuthology.orchestra.run.smithi018.stdout:prometheus.smithi018 smithi018 *:9095 running (56m) 108s ago 69m 74.2M - 2.43.0 a07b618ecd1d f58841127e21 2024-04-12T21:55:17.572 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:17.568+0000 7f2ec6800700 1 -- 172.21.15.18:0/2981068171 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f2ea406ec00 msgr2=0x7f2ea40710b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:55:17.572 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:17.568+0000 7f2ec6800700 1 --2- 172.21.15.18:0/2981068171 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f2ea406ec00 0x7f2ea40710b0 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7f2eac006fd0 tx=0x7f2eac008040 comp rx=0 tx=0).stop 2024-04-12T21:55:17.572 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:17.568+0000 7f2ec6800700 1 -- 172.21.15.18:0/2981068171 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2ec00724f0 msgr2=0x7f2ec011a120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:55:17.572 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:17.568+0000 7f2ec6800700 1 --2- 172.21.15.18:0/2981068171 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2ec00724f0 0x7f2ec011a120 secure :-1 s=READY pgs=1377 cs=0 l=1 rev1=1 crypto rx=0x7f2eb0004890 tx=0x7f2eb0009e30 comp rx=0 tx=0).stop 2024-04-12T21:55:17.572 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:17.568+0000 7f2ec6800700 1 -- 172.21.15.18:0/2981068171 shutdown_connections 2024-04-12T21:55:17.572 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:17.568+0000 7f2ec6800700 1 --2- 172.21.15.18:0/2981068171 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f2ea406ec00 0x7f2ea40710b0 unknown :-1 s=CLOSED pgs=730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:55:17.573 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:17.568+0000 7f2ec6800700 1 --2- 172.21.15.18:0/2981068171 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2ec00724f0 0x7f2ec011a120 unknown :-1 s=CLOSED pgs=1377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:55:17.573 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:17.568+0000 7f2ec6800700 1 --2- 172.21.15.18:0/2981068171 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2ec0072e00 0x7f2ec011a660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:55:17.573 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:17.568+0000 7f2ec6800700 1 -- 172.21.15.18:0/2981068171 >> 172.21.15.18:0/2981068171 conn(0x7f2ec00fabd0 msgr2=0x7f2ec0107730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:55:17.573 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:17.569+0000 7f2ec6800700 1 -- 172.21.15.18:0/2981068171 shutdown_connections 2024-04-12T21:55:17.573 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:17.569+0000 7f2ec6800700 1 -- 172.21.15.18:0/2981068171 wait complete. 2024-04-12T21:55:18.337 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:55:18 smithi018 ceph-mon[112590]: from='client.36824 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:55:18.337 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:55:18 smithi018 ceph-mon[112590]: pgmap v1671: 65 pgs: 65 active+clean; 257 MiB data, 976 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:55:18.423 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1cbe407a-f90d-11ee-bc8d-c7b262605968 -e sha1=324baf8ddb91dea46cac50cb535c306115791355 -- bash -c 'ceph versions' 2024-04-12T21:55:18.535 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:55:18 smithi186 ceph-mon[86379]: from='client.36824 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T21:55:18.535 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:55:18 smithi186 ceph-mon[86379]: pgmap v1671: 65 pgs: 65 active+clean; 257 MiB data, 976 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:55:18.937 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/mon.smithi018/config 2024-04-12T21:55:20.537 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:55:20 smithi186 ceph-mon[86379]: pgmap v1672: 65 pgs: 65 active+clean; 257 MiB data, 976 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:55:20.541 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:55:20 smithi018 ceph-mon[112590]: pgmap v1672: 65 pgs: 65 active+clean; 257 MiB data, 976 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:55:21.030 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:21.027+0000 7f47e3e00700 1 -- 172.21.15.18:0/639313380 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f47dc0fe830 msgr2=0x7f47dc106f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:55:21.030 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:21.027+0000 7f47e3e00700 1 --2- 172.21.15.18:0/639313380 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f47dc0fe830 0x7f47dc106f00 secure :-1 s=READY pgs=1378 cs=0 l=1 rev1=1 crypto rx=0x7f47c8004940 tx=0x7f47c8009e30 comp rx=0 tx=0).stop 2024-04-12T21:55:21.030 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:21.027+0000 7f47e3e00700 1 -- 172.21.15.18:0/639313380 shutdown_connections 2024-04-12T21:55:21.030 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:21.027+0000 7f47e3e00700 1 --2- 172.21.15.18:0/639313380 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f47dc0fe830 0x7f47dc106f00 unknown :-1 s=CLOSED pgs=1378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:55:21.030 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:21.027+0000 7f47e3e00700 1 --2- 172.21.15.18:0/639313380 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f47dc0fde90 0x7f47dc0fe260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:55:21.030 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:21.027+0000 7f47e3e00700 1 -- 172.21.15.18:0/639313380 >> 172.21.15.18:0/639313380 conn(0x7f47dc072bc0 msgr2=0x7f47dc072fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:55:21.031 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:21.027+0000 7f47e3e00700 1 -- 172.21.15.18:0/639313380 shutdown_connections 2024-04-12T21:55:21.031 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:21.027+0000 7f47e3e00700 1 -- 172.21.15.18:0/639313380 wait complete. 2024-04-12T21:55:21.031 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:21.028+0000 7f47e3e00700 1 Processor -- start 2024-04-12T21:55:21.031 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:21.028+0000 7f47e3e00700 1 -- start start 2024-04-12T21:55:21.032 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:21.028+0000 7f47e3e00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f47dc0fde90 0x7f47dc115db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:55:21.032 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:21.028+0000 7f47e3e00700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f47dc0fe830 0x7f47dc1162f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:55:21.032 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:21.028+0000 7f47e3e00700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47dc116940 con 0x7f47dc0fde90 2024-04-12T21:55:21.032 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:21.028+0000 7f47e3e00700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47dc10fd10 con 0x7f47dc0fe830 2024-04-12T21:55:21.032 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:21.029+0000 7f47e1400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f47dc0fde90 0x7f47dc115db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:55:21.032 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:21.029+0000 7f47e1400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f47dc0fde90 0x7f47dc115db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:43672/0 (socket says 172.21.15.18:43672) 2024-04-12T21:55:21.032 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:21.029+0000 7f47e1400700 1 -- 172.21.15.18:0/1980930167 learned_addr learned my addr 172.21.15.18:0/1980930167 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:55:21.032 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:21.029+0000 7f47e0a00700 1 --2- 172.21.15.18:0/1980930167 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f47dc0fe830 0x7f47dc1162f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:55:21.032 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:21.029+0000 7f47e1400700 1 -- 172.21.15.18:0/1980930167 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f47dc0fe830 msgr2=0x7f47dc1162f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:55:21.032 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:21.029+0000 7f47e1400700 1 --2- 172.21.15.18:0/1980930167 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f47dc0fe830 0x7f47dc1162f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:55:21.032 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:21.030+0000 7f47e1400700 1 -- 172.21.15.18:0/1980930167 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47c8005040 con 0x7f47dc0fde90 2024-04-12T21:55:21.033 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:21.030+0000 7f47e1400700 1 --2- 172.21.15.18:0/1980930167 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f47dc0fde90 0x7f47dc115db0 secure :-1 s=READY pgs=1379 cs=0 l=1 rev1=1 crypto rx=0x7f47d0002a50 tx=0x7f47d00046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:55:21.033 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:21.030+0000 7f47d9600700 1 -- 172.21.15.18:0/1980930167 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f47d0007cc0 con 0x7f47dc0fde90 2024-04-12T21:55:21.033 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:21.030+0000 7f47d9600700 1 -- 172.21.15.18:0/1980930167 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f47d000f040 con 0x7f47dc0fde90 2024-04-12T21:55:21.033 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:21.030+0000 7f47d9600700 1 -- 172.21.15.18:0/1980930167 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f47d000da70 con 0x7f47dc0fde90 2024-04-12T21:55:21.033 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:21.030+0000 7f47e3e00700 1 -- 172.21.15.18:0/1980930167 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f47dc10fff0 con 0x7f47dc0fde90 2024-04-12T21:55:21.033 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:21.031+0000 7f47e3e00700 1 -- 172.21.15.18:0/1980930167 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f47dc110530 con 0x7f47dc0fde90 2024-04-12T21:55:21.034 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:21.032+0000 7f47e3e00700 1 -- 172.21.15.18:0/1980930167 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f47dc106a20 con 0x7f47dc0fde90 2024-04-12T21:55:21.035 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:21.033+0000 7f47d9600700 1 -- 172.21.15.18:0/1980930167 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7f47d000dc10 con 0x7f47dc0fde90 2024-04-12T21:55:21.036 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:21.034+0000 7f47d9600700 1 --2- 172.21.15.18:0/1980930167 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f47c406ec10 0x7f47c40710c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:55:21.036 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:21.034+0000 7f47d9600700 1 -- 172.21.15.18:0/1980930167 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 6757+0+0 (secure 0 0 0) 0x7f47d008dfd0 con 0x7f47dc0fde90 2024-04-12T21:55:21.036 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:21.034+0000 7f47e0a00700 1 --2- 172.21.15.18:0/1980930167 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f47c406ec10 0x7f47c40710c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:55:21.037 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:21.035+0000 7f47e0a00700 1 --2- 172.21.15.18:0/1980930167 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f47c406ec10 0x7f47c40710c0 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7f47dc111650 tx=0x7f47c8009e30 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:55:21.041 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:21.039+0000 7f47d9600700 1 -- 172.21.15.18:0/1980930167 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f47d005a780 con 0x7f47dc0fde90 2024-04-12T21:55:21.368 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:21.365+0000 7f47e3e00700 1 -- 172.21.15.18:0/1980930167 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f47dc109f30 con 0x7f47dc0fde90 2024-04-12T21:55:21.368 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:21.366+0000 7f47d9600700 1 -- 172.21.15.18:0/1980930167 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+623 (secure 0 0 0) 0x7f47d005a310 con 0x7f47dc0fde90 2024-04-12T21:55:21.369 INFO:teuthology.orchestra.run.smithi018.stdout:{ 2024-04-12T21:55:21.369 INFO:teuthology.orchestra.run.smithi018.stdout: "mon": { 2024-04-12T21:55:21.369 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 19.0.0-2879-g324baf8d (324baf8ddb91dea46cac50cb535c306115791355) squid (dev)": 2 2024-04-12T21:55:21.369 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-12T21:55:21.369 INFO:teuthology.orchestra.run.smithi018.stdout: "mgr": { 2024-04-12T21:55:21.369 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 19.0.0-2879-g324baf8d (324baf8ddb91dea46cac50cb535c306115791355) squid (dev)": 2 2024-04-12T21:55:21.369 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-12T21:55:21.369 INFO:teuthology.orchestra.run.smithi018.stdout: "osd": { 2024-04-12T21:55:21.369 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 19.0.0-2879-g324baf8d (324baf8ddb91dea46cac50cb535c306115791355) squid (dev)": 6 2024-04-12T21:55:21.369 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-12T21:55:21.369 INFO:teuthology.orchestra.run.smithi018.stdout: "mds": { 2024-04-12T21:55:21.369 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 19.0.0-2879-g324baf8d (324baf8ddb91dea46cac50cb535c306115791355) squid (dev)": 4 2024-04-12T21:55:21.369 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-04-12T21:55:21.370 INFO:teuthology.orchestra.run.smithi018.stdout: "overall": { 2024-04-12T21:55:21.370 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 19.0.0-2879-g324baf8d (324baf8ddb91dea46cac50cb535c306115791355) squid (dev)": 14 2024-04-12T21:55:21.370 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-04-12T21:55:21.370 INFO:teuthology.orchestra.run.smithi018.stdout:} 2024-04-12T21:55:21.371 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:21.369+0000 7f47e3e00700 1 -- 172.21.15.18:0/1980930167 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f47c406ec10 msgr2=0x7f47c40710c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:55:21.372 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:21.369+0000 7f47e3e00700 1 --2- 172.21.15.18:0/1980930167 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f47c406ec10 0x7f47c40710c0 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7f47dc111650 tx=0x7f47c8009e30 comp rx=0 tx=0).stop 2024-04-12T21:55:21.372 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:21.369+0000 7f47e3e00700 1 -- 172.21.15.18:0/1980930167 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f47dc0fde90 msgr2=0x7f47dc115db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:55:21.372 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:21.369+0000 7f47e3e00700 1 --2- 172.21.15.18:0/1980930167 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f47dc0fde90 0x7f47dc115db0 secure :-1 s=READY pgs=1379 cs=0 l=1 rev1=1 crypto rx=0x7f47d0002a50 tx=0x7f47d00046b0 comp rx=0 tx=0).stop 2024-04-12T21:55:21.372 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:21.369+0000 7f47e3e00700 1 -- 172.21.15.18:0/1980930167 shutdown_connections 2024-04-12T21:55:21.372 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:21.369+0000 7f47e3e00700 1 --2- 172.21.15.18:0/1980930167 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f47c406ec10 0x7f47c40710c0 unknown :-1 s=CLOSED pgs=731 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:55:21.372 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:21.369+0000 7f47e3e00700 1 --2- 172.21.15.18:0/1980930167 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f47dc0fde90 0x7f47dc115db0 unknown :-1 s=CLOSED pgs=1379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:55:21.372 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:21.369+0000 7f47e3e00700 1 --2- 172.21.15.18:0/1980930167 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f47dc0fe830 0x7f47dc1162f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:55:21.372 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:21.369+0000 7f47e3e00700 1 -- 172.21.15.18:0/1980930167 >> 172.21.15.18:0/1980930167 conn(0x7f47dc072bc0 msgr2=0x7f47dc0fb730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:55:21.372 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:21.370+0000 7f47e3e00700 1 -- 172.21.15.18:0/1980930167 shutdown_connections 2024-04-12T21:55:21.372 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:21.370+0000 7f47e3e00700 1 -- 172.21.15.18:0/1980930167 wait complete. 2024-04-12T21:55:21.733 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:55:21 smithi018 ceph-mon[112590]: from='client.? 172.21.15.18:0/1980930167' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T21:55:21.784 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:55:21 smithi186 ceph-mon[86379]: from='client.? 172.21.15.18:0/1980930167' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T21:55:22.168 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1cbe407a-f90d-11ee-bc8d-c7b262605968 -e sha1=324baf8ddb91dea46cac50cb535c306115791355 -- bash -c 'ceph versions | jq -e '"'"'.overall | length == 1'"'"'' 2024-04-12T21:55:22.691 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/mon.smithi018/config 2024-04-12T21:55:22.785 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:55:22 smithi186 ceph-mon[86379]: pgmap v1673: 65 pgs: 65 active+clean; 257 MiB data, 976 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:55:22.846 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:55:22 smithi018 ceph-mon[112590]: pgmap v1673: 65 pgs: 65 active+clean; 257 MiB data, 976 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:55:24.525 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:55:24 smithi018 ceph-mon[112590]: pgmap v1674: 65 pgs: 65 active+clean; 257 MiB data, 976 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:55:24.534 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:55:24 smithi186 ceph-mon[86379]: pgmap v1674: 65 pgs: 65 active+clean; 257 MiB data, 976 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:55:24.842 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:24.839+0000 7f68cb200700 1 -- 172.21.15.18:0/3811579645 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f68c40724f0 msgr2=0x7f68c40728c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:55:24.842 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:24.839+0000 7f68cb200700 1 --2- 172.21.15.18:0/3811579645 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f68c40724f0 0x7f68c40728c0 secure :-1 s=READY pgs=1380 cs=0 l=1 rev1=1 crypto rx=0x7f68ac004940 tx=0x7f68ac009e30 comp rx=0 tx=0).stop 2024-04-12T21:55:24.842 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:24.839+0000 7f68cb200700 1 -- 172.21.15.18:0/3811579645 shutdown_connections 2024-04-12T21:55:24.842 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:24.839+0000 7f68cb200700 1 --2- 172.21.15.18:0/3811579645 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f68c4072e00 0x7f68c410d780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:55:24.842 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:24.839+0000 7f68cb200700 1 --2- 172.21.15.18:0/3811579645 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f68c40724f0 0x7f68c40728c0 unknown :-1 s=CLOSED pgs=1380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:55:24.842 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:24.839+0000 7f68cb200700 1 -- 172.21.15.18:0/3811579645 >> 172.21.15.18:0/3811579645 conn(0x7f68c40faba0 msgr2=0x7f68c40fcfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:55:24.842 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:24.839+0000 7f68cb200700 1 -- 172.21.15.18:0/3811579645 shutdown_connections 2024-04-12T21:55:24.842 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:24.839+0000 7f68cb200700 1 -- 172.21.15.18:0/3811579645 wait complete. 2024-04-12T21:55:24.844 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:24.840+0000 7f68cb200700 1 Processor -- start 2024-04-12T21:55:24.844 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:24.840+0000 7f68cb200700 1 -- start start 2024-04-12T21:55:24.844 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:24.841+0000 7f68cb200700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f68c40724f0 0x7f68c411a130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:55:24.844 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:24.841+0000 7f68cb200700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f68c4072e00 0x7f68c411a670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:55:24.844 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:24.841+0000 7f68cb200700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68c411acc0 con 0x7f68c40724f0 2024-04-12T21:55:24.844 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:24.841+0000 7f68cb200700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68c4114090 con 0x7f68c4072e00 2024-04-12T21:55:24.844 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:24.841+0000 7f68c3e00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f68c40724f0 0x7f68c411a130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:55:24.844 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:24.841+0000 7f68c3e00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f68c40724f0 0x7f68c411a130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:43694/0 (socket says 172.21.15.18:43694) 2024-04-12T21:55:24.844 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:24.841+0000 7f68c3e00700 1 -- 172.21.15.18:0/3050708461 learned_addr learned my addr 172.21.15.18:0/3050708461 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:55:24.844 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:24.841+0000 7f68bb400700 1 --2- 172.21.15.18:0/3050708461 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f68c4072e00 0x7f68c411a670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:55:24.844 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:24.841+0000 7f68c3e00700 1 -- 172.21.15.18:0/3050708461 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f68c4072e00 msgr2=0x7f68c411a670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:55:24.844 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:24.841+0000 7f68c3e00700 1 --2- 172.21.15.18:0/3050708461 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f68c4072e00 0x7f68c411a670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:55:24.844 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:24.842+0000 7f68c3e00700 1 -- 172.21.15.18:0/3050708461 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68ac005040 con 0x7f68c40724f0 2024-04-12T21:55:24.845 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:24.842+0000 7f68c3e00700 1 --2- 172.21.15.18:0/3050708461 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f68c40724f0 0x7f68c411a130 secure :-1 s=READY pgs=1381 cs=0 l=1 rev1=1 crypto rx=0x7f68ac004890 tx=0x7f68ac009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:55:24.846 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:24.842+0000 7f68c0c00700 1 -- 172.21.15.18:0/3050708461 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f68ac00e040 con 0x7f68c40724f0 2024-04-12T21:55:24.846 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:24.842+0000 7f68c0c00700 1 -- 172.21.15.18:0/3050708461 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f68ac009af0 con 0x7f68c40724f0 2024-04-12T21:55:24.846 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:24.842+0000 7f68c0c00700 1 -- 172.21.15.18:0/3050708461 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f68ac01c630 con 0x7f68c40724f0 2024-04-12T21:55:24.846 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:24.842+0000 7f68cb200700 1 -- 172.21.15.18:0/3050708461 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f68c4114310 con 0x7f68c40724f0 2024-04-12T21:55:24.846 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:24.842+0000 7f68cb200700 1 -- 172.21.15.18:0/3050708461 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f68c41148d0 con 0x7f68c40724f0 2024-04-12T21:55:24.846 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:24.843+0000 7f68cb200700 1 -- 172.21.15.18:0/3050708461 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f68c410a5c0 con 0x7f68c40724f0 2024-04-12T21:55:24.847 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:24.845+0000 7f68c0c00700 1 -- 172.21.15.18:0/3050708461 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7f68ac00bbc0 con 0x7f68c40724f0 2024-04-12T21:55:24.848 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:24.846+0000 7f68c0c00700 1 --2- 172.21.15.18:0/3050708461 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f68a806ec00 0x7f68a80710b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:55:24.848 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:24.846+0000 7f68c0c00700 1 -- 172.21.15.18:0/3050708461 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 6757+0+0 (secure 0 0 0) 0x7f68ac08fca0 con 0x7f68c40724f0 2024-04-12T21:55:24.849 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:24.847+0000 7f68bb400700 1 --2- 172.21.15.18:0/3050708461 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f68a806ec00 0x7f68a80710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:55:24.849 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:24.847+0000 7f68bb400700 1 --2- 172.21.15.18:0/3050708461 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f68a806ec00 0x7f68a80710b0 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7f68c41159d0 tx=0x7f68b400b040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:55:24.853 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:24.851+0000 7f68c0c00700 1 -- 172.21.15.18:0/3050708461 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f68ac05c4d0 con 0x7f68c40724f0 2024-04-12T21:55:25.200 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:25.197+0000 7f68cb200700 1 -- 172.21.15.18:0/3050708461 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f68c404bed0 con 0x7f68c40724f0 2024-04-12T21:55:25.201 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:25.198+0000 7f68c0c00700 1 -- 172.21.15.18:0/3050708461 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+623 (secure 0 0 0) 0x7f68ac05c060 con 0x7f68c40724f0 2024-04-12T21:55:25.204 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:25.201+0000 7f68cb200700 1 -- 172.21.15.18:0/3050708461 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f68a806ec00 msgr2=0x7f68a80710b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:55:25.204 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:25.201+0000 7f68cb200700 1 --2- 172.21.15.18:0/3050708461 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f68a806ec00 0x7f68a80710b0 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7f68c41159d0 tx=0x7f68b400b040 comp rx=0 tx=0).stop 2024-04-12T21:55:25.204 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:25.201+0000 7f68cb200700 1 -- 172.21.15.18:0/3050708461 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f68c40724f0 msgr2=0x7f68c411a130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:55:25.204 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:25.201+0000 7f68cb200700 1 --2- 172.21.15.18:0/3050708461 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f68c40724f0 0x7f68c411a130 secure :-1 s=READY pgs=1381 cs=0 l=1 rev1=1 crypto rx=0x7f68ac004890 tx=0x7f68ac009e30 comp rx=0 tx=0).stop 2024-04-12T21:55:25.205 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:25.201+0000 7f68cb200700 1 -- 172.21.15.18:0/3050708461 shutdown_connections 2024-04-12T21:55:25.205 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:25.201+0000 7f68cb200700 1 --2- 172.21.15.18:0/3050708461 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f68a806ec00 0x7f68a80710b0 unknown :-1 s=CLOSED pgs=732 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:55:25.205 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:25.201+0000 7f68cb200700 1 --2- 172.21.15.18:0/3050708461 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f68c40724f0 0x7f68c411a130 unknown :-1 s=CLOSED pgs=1381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:55:25.205 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:25.201+0000 7f68cb200700 1 --2- 172.21.15.18:0/3050708461 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f68c4072e00 0x7f68c411a670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:55:25.205 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:25.201+0000 7f68cb200700 1 -- 172.21.15.18:0/3050708461 >> 172.21.15.18:0/3050708461 conn(0x7f68c40faba0 msgr2=0x7f68c4107740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:55:25.205 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:25.202+0000 7f68cb200700 1 -- 172.21.15.18:0/3050708461 shutdown_connections 2024-04-12T21:55:25.205 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:25.202+0000 7f68cb200700 1 -- 172.21.15.18:0/3050708461 wait complete. 2024-04-12T21:55:25.221 INFO:teuthology.orchestra.run.smithi018.stdout:true 2024-04-12T21:55:25.534 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:55:25 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T21:55:25.535 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:55:25 smithi186 ceph-mon[86379]: from='client.? 172.21.15.18:0/3050708461' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T21:55:25.572 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:55:25 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T21:55:25.572 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:55:25 smithi018 ceph-mon[112590]: from='client.? 172.21.15.18:0/3050708461' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T21:55:26.017 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1cbe407a-f90d-11ee-bc8d-c7b262605968 -e sha1=324baf8ddb91dea46cac50cb535c306115791355 -- bash -c 'ceph versions | jq -e '"'"'.overall | keys'"'"' | grep $sha1' 2024-04-12T21:55:26.534 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:55:26 smithi186 ceph-mon[86379]: pgmap v1675: 65 pgs: 65 active+clean; 257 MiB data, 976 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:55:26.536 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/mon.smithi018/config 2024-04-12T21:55:26.605 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:55:26 smithi018 ceph-mon[112590]: pgmap v1675: 65 pgs: 65 active+clean; 257 MiB data, 976 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:55:28.498 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:55:28 smithi018 ceph-mon[112590]: pgmap v1676: 65 pgs: 65 active+clean; 257 MiB data, 976 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:55:28.534 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:55:28 smithi186 ceph-mon[86379]: pgmap v1676: 65 pgs: 65 active+clean; 257 MiB data, 976 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:55:28.794 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:28.791+0000 7fd58cc00700 1 -- 172.21.15.18:0/2670274180 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd588109430 msgr2=0x7fd588109800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:55:28.794 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:28.791+0000 7fd58cc00700 1 --2- 172.21.15.18:0/2670274180 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd588109430 0x7fd588109800 secure :-1 s=READY pgs=1382 cs=0 l=1 rev1=1 crypto rx=0x7fd578004940 tx=0x7fd578009e30 comp rx=0 tx=0).stop 2024-04-12T21:55:28.794 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:28.791+0000 7fd58cc00700 1 -- 172.21.15.18:0/2670274180 shutdown_connections 2024-04-12T21:55:28.794 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:28.791+0000 7fd58cc00700 1 --2- 172.21.15.18:0/2670274180 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd588109dd0 0x7fd588103440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:55:28.794 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:28.791+0000 7fd58cc00700 1 --2- 172.21.15.18:0/2670274180 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd588109430 0x7fd588109800 unknown :-1 s=CLOSED pgs=1382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:55:28.794 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:28.791+0000 7fd58cc00700 1 -- 172.21.15.18:0/2670274180 >> 172.21.15.18:0/2670274180 conn(0x7fd588072c00 msgr2=0x7fd588075010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:55:28.794 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:28.791+0000 7fd58cc00700 1 -- 172.21.15.18:0/2670274180 shutdown_connections 2024-04-12T21:55:28.794 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:28.791+0000 7fd58cc00700 1 -- 172.21.15.18:0/2670274180 wait complete. 2024-04-12T21:55:28.795 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:28.792+0000 7fd58cc00700 1 Processor -- start 2024-04-12T21:55:28.795 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:28.792+0000 7fd58cc00700 1 -- start start 2024-04-12T21:55:28.795 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:28.792+0000 7fd58cc00700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd588109430 0x7fd588119f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:55:28.795 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:28.793+0000 7fd58cc00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd588109dd0 0x7fd58811a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:55:28.795 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:28.793+0000 7fd58cc00700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd58811ab60 con 0x7fd588109dd0 2024-04-12T21:55:28.796 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:28.793+0000 7fd58cc00700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5881140b0 con 0x7fd588109430 2024-04-12T21:55:28.796 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:28.793+0000 7fd57d400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd588109dd0 0x7fd58811a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:55:28.796 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:28.793+0000 7fd57d400700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd588109dd0 0x7fd58811a480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:43712/0 (socket says 172.21.15.18:43712) 2024-04-12T21:55:28.796 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:28.793+0000 7fd57d400700 1 -- 172.21.15.18:0/3194102087 learned_addr learned my addr 172.21.15.18:0/3194102087 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:55:28.796 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:28.793+0000 7fd585e00700 1 --2- 172.21.15.18:0/3194102087 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd588109430 0x7fd588119f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:55:28.797 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:28.793+0000 7fd57d400700 1 -- 172.21.15.18:0/3194102087 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd588109430 msgr2=0x7fd588119f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:55:28.797 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:28.793+0000 7fd57d400700 1 --2- 172.21.15.18:0/3194102087 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd588109430 0x7fd588119f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:55:28.797 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:28.793+0000 7fd57d400700 1 -- 172.21.15.18:0/3194102087 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd578005040 con 0x7fd588109dd0 2024-04-12T21:55:28.797 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:28.794+0000 7fd57d400700 1 --2- 172.21.15.18:0/3194102087 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd588109dd0 0x7fd58811a480 secure :-1 s=READY pgs=1383 cs=0 l=1 rev1=1 crypto rx=0x7fd57000bae0 tx=0x7fd570009810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:55:28.797 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:28.794+0000 7fd57ea00700 1 -- 172.21.15.18:0/3194102087 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd57000c8c0 con 0x7fd588109dd0 2024-04-12T21:55:28.797 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:28.794+0000 7fd57ea00700 1 -- 172.21.15.18:0/3194102087 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd570014410 con 0x7fd588109dd0 2024-04-12T21:55:28.797 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:28.794+0000 7fd57ea00700 1 -- 172.21.15.18:0/3194102087 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd570007610 con 0x7fd588109dd0 2024-04-12T21:55:28.797 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:28.794+0000 7fd58cc00700 1 -- 172.21.15.18:0/3194102087 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd588114330 con 0x7fd588109dd0 2024-04-12T21:55:28.797 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:28.794+0000 7fd58cc00700 1 -- 172.21.15.18:0/3194102087 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd5881148f0 con 0x7fd588109dd0 2024-04-12T21:55:28.798 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:28.796+0000 7fd58cc00700 1 -- 172.21.15.18:0/3194102087 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd588104b80 con 0x7fd588109dd0 2024-04-12T21:55:28.799 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:28.797+0000 7fd57ea00700 1 -- 172.21.15.18:0/3194102087 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7fd570014580 con 0x7fd588109dd0 2024-04-12T21:55:28.800 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:28.798+0000 7fd57ea00700 1 --2- 172.21.15.18:0/3194102087 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fd56c06ec00 0x7fd56c0710b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:55:28.800 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:28.798+0000 7fd57ea00700 1 -- 172.21.15.18:0/3194102087 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 6757+0+0 (secure 0 0 0) 0x7fd57008eb40 con 0x7fd588109dd0 2024-04-12T21:55:28.800 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:28.798+0000 7fd585e00700 1 --2- 172.21.15.18:0/3194102087 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fd56c06ec00 0x7fd56c0710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:55:28.801 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:28.799+0000 7fd585e00700 1 --2- 172.21.15.18:0/3194102087 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fd56c06ec00 0x7fd56c0710b0 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7fd57800b020 tx=0x7fd578009e30 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:55:28.805 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:28.803+0000 7fd57ea00700 1 -- 172.21.15.18:0/3194102087 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7fd57005b270 con 0x7fd588109dd0 2024-04-12T21:55:29.135 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:29.132+0000 7fd58cc00700 1 -- 172.21.15.18:0/3194102087 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fd58804c710 con 0x7fd588109dd0 2024-04-12T21:55:29.135 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:29.133+0000 7fd57ea00700 1 -- 172.21.15.18:0/3194102087 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+623 (secure 0 0 0) 0x7fd57005ae00 con 0x7fd588109dd0 2024-04-12T21:55:29.138 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:29.136+0000 7fd58cc00700 1 -- 172.21.15.18:0/3194102087 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fd56c06ec00 msgr2=0x7fd56c0710b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:55:29.138 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:29.136+0000 7fd58cc00700 1 --2- 172.21.15.18:0/3194102087 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fd56c06ec00 0x7fd56c0710b0 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7fd57800b020 tx=0x7fd578009e30 comp rx=0 tx=0).stop 2024-04-12T21:55:29.138 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:29.136+0000 7fd58cc00700 1 -- 172.21.15.18:0/3194102087 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd588109dd0 msgr2=0x7fd58811a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:55:29.139 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:29.136+0000 7fd58cc00700 1 --2- 172.21.15.18:0/3194102087 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd588109dd0 0x7fd58811a480 secure :-1 s=READY pgs=1383 cs=0 l=1 rev1=1 crypto rx=0x7fd57000bae0 tx=0x7fd570009810 comp rx=0 tx=0).stop 2024-04-12T21:55:29.139 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:29.136+0000 7fd58cc00700 1 -- 172.21.15.18:0/3194102087 shutdown_connections 2024-04-12T21:55:29.139 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:29.136+0000 7fd58cc00700 1 --2- 172.21.15.18:0/3194102087 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fd56c06ec00 0x7fd56c0710b0 unknown :-1 s=CLOSED pgs=733 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:55:29.139 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:29.136+0000 7fd58cc00700 1 --2- 172.21.15.18:0/3194102087 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd588109430 0x7fd588119f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:55:29.139 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:29.136+0000 7fd58cc00700 1 --2- 172.21.15.18:0/3194102087 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd588109dd0 0x7fd58811a480 unknown :-1 s=CLOSED pgs=1383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:55:29.139 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:29.136+0000 7fd58cc00700 1 -- 172.21.15.18:0/3194102087 >> 172.21.15.18:0/3194102087 conn(0x7fd588072c00 msgr2=0x7fd588074c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:55:29.139 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:29.137+0000 7fd58cc00700 1 -- 172.21.15.18:0/3194102087 shutdown_connections 2024-04-12T21:55:29.140 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:29.137+0000 7fd58cc00700 1 -- 172.21.15.18:0/3194102087 wait complete. 2024-04-12T21:55:29.155 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph version 19.0.0-2879-g324baf8d (324baf8ddb91dea46cac50cb535c306115791355) squid (dev)" 2024-04-12T21:55:29.528 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:55:29 smithi018 ceph-mon[112590]: from='client.? 172.21.15.18:0/3194102087' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T21:55:29.534 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:55:29 smithi186 ceph-mon[86379]: from='client.? 172.21.15.18:0/3194102087' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T21:55:29.956 DEBUG:teuthology.parallel:result is None 2024-04-12T21:55:29.956 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-04-12T21:55:29.968 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi018.front.sepia.ceph.com 2024-04-12T21:55:29.968 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1cbe407a-f90d-11ee-bc8d-c7b262605968 -- bash -c 'ceph fs dump' 2024-04-12T21:55:30.487 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/mon.smithi018/config 2024-04-12T21:55:30.534 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:55:30 smithi186 ceph-mon[86379]: pgmap v1677: 65 pgs: 65 active+clean; 257 MiB data, 976 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:55:30.605 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:55:30 smithi018 ceph-mon[112590]: pgmap v1677: 65 pgs: 65 active+clean; 257 MiB data, 976 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:55:32.395 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:55:32 smithi018 ceph-mon[112590]: pgmap v1678: 65 pgs: 65 active+clean; 257 MiB data, 976 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:55:32.534 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:55:32 smithi186 ceph-mon[86379]: pgmap v1678: 65 pgs: 65 active+clean; 257 MiB data, 976 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:55:32.670 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:32.667+0000 7f4fe9e00700 1 -- 172.21.15.18:0/368287039 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4fe40fbbf0 msgr2=0x7f4fe40fbfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:55:32.670 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:32.667+0000 7f4fe9e00700 1 --2- 172.21.15.18:0/368287039 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4fe40fbbf0 0x7f4fe40fbfc0 secure :-1 s=READY pgs=1384 cs=0 l=1 rev1=1 crypto rx=0x7f4fd4004940 tx=0x7f4fd4009e30 comp rx=0 tx=0).stop 2024-04-12T21:55:32.670 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:32.667+0000 7f4fe9e00700 1 -- 172.21.15.18:0/368287039 shutdown_connections 2024-04-12T21:55:32.670 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:32.667+0000 7f4fe9e00700 1 --2- 172.21.15.18:0/368287039 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4fe40fc500 0x7f4fe4109260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:55:32.670 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:32.667+0000 7f4fe9e00700 1 --2- 172.21.15.18:0/368287039 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4fe40fbbf0 0x7f4fe40fbfc0 unknown :-1 s=CLOSED pgs=1384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:55:32.670 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:32.667+0000 7f4fe9e00700 1 -- 172.21.15.18:0/368287039 >> 172.21.15.18:0/368287039 conn(0x7f4fe40f7880 msgr2=0x7f4fe40f9c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:55:32.670 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:32.667+0000 7f4fe9e00700 1 -- 172.21.15.18:0/368287039 shutdown_connections 2024-04-12T21:55:32.671 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:32.667+0000 7f4fe9e00700 1 -- 172.21.15.18:0/368287039 wait complete. 2024-04-12T21:55:32.672 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:32.668+0000 7f4fe9e00700 1 Processor -- start 2024-04-12T21:55:32.672 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:32.668+0000 7f4fe9e00700 1 -- start start 2024-04-12T21:55:32.672 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:32.669+0000 7f4fe9e00700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4fe40fbbf0 0x7f4fe41160c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:55:32.672 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:32.669+0000 7f4fe9e00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4fe40fc500 0x7f4fe4116600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:55:32.672 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:32.669+0000 7f4fe9e00700 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fe4116c50 con 0x7f4fe40fc500 2024-04-12T21:55:32.672 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:32.669+0000 7f4fe9e00700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fe4110020 con 0x7f4fe40fbbf0 2024-04-12T21:55:32.672 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:32.669+0000 7f4fdbe00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4fe40fc500 0x7f4fe4116600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:55:32.672 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:32.669+0000 7f4fdbe00700 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4fe40fc500 0x7f4fe4116600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:50918/0 (socket says 172.21.15.18:50918) 2024-04-12T21:55:32.672 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:32.669+0000 7f4fdbe00700 1 -- 172.21.15.18:0/2641872696 learned_addr learned my addr 172.21.15.18:0/2641872696 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:55:32.672 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:32.670+0000 7f4fe2a00700 1 --2- 172.21.15.18:0/2641872696 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4fe40fbbf0 0x7f4fe41160c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:55:32.672 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:32.670+0000 7f4fdbe00700 1 -- 172.21.15.18:0/2641872696 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4fe40fbbf0 msgr2=0x7f4fe41160c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:55:32.673 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:32.670+0000 7f4fdbe00700 1 --2- 172.21.15.18:0/2641872696 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4fe40fbbf0 0x7f4fe41160c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:55:32.673 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:32.670+0000 7f4fdbe00700 1 -- 172.21.15.18:0/2641872696 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4fd4005040 con 0x7f4fe40fc500 2024-04-12T21:55:32.673 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:32.670+0000 7f4fdbe00700 1 --2- 172.21.15.18:0/2641872696 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4fe40fc500 0x7f4fe4116600 secure :-1 s=READY pgs=1385 cs=0 l=1 rev1=1 crypto rx=0x7f4fcc002a50 tx=0x7f4fcc0046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:55:32.674 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:32.670+0000 7f4fdaa00700 1 -- 172.21.15.18:0/2641872696 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4fcc007cc0 con 0x7f4fe40fc500 2024-04-12T21:55:32.674 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:32.670+0000 7f4fdaa00700 1 -- 172.21.15.18:0/2641872696 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f4fcc00f040 con 0x7f4fe40fc500 2024-04-12T21:55:32.674 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:32.670+0000 7f4fe9e00700 1 -- 172.21.15.18:0/2641872696 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4fe4110300 con 0x7f4fe40fc500 2024-04-12T21:55:32.674 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:32.671+0000 7f4fdaa00700 1 -- 172.21.15.18:0/2641872696 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4fcc009640 con 0x7f4fe40fc500 2024-04-12T21:55:32.674 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:32.671+0000 7f4fe9e00700 1 -- 172.21.15.18:0/2641872696 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4fe41108c0 con 0x7f4fe40fc500 2024-04-12T21:55:32.674 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:32.672+0000 7f4fe9e00700 1 -- 172.21.15.18:0/2641872696 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4fe4106990 con 0x7f4fe40fc500 2024-04-12T21:55:32.675 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:32.673+0000 7f4fdaa00700 1 -- 172.21.15.18:0/2641872696 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7f4fcc0097a0 con 0x7f4fe40fc500 2024-04-12T21:55:32.676 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:32.674+0000 7f4fdaa00700 1 --2- 172.21.15.18:0/2641872696 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f4fc806ec00 0x7f4fc80710b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:55:32.676 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:32.674+0000 7f4fdaa00700 1 -- 172.21.15.18:0/2641872696 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 6757+0+0 (secure 0 0 0) 0x7f4fcc08e760 con 0x7f4fe40fc500 2024-04-12T21:55:32.676 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:32.674+0000 7f4fe2a00700 1 --2- 172.21.15.18:0/2641872696 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f4fc806ec00 0x7f4fc80710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:55:32.677 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:32.675+0000 7f4fe2a00700 1 --2- 172.21.15.18:0/2641872696 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f4fc806ec00 0x7f4fc80710b0 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7f4fd400b3b0 tx=0x7f4fd4009e30 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:55:32.681 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:32.679+0000 7f4fdaa00700 1 -- 172.21.15.18:0/2641872696 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f4fcc05af10 con 0x7f4fe40fc500 2024-04-12T21:55:32.966 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:32.963+0000 7f4fe9e00700 1 -- 172.21.15.18:0/2641872696 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f4fe4111710 con 0x7f4fe40fc500 2024-04-12T21:55:32.967 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:32.964+0000 7f4fdaa00700 1 -- 172.21.15.18:0/2641872696 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 38 v38) v1 ==== 76+0+1956 (secure 0 0 0) 0x7f4fcc05aaa0 con 0x7f4fe40fc500 2024-04-12T21:55:32.967 INFO:teuthology.orchestra.run.smithi018.stdout:e38 2024-04-12T21:55:32.967 INFO:teuthology.orchestra.run.smithi018.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-12T21:55:32.967 INFO:teuthology.orchestra.run.smithi018.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-12T21:55:32.967 INFO:teuthology.orchestra.run.smithi018.stdout:legacy client fscid: 1 2024-04-12T21:55:32.968 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-12T21:55:32.968 INFO:teuthology.orchestra.run.smithi018.stdout:Filesystem 'cephfs' (1) 2024-04-12T21:55:32.968 INFO:teuthology.orchestra.run.smithi018.stdout:fs_name cephfs 2024-04-12T21:55:32.968 INFO:teuthology.orchestra.run.smithi018.stdout:epoch 38 2024-04-12T21:55:32.968 INFO:teuthology.orchestra.run.smithi018.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-04-12T21:55:32.968 INFO:teuthology.orchestra.run.smithi018.stdout:created 2024-04-12T20:51:57.362850+0000 2024-04-12T21:55:32.968 INFO:teuthology.orchestra.run.smithi018.stdout:modified 2024-04-12T21:53:30.672912+0000 2024-04-12T21:55:32.968 INFO:teuthology.orchestra.run.smithi018.stdout:tableserver 0 2024-04-12T21:55:32.968 INFO:teuthology.orchestra.run.smithi018.stdout:root 0 2024-04-12T21:55:32.968 INFO:teuthology.orchestra.run.smithi018.stdout:session_timeout 60 2024-04-12T21:55:32.968 INFO:teuthology.orchestra.run.smithi018.stdout:session_autoclose 300 2024-04-12T21:55:32.968 INFO:teuthology.orchestra.run.smithi018.stdout:max_file_size 1099511627776 2024-04-12T21:55:32.968 INFO:teuthology.orchestra.run.smithi018.stdout:max_xattr_size 65536 2024-04-12T21:55:32.968 INFO:teuthology.orchestra.run.smithi018.stdout:required_client_features {} 2024-04-12T21:55:32.968 INFO:teuthology.orchestra.run.smithi018.stdout:last_failure 0 2024-04-12T21:55:32.968 INFO:teuthology.orchestra.run.smithi018.stdout:last_failure_osd_epoch 100 2024-04-12T21:55:32.968 INFO:teuthology.orchestra.run.smithi018.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-04-12T21:55:32.969 INFO:teuthology.orchestra.run.smithi018.stdout:max_mds 2 2024-04-12T21:55:32.969 INFO:teuthology.orchestra.run.smithi018.stdout:in 0,1 2024-04-12T21:55:32.969 INFO:teuthology.orchestra.run.smithi018.stdout:up {0=34436,1=34468} 2024-04-12T21:55:32.969 INFO:teuthology.orchestra.run.smithi018.stdout:failed 2024-04-12T21:55:32.969 INFO:teuthology.orchestra.run.smithi018.stdout:damaged 2024-04-12T21:55:32.969 INFO:teuthology.orchestra.run.smithi018.stdout:stopped 2024-04-12T21:55:32.969 INFO:teuthology.orchestra.run.smithi018.stdout:data_pools [3] 2024-04-12T21:55:32.969 INFO:teuthology.orchestra.run.smithi018.stdout:metadata_pool 2 2024-04-12T21:55:32.969 INFO:teuthology.orchestra.run.smithi018.stdout:inline_data disabled 2024-04-12T21:55:32.969 INFO:teuthology.orchestra.run.smithi018.stdout:balancer 2024-04-12T21:55:32.969 INFO:teuthology.orchestra.run.smithi018.stdout:bal_rank_mask -1 2024-04-12T21:55:32.969 INFO:teuthology.orchestra.run.smithi018.stdout:standby_count_wanted 1 2024-04-12T21:55:32.969 INFO:teuthology.orchestra.run.smithi018.stdout:qdb_cluster leader: 34436 members: 34436,34468 2024-04-12T21:55:32.969 INFO:teuthology.orchestra.run.smithi018.stdout:[mds.cephfs.smithi018.acqbsc{0:34436} state up:active seq 699 join_fscid=1 addr [v2:172.21.15.18:6828/755179764,v1:172.21.15.18:6829/755179764] compat {c=[1],r=[1],i=[1fff]}] 2024-04-12T21:55:32.969 INFO:teuthology.orchestra.run.smithi018.stdout:[mds.cephfs.smithi018.szjluo{1:34468} state up:active seq 692 join_fscid=1 addr [v2:172.21.15.18:6826/2407992653,v1:172.21.15.18:6827/2407992653] compat {c=[1],r=[1],i=[1fff]}] 2024-04-12T21:55:32.969 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-12T21:55:32.970 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-12T21:55:32.970 INFO:teuthology.orchestra.run.smithi018.stdout:Standby daemons: 2024-04-12T21:55:32.970 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-12T21:55:32.970 INFO:teuthology.orchestra.run.smithi018.stdout:[mds.cephfs.smithi186.jdllyn{-1:44595} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.186:6824/3523444202,v1:172.21.15.186:6825/3523444202] compat {c=[1],r=[1],i=[1fff]}] 2024-04-12T21:55:32.970 INFO:teuthology.orchestra.run.smithi018.stdout:[mds.cephfs.smithi186.wjnafs{-1:44875} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.186:6826/3122443766,v1:172.21.15.186:6827/3122443766] compat {c=[1],r=[1],i=[1fff]}] 2024-04-12T21:55:32.971 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:32.969+0000 7f4fe9e00700 1 -- 172.21.15.18:0/2641872696 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f4fc806ec00 msgr2=0x7f4fc80710b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:55:32.971 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:32.969+0000 7f4fe9e00700 1 --2- 172.21.15.18:0/2641872696 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f4fc806ec00 0x7f4fc80710b0 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7f4fd400b3b0 tx=0x7f4fd4009e30 comp rx=0 tx=0).stop 2024-04-12T21:55:32.971 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:32.969+0000 7f4fe9e00700 1 -- 172.21.15.18:0/2641872696 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4fe40fc500 msgr2=0x7f4fe4116600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:55:32.971 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:32.969+0000 7f4fe9e00700 1 --2- 172.21.15.18:0/2641872696 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4fe40fc500 0x7f4fe4116600 secure :-1 s=READY pgs=1385 cs=0 l=1 rev1=1 crypto rx=0x7f4fcc002a50 tx=0x7f4fcc0046b0 comp rx=0 tx=0).stop 2024-04-12T21:55:32.972 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:32.969+0000 7f4fe9e00700 1 -- 172.21.15.18:0/2641872696 shutdown_connections 2024-04-12T21:55:32.972 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:32.969+0000 7f4fe9e00700 1 --2- 172.21.15.18:0/2641872696 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f4fc806ec00 0x7f4fc80710b0 unknown :-1 s=CLOSED pgs=734 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:55:32.972 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:32.969+0000 7f4fe9e00700 1 --2- 172.21.15.18:0/2641872696 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4fe40fbbf0 0x7f4fe41160c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:55:32.972 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:32.969+0000 7f4fe9e00700 1 --2- 172.21.15.18:0/2641872696 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4fe40fc500 0x7f4fe4116600 unknown :-1 s=CLOSED pgs=1385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:55:32.972 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:32.969+0000 7f4fe9e00700 1 -- 172.21.15.18:0/2641872696 >> 172.21.15.18:0/2641872696 conn(0x7f4fe40f7880 msgr2=0x7f4fe40f8140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:55:32.972 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:32.969+0000 7f4fe9e00700 1 -- 172.21.15.18:0/2641872696 shutdown_connections 2024-04-12T21:55:32.972 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:32.969+0000 7f4fe9e00700 1 -- 172.21.15.18:0/2641872696 wait complete. 2024-04-12T21:55:32.973 INFO:teuthology.orchestra.run.smithi018.stderr:dumped fsmap epoch 38 2024-04-12T21:55:33.534 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:55:33 smithi186 ceph-mon[86379]: from='client.? 172.21.15.18:0/2641872696' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-12T21:55:33.605 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:55:33 smithi018 ceph-mon[112590]: from='client.? 172.21.15.18:0/2641872696' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-12T21:55:33.812 INFO:teuthology.run_tasks:Running task fs.post_upgrade_checks... 2024-04-12T21:55:33.823 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-12T21:55:34.053 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:34.050+0000 7f488be00640 1 -- 172.21.15.18:0/3123511668 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f488c104220 msgr2=0x7f488c1045f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:55:34.053 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:34.050+0000 7f488be00640 1 --2- 172.21.15.18:0/3123511668 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f488c104220 0x7f488c1045f0 secure :-1 s=READY pgs=1386 cs=0 l=1 rev1=1 crypto rx=0x7f487c0048d0 tx=0x7f487c009da0 comp rx=0 tx=0).stop 2024-04-12T21:55:34.053 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:34.051+0000 7f488be00640 1 -- 172.21.15.18:0/3123511668 shutdown_connections 2024-04-12T21:55:34.053 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:34.051+0000 7f488be00640 1 --2- 172.21.15.18:0/3123511668 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f488c104bc0 0x7f488c0fe2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:55:34.053 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:34.051+0000 7f488be00640 1 --2- 172.21.15.18:0/3123511668 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f488c104220 0x7f488c1045f0 unknown :-1 s=CLOSED pgs=1386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:55:34.053 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:34.051+0000 7f488be00640 1 -- 172.21.15.18:0/3123511668 >> 172.21.15.18:0/3123511668 conn(0x7f488c0f9fc0 msgr2=0x7f488c0fc3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:55:34.054 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:34.051+0000 7f488be00640 1 -- 172.21.15.18:0/3123511668 shutdown_connections 2024-04-12T21:55:34.054 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:34.051+0000 7f488be00640 1 -- 172.21.15.18:0/3123511668 wait complete. 2024-04-12T21:55:34.055 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:34.052+0000 7f488be00640 1 Processor -- start 2024-04-12T21:55:34.055 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:34.052+0000 7f488be00640 1 -- start start 2024-04-12T21:55:34.055 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:34.052+0000 7f488be00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f488c104220 0x7f488c114d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:55:34.055 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:34.052+0000 7f488be00640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f488c104bc0 0x7f488c115280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:55:34.055 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:34.052+0000 7f488be00640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f488c115880 con 0x7f488c104220 2024-04-12T21:55:34.055 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:34.052+0000 7f488be00640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f488c10eed0 con 0x7f488c104bc0 2024-04-12T21:55:34.055 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:34.052+0000 7f488aa00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f488c104220 0x7f488c114d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:55:34.055 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:34.052+0000 7f488aa00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f488c104220 0x7f488c114d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:50930/0 (socket says 172.21.15.18:50930) 2024-04-12T21:55:34.055 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:34.052+0000 7f488aa00640 1 -- 172.21.15.18:0/4273166914 learned_addr learned my addr 172.21.15.18:0/4273166914 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:55:34.056 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:34.053+0000 7f488aa00640 1 -- 172.21.15.18:0/4273166914 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f488c104bc0 msgr2=0x7f488c115280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:55:34.056 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:34.053+0000 7f488aa00640 1 --2- 172.21.15.18:0/4273166914 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f488c104bc0 0x7f488c115280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:55:34.056 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:34.053+0000 7f488aa00640 1 -- 172.21.15.18:0/4273166914 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f487c005040 con 0x7f488c104220 2024-04-12T21:55:34.056 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:34.053+0000 7f488aa00640 1 --2- 172.21.15.18:0/4273166914 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f488c104220 0x7f488c114d40 secure :-1 s=READY pgs=1387 cs=0 l=1 rev1=1 crypto rx=0x7f487c004930 tx=0x7f487c009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:55:34.056 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:34.054+0000 7f4882a00640 1 -- 172.21.15.18:0/4273166914 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f487c0094c0 con 0x7f488c104220 2024-04-12T21:55:34.056 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:34.054+0000 7f488be00640 1 -- 172.21.15.18:0/4273166914 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f488c10f150 con 0x7f488c104220 2024-04-12T21:55:34.056 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:34.054+0000 7f488be00640 1 -- 172.21.15.18:0/4273166914 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f488c10f610 con 0x7f488c104220 2024-04-12T21:55:34.056 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:34.054+0000 7f4882a00640 1 -- 172.21.15.18:0/4273166914 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f487c032e50 con 0x7f488c104220 2024-04-12T21:55:34.056 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:34.054+0000 7f4882a00640 1 -- 172.21.15.18:0/4273166914 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f487c0433b0 con 0x7f488c104220 2024-04-12T21:55:34.057 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:34.055+0000 7f488be00640 1 -- 172.21.15.18:0/4273166914 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f486c005210 con 0x7f488c104220 2024-04-12T21:55:34.058 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:34.056+0000 7f4882a00640 1 -- 172.21.15.18:0/4273166914 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7f487c02f070 con 0x7f488c104220 2024-04-12T21:55:34.059 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:34.057+0000 7f4882a00640 1 --2- 172.21.15.18:0/4273166914 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f487006eb70 0x7f4870071020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:55:34.059 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:34.057+0000 7f4882a00640 1 -- 172.21.15.18:0/4273166914 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 6757+0+0 (secure 0 0 0) 0x7f487c0b06f0 con 0x7f488c104220 2024-04-12T21:55:34.059 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:34.057+0000 7f488a000640 1 --2- 172.21.15.18:0/4273166914 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f487006eb70 0x7f4870071020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:55:34.060 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:34.058+0000 7f488a000640 1 --2- 172.21.15.18:0/4273166914 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f487006eb70 0x7f4870071020 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7f488c110600 tx=0x7f4878009c50 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:55:34.064 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:34.062+0000 7f4882a00640 1 -- 172.21.15.18:0/4273166914 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f487c0792f0 con 0x7f488c104220 2024-04-12T21:55:34.271 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:34.269+0000 7f488be00640 1 -- 172.21.15.18:0/4273166914 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f486c0054c0 con 0x7f488c104220 2024-04-12T21:55:34.274 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:34.272+0000 7f4882a00640 1 -- 172.21.15.18:0/4273166914 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v104) v1 ==== 74+0+17095 (secure 0 0 0) 0x7f487c07c9b0 con 0x7f488c104220 2024-04-12T21:55:34.275 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-12T21:55:34.275 INFO:teuthology.orchestra.run.smithi018.stdout:{"epoch":104,"fsid":"1cbe407a-f90d-11ee-bc8d-c7b262605968","created":"2024-04-12T20:43:00.692361+0000","modified":"2024-04-12T21:38:07.461428+0000","last_up_change":"2024-04-12T21:06:52.454846+0000","last_in_change":"2024-04-12T20:49:54.190644+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":20,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-12T20:48:37.529081+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":6,"score_stable":6,"optimal_score":0.5,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs.cephfs.meta","create_time":"2024-04-12T20:51:56.284414+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"44","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.309999942779541,"score_stable":1.309999942779541,"optimal_score":1,"raw_score_acting":1.309999942779541,"raw_score_stable":1.309999942779541,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs.cephfs.data","create_time":"2024-04-12T20:51:56.927802+0000","flags":131073,"flags_names":"hashpspool,bulk","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"45","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2af887cc-0447-42aa-a8f3-1aa7fcd4dc8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":9,"last_clean_end":49,"up_from":64,"up_thru":97,"down_at":50,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6802","nonce":2573875940},{"type":"v1","addr":"172.21.15.18:6803","nonce":2573875940}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6804","nonce":2573875940},{"type":"v1","addr":"172.21.15.18:6805","nonce":2573875940}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6808","nonce":2573875940},{"type":"v1","addr":"172.21.15.18:6809","nonce":2573875940}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6806","nonce":2573875940},{"type":"v1","addr":"172.21.15.18:6807","nonce":2573875940}]},"public_addr":"172.21.15.18:6803/2573875940","cluster_addr":"172.21.15.18:6805/2573875940","heartbeat_back_addr":"172.21.15.18:6809/2573875940","heartbeat_front_addr":"172.21.15.18:6807/2573875940","state":["exists","up"]},{"osd":1,"uuid":"6ba7b0d3-a03d-45cc-b4e9-89a92c8c6ca8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":14,"last_clean_end":73,"up_from":77,"up_thru":97,"down_at":74,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6810","nonce":1955801845},{"type":"v1","addr":"172.21.15.18:6811","nonce":1955801845}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6812","nonce":1955801845},{"type":"v1","addr":"172.21.15.18:6813","nonce":1955801845}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6816","nonce":1955801845},{"type":"v1","addr":"172.21.15.18:6817","nonce":1955801845}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6814","nonce":1955801845},{"type":"v1","addr":"172.21.15.18:6815","nonce":1955801845}]},"public_addr":"172.21.15.18:6811/1955801845","cluster_addr":"172.21.15.18:6813/1955801845","heartbeat_back_addr":"172.21.15.18:6817/1955801845","heartbeat_front_addr":"172.21.15.18:6815/1955801845","state":["exists","up"]},{"osd":2,"uuid":"5bb1137d-5a3c-40bc-ae08-fc792b8ee547","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":19,"last_clean_end":78,"up_from":82,"up_thru":97,"down_at":79,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6818","nonce":1361831750},{"type":"v1","addr":"172.21.15.18:6819","nonce":1361831750}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6820","nonce":1361831750},{"type":"v1","addr":"172.21.15.18:6821","nonce":1361831750}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6824","nonce":1361831750},{"type":"v1","addr":"172.21.15.18:6825","nonce":1361831750}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6822","nonce":1361831750},{"type":"v1","addr":"172.21.15.18:6823","nonce":1361831750}]},"public_addr":"172.21.15.18:6819/1361831750","cluster_addr":"172.21.15.18:6821/1361831750","heartbeat_back_addr":"172.21.15.18:6825/1361831750","heartbeat_front_addr":"172.21.15.18:6823/1361831750","state":["exists","up"]},{"osd":3,"uuid":"a5140a33-1e90-4980-aa9a-b0afe6b155f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":26,"last_clean_end":83,"up_from":87,"up_thru":97,"down_at":84,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6800","nonce":1952273925},{"type":"v1","addr":"172.21.15.186:6801","nonce":1952273925}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6802","nonce":1952273925},{"type":"v1","addr":"172.21.15.186:6803","nonce":1952273925}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6806","nonce":1952273925},{"type":"v1","addr":"172.21.15.186:6807","nonce":1952273925}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6804","nonce":1952273925},{"type":"v1","addr":"172.21.15.186:6805","nonce":1952273925}]},"public_addr":"172.21.15.186:6801/1952273925","cluster_addr":"172.21.15.186:6803/1952273925","heartbeat_back_addr":"172.21.15.186:6807/1952273925","heartbeat_front_addr":"172.21.15.186:6805/1952273925","state":["exists","up"]},{"osd":4,"uuid":"d6b2c6b3-6c78-4261-bcbe-c33358407198","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":34,"last_clean_end":88,"up_from":92,"up_thru":97,"down_at":89,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6808","nonce":2123883537},{"type":"v1","addr":"172.21.15.186:6809","nonce":2123883537}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6810","nonce":2123883537},{"type":"v1","addr":"172.21.15.186:6811","nonce":2123883537}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6814","nonce":2123883537},{"type":"v1","addr":"172.21.15.186:6815","nonce":2123883537}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6812","nonce":2123883537},{"type":"v1","addr":"172.21.15.186:6813","nonce":2123883537}]},"public_addr":"172.21.15.186:6809/2123883537","cluster_addr":"172.21.15.186:6811/2123883537","heartbeat_back_addr":"172.21.15.186:6815/2123883537","heartbeat_front_addr":"172.21.15.186:6813/2123883537","state":["exists","up"]},{"osd":5,"uuid":"7b5c9f17-6306-4fb5-aa77-fc9a73bcc663","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":39,"last_clean_end":93,"up_from":97,"up_thru":97,"down_at":94,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6816","nonce":2849575493},{"type":"v1","addr":"172.21.15.186:6817","nonce":2849575493}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6818","nonce":2849575493},{"type":"v1","addr":"172.21.15.186:6819","nonce":2849575493}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6822","nonce":2849575493},{"type":"v1","addr":"172.21.15.186:6823","nonce":2849575493}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6820","nonce":2849575493},{"type":"v1","addr":"172.21.15.186:6821","nonce":2849575493}]},"public_addr":"172.21.15.186:6817/2849575493","cluster_addr":"172.21.15.186:6819/2849575493","heartbeat_back_addr":"172.21.15.186:6823/2849575493","heartbeat_front_addr":"172.21.15.186:6821/2849575493","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-12T20:47:21.231060+0000","dead_epoch":49},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-12T20:47:57.699205+0000","dead_epoch":73},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-12T20:48:35.096648+0000","dead_epoch":78},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-12T20:49:08.252810+0000","dead_epoch":83},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-12T20:49:41.102187+0000","dead_epoch":88},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-12T20:50:16.009813+0000","dead_epoch":93}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.8","mappings":[{"from":3,"to":2}]},{"pgid":"2.a","mappings":[{"from":3,"to":0}]},{"pgid":"2.1d","mappings":[{"from":3,"to":2}]},{"pgid":"3.6","mappings":[{"from":1,"to":5}]},{"pgid":"3.a","mappings":[{"from":4,"to":0}]},{"pgid":"3.e","mappings":[{"from":1,"to":2}]},{"pgid":"3.11","mappings":[{"from":3,"to":2}]},{"pgid":"3.14","mappings":[{"from":3,"to":0}]},{"pgid":"3.1a","mappings":[{"from":4,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.18:6826/1564065434":"2024-04-13T21:06:59.717116+0000","172.21.15.186:0/825752072":"2024-04-13T20:57:12.501044+0000","172.21.15.18:6800/1391635591":"2024-04-13T20:45:44.776072+0000","172.21.15.18:0/4018777476":"2024-04-13T20:45:44.776072+0000","172.21.15.186:6824/1645616336":"2024-04-13T20:52:14.968672+0000","172.21.15.18:0/3387911038":"2024-04-13T20:44:15.143139+0000","172.21.15.18:0/714775767":"2024-04-13T20:54:59.478287+0000","172.21.15.18:6801/3048894687":"2024-04-13T20:54:59.478287+0000","172.21.15.18:0/2171204556":"2024-04-13T21:01:39.724018+0000","172.21.15.186:0/2147813742":"2024-04-13T20:57:12.501044+0000","172.21.15.18:6829/722443353":"2024-04-13T21:06:59.717165+0000","172.21.15.18:0/2945432483":"2024-04-13T20:44:15.143139+0000","172.21.15.18:6827/1564065434":"2024-04-13T21:06:59.717116+0000","172.21.15.186:6829/2316349902":"2024-04-13T20:57:12.501044+0000","172.21.15.18:0/2956836690":"2024-04-13T20:54:59.478287+0000","172.21.15.18:0/1732042827":"2024-04-13T20:43:34.269559+0000","172.21.15.18:6800/3403971131":"2024-04-13T20:44:15.143139+0000","172.21.15.18:6800/1605950634":"2024-04-13T21:01:39.724018+0000","172.21.15.186:0/970502145":"2024-04-13T20:57:12.501044+0000","172.21.15.18:6800/3048894687":"2024-04-13T20:54:59.478287+0000","172.21.15.18:0/2330205732":"2024-04-13T20:54:59.478287+0000","172.21.15.18:0/3227425848":"2024-04-13T20:54:59.478287+0000","172.21.15.18:6801/1312802974":"2024-04-13T20:43:34.269559+0000","172.21.15.186:0/2459326506":"2024-04-13T20:57:12.501044+0000","172.21.15.18:0/3896210349":"2024-04-13T20:44:15.143139+0000","172.21.15.18:0/1720919549":"2024-04-13T20:45:44.776072+0000","172.21.15.18:0/766256564":"2024-04-13T20:45:44.776072+0000","172.21.15.18:6801/3403971131":"2024-04-13T20:44:15.143139+0000","172.21.15.18:6828/722443353":"2024-04-13T21:06:59.717165+0000","172.21.15.18:0/3295870133":"2024-04-13T20:43:34.269559+0000","172.21.15.18:6800/1312802974":"2024-04-13T20:43:34.269559+0000","172.21.15.186:6828/2316349902":"2024-04-13T20:57:12.501044+0000","172.21.15.186:6825/1645616336":"2024-04-13T20:52:14.968672+0000","172.21.15.18:0/3745234714":"2024-04-13T21:01:39.724018+0000","172.21.15.18:0/2356171563":"2024-04-13T21:01:39.724018+0000","172.21.15.18:6801/1391635591":"2024-04-13T20:45:44.776072+0000","172.21.15.18:0/2517252399":"2024-04-13T20:43:34.269559+0000","172.21.15.18:6801/1605950634":"2024-04-13T21:01:39.724018+0000","172.21.15.18:0/3861256640":"2024-04-13T21:01:39.724018+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-12T21:55:34.278 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:34.275+0000 7f488be00640 1 -- 172.21.15.18:0/4273166914 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f487006eb70 msgr2=0x7f4870071020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:55:34.278 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:34.275+0000 7f488be00640 1 --2- 172.21.15.18:0/4273166914 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f487006eb70 0x7f4870071020 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7f488c110600 tx=0x7f4878009c50 comp rx=0 tx=0).stop 2024-04-12T21:55:34.278 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:34.275+0000 7f488be00640 1 -- 172.21.15.18:0/4273166914 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f488c104220 msgr2=0x7f488c114d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:55:34.278 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:34.275+0000 7f488be00640 1 --2- 172.21.15.18:0/4273166914 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f488c104220 0x7f488c114d40 secure :-1 s=READY pgs=1387 cs=0 l=1 rev1=1 crypto rx=0x7f487c004930 tx=0x7f487c009da0 comp rx=0 tx=0).stop 2024-04-12T21:55:34.278 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:34.275+0000 7f488be00640 1 -- 172.21.15.18:0/4273166914 shutdown_connections 2024-04-12T21:55:34.278 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:34.275+0000 7f488be00640 1 --2- 172.21.15.18:0/4273166914 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f487006eb70 0x7f4870071020 unknown :-1 s=CLOSED pgs=735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:55:34.278 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:34.275+0000 7f488be00640 1 --2- 172.21.15.18:0/4273166914 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f488c104bc0 0x7f488c115280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:55:34.278 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:34.275+0000 7f488be00640 1 --2- 172.21.15.18:0/4273166914 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f488c104220 0x7f488c114d40 unknown :-1 s=CLOSED pgs=1387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:55:34.278 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:34.275+0000 7f488be00640 1 -- 172.21.15.18:0/4273166914 >> 172.21.15.18:0/4273166914 conn(0x7f488c0f9fc0 msgr2=0x7f488c0fbf10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:55:34.278 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:34.276+0000 7f488be00640 1 -- 172.21.15.18:0/4273166914 shutdown_connections 2024-04-12T21:55:34.278 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:34.276+0000 7f488be00640 1 -- 172.21.15.18:0/4273166914 wait complete. 2024-04-12T21:55:34.296 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-12T20:48:37.529081+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 6, 'score_stable': 6, 'optimal_score': 0.5, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs.cephfs.meta', 'create_time': '2024-04-12T20:51:56.284414+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '44', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.309999942779541, 'score_stable': 1.309999942779541, 'optimal_score': 1, 'raw_score_acting': 1.309999942779541, 'raw_score_stable': 1.309999942779541, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs.cephfs.data', 'create_time': '2024-04-12T20:51:56.927802+0000', 'flags': 131073, 'flags_names': 'hashpspool,bulk', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '45', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-12T21:55:34.296 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-12T21:55:34.518 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:34.515+0000 7fb967e00640 1 -- 172.21.15.18:0/4031296469 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb968104bf0 msgr2=0x7fb9680fe280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:55:34.518 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:34.516+0000 7fb967e00640 1 --2- 172.21.15.18:0/4031296469 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb968104bf0 0x7fb9680fe280 secure :-1 s=READY pgs=1388 cs=0 l=1 rev1=1 crypto rx=0x7fb9500048d0 tx=0x7fb950009da0 comp rx=0 tx=0).stop 2024-04-12T21:55:34.518 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:34.516+0000 7fb967e00640 1 -- 172.21.15.18:0/4031296469 shutdown_connections 2024-04-12T21:55:34.519 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:34.516+0000 7fb967e00640 1 --2- 172.21.15.18:0/4031296469 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb968104bf0 0x7fb9680fe280 unknown :-1 s=CLOSED pgs=1388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:55:34.519 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:34.516+0000 7fb967e00640 1 --2- 172.21.15.18:0/4031296469 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb968104250 0x7fb968104620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:55:34.519 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:34.516+0000 7fb967e00640 1 -- 172.21.15.18:0/4031296469 >> 172.21.15.18:0/4031296469 conn(0x7fb9680f9fc0 msgr2=0x7fb9680fc3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:55:34.519 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:34.516+0000 7fb967e00640 1 -- 172.21.15.18:0/4031296469 shutdown_connections 2024-04-12T21:55:34.519 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:34.516+0000 7fb967e00640 1 -- 172.21.15.18:0/4031296469 wait complete. 2024-04-12T21:55:34.520 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:34.517+0000 7fb967e00640 1 Processor -- start 2024-04-12T21:55:34.520 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:34.517+0000 7fb967e00640 1 -- start start 2024-04-12T21:55:34.520 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:34.517+0000 7fb967e00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb968104250 0x7fb968114da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:55:34.520 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:34.517+0000 7fb967e00640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb968104bf0 0x7fb9681152e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:55:34.520 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:34.517+0000 7fb967e00640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb968115970 con 0x7fb968104250 2024-04-12T21:55:34.520 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:34.517+0000 7fb967e00640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb96810ef30 con 0x7fb968104bf0 2024-04-12T21:55:34.520 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:34.518+0000 7fb966a00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb968104250 0x7fb968114da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:55:34.520 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:34.518+0000 7fb966a00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb968104250 0x7fb968114da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:50948/0 (socket says 172.21.15.18:50948) 2024-04-12T21:55:34.521 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:34.518+0000 7fb966a00640 1 -- 172.21.15.18:0/547996850 learned_addr learned my addr 172.21.15.18:0/547996850 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:55:34.521 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:34.518+0000 7fb95fe00640 1 --2- 172.21.15.18:0/547996850 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb968104bf0 0x7fb9681152e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:55:34.521 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:34.518+0000 7fb966a00640 1 -- 172.21.15.18:0/547996850 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb968104bf0 msgr2=0x7fb9681152e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:55:34.521 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:34.518+0000 7fb966a00640 1 --2- 172.21.15.18:0/547996850 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb968104bf0 0x7fb9681152e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:55:34.521 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:34.518+0000 7fb966a00640 1 -- 172.21.15.18:0/547996850 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb950005040 con 0x7fb968104250 2024-04-12T21:55:34.522 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:34.518+0000 7fb966a00640 1 --2- 172.21.15.18:0/547996850 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb968104250 0x7fb968114da0 secure :-1 s=READY pgs=1389 cs=0 l=1 rev1=1 crypto rx=0x7fb95800b790 tx=0x7fb9580091d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:55:34.522 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:34.519+0000 7fb95ea00640 1 -- 172.21.15.18:0/547996850 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb95800c700 con 0x7fb968104250 2024-04-12T21:55:34.522 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:34.519+0000 7fb95ea00640 1 -- 172.21.15.18:0/547996850 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb95800cd30 con 0x7fb968104250 2024-04-12T21:55:34.522 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:34.519+0000 7fb95ea00640 1 -- 172.21.15.18:0/547996850 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb958013370 con 0x7fb968104250 2024-04-12T21:55:34.522 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:34.519+0000 7fb967e00640 1 -- 172.21.15.18:0/547996850 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb96810f210 con 0x7fb968104250 2024-04-12T21:55:34.522 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:34.519+0000 7fb967e00640 1 -- 172.21.15.18:0/547996850 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb96810f730 con 0x7fb968104250 2024-04-12T21:55:34.522 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:34.520+0000 7fb967e00640 1 -- 172.21.15.18:0/547996850 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb9680ff9c0 con 0x7fb968104250 2024-04-12T21:55:34.523 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:34.521+0000 7fb95ea00640 1 -- 172.21.15.18:0/547996850 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7fb958012070 con 0x7fb968104250 2024-04-12T21:55:34.524 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:34.522+0000 7fb95ea00640 1 --2- 172.21.15.18:0/547996850 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fb94c06eb80 0x7fb94c071030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:55:34.524 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:34.522+0000 7fb95ea00640 1 -- 172.21.15.18:0/547996850 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 6757+0+0 (secure 0 0 0) 0x7fb95808e5b0 con 0x7fb968104250 2024-04-12T21:55:34.525 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:34.522+0000 7fb95fe00640 1 --2- 172.21.15.18:0/547996850 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fb94c06eb80 0x7fb94c071030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:55:34.525 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:34.523+0000 7fb95fe00640 1 --2- 172.21.15.18:0/547996850 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fb94c06eb80 0x7fb94c071030 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7fb950000c00 tx=0x7fb950009da0 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:55:34.529 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:34.527+0000 7fb95ea00640 1 -- 172.21.15.18:0/547996850 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7fb958057230 con 0x7fb968104250 2024-04-12T21:55:34.534 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:55:34 smithi186 ceph-mon[86379]: pgmap v1679: 65 pgs: 65 active+clean; 257 MiB data, 976 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:55:34.606 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:55:34 smithi018 ceph-mon[112590]: pgmap v1679: 65 pgs: 65 active+clean; 257 MiB data, 976 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:55:34.737 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:34.735+0000 7fb967e00640 1 -- 172.21.15.18:0/547996850 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fb96810b410 con 0x7fb968104250 2024-04-12T21:55:34.738 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:34.736+0000 7fb95ea00640 1 -- 172.21.15.18:0/547996850 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v104) v1 ==== 93+0+10 (secure 0 0 0) 0x7fb95805a8f0 con 0x7fb968104250 2024-04-12T21:55:34.738 INFO:teuthology.orchestra.run.smithi018.stdout:pg_num: 1 2024-04-12T21:55:34.741 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:34.739+0000 7fb967e00640 1 -- 172.21.15.18:0/547996850 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fb94c06eb80 msgr2=0x7fb94c071030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:55:34.741 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:34.739+0000 7fb967e00640 1 --2- 172.21.15.18:0/547996850 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fb94c06eb80 0x7fb94c071030 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7fb950000c00 tx=0x7fb950009da0 comp rx=0 tx=0).stop 2024-04-12T21:55:34.741 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:34.739+0000 7fb967e00640 1 -- 172.21.15.18:0/547996850 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb968104250 msgr2=0x7fb968114da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:55:34.741 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:34.739+0000 7fb967e00640 1 --2- 172.21.15.18:0/547996850 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb968104250 0x7fb968114da0 secure :-1 s=READY pgs=1389 cs=0 l=1 rev1=1 crypto rx=0x7fb95800b790 tx=0x7fb9580091d0 comp rx=0 tx=0).stop 2024-04-12T21:55:34.741 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:34.739+0000 7fb967e00640 1 -- 172.21.15.18:0/547996850 shutdown_connections 2024-04-12T21:55:34.742 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:34.739+0000 7fb967e00640 1 --2- 172.21.15.18:0/547996850 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fb94c06eb80 0x7fb94c071030 unknown :-1 s=CLOSED pgs=736 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:55:34.742 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:34.739+0000 7fb967e00640 1 --2- 172.21.15.18:0/547996850 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb968104bf0 0x7fb9681152e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:55:34.742 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:34.739+0000 7fb967e00640 1 --2- 172.21.15.18:0/547996850 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb968104250 0x7fb968114da0 unknown :-1 s=CLOSED pgs=1389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:55:34.742 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:34.739+0000 7fb967e00640 1 -- 172.21.15.18:0/547996850 >> 172.21.15.18:0/547996850 conn(0x7fb9680f9fc0 msgr2=0x7fb9680fbed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:55:34.742 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:34.739+0000 7fb967e00640 1 -- 172.21.15.18:0/547996850 shutdown_connections 2024-04-12T21:55:34.742 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:34.739+0000 7fb967e00640 1 -- 172.21.15.18:0/547996850 wait complete. 2024-04-12T21:55:34.760 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs.cephfs.meta pg_num 2024-04-12T21:55:34.984 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:34.982+0000 7fb532200640 1 -- 172.21.15.18:0/291381402 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb52c102d80 msgr2=0x7fb52c0fc410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:55:34.985 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:34.982+0000 7fb532200640 1 --2- 172.21.15.18:0/291381402 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb52c102d80 0x7fb52c0fc410 secure :-1 s=READY pgs=1390 cs=0 l=1 rev1=1 crypto rx=0x7fb5140048d0 tx=0x7fb514009da0 comp rx=0 tx=0).stop 2024-04-12T21:55:34.985 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:34.982+0000 7fb532200640 1 -- 172.21.15.18:0/291381402 shutdown_connections 2024-04-12T21:55:34.985 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:34.982+0000 7fb532200640 1 --2- 172.21.15.18:0/291381402 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb52c102d80 0x7fb52c0fc410 unknown :-1 s=CLOSED pgs=1390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:55:34.985 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:34.982+0000 7fb532200640 1 --2- 172.21.15.18:0/291381402 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb52c1023e0 0x7fb52c1027b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:55:34.985 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:34.982+0000 7fb532200640 1 -- 172.21.15.18:0/291381402 >> 172.21.15.18:0/291381402 conn(0x7fb52c0f7fb0 msgr2=0x7fb52c0fa3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:55:34.985 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:34.982+0000 7fb532200640 1 -- 172.21.15.18:0/291381402 shutdown_connections 2024-04-12T21:55:34.985 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:34.982+0000 7fb532200640 1 -- 172.21.15.18:0/291381402 wait complete. 2024-04-12T21:55:34.986 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:34.983+0000 7fb532200640 1 Processor -- start 2024-04-12T21:55:34.986 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:34.983+0000 7fb532200640 1 -- start start 2024-04-12T21:55:34.986 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:34.984+0000 7fb532200640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb52c1023e0 0x7fb52c115170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:55:34.986 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:34.984+0000 7fb532200640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb52c102d80 0x7fb52c1156b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:55:34.986 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:34.984+0000 7fb532200640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb52c115cb0 con 0x7fb52c102d80 2024-04-12T21:55:34.986 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:34.984+0000 7fb532200640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb52c10f300 con 0x7fb52c1023e0 2024-04-12T21:55:34.986 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:34.984+0000 7fb522a00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb52c102d80 0x7fb52c1156b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:55:34.986 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:34.984+0000 7fb522a00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb52c102d80 0x7fb52c1156b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:50964/0 (socket says 172.21.15.18:50964) 2024-04-12T21:55:34.986 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:34.984+0000 7fb522a00640 1 -- 172.21.15.18:0/680712186 learned_addr learned my addr 172.21.15.18:0/680712186 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:55:34.987 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:34.984+0000 7fb522a00640 1 -- 172.21.15.18:0/680712186 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb52c1023e0 msgr2=0x7fb52c115170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:55:34.987 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:34.984+0000 7fb522a00640 1 --2- 172.21.15.18:0/680712186 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb52c1023e0 0x7fb52c115170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:55:34.988 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:34.984+0000 7fb522a00640 1 -- 172.21.15.18:0/680712186 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb514005040 con 0x7fb52c102d80 2024-04-12T21:55:34.988 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:34.985+0000 7fb522a00640 1 --2- 172.21.15.18:0/680712186 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb52c102d80 0x7fb52c1156b0 secure :-1 s=READY pgs=1391 cs=0 l=1 rev1=1 crypto rx=0x7fb5140047f0 tx=0x7fb514009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:55:34.988 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:34.985+0000 7fb523e00640 1 -- 172.21.15.18:0/680712186 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb5140094c0 con 0x7fb52c102d80 2024-04-12T21:55:34.988 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:34.985+0000 7fb523e00640 1 -- 172.21.15.18:0/680712186 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb514032e50 con 0x7fb52c102d80 2024-04-12T21:55:34.988 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:34.985+0000 7fb532200640 1 -- 172.21.15.18:0/680712186 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb52c10f580 con 0x7fb52c102d80 2024-04-12T21:55:34.988 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:34.985+0000 7fb523e00640 1 -- 172.21.15.18:0/680712186 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb5140433b0 con 0x7fb52c102d80 2024-04-12T21:55:34.988 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:34.985+0000 7fb532200640 1 -- 172.21.15.18:0/680712186 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb52c10fa40 con 0x7fb52c102d80 2024-04-12T21:55:34.988 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:34.986+0000 7fb532200640 1 -- 172.21.15.18:0/680712186 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb52c0fdb50 con 0x7fb52c102d80 2024-04-12T21:55:34.989 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:34.987+0000 7fb523e00640 1 -- 172.21.15.18:0/680712186 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7fb51402f070 con 0x7fb52c102d80 2024-04-12T21:55:34.990 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:34.988+0000 7fb523e00640 1 --2- 172.21.15.18:0/680712186 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fb51006eb80 0x7fb510071030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:55:34.990 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:34.988+0000 7fb523e00640 1 -- 172.21.15.18:0/680712186 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 6757+0+0 (secure 0 0 0) 0x7fb5140b0890 con 0x7fb52c102d80 2024-04-12T21:55:34.991 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:34.989+0000 7fb52b400640 1 --2- 172.21.15.18:0/680712186 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fb51006eb80 0x7fb510071030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:55:34.991 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:34.989+0000 7fb52b400640 1 --2- 172.21.15.18:0/680712186 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fb51006eb80 0x7fb510071030 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7fb51c004c80 tx=0x7fb51c009c50 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:55:34.995 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:34.993+0000 7fb523e00640 1 -- 172.21.15.18:0/680712186 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7fb514079490 con 0x7fb52c102d80 2024-04-12T21:55:35.204 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:35.201+0000 7fb532200640 1 -- 172.21.15.18:0/680712186 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"} v 0) v1 -- 0x7fb52c109540 con 0x7fb52c102d80 2024-04-12T21:55:35.205 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:35.203+0000 7fb523e00640 1 -- 172.21.15.18:0/680712186 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"}]=0 v104) v1 ==== 107+0+11 (secure 0 0 0) 0x7fb51407cb50 con 0x7fb52c102d80 2024-04-12T21:55:35.205 INFO:teuthology.orchestra.run.smithi018.stdout:pg_num: 32 2024-04-12T21:55:35.208 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:35.206+0000 7fb532200640 1 -- 172.21.15.18:0/680712186 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fb51006eb80 msgr2=0x7fb510071030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:55:35.208 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:35.206+0000 7fb532200640 1 --2- 172.21.15.18:0/680712186 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fb51006eb80 0x7fb510071030 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7fb51c004c80 tx=0x7fb51c009c50 comp rx=0 tx=0).stop 2024-04-12T21:55:35.208 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:35.206+0000 7fb532200640 1 -- 172.21.15.18:0/680712186 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb52c102d80 msgr2=0x7fb52c1156b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:55:35.208 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:35.206+0000 7fb532200640 1 --2- 172.21.15.18:0/680712186 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb52c102d80 0x7fb52c1156b0 secure :-1 s=READY pgs=1391 cs=0 l=1 rev1=1 crypto rx=0x7fb5140047f0 tx=0x7fb514009da0 comp rx=0 tx=0).stop 2024-04-12T21:55:35.208 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:35.206+0000 7fb532200640 1 -- 172.21.15.18:0/680712186 shutdown_connections 2024-04-12T21:55:35.208 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:35.206+0000 7fb532200640 1 --2- 172.21.15.18:0/680712186 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fb51006eb80 0x7fb510071030 unknown :-1 s=CLOSED pgs=737 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:55:35.209 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:35.206+0000 7fb532200640 1 --2- 172.21.15.18:0/680712186 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb52c102d80 0x7fb52c1156b0 unknown :-1 s=CLOSED pgs=1391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:55:35.209 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:35.206+0000 7fb532200640 1 --2- 172.21.15.18:0/680712186 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb52c1023e0 0x7fb52c115170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:55:35.209 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:35.206+0000 7fb532200640 1 -- 172.21.15.18:0/680712186 >> 172.21.15.18:0/680712186 conn(0x7fb52c0f7fb0 msgr2=0x7fb52c0711f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:55:35.209 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:35.206+0000 7fb532200640 1 -- 172.21.15.18:0/680712186 shutdown_connections 2024-04-12T21:55:35.209 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:35.206+0000 7fb532200640 1 -- 172.21.15.18:0/680712186 wait complete. 2024-04-12T21:55:35.227 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs.cephfs.data pg_num 2024-04-12T21:55:35.448 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:35.445+0000 7f3efaa00640 1 -- 172.21.15.18:0/2905291728 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3ef41029b0 msgr2=0x7f3ef40fc040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:55:35.448 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:35.445+0000 7f3efaa00640 1 --2- 172.21.15.18:0/2905291728 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3ef41029b0 0x7f3ef40fc040 secure :-1 s=READY pgs=1392 cs=0 l=1 rev1=1 crypto rx=0x7f3ee00048d0 tx=0x7f3ee0009da0 comp rx=0 tx=0).stop 2024-04-12T21:55:35.448 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:35.445+0000 7f3efaa00640 1 -- 172.21.15.18:0/2905291728 shutdown_connections 2024-04-12T21:55:35.448 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:35.445+0000 7f3efaa00640 1 --2- 172.21.15.18:0/2905291728 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3ef41029b0 0x7f3ef40fc040 unknown :-1 s=CLOSED pgs=1392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:55:35.449 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:35.445+0000 7f3efaa00640 1 --2- 172.21.15.18:0/2905291728 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3ef4102010 0x7f3ef41023e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:55:35.449 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:35.445+0000 7f3efaa00640 1 -- 172.21.15.18:0/2905291728 >> 172.21.15.18:0/2905291728 conn(0x7f3ef40f7c20 msgr2=0x7f3ef40fa030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:55:35.449 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:35.446+0000 7f3efaa00640 1 -- 172.21.15.18:0/2905291728 shutdown_connections 2024-04-12T21:55:35.449 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:35.446+0000 7f3efaa00640 1 -- 172.21.15.18:0/2905291728 wait complete. 2024-04-12T21:55:35.451 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:35.446+0000 7f3efaa00640 1 Processor -- start 2024-04-12T21:55:35.451 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:35.447+0000 7f3efaa00640 1 -- start start 2024-04-12T21:55:35.451 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:35.447+0000 7f3efaa00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3ef4102010 0x7f3ef4114ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:55:35.451 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:35.447+0000 7f3efaa00640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3ef41029b0 0x7f3ef4115220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:55:35.451 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:35.447+0000 7f3efaa00640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ef4115820 con 0x7f3ef4102010 2024-04-12T21:55:35.451 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:35.447+0000 7f3efaa00640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ef4115990 con 0x7f3ef41029b0 2024-04-12T21:55:35.451 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:35.447+0000 7f3ef9600640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3ef4102010 0x7f3ef4114ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:55:35.451 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:35.447+0000 7f3ef9600640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3ef4102010 0x7f3ef4114ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:50980/0 (socket says 172.21.15.18:50980) 2024-04-12T21:55:35.451 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:35.447+0000 7f3ef9600640 1 -- 172.21.15.18:0/4116227053 learned_addr learned my addr 172.21.15.18:0/4116227053 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:55:35.451 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:35.448+0000 7f3ef8c00640 1 --2- 172.21.15.18:0/4116227053 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3ef41029b0 0x7f3ef4115220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:55:35.451 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:35.448+0000 7f3ef9600640 1 -- 172.21.15.18:0/4116227053 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3ef41029b0 msgr2=0x7f3ef4115220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:55:35.451 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:35.448+0000 7f3ef9600640 1 --2- 172.21.15.18:0/4116227053 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3ef41029b0 0x7f3ef4115220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:55:35.451 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:35.448+0000 7f3ef9600640 1 -- 172.21.15.18:0/4116227053 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ee0005040 con 0x7f3ef4102010 2024-04-12T21:55:35.452 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:35.448+0000 7f3ef8c00640 1 --2- 172.21.15.18:0/4116227053 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3ef41029b0 0x7f3ef4115220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-12T21:55:35.452 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:35.448+0000 7f3ef9600640 1 --2- 172.21.15.18:0/4116227053 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3ef4102010 0x7f3ef4114ce0 secure :-1 s=READY pgs=1393 cs=0 l=1 rev1=1 crypto rx=0x7f3ee8002a50 tx=0x7f3ee8004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:55:35.452 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:35.448+0000 7f3ef1600640 1 -- 172.21.15.18:0/4116227053 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3ee8007c30 con 0x7f3ef4102010 2024-04-12T21:55:35.452 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:35.448+0000 7f3ef1600640 1 -- 172.21.15.18:0/4116227053 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f3ee800f040 con 0x7f3ef4102010 2024-04-12T21:55:35.452 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:35.448+0000 7f3efaa00640 1 -- 172.21.15.18:0/4116227053 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3ef410f020 con 0x7f3ef4102010 2024-04-12T21:55:35.452 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:35.448+0000 7f3ef1600640 1 -- 172.21.15.18:0/4116227053 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3ee800db10 con 0x7f3ef4102010 2024-04-12T21:55:35.452 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:35.449+0000 7f3efaa00640 1 -- 172.21.15.18:0/4116227053 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3ef410f540 con 0x7f3ef4102010 2024-04-12T21:55:35.453 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:35.450+0000 7f3efaa00640 1 -- 172.21.15.18:0/4116227053 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3ef40fd780 con 0x7f3ef4102010 2024-04-12T21:55:35.453 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:35.451+0000 7f3ef1600640 1 -- 172.21.15.18:0/4116227053 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7f3ee800dc90 con 0x7f3ef4102010 2024-04-12T21:55:35.454 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:35.451+0000 7f3ef1600640 1 --2- 172.21.15.18:0/4116227053 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f3edc06eb80 0x7f3edc071030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:55:35.454 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:35.452+0000 7f3ef1600640 1 -- 172.21.15.18:0/4116227053 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 6757+0+0 (secure 0 0 0) 0x7f3ee808e260 con 0x7f3ef4102010 2024-04-12T21:55:35.454 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:35.452+0000 7f3ef8c00640 1 --2- 172.21.15.18:0/4116227053 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f3edc06eb80 0x7f3edc071030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:55:35.455 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:35.453+0000 7f3ef8c00640 1 --2- 172.21.15.18:0/4116227053 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f3edc06eb80 0x7f3edc071030 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7f3ee0000c00 tx=0x7f3ee0009da0 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:55:35.459 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:35.457+0000 7f3ef1600640 1 -- 172.21.15.18:0/4116227053 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f3ee8056f00 con 0x7f3ef4102010 2024-04-12T21:55:35.535 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:55:35 smithi186 ceph-mon[86379]: from='client.? 172.21.15.18:0/4273166914' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-12T21:55:35.535 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:55:35 smithi186 ceph-mon[86379]: from='client.? 172.21.15.18:0/547996850' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-12T21:55:35.535 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:55:35 smithi186 ceph-mon[86379]: from='client.? 172.21.15.18:0/680712186' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"}]: dispatch 2024-04-12T21:55:35.606 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:55:35 smithi018 ceph-mon[112590]: from='client.? 172.21.15.18:0/4273166914' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-12T21:55:35.606 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:55:35 smithi018 ceph-mon[112590]: from='client.? 172.21.15.18:0/547996850' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-12T21:55:35.606 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:55:35 smithi018 ceph-mon[112590]: from='client.? 172.21.15.18:0/680712186' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"}]: dispatch 2024-04-12T21:55:35.667 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:35.665+0000 7f3efaa00640 1 -- 172.21.15.18:0/4116227053 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"} v 0) v1 -- 0x7f3ef404bcc0 con 0x7f3ef4102010 2024-04-12T21:55:35.668 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:35.666+0000 7f3ef1600640 1 -- 172.21.15.18:0/4116227053 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"}]=0 v104) v1 ==== 107+0+11 (secure 0 0 0) 0x7f3ee805a5c0 con 0x7f3ef4102010 2024-04-12T21:55:35.668 INFO:teuthology.orchestra.run.smithi018.stdout:pg_num: 32 2024-04-12T21:55:35.671 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:35.668+0000 7f3efaa00640 1 -- 172.21.15.18:0/4116227053 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f3edc06eb80 msgr2=0x7f3edc071030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:55:35.671 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:35.668+0000 7f3efaa00640 1 --2- 172.21.15.18:0/4116227053 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f3edc06eb80 0x7f3edc071030 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7f3ee0000c00 tx=0x7f3ee0009da0 comp rx=0 tx=0).stop 2024-04-12T21:55:35.671 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:35.669+0000 7f3efaa00640 1 -- 172.21.15.18:0/4116227053 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3ef4102010 msgr2=0x7f3ef4114ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:55:35.671 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:35.669+0000 7f3efaa00640 1 --2- 172.21.15.18:0/4116227053 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3ef4102010 0x7f3ef4114ce0 secure :-1 s=READY pgs=1393 cs=0 l=1 rev1=1 crypto rx=0x7f3ee8002a50 tx=0x7f3ee8004600 comp rx=0 tx=0).stop 2024-04-12T21:55:35.671 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:35.669+0000 7f3efaa00640 1 -- 172.21.15.18:0/4116227053 shutdown_connections 2024-04-12T21:55:35.672 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:35.669+0000 7f3efaa00640 1 --2- 172.21.15.18:0/4116227053 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f3edc06eb80 0x7f3edc071030 unknown :-1 s=CLOSED pgs=738 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:55:35.672 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:35.669+0000 7f3efaa00640 1 --2- 172.21.15.18:0/4116227053 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3ef41029b0 0x7f3ef4115220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:55:35.672 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:35.669+0000 7f3efaa00640 1 --2- 172.21.15.18:0/4116227053 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3ef4102010 0x7f3ef4114ce0 unknown :-1 s=CLOSED pgs=1393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:55:35.672 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:35.669+0000 7f3efaa00640 1 -- 172.21.15.18:0/4116227053 >> 172.21.15.18:0/4116227053 conn(0x7f3ef40f7c20 msgr2=0x7f3ef4071040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:55:35.672 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:35.669+0000 7f3efaa00640 1 -- 172.21.15.18:0/4116227053 shutdown_connections 2024-04-12T21:55:35.672 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:35.669+0000 7f3efaa00640 1 -- 172.21.15.18:0/4116227053 wait complete. 2024-04-12T21:55:35.692 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-12T21:55:35.905 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:35.902+0000 7efc9fa00640 1 -- 172.21.15.18:0/39451077 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7efc98071a00 msgr2=0x7efc98071dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:55:35.905 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:35.902+0000 7efc9fa00640 1 --2- 172.21.15.18:0/39451077 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7efc98071a00 0x7efc98071dd0 secure :-1 s=READY pgs=1394 cs=0 l=1 rev1=1 crypto rx=0x7efc8c0048d0 tx=0x7efc8c009da0 comp rx=0 tx=0).stop 2024-04-12T21:55:35.905 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:35.902+0000 7efc9fa00640 1 -- 172.21.15.18:0/39451077 shutdown_connections 2024-04-12T21:55:35.905 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:35.902+0000 7efc9fa00640 1 --2- 172.21.15.18:0/39451077 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7efc98072310 0x7efc9810cb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:55:35.905 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:35.902+0000 7efc9fa00640 1 --2- 172.21.15.18:0/39451077 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7efc98071a00 0x7efc98071dd0 unknown :-1 s=CLOSED pgs=1394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:55:35.905 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:35.902+0000 7efc9fa00640 1 -- 172.21.15.18:0/39451077 >> 172.21.15.18:0/39451077 conn(0x7efc980f9fc0 msgr2=0x7efc980fc3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:55:35.905 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:35.902+0000 7efc9fa00640 1 -- 172.21.15.18:0/39451077 shutdown_connections 2024-04-12T21:55:35.905 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:35.902+0000 7efc9fa00640 1 -- 172.21.15.18:0/39451077 wait complete. 2024-04-12T21:55:35.908 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:35.903+0000 7efc9fa00640 1 Processor -- start 2024-04-12T21:55:35.908 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:35.903+0000 7efc9fa00640 1 -- start start 2024-04-12T21:55:35.908 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:35.904+0000 7efc9fa00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7efc98071a00 0x7efc981192d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:55:35.908 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:35.904+0000 7efc9fa00640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7efc98072310 0x7efc98119810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:55:35.908 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:35.904+0000 7efc9fa00640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc98119e10 con 0x7efc98071a00 2024-04-12T21:55:35.908 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:35.904+0000 7efc9fa00640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc98113460 con 0x7efc98072310 2024-04-12T21:55:35.908 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:35.904+0000 7efc9d000640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7efc98071a00 0x7efc981192d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:55:35.908 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:35.904+0000 7efc9d000640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7efc98071a00 0x7efc981192d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:50992/0 (socket says 172.21.15.18:50992) 2024-04-12T21:55:35.908 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:35.904+0000 7efc9d000640 1 -- 172.21.15.18:0/2382893698 learned_addr learned my addr 172.21.15.18:0/2382893698 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:55:35.908 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:35.904+0000 7efc97e00640 1 --2- 172.21.15.18:0/2382893698 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7efc98072310 0x7efc98119810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:55:35.909 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:35.904+0000 7efc9d000640 1 -- 172.21.15.18:0/2382893698 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7efc98072310 msgr2=0x7efc98119810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:55:35.909 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:35.904+0000 7efc9d000640 1 --2- 172.21.15.18:0/2382893698 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7efc98072310 0x7efc98119810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:55:35.909 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:35.904+0000 7efc9d000640 1 -- 172.21.15.18:0/2382893698 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc8c005040 con 0x7efc98071a00 2024-04-12T21:55:35.909 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:35.905+0000 7efc97e00640 1 --2- 172.21.15.18:0/2382893698 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7efc98072310 0x7efc98119810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-12T21:55:35.909 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:35.905+0000 7efc9d000640 1 --2- 172.21.15.18:0/2382893698 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7efc98071a00 0x7efc981192d0 secure :-1 s=READY pgs=1395 cs=0 l=1 rev1=1 crypto rx=0x7efc8c0047f0 tx=0x7efc8c009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:55:35.909 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:35.905+0000 7efc94c00640 1 -- 172.21.15.18:0/2382893698 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7efc8c0094c0 con 0x7efc98071a00 2024-04-12T21:55:35.909 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:35.905+0000 7efc94c00640 1 -- 172.21.15.18:0/2382893698 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7efc8c032e50 con 0x7efc98071a00 2024-04-12T21:55:35.909 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:35.905+0000 7efc94c00640 1 -- 172.21.15.18:0/2382893698 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7efc8c0435f0 con 0x7efc98071a00 2024-04-12T21:55:35.909 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:35.905+0000 7efc9fa00640 1 -- 172.21.15.18:0/2382893698 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efc981136e0 con 0x7efc98071a00 2024-04-12T21:55:35.909 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:35.905+0000 7efc9fa00640 1 -- 172.21.15.18:0/2382893698 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efc98113ba0 con 0x7efc98071a00 2024-04-12T21:55:35.910 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:35.907+0000 7efc9fa00640 1 -- 172.21.15.18:0/2382893698 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efc98073140 con 0x7efc98071a00 2024-04-12T21:55:35.910 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:35.908+0000 7efc94c00640 1 -- 172.21.15.18:0/2382893698 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7efc8c02f070 con 0x7efc98071a00 2024-04-12T21:55:35.912 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:35.909+0000 7efc94c00640 1 --2- 172.21.15.18:0/2382893698 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7efc8006eb70 0x7efc80071020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:55:35.912 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:35.909+0000 7efc94c00640 1 -- 172.21.15.18:0/2382893698 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 6757+0+0 (secure 0 0 0) 0x7efc8c0b0890 con 0x7efc98071a00 2024-04-12T21:55:35.912 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:35.910+0000 7efc97e00640 1 --2- 172.21.15.18:0/2382893698 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7efc8006eb70 0x7efc80071020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:55:35.912 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:35.910+0000 7efc97e00640 1 --2- 172.21.15.18:0/2382893698 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7efc8006eb70 0x7efc80071020 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7efc98114b90 tx=0x7efc84009040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:55:35.917 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:35.915+0000 7efc94c00640 1 -- 172.21.15.18:0/2382893698 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7efc8c079490 con 0x7efc98071a00 2024-04-12T21:55:36.162 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:36.160+0000 7efc9fa00640 1 -- 172.21.15.18:0/2382893698 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7efc9810f910 con 0x7efc98071a00 2024-04-12T21:55:36.163 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:36.161+0000 7efc94c00640 1 -- 172.21.15.18:0/2382893698 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 38 v38) v1 ==== 94+0+5236 (secure 0 0 0) 0x7efc8c07cb50 con 0x7efc98071a00 2024-04-12T21:55:36.163 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-12T21:55:36.164 INFO:teuthology.orchestra.run.smithi018.stdout:{"epoch":38,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":44595,"name":"cephfs.smithi186.jdllyn","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6825/3523444202","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6824","nonce":3523444202},{"type":"v1","addr":"172.21.15.186:6825","nonce":3523444202}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":31},{"gid":44875,"name":"cephfs.smithi186.wjnafs","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6827/3122443766","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6826","nonce":3122443766},{"type":"v1","addr":"172.21.15.186:6827","nonce":3122443766}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":33}],"filesystems":[{"mdsmap":{"epoch":38,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-12T20:51:57.362850+0000","modified":"2024-04-12T21:53:30.672912+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":100,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":2,"in":[0,1],"up":{"mds_0":34436,"mds_1":34468},"failed":[],"damaged":[],"stopped":[],"info":{"gid_34436":{"gid":34436,"name":"cephfs.smithi018.acqbsc","rank":0,"incarnation":34,"state":"up:active","state_seq":699,"addr":"172.21.15.18:6829/755179764","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6828","nonce":755179764},{"type":"v1","addr":"172.21.15.18:6829","nonce":755179764}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34468":{"gid":34468,"name":"cephfs.smithi018.szjluo","rank":1,"incarnation":34,"state":"up:active","state_seq":692,"addr":"172.21.15.18:6827/2407992653","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6826","nonce":2407992653},{"type":"v1","addr":"172.21.15.18:6827","nonce":2407992653}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":34436,"qdb_cluster":[34436,34468]},"id":1}]} 2024-04-12T21:55:36.164 INFO:teuthology.orchestra.run.smithi018.stderr:dumped fsmap epoch 38 2024-04-12T21:55:36.166 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:36.164+0000 7efc9fa00640 1 -- 172.21.15.18:0/2382893698 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7efc8006eb70 msgr2=0x7efc80071020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:55:36.166 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:36.164+0000 7efc9fa00640 1 --2- 172.21.15.18:0/2382893698 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7efc8006eb70 0x7efc80071020 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7efc98114b90 tx=0x7efc84009040 comp rx=0 tx=0).stop 2024-04-12T21:55:36.166 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:36.164+0000 7efc9fa00640 1 -- 172.21.15.18:0/2382893698 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7efc98071a00 msgr2=0x7efc981192d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:55:36.166 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:36.164+0000 7efc9fa00640 1 --2- 172.21.15.18:0/2382893698 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7efc98071a00 0x7efc981192d0 secure :-1 s=READY pgs=1395 cs=0 l=1 rev1=1 crypto rx=0x7efc8c0047f0 tx=0x7efc8c009da0 comp rx=0 tx=0).stop 2024-04-12T21:55:36.167 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:36.164+0000 7efc9fa00640 1 -- 172.21.15.18:0/2382893698 shutdown_connections 2024-04-12T21:55:36.167 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:36.164+0000 7efc9fa00640 1 --2- 172.21.15.18:0/2382893698 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7efc8006eb70 0x7efc80071020 unknown :-1 s=CLOSED pgs=739 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:55:36.167 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:36.164+0000 7efc9fa00640 1 --2- 172.21.15.18:0/2382893698 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7efc98072310 0x7efc98119810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:55:36.167 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:36.164+0000 7efc9fa00640 1 --2- 172.21.15.18:0/2382893698 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7efc98071a00 0x7efc981192d0 unknown :-1 s=CLOSED pgs=1395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:55:36.167 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:36.164+0000 7efc9fa00640 1 -- 172.21.15.18:0/2382893698 >> 172.21.15.18:0/2382893698 conn(0x7efc980f9fc0 msgr2=0x7efc98074550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:55:36.167 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:36.164+0000 7efc9fa00640 1 -- 172.21.15.18:0/2382893698 shutdown_connections 2024-04-12T21:55:36.167 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:36.164+0000 7efc9fa00640 1 -- 172.21.15.18:0/2382893698 wait complete. 2024-04-12T21:55:36.185 DEBUG:tasks.fs:checking fs fscid=1,name=cephfs state = {'epoch': 19, 'max_mds': 2, 'flags': 18} 2024-04-12T21:55:36.185 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 20 2024-04-12T21:55:36.405 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:36.403+0000 7fb1d2400640 1 -- 172.21.15.18:0/1492205691 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb1cc108f90 msgr2=0x7fb1cc102650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:55:36.406 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:36.403+0000 7fb1d2400640 1 --2- 172.21.15.18:0/1492205691 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb1cc108f90 0x7fb1cc102650 secure :-1 s=READY pgs=1396 cs=0 l=1 rev1=1 crypto rx=0x7fb1b80048d0 tx=0x7fb1b8009da0 comp rx=0 tx=0).stop 2024-04-12T21:55:36.406 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:36.403+0000 7fb1d2400640 1 -- 172.21.15.18:0/1492205691 shutdown_connections 2024-04-12T21:55:36.406 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:36.403+0000 7fb1d2400640 1 --2- 172.21.15.18:0/1492205691 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb1cc108f90 0x7fb1cc102650 unknown :-1 s=CLOSED pgs=1396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:55:36.406 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:36.403+0000 7fb1d2400640 1 --2- 172.21.15.18:0/1492205691 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb1cc1085f0 0x7fb1cc1089c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:55:36.406 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:36.403+0000 7fb1d2400640 1 -- 172.21.15.18:0/1492205691 >> 172.21.15.18:0/1492205691 conn(0x7fb1cc071e30 msgr2=0x7fb1cc074240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:55:36.406 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:36.403+0000 7fb1d2400640 1 -- 172.21.15.18:0/1492205691 shutdown_connections 2024-04-12T21:55:36.406 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:36.403+0000 7fb1d2400640 1 -- 172.21.15.18:0/1492205691 wait complete. 2024-04-12T21:55:36.408 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:36.404+0000 7fb1d2400640 1 Processor -- start 2024-04-12T21:55:36.408 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:36.404+0000 7fb1d2400640 1 -- start start 2024-04-12T21:55:36.408 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:36.405+0000 7fb1d2400640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb1cc1085f0 0x7fb1cc1191b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:55:36.408 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:36.405+0000 7fb1d2400640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb1cc108f90 0x7fb1cc1196f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:55:36.408 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:36.405+0000 7fb1d2400640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1cc119d80 con 0x7fb1cc108f90 2024-04-12T21:55:36.408 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:36.405+0000 7fb1d2400640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1cc113340 con 0x7fb1cc1085f0 2024-04-12T21:55:36.408 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:36.405+0000 7fb1cbe00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb1cc108f90 0x7fb1cc1196f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:55:36.408 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:36.405+0000 7fb1cbe00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb1cc108f90 0x7fb1cc1196f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:51016/0 (socket says 172.21.15.18:51016) 2024-04-12T21:55:36.408 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:36.405+0000 7fb1cbe00640 1 -- 172.21.15.18:0/3045360631 learned_addr learned my addr 172.21.15.18:0/3045360631 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:55:36.408 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:36.406+0000 7fb1cbe00640 1 -- 172.21.15.18:0/3045360631 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb1cc1085f0 msgr2=0x7fb1cc1191b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:55:36.409 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:36.406+0000 7fb1d1000640 1 --2- 172.21.15.18:0/3045360631 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb1cc1085f0 0x7fb1cc1191b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:55:36.409 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:36.406+0000 7fb1cbe00640 1 --2- 172.21.15.18:0/3045360631 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb1cc1085f0 0x7fb1cc1191b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:55:36.409 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:36.406+0000 7fb1cbe00640 1 -- 172.21.15.18:0/3045360631 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1b8005040 con 0x7fb1cc108f90 2024-04-12T21:55:36.409 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:36.406+0000 7fb1d1000640 1 --2- 172.21.15.18:0/3045360631 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb1cc1085f0 0x7fb1cc1191b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-12T21:55:36.410 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:36.406+0000 7fb1cbe00640 1 --2- 172.21.15.18:0/3045360631 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb1cc108f90 0x7fb1cc1196f0 secure :-1 s=READY pgs=1397 cs=0 l=1 rev1=1 crypto rx=0x7fb1b802be70 tx=0x7fb1b8009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:55:36.410 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:36.406+0000 7fb1c8c00640 1 -- 172.21.15.18:0/3045360631 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb1b80094c0 con 0x7fb1cc108f90 2024-04-12T21:55:36.410 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:36.406+0000 7fb1d2400640 1 -- 172.21.15.18:0/3045360631 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb1cc1135c0 con 0x7fb1cc108f90 2024-04-12T21:55:36.410 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:36.406+0000 7fb1c8c00640 1 -- 172.21.15.18:0/3045360631 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb1b8009620 con 0x7fb1cc108f90 2024-04-12T21:55:36.410 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:36.406+0000 7fb1c8c00640 1 -- 172.21.15.18:0/3045360631 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb1b80435f0 con 0x7fb1cc108f90 2024-04-12T21:55:36.410 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:36.407+0000 7fb1d2400640 1 -- 172.21.15.18:0/3045360631 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb1cc113a80 con 0x7fb1cc108f90 2024-04-12T21:55:36.410 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:36.408+0000 7fb1d2400640 1 -- 172.21.15.18:0/3045360631 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb1cc103d70 con 0x7fb1cc108f90 2024-04-12T21:55:36.411 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:36.409+0000 7fb1c8c00640 1 -- 172.21.15.18:0/3045360631 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7fb1b802f070 con 0x7fb1cc108f90 2024-04-12T21:55:36.412 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:36.410+0000 7fb1c8c00640 1 --2- 172.21.15.18:0/3045360631 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fb1b006eb70 0x7fb1b0071020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:55:36.412 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:36.410+0000 7fb1c8c00640 1 -- 172.21.15.18:0/3045360631 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 6757+0+0 (secure 0 0 0) 0x7fb1b80b08b0 con 0x7fb1cc108f90 2024-04-12T21:55:36.412 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:36.410+0000 7fb1d1000640 1 --2- 172.21.15.18:0/3045360631 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fb1b006eb70 0x7fb1b0071020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:55:36.413 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:36.411+0000 7fb1d1000640 1 --2- 172.21.15.18:0/3045360631 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fb1b006eb70 0x7fb1b0071020 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7fb1bc006fd0 tx=0x7fb1bc008040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:55:36.417 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:36.415+0000 7fb1c8c00640 1 -- 172.21.15.18:0/3045360631 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7fb1b80794b0 con 0x7fb1cc108f90 2024-04-12T21:55:36.535 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:55:36 smithi186 ceph-mon[86379]: from='client.? 172.21.15.18:0/4116227053' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"}]: dispatch 2024-04-12T21:55:36.535 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:55:36 smithi186 ceph-mon[86379]: pgmap v1680: 65 pgs: 65 active+clean; 257 MiB data, 976 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:55:36.535 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:55:36 smithi186 ceph-mon[86379]: from='client.? 172.21.15.18:0/2382893698' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-12T21:55:36.606 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:55:36 smithi018 ceph-mon[112590]: from='client.? 172.21.15.18:0/4116227053' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"}]: dispatch 2024-04-12T21:55:36.606 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:55:36 smithi018 ceph-mon[112590]: pgmap v1680: 65 pgs: 65 active+clean; 257 MiB data, 976 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:55:36.606 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:55:36 smithi018 ceph-mon[112590]: from='client.? 172.21.15.18:0/2382893698' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-12T21:55:36.670 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:36.667+0000 7fb1d2400640 1 -- 172.21.15.18:0/3045360631 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 20, "format": "json"} v 0) v1 -- 0x7fb1cc04bcc0 con 0x7fb1cc108f90 2024-04-12T21:55:36.671 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:36.668+0000 7fb1c8c00640 1 -- 172.21.15.18:0/3045360631 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 20, "format": "json"}]=0 dumped fsmap epoch 20 v38) v1 ==== 107+0+4873 (secure 0 0 0) 0x7fb1b807cb70 con 0x7fb1cc108f90 2024-04-12T21:55:36.671 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-12T21:55:36.671 INFO:teuthology.orchestra.run.smithi018.stdout:{"epoch":20,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24277,"name":"cephfs.smithi186.wjnafs","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.186:6827/438347207","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6826","nonce":438347207},{"type":"v1","addr":"172.21.15.186:6827","nonce":438347207}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":18},{"gid":24281,"name":"cephfs.smithi186.jdllyn","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6825/1464359952","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6824","nonce":1464359952},{"type":"v1","addr":"172.21.15.186:6825","nonce":1464359952}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":16}],"filesystems":[{"mdsmap":{"epoch":20,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-12T20:51:57.362850+0000","modified":"2024-04-12T20:54:48.315111+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":46,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":14452,"mds_1":14464},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14452":{"gid":14452,"name":"cephfs.smithi018.acqbsc","rank":0,"incarnation":4,"state":"up:active","state_seq":5,"addr":"172.21.15.18:6827/1564065434","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6826","nonce":1564065434},{"type":"v1","addr":"172.21.15.18:6827","nonce":1564065434}]},"join_fscid":1,"export_targets":[1],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_14464":{"gid":14464,"name":"cephfs.smithi018.szjluo","rank":1,"incarnation":15,"state":"up:active","state_seq":6,"addr":"172.21.15.18:6829/722443353","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6828","nonce":722443353},{"type":"v1","addr":"172.21.15.18:6829","nonce":722443353}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-04-12T21:55:36.671 INFO:teuthology.orchestra.run.smithi018.stderr:dumped fsmap epoch 20 2024-04-12T21:55:36.674 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:36.672+0000 7fb1d2400640 1 -- 172.21.15.18:0/3045360631 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fb1b006eb70 msgr2=0x7fb1b0071020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:55:36.674 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:36.672+0000 7fb1d2400640 1 --2- 172.21.15.18:0/3045360631 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fb1b006eb70 0x7fb1b0071020 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7fb1bc006fd0 tx=0x7fb1bc008040 comp rx=0 tx=0).stop 2024-04-12T21:55:36.674 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:36.672+0000 7fb1d2400640 1 -- 172.21.15.18:0/3045360631 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb1cc108f90 msgr2=0x7fb1cc1196f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:55:36.675 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:36.672+0000 7fb1d2400640 1 --2- 172.21.15.18:0/3045360631 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb1cc108f90 0x7fb1cc1196f0 secure :-1 s=READY pgs=1397 cs=0 l=1 rev1=1 crypto rx=0x7fb1b802be70 tx=0x7fb1b8009da0 comp rx=0 tx=0).stop 2024-04-12T21:55:36.675 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:36.672+0000 7fb1d2400640 1 -- 172.21.15.18:0/3045360631 shutdown_connections 2024-04-12T21:55:36.675 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:36.672+0000 7fb1d2400640 1 --2- 172.21.15.18:0/3045360631 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fb1b006eb70 0x7fb1b0071020 unknown :-1 s=CLOSED pgs=740 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:55:36.675 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:36.672+0000 7fb1d2400640 1 --2- 172.21.15.18:0/3045360631 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb1cc108f90 0x7fb1cc1196f0 unknown :-1 s=CLOSED pgs=1397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:55:36.675 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:36.672+0000 7fb1d2400640 1 --2- 172.21.15.18:0/3045360631 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb1cc1085f0 0x7fb1cc1191b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:55:36.675 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:36.672+0000 7fb1d2400640 1 -- 172.21.15.18:0/3045360631 >> 172.21.15.18:0/3045360631 conn(0x7fb1cc071e30 msgr2=0x7fb1cc073d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:55:36.675 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:36.672+0000 7fb1d2400640 1 -- 172.21.15.18:0/3045360631 shutdown_connections 2024-04-12T21:55:36.675 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:36.672+0000 7fb1d2400640 1 -- 172.21.15.18:0/3045360631 wait complete. 2024-04-12T21:55:36.695 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 21 2024-04-12T21:55:36.921 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:36.919+0000 7f3fb8c00640 1 -- 172.21.15.18:0/120566911 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3fb4071a00 msgr2=0x7f3fb4071dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:55:36.921 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:36.919+0000 7f3fb8c00640 1 --2- 172.21.15.18:0/120566911 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3fb4071a00 0x7f3fb4071dd0 secure :-1 s=READY pgs=1398 cs=0 l=1 rev1=1 crypto rx=0x7f3fa4009c50 tx=0x7f3fa4004d10 comp rx=0 tx=0).stop 2024-04-12T21:55:36.921 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:36.919+0000 7f3fb8c00640 1 -- 172.21.15.18:0/120566911 shutdown_connections 2024-04-12T21:55:36.921 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:36.919+0000 7f3fb8c00640 1 --2- 172.21.15.18:0/120566911 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3fb4072310 0x7f3fb410cb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:55:36.922 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:36.919+0000 7f3fb8c00640 1 --2- 172.21.15.18:0/120566911 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3fb4071a00 0x7f3fb4071dd0 unknown :-1 s=CLOSED pgs=1398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:55:36.922 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:36.919+0000 7f3fb8c00640 1 -- 172.21.15.18:0/120566911 >> 172.21.15.18:0/120566911 conn(0x7f3fb40f9ff0 msgr2=0x7f3fb40fc400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:55:36.922 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:36.919+0000 7f3fb8c00640 1 -- 172.21.15.18:0/120566911 shutdown_connections 2024-04-12T21:55:36.922 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:36.919+0000 7f3fb8c00640 1 -- 172.21.15.18:0/120566911 wait complete. 2024-04-12T21:55:36.923 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:36.920+0000 7f3fb8c00640 1 Processor -- start 2024-04-12T21:55:36.923 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:36.920+0000 7f3fb8c00640 1 -- start start 2024-04-12T21:55:36.923 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:36.920+0000 7f3fb8c00640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3fb4071a00 0x7f3fb41192e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:55:36.924 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:36.920+0000 7f3fb8c00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3fb4072310 0x7f3fb4119820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:55:36.924 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:36.920+0000 7f3fb8c00640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fb4119e20 con 0x7f3fb4072310 2024-04-12T21:55:36.924 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:36.920+0000 7f3fb8c00640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fb4113470 con 0x7f3fb4071a00 2024-04-12T21:55:36.924 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:36.921+0000 7f3fa9400640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3fb4072310 0x7f3fb4119820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:55:36.924 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:36.921+0000 7f3fa9400640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3fb4072310 0x7f3fb4119820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:51022/0 (socket says 172.21.15.18:51022) 2024-04-12T21:55:36.924 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:36.921+0000 7f3fa9400640 1 -- 172.21.15.18:0/163555954 learned_addr learned my addr 172.21.15.18:0/163555954 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:55:36.924 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:36.921+0000 7f3fb1e00640 1 --2- 172.21.15.18:0/163555954 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3fb4071a00 0x7f3fb41192e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:55:36.924 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:36.921+0000 7f3fa9400640 1 -- 172.21.15.18:0/163555954 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3fb4071a00 msgr2=0x7f3fb41192e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:55:36.924 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:36.921+0000 7f3fa9400640 1 --2- 172.21.15.18:0/163555954 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3fb4071a00 0x7f3fb41192e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:55:36.924 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:36.921+0000 7f3fa9400640 1 -- 172.21.15.18:0/163555954 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3fa40098b0 con 0x7f3fb4072310 2024-04-12T21:55:36.924 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:36.921+0000 7f3fb1e00640 1 --2- 172.21.15.18:0/163555954 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3fb4071a00 0x7f3fb41192e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-12T21:55:36.924 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:36.921+0000 7f3fa9400640 1 --2- 172.21.15.18:0/163555954 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3fb4072310 0x7f3fb4119820 secure :-1 s=READY pgs=1399 cs=0 l=1 rev1=1 crypto rx=0x7f3f9c002a50 tx=0x7f3f9c004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:55:36.925 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:36.922+0000 7f3faaa00640 1 -- 172.21.15.18:0/163555954 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3f9c007c30 con 0x7f3fb4072310 2024-04-12T21:55:36.925 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:36.922+0000 7f3faaa00640 1 -- 172.21.15.18:0/163555954 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f3f9c00f040 con 0x7f3fb4072310 2024-04-12T21:55:36.925 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:36.922+0000 7f3faaa00640 1 -- 172.21.15.18:0/163555954 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3f9c00db50 con 0x7f3fb4072310 2024-04-12T21:55:36.925 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:36.922+0000 7f3fb8c00640 1 -- 172.21.15.18:0/163555954 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3fb4113750 con 0x7f3fb4072310 2024-04-12T21:55:36.925 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:36.922+0000 7f3fb8c00640 1 -- 172.21.15.18:0/163555954 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3fb4113cf0 con 0x7f3fb4072310 2024-04-12T21:55:36.926 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:36.923+0000 7f3fb8c00640 1 -- 172.21.15.18:0/163555954 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3fb4073140 con 0x7f3fb4072310 2024-04-12T21:55:36.926 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:36.924+0000 7f3faaa00640 1 -- 172.21.15.18:0/163555954 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7f3f9c00dcf0 con 0x7f3fb4072310 2024-04-12T21:55:36.927 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:36.925+0000 7f3faaa00640 1 --2- 172.21.15.18:0/163555954 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f3f9806eb70 0x7f3f98071020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:55:36.927 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:36.925+0000 7f3faaa00640 1 -- 172.21.15.18:0/163555954 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 6757+0+0 (secure 0 0 0) 0x7f3f9c08e2c0 con 0x7f3fb4072310 2024-04-12T21:55:36.928 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:36.925+0000 7f3fb1e00640 1 --2- 172.21.15.18:0/163555954 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f3f9806eb70 0x7f3f98071020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:55:36.928 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:36.926+0000 7f3fb1e00640 1 --2- 172.21.15.18:0/163555954 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f3f9806eb70 0x7f3f98071020 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7f3fa4004a30 tx=0x7f3fa402e040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:55:36.932 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:36.930+0000 7f3faaa00640 1 -- 172.21.15.18:0/163555954 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f3f9c056f40 con 0x7f3fb4072310 2024-04-12T21:55:37.182 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:37.180+0000 7f3fb8c00640 1 -- 172.21.15.18:0/163555954 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 21, "format": "json"} v 0) v1 -- 0x7f3fb410f950 con 0x7f3fb4072310 2024-04-12T21:55:37.183 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:37.181+0000 7f3faaa00640 1 -- 172.21.15.18:0/163555954 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 21, "format": "json"}]=0 dumped fsmap epoch 21 v38) v1 ==== 107+0+4874 (secure 0 0 0) 0x7f3f9c05a600 con 0x7f3fb4072310 2024-04-12T21:55:37.184 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-12T21:55:37.184 INFO:teuthology.orchestra.run.smithi018.stdout:{"epoch":21,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24277,"name":"cephfs.smithi186.wjnafs","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.186:6827/438347207","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6826","nonce":438347207},{"type":"v1","addr":"172.21.15.186:6827","nonce":438347207}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":18},{"gid":24281,"name":"cephfs.smithi186.jdllyn","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6825/1464359952","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6824","nonce":1464359952},{"type":"v1","addr":"172.21.15.186:6825","nonce":1464359952}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":16}],"filesystems":[{"mdsmap":{"epoch":21,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-12T20:51:57.362850+0000","modified":"2024-04-12T20:56:40.253566+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":46,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":14452,"mds_1":14464},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14452":{"gid":14452,"name":"cephfs.smithi018.acqbsc","rank":0,"incarnation":4,"state":"up:active","state_seq":5,"addr":"172.21.15.18:6827/1564065434","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6826","nonce":1564065434},{"type":"v1","addr":"172.21.15.18:6827","nonce":1564065434}]},"join_fscid":1,"export_targets":[1],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_14464":{"gid":14464,"name":"cephfs.smithi018.szjluo","rank":1,"incarnation":15,"state":"up:active","state_seq":6,"addr":"172.21.15.18:6829/722443353","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6828","nonce":722443353},{"type":"v1","addr":"172.21.15.18:6829","nonce":722443353}]},"join_fscid":1,"export_targets":[0],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-04-12T21:55:37.184 INFO:teuthology.orchestra.run.smithi018.stderr:dumped fsmap epoch 21 2024-04-12T21:55:37.186 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:37.184+0000 7f3fb8c00640 1 -- 172.21.15.18:0/163555954 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f3f9806eb70 msgr2=0x7f3f98071020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:55:37.187 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:37.184+0000 7f3fb8c00640 1 --2- 172.21.15.18:0/163555954 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f3f9806eb70 0x7f3f98071020 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7f3fa4004a30 tx=0x7f3fa402e040 comp rx=0 tx=0).stop 2024-04-12T21:55:37.187 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:37.184+0000 7f3fb8c00640 1 -- 172.21.15.18:0/163555954 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3fb4072310 msgr2=0x7f3fb4119820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:55:37.187 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:37.184+0000 7f3fb8c00640 1 --2- 172.21.15.18:0/163555954 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3fb4072310 0x7f3fb4119820 secure :-1 s=READY pgs=1399 cs=0 l=1 rev1=1 crypto rx=0x7f3f9c002a50 tx=0x7f3f9c004600 comp rx=0 tx=0).stop 2024-04-12T21:55:37.187 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:37.184+0000 7f3fb8c00640 1 -- 172.21.15.18:0/163555954 shutdown_connections 2024-04-12T21:55:37.187 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:37.184+0000 7f3fb8c00640 1 --2- 172.21.15.18:0/163555954 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f3f9806eb70 0x7f3f98071020 unknown :-1 s=CLOSED pgs=741 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:55:37.187 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:37.184+0000 7f3fb8c00640 1 --2- 172.21.15.18:0/163555954 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3fb4072310 0x7f3fb4119820 unknown :-1 s=CLOSED pgs=1399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:55:37.187 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:37.184+0000 7f3fb8c00640 1 --2- 172.21.15.18:0/163555954 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3fb4071a00 0x7f3fb41192e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:55:37.187 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:37.184+0000 7f3fb8c00640 1 -- 172.21.15.18:0/163555954 >> 172.21.15.18:0/163555954 conn(0x7f3fb40f9ff0 msgr2=0x7f3fb40fac20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:55:37.188 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:37.185+0000 7f3fb8c00640 1 -- 172.21.15.18:0/163555954 shutdown_connections 2024-04-12T21:55:37.188 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:37.185+0000 7f3fb8c00640 1 -- 172.21.15.18:0/163555954 wait complete. 2024-04-12T21:55:37.205 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 22 2024-04-12T21:55:37.424 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:37.421+0000 7fd77fe00640 1 -- 172.21.15.18:0/2490249230 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd7801029a0 msgr2=0x7fd7800fc030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:55:37.424 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:37.421+0000 7fd77fe00640 1 --2- 172.21.15.18:0/2490249230 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd7801029a0 0x7fd7800fc030 secure :-1 s=READY pgs=1400 cs=0 l=1 rev1=1 crypto rx=0x7fd7680048d0 tx=0x7fd768009da0 comp rx=0 tx=0).stop 2024-04-12T21:55:37.424 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:37.421+0000 7fd77fe00640 1 -- 172.21.15.18:0/2490249230 shutdown_connections 2024-04-12T21:55:37.424 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:37.421+0000 7fd77fe00640 1 --2- 172.21.15.18:0/2490249230 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd7801029a0 0x7fd7800fc030 unknown :-1 s=CLOSED pgs=1400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:55:37.425 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:37.421+0000 7fd77fe00640 1 --2- 172.21.15.18:0/2490249230 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd780102000 0x7fd7801023d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:55:37.425 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:37.421+0000 7fd77fe00640 1 -- 172.21.15.18:0/2490249230 >> 172.21.15.18:0/2490249230 conn(0x7fd7800f7b70 msgr2=0x7fd7800f9f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:55:37.425 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:37.421+0000 7fd77fe00640 1 -- 172.21.15.18:0/2490249230 shutdown_connections 2024-04-12T21:55:37.425 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:37.421+0000 7fd77fe00640 1 -- 172.21.15.18:0/2490249230 wait complete. 2024-04-12T21:55:37.425 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:37.422+0000 7fd77fe00640 1 Processor -- start 2024-04-12T21:55:37.425 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:37.422+0000 7fd77fe00640 1 -- start start 2024-04-12T21:55:37.427 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:37.423+0000 7fd77fe00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd780102000 0x7fd780112ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:55:37.427 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:37.423+0000 7fd77fe00640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd7801029a0 0x7fd780113010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:55:37.427 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:37.423+0000 7fd77fe00640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd780113610 con 0x7fd780102000 2024-04-12T21:55:37.427 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:37.423+0000 7fd77fe00640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd78010cc60 con 0x7fd7801029a0 2024-04-12T21:55:37.427 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:37.423+0000 7fd77ea00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd780102000 0x7fd780112ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:55:37.427 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:37.423+0000 7fd77ea00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd780102000 0x7fd780112ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:51028/0 (socket says 172.21.15.18:51028) 2024-04-12T21:55:37.427 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:37.423+0000 7fd77ea00640 1 -- 172.21.15.18:0/2740490368 learned_addr learned my addr 172.21.15.18:0/2740490368 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:55:37.427 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:37.423+0000 7fd776000640 1 --2- 172.21.15.18:0/2740490368 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd7801029a0 0x7fd780113010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:55:37.427 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:37.423+0000 7fd77ea00640 1 -- 172.21.15.18:0/2740490368 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd7801029a0 msgr2=0x7fd780113010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:55:37.427 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:37.423+0000 7fd77ea00640 1 --2- 172.21.15.18:0/2740490368 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd7801029a0 0x7fd780113010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:55:37.427 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:37.423+0000 7fd77ea00640 1 -- 172.21.15.18:0/2740490368 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd768005040 con 0x7fd780102000 2024-04-12T21:55:37.427 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:37.424+0000 7fd77ea00640 1 --2- 172.21.15.18:0/2740490368 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd780102000 0x7fd780112ad0 secure :-1 s=READY pgs=1401 cs=0 l=1 rev1=1 crypto rx=0x7fd77000cae0 tx=0x7fd770007e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:55:37.427 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:37.424+0000 7fd777400640 1 -- 172.21.15.18:0/2740490368 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd77000ab00 con 0x7fd780102000 2024-04-12T21:55:37.428 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:37.424+0000 7fd777400640 1 -- 172.21.15.18:0/2740490368 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd77000ac60 con 0x7fd780102000 2024-04-12T21:55:37.428 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:37.424+0000 7fd777400640 1 -- 172.21.15.18:0/2740490368 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd7700116f0 con 0x7fd780102000 2024-04-12T21:55:37.428 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:37.424+0000 7fd77fe00640 1 -- 172.21.15.18:0/2740490368 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd78010cf40 con 0x7fd780102000 2024-04-12T21:55:37.428 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:37.424+0000 7fd77fe00640 1 -- 172.21.15.18:0/2740490368 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd78010d460 con 0x7fd780102000 2024-04-12T21:55:37.428 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:37.426+0000 7fd77fe00640 1 -- 172.21.15.18:0/2740490368 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd7800fd770 con 0x7fd780102000 2024-04-12T21:55:37.429 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:37.427+0000 7fd777400640 1 -- 172.21.15.18:0/2740490368 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7fd770012440 con 0x7fd780102000 2024-04-12T21:55:37.430 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:37.427+0000 7fd777400640 1 --2- 172.21.15.18:0/2740490368 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fd76406ebc0 0x7fd764071070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:55:37.430 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:37.428+0000 7fd777400640 1 -- 172.21.15.18:0/2740490368 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 6757+0+0 (secure 0 0 0) 0x7fd77008f360 con 0x7fd780102000 2024-04-12T21:55:37.430 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:37.428+0000 7fd776000640 1 --2- 172.21.15.18:0/2740490368 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fd76406ebc0 0x7fd764071070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:55:37.431 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:37.429+0000 7fd776000640 1 --2- 172.21.15.18:0/2740490368 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fd76406ebc0 0x7fd764071070 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7fd768000c00 tx=0x7fd768009da0 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:55:37.434 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:37.432+0000 7fd777400640 1 -- 172.21.15.18:0/2740490368 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7fd770057f60 con 0x7fd780102000 2024-04-12T21:55:37.535 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:55:37 smithi186 ceph-mon[86379]: from='client.? 172.21.15.18:0/3045360631' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 20, "format": "json"}]: dispatch 2024-04-12T21:55:37.535 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:55:37 smithi186 ceph-mon[86379]: from='client.? 172.21.15.18:0/163555954' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 21, "format": "json"}]: dispatch 2024-04-12T21:55:37.605 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:55:37 smithi018 ceph-mon[112590]: from='client.? 172.21.15.18:0/3045360631' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 20, "format": "json"}]: dispatch 2024-04-12T21:55:37.606 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:55:37 smithi018 ceph-mon[112590]: from='client.? 172.21.15.18:0/163555954' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 21, "format": "json"}]: dispatch 2024-04-12T21:55:37.683 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:37.681+0000 7fd77fe00640 1 -- 172.21.15.18:0/2740490368 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 22, "format": "json"} v 0) v1 -- 0x7fd78004bcc0 con 0x7fd780102000 2024-04-12T21:55:37.684 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:37.682+0000 7fd777400640 1 -- 172.21.15.18:0/2740490368 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 22, "format": "json"}]=0 dumped fsmap epoch 22 v38) v1 ==== 107+0+4873 (secure 0 0 0) 0x7fd77005b620 con 0x7fd780102000 2024-04-12T21:55:37.685 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-12T21:55:37.685 INFO:teuthology.orchestra.run.smithi018.stdout:{"epoch":22,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24277,"name":"cephfs.smithi186.wjnafs","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.186:6827/438347207","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6826","nonce":438347207},{"type":"v1","addr":"172.21.15.186:6827","nonce":438347207}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":18},{"gid":24281,"name":"cephfs.smithi186.jdllyn","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6825/1464359952","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6824","nonce":1464359952},{"type":"v1","addr":"172.21.15.186:6825","nonce":1464359952}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":16}],"filesystems":[{"mdsmap":{"epoch":22,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-12T20:51:57.362850+0000","modified":"2024-04-12T21:00:57.677022+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":46,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":14452,"mds_1":14464},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14452":{"gid":14452,"name":"cephfs.smithi018.acqbsc","rank":0,"incarnation":4,"state":"up:active","state_seq":5,"addr":"172.21.15.18:6827/1564065434","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6826","nonce":1564065434},{"type":"v1","addr":"172.21.15.18:6827","nonce":1564065434}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_14464":{"gid":14464,"name":"cephfs.smithi018.szjluo","rank":1,"incarnation":15,"state":"up:active","state_seq":6,"addr":"172.21.15.18:6829/722443353","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6828","nonce":722443353},{"type":"v1","addr":"172.21.15.18:6829","nonce":722443353}]},"join_fscid":1,"export_targets":[0],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-04-12T21:55:37.685 INFO:teuthology.orchestra.run.smithi018.stderr:dumped fsmap epoch 22 2024-04-12T21:55:37.687 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:37.685+0000 7fd77fe00640 1 -- 172.21.15.18:0/2740490368 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fd76406ebc0 msgr2=0x7fd764071070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:55:37.688 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:37.685+0000 7fd77fe00640 1 --2- 172.21.15.18:0/2740490368 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fd76406ebc0 0x7fd764071070 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7fd768000c00 tx=0x7fd768009da0 comp rx=0 tx=0).stop 2024-04-12T21:55:37.688 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:37.685+0000 7fd77fe00640 1 -- 172.21.15.18:0/2740490368 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd780102000 msgr2=0x7fd780112ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:55:37.688 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:37.685+0000 7fd77fe00640 1 --2- 172.21.15.18:0/2740490368 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd780102000 0x7fd780112ad0 secure :-1 s=READY pgs=1401 cs=0 l=1 rev1=1 crypto rx=0x7fd77000cae0 tx=0x7fd770007e50 comp rx=0 tx=0).stop 2024-04-12T21:55:37.688 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:37.685+0000 7fd77fe00640 1 -- 172.21.15.18:0/2740490368 shutdown_connections 2024-04-12T21:55:37.688 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:37.685+0000 7fd77fe00640 1 --2- 172.21.15.18:0/2740490368 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fd76406ebc0 0x7fd764071070 unknown :-1 s=CLOSED pgs=742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:55:37.688 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:37.685+0000 7fd77fe00640 1 --2- 172.21.15.18:0/2740490368 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd7801029a0 0x7fd780113010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:55:37.688 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:37.685+0000 7fd77fe00640 1 --2- 172.21.15.18:0/2740490368 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd780102000 0x7fd780112ad0 unknown :-1 s=CLOSED pgs=1401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:55:37.688 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:37.685+0000 7fd77fe00640 1 -- 172.21.15.18:0/2740490368 >> 172.21.15.18:0/2740490368 conn(0x7fd7800f7b70 msgr2=0x7fd7800f8ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:55:37.688 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:37.685+0000 7fd77fe00640 1 -- 172.21.15.18:0/2740490368 shutdown_connections 2024-04-12T21:55:37.688 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:37.685+0000 7fd77fe00640 1 -- 172.21.15.18:0/2740490368 wait complete. 2024-04-12T21:55:37.707 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 23 2024-04-12T21:55:37.933 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:37.930+0000 7f06be800640 1 -- 172.21.15.18:0/3807722478 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f06b8104d40 msgr2=0x7f06b80fe3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:55:37.933 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:37.930+0000 7f06be800640 1 --2- 172.21.15.18:0/3807722478 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f06b8104d40 0x7f06b80fe3d0 secure :-1 s=READY pgs=1402 cs=0 l=1 rev1=1 crypto rx=0x7f06a00048d0 tx=0x7f06a0009da0 comp rx=0 tx=0).stop 2024-04-12T21:55:37.933 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:37.930+0000 7f06be800640 1 -- 172.21.15.18:0/3807722478 shutdown_connections 2024-04-12T21:55:37.933 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:37.930+0000 7f06be800640 1 --2- 172.21.15.18:0/3807722478 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f06b8104d40 0x7f06b80fe3d0 unknown :-1 s=CLOSED pgs=1402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:55:37.933 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:37.930+0000 7f06be800640 1 --2- 172.21.15.18:0/3807722478 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f06b81043a0 0x7f06b8104770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:55:37.934 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:37.930+0000 7f06be800640 1 -- 172.21.15.18:0/3807722478 >> 172.21.15.18:0/3807722478 conn(0x7f06b80f9fc0 msgr2=0x7f06b80fc3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:55:37.934 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:37.930+0000 7f06be800640 1 -- 172.21.15.18:0/3807722478 shutdown_connections 2024-04-12T21:55:37.934 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:37.930+0000 7f06be800640 1 -- 172.21.15.18:0/3807722478 wait complete. 2024-04-12T21:55:37.936 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:37.931+0000 7f06be800640 1 Processor -- start 2024-04-12T21:55:37.936 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:37.931+0000 7f06be800640 1 -- start start 2024-04-12T21:55:37.936 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:37.932+0000 7f06be800640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f06b81043a0 0x7f06b8114f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:55:37.936 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:37.932+0000 7f06be800640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f06b8104d40 0x7f06b8115460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:55:37.936 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:37.932+0000 7f06be800640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06b8115af0 con 0x7f06b81043a0 2024-04-12T21:55:37.936 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:37.932+0000 7f06be800640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06b810f0b0 con 0x7f06b8104d40 2024-04-12T21:55:37.936 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:37.932+0000 7f06b7400640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f06b81043a0 0x7f06b8114f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:55:37.936 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:37.932+0000 7f06b7400640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f06b81043a0 0x7f06b8114f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:51056/0 (socket says 172.21.15.18:51056) 2024-04-12T21:55:37.936 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:37.932+0000 7f06b7400640 1 -- 172.21.15.18:0/3102621533 learned_addr learned my addr 172.21.15.18:0/3102621533 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:55:37.936 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:37.932+0000 7f06b6a00640 1 --2- 172.21.15.18:0/3102621533 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f06b8104d40 0x7f06b8115460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:55:37.936 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:37.932+0000 7f06b7400640 1 -- 172.21.15.18:0/3102621533 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f06b8104d40 msgr2=0x7f06b8115460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:55:37.937 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:37.932+0000 7f06b7400640 1 --2- 172.21.15.18:0/3102621533 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f06b8104d40 0x7f06b8115460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:55:37.937 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:37.932+0000 7f06b7400640 1 -- 172.21.15.18:0/3102621533 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06a0005040 con 0x7f06b81043a0 2024-04-12T21:55:37.937 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:37.932+0000 7f06b6a00640 1 --2- 172.21.15.18:0/3102621533 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f06b8104d40 0x7f06b8115460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-12T21:55:37.937 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:37.933+0000 7f06b7400640 1 --2- 172.21.15.18:0/3102621533 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f06b81043a0 0x7f06b8114f20 secure :-1 s=READY pgs=1403 cs=0 l=1 rev1=1 crypto rx=0x7f06a800b790 tx=0x7f06a80091d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:55:37.937 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:37.933+0000 7f06af400640 1 -- 172.21.15.18:0/3102621533 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f06a800c700 con 0x7f06b81043a0 2024-04-12T21:55:37.937 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:37.933+0000 7f06af400640 1 -- 172.21.15.18:0/3102621533 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f06a800cd30 con 0x7f06b81043a0 2024-04-12T21:55:37.937 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:37.933+0000 7f06af400640 1 -- 172.21.15.18:0/3102621533 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f06a8013370 con 0x7f06b81043a0 2024-04-12T21:55:37.937 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:37.933+0000 7f06be800640 1 -- 172.21.15.18:0/3102621533 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f06b810f390 con 0x7f06b81043a0 2024-04-12T21:55:37.937 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:37.933+0000 7f06be800640 1 -- 172.21.15.18:0/3102621533 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f06b810f860 con 0x7f06b81043a0 2024-04-12T21:55:37.938 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:37.934+0000 7f06be800640 1 -- 172.21.15.18:0/3102621533 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f06b80ffb10 con 0x7f06b81043a0 2024-04-12T21:55:37.938 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:37.936+0000 7f06af400640 1 -- 172.21.15.18:0/3102621533 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7f06a8012070 con 0x7f06b81043a0 2024-04-12T21:55:37.939 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:37.936+0000 7f06af400640 1 --2- 172.21.15.18:0/3102621533 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f069c06eb70 0x7f069c071020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:55:37.939 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:37.937+0000 7f06af400640 1 -- 172.21.15.18:0/3102621533 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 6757+0+0 (secure 0 0 0) 0x7f06a808e600 con 0x7f06b81043a0 2024-04-12T21:55:37.939 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:37.937+0000 7f06b6a00640 1 --2- 172.21.15.18:0/3102621533 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f069c06eb70 0x7f069c071020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:55:37.939 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:37.938+0000 7f06b6a00640 1 --2- 172.21.15.18:0/3102621533 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f069c06eb70 0x7f069c071020 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7f06a0000c00 tx=0x7f06a0009da0 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:55:37.943 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:37.941+0000 7f06af400640 1 -- 172.21.15.18:0/3102621533 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f06a80572a0 con 0x7f06b81043a0 2024-04-12T21:55:38.191 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:38.189+0000 7f06be800640 1 -- 172.21.15.18:0/3102621533 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 23, "format": "json"} v 0) v1 -- 0x7f06b81105f0 con 0x7f06b81043a0 2024-04-12T21:55:38.193 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:38.190+0000 7f06af400640 1 -- 172.21.15.18:0/3102621533 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 23, "format": "json"}]=0 dumped fsmap epoch 23 v38) v1 ==== 107+0+4872 (secure 0 0 0) 0x7f06a805a960 con 0x7f06b81043a0 2024-04-12T21:55:38.193 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-12T21:55:38.193 INFO:teuthology.orchestra.run.smithi018.stdout:{"epoch":23,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24277,"name":"cephfs.smithi186.wjnafs","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.186:6827/438347207","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6826","nonce":438347207},{"type":"v1","addr":"172.21.15.186:6827","nonce":438347207}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":18},{"gid":24281,"name":"cephfs.smithi186.jdllyn","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6825/1464359952","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6824","nonce":1464359952},{"type":"v1","addr":"172.21.15.186:6825","nonce":1464359952}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":16}],"filesystems":[{"mdsmap":{"epoch":23,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-12T20:51:57.362850+0000","modified":"2024-04-12T21:01:10.670287+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":46,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":14452,"mds_1":14464},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14452":{"gid":14452,"name":"cephfs.smithi018.acqbsc","rank":0,"incarnation":4,"state":"up:active","state_seq":5,"addr":"172.21.15.18:6827/1564065434","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6826","nonce":1564065434},{"type":"v1","addr":"172.21.15.18:6827","nonce":1564065434}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_14464":{"gid":14464,"name":"cephfs.smithi018.szjluo","rank":1,"incarnation":15,"state":"up:active","state_seq":6,"addr":"172.21.15.18:6829/722443353","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6828","nonce":722443353},{"type":"v1","addr":"172.21.15.18:6829","nonce":722443353}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-04-12T21:55:38.194 INFO:teuthology.orchestra.run.smithi018.stderr:dumped fsmap epoch 23 2024-04-12T21:55:38.195 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:38.193+0000 7f06be800640 1 -- 172.21.15.18:0/3102621533 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f069c06eb70 msgr2=0x7f069c071020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:55:38.195 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:38.193+0000 7f06be800640 1 --2- 172.21.15.18:0/3102621533 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f069c06eb70 0x7f069c071020 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7f06a0000c00 tx=0x7f06a0009da0 comp rx=0 tx=0).stop 2024-04-12T21:55:38.195 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:38.193+0000 7f06be800640 1 -- 172.21.15.18:0/3102621533 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f06b81043a0 msgr2=0x7f06b8114f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:55:38.195 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:38.193+0000 7f06be800640 1 --2- 172.21.15.18:0/3102621533 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f06b81043a0 0x7f06b8114f20 secure :-1 s=READY pgs=1403 cs=0 l=1 rev1=1 crypto rx=0x7f06a800b790 tx=0x7f06a80091d0 comp rx=0 tx=0).stop 2024-04-12T21:55:38.196 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:38.193+0000 7f06be800640 1 -- 172.21.15.18:0/3102621533 shutdown_connections 2024-04-12T21:55:38.196 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:38.193+0000 7f06be800640 1 --2- 172.21.15.18:0/3102621533 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f069c06eb70 0x7f069c071020 unknown :-1 s=CLOSED pgs=743 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:55:38.196 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:38.193+0000 7f06be800640 1 --2- 172.21.15.18:0/3102621533 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f06b8104d40 0x7f06b8115460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:55:38.196 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:38.193+0000 7f06be800640 1 --2- 172.21.15.18:0/3102621533 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f06b81043a0 0x7f06b8114f20 unknown :-1 s=CLOSED pgs=1403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:55:38.196 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:38.193+0000 7f06be800640 1 -- 172.21.15.18:0/3102621533 >> 172.21.15.18:0/3102621533 conn(0x7f06b80f9fc0 msgr2=0x7f06b80faf40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:55:38.196 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:38.193+0000 7f06be800640 1 -- 172.21.15.18:0/3102621533 shutdown_connections 2024-04-12T21:55:38.196 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:38.194+0000 7f06be800640 1 -- 172.21.15.18:0/3102621533 wait complete. 2024-04-12T21:55:38.216 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 24 2024-04-12T21:55:38.433 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:38.431+0000 7fa2c4c00640 1 -- 172.21.15.18:0/3943793506 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa2c0104210 msgr2=0x7fa2c01045e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:55:38.434 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:38.431+0000 7fa2c4c00640 1 --2- 172.21.15.18:0/3943793506 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa2c0104210 0x7fa2c01045e0 secure :-1 s=READY pgs=1404 cs=0 l=1 rev1=1 crypto rx=0x7fa2b00048d0 tx=0x7fa2b0009da0 comp rx=0 tx=0).stop 2024-04-12T21:55:38.434 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:38.431+0000 7fa2c4c00640 1 -- 172.21.15.18:0/3943793506 shutdown_connections 2024-04-12T21:55:38.434 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:38.431+0000 7fa2c4c00640 1 --2- 172.21.15.18:0/3943793506 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa2c0104bb0 0x7fa2c00fe240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:55:38.434 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:38.431+0000 7fa2c4c00640 1 --2- 172.21.15.18:0/3943793506 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa2c0104210 0x7fa2c01045e0 unknown :-1 s=CLOSED pgs=1404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:55:38.434 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:38.431+0000 7fa2c4c00640 1 -- 172.21.15.18:0/3943793506 >> 172.21.15.18:0/3943793506 conn(0x7fa2c00f9fc0 msgr2=0x7fa2c00fc3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:55:38.434 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:38.431+0000 7fa2c4c00640 1 -- 172.21.15.18:0/3943793506 shutdown_connections 2024-04-12T21:55:38.434 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:38.431+0000 7fa2c4c00640 1 -- 172.21.15.18:0/3943793506 wait complete. 2024-04-12T21:55:38.436 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:38.432+0000 7fa2c4c00640 1 Processor -- start 2024-04-12T21:55:38.436 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:38.432+0000 7fa2c4c00640 1 -- start start 2024-04-12T21:55:38.436 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:38.433+0000 7fa2c4c00640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa2c0104210 0x7fa2c0114d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:55:38.436 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:38.433+0000 7fa2c4c00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa2c0104bb0 0x7fa2c01152d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:55:38.436 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:38.433+0000 7fa2c4c00640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2c01158d0 con 0x7fa2c0104bb0 2024-04-12T21:55:38.436 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:38.433+0000 7fa2c4c00640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2c010ef20 con 0x7fa2c0104210 2024-04-12T21:55:38.436 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:38.433+0000 7fa2bea00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa2c0104bb0 0x7fa2c01152d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:55:38.436 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:38.433+0000 7fa2bea00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa2c0104bb0 0x7fa2c01152d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:51076/0 (socket says 172.21.15.18:51076) 2024-04-12T21:55:38.436 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:38.433+0000 7fa2bea00640 1 -- 172.21.15.18:0/3009616541 learned_addr learned my addr 172.21.15.18:0/3009616541 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:55:38.436 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:38.433+0000 7fa2bf400640 1 --2- 172.21.15.18:0/3009616541 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa2c0104210 0x7fa2c0114d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:55:38.436 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:38.433+0000 7fa2bea00640 1 -- 172.21.15.18:0/3009616541 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa2c0104210 msgr2=0x7fa2c0114d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:55:38.436 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:38.433+0000 7fa2bea00640 1 --2- 172.21.15.18:0/3009616541 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa2c0104210 0x7fa2c0114d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:55:38.436 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:38.433+0000 7fa2bea00640 1 -- 172.21.15.18:0/3009616541 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2b0005040 con 0x7fa2c0104bb0 2024-04-12T21:55:38.436 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:38.434+0000 7fa2bea00640 1 --2- 172.21.15.18:0/3009616541 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa2c0104bb0 0x7fa2c01152d0 secure :-1 s=READY pgs=1405 cs=0 l=1 rev1=1 crypto rx=0x7fa2a800db00 tx=0x7fa2a8004e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:55:38.437 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:38.434+0000 7fa2b7400640 1 -- 172.21.15.18:0/3009616541 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa2a8017070 con 0x7fa2c0104bb0 2024-04-12T21:55:38.437 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:38.434+0000 7fa2c4c00640 1 -- 172.21.15.18:0/3009616541 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa2c010f200 con 0x7fa2c0104bb0 2024-04-12T21:55:38.437 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:38.434+0000 7fa2b7400640 1 -- 172.21.15.18:0/3009616541 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa2a800f040 con 0x7fa2c0104bb0 2024-04-12T21:55:38.437 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:38.434+0000 7fa2b7400640 1 -- 172.21.15.18:0/3009616541 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa2a8003a70 con 0x7fa2c0104bb0 2024-04-12T21:55:38.438 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:38.434+0000 7fa2c4c00640 1 -- 172.21.15.18:0/3009616541 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa2c010f720 con 0x7fa2c0104bb0 2024-04-12T21:55:38.438 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:38.435+0000 7fa2c4c00640 1 -- 172.21.15.18:0/3009616541 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa2c00ff980 con 0x7fa2c0104bb0 2024-04-12T21:55:38.438 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:38.436+0000 7fa2b7400640 1 -- 172.21.15.18:0/3009616541 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7fa2a8003bd0 con 0x7fa2c0104bb0 2024-04-12T21:55:38.440 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:38.437+0000 7fa2b7400640 1 --2- 172.21.15.18:0/3009616541 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fa2a406eb70 0x7fa2a4071020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:55:38.440 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:38.437+0000 7fa2b7400640 1 -- 172.21.15.18:0/3009616541 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 6757+0+0 (secure 0 0 0) 0x7fa2a808eb50 con 0x7fa2c0104bb0 2024-04-12T21:55:38.440 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:38.438+0000 7fa2bf400640 1 --2- 172.21.15.18:0/3009616541 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fa2a406eb70 0x7fa2a4071020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:55:38.440 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:38.438+0000 7fa2bf400640 1 --2- 172.21.15.18:0/3009616541 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fa2a406eb70 0x7fa2a4071020 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7fa2b00048a0 tx=0x7fa2b0009da0 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:55:38.444 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:38.442+0000 7fa2b7400640 1 -- 172.21.15.18:0/3009616541 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7fa2a8057750 con 0x7fa2c0104bb0 2024-04-12T21:55:38.535 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:55:38 smithi186 ceph-mon[86379]: from='client.? 172.21.15.18:0/2740490368' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 22, "format": "json"}]: dispatch 2024-04-12T21:55:38.535 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:55:38 smithi186 ceph-mon[86379]: pgmap v1681: 65 pgs: 65 active+clean; 257 MiB data, 976 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:55:38.535 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:55:38 smithi186 ceph-mon[86379]: from='client.? 172.21.15.18:0/3102621533' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 23, "format": "json"}]: dispatch 2024-04-12T21:55:38.606 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:55:38 smithi018 ceph-mon[112590]: from='client.? 172.21.15.18:0/2740490368' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 22, "format": "json"}]: dispatch 2024-04-12T21:55:38.606 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:55:38 smithi018 ceph-mon[112590]: pgmap v1681: 65 pgs: 65 active+clean; 257 MiB data, 976 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:55:38.606 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:55:38 smithi018 ceph-mon[112590]: from='client.? 172.21.15.18:0/3102621533' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 23, "format": "json"}]: dispatch 2024-04-12T21:55:38.697 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:38.695+0000 7fa2c4c00640 1 -- 172.21.15.18:0/3009616541 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 24, "format": "json"} v 0) v1 -- 0x7fa2c010b3d0 con 0x7fa2c0104bb0 2024-04-12T21:55:38.699 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:38.696+0000 7fa2b7400640 1 -- 172.21.15.18:0/3009616541 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 24, "format": "json"}]=0 dumped fsmap epoch 24 v38) v1 ==== 107+0+3297 (secure 0 0 0) 0x7fa2a805ae10 con 0x7fa2c0104bb0 2024-04-12T21:55:38.699 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-12T21:55:38.699 INFO:teuthology.orchestra.run.smithi018.stdout:{"epoch":24,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24277,"name":"cephfs.smithi186.wjnafs","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.186:6827/438347207","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6826","nonce":438347207},{"type":"v1","addr":"172.21.15.186:6827","nonce":438347207}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":18},{"gid":24281,"name":"cephfs.smithi186.jdllyn","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6825/1464359952","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6824","nonce":1464359952},{"type":"v1","addr":"172.21.15.186:6825","nonce":1464359952}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":16}],"filesystems":[{"mdsmap":{"epoch":24,"flags":19,"flags_state":{"joinable":false,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-12T20:51:57.362850+0000","modified":"2024-04-12T21:06:59.717190+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":100,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{},"failed":[0,1],"damaged":[],"stopped":[],"info":{},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-04-12T21:55:38.699 INFO:teuthology.orchestra.run.smithi018.stderr:dumped fsmap epoch 24 2024-04-12T21:55:38.702 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:38.699+0000 7fa2c4c00640 1 -- 172.21.15.18:0/3009616541 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fa2a406eb70 msgr2=0x7fa2a4071020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:55:38.702 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:38.699+0000 7fa2c4c00640 1 --2- 172.21.15.18:0/3009616541 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fa2a406eb70 0x7fa2a4071020 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7fa2b00048a0 tx=0x7fa2b0009da0 comp rx=0 tx=0).stop 2024-04-12T21:55:38.702 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:38.700+0000 7fa2c4c00640 1 -- 172.21.15.18:0/3009616541 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa2c0104bb0 msgr2=0x7fa2c01152d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:55:38.702 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:38.700+0000 7fa2c4c00640 1 --2- 172.21.15.18:0/3009616541 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa2c0104bb0 0x7fa2c01152d0 secure :-1 s=READY pgs=1405 cs=0 l=1 rev1=1 crypto rx=0x7fa2a800db00 tx=0x7fa2a8004e60 comp rx=0 tx=0).stop 2024-04-12T21:55:38.702 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:38.700+0000 7fa2c4c00640 1 -- 172.21.15.18:0/3009616541 shutdown_connections 2024-04-12T21:55:38.702 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:38.700+0000 7fa2c4c00640 1 --2- 172.21.15.18:0/3009616541 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fa2a406eb70 0x7fa2a4071020 unknown :-1 s=CLOSED pgs=744 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:55:38.702 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:38.700+0000 7fa2c4c00640 1 --2- 172.21.15.18:0/3009616541 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa2c0104bb0 0x7fa2c01152d0 unknown :-1 s=CLOSED pgs=1405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:55:38.702 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:38.700+0000 7fa2c4c00640 1 --2- 172.21.15.18:0/3009616541 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa2c0104210 0x7fa2c0114d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:55:38.702 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:38.700+0000 7fa2c4c00640 1 -- 172.21.15.18:0/3009616541 >> 172.21.15.18:0/3009616541 conn(0x7fa2c00f9fc0 msgr2=0x7fa2c00fbf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:55:38.703 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:38.700+0000 7fa2c4c00640 1 -- 172.21.15.18:0/3009616541 shutdown_connections 2024-04-12T21:55:38.703 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:38.700+0000 7fa2c4c00640 1 -- 172.21.15.18:0/3009616541 wait complete. 2024-04-12T21:55:38.721 DEBUG:tasks.fs:max_mds not reduced in epoch 24 as fs was failed for carrying out rapid multi-rank mds upgrade 2024-04-12T21:55:38.722 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 25 2024-04-12T21:55:38.929 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:38.926+0000 7fe498400640 1 -- 172.21.15.18:0/1594115038 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe4901043e0 msgr2=0x7fe4901047b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:55:38.929 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:38.926+0000 7fe498400640 1 --2- 172.21.15.18:0/1594115038 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe4901043e0 0x7fe4901047b0 secure :-1 s=READY pgs=1406 cs=0 l=1 rev1=1 crypto rx=0x7fe4840048d0 tx=0x7fe484009da0 comp rx=0 tx=0).stop 2024-04-12T21:55:38.929 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:38.926+0000 7fe498400640 1 -- 172.21.15.18:0/1594115038 shutdown_connections 2024-04-12T21:55:38.929 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:38.926+0000 7fe498400640 1 --2- 172.21.15.18:0/1594115038 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe490104d80 0x7fe4900fe410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:55:38.929 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:38.926+0000 7fe498400640 1 --2- 172.21.15.18:0/1594115038 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe4901043e0 0x7fe4901047b0 unknown :-1 s=CLOSED pgs=1406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:55:38.929 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:38.926+0000 7fe498400640 1 -- 172.21.15.18:0/1594115038 >> 172.21.15.18:0/1594115038 conn(0x7fe4900f9ff0 msgr2=0x7fe4900fc400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:55:38.929 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:38.926+0000 7fe498400640 1 -- 172.21.15.18:0/1594115038 shutdown_connections 2024-04-12T21:55:38.929 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:38.927+0000 7fe498400640 1 -- 172.21.15.18:0/1594115038 wait complete. 2024-04-12T21:55:38.931 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:38.927+0000 7fe498400640 1 Processor -- start 2024-04-12T21:55:38.931 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:38.927+0000 7fe498400640 1 -- start start 2024-04-12T21:55:38.931 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:38.928+0000 7fe498400640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe4901043e0 0x7fe490114ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:55:38.932 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:38.928+0000 7fe498400640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe490104d80 0x7fe490115400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:55:38.932 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:38.928+0000 7fe498400640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe490115a00 con 0x7fe4901043e0 2024-04-12T21:55:38.932 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:38.928+0000 7fe498400640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe49010f050 con 0x7fe490104d80 2024-04-12T21:55:38.932 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:38.928+0000 7fe495a00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe4901043e0 0x7fe490114ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:55:38.932 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:38.928+0000 7fe495a00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe4901043e0 0x7fe490114ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:51086/0 (socket says 172.21.15.18:51086) 2024-04-12T21:55:38.932 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:38.928+0000 7fe495a00640 1 -- 172.21.15.18:0/3371243964 learned_addr learned my addr 172.21.15.18:0/3371243964 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:55:38.932 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:38.928+0000 7fe495000640 1 --2- 172.21.15.18:0/3371243964 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe490104d80 0x7fe490115400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:55:38.932 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:38.928+0000 7fe495a00640 1 -- 172.21.15.18:0/3371243964 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe490104d80 msgr2=0x7fe490115400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:55:38.932 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:38.928+0000 7fe495a00640 1 --2- 172.21.15.18:0/3371243964 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe490104d80 0x7fe490115400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:55:38.932 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:38.928+0000 7fe495a00640 1 -- 172.21.15.18:0/3371243964 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe484005040 con 0x7fe4901043e0 2024-04-12T21:55:38.932 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:38.929+0000 7fe495a00640 1 --2- 172.21.15.18:0/3371243964 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe4901043e0 0x7fe490114ec0 secure :-1 s=READY pgs=1407 cs=0 l=1 rev1=1 crypto rx=0x7fe4840047f0 tx=0x7fe484009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:55:38.932 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:38.929+0000 7fe48d600640 1 -- 172.21.15.18:0/3371243964 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe4840094c0 con 0x7fe4901043e0 2024-04-12T21:55:38.933 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:38.929+0000 7fe48d600640 1 -- 172.21.15.18:0/3371243964 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe484032e50 con 0x7fe4901043e0 2024-04-12T21:55:38.933 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:38.929+0000 7fe48d600640 1 -- 172.21.15.18:0/3371243964 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe4840433b0 con 0x7fe4901043e0 2024-04-12T21:55:38.933 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:38.929+0000 7fe498400640 1 -- 172.21.15.18:0/3371243964 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe49010f2d0 con 0x7fe4901043e0 2024-04-12T21:55:38.933 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:38.929+0000 7fe498400640 1 -- 172.21.15.18:0/3371243964 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe49010f790 con 0x7fe4901043e0 2024-04-12T21:55:38.933 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:38.930+0000 7fe498400640 1 -- 172.21.15.18:0/3371243964 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe4900ffb50 con 0x7fe4901043e0 2024-04-12T21:55:38.933 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:38.932+0000 7fe48d600640 1 -- 172.21.15.18:0/3371243964 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7fe48402f070 con 0x7fe4901043e0 2024-04-12T21:55:38.934 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:38.932+0000 7fe48d600640 1 --2- 172.21.15.18:0/3371243964 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fe47806eb70 0x7fe478071020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:55:38.935 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:38.933+0000 7fe48d600640 1 -- 172.21.15.18:0/3371243964 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 6757+0+0 (secure 0 0 0) 0x7fe4840b07f0 con 0x7fe4901043e0 2024-04-12T21:55:38.935 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:38.933+0000 7fe495000640 1 --2- 172.21.15.18:0/3371243964 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fe47806eb70 0x7fe478071020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:55:38.936 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:38.934+0000 7fe495000640 1 --2- 172.21.15.18:0/3371243964 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fe47806eb70 0x7fe478071020 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7fe490110780 tx=0x7fe47c00b040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:55:38.939 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:38.937+0000 7fe48d600640 1 -- 172.21.15.18:0/3371243964 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7fe484079490 con 0x7fe4901043e0 2024-04-12T21:55:39.185 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:39.182+0000 7fe498400640 1 -- 172.21.15.18:0/3371243964 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 25, "format": "json"} v 0) v1 -- 0x7fe490110530 con 0x7fe4901043e0 2024-04-12T21:55:39.186 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:39.183+0000 7fe48d600640 1 -- 172.21.15.18:0/3371243964 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 25, "format": "json"}]=0 dumped fsmap epoch 25 v38) v1 ==== 107+0+4852 (secure 0 0 0) 0x7fe48407cb50 con 0x7fe4901043e0 2024-04-12T21:55:39.186 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-12T21:55:39.186 INFO:teuthology.orchestra.run.smithi018.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24277,"name":"cephfs.smithi186.wjnafs","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.186:6827/438347207","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6826","nonce":438347207},{"type":"v1","addr":"172.21.15.186:6827","nonce":438347207}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":18},{"gid":24281,"name":"cephfs.smithi186.jdllyn","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6825/1464359952","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6824","nonce":1464359952},{"type":"v1","addr":"172.21.15.186:6825","nonce":1464359952}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":16},{"gid":34430,"name":"cephfs.smithi018.szjluo","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.18:6827/785330430","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6826","nonce":785330430},{"type":"v1","addr":"172.21.15.18:6827","nonce":785330430}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":25},{"gid":34432,"name":"cephfs.smithi018.acqbsc","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.18:6829/2928314745","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6828","nonce":2928314745},{"type":"v1","addr":"172.21.15.18:6829","nonce":2928314745}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":25}],"filesystems":[{"mdsmap":{"epoch":24,"flags":19,"flags_state":{"joinable":false,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-12T20:51:57.362850+0000","modified":"2024-04-12T21:06:59.717190+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":100,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{},"failed":[0,1],"damaged":[],"stopped":[],"info":{},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-04-12T21:55:39.186 INFO:teuthology.orchestra.run.smithi018.stderr:dumped fsmap epoch 25 2024-04-12T21:55:39.189 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:39.186+0000 7fe498400640 1 -- 172.21.15.18:0/3371243964 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fe47806eb70 msgr2=0x7fe478071020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:55:39.189 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:39.186+0000 7fe498400640 1 --2- 172.21.15.18:0/3371243964 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fe47806eb70 0x7fe478071020 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7fe490110780 tx=0x7fe47c00b040 comp rx=0 tx=0).stop 2024-04-12T21:55:39.189 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:39.186+0000 7fe498400640 1 -- 172.21.15.18:0/3371243964 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe4901043e0 msgr2=0x7fe490114ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:55:39.189 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:39.186+0000 7fe498400640 1 --2- 172.21.15.18:0/3371243964 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe4901043e0 0x7fe490114ec0 secure :-1 s=READY pgs=1407 cs=0 l=1 rev1=1 crypto rx=0x7fe4840047f0 tx=0x7fe484009da0 comp rx=0 tx=0).stop 2024-04-12T21:55:39.189 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:39.186+0000 7fe498400640 1 -- 172.21.15.18:0/3371243964 shutdown_connections 2024-04-12T21:55:39.189 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:39.186+0000 7fe498400640 1 --2- 172.21.15.18:0/3371243964 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fe47806eb70 0x7fe478071020 unknown :-1 s=CLOSED pgs=745 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:55:39.189 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:39.186+0000 7fe498400640 1 --2- 172.21.15.18:0/3371243964 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe490104d80 0x7fe490115400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:55:39.189 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:39.186+0000 7fe498400640 1 --2- 172.21.15.18:0/3371243964 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe4901043e0 0x7fe490114ec0 unknown :-1 s=CLOSED pgs=1407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:55:39.189 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:39.186+0000 7fe498400640 1 -- 172.21.15.18:0/3371243964 >> 172.21.15.18:0/3371243964 conn(0x7fe4900f9ff0 msgr2=0x7fe4900faf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:55:39.189 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:39.186+0000 7fe498400640 1 -- 172.21.15.18:0/3371243964 shutdown_connections 2024-04-12T21:55:39.189 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:39.186+0000 7fe498400640 1 -- 172.21.15.18:0/3371243964 wait complete. 2024-04-12T21:55:39.207 DEBUG:tasks.fs:max_mds not reduced in epoch 25 as fs was failed for carrying out rapid multi-rank mds upgrade 2024-04-12T21:55:39.207 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 26 2024-04-12T21:55:39.420 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:39.417+0000 7f012f200640 1 -- 172.21.15.18:0/1366428846 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0128104bb0 msgr2=0x7f01280fe240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:55:39.420 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:39.417+0000 7f012f200640 1 --2- 172.21.15.18:0/1366428846 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0128104bb0 0x7f01280fe240 secure :-1 s=READY pgs=1408 cs=0 l=1 rev1=1 crypto rx=0x7f01180048d0 tx=0x7f0118009da0 comp rx=0 tx=0).stop 2024-04-12T21:55:39.420 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:39.417+0000 7f012f200640 1 -- 172.21.15.18:0/1366428846 shutdown_connections 2024-04-12T21:55:39.420 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:39.417+0000 7f012f200640 1 --2- 172.21.15.18:0/1366428846 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0128104bb0 0x7f01280fe240 unknown :-1 s=CLOSED pgs=1408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:55:39.420 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:39.417+0000 7f012f200640 1 --2- 172.21.15.18:0/1366428846 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0128104210 0x7f01281045e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:55:39.420 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:39.417+0000 7f012f200640 1 -- 172.21.15.18:0/1366428846 >> 172.21.15.18:0/1366428846 conn(0x7f01280f9fc0 msgr2=0x7f01280fc3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:55:39.420 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:39.417+0000 7f012f200640 1 -- 172.21.15.18:0/1366428846 shutdown_connections 2024-04-12T21:55:39.420 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:39.417+0000 7f012f200640 1 -- 172.21.15.18:0/1366428846 wait complete. 2024-04-12T21:55:39.420 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:39.418+0000 7f012f200640 1 Processor -- start 2024-04-12T21:55:39.420 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:39.418+0000 7f012f200640 1 -- start start 2024-04-12T21:55:39.422 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:39.418+0000 7f012f200640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0128104210 0x7f0128114dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:55:39.422 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:39.418+0000 7f012f200640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0128104bb0 0x7f0128115310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:55:39.422 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:39.418+0000 7f012f200640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01281159a0 con 0x7f0128104bb0 2024-04-12T21:55:39.422 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:39.418+0000 7f012f200640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f012810ef60 con 0x7f0128104210 2024-04-12T21:55:39.422 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:39.418+0000 7f012d400640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0128104bb0 0x7f0128115310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:55:39.423 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:39.418+0000 7f012d400640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0128104bb0 0x7f0128115310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:51092/0 (socket says 172.21.15.18:51092) 2024-04-12T21:55:39.423 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:39.418+0000 7f012d400640 1 -- 172.21.15.18:0/3877102847 learned_addr learned my addr 172.21.15.18:0/3877102847 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:55:39.423 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:39.419+0000 7f012d400640 1 -- 172.21.15.18:0/3877102847 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0128104210 msgr2=0x7f0128114dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:55:39.423 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:39.419+0000 7f012de00640 1 --2- 172.21.15.18:0/3877102847 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0128104210 0x7f0128114dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:55:39.423 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:39.419+0000 7f012d400640 1 --2- 172.21.15.18:0/3877102847 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0128104210 0x7f0128114dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:55:39.423 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:39.419+0000 7f012d400640 1 -- 172.21.15.18:0/3877102847 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0118005040 con 0x7f0128104bb0 2024-04-12T21:55:39.423 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:39.419+0000 7f012d400640 1 --2- 172.21.15.18:0/3877102847 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0128104bb0 0x7f0128115310 secure :-1 s=READY pgs=1409 cs=0 l=1 rev1=1 crypto rx=0x7f0118004930 tx=0x7f0118009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:55:39.423 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:39.419+0000 7f0122000640 1 -- 172.21.15.18:0/3877102847 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f01180094c0 con 0x7f0128104bb0 2024-04-12T21:55:39.423 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:39.419+0000 7f0122000640 1 -- 172.21.15.18:0/3877102847 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f0118032e50 con 0x7f0128104bb0 2024-04-12T21:55:39.423 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:39.419+0000 7f012f200640 1 -- 172.21.15.18:0/3877102847 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f012810f1e0 con 0x7f0128104bb0 2024-04-12T21:55:39.423 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:39.420+0000 7f0122000640 1 -- 172.21.15.18:0/3877102847 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f01180433b0 con 0x7f0128104bb0 2024-04-12T21:55:39.423 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:39.420+0000 7f012f200640 1 -- 172.21.15.18:0/3877102847 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f012810f6a0 con 0x7f0128104bb0 2024-04-12T21:55:39.424 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:39.421+0000 7f012f200640 1 -- 172.21.15.18:0/3877102847 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f01280ff980 con 0x7f0128104bb0 2024-04-12T21:55:39.424 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:39.422+0000 7f0122000640 1 -- 172.21.15.18:0/3877102847 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7f011802f070 con 0x7f0128104bb0 2024-04-12T21:55:39.425 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:39.423+0000 7f0122000640 1 --2- 172.21.15.18:0/3877102847 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f011006eb70 0x7f0110071020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:55:39.425 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:39.423+0000 7f0122000640 1 -- 172.21.15.18:0/3877102847 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 6757+0+0 (secure 0 0 0) 0x7f01180b0a00 con 0x7f0128104bb0 2024-04-12T21:55:39.425 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:39.423+0000 7f012de00640 1 --2- 172.21.15.18:0/3877102847 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f011006eb70 0x7f0110071020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:55:39.425 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:39.423+0000 7f012de00640 1 --2- 172.21.15.18:0/3877102847 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f011006eb70 0x7f0110071020 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7f011c00a900 tx=0x7f011c008040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:55:39.430 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:39.428+0000 7f0122000640 1 -- 172.21.15.18:0/3877102847 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f0118079600 con 0x7f0128104bb0 2024-04-12T21:55:39.534 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:55:39 smithi186 ceph-mon[86379]: from='client.? 172.21.15.18:0/3009616541' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 24, "format": "json"}]: dispatch 2024-04-12T21:55:39.535 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:55:39 smithi186 ceph-mon[86379]: from='client.? 172.21.15.18:0/3371243964' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 25, "format": "json"}]: dispatch 2024-04-12T21:55:39.606 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:55:39 smithi018 ceph-mon[112590]: from='client.? 172.21.15.18:0/3009616541' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 24, "format": "json"}]: dispatch 2024-04-12T21:55:39.606 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:55:39 smithi018 ceph-mon[112590]: from='client.? 172.21.15.18:0/3371243964' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 25, "format": "json"}]: dispatch 2024-04-12T21:55:39.671 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:39.669+0000 7f012f200640 1 -- 172.21.15.18:0/3877102847 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 26, "format": "json"} v 0) v1 -- 0x7f012810f8c0 con 0x7f0128104bb0 2024-04-12T21:55:39.672 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:39.670+0000 7f0122000640 1 -- 172.21.15.18:0/3877102847 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 26, "format": "json"}]=0 dumped fsmap epoch 26 v38) v1 ==== 107+0+4073 (secure 0 0 0) 0x7f011807ccc0 con 0x7f0128104bb0 2024-04-12T21:55:39.673 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-12T21:55:39.673 INFO:teuthology.orchestra.run.smithi018.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24277,"name":"cephfs.smithi186.wjnafs","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.186:6827/438347207","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6826","nonce":438347207},{"type":"v1","addr":"172.21.15.186:6827","nonce":438347207}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":18},{"gid":24281,"name":"cephfs.smithi186.jdllyn","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6825/1464359952","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6824","nonce":1464359952},{"type":"v1","addr":"172.21.15.186:6825","nonce":1464359952}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":16},{"gid":34430,"name":"cephfs.smithi018.szjluo","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.18:6827/785330430","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6826","nonce":785330430},{"type":"v1","addr":"172.21.15.18:6827","nonce":785330430}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":25}],"filesystems":[{"mdsmap":{"epoch":24,"flags":19,"flags_state":{"joinable":false,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-12T20:51:57.362850+0000","modified":"2024-04-12T21:06:59.717190+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":100,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{},"failed":[0,1],"damaged":[],"stopped":[],"info":{},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-04-12T21:55:39.673 INFO:teuthology.orchestra.run.smithi018.stderr:dumped fsmap epoch 26 2024-04-12T21:55:39.676 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:39.673+0000 7f012f200640 1 -- 172.21.15.18:0/3877102847 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f011006eb70 msgr2=0x7f0110071020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:55:39.676 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:39.673+0000 7f012f200640 1 --2- 172.21.15.18:0/3877102847 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f011006eb70 0x7f0110071020 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7f011c00a900 tx=0x7f011c008040 comp rx=0 tx=0).stop 2024-04-12T21:55:39.676 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:39.673+0000 7f012f200640 1 -- 172.21.15.18:0/3877102847 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0128104bb0 msgr2=0x7f0128115310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:55:39.676 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:39.673+0000 7f012f200640 1 --2- 172.21.15.18:0/3877102847 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0128104bb0 0x7f0128115310 secure :-1 s=READY pgs=1409 cs=0 l=1 rev1=1 crypto rx=0x7f0118004930 tx=0x7f0118009da0 comp rx=0 tx=0).stop 2024-04-12T21:55:39.676 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:39.673+0000 7f012f200640 1 -- 172.21.15.18:0/3877102847 shutdown_connections 2024-04-12T21:55:39.676 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:39.673+0000 7f012f200640 1 --2- 172.21.15.18:0/3877102847 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f011006eb70 0x7f0110071020 unknown :-1 s=CLOSED pgs=746 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:55:39.676 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:39.673+0000 7f012f200640 1 --2- 172.21.15.18:0/3877102847 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0128104bb0 0x7f0128115310 unknown :-1 s=CLOSED pgs=1409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:55:39.676 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:39.673+0000 7f012f200640 1 --2- 172.21.15.18:0/3877102847 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0128104210 0x7f0128114dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:55:39.676 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:39.673+0000 7f012f200640 1 -- 172.21.15.18:0/3877102847 >> 172.21.15.18:0/3877102847 conn(0x7f01280f9fc0 msgr2=0x7f01280fbe90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:55:39.676 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:39.674+0000 7f012f200640 1 -- 172.21.15.18:0/3877102847 shutdown_connections 2024-04-12T21:55:39.676 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:39.674+0000 7f012f200640 1 -- 172.21.15.18:0/3877102847 wait complete. 2024-04-12T21:55:39.693 DEBUG:tasks.fs:max_mds not reduced in epoch 26 as fs was failed for carrying out rapid multi-rank mds upgrade 2024-04-12T21:55:39.693 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 27 2024-04-12T21:55:39.905 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:39.902+0000 7f0dd9200640 1 -- 172.21.15.18:0/1040804250 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0dd4102940 msgr2=0x7f0dd40fbfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:55:39.905 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:39.902+0000 7f0dd9200640 1 --2- 172.21.15.18:0/1040804250 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0dd4102940 0x7f0dd40fbfd0 secure :-1 s=READY pgs=1410 cs=0 l=1 rev1=1 crypto rx=0x7f0dbc0048d0 tx=0x7f0dbc009da0 comp rx=0 tx=0).stop 2024-04-12T21:55:39.906 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:39.903+0000 7f0dd9200640 1 -- 172.21.15.18:0/1040804250 shutdown_connections 2024-04-12T21:55:39.906 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:39.903+0000 7f0dd9200640 1 --2- 172.21.15.18:0/1040804250 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0dd4102940 0x7f0dd40fbfd0 unknown :-1 s=CLOSED pgs=1410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:55:39.906 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:39.903+0000 7f0dd9200640 1 --2- 172.21.15.18:0/1040804250 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0dd4101fa0 0x7f0dd4102370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:55:39.906 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:39.903+0000 7f0dd9200640 1 -- 172.21.15.18:0/1040804250 >> 172.21.15.18:0/1040804250 conn(0x7f0dd40f7b70 msgr2=0x7f0dd40f9f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:55:39.906 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:39.903+0000 7f0dd9200640 1 -- 172.21.15.18:0/1040804250 shutdown_connections 2024-04-12T21:55:39.906 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:39.903+0000 7f0dd9200640 1 -- 172.21.15.18:0/1040804250 wait complete. 2024-04-12T21:55:39.907 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:39.904+0000 7f0dd9200640 1 Processor -- start 2024-04-12T21:55:39.907 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:39.904+0000 7f0dd9200640 1 -- start start 2024-04-12T21:55:39.908 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:39.904+0000 7f0dd9200640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0dd4101fa0 0x7f0dd406f7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:55:39.908 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:39.905+0000 7f0dd9200640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0dd4102940 0x7f0dd406fd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:55:39.908 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:39.905+0000 7f0dd9200640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0dd4071740 con 0x7f0dd4102940 2024-04-12T21:55:39.908 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:39.905+0000 7f0dd9200640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0dd4070250 con 0x7f0dd4101fa0 2024-04-12T21:55:39.908 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:39.905+0000 7f0dd2a00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0dd4102940 0x7f0dd406fd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:55:39.908 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:39.905+0000 7f0dd2a00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0dd4102940 0x7f0dd406fd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:51108/0 (socket says 172.21.15.18:51108) 2024-04-12T21:55:39.908 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:39.905+0000 7f0dd2a00640 1 -- 172.21.15.18:0/278266599 learned_addr learned my addr 172.21.15.18:0/278266599 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:55:39.908 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:39.905+0000 7f0dd3400640 1 --2- 172.21.15.18:0/278266599 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0dd4101fa0 0x7f0dd406f7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:55:39.908 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:39.905+0000 7f0dd2a00640 1 -- 172.21.15.18:0/278266599 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0dd4101fa0 msgr2=0x7f0dd406f7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:55:39.908 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:39.905+0000 7f0dd2a00640 1 --2- 172.21.15.18:0/278266599 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0dd4101fa0 0x7f0dd406f7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:55:39.908 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:39.905+0000 7f0dd2a00640 1 -- 172.21.15.18:0/278266599 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0dbc005040 con 0x7f0dd4102940 2024-04-12T21:55:39.909 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:39.906+0000 7f0dd2a00640 1 --2- 172.21.15.18:0/278266599 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0dd4102940 0x7f0dd406fd10 secure :-1 s=READY pgs=1411 cs=0 l=1 rev1=1 crypto rx=0x7f0dbc0047f0 tx=0x7f0dbc009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:55:39.909 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:39.906+0000 7f0dcb400640 1 -- 172.21.15.18:0/278266599 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0dbc0094c0 con 0x7f0dd4102940 2024-04-12T21:55:39.909 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:39.906+0000 7f0dcb400640 1 -- 172.21.15.18:0/278266599 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f0dbc032e50 con 0x7f0dd4102940 2024-04-12T21:55:39.909 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:39.906+0000 7f0dcb400640 1 -- 172.21.15.18:0/278266599 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0dbc0435f0 con 0x7f0dd4102940 2024-04-12T21:55:39.910 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:39.906+0000 7f0dd9200640 1 -- 172.21.15.18:0/278266599 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0dd4070470 con 0x7f0dd4102940 2024-04-12T21:55:39.910 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:39.906+0000 7f0dd9200640 1 -- 172.21.15.18:0/278266599 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0dd41191f0 con 0x7f0dd4102940 2024-04-12T21:55:39.910 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:39.907+0000 7f0dd9200640 1 -- 172.21.15.18:0/278266599 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0dd40fd710 con 0x7f0dd4102940 2024-04-12T21:55:39.910 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:39.908+0000 7f0dcb400640 1 -- 172.21.15.18:0/278266599 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7f0dbc02f070 con 0x7f0dd4102940 2024-04-12T21:55:39.911 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:39.909+0000 7f0dcb400640 1 --2- 172.21.15.18:0/278266599 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f0db806eb70 0x7f0db8071020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:55:39.911 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:39.909+0000 7f0dcb400640 1 -- 172.21.15.18:0/278266599 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 6757+0+0 (secure 0 0 0) 0x7f0dbc0b08b0 con 0x7f0dd4102940 2024-04-12T21:55:39.912 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:39.910+0000 7f0dd3400640 1 --2- 172.21.15.18:0/278266599 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f0db806eb70 0x7f0db8071020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:55:39.912 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:39.910+0000 7f0dd3400640 1 --2- 172.21.15.18:0/278266599 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f0db806eb70 0x7f0db8071020 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7f0dc4001250 tx=0x7f0dc4009620 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:55:39.916 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:39.914+0000 7f0dcb400640 1 -- 172.21.15.18:0/278266599 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f0dbc0794b0 con 0x7f0dd4102940 2024-04-12T21:55:40.165 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:40.163+0000 7f0dd9200640 1 -- 172.21.15.18:0/278266599 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 27, "format": "json"} v 0) v1 -- 0x7f0dd404bcc0 con 0x7f0dd4102940 2024-04-12T21:55:40.166 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:40.164+0000 7f0dcb400640 1 -- 172.21.15.18:0/278266599 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 27, "format": "json"}]=0 dumped fsmap epoch 27 v38) v1 ==== 107+0+4917 (secure 0 0 0) 0x7f0dbc07cb70 con 0x7f0dd4102940 2024-04-12T21:55:40.167 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-12T21:55:40.167 INFO:teuthology.orchestra.run.smithi018.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24277,"name":"cephfs.smithi186.wjnafs","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.186:6827/438347207","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6826","nonce":438347207},{"type":"v1","addr":"172.21.15.186:6827","nonce":438347207}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":18},{"gid":24281,"name":"cephfs.smithi186.jdllyn","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6825/1464359952","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6824","nonce":1464359952},{"type":"v1","addr":"172.21.15.186:6825","nonce":1464359952}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":16},{"gid":34430,"name":"cephfs.smithi018.szjluo","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.18:6827/785330430","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6826","nonce":785330430},{"type":"v1","addr":"172.21.15.18:6827","nonce":785330430}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":25},{"gid":34436,"name":"cephfs.smithi018.acqbsc","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.18:6829/755179764","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6828","nonce":755179764},{"type":"v1","addr":"172.21.15.18:6829","nonce":755179764}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":27}],"filesystems":[{"mdsmap":{"epoch":24,"flags":19,"flags_state":{"joinable":false,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-12T20:51:57.362850+0000","modified":"2024-04-12T21:06:59.717190+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":100,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{},"failed":[0,1],"damaged":[],"stopped":[],"info":{},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-04-12T21:55:40.167 INFO:teuthology.orchestra.run.smithi018.stderr:dumped fsmap epoch 27 2024-04-12T21:55:40.169 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:40.167+0000 7f0dd9200640 1 -- 172.21.15.18:0/278266599 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f0db806eb70 msgr2=0x7f0db8071020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:55:40.169 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:40.167+0000 7f0dd9200640 1 --2- 172.21.15.18:0/278266599 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f0db806eb70 0x7f0db8071020 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7f0dc4001250 tx=0x7f0dc4009620 comp rx=0 tx=0).stop 2024-04-12T21:55:40.170 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:40.167+0000 7f0dd9200640 1 -- 172.21.15.18:0/278266599 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0dd4102940 msgr2=0x7f0dd406fd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:55:40.170 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:40.167+0000 7f0dd9200640 1 --2- 172.21.15.18:0/278266599 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0dd4102940 0x7f0dd406fd10 secure :-1 s=READY pgs=1411 cs=0 l=1 rev1=1 crypto rx=0x7f0dbc0047f0 tx=0x7f0dbc009da0 comp rx=0 tx=0).stop 2024-04-12T21:55:40.170 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:40.167+0000 7f0dd9200640 1 -- 172.21.15.18:0/278266599 shutdown_connections 2024-04-12T21:55:40.170 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:40.167+0000 7f0dd9200640 1 --2- 172.21.15.18:0/278266599 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f0db806eb70 0x7f0db8071020 unknown :-1 s=CLOSED pgs=747 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:55:40.170 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:40.167+0000 7f0dd9200640 1 --2- 172.21.15.18:0/278266599 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0dd4102940 0x7f0dd406fd10 unknown :-1 s=CLOSED pgs=1411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:55:40.170 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:40.167+0000 7f0dd9200640 1 --2- 172.21.15.18:0/278266599 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0dd4101fa0 0x7f0dd406f7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:55:40.170 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:40.167+0000 7f0dd9200640 1 -- 172.21.15.18:0/278266599 >> 172.21.15.18:0/278266599 conn(0x7f0dd40f7b70 msgr2=0x7f0dd40f8b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:55:40.170 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:40.167+0000 7f0dd9200640 1 -- 172.21.15.18:0/278266599 shutdown_connections 2024-04-12T21:55:40.170 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:40.167+0000 7f0dd9200640 1 -- 172.21.15.18:0/278266599 wait complete. 2024-04-12T21:55:40.188 DEBUG:tasks.fs:max_mds not reduced in epoch 27 as fs was failed for carrying out rapid multi-rank mds upgrade 2024-04-12T21:55:40.188 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 28 2024-04-12T21:55:40.389 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:40.386+0000 7fe778a00640 1 -- 172.21.15.18:0/1590500484 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe7740643a0 msgr2=0x7fe7740647d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:55:40.389 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:40.386+0000 7fe778a00640 1 --2- 172.21.15.18:0/1590500484 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe7740643a0 0x7fe7740647d0 secure :-1 s=READY pgs=1412 cs=0 l=1 rev1=1 crypto rx=0x7fe75c0048d0 tx=0x7fe75c009da0 comp rx=0 tx=0).stop 2024-04-12T21:55:40.389 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:40.386+0000 7fe778a00640 1 -- 172.21.15.18:0/1590500484 shutdown_connections 2024-04-12T21:55:40.389 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:40.386+0000 7fe778a00640 1 --2- 172.21.15.18:0/1590500484 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe7740643a0 0x7fe7740647d0 unknown :-1 s=CLOSED pgs=1412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:55:40.389 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:40.386+0000 7fe778a00640 1 --2- 172.21.15.18:0/1590500484 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe774102050 0x7fe774102420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:55:40.389 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:40.386+0000 7fe778a00640 1 -- 172.21.15.18:0/1590500484 >> 172.21.15.18:0/1590500484 conn(0x7fe774070f80 msgr2=0x7fe774071380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:55:40.389 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:40.387+0000 7fe778a00640 1 -- 172.21.15.18:0/1590500484 shutdown_connections 2024-04-12T21:55:40.390 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:40.387+0000 7fe778a00640 1 -- 172.21.15.18:0/1590500484 wait complete. 2024-04-12T21:55:40.391 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:40.387+0000 7fe778a00640 1 Processor -- start 2024-04-12T21:55:40.391 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:40.388+0000 7fe778a00640 1 -- start start 2024-04-12T21:55:40.391 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:40.388+0000 7fe778a00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe7740643a0 0x7fe774114d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:55:40.391 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:40.388+0000 7fe778a00640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe774102050 0x7fe774115240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:55:40.391 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:40.388+0000 7fe778a00640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe774115840 con 0x7fe7740643a0 2024-04-12T21:55:40.391 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:40.388+0000 7fe778a00640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe77410ee90 con 0x7fe774102050 2024-04-12T21:55:40.391 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:40.388+0000 7fe772800640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe7740643a0 0x7fe774114d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:55:40.391 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:40.388+0000 7fe772800640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe7740643a0 0x7fe774114d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:51124/0 (socket says 172.21.15.18:51124) 2024-04-12T21:55:40.391 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:40.388+0000 7fe772800640 1 -- 172.21.15.18:0/1951141307 learned_addr learned my addr 172.21.15.18:0/1951141307 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:55:40.391 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:40.389+0000 7fe772800640 1 -- 172.21.15.18:0/1951141307 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe774102050 msgr2=0x7fe774115240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:55:40.392 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:40.389+0000 7fe771e00640 1 --2- 172.21.15.18:0/1951141307 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe774102050 0x7fe774115240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:55:40.392 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:40.389+0000 7fe772800640 1 --2- 172.21.15.18:0/1951141307 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe774102050 0x7fe774115240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:55:40.392 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:40.389+0000 7fe772800640 1 -- 172.21.15.18:0/1951141307 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe75c005040 con 0x7fe7740643a0 2024-04-12T21:55:40.392 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:40.389+0000 7fe771e00640 1 --2- 172.21.15.18:0/1951141307 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe774102050 0x7fe774115240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-12T21:55:40.392 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:40.389+0000 7fe772800640 1 --2- 172.21.15.18:0/1951141307 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe7740643a0 0x7fe774114d00 secure :-1 s=READY pgs=1413 cs=0 l=1 rev1=1 crypto rx=0x7fe76400c810 tx=0x7fe7640043c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:55:40.393 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:40.389+0000 7fe76aa00640 1 -- 172.21.15.18:0/1951141307 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe76400a820 con 0x7fe7740643a0 2024-04-12T21:55:40.393 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:40.390+0000 7fe778a00640 1 -- 172.21.15.18:0/1951141307 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe77410f170 con 0x7fe7740643a0 2024-04-12T21:55:40.393 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:40.390+0000 7fe76aa00640 1 -- 172.21.15.18:0/1951141307 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe76400ae50 con 0x7fe7740643a0 2024-04-12T21:55:40.393 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:40.390+0000 7fe76aa00640 1 -- 172.21.15.18:0/1951141307 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe7640115c0 con 0x7fe7740643a0 2024-04-12T21:55:40.393 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:40.390+0000 7fe778a00640 1 -- 172.21.15.18:0/1951141307 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe77410f690 con 0x7fe7740643a0 2024-04-12T21:55:40.393 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:40.391+0000 7fe778a00640 1 -- 172.21.15.18:0/1951141307 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe7740fd750 con 0x7fe7740643a0 2024-04-12T21:55:40.394 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:40.392+0000 7fe76aa00640 1 -- 172.21.15.18:0/1951141307 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7fe76400a980 con 0x7fe7740643a0 2024-04-12T21:55:40.395 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:40.393+0000 7fe76aa00640 1 --2- 172.21.15.18:0/1951141307 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fe75806eb70 0x7fe758071020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:55:40.395 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:40.393+0000 7fe76aa00640 1 -- 172.21.15.18:0/1951141307 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 6757+0+0 (secure 0 0 0) 0x7fe76408ef20 con 0x7fe7740643a0 2024-04-12T21:55:40.395 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:40.393+0000 7fe771e00640 1 --2- 172.21.15.18:0/1951141307 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fe75806eb70 0x7fe758071020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:55:40.396 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:40.394+0000 7fe771e00640 1 --2- 172.21.15.18:0/1951141307 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fe75806eb70 0x7fe758071020 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7fe75c000c00 tx=0x7fe75c009da0 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:55:40.399 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:40.397+0000 7fe76aa00640 1 -- 172.21.15.18:0/1951141307 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7fe764057ba0 con 0x7fe7740643a0 2024-04-12T21:55:40.535 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:55:40 smithi186 ceph-mon[86379]: from='client.? 172.21.15.18:0/3877102847' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 26, "format": "json"}]: dispatch 2024-04-12T21:55:40.535 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:55:40 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T21:55:40.535 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:55:40 smithi186 ceph-mon[86379]: pgmap v1682: 65 pgs: 65 active+clean; 257 MiB data, 976 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:55:40.535 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:55:40 smithi186 ceph-mon[86379]: from='client.? 172.21.15.18:0/278266599' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 27, "format": "json"}]: dispatch 2024-04-12T21:55:40.606 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:55:40 smithi018 ceph-mon[112590]: from='client.? 172.21.15.18:0/3877102847' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 26, "format": "json"}]: dispatch 2024-04-12T21:55:40.606 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:55:40 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T21:55:40.606 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:55:40 smithi018 ceph-mon[112590]: pgmap v1682: 65 pgs: 65 active+clean; 257 MiB data, 976 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:55:40.606 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:55:40 smithi018 ceph-mon[112590]: from='client.? 172.21.15.18:0/278266599' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 27, "format": "json"}]: dispatch 2024-04-12T21:55:40.640 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:40.638+0000 7fe778a00640 1 -- 172.21.15.18:0/1951141307 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 28, "format": "json"} v 0) v1 -- 0x7fe774110370 con 0x7fe7740643a0 2024-04-12T21:55:40.642 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:40.640+0000 7fe76aa00640 1 -- 172.21.15.18:0/1951141307 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 28, "format": "json"}]=0 dumped fsmap epoch 28 v38) v1 ==== 107+0+4141 (secure 0 0 0) 0x7fe76405b260 con 0x7fe7740643a0 2024-04-12T21:55:40.642 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-12T21:55:40.642 INFO:teuthology.orchestra.run.smithi018.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24277,"name":"cephfs.smithi186.wjnafs","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.186:6827/438347207","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6826","nonce":438347207},{"type":"v1","addr":"172.21.15.186:6827","nonce":438347207}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":18},{"gid":24281,"name":"cephfs.smithi186.jdllyn","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6825/1464359952","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6824","nonce":1464359952},{"type":"v1","addr":"172.21.15.186:6825","nonce":1464359952}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":16},{"gid":34436,"name":"cephfs.smithi018.acqbsc","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.18:6829/755179764","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6828","nonce":755179764},{"type":"v1","addr":"172.21.15.18:6829","nonce":755179764}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":27}],"filesystems":[{"mdsmap":{"epoch":24,"flags":19,"flags_state":{"joinable":false,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-12T20:51:57.362850+0000","modified":"2024-04-12T21:06:59.717190+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":100,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{},"failed":[0,1],"damaged":[],"stopped":[],"info":{},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-04-12T21:55:40.643 INFO:teuthology.orchestra.run.smithi018.stderr:dumped fsmap epoch 28 2024-04-12T21:55:40.644 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:40.642+0000 7fe778a00640 1 -- 172.21.15.18:0/1951141307 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fe75806eb70 msgr2=0x7fe758071020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:55:40.645 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:40.642+0000 7fe778a00640 1 --2- 172.21.15.18:0/1951141307 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fe75806eb70 0x7fe758071020 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7fe75c000c00 tx=0x7fe75c009da0 comp rx=0 tx=0).stop 2024-04-12T21:55:40.645 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:40.642+0000 7fe778a00640 1 -- 172.21.15.18:0/1951141307 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe7740643a0 msgr2=0x7fe774114d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:55:40.645 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:40.642+0000 7fe778a00640 1 --2- 172.21.15.18:0/1951141307 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe7740643a0 0x7fe774114d00 secure :-1 s=READY pgs=1413 cs=0 l=1 rev1=1 crypto rx=0x7fe76400c810 tx=0x7fe7640043c0 comp rx=0 tx=0).stop 2024-04-12T21:55:40.645 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:40.642+0000 7fe778a00640 1 -- 172.21.15.18:0/1951141307 shutdown_connections 2024-04-12T21:55:40.645 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:40.642+0000 7fe778a00640 1 --2- 172.21.15.18:0/1951141307 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fe75806eb70 0x7fe758071020 unknown :-1 s=CLOSED pgs=748 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:55:40.645 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:40.642+0000 7fe778a00640 1 --2- 172.21.15.18:0/1951141307 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe774102050 0x7fe774115240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:55:40.645 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:40.642+0000 7fe778a00640 1 --2- 172.21.15.18:0/1951141307 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe7740643a0 0x7fe774114d00 unknown :-1 s=CLOSED pgs=1413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:55:40.645 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:40.642+0000 7fe778a00640 1 -- 172.21.15.18:0/1951141307 >> 172.21.15.18:0/1951141307 conn(0x7fe774070f80 msgr2=0x7fe7740fc9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:55:40.645 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:40.642+0000 7fe778a00640 1 -- 172.21.15.18:0/1951141307 shutdown_connections 2024-04-12T21:55:40.645 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:40.643+0000 7fe778a00640 1 -- 172.21.15.18:0/1951141307 wait complete. 2024-04-12T21:55:40.661 DEBUG:tasks.fs:max_mds not reduced in epoch 28 as fs was failed for carrying out rapid multi-rank mds upgrade 2024-04-12T21:55:40.661 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 29 2024-04-12T21:55:40.867 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:40.864+0000 7ff301200640 1 -- 172.21.15.18:0/784510010 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff2fc102000 msgr2=0x7ff2fc1023d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:55:40.867 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:40.864+0000 7ff301200640 1 --2- 172.21.15.18:0/784510010 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff2fc102000 0x7ff2fc1023d0 secure :-1 s=READY pgs=1414 cs=0 l=1 rev1=1 crypto rx=0x7ff2ec009c50 tx=0x7ff2ec004d10 comp rx=0 tx=0).stop 2024-04-12T21:55:40.867 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:40.864+0000 7ff301200640 1 -- 172.21.15.18:0/784510010 shutdown_connections 2024-04-12T21:55:40.867 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:40.864+0000 7ff301200640 1 --2- 172.21.15.18:0/784510010 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff2fc1029a0 0x7ff2fc0fc030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:55:40.867 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:40.864+0000 7ff301200640 1 --2- 172.21.15.18:0/784510010 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff2fc102000 0x7ff2fc1023d0 unknown :-1 s=CLOSED pgs=1414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:55:40.868 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:40.864+0000 7ff301200640 1 -- 172.21.15.18:0/784510010 >> 172.21.15.18:0/784510010 conn(0x7ff2fc0f7b70 msgr2=0x7ff2fc0f9f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:55:40.868 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:40.864+0000 7ff301200640 1 -- 172.21.15.18:0/784510010 shutdown_connections 2024-04-12T21:55:40.868 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:40.864+0000 7ff301200640 1 -- 172.21.15.18:0/784510010 wait complete. 2024-04-12T21:55:40.868 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:40.865+0000 7ff301200640 1 Processor -- start 2024-04-12T21:55:40.868 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:40.865+0000 7ff301200640 1 -- start start 2024-04-12T21:55:40.870 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:40.865+0000 7ff301200640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff2fc102000 0x7ff2fc114cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:55:40.870 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:40.865+0000 7ff301200640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff2fc1029a0 0x7ff2fc115230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:55:40.870 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:40.865+0000 7ff301200640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2fc115830 con 0x7ff2fc1029a0 2024-04-12T21:55:40.870 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:40.865+0000 7ff301200640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2fc10ee80 con 0x7ff2fc102000 2024-04-12T21:55:40.870 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:40.866+0000 7ff2f2a00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff2fc1029a0 0x7ff2fc115230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:55:40.870 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:40.866+0000 7ff2f2a00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff2fc1029a0 0x7ff2fc115230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:43334/0 (socket says 172.21.15.18:43334) 2024-04-12T21:55:40.870 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:40.866+0000 7ff2f2a00640 1 -- 172.21.15.18:0/152108083 learned_addr learned my addr 172.21.15.18:0/152108083 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:55:40.870 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:40.866+0000 7ff2f2a00640 1 -- 172.21.15.18:0/152108083 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff2fc102000 msgr2=0x7ff2fc114cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:55:40.871 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:40.866+0000 7ff2f2a00640 1 --2- 172.21.15.18:0/152108083 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff2fc102000 0x7ff2fc114cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:55:40.871 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:40.866+0000 7ff2f2a00640 1 -- 172.21.15.18:0/152108083 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2ec0098b0 con 0x7ff2fc1029a0 2024-04-12T21:55:40.871 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:40.866+0000 7ff2f2a00640 1 --2- 172.21.15.18:0/152108083 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff2fc1029a0 0x7ff2fc115230 secure :-1 s=READY pgs=1415 cs=0 l=1 rev1=1 crypto rx=0x7ff2e4002a50 tx=0x7ff2e4004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:55:40.871 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:40.867+0000 7ff2f3e00640 1 -- 172.21.15.18:0/152108083 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff2e4007c30 con 0x7ff2fc1029a0 2024-04-12T21:55:40.871 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:40.867+0000 7ff301200640 1 -- 172.21.15.18:0/152108083 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff2fc10f160 con 0x7ff2fc1029a0 2024-04-12T21:55:40.871 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:40.867+0000 7ff2f3e00640 1 -- 172.21.15.18:0/152108083 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff2e400f040 con 0x7ff2fc1029a0 2024-04-12T21:55:40.871 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:40.867+0000 7ff2f3e00640 1 -- 172.21.15.18:0/152108083 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff2e400db50 con 0x7ff2fc1029a0 2024-04-12T21:55:40.871 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:40.867+0000 7ff301200640 1 -- 172.21.15.18:0/152108083 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff2fc10f700 con 0x7ff2fc1029a0 2024-04-12T21:55:40.872 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:40.868+0000 7ff301200640 1 -- 172.21.15.18:0/152108083 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff2fc0fd770 con 0x7ff2fc1029a0 2024-04-12T21:55:40.872 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:40.869+0000 7ff2f3e00640 1 -- 172.21.15.18:0/152108083 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7ff2e401c400 con 0x7ff2fc1029a0 2024-04-12T21:55:40.872 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:40.870+0000 7ff2f3e00640 1 --2- 172.21.15.18:0/152108083 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7ff2e006eb70 0x7ff2e0071020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:55:40.873 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:40.870+0000 7ff2f3e00640 1 -- 172.21.15.18:0/152108083 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 6757+0+0 (secure 0 0 0) 0x7ff2e408e340 con 0x7ff2fc1029a0 2024-04-12T21:55:40.873 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:40.871+0000 7ff2fb400640 1 --2- 172.21.15.18:0/152108083 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7ff2e006eb70 0x7ff2e0071020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:55:40.873 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:40.871+0000 7ff2fb400640 1 --2- 172.21.15.18:0/152108083 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7ff2e006eb70 0x7ff2e0071020 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7ff2ec005010 tx=0x7ff2ec004d10 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:55:40.878 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:40.876+0000 7ff2f3e00640 1 -- 172.21.15.18:0/152108083 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7ff2e4056f40 con 0x7ff2fc1029a0 2024-04-12T21:55:41.129 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:41.127+0000 7ff301200640 1 -- 172.21.15.18:0/152108083 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 29, "format": "json"} v 0) v1 -- 0x7ff2fc109190 con 0x7ff2fc1029a0 2024-04-12T21:55:41.131 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:41.128+0000 7ff2f3e00640 1 -- 172.21.15.18:0/152108083 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 29, "format": "json"}]=0 dumped fsmap epoch 29 v38) v1 ==== 107+0+4988 (secure 0 0 0) 0x7ff2e405a600 con 0x7ff2fc1029a0 2024-04-12T21:55:41.131 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-12T21:55:41.131 INFO:teuthology.orchestra.run.smithi018.stdout:{"epoch":29,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24277,"name":"cephfs.smithi186.wjnafs","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.186:6827/438347207","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6826","nonce":438347207},{"type":"v1","addr":"172.21.15.186:6827","nonce":438347207}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":18},{"gid":24281,"name":"cephfs.smithi186.jdllyn","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6825/1464359952","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6824","nonce":1464359952},{"type":"v1","addr":"172.21.15.186:6825","nonce":1464359952}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":16},{"gid":34436,"name":"cephfs.smithi018.acqbsc","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.18:6829/755179764","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6828","nonce":755179764},{"type":"v1","addr":"172.21.15.18:6829","nonce":755179764}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":27},{"gid":34468,"name":"cephfs.smithi018.szjluo","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.18:6827/2407992653","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6826","nonce":2407992653},{"type":"v1","addr":"172.21.15.18:6827","nonce":2407992653}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":29}],"filesystems":[{"mdsmap":{"epoch":24,"flags":19,"flags_state":{"joinable":false,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-12T20:51:57.362850+0000","modified":"2024-04-12T21:06:59.717190+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":100,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{},"failed":[0,1],"damaged":[],"stopped":[],"info":{},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-04-12T21:55:41.131 INFO:teuthology.orchestra.run.smithi018.stderr:dumped fsmap epoch 29 2024-04-12T21:55:41.134 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:41.132+0000 7ff301200640 1 -- 172.21.15.18:0/152108083 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7ff2e006eb70 msgr2=0x7ff2e0071020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:55:41.134 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:41.132+0000 7ff301200640 1 --2- 172.21.15.18:0/152108083 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7ff2e006eb70 0x7ff2e0071020 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7ff2ec005010 tx=0x7ff2ec004d10 comp rx=0 tx=0).stop 2024-04-12T21:55:41.134 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:41.132+0000 7ff301200640 1 -- 172.21.15.18:0/152108083 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff2fc1029a0 msgr2=0x7ff2fc115230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:55:41.134 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:41.132+0000 7ff301200640 1 --2- 172.21.15.18:0/152108083 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff2fc1029a0 0x7ff2fc115230 secure :-1 s=READY pgs=1415 cs=0 l=1 rev1=1 crypto rx=0x7ff2e4002a50 tx=0x7ff2e4004600 comp rx=0 tx=0).stop 2024-04-12T21:55:41.134 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:41.132+0000 7ff301200640 1 -- 172.21.15.18:0/152108083 shutdown_connections 2024-04-12T21:55:41.134 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:41.132+0000 7ff301200640 1 --2- 172.21.15.18:0/152108083 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7ff2e006eb70 0x7ff2e0071020 unknown :-1 s=CLOSED pgs=749 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:55:41.135 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:41.132+0000 7ff301200640 1 --2- 172.21.15.18:0/152108083 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff2fc1029a0 0x7ff2fc115230 unknown :-1 s=CLOSED pgs=1415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:55:41.135 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:41.132+0000 7ff301200640 1 --2- 172.21.15.18:0/152108083 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff2fc102000 0x7ff2fc114cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:55:41.135 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:41.132+0000 7ff301200640 1 -- 172.21.15.18:0/152108083 >> 172.21.15.18:0/152108083 conn(0x7ff2fc0f7b70 msgr2=0x7ff2fc071160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:55:41.135 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:41.132+0000 7ff301200640 1 -- 172.21.15.18:0/152108083 shutdown_connections 2024-04-12T21:55:41.135 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:41.132+0000 7ff301200640 1 -- 172.21.15.18:0/152108083 wait complete. 2024-04-12T21:55:41.152 DEBUG:tasks.fs:max_mds not reduced in epoch 29 as fs was failed for carrying out rapid multi-rank mds upgrade 2024-04-12T21:55:41.152 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 30 2024-04-12T21:55:41.372 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:41.369+0000 7fbc5a200640 1 -- 172.21.15.18:0/2300755276 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fbc54108f90 msgr2=0x7fbc54102650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:55:41.372 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:41.369+0000 7fbc5a200640 1 --2- 172.21.15.18:0/2300755276 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fbc54108f90 0x7fbc54102650 secure :-1 s=READY pgs=1416 cs=0 l=1 rev1=1 crypto rx=0x7fbc440048d0 tx=0x7fbc44009da0 comp rx=0 tx=0).stop 2024-04-12T21:55:41.372 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:41.369+0000 7fbc5a200640 1 -- 172.21.15.18:0/2300755276 shutdown_connections 2024-04-12T21:55:41.372 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:41.369+0000 7fbc5a200640 1 --2- 172.21.15.18:0/2300755276 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fbc54108f90 0x7fbc54102650 unknown :-1 s=CLOSED pgs=1416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:55:41.372 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:41.370+0000 7fbc5a200640 1 --2- 172.21.15.18:0/2300755276 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbc541085f0 0x7fbc541089c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:55:41.372 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:41.370+0000 7fbc5a200640 1 -- 172.21.15.18:0/2300755276 >> 172.21.15.18:0/2300755276 conn(0x7fbc54071e30 msgr2=0x7fbc54074240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:55:41.372 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:41.370+0000 7fbc5a200640 1 -- 172.21.15.18:0/2300755276 shutdown_connections 2024-04-12T21:55:41.373 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:41.370+0000 7fbc5a200640 1 -- 172.21.15.18:0/2300755276 wait complete. 2024-04-12T21:55:41.374 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:41.370+0000 7fbc5a200640 1 Processor -- start 2024-04-12T21:55:41.374 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:41.370+0000 7fbc5a200640 1 -- start start 2024-04-12T21:55:41.374 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:41.371+0000 7fbc5a200640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fbc541085f0 0x7fbc54119150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:55:41.374 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:41.371+0000 7fbc5a200640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbc54108f90 0x7fbc54119690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:55:41.374 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:41.371+0000 7fbc5a200640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc54119c90 con 0x7fbc541085f0 2024-04-12T21:55:41.374 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:41.371+0000 7fbc5a200640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc541132e0 con 0x7fbc54108f90 2024-04-12T21:55:41.374 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:41.371+0000 7fbc58e00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fbc541085f0 0x7fbc54119150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:55:41.374 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:41.371+0000 7fbc58e00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fbc541085f0 0x7fbc54119150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:43352/0 (socket says 172.21.15.18:43352) 2024-04-12T21:55:41.374 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:41.371+0000 7fbc58e00640 1 -- 172.21.15.18:0/60817913 learned_addr learned my addr 172.21.15.18:0/60817913 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:55:41.375 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:41.371+0000 7fbc58e00640 1 -- 172.21.15.18:0/60817913 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbc54108f90 msgr2=0x7fbc54119690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:55:41.375 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:41.371+0000 7fbc58e00640 1 --2- 172.21.15.18:0/60817913 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbc54108f90 0x7fbc54119690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:55:41.375 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:41.372+0000 7fbc58e00640 1 -- 172.21.15.18:0/60817913 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc44005040 con 0x7fbc541085f0 2024-04-12T21:55:41.375 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:41.372+0000 7fbc58e00640 1 --2- 172.21.15.18:0/60817913 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fbc541085f0 0x7fbc54119150 secure :-1 s=READY pgs=1417 cs=0 l=1 rev1=1 crypto rx=0x7fbc4800eb70 tx=0x7fbc4800b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:55:41.375 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:41.372+0000 7fbc50c00640 1 -- 172.21.15.18:0/60817913 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbc48007c30 con 0x7fbc541085f0 2024-04-12T21:55:41.375 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:41.372+0000 7fbc50c00640 1 -- 172.21.15.18:0/60817913 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fbc4800f040 con 0x7fbc541085f0 2024-04-12T21:55:41.376 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:41.372+0000 7fbc50c00640 1 -- 172.21.15.18:0/60817913 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbc480095b0 con 0x7fbc541085f0 2024-04-12T21:55:41.376 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:41.372+0000 7fbc5a200640 1 -- 172.21.15.18:0/60817913 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbc54113560 con 0x7fbc541085f0 2024-04-12T21:55:41.376 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:41.372+0000 7fbc5a200640 1 -- 172.21.15.18:0/60817913 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbc54113a80 con 0x7fbc541085f0 2024-04-12T21:55:41.376 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:41.373+0000 7fbc5a200640 1 -- 172.21.15.18:0/60817913 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbc54103d70 con 0x7fbc541085f0 2024-04-12T21:55:41.376 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:41.375+0000 7fbc50c00640 1 -- 172.21.15.18:0/60817913 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7fbc48009710 con 0x7fbc541085f0 2024-04-12T21:55:41.377 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:41.375+0000 7fbc50c00640 1 --2- 172.21.15.18:0/60817913 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fbc3c06eb70 0x7fbc3c071020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:55:41.378 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:41.376+0000 7fbc50c00640 1 -- 172.21.15.18:0/60817913 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 6757+0+0 (secure 0 0 0) 0x7fbc4808e640 con 0x7fbc541085f0 2024-04-12T21:55:41.378 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:41.376+0000 7fbc53e00640 1 --2- 172.21.15.18:0/60817913 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fbc3c06eb70 0x7fbc3c071020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:55:41.378 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:41.377+0000 7fbc53e00640 1 --2- 172.21.15.18:0/60817913 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fbc3c06eb70 0x7fbc3c071020 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7fbc44000c00 tx=0x7fbc44009da0 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:55:41.383 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:41.381+0000 7fbc50c00640 1 -- 172.21.15.18:0/60817913 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7fbc48057360 con 0x7fbc541085f0 2024-04-12T21:55:41.605 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:55:41 smithi018 ceph-mon[112590]: from='client.? 172.21.15.18:0/1951141307' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 28, "format": "json"}]: dispatch 2024-04-12T21:55:41.606 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:55:41 smithi018 ceph-mon[112590]: from='client.? 172.21.15.18:0/152108083' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 29, "format": "json"}]: dispatch 2024-04-12T21:55:41.631 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:41.629+0000 7fbc5a200640 1 -- 172.21.15.18:0/60817913 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 30, "format": "json"} v 0) v1 -- 0x7fbc5410f800 con 0x7fbc541085f0 2024-04-12T21:55:41.632 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:41.630+0000 7fbc50c00640 1 -- 172.21.15.18:0/60817913 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 30, "format": "json"}]=0 dumped fsmap epoch 30 v38) v1 ==== 107+0+4206 (secure 0 0 0) 0x7fbc4805aa20 con 0x7fbc541085f0 2024-04-12T21:55:41.633 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-12T21:55:41.633 INFO:teuthology.orchestra.run.smithi018.stdout:{"epoch":30,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24277,"name":"cephfs.smithi186.wjnafs","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.186:6827/438347207","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6826","nonce":438347207},{"type":"v1","addr":"172.21.15.186:6827","nonce":438347207}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":18},{"gid":34436,"name":"cephfs.smithi018.acqbsc","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.18:6829/755179764","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6828","nonce":755179764},{"type":"v1","addr":"172.21.15.18:6829","nonce":755179764}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":27},{"gid":34468,"name":"cephfs.smithi018.szjluo","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.18:6827/2407992653","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6826","nonce":2407992653},{"type":"v1","addr":"172.21.15.18:6827","nonce":2407992653}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":29}],"filesystems":[{"mdsmap":{"epoch":24,"flags":19,"flags_state":{"joinable":false,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-12T20:51:57.362850+0000","modified":"2024-04-12T21:06:59.717190+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":100,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{},"failed":[0,1],"damaged":[],"stopped":[],"info":{},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-04-12T21:55:41.633 INFO:teuthology.orchestra.run.smithi018.stderr:dumped fsmap epoch 30 2024-04-12T21:55:41.635 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:41.633+0000 7fbc5a200640 1 -- 172.21.15.18:0/60817913 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fbc3c06eb70 msgr2=0x7fbc3c071020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:55:41.635 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:41.633+0000 7fbc5a200640 1 --2- 172.21.15.18:0/60817913 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fbc3c06eb70 0x7fbc3c071020 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7fbc44000c00 tx=0x7fbc44009da0 comp rx=0 tx=0).stop 2024-04-12T21:55:41.636 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:41.633+0000 7fbc5a200640 1 -- 172.21.15.18:0/60817913 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fbc541085f0 msgr2=0x7fbc54119150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:55:41.636 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:41.633+0000 7fbc5a200640 1 --2- 172.21.15.18:0/60817913 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fbc541085f0 0x7fbc54119150 secure :-1 s=READY pgs=1417 cs=0 l=1 rev1=1 crypto rx=0x7fbc4800eb70 tx=0x7fbc4800b040 comp rx=0 tx=0).stop 2024-04-12T21:55:41.636 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:41.634+0000 7fbc5a200640 1 -- 172.21.15.18:0/60817913 shutdown_connections 2024-04-12T21:55:41.636 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:41.634+0000 7fbc5a200640 1 --2- 172.21.15.18:0/60817913 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fbc3c06eb70 0x7fbc3c071020 unknown :-1 s=CLOSED pgs=750 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:55:41.636 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:41.634+0000 7fbc5a200640 1 --2- 172.21.15.18:0/60817913 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbc54108f90 0x7fbc54119690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:55:41.636 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:41.634+0000 7fbc5a200640 1 --2- 172.21.15.18:0/60817913 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fbc541085f0 0x7fbc54119150 unknown :-1 s=CLOSED pgs=1417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:55:41.636 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:41.634+0000 7fbc5a200640 1 -- 172.21.15.18:0/60817913 >> 172.21.15.18:0/60817913 conn(0x7fbc54071e30 msgr2=0x7fbc54073d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:55:41.636 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:41.634+0000 7fbc5a200640 1 -- 172.21.15.18:0/60817913 shutdown_connections 2024-04-12T21:55:41.637 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:41.634+0000 7fbc5a200640 1 -- 172.21.15.18:0/60817913 wait complete. 2024-04-12T21:55:41.653 DEBUG:tasks.fs:max_mds not reduced in epoch 30 as fs was failed for carrying out rapid multi-rank mds upgrade 2024-04-12T21:55:41.653 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 31 2024-04-12T21:55:41.785 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:55:41 smithi186 ceph-mon[86379]: from='client.? 172.21.15.18:0/1951141307' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 28, "format": "json"}]: dispatch 2024-04-12T21:55:41.785 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:55:41 smithi186 ceph-mon[86379]: from='client.? 172.21.15.18:0/152108083' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 29, "format": "json"}]: dispatch 2024-04-12T21:55:41.855 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:41.853+0000 7f2488e00640 1 -- 172.21.15.18:0/2617781941 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2484072310 msgr2=0x7f248410cb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:55:41.855 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:41.853+0000 7f2488e00640 1 --2- 172.21.15.18:0/2617781941 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2484072310 0x7f248410cb70 secure :-1 s=READY pgs=1418 cs=0 l=1 rev1=1 crypto rx=0x7f246c0048d0 tx=0x7f246c009da0 comp rx=0 tx=0).stop 2024-04-12T21:55:41.855 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:41.853+0000 7f2488e00640 1 -- 172.21.15.18:0/2617781941 shutdown_connections 2024-04-12T21:55:41.856 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:41.853+0000 7f2488e00640 1 --2- 172.21.15.18:0/2617781941 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2484072310 0x7f248410cb70 unknown :-1 s=CLOSED pgs=1418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:55:41.856 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:41.853+0000 7f2488e00640 1 --2- 172.21.15.18:0/2617781941 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2484071a00 0x7f2484071dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:55:41.856 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:41.853+0000 7f2488e00640 1 -- 172.21.15.18:0/2617781941 >> 172.21.15.18:0/2617781941 conn(0x7f24840f9ff0 msgr2=0x7f24840fc400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:55:41.856 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:41.853+0000 7f2488e00640 1 -- 172.21.15.18:0/2617781941 shutdown_connections 2024-04-12T21:55:41.856 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:41.853+0000 7f2488e00640 1 -- 172.21.15.18:0/2617781941 wait complete. 2024-04-12T21:55:41.856 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:41.854+0000 7f2488e00640 1 Processor -- start 2024-04-12T21:55:41.856 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:41.854+0000 7f2488e00640 1 -- start start 2024-04-12T21:55:41.857 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:41.855+0000 7f2488e00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2484071a00 0x7f24841192d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:55:41.857 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:41.855+0000 7f2488e00640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2484072310 0x7f2484119810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:55:41.857 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:41.855+0000 7f2488e00640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2484119e10 con 0x7f2484071a00 2024-04-12T21:55:41.857 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:41.855+0000 7f2488e00640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2484113460 con 0x7f2484072310 2024-04-12T21:55:41.857 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:41.855+0000 7f2481e00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2484071a00 0x7f24841192d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:55:41.857 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:41.855+0000 7f2481e00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2484071a00 0x7f24841192d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:43372/0 (socket says 172.21.15.18:43372) 2024-04-12T21:55:41.858 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:41.855+0000 7f2481e00640 1 -- 172.21.15.18:0/927739846 learned_addr learned my addr 172.21.15.18:0/927739846 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:55:41.859 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:41.855+0000 7f247be00640 1 --2- 172.21.15.18:0/927739846 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2484072310 0x7f2484119810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:55:41.859 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:41.856+0000 7f2481e00640 1 -- 172.21.15.18:0/927739846 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2484072310 msgr2=0x7f2484119810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:55:41.859 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:41.856+0000 7f2481e00640 1 --2- 172.21.15.18:0/927739846 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2484072310 0x7f2484119810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:55:41.859 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:41.856+0000 7f2481e00640 1 -- 172.21.15.18:0/927739846 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f246c005040 con 0x7f2484071a00 2024-04-12T21:55:41.859 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:41.856+0000 7f2481e00640 1 --2- 172.21.15.18:0/927739846 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2484071a00 0x7f24841192d0 secure :-1 s=READY pgs=1419 cs=0 l=1 rev1=1 crypto rx=0x7f2474002a50 tx=0x7f2474004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:55:41.859 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:41.856+0000 7f247a000640 1 -- 172.21.15.18:0/927739846 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2474007c30 con 0x7f2484071a00 2024-04-12T21:55:41.859 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:41.856+0000 7f247a000640 1 -- 172.21.15.18:0/927739846 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f247400f040 con 0x7f2484071a00 2024-04-12T21:55:41.859 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:41.856+0000 7f247a000640 1 -- 172.21.15.18:0/927739846 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f247400dad0 con 0x7f2484071a00 2024-04-12T21:55:41.859 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:41.856+0000 7f2488e00640 1 -- 172.21.15.18:0/927739846 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2484113740 con 0x7f2484071a00 2024-04-12T21:55:41.859 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:41.856+0000 7f2488e00640 1 -- 172.21.15.18:0/927739846 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2484113c30 con 0x7f2484071a00 2024-04-12T21:55:41.859 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:41.857+0000 7f2488e00640 1 -- 172.21.15.18:0/927739846 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2484073140 con 0x7f2484071a00 2024-04-12T21:55:41.861 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:41.859+0000 7f247a000640 1 -- 172.21.15.18:0/927739846 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7f247400dc70 con 0x7f2484071a00 2024-04-12T21:55:41.862 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:41.860+0000 7f247a000640 1 --2- 172.21.15.18:0/927739846 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f246806eb70 0x7f2468071020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:55:41.862 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:41.860+0000 7f247a000640 1 -- 172.21.15.18:0/927739846 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 6757+0+0 (secure 0 0 0) 0x7f247408e150 con 0x7f2484071a00 2024-04-12T21:55:41.862 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:41.860+0000 7f247be00640 1 --2- 172.21.15.18:0/927739846 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f246806eb70 0x7f2468071020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:55:41.863 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:41.861+0000 7f247be00640 1 --2- 172.21.15.18:0/927739846 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f246806eb70 0x7f2468071020 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7f246c0048a0 tx=0x7f246c009da0 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:55:41.867 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:41.865+0000 7f247a000640 1 -- 172.21.15.18:0/927739846 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f2474056e70 con 0x7f2484071a00 2024-04-12T21:55:42.112 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:42.110+0000 7f2488e00640 1 -- 172.21.15.18:0/927739846 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 31, "format": "json"} v 0) v1 -- 0x7f24841149d0 con 0x7f2484071a00 2024-04-12T21:55:42.114 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:42.111+0000 7f247a000640 1 -- 172.21.15.18:0/927739846 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 31, "format": "json"}]=0 dumped fsmap epoch 31 v38) v1 ==== 107+0+5056 (secure 0 0 0) 0x7f247405a530 con 0x7f2484071a00 2024-04-12T21:55:42.114 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-12T21:55:42.114 INFO:teuthology.orchestra.run.smithi018.stdout:{"epoch":31,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24277,"name":"cephfs.smithi186.wjnafs","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.186:6827/438347207","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6826","nonce":438347207},{"type":"v1","addr":"172.21.15.186:6827","nonce":438347207}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":18},{"gid":34436,"name":"cephfs.smithi018.acqbsc","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.18:6829/755179764","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6828","nonce":755179764},{"type":"v1","addr":"172.21.15.18:6829","nonce":755179764}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":27},{"gid":34468,"name":"cephfs.smithi018.szjluo","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.18:6827/2407992653","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6826","nonce":2407992653},{"type":"v1","addr":"172.21.15.18:6827","nonce":2407992653}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":29},{"gid":44595,"name":"cephfs.smithi186.jdllyn","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6825/3523444202","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6824","nonce":3523444202},{"type":"v1","addr":"172.21.15.186:6825","nonce":3523444202}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":31}],"filesystems":[{"mdsmap":{"epoch":24,"flags":19,"flags_state":{"joinable":false,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-12T20:51:57.362850+0000","modified":"2024-04-12T21:06:59.717190+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":100,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{},"failed":[0,1],"damaged":[],"stopped":[],"info":{},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-04-12T21:55:42.114 INFO:teuthology.orchestra.run.smithi018.stderr:dumped fsmap epoch 31 2024-04-12T21:55:42.117 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:42.115+0000 7f2488e00640 1 -- 172.21.15.18:0/927739846 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f246806eb70 msgr2=0x7f2468071020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:55:42.117 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:42.115+0000 7f2488e00640 1 --2- 172.21.15.18:0/927739846 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f246806eb70 0x7f2468071020 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7f246c0048a0 tx=0x7f246c009da0 comp rx=0 tx=0).stop 2024-04-12T21:55:42.117 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:42.115+0000 7f2488e00640 1 -- 172.21.15.18:0/927739846 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2484071a00 msgr2=0x7f24841192d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:55:42.117 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:42.115+0000 7f2488e00640 1 --2- 172.21.15.18:0/927739846 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2484071a00 0x7f24841192d0 secure :-1 s=READY pgs=1419 cs=0 l=1 rev1=1 crypto rx=0x7f2474002a50 tx=0x7f2474004600 comp rx=0 tx=0).stop 2024-04-12T21:55:42.117 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:42.115+0000 7f2488e00640 1 -- 172.21.15.18:0/927739846 shutdown_connections 2024-04-12T21:55:42.117 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:42.115+0000 7f2488e00640 1 --2- 172.21.15.18:0/927739846 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f246806eb70 0x7f2468071020 unknown :-1 s=CLOSED pgs=751 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:55:42.117 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:42.115+0000 7f2488e00640 1 --2- 172.21.15.18:0/927739846 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2484072310 0x7f2484119810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:55:42.117 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:42.115+0000 7f2488e00640 1 --2- 172.21.15.18:0/927739846 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2484071a00 0x7f24841192d0 unknown :-1 s=CLOSED pgs=1419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:55:42.117 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:42.115+0000 7f2488e00640 1 -- 172.21.15.18:0/927739846 >> 172.21.15.18:0/927739846 conn(0x7f24840f9ff0 msgr2=0x7f2484106b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:55:42.117 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:42.115+0000 7f2488e00640 1 -- 172.21.15.18:0/927739846 shutdown_connections 2024-04-12T21:55:42.117 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:42.115+0000 7f2488e00640 1 -- 172.21.15.18:0/927739846 wait complete. 2024-04-12T21:55:42.136 DEBUG:tasks.fs:max_mds not reduced in epoch 31 as fs was failed for carrying out rapid multi-rank mds upgrade 2024-04-12T21:55:42.136 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 32 2024-04-12T21:55:42.360 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:42.358+0000 7fbc54a00640 1 -- 172.21.15.18:0/529678334 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fbc50104bb0 msgr2=0x7fbc500fe240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:55:42.360 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:42.358+0000 7fbc54a00640 1 --2- 172.21.15.18:0/529678334 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fbc50104bb0 0x7fbc500fe240 secure :-1 s=READY pgs=1420 cs=0 l=1 rev1=1 crypto rx=0x7fbc380048d0 tx=0x7fbc38009da0 comp rx=0 tx=0).stop 2024-04-12T21:55:42.360 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:42.358+0000 7fbc54a00640 1 -- 172.21.15.18:0/529678334 shutdown_connections 2024-04-12T21:55:42.360 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:42.358+0000 7fbc54a00640 1 --2- 172.21.15.18:0/529678334 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fbc50104bb0 0x7fbc500fe240 unknown :-1 s=CLOSED pgs=1420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:55:42.360 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:42.358+0000 7fbc54a00640 1 --2- 172.21.15.18:0/529678334 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbc50104210 0x7fbc501045e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:55:42.360 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:42.358+0000 7fbc54a00640 1 -- 172.21.15.18:0/529678334 >> 172.21.15.18:0/529678334 conn(0x7fbc500f9fc0 msgr2=0x7fbc500fc3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:55:42.360 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:42.358+0000 7fbc54a00640 1 -- 172.21.15.18:0/529678334 shutdown_connections 2024-04-12T21:55:42.360 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:42.358+0000 7fbc54a00640 1 -- 172.21.15.18:0/529678334 wait complete. 2024-04-12T21:55:42.362 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:42.358+0000 7fbc54a00640 1 Processor -- start 2024-04-12T21:55:42.362 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:42.359+0000 7fbc54a00640 1 -- start start 2024-04-12T21:55:42.362 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:42.359+0000 7fbc54a00640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbc50104210 0x7fbc50114d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:55:42.362 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:42.359+0000 7fbc54a00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fbc50104bb0 0x7fbc50115290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:55:42.362 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:42.359+0000 7fbc54a00640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc50115890 con 0x7fbc50104bb0 2024-04-12T21:55:42.362 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:42.359+0000 7fbc54a00640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc5010eee0 con 0x7fbc50104210 2024-04-12T21:55:42.362 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:42.359+0000 7fbc47e00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fbc50104bb0 0x7fbc50115290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:55:42.362 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:42.359+0000 7fbc47e00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fbc50104bb0 0x7fbc50115290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:43388/0 (socket says 172.21.15.18:43388) 2024-04-12T21:55:42.363 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:42.359+0000 7fbc47e00640 1 -- 172.21.15.18:0/4053844814 learned_addr learned my addr 172.21.15.18:0/4053844814 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:55:42.363 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:42.360+0000 7fbc4e800640 1 --2- 172.21.15.18:0/4053844814 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbc50104210 0x7fbc50114d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:55:42.363 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:42.360+0000 7fbc47e00640 1 -- 172.21.15.18:0/4053844814 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbc50104210 msgr2=0x7fbc50114d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:55:42.363 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:42.360+0000 7fbc47e00640 1 --2- 172.21.15.18:0/4053844814 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbc50104210 0x7fbc50114d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:55:42.363 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:42.360+0000 7fbc47e00640 1 -- 172.21.15.18:0/4053844814 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc38005040 con 0x7fbc50104bb0 2024-04-12T21:55:42.363 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:42.360+0000 7fbc4e800640 1 --2- 172.21.15.18:0/4053844814 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbc50104210 0x7fbc50114d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-12T21:55:42.363 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:42.360+0000 7fbc47e00640 1 --2- 172.21.15.18:0/4053844814 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fbc50104bb0 0x7fbc50115290 secure :-1 s=READY pgs=1421 cs=0 l=1 rev1=1 crypto rx=0x7fbc380047f0 tx=0x7fbc38009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:55:42.364 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:42.360+0000 7fbc46a00640 1 -- 172.21.15.18:0/4053844814 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbc380094c0 con 0x7fbc50104bb0 2024-04-12T21:55:42.364 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:42.360+0000 7fbc46a00640 1 -- 172.21.15.18:0/4053844814 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fbc38032e50 con 0x7fbc50104bb0 2024-04-12T21:55:42.364 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:42.360+0000 7fbc46a00640 1 -- 172.21.15.18:0/4053844814 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbc380433b0 con 0x7fbc50104bb0 2024-04-12T21:55:42.364 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:42.360+0000 7fbc54a00640 1 -- 172.21.15.18:0/4053844814 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbc5010f160 con 0x7fbc50104bb0 2024-04-12T21:55:42.364 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:42.361+0000 7fbc54a00640 1 -- 172.21.15.18:0/4053844814 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbc5010f620 con 0x7fbc50104bb0 2024-04-12T21:55:42.364 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:42.362+0000 7fbc54a00640 1 -- 172.21.15.18:0/4053844814 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbc500ff980 con 0x7fbc50104bb0 2024-04-12T21:55:42.365 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:42.363+0000 7fbc46a00640 1 -- 172.21.15.18:0/4053844814 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7fbc3802f070 con 0x7fbc50104bb0 2024-04-12T21:55:42.366 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:42.365+0000 7fbc46a00640 1 --2- 172.21.15.18:0/4053844814 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fbc3406eb70 0x7fbc34071020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:55:42.366 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:42.365+0000 7fbc46a00640 1 -- 172.21.15.18:0/4053844814 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 6757+0+0 (secure 0 0 0) 0x7fbc380b08c0 con 0x7fbc50104bb0 2024-04-12T21:55:42.366 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:42.365+0000 7fbc4e800640 1 --2- 172.21.15.18:0/4053844814 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fbc3406eb70 0x7fbc34071020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:55:42.366 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:42.365+0000 7fbc4e800640 1 --2- 172.21.15.18:0/4053844814 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fbc3406eb70 0x7fbc34071020 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7fbc40004d80 tx=0x7fbc4000b040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:55:42.371 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:42.369+0000 7fbc46a00640 1 -- 172.21.15.18:0/4053844814 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7fbc380794c0 con 0x7fbc50104bb0 2024-04-12T21:55:42.606 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:55:42 smithi018 ceph-mon[112590]: from='client.? 172.21.15.18:0/60817913' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 30, "format": "json"}]: dispatch 2024-04-12T21:55:42.606 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:55:42 smithi018 ceph-mon[112590]: from='client.? 172.21.15.18:0/927739846' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 31, "format": "json"}]: dispatch 2024-04-12T21:55:42.606 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:55:42 smithi018 ceph-mon[112590]: pgmap v1683: 65 pgs: 65 active+clean; 257 MiB data, 976 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:55:42.618 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:42.616+0000 7fbc54a00640 1 -- 172.21.15.18:0/4053844814 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 32, "format": "json"} v 0) v1 -- 0x7fbc5004bcc0 con 0x7fbc50104bb0 2024-04-12T21:55:42.619 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:42.617+0000 7fbc46a00640 1 -- 172.21.15.18:0/4053844814 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 32, "format": "json"}]=0 dumped fsmap epoch 32 v38) v1 ==== 107+0+4277 (secure 0 0 0) 0x7fbc3807cb80 con 0x7fbc50104bb0 2024-04-12T21:55:42.619 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-12T21:55:42.620 INFO:teuthology.orchestra.run.smithi018.stdout:{"epoch":32,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":34436,"name":"cephfs.smithi018.acqbsc","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.18:6829/755179764","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6828","nonce":755179764},{"type":"v1","addr":"172.21.15.18:6829","nonce":755179764}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":27},{"gid":34468,"name":"cephfs.smithi018.szjluo","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.18:6827/2407992653","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6826","nonce":2407992653},{"type":"v1","addr":"172.21.15.18:6827","nonce":2407992653}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":29},{"gid":44595,"name":"cephfs.smithi186.jdllyn","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6825/3523444202","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6824","nonce":3523444202},{"type":"v1","addr":"172.21.15.186:6825","nonce":3523444202}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":31}],"filesystems":[{"mdsmap":{"epoch":24,"flags":19,"flags_state":{"joinable":false,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-12T20:51:57.362850+0000","modified":"2024-04-12T21:06:59.717190+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":100,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{},"failed":[0,1],"damaged":[],"stopped":[],"info":{},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-04-12T21:55:42.620 INFO:teuthology.orchestra.run.smithi018.stderr:dumped fsmap epoch 32 2024-04-12T21:55:42.622 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:42.621+0000 7fbc54a00640 1 -- 172.21.15.18:0/4053844814 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fbc3406eb70 msgr2=0x7fbc34071020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:55:42.622 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:42.621+0000 7fbc54a00640 1 --2- 172.21.15.18:0/4053844814 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fbc3406eb70 0x7fbc34071020 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7fbc40004d80 tx=0x7fbc4000b040 comp rx=0 tx=0).stop 2024-04-12T21:55:42.622 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:42.621+0000 7fbc54a00640 1 -- 172.21.15.18:0/4053844814 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fbc50104bb0 msgr2=0x7fbc50115290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:55:42.622 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:42.621+0000 7fbc54a00640 1 --2- 172.21.15.18:0/4053844814 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fbc50104bb0 0x7fbc50115290 secure :-1 s=READY pgs=1421 cs=0 l=1 rev1=1 crypto rx=0x7fbc380047f0 tx=0x7fbc38009da0 comp rx=0 tx=0).stop 2024-04-12T21:55:42.623 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:42.621+0000 7fbc54a00640 1 -- 172.21.15.18:0/4053844814 shutdown_connections 2024-04-12T21:55:42.623 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:42.621+0000 7fbc54a00640 1 --2- 172.21.15.18:0/4053844814 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fbc3406eb70 0x7fbc34071020 unknown :-1 s=CLOSED pgs=752 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:55:42.623 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:42.621+0000 7fbc54a00640 1 --2- 172.21.15.18:0/4053844814 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fbc50104bb0 0x7fbc50115290 unknown :-1 s=CLOSED pgs=1421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:55:42.623 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:42.621+0000 7fbc54a00640 1 --2- 172.21.15.18:0/4053844814 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbc50104210 0x7fbc50114d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:55:42.623 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:42.621+0000 7fbc54a00640 1 -- 172.21.15.18:0/4053844814 >> 172.21.15.18:0/4053844814 conn(0x7fbc500f9fc0 msgr2=0x7fbc500fbf40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:55:42.623 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:42.621+0000 7fbc54a00640 1 -- 172.21.15.18:0/4053844814 shutdown_connections 2024-04-12T21:55:42.623 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:42.621+0000 7fbc54a00640 1 -- 172.21.15.18:0/4053844814 wait complete. 2024-04-12T21:55:42.640 DEBUG:tasks.fs:max_mds not reduced in epoch 32 as fs was failed for carrying out rapid multi-rank mds upgrade 2024-04-12T21:55:42.640 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 33 2024-04-12T21:55:42.785 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:55:42 smithi186 ceph-mon[86379]: from='client.? 172.21.15.18:0/60817913' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 30, "format": "json"}]: dispatch 2024-04-12T21:55:42.785 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:55:42 smithi186 ceph-mon[86379]: from='client.? 172.21.15.18:0/927739846' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 31, "format": "json"}]: dispatch 2024-04-12T21:55:42.785 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:55:42 smithi186 ceph-mon[86379]: pgmap v1683: 65 pgs: 65 active+clean; 257 MiB data, 976 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:55:42.857 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:42.851+0000 7f7197e00640 1 -- 172.21.15.18:0/4265429789 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f71981029a0 msgr2=0x7f71980fc030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:55:42.857 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:42.851+0000 7f7197e00640 1 --2- 172.21.15.18:0/4265429789 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f71981029a0 0x7f71980fc030 secure :-1 s=READY pgs=1422 cs=0 l=1 rev1=1 crypto rx=0x7f71840048d0 tx=0x7f7184009da0 comp rx=0 tx=0).stop 2024-04-12T21:55:42.858 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:42.851+0000 7f7197e00640 1 -- 172.21.15.18:0/4265429789 shutdown_connections 2024-04-12T21:55:42.858 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:42.851+0000 7f7197e00640 1 --2- 172.21.15.18:0/4265429789 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f71981029a0 0x7f71980fc030 unknown :-1 s=CLOSED pgs=1422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:55:42.858 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:42.851+0000 7f7197e00640 1 --2- 172.21.15.18:0/4265429789 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7198102000 0x7f71981023d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:55:42.858 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:42.851+0000 7f7197e00640 1 -- 172.21.15.18:0/4265429789 >> 172.21.15.18:0/4265429789 conn(0x7f71980f7b70 msgr2=0x7f71980f9f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:55:42.858 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:42.851+0000 7f7197e00640 1 -- 172.21.15.18:0/4265429789 shutdown_connections 2024-04-12T21:55:42.859 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:42.851+0000 7f7197e00640 1 -- 172.21.15.18:0/4265429789 wait complete. 2024-04-12T21:55:42.859 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:42.852+0000 7f7197e00640 1 Processor -- start 2024-04-12T21:55:42.859 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:42.852+0000 7f7197e00640 1 -- start start 2024-04-12T21:55:42.859 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:42.852+0000 7f7197e00640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7198102000 0x7f7198114cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:55:42.859 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:42.852+0000 7f7197e00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f71981029a0 0x7f7198115230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:55:42.859 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:42.852+0000 7f7197e00640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7198115830 con 0x7f71981029a0 2024-04-12T21:55:42.859 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:42.852+0000 7f7197e00640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f719810ee80 con 0x7f7198102000 2024-04-12T21:55:42.860 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:42.853+0000 7f7196000640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f71981029a0 0x7f7198115230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:55:42.860 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:42.853+0000 7f7196000640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f71981029a0 0x7f7198115230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:43396/0 (socket says 172.21.15.18:43396) 2024-04-12T21:55:42.860 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:42.853+0000 7f7196000640 1 -- 172.21.15.18:0/1612377175 learned_addr learned my addr 172.21.15.18:0/1612377175 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:55:42.860 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:42.853+0000 7f7196000640 1 -- 172.21.15.18:0/1612377175 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7198102000 msgr2=0x7f7198114cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:55:42.860 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:42.853+0000 7f7196000640 1 --2- 172.21.15.18:0/1612377175 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7198102000 0x7f7198114cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:55:42.860 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:42.853+0000 7f7196000640 1 -- 172.21.15.18:0/1612377175 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7184005040 con 0x7f71981029a0 2024-04-12T21:55:42.861 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:42.853+0000 7f7196000640 1 --2- 172.21.15.18:0/1612377175 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f71981029a0 0x7f7198115230 secure :-1 s=READY pgs=1423 cs=0 l=1 rev1=1 crypto rx=0x7f71840047f0 tx=0x7f7184009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:55:42.861 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:42.854+0000 7f718ea00640 1 -- 172.21.15.18:0/1612377175 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f71840094c0 con 0x7f71981029a0 2024-04-12T21:55:42.861 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:42.854+0000 7f718ea00640 1 -- 172.21.15.18:0/1612377175 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f7184032e50 con 0x7f71981029a0 2024-04-12T21:55:42.861 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:42.854+0000 7f7197e00640 1 -- 172.21.15.18:0/1612377175 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f719810f100 con 0x7f71981029a0 2024-04-12T21:55:42.861 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:42.854+0000 7f718ea00640 1 -- 172.21.15.18:0/1612377175 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f71840435f0 con 0x7f71981029a0 2024-04-12T21:55:42.861 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:42.854+0000 7f7197e00640 1 -- 172.21.15.18:0/1612377175 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f719810f5c0 con 0x7f71981029a0 2024-04-12T21:55:42.864 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:42.855+0000 7f7197e00640 1 -- 172.21.15.18:0/1612377175 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f71980fd770 con 0x7f71981029a0 2024-04-12T21:55:42.864 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:42.856+0000 7f718ea00640 1 -- 172.21.15.18:0/1612377175 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7f718402f070 con 0x7f71981029a0 2024-04-12T21:55:42.864 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:42.857+0000 7f718ea00640 1 --2- 172.21.15.18:0/1612377175 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f717c06eb70 0x7f717c071020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:55:42.864 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:42.857+0000 7f718ea00640 1 -- 172.21.15.18:0/1612377175 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 6757+0+0 (secure 0 0 0) 0x7f71840b0920 con 0x7f71981029a0 2024-04-12T21:55:42.864 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:42.857+0000 7f7196a00640 1 --2- 172.21.15.18:0/1612377175 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f717c06eb70 0x7f717c071020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:55:42.865 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:42.858+0000 7f7196a00640 1 --2- 172.21.15.18:0/1612377175 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f717c06eb70 0x7f717c071020 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7f7188004af0 tx=0x7f7188007c30 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:55:42.865 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:42.862+0000 7f718ea00640 1 -- 172.21.15.18:0/1612377175 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f71840795c0 con 0x7f71981029a0 2024-04-12T21:55:43.104 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:43.101+0000 7f7197e00640 1 -- 172.21.15.18:0/1612377175 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 33, "format": "json"} v 0) v1 -- 0x7f7198109140 con 0x7f71981029a0 2024-04-12T21:55:43.105 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:43.103+0000 7f718ea00640 1 -- 172.21.15.18:0/1612377175 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 33, "format": "json"}]=0 dumped fsmap epoch 33 v38) v1 ==== 107+0+5127 (secure 0 0 0) 0x7f718407cc80 con 0x7f71981029a0 2024-04-12T21:55:43.105 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-12T21:55:43.105 INFO:teuthology.orchestra.run.smithi018.stdout:{"epoch":33,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":34436,"name":"cephfs.smithi018.acqbsc","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.18:6829/755179764","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6828","nonce":755179764},{"type":"v1","addr":"172.21.15.18:6829","nonce":755179764}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":27},{"gid":34468,"name":"cephfs.smithi018.szjluo","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.18:6827/2407992653","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6826","nonce":2407992653},{"type":"v1","addr":"172.21.15.18:6827","nonce":2407992653}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":29},{"gid":44595,"name":"cephfs.smithi186.jdllyn","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6825/3523444202","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6824","nonce":3523444202},{"type":"v1","addr":"172.21.15.186:6825","nonce":3523444202}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":31},{"gid":44875,"name":"cephfs.smithi186.wjnafs","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6827/3122443766","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6826","nonce":3122443766},{"type":"v1","addr":"172.21.15.186:6827","nonce":3122443766}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":33}],"filesystems":[{"mdsmap":{"epoch":24,"flags":19,"flags_state":{"joinable":false,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-12T20:51:57.362850+0000","modified":"2024-04-12T21:06:59.717190+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":100,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{},"failed":[0,1],"damaged":[],"stopped":[],"info":{},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-04-12T21:55:43.106 INFO:teuthology.orchestra.run.smithi018.stderr:dumped fsmap epoch 33 2024-04-12T21:55:43.109 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:43.106+0000 7f7197e00640 1 -- 172.21.15.18:0/1612377175 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f717c06eb70 msgr2=0x7f717c071020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:55:43.109 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:43.106+0000 7f7197e00640 1 --2- 172.21.15.18:0/1612377175 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f717c06eb70 0x7f717c071020 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7f7188004af0 tx=0x7f7188007c30 comp rx=0 tx=0).stop 2024-04-12T21:55:43.109 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:43.106+0000 7f7197e00640 1 -- 172.21.15.18:0/1612377175 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f71981029a0 msgr2=0x7f7198115230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:55:43.109 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:43.106+0000 7f7197e00640 1 --2- 172.21.15.18:0/1612377175 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f71981029a0 0x7f7198115230 secure :-1 s=READY pgs=1423 cs=0 l=1 rev1=1 crypto rx=0x7f71840047f0 tx=0x7f7184009da0 comp rx=0 tx=0).stop 2024-04-12T21:55:43.109 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:43.107+0000 7f7197e00640 1 -- 172.21.15.18:0/1612377175 shutdown_connections 2024-04-12T21:55:43.109 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:43.107+0000 7f7197e00640 1 --2- 172.21.15.18:0/1612377175 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f717c06eb70 0x7f717c071020 unknown :-1 s=CLOSED pgs=753 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:55:43.109 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:43.107+0000 7f7197e00640 1 --2- 172.21.15.18:0/1612377175 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f71981029a0 0x7f7198115230 unknown :-1 s=CLOSED pgs=1423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:55:43.110 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:43.107+0000 7f7197e00640 1 --2- 172.21.15.18:0/1612377175 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7198102000 0x7f7198114cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:55:43.110 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:43.107+0000 7f7197e00640 1 -- 172.21.15.18:0/1612377175 >> 172.21.15.18:0/1612377175 conn(0x7f71980f7b70 msgr2=0x7f71980711d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:55:43.110 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:43.107+0000 7f7197e00640 1 -- 172.21.15.18:0/1612377175 shutdown_connections 2024-04-12T21:55:43.110 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:43.107+0000 7f7197e00640 1 -- 172.21.15.18:0/1612377175 wait complete. 2024-04-12T21:55:43.128 DEBUG:tasks.fs:max_mds not reduced in epoch 33 as fs was failed for carrying out rapid multi-rank mds upgrade 2024-04-12T21:55:43.128 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 34 2024-04-12T21:55:43.341 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:43.340+0000 7f5b59a00640 1 -- 172.21.15.18:0/435510569 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5b54108f90 msgr2=0x7f5b54102650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:55:43.342 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:43.340+0000 7f5b59a00640 1 --2- 172.21.15.18:0/435510569 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5b54108f90 0x7f5b54102650 secure :-1 s=READY pgs=1424 cs=0 l=1 rev1=1 crypto rx=0x7f5b3c0048d0 tx=0x7f5b3c009da0 comp rx=0 tx=0).stop 2024-04-12T21:55:43.342 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:43.340+0000 7f5b59a00640 1 -- 172.21.15.18:0/435510569 shutdown_connections 2024-04-12T21:55:43.342 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:43.340+0000 7f5b59a00640 1 --2- 172.21.15.18:0/435510569 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5b54108f90 0x7f5b54102650 unknown :-1 s=CLOSED pgs=1424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:55:43.342 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:43.340+0000 7f5b59a00640 1 --2- 172.21.15.18:0/435510569 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5b541085f0 0x7f5b541089c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:55:43.342 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:43.340+0000 7f5b59a00640 1 -- 172.21.15.18:0/435510569 >> 172.21.15.18:0/435510569 conn(0x7f5b54071e30 msgr2=0x7f5b54074240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:55:43.342 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:43.340+0000 7f5b59a00640 1 -- 172.21.15.18:0/435510569 shutdown_connections 2024-04-12T21:55:43.342 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:43.340+0000 7f5b59a00640 1 -- 172.21.15.18:0/435510569 wait complete. 2024-04-12T21:55:43.344 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:43.341+0000 7f5b59a00640 1 Processor -- start 2024-04-12T21:55:43.344 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:43.341+0000 7f5b59a00640 1 -- start start 2024-04-12T21:55:43.344 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:43.341+0000 7f5b59a00640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5b541085f0 0x7f5b54119130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:55:43.344 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:43.341+0000 7f5b59a00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5b54108f90 0x7f5b54119670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:55:43.344 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:43.341+0000 7f5b59a00640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b54119c70 con 0x7f5b54108f90 2024-04-12T21:55:43.344 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:43.341+0000 7f5b59a00640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b541132c0 con 0x7f5b541085f0 2024-04-12T21:55:43.344 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:43.342+0000 7f5b4b400640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5b54108f90 0x7f5b54119670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:55:43.344 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:43.342+0000 7f5b4b400640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5b54108f90 0x7f5b54119670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:43414/0 (socket says 172.21.15.18:43414) 2024-04-12T21:55:43.344 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:43.342+0000 7f5b4b400640 1 -- 172.21.15.18:0/1683536601 learned_addr learned my addr 172.21.15.18:0/1683536601 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:55:43.344 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:43.342+0000 7f5b4b400640 1 -- 172.21.15.18:0/1683536601 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5b541085f0 msgr2=0x7f5b54119130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:55:43.345 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:43.342+0000 7f5b4b400640 1 --2- 172.21.15.18:0/1683536601 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5b541085f0 0x7f5b54119130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:55:43.345 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:43.342+0000 7f5b4b400640 1 -- 172.21.15.18:0/1683536601 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b3c005040 con 0x7f5b54108f90 2024-04-12T21:55:43.345 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:43.342+0000 7f5b4b400640 1 --2- 172.21.15.18:0/1683536601 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5b54108f90 0x7f5b54119670 secure :-1 s=READY pgs=1425 cs=0 l=1 rev1=1 crypto rx=0x7f5b3c02be70 tx=0x7f5b3c009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:55:43.345 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:43.343+0000 7f5b50c00640 1 -- 172.21.15.18:0/1683536601 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5b3c0094c0 con 0x7f5b54108f90 2024-04-12T21:55:43.345 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:43.343+0000 7f5b50c00640 1 -- 172.21.15.18:0/1683536601 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f5b3c009620 con 0x7f5b54108f90 2024-04-12T21:55:43.345 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:43.343+0000 7f5b50c00640 1 -- 172.21.15.18:0/1683536601 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5b3c0435f0 con 0x7f5b54108f90 2024-04-12T21:55:43.345 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:43.343+0000 7f5b59a00640 1 -- 172.21.15.18:0/1683536601 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5b54113540 con 0x7f5b54108f90 2024-04-12T21:55:43.345 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:43.343+0000 7f5b59a00640 1 -- 172.21.15.18:0/1683536601 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5b54113a00 con 0x7f5b54108f90 2024-04-12T21:55:43.345 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:43.344+0000 7f5b59a00640 1 -- 172.21.15.18:0/1683536601 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5b54103d70 con 0x7f5b54108f90 2024-04-12T21:55:43.346 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:43.345+0000 7f5b50c00640 1 -- 172.21.15.18:0/1683536601 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7f5b3c02f070 con 0x7f5b54108f90 2024-04-12T21:55:43.347 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:43.346+0000 7f5b50c00640 1 --2- 172.21.15.18:0/1683536601 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f5b3806eb70 0x7f5b38071020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:55:43.347 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:43.346+0000 7f5b50c00640 1 -- 172.21.15.18:0/1683536601 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 6757+0+0 (secure 0 0 0) 0x7f5b3c0b0830 con 0x7f5b54108f90 2024-04-12T21:55:43.347 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:43.346+0000 7f5b53e00640 1 --2- 172.21.15.18:0/1683536601 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f5b3806eb70 0x7f5b38071020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:55:43.348 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:43.347+0000 7f5b53e00640 1 --2- 172.21.15.18:0/1683536601 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f5b3806eb70 0x7f5b38071020 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7f5b44004af0 tx=0x7f5b44007c30 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:55:43.352 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:43.351+0000 7f5b50c00640 1 -- 172.21.15.18:0/1683536601 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f5b3c079430 con 0x7f5b54108f90 2024-04-12T21:55:43.606 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:55:43 smithi018 ceph-mon[112590]: from='client.? 172.21.15.18:0/4053844814' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 32, "format": "json"}]: dispatch 2024-04-12T21:55:43.606 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:55:43 smithi018 ceph-mon[112590]: from='client.? 172.21.15.18:0/1612377175' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 33, "format": "json"}]: dispatch 2024-04-12T21:55:43.609 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:43.607+0000 7f5b59a00640 1 -- 172.21.15.18:0/1683536601 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 34, "format": "json"} v 0) v1 -- 0x7f5b5404bcc0 con 0x7f5b54108f90 2024-04-12T21:55:43.610 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:43.609+0000 7f5b50c00640 1 -- 172.21.15.18:0/1683536601 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 34, "format": "json"}]=0 dumped fsmap epoch 34 v38) v1 ==== 107+0+5217 (secure 0 0 0) 0x7f5b3c07caf0 con 0x7f5b54108f90 2024-04-12T21:55:43.611 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-12T21:55:43.611 INFO:teuthology.orchestra.run.smithi018.stdout:{"epoch":34,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":44595,"name":"cephfs.smithi186.jdllyn","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6825/3523444202","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6824","nonce":3523444202},{"type":"v1","addr":"172.21.15.186:6825","nonce":3523444202}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":31},{"gid":44875,"name":"cephfs.smithi186.wjnafs","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6827/3122443766","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6826","nonce":3122443766},{"type":"v1","addr":"172.21.15.186:6827","nonce":3122443766}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":33}],"filesystems":[{"mdsmap":{"epoch":34,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-12T20:51:57.362850+0000","modified":"2024-04-12T21:53:20.700323+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":100,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":2,"in":[0,1],"up":{"mds_0":34436,"mds_1":34468},"failed":[],"damaged":[],"stopped":[],"info":{"gid_34436":{"gid":34436,"name":"cephfs.smithi018.acqbsc","rank":0,"incarnation":34,"state":"up:replay","state_seq":1,"addr":"172.21.15.18:6829/755179764","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6828","nonce":755179764},{"type":"v1","addr":"172.21.15.18:6829","nonce":755179764}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34468":{"gid":34468,"name":"cephfs.smithi018.szjluo","rank":1,"incarnation":34,"state":"up:replay","state_seq":1,"addr":"172.21.15.18:6827/2407992653","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6826","nonce":2407992653},{"type":"v1","addr":"172.21.15.18:6827","nonce":2407992653}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-04-12T21:55:43.611 INFO:teuthology.orchestra.run.smithi018.stderr:dumped fsmap epoch 34 2024-04-12T21:55:43.613 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:43.612+0000 7f5b59a00640 1 -- 172.21.15.18:0/1683536601 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f5b3806eb70 msgr2=0x7f5b38071020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:55:43.614 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:43.612+0000 7f5b59a00640 1 --2- 172.21.15.18:0/1683536601 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f5b3806eb70 0x7f5b38071020 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7f5b44004af0 tx=0x7f5b44007c30 comp rx=0 tx=0).stop 2024-04-12T21:55:43.614 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:43.612+0000 7f5b59a00640 1 -- 172.21.15.18:0/1683536601 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5b54108f90 msgr2=0x7f5b54119670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:55:43.614 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:43.612+0000 7f5b59a00640 1 --2- 172.21.15.18:0/1683536601 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5b54108f90 0x7f5b54119670 secure :-1 s=READY pgs=1425 cs=0 l=1 rev1=1 crypto rx=0x7f5b3c02be70 tx=0x7f5b3c009da0 comp rx=0 tx=0).stop 2024-04-12T21:55:43.614 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:43.612+0000 7f5b59a00640 1 -- 172.21.15.18:0/1683536601 shutdown_connections 2024-04-12T21:55:43.614 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:43.612+0000 7f5b59a00640 1 --2- 172.21.15.18:0/1683536601 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f5b3806eb70 0x7f5b38071020 unknown :-1 s=CLOSED pgs=754 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:55:43.614 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:43.612+0000 7f5b59a00640 1 --2- 172.21.15.18:0/1683536601 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5b54108f90 0x7f5b54119670 unknown :-1 s=CLOSED pgs=1425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:55:43.614 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:43.612+0000 7f5b59a00640 1 --2- 172.21.15.18:0/1683536601 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5b541085f0 0x7f5b54119130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:55:43.614 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:43.612+0000 7f5b59a00640 1 -- 172.21.15.18:0/1683536601 >> 172.21.15.18:0/1683536601 conn(0x7f5b54071e30 msgr2=0x7f5b54073d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:55:43.614 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:43.612+0000 7f5b59a00640 1 -- 172.21.15.18:0/1683536601 shutdown_connections 2024-04-12T21:55:43.614 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:43.612+0000 7f5b59a00640 1 -- 172.21.15.18:0/1683536601 wait complete. 2024-04-12T21:55:43.631 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 35 2024-04-12T21:55:43.785 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:55:43 smithi186 ceph-mon[86379]: from='client.? 172.21.15.18:0/4053844814' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 32, "format": "json"}]: dispatch 2024-04-12T21:55:43.785 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:55:43 smithi186 ceph-mon[86379]: from='client.? 172.21.15.18:0/1612377175' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 33, "format": "json"}]: dispatch 2024-04-12T21:55:43.852 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:43.850+0000 7fb491e00640 1 -- 172.21.15.18:0/2652981020 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb48c070180 msgr2=0x7fb48c108530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:55:43.852 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:43.850+0000 7fb491e00640 1 --2- 172.21.15.18:0/2652981020 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb48c070180 0x7fb48c108530 secure :-1 s=READY pgs=1426 cs=0 l=1 rev1=1 crypto rx=0x7fb4780048d0 tx=0x7fb478009da0 comp rx=0 tx=0).stop 2024-04-12T21:55:43.852 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:43.850+0000 7fb491e00640 1 -- 172.21.15.18:0/2652981020 shutdown_connections 2024-04-12T21:55:43.852 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:43.850+0000 7fb491e00640 1 --2- 172.21.15.18:0/2652981020 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb48c070180 0x7fb48c108530 unknown :-1 s=CLOSED pgs=1426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:55:43.852 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:43.850+0000 7fb491e00640 1 --2- 172.21.15.18:0/2652981020 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb48c06f870 0x7fb48c06fc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:55:43.852 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:43.850+0000 7fb491e00640 1 -- 172.21.15.18:0/2652981020 >> 172.21.15.18:0/2652981020 conn(0x7fb48c0f7dd0 msgr2=0x7fb48c0fa1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:55:43.852 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:43.851+0000 7fb491e00640 1 -- 172.21.15.18:0/2652981020 shutdown_connections 2024-04-12T21:55:43.852 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:43.851+0000 7fb491e00640 1 -- 172.21.15.18:0/2652981020 wait complete. 2024-04-12T21:55:43.853 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:43.852+0000 7fb491e00640 1 Processor -- start 2024-04-12T21:55:43.854 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:43.852+0000 7fb491e00640 1 -- start start 2024-04-12T21:55:43.854 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:43.852+0000 7fb491e00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb48c06f870 0x7fb48c114f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:55:43.854 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:43.852+0000 7fb491e00640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb48c070180 0x7fb48c115440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:55:43.854 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:43.852+0000 7fb491e00640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb48c115a40 con 0x7fb48c06f870 2024-04-12T21:55:43.854 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:43.852+0000 7fb491e00640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb48c10f090 con 0x7fb48c070180 2024-04-12T21:55:43.854 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:43.852+0000 7fb48aa00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb48c06f870 0x7fb48c114f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:55:43.854 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:43.852+0000 7fb48aa00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb48c06f870 0x7fb48c114f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:43442/0 (socket says 172.21.15.18:43442) 2024-04-12T21:55:43.854 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:43.852+0000 7fb48aa00640 1 -- 172.21.15.18:0/3995985946 learned_addr learned my addr 172.21.15.18:0/3995985946 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:55:43.855 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:43.853+0000 7fb48aa00640 1 -- 172.21.15.18:0/3995985946 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb48c070180 msgr2=0x7fb48c115440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:55:43.855 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:43.853+0000 7fb48aa00640 1 --2- 172.21.15.18:0/3995985946 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb48c070180 0x7fb48c115440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:55:43.855 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:43.853+0000 7fb48aa00640 1 -- 172.21.15.18:0/3995985946 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb478005040 con 0x7fb48c06f870 2024-04-12T21:55:43.855 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:43.853+0000 7fb48aa00640 1 --2- 172.21.15.18:0/3995985946 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb48c06f870 0x7fb48c114f00 secure :-1 s=READY pgs=1427 cs=0 l=1 rev1=1 crypto rx=0x7fb47c002a50 tx=0x7fb47c004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:55:43.855 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:43.853+0000 7fb482a00640 1 -- 172.21.15.18:0/3995985946 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb47c007c30 con 0x7fb48c06f870 2024-04-12T21:55:43.855 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:43.853+0000 7fb491e00640 1 -- 172.21.15.18:0/3995985946 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb48c10f370 con 0x7fb48c06f870 2024-04-12T21:55:43.855 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:43.854+0000 7fb482a00640 1 -- 172.21.15.18:0/3995985946 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb47c00f040 con 0x7fb48c06f870 2024-04-12T21:55:43.856 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:43.854+0000 7fb482a00640 1 -- 172.21.15.18:0/3995985946 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb47c00db50 con 0x7fb48c06f870 2024-04-12T21:55:43.856 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:43.854+0000 7fb491e00640 1 -- 172.21.15.18:0/3995985946 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb48c10f890 con 0x7fb48c06f870 2024-04-12T21:55:43.856 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:43.855+0000 7fb491e00640 1 -- 172.21.15.18:0/3995985946 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb48c070fb0 con 0x7fb48c06f870 2024-04-12T21:55:43.857 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:43.856+0000 7fb482a00640 1 -- 172.21.15.18:0/3995985946 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7fb47c01c400 con 0x7fb48c06f870 2024-04-12T21:55:43.858 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:43.857+0000 7fb482a00640 1 --2- 172.21.15.18:0/3995985946 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fb47006eb70 0x7fb470071020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:55:43.858 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:43.857+0000 7fb482a00640 1 -- 172.21.15.18:0/3995985946 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 6757+0+0 (secure 0 0 0) 0x7fb47c08e390 con 0x7fb48c06f870 2024-04-12T21:55:43.858 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:43.857+0000 7fb48a000640 1 --2- 172.21.15.18:0/3995985946 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fb47006eb70 0x7fb470071020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:55:43.859 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:43.858+0000 7fb48a000640 1 --2- 172.21.15.18:0/3995985946 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fb47006eb70 0x7fb470071020 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7fb47802bda0 tx=0x7fb478009da0 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:55:43.863 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:43.862+0000 7fb482a00640 1 -- 172.21.15.18:0/3995985946 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7fb47c056f90 con 0x7fb48c06f870 2024-04-12T21:55:44.100 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:44.099+0000 7fb491e00640 1 -- 172.21.15.18:0/3995985946 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 35, "format": "json"} v 0) v1 -- 0x7fb48c10b5b0 con 0x7fb48c06f870 2024-04-12T21:55:44.102 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:44.100+0000 7fb482a00640 1 -- 172.21.15.18:0/3995985946 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 35, "format": "json"}]=0 dumped fsmap epoch 35 v38) v1 ==== 107+0+5223 (secure 0 0 0) 0x7fb47c05a650 con 0x7fb48c06f870 2024-04-12T21:55:44.102 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-12T21:55:44.102 INFO:teuthology.orchestra.run.smithi018.stdout:{"epoch":35,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":44595,"name":"cephfs.smithi186.jdllyn","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6825/3523444202","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6824","nonce":3523444202},{"type":"v1","addr":"172.21.15.186:6825","nonce":3523444202}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":31},{"gid":44875,"name":"cephfs.smithi186.wjnafs","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6827/3122443766","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6826","nonce":3122443766},{"type":"v1","addr":"172.21.15.186:6827","nonce":3122443766}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":33}],"filesystems":[{"mdsmap":{"epoch":35,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-12T20:51:57.362850+0000","modified":"2024-04-12T21:53:27.551174+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":100,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":2,"in":[0,1],"up":{"mds_0":34436,"mds_1":34468},"failed":[],"damaged":[],"stopped":[],"info":{"gid_34436":{"gid":34436,"name":"cephfs.smithi018.acqbsc","rank":0,"incarnation":34,"state":"up:resolve","state_seq":696,"addr":"172.21.15.18:6829/755179764","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6828","nonce":755179764},{"type":"v1","addr":"172.21.15.18:6829","nonce":755179764}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34468":{"gid":34468,"name":"cephfs.smithi018.szjluo","rank":1,"incarnation":34,"state":"up:resolve","state_seq":689,"addr":"172.21.15.18:6827/2407992653","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6826","nonce":2407992653},{"type":"v1","addr":"172.21.15.18:6827","nonce":2407992653}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-04-12T21:55:44.103 INFO:teuthology.orchestra.run.smithi018.stderr:dumped fsmap epoch 35 2024-04-12T21:55:44.105 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:44.103+0000 7fb491e00640 1 -- 172.21.15.18:0/3995985946 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fb47006eb70 msgr2=0x7fb470071020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:55:44.105 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:44.103+0000 7fb491e00640 1 --2- 172.21.15.18:0/3995985946 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fb47006eb70 0x7fb470071020 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7fb47802bda0 tx=0x7fb478009da0 comp rx=0 tx=0).stop 2024-04-12T21:55:44.105 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:44.103+0000 7fb491e00640 1 -- 172.21.15.18:0/3995985946 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb48c06f870 msgr2=0x7fb48c114f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:55:44.105 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:44.103+0000 7fb491e00640 1 --2- 172.21.15.18:0/3995985946 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb48c06f870 0x7fb48c114f00 secure :-1 s=READY pgs=1427 cs=0 l=1 rev1=1 crypto rx=0x7fb47c002a50 tx=0x7fb47c004600 comp rx=0 tx=0).stop 2024-04-12T21:55:44.105 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:44.103+0000 7fb491e00640 1 -- 172.21.15.18:0/3995985946 shutdown_connections 2024-04-12T21:55:44.105 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:44.103+0000 7fb491e00640 1 --2- 172.21.15.18:0/3995985946 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fb47006eb70 0x7fb470071020 unknown :-1 s=CLOSED pgs=755 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:55:44.105 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:44.103+0000 7fb491e00640 1 --2- 172.21.15.18:0/3995985946 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb48c070180 0x7fb48c115440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:55:44.105 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:44.103+0000 7fb491e00640 1 --2- 172.21.15.18:0/3995985946 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb48c06f870 0x7fb48c114f00 unknown :-1 s=CLOSED pgs=1427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:55:44.105 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:44.103+0000 7fb491e00640 1 -- 172.21.15.18:0/3995985946 >> 172.21.15.18:0/3995985946 conn(0x7fb48c0f7dd0 msgr2=0x7fb48c104d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:55:44.105 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:44.104+0000 7fb491e00640 1 -- 172.21.15.18:0/3995985946 shutdown_connections 2024-04-12T21:55:44.105 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:44.104+0000 7fb491e00640 1 -- 172.21.15.18:0/3995985946 wait complete. 2024-04-12T21:55:44.121 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 36 2024-04-12T21:55:44.338 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:44.336+0000 7f644ba00640 1 -- 172.21.15.18:0/3614815863 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f64441043e0 msgr2=0x7f64441047b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:55:44.338 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:44.336+0000 7f644ba00640 1 --2- 172.21.15.18:0/3614815863 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f64441043e0 0x7f64441047b0 secure :-1 s=READY pgs=1428 cs=0 l=1 rev1=1 crypto rx=0x7f6434009c50 tx=0x7f6434004d10 comp rx=0 tx=0).stop 2024-04-12T21:55:44.338 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:44.336+0000 7f644ba00640 1 -- 172.21.15.18:0/3614815863 shutdown_connections 2024-04-12T21:55:44.338 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:44.336+0000 7f644ba00640 1 --2- 172.21.15.18:0/3614815863 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6444104d80 0x7f64440fe410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:55:44.338 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:44.336+0000 7f644ba00640 1 --2- 172.21.15.18:0/3614815863 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f64441043e0 0x7f64441047b0 unknown :-1 s=CLOSED pgs=1428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:55:44.338 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:44.336+0000 7f644ba00640 1 -- 172.21.15.18:0/3614815863 >> 172.21.15.18:0/3614815863 conn(0x7f64440f9ff0 msgr2=0x7f64440fc400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:55:44.338 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:44.336+0000 7f644ba00640 1 -- 172.21.15.18:0/3614815863 shutdown_connections 2024-04-12T21:55:44.338 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:44.336+0000 7f644ba00640 1 -- 172.21.15.18:0/3614815863 wait complete. 2024-04-12T21:55:44.340 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:44.337+0000 7f644ba00640 1 Processor -- start 2024-04-12T21:55:44.340 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:44.337+0000 7f644ba00640 1 -- start start 2024-04-12T21:55:44.340 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:44.338+0000 7f644ba00640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f64441043e0 0x7f6444114ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:55:44.340 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:44.338+0000 7f644ba00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6444104d80 0x7f6444115400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:55:44.341 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:44.338+0000 7f644ba00640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6444115a00 con 0x7f6444104d80 2024-04-12T21:55:44.341 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:44.338+0000 7f644ba00640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f644410f050 con 0x7f64441043e0 2024-04-12T21:55:44.341 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:44.338+0000 7f6443e00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6444104d80 0x7f6444115400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:55:44.341 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:44.338+0000 7f6443e00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6444104d80 0x7f6444115400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:43454/0 (socket says 172.21.15.18:43454) 2024-04-12T21:55:44.341 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:44.338+0000 7f6443e00640 1 -- 172.21.15.18:0/1129949094 learned_addr learned my addr 172.21.15.18:0/1129949094 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:55:44.341 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:44.338+0000 7f6449000640 1 --2- 172.21.15.18:0/1129949094 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f64441043e0 0x7f6444114ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:55:44.341 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:44.338+0000 7f6443e00640 1 -- 172.21.15.18:0/1129949094 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f64441043e0 msgr2=0x7f6444114ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:55:44.341 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:44.338+0000 7f6443e00640 1 --2- 172.21.15.18:0/1129949094 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f64441043e0 0x7f6444114ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:55:44.341 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:44.338+0000 7f6443e00640 1 -- 172.21.15.18:0/1129949094 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64340098b0 con 0x7f6444104d80 2024-04-12T21:55:44.341 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:44.339+0000 7f6443e00640 1 --2- 172.21.15.18:0/1129949094 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6444104d80 0x7f6444115400 secure :-1 s=READY pgs=1429 cs=0 l=1 rev1=1 crypto rx=0x7f642c00eb70 tx=0x7f642c00b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:55:44.341 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:44.339+0000 7f6440c00640 1 -- 172.21.15.18:0/1129949094 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f642c007c30 con 0x7f6444104d80 2024-04-12T21:55:44.341 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:44.339+0000 7f6440c00640 1 -- 172.21.15.18:0/1129949094 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f642c00f040 con 0x7f6444104d80 2024-04-12T21:55:44.341 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:44.339+0000 7f644ba00640 1 -- 172.21.15.18:0/1129949094 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f644410f2d0 con 0x7f6444104d80 2024-04-12T21:55:44.341 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:44.339+0000 7f6440c00640 1 -- 172.21.15.18:0/1129949094 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f642c0095b0 con 0x7f6444104d80 2024-04-12T21:55:44.342 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:44.339+0000 7f644ba00640 1 -- 172.21.15.18:0/1129949094 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f644410f870 con 0x7f6444104d80 2024-04-12T21:55:44.342 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:44.340+0000 7f644ba00640 1 -- 172.21.15.18:0/1129949094 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f64440ffb50 con 0x7f6444104d80 2024-04-12T21:55:44.342 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:44.341+0000 7f6440c00640 1 -- 172.21.15.18:0/1129949094 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7f642c009730 con 0x7f6444104d80 2024-04-12T21:55:44.344 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:44.342+0000 7f6440c00640 1 --2- 172.21.15.18:0/1129949094 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f642806eb70 0x7f6428071020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:55:44.344 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:44.343+0000 7f6440c00640 1 -- 172.21.15.18:0/1129949094 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 6757+0+0 (secure 0 0 0) 0x7f642c08e7a0 con 0x7f6444104d80 2024-04-12T21:55:44.344 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:44.343+0000 7f6449000640 1 --2- 172.21.15.18:0/1129949094 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f642806eb70 0x7f6428071020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:55:44.344 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:44.344+0000 7f6449000640 1 --2- 172.21.15.18:0/1129949094 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f642806eb70 0x7f6428071020 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7f6434005010 tx=0x7f643402e040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:55:44.348 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:44.347+0000 7f6440c00640 1 -- 172.21.15.18:0/1129949094 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f642c0573a0 con 0x7f6444104d80 2024-04-12T21:55:44.599 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:44.598+0000 7f644ba00640 1 -- 172.21.15.18:0/1129949094 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 36, "format": "json"} v 0) v1 -- 0x7f644410b5a0 con 0x7f6444104d80 2024-04-12T21:55:44.600 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:44.599+0000 7f6440c00640 1 -- 172.21.15.18:0/1129949094 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 36, "format": "json"}]=0 dumped fsmap epoch 36 v38) v1 ==== 107+0+5227 (secure 0 0 0) 0x7f642c05aa60 con 0x7f6444104d80 2024-04-12T21:55:44.601 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-12T21:55:44.601 INFO:teuthology.orchestra.run.smithi018.stdout:{"epoch":36,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":44595,"name":"cephfs.smithi186.jdllyn","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6825/3523444202","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6824","nonce":3523444202},{"type":"v1","addr":"172.21.15.186:6825","nonce":3523444202}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":31},{"gid":44875,"name":"cephfs.smithi186.wjnafs","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6827/3122443766","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6826","nonce":3122443766},{"type":"v1","addr":"172.21.15.186:6827","nonce":3122443766}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":33}],"filesystems":[{"mdsmap":{"epoch":36,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-12T20:51:57.362850+0000","modified":"2024-04-12T21:53:27.701777+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":100,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":2,"in":[0,1],"up":{"mds_0":34436,"mds_1":34468},"failed":[],"damaged":[],"stopped":[],"info":{"gid_34436":{"gid":34436,"name":"cephfs.smithi018.acqbsc","rank":0,"incarnation":34,"state":"up:reconnect","state_seq":697,"addr":"172.21.15.18:6829/755179764","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6828","nonce":755179764},{"type":"v1","addr":"172.21.15.18:6829","nonce":755179764}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34468":{"gid":34468,"name":"cephfs.smithi018.szjluo","rank":1,"incarnation":34,"state":"up:reconnect","state_seq":690,"addr":"172.21.15.18:6827/2407992653","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6826","nonce":2407992653},{"type":"v1","addr":"172.21.15.18:6827","nonce":2407992653}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-04-12T21:55:44.601 INFO:teuthology.orchestra.run.smithi018.stderr:dumped fsmap epoch 36 2024-04-12T21:55:44.603 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:44.602+0000 7f644ba00640 1 -- 172.21.15.18:0/1129949094 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f642806eb70 msgr2=0x7f6428071020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:55:44.603 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:44.602+0000 7f644ba00640 1 --2- 172.21.15.18:0/1129949094 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f642806eb70 0x7f6428071020 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7f6434005010 tx=0x7f643402e040 comp rx=0 tx=0).stop 2024-04-12T21:55:44.603 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:44.602+0000 7f644ba00640 1 -- 172.21.15.18:0/1129949094 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6444104d80 msgr2=0x7f6444115400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:55:44.603 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:44.602+0000 7f644ba00640 1 --2- 172.21.15.18:0/1129949094 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6444104d80 0x7f6444115400 secure :-1 s=READY pgs=1429 cs=0 l=1 rev1=1 crypto rx=0x7f642c00eb70 tx=0x7f642c00b040 comp rx=0 tx=0).stop 2024-04-12T21:55:44.603 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:44.602+0000 7f644ba00640 1 -- 172.21.15.18:0/1129949094 shutdown_connections 2024-04-12T21:55:44.604 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:44.602+0000 7f644ba00640 1 --2- 172.21.15.18:0/1129949094 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f642806eb70 0x7f6428071020 unknown :-1 s=CLOSED pgs=756 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:55:44.604 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:44.602+0000 7f644ba00640 1 --2- 172.21.15.18:0/1129949094 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6444104d80 0x7f6444115400 unknown :-1 s=CLOSED pgs=1429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:55:44.604 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:44.602+0000 7f644ba00640 1 --2- 172.21.15.18:0/1129949094 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f64441043e0 0x7f6444114ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:55:44.604 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:44.602+0000 7f644ba00640 1 -- 172.21.15.18:0/1129949094 >> 172.21.15.18:0/1129949094 conn(0x7f64440f9ff0 msgr2=0x7f64440faf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:55:44.604 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:44.602+0000 7f644ba00640 1 -- 172.21.15.18:0/1129949094 shutdown_connections 2024-04-12T21:55:44.604 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:44.603+0000 7f644ba00640 1 -- 172.21.15.18:0/1129949094 wait complete. 2024-04-12T21:55:44.605 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:55:44 smithi018 ceph-mon[112590]: from='client.? 172.21.15.18:0/1683536601' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 34, "format": "json"}]: dispatch 2024-04-12T21:55:44.605 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:55:44 smithi018 ceph-mon[112590]: from='client.? 172.21.15.18:0/3995985946' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 35, "format": "json"}]: dispatch 2024-04-12T21:55:44.605 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:55:44 smithi018 ceph-mon[112590]: pgmap v1684: 65 pgs: 65 active+clean; 257 MiB data, 976 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:55:44.622 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 37 2024-04-12T21:55:44.785 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:55:44 smithi186 ceph-mon[86379]: from='client.? 172.21.15.18:0/1683536601' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 34, "format": "json"}]: dispatch 2024-04-12T21:55:44.785 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:55:44 smithi186 ceph-mon[86379]: from='client.? 172.21.15.18:0/3995985946' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 35, "format": "json"}]: dispatch 2024-04-12T21:55:44.785 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:55:44 smithi186 ceph-mon[86379]: pgmap v1684: 65 pgs: 65 active+clean; 257 MiB data, 976 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:55:44.813 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:44.811+0000 7f2a22200640 1 -- 172.21.15.18:0/1661057073 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2a1c0fdb70 msgr2=0x7f2a1c1061b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:55:44.813 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:44.811+0000 7f2a22200640 1 --2- 172.21.15.18:0/1661057073 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2a1c0fdb70 0x7f2a1c1061b0 secure :-1 s=READY pgs=1430 cs=0 l=1 rev1=1 crypto rx=0x7f2a080048d0 tx=0x7f2a08009da0 comp rx=0 tx=0).stop 2024-04-12T21:55:44.813 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:44.811+0000 7f2a22200640 1 -- 172.21.15.18:0/1661057073 shutdown_connections 2024-04-12T21:55:44.813 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:44.811+0000 7f2a22200640 1 --2- 172.21.15.18:0/1661057073 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2a1c0fdb70 0x7f2a1c1061b0 unknown :-1 s=CLOSED pgs=1430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:55:44.813 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:44.811+0000 7f2a22200640 1 --2- 172.21.15.18:0/1661057073 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2a1c0fd1d0 0x7f2a1c0fd5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:55:44.813 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:44.811+0000 7f2a22200640 1 -- 172.21.15.18:0/1661057073 >> 172.21.15.18:0/1661057073 conn(0x7f2a1c0f6c20 msgr2=0x7f2a1c0f9030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:55:44.813 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:44.811+0000 7f2a22200640 1 -- 172.21.15.18:0/1661057073 shutdown_connections 2024-04-12T21:55:44.813 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:44.811+0000 7f2a22200640 1 -- 172.21.15.18:0/1661057073 wait complete. 2024-04-12T21:55:44.814 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:44.812+0000 7f2a22200640 1 Processor -- start 2024-04-12T21:55:44.815 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:44.812+0000 7f2a22200640 1 -- start start 2024-04-12T21:55:44.815 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:44.812+0000 7f2a22200640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2a1c0fd1d0 0x7f2a1c114f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:55:44.815 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:44.813+0000 7f2a22200640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2a1c0fdb70 0x7f2a1c1154b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:55:44.815 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:44.813+0000 7f2a22200640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a1c115ab0 con 0x7f2a1c0fdb70 2024-04-12T21:55:44.815 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:44.813+0000 7f2a22200640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a1c10f100 con 0x7f2a1c0fd1d0 2024-04-12T21:55:44.815 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:44.813+0000 7f2a1aa00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2a1c0fdb70 0x7f2a1c1154b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:55:44.815 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:44.813+0000 7f2a1aa00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2a1c0fdb70 0x7f2a1c1154b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:43476/0 (socket says 172.21.15.18:43476) 2024-04-12T21:55:44.815 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:44.813+0000 7f2a1aa00640 1 -- 172.21.15.18:0/1109916803 learned_addr learned my addr 172.21.15.18:0/1109916803 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:55:44.815 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:44.813+0000 7f2a1b400640 1 --2- 172.21.15.18:0/1109916803 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2a1c0fd1d0 0x7f2a1c114f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:55:44.815 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:44.813+0000 7f2a1aa00640 1 -- 172.21.15.18:0/1109916803 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2a1c0fd1d0 msgr2=0x7f2a1c114f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:55:44.815 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:44.813+0000 7f2a1aa00640 1 --2- 172.21.15.18:0/1109916803 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2a1c0fd1d0 0x7f2a1c114f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:55:44.815 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:44.813+0000 7f2a1aa00640 1 -- 172.21.15.18:0/1109916803 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a08005040 con 0x7f2a1c0fdb70 2024-04-12T21:55:44.816 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:44.814+0000 7f2a1aa00640 1 --2- 172.21.15.18:0/1109916803 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2a1c0fdb70 0x7f2a1c1154b0 secure :-1 s=READY pgs=1431 cs=0 l=1 rev1=1 crypto rx=0x7f2a080047f0 tx=0x7f2a08009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:55:44.816 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:44.814+0000 7f2a13400640 1 -- 172.21.15.18:0/1109916803 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2a080092b0 con 0x7f2a1c0fdb70 2024-04-12T21:55:44.816 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:44.814+0000 7f2a13400640 1 -- 172.21.15.18:0/1109916803 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f2a0803b400 con 0x7f2a1c0fdb70 2024-04-12T21:55:44.816 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:44.814+0000 7f2a22200640 1 -- 172.21.15.18:0/1109916803 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2a1c10f380 con 0x7f2a1c0fdb70 2024-04-12T21:55:44.816 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:44.814+0000 7f2a13400640 1 -- 172.21.15.18:0/1109916803 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2a080435b0 con 0x7f2a1c0fdb70 2024-04-12T21:55:44.817 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:44.814+0000 7f2a22200640 1 -- 172.21.15.18:0/1109916803 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2a1c10f840 con 0x7f2a1c0fdb70 2024-04-12T21:55:44.817 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:44.816+0000 7f2a22200640 1 -- 172.21.15.18:0/1109916803 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2a1c066060 con 0x7f2a1c0fdb70 2024-04-12T21:55:44.818 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:44.817+0000 7f2a13400640 1 -- 172.21.15.18:0/1109916803 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7f2a0802f070 con 0x7f2a1c0fdb70 2024-04-12T21:55:44.820 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:44.819+0000 7f2a13400640 1 --2- 172.21.15.18:0/1109916803 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f2a0006eb70 0x7f2a00071020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:55:44.820 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:44.819+0000 7f2a13400640 1 -- 172.21.15.18:0/1109916803 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 6757+0+0 (secure 0 0 0) 0x7f2a080b05d0 con 0x7f2a1c0fdb70 2024-04-12T21:55:44.820 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:44.819+0000 7f2a1b400640 1 --2- 172.21.15.18:0/1109916803 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f2a0006eb70 0x7f2a00071020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:55:44.821 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:44.820+0000 7f2a1b400640 1 --2- 172.21.15.18:0/1109916803 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f2a0006eb70 0x7f2a00071020 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7f2a0c009af0 tx=0x7f2a0c00b040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:55:44.824 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:44.823+0000 7f2a13400640 1 -- 172.21.15.18:0/1109916803 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f2a08079300 con 0x7f2a1c0fdb70 2024-04-12T21:55:45.071 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:45.069+0000 7f2a22200640 1 -- 172.21.15.18:0/1109916803 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 37, "format": "json"} v 0) v1 -- 0x7f2a1c109320 con 0x7f2a1c0fdb70 2024-04-12T21:55:45.072 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:45.071+0000 7f2a13400640 1 -- 172.21.15.18:0/1109916803 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 37, "format": "json"}]=0 dumped fsmap epoch 37 v38) v1 ==== 107+0+5221 (secure 0 0 0) 0x7f2a0807c9c0 con 0x7f2a1c0fdb70 2024-04-12T21:55:45.072 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-12T21:55:45.072 INFO:teuthology.orchestra.run.smithi018.stdout:{"epoch":37,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":44595,"name":"cephfs.smithi186.jdllyn","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6825/3523444202","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6824","nonce":3523444202},{"type":"v1","addr":"172.21.15.186:6825","nonce":3523444202}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":31},{"gid":44875,"name":"cephfs.smithi186.wjnafs","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6827/3122443766","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6826","nonce":3122443766},{"type":"v1","addr":"172.21.15.186:6827","nonce":3122443766}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":33}],"filesystems":[{"mdsmap":{"epoch":37,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-12T20:51:57.362850+0000","modified":"2024-04-12T21:53:28.664191+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":100,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":2,"in":[0,1],"up":{"mds_0":34436,"mds_1":34468},"failed":[],"damaged":[],"stopped":[],"info":{"gid_34436":{"gid":34436,"name":"cephfs.smithi018.acqbsc","rank":0,"incarnation":34,"state":"up:rejoin","state_seq":698,"addr":"172.21.15.18:6829/755179764","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6828","nonce":755179764},{"type":"v1","addr":"172.21.15.18:6829","nonce":755179764}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34468":{"gid":34468,"name":"cephfs.smithi018.szjluo","rank":1,"incarnation":34,"state":"up:rejoin","state_seq":691,"addr":"172.21.15.18:6827/2407992653","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6826","nonce":2407992653},{"type":"v1","addr":"172.21.15.18:6827","nonce":2407992653}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-04-12T21:55:45.073 INFO:teuthology.orchestra.run.smithi018.stderr:dumped fsmap epoch 37 2024-04-12T21:55:45.075 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:45.073+0000 7f2a22200640 1 -- 172.21.15.18:0/1109916803 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f2a0006eb70 msgr2=0x7f2a00071020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:55:45.075 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:45.073+0000 7f2a22200640 1 --2- 172.21.15.18:0/1109916803 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f2a0006eb70 0x7f2a00071020 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7f2a0c009af0 tx=0x7f2a0c00b040 comp rx=0 tx=0).stop 2024-04-12T21:55:45.075 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:45.073+0000 7f2a22200640 1 -- 172.21.15.18:0/1109916803 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2a1c0fdb70 msgr2=0x7f2a1c1154b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:55:45.075 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:45.073+0000 7f2a22200640 1 --2- 172.21.15.18:0/1109916803 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2a1c0fdb70 0x7f2a1c1154b0 secure :-1 s=READY pgs=1431 cs=0 l=1 rev1=1 crypto rx=0x7f2a080047f0 tx=0x7f2a08009da0 comp rx=0 tx=0).stop 2024-04-12T21:55:45.075 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:45.073+0000 7f2a22200640 1 -- 172.21.15.18:0/1109916803 shutdown_connections 2024-04-12T21:55:45.075 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:45.073+0000 7f2a22200640 1 --2- 172.21.15.18:0/1109916803 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f2a0006eb70 0x7f2a00071020 unknown :-1 s=CLOSED pgs=757 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:55:45.075 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:45.073+0000 7f2a22200640 1 --2- 172.21.15.18:0/1109916803 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2a1c0fdb70 0x7f2a1c1154b0 unknown :-1 s=CLOSED pgs=1431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:55:45.075 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:45.073+0000 7f2a22200640 1 --2- 172.21.15.18:0/1109916803 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2a1c0fd1d0 0x7f2a1c114f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:55:45.075 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:45.073+0000 7f2a22200640 1 -- 172.21.15.18:0/1109916803 >> 172.21.15.18:0/1109916803 conn(0x7f2a1c0f6c20 msgr2=0x7f2a1c0f81b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:55:45.075 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:45.073+0000 7f2a22200640 1 -- 172.21.15.18:0/1109916803 shutdown_connections 2024-04-12T21:55:45.075 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:45.074+0000 7f2a22200640 1 -- 172.21.15.18:0/1109916803 wait complete. 2024-04-12T21:55:45.091 DEBUG:teuthology.run_tasks:Unwinding manager ceph-fuse 2024-04-12T21:55:45.102 INFO:tasks.ceph_fuse:Unmounting ceph-fuse clients... 2024-04-12T21:55:45.102 DEBUG:teuthology.orchestra.run.smithi018:> set -ex 2024-04-12T21:55:45.102 DEBUG:teuthology.orchestra.run.smithi018:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-12T21:55:45.154 DEBUG:teuthology.orchestra.run.smithi018:> set -ex 2024-04-12T21:55:45.154 DEBUG:teuthology.orchestra.run.smithi018:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-12T21:55:45.238 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-12T21:55:45.349 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:55:45 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-12T21:55:45.349 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:55:45 smithi018 ceph-mon[112590]: from='client.? 172.21.15.18:0/1129949094' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 36, "format": "json"}]: dispatch 2024-04-12T21:55:45.349 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:55:45 smithi018 ceph-mon[112590]: from='client.? 172.21.15.18:0/1109916803' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 37, "format": "json"}]: dispatch 2024-04-12T21:55:45.467 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:45.465+0000 7ff654000640 1 -- 172.21.15.18:0/2038193417 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff64c0fdbe0 msgr2=0x7ff64c106340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:55:45.467 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:45.465+0000 7ff654000640 1 --2- 172.21.15.18:0/2038193417 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff64c0fdbe0 0x7ff64c106340 secure :-1 s=READY pgs=1432 cs=0 l=1 rev1=1 crypto rx=0x7ff6380048d0 tx=0x7ff638009da0 comp rx=0 tx=0).stop 2024-04-12T21:55:45.467 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:45.466+0000 7ff654000640 1 -- 172.21.15.18:0/2038193417 shutdown_connections 2024-04-12T21:55:45.467 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:45.466+0000 7ff654000640 1 --2- 172.21.15.18:0/2038193417 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff64c0fdbe0 0x7ff64c106340 unknown :-1 s=CLOSED pgs=1432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:55:45.467 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:45.466+0000 7ff654000640 1 --2- 172.21.15.18:0/2038193417 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff64c0fd2d0 0x7ff64c0fd6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:55:45.467 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:45.466+0000 7ff654000640 1 -- 172.21.15.18:0/2038193417 >> 172.21.15.18:0/2038193417 conn(0x7ff64c0f6d70 msgr2=0x7ff64c0f9180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:55:45.468 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:45.466+0000 7ff654000640 1 -- 172.21.15.18:0/2038193417 shutdown_connections 2024-04-12T21:55:45.468 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:45.466+0000 7ff654000640 1 -- 172.21.15.18:0/2038193417 wait complete. 2024-04-12T21:55:45.469 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:45.466+0000 7ff654000640 1 Processor -- start 2024-04-12T21:55:45.469 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:45.467+0000 7ff654000640 1 -- start start 2024-04-12T21:55:45.469 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:45.467+0000 7ff654000640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff64c0fd2d0 0x7ff64c0fb190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:55:45.469 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:45.467+0000 7ff654000640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff64c0fdbe0 0x7ff64c0fb6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:55:45.469 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:45.467+0000 7ff654000640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff64c0fd0f0 con 0x7ff64c0fd2d0 2024-04-12T21:55:45.469 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:45.467+0000 7ff654000640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff64c0fbc10 con 0x7ff64c0fdbe0 2024-04-12T21:55:45.470 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:45.467+0000 7ff651600640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff64c0fd2d0 0x7ff64c0fb190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:55:45.470 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:45.467+0000 7ff651600640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff64c0fd2d0 0x7ff64c0fb190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:43494/0 (socket says 172.21.15.18:43494) 2024-04-12T21:55:45.470 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:45.467+0000 7ff651600640 1 -- 172.21.15.18:0/555862095 learned_addr learned my addr 172.21.15.18:0/555862095 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:55:45.470 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:45.468+0000 7ff650c00640 1 --2- 172.21.15.18:0/555862095 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff64c0fdbe0 0x7ff64c0fb6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:55:45.470 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:45.468+0000 7ff651600640 1 -- 172.21.15.18:0/555862095 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff64c0fdbe0 msgr2=0x7ff64c0fb6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:55:45.470 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:45.468+0000 7ff651600640 1 --2- 172.21.15.18:0/555862095 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff64c0fdbe0 0x7ff64c0fb6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:55:45.470 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:45.468+0000 7ff651600640 1 -- 172.21.15.18:0/555862095 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff638005040 con 0x7ff64c0fd2d0 2024-04-12T21:55:45.470 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:45.468+0000 7ff651600640 1 --2- 172.21.15.18:0/555862095 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff64c0fd2d0 0x7ff64c0fb190 secure :-1 s=READY pgs=1433 cs=0 l=1 rev1=1 crypto rx=0x7ff640002a50 tx=0x7ff640004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:55:45.471 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:45.468+0000 7ff649600640 1 -- 172.21.15.18:0/555862095 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff640007c30 con 0x7ff64c0fd2d0 2024-04-12T21:55:45.471 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:45.468+0000 7ff654000640 1 -- 172.21.15.18:0/555862095 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff64c0fbef0 con 0x7ff64c0fd2d0 2024-04-12T21:55:45.471 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:45.468+0000 7ff649600640 1 -- 172.21.15.18:0/555862095 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff64000f040 con 0x7ff64c0fd2d0 2024-04-12T21:55:45.471 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:45.468+0000 7ff649600640 1 -- 172.21.15.18:0/555862095 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff64000dad0 con 0x7ff64c0fd2d0 2024-04-12T21:55:45.471 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:45.469+0000 7ff654000640 1 -- 172.21.15.18:0/555862095 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff64c1194f0 con 0x7ff64c0fd2d0 2024-04-12T21:55:45.471 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:45.470+0000 7ff654000640 1 -- 172.21.15.18:0/555862095 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff64c065d90 con 0x7ff64c0fd2d0 2024-04-12T21:55:45.472 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:45.472+0000 7ff649600640 1 -- 172.21.15.18:0/555862095 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7ff64000dd40 con 0x7ff64c0fd2d0 2024-04-12T21:55:45.474 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:45.472+0000 7ff649600640 1 --2- 172.21.15.18:0/555862095 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7ff63406eb70 0x7ff634071020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:55:45.474 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:45.473+0000 7ff649600640 1 -- 172.21.15.18:0/555862095 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 6757+0+0 (secure 0 0 0) 0x7ff64008e300 con 0x7ff64c0fd2d0 2024-04-12T21:55:45.474 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:45.473+0000 7ff650c00640 1 --2- 172.21.15.18:0/555862095 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7ff63406eb70 0x7ff634071020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:55:45.475 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:45.474+0000 7ff650c00640 1 --2- 172.21.15.18:0/555862095 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7ff63406eb70 0x7ff634071020 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7ff6380048a0 tx=0x7ff638009da0 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:55:45.478 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:45.477+0000 7ff649600640 1 -- 172.21.15.18:0/555862095 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7ff640056f00 con 0x7ff64c0fd2d0 2024-04-12T21:55:45.693 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:45.689+0000 7ff654000640 1 -- 172.21.15.18:0/555862095 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff64c100b20 con 0x7ff64c0fd2d0 2024-04-12T21:55:45.694 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:45.690+0000 7ff649600640 1 -- 172.21.15.18:0/555862095 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v104) v1 ==== 74+0+17095 (secure 0 0 0) 0x7ff64005a5c0 con 0x7ff64c0fd2d0 2024-04-12T21:55:45.695 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-12T21:55:45.695 INFO:teuthology.orchestra.run.smithi018.stdout:{"epoch":104,"fsid":"1cbe407a-f90d-11ee-bc8d-c7b262605968","created":"2024-04-12T20:43:00.692361+0000","modified":"2024-04-12T21:38:07.461428+0000","last_up_change":"2024-04-12T21:06:52.454846+0000","last_in_change":"2024-04-12T20:49:54.190644+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":20,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-12T20:48:37.529081+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":6,"score_stable":6,"optimal_score":0.5,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs.cephfs.meta","create_time":"2024-04-12T20:51:56.284414+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"44","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.309999942779541,"score_stable":1.309999942779541,"optimal_score":1,"raw_score_acting":1.309999942779541,"raw_score_stable":1.309999942779541,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs.cephfs.data","create_time":"2024-04-12T20:51:56.927802+0000","flags":131073,"flags_names":"hashpspool,bulk","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"45","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2af887cc-0447-42aa-a8f3-1aa7fcd4dc8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":9,"last_clean_end":49,"up_from":64,"up_thru":97,"down_at":50,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6802","nonce":2573875940},{"type":"v1","addr":"172.21.15.18:6803","nonce":2573875940}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6804","nonce":2573875940},{"type":"v1","addr":"172.21.15.18:6805","nonce":2573875940}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6808","nonce":2573875940},{"type":"v1","addr":"172.21.15.18:6809","nonce":2573875940}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6806","nonce":2573875940},{"type":"v1","addr":"172.21.15.18:6807","nonce":2573875940}]},"public_addr":"172.21.15.18:6803/2573875940","cluster_addr":"172.21.15.18:6805/2573875940","heartbeat_back_addr":"172.21.15.18:6809/2573875940","heartbeat_front_addr":"172.21.15.18:6807/2573875940","state":["exists","up"]},{"osd":1,"uuid":"6ba7b0d3-a03d-45cc-b4e9-89a92c8c6ca8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":14,"last_clean_end":73,"up_from":77,"up_thru":97,"down_at":74,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6810","nonce":1955801845},{"type":"v1","addr":"172.21.15.18:6811","nonce":1955801845}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6812","nonce":1955801845},{"type":"v1","addr":"172.21.15.18:6813","nonce":1955801845}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6816","nonce":1955801845},{"type":"v1","addr":"172.21.15.18:6817","nonce":1955801845}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6814","nonce":1955801845},{"type":"v1","addr":"172.21.15.18:6815","nonce":1955801845}]},"public_addr":"172.21.15.18:6811/1955801845","cluster_addr":"172.21.15.18:6813/1955801845","heartbeat_back_addr":"172.21.15.18:6817/1955801845","heartbeat_front_addr":"172.21.15.18:6815/1955801845","state":["exists","up"]},{"osd":2,"uuid":"5bb1137d-5a3c-40bc-ae08-fc792b8ee547","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":19,"last_clean_end":78,"up_from":82,"up_thru":97,"down_at":79,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6818","nonce":1361831750},{"type":"v1","addr":"172.21.15.18:6819","nonce":1361831750}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6820","nonce":1361831750},{"type":"v1","addr":"172.21.15.18:6821","nonce":1361831750}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6824","nonce":1361831750},{"type":"v1","addr":"172.21.15.18:6825","nonce":1361831750}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6822","nonce":1361831750},{"type":"v1","addr":"172.21.15.18:6823","nonce":1361831750}]},"public_addr":"172.21.15.18:6819/1361831750","cluster_addr":"172.21.15.18:6821/1361831750","heartbeat_back_addr":"172.21.15.18:6825/1361831750","heartbeat_front_addr":"172.21.15.18:6823/1361831750","state":["exists","up"]},{"osd":3,"uuid":"a5140a33-1e90-4980-aa9a-b0afe6b155f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":26,"last_clean_end":83,"up_from":87,"up_thru":97,"down_at":84,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6800","nonce":1952273925},{"type":"v1","addr":"172.21.15.186:6801","nonce":1952273925}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6802","nonce":1952273925},{"type":"v1","addr":"172.21.15.186:6803","nonce":1952273925}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6806","nonce":1952273925},{"type":"v1","addr":"172.21.15.186:6807","nonce":1952273925}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6804","nonce":1952273925},{"type":"v1","addr":"172.21.15.186:6805","nonce":1952273925}]},"public_addr":"172.21.15.186:6801/1952273925","cluster_addr":"172.21.15.186:6803/1952273925","heartbeat_back_addr":"172.21.15.186:6807/1952273925","heartbeat_front_addr":"172.21.15.186:6805/1952273925","state":["exists","up"]},{"osd":4,"uuid":"d6b2c6b3-6c78-4261-bcbe-c33358407198","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":34,"last_clean_end":88,"up_from":92,"up_thru":97,"down_at":89,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6808","nonce":2123883537},{"type":"v1","addr":"172.21.15.186:6809","nonce":2123883537}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6810","nonce":2123883537},{"type":"v1","addr":"172.21.15.186:6811","nonce":2123883537}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6814","nonce":2123883537},{"type":"v1","addr":"172.21.15.186:6815","nonce":2123883537}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6812","nonce":2123883537},{"type":"v1","addr":"172.21.15.186:6813","nonce":2123883537}]},"public_addr":"172.21.15.186:6809/2123883537","cluster_addr":"172.21.15.186:6811/2123883537","heartbeat_back_addr":"172.21.15.186:6815/2123883537","heartbeat_front_addr":"172.21.15.186:6813/2123883537","state":["exists","up"]},{"osd":5,"uuid":"7b5c9f17-6306-4fb5-aa77-fc9a73bcc663","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":39,"last_clean_end":93,"up_from":97,"up_thru":97,"down_at":94,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6816","nonce":2849575493},{"type":"v1","addr":"172.21.15.186:6817","nonce":2849575493}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6818","nonce":2849575493},{"type":"v1","addr":"172.21.15.186:6819","nonce":2849575493}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6822","nonce":2849575493},{"type":"v1","addr":"172.21.15.186:6823","nonce":2849575493}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6820","nonce":2849575493},{"type":"v1","addr":"172.21.15.186:6821","nonce":2849575493}]},"public_addr":"172.21.15.186:6817/2849575493","cluster_addr":"172.21.15.186:6819/2849575493","heartbeat_back_addr":"172.21.15.186:6823/2849575493","heartbeat_front_addr":"172.21.15.186:6821/2849575493","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-12T20:47:21.231060+0000","dead_epoch":49},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-12T20:47:57.699205+0000","dead_epoch":73},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-12T20:48:35.096648+0000","dead_epoch":78},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-12T20:49:08.252810+0000","dead_epoch":83},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-12T20:49:41.102187+0000","dead_epoch":88},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-12T20:50:16.009813+0000","dead_epoch":93}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.8","mappings":[{"from":3,"to":2}]},{"pgid":"2.a","mappings":[{"from":3,"to":0}]},{"pgid":"2.1d","mappings":[{"from":3,"to":2}]},{"pgid":"3.6","mappings":[{"from":1,"to":5}]},{"pgid":"3.a","mappings":[{"from":4,"to":0}]},{"pgid":"3.e","mappings":[{"from":1,"to":2}]},{"pgid":"3.11","mappings":[{"from":3,"to":2}]},{"pgid":"3.14","mappings":[{"from":3,"to":0}]},{"pgid":"3.1a","mappings":[{"from":4,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.18:6826/1564065434":"2024-04-13T21:06:59.717116+0000","172.21.15.186:0/825752072":"2024-04-13T20:57:12.501044+0000","172.21.15.18:6800/1391635591":"2024-04-13T20:45:44.776072+0000","172.21.15.18:0/4018777476":"2024-04-13T20:45:44.776072+0000","172.21.15.186:6824/1645616336":"2024-04-13T20:52:14.968672+0000","172.21.15.18:0/3387911038":"2024-04-13T20:44:15.143139+0000","172.21.15.18:0/714775767":"2024-04-13T20:54:59.478287+0000","172.21.15.18:6801/3048894687":"2024-04-13T20:54:59.478287+0000","172.21.15.18:0/2171204556":"2024-04-13T21:01:39.724018+0000","172.21.15.186:0/2147813742":"2024-04-13T20:57:12.501044+0000","172.21.15.18:6829/722443353":"2024-04-13T21:06:59.717165+0000","172.21.15.18:0/2945432483":"2024-04-13T20:44:15.143139+0000","172.21.15.18:6827/1564065434":"2024-04-13T21:06:59.717116+0000","172.21.15.186:6829/2316349902":"2024-04-13T20:57:12.501044+0000","172.21.15.18:0/2956836690":"2024-04-13T20:54:59.478287+0000","172.21.15.18:0/1732042827":"2024-04-13T20:43:34.269559+0000","172.21.15.18:6800/3403971131":"2024-04-13T20:44:15.143139+0000","172.21.15.18:6800/1605950634":"2024-04-13T21:01:39.724018+0000","172.21.15.186:0/970502145":"2024-04-13T20:57:12.501044+0000","172.21.15.18:6800/3048894687":"2024-04-13T20:54:59.478287+0000","172.21.15.18:0/2330205732":"2024-04-13T20:54:59.478287+0000","172.21.15.18:0/3227425848":"2024-04-13T20:54:59.478287+0000","172.21.15.18:6801/1312802974":"2024-04-13T20:43:34.269559+0000","172.21.15.186:0/2459326506":"2024-04-13T20:57:12.501044+0000","172.21.15.18:0/3896210349":"2024-04-13T20:44:15.143139+0000","172.21.15.18:0/1720919549":"2024-04-13T20:45:44.776072+0000","172.21.15.18:0/766256564":"2024-04-13T20:45:44.776072+0000","172.21.15.18:6801/3403971131":"2024-04-13T20:44:15.143139+0000","172.21.15.18:6828/722443353":"2024-04-13T21:06:59.717165+0000","172.21.15.18:0/3295870133":"2024-04-13T20:43:34.269559+0000","172.21.15.18:6800/1312802974":"2024-04-13T20:43:34.269559+0000","172.21.15.186:6828/2316349902":"2024-04-13T20:57:12.501044+0000","172.21.15.186:6825/1645616336":"2024-04-13T20:52:14.968672+0000","172.21.15.18:0/3745234714":"2024-04-13T21:01:39.724018+0000","172.21.15.18:0/2356171563":"2024-04-13T21:01:39.724018+0000","172.21.15.18:6801/1391635591":"2024-04-13T20:45:44.776072+0000","172.21.15.18:0/2517252399":"2024-04-13T20:43:34.269559+0000","172.21.15.18:6801/1605950634":"2024-04-13T21:01:39.724018+0000","172.21.15.18:0/3861256640":"2024-04-13T21:01:39.724018+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-12T21:55:45.695 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:45.693+0000 7ff654000640 1 -- 172.21.15.18:0/555862095 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7ff63406eb70 msgr2=0x7ff634071020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:55:45.696 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:45.693+0000 7ff654000640 1 --2- 172.21.15.18:0/555862095 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7ff63406eb70 0x7ff634071020 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7ff6380048a0 tx=0x7ff638009da0 comp rx=0 tx=0).stop 2024-04-12T21:55:45.696 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:45.693+0000 7ff654000640 1 -- 172.21.15.18:0/555862095 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff64c0fd2d0 msgr2=0x7ff64c0fb190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:55:45.696 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:45.693+0000 7ff654000640 1 --2- 172.21.15.18:0/555862095 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff64c0fd2d0 0x7ff64c0fb190 secure :-1 s=READY pgs=1433 cs=0 l=1 rev1=1 crypto rx=0x7ff640002a50 tx=0x7ff640004600 comp rx=0 tx=0).stop 2024-04-12T21:55:45.696 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:45.693+0000 7ff654000640 1 -- 172.21.15.18:0/555862095 shutdown_connections 2024-04-12T21:55:45.696 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:45.693+0000 7ff654000640 1 --2- 172.21.15.18:0/555862095 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7ff63406eb70 0x7ff634071020 unknown :-1 s=CLOSED pgs=758 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:55:45.696 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:45.693+0000 7ff654000640 1 --2- 172.21.15.18:0/555862095 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff64c0fdbe0 0x7ff64c0fb6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:55:45.696 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:45.693+0000 7ff654000640 1 --2- 172.21.15.18:0/555862095 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff64c0fd2d0 0x7ff64c0fb190 unknown :-1 s=CLOSED pgs=1433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:55:45.696 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:45.693+0000 7ff654000640 1 -- 172.21.15.18:0/555862095 >> 172.21.15.18:0/555862095 conn(0x7ff64c0f6d70 msgr2=0x7ff64c0fe830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:55:45.696 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:45.693+0000 7ff654000640 1 -- 172.21.15.18:0/555862095 shutdown_connections 2024-04-12T21:55:45.696 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:45.693+0000 7ff654000640 1 -- 172.21.15.18:0/555862095 wait complete. 2024-04-12T21:55:45.711 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-12T20:48:37.529081+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 6, 'score_stable': 6, 'optimal_score': 0.5, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs.cephfs.meta', 'create_time': '2024-04-12T20:51:56.284414+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '44', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.309999942779541, 'score_stable': 1.309999942779541, 'optimal_score': 1, 'raw_score_acting': 1.309999942779541, 'raw_score_stable': 1.309999942779541, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs.cephfs.data', 'create_time': '2024-04-12T20:51:56.927802+0000', 'flags': 131073, 'flags_names': 'hashpspool,bulk', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '45', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-12T21:55:45.712 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-12T21:55:45.785 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:55:45 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-12T21:55:45.785 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:55:45 smithi186 ceph-mon[86379]: from='client.? 172.21.15.18:0/1129949094' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 36, "format": "json"}]: dispatch 2024-04-12T21:55:45.785 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:55:45 smithi186 ceph-mon[86379]: from='client.? 172.21.15.18:0/1109916803' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 37, "format": "json"}]: dispatch 2024-04-12T21:55:45.900 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:45.898+0000 7fd9c9200640 1 -- 172.21.15.18:0/3470419919 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd9c41043c0 msgr2=0x7fd9c4104790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:55:45.900 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:45.898+0000 7fd9c9200640 1 --2- 172.21.15.18:0/3470419919 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd9c41043c0 0x7fd9c4104790 secure :-1 s=READY pgs=1434 cs=0 l=1 rev1=1 crypto rx=0x7fd9b40048d0 tx=0x7fd9b4009da0 comp rx=0 tx=0).stop 2024-04-12T21:55:45.900 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:45.899+0000 7fd9c9200640 1 -- 172.21.15.18:0/3470419919 shutdown_connections 2024-04-12T21:55:45.900 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:45.899+0000 7fd9c9200640 1 --2- 172.21.15.18:0/3470419919 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd9c4104d60 0x7fd9c40fe3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:55:45.900 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:45.899+0000 7fd9c9200640 1 --2- 172.21.15.18:0/3470419919 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd9c41043c0 0x7fd9c4104790 unknown :-1 s=CLOSED pgs=1434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:55:45.901 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:45.899+0000 7fd9c9200640 1 -- 172.21.15.18:0/3470419919 >> 172.21.15.18:0/3470419919 conn(0x7fd9c40f9fc0 msgr2=0x7fd9c40fc3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:55:45.901 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:45.899+0000 7fd9c9200640 1 -- 172.21.15.18:0/3470419919 shutdown_connections 2024-04-12T21:55:45.901 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:45.899+0000 7fd9c9200640 1 -- 172.21.15.18:0/3470419919 wait complete. 2024-04-12T21:55:45.902 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:45.899+0000 7fd9c9200640 1 Processor -- start 2024-04-12T21:55:45.902 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:45.900+0000 7fd9c9200640 1 -- start start 2024-04-12T21:55:45.902 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:45.900+0000 7fd9c9200640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd9c41043c0 0x7fd9c4114ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:55:45.902 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:45.900+0000 7fd9c9200640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd9c4104d60 0x7fd9c4115410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:55:45.902 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:45.900+0000 7fd9c9200640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9c4115a10 con 0x7fd9c4104d60 2024-04-12T21:55:45.902 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:45.900+0000 7fd9c9200640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9c410f060 con 0x7fd9c41043c0 2024-04-12T21:55:45.903 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:45.900+0000 7fd9c1400640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd9c4104d60 0x7fd9c4115410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:55:45.903 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:45.900+0000 7fd9c1400640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd9c4104d60 0x7fd9c4115410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:43502/0 (socket says 172.21.15.18:43502) 2024-04-12T21:55:45.903 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:45.900+0000 7fd9c1400640 1 -- 172.21.15.18:0/1629551101 learned_addr learned my addr 172.21.15.18:0/1629551101 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:55:45.903 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:45.901+0000 7fd9c1e00640 1 --2- 172.21.15.18:0/1629551101 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd9c41043c0 0x7fd9c4114ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:55:45.903 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:45.901+0000 7fd9c1400640 1 -- 172.21.15.18:0/1629551101 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd9c41043c0 msgr2=0x7fd9c4114ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:55:45.903 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:45.901+0000 7fd9c1400640 1 --2- 172.21.15.18:0/1629551101 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd9c41043c0 0x7fd9c4114ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:55:45.903 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:45.901+0000 7fd9c1400640 1 -- 172.21.15.18:0/1629551101 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9b4005040 con 0x7fd9c4104d60 2024-04-12T21:55:45.903 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:45.901+0000 7fd9c1400640 1 --2- 172.21.15.18:0/1629551101 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd9c4104d60 0x7fd9c4115410 secure :-1 s=READY pgs=1435 cs=0 l=1 rev1=1 crypto rx=0x7fd9b0002a50 tx=0x7fd9b0004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:55:45.904 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:45.901+0000 7fd9ba000640 1 -- 172.21.15.18:0/1629551101 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd9b0007c30 con 0x7fd9c4104d60 2024-04-12T21:55:45.904 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:45.902+0000 7fd9c9200640 1 -- 172.21.15.18:0/1629551101 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd9c410f340 con 0x7fd9c4104d60 2024-04-12T21:55:45.904 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:45.902+0000 7fd9c9200640 1 -- 172.21.15.18:0/1629551101 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd9c410f860 con 0x7fd9c4104d60 2024-04-12T21:55:45.904 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:45.902+0000 7fd9ba000640 1 -- 172.21.15.18:0/1629551101 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd9b000f040 con 0x7fd9c4104d60 2024-04-12T21:55:45.904 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:45.902+0000 7fd9ba000640 1 -- 172.21.15.18:0/1629551101 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd9b000db50 con 0x7fd9c4104d60 2024-04-12T21:55:45.904 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:45.903+0000 7fd9c9200640 1 -- 172.21.15.18:0/1629551101 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd9a4005210 con 0x7fd9c4104d60 2024-04-12T21:55:45.906 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:45.905+0000 7fd9ba000640 1 -- 172.21.15.18:0/1629551101 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7fd9b001c400 con 0x7fd9c4104d60 2024-04-12T21:55:45.907 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:45.906+0000 7fd9ba000640 1 --2- 172.21.15.18:0/1629551101 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fd9a806eb70 0x7fd9a8071020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:55:45.907 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:45.906+0000 7fd9ba000640 1 -- 172.21.15.18:0/1629551101 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 6757+0+0 (secure 0 0 0) 0x7fd9b008e250 con 0x7fd9c4104d60 2024-04-12T21:55:45.907 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:45.906+0000 7fd9c1e00640 1 --2- 172.21.15.18:0/1629551101 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fd9a806eb70 0x7fd9a8071020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:55:45.908 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:45.907+0000 7fd9c1e00640 1 --2- 172.21.15.18:0/1629551101 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fd9a806eb70 0x7fd9a8071020 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7fd9b4003080 tx=0x7fd9b4009da0 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:55:45.912 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:45.911+0000 7fd9ba000640 1 -- 172.21.15.18:0/1629551101 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7fd9b0056e50 con 0x7fd9c4104d60 2024-04-12T21:55:46.115 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:46.113+0000 7fd9c9200640 1 -- 172.21.15.18:0/1629551101 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fd9a4005a40 con 0x7fd9c4104d60 2024-04-12T21:55:46.115 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:46.114+0000 7fd9ba000640 1 -- 172.21.15.18:0/1629551101 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v104) v1 ==== 93+0+10 (secure 0 0 0) 0x7fd9b005a510 con 0x7fd9c4104d60 2024-04-12T21:55:46.116 INFO:teuthology.orchestra.run.smithi018.stdout:pg_num: 1 2024-04-12T21:55:46.118 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:46.117+0000 7fd9c9200640 1 -- 172.21.15.18:0/1629551101 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fd9a806eb70 msgr2=0x7fd9a8071020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:55:46.118 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:46.117+0000 7fd9c9200640 1 --2- 172.21.15.18:0/1629551101 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fd9a806eb70 0x7fd9a8071020 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7fd9b4003080 tx=0x7fd9b4009da0 comp rx=0 tx=0).stop 2024-04-12T21:55:46.118 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:46.117+0000 7fd9c9200640 1 -- 172.21.15.18:0/1629551101 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd9c4104d60 msgr2=0x7fd9c4115410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:55:46.119 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:46.117+0000 7fd9c9200640 1 --2- 172.21.15.18:0/1629551101 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd9c4104d60 0x7fd9c4115410 secure :-1 s=READY pgs=1435 cs=0 l=1 rev1=1 crypto rx=0x7fd9b0002a50 tx=0x7fd9b0004600 comp rx=0 tx=0).stop 2024-04-12T21:55:46.119 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:46.117+0000 7fd9c9200640 1 -- 172.21.15.18:0/1629551101 shutdown_connections 2024-04-12T21:55:46.119 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:46.117+0000 7fd9c9200640 1 --2- 172.21.15.18:0/1629551101 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fd9a806eb70 0x7fd9a8071020 unknown :-1 s=CLOSED pgs=759 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:55:46.119 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:46.117+0000 7fd9c9200640 1 --2- 172.21.15.18:0/1629551101 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd9c4104d60 0x7fd9c4115410 unknown :-1 s=CLOSED pgs=1435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:55:46.119 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:46.117+0000 7fd9c9200640 1 --2- 172.21.15.18:0/1629551101 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd9c41043c0 0x7fd9c4114ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:55:46.119 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:46.117+0000 7fd9c9200640 1 -- 172.21.15.18:0/1629551101 >> 172.21.15.18:0/1629551101 conn(0x7fd9c40f9fc0 msgr2=0x7fd9c40fae90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:55:46.119 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:46.117+0000 7fd9c9200640 1 -- 172.21.15.18:0/1629551101 shutdown_connections 2024-04-12T21:55:46.119 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:46.117+0000 7fd9c9200640 1 -- 172.21.15.18:0/1629551101 wait complete. 2024-04-12T21:55:46.134 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs.cephfs.meta pg_num 2024-04-12T21:55:46.339 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:46.337+0000 7feed5600640 1 -- 172.21.15.18:0/1549138849 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7feed00fd3f0 msgr2=0x7feed00fd7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:55:46.339 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:46.337+0000 7feed5600640 1 --2- 172.21.15.18:0/1549138849 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7feed00fd3f0 0x7feed00fd7c0 secure :-1 s=READY pgs=1436 cs=0 l=1 rev1=1 crypto rx=0x7feec00048d0 tx=0x7feec0009da0 comp rx=0 tx=0).stop 2024-04-12T21:55:46.339 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:46.337+0000 7feed5600640 1 -- 172.21.15.18:0/1549138849 shutdown_connections 2024-04-12T21:55:46.339 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:46.337+0000 7feed5600640 1 --2- 172.21.15.18:0/1549138849 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7feed00fdd00 0x7feed0106460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:55:46.339 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:46.337+0000 7feed5600640 1 --2- 172.21.15.18:0/1549138849 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7feed00fd3f0 0x7feed00fd7c0 unknown :-1 s=CLOSED pgs=1436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:55:46.339 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:46.337+0000 7feed5600640 1 -- 172.21.15.18:0/1549138849 >> 172.21.15.18:0/1549138849 conn(0x7feed00f6e70 msgr2=0x7feed00f9280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:55:46.339 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:46.337+0000 7feed5600640 1 -- 172.21.15.18:0/1549138849 shutdown_connections 2024-04-12T21:55:46.339 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:46.337+0000 7feed5600640 1 -- 172.21.15.18:0/1549138849 wait complete. 2024-04-12T21:55:46.340 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:46.338+0000 7feed5600640 1 Processor -- start 2024-04-12T21:55:46.341 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:46.339+0000 7feed5600640 1 -- start start 2024-04-12T21:55:46.341 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:46.339+0000 7feed5600640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7feed00fd3f0 0x7feed0112f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:55:46.341 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:46.339+0000 7feed5600640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7feed00fdd00 0x7feed01134c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:55:46.341 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:46.339+0000 7feed5600640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feed0113ac0 con 0x7feed00fd3f0 2024-04-12T21:55:46.341 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:46.339+0000 7feed5600640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feed0113c30 con 0x7feed00fdd00 2024-04-12T21:55:46.341 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:46.339+0000 7feecea00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7feed00fd3f0 0x7feed0112f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:55:46.341 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:46.339+0000 7feecea00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7feed00fd3f0 0x7feed0112f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:43520/0 (socket says 172.21.15.18:43520) 2024-04-12T21:55:46.341 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:46.339+0000 7feecea00640 1 -- 172.21.15.18:0/4045994680 learned_addr learned my addr 172.21.15.18:0/4045994680 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:55:46.342 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:46.340+0000 7feece000640 1 --2- 172.21.15.18:0/4045994680 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7feed00fdd00 0x7feed01134c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:55:46.342 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:46.340+0000 7feecea00640 1 -- 172.21.15.18:0/4045994680 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7feed00fdd00 msgr2=0x7feed01134c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:55:46.342 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:46.340+0000 7feecea00640 1 --2- 172.21.15.18:0/4045994680 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7feed00fdd00 0x7feed01134c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:55:46.342 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:46.340+0000 7feecea00640 1 -- 172.21.15.18:0/4045994680 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feec0005040 con 0x7feed00fd3f0 2024-04-12T21:55:46.343 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:46.340+0000 7feecea00640 1 --2- 172.21.15.18:0/4045994680 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7feed00fd3f0 0x7feed0112f80 secure :-1 s=READY pgs=1437 cs=0 l=1 rev1=1 crypto rx=0x7feec00047f0 tx=0x7feec0009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:55:46.343 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:46.340+0000 7feec6a00640 1 -- 172.21.15.18:0/4045994680 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7feec00094c0 con 0x7feed00fd3f0 2024-04-12T21:55:46.343 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:46.340+0000 7feec6a00640 1 -- 172.21.15.18:0/4045994680 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7feec0032e50 con 0x7feed00fd3f0 2024-04-12T21:55:46.343 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:46.340+0000 7feed5600640 1 -- 172.21.15.18:0/4045994680 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feed010d260 con 0x7feed00fd3f0 2024-04-12T21:55:46.343 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:46.340+0000 7feec6a00640 1 -- 172.21.15.18:0/4045994680 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7feec00433b0 con 0x7feed00fd3f0 2024-04-12T21:55:46.343 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:46.341+0000 7feed5600640 1 -- 172.21.15.18:0/4045994680 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feed010d720 con 0x7feed00fd3f0 2024-04-12T21:55:46.343 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:46.342+0000 7feed5600640 1 -- 172.21.15.18:0/4045994680 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feed0065db0 con 0x7feed00fd3f0 2024-04-12T21:55:46.344 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:46.343+0000 7feec6a00640 1 -- 172.21.15.18:0/4045994680 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7feec002f070 con 0x7feed00fd3f0 2024-04-12T21:55:46.345 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:46.343+0000 7feec6a00640 1 --2- 172.21.15.18:0/4045994680 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7feeb406eb80 0x7feeb4071030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:55:46.345 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:46.344+0000 7feec6a00640 1 -- 172.21.15.18:0/4045994680 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 6757+0+0 (secure 0 0 0) 0x7feec00b0840 con 0x7feed00fd3f0 2024-04-12T21:55:46.345 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:46.344+0000 7feece000640 1 --2- 172.21.15.18:0/4045994680 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7feeb406eb80 0x7feeb4071030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:55:46.346 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:46.345+0000 7feece000640 1 --2- 172.21.15.18:0/4045994680 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7feeb406eb80 0x7feeb4071030 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7feed010e880 tx=0x7feebc00b040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:55:46.349 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:46.348+0000 7feec6a00640 1 -- 172.21.15.18:0/4045994680 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7feec0079440 con 0x7feed00fd3f0 2024-04-12T21:55:46.548 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:46.547+0000 7feed5600640 1 -- 172.21.15.18:0/4045994680 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"} v 0) v1 -- 0x7feed004bcc0 con 0x7feed00fd3f0 2024-04-12T21:55:46.549 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:46.548+0000 7feec6a00640 1 -- 172.21.15.18:0/4045994680 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"}]=0 v104) v1 ==== 107+0+11 (secure 0 0 0) 0x7feec007cb00 con 0x7feed00fd3f0 2024-04-12T21:55:46.549 INFO:teuthology.orchestra.run.smithi018.stdout:pg_num: 32 2024-04-12T21:55:46.552 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:46.550+0000 7feed5600640 1 -- 172.21.15.18:0/4045994680 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7feeb406eb80 msgr2=0x7feeb4071030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:55:46.552 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:46.550+0000 7feed5600640 1 --2- 172.21.15.18:0/4045994680 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7feeb406eb80 0x7feeb4071030 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7feed010e880 tx=0x7feebc00b040 comp rx=0 tx=0).stop 2024-04-12T21:55:46.552 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:46.550+0000 7feed5600640 1 -- 172.21.15.18:0/4045994680 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7feed00fd3f0 msgr2=0x7feed0112f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:55:46.552 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:46.550+0000 7feed5600640 1 --2- 172.21.15.18:0/4045994680 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7feed00fd3f0 0x7feed0112f80 secure :-1 s=READY pgs=1437 cs=0 l=1 rev1=1 crypto rx=0x7feec00047f0 tx=0x7feec0009da0 comp rx=0 tx=0).stop 2024-04-12T21:55:46.552 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:46.550+0000 7feed5600640 1 -- 172.21.15.18:0/4045994680 shutdown_connections 2024-04-12T21:55:46.552 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:46.550+0000 7feed5600640 1 --2- 172.21.15.18:0/4045994680 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7feeb406eb80 0x7feeb4071030 unknown :-1 s=CLOSED pgs=760 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:55:46.552 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:46.550+0000 7feed5600640 1 --2- 172.21.15.18:0/4045994680 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7feed00fdd00 0x7feed01134c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:55:46.552 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:46.550+0000 7feed5600640 1 --2- 172.21.15.18:0/4045994680 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7feed00fd3f0 0x7feed0112f80 unknown :-1 s=CLOSED pgs=1437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:55:46.552 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:46.550+0000 7feed5600640 1 -- 172.21.15.18:0/4045994680 >> 172.21.15.18:0/4045994680 conn(0x7feed00f6e70 msgr2=0x7feed00f8390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:55:46.552 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:46.551+0000 7feed5600640 1 -- 172.21.15.18:0/4045994680 shutdown_connections 2024-04-12T21:55:46.552 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:46.551+0000 7feed5600640 1 -- 172.21.15.18:0/4045994680 wait complete. 2024-04-12T21:55:46.566 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:55:46 smithi018 ceph-mon[112590]: from='client.? 172.21.15.18:0/555862095' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-12T21:55:46.566 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:55:46 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-12T21:55:46.566 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:55:46 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-12T21:55:46.567 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:55:46 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' 2024-04-12T21:55:46.567 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:55:46 smithi018 ceph-mon[112590]: from='client.? 172.21.15.18:0/1629551101' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-12T21:55:46.567 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:55:46 smithi018 ceph-mon[112590]: pgmap v1685: 65 pgs: 65 active+clean; 257 MiB data, 976 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:55:46.569 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs.cephfs.data pg_num 2024-04-12T21:55:46.769 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:46.767+0000 7f3443200640 1 -- 172.21.15.18:0/3329676673 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3444102010 msgr2=0x7f34441023e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:55:46.769 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:46.767+0000 7f3443200640 1 --2- 172.21.15.18:0/3329676673 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3444102010 0x7f34441023e0 secure :-1 s=READY pgs=1438 cs=0 l=1 rev1=1 crypto rx=0x7f3434009c50 tx=0x7f3434004d10 comp rx=0 tx=0).stop 2024-04-12T21:55:46.770 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:46.768+0000 7f3443200640 1 -- 172.21.15.18:0/3329676673 shutdown_connections 2024-04-12T21:55:46.770 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:46.768+0000 7f3443200640 1 --2- 172.21.15.18:0/3329676673 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f34441029b0 0x7f34440fc040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:55:46.770 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:46.768+0000 7f3443200640 1 --2- 172.21.15.18:0/3329676673 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3444102010 0x7f34441023e0 unknown :-1 s=CLOSED pgs=1438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:55:46.770 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:46.768+0000 7f3443200640 1 -- 172.21.15.18:0/3329676673 >> 172.21.15.18:0/3329676673 conn(0x7f34440f7c20 msgr2=0x7f34440fa030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:55:46.770 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:46.768+0000 7f3443200640 1 -- 172.21.15.18:0/3329676673 shutdown_connections 2024-04-12T21:55:46.770 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:46.768+0000 7f3443200640 1 -- 172.21.15.18:0/3329676673 wait complete. 2024-04-12T21:55:46.771 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:46.769+0000 7f3443200640 1 Processor -- start 2024-04-12T21:55:46.771 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:46.769+0000 7f3443200640 1 -- start start 2024-04-12T21:55:46.771 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:46.769+0000 7f3443200640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3444102010 0x7f3444114cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:55:46.772 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:46.769+0000 7f3443200640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f34441029b0 0x7f3444115200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:55:46.772 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:46.769+0000 7f3443200640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3444115800 con 0x7f34441029b0 2024-04-12T21:55:46.772 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:46.769+0000 7f3443200640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3444115970 con 0x7f3444102010 2024-04-12T21:55:46.772 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:46.770+0000 7f3441400640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f34441029b0 0x7f3444115200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:55:46.772 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:46.770+0000 7f3441400640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f34441029b0 0x7f3444115200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:43536/0 (socket says 172.21.15.18:43536) 2024-04-12T21:55:46.772 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:46.770+0000 7f3441400640 1 -- 172.21.15.18:0/39114357 learned_addr learned my addr 172.21.15.18:0/39114357 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:55:46.772 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:46.770+0000 7f3441e00640 1 --2- 172.21.15.18:0/39114357 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3444102010 0x7f3444114cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:55:46.772 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:46.770+0000 7f3441400640 1 -- 172.21.15.18:0/39114357 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3444102010 msgr2=0x7f3444114cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:55:46.772 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:46.770+0000 7f3441400640 1 --2- 172.21.15.18:0/39114357 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3444102010 0x7f3444114cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:55:46.772 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:46.770+0000 7f3441400640 1 -- 172.21.15.18:0/39114357 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34340098b0 con 0x7f34441029b0 2024-04-12T21:55:46.772 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:46.770+0000 7f3441400640 1 --2- 172.21.15.18:0/39114357 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f34441029b0 0x7f3444115200 secure :-1 s=READY pgs=1439 cs=0 l=1 rev1=1 crypto rx=0x7f342c00c830 tx=0x7f342c004430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:55:46.773 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:46.771+0000 7f343a000640 1 -- 172.21.15.18:0/39114357 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f342c00a850 con 0x7f34441029b0 2024-04-12T21:55:46.773 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:46.771+0000 7f3443200640 1 -- 172.21.15.18:0/39114357 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f344410f000 con 0x7f34441029b0 2024-04-12T21:55:46.773 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:46.771+0000 7f343a000640 1 -- 172.21.15.18:0/39114357 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f342c00ae80 con 0x7f34441029b0 2024-04-12T21:55:46.773 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:46.771+0000 7f343a000640 1 -- 172.21.15.18:0/39114357 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f342c012ea0 con 0x7f34441029b0 2024-04-12T21:55:46.773 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:46.771+0000 7f3443200640 1 -- 172.21.15.18:0/39114357 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f344410f5a0 con 0x7f34441029b0 2024-04-12T21:55:46.773 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:46.772+0000 7f3443200640 1 -- 172.21.15.18:0/39114357 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f34440fd780 con 0x7f34441029b0 2024-04-12T21:55:46.774 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:46.773+0000 7f343a000640 1 -- 172.21.15.18:0/39114357 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7f342c012400 con 0x7f34441029b0 2024-04-12T21:55:46.775 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:46.774+0000 7f343a000640 1 --2- 172.21.15.18:0/39114357 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f342806eb80 0x7f3428071030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:55:46.775 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:46.774+0000 7f343a000640 1 -- 172.21.15.18:0/39114357 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 6757+0+0 (secure 0 0 0) 0x7f342c08ecd0 con 0x7f34441029b0 2024-04-12T21:55:46.776 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:46.774+0000 7f3441e00640 1 --2- 172.21.15.18:0/39114357 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f342806eb80 0x7f3428071030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:55:46.776 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:46.775+0000 7f3441e00640 1 --2- 172.21.15.18:0/39114357 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f342806eb80 0x7f3428071030 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7f3434004a30 tx=0x7f343402e040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:55:46.780 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:46.779+0000 7f343a000640 1 -- 172.21.15.18:0/39114357 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f342c057950 con 0x7f34441029b0 2024-04-12T21:55:46.785 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:55:46 smithi186 ceph-mon[86379]: from='client.? 172.21.15.18:0/555862095' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-12T21:55:46.785 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:55:46 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-12T21:55:46.785 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:55:46 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-12T21:55:46.785 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:55:46 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' 2024-04-12T21:55:46.785 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:55:46 smithi186 ceph-mon[86379]: from='client.? 172.21.15.18:0/1629551101' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-12T21:55:46.785 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:55:46 smithi186 ceph-mon[86379]: pgmap v1685: 65 pgs: 65 active+clean; 257 MiB data, 976 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:55:46.983 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:46.982+0000 7f3443200640 1 -- 172.21.15.18:0/39114357 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"} v 0) v1 -- 0x7f34441091c0 con 0x7f34441029b0 2024-04-12T21:55:46.985 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:46.983+0000 7f343a000640 1 -- 172.21.15.18:0/39114357 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"}]=0 v104) v1 ==== 107+0+11 (secure 0 0 0) 0x7f342c0126b0 con 0x7f34441029b0 2024-04-12T21:55:46.985 INFO:teuthology.orchestra.run.smithi018.stdout:pg_num: 32 2024-04-12T21:55:46.989 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:46.987+0000 7f3443200640 1 -- 172.21.15.18:0/39114357 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f342806eb80 msgr2=0x7f3428071030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:55:46.989 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:46.987+0000 7f3443200640 1 --2- 172.21.15.18:0/39114357 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f342806eb80 0x7f3428071030 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7f3434004a30 tx=0x7f343402e040 comp rx=0 tx=0).stop 2024-04-12T21:55:46.989 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:46.987+0000 7f3443200640 1 -- 172.21.15.18:0/39114357 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f34441029b0 msgr2=0x7f3444115200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:55:46.989 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:46.987+0000 7f3443200640 1 --2- 172.21.15.18:0/39114357 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f34441029b0 0x7f3444115200 secure :-1 s=READY pgs=1439 cs=0 l=1 rev1=1 crypto rx=0x7f342c00c830 tx=0x7f342c004430 comp rx=0 tx=0).stop 2024-04-12T21:55:46.989 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:46.988+0000 7f3443200640 1 -- 172.21.15.18:0/39114357 shutdown_connections 2024-04-12T21:55:46.989 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:46.988+0000 7f3443200640 1 --2- 172.21.15.18:0/39114357 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f342806eb80 0x7f3428071030 unknown :-1 s=CLOSED pgs=761 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:55:46.989 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:46.988+0000 7f3443200640 1 --2- 172.21.15.18:0/39114357 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f34441029b0 0x7f3444115200 unknown :-1 s=CLOSED pgs=1439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:55:46.989 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:46.988+0000 7f3443200640 1 --2- 172.21.15.18:0/39114357 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3444102010 0x7f3444114cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:55:46.989 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:46.988+0000 7f3443200640 1 -- 172.21.15.18:0/39114357 >> 172.21.15.18:0/39114357 conn(0x7f34440f7c20 msgr2=0x7f3444071140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:55:46.989 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:46.988+0000 7f3443200640 1 -- 172.21.15.18:0/39114357 shutdown_connections 2024-04-12T21:55:46.990 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:46.988+0000 7f3443200640 1 -- 172.21.15.18:0/39114357 wait complete. 2024-04-12T21:55:47.005 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-12T21:55:47.193 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:47.191+0000 7f289ae00640 1 -- 172.21.15.18:0/3845592516 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f289406f810 msgr2=0x7f289406fbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:55:47.193 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:47.191+0000 7f289ae00640 1 --2- 172.21.15.18:0/3845592516 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f289406f810 0x7f289406fbe0 secure :-1 s=READY pgs=1440 cs=0 l=1 rev1=1 crypto rx=0x7f2888009c50 tx=0x7f2888004d10 comp rx=0 tx=0).stop 2024-04-12T21:55:47.193 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:47.192+0000 7f289ae00640 1 -- 172.21.15.18:0/3845592516 shutdown_connections 2024-04-12T21:55:47.193 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:47.192+0000 7f289ae00640 1 --2- 172.21.15.18:0/3845592516 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2894070120 0x7f28941083e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:55:47.193 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:47.192+0000 7f289ae00640 1 --2- 172.21.15.18:0/3845592516 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f289406f810 0x7f289406fbe0 unknown :-1 s=CLOSED pgs=1440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:55:47.193 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:47.192+0000 7f289ae00640 1 -- 172.21.15.18:0/3845592516 >> 172.21.15.18:0/3845592516 conn(0x7f28940f7c60 msgr2=0x7f28940fa070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:55:47.193 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:47.192+0000 7f289ae00640 1 -- 172.21.15.18:0/3845592516 shutdown_connections 2024-04-12T21:55:47.194 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:47.192+0000 7f289ae00640 1 -- 172.21.15.18:0/3845592516 wait complete. 2024-04-12T21:55:47.195 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:47.192+0000 7f289ae00640 1 Processor -- start 2024-04-12T21:55:47.196 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:47.193+0000 7f289ae00640 1 -- start start 2024-04-12T21:55:47.196 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:47.193+0000 7f289ae00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f289406f810 0x7f2894114da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:55:47.196 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:47.193+0000 7f289ae00640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2894070120 0x7f28941152e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:55:47.196 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:47.193+0000 7f289ae00640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28941158e0 con 0x7f289406f810 2024-04-12T21:55:47.196 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:47.193+0000 7f289ae00640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f289410ef30 con 0x7f2894070120 2024-04-12T21:55:47.196 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:47.193+0000 7f2899a00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f289406f810 0x7f2894114da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:55:47.196 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:47.193+0000 7f2899a00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f289406f810 0x7f2894114da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:43556/0 (socket says 172.21.15.18:43556) 2024-04-12T21:55:47.196 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:47.193+0000 7f2899a00640 1 -- 172.21.15.18:0/3803902058 learned_addr learned my addr 172.21.15.18:0/3803902058 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:55:47.196 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:47.194+0000 7f2899000640 1 --2- 172.21.15.18:0/3803902058 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2894070120 0x7f28941152e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:55:47.196 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:47.194+0000 7f2899a00640 1 -- 172.21.15.18:0/3803902058 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2894070120 msgr2=0x7f28941152e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:55:47.196 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:47.194+0000 7f2899a00640 1 --2- 172.21.15.18:0/3803902058 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2894070120 0x7f28941152e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:55:47.196 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:47.194+0000 7f2899a00640 1 -- 172.21.15.18:0/3803902058 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28880098b0 con 0x7f289406f810 2024-04-12T21:55:47.196 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:47.194+0000 7f2899000640 1 --2- 172.21.15.18:0/3803902058 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2894070120 0x7f28941152e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-12T21:55:47.196 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:47.194+0000 7f2899a00640 1 --2- 172.21.15.18:0/3803902058 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f289406f810 0x7f2894114da0 secure :-1 s=READY pgs=1441 cs=0 l=1 rev1=1 crypto rx=0x7f288802c1f0 tx=0x7f2888004d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:55:47.197 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:47.194+0000 7f2891600640 1 -- 172.21.15.18:0/3803902058 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f288803a070 con 0x7f289406f810 2024-04-12T21:55:47.197 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:47.194+0000 7f2891600640 1 -- 172.21.15.18:0/3803902058 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f2888004610 con 0x7f289406f810 2024-04-12T21:55:47.197 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:47.194+0000 7f289ae00640 1 -- 172.21.15.18:0/3803902058 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f289410f1b0 con 0x7f289406f810 2024-04-12T21:55:47.197 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:47.194+0000 7f2891600640 1 -- 172.21.15.18:0/3803902058 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f288803ebe0 con 0x7f289406f810 2024-04-12T21:55:47.197 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:47.194+0000 7f289ae00640 1 -- 172.21.15.18:0/3803902058 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f289410f6d0 con 0x7f289406f810 2024-04-12T21:55:47.197 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:47.195+0000 7f289ae00640 1 -- 172.21.15.18:0/3803902058 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2894070f50 con 0x7f289406f810 2024-04-12T21:55:47.198 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:47.197+0000 7f2891600640 1 -- 172.21.15.18:0/3803902058 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7f2888032070 con 0x7f289406f810 2024-04-12T21:55:47.198 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:47.197+0000 7f2891600640 1 --2- 172.21.15.18:0/3803902058 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f287c06eb80 0x7f287c071030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:55:47.198 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:47.198+0000 7f2891600640 1 -- 172.21.15.18:0/3803902058 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 6757+0+0 (secure 0 0 0) 0x7f28880b0df0 con 0x7f289406f810 2024-04-12T21:55:47.199 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:47.198+0000 7f2899000640 1 --2- 172.21.15.18:0/3803902058 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f287c06eb80 0x7f287c071030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:55:47.199 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:47.198+0000 7f2899000640 1 --2- 172.21.15.18:0/3803902058 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f287c06eb80 0x7f287c071030 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7f2894110660 tx=0x7f288000b040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:55:47.203 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:47.202+0000 7f2891600640 1 -- 172.21.15.18:0/3803902058 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f2888079be0 con 0x7f289406f810 2024-04-12T21:55:47.422 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:47.421+0000 7f289ae00640 1 -- 172.21.15.18:0/3803902058 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f289410f340 con 0x7f289406f810 2024-04-12T21:55:47.423 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:47.422+0000 7f2891600640 1 -- 172.21.15.18:0/3803902058 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 39 entries v104) v1 ==== 81+0+2318 (secure 0 0 0) 0x7f289410f340 con 0x7f289406f810 2024-04-12T21:55:47.424 INFO:teuthology.orchestra.run.smithi018.stdout:172.21.15.18:6826/1564065434 2024-04-13T21:06:59.717116+0000 2024-04-12T21:55:47.424 INFO:teuthology.orchestra.run.smithi018.stdout:172.21.15.186:0/825752072 2024-04-13T20:57:12.501044+0000 2024-04-12T21:55:47.424 INFO:teuthology.orchestra.run.smithi018.stdout:172.21.15.18:6800/1391635591 2024-04-13T20:45:44.776072+0000 2024-04-12T21:55:47.424 INFO:teuthology.orchestra.run.smithi018.stdout:172.21.15.18:0/4018777476 2024-04-13T20:45:44.776072+0000 2024-04-12T21:55:47.424 INFO:teuthology.orchestra.run.smithi018.stdout:172.21.15.186:6824/1645616336 2024-04-13T20:52:14.968672+0000 2024-04-12T21:55:47.424 INFO:teuthology.orchestra.run.smithi018.stdout:172.21.15.18:0/3387911038 2024-04-13T20:44:15.143139+0000 2024-04-12T21:55:47.424 INFO:teuthology.orchestra.run.smithi018.stdout:172.21.15.18:0/714775767 2024-04-13T20:54:59.478287+0000 2024-04-12T21:55:47.424 INFO:teuthology.orchestra.run.smithi018.stdout:172.21.15.18:6801/3048894687 2024-04-13T20:54:59.478287+0000 2024-04-12T21:55:47.424 INFO:teuthology.orchestra.run.smithi018.stdout:172.21.15.18:0/2171204556 2024-04-13T21:01:39.724018+0000 2024-04-12T21:55:47.424 INFO:teuthology.orchestra.run.smithi018.stdout:172.21.15.186:0/2147813742 2024-04-13T20:57:12.501044+0000 2024-04-12T21:55:47.424 INFO:teuthology.orchestra.run.smithi018.stdout:172.21.15.18:6829/722443353 2024-04-13T21:06:59.717165+0000 2024-04-12T21:55:47.424 INFO:teuthology.orchestra.run.smithi018.stdout:172.21.15.18:0/2945432483 2024-04-13T20:44:15.143139+0000 2024-04-12T21:55:47.424 INFO:teuthology.orchestra.run.smithi018.stdout:172.21.15.18:6827/1564065434 2024-04-13T21:06:59.717116+0000 2024-04-12T21:55:47.425 INFO:teuthology.orchestra.run.smithi018.stdout:172.21.15.186:6829/2316349902 2024-04-13T20:57:12.501044+0000 2024-04-12T21:55:47.425 INFO:teuthology.orchestra.run.smithi018.stdout:172.21.15.18:0/2956836690 2024-04-13T20:54:59.478287+0000 2024-04-12T21:55:47.425 INFO:teuthology.orchestra.run.smithi018.stdout:172.21.15.18:0/1732042827 2024-04-13T20:43:34.269559+0000 2024-04-12T21:55:47.425 INFO:teuthology.orchestra.run.smithi018.stdout:172.21.15.18:6800/3403971131 2024-04-13T20:44:15.143139+0000 2024-04-12T21:55:47.425 INFO:teuthology.orchestra.run.smithi018.stdout:172.21.15.18:6800/1605950634 2024-04-13T21:01:39.724018+0000 2024-04-12T21:55:47.425 INFO:teuthology.orchestra.run.smithi018.stdout:172.21.15.186:0/970502145 2024-04-13T20:57:12.501044+0000 2024-04-12T21:55:47.425 INFO:teuthology.orchestra.run.smithi018.stdout:172.21.15.18:6800/3048894687 2024-04-13T20:54:59.478287+0000 2024-04-12T21:55:47.425 INFO:teuthology.orchestra.run.smithi018.stdout:172.21.15.18:0/2330205732 2024-04-13T20:54:59.478287+0000 2024-04-12T21:55:47.425 INFO:teuthology.orchestra.run.smithi018.stdout:172.21.15.18:0/3227425848 2024-04-13T20:54:59.478287+0000 2024-04-12T21:55:47.425 INFO:teuthology.orchestra.run.smithi018.stdout:172.21.15.18:6801/1312802974 2024-04-13T20:43:34.269559+0000 2024-04-12T21:55:47.425 INFO:teuthology.orchestra.run.smithi018.stdout:172.21.15.186:0/2459326506 2024-04-13T20:57:12.501044+0000 2024-04-12T21:55:47.425 INFO:teuthology.orchestra.run.smithi018.stdout:172.21.15.18:0/3896210349 2024-04-13T20:44:15.143139+0000 2024-04-12T21:55:47.425 INFO:teuthology.orchestra.run.smithi018.stdout:172.21.15.18:0/1720919549 2024-04-13T20:45:44.776072+0000 2024-04-12T21:55:47.425 INFO:teuthology.orchestra.run.smithi018.stdout:172.21.15.18:0/766256564 2024-04-13T20:45:44.776072+0000 2024-04-12T21:55:47.425 INFO:teuthology.orchestra.run.smithi018.stdout:172.21.15.18:6801/3403971131 2024-04-13T20:44:15.143139+0000 2024-04-12T21:55:47.426 INFO:teuthology.orchestra.run.smithi018.stdout:172.21.15.18:6828/722443353 2024-04-13T21:06:59.717165+0000 2024-04-12T21:55:47.426 INFO:teuthology.orchestra.run.smithi018.stdout:172.21.15.18:0/3295870133 2024-04-13T20:43:34.269559+0000 2024-04-12T21:55:47.426 INFO:teuthology.orchestra.run.smithi018.stdout:172.21.15.18:6800/1312802974 2024-04-13T20:43:34.269559+0000 2024-04-12T21:55:47.426 INFO:teuthology.orchestra.run.smithi018.stdout:172.21.15.186:6828/2316349902 2024-04-13T20:57:12.501044+0000 2024-04-12T21:55:47.426 INFO:teuthology.orchestra.run.smithi018.stdout:172.21.15.186:6825/1645616336 2024-04-13T20:52:14.968672+0000 2024-04-12T21:55:47.426 INFO:teuthology.orchestra.run.smithi018.stdout:172.21.15.18:0/3745234714 2024-04-13T21:01:39.724018+0000 2024-04-12T21:55:47.426 INFO:teuthology.orchestra.run.smithi018.stdout:172.21.15.18:0/2356171563 2024-04-13T21:01:39.724018+0000 2024-04-12T21:55:47.426 INFO:teuthology.orchestra.run.smithi018.stdout:172.21.15.18:6801/1391635591 2024-04-13T20:45:44.776072+0000 2024-04-12T21:55:47.426 INFO:teuthology.orchestra.run.smithi018.stdout:172.21.15.18:0/2517252399 2024-04-13T20:43:34.269559+0000 2024-04-12T21:55:47.426 INFO:teuthology.orchestra.run.smithi018.stdout:172.21.15.18:6801/1605950634 2024-04-13T21:01:39.724018+0000 2024-04-12T21:55:47.426 INFO:teuthology.orchestra.run.smithi018.stdout:172.21.15.18:0/3861256640 2024-04-13T21:01:39.724018+0000 2024-04-12T21:55:47.427 INFO:teuthology.orchestra.run.smithi018.stderr:listed 39 entries 2024-04-12T21:55:47.427 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:47.425+0000 7f289ae00640 1 -- 172.21.15.18:0/3803902058 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f287c06eb80 msgr2=0x7f287c071030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:55:47.427 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:47.425+0000 7f289ae00640 1 --2- 172.21.15.18:0/3803902058 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f287c06eb80 0x7f287c071030 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7f2894110660 tx=0x7f288000b040 comp rx=0 tx=0).stop 2024-04-12T21:55:47.427 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:47.425+0000 7f289ae00640 1 -- 172.21.15.18:0/3803902058 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f289406f810 msgr2=0x7f2894114da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:55:47.427 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:47.425+0000 7f289ae00640 1 --2- 172.21.15.18:0/3803902058 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f289406f810 0x7f2894114da0 secure :-1 s=READY pgs=1441 cs=0 l=1 rev1=1 crypto rx=0x7f288802c1f0 tx=0x7f2888004d10 comp rx=0 tx=0).stop 2024-04-12T21:55:47.427 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:47.425+0000 7f289ae00640 1 -- 172.21.15.18:0/3803902058 shutdown_connections 2024-04-12T21:55:47.427 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:47.425+0000 7f289ae00640 1 --2- 172.21.15.18:0/3803902058 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f287c06eb80 0x7f287c071030 unknown :-1 s=CLOSED pgs=762 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:55:47.427 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:47.425+0000 7f289ae00640 1 --2- 172.21.15.18:0/3803902058 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2894070120 0x7f28941152e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:55:47.427 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:47.425+0000 7f289ae00640 1 --2- 172.21.15.18:0/3803902058 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f289406f810 0x7f2894114da0 unknown :-1 s=CLOSED pgs=1441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:55:47.427 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:47.425+0000 7f289ae00640 1 -- 172.21.15.18:0/3803902058 >> 172.21.15.18:0/3803902058 conn(0x7f28940f7c60 msgr2=0x7f28940f9110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:55:47.428 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:47.426+0000 7f289ae00640 1 -- 172.21.15.18:0/3803902058 shutdown_connections 2024-04-12T21:55:47.428 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:47.426+0000 7f289ae00640 1 -- 172.21.15.18:0/3803902058 wait complete. 2024-04-12T21:55:47.445 DEBUG:teuthology.orchestra.run.smithi018:> set -ex 2024-04-12T21:55:47.445 DEBUG:teuthology.orchestra.run.smithi018:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-12T21:55:47.501 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-12T21:55:47.587 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:55:47 smithi018 ceph-mon[112590]: from='client.? 172.21.15.18:0/4045994680' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"}]: dispatch 2024-04-12T21:55:47.587 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:55:47 smithi018 ceph-mon[112590]: from='client.? 172.21.15.18:0/39114357' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"}]: dispatch 2024-04-12T21:55:47.720 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:47.719+0000 7fa67cc00640 1 -- 172.21.15.18:0/3214766550 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa6781026d0 msgr2=0x7fa678102b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:55:47.721 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:47.719+0000 7fa67cc00640 1 --2- 172.21.15.18:0/3214766550 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa6781026d0 0x7fa678102b40 secure :-1 s=READY pgs=1442 cs=0 l=1 rev1=1 crypto rx=0x7fa6640048d0 tx=0x7fa664009da0 comp rx=0 tx=0).stop 2024-04-12T21:55:47.721 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:47.719+0000 7fa67cc00640 1 -- 172.21.15.18:0/3214766550 shutdown_connections 2024-04-12T21:55:47.721 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:47.719+0000 7fa67cc00640 1 --2- 172.21.15.18:0/3214766550 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa6781026d0 0x7fa678102b40 unknown :-1 s=CLOSED pgs=1442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:55:47.721 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:47.719+0000 7fa67cc00640 1 --2- 172.21.15.18:0/3214766550 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa678108690 0x7fa678108a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:55:47.721 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:47.719+0000 7fa67cc00640 1 -- 172.21.15.18:0/3214766550 >> 172.21.15.18:0/3214766550 conn(0x7fa678071e60 msgr2=0x7fa678074270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:55:47.721 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:47.719+0000 7fa67cc00640 1 -- 172.21.15.18:0/3214766550 shutdown_connections 2024-04-12T21:55:47.721 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:47.719+0000 7fa67cc00640 1 -- 172.21.15.18:0/3214766550 wait complete. 2024-04-12T21:55:47.723 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:47.720+0000 7fa67cc00640 1 Processor -- start 2024-04-12T21:55:47.723 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:47.720+0000 7fa67cc00640 1 -- start start 2024-04-12T21:55:47.723 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:47.720+0000 7fa67cc00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa6781026d0 0x7fa6781190f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:55:47.723 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:47.720+0000 7fa67cc00640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa678108690 0x7fa678119630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:55:47.724 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:47.720+0000 7fa67cc00640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa678119c30 con 0x7fa6781026d0 2024-04-12T21:55:47.724 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:47.720+0000 7fa67cc00640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa678113280 con 0x7fa678108690 2024-04-12T21:55:47.724 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:47.721+0000 7fa677400640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa6781026d0 0x7fa6781190f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:55:47.724 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:47.721+0000 7fa677400640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa6781026d0 0x7fa6781190f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:43582/0 (socket says 172.21.15.18:43582) 2024-04-12T21:55:47.724 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:47.721+0000 7fa677400640 1 -- 172.21.15.18:0/1509714726 learned_addr learned my addr 172.21.15.18:0/1509714726 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:55:47.724 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:47.721+0000 7fa676a00640 1 --2- 172.21.15.18:0/1509714726 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa678108690 0x7fa678119630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:55:47.724 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:47.721+0000 7fa677400640 1 -- 172.21.15.18:0/1509714726 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa678108690 msgr2=0x7fa678119630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:55:47.724 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:47.721+0000 7fa677400640 1 --2- 172.21.15.18:0/1509714726 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa678108690 0x7fa678119630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:55:47.724 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:47.721+0000 7fa677400640 1 -- 172.21.15.18:0/1509714726 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa664005040 con 0x7fa6781026d0 2024-04-12T21:55:47.724 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:47.722+0000 7fa677400640 1 --2- 172.21.15.18:0/1509714726 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa6781026d0 0x7fa6781190f0 secure :-1 s=READY pgs=1443 cs=0 l=1 rev1=1 crypto rx=0x7fa66800c830 tx=0x7fa668004430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:55:47.724 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:47.722+0000 7fa66f400640 1 -- 172.21.15.18:0/1509714726 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa66800a7d0 con 0x7fa6781026d0 2024-04-12T21:55:47.724 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:47.722+0000 7fa66f400640 1 -- 172.21.15.18:0/1509714726 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa66800ae00 con 0x7fa6781026d0 2024-04-12T21:55:47.724 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:47.722+0000 7fa66f400640 1 -- 172.21.15.18:0/1509714726 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa6680113a0 con 0x7fa6781026d0 2024-04-12T21:55:47.724 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:47.722+0000 7fa67cc00640 1 -- 172.21.15.18:0/1509714726 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa678113560 con 0x7fa6781026d0 2024-04-12T21:55:47.725 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:47.722+0000 7fa67cc00640 1 -- 172.21.15.18:0/1509714726 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa678113a50 con 0x7fa6781026d0 2024-04-12T21:55:47.725 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:47.723+0000 7fa67cc00640 1 -- 172.21.15.18:0/1509714726 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa678103db0 con 0x7fa6781026d0 2024-04-12T21:55:47.726 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:47.724+0000 7fa66f400640 1 -- 172.21.15.18:0/1509714726 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7fa668012400 con 0x7fa6781026d0 2024-04-12T21:55:47.726 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:47.725+0000 7fa66f400640 1 --2- 172.21.15.18:0/1509714726 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fa65c06eb70 0x7fa65c071020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:55:47.726 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:47.725+0000 7fa66f400640 1 -- 172.21.15.18:0/1509714726 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 6757+0+0 (secure 0 0 0) 0x7fa66808ea70 con 0x7fa6781026d0 2024-04-12T21:55:47.727 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:47.726+0000 7fa676a00640 1 --2- 172.21.15.18:0/1509714726 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fa65c06eb70 0x7fa65c071020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:55:47.728 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:47.727+0000 7fa676a00640 1 --2- 172.21.15.18:0/1509714726 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fa65c06eb70 0x7fa65c071020 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7fa6640048a0 tx=0x7fa664009da0 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:55:47.732 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:47.731+0000 7fa66f400640 1 -- 172.21.15.18:0/1509714726 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7fa668057830 con 0x7fa6781026d0 2024-04-12T21:55:47.785 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:55:47 smithi186 ceph-mon[86379]: from='client.? 172.21.15.18:0/4045994680' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"}]: dispatch 2024-04-12T21:55:47.785 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:55:47 smithi186 ceph-mon[86379]: from='client.? 172.21.15.18:0/39114357' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"}]: dispatch 2024-04-12T21:55:47.932 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:47.931+0000 7fa67cc00640 1 -- 172.21.15.18:0/1509714726 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa67804bcc0 con 0x7fa6781026d0 2024-04-12T21:55:47.934 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:47.933+0000 7fa66f400640 1 -- 172.21.15.18:0/1509714726 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v104) v1 ==== 74+0+17095 (secure 0 0 0) 0x7fa66805aef0 con 0x7fa6781026d0 2024-04-12T21:55:47.935 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-04-12T21:55:47.935 INFO:teuthology.orchestra.run.smithi018.stdout:{"epoch":104,"fsid":"1cbe407a-f90d-11ee-bc8d-c7b262605968","created":"2024-04-12T20:43:00.692361+0000","modified":"2024-04-12T21:38:07.461428+0000","last_up_change":"2024-04-12T21:06:52.454846+0000","last_in_change":"2024-04-12T20:49:54.190644+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":20,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-12T20:48:37.529081+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":6,"score_stable":6,"optimal_score":0.5,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs.cephfs.meta","create_time":"2024-04-12T20:51:56.284414+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"44","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.309999942779541,"score_stable":1.309999942779541,"optimal_score":1,"raw_score_acting":1.309999942779541,"raw_score_stable":1.309999942779541,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs.cephfs.data","create_time":"2024-04-12T20:51:56.927802+0000","flags":131073,"flags_names":"hashpspool,bulk","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"45","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2af887cc-0447-42aa-a8f3-1aa7fcd4dc8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":9,"last_clean_end":49,"up_from":64,"up_thru":97,"down_at":50,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6802","nonce":2573875940},{"type":"v1","addr":"172.21.15.18:6803","nonce":2573875940}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6804","nonce":2573875940},{"type":"v1","addr":"172.21.15.18:6805","nonce":2573875940}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6808","nonce":2573875940},{"type":"v1","addr":"172.21.15.18:6809","nonce":2573875940}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6806","nonce":2573875940},{"type":"v1","addr":"172.21.15.18:6807","nonce":2573875940}]},"public_addr":"172.21.15.18:6803/2573875940","cluster_addr":"172.21.15.18:6805/2573875940","heartbeat_back_addr":"172.21.15.18:6809/2573875940","heartbeat_front_addr":"172.21.15.18:6807/2573875940","state":["exists","up"]},{"osd":1,"uuid":"6ba7b0d3-a03d-45cc-b4e9-89a92c8c6ca8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":14,"last_clean_end":73,"up_from":77,"up_thru":97,"down_at":74,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6810","nonce":1955801845},{"type":"v1","addr":"172.21.15.18:6811","nonce":1955801845}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6812","nonce":1955801845},{"type":"v1","addr":"172.21.15.18:6813","nonce":1955801845}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6816","nonce":1955801845},{"type":"v1","addr":"172.21.15.18:6817","nonce":1955801845}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6814","nonce":1955801845},{"type":"v1","addr":"172.21.15.18:6815","nonce":1955801845}]},"public_addr":"172.21.15.18:6811/1955801845","cluster_addr":"172.21.15.18:6813/1955801845","heartbeat_back_addr":"172.21.15.18:6817/1955801845","heartbeat_front_addr":"172.21.15.18:6815/1955801845","state":["exists","up"]},{"osd":2,"uuid":"5bb1137d-5a3c-40bc-ae08-fc792b8ee547","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":19,"last_clean_end":78,"up_from":82,"up_thru":97,"down_at":79,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6818","nonce":1361831750},{"type":"v1","addr":"172.21.15.18:6819","nonce":1361831750}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6820","nonce":1361831750},{"type":"v1","addr":"172.21.15.18:6821","nonce":1361831750}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6824","nonce":1361831750},{"type":"v1","addr":"172.21.15.18:6825","nonce":1361831750}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6822","nonce":1361831750},{"type":"v1","addr":"172.21.15.18:6823","nonce":1361831750}]},"public_addr":"172.21.15.18:6819/1361831750","cluster_addr":"172.21.15.18:6821/1361831750","heartbeat_back_addr":"172.21.15.18:6825/1361831750","heartbeat_front_addr":"172.21.15.18:6823/1361831750","state":["exists","up"]},{"osd":3,"uuid":"a5140a33-1e90-4980-aa9a-b0afe6b155f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":26,"last_clean_end":83,"up_from":87,"up_thru":97,"down_at":84,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6800","nonce":1952273925},{"type":"v1","addr":"172.21.15.186:6801","nonce":1952273925}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6802","nonce":1952273925},{"type":"v1","addr":"172.21.15.186:6803","nonce":1952273925}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6806","nonce":1952273925},{"type":"v1","addr":"172.21.15.186:6807","nonce":1952273925}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6804","nonce":1952273925},{"type":"v1","addr":"172.21.15.186:6805","nonce":1952273925}]},"public_addr":"172.21.15.186:6801/1952273925","cluster_addr":"172.21.15.186:6803/1952273925","heartbeat_back_addr":"172.21.15.186:6807/1952273925","heartbeat_front_addr":"172.21.15.186:6805/1952273925","state":["exists","up"]},{"osd":4,"uuid":"d6b2c6b3-6c78-4261-bcbe-c33358407198","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":34,"last_clean_end":88,"up_from":92,"up_thru":97,"down_at":89,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6808","nonce":2123883537},{"type":"v1","addr":"172.21.15.186:6809","nonce":2123883537}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6810","nonce":2123883537},{"type":"v1","addr":"172.21.15.186:6811","nonce":2123883537}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6814","nonce":2123883537},{"type":"v1","addr":"172.21.15.186:6815","nonce":2123883537}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6812","nonce":2123883537},{"type":"v1","addr":"172.21.15.186:6813","nonce":2123883537}]},"public_addr":"172.21.15.186:6809/2123883537","cluster_addr":"172.21.15.186:6811/2123883537","heartbeat_back_addr":"172.21.15.186:6815/2123883537","heartbeat_front_addr":"172.21.15.186:6813/2123883537","state":["exists","up"]},{"osd":5,"uuid":"7b5c9f17-6306-4fb5-aa77-fc9a73bcc663","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":39,"last_clean_end":93,"up_from":97,"up_thru":97,"down_at":94,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6816","nonce":2849575493},{"type":"v1","addr":"172.21.15.186:6817","nonce":2849575493}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6818","nonce":2849575493},{"type":"v1","addr":"172.21.15.186:6819","nonce":2849575493}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6822","nonce":2849575493},{"type":"v1","addr":"172.21.15.186:6823","nonce":2849575493}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6820","nonce":2849575493},{"type":"v1","addr":"172.21.15.186:6821","nonce":2849575493}]},"public_addr":"172.21.15.186:6817/2849575493","cluster_addr":"172.21.15.186:6819/2849575493","heartbeat_back_addr":"172.21.15.186:6823/2849575493","heartbeat_front_addr":"172.21.15.186:6821/2849575493","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-12T20:47:21.231060+0000","dead_epoch":49},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-12T20:47:57.699205+0000","dead_epoch":73},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-12T20:48:35.096648+0000","dead_epoch":78},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-12T20:49:08.252810+0000","dead_epoch":83},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-12T20:49:41.102187+0000","dead_epoch":88},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-12T20:50:16.009813+0000","dead_epoch":93}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.8","mappings":[{"from":3,"to":2}]},{"pgid":"2.a","mappings":[{"from":3,"to":0}]},{"pgid":"2.1d","mappings":[{"from":3,"to":2}]},{"pgid":"3.6","mappings":[{"from":1,"to":5}]},{"pgid":"3.a","mappings":[{"from":4,"to":0}]},{"pgid":"3.e","mappings":[{"from":1,"to":2}]},{"pgid":"3.11","mappings":[{"from":3,"to":2}]},{"pgid":"3.14","mappings":[{"from":3,"to":0}]},{"pgid":"3.1a","mappings":[{"from":4,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.18:6826/1564065434":"2024-04-13T21:06:59.717116+0000","172.21.15.186:0/825752072":"2024-04-13T20:57:12.501044+0000","172.21.15.18:6800/1391635591":"2024-04-13T20:45:44.776072+0000","172.21.15.18:0/4018777476":"2024-04-13T20:45:44.776072+0000","172.21.15.186:6824/1645616336":"2024-04-13T20:52:14.968672+0000","172.21.15.18:0/3387911038":"2024-04-13T20:44:15.143139+0000","172.21.15.18:0/714775767":"2024-04-13T20:54:59.478287+0000","172.21.15.18:6801/3048894687":"2024-04-13T20:54:59.478287+0000","172.21.15.18:0/2171204556":"2024-04-13T21:01:39.724018+0000","172.21.15.186:0/2147813742":"2024-04-13T20:57:12.501044+0000","172.21.15.18:6829/722443353":"2024-04-13T21:06:59.717165+0000","172.21.15.18:0/2945432483":"2024-04-13T20:44:15.143139+0000","172.21.15.18:6827/1564065434":"2024-04-13T21:06:59.717116+0000","172.21.15.186:6829/2316349902":"2024-04-13T20:57:12.501044+0000","172.21.15.18:0/2956836690":"2024-04-13T20:54:59.478287+0000","172.21.15.18:0/1732042827":"2024-04-13T20:43:34.269559+0000","172.21.15.18:6800/3403971131":"2024-04-13T20:44:15.143139+0000","172.21.15.18:6800/1605950634":"2024-04-13T21:01:39.724018+0000","172.21.15.186:0/970502145":"2024-04-13T20:57:12.501044+0000","172.21.15.18:6800/3048894687":"2024-04-13T20:54:59.478287+0000","172.21.15.18:0/2330205732":"2024-04-13T20:54:59.478287+0000","172.21.15.18:0/3227425848":"2024-04-13T20:54:59.478287+0000","172.21.15.18:6801/1312802974":"2024-04-13T20:43:34.269559+0000","172.21.15.186:0/2459326506":"2024-04-13T20:57:12.501044+0000","172.21.15.18:0/3896210349":"2024-04-13T20:44:15.143139+0000","172.21.15.18:0/1720919549":"2024-04-13T20:45:44.776072+0000","172.21.15.18:0/766256564":"2024-04-13T20:45:44.776072+0000","172.21.15.18:6801/3403971131":"2024-04-13T20:44:15.143139+0000","172.21.15.18:6828/722443353":"2024-04-13T21:06:59.717165+0000","172.21.15.18:0/3295870133":"2024-04-13T20:43:34.269559+0000","172.21.15.18:6800/1312802974":"2024-04-13T20:43:34.269559+0000","172.21.15.186:6828/2316349902":"2024-04-13T20:57:12.501044+0000","172.21.15.186:6825/1645616336":"2024-04-13T20:52:14.968672+0000","172.21.15.18:0/3745234714":"2024-04-13T21:01:39.724018+0000","172.21.15.18:0/2356171563":"2024-04-13T21:01:39.724018+0000","172.21.15.18:6801/1391635591":"2024-04-13T20:45:44.776072+0000","172.21.15.18:0/2517252399":"2024-04-13T20:43:34.269559+0000","172.21.15.18:6801/1605950634":"2024-04-13T21:01:39.724018+0000","172.21.15.18:0/3861256640":"2024-04-13T21:01:39.724018+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-12T21:55:47.938 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:47.936+0000 7fa67cc00640 1 -- 172.21.15.18:0/1509714726 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fa65c06eb70 msgr2=0x7fa65c071020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:55:47.938 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:47.936+0000 7fa67cc00640 1 --2- 172.21.15.18:0/1509714726 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fa65c06eb70 0x7fa65c071020 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7fa6640048a0 tx=0x7fa664009da0 comp rx=0 tx=0).stop 2024-04-12T21:55:47.938 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:47.936+0000 7fa67cc00640 1 -- 172.21.15.18:0/1509714726 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa6781026d0 msgr2=0x7fa6781190f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:55:47.938 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:47.936+0000 7fa67cc00640 1 --2- 172.21.15.18:0/1509714726 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa6781026d0 0x7fa6781190f0 secure :-1 s=READY pgs=1443 cs=0 l=1 rev1=1 crypto rx=0x7fa66800c830 tx=0x7fa668004430 comp rx=0 tx=0).stop 2024-04-12T21:55:47.938 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:47.936+0000 7fa67cc00640 1 -- 172.21.15.18:0/1509714726 shutdown_connections 2024-04-12T21:55:47.938 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:47.936+0000 7fa67cc00640 1 --2- 172.21.15.18:0/1509714726 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fa65c06eb70 0x7fa65c071020 unknown :-1 s=CLOSED pgs=763 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:55:47.938 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:47.936+0000 7fa67cc00640 1 --2- 172.21.15.18:0/1509714726 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa678108690 0x7fa678119630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:55:47.938 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:47.936+0000 7fa67cc00640 1 --2- 172.21.15.18:0/1509714726 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa6781026d0 0x7fa6781190f0 unknown :-1 s=CLOSED pgs=1443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:55:47.938 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:47.936+0000 7fa67cc00640 1 -- 172.21.15.18:0/1509714726 >> 172.21.15.18:0/1509714726 conn(0x7fa678071e60 msgr2=0x7fa678073cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:55:47.938 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:47.937+0000 7fa67cc00640 1 -- 172.21.15.18:0/1509714726 shutdown_connections 2024-04-12T21:55:47.938 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:47.937+0000 7fa67cc00640 1 -- 172.21.15.18:0/1509714726 wait complete. 2024-04-12T21:55:47.957 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-12T20:48:37.529081+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 6, 'score_stable': 6, 'optimal_score': 0.5, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs.cephfs.meta', 'create_time': '2024-04-12T20:51:56.284414+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '44', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.309999942779541, 'score_stable': 1.309999942779541, 'optimal_score': 1, 'raw_score_acting': 1.309999942779541, 'raw_score_stable': 1.309999942779541, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs.cephfs.data', 'create_time': '2024-04-12T20:51:56.927802+0000', 'flags': 131073, 'flags_names': 'hashpspool,bulk', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '45', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-12T21:55:47.958 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-12T21:55:48.169 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:48.167+0000 7fdb9cc00640 1 -- 172.21.15.18:0/2288906059 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdb98104bf0 msgr2=0x7fdb980fe280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:55:48.169 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:48.167+0000 7fdb9cc00640 1 --2- 172.21.15.18:0/2288906059 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdb98104bf0 0x7fdb980fe280 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fdb840048d0 tx=0x7fdb84009da0 comp rx=0 tx=0).stop 2024-04-12T21:55:48.169 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:48.167+0000 7fdb9cc00640 1 -- 172.21.15.18:0/2288906059 shutdown_connections 2024-04-12T21:55:48.169 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:48.167+0000 7fdb9cc00640 1 --2- 172.21.15.18:0/2288906059 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdb98104bf0 0x7fdb980fe280 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:55:48.169 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:48.167+0000 7fdb9cc00640 1 --2- 172.21.15.18:0/2288906059 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fdb98104250 0x7fdb98104620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:55:48.169 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:48.167+0000 7fdb9cc00640 1 -- 172.21.15.18:0/2288906059 >> 172.21.15.18:0/2288906059 conn(0x7fdb980f9fc0 msgr2=0x7fdb980fc3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:55:48.169 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:48.167+0000 7fdb9cc00640 1 -- 172.21.15.18:0/2288906059 shutdown_connections 2024-04-12T21:55:48.169 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:48.167+0000 7fdb9cc00640 1 -- 172.21.15.18:0/2288906059 wait complete. 2024-04-12T21:55:48.171 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:48.168+0000 7fdb9cc00640 1 Processor -- start 2024-04-12T21:55:48.171 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:48.168+0000 7fdb9cc00640 1 -- start start 2024-04-12T21:55:48.171 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:48.169+0000 7fdb9cc00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fdb98104250 0x7fdb98114da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:55:48.171 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:48.169+0000 7fdb9cc00640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdb98104bf0 0x7fdb981152e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:55:48.172 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:48.169+0000 7fdb9cc00640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb981158e0 con 0x7fdb98104250 2024-04-12T21:55:48.172 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:48.169+0000 7fdb9cc00640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb9810ef30 con 0x7fdb98104bf0 2024-04-12T21:55:48.172 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:48.169+0000 7fdb97400640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fdb98104250 0x7fdb98114da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:55:48.172 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:48.169+0000 7fdb97400640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fdb98104250 0x7fdb98114da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:43614/0 (socket says 172.21.15.18:43614) 2024-04-12T21:55:48.172 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:48.169+0000 7fdb97400640 1 -- 172.21.15.18:0/3545982562 learned_addr learned my addr 172.21.15.18:0/3545982562 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:55:48.172 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:48.169+0000 7fdb97400640 1 -- 172.21.15.18:0/3545982562 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdb98104bf0 msgr2=0x7fdb981152e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:55:48.172 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:48.169+0000 7fdb97400640 1 --2- 172.21.15.18:0/3545982562 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdb98104bf0 0x7fdb981152e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:55:48.172 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:48.169+0000 7fdb97400640 1 -- 172.21.15.18:0/3545982562 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb88005040 con 0x7fdb98104250 2024-04-12T21:55:48.172 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:48.170+0000 7fdb97400640 1 --2- 172.21.15.18:0/3545982562 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fdb98104250 0x7fdb98114da0 secure :-1 s=READY pgs=1444 cs=0 l=1 rev1=1 crypto rx=0x7fdb88009700 tx=0x7fdb88009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:55:48.172 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:48.170+0000 7fdb8f400640 1 -- 172.21.15.18:0/3545982562 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdb8800e930 con 0x7fdb98104250 2024-04-12T21:55:48.172 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:48.170+0000 7fdb8f400640 1 -- 172.21.15.18:0/3545982562 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fdb8800f040 con 0x7fdb98104250 2024-04-12T21:55:48.172 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:48.170+0000 7fdb8f400640 1 -- 172.21.15.18:0/3545982562 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdb88012360 con 0x7fdb98104250 2024-04-12T21:55:48.172 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:48.170+0000 7fdb9cc00640 1 -- 172.21.15.18:0/3545982562 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdb84005040 con 0x7fdb98104250 2024-04-12T21:55:48.172 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:48.170+0000 7fdb9cc00640 1 -- 172.21.15.18:0/3545982562 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdb9810f540 con 0x7fdb98104250 2024-04-12T21:55:48.173 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:48.172+0000 7fdb9cc00640 1 -- 172.21.15.18:0/3545982562 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdb980ff9c0 con 0x7fdb98104250 2024-04-12T21:55:48.173 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:48.172+0000 7fdb8f400640 1 -- 172.21.15.18:0/3545982562 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7fdb8800ea90 con 0x7fdb98104250 2024-04-12T21:55:48.174 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:48.173+0000 7fdb8f400640 1 --2- 172.21.15.18:0/3545982562 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fdb7c06eb80 0x7fdb7c071030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:55:48.175 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:48.174+0000 7fdb8f400640 1 -- 172.21.15.18:0/3545982562 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 6757+0+0 (secure 0 0 0) 0x7fdb8808e930 con 0x7fdb98104250 2024-04-12T21:55:48.175 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:48.174+0000 7fdb96a00640 1 --2- 172.21.15.18:0/3545982562 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fdb7c06eb80 0x7fdb7c071030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:55:48.175 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:48.175+0000 7fdb96a00640 1 --2- 172.21.15.18:0/3545982562 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fdb7c06eb80 0x7fdb7c071030 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7fdb8402bda0 tx=0x7fdb84009da0 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:55:48.179 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:48.178+0000 7fdb8f400640 1 -- 172.21.15.18:0/3545982562 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7fdb88057650 con 0x7fdb98104250 2024-04-12T21:55:48.387 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:48.386+0000 7fdb9cc00640 1 -- 172.21.15.18:0/3545982562 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fdb9804bcc0 con 0x7fdb98104250 2024-04-12T21:55:48.388 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:48.387+0000 7fdb8f400640 1 -- 172.21.15.18:0/3545982562 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v104) v1 ==== 93+0+10 (secure 0 0 0) 0x7fdb9804bcc0 con 0x7fdb98104250 2024-04-12T21:55:48.388 INFO:teuthology.orchestra.run.smithi018.stdout:pg_num: 1 2024-04-12T21:55:48.391 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:48.389+0000 7fdb9cc00640 1 -- 172.21.15.18:0/3545982562 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fdb7c06eb80 msgr2=0x7fdb7c071030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:55:48.391 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:48.389+0000 7fdb9cc00640 1 --2- 172.21.15.18:0/3545982562 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fdb7c06eb80 0x7fdb7c071030 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7fdb8402bda0 tx=0x7fdb84009da0 comp rx=0 tx=0).stop 2024-04-12T21:55:48.391 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:48.389+0000 7fdb9cc00640 1 -- 172.21.15.18:0/3545982562 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fdb98104250 msgr2=0x7fdb98114da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:55:48.391 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:48.389+0000 7fdb9cc00640 1 --2- 172.21.15.18:0/3545982562 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fdb98104250 0x7fdb98114da0 secure :-1 s=READY pgs=1444 cs=0 l=1 rev1=1 crypto rx=0x7fdb88009700 tx=0x7fdb88009da0 comp rx=0 tx=0).stop 2024-04-12T21:55:48.391 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:48.389+0000 7fdb9cc00640 1 -- 172.21.15.18:0/3545982562 shutdown_connections 2024-04-12T21:55:48.391 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:48.389+0000 7fdb9cc00640 1 --2- 172.21.15.18:0/3545982562 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fdb7c06eb80 0x7fdb7c071030 unknown :-1 s=CLOSED pgs=764 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:55:48.391 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:48.389+0000 7fdb9cc00640 1 --2- 172.21.15.18:0/3545982562 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdb98104bf0 0x7fdb981152e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:55:48.391 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:48.389+0000 7fdb9cc00640 1 --2- 172.21.15.18:0/3545982562 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fdb98104250 0x7fdb98114da0 unknown :-1 s=CLOSED pgs=1444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:55:48.391 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:48.389+0000 7fdb9cc00640 1 -- 172.21.15.18:0/3545982562 >> 172.21.15.18:0/3545982562 conn(0x7fdb980f9fc0 msgr2=0x7fdb980fbe70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:55:48.391 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:48.390+0000 7fdb9cc00640 1 -- 172.21.15.18:0/3545982562 shutdown_connections 2024-04-12T21:55:48.392 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:48.390+0000 7fdb9cc00640 1 -- 172.21.15.18:0/3545982562 wait complete. 2024-04-12T21:55:48.410 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs.cephfs.meta pg_num 2024-04-12T21:55:48.606 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:55:48 smithi018 ceph-mon[112590]: from='client.? 172.21.15.18:0/3803902058' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-04-12T21:55:48.606 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:55:48 smithi018 ceph-mon[112590]: from='client.? 172.21.15.18:0/1509714726' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-12T21:55:48.606 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:55:48 smithi018 ceph-mon[112590]: pgmap v1686: 65 pgs: 65 active+clean; 257 MiB data, 976 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:55:48.637 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:48.635+0000 7fa36ea00640 1 -- 172.21.15.18:0/2095734383 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa36806f7f0 msgr2=0x7fa36806fbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:55:48.637 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:48.635+0000 7fa36ea00640 1 --2- 172.21.15.18:0/2095734383 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa36806f7f0 0x7fa36806fbc0 secure :-1 s=READY pgs=1445 cs=0 l=1 rev1=1 crypto rx=0x7fa35c0048d0 tx=0x7fa35c009da0 comp rx=0 tx=0).stop 2024-04-12T21:55:48.637 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:48.635+0000 7fa36ea00640 1 -- 172.21.15.18:0/2095734383 shutdown_connections 2024-04-12T21:55:48.637 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:48.635+0000 7fa36ea00640 1 --2- 172.21.15.18:0/2095734383 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa368070100 0x7fa368108400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:55:48.637 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:48.635+0000 7fa36ea00640 1 --2- 172.21.15.18:0/2095734383 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa36806f7f0 0x7fa36806fbc0 unknown :-1 s=CLOSED pgs=1445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:55:48.637 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:48.635+0000 7fa36ea00640 1 -- 172.21.15.18:0/2095734383 >> 172.21.15.18:0/2095734383 conn(0x7fa3680f7c60 msgr2=0x7fa3680fa070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:55:48.637 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:48.636+0000 7fa36ea00640 1 -- 172.21.15.18:0/2095734383 shutdown_connections 2024-04-12T21:55:48.637 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:48.636+0000 7fa36ea00640 1 -- 172.21.15.18:0/2095734383 wait complete. 2024-04-12T21:55:48.638 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:48.636+0000 7fa36ea00640 1 Processor -- start 2024-04-12T21:55:48.638 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:48.637+0000 7fa36ea00640 1 -- start start 2024-04-12T21:55:48.638 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:48.637+0000 7fa36ea00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa36806f7f0 0x7fa368114d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:55:48.639 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:48.637+0000 7fa36ea00640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa368070100 0x7fa3681152c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:55:48.639 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:48.637+0000 7fa36ea00640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3681158c0 con 0x7fa36806f7f0 2024-04-12T21:55:48.639 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:48.637+0000 7fa36ea00640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa36810ef10 con 0x7fa368070100 2024-04-12T21:55:48.639 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:48.637+0000 7fa36d600640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa36806f7f0 0x7fa368114d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:55:48.639 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:48.637+0000 7fa36d600640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa36806f7f0 0x7fa368114d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:43632/0 (socket says 172.21.15.18:43632) 2024-04-12T21:55:48.639 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:48.637+0000 7fa36d600640 1 -- 172.21.15.18:0/313255472 learned_addr learned my addr 172.21.15.18:0/313255472 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:55:48.640 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:48.638+0000 7fa36d600640 1 -- 172.21.15.18:0/313255472 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa368070100 msgr2=0x7fa3681152c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:55:48.640 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:48.638+0000 7fa36d600640 1 --2- 172.21.15.18:0/313255472 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa368070100 0x7fa3681152c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:55:48.640 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:48.638+0000 7fa36d600640 1 -- 172.21.15.18:0/313255472 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa35c005040 con 0x7fa36806f7f0 2024-04-12T21:55:48.640 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:48.638+0000 7fa36d600640 1 --2- 172.21.15.18:0/313255472 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa36806f7f0 0x7fa368114d80 secure :-1 s=READY pgs=1446 cs=0 l=1 rev1=1 crypto rx=0x7fa35c0047f0 tx=0x7fa35c009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:55:48.640 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:48.638+0000 7fa365600640 1 -- 172.21.15.18:0/313255472 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa35c0094c0 con 0x7fa36806f7f0 2024-04-12T21:55:48.640 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:48.638+0000 7fa365600640 1 -- 172.21.15.18:0/313255472 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa35c032e50 con 0x7fa36806f7f0 2024-04-12T21:55:48.640 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:48.638+0000 7fa365600640 1 -- 172.21.15.18:0/313255472 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa35c0435f0 con 0x7fa36806f7f0 2024-04-12T21:55:48.640 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:48.638+0000 7fa36ea00640 1 -- 172.21.15.18:0/313255472 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa36810f190 con 0x7fa36806f7f0 2024-04-12T21:55:48.640 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:48.639+0000 7fa36ea00640 1 -- 172.21.15.18:0/313255472 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa36810f650 con 0x7fa36806f7f0 2024-04-12T21:55:48.641 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:48.640+0000 7fa36ea00640 1 -- 172.21.15.18:0/313255472 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa368070f30 con 0x7fa36806f7f0 2024-04-12T21:55:48.642 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:48.641+0000 7fa365600640 1 -- 172.21.15.18:0/313255472 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7fa35c02f070 con 0x7fa36806f7f0 2024-04-12T21:55:48.643 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:48.642+0000 7fa365600640 1 --2- 172.21.15.18:0/313255472 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fa35006eb80 0x7fa350071030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:55:48.643 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:48.642+0000 7fa365600640 1 -- 172.21.15.18:0/313255472 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 6757+0+0 (secure 0 0 0) 0x7fa35c0b0790 con 0x7fa36806f7f0 2024-04-12T21:55:48.643 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:48.642+0000 7fa36cc00640 1 --2- 172.21.15.18:0/313255472 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fa35006eb80 0x7fa350071030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:55:48.644 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:48.643+0000 7fa36cc00640 1 --2- 172.21.15.18:0/313255472 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fa35006eb80 0x7fa350071030 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7fa368110640 tx=0x7fa354007c30 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:55:48.647 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:48.647+0000 7fa365600640 1 -- 172.21.15.18:0/313255472 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7fa35c079430 con 0x7fa36806f7f0 2024-04-12T21:55:48.785 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:55:48 smithi186 ceph-mon[86379]: from='client.? 172.21.15.18:0/3803902058' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-04-12T21:55:48.785 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:55:48 smithi186 ceph-mon[86379]: from='client.? 172.21.15.18:0/1509714726' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-12T21:55:48.785 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:55:48 smithi186 ceph-mon[86379]: pgmap v1686: 65 pgs: 65 active+clean; 257 MiB data, 976 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:55:48.854 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:48.853+0000 7fa36ea00640 1 -- 172.21.15.18:0/313255472 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"} v 0) v1 -- 0x7fa36810b3a0 con 0x7fa36806f7f0 2024-04-12T21:55:48.855 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:48.854+0000 7fa365600640 1 -- 172.21.15.18:0/313255472 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"}]=0 v104) v1 ==== 107+0+11 (secure 0 0 0) 0x7fa35c07caf0 con 0x7fa36806f7f0 2024-04-12T21:55:48.856 INFO:teuthology.orchestra.run.smithi018.stdout:pg_num: 32 2024-04-12T21:55:48.859 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:48.857+0000 7fa36ea00640 1 -- 172.21.15.18:0/313255472 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fa35006eb80 msgr2=0x7fa350071030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:55:48.859 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:48.857+0000 7fa36ea00640 1 --2- 172.21.15.18:0/313255472 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fa35006eb80 0x7fa350071030 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7fa368110640 tx=0x7fa354007c30 comp rx=0 tx=0).stop 2024-04-12T21:55:48.859 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:48.857+0000 7fa36ea00640 1 -- 172.21.15.18:0/313255472 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa36806f7f0 msgr2=0x7fa368114d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:55:48.859 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:48.857+0000 7fa36ea00640 1 --2- 172.21.15.18:0/313255472 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa36806f7f0 0x7fa368114d80 secure :-1 s=READY pgs=1446 cs=0 l=1 rev1=1 crypto rx=0x7fa35c0047f0 tx=0x7fa35c009da0 comp rx=0 tx=0).stop 2024-04-12T21:55:48.859 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:48.857+0000 7fa36ea00640 1 -- 172.21.15.18:0/313255472 shutdown_connections 2024-04-12T21:55:48.859 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:48.857+0000 7fa36ea00640 1 --2- 172.21.15.18:0/313255472 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7fa35006eb80 0x7fa350071030 unknown :-1 s=CLOSED pgs=765 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:55:48.859 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:48.857+0000 7fa36ea00640 1 --2- 172.21.15.18:0/313255472 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa368070100 0x7fa3681152c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:55:48.859 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:48.857+0000 7fa36ea00640 1 --2- 172.21.15.18:0/313255472 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa36806f7f0 0x7fa368114d80 unknown :-1 s=CLOSED pgs=1446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:55:48.859 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:48.857+0000 7fa36ea00640 1 -- 172.21.15.18:0/313255472 >> 172.21.15.18:0/313255472 conn(0x7fa3680f7c60 msgr2=0x7fa3680f90d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:55:48.859 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:48.857+0000 7fa36ea00640 1 -- 172.21.15.18:0/313255472 shutdown_connections 2024-04-12T21:55:48.859 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:48.857+0000 7fa36ea00640 1 -- 172.21.15.18:0/313255472 wait complete. 2024-04-12T21:55:48.879 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs.cephfs.data pg_num 2024-04-12T21:55:49.093 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:49.092+0000 7f6342a00640 1 -- 172.21.15.18:0/349764706 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6334035470 con 0x7f6344104d60 2024-04-12T21:55:49.094 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:49.092+0000 7f634ac00640 1 -- 172.21.15.18:0/349764706 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6344104d60 msgr2=0x7f63440fe3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:55:49.094 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:49.092+0000 7f634ac00640 1 --2- 172.21.15.18:0/349764706 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6344104d60 0x7f63440fe3f0 secure :-1 s=READY pgs=1447 cs=0 l=1 rev1=1 crypto rx=0x7f6334009c50 tx=0x7f6334004d10 comp rx=0 tx=0).stop 2024-04-12T21:55:49.094 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:49.092+0000 7f634ac00640 1 -- 172.21.15.18:0/349764706 shutdown_connections 2024-04-12T21:55:49.094 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:49.092+0000 7f634ac00640 1 --2- 172.21.15.18:0/349764706 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6344104d60 0x7f63440fe3f0 unknown :-1 s=CLOSED pgs=1447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:55:49.094 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:49.092+0000 7f634ac00640 1 --2- 172.21.15.18:0/349764706 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f63441043c0 0x7f6344104790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:55:49.094 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:49.092+0000 7f634ac00640 1 -- 172.21.15.18:0/349764706 >> 172.21.15.18:0/349764706 conn(0x7f63440f9fc0 msgr2=0x7f63440fc3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:55:49.094 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:49.093+0000 7f634ac00640 1 -- 172.21.15.18:0/349764706 shutdown_connections 2024-04-12T21:55:49.094 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:49.093+0000 7f634ac00640 1 -- 172.21.15.18:0/349764706 wait complete. 2024-04-12T21:55:49.096 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:49.093+0000 7f634ac00640 1 Processor -- start 2024-04-12T21:55:49.096 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:49.094+0000 7f634ac00640 1 -- start start 2024-04-12T21:55:49.096 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:49.094+0000 7f634ac00640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f63441043c0 0x7f63441182d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:55:49.096 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:49.094+0000 7f634ac00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6344104d60 0x7f6344118810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:55:49.096 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:49.094+0000 7f634ac00640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6344118ef0 con 0x7f6344104d60 2024-04-12T21:55:49.096 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:49.094+0000 7f634ac00640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6344113930 con 0x7f63441043c0 2024-04-12T21:55:49.096 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:49.094+0000 7f6343400640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6344104d60 0x7f6344118810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:55:49.096 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:49.094+0000 7f6343400640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6344104d60 0x7f6344118810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:43662/0 (socket says 172.21.15.18:43662) 2024-04-12T21:55:49.096 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:49.094+0000 7f6343400640 1 -- 172.21.15.18:0/1065288014 learned_addr learned my addr 172.21.15.18:0/1065288014 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:55:49.096 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:49.095+0000 7f6343e00640 1 --2- 172.21.15.18:0/1065288014 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f63441043c0 0x7f63441182d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:55:49.096 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:49.095+0000 7f6343400640 1 -- 172.21.15.18:0/1065288014 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f63441043c0 msgr2=0x7f63441182d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:55:49.096 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:49.095+0000 7f6343400640 1 --2- 172.21.15.18:0/1065288014 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f63441043c0 0x7f63441182d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:55:49.097 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:49.095+0000 7f6343400640 1 -- 172.21.15.18:0/1065288014 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63340098b0 con 0x7f6344104d60 2024-04-12T21:55:49.097 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:49.095+0000 7f6343e00640 1 --2- 172.21.15.18:0/1065288014 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f63441043c0 0x7f63441182d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-12T21:55:49.097 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:49.095+0000 7f6343400640 1 --2- 172.21.15.18:0/1065288014 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6344104d60 0x7f6344118810 secure :-1 s=READY pgs=1448 cs=0 l=1 rev1=1 crypto rx=0x7f6334004a30 tx=0x7f6334004d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:55:49.097 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:49.095+0000 7f633be00640 1 -- 172.21.15.18:0/1065288014 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f633403a070 con 0x7f6344104d60 2024-04-12T21:55:49.098 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:49.095+0000 7f633be00640 1 -- 172.21.15.18:0/1065288014 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f6334035b20 con 0x7f6344104d60 2024-04-12T21:55:49.098 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:49.095+0000 7f634ac00640 1 -- 172.21.15.18:0/1065288014 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6344113bb0 con 0x7f6344104d60 2024-04-12T21:55:49.098 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:49.095+0000 7f633be00640 1 -- 172.21.15.18:0/1065288014 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f633403eaa0 con 0x7f6344104d60 2024-04-12T21:55:49.098 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:49.096+0000 7f634ac00640 1 -- 172.21.15.18:0/1065288014 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f63441140d0 con 0x7f6344104d60 2024-04-12T21:55:49.098 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:49.097+0000 7f634ac00640 1 -- 172.21.15.18:0/1065288014 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f63440ffb30 con 0x7f6344104d60 2024-04-12T21:55:49.099 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:49.098+0000 7f633be00640 1 -- 172.21.15.18:0/1065288014 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7f6334032070 con 0x7f6344104d60 2024-04-12T21:55:49.100 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:49.099+0000 7f633be00640 1 --2- 172.21.15.18:0/1065288014 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f632806eb70 0x7f6328071020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:55:49.100 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:49.099+0000 7f633be00640 1 -- 172.21.15.18:0/1065288014 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 6757+0+0 (secure 0 0 0) 0x7f63340b1260 con 0x7f6344104d60 2024-04-12T21:55:49.100 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:49.099+0000 7f6343e00640 1 --2- 172.21.15.18:0/1065288014 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f632806eb70 0x7f6328071020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:55:49.101 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:49.100+0000 7f6343e00640 1 --2- 172.21.15.18:0/1065288014 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f632806eb70 0x7f6328071020 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7f632c006fd0 tx=0x7f632c008040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:55:49.104 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:49.103+0000 7f633be00640 1 -- 172.21.15.18:0/1065288014 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f6334079f00 con 0x7f6344104d60 2024-04-12T21:55:49.311 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:49.309+0000 7f634ac00640 1 -- 172.21.15.18:0/1065288014 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"} v 0) v1 -- 0x7f6344071d70 con 0x7f6344104d60 2024-04-12T21:55:49.312 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:49.310+0000 7f633be00640 1 -- 172.21.15.18:0/1065288014 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"}]=0 v104) v1 ==== 107+0+11 (secure 0 0 0) 0x7f633407d5c0 con 0x7f6344104d60 2024-04-12T21:55:49.312 INFO:teuthology.orchestra.run.smithi018.stdout:pg_num: 32 2024-04-12T21:55:49.315 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:49.313+0000 7f634ac00640 1 -- 172.21.15.18:0/1065288014 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f632806eb70 msgr2=0x7f6328071020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:55:49.315 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:49.313+0000 7f634ac00640 1 --2- 172.21.15.18:0/1065288014 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f632806eb70 0x7f6328071020 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7f632c006fd0 tx=0x7f632c008040 comp rx=0 tx=0).stop 2024-04-12T21:55:49.315 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:49.313+0000 7f634ac00640 1 -- 172.21.15.18:0/1065288014 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6344104d60 msgr2=0x7f6344118810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:55:49.315 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:49.313+0000 7f634ac00640 1 --2- 172.21.15.18:0/1065288014 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6344104d60 0x7f6344118810 secure :-1 s=READY pgs=1448 cs=0 l=1 rev1=1 crypto rx=0x7f6334004a30 tx=0x7f6334004d10 comp rx=0 tx=0).stop 2024-04-12T21:55:49.315 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:49.313+0000 7f634ac00640 1 -- 172.21.15.18:0/1065288014 shutdown_connections 2024-04-12T21:55:49.315 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:49.313+0000 7f634ac00640 1 --2- 172.21.15.18:0/1065288014 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7f632806eb70 0x7f6328071020 unknown :-1 s=CLOSED pgs=766 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:55:49.315 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:49.313+0000 7f634ac00640 1 --2- 172.21.15.18:0/1065288014 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6344104d60 0x7f6344118810 unknown :-1 s=CLOSED pgs=1448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:55:49.315 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:49.313+0000 7f634ac00640 1 --2- 172.21.15.18:0/1065288014 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f63441043c0 0x7f63441182d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:55:49.316 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:49.313+0000 7f634ac00640 1 -- 172.21.15.18:0/1065288014 >> 172.21.15.18:0/1065288014 conn(0x7f63440f9fc0 msgr2=0x7f63440faef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:55:49.316 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:49.314+0000 7f634ac00640 1 -- 172.21.15.18:0/1065288014 shutdown_connections 2024-04-12T21:55:49.316 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:49.314+0000 7f634ac00640 1 -- 172.21.15.18:0/1065288014 wait complete. 2024-04-12T21:55:49.335 DEBUG:teuthology.orchestra.run.smithi018:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-12T21:55:49.559 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:49.557+0000 7ff243200640 1 -- 172.21.15.18:0/1184501757 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff244102010 msgr2=0x7ff2441023e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:55:49.559 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:49.557+0000 7ff243200640 1 --2- 172.21.15.18:0/1184501757 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff244102010 0x7ff2441023e0 secure :-1 s=READY pgs=1449 cs=0 l=1 rev1=1 crypto rx=0x7ff2340048d0 tx=0x7ff234009da0 comp rx=0 tx=0).stop 2024-04-12T21:55:49.559 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:49.557+0000 7ff243200640 1 -- 172.21.15.18:0/1184501757 shutdown_connections 2024-04-12T21:55:49.559 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:49.557+0000 7ff243200640 1 --2- 172.21.15.18:0/1184501757 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff2441029b0 0x7ff2440fc040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:55:49.559 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:49.557+0000 7ff243200640 1 --2- 172.21.15.18:0/1184501757 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff244102010 0x7ff2441023e0 unknown :-1 s=CLOSED pgs=1449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:55:49.559 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:49.557+0000 7ff243200640 1 -- 172.21.15.18:0/1184501757 >> 172.21.15.18:0/1184501757 conn(0x7ff2440f7c20 msgr2=0x7ff2440fa030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:55:49.559 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:49.557+0000 7ff243200640 1 -- 172.21.15.18:0/1184501757 shutdown_connections 2024-04-12T21:55:49.559 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:49.557+0000 7ff243200640 1 -- 172.21.15.18:0/1184501757 wait complete. 2024-04-12T21:55:49.560 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:49.558+0000 7ff243200640 1 Processor -- start 2024-04-12T21:55:49.560 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:49.558+0000 7ff243200640 1 -- start start 2024-04-12T21:55:49.560 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:49.559+0000 7ff243200640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff244102010 0x7ff244112bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:55:49.561 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:49.559+0000 7ff243200640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff2441029b0 0x7ff244113100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:55:49.561 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:49.559+0000 7ff243200640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff244113790 con 0x7ff2441029b0 2024-04-12T21:55:49.561 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:49.559+0000 7ff243200640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff24410cd50 con 0x7ff244102010 2024-04-12T21:55:49.561 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:49.559+0000 7ff241400640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff2441029b0 0x7ff244113100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:55:49.561 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:49.559+0000 7ff241400640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff2441029b0 0x7ff244113100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:43684/0 (socket says 172.21.15.18:43684) 2024-04-12T21:55:49.561 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:49.559+0000 7ff241400640 1 -- 172.21.15.18:0/704394508 learned_addr learned my addr 172.21.15.18:0/704394508 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-04-12T21:55:49.562 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:49.560+0000 7ff241e00640 1 --2- 172.21.15.18:0/704394508 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff244102010 0x7ff244112bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:55:49.562 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:49.560+0000 7ff241400640 1 -- 172.21.15.18:0/704394508 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff244102010 msgr2=0x7ff244112bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:55:49.562 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:49.560+0000 7ff241400640 1 --2- 172.21.15.18:0/704394508 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff244102010 0x7ff244112bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:55:49.562 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:49.560+0000 7ff241400640 1 -- 172.21.15.18:0/704394508 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff234005040 con 0x7ff2441029b0 2024-04-12T21:55:49.562 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:49.560+0000 7ff241400640 1 --2- 172.21.15.18:0/704394508 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff2441029b0 0x7ff244113100 secure :-1 s=READY pgs=1450 cs=0 l=1 rev1=1 crypto rx=0x7ff23000b790 tx=0x7ff2300091d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:55:49.562 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:49.560+0000 7ff23a000640 1 -- 172.21.15.18:0/704394508 <== mon.0 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff23000c700 con 0x7ff2441029b0 2024-04-12T21:55:49.562 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:49.560+0000 7ff23a000640 1 -- 172.21.15.18:0/704394508 <== mon.0 v2:172.21.15.18:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff23000cd30 con 0x7ff2441029b0 2024-04-12T21:55:49.563 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:49.560+0000 7ff23a000640 1 -- 172.21.15.18:0/704394508 <== mon.0 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff230013370 con 0x7ff2441029b0 2024-04-12T21:55:49.563 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:49.560+0000 7ff243200640 1 -- 172.21.15.18:0/704394508 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff24410d030 con 0x7ff2441029b0 2024-04-12T21:55:49.563 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:49.561+0000 7ff243200640 1 -- 172.21.15.18:0/704394508 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff24410d550 con 0x7ff2441029b0 2024-04-12T21:55:49.563 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:49.562+0000 7ff243200640 1 -- 172.21.15.18:0/704394508 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff2440fd780 con 0x7ff2441029b0 2024-04-12T21:55:49.564 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:49.563+0000 7ff23a000640 1 -- 172.21.15.18:0/704394508 <== mon.0 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 39) v1 ==== 92745+0+0 (secure 0 0 0) 0x7ff230012070 con 0x7ff2441029b0 2024-04-12T21:55:49.566 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:49.564+0000 7ff23a000640 1 --2- 172.21.15.18:0/704394508 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7ff22806eb80 0x7ff228071030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T21:55:49.566 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:49.565+0000 7ff23a000640 1 -- 172.21.15.18:0/704394508 <== mon.0 v2:172.21.15.18:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 6757+0+0 (secure 0 0 0) 0x7ff23008e3c0 con 0x7ff2441029b0 2024-04-12T21:55:49.566 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:49.565+0000 7ff241e00640 1 --2- 172.21.15.18:0/704394508 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7ff22806eb80 0x7ff228071030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T21:55:49.567 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:49.566+0000 7ff241e00640 1 --2- 172.21.15.18:0/704394508 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7ff22806eb80 0x7ff228071030 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7ff2340048a0 tx=0x7ff234009da0 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T21:55:49.570 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:49.569+0000 7ff23a000640 1 -- 172.21.15.18:0/704394508 <== mon.0 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7ff230057040 con 0x7ff2441029b0 2024-04-12T21:55:49.606 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:55:49 smithi018 ceph-mon[112590]: from='client.? 172.21.15.18:0/3545982562' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-12T21:55:49.606 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:55:49 smithi018 ceph-mon[112590]: from='client.? 172.21.15.18:0/313255472' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"}]: dispatch 2024-04-12T21:55:49.606 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:55:49 smithi018 ceph-mon[112590]: from='client.? 172.21.15.18:0/1065288014' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"}]: dispatch 2024-04-12T21:55:49.785 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:55:49 smithi186 ceph-mon[86379]: from='client.? 172.21.15.18:0/3545982562' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-12T21:55:49.785 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:55:49 smithi186 ceph-mon[86379]: from='client.? 172.21.15.18:0/313255472' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"}]: dispatch 2024-04-12T21:55:49.785 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:55:49 smithi186 ceph-mon[86379]: from='client.? 172.21.15.18:0/1065288014' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"}]: dispatch 2024-04-12T21:55:49.797 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:49.796+0000 7ff243200640 1 -- 172.21.15.18:0/704394508 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7ff244109230 con 0x7ff2441029b0 2024-04-12T21:55:49.798 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:49.797+0000 7ff23a000640 1 -- 172.21.15.18:0/704394508 <== mon.0 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 39 entries v104) v1 ==== 81+0+2318 (secure 0 0 0) 0x7ff23005a700 con 0x7ff2441029b0 2024-04-12T21:55:49.799 INFO:teuthology.orchestra.run.smithi018.stdout:172.21.15.18:6826/1564065434 2024-04-13T21:06:59.717116+0000 2024-04-12T21:55:49.799 INFO:teuthology.orchestra.run.smithi018.stdout:172.21.15.186:0/825752072 2024-04-13T20:57:12.501044+0000 2024-04-12T21:55:49.799 INFO:teuthology.orchestra.run.smithi018.stdout:172.21.15.18:6800/1391635591 2024-04-13T20:45:44.776072+0000 2024-04-12T21:55:49.799 INFO:teuthology.orchestra.run.smithi018.stdout:172.21.15.18:0/4018777476 2024-04-13T20:45:44.776072+0000 2024-04-12T21:55:49.799 INFO:teuthology.orchestra.run.smithi018.stdout:172.21.15.186:6824/1645616336 2024-04-13T20:52:14.968672+0000 2024-04-12T21:55:49.799 INFO:teuthology.orchestra.run.smithi018.stdout:172.21.15.18:0/3387911038 2024-04-13T20:44:15.143139+0000 2024-04-12T21:55:49.799 INFO:teuthology.orchestra.run.smithi018.stdout:172.21.15.18:0/714775767 2024-04-13T20:54:59.478287+0000 2024-04-12T21:55:49.799 INFO:teuthology.orchestra.run.smithi018.stdout:172.21.15.18:6801/3048894687 2024-04-13T20:54:59.478287+0000 2024-04-12T21:55:49.799 INFO:teuthology.orchestra.run.smithi018.stdout:172.21.15.18:0/2171204556 2024-04-13T21:01:39.724018+0000 2024-04-12T21:55:49.799 INFO:teuthology.orchestra.run.smithi018.stdout:172.21.15.186:0/2147813742 2024-04-13T20:57:12.501044+0000 2024-04-12T21:55:49.799 INFO:teuthology.orchestra.run.smithi018.stdout:172.21.15.18:6829/722443353 2024-04-13T21:06:59.717165+0000 2024-04-12T21:55:49.799 INFO:teuthology.orchestra.run.smithi018.stdout:172.21.15.18:0/2945432483 2024-04-13T20:44:15.143139+0000 2024-04-12T21:55:49.799 INFO:teuthology.orchestra.run.smithi018.stdout:172.21.15.18:6827/1564065434 2024-04-13T21:06:59.717116+0000 2024-04-12T21:55:49.800 INFO:teuthology.orchestra.run.smithi018.stdout:172.21.15.186:6829/2316349902 2024-04-13T20:57:12.501044+0000 2024-04-12T21:55:49.800 INFO:teuthology.orchestra.run.smithi018.stdout:172.21.15.18:0/2956836690 2024-04-13T20:54:59.478287+0000 2024-04-12T21:55:49.800 INFO:teuthology.orchestra.run.smithi018.stdout:172.21.15.18:0/1732042827 2024-04-13T20:43:34.269559+0000 2024-04-12T21:55:49.800 INFO:teuthology.orchestra.run.smithi018.stdout:172.21.15.18:6800/3403971131 2024-04-13T20:44:15.143139+0000 2024-04-12T21:55:49.800 INFO:teuthology.orchestra.run.smithi018.stdout:172.21.15.18:6800/1605950634 2024-04-13T21:01:39.724018+0000 2024-04-12T21:55:49.800 INFO:teuthology.orchestra.run.smithi018.stdout:172.21.15.186:0/970502145 2024-04-13T20:57:12.501044+0000 2024-04-12T21:55:49.800 INFO:teuthology.orchestra.run.smithi018.stdout:172.21.15.18:6800/3048894687 2024-04-13T20:54:59.478287+0000 2024-04-12T21:55:49.800 INFO:teuthology.orchestra.run.smithi018.stdout:172.21.15.18:0/2330205732 2024-04-13T20:54:59.478287+0000 2024-04-12T21:55:49.800 INFO:teuthology.orchestra.run.smithi018.stdout:172.21.15.18:0/3227425848 2024-04-13T20:54:59.478287+0000 2024-04-12T21:55:49.800 INFO:teuthology.orchestra.run.smithi018.stdout:172.21.15.18:6801/1312802974 2024-04-13T20:43:34.269559+0000 2024-04-12T21:55:49.800 INFO:teuthology.orchestra.run.smithi018.stdout:172.21.15.186:0/2459326506 2024-04-13T20:57:12.501044+0000 2024-04-12T21:55:49.800 INFO:teuthology.orchestra.run.smithi018.stdout:172.21.15.18:0/3896210349 2024-04-13T20:44:15.143139+0000 2024-04-12T21:55:49.800 INFO:teuthology.orchestra.run.smithi018.stdout:172.21.15.18:0/1720919549 2024-04-13T20:45:44.776072+0000 2024-04-12T21:55:49.801 INFO:teuthology.orchestra.run.smithi018.stdout:172.21.15.18:0/766256564 2024-04-13T20:45:44.776072+0000 2024-04-12T21:55:49.801 INFO:teuthology.orchestra.run.smithi018.stdout:172.21.15.18:6801/3403971131 2024-04-13T20:44:15.143139+0000 2024-04-12T21:55:49.801 INFO:teuthology.orchestra.run.smithi018.stdout:172.21.15.18:6828/722443353 2024-04-13T21:06:59.717165+0000 2024-04-12T21:55:49.801 INFO:teuthology.orchestra.run.smithi018.stdout:172.21.15.18:0/3295870133 2024-04-13T20:43:34.269559+0000 2024-04-12T21:55:49.801 INFO:teuthology.orchestra.run.smithi018.stdout:172.21.15.18:6800/1312802974 2024-04-13T20:43:34.269559+0000 2024-04-12T21:55:49.801 INFO:teuthology.orchestra.run.smithi018.stdout:172.21.15.186:6828/2316349902 2024-04-13T20:57:12.501044+0000 2024-04-12T21:55:49.801 INFO:teuthology.orchestra.run.smithi018.stdout:172.21.15.186:6825/1645616336 2024-04-13T20:52:14.968672+0000 2024-04-12T21:55:49.801 INFO:teuthology.orchestra.run.smithi018.stdout:172.21.15.18:0/3745234714 2024-04-13T21:01:39.724018+0000 2024-04-12T21:55:49.801 INFO:teuthology.orchestra.run.smithi018.stdout:172.21.15.18:0/2356171563 2024-04-13T21:01:39.724018+0000 2024-04-12T21:55:49.801 INFO:teuthology.orchestra.run.smithi018.stdout:172.21.15.18:6801/1391635591 2024-04-13T20:45:44.776072+0000 2024-04-12T21:55:49.801 INFO:teuthology.orchestra.run.smithi018.stdout:172.21.15.18:0/2517252399 2024-04-13T20:43:34.269559+0000 2024-04-12T21:55:49.801 INFO:teuthology.orchestra.run.smithi018.stdout:172.21.15.18:6801/1605950634 2024-04-13T21:01:39.724018+0000 2024-04-12T21:55:49.801 INFO:teuthology.orchestra.run.smithi018.stdout:172.21.15.18:0/3861256640 2024-04-13T21:01:39.724018+0000 2024-04-12T21:55:49.802 INFO:teuthology.orchestra.run.smithi018.stderr:listed 39 entries 2024-04-12T21:55:49.802 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:49.800+0000 7ff243200640 1 -- 172.21.15.18:0/704394508 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7ff22806eb80 msgr2=0x7ff228071030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:55:49.802 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:49.800+0000 7ff243200640 1 --2- 172.21.15.18:0/704394508 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7ff22806eb80 0x7ff228071030 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7ff2340048a0 tx=0x7ff234009da0 comp rx=0 tx=0).stop 2024-04-12T21:55:49.803 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:49.800+0000 7ff243200640 1 -- 172.21.15.18:0/704394508 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff2441029b0 msgr2=0x7ff244113100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T21:55:49.803 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:49.800+0000 7ff243200640 1 --2- 172.21.15.18:0/704394508 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff2441029b0 0x7ff244113100 secure :-1 s=READY pgs=1450 cs=0 l=1 rev1=1 crypto rx=0x7ff23000b790 tx=0x7ff2300091d0 comp rx=0 tx=0).stop 2024-04-12T21:55:49.803 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:49.800+0000 7ff243200640 1 -- 172.21.15.18:0/704394508 shutdown_connections 2024-04-12T21:55:49.803 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:49.800+0000 7ff243200640 1 --2- 172.21.15.18:0/704394508 >> [v2:172.21.15.18:6800/3812891919,v1:172.21.15.18:6801/3812891919] conn(0x7ff22806eb80 0x7ff228071030 unknown :-1 s=CLOSED pgs=767 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:55:49.803 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:49.800+0000 7ff243200640 1 --2- 172.21.15.18:0/704394508 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff2441029b0 0x7ff244113100 unknown :-1 s=CLOSED pgs=1450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:55:49.803 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:49.800+0000 7ff243200640 1 --2- 172.21.15.18:0/704394508 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff244102010 0x7ff244112bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T21:55:49.803 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:49.800+0000 7ff243200640 1 -- 172.21.15.18:0/704394508 >> 172.21.15.18:0/704394508 conn(0x7ff2440f7c20 msgr2=0x7ff2440f8b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T21:55:49.803 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:49.800+0000 7ff243200640 1 -- 172.21.15.18:0/704394508 shutdown_connections 2024-04-12T21:55:49.803 INFO:teuthology.orchestra.run.smithi018.stderr:2024-04-12T21:55:49.800+0000 7ff243200640 1 -- 172.21.15.18:0/704394508 wait complete. 2024-04-12T21:55:49.821 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi018.front.sepia.ceph.com... 2024-04-12T21:55:49.822 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-12T21:55:49.822 DEBUG:teuthology.orchestra.run.smithi018:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-04-12T21:55:49.918 INFO:teuthology.orchestra.run:waiting for 300 2024-04-12T21:55:50.606 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:55:50 smithi018 ceph-mon[112590]: from='client.? 172.21.15.18:0/704394508' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-04-12T21:55:50.606 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:55:50 smithi018 ceph-mon[112590]: pgmap v1687: 65 pgs: 65 active+clean; 257 MiB data, 976 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:55:50.785 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:55:50 smithi186 ceph-mon[86379]: from='client.? 172.21.15.18:0/704394508' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-04-12T21:55:50.785 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:55:50 smithi186 ceph-mon[86379]: pgmap v1687: 65 pgs: 65 active+clean; 257 MiB data, 976 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:55:52.534 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:55:52 smithi186 ceph-mon[86379]: pgmap v1688: 65 pgs: 65 active+clean; 257 MiB data, 976 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:55:52.605 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:55:52 smithi018 ceph-mon[112590]: pgmap v1688: 65 pgs: 65 active+clean; 257 MiB data, 976 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:55:54.535 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:55:54 smithi186 ceph-mon[86379]: pgmap v1689: 65 pgs: 65 active+clean; 257 MiB data, 976 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:55:54.605 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:55:54 smithi018 ceph-mon[112590]: pgmap v1689: 65 pgs: 65 active+clean; 257 MiB data, 976 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:55:55.535 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:55:55 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T21:55:55.606 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:55:55 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T21:55:56.535 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:55:56 smithi186 ceph-mon[86379]: pgmap v1690: 65 pgs: 65 active+clean; 257 MiB data, 976 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:55:56.606 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:55:56 smithi018 ceph-mon[112590]: pgmap v1690: 65 pgs: 65 active+clean; 257 MiB data, 976 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:55:58.534 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:55:58 smithi186 ceph-mon[86379]: pgmap v1691: 65 pgs: 65 active+clean; 257 MiB data, 976 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:55:58.606 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:55:58 smithi018 ceph-mon[112590]: pgmap v1691: 65 pgs: 65 active+clean; 257 MiB data, 976 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:56:00.535 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:56:00 smithi186 ceph-mon[86379]: pgmap v1692: 65 pgs: 65 active+clean; 257 MiB data, 976 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:56:00.606 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:56:00 smithi018 ceph-mon[112590]: pgmap v1692: 65 pgs: 65 active+clean; 257 MiB data, 976 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:56:02.535 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:56:02 smithi186 ceph-mon[86379]: pgmap v1693: 65 pgs: 65 active+clean; 257 MiB data, 976 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:56:02.606 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:56:02 smithi018 ceph-mon[112590]: pgmap v1693: 65 pgs: 65 active+clean; 257 MiB data, 976 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:56:04.535 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:56:04 smithi186 ceph-mon[86379]: pgmap v1694: 65 pgs: 65 active+clean; 257 MiB data, 976 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:56:04.606 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:56:04 smithi018 ceph-mon[112590]: pgmap v1694: 65 pgs: 65 active+clean; 257 MiB data, 976 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:56:06.534 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:56:06 smithi186 ceph-mon[86379]: pgmap v1695: 65 pgs: 65 active+clean; 257 MiB data, 976 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:56:06.606 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:56:06 smithi018 ceph-mon[112590]: pgmap v1695: 65 pgs: 65 active+clean; 257 MiB data, 976 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:56:08.535 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:56:08 smithi186 ceph-mon[86379]: pgmap v1696: 65 pgs: 65 active+clean; 257 MiB data, 976 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:56:08.606 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:56:08 smithi018 ceph-mon[112590]: pgmap v1696: 65 pgs: 65 active+clean; 257 MiB data, 976 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:56:10.285 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:56:10 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T21:56:10.355 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:56:10 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T21:56:11.284 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:56:11 smithi186 ceph-mon[86379]: pgmap v1697: 65 pgs: 65 active+clean; 257 MiB data, 976 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:56:11.355 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:56:11 smithi018 ceph-mon[112590]: pgmap v1697: 65 pgs: 65 active+clean; 257 MiB data, 976 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:56:12.535 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:56:12 smithi186 ceph-mon[86379]: pgmap v1698: 65 pgs: 65 active+clean; 257 MiB data, 976 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:56:12.606 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:56:12 smithi018 ceph-mon[112590]: pgmap v1698: 65 pgs: 65 active+clean; 257 MiB data, 976 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:56:14.535 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:56:14 smithi186 ceph-mon[86379]: pgmap v1699: 65 pgs: 65 active+clean; 257 MiB data, 976 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:56:14.606 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:56:14 smithi018 ceph-mon[112590]: pgmap v1699: 65 pgs: 65 active+clean; 257 MiB data, 976 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:56:16.535 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:56:16 smithi186 ceph-mon[86379]: pgmap v1700: 65 pgs: 65 active+clean; 257 MiB data, 976 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:56:16.540 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:56:16 smithi018 ceph-mon[112590]: pgmap v1700: 65 pgs: 65 active+clean; 257 MiB data, 976 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:56:18.535 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:56:18 smithi186 ceph-mon[86379]: pgmap v1701: 65 pgs: 65 active+clean; 257 MiB data, 976 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:56:18.605 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:56:18 smithi018 ceph-mon[112590]: pgmap v1701: 65 pgs: 65 active+clean; 257 MiB data, 976 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:56:20.535 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:56:20 smithi186 ceph-mon[86379]: pgmap v1702: 65 pgs: 65 active+clean; 257 MiB data, 976 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:56:20.605 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:56:20 smithi018 ceph-mon[112590]: pgmap v1702: 65 pgs: 65 active+clean; 257 MiB data, 976 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:56:22.535 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:56:22 smithi186 ceph-mon[86379]: pgmap v1703: 65 pgs: 65 active+clean; 257 MiB data, 976 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:56:22.606 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:56:22 smithi018 ceph-mon[112590]: pgmap v1703: 65 pgs: 65 active+clean; 257 MiB data, 976 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:56:24.534 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:56:24 smithi186 ceph-mon[86379]: pgmap v1704: 65 pgs: 65 active+clean; 257 MiB data, 976 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:56:24.606 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:56:24 smithi018 ceph-mon[112590]: pgmap v1704: 65 pgs: 65 active+clean; 257 MiB data, 976 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:56:25.534 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:56:25 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T21:56:25.606 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:56:25 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T21:56:26.534 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:56:26 smithi186 ceph-mon[86379]: pgmap v1705: 65 pgs: 65 active+clean; 257 MiB data, 976 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:56:26.606 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:56:26 smithi018 ceph-mon[112590]: pgmap v1705: 65 pgs: 65 active+clean; 257 MiB data, 976 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:56:28.535 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:56:28 smithi186 ceph-mon[86379]: pgmap v1706: 65 pgs: 65 active+clean; 257 MiB data, 976 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:56:28.606 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:56:28 smithi018 ceph-mon[112590]: pgmap v1706: 65 pgs: 65 active+clean; 257 MiB data, 976 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:56:30.535 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:56:30 smithi186 ceph-mon[86379]: pgmap v1707: 65 pgs: 65 active+clean; 257 MiB data, 976 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:56:30.606 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:56:30 smithi018 ceph-mon[112590]: pgmap v1707: 65 pgs: 65 active+clean; 257 MiB data, 976 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:56:32.535 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:56:32 smithi186 ceph-mon[86379]: pgmap v1708: 65 pgs: 65 active+clean; 257 MiB data, 976 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:56:32.606 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:56:32 smithi018 ceph-mon[112590]: pgmap v1708: 65 pgs: 65 active+clean; 257 MiB data, 976 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:56:34.535 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:56:34 smithi186 ceph-mon[86379]: pgmap v1709: 65 pgs: 65 active+clean; 257 MiB data, 976 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:56:34.606 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:56:34 smithi018 ceph-mon[112590]: pgmap v1709: 65 pgs: 65 active+clean; 257 MiB data, 976 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:56:36.535 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:56:36 smithi186 ceph-mon[86379]: pgmap v1710: 65 pgs: 65 active+clean; 257 MiB data, 976 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:56:36.606 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:56:36 smithi018 ceph-mon[112590]: pgmap v1710: 65 pgs: 65 active+clean; 257 MiB data, 976 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:56:38.535 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:56:38 smithi186 ceph-mon[86379]: pgmap v1711: 65 pgs: 65 active+clean; 257 MiB data, 976 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:56:38.606 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:56:38 smithi018 ceph-mon[112590]: pgmap v1711: 65 pgs: 65 active+clean; 257 MiB data, 976 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:56:40.285 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:56:40 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T21:56:40.356 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:56:40 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T21:56:41.285 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:56:41 smithi186 ceph-mon[86379]: pgmap v1712: 65 pgs: 65 active+clean; 257 MiB data, 976 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:56:41.356 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:56:41 smithi018 ceph-mon[112590]: pgmap v1712: 65 pgs: 65 active+clean; 257 MiB data, 976 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:56:42.535 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:56:42 smithi186 ceph-mon[86379]: pgmap v1713: 65 pgs: 65 active+clean; 257 MiB data, 976 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:56:42.606 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:56:42 smithi018 ceph-mon[112590]: pgmap v1713: 65 pgs: 65 active+clean; 257 MiB data, 976 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:56:44.535 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:56:44 smithi186 ceph-mon[86379]: pgmap v1714: 65 pgs: 65 active+clean; 257 MiB data, 976 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:56:44.606 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:56:44 smithi018 ceph-mon[112590]: pgmap v1714: 65 pgs: 65 active+clean; 257 MiB data, 976 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:56:46.535 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:56:46 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-12T21:56:46.535 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:56:46 smithi186 ceph-mon[86379]: pgmap v1715: 65 pgs: 65 active+clean; 257 MiB data, 976 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:56:46.605 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:56:46 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-12T21:56:46.606 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:56:46 smithi018 ceph-mon[112590]: pgmap v1715: 65 pgs: 65 active+clean; 257 MiB data, 976 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:56:48.035 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:56:47 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-12T21:56:48.035 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:56:47 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-12T21:56:48.035 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:56:47 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' 2024-04-12T21:56:48.106 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:56:47 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-12T21:56:48.106 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:56:47 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-12T21:56:48.106 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:56:47 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' 2024-04-12T21:56:49.035 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:56:48 smithi186 ceph-mon[86379]: pgmap v1716: 65 pgs: 65 active+clean; 257 MiB data, 976 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:56:49.106 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:56:48 smithi018 ceph-mon[112590]: pgmap v1716: 65 pgs: 65 active+clean; 257 MiB data, 976 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:56:50.535 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:56:50 smithi186 ceph-mon[86379]: pgmap v1717: 65 pgs: 65 active+clean; 257 MiB data, 976 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:56:50.606 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:56:50 smithi018 ceph-mon[112590]: pgmap v1717: 65 pgs: 65 active+clean; 257 MiB data, 976 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:56:52.535 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:56:52 smithi186 ceph-mon[86379]: pgmap v1718: 65 pgs: 65 active+clean; 257 MiB data, 976 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:56:52.606 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:56:52 smithi018 ceph-mon[112590]: pgmap v1718: 65 pgs: 65 active+clean; 257 MiB data, 976 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:56:54.535 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:56:54 smithi186 ceph-mon[86379]: pgmap v1719: 65 pgs: 65 active+clean; 257 MiB data, 976 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:56:54.606 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:56:54 smithi018 ceph-mon[112590]: pgmap v1719: 65 pgs: 65 active+clean; 257 MiB data, 976 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:56:55.535 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:56:55 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T21:56:55.606 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:56:55 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T21:56:56.535 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:56:56 smithi186 ceph-mon[86379]: pgmap v1720: 65 pgs: 65 active+clean; 257 MiB data, 976 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:56:56.606 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:56:56 smithi018 ceph-mon[112590]: pgmap v1720: 65 pgs: 65 active+clean; 257 MiB data, 976 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:56:58.535 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:56:58 smithi186 ceph-mon[86379]: pgmap v1721: 65 pgs: 65 active+clean; 257 MiB data, 976 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:56:58.606 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:56:58 smithi018 ceph-mon[112590]: pgmap v1721: 65 pgs: 65 active+clean; 257 MiB data, 976 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:57:00.535 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:57:00 smithi186 ceph-mon[86379]: pgmap v1722: 65 pgs: 65 active+clean; 257 MiB data, 976 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:57:00.606 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:57:00 smithi018 ceph-mon[112590]: pgmap v1722: 65 pgs: 65 active+clean; 257 MiB data, 976 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:57:02.535 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:57:02 smithi186 ceph-mon[86379]: pgmap v1723: 65 pgs: 65 active+clean; 257 MiB data, 976 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:57:02.606 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:57:02 smithi018 ceph-mon[112590]: pgmap v1723: 65 pgs: 65 active+clean; 257 MiB data, 976 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:57:04.535 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:57:04 smithi186 ceph-mon[86379]: pgmap v1724: 65 pgs: 65 active+clean; 257 MiB data, 976 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:57:04.606 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:57:04 smithi018 ceph-mon[112590]: pgmap v1724: 65 pgs: 65 active+clean; 257 MiB data, 976 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:57:06.535 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:57:06 smithi186 ceph-mon[86379]: pgmap v1725: 65 pgs: 65 active+clean; 257 MiB data, 976 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:57:06.606 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:57:06 smithi018 ceph-mon[112590]: pgmap v1725: 65 pgs: 65 active+clean; 257 MiB data, 976 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:57:08.535 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:57:08 smithi186 ceph-mon[86379]: pgmap v1726: 65 pgs: 65 active+clean; 257 MiB data, 976 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:57:08.606 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:57:08 smithi018 ceph-mon[112590]: pgmap v1726: 65 pgs: 65 active+clean; 257 MiB data, 976 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:57:10.285 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:57:10 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T21:57:10.356 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:57:10 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T21:57:11.285 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:57:11 smithi186 ceph-mon[86379]: pgmap v1727: 65 pgs: 65 active+clean; 257 MiB data, 976 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:57:11.356 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:57:11 smithi018 ceph-mon[112590]: pgmap v1727: 65 pgs: 65 active+clean; 257 MiB data, 976 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:57:12.535 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:57:12 smithi186 ceph-mon[86379]: pgmap v1728: 65 pgs: 65 active+clean; 257 MiB data, 976 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:57:12.606 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:57:12 smithi018 ceph-mon[112590]: pgmap v1728: 65 pgs: 65 active+clean; 257 MiB data, 976 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:57:14.535 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:57:14 smithi186 ceph-mon[86379]: pgmap v1729: 65 pgs: 65 active+clean; 257 MiB data, 976 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:57:14.606 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:57:14 smithi018 ceph-mon[112590]: pgmap v1729: 65 pgs: 65 active+clean; 257 MiB data, 976 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:57:16.535 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:57:16 smithi186 ceph-mon[86379]: pgmap v1730: 65 pgs: 65 active+clean; 257 MiB data, 976 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:57:16.606 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:57:16 smithi018 ceph-mon[112590]: pgmap v1730: 65 pgs: 65 active+clean; 257 MiB data, 976 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:57:18.535 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:57:18 smithi186 ceph-mon[86379]: pgmap v1731: 65 pgs: 65 active+clean; 257 MiB data, 976 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:57:18.606 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:57:18 smithi018 ceph-mon[112590]: pgmap v1731: 65 pgs: 65 active+clean; 257 MiB data, 976 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:57:20.535 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:57:20 smithi186 ceph-mon[86379]: pgmap v1732: 65 pgs: 65 active+clean; 257 MiB data, 976 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:57:20.606 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:57:20 smithi018 ceph-mon[112590]: pgmap v1732: 65 pgs: 65 active+clean; 257 MiB data, 976 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:57:22.535 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:57:22 smithi186 ceph-mon[86379]: pgmap v1733: 65 pgs: 65 active+clean; 257 MiB data, 976 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:57:22.606 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:57:22 smithi018 ceph-mon[112590]: pgmap v1733: 65 pgs: 65 active+clean; 257 MiB data, 976 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:57:24.535 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:57:24 smithi186 ceph-mon[86379]: pgmap v1734: 65 pgs: 65 active+clean; 257 MiB data, 976 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:57:24.606 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:57:24 smithi018 ceph-mon[112590]: pgmap v1734: 65 pgs: 65 active+clean; 257 MiB data, 976 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:57:25.535 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:57:25 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T21:57:25.606 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:57:25 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T21:57:26.535 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:57:26 smithi186 ceph-mon[86379]: pgmap v1735: 65 pgs: 65 active+clean; 257 MiB data, 976 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:57:26.606 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:57:26 smithi018 ceph-mon[112590]: pgmap v1735: 65 pgs: 65 active+clean; 257 MiB data, 976 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:57:28.535 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:57:28 smithi186 ceph-mon[86379]: pgmap v1736: 65 pgs: 65 active+clean; 257 MiB data, 976 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:57:28.605 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:57:28 smithi018 ceph-mon[112590]: pgmap v1736: 65 pgs: 65 active+clean; 257 MiB data, 976 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:57:30.535 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:57:30 smithi186 ceph-mon[86379]: pgmap v1737: 65 pgs: 65 active+clean; 257 MiB data, 976 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:57:30.606 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:57:30 smithi018 ceph-mon[112590]: pgmap v1737: 65 pgs: 65 active+clean; 257 MiB data, 976 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:57:32.535 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:57:32 smithi186 ceph-mon[86379]: pgmap v1738: 65 pgs: 65 active+clean; 257 MiB data, 976 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:57:32.606 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:57:32 smithi018 ceph-mon[112590]: pgmap v1738: 65 pgs: 65 active+clean; 257 MiB data, 976 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:57:34.535 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:57:34 smithi186 ceph-mon[86379]: pgmap v1739: 65 pgs: 65 active+clean; 257 MiB data, 976 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:57:34.606 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:57:34 smithi018 ceph-mon[112590]: pgmap v1739: 65 pgs: 65 active+clean; 257 MiB data, 976 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:57:36.535 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:57:36 smithi186 ceph-mon[86379]: pgmap v1740: 65 pgs: 65 active+clean; 257 MiB data, 976 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:57:36.606 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:57:36 smithi018 ceph-mon[112590]: pgmap v1740: 65 pgs: 65 active+clean; 257 MiB data, 976 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:57:38.535 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:57:38 smithi186 ceph-mon[86379]: pgmap v1741: 65 pgs: 65 active+clean; 257 MiB data, 976 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:57:38.606 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:57:38 smithi018 ceph-mon[112590]: pgmap v1741: 65 pgs: 65 active+clean; 257 MiB data, 976 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:57:40.285 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:57:40 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T21:57:40.356 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:57:40 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T21:57:41.285 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:57:41 smithi186 ceph-mon[86379]: pgmap v1742: 65 pgs: 65 active+clean; 257 MiB data, 976 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:57:41.356 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:57:41 smithi018 ceph-mon[112590]: pgmap v1742: 65 pgs: 65 active+clean; 257 MiB data, 976 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:57:42.535 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:57:42 smithi186 ceph-mon[86379]: pgmap v1743: 65 pgs: 65 active+clean; 257 MiB data, 976 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:57:42.606 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:57:42 smithi018 ceph-mon[112590]: pgmap v1743: 65 pgs: 65 active+clean; 257 MiB data, 976 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:57:44.535 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:57:44 smithi186 ceph-mon[86379]: pgmap v1744: 65 pgs: 65 active+clean; 257 MiB data, 976 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:57:44.606 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:57:44 smithi018 ceph-mon[112590]: pgmap v1744: 65 pgs: 65 active+clean; 257 MiB data, 976 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:57:46.535 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:57:46 smithi186 ceph-mon[86379]: pgmap v1745: 65 pgs: 65 active+clean; 257 MiB data, 976 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:57:46.606 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:57:46 smithi018 ceph-mon[112590]: pgmap v1745: 65 pgs: 65 active+clean; 257 MiB data, 976 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:57:48.285 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:57:47 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-12T21:57:48.355 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:57:47 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-12T21:57:50.285 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:57:49 smithi186 ceph-mon[86379]: pgmap v1746: 65 pgs: 65 active+clean; 257 MiB data, 976 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:57:50.285 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:57:49 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-12T21:57:50.285 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:57:49 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-12T21:57:50.285 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:57:49 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' 2024-04-12T21:57:50.356 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:57:49 smithi018 ceph-mon[112590]: pgmap v1746: 65 pgs: 65 active+clean; 257 MiB data, 976 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:57:50.356 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:57:49 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-12T21:57:50.356 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:57:49 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-12T21:57:50.356 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:57:49 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' 2024-04-12T21:57:52.285 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:57:51 smithi186 ceph-mon[86379]: pgmap v1747: 65 pgs: 65 active+clean; 257 MiB data, 976 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:57:52.356 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:57:51 smithi018 ceph-mon[112590]: pgmap v1747: 65 pgs: 65 active+clean; 257 MiB data, 976 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:57:54.285 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:57:54 smithi186 ceph-mon[86379]: pgmap v1748: 65 pgs: 65 active+clean; 257 MiB data, 976 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:57:54.356 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:57:54 smithi018 ceph-mon[112590]: pgmap v1748: 65 pgs: 65 active+clean; 257 MiB data, 976 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:57:55.285 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:57:55 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T21:57:55.356 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:57:55 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T21:57:56.285 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:57:56 smithi186 ceph-mon[86379]: pgmap v1749: 65 pgs: 65 active+clean; 257 MiB data, 976 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:57:56.356 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:57:56 smithi018 ceph-mon[112590]: pgmap v1749: 65 pgs: 65 active+clean; 257 MiB data, 976 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:57:58.285 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:57:58 smithi186 ceph-mon[86379]: pgmap v1750: 65 pgs: 65 active+clean; 257 MiB data, 976 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:57:58.356 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:57:58 smithi018 ceph-mon[112590]: pgmap v1750: 65 pgs: 65 active+clean; 257 MiB data, 976 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:58:00.285 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:58:00 smithi186 ceph-mon[86379]: pgmap v1751: 65 pgs: 65 active+clean; 257 MiB data, 976 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:58:00.356 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:58:00 smithi018 ceph-mon[112590]: pgmap v1751: 65 pgs: 65 active+clean; 257 MiB data, 976 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:58:02.356 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:58:02 smithi018 ceph-mon[112590]: pgmap v1752: 65 pgs: 65 active+clean; 257 MiB data, 976 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:58:02.535 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:58:02 smithi186 ceph-mon[86379]: pgmap v1752: 65 pgs: 65 active+clean; 257 MiB data, 976 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:58:04.356 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:58:04 smithi018 ceph-mon[112590]: pgmap v1753: 65 pgs: 65 active+clean; 257 MiB data, 976 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:58:04.535 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:58:04 smithi186 ceph-mon[86379]: pgmap v1753: 65 pgs: 65 active+clean; 257 MiB data, 976 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:58:06.356 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:58:06 smithi018 ceph-mon[112590]: pgmap v1754: 65 pgs: 65 active+clean; 257 MiB data, 976 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:58:06.535 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:58:06 smithi186 ceph-mon[86379]: pgmap v1754: 65 pgs: 65 active+clean; 257 MiB data, 976 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:58:08.356 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:58:08 smithi018 ceph-mon[112590]: pgmap v1755: 65 pgs: 65 active+clean; 257 MiB data, 976 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:58:08.535 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:58:08 smithi186 ceph-mon[86379]: pgmap v1755: 65 pgs: 65 active+clean; 257 MiB data, 976 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:58:10.356 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:58:10 smithi018 ceph-mon[112590]: pgmap v1756: 65 pgs: 65 active+clean; 257 MiB data, 976 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:58:10.356 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:58:10 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T21:58:10.535 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:58:10 smithi186 ceph-mon[86379]: pgmap v1756: 65 pgs: 65 active+clean; 257 MiB data, 976 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:58:10.535 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:58:10 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T21:58:12.356 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:58:12 smithi018 ceph-mon[112590]: pgmap v1757: 65 pgs: 65 active+clean; 257 MiB data, 976 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:58:12.535 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:58:12 smithi186 ceph-mon[86379]: pgmap v1757: 65 pgs: 65 active+clean; 257 MiB data, 976 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:58:14.356 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:58:14 smithi018 ceph-mon[112590]: pgmap v1758: 65 pgs: 65 active+clean; 257 MiB data, 976 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:58:14.535 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:58:14 smithi186 ceph-mon[86379]: pgmap v1758: 65 pgs: 65 active+clean; 257 MiB data, 976 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:58:16.356 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:58:16 smithi018 ceph-mon[112590]: pgmap v1759: 65 pgs: 65 active+clean; 257 MiB data, 976 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:58:16.535 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:58:16 smithi186 ceph-mon[86379]: pgmap v1759: 65 pgs: 65 active+clean; 257 MiB data, 976 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:58:18.356 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:58:18 smithi018 ceph-mon[112590]: pgmap v1760: 65 pgs: 65 active+clean; 257 MiB data, 976 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:58:18.535 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:58:18 smithi186 ceph-mon[86379]: pgmap v1760: 65 pgs: 65 active+clean; 257 MiB data, 976 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:58:20.356 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:58:20 smithi018 ceph-mon[112590]: pgmap v1761: 65 pgs: 65 active+clean; 257 MiB data, 976 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:58:20.535 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:58:20 smithi186 ceph-mon[86379]: pgmap v1761: 65 pgs: 65 active+clean; 257 MiB data, 976 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:58:22.356 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:58:22 smithi018 ceph-mon[112590]: pgmap v1762: 65 pgs: 65 active+clean; 257 MiB data, 976 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:58:22.534 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:58:22 smithi186 ceph-mon[86379]: pgmap v1762: 65 pgs: 65 active+clean; 257 MiB data, 976 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:58:24.355 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:58:24 smithi018 ceph-mon[112590]: pgmap v1763: 65 pgs: 65 active+clean; 257 MiB data, 976 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:58:24.534 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:58:24 smithi186 ceph-mon[86379]: pgmap v1763: 65 pgs: 65 active+clean; 257 MiB data, 976 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:58:25.386 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:58:25 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T21:58:25.534 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:58:25 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T21:58:26.535 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:58:26 smithi186 ceph-mon[86379]: pgmap v1764: 65 pgs: 65 active+clean; 257 MiB data, 976 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:58:26.606 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:58:26 smithi018 ceph-mon[112590]: pgmap v1764: 65 pgs: 65 active+clean; 257 MiB data, 976 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:58:28.534 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:58:28 smithi186 ceph-mon[86379]: pgmap v1765: 65 pgs: 65 active+clean; 257 MiB data, 976 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:58:28.606 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:58:28 smithi018 ceph-mon[112590]: pgmap v1765: 65 pgs: 65 active+clean; 257 MiB data, 976 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:58:30.535 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:58:30 smithi186 ceph-mon[86379]: pgmap v1766: 65 pgs: 65 active+clean; 257 MiB data, 976 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:58:30.605 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:58:30 smithi018 ceph-mon[112590]: pgmap v1766: 65 pgs: 65 active+clean; 257 MiB data, 976 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:58:32.535 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:58:32 smithi186 ceph-mon[86379]: pgmap v1767: 65 pgs: 65 active+clean; 257 MiB data, 976 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:58:32.605 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:58:32 smithi018 ceph-mon[112590]: pgmap v1767: 65 pgs: 65 active+clean; 257 MiB data, 976 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:58:34.534 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:58:34 smithi186 ceph-mon[86379]: pgmap v1768: 65 pgs: 65 active+clean; 257 MiB data, 976 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:58:34.605 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:58:34 smithi018 ceph-mon[112590]: pgmap v1768: 65 pgs: 65 active+clean; 257 MiB data, 976 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:58:36.537 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:58:36 smithi186 ceph-mon[86379]: pgmap v1769: 65 pgs: 65 active+clean; 257 MiB data, 976 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:58:36.605 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:58:36 smithi018 ceph-mon[112590]: pgmap v1769: 65 pgs: 65 active+clean; 257 MiB data, 976 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:58:38.534 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:58:38 smithi186 ceph-mon[86379]: pgmap v1770: 65 pgs: 65 active+clean; 257 MiB data, 976 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:58:38.605 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:58:38 smithi018 ceph-mon[112590]: pgmap v1770: 65 pgs: 65 active+clean; 257 MiB data, 976 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:58:40.534 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:58:40 smithi186 ceph-mon[86379]: pgmap v1771: 65 pgs: 65 active+clean; 257 MiB data, 976 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:58:40.535 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:58:40 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T21:58:40.605 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:58:40 smithi018 ceph-mon[112590]: pgmap v1771: 65 pgs: 65 active+clean; 257 MiB data, 976 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:58:40.606 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:58:40 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T21:58:42.534 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:58:42 smithi186 ceph-mon[86379]: pgmap v1772: 65 pgs: 65 active+clean; 257 MiB data, 976 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:58:42.605 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:58:42 smithi018 ceph-mon[112590]: pgmap v1772: 65 pgs: 65 active+clean; 257 MiB data, 976 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:58:44.534 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:58:44 smithi186 ceph-mon[86379]: pgmap v1773: 65 pgs: 65 active+clean; 257 MiB data, 976 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:58:44.605 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:58:44 smithi018 ceph-mon[112590]: pgmap v1773: 65 pgs: 65 active+clean; 257 MiB data, 976 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:58:46.534 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:58:46 smithi186 ceph-mon[86379]: pgmap v1774: 65 pgs: 65 active+clean; 257 MiB data, 976 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:58:46.606 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:58:46 smithi018 ceph-mon[112590]: pgmap v1774: 65 pgs: 65 active+clean; 257 MiB data, 976 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:58:48.534 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:58:48 smithi186 ceph-mon[86379]: pgmap v1775: 65 pgs: 65 active+clean; 257 MiB data, 976 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:58:48.605 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:58:48 smithi018 ceph-mon[112590]: pgmap v1775: 65 pgs: 65 active+clean; 257 MiB data, 976 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:58:50.534 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:58:50 smithi186 ceph-mon[86379]: pgmap v1776: 65 pgs: 65 active+clean; 257 MiB data, 976 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:58:50.535 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:58:50 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-12T21:58:50.605 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:58:50 smithi018 ceph-mon[112590]: pgmap v1776: 65 pgs: 65 active+clean; 257 MiB data, 976 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:58:50.606 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:58:50 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-12T21:58:51.535 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:58:51 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-12T21:58:51.535 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:58:51 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-12T21:58:51.535 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:58:51 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' 2024-04-12T21:58:51.606 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:58:51 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-12T21:58:51.606 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:58:51 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-12T21:58:51.606 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:58:51 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' 2024-04-12T21:58:52.535 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:58:52 smithi186 ceph-mon[86379]: pgmap v1777: 65 pgs: 65 active+clean; 257 MiB data, 976 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:58:52.606 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:58:52 smithi018 ceph-mon[112590]: pgmap v1777: 65 pgs: 65 active+clean; 257 MiB data, 976 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:58:54.534 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:58:54 smithi186 ceph-mon[86379]: pgmap v1778: 65 pgs: 65 active+clean; 257 MiB data, 976 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:58:54.606 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:58:54 smithi018 ceph-mon[112590]: pgmap v1778: 65 pgs: 65 active+clean; 257 MiB data, 976 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:58:55.534 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:58:55 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T21:58:55.605 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:58:55 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T21:58:56.534 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:58:56 smithi186 ceph-mon[86379]: pgmap v1779: 65 pgs: 65 active+clean; 257 MiB data, 976 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:58:56.606 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:58:56 smithi018 ceph-mon[112590]: pgmap v1779: 65 pgs: 65 active+clean; 257 MiB data, 976 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:58:58.534 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:58:58 smithi186 ceph-mon[86379]: pgmap v1780: 65 pgs: 65 active+clean; 257 MiB data, 976 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:58:58.605 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:58:58 smithi018 ceph-mon[112590]: pgmap v1780: 65 pgs: 65 active+clean; 257 MiB data, 976 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:58:59.606 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:58:59 smithi018 ceph-mon[112590]: pgmap v1781: 65 pgs: 65 active+clean; 257 MiB data, 976 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:58:59.785 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:58:59 smithi186 ceph-mon[86379]: pgmap v1781: 65 pgs: 65 active+clean; 257 MiB data, 976 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:59:01.587 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:59:01 smithi186 ceph-mon[86379]: pgmap v1782: 65 pgs: 65 active+clean; 257 MiB data, 976 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:59:01.605 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:59:01 smithi018 ceph-mon[112590]: pgmap v1782: 65 pgs: 65 active+clean; 257 MiB data, 976 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:59:03.606 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:59:03 smithi018 ceph-mon[112590]: pgmap v1783: 65 pgs: 65 active+clean; 257 MiB data, 976 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:59:03.785 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:59:03 smithi186 ceph-mon[86379]: pgmap v1783: 65 pgs: 65 active+clean; 257 MiB data, 976 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:59:05.605 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:59:05 smithi018 ceph-mon[112590]: pgmap v1784: 65 pgs: 65 active+clean; 257 MiB data, 976 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:59:05.785 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:59:05 smithi186 ceph-mon[86379]: pgmap v1784: 65 pgs: 65 active+clean; 257 MiB data, 976 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:59:08.105 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:59:07 smithi018 ceph-mon[112590]: pgmap v1785: 65 pgs: 65 active+clean; 257 MiB data, 976 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:59:08.285 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:59:07 smithi186 ceph-mon[86379]: pgmap v1785: 65 pgs: 65 active+clean; 257 MiB data, 976 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:59:10.105 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:59:09 smithi018 ceph-mon[112590]: pgmap v1786: 65 pgs: 65 active+clean; 257 MiB data, 976 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:59:10.284 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:59:09 smithi186 ceph-mon[86379]: pgmap v1786: 65 pgs: 65 active+clean; 257 MiB data, 976 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:59:11.105 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:59:10 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T21:59:11.284 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:59:10 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T21:59:12.105 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:59:11 smithi018 ceph-mon[112590]: pgmap v1787: 65 pgs: 65 active+clean; 257 MiB data, 976 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:59:12.284 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:59:11 smithi186 ceph-mon[86379]: pgmap v1787: 65 pgs: 65 active+clean; 257 MiB data, 976 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:59:14.105 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:59:13 smithi018 ceph-mon[112590]: pgmap v1788: 65 pgs: 65 active+clean; 257 MiB data, 976 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:59:14.284 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:59:13 smithi186 ceph-mon[86379]: pgmap v1788: 65 pgs: 65 active+clean; 257 MiB data, 976 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:59:16.105 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:59:15 smithi018 ceph-mon[112590]: pgmap v1789: 65 pgs: 65 active+clean; 257 MiB data, 976 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:59:16.284 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:59:15 smithi186 ceph-mon[86379]: pgmap v1789: 65 pgs: 65 active+clean; 257 MiB data, 976 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:59:18.105 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:59:17 smithi018 ceph-mon[112590]: pgmap v1790: 65 pgs: 65 active+clean; 257 MiB data, 976 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:59:18.284 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:59:17 smithi186 ceph-mon[86379]: pgmap v1790: 65 pgs: 65 active+clean; 257 MiB data, 976 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:59:20.285 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:59:19 smithi186 ceph-mon[86379]: pgmap v1791: 65 pgs: 65 active+clean; 257 MiB data, 976 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:59:20.355 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:59:19 smithi018 ceph-mon[112590]: pgmap v1791: 65 pgs: 65 active+clean; 257 MiB data, 976 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:59:22.284 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:59:21 smithi186 ceph-mon[86379]: pgmap v1792: 65 pgs: 65 active+clean; 257 MiB data, 976 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:59:22.355 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:59:21 smithi018 ceph-mon[112590]: pgmap v1792: 65 pgs: 65 active+clean; 257 MiB data, 976 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:59:24.284 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:59:23 smithi186 ceph-mon[86379]: pgmap v1793: 65 pgs: 65 active+clean; 257 MiB data, 976 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:59:24.355 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:59:23 smithi018 ceph-mon[112590]: pgmap v1793: 65 pgs: 65 active+clean; 257 MiB data, 976 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:59:26.284 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:59:25 smithi186 ceph-mon[86379]: pgmap v1794: 65 pgs: 65 active+clean; 257 MiB data, 976 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:59:26.285 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:59:25 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T21:59:26.355 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:59:25 smithi018 ceph-mon[112590]: pgmap v1794: 65 pgs: 65 active+clean; 257 MiB data, 976 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:59:26.356 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:59:25 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T21:59:28.284 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:59:27 smithi186 ceph-mon[86379]: pgmap v1795: 65 pgs: 65 active+clean; 257 MiB data, 976 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:59:28.355 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:59:27 smithi018 ceph-mon[112590]: pgmap v1795: 65 pgs: 65 active+clean; 257 MiB data, 976 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:59:30.284 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:59:29 smithi186 ceph-mon[86379]: pgmap v1796: 65 pgs: 65 active+clean; 257 MiB data, 976 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:59:30.355 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:59:29 smithi018 ceph-mon[112590]: pgmap v1796: 65 pgs: 65 active+clean; 257 MiB data, 976 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:59:32.284 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:59:31 smithi186 ceph-mon[86379]: pgmap v1797: 65 pgs: 65 active+clean; 257 MiB data, 976 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:59:32.355 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:59:31 smithi018 ceph-mon[112590]: pgmap v1797: 65 pgs: 65 active+clean; 257 MiB data, 976 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:59:34.284 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:59:33 smithi186 ceph-mon[86379]: pgmap v1798: 65 pgs: 65 active+clean; 257 MiB data, 976 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:59:34.355 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:59:33 smithi018 ceph-mon[112590]: pgmap v1798: 65 pgs: 65 active+clean; 257 MiB data, 976 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:59:36.284 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:59:35 smithi186 ceph-mon[86379]: pgmap v1799: 65 pgs: 65 active+clean; 257 MiB data, 976 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:59:36.355 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:59:35 smithi018 ceph-mon[112590]: pgmap v1799: 65 pgs: 65 active+clean; 257 MiB data, 976 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:59:38.284 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:59:37 smithi186 ceph-mon[86379]: pgmap v1800: 65 pgs: 65 active+clean; 257 MiB data, 976 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:59:38.355 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:59:37 smithi018 ceph-mon[112590]: pgmap v1800: 65 pgs: 65 active+clean; 257 MiB data, 976 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:59:40.284 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:59:39 smithi186 ceph-mon[86379]: pgmap v1801: 65 pgs: 65 active+clean; 257 MiB data, 976 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:59:40.355 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:59:39 smithi018 ceph-mon[112590]: pgmap v1801: 65 pgs: 65 active+clean; 257 MiB data, 976 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:59:41.284 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:59:40 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T21:59:41.355 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:59:40 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T21:59:42.284 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:59:41 smithi186 ceph-mon[86379]: pgmap v1802: 65 pgs: 65 active+clean; 257 MiB data, 976 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:59:42.355 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:59:41 smithi018 ceph-mon[112590]: pgmap v1802: 65 pgs: 65 active+clean; 257 MiB data, 976 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:59:44.284 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:59:43 smithi186 ceph-mon[86379]: pgmap v1803: 65 pgs: 65 active+clean; 257 MiB data, 976 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:59:44.355 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:59:43 smithi018 ceph-mon[112590]: pgmap v1803: 65 pgs: 65 active+clean; 257 MiB data, 976 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:59:46.284 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:59:45 smithi186 ceph-mon[86379]: pgmap v1804: 65 pgs: 65 active+clean; 257 MiB data, 976 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:59:46.355 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:59:45 smithi018 ceph-mon[112590]: pgmap v1804: 65 pgs: 65 active+clean; 257 MiB data, 976 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:59:48.284 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:59:47 smithi186 ceph-mon[86379]: pgmap v1805: 65 pgs: 65 active+clean; 257 MiB data, 976 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:59:48.355 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:59:47 smithi018 ceph-mon[112590]: pgmap v1805: 65 pgs: 65 active+clean; 257 MiB data, 976 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:59:50.284 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:59:49 smithi186 ceph-mon[86379]: pgmap v1806: 65 pgs: 65 active+clean; 257 MiB data, 976 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:59:50.355 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:59:49 smithi018 ceph-mon[112590]: pgmap v1806: 65 pgs: 65 active+clean; 257 MiB data, 976 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:59:52.284 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:59:51 smithi186 ceph-mon[86379]: pgmap v1807: 65 pgs: 65 active+clean; 257 MiB data, 976 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:59:52.284 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:59:51 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-12T21:59:52.355 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:59:51 smithi018 ceph-mon[112590]: pgmap v1807: 65 pgs: 65 active+clean; 257 MiB data, 976 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:59:52.355 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:59:51 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-12T21:59:53.284 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:59:52 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-12T21:59:53.284 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:59:52 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-12T21:59:53.285 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:59:52 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' 2024-04-12T21:59:53.355 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:59:52 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-12T21:59:53.356 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:59:52 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-12T21:59:53.356 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:59:52 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' 2024-04-12T21:59:54.284 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:59:53 smithi186 ceph-mon[86379]: pgmap v1808: 65 pgs: 65 active+clean; 257 MiB data, 976 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:59:54.355 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:59:53 smithi018 ceph-mon[112590]: pgmap v1808: 65 pgs: 65 active+clean; 257 MiB data, 976 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:59:55.284 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:59:54 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T21:59:55.355 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:59:54 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T21:59:56.284 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:59:55 smithi186 ceph-mon[86379]: pgmap v1809: 65 pgs: 65 active+clean; 257 MiB data, 976 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:59:56.355 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:59:55 smithi018 ceph-mon[112590]: pgmap v1809: 65 pgs: 65 active+clean; 257 MiB data, 976 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:59:58.284 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 21:59:58 smithi186 ceph-mon[86379]: pgmap v1810: 65 pgs: 65 active+clean; 257 MiB data, 976 MiB used, 535 GiB / 536 GiB avail 2024-04-12T21:59:58.355 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 21:59:58 smithi018 ceph-mon[112590]: pgmap v1810: 65 pgs: 65 active+clean; 257 MiB data, 976 MiB used, 535 GiB / 536 GiB avail 2024-04-12T22:00:00.284 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 22:00:00 smithi186 ceph-mon[86379]: pgmap v1811: 65 pgs: 65 active+clean; 257 MiB data, 976 MiB used, 535 GiB / 536 GiB avail 2024-04-12T22:00:00.284 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 22:00:00 smithi186 ceph-mon[86379]: overall HEALTH_OK 2024-04-12T22:00:00.355 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 22:00:00 smithi018 ceph-mon[112590]: pgmap v1811: 65 pgs: 65 active+clean; 257 MiB data, 976 MiB used, 535 GiB / 536 GiB avail 2024-04-12T22:00:00.356 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 22:00:00 smithi018 ceph-mon[112590]: overall HEALTH_OK 2024-04-12T22:00:02.284 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 22:00:02 smithi186 ceph-mon[86379]: pgmap v1812: 65 pgs: 65 active+clean; 257 MiB data, 976 MiB used, 535 GiB / 536 GiB avail 2024-04-12T22:00:02.355 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 22:00:02 smithi018 ceph-mon[112590]: pgmap v1812: 65 pgs: 65 active+clean; 257 MiB data, 976 MiB used, 535 GiB / 536 GiB avail 2024-04-12T22:00:04.284 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 22:00:04 smithi186 ceph-mon[86379]: pgmap v1813: 65 pgs: 65 active+clean; 257 MiB data, 976 MiB used, 535 GiB / 536 GiB avail 2024-04-12T22:00:04.355 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 22:00:04 smithi018 ceph-mon[112590]: pgmap v1813: 65 pgs: 65 active+clean; 257 MiB data, 976 MiB used, 535 GiB / 536 GiB avail 2024-04-12T22:00:06.284 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 22:00:06 smithi186 ceph-mon[86379]: pgmap v1814: 65 pgs: 65 active+clean; 257 MiB data, 976 MiB used, 535 GiB / 536 GiB avail 2024-04-12T22:00:06.355 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 22:00:06 smithi018 ceph-mon[112590]: pgmap v1814: 65 pgs: 65 active+clean; 257 MiB data, 976 MiB used, 535 GiB / 536 GiB avail 2024-04-12T22:00:08.355 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 22:00:08 smithi018 ceph-mon[112590]: pgmap v1815: 65 pgs: 65 active+clean; 257 MiB data, 976 MiB used, 535 GiB / 536 GiB avail 2024-04-12T22:00:08.534 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 22:00:08 smithi186 ceph-mon[86379]: pgmap v1815: 65 pgs: 65 active+clean; 257 MiB data, 976 MiB used, 535 GiB / 536 GiB avail 2024-04-12T22:00:10.356 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 22:00:10 smithi018 ceph-mon[112590]: pgmap v1816: 65 pgs: 65 active+clean; 257 MiB data, 976 MiB used, 535 GiB / 536 GiB avail 2024-04-12T22:00:10.356 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 22:00:10 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T22:00:10.534 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 22:00:10 smithi186 ceph-mon[86379]: pgmap v1816: 65 pgs: 65 active+clean; 257 MiB data, 976 MiB used, 535 GiB / 536 GiB avail 2024-04-12T22:00:10.534 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 22:00:10 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T22:00:12.357 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 22:00:12 smithi018 ceph-mon[112590]: pgmap v1817: 65 pgs: 65 active+clean; 257 MiB data, 976 MiB used, 535 GiB / 536 GiB avail 2024-04-12T22:00:12.535 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 22:00:12 smithi186 ceph-mon[86379]: pgmap v1817: 65 pgs: 65 active+clean; 257 MiB data, 976 MiB used, 535 GiB / 536 GiB avail 2024-04-12T22:00:14.355 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 22:00:14 smithi018 ceph-mon[112590]: pgmap v1818: 65 pgs: 65 active+clean; 257 MiB data, 976 MiB used, 535 GiB / 536 GiB avail 2024-04-12T22:00:14.534 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 22:00:14 smithi186 ceph-mon[86379]: pgmap v1818: 65 pgs: 65 active+clean; 257 MiB data, 976 MiB used, 535 GiB / 536 GiB avail 2024-04-12T22:00:16.355 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 22:00:16 smithi018 ceph-mon[112590]: pgmap v1819: 65 pgs: 65 active+clean; 257 MiB data, 976 MiB used, 535 GiB / 536 GiB avail 2024-04-12T22:00:16.534 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 22:00:16 smithi186 ceph-mon[86379]: pgmap v1819: 65 pgs: 65 active+clean; 257 MiB data, 976 MiB used, 535 GiB / 536 GiB avail 2024-04-12T22:00:18.355 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 22:00:18 smithi018 ceph-mon[112590]: pgmap v1820: 65 pgs: 65 active+clean; 257 MiB data, 976 MiB used, 535 GiB / 536 GiB avail 2024-04-12T22:00:18.534 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 22:00:18 smithi186 ceph-mon[86379]: pgmap v1820: 65 pgs: 65 active+clean; 257 MiB data, 976 MiB used, 535 GiB / 536 GiB avail 2024-04-12T22:00:20.355 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 22:00:20 smithi018 ceph-mon[112590]: pgmap v1821: 65 pgs: 65 active+clean; 257 MiB data, 976 MiB used, 535 GiB / 536 GiB avail 2024-04-12T22:00:20.534 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 22:00:20 smithi186 ceph-mon[86379]: pgmap v1821: 65 pgs: 65 active+clean; 257 MiB data, 976 MiB used, 535 GiB / 536 GiB avail 2024-04-12T22:00:22.355 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 22:00:22 smithi018 ceph-mon[112590]: pgmap v1822: 65 pgs: 65 active+clean; 257 MiB data, 976 MiB used, 535 GiB / 536 GiB avail 2024-04-12T22:00:22.534 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 22:00:22 smithi186 ceph-mon[86379]: pgmap v1822: 65 pgs: 65 active+clean; 257 MiB data, 976 MiB used, 535 GiB / 536 GiB avail 2024-04-12T22:00:24.355 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 22:00:24 smithi018 ceph-mon[112590]: pgmap v1823: 65 pgs: 65 active+clean; 257 MiB data, 976 MiB used, 535 GiB / 536 GiB avail 2024-04-12T22:00:24.534 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 22:00:24 smithi186 ceph-mon[86379]: pgmap v1823: 65 pgs: 65 active+clean; 257 MiB data, 976 MiB used, 535 GiB / 536 GiB avail 2024-04-12T22:00:25.355 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 22:00:25 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T22:00:25.534 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 22:00:25 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T22:00:26.355 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 22:00:26 smithi018 ceph-mon[112590]: pgmap v1824: 65 pgs: 65 active+clean; 257 MiB data, 976 MiB used, 535 GiB / 536 GiB avail 2024-04-12T22:00:26.534 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 22:00:26 smithi186 ceph-mon[86379]: pgmap v1824: 65 pgs: 65 active+clean; 257 MiB data, 976 MiB used, 535 GiB / 536 GiB avail 2024-04-12T22:00:28.355 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 22:00:28 smithi018 ceph-mon[112590]: pgmap v1825: 65 pgs: 65 active+clean; 257 MiB data, 976 MiB used, 535 GiB / 536 GiB avail 2024-04-12T22:00:28.534 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 22:00:28 smithi186 ceph-mon[86379]: pgmap v1825: 65 pgs: 65 active+clean; 257 MiB data, 976 MiB used, 535 GiB / 536 GiB avail 2024-04-12T22:00:30.534 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 22:00:30 smithi186 ceph-mon[86379]: pgmap v1826: 65 pgs: 65 active+clean; 257 MiB data, 976 MiB used, 535 GiB / 536 GiB avail 2024-04-12T22:00:30.605 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 22:00:30 smithi018 ceph-mon[112590]: pgmap v1826: 65 pgs: 65 active+clean; 257 MiB data, 976 MiB used, 535 GiB / 536 GiB avail 2024-04-12T22:00:32.534 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 22:00:32 smithi186 ceph-mon[86379]: pgmap v1827: 65 pgs: 65 active+clean; 257 MiB data, 976 MiB used, 535 GiB / 536 GiB avail 2024-04-12T22:00:32.605 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 22:00:32 smithi018 ceph-mon[112590]: pgmap v1827: 65 pgs: 65 active+clean; 257 MiB data, 976 MiB used, 535 GiB / 536 GiB avail 2024-04-12T22:00:34.534 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 22:00:34 smithi186 ceph-mon[86379]: pgmap v1828: 65 pgs: 65 active+clean; 257 MiB data, 976 MiB used, 535 GiB / 536 GiB avail 2024-04-12T22:00:34.605 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 22:00:34 smithi018 ceph-mon[112590]: pgmap v1828: 65 pgs: 65 active+clean; 257 MiB data, 976 MiB used, 535 GiB / 536 GiB avail 2024-04-12T22:00:36.534 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 22:00:36 smithi186 ceph-mon[86379]: pgmap v1829: 65 pgs: 65 active+clean; 257 MiB data, 976 MiB used, 535 GiB / 536 GiB avail 2024-04-12T22:00:36.605 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 22:00:36 smithi018 ceph-mon[112590]: pgmap v1829: 65 pgs: 65 active+clean; 257 MiB data, 976 MiB used, 535 GiB / 536 GiB avail 2024-04-12T22:00:38.534 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 22:00:38 smithi186 ceph-mon[86379]: pgmap v1830: 65 pgs: 65 active+clean; 257 MiB data, 976 MiB used, 535 GiB / 536 GiB avail 2024-04-12T22:00:38.605 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 22:00:38 smithi018 ceph-mon[112590]: pgmap v1830: 65 pgs: 65 active+clean; 257 MiB data, 976 MiB used, 535 GiB / 536 GiB avail 2024-04-12T22:00:40.534 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 22:00:40 smithi186 ceph-mon[86379]: pgmap v1831: 65 pgs: 65 active+clean; 257 MiB data, 976 MiB used, 535 GiB / 536 GiB avail 2024-04-12T22:00:40.535 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 22:00:40 smithi186 ceph-mon[86379]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T22:00:40.605 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 22:00:40 smithi018 ceph-mon[112590]: pgmap v1831: 65 pgs: 65 active+clean; 257 MiB data, 976 MiB used, 535 GiB / 536 GiB avail 2024-04-12T22:00:40.605 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 22:00:40 smithi018 ceph-mon[112590]: from='mgr.34102 172.21.15.18:0/1880859214' entity='mgr.smithi018.xjzwga' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T22:00:42.534 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 22:00:42 smithi186 ceph-mon[86379]: pgmap v1832: 65 pgs: 65 active+clean; 257 MiB data, 976 MiB used, 535 GiB / 536 GiB avail 2024-04-12T22:00:42.605 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 22:00:42 smithi018 ceph-mon[112590]: pgmap v1832: 65 pgs: 65 active+clean; 257 MiB data, 976 MiB used, 535 GiB / 536 GiB avail 2024-04-12T22:00:44.534 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 22:00:44 smithi186 ceph-mon[86379]: pgmap v1833: 65 pgs: 65 active+clean; 257 MiB data, 976 MiB used, 535 GiB / 536 GiB avail 2024-04-12T22:00:44.605 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 22:00:44 smithi018 ceph-mon[112590]: pgmap v1833: 65 pgs: 65 active+clean; 257 MiB data, 976 MiB used, 535 GiB / 536 GiB avail 2024-04-12T22:00:46.534 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 22:00:46 smithi186 ceph-mon[86379]: pgmap v1834: 65 pgs: 65 active+clean; 257 MiB data, 976 MiB used, 535 GiB / 536 GiB avail 2024-04-12T22:00:46.605 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 22:00:46 smithi018 ceph-mon[112590]: pgmap v1834: 65 pgs: 65 active+clean; 257 MiB data, 976 MiB used, 535 GiB / 536 GiB avail 2024-04-12T22:00:48.534 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 22:00:48 smithi186 ceph-mon[86379]: pgmap v1835: 65 pgs: 65 active+clean; 257 MiB data, 976 MiB used, 535 GiB / 536 GiB avail 2024-04-12T22:00:48.605 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 22:00:48 smithi018 ceph-mon[112590]: pgmap v1835: 65 pgs: 65 active+clean; 257 MiB data, 976 MiB used, 535 GiB / 536 GiB avail 2024-04-12T22:00:48.986 ERROR:tasks.cephfs.fuse_mount:process failed to terminate after unmount. This probably indicates a bug within ceph-fuse. 2024-04-12T22:00:48.986 ERROR:teuthology.run_tasks:Manager failed: ceph-fuse Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_6c637841c215537a4502385240412f1966e0faab/teuthology/run_tasks.py", line 154, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ next(self.gen) File "/home/teuthworker/src/git.ceph.com_ceph-c_324baf8ddb91dea46cac50cb535c306115791355/qa/tasks/ceph_fuse.py", line 183, in task mount.umount_wait() File "/home/teuthworker/src/git.ceph.com_ceph-c_324baf8ddb91dea46cac50cb535c306115791355/qa/tasks/cephfs/fuse_mount.py", line 403, in umount_wait run.wait([self.fuse_daemon], timeout) File "/home/teuthworker/src/git.ceph.com_teuthology_6c637841c215537a4502385240412f1966e0faab/teuthology/orchestra/run.py", line 473, in wait check_time() File "/home/teuthworker/src/git.ceph.com_teuthology_6c637841c215537a4502385240412f1966e0faab/teuthology/contextutil.py", line 134, in __call__ raise MaxWhileTries(error_msg) teuthology.exceptions.MaxWhileTries: reached maximum tries (51) after waiting for 300 seconds 2024-04-12T22:00:48.988 DEBUG:teuthology.run_tasks:Unwinding manager cephadm 2024-04-12T22:00:49.000 INFO:tasks.cephadm:Teardown begin 2024-04-12T22:00:49.000 ERROR:teuthology.contextutil:Saw exception from nested tasks Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_6c637841c215537a4502385240412f1966e0faab/teuthology/contextutil.py", line 32, in nested yield vars File "/home/teuthworker/src/git.ceph.com_ceph-c_324baf8ddb91dea46cac50cb535c306115791355/qa/tasks/cephadm.py", line 2230, in task yield File "/home/teuthworker/src/git.ceph.com_teuthology_6c637841c215537a4502385240412f1966e0faab/teuthology/run_tasks.py", line 154, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ next(self.gen) File "/home/teuthworker/src/git.ceph.com_ceph-c_324baf8ddb91dea46cac50cb535c306115791355/qa/tasks/ceph_fuse.py", line 183, in task mount.umount_wait() File "/home/teuthworker/src/git.ceph.com_ceph-c_324baf8ddb91dea46cac50cb535c306115791355/qa/tasks/cephfs/fuse_mount.py", line 403, in umount_wait run.wait([self.fuse_daemon], timeout) File "/home/teuthworker/src/git.ceph.com_teuthology_6c637841c215537a4502385240412f1966e0faab/teuthology/orchestra/run.py", line 473, in wait check_time() File "/home/teuthworker/src/git.ceph.com_teuthology_6c637841c215537a4502385240412f1966e0faab/teuthology/contextutil.py", line 134, in __call__ raise MaxWhileTries(error_msg) teuthology.exceptions.MaxWhileTries: reached maximum tries (51) after waiting for 300 seconds 2024-04-12T22:00:49.001 DEBUG:teuthology.orchestra.run.smithi018:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-12T22:00:49.118 DEBUG:teuthology.orchestra.run.smithi186:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-12T22:00:49.221 INFO:tasks.cephadm:Cleaning up testdir ceph.* files... 2024-04-12T22:00:49.222 DEBUG:teuthology.orchestra.run.smithi018:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-04-12T22:00:49.283 DEBUG:teuthology.orchestra.run.smithi186:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-04-12T22:00:49.339 INFO:tasks.cephadm:Stopping all daemons... 2024-04-12T22:00:49.339 INFO:tasks.cephadm.mon.smithi018:Stopping mon.smithi018... 2024-04-12T22:00:49.339 DEBUG:teuthology.orchestra.run.smithi018:> sudo systemctl stop ceph-1cbe407a-f90d-11ee-bc8d-c7b262605968@mon.smithi018 2024-04-12T22:00:49.855 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 22:00:49 smithi018 systemd[1]: Stopping Ceph mon.smithi018 for 1cbe407a-f90d-11ee-bc8d-c7b262605968... 2024-04-12T22:00:50.267 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 22:00:49 smithi018 ceph-1cbe407a-f90d-11ee-bc8d-c7b262605968-mon-smithi018[112586]: 2024-04-12T22:00:49.986+0000 7fd45e200700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.smithi018 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-04-12T22:00:50.267 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 22:00:49 smithi018 ceph-1cbe407a-f90d-11ee-bc8d-c7b262605968-mon-smithi018[112586]: 2024-04-12T22:00:49.986+0000 7fd45e200700 -1 mon.smithi018@0(leader) e3 *** Got Signal Terminated *** 2024-04-12T22:00:50.572 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 22:00:50 smithi018 podman[175978]: 2024-04-12 22:00:50.265926685 +0000 UTC m=+0.413713193 container died df3e201dfed6be276236a5d4fccee37b4841b35ee7306e2eb01c136fde328dce (image=quay.ceph.io/ceph-ci/ceph@sha256:f4452def3be4d350f4a8e5feaf7ebb1511be9298fe68478ba7a2d6fbd7d69a63, name=ceph-1cbe407a-f90d-11ee-bc8d-c7b262605968-mon-smithi018, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-vshankar-testing-20240411.061452-324baf8, GIT_CLEAN=True, org.label-schema.build-date=20240409, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=) 2024-04-12T22:00:51.167 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 22:00:50 smithi018 bash[175978]: ceph-1cbe407a-f90d-11ee-bc8d-c7b262605968-mon-smithi018 2024-04-12T22:00:51.167 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 22:00:50 smithi018 podman[175978]: 2024-04-12 22:00:50.855621671 +0000 UTC m=+1.003408175 container cleanup df3e201dfed6be276236a5d4fccee37b4841b35ee7306e2eb01c136fde328dce (image=quay.ceph.io/ceph-ci/ceph@sha256:f4452def3be4d350f4a8e5feaf7ebb1511be9298fe68478ba7a2d6fbd7d69a63, name=ceph-1cbe407a-f90d-11ee-bc8d-c7b262605968-mon-smithi018, org.label-schema.build-date=20240409, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, ceph=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , RELEASE=wip-vshankar-testing-20240411.061452-324baf8) 2024-04-12T22:00:51.428 INFO:journalctl@ceph.mon.smithi018.smithi018.stdout:Apr 12 22:00:51 smithi018 podman[176003]: 2024-04-12 22:00:51.165566155 +0000 UTC m=+0.882970066 container remove df3e201dfed6be276236a5d4fccee37b4841b35ee7306e2eb01c136fde328dce (image=quay.ceph.io/ceph-ci/ceph@sha256:f4452def3be4d350f4a8e5feaf7ebb1511be9298fe68478ba7a2d6fbd7d69a63, name=ceph-1cbe407a-f90d-11ee-bc8d-c7b262605968-mon-smithi018, RELEASE=wip-vshankar-testing-20240411.061452-324baf8, org.label-schema.build-date=20240409, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, GIT_CLEAN=True, ceph=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, io.buildah.version=1.33.5) 2024-04-12T22:00:51.444 DEBUG:teuthology.orchestra.run.smithi018:> sudo pkill -f 'journalctl -f -n 0 -u ceph-1cbe407a-f90d-11ee-bc8d-c7b262605968@mon.smithi018.service' 2024-04-12T22:00:51.586 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-12T22:00:51.586 INFO:tasks.cephadm.mon.smithi018:Stopped mon.smithi018 2024-04-12T22:00:51.586 INFO:tasks.cephadm.mon.smithi186:Stopping mon.smithi186... 2024-04-12T22:00:51.586 DEBUG:teuthology.orchestra.run.smithi186:> sudo systemctl stop ceph-1cbe407a-f90d-11ee-bc8d-c7b262605968@mon.smithi186 2024-04-12T22:00:52.034 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 22:00:51 smithi186 systemd[1]: Stopping Ceph mon.smithi186 for 1cbe407a-f90d-11ee-bc8d-c7b262605968... 2024-04-12T22:00:52.535 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 22:00:52 smithi186 ceph-1cbe407a-f90d-11ee-bc8d-c7b262605968-mon-smithi186[86375]: 2024-04-12T22:00:52.155+0000 7f38e3800700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.smithi186 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-04-12T22:00:52.535 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 22:00:52 smithi186 ceph-1cbe407a-f90d-11ee-bc8d-c7b262605968-mon-smithi186[86375]: 2024-04-12T22:00:52.155+0000 7f38e3800700 -1 mon.smithi186@1(peon) e3 *** Got Signal Terminated *** 2024-04-12T22:00:52.535 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 22:00:52 smithi186 podman[123041]: 2024-04-12 22:00:52.404727565 +0000 UTC m=+0.389472821 container died c77498da9b2bf5febb1767de4918612d9c9398a028542311285e3b9dcdc4e7a6 (image=quay.ceph.io/ceph-ci/ceph@sha256:f4452def3be4d350f4a8e5feaf7ebb1511be9298fe68478ba7a2d6fbd7d69a63, name=ceph-1cbe407a-f90d-11ee-bc8d-c7b262605968-mon-smithi186, RELEASE=wip-vshankar-testing-20240411.061452-324baf8, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240409) 2024-04-12T22:00:52.968 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 22:00:52 smithi186 podman[123041]: 2024-04-12 22:00:52.933560488 +0000 UTC m=+0.918305741 container cleanup c77498da9b2bf5febb1767de4918612d9c9398a028542311285e3b9dcdc4e7a6 (image=quay.ceph.io/ceph-ci/ceph@sha256:f4452def3be4d350f4a8e5feaf7ebb1511be9298fe68478ba7a2d6fbd7d69a63, name=ceph-1cbe407a-f90d-11ee-bc8d-c7b262605968-mon-smithi186, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, org.label-schema.build-date=20240409, RELEASE=wip-vshankar-testing-20240411.061452-324baf8, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5) 2024-04-12T22:00:52.968 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 22:00:52 smithi186 bash[123041]: ceph-1cbe407a-f90d-11ee-bc8d-c7b262605968-mon-smithi186 2024-04-12T22:00:53.284 INFO:journalctl@ceph.mon.smithi186.smithi186.stdout:Apr 12 22:00:53 smithi186 podman[123055]: 2024-04-12 22:00:53.185021431 +0000 UTC m=+0.766998381 container remove c77498da9b2bf5febb1767de4918612d9c9398a028542311285e3b9dcdc4e7a6 (image=quay.ceph.io/ceph-ci/ceph@sha256:f4452def3be4d350f4a8e5feaf7ebb1511be9298fe68478ba7a2d6fbd7d69a63, name=ceph-1cbe407a-f90d-11ee-bc8d-c7b262605968-mon-smithi186, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, RELEASE=wip-vshankar-testing-20240411.061452-324baf8, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.build-date=20240409, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-12T22:00:53.700 DEBUG:teuthology.orchestra.run.smithi186:> sudo pkill -f 'journalctl -f -n 0 -u ceph-1cbe407a-f90d-11ee-bc8d-c7b262605968@mon.smithi186.service' 2024-04-12T22:00:53.836 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-12T22:00:53.836 INFO:tasks.cephadm.mon.smithi186:Stopped mon.smithi186 2024-04-12T22:00:53.836 INFO:tasks.cephadm.osd.0:Stopping osd.0... 2024-04-12T22:00:53.837 DEBUG:teuthology.orchestra.run.smithi018:> sudo systemctl stop ceph-1cbe407a-f90d-11ee-bc8d-c7b262605968@osd.0 2024-04-12T22:00:54.356 INFO:journalctl@ceph.osd.0.smithi018.stdout:Apr 12 22:00:53 smithi018 systemd[1]: Stopping Ceph osd.0 for 1cbe407a-f90d-11ee-bc8d-c7b262605968... 2024-04-12T22:00:54.856 INFO:journalctl@ceph.osd.0.smithi018.stdout:Apr 12 22:00:54 smithi018 ceph-1cbe407a-f90d-11ee-bc8d-c7b262605968-osd-0[124276]: 2024-04-12T22:00:54.433+0000 7f2f43000700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-12T22:00:54.856 INFO:journalctl@ceph.osd.0.smithi018.stdout:Apr 12 22:00:54 smithi018 ceph-1cbe407a-f90d-11ee-bc8d-c7b262605968-osd-0[124276]: 2024-04-12T22:00:54.433+0000 7f2f43000700 -1 osd.0 104 *** Got signal Terminated *** 2024-04-12T22:00:54.856 INFO:journalctl@ceph.osd.0.smithi018.stdout:Apr 12 22:00:54 smithi018 ceph-1cbe407a-f90d-11ee-bc8d-c7b262605968-osd-0[124276]: 2024-04-12T22:00:54.433+0000 7f2f43000700 -1 osd.0 104 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-12T22:01:00.356 INFO:journalctl@ceph.osd.0.smithi018.stdout:Apr 12 22:00:59 smithi018 podman[176111]: 2024-04-12 22:00:59.970003641 +0000 UTC m=+5.697641466 container died 60b1273ca0475debf2761f0f1c017a030f5b7d0923f1f790cdb5b2ac8dfe8e13 (image=quay.ceph.io/ceph-ci/ceph@sha256:f4452def3be4d350f4a8e5feaf7ebb1511be9298fe68478ba7a2d6fbd7d69a63, name=ceph-1cbe407a-f90d-11ee-bc8d-c7b262605968-osd-0, ceph=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, RELEASE=wip-vshankar-testing-20240411.061452-324baf8, org.label-schema.build-date=20240409, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-12T22:01:00.770 INFO:journalctl@ceph.osd.0.smithi018.stdout:Apr 12 22:01:00 smithi018 podman[176111]: 2024-04-12 22:01:00.501361643 +0000 UTC m=+6.228999461 container cleanup 60b1273ca0475debf2761f0f1c017a030f5b7d0923f1f790cdb5b2ac8dfe8e13 (image=quay.ceph.io/ceph-ci/ceph@sha256:f4452def3be4d350f4a8e5feaf7ebb1511be9298fe68478ba7a2d6fbd7d69a63, name=ceph-1cbe407a-f90d-11ee-bc8d-c7b262605968-osd-0, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240409, GIT_CLEAN=True, ceph=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, RELEASE=wip-vshankar-testing-20240411.061452-324baf8, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-12T22:01:00.771 INFO:journalctl@ceph.osd.0.smithi018.stdout:Apr 12 22:01:00 smithi018 bash[176111]: ceph-1cbe407a-f90d-11ee-bc8d-c7b262605968-osd-0 2024-04-12T22:01:01.106 INFO:journalctl@ceph.osd.0.smithi018.stdout:Apr 12 22:01:00 smithi018 podman[176131]: 2024-04-12 22:01:00.769653312 +0000 UTC m=+0.791646215 container remove 60b1273ca0475debf2761f0f1c017a030f5b7d0923f1f790cdb5b2ac8dfe8e13 (image=quay.ceph.io/ceph-ci/ceph@sha256:f4452def3be4d350f4a8e5feaf7ebb1511be9298fe68478ba7a2d6fbd7d69a63, name=ceph-1cbe407a-f90d-11ee-bc8d-c7b262605968-osd-0, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240409, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, RELEASE=wip-vshankar-testing-20240411.061452-324baf8, ceph=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2) 2024-04-12T22:01:01.856 INFO:journalctl@ceph.osd.0.smithi018.stdout:Apr 12 22:01:01 smithi018 podman[176229]: 2024-04-12 22:01:01.485047554 +0000 UTC m=+0.037324513 image pull 8717720dc5e5c56303818ea4d4f0a11f6e2c03dee521fb668f411f1304e0afca quay.ceph.io/ceph-ci/ceph@sha256:f4452def3be4d350f4a8e5feaf7ebb1511be9298fe68478ba7a2d6fbd7d69a63 2024-04-12T22:01:01.856 INFO:journalctl@ceph.osd.0.smithi018.stdout:Apr 12 22:01:01 smithi018 podman[176229]: 2024-04-12 22:01:01.603896271 +0000 UTC m=+0.156173261 container create 543bce4955ba2cad21a4a7a32cb0f90fa113707cc53056d48ea48c90483aed0c (image=quay.ceph.io/ceph-ci/ceph@sha256:f4452def3be4d350f4a8e5feaf7ebb1511be9298fe68478ba7a2d6fbd7d69a63, name=ceph-1cbe407a-f90d-11ee-bc8d-c7b262605968-osd-0-deactivate, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240409, ceph=True, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, RELEASE=wip-vshankar-testing-20240411.061452-324baf8, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True) 2024-04-12T22:01:02.356 INFO:journalctl@ceph.osd.0.smithi018.stdout:Apr 12 22:01:01 smithi018 podman[176229]: 2024-04-12 22:01:01.98293208 +0000 UTC m=+0.535209077 container init 543bce4955ba2cad21a4a7a32cb0f90fa113707cc53056d48ea48c90483aed0c (image=quay.ceph.io/ceph-ci/ceph@sha256:f4452def3be4d350f4a8e5feaf7ebb1511be9298fe68478ba7a2d6fbd7d69a63, name=ceph-1cbe407a-f90d-11ee-bc8d-c7b262605968-osd-0-deactivate, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240409, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, RELEASE=wip-vshankar-testing-20240411.061452-324baf8, ceph=True, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0) 2024-04-12T22:01:02.356 INFO:journalctl@ceph.osd.0.smithi018.stdout:Apr 12 22:01:01 smithi018 podman[176229]: 2024-04-12 22:01:01.996816172 +0000 UTC m=+0.549093142 container start 543bce4955ba2cad21a4a7a32cb0f90fa113707cc53056d48ea48c90483aed0c (image=quay.ceph.io/ceph-ci/ceph@sha256:f4452def3be4d350f4a8e5feaf7ebb1511be9298fe68478ba7a2d6fbd7d69a63, name=ceph-1cbe407a-f90d-11ee-bc8d-c7b262605968-osd-0-deactivate, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, RELEASE=wip-vshankar-testing-20240411.061452-324baf8, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240409, org.label-schema.license=GPLv2) 2024-04-12T22:01:02.356 INFO:journalctl@ceph.osd.0.smithi018.stdout:Apr 12 22:01:02 smithi018 podman[176229]: 2024-04-12 22:01:02.117445881 +0000 UTC m=+0.669722905 container attach 543bce4955ba2cad21a4a7a32cb0f90fa113707cc53056d48ea48c90483aed0c (image=quay.ceph.io/ceph-ci/ceph@sha256:f4452def3be4d350f4a8e5feaf7ebb1511be9298fe68478ba7a2d6fbd7d69a63, name=ceph-1cbe407a-f90d-11ee-bc8d-c7b262605968-osd-0-deactivate, GIT_CLEAN=True, io.buildah.version=1.33.5, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, RELEASE=wip-vshankar-testing-20240411.061452-324baf8, org.label-schema.build-date=20240409, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0) 2024-04-12T22:01:02.816 INFO:journalctl@ceph.osd.0.smithi018.stdout:Apr 12 22:01:02 smithi018 podman[176229]: 2024-04-12 22:01:02.45864205 +0000 UTC m=+1.010919083 container died 543bce4955ba2cad21a4a7a32cb0f90fa113707cc53056d48ea48c90483aed0c (image=quay.ceph.io/ceph-ci/ceph@sha256:f4452def3be4d350f4a8e5feaf7ebb1511be9298fe68478ba7a2d6fbd7d69a63, name=ceph-1cbe407a-f90d-11ee-bc8d-c7b262605968-osd-0-deactivate, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, RELEASE=wip-vshankar-testing-20240411.061452-324baf8, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.build-date=20240409, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True) 2024-04-12T22:01:03.271 DEBUG:teuthology.orchestra.run.smithi018:> sudo pkill -f 'journalctl -f -n 0 -u ceph-1cbe407a-f90d-11ee-bc8d-c7b262605968@osd.0.service' 2024-04-12T22:01:03.394 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-12T22:01:03.394 INFO:tasks.cephadm.osd.0:Stopped osd.0 2024-04-12T22:01:03.394 INFO:tasks.cephadm.osd.1:Stopping osd.1... 2024-04-12T22:01:03.394 DEBUG:teuthology.orchestra.run.smithi018:> sudo systemctl stop ceph-1cbe407a-f90d-11ee-bc8d-c7b262605968@osd.1 2024-04-12T22:01:03.855 INFO:journalctl@ceph.osd.1.smithi018.stdout:Apr 12 22:01:03 smithi018 systemd[1]: Stopping Ceph osd.1 for 1cbe407a-f90d-11ee-bc8d-c7b262605968... 2024-04-12T22:01:04.356 INFO:journalctl@ceph.osd.1.smithi018.stdout:Apr 12 22:01:04 smithi018 ceph-1cbe407a-f90d-11ee-bc8d-c7b262605968-osd-1[130044]: 2024-04-12T22:01:04.029+0000 7f5eb2e00700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-12T22:01:04.356 INFO:journalctl@ceph.osd.1.smithi018.stdout:Apr 12 22:01:04 smithi018 ceph-1cbe407a-f90d-11ee-bc8d-c7b262605968-osd-1[130044]: 2024-04-12T22:01:04.029+0000 7f5eb2e00700 -1 osd.1 104 *** Got signal Terminated *** 2024-04-12T22:01:04.356 INFO:journalctl@ceph.osd.1.smithi018.stdout:Apr 12 22:01:04 smithi018 ceph-1cbe407a-f90d-11ee-bc8d-c7b262605968-osd-1[130044]: 2024-04-12T22:01:04.029+0000 7f5eb2e00700 -1 osd.1 104 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-12T22:01:09.788 INFO:journalctl@ceph.osd.1.smithi018.stdout:Apr 12 22:01:09 smithi018 podman[176336]: 2024-04-12 22:01:09.532618497 +0000 UTC m=+5.641343209 container died 170ce1e5fb8dedd565d7c8fc77f3e8eb8db353188932b3fc1f307ddd4d64fac1 (image=quay.ceph.io/ceph-ci/ceph@sha256:f4452def3be4d350f4a8e5feaf7ebb1511be9298fe68478ba7a2d6fbd7d69a63, name=ceph-1cbe407a-f90d-11ee-bc8d-c7b262605968-osd-1, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_CLEAN=True, ceph=True, org.label-schema.build-date=20240409, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-vshankar-testing-20240411.061452-324baf8, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-12T22:01:10.106 INFO:journalctl@ceph.osd.1.smithi018.stdout:Apr 12 22:01:09 smithi018 podman[176336]: 2024-04-12 22:01:09.913967551 +0000 UTC m=+6.022692299 container cleanup 170ce1e5fb8dedd565d7c8fc77f3e8eb8db353188932b3fc1f307ddd4d64fac1 (image=quay.ceph.io/ceph-ci/ceph@sha256:f4452def3be4d350f4a8e5feaf7ebb1511be9298fe68478ba7a2d6fbd7d69a63, name=ceph-1cbe407a-f90d-11ee-bc8d-c7b262605968-osd-1, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, ceph=True, org.label-schema.license=GPLv2, RELEASE=wip-vshankar-testing-20240411.061452-324baf8, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.build-date=20240409, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, io.buildah.version=1.33.5) 2024-04-12T22:01:10.106 INFO:journalctl@ceph.osd.1.smithi018.stdout:Apr 12 22:01:09 smithi018 bash[176336]: ceph-1cbe407a-f90d-11ee-bc8d-c7b262605968-osd-1 2024-04-12T22:01:10.605 INFO:journalctl@ceph.osd.1.smithi018.stdout:Apr 12 22:01:10 smithi018 podman[176356]: 2024-04-12 22:01:10.140134902 +0000 UTC m=+0.592706446 container remove 170ce1e5fb8dedd565d7c8fc77f3e8eb8db353188932b3fc1f307ddd4d64fac1 (image=quay.ceph.io/ceph-ci/ceph@sha256:f4452def3be4d350f4a8e5feaf7ebb1511be9298fe68478ba7a2d6fbd7d69a63, name=ceph-1cbe407a-f90d-11ee-bc8d-c7b262605968-osd-1, RELEASE=wip-vshankar-testing-20240411.061452-324baf8, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240409, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, ceph=True, GIT_CLEAN=True) 2024-04-12T22:01:11.280 INFO:journalctl@ceph.osd.1.smithi018.stdout:Apr 12 22:01:10 smithi018 podman[176443]: 2024-04-12 22:01:10.889735023 +0000 UTC m=+0.036106630 image pull 8717720dc5e5c56303818ea4d4f0a11f6e2c03dee521fb668f411f1304e0afca quay.ceph.io/ceph-ci/ceph@sha256:f4452def3be4d350f4a8e5feaf7ebb1511be9298fe68478ba7a2d6fbd7d69a63 2024-04-12T22:01:11.281 INFO:journalctl@ceph.osd.1.smithi018.stdout:Apr 12 22:01:11 smithi018 podman[176443]: 2024-04-12 22:01:11.024747434 +0000 UTC m=+0.171119064 container create 1c52800cdf6c9e4837709b5e3c5663914ffdbfe08219e7a0441d3043b22bf8cd (image=quay.ceph.io/ceph-ci/ceph@sha256:f4452def3be4d350f4a8e5feaf7ebb1511be9298fe68478ba7a2d6fbd7d69a63, name=ceph-1cbe407a-f90d-11ee-bc8d-c7b262605968-osd-1-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-vshankar-testing-20240411.061452-324baf8, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.build-date=20240409, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_CLEAN=True) 2024-04-12T22:01:11.281 INFO:journalctl@ceph.osd.1.smithi018.stdout:Apr 12 22:01:11 smithi018 podman[176443]: 2024-04-12 22:01:11.278779845 +0000 UTC m=+0.425151518 container init 1c52800cdf6c9e4837709b5e3c5663914ffdbfe08219e7a0441d3043b22bf8cd (image=quay.ceph.io/ceph-ci/ceph@sha256:f4452def3be4d350f4a8e5feaf7ebb1511be9298fe68478ba7a2d6fbd7d69a63, name=ceph-1cbe407a-f90d-11ee-bc8d-c7b262605968-osd-1-deactivate, org.label-schema.build-date=20240409, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-vshankar-testing-20240411.061452-324baf8, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, ceph=True, GIT_BRANCH=HEAD) 2024-04-12T22:01:11.605 INFO:journalctl@ceph.osd.1.smithi018.stdout:Apr 12 22:01:11 smithi018 podman[176443]: 2024-04-12 22:01:11.29132352 +0000 UTC m=+0.437695173 container start 1c52800cdf6c9e4837709b5e3c5663914ffdbfe08219e7a0441d3043b22bf8cd (image=quay.ceph.io/ceph-ci/ceph@sha256:f4452def3be4d350f4a8e5feaf7ebb1511be9298fe68478ba7a2d6fbd7d69a63, name=ceph-1cbe407a-f90d-11ee-bc8d-c7b262605968-osd-1-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, RELEASE=wip-vshankar-testing-20240411.061452-324baf8, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240409, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-12T22:01:11.605 INFO:journalctl@ceph.osd.1.smithi018.stdout:Apr 12 22:01:11 smithi018 podman[176443]: 2024-04-12 22:01:11.377368159 +0000 UTC m=+0.523739795 container attach 1c52800cdf6c9e4837709b5e3c5663914ffdbfe08219e7a0441d3043b22bf8cd (image=quay.ceph.io/ceph-ci/ceph@sha256:f4452def3be4d350f4a8e5feaf7ebb1511be9298fe68478ba7a2d6fbd7d69a63, name=ceph-1cbe407a-f90d-11ee-bc8d-c7b262605968-osd-1-deactivate, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, RELEASE=wip-vshankar-testing-20240411.061452-324baf8, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240409, ceph=True, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD) 2024-04-12T22:01:12.105 INFO:journalctl@ceph.osd.1.smithi018.stdout:Apr 12 22:01:11 smithi018 podman[176443]: 2024-04-12 22:01:11.759875945 +0000 UTC m=+0.906247568 container died 1c52800cdf6c9e4837709b5e3c5663914ffdbfe08219e7a0441d3043b22bf8cd (image=quay.ceph.io/ceph-ci/ceph@sha256:f4452def3be4d350f4a8e5feaf7ebb1511be9298fe68478ba7a2d6fbd7d69a63, name=ceph-1cbe407a-f90d-11ee-bc8d-c7b262605968-osd-1-deactivate, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240409, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, RELEASE=wip-vshankar-testing-20240411.061452-324baf8, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-12T22:01:12.499 DEBUG:teuthology.orchestra.run.smithi018:> sudo pkill -f 'journalctl -f -n 0 -u ceph-1cbe407a-f90d-11ee-bc8d-c7b262605968@osd.1.service' 2024-04-12T22:01:12.630 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-12T22:01:12.631 INFO:tasks.cephadm.osd.1:Stopped osd.1 2024-04-12T22:01:12.631 INFO:tasks.cephadm.osd.2:Stopping osd.2... 2024-04-12T22:01:12.631 DEBUG:teuthology.orchestra.run.smithi018:> sudo systemctl stop ceph-1cbe407a-f90d-11ee-bc8d-c7b262605968@osd.2 2024-04-12T22:01:13.106 INFO:journalctl@ceph.osd.2.smithi018.stdout:Apr 12 22:01:12 smithi018 systemd[1]: Stopping Ceph osd.2 for 1cbe407a-f90d-11ee-bc8d-c7b262605968... 2024-04-12T22:01:13.606 INFO:journalctl@ceph.osd.2.smithi018.stdout:Apr 12 22:01:13 smithi018 ceph-1cbe407a-f90d-11ee-bc8d-c7b262605968-osd-2[135391]: 2024-04-12T22:01:13.241+0000 7fb07d800700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-12T22:01:13.606 INFO:journalctl@ceph.osd.2.smithi018.stdout:Apr 12 22:01:13 smithi018 ceph-1cbe407a-f90d-11ee-bc8d-c7b262605968-osd-2[135391]: 2024-04-12T22:01:13.241+0000 7fb07d800700 -1 osd.2 104 *** Got signal Terminated *** 2024-04-12T22:01:13.606 INFO:journalctl@ceph.osd.2.smithi018.stdout:Apr 12 22:01:13 smithi018 ceph-1cbe407a-f90d-11ee-bc8d-c7b262605968-osd-2[135391]: 2024-04-12T22:01:13.241+0000 7fb07d800700 -1 osd.2 104 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-12T22:01:19.106 INFO:journalctl@ceph.osd.2.smithi018.stdout:Apr 12 22:01:18 smithi018 podman[176549]: 2024-04-12 22:01:18.833348983 +0000 UTC m=+5.714907783 container died c2cbc6aa2839b571d7972062a0f7b290aac7616bef836d4755b6844b541f65ad (image=quay.ceph.io/ceph-ci/ceph@sha256:f4452def3be4d350f4a8e5feaf7ebb1511be9298fe68478ba7a2d6fbd7d69a63, name=ceph-1cbe407a-f90d-11ee-bc8d-c7b262605968-osd-2, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.build-date=20240409, io.buildah.version=1.33.5, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, RELEASE=wip-vshankar-testing-20240411.061452-324baf8) 2024-04-12T22:01:19.536 INFO:journalctl@ceph.osd.2.smithi018.stdout:Apr 12 22:01:19 smithi018 podman[176549]: 2024-04-12 22:01:19.267174132 +0000 UTC m=+6.148732925 container cleanup c2cbc6aa2839b571d7972062a0f7b290aac7616bef836d4755b6844b541f65ad (image=quay.ceph.io/ceph-ci/ceph@sha256:f4452def3be4d350f4a8e5feaf7ebb1511be9298fe68478ba7a2d6fbd7d69a63, name=ceph-1cbe407a-f90d-11ee-bc8d-c7b262605968-osd-2, ceph=True, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, RELEASE=wip-vshankar-testing-20240411.061452-324baf8, org.label-schema.build-date=20240409, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux ) 2024-04-12T22:01:19.537 INFO:journalctl@ceph.osd.2.smithi018.stdout:Apr 12 22:01:19 smithi018 bash[176549]: ceph-1cbe407a-f90d-11ee-bc8d-c7b262605968-osd-2 2024-04-12T22:01:19.855 INFO:journalctl@ceph.osd.2.smithi018.stdout:Apr 12 22:01:19 smithi018 podman[176581]: 2024-04-12 22:01:19.535034792 +0000 UTC m=+0.683316172 container remove c2cbc6aa2839b571d7972062a0f7b290aac7616bef836d4755b6844b541f65ad (image=quay.ceph.io/ceph-ci/ceph@sha256:f4452def3be4d350f4a8e5feaf7ebb1511be9298fe68478ba7a2d6fbd7d69a63, name=ceph-1cbe407a-f90d-11ee-bc8d-c7b262605968-osd-2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.build-date=20240409, RELEASE=wip-vshankar-testing-20240411.061452-324baf8, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux ) 2024-04-12T22:01:20.605 INFO:journalctl@ceph.osd.2.smithi018.stdout:Apr 12 22:01:20 smithi018 podman[176670]: 2024-04-12 22:01:20.248359519 +0000 UTC m=+0.033733585 image pull 8717720dc5e5c56303818ea4d4f0a11f6e2c03dee521fb668f411f1304e0afca quay.ceph.io/ceph-ci/ceph@sha256:f4452def3be4d350f4a8e5feaf7ebb1511be9298fe68478ba7a2d6fbd7d69a63 2024-04-12T22:01:20.605 INFO:journalctl@ceph.osd.2.smithi018.stdout:Apr 12 22:01:20 smithi018 podman[176670]: 2024-04-12 22:01:20.36095583 +0000 UTC m=+0.146329957 container create 7b66d2d389060e70cfb67f09d4a3a09f5fbc91fb28e90c7bbd97d0fdab1de7e8 (image=quay.ceph.io/ceph-ci/ceph@sha256:f4452def3be4d350f4a8e5feaf7ebb1511be9298fe68478ba7a2d6fbd7d69a63, name=ceph-1cbe407a-f90d-11ee-bc8d-c7b262605968-osd-2-deactivate, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-vshankar-testing-20240411.061452-324baf8, GIT_CLEAN=True, org.label-schema.build-date=20240409, CEPH_POINT_RELEASE=, ceph=True) 2024-04-12T22:01:21.105 INFO:journalctl@ceph.osd.2.smithi018.stdout:Apr 12 22:01:20 smithi018 podman[176670]: 2024-04-12 22:01:20.65805945 +0000 UTC m=+0.443433594 container init 7b66d2d389060e70cfb67f09d4a3a09f5fbc91fb28e90c7bbd97d0fdab1de7e8 (image=quay.ceph.io/ceph-ci/ceph@sha256:f4452def3be4d350f4a8e5feaf7ebb1511be9298fe68478ba7a2d6fbd7d69a63, name=ceph-1cbe407a-f90d-11ee-bc8d-c7b262605968-osd-2-deactivate, ceph=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, RELEASE=wip-vshankar-testing-20240411.061452-324baf8, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.build-date=20240409, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD) 2024-04-12T22:01:21.105 INFO:journalctl@ceph.osd.2.smithi018.stdout:Apr 12 22:01:20 smithi018 podman[176670]: 2024-04-12 22:01:20.671575478 +0000 UTC m=+0.456949602 container start 7b66d2d389060e70cfb67f09d4a3a09f5fbc91fb28e90c7bbd97d0fdab1de7e8 (image=quay.ceph.io/ceph-ci/ceph@sha256:f4452def3be4d350f4a8e5feaf7ebb1511be9298fe68478ba7a2d6fbd7d69a63, name=ceph-1cbe407a-f90d-11ee-bc8d-c7b262605968-osd-2-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-vshankar-testing-20240411.061452-324baf8, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , ceph=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.build-date=20240409, org.label-schema.vendor=CentOS) 2024-04-12T22:01:21.105 INFO:journalctl@ceph.osd.2.smithi018.stdout:Apr 12 22:01:20 smithi018 podman[176670]: 2024-04-12 22:01:20.772134409 +0000 UTC m=+0.557508488 container attach 7b66d2d389060e70cfb67f09d4a3a09f5fbc91fb28e90c7bbd97d0fdab1de7e8 (image=quay.ceph.io/ceph-ci/ceph@sha256:f4452def3be4d350f4a8e5feaf7ebb1511be9298fe68478ba7a2d6fbd7d69a63, name=ceph-1cbe407a-f90d-11ee-bc8d-c7b262605968-osd-2-deactivate, org.label-schema.build-date=20240409, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, ceph=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.schema-version=1.0, RELEASE=wip-vshankar-testing-20240411.061452-324baf8, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-12T22:01:21.485 INFO:journalctl@ceph.osd.2.smithi018.stdout:Apr 12 22:01:21 smithi018 podman[176670]: 2024-04-12 22:01:21.141916566 +0000 UTC m=+0.927290667 container died 7b66d2d389060e70cfb67f09d4a3a09f5fbc91fb28e90c7bbd97d0fdab1de7e8 (image=quay.ceph.io/ceph-ci/ceph@sha256:f4452def3be4d350f4a8e5feaf7ebb1511be9298fe68478ba7a2d6fbd7d69a63, name=ceph-1cbe407a-f90d-11ee-bc8d-c7b262605968-osd-2-deactivate, RELEASE=wip-vshankar-testing-20240411.061452-324baf8, GIT_BRANCH=HEAD, org.label-schema.build-date=20240409, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, org.label-schema.vendor=CentOS) 2024-04-12T22:01:21.844 DEBUG:teuthology.orchestra.run.smithi018:> sudo pkill -f 'journalctl -f -n 0 -u ceph-1cbe407a-f90d-11ee-bc8d-c7b262605968@osd.2.service' 2024-04-12T22:01:21.967 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-12T22:01:21.968 INFO:tasks.cephadm.osd.2:Stopped osd.2 2024-04-12T22:01:21.968 INFO:tasks.cephadm.osd.3:Stopping osd.3... 2024-04-12T22:01:21.968 DEBUG:teuthology.orchestra.run.smithi186:> sudo systemctl stop ceph-1cbe407a-f90d-11ee-bc8d-c7b262605968@osd.3 2024-04-12T22:01:22.284 INFO:journalctl@ceph.osd.3.smithi186.stdout:Apr 12 22:01:21 smithi186 ceph-1cbe407a-f90d-11ee-bc8d-c7b262605968-osd-3[95422]: 2024-04-12T22:01:21.939+0000 7f171a000700 -1 osd.3 104 heartbeat_check: no reply from 172.21.15.18:6806 osd.0 since back 2024-04-12T22:00:57.646614+0000 front 2024-04-12T22:00:57.646795+0000 (oldest deadline 2024-04-12T22:01:21.746186+0000) 2024-04-12T22:01:22.284 INFO:journalctl@ceph.osd.3.smithi186.stdout:Apr 12 22:01:22 smithi186 systemd[1]: Stopping Ceph osd.3 for 1cbe407a-f90d-11ee-bc8d-c7b262605968... 2024-04-12T22:01:22.784 INFO:journalctl@ceph.osd.4.smithi186.stdout:Apr 12 22:01:22 smithi186 ceph-1cbe407a-f90d-11ee-bc8d-c7b262605968-osd-4[100708]: 2024-04-12T22:01:22.720+0000 7fbcd6800700 -1 osd.4 104 heartbeat_check: no reply from 172.21.15.18:6806 osd.0 since back 2024-04-12T22:00:57.616714+0000 front 2024-04-12T22:00:57.616664+0000 (oldest deadline 2024-04-12T22:01:22.316121+0000) 2024-04-12T22:01:22.785 INFO:journalctl@ceph.osd.3.smithi186.stdout:Apr 12 22:01:22 smithi186 ceph-1cbe407a-f90d-11ee-bc8d-c7b262605968-osd-3[95422]: 2024-04-12T22:01:22.518+0000 7f1723800700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.3 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-12T22:01:22.785 INFO:journalctl@ceph.osd.3.smithi186.stdout:Apr 12 22:01:22 smithi186 ceph-1cbe407a-f90d-11ee-bc8d-c7b262605968-osd-3[95422]: 2024-04-12T22:01:22.518+0000 7f1723800700 -1 osd.3 104 *** Got signal Terminated *** 2024-04-12T22:01:22.785 INFO:journalctl@ceph.osd.3.smithi186.stdout:Apr 12 22:01:22 smithi186 ceph-1cbe407a-f90d-11ee-bc8d-c7b262605968-osd-3[95422]: 2024-04-12T22:01:22.519+0000 7f1723800700 -1 osd.3 104 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-12T22:01:23.284 INFO:journalctl@ceph.osd.3.smithi186.stdout:Apr 12 22:01:22 smithi186 ceph-1cbe407a-f90d-11ee-bc8d-c7b262605968-osd-3[95422]: 2024-04-12T22:01:22.967+0000 7f171a000700 -1 osd.3 104 heartbeat_check: no reply from 172.21.15.18:6806 osd.0 since back 2024-04-12T22:00:57.646614+0000 front 2024-04-12T22:00:57.646795+0000 (oldest deadline 2024-04-12T22:01:21.746186+0000) 2024-04-12T22:01:23.943 INFO:journalctl@ceph.osd.4.smithi186.stdout:Apr 12 22:01:23 smithi186 ceph-1cbe407a-f90d-11ee-bc8d-c7b262605968-osd-4[100708]: 2024-04-12T22:01:23.767+0000 7fbcd6800700 -1 osd.4 104 heartbeat_check: no reply from 172.21.15.18:6806 osd.0 since back 2024-04-12T22:00:57.616714+0000 front 2024-04-12T22:00:57.616664+0000 (oldest deadline 2024-04-12T22:01:22.316121+0000) 2024-04-12T22:01:23.943 INFO:journalctl@ceph.osd.5.smithi186.stdout:Apr 12 22:01:23 smithi186 ceph-1cbe407a-f90d-11ee-bc8d-c7b262605968-osd-5[105976]: 2024-04-12T22:01:23.618+0000 7f8211600700 -1 osd.5 104 heartbeat_check: no reply from 172.21.15.18:6806 osd.0 since back 2024-04-12T22:00:58.233676+0000 front 2024-04-12T22:00:58.233957+0000 (oldest deadline 2024-04-12T22:01:23.533389+0000) 2024-04-12T22:01:24.284 INFO:journalctl@ceph.osd.3.smithi186.stdout:Apr 12 22:01:23 smithi186 ceph-1cbe407a-f90d-11ee-bc8d-c7b262605968-osd-3[95422]: 2024-04-12T22:01:23.941+0000 7f171a000700 -1 osd.3 104 heartbeat_check: no reply from 172.21.15.18:6806 osd.0 since back 2024-04-12T22:00:57.646614+0000 front 2024-04-12T22:00:57.646795+0000 (oldest deadline 2024-04-12T22:01:21.746186+0000) 2024-04-12T22:01:24.948 INFO:journalctl@ceph.osd.4.smithi186.stdout:Apr 12 22:01:24 smithi186 ceph-1cbe407a-f90d-11ee-bc8d-c7b262605968-osd-4[100708]: 2024-04-12T22:01:24.720+0000 7fbcd6800700 -1 osd.4 104 heartbeat_check: no reply from 172.21.15.18:6806 osd.0 since back 2024-04-12T22:00:57.616714+0000 front 2024-04-12T22:00:57.616664+0000 (oldest deadline 2024-04-12T22:01:22.316121+0000) 2024-04-12T22:01:24.948 INFO:journalctl@ceph.osd.5.smithi186.stdout:Apr 12 22:01:24 smithi186 ceph-1cbe407a-f90d-11ee-bc8d-c7b262605968-osd-5[105976]: 2024-04-12T22:01:24.629+0000 7f8211600700 -1 osd.5 104 heartbeat_check: no reply from 172.21.15.18:6806 osd.0 since back 2024-04-12T22:00:58.233676+0000 front 2024-04-12T22:00:58.233957+0000 (oldest deadline 2024-04-12T22:01:23.533389+0000) 2024-04-12T22:01:25.284 INFO:journalctl@ceph.osd.3.smithi186.stdout:Apr 12 22:01:24 smithi186 ceph-1cbe407a-f90d-11ee-bc8d-c7b262605968-osd-3[95422]: 2024-04-12T22:01:24.947+0000 7f171a000700 -1 osd.3 104 heartbeat_check: no reply from 172.21.15.18:6806 osd.0 since back 2024-04-12T22:00:57.646614+0000 front 2024-04-12T22:00:57.646795+0000 (oldest deadline 2024-04-12T22:01:21.746186+0000) 2024-04-12T22:01:25.987 INFO:journalctl@ceph.osd.4.smithi186.stdout:Apr 12 22:01:25 smithi186 ceph-1cbe407a-f90d-11ee-bc8d-c7b262605968-osd-4[100708]: 2024-04-12T22:01:25.766+0000 7fbcd6800700 -1 osd.4 104 heartbeat_check: no reply from 172.21.15.18:6806 osd.0 since back 2024-04-12T22:00:57.616714+0000 front 2024-04-12T22:00:57.616664+0000 (oldest deadline 2024-04-12T22:01:22.316121+0000) 2024-04-12T22:01:25.987 INFO:journalctl@ceph.osd.5.smithi186.stdout:Apr 12 22:01:25 smithi186 ceph-1cbe407a-f90d-11ee-bc8d-c7b262605968-osd-5[105976]: 2024-04-12T22:01:25.646+0000 7f8211600700 -1 osd.5 104 heartbeat_check: no reply from 172.21.15.18:6806 osd.0 since back 2024-04-12T22:00:58.233676+0000 front 2024-04-12T22:00:58.233957+0000 (oldest deadline 2024-04-12T22:01:23.533389+0000) 2024-04-12T22:01:26.286 INFO:journalctl@ceph.osd.3.smithi186.stdout:Apr 12 22:01:25 smithi186 ceph-1cbe407a-f90d-11ee-bc8d-c7b262605968-osd-3[95422]: 2024-04-12T22:01:25.985+0000 7f171a000700 -1 osd.3 104 heartbeat_check: no reply from 172.21.15.18:6806 osd.0 since back 2024-04-12T22:00:57.646614+0000 front 2024-04-12T22:00:57.646795+0000 (oldest deadline 2024-04-12T22:01:21.746186+0000) 2024-04-12T22:01:26.989 INFO:journalctl@ceph.osd.4.smithi186.stdout:Apr 12 22:01:26 smithi186 ceph-1cbe407a-f90d-11ee-bc8d-c7b262605968-osd-4[100708]: 2024-04-12T22:01:26.776+0000 7fbcd6800700 -1 osd.4 104 heartbeat_check: no reply from 172.21.15.18:6806 osd.0 since back 2024-04-12T22:00:57.616714+0000 front 2024-04-12T22:00:57.616664+0000 (oldest deadline 2024-04-12T22:01:22.316121+0000) 2024-04-12T22:01:26.989 INFO:journalctl@ceph.osd.5.smithi186.stdout:Apr 12 22:01:26 smithi186 ceph-1cbe407a-f90d-11ee-bc8d-c7b262605968-osd-5[105976]: 2024-04-12T22:01:26.690+0000 7f8211600700 -1 osd.5 104 heartbeat_check: no reply from 172.21.15.18:6806 osd.0 since back 2024-04-12T22:00:58.233676+0000 front 2024-04-12T22:00:58.233957+0000 (oldest deadline 2024-04-12T22:01:23.533389+0000) 2024-04-12T22:01:27.284 INFO:journalctl@ceph.osd.3.smithi186.stdout:Apr 12 22:01:26 smithi186 ceph-1cbe407a-f90d-11ee-bc8d-c7b262605968-osd-3[95422]: 2024-04-12T22:01:26.987+0000 7f171a000700 -1 osd.3 104 heartbeat_check: no reply from 172.21.15.18:6806 osd.0 since back 2024-04-12T22:00:57.646614+0000 front 2024-04-12T22:00:57.646795+0000 (oldest deadline 2024-04-12T22:01:21.746186+0000) 2024-04-12T22:01:28.034 INFO:journalctl@ceph.osd.4.smithi186.stdout:Apr 12 22:01:27 smithi186 ceph-1cbe407a-f90d-11ee-bc8d-c7b262605968-osd-4[100708]: 2024-04-12T22:01:27.730+0000 7fbcd6800700 -1 osd.4 104 heartbeat_check: no reply from 172.21.15.18:6806 osd.0 since back 2024-04-12T22:00:57.616714+0000 front 2024-04-12T22:00:57.616664+0000 (oldest deadline 2024-04-12T22:01:22.316121+0000) 2024-04-12T22:01:28.034 INFO:journalctl@ceph.osd.5.smithi186.stdout:Apr 12 22:01:27 smithi186 ceph-1cbe407a-f90d-11ee-bc8d-c7b262605968-osd-5[105976]: 2024-04-12T22:01:27.732+0000 7f8211600700 -1 osd.5 104 heartbeat_check: no reply from 172.21.15.18:6806 osd.0 since back 2024-04-12T22:00:58.233676+0000 front 2024-04-12T22:00:58.233957+0000 (oldest deadline 2024-04-12T22:01:23.533389+0000) 2024-04-12T22:01:28.453 INFO:journalctl@ceph.osd.3.smithi186.stdout:Apr 12 22:01:28 smithi186 podman[123186]: 2024-04-12 22:01:28.13003477 +0000 UTC m=+5.735086008 container died 0138e8262e4bab328e00c422fb40fa78a35c6daf7fe25c2b12b216036536a403 (image=quay.ceph.io/ceph-ci/ceph@sha256:f4452def3be4d350f4a8e5feaf7ebb1511be9298fe68478ba7a2d6fbd7d69a63, name=ceph-1cbe407a-f90d-11ee-bc8d-c7b262605968-osd-3, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-vshankar-testing-20240411.061452-324baf8, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240409, GIT_CLEAN=True, CEPH_POINT_RELEASE=) 2024-04-12T22:01:28.763 INFO:journalctl@ceph.osd.5.smithi186.stdout:Apr 12 22:01:28 smithi186 ceph-1cbe407a-f90d-11ee-bc8d-c7b262605968-osd-5[105976]: 2024-04-12T22:01:28.686+0000 7f8211600700 -1 osd.5 104 heartbeat_check: no reply from 172.21.15.18:6806 osd.0 since back 2024-04-12T22:00:58.233676+0000 front 2024-04-12T22:00:58.233957+0000 (oldest deadline 2024-04-12T22:01:23.533389+0000) 2024-04-12T22:01:28.763 INFO:journalctl@ceph.osd.3.smithi186.stdout:Apr 12 22:01:28 smithi186 podman[123186]: 2024-04-12 22:01:28.569615352 +0000 UTC m=+6.174666584 container cleanup 0138e8262e4bab328e00c422fb40fa78a35c6daf7fe25c2b12b216036536a403 (image=quay.ceph.io/ceph-ci/ceph@sha256:f4452def3be4d350f4a8e5feaf7ebb1511be9298fe68478ba7a2d6fbd7d69a63, name=ceph-1cbe407a-f90d-11ee-bc8d-c7b262605968-osd-3, GIT_BRANCH=HEAD, RELEASE=wip-vshankar-testing-20240411.061452-324baf8, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.build-date=20240409, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0) 2024-04-12T22:01:28.763 INFO:journalctl@ceph.osd.3.smithi186.stdout:Apr 12 22:01:28 smithi186 bash[123186]: ceph-1cbe407a-f90d-11ee-bc8d-c7b262605968-osd-3 2024-04-12T22:01:29.034 INFO:journalctl@ceph.osd.4.smithi186.stdout:Apr 12 22:01:28 smithi186 ceph-1cbe407a-f90d-11ee-bc8d-c7b262605968-osd-4[100708]: 2024-04-12T22:01:28.761+0000 7fbcd6800700 -1 osd.4 104 heartbeat_check: no reply from 172.21.15.18:6806 osd.0 since back 2024-04-12T22:00:57.616714+0000 front 2024-04-12T22:00:57.616664+0000 (oldest deadline 2024-04-12T22:01:22.316121+0000) 2024-04-12T22:01:29.035 INFO:journalctl@ceph.osd.3.smithi186.stdout:Apr 12 22:01:28 smithi186 podman[123204]: 2024-04-12 22:01:28.812466023 +0000 UTC m=+0.678203937 container remove 0138e8262e4bab328e00c422fb40fa78a35c6daf7fe25c2b12b216036536a403 (image=quay.ceph.io/ceph-ci/ceph@sha256:f4452def3be4d350f4a8e5feaf7ebb1511be9298fe68478ba7a2d6fbd7d69a63, name=ceph-1cbe407a-f90d-11ee-bc8d-c7b262605968-osd-3, org.label-schema.build-date=20240409, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-vshankar-testing-20240411.061452-324baf8, org.label-schema.vendor=CentOS, GIT_CLEAN=True, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=) 2024-04-12T22:01:29.950 INFO:journalctl@ceph.osd.5.smithi186.stdout:Apr 12 22:01:29 smithi186 ceph-1cbe407a-f90d-11ee-bc8d-c7b262605968-osd-5[105976]: 2024-04-12T22:01:29.668+0000 7f8211600700 -1 osd.5 104 heartbeat_check: no reply from 172.21.15.18:6806 osd.0 since back 2024-04-12T22:00:58.233676+0000 front 2024-04-12T22:00:58.233957+0000 (oldest deadline 2024-04-12T22:01:23.533389+0000) 2024-04-12T22:01:29.950 INFO:journalctl@ceph.osd.3.smithi186.stdout:Apr 12 22:01:29 smithi186 podman[123291]: 2024-04-12 22:01:29.518179256 +0000 UTC m=+0.034526010 image pull 8717720dc5e5c56303818ea4d4f0a11f6e2c03dee521fb668f411f1304e0afca quay.ceph.io/ceph-ci/ceph@sha256:f4452def3be4d350f4a8e5feaf7ebb1511be9298fe68478ba7a2d6fbd7d69a63 2024-04-12T22:01:29.950 INFO:journalctl@ceph.osd.3.smithi186.stdout:Apr 12 22:01:29 smithi186 podman[123291]: 2024-04-12 22:01:29.646722736 +0000 UTC m=+0.163069531 container create a8f7a8ee4ddcca02de890780838ed51943b79708108629fc789d401daebc1754 (image=quay.ceph.io/ceph-ci/ceph@sha256:f4452def3be4d350f4a8e5feaf7ebb1511be9298fe68478ba7a2d6fbd7d69a63, name=ceph-1cbe407a-f90d-11ee-bc8d-c7b262605968-osd-3-deactivate, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240409, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, ceph=True, GIT_BRANCH=HEAD, RELEASE=wip-vshankar-testing-20240411.061452-324baf8, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=) 2024-04-12T22:01:29.950 INFO:journalctl@ceph.osd.3.smithi186.stdout:Apr 12 22:01:29 smithi186 podman[123291]: 2024-04-12 22:01:29.948264238 +0000 UTC m=+0.464611039 container init a8f7a8ee4ddcca02de890780838ed51943b79708108629fc789d401daebc1754 (image=quay.ceph.io/ceph-ci/ceph@sha256:f4452def3be4d350f4a8e5feaf7ebb1511be9298fe68478ba7a2d6fbd7d69a63, name=ceph-1cbe407a-f90d-11ee-bc8d-c7b262605968-osd-3-deactivate, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, RELEASE=wip-vshankar-testing-20240411.061452-324baf8, org.label-schema.build-date=20240409, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS) 2024-04-12T22:01:29.950 INFO:journalctl@ceph.osd.4.smithi186.stdout:Apr 12 22:01:29 smithi186 ceph-1cbe407a-f90d-11ee-bc8d-c7b262605968-osd-4[100708]: 2024-04-12T22:01:29.768+0000 7fbcd6800700 -1 osd.4 104 heartbeat_check: no reply from 172.21.15.18:6806 osd.0 since back 2024-04-12T22:00:57.616714+0000 front 2024-04-12T22:00:57.616664+0000 (oldest deadline 2024-04-12T22:01:22.316121+0000) 2024-04-12T22:01:30.284 INFO:journalctl@ceph.osd.3.smithi186.stdout:Apr 12 22:01:29 smithi186 podman[123291]: 2024-04-12 22:01:29.959466962 +0000 UTC m=+0.475813733 container start a8f7a8ee4ddcca02de890780838ed51943b79708108629fc789d401daebc1754 (image=quay.ceph.io/ceph-ci/ceph@sha256:f4452def3be4d350f4a8e5feaf7ebb1511be9298fe68478ba7a2d6fbd7d69a63, name=ceph-1cbe407a-f90d-11ee-bc8d-c7b262605968-osd-3-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, RELEASE=wip-vshankar-testing-20240411.061452-324baf8, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240409) 2024-04-12T22:01:30.284 INFO:journalctl@ceph.osd.3.smithi186.stdout:Apr 12 22:01:30 smithi186 podman[123291]: 2024-04-12 22:01:30.038422299 +0000 UTC m=+0.554769110 container attach a8f7a8ee4ddcca02de890780838ed51943b79708108629fc789d401daebc1754 (image=quay.ceph.io/ceph-ci/ceph@sha256:f4452def3be4d350f4a8e5feaf7ebb1511be9298fe68478ba7a2d6fbd7d69a63, name=ceph-1cbe407a-f90d-11ee-bc8d-c7b262605968-osd-3-deactivate, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_CLEAN=True, RELEASE=wip-vshankar-testing-20240411.061452-324baf8, org.label-schema.build-date=20240409, ceph=True, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-12T22:01:30.745 INFO:journalctl@ceph.osd.5.smithi186.stdout:Apr 12 22:01:30 smithi186 ceph-1cbe407a-f90d-11ee-bc8d-c7b262605968-osd-5[105976]: 2024-04-12T22:01:30.619+0000 7f8211600700 -1 osd.5 104 heartbeat_check: no reply from 172.21.15.18:6806 osd.0 since back 2024-04-12T22:00:58.233676+0000 front 2024-04-12T22:00:58.233957+0000 (oldest deadline 2024-04-12T22:01:23.533389+0000) 2024-04-12T22:01:30.745 INFO:journalctl@ceph.osd.3.smithi186.stdout:Apr 12 22:01:30 smithi186 podman[123291]: 2024-04-12 22:01:30.415020267 +0000 UTC m=+0.931367065 container died a8f7a8ee4ddcca02de890780838ed51943b79708108629fc789d401daebc1754 (image=quay.ceph.io/ceph-ci/ceph@sha256:f4452def3be4d350f4a8e5feaf7ebb1511be9298fe68478ba7a2d6fbd7d69a63, name=ceph-1cbe407a-f90d-11ee-bc8d-c7b262605968-osd-3-deactivate, GIT_BRANCH=HEAD, org.label-schema.build-date=20240409, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-vshankar-testing-20240411.061452-324baf8, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_CLEAN=True, ceph=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-12T22:01:30.746 INFO:journalctl@ceph.osd.4.smithi186.stdout:Apr 12 22:01:30 smithi186 ceph-1cbe407a-f90d-11ee-bc8d-c7b262605968-osd-4[100708]: 2024-04-12T22:01:30.743+0000 7fbcd6800700 -1 osd.4 104 heartbeat_check: no reply from 172.21.15.18:6806 osd.0 since back 2024-04-12T22:00:57.616714+0000 front 2024-04-12T22:00:57.616664+0000 (oldest deadline 2024-04-12T22:01:22.316121+0000) 2024-04-12T22:01:30.746 INFO:journalctl@ceph.osd.4.smithi186.stdout:Apr 12 22:01:30 smithi186 ceph-1cbe407a-f90d-11ee-bc8d-c7b262605968-osd-4[100708]: 2024-04-12T22:01:30.743+0000 7fbcd6800700 -1 osd.4 104 heartbeat_check: no reply from 172.21.15.18:6814 osd.1 since back 2024-04-12T22:01:04.018167+0000 front 2024-04-12T22:01:04.017636+0000 (oldest deadline 2024-04-12T22:01:29.917422+0000) 2024-04-12T22:01:31.157 DEBUG:teuthology.orchestra.run.smithi186:> sudo pkill -f 'journalctl -f -n 0 -u ceph-1cbe407a-f90d-11ee-bc8d-c7b262605968@osd.3.service' 2024-04-12T22:01:31.259 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-12T22:01:31.259 INFO:tasks.cephadm.osd.3:Stopped osd.3 2024-04-12T22:01:31.259 INFO:tasks.cephadm.osd.4:Stopping osd.4... 2024-04-12T22:01:31.259 DEBUG:teuthology.orchestra.run.smithi186:> sudo systemctl stop ceph-1cbe407a-f90d-11ee-bc8d-c7b262605968@osd.4 2024-04-12T22:01:31.763 INFO:journalctl@ceph.osd.4.smithi186.stdout:Apr 12 22:01:31 smithi186 systemd[1]: Stopping Ceph osd.4 for 1cbe407a-f90d-11ee-bc8d-c7b262605968... 2024-04-12T22:01:31.764 INFO:journalctl@ceph.osd.5.smithi186.stdout:Apr 12 22:01:31 smithi186 ceph-1cbe407a-f90d-11ee-bc8d-c7b262605968-osd-5[105976]: 2024-04-12T22:01:31.599+0000 7f8211600700 -1 osd.5 104 heartbeat_check: no reply from 172.21.15.18:6806 osd.0 since back 2024-04-12T22:00:58.233676+0000 front 2024-04-12T22:00:58.233957+0000 (oldest deadline 2024-04-12T22:01:23.533389+0000) 2024-04-12T22:01:32.034 INFO:journalctl@ceph.osd.4.smithi186.stdout:Apr 12 22:01:31 smithi186 ceph-1cbe407a-f90d-11ee-bc8d-c7b262605968-osd-4[100708]: 2024-04-12T22:01:31.762+0000 7fbcdb000700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.4 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-12T22:01:32.034 INFO:journalctl@ceph.osd.4.smithi186.stdout:Apr 12 22:01:31 smithi186 ceph-1cbe407a-f90d-11ee-bc8d-c7b262605968-osd-4[100708]: 2024-04-12T22:01:31.762+0000 7fbcdb000700 -1 osd.4 104 *** Got signal Terminated *** 2024-04-12T22:01:32.034 INFO:journalctl@ceph.osd.4.smithi186.stdout:Apr 12 22:01:31 smithi186 ceph-1cbe407a-f90d-11ee-bc8d-c7b262605968-osd-4[100708]: 2024-04-12T22:01:31.762+0000 7fbcdb000700 -1 osd.4 104 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-12T22:01:32.034 INFO:journalctl@ceph.osd.4.smithi186.stdout:Apr 12 22:01:31 smithi186 ceph-1cbe407a-f90d-11ee-bc8d-c7b262605968-osd-4[100708]: 2024-04-12T22:01:31.773+0000 7fbcd6800700 -1 osd.4 104 heartbeat_check: no reply from 172.21.15.18:6806 osd.0 since back 2024-04-12T22:00:57.616714+0000 front 2024-04-12T22:00:57.616664+0000 (oldest deadline 2024-04-12T22:01:22.316121+0000) 2024-04-12T22:01:32.035 INFO:journalctl@ceph.osd.4.smithi186.stdout:Apr 12 22:01:31 smithi186 ceph-1cbe407a-f90d-11ee-bc8d-c7b262605968-osd-4[100708]: 2024-04-12T22:01:31.774+0000 7fbcd6800700 -1 osd.4 104 heartbeat_check: no reply from 172.21.15.18:6814 osd.1 since back 2024-04-12T22:01:04.018167+0000 front 2024-04-12T22:01:04.017636+0000 (oldest deadline 2024-04-12T22:01:29.917422+0000) 2024-04-12T22:01:33.034 INFO:journalctl@ceph.osd.4.smithi186.stdout:Apr 12 22:01:32 smithi186 ceph-1cbe407a-f90d-11ee-bc8d-c7b262605968-osd-4[100708]: 2024-04-12T22:01:32.799+0000 7fbcd6800700 -1 osd.4 104 heartbeat_check: no reply from 172.21.15.18:6806 osd.0 since back 2024-04-12T22:00:57.616714+0000 front 2024-04-12T22:00:57.616664+0000 (oldest deadline 2024-04-12T22:01:22.316121+0000) 2024-04-12T22:01:33.034 INFO:journalctl@ceph.osd.4.smithi186.stdout:Apr 12 22:01:32 smithi186 ceph-1cbe407a-f90d-11ee-bc8d-c7b262605968-osd-4[100708]: 2024-04-12T22:01:32.799+0000 7fbcd6800700 -1 osd.4 104 heartbeat_check: no reply from 172.21.15.18:6814 osd.1 since back 2024-04-12T22:01:04.018167+0000 front 2024-04-12T22:01:04.017636+0000 (oldest deadline 2024-04-12T22:01:29.917422+0000) 2024-04-12T22:01:33.035 INFO:journalctl@ceph.osd.5.smithi186.stdout:Apr 12 22:01:32 smithi186 ceph-1cbe407a-f90d-11ee-bc8d-c7b262605968-osd-5[105976]: 2024-04-12T22:01:32.555+0000 7f8211600700 -1 osd.5 104 heartbeat_check: no reply from 172.21.15.18:6806 osd.0 since back 2024-04-12T22:00:58.233676+0000 front 2024-04-12T22:00:58.233957+0000 (oldest deadline 2024-04-12T22:01:23.533389+0000) 2024-04-12T22:01:34.034 INFO:journalctl@ceph.osd.5.smithi186.stdout:Apr 12 22:01:33 smithi186 ceph-1cbe407a-f90d-11ee-bc8d-c7b262605968-osd-5[105976]: 2024-04-12T22:01:33.557+0000 7f8211600700 -1 osd.5 104 heartbeat_check: no reply from 172.21.15.18:6806 osd.0 since back 2024-04-12T22:00:58.233676+0000 front 2024-04-12T22:00:58.233957+0000 (oldest deadline 2024-04-12T22:01:23.533389+0000) 2024-04-12T22:01:34.035 INFO:journalctl@ceph.osd.4.smithi186.stdout:Apr 12 22:01:33 smithi186 ceph-1cbe407a-f90d-11ee-bc8d-c7b262605968-osd-4[100708]: 2024-04-12T22:01:33.800+0000 7fbcd6800700 -1 osd.4 104 heartbeat_check: no reply from 172.21.15.18:6806 osd.0 since back 2024-04-12T22:00:57.616714+0000 front 2024-04-12T22:00:57.616664+0000 (oldest deadline 2024-04-12T22:01:22.316121+0000) 2024-04-12T22:01:34.035 INFO:journalctl@ceph.osd.4.smithi186.stdout:Apr 12 22:01:33 smithi186 ceph-1cbe407a-f90d-11ee-bc8d-c7b262605968-osd-4[100708]: 2024-04-12T22:01:33.800+0000 7fbcd6800700 -1 osd.4 104 heartbeat_check: no reply from 172.21.15.18:6814 osd.1 since back 2024-04-12T22:01:04.018167+0000 front 2024-04-12T22:01:04.017636+0000 (oldest deadline 2024-04-12T22:01:29.917422+0000) 2024-04-12T22:01:34.791 INFO:journalctl@ceph.osd.5.smithi186.stdout:Apr 12 22:01:34 smithi186 ceph-1cbe407a-f90d-11ee-bc8d-c7b262605968-osd-5[105976]: 2024-04-12T22:01:34.534+0000 7f8211600700 -1 osd.5 104 heartbeat_check: no reply from 172.21.15.18:6806 osd.0 since back 2024-04-12T22:00:58.233676+0000 front 2024-04-12T22:00:58.233957+0000 (oldest deadline 2024-04-12T22:01:23.533389+0000) 2024-04-12T22:01:35.284 INFO:journalctl@ceph.osd.4.smithi186.stdout:Apr 12 22:01:34 smithi186 ceph-1cbe407a-f90d-11ee-bc8d-c7b262605968-osd-4[100708]: 2024-04-12T22:01:34.790+0000 7fbcd6800700 -1 osd.4 104 heartbeat_check: no reply from 172.21.15.18:6806 osd.0 since back 2024-04-12T22:00:57.616714+0000 front 2024-04-12T22:00:57.616664+0000 (oldest deadline 2024-04-12T22:01:22.316121+0000) 2024-04-12T22:01:35.284 INFO:journalctl@ceph.osd.4.smithi186.stdout:Apr 12 22:01:34 smithi186 ceph-1cbe407a-f90d-11ee-bc8d-c7b262605968-osd-4[100708]: 2024-04-12T22:01:34.790+0000 7fbcd6800700 -1 osd.4 104 heartbeat_check: no reply from 172.21.15.18:6814 osd.1 since back 2024-04-12T22:01:04.018167+0000 front 2024-04-12T22:01:04.017636+0000 (oldest deadline 2024-04-12T22:01:29.917422+0000) 2024-04-12T22:01:35.806 INFO:journalctl@ceph.osd.5.smithi186.stdout:Apr 12 22:01:35 smithi186 ceph-1cbe407a-f90d-11ee-bc8d-c7b262605968-osd-5[105976]: 2024-04-12T22:01:35.549+0000 7f8211600700 -1 osd.5 104 heartbeat_check: no reply from 172.21.15.18:6806 osd.0 since back 2024-04-12T22:00:58.233676+0000 front 2024-04-12T22:00:58.233957+0000 (oldest deadline 2024-04-12T22:01:23.533389+0000) 2024-04-12T22:01:35.807 INFO:journalctl@ceph.osd.5.smithi186.stdout:Apr 12 22:01:35 smithi186 ceph-1cbe407a-f90d-11ee-bc8d-c7b262605968-osd-5[105976]: 2024-04-12T22:01:35.549+0000 7f8211600700 -1 osd.5 104 heartbeat_check: no reply from 172.21.15.18:6814 osd.1 since back 2024-04-12T22:01:03.534222+0000 front 2024-04-12T22:01:03.534417+0000 (oldest deadline 2024-04-12T22:01:34.734828+0000) 2024-04-12T22:01:36.284 INFO:journalctl@ceph.osd.4.smithi186.stdout:Apr 12 22:01:35 smithi186 ceph-1cbe407a-f90d-11ee-bc8d-c7b262605968-osd-4[100708]: 2024-04-12T22:01:35.804+0000 7fbcd6800700 -1 osd.4 104 heartbeat_check: no reply from 172.21.15.18:6806 osd.0 since back 2024-04-12T22:00:57.616714+0000 front 2024-04-12T22:00:57.616664+0000 (oldest deadline 2024-04-12T22:01:22.316121+0000) 2024-04-12T22:01:36.284 INFO:journalctl@ceph.osd.4.smithi186.stdout:Apr 12 22:01:35 smithi186 ceph-1cbe407a-f90d-11ee-bc8d-c7b262605968-osd-4[100708]: 2024-04-12T22:01:35.805+0000 7fbcd6800700 -1 osd.4 104 heartbeat_check: no reply from 172.21.15.18:6814 osd.1 since back 2024-04-12T22:01:04.018167+0000 front 2024-04-12T22:01:04.017636+0000 (oldest deadline 2024-04-12T22:01:29.917422+0000) 2024-04-12T22:01:37.034 INFO:journalctl@ceph.osd.5.smithi186.stdout:Apr 12 22:01:36 smithi186 ceph-1cbe407a-f90d-11ee-bc8d-c7b262605968-osd-5[105976]: 2024-04-12T22:01:36.541+0000 7f8211600700 -1 osd.5 104 heartbeat_check: no reply from 172.21.15.18:6806 osd.0 since back 2024-04-12T22:00:58.233676+0000 front 2024-04-12T22:00:58.233957+0000 (oldest deadline 2024-04-12T22:01:23.533389+0000) 2024-04-12T22:01:37.034 INFO:journalctl@ceph.osd.5.smithi186.stdout:Apr 12 22:01:36 smithi186 ceph-1cbe407a-f90d-11ee-bc8d-c7b262605968-osd-5[105976]: 2024-04-12T22:01:36.541+0000 7f8211600700 -1 osd.5 104 heartbeat_check: no reply from 172.21.15.18:6814 osd.1 since back 2024-04-12T22:01:03.534222+0000 front 2024-04-12T22:01:03.534417+0000 (oldest deadline 2024-04-12T22:01:34.734828+0000) 2024-04-12T22:01:37.693 INFO:journalctl@ceph.osd.4.smithi186.stdout:Apr 12 22:01:37 smithi186 podman[123406]: 2024-04-12 22:01:37.32446095 +0000 UTC m=+5.695415219 container died b20cb63b25c9f6653442c9f5146dd3b2af3ffe45af103d11ab22c9991f5e799d (image=quay.ceph.io/ceph-ci/ceph@sha256:f4452def3be4d350f4a8e5feaf7ebb1511be9298fe68478ba7a2d6fbd7d69a63, name=ceph-1cbe407a-f90d-11ee-bc8d-c7b262605968-osd-4, GIT_BRANCH=HEAD, RELEASE=wip-vshankar-testing-20240411.061452-324baf8, org.label-schema.build-date=20240409, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.license=GPLv2, GIT_CLEAN=True) 2024-04-12T22:01:37.693 INFO:journalctl@ceph.osd.5.smithi186.stdout:Apr 12 22:01:37 smithi186 ceph-1cbe407a-f90d-11ee-bc8d-c7b262605968-osd-5[105976]: 2024-04-12T22:01:37.515+0000 7f8211600700 -1 osd.5 104 heartbeat_check: no reply from 172.21.15.18:6806 osd.0 since back 2024-04-12T22:00:58.233676+0000 front 2024-04-12T22:00:58.233957+0000 (oldest deadline 2024-04-12T22:01:23.533389+0000) 2024-04-12T22:01:37.693 INFO:journalctl@ceph.osd.5.smithi186.stdout:Apr 12 22:01:37 smithi186 ceph-1cbe407a-f90d-11ee-bc8d-c7b262605968-osd-5[105976]: 2024-04-12T22:01:37.515+0000 7f8211600700 -1 osd.5 104 heartbeat_check: no reply from 172.21.15.18:6814 osd.1 since back 2024-04-12T22:01:03.534222+0000 front 2024-04-12T22:01:03.534417+0000 (oldest deadline 2024-04-12T22:01:34.734828+0000) 2024-04-12T22:01:38.022 INFO:journalctl@ceph.osd.4.smithi186.stdout:Apr 12 22:01:37 smithi186 podman[123406]: 2024-04-12 22:01:37.803302129 +0000 UTC m=+6.174256465 container cleanup b20cb63b25c9f6653442c9f5146dd3b2af3ffe45af103d11ab22c9991f5e799d (image=quay.ceph.io/ceph-ci/ceph@sha256:f4452def3be4d350f4a8e5feaf7ebb1511be9298fe68478ba7a2d6fbd7d69a63, name=ceph-1cbe407a-f90d-11ee-bc8d-c7b262605968-osd-4, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240409, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-vshankar-testing-20240411.061452-324baf8, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-12T22:01:38.022 INFO:journalctl@ceph.osd.4.smithi186.stdout:Apr 12 22:01:37 smithi186 bash[123406]: ceph-1cbe407a-f90d-11ee-bc8d-c7b262605968-osd-4 2024-04-12T22:01:38.284 INFO:journalctl@ceph.osd.4.smithi186.stdout:Apr 12 22:01:38 smithi186 podman[123427]: 2024-04-12 22:01:38.021101884 +0000 UTC m=+0.682245254 container remove b20cb63b25c9f6653442c9f5146dd3b2af3ffe45af103d11ab22c9991f5e799d (image=quay.ceph.io/ceph-ci/ceph@sha256:f4452def3be4d350f4a8e5feaf7ebb1511be9298fe68478ba7a2d6fbd7d69a63, name=ceph-1cbe407a-f90d-11ee-bc8d-c7b262605968-osd-4, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-vshankar-testing-20240411.061452-324baf8, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240409, org.label-schema.license=GPLv2) 2024-04-12T22:01:38.784 INFO:journalctl@ceph.osd.5.smithi186.stdout:Apr 12 22:01:38 smithi186 ceph-1cbe407a-f90d-11ee-bc8d-c7b262605968-osd-5[105976]: 2024-04-12T22:01:38.524+0000 7f8211600700 -1 osd.5 104 heartbeat_check: no reply from 172.21.15.18:6806 osd.0 since back 2024-04-12T22:00:58.233676+0000 front 2024-04-12T22:00:58.233957+0000 (oldest deadline 2024-04-12T22:01:23.533389+0000) 2024-04-12T22:01:38.784 INFO:journalctl@ceph.osd.5.smithi186.stdout:Apr 12 22:01:38 smithi186 ceph-1cbe407a-f90d-11ee-bc8d-c7b262605968-osd-5[105976]: 2024-04-12T22:01:38.524+0000 7f8211600700 -1 osd.5 104 heartbeat_check: no reply from 172.21.15.18:6814 osd.1 since back 2024-04-12T22:01:03.534222+0000 front 2024-04-12T22:01:03.534417+0000 (oldest deadline 2024-04-12T22:01:34.734828+0000) 2024-04-12T22:01:39.143 INFO:journalctl@ceph.osd.4.smithi186.stdout:Apr 12 22:01:38 smithi186 podman[123515]: 2024-04-12 22:01:38.773008171 +0000 UTC m=+0.032118623 image pull 8717720dc5e5c56303818ea4d4f0a11f6e2c03dee521fb668f411f1304e0afca quay.ceph.io/ceph-ci/ceph@sha256:f4452def3be4d350f4a8e5feaf7ebb1511be9298fe68478ba7a2d6fbd7d69a63 2024-04-12T22:01:39.144 INFO:journalctl@ceph.osd.4.smithi186.stdout:Apr 12 22:01:38 smithi186 podman[123515]: 2024-04-12 22:01:38.871963457 +0000 UTC m=+0.131073940 container create 8b5067236dbe50d3c88f250360e5ae234e69465ff9fbccad6856b5c470117eb4 (image=quay.ceph.io/ceph-ci/ceph@sha256:f4452def3be4d350f4a8e5feaf7ebb1511be9298fe68478ba7a2d6fbd7d69a63, name=ceph-1cbe407a-f90d-11ee-bc8d-c7b262605968-osd-4-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-vshankar-testing-20240411.061452-324baf8, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.build-date=20240409) 2024-04-12T22:01:39.144 INFO:journalctl@ceph.osd.4.smithi186.stdout:Apr 12 22:01:39 smithi186 podman[123515]: 2024-04-12 22:01:39.142449567 +0000 UTC m=+0.401560062 container init 8b5067236dbe50d3c88f250360e5ae234e69465ff9fbccad6856b5c470117eb4 (image=quay.ceph.io/ceph-ci/ceph@sha256:f4452def3be4d350f4a8e5feaf7ebb1511be9298fe68478ba7a2d6fbd7d69a63, name=ceph-1cbe407a-f90d-11ee-bc8d-c7b262605968-osd-4-deactivate, org.label-schema.vendor=CentOS, ceph=True, io.buildah.version=1.33.5, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-vshankar-testing-20240411.061452-324baf8, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240409, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=) 2024-04-12T22:01:39.534 INFO:journalctl@ceph.osd.4.smithi186.stdout:Apr 12 22:01:39 smithi186 podman[123515]: 2024-04-12 22:01:39.154604057 +0000 UTC m=+0.413714519 container start 8b5067236dbe50d3c88f250360e5ae234e69465ff9fbccad6856b5c470117eb4 (image=quay.ceph.io/ceph-ci/ceph@sha256:f4452def3be4d350f4a8e5feaf7ebb1511be9298fe68478ba7a2d6fbd7d69a63, name=ceph-1cbe407a-f90d-11ee-bc8d-c7b262605968-osd-4-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, RELEASE=wip-vshankar-testing-20240411.061452-324baf8, ceph=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.build-date=20240409, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5) 2024-04-12T22:01:39.534 INFO:journalctl@ceph.osd.4.smithi186.stdout:Apr 12 22:01:39 smithi186 podman[123515]: 2024-04-12 22:01:39.240875716 +0000 UTC m=+0.499986211 container attach 8b5067236dbe50d3c88f250360e5ae234e69465ff9fbccad6856b5c470117eb4 (image=quay.ceph.io/ceph-ci/ceph@sha256:f4452def3be4d350f4a8e5feaf7ebb1511be9298fe68478ba7a2d6fbd7d69a63, name=ceph-1cbe407a-f90d-11ee-bc8d-c7b262605968-osd-4-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, RELEASE=wip-vshankar-testing-20240411.061452-324baf8, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.build-date=20240409, maintainer=Guillaume Abrioux ) 2024-04-12T22:01:39.983 INFO:journalctl@ceph.osd.4.smithi186.stdout:Apr 12 22:01:39 smithi186 podman[123515]: 2024-04-12 22:01:39.618886377 +0000 UTC m=+0.877996872 container died 8b5067236dbe50d3c88f250360e5ae234e69465ff9fbccad6856b5c470117eb4 (image=quay.ceph.io/ceph-ci/ceph@sha256:f4452def3be4d350f4a8e5feaf7ebb1511be9298fe68478ba7a2d6fbd7d69a63, name=ceph-1cbe407a-f90d-11ee-bc8d-c7b262605968-osd-4-deactivate, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, org.label-schema.build-date=20240409, RELEASE=wip-vshankar-testing-20240411.061452-324baf8, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS) 2024-04-12T22:01:39.983 INFO:journalctl@ceph.osd.5.smithi186.stdout:Apr 12 22:01:39 smithi186 ceph-1cbe407a-f90d-11ee-bc8d-c7b262605968-osd-5[105976]: 2024-04-12T22:01:39.545+0000 7f8211600700 -1 osd.5 104 heartbeat_check: no reply from 172.21.15.18:6806 osd.0 since back 2024-04-12T22:00:58.233676+0000 front 2024-04-12T22:00:58.233957+0000 (oldest deadline 2024-04-12T22:01:23.533389+0000) 2024-04-12T22:01:39.984 INFO:journalctl@ceph.osd.5.smithi186.stdout:Apr 12 22:01:39 smithi186 ceph-1cbe407a-f90d-11ee-bc8d-c7b262605968-osd-5[105976]: 2024-04-12T22:01:39.545+0000 7f8211600700 -1 osd.5 104 heartbeat_check: no reply from 172.21.15.18:6814 osd.1 since back 2024-04-12T22:01:03.534222+0000 front 2024-04-12T22:01:03.534417+0000 (oldest deadline 2024-04-12T22:01:34.734828+0000) 2024-04-12T22:01:40.397 DEBUG:teuthology.orchestra.run.smithi186:> sudo pkill -f 'journalctl -f -n 0 -u ceph-1cbe407a-f90d-11ee-bc8d-c7b262605968@osd.4.service' 2024-04-12T22:01:40.523 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-12T22:01:40.523 INFO:tasks.cephadm.osd.4:Stopped osd.4 2024-04-12T22:01:40.524 INFO:tasks.cephadm.osd.5:Stopping osd.5... 2024-04-12T22:01:40.524 DEBUG:teuthology.orchestra.run.smithi186:> sudo systemctl stop ceph-1cbe407a-f90d-11ee-bc8d-c7b262605968@osd.5 2024-04-12T22:01:40.654 INFO:journalctl@ceph.osd.5.smithi186.stdout:Apr 12 22:01:40 smithi186 ceph-1cbe407a-f90d-11ee-bc8d-c7b262605968-osd-5[105976]: 2024-04-12T22:01:40.565+0000 7f8211600700 -1 osd.5 104 heartbeat_check: no reply from 172.21.15.18:6806 osd.0 since back 2024-04-12T22:00:58.233676+0000 front 2024-04-12T22:00:58.233957+0000 (oldest deadline 2024-04-12T22:01:23.533389+0000) 2024-04-12T22:01:40.654 INFO:journalctl@ceph.osd.5.smithi186.stdout:Apr 12 22:01:40 smithi186 ceph-1cbe407a-f90d-11ee-bc8d-c7b262605968-osd-5[105976]: 2024-04-12T22:01:40.565+0000 7f8211600700 -1 osd.5 104 heartbeat_check: no reply from 172.21.15.18:6814 osd.1 since back 2024-04-12T22:01:03.534222+0000 front 2024-04-12T22:01:03.534417+0000 (oldest deadline 2024-04-12T22:01:34.734828+0000) 2024-04-12T22:01:41.034 INFO:journalctl@ceph.osd.5.smithi186.stdout:Apr 12 22:01:40 smithi186 systemd[1]: Stopping Ceph osd.5 for 1cbe407a-f90d-11ee-bc8d-c7b262605968... 2024-04-12T22:01:41.534 INFO:journalctl@ceph.osd.5.smithi186.stdout:Apr 12 22:01:41 smithi186 ceph-1cbe407a-f90d-11ee-bc8d-c7b262605968-osd-5[105976]: 2024-04-12T22:01:41.153+0000 7f8215e00700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.5 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-12T22:01:41.534 INFO:journalctl@ceph.osd.5.smithi186.stdout:Apr 12 22:01:41 smithi186 ceph-1cbe407a-f90d-11ee-bc8d-c7b262605968-osd-5[105976]: 2024-04-12T22:01:41.153+0000 7f8215e00700 -1 osd.5 104 *** Got signal Terminated *** 2024-04-12T22:01:41.535 INFO:journalctl@ceph.osd.5.smithi186.stdout:Apr 12 22:01:41 smithi186 ceph-1cbe407a-f90d-11ee-bc8d-c7b262605968-osd-5[105976]: 2024-04-12T22:01:41.153+0000 7f8215e00700 -1 osd.5 104 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-12T22:01:42.034 INFO:journalctl@ceph.osd.5.smithi186.stdout:Apr 12 22:01:41 smithi186 ceph-1cbe407a-f90d-11ee-bc8d-c7b262605968-osd-5[105976]: 2024-04-12T22:01:41.557+0000 7f8211600700 -1 osd.5 104 heartbeat_check: no reply from 172.21.15.18:6806 osd.0 since back 2024-04-12T22:00:58.233676+0000 front 2024-04-12T22:00:58.233957+0000 (oldest deadline 2024-04-12T22:01:23.533389+0000) 2024-04-12T22:01:42.034 INFO:journalctl@ceph.osd.5.smithi186.stdout:Apr 12 22:01:41 smithi186 ceph-1cbe407a-f90d-11ee-bc8d-c7b262605968-osd-5[105976]: 2024-04-12T22:01:41.557+0000 7f8211600700 -1 osd.5 104 heartbeat_check: no reply from 172.21.15.18:6814 osd.1 since back 2024-04-12T22:01:03.534222+0000 front 2024-04-12T22:01:03.534417+0000 (oldest deadline 2024-04-12T22:01:34.734828+0000) 2024-04-12T22:01:43.034 INFO:journalctl@ceph.osd.5.smithi186.stdout:Apr 12 22:01:42 smithi186 ceph-1cbe407a-f90d-11ee-bc8d-c7b262605968-osd-5[105976]: 2024-04-12T22:01:42.601+0000 7f8211600700 -1 osd.5 104 heartbeat_check: no reply from 172.21.15.18:6806 osd.0 since back 2024-04-12T22:00:58.233676+0000 front 2024-04-12T22:00:58.233957+0000 (oldest deadline 2024-04-12T22:01:23.533389+0000) 2024-04-12T22:01:43.034 INFO:journalctl@ceph.osd.5.smithi186.stdout:Apr 12 22:01:42 smithi186 ceph-1cbe407a-f90d-11ee-bc8d-c7b262605968-osd-5[105976]: 2024-04-12T22:01:42.601+0000 7f8211600700 -1 osd.5 104 heartbeat_check: no reply from 172.21.15.18:6814 osd.1 since back 2024-04-12T22:01:03.534222+0000 front 2024-04-12T22:01:03.534417+0000 (oldest deadline 2024-04-12T22:01:34.734828+0000) 2024-04-12T22:01:43.035 INFO:journalctl@ceph.osd.5.smithi186.stdout:Apr 12 22:01:42 smithi186 ceph-1cbe407a-f90d-11ee-bc8d-c7b262605968-osd-5[105976]: 2024-04-12T22:01:42.601+0000 7f8211600700 -1 osd.5 104 heartbeat_check: no reply from 172.21.15.18:6822 osd.2 since back 2024-04-12T22:01:17.636834+0000 front 2024-04-12T22:01:17.636800+0000 (oldest deadline 2024-04-12T22:01:41.736242+0000) 2024-04-12T22:01:44.034 INFO:journalctl@ceph.osd.5.smithi186.stdout:Apr 12 22:01:43 smithi186 ceph-1cbe407a-f90d-11ee-bc8d-c7b262605968-osd-5[105976]: 2024-04-12T22:01:43.582+0000 7f8211600700 -1 osd.5 104 heartbeat_check: no reply from 172.21.15.18:6806 osd.0 since back 2024-04-12T22:00:58.233676+0000 front 2024-04-12T22:00:58.233957+0000 (oldest deadline 2024-04-12T22:01:23.533389+0000) 2024-04-12T22:01:44.035 INFO:journalctl@ceph.osd.5.smithi186.stdout:Apr 12 22:01:43 smithi186 ceph-1cbe407a-f90d-11ee-bc8d-c7b262605968-osd-5[105976]: 2024-04-12T22:01:43.582+0000 7f8211600700 -1 osd.5 104 heartbeat_check: no reply from 172.21.15.18:6814 osd.1 since back 2024-04-12T22:01:03.534222+0000 front 2024-04-12T22:01:03.534417+0000 (oldest deadline 2024-04-12T22:01:34.734828+0000) 2024-04-12T22:01:44.035 INFO:journalctl@ceph.osd.5.smithi186.stdout:Apr 12 22:01:43 smithi186 ceph-1cbe407a-f90d-11ee-bc8d-c7b262605968-osd-5[105976]: 2024-04-12T22:01:43.582+0000 7f8211600700 -1 osd.5 104 heartbeat_check: no reply from 172.21.15.18:6822 osd.2 since back 2024-04-12T22:01:17.636834+0000 front 2024-04-12T22:01:17.636800+0000 (oldest deadline 2024-04-12T22:01:41.736242+0000) 2024-04-12T22:01:45.034 INFO:journalctl@ceph.osd.5.smithi186.stdout:Apr 12 22:01:44 smithi186 ceph-1cbe407a-f90d-11ee-bc8d-c7b262605968-osd-5[105976]: 2024-04-12T22:01:44.591+0000 7f8211600700 -1 osd.5 104 heartbeat_check: no reply from 172.21.15.18:6806 osd.0 since back 2024-04-12T22:00:58.233676+0000 front 2024-04-12T22:00:58.233957+0000 (oldest deadline 2024-04-12T22:01:23.533389+0000) 2024-04-12T22:01:45.034 INFO:journalctl@ceph.osd.5.smithi186.stdout:Apr 12 22:01:44 smithi186 ceph-1cbe407a-f90d-11ee-bc8d-c7b262605968-osd-5[105976]: 2024-04-12T22:01:44.591+0000 7f8211600700 -1 osd.5 104 heartbeat_check: no reply from 172.21.15.18:6814 osd.1 since back 2024-04-12T22:01:03.534222+0000 front 2024-04-12T22:01:03.534417+0000 (oldest deadline 2024-04-12T22:01:34.734828+0000) 2024-04-12T22:01:45.034 INFO:journalctl@ceph.osd.5.smithi186.stdout:Apr 12 22:01:44 smithi186 ceph-1cbe407a-f90d-11ee-bc8d-c7b262605968-osd-5[105976]: 2024-04-12T22:01:44.591+0000 7f8211600700 -1 osd.5 104 heartbeat_check: no reply from 172.21.15.18:6822 osd.2 since back 2024-04-12T22:01:17.636834+0000 front 2024-04-12T22:01:17.636800+0000 (oldest deadline 2024-04-12T22:01:41.736242+0000) 2024-04-12T22:01:46.035 INFO:journalctl@ceph.osd.5.smithi186.stdout:Apr 12 22:01:45 smithi186 ceph-1cbe407a-f90d-11ee-bc8d-c7b262605968-osd-5[105976]: 2024-04-12T22:01:45.546+0000 7f8211600700 -1 osd.5 104 heartbeat_check: no reply from 172.21.15.18:6806 osd.0 since back 2024-04-12T22:00:58.233676+0000 front 2024-04-12T22:00:58.233957+0000 (oldest deadline 2024-04-12T22:01:23.533389+0000) 2024-04-12T22:01:46.035 INFO:journalctl@ceph.osd.5.smithi186.stdout:Apr 12 22:01:45 smithi186 ceph-1cbe407a-f90d-11ee-bc8d-c7b262605968-osd-5[105976]: 2024-04-12T22:01:45.546+0000 7f8211600700 -1 osd.5 104 heartbeat_check: no reply from 172.21.15.18:6814 osd.1 since back 2024-04-12T22:01:03.534222+0000 front 2024-04-12T22:01:03.534417+0000 (oldest deadline 2024-04-12T22:01:34.734828+0000) 2024-04-12T22:01:46.035 INFO:journalctl@ceph.osd.5.smithi186.stdout:Apr 12 22:01:45 smithi186 ceph-1cbe407a-f90d-11ee-bc8d-c7b262605968-osd-5[105976]: 2024-04-12T22:01:45.546+0000 7f8211600700 -1 osd.5 104 heartbeat_check: no reply from 172.21.15.18:6822 osd.2 since back 2024-04-12T22:01:17.636834+0000 front 2024-04-12T22:01:17.636800+0000 (oldest deadline 2024-04-12T22:01:41.736242+0000) 2024-04-12T22:01:46.784 INFO:journalctl@ceph.osd.5.smithi186.stdout:Apr 12 22:01:46 smithi186 podman[123621]: 2024-04-12 22:01:46.531487354 +0000 UTC m=+5.499809407 container died 2513bea2610e64f5e1b138f0de68b834e0764f2534134d402ed08f2ada5b32b6 (image=quay.ceph.io/ceph-ci/ceph@sha256:f4452def3be4d350f4a8e5feaf7ebb1511be9298fe68478ba7a2d6fbd7d69a63, name=ceph-1cbe407a-f90d-11ee-bc8d-c7b262605968-osd-5, RELEASE=wip-vshankar-testing-20240411.061452-324baf8, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240409, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, ceph=True, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux ) 2024-04-12T22:01:47.091 INFO:journalctl@ceph.osd.5.smithi186.stdout:Apr 12 22:01:46 smithi186 podman[123621]: 2024-04-12 22:01:46.913555975 +0000 UTC m=+5.881878022 container cleanup 2513bea2610e64f5e1b138f0de68b834e0764f2534134d402ed08f2ada5b32b6 (image=quay.ceph.io/ceph-ci/ceph@sha256:f4452def3be4d350f4a8e5feaf7ebb1511be9298fe68478ba7a2d6fbd7d69a63, name=ceph-1cbe407a-f90d-11ee-bc8d-c7b262605968-osd-5, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-vshankar-testing-20240411.061452-324baf8, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, ceph=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240409) 2024-04-12T22:01:47.091 INFO:journalctl@ceph.osd.5.smithi186.stdout:Apr 12 22:01:46 smithi186 bash[123621]: ceph-1cbe407a-f90d-11ee-bc8d-c7b262605968-osd-5 2024-04-12T22:01:47.534 INFO:journalctl@ceph.osd.5.smithi186.stdout:Apr 12 22:01:47 smithi186 podman[123640]: 2024-04-12 22:01:47.089562251 +0000 UTC m=+0.541478609 container remove 2513bea2610e64f5e1b138f0de68b834e0764f2534134d402ed08f2ada5b32b6 (image=quay.ceph.io/ceph-ci/ceph@sha256:f4452def3be4d350f4a8e5feaf7ebb1511be9298fe68478ba7a2d6fbd7d69a63, name=ceph-1cbe407a-f90d-11ee-bc8d-c7b262605968-osd-5, RELEASE=wip-vshankar-testing-20240411.061452-324baf8, GIT_BRANCH=HEAD, ceph=True, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240409) 2024-04-12T22:01:48.212 INFO:journalctl@ceph.osd.5.smithi186.stdout:Apr 12 22:01:47 smithi186 podman[123725]: 2024-04-12 22:01:47.795865303 +0000 UTC m=+0.048601823 image pull 8717720dc5e5c56303818ea4d4f0a11f6e2c03dee521fb668f411f1304e0afca quay.ceph.io/ceph-ci/ceph@sha256:f4452def3be4d350f4a8e5feaf7ebb1511be9298fe68478ba7a2d6fbd7d69a63 2024-04-12T22:01:48.212 INFO:journalctl@ceph.osd.5.smithi186.stdout:Apr 12 22:01:47 smithi186 podman[123725]: 2024-04-12 22:01:47.940483773 +0000 UTC m=+0.193220250 container create aa0eb8466c55834e35aa283e7e93798b0e4d56038386f404e82d9f29f5f8d531 (image=quay.ceph.io/ceph-ci/ceph@sha256:f4452def3be4d350f4a8e5feaf7ebb1511be9298fe68478ba7a2d6fbd7d69a63, name=ceph-1cbe407a-f90d-11ee-bc8d-c7b262605968-osd-5-deactivate, RELEASE=wip-vshankar-testing-20240411.061452-324baf8, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, ceph=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240409, GIT_CLEAN=True, io.buildah.version=1.33.5) 2024-04-12T22:01:48.212 INFO:journalctl@ceph.osd.5.smithi186.stdout:Apr 12 22:01:48 smithi186 podman[123725]: 2024-04-12 22:01:48.211289541 +0000 UTC m=+0.464026061 container init aa0eb8466c55834e35aa283e7e93798b0e4d56038386f404e82d9f29f5f8d531 (image=quay.ceph.io/ceph-ci/ceph@sha256:f4452def3be4d350f4a8e5feaf7ebb1511be9298fe68478ba7a2d6fbd7d69a63, name=ceph-1cbe407a-f90d-11ee-bc8d-c7b262605968-osd-5-deactivate, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240409, org.label-schema.license=GPLv2, GIT_CLEAN=True, RELEASE=wip-vshankar-testing-20240411.061452-324baf8, GIT_BRANCH=HEAD, ceph=True) 2024-04-12T22:01:48.534 INFO:journalctl@ceph.osd.5.smithi186.stdout:Apr 12 22:01:48 smithi186 podman[123725]: 2024-04-12 22:01:48.223456305 +0000 UTC m=+0.476192753 container start aa0eb8466c55834e35aa283e7e93798b0e4d56038386f404e82d9f29f5f8d531 (image=quay.ceph.io/ceph-ci/ceph@sha256:f4452def3be4d350f4a8e5feaf7ebb1511be9298fe68478ba7a2d6fbd7d69a63, name=ceph-1cbe407a-f90d-11ee-bc8d-c7b262605968-osd-5-deactivate, GIT_CLEAN=True, org.label-schema.build-date=20240409, org.label-schema.license=GPLv2, ceph=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, RELEASE=wip-vshankar-testing-20240411.061452-324baf8, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-12T22:01:48.534 INFO:journalctl@ceph.osd.5.smithi186.stdout:Apr 12 22:01:48 smithi186 podman[123725]: 2024-04-12 22:01:48.301541349 +0000 UTC m=+0.554277834 container attach aa0eb8466c55834e35aa283e7e93798b0e4d56038386f404e82d9f29f5f8d531 (image=quay.ceph.io/ceph-ci/ceph@sha256:f4452def3be4d350f4a8e5feaf7ebb1511be9298fe68478ba7a2d6fbd7d69a63, name=ceph-1cbe407a-f90d-11ee-bc8d-c7b262605968-osd-5-deactivate, org.label-schema.build-date=20240409, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, GIT_CLEAN=True, RELEASE=wip-vshankar-testing-20240411.061452-324baf8, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-12T22:01:48.952 INFO:journalctl@ceph.osd.5.smithi186.stdout:Apr 12 22:01:48 smithi186 podman[123725]: 2024-04-12 22:01:48.661007319 +0000 UTC m=+0.913743759 container died aa0eb8466c55834e35aa283e7e93798b0e4d56038386f404e82d9f29f5f8d531 (image=quay.ceph.io/ceph-ci/ceph@sha256:f4452def3be4d350f4a8e5feaf7ebb1511be9298fe68478ba7a2d6fbd7d69a63, name=ceph-1cbe407a-f90d-11ee-bc8d-c7b262605968-osd-5-deactivate, io.buildah.version=1.33.5, GIT_CLEAN=True, RELEASE=wip-vshankar-testing-20240411.061452-324baf8, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240409) 2024-04-12T22:01:49.302 DEBUG:teuthology.orchestra.run.smithi186:> sudo pkill -f 'journalctl -f -n 0 -u ceph-1cbe407a-f90d-11ee-bc8d-c7b262605968@osd.5.service' 2024-04-12T22:01:49.420 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-12T22:01:49.420 INFO:tasks.cephadm.osd.5:Stopped osd.5 2024-04-12T22:01:49.420 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 1cbe407a-f90d-11ee-bc8d-c7b262605968 --force --keep-logs 2024-04-12T22:01:55.126 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi018.stderr:ceph-fuse[76022]: fuse finished with error 0 and tester_r 0 2024-04-12T22:02:17.132 DEBUG:teuthology.orchestra.run.smithi186:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 1cbe407a-f90d-11ee-bc8d-c7b262605968 --force --keep-logs 2024-04-12T22:02:34.910 DEBUG:teuthology.orchestra.run.smithi018:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-12T22:02:35.025 DEBUG:teuthology.orchestra.run.smithi186:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-12T22:02:35.130 INFO:tasks.cephadm:Archiving crash dumps... 2024-04-12T22:02:35.132 DEBUG:teuthology.misc:Transferring archived files from smithi018:/var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/crash to /home/teuthworker/archive/vshankar-2024-04-12_03:11:38-fs-wip-vshankar-testing-20240411.061452-testing-default-smithi/7653578/remote/smithi018/crash 2024-04-12T22:02:35.133 DEBUG:teuthology.orchestra.run.smithi018:> sudo tar c -f - -C /var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/crash -- . 2024-04-12T22:02:35.238 INFO:teuthology.orchestra.run.smithi018.stderr:tar: /var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/crash: Cannot open: No such file or directory 2024-04-12T22:02:35.238 INFO:teuthology.orchestra.run.smithi018.stderr:tar: Error is not recoverable: exiting now 2024-04-12T22:02:35.243 DEBUG:teuthology.misc:Transferring archived files from smithi186:/var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/crash to /home/teuthworker/archive/vshankar-2024-04-12_03:11:38-fs-wip-vshankar-testing-20240411.061452-testing-default-smithi/7653578/remote/smithi186/crash 2024-04-12T22:02:35.244 DEBUG:teuthology.orchestra.run.smithi186:> sudo tar c -f - -C /var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/crash -- . 2024-04-12T22:02:35.342 INFO:teuthology.orchestra.run.smithi186.stderr:tar: /var/lib/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/crash: Cannot open: No such file or directory 2024-04-12T22:02:35.342 INFO:teuthology.orchestra.run.smithi186.stderr:tar: Error is not recoverable: exiting now 2024-04-12T22:02:35.346 INFO:tasks.cephadm:Checking cluster log for badness... 2024-04-12T22:02:35.346 DEBUG:teuthology.orchestra.run.smithi018:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v OSD_DOWN | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'Reduced data availability' | head -n 1 2024-04-12T22:02:35.462 INFO:teuthology.orchestra.run.smithi018.stdout:2024-04-12T21:10:00.000164+0000 mon.smithi018 (mon.0) 650 : cluster [ERR] Health detail: HEALTH_ERR 1 filesystem is degraded; 1 filesystem is offline 2024-04-12T22:02:35.463 WARNING:tasks.cephadm:Found errors (ERR|WRN|SEC) in cluster log 2024-04-12T22:02:35.463 INFO:tasks.cephadm:Compressing logs... 2024-04-12T22:02:35.463 DEBUG:teuthology.orchestra.run.smithi018:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-04-12T22:02:35.467 DEBUG:teuthology.orchestra.run.smithi186:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-04-12T22:02:35.539 INFO:teuthology.orchestra.run.smithi018.stderr:find: ‘/var/log/rbd-target-api’: No such file or directory 2024-04-12T22:02:35.539 INFO:teuthology.orchestra.run.smithi018.stderr:gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-04-12T22:02:35.540 INFO:teuthology.orchestra.run.smithi018.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.log 2024-04-12T22:02:35.541 INFO:teuthology.orchestra.run.smithi018.stderr:/var/log/ceph/cephadm.log: gzip -5 --verbose -- /var/log/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/ceph-mds.cephfs.smithi018.acqbsc.log 2024-04-12T22:02:35.543 INFO:teuthology.orchestra.run.smithi018.stderr:/var/log/ceph/ceph-client.0.log: gzip -5 --verbose -- /var/log/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/ceph.audit.log 2024-04-12T22:02:35.544 INFO:teuthology.orchestra.run.smithi018.stderr:/var/log/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/ceph-mds.cephfs.smithi018.acqbsc.log: gzip -5 --verbose -- /var/log/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/ceph-mgr.smithi018.xjzwga.log 2024-04-12T22:02:35.545 INFO:teuthology.orchestra.run.smithi018.stderr:/var/log/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/ceph-osd.1.log 2024-04-12T22:02:35.547 INFO:teuthology.orchestra.run.smithi018.stderr:/var/log/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/ceph-mgr.smithi018.xjzwga.log: gzip -5 --verbose -- /var/log/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/ceph-mds.cephfs.smithi018.szjluo.log 2024-04-12T22:02:35.549 INFO:teuthology.orchestra.run.smithi018.stderr:/var/log/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/ceph-osd.0.log 2024-04-12T22:02:35.551 INFO:teuthology.orchestra.run.smithi018.stderr:/var/log/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/ceph-mds.cephfs.smithi018.szjluo.log: gzip -5 --verbose -- /var/log/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/ceph.cephadm.log 2024-04-12T22:02:35.554 INFO:teuthology.orchestra.run.smithi018.stderr:/var/log/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/ceph-osd.0.log: 91.0% -- replaced with /var/log/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/ceph.audit.log.gz 2024-04-12T22:02:35.556 INFO:teuthology.orchestra.run.smithi018.stderr:/var/log/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/ceph.cephadm.log: 85.2% -- replaced with /var/log/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/ceph.cephadm.log.gz 2024-04-12T22:02:35.556 INFO:teuthology.orchestra.run.smithi018.stderr:gzip -5 --verbose -- /var/log/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/ceph.log 2024-04-12T22:02:35.558 INFO:teuthology.orchestra.run.smithi018.stderr:gzip -5 --verbose -- /var/log/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/ceph-mon.smithi018.log 2024-04-12T22:02:35.561 INFO:teuthology.orchestra.run.smithi186.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-04-12T22:02:35.561 INFO:teuthology.orchestra.run.smithi186.stderr:: No such file or directory 2024-04-12T22:02:35.562 INFO:teuthology.orchestra.run.smithi186.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.1.log 2024-04-12T22:02:35.563 INFO:teuthology.orchestra.run.smithi186.stderr:/var/log/ceph/cephadm.log: gzip -5 --verbose -- /var/log/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/ceph-mgr.smithi186.kqcfxk.log 2024-04-12T22:02:35.564 INFO:teuthology.orchestra.run.smithi186.stderr:/var/log/ceph/ceph-client.1.log: gzip -5 --verbose -- /var/log/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/ceph-mds.cephfs.smithi186.wjnafs.log 2024-04-12T22:02:35.565 INFO:teuthology.orchestra.run.smithi018.stderr:/var/log/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/ceph.log: /var/log/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/ceph-mon.smithi018.log: gzip -5 --verbose -- /var/log/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/ceph-osd.2.log 2024-04-12T22:02:35.565 INFO:teuthology.orchestra.run.smithi186.stderr:/var/log/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/ceph-mgr.smithi186.kqcfxk.log: gzip -5 --verbose -- /var/log/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/ceph.audit.log 2024-04-12T22:02:35.566 INFO:teuthology.orchestra.run.smithi186.stderr: 92.1%/var/log/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/ceph-mds.cephfs.smithi186.wjnafs.log: -- replaced with /var/log/ceph/cephadm.log.gz 2024-04-12T22:02:35.567 INFO:teuthology.orchestra.run.smithi186.stderr:gzip -5 --verbose -- /var/log/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/ceph-osd.4.log 2024-04-12T22:02:35.568 INFO:teuthology.orchestra.run.smithi018.stderr:/var/log/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/ceph-osd.2.log: 89.7% -- replaced with /var/log/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/ceph.log.gz 2024-04-12T22:02:35.569 INFO:teuthology.orchestra.run.smithi186.stderr:/var/log/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/ceph-osd.5.log 2024-04-12T22:02:35.569 INFO:teuthology.orchestra.run.smithi018.stderr:gzip -5 --verbose -- /var/log/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/ceph-volume.log 2024-04-12T22:02:35.570 INFO:teuthology.orchestra.run.smithi186.stderr:/var/log/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/ceph-mon.smithi186.log 2024-04-12T22:02:35.571 INFO:teuthology.orchestra.run.smithi186.stderr:/var/log/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/ceph-osd.3.log 2024-04-12T22:02:35.577 INFO:teuthology.orchestra.run.smithi186.stderr:/var/log/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/ceph-mon.smithi186.log: /var/log/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/ceph-osd.3.log: gzip -5 --verbose -- /var/log/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/ceph.cephadm.log 2024-04-12T22:02:35.578 INFO:teuthology.orchestra.run.smithi186.stderr: 91.2%gzip -5 --verbose -- /var/log/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/ceph.log 2024-04-12T22:02:35.578 INFO:teuthology.orchestra.run.smithi186.stderr: -- replaced with /var/log/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/ceph.audit.log.gz 2024-04-12T22:02:35.580 INFO:teuthology.orchestra.run.smithi186.stderr:/var/log/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/ceph.cephadm.log: 85.1% -- replaced with /var/log/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/ceph.cephadm.log.gz 2024-04-12T22:02:35.585 INFO:teuthology.orchestra.run.smithi186.stderr:gzip -5 --verbose -- /var/log/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/ceph-volume.log 2024-04-12T22:02:35.585 INFO:teuthology.orchestra.run.smithi186.stderr:/var/log/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/ceph.log: 94.6% -- replaced with /var/log/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/ceph-mds.cephfs.smithi186.wjnafs.log.gz 2024-04-12T22:02:35.586 INFO:teuthology.orchestra.run.smithi186.stderr:gzip -5 --verbose -- /var/log/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/ceph-mds.cephfs.smithi186.jdllyn.log 2024-04-12T22:02:35.590 INFO:teuthology.orchestra.run.smithi018.stderr:/var/log/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/ceph-volume.log: 92.0% -- replaced with /var/log/ceph/cephadm.log.gz 2024-04-12T22:02:35.600 INFO:teuthology.orchestra.run.smithi186.stderr:/var/log/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/ceph-volume.log: /var/log/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/ceph-mds.cephfs.smithi186.jdllyn.log: 89.8% -- replaced with /var/log/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/ceph.log.gz 2024-04-12T22:02:35.615 INFO:teuthology.orchestra.run.smithi186.stderr: 90.9% -- replaced with /var/log/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/ceph-volume.log.gz 2024-04-12T22:02:35.621 INFO:teuthology.orchestra.run.smithi186.stderr: 93.4% -- replaced with /var/log/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/ceph-mds.cephfs.smithi186.jdllyn.log.gz 2024-04-12T22:02:35.634 INFO:teuthology.orchestra.run.smithi018.stderr: 90.9% -- replaced with /var/log/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/ceph-volume.log.gz 2024-04-12T22:02:35.660 INFO:teuthology.orchestra.run.smithi186.stderr: 90.4% -- replaced with /var/log/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/ceph-mgr.smithi186.kqcfxk.log.gz 2024-04-12T22:02:36.329 INFO:teuthology.orchestra.run.smithi186.stderr: 92.6% -- replaced with /var/log/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/ceph-mon.smithi186.log.gz 2024-04-12T22:02:36.444 INFO:teuthology.orchestra.run.smithi018.stderr: 89.8% -- replaced with /var/log/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/ceph-mgr.smithi018.xjzwga.log.gz 2024-04-12T22:02:38.958 INFO:teuthology.orchestra.run.smithi018.stderr: 89.5% -- replaced with /var/log/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/ceph-mon.smithi018.log.gz 2024-04-12T22:02:41.067 INFO:teuthology.orchestra.run.smithi186.stderr: 93.1% -- replaced with /var/log/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/ceph-osd.4.log.gz 2024-04-12T22:02:43.160 INFO:teuthology.orchestra.run.smithi186.stderr: 93.3% -- replaced with /var/log/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/ceph-osd.5.log.gz 2024-04-12T22:02:43.678 INFO:teuthology.orchestra.run.smithi186.stderr: 93.0% -- replaced with /var/log/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/ceph-osd.3.log.gz 2024-04-12T22:02:44.037 INFO:teuthology.orchestra.run.smithi018.stderr: 93.3% -- replaced with /var/log/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/ceph-osd.2.log.gz 2024-04-12T22:02:44.836 INFO:teuthology.orchestra.run.smithi018.stderr: 93.3% -- replaced with /var/log/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/ceph-osd.0.log.gz 2024-04-12T22:02:46.372 INFO:teuthology.orchestra.run.smithi018.stderr: 92.9% -- replaced with /var/log/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/ceph-osd.1.log.gz 2024-04-12T22:02:50.261 INFO:teuthology.orchestra.run.smithi186.stderr:gzip: /var/log/ceph/ceph-client.1.log: file size changed while zipping 2024-04-12T22:02:50.261 INFO:teuthology.orchestra.run.smithi186.stderr: 92.9% -- replaced with /var/log/ceph/ceph-client.1.log.gz 2024-04-12T22:02:50.267 INFO:teuthology.orchestra.run.smithi186.stderr: 2024-04-12T22:02:50.267 INFO:teuthology.orchestra.run.smithi186.stderr:real 0m14.746s 2024-04-12T22:02:50.267 INFO:teuthology.orchestra.run.smithi186.stderr:user 0m34.517s 2024-04-12T22:02:50.267 INFO:teuthology.orchestra.run.smithi186.stderr:sys 0m2.425s 2024-04-12T22:02:54.580 INFO:teuthology.orchestra.run.smithi018.stderr: 92.9% -- replaced with /var/log/ceph/ceph-client.0.log.gz 2024-04-12T22:03:27.293 INFO:teuthology.orchestra.run.smithi018.stderr: 93.4% -- replaced with /var/log/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/ceph-mds.cephfs.smithi018.szjluo.log.gz 2024-04-12T22:03:28.959 INFO:teuthology.orchestra.run.smithi018.stderr: 93.4% -- replaced with /var/log/ceph/1cbe407a-f90d-11ee-bc8d-c7b262605968/ceph-mds.cephfs.smithi018.acqbsc.log.gz 2024-04-12T22:03:28.962 INFO:teuthology.orchestra.run.smithi018.stderr: 2024-04-12T22:03:28.963 INFO:teuthology.orchestra.run.smithi018.stderr:real 0m53.463s 2024-04-12T22:03:28.963 INFO:teuthology.orchestra.run.smithi018.stderr:user 2m26.104s 2024-04-12T22:03:28.963 INFO:teuthology.orchestra.run.smithi018.stderr:sys 0m10.763s 2024-04-12T22:03:28.964 INFO:tasks.cephadm:Archiving logs... 2024-04-12T22:03:28.964 DEBUG:teuthology.misc:Transferring archived files from smithi018:/var/log/ceph to /home/teuthworker/archive/vshankar-2024-04-12_03:11:38-fs-wip-vshankar-testing-20240411.061452-testing-default-smithi/7653578/remote/smithi018/log 2024-04-12T22:03:28.966 DEBUG:teuthology.orchestra.run.smithi018:> sudo tar c -f - -C /var/log/ceph -- . 2024-04-12T22:03:48.079 DEBUG:teuthology.misc:Transferring archived files from smithi186:/var/log/ceph to /home/teuthworker/archive/vshankar-2024-04-12_03:11:38-fs-wip-vshankar-testing-20240411.061452-testing-default-smithi/7653578/remote/smithi186/log 2024-04-12T22:03:48.080 DEBUG:teuthology.orchestra.run.smithi186:> sudo tar c -f - -C /var/log/ceph -- . 2024-04-12T22:03:53.617 INFO:tasks.cephadm:Removing cluster... 2024-04-12T22:03:53.617 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 1cbe407a-f90d-11ee-bc8d-c7b262605968 --force 2024-04-12T22:03:55.105 DEBUG:teuthology.orchestra.run.smithi186:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 1cbe407a-f90d-11ee-bc8d-c7b262605968 --force 2024-04-12T22:03:56.178 INFO:tasks.cephadm:Removing cephadm ... 2024-04-12T22:03:56.178 DEBUG:teuthology.orchestra.run.smithi018:> rm -rf /home/ubuntu/cephtest/cephadm 2024-04-12T22:03:56.240 DEBUG:teuthology.orchestra.run.smithi186:> rm -rf /home/ubuntu/cephtest/cephadm 2024-04-12T22:03:56.297 INFO:tasks.cephadm:Teardown complete 2024-04-12T22:03:56.297 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-04-12T22:03:56.311 ERROR:teuthology.contextutil:Saw exception from nested tasks Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_6c637841c215537a4502385240412f1966e0faab/teuthology/contextutil.py", line 32, in nested yield vars File "/home/teuthworker/src/git.ceph.com_teuthology_6c637841c215537a4502385240412f1966e0faab/teuthology/task/install/__init__.py", line 626, in task yield File "/home/teuthworker/src/git.ceph.com_teuthology_6c637841c215537a4502385240412f1966e0faab/teuthology/run_tasks.py", line 154, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ next(self.gen) File "/home/teuthworker/src/git.ceph.com_ceph-c_324baf8ddb91dea46cac50cb535c306115791355/qa/tasks/ceph_fuse.py", line 183, in task mount.umount_wait() File "/home/teuthworker/src/git.ceph.com_ceph-c_324baf8ddb91dea46cac50cb535c306115791355/qa/tasks/cephfs/fuse_mount.py", line 403, in umount_wait run.wait([self.fuse_daemon], timeout) File "/home/teuthworker/src/git.ceph.com_teuthology_6c637841c215537a4502385240412f1966e0faab/teuthology/orchestra/run.py", line 473, in wait check_time() File "/home/teuthworker/src/git.ceph.com_teuthology_6c637841c215537a4502385240412f1966e0faab/teuthology/contextutil.py", line 134, in __call__ raise MaxWhileTries(error_msg) teuthology.exceptions.MaxWhileTries: reached maximum tries (51) after waiting for 300 seconds 2024-04-12T22:03:56.312 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-04-12T22:03:56.312 DEBUG:teuthology.orchestra.run.smithi018:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-04-12T22:03:56.320 DEBUG:teuthology.orchestra.run.smithi186:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-04-12T22:03:56.469 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-04-12T22:03:56.485 INFO:teuthology.task.clock:Checking final clock skew... 2024-04-12T22:03:56.486 DEBUG:teuthology.orchestra.run.smithi018:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-12T22:03:56.494 DEBUG:teuthology.orchestra.run.smithi186:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-12T22:03:56.543 INFO:teuthology.orchestra.run.smithi018.stderr:bash: line 1: ntpq: command not found 2024-04-12T22:03:56.548 INFO:teuthology.orchestra.run.smithi186.stderr:bash: line 1: ntpq: command not found 2024-04-12T22:03:56.553 INFO:teuthology.orchestra.run.smithi018.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-12T22:03:56.554 INFO:teuthology.orchestra.run.smithi018.stdout:=============================================================================== 2024-04-12T22:03:56.554 INFO:teuthology.orchestra.run.smithi018.stdout:^- hv01.front.sepia.ceph.com 2 7 377 35 +1125us[+1125us] +/- 358ms 2024-04-12T22:03:56.554 INFO:teuthology.orchestra.run.smithi018.stdout:^* hv02.front.sepia.ceph.com 3 7 377 40 -469us[ -466us] +/- 52ms 2024-04-12T22:03:56.554 INFO:teuthology.orchestra.run.smithi018.stdout:^+ hv03.front.sepia.ceph.com 3 7 377 36 +539us[ +539us] +/- 62ms 2024-04-12T22:03:56.554 INFO:teuthology.orchestra.run.smithi018.stdout:^? hv04.front.sepia.ceph.com 0 10 0 - +0ns[ +0ns] +/- 0ns 2024-04-12T22:03:56.558 INFO:teuthology.orchestra.run.smithi186.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-12T22:03:56.558 INFO:teuthology.orchestra.run.smithi186.stdout:=============================================================================== 2024-04-12T22:03:56.558 INFO:teuthology.orchestra.run.smithi186.stdout:^- hv01.front.sepia.ceph.com 2 7 377 40 +1154us[+1154us] +/- 358ms 2024-04-12T22:03:56.558 INFO:teuthology.orchestra.run.smithi186.stdout:^* hv02.front.sepia.ceph.com 3 8 377 100 -374us[ -382us] +/- 51ms 2024-04-12T22:03:56.558 INFO:teuthology.orchestra.run.smithi186.stdout:^+ hv03.front.sepia.ceph.com 3 8 377 167 +621us[ +612us] +/- 60ms 2024-04-12T22:03:56.558 INFO:teuthology.orchestra.run.smithi186.stdout:^? hv04.front.sepia.ceph.com 0 10 0 - +0ns[ +0ns] +/- 0ns 2024-04-12T22:03:56.560 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-04-12T22:03:56.572 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-04-12T22:03:56.573 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-04-12T22:03:56.586 DEBUG:teuthology.orchestra.run.smithi018:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-04-12T22:03:56.852 DEBUG:teuthology.orchestra.run.smithi186:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-04-12T22:03:57.079 DEBUG:teuthology.orchestra.run.smithi018:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-12T22:03:57.193 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-12T22:03:57.194 DEBUG:teuthology.orchestra.run.smithi186:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-12T22:03:57.298 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-12T22:03:57.299 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-04-12T22:03:57.311 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-04-12T22:03:57.328 INFO:teuthology.task.internal:Duration was 5469.626302 seconds 2024-04-12T22:03:57.328 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-04-12T22:03:57.345 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-04-12T22:03:57.345 DEBUG:teuthology.orchestra.run.smithi018:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-04-12T22:03:57.353 DEBUG:teuthology.orchestra.run.smithi186:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-04-12T22:03:57.521 INFO:teuthology.orchestra.run.smithi186.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-12T22:03:57.528 INFO:teuthology.orchestra.run.smithi018.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-12T22:03:57.950 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-04-12T22:03:57.950 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi018.front.sepia.ceph.com 2024-04-12T22:03:57.950 DEBUG:teuthology.orchestra.run.smithi018:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-04-12T22:03:58.020 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi186.front.sepia.ceph.com 2024-04-12T22:03:58.021 DEBUG:teuthology.orchestra.run.smithi186:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-04-12T22:03:58.095 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-04-12T22:03:58.095 DEBUG:teuthology.orchestra.run.smithi018:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-04-12T22:03:58.103 DEBUG:teuthology.orchestra.run.smithi186:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-04-12T22:03:58.203 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-04-12T22:03:58.204 DEBUG:teuthology.orchestra.run.smithi018:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-04-12T22:03:58.209 DEBUG:teuthology.orchestra.run.smithi186:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-04-12T22:03:59.841 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-04-12T22:03:59.855 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-04-12T22:03:59.856 DEBUG:teuthology.orchestra.run.smithi018:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-04-12T22:03:59.953 DEBUG:teuthology.orchestra.run.smithi186:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-04-12T22:04:00.054 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-04-12T22:04:00.069 DEBUG:teuthology.orchestra.run.smithi018:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-04-12T22:04:00.077 DEBUG:teuthology.orchestra.run.smithi186:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-04-12T22:04:00.185 INFO:teuthology.orchestra.run.smithi018.stdout:kernel.core_pattern = core 2024-04-12T22:04:00.189 INFO:teuthology.orchestra.run.smithi186.stdout:kernel.core_pattern = core 2024-04-12T22:04:00.261 DEBUG:teuthology.orchestra.run.smithi018:> test -e /home/ubuntu/cephtest/archive/coredump 2024-04-12T22:04:00.318 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-12T22:04:00.318 DEBUG:teuthology.orchestra.run.smithi186:> test -e /home/ubuntu/cephtest/archive/coredump 2024-04-12T22:04:00.372 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-12T22:04:00.373 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-04-12T22:04:00.385 INFO:teuthology.task.internal:Transferring archived files... 2024-04-12T22:04:00.387 DEBUG:teuthology.misc:Transferring archived files from smithi018:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/vshankar-2024-04-12_03:11:38-fs-wip-vshankar-testing-20240411.061452-testing-default-smithi/7653578/remote/smithi018 2024-04-12T22:04:00.387 DEBUG:teuthology.orchestra.run.smithi018:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-04-12T22:04:00.537 DEBUG:teuthology.misc:Transferring archived files from smithi186:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/vshankar-2024-04-12_03:11:38-fs-wip-vshankar-testing-20240411.061452-testing-default-smithi/7653578/remote/smithi186 2024-04-12T22:04:00.537 DEBUG:teuthology.orchestra.run.smithi186:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-04-12T22:04:00.650 INFO:teuthology.task.internal:Removing archive directory... 2024-04-12T22:04:00.650 DEBUG:teuthology.orchestra.run.smithi018:> rm -rf -- /home/ubuntu/cephtest/archive 2024-04-12T22:04:00.658 DEBUG:teuthology.orchestra.run.smithi186:> rm -rf -- /home/ubuntu/cephtest/archive 2024-04-12T22:04:00.714 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-04-12T22:04:00.731 INFO:teuthology.task.internal:Not uploading archives. 2024-04-12T22:04:00.731 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-04-12T22:04:00.748 INFO:teuthology.task.internal:Tidying up after the test... 2024-04-12T22:04:00.749 DEBUG:teuthology.orchestra.run.smithi018:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-04-12T22:04:00.756 DEBUG:teuthology.orchestra.run.smithi186:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-04-12T22:04:00.809 INFO:teuthology.orchestra.run.smithi018.stdout: 394535 4 drwxr-xr-x 3 ubuntu ubuntu 4096 Apr 12 22:04 /home/ubuntu/cephtest 2024-04-12T22:04:00.809 INFO:teuthology.orchestra.run.smithi018.stdout: 787518 4 d--------- 2 ubuntu ubuntu 4096 Apr 12 20:52 /home/ubuntu/cephtest/mnt.0 2024-04-12T22:04:00.809 INFO:teuthology.orchestra.run.smithi018.stderr:find: ‘/home/ubuntu/cephtest/mnt.0’: Permission denied 2024-04-12T22:04:00.812 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-12T22:04:00.812 INFO:teuthology.orchestra.run.smithi018.stderr:rmdir: failed to remove '/home/ubuntu/cephtest': Directory not empty 2024-04-12T22:04:00.812 ERROR:teuthology.run_tasks:Manager failed: internal.base Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_6c637841c215537a4502385240412f1966e0faab/teuthology/task/internal/__init__.py", line 48, in base yield File "/home/teuthworker/src/git.ceph.com_teuthology_6c637841c215537a4502385240412f1966e0faab/teuthology/run_tasks.py", line 154, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ next(self.gen) File "/home/teuthworker/src/git.ceph.com_ceph-c_324baf8ddb91dea46cac50cb535c306115791355/qa/tasks/ceph_fuse.py", line 183, in task mount.umount_wait() File "/home/teuthworker/src/git.ceph.com_ceph-c_324baf8ddb91dea46cac50cb535c306115791355/qa/tasks/cephfs/fuse_mount.py", line 403, in umount_wait run.wait([self.fuse_daemon], timeout) File "/home/teuthworker/src/git.ceph.com_teuthology_6c637841c215537a4502385240412f1966e0faab/teuthology/orchestra/run.py", line 473, in wait check_time() File "/home/teuthworker/src/git.ceph.com_teuthology_6c637841c215537a4502385240412f1966e0faab/teuthology/contextutil.py", line 134, in __call__ raise MaxWhileTries(error_msg) teuthology.exceptions.MaxWhileTries: reached maximum tries (51) after waiting for 300 seconds During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_6c637841c215537a4502385240412f1966e0faab/teuthology/run_tasks.py", line 154, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.8/contextlib.py", line 131, in __exit__ self.gen.throw(type, value, traceback) File "/home/teuthworker/src/git.ceph.com_teuthology_6c637841c215537a4502385240412f1966e0faab/teuthology/task/internal/__init__.py", line 53, in base run.wait( File "/home/teuthworker/src/git.ceph.com_teuthology_6c637841c215537a4502385240412f1966e0faab/teuthology/orchestra/run.py", line 479, in wait proc.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_6c637841c215537a4502385240412f1966e0faab/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_6c637841c215537a4502385240412f1966e0faab/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on smithi018 with status 1: 'find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest' 2024-04-12T22:04:00.813 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-04-12T22:04:00.829 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-04-12T22:04:00.847 DEBUG:teuthology.run_tasks:Exception was not quenched, exiting: MaxWhileTries: reached maximum tries (51) after waiting for 300 seconds 2024-04-12T22:04:00.847 INFO:teuthology.nuke:Checking targets against current locks 2024-04-12T22:04:00.875 DEBUG:teuthology.nuke:shortname: smithi018 2024-04-12T22:04:00.875 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-04-12T22:04:00.898 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi018.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/vshankar-2024-04-12_03:11:38-fs-wip-vshankar-testing-20240411.061452-testing-default-smithi/7653578', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-12 20:22:31.827896', 'locked_by': 'scheduled_vshankar@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBNGyQkOmched09Efq4bSE3KgVpnlI3J6whhqkmy38BCo3vj5GRkDnqsXwwiryS9ADIjlegH6l0N7Y8JY4lxk/dQ='} 2024-04-12T22:04:00.902 DEBUG:teuthology.nuke:shortname: smithi186 2024-04-12T22:04:00.902 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-04-12T22:04:00.928 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi186.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/vshankar-2024-04-12_03:11:38-fs-wip-vshankar-testing-20240411.061452-testing-default-smithi/7653578', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-12 20:22:31.829527', 'locked_by': 'scheduled_vshankar@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBKMqUaLnqL7jyJPKjPWFwXI6tZGa6LWRUQG61/E3XmkQnxUMVoO0pdZq24E8iyHT4Tq+F9Y+tckvwkjO0DzdbJY='} 2024-04-12T22:04:00.958 INFO:teuthology.orchestra.console.smithi018:Power off 2024-04-12T22:04:00.958 DEBUG:teuthology.orchestra.console.smithi018:pexpect command: ipmitool -H smithi018.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-04-12T22:04:00.984 DEBUG:teuthology.orchestra.console.smithi018:power off output: Chassis Power Control: Down/Off 2024-04-12T22:04:00.985 DEBUG:teuthology.orchestra.console.smithi018:pexpect command: ipmitool -H smithi018.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-12T22:04:01.000 INFO:teuthology.orchestra.console.smithi186:Power off 2024-04-12T22:04:01.000 DEBUG:teuthology.orchestra.console.smithi186:pexpect command: ipmitool -H smithi186.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-04-12T22:04:01.009 DEBUG:teuthology.orchestra.console.smithi018:check power output: Chassis Power is on 2024-04-12T22:04:01.035 DEBUG:teuthology.orchestra.console.smithi186:power off output: Chassis Power Control: Down/Off 2024-04-12T22:04:01.035 DEBUG:teuthology.orchestra.console.smithi186:pexpect command: ipmitool -H smithi186.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-12T22:04:01.059 DEBUG:teuthology.orchestra.console.smithi186:check power output: Chassis Power is on 2024-04-12T22:04:05.010 DEBUG:teuthology.orchestra.console.smithi018:pexpect command: ipmitool -H smithi018.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-12T22:04:05.060 DEBUG:teuthology.orchestra.console.smithi186:pexpect command: ipmitool -H smithi186.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-12T22:04:05.120 DEBUG:teuthology.orchestra.console.smithi018:check power output: Chassis Power is on 2024-04-12T22:04:05.172 DEBUG:teuthology.orchestra.console.smithi186:check power output: Chassis Power is on 2024-04-12T22:04:09.125 DEBUG:teuthology.orchestra.console.smithi018:pexpect command: ipmitool -H smithi018.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-12T22:04:09.173 DEBUG:teuthology.orchestra.console.smithi186:pexpect command: ipmitool -H smithi186.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-12T22:04:09.236 DEBUG:teuthology.orchestra.console.smithi018:check power output: Chassis Power is off 2024-04-12T22:04:09.285 DEBUG:teuthology.orchestra.console.smithi186:check power output: Chassis Power is off 2024-04-12T22:04:09.337 INFO:teuthology.orchestra.console.smithi018:Power off completed 2024-04-12T22:04:09.386 INFO:teuthology.orchestra.console.smithi186:Power off completed 2024-04-12T22:04:09.491 INFO:teuthology.run:Summary data: description: fs/upgrade/mds_upgrade_sequence/{bluestore-bitmap centos_9.stream conf/{client mds mgr mon osd} fail_fs/yes overrides/{ignorelist_health ignorelist_upgrade ignorelist_wrongly_marked_down pg-warn pg_health syntax} roles tasks/{0-from/quincy 1-volume/{0-create 1-ranks/2 2-allow_standby_replay/no 3-inline/no 4-verify} 2-client/fuse 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}} duration: 5469.626302480698 failure_reason: reached maximum tries (51) after waiting for 300 seconds flavor: default host.a-kernel-sha1: 3f60ed14f7c110c7ce3c8d83dd2ed0d08e80d0c8 host.b-kernel-sha1: 3f60ed14f7c110c7ce3c8d83dd2ed0d08e80d0c8 owner: scheduled_vshankar@teuthology status: fail success: false 2024-04-12T22:04:09.491 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-12T22:04:09.600 INFO:teuthology.run:FAIL